2022-09-03T05:04:23.031 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18 2022-09-03T05:04:23.033 DEBUG:teuthology.run:Teuthology command: teuthology --description rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zone-plus-pubsub supported-random-distro$/{centos_8} tasks/test_multi} --verbose --name dang-2022-09-02_07:58:15-rgw-wip-dang-zipper-admin-api-distro-default-smithi --archive /home/teuthworker/archive/dang-2022-09-02_07:58:15-rgw-wip-dang-zipper-admin-api-distro-default-smithi/7007679 --owner scheduled_dang@sidious.arb.redhat.com -- /home/teuthworker/archive/dang-2022-09-02_07:58:15-rgw-wip-dang-zipper-admin-api-distro-default-smithi/7007679/orig.config.yaml 2022-09-03T05:04:23.086 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2022-09-03T05:04:23.174 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/dang-2022-09-02_07:58:15-rgw-wip-dang-zipper-admin-api-distro-default-smithi/7007679 branch: wip-dang-zipper-admin-api description: rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zone-plus-pubsub supported-random-distro$/{centos_8} tasks/test_multi} email: null first_in_suite: false job_id: '7007679' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: dang-2022-09-02_07:58:15-rgw-wip-dang-zipper-admin-api-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-dang-zipper-admin-api ceph: conf: client: debug rgw: 20 debug rgw sync: 20 rgw crypt require ssl: false rgw crypt s3 kms backend: testing rgw crypt s3 kms encryption keys: testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= rgw curl low speed time: 300 rgw data log num shards: 4 rgw md log max shards: 4 rgw sync data inject err probability: 0.1 rgw sync log trim interval: 0 rgw sync meta inject err probability: 0.1 rgw sync obj etag verify: true setgroup: ceph setuser: ceph client.0: rgw data notify interval msec: 0 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 rocksdb delete range threshold: 0 osd.0: osd_max_omap_entries_per_request: 10 osd.1: osd_max_omap_entries_per_request: 1000 osd.2: osd_max_omap_entries_per_request: 10000 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) sha1: 4d487742eda56d4badbc9bdb33761243365bb37a wait-for-scrub: false ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: default sha1: 4d487742eda56d4badbc9bdb33761243365bb37a rgw: compression type: random frontend: beast rgw-multisite: realm: is default: true name: test-realm zonegroups: - enabled_features: - resharding endpoints: - c1.client.0 is_default: true is_master: true name: test-zonegroup zones: - endpoints: - c1.client.0 is_default: true is_master: true name: test-zone1 - endpoints: - c2.client.0 is_default: true name: test-zone2 - endpoints: - c1.client.1 name: test-zone3 - endpoints: - c2.client.1 is_pubsub: true name: test-zone4 rgw-multisite-tests: args: - tests.py - tests_ps.py selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-dang-zipper-admin-api sha1: 4d487742eda56d4badbc9bdb33761243365bb37a owner: scheduled_dang@sidious.arb.redhat.com priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - c1.mon.a - c1.mgr.x - c1.osd.0 - c1.osd.1 - c1.osd.2 - c1.client.0 - c1.client.1 - - c2.mon.a - c2.mgr.x - c2.osd.0 - c2.osd.1 - c2.osd.2 - c2.client.0 - c2.client.1 sha1: 4d487742eda56d4badbc9bdb33761243365bb37a sleep_before_teardown: 0 suite: rgw suite_branch: wip-dang-zipper-admin-api suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_4d487742eda56d4badbc9bdb33761243365bb37a/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 4d487742eda56d4badbc9bdb33761243365bb37a targets: smithi181.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDpTjEkBg4fXYR5zbI8X1aIX0ry1owo8ATLzirNBazD0ie3B5bmEQ5uTTpXbhbf38TCOicU7Kr7jvVMjp3AWagRW1Z/slXFYQo7xV7wq9O5eljNYgdg4GGmLc4ySZrcCfOzKPfS+3SZVPb/66kxGPJmtIn39hoAN2wj6oFZVCGVADXDLjcPvRXmWHrdJypT2cEY5Ws4o7hOwfxKLus2446DBRdhteOUYqOQR8U4mYU6SLNqcSJRdYQWduH4fqZ+XH9rrf25EwFaPqC4qaj0bngkp5NVSHxqeNE2E2p69KIeQmPr0hbjH5GENnQu4m6NlRJijtGz87r1IlxjC8mzspit5PKbEWfawJWbfstOcTATxYlZh0SP2Qm7HP+y7oVxls18yezhrYIzMM9x0pCtWM/Gbtj8uOz8ZwfKgJ7ojFfz4dPnvFxlqNPjvAsCVLvGog4YWDtG6T08pvidSfJU3QIcA6ZWm7UNPCddUzZHYkvF5XqM2D7gE45qGm++vsNIOBs= smithi189.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCxjbJLTVVZDanIouEGFrJENZLRX4RqVQVKScenNpB1JvF6Qxa0CEAI+dllK2d1mgZBvdyQ+8cAVgmZODJHYi/2nBT2G6zXdMuCp6vYu7+HM8/ZgrBwig6Nx+Y4Oj0/QwwlVJDp4GQuQZKRnR/Wmej1JJRnzcPpd7sLK95+DdEwmBX+3kJPM7SZ+xT+jxri9l6zwgFA2LYPwS2s3WhFA43Tks2OzzN/VgCVZHxbqbH4DBThmpXyWKJz1X1ufqc208z1xFhMy4e8UDJb8VHiOALYQTuT4atNidJELjdaY915boUU/2ACXxovBgqiXHyvJFF+x9N9MPVPSm43GMCb7IxQffBP0UB1m3E5FoQIJtsp8VYILST6U104HNWJ0NGDD8Euh2RUYlyEcxvjwSemVBHw3/QWjwZif3k9GcdOZOKH35gNKGF+UX82+aW39UwYrm7axfkO/HnP57ZZJdsq2334ENS5b607SMAnHrOyCDhkQvaYBoSq9X5JqLkVuBR4/8= tasks: - install: null - ceph: cluster: c1 - ceph: cluster: c2 - rgw: c1.client.0: port: 8000 c1.client.1: port: 8001 c2.client.0: port: 8000 c2.client.1: port: 8001 - rgw-multisite: null - rgw-multisite-tests: config: reconfigure_delay: 60 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: b58f2c18636eb10faa77ed3614abd00cb85dfc2c timestamp: 2022-09-02_07:58:15 tube: smithi user: dang verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1713 2022-09-03T05:04:23.175 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_4d487742eda56d4badbc9bdb33761243365bb37a/qa; will attempt to use it 2022-09-03T05:04:23.175 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_4d487742eda56d4badbc9bdb33761243365bb37a/qa/tasks 2022-09-03T05:04:23.176 INFO:teuthology.run_tasks:Running task internal.check_packages... 2022-09-03T05:04:23.178 INFO:teuthology.task.internal:Checking packages... 2022-09-03T05:04:23.208 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '4d487742eda56d4badbc9bdb33761243365bb37a' 2022-09-03T05:04:23.209 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-09-03T05:04:23.210 INFO:teuthology.packaging:ref: None 2022-09-03T05:04:23.211 INFO:teuthology.packaging:tag: None 2022-09-03T05:04:23.211 INFO:teuthology.packaging:branch: wip-dang-zipper-admin-api 2022-09-03T05:04:23.212 INFO:teuthology.packaging:sha1: 4d487742eda56d4badbc9bdb33761243365bb37a 2022-09-03T05:04:23.213 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-dang-zipper-admin-api 2022-09-03T05:04:23.373 INFO:teuthology.task.internal:Found packages for ceph version 17.0.0-14661.g4d487742 2022-09-03T05:04:23.373 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2022-09-03T05:04:23.386 INFO:teuthology.task.internal:no buildpackages task found 2022-09-03T05:04:23.387 INFO:teuthology.run_tasks:Running task internal.save_config... 2022-09-03T05:04:23.419 INFO:teuthology.task.internal:Saving configuration 2022-09-03T05:04:23.449 INFO:teuthology.run_tasks:Running task internal.check_lock... 2022-09-03T05:04:23.456 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-09-03T05:04:23.487 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dang-2022-09-02_07:58:15-rgw-wip-dang-zipper-admin-api-distro-default-smithi/7007679', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-09-03 04:59:12.845617', 'locked_by': 'scheduled_dang@sidious.arb.redhat.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF2fBbcPD/eZjxc2OGR1mGkpypv1UicwGSAzzce2PHEfQNbRRtlTlaco1cml5UuIy4ek6uiRvHopPzZ2U87C4vM='} 2022-09-03T05:04:23.518 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi189.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dang-2022-09-02_07:58:15-rgw-wip-dang-zipper-admin-api-distro-default-smithi/7007679', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-09-03 04:59:12.844176', 'locked_by': 'scheduled_dang@sidious.arb.redhat.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIAPLhAcs6WqPsAAKTyQL8NqQSAildYzBsKkEDEIayCGt0wJUN3mCK4S8aw5h0qfmZOiZmygZcI5WYHpB7wQM1c='} 2022-09-03T05:04:23.519 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2022-09-03T05:04:23.531 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'] 2022-09-03T05:04:23.532 INFO:teuthology.task.internal:roles: ubuntu@smithi189.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1'] 2022-09-03T05:04:23.532 INFO:teuthology.run_tasks:Running task console_log... 2022-09-03T05:04:23.666 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0fee5462f0>, signals=[15]) 2022-09-03T05:04:23.668 INFO:teuthology.run_tasks:Running task internal.connect... 2022-09-03T05:04:23.679 INFO:teuthology.task.internal:Opening connections... 2022-09-03T05:04:23.679 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi181.front.sepia.ceph.com 2022-09-03T05:04:23.681 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-09-03T05:04:23.760 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi189.front.sepia.ceph.com 2022-09-03T05:04:23.761 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-09-03T05:04:23.845 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2022-09-03T05:04:23.854 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2022-09-03T05:04:23.875 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2022-09-03T05:04:23.876 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2022-09-03T05:04:23.934 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream" 2022-09-03T05:04:23.935 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="8" 2022-09-03T05:04:23.935 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos" 2022-09-03T05:04:23.935 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora" 2022-09-03T05:04:23.936 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="8" 2022-09-03T05:04:23.936 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el8" 2022-09-03T05:04:23.936 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 8" 2022-09-03T05:04:23.936 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2022-09-03T05:04:23.937 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-09-03T05:04:23.937 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/" 2022-09-03T05:04:23.937 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-03T05:04:23.937 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-03T05:04:23.938 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-09-03T05:04:23.938 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2022-09-03T05:04:23.959 DEBUG:teuthology.orchestra.run.smithi189:> uname -m 2022-09-03T05:04:23.981 INFO:teuthology.orchestra.run.smithi189.stdout:x86_64 2022-09-03T05:04:23.981 DEBUG:teuthology.orchestra.run.smithi189:> cat /etc/os-release 2022-09-03T05:04:24.043 INFO:teuthology.orchestra.run.smithi189.stdout:NAME="CentOS Stream" 2022-09-03T05:04:24.043 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION="8" 2022-09-03T05:04:24.044 INFO:teuthology.orchestra.run.smithi189.stdout:ID="centos" 2022-09-03T05:04:24.044 INFO:teuthology.orchestra.run.smithi189.stdout:ID_LIKE="rhel fedora" 2022-09-03T05:04:24.045 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION_ID="8" 2022-09-03T05:04:24.045 INFO:teuthology.orchestra.run.smithi189.stdout:PLATFORM_ID="platform:el8" 2022-09-03T05:04:24.045 INFO:teuthology.orchestra.run.smithi189.stdout:PRETTY_NAME="CentOS Stream 8" 2022-09-03T05:04:24.046 INFO:teuthology.orchestra.run.smithi189.stdout:ANSI_COLOR="0;31" 2022-09-03T05:04:24.046 INFO:teuthology.orchestra.run.smithi189.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-09-03T05:04:24.047 INFO:teuthology.orchestra.run.smithi189.stdout:HOME_URL="https://centos.org/" 2022-09-03T05:04:24.047 INFO:teuthology.orchestra.run.smithi189.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-09-03T05:04:24.047 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-09-03T05:04:24.048 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-09-03T05:04:24.048 INFO:teuthology.lock.ops:Updating smithi189.front.sepia.ceph.com on lock server 2022-09-03T05:04:24.070 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2022-09-03T05:04:24.083 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2022-09-03T05:04:24.101 INFO:teuthology.task.internal:Checking for old test directory... 2022-09-03T05:04:24.102 DEBUG:teuthology.orchestra.run.smithi181:> test '!' -e /home/ubuntu/cephtest 2022-09-03T05:04:24.111 DEBUG:teuthology.orchestra.run.smithi189:> test '!' -e /home/ubuntu/cephtest 2022-09-03T05:04:24.131 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2022-09-03T05:04:24.141 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2022-09-03T05:04:24.141 DEBUG:teuthology.orchestra.run.smithi181:> test -z $(ls -A /var/lib/ceph) 2022-09-03T05:04:24.169 DEBUG:teuthology.orchestra.run.smithi189:> test -z $(ls -A /var/lib/ceph) 2022-09-03T05:04:24.208 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2022-09-03T05:04:24.270 INFO:teuthology.run_tasks:Running task kernel... 2022-09-03T05:04:24.284 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2022-09-03T05:04:24.285 INFO:teuthology.task.kernel:config {'c1.mon.a': {'kdb': True, 'sha1': 'distro'}, 'c2.mon.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2022-09-03T05:04:24.285 DEBUG:teuthology.orchestra.run.smithi181:> test -f /run/.containerenv -o -f /.dockerenv 2022-09-03T05:04:24.287 DEBUG:teuthology.orchestra.run.smithi189:> test -f /run/.containerenv -o -f /.dockerenv 2022-09-03T05:04:24.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-09-03T05:04:24.305 DEBUG:teuthology.orchestra.run.smithi181:> uname -r 2022-09-03T05:04:24.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-09-03T05:04:24.307 DEBUG:teuthology.orchestra.run.smithi189:> uname -r 2022-09-03T05:04:24.365 INFO:teuthology.orchestra.run.smithi181.stdout:4.18.0-408.el8.x86_64 2022-09-03T05:04:24.366 INFO:teuthology.task.kernel:Running kernel on smithi181: 4.18.0-408.el8.x86_64 2022-09-03T05:04:24.366 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum install -y kernel 2022-09-03T05:04:24.369 INFO:teuthology.orchestra.run.smithi189.stdout:4.18.0-408.el8.x86_64 2022-09-03T05:04:24.369 INFO:teuthology.task.kernel:Running kernel on smithi189: 4.18.0-408.el8.x86_64 2022-09-03T05:04:24.370 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum install -y kernel 2022-09-03T05:04:27.323 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS-8 - AppStream 39 MB/s | 25 MB 00:00 2022-09-03T05:04:27.637 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS-8 - AppStream 35 MB/s | 25 MB 00:00 2022-09-03T05:04:32.374 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS-8 - BaseOS 45 MB/s | 25 MB 00:00 2022-09-03T05:04:32.695 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS-8 - BaseOS 41 MB/s | 25 MB 00:00 2022-09-03T05:04:36.231 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS-8 - Extras 176 kB/s | 18 kB 00:00 2022-09-03T05:04:36.536 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS-8 - Extras 150 kB/s | 18 kB 00:00 2022-09-03T05:04:36.608 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS-8 - PowerTools 19 MB/s | 5.0 MB 00:00 2022-09-03T05:04:36.888 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS-8 - PowerTools 21 MB/s | 5.0 MB 00:00 2022-09-03T05:04:37.791 INFO:teuthology.orchestra.run.smithi181.stdout:Copr repo for python3-asyncssh owned by ceph 17 kB/s | 3.5 kB 00:00 2022-09-03T05:04:38.038 INFO:teuthology.orchestra.run.smithi189.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2022-09-03T05:04:39.707 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 7.1 MB/s | 13 MB 00:01 2022-09-03T05:04:39.973 INFO:teuthology.orchestra.run.smithi189.stdout:Extra Packages for Enterprise Linux 7.0 MB/s | 13 MB 00:01 2022-09-03T05:04:42.741 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 425 kB/s | 24 kB 00:00 2022-09-03T05:04:43.004 INFO:teuthology.orchestra.run.smithi189.stdout:lab-extras 575 kB/s | 24 kB 00:00 2022-09-03T05:04:45.514 INFO:teuthology.orchestra.run.smithi181.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2022-09-03T05:04:45.515 INFO:teuthology.orchestra.run.smithi181.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2022-09-03T05:04:45.654 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:04:45.656 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2022-09-03T05:04:45.656 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:04:45.785 DEBUG:teuthology.orchestra.run.smithi181:> echo no | sudo yum reinstall kernel || true 2022-09-03T05:04:45.786 INFO:teuthology.orchestra.run.smithi189.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2022-09-03T05:04:45.786 INFO:teuthology.orchestra.run.smithi189.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2022-09-03T05:04:45.914 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:04:45.916 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2022-09-03T05:04:45.917 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:04:46.047 DEBUG:teuthology.orchestra.run.smithi189:> echo no | sudo yum reinstall kernel || true 2022-09-03T05:04:46.353 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:00:04 ago on Sat 03 Sep 2022 05:04:42 AM UTC. 2022-09-03T05:04:46.660 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:04 ago on Sat 03 Sep 2022 05:04:42 AM UTC. 2022-09-03T05:04:47.387 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:04:47.388 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:04:47.388 INFO:teuthology.orchestra.run.smithi181.stdout: Package Architecture Version Repository Size 2022-09-03T05:04:47.389 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:04:47.389 INFO:teuthology.orchestra.run.smithi181.stdout:Reinstalling: 2022-09-03T05:04:47.389 INFO:teuthology.orchestra.run.smithi181.stdout: kernel x86_64 4.18.0-408.el8 CentOS-BaseOS 8.5 M 2022-09-03T05:04:47.390 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:04:47.390 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2022-09-03T05:04:47.390 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:04:47.390 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:04:47.391 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 8.5 M 2022-09-03T05:04:47.391 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 0 2022-09-03T05:04:47.391 INFO:teuthology.orchestra.run.smithi181.stderr:Operation aborted. 2022-09-03T05:04:47.447 INFO:teuthology.orchestra.run.smithi181.stdout:Is this ok [y/N]: 2022-09-03T05:04:47.448 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum reinstall -y kernel || true 2022-09-03T05:04:47.689 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:04:47.691 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:04:47.692 INFO:teuthology.orchestra.run.smithi189.stdout: Package Architecture Version Repository Size 2022-09-03T05:04:47.692 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:04:47.693 INFO:teuthology.orchestra.run.smithi189.stdout:Reinstalling: 2022-09-03T05:04:47.693 INFO:teuthology.orchestra.run.smithi189.stdout: kernel x86_64 4.18.0-408.el8 CentOS-BaseOS 8.5 M 2022-09-03T05:04:47.694 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:04:47.694 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2022-09-03T05:04:47.694 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:04:47.695 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:04:47.695 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 8.5 M 2022-09-03T05:04:47.695 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 0 2022-09-03T05:04:47.695 INFO:teuthology.orchestra.run.smithi189.stderr:Operation aborted. 2022-09-03T05:04:47.756 INFO:teuthology.orchestra.run.smithi189.stdout:Is this ok [y/N]: 2022-09-03T05:04:47.757 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum reinstall -y kernel || true 2022-09-03T05:04:48.014 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:00:06 ago on Sat 03 Sep 2022 05:04:42 AM UTC. 2022-09-03T05:04:48.325 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:06 ago on Sat 03 Sep 2022 05:04:42 AM UTC. 2022-09-03T05:04:49.012 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:04:49.014 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:04:49.015 INFO:teuthology.orchestra.run.smithi181.stdout: Package Architecture Version Repository Size 2022-09-03T05:04:49.015 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:04:49.016 INFO:teuthology.orchestra.run.smithi181.stdout:Reinstalling: 2022-09-03T05:04:49.016 INFO:teuthology.orchestra.run.smithi181.stdout: kernel x86_64 4.18.0-408.el8 CentOS-BaseOS 8.5 M 2022-09-03T05:04:49.017 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:04:49.017 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2022-09-03T05:04:49.018 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:04:49.018 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:04:49.019 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 8.5 M 2022-09-03T05:04:49.019 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 0 2022-09-03T05:04:49.020 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2022-09-03T05:04:49.251 INFO:teuthology.orchestra.run.smithi181.stdout:kernel-4.18.0-408.el8.x86_64.rpm 36 MB/s | 8.5 MB 00:00 2022-09-03T05:04:49.252 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:04:49.252 INFO:teuthology.orchestra.run.smithi181.stdout:Total 36 MB/s | 8.5 MB 00:00 2022-09-03T05:04:49.332 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:04:49.333 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:04:49.333 INFO:teuthology.orchestra.run.smithi189.stdout: Package Architecture Version Repository Size 2022-09-03T05:04:49.333 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:04:49.334 INFO:teuthology.orchestra.run.smithi189.stdout:Reinstalling: 2022-09-03T05:04:49.334 INFO:teuthology.orchestra.run.smithi189.stdout: kernel x86_64 4.18.0-408.el8 CentOS-BaseOS 8.5 M 2022-09-03T05:04:49.334 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:04:49.335 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2022-09-03T05:04:49.335 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:04:49.335 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:04:49.335 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 8.5 M 2022-09-03T05:04:49.335 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 0 2022-09-03T05:04:49.336 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2022-09-03T05:04:49.354 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2022-09-03T05:04:49.541 INFO:teuthology.orchestra.run.smithi189.stdout:kernel-4.18.0-408.el8.x86_64.rpm 41 MB/s | 8.5 MB 00:00 2022-09-03T05:04:49.541 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:04:49.542 INFO:teuthology.orchestra.run.smithi189.stdout:Total 41 MB/s | 8.5 MB 00:00 2022-09-03T05:04:49.639 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2022-09-03T05:04:49.721 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2022-09-03T05:04:49.721 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2022-09-03T05:04:49.869 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2022-09-03T05:04:49.875 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2022-09-03T05:04:50.021 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2022-09-03T05:04:50.022 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2022-09-03T05:04:50.170 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2022-09-03T05:04:50.181 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2022-09-03T05:04:50.200 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2022-09-03T05:04:50.391 INFO:teuthology.orchestra.run.smithi181.stdout: Reinstalling : kernel-4.18.0-408.el8.x86_64 1/2 2022-09-03T05:04:50.391 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : kernel-4.18.0-408.el8.x86_64 2/2 2022-09-03T05:04:50.489 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2022-09-03T05:04:50.711 INFO:teuthology.orchestra.run.smithi189.stdout: Reinstalling : kernel-4.18.0-408.el8.x86_64 1/2 2022-09-03T05:04:50.712 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : kernel-4.18.0-408.el8.x86_64 2/2 2022-09-03T05:04:50.771 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: kernel-4.18.0-408.el8.x86_64 2/2 2022-09-03T05:04:50.772 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : kernel-4.18.0-408.el8.x86_64 1/2 2022-09-03T05:04:51.123 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-4.18.0-408.el8.x86_64 2/2 2022-09-03T05:04:51.124 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : kernel-4.18.0-408.el8.x86_64 1/2 2022-09-03T05:04:51.219 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : kernel-4.18.0-408.el8.x86_64 2/2 2022-09-03T05:04:51.219 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:04:51.220 INFO:teuthology.orchestra.run.smithi181.stdout:Reinstalled: 2022-09-03T05:04:51.220 INFO:teuthology.orchestra.run.smithi181.stdout: kernel-4.18.0-408.el8.x86_64 2022-09-03T05:04:51.220 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:04:51.221 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:04:51.295 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q kernel --last | head -n 1 2022-09-03T05:04:51.422 INFO:teuthology.orchestra.run.smithi181.stdout:kernel-4.18.0-408.el8.x86_64 Sat 03 Sep 2022 05:04:50 AM UTC 2022-09-03T05:04:51.424 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-408.el8.x86_64 2022-09-03T05:04:51.425 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-09-03T05:04:51.425 INFO:teuthology.task.kernel:Enabling kdb on c1.mon.a... 2022-09-03T05:04:51.426 DEBUG:teuthology.orchestra.run.smithi181:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-09-03T05:04:51.460 INFO:teuthology.orchestra.run.smithi181.stdout:ttyS1 2022-09-03T05:04:51.473 DEBUG:teuthology.parallel:result is None 2022-09-03T05:04:51.643 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : kernel-4.18.0-408.el8.x86_64 2/2 2022-09-03T05:04:51.644 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:04:51.644 INFO:teuthology.orchestra.run.smithi189.stdout:Reinstalled: 2022-09-03T05:04:51.645 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-4.18.0-408.el8.x86_64 2022-09-03T05:04:51.645 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:04:51.645 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:04:51.723 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q kernel --last | head -n 1 2022-09-03T05:04:51.851 INFO:teuthology.orchestra.run.smithi189.stdout:kernel-4.18.0-408.el8.x86_64 Sat 03 Sep 2022 05:04:50 AM UTC 2022-09-03T05:04:51.853 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-408.el8.x86_64 2022-09-03T05:04:51.853 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-09-03T05:04:51.853 INFO:teuthology.task.kernel:Enabling kdb on c2.mon.a... 2022-09-03T05:04:51.854 DEBUG:teuthology.orchestra.run.smithi189:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-09-03T05:04:51.896 INFO:teuthology.orchestra.run.smithi189.stdout:ttyS1 2022-09-03T05:04:51.909 DEBUG:teuthology.parallel:result is None 2022-09-03T05:04:51.909 INFO:teuthology.run_tasks:Running task internal.base... 2022-09-03T05:04:51.918 INFO:teuthology.task.internal:Creating test directory... 2022-09-03T05:04:51.919 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-09-03T05:04:51.925 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-09-03T05:04:51.969 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2022-09-03T05:04:51.979 INFO:teuthology.run_tasks:Running task internal.archive... 2022-09-03T05:04:51.999 INFO:teuthology.task.internal:Creating archive directory... 2022-09-03T05:04:52.000 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-09-03T05:04:52.002 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-09-03T05:04:52.044 INFO:teuthology.run_tasks:Running task internal.coredump... 2022-09-03T05:04:52.061 INFO:teuthology.task.internal:Enabling coredump saving... 2022-09-03T05:04:52.061 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-09-03T05:04:52.071 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-09-03T05:04:52.104 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-09-03T05:04:52.115 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-09-03T05:04:52.122 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-09-03T05:04:52.132 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-09-03T05:04:52.133 INFO:teuthology.run_tasks:Running task internal.sudo... 2022-09-03T05:04:52.143 INFO:teuthology.task.internal:Configuring sudo... 2022-09-03T05:04:52.144 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-09-03T05:04:52.158 DEBUG:teuthology.orchestra.run.smithi189:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-09-03T05:04:52.201 INFO:teuthology.run_tasks:Running task internal.syslog... 2022-09-03T05:04:52.211 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2022-09-03T05:04:52.212 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-09-03T05:04:52.225 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-09-03T05:04:52.259 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-09-03T05:04:52.334 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-09-03T05:04:52.380 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-09-03T05:04:52.475 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-09-03T05:04:52.511 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2022-09-03T05:04:52.512 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-09-03T05:04:52.579 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-09-03T05:04:52.628 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-09-03T05:04:52.715 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-09-03T05:04:52.763 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-09-03T05:04:52.806 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2022-09-03T05:04:52.807 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-09-03T05:04:52.884 DEBUG:teuthology.orchestra.run.smithi181:> sudo service rsyslog restart 2022-09-03T05:04:52.887 DEBUG:teuthology.orchestra.run.smithi189:> sudo service rsyslog restart 2022-09-03T05:04:52.946 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-09-03T05:04:52.993 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-09-03T05:04:53.362 INFO:teuthology.run_tasks:Running task internal.timer... 2022-09-03T05:04:53.380 INFO:teuthology.task.internal:Starting timer... 2022-09-03T05:04:53.382 INFO:teuthology.run_tasks:Running task pcp... 2022-09-03T05:04:53.423 INFO:teuthology.run_tasks:Running task selinux... 2022-09-03T05:04:53.446 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2022-09-03T05:04:53.447 DEBUG:teuthology.orchestra.run.smithi181:> sudo service auditd rotate 2022-09-03T05:04:53.506 INFO:teuthology.orchestra.run.smithi181.stdout:Rotating logs: 2022-09-03T05:04:53.507 DEBUG:teuthology.orchestra.run.smithi189:> sudo service auditd rotate 2022-09-03T05:04:53.632 INFO:teuthology.orchestra.run.smithi189.stdout:Rotating logs: 2022-09-03T05:04:53.633 DEBUG:teuthology.task.selinux:Getting current SELinux state 2022-09-03T05:04:53.634 DEBUG:teuthology.orchestra.run.smithi181:> /usr/sbin/getenforce 2022-09-03T05:04:53.670 INFO:teuthology.orchestra.run.smithi181.stdout:Permissive 2022-09-03T05:04:53.671 DEBUG:teuthology.orchestra.run.smithi189:> /usr/sbin/getenforce 2022-09-03T05:04:53.695 INFO:teuthology.orchestra.run.smithi189.stdout:Permissive 2022-09-03T05:04:53.696 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi181.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi189.front.sepia.ceph.com': 'permissive'} 2022-09-03T05:04:53.696 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-09-03T05:04:53.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-09-03T05:04:53.740 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-09-03T05:04:53.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-09-03T05:04:53.770 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2022-09-03T05:04:53.771 DEBUG:teuthology.orchestra.run.smithi181:> sudo /usr/sbin/setenforce permissive 2022-09-03T05:04:53.807 DEBUG:teuthology.orchestra.run.smithi189:> sudo /usr/sbin/setenforce permissive 2022-09-03T05:04:53.838 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2022-09-03T05:04:53.863 DEBUG:teuthology.repo_utils:Setting repo remote to git://git.ceph.com/git/ceph-cm-ansible.git 2022-09-03T05:04:53.873 INFO:teuthology.repo_utils:Fetching main from origin 2022-09-03T05:04:53.943 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_main to origin/main 2022-09-03T05:04:53.994 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2022-09-03T05:04:53.996 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi181.front.sepia.ceph.com,smithi189.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_main/cephlab.yml 2022-09-03T05:12:31.560 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi181.front.sepia.ceph.com'), Remote(name='ubuntu@smithi189.front.sepia.ceph.com')] 2022-09-03T05:12:31.612 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2022-09-03T05:12:31.614 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-09-03T05:12:31.811 DEBUG:teuthology.orchestra.run.smithi181:> true 2022-09-03T05:12:31.840 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2022-09-03T05:12:31.842 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-09-03T05:12:32.009 DEBUG:teuthology.orchestra.run.smithi189:> true 2022-09-03T05:12:32.082 INFO:teuthology.run_tasks:Running task clock... 2022-09-03T05:12:32.127 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2022-09-03T05:12:32.130 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-09-03T05:12:32.139 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-09-03T05:12:32.145 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-09-03T05:12:32.147 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-09-03T05:12:32.172 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-09-03T05:12:32.181 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-09-03T05:12:32.190 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-09-03T05:12:32.209 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-09-03T05:12:32.254 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: ntpd: command not found 2022-09-03T05:12:32.269 INFO:teuthology.orchestra.run.smithi189.stderr:sudo: ntpd: command not found 2022-09-03T05:12:32.270 INFO:teuthology.orchestra.run.smithi181.stdout:506 Cannot talk to daemon 2022-09-03T05:12:32.277 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-09-03T05:12:32.299 INFO:teuthology.orchestra.run.smithi189.stdout:506 Cannot talk to daemon 2022-09-03T05:12:32.299 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-09-03T05:12:32.300 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-09-03T05:12:32.306 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-09-03T05:12:32.349 INFO:teuthology.orchestra.run.smithi181.stderr:bash: ntpq: command not found 2022-09-03T05:12:32.370 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-09-03T05:12:32.385 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2022-09-03T05:12:32.392 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-09-03T05:12:32.393 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-09-03T05:12:32.393 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-09-03T05:12:32.400 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-09-03T05:12:32.401 INFO:teuthology.orchestra.run.smithi189.stderr:bash: ntpq: command not found 2022-09-03T05:12:32.403 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-09-03T05:12:32.412 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2022-09-03T05:12:32.413 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-09-03T05:12:32.413 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-09-03T05:12:32.413 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-09-03T05:12:32.413 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-09-03T05:12:32.414 INFO:teuthology.run_tasks:Running task install... 2022-09-03T05:12:32.437 DEBUG:teuthology.task.install:project ceph 2022-09-03T05:12:32.438 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '4d487742eda56d4badbc9bdb33761243365bb37a'}} 2022-09-03T05:12:32.439 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '4d487742eda56d4badbc9bdb33761243365bb37a'} 2022-09-03T05:12:32.439 INFO:teuthology.task.install:Using flavor: default 2022-09-03T05:12:32.452 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2022-09-03T05:12:32.453 INFO:teuthology.task.install:extra packages: [] 2022-09-03T05:12:32.453 DEBUG:teuthology.orchestra.run.smithi181:> sudo lsb_release -is 2022-09-03T05:12:32.491 INFO:teuthology.orchestra.run.smithi181.stdout:CentOSStream 2022-09-03T05:12:32.493 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-09-03T05:12:32.493 DEBUG:teuthology.orchestra.run.smithi189:> sudo lsb_release -is 2022-09-03T05:12:32.494 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4d487742eda56d4badbc9bdb33761243365bb37a', 'tag': None, 'wait_for_package': False} 2022-09-03T05:12:32.495 DEBUG:teuthology.orchestra.run.smithi181:> sudo lsb_release -is 2022-09-03T05:12:32.538 INFO:teuthology.orchestra.run.smithi189.stdout:CentOSStream 2022-09-03T05:12:32.538 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-09-03T05:12:32.539 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4d487742eda56d4badbc9bdb33761243365bb37a', 'tag': None, 'wait_for_package': False} 2022-09-03T05:12:32.539 DEBUG:teuthology.orchestra.run.smithi189:> sudo lsb_release -is 2022-09-03T05:12:32.571 INFO:teuthology.orchestra.run.smithi181.stdout:CentOSStream 2022-09-03T05:12:32.574 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-09-03T05:12:32.576 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4d487742eda56d4badbc9bdb33761243365bb37a 2022-09-03T05:12:32.655 INFO:teuthology.orchestra.run.smithi189.stdout:CentOSStream 2022-09-03T05:12:32.655 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-09-03T05:12:32.656 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4d487742eda56d4badbc9bdb33761243365bb37a 2022-09-03T05:12:32.815 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-dang-zipper-admin-api/4d487742eda56d4badbc9bdb33761243365bb37a/centos/8/flavors/default/ 2022-09-03T05:12:32.824 INFO:teuthology.task.install.rpm:Package version is 17.0.0-14661.g4d487742 2022-09-03T05:12:32.843 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-dang-zipper-admin-api/4d487742eda56d4badbc9bdb33761243365bb37a/centos/8/flavors/default/ 2022-09-03T05:12:32.844 INFO:teuthology.task.install.rpm:Package version is 17.0.0-14661.g4d487742 2022-09-03T05:12:33.076 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-dang-zipper-admin-api/4d487742eda56d4badbc9bdb33761243365bb37a/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-dang-zipper-admin-api/4d487742eda56d4badbc9bdb33761243365bb37a/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-dang-zipper-admin-api/4d487742eda56d4badbc9bdb33761243365bb37a/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-09-03T05:12:33.077 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2022-09-03T05:12:33.077 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-09-03T05:12:33.086 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-dang-zipper-admin-api/4d487742eda56d4badbc9bdb33761243365bb37a/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-dang-zipper-admin-api/4d487742eda56d4badbc9bdb33761243365bb37a/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-dang-zipper-admin-api/4d487742eda56d4badbc9bdb33761243365bb37a/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-09-03T05:12:33.087 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2022-09-03T05:12:33.088 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-09-03T05:12:33.117 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2022-09-03T05:12:33.124 DEBUG:teuthology.orchestra.run.smithi181:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4d487742eda56d4badbc9bdb33761243365bb37a/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-09-03T05:12:33.192 DEBUG:teuthology.orchestra.run.smithi181:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-09-03T05:12:33.194 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2022-09-03T05:12:33.194 DEBUG:teuthology.orchestra.run.smithi189:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4d487742eda56d4badbc9bdb33761243365bb37a/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-09-03T05:12:33.272 DEBUG:teuthology.orchestra.run.smithi189:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-09-03T05:12:33.275 DEBUG:teuthology.orchestra.run.smithi181:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-09-03T05:12:33.345 INFO:teuthology.orchestra.run.smithi181.stdout:check_obsoletes = 1 2022-09-03T05:12:33.347 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum clean all 2022-09-03T05:12:33.361 DEBUG:teuthology.orchestra.run.smithi189:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-09-03T05:12:33.428 INFO:teuthology.orchestra.run.smithi189.stdout:check_obsoletes = 1 2022-09-03T05:12:33.431 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum clean all 2022-09-03T05:12:33.598 INFO:teuthology.orchestra.run.smithi181.stdout:63 files removed 2022-09-03T05:12:33.622 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-radosgw 2022-09-03T05:12:33.689 INFO:teuthology.orchestra.run.smithi189.stdout:63 files removed 2022-09-03T05:12:33.719 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-radosgw 2022-09-03T05:12:34.266 INFO:teuthology.orchestra.run.smithi181.stdout:ceph packages for x86_64 289 kB/s | 81 kB 00:00 2022-09-03T05:12:34.387 INFO:teuthology.orchestra.run.smithi189.stdout:ceph packages for x86_64 264 kB/s | 81 kB 00:00 2022-09-03T05:12:34.550 INFO:teuthology.orchestra.run.smithi181.stdout:ceph noarch packages 38 kB/s | 9.4 kB 00:00 2022-09-03T05:12:34.673 INFO:teuthology.orchestra.run.smithi189.stdout:ceph noarch packages 38 kB/s | 9.4 kB 00:00 2022-09-03T05:12:34.817 INFO:teuthology.orchestra.run.smithi181.stdout:ceph source packages 6.3 kB/s | 1.5 kB 00:00 2022-09-03T05:12:34.933 INFO:teuthology.orchestra.run.smithi189.stdout:ceph source packages 6.2 kB/s | 1.5 kB 00:00 2022-09-03T05:12:35.485 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS-8 - AppStream 39 MB/s | 25 MB 00:00 2022-09-03T05:12:35.732 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS-8 - AppStream 32 MB/s | 25 MB 00:00 2022-09-03T05:12:40.446 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS-8 - BaseOS 43 MB/s | 25 MB 00:00 2022-09-03T05:12:40.742 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS-8 - BaseOS 36 MB/s | 25 MB 00:00 2022-09-03T05:12:44.333 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS-8 - Extras 164 kB/s | 18 kB 00:00 2022-09-03T05:12:44.563 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS-8 - Extras 156 kB/s | 18 kB 00:00 2022-09-03T05:12:44.673 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS-8 - PowerTools 22 MB/s | 5.0 MB 00:00 2022-09-03T05:12:44.949 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS-8 - PowerTools 19 MB/s | 5.0 MB 00:00 2022-09-03T05:12:45.931 INFO:teuthology.orchestra.run.smithi181.stdout:Copr repo for python3-asyncssh owned by ceph 12 kB/s | 3.5 kB 00:00 2022-09-03T05:12:46.080 INFO:teuthology.orchestra.run.smithi189.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2022-09-03T05:12:47.007 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 13 MB/s | 13 MB 00:00 2022-09-03T05:12:47.146 INFO:teuthology.orchestra.run.smithi189.stdout:Extra Packages for Enterprise Linux 13 MB/s | 13 MB 00:00 2022-09-03T05:12:50.005 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 534 kB/s | 24 kB 00:00 2022-09-03T05:12:50.138 INFO:teuthology.orchestra.run.smithi189.stdout:lab-extras 400 kB/s | 24 kB 00:00 2022-09-03T05:12:53.084 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:12:53.084 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:12:53.085 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2022-09-03T05:12:53.085 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:12:53.089 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2022-09-03T05:12:53.090 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw x86_64 2:17.0.0-14661.g4d487742.el8 ceph 18 M 2022-09-03T05:12:53.090 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrading: 2022-09-03T05:12:53.100 INFO:teuthology.orchestra.run.smithi181.stdout: librados2 x86_64 2:17.0.0-14661.g4d487742.el8 ceph 3.6 M 2022-09-03T05:12:53.100 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1 x86_64 2:17.0.0-14661.g4d487742.el8 ceph 3.6 M 2022-09-03T05:12:53.101 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2022-09-03T05:12:53.101 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base x86_64 2:17.0.0-14661.g4d487742.el8 ceph 5.2 M 2022-09-03T05:12:53.101 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common x86_64 2:17.0.0-14661.g4d487742.el8 ceph 24 M 2022-09-03T05:12:53.101 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux x86_64 2:17.0.0-14661.g4d487742.el8 ceph 24 k 2022-09-03T05:12:53.102 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-09-03T05:12:53.102 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2 x86_64 2:17.0.0-14661.g4d487742.el8 ceph 766 k 2022-09-03T05:12:53.102 INFO:teuthology.orchestra.run.smithi181.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-09-03T05:12:53.103 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2022-09-03T05:12:53.103 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1 x86_64 2:17.0.0-14661.g4d487742.el8 ceph 476 k 2022-09-03T05:12:53.103 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2022-09-03T05:12:53.104 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2 x86_64 2:17.0.0-14661.g4d487742.el8 ceph 5.8 M 2022-09-03T05:12:53.105 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-09-03T05:12:53.105 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2022-09-03T05:12:53.105 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse 2022-09-03T05:12:53.105 INFO:teuthology.orchestra.run.smithi181.stdout: x86_64 2:17.0.0-14661.g4d487742.el8 ceph 45 k 2022-09-03T05:12:53.107 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common x86_64 2:17.0.0-14661.g4d487742.el8 ceph 109 k 2022-09-03T05:12:53.107 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs x86_64 2:17.0.0-14661.g4d487742.el8 ceph 190 k 2022-09-03T05:12:53.107 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2022-09-03T05:12:53.107 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados x86_64 2:17.0.0-14661.g4d487742.el8 ceph 371 k 2022-09-03T05:12:53.108 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd x86_64 2:17.0.0-14661.g4d487742.el8 ceph 358 k 2022-09-03T05:12:53.108 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw x86_64 2:17.0.0-14661.g4d487742.el8 ceph 107 k 2022-09-03T05:12:53.108 INFO:teuthology.orchestra.run.smithi181.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2022-09-03T05:12:53.108 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:12:53.109 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2022-09-03T05:12:53.109 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:12:53.109 INFO:teuthology.orchestra.run.smithi181.stdout:Install 21 Packages 2022-09-03T05:12:53.109 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrade 2 Packages 2022-09-03T05:12:53.109 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:12:53.110 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 65 M 2022-09-03T05:12:53.110 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2022-09-03T05:12:53.288 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:12:53.293 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:12:53.295 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2022-09-03T05:12:53.295 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:12:53.297 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2022-09-03T05:12:53.297 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw x86_64 2:17.0.0-14661.g4d487742.el8 ceph 18 M 2022-09-03T05:12:53.298 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrading: 2022-09-03T05:12:53.298 INFO:teuthology.orchestra.run.smithi189.stdout: librados2 x86_64 2:17.0.0-14661.g4d487742.el8 ceph 3.6 M 2022-09-03T05:12:53.298 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1 x86_64 2:17.0.0-14661.g4d487742.el8 ceph 3.6 M 2022-09-03T05:12:53.298 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2022-09-03T05:12:53.299 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base x86_64 2:17.0.0-14661.g4d487742.el8 ceph 5.2 M 2022-09-03T05:12:53.299 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common x86_64 2:17.0.0-14661.g4d487742.el8 ceph 24 M 2022-09-03T05:12:53.299 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux x86_64 2:17.0.0-14661.g4d487742.el8 ceph 24 k 2022-09-03T05:12:53.299 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-09-03T05:12:53.300 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2 x86_64 2:17.0.0-14661.g4d487742.el8 ceph 766 k 2022-09-03T05:12:53.300 INFO:teuthology.orchestra.run.smithi189.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-09-03T05:12:53.300 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2022-09-03T05:12:53.300 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1 x86_64 2:17.0.0-14661.g4d487742.el8 ceph 476 k 2022-09-03T05:12:53.301 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2022-09-03T05:12:53.301 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2 x86_64 2:17.0.0-14661.g4d487742.el8 ceph 5.8 M 2022-09-03T05:12:53.301 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-09-03T05:12:53.301 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2022-09-03T05:12:53.302 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse 2022-09-03T05:12:53.302 INFO:teuthology.orchestra.run.smithi189.stdout: x86_64 2:17.0.0-14661.g4d487742.el8 ceph 45 k 2022-09-03T05:12:53.302 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common x86_64 2:17.0.0-14661.g4d487742.el8 ceph 109 k 2022-09-03T05:12:53.302 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs x86_64 2:17.0.0-14661.g4d487742.el8 ceph 190 k 2022-09-03T05:12:53.303 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2022-09-03T05:12:53.303 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados x86_64 2:17.0.0-14661.g4d487742.el8 ceph 371 k 2022-09-03T05:12:53.303 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd x86_64 2:17.0.0-14661.g4d487742.el8 ceph 358 k 2022-09-03T05:12:53.303 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw x86_64 2:17.0.0-14661.g4d487742.el8 ceph 107 k 2022-09-03T05:12:53.303 INFO:teuthology.orchestra.run.smithi189.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2022-09-03T05:12:53.304 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:12:53.304 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2022-09-03T05:12:53.304 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:12:53.304 INFO:teuthology.orchestra.run.smithi189.stdout:Install 21 Packages 2022-09-03T05:12:53.305 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrade 2 Packages 2022-09-03T05:12:53.305 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:12:53.305 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 65 M 2022-09-03T05:12:53.305 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2022-09-03T05:12:53.783 INFO:teuthology.orchestra.run.smithi181.stdout:(1/23): ceph-base-17.0.0-14661.g4d487742.el8.x8 7.4 MB/s | 5.2 MB 00:00 2022-09-03T05:12:53.884 INFO:teuthology.orchestra.run.smithi181.stdout:(2/23): ceph-selinux-17.0.0-14661.g4d487742.el8 242 kB/s | 24 kB 00:00 2022-09-03T05:12:54.565 INFO:teuthology.orchestra.run.smithi189.stdout:(1/23): ceph-base-17.0.0-14661.g4d487742.el8.x8 4.1 MB/s | 5.2 MB 00:01 2022-09-03T05:12:54.656 INFO:teuthology.orchestra.run.smithi189.stdout:(2/23): ceph-selinux-17.0.0-14661.g4d487742.el8 291 kB/s | 24 kB 00:00 2022-09-03T05:12:54.682 INFO:teuthology.orchestra.run.smithi181.stdout:(3/23): libcephfs2-17.0.0-14661.g4d487742.el8.x 958 kB/s | 766 kB 00:00 2022-09-03T05:12:54.842 INFO:teuthology.orchestra.run.smithi189.stdout:(3/23): libcephfs2-17.0.0-14661.g4d487742.el8.x 3.9 MB/s | 766 kB 00:00 2022-09-03T05:12:54.945 INFO:teuthology.orchestra.run.smithi189.stdout:(4/23): libradosstriper1-17.0.0-14661.g4d487742 4.6 MB/s | 476 kB 00:00 2022-09-03T05:12:55.573 INFO:teuthology.orchestra.run.smithi181.stdout:(4/23): libradosstriper1-17.0.0-14661.g4d487742 533 kB/s | 476 kB 00:00 2022-09-03T05:12:55.837 INFO:teuthology.orchestra.run.smithi189.stdout:(5/23): librgw2-17.0.0-14661.g4d487742.el8.x86_ 6.5 MB/s | 5.8 MB 00:00 2022-09-03T05:12:55.902 INFO:teuthology.orchestra.run.smithi189.stdout:(6/23): python3-ceph-argparse-17.0.0-14661.g4d4 670 kB/s | 45 kB 00:00 2022-09-03T05:12:55.959 INFO:teuthology.orchestra.run.smithi189.stdout:(7/23): python3-ceph-common-17.0.0-14661.g4d487 1.8 MB/s | 109 kB 00:00 2022-09-03T05:12:56.016 INFO:teuthology.orchestra.run.smithi189.stdout:(8/23): python3-cephfs-17.0.0-14661.g4d487742.e 3.2 MB/s | 190 kB 00:00 2022-09-03T05:12:56.083 INFO:teuthology.orchestra.run.smithi189.stdout:(9/23): python3-rados-17.0.0-14661.g4d487742.el 5.5 MB/s | 371 kB 00:00 2022-09-03T05:12:56.100 INFO:teuthology.orchestra.run.smithi181.stdout:(5/23): ceph-radosgw-17.0.0-14661.g4d487742.el8 5.8 MB/s | 18 MB 00:03 2022-09-03T05:12:56.178 INFO:teuthology.orchestra.run.smithi189.stdout:(10/23): python3-rbd-17.0.0-14661.g4d487742.el8 3.8 MB/s | 358 kB 00:00 2022-09-03T05:12:56.242 INFO:teuthology.orchestra.run.smithi189.stdout:(11/23): python3-rgw-17.0.0-14661.g4d487742.el8 1.7 MB/s | 107 kB 00:00 2022-09-03T05:12:56.300 INFO:teuthology.orchestra.run.smithi189.stdout:(12/23): librdkafka-0.11.4-3.el8.x86_64.rpm 5.9 MB/s | 354 kB 00:00 2022-09-03T05:12:56.319 INFO:teuthology.orchestra.run.smithi189.stdout:(13/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 10 MB/s | 259 kB 00:00 2022-09-03T05:12:56.341 INFO:teuthology.orchestra.run.smithi181.stdout:(6/23): ceph-common-17.0.0-14661.g4d487742.el8. 7.4 MB/s | 24 MB 00:03 2022-09-03T05:12:56.350 INFO:teuthology.orchestra.run.smithi189.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 1.7 MB/s | 44 kB 00:00 2022-09-03T05:12:56.370 INFO:teuthology.orchestra.run.smithi181.stdout:(7/23): python3-ceph-argparse-17.0.0-14661.g4d4 167 kB/s | 45 kB 00:00 2022-09-03T05:12:56.371 INFO:teuthology.orchestra.run.smithi189.stdout:(15/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.8 MB/s | 47 kB 00:00 2022-09-03T05:12:56.409 INFO:teuthology.orchestra.run.smithi181.stdout:(8/23): python3-ceph-common-17.0.0-14661.g4d487 1.6 MB/s | 109 kB 00:00 2022-09-03T05:12:56.474 INFO:teuthology.orchestra.run.smithi181.stdout:(9/23): python3-cephfs-17.0.0-14661.g4d487742.e 1.7 MB/s | 190 kB 00:00 2022-09-03T05:12:56.525 INFO:teuthology.orchestra.run.smithi181.stdout:(10/23): python3-rados-17.0.0-14661.g4d487742.e 3.1 MB/s | 371 kB 00:00 2022-09-03T05:12:56.603 INFO:teuthology.orchestra.run.smithi181.stdout:(11/23): python3-rbd-17.0.0-14661.g4d487742.el8 2.8 MB/s | 358 kB 00:00 2022-09-03T05:12:56.627 INFO:teuthology.orchestra.run.smithi181.stdout:(12/23): python3-rgw-17.0.0-14661.g4d487742.el8 1.0 MB/s | 107 kB 00:00 2022-09-03T05:12:56.697 INFO:teuthology.orchestra.run.smithi181.stdout:(13/23): librdkafka-0.11.4-3.el8.x86_64.rpm 3.8 MB/s | 354 kB 00:00 2022-09-03T05:12:56.707 INFO:teuthology.orchestra.run.smithi189.stdout:(16/23): gperftools-libs-2.7-9.el8.x86_64.rpm 918 kB/s | 306 kB 00:00 2022-09-03T05:12:56.718 INFO:teuthology.orchestra.run.smithi181.stdout:(14/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.8 MB/s | 259 kB 00:00 2022-09-03T05:12:56.738 INFO:teuthology.orchestra.run.smithi181.stdout:(15/23): python3-prettytable-0.7.2-14.el8.noarc 1.0 MB/s | 44 kB 00:00 2022-09-03T05:12:56.751 INFO:teuthology.orchestra.run.smithi181.stdout:(16/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2022-09-03T05:12:56.799 INFO:teuthology.orchestra.run.smithi189.stdout:(17/23): liboath-2.6.2-3.el8.x86_64.rpm 642 kB/s | 59 kB 00:00 2022-09-03T05:12:56.835 INFO:teuthology.orchestra.run.smithi181.stdout:(17/23): librgw2-17.0.0-14661.g4d487742.el8.x86 4.6 MB/s | 5.8 MB 00:01 2022-09-03T05:12:56.896 INFO:teuthology.orchestra.run.smithi189.stdout:(18/23): libunwind-1.3.1-3.el8.x86_64.rpm 750 kB/s | 75 kB 00:00 2022-09-03T05:12:57.032 INFO:teuthology.orchestra.run.smithi181.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 214 kB/s | 59 kB 00:00 2022-09-03T05:12:57.069 INFO:teuthology.orchestra.run.smithi181.stdout:(19/23): libunwind-1.3.1-3.el8.x86_64.rpm 321 kB/s | 75 kB 00:00 2022-09-03T05:12:57.167 INFO:teuthology.orchestra.run.smithi189.stdout:(19/23): thrift-0.13.0-2.el8.x86_64.rpm 6.5 MB/s | 1.7 MB 00:00 2022-09-03T05:12:57.211 INFO:teuthology.orchestra.run.smithi181.stdout:(20/23): gperftools-libs-2.7-9.el8.x86_64.rpm 643 kB/s | 306 kB 00:00 2022-09-03T05:12:57.597 INFO:teuthology.orchestra.run.smithi189.stdout:(20/23): ceph-radosgw-17.0.0-14661.g4d487742.el 4.1 MB/s | 18 MB 00:04 2022-09-03T05:12:57.778 INFO:teuthology.orchestra.run.smithi181.stdout:(21/23): librados2-17.0.0-14661.g4d487742.el8.x 5.1 MB/s | 3.6 MB 00:00 2022-09-03T05:12:57.828 INFO:teuthology.orchestra.run.smithi181.stdout:(22/23): librbd1-17.0.0-14661.g4d487742.el8.x86 5.8 MB/s | 3.6 MB 00:00 2022-09-03T05:12:57.863 INFO:teuthology.orchestra.run.smithi189.stdout:(21/23): librados2-17.0.0-14661.g4d487742.el8.x 5.2 MB/s | 3.6 MB 00:00 2022-09-03T05:12:58.077 INFO:teuthology.orchestra.run.smithi181.stdout:(23/23): thrift-0.13.0-2.el8.x86_64.rpm 1.7 MB/s | 1.7 MB 00:01 2022-09-03T05:12:58.086 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:12:58.087 INFO:teuthology.orchestra.run.smithi181.stdout:Total 13 MB/s | 65 MB 00:04 2022-09-03T05:12:58.105 INFO:teuthology.orchestra.run.smithi189.stdout:(22/23): librbd1-17.0.0-14661.g4d487742.el8.x86 7.0 MB/s | 3.6 MB 00:00 2022-09-03T05:12:58.119 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2022-09-03T05:12:58.166 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2022-09-03T05:12:58.166 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2022-09-03T05:12:58.628 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2022-09-03T05:12:58.629 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2022-09-03T05:12:59.105 INFO:teuthology.orchestra.run.smithi189.stdout:(23/23): ceph-common-17.0.0-14661.g4d487742.el8 4.1 MB/s | 24 MB 00:05 2022-09-03T05:12:59.108 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:12:59.108 INFO:teuthology.orchestra.run.smithi189.stdout:Total 11 MB/s | 65 MB 00:05 2022-09-03T05:12:59.147 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2022-09-03T05:12:59.191 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2022-09-03T05:12:59.191 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2022-09-03T05:12:59.679 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2022-09-03T05:12:59.682 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2022-09-03T05:12:59.794 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2022-09-03T05:12:59.958 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2022-09-03T05:13:00.136 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/25 2022-09-03T05:13:00.335 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/25 2022-09-03T05:13:00.735 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/25 2022-09-03T05:13:00.927 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2022-09-03T05:13:00.935 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librados2-2:17.0.0-14661.g4d487742.el8.x86_64 4/25 2022-09-03T05:13:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:17.0.0-14661.g4d487742.el8.x86_64 4/25 2022-09-03T05:13:01.118 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2022-09-03T05:13:01.236 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rados-2:17.0.0-14661.g4d487742.el8.x86_64 5/25 2022-09-03T05:13:01.347 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/25 2022-09-03T05:13:01.376 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs2-2:17.0.0-14661.g4d487742.el8.x86_64 6/25 2022-09-03T05:13:01.550 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/25 2022-09-03T05:13:01.769 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephfs2-2:17.0.0-14661.g4d487742.el8.x86_64 6/25 2022-09-03T05:13:01.919 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librbd1-2:17.0.0-14661.g4d487742.el8.x86_64 7/25 2022-09-03T05:13:01.929 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/25 2022-09-03T05:13:01.967 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:17.0.0-14661.g4d487742.el8.x86_64 7/25 2022-09-03T05:13:02.155 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librados2-2:17.0.0-14661.g4d487742.el8.x86_64 4/25 2022-09-03T05:13:02.159 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2022-09-03T05:13:02.239 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:17.0.0-14661.g4d487742.el8.x86_64 4/25 2022-09-03T05:13:02.306 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2022-09-03T05:13:02.371 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2022-09-03T05:13:02.506 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-argparse-2:17.0.0-14661.g4d487742.el8 10/25 2022-09-03T05:13:02.540 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rados-2:17.0.0-14661.g4d487742.el8.x86_64 5/25 2022-09-03T05:13:02.720 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cephfs-2:17.0.0-14661.g4d487742.el8.x86_64 11/25 2022-09-03T05:13:02.763 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs2-2:17.0.0-14661.g4d487742.el8.x86_64 6/25 2022-09-03T05:13:02.936 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rbd-2:17.0.0-14661.g4d487742.el8.x86_64 12/25 2022-09-03T05:13:03.102 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libradosstriper1-2:17.0.0-14661.g4d487742.el8.x86_ 13/25 2022-09-03T05:13:03.162 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libradosstriper1-2:17.0.0-14661.g4d487742.el8.x86_ 13/25 2022-09-03T05:13:03.178 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephfs2-2:17.0.0-14661.g4d487742.el8.x86_64 6/25 2022-09-03T05:13:03.352 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2022-09-03T05:13:03.404 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librbd1-2:17.0.0-14661.g4d487742.el8.x86_64 7/25 2022-09-03T05:13:03.468 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:17.0.0-14661.g4d487742.el8.x86_64 7/25 2022-09-03T05:13:03.695 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2022-09-03T05:13:03.846 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2022-09-03T05:13:03.905 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2022-09-03T05:13:04.051 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2022-09-03T05:13:04.082 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-argparse-2:17.0.0-14661.g4d487742.el8 10/25 2022-09-03T05:13:04.226 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librgw2-2:17.0.0-14661.g4d487742.el8.x86_64 16/25 2022-09-03T05:13:04.297 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librgw2-2:17.0.0-14661.g4d487742.el8.x86_64 16/25 2022-09-03T05:13:04.397 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cephfs-2:17.0.0-14661.g4d487742.el8.x86_64 11/25 2022-09-03T05:13:04.468 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rgw-2:17.0.0-14661.g4d487742.el8.x86_64 17/25 2022-09-03T05:13:04.618 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2022-09-03T05:13:04.655 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rbd-2:17.0.0-14661.g4d487742.el8.x86_64 12/25 2022-09-03T05:13:04.761 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-common-2:17.0.0-14661.g4d487742.el8.x 19/25 2022-09-03T05:13:04.830 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libradosstriper1-2:17.0.0-14661.g4d487742.el8.x86_ 13/25 2022-09-03T05:13:04.884 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libradosstriper1-2:17.0.0-14661.g4d487742.el8.x86_ 13/25 2022-09-03T05:13:05.111 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2022-09-03T05:13:05.865 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2022-09-03T05:13:06.073 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librgw2-2:17.0.0-14661.g4d487742.el8.x86_64 16/25 2022-09-03T05:13:06.181 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librgw2-2:17.0.0-14661.g4d487742.el8.x86_64 16/25 2022-09-03T05:13:06.420 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rgw-2:17.0.0-14661.g4d487742.el8.x86_64 17/25 2022-09-03T05:13:06.604 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2022-09-03T05:13:06.780 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-common-2:17.0.0-14661.g4d487742.el8.x 19/25 2022-09-03T05:13:08.593 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:17.0.0-14661.g4d487742.el8.x86_64 20/25 2022-09-03T05:13:08.838 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-common-2:17.0.0-14661.g4d487742.el8.x86_64 20/25 2022-09-03T05:13:09.430 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:17.0.0-14661.g4d487742.el8.x86_64 20/25 2022-09-03T05:13:09.657 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-base-2:17.0.0-14661.g4d487742.el8.x86_64 21/25 2022-09-03T05:13:09.754 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-base-2:17.0.0-14661.g4d487742.el8.x86_64 21/25 2022-09-03T05:13:09.901 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-selinux-2:17.0.0-14661.g4d487742.el8.x86_64 22/25 2022-09-03T05:13:10.647 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:17.0.0-14661.g4d487742.el8.x86_64 20/25 2022-09-03T05:13:10.922 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-common-2:17.0.0-14661.g4d487742.el8.x86_64 20/25 2022-09-03T05:13:11.518 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:17.0.0-14661.g4d487742.el8.x86_64 20/25 2022-09-03T05:13:11.830 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-base-2:17.0.0-14661.g4d487742.el8.x86_64 21/25 2022-09-03T05:13:11.939 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-base-2:17.0.0-14661.g4d487742.el8.x86_64 21/25 2022-09-03T05:13:12.120 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-selinux-2:17.0.0-14661.g4d487742.el8.x86_64 22/25 2022-09-03T05:13:28.307 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-selinux-2:17.0.0-14661.g4d487742.el8.x86_64 22/25 2022-09-03T05:13:28.308 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /sys 2022-09-03T05:13:28.308 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /proc 2022-09-03T05:13:28.309 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /dev 2022-09-03T05:13:28.309 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /run 2022-09-03T05:13:28.310 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /mnt 2022-09-03T05:13:28.311 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /var/tmp 2022-09-03T05:13:28.312 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /home 2022-09-03T05:13:28.312 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /tmp 2022-09-03T05:13:28.312 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /dev 2022-09-03T05:13:28.313 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:13:30.013 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-radosgw-2:17.0.0-14661.g4d487742.el8.x86_64 23/25 2022-09-03T05:13:30.035 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-radosgw-2:17.0.0-14661.g4d487742.el8.x86_64 23/25 2022-09-03T05:13:30.036 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2022-09-03T05:13:30.250 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2022-09-03T05:13:30.251 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2022-09-03T05:13:30.855 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2022-09-03T05:13:30.855 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-base-2:17.0.0-14661.g4d487742.el8.x86_64 1/25 2022-09-03T05:13:30.856 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-common-2:17.0.0-14661.g4d487742.el8.x86_64 2/25 2022-09-03T05:13:30.856 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-radosgw-2:17.0.0-14661.g4d487742.el8.x86_64 3/25 2022-09-03T05:13:30.856 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-selinux-2:17.0.0-14661.g4d487742.el8.x86_64 4/25 2022-09-03T05:13:30.856 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs2-2:17.0.0-14661.g4d487742.el8.x86_64 5/25 2022-09-03T05:13:30.857 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libradosstriper1-2:17.0.0-14661.g4d487742.el8.x86_ 6/25 2022-09-03T05:13:30.857 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librgw2-2:17.0.0-14661.g4d487742.el8.x86_64 7/25 2022-09-03T05:13:30.857 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-argparse-2:17.0.0-14661.g4d487742.el8 8/25 2022-09-03T05:13:30.857 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-common-2:17.0.0-14661.g4d487742.el8.x 9/25 2022-09-03T05:13:30.866 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cephfs-2:17.0.0-14661.g4d487742.el8.x86_64 10/25 2022-09-03T05:13:30.866 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rados-2:17.0.0-14661.g4d487742.el8.x86_64 11/25 2022-09-03T05:13:30.867 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rbd-2:17.0.0-14661.g4d487742.el8.x86_64 12/25 2022-09-03T05:13:30.868 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rgw-2:17.0.0-14661.g4d487742.el8.x86_64 13/25 2022-09-03T05:13:30.868 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2022-09-03T05:13:30.869 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2022-09-03T05:13:30.870 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2022-09-03T05:13:30.870 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/25 2022-09-03T05:13:30.871 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2022-09-03T05:13:30.872 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2022-09-03T05:13:30.872 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2022-09-03T05:13:30.873 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2022-09-03T05:13:30.873 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:17.0.0-14661.g4d487742.el8.x86_64 22/25 2022-09-03T05:13:30.873 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2022-09-03T05:13:30.874 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:17.0.0-14661.g4d487742.el8.x86_64 24/25 2022-09-03T05:13:30.915 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-selinux-2:17.0.0-14661.g4d487742.el8.x86_64 22/25 2022-09-03T05:13:30.916 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /sys 2022-09-03T05:13:30.917 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /proc 2022-09-03T05:13:30.917 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /dev 2022-09-03T05:13:30.917 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /run 2022-09-03T05:13:30.917 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /mnt 2022-09-03T05:13:30.918 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /var/tmp 2022-09-03T05:13:30.918 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /home 2022-09-03T05:13:30.918 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /tmp 2022-09-03T05:13:30.918 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /dev 2022-09-03T05:13:30.919 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:13:31.630 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2022-09-03T05:13:31.631 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:13:31.631 INFO:teuthology.orchestra.run.smithi181.stdout:Upgraded: 2022-09-03T05:13:31.631 INFO:teuthology.orchestra.run.smithi181.stdout: librados2-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:31.632 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:31.632 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2022-09-03T05:13:31.632 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:31.633 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:31.633 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:31.641 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:31.641 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-09-03T05:13:31.642 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:31.642 INFO:teuthology.orchestra.run.smithi181.stdout: liboath-2.6.2-3.el8.x86_64 2022-09-03T05:13:31.643 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-09-03T05:13:31.643 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:31.643 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-09-03T05:13:31.644 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:31.645 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind-1.3.1-3.el8.x86_64 2022-09-03T05:13:31.646 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-09-03T05:13:31.647 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:31.648 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:31.648 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:31.649 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-09-03T05:13:31.649 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:31.650 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:31.651 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:31.652 INFO:teuthology.orchestra.run.smithi181.stdout: thrift-0.13.0-2.el8.x86_64 2022-09-03T05:13:31.653 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:13:31.653 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:13:31.909 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-test 2022-09-03T05:13:32.502 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:00:43 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:13:32.708 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-radosgw-2:17.0.0-14661.g4d487742.el8.x86_64 23/25 2022-09-03T05:13:32.729 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-radosgw-2:17.0.0-14661.g4d487742.el8.x86_64 23/25 2022-09-03T05:13:32.729 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2022-09-03T05:13:32.980 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2022-09-03T05:13:32.980 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2022-09-03T05:13:33.511 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:13:33.512 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:13:33.512 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2022-09-03T05:13:33.513 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:13:33.513 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2022-09-03T05:13:33.514 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test x86_64 2:17.0.0-14661.g4d487742.el8 ceph 54 M 2022-09-03T05:13:33.515 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2022-09-03T05:13:33.515 INFO:teuthology.orchestra.run.smithi181.stdout: jq x86_64 1.6-3.el8 CentOS-AppStream 202 k 2022-09-03T05:13:33.516 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite x86_64 2:17.0.0-14661.g4d487742.el8 ceph 136 k 2022-09-03T05:13:33.516 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2022-09-03T05:13:33.517 INFO:teuthology.orchestra.run.smithi181.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2022-09-03T05:13:33.517 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-PowerTools 71 k 2022-09-03T05:13:33.517 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:13:33.538 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2022-09-03T05:13:33.539 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:13:33.539 INFO:teuthology.orchestra.run.smithi181.stdout:Install 6 Packages 2022-09-03T05:13:33.539 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:13:33.539 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 55 M 2022-09-03T05:13:33.540 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 237 M 2022-09-03T05:13:33.540 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2022-09-03T05:13:33.579 INFO:teuthology.orchestra.run.smithi181.stdout:(1/6): jq-1.6-3.el8.x86_64.rpm 3.1 MB/s | 202 kB 00:00 2022-09-03T05:13:33.598 INFO:teuthology.orchestra.run.smithi181.stdout:(2/6): oniguruma-6.8.2-2.el8.x86_64.rpm 7.6 MB/s | 187 kB 00:00 2022-09-03T05:13:33.626 INFO:teuthology.orchestra.run.smithi181.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 13 MB/s | 323 kB 00:00 2022-09-03T05:13:33.666 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2022-09-03T05:13:33.674 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-base-2:17.0.0-14661.g4d487742.el8.x86_64 1/25 2022-09-03T05:13:33.675 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-common-2:17.0.0-14661.g4d487742.el8.x86_64 2/25 2022-09-03T05:13:33.675 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-radosgw-2:17.0.0-14661.g4d487742.el8.x86_64 3/25 2022-09-03T05:13:33.685 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-selinux-2:17.0.0-14661.g4d487742.el8.x86_64 4/25 2022-09-03T05:13:33.685 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs2-2:17.0.0-14661.g4d487742.el8.x86_64 5/25 2022-09-03T05:13:33.686 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libradosstriper1-2:17.0.0-14661.g4d487742.el8.x86_ 6/25 2022-09-03T05:13:33.687 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librgw2-2:17.0.0-14661.g4d487742.el8.x86_64 7/25 2022-09-03T05:13:33.687 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-argparse-2:17.0.0-14661.g4d487742.el8 8/25 2022-09-03T05:13:33.688 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-common-2:17.0.0-14661.g4d487742.el8.x 9/25 2022-09-03T05:13:33.688 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cephfs-2:17.0.0-14661.g4d487742.el8.x86_64 10/25 2022-09-03T05:13:33.688 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rados-2:17.0.0-14661.g4d487742.el8.x86_64 11/25 2022-09-03T05:13:33.689 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rbd-2:17.0.0-14661.g4d487742.el8.x86_64 12/25 2022-09-03T05:13:33.689 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rgw-2:17.0.0-14661.g4d487742.el8.x86_64 13/25 2022-09-03T05:13:33.690 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2022-09-03T05:13:33.690 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2022-09-03T05:13:33.691 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2022-09-03T05:13:33.691 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/25 2022-09-03T05:13:33.693 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2022-09-03T05:13:33.693 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2022-09-03T05:13:33.693 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2022-09-03T05:13:33.694 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2022-09-03T05:13:33.694 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:17.0.0-14661.g4d487742.el8.x86_64 22/25 2022-09-03T05:13:33.695 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2022-09-03T05:13:33.695 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:17.0.0-14661.g4d487742.el8.x86_64 24/25 2022-09-03T05:13:33.696 INFO:teuthology.orchestra.run.smithi181.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2022-09-03T05:13:33.727 INFO:teuthology.orchestra.run.smithi181.stdout:(5/6): libcephsqlite-17.0.0-14661.g4d487742.el8 633 kB/s | 136 kB 00:00 2022-09-03T05:13:34.535 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2022-09-03T05:13:34.536 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:13:34.536 INFO:teuthology.orchestra.run.smithi189.stdout:Upgraded: 2022-09-03T05:13:34.537 INFO:teuthology.orchestra.run.smithi189.stdout: librados2-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:34.537 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:34.537 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2022-09-03T05:13:34.543 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:34.544 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:34.544 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:34.545 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:34.545 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-09-03T05:13:34.546 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:34.547 INFO:teuthology.orchestra.run.smithi189.stdout: liboath-2.6.2-3.el8.x86_64 2022-09-03T05:13:34.547 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-09-03T05:13:34.548 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:34.548 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-09-03T05:13:34.549 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:34.549 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind-1.3.1-3.el8.x86_64 2022-09-03T05:13:34.550 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-09-03T05:13:34.550 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:34.552 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:34.553 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:34.553 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-09-03T05:13:34.554 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:34.554 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:34.555 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:34.556 INFO:teuthology.orchestra.run.smithi189.stdout: thrift-0.13.0-2.el8.x86_64 2022-09-03T05:13:34.556 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:13:34.557 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:13:34.736 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-test 2022-09-03T05:13:35.315 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:45 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:13:36.194 INFO:teuthology.orchestra.run.smithi181.stdout:(6/6): ceph-test-17.0.0-14661.g4d487742.el8.x86 20 MB/s | 54 MB 00:02 2022-09-03T05:13:36.195 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:13:36.195 INFO:teuthology.orchestra.run.smithi181.stdout:Total 20 MB/s | 55 MB 00:02 2022-09-03T05:13:36.236 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2022-09-03T05:13:36.256 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2022-09-03T05:13:36.256 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2022-09-03T05:13:36.343 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:13:36.343 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:13:36.347 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2022-09-03T05:13:36.348 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:13:36.348 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2022-09-03T05:13:36.348 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test x86_64 2:17.0.0-14661.g4d487742.el8 ceph 54 M 2022-09-03T05:13:36.348 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2022-09-03T05:13:36.349 INFO:teuthology.orchestra.run.smithi189.stdout: jq x86_64 1.6-3.el8 CentOS-AppStream 202 k 2022-09-03T05:13:36.349 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite x86_64 2:17.0.0-14661.g4d487742.el8 ceph 136 k 2022-09-03T05:13:36.349 INFO:teuthology.orchestra.run.smithi189.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2022-09-03T05:13:36.349 INFO:teuthology.orchestra.run.smithi189.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2022-09-03T05:13:36.349 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-PowerTools 71 k 2022-09-03T05:13:36.350 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:13:36.350 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2022-09-03T05:13:36.350 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:13:36.350 INFO:teuthology.orchestra.run.smithi189.stdout:Install 6 Packages 2022-09-03T05:13:36.351 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:13:36.351 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 55 M 2022-09-03T05:13:36.353 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 237 M 2022-09-03T05:13:36.353 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2022-09-03T05:13:36.418 INFO:teuthology.orchestra.run.smithi189.stdout:(1/6): jq-1.6-3.el8.x86_64.rpm 2.5 MB/s | 202 kB 00:00 2022-09-03T05:13:36.440 INFO:teuthology.orchestra.run.smithi189.stdout:(2/6): oniguruma-6.8.2-2.el8.x86_64.rpm 7.6 MB/s | 187 kB 00:00 2022-09-03T05:13:36.484 INFO:teuthology.orchestra.run.smithi189.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 9.6 MB/s | 323 kB 00:00 2022-09-03T05:13:36.499 INFO:teuthology.orchestra.run.smithi189.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 71 kB 00:00 2022-09-03T05:13:36.555 INFO:teuthology.orchestra.run.smithi189.stdout:(5/6): libcephsqlite-17.0.0-14661.g4d487742.el8 616 kB/s | 136 kB 00:00 2022-09-03T05:13:36.570 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2022-09-03T05:13:36.575 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2022-09-03T05:13:37.061 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2022-09-03T05:13:37.247 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2022-09-03T05:13:37.396 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2022-09-03T05:13:37.521 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/6 2022-09-03T05:13:37.603 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/6 2022-09-03T05:13:37.766 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : jq-1.6-3.el8.x86_64 4/6 2022-09-03T05:13:37.922 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephsqlite-2:17.0.0-14661.g4d487742.el8.x86_64 5/6 2022-09-03T05:13:38.895 INFO:teuthology.orchestra.run.smithi189.stdout:(6/6): ceph-test-17.0.0-14661.g4d487742.el8.x86 21 MB/s | 54 MB 00:02 2022-09-03T05:13:38.895 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:13:38.895 INFO:teuthology.orchestra.run.smithi189.stdout:Total 21 MB/s | 55 MB 00:02 2022-09-03T05:13:38.933 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2022-09-03T05:13:38.953 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2022-09-03T05:13:38.953 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2022-09-03T05:13:39.260 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2022-09-03T05:13:39.263 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2022-09-03T05:13:39.771 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2022-09-03T05:13:39.972 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2022-09-03T05:13:40.175 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2022-09-03T05:13:40.365 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/6 2022-09-03T05:13:40.476 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/6 2022-09-03T05:13:40.660 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : jq-1.6-3.el8.x86_64 4/6 2022-09-03T05:13:40.882 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephsqlite-2:17.0.0-14661.g4d487742.el8.x86_64 5/6 2022-09-03T05:13:43.050 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephsqlite-2:17.0.0-14661.g4d487742.el8.x86_64 5/6 2022-09-03T05:13:43.249 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-test-2:17.0.0-14661.g4d487742.el8.x86_64 6/6 2022-09-03T05:13:43.484 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-test-2:17.0.0-14661.g4d487742.el8.x86_64 6/6 2022-09-03T05:13:43.485 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-test-2:17.0.0-14661.g4d487742.el8.x86_64 1/6 2022-09-03T05:13:43.485 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephsqlite-2:17.0.0-14661.g4d487742.el8.x86_64 2/6 2022-09-03T05:13:43.485 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : jq-1.6-3.el8.x86_64 3/6 2022-09-03T05:13:43.486 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 4/6 2022-09-03T05:13:43.486 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2022-09-03T05:13:43.965 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2022-09-03T05:13:43.965 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:13:43.966 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2022-09-03T05:13:43.966 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:43.967 INFO:teuthology.orchestra.run.smithi181.stdout: jq-1.6-3.el8.x86_64 2022-09-03T05:13:43.967 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:43.967 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma-6.8.2-2.el8.x86_64 2022-09-03T05:13:43.968 INFO:teuthology.orchestra.run.smithi181.stdout: socat-1.7.4.1-1.el8.x86_64 2022-09-03T05:13:43.968 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-09-03T05:13:43.969 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:13:43.969 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:13:44.185 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph 2022-09-03T05:13:44.768 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:00:55 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:13:45.762 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:13:45.766 INFO:teuthology.orchestra.run.smithi181.stdout:======================================================================================== 2022-09-03T05:13:45.766 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2022-09-03T05:13:45.767 INFO:teuthology.orchestra.run.smithi181.stdout:======================================================================================== 2022-09-03T05:13:45.767 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2022-09-03T05:13:45.768 INFO:teuthology.orchestra.run.smithi181.stdout: ceph x86_64 2:17.0.0-14661.g4d487742.el8 ceph 6.4 k 2022-09-03T05:13:45.768 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2022-09-03T05:13:45.768 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds x86_64 2:17.0.0-14661.g4d487742.el8 ceph 2.3 M 2022-09-03T05:13:45.769 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr x86_64 2:17.0.0-14661.g4d487742.el8 ceph 1.5 M 2022-09-03T05:13:45.769 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core noarch 2:17.0.0-14661.g4d487742.el8 ceph-noarch 233 k 2022-09-03T05:13:45.770 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon x86_64 2:17.0.0-14661.g4d487742.el8 ceph 3.9 M 2022-09-03T05:13:45.770 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd x86_64 2:17.0.0-14661.g4d487742.el8 ceph 17 M 2022-09-03T05:13:45.771 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-09-03T05:13:45.771 INFO:teuthology.orchestra.run.smithi181.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-09-03T05:13:45.772 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2022-09-03T05:13:45.774 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-09-03T05:13:45.774 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-09-03T05:13:45.775 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2022-09-03T05:13:45.775 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-09-03T05:13:45.776 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-09-03T05:13:45.777 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-09-03T05:13:45.777 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2022-09-03T05:13:45.777 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2022-09-03T05:13:45.778 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-09-03T05:13:45.778 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-09-03T05:13:45.778 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-09-03T05:13:45.779 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2022-09-03T05:13:45.779 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2022-09-03T05:13:45.779 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2022-09-03T05:13:45.779 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-09-03T05:13:45.780 INFO:teuthology.orchestra.run.smithi181.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-09-03T05:13:45.780 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-09-03T05:13:45.780 INFO:teuthology.orchestra.run.smithi181.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-09-03T05:13:45.781 INFO:teuthology.orchestra.run.smithi181.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-09-03T05:13:45.781 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-09-03T05:13:45.781 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-09-03T05:13:45.781 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2022-09-03T05:13:45.781 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-09-03T05:13:45.782 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:13:45.782 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2022-09-03T05:13:45.783 INFO:teuthology.orchestra.run.smithi181.stdout:======================================================================================== 2022-09-03T05:13:45.783 INFO:teuthology.orchestra.run.smithi181.stdout:Install 32 Packages 2022-09-03T05:13:45.783 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:13:45.785 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 28 M 2022-09-03T05:13:45.785 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 110 M 2022-09-03T05:13:45.785 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2022-09-03T05:13:45.925 INFO:teuthology.orchestra.run.smithi181.stdout:(1/32): ceph-17.0.0-14661.g4d487742.el8.x86_64. 42 kB/s | 6.4 kB 00:00 2022-09-03T05:13:46.035 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephsqlite-2:17.0.0-14661.g4d487742.el8.x86_64 5/6 2022-09-03T05:13:46.125 INFO:teuthology.orchestra.run.smithi181.stdout:(2/32): ceph-mgr-17.0.0-14661.g4d487742.el8.x86 4.3 MB/s | 1.5 MB 00:00 2022-09-03T05:13:46.192 INFO:teuthology.orchestra.run.smithi181.stdout:(3/32): ceph-mds-17.0.0-14661.g4d487742.el8.x86 5.4 MB/s | 2.3 MB 00:00 2022-09-03T05:13:46.268 INFO:teuthology.orchestra.run.smithi181.stdout:(4/32): ceph-mgr-modules-core-17.0.0-14661.g4d4 3.0 MB/s | 233 kB 00:00 2022-09-03T05:13:46.320 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-test-2:17.0.0-14661.g4d487742.el8.x86_64 6/6 2022-09-03T05:13:46.352 INFO:teuthology.orchestra.run.smithi181.stdout:(5/32): python3-mako-1.0.6-13.el8.noarch.rpm 1.8 MB/s | 157 kB 00:00 2022-09-03T05:13:46.385 INFO:teuthology.orchestra.run.smithi181.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 1.1 MB/s | 39 kB 00:00 2022-09-03T05:13:46.419 INFO:teuthology.orchestra.run.smithi181.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 3.0 MB/s | 103 kB 00:00 2022-09-03T05:13:46.444 INFO:teuthology.orchestra.run.smithi181.stdout:(8/32): python3-pytz-2017.2-9.el8.noarch.rpm 2.1 MB/s | 54 kB 00:00 2022-09-03T05:13:46.479 INFO:teuthology.orchestra.run.smithi181.stdout:(9/32): python3-werkzeug-0.12.2-4.el8.noarch.rp 13 MB/s | 457 kB 00:00 2022-09-03T05:13:46.507 INFO:teuthology.orchestra.run.smithi181.stdout:(10/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 9.3 MB/s | 237 kB 00:00 2022-09-03T05:13:46.545 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-test-2:17.0.0-14661.g4d487742.el8.x86_64 6/6 2022-09-03T05:13:46.545 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-test-2:17.0.0-14661.g4d487742.el8.x86_64 1/6 2022-09-03T05:13:46.546 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephsqlite-2:17.0.0-14661.g4d487742.el8.x86_64 2/6 2022-09-03T05:13:46.546 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : jq-1.6-3.el8.x86_64 3/6 2022-09-03T05:13:46.546 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 4/6 2022-09-03T05:13:46.546 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2022-09-03T05:13:46.561 INFO:teuthology.orchestra.run.smithi181.stdout:(11/32): ceph-mon-17.0.0-14661.g4d487742.el8.x8 6.1 MB/s | 3.9 MB 00:00 2022-09-03T05:13:46.603 INFO:teuthology.orchestra.run.smithi181.stdout:(12/32): python3-cryptography-3.2.1-5.el8.x86_6 5.5 MB/s | 559 kB 00:00 2022-09-03T05:13:46.630 INFO:teuthology.orchestra.run.smithi181.stdout:(13/32): python3-pycparser-2.14-14.el8.noarch.r 1.6 MB/s | 109 kB 00:00 2022-09-03T05:13:46.737 INFO:teuthology.orchestra.run.smithi181.stdout:(14/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 328 kB/s | 44 kB 00:00 2022-09-03T05:13:46.796 INFO:teuthology.orchestra.run.smithi181.stdout:(15/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 1.1 MB/s | 185 kB 00:00 2022-09-03T05:13:46.846 INFO:teuthology.orchestra.run.smithi181.stdout:(16/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.6 MB/s | 173 kB 00:00 2022-09-03T05:13:46.880 INFO:teuthology.orchestra.run.smithi181.stdout:(17/32): python3-cherrypy-18.4.0-1.el8.noarch.r 4.5 MB/s | 384 kB 00:00 2022-09-03T05:13:46.905 INFO:teuthology.orchestra.run.smithi181.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2022-09-03T05:13:46.930 INFO:teuthology.orchestra.run.smithi181.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 350 kB/s | 18 kB 00:00 2022-09-03T05:13:46.964 INFO:teuthology.orchestra.run.smithi181.stdout:(20/32): python3-logutils-0.3.5-11.el8.noarch.r 831 kB/s | 49 kB 00:00 2022-09-03T05:13:46.989 INFO:teuthology.orchestra.run.smithi181.stdout:(21/32): python3-more-itertools-7.2.0-3.el8.noa 1.0 MB/s | 59 kB 00:00 2022-09-03T05:13:47.006 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2022-09-03T05:13:47.007 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:13:47.008 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2022-09-03T05:13:47.008 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:47.009 INFO:teuthology.orchestra.run.smithi189.stdout: jq-1.6-3.el8.x86_64 2022-09-03T05:13:47.009 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:47.010 INFO:teuthology.orchestra.run.smithi189.stdout: oniguruma-6.8.2-2.el8.x86_64 2022-09-03T05:13:47.010 INFO:teuthology.orchestra.run.smithi189.stdout: socat-1.7.4.1-1.el8.x86_64 2022-09-03T05:13:47.011 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-09-03T05:13:47.011 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:13:47.011 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:13:47.046 INFO:teuthology.orchestra.run.smithi181.stdout:(22/32): python3-pecan-1.3.2-9.el8.noarch.rpm 3.4 MB/s | 283 kB 00:00 2022-09-03T05:13:47.073 INFO:teuthology.orchestra.run.smithi181.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 186 kB/s | 16 kB 00:00 2022-09-03T05:13:47.101 INFO:teuthology.orchestra.run.smithi181.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 352 kB/s | 19 kB 00:00 2022-09-03T05:13:47.124 INFO:teuthology.orchestra.run.smithi181.stdout:(25/32): python3-singledispatch-3.4.0.3-18.el8. 474 kB/s | 24 kB 00:00 2022-09-03T05:13:47.152 INFO:teuthology.orchestra.run.smithi181.stdout:(26/32): python3-tempora-1.14.1-5.el8.noarch.rp 579 kB/s | 29 kB 00:00 2022-09-03T05:13:47.174 INFO:teuthology.orchestra.run.smithi181.stdout:(27/32): python3-trustme-0.6.0-4.el8.noarch.rpm 546 kB/s | 27 kB 00:00 2022-09-03T05:13:47.208 INFO:teuthology.orchestra.run.smithi181.stdout:(28/32): python3-waitress-1.2.1-2.el8.1.noarch. 4.0 MB/s | 241 kB 00:00 2022-09-03T05:13:47.234 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph 2022-09-03T05:13:47.376 INFO:teuthology.orchestra.run.smithi181.stdout:(29/32): ceph-osd-17.0.0-14661.g4d487742.el8.x8 13 MB/s | 17 MB 00:01 2022-09-03T05:13:47.401 INFO:teuthology.orchestra.run.smithi181.stdout:(30/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.1 MB/s | 251 kB 00:00 2022-09-03T05:13:47.425 INFO:teuthology.orchestra.run.smithi181.stdout:(31/32): python3-webtest-2.0.33-1.el8.noarch.rp 393 kB/s | 85 kB 00:00 2022-09-03T05:13:47.492 INFO:teuthology.orchestra.run.smithi181.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 197 kB/s | 23 kB 00:00 2022-09-03T05:13:47.496 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:13:47.497 INFO:teuthology.orchestra.run.smithi181.stdout:Total 16 MB/s | 28 MB 00:01 2022-09-03T05:13:47.570 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2022-09-03T05:13:47.607 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2022-09-03T05:13:47.608 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2022-09-03T05:13:47.810 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:57 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:13:47.908 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2022-09-03T05:13:47.911 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2022-09-03T05:13:48.832 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:13:48.835 INFO:teuthology.orchestra.run.smithi189.stdout:======================================================================================== 2022-09-03T05:13:48.836 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2022-09-03T05:13:48.836 INFO:teuthology.orchestra.run.smithi189.stdout:======================================================================================== 2022-09-03T05:13:48.836 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2022-09-03T05:13:48.837 INFO:teuthology.orchestra.run.smithi189.stdout: ceph x86_64 2:17.0.0-14661.g4d487742.el8 ceph 6.4 k 2022-09-03T05:13:48.837 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2022-09-03T05:13:48.837 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds x86_64 2:17.0.0-14661.g4d487742.el8 ceph 2.3 M 2022-09-03T05:13:48.837 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr x86_64 2:17.0.0-14661.g4d487742.el8 ceph 1.5 M 2022-09-03T05:13:48.838 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core noarch 2:17.0.0-14661.g4d487742.el8 ceph-noarch 233 k 2022-09-03T05:13:48.838 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon x86_64 2:17.0.0-14661.g4d487742.el8 ceph 3.9 M 2022-09-03T05:13:48.838 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd x86_64 2:17.0.0-14661.g4d487742.el8 ceph 17 M 2022-09-03T05:13:48.839 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-09-03T05:13:48.839 INFO:teuthology.orchestra.run.smithi189.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-09-03T05:13:48.839 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2022-09-03T05:13:48.839 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-09-03T05:13:48.840 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-09-03T05:13:48.840 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2022-09-03T05:13:48.840 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-09-03T05:13:48.840 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-09-03T05:13:48.841 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-09-03T05:13:48.841 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2022-09-03T05:13:48.842 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2022-09-03T05:13:48.843 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-09-03T05:13:48.843 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-09-03T05:13:48.843 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-09-03T05:13:48.843 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2022-09-03T05:13:48.844 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2022-09-03T05:13:48.844 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2022-09-03T05:13:48.844 INFO:teuthology.orchestra.run.smithi189.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-09-03T05:13:48.844 INFO:teuthology.orchestra.run.smithi189.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-09-03T05:13:48.844 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-09-03T05:13:48.845 INFO:teuthology.orchestra.run.smithi189.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-09-03T05:13:48.845 INFO:teuthology.orchestra.run.smithi189.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-09-03T05:13:48.845 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-09-03T05:13:48.846 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-09-03T05:13:48.846 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2022-09-03T05:13:48.846 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-09-03T05:13:48.846 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:13:48.847 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2022-09-03T05:13:48.847 INFO:teuthology.orchestra.run.smithi189.stdout:======================================================================================== 2022-09-03T05:13:48.847 INFO:teuthology.orchestra.run.smithi189.stdout:Install 32 Packages 2022-09-03T05:13:48.847 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:13:48.848 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 28 M 2022-09-03T05:13:48.849 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 110 M 2022-09-03T05:13:48.849 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2022-09-03T05:13:48.972 INFO:teuthology.orchestra.run.smithi189.stdout:(1/32): ceph-17.0.0-14661.g4d487742.el8.x86_64. 49 kB/s | 6.4 kB 00:00 2022-09-03T05:13:49.045 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2022-09-03T05:13:49.206 INFO:teuthology.orchestra.run.smithi189.stdout:(2/32): ceph-mds-17.0.0-14661.g4d487742.el8.x86 6.2 MB/s | 2.3 MB 00:00 2022-09-03T05:13:49.248 INFO:teuthology.orchestra.run.smithi189.stdout:(3/32): ceph-mgr-17.0.0-14661.g4d487742.el8.x86 3.8 MB/s | 1.5 MB 00:00 2022-09-03T05:13:49.268 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2022-09-03T05:13:49.358 INFO:teuthology.orchestra.run.smithi189.stdout:(4/32): ceph-mgr-modules-core-17.0.0-14661.g4d4 2.1 MB/s | 233 kB 00:00 2022-09-03T05:13:49.415 INFO:teuthology.orchestra.run.smithi189.stdout:(5/32): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2022-09-03T05:13:49.443 INFO:teuthology.orchestra.run.smithi189.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 1.5 MB/s | 39 kB 00:00 2022-09-03T05:13:49.466 INFO:teuthology.orchestra.run.smithi189.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 4.0 MB/s | 103 kB 00:00 2022-09-03T05:13:49.491 INFO:teuthology.orchestra.run.smithi189.stdout:(8/32): python3-pytz-2017.2-9.el8.noarch.rpm 2.1 MB/s | 54 kB 00:00 2022-09-03T05:13:49.502 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2022-09-03T05:13:49.560 INFO:teuthology.orchestra.run.smithi189.stdout:(9/32): python3-werkzeug-0.12.2-4.el8.noarch.rp 6.5 MB/s | 457 kB 00:00 2022-09-03T05:13:49.599 INFO:teuthology.orchestra.run.smithi189.stdout:(10/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 5.9 MB/s | 237 kB 00:00 2022-09-03T05:13:49.633 INFO:teuthology.orchestra.run.smithi189.stdout:(11/32): python3-cryptography-3.2.1-5.el8.x86_6 16 MB/s | 559 kB 00:00 2022-09-03T05:13:49.658 INFO:teuthology.orchestra.run.smithi189.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 4.2 MB/s | 109 kB 00:00 2022-09-03T05:13:49.689 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2022-09-03T05:13:49.809 INFO:teuthology.orchestra.run.smithi189.stdout:(13/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 291 kB/s | 44 kB 00:00 2022-09-03T05:13:49.867 INFO:teuthology.orchestra.run.smithi189.stdout:(14/32): ceph-mon-17.0.0-14661.g4d487742.el8.x8 4.3 MB/s | 3.9 MB 00:00 2022-09-03T05:13:49.884 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2022-09-03T05:13:49.909 INFO:teuthology.orchestra.run.smithi189.stdout:(15/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 1.8 MB/s | 185 kB 00:00 2022-09-03T05:13:50.039 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2022-09-03T05:13:50.102 INFO:teuthology.orchestra.run.smithi189.stdout:(16/32): ceph-osd-17.0.0-14661.g4d487742.el8.x8 19 MB/s | 17 MB 00:00 2022-09-03T05:13:50.126 INFO:teuthology.orchestra.run.smithi189.stdout:(17/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 667 kB/s | 173 kB 00:00 2022-09-03T05:13:50.152 INFO:teuthology.orchestra.run.smithi189.stdout:(18/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.6 MB/s | 384 kB 00:00 2022-09-03T05:13:50.177 INFO:teuthology.orchestra.run.smithi189.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 349 kB/s | 18 kB 00:00 2022-09-03T05:13:50.186 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2022-09-03T05:13:50.202 INFO:teuthology.orchestra.run.smithi189.stdout:(20/32): python3-logutils-0.3.5-11.el8.noarch.r 972 kB/s | 49 kB 00:00 2022-09-03T05:13:50.227 INFO:teuthology.orchestra.run.smithi189.stdout:(21/32): python3-jaraco-6.2-6.el8.noarch.rpm 88 kB/s | 11 kB 00:00 2022-09-03T05:13:50.294 INFO:teuthology.orchestra.run.smithi189.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 500 kB/s | 59 kB 00:00 2022-09-03T05:13:50.321 INFO:teuthology.orchestra.run.smithi189.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 170 kB/s | 16 kB 00:00 2022-09-03T05:13:50.345 INFO:teuthology.orchestra.run.smithi189.stdout:(24/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.9 MB/s | 283 kB 00:00 2022-09-03T05:13:50.355 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2022-09-03T05:13:50.370 INFO:teuthology.orchestra.run.smithi189.stdout:(25/32): python3-simplegeneric-0.8.1-17.el8.noa 248 kB/s | 19 kB 00:00 2022-09-03T05:13:50.396 INFO:teuthology.orchestra.run.smithi189.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 316 kB/s | 24 kB 00:00 2022-09-03T05:13:50.421 INFO:teuthology.orchestra.run.smithi189.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 388 kB/s | 29 kB 00:00 2022-09-03T05:13:50.455 INFO:teuthology.orchestra.run.smithi189.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 326 kB/s | 27 kB 00:00 2022-09-03T05:13:50.491 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2022-09-03T05:13:50.505 INFO:teuthology.orchestra.run.smithi189.stdout:(29/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.9 MB/s | 251 kB 00:00 2022-09-03T05:13:50.530 INFO:teuthology.orchestra.run.smithi189.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.1 MB/s | 85 kB 00:00 2022-09-03T05:13:50.556 INFO:teuthology.orchestra.run.smithi189.stdout:(31/32): python3-waitress-1.2.1-2.el8.1.noarch. 1.5 MB/s | 241 kB 00:00 2022-09-03T05:13:50.580 INFO:teuthology.orchestra.run.smithi189.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 306 kB/s | 23 kB 00:00 2022-09-03T05:13:50.583 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:13:50.583 INFO:teuthology.orchestra.run.smithi189.stdout:Total 16 MB/s | 28 MB 00:01 2022-09-03T05:13:50.659 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2022-09-03T05:13:50.696 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2022-09-03T05:13:50.697 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2022-09-03T05:13:50.706 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2022-09-03T05:13:50.918 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2022-09-03T05:13:50.985 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2022-09-03T05:13:50.994 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2022-09-03T05:13:51.130 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2022-09-03T05:13:51.373 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2022-09-03T05:13:51.744 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2022-09-03T05:13:51.999 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2022-09-03T05:13:52.151 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2022-09-03T05:13:52.268 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2022-09-03T05:13:52.350 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2022-09-03T05:13:52.526 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2022-09-03T05:13:52.544 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2022-09-03T05:13:52.821 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2022-09-03T05:13:52.886 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2022-09-03T05:13:53.059 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2022-09-03T05:13:53.091 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2022-09-03T05:13:53.246 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2022-09-03T05:13:53.283 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2022-09-03T05:13:53.482 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2022-09-03T05:13:53.483 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2022-09-03T05:13:53.688 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2022-09-03T05:13:53.708 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2022-09-03T05:13:53.889 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2022-09-03T05:13:53.892 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2022-09-03T05:13:54.055 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2022-09-03T05:13:54.087 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2022-09-03T05:13:54.264 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2022-09-03T05:13:54.420 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 25/32 2022-09-03T05:13:54.546 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2022-09-03T05:13:54.739 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2022-09-03T05:13:54.784 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2022-09-03T05:13:55.005 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2022-09-03T05:13:55.150 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-modules-core-2:17.0.0-14661.g4d487742.el8 27/32 2022-09-03T05:13:55.289 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-2:17.0.0-14661.g4d487742.el8.x86_64 28/32 2022-09-03T05:13:55.444 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2022-09-03T05:13:55.723 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2022-09-03T05:13:55.920 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2022-09-03T05:13:56.169 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2022-09-03T05:13:56.397 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2022-09-03T05:13:56.721 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2022-09-03T05:13:56.781 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-2:17.0.0-14661.g4d487742.el8.x86_64 28/32 2022-09-03T05:13:56.976 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2022-09-03T05:13:56.986 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-osd-2:17.0.0-14661.g4d487742.el8.x86_64 29/32 2022-09-03T05:13:57.158 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2022-09-03T05:13:57.364 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-osd-2:17.0.0-14661.g4d487742.el8.x86_64 29/32 2022-09-03T05:13:57.456 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2022-09-03T05:13:57.534 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mon-2:17.0.0-14661.g4d487742.el8.x86_64 30/32 2022-09-03T05:13:57.696 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2022-09-03T05:13:57.738 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mon-2:17.0.0-14661.g4d487742.el8.x86_64 30/32 2022-09-03T05:13:57.923 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mds-2:17.0.0-14661.g4d487742.el8.x86_64 31/32 2022-09-03T05:13:57.935 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mds-2:17.0.0-14661.g4d487742.el8.x86_64 31/32 2022-09-03T05:13:57.958 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2022-09-03T05:13:58.058 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-2:17.0.0-14661.g4d487742.el8.x86_64 32/32 2022-09-03T05:13:58.223 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2022-09-03T05:13:58.454 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-2:17.0.0-14661.g4d487742.el8.x86_64 32/32 2022-09-03T05:13:58.455 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-2:17.0.0-14661.g4d487742.el8.x86_64 1/32 2022-09-03T05:13:58.456 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mds-2:17.0.0-14661.g4d487742.el8.x86_64 2/32 2022-09-03T05:13:58.486 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-2:17.0.0-14661.g4d487742.el8.x86_64 3/32 2022-09-03T05:13:58.488 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mon-2:17.0.0-14661.g4d487742.el8.x86_64 4/32 2022-09-03T05:13:58.489 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-osd-2:17.0.0-14661.g4d487742.el8.x86_64 5/32 2022-09-03T05:13:58.491 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-modules-core-2:17.0.0-14661.g4d487742.el8 6/32 2022-09-03T05:13:58.491 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 7/32 2022-09-03T05:13:58.492 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2022-09-03T05:13:58.493 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2022-09-03T05:13:58.495 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2022-09-03T05:13:58.496 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2022-09-03T05:13:58.497 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2022-09-03T05:13:58.497 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2022-09-03T05:13:58.498 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2022-09-03T05:13:58.500 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2022-09-03T05:13:58.500 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2022-09-03T05:13:58.502 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2022-09-03T05:13:58.502 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2022-09-03T05:13:58.503 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2022-09-03T05:13:58.503 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2022-09-03T05:13:58.504 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2022-09-03T05:13:58.504 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2022-09-03T05:13:58.505 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2022-09-03T05:13:58.506 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2022-09-03T05:13:58.508 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2022-09-03T05:13:58.509 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2022-09-03T05:13:58.509 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2022-09-03T05:13:58.510 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2022-09-03T05:13:58.510 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2022-09-03T05:13:58.511 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2022-09-03T05:13:58.511 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2022-09-03T05:13:58.600 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 25/32 2022-09-03T05:13:58.940 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2022-09-03T05:13:59.279 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2022-09-03T05:13:59.279 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:13:59.280 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2022-09-03T05:13:59.280 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:59.281 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:59.281 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:59.286 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core-2:17.0.0-14661.g4d487742.el8.noarch 2022-09-03T05:13:59.286 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:59.287 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:13:59.288 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-09-03T05:13:59.288 INFO:teuthology.orchestra.run.smithi181.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-09-03T05:13:59.289 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-09-03T05:13:59.289 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-09-03T05:13:59.290 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-09-03T05:13:59.290 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-09-03T05:13:59.291 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-6.2-6.el8.noarch 2022-09-03T05:13:59.291 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-09-03T05:13:59.292 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-09-03T05:13:59.298 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako-1.0.6-13.el8.noarch 2022-09-03T05:13:59.303 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-09-03T05:13:59.307 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-09-03T05:13:59.311 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-09-03T05:13:59.313 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend-2.6-1.el8.noarch 2022-09-03T05:13:59.314 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-09-03T05:13:59.315 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser-2.14-14.el8.noarch 2022-09-03T05:13:59.319 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz-2017.2-9.el8.noarch 2022-09-03T05:13:59.320 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-09-03T05:13:59.321 INFO:teuthology.orchestra.run.smithi181.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-09-03T05:13:59.321 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-09-03T05:13:59.322 INFO:teuthology.orchestra.run.smithi181.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-09-03T05:13:59.359 INFO:teuthology.orchestra.run.smithi181.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-09-03T05:13:59.360 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-09-03T05:13:59.361 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-09-03T05:13:59.363 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-09-03T05:13:59.364 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-09-03T05:13:59.364 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:13:59.364 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:13:59.374 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-modules-core-2:17.0.0-14661.g4d487742.el8 27/32 2022-09-03T05:13:59.480 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-base 2022-09-03T05:13:59.617 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-2:17.0.0-14661.g4d487742.el8.x86_64 28/32 2022-09-03T05:14:00.054 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:11 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:14:00.960 INFO:teuthology.orchestra.run.smithi181.stdout:Package ceph-base-2:17.0.0-14661.g4d487742.el8.x86_64 is already installed. 2022-09-03T05:14:01.036 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:14:01.038 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2022-09-03T05:14:01.038 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:14:01.084 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-2:17.0.0-14661.g4d487742.el8.x86_64 28/32 2022-09-03T05:14:01.110 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install cephadm 2022-09-03T05:14:01.333 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-osd-2:17.0.0-14661.g4d487742.el8.x86_64 29/32 2022-09-03T05:14:01.678 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:12 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:14:01.716 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-osd-2:17.0.0-14661.g4d487742.el8.x86_64 29/32 2022-09-03T05:14:01.931 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mon-2:17.0.0-14661.g4d487742.el8.x86_64 30/32 2022-09-03T05:14:02.141 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mon-2:17.0.0-14661.g4d487742.el8.x86_64 30/32 2022-09-03T05:14:02.304 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mds-2:17.0.0-14661.g4d487742.el8.x86_64 31/32 2022-09-03T05:14:02.321 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mds-2:17.0.0-14661.g4d487742.el8.x86_64 31/32 2022-09-03T05:14:02.468 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-2:17.0.0-14661.g4d487742.el8.x86_64 32/32 2022-09-03T05:14:02.682 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:14:02.683 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:02.683 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2022-09-03T05:14:02.683 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:02.684 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2022-09-03T05:14:02.684 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm noarch 2:17.0.0-14661.g4d487742.el8 ceph-noarch 81 k 2022-09-03T05:14:02.684 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:02.684 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2022-09-03T05:14:02.685 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:02.685 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2022-09-03T05:14:02.685 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:02.685 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 81 k 2022-09-03T05:14:02.685 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 367 k 2022-09-03T05:14:02.686 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2022-09-03T05:14:02.883 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-2:17.0.0-14661.g4d487742.el8.x86_64 32/32 2022-09-03T05:14:02.884 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-2:17.0.0-14661.g4d487742.el8.x86_64 1/32 2022-09-03T05:14:02.884 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mds-2:17.0.0-14661.g4d487742.el8.x86_64 2/32 2022-09-03T05:14:02.884 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-2:17.0.0-14661.g4d487742.el8.x86_64 3/32 2022-09-03T05:14:02.885 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mon-2:17.0.0-14661.g4d487742.el8.x86_64 4/32 2022-09-03T05:14:02.885 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-osd-2:17.0.0-14661.g4d487742.el8.x86_64 5/32 2022-09-03T05:14:02.885 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-modules-core-2:17.0.0-14661.g4d487742.el8 6/32 2022-09-03T05:14:02.885 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 7/32 2022-09-03T05:14:02.885 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2022-09-03T05:14:02.886 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2022-09-03T05:14:02.886 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2022-09-03T05:14:02.886 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2022-09-03T05:14:02.886 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2022-09-03T05:14:02.887 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2022-09-03T05:14:02.887 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2022-09-03T05:14:02.887 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2022-09-03T05:14:02.887 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2022-09-03T05:14:02.888 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2022-09-03T05:14:02.888 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2022-09-03T05:14:02.888 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2022-09-03T05:14:02.888 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2022-09-03T05:14:02.889 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2022-09-03T05:14:02.889 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2022-09-03T05:14:02.889 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2022-09-03T05:14:02.889 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2022-09-03T05:14:02.890 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2022-09-03T05:14:02.890 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2022-09-03T05:14:02.890 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2022-09-03T05:14:02.890 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2022-09-03T05:14:02.891 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2022-09-03T05:14:02.892 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2022-09-03T05:14:02.892 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2022-09-03T05:14:03.359 INFO:teuthology.orchestra.run.smithi181.stdout:cephadm-17.0.0-14661.g4d487742.el8.noarch.rpm 120 kB/s | 81 kB 00:00 2022-09-03T05:14:03.359 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:14:03.360 INFO:teuthology.orchestra.run.smithi181.stdout:Total 120 kB/s | 81 kB 00:00 2022-09-03T05:14:03.360 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2022-09-03T05:14:03.374 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2022-09-03T05:14:03.375 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2022-09-03T05:14:03.375 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2022-09-03T05:14:03.375 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2022-09-03T05:14:03.625 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2022-09-03T05:14:03.637 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2022-09-03T05:14:03.638 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:03.638 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2022-09-03T05:14:03.640 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:14:03.640 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:14:03.642 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:14:03.642 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core-2:17.0.0-14661.g4d487742.el8.noarch 2022-09-03T05:14:03.643 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:14:03.643 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:14:03.644 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-09-03T05:14:03.644 INFO:teuthology.orchestra.run.smithi189.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-09-03T05:14:03.644 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-09-03T05:14:03.645 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-09-03T05:14:03.645 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-09-03T05:14:03.646 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-09-03T05:14:03.646 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-6.2-6.el8.noarch 2022-09-03T05:14:03.648 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-09-03T05:14:03.648 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-09-03T05:14:03.648 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako-1.0.6-13.el8.noarch 2022-09-03T05:14:03.649 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-09-03T05:14:03.649 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-09-03T05:14:03.650 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-09-03T05:14:03.650 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend-2.6-1.el8.noarch 2022-09-03T05:14:03.651 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-09-03T05:14:03.651 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser-2.14-14.el8.noarch 2022-09-03T05:14:03.652 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz-2017.2-9.el8.noarch 2022-09-03T05:14:03.652 INFO:teuthology.orchestra.run.smithi189.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-09-03T05:14:03.652 INFO:teuthology.orchestra.run.smithi189.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-09-03T05:14:03.653 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-09-03T05:14:03.653 INFO:teuthology.orchestra.run.smithi189.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-09-03T05:14:03.654 INFO:teuthology.orchestra.run.smithi189.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-09-03T05:14:03.654 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-09-03T05:14:03.654 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-09-03T05:14:03.655 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-09-03T05:14:03.655 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-09-03T05:14:03.656 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:03.656 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:14:03.879 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-base 2022-09-03T05:14:04.459 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:14 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:14:05.144 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephadm-2:17.0.0-14661.g4d487742.el8.noarch 1/1 2022-09-03T05:14:05.316 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephadm-2:17.0.0-14661.g4d487742.el8.noarch 1/1 2022-09-03T05:14:05.385 INFO:teuthology.orchestra.run.smithi189.stdout:Package ceph-base-2:17.0.0-14661.g4d487742.el8.x86_64 is already installed. 2022-09-03T05:14:05.459 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:14:05.461 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2022-09-03T05:14:05.463 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:14:05.507 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephadm-2:17.0.0-14661.g4d487742.el8.noarch 1/1 2022-09-03T05:14:05.855 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install cephadm 2022-09-03T05:14:05.919 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephadm-2:17.0.0-14661.g4d487742.el8.noarch 1/1 2022-09-03T05:14:05.919 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:05.920 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2022-09-03T05:14:05.920 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm-2:17.0.0-14661.g4d487742.el8.noarch 2022-09-03T05:14:05.920 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:05.920 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:14:06.154 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-immutable-object-cache 2022-09-03T05:14:06.432 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:16 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:14:06.760 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:17 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:14:07.454 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:14:07.455 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:14:07.455 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2022-09-03T05:14:07.455 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:14:07.455 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2022-09-03T05:14:07.456 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm noarch 2:17.0.0-14661.g4d487742.el8 ceph-noarch 81 k 2022-09-03T05:14:07.456 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:07.456 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2022-09-03T05:14:07.456 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:14:07.457 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2022-09-03T05:14:07.457 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:07.465 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 81 k 2022-09-03T05:14:07.466 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 367 k 2022-09-03T05:14:07.473 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2022-09-03T05:14:07.788 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:14:07.788 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:07.789 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repo Size 2022-09-03T05:14:07.789 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:07.789 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2022-09-03T05:14:07.789 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache x86_64 2:17.0.0-14661.g4d487742.el8 ceph 165 k 2022-09-03T05:14:07.790 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:07.791 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2022-09-03T05:14:07.791 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:07.791 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2022-09-03T05:14:07.792 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:07.792 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 165 k 2022-09-03T05:14:07.792 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 503 k 2022-09-03T05:14:07.792 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2022-09-03T05:14:07.884 INFO:teuthology.orchestra.run.smithi189.stdout:cephadm-17.0.0-14661.g4d487742.el8.noarch.rpm 189 kB/s | 81 kB 00:00 2022-09-03T05:14:07.885 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:14:07.886 INFO:teuthology.orchestra.run.smithi189.stdout:Total 188 kB/s | 81 kB 00:00 2022-09-03T05:14:07.886 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2022-09-03T05:14:07.893 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2022-09-03T05:14:07.893 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2022-09-03T05:14:07.895 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2022-09-03T05:14:07.899 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2022-09-03T05:14:07.947 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2022-09-03T05:14:08.195 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-immutable-object-cache-17.0.0-14661.g4d487 407 kB/s | 165 kB 00:00 2022-09-03T05:14:08.196 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:14:08.196 INFO:teuthology.orchestra.run.smithi181.stdout:Total 405 kB/s | 165 kB 00:00 2022-09-03T05:14:08.196 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2022-09-03T05:14:08.207 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2022-09-03T05:14:08.207 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2022-09-03T05:14:08.254 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2022-09-03T05:14:08.257 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2022-09-03T05:14:08.381 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2022-09-03T05:14:08.542 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-immutable-object-cache-2:17.0.0-14661.g4d487742 1/1 2022-09-03T05:14:09.354 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.0.0-14661.g4d487742 1/1 2022-09-03T05:14:09.542 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephadm-2:17.0.0-14661.g4d487742.el8.noarch 1/1 2022-09-03T05:14:09.761 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-immutable-object-cache-2:17.0.0-14661.g4d487742 1/1 2022-09-03T05:14:09.762 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:09.762 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2022-09-03T05:14:09.765 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:14:09.765 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:09.766 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:14:09.785 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephadm-2:17.0.0-14661.g4d487742.el8.noarch 1/1 2022-09-03T05:14:09.937 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-mgr 2022-09-03T05:14:09.960 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephadm-2:17.0.0-14661.g4d487742.el8.noarch 1/1 2022-09-03T05:14:10.358 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephadm-2:17.0.0-14661.g4d487742.el8.noarch 1/1 2022-09-03T05:14:10.359 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:10.360 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2022-09-03T05:14:10.369 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm-2:17.0.0-14661.g4d487742.el8.noarch 2022-09-03T05:14:10.371 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:10.371 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:14:10.559 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:21 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:14:10.626 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-immutable-object-cache 2022-09-03T05:14:11.254 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:21 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:14:11.489 INFO:teuthology.orchestra.run.smithi181.stdout:Package ceph-mgr-2:17.0.0-14661.g4d487742.el8.x86_64 is already installed. 2022-09-03T05:14:11.565 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:14:11.566 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2022-09-03T05:14:11.566 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:14:11.659 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-mgr-dashboard 2022-09-03T05:14:12.273 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:23 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:14:12.307 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:14:12.307 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:14:12.307 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repo Size 2022-09-03T05:14:12.308 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:14:12.308 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2022-09-03T05:14:12.308 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache x86_64 2:17.0.0-14661.g4d487742.el8 ceph 165 k 2022-09-03T05:14:12.308 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:12.309 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2022-09-03T05:14:12.309 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:14:12.309 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2022-09-03T05:14:12.310 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:12.311 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 165 k 2022-09-03T05:14:12.313 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 503 k 2022-09-03T05:14:12.333 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2022-09-03T05:14:12.527 INFO:teuthology.orchestra.run.smithi189.stdout:ceph-immutable-object-cache-17.0.0-14661.g4d487 713 kB/s | 165 kB 00:00 2022-09-03T05:14:12.528 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:14:12.528 INFO:teuthology.orchestra.run.smithi189.stdout:Total 710 kB/s | 165 kB 00:00 2022-09-03T05:14:12.529 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2022-09-03T05:14:12.538 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2022-09-03T05:14:12.539 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2022-09-03T05:14:12.582 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2022-09-03T05:14:12.585 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2022-09-03T05:14:12.699 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2022-09-03T05:14:12.914 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-immutable-object-cache-2:17.0.0-14661.g4d487742 1/1 2022-09-03T05:14:13.248 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:14:13.269 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:13.291 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2022-09-03T05:14:13.291 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:13.292 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2022-09-03T05:14:13.292 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard noarch 2:17.0.0-14661.g4d487742.el8 ceph-noarch 1.6 M 2022-09-03T05:14:13.293 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2022-09-03T05:14:13.293 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards 2022-09-03T05:14:13.311 INFO:teuthology.orchestra.run.smithi181.stdout: noarch 2:17.0.0-14661.g4d487742.el8 ceph-noarch 23 k 2022-09-03T05:14:13.316 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts noarch 2:17.0.0-14661.g4d487742.el8 ceph-noarch 14 k 2022-09-03T05:14:13.317 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2022-09-03T05:14:13.317 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2022-09-03T05:14:13.317 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2022-09-03T05:14:13.320 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:13.321 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2022-09-03T05:14:13.321 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:13.321 INFO:teuthology.orchestra.run.smithi181.stdout:Install 6 Packages 2022-09-03T05:14:13.322 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:13.322 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 1.9 M 2022-09-03T05:14:13.323 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 7.8 M 2022-09-03T05:14:13.323 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2022-09-03T05:14:13.324 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.0.0-14661.g4d487742 1/1 2022-09-03T05:14:13.406 INFO:teuthology.orchestra.run.smithi181.stdout:(1/6): ceph-grafana-dashboards-17.0.0-14661.g4d 144 kB/s | 23 kB 00:00 2022-09-03T05:14:13.430 INFO:teuthology.orchestra.run.smithi181.stdout:(2/6): ceph-prometheus-alerts-17.0.0-14661.g4d4 82 kB/s | 14 kB 00:00 2022-09-03T05:14:13.486 INFO:teuthology.orchestra.run.smithi181.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 637 kB/s | 43 kB 00:00 2022-09-03T05:14:13.575 INFO:teuthology.orchestra.run.smithi181.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 223 kB/s | 34 kB 00:00 2022-09-03T05:14:13.685 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-immutable-object-cache-2:17.0.0-14661.g4d487742 1/1 2022-09-03T05:14:13.685 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:13.686 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2022-09-03T05:14:13.687 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:14:13.687 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:13.688 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:14:13.706 INFO:teuthology.orchestra.run.smithi181.stdout:(5/6): ceph-mgr-dashboard-17.0.0-14661.g4d48774 3.6 MB/s | 1.6 MB 00:00 2022-09-03T05:14:13.724 INFO:teuthology.orchestra.run.smithi181.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 785 kB/s | 196 kB 00:00 2022-09-03T05:14:13.725 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:14:13.726 INFO:teuthology.orchestra.run.smithi181.stdout:Total 4.1 MB/s | 1.9 MB 00:00 2022-09-03T05:14:13.736 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2022-09-03T05:14:13.746 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2022-09-03T05:14:13.746 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2022-09-03T05:14:13.820 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2022-09-03T05:14:13.824 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2022-09-03T05:14:13.880 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-mgr 2022-09-03T05:14:14.109 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2022-09-03T05:14:14.461 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:24 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:14:15.332 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2022-09-03T05:14:15.406 INFO:teuthology.orchestra.run.smithi189.stdout:Package ceph-mgr-2:17.0.0-14661.g4d487742.el8.x86_64 is already installed. 2022-09-03T05:14:15.473 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:14:15.475 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2022-09-03T05:14:15.476 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:14:15.559 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-mgr-dashboard 2022-09-03T05:14:16.093 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2022-09-03T05:14:16.136 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:26 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:14:16.327 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2022-09-03T05:14:16.475 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-prometheus-alerts-2:17.0.0-14661.g4d487742.el8. 4/6 2022-09-03T05:14:16.806 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-grafana-dashboards-2:17.0.0-14661.g4d487742.el8 5/6 2022-09-03T05:14:17.003 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-dashboard-2:17.0.0-14661.g4d487742.el8.noar 6/6 2022-09-03T05:14:17.155 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:14:17.156 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:14:17.162 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2022-09-03T05:14:17.162 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:14:17.163 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2022-09-03T05:14:17.163 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard noarch 2:17.0.0-14661.g4d487742.el8 ceph-noarch 1.6 M 2022-09-03T05:14:17.163 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2022-09-03T05:14:17.164 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards 2022-09-03T05:14:17.164 INFO:teuthology.orchestra.run.smithi189.stdout: noarch 2:17.0.0-14661.g4d487742.el8 ceph-noarch 23 k 2022-09-03T05:14:17.164 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts noarch 2:17.0.0-14661.g4d487742.el8 ceph-noarch 14 k 2022-09-03T05:14:17.165 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2022-09-03T05:14:17.165 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2022-09-03T05:14:17.165 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2022-09-03T05:14:17.182 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:17.183 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2022-09-03T05:14:17.183 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:14:17.183 INFO:teuthology.orchestra.run.smithi189.stdout:Install 6 Packages 2022-09-03T05:14:17.184 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:17.184 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 1.9 M 2022-09-03T05:14:17.185 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 7.8 M 2022-09-03T05:14:17.185 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2022-09-03T05:14:17.254 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.0.0-14661.g4d487742.el8.noar 6/6 2022-09-03T05:14:17.254 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-grafana-dashboards-2:17.0.0-14661.g4d487742.el8 1/6 2022-09-03T05:14:17.255 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-dashboard-2:17.0.0-14661.g4d487742.el8.noar 2/6 2022-09-03T05:14:17.255 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-prometheus-alerts-2:17.0.0-14661.g4d487742.el8. 3/6 2022-09-03T05:14:17.256 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2022-09-03T05:14:17.256 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2022-09-03T05:14:17.648 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2022-09-03T05:14:17.648 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:17.649 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2022-09-03T05:14:17.649 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards-2:17.0.0-14661.g4d487742.el8.noarch 2022-09-03T05:14:17.649 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard-2:17.0.0-14661.g4d487742.el8.noarch 2022-09-03T05:14:17.649 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts-2:17.0.0-14661.g4d487742.el8.noarch 2022-09-03T05:14:17.650 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt-1.6.1-2.el8.noarch 2022-09-03T05:14:17.650 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru-0.7-6.el8.noarch 2022-09-03T05:14:17.650 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes-2.4.1-12.el8.noarch 2022-09-03T05:14:17.651 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:17.651 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:14:17.796 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-mgr-diskprediction-local 2022-09-03T05:14:17.803 INFO:teuthology.orchestra.run.smithi189.stdout:(1/6): ceph-prometheus-alerts-17.0.0-14661.g4d4 23 kB/s | 14 kB 00:00 2022-09-03T05:14:18.246 INFO:teuthology.orchestra.run.smithi189.stdout:(2/6): ceph-grafana-dashboards-17.0.0-14661.g4d 21 kB/s | 23 kB 00:01 2022-09-03T05:14:18.376 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:29 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:14:18.512 INFO:teuthology.orchestra.run.smithi189.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 59 kB/s | 43 kB 00:00 2022-09-03T05:14:18.611 INFO:teuthology.orchestra.run.smithi189.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 93 kB/s | 34 kB 00:00 2022-09-03T05:14:18.798 INFO:teuthology.orchestra.run.smithi189.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 693 kB/s | 196 kB 00:00 2022-09-03T05:14:18.859 INFO:teuthology.orchestra.run.smithi189.stdout:(6/6): ceph-mgr-dashboard-17.0.0-14661.g4d48774 992 kB/s | 1.6 MB 00:01 2022-09-03T05:14:18.859 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:14:18.860 INFO:teuthology.orchestra.run.smithi189.stdout:Total 1.1 MB/s | 1.9 MB 00:01 2022-09-03T05:14:18.880 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2022-09-03T05:14:18.881 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2022-09-03T05:14:18.881 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2022-09-03T05:14:18.956 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2022-09-03T05:14:18.957 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2022-09-03T05:14:19.240 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2022-09-03T05:14:19.380 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:14:19.423 INFO:teuthology.orchestra.run.smithi181.stdout:===================================================================================================== 2022-09-03T05:14:19.427 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2022-09-03T05:14:19.428 INFO:teuthology.orchestra.run.smithi181.stdout:===================================================================================================== 2022-09-03T05:14:19.429 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2022-09-03T05:14:19.430 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local noarch 2:17.0.0-14661.g4d487742.el8 ceph-noarch 7.4 M 2022-09-03T05:14:19.430 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2022-09-03T05:14:19.435 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran x86_64 8.5.0-15.el8 CentOS-BaseOS 644 k 2022-09-03T05:14:19.439 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath x86_64 8.5.0-15.el8 CentOS-BaseOS 172 k 2022-09-03T05:14:19.439 INFO:teuthology.orchestra.run.smithi181.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2022-09-03T05:14:19.476 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2022-09-03T05:14:19.477 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2022-09-03T05:14:19.477 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2022-09-03T05:14:19.477 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2022-09-03T05:14:19.478 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:19.489 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2022-09-03T05:14:19.492 INFO:teuthology.orchestra.run.smithi181.stdout:===================================================================================================== 2022-09-03T05:14:19.493 INFO:teuthology.orchestra.run.smithi181.stdout:Install 8 Packages 2022-09-03T05:14:19.496 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:19.499 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 35 M 2022-09-03T05:14:19.499 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 210 M 2022-09-03T05:14:19.501 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2022-09-03T05:14:19.502 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2022-09-03T05:14:19.626 INFO:teuthology.orchestra.run.smithi181.stdout:(1/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 21 MB/s | 4.9 MB 00:00 2022-09-03T05:14:19.692 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2022-09-03T05:14:19.742 INFO:teuthology.orchestra.run.smithi181.stdout:(2/8): openblas-0.3.15-4.el8.x86_64.rpm 13 MB/s | 4.7 MB 00:00 2022-09-03T05:14:19.774 INFO:teuthology.orchestra.run.smithi181.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 6.6 MB/s | 225 kB 00:00 2022-09-03T05:14:19.863 INFO:teuthology.orchestra.run.smithi181.stdout:(4/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 15 MB/s | 3.7 MB 00:00 2022-09-03T05:14:19.905 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2022-09-03T05:14:19.922 INFO:teuthology.orchestra.run.smithi181.stdout:(5/8): libgfortran-8.5.0-15.el8.x86_64.rpm 13 MB/s | 644 kB 00:00 2022-09-03T05:14:19.942 INFO:teuthology.orchestra.run.smithi181.stdout:(6/8): libquadmath-8.5.0-15.el8.x86_64.rpm 5.1 MB/s | 172 kB 00:00 2022-09-03T05:14:20.088 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-prometheus-alerts-2:17.0.0-14661.g4d487742.el8. 4/6 2022-09-03T05:14:20.215 INFO:teuthology.orchestra.run.smithi181.stdout:(7/8): python3-scipy-1.0.0-21.module_el8.5.0+77 32 MB/s | 14 MB 00:00 2022-09-03T05:14:20.432 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-grafana-dashboards-2:17.0.0-14661.g4d487742.el8 5/6 2022-09-03T05:14:20.592 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-dashboard-2:17.0.0-14661.g4d487742.el8.noar 6/6 2022-09-03T05:14:20.857 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.0.0-14661.g4d487742.el8.noar 6/6 2022-09-03T05:14:20.857 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-grafana-dashboards-2:17.0.0-14661.g4d487742.el8 1/6 2022-09-03T05:14:20.858 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-dashboard-2:17.0.0-14661.g4d487742.el8.noar 2/6 2022-09-03T05:14:20.858 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-prometheus-alerts-2:17.0.0-14661.g4d487742.el8. 3/6 2022-09-03T05:14:20.859 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2022-09-03T05:14:20.859 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2022-09-03T05:14:21.132 INFO:teuthology.orchestra.run.smithi181.stdout:(8/8): ceph-mgr-diskprediction-local-17.0.0-146 4.2 MB/s | 7.4 MB 00:01 2022-09-03T05:14:21.135 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:14:21.136 INFO:teuthology.orchestra.run.smithi181.stdout:Total 20 MB/s | 35 MB 00:01 2022-09-03T05:14:21.266 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2022-09-03T05:14:21.267 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:21.267 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2022-09-03T05:14:21.268 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards-2:17.0.0-14661.g4d487742.el8.noarch 2022-09-03T05:14:21.268 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard-2:17.0.0-14661.g4d487742.el8.noarch 2022-09-03T05:14:21.269 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts-2:17.0.0-14661.g4d487742.el8.noarch 2022-09-03T05:14:21.269 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jwt-1.6.1-2.el8.noarch 2022-09-03T05:14:21.270 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru-0.7-6.el8.noarch 2022-09-03T05:14:21.270 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes-2.4.1-12.el8.noarch 2022-09-03T05:14:21.270 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:21.271 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:14:21.374 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2022-09-03T05:14:21.382 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2022-09-03T05:14:21.383 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2022-09-03T05:14:21.413 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-mgr-diskprediction-local 2022-09-03T05:14:21.759 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2022-09-03T05:14:21.772 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2022-09-03T05:14:21.991 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:31 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:14:22.318 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2022-09-03T05:14:22.469 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libquadmath-8.5.0-15.el8.x86_64 1/8 2022-09-03T05:14:22.604 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libquadmath-8.5.0-15.el8.x86_64 1/8 2022-09-03T05:14:22.763 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libgfortran-8.5.0-15.el8.x86_64 2/8 2022-09-03T05:14:22.995 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:14:22.995 INFO:teuthology.orchestra.run.smithi189.stdout:===================================================================================================== 2022-09-03T05:14:22.995 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2022-09-03T05:14:22.996 INFO:teuthology.orchestra.run.smithi189.stdout:===================================================================================================== 2022-09-03T05:14:22.996 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2022-09-03T05:14:22.996 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local noarch 2:17.0.0-14661.g4d487742.el8 ceph-noarch 7.4 M 2022-09-03T05:14:22.996 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2022-09-03T05:14:22.997 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran x86_64 8.5.0-15.el8 CentOS-BaseOS 644 k 2022-09-03T05:14:22.997 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath x86_64 8.5.0-15.el8 CentOS-BaseOS 172 k 2022-09-03T05:14:22.997 INFO:teuthology.orchestra.run.smithi189.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2022-09-03T05:14:22.997 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2022-09-03T05:14:23.010 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2022-09-03T05:14:23.011 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2022-09-03T05:14:23.017 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2022-09-03T05:14:23.049 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:23.050 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2022-09-03T05:14:23.050 INFO:teuthology.orchestra.run.smithi189.stdout:===================================================================================================== 2022-09-03T05:14:23.050 INFO:teuthology.orchestra.run.smithi189.stdout:Install 8 Packages 2022-09-03T05:14:23.050 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:23.051 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 35 M 2022-09-03T05:14:23.051 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 210 M 2022-09-03T05:14:23.051 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2022-09-03T05:14:23.186 INFO:teuthology.orchestra.run.smithi189.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 27 MB/s | 4.7 MB 00:00 2022-09-03T05:14:23.302 INFO:teuthology.orchestra.run.smithi189.stdout:(2/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 30 MB/s | 3.7 MB 00:00 2022-09-03T05:14:23.337 INFO:teuthology.orchestra.run.smithi189.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 6.6 MB/s | 225 kB 00:00 2022-09-03T05:14:23.359 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libgfortran-8.5.0-15.el8.x86_64 2/8 2022-09-03T05:14:23.484 INFO:teuthology.orchestra.run.smithi189.stdout:(4/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 10 MB/s | 4.9 MB 00:00 2022-09-03T05:14:23.533 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2022-09-03T05:14:23.534 INFO:teuthology.orchestra.run.smithi189.stdout:(5/8): libgfortran-8.5.0-15.el8.x86_64.rpm 13 MB/s | 644 kB 00:00 2022-09-03T05:14:23.618 INFO:teuthology.orchestra.run.smithi189.stdout:(6/8): ceph-mgr-diskprediction-local-17.0.0-146 12 MB/s | 7.4 MB 00:00 2022-09-03T05:14:23.646 INFO:teuthology.orchestra.run.smithi189.stdout:(7/8): libquadmath-8.5.0-15.el8.x86_64.rpm 1.5 MB/s | 172 kB 00:00 2022-09-03T05:14:23.905 INFO:teuthology.orchestra.run.smithi189.stdout:(8/8): python3-scipy-1.0.0-21.module_el8.5.0+77 24 MB/s | 14 MB 00:00 2022-09-03T05:14:23.924 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:14:23.925 INFO:teuthology.orchestra.run.smithi189.stdout:Total 39 MB/s | 35 MB 00:00 2022-09-03T05:14:24.134 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2022-09-03T05:14:24.142 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2022-09-03T05:14:24.146 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2022-09-03T05:14:24.146 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2022-09-03T05:14:24.322 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2022-09-03T05:14:24.526 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2022-09-03T05:14:24.537 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2022-09-03T05:14:24.988 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2022-09-03T05:14:25.078 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2022-09-03T05:14:25.258 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libquadmath-8.5.0-15.el8.x86_64 1/8 2022-09-03T05:14:25.286 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2022-09-03T05:14:25.402 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libquadmath-8.5.0-15.el8.x86_64 1/8 2022-09-03T05:14:25.507 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2022-09-03T05:14:25.551 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libgfortran-8.5.0-15.el8.x86_64 2/8 2022-09-03T05:14:26.169 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libgfortran-8.5.0-15.el8.x86_64 2/8 2022-09-03T05:14:26.392 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2022-09-03T05:14:27.011 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2022-09-03T05:14:27.199 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2022-09-03T05:14:27.262 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2022-09-03T05:14:27.919 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2022-09-03T05:14:28.270 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2022-09-03T05:14:28.503 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2022-09-03T05:14:28.798 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2022-09-03T05:14:28.942 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-diskprediction-local-2:17.0.0-14661.g4d4877 8/8 2022-09-03T05:14:29.238 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.0.0-14661.g4d4877 8/8 2022-09-03T05:14:29.239 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.0.0-14661.g4d4877 1/8 2022-09-03T05:14:29.239 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2022-09-03T05:14:29.240 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2022-09-03T05:14:29.240 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2022-09-03T05:14:29.276 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2022-09-03T05:14:29.277 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2022-09-03T05:14:29.277 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libgfortran-8.5.0-15.el8.x86_64 7/8 2022-09-03T05:14:29.650 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libquadmath-8.5.0-15.el8.x86_64 8/8 2022-09-03T05:14:29.651 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:29.651 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2022-09-03T05:14:29.652 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local-2:17.0.0-14661.g4d487742.el8.noarch 2022-09-03T05:14:29.652 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran-8.5.0-15.el8.x86_64 2022-09-03T05:14:29.652 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath-8.5.0-15.el8.x86_64 2022-09-03T05:14:29.653 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-0.3.15-4.el8.x86_64 2022-09-03T05:14:29.654 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-threads-0.3.15-4.el8.x86_64 2022-09-03T05:14:29.654 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2022-09-03T05:14:29.654 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2022-09-03T05:14:29.655 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2022-09-03T05:14:29.655 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:29.655 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:14:29.784 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-mgr-rook 2022-09-03T05:14:30.253 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2022-09-03T05:14:30.360 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:41 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:14:31.346 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:14:31.348 INFO:teuthology.orchestra.run.smithi181.stdout:========================================================================================= 2022-09-03T05:14:31.348 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2022-09-03T05:14:31.350 INFO:teuthology.orchestra.run.smithi181.stdout:========================================================================================= 2022-09-03T05:14:31.350 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2022-09-03T05:14:31.351 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook noarch 2:17.0.0-14661.g4d487742.el8 ceph-noarch 47 k 2022-09-03T05:14:31.351 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2022-09-03T05:14:31.352 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2022-09-03T05:14:31.352 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2022-09-03T05:14:31.353 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2022-09-03T05:14:31.355 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2022-09-03T05:14:31.355 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2022-09-03T05:14:31.356 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2022-09-03T05:14:31.356 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2022-09-03T05:14:31.356 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2022-09-03T05:14:31.357 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2022-09-03T05:14:31.359 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2022-09-03T05:14:31.359 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2022-09-03T05:14:31.360 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2022-09-03T05:14:31.360 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:31.360 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2022-09-03T05:14:31.361 INFO:teuthology.orchestra.run.smithi181.stdout:========================================================================================= 2022-09-03T05:14:31.361 INFO:teuthology.orchestra.run.smithi181.stdout:Install 13 Packages 2022-09-03T05:14:31.362 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:31.365 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 1.9 M 2022-09-03T05:14:31.365 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 25 M 2022-09-03T05:14:31.366 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2022-09-03T05:14:31.416 INFO:teuthology.orchestra.run.smithi181.stdout:(1/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 432 kB/s | 27 kB 00:00 2022-09-03T05:14:31.430 INFO:teuthology.orchestra.run.smithi181.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 255 kB/s | 20 kB 00:00 2022-09-03T05:14:31.457 INFO:teuthology.orchestra.run.smithi181.stdout:(3/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 3.0 MB/s | 126 kB 00:00 2022-09-03T05:14:31.506 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2022-09-03T05:14:31.507 INFO:teuthology.orchestra.run.smithi181.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 2.1 MB/s | 110 kB 00:00 2022-09-03T05:14:31.507 INFO:teuthology.orchestra.run.smithi181.stdout:(5/13): python3-requests-oauthlib-1.0.0-1.el8.n 1.7 MB/s | 43 kB 00:00 2022-09-03T05:14:31.535 INFO:teuthology.orchestra.run.smithi181.stdout:(6/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 2.0 MB/s | 155 kB 00:00 2022-09-03T05:14:31.558 INFO:teuthology.orchestra.run.smithi181.stdout:(7/13): ceph-mgr-rook-17.0.0-14661.g4d487742.el 229 kB/s | 47 kB 00:00 2022-09-03T05:14:31.651 INFO:teuthology.orchestra.run.smithi181.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 247 kB/s | 33 kB 00:00 2022-09-03T05:14:31.675 INFO:teuthology.orchestra.run.smithi181.stdout:(9/13): python3-certifi-2018.10.15-7.el8.noarch 122 kB/s | 16 kB 00:00 2022-09-03T05:14:31.690 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-diskprediction-local-2:17.0.0-14661.g4d4877 8/8 2022-09-03T05:14:31.691 INFO:teuthology.orchestra.run.smithi181.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 607 kB/s | 81 kB 00:00 2022-09-03T05:14:31.746 INFO:teuthology.orchestra.run.smithi181.stdout:(11/13): python3-websocket-client-0.56.0-5.el8. 1.2 MB/s | 61 kB 00:00 2022-09-03T05:14:31.779 INFO:teuthology.orchestra.run.smithi181.stdout:(12/13): python3-rsa-4.8-1.el8.noarch.rpm 613 kB/s | 61 kB 00:00 2022-09-03T05:14:31.981 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.0.0-14661.g4d4877 8/8 2022-09-03T05:14:31.981 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.0.0-14661.g4d4877 1/8 2022-09-03T05:14:31.982 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2022-09-03T05:14:31.982 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2022-09-03T05:14:31.983 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2022-09-03T05:14:31.983 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2022-09-03T05:14:31.984 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2022-09-03T05:14:31.984 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libgfortran-8.5.0-15.el8.x86_64 7/8 2022-09-03T05:14:32.000 INFO:teuthology.orchestra.run.smithi181.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 3.2 MB/s | 1.2 MB 00:00 2022-09-03T05:14:32.009 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:14:32.009 INFO:teuthology.orchestra.run.smithi181.stdout:Total 3.0 MB/s | 1.9 MB 00:00 2022-09-03T05:14:32.052 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2022-09-03T05:14:32.068 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2022-09-03T05:14:32.068 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2022-09-03T05:14:32.154 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2022-09-03T05:14:32.158 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2022-09-03T05:14:32.366 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libquadmath-8.5.0-15.el8.x86_64 8/8 2022-09-03T05:14:32.366 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:32.367 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2022-09-03T05:14:32.367 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local-2:17.0.0-14661.g4d487742.el8.noarch 2022-09-03T05:14:32.368 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran-8.5.0-15.el8.x86_64 2022-09-03T05:14:32.368 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath-8.5.0-15.el8.x86_64 2022-09-03T05:14:32.369 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-0.3.15-4.el8.x86_64 2022-09-03T05:14:32.370 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-threads-0.3.15-4.el8.x86_64 2022-09-03T05:14:32.370 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2022-09-03T05:14:32.370 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2022-09-03T05:14:32.371 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2022-09-03T05:14:32.371 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:32.372 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:14:32.474 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-mgr-rook 2022-09-03T05:14:32.627 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2022-09-03T05:14:32.959 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2022-09-03T05:14:33.053 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:43 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:14:33.150 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2022-09-03T05:14:33.373 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rsa-4.8-1.el8.noarch 3/13 2022-09-03T05:14:33.554 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2022-09-03T05:14:33.723 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2022-09-03T05:14:33.891 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2022-09-03T05:14:34.084 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:14:34.085 INFO:teuthology.orchestra.run.smithi189.stdout:========================================================================================= 2022-09-03T05:14:34.085 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2022-09-03T05:14:34.086 INFO:teuthology.orchestra.run.smithi189.stdout:========================================================================================= 2022-09-03T05:14:34.092 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2022-09-03T05:14:34.092 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook noarch 2:17.0.0-14661.g4d487742.el8 ceph-noarch 47 k 2022-09-03T05:14:34.094 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2022-09-03T05:14:34.095 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2022-09-03T05:14:34.109 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2022-09-03T05:14:34.110 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2022-09-03T05:14:34.114 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2022-09-03T05:14:34.115 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2022-09-03T05:14:34.115 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2022-09-03T05:14:34.115 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2022-09-03T05:14:34.116 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2022-09-03T05:14:34.116 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2022-09-03T05:14:34.117 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2022-09-03T05:14:34.117 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2022-09-03T05:14:34.128 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2022-09-03T05:14:34.128 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:34.129 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2022-09-03T05:14:34.129 INFO:teuthology.orchestra.run.smithi189.stdout:========================================================================================= 2022-09-03T05:14:34.130 INFO:teuthology.orchestra.run.smithi189.stdout:Install 13 Packages 2022-09-03T05:14:34.131 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:34.131 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 1.9 M 2022-09-03T05:14:34.132 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 25 M 2022-09-03T05:14:34.132 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2022-09-03T05:14:34.143 INFO:teuthology.orchestra.run.smithi189.stdout:(1/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 427 kB/s | 27 kB 00:00 2022-09-03T05:14:34.179 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2022-09-03T05:14:34.197 INFO:teuthology.orchestra.run.smithi189.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 177 kB/s | 20 kB 00:00 2022-09-03T05:14:34.246 INFO:teuthology.orchestra.run.smithi189.stdout:(3/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.2 MB/s | 126 kB 00:00 2022-09-03T05:14:34.277 INFO:teuthology.orchestra.run.smithi189.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 1.3 MB/s | 110 kB 00:00 2022-09-03T05:14:34.303 INFO:teuthology.orchestra.run.smithi189.stdout:(5/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 2.7 MB/s | 155 kB 00:00 2022-09-03T05:14:34.328 INFO:teuthology.orchestra.run.smithi189.stdout:(6/13): python3-requests-oauthlib-1.0.0-1.el8.n 850 kB/s | 43 kB 00:00 2022-09-03T05:14:34.365 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2022-09-03T05:14:34.366 INFO:teuthology.orchestra.run.smithi189.stdout:(7/13): ceph-mgr-rook-17.0.0-14661.g4d487742.el 172 kB/s | 47 kB 00:00 2022-09-03T05:14:34.484 INFO:teuthology.orchestra.run.smithi189.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 105 kB/s | 16 kB 00:00 2022-09-03T05:14:34.518 INFO:teuthology.orchestra.run.smithi189.stdout:(9/13): python3-cachetools-3.1.1-4.el8.noarch.r 158 kB/s | 33 kB 00:00 2022-09-03T05:14:34.554 INFO:teuthology.orchestra.run.smithi189.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 442 kB/s | 81 kB 00:00 2022-09-03T05:14:34.693 INFO:teuthology.orchestra.run.smithi189.stdout:(11/13): python3-websocket-client-0.56.0-5.el8. 401 kB/s | 61 kB 00:00 2022-09-03T05:14:34.963 INFO:teuthology.orchestra.run.smithi189.stdout:(12/13): python3-rsa-4.8-1.el8.noarch.rpm 136 kB/s | 61 kB 00:00 2022-09-03T05:14:35.375 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2022-09-03T05:14:35.497 INFO:teuthology.orchestra.run.smithi189.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 1.2 MB/s | 1.2 MB 00:01 2022-09-03T05:14:35.499 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:14:35.500 INFO:teuthology.orchestra.run.smithi189.stdout:Total 1.4 MB/s | 1.9 MB 00:01 2022-09-03T05:14:35.557 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2022-09-03T05:14:35.575 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2022-09-03T05:14:35.577 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2022-09-03T05:14:35.606 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2022-09-03T05:14:35.667 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2022-09-03T05:14:35.668 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2022-09-03T05:14:35.774 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2022-09-03T05:14:35.972 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2022-09-03T05:14:36.114 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-rook-2:17.0.0-14661.g4d487742.el8.noarch 13/13 2022-09-03T05:14:36.195 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2022-09-03T05:14:36.381 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-rook-2:17.0.0-14661.g4d487742.el8.noarch 13/13 2022-09-03T05:14:36.381 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-rook-2:17.0.0-14661.g4d487742.el8.noarch 1/13 2022-09-03T05:14:36.393 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2022-09-03T05:14:36.395 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2022-09-03T05:14:36.396 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2022-09-03T05:14:36.396 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2022-09-03T05:14:36.397 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2022-09-03T05:14:36.397 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2022-09-03T05:14:36.397 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2022-09-03T05:14:36.399 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2022-09-03T05:14:36.399 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2022-09-03T05:14:36.400 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2022-09-03T05:14:36.400 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 12/13 2022-09-03T05:14:36.474 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2022-09-03T05:14:36.706 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2022-09-03T05:14:36.855 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2022-09-03T05:14:36.855 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:36.856 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2022-09-03T05:14:36.856 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook-2:17.0.0-14661.g4d487742.el8.noarch 2022-09-03T05:14:36.857 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools-3.1.1-4.el8.noarch 2022-09-03T05:14:36.857 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi-2018.10.15-7.el8.noarch 2022-09-03T05:14:36.860 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2022-09-03T05:14:36.861 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch-1.21-2.el8.noarch 2022-09-03T05:14:36.861 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer-1.10-11.el8.noarch 2022-09-03T05:14:36.861 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2022-09-03T05:14:36.862 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2022-09-03T05:14:36.862 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2022-09-03T05:14:36.862 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2022-09-03T05:14:36.862 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2022-09-03T05:14:36.863 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa-4.8-1.el8.noarch 2022-09-03T05:14:36.863 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2022-09-03T05:14:36.863 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:36.863 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:14:36.958 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rsa-4.8-1.el8.noarch 3/13 2022-09-03T05:14:36.972 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-mgr-cephadm 2022-09-03T05:14:37.235 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2022-09-03T05:14:37.411 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2022-09-03T05:14:37.546 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:48 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:14:37.648 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2022-09-03T05:14:37.943 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2022-09-03T05:14:38.161 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2022-09-03T05:14:38.552 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:14:38.552 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:38.553 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2022-09-03T05:14:38.553 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:38.553 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2022-09-03T05:14:38.553 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm noarch 2:17.0.0-14661.g4d487742.el8 ceph-noarch 114 k 2022-09-03T05:14:38.554 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2022-09-03T05:14:38.554 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2022-09-03T05:14:38.554 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2022-09-03T05:14:38.555 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2022-09-03T05:14:38.555 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2022-09-03T05:14:38.555 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:38.555 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2022-09-03T05:14:38.555 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:38.556 INFO:teuthology.orchestra.run.smithi181.stdout:Install 5 Packages 2022-09-03T05:14:38.556 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:38.557 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 5.9 M 2022-09-03T05:14:38.557 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 26 M 2022-09-03T05:14:38.557 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2022-09-03T05:14:38.658 INFO:teuthology.orchestra.run.smithi181.stdout:(1/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 5.1 MB/s | 538 kB 00:00 2022-09-03T05:14:38.770 INFO:teuthology.orchestra.run.smithi181.stdout:(2/5): python3-babel-2.5.1-7.el8.noarch.rpm 23 MB/s | 4.8 MB 00:00 2022-09-03T05:14:38.799 INFO:teuthology.orchestra.run.smithi181.stdout:(3/5): ceph-mgr-cephadm-17.0.0-14661.g4d487742. 484 kB/s | 114 kB 00:00 2022-09-03T05:14:38.908 INFO:teuthology.orchestra.run.smithi181.stdout:(4/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.9 MB/s | 476 kB 00:00 2022-09-03T05:14:38.940 INFO:teuthology.orchestra.run.smithi181.stdout:(5/5): python3-natsort-7.1.1-2.el8.noarch.rpm 361 kB/s | 60 kB 00:00 2022-09-03T05:14:38.941 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:14:38.941 INFO:teuthology.orchestra.run.smithi181.stdout:Total 16 MB/s | 5.9 MB 00:00 2022-09-03T05:14:38.986 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2022-09-03T05:14:38.998 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2022-09-03T05:14:38.999 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2022-09-03T05:14:39.096 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2022-09-03T05:14:39.096 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2022-09-03T05:14:39.202 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2022-09-03T05:14:39.365 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2022-09-03T05:14:39.453 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2022-09-03T05:14:39.650 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2022-09-03T05:14:39.658 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2022-09-03T05:14:39.901 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2022-09-03T05:14:40.089 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-rook-2:17.0.0-14661.g4d487742.el8.noarch 13/13 2022-09-03T05:14:40.348 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-rook-2:17.0.0-14661.g4d487742.el8.noarch 13/13 2022-09-03T05:14:40.349 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-rook-2:17.0.0-14661.g4d487742.el8.noarch 1/13 2022-09-03T05:14:40.349 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2022-09-03T05:14:40.350 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2022-09-03T05:14:40.351 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2022-09-03T05:14:40.351 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2022-09-03T05:14:40.352 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2022-09-03T05:14:40.352 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2022-09-03T05:14:40.353 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2022-09-03T05:14:40.353 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2022-09-03T05:14:40.353 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2022-09-03T05:14:40.354 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2022-09-03T05:14:40.355 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 12/13 2022-09-03T05:14:40.587 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2022-09-03T05:14:40.768 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2022-09-03T05:14:40.769 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:40.769 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2022-09-03T05:14:40.769 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook-2:17.0.0-14661.g4d487742.el8.noarch 2022-09-03T05:14:40.770 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools-3.1.1-4.el8.noarch 2022-09-03T05:14:40.770 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi-2018.10.15-7.el8.noarch 2022-09-03T05:14:40.770 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2022-09-03T05:14:40.770 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch-1.21-2.el8.noarch 2022-09-03T05:14:40.771 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer-1.10-11.el8.noarch 2022-09-03T05:14:40.771 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2022-09-03T05:14:40.771 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2022-09-03T05:14:40.771 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2022-09-03T05:14:40.772 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2022-09-03T05:14:40.772 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2022-09-03T05:14:40.772 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa-4.8-1.el8.noarch 2022-09-03T05:14:40.772 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2022-09-03T05:14:40.773 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:40.773 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:14:40.921 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2022-09-03T05:14:41.193 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2022-09-03T05:14:41.329 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-mgr-cephadm 2022-09-03T05:14:41.388 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-cephadm-2:17.0.0-14661.g4d487742.el8.noarch 5/5 2022-09-03T05:14:41.624 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.0.0-14661.g4d487742.el8.noarch 5/5 2022-09-03T05:14:41.626 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-cephadm-2:17.0.0-14661.g4d487742.el8.noarch 1/5 2022-09-03T05:14:41.628 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2022-09-03T05:14:41.628 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2022-09-03T05:14:41.629 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2022-09-03T05:14:41.913 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:51 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:14:42.020 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2022-09-03T05:14:42.022 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:42.022 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2022-09-03T05:14:42.022 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm-2:17.0.0-14661.g4d487742.el8.noarch 2022-09-03T05:14:42.023 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2022-09-03T05:14:42.023 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel-2.5.1-7.el8.noarch 2022-09-03T05:14:42.023 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2-2.10.1-3.el8.noarch 2022-09-03T05:14:42.023 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort-7.1.1-2.el8.noarch 2022-09-03T05:14:42.024 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:42.024 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:14:42.136 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-fuse 2022-09-03T05:14:42.717 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:53 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:14:42.933 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:14:42.933 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:14:42.934 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2022-09-03T05:14:42.934 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:14:42.935 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2022-09-03T05:14:42.935 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm noarch 2:17.0.0-14661.g4d487742.el8 ceph-noarch 114 k 2022-09-03T05:14:42.936 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2022-09-03T05:14:42.936 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2022-09-03T05:14:42.937 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2022-09-03T05:14:42.937 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2022-09-03T05:14:42.937 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2022-09-03T05:14:42.938 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:42.940 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2022-09-03T05:14:42.940 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:14:42.941 INFO:teuthology.orchestra.run.smithi189.stdout:Install 5 Packages 2022-09-03T05:14:42.941 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:42.942 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 5.9 M 2022-09-03T05:14:42.950 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 26 M 2022-09-03T05:14:42.951 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2022-09-03T05:14:43.038 INFO:teuthology.orchestra.run.smithi189.stdout:(1/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 4.9 MB/s | 538 kB 00:00 2022-09-03T05:14:43.236 INFO:teuthology.orchestra.run.smithi189.stdout:(2/5): python3-babel-2.5.1-7.el8.noarch.rpm 16 MB/s | 4.8 MB 00:00 2022-09-03T05:14:43.262 INFO:teuthology.orchestra.run.smithi189.stdout:(3/5): ceph-mgr-cephadm-17.0.0-14661.g4d487742. 351 kB/s | 114 kB 00:00 2022-09-03T05:14:43.413 INFO:teuthology.orchestra.run.smithi189.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 328 kB/s | 60 kB 00:00 2022-09-03T05:14:43.441 INFO:teuthology.orchestra.run.smithi189.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.2 MB/s | 476 kB 00:00 2022-09-03T05:14:43.442 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:14:43.446 INFO:teuthology.orchestra.run.smithi189.stdout:Total 12 MB/s | 5.9 MB 00:00 2022-09-03T05:14:43.502 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2022-09-03T05:14:43.508 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2022-09-03T05:14:43.508 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2022-09-03T05:14:43.592 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2022-09-03T05:14:43.600 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2022-09-03T05:14:43.750 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:14:43.751 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:43.752 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2022-09-03T05:14:43.752 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:43.752 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2022-09-03T05:14:43.753 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse x86_64 2:17.0.0-14661.g4d487742.el8 ceph 843 k 2022-09-03T05:14:43.753 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2022-09-03T05:14:43.763 INFO:teuthology.orchestra.run.smithi181.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2022-09-03T05:14:43.763 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:43.764 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2022-09-03T05:14:43.764 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:43.765 INFO:teuthology.orchestra.run.smithi181.stdout:Install 2 Packages 2022-09-03T05:14:43.765 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:43.766 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 926 k 2022-09-03T05:14:43.766 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 2.9 M 2022-09-03T05:14:43.767 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2022-09-03T05:14:43.814 INFO:teuthology.orchestra.run.smithi181.stdout:(1/2): fuse-2.9.7-16.el8.x86_64.rpm 1.3 MB/s | 83 kB 00:00 2022-09-03T05:14:43.872 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2022-09-03T05:14:44.159 INFO:teuthology.orchestra.run.smithi181.stdout:(2/2): ceph-fuse-17.0.0-14661.g4d487742.el8.x86 2.0 MB/s | 843 kB 00:00 2022-09-03T05:14:44.159 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:14:44.160 INFO:teuthology.orchestra.run.smithi181.stdout:Total 2.2 MB/s | 926 kB 00:00 2022-09-03T05:14:44.170 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2022-09-03T05:14:44.203 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2022-09-03T05:14:44.203 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2022-09-03T05:14:44.211 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2022-09-03T05:14:44.267 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2022-09-03T05:14:44.269 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2022-09-03T05:14:44.419 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2022-09-03T05:14:44.688 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2022-09-03T05:14:44.818 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-fuse-2:17.0.0-14661.g4d487742.el8.x86_64 2/2 2022-09-03T05:14:45.168 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2022-09-03T05:14:45.170 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-fuse-2:17.0.0-14661.g4d487742.el8.x86_64 2/2 2022-09-03T05:14:45.170 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-fuse-2:17.0.0-14661.g4d487742.el8.x86_64 1/2 2022-09-03T05:14:45.519 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2022-09-03T05:14:45.578 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2022-09-03T05:14:45.579 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:45.579 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2022-09-03T05:14:45.580 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse-2:17.0.0-14661.g4d487742.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2022-09-03T05:14:45.580 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:45.581 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:14:45.775 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2022-09-03T05:14:45.787 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-volume 2022-09-03T05:14:45.966 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-cephadm-2:17.0.0-14661.g4d487742.el8.noarch 5/5 2022-09-03T05:14:46.227 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.0.0-14661.g4d487742.el8.noarch 5/5 2022-09-03T05:14:46.228 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-cephadm-2:17.0.0-14661.g4d487742.el8.noarch 1/5 2022-09-03T05:14:46.228 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2022-09-03T05:14:46.228 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2022-09-03T05:14:46.228 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2022-09-03T05:14:46.371 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:01:57 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:14:46.640 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2022-09-03T05:14:46.641 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:46.641 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2022-09-03T05:14:46.642 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm-2:17.0.0-14661.g4d487742.el8.noarch 2022-09-03T05:14:46.642 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2022-09-03T05:14:46.643 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel-2.5.1-7.el8.noarch 2022-09-03T05:14:46.643 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2-2.10.1-3.el8.noarch 2022-09-03T05:14:46.644 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort-7.1.1-2.el8.noarch 2022-09-03T05:14:46.644 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:46.644 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:14:46.781 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-fuse 2022-09-03T05:14:47.357 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:57 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:14:47.373 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:14:47.375 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:47.375 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2022-09-03T05:14:47.376 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:47.376 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2022-09-03T05:14:47.377 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume noarch 2:17.0.0-14661.g4d487742.el8 ceph-noarch 262 k 2022-09-03T05:14:47.378 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:47.378 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2022-09-03T05:14:47.379 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:47.379 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2022-09-03T05:14:47.380 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:47.380 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 262 k 2022-09-03T05:14:47.381 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 1.3 M 2022-09-03T05:14:47.381 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2022-09-03T05:14:47.669 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-volume-17.0.0-14661.g4d487742.el8.noarch.r 891 kB/s | 262 kB 00:00 2022-09-03T05:14:47.671 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:14:47.675 INFO:teuthology.orchestra.run.smithi181.stdout:Total 888 kB/s | 262 kB 00:00 2022-09-03T05:14:47.676 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2022-09-03T05:14:47.681 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2022-09-03T05:14:47.681 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2022-09-03T05:14:47.706 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2022-09-03T05:14:47.706 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2022-09-03T05:14:47.872 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2022-09-03T05:14:48.113 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-volume-2:17.0.0-14661.g4d487742.el8.noarch 1/1 2022-09-03T05:14:48.379 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:14:48.380 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:14:48.380 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2022-09-03T05:14:48.380 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:14:48.381 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2022-09-03T05:14:48.381 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse x86_64 2:17.0.0-14661.g4d487742.el8 ceph 843 k 2022-09-03T05:14:48.382 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2022-09-03T05:14:48.383 INFO:teuthology.orchestra.run.smithi189.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2022-09-03T05:14:48.383 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:48.384 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2022-09-03T05:14:48.384 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:14:48.384 INFO:teuthology.orchestra.run.smithi189.stdout:Install 2 Packages 2022-09-03T05:14:48.385 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:48.385 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 926 k 2022-09-03T05:14:48.386 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 2.9 M 2022-09-03T05:14:48.387 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2022-09-03T05:14:48.455 INFO:teuthology.orchestra.run.smithi189.stdout:(1/2): fuse-2.9.7-16.el8.x86_64.rpm 1.0 MB/s | 83 kB 00:00 2022-09-03T05:14:48.484 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-volume-2:17.0.0-14661.g4d487742.el8.noarch 1/1 2022-09-03T05:14:48.851 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-volume-2:17.0.0-14661.g4d487742.el8.noarch 1/1 2022-09-03T05:14:48.852 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:48.854 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2022-09-03T05:14:48.854 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume-2:17.0.0-14661.g4d487742.el8.noarch 2022-09-03T05:14:48.855 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:48.855 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:14:49.048 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install librados-devel 2022-09-03T05:14:49.053 INFO:teuthology.orchestra.run.smithi189.stdout:(2/2): ceph-fuse-17.0.0-14661.g4d487742.el8.x86 1.2 MB/s | 843 kB 00:00 2022-09-03T05:14:49.056 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:14:49.057 INFO:teuthology.orchestra.run.smithi189.stdout:Total 1.3 MB/s | 926 kB 00:00 2022-09-03T05:14:49.065 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2022-09-03T05:14:49.090 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2022-09-03T05:14:49.091 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2022-09-03T05:14:49.162 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2022-09-03T05:14:49.166 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2022-09-03T05:14:49.331 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2022-09-03T05:14:49.633 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2022-09-03T05:14:49.635 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:00 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:14:49.816 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-fuse-2:17.0.0-14661.g4d487742.el8.x86_64 2/2 2022-09-03T05:14:50.177 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-fuse-2:17.0.0-14661.g4d487742.el8.x86_64 2/2 2022-09-03T05:14:50.178 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-fuse-2:17.0.0-14661.g4d487742.el8.x86_64 1/2 2022-09-03T05:14:50.644 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2022-09-03T05:14:50.645 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:50.645 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2022-09-03T05:14:50.645 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse-2:17.0.0-14661.g4d487742.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2022-09-03T05:14:50.646 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:50.646 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:14:50.659 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:14:50.660 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:50.661 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repo Size 2022-09-03T05:14:50.662 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:50.663 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2022-09-03T05:14:50.664 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel x86_64 2:17.0.0-14661.g4d487742.el8 ceph 139 k 2022-09-03T05:14:50.670 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:50.671 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2022-09-03T05:14:50.671 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:50.679 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2022-09-03T05:14:50.680 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:50.680 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 139 k 2022-09-03T05:14:50.681 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 511 k 2022-09-03T05:14:50.681 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2022-09-03T05:14:50.880 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-volume 2022-09-03T05:14:50.985 INFO:teuthology.orchestra.run.smithi181.stdout:librados-devel-17.0.0-14661.g4d487742.el8.x86_6 430 kB/s | 139 kB 00:00 2022-09-03T05:14:50.985 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:14:50.986 INFO:teuthology.orchestra.run.smithi181.stdout:Total 428 kB/s | 139 kB 00:00 2022-09-03T05:14:50.987 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2022-09-03T05:14:50.989 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2022-09-03T05:14:50.990 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2022-09-03T05:14:51.046 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2022-09-03T05:14:51.047 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2022-09-03T05:14:51.165 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2022-09-03T05:14:51.316 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librados-devel-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:14:51.466 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:02:01 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:14:51.570 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados-devel-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:14:51.947 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados-devel-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:14:51.948 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:51.948 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2022-09-03T05:14:51.949 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:14:51.949 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:51.951 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:14:52.144 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install libcephfs2 2022-09-03T05:14:52.473 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:14:52.473 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:14:52.473 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2022-09-03T05:14:52.474 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:14:52.474 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2022-09-03T05:14:52.474 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume noarch 2:17.0.0-14661.g4d487742.el8 ceph-noarch 262 k 2022-09-03T05:14:52.479 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:52.480 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2022-09-03T05:14:52.480 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:14:52.480 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2022-09-03T05:14:52.480 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:52.481 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 262 k 2022-09-03T05:14:52.481 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 1.3 M 2022-09-03T05:14:52.481 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2022-09-03T05:14:52.703 INFO:teuthology.orchestra.run.smithi189.stdout:ceph-volume-17.0.0-14661.g4d487742.el8.noarch.r 1.1 MB/s | 262 kB 00:00 2022-09-03T05:14:52.703 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:14:52.704 INFO:teuthology.orchestra.run.smithi189.stdout:Total 1.1 MB/s | 262 kB 00:00 2022-09-03T05:14:52.704 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2022-09-03T05:14:52.712 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2022-09-03T05:14:52.713 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2022-09-03T05:14:52.733 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:03 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:14:52.736 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2022-09-03T05:14:52.739 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2022-09-03T05:14:52.902 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2022-09-03T05:14:53.189 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-volume-2:17.0.0-14661.g4d487742.el8.noarch 1/1 2022-09-03T05:14:53.554 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-volume-2:17.0.0-14661.g4d487742.el8.noarch 1/1 2022-09-03T05:14:53.665 INFO:teuthology.orchestra.run.smithi181.stdout:Package libcephfs2-2:17.0.0-14661.g4d487742.el8.x86_64 is already installed. 2022-09-03T05:14:53.741 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:14:53.741 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2022-09-03T05:14:53.742 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:14:53.988 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-volume-2:17.0.0-14661.g4d487742.el8.noarch 1/1 2022-09-03T05:14:53.989 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:53.990 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2022-09-03T05:14:53.990 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume-2:17.0.0-14661.g4d487742.el8.noarch 2022-09-03T05:14:53.991 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:53.991 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:14:54.120 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install libcephfs-devel 2022-09-03T05:14:54.167 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install librados-devel 2022-09-03T05:14:54.698 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:05 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:14:54.768 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:02:04 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:14:55.700 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:14:55.701 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:55.701 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repo Size 2022-09-03T05:14:55.702 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:55.702 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2022-09-03T05:14:55.702 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel x86_64 2:17.0.0-14661.g4d487742.el8 ceph 30 k 2022-09-03T05:14:55.702 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:55.703 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2022-09-03T05:14:55.703 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:14:55.703 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2022-09-03T05:14:55.703 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:55.704 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 30 k 2022-09-03T05:14:55.704 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 137 k 2022-09-03T05:14:55.704 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2022-09-03T05:14:55.781 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:14:55.782 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:14:55.782 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repo Size 2022-09-03T05:14:55.787 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:14:55.788 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2022-09-03T05:14:55.788 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel x86_64 2:17.0.0-14661.g4d487742.el8 ceph 139 k 2022-09-03T05:14:55.789 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:55.789 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2022-09-03T05:14:55.790 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:14:55.791 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2022-09-03T05:14:55.791 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:55.792 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 139 k 2022-09-03T05:14:55.792 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 511 k 2022-09-03T05:14:55.792 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2022-09-03T05:14:55.853 INFO:teuthology.orchestra.run.smithi181.stdout:libcephfs-devel-17.0.0-14661.g4d487742.el8.x86_ 204 kB/s | 30 kB 00:00 2022-09-03T05:14:55.853 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:14:55.853 INFO:teuthology.orchestra.run.smithi181.stdout:Total 203 kB/s | 30 kB 00:00 2022-09-03T05:14:55.854 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2022-09-03T05:14:55.856 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2022-09-03T05:14:55.856 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2022-09-03T05:14:55.906 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2022-09-03T05:14:55.914 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2022-09-03T05:14:55.956 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2022-09-03T05:14:55.988 INFO:teuthology.orchestra.run.smithi189.stdout:librados-devel-17.0.0-14661.g4d487742.el8.x86_6 674 kB/s | 139 kB 00:00 2022-09-03T05:14:55.990 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:14:55.991 INFO:teuthology.orchestra.run.smithi189.stdout:Total 670 kB/s | 139 kB 00:00 2022-09-03T05:14:55.991 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2022-09-03T05:14:55.995 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2022-09-03T05:14:55.996 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2022-09-03T05:14:56.043 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2022-09-03T05:14:56.045 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2022-09-03T05:14:56.120 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs-devel-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:14:56.157 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2022-09-03T05:14:56.329 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephfs-devel-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:14:56.371 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librados-devel-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:14:56.605 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados-devel-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:14:56.680 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs-devel-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:14:56.681 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:56.681 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2022-09-03T05:14:56.681 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:14:56.682 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:14:56.682 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:14:56.785 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install librados2 2022-09-03T05:14:57.361 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:08 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:14:57.522 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados-devel-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:14:57.523 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:57.523 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2022-09-03T05:14:57.525 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:14:57.526 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:14:57.527 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:14:57.742 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install libcephfs2 2022-09-03T05:14:58.294 INFO:teuthology.orchestra.run.smithi181.stdout:Package librados2-2:17.0.0-14661.g4d487742.el8.x86_64 is already installed. 2022-09-03T05:14:58.328 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:02:08 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:14:58.369 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:14:58.371 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2022-09-03T05:14:58.371 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:14:58.470 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install librbd1 2022-09-03T05:14:59.045 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:10 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:14:59.262 INFO:teuthology.orchestra.run.smithi189.stdout:Package libcephfs2-2:17.0.0-14661.g4d487742.el8.x86_64 is already installed. 2022-09-03T05:14:59.338 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:14:59.339 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2022-09-03T05:14:59.340 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:14:59.416 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install libcephfs-devel 2022-09-03T05:14:59.982 INFO:teuthology.orchestra.run.smithi181.stdout:Package librbd1-2:17.0.0-14661.g4d487742.el8.x86_64 is already installed. 2022-09-03T05:14:59.992 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:02:09 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:15:00.053 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:15:00.055 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2022-09-03T05:15:00.055 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:15:00.713 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install python3-rados 2022-09-03T05:15:01.000 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:15:01.001 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:15:01.001 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repo Size 2022-09-03T05:15:01.002 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:15:01.002 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2022-09-03T05:15:01.003 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel x86_64 2:17.0.0-14661.g4d487742.el8 ceph 30 k 2022-09-03T05:15:01.007 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:01.008 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2022-09-03T05:15:01.008 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:15:01.008 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2022-09-03T05:15:01.008 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:01.009 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 30 k 2022-09-03T05:15:01.009 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 137 k 2022-09-03T05:15:01.009 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2022-09-03T05:15:01.165 INFO:teuthology.orchestra.run.smithi189.stdout:libcephfs-devel-17.0.0-14661.g4d487742.el8.x86_ 193 kB/s | 30 kB 00:00 2022-09-03T05:15:01.166 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:15:01.167 INFO:teuthology.orchestra.run.smithi189.stdout:Total 192 kB/s | 30 kB 00:00 2022-09-03T05:15:01.167 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2022-09-03T05:15:01.168 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2022-09-03T05:15:01.168 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2022-09-03T05:15:01.203 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2022-09-03T05:15:01.203 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2022-09-03T05:15:01.253 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2022-09-03T05:15:01.285 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:12 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:15:01.435 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs-devel-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:15:01.660 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephfs-devel-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:15:02.003 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs-devel-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:15:02.004 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:02.012 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2022-09-03T05:15:02.013 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:15:02.013 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:02.013 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:15:02.127 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install librados2 2022-09-03T05:15:02.214 INFO:teuthology.orchestra.run.smithi181.stdout:Package python3-rados-2:17.0.0-14661.g4d487742.el8.x86_64 is already installed. 2022-09-03T05:15:02.305 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:15:02.305 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2022-09-03T05:15:02.305 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:15:02.379 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install python3-rgw 2022-09-03T05:15:02.709 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:02:12 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:15:02.950 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:13 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:15:03.639 INFO:teuthology.orchestra.run.smithi189.stdout:Package librados2-2:17.0.0-14661.g4d487742.el8.x86_64 is already installed. 2022-09-03T05:15:03.715 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:15:03.715 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2022-09-03T05:15:03.716 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:15:03.879 INFO:teuthology.orchestra.run.smithi181.stdout:Package python3-rgw-2:17.0.0-14661.g4d487742.el8.x86_64 is already installed. 2022-09-03T05:15:03.954 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:15:03.956 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2022-09-03T05:15:03.956 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:15:04.035 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install python3-cephfs 2022-09-03T05:15:04.367 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install librbd1 2022-09-03T05:15:04.610 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:15 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:15:04.944 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:02:14 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:15:05.555 INFO:teuthology.orchestra.run.smithi181.stdout:Package python3-cephfs-2:17.0.0-14661.g4d487742.el8.x86_64 is already installed. 2022-09-03T05:15:05.631 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:15:05.632 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2022-09-03T05:15:05.633 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:15:05.720 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install python3-rbd 2022-09-03T05:15:05.874 INFO:teuthology.orchestra.run.smithi189.stdout:Package librbd1-2:17.0.0-14661.g4d487742.el8.x86_64 is already installed. 2022-09-03T05:15:05.954 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:15:05.955 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2022-09-03T05:15:05.955 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:15:06.035 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install python3-rados 2022-09-03T05:15:06.301 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:17 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:15:06.658 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:02:16 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:15:07.227 INFO:teuthology.orchestra.run.smithi181.stdout:Package python3-rbd-2:17.0.0-14661.g4d487742.el8.x86_64 is already installed. 2022-09-03T05:15:07.301 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:15:07.303 INFO:teuthology.orchestra.run.smithi181.stdout:Nothing to do. 2022-09-03T05:15:07.304 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:15:07.388 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install rbd-fuse 2022-09-03T05:15:07.562 INFO:teuthology.orchestra.run.smithi189.stdout:Package python3-rados-2:17.0.0-14661.g4d487742.el8.x86_64 is already installed. 2022-09-03T05:15:07.639 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:15:07.640 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2022-09-03T05:15:07.641 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:15:07.705 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install python3-rgw 2022-09-03T05:15:07.962 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:18 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:15:08.291 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:02:18 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:15:08.956 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:15:08.958 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:15:08.958 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repo Size 2022-09-03T05:15:08.959 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:15:08.959 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2022-09-03T05:15:08.960 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse x86_64 2:17.0.0-14661.g4d487742.el8 ceph 94 k 2022-09-03T05:15:08.960 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:08.960 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2022-09-03T05:15:08.961 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:15:08.961 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2022-09-03T05:15:08.961 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:08.962 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 94 k 2022-09-03T05:15:08.962 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 280 k 2022-09-03T05:15:08.963 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2022-09-03T05:15:09.223 INFO:teuthology.orchestra.run.smithi189.stdout:Package python3-rgw-2:17.0.0-14661.g4d487742.el8.x86_64 is already installed. 2022-09-03T05:15:09.275 INFO:teuthology.orchestra.run.smithi181.stdout:rbd-fuse-17.0.0-14661.g4d487742.el8.x86_64.rpm 304 kB/s | 94 kB 00:00 2022-09-03T05:15:09.276 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:15:09.276 INFO:teuthology.orchestra.run.smithi181.stdout:Total 303 kB/s | 94 kB 00:00 2022-09-03T05:15:09.277 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2022-09-03T05:15:09.279 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2022-09-03T05:15:09.280 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2022-09-03T05:15:09.298 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:15:09.300 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2022-09-03T05:15:09.300 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:15:09.328 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2022-09-03T05:15:09.333 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2022-09-03T05:15:09.414 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install python3-cephfs 2022-09-03T05:15:09.452 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2022-09-03T05:15:09.594 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-fuse-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:15:09.834 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-fuse-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:15:10.018 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:02:20 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:15:10.218 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-fuse-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:15:10.218 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:10.219 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2022-09-03T05:15:10.219 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:15:10.219 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:10.219 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:15:10.435 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install rbd-mirror 2022-09-03T05:15:10.943 INFO:teuthology.orchestra.run.smithi189.stdout:Package python3-cephfs-2:17.0.0-14661.g4d487742.el8.x86_64 is already installed. 2022-09-03T05:15:11.019 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:15:11.019 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2022-09-03T05:15:11.020 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:15:11.032 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:22 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:15:11.098 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install python3-rbd 2022-09-03T05:15:11.679 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:02:21 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:15:12.039 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:15:12.041 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:15:12.041 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repo Size 2022-09-03T05:15:12.041 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:15:12.042 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2022-09-03T05:15:12.042 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror x86_64 2:17.0.0-14661.g4d487742.el8 ceph 3.5 M 2022-09-03T05:15:12.042 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:12.042 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2022-09-03T05:15:12.043 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:15:12.043 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2022-09-03T05:15:12.044 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:12.044 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 3.5 M 2022-09-03T05:15:12.044 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 16 M 2022-09-03T05:15:12.045 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2022-09-03T05:15:12.617 INFO:teuthology.orchestra.run.smithi189.stdout:Package python3-rbd-2:17.0.0-14661.g4d487742.el8.x86_64 is already installed. 2022-09-03T05:15:12.687 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:15:12.687 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2022-09-03T05:15:12.687 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:15:12.776 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install rbd-fuse 2022-09-03T05:15:13.350 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:02:23 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:15:13.873 INFO:teuthology.orchestra.run.smithi181.stdout:rbd-mirror-17.0.0-14661.g4d487742.el8.x86_64.rp 1.9 MB/s | 3.5 MB 00:01 2022-09-03T05:15:13.874 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:15:13.875 INFO:teuthology.orchestra.run.smithi181.stdout:Total 1.9 MB/s | 3.5 MB 00:01 2022-09-03T05:15:13.875 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2022-09-03T05:15:13.886 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2022-09-03T05:15:13.887 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2022-09-03T05:15:13.949 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2022-09-03T05:15:13.952 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2022-09-03T05:15:14.366 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:15:14.367 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:15:14.367 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repo Size 2022-09-03T05:15:14.367 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:15:14.367 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2022-09-03T05:15:14.367 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse x86_64 2:17.0.0-14661.g4d487742.el8 ceph 94 k 2022-09-03T05:15:14.368 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:14.368 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2022-09-03T05:15:14.368 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:15:14.368 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2022-09-03T05:15:14.369 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:14.369 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 94 k 2022-09-03T05:15:14.369 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 280 k 2022-09-03T05:15:14.369 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2022-09-03T05:15:14.387 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2022-09-03T05:15:14.543 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-mirror-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:15:14.785 INFO:teuthology.orchestra.run.smithi189.stdout:rbd-fuse-17.0.0-14661.g4d487742.el8.x86_64.rpm 228 kB/s | 94 kB 00:00 2022-09-03T05:15:14.785 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:15:14.786 INFO:teuthology.orchestra.run.smithi189.stdout:Total 228 kB/s | 94 kB 00:00 2022-09-03T05:15:14.786 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2022-09-03T05:15:14.802 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2022-09-03T05:15:14.802 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2022-09-03T05:15:14.841 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2022-09-03T05:15:14.842 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2022-09-03T05:15:14.925 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-mirror-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:15:14.958 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2022-09-03T05:15:15.143 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-fuse-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:15:15.296 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-mirror-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:15:15.297 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:15.297 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2022-09-03T05:15:15.298 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:15:15.298 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:15.299 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:15:15.397 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: rbd-fuse-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:15:15.505 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install rbd-nbd 2022-09-03T05:15:15.791 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-fuse-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:15:15.791 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:15.791 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2022-09-03T05:15:15.792 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:15:15.792 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:15.792 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:15:15.973 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install rbd-mirror 2022-09-03T05:15:16.092 INFO:teuthology.orchestra.run.smithi181.stdout:Last metadata expiration check: 0:02:27 ago on Sat 03 Sep 2022 05:12:49 AM UTC. 2022-09-03T05:15:16.570 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:02:26 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:15:17.103 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2022-09-03T05:15:17.106 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:15:17.106 INFO:teuthology.orchestra.run.smithi181.stdout: Package Architecture Version Repository Size 2022-09-03T05:15:17.107 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:15:17.107 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2022-09-03T05:15:17.109 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd x86_64 2:17.0.0-14661.g4d487742.el8 ceph 175 k 2022-09-03T05:15:17.128 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:17.128 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2022-09-03T05:15:17.129 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2022-09-03T05:15:17.129 INFO:teuthology.orchestra.run.smithi181.stdout:Install 1 Package 2022-09-03T05:15:17.130 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:17.130 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 175 k 2022-09-03T05:15:17.131 INFO:teuthology.orchestra.run.smithi181.stdout:Installed size: 556 k 2022-09-03T05:15:17.131 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2022-09-03T05:15:17.584 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:15:17.586 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:15:17.586 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repo Size 2022-09-03T05:15:17.587 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:15:17.587 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2022-09-03T05:15:17.588 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror x86_64 2:17.0.0-14661.g4d487742.el8 ceph 3.5 M 2022-09-03T05:15:17.588 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:17.589 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2022-09-03T05:15:17.589 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:15:17.589 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2022-09-03T05:15:17.590 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:17.590 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 3.5 M 2022-09-03T05:15:17.591 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 16 M 2022-09-03T05:15:17.591 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2022-09-03T05:15:18.488 INFO:teuthology.orchestra.run.smithi189.stdout:rbd-mirror-17.0.0-14661.g4d487742.el8.x86_64.rp 3.8 MB/s | 3.5 MB 00:00 2022-09-03T05:15:18.489 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:15:18.489 INFO:teuthology.orchestra.run.smithi189.stdout:Total 3.8 MB/s | 3.5 MB 00:00 2022-09-03T05:15:18.489 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2022-09-03T05:15:18.500 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2022-09-03T05:15:18.500 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2022-09-03T05:15:18.561 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2022-09-03T05:15:18.565 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2022-09-03T05:15:19.003 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2022-09-03T05:15:19.184 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-mirror-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:15:19.574 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: rbd-mirror-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:15:20.176 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-mirror-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:15:20.177 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:20.178 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2022-09-03T05:15:20.178 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:15:20.178 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:20.179 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:15:20.347 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install rbd-nbd 2022-09-03T05:15:20.938 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:02:30 ago on Sat 03 Sep 2022 05:12:50 AM UTC. 2022-09-03T05:15:21.020 INFO:teuthology.orchestra.run.smithi181.stdout:rbd-nbd-17.0.0-14661.g4d487742.el8.x86_64.rpm 45 kB/s | 175 kB 00:03 2022-09-03T05:15:21.022 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:15:21.022 INFO:teuthology.orchestra.run.smithi181.stdout:Total 45 kB/s | 175 kB 00:03 2022-09-03T05:15:21.023 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2022-09-03T05:15:21.032 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2022-09-03T05:15:21.033 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2022-09-03T05:15:21.087 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2022-09-03T05:15:21.087 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2022-09-03T05:15:21.201 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2022-09-03T05:15:21.369 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-nbd-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:15:21.618 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-nbd-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:15:21.952 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2022-09-03T05:15:21.953 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:15:21.954 INFO:teuthology.orchestra.run.smithi189.stdout: Package Architecture Version Repository Size 2022-09-03T05:15:21.955 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:15:21.955 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2022-09-03T05:15:21.955 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd x86_64 2:17.0.0-14661.g4d487742.el8 ceph 175 k 2022-09-03T05:15:21.955 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:21.956 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2022-09-03T05:15:21.956 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2022-09-03T05:15:21.956 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2022-09-03T05:15:21.956 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:21.957 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 175 k 2022-09-03T05:15:21.957 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 556 k 2022-09-03T05:15:21.957 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2022-09-03T05:15:22.022 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-nbd-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:15:22.022 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:22.022 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2022-09-03T05:15:22.023 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:15:22.023 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:22.023 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2022-09-03T05:15:22.201 DEBUG:teuthology.parallel:result is None 2022-09-03T05:15:22.209 INFO:teuthology.orchestra.run.smithi189.stdout:rbd-nbd-17.0.0-14661.g4d487742.el8.x86_64.rpm 680 kB/s | 175 kB 00:00 2022-09-03T05:15:22.213 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2022-09-03T05:15:22.213 INFO:teuthology.orchestra.run.smithi189.stdout:Total 672 kB/s | 175 kB 00:00 2022-09-03T05:15:22.214 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2022-09-03T05:15:22.223 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2022-09-03T05:15:22.224 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2022-09-03T05:15:22.273 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2022-09-03T05:15:22.275 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2022-09-03T05:15:22.412 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2022-09-03T05:15:22.630 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-nbd-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:15:22.887 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: rbd-nbd-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:15:23.292 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-nbd-2:17.0.0-14661.g4d487742.el8.x86_64 1/1 2022-09-03T05:15:23.292 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:23.293 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2022-09-03T05:15:23.293 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd-2:17.0.0-14661.g4d487742.el8.x86_64 2022-09-03T05:15:23.294 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:23.294 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2022-09-03T05:15:23.512 DEBUG:teuthology.parallel:result is None 2022-09-03T05:15:23.513 DEBUG:teuthology.orchestra.run.smithi181:> sudo lsb_release -is 2022-09-03T05:15:23.559 INFO:teuthology.orchestra.run.smithi181.stdout:CentOSStream 2022-09-03T05:15:23.562 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-09-03T05:15:23.563 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4d487742eda56d4badbc9bdb33761243365bb37a 2022-09-03T05:15:23.745 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-09-03T05:15:23.770 INFO:teuthology.orchestra.run.smithi181.stdout:17.0.0-14661.g4d487742.el8 2022-09-03T05:15:23.771 INFO:teuthology.packaging:The installed version of ceph is 17.0.0-14661.g4d487742.el8 2022-09-03T05:15:23.772 INFO:teuthology.task.install:The correct ceph version 17.0.0-14661.g4d487742 is installed. 2022-09-03T05:15:23.772 DEBUG:teuthology.orchestra.run.smithi189:> sudo lsb_release -is 2022-09-03T05:15:23.810 INFO:teuthology.orchestra.run.smithi189.stdout:CentOSStream 2022-09-03T05:15:23.817 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-09-03T05:15:23.818 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4d487742eda56d4badbc9bdb33761243365bb37a 2022-09-03T05:15:24.006 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-09-03T05:15:24.034 INFO:teuthology.orchestra.run.smithi189.stdout:17.0.0-14661.g4d487742.el8 2022-09-03T05:15:24.035 INFO:teuthology.packaging:The installed version of ceph is 17.0.0-14661.g4d487742.el8 2022-09-03T05:15:24.036 INFO:teuthology.task.install:The correct ceph version 17.0.0-14661.g4d487742 is installed. 2022-09-03T05:15:24.036 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2022-09-03T05:15:24.037 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2022-09-03T05:15:24.039 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-09-03T05:15:24.073 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2022-09-03T05:15:24.074 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-09-03T05:15:24.101 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2022-09-03T05:15:24.102 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2022-09-03T05:15:24.102 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/daemon-helper 2022-09-03T05:15:24.151 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-09-03T05:15:24.226 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2022-09-03T05:15:24.235 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/daemon-helper 2022-09-03T05:15:24.270 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-09-03T05:15:24.341 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2022-09-03T05:15:24.342 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2022-09-03T05:15:24.343 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/adjust-ulimits 2022-09-03T05:15:24.377 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-09-03T05:15:24.447 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2022-09-03T05:15:24.448 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/adjust-ulimits 2022-09-03T05:15:24.475 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-09-03T05:15:24.543 INFO:teuthology.run_tasks:Running task ceph... 2022-09-03T05:15:24.685 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2022-09-03T05:15:24.687 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 777 /var/log/ceph 2022-09-03T05:15:24.690 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 777 /var/log/ceph 2022-09-03T05:15:24.719 INFO:tasks.ceph:Disabling ceph logrotate... 2022-09-03T05:15:24.720 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/logrotate.d/ceph 2022-09-03T05:15:24.760 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /etc/logrotate.d/ceph 2022-09-03T05:15:24.788 INFO:tasks.ceph:Creating extra log directories... 2022-09-03T05:15:24.789 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-09-03T05:15:24.833 DEBUG:teuthology.orchestra.run.smithi189:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-09-03T05:15:24.886 INFO:tasks.ceph:Creating ceph cluster c1... 2022-09-03T05:15:24.887 INFO:tasks.ceph:config {'conf': {'client': {'debug rgw': 20, 'debug rgw sync': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync data inject err probability': 0.1, 'rgw sync log trim interval': 0, 'rgw sync meta inject err probability': 0.1, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'client.0': {'rgw data notify interval msec': 0}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'rocksdb delete range threshold': 0}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'cpu_profile': set(), 'cluster': 'c1', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2022-09-03T05:15:24.888 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/dang-2022-09-02_07:58:15-rgw-wip-dang-zipper-admin-api-distro-default-smithi/7007679', 'branch': 'wip-dang-zipper-admin-api', 'description': 'rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zone-plus-pubsub supported-random-distro$/{centos_8} tasks/test_multi}', 'email': None, 'first_in_suite': False, 'job_id': '7007679', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'dang-2022-09-02_07:58:15-rgw-wip-dang-zipper-admin-api-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-dang-zipper-admin-api'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'debug rgw sync': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync data inject err probability': 0.1, 'rgw sync log trim interval': 0, 'rgw sync meta inject err probability': 0.1, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'client.0': {'rgw data notify interval msec': 0}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'rocksdb delete range threshold': 0}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '4d487742eda56d4badbc9bdb33761243365bb37a', 'wait-for-scrub': False}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '4d487742eda56d4badbc9bdb33761243365bb37a'}}, 'rgw': {'compression type': 'random', 'frontend': 'beast'}, 'rgw-multisite': {'realm': {'is default': True, 'name': 'test-realm'}, 'zonegroups': [{'enabled_features': ['resharding'], 'endpoints': ['c1.client.0'], 'is_default': True, 'is_master': True, 'name': 'test-zonegroup', 'zones': [{'endpoints': ['c1.client.0'], 'is_default': True, 'is_master': True, 'name': 'test-zone1'}, {'endpoints': ['c2.client.0'], 'is_default': True, 'name': 'test-zone2'}, {'endpoints': ['c1.client.1'], 'name': 'test-zone3'}, {'endpoints': ['c2.client.1'], 'is_pubsub': True, 'name': 'test-zone4'}]}]}, 'rgw-multisite-tests': {'args': ['tests.py', 'tests_ps.py']}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'workunit': {'branch': 'wip-dang-zipper-admin-api', 'sha1': '4d487742eda56d4badbc9bdb33761243365bb37a'}}, 'owner': 'scheduled_dang@sidious.arb.redhat.com', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'], ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1']], 'sha1': '4d487742eda56d4badbc9bdb33761243365bb37a', 'sleep_before_teardown': 0, 'suite': 'rgw', 'suite_branch': 'wip-dang-zipper-admin-api', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_4d487742eda56d4badbc9bdb33761243365bb37a/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '4d487742eda56d4badbc9bdb33761243365bb37a', 'targets': {'smithi181.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDpTjEkBg4fXYR5zbI8X1aIX0ry1owo8ATLzirNBazD0ie3B5bmEQ5uTTpXbhbf38TCOicU7Kr7jvVMjp3AWagRW1Z/slXFYQo7xV7wq9O5eljNYgdg4GGmLc4ySZrcCfOzKPfS+3SZVPb/66kxGPJmtIn39hoAN2wj6oFZVCGVADXDLjcPvRXmWHrdJypT2cEY5Ws4o7hOwfxKLus2446DBRdhteOUYqOQR8U4mYU6SLNqcSJRdYQWduH4fqZ+XH9rrf25EwFaPqC4qaj0bngkp5NVSHxqeNE2E2p69KIeQmPr0hbjH5GENnQu4m6NlRJijtGz87r1IlxjC8mzspit5PKbEWfawJWbfstOcTATxYlZh0SP2Qm7HP+y7oVxls18yezhrYIzMM9x0pCtWM/Gbtj8uOz8ZwfKgJ7ojFfz4dPnvFxlqNPjvAsCVLvGog4YWDtG6T08pvidSfJU3QIcA6ZWm7UNPCddUzZHYkvF5XqM2D7gE45qGm++vsNIOBs=', 'smithi189.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCxjbJLTVVZDanIouEGFrJENZLRX4RqVQVKScenNpB1JvF6Qxa0CEAI+dllK2d1mgZBvdyQ+8cAVgmZODJHYi/2nBT2G6zXdMuCp6vYu7+HM8/ZgrBwig6Nx+Y4Oj0/QwwlVJDp4GQuQZKRnR/Wmej1JJRnzcPpd7sLK95+DdEwmBX+3kJPM7SZ+xT+jxri9l6zwgFA2LYPwS2s3WhFA43Tks2OzzN/VgCVZHxbqbH4DBThmpXyWKJz1X1ufqc208z1xFhMy4e8UDJb8VHiOALYQTuT4atNidJELjdaY915boUU/2ACXxovBgqiXHyvJFF+x9N9MPVPSm43GMCb7IxQffBP0UB1m3E5FoQIJtsp8VYILST6U104HNWJ0NGDD8Euh2RUYlyEcxvjwSemVBHw3/QWjwZif3k9GcdOZOKH35gNKGF+UX82+aW39UwYrm7axfkO/HnP57ZZJdsq2334ENS5b607SMAnHrOyCDhkQvaYBoSq9X5JqLkVuBR4/8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'cluster': 'c1', 'conf': {'client': {'debug rgw': 20, 'debug rgw sync': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync data inject err probability': 0.1, 'rgw sync log trim interval': 0, 'rgw sync meta inject err probability': 0.1, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'client.0': {'rgw data notify interval msec': 0}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'rocksdb delete range threshold': 0}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '4d487742eda56d4badbc9bdb33761243365bb37a', 'wait-for-scrub': False}}, {'ceph': {'cluster': 'c2'}}, {'rgw': {'c1.client.0': {'port': 8000}, 'c1.client.1': {'port': 8001}, 'c2.client.0': {'port': 8000}, 'c2.client.1': {'port': 8001}}}, {'rgw-multisite': None}, {'rgw-multisite-tests': {'config': {'reconfigure_delay': 60}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'b58f2c18636eb10faa77ed3614abd00cb85dfc2c', 'timestamp': '2022-09-02_07:58:15', 'tube': 'smithi', 'user': 'dang', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.1713'} 2022-09-03T05:15:24.889 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/c1.data 2022-09-03T05:15:24.913 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/c1.data 2022-09-03T05:15:24.947 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m0777 -- /var/run/ceph 2022-09-03T05:15:25.000 DEBUG:teuthology.orchestra.run.smithi189:> sudo install -d -m0777 -- /var/run/ceph 2022-09-03T05:15:25.045 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2022-09-03T05:15:25.046 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/scratch_devs of=/dev/stdout 2022-09-03T05:15:25.103 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-09-03T05:15:25.106 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2022-09-03T05:15:25.169 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-09-03T05:15:25.169 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-09-03T05:15:25.169 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 359214 Links: 1 2022-09-03T05:15:25.170 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-09-03T05:15:25.172 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2022-09-03T05:15:25.173 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2022-09-03 05:15:14.710762375 +0000 2022-09-03T05:15:25.173 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2022-09-03 05:11:49.020724370 +0000 2022-09-03T05:15:25.174 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2022-09-03 05:11:49.020724370 +0000 2022-09-03T05:15:25.174 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2022-09-03T05:15:25.180 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2022-09-03T05:15:25.181 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2022-09-03T05:15:25.188 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000211752 s, 2.4 MB/s 2022-09-03T05:15:25.191 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2022-09-03T05:15:25.259 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-09-03T05:15:25.259 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-09-03T05:15:25.259 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 361334 Links: 1 2022-09-03T05:15:25.260 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-09-03T05:15:25.260 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2022-09-03T05:15:25.261 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2022-09-03 05:15:14.710762375 +0000 2022-09-03T05:15:25.261 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2022-09-03 05:11:49.779709105 +0000 2022-09-03T05:15:25.261 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2022-09-03 05:11:49.779709105 +0000 2022-09-03T05:15:25.262 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2022-09-03T05:15:25.267 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2022-09-03T05:15:25.268 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2022-09-03T05:15:25.268 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000209845 s, 2.4 MB/s 2022-09-03T05:15:25.273 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2022-09-03T05:15:25.338 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-09-03T05:15:25.338 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-09-03T05:15:25.339 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 370015 Links: 1 2022-09-03T05:15:25.339 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-09-03T05:15:25.339 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2022-09-03T05:15:25.339 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2022-09-03 05:15:14.710762375 +0000 2022-09-03T05:15:25.340 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2022-09-03 05:11:50.482694967 +0000 2022-09-03T05:15:25.340 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2022-09-03 05:11:50.482694967 +0000 2022-09-03T05:15:25.340 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2022-09-03T05:15:25.348 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2022-09-03T05:15:25.348 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2022-09-03T05:15:25.349 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000213542 s, 2.4 MB/s 2022-09-03T05:15:25.352 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2022-09-03T05:15:25.420 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-09-03T05:15:25.420 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-09-03T05:15:25.421 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 6h/6d Inode: 373780 Links: 1 2022-09-03T05:15:25.421 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-09-03T05:15:25.421 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2022-09-03T05:15:25.422 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2022-09-03 05:15:14.710762375 +0000 2022-09-03T05:15:25.422 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2022-09-03 05:11:51.221680105 +0000 2022-09-03T05:15:25.422 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2022-09-03 05:11:51.221680105 +0000 2022-09-03T05:15:25.423 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2022-09-03T05:15:25.425 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2022-09-03T05:15:25.426 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2022-09-03T05:15:25.427 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000214651 s, 2.4 MB/s 2022-09-03T05:15:25.427 INFO:tasks.ceph:osd dev map: {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'} 2022-09-03T05:15:25.428 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi181.front.sepia.ceph.com'): {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'}} 2022-09-03T05:15:25.428 INFO:tasks.ceph:Generating config... 2022-09-03T05:15:25.431 INFO:tasks.ceph:[client] debug rgw = 20 2022-09-03T05:15:25.432 INFO:tasks.ceph:[client] debug rgw sync = 20 2022-09-03T05:15:25.432 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2022-09-03T05:15:25.433 INFO:tasks.ceph:[client] rgw crypt s3 kms backend = testing 2022-09-03T05:15:25.433 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2022-09-03T05:15:25.434 INFO:tasks.ceph:[client] rgw curl low speed time = 300 2022-09-03T05:15:25.435 INFO:tasks.ceph:[client] rgw data log num shards = 4 2022-09-03T05:15:25.435 INFO:tasks.ceph:[client] rgw md log max shards = 4 2022-09-03T05:15:25.436 INFO:tasks.ceph:[client] rgw sync data inject err probability = 0.1 2022-09-03T05:15:25.436 INFO:tasks.ceph:[client] rgw sync log trim interval = 0 2022-09-03T05:15:25.436 INFO:tasks.ceph:[client] rgw sync meta inject err probability = 0.1 2022-09-03T05:15:25.437 INFO:tasks.ceph:[client] rgw sync obj etag verify = True 2022-09-03T05:15:25.437 INFO:tasks.ceph:[client] setgroup = ceph 2022-09-03T05:15:25.438 INFO:tasks.ceph:[client] setuser = ceph 2022-09-03T05:15:25.438 INFO:tasks.ceph:[client.0] rgw data notify interval msec = 0 2022-09-03T05:15:25.438 INFO:tasks.ceph:[mgr] debug mgr = 20 2022-09-03T05:15:25.439 INFO:tasks.ceph:[mgr] debug ms = 1 2022-09-03T05:15:25.440 INFO:tasks.ceph:[mon] debug mon = 20 2022-09-03T05:15:25.440 INFO:tasks.ceph:[mon] debug ms = 1 2022-09-03T05:15:25.440 INFO:tasks.ceph:[mon] debug paxos = 20 2022-09-03T05:15:25.441 INFO:tasks.ceph:[osd] debug ms = 1 2022-09-03T05:15:25.441 INFO:tasks.ceph:[osd] debug osd = 20 2022-09-03T05:15:25.441 INFO:tasks.ceph:[osd] rocksdb delete range threshold = 0 2022-09-03T05:15:25.442 INFO:tasks.ceph:[osd.0] osd_max_omap_entries_per_request = 10 2022-09-03T05:15:25.444 INFO:tasks.ceph:[osd.1] osd_max_omap_entries_per_request = 1000 2022-09-03T05:15:25.444 INFO:tasks.ceph:[osd.2] osd_max_omap_entries_per_request = 10000 2022-09-03T05:15:25.445 INFO:tasks.ceph:Setting up c1.mon.a... 2022-09-03T05:15:25.445 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/c1.keyring 2022-09-03T05:15:25.513 INFO:teuthology.orchestra.run.smithi181.stdout:creating /etc/ceph/c1.keyring 2022-09-03T05:15:25.517 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/c1.keyring 2022-09-03T05:15:25.602 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0644 /etc/ceph/c1.keyring 2022-09-03T05:15:25.671 DEBUG:tasks.ceph:Ceph mon addresses: [('c1.mon.a', '172.21.15.181')] 2022-09-03T05:15:25.677 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.181'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug ms': 1, 'debug osd': 20, 'rocksdb delete range threshold': 0}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rgw': 20, 'debug rgw sync': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync data inject err probability': 0.1, 'rgw sync log trim interval': 0, 'rgw sync meta inject err probability': 0.1, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}, 'client.0': {'rgw data notify interval msec': 0}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}} 2022-09-03T05:15:25.678 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2022-09-03T05:15:25.679 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2022-09-03T05:15:25.734 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.181 --print /home/ubuntu/cephtest/c1.monmap 2022-09-03T05:15:25.820 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:15:25.820 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:15:25.823 INFO:teuthology.orchestra.run.smithi181.stdout:monmaptool: monmap file /home/ubuntu/cephtest/c1.monmap 2022-09-03T05:15:25.823 INFO:teuthology.orchestra.run.smithi181.stdout:monmaptool: generated fsid 1b22fd20-d842-48ea-bdcc-6e548483fbc3 2022-09-03T05:15:25.823 INFO:teuthology.orchestra.run.smithi181.stdout:setting min_mon_release = octopus 2022-09-03T05:15:25.824 INFO:teuthology.orchestra.run.smithi181.stdout:epoch 0 2022-09-03T05:15:25.824 INFO:teuthology.orchestra.run.smithi181.stdout:fsid 1b22fd20-d842-48ea-bdcc-6e548483fbc3 2022-09-03T05:15:25.824 INFO:teuthology.orchestra.run.smithi181.stdout:last_changed 2022-09-03T05:15:25.819335+0000 2022-09-03T05:15:25.824 INFO:teuthology.orchestra.run.smithi181.stdout:created 2022-09-03T05:15:25.819335+0000 2022-09-03T05:15:25.825 INFO:teuthology.orchestra.run.smithi181.stdout:min_mon_release 15 (octopus) 2022-09-03T05:15:25.825 INFO:teuthology.orchestra.run.smithi181.stdout:election_strategy: 1 2022-09-03T05:15:25.825 INFO:teuthology.orchestra.run.smithi181.stdout:0: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.a 2022-09-03T05:15:25.826 INFO:teuthology.orchestra.run.smithi181.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/c1.monmap (1 monitors) 2022-09-03T05:15:25.826 DEBUG:teuthology.orchestra.run.smithi181:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2022-09-03T05:15:25.882 INFO:tasks.ceph:Writing /etc/ceph/c1.conf for FSID 1b22fd20-d842-48ea-bdcc-6e548483fbc3... 2022-09-03T05:15:25.884 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c1.conf && sudo chmod 0644 /etc/ceph/c1.conf > /dev/null 2022-09-03T05:15:25.930 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c1.conf && sudo chmod 0644 /etc/ceph/c1.conf > /dev/null 2022-09-03T05:15:25.973 INFO:teuthology.orchestra.run.smithi181.stdout:[global] 2022-09-03T05:15:25.973 INFO:teuthology.orchestra.run.smithi181.stdout: chdir = "" 2022-09-03T05:15:25.973 INFO:teuthology.orchestra.run.smithi181.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-09-03T05:15:25.973 INFO:teuthology.orchestra.run.smithi181.stdout: auth supported = cephx 2022-09-03T05:15:25.974 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.974 INFO:teuthology.orchestra.run.smithi181.stdout: filestore xattr use omap = true 2022-09-03T05:15:25.974 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.974 INFO:teuthology.orchestra.run.smithi181.stdout: mon clock drift allowed = 1.000 2022-09-03T05:15:25.975 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.975 INFO:teuthology.orchestra.run.smithi181.stdout: osd crush chooseleaf type = 0 2022-09-03T05:15:25.975 INFO:teuthology.orchestra.run.smithi181.stdout: auth debug = true 2022-09-03T05:15:25.975 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.976 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on old message = true 2022-09-03T05:15:25.976 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on bug = true 2022-09-03T05:15:25.976 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.976 INFO:teuthology.orchestra.run.smithi181.stdout: mon max pg per osd = 10000 # >= luminous 2022-09-03T05:15:25.977 INFO:teuthology.orchestra.run.smithi181.stdout: mon pg warn max object skew = 0 2022-09-03T05:15:25.977 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.977 INFO:teuthology.orchestra.run.smithi181.stdout: # disable pg_autoscaler by default for new pools 2022-09-03T05:15:25.977 INFO:teuthology.orchestra.run.smithi181.stdout: osd_pool_default_pg_autoscale_mode = off 2022-09-03T05:15:25.978 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.978 INFO:teuthology.orchestra.run.smithi181.stdout: osd pool default size = 2 2022-09-03T05:15:25.978 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.979 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd allow primary affinity = true 2022-09-03T05:15:25.979 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd allow pg remap = true 2022-09-03T05:15:25.979 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on legacy crush tunables = false 2022-09-03T05:15:25.979 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on crush straw calc version zero = false 2022-09-03T05:15:25.979 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on no sortbitwise = false 2022-09-03T05:15:25.980 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on osd down out interval zero = false 2022-09-03T05:15:25.980 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on too few osds = false 2022-09-03T05:15:25.980 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-09-03T05:15:25.980 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_pool_no_redundancy = false 2022-09-03T05:15:25.981 INFO:teuthology.orchestra.run.smithi181.stdout: mon_allow_pool_size_one = true 2022-09-03T05:15:25.981 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.982 INFO:teuthology.orchestra.run.smithi181.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-09-03T05:15:25.982 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.982 INFO:teuthology.orchestra.run.smithi181.stdout: osd default data pool replay window = 5 2022-09-03T05:15:25.982 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.983 INFO:teuthology.orchestra.run.smithi181.stdout: mon allow pool delete = true 2022-09-03T05:15:25.983 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.983 INFO:teuthology.orchestra.run.smithi181.stdout: mon cluster log file level = debug 2022-09-03T05:15:25.983 INFO:teuthology.orchestra.run.smithi181.stdout: debug asserts on shutdown = true 2022-09-03T05:15:25.983 INFO:teuthology.orchestra.run.smithi181.stdout: mon health detail to clog = false 2022-09-03T05:15:25.984 INFO:teuthology.orchestra.run.smithi181.stdout: mon host = 172.21.15.181 2022-09-03T05:15:25.984 INFO:teuthology.orchestra.run.smithi181.stdout: fsid = 1b22fd20-d842-48ea-bdcc-6e548483fbc3 2022-09-03T05:15:25.984 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.984 INFO:teuthology.orchestra.run.smithi181.stdout:[osd] 2022-09-03T05:15:25.985 INFO:teuthology.orchestra.run.smithi181.stdout: osd journal size = 100 2022-09-03T05:15:25.985 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.985 INFO:teuthology.orchestra.run.smithi181.stdout: osd scrub load threshold = 5.0 2022-09-03T05:15:25.986 INFO:teuthology.orchestra.run.smithi181.stdout: osd scrub max interval = 600 2022-09-03T05:15:25.986 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.986 INFO:teuthology.orchestra.run.smithi181.stdout: osd recover clone overlap = true 2022-09-03T05:15:25.986 INFO:teuthology.orchestra.run.smithi181.stdout: osd recovery max chunk = 1048576 2022-09-03T05:15:25.987 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.987 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug shutdown = true 2022-09-03T05:15:25.987 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug op order = true 2022-09-03T05:15:25.987 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug verify stray on activate = true 2022-09-03T05:15:25.987 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.988 INFO:teuthology.orchestra.run.smithi181.stdout: osd open classes on start = true 2022-09-03T05:15:25.988 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug pg log writeout = true 2022-09-03T05:15:25.988 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.988 INFO:teuthology.orchestra.run.smithi181.stdout: osd deep scrub update digest min age = 30 2022-09-03T05:15:25.989 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.989 INFO:teuthology.orchestra.run.smithi181.stdout: osd map max advance = 10 2022-09-03T05:15:25.989 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.989 INFO:teuthology.orchestra.run.smithi181.stdout: journal zero on create = true 2022-09-03T05:15:25.989 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.990 INFO:teuthology.orchestra.run.smithi181.stdout: filestore ondisk finisher threads = 3 2022-09-03T05:15:25.990 INFO:teuthology.orchestra.run.smithi181.stdout: filestore apply finisher threads = 3 2022-09-03T05:15:25.990 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.990 INFO:teuthology.orchestra.run.smithi181.stdout: bdev debug aio = true 2022-09-03T05:15:25.991 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug misdirected ops = true 2022-09-03T05:15:25.991 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2022-09-03T05:15:25.991 INFO:teuthology.orchestra.run.smithi181.stdout: debug osd = 20 2022-09-03T05:15:25.992 INFO:teuthology.orchestra.run.smithi181.stdout: rocksdb delete range threshold = 0 2022-09-03T05:15:25.992 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.992 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr] 2022-09-03T05:15:25.993 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2022-09-03T05:15:25.993 INFO:teuthology.orchestra.run.smithi181.stdout: debug mgr = 20 2022-09-03T05:15:25.993 INFO:teuthology.orchestra.run.smithi181.stdout: debug mon = 20 2022-09-03T05:15:25.993 INFO:teuthology.orchestra.run.smithi181.stdout: debug auth = 20 2022-09-03T05:15:25.994 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min pgs per osd = 4 2022-09-03T05:15:25.994 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min bytes per osd = 10 2022-09-03T05:15:25.994 INFO:teuthology.orchestra.run.smithi181.stdout: mgr/telemetry/nag = false 2022-09-03T05:15:25.994 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.994 INFO:teuthology.orchestra.run.smithi181.stdout:[mon] 2022-09-03T05:15:25.995 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2022-09-03T05:15:25.995 INFO:teuthology.orchestra.run.smithi181.stdout: debug mon = 20 2022-09-03T05:15:25.995 INFO:teuthology.orchestra.run.smithi181.stdout: debug paxos = 20 2022-09-03T05:15:25.995 INFO:teuthology.orchestra.run.smithi181.stdout: debug auth = 20 2022-09-03T05:15:25.996 INFO:teuthology.orchestra.run.smithi181.stdout: mon data avail warn = 5 2022-09-03T05:15:25.996 INFO:teuthology.orchestra.run.smithi181.stdout: mon mgr mkfs grace = 240 2022-09-03T05:15:25.996 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min pgs per osd = 4 2022-09-03T05:15:25.996 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd reporter subtree level = osd 2022-09-03T05:15:25.996 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd prime pg temp = true 2022-09-03T05:15:25.997 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min bytes per osd = 10 2022-09-03T05:15:25.997 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.997 INFO:teuthology.orchestra.run.smithi181.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-09-03T05:15:25.997 INFO:teuthology.orchestra.run.smithi181.stdout: auth mon ticket ttl = 660 # 11m 2022-09-03T05:15:25.998 INFO:teuthology.orchestra.run.smithi181.stdout: auth service ticket ttl = 240 # 4m 2022-09-03T05:15:25.998 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.998 INFO:teuthology.orchestra.run.smithi181.stdout: # don't complain about insecure global_id in the test suite 2022-09-03T05:15:25.998 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-09-03T05:15:25.999 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-09-03T05:15:25.999 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:25.999 INFO:teuthology.orchestra.run.smithi181.stdout: # 1m isn't quite enough 2022-09-03T05:15:26.000 INFO:teuthology.orchestra.run.smithi181.stdout: mon_down_mkfs_grace = 2m 2022-09-03T05:15:26.000 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:26.000 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_filestore_osds = false 2022-09-03T05:15:26.000 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:26.001 INFO:teuthology.orchestra.run.smithi181.stdout:[client] 2022-09-03T05:15:26.001 INFO:teuthology.orchestra.run.smithi181.stdout: rgw cache enabled = true 2022-09-03T05:15:26.001 INFO:teuthology.orchestra.run.smithi181.stdout: rgw enable ops log = true 2022-09-03T05:15:26.001 INFO:teuthology.orchestra.run.smithi181.stdout: rgw enable usage log = true 2022-09-03T05:15:26.001 INFO:teuthology.orchestra.run.smithi181.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-09-03T05:15:26.002 INFO:teuthology.orchestra.run.smithi181.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-09-03T05:15:26.002 INFO:teuthology.orchestra.run.smithi181.stdout: debug rgw = 20 2022-09-03T05:15:26.002 INFO:teuthology.orchestra.run.smithi181.stdout: debug rgw sync = 20 2022-09-03T05:15:26.002 INFO:teuthology.orchestra.run.smithi181.stdout: rgw crypt require ssl = False 2022-09-03T05:15:26.003 INFO:teuthology.orchestra.run.smithi181.stdout: rgw crypt s3 kms backend = testing 2022-09-03T05:15:26.003 INFO:teuthology.orchestra.run.smithi181.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2022-09-03T05:15:26.003 INFO:teuthology.orchestra.run.smithi181.stdout: rgw curl low speed time = 300 2022-09-03T05:15:26.003 INFO:teuthology.orchestra.run.smithi181.stdout: rgw data log num shards = 4 2022-09-03T05:15:26.004 INFO:teuthology.orchestra.run.smithi181.stdout: rgw md log max shards = 4 2022-09-03T05:15:26.004 INFO:teuthology.orchestra.run.smithi181.stdout: rgw sync data inject err probability = 0.1 2022-09-03T05:15:26.004 INFO:teuthology.orchestra.run.smithi181.stdout: rgw sync log trim interval = 0 2022-09-03T05:15:26.004 INFO:teuthology.orchestra.run.smithi181.stdout: rgw sync meta inject err probability = 0.1 2022-09-03T05:15:26.004 INFO:teuthology.orchestra.run.smithi181.stdout: rgw sync obj etag verify = True 2022-09-03T05:15:26.005 INFO:teuthology.orchestra.run.smithi181.stdout: setgroup = ceph 2022-09-03T05:15:26.005 INFO:teuthology.orchestra.run.smithi181.stdout: setuser = ceph 2022-09-03T05:15:26.005 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.a] 2022-09-03T05:15:26.005 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2022-09-03T05:15:26.006 INFO:teuthology.orchestra.run.smithi181.stdout: rgw data notify interval msec = 0 2022-09-03T05:15:26.006 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.0] 2022-09-03T05:15:26.006 INFO:teuthology.orchestra.run.smithi181.stdout: osd_max_omap_entries_per_request = 10 2022-09-03T05:15:26.006 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.1] 2022-09-03T05:15:26.006 INFO:teuthology.orchestra.run.smithi181.stdout: osd_max_omap_entries_per_request = 1000 2022-09-03T05:15:26.007 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.2] 2022-09-03T05:15:26.007 INFO:teuthology.orchestra.run.smithi181.stdout: osd_max_omap_entries_per_request = 10000 2022-09-03T05:15:26.009 INFO:teuthology.orchestra.run.smithi189.stdout:[global] 2022-09-03T05:15:26.010 INFO:teuthology.orchestra.run.smithi189.stdout: chdir = "" 2022-09-03T05:15:26.010 INFO:teuthology.orchestra.run.smithi189.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-09-03T05:15:26.010 INFO:teuthology.orchestra.run.smithi189.stdout: auth supported = cephx 2022-09-03T05:15:26.010 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.011 INFO:teuthology.orchestra.run.smithi189.stdout: filestore xattr use omap = true 2022-09-03T05:15:26.011 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.011 INFO:teuthology.orchestra.run.smithi189.stdout: mon clock drift allowed = 1.000 2022-09-03T05:15:26.011 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.012 INFO:teuthology.orchestra.run.smithi189.stdout: osd crush chooseleaf type = 0 2022-09-03T05:15:26.012 INFO:teuthology.orchestra.run.smithi189.stdout: auth debug = true 2022-09-03T05:15:26.012 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.012 INFO:teuthology.orchestra.run.smithi189.stdout: ms die on old message = true 2022-09-03T05:15:26.013 INFO:teuthology.orchestra.run.smithi189.stdout: ms die on bug = true 2022-09-03T05:15:26.018 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.018 INFO:teuthology.orchestra.run.smithi189.stdout: mon max pg per osd = 10000 # >= luminous 2022-09-03T05:15:26.018 INFO:teuthology.orchestra.run.smithi189.stdout: mon pg warn max object skew = 0 2022-09-03T05:15:26.019 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.019 INFO:teuthology.orchestra.run.smithi189.stdout: # disable pg_autoscaler by default for new pools 2022-09-03T05:15:26.020 INFO:teuthology.orchestra.run.smithi189.stdout: osd_pool_default_pg_autoscale_mode = off 2022-09-03T05:15:26.020 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.020 INFO:teuthology.orchestra.run.smithi189.stdout: osd pool default size = 2 2022-09-03T05:15:26.020 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.021 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd allow primary affinity = true 2022-09-03T05:15:26.021 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd allow pg remap = true 2022-09-03T05:15:26.021 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on legacy crush tunables = false 2022-09-03T05:15:26.021 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on crush straw calc version zero = false 2022-09-03T05:15:26.021 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on no sortbitwise = false 2022-09-03T05:15:26.022 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on osd down out interval zero = false 2022-09-03T05:15:26.022 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on too few osds = false 2022-09-03T05:15:26.022 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-09-03T05:15:26.022 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_pool_no_redundancy = false 2022-09-03T05:15:26.023 INFO:teuthology.orchestra.run.smithi189.stdout: mon_allow_pool_size_one = true 2022-09-03T05:15:26.023 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.023 INFO:teuthology.orchestra.run.smithi189.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-09-03T05:15:26.024 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.024 INFO:teuthology.orchestra.run.smithi189.stdout: osd default data pool replay window = 5 2022-09-03T05:15:26.024 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.025 INFO:teuthology.orchestra.run.smithi189.stdout: mon allow pool delete = true 2022-09-03T05:15:26.025 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.025 INFO:teuthology.orchestra.run.smithi189.stdout: mon cluster log file level = debug 2022-09-03T05:15:26.025 INFO:teuthology.orchestra.run.smithi189.stdout: debug asserts on shutdown = true 2022-09-03T05:15:26.025 INFO:teuthology.orchestra.run.smithi189.stdout: mon health detail to clog = false 2022-09-03T05:15:26.026 INFO:teuthology.orchestra.run.smithi189.stdout: mon host = 172.21.15.181 2022-09-03T05:15:26.026 INFO:teuthology.orchestra.run.smithi189.stdout: fsid = 1b22fd20-d842-48ea-bdcc-6e548483fbc3 2022-09-03T05:15:26.026 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.026 INFO:teuthology.orchestra.run.smithi189.stdout:[osd] 2022-09-03T05:15:26.027 INFO:teuthology.orchestra.run.smithi189.stdout: osd journal size = 100 2022-09-03T05:15:26.027 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.027 INFO:teuthology.orchestra.run.smithi189.stdout: osd scrub load threshold = 5.0 2022-09-03T05:15:26.027 INFO:teuthology.orchestra.run.smithi189.stdout: osd scrub max interval = 600 2022-09-03T05:15:26.027 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.028 INFO:teuthology.orchestra.run.smithi189.stdout: osd recover clone overlap = true 2022-09-03T05:15:26.028 INFO:teuthology.orchestra.run.smithi189.stdout: osd recovery max chunk = 1048576 2022-09-03T05:15:26.028 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.028 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug shutdown = true 2022-09-03T05:15:26.028 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug op order = true 2022-09-03T05:15:26.029 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug verify stray on activate = true 2022-09-03T05:15:26.029 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.029 INFO:teuthology.orchestra.run.smithi189.stdout: osd open classes on start = true 2022-09-03T05:15:26.029 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug pg log writeout = true 2022-09-03T05:15:26.030 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.030 INFO:teuthology.orchestra.run.smithi189.stdout: osd deep scrub update digest min age = 30 2022-09-03T05:15:26.031 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.031 INFO:teuthology.orchestra.run.smithi189.stdout: osd map max advance = 10 2022-09-03T05:15:26.031 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.031 INFO:teuthology.orchestra.run.smithi189.stdout: journal zero on create = true 2022-09-03T05:15:26.031 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.032 INFO:teuthology.orchestra.run.smithi189.stdout: filestore ondisk finisher threads = 3 2022-09-03T05:15:26.032 INFO:teuthology.orchestra.run.smithi189.stdout: filestore apply finisher threads = 3 2022-09-03T05:15:26.032 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.032 INFO:teuthology.orchestra.run.smithi189.stdout: bdev debug aio = true 2022-09-03T05:15:26.033 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug misdirected ops = true 2022-09-03T05:15:26.033 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2022-09-03T05:15:26.033 INFO:teuthology.orchestra.run.smithi189.stdout: debug osd = 20 2022-09-03T05:15:26.033 INFO:teuthology.orchestra.run.smithi189.stdout: rocksdb delete range threshold = 0 2022-09-03T05:15:26.033 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.034 INFO:teuthology.orchestra.run.smithi189.stdout:[mgr] 2022-09-03T05:15:26.035 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2022-09-03T05:15:26.035 INFO:teuthology.orchestra.run.smithi189.stdout: debug mgr = 20 2022-09-03T05:15:26.035 INFO:teuthology.orchestra.run.smithi189.stdout: debug mon = 20 2022-09-03T05:15:26.035 INFO:teuthology.orchestra.run.smithi189.stdout: debug auth = 20 2022-09-03T05:15:26.036 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min pgs per osd = 4 2022-09-03T05:15:26.036 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min bytes per osd = 10 2022-09-03T05:15:26.036 INFO:teuthology.orchestra.run.smithi189.stdout: mgr/telemetry/nag = false 2022-09-03T05:15:26.036 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.036 INFO:teuthology.orchestra.run.smithi189.stdout:[mon] 2022-09-03T05:15:26.037 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2022-09-03T05:15:26.037 INFO:teuthology.orchestra.run.smithi189.stdout: debug mon = 20 2022-09-03T05:15:26.037 INFO:teuthology.orchestra.run.smithi189.stdout: debug paxos = 20 2022-09-03T05:15:26.037 INFO:teuthology.orchestra.run.smithi189.stdout: debug auth = 20 2022-09-03T05:15:26.038 INFO:teuthology.orchestra.run.smithi189.stdout: mon data avail warn = 5 2022-09-03T05:15:26.038 INFO:teuthology.orchestra.run.smithi189.stdout: mon mgr mkfs grace = 240 2022-09-03T05:15:26.038 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min pgs per osd = 4 2022-09-03T05:15:26.038 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd reporter subtree level = osd 2022-09-03T05:15:26.039 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd prime pg temp = true 2022-09-03T05:15:26.039 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min bytes per osd = 10 2022-09-03T05:15:26.039 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.039 INFO:teuthology.orchestra.run.smithi189.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-09-03T05:15:26.040 INFO:teuthology.orchestra.run.smithi189.stdout: auth mon ticket ttl = 660 # 11m 2022-09-03T05:15:26.040 INFO:teuthology.orchestra.run.smithi189.stdout: auth service ticket ttl = 240 # 4m 2022-09-03T05:15:26.040 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.041 INFO:teuthology.orchestra.run.smithi189.stdout: # don't complain about insecure global_id in the test suite 2022-09-03T05:15:26.041 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-09-03T05:15:26.041 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-09-03T05:15:26.041 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.041 INFO:teuthology.orchestra.run.smithi189.stdout: # 1m isn't quite enough 2022-09-03T05:15:26.042 INFO:teuthology.orchestra.run.smithi189.stdout: mon_down_mkfs_grace = 2m 2022-09-03T05:15:26.043 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.043 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_filestore_osds = false 2022-09-03T05:15:26.043 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:15:26.043 INFO:teuthology.orchestra.run.smithi189.stdout:[client] 2022-09-03T05:15:26.043 INFO:teuthology.orchestra.run.smithi189.stdout: rgw cache enabled = true 2022-09-03T05:15:26.044 INFO:teuthology.orchestra.run.smithi189.stdout: rgw enable ops log = true 2022-09-03T05:15:26.044 INFO:teuthology.orchestra.run.smithi189.stdout: rgw enable usage log = true 2022-09-03T05:15:26.044 INFO:teuthology.orchestra.run.smithi189.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-09-03T05:15:26.045 INFO:teuthology.orchestra.run.smithi189.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-09-03T05:15:26.045 INFO:teuthology.orchestra.run.smithi189.stdout: debug rgw = 20 2022-09-03T05:15:26.046 INFO:teuthology.orchestra.run.smithi189.stdout: debug rgw sync = 20 2022-09-03T05:15:26.046 INFO:teuthology.orchestra.run.smithi189.stdout: rgw crypt require ssl = False 2022-09-03T05:15:26.047 INFO:teuthology.orchestra.run.smithi189.stdout: rgw crypt s3 kms backend = testing 2022-09-03T05:15:26.047 INFO:teuthology.orchestra.run.smithi189.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2022-09-03T05:15:26.047 INFO:teuthology.orchestra.run.smithi189.stdout: rgw curl low speed time = 300 2022-09-03T05:15:26.048 INFO:teuthology.orchestra.run.smithi189.stdout: rgw data log num shards = 4 2022-09-03T05:15:26.048 INFO:teuthology.orchestra.run.smithi189.stdout: rgw md log max shards = 4 2022-09-03T05:15:26.048 INFO:teuthology.orchestra.run.smithi189.stdout: rgw sync data inject err probability = 0.1 2022-09-03T05:15:26.048 INFO:teuthology.orchestra.run.smithi189.stdout: rgw sync log trim interval = 0 2022-09-03T05:15:26.049 INFO:teuthology.orchestra.run.smithi189.stdout: rgw sync meta inject err probability = 0.1 2022-09-03T05:15:26.049 INFO:teuthology.orchestra.run.smithi189.stdout: rgw sync obj etag verify = True 2022-09-03T05:15:26.050 INFO:teuthology.orchestra.run.smithi189.stdout: setgroup = ceph 2022-09-03T05:15:26.050 INFO:teuthology.orchestra.run.smithi189.stdout: setuser = ceph 2022-09-03T05:15:26.050 INFO:teuthology.orchestra.run.smithi189.stdout:[mon.a] 2022-09-03T05:15:26.051 INFO:teuthology.orchestra.run.smithi189.stdout:[client.0] 2022-09-03T05:15:26.052 INFO:teuthology.orchestra.run.smithi189.stdout: rgw data notify interval msec = 0 2022-09-03T05:15:26.052 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.0] 2022-09-03T05:15:26.053 INFO:teuthology.orchestra.run.smithi189.stdout: osd_max_omap_entries_per_request = 10 2022-09-03T05:15:26.053 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.1] 2022-09-03T05:15:26.054 INFO:teuthology.orchestra.run.smithi189.stdout: osd_max_omap_entries_per_request = 1000 2022-09-03T05:15:26.055 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.2] 2022-09-03T05:15:26.056 INFO:teuthology.orchestra.run.smithi189.stdout: osd_max_omap_entries_per_request = 10000 2022-09-03T05:15:26.060 INFO:tasks.ceph:Creating admin key on c1.mon.a... 2022-09-03T05:15:26.061 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/c1.keyring 2022-09-03T05:15:26.121 INFO:tasks.ceph:Copying monmap to all nodes... 2022-09-03T05:15:26.123 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2022-09-03T05:15:26.124 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/etc/ceph/c1.keyring of=/dev/stdout 2022-09-03T05:15:26.146 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2022-09-03T05:15:26.147 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/home/ubuntu/cephtest/c1.monmap of=/dev/stdout 2022-09-03T05:15:26.210 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi181.front.sepia.ceph.com 2022-09-03T05:15:26.210 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2022-09-03T05:15:26.211 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/c1.keyring 2022-09-03T05:15:26.211 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0644 /etc/ceph/c1.keyring 2022-09-03T05:15:26.293 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2022-09-03T05:15:26.293 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/home/ubuntu/cephtest/c1.monmap 2022-09-03T05:15:26.352 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi189.front.sepia.ceph.com 2022-09-03T05:15:26.353 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2022-09-03T05:15:26.354 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/c1.keyring 2022-09-03T05:15:26.354 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 0644 /etc/ceph/c1.keyring 2022-09-03T05:15:26.394 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2022-09-03T05:15:26.395 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/home/ubuntu/cephtest/c1.monmap 2022-09-03T05:15:26.458 INFO:tasks.ceph:Setting up mon nodes... 2022-09-03T05:15:26.459 INFO:tasks.ceph:Setting up mgr nodes... 2022-09-03T05:15:26.460 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mgr/c1-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/c1-x/keyring 2022-09-03T05:15:26.518 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mgr/c1-x/keyring 2022-09-03T05:15:26.519 INFO:tasks.ceph:Setting up mds nodes... 2022-09-03T05:15:26.519 INFO:tasks.ceph_client:Setting up client nodes... 2022-09-03T05:15:26.520 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/c1.client.0.keyring && sudo chmod 0644 /etc/ceph/c1.client.0.keyring 2022-09-03T05:15:26.563 INFO:teuthology.orchestra.run.smithi181.stdout:creating /etc/ceph/c1.client.0.keyring 2022-09-03T05:15:26.580 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/c1.client.1.keyring && sudo chmod 0644 /etc/ceph/c1.client.1.keyring 2022-09-03T05:15:26.659 INFO:teuthology.orchestra.run.smithi181.stdout:creating /etc/ceph/c1.client.1.keyring 2022-09-03T05:15:26.672 INFO:tasks.ceph:Running mkfs on osd nodes... 2022-09-03T05:15:26.673 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi181.front.sepia.ceph.com'): {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'}} 2022-09-03T05:15:26.674 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/c1-0 2022-09-03T05:15:26.739 INFO:tasks.ceph:roles_to_devs: {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'} 2022-09-03T05:15:26.739 INFO:tasks.ceph:role: c1.osd.0 2022-09-03T05:15:26.740 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi181.front.sepia.ceph.com 2022-09-03T05:15:26.740 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-09-03T05:15:26.821 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-09-03T05:15:26.821 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2022-09-03T05:15:26.822 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-09-03T05:15:26.822 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-09-03T05:15:26.822 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-09-03T05:15:26.823 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2022-09-03T05:15:26.823 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-09-03T05:15:26.823 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-09-03T05:15:26.823 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-09-03T05:15:26.824 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-09-03T05:15:27.244 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2022-09-03T05:15:27.246 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2022-09-03T05:15:27.246 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/c1-0 2022-09-03T05:15:27.441 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/c1-0 2022-09-03T05:15:27.479 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/c1-1 2022-09-03T05:15:27.547 INFO:tasks.ceph:roles_to_devs: {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'} 2022-09-03T05:15:27.547 INFO:tasks.ceph:role: c1.osd.1 2022-09-03T05:15:27.548 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi181.front.sepia.ceph.com 2022-09-03T05:15:27.549 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-09-03T05:15:27.642 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-09-03T05:15:27.643 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2022-09-03T05:15:27.644 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-09-03T05:15:27.644 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-09-03T05:15:27.645 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-09-03T05:15:27.645 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2022-09-03T05:15:27.645 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-09-03T05:15:27.646 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-09-03T05:15:27.646 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-09-03T05:15:27.647 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-09-03T05:15:28.042 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2022-09-03T05:15:28.044 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2022-09-03T05:15:28.045 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/c1-1 2022-09-03T05:15:28.156 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/c1-1 2022-09-03T05:15:28.188 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/c1-2 2022-09-03T05:15:28.255 INFO:tasks.ceph:roles_to_devs: {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'} 2022-09-03T05:15:28.256 INFO:tasks.ceph:role: c1.osd.2 2022-09-03T05:15:28.257 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi181.front.sepia.ceph.com 2022-09-03T05:15:28.257 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-09-03T05:15:28.338 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-09-03T05:15:28.338 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2022-09-03T05:15:28.339 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-09-03T05:15:28.339 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-09-03T05:15:28.339 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-09-03T05:15:28.339 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2022-09-03T05:15:28.340 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-09-03T05:15:28.340 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-09-03T05:15:28.340 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-09-03T05:15:28.340 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-09-03T05:15:28.783 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2022-09-03T05:15:28.784 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2022-09-03T05:15:28.784 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/c1-2 2022-09-03T05:15:28.953 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/c1-2 2022-09-03T05:15:28.986 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c1 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/c1.monmap 2022-09-03T05:15:29.084 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:15:29.079+0000 7f8cac4cd440 -1 auth: error reading file: /var/lib/ceph/osd/c1-0/keyring: can't open /var/lib/ceph/osd/c1-0/keyring: (2) No such file or directory 2022-09-03T05:15:29.087 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:15:29.082+0000 7f8cac4cd440 -1 created new key in keyring /var/lib/ceph/osd/c1-0/keyring 2022-09-03T05:15:29.087 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:15:29.082+0000 7f8cac4cd440 -1 bluestore(/var/lib/ceph/osd/c1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-0/block: (2) No such file or directory 2022-09-03T05:15:29.088 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:15:29.082+0000 7f8cac4cd440 -1 bluestore(/var/lib/ceph/osd/c1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-0/block: (2) No such file or directory 2022-09-03T05:15:29.088 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:15:29.082+0000 7f8cac4cd440 -1 bluestore(/var/lib/ceph/osd/c1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-0/block: (2) No such file or directory 2022-09-03T05:15:29.089 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:15:29.083+0000 7f8cac4cd440 -1 bluestore(/var/lib/ceph/osd/c1-0) _read_fsid unparsable uuid 2022-09-03T05:15:31.153 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c1-0 2022-09-03T05:15:31.184 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c1 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/c1.monmap 2022-09-03T05:15:31.273 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:15:31.275+0000 7fe47b110440 -1 auth: error reading file: /var/lib/ceph/osd/c1-1/keyring: can't open /var/lib/ceph/osd/c1-1/keyring: (2) No such file or directory 2022-09-03T05:15:31.273 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:15:31.275+0000 7fe47b110440 -1 created new key in keyring /var/lib/ceph/osd/c1-1/keyring 2022-09-03T05:15:31.274 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:15:31.275+0000 7fe47b110440 -1 bluestore(/var/lib/ceph/osd/c1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-1/block: (2) No such file or directory 2022-09-03T05:15:31.274 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:15:31.275+0000 7fe47b110440 -1 bluestore(/var/lib/ceph/osd/c1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-1/block: (2) No such file or directory 2022-09-03T05:15:31.274 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:15:31.275+0000 7fe47b110440 -1 bluestore(/var/lib/ceph/osd/c1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-1/block: (2) No such file or directory 2022-09-03T05:15:31.274 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:15:31.276+0000 7fe47b110440 -1 bluestore(/var/lib/ceph/osd/c1-1) _read_fsid unparsable uuid 2022-09-03T05:15:33.354 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c1-1 2022-09-03T05:15:33.386 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c1 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/c1.monmap 2022-09-03T05:15:33.474 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:15:33.477+0000 7f27701e1440 -1 auth: error reading file: /var/lib/ceph/osd/c1-2/keyring: can't open /var/lib/ceph/osd/c1-2/keyring: (2) No such file or directory 2022-09-03T05:15:33.475 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:15:33.477+0000 7f27701e1440 -1 created new key in keyring /var/lib/ceph/osd/c1-2/keyring 2022-09-03T05:15:33.475 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:15:33.477+0000 7f27701e1440 -1 bluestore(/var/lib/ceph/osd/c1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-2/block: (2) No such file or directory 2022-09-03T05:15:33.475 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:15:33.477+0000 7f27701e1440 -1 bluestore(/var/lib/ceph/osd/c1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-2/block: (2) No such file or directory 2022-09-03T05:15:33.476 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:15:33.477+0000 7f27701e1440 -1 bluestore(/var/lib/ceph/osd/c1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c1-2/block: (2) No such file or directory 2022-09-03T05:15:33.476 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:15:33.477+0000 7f27701e1440 -1 bluestore(/var/lib/ceph/osd/c1-2) _read_fsid unparsable uuid 2022-09-03T05:15:35.543 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c1-2 2022-09-03T05:15:35.572 INFO:tasks.ceph:Reading keys from all nodes... 2022-09-03T05:15:35.573 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2022-09-03T05:15:35.574 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mgr/c1-x/keyring of=/dev/stdout 2022-09-03T05:15:35.642 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2022-09-03T05:15:35.643 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/c1-0/keyring of=/dev/stdout 2022-09-03T05:15:35.708 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2022-09-03T05:15:35.708 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/c1-1/keyring of=/dev/stdout 2022-09-03T05:15:35.776 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2022-09-03T05:15:35.777 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/c1-2/keyring of=/dev/stdout 2022-09-03T05:15:35.845 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2022-09-03T05:15:35.845 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/etc/ceph/c1.client.0.keyring of=/dev/stdout 2022-09-03T05:15:35.907 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2022-09-03T05:15:35.908 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/etc/ceph/c1.client.1.keyring of=/dev/stdout 2022-09-03T05:15:35.965 INFO:tasks.ceph:Adding keys to all mons... 2022-09-03T05:15:35.966 DEBUG:teuthology.orchestra.run.smithi181:> sudo tee -a /etc/ceph/c1.keyring 2022-09-03T05:15:36.037 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr.x] 2022-09-03T05:15:36.038 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDu4hJjmlLnHhAAr84+NAcVLATEULAW1FgNKw== 2022-09-03T05:15:36.039 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.0] 2022-09-03T05:15:36.040 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDx4hJjd6HJBBAA2rmx7zh+JlmRrkUwSI0u8g== 2022-09-03T05:15:36.041 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.1] 2022-09-03T05:15:36.041 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDz4hJj6odwEBAAjhHhn0plDYyOkUGgm7AkOw== 2022-09-03T05:15:36.041 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.2] 2022-09-03T05:15:36.042 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQD14hJjKYV4HBAAun4Al5aI6cq4kRq1I0x7Vg== 2022-09-03T05:15:36.042 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2022-09-03T05:15:36.043 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDu4hJjedvLIRAAH8qN4/JS8eEJLkTcRgdxiw== 2022-09-03T05:15:36.044 INFO:teuthology.orchestra.run.smithi181.stdout:[client.1] 2022-09-03T05:15:36.044 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQDu4hJjjyV/JxAAIbbXqps2xoqNmXhKw/+qVg== 2022-09-03T05:15:36.047 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-09-03T05:15:36.146 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-09-03T05:15:36.203 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-09-03T05:15:36.256 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-09-03T05:15:36.312 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-09-03T05:15:36.371 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c1.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-09-03T05:15:36.433 INFO:tasks.ceph:Running mkfs on mon nodes... 2022-09-03T05:15:36.434 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mon/c1-a 2022-09-03T05:15:36.464 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster c1 --mkfs -i a --monmap /home/ubuntu/cephtest/c1.monmap --keyring /etc/ceph/c1.keyring 2022-09-03T05:15:36.561 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mon/c1-a 2022-09-03T05:15:36.591 DEBUG:teuthology.orchestra.run.smithi181:> rm -- /home/ubuntu/cephtest/c1.monmap 2022-09-03T05:15:36.650 INFO:tasks.ceph:Starting mon daemons in cluster c1... 2022-09-03T05:15:36.651 INFO:tasks.ceph.c1.mon.a:Restarting daemon 2022-09-03T05:15:36.652 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster c1 -i a 2022-09-03T05:15:36.693 INFO:tasks.ceph.c1.mon.a:Started 2022-09-03T05:15:36.694 INFO:tasks.ceph:Starting mgr daemons in cluster c1... 2022-09-03T05:15:36.695 INFO:tasks.ceph.c1.mgr.x:Restarting daemon 2022-09-03T05:15:36.696 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster c1 -i x 2022-09-03T05:15:36.698 INFO:tasks.ceph.c1.mgr.x:Started 2022-09-03T05:15:36.699 INFO:tasks.ceph:Setting crush tunables to default 2022-09-03T05:15:36.699 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster c1 osd crush tunables default 2022-09-03T05:15:36.902 INFO:teuthology.orchestra.run.smithi181.stderr:adjusted tunables profile to default 2022-09-03T05:15:36.912 INFO:tasks.ceph:Starting osd daemons in cluster c1... 2022-09-03T05:15:36.913 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2022-09-03T05:15:36.913 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/c1-0/fsid of=/dev/stdout 2022-09-03T05:15:36.942 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2022-09-03T05:15:36.943 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/c1-1/fsid of=/dev/stdout 2022-09-03T05:15:37.009 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2022-09-03T05:15:37.010 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/c1-2/fsid of=/dev/stdout 2022-09-03T05:15:37.078 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster c1 osd new 367873db-887a-4b6d-bb67-c7212da52e73 0 2022-09-03T05:15:37.330 INFO:teuthology.orchestra.run.smithi181.stdout:0 2022-09-03T05:15:37.343 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster c1 osd new 03fde9d4-1515-40cc-9ae1-8be880cfeb2f 1 2022-09-03T05:15:37.556 INFO:teuthology.orchestra.run.smithi181.stdout:1 2022-09-03T05:15:37.569 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster c1 osd new f0e7b6ab-4ec1-4b97-8a9f-2f0250a1279c 2 2022-09-03T05:15:37.780 INFO:teuthology.orchestra.run.smithi181.stdout:2 2022-09-03T05:15:37.795 INFO:tasks.ceph.c1.osd.0:Restarting daemon 2022-09-03T05:15:37.796 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c1 -i 0 2022-09-03T05:15:37.798 INFO:tasks.ceph.c1.osd.0:Started 2022-09-03T05:15:37.799 INFO:tasks.ceph.c1.osd.1:Restarting daemon 2022-09-03T05:15:37.799 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c1 -i 1 2022-09-03T05:15:37.801 INFO:tasks.ceph.c1.osd.1:Started 2022-09-03T05:15:37.802 INFO:tasks.ceph.c1.osd.2:Restarting daemon 2022-09-03T05:15:37.802 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c1 -i 2 2022-09-03T05:15:37.804 INFO:tasks.ceph.c1.osd.2:Started 2022-09-03T05:15:37.805 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd dump --format=json 2022-09-03T05:15:37.949 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:37.951+0000 7f4ca8e5f080 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2022-09-03T05:15:38.028 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:38.031+0000 7f4ca8e5f080 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2022-09-03T05:15:38.047 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:38.048 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":5,"fsid":"1b22fd20-d842-48ea-bdcc-6e548483fbc3","created":"2022-09-03T05:15:36.812798+0000","modified":"2022-09-03T05:15:37.782081+0000","last_up_change":"0.000000","last_in_change":"2022-09-03T05:15:37.782081+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"367873db-887a-4b6d-bb67-c7212da52e73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"03fde9d4-1515-40cc-9ae1-8be880cfeb2f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f0e7b6ab-4ec1-4b97-8a9f-2f0250a1279c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-09-03T05:15:38.060 INFO:tasks.ceph.ceph_manager.c1:[] 2022-09-03T05:15:38.061 INFO:tasks.ceph:Waiting for OSDs to come up 2022-09-03T05:15:38.112 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:38.115+0000 7f4ca8e5f080 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2022-09-03T05:15:38.297 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:38.297+0000 7f4ca8e5f080 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2022-09-03T05:15:38.365 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c1 osd dump --format=json 2022-09-03T05:15:38.418 INFO:tasks.ceph.c1.osd.1.smithi181.stderr:2022-09-03T05:15:38.421+0000 7fb39ed3f440 -1 Falling back to public interface 2022-09-03T05:15:38.419 INFO:tasks.ceph.c1.osd.0.smithi181.stderr:2022-09-03T05:15:38.421+0000 7f9170b2a440 -1 Falling back to public interface 2022-09-03T05:15:38.426 INFO:tasks.ceph.c1.osd.2.smithi181.stderr:2022-09-03T05:15:38.426+0000 7fcbbb416440 -1 Falling back to public interface 2022-09-03T05:15:38.560 INFO:teuthology.misc.health.smithi181.stdout: 2022-09-03T05:15:38.561 INFO:teuthology.misc.health.smithi181.stdout:{"epoch":5,"fsid":"1b22fd20-d842-48ea-bdcc-6e548483fbc3","created":"2022-09-03T05:15:36.812798+0000","modified":"2022-09-03T05:15:37.782081+0000","last_up_change":"0.000000","last_in_change":"2022-09-03T05:15:37.782081+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"367873db-887a-4b6d-bb67-c7212da52e73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"03fde9d4-1515-40cc-9ae1-8be880cfeb2f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f0e7b6ab-4ec1-4b97-8a9f-2f0250a1279c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-09-03T05:15:38.571 DEBUG:teuthology.misc:0 of 3 OSDs are up 2022-09-03T05:15:39.084 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:39.086+0000 7f4ca8e5f080 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2022-09-03T05:15:39.221 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:39.223+0000 7f4ca8e5f080 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2022-09-03T05:15:39.443 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:39.445+0000 7f4ca8e5f080 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2022-09-03T05:15:39.512 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:39.515+0000 7f4ca8e5f080 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2022-09-03T05:15:39.647 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:39.650+0000 7f4ca8e5f080 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2022-09-03T05:15:39.907 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:39.910+0000 7f4ca8e5f080 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2022-09-03T05:15:40.092 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:40.094+0000 7f4ca8e5f080 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2022-09-03T05:15:40.226 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:40.229+0000 7f4ca8e5f080 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2022-09-03T05:15:40.344 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:40.343+0000 7f4ca8e5f080 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2022-09-03T05:15:40.479 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:40.481+0000 7f4ca8e5f080 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2022-09-03T05:15:40.563 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:40.559+0000 7f4ca8e5f080 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2022-09-03T05:15:40.775 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:40.777+0000 7f4ca8e5f080 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2022-09-03T05:15:40.855 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:40.854+0000 7f4ca8e5f080 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2022-09-03T05:15:40.897 INFO:tasks.ceph.c1.osd.2.smithi181.stderr:2022-09-03T05:15:40.899+0000 7fcbbb416440 -1 osd.2 0 log_to_monitors true 2022-09-03T05:15:41.452 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:41.455+0000 7f4ca8e5f080 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2022-09-03T05:15:41.714 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:41.714+0000 7f4ca8e5f080 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2022-09-03T05:15:41.784 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:41.787+0000 7f4ca8e5f080 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2022-09-03T05:15:41.865 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:41.867+0000 7f4ca8e5f080 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2022-09-03T05:15:42.056 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:42.058+0000 7f4ca8e5f080 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2022-09-03T05:15:42.130 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:42.133+0000 7f4ca8e5f080 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2022-09-03T05:15:42.289 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:42.291+0000 7f4ca8e5f080 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2022-09-03T05:15:42.449 INFO:tasks.ceph.c1.osd.2.smithi181.stderr:2022-09-03T05:15:42.450+0000 7fcbaf1c3700 -1 osd.2 0 waiting for initial osdmap 2022-09-03T05:15:42.467 INFO:tasks.ceph.c1.osd.2.smithi181.stderr:2022-09-03T05:15:42.460+0000 7fcba7d9f700 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-09-03T05:15:43.327 INFO:tasks.ceph.c1.mgr.x.smithi181.stderr:2022-09-03T05:15:43.328+0000 7f4c62c3f700 -1 mgr.server handle_report got status from non-daemon mon.a 2022-09-03T05:15:44.875 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c1 osd dump --format=json 2022-09-03T05:15:45.184 INFO:teuthology.misc.health.smithi181.stdout: 2022-09-03T05:15:45.184 INFO:teuthology.misc.health.smithi181.stdout:{"epoch":9,"fsid":"1b22fd20-d842-48ea-bdcc-6e548483fbc3","created":"2022-09-03T05:15:36.812798+0000","modified":"2022-09-03T05:15:44.449550+0000","last_up_change":"2022-09-03T05:15:43.448177+0000","last_in_change":"2022-09-03T05:15:37.782081+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"367873db-887a-4b6d-bb67-c7212da52e73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"03fde9d4-1515-40cc-9ae1-8be880cfeb2f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f0e7b6ab-4ec1-4b97-8a9f-2f0250a1279c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":104072},{"type":"v1","addr":"172.21.15.181:6817","nonce":104072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":104072},{"type":"v1","addr":"172.21.15.181:6819","nonce":104072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":104072},{"type":"v1","addr":"172.21.15.181:6823","nonce":104072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":104072},{"type":"v1","addr":"172.21.15.181:6821","nonce":104072}]},"public_addr":"172.21.15.181:6817/104072","cluster_addr":"172.21.15.181:6819/104072","heartbeat_back_addr":"172.21.15.181:6823/104072","heartbeat_front_addr":"172.21.15.181:6821/104072","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-03T05:15:41.911175+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-09-03T05:15:45.196 DEBUG:teuthology.misc:1 of 3 OSDs are up 2022-09-03T05:15:45.872 INFO:tasks.ceph.c1.osd.1.smithi181.stderr:2022-09-03T05:15:45.875+0000 7fb39ed3f440 -1 osd.1 0 log_to_monitors true 2022-09-03T05:15:46.113 INFO:tasks.ceph.c1.osd.0.smithi181.stderr:2022-09-03T05:15:46.114+0000 7f9170b2a440 -1 osd.0 0 log_to_monitors true 2022-09-03T05:15:48.296 INFO:tasks.ceph.c1.osd.1.smithi181.stderr:2022-09-03T05:15:48.298+0000 7fb392aec700 -1 osd.1 0 waiting for initial osdmap 2022-09-03T05:15:48.308 INFO:tasks.ceph.c1.osd.1.smithi181.stderr:2022-09-03T05:15:48.311+0000 7fb38b6c8700 -1 osd.1 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-09-03T05:15:48.309 INFO:tasks.ceph.c1.osd.0.smithi181.stderr:2022-09-03T05:15:48.311+0000 7f91648d7700 -1 osd.0 0 waiting for initial osdmap 2022-09-03T05:15:48.323 INFO:tasks.ceph.c1.osd.0.smithi181.stderr:2022-09-03T05:15:48.325+0000 7f915d4b3700 -1 osd.0 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-09-03T05:15:51.503 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c1 osd dump --format=json 2022-09-03T05:15:51.810 INFO:teuthology.misc.health.smithi181.stdout: 2022-09-03T05:15:51.810 INFO:teuthology.misc.health.smithi181.stdout:{"epoch":14,"fsid":"1b22fd20-d842-48ea-bdcc-6e548483fbc3","created":"2022-09-03T05:15:36.812798+0000","modified":"2022-09-03T05:15:51.460991+0000","last_up_change":"2022-09-03T05:15:49.310155+0000","last_in_change":"2022-09-03T05:15:37.782081+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-09-03T05:15:50.368822+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"367873db-887a-4b6d-bb67-c7212da52e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":104070},{"type":"v1","addr":"172.21.15.181:6809","nonce":104070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":104070},{"type":"v1","addr":"172.21.15.181:6811","nonce":104070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":104070},{"type":"v1","addr":"172.21.15.181:6815","nonce":104070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":104070},{"type":"v1","addr":"172.21.15.181:6813","nonce":104070}]},"public_addr":"172.21.15.181:6809/104070","cluster_addr":"172.21.15.181:6811/104070","heartbeat_back_addr":"172.21.15.181:6815/104070","heartbeat_front_addr":"172.21.15.181:6813/104070","state":["exists","up"]},{"osd":1,"uuid":"03fde9d4-1515-40cc-9ae1-8be880cfeb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":104071},{"type":"v1","addr":"172.21.15.181:6801","nonce":104071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":104071},{"type":"v1","addr":"172.21.15.181:6803","nonce":104071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":104071},{"type":"v1","addr":"172.21.15.181:6808","nonce":104071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":104071},{"type":"v1","addr":"172.21.15.181:6805","nonce":104071}]},"public_addr":"172.21.15.181:6801/104071","cluster_addr":"172.21.15.181:6803/104071","heartbeat_back_addr":"172.21.15.181:6808/104071","heartbeat_front_addr":"172.21.15.181:6805/104071","state":["exists","up"]},{"osd":2,"uuid":"f0e7b6ab-4ec1-4b97-8a9f-2f0250a1279c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":104072},{"type":"v1","addr":"172.21.15.181:6817","nonce":104072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":104072},{"type":"v1","addr":"172.21.15.181:6819","nonce":104072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":104072},{"type":"v1","addr":"172.21.15.181:6823","nonce":104072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":104072},{"type":"v1","addr":"172.21.15.181:6821","nonce":104072}]},"public_addr":"172.21.15.181:6817/104072","cluster_addr":"172.21.15.181:6819/104072","heartbeat_back_addr":"172.21.15.181:6823/104072","heartbeat_front_addr":"172.21.15.181:6821/104072","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-03T05:15:47.123206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-03T05:15:46.848141+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-03T05:15:41.911175+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-09-03T05:15:51.823 DEBUG:teuthology.misc:3 of 3 OSDs are up 2022-09-03T05:15:51.824 INFO:tasks.ceph:Creating RBD pool 2022-09-03T05:15:51.824 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster c1 osd pool create rbd 8 2022-09-03T05:15:52.789 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'rbd' created 2022-09-03T05:15:52.801 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster c1 osd pool application enable rbd rbd --yes-i-really-mean-it 2022-09-03T05:15:53.797 INFO:teuthology.orchestra.run.smithi181.stderr:enabled application 'rbd' on pool 'rbd' 2022-09-03T05:15:53.810 INFO:tasks.ceph:Starting mds daemons in cluster c1... 2022-09-03T05:15:53.811 INFO:tasks.ceph:Waiting until c1 daemons up and pgs clean... 2022-09-03T05:15:53.811 INFO:tasks.ceph.ceph_manager.c1:waiting for mgr available 2022-09-03T05:15:53.812 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 mgr dump --format=json 2022-09-03T05:15:53.812 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2022-09-03T05:15:54.178 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:54.188 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":5,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":103663},{"type":"v1","addr":"172.21.15.181:6825","nonce":103663}]},"active_addr":"172.21.15.181:6825/103663","active_change":"2022-09-03T05:15:42.305354+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":475737435}]},{"addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":2660084398}]},{"addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":3963137742}]},{"addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":21641186}]}]}} 2022-09-03T05:15:54.195 INFO:tasks.ceph.ceph_manager.c1:mgr available! 2022-09-03T05:15:54.196 INFO:tasks.ceph.ceph_manager.c1:waiting for all up 2022-09-03T05:15:54.197 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd dump --format=json 2022-09-03T05:15:54.515 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:54.515 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":16,"fsid":"1b22fd20-d842-48ea-bdcc-6e548483fbc3","created":"2022-09-03T05:15:36.812798+0000","modified":"2022-09-03T05:15:53.786494+0000","last_up_change":"2022-09-03T05:15:49.310155+0000","last_in_change":"2022-09-03T05:15:37.782081+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-09-03T05:15:50.368822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-09-03T05:15:52.175757+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"367873db-887a-4b6d-bb67-c7212da52e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":104070},{"type":"v1","addr":"172.21.15.181:6809","nonce":104070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":104070},{"type":"v1","addr":"172.21.15.181:6811","nonce":104070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":104070},{"type":"v1","addr":"172.21.15.181:6815","nonce":104070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":104070},{"type":"v1","addr":"172.21.15.181:6813","nonce":104070}]},"public_addr":"172.21.15.181:6809/104070","cluster_addr":"172.21.15.181:6811/104070","heartbeat_back_addr":"172.21.15.181:6815/104070","heartbeat_front_addr":"172.21.15.181:6813/104070","state":["exists","up"]},{"osd":1,"uuid":"03fde9d4-1515-40cc-9ae1-8be880cfeb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":104071},{"type":"v1","addr":"172.21.15.181:6801","nonce":104071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":104071},{"type":"v1","addr":"172.21.15.181:6803","nonce":104071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":104071},{"type":"v1","addr":"172.21.15.181:6808","nonce":104071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":104071},{"type":"v1","addr":"172.21.15.181:6805","nonce":104071}]},"public_addr":"172.21.15.181:6801/104071","cluster_addr":"172.21.15.181:6803/104071","heartbeat_back_addr":"172.21.15.181:6808/104071","heartbeat_front_addr":"172.21.15.181:6805/104071","state":["exists","up"]},{"osd":2,"uuid":"f0e7b6ab-4ec1-4b97-8a9f-2f0250a1279c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":104072},{"type":"v1","addr":"172.21.15.181:6817","nonce":104072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":104072},{"type":"v1","addr":"172.21.15.181:6819","nonce":104072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":104072},{"type":"v1","addr":"172.21.15.181:6823","nonce":104072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":104072},{"type":"v1","addr":"172.21.15.181:6821","nonce":104072}]},"public_addr":"172.21.15.181:6817/104072","cluster_addr":"172.21.15.181:6819/104072","heartbeat_back_addr":"172.21.15.181:6823/104072","heartbeat_front_addr":"172.21.15.181:6821/104072","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-03T05:15:47.123206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-03T05:15:46.848141+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-03T05:15:41.911175+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-09-03T05:15:54.527 INFO:tasks.ceph.ceph_manager.c1:all up! 2022-09-03T05:15:54.528 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd dump --format=json 2022-09-03T05:15:54.837 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:54.838 INFO:teuthology.orchestra.run.smithi181.stdout:{"epoch":17,"fsid":"1b22fd20-d842-48ea-bdcc-6e548483fbc3","created":"2022-09-03T05:15:36.812798+0000","modified":"2022-09-03T05:15:54.791113+0000","last_up_change":"2022-09-03T05:15:49.310155+0000","last_in_change":"2022-09-03T05:15:37.782081+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-09-03T05:15:50.368822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-09-03T05:15:52.175757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"367873db-887a-4b6d-bb67-c7212da52e73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":104070},{"type":"v1","addr":"172.21.15.181:6809","nonce":104070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":104070},{"type":"v1","addr":"172.21.15.181:6811","nonce":104070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":104070},{"type":"v1","addr":"172.21.15.181:6815","nonce":104070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":104070},{"type":"v1","addr":"172.21.15.181:6813","nonce":104070}]},"public_addr":"172.21.15.181:6809/104070","cluster_addr":"172.21.15.181:6811/104070","heartbeat_back_addr":"172.21.15.181:6815/104070","heartbeat_front_addr":"172.21.15.181:6813/104070","state":["exists","up"]},{"osd":1,"uuid":"03fde9d4-1515-40cc-9ae1-8be880cfeb2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":104071},{"type":"v1","addr":"172.21.15.181:6801","nonce":104071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":104071},{"type":"v1","addr":"172.21.15.181:6803","nonce":104071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":104071},{"type":"v1","addr":"172.21.15.181:6808","nonce":104071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":104071},{"type":"v1","addr":"172.21.15.181:6805","nonce":104071}]},"public_addr":"172.21.15.181:6801/104071","cluster_addr":"172.21.15.181:6803/104071","heartbeat_back_addr":"172.21.15.181:6808/104071","heartbeat_front_addr":"172.21.15.181:6805/104071","state":["exists","up"]},{"osd":2,"uuid":"f0e7b6ab-4ec1-4b97-8a9f-2f0250a1279c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":104072},{"type":"v1","addr":"172.21.15.181:6817","nonce":104072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":104072},{"type":"v1","addr":"172.21.15.181:6819","nonce":104072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":104072},{"type":"v1","addr":"172.21.15.181:6823","nonce":104072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":104072},{"type":"v1","addr":"172.21.15.181:6821","nonce":104072}]},"public_addr":"172.21.15.181:6817/104072","cluster_addr":"172.21.15.181:6819/104072","heartbeat_back_addr":"172.21.15.181:6823/104072","heartbeat_front_addr":"172.21.15.181:6821/104072","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-03T05:15:47.123206+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-03T05:15:46.848141+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-03T05:15:41.911175+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-09-03T05:15:54.852 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 tell osd.0 flush_pg_stats 2022-09-03T05:15:54.852 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 tell osd.1 flush_pg_stats 2022-09-03T05:15:54.853 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 tell osd.2 flush_pg_stats 2022-09-03T05:15:55.020 INFO:teuthology.orchestra.run.smithi181.stdout:34359738372 2022-09-03T05:15:55.021 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.2 2022-09-03T05:15:55.027 INFO:teuthology.orchestra.run.smithi181.stdout:51539607557 2022-09-03T05:15:55.027 INFO:teuthology.orchestra.run.smithi181.stdout:51539607556 2022-09-03T05:15:55.028 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.1 2022-09-03T05:15:55.029 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.0 2022-09-03T05:15:55.382 INFO:teuthology.orchestra.run.smithi181.stdout:34359738370 2022-09-03T05:15:55.395 INFO:teuthology.orchestra.run.smithi181.stdout:51539607554 2022-09-03T05:15:55.397 INFO:tasks.ceph.ceph_manager.c1:need seq 34359738372 got 34359738370 for osd.2 2022-09-03T05:15:55.409 INFO:tasks.ceph.ceph_manager.c1:need seq 51539607556 got 51539607554 for osd.1 2022-09-03T05:15:55.426 INFO:teuthology.orchestra.run.smithi181.stdout:51539607555 2022-09-03T05:15:55.440 INFO:tasks.ceph.ceph_manager.c1:need seq 51539607557 got 51539607555 for osd.0 2022-09-03T05:15:56.398 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.2 2022-09-03T05:15:56.411 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.1 2022-09-03T05:15:56.441 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.0 2022-09-03T05:15:56.763 INFO:teuthology.orchestra.run.smithi181.stdout:34359738370 2022-09-03T05:15:56.772 INFO:tasks.ceph.ceph_manager.c1:need seq 34359738372 got 34359738370 for osd.2 2022-09-03T05:15:56.777 INFO:teuthology.orchestra.run.smithi181.stdout:51539607554 2022-09-03T05:15:56.788 INFO:tasks.ceph.ceph_manager.c1:need seq 51539607556 got 51539607554 for osd.1 2022-09-03T05:15:56.794 INFO:teuthology.orchestra.run.smithi181.stdout:51539607555 2022-09-03T05:15:56.804 INFO:tasks.ceph.ceph_manager.c1:need seq 51539607557 got 51539607555 for osd.0 2022-09-03T05:15:57.774 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.2 2022-09-03T05:15:57.789 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.1 2022-09-03T05:15:57.806 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 osd last-stat-seq osd.0 2022-09-03T05:15:58.117 INFO:teuthology.orchestra.run.smithi181.stdout:34359738372 2022-09-03T05:15:58.135 INFO:tasks.ceph.ceph_manager.c1:need seq 34359738372 got 34359738372 for osd.2 2022-09-03T05:15:58.136 DEBUG:teuthology.parallel:result is None 2022-09-03T05:15:58.159 INFO:teuthology.orchestra.run.smithi181.stdout:51539607556 2022-09-03T05:15:58.163 INFO:teuthology.orchestra.run.smithi181.stdout:51539607557 2022-09-03T05:15:58.180 INFO:tasks.ceph.ceph_manager.c1:need seq 51539607556 got 51539607556 for osd.1 2022-09-03T05:15:58.182 DEBUG:teuthology.parallel:result is None 2022-09-03T05:15:58.184 INFO:tasks.ceph.ceph_manager.c1:need seq 51539607557 got 51539607557 for osd.0 2022-09-03T05:15:58.184 DEBUG:teuthology.parallel:result is None 2022-09-03T05:15:58.185 INFO:tasks.ceph.ceph_manager.c1:waiting for clean 2022-09-03T05:15:58.186 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 pg dump --format=json 2022-09-03T05:15:58.526 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:58.530 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2022-09-03T05:15:58.540 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2022-09-03T05:15:58.328137+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":25032,"kb_used_data":1584,"kb_used_omap":0,"kb_used_meta":23424,"kb_avail":314547768,"statfs":{"total":322122547200,"available":322096914432,"internally_reserved":0,"allocated":1622016,"data_stored":1297282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23986176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.534151"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2022-09-03T05:15:54.463762+0000","last_change":"2022-09-03T05:15:53.797007+0000","last_active":"2022-09-03T05:15:54.463762+0000","last_peered":"2022-09-03T05:15:54.463762+0000","last_clean":"2022-09-03T05:15:54.463762+0000","last_became_active":"2022-09-03T05:15:53.796757+0000","last_became_peered":"2022-09-03T05:15:53.796757+0000","last_unstale":"2022-09-03T05:15:54.463762+0000","last_undegraded":"2022-09-03T05:15:54.463762+0000","last_fullsized":"2022-09-03T05:15:54.463762+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_clean_scrub_stamp":"2022-09-03T05:15:52.785632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T13:46:23.268368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2022-09-03T05:15:54.463745+0000","last_change":"2022-09-03T05:15:53.797053+0000","last_active":"2022-09-03T05:15:54.463745+0000","last_peered":"2022-09-03T05:15:54.463745+0000","last_clean":"2022-09-03T05:15:54.463745+0000","last_became_active":"2022-09-03T05:15:53.796816+0000","last_became_peered":"2022-09-03T05:15:53.796816+0000","last_unstale":"2022-09-03T05:15:54.463745+0000","last_undegraded":"2022-09-03T05:15:54.463745+0000","last_fullsized":"2022-09-03T05:15:54.463745+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_clean_scrub_stamp":"2022-09-03T05:15:52.785632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T15:04:48.988712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2022-09-03T05:15:54.463721+0000","last_change":"2022-09-03T05:15:53.796204+0000","last_active":"2022-09-03T05:15:54.463721+0000","last_peered":"2022-09-03T05:15:54.463721+0000","last_clean":"2022-09-03T05:15:54.463721+0000","last_became_active":"2022-09-03T05:15:53.795919+0000","last_became_peered":"2022-09-03T05:15:53.795919+0000","last_unstale":"2022-09-03T05:15:54.463721+0000","last_undegraded":"2022-09-03T05:15:54.463721+0000","last_fullsized":"2022-09-03T05:15:54.463721+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_clean_scrub_stamp":"2022-09-03T05:15:52.785632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T15:56:06.848020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2022-09-03T05:15:54.327693+0000","last_change":"2022-09-03T05:15:53.793581+0000","last_active":"2022-09-03T05:15:54.327693+0000","last_peered":"2022-09-03T05:15:54.327693+0000","last_clean":"2022-09-03T05:15:54.327693+0000","last_became_active":"2022-09-03T05:15:53.793291+0000","last_became_peered":"2022-09-03T05:15:53.793291+0000","last_unstale":"2022-09-03T05:15:54.327693+0000","last_undegraded":"2022-09-03T05:15:54.327693+0000","last_fullsized":"2022-09-03T05:15:54.327693+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_clean_scrub_stamp":"2022-09-03T05:15:52.785632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T06:55:54.902867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":139,"reported_epoch":16,"state":"active+clean","last_fresh":"2022-09-03T05:15:54.463651+0000","last_change":"2022-09-03T05:15:52.019569+0000","last_active":"2022-09-03T05:15:54.463651+0000","last_peered":"2022-09-03T05:15:54.463651+0000","last_clean":"2022-09-03T05:15:54.463651+0000","last_became_active":"2022-09-03T05:15:52.018804+0000","last_became_peered":"2022-09-03T05:15:52.018804+0000","last_unstale":"2022-09-03T05:15:54.463651+0000","last_undegraded":"2022-09-03T05:15:54.463651+0000","last_fullsized":"2022-09-03T05:15:54.463651+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:15:50.458713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:15:50.458713+0000","last_clean_scrub_stamp":"2022-09-03T05:15:50.458713+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T14:07:54.290621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2022-09-03T05:15:54.463681+0000","last_change":"2022-09-03T05:15:53.795101+0000","last_active":"2022-09-03T05:15:54.463681+0000","last_peered":"2022-09-03T05:15:54.463681+0000","last_clean":"2022-09-03T05:15:54.463681+0000","last_became_active":"2022-09-03T05:15:53.794823+0000","last_became_peered":"2022-09-03T05:15:53.794823+0000","last_unstale":"2022-09-03T05:15:54.463681+0000","last_undegraded":"2022-09-03T05:15:54.463681+0000","last_fullsized":"2022-09-03T05:15:54.463681+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_clean_scrub_stamp":"2022-09-03T05:15:52.785632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T10:16:38.157284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2022-09-03T05:15:54.327724+0000","last_change":"2022-09-03T05:15:53.793587+0000","last_active":"2022-09-03T05:15:54.327724+0000","last_peered":"2022-09-03T05:15:54.327724+0000","last_clean":"2022-09-03T05:15:54.327724+0000","last_became_active":"2022-09-03T05:15:53.793319+0000","last_became_peered":"2022-09-03T05:15:53.793319+0000","last_unstale":"2022-09-03T05:15:54.327724+0000","last_undegraded":"2022-09-03T05:15:54.327724+0000","last_fullsized":"2022-09-03T05:15:54.327724+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_clean_scrub_stamp":"2022-09-03T05:15:52.785632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T14:44:48.349499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2022-09-03T05:15:54.463705+0000","last_change":"2022-09-03T05:15:53.793582+0000","last_active":"2022-09-03T05:15:54.463705+0000","last_peered":"2022-09-03T05:15:54.463705+0000","last_clean":"2022-09-03T05:15:54.463705+0000","last_became_active":"2022-09-03T05:15:53.793299+0000","last_became_peered":"2022-09-03T05:15:53.793299+0000","last_unstale":"2022-09-03T05:15:54.463705+0000","last_undegraded":"2022-09-03T05:15:54.463705+0000","last_fullsized":"2022-09-03T05:15:54.463705+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_clean_scrub_stamp":"2022-09-03T05:15:52.785632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T07:20:58.671356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2022-09-03T05:15:54.463701+0000","last_change":"2022-09-03T05:15:53.795509+0000","last_active":"2022-09-03T05:15:54.463701+0000","last_peered":"2022-09-03T05:15:54.463701+0000","last_clean":"2022-09-03T05:15:54.463701+0000","last_became_active":"2022-09-03T05:15:53.795272+0000","last_became_peered":"2022-09-03T05:15:53.795272+0000","last_unstale":"2022-09-03T05:15:54.463701+0000","last_undegraded":"2022-09-03T05:15:54.463701+0000","last_fullsized":"2022-09-03T05:15:54.463701+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_clean_scrub_stamp":"2022-09-03T05:15:52.785632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T06:59:23.459753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":12,"seq":51539607556,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":8668,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":104848932,"statfs":{"total":107374182400,"available":107365306368,"internally_reserved":0,"allocated":741376,"data_stored":630372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]}]},{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":7704,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":7552,"kb_avail":104849896,"statfs":{"total":107374182400,"available":107366293504,"internally_reserved":0,"allocated":147456,"data_stored":40004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7733248},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.216}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]}]},{"osd":0,"up_from":12,"seq":51539607557,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":8660,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":104848940,"statfs":{"total":107374182400,"available":107365314560,"internally_reserved":0,"allocated":733184,"data_stored":626906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2022-09-03T05:15:58.542 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 pg dump --format=json 2022-09-03T05:15:58.846 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:58.847 INFO:teuthology.orchestra.run.smithi181.stderr:dumped all 2022-09-03T05:15:58.858 INFO:teuthology.orchestra.run.smithi181.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2022-09-03T05:15:58.328137+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":25032,"kb_used_data":1584,"kb_used_omap":0,"kb_used_meta":23424,"kb_avail":314547768,"statfs":{"total":322122547200,"available":322096914432,"internally_reserved":0,"allocated":1622016,"data_stored":1297282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23986176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.534151"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2022-09-03T05:15:54.463762+0000","last_change":"2022-09-03T05:15:53.797007+0000","last_active":"2022-09-03T05:15:54.463762+0000","last_peered":"2022-09-03T05:15:54.463762+0000","last_clean":"2022-09-03T05:15:54.463762+0000","last_became_active":"2022-09-03T05:15:53.796757+0000","last_became_peered":"2022-09-03T05:15:53.796757+0000","last_unstale":"2022-09-03T05:15:54.463762+0000","last_undegraded":"2022-09-03T05:15:54.463762+0000","last_fullsized":"2022-09-03T05:15:54.463762+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_clean_scrub_stamp":"2022-09-03T05:15:52.785632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T13:46:23.268368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2022-09-03T05:15:54.463745+0000","last_change":"2022-09-03T05:15:53.797053+0000","last_active":"2022-09-03T05:15:54.463745+0000","last_peered":"2022-09-03T05:15:54.463745+0000","last_clean":"2022-09-03T05:15:54.463745+0000","last_became_active":"2022-09-03T05:15:53.796816+0000","last_became_peered":"2022-09-03T05:15:53.796816+0000","last_unstale":"2022-09-03T05:15:54.463745+0000","last_undegraded":"2022-09-03T05:15:54.463745+0000","last_fullsized":"2022-09-03T05:15:54.463745+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_clean_scrub_stamp":"2022-09-03T05:15:52.785632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T15:04:48.988712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2022-09-03T05:15:54.463721+0000","last_change":"2022-09-03T05:15:53.796204+0000","last_active":"2022-09-03T05:15:54.463721+0000","last_peered":"2022-09-03T05:15:54.463721+0000","last_clean":"2022-09-03T05:15:54.463721+0000","last_became_active":"2022-09-03T05:15:53.795919+0000","last_became_peered":"2022-09-03T05:15:53.795919+0000","last_unstale":"2022-09-03T05:15:54.463721+0000","last_undegraded":"2022-09-03T05:15:54.463721+0000","last_fullsized":"2022-09-03T05:15:54.463721+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_clean_scrub_stamp":"2022-09-03T05:15:52.785632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T15:56:06.848020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2022-09-03T05:15:54.327693+0000","last_change":"2022-09-03T05:15:53.793581+0000","last_active":"2022-09-03T05:15:54.327693+0000","last_peered":"2022-09-03T05:15:54.327693+0000","last_clean":"2022-09-03T05:15:54.327693+0000","last_became_active":"2022-09-03T05:15:53.793291+0000","last_became_peered":"2022-09-03T05:15:53.793291+0000","last_unstale":"2022-09-03T05:15:54.327693+0000","last_undegraded":"2022-09-03T05:15:54.327693+0000","last_fullsized":"2022-09-03T05:15:54.327693+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_clean_scrub_stamp":"2022-09-03T05:15:52.785632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T06:55:54.902867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'97","reported_seq":139,"reported_epoch":16,"state":"active+clean","last_fresh":"2022-09-03T05:15:54.463651+0000","last_change":"2022-09-03T05:15:52.019569+0000","last_active":"2022-09-03T05:15:54.463651+0000","last_peered":"2022-09-03T05:15:54.463651+0000","last_clean":"2022-09-03T05:15:54.463651+0000","last_became_active":"2022-09-03T05:15:52.018804+0000","last_became_peered":"2022-09-03T05:15:52.018804+0000","last_unstale":"2022-09-03T05:15:54.463651+0000","last_undegraded":"2022-09-03T05:15:54.463651+0000","last_fullsized":"2022-09-03T05:15:54.463651+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:15:50.458713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:15:50.458713+0000","last_clean_scrub_stamp":"2022-09-03T05:15:50.458713+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T14:07:54.290621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2022-09-03T05:15:54.463681+0000","last_change":"2022-09-03T05:15:53.795101+0000","last_active":"2022-09-03T05:15:54.463681+0000","last_peered":"2022-09-03T05:15:54.463681+0000","last_clean":"2022-09-03T05:15:54.463681+0000","last_became_active":"2022-09-03T05:15:53.794823+0000","last_became_peered":"2022-09-03T05:15:53.794823+0000","last_unstale":"2022-09-03T05:15:54.463681+0000","last_undegraded":"2022-09-03T05:15:54.463681+0000","last_fullsized":"2022-09-03T05:15:54.463681+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_clean_scrub_stamp":"2022-09-03T05:15:52.785632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T10:16:38.157284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2022-09-03T05:15:54.327724+0000","last_change":"2022-09-03T05:15:53.793587+0000","last_active":"2022-09-03T05:15:54.327724+0000","last_peered":"2022-09-03T05:15:54.327724+0000","last_clean":"2022-09-03T05:15:54.327724+0000","last_became_active":"2022-09-03T05:15:53.793319+0000","last_became_peered":"2022-09-03T05:15:53.793319+0000","last_unstale":"2022-09-03T05:15:54.327724+0000","last_undegraded":"2022-09-03T05:15:54.327724+0000","last_fullsized":"2022-09-03T05:15:54.327724+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_clean_scrub_stamp":"2022-09-03T05:15:52.785632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T14:44:48.349499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2022-09-03T05:15:54.463705+0000","last_change":"2022-09-03T05:15:53.793582+0000","last_active":"2022-09-03T05:15:54.463705+0000","last_peered":"2022-09-03T05:15:54.463705+0000","last_clean":"2022-09-03T05:15:54.463705+0000","last_became_active":"2022-09-03T05:15:53.793299+0000","last_became_peered":"2022-09-03T05:15:53.793299+0000","last_unstale":"2022-09-03T05:15:54.463705+0000","last_undegraded":"2022-09-03T05:15:54.463705+0000","last_fullsized":"2022-09-03T05:15:54.463705+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_clean_scrub_stamp":"2022-09-03T05:15:52.785632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T07:20:58.671356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2022-09-03T05:15:54.463701+0000","last_change":"2022-09-03T05:15:53.795509+0000","last_active":"2022-09-03T05:15:54.463701+0000","last_peered":"2022-09-03T05:15:54.463701+0000","last_clean":"2022-09-03T05:15:54.463701+0000","last_became_active":"2022-09-03T05:15:53.795272+0000","last_became_peered":"2022-09-03T05:15:53.795272+0000","last_unstale":"2022-09-03T05:15:54.463701+0000","last_undegraded":"2022-09-03T05:15:54.463701+0000","last_fullsized":"2022-09-03T05:15:54.463701+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:15:52.785632+0000","last_clean_scrub_stamp":"2022-09-03T05:15:52.785632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T06:59:23.459753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":12,"seq":51539607556,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":8668,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":104848932,"statfs":{"total":107374182400,"available":107365306368,"internally_reserved":0,"allocated":741376,"data_stored":630372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]}]},{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":7704,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":7552,"kb_avail":104849896,"statfs":{"total":107374182400,"available":107366293504,"internally_reserved":0,"allocated":147456,"data_stored":40004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7733248},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.216}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]}]},{"osd":0,"up_from":12,"seq":51539607557,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":8660,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":104848940,"statfs":{"total":107374182400,"available":107365314560,"internally_reserved":0,"allocated":733184,"data_stored":626906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2022-09-03T05:15:58.860 INFO:tasks.ceph.ceph_manager.c1:clean! 2022-09-03T05:15:58.860 INFO:tasks.ceph:Waiting until ceph cluster c1 is healthy... 2022-09-03T05:15:58.861 INFO:tasks.ceph.ceph_manager.c1:wait_until_healthy 2022-09-03T05:15:58.861 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c1 health --format=json 2022-09-03T05:15:59.203 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:15:59.204 INFO:teuthology.orchestra.run.smithi181.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2022-09-03T05:15:59.217 INFO:tasks.ceph.ceph_manager.c1:wait_until_healthy done 2022-09-03T05:15:59.218 INFO:teuthology.run_tasks:Running task ceph... 2022-09-03T05:15:59.234 INFO:tasks.ceph:Creating ceph cluster c2... 2022-09-03T05:15:59.235 INFO:tasks.ceph:config {'conf': {'client': {'debug rgw': 20, 'debug rgw sync': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync data inject err probability': 0.1, 'rgw sync log trim interval': 0, 'rgw sync meta inject err probability': 0.1, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'client.0': {'rgw data notify interval msec': 0}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'rocksdb delete range threshold': 0}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'cpu_profile': set(), 'cluster': 'c2', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2022-09-03T05:15:59.235 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/dang-2022-09-02_07:58:15-rgw-wip-dang-zipper-admin-api-distro-default-smithi/7007679', 'branch': 'wip-dang-zipper-admin-api', 'description': 'rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zone-plus-pubsub supported-random-distro$/{centos_8} tasks/test_multi}', 'email': None, 'first_in_suite': False, 'job_id': '7007679', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'dang-2022-09-02_07:58:15-rgw-wip-dang-zipper-admin-api-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-dang-zipper-admin-api'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'debug rgw sync': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync data inject err probability': 0.1, 'rgw sync log trim interval': 0, 'rgw sync meta inject err probability': 0.1, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'client.0': {'rgw data notify interval msec': 0}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'rocksdb delete range threshold': 0}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '4d487742eda56d4badbc9bdb33761243365bb37a', 'wait-for-scrub': False}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '4d487742eda56d4badbc9bdb33761243365bb37a'}}, 'rgw': {'compression type': 'random', 'frontend': 'beast'}, 'rgw-multisite': {'realm': {'is default': True, 'name': 'test-realm'}, 'zonegroups': [{'enabled_features': ['resharding'], 'endpoints': ['c1.client.0'], 'is_default': True, 'is_master': True, 'name': 'test-zonegroup', 'zones': [{'endpoints': ['c1.client.0'], 'is_default': True, 'is_master': True, 'name': 'test-zone1'}, {'endpoints': ['c2.client.0'], 'is_default': True, 'name': 'test-zone2'}, {'endpoints': ['c1.client.1'], 'name': 'test-zone3'}, {'endpoints': ['c2.client.1'], 'is_pubsub': True, 'name': 'test-zone4'}]}]}, 'rgw-multisite-tests': {'args': ['tests.py', 'tests_ps.py']}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'workunit': {'branch': 'wip-dang-zipper-admin-api', 'sha1': '4d487742eda56d4badbc9bdb33761243365bb37a'}}, 'owner': 'scheduled_dang@sidious.arb.redhat.com', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'], ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1']], 'sha1': '4d487742eda56d4badbc9bdb33761243365bb37a', 'sleep_before_teardown': 0, 'suite': 'rgw', 'suite_branch': 'wip-dang-zipper-admin-api', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_4d487742eda56d4badbc9bdb33761243365bb37a/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '4d487742eda56d4badbc9bdb33761243365bb37a', 'targets': {'smithi181.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDpTjEkBg4fXYR5zbI8X1aIX0ry1owo8ATLzirNBazD0ie3B5bmEQ5uTTpXbhbf38TCOicU7Kr7jvVMjp3AWagRW1Z/slXFYQo7xV7wq9O5eljNYgdg4GGmLc4ySZrcCfOzKPfS+3SZVPb/66kxGPJmtIn39hoAN2wj6oFZVCGVADXDLjcPvRXmWHrdJypT2cEY5Ws4o7hOwfxKLus2446DBRdhteOUYqOQR8U4mYU6SLNqcSJRdYQWduH4fqZ+XH9rrf25EwFaPqC4qaj0bngkp5NVSHxqeNE2E2p69KIeQmPr0hbjH5GENnQu4m6NlRJijtGz87r1IlxjC8mzspit5PKbEWfawJWbfstOcTATxYlZh0SP2Qm7HP+y7oVxls18yezhrYIzMM9x0pCtWM/Gbtj8uOz8ZwfKgJ7ojFfz4dPnvFxlqNPjvAsCVLvGog4YWDtG6T08pvidSfJU3QIcA6ZWm7UNPCddUzZHYkvF5XqM2D7gE45qGm++vsNIOBs=', 'smithi189.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCxjbJLTVVZDanIouEGFrJENZLRX4RqVQVKScenNpB1JvF6Qxa0CEAI+dllK2d1mgZBvdyQ+8cAVgmZODJHYi/2nBT2G6zXdMuCp6vYu7+HM8/ZgrBwig6Nx+Y4Oj0/QwwlVJDp4GQuQZKRnR/Wmej1JJRnzcPpd7sLK95+DdEwmBX+3kJPM7SZ+xT+jxri9l6zwgFA2LYPwS2s3WhFA43Tks2OzzN/VgCVZHxbqbH4DBThmpXyWKJz1X1ufqc208z1xFhMy4e8UDJb8VHiOALYQTuT4atNidJELjdaY915boUU/2ACXxovBgqiXHyvJFF+x9N9MPVPSm43GMCb7IxQffBP0UB1m3E5FoQIJtsp8VYILST6U104HNWJ0NGDD8Euh2RUYlyEcxvjwSemVBHw3/QWjwZif3k9GcdOZOKH35gNKGF+UX82+aW39UwYrm7axfkO/HnP57ZZJdsq2334ENS5b607SMAnHrOyCDhkQvaYBoSq9X5JqLkVuBR4/8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'cluster': 'c1', 'conf': {'client': {'debug rgw': 20, 'debug rgw sync': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync data inject err probability': 0.1, 'rgw sync log trim interval': 0, 'rgw sync meta inject err probability': 0.1, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'client.0': {'rgw data notify interval msec': 0}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'rocksdb delete range threshold': 0}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '4d487742eda56d4badbc9bdb33761243365bb37a', 'wait-for-scrub': False}}, {'ceph': {'cluster': 'c2', 'conf': {'client': {'debug rgw': 20, 'debug rgw sync': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync data inject err probability': 0.1, 'rgw sync log trim interval': 0, 'rgw sync meta inject err probability': 0.1, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'client.0': {'rgw data notify interval msec': 0}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'rocksdb delete range threshold': 0}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '4d487742eda56d4badbc9bdb33761243365bb37a', 'wait-for-scrub': False}}, {'rgw': {'c1.client.0': {'port': 8000}, 'c1.client.1': {'port': 8001}, 'c2.client.0': {'port': 8000}, 'c2.client.1': {'port': 8001}}}, {'rgw-multisite': None}, {'rgw-multisite-tests': {'config': {'reconfigure_delay': 60}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'b58f2c18636eb10faa77ed3614abd00cb85dfc2c', 'timestamp': '2022-09-02_07:58:15', 'tube': 'smithi', 'user': 'dang', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.1713'} 2022-09-03T05:15:59.236 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/c2.data 2022-09-03T05:15:59.239 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/c2.data 2022-09-03T05:15:59.260 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m0777 -- /var/run/ceph 2022-09-03T05:15:59.299 DEBUG:teuthology.orchestra.run.smithi189:> sudo install -d -m0777 -- /var/run/ceph 2022-09-03T05:15:59.337 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2022-09-03T05:15:59.337 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/scratch_devs of=/dev/stdout 2022-09-03T05:15:59.397 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-09-03T05:15:59.398 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2022-09-03T05:15:59.456 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-09-03T05:15:59.456 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-09-03T05:15:59.457 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 6h/6d Inode: 372755 Links: 1 2022-09-03T05:15:59.457 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-09-03T05:15:59.457 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2022-09-03T05:15:59.457 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2022-09-03 05:15:19.362949103 +0000 2022-09-03T05:15:59.458 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2022-09-03 05:11:49.013018044 +0000 2022-09-03T05:15:59.458 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2022-09-03 05:11:49.013018044 +0000 2022-09-03T05:15:59.458 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: - 2022-09-03T05:15:59.476 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2022-09-03T05:15:59.476 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2022-09-03T05:15:59.477 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000223634 s, 2.3 MB/s 2022-09-03T05:15:59.478 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2022-09-03T05:15:59.535 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-09-03T05:15:59.535 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-09-03T05:15:59.535 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 6h/6d Inode: 371915 Links: 1 2022-09-03T05:15:59.535 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-09-03T05:15:59.536 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2022-09-03T05:15:59.537 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2022-09-03 05:15:19.362949103 +0000 2022-09-03T05:15:59.537 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2022-09-03 05:11:49.691004787 +0000 2022-09-03T05:15:59.538 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2022-09-03 05:11:49.691004787 +0000 2022-09-03T05:15:59.539 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: - 2022-09-03T05:15:59.544 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2022-09-03T05:15:59.545 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2022-09-03T05:15:59.545 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000194099 s, 2.6 MB/s 2022-09-03T05:15:59.547 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2022-09-03T05:15:59.606 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-09-03T05:15:59.607 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-09-03T05:15:59.607 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 6h/6d Inode: 365568 Links: 1 2022-09-03T05:15:59.608 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-09-03T05:15:59.608 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2022-09-03T05:15:59.609 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2022-09-03 05:15:19.362949103 +0000 2022-09-03T05:15:59.609 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2022-09-03 05:11:50.342992040 +0000 2022-09-03T05:15:59.610 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2022-09-03 05:11:50.342992040 +0000 2022-09-03T05:15:59.610 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: - 2022-09-03T05:15:59.616 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2022-09-03T05:15:59.616 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2022-09-03T05:15:59.616 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000318857 s, 1.6 MB/s 2022-09-03T05:15:59.619 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2022-09-03T05:15:59.679 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-09-03T05:15:59.680 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-09-03T05:15:59.680 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 6h/6d Inode: 369048 Links: 1 2022-09-03T05:15:59.680 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-09-03T05:15:59.681 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2022-09-03T05:15:59.681 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2022-09-03 05:15:19.362949103 +0000 2022-09-03T05:15:59.681 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2022-09-03 05:11:50.992979332 +0000 2022-09-03T05:15:59.682 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2022-09-03 05:11:50.992979332 +0000 2022-09-03T05:15:59.683 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: - 2022-09-03T05:15:59.688 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2022-09-03T05:15:59.689 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2022-09-03T05:15:59.689 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000190191 s, 2.7 MB/s 2022-09-03T05:15:59.691 INFO:tasks.ceph:osd dev map: {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'} 2022-09-03T05:15:59.691 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi189.front.sepia.ceph.com'): {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'}} 2022-09-03T05:15:59.692 INFO:tasks.ceph:Generating config... 2022-09-03T05:15:59.693 INFO:tasks.ceph:[client] debug rgw = 20 2022-09-03T05:15:59.694 INFO:tasks.ceph:[client] debug rgw sync = 20 2022-09-03T05:15:59.694 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2022-09-03T05:15:59.694 INFO:tasks.ceph:[client] rgw crypt s3 kms backend = testing 2022-09-03T05:15:59.695 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2022-09-03T05:15:59.695 INFO:tasks.ceph:[client] rgw curl low speed time = 300 2022-09-03T05:15:59.696 INFO:tasks.ceph:[client] rgw data log num shards = 4 2022-09-03T05:15:59.699 INFO:tasks.ceph:[client] rgw md log max shards = 4 2022-09-03T05:15:59.699 INFO:tasks.ceph:[client] rgw sync data inject err probability = 0.1 2022-09-03T05:15:59.700 INFO:tasks.ceph:[client] rgw sync log trim interval = 0 2022-09-03T05:15:59.701 INFO:tasks.ceph:[client] rgw sync meta inject err probability = 0.1 2022-09-03T05:15:59.702 INFO:tasks.ceph:[client] rgw sync obj etag verify = True 2022-09-03T05:15:59.703 INFO:tasks.ceph:[client] setgroup = ceph 2022-09-03T05:15:59.703 INFO:tasks.ceph:[client] setuser = ceph 2022-09-03T05:15:59.704 INFO:tasks.ceph:[client.0] rgw data notify interval msec = 0 2022-09-03T05:15:59.706 INFO:tasks.ceph:[mgr] debug mgr = 20 2022-09-03T05:15:59.706 INFO:tasks.ceph:[mgr] debug ms = 1 2022-09-03T05:15:59.707 INFO:tasks.ceph:[mon] debug mon = 20 2022-09-03T05:15:59.708 INFO:tasks.ceph:[mon] debug ms = 1 2022-09-03T05:15:59.708 INFO:tasks.ceph:[mon] debug paxos = 20 2022-09-03T05:15:59.715 INFO:tasks.ceph:[osd] debug ms = 1 2022-09-03T05:15:59.716 INFO:tasks.ceph:[osd] debug osd = 20 2022-09-03T05:15:59.716 INFO:tasks.ceph:[osd] rocksdb delete range threshold = 0 2022-09-03T05:15:59.716 INFO:tasks.ceph:[osd.0] osd_max_omap_entries_per_request = 10 2022-09-03T05:15:59.717 INFO:tasks.ceph:[osd.1] osd_max_omap_entries_per_request = 1000 2022-09-03T05:15:59.717 INFO:tasks.ceph:[osd.2] osd_max_omap_entries_per_request = 10000 2022-09-03T05:15:59.717 INFO:tasks.ceph:Setting up c2.mon.a... 2022-09-03T05:15:59.718 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/c2.keyring 2022-09-03T05:15:59.786 INFO:teuthology.orchestra.run.smithi189.stdout:creating /etc/ceph/c2.keyring 2022-09-03T05:15:59.789 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/c2.keyring 2022-09-03T05:15:59.835 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 0644 /etc/ceph/c2.keyring 2022-09-03T05:15:59.907 DEBUG:tasks.ceph:Ceph mon addresses: [('c2.mon.a', '172.21.15.189')] 2022-09-03T05:15:59.909 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.189'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug ms': 1, 'debug osd': 20, 'rocksdb delete range threshold': 0}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rgw': 20, 'debug rgw sync': 20, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo=', 'rgw curl low speed time': 300, 'rgw data log num shards': 4, 'rgw md log max shards': 4, 'rgw sync data inject err probability': 0.1, 'rgw sync log trim interval': 0, 'rgw sync meta inject err probability': 0.1, 'rgw sync obj etag verify': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}, 'client.0': {'rgw data notify interval msec': 0}, 'osd.0': {'osd_max_omap_entries_per_request': 10}, 'osd.1': {'osd_max_omap_entries_per_request': 1000}, 'osd.2': {'osd_max_omap_entries_per_request': 10000}} 2022-09-03T05:15:59.911 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2022-09-03T05:15:59.911 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2022-09-03T05:15:59.966 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.189 --print /home/ubuntu/cephtest/c2.monmap 2022-09-03T05:16:00.052 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:16:00.052 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:16:00.054 INFO:teuthology.orchestra.run.smithi189.stdout:monmaptool: monmap file /home/ubuntu/cephtest/c2.monmap 2022-09-03T05:16:00.054 INFO:teuthology.orchestra.run.smithi189.stdout:monmaptool: generated fsid 08f0e03a-4e9e-4009-81f6-ef837e373516 2022-09-03T05:16:00.057 INFO:teuthology.orchestra.run.smithi189.stdout:setting min_mon_release = octopus 2022-09-03T05:16:00.057 INFO:teuthology.orchestra.run.smithi189.stdout:epoch 0 2022-09-03T05:16:00.057 INFO:teuthology.orchestra.run.smithi189.stdout:fsid 08f0e03a-4e9e-4009-81f6-ef837e373516 2022-09-03T05:16:00.057 INFO:teuthology.orchestra.run.smithi189.stdout:last_changed 2022-09-03T05:16:00.054991+0000 2022-09-03T05:16:00.058 INFO:teuthology.orchestra.run.smithi189.stdout:created 2022-09-03T05:16:00.054991+0000 2022-09-03T05:16:00.058 INFO:teuthology.orchestra.run.smithi189.stdout:min_mon_release 15 (octopus) 2022-09-03T05:16:00.058 INFO:teuthology.orchestra.run.smithi189.stdout:election_strategy: 1 2022-09-03T05:16:00.058 INFO:teuthology.orchestra.run.smithi189.stdout:0: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.a 2022-09-03T05:16:00.059 INFO:teuthology.orchestra.run.smithi189.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/c2.monmap (1 monitors) 2022-09-03T05:16:00.059 DEBUG:teuthology.orchestra.run.smithi189:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2022-09-03T05:16:00.113 INFO:tasks.ceph:Writing /etc/ceph/c2.conf for FSID 08f0e03a-4e9e-4009-81f6-ef837e373516... 2022-09-03T05:16:00.114 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c2.conf && sudo chmod 0644 /etc/ceph/c2.conf > /dev/null 2022-09-03T05:16:00.117 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/c2.conf && sudo chmod 0644 /etc/ceph/c2.conf > /dev/null 2022-09-03T05:16:00.164 INFO:teuthology.orchestra.run.smithi181.stdout:[global] 2022-09-03T05:16:00.164 INFO:teuthology.orchestra.run.smithi181.stdout: chdir = "" 2022-09-03T05:16:00.165 INFO:teuthology.orchestra.run.smithi181.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-09-03T05:16:00.165 INFO:teuthology.orchestra.run.smithi181.stdout: auth supported = cephx 2022-09-03T05:16:00.166 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.166 INFO:teuthology.orchestra.run.smithi181.stdout: filestore xattr use omap = true 2022-09-03T05:16:00.167 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.167 INFO:teuthology.orchestra.run.smithi181.stdout: mon clock drift allowed = 1.000 2022-09-03T05:16:00.167 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.167 INFO:teuthology.orchestra.run.smithi181.stdout: osd crush chooseleaf type = 0 2022-09-03T05:16:00.168 INFO:teuthology.orchestra.run.smithi181.stdout: auth debug = true 2022-09-03T05:16:00.168 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.168 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on old message = true 2022-09-03T05:16:00.168 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on bug = true 2022-09-03T05:16:00.169 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.169 INFO:teuthology.orchestra.run.smithi181.stdout: mon max pg per osd = 10000 # >= luminous 2022-09-03T05:16:00.169 INFO:teuthology.orchestra.run.smithi181.stdout: mon pg warn max object skew = 0 2022-09-03T05:16:00.170 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.170 INFO:teuthology.orchestra.run.smithi181.stdout: # disable pg_autoscaler by default for new pools 2022-09-03T05:16:00.171 INFO:teuthology.orchestra.run.smithi181.stdout: osd_pool_default_pg_autoscale_mode = off 2022-09-03T05:16:00.171 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.172 INFO:teuthology.orchestra.run.smithi181.stdout: osd pool default size = 2 2022-09-03T05:16:00.172 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.172 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd allow primary affinity = true 2022-09-03T05:16:00.172 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd allow pg remap = true 2022-09-03T05:16:00.173 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on legacy crush tunables = false 2022-09-03T05:16:00.173 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on crush straw calc version zero = false 2022-09-03T05:16:00.173 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on no sortbitwise = false 2022-09-03T05:16:00.173 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on osd down out interval zero = false 2022-09-03T05:16:00.173 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on too few osds = false 2022-09-03T05:16:00.174 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-09-03T05:16:00.174 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_pool_no_redundancy = false 2022-09-03T05:16:00.174 INFO:teuthology.orchestra.run.smithi181.stdout: mon_allow_pool_size_one = true 2022-09-03T05:16:00.175 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.175 INFO:teuthology.orchestra.run.smithi181.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-09-03T05:16:00.175 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.176 INFO:teuthology.orchestra.run.smithi181.stdout: osd default data pool replay window = 5 2022-09-03T05:16:00.176 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.176 INFO:teuthology.orchestra.run.smithi181.stdout: mon allow pool delete = true 2022-09-03T05:16:00.176 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.177 INFO:teuthology.orchestra.run.smithi181.stdout: mon cluster log file level = debug 2022-09-03T05:16:00.177 INFO:teuthology.orchestra.run.smithi181.stdout: debug asserts on shutdown = true 2022-09-03T05:16:00.177 INFO:teuthology.orchestra.run.smithi181.stdout: mon health detail to clog = false 2022-09-03T05:16:00.177 INFO:teuthology.orchestra.run.smithi181.stdout: mon host = 172.21.15.189 2022-09-03T05:16:00.178 INFO:teuthology.orchestra.run.smithi181.stdout: fsid = 08f0e03a-4e9e-4009-81f6-ef837e373516 2022-09-03T05:16:00.178 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.179 INFO:teuthology.orchestra.run.smithi181.stdout:[osd] 2022-09-03T05:16:00.179 INFO:teuthology.orchestra.run.smithi181.stdout: osd journal size = 100 2022-09-03T05:16:00.179 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.179 INFO:teuthology.orchestra.run.smithi181.stdout: osd scrub load threshold = 5.0 2022-09-03T05:16:00.180 INFO:teuthology.orchestra.run.smithi181.stdout: osd scrub max interval = 600 2022-09-03T05:16:00.180 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.180 INFO:teuthology.orchestra.run.smithi181.stdout: osd recover clone overlap = true 2022-09-03T05:16:00.180 INFO:teuthology.orchestra.run.smithi181.stdout: osd recovery max chunk = 1048576 2022-09-03T05:16:00.180 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.181 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug shutdown = true 2022-09-03T05:16:00.181 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug op order = true 2022-09-03T05:16:00.181 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug verify stray on activate = true 2022-09-03T05:16:00.182 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.183 INFO:teuthology.orchestra.run.smithi181.stdout: osd open classes on start = true 2022-09-03T05:16:00.183 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug pg log writeout = true 2022-09-03T05:16:00.183 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.183 INFO:teuthology.orchestra.run.smithi181.stdout: osd deep scrub update digest min age = 30 2022-09-03T05:16:00.184 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.184 INFO:teuthology.orchestra.run.smithi181.stdout: osd map max advance = 10 2022-09-03T05:16:00.184 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.184 INFO:teuthology.orchestra.run.smithi181.stdout: journal zero on create = true 2022-09-03T05:16:00.185 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.185 INFO:teuthology.orchestra.run.smithi181.stdout: filestore ondisk finisher threads = 3 2022-09-03T05:16:00.186 INFO:teuthology.orchestra.run.smithi181.stdout: filestore apply finisher threads = 3 2022-09-03T05:16:00.186 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.186 INFO:teuthology.orchestra.run.smithi181.stdout: bdev debug aio = true 2022-09-03T05:16:00.187 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug misdirected ops = true 2022-09-03T05:16:00.187 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2022-09-03T05:16:00.187 INFO:teuthology.orchestra.run.smithi181.stdout: debug osd = 20 2022-09-03T05:16:00.189 INFO:teuthology.orchestra.run.smithi181.stdout: rocksdb delete range threshold = 0 2022-09-03T05:16:00.189 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.189 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr] 2022-09-03T05:16:00.190 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2022-09-03T05:16:00.190 INFO:teuthology.orchestra.run.smithi181.stdout: debug mgr = 20 2022-09-03T05:16:00.190 INFO:teuthology.orchestra.run.smithi181.stdout: debug mon = 20 2022-09-03T05:16:00.190 INFO:teuthology.orchestra.run.smithi181.stdout: debug auth = 20 2022-09-03T05:16:00.191 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min pgs per osd = 4 2022-09-03T05:16:00.191 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min bytes per osd = 10 2022-09-03T05:16:00.191 INFO:teuthology.orchestra.run.smithi181.stdout: mgr/telemetry/nag = false 2022-09-03T05:16:00.191 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.192 INFO:teuthology.orchestra.run.smithi181.stdout:[mon] 2022-09-03T05:16:00.192 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2022-09-03T05:16:00.192 INFO:teuthology.orchestra.run.smithi181.stdout: debug mon = 20 2022-09-03T05:16:00.192 INFO:teuthology.orchestra.run.smithi181.stdout: debug paxos = 20 2022-09-03T05:16:00.192 INFO:teuthology.orchestra.run.smithi181.stdout: debug auth = 20 2022-09-03T05:16:00.193 INFO:teuthology.orchestra.run.smithi181.stdout: mon data avail warn = 5 2022-09-03T05:16:00.193 INFO:teuthology.orchestra.run.smithi181.stdout: mon mgr mkfs grace = 240 2022-09-03T05:16:00.193 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min pgs per osd = 4 2022-09-03T05:16:00.193 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd reporter subtree level = osd 2022-09-03T05:16:00.194 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd prime pg temp = true 2022-09-03T05:16:00.194 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min bytes per osd = 10 2022-09-03T05:16:00.194 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.195 INFO:teuthology.orchestra.run.smithi181.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-09-03T05:16:00.195 INFO:teuthology.orchestra.run.smithi181.stdout: auth mon ticket ttl = 660 # 11m 2022-09-03T05:16:00.195 INFO:teuthology.orchestra.run.smithi181.stdout: auth service ticket ttl = 240 # 4m 2022-09-03T05:16:00.196 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.196 INFO:teuthology.orchestra.run.smithi181.stdout: # don't complain about insecure global_id in the test suite 2022-09-03T05:16:00.196 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-09-03T05:16:00.196 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-09-03T05:16:00.197 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.197 INFO:teuthology.orchestra.run.smithi181.stdout: # 1m isn't quite enough 2022-09-03T05:16:00.197 INFO:teuthology.orchestra.run.smithi181.stdout: mon_down_mkfs_grace = 2m 2022-09-03T05:16:00.197 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.198 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_filestore_osds = false 2022-09-03T05:16:00.198 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:16:00.198 INFO:teuthology.orchestra.run.smithi181.stdout:[client] 2022-09-03T05:16:00.198 INFO:teuthology.orchestra.run.smithi181.stdout: rgw cache enabled = true 2022-09-03T05:16:00.198 INFO:teuthology.orchestra.run.smithi181.stdout: rgw enable ops log = true 2022-09-03T05:16:00.199 INFO:teuthology.orchestra.run.smithi181.stdout: rgw enable usage log = true 2022-09-03T05:16:00.199 INFO:teuthology.orchestra.run.smithi181.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-09-03T05:16:00.199 INFO:teuthology.orchestra.run.smithi181.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-09-03T05:16:00.199 INFO:teuthology.orchestra.run.smithi181.stdout: debug rgw = 20 2022-09-03T05:16:00.200 INFO:teuthology.orchestra.run.smithi181.stdout: debug rgw sync = 20 2022-09-03T05:16:00.200 INFO:teuthology.orchestra.run.smithi181.stdout: rgw crypt require ssl = False 2022-09-03T05:16:00.200 INFO:teuthology.orchestra.run.smithi181.stdout: rgw crypt s3 kms backend = testing 2022-09-03T05:16:00.200 INFO:teuthology.orchestra.run.smithi181.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2022-09-03T05:16:00.200 INFO:teuthology.orchestra.run.smithi181.stdout: rgw curl low speed time = 300 2022-09-03T05:16:00.201 INFO:teuthology.orchestra.run.smithi181.stdout: rgw data log num shards = 4 2022-09-03T05:16:00.201 INFO:teuthology.orchestra.run.smithi181.stdout: rgw md log max shards = 4 2022-09-03T05:16:00.201 INFO:teuthology.orchestra.run.smithi181.stdout: rgw sync data inject err probability = 0.1 2022-09-03T05:16:00.201 INFO:teuthology.orchestra.run.smithi181.stdout: rgw sync log trim interval = 0 2022-09-03T05:16:00.202 INFO:teuthology.orchestra.run.smithi181.stdout: rgw sync meta inject err probability = 0.1 2022-09-03T05:16:00.202 INFO:teuthology.orchestra.run.smithi181.stdout: rgw sync obj etag verify = True 2022-09-03T05:16:00.202 INFO:teuthology.orchestra.run.smithi181.stdout: setgroup = ceph 2022-09-03T05:16:00.202 INFO:teuthology.orchestra.run.smithi181.stdout: setuser = ceph 2022-09-03T05:16:00.203 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.a] 2022-09-03T05:16:00.203 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2022-09-03T05:16:00.203 INFO:teuthology.orchestra.run.smithi181.stdout: rgw data notify interval msec = 0 2022-09-03T05:16:00.203 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.0] 2022-09-03T05:16:00.204 INFO:teuthology.orchestra.run.smithi181.stdout: osd_max_omap_entries_per_request = 10 2022-09-03T05:16:00.204 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.1] 2022-09-03T05:16:00.204 INFO:teuthology.orchestra.run.smithi181.stdout: osd_max_omap_entries_per_request = 1000 2022-09-03T05:16:00.204 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.2] 2022-09-03T05:16:00.204 INFO:teuthology.orchestra.run.smithi181.stdout: osd_max_omap_entries_per_request = 10000 2022-09-03T05:16:00.205 INFO:teuthology.orchestra.run.smithi189.stdout:[global] 2022-09-03T05:16:00.206 INFO:teuthology.orchestra.run.smithi189.stdout: chdir = "" 2022-09-03T05:16:00.206 INFO:teuthology.orchestra.run.smithi189.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-09-03T05:16:00.206 INFO:teuthology.orchestra.run.smithi189.stdout: auth supported = cephx 2022-09-03T05:16:00.206 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.207 INFO:teuthology.orchestra.run.smithi189.stdout: filestore xattr use omap = true 2022-09-03T05:16:00.207 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.207 INFO:teuthology.orchestra.run.smithi189.stdout: mon clock drift allowed = 1.000 2022-09-03T05:16:00.207 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.208 INFO:teuthology.orchestra.run.smithi189.stdout: osd crush chooseleaf type = 0 2022-09-03T05:16:00.208 INFO:teuthology.orchestra.run.smithi189.stdout: auth debug = true 2022-09-03T05:16:00.208 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.208 INFO:teuthology.orchestra.run.smithi189.stdout: ms die on old message = true 2022-09-03T05:16:00.209 INFO:teuthology.orchestra.run.smithi189.stdout: ms die on bug = true 2022-09-03T05:16:00.209 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.209 INFO:teuthology.orchestra.run.smithi189.stdout: mon max pg per osd = 10000 # >= luminous 2022-09-03T05:16:00.209 INFO:teuthology.orchestra.run.smithi189.stdout: mon pg warn max object skew = 0 2022-09-03T05:16:00.209 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.210 INFO:teuthology.orchestra.run.smithi189.stdout: # disable pg_autoscaler by default for new pools 2022-09-03T05:16:00.211 INFO:teuthology.orchestra.run.smithi189.stdout: osd_pool_default_pg_autoscale_mode = off 2022-09-03T05:16:00.211 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.211 INFO:teuthology.orchestra.run.smithi189.stdout: osd pool default size = 2 2022-09-03T05:16:00.212 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.212 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd allow primary affinity = true 2022-09-03T05:16:00.212 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd allow pg remap = true 2022-09-03T05:16:00.212 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on legacy crush tunables = false 2022-09-03T05:16:00.212 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on crush straw calc version zero = false 2022-09-03T05:16:00.213 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on no sortbitwise = false 2022-09-03T05:16:00.213 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on osd down out interval zero = false 2022-09-03T05:16:00.213 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on too few osds = false 2022-09-03T05:16:00.213 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-09-03T05:16:00.214 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_pool_no_redundancy = false 2022-09-03T05:16:00.214 INFO:teuthology.orchestra.run.smithi189.stdout: mon_allow_pool_size_one = true 2022-09-03T05:16:00.214 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.214 INFO:teuthology.orchestra.run.smithi189.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-09-03T05:16:00.215 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.215 INFO:teuthology.orchestra.run.smithi189.stdout: osd default data pool replay window = 5 2022-09-03T05:16:00.215 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.215 INFO:teuthology.orchestra.run.smithi189.stdout: mon allow pool delete = true 2022-09-03T05:16:00.215 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.216 INFO:teuthology.orchestra.run.smithi189.stdout: mon cluster log file level = debug 2022-09-03T05:16:00.216 INFO:teuthology.orchestra.run.smithi189.stdout: debug asserts on shutdown = true 2022-09-03T05:16:00.216 INFO:teuthology.orchestra.run.smithi189.stdout: mon health detail to clog = false 2022-09-03T05:16:00.216 INFO:teuthology.orchestra.run.smithi189.stdout: mon host = 172.21.15.189 2022-09-03T05:16:00.216 INFO:teuthology.orchestra.run.smithi189.stdout: fsid = 08f0e03a-4e9e-4009-81f6-ef837e373516 2022-09-03T05:16:00.217 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.217 INFO:teuthology.orchestra.run.smithi189.stdout:[osd] 2022-09-03T05:16:00.217 INFO:teuthology.orchestra.run.smithi189.stdout: osd journal size = 100 2022-09-03T05:16:00.217 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.218 INFO:teuthology.orchestra.run.smithi189.stdout: osd scrub load threshold = 5.0 2022-09-03T05:16:00.218 INFO:teuthology.orchestra.run.smithi189.stdout: osd scrub max interval = 600 2022-09-03T05:16:00.218 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.218 INFO:teuthology.orchestra.run.smithi189.stdout: osd recover clone overlap = true 2022-09-03T05:16:00.219 INFO:teuthology.orchestra.run.smithi189.stdout: osd recovery max chunk = 1048576 2022-09-03T05:16:00.219 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.219 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug shutdown = true 2022-09-03T05:16:00.219 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug op order = true 2022-09-03T05:16:00.219 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug verify stray on activate = true 2022-09-03T05:16:00.220 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.220 INFO:teuthology.orchestra.run.smithi189.stdout: osd open classes on start = true 2022-09-03T05:16:00.220 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug pg log writeout = true 2022-09-03T05:16:00.220 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.221 INFO:teuthology.orchestra.run.smithi189.stdout: osd deep scrub update digest min age = 30 2022-09-03T05:16:00.221 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.221 INFO:teuthology.orchestra.run.smithi189.stdout: osd map max advance = 10 2022-09-03T05:16:00.221 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.221 INFO:teuthology.orchestra.run.smithi189.stdout: journal zero on create = true 2022-09-03T05:16:00.222 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.222 INFO:teuthology.orchestra.run.smithi189.stdout: filestore ondisk finisher threads = 3 2022-09-03T05:16:00.222 INFO:teuthology.orchestra.run.smithi189.stdout: filestore apply finisher threads = 3 2022-09-03T05:16:00.222 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.223 INFO:teuthology.orchestra.run.smithi189.stdout: bdev debug aio = true 2022-09-03T05:16:00.223 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug misdirected ops = true 2022-09-03T05:16:00.223 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2022-09-03T05:16:00.223 INFO:teuthology.orchestra.run.smithi189.stdout: debug osd = 20 2022-09-03T05:16:00.224 INFO:teuthology.orchestra.run.smithi189.stdout: rocksdb delete range threshold = 0 2022-09-03T05:16:00.224 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.224 INFO:teuthology.orchestra.run.smithi189.stdout:[mgr] 2022-09-03T05:16:00.224 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2022-09-03T05:16:00.224 INFO:teuthology.orchestra.run.smithi189.stdout: debug mgr = 20 2022-09-03T05:16:00.225 INFO:teuthology.orchestra.run.smithi189.stdout: debug mon = 20 2022-09-03T05:16:00.225 INFO:teuthology.orchestra.run.smithi189.stdout: debug auth = 20 2022-09-03T05:16:00.225 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min pgs per osd = 4 2022-09-03T05:16:00.225 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min bytes per osd = 10 2022-09-03T05:16:00.226 INFO:teuthology.orchestra.run.smithi189.stdout: mgr/telemetry/nag = false 2022-09-03T05:16:00.226 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.227 INFO:teuthology.orchestra.run.smithi189.stdout:[mon] 2022-09-03T05:16:00.227 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2022-09-03T05:16:00.227 INFO:teuthology.orchestra.run.smithi189.stdout: debug mon = 20 2022-09-03T05:16:00.227 INFO:teuthology.orchestra.run.smithi189.stdout: debug paxos = 20 2022-09-03T05:16:00.228 INFO:teuthology.orchestra.run.smithi189.stdout: debug auth = 20 2022-09-03T05:16:00.228 INFO:teuthology.orchestra.run.smithi189.stdout: mon data avail warn = 5 2022-09-03T05:16:00.228 INFO:teuthology.orchestra.run.smithi189.stdout: mon mgr mkfs grace = 240 2022-09-03T05:16:00.228 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min pgs per osd = 4 2022-09-03T05:16:00.228 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd reporter subtree level = osd 2022-09-03T05:16:00.229 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd prime pg temp = true 2022-09-03T05:16:00.229 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min bytes per osd = 10 2022-09-03T05:16:00.229 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.229 INFO:teuthology.orchestra.run.smithi189.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-09-03T05:16:00.230 INFO:teuthology.orchestra.run.smithi189.stdout: auth mon ticket ttl = 660 # 11m 2022-09-03T05:16:00.230 INFO:teuthology.orchestra.run.smithi189.stdout: auth service ticket ttl = 240 # 4m 2022-09-03T05:16:00.230 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.230 INFO:teuthology.orchestra.run.smithi189.stdout: # don't complain about insecure global_id in the test suite 2022-09-03T05:16:00.230 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-09-03T05:16:00.231 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-09-03T05:16:00.231 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.231 INFO:teuthology.orchestra.run.smithi189.stdout: # 1m isn't quite enough 2022-09-03T05:16:00.231 INFO:teuthology.orchestra.run.smithi189.stdout: mon_down_mkfs_grace = 2m 2022-09-03T05:16:00.232 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.232 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_filestore_osds = false 2022-09-03T05:16:00.232 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:00.232 INFO:teuthology.orchestra.run.smithi189.stdout:[client] 2022-09-03T05:16:00.233 INFO:teuthology.orchestra.run.smithi189.stdout: rgw cache enabled = true 2022-09-03T05:16:00.233 INFO:teuthology.orchestra.run.smithi189.stdout: rgw enable ops log = true 2022-09-03T05:16:00.233 INFO:teuthology.orchestra.run.smithi189.stdout: rgw enable usage log = true 2022-09-03T05:16:00.233 INFO:teuthology.orchestra.run.smithi189.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-09-03T05:16:00.233 INFO:teuthology.orchestra.run.smithi189.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-09-03T05:16:00.234 INFO:teuthology.orchestra.run.smithi189.stdout: debug rgw = 20 2022-09-03T05:16:00.234 INFO:teuthology.orchestra.run.smithi189.stdout: debug rgw sync = 20 2022-09-03T05:16:00.234 INFO:teuthology.orchestra.run.smithi189.stdout: rgw crypt require ssl = False 2022-09-03T05:16:00.234 INFO:teuthology.orchestra.run.smithi189.stdout: rgw crypt s3 kms backend = testing 2022-09-03T05:16:00.235 INFO:teuthology.orchestra.run.smithi189.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= 2022-09-03T05:16:00.235 INFO:teuthology.orchestra.run.smithi189.stdout: rgw curl low speed time = 300 2022-09-03T05:16:00.235 INFO:teuthology.orchestra.run.smithi189.stdout: rgw data log num shards = 4 2022-09-03T05:16:00.235 INFO:teuthology.orchestra.run.smithi189.stdout: rgw md log max shards = 4 2022-09-03T05:16:00.236 INFO:teuthology.orchestra.run.smithi189.stdout: rgw sync data inject err probability = 0.1 2022-09-03T05:16:00.236 INFO:teuthology.orchestra.run.smithi189.stdout: rgw sync log trim interval = 0 2022-09-03T05:16:00.236 INFO:teuthology.orchestra.run.smithi189.stdout: rgw sync meta inject err probability = 0.1 2022-09-03T05:16:00.236 INFO:teuthology.orchestra.run.smithi189.stdout: rgw sync obj etag verify = True 2022-09-03T05:16:00.236 INFO:teuthology.orchestra.run.smithi189.stdout: setgroup = ceph 2022-09-03T05:16:00.237 INFO:teuthology.orchestra.run.smithi189.stdout: setuser = ceph 2022-09-03T05:16:00.237 INFO:teuthology.orchestra.run.smithi189.stdout:[mon.a] 2022-09-03T05:16:00.237 INFO:teuthology.orchestra.run.smithi189.stdout:[client.0] 2022-09-03T05:16:00.237 INFO:teuthology.orchestra.run.smithi189.stdout: rgw data notify interval msec = 0 2022-09-03T05:16:00.238 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.0] 2022-09-03T05:16:00.238 INFO:teuthology.orchestra.run.smithi189.stdout: osd_max_omap_entries_per_request = 10 2022-09-03T05:16:00.238 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.1] 2022-09-03T05:16:00.238 INFO:teuthology.orchestra.run.smithi189.stdout: osd_max_omap_entries_per_request = 1000 2022-09-03T05:16:00.238 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.2] 2022-09-03T05:16:00.239 INFO:teuthology.orchestra.run.smithi189.stdout: osd_max_omap_entries_per_request = 10000 2022-09-03T05:16:00.241 INFO:tasks.ceph:Creating admin key on c2.mon.a... 2022-09-03T05:16:00.241 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/c2.keyring 2022-09-03T05:16:00.339 INFO:tasks.ceph:Copying monmap to all nodes... 2022-09-03T05:16:00.340 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2022-09-03T05:16:00.340 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/etc/ceph/c2.keyring of=/dev/stdout 2022-09-03T05:16:00.361 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2022-09-03T05:16:00.362 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/home/ubuntu/cephtest/c2.monmap of=/dev/stdout 2022-09-03T05:16:00.418 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi181.front.sepia.ceph.com 2022-09-03T05:16:00.419 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2022-09-03T05:16:00.419 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/c2.keyring 2022-09-03T05:16:00.420 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0644 /etc/ceph/c2.keyring 2022-09-03T05:16:00.461 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2022-09-03T05:16:00.462 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/home/ubuntu/cephtest/c2.monmap 2022-09-03T05:16:00.528 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi189.front.sepia.ceph.com 2022-09-03T05:16:00.530 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2022-09-03T05:16:00.531 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/c2.keyring 2022-09-03T05:16:00.532 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 0644 /etc/ceph/c2.keyring 2022-09-03T05:16:00.570 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2022-09-03T05:16:00.571 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/home/ubuntu/cephtest/c2.monmap 2022-09-03T05:16:00.630 INFO:tasks.ceph:Setting up mon nodes... 2022-09-03T05:16:00.630 INFO:tasks.ceph:Setting up mgr nodes... 2022-09-03T05:16:00.631 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/mgr/c2-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/c2-x/keyring 2022-09-03T05:16:00.723 INFO:teuthology.orchestra.run.smithi189.stdout:creating /var/lib/ceph/mgr/c2-x/keyring 2022-09-03T05:16:00.726 INFO:tasks.ceph:Setting up mds nodes... 2022-09-03T05:16:00.726 INFO:tasks.ceph_client:Setting up client nodes... 2022-09-03T05:16:00.727 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/c2.client.0.keyring && sudo chmod 0644 /etc/ceph/c2.client.0.keyring 2022-09-03T05:16:00.771 INFO:teuthology.orchestra.run.smithi189.stdout:creating /etc/ceph/c2.client.0.keyring 2022-09-03T05:16:00.783 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/c2.client.1.keyring && sudo chmod 0644 /etc/ceph/c2.client.1.keyring 2022-09-03T05:16:00.870 INFO:teuthology.orchestra.run.smithi189.stdout:creating /etc/ceph/c2.client.1.keyring 2022-09-03T05:16:00.879 INFO:tasks.ceph:Running mkfs on osd nodes... 2022-09-03T05:16:00.879 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi181.front.sepia.ceph.com'): {'c1.osd.0': '/dev/vg_nvme/lv_1', 'c1.osd.1': '/dev/vg_nvme/lv_2', 'c1.osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi189.front.sepia.ceph.com'): {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'}} 2022-09-03T05:16:00.880 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/osd/c2-0 2022-09-03T05:16:00.955 INFO:tasks.ceph:roles_to_devs: {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'} 2022-09-03T05:16:00.956 INFO:tasks.ceph:role: c2.osd.0 2022-09-03T05:16:00.957 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi189.front.sepia.ceph.com 2022-09-03T05:16:00.957 DEBUG:teuthology.orchestra.run.smithi189:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-09-03T05:16:01.029 INFO:teuthology.orchestra.run.smithi189.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-09-03T05:16:01.030 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 attr=2, projid32bit=1 2022-09-03T05:16:01.030 INFO:teuthology.orchestra.run.smithi189.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-09-03T05:16:01.030 INFO:teuthology.orchestra.run.smithi189.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-09-03T05:16:01.031 INFO:teuthology.orchestra.run.smithi189.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-09-03T05:16:01.031 INFO:teuthology.orchestra.run.smithi189.stdout: = sunit=0 swidth=0 blks 2022-09-03T05:16:01.031 INFO:teuthology.orchestra.run.smithi189.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-09-03T05:16:01.031 INFO:teuthology.orchestra.run.smithi189.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-09-03T05:16:01.032 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-09-03T05:16:01.032 INFO:teuthology.orchestra.run.smithi189.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-09-03T05:16:01.457 INFO:teuthology.orchestra.run.smithi189.stdout:Discarding blocks...Done. 2022-09-03T05:16:01.460 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi189.front.sepia.ceph.com -o noatime 2022-09-03T05:16:01.460 DEBUG:teuthology.orchestra.run.smithi189:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/c2-0 2022-09-03T05:16:01.634 DEBUG:teuthology.orchestra.run.smithi189:> sudo /sbin/restorecon /var/lib/ceph/osd/c2-0 2022-09-03T05:16:01.675 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/osd/c2-1 2022-09-03T05:16:01.744 INFO:tasks.ceph:roles_to_devs: {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'} 2022-09-03T05:16:01.745 INFO:tasks.ceph:role: c2.osd.1 2022-09-03T05:16:01.746 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi189.front.sepia.ceph.com 2022-09-03T05:16:01.746 DEBUG:teuthology.orchestra.run.smithi189:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-09-03T05:16:01.824 INFO:teuthology.orchestra.run.smithi189.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-09-03T05:16:01.825 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 attr=2, projid32bit=1 2022-09-03T05:16:01.825 INFO:teuthology.orchestra.run.smithi189.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-09-03T05:16:01.826 INFO:teuthology.orchestra.run.smithi189.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-09-03T05:16:01.826 INFO:teuthology.orchestra.run.smithi189.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-09-03T05:16:01.827 INFO:teuthology.orchestra.run.smithi189.stdout: = sunit=0 swidth=0 blks 2022-09-03T05:16:01.827 INFO:teuthology.orchestra.run.smithi189.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-09-03T05:16:01.828 INFO:teuthology.orchestra.run.smithi189.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-09-03T05:16:01.828 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-09-03T05:16:01.829 INFO:teuthology.orchestra.run.smithi189.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-09-03T05:16:02.244 INFO:teuthology.orchestra.run.smithi189.stdout:Discarding blocks...Done. 2022-09-03T05:16:02.246 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi189.front.sepia.ceph.com -o noatime 2022-09-03T05:16:02.247 DEBUG:teuthology.orchestra.run.smithi189:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/c2-1 2022-09-03T05:16:02.358 DEBUG:teuthology.orchestra.run.smithi189:> sudo /sbin/restorecon /var/lib/ceph/osd/c2-1 2022-09-03T05:16:02.394 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/osd/c2-2 2022-09-03T05:16:02.473 INFO:tasks.ceph:roles_to_devs: {'c2.osd.0': '/dev/vg_nvme/lv_1', 'c2.osd.1': '/dev/vg_nvme/lv_2', 'c2.osd.2': '/dev/vg_nvme/lv_3'} 2022-09-03T05:16:02.474 INFO:tasks.ceph:role: c2.osd.2 2022-09-03T05:16:02.474 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi189.front.sepia.ceph.com 2022-09-03T05:16:02.475 DEBUG:teuthology.orchestra.run.smithi189:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-09-03T05:16:02.556 INFO:teuthology.orchestra.run.smithi189.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-09-03T05:16:02.557 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 attr=2, projid32bit=1 2022-09-03T05:16:02.558 INFO:teuthology.orchestra.run.smithi189.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-09-03T05:16:02.558 INFO:teuthology.orchestra.run.smithi189.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-09-03T05:16:02.559 INFO:teuthology.orchestra.run.smithi189.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-09-03T05:16:02.559 INFO:teuthology.orchestra.run.smithi189.stdout: = sunit=0 swidth=0 blks 2022-09-03T05:16:02.560 INFO:teuthology.orchestra.run.smithi189.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-09-03T05:16:02.560 INFO:teuthology.orchestra.run.smithi189.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-09-03T05:16:02.561 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-09-03T05:16:02.561 INFO:teuthology.orchestra.run.smithi189.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-09-03T05:16:02.995 INFO:teuthology.orchestra.run.smithi189.stdout:Discarding blocks...Done. 2022-09-03T05:16:02.997 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi189.front.sepia.ceph.com -o noatime 2022-09-03T05:16:02.997 DEBUG:teuthology.orchestra.run.smithi189:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/c2-2 2022-09-03T05:16:03.192 DEBUG:teuthology.orchestra.run.smithi189:> sudo /sbin/restorecon /var/lib/ceph/osd/c2-2 2022-09-03T05:16:03.235 DEBUG:teuthology.orchestra.run.smithi189:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c2 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/c2.monmap 2022-09-03T05:16:03.316 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:03.317+0000 7ff57ff27440 -1 auth: error reading file: /var/lib/ceph/osd/c2-0/keyring: can't open /var/lib/ceph/osd/c2-0/keyring: (2) No such file or directory 2022-09-03T05:16:03.317 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:03.319+0000 7ff57ff27440 -1 created new key in keyring /var/lib/ceph/osd/c2-0/keyring 2022-09-03T05:16:03.318 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:03.319+0000 7ff57ff27440 -1 bluestore(/var/lib/ceph/osd/c2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-0/block: (2) No such file or directory 2022-09-03T05:16:03.318 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:03.319+0000 7ff57ff27440 -1 bluestore(/var/lib/ceph/osd/c2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-0/block: (2) No such file or directory 2022-09-03T05:16:03.318 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:03.319+0000 7ff57ff27440 -1 bluestore(/var/lib/ceph/osd/c2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-0/block: (2) No such file or directory 2022-09-03T05:16:03.326 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:03.326+0000 7ff57ff27440 -1 bluestore(/var/lib/ceph/osd/c2-0) _read_fsid unparsable uuid 2022-09-03T05:16:05.396 DEBUG:teuthology.orchestra.run.smithi189:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c2-0 2022-09-03T05:16:05.428 DEBUG:teuthology.orchestra.run.smithi189:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c2 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/c2.monmap 2022-09-03T05:16:05.519 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:05.520+0000 7f37b0333440 -1 auth: error reading file: /var/lib/ceph/osd/c2-1/keyring: can't open /var/lib/ceph/osd/c2-1/keyring: (2) No such file or directory 2022-09-03T05:16:05.519 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:05.520+0000 7f37b0333440 -1 created new key in keyring /var/lib/ceph/osd/c2-1/keyring 2022-09-03T05:16:05.520 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:05.520+0000 7f37b0333440 -1 bluestore(/var/lib/ceph/osd/c2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-1/block: (2) No such file or directory 2022-09-03T05:16:05.520 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:05.520+0000 7f37b0333440 -1 bluestore(/var/lib/ceph/osd/c2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-1/block: (2) No such file or directory 2022-09-03T05:16:05.520 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:05.520+0000 7f37b0333440 -1 bluestore(/var/lib/ceph/osd/c2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-1/block: (2) No such file or directory 2022-09-03T05:16:05.520 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:05.521+0000 7f37b0333440 -1 bluestore(/var/lib/ceph/osd/c2-1) _read_fsid unparsable uuid 2022-09-03T05:16:07.589 DEBUG:teuthology.orchestra.run.smithi189:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c2-1 2022-09-03T05:16:07.634 DEBUG:teuthology.orchestra.run.smithi189:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster c2 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/c2.monmap 2022-09-03T05:16:07.730 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:07.731+0000 7fd86d6ce440 -1 auth: error reading file: /var/lib/ceph/osd/c2-2/keyring: can't open /var/lib/ceph/osd/c2-2/keyring: (2) No such file or directory 2022-09-03T05:16:07.730 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:07.731+0000 7fd86d6ce440 -1 created new key in keyring /var/lib/ceph/osd/c2-2/keyring 2022-09-03T05:16:07.731 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:07.731+0000 7fd86d6ce440 -1 bluestore(/var/lib/ceph/osd/c2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-2/block: (2) No such file or directory 2022-09-03T05:16:07.731 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:07.731+0000 7fd86d6ce440 -1 bluestore(/var/lib/ceph/osd/c2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-2/block: (2) No such file or directory 2022-09-03T05:16:07.731 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:07.731+0000 7fd86d6ce440 -1 bluestore(/var/lib/ceph/osd/c2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/c2-2/block: (2) No such file or directory 2022-09-03T05:16:07.731 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:07.732+0000 7fd86d6ce440 -1 bluestore(/var/lib/ceph/osd/c2-2) _read_fsid unparsable uuid 2022-09-03T05:16:09.807 DEBUG:teuthology.orchestra.run.smithi189:> sudo chown -R ceph:ceph /var/lib/ceph/osd/c2-2 2022-09-03T05:16:09.843 INFO:tasks.ceph:Reading keys from all nodes... 2022-09-03T05:16:09.844 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2022-09-03T05:16:09.845 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/mgr/c2-x/keyring of=/dev/stdout 2022-09-03T05:16:09.919 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2022-09-03T05:16:09.919 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/c2-0/keyring of=/dev/stdout 2022-09-03T05:16:10.007 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2022-09-03T05:16:10.008 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/c2-1/keyring of=/dev/stdout 2022-09-03T05:16:10.079 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2022-09-03T05:16:10.080 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/c2-2/keyring of=/dev/stdout 2022-09-03T05:16:10.149 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2022-09-03T05:16:10.150 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/etc/ceph/c2.client.0.keyring of=/dev/stdout 2022-09-03T05:16:10.208 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2022-09-03T05:16:10.209 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/etc/ceph/c2.client.1.keyring of=/dev/stdout 2022-09-03T05:16:10.271 INFO:tasks.ceph:Adding keys to all mons... 2022-09-03T05:16:10.272 DEBUG:teuthology.orchestra.run.smithi189:> sudo tee -a /etc/ceph/c2.keyring 2022-09-03T05:16:10.338 INFO:teuthology.orchestra.run.smithi189.stdout:[mgr.x] 2022-09-03T05:16:10.338 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQAQ4xJjEqVMKxAAxQDj+twID8a4tPcXdVGLsw== 2022-09-03T05:16:10.339 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.0] 2022-09-03T05:16:10.340 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQAT4xJjWTICExAAoJVxBLn6qgyPC0R+po9ioA== 2022-09-03T05:16:10.340 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.1] 2022-09-03T05:16:10.341 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQAV4xJjAKkTHxAACc/yt0vLFbPc2PdRUBK4OA== 2022-09-03T05:16:10.341 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.2] 2022-09-03T05:16:10.342 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQAX4xJj/a2nKxAA3vF5owXNZkdLwsM4nwF7qA== 2022-09-03T05:16:10.342 INFO:teuthology.orchestra.run.smithi189.stdout:[client.0] 2022-09-03T05:16:10.342 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQAQ4xJj99gnLhAATuIJCo2gEts6Ig8U+je7MA== 2022-09-03T05:16:10.343 INFO:teuthology.orchestra.run.smithi189.stdout:[client.1] 2022-09-03T05:16:10.343 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQAQ4xJjMkzeMxAALzmZtgTJSYcYTkCUOXYHcQ== 2022-09-03T05:16:10.346 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-09-03T05:16:10.439 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-09-03T05:16:10.496 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-09-03T05:16:10.550 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-09-03T05:16:10.604 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-09-03T05:16:10.662 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/c2.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-09-03T05:16:10.723 INFO:tasks.ceph:Running mkfs on mon nodes... 2022-09-03T05:16:10.728 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/mon/c2-a 2022-09-03T05:16:10.758 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster c2 --mkfs -i a --monmap /home/ubuntu/cephtest/c2.monmap --keyring /etc/ceph/c2.keyring 2022-09-03T05:16:10.854 DEBUG:teuthology.orchestra.run.smithi189:> sudo chown -R ceph:ceph /var/lib/ceph/mon/c2-a 2022-09-03T05:16:10.891 DEBUG:teuthology.orchestra.run.smithi189:> rm -- /home/ubuntu/cephtest/c2.monmap 2022-09-03T05:16:10.952 INFO:tasks.ceph:Starting mon daemons in cluster c2... 2022-09-03T05:16:10.953 INFO:tasks.ceph.c2.mon.a:Restarting daemon 2022-09-03T05:16:10.954 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster c2 -i a 2022-09-03T05:16:10.995 INFO:tasks.ceph.c2.mon.a:Started 2022-09-03T05:16:10.996 INFO:tasks.ceph:Starting mgr daemons in cluster c2... 2022-09-03T05:16:10.996 INFO:tasks.ceph.c2.mgr.x:Restarting daemon 2022-09-03T05:16:10.997 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster c2 -i x 2022-09-03T05:16:11.000 INFO:tasks.ceph.c2.mgr.x:Started 2022-09-03T05:16:11.000 INFO:tasks.ceph:Setting crush tunables to default 2022-09-03T05:16:11.001 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster c2 osd crush tunables default 2022-09-03T05:16:11.205 INFO:teuthology.orchestra.run.smithi189.stderr:adjusted tunables profile to default 2022-09-03T05:16:11.216 INFO:tasks.ceph:Starting osd daemons in cluster c2... 2022-09-03T05:16:11.217 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2022-09-03T05:16:11.217 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/c2-0/fsid of=/dev/stdout 2022-09-03T05:16:11.245 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2022-09-03T05:16:11.246 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/c2-1/fsid of=/dev/stdout 2022-09-03T05:16:11.317 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2022-09-03T05:16:11.318 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/c2-2/fsid of=/dev/stdout 2022-09-03T05:16:11.387 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster c2 osd new 63d90306-a258-4d2a-b0d5-385b45e9ad59 0 2022-09-03T05:16:11.641 INFO:teuthology.orchestra.run.smithi189.stdout:0 2022-09-03T05:16:11.655 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster c2 osd new 0960eccf-8b90-4288-ace5-09c7b13c8994 1 2022-09-03T05:16:11.865 INFO:teuthology.orchestra.run.smithi189.stdout:1 2022-09-03T05:16:11.883 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster c2 osd new ca938b69-d40f-4d4c-8494-cc5ed19b8d9b 2 2022-09-03T05:16:12.094 INFO:teuthology.orchestra.run.smithi189.stdout:2 2022-09-03T05:16:12.117 INFO:tasks.ceph.c2.osd.0:Restarting daemon 2022-09-03T05:16:12.117 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c2 -i 0 2022-09-03T05:16:12.128 INFO:tasks.ceph.c2.osd.0:Started 2022-09-03T05:16:12.130 INFO:tasks.ceph.c2.osd.1:Restarting daemon 2022-09-03T05:16:12.131 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c2 -i 1 2022-09-03T05:16:12.133 INFO:tasks.ceph.c2.osd.1:Started 2022-09-03T05:16:12.134 INFO:tasks.ceph.c2.osd.2:Restarting daemon 2022-09-03T05:16:12.134 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster c2 -i 2 2022-09-03T05:16:12.139 INFO:tasks.ceph.c2.osd.2:Started 2022-09-03T05:16:12.139 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd dump --format=json 2022-09-03T05:16:12.288 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:12.289+0000 7f7de2b14080 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2022-09-03T05:16:12.351 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:12.352 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":5,"fsid":"08f0e03a-4e9e-4009-81f6-ef837e373516","created":"2022-09-03T05:16:11.112141+0000","modified":"2022-09-03T05:16:12.096007+0000","last_up_change":"0.000000","last_in_change":"2022-09-03T05:16:12.096007+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"63d90306-a258-4d2a-b0d5-385b45e9ad59","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0960eccf-8b90-4288-ace5-09c7b13c8994","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ca938b69-d40f-4d4c-8494-cc5ed19b8d9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-09-03T05:16:12.359 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:12.360+0000 7f7de2b14080 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2022-09-03T05:16:12.364 INFO:tasks.ceph.ceph_manager.c2:[] 2022-09-03T05:16:12.364 INFO:tasks.ceph:Waiting for OSDs to come up 2022-09-03T05:16:12.450 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:12.450+0000 7f7de2b14080 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2022-09-03T05:16:12.482 INFO:tasks.ceph.c2.osd.0.smithi189.stderr:2022-09-03T05:16:12.483+0000 7f651dfc9440 -1 Falling back to public interface 2022-09-03T05:16:12.633 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:12.634+0000 7f7de2b14080 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2022-09-03T05:16:12.668 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c2 osd dump --format=json 2022-09-03T05:16:12.741 INFO:tasks.ceph.c2.osd.1.smithi189.stderr:2022-09-03T05:16:12.742+0000 7f7c201bc440 -1 Falling back to public interface 2022-09-03T05:16:12.771 INFO:tasks.ceph.c2.osd.2.smithi189.stderr:2022-09-03T05:16:12.772+0000 7f7cac409440 -1 Falling back to public interface 2022-09-03T05:16:12.858 INFO:teuthology.misc.health.smithi189.stdout: 2022-09-03T05:16:12.858 INFO:teuthology.misc.health.smithi189.stdout:{"epoch":5,"fsid":"08f0e03a-4e9e-4009-81f6-ef837e373516","created":"2022-09-03T05:16:11.112141+0000","modified":"2022-09-03T05:16:12.096007+0000","last_up_change":"0.000000","last_in_change":"2022-09-03T05:16:12.096007+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"63d90306-a258-4d2a-b0d5-385b45e9ad59","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0960eccf-8b90-4288-ace5-09c7b13c8994","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ca938b69-d40f-4d4c-8494-cc5ed19b8d9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-09-03T05:16:12.871 DEBUG:teuthology.misc:0 of 3 OSDs are up 2022-09-03T05:16:13.413 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:13.414+0000 7f7de2b14080 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2022-09-03T05:16:13.552 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:13.551+0000 7f7de2b14080 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2022-09-03T05:16:13.772 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:13.773+0000 7f7de2b14080 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2022-09-03T05:16:13.841 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:13.842+0000 7f7de2b14080 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2022-09-03T05:16:13.977 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:13.978+0000 7f7de2b14080 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2022-09-03T05:16:14.239 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:14.240+0000 7f7de2b14080 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2022-09-03T05:16:14.423 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:14.424+0000 7f7de2b14080 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2022-09-03T05:16:14.558 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:14.559+0000 7f7de2b14080 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2022-09-03T05:16:14.672 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:14.674+0000 7f7de2b14080 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2022-09-03T05:16:14.815 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:14.812+0000 7f7de2b14080 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2022-09-03T05:16:14.887 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:14.889+0000 7f7de2b14080 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2022-09-03T05:16:15.106 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:15.108+0000 7f7de2b14080 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2022-09-03T05:16:15.181 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:15.184+0000 7f7de2b14080 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2022-09-03T05:16:15.778 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:15.780+0000 7f7de2b14080 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2022-09-03T05:16:16.032 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:16.034+0000 7f7de2b14080 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2022-09-03T05:16:16.105 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:16.107+0000 7f7de2b14080 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2022-09-03T05:16:16.185 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:16.187+0000 7f7de2b14080 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2022-09-03T05:16:16.364 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:16.365+0000 7f7de2b14080 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2022-09-03T05:16:16.432 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:16.434+0000 7f7de2b14080 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2022-09-03T05:16:16.582 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:16.581+0000 7f7de2b14080 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2022-09-03T05:16:17.598 INFO:tasks.ceph.c2.mgr.x.smithi189.stderr:2022-09-03T05:16:17.600+0000 7f7d9c8f4700 -1 mgr.server handle_report got status from non-daemon mon.a 2022-09-03T05:16:19.175 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c2 osd dump --format=json 2022-09-03T05:16:19.488 INFO:teuthology.misc.health.smithi189.stdout: 2022-09-03T05:16:19.489 INFO:teuthology.misc.health.smithi189.stdout:{"epoch":5,"fsid":"08f0e03a-4e9e-4009-81f6-ef837e373516","created":"2022-09-03T05:16:11.112141+0000","modified":"2022-09-03T05:16:12.096007+0000","last_up_change":"0.000000","last_in_change":"2022-09-03T05:16:12.096007+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"63d90306-a258-4d2a-b0d5-385b45e9ad59","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0960eccf-8b90-4288-ace5-09c7b13c8994","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ca938b69-d40f-4d4c-8494-cc5ed19b8d9b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-09-03T05:16:19.505 DEBUG:teuthology.misc:0 of 3 OSDs are up 2022-09-03T05:16:20.197 INFO:tasks.ceph.c2.osd.0.smithi189.stderr:2022-09-03T05:16:20.198+0000 7f651dfc9440 -1 osd.0 0 log_to_monitors true 2022-09-03T05:16:20.198 INFO:tasks.ceph.c2.osd.1.smithi189.stderr:2022-09-03T05:16:20.200+0000 7f7c201bc440 -1 osd.1 0 log_to_monitors true 2022-09-03T05:16:20.483 INFO:tasks.ceph.c2.osd.2.smithi189.stderr:2022-09-03T05:16:20.485+0000 7f7cac409440 -1 osd.2 0 log_to_monitors true 2022-09-03T05:16:22.781 INFO:tasks.ceph.c2.osd.0.smithi189.stderr:2022-09-03T05:16:22.783+0000 7f6511d76700 -1 osd.0 0 waiting for initial osdmap 2022-09-03T05:16:22.794 INFO:tasks.ceph.c2.osd.0.smithi189.stderr:2022-09-03T05:16:22.797+0000 7f650a952700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-09-03T05:16:22.797 INFO:tasks.ceph.c2.osd.2.smithi189.stderr:2022-09-03T05:16:22.799+0000 7f7ca01b6700 -1 osd.2 0 waiting for initial osdmap 2022-09-03T05:16:22.812 INFO:tasks.ceph.c2.osd.2.smithi189.stderr:2022-09-03T05:16:22.815+0000 7f7c98d92700 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-09-03T05:16:22.817 INFO:tasks.ceph.c2.osd.1.smithi189.stderr:2022-09-03T05:16:22.819+0000 7f7c13f69700 -1 osd.1 0 waiting for initial osdmap 2022-09-03T05:16:22.827 INFO:tasks.ceph.c2.osd.1.smithi189.stderr:2022-09-03T05:16:22.829+0000 7f7c0cb45700 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-09-03T05:16:25.811 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster c2 osd dump --format=json 2022-09-03T05:16:26.206 INFO:teuthology.misc.health.smithi189.stdout: 2022-09-03T05:16:26.207 INFO:teuthology.misc.health.smithi189.stdout:{"epoch":10,"fsid":"08f0e03a-4e9e-4009-81f6-ef837e373516","created":"2022-09-03T05:16:11.112141+0000","modified":"2022-09-03T05:16:25.804879+0000","last_up_change":"2022-09-03T05:16:23.797791+0000","last_in_change":"2022-09-03T05:16:12.096007+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-09-03T05:16:24.640677+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"63d90306-a258-4d2a-b0d5-385b45e9ad59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":104440},{"type":"v1","addr":"172.21.15.189:6801","nonce":104440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":104440},{"type":"v1","addr":"172.21.15.189:6803","nonce":104440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":104440},{"type":"v1","addr":"172.21.15.189:6807","nonce":104440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":104440},{"type":"v1","addr":"172.21.15.189:6805","nonce":104440}]},"public_addr":"172.21.15.189:6801/104440","cluster_addr":"172.21.15.189:6803/104440","heartbeat_back_addr":"172.21.15.189:6807/104440","heartbeat_front_addr":"172.21.15.189:6805/104440","state":["exists","up"]},{"osd":1,"uuid":"0960eccf-8b90-4288-ace5-09c7b13c8994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":104441},{"type":"v1","addr":"172.21.15.189:6809","nonce":104441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":104441},{"type":"v1","addr":"172.21.15.189:6811","nonce":104441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":104441},{"type":"v1","addr":"172.21.15.189:6815","nonce":104441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":104441},{"type":"v1","addr":"172.21.15.189:6813","nonce":104441}]},"public_addr":"172.21.15.189:6809/104441","cluster_addr":"172.21.15.189:6811/104441","heartbeat_back_addr":"172.21.15.189:6815/104441","heartbeat_front_addr":"172.21.15.189:6813/104441","state":["exists","up"]},{"osd":2,"uuid":"ca938b69-d40f-4d4c-8494-cc5ed19b8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":104470},{"type":"v1","addr":"172.21.15.189:6817","nonce":104470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":104470},{"type":"v1","addr":"172.21.15.189:6819","nonce":104470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":104470},{"type":"v1","addr":"172.21.15.189:6823","nonce":104470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":104470},{"type":"v1","addr":"172.21.15.189:6821","nonce":104470}]},"public_addr":"172.21.15.189:6817/104470","cluster_addr":"172.21.15.189:6819/104470","heartbeat_back_addr":"172.21.15.189:6823/104470","heartbeat_front_addr":"172.21.15.189:6821/104470","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-03T05:16:21.157391+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-03T05:16:21.235398+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-03T05:16:21.519905+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-09-03T05:16:26.220 DEBUG:teuthology.misc:3 of 3 OSDs are up 2022-09-03T05:16:26.221 INFO:tasks.ceph:Creating RBD pool 2022-09-03T05:16:26.222 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster c2 osd pool create rbd 8 2022-09-03T05:16:26.812 INFO:teuthology.orchestra.run.smithi189.stderr:pool 'rbd' created 2022-09-03T05:16:26.824 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster c2 osd pool application enable rbd rbd --yes-i-really-mean-it 2022-09-03T05:16:27.819 INFO:teuthology.orchestra.run.smithi189.stderr:enabled application 'rbd' on pool 'rbd' 2022-09-03T05:16:27.832 INFO:tasks.ceph:Starting mds daemons in cluster c2... 2022-09-03T05:16:27.833 INFO:tasks.ceph:Waiting until c2 daemons up and pgs clean... 2022-09-03T05:16:27.833 INFO:tasks.ceph.ceph_manager.c2:waiting for mgr available 2022-09-03T05:16:27.834 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 mgr dump --format=json 2022-09-03T05:16:27.834 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2022-09-03T05:16:28.200 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:28.211 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":5,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":104033},{"type":"v1","addr":"172.21.15.189:6825","nonce":104033}]},"active_addr":"172.21.15.189:6825/104033","active_change":"2022-09-03T05:16:16.594703+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.189:0","nonce":1353777226}]},{"addrvec":[{"type":"v2","addr":"172.21.15.189:0","nonce":545096157}]},{"addrvec":[{"type":"v2","addr":"172.21.15.189:0","nonce":4107047571}]},{"addrvec":[{"type":"v2","addr":"172.21.15.189:0","nonce":1503329436}]}]}} 2022-09-03T05:16:28.217 INFO:tasks.ceph.ceph_manager.c2:mgr available! 2022-09-03T05:16:28.218 INFO:tasks.ceph.ceph_manager.c2:waiting for all up 2022-09-03T05:16:28.218 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd dump --format=json 2022-09-03T05:16:28.524 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:28.525 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":12,"fsid":"08f0e03a-4e9e-4009-81f6-ef837e373516","created":"2022-09-03T05:16:11.112141+0000","modified":"2022-09-03T05:16:27.811488+0000","last_up_change":"2022-09-03T05:16:23.797791+0000","last_in_change":"2022-09-03T05:16:12.096007+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-09-03T05:16:24.640677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-09-03T05:16:26.546680+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"63d90306-a258-4d2a-b0d5-385b45e9ad59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":104440},{"type":"v1","addr":"172.21.15.189:6801","nonce":104440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":104440},{"type":"v1","addr":"172.21.15.189:6803","nonce":104440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":104440},{"type":"v1","addr":"172.21.15.189:6807","nonce":104440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":104440},{"type":"v1","addr":"172.21.15.189:6805","nonce":104440}]},"public_addr":"172.21.15.189:6801/104440","cluster_addr":"172.21.15.189:6803/104440","heartbeat_back_addr":"172.21.15.189:6807/104440","heartbeat_front_addr":"172.21.15.189:6805/104440","state":["exists","up"]},{"osd":1,"uuid":"0960eccf-8b90-4288-ace5-09c7b13c8994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":104441},{"type":"v1","addr":"172.21.15.189:6809","nonce":104441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":104441},{"type":"v1","addr":"172.21.15.189:6811","nonce":104441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":104441},{"type":"v1","addr":"172.21.15.189:6815","nonce":104441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":104441},{"type":"v1","addr":"172.21.15.189:6813","nonce":104441}]},"public_addr":"172.21.15.189:6809/104441","cluster_addr":"172.21.15.189:6811/104441","heartbeat_back_addr":"172.21.15.189:6815/104441","heartbeat_front_addr":"172.21.15.189:6813/104441","state":["exists","up"]},{"osd":2,"uuid":"ca938b69-d40f-4d4c-8494-cc5ed19b8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":104470},{"type":"v1","addr":"172.21.15.189:6817","nonce":104470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":104470},{"type":"v1","addr":"172.21.15.189:6819","nonce":104470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":104470},{"type":"v1","addr":"172.21.15.189:6823","nonce":104470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":104470},{"type":"v1","addr":"172.21.15.189:6821","nonce":104470}]},"public_addr":"172.21.15.189:6817/104470","cluster_addr":"172.21.15.189:6819/104470","heartbeat_back_addr":"172.21.15.189:6823/104470","heartbeat_front_addr":"172.21.15.189:6821/104470","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-03T05:16:21.157391+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-03T05:16:21.235398+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-03T05:16:21.519905+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-09-03T05:16:28.540 INFO:tasks.ceph.ceph_manager.c2:all up! 2022-09-03T05:16:28.548 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd dump --format=json 2022-09-03T05:16:28.860 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:28.860 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":13,"fsid":"08f0e03a-4e9e-4009-81f6-ef837e373516","created":"2022-09-03T05:16:11.112141+0000","modified":"2022-09-03T05:16:28.816585+0000","last_up_change":"2022-09-03T05:16:23.797791+0000","last_in_change":"2022-09-03T05:16:12.096007+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-09-03T05:16:24.640677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-09-03T05:16:26.546680+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"63d90306-a258-4d2a-b0d5-385b45e9ad59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":104440},{"type":"v1","addr":"172.21.15.189:6801","nonce":104440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":104440},{"type":"v1","addr":"172.21.15.189:6803","nonce":104440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":104440},{"type":"v1","addr":"172.21.15.189:6807","nonce":104440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":104440},{"type":"v1","addr":"172.21.15.189:6805","nonce":104440}]},"public_addr":"172.21.15.189:6801/104440","cluster_addr":"172.21.15.189:6803/104440","heartbeat_back_addr":"172.21.15.189:6807/104440","heartbeat_front_addr":"172.21.15.189:6805/104440","state":["exists","up"]},{"osd":1,"uuid":"0960eccf-8b90-4288-ace5-09c7b13c8994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":104441},{"type":"v1","addr":"172.21.15.189:6809","nonce":104441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":104441},{"type":"v1","addr":"172.21.15.189:6811","nonce":104441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":104441},{"type":"v1","addr":"172.21.15.189:6815","nonce":104441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":104441},{"type":"v1","addr":"172.21.15.189:6813","nonce":104441}]},"public_addr":"172.21.15.189:6809/104441","cluster_addr":"172.21.15.189:6811/104441","heartbeat_back_addr":"172.21.15.189:6815/104441","heartbeat_front_addr":"172.21.15.189:6813/104441","state":["exists","up"]},{"osd":2,"uuid":"ca938b69-d40f-4d4c-8494-cc5ed19b8d9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":104470},{"type":"v1","addr":"172.21.15.189:6817","nonce":104470}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":104470},{"type":"v1","addr":"172.21.15.189:6819","nonce":104470}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":104470},{"type":"v1","addr":"172.21.15.189:6823","nonce":104470}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":104470},{"type":"v1","addr":"172.21.15.189:6821","nonce":104470}]},"public_addr":"172.21.15.189:6817/104470","cluster_addr":"172.21.15.189:6819/104470","heartbeat_back_addr":"172.21.15.189:6823/104470","heartbeat_front_addr":"172.21.15.189:6821/104470","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-03T05:16:21.157391+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-03T05:16:21.235398+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2022-09-03T05:16:21.519905+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-09-03T05:16:28.875 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 tell osd.0 flush_pg_stats 2022-09-03T05:16:28.876 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 tell osd.1 flush_pg_stats 2022-09-03T05:16:28.877 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 tell osd.2 flush_pg_stats 2022-09-03T05:16:29.045 INFO:teuthology.orchestra.run.smithi189.stdout:34359738373 2022-09-03T05:16:29.047 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.2 2022-09-03T05:16:29.060 INFO:teuthology.orchestra.run.smithi189.stdout:34359738373 2022-09-03T05:16:29.062 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.1 2022-09-03T05:16:29.067 INFO:teuthology.orchestra.run.smithi189.stdout:34359738373 2022-09-03T05:16:29.068 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.0 2022-09-03T05:16:29.401 INFO:teuthology.orchestra.run.smithi189.stdout:34359738372 2022-09-03T05:16:29.414 INFO:tasks.ceph.ceph_manager.c2:need seq 34359738373 got 34359738372 for osd.2 2022-09-03T05:16:29.438 INFO:teuthology.orchestra.run.smithi189.stdout:34359738372 2022-09-03T05:16:29.447 INFO:tasks.ceph.ceph_manager.c2:need seq 34359738373 got 34359738372 for osd.0 2022-09-03T05:16:29.465 INFO:teuthology.orchestra.run.smithi189.stdout:34359738372 2022-09-03T05:16:29.477 INFO:tasks.ceph.ceph_manager.c2:need seq 34359738373 got 34359738372 for osd.1 2022-09-03T05:16:30.415 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.2 2022-09-03T05:16:30.455 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.0 2022-09-03T05:16:30.479 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.1 2022-09-03T05:16:30.760 INFO:teuthology.orchestra.run.smithi189.stdout:34359738372 2022-09-03T05:16:30.775 INFO:tasks.ceph.ceph_manager.c2:need seq 34359738373 got 34359738372 for osd.2 2022-09-03T05:16:30.805 INFO:teuthology.orchestra.run.smithi189.stdout:34359738372 2022-09-03T05:16:30.815 INFO:teuthology.orchestra.run.smithi189.stdout:34359738372 2022-09-03T05:16:30.820 INFO:tasks.ceph.ceph_manager.c2:need seq 34359738373 got 34359738372 for osd.0 2022-09-03T05:16:30.830 INFO:tasks.ceph.ceph_manager.c2:need seq 34359738373 got 34359738372 for osd.1 2022-09-03T05:16:31.777 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.2 2022-09-03T05:16:31.821 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.0 2022-09-03T05:16:31.831 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 osd last-stat-seq osd.1 2022-09-03T05:16:32.128 INFO:teuthology.orchestra.run.smithi189.stdout:34359738374 2022-09-03T05:16:32.143 INFO:tasks.ceph.ceph_manager.c2:need seq 34359738373 got 34359738374 for osd.2 2022-09-03T05:16:32.144 DEBUG:teuthology.parallel:result is None 2022-09-03T05:16:32.169 INFO:teuthology.orchestra.run.smithi189.stdout:34359738374 2022-09-03T05:16:32.184 INFO:tasks.ceph.ceph_manager.c2:need seq 34359738373 got 34359738374 for osd.0 2022-09-03T05:16:32.185 DEBUG:teuthology.parallel:result is None 2022-09-03T05:16:32.191 INFO:teuthology.orchestra.run.smithi189.stdout:34359738374 2022-09-03T05:16:32.204 INFO:tasks.ceph.ceph_manager.c2:need seq 34359738373 got 34359738374 for osd.1 2022-09-03T05:16:32.205 DEBUG:teuthology.parallel:result is None 2022-09-03T05:16:32.206 INFO:tasks.ceph.ceph_manager.c2:waiting for clean 2022-09-03T05:16:32.206 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 pg dump --format=json 2022-09-03T05:16:32.547 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:32.549 INFO:teuthology.orchestra.run.smithi189.stderr:dumped all 2022-09-03T05:16:32.560 INFO:teuthology.orchestra.run.smithi189.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2022-09-03T05:16:30.599922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":24932,"kb_used_data":1484,"kb_used_omap":0,"kb_used_meta":23424,"kb_avail":314547868,"statfs":{"total":322122547200,"available":322097016832,"internally_reserved":0,"allocated":1519616,"data_stored":1272047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23986176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2022-09-03T05:16:28.821393+0000","last_change":"2022-09-03T05:16:27.821343+0000","last_active":"2022-09-03T05:16:28.821393+0000","last_peered":"2022-09-03T05:16:28.821393+0000","last_clean":"2022-09-03T05:16:28.821393+0000","last_became_active":"2022-09-03T05:16:27.821093+0000","last_became_peered":"2022-09-03T05:16:27.821093+0000","last_unstale":"2022-09-03T05:16:28.821393+0000","last_undegraded":"2022-09-03T05:16:28.821393+0000","last_fullsized":"2022-09-03T05:16:28.821393+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_clean_scrub_stamp":"2022-09-03T05:16:26.808170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T16:12:36.556544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2022-09-03T05:16:28.821290+0000","last_change":"2022-09-03T05:16:27.820935+0000","last_active":"2022-09-03T05:16:28.821290+0000","last_peered":"2022-09-03T05:16:28.821290+0000","last_clean":"2022-09-03T05:16:28.821290+0000","last_became_active":"2022-09-03T05:16:27.820675+0000","last_became_peered":"2022-09-03T05:16:27.820675+0000","last_unstale":"2022-09-03T05:16:28.821290+0000","last_undegraded":"2022-09-03T05:16:28.821290+0000","last_fullsized":"2022-09-03T05:16:28.821290+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_clean_scrub_stamp":"2022-09-03T05:16:26.808170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T07:56:04.321987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2022-09-03T05:16:28.821261+0000","last_change":"2022-09-03T05:16:27.820955+0000","last_active":"2022-09-03T05:16:28.821261+0000","last_peered":"2022-09-03T05:16:28.821261+0000","last_clean":"2022-09-03T05:16:28.821261+0000","last_became_active":"2022-09-03T05:16:27.820646+0000","last_became_peered":"2022-09-03T05:16:27.820646+0000","last_unstale":"2022-09-03T05:16:28.821261+0000","last_undegraded":"2022-09-03T05:16:28.821261+0000","last_fullsized":"2022-09-03T05:16:28.821261+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_clean_scrub_stamp":"2022-09-03T05:16:26.808170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T10:58:02.356757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":12,"state":"active+clean","last_fresh":"2022-09-03T05:16:29.034877+0000","last_change":"2022-09-03T05:16:27.819362+0000","last_active":"2022-09-03T05:16:29.034877+0000","last_peered":"2022-09-03T05:16:29.034877+0000","last_clean":"2022-09-03T05:16:29.034877+0000","last_became_active":"2022-09-03T05:16:27.819122+0000","last_became_peered":"2022-09-03T05:16:27.819122+0000","last_unstale":"2022-09-03T05:16:29.034877+0000","last_undegraded":"2022-09-03T05:16:29.034877+0000","last_fullsized":"2022-09-03T05:16:29.034877+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_clean_scrub_stamp":"2022-09-03T05:16:26.808170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T11:29:05.586523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":139,"reported_epoch":13,"state":"active+clean","last_fresh":"2022-09-03T05:16:28.821193+0000","last_change":"2022-09-03T05:16:25.810340+0000","last_active":"2022-09-03T05:16:28.821193+0000","last_peered":"2022-09-03T05:16:28.821193+0000","last_clean":"2022-09-03T05:16:28.821193+0000","last_became_active":"2022-09-03T05:16:25.810084+0000","last_became_peered":"2022-09-03T05:16:25.810084+0000","last_unstale":"2022-09-03T05:16:28.821193+0000","last_undegraded":"2022-09-03T05:16:28.821193+0000","last_fullsized":"2022-09-03T05:16:28.821193+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:16:24.801159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:16:24.801159+0000","last_clean_scrub_stamp":"2022-09-03T05:16:24.801159+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T10:43:50.749406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2022-09-03T05:16:28.821375+0000","last_change":"2022-09-03T05:16:27.820969+0000","last_active":"2022-09-03T05:16:28.821375+0000","last_peered":"2022-09-03T05:16:28.821375+0000","last_clean":"2022-09-03T05:16:28.821375+0000","last_became_active":"2022-09-03T05:16:27.820719+0000","last_became_peered":"2022-09-03T05:16:27.820719+0000","last_unstale":"2022-09-03T05:16:28.821375+0000","last_undegraded":"2022-09-03T05:16:28.821375+0000","last_fullsized":"2022-09-03T05:16:28.821375+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_clean_scrub_stamp":"2022-09-03T05:16:26.808170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T15:50:18.538796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":12,"state":"active+clean","last_fresh":"2022-09-03T05:16:29.034904+0000","last_change":"2022-09-03T05:16:27.821123+0000","last_active":"2022-09-03T05:16:29.034904+0000","last_peered":"2022-09-03T05:16:29.034904+0000","last_clean":"2022-09-03T05:16:29.034904+0000","last_became_active":"2022-09-03T05:16:27.820829+0000","last_became_peered":"2022-09-03T05:16:27.820829+0000","last_unstale":"2022-09-03T05:16:29.034904+0000","last_undegraded":"2022-09-03T05:16:29.034904+0000","last_fullsized":"2022-09-03T05:16:29.034904+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_clean_scrub_stamp":"2022-09-03T05:16:26.808170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T05:48:04.887516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2022-09-03T05:16:27.820494+0000","last_change":"2022-09-03T05:16:27.819591+0000","last_active":"2022-09-03T05:16:27.820494+0000","last_peered":"2022-09-03T05:16:27.820494+0000","last_clean":"2022-09-03T05:16:27.820494+0000","last_became_active":"2022-09-03T05:16:27.819239+0000","last_became_peered":"2022-09-03T05:16:27.819239+0000","last_unstale":"2022-09-03T05:16:27.820494+0000","last_undegraded":"2022-09-03T05:16:27.820494+0000","last_fullsized":"2022-09-03T05:16:27.820494+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_clean_scrub_stamp":"2022-09-03T05:16:26.808170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T15:36:46.679872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2022-09-03T05:16:28.821313+0000","last_change":"2022-09-03T05:16:27.822508+0000","last_active":"2022-09-03T05:16:28.821313+0000","last_peered":"2022-09-03T05:16:28.821313+0000","last_clean":"2022-09-03T05:16:28.821313+0000","last_became_active":"2022-09-03T05:16:27.822207+0000","last_became_peered":"2022-09-03T05:16:27.822207+0000","last_unstale":"2022-09-03T05:16:28.821313+0000","last_undegraded":"2022-09-03T05:16:28.821313+0000","last_fullsized":"2022-09-03T05:16:28.821313+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_clean_scrub_stamp":"2022-09-03T05:16:26.808170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T10:34:32.035579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":7672,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":7552,"kb_avail":104849928,"statfs":{"total":107374182400,"available":107366326272,"internally_reserved":0,"allocated":114688,"data_stored":31846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7733248},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":8624,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":104848976,"statfs":{"total":107374182400,"available":107365351424,"internally_reserved":0,"allocated":696320,"data_stored":617987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]}]},{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":8636,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":104848964,"statfs":{"total":107374182400,"available":107365339136,"internally_reserved":0,"allocated":708608,"data_stored":622214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2022-09-03T05:16:32.562 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 pg dump --format=json 2022-09-03T05:16:32.867 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:32.869 INFO:teuthology.orchestra.run.smithi189.stderr:dumped all 2022-09-03T05:16:32.882 INFO:teuthology.orchestra.run.smithi189.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2022-09-03T05:16:32.600197+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":24932,"kb_used_data":1484,"kb_used_omap":0,"kb_used_meta":23424,"kb_avail":314547868,"statfs":{"total":322122547200,"available":322097016832,"internally_reserved":0,"allocated":1519616,"data_stored":1272047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":23986176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000275"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2022-09-03T05:16:28.821393+0000","last_change":"2022-09-03T05:16:27.821343+0000","last_active":"2022-09-03T05:16:28.821393+0000","last_peered":"2022-09-03T05:16:28.821393+0000","last_clean":"2022-09-03T05:16:28.821393+0000","last_became_active":"2022-09-03T05:16:27.821093+0000","last_became_peered":"2022-09-03T05:16:27.821093+0000","last_unstale":"2022-09-03T05:16:28.821393+0000","last_undegraded":"2022-09-03T05:16:28.821393+0000","last_fullsized":"2022-09-03T05:16:28.821393+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_clean_scrub_stamp":"2022-09-03T05:16:26.808170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T16:12:36.556544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2022-09-03T05:16:28.821290+0000","last_change":"2022-09-03T05:16:27.820935+0000","last_active":"2022-09-03T05:16:28.821290+0000","last_peered":"2022-09-03T05:16:28.821290+0000","last_clean":"2022-09-03T05:16:28.821290+0000","last_became_active":"2022-09-03T05:16:27.820675+0000","last_became_peered":"2022-09-03T05:16:27.820675+0000","last_unstale":"2022-09-03T05:16:28.821290+0000","last_undegraded":"2022-09-03T05:16:28.821290+0000","last_fullsized":"2022-09-03T05:16:28.821290+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_clean_scrub_stamp":"2022-09-03T05:16:26.808170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T07:56:04.321987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2022-09-03T05:16:28.821261+0000","last_change":"2022-09-03T05:16:27.820955+0000","last_active":"2022-09-03T05:16:28.821261+0000","last_peered":"2022-09-03T05:16:28.821261+0000","last_clean":"2022-09-03T05:16:28.821261+0000","last_became_active":"2022-09-03T05:16:27.820646+0000","last_became_peered":"2022-09-03T05:16:27.820646+0000","last_unstale":"2022-09-03T05:16:28.821261+0000","last_undegraded":"2022-09-03T05:16:28.821261+0000","last_fullsized":"2022-09-03T05:16:28.821261+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_clean_scrub_stamp":"2022-09-03T05:16:26.808170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T10:58:02.356757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":12,"state":"active+clean","last_fresh":"2022-09-03T05:16:29.034877+0000","last_change":"2022-09-03T05:16:27.819362+0000","last_active":"2022-09-03T05:16:29.034877+0000","last_peered":"2022-09-03T05:16:29.034877+0000","last_clean":"2022-09-03T05:16:29.034877+0000","last_became_active":"2022-09-03T05:16:27.819122+0000","last_became_peered":"2022-09-03T05:16:27.819122+0000","last_unstale":"2022-09-03T05:16:29.034877+0000","last_undegraded":"2022-09-03T05:16:29.034877+0000","last_fullsized":"2022-09-03T05:16:29.034877+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_clean_scrub_stamp":"2022-09-03T05:16:26.808170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T11:29:05.586523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":139,"reported_epoch":13,"state":"active+clean","last_fresh":"2022-09-03T05:16:28.821193+0000","last_change":"2022-09-03T05:16:25.810340+0000","last_active":"2022-09-03T05:16:28.821193+0000","last_peered":"2022-09-03T05:16:28.821193+0000","last_clean":"2022-09-03T05:16:28.821193+0000","last_became_active":"2022-09-03T05:16:25.810084+0000","last_became_peered":"2022-09-03T05:16:25.810084+0000","last_unstale":"2022-09-03T05:16:28.821193+0000","last_undegraded":"2022-09-03T05:16:28.821193+0000","last_fullsized":"2022-09-03T05:16:28.821193+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:16:24.801159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:16:24.801159+0000","last_clean_scrub_stamp":"2022-09-03T05:16:24.801159+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T10:43:50.749406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2022-09-03T05:16:28.821375+0000","last_change":"2022-09-03T05:16:27.820969+0000","last_active":"2022-09-03T05:16:28.821375+0000","last_peered":"2022-09-03T05:16:28.821375+0000","last_clean":"2022-09-03T05:16:28.821375+0000","last_became_active":"2022-09-03T05:16:27.820719+0000","last_became_peered":"2022-09-03T05:16:27.820719+0000","last_unstale":"2022-09-03T05:16:28.821375+0000","last_undegraded":"2022-09-03T05:16:28.821375+0000","last_fullsized":"2022-09-03T05:16:28.821375+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_clean_scrub_stamp":"2022-09-03T05:16:26.808170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T15:50:18.538796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":12,"state":"active+clean","last_fresh":"2022-09-03T05:16:29.034904+0000","last_change":"2022-09-03T05:16:27.821123+0000","last_active":"2022-09-03T05:16:29.034904+0000","last_peered":"2022-09-03T05:16:29.034904+0000","last_clean":"2022-09-03T05:16:29.034904+0000","last_became_active":"2022-09-03T05:16:27.820829+0000","last_became_peered":"2022-09-03T05:16:27.820829+0000","last_unstale":"2022-09-03T05:16:29.034904+0000","last_undegraded":"2022-09-03T05:16:29.034904+0000","last_fullsized":"2022-09-03T05:16:29.034904+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_clean_scrub_stamp":"2022-09-03T05:16:26.808170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T05:48:04.887516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":12,"state":"active+clean","last_fresh":"2022-09-03T05:16:27.820494+0000","last_change":"2022-09-03T05:16:27.819591+0000","last_active":"2022-09-03T05:16:27.820494+0000","last_peered":"2022-09-03T05:16:27.820494+0000","last_clean":"2022-09-03T05:16:27.820494+0000","last_became_active":"2022-09-03T05:16:27.819239+0000","last_became_peered":"2022-09-03T05:16:27.819239+0000","last_unstale":"2022-09-03T05:16:27.820494+0000","last_undegraded":"2022-09-03T05:16:27.820494+0000","last_fullsized":"2022-09-03T05:16:27.820494+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_clean_scrub_stamp":"2022-09-03T05:16:26.808170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T15:36:46.679872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":13,"state":"active+clean","last_fresh":"2022-09-03T05:16:28.821313+0000","last_change":"2022-09-03T05:16:27.822508+0000","last_active":"2022-09-03T05:16:28.821313+0000","last_peered":"2022-09-03T05:16:28.821313+0000","last_clean":"2022-09-03T05:16:28.821313+0000","last_became_active":"2022-09-03T05:16:27.822207+0000","last_became_peered":"2022-09-03T05:16:27.822207+0000","last_unstale":"2022-09-03T05:16:28.821313+0000","last_undegraded":"2022-09-03T05:16:28.821313+0000","last_fullsized":"2022-09-03T05:16:28.821313+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-09-03T05:16:26.808170+0000","last_clean_scrub_stamp":"2022-09-03T05:16:26.808170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2022-09-04T10:34:32.035579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":7672,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":7552,"kb_avail":104849928,"statfs":{"total":107374182400,"available":107366326272,"internally_reserved":0,"allocated":114688,"data_stored":31846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7733248},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":8624,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":104848976,"statfs":{"total":107374182400,"available":107365351424,"internally_reserved":0,"allocated":696320,"data_stored":617987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]}]},{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":8636,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":104848964,"statfs":{"total":107374182400,"available":107365339136,"internally_reserved":0,"allocated":708608,"data_stored":622214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2022-09-03T05:16:32.884 INFO:tasks.ceph.ceph_manager.c2:clean! 2022-09-03T05:16:32.884 INFO:tasks.ceph:Waiting until ceph cluster c2 is healthy... 2022-09-03T05:16:32.884 INFO:tasks.ceph.ceph_manager.c2:wait_until_healthy 2022-09-03T05:16:32.885 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster c2 health --format=json 2022-09-03T05:16:33.246 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:16:33.246 INFO:teuthology.orchestra.run.smithi189.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2022-09-03T05:16:33.247 INFO:tasks.ceph.ceph_manager.c2:wait_until_healthy done 2022-09-03T05:16:33.248 INFO:teuthology.run_tasks:Running task rgw... 2022-09-03T05:16:33.265 DEBUG:tasks.rgw:config is {'c1.client.0': {'port': 8000}, 'c1.client.1': {'port': 8001}, 'c2.client.0': {'port': 8000}, 'c2.client.1': {'port': 8001}} 2022-09-03T05:16:33.266 DEBUG:tasks.rgw:client list is dict_keys(['c1.client.0', 'c1.client.1', 'c2.client.0', 'c2.client.1']) 2022-09-03T05:16:33.267 INFO:tasks.rgw:Creating data pools 2022-09-03T05:16:33.267 DEBUG:tasks.rgw:Obtaining remote for client c1.client.0 2022-09-03T05:16:33.267 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster c1 2022-09-03T05:16:33.836 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'default.rgw.buckets.data' created 2022-09-03T05:16:33.850 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster c1 2022-09-03T05:16:34.844 INFO:teuthology.orchestra.run.smithi181.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2022-09-03T05:16:34.864 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster c1 2022-09-03T05:16:35.849 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'default.rgw.buckets.index' created 2022-09-03T05:16:35.861 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster c1 2022-09-03T05:16:36.862 INFO:teuthology.orchestra.run.smithi181.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2022-09-03T05:16:36.878 DEBUG:tasks.rgw:Obtaining remote for client c1.client.1 2022-09-03T05:16:36.879 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster c1 2022-09-03T05:16:37.200 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'default.rgw.buckets.data' already exists 2022-09-03T05:16:37.209 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster c1 2022-09-03T05:16:37.855 INFO:teuthology.orchestra.run.smithi181.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2022-09-03T05:16:37.865 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster c1 2022-09-03T05:16:38.174 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'default.rgw.buckets.index' already exists 2022-09-03T05:16:38.186 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster c1 2022-09-03T05:16:38.859 INFO:teuthology.orchestra.run.smithi181.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2022-09-03T05:16:38.871 DEBUG:tasks.rgw:Obtaining remote for client c2.client.0 2022-09-03T05:16:38.872 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster c2 2022-09-03T05:16:40.110 INFO:teuthology.orchestra.run.smithi189.stderr:pool 'default.rgw.buckets.data' created 2022-09-03T05:16:40.137 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster c2 2022-09-03T05:16:41.131 INFO:teuthology.orchestra.run.smithi189.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2022-09-03T05:16:41.156 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster c2 2022-09-03T05:16:42.128 INFO:teuthology.orchestra.run.smithi189.stderr:pool 'default.rgw.buckets.index' created 2022-09-03T05:16:42.224 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster c2 2022-09-03T05:16:43.147 INFO:teuthology.orchestra.run.smithi189.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2022-09-03T05:16:43.161 DEBUG:tasks.rgw:Obtaining remote for client c2.client.1 2022-09-03T05:16:43.161 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster c2 2022-09-03T05:16:43.486 INFO:teuthology.orchestra.run.smithi189.stderr:pool 'default.rgw.buckets.data' already exists 2022-09-03T05:16:43.493 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster c2 2022-09-03T05:16:44.127 INFO:teuthology.orchestra.run.smithi189.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2022-09-03T05:16:44.140 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster c2 2022-09-03T05:16:44.450 INFO:teuthology.orchestra.run.smithi189.stderr:pool 'default.rgw.buckets.index' already exists 2022-09-03T05:16:44.460 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster c2 2022-09-03T05:16:45.130 INFO:teuthology.orchestra.run.smithi189.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2022-09-03T05:16:45.143 DEBUG:tasks.rgw:Pools created 2022-09-03T05:16:45.144 INFO:tasks.rgw:Configuring compression type = random 2022-09-03T05:16:45.145 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['user', 'list'] 2022-09-03T05:16:45.146 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'user', 'list'] 2022-09-03T05:16:45.146 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 user list 2022-09-03T05:16:45.221 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:16:45.221 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:16:45.227 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:45.229+0000 7f6fe63f6600 20 rados->read ofs=0 len=0 2022-09-03T05:16:46.887 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.885+0000 7f6fe63f6600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:46.888 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.885+0000 7f6fe63f6600 20 realm 2022-09-03T05:16:46.889 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.885+0000 7f6fe63f6600 20 rados->read ofs=0 len=0 2022-09-03T05:16:46.889 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.887+0000 7f6fe63f6600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:46.889 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.887+0000 7f6fe63f6600 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2022-09-03T05:16:46.890 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.887+0000 7f6fe63f6600 20 rados->read ofs=0 len=0 2022-09-03T05:16:46.891 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.887+0000 7f6fe63f6600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:46.891 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.887+0000 7f6fe63f6600 20 rados->read ofs=0 len=0 2022-09-03T05:16:46.894 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.895+0000 7f6fe63f6600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:46.894 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.895+0000 7f6fe63f6600 20 rados->read ofs=0 len=0 2022-09-03T05:16:46.898 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.896+0000 7f6fe63f6600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:46.898 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.896+0000 7f6fe63f6600 20 rados->read ofs=0 len=0 2022-09-03T05:16:46.898 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.896+0000 7f6fe63f6600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:46.899 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.896+0000 7f6fe63f6600 10 Creating default zonegroup 2022-09-03T05:16:46.899 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.897+0000 7f6fe63f6600 10 couldn't find old data placement pools config, setting up new ones for the zone 2022-09-03T05:16:46.905 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.906+0000 7f6fe63f6600 10 WARNING: store->list_zones() returned r=-2 2022-09-03T05:16:46.906 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.907+0000 7f6fe63f6600 20 rados->read ofs=0 len=0 2022-09-03T05:16:46.906 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.907+0000 7f6fe63f6600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:46.909 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.911+0000 7f6fe63f6600 20 rados->read ofs=0 len=0 2022-09-03T05:16:46.910 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.911+0000 7f6fe63f6600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:46.910 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.911+0000 7f6fe63f6600 10 could not read realm id: (2) No such file or directory 2022-09-03T05:16:46.910 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.911+0000 7f6fe63f6600 10 WARNING: failed to set zone as default, r=-22 2022-09-03T05:16:46.911 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.911+0000 7f6fe63f6600 20 rados->read ofs=0 len=0 2022-09-03T05:16:46.911 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.911+0000 7f6fe63f6600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:46.913 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.915+0000 7f6fe63f6600 20 rados->read ofs=0 len=0 2022-09-03T05:16:46.915 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.915+0000 7f6fe63f6600 20 rados_obj.operate() r=0 bl.length=873 2022-09-03T05:16:46.915 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.915+0000 7f6fe63f6600 20 rados->read ofs=0 len=0 2022-09-03T05:16:46.915 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.916+0000 7f6fe63f6600 20 rados_obj.operate() r=0 bl.length=398 2022-09-03T05:16:46.916 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.916+0000 7f6fe63f6600 10 Using default name default 2022-09-03T05:16:46.916 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.916+0000 7f6fe63f6600 20 rados->read ofs=0 len=0 2022-09-03T05:16:46.916 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.916+0000 7f6fe63f6600 20 rados_obj.operate() r=0 bl.length=46 2022-09-03T05:16:46.916 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.916+0000 7f6fe63f6600 20 rados->read ofs=0 len=0 2022-09-03T05:16:46.916 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.916+0000 7f6fe63f6600 20 rados_obj.operate() r=0 bl.length=873 2022-09-03T05:16:46.917 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.916+0000 7f6fe63f6600 20 zone default found 2022-09-03T05:16:46.917 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.916+0000 7f6fe63f6600 4 Realm: () 2022-09-03T05:16:46.917 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.916+0000 7f6fe63f6600 4 ZoneGroup: default (9ad14a2b-08e7-4a70-91e3-2b1956365d5d) 2022-09-03T05:16:46.917 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.916+0000 7f6fe63f6600 4 Zone: default (ce9d29af-1d2c-44d9-98e6-223162653e78) 2022-09-03T05:16:46.918 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.916+0000 7f6fe63f6600 10 cannot find current period zonegroup using local zonegroup configuration 2022-09-03T05:16:46.918 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.916+0000 7f6fe63f6600 20 zonegroup default 2022-09-03T05:16:46.918 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.916+0000 7f6fe63f6600 20 rados->read ofs=0 len=0 2022-09-03T05:16:46.918 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.917+0000 7f6fe63f6600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:46.919 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:46.917+0000 7f6fe63f6600 20 rados->read ofs=0 len=0 2022-09-03T05:16:48.909 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:48.910+0000 7f6fe63f6600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:48.909 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:48.910+0000 7f6fe63f6600 20 rados->read ofs=0 len=0 2022-09-03T05:16:48.910 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:48.910+0000 7f6fe63f6600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:48.910 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:48.910+0000 7f6fe63f6600 20 started sync module instance, tier type = 2022-09-03T05:16:48.911 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:48.910+0000 7f6fe63f6600 20 started zone id=ce9d29af-1d2c-44d9-98e6-223162653e78 (name=default) with tier type = 2022-09-03T05:16:50.929 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.931+0000 7f6fe63f6600 20 add_watcher() i=0 2022-09-03T05:16:50.930 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.931+0000 7f6fe63f6600 20 add_watcher() i=1 2022-09-03T05:16:50.930 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.931+0000 7f6fe63f6600 20 add_watcher() i=2 2022-09-03T05:16:50.931 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.931+0000 7f6fe63f6600 20 add_watcher() i=3 2022-09-03T05:16:50.931 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.931+0000 7f6fe63f6600 20 add_watcher() i=4 2022-09-03T05:16:50.931 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.931+0000 7f6fe63f6600 20 add_watcher() i=5 2022-09-03T05:16:50.931 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.931+0000 7f6fe63f6600 20 add_watcher() i=6 2022-09-03T05:16:50.932 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.932+0000 7f6fe63f6600 20 add_watcher() i=7 2022-09-03T05:16:50.932 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.932+0000 7f6fe63f6600 2 all 8 watchers are set, enabling cache 2022-09-03T05:16:50.933 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.932+0000 7f6fe63f6600 5 tl::expected, obj_version>, boost::system::error_code> logback_generations::read(const DoutPrefixProvider*, optional_yield):410: oid=data_loggenerations_metadata not found 2022-09-03T05:16:50.933 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.932+0000 7f6fe63f6600 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.0 2022-09-03T05:16:50.934 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.932+0000 7f6fe63f6600 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1161 entering 2022-09-03T05:16:50.937 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.932+0000 7f6fe63f6600 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.0 does not exist 2022-09-03T05:16:50.937 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.932+0000 7f6fe63f6600 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.1 2022-09-03T05:16:50.938 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.932+0000 7f6fe63f6600 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1161 entering 2022-09-03T05:16:50.938 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.932+0000 7f6fe63f6600 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.1 does not exist 2022-09-03T05:16:50.938 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.932+0000 7f6fe63f6600 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.2 2022-09-03T05:16:50.938 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.932+0000 7f6fe63f6600 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1161 entering 2022-09-03T05:16:50.939 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.933+0000 7f6fe63f6600 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.2 does not exist 2022-09-03T05:16:50.939 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.933+0000 7f6fe63f6600 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.3 2022-09-03T05:16:50.939 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.933+0000 7f6fe63f6600 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1161 entering 2022-09-03T05:16:50.939 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.933+0000 7f6fe63f6600 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.3 does not exist 2022-09-03T05:16:50.940 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.933+0000 7f6fe63f6600 20 static int rgw::cls::fifo::FIFO::create(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, std::optional >, bool, uint64_t, uint64_t):1203 entering 2022-09-03T05:16:50.940 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.934+0000 7f6fe63f6600 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1161 entering 2022-09-03T05:16:50.940 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.937+0000 7f6fceb77700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2022-09-03T05:16:50.941 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.937+0000 7f6fe63f6600 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2022-09-03T05:16:50.941 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:50.937+0000 7f6fe63f6600 20 check_secure_mon_conn(): mode 1 is insecure 2022-09-03T05:16:53.914 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:53.916+0000 7f6fe63f6600 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.meta 2022-09-03T05:16:53.915 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:53.916+0000 7f6fe63f6600 5 note: GC not initialized 2022-09-03T05:16:53.916 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:53.917+0000 7f6fca36e700 20 reqs_thread_entry: start 2022-09-03T05:16:53.917 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:53.917+0000 7f6fe63f6600 20 init_complete bucket index max shards: 11 2022-09-03T05:16:53.918 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:53.917+0000 7f6fe63f6600 10 rgw notify: Started notification manager with: 1 workers 2022-09-03T05:16:53.918 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:53.917+0000 7f6fc836a700 20 rgw notify: INFO: next queues processing will happen at: Sat Sep 3 05:17:24 2022 2022-09-03T05:16:53.918 INFO:teuthology.orchestra.run.smithi181.stderr: 2022-09-03T05:16:53.919 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:53.917+0000 7f6fc7b48700 20 reqs_thread_entry: start 2022-09-03T05:16:53.939 INFO:teuthology.orchestra.run.smithi181.stdout:[] 2022-09-03T05:16:53.940 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:53.940+0000 7f6fe63f6600 20 remove_watcher() i=0 2022-09-03T05:16:53.940 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:53.940+0000 7f6fe63f6600 2 removed watcher, disabling cache 2022-09-03T05:16:53.941 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:53.943+0000 7f6fe63f6600 20 remove_watcher() i=1 2022-09-03T05:16:53.945 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:53.944+0000 7f6fe63f6600 20 remove_watcher() i=2 2022-09-03T05:16:53.946 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:53.946+0000 7f6fe63f6600 20 remove_watcher() i=3 2022-09-03T05:16:53.947 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:53.949+0000 7f6fe63f6600 20 remove_watcher() i=4 2022-09-03T05:16:53.948 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:53.950+0000 7f6fe63f6600 20 remove_watcher() i=5 2022-09-03T05:16:53.949 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:53.951+0000 7f6fe63f6600 20 remove_watcher() i=6 2022-09-03T05:16:53.951 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:53.952+0000 7f6fe63f6600 20 remove_watcher() i=7 2022-09-03T05:16:53.951 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:53.953+0000 7f6fceb77700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2022-09-03T05:16:53.959 DEBUG:tasks.util.rgw: json result: [] 2022-09-03T05:16:53.960 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2022-09-03T05:16:53.961 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2022-09-03T05:16:53.961 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone placement modify --rgw-zone default --placement-id default-placement --compression random 2022-09-03T05:16:54.016 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:16:54.017 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:16:54.022 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.024+0000 7f1317e9f600 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2022-09-03T05:16:54.023 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.024+0000 7f1317e9f600 20 check_secure_mon_conn(): mode 1 is insecure 2022-09-03T05:16:54.027 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.028+0000 7f12fce19700 20 reqs_thread_entry: start 2022-09-03T05:16:54.028 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.028+0000 7f1317e9f600 20 try_to_resolve_local_entities(): before: realm_id= realm_name= zonegroup_id= zonegroup_name= zone_id= zone_name=default 2022-09-03T05:16:54.028 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.028+0000 7f1317e9f600 20 rados->read ofs=0 len=0 2022-09-03T05:16:54.029 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.028+0000 7f1317e9f600 20 rados_obj.operate() r=0 bl.length=46 2022-09-03T05:16:54.029 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.028+0000 7f1317e9f600 20 rados->read ofs=0 len=0 2022-09-03T05:16:54.029 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.029+0000 7f1317e9f600 20 rados_obj.operate() r=0 bl.length=873 2022-09-03T05:16:54.042 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.036+0000 7f1317e9f600 20 RGWRados::pool_iterate: got zonegroup_info.9ad14a2b-08e7-4a70-91e3-2b1956365d5d 2022-09-03T05:16:54.043 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.036+0000 7f1317e9f600 20 RGWRados::pool_iterate: got zone_info.ce9d29af-1d2c-44d9-98e6-223162653e78 2022-09-03T05:16:54.043 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.036+0000 7f1317e9f600 20 RGWRados::pool_iterate: got zone_names.default 2022-09-03T05:16:54.043 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.036+0000 7f1317e9f600 20 RGWRados::pool_iterate: got zonegroups_names.default 2022-09-03T05:16:54.044 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.036+0000 7f1317e9f600 20 rados->read ofs=0 len=0 2022-09-03T05:16:54.044 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.037+0000 7f1317e9f600 20 rados_obj.operate() r=0 bl.length=873 2022-09-03T05:16:54.045 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.037+0000 7f1317e9f600 20 rados->read ofs=0 len=0 2022-09-03T05:16:54.045 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.037+0000 7f1317e9f600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:54.046 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.037+0000 7f1317e9f600 20 rados->read ofs=0 len=0 2022-09-03T05:16:54.047 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.037+0000 7f1317e9f600 20 rados_obj.operate() r=0 bl.length=46 2022-09-03T05:16:54.047 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.037+0000 7f1317e9f600 20 rados->read ofs=0 len=0 2022-09-03T05:16:54.047 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.037+0000 7f1317e9f600 20 rados_obj.operate() r=0 bl.length=398 2022-09-03T05:16:54.050 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:16:54.050 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "ce9d29af-1d2c-44d9-98e6-223162653e78", 2022-09-03T05:16:54.051 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "default", 2022-09-03T05:16:54.051 INFO:teuthology.orchestra.run.smithi181.stdout: "domain_root": "default.rgw.meta:root", 2022-09-03T05:16:54.052 INFO:teuthology.orchestra.run.smithi181.stdout: "control_pool": "default.rgw.control", 2022-09-03T05:16:54.052 INFO:teuthology.orchestra.run.smithi181.stdout: "gc_pool": "default.rgw.log:gc", 2022-09-03T05:16:54.053 INFO:teuthology.orchestra.run.smithi181.stdout: "lc_pool": "default.rgw.log:lc", 2022-09-03T05:16:54.053 INFO:teuthology.orchestra.run.smithi181.stdout: "log_pool": "default.rgw.log", 2022-09-03T05:16:54.054 INFO:teuthology.orchestra.run.smithi181.stdout: "intent_log_pool": "default.rgw.log:intent", 2022-09-03T05:16:54.054 INFO:teuthology.orchestra.run.smithi181.stdout: "usage_log_pool": "default.rgw.log:usage", 2022-09-03T05:16:54.054 INFO:teuthology.orchestra.run.smithi181.stdout: "roles_pool": "default.rgw.meta:roles", 2022-09-03T05:16:54.055 INFO:teuthology.orchestra.run.smithi181.stdout: "reshard_pool": "default.rgw.log:reshard", 2022-09-03T05:16:54.056 INFO:teuthology.orchestra.run.smithi181.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2022-09-03T05:16:54.056 INFO:teuthology.orchestra.run.smithi181.stdout: "user_email_pool": "default.rgw.meta:users.email", 2022-09-03T05:16:54.056 INFO:teuthology.orchestra.run.smithi181.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2022-09-03T05:16:54.057 INFO:teuthology.orchestra.run.smithi181.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2022-09-03T05:16:54.058 INFO:teuthology.orchestra.run.smithi181.stdout: "otp_pool": "default.rgw.otp", 2022-09-03T05:16:54.058 INFO:teuthology.orchestra.run.smithi181.stdout: "system_key": { 2022-09-03T05:16:54.058 INFO:teuthology.orchestra.run.smithi181.stdout: "access_key": "", 2022-09-03T05:16:54.059 INFO:teuthology.orchestra.run.smithi181.stdout: "secret_key": "" 2022-09-03T05:16:54.059 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:16:54.060 INFO:teuthology.orchestra.run.smithi181.stdout: "placement_pools": [ 2022-09-03T05:16:54.060 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:16:54.060 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "default-placement", 2022-09-03T05:16:54.060 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:16:54.061 INFO:teuthology.orchestra.run.smithi181.stdout: "index_pool": "default.rgw.buckets.index", 2022-09-03T05:16:54.061 INFO:teuthology.orchestra.run.smithi181.stdout: "storage_classes": { 2022-09-03T05:16:54.061 INFO:teuthology.orchestra.run.smithi181.stdout: "STANDARD": { 2022-09-03T05:16:54.061 INFO:teuthology.orchestra.run.smithi181.stdout: "data_pool": "default.rgw.buckets.data", 2022-09-03T05:16:54.062 INFO:teuthology.orchestra.run.smithi181.stdout: "compression_type": "random" 2022-09-03T05:16:54.062 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:16:54.062 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:16:54.063 INFO:teuthology.orchestra.run.smithi181.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2022-09-03T05:16:54.063 INFO:teuthology.orchestra.run.smithi181.stdout: "index_type": 0 2022-09-03T05:16:54.063 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:16:54.063 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:16:54.063 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:16:54.064 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "", 2022-09-03T05:16:54.064 INFO:teuthology.orchestra.run.smithi181.stdout: "notif_pool": "default.rgw.log:notif" 2022-09-03T05:16:54.064 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:16:54.065 DEBUG:tasks.util.rgw: json result: {'id': 'ce9d29af-1d2c-44d9-98e6-223162653e78', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2022-09-03T05:16:54.065 INFO:tasks.util.rgw:rgwadmin: c1.client.1 : ['user', 'list'] 2022-09-03T05:16:54.066 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.1', '--cluster', 'c1', 'user', 'list'] 2022-09-03T05:16:54.066 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.1 --cluster c1 user list 2022-09-03T05:16:54.112 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:16:54.112 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:16:54.120 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.118+0000 7f8e1511a600 20 rados->read ofs=0 len=0 2022-09-03T05:16:54.121 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.119+0000 7f8e1511a600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:54.121 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.119+0000 7f8e1511a600 20 realm 2022-09-03T05:16:54.122 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.119+0000 7f8e1511a600 20 rados->read ofs=0 len=0 2022-09-03T05:16:54.128 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.120+0000 7f8e1511a600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:54.128 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.120+0000 7f8e1511a600 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2022-09-03T05:16:54.128 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.120+0000 7f8e1511a600 20 rados->read ofs=0 len=0 2022-09-03T05:16:54.129 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.120+0000 7f8e1511a600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:54.129 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.120+0000 7f8e1511a600 20 rados->read ofs=0 len=0 2022-09-03T05:16:54.129 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.120+0000 7f8e1511a600 20 rados_obj.operate() r=0 bl.length=46 2022-09-03T05:16:54.130 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.120+0000 7f8e1511a600 20 rados->read ofs=0 len=0 2022-09-03T05:16:54.130 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.120+0000 7f8e1511a600 20 rados_obj.operate() r=0 bl.length=889 2022-09-03T05:16:54.130 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.120+0000 7f8e1511a600 20 searching for the correct realm 2022-09-03T05:16:54.134 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.127+0000 7f8e1511a600 20 RGWRados::pool_iterate: got zonegroup_info.9ad14a2b-08e7-4a70-91e3-2b1956365d5d 2022-09-03T05:16:54.134 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.127+0000 7f8e1511a600 20 RGWRados::pool_iterate: got zone_info.ce9d29af-1d2c-44d9-98e6-223162653e78 2022-09-03T05:16:54.134 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.127+0000 7f8e1511a600 20 RGWRados::pool_iterate: got zone_names.default 2022-09-03T05:16:54.135 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.127+0000 7f8e1511a600 20 RGWRados::pool_iterate: got zonegroups_names.default 2022-09-03T05:16:54.135 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.127+0000 7f8e1511a600 20 rados->read ofs=0 len=0 2022-09-03T05:16:54.135 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.127+0000 7f8e1511a600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:54.136 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.127+0000 7f8e1511a600 20 rados->read ofs=0 len=0 2022-09-03T05:16:54.136 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.127+0000 7f8e1511a600 20 rados_obj.operate() r=0 bl.length=46 2022-09-03T05:16:54.136 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.127+0000 7f8e1511a600 20 rados->read ofs=0 len=0 2022-09-03T05:16:54.136 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.128+0000 7f8e1511a600 20 rados_obj.operate() r=0 bl.length=398 2022-09-03T05:16:54.137 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.128+0000 7f8e1511a600 20 zone default found 2022-09-03T05:16:54.137 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.128+0000 7f8e1511a600 4 Realm: () 2022-09-03T05:16:54.137 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.128+0000 7f8e1511a600 4 ZoneGroup: default (9ad14a2b-08e7-4a70-91e3-2b1956365d5d) 2022-09-03T05:16:54.137 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.128+0000 7f8e1511a600 4 Zone: default (ce9d29af-1d2c-44d9-98e6-223162653e78) 2022-09-03T05:16:54.138 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.128+0000 7f8e1511a600 10 cannot find current period zonegroup using local zonegroup configuration 2022-09-03T05:16:54.138 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.128+0000 7f8e1511a600 20 zonegroup default 2022-09-03T05:16:54.139 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.128+0000 7f8e1511a600 20 rados->read ofs=0 len=0 2022-09-03T05:16:54.139 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.128+0000 7f8e1511a600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:54.140 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.128+0000 7f8e1511a600 20 rados->read ofs=0 len=0 2022-09-03T05:16:54.140 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.128+0000 7f8e1511a600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:54.141 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.128+0000 7f8e1511a600 20 rados->read ofs=0 len=0 2022-09-03T05:16:54.141 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.128+0000 7f8e1511a600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:54.141 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.128+0000 7f8e1511a600 20 started sync module instance, tier type = 2022-09-03T05:16:54.142 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.128+0000 7f8e1511a600 20 started zone id=ce9d29af-1d2c-44d9-98e6-223162653e78 (name=default) with tier type = 2022-09-03T05:16:54.145 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.137+0000 7f8e1511a600 20 add_watcher() i=0 2022-09-03T05:16:54.145 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.137+0000 7f8e1511a600 20 add_watcher() i=1 2022-09-03T05:16:54.146 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.137+0000 7f8e1511a600 20 add_watcher() i=2 2022-09-03T05:16:54.147 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.137+0000 7f8e1511a600 20 add_watcher() i=3 2022-09-03T05:16:54.147 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.137+0000 7f8e1511a600 20 add_watcher() i=4 2022-09-03T05:16:54.148 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.137+0000 7f8e1511a600 20 add_watcher() i=5 2022-09-03T05:16:54.148 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.137+0000 7f8e1511a600 20 add_watcher() i=6 2022-09-03T05:16:54.148 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.138+0000 7f8e1511a600 20 add_watcher() i=7 2022-09-03T05:16:54.148 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.138+0000 7f8e1511a600 2 all 8 watchers are set, enabling cache 2022-09-03T05:16:54.149 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.139+0000 7f8dfd89b700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2022-09-03T05:16:54.149 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.139+0000 7f8e1511a600 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2022-09-03T05:16:54.149 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.139+0000 7f8e1511a600 20 check_secure_mon_conn(): mode 1 is insecure 2022-09-03T05:16:54.149 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.143+0000 7f8e1511a600 5 note: GC not initialized 2022-09-03T05:16:54.150 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.144+0000 7f8df9092700 20 reqs_thread_entry: start 2022-09-03T05:16:54.150 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.144+0000 7f8e1511a600 20 init_complete bucket index max shards: 11 2022-09-03T05:16:54.150 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.144+0000 7f8e1511a600 10 rgw notify: Started notification manager with: 1 workers 2022-09-03T05:16:54.150 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.144+0000 7f8df708e700 20 rgw notify: INFO: next queues processing will happen at: Sat Sep 3 05:17:24 2022 2022-09-03T05:16:54.151 INFO:teuthology.orchestra.run.smithi181.stderr: 2022-09-03T05:16:54.151 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.144+0000 7f8df686c700 20 reqs_thread_entry: start 2022-09-03T05:16:54.152 INFO:teuthology.orchestra.run.smithi181.stdout:[] 2022-09-03T05:16:54.153 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.151+0000 7f8e1511a600 20 remove_watcher() i=0 2022-09-03T05:16:54.154 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.151+0000 7f8e1511a600 2 removed watcher, disabling cache 2022-09-03T05:16:54.154 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.152+0000 7f8e1511a600 20 remove_watcher() i=1 2022-09-03T05:16:54.154 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.153+0000 7f8e1511a600 20 remove_watcher() i=2 2022-09-03T05:16:54.154 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.153+0000 7f8e1511a600 20 remove_watcher() i=3 2022-09-03T05:16:54.155 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.154+0000 7f8e1511a600 20 remove_watcher() i=4 2022-09-03T05:16:54.155 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.155+0000 7f8e1511a600 20 remove_watcher() i=5 2022-09-03T05:16:54.156 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.156+0000 7f8e1511a600 20 remove_watcher() i=6 2022-09-03T05:16:54.156 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.156+0000 7f8e1511a600 20 remove_watcher() i=7 2022-09-03T05:16:54.156 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.157+0000 7f8dfd89b700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2022-09-03T05:16:54.159 DEBUG:tasks.util.rgw: json result: [] 2022-09-03T05:16:54.160 INFO:tasks.util.rgw:rgwadmin: c1.client.1 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2022-09-03T05:16:54.160 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.1', '--cluster', 'c1', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2022-09-03T05:16:54.161 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.1 --cluster c1 zone placement modify --rgw-zone default --placement-id default-placement --compression random 2022-09-03T05:16:54.251 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:16:54.252 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:16:54.257 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.258+0000 7f19ebf90600 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2022-09-03T05:16:54.257 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.258+0000 7f19ebf90600 20 check_secure_mon_conn(): mode 1 is insecure 2022-09-03T05:16:54.261 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.262+0000 7f19d0f0a700 20 reqs_thread_entry: start 2022-09-03T05:16:54.261 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.262+0000 7f19ebf90600 20 try_to_resolve_local_entities(): before: realm_id= realm_name= zonegroup_id= zonegroup_name= zone_id= zone_name=default 2022-09-03T05:16:54.261 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.262+0000 7f19ebf90600 20 rados->read ofs=0 len=0 2022-09-03T05:16:54.262 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.263+0000 7f19ebf90600 20 rados_obj.operate() r=0 bl.length=46 2022-09-03T05:16:54.262 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.263+0000 7f19ebf90600 20 rados->read ofs=0 len=0 2022-09-03T05:16:54.263 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.263+0000 7f19ebf90600 20 rados_obj.operate() r=0 bl.length=889 2022-09-03T05:16:54.268 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.270+0000 7f19ebf90600 20 RGWRados::pool_iterate: got zonegroup_info.9ad14a2b-08e7-4a70-91e3-2b1956365d5d 2022-09-03T05:16:54.268 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.270+0000 7f19ebf90600 20 RGWRados::pool_iterate: got zone_info.ce9d29af-1d2c-44d9-98e6-223162653e78 2022-09-03T05:16:54.269 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.270+0000 7f19ebf90600 20 RGWRados::pool_iterate: got zone_names.default 2022-09-03T05:16:54.269 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.270+0000 7f19ebf90600 20 RGWRados::pool_iterate: got zonegroups_names.default 2022-09-03T05:16:54.269 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.270+0000 7f19ebf90600 20 rados->read ofs=0 len=0 2022-09-03T05:16:54.275 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.270+0000 7f19ebf90600 20 rados_obj.operate() r=0 bl.length=889 2022-09-03T05:16:54.275 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.270+0000 7f19ebf90600 20 rados->read ofs=0 len=0 2022-09-03T05:16:54.275 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.270+0000 7f19ebf90600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:54.276 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.270+0000 7f19ebf90600 20 rados->read ofs=0 len=0 2022-09-03T05:16:54.276 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.270+0000 7f19ebf90600 20 rados_obj.operate() r=0 bl.length=46 2022-09-03T05:16:54.276 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.270+0000 7f19ebf90600 20 rados->read ofs=0 len=0 2022-09-03T05:16:54.276 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:16:54.271+0000 7f19ebf90600 20 rados_obj.operate() r=0 bl.length=398 2022-09-03T05:16:54.277 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:16:54.278 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "ce9d29af-1d2c-44d9-98e6-223162653e78", 2022-09-03T05:16:54.278 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "default", 2022-09-03T05:16:54.278 INFO:teuthology.orchestra.run.smithi181.stdout: "domain_root": "default.rgw.meta:root", 2022-09-03T05:16:54.278 INFO:teuthology.orchestra.run.smithi181.stdout: "control_pool": "default.rgw.control", 2022-09-03T05:16:54.279 INFO:teuthology.orchestra.run.smithi181.stdout: "gc_pool": "default.rgw.log:gc", 2022-09-03T05:16:54.279 INFO:teuthology.orchestra.run.smithi181.stdout: "lc_pool": "default.rgw.log:lc", 2022-09-03T05:16:54.279 INFO:teuthology.orchestra.run.smithi181.stdout: "log_pool": "default.rgw.log", 2022-09-03T05:16:54.279 INFO:teuthology.orchestra.run.smithi181.stdout: "intent_log_pool": "default.rgw.log:intent", 2022-09-03T05:16:54.280 INFO:teuthology.orchestra.run.smithi181.stdout: "usage_log_pool": "default.rgw.log:usage", 2022-09-03T05:16:54.280 INFO:teuthology.orchestra.run.smithi181.stdout: "roles_pool": "default.rgw.meta:roles", 2022-09-03T05:16:54.280 INFO:teuthology.orchestra.run.smithi181.stdout: "reshard_pool": "default.rgw.log:reshard", 2022-09-03T05:16:54.280 INFO:teuthology.orchestra.run.smithi181.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2022-09-03T05:16:54.281 INFO:teuthology.orchestra.run.smithi181.stdout: "user_email_pool": "default.rgw.meta:users.email", 2022-09-03T05:16:54.281 INFO:teuthology.orchestra.run.smithi181.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2022-09-03T05:16:54.281 INFO:teuthology.orchestra.run.smithi181.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2022-09-03T05:16:54.281 INFO:teuthology.orchestra.run.smithi181.stdout: "otp_pool": "default.rgw.otp", 2022-09-03T05:16:54.282 INFO:teuthology.orchestra.run.smithi181.stdout: "system_key": { 2022-09-03T05:16:54.282 INFO:teuthology.orchestra.run.smithi181.stdout: "access_key": "", 2022-09-03T05:16:54.282 INFO:teuthology.orchestra.run.smithi181.stdout: "secret_key": "" 2022-09-03T05:16:54.282 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:16:54.282 INFO:teuthology.orchestra.run.smithi181.stdout: "placement_pools": [ 2022-09-03T05:16:54.283 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:16:54.283 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "default-placement", 2022-09-03T05:16:54.283 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:16:54.283 INFO:teuthology.orchestra.run.smithi181.stdout: "index_pool": "default.rgw.buckets.index", 2022-09-03T05:16:54.284 INFO:teuthology.orchestra.run.smithi181.stdout: "storage_classes": { 2022-09-03T05:16:54.284 INFO:teuthology.orchestra.run.smithi181.stdout: "STANDARD": { 2022-09-03T05:16:54.284 INFO:teuthology.orchestra.run.smithi181.stdout: "data_pool": "default.rgw.buckets.data", 2022-09-03T05:16:54.284 INFO:teuthology.orchestra.run.smithi181.stdout: "compression_type": "random" 2022-09-03T05:16:54.285 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:16:54.285 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:16:54.285 INFO:teuthology.orchestra.run.smithi181.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2022-09-03T05:16:54.285 INFO:teuthology.orchestra.run.smithi181.stdout: "index_type": 0 2022-09-03T05:16:54.286 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:16:54.286 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:16:54.286 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:16:54.286 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "", 2022-09-03T05:16:54.286 INFO:teuthology.orchestra.run.smithi181.stdout: "notif_pool": "default.rgw.log:notif" 2022-09-03T05:16:54.287 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:16:54.287 DEBUG:tasks.util.rgw: json result: {'id': 'ce9d29af-1d2c-44d9-98e6-223162653e78', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2022-09-03T05:16:54.288 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['user', 'list'] 2022-09-03T05:16:54.288 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'user', 'list'] 2022-09-03T05:16:54.288 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 user list 2022-09-03T05:16:54.367 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:16:54.368 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:16:54.374 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:54.376+0000 7fe5d4f11600 20 rados->read ofs=0 len=0 2022-09-03T05:16:56.153 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.154+0000 7fe5d4f11600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:56.154 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.154+0000 7fe5d4f11600 20 realm 2022-09-03T05:16:56.154 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.154+0000 7fe5d4f11600 20 rados->read ofs=0 len=0 2022-09-03T05:16:56.154 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.155+0000 7fe5d4f11600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:56.155 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.155+0000 7fe5d4f11600 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2022-09-03T05:16:56.155 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.155+0000 7fe5d4f11600 20 rados->read ofs=0 len=0 2022-09-03T05:16:56.155 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.155+0000 7fe5d4f11600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:56.155 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.155+0000 7fe5d4f11600 20 rados->read ofs=0 len=0 2022-09-03T05:16:56.157 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.157+0000 7fe5d4f11600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:56.157 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.157+0000 7fe5d4f11600 20 rados->read ofs=0 len=0 2022-09-03T05:16:56.157 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.157+0000 7fe5d4f11600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:56.158 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.157+0000 7fe5d4f11600 20 rados->read ofs=0 len=0 2022-09-03T05:16:56.158 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.158+0000 7fe5d4f11600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:56.158 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.158+0000 7fe5d4f11600 10 Creating default zonegroup 2022-09-03T05:16:56.159 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.158+0000 7fe5d4f11600 10 couldn't find old data placement pools config, setting up new ones for the zone 2022-09-03T05:16:56.169 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.168+0000 7fe5d4f11600 10 WARNING: store->list_zones() returned r=-2 2022-09-03T05:16:56.169 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.168+0000 7fe5d4f11600 20 rados->read ofs=0 len=0 2022-09-03T05:16:56.169 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.168+0000 7fe5d4f11600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:56.170 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.170+0000 7fe5d4f11600 20 rados->read ofs=0 len=0 2022-09-03T05:16:56.170 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.170+0000 7fe5d4f11600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:56.170 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.170+0000 7fe5d4f11600 10 could not read realm id: (2) No such file or directory 2022-09-03T05:16:56.170 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.170+0000 7fe5d4f11600 10 WARNING: failed to set zone as default, r=-22 2022-09-03T05:16:56.171 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.170+0000 7fe5d4f11600 20 rados->read ofs=0 len=0 2022-09-03T05:16:56.171 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.171+0000 7fe5d4f11600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:56.173 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.173+0000 7fe5d4f11600 20 rados->read ofs=0 len=0 2022-09-03T05:16:56.173 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.173+0000 7fe5d4f11600 20 rados_obj.operate() r=0 bl.length=873 2022-09-03T05:16:56.173 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.173+0000 7fe5d4f11600 20 rados->read ofs=0 len=0 2022-09-03T05:16:56.174 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.174+0000 7fe5d4f11600 20 rados_obj.operate() r=0 bl.length=398 2022-09-03T05:16:56.174 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.174+0000 7fe5d4f11600 10 Using default name default 2022-09-03T05:16:56.174 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.174+0000 7fe5d4f11600 20 rados->read ofs=0 len=0 2022-09-03T05:16:56.174 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.174+0000 7fe5d4f11600 20 rados_obj.operate() r=0 bl.length=46 2022-09-03T05:16:56.175 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.174+0000 7fe5d4f11600 20 rados->read ofs=0 len=0 2022-09-03T05:16:56.175 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.174+0000 7fe5d4f11600 20 rados_obj.operate() r=0 bl.length=873 2022-09-03T05:16:56.175 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.174+0000 7fe5d4f11600 20 zone default found 2022-09-03T05:16:56.176 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.174+0000 7fe5d4f11600 4 Realm: () 2022-09-03T05:16:56.177 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.174+0000 7fe5d4f11600 4 ZoneGroup: default (224dab99-5f24-4511-9f76-68bf9b1b459e) 2022-09-03T05:16:56.177 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.174+0000 7fe5d4f11600 4 Zone: default (ef6feded-abd7-44e7-8115-4d2abed1a3da) 2022-09-03T05:16:56.178 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.174+0000 7fe5d4f11600 10 cannot find current period zonegroup using local zonegroup configuration 2022-09-03T05:16:56.178 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.174+0000 7fe5d4f11600 20 zonegroup default 2022-09-03T05:16:56.179 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.174+0000 7fe5d4f11600 20 rados->read ofs=0 len=0 2022-09-03T05:16:56.179 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.174+0000 7fe5d4f11600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:56.179 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:56.175+0000 7fe5d4f11600 20 rados->read ofs=0 len=0 2022-09-03T05:16:58.160 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:58.162+0000 7fe5d4f11600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:58.161 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:58.162+0000 7fe5d4f11600 20 rados->read ofs=0 len=0 2022-09-03T05:16:58.161 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:58.162+0000 7fe5d4f11600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:16:58.161 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:58.162+0000 7fe5d4f11600 20 started sync module instance, tier type = 2022-09-03T05:16:58.161 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:16:58.162+0000 7fe5d4f11600 20 started zone id=ef6feded-abd7-44e7-8115-4d2abed1a3da (name=default) with tier type = 2022-09-03T05:17:00.211 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.213+0000 7fe5d4f11600 20 add_watcher() i=0 2022-09-03T05:17:00.212 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.213+0000 7fe5d4f11600 20 add_watcher() i=1 2022-09-03T05:17:00.213 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.213+0000 7fe5d4f11600 20 add_watcher() i=2 2022-09-03T05:17:00.214 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.213+0000 7fe5d4f11600 20 add_watcher() i=3 2022-09-03T05:17:00.214 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.213+0000 7fe5d4f11600 20 add_watcher() i=4 2022-09-03T05:17:00.214 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.213+0000 7fe5d4f11600 20 add_watcher() i=5 2022-09-03T05:17:00.215 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.213+0000 7fe5d4f11600 20 add_watcher() i=6 2022-09-03T05:17:00.217 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.214+0000 7fe5d4f11600 20 add_watcher() i=7 2022-09-03T05:17:00.218 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.214+0000 7fe5d4f11600 2 all 8 watchers are set, enabling cache 2022-09-03T05:17:00.218 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.214+0000 7fe5d4f11600 5 tl::expected, obj_version>, boost::system::error_code> logback_generations::read(const DoutPrefixProvider*, optional_yield):410: oid=data_loggenerations_metadata not found 2022-09-03T05:17:00.219 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.214+0000 7fe5d4f11600 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.0 2022-09-03T05:17:00.219 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.214+0000 7fe5d4f11600 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1161 entering 2022-09-03T05:17:00.219 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.214+0000 7fe5d4f11600 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.0 does not exist 2022-09-03T05:17:00.219 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.214+0000 7fe5d4f11600 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.1 2022-09-03T05:17:00.220 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.214+0000 7fe5d4f11600 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1161 entering 2022-09-03T05:17:00.220 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.215+0000 7fe5d4f11600 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.1 does not exist 2022-09-03T05:17:00.220 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.215+0000 7fe5d4f11600 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.2 2022-09-03T05:17:00.220 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.215+0000 7fe5d4f11600 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1161 entering 2022-09-03T05:17:00.221 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.215+0000 7fe5d4f11600 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.2 does not exist 2022-09-03T05:17:00.221 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.215+0000 7fe5d4f11600 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.3 2022-09-03T05:17:00.221 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.215+0000 7fe5d4f11600 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1161 entering 2022-09-03T05:17:00.221 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.215+0000 7fe5d4f11600 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.3 does not exist 2022-09-03T05:17:00.222 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.215+0000 7fe5d4f11600 20 static int rgw::cls::fifo::FIFO::create(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, std::optional >, bool, uint64_t, uint64_t):1203 entering 2022-09-03T05:17:00.222 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.219+0000 7fe5d4f11600 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1161 entering 2022-09-03T05:17:00.223 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.221+0000 7fe5bd692700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2022-09-03T05:17:00.223 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.221+0000 7fe5d4f11600 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2022-09-03T05:17:00.224 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:00.221+0000 7fe5d4f11600 20 check_secure_mon_conn(): mode 1 is insecure 2022-09-03T05:17:03.177 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.176+0000 7fe5d4f11600 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.meta 2022-09-03T05:17:03.177 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.176+0000 7fe5d4f11600 5 note: GC not initialized 2022-09-03T05:17:03.177 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.178+0000 7fe5b8e89700 20 reqs_thread_entry: start 2022-09-03T05:17:03.178 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.178+0000 7fe5d4f11600 20 init_complete bucket index max shards: 11 2022-09-03T05:17:03.178 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.178+0000 7fe5d4f11600 10 rgw notify: Started notification manager with: 1 workers 2022-09-03T05:17:03.178 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.178+0000 7fe5b6e85700 20 rgw notify: INFO: next queues processing will happen at: Sat Sep 3 05:17:33 2022 2022-09-03T05:17:03.178 INFO:teuthology.orchestra.run.smithi189.stderr: 2022-09-03T05:17:03.179 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.178+0000 7fe5b6684700 20 reqs_thread_entry: start 2022-09-03T05:17:03.196 INFO:teuthology.orchestra.run.smithi189.stdout:[] 2022-09-03T05:17:03.197 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.196+0000 7fe5d4f11600 20 remove_watcher() i=0 2022-09-03T05:17:03.197 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.196+0000 7fe5d4f11600 2 removed watcher, disabling cache 2022-09-03T05:17:03.198 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.197+0000 7fe5d4f11600 20 remove_watcher() i=1 2022-09-03T05:17:03.198 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.198+0000 7fe5d4f11600 20 remove_watcher() i=2 2022-09-03T05:17:03.201 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.199+0000 7fe5d4f11600 20 remove_watcher() i=3 2022-09-03T05:17:03.201 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.200+0000 7fe5d4f11600 20 remove_watcher() i=4 2022-09-03T05:17:03.202 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.201+0000 7fe5d4f11600 20 remove_watcher() i=5 2022-09-03T05:17:03.202 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.202+0000 7fe5d4f11600 20 remove_watcher() i=6 2022-09-03T05:17:03.203 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.203+0000 7fe5d4f11600 20 remove_watcher() i=7 2022-09-03T05:17:03.203 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.204+0000 7fe5bd692700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2022-09-03T05:17:03.208 DEBUG:tasks.util.rgw: json result: [] 2022-09-03T05:17:03.208 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2022-09-03T05:17:03.209 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2022-09-03T05:17:03.209 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone placement modify --rgw-zone default --placement-id default-placement --compression random 2022-09-03T05:17:03.264 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:17:03.264 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:17:03.269 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.271+0000 7f611d85b600 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2022-09-03T05:17:03.270 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.271+0000 7f611d85b600 20 check_secure_mon_conn(): mode 1 is insecure 2022-09-03T05:17:03.275 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.274+0000 7f61027d5700 20 reqs_thread_entry: start 2022-09-03T05:17:03.275 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.274+0000 7f611d85b600 20 try_to_resolve_local_entities(): before: realm_id= realm_name= zonegroup_id= zonegroup_name= zone_id= zone_name=default 2022-09-03T05:17:03.276 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.274+0000 7f611d85b600 20 rados->read ofs=0 len=0 2022-09-03T05:17:03.276 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.275+0000 7f611d85b600 20 rados_obj.operate() r=0 bl.length=46 2022-09-03T05:17:03.277 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.275+0000 7f611d85b600 20 rados->read ofs=0 len=0 2022-09-03T05:17:03.277 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.276+0000 7f611d85b600 20 rados_obj.operate() r=0 bl.length=873 2022-09-03T05:17:03.285 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.284+0000 7f611d85b600 20 RGWRados::pool_iterate: got zone_info.ef6feded-abd7-44e7-8115-4d2abed1a3da 2022-09-03T05:17:03.285 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.284+0000 7f611d85b600 20 RGWRados::pool_iterate: got zonegroup_info.224dab99-5f24-4511-9f76-68bf9b1b459e 2022-09-03T05:17:03.285 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.284+0000 7f611d85b600 20 RGWRados::pool_iterate: got zone_names.default 2022-09-03T05:17:03.286 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.284+0000 7f611d85b600 20 RGWRados::pool_iterate: got zonegroups_names.default 2022-09-03T05:17:03.286 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.284+0000 7f611d85b600 20 rados->read ofs=0 len=0 2022-09-03T05:17:03.286 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.284+0000 7f611d85b600 20 rados_obj.operate() r=0 bl.length=873 2022-09-03T05:17:03.286 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.284+0000 7f611d85b600 20 rados->read ofs=0 len=0 2022-09-03T05:17:03.287 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.284+0000 7f611d85b600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:17:03.287 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.284+0000 7f611d85b600 20 rados->read ofs=0 len=0 2022-09-03T05:17:03.289 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.284+0000 7f611d85b600 20 rados_obj.operate() r=0 bl.length=46 2022-09-03T05:17:03.290 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.284+0000 7f611d85b600 20 rados->read ofs=0 len=0 2022-09-03T05:17:03.290 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.285+0000 7f611d85b600 20 rados_obj.operate() r=0 bl.length=398 2022-09-03T05:17:03.292 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:17:03.293 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "ef6feded-abd7-44e7-8115-4d2abed1a3da", 2022-09-03T05:17:03.294 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "default", 2022-09-03T05:17:03.294 INFO:teuthology.orchestra.run.smithi189.stdout: "domain_root": "default.rgw.meta:root", 2022-09-03T05:17:03.294 INFO:teuthology.orchestra.run.smithi189.stdout: "control_pool": "default.rgw.control", 2022-09-03T05:17:03.295 INFO:teuthology.orchestra.run.smithi189.stdout: "gc_pool": "default.rgw.log:gc", 2022-09-03T05:17:03.295 INFO:teuthology.orchestra.run.smithi189.stdout: "lc_pool": "default.rgw.log:lc", 2022-09-03T05:17:03.295 INFO:teuthology.orchestra.run.smithi189.stdout: "log_pool": "default.rgw.log", 2022-09-03T05:17:03.295 INFO:teuthology.orchestra.run.smithi189.stdout: "intent_log_pool": "default.rgw.log:intent", 2022-09-03T05:17:03.295 INFO:teuthology.orchestra.run.smithi189.stdout: "usage_log_pool": "default.rgw.log:usage", 2022-09-03T05:17:03.296 INFO:teuthology.orchestra.run.smithi189.stdout: "roles_pool": "default.rgw.meta:roles", 2022-09-03T05:17:03.297 INFO:teuthology.orchestra.run.smithi189.stdout: "reshard_pool": "default.rgw.log:reshard", 2022-09-03T05:17:03.297 INFO:teuthology.orchestra.run.smithi189.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2022-09-03T05:17:03.297 INFO:teuthology.orchestra.run.smithi189.stdout: "user_email_pool": "default.rgw.meta:users.email", 2022-09-03T05:17:03.298 INFO:teuthology.orchestra.run.smithi189.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2022-09-03T05:17:03.298 INFO:teuthology.orchestra.run.smithi189.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2022-09-03T05:17:03.298 INFO:teuthology.orchestra.run.smithi189.stdout: "otp_pool": "default.rgw.otp", 2022-09-03T05:17:03.298 INFO:teuthology.orchestra.run.smithi189.stdout: "system_key": { 2022-09-03T05:17:03.299 INFO:teuthology.orchestra.run.smithi189.stdout: "access_key": "", 2022-09-03T05:17:03.299 INFO:teuthology.orchestra.run.smithi189.stdout: "secret_key": "" 2022-09-03T05:17:03.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:17:03.300 INFO:teuthology.orchestra.run.smithi189.stdout: "placement_pools": [ 2022-09-03T05:17:03.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:17:03.300 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "default-placement", 2022-09-03T05:17:03.300 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:17:03.300 INFO:teuthology.orchestra.run.smithi189.stdout: "index_pool": "default.rgw.buckets.index", 2022-09-03T05:17:03.301 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_classes": { 2022-09-03T05:17:03.301 INFO:teuthology.orchestra.run.smithi189.stdout: "STANDARD": { 2022-09-03T05:17:03.301 INFO:teuthology.orchestra.run.smithi189.stdout: "data_pool": "default.rgw.buckets.data", 2022-09-03T05:17:03.302 INFO:teuthology.orchestra.run.smithi189.stdout: "compression_type": "random" 2022-09-03T05:17:03.302 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:17:03.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:17:03.302 INFO:teuthology.orchestra.run.smithi189.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2022-09-03T05:17:03.303 INFO:teuthology.orchestra.run.smithi189.stdout: "index_type": 0 2022-09-03T05:17:03.303 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:17:03.303 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:17:03.303 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:17:03.304 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "", 2022-09-03T05:17:03.304 INFO:teuthology.orchestra.run.smithi189.stdout: "notif_pool": "default.rgw.log:notif" 2022-09-03T05:17:03.305 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:17:03.305 DEBUG:tasks.util.rgw: json result: {'id': 'ef6feded-abd7-44e7-8115-4d2abed1a3da', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2022-09-03T05:17:03.306 INFO:tasks.util.rgw:rgwadmin: c2.client.1 : ['user', 'list'] 2022-09-03T05:17:03.306 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.1', '--cluster', 'c2', 'user', 'list'] 2022-09-03T05:17:03.307 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.1 --cluster c2 user list 2022-09-03T05:17:03.354 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:17:03.355 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:17:03.363 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.359+0000 7ff218ffe600 20 rados->read ofs=0 len=0 2022-09-03T05:17:03.363 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.360+0000 7ff218ffe600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:17:03.364 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.360+0000 7ff218ffe600 20 realm 2022-09-03T05:17:03.364 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.360+0000 7ff218ffe600 20 rados->read ofs=0 len=0 2022-09-03T05:17:03.364 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.361+0000 7ff218ffe600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:17:03.365 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.361+0000 7ff218ffe600 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2022-09-03T05:17:03.365 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.361+0000 7ff218ffe600 20 rados->read ofs=0 len=0 2022-09-03T05:17:03.365 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.361+0000 7ff218ffe600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:17:03.366 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.361+0000 7ff218ffe600 20 rados->read ofs=0 len=0 2022-09-03T05:17:03.366 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.361+0000 7ff218ffe600 20 rados_obj.operate() r=0 bl.length=46 2022-09-03T05:17:03.367 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.361+0000 7ff218ffe600 20 rados->read ofs=0 len=0 2022-09-03T05:17:03.367 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.362+0000 7ff218ffe600 20 rados_obj.operate() r=0 bl.length=889 2022-09-03T05:17:03.367 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.362+0000 7ff218ffe600 20 searching for the correct realm 2022-09-03T05:17:03.370 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.369+0000 7ff218ffe600 20 RGWRados::pool_iterate: got zone_info.ef6feded-abd7-44e7-8115-4d2abed1a3da 2022-09-03T05:17:03.370 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.369+0000 7ff218ffe600 20 RGWRados::pool_iterate: got zonegroup_info.224dab99-5f24-4511-9f76-68bf9b1b459e 2022-09-03T05:17:03.370 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.369+0000 7ff218ffe600 20 RGWRados::pool_iterate: got zone_names.default 2022-09-03T05:17:03.370 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.369+0000 7ff218ffe600 20 RGWRados::pool_iterate: got zonegroups_names.default 2022-09-03T05:17:03.371 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.369+0000 7ff218ffe600 20 rados->read ofs=0 len=0 2022-09-03T05:17:03.371 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.369+0000 7ff218ffe600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:17:03.371 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.369+0000 7ff218ffe600 20 rados->read ofs=0 len=0 2022-09-03T05:17:03.371 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.369+0000 7ff218ffe600 20 rados_obj.operate() r=0 bl.length=46 2022-09-03T05:17:03.372 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.369+0000 7ff218ffe600 20 rados->read ofs=0 len=0 2022-09-03T05:17:03.372 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.369+0000 7ff218ffe600 20 rados_obj.operate() r=0 bl.length=398 2022-09-03T05:17:03.372 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.369+0000 7ff218ffe600 20 zone default found 2022-09-03T05:17:03.372 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.369+0000 7ff218ffe600 4 Realm: () 2022-09-03T05:17:03.372 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.369+0000 7ff218ffe600 4 ZoneGroup: default (224dab99-5f24-4511-9f76-68bf9b1b459e) 2022-09-03T05:17:03.373 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.369+0000 7ff218ffe600 4 Zone: default (ef6feded-abd7-44e7-8115-4d2abed1a3da) 2022-09-03T05:17:03.373 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.369+0000 7ff218ffe600 10 cannot find current period zonegroup using local zonegroup configuration 2022-09-03T05:17:03.373 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.369+0000 7ff218ffe600 20 zonegroup default 2022-09-03T05:17:03.373 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.369+0000 7ff218ffe600 20 rados->read ofs=0 len=0 2022-09-03T05:17:03.373 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.369+0000 7ff218ffe600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:17:03.374 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.369+0000 7ff218ffe600 20 rados->read ofs=0 len=0 2022-09-03T05:17:03.374 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.370+0000 7ff218ffe600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:17:03.374 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.370+0000 7ff218ffe600 20 rados->read ofs=0 len=0 2022-09-03T05:17:03.374 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.370+0000 7ff218ffe600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:17:03.375 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.370+0000 7ff218ffe600 20 started sync module instance, tier type = 2022-09-03T05:17:03.375 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.370+0000 7ff218ffe600 20 started zone id=ef6feded-abd7-44e7-8115-4d2abed1a3da (name=default) with tier type = 2022-09-03T05:17:03.380 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.381+0000 7ff218ffe600 20 add_watcher() i=0 2022-09-03T05:17:03.381 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.381+0000 7ff218ffe600 20 add_watcher() i=1 2022-09-03T05:17:03.381 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.381+0000 7ff218ffe600 20 add_watcher() i=2 2022-09-03T05:17:03.381 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.381+0000 7ff218ffe600 20 add_watcher() i=3 2022-09-03T05:17:03.382 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.381+0000 7ff218ffe600 20 add_watcher() i=4 2022-09-03T05:17:03.382 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.381+0000 7ff218ffe600 20 add_watcher() i=5 2022-09-03T05:17:03.383 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.381+0000 7ff218ffe600 20 add_watcher() i=6 2022-09-03T05:17:03.384 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.382+0000 7ff218ffe600 20 add_watcher() i=7 2022-09-03T05:17:03.385 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.382+0000 7ff218ffe600 2 all 8 watchers are set, enabling cache 2022-09-03T05:17:03.385 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.383+0000 7ff20177f700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2022-09-03T05:17:03.385 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.383+0000 7ff218ffe600 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2022-09-03T05:17:03.388 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.383+0000 7ff218ffe600 20 check_secure_mon_conn(): mode 1 is insecure 2022-09-03T05:17:03.391 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.387+0000 7ff218ffe600 5 note: GC not initialized 2022-09-03T05:17:03.391 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.388+0000 7ff1fcf76700 20 reqs_thread_entry: start 2022-09-03T05:17:03.392 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.388+0000 7ff218ffe600 20 init_complete bucket index max shards: 11 2022-09-03T05:17:03.392 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.388+0000 7ff218ffe600 10 rgw notify: Started notification manager with: 1 workers 2022-09-03T05:17:03.393 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.388+0000 7ff1faf72700 20 rgw notify: INFO: next queues processing will happen at: Sat Sep 3 05:17:33 2022 2022-09-03T05:17:03.393 INFO:teuthology.orchestra.run.smithi189.stderr: 2022-09-03T05:17:03.393 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.388+0000 7ff1fa750700 20 reqs_thread_entry: start 2022-09-03T05:17:03.395 INFO:teuthology.orchestra.run.smithi189.stdout:[] 2022-09-03T05:17:03.398 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.395+0000 7ff218ffe600 20 remove_watcher() i=0 2022-09-03T05:17:03.399 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.395+0000 7ff218ffe600 2 removed watcher, disabling cache 2022-09-03T05:17:03.399 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.396+0000 7ff218ffe600 20 remove_watcher() i=1 2022-09-03T05:17:03.399 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.397+0000 7ff218ffe600 20 remove_watcher() i=2 2022-09-03T05:17:03.399 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.398+0000 7ff218ffe600 20 remove_watcher() i=3 2022-09-03T05:17:03.400 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.399+0000 7ff218ffe600 20 remove_watcher() i=4 2022-09-03T05:17:03.400 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.400+0000 7ff218ffe600 20 remove_watcher() i=5 2022-09-03T05:17:03.401 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.400+0000 7ff218ffe600 20 remove_watcher() i=6 2022-09-03T05:17:03.401 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.402+0000 7ff218ffe600 20 remove_watcher() i=7 2022-09-03T05:17:03.401 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.403+0000 7ff20177f700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2022-09-03T05:17:03.410 DEBUG:tasks.util.rgw: json result: [] 2022-09-03T05:17:03.410 INFO:tasks.util.rgw:rgwadmin: c2.client.1 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2022-09-03T05:17:03.411 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.1', '--cluster', 'c2', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2022-09-03T05:17:03.411 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.1 --cluster c2 zone placement modify --rgw-zone default --placement-id default-placement --compression random 2022-09-03T05:17:03.471 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:17:03.471 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:17:03.471 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.468+0000 7f4079622600 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2022-09-03T05:17:03.471 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.468+0000 7f4079622600 20 check_secure_mon_conn(): mode 1 is insecure 2022-09-03T05:17:03.472 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.472+0000 7f405e59c700 20 reqs_thread_entry: start 2022-09-03T05:17:03.472 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.472+0000 7f4079622600 20 try_to_resolve_local_entities(): before: realm_id= realm_name= zonegroup_id= zonegroup_name= zone_id= zone_name=default 2022-09-03T05:17:03.472 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.472+0000 7f4079622600 20 rados->read ofs=0 len=0 2022-09-03T05:17:03.473 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.473+0000 7f4079622600 20 rados_obj.operate() r=0 bl.length=46 2022-09-03T05:17:03.473 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.473+0000 7f4079622600 20 rados->read ofs=0 len=0 2022-09-03T05:17:03.474 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.473+0000 7f4079622600 20 rados_obj.operate() r=0 bl.length=889 2022-09-03T05:17:03.481 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.480+0000 7f4079622600 20 RGWRados::pool_iterate: got zone_info.ef6feded-abd7-44e7-8115-4d2abed1a3da 2022-09-03T05:17:03.486 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.480+0000 7f4079622600 20 RGWRados::pool_iterate: got zonegroup_info.224dab99-5f24-4511-9f76-68bf9b1b459e 2022-09-03T05:17:03.486 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.480+0000 7f4079622600 20 RGWRados::pool_iterate: got zone_names.default 2022-09-03T05:17:03.487 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.480+0000 7f4079622600 20 RGWRados::pool_iterate: got zonegroups_names.default 2022-09-03T05:17:03.487 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.480+0000 7f4079622600 20 rados->read ofs=0 len=0 2022-09-03T05:17:03.487 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.480+0000 7f4079622600 20 rados_obj.operate() r=0 bl.length=889 2022-09-03T05:17:03.487 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.480+0000 7f4079622600 20 rados->read ofs=0 len=0 2022-09-03T05:17:03.488 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.480+0000 7f4079622600 20 rados_obj.operate() r=-2 bl.length=0 2022-09-03T05:17:03.488 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.480+0000 7f4079622600 20 rados->read ofs=0 len=0 2022-09-03T05:17:03.488 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.480+0000 7f4079622600 20 rados_obj.operate() r=0 bl.length=46 2022-09-03T05:17:03.488 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.480+0000 7f4079622600 20 rados->read ofs=0 len=0 2022-09-03T05:17:03.489 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:03.480+0000 7f4079622600 20 rados_obj.operate() r=0 bl.length=398 2022-09-03T05:17:03.490 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:17:03.491 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "ef6feded-abd7-44e7-8115-4d2abed1a3da", 2022-09-03T05:17:03.491 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "default", 2022-09-03T05:17:03.491 INFO:teuthology.orchestra.run.smithi189.stdout: "domain_root": "default.rgw.meta:root", 2022-09-03T05:17:03.492 INFO:teuthology.orchestra.run.smithi189.stdout: "control_pool": "default.rgw.control", 2022-09-03T05:17:03.492 INFO:teuthology.orchestra.run.smithi189.stdout: "gc_pool": "default.rgw.log:gc", 2022-09-03T05:17:03.492 INFO:teuthology.orchestra.run.smithi189.stdout: "lc_pool": "default.rgw.log:lc", 2022-09-03T05:17:03.492 INFO:teuthology.orchestra.run.smithi189.stdout: "log_pool": "default.rgw.log", 2022-09-03T05:17:03.493 INFO:teuthology.orchestra.run.smithi189.stdout: "intent_log_pool": "default.rgw.log:intent", 2022-09-03T05:17:03.493 INFO:teuthology.orchestra.run.smithi189.stdout: "usage_log_pool": "default.rgw.log:usage", 2022-09-03T05:17:03.493 INFO:teuthology.orchestra.run.smithi189.stdout: "roles_pool": "default.rgw.meta:roles", 2022-09-03T05:17:03.493 INFO:teuthology.orchestra.run.smithi189.stdout: "reshard_pool": "default.rgw.log:reshard", 2022-09-03T05:17:03.493 INFO:teuthology.orchestra.run.smithi189.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2022-09-03T05:17:03.494 INFO:teuthology.orchestra.run.smithi189.stdout: "user_email_pool": "default.rgw.meta:users.email", 2022-09-03T05:17:03.494 INFO:teuthology.orchestra.run.smithi189.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2022-09-03T05:17:03.494 INFO:teuthology.orchestra.run.smithi189.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2022-09-03T05:17:03.494 INFO:teuthology.orchestra.run.smithi189.stdout: "otp_pool": "default.rgw.otp", 2022-09-03T05:17:03.495 INFO:teuthology.orchestra.run.smithi189.stdout: "system_key": { 2022-09-03T05:17:03.495 INFO:teuthology.orchestra.run.smithi189.stdout: "access_key": "", 2022-09-03T05:17:03.495 INFO:teuthology.orchestra.run.smithi189.stdout: "secret_key": "" 2022-09-03T05:17:03.495 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:17:03.496 INFO:teuthology.orchestra.run.smithi189.stdout: "placement_pools": [ 2022-09-03T05:17:03.496 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:17:03.496 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "default-placement", 2022-09-03T05:17:03.496 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:17:03.496 INFO:teuthology.orchestra.run.smithi189.stdout: "index_pool": "default.rgw.buckets.index", 2022-09-03T05:17:03.497 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_classes": { 2022-09-03T05:17:03.498 INFO:teuthology.orchestra.run.smithi189.stdout: "STANDARD": { 2022-09-03T05:17:03.498 INFO:teuthology.orchestra.run.smithi189.stdout: "data_pool": "default.rgw.buckets.data", 2022-09-03T05:17:03.499 INFO:teuthology.orchestra.run.smithi189.stdout: "compression_type": "random" 2022-09-03T05:17:03.499 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:17:03.499 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:17:03.499 INFO:teuthology.orchestra.run.smithi189.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2022-09-03T05:17:03.500 INFO:teuthology.orchestra.run.smithi189.stdout: "index_type": 0 2022-09-03T05:17:03.500 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:17:03.500 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:17:03.500 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:17:03.501 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "", 2022-09-03T05:17:03.501 INFO:teuthology.orchestra.run.smithi189.stdout: "notif_pool": "default.rgw.log:notif" 2022-09-03T05:17:03.501 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:17:03.502 DEBUG:tasks.util.rgw: json result: {'id': 'ef6feded-abd7-44e7-8115-4d2abed1a3da', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2022-09-03T05:17:03.502 INFO:tasks.rgw:Starting rgw... 2022-09-03T05:17:03.503 INFO:tasks.rgw:rgw c1.client.0 config is {'port': 8000} 2022-09-03T05:17:03.503 INFO:tasks.rgw:Using beast as radosgw frontend 2022-09-03T05:17:03.503 INFO:tasks.rgw.c1.client.0:Restarting daemon 2022-09-03T05:17:03.504 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c1 -k /etc/ceph/c1.client.0.keyring --log-file /var/log/ceph/rgw.c1.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.c1.client.0.stdout 2>&1 2022-09-03T05:17:03.514 INFO:tasks.rgw.c1.client.0:Started 2022-09-03T05:17:03.515 INFO:tasks.rgw:rgw c1.client.1 config is {'port': 8001} 2022-09-03T05:17:03.516 INFO:tasks.rgw:Using beast as radosgw frontend 2022-09-03T05:17:03.517 INFO:tasks.rgw.c1.client.1:Restarting daemon 2022-09-03T05:17:03.517 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8001' -n client.1 --cluster c1 -k /etc/ceph/c1.client.1.keyring --log-file /var/log/ceph/rgw.c1.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.1.sock --foreground | sudo tee /var/log/ceph/rgw.c1.client.1.stdout 2>&1 2022-09-03T05:17:03.524 INFO:tasks.rgw.c1.client.1:Started 2022-09-03T05:17:03.525 INFO:tasks.rgw:rgw c2.client.0 config is {'port': 8000} 2022-09-03T05:17:03.526 INFO:tasks.rgw:Using beast as radosgw frontend 2022-09-03T05:17:03.530 INFO:tasks.rgw.c2.client.0:Restarting daemon 2022-09-03T05:17:03.531 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.c2.client.0.stdout 2>&1 2022-09-03T05:17:03.544 INFO:tasks.rgw.c2.client.0:Started 2022-09-03T05:17:03.545 INFO:tasks.rgw:rgw c2.client.1 config is {'port': 8001} 2022-09-03T05:17:03.546 INFO:tasks.rgw:Using beast as radosgw frontend 2022-09-03T05:17:03.546 INFO:tasks.rgw.c2.client.1:Restarting daemon 2022-09-03T05:17:03.547 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8001' -n client.1 --cluster c2 -k /etc/ceph/c2.client.1.keyring --log-file /var/log/ceph/rgw.c2.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.1.sock --foreground | sudo tee /var/log/ceph/rgw.c2.client.1.stdout 2>&1 2022-09-03T05:17:03.563 INFO:tasks.rgw.c2.client.1:Started 2022-09-03T05:17:03.563 INFO:tasks.rgw:Polling c1.client.0 until it starts accepting connections on http://smithi181.front.sepia.ceph.com:8000/ 2022-09-03T05:17:03.564 DEBUG:teuthology.orchestra.run.smithi181:> curl http://smithi181.front.sepia.ceph.com:8000/ 2022-09-03T05:17:03.616 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T05:17:03.617 INFO:teuthology.orchestra.run.smithi181.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T05:17:03.617 INFO:teuthology.orchestra.run.smithi181.stderr: Dload Upload Total Spent Left Speed 2022-09-03T05:17:03.618 INFO:teuthology.orchestra.run.smithi181.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi181.front.sepia.ceph.com port 8000: Connection refused 2022-09-03T05:17:04.621 DEBUG:teuthology.orchestra.run.smithi181:> curl http://smithi181.front.sepia.ceph.com:8000/ 2022-09-03T05:17:04.649 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T05:17:04.650 INFO:teuthology.orchestra.run.smithi181.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T05:17:04.651 INFO:teuthology.orchestra.run.smithi181.stderr: Dload Upload Total Spent Left Speed 2022-09-03T05:17:04.651 INFO:teuthology.orchestra.run.smithi181.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi181.front.sepia.ceph.com port 8000: Connection refused 2022-09-03T05:17:06.656 DEBUG:teuthology.orchestra.run.smithi181:> curl http://smithi181.front.sepia.ceph.com:8000/ 2022-09-03T05:17:06.685 INFO:teuthology.orchestra.run.smithi181.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T05:17:06.685 INFO:teuthology.orchestra.run.smithi181.stderr: Dload Upload Total Spent Left Speed 2022-09-03T05:17:06.686 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T05:17:06.687 INFO:teuthology.orchestra.run.smithi181.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi181.front.sepia.ceph.com port 8000: Connection refused 2022-09-03T05:17:10.695 DEBUG:teuthology.orchestra.run.smithi181:> curl http://smithi181.front.sepia.ceph.com:8000/ 2022-09-03T05:17:10.720 INFO:teuthology.orchestra.run.smithi181.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T05:17:10.721 INFO:teuthology.orchestra.run.smithi181.stderr: Dload Upload Total Spent Left Speed 2022-09-03T05:17:10.726 INFO:teuthology.orchestra.run.smithi181.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 104k 0 --:--:-- --:--:-- --:--:-- 104k 2022-09-03T05:17:10.728 INFO:teuthology.orchestra.run.smithi181.stdout:anonymous 2022-09-03T05:17:10.728 INFO:tasks.rgw:Polling c1.client.1 until it starts accepting connections on http://smithi181.front.sepia.ceph.com:8001/ 2022-09-03T05:17:10.728 DEBUG:teuthology.orchestra.run.smithi181:> curl http://smithi181.front.sepia.ceph.com:8001/ 2022-09-03T05:17:10.794 INFO:teuthology.orchestra.run.smithi181.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T05:17:10.794 INFO:teuthology.orchestra.run.smithi181.stderr: Dload Upload Total Spent Left Speed 2022-09-03T05:17:10.794 INFO:teuthology.orchestra.run.smithi181.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 208k 0 --:--:-- --:--:-- --:--:-- 208k 2022-09-03T05:17:10.796 INFO:teuthology.orchestra.run.smithi181.stdout:anonymous 2022-09-03T05:17:10.796 INFO:tasks.rgw:Polling c2.client.0 until it starts accepting connections on http://smithi189.front.sepia.ceph.com:8000/ 2022-09-03T05:17:10.796 DEBUG:teuthology.orchestra.run.smithi189:> curl http://smithi189.front.sepia.ceph.com:8000/ 2022-09-03T05:17:10.861 INFO:teuthology.orchestra.run.smithi189.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T05:17:10.862 INFO:teuthology.orchestra.run.smithi189.stderr: Dload Upload Total Spent Left Speed 2022-09-03T05:17:10.862 INFO:teuthology.orchestra.run.smithi189.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 104k 0 --:--:-- --:--:-- --:--:-- 104k 2022-09-03T05:17:10.864 INFO:teuthology.orchestra.run.smithi189.stdout:anonymous 2022-09-03T05:17:10.870 INFO:tasks.rgw:Polling c2.client.1 until it starts accepting connections on http://smithi189.front.sepia.ceph.com:8001/ 2022-09-03T05:17:10.871 DEBUG:teuthology.orchestra.run.smithi189:> curl http://smithi189.front.sepia.ceph.com:8001/ 2022-09-03T05:17:10.899 INFO:teuthology.orchestra.run.smithi189.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T05:17:10.900 INFO:teuthology.orchestra.run.smithi189.stderr: Dload Upload Total Spent Left Speed 2022-09-03T05:17:10.900 INFO:teuthology.orchestra.run.smithi189.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 104k 0 --:--:-- --:--:-- --:--:-- 104k 2022-09-03T05:17:10.902 INFO:teuthology.orchestra.run.smithi189.stdout:anonymous 2022-09-03T05:17:10.903 INFO:teuthology.run_tasks:Running task rgw-multisite... 2022-09-03T05:17:11.595 INFO:tasks.rgw_multisite:creating realm.. 2022-09-03T05:17:11.596 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['realm', 'create', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:11.596 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'realm', 'create', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:11.597 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 realm create --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:17:11.650 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:17:11.651 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:17:11.684 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:17:11.685 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:17:11.685 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-realm", 2022-09-03T05:17:11.686 INFO:teuthology.orchestra.run.smithi181.stdout: "current_period": "26ebceb6-9cf2-4657-999c-e205262f80e9", 2022-09-03T05:17:11.686 INFO:teuthology.orchestra.run.smithi181.stdout: "epoch": 1 2022-09-03T05:17:11.686 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:17:11.687 INFO:tasks.rgw_multisite:creating master zonegroup.. 2022-09-03T05:17:11.687 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'set', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:11.688 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'set', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:11.688 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup set --rgw-zonegroup test-zonegroup --rgw-realm test-realm --default --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:17:11.738 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:17:11.738 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:17:11.749 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:17:11.752+0000 7f992abeb600 -1 old format 2022-09-03T05:17:11.754 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:17:11.755 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "test-zonegroup", 2022-09-03T05:17:11.755 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zonegroup", 2022-09-03T05:17:11.755 INFO:teuthology.orchestra.run.smithi181.stdout: "api_name": "", 2022-09-03T05:17:11.755 INFO:teuthology.orchestra.run.smithi181.stdout: "is_master": true, 2022-09-03T05:17:11.755 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:17:11.756 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:17:11.756 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:17:11.756 INFO:teuthology.orchestra.run.smithi181.stdout: "hostnames": [], 2022-09-03T05:17:11.756 INFO:teuthology.orchestra.run.smithi181.stdout: "hostnames_s3website": [], 2022-09-03T05:17:11.757 INFO:teuthology.orchestra.run.smithi181.stdout: "master_zone": "", 2022-09-03T05:17:11.757 INFO:teuthology.orchestra.run.smithi181.stdout: "zones": [], 2022-09-03T05:17:11.757 INFO:teuthology.orchestra.run.smithi181.stdout: "placement_targets": [], 2022-09-03T05:17:11.757 INFO:teuthology.orchestra.run.smithi181.stdout: "default_placement": "", 2022-09-03T05:17:11.758 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:17:11.758 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_policy": { 2022-09-03T05:17:11.758 INFO:teuthology.orchestra.run.smithi181.stdout: "groups": [] 2022-09-03T05:17:11.758 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:11.759 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled_features": [ 2022-09-03T05:17:11.759 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:17:11.759 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:17:11.759 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:17:11.761 INFO:tasks.rgw_multisite:creating master zone.. 2022-09-03T05:17:11.761 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'create', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--master', '--access-key', 'TW0E8EULCERVYU84', '--secret', 'EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP', '--endpoints', 'http://smithi181.front.sepia.ceph.com:8000', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:11.761 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'create', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--master', '--access-key', 'TW0E8EULCERVYU84', '--secret', 'EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP', '--endpoints', 'http://smithi181.front.sepia.ceph.com:8000', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:11.762 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone create --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --default --master --access-key TW0E8EULCERVYU84 --secret EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP --endpoints http://smithi181.front.sepia.ceph.com:8000 --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:17:11.856 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:17:11.856 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:17:11.883 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:17:11.883 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:17:11.883 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone1", 2022-09-03T05:17:11.883 INFO:teuthology.orchestra.run.smithi181.stdout: "domain_root": "test-zone1.rgw.meta:root", 2022-09-03T05:17:11.884 INFO:teuthology.orchestra.run.smithi181.stdout: "control_pool": "test-zone1.rgw.control", 2022-09-03T05:17:11.884 INFO:teuthology.orchestra.run.smithi181.stdout: "gc_pool": "test-zone1.rgw.log:gc", 2022-09-03T05:17:11.884 INFO:teuthology.orchestra.run.smithi181.stdout: "lc_pool": "test-zone1.rgw.log:lc", 2022-09-03T05:17:11.884 INFO:teuthology.orchestra.run.smithi181.stdout: "log_pool": "test-zone1.rgw.log", 2022-09-03T05:17:11.885 INFO:teuthology.orchestra.run.smithi181.stdout: "intent_log_pool": "test-zone1.rgw.log:intent", 2022-09-03T05:17:11.885 INFO:teuthology.orchestra.run.smithi181.stdout: "usage_log_pool": "test-zone1.rgw.log:usage", 2022-09-03T05:17:11.885 INFO:teuthology.orchestra.run.smithi181.stdout: "roles_pool": "test-zone1.rgw.meta:roles", 2022-09-03T05:17:11.885 INFO:teuthology.orchestra.run.smithi181.stdout: "reshard_pool": "test-zone1.rgw.log:reshard", 2022-09-03T05:17:11.886 INFO:teuthology.orchestra.run.smithi181.stdout: "user_keys_pool": "test-zone1.rgw.meta:users.keys", 2022-09-03T05:17:11.886 INFO:teuthology.orchestra.run.smithi181.stdout: "user_email_pool": "test-zone1.rgw.meta:users.email", 2022-09-03T05:17:11.886 INFO:teuthology.orchestra.run.smithi181.stdout: "user_swift_pool": "test-zone1.rgw.meta:users.swift", 2022-09-03T05:17:11.886 INFO:teuthology.orchestra.run.smithi181.stdout: "user_uid_pool": "test-zone1.rgw.meta:users.uid", 2022-09-03T05:17:11.887 INFO:teuthology.orchestra.run.smithi181.stdout: "otp_pool": "test-zone1.rgw.otp", 2022-09-03T05:17:11.887 INFO:teuthology.orchestra.run.smithi181.stdout: "system_key": { 2022-09-03T05:17:11.887 INFO:teuthology.orchestra.run.smithi181.stdout: "access_key": "TW0E8EULCERVYU84", 2022-09-03T05:17:11.887 INFO:teuthology.orchestra.run.smithi181.stdout: "secret_key": "EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP" 2022-09-03T05:17:11.888 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:11.888 INFO:teuthology.orchestra.run.smithi181.stdout: "placement_pools": [ 2022-09-03T05:17:11.888 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:17:11.888 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "default-placement", 2022-09-03T05:17:11.889 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:17:11.889 INFO:teuthology.orchestra.run.smithi181.stdout: "index_pool": "test-zone1.rgw.buckets.index", 2022-09-03T05:17:11.889 INFO:teuthology.orchestra.run.smithi181.stdout: "storage_classes": { 2022-09-03T05:17:11.889 INFO:teuthology.orchestra.run.smithi181.stdout: "STANDARD": { 2022-09-03T05:17:11.889 INFO:teuthology.orchestra.run.smithi181.stdout: "data_pool": "test-zone1.rgw.buckets.data" 2022-09-03T05:17:11.890 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:17:11.890 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:11.890 INFO:teuthology.orchestra.run.smithi181.stdout: "data_extra_pool": "test-zone1.rgw.buckets.non-ec", 2022-09-03T05:17:11.890 INFO:teuthology.orchestra.run.smithi181.stdout: "index_type": 0 2022-09-03T05:17:11.891 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:17:11.891 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:17:11.891 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:17:11.891 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:17:11.892 INFO:teuthology.orchestra.run.smithi181.stdout: "notif_pool": "test-zone1.rgw.log:notif" 2022-09-03T05:17:11.892 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:17:11.893 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph osd pool create test-zone1.rgw.buckets.data 64 64 --cluster c1 2022-09-03T05:17:12.948 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'test-zone1.rgw.buckets.data' created 2022-09-03T05:17:12.965 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph osd pool application enable test-zone1.rgw.buckets.data rgw --cluster c1 2022-09-03T05:17:13.963 INFO:teuthology.orchestra.run.smithi181.stderr:enabled application 'rgw' on pool 'test-zone1.rgw.buckets.data' 2022-09-03T05:17:13.978 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'placement', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:13.978 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'placement', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:13.979 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone placement --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm modify --placement-id default-placement --compression random --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:17:14.033 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:17:14.034 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:17:14.056 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:17:14.057 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:17:14.057 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone1", 2022-09-03T05:17:14.058 INFO:teuthology.orchestra.run.smithi181.stdout: "domain_root": "test-zone1.rgw.meta:root", 2022-09-03T05:17:14.058 INFO:teuthology.orchestra.run.smithi181.stdout: "control_pool": "test-zone1.rgw.control", 2022-09-03T05:17:14.059 INFO:teuthology.orchestra.run.smithi181.stdout: "gc_pool": "test-zone1.rgw.log:gc", 2022-09-03T05:17:14.059 INFO:teuthology.orchestra.run.smithi181.stdout: "lc_pool": "test-zone1.rgw.log:lc", 2022-09-03T05:17:14.060 INFO:teuthology.orchestra.run.smithi181.stdout: "log_pool": "test-zone1.rgw.log", 2022-09-03T05:17:14.060 INFO:teuthology.orchestra.run.smithi181.stdout: "intent_log_pool": "test-zone1.rgw.log:intent", 2022-09-03T05:17:14.061 INFO:teuthology.orchestra.run.smithi181.stdout: "usage_log_pool": "test-zone1.rgw.log:usage", 2022-09-03T05:17:14.061 INFO:teuthology.orchestra.run.smithi181.stdout: "roles_pool": "test-zone1.rgw.meta:roles", 2022-09-03T05:17:14.062 INFO:teuthology.orchestra.run.smithi181.stdout: "reshard_pool": "test-zone1.rgw.log:reshard", 2022-09-03T05:17:14.062 INFO:teuthology.orchestra.run.smithi181.stdout: "user_keys_pool": "test-zone1.rgw.meta:users.keys", 2022-09-03T05:17:14.063 INFO:teuthology.orchestra.run.smithi181.stdout: "user_email_pool": "test-zone1.rgw.meta:users.email", 2022-09-03T05:17:14.063 INFO:teuthology.orchestra.run.smithi181.stdout: "user_swift_pool": "test-zone1.rgw.meta:users.swift", 2022-09-03T05:17:14.063 INFO:teuthology.orchestra.run.smithi181.stdout: "user_uid_pool": "test-zone1.rgw.meta:users.uid", 2022-09-03T05:17:14.064 INFO:teuthology.orchestra.run.smithi181.stdout: "otp_pool": "test-zone1.rgw.otp", 2022-09-03T05:17:14.064 INFO:teuthology.orchestra.run.smithi181.stdout: "system_key": { 2022-09-03T05:17:14.066 INFO:teuthology.orchestra.run.smithi181.stdout: "access_key": "TW0E8EULCERVYU84", 2022-09-03T05:17:14.066 INFO:teuthology.orchestra.run.smithi181.stdout: "secret_key": "EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP" 2022-09-03T05:17:14.066 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:14.067 INFO:teuthology.orchestra.run.smithi181.stdout: "placement_pools": [ 2022-09-03T05:17:14.067 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:17:14.067 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "default-placement", 2022-09-03T05:17:14.068 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:17:14.068 INFO:teuthology.orchestra.run.smithi181.stdout: "index_pool": "test-zone1.rgw.buckets.index", 2022-09-03T05:17:14.068 INFO:teuthology.orchestra.run.smithi181.stdout: "storage_classes": { 2022-09-03T05:17:14.068 INFO:teuthology.orchestra.run.smithi181.stdout: "STANDARD": { 2022-09-03T05:17:14.069 INFO:teuthology.orchestra.run.smithi181.stdout: "data_pool": "test-zone1.rgw.buckets.data", 2022-09-03T05:17:14.069 INFO:teuthology.orchestra.run.smithi181.stdout: "compression_type": "random" 2022-09-03T05:17:14.070 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:17:14.070 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:14.071 INFO:teuthology.orchestra.run.smithi181.stdout: "data_extra_pool": "test-zone1.rgw.buckets.non-ec", 2022-09-03T05:17:14.071 INFO:teuthology.orchestra.run.smithi181.stdout: "index_type": 0 2022-09-03T05:17:14.071 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:17:14.071 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:17:14.072 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:17:14.072 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "", 2022-09-03T05:17:14.072 INFO:teuthology.orchestra.run.smithi181.stdout: "notif_pool": "test-zone1.rgw.log:notif" 2022-09-03T05:17:14.072 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:17:14.074 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:14.075 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:14.075 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:17:14.125 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:17:14.125 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:17:14.136 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:17:14.135+0000 7f3448f8e600 0 period (26ebceb6-9cf2-4657-999c-e205262f80e9 does not have zone 91786ef1-cdb4-4061-8a3e-2a28cc497285 configured 2022-09-03T05:17:21.035 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:17:21.036 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:17:21.037 INFO:teuthology.orchestra.run.smithi181.stdout: "epoch": 1, 2022-09-03T05:17:21.037 INFO:teuthology.orchestra.run.smithi181.stdout: "predecessor_uuid": "26ebceb6-9cf2-4657-999c-e205262f80e9", 2022-09-03T05:17:21.038 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": [], 2022-09-03T05:17:21.038 INFO:teuthology.orchestra.run.smithi181.stdout: "period_map": { 2022-09-03T05:17:21.039 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:17:21.039 INFO:teuthology.orchestra.run.smithi181.stdout: "zonegroups": [ 2022-09-03T05:17:21.039 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:17:21.040 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "test-zonegroup", 2022-09-03T05:17:21.041 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zonegroup", 2022-09-03T05:17:21.041 INFO:teuthology.orchestra.run.smithi181.stdout: "api_name": "", 2022-09-03T05:17:21.042 INFO:teuthology.orchestra.run.smithi181.stdout: "is_master": true, 2022-09-03T05:17:21.042 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:17:21.042 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:17:21.042 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:17:21.043 INFO:teuthology.orchestra.run.smithi181.stdout: "hostnames": [], 2022-09-03T05:17:21.043 INFO:teuthology.orchestra.run.smithi181.stdout: "hostnames_s3website": [], 2022-09-03T05:17:21.043 INFO:teuthology.orchestra.run.smithi181.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:17:21.043 INFO:teuthology.orchestra.run.smithi181.stdout: "zones": [ 2022-09-03T05:17:21.044 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:17:21.044 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:17:21.044 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone1", 2022-09-03T05:17:21.044 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:17:21.045 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:17:21.045 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:17:21.045 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:17:21.046 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": false, 2022-09-03T05:17:21.046 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:17:21.046 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:17:21.046 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "", 2022-09-03T05:17:21.047 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": true, 2022-09-03T05:17:21.047 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [], 2022-09-03T05:17:21.047 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:17:21.047 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:17:21.048 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:17:21.048 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:17:21.048 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:17:21.049 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:17:21.049 INFO:teuthology.orchestra.run.smithi181.stdout: "placement_targets": [ 2022-09-03T05:17:21.049 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:17:21.050 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "default-placement", 2022-09-03T05:17:21.050 INFO:teuthology.orchestra.run.smithi181.stdout: "tags": [], 2022-09-03T05:17:21.050 INFO:teuthology.orchestra.run.smithi181.stdout: "storage_classes": [ 2022-09-03T05:17:21.051 INFO:teuthology.orchestra.run.smithi181.stdout: "STANDARD" 2022-09-03T05:17:21.051 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:17:21.051 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:17:21.051 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:17:21.052 INFO:teuthology.orchestra.run.smithi181.stdout: "default_placement": "default-placement", 2022-09-03T05:17:21.052 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:17:21.052 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_policy": { 2022-09-03T05:17:21.052 INFO:teuthology.orchestra.run.smithi181.stdout: "groups": [] 2022-09-03T05:17:21.053 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:21.053 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled_features": [ 2022-09-03T05:17:21.053 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:17:21.053 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:17:21.054 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:17:21.054 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:17:21.054 INFO:teuthology.orchestra.run.smithi181.stdout: "short_zone_ids": [ 2022-09-03T05:17:21.055 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:17:21.055 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:17:21.055 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 1752562153 2022-09-03T05:17:21.056 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:17:21.056 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:17:21.056 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:21.056 INFO:teuthology.orchestra.run.smithi181.stdout: "master_zonegroup": "test-zonegroup", 2022-09-03T05:17:21.056 INFO:teuthology.orchestra.run.smithi181.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:17:21.057 INFO:teuthology.orchestra.run.smithi181.stdout: "period_config": { 2022-09-03T05:17:21.057 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_quota": { 2022-09-03T05:17:21.057 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false, 2022-09-03T05:17:21.057 INFO:teuthology.orchestra.run.smithi181.stdout: "check_on_raw": false, 2022-09-03T05:17:21.058 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size": -1, 2022-09-03T05:17:21.058 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size_kb": 0, 2022-09-03T05:17:21.058 INFO:teuthology.orchestra.run.smithi181.stdout: "max_objects": -1 2022-09-03T05:17:21.059 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:21.059 INFO:teuthology.orchestra.run.smithi181.stdout: "user_quota": { 2022-09-03T05:17:21.059 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false, 2022-09-03T05:17:21.060 INFO:teuthology.orchestra.run.smithi181.stdout: "check_on_raw": false, 2022-09-03T05:17:21.060 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size": -1, 2022-09-03T05:17:21.061 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size_kb": 0, 2022-09-03T05:17:21.061 INFO:teuthology.orchestra.run.smithi181.stdout: "max_objects": -1 2022-09-03T05:17:21.062 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:21.062 INFO:teuthology.orchestra.run.smithi181.stdout: "user_ratelimit": { 2022-09-03T05:17:21.062 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_ops": 0, 2022-09-03T05:17:21.063 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_ops": 0, 2022-09-03T05:17:21.064 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_bytes": 0, 2022-09-03T05:17:21.064 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_bytes": 0, 2022-09-03T05:17:21.065 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false 2022-09-03T05:17:21.065 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:21.066 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_ratelimit": { 2022-09-03T05:17:21.066 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_ops": 0, 2022-09-03T05:17:21.066 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_ops": 0, 2022-09-03T05:17:21.067 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_bytes": 0, 2022-09-03T05:17:21.067 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_bytes": 0, 2022-09-03T05:17:21.067 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false 2022-09-03T05:17:21.068 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:21.068 INFO:teuthology.orchestra.run.smithi181.stdout: "anonymous_ratelimit": { 2022-09-03T05:17:21.068 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_ops": 0, 2022-09-03T05:17:21.069 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_ops": 0, 2022-09-03T05:17:21.069 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_bytes": 0, 2022-09-03T05:17:21.069 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_bytes": 0, 2022-09-03T05:17:21.070 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false 2022-09-03T05:17:21.070 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:17:21.071 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:21.071 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:17:21.072 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_name": "test-realm", 2022-09-03T05:17:21.072 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:17:21.073 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:17:21.075 DEBUG:tasks.rgw.c1.client.0:waiting for process to exit 2022-09-03T05:17:21.076 INFO:teuthology.orchestra.run:waiting for 300 2022-09-03T05:17:21.078 INFO:tasks.rgw.c1.client.0.smithi181.stdout:2022-09-03T05:17:21.079+0000 7fbb64947700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term radosgw --rgw-frontends beast port=8000 -n client.0 --cluster c1 -k /etc/ceph/c1.client.0.keyring --log-file /var/log/ceph/rgw.c1.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.0.sock --foreground (PID: 109790) UID: 0 2022-09-03T05:17:21.078 INFO:tasks.rgw.c1.client.0.smithi181.stdout:2022-09-03T05:17:21.079+0000 7fbb68bb2600 -1 shutting down 2022-09-03T05:17:23.612 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c1.rgw.client.0 is failed for ~0s 2022-09-03T05:17:27.178 INFO:tasks.rgw.c1.client.0:Stopped 2022-09-03T05:17:27.179 INFO:tasks.rgw.c1.client.0:Restarting daemon 2022-09-03T05:17:27.180 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c1 -k /etc/ceph/c1.client.0.keyring --log-file /var/log/ceph/rgw.c1.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.0.sock --foreground --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c1.client.0.stdout 2>&1 2022-09-03T05:17:27.184 INFO:tasks.rgw.c1.client.0:Started 2022-09-03T05:17:27.184 DEBUG:teuthology.orchestra.run.smithi181:> curl http://smithi181.front.sepia.ceph.com:8000 2022-09-03T05:17:27.213 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T05:17:27.214 INFO:teuthology.orchestra.run.smithi181.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T05:17:27.215 INFO:teuthology.orchestra.run.smithi181.stderr: Dload Upload Total Spent Left Speed 2022-09-03T05:17:27.215 INFO:teuthology.orchestra.run.smithi181.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi181.front.sepia.ceph.com port 8000: Connection refused 2022-09-03T05:17:28.222 DEBUG:teuthology.orchestra.run.smithi181:> curl http://smithi181.front.sepia.ceph.com:8000 2022-09-03T05:17:28.263 INFO:teuthology.orchestra.run.smithi181.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T05:17:28.263 INFO:teuthology.orchestra.run.smithi181.stderr: Dload Upload Total Spent Left Speed 2022-09-03T05:17:28.265 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T05:17:28.265 INFO:teuthology.orchestra.run.smithi181.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi181.front.sepia.ceph.com port 8000: Connection refused 2022-09-03T05:17:29.319 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c1.rgw.client.0 has been restored 2022-09-03T05:17:30.278 DEBUG:teuthology.orchestra.run.smithi181:> curl http://smithi181.front.sepia.ceph.com:8000 2022-09-03T05:17:30.306 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T05:17:30.307 INFO:teuthology.orchestra.run.smithi181.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T05:17:30.307 INFO:teuthology.orchestra.run.smithi181.stderr: Dload Upload Total Spent Left Speed 2022-09-03T05:17:30.307 INFO:teuthology.orchestra.run.smithi181.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi181.front.sepia.ceph.com port 8000: Connection refused 2022-09-03T05:17:34.310 DEBUG:teuthology.orchestra.run.smithi181:> curl http://smithi181.front.sepia.ceph.com:8000 2022-09-03T05:17:34.342 INFO:teuthology.orchestra.run.smithi181.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T05:17:34.342 INFO:teuthology.orchestra.run.smithi181.stderr: Dload Upload Total Spent Left Speed 2022-09-03T05:17:34.344 INFO:teuthology.orchestra.run.smithi181.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 104k 0 --:--:-- --:--:-- --:--:-- 104k 2022-09-03T05:17:34.345 INFO:teuthology.orchestra.run.smithi181.stdout:anonymous 2022-09-03T05:17:34.345 INFO:tasks.rgw_multisite:creating admin user.. 2022-09-03T05:17:34.346 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['user', 'create', '--uid', 'realm-admin', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--display-name', 'Realm Admin', '--system', '--access-key', 'TW0E8EULCERVYU84', '--secret', 'EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:34.346 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'user', 'create', '--uid', 'realm-admin', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--display-name', 'Realm Admin', '--system', '--access-key', 'TW0E8EULCERVYU84', '--secret', 'EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:34.347 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 user create --uid realm-admin --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --display-name 'Realm Admin' --system --access-key TW0E8EULCERVYU84 --secret EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:17:34.446 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:17:34.446 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:17:34.483 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:17:34.484 INFO:teuthology.orchestra.run.smithi181.stdout: "user_id": "realm-admin", 2022-09-03T05:17:34.484 INFO:teuthology.orchestra.run.smithi181.stdout: "display_name": "Realm Admin", 2022-09-03T05:17:34.485 INFO:teuthology.orchestra.run.smithi181.stdout: "email": "", 2022-09-03T05:17:34.487 INFO:teuthology.orchestra.run.smithi181.stdout: "suspended": 0, 2022-09-03T05:17:34.487 INFO:teuthology.orchestra.run.smithi181.stdout: "max_buckets": 1000, 2022-09-03T05:17:34.488 INFO:teuthology.orchestra.run.smithi181.stdout: "subusers": [], 2022-09-03T05:17:34.488 INFO:teuthology.orchestra.run.smithi181.stdout: "keys": [ 2022-09-03T05:17:34.489 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:17:34.489 INFO:teuthology.orchestra.run.smithi181.stdout: "user": "realm-admin", 2022-09-03T05:17:34.490 INFO:teuthology.orchestra.run.smithi181.stdout: "access_key": "TW0E8EULCERVYU84", 2022-09-03T05:17:34.490 INFO:teuthology.orchestra.run.smithi181.stdout: "secret_key": "EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP" 2022-09-03T05:17:34.491 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:17:34.491 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:17:34.492 INFO:teuthology.orchestra.run.smithi181.stdout: "swift_keys": [], 2022-09-03T05:17:34.492 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": [], 2022-09-03T05:17:34.492 INFO:teuthology.orchestra.run.smithi181.stdout: "op_mask": "read, write, delete", 2022-09-03T05:17:34.493 INFO:teuthology.orchestra.run.smithi181.stdout: "system": true, 2022-09-03T05:17:34.493 INFO:teuthology.orchestra.run.smithi181.stdout: "default_placement": "", 2022-09-03T05:17:34.494 INFO:teuthology.orchestra.run.smithi181.stdout: "default_storage_class": "", 2022-09-03T05:17:34.494 INFO:teuthology.orchestra.run.smithi181.stdout: "placement_tags": [], 2022-09-03T05:17:34.495 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_quota": { 2022-09-03T05:17:34.495 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false, 2022-09-03T05:17:34.496 INFO:teuthology.orchestra.run.smithi181.stdout: "check_on_raw": false, 2022-09-03T05:17:34.496 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size": -1, 2022-09-03T05:17:34.497 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size_kb": 0, 2022-09-03T05:17:34.497 INFO:teuthology.orchestra.run.smithi181.stdout: "max_objects": -1 2022-09-03T05:17:34.498 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:34.498 INFO:teuthology.orchestra.run.smithi181.stdout: "user_quota": { 2022-09-03T05:17:34.499 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false, 2022-09-03T05:17:34.499 INFO:teuthology.orchestra.run.smithi181.stdout: "check_on_raw": false, 2022-09-03T05:17:34.500 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size": -1, 2022-09-03T05:17:34.500 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size_kb": 0, 2022-09-03T05:17:34.500 INFO:teuthology.orchestra.run.smithi181.stdout: "max_objects": -1 2022-09-03T05:17:34.501 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:34.501 INFO:teuthology.orchestra.run.smithi181.stdout: "temp_url_keys": [], 2022-09-03T05:17:34.502 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "rgw", 2022-09-03T05:17:34.502 INFO:teuthology.orchestra.run.smithi181.stdout: "mfa_ids": [] 2022-09-03T05:17:34.503 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:17:34.503 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:17:34.506 INFO:tasks.rgw_multisite:pulling realm configuration to c2 2022-09-03T05:17:34.506 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['realm', 'pull', '--rgw-realm', 'test-realm', '--url', 'http://smithi181.front.sepia.ceph.com:8000', '--access-key', 'TW0E8EULCERVYU84', '--secret', 'EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:34.507 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'realm', 'pull', '--rgw-realm', 'test-realm', '--url', 'http://smithi181.front.sepia.ceph.com:8000', '--access-key', 'TW0E8EULCERVYU84', '--secret', 'EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:34.507 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 realm pull --rgw-realm test-realm --url http://smithi181.front.sepia.ceph.com:8000 --access-key TW0E8EULCERVYU84 --secret EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:17:34.569 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:17:34.570 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:17:34.611 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:17:34.611 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:17:34.611 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-realm", 2022-09-03T05:17:34.612 INFO:teuthology.orchestra.run.smithi189.stdout: "current_period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:17:34.612 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 2 2022-09-03T05:17:34.612 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:17:34.613 INFO:tasks.rgw_multisite:creating zone.. 2022-09-03T05:17:34.614 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'create', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--access-key', 'TW0E8EULCERVYU84', '--secret', 'EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP', '--endpoints', 'http://smithi189.front.sepia.ceph.com:8000', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:34.614 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'create', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--default', '--access-key', 'TW0E8EULCERVYU84', '--secret', 'EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP', '--endpoints', 'http://smithi189.front.sepia.ceph.com:8000', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:34.615 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone create --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --default --access-key TW0E8EULCERVYU84 --secret EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP --endpoints http://smithi189.front.sepia.ceph.com:8000 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:17:34.672 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:17:34.673 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:17:34.699 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:34.701+0000 7f35e2ddd600 0 failed reading obj info from .rgw.root:zone_info.91786ef1-cdb4-4061-8a3e-2a28cc497285: (2) No such file or directory 2022-09-03T05:17:34.699 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:34.701+0000 7f35e2ddd600 0 WARNING: could not read zone params for zone id=91786ef1-cdb4-4061-8a3e-2a28cc497285 name=test-zone1 2022-09-03T05:17:34.707 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:17:34.708 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:17:34.709 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone2", 2022-09-03T05:17:34.709 INFO:teuthology.orchestra.run.smithi189.stdout: "domain_root": "test-zone2.rgw.meta:root", 2022-09-03T05:17:34.710 INFO:teuthology.orchestra.run.smithi189.stdout: "control_pool": "test-zone2.rgw.control", 2022-09-03T05:17:34.710 INFO:teuthology.orchestra.run.smithi189.stdout: "gc_pool": "test-zone2.rgw.log:gc", 2022-09-03T05:17:34.710 INFO:teuthology.orchestra.run.smithi189.stdout: "lc_pool": "test-zone2.rgw.log:lc", 2022-09-03T05:17:34.711 INFO:teuthology.orchestra.run.smithi189.stdout: "log_pool": "test-zone2.rgw.log", 2022-09-03T05:17:34.711 INFO:teuthology.orchestra.run.smithi189.stdout: "intent_log_pool": "test-zone2.rgw.log:intent", 2022-09-03T05:17:34.712 INFO:teuthology.orchestra.run.smithi189.stdout: "usage_log_pool": "test-zone2.rgw.log:usage", 2022-09-03T05:17:34.712 INFO:teuthology.orchestra.run.smithi189.stdout: "roles_pool": "test-zone2.rgw.meta:roles", 2022-09-03T05:17:34.713 INFO:teuthology.orchestra.run.smithi189.stdout: "reshard_pool": "test-zone2.rgw.log:reshard", 2022-09-03T05:17:34.713 INFO:teuthology.orchestra.run.smithi189.stdout: "user_keys_pool": "test-zone2.rgw.meta:users.keys", 2022-09-03T05:17:34.714 INFO:teuthology.orchestra.run.smithi189.stdout: "user_email_pool": "test-zone2.rgw.meta:users.email", 2022-09-03T05:17:34.714 INFO:teuthology.orchestra.run.smithi189.stdout: "user_swift_pool": "test-zone2.rgw.meta:users.swift", 2022-09-03T05:17:34.715 INFO:teuthology.orchestra.run.smithi189.stdout: "user_uid_pool": "test-zone2.rgw.meta:users.uid", 2022-09-03T05:17:34.715 INFO:teuthology.orchestra.run.smithi189.stdout: "otp_pool": "test-zone2.rgw.otp", 2022-09-03T05:17:34.715 INFO:teuthology.orchestra.run.smithi189.stdout: "system_key": { 2022-09-03T05:17:34.716 INFO:teuthology.orchestra.run.smithi189.stdout: "access_key": "TW0E8EULCERVYU84", 2022-09-03T05:17:34.716 INFO:teuthology.orchestra.run.smithi189.stdout: "secret_key": "EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP" 2022-09-03T05:17:34.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:17:34.717 INFO:teuthology.orchestra.run.smithi189.stdout: "placement_pools": [ 2022-09-03T05:17:34.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:17:34.718 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "default-placement", 2022-09-03T05:17:34.719 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:17:34.719 INFO:teuthology.orchestra.run.smithi189.stdout: "index_pool": "test-zone2.rgw.buckets.index", 2022-09-03T05:17:34.720 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_classes": { 2022-09-03T05:17:34.723 INFO:teuthology.orchestra.run.smithi189.stdout: "STANDARD": { 2022-09-03T05:17:34.724 INFO:teuthology.orchestra.run.smithi189.stdout: "data_pool": "test-zone2.rgw.buckets.data" 2022-09-03T05:17:34.724 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:17:34.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:17:34.724 INFO:teuthology.orchestra.run.smithi189.stdout: "data_extra_pool": "test-zone2.rgw.buckets.non-ec", 2022-09-03T05:17:34.725 INFO:teuthology.orchestra.run.smithi189.stdout: "index_type": 0 2022-09-03T05:17:34.725 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:17:34.725 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:17:34.725 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:17:34.726 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:17:34.726 INFO:teuthology.orchestra.run.smithi189.stdout: "notif_pool": "test-zone2.rgw.log:notif" 2022-09-03T05:17:34.726 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:17:34.728 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph osd pool create test-zone2.rgw.buckets.data 64 64 --cluster c2 2022-09-03T05:17:35.228 INFO:teuthology.orchestra.run.smithi189.stderr:pool 'test-zone2.rgw.buckets.data' created 2022-09-03T05:17:35.244 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph osd pool application enable test-zone2.rgw.buckets.data rgw --cluster c2 2022-09-03T05:17:36.251 INFO:teuthology.orchestra.run.smithi189.stderr:enabled application 'rgw' on pool 'test-zone2.rgw.buckets.data' 2022-09-03T05:17:36.268 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'placement', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:36.268 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'placement', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:36.269 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone placement --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm modify --placement-id default-placement --compression random --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:17:36.324 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:17:36.324 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:17:36.346 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:17:36.347 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:17:36.347 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone2", 2022-09-03T05:17:36.348 INFO:teuthology.orchestra.run.smithi189.stdout: "domain_root": "test-zone2.rgw.meta:root", 2022-09-03T05:17:36.348 INFO:teuthology.orchestra.run.smithi189.stdout: "control_pool": "test-zone2.rgw.control", 2022-09-03T05:17:36.348 INFO:teuthology.orchestra.run.smithi189.stdout: "gc_pool": "test-zone2.rgw.log:gc", 2022-09-03T05:17:36.349 INFO:teuthology.orchestra.run.smithi189.stdout: "lc_pool": "test-zone2.rgw.log:lc", 2022-09-03T05:17:36.349 INFO:teuthology.orchestra.run.smithi189.stdout: "log_pool": "test-zone2.rgw.log", 2022-09-03T05:17:36.349 INFO:teuthology.orchestra.run.smithi189.stdout: "intent_log_pool": "test-zone2.rgw.log:intent", 2022-09-03T05:17:36.349 INFO:teuthology.orchestra.run.smithi189.stdout: "usage_log_pool": "test-zone2.rgw.log:usage", 2022-09-03T05:17:36.350 INFO:teuthology.orchestra.run.smithi189.stdout: "roles_pool": "test-zone2.rgw.meta:roles", 2022-09-03T05:17:36.350 INFO:teuthology.orchestra.run.smithi189.stdout: "reshard_pool": "test-zone2.rgw.log:reshard", 2022-09-03T05:17:36.350 INFO:teuthology.orchestra.run.smithi189.stdout: "user_keys_pool": "test-zone2.rgw.meta:users.keys", 2022-09-03T05:17:36.350 INFO:teuthology.orchestra.run.smithi189.stdout: "user_email_pool": "test-zone2.rgw.meta:users.email", 2022-09-03T05:17:36.351 INFO:teuthology.orchestra.run.smithi189.stdout: "user_swift_pool": "test-zone2.rgw.meta:users.swift", 2022-09-03T05:17:36.351 INFO:teuthology.orchestra.run.smithi189.stdout: "user_uid_pool": "test-zone2.rgw.meta:users.uid", 2022-09-03T05:17:36.351 INFO:teuthology.orchestra.run.smithi189.stdout: "otp_pool": "test-zone2.rgw.otp", 2022-09-03T05:17:36.351 INFO:teuthology.orchestra.run.smithi189.stdout: "system_key": { 2022-09-03T05:17:36.351 INFO:teuthology.orchestra.run.smithi189.stdout: "access_key": "TW0E8EULCERVYU84", 2022-09-03T05:17:36.352 INFO:teuthology.orchestra.run.smithi189.stdout: "secret_key": "EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP" 2022-09-03T05:17:36.352 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:17:36.352 INFO:teuthology.orchestra.run.smithi189.stdout: "placement_pools": [ 2022-09-03T05:17:36.352 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:17:36.353 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "default-placement", 2022-09-03T05:17:36.353 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:17:36.353 INFO:teuthology.orchestra.run.smithi189.stdout: "index_pool": "test-zone2.rgw.buckets.index", 2022-09-03T05:17:36.354 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_classes": { 2022-09-03T05:17:36.354 INFO:teuthology.orchestra.run.smithi189.stdout: "STANDARD": { 2022-09-03T05:17:36.354 INFO:teuthology.orchestra.run.smithi189.stdout: "data_pool": "test-zone2.rgw.buckets.data", 2022-09-03T05:17:36.355 INFO:teuthology.orchestra.run.smithi189.stdout: "compression_type": "random" 2022-09-03T05:17:36.355 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:17:36.356 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:17:36.356 INFO:teuthology.orchestra.run.smithi189.stdout: "data_extra_pool": "test-zone2.rgw.buckets.non-ec", 2022-09-03T05:17:36.356 INFO:teuthology.orchestra.run.smithi189.stdout: "index_type": 0 2022-09-03T05:17:36.357 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:17:36.357 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:17:36.357 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:17:36.357 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "", 2022-09-03T05:17:36.358 INFO:teuthology.orchestra.run.smithi189.stdout: "notif_pool": "test-zone2.rgw.log:notif" 2022-09-03T05:17:36.359 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:17:36.361 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:36.362 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:36.363 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:17:36.414 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:17:36.415 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:17:36.427 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:36.425+0000 7f10ee587600 0 period (41ecd2a5-afc0-4eae-9349-7dfe39baa1cb does not have zone e40d88ba-d542-483a-afa2-5de687affd14 configured 2022-09-03T05:17:43.296 INFO:teuthology.orchestra.run.smithi189.stderr:Sending period to new master zone 91786ef1-cdb4-4061-8a3e-2a28cc497285 2022-09-03T05:17:43.322 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:17:43.323 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:17:43.323 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 2, 2022-09-03T05:17:43.324 INFO:teuthology.orchestra.run.smithi189.stdout: "predecessor_uuid": "26ebceb6-9cf2-4657-999c-e205262f80e9", 2022-09-03T05:17:43.324 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": [], 2022-09-03T05:17:43.324 INFO:teuthology.orchestra.run.smithi189.stdout: "period_map": { 2022-09-03T05:17:43.325 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:17:43.325 INFO:teuthology.orchestra.run.smithi189.stdout: "zonegroups": [ 2022-09-03T05:17:43.325 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:17:43.326 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "test-zonegroup", 2022-09-03T05:17:43.326 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zonegroup", 2022-09-03T05:17:43.326 INFO:teuthology.orchestra.run.smithi189.stdout: "api_name": "", 2022-09-03T05:17:43.326 INFO:teuthology.orchestra.run.smithi189.stdout: "is_master": true, 2022-09-03T05:17:43.327 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:17:43.327 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:17:43.327 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:17:43.328 INFO:teuthology.orchestra.run.smithi189.stdout: "hostnames": [], 2022-09-03T05:17:43.328 INFO:teuthology.orchestra.run.smithi189.stdout: "hostnames_s3website": [], 2022-09-03T05:17:43.328 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:17:43.329 INFO:teuthology.orchestra.run.smithi189.stdout: "zones": [ 2022-09-03T05:17:43.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:17:43.329 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:17:43.329 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone1", 2022-09-03T05:17:43.329 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:17:43.330 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:17:43.330 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:17:43.330 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:17:43.330 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:17:43.331 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:17:43.331 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:17:43.331 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:17:43.331 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": true, 2022-09-03T05:17:43.332 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:17:43.332 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:17:43.332 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:17:43.332 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:17:43.333 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:17:43.333 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:17:43.334 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:17:43.334 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:17:43.335 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone2", 2022-09-03T05:17:43.335 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:17:43.337 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi189.front.sepia.ceph.com:8000" 2022-09-03T05:17:43.337 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:17:43.337 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:17:43.337 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:17:43.338 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:17:43.338 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:17:43.338 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:17:43.338 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": true, 2022-09-03T05:17:43.339 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:17:43.339 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:17:43.339 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:17:43.339 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:17:43.339 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:17:43.340 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:17:43.340 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:17:43.340 INFO:teuthology.orchestra.run.smithi189.stdout: "placement_targets": [ 2022-09-03T05:17:43.340 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:17:43.341 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "default-placement", 2022-09-03T05:17:43.341 INFO:teuthology.orchestra.run.smithi189.stdout: "tags": [], 2022-09-03T05:17:43.341 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_classes": [ 2022-09-03T05:17:43.341 INFO:teuthology.orchestra.run.smithi189.stdout: "STANDARD" 2022-09-03T05:17:43.341 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:17:43.342 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:17:43.342 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:17:43.342 INFO:teuthology.orchestra.run.smithi189.stdout: "default_placement": "default-placement", 2022-09-03T05:17:43.343 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:17:43.343 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_policy": { 2022-09-03T05:17:43.344 INFO:teuthology.orchestra.run.smithi189.stdout: "groups": [] 2022-09-03T05:17:43.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:17:43.345 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled_features": [ 2022-09-03T05:17:43.345 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:17:43.345 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:17:43.346 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:17:43.346 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:17:43.346 INFO:teuthology.orchestra.run.smithi189.stdout: "short_zone_ids": [ 2022-09-03T05:17:43.346 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:17:43.347 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:17:43.347 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 1752562153 2022-09-03T05:17:43.347 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:17:43.347 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:17:43.348 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:17:43.348 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 2011734872 2022-09-03T05:17:43.348 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:17:43.348 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:17:43.349 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:17:43.349 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zonegroup": "test-zonegroup", 2022-09-03T05:17:43.349 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:17:43.349 INFO:teuthology.orchestra.run.smithi189.stdout: "period_config": { 2022-09-03T05:17:43.350 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_quota": { 2022-09-03T05:17:43.350 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false, 2022-09-03T05:17:43.350 INFO:teuthology.orchestra.run.smithi189.stdout: "check_on_raw": false, 2022-09-03T05:17:43.350 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size": -1, 2022-09-03T05:17:43.350 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size_kb": 0, 2022-09-03T05:17:43.351 INFO:teuthology.orchestra.run.smithi189.stdout: "max_objects": -1 2022-09-03T05:17:43.351 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:17:43.352 INFO:teuthology.orchestra.run.smithi189.stdout: "user_quota": { 2022-09-03T05:17:43.352 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false, 2022-09-03T05:17:43.352 INFO:teuthology.orchestra.run.smithi189.stdout: "check_on_raw": false, 2022-09-03T05:17:43.352 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size": -1, 2022-09-03T05:17:43.353 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size_kb": 0, 2022-09-03T05:17:43.353 INFO:teuthology.orchestra.run.smithi189.stdout: "max_objects": -1 2022-09-03T05:17:43.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:17:43.353 INFO:teuthology.orchestra.run.smithi189.stdout: "user_ratelimit": { 2022-09-03T05:17:43.353 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:17:43.354 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:17:43.354 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:17:43.354 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:17:43.354 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:17:43.355 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:17:43.355 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_ratelimit": { 2022-09-03T05:17:43.355 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:17:43.355 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:17:43.356 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:17:43.356 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:17:43.356 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:17:43.356 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:17:43.357 INFO:teuthology.orchestra.run.smithi189.stdout: "anonymous_ratelimit": { 2022-09-03T05:17:43.358 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:17:43.358 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:17:43.358 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:17:43.359 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:17:43.359 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:17:43.359 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:17:43.359 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:17:43.359 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:17:43.360 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_name": "test-realm", 2022-09-03T05:17:43.360 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:17:43.360 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:17:43.361 DEBUG:tasks.rgw.c2.client.0:waiting for process to exit 2022-09-03T05:17:43.361 INFO:teuthology.orchestra.run:waiting for 300 2022-09-03T05:17:43.362 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T05:17:43.364+0000 7fb826049700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term radosgw --rgw-frontends beast port=8000 -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground (PID: 110300) UID: 0 2022-09-03T05:17:43.362 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T05:17:43.364+0000 7fb82a2b4600 -1 shutting down 2022-09-03T05:17:43.460 INFO:tasks.rgw.c2.client.0:Stopped 2022-09-03T05:17:43.461 INFO:tasks.rgw.c2.client.0:Restarting daemon 2022-09-03T05:17:43.462 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c2.client.0.stdout 2>&1 2022-09-03T05:17:43.504 INFO:tasks.rgw.c2.client.0:Started 2022-09-03T05:17:43.505 DEBUG:teuthology.orchestra.run.smithi189:> curl http://smithi189.front.sepia.ceph.com:8000 2022-09-03T05:17:43.528 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T05:17:43.529 INFO:teuthology.orchestra.run.smithi189.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T05:17:43.530 INFO:teuthology.orchestra.run.smithi189.stderr: Dload Upload Total Spent Left Speed 2022-09-03T05:17:43.530 INFO:teuthology.orchestra.run.smithi189.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi189.front.sepia.ceph.com port 8000: Connection refused 2022-09-03T05:17:44.531 DEBUG:teuthology.orchestra.run.smithi189:> curl http://smithi189.front.sepia.ceph.com:8000 2022-09-03T05:17:44.559 INFO:teuthology.orchestra.run.smithi189.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T05:17:44.560 INFO:teuthology.orchestra.run.smithi189.stderr: Dload Upload Total Spent Left Speed 2022-09-03T05:17:44.561 INFO:teuthology.orchestra.run.smithi189.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 208k 0 --:--:-- --:--:-- --:--:-- 208k 2022-09-03T05:17:44.562 INFO:teuthology.orchestra.run.smithi189.stdout:anonymous 2022-09-03T05:17:44.562 INFO:tasks.rgw_multisite:creating zone.. 2022-09-03T05:17:44.563 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'create', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', 'TW0E8EULCERVYU84', '--secret', 'EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP', '--endpoints', 'http://smithi181.front.sepia.ceph.com:8001', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:44.563 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'create', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', 'TW0E8EULCERVYU84', '--secret', 'EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP', '--endpoints', 'http://smithi181.front.sepia.ceph.com:8001', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:44.563 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone create --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --access-key TW0E8EULCERVYU84 --secret EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP --endpoints http://smithi181.front.sepia.ceph.com:8001 --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:17:44.624 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:17:44.624 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:17:44.654 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:17:44.656+0000 7f3294152600 0 failed reading obj info from .rgw.root:zone_info.e40d88ba-d542-483a-afa2-5de687affd14: (2) No such file or directory 2022-09-03T05:17:44.655 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:17:44.656+0000 7f3294152600 0 WARNING: could not read zone params for zone id=e40d88ba-d542-483a-afa2-5de687affd14 name=test-zone2 2022-09-03T05:17:44.658 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:17:44.659 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:17:44.659 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone3", 2022-09-03T05:17:44.659 INFO:teuthology.orchestra.run.smithi181.stdout: "domain_root": "test-zone3.rgw.meta:root", 2022-09-03T05:17:44.660 INFO:teuthology.orchestra.run.smithi181.stdout: "control_pool": "test-zone3.rgw.control", 2022-09-03T05:17:44.660 INFO:teuthology.orchestra.run.smithi181.stdout: "gc_pool": "test-zone3.rgw.log:gc", 2022-09-03T05:17:44.661 INFO:teuthology.orchestra.run.smithi181.stdout: "lc_pool": "test-zone3.rgw.log:lc", 2022-09-03T05:17:44.661 INFO:teuthology.orchestra.run.smithi181.stdout: "log_pool": "test-zone3.rgw.log", 2022-09-03T05:17:44.662 INFO:teuthology.orchestra.run.smithi181.stdout: "intent_log_pool": "test-zone3.rgw.log:intent", 2022-09-03T05:17:44.662 INFO:teuthology.orchestra.run.smithi181.stdout: "usage_log_pool": "test-zone3.rgw.log:usage", 2022-09-03T05:17:44.662 INFO:teuthology.orchestra.run.smithi181.stdout: "roles_pool": "test-zone3.rgw.meta:roles", 2022-09-03T05:17:44.663 INFO:teuthology.orchestra.run.smithi181.stdout: "reshard_pool": "test-zone3.rgw.log:reshard", 2022-09-03T05:17:44.663 INFO:teuthology.orchestra.run.smithi181.stdout: "user_keys_pool": "test-zone3.rgw.meta:users.keys", 2022-09-03T05:17:44.664 INFO:teuthology.orchestra.run.smithi181.stdout: "user_email_pool": "test-zone3.rgw.meta:users.email", 2022-09-03T05:17:44.664 INFO:teuthology.orchestra.run.smithi181.stdout: "user_swift_pool": "test-zone3.rgw.meta:users.swift", 2022-09-03T05:17:44.664 INFO:teuthology.orchestra.run.smithi181.stdout: "user_uid_pool": "test-zone3.rgw.meta:users.uid", 2022-09-03T05:17:44.665 INFO:teuthology.orchestra.run.smithi181.stdout: "otp_pool": "test-zone3.rgw.otp", 2022-09-03T05:17:44.665 INFO:teuthology.orchestra.run.smithi181.stdout: "system_key": { 2022-09-03T05:17:44.665 INFO:teuthology.orchestra.run.smithi181.stdout: "access_key": "TW0E8EULCERVYU84", 2022-09-03T05:17:44.665 INFO:teuthology.orchestra.run.smithi181.stdout: "secret_key": "EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP" 2022-09-03T05:17:44.667 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:44.667 INFO:teuthology.orchestra.run.smithi181.stdout: "placement_pools": [ 2022-09-03T05:17:44.667 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:17:44.667 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "default-placement", 2022-09-03T05:17:44.668 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:17:44.668 INFO:teuthology.orchestra.run.smithi181.stdout: "index_pool": "test-zone3.rgw.buckets.index", 2022-09-03T05:17:44.668 INFO:teuthology.orchestra.run.smithi181.stdout: "storage_classes": { 2022-09-03T05:17:44.668 INFO:teuthology.orchestra.run.smithi181.stdout: "STANDARD": { 2022-09-03T05:17:44.669 INFO:teuthology.orchestra.run.smithi181.stdout: "data_pool": "test-zone3.rgw.buckets.data" 2022-09-03T05:17:44.669 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:17:44.669 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:44.670 INFO:teuthology.orchestra.run.smithi181.stdout: "data_extra_pool": "test-zone3.rgw.buckets.non-ec", 2022-09-03T05:17:44.670 INFO:teuthology.orchestra.run.smithi181.stdout: "index_type": 0 2022-09-03T05:17:44.670 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:17:44.670 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:17:44.671 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:17:44.671 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:17:44.671 INFO:teuthology.orchestra.run.smithi181.stdout: "notif_pool": "test-zone3.rgw.log:notif" 2022-09-03T05:17:44.671 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:17:44.673 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph osd pool create test-zone3.rgw.buckets.data 64 64 --cluster c1 2022-09-03T05:17:46.026 INFO:teuthology.orchestra.run.smithi181.stderr:pool 'test-zone3.rgw.buckets.data' created 2022-09-03T05:17:46.044 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph osd pool application enable test-zone3.rgw.buckets.data rgw --cluster c1 2022-09-03T05:17:47.047 INFO:teuthology.orchestra.run.smithi181.stderr:enabled application 'rgw' on pool 'test-zone3.rgw.buckets.data' 2022-09-03T05:17:47.060 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'placement', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:47.061 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'placement', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:47.061 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone placement --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm modify --placement-id default-placement --compression random --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:17:47.113 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:17:47.114 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:17:47.135 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:17:47.136 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:17:47.136 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone3", 2022-09-03T05:17:47.136 INFO:teuthology.orchestra.run.smithi181.stdout: "domain_root": "test-zone3.rgw.meta:root", 2022-09-03T05:17:47.137 INFO:teuthology.orchestra.run.smithi181.stdout: "control_pool": "test-zone3.rgw.control", 2022-09-03T05:17:47.137 INFO:teuthology.orchestra.run.smithi181.stdout: "gc_pool": "test-zone3.rgw.log:gc", 2022-09-03T05:17:47.137 INFO:teuthology.orchestra.run.smithi181.stdout: "lc_pool": "test-zone3.rgw.log:lc", 2022-09-03T05:17:47.138 INFO:teuthology.orchestra.run.smithi181.stdout: "log_pool": "test-zone3.rgw.log", 2022-09-03T05:17:47.139 INFO:teuthology.orchestra.run.smithi181.stdout: "intent_log_pool": "test-zone3.rgw.log:intent", 2022-09-03T05:17:47.139 INFO:teuthology.orchestra.run.smithi181.stdout: "usage_log_pool": "test-zone3.rgw.log:usage", 2022-09-03T05:17:47.139 INFO:teuthology.orchestra.run.smithi181.stdout: "roles_pool": "test-zone3.rgw.meta:roles", 2022-09-03T05:17:47.140 INFO:teuthology.orchestra.run.smithi181.stdout: "reshard_pool": "test-zone3.rgw.log:reshard", 2022-09-03T05:17:47.140 INFO:teuthology.orchestra.run.smithi181.stdout: "user_keys_pool": "test-zone3.rgw.meta:users.keys", 2022-09-03T05:17:47.140 INFO:teuthology.orchestra.run.smithi181.stdout: "user_email_pool": "test-zone3.rgw.meta:users.email", 2022-09-03T05:17:47.141 INFO:teuthology.orchestra.run.smithi181.stdout: "user_swift_pool": "test-zone3.rgw.meta:users.swift", 2022-09-03T05:17:47.141 INFO:teuthology.orchestra.run.smithi181.stdout: "user_uid_pool": "test-zone3.rgw.meta:users.uid", 2022-09-03T05:17:47.141 INFO:teuthology.orchestra.run.smithi181.stdout: "otp_pool": "test-zone3.rgw.otp", 2022-09-03T05:17:47.141 INFO:teuthology.orchestra.run.smithi181.stdout: "system_key": { 2022-09-03T05:17:47.142 INFO:teuthology.orchestra.run.smithi181.stdout: "access_key": "TW0E8EULCERVYU84", 2022-09-03T05:17:47.142 INFO:teuthology.orchestra.run.smithi181.stdout: "secret_key": "EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP" 2022-09-03T05:17:47.142 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:47.142 INFO:teuthology.orchestra.run.smithi181.stdout: "placement_pools": [ 2022-09-03T05:17:47.143 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:17:47.143 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "default-placement", 2022-09-03T05:17:47.143 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:17:47.143 INFO:teuthology.orchestra.run.smithi181.stdout: "index_pool": "test-zone3.rgw.buckets.index", 2022-09-03T05:17:47.144 INFO:teuthology.orchestra.run.smithi181.stdout: "storage_classes": { 2022-09-03T05:17:47.144 INFO:teuthology.orchestra.run.smithi181.stdout: "STANDARD": { 2022-09-03T05:17:47.144 INFO:teuthology.orchestra.run.smithi181.stdout: "data_pool": "test-zone3.rgw.buckets.data", 2022-09-03T05:17:47.144 INFO:teuthology.orchestra.run.smithi181.stdout: "compression_type": "random" 2022-09-03T05:17:47.145 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:17:47.145 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:47.145 INFO:teuthology.orchestra.run.smithi181.stdout: "data_extra_pool": "test-zone3.rgw.buckets.non-ec", 2022-09-03T05:17:47.145 INFO:teuthology.orchestra.run.smithi181.stdout: "index_type": 0 2022-09-03T05:17:47.146 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:17:47.146 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:17:47.146 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:17:47.146 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "", 2022-09-03T05:17:47.147 INFO:teuthology.orchestra.run.smithi181.stdout: "notif_pool": "test-zone3.rgw.log:notif" 2022-09-03T05:17:47.147 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:17:47.148 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:47.148 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:47.149 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:17:47.197 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:17:47.198 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:17:47.206 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:17:47.208+0000 7f13f362a600 0 period (41ecd2a5-afc0-4eae-9349-7dfe39baa1cb does not have zone 6ebb780b-73b6-47d3-ad4f-1dc62076c1f4 configured 2022-09-03T05:17:54.069 INFO:teuthology.orchestra.run.smithi181.stderr:Sending period to new master zone 91786ef1-cdb4-4061-8a3e-2a28cc497285 2022-09-03T05:17:55.099 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:17:55.099 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:17:55.099 INFO:teuthology.orchestra.run.smithi181.stdout: "epoch": 3, 2022-09-03T05:17:55.100 INFO:teuthology.orchestra.run.smithi181.stdout: "predecessor_uuid": "26ebceb6-9cf2-4657-999c-e205262f80e9", 2022-09-03T05:17:55.100 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": [], 2022-09-03T05:17:55.100 INFO:teuthology.orchestra.run.smithi181.stdout: "period_map": { 2022-09-03T05:17:55.101 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:17:55.101 INFO:teuthology.orchestra.run.smithi181.stdout: "zonegroups": [ 2022-09-03T05:17:55.102 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:17:55.102 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "test-zonegroup", 2022-09-03T05:17:55.102 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zonegroup", 2022-09-03T05:17:55.103 INFO:teuthology.orchestra.run.smithi181.stdout: "api_name": "", 2022-09-03T05:17:55.103 INFO:teuthology.orchestra.run.smithi181.stdout: "is_master": true, 2022-09-03T05:17:55.103 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:17:55.103 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:17:55.104 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:17:55.104 INFO:teuthology.orchestra.run.smithi181.stdout: "hostnames": [], 2022-09-03T05:17:55.104 INFO:teuthology.orchestra.run.smithi181.stdout: "hostnames_s3website": [], 2022-09-03T05:17:55.104 INFO:teuthology.orchestra.run.smithi181.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:17:55.105 INFO:teuthology.orchestra.run.smithi181.stdout: "zones": [ 2022-09-03T05:17:55.105 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:17:55.105 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:17:55.105 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone3", 2022-09-03T05:17:55.106 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:17:55.106 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi181.front.sepia.ceph.com:8001" 2022-09-03T05:17:55.106 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:17:55.106 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:17:55.107 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": true, 2022-09-03T05:17:55.107 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:17:55.107 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:17:55.107 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "", 2022-09-03T05:17:55.108 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": true, 2022-09-03T05:17:55.108 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [], 2022-09-03T05:17:55.108 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:17:55.108 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:17:55.109 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:17:55.109 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:17:55.109 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:55.109 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:17:55.109 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:17:55.110 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone1", 2022-09-03T05:17:55.110 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:17:55.110 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:17:55.110 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:17:55.111 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:17:55.111 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": true, 2022-09-03T05:17:55.111 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:17:55.111 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:17:55.112 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "", 2022-09-03T05:17:55.112 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": true, 2022-09-03T05:17:55.112 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [], 2022-09-03T05:17:55.112 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:17:55.112 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:17:55.113 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:17:55.113 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:17:55.113 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:55.113 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:17:55.114 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:17:55.114 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone2", 2022-09-03T05:17:55.114 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:17:55.114 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi189.front.sepia.ceph.com:8000" 2022-09-03T05:17:55.115 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:17:55.115 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:17:55.115 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": true, 2022-09-03T05:17:55.115 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:17:55.116 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:17:55.116 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "", 2022-09-03T05:17:55.116 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": true, 2022-09-03T05:17:55.117 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [], 2022-09-03T05:17:55.117 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:17:55.117 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:17:55.117 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:17:55.117 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:17:55.118 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:17:55.118 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:17:55.118 INFO:teuthology.orchestra.run.smithi181.stdout: "placement_targets": [ 2022-09-03T05:17:55.118 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:17:55.119 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "default-placement", 2022-09-03T05:17:55.119 INFO:teuthology.orchestra.run.smithi181.stdout: "tags": [], 2022-09-03T05:17:55.119 INFO:teuthology.orchestra.run.smithi181.stdout: "storage_classes": [ 2022-09-03T05:17:55.119 INFO:teuthology.orchestra.run.smithi181.stdout: "STANDARD" 2022-09-03T05:17:55.119 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:17:55.120 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:17:55.120 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:17:55.120 INFO:teuthology.orchestra.run.smithi181.stdout: "default_placement": "default-placement", 2022-09-03T05:17:55.120 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:17:55.121 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_policy": { 2022-09-03T05:17:55.121 INFO:teuthology.orchestra.run.smithi181.stdout: "groups": [] 2022-09-03T05:17:55.121 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:55.121 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled_features": [ 2022-09-03T05:17:55.121 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:17:55.122 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:17:55.122 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:17:55.122 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:17:55.122 INFO:teuthology.orchestra.run.smithi181.stdout: "short_zone_ids": [ 2022-09-03T05:17:55.123 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:17:55.123 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:17:55.123 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 327838863 2022-09-03T05:17:55.123 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:55.123 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:17:55.124 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:17:55.124 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 1752562153 2022-09-03T05:17:55.125 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:55.125 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:17:55.125 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:17:55.126 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 2011734872 2022-09-03T05:17:55.126 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:17:55.126 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:17:55.127 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:55.128 INFO:teuthology.orchestra.run.smithi181.stdout: "master_zonegroup": "test-zonegroup", 2022-09-03T05:17:55.129 INFO:teuthology.orchestra.run.smithi181.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:17:55.129 INFO:teuthology.orchestra.run.smithi181.stdout: "period_config": { 2022-09-03T05:17:55.129 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_quota": { 2022-09-03T05:17:55.129 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false, 2022-09-03T05:17:55.130 INFO:teuthology.orchestra.run.smithi181.stdout: "check_on_raw": false, 2022-09-03T05:17:55.130 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size": -1, 2022-09-03T05:17:55.130 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size_kb": 0, 2022-09-03T05:17:55.131 INFO:teuthology.orchestra.run.smithi181.stdout: "max_objects": -1 2022-09-03T05:17:55.131 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:55.131 INFO:teuthology.orchestra.run.smithi181.stdout: "user_quota": { 2022-09-03T05:17:55.131 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false, 2022-09-03T05:17:55.132 INFO:teuthology.orchestra.run.smithi181.stdout: "check_on_raw": false, 2022-09-03T05:17:55.132 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size": -1, 2022-09-03T05:17:55.133 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size_kb": 0, 2022-09-03T05:17:55.133 INFO:teuthology.orchestra.run.smithi181.stdout: "max_objects": -1 2022-09-03T05:17:55.133 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:55.134 INFO:teuthology.orchestra.run.smithi181.stdout: "user_ratelimit": { 2022-09-03T05:17:55.134 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_ops": 0, 2022-09-03T05:17:55.135 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_ops": 0, 2022-09-03T05:17:55.135 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_bytes": 0, 2022-09-03T05:17:55.136 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_bytes": 0, 2022-09-03T05:17:55.136 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false 2022-09-03T05:17:55.136 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:55.137 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_ratelimit": { 2022-09-03T05:17:55.137 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_ops": 0, 2022-09-03T05:17:55.137 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_ops": 0, 2022-09-03T05:17:55.138 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_bytes": 0, 2022-09-03T05:17:55.138 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_bytes": 0, 2022-09-03T05:17:55.138 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false 2022-09-03T05:17:55.139 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:55.139 INFO:teuthology.orchestra.run.smithi181.stdout: "anonymous_ratelimit": { 2022-09-03T05:17:55.139 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_ops": 0, 2022-09-03T05:17:55.139 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_ops": 0, 2022-09-03T05:17:55.140 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_bytes": 0, 2022-09-03T05:17:55.140 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_bytes": 0, 2022-09-03T05:17:55.140 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false 2022-09-03T05:17:55.140 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:17:55.141 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:17:55.141 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:17:55.141 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_name": "test-realm", 2022-09-03T05:17:55.141 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:17:55.141 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:17:55.142 DEBUG:tasks.rgw.c1.client.1:waiting for process to exit 2022-09-03T05:17:55.143 INFO:teuthology.orchestra.run:waiting for 300 2022-09-03T05:17:55.144 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T05:17:55.146+0000 7fa6ba4fa700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term radosgw --rgw-frontends beast port=8001 -n client.1 --cluster c1 -k /etc/ceph/c1.client.1.keyring --log-file /var/log/ceph/rgw.c1.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.1.sock --foreground (PID: 109799) UID: 0 2022-09-03T05:17:55.144 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T05:17:55.146+0000 7fa6be765600 -1 shutting down 2022-09-03T05:17:55.211 INFO:tasks.rgw.c1.client.1:Stopped 2022-09-03T05:17:55.211 INFO:tasks.rgw.c1.client.1:Restarting daemon 2022-09-03T05:17:55.212 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8001' -n client.1 --cluster c1 -k /etc/ceph/c1.client.1.keyring --log-file /var/log/ceph/rgw.c1.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.1.sock --foreground --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c1.client.1.stdout 2>&1 2022-09-03T05:17:55.254 INFO:tasks.rgw.c1.client.1:Started 2022-09-03T05:17:55.254 DEBUG:teuthology.orchestra.run.smithi181:> curl http://smithi181.front.sepia.ceph.com:8001 2022-09-03T05:17:55.276 INFO:teuthology.orchestra.run.smithi181.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T05:17:55.276 INFO:teuthology.orchestra.run.smithi181.stderr: Dload Upload Total Spent Left Speed 2022-09-03T05:17:55.277 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T05:17:55.278 INFO:teuthology.orchestra.run.smithi181.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi181.front.sepia.ceph.com port 8001: Connection refused 2022-09-03T05:17:56.280 DEBUG:teuthology.orchestra.run.smithi181:> curl http://smithi181.front.sepia.ceph.com:8001 2022-09-03T05:17:56.308 INFO:teuthology.orchestra.run.smithi181.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T05:17:56.308 INFO:teuthology.orchestra.run.smithi181.stderr: Dload Upload Total Spent Left Speed 2022-09-03T05:17:56.309 INFO:teuthology.orchestra.run.smithi181.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 208k 0 --:--:-- --:--:-- --:--:-- 208k 2022-09-03T05:17:56.310 INFO:teuthology.orchestra.run.smithi181.stdout:anonymous 2022-09-03T05:17:56.311 INFO:tasks.rgw_multisite:pulling realm configuration to c2 2022-09-03T05:17:56.311 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['realm', 'pull', '--rgw-realm', 'test-realm', '--url', 'http://smithi181.front.sepia.ceph.com:8000', '--access-key', 'TW0E8EULCERVYU84', '--secret', 'EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP', '--url', 'http://smithi181.front.sepia.ceph.com:8000', '--access-key', 'TW0E8EULCERVYU84', '--secret', 'EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:56.313 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'realm', 'pull', '--rgw-realm', 'test-realm', '--url', 'http://smithi181.front.sepia.ceph.com:8000', '--access-key', 'TW0E8EULCERVYU84', '--secret', 'EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP', '--url', 'http://smithi181.front.sepia.ceph.com:8000', '--access-key', 'TW0E8EULCERVYU84', '--secret', 'EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:56.313 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 realm pull --rgw-realm test-realm --url http://smithi181.front.sepia.ceph.com:8000 --access-key TW0E8EULCERVYU84 --secret EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP --url http://smithi181.front.sepia.ceph.com:8000 --access-key TW0E8EULCERVYU84 --secret EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:17:56.387 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:17:56.387 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:17:56.440 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T05:17:56.441+0000 7f6b1a0f8700 -1 rgw realm watcher: RGWRealmWatcher::handle_error oid=realms.06c68c5c-7fff-4622-880e-066dbd09a96b.control err=-107 2022-09-03T05:17:56.479 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:17:56.479 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:17:56.479 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-realm", 2022-09-03T05:17:56.480 INFO:teuthology.orchestra.run.smithi189.stdout: "current_period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:17:56.480 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 2 2022-09-03T05:17:56.480 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:17:56.481 INFO:tasks.rgw_multisite:creating zone.. 2022-09-03T05:17:56.482 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'create', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', 'TW0E8EULCERVYU84', '--secret', 'EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP', '--endpoints', 'http://smithi189.front.sepia.ceph.com:8001', '--tier-type', 'pubsub', '--sync-from-all=0', '--sync-from', 'test-zone1', '--tier-config', 'start_with_full_sync=false,event_retention_days=7', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:56.483 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'create', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', 'TW0E8EULCERVYU84', '--secret', 'EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP', '--endpoints', 'http://smithi189.front.sepia.ceph.com:8001', '--tier-type', 'pubsub', '--sync-from-all=0', '--sync-from', 'test-zone1', '--tier-config', 'start_with_full_sync=false,event_retention_days=7', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:56.483 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone create --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --access-key TW0E8EULCERVYU84 --secret EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP --endpoints http://smithi189.front.sepia.ceph.com:8001 --tier-type pubsub --sync-from-all=0 --sync-from test-zone1 --tier-config start_with_full_sync=false,event_retention_days=7 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:17:56.556 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:17:56.556 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:17:56.593 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:56.594+0000 7f9618722600 0 failed reading obj info from .rgw.root:zone_info.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4: (2) No such file or directory 2022-09-03T05:17:56.593 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:56.594+0000 7f9618722600 0 WARNING: could not read zone params for zone id=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4 name=test-zone3 2022-09-03T05:17:56.594 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:56.595+0000 7f9618722600 0 failed reading obj info from .rgw.root:zone_info.91786ef1-cdb4-4061-8a3e-2a28cc497285: (2) No such file or directory 2022-09-03T05:17:56.594 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:56.595+0000 7f9618722600 0 WARNING: could not read zone params for zone id=91786ef1-cdb4-4061-8a3e-2a28cc497285 name=test-zone1 2022-09-03T05:17:56.600 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:17:56.601 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:17:56.601 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone4", 2022-09-03T05:17:56.601 INFO:teuthology.orchestra.run.smithi189.stdout: "domain_root": "test-zone4.rgw.meta:root", 2022-09-03T05:17:56.602 INFO:teuthology.orchestra.run.smithi189.stdout: "control_pool": "test-zone4.rgw.control", 2022-09-03T05:17:56.602 INFO:teuthology.orchestra.run.smithi189.stdout: "gc_pool": "test-zone4.rgw.log:gc", 2022-09-03T05:17:56.602 INFO:teuthology.orchestra.run.smithi189.stdout: "lc_pool": "test-zone4.rgw.log:lc", 2022-09-03T05:17:56.602 INFO:teuthology.orchestra.run.smithi189.stdout: "log_pool": "test-zone4.rgw.log", 2022-09-03T05:17:56.603 INFO:teuthology.orchestra.run.smithi189.stdout: "intent_log_pool": "test-zone4.rgw.log:intent", 2022-09-03T05:17:56.603 INFO:teuthology.orchestra.run.smithi189.stdout: "usage_log_pool": "test-zone4.rgw.log:usage", 2022-09-03T05:17:56.603 INFO:teuthology.orchestra.run.smithi189.stdout: "roles_pool": "test-zone4.rgw.meta:roles", 2022-09-03T05:17:56.603 INFO:teuthology.orchestra.run.smithi189.stdout: "reshard_pool": "test-zone4.rgw.log:reshard", 2022-09-03T05:17:56.604 INFO:teuthology.orchestra.run.smithi189.stdout: "user_keys_pool": "test-zone4.rgw.meta:users.keys", 2022-09-03T05:17:56.604 INFO:teuthology.orchestra.run.smithi189.stdout: "user_email_pool": "test-zone4.rgw.meta:users.email", 2022-09-03T05:17:56.604 INFO:teuthology.orchestra.run.smithi189.stdout: "user_swift_pool": "test-zone4.rgw.meta:users.swift", 2022-09-03T05:17:56.604 INFO:teuthology.orchestra.run.smithi189.stdout: "user_uid_pool": "test-zone4.rgw.meta:users.uid", 2022-09-03T05:17:56.605 INFO:teuthology.orchestra.run.smithi189.stdout: "otp_pool": "test-zone4.rgw.otp", 2022-09-03T05:17:56.605 INFO:teuthology.orchestra.run.smithi189.stdout: "system_key": { 2022-09-03T05:17:56.605 INFO:teuthology.orchestra.run.smithi189.stdout: "access_key": "TW0E8EULCERVYU84", 2022-09-03T05:17:56.605 INFO:teuthology.orchestra.run.smithi189.stdout: "secret_key": "EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP" 2022-09-03T05:17:56.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:17:56.606 INFO:teuthology.orchestra.run.smithi189.stdout: "placement_pools": [ 2022-09-03T05:17:56.606 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:17:56.606 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "default-placement", 2022-09-03T05:17:56.607 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:17:56.607 INFO:teuthology.orchestra.run.smithi189.stdout: "index_pool": "test-zone4.rgw.buckets.index", 2022-09-03T05:17:56.607 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_classes": { 2022-09-03T05:17:56.607 INFO:teuthology.orchestra.run.smithi189.stdout: "STANDARD": { 2022-09-03T05:17:56.608 INFO:teuthology.orchestra.run.smithi189.stdout: "data_pool": "test-zone4.rgw.buckets.data" 2022-09-03T05:17:56.608 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:17:56.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:17:56.608 INFO:teuthology.orchestra.run.smithi189.stdout: "data_extra_pool": "test-zone4.rgw.buckets.non-ec", 2022-09-03T05:17:56.608 INFO:teuthology.orchestra.run.smithi189.stdout: "index_type": 0 2022-09-03T05:17:56.609 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:17:56.609 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:17:56.609 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:17:56.609 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_config": { 2022-09-03T05:17:56.610 INFO:teuthology.orchestra.run.smithi189.stdout: "event_retention_days": 7, 2022-09-03T05:17:56.610 INFO:teuthology.orchestra.run.smithi189.stdout: "start_with_full_sync": false 2022-09-03T05:17:56.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:17:56.610 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:17:56.611 INFO:teuthology.orchestra.run.smithi189.stdout: "notif_pool": "test-zone4.rgw.log:notif" 2022-09-03T05:17:56.611 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:17:56.612 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph osd pool create test-zone4.rgw.buckets.data 64 64 --cluster c2 2022-09-03T05:17:57.296 INFO:teuthology.orchestra.run.smithi189.stderr:pool 'test-zone4.rgw.buckets.data' created 2022-09-03T05:17:57.316 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph osd pool application enable test-zone4.rgw.buckets.data rgw --cluster c2 2022-09-03T05:17:58.319 INFO:teuthology.orchestra.run.smithi189.stderr:enabled application 'rgw' on pool 'test-zone4.rgw.buckets.data' 2022-09-03T05:17:58.336 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'placement', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:58.336 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'placement', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', 'modify', '--placement-id', 'default-placement', '--compression', 'random', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:58.337 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone placement --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm modify --placement-id default-placement --compression random --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:17:58.388 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:17:58.389 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:17:58.411 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:17:58.412 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:17:58.412 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone4", 2022-09-03T05:17:58.413 INFO:teuthology.orchestra.run.smithi189.stdout: "domain_root": "test-zone4.rgw.meta:root", 2022-09-03T05:17:58.413 INFO:teuthology.orchestra.run.smithi189.stdout: "control_pool": "test-zone4.rgw.control", 2022-09-03T05:17:58.414 INFO:teuthology.orchestra.run.smithi189.stdout: "gc_pool": "test-zone4.rgw.log:gc", 2022-09-03T05:17:58.414 INFO:teuthology.orchestra.run.smithi189.stdout: "lc_pool": "test-zone4.rgw.log:lc", 2022-09-03T05:17:58.415 INFO:teuthology.orchestra.run.smithi189.stdout: "log_pool": "test-zone4.rgw.log", 2022-09-03T05:17:58.415 INFO:teuthology.orchestra.run.smithi189.stdout: "intent_log_pool": "test-zone4.rgw.log:intent", 2022-09-03T05:17:58.416 INFO:teuthology.orchestra.run.smithi189.stdout: "usage_log_pool": "test-zone4.rgw.log:usage", 2022-09-03T05:17:58.416 INFO:teuthology.orchestra.run.smithi189.stdout: "roles_pool": "test-zone4.rgw.meta:roles", 2022-09-03T05:17:58.416 INFO:teuthology.orchestra.run.smithi189.stdout: "reshard_pool": "test-zone4.rgw.log:reshard", 2022-09-03T05:17:58.417 INFO:teuthology.orchestra.run.smithi189.stdout: "user_keys_pool": "test-zone4.rgw.meta:users.keys", 2022-09-03T05:17:58.417 INFO:teuthology.orchestra.run.smithi189.stdout: "user_email_pool": "test-zone4.rgw.meta:users.email", 2022-09-03T05:17:58.418 INFO:teuthology.orchestra.run.smithi189.stdout: "user_swift_pool": "test-zone4.rgw.meta:users.swift", 2022-09-03T05:17:58.418 INFO:teuthology.orchestra.run.smithi189.stdout: "user_uid_pool": "test-zone4.rgw.meta:users.uid", 2022-09-03T05:17:58.418 INFO:teuthology.orchestra.run.smithi189.stdout: "otp_pool": "test-zone4.rgw.otp", 2022-09-03T05:17:58.419 INFO:teuthology.orchestra.run.smithi189.stdout: "system_key": { 2022-09-03T05:17:58.419 INFO:teuthology.orchestra.run.smithi189.stdout: "access_key": "TW0E8EULCERVYU84", 2022-09-03T05:17:58.420 INFO:teuthology.orchestra.run.smithi189.stdout: "secret_key": "EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP" 2022-09-03T05:17:58.420 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:17:58.420 INFO:teuthology.orchestra.run.smithi189.stdout: "placement_pools": [ 2022-09-03T05:17:58.421 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:17:58.421 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "default-placement", 2022-09-03T05:17:58.421 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:17:58.422 INFO:teuthology.orchestra.run.smithi189.stdout: "index_pool": "test-zone4.rgw.buckets.index", 2022-09-03T05:17:58.422 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_classes": { 2022-09-03T05:17:58.423 INFO:teuthology.orchestra.run.smithi189.stdout: "STANDARD": { 2022-09-03T05:17:58.423 INFO:teuthology.orchestra.run.smithi189.stdout: "data_pool": "test-zone4.rgw.buckets.data", 2022-09-03T05:17:58.424 INFO:teuthology.orchestra.run.smithi189.stdout: "compression_type": "random" 2022-09-03T05:17:58.424 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:17:58.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:17:58.425 INFO:teuthology.orchestra.run.smithi189.stdout: "data_extra_pool": "test-zone4.rgw.buckets.non-ec", 2022-09-03T05:17:58.425 INFO:teuthology.orchestra.run.smithi189.stdout: "index_type": 0 2022-09-03T05:17:58.426 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:17:58.426 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:17:58.427 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:17:58.427 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_config": { 2022-09-03T05:17:58.428 INFO:teuthology.orchestra.run.smithi189.stdout: "event_retention_days": 7, 2022-09-03T05:17:58.428 INFO:teuthology.orchestra.run.smithi189.stdout: "start_with_full_sync": false 2022-09-03T05:17:58.429 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:17:58.429 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "", 2022-09-03T05:17:58.429 INFO:teuthology.orchestra.run.smithi189.stdout: "notif_pool": "test-zone4.rgw.log:notif" 2022-09-03T05:17:58.430 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:17:58.431 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:58.432 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:17:58.432 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:17:58.481 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:17:58.481 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:17:58.491 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:17:58.492+0000 7f592fc0d600 0 period (41ecd2a5-afc0-4eae-9349-7dfe39baa1cb does not have zone 878fc3d4-c5f9-4296-8cc0-f2a8e029bc48 configured 2022-09-03T05:18:05.374 INFO:teuthology.orchestra.run.smithi189.stderr:Sending period to new master zone 91786ef1-cdb4-4061-8a3e-2a28cc497285 2022-09-03T05:18:05.415 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:18:05.415 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:18:05.416 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 4, 2022-09-03T05:18:05.416 INFO:teuthology.orchestra.run.smithi189.stdout: "predecessor_uuid": "26ebceb6-9cf2-4657-999c-e205262f80e9", 2022-09-03T05:18:05.417 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": [], 2022-09-03T05:18:05.417 INFO:teuthology.orchestra.run.smithi189.stdout: "period_map": { 2022-09-03T05:18:05.417 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:18:05.418 INFO:teuthology.orchestra.run.smithi189.stdout: "zonegroups": [ 2022-09-03T05:18:05.418 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:05.419 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "test-zonegroup", 2022-09-03T05:18:05.419 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zonegroup", 2022-09-03T05:18:05.420 INFO:teuthology.orchestra.run.smithi189.stdout: "api_name": "", 2022-09-03T05:18:05.420 INFO:teuthology.orchestra.run.smithi189.stdout: "is_master": true, 2022-09-03T05:18:05.420 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:18:05.421 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:18:05.421 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:18:05.421 INFO:teuthology.orchestra.run.smithi189.stdout: "hostnames": [], 2022-09-03T05:18:05.422 INFO:teuthology.orchestra.run.smithi189.stdout: "hostnames_s3website": [], 2022-09-03T05:18:05.422 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:18:05.422 INFO:teuthology.orchestra.run.smithi189.stdout: "zones": [ 2022-09-03T05:18:05.423 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:05.423 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:18:05.424 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone3", 2022-09-03T05:18:05.426 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:18:05.427 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi181.front.sepia.ceph.com:8001" 2022-09-03T05:18:05.427 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:18:05.428 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:18:05.428 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:18:05.428 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:18:05.429 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:18:05.429 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:18:05.430 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": true, 2022-09-03T05:18:05.430 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:18:05.431 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:18:05.431 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:18:05.432 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:18:05.432 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:18:05.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:05.433 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:05.433 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:18:05.434 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone4", 2022-09-03T05:18:05.434 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:18:05.435 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi189.front.sepia.ceph.com:8001" 2022-09-03T05:18:05.435 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:18:05.436 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:18:05.436 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:18:05.437 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:18:05.437 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:18:05.438 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "pubsub", 2022-09-03T05:18:05.438 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": false, 2022-09-03T05:18:05.439 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [ 2022-09-03T05:18:05.439 INFO:teuthology.orchestra.run.smithi189.stdout: "test-zone1" 2022-09-03T05:18:05.440 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:18:05.440 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:18:05.441 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:18:05.441 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:18:05.442 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:18:05.442 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:05.443 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:05.443 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:18:05.444 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone1", 2022-09-03T05:18:05.444 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:18:05.445 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:18:05.445 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:18:05.445 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:18:05.446 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:18:05.446 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:18:05.447 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:18:05.447 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:18:05.448 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": true, 2022-09-03T05:18:05.448 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:18:05.449 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:18:05.449 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:18:05.450 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:18:05.450 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:18:05.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:05.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:05.451 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:18:05.452 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone2", 2022-09-03T05:18:05.452 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:18:05.453 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi189.front.sepia.ceph.com:8000" 2022-09-03T05:18:05.453 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:18:05.454 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:18:05.454 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:18:05.454 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:18:05.455 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:18:05.455 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:18:05.456 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": true, 2022-09-03T05:18:05.457 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:18:05.457 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:18:05.458 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:18:05.458 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:18:05.459 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:18:05.459 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:05.459 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:18:05.460 INFO:teuthology.orchestra.run.smithi189.stdout: "placement_targets": [ 2022-09-03T05:18:05.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:05.461 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "default-placement", 2022-09-03T05:18:05.462 INFO:teuthology.orchestra.run.smithi189.stdout: "tags": [], 2022-09-03T05:18:05.462 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_classes": [ 2022-09-03T05:18:05.465 INFO:teuthology.orchestra.run.smithi189.stdout: "STANDARD" 2022-09-03T05:18:05.466 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:18:05.466 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:05.467 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:18:05.467 INFO:teuthology.orchestra.run.smithi189.stdout: "default_placement": "default-placement", 2022-09-03T05:18:05.467 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:18:05.468 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_policy": { 2022-09-03T05:18:05.469 INFO:teuthology.orchestra.run.smithi189.stdout: "groups": [] 2022-09-03T05:18:05.469 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:05.469 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled_features": [ 2022-09-03T05:18:05.470 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:18:05.470 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:18:05.471 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:05.471 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:18:05.472 INFO:teuthology.orchestra.run.smithi189.stdout: "short_zone_ids": [ 2022-09-03T05:18:05.472 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:05.472 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:18:05.472 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 327838863 2022-09-03T05:18:05.472 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:05.473 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:05.473 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:18:05.473 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 1690651894 2022-09-03T05:18:05.473 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:05.474 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:05.474 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:18:05.474 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 1752562153 2022-09-03T05:18:05.474 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:05.475 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:05.475 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:18:05.475 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 2011734872 2022-09-03T05:18:05.475 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:05.475 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:18:05.476 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:05.476 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zonegroup": "test-zonegroup", 2022-09-03T05:18:05.476 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:18:05.476 INFO:teuthology.orchestra.run.smithi189.stdout: "period_config": { 2022-09-03T05:18:05.477 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_quota": { 2022-09-03T05:18:05.477 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false, 2022-09-03T05:18:05.477 INFO:teuthology.orchestra.run.smithi189.stdout: "check_on_raw": false, 2022-09-03T05:18:05.477 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size": -1, 2022-09-03T05:18:05.477 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size_kb": 0, 2022-09-03T05:18:05.478 INFO:teuthology.orchestra.run.smithi189.stdout: "max_objects": -1 2022-09-03T05:18:05.478 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:05.478 INFO:teuthology.orchestra.run.smithi189.stdout: "user_quota": { 2022-09-03T05:18:05.478 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false, 2022-09-03T05:18:05.479 INFO:teuthology.orchestra.run.smithi189.stdout: "check_on_raw": false, 2022-09-03T05:18:05.479 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size": -1, 2022-09-03T05:18:05.479 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size_kb": 0, 2022-09-03T05:18:05.479 INFO:teuthology.orchestra.run.smithi189.stdout: "max_objects": -1 2022-09-03T05:18:05.479 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:05.480 INFO:teuthology.orchestra.run.smithi189.stdout: "user_ratelimit": { 2022-09-03T05:18:05.480 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:18:05.480 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:18:05.480 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:18:05.481 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:18:05.481 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:18:05.481 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:05.481 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_ratelimit": { 2022-09-03T05:18:05.481 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:18:05.482 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:18:05.482 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:18:05.482 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:18:05.482 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:18:05.483 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:05.483 INFO:teuthology.orchestra.run.smithi189.stdout: "anonymous_ratelimit": { 2022-09-03T05:18:05.483 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:18:05.483 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:18:05.484 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:18:05.484 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:18:05.484 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:18:05.484 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:05.485 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:05.485 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:18:05.485 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_name": "test-realm", 2022-09-03T05:18:05.485 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:05.486 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:18:05.486 DEBUG:tasks.rgw.c2.client.1:waiting for process to exit 2022-09-03T05:18:05.487 INFO:teuthology.orchestra.run:waiting for 300 2022-09-03T05:18:05.488 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T05:18:05.490+0000 7f30ecbce700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term radosgw --rgw-frontends beast port=8001 -n client.1 --cluster c2 -k /etc/ceph/c2.client.1.keyring --log-file /var/log/ceph/rgw.c2.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.1.sock --foreground (PID: 110323) UID: 0 2022-09-03T05:18:05.488 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T05:18:05.490+0000 7f30f0e39600 -1 shutting down 2022-09-03T05:18:09.797 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.1 is failed for ~0s 2022-09-03T05:18:11.588 INFO:tasks.rgw.c2.client.1:Stopped 2022-09-03T05:18:11.589 INFO:tasks.rgw.c2.client.1:Restarting daemon 2022-09-03T05:18:11.589 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8001' -n client.1 --cluster c2 -k /etc/ceph/c2.client.1.keyring --log-file /var/log/ceph/rgw.c2.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.1.sock --foreground --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c2.client.1.stdout 2>&1 2022-09-03T05:18:11.593 INFO:tasks.rgw.c2.client.1:Started 2022-09-03T05:18:11.594 DEBUG:teuthology.orchestra.run.smithi189:> curl http://smithi189.front.sepia.ceph.com:8001 2022-09-03T05:18:11.622 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T05:18:11.623 INFO:teuthology.orchestra.run.smithi189.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T05:18:11.623 INFO:teuthology.orchestra.run.smithi189.stderr: Dload Upload Total Spent Left Speed 2022-09-03T05:18:11.623 INFO:teuthology.orchestra.run.smithi189.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi189.front.sepia.ceph.com port 8001: Connection refused 2022-09-03T05:18:12.624 DEBUG:teuthology.orchestra.run.smithi189:> curl http://smithi189.front.sepia.ceph.com:8001 2022-09-03T05:18:12.651 INFO:teuthology.orchestra.run.smithi189.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T05:18:12.651 INFO:teuthology.orchestra.run.smithi189.stderr: Dload Upload Total Spent Left Speed 2022-09-03T05:18:12.652 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T05:18:12.653 INFO:teuthology.orchestra.run.smithi189.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi189.front.sepia.ceph.com port 8001: Connection refused 2022-09-03T05:18:14.654 DEBUG:teuthology.orchestra.run.smithi189:> curl http://smithi189.front.sepia.ceph.com:8001 2022-09-03T05:18:14.693 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T05:18:14.694 INFO:teuthology.orchestra.run.smithi189.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T05:18:14.695 INFO:teuthology.orchestra.run.smithi189.stderr: Dload Upload Total Spent Left Speed 2022-09-03T05:18:14.695 INFO:teuthology.orchestra.run.smithi189.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi189.front.sepia.ceph.com port 8001: Connection refused 2022-09-03T05:18:15.503 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.1 has been restored 2022-09-03T05:18:18.697 DEBUG:teuthology.orchestra.run.smithi189:> curl http://smithi189.front.sepia.ceph.com:8001 2022-09-03T05:18:18.728 INFO:teuthology.orchestra.run.smithi189.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T05:18:18.728 INFO:teuthology.orchestra.run.smithi189.stderr: Dload Upload Total Spent Left Speed 2022-09-03T05:18:18.729 INFO:teuthology.orchestra.run.smithi189.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 134 100 134 0 0 130k 0 --:--:-- --:--:-- --:--:-- 130k 2022-09-03T05:18:18.730 INFO:teuthology.orchestra.run.smithi189.stdout:{"Code":"MethodNotAllowed","RequestId":"tx0000084bfa40ff05bf055-006312e39a-10f0-test-zone4","HostId":"10f0-test-zone4-test-zonegroup"} 2022-09-03T05:18:18.731 INFO:tasks.rgw_multisite:rgw multisite configuration completed 2022-09-03T05:18:18.731 INFO:teuthology.run_tasks:Running task rgw-multisite-tests... 2022-09-03T05:18:18.757 INFO:tasks.rgw_multisite_tests:importing tests from /home/teuthworker/src/github.com_ceph_ceph-c_4d487742eda56d4badbc9bdb33761243365bb37a/qa/../src/test/rgw/rgw_multi 2022-09-03T05:18:18.794 INFO:tasks.rgw_multisite_tests:creating test user.. 2022-09-03T05:18:18.795 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['user', 'create', '--uid', 'rgw-multisite-test-user', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--display-name', 'Multisite Test User', '--gen-access-key', '--gen-secret', '--caps', 'roles=*', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:18:18.796 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'user', 'create', '--uid', 'rgw-multisite-test-user', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--display-name', 'Multisite Test User', '--gen-access-key', '--gen-secret', '--caps', 'roles=*', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:18:18.796 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 user create --uid rgw-multisite-test-user --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --display-name 'Multisite Test User' --gen-access-key --gen-secret --caps 'roles=*' --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:18:18.864 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:18:18.864 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:18:18.910 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:18:18.914 INFO:teuthology.orchestra.run.smithi181.stdout: "user_id": "rgw-multisite-test-user", 2022-09-03T05:18:18.915 INFO:teuthology.orchestra.run.smithi181.stdout: "display_name": "Multisite Test User", 2022-09-03T05:18:18.915 INFO:teuthology.orchestra.run.smithi181.stdout: "email": "", 2022-09-03T05:18:18.916 INFO:teuthology.orchestra.run.smithi181.stdout: "suspended": 0, 2022-09-03T05:18:18.916 INFO:teuthology.orchestra.run.smithi181.stdout: "max_buckets": 1000, 2022-09-03T05:18:18.916 INFO:teuthology.orchestra.run.smithi181.stdout: "subusers": [], 2022-09-03T05:18:18.916 INFO:teuthology.orchestra.run.smithi181.stdout: "keys": [ 2022-09-03T05:18:18.917 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:18:18.917 INFO:teuthology.orchestra.run.smithi181.stdout: "user": "rgw-multisite-test-user", 2022-09-03T05:18:18.917 INFO:teuthology.orchestra.run.smithi181.stdout: "access_key": "SA4D9959RL1I1FMQ4Y2O", 2022-09-03T05:18:18.917 INFO:teuthology.orchestra.run.smithi181.stdout: "secret_key": "DSQn6tV1cxX9BMV8pWAA2kwbjnf7l8Kc2Lh6qTnB" 2022-09-03T05:18:18.918 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:18:18.918 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:18:18.918 INFO:teuthology.orchestra.run.smithi181.stdout: "swift_keys": [], 2022-09-03T05:18:18.919 INFO:teuthology.orchestra.run.smithi181.stdout: "caps": [ 2022-09-03T05:18:18.919 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:18:18.919 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "roles", 2022-09-03T05:18:18.920 INFO:teuthology.orchestra.run.smithi181.stdout: "perm": "*" 2022-09-03T05:18:18.920 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:18:18.920 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:18:18.921 INFO:teuthology.orchestra.run.smithi181.stdout: "op_mask": "read, write, delete", 2022-09-03T05:18:18.921 INFO:teuthology.orchestra.run.smithi181.stdout: "default_placement": "", 2022-09-03T05:18:18.921 INFO:teuthology.orchestra.run.smithi181.stdout: "default_storage_class": "", 2022-09-03T05:18:18.922 INFO:teuthology.orchestra.run.smithi181.stdout: "placement_tags": [], 2022-09-03T05:18:18.922 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_quota": { 2022-09-03T05:18:18.922 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false, 2022-09-03T05:18:18.922 INFO:teuthology.orchestra.run.smithi181.stdout: "check_on_raw": false, 2022-09-03T05:18:18.923 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size": -1, 2022-09-03T05:18:18.923 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size_kb": 0, 2022-09-03T05:18:18.923 INFO:teuthology.orchestra.run.smithi181.stdout: "max_objects": -1 2022-09-03T05:18:18.923 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:18:18.923 INFO:teuthology.orchestra.run.smithi181.stdout: "user_quota": { 2022-09-03T05:18:18.924 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false, 2022-09-03T05:18:18.924 INFO:teuthology.orchestra.run.smithi181.stdout: "check_on_raw": false, 2022-09-03T05:18:18.924 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size": -1, 2022-09-03T05:18:18.924 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size_kb": 0, 2022-09-03T05:18:18.925 INFO:teuthology.orchestra.run.smithi181.stdout: "max_objects": -1 2022-09-03T05:18:18.925 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:18:18.925 INFO:teuthology.orchestra.run.smithi181.stdout: "temp_url_keys": [], 2022-09-03T05:18:18.926 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "rgw", 2022-09-03T05:18:18.927 INFO:teuthology.orchestra.run.smithi181.stdout: "mfa_ids": [] 2022-09-03T05:18:18.927 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:18:18.928 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:18:18.930 INFO:rgw_multi.tests:meta checkpoint 2022-09-03T05:18:18.931 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:18:18.931 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:18:18.932 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:18:18.982 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:18:18.983 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:18:19.011 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=41ecd2a5-afc0-4eae-9349-7dfe39baa1cb 2022-09-03T05:18:19.022 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:18:19.023 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:18:19.023 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:18:19.023 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:18:18.906852Z" 2022-09-03T05:18:19.023 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:18:19.024 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:18:19.024 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:18:19.024 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:18:19.025 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:18:19.025 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:18:19.025 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:18:19.025 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:18:19.026 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:18:19.026 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:18:19.027 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:18:19.027 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:18:19.028 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:18:19.028 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:18:19.030 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:18:19.031 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:18:19.032 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:18:19.033 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:18:19.090 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:18:19.091 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:18:19.120 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:18:19.122+0000 7ff3238ec600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:18:19.140 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:18:19.143+0000 7ff3238ec600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:18:19.155 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:18:19.155 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:18:19.155 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:18:19.156 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:18:19.156 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:18:19.156 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:18:19.156 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:19.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:19.157 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:18:19.157 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:19.157 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:18:19.158 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:19.158 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:19.158 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:19.158 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:19.159 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:18:19.159 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:19.159 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:19.159 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:19.159 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:19.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:19.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:19.160 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:18:19.161 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:19.161 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:19.161 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:19.161 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:19.162 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:19.162 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:19.162 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:19.163 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:19.163 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:19.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:19.163 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:19.164 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:18:19.164 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:19.165 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:19.165 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:19.166 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:19.166 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:19.167 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:19.167 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:19.168 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:19.168 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:19.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:19.169 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:19.169 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:18:19.170 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:19.170 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:19.171 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:19.172 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:19.172 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:19.172 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:19.174 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:19.174 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:19.175 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:19.175 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:19.175 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:18:19.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:19.175 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:18:19.176 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:18:19.176 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:18:19.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:19.176 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:18:19Z" 2022-09-03T05:18:19.177 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:18:19.178 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662182298.906852_69.1 target= 2022-09-03T05:18:24.179 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:18:24.180 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:18:24.180 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:18:24.238 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:18:24.238 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:18:24.268 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:18:24.270+0000 7fa397995600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:18:24.269 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:18:24.272+0000 7fa397995600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:18:24.281 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:18:24.281 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:18:24.281 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:18:24.282 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:18:24.282 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:18:24.282 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:18:24.283 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:24.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:24.284 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:18:24.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:24.284 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:18:24.285 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:24.285 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:24.285 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:24.285 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:24.286 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:18:24.286 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:24.287 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:24.287 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:24.287 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:24.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:24.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:24.289 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:18:24.289 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:24.290 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:24.290 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:24.290 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:24.291 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:24.291 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:24.291 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:24.291 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:24.292 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:24.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:24.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:24.293 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:18:24.293 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:24.293 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:24.294 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:24.294 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:24.294 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:24.295 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:24.295 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:24.295 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:24.295 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:24.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:24.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:24.296 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:18:24.296 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:24.297 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:24.298 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:24.298 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:24.298 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:24.299 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:24.299 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:24.300 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:24.300 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:24.301 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:24.301 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:18:24.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:24.301 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:18:24.302 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:18:24.302 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:18:24.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:24.302 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:18:24Z" 2022-09-03T05:18:24.303 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:18:24.304 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662182298.906852_69.1 target= 2022-09-03T05:18:29.306 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:18:29.306 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:18:29.307 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:18:29.365 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:18:29.366 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:18:29.405 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:18:29.407+0000 7f8c1a546600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:18:29.405 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:18:29.408+0000 7f8c1a546600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:18:29.421 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:18:29.421 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:18:29.421 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:18:29.422 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:18:29.422 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:18:29.422 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:18:29.422 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:29.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:29.423 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:18:29.423 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:29.423 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:18:29.424 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:29.424 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:29.424 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:29.424 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:29.425 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:18:29.425 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:29.425 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:29.425 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:29.426 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:29.426 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:29.426 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:29.426 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:18:29.427 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:29.428 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:29.428 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:29.428 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:29.429 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:29.430 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:29.430 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:29.431 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:29.431 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:29.431 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:29.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:29.433 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:18:29.433 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:29.433 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:29.434 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:29.434 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:29.435 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:29.435 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:29.436 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:29.436 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:29.436 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:29.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:29.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:29.438 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:18:29.438 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:29.439 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:29.439 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:29.440 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:29.440 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:29.441 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:29.441 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:29.442 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:29.442 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:29.442 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:29.443 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:18:29.443 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:29.443 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:18:29.444 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:18:29.444 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:18:29.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:29.445 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:18:29Z" 2022-09-03T05:18:29.445 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:18:29.447 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662182298.906852_69.1 target= 2022-09-03T05:18:34.448 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:18:34.448 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:18:34.449 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:18:34.510 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:18:34.511 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:18:34.539 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:18:34.541+0000 7fc2c9d0b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:18:34.540 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:18:34.542+0000 7fc2c9d0b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:18:34.555 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:18:34.556 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:18:34.557 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:18:34.557 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:18:34.558 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:18:34.558 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:18:34.559 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:34.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:34.560 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:18:34.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:34.561 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:18:34.561 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:34.562 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:34.562 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:34.563 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:34.563 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:18:34.564 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:34.564 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:34.564 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:34.565 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:34.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:34.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:34.566 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:18:34.566 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:34.566 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:34.566 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:34.567 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:34.567 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:34.567 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:34.567 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:34.568 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:34.568 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:34.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:34.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:34.569 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:18:34.569 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:34.569 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:34.569 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:34.569 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:34.570 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:34.570 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:34.570 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:34.571 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:34.571 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:34.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:34.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:34.572 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:18:34.572 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:34.573 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:34.573 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:34.574 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:34.577 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:34.578 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:34.578 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:34.579 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:34.579 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:34.580 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:34.580 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:18:34.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:34.581 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:18:34.581 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:18:34.581 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:18:34.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:34.582 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:18:34Z" 2022-09-03T05:18:34.583 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:18:34.583 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662182298.906852_69.1 target= 2022-09-03T05:18:39.585 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:18:39.586 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:18:39.586 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:18:39.642 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:18:39.643 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:18:39.675 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:18:39.678+0000 7f0a1dbf8600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:18:39.676 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:18:39.679+0000 7f0a1dbf8600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:18:39.688 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:18:39.689 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:18:39.689 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:18:39.689 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:18:39.690 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:18:39.690 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:18:39.691 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:39.691 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:39.692 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:18:39.692 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:39.693 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:18:39.693 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:39.693 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:39.694 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:39.694 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:39.694 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:18:39.695 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:39.695 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:39.695 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:39.696 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:39.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:39.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:39.697 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:18:39.697 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:39.697 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:39.698 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:39.698 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:39.698 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:39.698 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:39.699 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:39.699 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:39.699 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:39.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:39.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:39.700 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:18:39.700 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:39.700 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:39.702 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:39.702 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:39.703 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:39.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:39.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:39.704 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:18:39.704 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:39.705 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:39.705 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:39.706 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:39.706 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:39.707 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:39.707 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:39.707 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:39.708 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:39.708 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:39.708 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:18:39.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:39.709 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:18:39.710 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:18:39.710 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:18:39.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:39.710 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:18:39Z" 2022-09-03T05:18:39.711 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:18:39.712 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662182298.906852_69.1 target= 2022-09-03T05:18:44.713 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:18:44.714 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:18:44.715 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:18:44.771 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:18:44.771 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:18:44.801 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:18:44.803+0000 7f6bfdb96600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:18:44.801 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:18:44.804+0000 7f6bfdb96600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:18:44.815 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:18:44.816 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:18:44.816 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:18:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:18:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:18:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:18:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:18:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:18:44.820 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:44.820 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:44.820 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:44.820 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:18:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:18:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:18:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:44.826 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:44.826 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:44.826 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:44.826 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:44.827 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:44.827 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:44.827 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:44.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:44.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:44.828 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:18:44.828 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:44.828 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:44.828 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:44.829 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:44.829 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:44.829 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:44.830 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:44.830 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:44.830 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:44.830 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:44.831 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:18:44.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:44.831 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:18:44.831 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:18:44.832 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:18:44.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:44.832 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:18:44Z" 2022-09-03T05:18:44.832 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:18:44.834 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662182298.906852_69.1 target= 2022-09-03T05:18:49.835 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:18:49.835 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:18:49.836 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:18:49.894 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:18:49.895 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:18:49.930 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:18:49.931+0000 7ff51d2fd600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:18:49.930 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:18:49.932+0000 7ff51d2fd600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:18:49.947 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:18:49.948 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:18:49.948 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:18:49.949 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:18:49.950 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:18:49.950 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:18:49.951 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:49.951 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:49.951 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:18:49.952 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:49.952 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:18:49.953 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:49.953 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:49.954 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:49.954 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:49.955 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:18:49.955 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:49.956 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:49.956 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:49.957 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:49.957 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:49.957 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:49.958 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:18:49.958 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:49.959 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:49.959 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:49.959 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:49.960 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:49.960 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:49.961 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:49.961 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:49.961 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:49.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:49.962 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:49.962 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:18:49.962 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:49.962 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:49.962 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:49.963 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:49.963 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:49.963 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:49.964 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:49.964 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:49.965 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:49.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:49.965 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:49.966 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:18:49.966 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:49.967 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:49.967 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:49.968 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:49.968 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:49.969 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:49.969 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:49.970 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:49.970 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:49.971 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:49.971 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:18:49.972 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:49.972 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:18:49.972 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:18:49.973 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:18:49.973 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:49.974 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:18:49Z" 2022-09-03T05:18:49.974 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:18:49.975 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662182298.906852_69.1 target= 2022-09-03T05:18:54.976 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:18:54.977 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:18:54.978 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:18:55.046 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:18:55.047 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:18:55.069 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:18:55.071+0000 7fba4f4ca600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:18:55.070 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:18:55.073+0000 7fba4f4ca600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:18:55.082 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:18:55.082 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:18:55.082 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:18:55.083 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:18:55.083 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:18:55.083 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:18:55.084 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:55.084 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:55.084 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:18:55.084 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:55.084 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:18:55.085 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:55.085 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:55.085 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:55.085 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:55.086 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:18:55.086 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:55.086 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:55.086 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:55.087 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:55.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:55.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:55.087 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:18:55.087 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:55.088 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:55.088 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:55.088 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:55.088 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:55.089 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:55.089 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:55.089 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:55.089 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:55.089 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:55.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:55.090 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:18:55.090 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:55.090 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:55.091 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:55.091 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:55.091 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:55.091 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:55.091 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:55.092 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:55.092 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:55.092 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:55.092 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:18:55.093 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:18:55.093 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:18:55.093 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:18:55.093 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:18:55.093 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:18:55.094 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:18:55.094 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:18:55.094 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:18:55.094 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:18:55.095 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:55.095 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:18:55.095 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:18:55.095 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:55.096 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:18:55.096 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:18:55.096 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:18:55.096 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:18:55.096 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:18:55Z" 2022-09-03T05:18:55.097 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:18:55.098 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662182298.906852_69.1 target= 2022-09-03T05:19:00.100 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:00.100 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:00.101 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:19:00.160 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:19:00.161 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:19:00.190 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:00.192+0000 7f7b7c5c4600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:00.191 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:00.193+0000 7f7b7c5c4600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:00.202 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:19:00.202 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:19:00.203 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:19:00.203 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:19:00.203 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:19:00.204 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:19:00.204 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:00.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:00.204 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:19:00.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:00.205 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:19:00.205 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:00.205 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:00.205 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:00.206 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:00.206 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:19:00.206 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:00.206 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:00.207 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:00.207 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:00.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:00.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:00.207 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:19:00.208 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:00.208 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:00.208 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:00.208 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:00.209 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:00.209 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:00.209 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:00.209 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:00.209 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:00.210 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:00.210 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:00.210 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:19:00.210 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:00.211 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:00.211 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:00.211 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:00.212 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:00.212 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:00.212 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:00.212 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:00.213 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:00.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:00.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:00.213 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:19:00.214 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:00.214 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:00.214 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:00.214 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:00.215 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:00.215 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:00.215 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:00.215 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:00.215 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:00.216 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:00.216 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:19:00.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:00.216 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:19:00.217 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:19:00.217 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:19:00.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:00.217 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:19:00Z" 2022-09-03T05:19:00.217 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:19:00.219 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662182298.906852_69.1 target= 2022-09-03T05:19:05.220 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:05.221 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:05.221 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:19:05.281 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:19:05.281 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:19:05.313 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:05.315+0000 7f91134ed600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:05.314 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:05.317+0000 7f91134ed600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:05.327 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:19:05.328 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:19:05.328 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:19:05.328 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:19:05.328 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:19:05.329 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:19:05.329 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:05.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:05.330 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:19:05.330 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:05.330 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:19:05.330 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:05.331 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:05.331 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:05.331 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:05.331 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:19:05.331 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:05.332 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:05.332 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:05.332 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:05.332 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:05.333 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:05.333 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:19:05.333 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:05.333 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:05.334 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:05.334 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:05.334 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:05.334 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:05.335 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:05.335 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:05.335 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:05.335 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:05.336 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:05.336 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:19:05.336 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:05.336 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:05.336 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:05.337 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:05.337 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:05.337 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:05.337 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:05.338 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:05.338 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:05.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:05.338 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:05.339 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:19:05.339 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:05.339 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:05.339 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:05.339 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:05.340 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:05.340 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:05.340 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:05.340 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:05.341 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:05.341 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:05.341 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:19:05.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:05.342 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:19:05.342 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:19:05.342 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:19:05.342 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:05.343 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:19:05Z" 2022-09-03T05:19:05.343 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:19:05.344 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662182298.906852_69.1 target= 2022-09-03T05:19:10.345 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:10.346 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:10.346 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:19:10.403 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:19:10.403 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:19:10.429 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:10.432+0000 7fda8ba91600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:10.430 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:10.433+0000 7fda8ba91600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:10.442 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:19:10.442 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:19:10.442 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:19:10.443 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:19:10.443 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:19:10.443 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:19:10.443 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:10.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:10.444 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:19:10.445 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:10.445 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:19:10.445 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:10.445 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:10.445 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:10.446 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:10.446 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:19:10.446 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:10.446 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:10.447 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:10.447 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:10.447 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:10.447 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:10.448 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:19:10.448 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:10.448 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:10.448 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:10.449 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:10.449 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:10.449 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:10.449 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:10.449 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:10.450 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:10.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:10.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:10.451 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:19:10.451 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:10.452 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:10.452 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:10.452 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:10.453 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:10.453 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:10.453 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:10.453 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:10.454 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:10.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:10.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:10.454 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:19:10.455 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:10.455 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:10.455 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:10.456 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:10.456 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:10.456 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:10.457 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:10.457 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:10.457 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:10.458 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:10.458 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:19:10.458 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:10.459 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:19:10.459 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:19:10.459 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:19:10.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:10.460 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:19:10Z" 2022-09-03T05:19:10.460 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:19:10.462 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662182298.906852_69.1 target= 2022-09-03T05:19:15.463 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:15.464 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:15.464 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:19:15.522 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:19:15.522 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:19:15.557 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:15.559+0000 7fbc285d2600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:15.557 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:15.560+0000 7fbc285d2600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:15.570 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:19:15.570 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:19:15.570 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:19:15.571 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:19:15.571 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:19:15.571 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:19:15.571 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:15.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:15.572 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:19:15.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:15.572 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:19:15.573 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:15.573 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:15.573 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:15.573 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:15.573 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:19:15.574 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:15.574 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:15.574 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:15.574 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:15.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:15.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:15.575 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:19:15.575 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:15.576 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:15.576 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:15.576 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:15.576 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:15.577 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:15.577 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:15.577 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:15.577 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:15.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:15.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:15.578 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:19:15.578 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:15.578 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:15.579 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:15.579 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:15.579 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:15.579 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:15.580 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:15.580 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:15.580 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:15.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:15.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:15.581 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:19:15.581 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:15.581 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:15.581 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:15.582 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:15.582 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:15.582 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:15.582 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:15.583 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:15.583 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:15.583 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:15.583 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:19:15.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:15.584 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:19:15.584 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:19:15.584 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:19:15.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:15.585 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:19:15Z" 2022-09-03T05:19:15.585 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:19:15.586 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662182298.906852_69.1 target= 2022-09-03T05:19:20.588 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:20.588 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:20.589 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:19:20.647 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:19:20.647 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:19:20.677 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:20.680+0000 7fcf226b1600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:20.678 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:20.681+0000 7fcf226b1600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:20.690 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:19:20.690 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:19:20.690 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:19:20.690 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:19:20.691 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:19:20.691 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:19:20.691 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:20.692 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:20.692 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:19:20.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:20.693 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:19:20.694 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:20.694 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:20.695 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:20.695 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:20.695 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:19:20.695 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:20.696 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:20.696 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:20.697 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:20.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:20.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:20.697 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:19:20.698 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:20.698 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:20.698 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:20.699 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:20.699 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:20.699 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:20.699 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:20.699 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:20.700 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:20.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:20.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:20.700 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:19:20.701 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:20.701 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:20.701 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:20.701 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:20.701 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:20.702 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:20.702 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:20.702 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:20.702 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:20.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:20.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:20.703 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:19:20.704 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:20.704 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:20.704 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:20.705 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:20.705 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:20.705 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:20.705 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:20.705 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:20.706 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:20.706 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:20.706 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:19:20.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:20.707 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:19:20.707 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:19:20.707 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:19:20.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:20.707 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:19:20Z" 2022-09-03T05:19:20.708 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:19:20.709 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662182298.906852_69.1 target= 2022-09-03T05:19:25.710 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:25.711 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:25.711 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:19:25.770 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:19:25.771 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:19:25.798 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:25.801+0000 7f239e7fe600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:25.799 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:25.802+0000 7f239e7fe600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:25.815 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:19:25.815 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:19:25.815 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:19:25.816 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:19:25.816 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:19:25.816 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:19:25.816 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:25.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:25.817 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:19:25.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:25.817 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:19:25.818 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:25.818 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:25.818 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:25.818 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:25.819 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:19:25.819 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:25.819 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:25.819 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:25.820 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:25.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:25.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:25.821 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:19:25.821 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:25.821 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:25.822 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:25.822 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:25.822 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:25.823 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:25.823 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:25.824 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:25.824 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:25.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:25.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:25.825 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:19:25.825 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:25.826 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:25.826 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:25.826 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:25.826 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:25.826 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:25.827 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:25.827 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:25.827 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:25.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:25.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:25.828 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:19:25.828 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:25.828 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:25.828 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:25.829 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:25.829 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:25.829 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:25.829 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:25.830 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:25.830 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:25.830 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:25.831 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:19:25.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:25.831 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:19:25.831 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:19:25.832 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:19:25.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:25.832 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:19:25Z" 2022-09-03T05:19:25.832 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:19:25.833 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662182298.906852_69.1 target= 2022-09-03T05:19:30.835 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:30.835 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:30.836 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:19:30.892 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:19:30.893 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:19:30.924 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:30.926+0000 7f734aeb0600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:30.924 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:30.927+0000 7f734aeb0600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:30.940 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:19:30.940 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:19:30.941 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:19:30.941 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:19:30.942 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:19:30.942 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:19:30.943 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:30.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:30.944 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:19:30.944 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:30.944 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:19:30.945 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:30.945 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:30.946 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:30.946 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:30.947 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:19:30.947 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:30.947 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:30.948 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:30.948 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:30.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:30.949 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:30.949 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:19:30.949 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:30.950 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:30.950 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:30.950 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:30.951 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:30.951 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:30.951 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:30.951 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:30.952 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:30.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:30.952 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:30.952 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:19:30.953 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:30.953 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:30.953 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:30.953 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:30.954 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:30.954 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:30.954 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:30.954 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:30.955 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:30.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:30.955 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:30.956 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:19:30.956 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:30.956 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:30.956 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:30.957 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:30.957 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:30.957 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:30.957 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:30.958 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:30.958 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:30.958 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:30.958 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:19:30.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:30.959 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:19:30.959 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:19:30.959 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:19:30.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:30.960 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:19:30Z" 2022-09-03T05:19:30.960 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:19:30.961 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662182298.906852_69.1 target= 2022-09-03T05:19:35.963 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:35.963 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:35.964 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:19:36.022 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:19:36.023 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:19:36.050 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:36.052+0000 7f8bece90600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:36.051 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:36.053+0000 7f8bece90600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:36.063 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:19:36.063 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:19:36.063 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:19:36.064 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:19:36.064 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:19:36.064 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:19:36.064 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:36.065 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:36.065 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:19:36.066 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:36.066 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:19:36.066 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:36.067 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:36.067 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:36.067 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:36.068 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:19:36.068 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:36.069 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:36.069 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:36.069 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:36.070 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:36.071 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:36.071 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:19:36.072 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:36.072 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:36.073 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:36.073 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:36.074 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:36.074 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:36.074 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:36.075 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:36.075 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:36.075 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:36.076 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:36.076 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:19:36.077 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:36.077 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:36.077 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:36.078 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:36.078 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:36.078 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:36.079 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:36.079 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:36.079 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:36.080 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:36.080 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:36.080 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:19:36.081 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:36.081 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:36.081 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:36.081 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:36.082 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:36.082 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:36.082 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:36.082 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:36.083 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:36.083 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:36.083 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:19:36.083 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:36.084 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:19:36.084 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:19:36.084 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:19:36.084 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:36.085 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:19:36Z" 2022-09-03T05:19:36.085 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:19:36.087 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662182298.906852_69.1 target= 2022-09-03T05:19:41.089 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:41.090 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:41.091 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:19:41.149 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:19:41.150 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:19:41.183 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:41.185+0000 7fecb90a3600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:41.184 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:41.186+0000 7fecb90a3600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:41.198 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:19:41.198 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:19:41.199 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:19:41.199 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:19:41.199 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:19:41.199 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:19:41.200 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:41.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:41.200 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:19:41.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:41.201 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:19:41.201 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:41.201 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:41.201 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:41.202 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:41.202 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:19:41.202 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:41.202 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:41.203 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:41.203 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:41.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:41.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:41.204 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:19:41.204 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:41.204 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:41.204 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:41.205 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:41.205 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:41.205 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:41.205 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:41.206 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:41.206 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:41.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:41.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:41.207 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:19:41.207 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:41.207 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:41.207 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:41.208 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:41.208 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:41.208 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:41.208 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:41.209 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:41.209 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:41.209 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:41.209 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:41.209 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:19:41.210 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:41.210 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:41.210 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:41.210 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:41.211 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:41.211 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:41.211 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:41.212 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:41.212 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:41.212 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:41.212 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:19:41.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:41.214 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:19:41.214 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:19:41.215 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:19:41.215 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:41.215 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:19:41Z" 2022-09-03T05:19:41.216 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:19:41.217 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662182298.906852_69.1 target= 2022-09-03T05:19:46.218 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:46.219 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:46.220 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:19:46.276 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:19:46.276 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:19:46.304 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:46.307+0000 7f7e23791600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:46.305 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:46.308+0000 7f7e23791600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:46.319 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:19:46.319 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:19:46.320 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:19:46.320 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:19:46.321 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:19:46.321 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:19:46.322 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:46.322 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:46.323 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:19:46.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:46.326 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:19:46.326 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:46.326 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:46.327 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:46.327 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:46.327 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:19:46.327 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:46.328 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:46.328 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:46.328 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:46.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:46.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:46.329 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:19:46.329 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:46.329 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:46.330 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:46.330 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:46.330 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:46.330 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:46.331 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:46.331 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:46.331 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:46.331 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:46.332 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:46.332 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:19:46.332 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:46.332 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:46.332 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:46.333 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:46.333 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:46.333 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:46.333 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:46.334 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:46.334 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:46.334 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:46.334 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:46.335 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:19:46.335 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:46.335 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:46.335 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:46.336 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:46.336 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:46.336 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:46.336 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:46.337 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:46.337 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:46.337 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:46.337 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:19:46.337 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:46.338 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:19:46.338 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:19:46.338 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:19:46.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:46.339 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:19:46Z" 2022-09-03T05:19:46.339 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:19:46.339 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662182298.906852_69.1 target= 2022-09-03T05:19:51.341 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:51.342 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:51.342 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:19:51.398 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:19:51.398 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:19:51.426 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:51.428+0000 7f61c5027600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:51.426 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:51.429+0000 7f61c5027600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:51.439 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:19:51.440 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:19:51.440 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:19:51.440 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:19:51.441 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:19:51.441 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:19:51.441 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:51.441 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:51.442 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:19:51.442 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:51.442 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:19:51.442 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:51.443 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:51.443 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:51.443 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:51.443 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:19:51.443 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:51.444 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:51.444 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:51.444 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:51.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:51.445 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:51.445 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:19:51.445 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:51.445 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:51.445 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:51.446 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:51.446 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:51.446 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:51.447 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:51.447 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:51.447 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:51.447 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:51.448 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:51.448 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:19:51.448 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:51.448 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:51.448 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:51.449 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:51.449 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:51.449 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:51.449 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:51.450 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:51.450 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:51.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:51.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:51.451 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:19:51.451 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:51.451 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:51.451 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:51.451 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:51.452 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:51.452 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:51.452 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:51.452 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:51.453 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:51.453 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:51.453 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:19:51.453 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:51.453 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:19:51.454 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:19:51.454 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:19:51.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:51.454 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:19:51Z" 2022-09-03T05:19:51.455 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:19:51.455 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662182298.906852_69.1 target= 2022-09-03T05:19:56.456 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:56.457 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:19:56.458 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:19:56.515 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:19:56.516 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:19:56.548 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:56.550+0000 7efd9efeb600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:56.549 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:19:56.552+0000 7efd9efeb600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:19:56.566 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:19:56.567 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:19:56.567 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:19:56.567 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:19:56.568 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:19:56.568 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:19:56.568 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:56.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:56.569 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:19:56.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:56.569 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:19:56.569 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:56.569 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:56.570 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:56.570 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:56.570 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:19:56.570 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:56.571 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:56.571 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:56.571 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:56.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:56.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:56.572 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:19:56.573 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:56.573 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:56.573 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:56.573 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:56.574 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:56.574 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:56.574 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:56.574 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:56.575 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:56.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:56.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:56.575 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:19:56.575 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:56.576 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:56.576 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:56.576 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:56.576 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:56.577 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:56.577 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:56.577 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:56.577 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:56.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:56.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:19:56.578 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:19:56.578 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:19:56.578 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:19:56.579 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:19:56.579 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:19:56.579 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:19:56.579 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:19:56.580 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:19:56.580 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:19:56.580 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:56.580 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:19:56.581 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:19:56.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:56.581 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:19:56.581 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:19:56.581 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:19:56.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:19:56.582 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:19:56Z" 2022-09-03T05:19:56.582 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:19:56.583 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662182298.906852_69.1 target= 2022-09-03T05:20:01.584 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:01.585 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:01.585 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:01.644 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:01.644 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:01.678 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:01.679+0000 7f4912d55600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:01.678 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:01.680+0000 7f4912d55600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:01.691 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:20:01.692 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:20:01.692 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:20:01.692 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:20:01.693 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:20:01.693 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:01.694 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:01.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:01.695 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:20:01.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:01.695 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:20:01.696 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:01.696 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:01.697 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:01.697 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:01.698 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:20:01.698 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:01.698 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:01.698 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:01.699 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:01.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:01.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:01.699 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:20:01.700 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:01.700 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:01.700 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:01.700 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:01.700 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:01.701 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:01.701 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:01.701 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:01.701 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:01.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:01.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:01.702 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:20:01.702 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:01.703 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:01.703 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:01.703 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:01.703 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:01.704 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:01.704 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:01.704 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:01.705 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:01.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:01.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:01.705 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:20:01.706 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:01.706 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:01.706 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:01.706 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:01.707 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:01.707 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:01.707 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:01.707 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:01.708 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:01.708 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:01.708 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:20:01.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:01.709 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:20:01.709 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:20:01.709 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:20:01.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:01.709 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:20:01Z" 2022-09-03T05:20:01.710 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:20:01.711 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662182298.906852_69.1 target= 2022-09-03T05:20:06.712 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:06.713 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:06.714 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:06.770 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:06.771 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:06.799 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:06.802+0000 7f7e6c772600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:06.800 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:06.803+0000 7f7e6c772600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:06.815 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:20:06.816 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:20:06.816 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:20:06.817 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:20:06.817 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:20:06.818 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:06.818 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:06.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:06.819 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:20:06.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:06.819 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:20:06.820 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:06.820 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:06.821 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:06.821 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:06.821 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:20:06.822 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:06.822 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:06.822 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:06.822 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:06.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:06.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:06.823 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:20:06.823 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:06.824 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:06.824 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:06.824 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:06.825 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:06.825 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:06.825 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:06.825 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:06.826 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:06.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:06.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:06.826 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:20:06.827 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:06.827 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:06.827 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:06.827 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:06.828 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:06.828 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:06.828 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:06.828 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:06.828 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:06.829 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:06.829 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:06.829 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:20:06.829 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:06.830 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:06.830 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:06.831 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:06.831 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:06.831 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:06.831 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:06.832 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:06.832 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:06.832 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:06.833 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:20:06.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:06.833 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:20:06.833 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:20:06.834 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:20:06.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:06.834 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:20:06Z" 2022-09-03T05:20:06.834 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:20:06.837 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662182298.906852_69.1 target= 2022-09-03T05:20:11.838 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:11.839 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:11.840 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:11.897 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:11.897 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:11.925 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:11.927+0000 7f3c4c4c8600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:11.925 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:11.928+0000 7f3c4c4c8600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:11.937 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:20:11.938 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:20:11.938 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:20:11.939 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:20:11.939 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:20:11.940 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:11.940 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:11.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:11.941 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:20:11.941 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:11.942 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:20:11.942 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:11.943 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:11.943 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:11.943 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:11.944 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:20:11.944 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:11.944 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:11.944 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:11.944 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:11.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:11.945 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:11.945 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:20:11.945 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:11.946 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:11.946 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:11.946 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:11.946 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:11.947 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:11.947 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:11.947 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:11.948 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:11.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:11.950 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:11.951 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:20:11.951 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:11.952 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:11.952 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:11.953 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:11.953 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:11.953 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:11.953 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:11.954 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:11.954 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:11.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:11.954 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:11.955 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:20:11.955 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:11.955 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:11.956 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:11.956 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:11.956 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:11.957 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:11.957 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:11.957 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:11.957 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:11.958 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:11.958 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:20:11.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:11.959 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:20:11.959 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:20:11.959 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:20:11.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:11.960 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:20:11Z" 2022-09-03T05:20:11.961 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:20:11.963 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:20:11.966 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:20:11.967 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:11.968 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:11.969 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:12.028 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:12.028 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:12.061 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:20:12.062+0000 7f4504c0d600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:12.062 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:20:12.064+0000 7f4504c0d600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:12.089 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:20:12.089 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:20:12.089 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:20:12.089 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:20:12.090 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:20:12.090 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:12.090 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:12.090 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:12.091 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:20:12.091 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:12.091 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:20:12.091 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:12.092 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:12.092 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:12.092 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:12.092 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:20:12.092 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:12.093 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:12.093 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:12.093 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:12.093 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:12.094 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:12.094 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:20:12.094 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:12.094 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:12.095 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:12.095 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:12.095 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:12.095 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:12.095 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:20:12.096 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:12.096 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:12.096 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:12.096 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:12.097 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:20:12.097 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:12.097 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:12.097 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:12.097 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:12.098 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:12.098 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:12.098 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:20:12.098 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:12.099 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:12.099 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:12.099 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:12.099 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:20:12.099 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:12.100 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:12.100 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:12.100 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:12.100 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:12.101 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:12.101 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:20:12.101 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:12.101 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:12.101 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:12.102 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:20:12.102 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:12.102 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:20:12.102 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:20:12.103 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:20:12.103 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:12.103 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:20:12Z" 2022-09-03T05:20:12.103 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:20:12.105 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:20:12.105 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:20:12.105 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:12.106 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:12.106 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:12.158 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:12.159 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:12.187 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:12.190+0000 7f648b187600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:12.188 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:12.191+0000 7f648b187600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:12.199 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:20:12.199 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:20:12.199 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:20:12.200 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:20:12.200 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:20:12.200 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:12.200 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:12.201 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:12.201 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:20:12.201 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:12.201 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:20:12.202 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:12.202 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:12.202 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:12.202 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:12.203 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:20:12.204 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:12.205 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:12.205 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:12.205 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:12.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:12.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:12.206 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:20:12.207 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:12.207 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:12.208 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:12.208 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:12.208 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:12.208 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:12.209 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:12.209 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:12.209 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:12.209 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:12.210 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:12.210 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:20:12.210 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:12.210 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:12.210 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:12.211 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:12.211 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:12.211 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:12.211 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:12.212 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:12.212 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:12.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:12.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:12.213 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:20:12.213 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:12.213 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:12.213 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:12.214 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:12.215 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:12.215 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:12.215 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:12.215 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:12.215 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:12.216 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:12.216 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:20:12.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:12.216 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:20:12.217 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:20:12.217 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:20:12.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:12.217 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:20:12Z" 2022-09-03T05:20:12.217 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:20:12.220 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:20:12.220 INFO:rgw_multi.tests:meta checkpoint 2022-09-03T05:20:12.220 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:12.221 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:12.221 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:12.273 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:12.273 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:12.327 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=41ecd2a5-afc0-4eae-9349-7dfe39baa1cb 2022-09-03T05:20:12.345 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:20:12.345 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:12.346 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:12.346 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:18:18.906852Z" 2022-09-03T05:20:12.346 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:12.347 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:12.347 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:12.348 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:20:12.348 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:12.348 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:12.349 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:12.349 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:20:12.349 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:12.350 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:12.350 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:12.350 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:20:12.350 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:12.351 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:20:12.352 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:20:12.352 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:12.353 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:12.353 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:12.402 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:12.402 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:12.427 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:12.430+0000 7f70bb906600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:12.428 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:12.431+0000 7f70bb906600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:12.439 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:20:12.439 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:20:12.439 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:20:12.439 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:20:12.440 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:20:12.440 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:12.440 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:12.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:12.441 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:20:12.441 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:12.441 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:20:12.441 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:12.442 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:12.442 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:12.442 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:12.443 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:20:12.443 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:12.443 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:12.443 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:12.443 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:12.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:12.444 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:12.444 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:20:12.444 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:12.445 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:12.445 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:12.445 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:12.445 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:12.446 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:12.446 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:12.446 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:12.446 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:12.447 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:12.447 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:12.447 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:20:12.447 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:12.448 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:12.448 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:12.448 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:12.448 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:12.449 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:12.449 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:12.449 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:12.449 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:12.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:12.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:12.450 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:20:12.451 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:12.451 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:12.452 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:12.452 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:12.452 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:12.452 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:12.453 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:12.453 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:12.453 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:12.453 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:12.453 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:20:12.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:12.454 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:20:12.455 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:20:12.455 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:20:12.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:12.455 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:20:12Z" 2022-09-03T05:20:12.455 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:20:12.457 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:20:12.457 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:20:12.458 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:12.458 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:12.459 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:12.508 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:12.509 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:12.536 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:20:12.537+0000 7f4c6f80e600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:12.536 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:20:12.538+0000 7f4c6f80e600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:12.548 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:20:12.548 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:20:12.549 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:20:12.549 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:20:12.549 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:20:12.550 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:12.550 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:12.551 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:12.551 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:20:12.552 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:12.552 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:20:12.552 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:12.552 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:12.553 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:12.553 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:12.553 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:20:12.554 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:12.554 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:12.554 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:12.554 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:12.555 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:12.555 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:12.555 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:20:12.555 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:12.556 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:12.556 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:12.556 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:12.556 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:12.556 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:12.557 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:20:12.557 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:12.557 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:12.557 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:12.558 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:12.558 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:20:12.558 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:12.558 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:12.559 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:12.559 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:12.559 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:12.560 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:12.560 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:20:12.560 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:12.560 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:12.561 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:12.561 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:12.561 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:20:12.561 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:12.561 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:12.562 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:12.562 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:12.562 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:12.562 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:12.563 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:20:12.563 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:12.563 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:12.563 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:12.564 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:20:12.564 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:12.564 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:20:12.564 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:20:12.565 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:20:12.565 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:12.565 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:20:12Z" 2022-09-03T05:20:12.565 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:20:12.567 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:20:12.567 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:20:12.568 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:12.568 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:12.569 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:12.618 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:12.619 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:12.643 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:12.646+0000 7fa8bd4c2600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:12.644 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:12.647+0000 7fa8bd4c2600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:12.656 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:20:12.657 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:20:12.657 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:20:12.657 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:20:12.657 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:20:12.658 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:12.658 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:12.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:12.659 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:20:12.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:12.659 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:20:12.659 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:12.660 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:12.660 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:12.660 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:12.660 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:20:12.661 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:12.661 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:12.661 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:12.661 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:12.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:12.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:12.662 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:20:12.662 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:12.663 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:12.663 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:12.663 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:12.663 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:12.664 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:12.664 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:12.664 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:12.664 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:12.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:12.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:12.665 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:20:12.665 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:12.665 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:12.666 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:12.666 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:12.666 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:12.667 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:12.667 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:12.667 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:12.667 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:12.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:12.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:12.668 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:20:12.668 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:12.668 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:12.669 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:12.669 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:12.669 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:12.669 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:12.670 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:12.670 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:12.670 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:12.670 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:12.671 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:20:12.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:12.671 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:20:12.671 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:20:12.672 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:20:12.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:12.672 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:20:12Z" 2022-09-03T05:20:12.672 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:20:12.674 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:20:12.675 INFO:tasks.rgw_multisite_tests:running rgw multisite tests on '/home/teuthworker/src/github.com_ceph_ceph-c_4d487742eda56d4badbc9bdb33761243365bb37a/qa/../src/test/rgw/rgw_multi' with args=['tests.py', 'tests_ps.py'] 2022-09-03T05:20:12.764 DEBUG:boto:Using access key provided by client. 2022-09-03T05:20:12.777 DEBUG:boto:Using secret key provided by client. 2022-09-03T05:20:12.779 DEBUG:boto:Using access key provided by client. 2022-09-03T05:20:12.782 DEBUG:boto:Using secret key provided by client. 2022-09-03T05:20:12.783 DEBUG:boto:Using access key provided by client. 2022-09-03T05:20:12.784 DEBUG:boto:Using secret key provided by client. 2022-09-03T05:20:12.784 DEBUG:boto:Using access key provided by client. 2022-09-03T05:20:12.785 DEBUG:boto:Using secret key provided by client. 2022-09-03T05:20:12.786 DEBUG:boto:Using access key provided by client. 2022-09-03T05:20:12.787 DEBUG:boto:Using secret key provided by client. 2022-09-03T05:20:12.788 DEBUG:boto:Using access key provided by client. 2022-09-03T05:20:12.789 DEBUG:boto:Using secret key provided by client. 2022-09-03T05:20:12.790 DEBUG:boto:Using access key provided by client. 2022-09-03T05:20:12.791 DEBUG:boto:Using secret key provided by client. 2022-09-03T05:20:12.792 DEBUG:boto:Using access key provided by client. 2022-09-03T05:20:12.792 DEBUG:boto:Using secret key provided by client. 2022-09-03T05:20:12.793 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-1 2022-09-03T05:20:12.793 DEBUG:boto:path=/dsgvcy-1/ 2022-09-03T05:20:12.794 DEBUG:boto:auth_path=/dsgvcy-1/ 2022-09-03T05:20:12.794 DEBUG:boto:Method: PUT 2022-09-03T05:20:12.795 DEBUG:boto:Path: /dsgvcy-1/ 2022-09-03T05:20:12.796 DEBUG:boto:Data: 2022-09-03T05:20:12.796 DEBUG:boto:Headers: {} 2022-09-03T05:20:12.797 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:12.797 DEBUG:boto:Port: 8000 2022-09-03T05:20:12.798 DEBUG:boto:Params: {} 2022-09-03T05:20:12.799 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:20:12.800 DEBUG:boto:Token: None 2022-09-03T05:20:12.802 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:20:12 GMT /dsgvcy-1/ 2022-09-03T05:20:12.804 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yl0SRpfJlP/FyFB3krxE1xx0f50= 2022-09-03T05:20:12.805 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yl0SRpfJlP/FyFB3krxE1xx0f50=', 'Content-Length': '0'} 2022-09-03T05:20:14.971 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000c27dc67914eea1f6-006312e40c-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:14.972 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=dsgvcy-2 2022-09-03T05:20:14.973 DEBUG:boto:path=/dsgvcy-2/ 2022-09-03T05:20:14.974 DEBUG:boto:auth_path=/dsgvcy-2/ 2022-09-03T05:20:14.975 DEBUG:boto:Method: PUT 2022-09-03T05:20:14.976 DEBUG:boto:Path: /dsgvcy-2/ 2022-09-03T05:20:14.977 DEBUG:boto:Data: 2022-09-03T05:20:14.978 DEBUG:boto:Headers: {} 2022-09-03T05:20:14.979 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:14.980 DEBUG:boto:Port: 8000 2022-09-03T05:20:14.981 DEBUG:boto:Params: {} 2022-09-03T05:20:14.982 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:20:14.983 DEBUG:boto:Token: None 2022-09-03T05:20:14.984 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:20:14 GMT /dsgvcy-2/ 2022-09-03T05:20:14.984 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DAQYqD7z92C6ITJppGizbtPkQPQ= 2022-09-03T05:20:14.985 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DAQYqD7z92C6ITJppGizbtPkQPQ=', 'Content-Length': '0'} 2022-09-03T05:20:17.277 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000002d450b95eae3cd17-006312e40e-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:17.278 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=dsgvcy-3 2022-09-03T05:20:17.279 DEBUG:boto:path=/dsgvcy-3/ 2022-09-03T05:20:17.279 DEBUG:boto:auth_path=/dsgvcy-3/ 2022-09-03T05:20:17.280 DEBUG:boto:Method: PUT 2022-09-03T05:20:17.281 DEBUG:boto:Path: /dsgvcy-3/ 2022-09-03T05:20:17.283 DEBUG:boto:Data: 2022-09-03T05:20:17.284 DEBUG:boto:Headers: {} 2022-09-03T05:20:17.284 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:17.285 DEBUG:boto:Port: 8001 2022-09-03T05:20:17.285 DEBUG:boto:Params: {} 2022-09-03T05:20:17.286 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T05:20:17.287 DEBUG:boto:Token: None 2022-09-03T05:20:17.287 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:20:17 GMT /dsgvcy-3/ 2022-09-03T05:20:17.288 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:euyUM7f/hUgYWNPBNex0fgs7Q9s= 2022-09-03T05:20:17.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:euyUM7f/hUgYWNPBNex0fgs7Q9s=', 'Content-Length': '0'} 2022-09-03T05:20:17.334 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000003324f712b320e97-006312e411-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:17.337 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:17.338 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:17.339 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:17.395 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:17.395 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:17.440 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=41ecd2a5-afc0-4eae-9349-7dfe39baa1cb 2022-09-03T05:20:17.455 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:20:17.455 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:17.456 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:17.456 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:18:18.906852Z" 2022-09-03T05:20:17.457 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:17.458 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:17.458 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:17.459 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:20:17.459 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:17.459 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:17.460 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:17.460 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:20:17.460 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:17.460 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:17.461 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182417.319609_30.1", 2022-09-03T05:20:17.461 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:20:17.319609Z" 2022-09-03T05:20:17.461 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:17.462 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:20:17.465 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182417.319609_30.1'} 2022-09-03T05:20:17.465 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:20:17.466 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:17.467 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:17.467 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:17.519 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:17.520 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:17.559 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:17.561+0000 7f91c338f600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:17.560 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:17.562+0000 7f91c338f600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:17.575 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:20:17.575 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:20:17.576 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:20:17.576 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:20:17.577 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:20:17.577 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:17.578 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:17.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:17.579 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:20:17.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:17.580 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:20:17.580 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:17.581 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:17.581 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:17.582 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:17.582 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:20:17.583 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:17.583 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:17.584 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:17.585 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:17.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:17.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:17.585 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:20:17.586 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:17.586 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:17.586 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:17.586 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:17.587 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:17.587 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:17.587 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:17.587 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:17.588 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:17.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:17.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:17.589 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:20:17.589 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:17.589 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:17.589 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:17.590 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:17.590 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:17.590 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:17.590 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:17.591 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:17.591 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:17.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:17.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:17.592 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:20:17.592 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:17.592 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:17.593 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182414.943600_19.1", 2022-09-03T05:20:17.593 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:17.593 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:17.593 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:17.593 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:14.943600Z", 2022-09-03T05:20:17.594 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:17.594 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:17.594 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:17.595 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:20:17.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:17.595 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:20:17.596 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:20:17.596 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:20:17.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:17.596 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:20:17Z" 2022-09-03T05:20:17.597 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:20:17.598 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182414.943600_19.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:14.943600Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:20:17Z" } 2022-09-03T05:20:17.599 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182414.943600_19.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:14.943600Z', 'realm_epoch': 2}}] 2022-09-03T05:20:17.599 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182417.319609_30.1'} 2022-09-03T05:20:17.600 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182414.943600_19.1'} 2022-09-03T05:20:17.600 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=3 master=1_1662182417.319609_30.1 target=1_1662182414.943600_19.1 2022-09-03T05:20:22.602 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:22.603 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:22.603 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:22.661 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:22.662 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:22.701 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:22.703+0000 7f9a6e5fe600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:22.702 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:22.704+0000 7f9a6e5fe600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:22.713 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:20:22.713 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:20:22.714 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:20:22.714 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:20:22.714 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:20:22.715 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:22.715 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:22.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:22.715 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:20:22.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:22.716 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:20:22.716 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:22.717 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:22.717 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:22.718 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:22.718 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:20:22.718 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:22.719 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:22.719 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:22.720 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:22.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:22.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:22.721 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:20:22.721 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:22.721 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:22.721 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:22.722 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:22.722 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:22.722 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:22.722 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:22.723 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:22.723 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:22.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:22.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:22.724 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:20:22.724 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:22.724 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:22.724 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:22.725 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:22.725 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:22.725 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:22.725 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:22.725 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:22.726 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:22.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:22.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:22.728 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:20:22.729 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:22.729 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:22.729 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182417.319609_30.1", 2022-09-03T05:20:22.730 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:22.730 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:22.730 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:22.731 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:17.319609Z", 2022-09-03T05:20:22.731 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:22.731 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:22.732 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:22.732 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:20:22.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:22.732 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:20:22.733 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:20:22.733 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:20:22.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:22.733 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:20:22Z" 2022-09-03T05:20:22.734 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:20:22.735 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182417.319609_30.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:17.319609Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:20:22Z" } 2022-09-03T05:20:22.736 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182417.319609_30.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:17.319609Z', 'realm_epoch': 2}}] 2022-09-03T05:20:22.736 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182417.319609_30.1'} 2022-09-03T05:20:22.737 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182417.319609_30.1'} 2022-09-03T05:20:22.737 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:20:22.738 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:20:22.738 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:22.739 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:22.740 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:22.796 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:22.797 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:22.845 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:20:22.847+0000 7fb911ebc600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:22.846 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:20:22.848+0000 7fb911ebc600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:22.869 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:20:22.870 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:20:22.871 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:20:22.871 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:20:22.871 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:20:22.872 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:22.872 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:22.873 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:22.873 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:20:22.874 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:22.874 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:20:22.875 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:22.875 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:22.876 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:22.876 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:22.876 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:20:22.876 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:22.877 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:22.877 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:22.878 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:22.878 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:22.879 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:22.879 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:20:22.880 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:22.880 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:22.881 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:22.881 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:22.882 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:22.882 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:22.883 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:20:22.883 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:22.884 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:22.884 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:22.885 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:22.885 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:20:22.886 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:22.887 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:22.887 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:22.887 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:22.888 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:22.888 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:22.888 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:20:22.889 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:22.889 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:22.889 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:22.889 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:22.890 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:20:22.890 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:22.890 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:22.891 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182417.319609_30.1", 2022-09-03T05:20:22.891 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:22.892 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:22.892 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:22.893 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:17.319609Z", 2022-09-03T05:20:22.893 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:22.893 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:20:22.894 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:22.895 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:20:22.895 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:20:22.895 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:20:22.895 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:22.896 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:20:22Z" 2022-09-03T05:20:22.896 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:20:22.898 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182417.319609_30.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:17.319609Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:20:22Z" } 2022-09-03T05:20:22.898 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182417.319609_30.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:17.319609Z', 'realm_epoch': 2}}] 2022-09-03T05:20:22.899 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182417.319609_30.1'} 2022-09-03T05:20:22.899 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182417.319609_30.1'} 2022-09-03T05:20:22.900 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:20:22.901 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:20:22.901 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:22.902 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:22.903 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:22.960 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:22.961 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:22.986 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:22.989+0000 7f9256d83600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:22.988 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:22.990+0000 7f9256d83600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:22.998 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:20:22.998 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:20:22.999 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:20:22.999 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:20:23.000 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:20:23.000 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:23.001 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:23.001 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:23.002 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:20:23.002 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:23.003 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:20:23.003 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:23.004 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:23.004 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:23.005 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:23.005 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:20:23.006 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:23.006 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:23.006 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:23.007 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:23.007 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:23.008 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:23.008 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:20:23.008 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:23.009 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:23.009 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:23.010 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:23.010 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:23.011 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:23.011 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:23.011 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:23.011 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:23.012 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:23.012 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:23.012 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:20:23.012 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:23.013 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:23.013 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:23.013 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:23.014 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:23.014 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:23.014 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:23.015 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:23.015 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:23.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:23.015 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:23.016 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:20:23.016 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:23.016 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:23.016 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182417.319609_30.1", 2022-09-03T05:20:23.017 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:23.017 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:23.017 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:23.018 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:17.319609Z", 2022-09-03T05:20:23.018 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:23.018 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:23.019 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:23.019 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:20:23.019 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:23.019 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:20:23.020 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:20:23.020 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:20:23.020 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:23.020 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:20:22Z" 2022-09-03T05:20:23.021 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:20:23.022 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182417.319609_30.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:17.319609Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:20:22Z" } 2022-09-03T05:20:23.023 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182417.319609_30.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:17.319609Z', 'realm_epoch': 2}}] 2022-09-03T05:20:23.023 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182417.319609_30.1'} 2022-09-03T05:20:23.024 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182417.319609_30.1'} 2022-09-03T05:20:23.024 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:20:23.025 DEBUG:boto:path=/dsgvcy-1/ 2022-09-03T05:20:23.025 DEBUG:boto:auth_path=/dsgvcy-1/ 2022-09-03T05:20:23.026 DEBUG:boto:Method: HEAD 2022-09-03T05:20:23.026 DEBUG:boto:Path: /dsgvcy-1/ 2022-09-03T05:20:23.027 DEBUG:boto:Data: 2022-09-03T05:20:23.027 DEBUG:boto:Headers: {} 2022-09-03T05:20:23.028 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:23.028 DEBUG:boto:Port: 8000 2022-09-03T05:20:23.029 DEBUG:boto:Params: {} 2022-09-03T05:20:23.029 DEBUG:boto:Token: None 2022-09-03T05:20:23.030 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:23 GMT /dsgvcy-1/ 2022-09-03T05:20:23.031 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hViuWhlcWo5RVej5g5Fl2A6cwbk= 2022-09-03T05:20:23.031 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hViuWhlcWo5RVej5g5Fl2A6cwbk=', 'Content-Length': '0'} 2022-09-03T05:20:23.034 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000088aac8b5e6157e56-006312e417-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:23.034 DEBUG:boto:path=/dsgvcy-2/ 2022-09-03T05:20:23.035 DEBUG:boto:auth_path=/dsgvcy-2/ 2022-09-03T05:20:23.035 DEBUG:boto:Method: HEAD 2022-09-03T05:20:23.036 DEBUG:boto:Path: /dsgvcy-2/ 2022-09-03T05:20:23.036 DEBUG:boto:Data: 2022-09-03T05:20:23.037 DEBUG:boto:Headers: {} 2022-09-03T05:20:23.037 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:23.038 DEBUG:boto:Port: 8000 2022-09-03T05:20:23.039 DEBUG:boto:Params: {} 2022-09-03T05:20:23.039 DEBUG:boto:Token: None 2022-09-03T05:20:23.040 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:23 GMT /dsgvcy-2/ 2022-09-03T05:20:23.040 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ekyHpCWRHfmFhOUjCWaeRIvNDVg= 2022-09-03T05:20:23.041 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ekyHpCWRHfmFhOUjCWaeRIvNDVg=', 'Content-Length': '0'} 2022-09-03T05:20:23.044 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f070b4272cc7d109-006312e417-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:23.044 DEBUG:boto:path=/dsgvcy-3/ 2022-09-03T05:20:23.045 DEBUG:boto:auth_path=/dsgvcy-3/ 2022-09-03T05:20:23.045 DEBUG:boto:Method: HEAD 2022-09-03T05:20:23.046 DEBUG:boto:Path: /dsgvcy-3/ 2022-09-03T05:20:23.046 DEBUG:boto:Data: 2022-09-03T05:20:23.047 DEBUG:boto:Headers: {} 2022-09-03T05:20:23.047 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:23.048 DEBUG:boto:Port: 8000 2022-09-03T05:20:23.048 DEBUG:boto:Params: {} 2022-09-03T05:20:23.049 DEBUG:boto:Token: None 2022-09-03T05:20:23.050 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:23 GMT /dsgvcy-3/ 2022-09-03T05:20:23.050 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OV3SghxXewMfleF10Ht0qVHliXo= 2022-09-03T05:20:23.051 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OV3SghxXewMfleF10Ht0qVHliXo=', 'Content-Length': '0'} 2022-09-03T05:20:23.053 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000a7f3a6aa3abaeb3-006312e417-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:23.054 DEBUG:boto:path=/dsgvcy-1/ 2022-09-03T05:20:23.054 DEBUG:boto:auth_path=/dsgvcy-1/ 2022-09-03T05:20:23.055 DEBUG:boto:Method: HEAD 2022-09-03T05:20:23.055 DEBUG:boto:Path: /dsgvcy-1/ 2022-09-03T05:20:23.056 DEBUG:boto:Data: 2022-09-03T05:20:23.056 DEBUG:boto:Headers: {} 2022-09-03T05:20:23.057 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:23.057 DEBUG:boto:Port: 8000 2022-09-03T05:20:23.058 DEBUG:boto:Params: {} 2022-09-03T05:20:23.059 DEBUG:boto:Token: None 2022-09-03T05:20:23.059 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:23 GMT /dsgvcy-1/ 2022-09-03T05:20:23.060 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hViuWhlcWo5RVej5g5Fl2A6cwbk= 2022-09-03T05:20:23.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hViuWhlcWo5RVej5g5Fl2A6cwbk=', 'Content-Length': '0'} 2022-09-03T05:20:23.063 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000095bd9e35d0c7c2bf-006312e417-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:23.064 DEBUG:boto:path=/dsgvcy-2/ 2022-09-03T05:20:23.065 DEBUG:boto:auth_path=/dsgvcy-2/ 2022-09-03T05:20:23.066 DEBUG:boto:Method: HEAD 2022-09-03T05:20:23.067 DEBUG:boto:Path: /dsgvcy-2/ 2022-09-03T05:20:23.068 DEBUG:boto:Data: 2022-09-03T05:20:23.069 DEBUG:boto:Headers: {} 2022-09-03T05:20:23.070 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:23.071 DEBUG:boto:Port: 8000 2022-09-03T05:20:23.072 DEBUG:boto:Params: {} 2022-09-03T05:20:23.073 DEBUG:boto:Token: None 2022-09-03T05:20:23.074 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:23 GMT /dsgvcy-2/ 2022-09-03T05:20:23.075 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ekyHpCWRHfmFhOUjCWaeRIvNDVg= 2022-09-03T05:20:23.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ekyHpCWRHfmFhOUjCWaeRIvNDVg=', 'Content-Length': '0'} 2022-09-03T05:20:23.079 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000156409685fbf206-006312e417-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:23.080 DEBUG:boto:path=/dsgvcy-3/ 2022-09-03T05:20:23.080 DEBUG:boto:auth_path=/dsgvcy-3/ 2022-09-03T05:20:23.081 DEBUG:boto:Method: HEAD 2022-09-03T05:20:23.081 DEBUG:boto:Path: /dsgvcy-3/ 2022-09-03T05:20:23.082 DEBUG:boto:Data: 2022-09-03T05:20:23.082 DEBUG:boto:Headers: {} 2022-09-03T05:20:23.083 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:23.084 DEBUG:boto:Port: 8000 2022-09-03T05:20:23.085 DEBUG:boto:Params: {} 2022-09-03T05:20:23.086 DEBUG:boto:Token: None 2022-09-03T05:20:23.087 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:23 GMT /dsgvcy-3/ 2022-09-03T05:20:23.088 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OV3SghxXewMfleF10Ht0qVHliXo= 2022-09-03T05:20:23.089 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OV3SghxXewMfleF10Ht0qVHliXo=', 'Content-Length': '0'} 2022-09-03T05:20:23.092 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000087fbcf98aa07629b-006312e417-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:23.093 DEBUG:boto:path=/dsgvcy-1/ 2022-09-03T05:20:23.094 DEBUG:boto:auth_path=/dsgvcy-1/ 2022-09-03T05:20:23.095 DEBUG:boto:Method: HEAD 2022-09-03T05:20:23.096 DEBUG:boto:Path: /dsgvcy-1/ 2022-09-03T05:20:23.096 DEBUG:boto:Data: 2022-09-03T05:20:23.097 DEBUG:boto:Headers: {} 2022-09-03T05:20:23.097 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:23.098 DEBUG:boto:Port: 8001 2022-09-03T05:20:23.098 DEBUG:boto:Params: {} 2022-09-03T05:20:23.099 DEBUG:boto:Token: None 2022-09-03T05:20:23.099 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:23 GMT /dsgvcy-1/ 2022-09-03T05:20:23.100 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hViuWhlcWo5RVej5g5Fl2A6cwbk= 2022-09-03T05:20:23.101 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hViuWhlcWo5RVej5g5Fl2A6cwbk=', 'Content-Length': '0'} 2022-09-03T05:20:23.103 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000453513efaa8c9f50-006312e417-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:23.104 DEBUG:boto:path=/dsgvcy-2/ 2022-09-03T05:20:23.104 DEBUG:boto:auth_path=/dsgvcy-2/ 2022-09-03T05:20:23.105 DEBUG:boto:Method: HEAD 2022-09-03T05:20:23.105 DEBUG:boto:Path: /dsgvcy-2/ 2022-09-03T05:20:23.106 DEBUG:boto:Data: 2022-09-03T05:20:23.106 DEBUG:boto:Headers: {} 2022-09-03T05:20:23.107 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:23.107 DEBUG:boto:Port: 8001 2022-09-03T05:20:23.108 DEBUG:boto:Params: {} 2022-09-03T05:20:23.109 DEBUG:boto:Token: None 2022-09-03T05:20:23.111 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:23 GMT /dsgvcy-2/ 2022-09-03T05:20:23.112 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ekyHpCWRHfmFhOUjCWaeRIvNDVg= 2022-09-03T05:20:23.113 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ekyHpCWRHfmFhOUjCWaeRIvNDVg=', 'Content-Length': '0'} 2022-09-03T05:20:23.116 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f40b296d4170ce9d-006312e417-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:23.117 DEBUG:boto:path=/dsgvcy-3/ 2022-09-03T05:20:23.118 DEBUG:boto:auth_path=/dsgvcy-3/ 2022-09-03T05:20:23.119 DEBUG:boto:Method: HEAD 2022-09-03T05:20:23.119 DEBUG:boto:Path: /dsgvcy-3/ 2022-09-03T05:20:23.120 DEBUG:boto:Data: 2022-09-03T05:20:23.121 DEBUG:boto:Headers: {} 2022-09-03T05:20:23.121 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:23.122 DEBUG:boto:Port: 8001 2022-09-03T05:20:23.122 DEBUG:boto:Params: {} 2022-09-03T05:20:23.123 DEBUG:boto:Token: None 2022-09-03T05:20:23.123 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:23 GMT /dsgvcy-3/ 2022-09-03T05:20:23.124 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OV3SghxXewMfleF10Ht0qVHliXo= 2022-09-03T05:20:23.125 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OV3SghxXewMfleF10Ht0qVHliXo=', 'Content-Length': '0'} 2022-09-03T05:20:23.127 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c9375a2dfd28fa2e-006312e417-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:23.128 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_create ... ok 2022-09-03T05:20:23.130 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-4 2022-09-03T05:20:23.131 DEBUG:boto:path=/dsgvcy-4/ 2022-09-03T05:20:23.131 DEBUG:boto:auth_path=/dsgvcy-4/ 2022-09-03T05:20:23.132 DEBUG:boto:Method: PUT 2022-09-03T05:20:23.132 DEBUG:boto:Path: /dsgvcy-4/ 2022-09-03T05:20:23.133 DEBUG:boto:Data: 2022-09-03T05:20:23.134 DEBUG:boto:Headers: {} 2022-09-03T05:20:23.135 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:23.136 DEBUG:boto:Port: 8000 2022-09-03T05:20:23.136 DEBUG:boto:Params: {} 2022-09-03T05:20:23.137 DEBUG:boto:Token: None 2022-09-03T05:20:23.138 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:20:23 GMT /dsgvcy-4/ 2022-09-03T05:20:23.138 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j+fJQRFqTKlMUgNMBDWj6GYPIWE= 2022-09-03T05:20:23.139 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j+fJQRFqTKlMUgNMBDWj6GYPIWE=', 'Content-Length': '0'} 2022-09-03T05:20:23.162 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000c3162c1fbd003833-006312e417-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:23.162 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=dsgvcy-5 2022-09-03T05:20:23.163 DEBUG:boto:path=/dsgvcy-5/ 2022-09-03T05:20:23.163 DEBUG:boto:auth_path=/dsgvcy-5/ 2022-09-03T05:20:23.164 DEBUG:boto:Method: PUT 2022-09-03T05:20:23.165 DEBUG:boto:Path: /dsgvcy-5/ 2022-09-03T05:20:23.165 DEBUG:boto:Data: 2022-09-03T05:20:23.166 DEBUG:boto:Headers: {} 2022-09-03T05:20:23.166 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:23.167 DEBUG:boto:Port: 8000 2022-09-03T05:20:23.167 DEBUG:boto:Params: {} 2022-09-03T05:20:23.168 DEBUG:boto:Token: None 2022-09-03T05:20:23.169 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:20:23 GMT /dsgvcy-5/ 2022-09-03T05:20:23.169 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L3SYd31g9iJo2Ou+yYOH4dACEwY= 2022-09-03T05:20:23.170 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L3SYd31g9iJo2Ou+yYOH4dACEwY=', 'Content-Length': '0'} 2022-09-03T05:20:23.193 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000045b62ba75f00615f-006312e417-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:23.194 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=dsgvcy-6 2022-09-03T05:20:23.195 DEBUG:boto:path=/dsgvcy-6/ 2022-09-03T05:20:23.195 DEBUG:boto:auth_path=/dsgvcy-6/ 2022-09-03T05:20:23.197 DEBUG:boto:Method: PUT 2022-09-03T05:20:23.197 DEBUG:boto:Path: /dsgvcy-6/ 2022-09-03T05:20:23.198 DEBUG:boto:Data: 2022-09-03T05:20:23.199 DEBUG:boto:Headers: {} 2022-09-03T05:20:23.200 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:23.201 DEBUG:boto:Port: 8001 2022-09-03T05:20:23.202 DEBUG:boto:Params: {} 2022-09-03T05:20:23.203 DEBUG:boto:Token: None 2022-09-03T05:20:23.204 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:20:23 GMT /dsgvcy-6/ 2022-09-03T05:20:23.205 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8idOpetH8mrTUqLoP5X1aHyd/Ic= 2022-09-03T05:20:23.206 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8idOpetH8mrTUqLoP5X1aHyd/Ic=', 'Content-Length': '0'} 2022-09-03T05:20:23.244 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000009b83b95cdbf096f1-006312e417-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:23.245 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:23.246 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:23.247 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:23.298 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:23.299 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:23.340 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=41ecd2a5-afc0-4eae-9349-7dfe39baa1cb 2022-09-03T05:20:23.361 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:20:23.362 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:23.362 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:23.362 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:18:18.906852Z" 2022-09-03T05:20:23.363 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:23.363 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:23.363 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:23.363 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:20:23.364 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:23.364 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:23.364 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:23.364 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:20:23.364 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:23.365 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:23.365 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182423.238098_42.1", 2022-09-03T05:20:23.366 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:20:23.238098Z" 2022-09-03T05:20:23.366 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:23.366 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:20:23.366 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182423.238098_42.1'} 2022-09-03T05:20:23.367 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:20:23.368 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:23.368 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:23.369 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:23.421 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:23.421 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:23.446 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:23.448+0000 7f79a5872600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:23.447 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:23.449+0000 7f79a5872600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:23.464 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:20:23.464 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:20:23.465 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:20:23.465 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:20:23.465 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:20:23.465 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:23.466 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:23.466 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:23.466 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:20:23.466 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:23.467 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:20:23.467 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:23.467 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:23.468 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:23.468 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:23.469 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:20:23.469 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:23.469 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:23.470 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:23.470 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:23.471 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:23.471 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:23.472 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:20:23.472 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:23.472 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:23.472 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:23.473 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:23.473 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:23.473 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:23.474 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:23.474 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:23.474 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:23.475 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:23.475 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:23.475 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:20:23.475 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:23.476 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:23.476 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:23.476 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:23.476 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:23.477 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:23.477 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:23.477 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:23.477 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:23.478 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:23.478 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:23.478 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:20:23.479 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:23.479 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:23.479 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182423.221675_39.1", 2022-09-03T05:20:23.479 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:23.480 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:23.480 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:23.480 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.221675Z", 2022-09-03T05:20:23.481 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:23.481 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:23.481 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:23.481 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:20:23.482 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:23.482 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:20:23.482 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:20:23.483 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:20:23.483 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:23.483 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:20:23Z" 2022-09-03T05:20:23.483 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:20:23.485 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182423.221675_39.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:23.221675Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:20:23Z" } 2022-09-03T05:20:23.486 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182423.221675_39.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:23.221675Z', 'realm_epoch': 2}}] 2022-09-03T05:20:23.486 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182423.238098_42.1'} 2022-09-03T05:20:23.487 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182423.221675_39.1'} 2022-09-03T05:20:23.487 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=3 master=1_1662182423.238098_42.1 target=1_1662182423.221675_39.1 2022-09-03T05:20:28.489 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:28.490 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:28.491 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:28.549 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:28.549 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:28.589 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:28.591+0000 7ff4f91fc600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:28.590 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:28.593+0000 7ff4f91fc600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:28.605 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:20:28.605 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:20:28.606 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:20:28.606 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:20:28.607 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:20:28.607 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:28.607 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:28.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:28.608 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:20:28.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:28.608 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:20:28.608 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:28.609 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:28.609 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:28.609 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:28.610 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:20:28.610 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:28.610 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:28.610 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:28.611 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:28.611 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:28.611 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:28.611 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:20:28.612 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:28.612 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:28.612 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:28.612 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:28.613 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:28.613 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:28.613 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:28.613 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:28.614 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:28.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:28.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:28.614 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:20:28.615 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:28.615 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:28.615 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:28.615 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:28.616 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:28.616 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:28.616 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:28.616 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:28.617 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:28.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:28.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:28.617 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:20:28.618 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:28.618 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:28.618 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182423.238098_42.1", 2022-09-03T05:20:28.618 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:28.619 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:28.619 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:28.619 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.238098Z", 2022-09-03T05:20:28.619 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:28.620 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:28.620 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:28.620 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:20:28.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:28.621 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:20:28.621 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:20:28.621 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:20:28.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:28.622 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:20:28Z" 2022-09-03T05:20:28.622 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:20:28.623 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182423.238098_42.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:23.238098Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:20:28Z" } 2022-09-03T05:20:28.624 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182423.238098_42.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:23.238098Z', 'realm_epoch': 2}}] 2022-09-03T05:20:28.625 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182423.238098_42.1'} 2022-09-03T05:20:28.625 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182423.238098_42.1'} 2022-09-03T05:20:28.626 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:20:28.626 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:20:28.627 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:28.627 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:28.628 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:28.684 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:28.685 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:28.729 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:20:28.732+0000 7f2a0a5cb600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:28.730 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:20:28.733+0000 7f2a0a5cb600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:28.741 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:20:28.742 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:20:28.742 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:20:28.742 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:20:28.742 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:20:28.743 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:28.743 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:28.743 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:28.743 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:20:28.744 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:28.744 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:20:28.744 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:28.745 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:28.745 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:28.745 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:28.745 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:20:28.746 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:28.746 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:28.746 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:28.747 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:28.747 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:28.747 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:28.747 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:20:28.748 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:28.748 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:28.748 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:28.748 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:28.749 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:28.749 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:28.750 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:20:28.750 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:28.751 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:28.751 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:28.751 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:28.752 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:20:28.752 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:28.752 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:28.753 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:28.753 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:28.753 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:28.753 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:28.754 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:20:28.754 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:28.755 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:28.755 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:28.756 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:28.757 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:20:28.757 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:28.758 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:28.758 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182423.238098_42.1", 2022-09-03T05:20:28.759 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:28.759 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:28.759 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:28.760 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.238098Z", 2022-09-03T05:20:28.760 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:28.760 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:28.761 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:28.761 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:20:28.761 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:28.762 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:20:28.762 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:20:28.762 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:20:28.762 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:28.763 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:20:28Z" 2022-09-03T05:20:28.763 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:20:28.764 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182423.238098_42.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:23.238098Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:20:28Z" } 2022-09-03T05:20:28.765 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182423.238098_42.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:23.238098Z', 'realm_epoch': 2}}] 2022-09-03T05:20:28.766 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182423.238098_42.1'} 2022-09-03T05:20:28.767 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182423.238098_42.1'} 2022-09-03T05:20:28.767 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:20:28.768 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:20:28.769 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:28.769 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:28.770 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:28.823 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:28.824 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:28.850 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:28.852+0000 7f52f9fa5600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:28.851 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:28.853+0000 7f52f9fa5600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:28.861 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:20:28.862 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:20:28.862 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:20:28.862 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:20:28.863 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:20:28.863 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:28.863 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:28.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:28.864 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:20:28.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:28.864 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:20:28.865 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:28.865 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:28.865 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:28.865 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:28.866 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:20:28.866 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:28.866 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:28.866 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:28.867 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:28.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:28.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:28.867 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:20:28.868 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:28.868 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:28.868 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:28.868 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:28.869 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:28.869 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:28.869 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:28.869 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:28.870 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:28.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:28.870 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:28.871 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:20:28.871 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:28.871 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:28.871 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:28.872 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:28.872 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:28.872 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:28.872 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:28.873 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:28.873 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:28.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:28.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:28.874 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:20:28.874 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:28.874 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:28.875 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182423.238098_42.1", 2022-09-03T05:20:28.875 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:28.875 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:28.875 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:28.876 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.238098Z", 2022-09-03T05:20:28.876 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:28.876 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:28.876 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:28.877 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:20:28.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:28.877 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:20:28.877 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:20:28.878 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:20:28.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:28.878 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:20:28Z" 2022-09-03T05:20:28.878 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:20:28.880 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182423.238098_42.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:23.238098Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:20:28Z" } 2022-09-03T05:20:28.880 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182423.238098_42.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:23.238098Z', 'realm_epoch': 2}}] 2022-09-03T05:20:28.881 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182423.238098_42.1'} 2022-09-03T05:20:28.881 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182423.238098_42.1'} 2022-09-03T05:20:28.882 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:20:28.883 DEBUG:boto:path=/dsgvcy-4/ 2022-09-03T05:20:28.884 DEBUG:boto:auth_path=/dsgvcy-4/ 2022-09-03T05:20:28.885 DEBUG:boto:Method: HEAD 2022-09-03T05:20:28.886 DEBUG:boto:Path: /dsgvcy-4/ 2022-09-03T05:20:28.887 DEBUG:boto:Data: 2022-09-03T05:20:28.887 DEBUG:boto:Headers: {} 2022-09-03T05:20:28.888 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:28.889 DEBUG:boto:Port: 8000 2022-09-03T05:20:28.890 DEBUG:boto:Params: {} 2022-09-03T05:20:28.891 DEBUG:boto:Token: None 2022-09-03T05:20:28.892 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:28 GMT /dsgvcy-4/ 2022-09-03T05:20:28.893 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i83vGx0itHxldUwo1/6astwCW/c= 2022-09-03T05:20:28.894 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i83vGx0itHxldUwo1/6astwCW/c=', 'Content-Length': '0'} 2022-09-03T05:20:28.897 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000493a8f08a22003aa-006312e41c-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:28.898 DEBUG:boto:path=/dsgvcy-5/ 2022-09-03T05:20:28.898 DEBUG:boto:auth_path=/dsgvcy-5/ 2022-09-03T05:20:28.899 DEBUG:boto:Method: HEAD 2022-09-03T05:20:28.899 DEBUG:boto:Path: /dsgvcy-5/ 2022-09-03T05:20:28.900 DEBUG:boto:Data: 2022-09-03T05:20:28.900 DEBUG:boto:Headers: {} 2022-09-03T05:20:28.901 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:28.901 DEBUG:boto:Port: 8000 2022-09-03T05:20:28.902 DEBUG:boto:Params: {} 2022-09-03T05:20:28.903 DEBUG:boto:Token: None 2022-09-03T05:20:28.903 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:28 GMT /dsgvcy-5/ 2022-09-03T05:20:28.904 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WJhb90qI2kQSJG4zUq/QnA6uic8= 2022-09-03T05:20:28.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WJhb90qI2kQSJG4zUq/QnA6uic8=', 'Content-Length': '0'} 2022-09-03T05:20:28.906 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000de1749639d4cd734-006312e41c-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:28.907 DEBUG:boto:path=/dsgvcy-6/ 2022-09-03T05:20:28.908 DEBUG:boto:auth_path=/dsgvcy-6/ 2022-09-03T05:20:28.908 DEBUG:boto:Method: HEAD 2022-09-03T05:20:28.909 DEBUG:boto:Path: /dsgvcy-6/ 2022-09-03T05:20:28.909 DEBUG:boto:Data: 2022-09-03T05:20:28.910 DEBUG:boto:Headers: {} 2022-09-03T05:20:28.910 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:28.911 DEBUG:boto:Port: 8000 2022-09-03T05:20:28.911 DEBUG:boto:Params: {} 2022-09-03T05:20:28.912 DEBUG:boto:Token: None 2022-09-03T05:20:28.912 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:28 GMT /dsgvcy-6/ 2022-09-03T05:20:28.913 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:x47/J3ht6eF/h7RdFVqIttyVggQ= 2022-09-03T05:20:28.913 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:x47/J3ht6eF/h7RdFVqIttyVggQ=', 'Content-Length': '0'} 2022-09-03T05:20:28.916 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a471c97542fdd0af-006312e41c-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:28.916 DEBUG:boto:path=/dsgvcy-4/ 2022-09-03T05:20:28.917 DEBUG:boto:auth_path=/dsgvcy-4/ 2022-09-03T05:20:28.917 DEBUG:boto:Method: HEAD 2022-09-03T05:20:28.918 DEBUG:boto:Path: /dsgvcy-4/ 2022-09-03T05:20:28.919 DEBUG:boto:Data: 2022-09-03T05:20:28.919 DEBUG:boto:Headers: {} 2022-09-03T05:20:28.920 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:28.920 DEBUG:boto:Port: 8000 2022-09-03T05:20:28.921 DEBUG:boto:Params: {} 2022-09-03T05:20:28.921 DEBUG:boto:Token: None 2022-09-03T05:20:28.922 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:28 GMT /dsgvcy-4/ 2022-09-03T05:20:28.923 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i83vGx0itHxldUwo1/6astwCW/c= 2022-09-03T05:20:28.923 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i83vGx0itHxldUwo1/6astwCW/c=', 'Content-Length': '0'} 2022-09-03T05:20:28.926 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000054984b306280eae3-006312e41c-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:28.926 DEBUG:boto:path=/dsgvcy-5/ 2022-09-03T05:20:28.927 DEBUG:boto:auth_path=/dsgvcy-5/ 2022-09-03T05:20:28.927 DEBUG:boto:Method: HEAD 2022-09-03T05:20:28.928 DEBUG:boto:Path: /dsgvcy-5/ 2022-09-03T05:20:28.929 DEBUG:boto:Data: 2022-09-03T05:20:28.929 DEBUG:boto:Headers: {} 2022-09-03T05:20:28.930 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:28.930 DEBUG:boto:Port: 8000 2022-09-03T05:20:28.931 DEBUG:boto:Params: {} 2022-09-03T05:20:28.931 DEBUG:boto:Token: None 2022-09-03T05:20:28.932 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:28 GMT /dsgvcy-5/ 2022-09-03T05:20:28.933 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WJhb90qI2kQSJG4zUq/QnA6uic8= 2022-09-03T05:20:28.933 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WJhb90qI2kQSJG4zUq/QnA6uic8=', 'Content-Length': '0'} 2022-09-03T05:20:28.936 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000936a07ba5e9d0e29-006312e41c-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:28.937 DEBUG:boto:path=/dsgvcy-6/ 2022-09-03T05:20:28.937 DEBUG:boto:auth_path=/dsgvcy-6/ 2022-09-03T05:20:28.938 DEBUG:boto:Method: HEAD 2022-09-03T05:20:28.938 DEBUG:boto:Path: /dsgvcy-6/ 2022-09-03T05:20:28.939 DEBUG:boto:Data: 2022-09-03T05:20:28.939 DEBUG:boto:Headers: {} 2022-09-03T05:20:28.940 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:28.940 DEBUG:boto:Port: 8000 2022-09-03T05:20:28.941 DEBUG:boto:Params: {} 2022-09-03T05:20:28.941 DEBUG:boto:Token: None 2022-09-03T05:20:28.942 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:28 GMT /dsgvcy-6/ 2022-09-03T05:20:28.943 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:x47/J3ht6eF/h7RdFVqIttyVggQ= 2022-09-03T05:20:28.943 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:x47/J3ht6eF/h7RdFVqIttyVggQ=', 'Content-Length': '0'} 2022-09-03T05:20:28.946 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000065b2cd3464b29d96-006312e41c-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:28.946 DEBUG:boto:path=/dsgvcy-4/ 2022-09-03T05:20:28.947 DEBUG:boto:auth_path=/dsgvcy-4/ 2022-09-03T05:20:28.947 DEBUG:boto:Method: HEAD 2022-09-03T05:20:28.948 DEBUG:boto:Path: /dsgvcy-4/ 2022-09-03T05:20:28.948 DEBUG:boto:Data: 2022-09-03T05:20:28.949 DEBUG:boto:Headers: {} 2022-09-03T05:20:28.949 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:28.950 DEBUG:boto:Port: 8001 2022-09-03T05:20:28.950 DEBUG:boto:Params: {} 2022-09-03T05:20:28.951 DEBUG:boto:Token: None 2022-09-03T05:20:28.951 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:28 GMT /dsgvcy-4/ 2022-09-03T05:20:28.952 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i83vGx0itHxldUwo1/6astwCW/c= 2022-09-03T05:20:28.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i83vGx0itHxldUwo1/6astwCW/c=', 'Content-Length': '0'} 2022-09-03T05:20:28.955 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000069561c74037549d2-006312e41c-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:28.956 DEBUG:boto:path=/dsgvcy-5/ 2022-09-03T05:20:28.956 DEBUG:boto:auth_path=/dsgvcy-5/ 2022-09-03T05:20:28.957 DEBUG:boto:Method: HEAD 2022-09-03T05:20:28.957 DEBUG:boto:Path: /dsgvcy-5/ 2022-09-03T05:20:28.958 DEBUG:boto:Data: 2022-09-03T05:20:28.958 DEBUG:boto:Headers: {} 2022-09-03T05:20:28.959 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:28.959 DEBUG:boto:Port: 8001 2022-09-03T05:20:28.960 DEBUG:boto:Params: {} 2022-09-03T05:20:28.960 DEBUG:boto:Token: None 2022-09-03T05:20:28.961 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:28 GMT /dsgvcy-5/ 2022-09-03T05:20:28.961 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WJhb90qI2kQSJG4zUq/QnA6uic8= 2022-09-03T05:20:28.962 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WJhb90qI2kQSJG4zUq/QnA6uic8=', 'Content-Length': '0'} 2022-09-03T05:20:28.964 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000de0c17ea5fbf5d89-006312e41c-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:28.965 DEBUG:boto:path=/dsgvcy-6/ 2022-09-03T05:20:28.965 DEBUG:boto:auth_path=/dsgvcy-6/ 2022-09-03T05:20:28.966 DEBUG:boto:Method: HEAD 2022-09-03T05:20:28.966 DEBUG:boto:Path: /dsgvcy-6/ 2022-09-03T05:20:28.967 DEBUG:boto:Data: 2022-09-03T05:20:28.967 DEBUG:boto:Headers: {} 2022-09-03T05:20:28.968 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:28.968 DEBUG:boto:Port: 8001 2022-09-03T05:20:28.969 DEBUG:boto:Params: {} 2022-09-03T05:20:28.969 DEBUG:boto:Token: None 2022-09-03T05:20:28.970 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:28 GMT /dsgvcy-6/ 2022-09-03T05:20:28.971 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:x47/J3ht6eF/h7RdFVqIttyVggQ= 2022-09-03T05:20:28.971 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:x47/J3ht6eF/h7RdFVqIttyVggQ=', 'Content-Length': '0'} 2022-09-03T05:20:28.974 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f39bf21bb7fc1c1c-006312e41c-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:28.974 DEBUG:boto:path=/dsgvcy-4/ 2022-09-03T05:20:28.975 DEBUG:boto:auth_path=/dsgvcy-4/ 2022-09-03T05:20:28.975 DEBUG:boto:Method: PUT 2022-09-03T05:20:28.976 DEBUG:boto:Path: /dsgvcy-4/ 2022-09-03T05:20:28.976 DEBUG:boto:Data: 2022-09-03T05:20:28.977 DEBUG:boto:Headers: {} 2022-09-03T05:20:28.977 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:28.978 DEBUG:boto:Port: 8000 2022-09-03T05:20:28.978 DEBUG:boto:Params: {} 2022-09-03T05:20:28.979 DEBUG:boto:Token: None 2022-09-03T05:20:28.979 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:20:28 GMT /dsgvcy-4/ 2022-09-03T05:20:28.980 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f1uMUSpXiBZk0QjLX/CkQdGMXkc= 2022-09-03T05:20:28.980 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f1uMUSpXiBZk0QjLX/CkQdGMXkc=', 'Content-Length': '0'} 2022-09-03T05:20:28.984 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000032d3b55a87007317-006312e41c-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:28.984 DEBUG:boto:path=/dsgvcy-5/ 2022-09-03T05:20:28.985 DEBUG:boto:auth_path=/dsgvcy-5/ 2022-09-03T05:20:28.985 DEBUG:boto:Method: PUT 2022-09-03T05:20:28.986 DEBUG:boto:Path: /dsgvcy-5/ 2022-09-03T05:20:28.987 DEBUG:boto:Data: 2022-09-03T05:20:28.987 DEBUG:boto:Headers: {} 2022-09-03T05:20:28.988 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:28.989 DEBUG:boto:Port: 8000 2022-09-03T05:20:28.990 DEBUG:boto:Params: {} 2022-09-03T05:20:28.990 DEBUG:boto:Token: None 2022-09-03T05:20:28.991 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:20:28 GMT /dsgvcy-5/ 2022-09-03T05:20:28.991 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h+9Kx1WYU5Eb8HDSO7OHuvFcURc= 2022-09-03T05:20:28.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h+9Kx1WYU5Eb8HDSO7OHuvFcURc=', 'Content-Length': '0'} 2022-09-03T05:20:28.995 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000a8da160a685478ec-006312e41c-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:28.995 DEBUG:boto:path=/dsgvcy-6/ 2022-09-03T05:20:28.996 DEBUG:boto:auth_path=/dsgvcy-6/ 2022-09-03T05:20:28.996 DEBUG:boto:Method: PUT 2022-09-03T05:20:28.997 DEBUG:boto:Path: /dsgvcy-6/ 2022-09-03T05:20:28.997 DEBUG:boto:Data: 2022-09-03T05:20:28.998 DEBUG:boto:Headers: {} 2022-09-03T05:20:28.998 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:28.999 DEBUG:boto:Port: 8000 2022-09-03T05:20:28.999 DEBUG:boto:Params: {} 2022-09-03T05:20:29.000 DEBUG:boto:Token: None 2022-09-03T05:20:29.000 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:20:29 GMT /dsgvcy-6/ 2022-09-03T05:20:29.001 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7eSO8TrhcFi+UQfI7SEzNuSw3VA= 2022-09-03T05:20:29.002 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7eSO8TrhcFi+UQfI7SEzNuSw3VA=', 'Content-Length': '0'} 2022-09-03T05:20:29.004 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000a2756f570dee67fa-006312e41d-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:29.006 DEBUG:boto:path=/dsgvcy-4/ 2022-09-03T05:20:29.006 DEBUG:boto:auth_path=/dsgvcy-4/ 2022-09-03T05:20:29.007 DEBUG:boto:Method: PUT 2022-09-03T05:20:29.007 DEBUG:boto:Path: /dsgvcy-4/ 2022-09-03T05:20:29.008 DEBUG:boto:Data: 2022-09-03T05:20:29.009 DEBUG:boto:Headers: {} 2022-09-03T05:20:29.010 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:29.010 DEBUG:boto:Port: 8000 2022-09-03T05:20:29.011 DEBUG:boto:Params: {} 2022-09-03T05:20:29.011 DEBUG:boto:Token: None 2022-09-03T05:20:29.012 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:20:29 GMT /dsgvcy-4/ 2022-09-03T05:20:29.012 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/LCbW7VfXzXyJn/AaSZw0HJ7pWY= 2022-09-03T05:20:29.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/LCbW7VfXzXyJn/AaSZw0HJ7pWY=', 'Content-Length': '0'} 2022-09-03T05:20:29.018 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000f60f2791e1a4f4bb-006312e41d-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:29.019 DEBUG:boto:path=/dsgvcy-5/ 2022-09-03T05:20:29.020 DEBUG:boto:auth_path=/dsgvcy-5/ 2022-09-03T05:20:29.021 DEBUG:boto:Method: PUT 2022-09-03T05:20:29.022 DEBUG:boto:Path: /dsgvcy-5/ 2022-09-03T05:20:29.023 DEBUG:boto:Data: 2022-09-03T05:20:29.024 DEBUG:boto:Headers: {} 2022-09-03T05:20:29.025 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:29.025 DEBUG:boto:Port: 8000 2022-09-03T05:20:29.026 DEBUG:boto:Params: {} 2022-09-03T05:20:29.026 DEBUG:boto:Token: None 2022-09-03T05:20:29.027 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:20:29 GMT /dsgvcy-5/ 2022-09-03T05:20:29.027 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J21a/cK6bxPCZ0P4S3fwHTExCxE= 2022-09-03T05:20:29.028 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J21a/cK6bxPCZ0P4S3fwHTExCxE=', 'Content-Length': '0'} 2022-09-03T05:20:29.074 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000009e8d41c29d7e398b-006312e41d-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:29.075 DEBUG:boto:path=/dsgvcy-6/ 2022-09-03T05:20:29.076 DEBUG:boto:auth_path=/dsgvcy-6/ 2022-09-03T05:20:29.076 DEBUG:boto:Method: PUT 2022-09-03T05:20:29.077 DEBUG:boto:Path: /dsgvcy-6/ 2022-09-03T05:20:29.077 DEBUG:boto:Data: 2022-09-03T05:20:29.078 DEBUG:boto:Headers: {} 2022-09-03T05:20:29.078 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:29.079 DEBUG:boto:Port: 8000 2022-09-03T05:20:29.079 DEBUG:boto:Params: {} 2022-09-03T05:20:29.080 DEBUG:boto:Token: None 2022-09-03T05:20:29.080 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:20:29 GMT /dsgvcy-6/ 2022-09-03T05:20:29.081 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7eSO8TrhcFi+UQfI7SEzNuSw3VA= 2022-09-03T05:20:29.083 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7eSO8TrhcFi+UQfI7SEzNuSw3VA=', 'Content-Length': '0'} 2022-09-03T05:20:29.129 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000002a728764e0648108-006312e41d-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:29.130 DEBUG:boto:path=/dsgvcy-4/ 2022-09-03T05:20:29.131 DEBUG:boto:auth_path=/dsgvcy-4/ 2022-09-03T05:20:29.131 DEBUG:boto:Method: PUT 2022-09-03T05:20:29.132 DEBUG:boto:Path: /dsgvcy-4/ 2022-09-03T05:20:29.132 DEBUG:boto:Data: 2022-09-03T05:20:29.133 DEBUG:boto:Headers: {} 2022-09-03T05:20:29.133 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:29.134 DEBUG:boto:Port: 8001 2022-09-03T05:20:29.135 DEBUG:boto:Params: {} 2022-09-03T05:20:29.135 DEBUG:boto:Token: None 2022-09-03T05:20:29.136 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:20:29 GMT /dsgvcy-4/ 2022-09-03T05:20:29.136 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/LCbW7VfXzXyJn/AaSZw0HJ7pWY= 2022-09-03T05:20:29.137 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/LCbW7VfXzXyJn/AaSZw0HJ7pWY=', 'Content-Length': '0'} 2022-09-03T05:20:29.142 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000003fc7dae434c8f845-006312e41d-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:29.143 DEBUG:boto:path=/dsgvcy-5/ 2022-09-03T05:20:29.144 DEBUG:boto:auth_path=/dsgvcy-5/ 2022-09-03T05:20:29.144 DEBUG:boto:Method: PUT 2022-09-03T05:20:29.145 DEBUG:boto:Path: /dsgvcy-5/ 2022-09-03T05:20:29.145 DEBUG:boto:Data: 2022-09-03T05:20:29.146 DEBUG:boto:Headers: {} 2022-09-03T05:20:29.146 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:29.147 DEBUG:boto:Port: 8001 2022-09-03T05:20:29.148 DEBUG:boto:Params: {} 2022-09-03T05:20:29.148 DEBUG:boto:Token: None 2022-09-03T05:20:29.149 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:20:29 GMT /dsgvcy-5/ 2022-09-03T05:20:29.150 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J21a/cK6bxPCZ0P4S3fwHTExCxE= 2022-09-03T05:20:29.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J21a/cK6bxPCZ0P4S3fwHTExCxE=', 'Content-Length': '0'} 2022-09-03T05:20:29.199 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000008ba694a91c3bcea3-006312e41d-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:29.200 DEBUG:boto:path=/dsgvcy-6/ 2022-09-03T05:20:29.200 DEBUG:boto:auth_path=/dsgvcy-6/ 2022-09-03T05:20:29.201 DEBUG:boto:Method: PUT 2022-09-03T05:20:29.201 DEBUG:boto:Path: /dsgvcy-6/ 2022-09-03T05:20:29.202 DEBUG:boto:Data: 2022-09-03T05:20:29.202 DEBUG:boto:Headers: {} 2022-09-03T05:20:29.203 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:29.203 DEBUG:boto:Port: 8001 2022-09-03T05:20:29.204 DEBUG:boto:Params: {} 2022-09-03T05:20:29.204 DEBUG:boto:Token: None 2022-09-03T05:20:29.205 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:20:29 GMT /dsgvcy-6/ 2022-09-03T05:20:29.206 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7eSO8TrhcFi+UQfI7SEzNuSw3VA= 2022-09-03T05:20:29.206 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7eSO8TrhcFi+UQfI7SEzNuSw3VA=', 'Content-Length': '0'} 2022-09-03T05:20:29.252 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000a614fd8cbfd2e7a8-006312e41d-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:29.254 DEBUG:boto:path=/dsgvcy-4/ 2022-09-03T05:20:29.255 DEBUG:boto:auth_path=/dsgvcy-4/ 2022-09-03T05:20:29.255 DEBUG:boto:Method: HEAD 2022-09-03T05:20:29.256 DEBUG:boto:Path: /dsgvcy-4/ 2022-09-03T05:20:29.256 DEBUG:boto:Data: 2022-09-03T05:20:29.257 DEBUG:boto:Headers: {} 2022-09-03T05:20:29.258 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:29.258 DEBUG:boto:Port: 8000 2022-09-03T05:20:29.259 DEBUG:boto:Params: {} 2022-09-03T05:20:29.260 DEBUG:boto:Token: None 2022-09-03T05:20:29.261 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:29 GMT /dsgvcy-4/ 2022-09-03T05:20:29.262 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ekF3j7VNknhmIt8ya76TfwFNWZE= 2022-09-03T05:20:29.262 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ekF3j7VNknhmIt8ya76TfwFNWZE=', 'Content-Length': '0'} 2022-09-03T05:20:29.265 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b7bc8fa49a057cb4-006312e41d-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:29.266 DEBUG:boto:path=/dsgvcy-5/ 2022-09-03T05:20:29.267 DEBUG:boto:auth_path=/dsgvcy-5/ 2022-09-03T05:20:29.268 DEBUG:boto:Method: HEAD 2022-09-03T05:20:29.268 DEBUG:boto:Path: /dsgvcy-5/ 2022-09-03T05:20:29.269 DEBUG:boto:Data: 2022-09-03T05:20:29.269 DEBUG:boto:Headers: {} 2022-09-03T05:20:29.270 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:29.270 DEBUG:boto:Port: 8000 2022-09-03T05:20:29.271 DEBUG:boto:Params: {} 2022-09-03T05:20:29.271 DEBUG:boto:Token: None 2022-09-03T05:20:29.272 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:29 GMT /dsgvcy-5/ 2022-09-03T05:20:29.273 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SIOOWzleHeFaT1OJ0ri5ejaKHcY= 2022-09-03T05:20:29.273 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SIOOWzleHeFaT1OJ0ri5ejaKHcY=', 'Content-Length': '0'} 2022-09-03T05:20:29.275 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fa13aca45b0be7b3-006312e41d-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:29.276 DEBUG:boto:path=/dsgvcy-6/ 2022-09-03T05:20:29.277 DEBUG:boto:auth_path=/dsgvcy-6/ 2022-09-03T05:20:29.277 DEBUG:boto:Method: HEAD 2022-09-03T05:20:29.278 DEBUG:boto:Path: /dsgvcy-6/ 2022-09-03T05:20:29.278 DEBUG:boto:Data: 2022-09-03T05:20:29.279 DEBUG:boto:Headers: {} 2022-09-03T05:20:29.279 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:29.280 DEBUG:boto:Port: 8000 2022-09-03T05:20:29.280 DEBUG:boto:Params: {} 2022-09-03T05:20:29.281 DEBUG:boto:Token: None 2022-09-03T05:20:29.282 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:29 GMT /dsgvcy-6/ 2022-09-03T05:20:29.282 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xOi5YSxK4KizpuQ4tVOlN1m6j7U= 2022-09-03T05:20:29.283 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xOi5YSxK4KizpuQ4tVOlN1m6j7U=', 'Content-Length': '0'} 2022-09-03T05:20:29.285 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a209247f84547ec0-006312e41d-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:29.285 DEBUG:boto:path=/dsgvcy-4/ 2022-09-03T05:20:29.286 DEBUG:boto:auth_path=/dsgvcy-4/ 2022-09-03T05:20:29.287 DEBUG:boto:Method: HEAD 2022-09-03T05:20:29.287 DEBUG:boto:Path: /dsgvcy-4/ 2022-09-03T05:20:29.288 DEBUG:boto:Data: 2022-09-03T05:20:29.288 DEBUG:boto:Headers: {} 2022-09-03T05:20:29.289 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:29.289 DEBUG:boto:Port: 8000 2022-09-03T05:20:29.290 DEBUG:boto:Params: {} 2022-09-03T05:20:29.290 DEBUG:boto:Token: None 2022-09-03T05:20:29.291 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:29 GMT /dsgvcy-4/ 2022-09-03T05:20:29.291 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ekF3j7VNknhmIt8ya76TfwFNWZE= 2022-09-03T05:20:29.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ekF3j7VNknhmIt8ya76TfwFNWZE=', 'Content-Length': '0'} 2022-09-03T05:20:29.294 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000dd767493a1e8e26f-006312e41d-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:29.295 DEBUG:boto:path=/dsgvcy-5/ 2022-09-03T05:20:29.296 DEBUG:boto:auth_path=/dsgvcy-5/ 2022-09-03T05:20:29.296 DEBUG:boto:Method: HEAD 2022-09-03T05:20:29.297 DEBUG:boto:Path: /dsgvcy-5/ 2022-09-03T05:20:29.297 DEBUG:boto:Data: 2022-09-03T05:20:29.298 DEBUG:boto:Headers: {} 2022-09-03T05:20:29.298 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:29.299 DEBUG:boto:Port: 8000 2022-09-03T05:20:29.299 DEBUG:boto:Params: {} 2022-09-03T05:20:29.300 DEBUG:boto:Token: None 2022-09-03T05:20:29.301 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:29 GMT /dsgvcy-5/ 2022-09-03T05:20:29.301 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SIOOWzleHeFaT1OJ0ri5ejaKHcY= 2022-09-03T05:20:29.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SIOOWzleHeFaT1OJ0ri5ejaKHcY=', 'Content-Length': '0'} 2022-09-03T05:20:29.304 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000039d25ed95d5975fa-006312e41d-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:29.305 DEBUG:boto:path=/dsgvcy-6/ 2022-09-03T05:20:29.306 DEBUG:boto:auth_path=/dsgvcy-6/ 2022-09-03T05:20:29.306 DEBUG:boto:Method: HEAD 2022-09-03T05:20:29.307 DEBUG:boto:Path: /dsgvcy-6/ 2022-09-03T05:20:29.307 DEBUG:boto:Data: 2022-09-03T05:20:29.308 DEBUG:boto:Headers: {} 2022-09-03T05:20:29.308 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:29.309 DEBUG:boto:Port: 8000 2022-09-03T05:20:29.310 DEBUG:boto:Params: {} 2022-09-03T05:20:29.310 DEBUG:boto:Token: None 2022-09-03T05:20:29.311 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:29 GMT /dsgvcy-6/ 2022-09-03T05:20:29.311 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xOi5YSxK4KizpuQ4tVOlN1m6j7U= 2022-09-03T05:20:29.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xOi5YSxK4KizpuQ4tVOlN1m6j7U=', 'Content-Length': '0'} 2022-09-03T05:20:29.315 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e51e12c71e50ffac-006312e41d-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:29.316 DEBUG:boto:path=/dsgvcy-4/ 2022-09-03T05:20:29.316 DEBUG:boto:auth_path=/dsgvcy-4/ 2022-09-03T05:20:29.317 DEBUG:boto:Method: HEAD 2022-09-03T05:20:29.318 DEBUG:boto:Path: /dsgvcy-4/ 2022-09-03T05:20:29.319 DEBUG:boto:Data: 2022-09-03T05:20:29.320 DEBUG:boto:Headers: {} 2022-09-03T05:20:29.321 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:29.322 DEBUG:boto:Port: 8001 2022-09-03T05:20:29.323 DEBUG:boto:Params: {} 2022-09-03T05:20:29.324 DEBUG:boto:Token: None 2022-09-03T05:20:29.325 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:29 GMT /dsgvcy-4/ 2022-09-03T05:20:29.325 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ekF3j7VNknhmIt8ya76TfwFNWZE= 2022-09-03T05:20:29.326 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ekF3j7VNknhmIt8ya76TfwFNWZE=', 'Content-Length': '0'} 2022-09-03T05:20:29.329 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f385cf31ddb84661-006312e41d-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:29.329 DEBUG:boto:path=/dsgvcy-5/ 2022-09-03T05:20:29.330 DEBUG:boto:auth_path=/dsgvcy-5/ 2022-09-03T05:20:29.330 DEBUG:boto:Method: HEAD 2022-09-03T05:20:29.331 DEBUG:boto:Path: /dsgvcy-5/ 2022-09-03T05:20:29.331 DEBUG:boto:Data: 2022-09-03T05:20:29.332 DEBUG:boto:Headers: {} 2022-09-03T05:20:29.332 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:29.333 DEBUG:boto:Port: 8001 2022-09-03T05:20:29.333 DEBUG:boto:Params: {} 2022-09-03T05:20:29.334 DEBUG:boto:Token: None 2022-09-03T05:20:29.335 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:29 GMT /dsgvcy-5/ 2022-09-03T05:20:29.335 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SIOOWzleHeFaT1OJ0ri5ejaKHcY= 2022-09-03T05:20:29.336 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SIOOWzleHeFaT1OJ0ri5ejaKHcY=', 'Content-Length': '0'} 2022-09-03T05:20:29.338 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000960c9f5bd534cbb-006312e41d-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:29.339 DEBUG:boto:path=/dsgvcy-6/ 2022-09-03T05:20:29.339 DEBUG:boto:auth_path=/dsgvcy-6/ 2022-09-03T05:20:29.340 DEBUG:boto:Method: HEAD 2022-09-03T05:20:29.340 DEBUG:boto:Path: /dsgvcy-6/ 2022-09-03T05:20:29.341 DEBUG:boto:Data: 2022-09-03T05:20:29.341 DEBUG:boto:Headers: {} 2022-09-03T05:20:29.342 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:29.342 DEBUG:boto:Port: 8001 2022-09-03T05:20:29.343 DEBUG:boto:Params: {} 2022-09-03T05:20:29.343 DEBUG:boto:Token: None 2022-09-03T05:20:29.344 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:29 GMT /dsgvcy-6/ 2022-09-03T05:20:29.345 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xOi5YSxK4KizpuQ4tVOlN1m6j7U= 2022-09-03T05:20:29.345 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xOi5YSxK4KizpuQ4tVOlN1m6j7U=', 'Content-Length': '0'} 2022-09-03T05:20:29.347 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c9e428b99cce11b2-006312e41d-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:29.348 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:29.349 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:29.349 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:29.398 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:29.399 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:29.429 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=41ecd2a5-afc0-4eae-9349-7dfe39baa1cb 2022-09-03T05:20:29.467 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:20:29.468 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:29.468 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:29.468 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:18:18.906852Z" 2022-09-03T05:20:29.469 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:29.469 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:29.469 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:29.469 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:20:29.470 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:29.470 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:29.470 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:29.470 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:20:29.471 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:29.471 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:29.471 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182423.238098_42.1", 2022-09-03T05:20:29.471 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:20:23.238098Z" 2022-09-03T05:20:29.472 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:29.472 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:20:29.473 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182423.238098_42.1'} 2022-09-03T05:20:29.474 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:20:29.474 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:29.475 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:29.476 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:29.527 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:29.528 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:29.554 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:29.556+0000 7f866e9fc600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:29.555 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:29.557+0000 7f866e9fc600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:29.569 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:20:29.570 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:20:29.570 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:20:29.571 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:20:29.571 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:20:29.571 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:29.571 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:29.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:29.572 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:20:29.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:29.572 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:20:29.573 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:29.573 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:29.573 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:29.573 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:29.574 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:20:29.574 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:29.574 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:29.574 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:29.575 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:29.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:29.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:29.575 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:20:29.576 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:29.576 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:29.576 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:29.577 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:29.577 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:29.577 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:29.578 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:29.578 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:29.578 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:29.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:29.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:29.579 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:20:29.579 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:29.579 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:29.580 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:29.580 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:29.580 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:29.580 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:29.581 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:29.581 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:29.581 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:29.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:29.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:29.582 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:20:29.582 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:29.583 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:29.583 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182423.238098_42.1", 2022-09-03T05:20:29.583 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:29.583 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:29.584 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:29.584 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.238098Z", 2022-09-03T05:20:29.584 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:29.584 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:29.585 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:29.585 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:20:29.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:29.585 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:20:29.586 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:20:29.586 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:20:29.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:29.586 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:20:29Z" 2022-09-03T05:20:29.587 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:20:29.588 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182423.238098_42.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:23.238098Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:20:29Z" } 2022-09-03T05:20:29.589 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182423.238098_42.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:23.238098Z', 'realm_epoch': 2}}] 2022-09-03T05:20:29.589 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182423.238098_42.1'} 2022-09-03T05:20:29.590 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182423.238098_42.1'} 2022-09-03T05:20:29.590 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:20:29.591 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:20:29.591 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:29.592 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:29.593 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:29.642 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:29.643 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:29.691 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:20:29.694+0000 7fb417b3f600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:29.691 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:20:29.694+0000 7fb417b3f600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:29.712 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:20:29.713 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:20:29.713 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:20:29.714 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:20:29.714 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:20:29.715 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:29.715 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:29.716 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:29.716 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:20:29.717 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:29.718 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:20:29.718 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:29.719 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:29.719 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:29.720 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:29.720 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:20:29.721 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:29.721 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:29.722 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:29.722 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:29.723 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:29.724 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:29.724 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:20:29.725 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:29.725 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:29.726 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:29.726 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:29.727 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:29.727 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:29.728 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:20:29.728 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:29.729 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:29.729 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:29.730 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:29.730 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:20:29.731 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:29.731 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:29.731 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:29.732 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:29.732 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:29.732 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:29.732 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:20:29.733 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:29.733 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:29.733 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:29.733 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:29.734 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:20:29.734 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:29.734 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:29.735 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182423.238098_42.1", 2022-09-03T05:20:29.735 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:29.735 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:29.735 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:29.736 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.238098Z", 2022-09-03T05:20:29.736 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:29.737 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:29.737 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:29.738 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:20:29.738 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:29.739 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:20:29.739 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:20:29.740 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:20:29.740 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:29.741 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:20:29Z" 2022-09-03T05:20:29.741 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:20:29.743 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182423.238098_42.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:23.238098Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:20:29Z" } 2022-09-03T05:20:29.744 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182423.238098_42.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:23.238098Z', 'realm_epoch': 2}}] 2022-09-03T05:20:29.745 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182423.238098_42.1'} 2022-09-03T05:20:29.746 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182423.238098_42.1'} 2022-09-03T05:20:29.747 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:20:29.748 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:20:29.749 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:29.750 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:29.751 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:29.802 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:29.803 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:29.838 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:29.840+0000 7eff510ee600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:29.839 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:29.841+0000 7eff510ee600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:29.890 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:20:29.891 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:20:29.892 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:20:29.892 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:20:29.893 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:20:29.893 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:29.893 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:29.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:29.894 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:20:29.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:29.895 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:20:29.895 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:29.896 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:29.896 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:29.896 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:29.897 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:20:29.897 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:29.897 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:29.897 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:29.898 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:29.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:29.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:29.900 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:20:29.901 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:29.901 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:29.902 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:29.902 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:29.903 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:29.903 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:29.903 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:29.904 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:29.904 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:29.904 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:29.904 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:29.905 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:20:29.905 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:29.905 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:29.905 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:29.905 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:29.906 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:29.906 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:29.906 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:29.907 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:29.907 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:29.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:29.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:29.909 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:20:29.909 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:29.910 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:29.910 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182423.238098_42.1", 2022-09-03T05:20:29.911 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:29.911 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:29.911 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:29.912 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.238098Z", 2022-09-03T05:20:29.912 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:29.913 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:29.913 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:29.913 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:20:29.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:29.914 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:20:29.915 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:20:29.915 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:20:29.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:29.916 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:20:29Z" 2022-09-03T05:20:29.916 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:20:29.916 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182423.238098_42.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:23.238098Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:20:29Z" } 2022-09-03T05:20:29.917 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182423.238098_42.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:23.238098Z', 'realm_epoch': 2}}] 2022-09-03T05:20:29.918 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182423.238098_42.1'} 2022-09-03T05:20:29.919 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182423.238098_42.1'} 2022-09-03T05:20:29.920 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:20:29.921 DEBUG:boto:path=/dsgvcy-4/ 2022-09-03T05:20:29.922 DEBUG:boto:auth_path=/dsgvcy-4/ 2022-09-03T05:20:29.922 DEBUG:boto:Method: HEAD 2022-09-03T05:20:29.924 DEBUG:boto:Path: /dsgvcy-4/ 2022-09-03T05:20:29.924 DEBUG:boto:Data: 2022-09-03T05:20:29.925 DEBUG:boto:Headers: {} 2022-09-03T05:20:29.926 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:29.927 DEBUG:boto:Port: 8000 2022-09-03T05:20:29.927 DEBUG:boto:Params: {} 2022-09-03T05:20:29.929 DEBUG:boto:Token: None 2022-09-03T05:20:29.931 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:29 GMT /dsgvcy-4/ 2022-09-03T05:20:29.932 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ekF3j7VNknhmIt8ya76TfwFNWZE= 2022-09-03T05:20:29.933 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ekF3j7VNknhmIt8ya76TfwFNWZE=', 'Content-Length': '0'} 2022-09-03T05:20:29.937 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000cd7de07dd046c1b1-006312e41d-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:29.938 DEBUG:boto:path=/dsgvcy-5/ 2022-09-03T05:20:29.939 DEBUG:boto:auth_path=/dsgvcy-5/ 2022-09-03T05:20:29.939 DEBUG:boto:Method: HEAD 2022-09-03T05:20:29.940 DEBUG:boto:Path: /dsgvcy-5/ 2022-09-03T05:20:29.941 DEBUG:boto:Data: 2022-09-03T05:20:29.942 DEBUG:boto:Headers: {} 2022-09-03T05:20:29.943 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:29.944 DEBUG:boto:Port: 8000 2022-09-03T05:20:29.945 DEBUG:boto:Params: {} 2022-09-03T05:20:29.946 DEBUG:boto:Token: None 2022-09-03T05:20:29.947 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:29 GMT /dsgvcy-5/ 2022-09-03T05:20:29.947 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SIOOWzleHeFaT1OJ0ri5ejaKHcY= 2022-09-03T05:20:29.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SIOOWzleHeFaT1OJ0ri5ejaKHcY=', 'Content-Length': '0'} 2022-09-03T05:20:29.950 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006be579ae1d9aed36-006312e41d-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:29.951 DEBUG:boto:path=/dsgvcy-6/ 2022-09-03T05:20:29.952 DEBUG:boto:auth_path=/dsgvcy-6/ 2022-09-03T05:20:29.952 DEBUG:boto:Method: HEAD 2022-09-03T05:20:29.953 DEBUG:boto:Path: /dsgvcy-6/ 2022-09-03T05:20:29.953 DEBUG:boto:Data: 2022-09-03T05:20:29.954 DEBUG:boto:Headers: {} 2022-09-03T05:20:29.954 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:29.955 DEBUG:boto:Port: 8000 2022-09-03T05:20:29.955 DEBUG:boto:Params: {} 2022-09-03T05:20:29.956 DEBUG:boto:Token: None 2022-09-03T05:20:29.957 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:29 GMT /dsgvcy-6/ 2022-09-03T05:20:29.957 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xOi5YSxK4KizpuQ4tVOlN1m6j7U= 2022-09-03T05:20:29.958 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xOi5YSxK4KizpuQ4tVOlN1m6j7U=', 'Content-Length': '0'} 2022-09-03T05:20:29.960 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000108581b517f76fea-006312e41d-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:29.961 DEBUG:boto:path=/dsgvcy-4/ 2022-09-03T05:20:29.962 DEBUG:boto:auth_path=/dsgvcy-4/ 2022-09-03T05:20:29.962 DEBUG:boto:Method: HEAD 2022-09-03T05:20:29.963 DEBUG:boto:Path: /dsgvcy-4/ 2022-09-03T05:20:29.963 DEBUG:boto:Data: 2022-09-03T05:20:29.964 DEBUG:boto:Headers: {} 2022-09-03T05:20:29.964 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:29.965 DEBUG:boto:Port: 8000 2022-09-03T05:20:29.965 DEBUG:boto:Params: {} 2022-09-03T05:20:29.966 DEBUG:boto:Token: None 2022-09-03T05:20:29.967 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:29 GMT /dsgvcy-4/ 2022-09-03T05:20:29.967 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ekF3j7VNknhmIt8ya76TfwFNWZE= 2022-09-03T05:20:29.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ekF3j7VNknhmIt8ya76TfwFNWZE=', 'Content-Length': '0'} 2022-09-03T05:20:29.974 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001bfb06970aef8aff-006312e41d-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:29.975 DEBUG:boto:path=/dsgvcy-5/ 2022-09-03T05:20:29.975 DEBUG:boto:auth_path=/dsgvcy-5/ 2022-09-03T05:20:29.976 DEBUG:boto:Method: HEAD 2022-09-03T05:20:29.976 DEBUG:boto:Path: /dsgvcy-5/ 2022-09-03T05:20:29.977 DEBUG:boto:Data: 2022-09-03T05:20:29.977 DEBUG:boto:Headers: {} 2022-09-03T05:20:29.978 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:29.979 DEBUG:boto:Port: 8000 2022-09-03T05:20:29.979 DEBUG:boto:Params: {} 2022-09-03T05:20:29.980 DEBUG:boto:Token: None 2022-09-03T05:20:29.980 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:29 GMT /dsgvcy-5/ 2022-09-03T05:20:29.981 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SIOOWzleHeFaT1OJ0ri5ejaKHcY= 2022-09-03T05:20:29.981 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SIOOWzleHeFaT1OJ0ri5ejaKHcY=', 'Content-Length': '0'} 2022-09-03T05:20:29.984 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002d0b8deaeef6f86e-006312e41d-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:29.985 DEBUG:boto:path=/dsgvcy-6/ 2022-09-03T05:20:29.986 DEBUG:boto:auth_path=/dsgvcy-6/ 2022-09-03T05:20:29.988 DEBUG:boto:Method: HEAD 2022-09-03T05:20:29.989 DEBUG:boto:Path: /dsgvcy-6/ 2022-09-03T05:20:29.990 DEBUG:boto:Data: 2022-09-03T05:20:29.991 DEBUG:boto:Headers: {} 2022-09-03T05:20:29.992 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:29.993 DEBUG:boto:Port: 8000 2022-09-03T05:20:29.994 DEBUG:boto:Params: {} 2022-09-03T05:20:29.995 DEBUG:boto:Token: None 2022-09-03T05:20:29.996 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:29 GMT /dsgvcy-6/ 2022-09-03T05:20:29.997 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xOi5YSxK4KizpuQ4tVOlN1m6j7U= 2022-09-03T05:20:29.998 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xOi5YSxK4KizpuQ4tVOlN1m6j7U=', 'Content-Length': '0'} 2022-09-03T05:20:30.004 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005235a8193888fecb-006312e41e-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:30.005 DEBUG:boto:path=/dsgvcy-4/ 2022-09-03T05:20:30.008 DEBUG:boto:auth_path=/dsgvcy-4/ 2022-09-03T05:20:30.009 DEBUG:boto:Method: HEAD 2022-09-03T05:20:30.009 DEBUG:boto:Path: /dsgvcy-4/ 2022-09-03T05:20:30.010 DEBUG:boto:Data: 2022-09-03T05:20:30.011 DEBUG:boto:Headers: {} 2022-09-03T05:20:30.011 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:30.012 DEBUG:boto:Port: 8001 2022-09-03T05:20:30.012 DEBUG:boto:Params: {} 2022-09-03T05:20:30.013 DEBUG:boto:Token: None 2022-09-03T05:20:30.013 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:30 GMT /dsgvcy-4/ 2022-09-03T05:20:30.014 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aTks2z1nM4RWMHqcIlDpxX4NnEM= 2022-09-03T05:20:30.015 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aTks2z1nM4RWMHqcIlDpxX4NnEM=', 'Content-Length': '0'} 2022-09-03T05:20:30.017 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000095a8dbf9ab62e952-006312e41e-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:30.018 DEBUG:boto:path=/dsgvcy-5/ 2022-09-03T05:20:30.019 DEBUG:boto:auth_path=/dsgvcy-5/ 2022-09-03T05:20:30.019 DEBUG:boto:Method: HEAD 2022-09-03T05:20:30.020 DEBUG:boto:Path: /dsgvcy-5/ 2022-09-03T05:20:30.020 DEBUG:boto:Data: 2022-09-03T05:20:30.021 DEBUG:boto:Headers: {} 2022-09-03T05:20:30.021 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:30.022 DEBUG:boto:Port: 8001 2022-09-03T05:20:30.023 DEBUG:boto:Params: {} 2022-09-03T05:20:30.023 DEBUG:boto:Token: None 2022-09-03T05:20:30.024 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:30 GMT /dsgvcy-5/ 2022-09-03T05:20:30.024 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t+byaxtx7SilxYT02b8Xxim7OPI= 2022-09-03T05:20:30.025 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t+byaxtx7SilxYT02b8Xxim7OPI=', 'Content-Length': '0'} 2022-09-03T05:20:30.027 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f05f9325e3f0c3ae-006312e41e-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:30.028 DEBUG:boto:path=/dsgvcy-6/ 2022-09-03T05:20:30.029 DEBUG:boto:auth_path=/dsgvcy-6/ 2022-09-03T05:20:30.029 DEBUG:boto:Method: HEAD 2022-09-03T05:20:30.030 DEBUG:boto:Path: /dsgvcy-6/ 2022-09-03T05:20:30.030 DEBUG:boto:Data: 2022-09-03T05:20:30.031 DEBUG:boto:Headers: {} 2022-09-03T05:20:30.031 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:30.032 DEBUG:boto:Port: 8001 2022-09-03T05:20:30.032 DEBUG:boto:Params: {} 2022-09-03T05:20:30.034 DEBUG:boto:Token: None 2022-09-03T05:20:30.035 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:30 GMT /dsgvcy-6/ 2022-09-03T05:20:30.036 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KERSZwn54Ap9mHJ0nfqUBGRdmGE= 2022-09-03T05:20:30.037 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KERSZwn54Ap9mHJ0nfqUBGRdmGE=', 'Content-Length': '0'} 2022-09-03T05:20:30.039 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ff4ed1ef223a15bf-006312e41e-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:30.041 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_recreate ... ok 2022-09-03T05:20:30.042 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-7 2022-09-03T05:20:30.043 DEBUG:boto:path=/dsgvcy-7/ 2022-09-03T05:20:30.045 DEBUG:boto:auth_path=/dsgvcy-7/ 2022-09-03T05:20:30.045 DEBUG:boto:Method: PUT 2022-09-03T05:20:30.046 DEBUG:boto:Path: /dsgvcy-7/ 2022-09-03T05:20:30.047 DEBUG:boto:Data: 2022-09-03T05:20:30.048 DEBUG:boto:Headers: {} 2022-09-03T05:20:30.048 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:30.049 DEBUG:boto:Port: 8000 2022-09-03T05:20:30.049 DEBUG:boto:Params: {} 2022-09-03T05:20:30.050 DEBUG:boto:Token: None 2022-09-03T05:20:30.050 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:20:30 GMT /dsgvcy-7/ 2022-09-03T05:20:30.051 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YwpTDvGOPDzPl7GjKVWjPq8qVcA= 2022-09-03T05:20:30.051 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YwpTDvGOPDzPl7GjKVWjPq8qVcA=', 'Content-Length': '0'} 2022-09-03T05:20:30.076 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000a26c08aabbac7493-006312e41e-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:30.077 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=dsgvcy-8 2022-09-03T05:20:30.078 DEBUG:boto:path=/dsgvcy-8/ 2022-09-03T05:20:30.079 DEBUG:boto:auth_path=/dsgvcy-8/ 2022-09-03T05:20:30.080 DEBUG:boto:Method: PUT 2022-09-03T05:20:30.081 DEBUG:boto:Path: /dsgvcy-8/ 2022-09-03T05:20:30.082 DEBUG:boto:Data: 2022-09-03T05:20:30.082 DEBUG:boto:Headers: {} 2022-09-03T05:20:30.083 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:30.084 DEBUG:boto:Port: 8000 2022-09-03T05:20:30.084 DEBUG:boto:Params: {} 2022-09-03T05:20:30.085 DEBUG:boto:Token: None 2022-09-03T05:20:30.086 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:20:30 GMT /dsgvcy-8/ 2022-09-03T05:20:30.087 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lJCwGJlmy/q4kVN9BWwqmKbouRc= 2022-09-03T05:20:30.089 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lJCwGJlmy/q4kVN9BWwqmKbouRc=', 'Content-Length': '0'} 2022-09-03T05:20:30.116 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000ed087050d5edafaf-006312e41e-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:30.117 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=dsgvcy-9 2022-09-03T05:20:30.117 DEBUG:boto:path=/dsgvcy-9/ 2022-09-03T05:20:30.118 DEBUG:boto:auth_path=/dsgvcy-9/ 2022-09-03T05:20:30.118 DEBUG:boto:Method: PUT 2022-09-03T05:20:30.119 DEBUG:boto:Path: /dsgvcy-9/ 2022-09-03T05:20:30.119 DEBUG:boto:Data: 2022-09-03T05:20:30.120 DEBUG:boto:Headers: {} 2022-09-03T05:20:30.120 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:30.121 DEBUG:boto:Port: 8001 2022-09-03T05:20:30.122 DEBUG:boto:Params: {} 2022-09-03T05:20:30.123 DEBUG:boto:Token: None 2022-09-03T05:20:30.124 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:20:30 GMT /dsgvcy-9/ 2022-09-03T05:20:30.124 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zs4PzGbaWGYeq7VLCUwv8UcSeVM= 2022-09-03T05:20:30.125 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zs4PzGbaWGYeq7VLCUwv8UcSeVM=', 'Content-Length': '0'} 2022-09-03T05:20:30.147 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000cce61ef8fc61c1a3-006312e41e-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:30.148 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:30.149 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:30.150 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:30.198 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:30.199 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:30.236 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=41ecd2a5-afc0-4eae-9349-7dfe39baa1cb 2022-09-03T05:20:30.256 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:20:30.257 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:30.257 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:30.257 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:18:18.906852Z" 2022-09-03T05:20:30.258 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:30.258 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:30.258 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:30.259 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:20:30.259 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:30.259 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:30.259 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:30.260 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:20:30.260 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:30.260 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:30.261 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.141916_54.1", 2022-09-03T05:20:30.261 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:20:30.141916Z" 2022-09-03T05:20:30.261 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:30.262 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:20:30.263 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182430.141916_54.1'} 2022-09-03T05:20:30.264 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:20:30.265 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:30.265 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:30.267 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:30.319 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:30.320 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:30.355 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:30.356+0000 7f8e71792600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:30.356 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:30.357+0000 7f8e71792600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:30.366 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:20:30.367 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:20:30.368 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:20:30.368 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:20:30.369 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:20:30.369 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:30.369 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:30.370 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:30.370 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:20:30.370 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:30.370 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:20:30.371 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:30.371 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:30.371 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:30.372 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:30.372 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:20:30.373 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:30.373 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:30.373 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:30.374 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:30.374 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:30.374 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:30.374 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:20:30.375 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:30.375 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:30.375 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:30.376 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:30.376 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:30.376 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:30.376 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:30.377 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:30.377 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:30.377 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:30.377 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:30.378 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:20:30.378 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:30.378 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:30.378 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:30.379 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:30.379 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:30.379 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:30.380 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:30.380 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:30.380 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:30.380 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:30.381 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:30.381 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:20:30.381 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:30.381 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:30.382 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.141916_54.1", 2022-09-03T05:20:30.382 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:30.382 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:30.382 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:30.383 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.141916Z", 2022-09-03T05:20:30.383 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:30.383 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:30.384 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:30.384 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:20:30.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:30.384 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:20:30.385 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:20:30.385 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:20:30.385 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:30.385 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:20:30Z" 2022-09-03T05:20:30.386 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:20:30.387 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.141916_54.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.141916Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:20:30Z" } 2022-09-03T05:20:30.389 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.141916_54.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.141916Z', 'realm_epoch': 2}}] 2022-09-03T05:20:30.389 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182430.141916_54.1'} 2022-09-03T05:20:30.390 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182430.141916_54.1'} 2022-09-03T05:20:30.391 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:20:30.392 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:20:30.393 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:30.393 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:30.394 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:30.447 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:30.448 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:30.491 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:20:30.493+0000 7fc89da5d600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:30.492 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:20:30.494+0000 7fc89da5d600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:30.507 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:20:30.507 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:20:30.508 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:20:30.508 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:20:30.508 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:20:30.508 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:30.509 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:30.509 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:30.509 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:20:30.510 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:30.510 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:20:30.510 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:30.510 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:30.511 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:30.511 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:30.512 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:20:30.512 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:30.512 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:30.512 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:30.513 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:30.513 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:30.513 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:30.513 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:20:30.514 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:30.514 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:30.515 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:30.515 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:30.516 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:30.516 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:30.517 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:20:30.517 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:30.518 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:30.518 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:30.519 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:30.519 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:20:30.520 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:30.520 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:30.520 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:30.521 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:30.521 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:30.522 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:30.522 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:20:30.523 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:30.523 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:30.524 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:30.525 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:30.525 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:20:30.525 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:30.526 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:30.526 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.141916_54.1", 2022-09-03T05:20:30.527 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:30.527 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:30.528 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:30.528 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.141916Z", 2022-09-03T05:20:30.529 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:30.529 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:30.529 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:30.530 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:20:30.530 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:30.530 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:20:30.530 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:20:30.531 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:20:30.531 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:30.532 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:20:30Z" 2022-09-03T05:20:30.532 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:20:30.534 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.141916_54.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.141916Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:20:30Z" } 2022-09-03T05:20:30.535 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.141916_54.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.141916Z', 'realm_epoch': 2}}] 2022-09-03T05:20:30.536 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182430.141916_54.1'} 2022-09-03T05:20:30.537 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182430.141916_54.1'} 2022-09-03T05:20:30.537 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:20:30.538 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:20:30.539 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:30.539 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:30.540 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:30.592 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:30.593 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:30.619 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:30.620+0000 7f33fab65600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:30.619 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:30.621+0000 7f33fab65600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:30.630 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:20:30.630 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:20:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:20:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:20:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:20:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:20:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:20:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:20:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:20:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:20:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:20:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.141916_54.1", 2022-09-03T05:20:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.141916Z", 2022-09-03T05:20:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:20:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:20:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:20:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:20:30.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:30.653 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:20:30Z" 2022-09-03T05:20:30.653 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:20:30.655 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.141916_54.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.141916Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:20:30Z" } 2022-09-03T05:20:30.655 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.141916_54.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.141916Z', 'realm_epoch': 2}}] 2022-09-03T05:20:30.656 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182430.141916_54.1'} 2022-09-03T05:20:30.656 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182430.141916_54.1'} 2022-09-03T05:20:30.657 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:20:30.657 DEBUG:boto:path=/dsgvcy-7/ 2022-09-03T05:20:30.658 DEBUG:boto:auth_path=/dsgvcy-7/ 2022-09-03T05:20:30.658 DEBUG:boto:Method: HEAD 2022-09-03T05:20:30.659 DEBUG:boto:Path: /dsgvcy-7/ 2022-09-03T05:20:30.659 DEBUG:boto:Data: 2022-09-03T05:20:30.660 DEBUG:boto:Headers: {} 2022-09-03T05:20:30.660 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:30.661 DEBUG:boto:Port: 8000 2022-09-03T05:20:30.661 DEBUG:boto:Params: {} 2022-09-03T05:20:30.662 DEBUG:boto:Token: None 2022-09-03T05:20:30.663 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:30 GMT /dsgvcy-7/ 2022-09-03T05:20:30.663 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6Oj4+H2/I3iunBOeRkvwIxRbHP4= 2022-09-03T05:20:30.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6Oj4+H2/I3iunBOeRkvwIxRbHP4=', 'Content-Length': '0'} 2022-09-03T05:20:30.666 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004e1c1eb177d70a86-006312e41e-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:30.667 DEBUG:boto:path=/dsgvcy-8/ 2022-09-03T05:20:30.668 DEBUG:boto:auth_path=/dsgvcy-8/ 2022-09-03T05:20:30.668 DEBUG:boto:Method: HEAD 2022-09-03T05:20:30.669 DEBUG:boto:Path: /dsgvcy-8/ 2022-09-03T05:20:30.669 DEBUG:boto:Data: 2022-09-03T05:20:30.670 DEBUG:boto:Headers: {} 2022-09-03T05:20:30.670 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:30.671 DEBUG:boto:Port: 8000 2022-09-03T05:20:30.671 DEBUG:boto:Params: {} 2022-09-03T05:20:30.672 DEBUG:boto:Token: None 2022-09-03T05:20:30.673 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:30 GMT /dsgvcy-8/ 2022-09-03T05:20:30.673 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bArfq3deKvvSHgE01VJdoGqVxNg= 2022-09-03T05:20:30.674 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bArfq3deKvvSHgE01VJdoGqVxNg=', 'Content-Length': '0'} 2022-09-03T05:20:30.677 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e1c37090451bf355-006312e41e-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:30.678 DEBUG:boto:path=/dsgvcy-9/ 2022-09-03T05:20:30.679 DEBUG:boto:auth_path=/dsgvcy-9/ 2022-09-03T05:20:30.680 DEBUG:boto:Method: HEAD 2022-09-03T05:20:30.681 DEBUG:boto:Path: /dsgvcy-9/ 2022-09-03T05:20:30.682 DEBUG:boto:Data: 2022-09-03T05:20:30.683 DEBUG:boto:Headers: {} 2022-09-03T05:20:30.683 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:30.684 DEBUG:boto:Port: 8000 2022-09-03T05:20:30.684 DEBUG:boto:Params: {} 2022-09-03T05:20:30.685 DEBUG:boto:Token: None 2022-09-03T05:20:30.685 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:30 GMT /dsgvcy-9/ 2022-09-03T05:20:30.687 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SVqvWzZWJ89ZiI35tc7Y1uf66ps= 2022-09-03T05:20:30.688 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SVqvWzZWJ89ZiI35tc7Y1uf66ps=', 'Content-Length': '0'} 2022-09-03T05:20:30.691 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003d5a2968df417594-006312e41e-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:30.692 DEBUG:boto:path=/dsgvcy-7/ 2022-09-03T05:20:30.693 DEBUG:boto:auth_path=/dsgvcy-7/ 2022-09-03T05:20:30.694 DEBUG:boto:Method: HEAD 2022-09-03T05:20:30.695 DEBUG:boto:Path: /dsgvcy-7/ 2022-09-03T05:20:30.696 DEBUG:boto:Data: 2022-09-03T05:20:30.697 DEBUG:boto:Headers: {} 2022-09-03T05:20:30.698 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:30.699 DEBUG:boto:Port: 8000 2022-09-03T05:20:30.699 DEBUG:boto:Params: {} 2022-09-03T05:20:30.700 DEBUG:boto:Token: None 2022-09-03T05:20:30.701 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:30 GMT /dsgvcy-7/ 2022-09-03T05:20:30.701 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6Oj4+H2/I3iunBOeRkvwIxRbHP4= 2022-09-03T05:20:30.702 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6Oj4+H2/I3iunBOeRkvwIxRbHP4=', 'Content-Length': '0'} 2022-09-03T05:20:30.704 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003722adfc2f0f3c57-006312e41e-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:30.705 DEBUG:boto:path=/dsgvcy-8/ 2022-09-03T05:20:30.706 DEBUG:boto:auth_path=/dsgvcy-8/ 2022-09-03T05:20:30.706 DEBUG:boto:Method: HEAD 2022-09-03T05:20:30.707 DEBUG:boto:Path: /dsgvcy-8/ 2022-09-03T05:20:30.707 DEBUG:boto:Data: 2022-09-03T05:20:30.708 DEBUG:boto:Headers: {} 2022-09-03T05:20:30.708 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:30.709 DEBUG:boto:Port: 8000 2022-09-03T05:20:30.709 DEBUG:boto:Params: {} 2022-09-03T05:20:30.710 DEBUG:boto:Token: None 2022-09-03T05:20:30.710 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:30 GMT /dsgvcy-8/ 2022-09-03T05:20:30.711 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bArfq3deKvvSHgE01VJdoGqVxNg= 2022-09-03T05:20:30.711 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bArfq3deKvvSHgE01VJdoGqVxNg=', 'Content-Length': '0'} 2022-09-03T05:20:30.714 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000060c70ae954200ddc-006312e41e-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:30.714 DEBUG:boto:path=/dsgvcy-9/ 2022-09-03T05:20:30.715 DEBUG:boto:auth_path=/dsgvcy-9/ 2022-09-03T05:20:30.715 DEBUG:boto:Method: HEAD 2022-09-03T05:20:30.716 DEBUG:boto:Path: /dsgvcy-9/ 2022-09-03T05:20:30.716 DEBUG:boto:Data: 2022-09-03T05:20:30.717 DEBUG:boto:Headers: {} 2022-09-03T05:20:30.717 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:30.718 DEBUG:boto:Port: 8000 2022-09-03T05:20:30.718 DEBUG:boto:Params: {} 2022-09-03T05:20:30.719 DEBUG:boto:Token: None 2022-09-03T05:20:30.719 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:30 GMT /dsgvcy-9/ 2022-09-03T05:20:30.720 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SVqvWzZWJ89ZiI35tc7Y1uf66ps= 2022-09-03T05:20:30.721 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SVqvWzZWJ89ZiI35tc7Y1uf66ps=', 'Content-Length': '0'} 2022-09-03T05:20:30.723 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c3b82fd253b9a1f2-006312e41e-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:30.724 DEBUG:boto:path=/dsgvcy-7/ 2022-09-03T05:20:30.724 DEBUG:boto:auth_path=/dsgvcy-7/ 2022-09-03T05:20:30.725 DEBUG:boto:Method: HEAD 2022-09-03T05:20:30.725 DEBUG:boto:Path: /dsgvcy-7/ 2022-09-03T05:20:30.726 DEBUG:boto:Data: 2022-09-03T05:20:30.727 DEBUG:boto:Headers: {} 2022-09-03T05:20:30.728 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:30.728 DEBUG:boto:Port: 8001 2022-09-03T05:20:30.729 DEBUG:boto:Params: {} 2022-09-03T05:20:30.730 DEBUG:boto:Token: None 2022-09-03T05:20:30.731 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:30 GMT /dsgvcy-7/ 2022-09-03T05:20:30.732 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6Oj4+H2/I3iunBOeRkvwIxRbHP4= 2022-09-03T05:20:30.733 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6Oj4+H2/I3iunBOeRkvwIxRbHP4=', 'Content-Length': '0'} 2022-09-03T05:20:30.738 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000043a8127090063203-006312e41e-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:30.739 DEBUG:boto:path=/dsgvcy-8/ 2022-09-03T05:20:30.739 DEBUG:boto:auth_path=/dsgvcy-8/ 2022-09-03T05:20:30.740 DEBUG:boto:Method: HEAD 2022-09-03T05:20:30.740 DEBUG:boto:Path: /dsgvcy-8/ 2022-09-03T05:20:30.741 DEBUG:boto:Data: 2022-09-03T05:20:30.741 DEBUG:boto:Headers: {} 2022-09-03T05:20:30.742 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:30.742 DEBUG:boto:Port: 8001 2022-09-03T05:20:30.743 DEBUG:boto:Params: {} 2022-09-03T05:20:30.743 DEBUG:boto:Token: None 2022-09-03T05:20:30.744 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:30 GMT /dsgvcy-8/ 2022-09-03T05:20:30.745 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bArfq3deKvvSHgE01VJdoGqVxNg= 2022-09-03T05:20:30.745 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bArfq3deKvvSHgE01VJdoGqVxNg=', 'Content-Length': '0'} 2022-09-03T05:20:30.747 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000085069310d12a4842-006312e41e-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:30.748 DEBUG:boto:path=/dsgvcy-9/ 2022-09-03T05:20:30.749 DEBUG:boto:auth_path=/dsgvcy-9/ 2022-09-03T05:20:30.750 DEBUG:boto:Method: HEAD 2022-09-03T05:20:30.750 DEBUG:boto:Path: /dsgvcy-9/ 2022-09-03T05:20:30.751 DEBUG:boto:Data: 2022-09-03T05:20:30.752 DEBUG:boto:Headers: {} 2022-09-03T05:20:30.752 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:30.753 DEBUG:boto:Port: 8001 2022-09-03T05:20:30.753 DEBUG:boto:Params: {} 2022-09-03T05:20:30.754 DEBUG:boto:Token: None 2022-09-03T05:20:30.754 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:30 GMT /dsgvcy-9/ 2022-09-03T05:20:30.755 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SVqvWzZWJ89ZiI35tc7Y1uf66ps= 2022-09-03T05:20:30.755 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SVqvWzZWJ89ZiI35tc7Y1uf66ps=', 'Content-Length': '0'} 2022-09-03T05:20:30.758 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007d00b88300a22754-006312e41e-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:30.759 DEBUG:boto:path=/dsgvcy-7/ 2022-09-03T05:20:30.759 DEBUG:boto:auth_path=/dsgvcy-7/ 2022-09-03T05:20:30.760 DEBUG:boto:Method: DELETE 2022-09-03T05:20:30.760 DEBUG:boto:Path: /dsgvcy-7/ 2022-09-03T05:20:30.761 DEBUG:boto:Data: 2022-09-03T05:20:30.761 DEBUG:boto:Headers: {} 2022-09-03T05:20:30.762 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:30.762 DEBUG:boto:Port: 8000 2022-09-03T05:20:30.763 DEBUG:boto:Params: {} 2022-09-03T05:20:30.763 DEBUG:boto:Token: None 2022-09-03T05:20:30.764 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:20:30 GMT /dsgvcy-7/ 2022-09-03T05:20:30.765 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MBkMu2gkoG++KoPsuiAK6Zd2OLw= 2022-09-03T05:20:30.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MBkMu2gkoG++KoPsuiAK6Zd2OLw=', 'Content-Length': '0'} 2022-09-03T05:20:30.799 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000087cde12d1bbf74d9-006312e41e-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:20:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:30.799 DEBUG:boto:path=/dsgvcy-8/ 2022-09-03T05:20:30.800 DEBUG:boto:auth_path=/dsgvcy-8/ 2022-09-03T05:20:30.801 DEBUG:boto:Method: DELETE 2022-09-03T05:20:30.802 DEBUG:boto:Path: /dsgvcy-8/ 2022-09-03T05:20:30.802 DEBUG:boto:Data: 2022-09-03T05:20:30.803 DEBUG:boto:Headers: {} 2022-09-03T05:20:30.804 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:30.804 DEBUG:boto:Port: 8000 2022-09-03T05:20:30.805 DEBUG:boto:Params: {} 2022-09-03T05:20:30.806 DEBUG:boto:Token: None 2022-09-03T05:20:30.807 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:20:30 GMT /dsgvcy-8/ 2022-09-03T05:20:30.808 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MdYtH4cMeFFf0F8EEZxNQzWvS20= 2022-09-03T05:20:30.809 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MdYtH4cMeFFf0F8EEZxNQzWvS20=', 'Content-Length': '0'} 2022-09-03T05:20:30.880 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000012bb9df06c0e0de2-006312e41e-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:20:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:30.881 DEBUG:boto:path=/dsgvcy-9/ 2022-09-03T05:20:30.882 DEBUG:boto:auth_path=/dsgvcy-9/ 2022-09-03T05:20:30.882 DEBUG:boto:Method: DELETE 2022-09-03T05:20:30.883 DEBUG:boto:Path: /dsgvcy-9/ 2022-09-03T05:20:30.884 DEBUG:boto:Data: 2022-09-03T05:20:30.884 DEBUG:boto:Headers: {} 2022-09-03T05:20:30.885 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:30.885 DEBUG:boto:Port: 8001 2022-09-03T05:20:30.886 DEBUG:boto:Params: {} 2022-09-03T05:20:30.886 DEBUG:boto:Token: None 2022-09-03T05:20:30.887 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:20:30 GMT /dsgvcy-9/ 2022-09-03T05:20:30.887 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s0y1i+C8dPSc4EMCp5Wg27JH/EA= 2022-09-03T05:20:30.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s0y1i+C8dPSc4EMCp5Wg27JH/EA=', 'Content-Length': '0'} 2022-09-03T05:20:30.960 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000003fc35ad465f3c3a0-006312e41e-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:20:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:30.961 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:30.962 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:30.963 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:31.013 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:31.013 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:31.053 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=41ecd2a5-afc0-4eae-9349-7dfe39baa1cb 2022-09-03T05:20:31.070 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:20:31.071 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:31.071 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:31.071 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:18:18.906852Z" 2022-09-03T05:20:31.072 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:31.072 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:31.072 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:31.072 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:20:31.073 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:31.073 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:31.073 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:31.074 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:20:31.074 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:31.074 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:31.074 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:20:31.075 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:20:30.928498Z" 2022-09-03T05:20:31.075 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:31.075 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:20:31.077 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:20:31.078 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:20:31.079 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:31.080 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:31.081 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:31.132 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:31.132 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:31.161 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:31.163+0000 7f48e051a600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:31.162 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:31.164+0000 7f48e051a600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:31.175 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:20:31.176 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:20:31.176 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:20:31.176 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:20:31.177 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:20:31.177 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:31.177 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:31.177 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:31.178 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:20:31.178 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:31.178 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:20:31.179 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:31.179 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:31.179 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:31.179 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:31.180 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:20:31.180 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:31.180 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:31.180 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:31.181 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:31.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:31.181 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:31.181 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:20:31.182 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:31.182 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:31.182 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:31.183 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:31.183 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:31.184 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:31.184 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:31.184 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:31.184 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:31.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:31.185 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:31.185 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:20:31.185 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:31.186 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:31.186 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:31.186 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:31.187 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:31.187 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:31.187 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:31.187 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:31.188 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:31.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:31.188 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:31.188 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:20:31.189 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:31.189 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:31.189 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.782802_55.1", 2022-09-03T05:20:31.189 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:31.189 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:31.190 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:31.190 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.782802Z", 2022-09-03T05:20:31.190 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:31.191 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:31.191 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:31.191 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:20:31.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:31.192 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:20:31.192 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:20:31.192 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:20:31.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:31.193 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:20:31Z" 2022-09-03T05:20:31.193 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:20:31.195 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.782802_55.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.782802Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:20:31Z" } 2022-09-03T05:20:31.195 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.782802_55.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.782802Z', 'realm_epoch': 2}}] 2022-09-03T05:20:31.196 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:20:31.196 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182430.782802_55.1'} 2022-09-03T05:20:31.197 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=3 master=1_1662182430.928498_66.1 target=1_1662182430.782802_55.1 2022-09-03T05:20:36.198 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:36.199 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:36.200 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:36.257 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:36.258 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:36.287 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:36.289+0000 7f742b8a5600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:36.288 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:36.291+0000 7f742b8a5600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:36.299 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:20:36.300 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:20:36.300 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:20:36.300 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:20:36.301 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:20:36.301 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:36.301 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:36.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:36.302 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:20:36.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:36.303 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:20:36.303 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:36.303 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:36.304 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:36.304 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:36.304 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:20:36.304 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:36.305 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:36.305 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:36.305 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:36.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:36.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:36.306 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:20:36.306 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:36.307 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:36.307 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:36.307 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:36.307 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:36.308 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:36.308 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:36.308 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:36.308 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:36.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:36.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:36.309 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:20:36.309 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:36.310 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:36.310 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:36.310 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:36.310 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:36.311 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:36.311 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:36.311 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:36.311 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:36.312 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:36.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:36.312 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:20:36.312 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:36.312 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:36.313 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:20:36.313 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:36.313 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:36.313 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:36.314 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:20:36.314 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:36.314 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:36.314 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:36.315 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:20:36.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:36.315 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:20:36.315 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:20:36.316 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:20:36.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:36.316 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:20:36Z" 2022-09-03T05:20:36.316 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:20:36.318 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:20:36Z" } 2022-09-03T05:20:36.318 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:20:36.319 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:20:36.319 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:20:36.320 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:20:36.320 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:20:36.321 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:36.321 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:36.322 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:36.381 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:36.381 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:36.429 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:20:36.432+0000 7f51f045b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:36.430 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:20:36.433+0000 7f51f045b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:36.445 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:20:36.446 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:20:36.446 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:20:36.446 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:20:36.447 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:20:36.447 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:36.447 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:36.448 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:36.448 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:20:36.449 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:36.449 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:20:36.450 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:36.450 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:36.451 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:36.451 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:36.452 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:20:36.452 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:36.453 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:36.453 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:36.454 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:36.454 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:36.455 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:36.455 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:20:36.455 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:36.456 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:36.456 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:36.457 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:36.457 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:36.458 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:36.458 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:20:36.458 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:36.459 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:36.459 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:36.459 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:36.460 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:20:36.460 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:36.460 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:36.460 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:36.461 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:36.461 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:36.461 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:36.461 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:20:36.462 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:36.462 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:36.463 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:36.463 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:36.464 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:20:36.464 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:36.465 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:36.465 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:20:36.465 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:36.466 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:36.466 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:36.466 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:20:36.466 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:36.467 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:36.467 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:36.467 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:20:36.468 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:36.468 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:20:36.468 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:20:36.468 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:20:36.469 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:36.469 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:20:36Z" 2022-09-03T05:20:36.469 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:20:36.471 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:20:36Z" } 2022-09-03T05:20:36.471 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:20:36.472 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:20:36.473 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:20:36.473 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:20:36.474 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:20:36.475 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:36.476 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:36.477 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:36.528 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:36.528 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:36.559 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:36.561+0000 7fb7febfd600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:36.560 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:36.562+0000 7fb7febfd600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:36.573 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:20:36.574 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:20:36.574 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:20:36.574 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:20:36.575 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:20:36.575 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:36.575 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:36.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:36.576 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:20:36.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:36.577 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:20:36.577 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:36.577 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:36.577 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:36.578 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:36.578 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:20:36.578 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:36.579 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:36.579 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:36.579 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:36.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:36.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:36.580 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:20:36.580 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:36.580 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:36.581 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:36.581 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:36.581 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:36.581 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:36.582 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:36.582 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:36.582 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:36.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:36.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:36.584 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:20:36.584 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:36.584 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:36.585 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:36.585 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:36.585 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:36.585 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:36.586 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:36.586 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:36.587 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:36.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:36.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:36.587 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:20:36.588 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:36.588 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:36.588 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:20:36.589 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:36.589 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:36.589 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:36.589 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:20:36.590 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:36.590 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:36.590 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:36.590 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:20:36.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:36.591 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:20:36.591 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:20:36.592 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:20:36.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:36.592 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:20:36Z" 2022-09-03T05:20:36.593 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:20:36.594 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:20:36Z" } 2022-09-03T05:20:36.595 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:20:36.595 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:20:36.596 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:20:36.596 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:20:36.597 DEBUG:boto:path=/dsgvcy-7/ 2022-09-03T05:20:36.597 DEBUG:boto:auth_path=/dsgvcy-7/ 2022-09-03T05:20:36.598 DEBUG:boto:Method: HEAD 2022-09-03T05:20:36.599 DEBUG:boto:Path: /dsgvcy-7/ 2022-09-03T05:20:36.599 DEBUG:boto:Data: 2022-09-03T05:20:36.600 DEBUG:boto:Headers: {} 2022-09-03T05:20:36.600 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:36.601 DEBUG:boto:Port: 8000 2022-09-03T05:20:36.601 DEBUG:boto:Params: {} 2022-09-03T05:20:36.602 DEBUG:boto:Token: None 2022-09-03T05:20:36.602 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:36 GMT /dsgvcy-7/ 2022-09-03T05:20:36.603 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uK0mwLMwKUxUcWfsr+8PGSA8nJs= 2022-09-03T05:20:36.604 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uK0mwLMwKUxUcWfsr+8PGSA8nJs=', 'Content-Length': '0'} 2022-09-03T05:20:36.605 DEBUG:boto:Response headers: [('Content-Length', '231'), ('x-amz-request-id', 'tx000002d1b9be2aac0a889-006312e424-10ff-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:36.606 DEBUG:boto:path=/dsgvcy-8/ 2022-09-03T05:20:36.606 DEBUG:boto:auth_path=/dsgvcy-8/ 2022-09-03T05:20:36.607 DEBUG:boto:Method: HEAD 2022-09-03T05:20:36.608 DEBUG:boto:Path: /dsgvcy-8/ 2022-09-03T05:20:36.608 DEBUG:boto:Data: 2022-09-03T05:20:36.609 DEBUG:boto:Headers: {} 2022-09-03T05:20:36.610 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:36.610 DEBUG:boto:Port: 8000 2022-09-03T05:20:36.611 DEBUG:boto:Params: {} 2022-09-03T05:20:36.611 DEBUG:boto:Token: None 2022-09-03T05:20:36.612 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:36 GMT /dsgvcy-8/ 2022-09-03T05:20:36.613 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QNW1q8u6Bci4UnYJ6c9W1FpExrM= 2022-09-03T05:20:36.614 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QNW1q8u6Bci4UnYJ6c9W1FpExrM=', 'Content-Length': '0'} 2022-09-03T05:20:36.615 DEBUG:boto:Response headers: [('Content-Length', '231'), ('x-amz-request-id', 'tx000008a8ed1c8a2a4971f-006312e424-10ff-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:36.616 DEBUG:boto:path=/dsgvcy-9/ 2022-09-03T05:20:36.616 DEBUG:boto:auth_path=/dsgvcy-9/ 2022-09-03T05:20:36.617 DEBUG:boto:Method: HEAD 2022-09-03T05:20:36.617 DEBUG:boto:Path: /dsgvcy-9/ 2022-09-03T05:20:36.618 DEBUG:boto:Data: 2022-09-03T05:20:36.618 DEBUG:boto:Headers: {} 2022-09-03T05:20:36.619 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:36.619 DEBUG:boto:Port: 8000 2022-09-03T05:20:36.620 DEBUG:boto:Params: {} 2022-09-03T05:20:36.621 DEBUG:boto:Token: None 2022-09-03T05:20:36.621 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:36 GMT /dsgvcy-9/ 2022-09-03T05:20:36.622 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Cyhya7sJ0Kuy+C5qrRaICwWMVyQ= 2022-09-03T05:20:36.622 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Cyhya7sJ0Kuy+C5qrRaICwWMVyQ=', 'Content-Length': '0'} 2022-09-03T05:20:36.624 DEBUG:boto:Response headers: [('Content-Length', '231'), ('x-amz-request-id', 'tx00000a1e9caca705cafa8-006312e424-10ff-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:36.624 DEBUG:boto:path=/dsgvcy-7/ 2022-09-03T05:20:36.625 DEBUG:boto:auth_path=/dsgvcy-7/ 2022-09-03T05:20:36.625 DEBUG:boto:Method: HEAD 2022-09-03T05:20:36.626 DEBUG:boto:Path: /dsgvcy-7/ 2022-09-03T05:20:36.626 DEBUG:boto:Data: 2022-09-03T05:20:36.627 DEBUG:boto:Headers: {} 2022-09-03T05:20:36.627 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:36.628 DEBUG:boto:Port: 8000 2022-09-03T05:20:36.628 DEBUG:boto:Params: {} 2022-09-03T05:20:36.629 DEBUG:boto:Token: None 2022-09-03T05:20:36.630 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:36 GMT /dsgvcy-7/ 2022-09-03T05:20:36.630 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uK0mwLMwKUxUcWfsr+8PGSA8nJs= 2022-09-03T05:20:36.631 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uK0mwLMwKUxUcWfsr+8PGSA8nJs=', 'Content-Length': '0'} 2022-09-03T05:20:36.632 DEBUG:boto:Response headers: [('Content-Length', '231'), ('x-amz-request-id', 'tx0000067df51944d4e0391-006312e424-10ea-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:36.633 DEBUG:boto:path=/dsgvcy-8/ 2022-09-03T05:20:36.633 DEBUG:boto:auth_path=/dsgvcy-8/ 2022-09-03T05:20:36.634 DEBUG:boto:Method: HEAD 2022-09-03T05:20:36.635 DEBUG:boto:Path: /dsgvcy-8/ 2022-09-03T05:20:36.635 DEBUG:boto:Data: 2022-09-03T05:20:36.635 DEBUG:boto:Headers: {} 2022-09-03T05:20:36.636 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:36.636 DEBUG:boto:Port: 8000 2022-09-03T05:20:36.637 DEBUG:boto:Params: {} 2022-09-03T05:20:36.638 DEBUG:boto:Token: None 2022-09-03T05:20:36.638 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:36 GMT /dsgvcy-8/ 2022-09-03T05:20:36.639 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QNW1q8u6Bci4UnYJ6c9W1FpExrM= 2022-09-03T05:20:36.639 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QNW1q8u6Bci4UnYJ6c9W1FpExrM=', 'Content-Length': '0'} 2022-09-03T05:20:36.641 DEBUG:boto:Response headers: [('Content-Length', '231'), ('x-amz-request-id', 'tx00000b3ef32aa6a14aa8a-006312e424-10ea-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:36.642 DEBUG:boto:path=/dsgvcy-9/ 2022-09-03T05:20:36.642 DEBUG:boto:auth_path=/dsgvcy-9/ 2022-09-03T05:20:36.643 DEBUG:boto:Method: HEAD 2022-09-03T05:20:36.643 DEBUG:boto:Path: /dsgvcy-9/ 2022-09-03T05:20:36.644 DEBUG:boto:Data: 2022-09-03T05:20:36.644 DEBUG:boto:Headers: {} 2022-09-03T05:20:36.645 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:36.645 DEBUG:boto:Port: 8000 2022-09-03T05:20:36.646 DEBUG:boto:Params: {} 2022-09-03T05:20:36.646 DEBUG:boto:Token: None 2022-09-03T05:20:36.647 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:36 GMT /dsgvcy-9/ 2022-09-03T05:20:36.647 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Cyhya7sJ0Kuy+C5qrRaICwWMVyQ= 2022-09-03T05:20:36.648 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Cyhya7sJ0Kuy+C5qrRaICwWMVyQ=', 'Content-Length': '0'} 2022-09-03T05:20:36.649 DEBUG:boto:Response headers: [('Content-Length', '231'), ('x-amz-request-id', 'tx00000e6eda2dddb0c2b88-006312e424-10ea-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:36.650 DEBUG:boto:path=/dsgvcy-7/ 2022-09-03T05:20:36.651 DEBUG:boto:auth_path=/dsgvcy-7/ 2022-09-03T05:20:36.651 DEBUG:boto:Method: HEAD 2022-09-03T05:20:36.652 DEBUG:boto:Path: /dsgvcy-7/ 2022-09-03T05:20:36.652 DEBUG:boto:Data: 2022-09-03T05:20:36.653 DEBUG:boto:Headers: {} 2022-09-03T05:20:36.653 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:36.654 DEBUG:boto:Port: 8001 2022-09-03T05:20:36.654 DEBUG:boto:Params: {} 2022-09-03T05:20:36.655 DEBUG:boto:Token: None 2022-09-03T05:20:36.655 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:36 GMT /dsgvcy-7/ 2022-09-03T05:20:36.656 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uK0mwLMwKUxUcWfsr+8PGSA8nJs= 2022-09-03T05:20:36.657 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uK0mwLMwKUxUcWfsr+8PGSA8nJs=', 'Content-Length': '0'} 2022-09-03T05:20:36.658 DEBUG:boto:Response headers: [('Content-Length', '231'), ('x-amz-request-id', 'tx00000a23ee25b1404b2ea-006312e424-1103-test-zone3'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:36.659 DEBUG:boto:path=/dsgvcy-8/ 2022-09-03T05:20:36.660 DEBUG:boto:auth_path=/dsgvcy-8/ 2022-09-03T05:20:36.660 DEBUG:boto:Method: HEAD 2022-09-03T05:20:36.661 DEBUG:boto:Path: /dsgvcy-8/ 2022-09-03T05:20:36.661 DEBUG:boto:Data: 2022-09-03T05:20:36.662 DEBUG:boto:Headers: {} 2022-09-03T05:20:36.662 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:36.663 DEBUG:boto:Port: 8001 2022-09-03T05:20:36.663 DEBUG:boto:Params: {} 2022-09-03T05:20:36.664 DEBUG:boto:Token: None 2022-09-03T05:20:36.664 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:36 GMT /dsgvcy-8/ 2022-09-03T05:20:36.665 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QNW1q8u6Bci4UnYJ6c9W1FpExrM= 2022-09-03T05:20:36.666 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QNW1q8u6Bci4UnYJ6c9W1FpExrM=', 'Content-Length': '0'} 2022-09-03T05:20:36.667 DEBUG:boto:Response headers: [('Content-Length', '231'), ('x-amz-request-id', 'tx000006483b792f504f8d6-006312e424-1103-test-zone3'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:36.668 DEBUG:boto:path=/dsgvcy-9/ 2022-09-03T05:20:36.668 DEBUG:boto:auth_path=/dsgvcy-9/ 2022-09-03T05:20:36.669 DEBUG:boto:Method: HEAD 2022-09-03T05:20:36.669 DEBUG:boto:Path: /dsgvcy-9/ 2022-09-03T05:20:36.670 DEBUG:boto:Data: 2022-09-03T05:20:36.670 DEBUG:boto:Headers: {} 2022-09-03T05:20:36.671 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:36.671 DEBUG:boto:Port: 8001 2022-09-03T05:20:36.672 DEBUG:boto:Params: {} 2022-09-03T05:20:36.672 DEBUG:boto:Token: None 2022-09-03T05:20:36.673 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:36 GMT /dsgvcy-9/ 2022-09-03T05:20:36.673 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Cyhya7sJ0Kuy+C5qrRaICwWMVyQ= 2022-09-03T05:20:36.674 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Cyhya7sJ0Kuy+C5qrRaICwWMVyQ=', 'Content-Length': '0'} 2022-09-03T05:20:36.675 DEBUG:boto:Response headers: [('Content-Length', '231'), ('x-amz-request-id', 'tx0000079f8d0a494b24267-006312e424-1103-test-zone3'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:36.676 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_remove ... ok 2022-09-03T05:20:36.677 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-10 2022-09-03T05:20:36.677 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:36.678 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:36.678 DEBUG:boto:Method: PUT 2022-09-03T05:20:36.679 DEBUG:boto:Path: /dsgvcy-10/ 2022-09-03T05:20:36.680 DEBUG:boto:Data: 2022-09-03T05:20:36.680 DEBUG:boto:Headers: {} 2022-09-03T05:20:36.681 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:36.681 DEBUG:boto:Port: 8000 2022-09-03T05:20:36.682 DEBUG:boto:Params: {} 2022-09-03T05:20:36.682 DEBUG:boto:Token: None 2022-09-03T05:20:36.683 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:20:36 GMT /dsgvcy-10/ 2022-09-03T05:20:36.684 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mXZBOYQGxqLGoTtGmVSU32pwhwc= 2022-09-03T05:20:36.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mXZBOYQGxqLGoTtGmVSU32pwhwc=', 'Content-Length': '0'} 2022-09-03T05:20:36.704 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000af00380bb2c0d684-006312e424-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:36.705 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=dsgvcy-11 2022-09-03T05:20:36.706 DEBUG:boto:path=/dsgvcy-11/ 2022-09-03T05:20:36.706 DEBUG:boto:auth_path=/dsgvcy-11/ 2022-09-03T05:20:36.707 DEBUG:boto:Method: PUT 2022-09-03T05:20:36.707 DEBUG:boto:Path: /dsgvcy-11/ 2022-09-03T05:20:36.708 DEBUG:boto:Data: 2022-09-03T05:20:36.708 DEBUG:boto:Headers: {} 2022-09-03T05:20:36.709 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:36.709 DEBUG:boto:Port: 8000 2022-09-03T05:20:36.710 DEBUG:boto:Params: {} 2022-09-03T05:20:36.710 DEBUG:boto:Token: None 2022-09-03T05:20:36.711 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:20:36 GMT /dsgvcy-11/ 2022-09-03T05:20:36.711 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:InPmlc3frk//S/PIGN/bS6SB2No= 2022-09-03T05:20:36.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:InPmlc3frk//S/PIGN/bS6SB2No=', 'Content-Length': '0'} 2022-09-03T05:20:36.739 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000744eced7f2a211fb-006312e424-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:36.740 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=dsgvcy-12 2022-09-03T05:20:36.741 DEBUG:boto:path=/dsgvcy-12/ 2022-09-03T05:20:36.742 DEBUG:boto:auth_path=/dsgvcy-12/ 2022-09-03T05:20:36.743 DEBUG:boto:Method: PUT 2022-09-03T05:20:36.744 DEBUG:boto:Path: /dsgvcy-12/ 2022-09-03T05:20:36.745 DEBUG:boto:Data: 2022-09-03T05:20:36.745 DEBUG:boto:Headers: {} 2022-09-03T05:20:36.746 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:36.746 DEBUG:boto:Port: 8001 2022-09-03T05:20:36.747 DEBUG:boto:Params: {} 2022-09-03T05:20:36.748 DEBUG:boto:Token: None 2022-09-03T05:20:36.748 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:20:36 GMT /dsgvcy-12/ 2022-09-03T05:20:36.749 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vPOW4TpD1EA9eVbNGXDt1pdf6sk= 2022-09-03T05:20:36.749 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vPOW4TpD1EA9eVbNGXDt1pdf6sk=', 'Content-Length': '0'} 2022-09-03T05:20:36.773 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000511182f4e5149392-006312e424-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:36.774 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:36.774 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:36.775 DEBUG:boto:Method: HEAD 2022-09-03T05:20:36.775 DEBUG:boto:Path: /dsgvcy-10/ 2022-09-03T05:20:36.776 DEBUG:boto:Data: 2022-09-03T05:20:36.776 DEBUG:boto:Headers: {} 2022-09-03T05:20:36.777 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:36.777 DEBUG:boto:Port: 8000 2022-09-03T05:20:36.778 DEBUG:boto:Params: {} 2022-09-03T05:20:36.779 DEBUG:boto:Token: None 2022-09-03T05:20:36.779 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:36 GMT /dsgvcy-10/ 2022-09-03T05:20:36.780 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:60LgsBKDotAXw3G+rlLwLE+Z9s8= 2022-09-03T05:20:36.781 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:60LgsBKDotAXw3G+rlLwLE+Z9s8=', 'Content-Length': '0'} 2022-09-03T05:20:36.783 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ad72a578cdd37f14-006312e424-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:36.784 DEBUG:boto:path=/dsgvcy-10/myobj 2022-09-03T05:20:36.788 DEBUG:boto:auth_path=/dsgvcy-10/myobj 2022-09-03T05:20:36.788 DEBUG:boto:Method: PUT 2022-09-03T05:20:36.789 DEBUG:boto:Path: /dsgvcy-10/myobj 2022-09-03T05:20:36.789 DEBUG:boto:Data: 2022-09-03T05:20:36.790 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:20:36.790 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:36.791 DEBUG:boto:Port: 8000 2022-09-03T05:20:36.791 DEBUG:boto:Params: {} 2022-09-03T05:20:36.792 DEBUG:boto:Token: None 2022-09-03T05:20:36.792 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:20:36 GMT /dsgvcy-10/myobj 2022-09-03T05:20:36.793 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9HCBr2nDTeu44dAhhsKRCNoMKIc= 2022-09-03T05:20:36.794 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:20:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9HCBr2nDTeu44dAhhsKRCNoMKIc='} 2022-09-03T05:20:36.838 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009f25738d2af73e30-006312e424-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:36.839 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:36.840 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:36.840 DEBUG:boto:Method: HEAD 2022-09-03T05:20:36.841 DEBUG:boto:Path: /dsgvcy-10/ 2022-09-03T05:20:36.841 DEBUG:boto:Data: 2022-09-03T05:20:36.842 DEBUG:boto:Headers: {} 2022-09-03T05:20:36.842 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:36.843 DEBUG:boto:Port: 8000 2022-09-03T05:20:36.843 DEBUG:boto:Params: {} 2022-09-03T05:20:36.844 DEBUG:boto:Token: None 2022-09-03T05:20:36.844 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:36 GMT /dsgvcy-10/ 2022-09-03T05:20:36.845 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:60LgsBKDotAXw3G+rlLwLE+Z9s8= 2022-09-03T05:20:36.845 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:60LgsBKDotAXw3G+rlLwLE+Z9s8=', 'Content-Length': '0'} 2022-09-03T05:20:36.848 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000085ac4f515067b511-006312e424-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:36.849 DEBUG:boto:path=/dsgvcy-10/_myobj 2022-09-03T05:20:36.849 DEBUG:boto:auth_path=/dsgvcy-10/_myobj 2022-09-03T05:20:36.850 DEBUG:boto:Method: PUT 2022-09-03T05:20:36.850 DEBUG:boto:Path: /dsgvcy-10/_myobj 2022-09-03T05:20:36.851 DEBUG:boto:Data: 2022-09-03T05:20:36.851 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:20:36.852 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:36.852 DEBUG:boto:Port: 8000 2022-09-03T05:20:36.853 DEBUG:boto:Params: {} 2022-09-03T05:20:36.853 DEBUG:boto:Token: None 2022-09-03T05:20:36.854 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:20:36 GMT /dsgvcy-10/_myobj 2022-09-03T05:20:36.854 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TKjWkYyoJcxFq1s8vAAKCsjo4vQ= 2022-09-03T05:20:36.855 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:20:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TKjWkYyoJcxFq1s8vAAKCsjo4vQ='} 2022-09-03T05:20:36.880 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000064c979ffbbacb1fa-006312e424-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:36.881 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:36.882 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:36.882 DEBUG:boto:Method: HEAD 2022-09-03T05:20:36.883 DEBUG:boto:Path: /dsgvcy-10/ 2022-09-03T05:20:36.884 DEBUG:boto:Data: 2022-09-03T05:20:36.884 DEBUG:boto:Headers: {} 2022-09-03T05:20:36.885 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:36.885 DEBUG:boto:Port: 8000 2022-09-03T05:20:36.885 DEBUG:boto:Params: {} 2022-09-03T05:20:36.886 DEBUG:boto:Token: None 2022-09-03T05:20:36.887 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:36 GMT /dsgvcy-10/ 2022-09-03T05:20:36.887 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:60LgsBKDotAXw3G+rlLwLE+Z9s8= 2022-09-03T05:20:36.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:60LgsBKDotAXw3G+rlLwLE+Z9s8=', 'Content-Length': '0'} 2022-09-03T05:20:36.890 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001171ee975806d1d6-006312e424-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:36.891 DEBUG:boto:path=/dsgvcy-10/%3A 2022-09-03T05:20:36.892 DEBUG:boto:auth_path=/dsgvcy-10/%3A 2022-09-03T05:20:36.892 DEBUG:boto:Method: PUT 2022-09-03T05:20:36.893 DEBUG:boto:Path: /dsgvcy-10/%3A 2022-09-03T05:20:36.893 DEBUG:boto:Data: 2022-09-03T05:20:36.894 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:20:36.894 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:36.895 DEBUG:boto:Port: 8000 2022-09-03T05:20:36.895 DEBUG:boto:Params: {} 2022-09-03T05:20:36.896 DEBUG:boto:Token: None 2022-09-03T05:20:36.897 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:20:36 GMT /dsgvcy-10/%3A 2022-09-03T05:20:36.897 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X2aZlJtuhCFxHk8aEJN3l3K5sAA= 2022-09-03T05:20:36.898 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:20:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X2aZlJtuhCFxHk8aEJN3l3K5sAA='} 2022-09-03T05:20:36.946 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000740170d445291a29-006312e424-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:36.948 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:36.949 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:36.950 DEBUG:boto:Method: HEAD 2022-09-03T05:20:36.950 DEBUG:boto:Path: /dsgvcy-10/ 2022-09-03T05:20:36.951 DEBUG:boto:Data: 2022-09-03T05:20:36.951 DEBUG:boto:Headers: {} 2022-09-03T05:20:36.952 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:36.952 DEBUG:boto:Port: 8000 2022-09-03T05:20:36.953 DEBUG:boto:Params: {} 2022-09-03T05:20:36.953 DEBUG:boto:Token: None 2022-09-03T05:20:36.954 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:36 GMT /dsgvcy-10/ 2022-09-03T05:20:36.955 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:60LgsBKDotAXw3G+rlLwLE+Z9s8= 2022-09-03T05:20:36.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:60LgsBKDotAXw3G+rlLwLE+Z9s8=', 'Content-Length': '0'} 2022-09-03T05:20:36.959 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ce0489afdfc840af-006312e424-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:36.960 DEBUG:boto:path=/dsgvcy-10/%26 2022-09-03T05:20:36.960 DEBUG:boto:auth_path=/dsgvcy-10/%26 2022-09-03T05:20:36.961 DEBUG:boto:Method: PUT 2022-09-03T05:20:36.962 DEBUG:boto:Path: /dsgvcy-10/%26 2022-09-03T05:20:36.963 DEBUG:boto:Data: 2022-09-03T05:20:36.963 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:20:36.964 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:36.964 DEBUG:boto:Port: 8000 2022-09-03T05:20:36.965 DEBUG:boto:Params: {} 2022-09-03T05:20:36.966 DEBUG:boto:Token: None 2022-09-03T05:20:36.966 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:20:36 GMT /dsgvcy-10/%26 2022-09-03T05:20:36.967 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gRRYfLpoMYkTr3OcErEQnOFz+Yg= 2022-09-03T05:20:36.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:20:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gRRYfLpoMYkTr3OcErEQnOFz+Yg='} 2022-09-03T05:20:36.976 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c8b0c66e78a745c9-006312e424-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:36.978 DEBUG:boto:path=/dsgvcy-11/ 2022-09-03T05:20:36.979 DEBUG:boto:auth_path=/dsgvcy-11/ 2022-09-03T05:20:36.979 DEBUG:boto:Method: HEAD 2022-09-03T05:20:36.980 DEBUG:boto:Path: /dsgvcy-11/ 2022-09-03T05:20:36.981 DEBUG:boto:Data: 2022-09-03T05:20:36.982 DEBUG:boto:Headers: {} 2022-09-03T05:20:36.982 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:36.983 DEBUG:boto:Port: 8000 2022-09-03T05:20:36.983 DEBUG:boto:Params: {} 2022-09-03T05:20:36.984 DEBUG:boto:Token: None 2022-09-03T05:20:36.985 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:36 GMT /dsgvcy-11/ 2022-09-03T05:20:36.985 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OSR07lTDJb5HSEdBwmXkkbaadJI= 2022-09-03T05:20:36.986 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OSR07lTDJb5HSEdBwmXkkbaadJI=', 'Content-Length': '0'} 2022-09-03T05:20:36.988 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000003e5fb65501f9fd7-006312e424-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:36.989 DEBUG:boto:path=/dsgvcy-11/myobj 2022-09-03T05:20:36.990 DEBUG:boto:auth_path=/dsgvcy-11/myobj 2022-09-03T05:20:36.991 DEBUG:boto:Method: PUT 2022-09-03T05:20:36.991 DEBUG:boto:Path: /dsgvcy-11/myobj 2022-09-03T05:20:36.992 DEBUG:boto:Data: 2022-09-03T05:20:36.992 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:20:36.993 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:36.993 DEBUG:boto:Port: 8000 2022-09-03T05:20:36.994 DEBUG:boto:Params: {} 2022-09-03T05:20:36.995 DEBUG:boto:Token: None 2022-09-03T05:20:36.995 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:20:36 GMT /dsgvcy-11/myobj 2022-09-03T05:20:36.996 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D3qI/AbP4Mf1GUYTP2Ox/mVX13U= 2022-09-03T05:20:36.997 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:20:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D3qI/AbP4Mf1GUYTP2Ox/mVX13U='} 2022-09-03T05:20:37.008 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000002f49125d0f8599ae-006312e425-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:20:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:37.009 DEBUG:boto:path=/dsgvcy-11/ 2022-09-03T05:20:37.010 DEBUG:boto:auth_path=/dsgvcy-11/ 2022-09-03T05:20:37.010 DEBUG:boto:Method: HEAD 2022-09-03T05:20:37.011 DEBUG:boto:Path: /dsgvcy-11/ 2022-09-03T05:20:37.012 DEBUG:boto:Data: 2022-09-03T05:20:37.012 DEBUG:boto:Headers: {} 2022-09-03T05:20:37.013 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:37.013 DEBUG:boto:Port: 8000 2022-09-03T05:20:37.014 DEBUG:boto:Params: {} 2022-09-03T05:20:37.015 DEBUG:boto:Token: None 2022-09-03T05:20:37.016 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:37 GMT /dsgvcy-11/ 2022-09-03T05:20:37.016 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nkGTtJ+r6+OopJfYhq7cSKq7GWE= 2022-09-03T05:20:37.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nkGTtJ+r6+OopJfYhq7cSKq7GWE=', 'Content-Length': '0'} 2022-09-03T05:20:37.019 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000957701409a0cc2ff-006312e425-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:37.021 DEBUG:boto:path=/dsgvcy-11/_myobj 2022-09-03T05:20:37.021 DEBUG:boto:auth_path=/dsgvcy-11/_myobj 2022-09-03T05:20:37.022 DEBUG:boto:Method: PUT 2022-09-03T05:20:37.023 DEBUG:boto:Path: /dsgvcy-11/_myobj 2022-09-03T05:20:37.023 DEBUG:boto:Data: 2022-09-03T05:20:37.024 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:20:37.024 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:37.025 DEBUG:boto:Port: 8000 2022-09-03T05:20:37.025 DEBUG:boto:Params: {} 2022-09-03T05:20:37.026 DEBUG:boto:Token: None 2022-09-03T05:20:37.027 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:20:37 GMT /dsgvcy-11/_myobj 2022-09-03T05:20:37.028 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LKckGttaq+zrCUqJ1cumunstIPM= 2022-09-03T05:20:37.028 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:20:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LKckGttaq+zrCUqJ1cumunstIPM='} 2022-09-03T05:20:37.076 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000007ff57f8f1d3f197a-006312e425-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:20:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:37.077 DEBUG:boto:path=/dsgvcy-11/ 2022-09-03T05:20:37.078 DEBUG:boto:auth_path=/dsgvcy-11/ 2022-09-03T05:20:37.079 DEBUG:boto:Method: HEAD 2022-09-03T05:20:37.079 DEBUG:boto:Path: /dsgvcy-11/ 2022-09-03T05:20:37.080 DEBUG:boto:Data: 2022-09-03T05:20:37.080 DEBUG:boto:Headers: {} 2022-09-03T05:20:37.081 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:37.082 DEBUG:boto:Port: 8000 2022-09-03T05:20:37.082 DEBUG:boto:Params: {} 2022-09-03T05:20:37.083 DEBUG:boto:Token: None 2022-09-03T05:20:37.084 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:37 GMT /dsgvcy-11/ 2022-09-03T05:20:37.085 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nkGTtJ+r6+OopJfYhq7cSKq7GWE= 2022-09-03T05:20:37.086 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nkGTtJ+r6+OopJfYhq7cSKq7GWE=', 'Content-Length': '0'} 2022-09-03T05:20:37.089 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b38b7e724809e9f4-006312e425-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:37.090 DEBUG:boto:path=/dsgvcy-11/%3A 2022-09-03T05:20:37.091 DEBUG:boto:auth_path=/dsgvcy-11/%3A 2022-09-03T05:20:37.092 DEBUG:boto:Method: PUT 2022-09-03T05:20:37.093 DEBUG:boto:Path: /dsgvcy-11/%3A 2022-09-03T05:20:37.093 DEBUG:boto:Data: 2022-09-03T05:20:37.094 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:20:37.095 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:37.095 DEBUG:boto:Port: 8000 2022-09-03T05:20:37.096 DEBUG:boto:Params: {} 2022-09-03T05:20:37.097 DEBUG:boto:Token: None 2022-09-03T05:20:37.097 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:20:37 GMT /dsgvcy-11/%3A 2022-09-03T05:20:37.098 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2PNDx8eDkKeiXZsWqEhHFz/KF/w= 2022-09-03T05:20:37.099 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:20:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2PNDx8eDkKeiXZsWqEhHFz/KF/w='} 2022-09-03T05:20:37.109 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000fac6dc7d4053c825-006312e425-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:20:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:37.110 DEBUG:boto:path=/dsgvcy-11/ 2022-09-03T05:20:37.111 DEBUG:boto:auth_path=/dsgvcy-11/ 2022-09-03T05:20:37.111 DEBUG:boto:Method: HEAD 2022-09-03T05:20:37.112 DEBUG:boto:Path: /dsgvcy-11/ 2022-09-03T05:20:37.114 DEBUG:boto:Data: 2022-09-03T05:20:37.115 DEBUG:boto:Headers: {} 2022-09-03T05:20:37.117 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:37.118 DEBUG:boto:Port: 8000 2022-09-03T05:20:37.119 DEBUG:boto:Params: {} 2022-09-03T05:20:37.120 DEBUG:boto:Token: None 2022-09-03T05:20:37.122 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:37 GMT /dsgvcy-11/ 2022-09-03T05:20:37.123 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nkGTtJ+r6+OopJfYhq7cSKq7GWE= 2022-09-03T05:20:37.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nkGTtJ+r6+OopJfYhq7cSKq7GWE=', 'Content-Length': '0'} 2022-09-03T05:20:37.127 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000036072adf49037dd2-006312e425-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:37.128 DEBUG:boto:path=/dsgvcy-11/%26 2022-09-03T05:20:37.129 DEBUG:boto:auth_path=/dsgvcy-11/%26 2022-09-03T05:20:37.130 DEBUG:boto:Method: PUT 2022-09-03T05:20:37.131 DEBUG:boto:Path: /dsgvcy-11/%26 2022-09-03T05:20:37.132 DEBUG:boto:Data: 2022-09-03T05:20:37.133 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:20:37.134 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:37.135 DEBUG:boto:Port: 8000 2022-09-03T05:20:37.136 DEBUG:boto:Params: {} 2022-09-03T05:20:37.137 DEBUG:boto:Token: None 2022-09-03T05:20:37.138 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:20:37 GMT /dsgvcy-11/%26 2022-09-03T05:20:37.138 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FWRkr+fGSYq5tPQpB+qJNn63714= 2022-09-03T05:20:37.139 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:20:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FWRkr+fGSYq5tPQpB+qJNn63714='} 2022-09-03T05:20:37.186 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009e65094d7cf9c6ea-006312e425-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:20:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:37.187 DEBUG:boto:path=/dsgvcy-12/ 2022-09-03T05:20:37.188 DEBUG:boto:auth_path=/dsgvcy-12/ 2022-09-03T05:20:37.188 DEBUG:boto:Method: HEAD 2022-09-03T05:20:37.189 DEBUG:boto:Path: /dsgvcy-12/ 2022-09-03T05:20:37.189 DEBUG:boto:Data: 2022-09-03T05:20:37.190 DEBUG:boto:Headers: {} 2022-09-03T05:20:37.191 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:37.191 DEBUG:boto:Port: 8001 2022-09-03T05:20:37.192 DEBUG:boto:Params: {} 2022-09-03T05:20:37.193 DEBUG:boto:Token: None 2022-09-03T05:20:37.194 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:37 GMT /dsgvcy-12/ 2022-09-03T05:20:37.195 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cGGoATGgWmql1LEUWoIRfL1JVKw= 2022-09-03T05:20:37.196 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cGGoATGgWmql1LEUWoIRfL1JVKw=', 'Content-Length': '0'} 2022-09-03T05:20:37.199 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000077bf34e6f55942d8-006312e425-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:37.200 DEBUG:boto:path=/dsgvcy-12/myobj 2022-09-03T05:20:37.200 DEBUG:boto:auth_path=/dsgvcy-12/myobj 2022-09-03T05:20:37.201 DEBUG:boto:Method: PUT 2022-09-03T05:20:37.201 DEBUG:boto:Path: /dsgvcy-12/myobj 2022-09-03T05:20:37.202 DEBUG:boto:Data: 2022-09-03T05:20:37.202 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:20:37.203 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:37.203 DEBUG:boto:Port: 8001 2022-09-03T05:20:37.204 DEBUG:boto:Params: {} 2022-09-03T05:20:37.204 DEBUG:boto:Token: None 2022-09-03T05:20:37.205 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:20:37 GMT /dsgvcy-12/myobj 2022-09-03T05:20:37.206 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A3g6rgSZw95U5dQ2VXC0s138fcg= 2022-09-03T05:20:37.206 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:20:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A3g6rgSZw95U5dQ2VXC0s138fcg='} 2022-09-03T05:20:37.226 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c6c0d1c01f153ddb-006312e425-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:20:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:37.227 DEBUG:boto:path=/dsgvcy-12/ 2022-09-03T05:20:37.227 DEBUG:boto:auth_path=/dsgvcy-12/ 2022-09-03T05:20:37.228 DEBUG:boto:Method: HEAD 2022-09-03T05:20:37.228 DEBUG:boto:Path: /dsgvcy-12/ 2022-09-03T05:20:37.229 DEBUG:boto:Data: 2022-09-03T05:20:37.229 DEBUG:boto:Headers: {} 2022-09-03T05:20:37.230 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:37.230 DEBUG:boto:Port: 8001 2022-09-03T05:20:37.231 DEBUG:boto:Params: {} 2022-09-03T05:20:37.232 DEBUG:boto:Token: None 2022-09-03T05:20:37.232 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:37 GMT /dsgvcy-12/ 2022-09-03T05:20:37.233 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cGGoATGgWmql1LEUWoIRfL1JVKw= 2022-09-03T05:20:37.233 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cGGoATGgWmql1LEUWoIRfL1JVKw=', 'Content-Length': '0'} 2022-09-03T05:20:37.236 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b31372e5376d6bb7-006312e425-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:37.237 DEBUG:boto:path=/dsgvcy-12/_myobj 2022-09-03T05:20:37.237 DEBUG:boto:auth_path=/dsgvcy-12/_myobj 2022-09-03T05:20:37.238 DEBUG:boto:Method: PUT 2022-09-03T05:20:37.239 DEBUG:boto:Path: /dsgvcy-12/_myobj 2022-09-03T05:20:37.239 DEBUG:boto:Data: 2022-09-03T05:20:37.240 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:20:37.240 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:37.241 DEBUG:boto:Port: 8001 2022-09-03T05:20:37.241 DEBUG:boto:Params: {} 2022-09-03T05:20:37.242 DEBUG:boto:Token: None 2022-09-03T05:20:37.243 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:20:37 GMT /dsgvcy-12/_myobj 2022-09-03T05:20:37.243 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CPVnmOj+Ic1p6nxmsbsAe/Wpf4o= 2022-09-03T05:20:37.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:20:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CPVnmOj+Ic1p6nxmsbsAe/Wpf4o='} 2022-09-03T05:20:37.260 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000cd9272a35d02311d-006312e425-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:20:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:37.261 DEBUG:boto:path=/dsgvcy-12/ 2022-09-03T05:20:37.262 DEBUG:boto:auth_path=/dsgvcy-12/ 2022-09-03T05:20:37.262 DEBUG:boto:Method: HEAD 2022-09-03T05:20:37.263 DEBUG:boto:Path: /dsgvcy-12/ 2022-09-03T05:20:37.263 DEBUG:boto:Data: 2022-09-03T05:20:37.264 DEBUG:boto:Headers: {} 2022-09-03T05:20:37.264 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:37.265 DEBUG:boto:Port: 8001 2022-09-03T05:20:37.265 DEBUG:boto:Params: {} 2022-09-03T05:20:37.266 DEBUG:boto:Token: None 2022-09-03T05:20:37.267 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:37 GMT /dsgvcy-12/ 2022-09-03T05:20:37.267 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cGGoATGgWmql1LEUWoIRfL1JVKw= 2022-09-03T05:20:37.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cGGoATGgWmql1LEUWoIRfL1JVKw=', 'Content-Length': '0'} 2022-09-03T05:20:37.270 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000aa64d3dfd9939ab8-006312e425-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:37.271 DEBUG:boto:path=/dsgvcy-12/%3A 2022-09-03T05:20:37.272 DEBUG:boto:auth_path=/dsgvcy-12/%3A 2022-09-03T05:20:37.272 DEBUG:boto:Method: PUT 2022-09-03T05:20:37.273 DEBUG:boto:Path: /dsgvcy-12/%3A 2022-09-03T05:20:37.273 DEBUG:boto:Data: 2022-09-03T05:20:37.274 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:20:37.274 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:37.275 DEBUG:boto:Port: 8001 2022-09-03T05:20:37.276 DEBUG:boto:Params: {} 2022-09-03T05:20:37.276 DEBUG:boto:Token: None 2022-09-03T05:20:37.277 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:20:37 GMT /dsgvcy-12/%3A 2022-09-03T05:20:37.277 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SBd9hFv3DVKZd+f/bwiLtRiRJ4Q= 2022-09-03T05:20:37.278 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:20:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SBd9hFv3DVKZd+f/bwiLtRiRJ4Q='} 2022-09-03T05:20:37.322 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d6251f2865b0275a-006312e425-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:20:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:37.324 DEBUG:boto:path=/dsgvcy-12/ 2022-09-03T05:20:37.325 DEBUG:boto:auth_path=/dsgvcy-12/ 2022-09-03T05:20:37.325 DEBUG:boto:Method: HEAD 2022-09-03T05:20:37.326 DEBUG:boto:Path: /dsgvcy-12/ 2022-09-03T05:20:37.326 DEBUG:boto:Data: 2022-09-03T05:20:37.327 DEBUG:boto:Headers: {} 2022-09-03T05:20:37.327 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:37.328 DEBUG:boto:Port: 8001 2022-09-03T05:20:37.328 DEBUG:boto:Params: {} 2022-09-03T05:20:37.329 DEBUG:boto:Token: None 2022-09-03T05:20:37.329 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:37 GMT /dsgvcy-12/ 2022-09-03T05:20:37.330 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cGGoATGgWmql1LEUWoIRfL1JVKw= 2022-09-03T05:20:37.331 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cGGoATGgWmql1LEUWoIRfL1JVKw=', 'Content-Length': '0'} 2022-09-03T05:20:37.333 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000053b50651d4912629-006312e425-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:37.334 DEBUG:boto:path=/dsgvcy-12/%26 2022-09-03T05:20:37.334 DEBUG:boto:auth_path=/dsgvcy-12/%26 2022-09-03T05:20:37.335 DEBUG:boto:Method: PUT 2022-09-03T05:20:37.335 DEBUG:boto:Path: /dsgvcy-12/%26 2022-09-03T05:20:37.336 DEBUG:boto:Data: 2022-09-03T05:20:37.336 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:20:37.337 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:37.337 DEBUG:boto:Port: 8001 2022-09-03T05:20:37.338 DEBUG:boto:Params: {} 2022-09-03T05:20:37.339 DEBUG:boto:Token: None 2022-09-03T05:20:37.339 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:20:37 GMT /dsgvcy-12/%26 2022-09-03T05:20:37.340 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5O5IU+MtXql2E56OSbyskcSpIMI= 2022-09-03T05:20:37.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:20:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5O5IU+MtXql2E56OSbyskcSpIMI='} 2022-09-03T05:20:37.346 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000002d2905dd8bdcbc25-006312e425-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:20:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:37.347 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:37.348 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:37.348 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:37.397 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:37.398 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:37.424 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=41ecd2a5-afc0-4eae-9349-7dfe39baa1cb 2022-09-03T05:20:37.435 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:20:37.435 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:37.435 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:37.436 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:18:18.906852Z" 2022-09-03T05:20:37.436 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:37.436 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:37.436 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:37.437 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:20:37.437 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:37.437 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:37.438 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182436.766898_99.1", 2022-09-03T05:20:37.438 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:20:36.766898Z" 2022-09-03T05:20:37.438 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:37.438 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:37.439 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:20:37.439 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:20:30.928498Z" 2022-09-03T05:20:37.439 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:37.439 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:20:37.441 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182436.766898_99.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:20:37.441 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:20:37.442 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:37.442 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:37.443 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:37.496 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:37.496 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:37.523 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:37.526+0000 7f488aeb7600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:37.524 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:37.527+0000 7f488aeb7600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:37.535 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:20:37.536 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:20:37.536 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:20:37.536 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:20:37.537 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:20:37.537 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:37.538 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:37.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:37.539 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:20:37.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:37.539 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:20:37.540 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:37.540 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:37.541 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:37.541 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:37.542 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:20:37.542 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:37.543 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:37.543 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:37.544 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:37.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:37.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:37.545 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:20:37.545 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:37.546 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:37.546 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:37.546 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:37.547 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:37.547 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:37.547 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:37.547 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:37.548 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:37.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:37.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:37.548 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:20:37.549 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:37.549 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:37.549 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:37.549 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:37.550 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:37.550 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:37.550 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:37.551 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:37.551 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:37.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:37.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:37.552 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:20:37.552 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:37.552 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:37.552 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:20:37.553 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:37.553 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:37.553 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:37.553 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:20:37.554 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:37.554 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:37.554 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:37.555 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:20:37.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:37.555 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:20:37.555 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:20:37.556 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:20:37.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:37.556 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:20:37Z" 2022-09-03T05:20:37.556 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:20:37.558 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:20:37Z" } 2022-09-03T05:20:37.558 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:20:37.559 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182436.766898_99.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:20:37.559 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:20:37.560 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662182436.766898_99.1 target= 2022-09-03T05:20:42.561 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:42.562 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:42.563 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:42.622 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:42.622 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:42.660 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:42.662+0000 7efc57d32600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:42.661 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:42.663+0000 7efc57d32600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:42.673 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:20:42.673 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:20:42.674 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:20:42.674 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:20:42.675 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:20:42.675 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:42.675 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:42.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:42.676 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:20:42.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:42.677 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:20:42.678 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:42.679 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:42.679 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:42.679 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:42.680 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:20:42.680 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:42.681 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:42.681 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:42.682 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:42.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:42.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:42.683 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:20:42.683 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:42.684 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:42.684 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:42.684 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:42.684 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:42.685 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:42.685 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:42.685 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:42.685 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:42.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:42.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:42.686 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:20:42.686 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:42.687 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:42.687 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182436.766898_99.1", 2022-09-03T05:20:42.687 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:42.687 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:42.688 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:42.688 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.766898Z", 2022-09-03T05:20:42.688 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:42.689 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:42.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:42.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:42.690 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:20:42.690 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:42.690 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:42.691 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:20:42.691 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:42.691 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:42.692 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:42.692 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:20:42.692 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:42.693 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:42.693 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:42.694 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:20:42.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:42.695 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:20:42.695 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:20:42.695 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:20:42.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:42.696 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:20:42Z" 2022-09-03T05:20:42.696 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:20:42.699 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662182436.766898_99.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:36.766898Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:20:42Z" } 2022-09-03T05:20:42.700 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662182436.766898_99.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:36.766898Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:20:42.701 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182436.766898_99.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:20:42.702 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182436.766898_99.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:20:42.703 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:20:42.703 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:20:42.704 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:42.705 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:42.706 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:42.770 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:42.770 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:42.800 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:20:42.803+0000 7f1ffc776600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:42.801 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:20:42.804+0000 7f1ffc776600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:42.813 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:20:42.813 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:20:42.814 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:20:42.814 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:20:42.814 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:20:42.815 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:42.815 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:42.815 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:42.815 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:20:42.816 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:42.816 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:20:42.816 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:42.816 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:42.817 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:42.817 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:42.817 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:20:42.817 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:42.818 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:42.818 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:42.818 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:42.819 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:42.819 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:42.819 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:20:42.819 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:42.820 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:42.820 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:42.820 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:42.821 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:42.821 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:42.821 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:20:42.821 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:42.822 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:42.822 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:42.822 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:42.823 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:20:42.823 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:42.824 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:42.824 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182436.694951_88.1", 2022-09-03T05:20:42.824 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:42.824 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:42.825 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:42.825 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.694951Z", 2022-09-03T05:20:42.825 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:42.825 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:42.826 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:42.826 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:42.826 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:20:42.827 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:42.827 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:42.827 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:20:42.828 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:42.828 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:42.828 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:42.828 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:20:42.829 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:42.829 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:42.829 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:42.829 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:20:42.830 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:42.830 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:20:42.830 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:20:42.830 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:20:42.831 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:42.831 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:20:42Z" 2022-09-03T05:20:42.832 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:20:42.833 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662182436.694951_88.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:36.694951Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:20:42Z" } 2022-09-03T05:20:42.834 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662182436.694951_88.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:36.694951Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:20:42.835 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182436.766898_99.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:20:42.835 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182436.694951_88.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:20:42.836 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662182436.766898_99.1 target=1_1662182436.694951_88.1 2022-09-03T05:20:47.837 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:47.838 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:47.839 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:47.895 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:47.895 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:47.939 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:20:47.942+0000 7f3ef68f1600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:47.940 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:20:47.943+0000 7f3ef68f1600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:47.954 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:20:47.955 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:20:47.955 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:20:47.955 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:20:47.956 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:20:47.956 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:47.956 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:47.956 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:47.957 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:20:47.957 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:47.957 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:20:47.957 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:47.958 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:47.958 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:47.958 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:47.959 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:20:47.959 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:47.959 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:47.959 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:47.960 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:47.960 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:47.960 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:47.960 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:20:47.961 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:47.961 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:47.962 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:20:47.962 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:47.962 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:47.962 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:47.963 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:20:47.963 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:47.963 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:47.964 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:47.964 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:47.964 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:20:47.966 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:47.966 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:47.967 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182436.766898_99.1", 2022-09-03T05:20:47.967 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:47.968 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:47.968 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:47.969 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.766898Z", 2022-09-03T05:20:47.969 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:47.969 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:47.970 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:47.970 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:20:47.971 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:20:47.971 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:20:47.972 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:20:47.972 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:20:47.973 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:20:47.973 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:20:47.973 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:20:47.974 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:20:47.974 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:20:47.974 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:47.974 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:20:47.975 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:20:47.975 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:47.975 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:20:47.975 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:20:47.976 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:20:47.976 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:20:47.976 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:20:47Z" 2022-09-03T05:20:47.976 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:20:47.978 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662182436.766898_99.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:36.766898Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:20:47Z" } 2022-09-03T05:20:47.978 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662182436.766898_99.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:36.766898Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:20:47.979 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182436.766898_99.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:20:47.979 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182436.766898_99.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:20:47.980 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:20:47.980 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:20:47.981 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:47.981 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:20:47.982 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:20:48.041 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:48.041 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:48.083 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:48.086+0000 7fce96bd8600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:48.084 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:48.087+0000 7fce96bd8600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:20:48.095 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:20:48.096 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:20:48.096 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:20:48.096 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:20:48.097 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:20:48.097 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:20:48.097 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:48.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:48.098 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:20:48.098 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:48.098 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:20:48.099 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:48.099 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:48.099 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:20:48.100 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:48.100 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:20:48.100 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:48.100 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:20:48.101 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:48.101 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:48.101 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:48.101 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:48.102 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:20:48.102 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:48.102 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:48.103 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:20:48.103 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:48.103 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:48.103 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:48.104 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:20:48.104 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:48.104 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:48.104 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:48.105 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:48.105 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:20:48.105 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:48.106 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:48.106 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182436.766898_99.1", 2022-09-03T05:20:48.106 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:48.106 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:48.107 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:48.107 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.766898Z", 2022-09-03T05:20:48.107 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:48.108 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:48.108 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:48.108 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:20:48.108 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:20:48.109 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:20:48.109 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:20:48.109 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:20:48.109 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:20:48.110 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:20:48.110 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:20:48.110 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:20:48.111 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:20:48.111 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:48.111 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:20:48.111 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:20:48.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:48.112 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:20:48.112 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:20:48.112 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:20:48.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:20:48.113 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:20:48Z" 2022-09-03T05:20:48.113 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:20:48.114 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662182436.766898_99.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:36.766898Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:20:48Z" } 2022-09-03T05:20:48.115 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662182436.766898_99.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:36.766898Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:20:48.116 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182436.766898_99.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:20:48.116 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182436.766898_99.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:20:48.117 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:20:48.117 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-10', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:20:48.118 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-10', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:20:48.119 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-10 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:20:48.177 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:48.178 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:48.211 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:48.214+0000 7fd651a4d600 1 bucket sync caught up with source: 2022-09-03T05:20:48.212 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:20:48.212 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:20:48.212 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:48.214+0000 7fd651a4d600 0 bucket checkpoint complete 2022-09-03T05:20:48.223 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-10 zones={test-zone2, test-zone1} 2022-09-03T05:20:48.224 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:48.224 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:48.225 DEBUG:boto:Method: HEAD 2022-09-03T05:20:48.226 DEBUG:boto:Path: /dsgvcy-10/ 2022-09-03T05:20:48.227 DEBUG:boto:Data: 2022-09-03T05:20:48.227 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.228 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:48.228 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.229 DEBUG:boto:Params: {} 2022-09-03T05:20:48.230 DEBUG:boto:Token: None 2022-09-03T05:20:48.230 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/ 2022-09-03T05:20:48.231 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B91Fwfy5MnPCZvU4Fdh46Eit+to= 2022-09-03T05:20:48.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B91Fwfy5MnPCZvU4Fdh46Eit+to=', 'Content-Length': '0'} 2022-09-03T05:20:48.234 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b79c9d2837b3f0bc-006312e430-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.235 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:48.236 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:48.237 DEBUG:boto:Method: HEAD 2022-09-03T05:20:48.238 DEBUG:boto:Path: /dsgvcy-10/ 2022-09-03T05:20:48.238 DEBUG:boto:Data: 2022-09-03T05:20:48.239 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.239 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:48.240 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.240 DEBUG:boto:Params: {} 2022-09-03T05:20:48.241 DEBUG:boto:Token: None 2022-09-03T05:20:48.241 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/ 2022-09-03T05:20:48.242 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B91Fwfy5MnPCZvU4Fdh46Eit+to= 2022-09-03T05:20:48.243 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B91Fwfy5MnPCZvU4Fdh46Eit+to=', 'Content-Length': '0'} 2022-09-03T05:20:48.245 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005c3d2622f4325453-006312e430-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.246 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:20:48.246 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:48.251 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:48.252 DEBUG:boto:path=/dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:48.252 DEBUG:boto:auth_path=/dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:48.253 DEBUG:boto:Method: GET 2022-09-03T05:20:48.253 DEBUG:boto:Path: /dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:48.254 DEBUG:boto:Data: 2022-09-03T05:20:48.254 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.255 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:48.255 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.256 DEBUG:boto:Params: {} 2022-09-03T05:20:48.257 DEBUG:boto:Token: None 2022-09-03T05:20:48.257 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/?versions 2022-09-03T05:20:48.258 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vwJ5oDIw4mb+0y02kBHnZC6nCno= 2022-09-03T05:20:48.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vwJ5oDIw4mb+0y02kBHnZC6nCno=', 'Content-Length': '0'} 2022-09-03T05:20:48.261 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001b4afc4c42c87142-006312e430-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.262 DEBUG:boto:b'dsgvcy-10999false&nulltrue2022-09-03T05:20:36.975Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal:nulltrue2022-09-03T05:20:36.905Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal_myobjnulltrue2022-09-03T05:20:36.861Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormalmyobjnulltrue2022-09-03T05:20:36.801Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:48.264 DEBUG:tasks.rgw_multi.zone_rados:o=& 2022-09-03T05:20:48.265 DEBUG:tasks.rgw_multi.zone_rados:o=: 2022-09-03T05:20:48.265 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2022-09-03T05:20:48.266 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2022-09-03T05:20:48.266 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:20:48.267 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:48.267 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:48.268 DEBUG:boto:path=/dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:48.268 DEBUG:boto:auth_path=/dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:48.269 DEBUG:boto:Method: GET 2022-09-03T05:20:48.269 DEBUG:boto:Path: /dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:48.270 DEBUG:boto:Data: 2022-09-03T05:20:48.270 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.271 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:48.271 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.272 DEBUG:boto:Params: {} 2022-09-03T05:20:48.272 DEBUG:boto:Token: None 2022-09-03T05:20:48.273 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/?versions 2022-09-03T05:20:48.274 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vwJ5oDIw4mb+0y02kBHnZC6nCno= 2022-09-03T05:20:48.274 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vwJ5oDIw4mb+0y02kBHnZC6nCno=', 'Content-Length': '0'} 2022-09-03T05:20:48.281 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a191b214c240aa57-006312e430-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.283 DEBUG:boto:b'dsgvcy-10999false&nulltrue2022-09-03T05:20:36.975Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal:nulltrue2022-09-03T05:20:36.905Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal_myobjnulltrue2022-09-03T05:20:36.861Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormalmyobjnulltrue2022-09-03T05:20:36.801Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:48.284 DEBUG:tasks.rgw_multi.zone_rados:o=& 2022-09-03T05:20:48.284 DEBUG:tasks.rgw_multi.zone_rados:o=: 2022-09-03T05:20:48.285 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2022-09-03T05:20:48.286 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2022-09-03T05:20:48.286 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:48.287 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:48.287 DEBUG:boto:path=/dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:48.288 DEBUG:boto:auth_path=/dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:48.288 DEBUG:boto:Method: GET 2022-09-03T05:20:48.289 DEBUG:boto:Path: /dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:48.290 DEBUG:boto:Data: 2022-09-03T05:20:48.291 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.291 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:48.292 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.293 DEBUG:boto:Params: {} 2022-09-03T05:20:48.294 DEBUG:boto:Token: None 2022-09-03T05:20:48.294 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/?versions 2022-09-03T05:20:48.295 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vwJ5oDIw4mb+0y02kBHnZC6nCno= 2022-09-03T05:20:48.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vwJ5oDIw4mb+0y02kBHnZC6nCno=', 'Content-Length': '0'} 2022-09-03T05:20:48.300 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a762195e1559280d-006312e430-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.301 DEBUG:boto:b'dsgvcy-10999false&nulltrue2022-09-03T05:20:36.975Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal:nulltrue2022-09-03T05:20:36.905Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal_myobjnulltrue2022-09-03T05:20:36.861Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormalmyobjnulltrue2022-09-03T05:20:36.801Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:48.302 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:48.303 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:48.303 DEBUG:boto:path=/dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:48.304 DEBUG:boto:auth_path=/dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:48.304 DEBUG:boto:Method: GET 2022-09-03T05:20:48.305 DEBUG:boto:Path: /dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:48.306 DEBUG:boto:Data: 2022-09-03T05:20:48.306 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.307 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:48.308 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.308 DEBUG:boto:Params: {} 2022-09-03T05:20:48.309 DEBUG:boto:Token: None 2022-09-03T05:20:48.309 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/?versions 2022-09-03T05:20:48.310 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vwJ5oDIw4mb+0y02kBHnZC6nCno= 2022-09-03T05:20:48.311 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vwJ5oDIw4mb+0y02kBHnZC6nCno=', 'Content-Length': '0'} 2022-09-03T05:20:48.313 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e866ff18e25348bf-006312e430-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.314 DEBUG:boto:b'dsgvcy-10999false&nulltrue2022-09-03T05:20:36.975Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal:nulltrue2022-09-03T05:20:36.905Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal_myobjnulltrue2022-09-03T05:20:36.861Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormalmyobjnulltrue2022-09-03T05:20:36.801Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:48.315 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2022-09-03T05:20:48.315 DEBUG:boto:path=/dsgvcy-10/%26 2022-09-03T05:20:48.316 DEBUG:boto:auth_path=/dsgvcy-10/%26 2022-09-03T05:20:48.317 DEBUG:boto:path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.317 DEBUG:boto:auth_path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.318 DEBUG:boto:Method: GET 2022-09-03T05:20:48.319 DEBUG:boto:Path: /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.319 DEBUG:boto:Data: 2022-09-03T05:20:48.320 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.320 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:48.321 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.323 DEBUG:boto:Params: {} 2022-09-03T05:20:48.324 DEBUG:boto:Token: None 2022-09-03T05:20:48.325 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.326 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JcT9kqgEKEQN1X8w60VsxlmoHIc= 2022-09-03T05:20:48.327 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JcT9kqgEKEQN1X8w60VsxlmoHIc=', 'Content-Length': '0'} 2022-09-03T05:20:48.334 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000df9af5586e5aaaee-006312e430-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.335 DEBUG:boto:path=/dsgvcy-10/%26 2022-09-03T05:20:48.336 DEBUG:boto:auth_path=/dsgvcy-10/%26 2022-09-03T05:20:48.337 DEBUG:boto:path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.337 DEBUG:boto:auth_path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.338 DEBUG:boto:Method: GET 2022-09-03T05:20:48.339 DEBUG:boto:Path: /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.339 DEBUG:boto:Data: 2022-09-03T05:20:48.340 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.340 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:48.341 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.341 DEBUG:boto:Params: {} 2022-09-03T05:20:48.342 DEBUG:boto:Token: None 2022-09-03T05:20:48.343 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.344 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JcT9kqgEKEQN1X8w60VsxlmoHIc= 2022-09-03T05:20:48.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JcT9kqgEKEQN1X8w60VsxlmoHIc=', 'Content-Length': '0'} 2022-09-03T05:20:48.347 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f6d4763d11ef33f3-006312e430-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.349 DEBUG:boto:path=/dsgvcy-10/%26 2022-09-03T05:20:48.351 DEBUG:boto:auth_path=/dsgvcy-10/%26 2022-09-03T05:20:48.352 DEBUG:boto:path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.353 DEBUG:boto:auth_path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.354 DEBUG:boto:Method: HEAD 2022-09-03T05:20:48.355 DEBUG:boto:Path: /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.356 DEBUG:boto:Data: 2022-09-03T05:20:48.357 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.357 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:48.359 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.360 DEBUG:boto:Params: {} 2022-09-03T05:20:48.361 DEBUG:boto:Token: None 2022-09-03T05:20:48.362 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.363 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CT0XMUoa8rkbop2FXAVdGcSyfYc= 2022-09-03T05:20:48.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CT0XMUoa8rkbop2FXAVdGcSyfYc=', 'Content-Length': '0'} 2022-09-03T05:20:48.367 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000009fc0873932dbf287-006312e430-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.369 DEBUG:boto:path=/dsgvcy-10/%26 2022-09-03T05:20:48.370 DEBUG:boto:auth_path=/dsgvcy-10/%26 2022-09-03T05:20:48.371 DEBUG:boto:path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.372 DEBUG:boto:auth_path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.373 DEBUG:boto:Method: HEAD 2022-09-03T05:20:48.374 DEBUG:boto:Path: /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.375 DEBUG:boto:Data: 2022-09-03T05:20:48.375 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.377 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:48.378 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.379 DEBUG:boto:Params: {} 2022-09-03T05:20:48.380 DEBUG:boto:Token: None 2022-09-03T05:20:48.381 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.382 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CT0XMUoa8rkbop2FXAVdGcSyfYc= 2022-09-03T05:20:48.383 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CT0XMUoa8rkbop2FXAVdGcSyfYc=', 'Content-Length': '0'} 2022-09-03T05:20:48.386 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a51e1ae0a2775cf3-006312e430-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.387 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2022-09-03T05:20:48.388 DEBUG:boto:path=/dsgvcy-10/%26 2022-09-03T05:20:48.389 DEBUG:boto:auth_path=/dsgvcy-10/%26 2022-09-03T05:20:48.390 DEBUG:boto:path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.391 DEBUG:boto:auth_path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.392 DEBUG:boto:Method: GET 2022-09-03T05:20:48.393 DEBUG:boto:Path: /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.394 DEBUG:boto:Data: 2022-09-03T05:20:48.395 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.396 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:48.397 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.398 DEBUG:boto:Params: {} 2022-09-03T05:20:48.399 DEBUG:boto:Token: None 2022-09-03T05:20:48.400 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.400 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JcT9kqgEKEQN1X8w60VsxlmoHIc= 2022-09-03T05:20:48.401 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JcT9kqgEKEQN1X8w60VsxlmoHIc=', 'Content-Length': '0'} 2022-09-03T05:20:48.403 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000dd36e7638a4a40a8-006312e430-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.404 DEBUG:boto:path=/dsgvcy-10/%26 2022-09-03T05:20:48.404 DEBUG:boto:auth_path=/dsgvcy-10/%26 2022-09-03T05:20:48.405 DEBUG:boto:path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.405 DEBUG:boto:auth_path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.406 DEBUG:boto:Method: GET 2022-09-03T05:20:48.407 DEBUG:boto:Path: /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.407 DEBUG:boto:Data: 2022-09-03T05:20:48.408 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.409 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:48.409 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.410 DEBUG:boto:Params: {} 2022-09-03T05:20:48.410 DEBUG:boto:Token: None 2022-09-03T05:20:48.411 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:48.412 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JcT9kqgEKEQN1X8w60VsxlmoHIc= 2022-09-03T05:20:48.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JcT9kqgEKEQN1X8w60VsxlmoHIc=', 'Content-Length': '0'} 2022-09-03T05:20:48.415 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000428049615ea323d8-006312e430-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.415 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:48.416 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:48.417 DEBUG:boto:path=/dsgvcy-10/?prefix=%26 2022-09-03T05:20:48.418 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=%26 2022-09-03T05:20:48.418 DEBUG:boto:Method: GET 2022-09-03T05:20:48.419 DEBUG:boto:Path: /dsgvcy-10/?prefix=%26 2022-09-03T05:20:48.420 DEBUG:boto:Data: 2022-09-03T05:20:48.421 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.422 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:48.423 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.424 DEBUG:boto:Params: {} 2022-09-03T05:20:48.425 DEBUG:boto:Token: None 2022-09-03T05:20:48.426 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/ 2022-09-03T05:20:48.427 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r2KMpG0rN3GArz7Jb6vMwlG6T2s= 2022-09-03T05:20:48.429 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r2KMpG0rN3GArz7Jb6vMwlG6T2s=', 'Content-Length': '0'} 2022-09-03T05:20:48.431 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a2f991fc51dadb33-006312e430-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.478 DEBUG:boto:b'dsgvcy-10&1000false&2022-09-03T05:20:36.975Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:48.479 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:48.480 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:48.481 DEBUG:boto:path=/dsgvcy-10/?prefix=%26 2022-09-03T05:20:48.481 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=%26 2022-09-03T05:20:48.482 DEBUG:boto:Method: GET 2022-09-03T05:20:48.483 DEBUG:boto:Path: /dsgvcy-10/?prefix=%26 2022-09-03T05:20:48.483 DEBUG:boto:Data: 2022-09-03T05:20:48.484 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.484 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:48.486 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.487 DEBUG:boto:Params: {} 2022-09-03T05:20:48.488 DEBUG:boto:Token: None 2022-09-03T05:20:48.489 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/ 2022-09-03T05:20:48.489 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r2KMpG0rN3GArz7Jb6vMwlG6T2s= 2022-09-03T05:20:48.490 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r2KMpG0rN3GArz7Jb6vMwlG6T2s=', 'Content-Length': '0'} 2022-09-03T05:20:48.493 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000025d2c4bfc99b87f2-006312e430-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.493 DEBUG:boto:b'dsgvcy-10&1000false&2022-09-03T05:20:36.975Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:48.494 DEBUG:boto:path=/dsgvcy-10/%26 2022-09-03T05:20:48.495 DEBUG:boto:auth_path=/dsgvcy-10/%26 2022-09-03T05:20:48.496 DEBUG:boto:Method: HEAD 2022-09-03T05:20:48.496 DEBUG:boto:Path: /dsgvcy-10/%26 2022-09-03T05:20:48.497 DEBUG:boto:Data: 2022-09-03T05:20:48.497 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.499 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:48.499 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.500 DEBUG:boto:Params: {} 2022-09-03T05:20:48.500 DEBUG:boto:Token: None 2022-09-03T05:20:48.501 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/%26 2022-09-03T05:20:48.501 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bla0RLGnqe3BUVaCdqCiGjU3mZ4= 2022-09-03T05:20:48.502 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bla0RLGnqe3BUVaCdqCiGjU3mZ4=', 'Content-Length': '0'} 2022-09-03T05:20:48.504 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c183b83ad75c8764-006312e430-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.505 DEBUG:boto:path=/dsgvcy-10/%26 2022-09-03T05:20:48.506 DEBUG:boto:auth_path=/dsgvcy-10/%26 2022-09-03T05:20:48.507 DEBUG:boto:Method: HEAD 2022-09-03T05:20:48.508 DEBUG:boto:Path: /dsgvcy-10/%26 2022-09-03T05:20:48.508 DEBUG:boto:Data: 2022-09-03T05:20:48.509 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.509 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:48.510 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.511 DEBUG:boto:Params: {} 2022-09-03T05:20:48.511 DEBUG:boto:Token: None 2022-09-03T05:20:48.512 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/%26 2022-09-03T05:20:48.513 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bla0RLGnqe3BUVaCdqCiGjU3mZ4= 2022-09-03T05:20:48.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bla0RLGnqe3BUVaCdqCiGjU3mZ4=', 'Content-Length': '0'} 2022-09-03T05:20:48.515 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f93eea513e1b41db-006312e430-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.516 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2022-09-03T05:20:48.517 DEBUG:boto:path=/dsgvcy-10/%26 2022-09-03T05:20:48.518 DEBUG:boto:auth_path=/dsgvcy-10/%26 2022-09-03T05:20:48.519 DEBUG:boto:Method: GET 2022-09-03T05:20:48.520 DEBUG:boto:Path: /dsgvcy-10/%26 2022-09-03T05:20:48.520 DEBUG:boto:Data: 2022-09-03T05:20:48.521 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.521 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:48.522 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.522 DEBUG:boto:Params: {} 2022-09-03T05:20:48.523 DEBUG:boto:Token: None 2022-09-03T05:20:48.524 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/%26 2022-09-03T05:20:48.524 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MzU0QXYNNF7YrxkHayBKS6OsjaE= 2022-09-03T05:20:48.525 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MzU0QXYNNF7YrxkHayBKS6OsjaE=', 'Content-Length': '0'} 2022-09-03T05:20:48.527 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000066071cdc21d49c7d-006312e430-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.528 DEBUG:boto:path=/dsgvcy-10/%26 2022-09-03T05:20:48.529 DEBUG:boto:auth_path=/dsgvcy-10/%26 2022-09-03T05:20:48.530 DEBUG:boto:Method: GET 2022-09-03T05:20:48.530 DEBUG:boto:Path: /dsgvcy-10/%26 2022-09-03T05:20:48.531 DEBUG:boto:Data: 2022-09-03T05:20:48.532 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.533 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:48.533 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.534 DEBUG:boto:Params: {} 2022-09-03T05:20:48.534 DEBUG:boto:Token: None 2022-09-03T05:20:48.535 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/%26 2022-09-03T05:20:48.536 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MzU0QXYNNF7YrxkHayBKS6OsjaE= 2022-09-03T05:20:48.537 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MzU0QXYNNF7YrxkHayBKS6OsjaE=', 'Content-Length': '0'} 2022-09-03T05:20:48.539 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000087b1c81132c6dea4-006312e430-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.540 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:48.541 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:48.541 DEBUG:boto:path=/dsgvcy-10/?prefix=%26 2022-09-03T05:20:48.542 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=%26 2022-09-03T05:20:48.542 DEBUG:boto:Method: GET 2022-09-03T05:20:48.543 DEBUG:boto:Path: /dsgvcy-10/?prefix=%26 2022-09-03T05:20:48.544 DEBUG:boto:Data: 2022-09-03T05:20:48.544 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.545 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:48.545 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.546 DEBUG:boto:Params: {} 2022-09-03T05:20:48.546 DEBUG:boto:Token: None 2022-09-03T05:20:48.547 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/ 2022-09-03T05:20:48.548 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r2KMpG0rN3GArz7Jb6vMwlG6T2s= 2022-09-03T05:20:48.548 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r2KMpG0rN3GArz7Jb6vMwlG6T2s=', 'Content-Length': '0'} 2022-09-03T05:20:48.550 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a5bff040216ee9b4-006312e430-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.551 DEBUG:boto:b'dsgvcy-10&1000false&2022-09-03T05:20:36.975Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:48.552 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:48.552 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:48.553 DEBUG:boto:path=/dsgvcy-10/?prefix=%26 2022-09-03T05:20:48.554 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=%26 2022-09-03T05:20:48.554 DEBUG:boto:Method: GET 2022-09-03T05:20:48.555 DEBUG:boto:Path: /dsgvcy-10/?prefix=%26 2022-09-03T05:20:48.555 DEBUG:boto:Data: 2022-09-03T05:20:48.556 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.556 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:48.557 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.557 DEBUG:boto:Params: {} 2022-09-03T05:20:48.558 DEBUG:boto:Token: None 2022-09-03T05:20:48.558 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/ 2022-09-03T05:20:48.559 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r2KMpG0rN3GArz7Jb6vMwlG6T2s= 2022-09-03T05:20:48.560 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r2KMpG0rN3GArz7Jb6vMwlG6T2s=', 'Content-Length': '0'} 2022-09-03T05:20:48.562 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a17447f02c6f4e2c-006312e430-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.563 DEBUG:boto:b'dsgvcy-10&1000false&2022-09-03T05:20:36.975Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:48.564 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2022-09-03T05:20:48.564 DEBUG:boto:path=/dsgvcy-10/%3A 2022-09-03T05:20:48.565 DEBUG:boto:auth_path=/dsgvcy-10/%3A 2022-09-03T05:20:48.565 DEBUG:boto:path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.566 DEBUG:boto:auth_path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.566 DEBUG:boto:Method: GET 2022-09-03T05:20:48.567 DEBUG:boto:Path: /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.568 DEBUG:boto:Data: 2022-09-03T05:20:48.568 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.569 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:48.569 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.570 DEBUG:boto:Params: {} 2022-09-03T05:20:48.570 DEBUG:boto:Token: None 2022-09-03T05:20:48.571 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.572 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1hbRUXZ5jvkH/ePuyWYfrIncSlA= 2022-09-03T05:20:48.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1hbRUXZ5jvkH/ePuyWYfrIncSlA=', 'Content-Length': '0'} 2022-09-03T05:20:48.574 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a43202120d8684d0-006312e430-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.575 DEBUG:boto:path=/dsgvcy-10/%3A 2022-09-03T05:20:48.575 DEBUG:boto:auth_path=/dsgvcy-10/%3A 2022-09-03T05:20:48.576 DEBUG:boto:path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.577 DEBUG:boto:auth_path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.577 DEBUG:boto:Method: GET 2022-09-03T05:20:48.578 DEBUG:boto:Path: /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.578 DEBUG:boto:Data: 2022-09-03T05:20:48.579 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.579 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:48.580 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.580 DEBUG:boto:Params: {} 2022-09-03T05:20:48.581 DEBUG:boto:Token: None 2022-09-03T05:20:48.582 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.582 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1hbRUXZ5jvkH/ePuyWYfrIncSlA= 2022-09-03T05:20:48.583 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1hbRUXZ5jvkH/ePuyWYfrIncSlA=', 'Content-Length': '0'} 2022-09-03T05:20:48.585 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007440a0cdacf96363-006312e430-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.626 DEBUG:boto:path=/dsgvcy-10/%3A 2022-09-03T05:20:48.627 DEBUG:boto:auth_path=/dsgvcy-10/%3A 2022-09-03T05:20:48.628 DEBUG:boto:path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.629 DEBUG:boto:auth_path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.629 DEBUG:boto:Method: HEAD 2022-09-03T05:20:48.630 DEBUG:boto:Path: /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.631 DEBUG:boto:Data: 2022-09-03T05:20:48.632 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.633 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:48.633 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.634 DEBUG:boto:Params: {} 2022-09-03T05:20:48.636 DEBUG:boto:Token: None 2022-09-03T05:20:48.637 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.637 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i1whvMWidefH9p8pYsNOnU7zSB4= 2022-09-03T05:20:48.638 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i1whvMWidefH9p8pYsNOnU7zSB4=', 'Content-Length': '0'} 2022-09-03T05:20:48.641 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000097ab87131db62b9b-006312e430-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.642 DEBUG:boto:path=/dsgvcy-10/%3A 2022-09-03T05:20:48.643 DEBUG:boto:auth_path=/dsgvcy-10/%3A 2022-09-03T05:20:48.644 DEBUG:boto:path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.646 DEBUG:boto:auth_path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.646 DEBUG:boto:Method: HEAD 2022-09-03T05:20:48.647 DEBUG:boto:Path: /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.647 DEBUG:boto:Data: 2022-09-03T05:20:48.648 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.648 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:48.649 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.649 DEBUG:boto:Params: {} 2022-09-03T05:20:48.650 DEBUG:boto:Token: None 2022-09-03T05:20:48.650 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.651 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i1whvMWidefH9p8pYsNOnU7zSB4= 2022-09-03T05:20:48.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i1whvMWidefH9p8pYsNOnU7zSB4=', 'Content-Length': '0'} 2022-09-03T05:20:48.654 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000889487387c235778-006312e430-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.654 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2022-09-03T05:20:48.655 DEBUG:boto:path=/dsgvcy-10/%3A 2022-09-03T05:20:48.655 DEBUG:boto:auth_path=/dsgvcy-10/%3A 2022-09-03T05:20:48.656 DEBUG:boto:path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.656 DEBUG:boto:auth_path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.657 DEBUG:boto:Method: GET 2022-09-03T05:20:48.658 DEBUG:boto:Path: /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.658 DEBUG:boto:Data: 2022-09-03T05:20:48.659 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.659 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:48.660 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.660 DEBUG:boto:Params: {} 2022-09-03T05:20:48.661 DEBUG:boto:Token: None 2022-09-03T05:20:48.662 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.663 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1hbRUXZ5jvkH/ePuyWYfrIncSlA= 2022-09-03T05:20:48.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1hbRUXZ5jvkH/ePuyWYfrIncSlA=', 'Content-Length': '0'} 2022-09-03T05:20:48.665 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d8fab9b785860498-006312e430-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.667 DEBUG:boto:path=/dsgvcy-10/%3A 2022-09-03T05:20:48.668 DEBUG:boto:auth_path=/dsgvcy-10/%3A 2022-09-03T05:20:48.668 DEBUG:boto:path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.669 DEBUG:boto:auth_path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.669 DEBUG:boto:Method: GET 2022-09-03T05:20:48.670 DEBUG:boto:Path: /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.670 DEBUG:boto:Data: 2022-09-03T05:20:48.671 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.671 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:48.672 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.672 DEBUG:boto:Params: {} 2022-09-03T05:20:48.673 DEBUG:boto:Token: None 2022-09-03T05:20:48.674 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:48.674 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1hbRUXZ5jvkH/ePuyWYfrIncSlA= 2022-09-03T05:20:48.675 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1hbRUXZ5jvkH/ePuyWYfrIncSlA=', 'Content-Length': '0'} 2022-09-03T05:20:48.677 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000084e732bd39cdb57f-006312e430-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.678 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:48.678 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:48.679 DEBUG:boto:path=/dsgvcy-10/?prefix=%3A 2022-09-03T05:20:48.679 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=%3A 2022-09-03T05:20:48.680 DEBUG:boto:Method: GET 2022-09-03T05:20:48.681 DEBUG:boto:Path: /dsgvcy-10/?prefix=%3A 2022-09-03T05:20:48.681 DEBUG:boto:Data: 2022-09-03T05:20:48.682 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.682 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:48.683 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.683 DEBUG:boto:Params: {} 2022-09-03T05:20:48.684 DEBUG:boto:Token: None 2022-09-03T05:20:48.684 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/ 2022-09-03T05:20:48.685 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r2KMpG0rN3GArz7Jb6vMwlG6T2s= 2022-09-03T05:20:48.686 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r2KMpG0rN3GArz7Jb6vMwlG6T2s=', 'Content-Length': '0'} 2022-09-03T05:20:48.688 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cd3f302ca9ee1561-006312e430-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.730 DEBUG:boto:b'dsgvcy-10:1000false:2022-09-03T05:20:36.905Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:48.732 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:48.733 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:48.734 DEBUG:boto:path=/dsgvcy-10/?prefix=%3A 2022-09-03T05:20:48.735 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=%3A 2022-09-03T05:20:48.736 DEBUG:boto:Method: GET 2022-09-03T05:20:48.737 DEBUG:boto:Path: /dsgvcy-10/?prefix=%3A 2022-09-03T05:20:48.738 DEBUG:boto:Data: 2022-09-03T05:20:48.739 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.739 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:48.740 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.741 DEBUG:boto:Params: {} 2022-09-03T05:20:48.742 DEBUG:boto:Token: None 2022-09-03T05:20:48.743 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/ 2022-09-03T05:20:48.744 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r2KMpG0rN3GArz7Jb6vMwlG6T2s= 2022-09-03T05:20:48.745 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r2KMpG0rN3GArz7Jb6vMwlG6T2s=', 'Content-Length': '0'} 2022-09-03T05:20:48.748 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000021284a22b395bac7-006312e430-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.749 DEBUG:boto:b'dsgvcy-10:1000false:2022-09-03T05:20:36.905Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:48.751 DEBUG:boto:path=/dsgvcy-10/%3A 2022-09-03T05:20:48.752 DEBUG:boto:auth_path=/dsgvcy-10/%3A 2022-09-03T05:20:48.753 DEBUG:boto:Method: HEAD 2022-09-03T05:20:48.754 DEBUG:boto:Path: /dsgvcy-10/%3A 2022-09-03T05:20:48.755 DEBUG:boto:Data: 2022-09-03T05:20:48.755 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.756 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:48.756 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.757 DEBUG:boto:Params: {} 2022-09-03T05:20:48.759 DEBUG:boto:Token: None 2022-09-03T05:20:48.760 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/%3A 2022-09-03T05:20:48.761 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mfYGAG8vxTsWqfM+wibAgbsaADI= 2022-09-03T05:20:48.762 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mfYGAG8vxTsWqfM+wibAgbsaADI=', 'Content-Length': '0'} 2022-09-03T05:20:48.765 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ab85ad827c70ac59-006312e430-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.766 DEBUG:boto:path=/dsgvcy-10/%3A 2022-09-03T05:20:48.767 DEBUG:boto:auth_path=/dsgvcy-10/%3A 2022-09-03T05:20:48.768 DEBUG:boto:Method: HEAD 2022-09-03T05:20:48.769 DEBUG:boto:Path: /dsgvcy-10/%3A 2022-09-03T05:20:48.770 DEBUG:boto:Data: 2022-09-03T05:20:48.771 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.771 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:48.772 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.772 DEBUG:boto:Params: {} 2022-09-03T05:20:48.773 DEBUG:boto:Token: None 2022-09-03T05:20:48.774 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/%3A 2022-09-03T05:20:48.775 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mfYGAG8vxTsWqfM+wibAgbsaADI= 2022-09-03T05:20:48.775 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mfYGAG8vxTsWqfM+wibAgbsaADI=', 'Content-Length': '0'} 2022-09-03T05:20:48.777 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002ca1a5eb9ea229a6-006312e430-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.778 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2022-09-03T05:20:48.779 DEBUG:boto:path=/dsgvcy-10/%3A 2022-09-03T05:20:48.780 DEBUG:boto:auth_path=/dsgvcy-10/%3A 2022-09-03T05:20:48.781 DEBUG:boto:Method: GET 2022-09-03T05:20:48.781 DEBUG:boto:Path: /dsgvcy-10/%3A 2022-09-03T05:20:48.782 DEBUG:boto:Data: 2022-09-03T05:20:48.783 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.783 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:48.784 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.785 DEBUG:boto:Params: {} 2022-09-03T05:20:48.786 DEBUG:boto:Token: None 2022-09-03T05:20:48.788 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/%3A 2022-09-03T05:20:48.789 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FnhHtMNvAxejYShpZDG/xOYtES4= 2022-09-03T05:20:48.790 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FnhHtMNvAxejYShpZDG/xOYtES4=', 'Content-Length': '0'} 2022-09-03T05:20:48.793 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e948568073308411-006312e430-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.794 DEBUG:boto:path=/dsgvcy-10/%3A 2022-09-03T05:20:48.795 DEBUG:boto:auth_path=/dsgvcy-10/%3A 2022-09-03T05:20:48.796 DEBUG:boto:Method: GET 2022-09-03T05:20:48.797 DEBUG:boto:Path: /dsgvcy-10/%3A 2022-09-03T05:20:48.798 DEBUG:boto:Data: 2022-09-03T05:20:48.799 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.800 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:48.800 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.801 DEBUG:boto:Params: {} 2022-09-03T05:20:48.802 DEBUG:boto:Token: None 2022-09-03T05:20:48.803 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/%3A 2022-09-03T05:20:48.804 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FnhHtMNvAxejYShpZDG/xOYtES4= 2022-09-03T05:20:48.805 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FnhHtMNvAxejYShpZDG/xOYtES4=', 'Content-Length': '0'} 2022-09-03T05:20:48.809 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000000d2b0dd256850581-006312e430-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.811 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:48.811 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:48.812 DEBUG:boto:path=/dsgvcy-10/?prefix=%3A 2022-09-03T05:20:48.812 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=%3A 2022-09-03T05:20:48.813 DEBUG:boto:Method: GET 2022-09-03T05:20:48.813 DEBUG:boto:Path: /dsgvcy-10/?prefix=%3A 2022-09-03T05:20:48.814 DEBUG:boto:Data: 2022-09-03T05:20:48.815 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.816 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:48.817 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.818 DEBUG:boto:Params: {} 2022-09-03T05:20:48.819 DEBUG:boto:Token: None 2022-09-03T05:20:48.821 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/ 2022-09-03T05:20:48.822 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r2KMpG0rN3GArz7Jb6vMwlG6T2s= 2022-09-03T05:20:48.823 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r2KMpG0rN3GArz7Jb6vMwlG6T2s=', 'Content-Length': '0'} 2022-09-03T05:20:48.831 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000028a762fce80ac121-006312e430-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.832 DEBUG:boto:b'dsgvcy-10:1000false:2022-09-03T05:20:36.905Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:48.833 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:48.834 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:48.835 DEBUG:boto:path=/dsgvcy-10/?prefix=%3A 2022-09-03T05:20:48.835 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=%3A 2022-09-03T05:20:48.836 DEBUG:boto:Method: GET 2022-09-03T05:20:48.836 DEBUG:boto:Path: /dsgvcy-10/?prefix=%3A 2022-09-03T05:20:48.837 DEBUG:boto:Data: 2022-09-03T05:20:48.837 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.838 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:48.838 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.839 DEBUG:boto:Params: {} 2022-09-03T05:20:48.840 DEBUG:boto:Token: None 2022-09-03T05:20:48.840 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/ 2022-09-03T05:20:48.841 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r2KMpG0rN3GArz7Jb6vMwlG6T2s= 2022-09-03T05:20:48.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r2KMpG0rN3GArz7Jb6vMwlG6T2s=', 'Content-Length': '0'} 2022-09-03T05:20:48.844 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a31172b790c1c3ff-006312e430-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.845 DEBUG:boto:b'dsgvcy-10:1000false:2022-09-03T05:20:36.905Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:48.846 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2022-09-03T05:20:48.846 DEBUG:boto:path=/dsgvcy-10/_myobj 2022-09-03T05:20:48.848 DEBUG:boto:auth_path=/dsgvcy-10/_myobj 2022-09-03T05:20:48.848 DEBUG:boto:path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.849 DEBUG:boto:auth_path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.850 DEBUG:boto:Method: GET 2022-09-03T05:20:48.851 DEBUG:boto:Path: /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.852 DEBUG:boto:Data: 2022-09-03T05:20:48.852 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.853 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:48.853 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.854 DEBUG:boto:Params: {} 2022-09-03T05:20:48.855 DEBUG:boto:Token: None 2022-09-03T05:20:48.855 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.856 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ziWOPqvFQFsobgE6LgOkRfnMNXc= 2022-09-03T05:20:48.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ziWOPqvFQFsobgE6LgOkRfnMNXc=', 'Content-Length': '0'} 2022-09-03T05:20:48.859 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000053e1a11cb4ac495-006312e430-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.860 DEBUG:boto:path=/dsgvcy-10/_myobj 2022-09-03T05:20:48.862 DEBUG:boto:auth_path=/dsgvcy-10/_myobj 2022-09-03T05:20:48.863 DEBUG:boto:path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.864 DEBUG:boto:auth_path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.865 DEBUG:boto:Method: GET 2022-09-03T05:20:48.866 DEBUG:boto:Path: /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.867 DEBUG:boto:Data: 2022-09-03T05:20:48.868 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.869 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:48.870 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.871 DEBUG:boto:Params: {} 2022-09-03T05:20:48.872 DEBUG:boto:Token: None 2022-09-03T05:20:48.873 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.874 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ziWOPqvFQFsobgE6LgOkRfnMNXc= 2022-09-03T05:20:48.875 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ziWOPqvFQFsobgE6LgOkRfnMNXc=', 'Content-Length': '0'} 2022-09-03T05:20:48.878 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e79b8759893bac13-006312e430-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.878 DEBUG:boto:path=/dsgvcy-10/_myobj 2022-09-03T05:20:48.879 DEBUG:boto:auth_path=/dsgvcy-10/_myobj 2022-09-03T05:20:48.880 DEBUG:boto:path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.880 DEBUG:boto:auth_path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.881 DEBUG:boto:Method: HEAD 2022-09-03T05:20:48.881 DEBUG:boto:Path: /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.882 DEBUG:boto:Data: 2022-09-03T05:20:48.882 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.883 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:48.883 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.884 DEBUG:boto:Params: {} 2022-09-03T05:20:48.884 DEBUG:boto:Token: None 2022-09-03T05:20:48.885 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.885 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UgXxnN4/e0Yg51s1WCegJNCCyE0= 2022-09-03T05:20:48.886 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UgXxnN4/e0Yg51s1WCegJNCCyE0=', 'Content-Length': '0'} 2022-09-03T05:20:48.888 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000028dfa705c31ec54f-006312e430-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.889 DEBUG:boto:path=/dsgvcy-10/_myobj 2022-09-03T05:20:48.889 DEBUG:boto:auth_path=/dsgvcy-10/_myobj 2022-09-03T05:20:48.890 DEBUG:boto:path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.890 DEBUG:boto:auth_path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.891 DEBUG:boto:Method: HEAD 2022-09-03T05:20:48.891 DEBUG:boto:Path: /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.892 DEBUG:boto:Data: 2022-09-03T05:20:48.892 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.893 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:48.893 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.894 DEBUG:boto:Params: {} 2022-09-03T05:20:48.894 DEBUG:boto:Token: None 2022-09-03T05:20:48.895 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.895 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UgXxnN4/e0Yg51s1WCegJNCCyE0= 2022-09-03T05:20:48.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UgXxnN4/e0Yg51s1WCegJNCCyE0=', 'Content-Length': '0'} 2022-09-03T05:20:48.898 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000085474f7504482b12-006312e430-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.898 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2022-09-03T05:20:48.899 DEBUG:boto:path=/dsgvcy-10/_myobj 2022-09-03T05:20:48.900 DEBUG:boto:auth_path=/dsgvcy-10/_myobj 2022-09-03T05:20:48.900 DEBUG:boto:path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.901 DEBUG:boto:auth_path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.901 DEBUG:boto:Method: GET 2022-09-03T05:20:48.902 DEBUG:boto:Path: /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.902 DEBUG:boto:Data: 2022-09-03T05:20:48.903 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.903 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:48.904 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.905 DEBUG:boto:Params: {} 2022-09-03T05:20:48.905 DEBUG:boto:Token: None 2022-09-03T05:20:48.906 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.906 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ziWOPqvFQFsobgE6LgOkRfnMNXc= 2022-09-03T05:20:48.907 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ziWOPqvFQFsobgE6LgOkRfnMNXc=', 'Content-Length': '0'} 2022-09-03T05:20:48.909 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000000b4506cc946ee9d2-006312e430-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.910 DEBUG:boto:path=/dsgvcy-10/_myobj 2022-09-03T05:20:48.910 DEBUG:boto:auth_path=/dsgvcy-10/_myobj 2022-09-03T05:20:48.911 DEBUG:boto:path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.911 DEBUG:boto:auth_path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.912 DEBUG:boto:Method: GET 2022-09-03T05:20:48.912 DEBUG:boto:Path: /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.913 DEBUG:boto:Data: 2022-09-03T05:20:48.913 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.914 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:48.914 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.915 DEBUG:boto:Params: {} 2022-09-03T05:20:48.916 DEBUG:boto:Token: None 2022-09-03T05:20:48.916 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:48.917 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ziWOPqvFQFsobgE6LgOkRfnMNXc= 2022-09-03T05:20:48.917 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ziWOPqvFQFsobgE6LgOkRfnMNXc=', 'Content-Length': '0'} 2022-09-03T05:20:48.920 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000060b84dd4451ba6bb-006312e430-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.920 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:48.921 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:48.922 DEBUG:boto:path=/dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:48.922 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:48.923 DEBUG:boto:Method: GET 2022-09-03T05:20:48.923 DEBUG:boto:Path: /dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:48.924 DEBUG:boto:Data: 2022-09-03T05:20:48.924 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.925 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:48.925 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.926 DEBUG:boto:Params: {} 2022-09-03T05:20:48.926 DEBUG:boto:Token: None 2022-09-03T05:20:48.927 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/ 2022-09-03T05:20:48.928 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r2KMpG0rN3GArz7Jb6vMwlG6T2s= 2022-09-03T05:20:48.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r2KMpG0rN3GArz7Jb6vMwlG6T2s=', 'Content-Length': '0'} 2022-09-03T05:20:48.931 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006958824f3e45cc22-006312e430-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.931 DEBUG:boto:b'dsgvcy-10_myobj1000false_myobj2022-09-03T05:20:36.861Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:48.932 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:48.933 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:48.933 DEBUG:boto:path=/dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:48.934 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:48.934 DEBUG:boto:Method: GET 2022-09-03T05:20:48.935 DEBUG:boto:Path: /dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:48.935 DEBUG:boto:Data: 2022-09-03T05:20:48.936 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.936 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:48.937 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.937 DEBUG:boto:Params: {} 2022-09-03T05:20:48.938 DEBUG:boto:Token: None 2022-09-03T05:20:48.939 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/ 2022-09-03T05:20:48.939 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r2KMpG0rN3GArz7Jb6vMwlG6T2s= 2022-09-03T05:20:48.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r2KMpG0rN3GArz7Jb6vMwlG6T2s=', 'Content-Length': '0'} 2022-09-03T05:20:48.942 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002ad9ed2367586603-006312e430-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.943 DEBUG:boto:b'dsgvcy-10_myobj1000false_myobj2022-09-03T05:20:36.861Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:48.944 DEBUG:boto:path=/dsgvcy-10/_myobj 2022-09-03T05:20:48.944 DEBUG:boto:auth_path=/dsgvcy-10/_myobj 2022-09-03T05:20:48.945 DEBUG:boto:Method: HEAD 2022-09-03T05:20:48.945 DEBUG:boto:Path: /dsgvcy-10/_myobj 2022-09-03T05:20:48.946 DEBUG:boto:Data: 2022-09-03T05:20:48.946 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.947 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:48.947 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.948 DEBUG:boto:Params: {} 2022-09-03T05:20:48.949 DEBUG:boto:Token: None 2022-09-03T05:20:48.949 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/_myobj 2022-09-03T05:20:48.950 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BUiVgSFzNC1SVsQghjynp0TKrPE= 2022-09-03T05:20:48.951 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BUiVgSFzNC1SVsQghjynp0TKrPE=', 'Content-Length': '0'} 2022-09-03T05:20:48.952 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b86e29d12f41156d-006312e430-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.953 DEBUG:boto:path=/dsgvcy-10/_myobj 2022-09-03T05:20:48.954 DEBUG:boto:auth_path=/dsgvcy-10/_myobj 2022-09-03T05:20:48.954 DEBUG:boto:Method: HEAD 2022-09-03T05:20:48.955 DEBUG:boto:Path: /dsgvcy-10/_myobj 2022-09-03T05:20:48.955 DEBUG:boto:Data: 2022-09-03T05:20:48.956 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.956 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:48.957 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.957 DEBUG:boto:Params: {} 2022-09-03T05:20:48.958 DEBUG:boto:Token: None 2022-09-03T05:20:48.958 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/_myobj 2022-09-03T05:20:48.959 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BUiVgSFzNC1SVsQghjynp0TKrPE= 2022-09-03T05:20:48.960 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BUiVgSFzNC1SVsQghjynp0TKrPE=', 'Content-Length': '0'} 2022-09-03T05:20:48.961 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000058e85e181b52dcba-006312e430-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:48.962 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2022-09-03T05:20:48.963 DEBUG:boto:path=/dsgvcy-10/_myobj 2022-09-03T05:20:48.963 DEBUG:boto:auth_path=/dsgvcy-10/_myobj 2022-09-03T05:20:48.964 DEBUG:boto:Method: GET 2022-09-03T05:20:48.964 DEBUG:boto:Path: /dsgvcy-10/_myobj 2022-09-03T05:20:48.965 DEBUG:boto:Data: 2022-09-03T05:20:48.965 DEBUG:boto:Headers: {} 2022-09-03T05:20:48.966 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:48.966 DEBUG:boto:Port: 8000 2022-09-03T05:20:48.967 DEBUG:boto:Params: {} 2022-09-03T05:20:48.967 DEBUG:boto:Token: None 2022-09-03T05:20:48.968 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:48 GMT /dsgvcy-10/_myobj 2022-09-03T05:20:48.969 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5BYdhzZThEnSrlu/Txc6gKng1NA= 2022-09-03T05:20:48.969 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5BYdhzZThEnSrlu/Txc6gKng1NA=', 'Content-Length': '0'} 2022-09-03T05:20:48.971 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f41e6e9e236aa035-006312e430-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.014 DEBUG:boto:path=/dsgvcy-10/_myobj 2022-09-03T05:20:49.015 DEBUG:boto:auth_path=/dsgvcy-10/_myobj 2022-09-03T05:20:49.015 DEBUG:boto:Method: GET 2022-09-03T05:20:49.016 DEBUG:boto:Path: /dsgvcy-10/_myobj 2022-09-03T05:20:49.017 DEBUG:boto:Data: 2022-09-03T05:20:49.018 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.018 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:49.019 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.020 DEBUG:boto:Params: {} 2022-09-03T05:20:49.020 DEBUG:boto:Token: None 2022-09-03T05:20:49.021 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/_myobj 2022-09-03T05:20:49.022 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GuaLgosn9GcVS2Ur+xPxnfnuf4U= 2022-09-03T05:20:49.022 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GuaLgosn9GcVS2Ur+xPxnfnuf4U=', 'Content-Length': '0'} 2022-09-03T05:20:49.024 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ebffc9399bec5b54-006312e431-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.026 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:49.026 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:49.027 DEBUG:boto:path=/dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:49.027 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:49.028 DEBUG:boto:Method: GET 2022-09-03T05:20:49.028 DEBUG:boto:Path: /dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:49.029 DEBUG:boto:Data: 2022-09-03T05:20:49.029 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.030 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:49.031 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.032 DEBUG:boto:Params: {} 2022-09-03T05:20:49.032 DEBUG:boto:Token: None 2022-09-03T05:20:49.033 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/ 2022-09-03T05:20:49.034 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uWIRBtV5fk0bHr6mNUKUSO6dRME= 2022-09-03T05:20:49.034 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uWIRBtV5fk0bHr6mNUKUSO6dRME=', 'Content-Length': '0'} 2022-09-03T05:20:49.037 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000038c98d698da09a8a-006312e431-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.038 DEBUG:boto:b'dsgvcy-10_myobj1000false_myobj2022-09-03T05:20:36.861Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:49.039 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:49.039 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:49.040 DEBUG:boto:path=/dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:49.041 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:49.042 DEBUG:boto:Method: GET 2022-09-03T05:20:49.042 DEBUG:boto:Path: /dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:49.043 DEBUG:boto:Data: 2022-09-03T05:20:49.043 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.044 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:49.044 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.045 DEBUG:boto:Params: {} 2022-09-03T05:20:49.046 DEBUG:boto:Token: None 2022-09-03T05:20:49.046 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/ 2022-09-03T05:20:49.047 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uWIRBtV5fk0bHr6mNUKUSO6dRME= 2022-09-03T05:20:49.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uWIRBtV5fk0bHr6mNUKUSO6dRME=', 'Content-Length': '0'} 2022-09-03T05:20:49.050 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002aa774b0851ebb91-006312e431-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.051 DEBUG:boto:b'dsgvcy-10_myobj1000false_myobj2022-09-03T05:20:36.861Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:49.053 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:20:49.054 DEBUG:boto:path=/dsgvcy-10/myobj 2022-09-03T05:20:49.054 DEBUG:boto:auth_path=/dsgvcy-10/myobj 2022-09-03T05:20:49.055 DEBUG:boto:path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.055 DEBUG:boto:auth_path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.056 DEBUG:boto:Method: GET 2022-09-03T05:20:49.057 DEBUG:boto:Path: /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.057 DEBUG:boto:Data: 2022-09-03T05:20:49.058 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.058 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:49.059 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.060 DEBUG:boto:Params: {} 2022-09-03T05:20:49.060 DEBUG:boto:Token: None 2022-09-03T05:20:49.061 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.062 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FH9x+6QzwOrWFEBUB5V4uWCT2O4= 2022-09-03T05:20:49.063 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FH9x+6QzwOrWFEBUB5V4uWCT2O4=', 'Content-Length': '0'} 2022-09-03T05:20:49.065 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f5cd99b24318935d-006312e431-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.066 DEBUG:boto:path=/dsgvcy-10/myobj 2022-09-03T05:20:49.067 DEBUG:boto:auth_path=/dsgvcy-10/myobj 2022-09-03T05:20:49.068 DEBUG:boto:path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.069 DEBUG:boto:auth_path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.070 DEBUG:boto:Method: GET 2022-09-03T05:20:49.071 DEBUG:boto:Path: /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.072 DEBUG:boto:Data: 2022-09-03T05:20:49.073 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.074 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:49.075 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.076 DEBUG:boto:Params: {} 2022-09-03T05:20:49.077 DEBUG:boto:Token: None 2022-09-03T05:20:49.078 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.079 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FH9x+6QzwOrWFEBUB5V4uWCT2O4= 2022-09-03T05:20:49.080 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FH9x+6QzwOrWFEBUB5V4uWCT2O4=', 'Content-Length': '0'} 2022-09-03T05:20:49.083 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ff780361426fb6dd-006312e431-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.126 DEBUG:boto:path=/dsgvcy-10/myobj 2022-09-03T05:20:49.127 DEBUG:boto:auth_path=/dsgvcy-10/myobj 2022-09-03T05:20:49.128 DEBUG:boto:path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.129 DEBUG:boto:auth_path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.130 DEBUG:boto:Method: HEAD 2022-09-03T05:20:49.131 DEBUG:boto:Path: /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.132 DEBUG:boto:Data: 2022-09-03T05:20:49.133 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.134 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:49.135 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.136 DEBUG:boto:Params: {} 2022-09-03T05:20:49.137 DEBUG:boto:Token: None 2022-09-03T05:20:49.138 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.139 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qSXQDEXAz3JxDf4jcdB7pQ8JO48= 2022-09-03T05:20:49.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qSXQDEXAz3JxDf4jcdB7pQ8JO48=', 'Content-Length': '0'} 2022-09-03T05:20:49.142 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000bdcc3c0ddfc755eb-006312e431-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.144 DEBUG:boto:path=/dsgvcy-10/myobj 2022-09-03T05:20:49.145 DEBUG:boto:auth_path=/dsgvcy-10/myobj 2022-09-03T05:20:49.146 DEBUG:boto:path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.147 DEBUG:boto:auth_path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.147 DEBUG:boto:Method: HEAD 2022-09-03T05:20:49.148 DEBUG:boto:Path: /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.149 DEBUG:boto:Data: 2022-09-03T05:20:49.150 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.151 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:49.152 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.152 DEBUG:boto:Params: {} 2022-09-03T05:20:49.153 DEBUG:boto:Token: None 2022-09-03T05:20:49.154 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.155 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qSXQDEXAz3JxDf4jcdB7pQ8JO48= 2022-09-03T05:20:49.157 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qSXQDEXAz3JxDf4jcdB7pQ8JO48=', 'Content-Length': '0'} 2022-09-03T05:20:49.161 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c054f21190704354-006312e431-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.163 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:20:49.163 DEBUG:boto:path=/dsgvcy-10/myobj 2022-09-03T05:20:49.164 DEBUG:boto:auth_path=/dsgvcy-10/myobj 2022-09-03T05:20:49.164 DEBUG:boto:path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.165 DEBUG:boto:auth_path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.166 DEBUG:boto:Method: GET 2022-09-03T05:20:49.167 DEBUG:boto:Path: /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.167 DEBUG:boto:Data: 2022-09-03T05:20:49.168 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.169 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:49.169 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.170 DEBUG:boto:Params: {} 2022-09-03T05:20:49.171 DEBUG:boto:Token: None 2022-09-03T05:20:49.171 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.172 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FH9x+6QzwOrWFEBUB5V4uWCT2O4= 2022-09-03T05:20:49.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FH9x+6QzwOrWFEBUB5V4uWCT2O4=', 'Content-Length': '0'} 2022-09-03T05:20:49.175 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000668417923a403799-006312e431-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.176 DEBUG:boto:path=/dsgvcy-10/myobj 2022-09-03T05:20:49.177 DEBUG:boto:auth_path=/dsgvcy-10/myobj 2022-09-03T05:20:49.178 DEBUG:boto:path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.179 DEBUG:boto:auth_path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.180 DEBUG:boto:Method: GET 2022-09-03T05:20:49.181 DEBUG:boto:Path: /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.182 DEBUG:boto:Data: 2022-09-03T05:20:49.183 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.184 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:49.185 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.186 DEBUG:boto:Params: {} 2022-09-03T05:20:49.187 DEBUG:boto:Token: None 2022-09-03T05:20:49.188 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:49.189 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FH9x+6QzwOrWFEBUB5V4uWCT2O4= 2022-09-03T05:20:49.190 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FH9x+6QzwOrWFEBUB5V4uWCT2O4=', 'Content-Length': '0'} 2022-09-03T05:20:49.192 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007e556ecebc67beaa-006312e431-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.194 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:49.195 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:49.196 DEBUG:boto:path=/dsgvcy-10/?prefix=myobj 2022-09-03T05:20:49.197 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=myobj 2022-09-03T05:20:49.198 DEBUG:boto:Method: GET 2022-09-03T05:20:49.199 DEBUG:boto:Path: /dsgvcy-10/?prefix=myobj 2022-09-03T05:20:49.200 DEBUG:boto:Data: 2022-09-03T05:20:49.200 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.201 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:49.202 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.203 DEBUG:boto:Params: {} 2022-09-03T05:20:49.205 DEBUG:boto:Token: None 2022-09-03T05:20:49.206 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/ 2022-09-03T05:20:49.207 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uWIRBtV5fk0bHr6mNUKUSO6dRME= 2022-09-03T05:20:49.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uWIRBtV5fk0bHr6mNUKUSO6dRME=', 'Content-Length': '0'} 2022-09-03T05:20:49.211 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000010d5ebc9557e97aa-006312e431-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.254 DEBUG:boto:b'dsgvcy-10myobj1000falsemyobj2022-09-03T05:20:36.801Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:49.256 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:49.257 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:49.258 DEBUG:boto:path=/dsgvcy-10/?prefix=myobj 2022-09-03T05:20:49.260 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=myobj 2022-09-03T05:20:49.261 DEBUG:boto:Method: GET 2022-09-03T05:20:49.262 DEBUG:boto:Path: /dsgvcy-10/?prefix=myobj 2022-09-03T05:20:49.262 DEBUG:boto:Data: 2022-09-03T05:20:49.264 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.265 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:49.265 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.267 DEBUG:boto:Params: {} 2022-09-03T05:20:49.268 DEBUG:boto:Token: None 2022-09-03T05:20:49.269 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/ 2022-09-03T05:20:49.270 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uWIRBtV5fk0bHr6mNUKUSO6dRME= 2022-09-03T05:20:49.271 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uWIRBtV5fk0bHr6mNUKUSO6dRME=', 'Content-Length': '0'} 2022-09-03T05:20:49.274 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000048c828baf6b4c84e-006312e431-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.275 DEBUG:boto:b'dsgvcy-10myobj1000falsemyobj2022-09-03T05:20:36.801Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:49.276 DEBUG:boto:path=/dsgvcy-10/myobj 2022-09-03T05:20:49.277 DEBUG:boto:auth_path=/dsgvcy-10/myobj 2022-09-03T05:20:49.278 DEBUG:boto:Method: HEAD 2022-09-03T05:20:49.279 DEBUG:boto:Path: /dsgvcy-10/myobj 2022-09-03T05:20:49.279 DEBUG:boto:Data: 2022-09-03T05:20:49.280 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.280 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:49.281 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.281 DEBUG:boto:Params: {} 2022-09-03T05:20:49.282 DEBUG:boto:Token: None 2022-09-03T05:20:49.283 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/myobj 2022-09-03T05:20:49.284 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ugoltqphOqgb5siYY8AhVl7WmLU= 2022-09-03T05:20:49.285 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ugoltqphOqgb5siYY8AhVl7WmLU=', 'Content-Length': '0'} 2022-09-03T05:20:49.288 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000003750462e111e8f54-006312e431-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.288 DEBUG:boto:path=/dsgvcy-10/myobj 2022-09-03T05:20:49.289 DEBUG:boto:auth_path=/dsgvcy-10/myobj 2022-09-03T05:20:49.290 DEBUG:boto:Method: HEAD 2022-09-03T05:20:49.290 DEBUG:boto:Path: /dsgvcy-10/myobj 2022-09-03T05:20:49.291 DEBUG:boto:Data: 2022-09-03T05:20:49.291 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.292 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:49.292 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.293 DEBUG:boto:Params: {} 2022-09-03T05:20:49.293 DEBUG:boto:Token: None 2022-09-03T05:20:49.294 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/myobj 2022-09-03T05:20:49.295 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ugoltqphOqgb5siYY8AhVl7WmLU= 2022-09-03T05:20:49.295 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ugoltqphOqgb5siYY8AhVl7WmLU=', 'Content-Length': '0'} 2022-09-03T05:20:49.298 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ae1bc28588934ade-006312e431-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.298 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:20:49.299 DEBUG:boto:path=/dsgvcy-10/myobj 2022-09-03T05:20:49.299 DEBUG:boto:auth_path=/dsgvcy-10/myobj 2022-09-03T05:20:49.300 DEBUG:boto:Method: GET 2022-09-03T05:20:49.300 DEBUG:boto:Path: /dsgvcy-10/myobj 2022-09-03T05:20:49.301 DEBUG:boto:Data: 2022-09-03T05:20:49.302 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.302 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:49.303 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.303 DEBUG:boto:Params: {} 2022-09-03T05:20:49.304 DEBUG:boto:Token: None 2022-09-03T05:20:49.304 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/myobj 2022-09-03T05:20:49.305 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pOr5n/5RlIja834tZuSwqNLZ9RE= 2022-09-03T05:20:49.306 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pOr5n/5RlIja834tZuSwqNLZ9RE=', 'Content-Length': '0'} 2022-09-03T05:20:49.309 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000cbb29d5748286193-006312e431-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.311 DEBUG:boto:path=/dsgvcy-10/myobj 2022-09-03T05:20:49.312 DEBUG:boto:auth_path=/dsgvcy-10/myobj 2022-09-03T05:20:49.313 DEBUG:boto:Method: GET 2022-09-03T05:20:49.314 DEBUG:boto:Path: /dsgvcy-10/myobj 2022-09-03T05:20:49.315 DEBUG:boto:Data: 2022-09-03T05:20:49.316 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.317 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:49.317 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.318 DEBUG:boto:Params: {} 2022-09-03T05:20:49.319 DEBUG:boto:Token: None 2022-09-03T05:20:49.320 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/myobj 2022-09-03T05:20:49.321 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pOr5n/5RlIja834tZuSwqNLZ9RE= 2022-09-03T05:20:49.322 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pOr5n/5RlIja834tZuSwqNLZ9RE=', 'Content-Length': '0'} 2022-09-03T05:20:49.324 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000d7724fe6c5f1bb41-006312e431-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.325 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:49.326 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:49.327 DEBUG:boto:path=/dsgvcy-10/?prefix=myobj 2022-09-03T05:20:49.327 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=myobj 2022-09-03T05:20:49.328 DEBUG:boto:Method: GET 2022-09-03T05:20:49.328 DEBUG:boto:Path: /dsgvcy-10/?prefix=myobj 2022-09-03T05:20:49.329 DEBUG:boto:Data: 2022-09-03T05:20:49.329 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.330 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:20:49.331 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.331 DEBUG:boto:Params: {} 2022-09-03T05:20:49.332 DEBUG:boto:Token: None 2022-09-03T05:20:49.332 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/ 2022-09-03T05:20:49.335 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uWIRBtV5fk0bHr6mNUKUSO6dRME= 2022-09-03T05:20:49.336 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uWIRBtV5fk0bHr6mNUKUSO6dRME=', 'Content-Length': '0'} 2022-09-03T05:20:49.339 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e2acd662f7ae5c43-006312e431-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.340 DEBUG:boto:b'dsgvcy-10myobj1000falsemyobj2022-09-03T05:20:36.801Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:49.341 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:49.342 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:49.343 DEBUG:boto:path=/dsgvcy-10/?prefix=myobj 2022-09-03T05:20:49.344 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=myobj 2022-09-03T05:20:49.345 DEBUG:boto:Method: GET 2022-09-03T05:20:49.346 DEBUG:boto:Path: /dsgvcy-10/?prefix=myobj 2022-09-03T05:20:49.347 DEBUG:boto:Data: 2022-09-03T05:20:49.348 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.349 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:49.350 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.351 DEBUG:boto:Params: {} 2022-09-03T05:20:49.351 DEBUG:boto:Token: None 2022-09-03T05:20:49.352 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/ 2022-09-03T05:20:49.353 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uWIRBtV5fk0bHr6mNUKUSO6dRME= 2022-09-03T05:20:49.354 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uWIRBtV5fk0bHr6mNUKUSO6dRME=', 'Content-Length': '0'} 2022-09-03T05:20:49.357 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c6d781fcc1d324ef-006312e431-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.358 DEBUG:boto:b'dsgvcy-10myobj1000falsemyobj2022-09-03T05:20:36.801Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:49.360 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-10 zones={test-zone2, test-zone1} 2022-09-03T05:20:49.361 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-10', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:20:49.361 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-10', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:20:49.362 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-10 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:20:49.420 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:49.421 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:49.453 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:20:49.455+0000 7fd6ab141600 1 bucket sync caught up with source: 2022-09-03T05:20:49.454 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:20:49.455 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:20:49.456 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:20:49.455+0000 7fd6ab141600 0 bucket checkpoint complete 2022-09-03T05:20:49.466 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-10 zones={test-zone3, test-zone1} 2022-09-03T05:20:49.467 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:49.468 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:49.468 DEBUG:boto:Method: HEAD 2022-09-03T05:20:49.469 DEBUG:boto:Path: /dsgvcy-10/ 2022-09-03T05:20:49.469 DEBUG:boto:Data: 2022-09-03T05:20:49.470 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.470 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:49.471 DEBUG:boto:Port: 8001 2022-09-03T05:20:49.471 DEBUG:boto:Params: {} 2022-09-03T05:20:49.472 DEBUG:boto:Token: None 2022-09-03T05:20:49.472 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/ 2022-09-03T05:20:49.473 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fcQHZSYCmfGWp1q4upJ2cXCL/7M= 2022-09-03T05:20:49.474 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fcQHZSYCmfGWp1q4upJ2cXCL/7M=', 'Content-Length': '0'} 2022-09-03T05:20:49.476 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c10943d30a65ea7c-006312e431-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.477 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:49.477 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:49.478 DEBUG:boto:Method: HEAD 2022-09-03T05:20:49.478 DEBUG:boto:Path: /dsgvcy-10/ 2022-09-03T05:20:49.479 DEBUG:boto:Data: 2022-09-03T05:20:49.480 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.480 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:49.481 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.481 DEBUG:boto:Params: {} 2022-09-03T05:20:49.482 DEBUG:boto:Token: None 2022-09-03T05:20:49.483 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/ 2022-09-03T05:20:49.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fcQHZSYCmfGWp1q4upJ2cXCL/7M= 2022-09-03T05:20:49.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fcQHZSYCmfGWp1q4upJ2cXCL/7M=', 'Content-Length': '0'} 2022-09-03T05:20:49.486 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000eb78eda1f2925a7f-006312e431-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.487 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:20:49.487 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:49.488 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:49.488 DEBUG:boto:path=/dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:49.489 DEBUG:boto:auth_path=/dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:49.489 DEBUG:boto:Method: GET 2022-09-03T05:20:49.490 DEBUG:boto:Path: /dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:49.490 DEBUG:boto:Data: 2022-09-03T05:20:49.491 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.491 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:49.492 DEBUG:boto:Port: 8001 2022-09-03T05:20:49.493 DEBUG:boto:Params: {} 2022-09-03T05:20:49.493 DEBUG:boto:Token: None 2022-09-03T05:20:49.494 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/?versions 2022-09-03T05:20:49.494 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jipBilFcH7NJHy0enAS9ny5fAlA= 2022-09-03T05:20:49.495 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jipBilFcH7NJHy0enAS9ny5fAlA=', 'Content-Length': '0'} 2022-09-03T05:20:49.498 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e9e7c575b70df686-006312e431-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.498 DEBUG:boto:b'dsgvcy-10999false&nulltrue2022-09-03T05:20:36.975Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal:nulltrue2022-09-03T05:20:36.905Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal_myobjnulltrue2022-09-03T05:20:36.861Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormalmyobjnulltrue2022-09-03T05:20:36.801Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:49.499 DEBUG:tasks.rgw_multi.zone_rados:o=& 2022-09-03T05:20:49.500 DEBUG:tasks.rgw_multi.zone_rados:o=: 2022-09-03T05:20:49.500 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2022-09-03T05:20:49.501 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2022-09-03T05:20:49.501 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:20:49.502 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:49.502 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:49.503 DEBUG:boto:path=/dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:49.503 DEBUG:boto:auth_path=/dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:49.504 DEBUG:boto:Method: GET 2022-09-03T05:20:49.504 DEBUG:boto:Path: /dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:49.505 DEBUG:boto:Data: 2022-09-03T05:20:49.505 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.506 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:49.506 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.507 DEBUG:boto:Params: {} 2022-09-03T05:20:49.508 DEBUG:boto:Token: None 2022-09-03T05:20:49.508 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/?versions 2022-09-03T05:20:49.509 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jipBilFcH7NJHy0enAS9ny5fAlA= 2022-09-03T05:20:49.509 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jipBilFcH7NJHy0enAS9ny5fAlA=', 'Content-Length': '0'} 2022-09-03T05:20:49.512 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c81b64c6dc80129f-006312e431-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.512 DEBUG:boto:b'dsgvcy-10999false&nulltrue2022-09-03T05:20:36.975Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal:nulltrue2022-09-03T05:20:36.905Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal_myobjnulltrue2022-09-03T05:20:36.861Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormalmyobjnulltrue2022-09-03T05:20:36.801Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:49.513 DEBUG:tasks.rgw_multi.zone_rados:o=& 2022-09-03T05:20:49.514 DEBUG:tasks.rgw_multi.zone_rados:o=: 2022-09-03T05:20:49.514 DEBUG:tasks.rgw_multi.zone_rados:o=_myobj 2022-09-03T05:20:49.515 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2022-09-03T05:20:49.515 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:49.516 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:49.516 DEBUG:boto:path=/dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:49.517 DEBUG:boto:auth_path=/dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:49.517 DEBUG:boto:Method: GET 2022-09-03T05:20:49.518 DEBUG:boto:Path: /dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:49.518 DEBUG:boto:Data: 2022-09-03T05:20:49.519 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.519 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:49.520 DEBUG:boto:Port: 8001 2022-09-03T05:20:49.520 DEBUG:boto:Params: {} 2022-09-03T05:20:49.521 DEBUG:boto:Token: None 2022-09-03T05:20:49.522 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/?versions 2022-09-03T05:20:49.522 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jipBilFcH7NJHy0enAS9ny5fAlA= 2022-09-03T05:20:49.523 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jipBilFcH7NJHy0enAS9ny5fAlA=', 'Content-Length': '0'} 2022-09-03T05:20:49.526 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ec7f6ce6919332d9-006312e431-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.527 DEBUG:boto:b'dsgvcy-10999false&nulltrue2022-09-03T05:20:36.975Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal:nulltrue2022-09-03T05:20:36.905Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal_myobjnulltrue2022-09-03T05:20:36.861Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormalmyobjnulltrue2022-09-03T05:20:36.801Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:49.529 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:49.530 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:49.531 DEBUG:boto:path=/dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:49.532 DEBUG:boto:auth_path=/dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:49.533 DEBUG:boto:Method: GET 2022-09-03T05:20:49.534 DEBUG:boto:Path: /dsgvcy-10/?versions&max-keys=999 2022-09-03T05:20:49.535 DEBUG:boto:Data: 2022-09-03T05:20:49.536 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.537 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:49.538 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.539 DEBUG:boto:Params: {} 2022-09-03T05:20:49.540 DEBUG:boto:Token: None 2022-09-03T05:20:49.541 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/?versions 2022-09-03T05:20:49.543 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jipBilFcH7NJHy0enAS9ny5fAlA= 2022-09-03T05:20:49.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jipBilFcH7NJHy0enAS9ny5fAlA=', 'Content-Length': '0'} 2022-09-03T05:20:49.549 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dbec68b28bed0d2b-006312e431-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.550 DEBUG:boto:b'dsgvcy-10999false&nulltrue2022-09-03T05:20:36.975Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal:nulltrue2022-09-03T05:20:36.905Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal_myobjnulltrue2022-09-03T05:20:36.861Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormalmyobjnulltrue2022-09-03T05:20:36.801Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:49.551 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2022-09-03T05:20:49.552 DEBUG:boto:path=/dsgvcy-10/%26 2022-09-03T05:20:49.553 DEBUG:boto:auth_path=/dsgvcy-10/%26 2022-09-03T05:20:49.554 DEBUG:boto:path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.554 DEBUG:boto:auth_path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.555 DEBUG:boto:Method: GET 2022-09-03T05:20:49.555 DEBUG:boto:Path: /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.556 DEBUG:boto:Data: 2022-09-03T05:20:49.557 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.558 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:49.558 DEBUG:boto:Port: 8001 2022-09-03T05:20:49.559 DEBUG:boto:Params: {} 2022-09-03T05:20:49.559 DEBUG:boto:Token: None 2022-09-03T05:20:49.560 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.561 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jsUvmc+yBkN8ktQ1//EhjYmrBGc= 2022-09-03T05:20:49.561 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jsUvmc+yBkN8ktQ1//EhjYmrBGc=', 'Content-Length': '0'} 2022-09-03T05:20:49.563 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000798dbd2daac8fe39-006312e431-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.565 DEBUG:boto:path=/dsgvcy-10/%26 2022-09-03T05:20:49.566 DEBUG:boto:auth_path=/dsgvcy-10/%26 2022-09-03T05:20:49.568 DEBUG:boto:path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.568 DEBUG:boto:auth_path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.569 DEBUG:boto:Method: GET 2022-09-03T05:20:49.570 DEBUG:boto:Path: /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.571 DEBUG:boto:Data: 2022-09-03T05:20:49.572 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.573 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:49.574 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.575 DEBUG:boto:Params: {} 2022-09-03T05:20:49.576 DEBUG:boto:Token: None 2022-09-03T05:20:49.577 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.579 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jsUvmc+yBkN8ktQ1//EhjYmrBGc= 2022-09-03T05:20:49.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jsUvmc+yBkN8ktQ1//EhjYmrBGc=', 'Content-Length': '0'} 2022-09-03T05:20:49.582 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000079c0ff4a746eb02a-006312e431-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.626 DEBUG:boto:path=/dsgvcy-10/%26 2022-09-03T05:20:49.627 DEBUG:boto:auth_path=/dsgvcy-10/%26 2022-09-03T05:20:49.627 DEBUG:boto:path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.628 DEBUG:boto:auth_path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.628 DEBUG:boto:Method: HEAD 2022-09-03T05:20:49.629 DEBUG:boto:Path: /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.629 DEBUG:boto:Data: 2022-09-03T05:20:49.630 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.631 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:49.631 DEBUG:boto:Port: 8001 2022-09-03T05:20:49.632 DEBUG:boto:Params: {} 2022-09-03T05:20:49.632 DEBUG:boto:Token: None 2022-09-03T05:20:49.633 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.633 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mCJuyMwOBo8YrjUvNJqGBpxUKGQ= 2022-09-03T05:20:49.634 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mCJuyMwOBo8YrjUvNJqGBpxUKGQ=', 'Content-Length': '0'} 2022-09-03T05:20:49.636 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000028e98350d1b2d407-006312e431-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.637 DEBUG:boto:path=/dsgvcy-10/%26 2022-09-03T05:20:49.637 DEBUG:boto:auth_path=/dsgvcy-10/%26 2022-09-03T05:20:49.638 DEBUG:boto:path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.638 DEBUG:boto:auth_path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.639 DEBUG:boto:Method: HEAD 2022-09-03T05:20:49.639 DEBUG:boto:Path: /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.640 DEBUG:boto:Data: 2022-09-03T05:20:49.640 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.641 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:49.641 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.642 DEBUG:boto:Params: {} 2022-09-03T05:20:49.643 DEBUG:boto:Token: None 2022-09-03T05:20:49.643 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.644 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mCJuyMwOBo8YrjUvNJqGBpxUKGQ= 2022-09-03T05:20:49.644 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mCJuyMwOBo8YrjUvNJqGBpxUKGQ=', 'Content-Length': '0'} 2022-09-03T05:20:49.655 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000120d3279f8a5596e-006312e431-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.656 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2022-09-03T05:20:49.656 DEBUG:boto:path=/dsgvcy-10/%26 2022-09-03T05:20:49.657 DEBUG:boto:auth_path=/dsgvcy-10/%26 2022-09-03T05:20:49.658 DEBUG:boto:path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.658 DEBUG:boto:auth_path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.659 DEBUG:boto:Method: GET 2022-09-03T05:20:49.660 DEBUG:boto:Path: /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.661 DEBUG:boto:Data: 2022-09-03T05:20:49.662 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.664 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:49.665 DEBUG:boto:Port: 8001 2022-09-03T05:20:49.666 DEBUG:boto:Params: {} 2022-09-03T05:20:49.674 DEBUG:boto:Token: None 2022-09-03T05:20:49.675 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.676 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jsUvmc+yBkN8ktQ1//EhjYmrBGc= 2022-09-03T05:20:49.677 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jsUvmc+yBkN8ktQ1//EhjYmrBGc=', 'Content-Length': '0'} 2022-09-03T05:20:49.679 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000730727316921670a-006312e431-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.680 DEBUG:boto:path=/dsgvcy-10/%26 2022-09-03T05:20:49.681 DEBUG:boto:auth_path=/dsgvcy-10/%26 2022-09-03T05:20:49.681 DEBUG:boto:path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.682 DEBUG:boto:auth_path=/dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.683 DEBUG:boto:Method: GET 2022-09-03T05:20:49.683 DEBUG:boto:Path: /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.684 DEBUG:boto:Data: 2022-09-03T05:20:49.685 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.686 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:49.686 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.687 DEBUG:boto:Params: {} 2022-09-03T05:20:49.688 DEBUG:boto:Token: None 2022-09-03T05:20:49.688 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/%26?versionId=null 2022-09-03T05:20:49.689 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jsUvmc+yBkN8ktQ1//EhjYmrBGc= 2022-09-03T05:20:49.690 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jsUvmc+yBkN8ktQ1//EhjYmrBGc=', 'Content-Length': '0'} 2022-09-03T05:20:49.693 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000027a7f80c6a7cc516-006312e431-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.695 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:49.695 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:49.696 DEBUG:boto:path=/dsgvcy-10/?prefix=%26 2022-09-03T05:20:49.697 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=%26 2022-09-03T05:20:49.697 DEBUG:boto:Method: GET 2022-09-03T05:20:49.699 DEBUG:boto:Path: /dsgvcy-10/?prefix=%26 2022-09-03T05:20:49.700 DEBUG:boto:Data: 2022-09-03T05:20:49.701 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.702 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:49.703 DEBUG:boto:Port: 8001 2022-09-03T05:20:49.705 DEBUG:boto:Params: {} 2022-09-03T05:20:49.706 DEBUG:boto:Token: None 2022-09-03T05:20:49.708 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/ 2022-09-03T05:20:49.709 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uWIRBtV5fk0bHr6mNUKUSO6dRME= 2022-09-03T05:20:49.710 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uWIRBtV5fk0bHr6mNUKUSO6dRME=', 'Content-Length': '0'} 2022-09-03T05:20:49.713 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fd59062a49e75dca-006312e431-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.715 DEBUG:boto:b'dsgvcy-10&1000false&2022-09-03T05:20:36.975Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:49.716 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:49.716 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:49.717 DEBUG:boto:path=/dsgvcy-10/?prefix=%26 2022-09-03T05:20:49.717 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=%26 2022-09-03T05:20:49.718 DEBUG:boto:Method: GET 2022-09-03T05:20:49.719 DEBUG:boto:Path: /dsgvcy-10/?prefix=%26 2022-09-03T05:20:49.719 DEBUG:boto:Data: 2022-09-03T05:20:49.720 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.720 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:49.721 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.722 DEBUG:boto:Params: {} 2022-09-03T05:20:49.723 DEBUG:boto:Token: None 2022-09-03T05:20:49.724 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/ 2022-09-03T05:20:49.725 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uWIRBtV5fk0bHr6mNUKUSO6dRME= 2022-09-03T05:20:49.726 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uWIRBtV5fk0bHr6mNUKUSO6dRME=', 'Content-Length': '0'} 2022-09-03T05:20:49.729 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dd34af71e7058057-006312e431-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.729 DEBUG:boto:b'dsgvcy-10&1000false&2022-09-03T05:20:36.975Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:49.730 DEBUG:boto:path=/dsgvcy-10/%26 2022-09-03T05:20:49.731 DEBUG:boto:auth_path=/dsgvcy-10/%26 2022-09-03T05:20:49.731 DEBUG:boto:Method: HEAD 2022-09-03T05:20:49.732 DEBUG:boto:Path: /dsgvcy-10/%26 2022-09-03T05:20:49.732 DEBUG:boto:Data: 2022-09-03T05:20:49.733 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.733 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:49.734 DEBUG:boto:Port: 8001 2022-09-03T05:20:49.735 DEBUG:boto:Params: {} 2022-09-03T05:20:49.735 DEBUG:boto:Token: None 2022-09-03T05:20:49.736 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/%26 2022-09-03T05:20:49.736 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:28OSwv6tqt6GJB9vqqjokB3gdwE= 2022-09-03T05:20:49.737 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:28OSwv6tqt6GJB9vqqjokB3gdwE=', 'Content-Length': '0'} 2022-09-03T05:20:49.746 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000056a67406b97e7e80-006312e431-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.747 DEBUG:boto:path=/dsgvcy-10/%26 2022-09-03T05:20:49.748 DEBUG:boto:auth_path=/dsgvcy-10/%26 2022-09-03T05:20:49.749 DEBUG:boto:Method: HEAD 2022-09-03T05:20:49.749 DEBUG:boto:Path: /dsgvcy-10/%26 2022-09-03T05:20:49.751 DEBUG:boto:Data: 2022-09-03T05:20:49.752 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.753 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:49.753 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.755 DEBUG:boto:Params: {} 2022-09-03T05:20:49.756 DEBUG:boto:Token: None 2022-09-03T05:20:49.757 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/%26 2022-09-03T05:20:49.758 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:28OSwv6tqt6GJB9vqqjokB3gdwE= 2022-09-03T05:20:49.759 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:28OSwv6tqt6GJB9vqqjokB3gdwE=', 'Content-Length': '0'} 2022-09-03T05:20:49.763 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005c7869d1d8f67226-006312e431-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.764 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=& 2022-09-03T05:20:49.765 DEBUG:boto:path=/dsgvcy-10/%26 2022-09-03T05:20:49.765 DEBUG:boto:auth_path=/dsgvcy-10/%26 2022-09-03T05:20:49.766 DEBUG:boto:Method: GET 2022-09-03T05:20:49.767 DEBUG:boto:Path: /dsgvcy-10/%26 2022-09-03T05:20:49.767 DEBUG:boto:Data: 2022-09-03T05:20:49.768 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.768 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:49.769 DEBUG:boto:Port: 8001 2022-09-03T05:20:49.769 DEBUG:boto:Params: {} 2022-09-03T05:20:49.771 DEBUG:boto:Token: None 2022-09-03T05:20:49.772 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/%26 2022-09-03T05:20:49.772 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MYfvwn1dT0SdNbH4JB7JA1r+MtU= 2022-09-03T05:20:49.773 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MYfvwn1dT0SdNbH4JB7JA1r+MtU=', 'Content-Length': '0'} 2022-09-03T05:20:49.777 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000000e1599bd1fea77ba-006312e431-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.822 DEBUG:boto:path=/dsgvcy-10/%26 2022-09-03T05:20:49.823 DEBUG:boto:auth_path=/dsgvcy-10/%26 2022-09-03T05:20:49.824 DEBUG:boto:Method: GET 2022-09-03T05:20:49.824 DEBUG:boto:Path: /dsgvcy-10/%26 2022-09-03T05:20:49.825 DEBUG:boto:Data: 2022-09-03T05:20:49.825 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.826 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:49.827 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.828 DEBUG:boto:Params: {} 2022-09-03T05:20:49.829 DEBUG:boto:Token: None 2022-09-03T05:20:49.830 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/%26 2022-09-03T05:20:49.831 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MYfvwn1dT0SdNbH4JB7JA1r+MtU= 2022-09-03T05:20:49.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MYfvwn1dT0SdNbH4JB7JA1r+MtU=', 'Content-Length': '0'} 2022-09-03T05:20:49.834 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000bbac09c4c8bd6fa3-006312e431-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.835 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:49.836 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:49.837 DEBUG:boto:path=/dsgvcy-10/?prefix=%26 2022-09-03T05:20:49.837 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=%26 2022-09-03T05:20:49.838 DEBUG:boto:Method: GET 2022-09-03T05:20:49.839 DEBUG:boto:Path: /dsgvcy-10/?prefix=%26 2022-09-03T05:20:49.839 DEBUG:boto:Data: 2022-09-03T05:20:49.840 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.840 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:49.841 DEBUG:boto:Port: 8001 2022-09-03T05:20:49.841 DEBUG:boto:Params: {} 2022-09-03T05:20:49.842 DEBUG:boto:Token: None 2022-09-03T05:20:49.843 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/ 2022-09-03T05:20:49.843 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uWIRBtV5fk0bHr6mNUKUSO6dRME= 2022-09-03T05:20:49.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uWIRBtV5fk0bHr6mNUKUSO6dRME=', 'Content-Length': '0'} 2022-09-03T05:20:49.846 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009eac4e0209bc4286-006312e431-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.847 DEBUG:boto:b'dsgvcy-10&1000false&2022-09-03T05:20:36.975Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:49.848 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:49.849 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:49.849 DEBUG:boto:path=/dsgvcy-10/?prefix=%26 2022-09-03T05:20:49.850 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=%26 2022-09-03T05:20:49.850 DEBUG:boto:Method: GET 2022-09-03T05:20:49.851 DEBUG:boto:Path: /dsgvcy-10/?prefix=%26 2022-09-03T05:20:49.852 DEBUG:boto:Data: 2022-09-03T05:20:49.852 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.853 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:49.853 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.856 DEBUG:boto:Params: {} 2022-09-03T05:20:49.857 DEBUG:boto:Token: None 2022-09-03T05:20:49.860 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/ 2022-09-03T05:20:49.861 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uWIRBtV5fk0bHr6mNUKUSO6dRME= 2022-09-03T05:20:49.862 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uWIRBtV5fk0bHr6mNUKUSO6dRME=', 'Content-Length': '0'} 2022-09-03T05:20:49.866 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fd2d728a70c3d15d-006312e431-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.868 DEBUG:boto:b'dsgvcy-10&1000false&2022-09-03T05:20:36.975Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:49.869 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2022-09-03T05:20:49.870 DEBUG:boto:path=/dsgvcy-10/%3A 2022-09-03T05:20:49.872 DEBUG:boto:auth_path=/dsgvcy-10/%3A 2022-09-03T05:20:49.872 DEBUG:boto:path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.873 DEBUG:boto:auth_path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.873 DEBUG:boto:Method: GET 2022-09-03T05:20:49.874 DEBUG:boto:Path: /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.874 DEBUG:boto:Data: 2022-09-03T05:20:49.875 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.875 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:49.876 DEBUG:boto:Port: 8001 2022-09-03T05:20:49.876 DEBUG:boto:Params: {} 2022-09-03T05:20:49.878 DEBUG:boto:Token: None 2022-09-03T05:20:49.878 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.879 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Qsvvspj+Rnz6ZCnSlZUrBWYqREc= 2022-09-03T05:20:49.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Qsvvspj+Rnz6ZCnSlZUrBWYqREc=', 'Content-Length': '0'} 2022-09-03T05:20:49.882 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000051f1ef387d5a6a7b-006312e431-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.883 DEBUG:boto:path=/dsgvcy-10/%3A 2022-09-03T05:20:49.883 DEBUG:boto:auth_path=/dsgvcy-10/%3A 2022-09-03T05:20:49.884 DEBUG:boto:path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.884 DEBUG:boto:auth_path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.885 DEBUG:boto:Method: GET 2022-09-03T05:20:49.886 DEBUG:boto:Path: /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.886 DEBUG:boto:Data: 2022-09-03T05:20:49.887 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.887 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:49.888 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.891 DEBUG:boto:Params: {} 2022-09-03T05:20:49.892 DEBUG:boto:Token: None 2022-09-03T05:20:49.893 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.895 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Qsvvspj+Rnz6ZCnSlZUrBWYqREc= 2022-09-03T05:20:49.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Qsvvspj+Rnz6ZCnSlZUrBWYqREc=', 'Content-Length': '0'} 2022-09-03T05:20:49.898 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000bb294412b7c633a9-006312e431-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.942 DEBUG:boto:path=/dsgvcy-10/%3A 2022-09-03T05:20:49.943 DEBUG:boto:auth_path=/dsgvcy-10/%3A 2022-09-03T05:20:49.946 DEBUG:boto:path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.946 DEBUG:boto:auth_path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.947 DEBUG:boto:Method: HEAD 2022-09-03T05:20:49.947 DEBUG:boto:Path: /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.948 DEBUG:boto:Data: 2022-09-03T05:20:49.948 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.949 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:49.949 DEBUG:boto:Port: 8001 2022-09-03T05:20:49.950 DEBUG:boto:Params: {} 2022-09-03T05:20:49.951 DEBUG:boto:Token: None 2022-09-03T05:20:49.951 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.952 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6eFSDqDoJ+wHMJvCBh3YhwD50IQ= 2022-09-03T05:20:49.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6eFSDqDoJ+wHMJvCBh3YhwD50IQ=', 'Content-Length': '0'} 2022-09-03T05:20:49.955 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000cd071face94811cd-006312e431-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.957 DEBUG:boto:path=/dsgvcy-10/%3A 2022-09-03T05:20:49.958 DEBUG:boto:auth_path=/dsgvcy-10/%3A 2022-09-03T05:20:49.958 DEBUG:boto:path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.959 DEBUG:boto:auth_path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.960 DEBUG:boto:Method: HEAD 2022-09-03T05:20:49.961 DEBUG:boto:Path: /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.962 DEBUG:boto:Data: 2022-09-03T05:20:49.963 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.964 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:49.965 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.966 DEBUG:boto:Params: {} 2022-09-03T05:20:49.966 DEBUG:boto:Token: None 2022-09-03T05:20:49.967 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.968 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6eFSDqDoJ+wHMJvCBh3YhwD50IQ= 2022-09-03T05:20:49.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6eFSDqDoJ+wHMJvCBh3YhwD50IQ=', 'Content-Length': '0'} 2022-09-03T05:20:49.970 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000068e4092c829e2032-006312e431-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.971 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2022-09-03T05:20:49.972 DEBUG:boto:path=/dsgvcy-10/%3A 2022-09-03T05:20:49.972 DEBUG:boto:auth_path=/dsgvcy-10/%3A 2022-09-03T05:20:49.973 DEBUG:boto:path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.974 DEBUG:boto:auth_path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.975 DEBUG:boto:Method: GET 2022-09-03T05:20:49.975 DEBUG:boto:Path: /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.976 DEBUG:boto:Data: 2022-09-03T05:20:49.976 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.977 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:49.977 DEBUG:boto:Port: 8001 2022-09-03T05:20:49.978 DEBUG:boto:Params: {} 2022-09-03T05:20:49.979 DEBUG:boto:Token: None 2022-09-03T05:20:49.980 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.981 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Qsvvspj+Rnz6ZCnSlZUrBWYqREc= 2022-09-03T05:20:49.982 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Qsvvspj+Rnz6ZCnSlZUrBWYqREc=', 'Content-Length': '0'} 2022-09-03T05:20:49.985 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000003e45614380d46b2e-006312e431-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.986 DEBUG:boto:path=/dsgvcy-10/%3A 2022-09-03T05:20:49.987 DEBUG:boto:auth_path=/dsgvcy-10/%3A 2022-09-03T05:20:49.987 DEBUG:boto:path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.988 DEBUG:boto:auth_path=/dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.988 DEBUG:boto:Method: GET 2022-09-03T05:20:49.989 DEBUG:boto:Path: /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.990 DEBUG:boto:Data: 2022-09-03T05:20:49.990 DEBUG:boto:Headers: {} 2022-09-03T05:20:49.991 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:49.991 DEBUG:boto:Port: 8000 2022-09-03T05:20:49.992 DEBUG:boto:Params: {} 2022-09-03T05:20:49.993 DEBUG:boto:Token: None 2022-09-03T05:20:49.993 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:49 GMT /dsgvcy-10/%3A?versionId=null 2022-09-03T05:20:49.994 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Qsvvspj+Rnz6ZCnSlZUrBWYqREc= 2022-09-03T05:20:49.995 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Qsvvspj+Rnz6ZCnSlZUrBWYqREc=', 'Content-Length': '0'} 2022-09-03T05:20:49.997 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b239682f64eb303b-006312e432-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:49.998 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:49.999 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:50.000 DEBUG:boto:path=/dsgvcy-10/?prefix=%3A 2022-09-03T05:20:50.001 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=%3A 2022-09-03T05:20:50.001 DEBUG:boto:Method: GET 2022-09-03T05:20:50.002 DEBUG:boto:Path: /dsgvcy-10/?prefix=%3A 2022-09-03T05:20:50.002 DEBUG:boto:Data: 2022-09-03T05:20:50.003 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.003 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:50.004 DEBUG:boto:Port: 8001 2022-09-03T05:20:50.004 DEBUG:boto:Params: {} 2022-09-03T05:20:50.005 DEBUG:boto:Token: None 2022-09-03T05:20:50.006 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/ 2022-09-03T05:20:50.007 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0= 2022-09-03T05:20:50.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0=', 'Content-Length': '0'} 2022-09-03T05:20:50.011 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004c3d6d8c78d1e4e9-006312e432-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.012 DEBUG:boto:b'dsgvcy-10:1000false:2022-09-03T05:20:36.905Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:50.013 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:50.014 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:50.015 DEBUG:boto:path=/dsgvcy-10/?prefix=%3A 2022-09-03T05:20:50.015 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=%3A 2022-09-03T05:20:50.016 DEBUG:boto:Method: GET 2022-09-03T05:20:50.016 DEBUG:boto:Path: /dsgvcy-10/?prefix=%3A 2022-09-03T05:20:50.017 DEBUG:boto:Data: 2022-09-03T05:20:50.018 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.019 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:50.020 DEBUG:boto:Port: 8000 2022-09-03T05:20:50.021 DEBUG:boto:Params: {} 2022-09-03T05:20:50.022 DEBUG:boto:Token: None 2022-09-03T05:20:50.023 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/ 2022-09-03T05:20:50.025 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0= 2022-09-03T05:20:50.026 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0=', 'Content-Length': '0'} 2022-09-03T05:20:50.029 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c8566d76a302a5bc-006312e432-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.030 DEBUG:boto:b'dsgvcy-10:1000false:2022-09-03T05:20:36.905Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:50.031 DEBUG:boto:path=/dsgvcy-10/%3A 2022-09-03T05:20:50.032 DEBUG:boto:auth_path=/dsgvcy-10/%3A 2022-09-03T05:20:50.033 DEBUG:boto:Method: HEAD 2022-09-03T05:20:50.034 DEBUG:boto:Path: /dsgvcy-10/%3A 2022-09-03T05:20:50.035 DEBUG:boto:Data: 2022-09-03T05:20:50.036 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.037 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:50.038 DEBUG:boto:Port: 8001 2022-09-03T05:20:50.039 DEBUG:boto:Params: {} 2022-09-03T05:20:50.040 DEBUG:boto:Token: None 2022-09-03T05:20:50.041 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/%3A 2022-09-03T05:20:50.044 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sc+EX8Y0WtJYM1PThLBsvdxv54w= 2022-09-03T05:20:50.045 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sc+EX8Y0WtJYM1PThLBsvdxv54w=', 'Content-Length': '0'} 2022-09-03T05:20:50.047 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000006ac96cd7f0edb437-006312e432-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.048 DEBUG:boto:path=/dsgvcy-10/%3A 2022-09-03T05:20:50.048 DEBUG:boto:auth_path=/dsgvcy-10/%3A 2022-09-03T05:20:50.049 DEBUG:boto:Method: HEAD 2022-09-03T05:20:50.049 DEBUG:boto:Path: /dsgvcy-10/%3A 2022-09-03T05:20:50.050 DEBUG:boto:Data: 2022-09-03T05:20:50.051 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.052 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:50.053 DEBUG:boto:Port: 8000 2022-09-03T05:20:50.054 DEBUG:boto:Params: {} 2022-09-03T05:20:50.055 DEBUG:boto:Token: None 2022-09-03T05:20:50.056 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/%3A 2022-09-03T05:20:50.057 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sc+EX8Y0WtJYM1PThLBsvdxv54w= 2022-09-03T05:20:50.058 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sc+EX8Y0WtJYM1PThLBsvdxv54w=', 'Content-Length': '0'} 2022-09-03T05:20:50.061 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a22f696924a7e6e6-006312e432-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.062 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=: 2022-09-03T05:20:50.063 DEBUG:boto:path=/dsgvcy-10/%3A 2022-09-03T05:20:50.064 DEBUG:boto:auth_path=/dsgvcy-10/%3A 2022-09-03T05:20:50.065 DEBUG:boto:Method: GET 2022-09-03T05:20:50.066 DEBUG:boto:Path: /dsgvcy-10/%3A 2022-09-03T05:20:50.067 DEBUG:boto:Data: 2022-09-03T05:20:50.068 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.070 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:50.074 DEBUG:boto:Port: 8001 2022-09-03T05:20:50.075 DEBUG:boto:Params: {} 2022-09-03T05:20:50.076 DEBUG:boto:Token: None 2022-09-03T05:20:50.078 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/%3A 2022-09-03T05:20:50.079 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rioOZLYriqQxtMNgkrsz/FKXrSg= 2022-09-03T05:20:50.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rioOZLYriqQxtMNgkrsz/FKXrSg=', 'Content-Length': '0'} 2022-09-03T05:20:50.092 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e2c71d71f1a3520e-006312e432-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.094 DEBUG:boto:path=/dsgvcy-10/%3A 2022-09-03T05:20:50.094 DEBUG:boto:auth_path=/dsgvcy-10/%3A 2022-09-03T05:20:50.095 DEBUG:boto:Method: GET 2022-09-03T05:20:50.096 DEBUG:boto:Path: /dsgvcy-10/%3A 2022-09-03T05:20:50.097 DEBUG:boto:Data: 2022-09-03T05:20:50.098 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.099 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:50.100 DEBUG:boto:Port: 8000 2022-09-03T05:20:50.101 DEBUG:boto:Params: {} 2022-09-03T05:20:50.103 DEBUG:boto:Token: None 2022-09-03T05:20:50.104 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/%3A 2022-09-03T05:20:50.105 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rioOZLYriqQxtMNgkrsz/FKXrSg= 2022-09-03T05:20:50.107 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rioOZLYriqQxtMNgkrsz/FKXrSg=', 'Content-Length': '0'} 2022-09-03T05:20:50.109 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ca97dab044d0a72b-006312e432-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.111 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:50.112 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:50.113 DEBUG:boto:path=/dsgvcy-10/?prefix=%3A 2022-09-03T05:20:50.114 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=%3A 2022-09-03T05:20:50.115 DEBUG:boto:Method: GET 2022-09-03T05:20:50.117 DEBUG:boto:Path: /dsgvcy-10/?prefix=%3A 2022-09-03T05:20:50.118 DEBUG:boto:Data: 2022-09-03T05:20:50.119 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.120 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:50.121 DEBUG:boto:Port: 8001 2022-09-03T05:20:50.121 DEBUG:boto:Params: {} 2022-09-03T05:20:50.122 DEBUG:boto:Token: None 2022-09-03T05:20:50.123 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/ 2022-09-03T05:20:50.123 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0= 2022-09-03T05:20:50.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0=', 'Content-Length': '0'} 2022-09-03T05:20:50.127 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c1b2c24c771c6523-006312e432-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.128 DEBUG:boto:b'dsgvcy-10:1000false:2022-09-03T05:20:36.905Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:50.130 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:50.131 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:50.133 DEBUG:boto:path=/dsgvcy-10/?prefix=%3A 2022-09-03T05:20:50.134 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=%3A 2022-09-03T05:20:50.134 DEBUG:boto:Method: GET 2022-09-03T05:20:50.136 DEBUG:boto:Path: /dsgvcy-10/?prefix=%3A 2022-09-03T05:20:50.136 DEBUG:boto:Data: 2022-09-03T05:20:50.138 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.139 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:50.140 DEBUG:boto:Port: 8000 2022-09-03T05:20:50.141 DEBUG:boto:Params: {} 2022-09-03T05:20:50.142 DEBUG:boto:Token: None 2022-09-03T05:20:50.143 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/ 2022-09-03T05:20:50.145 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0= 2022-09-03T05:20:50.146 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0=', 'Content-Length': '0'} 2022-09-03T05:20:50.149 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000084259481eddd8505-006312e432-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.150 DEBUG:boto:b'dsgvcy-10:1000false:2022-09-03T05:20:36.905Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:50.152 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2022-09-03T05:20:50.153 DEBUG:boto:path=/dsgvcy-10/_myobj 2022-09-03T05:20:50.154 DEBUG:boto:auth_path=/dsgvcy-10/_myobj 2022-09-03T05:20:50.155 DEBUG:boto:path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.156 DEBUG:boto:auth_path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.157 DEBUG:boto:Method: GET 2022-09-03T05:20:50.158 DEBUG:boto:Path: /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.158 DEBUG:boto:Data: 2022-09-03T05:20:50.159 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.160 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:50.160 DEBUG:boto:Port: 8001 2022-09-03T05:20:50.161 DEBUG:boto:Params: {} 2022-09-03T05:20:50.161 DEBUG:boto:Token: None 2022-09-03T05:20:50.162 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.163 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dwihq+9SFD8NkT1euXnj/FoGaDs= 2022-09-03T05:20:50.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dwihq+9SFD8NkT1euXnj/FoGaDs=', 'Content-Length': '0'} 2022-09-03T05:20:50.166 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000a360beea3baba269-006312e432-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.167 DEBUG:boto:path=/dsgvcy-10/_myobj 2022-09-03T05:20:50.169 DEBUG:boto:auth_path=/dsgvcy-10/_myobj 2022-09-03T05:20:50.169 DEBUG:boto:path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.171 DEBUG:boto:auth_path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.172 DEBUG:boto:Method: GET 2022-09-03T05:20:50.173 DEBUG:boto:Path: /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.174 DEBUG:boto:Data: 2022-09-03T05:20:50.175 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.175 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:50.176 DEBUG:boto:Port: 8000 2022-09-03T05:20:50.176 DEBUG:boto:Params: {} 2022-09-03T05:20:50.177 DEBUG:boto:Token: None 2022-09-03T05:20:50.178 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.178 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dwihq+9SFD8NkT1euXnj/FoGaDs= 2022-09-03T05:20:50.179 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dwihq+9SFD8NkT1euXnj/FoGaDs=', 'Content-Length': '0'} 2022-09-03T05:20:50.181 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e4162452c3007370-006312e432-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.182 DEBUG:boto:path=/dsgvcy-10/_myobj 2022-09-03T05:20:50.183 DEBUG:boto:auth_path=/dsgvcy-10/_myobj 2022-09-03T05:20:50.183 DEBUG:boto:path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.184 DEBUG:boto:auth_path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.184 DEBUG:boto:Method: HEAD 2022-09-03T05:20:50.185 DEBUG:boto:Path: /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.185 DEBUG:boto:Data: 2022-09-03T05:20:50.186 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.186 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:50.187 DEBUG:boto:Port: 8001 2022-09-03T05:20:50.187 DEBUG:boto:Params: {} 2022-09-03T05:20:50.188 DEBUG:boto:Token: None 2022-09-03T05:20:50.189 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.190 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wHtZiy07e4WNbjdEN5i7JYnW/ls= 2022-09-03T05:20:50.191 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wHtZiy07e4WNbjdEN5i7JYnW/ls=', 'Content-Length': '0'} 2022-09-03T05:20:50.194 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007bdef961e81303a1-006312e432-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.194 DEBUG:boto:path=/dsgvcy-10/_myobj 2022-09-03T05:20:50.195 DEBUG:boto:auth_path=/dsgvcy-10/_myobj 2022-09-03T05:20:50.195 DEBUG:boto:path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.196 DEBUG:boto:auth_path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.197 DEBUG:boto:Method: HEAD 2022-09-03T05:20:50.197 DEBUG:boto:Path: /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.198 DEBUG:boto:Data: 2022-09-03T05:20:50.198 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.199 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:50.199 DEBUG:boto:Port: 8000 2022-09-03T05:20:50.200 DEBUG:boto:Params: {} 2022-09-03T05:20:50.200 DEBUG:boto:Token: None 2022-09-03T05:20:50.201 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.201 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wHtZiy07e4WNbjdEN5i7JYnW/ls= 2022-09-03T05:20:50.202 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wHtZiy07e4WNbjdEN5i7JYnW/ls=', 'Content-Length': '0'} 2022-09-03T05:20:50.204 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000001f7eff70976d0b9f-006312e432-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.205 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2022-09-03T05:20:50.205 DEBUG:boto:path=/dsgvcy-10/_myobj 2022-09-03T05:20:50.206 DEBUG:boto:auth_path=/dsgvcy-10/_myobj 2022-09-03T05:20:50.206 DEBUG:boto:path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.207 DEBUG:boto:auth_path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.207 DEBUG:boto:Method: GET 2022-09-03T05:20:50.208 DEBUG:boto:Path: /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.208 DEBUG:boto:Data: 2022-09-03T05:20:50.209 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.209 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:50.210 DEBUG:boto:Port: 8001 2022-09-03T05:20:50.210 DEBUG:boto:Params: {} 2022-09-03T05:20:50.211 DEBUG:boto:Token: None 2022-09-03T05:20:50.212 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.212 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dwihq+9SFD8NkT1euXnj/FoGaDs= 2022-09-03T05:20:50.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dwihq+9SFD8NkT1euXnj/FoGaDs=', 'Content-Length': '0'} 2022-09-03T05:20:50.215 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000099feafc6d7940e06-006312e432-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.216 DEBUG:boto:path=/dsgvcy-10/_myobj 2022-09-03T05:20:50.216 DEBUG:boto:auth_path=/dsgvcy-10/_myobj 2022-09-03T05:20:50.217 DEBUG:boto:path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.217 DEBUG:boto:auth_path=/dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.218 DEBUG:boto:Method: GET 2022-09-03T05:20:50.218 DEBUG:boto:Path: /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.219 DEBUG:boto:Data: 2022-09-03T05:20:50.219 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.220 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:50.220 DEBUG:boto:Port: 8000 2022-09-03T05:20:50.221 DEBUG:boto:Params: {} 2022-09-03T05:20:50.222 DEBUG:boto:Token: None 2022-09-03T05:20:50.222 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/_myobj?versionId=null 2022-09-03T05:20:50.223 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dwihq+9SFD8NkT1euXnj/FoGaDs= 2022-09-03T05:20:50.223 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dwihq+9SFD8NkT1euXnj/FoGaDs=', 'Content-Length': '0'} 2022-09-03T05:20:50.225 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000022213256da5b76ce-006312e432-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.226 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:50.227 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:50.228 DEBUG:boto:path=/dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:50.228 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:50.229 DEBUG:boto:Method: GET 2022-09-03T05:20:50.229 DEBUG:boto:Path: /dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:50.230 DEBUG:boto:Data: 2022-09-03T05:20:50.230 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.231 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:50.231 DEBUG:boto:Port: 8001 2022-09-03T05:20:50.232 DEBUG:boto:Params: {} 2022-09-03T05:20:50.232 DEBUG:boto:Token: None 2022-09-03T05:20:50.233 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/ 2022-09-03T05:20:50.233 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0= 2022-09-03T05:20:50.234 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0=', 'Content-Length': '0'} 2022-09-03T05:20:50.236 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000089b13f3680978208-006312e432-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.278 DEBUG:boto:b'dsgvcy-10_myobj1000false_myobj2022-09-03T05:20:36.861Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:50.279 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:50.280 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:50.280 DEBUG:boto:path=/dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:50.281 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:50.281 DEBUG:boto:Method: GET 2022-09-03T05:20:50.282 DEBUG:boto:Path: /dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:50.283 DEBUG:boto:Data: 2022-09-03T05:20:50.283 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.284 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:50.284 DEBUG:boto:Port: 8000 2022-09-03T05:20:50.285 DEBUG:boto:Params: {} 2022-09-03T05:20:50.285 DEBUG:boto:Token: None 2022-09-03T05:20:50.286 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/ 2022-09-03T05:20:50.287 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0= 2022-09-03T05:20:50.287 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0=', 'Content-Length': '0'} 2022-09-03T05:20:50.289 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007ab411ec9d3b3b9d-006312e432-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.290 DEBUG:boto:b'dsgvcy-10_myobj1000false_myobj2022-09-03T05:20:36.861Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:50.291 DEBUG:boto:path=/dsgvcy-10/_myobj 2022-09-03T05:20:50.292 DEBUG:boto:auth_path=/dsgvcy-10/_myobj 2022-09-03T05:20:50.292 DEBUG:boto:Method: HEAD 2022-09-03T05:20:50.293 DEBUG:boto:Path: /dsgvcy-10/_myobj 2022-09-03T05:20:50.293 DEBUG:boto:Data: 2022-09-03T05:20:50.293 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.294 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:50.294 DEBUG:boto:Port: 8001 2022-09-03T05:20:50.295 DEBUG:boto:Params: {} 2022-09-03T05:20:50.296 DEBUG:boto:Token: None 2022-09-03T05:20:50.296 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/_myobj 2022-09-03T05:20:50.297 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yMV396spHkckMyui949a7+MNoXo= 2022-09-03T05:20:50.297 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yMV396spHkckMyui949a7+MNoXo=', 'Content-Length': '0'} 2022-09-03T05:20:50.299 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007655a1bc8685f79c-006312e432-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.300 DEBUG:boto:path=/dsgvcy-10/_myobj 2022-09-03T05:20:50.301 DEBUG:boto:auth_path=/dsgvcy-10/_myobj 2022-09-03T05:20:50.301 DEBUG:boto:Method: HEAD 2022-09-03T05:20:50.302 DEBUG:boto:Path: /dsgvcy-10/_myobj 2022-09-03T05:20:50.302 DEBUG:boto:Data: 2022-09-03T05:20:50.303 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.303 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:50.304 DEBUG:boto:Port: 8000 2022-09-03T05:20:50.304 DEBUG:boto:Params: {} 2022-09-03T05:20:50.305 DEBUG:boto:Token: None 2022-09-03T05:20:50.305 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/_myobj 2022-09-03T05:20:50.306 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yMV396spHkckMyui949a7+MNoXo= 2022-09-03T05:20:50.306 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yMV396spHkckMyui949a7+MNoXo=', 'Content-Length': '0'} 2022-09-03T05:20:50.308 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e3a64516636df600-006312e432-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.309 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=_myobj 2022-09-03T05:20:50.310 DEBUG:boto:path=/dsgvcy-10/_myobj 2022-09-03T05:20:50.310 DEBUG:boto:auth_path=/dsgvcy-10/_myobj 2022-09-03T05:20:50.311 DEBUG:boto:Method: GET 2022-09-03T05:20:50.311 DEBUG:boto:Path: /dsgvcy-10/_myobj 2022-09-03T05:20:50.312 DEBUG:boto:Data: 2022-09-03T05:20:50.312 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.313 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:50.314 DEBUG:boto:Port: 8001 2022-09-03T05:20:50.314 DEBUG:boto:Params: {} 2022-09-03T05:20:50.315 DEBUG:boto:Token: None 2022-09-03T05:20:50.315 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/_myobj 2022-09-03T05:20:50.316 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:07/4Tjw/6i61JuY9wOWVNzzVvK8= 2022-09-03T05:20:50.317 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:07/4Tjw/6i61JuY9wOWVNzzVvK8=', 'Content-Length': '0'} 2022-09-03T05:20:50.318 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000051b486f9ff424cab-006312e432-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.319 DEBUG:boto:path=/dsgvcy-10/_myobj 2022-09-03T05:20:50.320 DEBUG:boto:auth_path=/dsgvcy-10/_myobj 2022-09-03T05:20:50.320 DEBUG:boto:Method: GET 2022-09-03T05:20:50.321 DEBUG:boto:Path: /dsgvcy-10/_myobj 2022-09-03T05:20:50.321 DEBUG:boto:Data: 2022-09-03T05:20:50.322 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.322 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:50.323 DEBUG:boto:Port: 8000 2022-09-03T05:20:50.323 DEBUG:boto:Params: {} 2022-09-03T05:20:50.324 DEBUG:boto:Token: None 2022-09-03T05:20:50.325 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/_myobj 2022-09-03T05:20:50.325 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:07/4Tjw/6i61JuY9wOWVNzzVvK8= 2022-09-03T05:20:50.326 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:07/4Tjw/6i61JuY9wOWVNzzVvK8=', 'Content-Length': '0'} 2022-09-03T05:20:50.328 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007b13773589a29c73-006312e432-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.329 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:50.329 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:50.330 DEBUG:boto:path=/dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:50.330 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:50.331 DEBUG:boto:Method: GET 2022-09-03T05:20:50.332 DEBUG:boto:Path: /dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:50.332 DEBUG:boto:Data: 2022-09-03T05:20:50.333 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.333 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:50.334 DEBUG:boto:Port: 8001 2022-09-03T05:20:50.334 DEBUG:boto:Params: {} 2022-09-03T05:20:50.335 DEBUG:boto:Token: None 2022-09-03T05:20:50.336 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/ 2022-09-03T05:20:50.336 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0= 2022-09-03T05:20:50.337 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0=', 'Content-Length': '0'} 2022-09-03T05:20:50.339 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e42287491d39323e-006312e432-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.340 DEBUG:boto:b'dsgvcy-10_myobj1000false_myobj2022-09-03T05:20:36.861Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:50.341 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:50.342 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:50.342 DEBUG:boto:path=/dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:50.343 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:50.343 DEBUG:boto:Method: GET 2022-09-03T05:20:50.344 DEBUG:boto:Path: /dsgvcy-10/?prefix=_myobj 2022-09-03T05:20:50.344 DEBUG:boto:Data: 2022-09-03T05:20:50.345 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.345 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:50.346 DEBUG:boto:Port: 8000 2022-09-03T05:20:50.347 DEBUG:boto:Params: {} 2022-09-03T05:20:50.347 DEBUG:boto:Token: None 2022-09-03T05:20:50.348 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/ 2022-09-03T05:20:50.349 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0= 2022-09-03T05:20:50.349 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0=', 'Content-Length': '0'} 2022-09-03T05:20:50.351 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000df1359626258b347-006312e432-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.352 DEBUG:boto:b'dsgvcy-10_myobj1000false_myobj2022-09-03T05:20:36.861Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:50.353 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:20:50.354 DEBUG:boto:path=/dsgvcy-10/myobj 2022-09-03T05:20:50.354 DEBUG:boto:auth_path=/dsgvcy-10/myobj 2022-09-03T05:20:50.355 DEBUG:boto:path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.355 DEBUG:boto:auth_path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.356 DEBUG:boto:Method: GET 2022-09-03T05:20:50.356 DEBUG:boto:Path: /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.357 DEBUG:boto:Data: 2022-09-03T05:20:50.357 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.358 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:50.359 DEBUG:boto:Port: 8001 2022-09-03T05:20:50.359 DEBUG:boto:Params: {} 2022-09-03T05:20:50.360 DEBUG:boto:Token: None 2022-09-03T05:20:50.360 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.361 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:42Y/PWOBD7zs84bSIU8D78qtG64= 2022-09-03T05:20:50.362 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:42Y/PWOBD7zs84bSIU8D78qtG64=', 'Content-Length': '0'} 2022-09-03T05:20:50.364 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004c9171dd0f505604-006312e432-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.406 DEBUG:boto:path=/dsgvcy-10/myobj 2022-09-03T05:20:50.407 DEBUG:boto:auth_path=/dsgvcy-10/myobj 2022-09-03T05:20:50.407 DEBUG:boto:path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.408 DEBUG:boto:auth_path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.409 DEBUG:boto:Method: GET 2022-09-03T05:20:50.410 DEBUG:boto:Path: /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.411 DEBUG:boto:Data: 2022-09-03T05:20:50.412 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.413 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:50.414 DEBUG:boto:Port: 8000 2022-09-03T05:20:50.415 DEBUG:boto:Params: {} 2022-09-03T05:20:50.416 DEBUG:boto:Token: None 2022-09-03T05:20:50.418 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.419 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:42Y/PWOBD7zs84bSIU8D78qtG64= 2022-09-03T05:20:50.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:42Y/PWOBD7zs84bSIU8D78qtG64=', 'Content-Length': '0'} 2022-09-03T05:20:50.423 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000001c008e8ae11ab38d-006312e432-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.424 DEBUG:boto:path=/dsgvcy-10/myobj 2022-09-03T05:20:50.425 DEBUG:boto:auth_path=/dsgvcy-10/myobj 2022-09-03T05:20:50.426 DEBUG:boto:path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.427 DEBUG:boto:auth_path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.428 DEBUG:boto:Method: HEAD 2022-09-03T05:20:50.429 DEBUG:boto:Path: /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.430 DEBUG:boto:Data: 2022-09-03T05:20:50.431 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.432 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:50.433 DEBUG:boto:Port: 8001 2022-09-03T05:20:50.434 DEBUG:boto:Params: {} 2022-09-03T05:20:50.435 DEBUG:boto:Token: None 2022-09-03T05:20:50.436 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.437 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WyfIdLrOypdlSYpT2t9mSjuZbFQ= 2022-09-03T05:20:50.437 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WyfIdLrOypdlSYpT2t9mSjuZbFQ=', 'Content-Length': '0'} 2022-09-03T05:20:50.439 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000003432725f671ca6f1-006312e432-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.440 DEBUG:boto:path=/dsgvcy-10/myobj 2022-09-03T05:20:50.441 DEBUG:boto:auth_path=/dsgvcy-10/myobj 2022-09-03T05:20:50.441 DEBUG:boto:path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.442 DEBUG:boto:auth_path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.442 DEBUG:boto:Method: HEAD 2022-09-03T05:20:50.443 DEBUG:boto:Path: /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.444 DEBUG:boto:Data: 2022-09-03T05:20:50.444 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.445 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:50.445 DEBUG:boto:Port: 8000 2022-09-03T05:20:50.446 DEBUG:boto:Params: {} 2022-09-03T05:20:50.447 DEBUG:boto:Token: None 2022-09-03T05:20:50.447 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.448 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WyfIdLrOypdlSYpT2t9mSjuZbFQ= 2022-09-03T05:20:50.449 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WyfIdLrOypdlSYpT2t9mSjuZbFQ=', 'Content-Length': '0'} 2022-09-03T05:20:50.450 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004c02e9e1f68795e4-006312e432-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.451 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:20:50.452 DEBUG:boto:path=/dsgvcy-10/myobj 2022-09-03T05:20:50.452 DEBUG:boto:auth_path=/dsgvcy-10/myobj 2022-09-03T05:20:50.453 DEBUG:boto:path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.454 DEBUG:boto:auth_path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.455 DEBUG:boto:Method: GET 2022-09-03T05:20:50.455 DEBUG:boto:Path: /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.456 DEBUG:boto:Data: 2022-09-03T05:20:50.457 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.457 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:50.458 DEBUG:boto:Port: 8001 2022-09-03T05:20:50.458 DEBUG:boto:Params: {} 2022-09-03T05:20:50.459 DEBUG:boto:Token: None 2022-09-03T05:20:50.459 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.460 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:42Y/PWOBD7zs84bSIU8D78qtG64= 2022-09-03T05:20:50.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:42Y/PWOBD7zs84bSIU8D78qtG64=', 'Content-Length': '0'} 2022-09-03T05:20:50.463 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000008446e5b8840e240e-006312e432-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.479 DEBUG:boto:path=/dsgvcy-10/myobj 2022-09-03T05:20:50.480 DEBUG:boto:auth_path=/dsgvcy-10/myobj 2022-09-03T05:20:50.481 DEBUG:boto:path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.481 DEBUG:boto:auth_path=/dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.482 DEBUG:boto:Method: GET 2022-09-03T05:20:50.483 DEBUG:boto:Path: /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.483 DEBUG:boto:Data: 2022-09-03T05:20:50.484 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.484 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:50.485 DEBUG:boto:Port: 8000 2022-09-03T05:20:50.485 DEBUG:boto:Params: {} 2022-09-03T05:20:50.486 DEBUG:boto:Token: None 2022-09-03T05:20:50.487 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/myobj?versionId=null 2022-09-03T05:20:50.487 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:42Y/PWOBD7zs84bSIU8D78qtG64= 2022-09-03T05:20:50.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:42Y/PWOBD7zs84bSIU8D78qtG64=', 'Content-Length': '0'} 2022-09-03T05:20:50.490 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007527f862d1aa3608-006312e432-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.534 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:50.536 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:50.537 DEBUG:boto:path=/dsgvcy-10/?prefix=myobj 2022-09-03T05:20:50.537 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=myobj 2022-09-03T05:20:50.538 DEBUG:boto:Method: GET 2022-09-03T05:20:50.538 DEBUG:boto:Path: /dsgvcy-10/?prefix=myobj 2022-09-03T05:20:50.539 DEBUG:boto:Data: 2022-09-03T05:20:50.540 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.540 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:50.541 DEBUG:boto:Port: 8001 2022-09-03T05:20:50.542 DEBUG:boto:Params: {} 2022-09-03T05:20:50.542 DEBUG:boto:Token: None 2022-09-03T05:20:50.543 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/ 2022-09-03T05:20:50.544 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0= 2022-09-03T05:20:50.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0=', 'Content-Length': '0'} 2022-09-03T05:20:50.547 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fa36a178486f291f-006312e432-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.547 DEBUG:boto:b'dsgvcy-10myobj1000falsemyobj2022-09-03T05:20:36.801Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:50.548 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:50.549 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:50.549 DEBUG:boto:path=/dsgvcy-10/?prefix=myobj 2022-09-03T05:20:50.550 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=myobj 2022-09-03T05:20:50.551 DEBUG:boto:Method: GET 2022-09-03T05:20:50.551 DEBUG:boto:Path: /dsgvcy-10/?prefix=myobj 2022-09-03T05:20:50.552 DEBUG:boto:Data: 2022-09-03T05:20:50.552 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.553 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:50.553 DEBUG:boto:Port: 8000 2022-09-03T05:20:50.554 DEBUG:boto:Params: {} 2022-09-03T05:20:50.554 DEBUG:boto:Token: None 2022-09-03T05:20:50.555 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/ 2022-09-03T05:20:50.556 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0= 2022-09-03T05:20:50.556 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0=', 'Content-Length': '0'} 2022-09-03T05:20:50.559 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006003ad6a22cf095e-006312e432-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.559 DEBUG:boto:b'dsgvcy-10myobj1000falsemyobj2022-09-03T05:20:36.801Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:50.560 DEBUG:boto:path=/dsgvcy-10/myobj 2022-09-03T05:20:50.561 DEBUG:boto:auth_path=/dsgvcy-10/myobj 2022-09-03T05:20:50.561 DEBUG:boto:Method: HEAD 2022-09-03T05:20:50.562 DEBUG:boto:Path: /dsgvcy-10/myobj 2022-09-03T05:20:50.562 DEBUG:boto:Data: 2022-09-03T05:20:50.563 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.563 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:50.564 DEBUG:boto:Port: 8001 2022-09-03T05:20:50.564 DEBUG:boto:Params: {} 2022-09-03T05:20:50.565 DEBUG:boto:Token: None 2022-09-03T05:20:50.566 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/myobj 2022-09-03T05:20:50.566 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MJODqziVujwFk28ctWmOmqiv5aU= 2022-09-03T05:20:50.567 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MJODqziVujwFk28ctWmOmqiv5aU=', 'Content-Length': '0'} 2022-09-03T05:20:50.569 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000006e29224e6473cf6e-006312e432-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.570 DEBUG:boto:path=/dsgvcy-10/myobj 2022-09-03T05:20:50.570 DEBUG:boto:auth_path=/dsgvcy-10/myobj 2022-09-03T05:20:50.571 DEBUG:boto:Method: HEAD 2022-09-03T05:20:50.571 DEBUG:boto:Path: /dsgvcy-10/myobj 2022-09-03T05:20:50.572 DEBUG:boto:Data: 2022-09-03T05:20:50.572 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.573 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:50.573 DEBUG:boto:Port: 8000 2022-09-03T05:20:50.574 DEBUG:boto:Params: {} 2022-09-03T05:20:50.574 DEBUG:boto:Token: None 2022-09-03T05:20:50.575 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/myobj 2022-09-03T05:20:50.575 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MJODqziVujwFk28ctWmOmqiv5aU= 2022-09-03T05:20:50.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MJODqziVujwFk28ctWmOmqiv5aU=', 'Content-Length': '0'} 2022-09-03T05:20:50.578 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004da31304e3e064ad-006312e432-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.579 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:20:50.579 DEBUG:boto:path=/dsgvcy-10/myobj 2022-09-03T05:20:50.580 DEBUG:boto:auth_path=/dsgvcy-10/myobj 2022-09-03T05:20:50.581 DEBUG:boto:Method: GET 2022-09-03T05:20:50.581 DEBUG:boto:Path: /dsgvcy-10/myobj 2022-09-03T05:20:50.582 DEBUG:boto:Data: 2022-09-03T05:20:50.582 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.583 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:50.583 DEBUG:boto:Port: 8001 2022-09-03T05:20:50.584 DEBUG:boto:Params: {} 2022-09-03T05:20:50.584 DEBUG:boto:Token: None 2022-09-03T05:20:50.585 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/myobj 2022-09-03T05:20:50.586 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yehJ92mjItLqf3+lm2MT9ElOWMw= 2022-09-03T05:20:50.586 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yehJ92mjItLqf3+lm2MT9ElOWMw=', 'Content-Length': '0'} 2022-09-03T05:20:50.588 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e5627e263509a668-006312e432-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.589 DEBUG:boto:path=/dsgvcy-10/myobj 2022-09-03T05:20:50.590 DEBUG:boto:auth_path=/dsgvcy-10/myobj 2022-09-03T05:20:50.590 DEBUG:boto:Method: GET 2022-09-03T05:20:50.591 DEBUG:boto:Path: /dsgvcy-10/myobj 2022-09-03T05:20:50.592 DEBUG:boto:Data: 2022-09-03T05:20:50.592 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.593 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:50.593 DEBUG:boto:Port: 8000 2022-09-03T05:20:50.594 DEBUG:boto:Params: {} 2022-09-03T05:20:50.594 DEBUG:boto:Token: None 2022-09-03T05:20:50.595 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/myobj 2022-09-03T05:20:50.596 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yehJ92mjItLqf3+lm2MT9ElOWMw= 2022-09-03T05:20:50.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yehJ92mjItLqf3+lm2MT9ElOWMw=', 'Content-Length': '0'} 2022-09-03T05:20:50.599 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:20:36 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000006c0cf81b5eb78186-006312e432-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.642 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:50.643 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:50.644 DEBUG:boto:path=/dsgvcy-10/?prefix=myobj 2022-09-03T05:20:50.644 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=myobj 2022-09-03T05:20:50.645 DEBUG:boto:Method: GET 2022-09-03T05:20:50.646 DEBUG:boto:Path: /dsgvcy-10/?prefix=myobj 2022-09-03T05:20:50.646 DEBUG:boto:Data: 2022-09-03T05:20:50.647 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.647 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:20:50.648 DEBUG:boto:Port: 8001 2022-09-03T05:20:50.648 DEBUG:boto:Params: {} 2022-09-03T05:20:50.649 DEBUG:boto:Token: None 2022-09-03T05:20:50.650 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/ 2022-09-03T05:20:50.651 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0= 2022-09-03T05:20:50.651 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0=', 'Content-Length': '0'} 2022-09-03T05:20:50.653 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005a23ff3179ff7bdd-006312e432-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.654 DEBUG:boto:b'dsgvcy-10myobj1000falsemyobj2022-09-03T05:20:36.801Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:50.655 DEBUG:boto:path=/dsgvcy-10/ 2022-09-03T05:20:50.656 DEBUG:boto:auth_path=/dsgvcy-10/ 2022-09-03T05:20:50.657 DEBUG:boto:path=/dsgvcy-10/?prefix=myobj 2022-09-03T05:20:50.658 DEBUG:boto:auth_path=/dsgvcy-10/?prefix=myobj 2022-09-03T05:20:50.659 DEBUG:boto:Method: GET 2022-09-03T05:20:50.661 DEBUG:boto:Path: /dsgvcy-10/?prefix=myobj 2022-09-03T05:20:50.662 DEBUG:boto:Data: 2022-09-03T05:20:50.663 DEBUG:boto:Headers: {} 2022-09-03T05:20:50.664 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:20:50.665 DEBUG:boto:Port: 8000 2022-09-03T05:20:50.666 DEBUG:boto:Params: {} 2022-09-03T05:20:50.670 DEBUG:boto:Token: None 2022-09-03T05:20:50.671 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:20:50 GMT /dsgvcy-10/ 2022-09-03T05:20:50.672 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0= 2022-09-03T05:20:50.673 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:20:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8YQyL1veLG3mK+59TcB4ljUH9W0=', 'Content-Length': '0'} 2022-09-03T05:20:50.676 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000b1ba2f3da604393-006312e432-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:20:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:20:50.677 DEBUG:boto:b'dsgvcy-10myobj1000falsemyobj2022-09-03T05:20:36.801Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:20:50.679 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-10 zones={test-zone3, test-zone1} 2022-09-03T05:20:50.680 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-10', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:20:50.681 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-10', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:20:50.682 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-10 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:20:50.747 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:20:50.748 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:20:50.799 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:50.801+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:20:50.799 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.7.5, , ] 2022-09-03T05:20:50.800 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:20:55.801 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:20:55.803+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:20:55.802 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.7.5, , ] 2022-09-03T05:20:55.802 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:00.802 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:21:00.805+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:21:00.803 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:00.804 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:05.803 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:21:05.806+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:21:05.804 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:05.804 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:10.805 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:21:10.807+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:21:10.806 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:10.806 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:15.807 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:21:15.809+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:21:15.807 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:15.808 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:20.808 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:21:20.810+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:21:20.808 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:20.808 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:25.809 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:21:25.810+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:21:25.809 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:25.810 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:30.811 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:21:30.813+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:21:30.812 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:30.812 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:35.812 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:21:35.814+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:21:35.813 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:35.813 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:40.813 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:21:40.816+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:21:40.814 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:40.815 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:45.815 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:21:45.817+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:21:45.816 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:45.816 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:50.817 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:21:50.819+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:21:50.817 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:50.818 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:55.818 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:21:55.821+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:21:55.819 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:21:55.819 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:00.819 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:22:00.822+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:22:00.820 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:00.820 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:05.824 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:22:05.823+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:22:05.825 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:05.825 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:10.826 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:22:10.825+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:22:10.827 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:10.827 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:15.824 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:22:15.825+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:22:15.825 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:15.825 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:20.825 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:22:20.828+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:22:20.826 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:20.827 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:25.827 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:22:25.829+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:22:25.827 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:25.828 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:30.828 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:22:30.830+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:22:30.829 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:30.829 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:35.830 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:22:35.832+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:22:35.830 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:35.831 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:40.831 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:22:40.833+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:22:40.832 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:40.846 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:45.834 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:22:45.835+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:22:45.839 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:45.840 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:50.836 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:22:50.836+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:22:50.837 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:50.837 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:55.835 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:22:55.838+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:22:55.836 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:22:55.836 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:00.837 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:23:00.839+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:23:00.838 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:00.838 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:05.838 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:23:05.840+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:23:05.839 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:05.839 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:10.840 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:23:10.841+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:23:10.841 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:10.841 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:15.841 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:23:15.843+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:23:15.842 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:15.842 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:20.843 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:23:20.845+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:23:20.844 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:20.845 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:25.844 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:23:25.846+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:23:25.845 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:25.845 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:30.847 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:23:30.848+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:23:30.850 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:30.851 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:35.848 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:23:35.849+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:23:35.849 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:35.849 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:40.849 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:23:40.851+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:23:40.850 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:40.850 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:45.850 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:23:45.852+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:23:45.851 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:45.851 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:50.855 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:23:50.853+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:23:50.855 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:50.855 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:55.853 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:23:55.855+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:23:55.853 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:23:55.853 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:00.854 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:24:00.855+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:24:00.855 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:00.856 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:05.856 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:24:05.857+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:24:05.857 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:05.858 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:10.857 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:24:10.859+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:24:10.858 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:10.858 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:15.858 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:24:15.861+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:24:15.859 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:15.860 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:20.860 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:24:20.862+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:24:20.861 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:20.861 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:25.861 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:24:25.863+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:24:25.862 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:25.862 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:30.863 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:24:30.865+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:24:30.863 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:30.864 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:35.864 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:24:35.866+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:24:35.865 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:35.865 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:40.870 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:24:40.868+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:24:40.871 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:40.872 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:45.867 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:24:45.869+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:24:45.868 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:45.868 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:50.869 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:24:50.870+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:24:50.870 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:50.871 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:55.870 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:24:55.872+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:24:55.870 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:24:55.871 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:25:00.876 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:25:00.874+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:25:00.877 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:25:00.877 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:25:05.873 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:25:05.875+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:25:05.873 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:25:05.874 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:25:10.875 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:25:10.877+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:25:10.875 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:25:10.876 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:25:15.876 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:25:15.878+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:25:15.876 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:25:15.876 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:25:20.883 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:25:20.880+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:25:20.884 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:25:20.884 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:25:25.880 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:25:25.881+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:25:25.880 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:25:25.881 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:25:30.881 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:25:30.883+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:25:30.882 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:25:30.882 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:25:35.882 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:25:35.884+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:25:35.882 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:25:35.883 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:25:40.889 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:25:40.884+0000 7f064969f600 1 waiting for incremental sync to catch up: 2022-09-03T05:25:40.890 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.4.5, , , , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:25:40.890 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.4.5, , 00000000001.8.5, , , , 00000000001.7.5, 00000000001.7.5, , ] 2022-09-03T05:25:45.884 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:25:45.887+0000 7f064969f600 0 bucket checkpoint timed out waiting for incremental sync to catch up 2022-09-03T05:25:45.885 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:25:45.887+0000 7f064969f600 0 bucket sync checkpoint failed: (110) Connection timed out 2022-09-03T05:25:45.885 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:25:45.887+0000 7f064969f600 -1 bucket sync checkpoint failed: (110) Connection timed out 2022-09-03T05:25:45.902 DEBUG:teuthology.orchestra.run:got remote process result: 110 2022-09-03T05:25:45.913 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_object_sync ... FAIL 2022-09-03T05:25:45.916 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-13 2022-09-03T05:25:45.917 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:25:45.918 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:25:45.919 DEBUG:boto:Method: PUT 2022-09-03T05:25:45.920 DEBUG:boto:Path: /dsgvcy-13/ 2022-09-03T05:25:45.920 DEBUG:boto:Data: 2022-09-03T05:25:45.921 DEBUG:boto:Headers: {} 2022-09-03T05:25:45.921 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:45.922 DEBUG:boto:Port: 8000 2022-09-03T05:25:45.922 DEBUG:boto:Params: {} 2022-09-03T05:25:45.925 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:25:45.925 DEBUG:boto:Token: None 2022-09-03T05:25:45.926 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:25:45 GMT /dsgvcy-13/ 2022-09-03T05:25:45.927 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cpcWVz2eCojHBKdvUjaReEJZraU= 2022-09-03T05:25:45.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cpcWVz2eCojHBKdvUjaReEJZraU=', 'Content-Length': '0'} 2022-09-03T05:25:45.959 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000004ccccc0f7fa0405a-006312e559-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:25:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:45.961 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=dsgvcy-14 2022-09-03T05:25:45.962 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:25:45.963 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:25:45.964 DEBUG:boto:Method: PUT 2022-09-03T05:25:45.964 DEBUG:boto:Path: /dsgvcy-14/ 2022-09-03T05:25:45.965 DEBUG:boto:Data: 2022-09-03T05:25:45.966 DEBUG:boto:Headers: {} 2022-09-03T05:25:45.967 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:45.968 DEBUG:boto:Port: 8000 2022-09-03T05:25:45.969 DEBUG:boto:Params: {} 2022-09-03T05:25:45.970 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:25:45.971 DEBUG:boto:Token: None 2022-09-03T05:25:45.972 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:25:45 GMT /dsgvcy-14/ 2022-09-03T05:25:45.974 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XjC+HVs6p7heJO6k99UrRDc4wPs= 2022-09-03T05:25:45.974 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XjC+HVs6p7heJO6k99UrRDc4wPs=', 'Content-Length': '0'} 2022-09-03T05:25:46.002 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000008cd64a4a5df8fba-006312e559-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:46.002 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=dsgvcy-15 2022-09-03T05:25:46.004 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:25:46.004 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:25:46.005 DEBUG:boto:Method: PUT 2022-09-03T05:25:46.005 DEBUG:boto:Path: /dsgvcy-15/ 2022-09-03T05:25:46.006 DEBUG:boto:Data: 2022-09-03T05:25:46.006 DEBUG:boto:Headers: {} 2022-09-03T05:25:46.007 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:25:46.007 DEBUG:boto:Port: 8001 2022-09-03T05:25:46.008 DEBUG:boto:Params: {} 2022-09-03T05:25:46.008 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T05:25:46.009 DEBUG:boto:Token: None 2022-09-03T05:25:46.010 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:25:46 GMT /dsgvcy-15/ 2022-09-03T05:25:46.011 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YipGOCCWB5OzYIiwXkYuguK1Dms= 2022-09-03T05:25:46.012 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YipGOCCWB5OzYIiwXkYuguK1Dms=', 'Content-Length': '0'} 2022-09-03T05:25:46.054 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000086db7f114d7c8f5-006312e55a-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:46.055 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:25:46.056 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:25:46.057 DEBUG:boto:Method: HEAD 2022-09-03T05:25:46.058 DEBUG:boto:Path: /dsgvcy-13/ 2022-09-03T05:25:46.059 DEBUG:boto:Data: 2022-09-03T05:25:46.060 DEBUG:boto:Headers: {} 2022-09-03T05:25:46.061 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:46.062 DEBUG:boto:Port: 8000 2022-09-03T05:25:46.062 DEBUG:boto:Params: {} 2022-09-03T05:25:46.064 DEBUG:boto:Token: None 2022-09-03T05:25:46.064 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:46 GMT /dsgvcy-13/ 2022-09-03T05:25:46.065 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X+gOv0Imbb8zXUHkDOYJbD1gevs= 2022-09-03T05:25:46.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X+gOv0Imbb8zXUHkDOYJbD1gevs=', 'Content-Length': '0'} 2022-09-03T05:25:46.071 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006f47305918533f1c-006312e55a-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:46.072 DEBUG:boto:path=/dsgvcy-13/myobj 2022-09-03T05:25:46.073 DEBUG:boto:auth_path=/dsgvcy-13/myobj 2022-09-03T05:25:46.073 DEBUG:boto:Method: PUT 2022-09-03T05:25:46.074 DEBUG:boto:Path: /dsgvcy-13/myobj 2022-09-03T05:25:46.074 DEBUG:boto:Data: 2022-09-03T05:25:46.075 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:25:46.075 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:46.076 DEBUG:boto:Port: 8000 2022-09-03T05:25:46.076 DEBUG:boto:Params: {} 2022-09-03T05:25:46.077 DEBUG:boto:Token: None 2022-09-03T05:25:46.078 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:25:46 GMT /dsgvcy-13/myobj 2022-09-03T05:25:46.078 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y8esx8R6y5Jx1qYea2N4WR3GuKw= 2022-09-03T05:25:46.079 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:25:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y8esx8R6y5Jx1qYea2N4WR3GuKw='} 2022-09-03T05:25:46.098 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000098b8cf42377d58eb-006312e55a-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:46.100 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:25:46.103 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:25:46.104 DEBUG:boto:Method: HEAD 2022-09-03T05:25:46.105 DEBUG:boto:Path: /dsgvcy-14/ 2022-09-03T05:25:46.106 DEBUG:boto:Data: 2022-09-03T05:25:46.107 DEBUG:boto:Headers: {} 2022-09-03T05:25:46.107 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:46.108 DEBUG:boto:Port: 8000 2022-09-03T05:25:46.109 DEBUG:boto:Params: {} 2022-09-03T05:25:46.110 DEBUG:boto:Token: None 2022-09-03T05:25:46.111 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:46 GMT /dsgvcy-14/ 2022-09-03T05:25:46.113 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t34dWZulIn+VrClh/ZzfVSdWei4= 2022-09-03T05:25:46.114 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t34dWZulIn+VrClh/ZzfVSdWei4=', 'Content-Length': '0'} 2022-09-03T05:25:46.117 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000066132fd3d306be94-006312e55a-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:46.119 DEBUG:boto:path=/dsgvcy-14/myobj 2022-09-03T05:25:46.120 DEBUG:boto:auth_path=/dsgvcy-14/myobj 2022-09-03T05:25:46.121 DEBUG:boto:Method: PUT 2022-09-03T05:25:46.121 DEBUG:boto:Path: /dsgvcy-14/myobj 2022-09-03T05:25:46.123 DEBUG:boto:Data: 2022-09-03T05:25:46.124 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:25:46.125 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:46.126 DEBUG:boto:Port: 8000 2022-09-03T05:25:46.127 DEBUG:boto:Params: {} 2022-09-03T05:25:46.128 DEBUG:boto:Token: None 2022-09-03T05:25:46.129 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:25:46 GMT /dsgvcy-14/myobj 2022-09-03T05:25:46.131 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oFuB4hMgwFgr2hItNKMc7P0BEcQ= 2022-09-03T05:25:46.131 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:25:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oFuB4hMgwFgr2hItNKMc7P0BEcQ='} 2022-09-03T05:25:46.138 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005f73c03552d25878-006312e55a-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:46.139 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:25:46.139 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:25:46.140 DEBUG:boto:Method: HEAD 2022-09-03T05:25:46.142 DEBUG:boto:Path: /dsgvcy-15/ 2022-09-03T05:25:46.143 DEBUG:boto:Data: 2022-09-03T05:25:46.143 DEBUG:boto:Headers: {} 2022-09-03T05:25:46.144 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:25:46.144 DEBUG:boto:Port: 8001 2022-09-03T05:25:46.144 DEBUG:boto:Params: {} 2022-09-03T05:25:46.145 DEBUG:boto:Token: None 2022-09-03T05:25:46.146 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:46 GMT /dsgvcy-15/ 2022-09-03T05:25:46.146 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m/8aeNTMe5+OyU72SUbJYxNuvkg= 2022-09-03T05:25:46.147 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m/8aeNTMe5+OyU72SUbJYxNuvkg=', 'Content-Length': '0'} 2022-09-03T05:25:46.149 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003f430b11180bebbf-006312e55a-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:46.150 DEBUG:boto:path=/dsgvcy-15/myobj 2022-09-03T05:25:46.151 DEBUG:boto:auth_path=/dsgvcy-15/myobj 2022-09-03T05:25:46.151 DEBUG:boto:Method: PUT 2022-09-03T05:25:46.152 DEBUG:boto:Path: /dsgvcy-15/myobj 2022-09-03T05:25:46.152 DEBUG:boto:Data: 2022-09-03T05:25:46.153 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:25:46.153 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:25:46.154 DEBUG:boto:Port: 8001 2022-09-03T05:25:46.155 DEBUG:boto:Params: {} 2022-09-03T05:25:46.156 DEBUG:boto:Token: None 2022-09-03T05:25:46.157 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:25:46 GMT /dsgvcy-15/myobj 2022-09-03T05:25:46.158 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YwpQVO3Q7EoSH05Uv5MLZWol9g4= 2022-09-03T05:25:46.159 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:25:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YwpQVO3Q7EoSH05Uv5MLZWol9g4='} 2022-09-03T05:25:46.172 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000039a9ad5bfec2ef09-006312e55a-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:46.173 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:25:46.174 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:25:46.175 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:25:46.232 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:25:46.233 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:25:46.275 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=41ecd2a5-afc0-4eae-9349-7dfe39baa1cb 2022-09-03T05:25:46.288 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:25:46.289 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:25:46.289 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:25:46.289 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:18:18.906852Z" 2022-09-03T05:25:46.290 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:25:46.290 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:25:46.290 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:25:46.290 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:25:46.291 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:25:46.291 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:25:46.292 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182746.041492_111.1", 2022-09-03T05:25:46.292 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:25:46.041492Z" 2022-09-03T05:25:46.292 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:25:46.292 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:25:46.293 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:25:46.293 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:20:30.928498Z" 2022-09-03T05:25:46.293 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:25:46.294 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:25:46.296 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182746.041492_111.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:25:46.297 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:25:46.298 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:25:46.299 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:25:46.300 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:25:46.353 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:25:46.353 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:25:46.384 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:25:46.387+0000 7fb7a5fea600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:25:46.385 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:25:46.388+0000 7fb7a5fea600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:25:46.399 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:25:46.399 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:25:46.400 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:25:46.400 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:25:46.400 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:25:46.400 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:25:46.401 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:25:46.401 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:25:46.401 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:25:46.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:25:46.402 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:25:46.403 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:25:46.403 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:25:46.404 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:25:46.404 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:25:46.405 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:25:46.405 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:25:46.405 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:25:46.406 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:25:46.406 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:25:46.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:25:46.407 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:25:46.408 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:25:46.408 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:25:46.409 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:25:46.409 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:25:46.410 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:25:46.410 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:25:46.411 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:25:46.411 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:25:46.412 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:25:46.412 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:25:46.413 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:25:46.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:25:46.414 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:25:46.414 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:25:46.415 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:25:46.415 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182746.041492_111.1", 2022-09-03T05:25:46.416 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:25:46.416 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:25:46.417 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:25:46.417 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:46.041492Z", 2022-09-03T05:25:46.418 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:25:46.418 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:25:46.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:25:46.419 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:25:46.420 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:25:46.420 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:25:46.421 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:25:46.421 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:25:46.422 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:25:46.422 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:25:46.423 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:25:46.423 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:25:46.424 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:25:46.424 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:25:46.425 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:25:46.425 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:25:46.425 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:25:46.426 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:25:46.426 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:25:46.427 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:25:46.427 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:25:46.428 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:25:46Z" 2022-09-03T05:25:46.428 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:25:46.429 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662182746.041492_111.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:25:46.041492Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:25:46Z" } 2022-09-03T05:25:46.429 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662182746.041492_111.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:25:46.041492Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:25:46.430 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182746.041492_111.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:25:46.430 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182746.041492_111.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:25:46.431 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:25:46.431 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:25:46.432 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:25:46.432 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:25:46.433 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:25:46.483 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:25:46.483 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:25:46.525 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:25:46.526+0000 7f3ac8f92600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:25:46.526 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:25:46.527+0000 7f3ac8f92600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:25:46.541 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:25:46.541 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:25:46.541 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:25:46.542 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:25:46.542 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:25:46.542 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:25:46.543 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:25:46.544 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:25:46.544 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:25:46.544 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:25:46.545 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:25:46.545 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:25:46.545 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:25:46.545 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:25:46.546 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:25:46.546 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:25:46.546 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:25:46.546 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:25:46.547 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:25:46.547 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:25:46.547 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:25:46.548 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:25:46.549 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:25:46.550 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:25:46.550 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:25:46.551 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:25:46.551 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:25:46.552 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:25:46.552 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:25:46.553 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:25:46.553 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:25:46.554 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:25:46.554 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:25:46.555 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:25:46.555 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:25:46.556 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:25:46.556 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:25:46.556 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182745.988740_104.1", 2022-09-03T05:25:46.556 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:25:46.557 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:25:46.557 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:25:46.557 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:45.988740Z", 2022-09-03T05:25:46.558 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:25:46.558 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:25:46.559 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:25:46.559 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:25:46.560 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:25:46.560 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:25:46.560 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:25:46.561 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:25:46.561 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:25:46.562 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:25:46.562 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:25:46.563 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:25:46.563 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:25:46.564 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:25:46.564 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:25:46.565 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:25:46.565 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:25:46.566 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:25:46.566 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:25:46.566 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:25:46.567 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:25:46.567 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:25:46Z" 2022-09-03T05:25:46.567 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:25:46.569 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662182745.988740_104.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:25:45.988740Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:25:46Z" } 2022-09-03T05:25:46.569 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662182745.988740_104.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:25:45.988740Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:25:46.570 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182746.041492_111.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:25:46.571 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182745.988740_104.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:25:46.571 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662182746.041492_111.1 target=1_1662182745.988740_104.1 2022-09-03T05:25:51.573 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:25:51.575 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:25:51.577 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:25:51.634 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:25:51.635 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:25:51.690 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:25:51.692+0000 7f163612f600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:25:51.690 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:25:51.693+0000 7f163612f600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:25:51.716 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:25:51.717 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:25:51.717 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:25:51.717 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:25:51.717 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:25:51.718 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:25:51.718 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:25:51.718 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:25:51.719 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:25:51.719 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:25:51.719 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:25:51.720 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:25:51.720 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:25:51.720 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:25:51.720 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:25:51.720 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:25:51.721 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:25:51.721 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:25:51.721 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:25:51.721 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:25:51.722 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:25:51.722 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:25:51.722 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:25:51.723 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:25:51.723 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:25:51.723 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:25:51.723 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:25:51.724 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:25:51.724 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:25:51.724 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:25:51.725 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:25:51.725 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:25:51.725 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:25:51.726 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:25:51.726 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:25:51.726 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:25:51.727 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:25:51.727 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182746.041492_111.1", 2022-09-03T05:25:51.727 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:25:51.727 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:25:51.728 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:25:51.728 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:46.041492Z", 2022-09-03T05:25:51.729 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:25:51.729 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:25:51.730 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:25:51.730 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:25:51.731 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:25:51.731 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:25:51.732 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:25:51.732 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:25:51.732 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:25:51.733 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:25:51.733 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:25:51.734 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:25:51.734 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:25:51.735 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:25:51.735 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:25:51.736 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:25:51.736 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:25:51.737 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:25:51.737 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:25:51.738 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:25:51.738 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:25:51.738 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:25:51Z" 2022-09-03T05:25:51.739 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:25:51.741 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662182746.041492_111.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:25:46.041492Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:25:51Z" } 2022-09-03T05:25:51.741 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662182746.041492_111.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:25:46.041492Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:25:51.742 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182746.041492_111.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:25:51.743 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182746.041492_111.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:25:51.743 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:25:51.744 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:25:51.744 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:25:51.745 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:25:51.746 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:25:51.805 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:25:51.805 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:25:51.835 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:25:51.837+0000 7f668d238600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:25:51.836 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:25:51.839+0000 7f668d238600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:25:51.853 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:25:51.854 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:25:51.854 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:25:51.854 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:25:51.855 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:25:51.855 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:25:51.856 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:25:51.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:25:51.856 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:25:51.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:25:51.857 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:25:51.858 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:25:51.858 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:25:51.858 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:25:51.858 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:25:51.859 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:25:51.859 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:25:51.859 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:25:51.860 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:25:51.860 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:25:51.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:25:51.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:25:51.861 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:25:51.861 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:25:51.862 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:25:51.862 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:25:51.862 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:25:51.863 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:25:51.863 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:25:51.863 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:25:51.863 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:25:51.864 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:25:51.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:25:51.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:25:51.864 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:25:51.865 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:25:51.865 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:25:51.865 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182746.041492_111.1", 2022-09-03T05:25:51.865 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:25:51.866 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:25:51.866 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:25:51.866 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:46.041492Z", 2022-09-03T05:25:51.867 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:25:51.867 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:25:51.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:25:51.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:25:51.868 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:25:51.869 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:25:51.869 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:25:51.869 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:25:51.870 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:25:51.870 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:25:51.870 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:25:51.871 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:25:51.871 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:25:51.871 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:25:51.871 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:25:51.872 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:25:51.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:25:51.873 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:25:51.873 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:25:51.873 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:25:51.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:25:51.874 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:25:51Z" 2022-09-03T05:25:51.874 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:25:51.876 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662182746.041492_111.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:25:46.041492Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:25:51Z" } 2022-09-03T05:25:51.876 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662182746.041492_111.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:25:46.041492Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:25:51.877 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182746.041492_111.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:25:51.878 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182746.041492_111.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:25:51.879 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:25:51.879 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-13', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:25:51.880 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-13', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:25:51.881 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-13 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:25:51.938 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:25:51.939 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:25:51.971 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:25:51.973+0000 7f65bdf66600 1 bucket sync caught up with source: 2022-09-03T05:25:51.972 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.6.5, , , , , , , , , ] 2022-09-03T05:25:51.972 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.6.5, , , , , , , , , ] 2022-09-03T05:25:51.973 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:25:51.973+0000 7f65bdf66600 0 bucket checkpoint complete 2022-09-03T05:25:51.983 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-13 zones={test-zone2, test-zone1} 2022-09-03T05:25:51.983 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:25:51.984 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:25:51.984 DEBUG:boto:Method: HEAD 2022-09-03T05:25:51.985 DEBUG:boto:Path: /dsgvcy-13/ 2022-09-03T05:25:51.985 DEBUG:boto:Data: 2022-09-03T05:25:51.986 DEBUG:boto:Headers: {} 2022-09-03T05:25:51.987 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:51.987 DEBUG:boto:Port: 8000 2022-09-03T05:25:51.988 DEBUG:boto:Params: {} 2022-09-03T05:25:51.988 DEBUG:boto:Token: None 2022-09-03T05:25:51.989 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:51 GMT /dsgvcy-13/ 2022-09-03T05:25:51.989 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0cPgRO3NuT7MSup1sdIyBP58ZtY= 2022-09-03T05:25:51.990 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0cPgRO3NuT7MSup1sdIyBP58ZtY=', 'Content-Length': '0'} 2022-09-03T05:25:51.993 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f140acd5d5274175-006312e55f-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:25:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:51.993 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:25:51.994 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:25:51.995 DEBUG:boto:Method: HEAD 2022-09-03T05:25:51.995 DEBUG:boto:Path: /dsgvcy-13/ 2022-09-03T05:25:51.996 DEBUG:boto:Data: 2022-09-03T05:25:51.996 DEBUG:boto:Headers: {} 2022-09-03T05:25:51.996 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:51.997 DEBUG:boto:Port: 8000 2022-09-03T05:25:51.997 DEBUG:boto:Params: {} 2022-09-03T05:25:51.998 DEBUG:boto:Token: None 2022-09-03T05:25:51.999 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:51 GMT /dsgvcy-13/ 2022-09-03T05:25:51.999 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0cPgRO3NuT7MSup1sdIyBP58ZtY= 2022-09-03T05:25:52.000 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0cPgRO3NuT7MSup1sdIyBP58ZtY=', 'Content-Length': '0'} 2022-09-03T05:25:52.003 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006a58ce1f2f272749-006312e560-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.004 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:25:52.006 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:25:52.007 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:25:52.008 DEBUG:boto:path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.009 DEBUG:boto:auth_path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.010 DEBUG:boto:Method: GET 2022-09-03T05:25:52.011 DEBUG:boto:Path: /dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.012 DEBUG:boto:Data: 2022-09-03T05:25:52.012 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.013 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:52.014 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.015 DEBUG:boto:Params: {} 2022-09-03T05:25:52.016 DEBUG:boto:Token: None 2022-09-03T05:25:52.017 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/?versions 2022-09-03T05:25:52.018 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rAj5wp5T9vzs8TUB/l9P2sJDkPc= 2022-09-03T05:25:52.019 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rAj5wp5T9vzs8TUB/l9P2sJDkPc=', 'Content-Length': '0'} 2022-09-03T05:25:52.021 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e9927290f4f912a2-006312e560-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.022 DEBUG:boto:b'dsgvcy-13999falsemyobjnulltrue2022-09-03T05:25:46.085Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:52.024 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2022-09-03T05:25:52.025 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:25:52.025 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:25:52.026 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:25:52.027 DEBUG:boto:path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.027 DEBUG:boto:auth_path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.028 DEBUG:boto:Method: GET 2022-09-03T05:25:52.029 DEBUG:boto:Path: /dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.029 DEBUG:boto:Data: 2022-09-03T05:25:52.030 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.031 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:52.031 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.032 DEBUG:boto:Params: {} 2022-09-03T05:25:52.033 DEBUG:boto:Token: None 2022-09-03T05:25:52.034 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/?versions 2022-09-03T05:25:52.034 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rAj5wp5T9vzs8TUB/l9P2sJDkPc= 2022-09-03T05:25:52.035 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rAj5wp5T9vzs8TUB/l9P2sJDkPc=', 'Content-Length': '0'} 2022-09-03T05:25:52.039 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000097ab1f63e889a605-006312e560-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.041 DEBUG:boto:b'dsgvcy-13999falsemyobjnulltrue2022-09-03T05:25:46.085Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:52.042 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2022-09-03T05:25:52.043 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:25:52.044 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:25:52.045 DEBUG:boto:path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.046 DEBUG:boto:auth_path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.047 DEBUG:boto:Method: GET 2022-09-03T05:25:52.048 DEBUG:boto:Path: /dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.049 DEBUG:boto:Data: 2022-09-03T05:25:52.050 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.052 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:52.053 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.054 DEBUG:boto:Params: {} 2022-09-03T05:25:52.055 DEBUG:boto:Token: None 2022-09-03T05:25:52.056 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/?versions 2022-09-03T05:25:52.057 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rAj5wp5T9vzs8TUB/l9P2sJDkPc= 2022-09-03T05:25:52.058 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rAj5wp5T9vzs8TUB/l9P2sJDkPc=', 'Content-Length': '0'} 2022-09-03T05:25:52.061 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bc5f42c8b2b8f783-006312e560-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.068 DEBUG:boto:b'dsgvcy-13999falsemyobjnulltrue2022-09-03T05:25:46.085Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:52.072 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:25:52.074 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:25:52.075 DEBUG:boto:path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.077 DEBUG:boto:auth_path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.077 DEBUG:boto:Method: GET 2022-09-03T05:25:52.078 DEBUG:boto:Path: /dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.079 DEBUG:boto:Data: 2022-09-03T05:25:52.081 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.081 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:52.082 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.083 DEBUG:boto:Params: {} 2022-09-03T05:25:52.084 DEBUG:boto:Token: None 2022-09-03T05:25:52.086 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/?versions 2022-09-03T05:25:52.087 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rAj5wp5T9vzs8TUB/l9P2sJDkPc= 2022-09-03T05:25:52.089 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rAj5wp5T9vzs8TUB/l9P2sJDkPc=', 'Content-Length': '0'} 2022-09-03T05:25:52.093 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000046812289f601e8ce-006312e560-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.096 DEBUG:boto:b'dsgvcy-13999falsemyobjnulltrue2022-09-03T05:25:46.085Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:52.098 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:25:52.100 DEBUG:boto:path=/dsgvcy-13/myobj 2022-09-03T05:25:52.101 DEBUG:boto:auth_path=/dsgvcy-13/myobj 2022-09-03T05:25:52.102 DEBUG:boto:path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.102 DEBUG:boto:auth_path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.103 DEBUG:boto:Method: GET 2022-09-03T05:25:52.104 DEBUG:boto:Path: /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.104 DEBUG:boto:Data: 2022-09-03T05:25:52.105 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.105 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:52.106 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.107 DEBUG:boto:Params: {} 2022-09-03T05:25:52.108 DEBUG:boto:Token: None 2022-09-03T05:25:52.108 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.109 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Uzjkl0upMergKmBKzcI1kOdQ++k= 2022-09-03T05:25:52.110 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Uzjkl0upMergKmBKzcI1kOdQ++k=', 'Content-Length': '0'} 2022-09-03T05:25:52.112 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004c8d9aae9717570f-006312e560-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.121 DEBUG:boto:path=/dsgvcy-13/myobj 2022-09-03T05:25:52.122 DEBUG:boto:auth_path=/dsgvcy-13/myobj 2022-09-03T05:25:52.123 DEBUG:boto:path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.124 DEBUG:boto:auth_path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.125 DEBUG:boto:Method: GET 2022-09-03T05:25:52.126 DEBUG:boto:Path: /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.127 DEBUG:boto:Data: 2022-09-03T05:25:52.128 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.129 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:52.129 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.130 DEBUG:boto:Params: {} 2022-09-03T05:25:52.131 DEBUG:boto:Token: None 2022-09-03T05:25:52.132 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.133 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Uzjkl0upMergKmBKzcI1kOdQ++k= 2022-09-03T05:25:52.134 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Uzjkl0upMergKmBKzcI1kOdQ++k=', 'Content-Length': '0'} 2022-09-03T05:25:52.136 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007dd5a6bbb4162fa9-006312e560-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.179 DEBUG:boto:path=/dsgvcy-13/myobj 2022-09-03T05:25:52.180 DEBUG:boto:auth_path=/dsgvcy-13/myobj 2022-09-03T05:25:52.181 DEBUG:boto:path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.182 DEBUG:boto:auth_path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.182 DEBUG:boto:Method: HEAD 2022-09-03T05:25:52.183 DEBUG:boto:Path: /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.183 DEBUG:boto:Data: 2022-09-03T05:25:52.184 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.184 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:52.185 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.186 DEBUG:boto:Params: {} 2022-09-03T05:25:52.187 DEBUG:boto:Token: None 2022-09-03T05:25:52.189 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.190 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WKdLrO6dbWXvpC/y1O93K3anAlY= 2022-09-03T05:25:52.191 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WKdLrO6dbWXvpC/y1O93K3anAlY=', 'Content-Length': '0'} 2022-09-03T05:25:52.194 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000bafd0bc78a2f4b28-006312e560-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.195 DEBUG:boto:path=/dsgvcy-13/myobj 2022-09-03T05:25:52.196 DEBUG:boto:auth_path=/dsgvcy-13/myobj 2022-09-03T05:25:52.197 DEBUG:boto:path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.199 DEBUG:boto:auth_path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.199 DEBUG:boto:Method: HEAD 2022-09-03T05:25:52.200 DEBUG:boto:Path: /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.200 DEBUG:boto:Data: 2022-09-03T05:25:52.201 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.201 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:52.202 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.203 DEBUG:boto:Params: {} 2022-09-03T05:25:52.203 DEBUG:boto:Token: None 2022-09-03T05:25:52.206 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.207 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WKdLrO6dbWXvpC/y1O93K3anAlY= 2022-09-03T05:25:52.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WKdLrO6dbWXvpC/y1O93K3anAlY=', 'Content-Length': '0'} 2022-09-03T05:25:52.214 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000001fd22f5f40fff265-006312e560-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.217 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:25:52.218 DEBUG:boto:path=/dsgvcy-13/myobj 2022-09-03T05:25:52.219 DEBUG:boto:auth_path=/dsgvcy-13/myobj 2022-09-03T05:25:52.220 DEBUG:boto:path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.221 DEBUG:boto:auth_path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.225 DEBUG:boto:Method: GET 2022-09-03T05:25:52.226 DEBUG:boto:Path: /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.226 DEBUG:boto:Data: 2022-09-03T05:25:52.227 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.228 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:52.228 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.229 DEBUG:boto:Params: {} 2022-09-03T05:25:52.230 DEBUG:boto:Token: None 2022-09-03T05:25:52.231 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.231 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Uzjkl0upMergKmBKzcI1kOdQ++k= 2022-09-03T05:25:52.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Uzjkl0upMergKmBKzcI1kOdQ++k=', 'Content-Length': '0'} 2022-09-03T05:25:52.237 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000000cfd4dc2e0039257-006312e560-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.284 DEBUG:boto:path=/dsgvcy-13/myobj 2022-09-03T05:25:52.285 DEBUG:boto:auth_path=/dsgvcy-13/myobj 2022-09-03T05:25:52.286 DEBUG:boto:path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.286 DEBUG:boto:auth_path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.287 DEBUG:boto:Method: GET 2022-09-03T05:25:52.288 DEBUG:boto:Path: /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.288 DEBUG:boto:Data: 2022-09-03T05:25:52.289 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.289 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:52.290 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.291 DEBUG:boto:Params: {} 2022-09-03T05:25:52.291 DEBUG:boto:Token: None 2022-09-03T05:25:52.292 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.293 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Uzjkl0upMergKmBKzcI1kOdQ++k= 2022-09-03T05:25:52.293 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Uzjkl0upMergKmBKzcI1kOdQ++k=', 'Content-Length': '0'} 2022-09-03T05:25:52.296 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000045d94c95e301f1f4-006312e560-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.297 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:25:52.298 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:25:52.298 DEBUG:boto:path=/dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.299 DEBUG:boto:auth_path=/dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.300 DEBUG:boto:Method: GET 2022-09-03T05:25:52.300 DEBUG:boto:Path: /dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.301 DEBUG:boto:Data: 2022-09-03T05:25:52.301 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.302 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:52.302 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.303 DEBUG:boto:Params: {} 2022-09-03T05:25:52.304 DEBUG:boto:Token: None 2022-09-03T05:25:52.305 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/ 2022-09-03T05:25:52.306 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lclcA9ixZoblQrWA3MouYRENh5c= 2022-09-03T05:25:52.307 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lclcA9ixZoblQrWA3MouYRENh5c=', 'Content-Length': '0'} 2022-09-03T05:25:52.310 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e574df4a220f26e3-006312e560-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.311 DEBUG:boto:b'dsgvcy-13myobj1000falsemyobj2022-09-03T05:25:46.085Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:52.312 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:25:52.312 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:25:52.313 DEBUG:boto:path=/dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.313 DEBUG:boto:auth_path=/dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.314 DEBUG:boto:Method: GET 2022-09-03T05:25:52.314 DEBUG:boto:Path: /dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.315 DEBUG:boto:Data: 2022-09-03T05:25:52.315 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.316 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:52.317 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.317 DEBUG:boto:Params: {} 2022-09-03T05:25:52.318 DEBUG:boto:Token: None 2022-09-03T05:25:52.318 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/ 2022-09-03T05:25:52.319 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lclcA9ixZoblQrWA3MouYRENh5c= 2022-09-03T05:25:52.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lclcA9ixZoblQrWA3MouYRENh5c=', 'Content-Length': '0'} 2022-09-03T05:25:52.322 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000097d1ad442e488a9-006312e560-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.323 DEBUG:boto:b'dsgvcy-13myobj1000falsemyobj2022-09-03T05:25:46.085Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:52.324 DEBUG:boto:path=/dsgvcy-13/myobj 2022-09-03T05:25:52.324 DEBUG:boto:auth_path=/dsgvcy-13/myobj 2022-09-03T05:25:52.325 DEBUG:boto:Method: HEAD 2022-09-03T05:25:52.325 DEBUG:boto:Path: /dsgvcy-13/myobj 2022-09-03T05:25:52.326 DEBUG:boto:Data: 2022-09-03T05:25:52.327 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.327 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:52.328 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.328 DEBUG:boto:Params: {} 2022-09-03T05:25:52.329 DEBUG:boto:Token: None 2022-09-03T05:25:52.329 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/myobj 2022-09-03T05:25:52.330 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eCjleVMWZrhCm0z50Oxw4V0HXWc= 2022-09-03T05:25:52.331 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eCjleVMWZrhCm0z50Oxw4V0HXWc=', 'Content-Length': '0'} 2022-09-03T05:25:52.333 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000dc4b20589f48572e-006312e560-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.333 DEBUG:boto:path=/dsgvcy-13/myobj 2022-09-03T05:25:52.334 DEBUG:boto:auth_path=/dsgvcy-13/myobj 2022-09-03T05:25:52.335 DEBUG:boto:Method: HEAD 2022-09-03T05:25:52.335 DEBUG:boto:Path: /dsgvcy-13/myobj 2022-09-03T05:25:52.336 DEBUG:boto:Data: 2022-09-03T05:25:52.336 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.337 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:52.337 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.338 DEBUG:boto:Params: {} 2022-09-03T05:25:52.338 DEBUG:boto:Token: None 2022-09-03T05:25:52.339 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/myobj 2022-09-03T05:25:52.340 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eCjleVMWZrhCm0z50Oxw4V0HXWc= 2022-09-03T05:25:52.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eCjleVMWZrhCm0z50Oxw4V0HXWc=', 'Content-Length': '0'} 2022-09-03T05:25:52.343 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000006f76d752088cb9d-006312e560-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.344 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:25:52.344 DEBUG:boto:path=/dsgvcy-13/myobj 2022-09-03T05:25:52.345 DEBUG:boto:auth_path=/dsgvcy-13/myobj 2022-09-03T05:25:52.345 DEBUG:boto:Method: GET 2022-09-03T05:25:52.346 DEBUG:boto:Path: /dsgvcy-13/myobj 2022-09-03T05:25:52.347 DEBUG:boto:Data: 2022-09-03T05:25:52.347 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.348 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:52.348 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.349 DEBUG:boto:Params: {} 2022-09-03T05:25:52.349 DEBUG:boto:Token: None 2022-09-03T05:25:52.350 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/myobj 2022-09-03T05:25:52.350 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yV69v24dqDw5Wf2FBLiA9X+bU/M= 2022-09-03T05:25:52.351 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yV69v24dqDw5Wf2FBLiA9X+bU/M=', 'Content-Length': '0'} 2022-09-03T05:25:52.353 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000053a4fb889a68eb59-006312e560-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.354 DEBUG:boto:path=/dsgvcy-13/myobj 2022-09-03T05:25:52.354 DEBUG:boto:auth_path=/dsgvcy-13/myobj 2022-09-03T05:25:52.355 DEBUG:boto:Method: GET 2022-09-03T05:25:52.355 DEBUG:boto:Path: /dsgvcy-13/myobj 2022-09-03T05:25:52.356 DEBUG:boto:Data: 2022-09-03T05:25:52.357 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.357 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:52.358 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.358 DEBUG:boto:Params: {} 2022-09-03T05:25:52.359 DEBUG:boto:Token: None 2022-09-03T05:25:52.359 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/myobj 2022-09-03T05:25:52.360 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yV69v24dqDw5Wf2FBLiA9X+bU/M= 2022-09-03T05:25:52.361 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yV69v24dqDw5Wf2FBLiA9X+bU/M=', 'Content-Length': '0'} 2022-09-03T05:25:52.363 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000832ef31e71fcf2c5-006312e560-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.406 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:25:52.407 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:25:52.408 DEBUG:boto:path=/dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.408 DEBUG:boto:auth_path=/dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.409 DEBUG:boto:Method: GET 2022-09-03T05:25:52.410 DEBUG:boto:Path: /dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.411 DEBUG:boto:Data: 2022-09-03T05:25:52.411 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.412 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:52.413 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.414 DEBUG:boto:Params: {} 2022-09-03T05:25:52.414 DEBUG:boto:Token: None 2022-09-03T05:25:52.416 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/ 2022-09-03T05:25:52.417 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lclcA9ixZoblQrWA3MouYRENh5c= 2022-09-03T05:25:52.417 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lclcA9ixZoblQrWA3MouYRENh5c=', 'Content-Length': '0'} 2022-09-03T05:25:52.420 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000018ab4d0dc9f2cb00-006312e560-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.421 DEBUG:boto:b'dsgvcy-13myobj1000falsemyobj2022-09-03T05:25:46.085Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:52.422 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:25:52.422 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:25:52.423 DEBUG:boto:path=/dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.424 DEBUG:boto:auth_path=/dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.425 DEBUG:boto:Method: GET 2022-09-03T05:25:52.425 DEBUG:boto:Path: /dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.426 DEBUG:boto:Data: 2022-09-03T05:25:52.427 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.427 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:52.428 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.429 DEBUG:boto:Params: {} 2022-09-03T05:25:52.429 DEBUG:boto:Token: None 2022-09-03T05:25:52.430 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/ 2022-09-03T05:25:52.430 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lclcA9ixZoblQrWA3MouYRENh5c= 2022-09-03T05:25:52.431 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lclcA9ixZoblQrWA3MouYRENh5c=', 'Content-Length': '0'} 2022-09-03T05:25:52.433 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b3ec9dd4172be106-006312e560-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.435 DEBUG:boto:b'dsgvcy-13myobj1000falsemyobj2022-09-03T05:25:46.085Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:52.436 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-13 zones={test-zone2, test-zone1} 2022-09-03T05:25:52.437 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-13', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:25:52.438 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-13', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:25:52.439 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-13 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:25:52.501 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:25:52.502 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:25:52.550 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:25:52.552+0000 7fd8db05f600 1 bucket sync caught up with source: 2022-09-03T05:25:52.550 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.6.5, , , , , , , , , ] 2022-09-03T05:25:52.551 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.6.5, , , , , , , , , ] 2022-09-03T05:25:52.551 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:25:52.552+0000 7fd8db05f600 0 bucket checkpoint complete 2022-09-03T05:25:52.563 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-13 zones={test-zone3, test-zone1} 2022-09-03T05:25:52.564 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:25:52.564 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:25:52.565 DEBUG:boto:Method: HEAD 2022-09-03T05:25:52.565 DEBUG:boto:Path: /dsgvcy-13/ 2022-09-03T05:25:52.567 DEBUG:boto:Data: 2022-09-03T05:25:52.568 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.569 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:25:52.570 DEBUG:boto:Port: 8001 2022-09-03T05:25:52.570 DEBUG:boto:Params: {} 2022-09-03T05:25:52.572 DEBUG:boto:Token: None 2022-09-03T05:25:52.573 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/ 2022-09-03T05:25:52.574 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xG7rgvkqhZVm1/iSrSxCGvvox0c= 2022-09-03T05:25:52.575 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xG7rgvkqhZVm1/iSrSxCGvvox0c=', 'Content-Length': '0'} 2022-09-03T05:25:52.580 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000067d412dd88f67f45-006312e560-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.581 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:25:52.583 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:25:52.584 DEBUG:boto:Method: HEAD 2022-09-03T05:25:52.584 DEBUG:boto:Path: /dsgvcy-13/ 2022-09-03T05:25:52.585 DEBUG:boto:Data: 2022-09-03T05:25:52.585 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.586 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:52.587 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.587 DEBUG:boto:Params: {} 2022-09-03T05:25:52.588 DEBUG:boto:Token: None 2022-09-03T05:25:52.589 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/ 2022-09-03T05:25:52.590 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xG7rgvkqhZVm1/iSrSxCGvvox0c= 2022-09-03T05:25:52.591 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xG7rgvkqhZVm1/iSrSxCGvvox0c=', 'Content-Length': '0'} 2022-09-03T05:25:52.593 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000597e15d1639de573-006312e560-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.594 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:25:52.594 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:25:52.595 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:25:52.595 DEBUG:boto:path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.596 DEBUG:boto:auth_path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.596 DEBUG:boto:Method: GET 2022-09-03T05:25:52.597 DEBUG:boto:Path: /dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.598 DEBUG:boto:Data: 2022-09-03T05:25:52.598 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.599 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:25:52.599 DEBUG:boto:Port: 8001 2022-09-03T05:25:52.600 DEBUG:boto:Params: {} 2022-09-03T05:25:52.600 DEBUG:boto:Token: None 2022-09-03T05:25:52.601 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/?versions 2022-09-03T05:25:52.601 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rAj5wp5T9vzs8TUB/l9P2sJDkPc= 2022-09-03T05:25:52.602 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rAj5wp5T9vzs8TUB/l9P2sJDkPc=', 'Content-Length': '0'} 2022-09-03T05:25:52.604 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fb9dc67c7830e9b7-006312e560-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.605 DEBUG:boto:b'dsgvcy-13999falsemyobjnulltrue2022-09-03T05:25:46.085Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:52.606 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2022-09-03T05:25:52.607 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:25:52.608 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:25:52.608 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:25:52.609 DEBUG:boto:path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.610 DEBUG:boto:auth_path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.610 DEBUG:boto:Method: GET 2022-09-03T05:25:52.611 DEBUG:boto:Path: /dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.611 DEBUG:boto:Data: 2022-09-03T05:25:52.612 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.612 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:52.613 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.613 DEBUG:boto:Params: {} 2022-09-03T05:25:52.614 DEBUG:boto:Token: None 2022-09-03T05:25:52.614 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/?versions 2022-09-03T05:25:52.615 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rAj5wp5T9vzs8TUB/l9P2sJDkPc= 2022-09-03T05:25:52.615 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rAj5wp5T9vzs8TUB/l9P2sJDkPc=', 'Content-Length': '0'} 2022-09-03T05:25:52.618 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ecb17d8f3e923d41-006312e560-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.619 DEBUG:boto:b'dsgvcy-13999falsemyobjnulltrue2022-09-03T05:25:46.085Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:52.620 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2022-09-03T05:25:52.621 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:25:52.621 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:25:52.622 DEBUG:boto:path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.622 DEBUG:boto:auth_path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.623 DEBUG:boto:Method: GET 2022-09-03T05:25:52.623 DEBUG:boto:Path: /dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.624 DEBUG:boto:Data: 2022-09-03T05:25:52.624 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.625 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:25:52.625 DEBUG:boto:Port: 8001 2022-09-03T05:25:52.626 DEBUG:boto:Params: {} 2022-09-03T05:25:52.626 DEBUG:boto:Token: None 2022-09-03T05:25:52.627 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/?versions 2022-09-03T05:25:52.628 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rAj5wp5T9vzs8TUB/l9P2sJDkPc= 2022-09-03T05:25:52.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rAj5wp5T9vzs8TUB/l9P2sJDkPc=', 'Content-Length': '0'} 2022-09-03T05:25:52.631 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000efcaf10eaa07e1ca-006312e560-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.632 DEBUG:boto:b'dsgvcy-13999falsemyobjnulltrue2022-09-03T05:25:46.085Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:52.633 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:25:52.633 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:25:52.634 DEBUG:boto:path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.635 DEBUG:boto:auth_path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.635 DEBUG:boto:Method: GET 2022-09-03T05:25:52.636 DEBUG:boto:Path: /dsgvcy-13/?versions&max-keys=999 2022-09-03T05:25:52.636 DEBUG:boto:Data: 2022-09-03T05:25:52.637 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.638 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:52.638 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.639 DEBUG:boto:Params: {} 2022-09-03T05:25:52.639 DEBUG:boto:Token: None 2022-09-03T05:25:52.640 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/?versions 2022-09-03T05:25:52.640 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rAj5wp5T9vzs8TUB/l9P2sJDkPc= 2022-09-03T05:25:52.641 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rAj5wp5T9vzs8TUB/l9P2sJDkPc=', 'Content-Length': '0'} 2022-09-03T05:25:52.643 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d85b6fa0d49115f4-006312e560-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.644 DEBUG:boto:b'dsgvcy-13999falsemyobjnulltrue2022-09-03T05:25:46.085Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:52.645 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:25:52.646 DEBUG:boto:path=/dsgvcy-13/myobj 2022-09-03T05:25:52.646 DEBUG:boto:auth_path=/dsgvcy-13/myobj 2022-09-03T05:25:52.647 DEBUG:boto:path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.648 DEBUG:boto:auth_path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.648 DEBUG:boto:Method: GET 2022-09-03T05:25:52.649 DEBUG:boto:Path: /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.649 DEBUG:boto:Data: 2022-09-03T05:25:52.650 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.651 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:25:52.651 DEBUG:boto:Port: 8001 2022-09-03T05:25:52.652 DEBUG:boto:Params: {} 2022-09-03T05:25:52.652 DEBUG:boto:Token: None 2022-09-03T05:25:52.653 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.653 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Uzjkl0upMergKmBKzcI1kOdQ++k= 2022-09-03T05:25:52.654 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Uzjkl0upMergKmBKzcI1kOdQ++k=', 'Content-Length': '0'} 2022-09-03T05:25:52.656 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000236ad4e6e7d425c0-006312e560-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.658 DEBUG:boto:path=/dsgvcy-13/myobj 2022-09-03T05:25:52.659 DEBUG:boto:auth_path=/dsgvcy-13/myobj 2022-09-03T05:25:52.660 DEBUG:boto:path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.661 DEBUG:boto:auth_path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.662 DEBUG:boto:Method: GET 2022-09-03T05:25:52.663 DEBUG:boto:Path: /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.663 DEBUG:boto:Data: 2022-09-03T05:25:52.664 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.664 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:52.665 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.665 DEBUG:boto:Params: {} 2022-09-03T05:25:52.666 DEBUG:boto:Token: None 2022-09-03T05:25:52.667 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.667 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Uzjkl0upMergKmBKzcI1kOdQ++k= 2022-09-03T05:25:52.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Uzjkl0upMergKmBKzcI1kOdQ++k=', 'Content-Length': '0'} 2022-09-03T05:25:52.671 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000001e1785293bf11b03-006312e560-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.672 DEBUG:boto:path=/dsgvcy-13/myobj 2022-09-03T05:25:52.672 DEBUG:boto:auth_path=/dsgvcy-13/myobj 2022-09-03T05:25:52.673 DEBUG:boto:path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.673 DEBUG:boto:auth_path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.674 DEBUG:boto:Method: HEAD 2022-09-03T05:25:52.675 DEBUG:boto:Path: /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.675 DEBUG:boto:Data: 2022-09-03T05:25:52.676 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.677 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:25:52.677 DEBUG:boto:Port: 8001 2022-09-03T05:25:52.678 DEBUG:boto:Params: {} 2022-09-03T05:25:52.678 DEBUG:boto:Token: None 2022-09-03T05:25:52.679 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.680 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WKdLrO6dbWXvpC/y1O93K3anAlY= 2022-09-03T05:25:52.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WKdLrO6dbWXvpC/y1O93K3anAlY=', 'Content-Length': '0'} 2022-09-03T05:25:52.682 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005f06e2db7e154e04-006312e560-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.683 DEBUG:boto:path=/dsgvcy-13/myobj 2022-09-03T05:25:52.683 DEBUG:boto:auth_path=/dsgvcy-13/myobj 2022-09-03T05:25:52.684 DEBUG:boto:path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.684 DEBUG:boto:auth_path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.685 DEBUG:boto:Method: HEAD 2022-09-03T05:25:52.686 DEBUG:boto:Path: /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.687 DEBUG:boto:Data: 2022-09-03T05:25:52.687 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.689 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:52.689 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.690 DEBUG:boto:Params: {} 2022-09-03T05:25:52.691 DEBUG:boto:Token: None 2022-09-03T05:25:52.691 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.692 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WKdLrO6dbWXvpC/y1O93K3anAlY= 2022-09-03T05:25:52.693 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WKdLrO6dbWXvpC/y1O93K3anAlY=', 'Content-Length': '0'} 2022-09-03T05:25:52.695 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000be198e4507333f20-006312e560-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.696 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:25:52.697 DEBUG:boto:path=/dsgvcy-13/myobj 2022-09-03T05:25:52.697 DEBUG:boto:auth_path=/dsgvcy-13/myobj 2022-09-03T05:25:52.698 DEBUG:boto:path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.698 DEBUG:boto:auth_path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.699 DEBUG:boto:Method: GET 2022-09-03T05:25:52.700 DEBUG:boto:Path: /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.701 DEBUG:boto:Data: 2022-09-03T05:25:52.701 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.702 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:25:52.702 DEBUG:boto:Port: 8001 2022-09-03T05:25:52.703 DEBUG:boto:Params: {} 2022-09-03T05:25:52.704 DEBUG:boto:Token: None 2022-09-03T05:25:52.704 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.705 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Uzjkl0upMergKmBKzcI1kOdQ++k= 2022-09-03T05:25:52.706 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Uzjkl0upMergKmBKzcI1kOdQ++k=', 'Content-Length': '0'} 2022-09-03T05:25:52.708 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000038b2e7317005e643-006312e560-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.709 DEBUG:boto:path=/dsgvcy-13/myobj 2022-09-03T05:25:52.710 DEBUG:boto:auth_path=/dsgvcy-13/myobj 2022-09-03T05:25:52.711 DEBUG:boto:path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.712 DEBUG:boto:auth_path=/dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.712 DEBUG:boto:Method: GET 2022-09-03T05:25:52.713 DEBUG:boto:Path: /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.713 DEBUG:boto:Data: 2022-09-03T05:25:52.714 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.715 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:52.715 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.716 DEBUG:boto:Params: {} 2022-09-03T05:25:52.717 DEBUG:boto:Token: None 2022-09-03T05:25:52.717 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/myobj?versionId=null 2022-09-03T05:25:52.718 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Uzjkl0upMergKmBKzcI1kOdQ++k= 2022-09-03T05:25:52.719 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Uzjkl0upMergKmBKzcI1kOdQ++k=', 'Content-Length': '0'} 2022-09-03T05:25:52.720 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000165d633c40c6fd94-006312e560-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.762 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:25:52.763 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:25:52.764 DEBUG:boto:path=/dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.764 DEBUG:boto:auth_path=/dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.765 DEBUG:boto:Method: GET 2022-09-03T05:25:52.766 DEBUG:boto:Path: /dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.767 DEBUG:boto:Data: 2022-09-03T05:25:52.768 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.770 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:25:52.771 DEBUG:boto:Port: 8001 2022-09-03T05:25:52.772 DEBUG:boto:Params: {} 2022-09-03T05:25:52.773 DEBUG:boto:Token: None 2022-09-03T05:25:52.774 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/ 2022-09-03T05:25:52.774 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lclcA9ixZoblQrWA3MouYRENh5c= 2022-09-03T05:25:52.775 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lclcA9ixZoblQrWA3MouYRENh5c=', 'Content-Length': '0'} 2022-09-03T05:25:52.778 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b43feff4d24881e6-006312e560-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.779 DEBUG:boto:b'dsgvcy-13myobj1000falsemyobj2022-09-03T05:25:46.085Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:52.780 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:25:52.780 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:25:52.781 DEBUG:boto:path=/dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.782 DEBUG:boto:auth_path=/dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.782 DEBUG:boto:Method: GET 2022-09-03T05:25:52.783 DEBUG:boto:Path: /dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.784 DEBUG:boto:Data: 2022-09-03T05:25:52.785 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.785 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:52.786 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.787 DEBUG:boto:Params: {} 2022-09-03T05:25:52.788 DEBUG:boto:Token: None 2022-09-03T05:25:52.788 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/ 2022-09-03T05:25:52.789 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lclcA9ixZoblQrWA3MouYRENh5c= 2022-09-03T05:25:52.790 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lclcA9ixZoblQrWA3MouYRENh5c=', 'Content-Length': '0'} 2022-09-03T05:25:52.792 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a50216ebc457d904-006312e560-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.793 DEBUG:boto:b'dsgvcy-13myobj1000falsemyobj2022-09-03T05:25:46.085Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:52.794 DEBUG:boto:path=/dsgvcy-13/myobj 2022-09-03T05:25:52.795 DEBUG:boto:auth_path=/dsgvcy-13/myobj 2022-09-03T05:25:52.795 DEBUG:boto:Method: HEAD 2022-09-03T05:25:52.796 DEBUG:boto:Path: /dsgvcy-13/myobj 2022-09-03T05:25:52.796 DEBUG:boto:Data: 2022-09-03T05:25:52.797 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.797 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:25:52.798 DEBUG:boto:Port: 8001 2022-09-03T05:25:52.799 DEBUG:boto:Params: {} 2022-09-03T05:25:52.799 DEBUG:boto:Token: None 2022-09-03T05:25:52.800 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/myobj 2022-09-03T05:25:52.801 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eCjleVMWZrhCm0z50Oxw4V0HXWc= 2022-09-03T05:25:52.801 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eCjleVMWZrhCm0z50Oxw4V0HXWc=', 'Content-Length': '0'} 2022-09-03T05:25:52.804 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c0cb26ef2fa8e454-006312e560-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.804 DEBUG:boto:path=/dsgvcy-13/myobj 2022-09-03T05:25:52.805 DEBUG:boto:auth_path=/dsgvcy-13/myobj 2022-09-03T05:25:52.805 DEBUG:boto:Method: HEAD 2022-09-03T05:25:52.806 DEBUG:boto:Path: /dsgvcy-13/myobj 2022-09-03T05:25:52.806 DEBUG:boto:Data: 2022-09-03T05:25:52.807 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.808 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:52.808 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.809 DEBUG:boto:Params: {} 2022-09-03T05:25:52.809 DEBUG:boto:Token: None 2022-09-03T05:25:52.810 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/myobj 2022-09-03T05:25:52.811 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eCjleVMWZrhCm0z50Oxw4V0HXWc= 2022-09-03T05:25:52.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eCjleVMWZrhCm0z50Oxw4V0HXWc=', 'Content-Length': '0'} 2022-09-03T05:25:52.814 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ee99d3be2fd535cc-006312e560-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.814 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:25:52.815 DEBUG:boto:path=/dsgvcy-13/myobj 2022-09-03T05:25:52.815 DEBUG:boto:auth_path=/dsgvcy-13/myobj 2022-09-03T05:25:52.816 DEBUG:boto:Method: GET 2022-09-03T05:25:52.817 DEBUG:boto:Path: /dsgvcy-13/myobj 2022-09-03T05:25:52.817 DEBUG:boto:Data: 2022-09-03T05:25:52.818 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.819 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:25:52.820 DEBUG:boto:Port: 8001 2022-09-03T05:25:52.821 DEBUG:boto:Params: {} 2022-09-03T05:25:52.822 DEBUG:boto:Token: None 2022-09-03T05:25:52.823 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/myobj 2022-09-03T05:25:52.824 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yV69v24dqDw5Wf2FBLiA9X+bU/M= 2022-09-03T05:25:52.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yV69v24dqDw5Wf2FBLiA9X+bU/M=', 'Content-Length': '0'} 2022-09-03T05:25:52.828 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ec058c85108671ad-006312e560-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.829 DEBUG:boto:path=/dsgvcy-13/myobj 2022-09-03T05:25:52.830 DEBUG:boto:auth_path=/dsgvcy-13/myobj 2022-09-03T05:25:52.830 DEBUG:boto:Method: GET 2022-09-03T05:25:52.831 DEBUG:boto:Path: /dsgvcy-13/myobj 2022-09-03T05:25:52.831 DEBUG:boto:Data: 2022-09-03T05:25:52.832 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.832 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:52.833 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.833 DEBUG:boto:Params: {} 2022-09-03T05:25:52.834 DEBUG:boto:Token: None 2022-09-03T05:25:52.835 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/myobj 2022-09-03T05:25:52.835 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yV69v24dqDw5Wf2FBLiA9X+bU/M= 2022-09-03T05:25:52.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yV69v24dqDw5Wf2FBLiA9X+bU/M=', 'Content-Length': '0'} 2022-09-03T05:25:52.838 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000026430f51e039cc99-006312e560-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.839 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:25:52.839 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:25:52.840 DEBUG:boto:path=/dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.840 DEBUG:boto:auth_path=/dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.841 DEBUG:boto:Method: GET 2022-09-03T05:25:52.841 DEBUG:boto:Path: /dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.842 DEBUG:boto:Data: 2022-09-03T05:25:52.843 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.843 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:25:52.844 DEBUG:boto:Port: 8001 2022-09-03T05:25:52.844 DEBUG:boto:Params: {} 2022-09-03T05:25:52.845 DEBUG:boto:Token: None 2022-09-03T05:25:52.845 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/ 2022-09-03T05:25:52.846 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lclcA9ixZoblQrWA3MouYRENh5c= 2022-09-03T05:25:52.847 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lclcA9ixZoblQrWA3MouYRENh5c=', 'Content-Length': '0'} 2022-09-03T05:25:52.850 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000061a350bc4b8736bc-006312e560-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.851 DEBUG:boto:b'dsgvcy-13myobj1000falsemyobj2022-09-03T05:25:46.085Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:52.851 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:25:52.852 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:25:52.853 DEBUG:boto:path=/dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.853 DEBUG:boto:auth_path=/dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.854 DEBUG:boto:Method: GET 2022-09-03T05:25:52.854 DEBUG:boto:Path: /dsgvcy-13/?prefix=myobj 2022-09-03T05:25:52.855 DEBUG:boto:Data: 2022-09-03T05:25:52.856 DEBUG:boto:Headers: {} 2022-09-03T05:25:52.856 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:52.857 DEBUG:boto:Port: 8000 2022-09-03T05:25:52.857 DEBUG:boto:Params: {} 2022-09-03T05:25:52.858 DEBUG:boto:Token: None 2022-09-03T05:25:52.858 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:52 GMT /dsgvcy-13/ 2022-09-03T05:25:52.859 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lclcA9ixZoblQrWA3MouYRENh5c= 2022-09-03T05:25:52.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lclcA9ixZoblQrWA3MouYRENh5c=', 'Content-Length': '0'} 2022-09-03T05:25:52.862 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a40b1ede1c053ba0-006312e560-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:52.863 DEBUG:boto:b'dsgvcy-13myobj1000falsemyobj2022-09-03T05:25:46.085Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:52.864 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-13 zones={test-zone3, test-zone1} 2022-09-03T05:25:52.864 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-13', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:25:52.865 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-13', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:25:52.866 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-13 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:25:52.926 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:25:52.926 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:25:52.958 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:25:52.961+0000 7f55352b9600 1 waiting to reach incremental sync.. 2022-09-03T05:25:57.961 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:25:57.963+0000 7f55352b9600 1 bucket sync caught up with source: 2022-09-03T05:25:57.961 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.6.5, , , , , , , , , ] 2022-09-03T05:25:57.962 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.6.5, , , , , , , , , ] 2022-09-03T05:25:57.962 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:25:57.963+0000 7f55352b9600 0 bucket checkpoint complete 2022-09-03T05:25:57.977 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-14', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:25:57.978 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-14', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:25:57.979 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-14 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:25:58.044 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:25:58.045 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:25:58.095 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:25:58.097+0000 7ff66bddf600 1 bucket sync caught up with source: 2022-09-03T05:25:58.095 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.6.5, , , , , , , , , ] 2022-09-03T05:25:58.096 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.6.5, , , , , , , , , ] 2022-09-03T05:25:58.096 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:25:58.097+0000 7ff66bddf600 0 bucket checkpoint complete 2022-09-03T05:25:58.115 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-14 zones={test-zone1, test-zone2} 2022-09-03T05:25:58.116 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:25:58.116 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:25:58.118 DEBUG:boto:Method: HEAD 2022-09-03T05:25:58.118 DEBUG:boto:Path: /dsgvcy-14/ 2022-09-03T05:25:58.119 DEBUG:boto:Data: 2022-09-03T05:25:58.119 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.120 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:58.120 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.121 DEBUG:boto:Params: {} 2022-09-03T05:25:58.122 DEBUG:boto:Token: None 2022-09-03T05:25:58.123 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/ 2022-09-03T05:25:58.124 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XJPAcYXrIohx1Qum1QdpuHjMmmg= 2022-09-03T05:25:58.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XJPAcYXrIohx1Qum1QdpuHjMmmg=', 'Content-Length': '0'} 2022-09-03T05:25:58.127 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000007978173b909cf52-006312e566-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.127 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:25:58.128 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:25:58.129 DEBUG:boto:Method: HEAD 2022-09-03T05:25:58.129 DEBUG:boto:Path: /dsgvcy-14/ 2022-09-03T05:25:58.130 DEBUG:boto:Data: 2022-09-03T05:25:58.131 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.132 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:58.133 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.133 DEBUG:boto:Params: {} 2022-09-03T05:25:58.134 DEBUG:boto:Token: None 2022-09-03T05:25:58.135 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/ 2022-09-03T05:25:58.136 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XJPAcYXrIohx1Qum1QdpuHjMmmg= 2022-09-03T05:25:58.136 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XJPAcYXrIohx1Qum1QdpuHjMmmg=', 'Content-Length': '0'} 2022-09-03T05:25:58.141 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a3376c9c0a9bedb3-006312e566-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.143 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:25:58.144 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:25:58.145 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:25:58.146 DEBUG:boto:path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.147 DEBUG:boto:auth_path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.147 DEBUG:boto:Method: GET 2022-09-03T05:25:58.148 DEBUG:boto:Path: /dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.149 DEBUG:boto:Data: 2022-09-03T05:25:58.150 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.151 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:58.152 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.153 DEBUG:boto:Params: {} 2022-09-03T05:25:58.154 DEBUG:boto:Token: None 2022-09-03T05:25:58.155 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/?versions 2022-09-03T05:25:58.156 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0RAFPEDbuRljx7rWL2BF/IKh7KI= 2022-09-03T05:25:58.157 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0RAFPEDbuRljx7rWL2BF/IKh7KI=', 'Content-Length': '0'} 2022-09-03T05:25:58.161 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b5bf3fbb66e82650-006312e566-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.202 DEBUG:boto:b'dsgvcy-14999falsemyobjnulltrue2022-09-03T05:25:46.137Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:58.203 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2022-09-03T05:25:58.204 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:25:58.204 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:25:58.205 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:25:58.205 DEBUG:boto:path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.206 DEBUG:boto:auth_path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.207 DEBUG:boto:Method: GET 2022-09-03T05:25:58.207 DEBUG:boto:Path: /dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.208 DEBUG:boto:Data: 2022-09-03T05:25:58.208 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.209 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:58.209 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.210 DEBUG:boto:Params: {} 2022-09-03T05:25:58.211 DEBUG:boto:Token: None 2022-09-03T05:25:58.212 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/?versions 2022-09-03T05:25:58.212 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0RAFPEDbuRljx7rWL2BF/IKh7KI= 2022-09-03T05:25:58.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0RAFPEDbuRljx7rWL2BF/IKh7KI=', 'Content-Length': '0'} 2022-09-03T05:25:58.216 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000065a66c750b9ce954-006312e566-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.216 DEBUG:boto:b'dsgvcy-14999falsemyobjnulltrue2022-09-03T05:25:46.137Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:58.217 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2022-09-03T05:25:58.218 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:25:58.218 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:25:58.219 DEBUG:boto:path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.219 DEBUG:boto:auth_path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.220 DEBUG:boto:Method: GET 2022-09-03T05:25:58.220 DEBUG:boto:Path: /dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.221 DEBUG:boto:Data: 2022-09-03T05:25:58.221 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.222 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:58.222 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.223 DEBUG:boto:Params: {} 2022-09-03T05:25:58.224 DEBUG:boto:Token: None 2022-09-03T05:25:58.225 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/?versions 2022-09-03T05:25:58.225 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0RAFPEDbuRljx7rWL2BF/IKh7KI= 2022-09-03T05:25:58.226 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0RAFPEDbuRljx7rWL2BF/IKh7KI=', 'Content-Length': '0'} 2022-09-03T05:25:58.228 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000094be33d65384239e-006312e566-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.229 DEBUG:boto:b'dsgvcy-14999falsemyobjnulltrue2022-09-03T05:25:46.137Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:58.230 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:25:58.231 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:25:58.231 DEBUG:boto:path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.232 DEBUG:boto:auth_path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.232 DEBUG:boto:Method: GET 2022-09-03T05:25:58.233 DEBUG:boto:Path: /dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.233 DEBUG:boto:Data: 2022-09-03T05:25:58.234 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.234 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:58.235 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.235 DEBUG:boto:Params: {} 2022-09-03T05:25:58.236 DEBUG:boto:Token: None 2022-09-03T05:25:58.236 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/?versions 2022-09-03T05:25:58.237 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0RAFPEDbuRljx7rWL2BF/IKh7KI= 2022-09-03T05:25:58.238 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0RAFPEDbuRljx7rWL2BF/IKh7KI=', 'Content-Length': '0'} 2022-09-03T05:25:58.240 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006457274a1a4de88e-006312e566-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.241 DEBUG:boto:b'dsgvcy-14999falsemyobjnulltrue2022-09-03T05:25:46.137Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:58.242 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:25:58.243 DEBUG:boto:path=/dsgvcy-14/myobj 2022-09-03T05:25:58.243 DEBUG:boto:auth_path=/dsgvcy-14/myobj 2022-09-03T05:25:58.244 DEBUG:boto:path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.244 DEBUG:boto:auth_path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.245 DEBUG:boto:Method: GET 2022-09-03T05:25:58.245 DEBUG:boto:Path: /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.246 DEBUG:boto:Data: 2022-09-03T05:25:58.247 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.247 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:58.248 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.248 DEBUG:boto:Params: {} 2022-09-03T05:25:58.249 DEBUG:boto:Token: None 2022-09-03T05:25:58.250 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.251 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vTZZ1osZ5MUcfEur6EfStmaywaA= 2022-09-03T05:25:58.251 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vTZZ1osZ5MUcfEur6EfStmaywaA=', 'Content-Length': '0'} 2022-09-03T05:25:58.253 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000008e17b055b2171681-006312e566-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.254 DEBUG:boto:path=/dsgvcy-14/myobj 2022-09-03T05:25:58.255 DEBUG:boto:auth_path=/dsgvcy-14/myobj 2022-09-03T05:25:58.255 DEBUG:boto:path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.256 DEBUG:boto:auth_path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.256 DEBUG:boto:Method: GET 2022-09-03T05:25:58.257 DEBUG:boto:Path: /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.257 DEBUG:boto:Data: 2022-09-03T05:25:58.258 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.259 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:58.259 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.260 DEBUG:boto:Params: {} 2022-09-03T05:25:58.261 DEBUG:boto:Token: None 2022-09-03T05:25:58.261 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.262 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vTZZ1osZ5MUcfEur6EfStmaywaA= 2022-09-03T05:25:58.263 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vTZZ1osZ5MUcfEur6EfStmaywaA=', 'Content-Length': '0'} 2022-09-03T05:25:58.265 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004fd0ca3b69d8402a-006312e566-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.266 DEBUG:boto:path=/dsgvcy-14/myobj 2022-09-03T05:25:58.267 DEBUG:boto:auth_path=/dsgvcy-14/myobj 2022-09-03T05:25:58.267 DEBUG:boto:path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.268 DEBUG:boto:auth_path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.268 DEBUG:boto:Method: HEAD 2022-09-03T05:25:58.269 DEBUG:boto:Path: /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.269 DEBUG:boto:Data: 2022-09-03T05:25:58.270 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.270 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:58.271 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.272 DEBUG:boto:Params: {} 2022-09-03T05:25:58.272 DEBUG:boto:Token: None 2022-09-03T05:25:58.273 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.274 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rrjNjxC5sJoe7NgTs4hqHp+lj6I= 2022-09-03T05:25:58.274 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rrjNjxC5sJoe7NgTs4hqHp+lj6I=', 'Content-Length': '0'} 2022-09-03T05:25:58.276 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c26be330687493c9-006312e566-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.277 DEBUG:boto:path=/dsgvcy-14/myobj 2022-09-03T05:25:58.278 DEBUG:boto:auth_path=/dsgvcy-14/myobj 2022-09-03T05:25:58.278 DEBUG:boto:path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.279 DEBUG:boto:auth_path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.279 DEBUG:boto:Method: HEAD 2022-09-03T05:25:58.280 DEBUG:boto:Path: /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.281 DEBUG:boto:Data: 2022-09-03T05:25:58.281 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.282 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:58.283 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.283 DEBUG:boto:Params: {} 2022-09-03T05:25:58.284 DEBUG:boto:Token: None 2022-09-03T05:25:58.284 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.285 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rrjNjxC5sJoe7NgTs4hqHp+lj6I= 2022-09-03T05:25:58.286 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rrjNjxC5sJoe7NgTs4hqHp+lj6I=', 'Content-Length': '0'} 2022-09-03T05:25:58.288 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000133790b9fd1f5920-006312e566-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.290 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:25:58.291 DEBUG:boto:path=/dsgvcy-14/myobj 2022-09-03T05:25:58.292 DEBUG:boto:auth_path=/dsgvcy-14/myobj 2022-09-03T05:25:58.293 DEBUG:boto:path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.294 DEBUG:boto:auth_path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.295 DEBUG:boto:Method: GET 2022-09-03T05:25:58.296 DEBUG:boto:Path: /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.297 DEBUG:boto:Data: 2022-09-03T05:25:58.297 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.298 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:58.299 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.299 DEBUG:boto:Params: {} 2022-09-03T05:25:58.300 DEBUG:boto:Token: None 2022-09-03T05:25:58.301 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.301 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vTZZ1osZ5MUcfEur6EfStmaywaA= 2022-09-03T05:25:58.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vTZZ1osZ5MUcfEur6EfStmaywaA=', 'Content-Length': '0'} 2022-09-03T05:25:58.304 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000497f70eaf5a036ea-006312e566-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.305 DEBUG:boto:path=/dsgvcy-14/myobj 2022-09-03T05:25:58.306 DEBUG:boto:auth_path=/dsgvcy-14/myobj 2022-09-03T05:25:58.306 DEBUG:boto:path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.307 DEBUG:boto:auth_path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.307 DEBUG:boto:Method: GET 2022-09-03T05:25:58.308 DEBUG:boto:Path: /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.309 DEBUG:boto:Data: 2022-09-03T05:25:58.309 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.310 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:58.310 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.311 DEBUG:boto:Params: {} 2022-09-03T05:25:58.312 DEBUG:boto:Token: None 2022-09-03T05:25:58.312 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.313 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vTZZ1osZ5MUcfEur6EfStmaywaA= 2022-09-03T05:25:58.314 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vTZZ1osZ5MUcfEur6EfStmaywaA=', 'Content-Length': '0'} 2022-09-03T05:25:58.317 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e059d774d33c9d6f-006312e566-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.319 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:25:58.320 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:25:58.321 DEBUG:boto:path=/dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.322 DEBUG:boto:auth_path=/dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.323 DEBUG:boto:Method: GET 2022-09-03T05:25:58.324 DEBUG:boto:Path: /dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.324 DEBUG:boto:Data: 2022-09-03T05:25:58.325 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.326 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:58.328 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.329 DEBUG:boto:Params: {} 2022-09-03T05:25:58.331 DEBUG:boto:Token: None 2022-09-03T05:25:58.332 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/ 2022-09-03T05:25:58.332 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkLcY0gPOIyCdu8LQhjVPhOmVus= 2022-09-03T05:25:58.333 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkLcY0gPOIyCdu8LQhjVPhOmVus=', 'Content-Length': '0'} 2022-09-03T05:25:58.336 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001493834eeed5b897-006312e566-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.337 DEBUG:boto:b'dsgvcy-14myobj1000falsemyobj2022-09-03T05:25:46.137Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:58.338 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:25:58.339 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:25:58.340 DEBUG:boto:path=/dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.340 DEBUG:boto:auth_path=/dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.341 DEBUG:boto:Method: GET 2022-09-03T05:25:58.341 DEBUG:boto:Path: /dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.342 DEBUG:boto:Data: 2022-09-03T05:25:58.342 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.343 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:58.343 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.344 DEBUG:boto:Params: {} 2022-09-03T05:25:58.345 DEBUG:boto:Token: None 2022-09-03T05:25:58.345 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/ 2022-09-03T05:25:58.346 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkLcY0gPOIyCdu8LQhjVPhOmVus= 2022-09-03T05:25:58.346 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkLcY0gPOIyCdu8LQhjVPhOmVus=', 'Content-Length': '0'} 2022-09-03T05:25:58.349 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000061cdb8ca8b08b699-006312e566-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.350 DEBUG:boto:b'dsgvcy-14myobj1000falsemyobj2022-09-03T05:25:46.137Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:58.351 DEBUG:boto:path=/dsgvcy-14/myobj 2022-09-03T05:25:58.351 DEBUG:boto:auth_path=/dsgvcy-14/myobj 2022-09-03T05:25:58.352 DEBUG:boto:Method: HEAD 2022-09-03T05:25:58.352 DEBUG:boto:Path: /dsgvcy-14/myobj 2022-09-03T05:25:58.353 DEBUG:boto:Data: 2022-09-03T05:25:58.353 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.354 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:58.354 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.355 DEBUG:boto:Params: {} 2022-09-03T05:25:58.355 DEBUG:boto:Token: None 2022-09-03T05:25:58.356 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/myobj 2022-09-03T05:25:58.356 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YmwtxgihhbS2tNSfiMFH5EjE9gQ= 2022-09-03T05:25:58.357 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YmwtxgihhbS2tNSfiMFH5EjE9gQ=', 'Content-Length': '0'} 2022-09-03T05:25:58.359 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000050162487fb19800e-006312e566-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.360 DEBUG:boto:path=/dsgvcy-14/myobj 2022-09-03T05:25:58.360 DEBUG:boto:auth_path=/dsgvcy-14/myobj 2022-09-03T05:25:58.361 DEBUG:boto:Method: HEAD 2022-09-03T05:25:58.361 DEBUG:boto:Path: /dsgvcy-14/myobj 2022-09-03T05:25:58.362 DEBUG:boto:Data: 2022-09-03T05:25:58.362 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.363 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:58.363 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.364 DEBUG:boto:Params: {} 2022-09-03T05:25:58.364 DEBUG:boto:Token: None 2022-09-03T05:25:58.365 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/myobj 2022-09-03T05:25:58.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YmwtxgihhbS2tNSfiMFH5EjE9gQ= 2022-09-03T05:25:58.366 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YmwtxgihhbS2tNSfiMFH5EjE9gQ=', 'Content-Length': '0'} 2022-09-03T05:25:58.368 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000cb3c94483f6715c4-006312e566-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.369 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:25:58.370 DEBUG:boto:path=/dsgvcy-14/myobj 2022-09-03T05:25:58.370 DEBUG:boto:auth_path=/dsgvcy-14/myobj 2022-09-03T05:25:58.371 DEBUG:boto:Method: GET 2022-09-03T05:25:58.372 DEBUG:boto:Path: /dsgvcy-14/myobj 2022-09-03T05:25:58.372 DEBUG:boto:Data: 2022-09-03T05:25:58.373 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.373 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:58.374 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.375 DEBUG:boto:Params: {} 2022-09-03T05:25:58.376 DEBUG:boto:Token: None 2022-09-03T05:25:58.377 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/myobj 2022-09-03T05:25:58.378 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Pjk5KiRi09fzYU6JvYXFHXU/pPY= 2022-09-03T05:25:58.379 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Pjk5KiRi09fzYU6JvYXFHXU/pPY=', 'Content-Length': '0'} 2022-09-03T05:25:58.381 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000bebda3ba2d9c9440-006312e566-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.382 DEBUG:boto:path=/dsgvcy-14/myobj 2022-09-03T05:25:58.383 DEBUG:boto:auth_path=/dsgvcy-14/myobj 2022-09-03T05:25:58.384 DEBUG:boto:Method: GET 2022-09-03T05:25:58.384 DEBUG:boto:Path: /dsgvcy-14/myobj 2022-09-03T05:25:58.385 DEBUG:boto:Data: 2022-09-03T05:25:58.385 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.386 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:58.386 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.387 DEBUG:boto:Params: {} 2022-09-03T05:25:58.388 DEBUG:boto:Token: None 2022-09-03T05:25:58.388 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/myobj 2022-09-03T05:25:58.389 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Pjk5KiRi09fzYU6JvYXFHXU/pPY= 2022-09-03T05:25:58.389 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Pjk5KiRi09fzYU6JvYXFHXU/pPY=', 'Content-Length': '0'} 2022-09-03T05:25:58.392 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000327b22c09a614438-006312e566-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.393 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:25:58.393 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:25:58.394 DEBUG:boto:path=/dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.394 DEBUG:boto:auth_path=/dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.395 DEBUG:boto:Method: GET 2022-09-03T05:25:58.395 DEBUG:boto:Path: /dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.396 DEBUG:boto:Data: 2022-09-03T05:25:58.396 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.397 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:25:58.397 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.398 DEBUG:boto:Params: {} 2022-09-03T05:25:58.399 DEBUG:boto:Token: None 2022-09-03T05:25:58.399 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/ 2022-09-03T05:25:58.400 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkLcY0gPOIyCdu8LQhjVPhOmVus= 2022-09-03T05:25:58.401 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkLcY0gPOIyCdu8LQhjVPhOmVus=', 'Content-Length': '0'} 2022-09-03T05:25:58.403 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009d61f5519edc4cae-006312e566-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.404 DEBUG:boto:b'dsgvcy-14myobj1000falsemyobj2022-09-03T05:25:46.137Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:58.405 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:25:58.405 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:25:58.406 DEBUG:boto:path=/dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.407 DEBUG:boto:auth_path=/dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.407 DEBUG:boto:Method: GET 2022-09-03T05:25:58.408 DEBUG:boto:Path: /dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.408 DEBUG:boto:Data: 2022-09-03T05:25:58.409 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.410 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:58.411 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.412 DEBUG:boto:Params: {} 2022-09-03T05:25:58.413 DEBUG:boto:Token: None 2022-09-03T05:25:58.414 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/ 2022-09-03T05:25:58.415 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkLcY0gPOIyCdu8LQhjVPhOmVus= 2022-09-03T05:25:58.416 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkLcY0gPOIyCdu8LQhjVPhOmVus=', 'Content-Length': '0'} 2022-09-03T05:25:58.420 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000033d01786763158d2-006312e566-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.421 DEBUG:boto:b'dsgvcy-14myobj1000falsemyobj2022-09-03T05:25:46.137Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:58.423 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-14 zones={test-zone1, test-zone2} 2022-09-03T05:25:58.423 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-14', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:25:58.424 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-14', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:25:58.424 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-14 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:25:58.482 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:25:58.483 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:25:58.538 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:25:58.540+0000 7fd1a786d600 1 bucket sync caught up with source: 2022-09-03T05:25:58.539 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.6.5, , , , , , , , , ] 2022-09-03T05:25:58.539 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.6.5, , , , , , , , , ] 2022-09-03T05:25:58.540 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:25:58.540+0000 7fd1a786d600 0 bucket checkpoint complete 2022-09-03T05:25:58.555 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-14 zones={test-zone3, test-zone2} 2022-09-03T05:25:58.556 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:25:58.556 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:25:58.557 DEBUG:boto:Method: HEAD 2022-09-03T05:25:58.557 DEBUG:boto:Path: /dsgvcy-14/ 2022-09-03T05:25:58.558 DEBUG:boto:Data: 2022-09-03T05:25:58.559 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.559 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:25:58.560 DEBUG:boto:Port: 8001 2022-09-03T05:25:58.560 DEBUG:boto:Params: {} 2022-09-03T05:25:58.561 DEBUG:boto:Token: None 2022-09-03T05:25:58.561 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/ 2022-09-03T05:25:58.562 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XJPAcYXrIohx1Qum1QdpuHjMmmg= 2022-09-03T05:25:58.563 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XJPAcYXrIohx1Qum1QdpuHjMmmg=', 'Content-Length': '0'} 2022-09-03T05:25:58.565 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a90ce7c5f8580bd3-006312e566-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.566 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:25:58.567 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:25:58.568 DEBUG:boto:Method: HEAD 2022-09-03T05:25:58.568 DEBUG:boto:Path: /dsgvcy-14/ 2022-09-03T05:25:58.569 DEBUG:boto:Data: 2022-09-03T05:25:58.570 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.570 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:58.571 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.571 DEBUG:boto:Params: {} 2022-09-03T05:25:58.572 DEBUG:boto:Token: None 2022-09-03T05:25:58.573 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/ 2022-09-03T05:25:58.573 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XJPAcYXrIohx1Qum1QdpuHjMmmg= 2022-09-03T05:25:58.574 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XJPAcYXrIohx1Qum1QdpuHjMmmg=', 'Content-Length': '0'} 2022-09-03T05:25:58.579 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008797a5ec4bc010d7-006312e566-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.580 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:25:58.581 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:25:58.581 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:25:58.582 DEBUG:boto:path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.582 DEBUG:boto:auth_path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.583 DEBUG:boto:Method: GET 2022-09-03T05:25:58.584 DEBUG:boto:Path: /dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.584 DEBUG:boto:Data: 2022-09-03T05:25:58.585 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.585 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:25:58.586 DEBUG:boto:Port: 8001 2022-09-03T05:25:58.586 DEBUG:boto:Params: {} 2022-09-03T05:25:58.587 DEBUG:boto:Token: None 2022-09-03T05:25:58.588 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/?versions 2022-09-03T05:25:58.588 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0RAFPEDbuRljx7rWL2BF/IKh7KI= 2022-09-03T05:25:58.589 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0RAFPEDbuRljx7rWL2BF/IKh7KI=', 'Content-Length': '0'} 2022-09-03T05:25:58.591 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d196a9ad2e63fa7a-006312e566-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.592 DEBUG:boto:b'dsgvcy-14999falsemyobjnulltrue2022-09-03T05:25:46.137Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:58.593 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2022-09-03T05:25:58.593 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:25:58.594 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:25:58.594 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:25:58.595 DEBUG:boto:path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.595 DEBUG:boto:auth_path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.596 DEBUG:boto:Method: GET 2022-09-03T05:25:58.596 DEBUG:boto:Path: /dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.597 DEBUG:boto:Data: 2022-09-03T05:25:58.597 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.598 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:58.598 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.599 DEBUG:boto:Params: {} 2022-09-03T05:25:58.600 DEBUG:boto:Token: None 2022-09-03T05:25:58.600 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/?versions 2022-09-03T05:25:58.601 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0RAFPEDbuRljx7rWL2BF/IKh7KI= 2022-09-03T05:25:58.602 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0RAFPEDbuRljx7rWL2BF/IKh7KI=', 'Content-Length': '0'} 2022-09-03T05:25:58.605 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e48a5a905c599a2e-006312e566-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.606 DEBUG:boto:b'dsgvcy-14999falsemyobjnulltrue2022-09-03T05:25:46.137Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:58.607 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2022-09-03T05:25:58.607 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:25:58.608 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:25:58.608 DEBUG:boto:path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.609 DEBUG:boto:auth_path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.609 DEBUG:boto:Method: GET 2022-09-03T05:25:58.610 DEBUG:boto:Path: /dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.610 DEBUG:boto:Data: 2022-09-03T05:25:58.611 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.611 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:25:58.612 DEBUG:boto:Port: 8001 2022-09-03T05:25:58.612 DEBUG:boto:Params: {} 2022-09-03T05:25:58.613 DEBUG:boto:Token: None 2022-09-03T05:25:58.614 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/?versions 2022-09-03T05:25:58.614 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0RAFPEDbuRljx7rWL2BF/IKh7KI= 2022-09-03T05:25:58.615 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0RAFPEDbuRljx7rWL2BF/IKh7KI=', 'Content-Length': '0'} 2022-09-03T05:25:58.617 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cc376b0963ce2f35-006312e566-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.658 DEBUG:boto:b'dsgvcy-14999falsemyobjnulltrue2022-09-03T05:25:46.137Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:58.659 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:25:58.660 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:25:58.660 DEBUG:boto:path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.661 DEBUG:boto:auth_path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.661 DEBUG:boto:Method: GET 2022-09-03T05:25:58.662 DEBUG:boto:Path: /dsgvcy-14/?versions&max-keys=999 2022-09-03T05:25:58.662 DEBUG:boto:Data: 2022-09-03T05:25:58.663 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.663 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:58.664 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.664 DEBUG:boto:Params: {} 2022-09-03T05:25:58.665 DEBUG:boto:Token: None 2022-09-03T05:25:58.666 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/?versions 2022-09-03T05:25:58.666 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0RAFPEDbuRljx7rWL2BF/IKh7KI= 2022-09-03T05:25:58.667 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0RAFPEDbuRljx7rWL2BF/IKh7KI=', 'Content-Length': '0'} 2022-09-03T05:25:58.669 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007d6cb44c61b10c82-006312e566-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.670 DEBUG:boto:b'dsgvcy-14999falsemyobjnulltrue2022-09-03T05:25:46.137Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:58.671 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:25:58.671 DEBUG:boto:path=/dsgvcy-14/myobj 2022-09-03T05:25:58.672 DEBUG:boto:auth_path=/dsgvcy-14/myobj 2022-09-03T05:25:58.673 DEBUG:boto:path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.673 DEBUG:boto:auth_path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.674 DEBUG:boto:Method: GET 2022-09-03T05:25:58.674 DEBUG:boto:Path: /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.675 DEBUG:boto:Data: 2022-09-03T05:25:58.675 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.676 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:25:58.676 DEBUG:boto:Port: 8001 2022-09-03T05:25:58.677 DEBUG:boto:Params: {} 2022-09-03T05:25:58.677 DEBUG:boto:Token: None 2022-09-03T05:25:58.678 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.679 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vTZZ1osZ5MUcfEur6EfStmaywaA= 2022-09-03T05:25:58.679 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vTZZ1osZ5MUcfEur6EfStmaywaA=', 'Content-Length': '0'} 2022-09-03T05:25:58.681 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f8d3aad03d9be323-006312e566-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.682 DEBUG:boto:path=/dsgvcy-14/myobj 2022-09-03T05:25:58.683 DEBUG:boto:auth_path=/dsgvcy-14/myobj 2022-09-03T05:25:58.683 DEBUG:boto:path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.684 DEBUG:boto:auth_path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.685 DEBUG:boto:Method: GET 2022-09-03T05:25:58.685 DEBUG:boto:Path: /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.686 DEBUG:boto:Data: 2022-09-03T05:25:58.686 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.687 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:58.687 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.688 DEBUG:boto:Params: {} 2022-09-03T05:25:58.688 DEBUG:boto:Token: None 2022-09-03T05:25:58.689 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.690 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vTZZ1osZ5MUcfEur6EfStmaywaA= 2022-09-03T05:25:58.691 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vTZZ1osZ5MUcfEur6EfStmaywaA=', 'Content-Length': '0'} 2022-09-03T05:25:58.693 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000008d068f3a1f3270f6-006312e566-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.695 DEBUG:boto:path=/dsgvcy-14/myobj 2022-09-03T05:25:58.696 DEBUG:boto:auth_path=/dsgvcy-14/myobj 2022-09-03T05:25:58.697 DEBUG:boto:path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.698 DEBUG:boto:auth_path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.699 DEBUG:boto:Method: HEAD 2022-09-03T05:25:58.700 DEBUG:boto:Path: /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.701 DEBUG:boto:Data: 2022-09-03T05:25:58.702 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.702 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:25:58.703 DEBUG:boto:Port: 8001 2022-09-03T05:25:58.703 DEBUG:boto:Params: {} 2022-09-03T05:25:58.704 DEBUG:boto:Token: None 2022-09-03T05:25:58.705 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.705 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rrjNjxC5sJoe7NgTs4hqHp+lj6I= 2022-09-03T05:25:58.706 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rrjNjxC5sJoe7NgTs4hqHp+lj6I=', 'Content-Length': '0'} 2022-09-03T05:25:58.708 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000005e8d11bd04dc3f36-006312e566-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.709 DEBUG:boto:path=/dsgvcy-14/myobj 2022-09-03T05:25:58.709 DEBUG:boto:auth_path=/dsgvcy-14/myobj 2022-09-03T05:25:58.710 DEBUG:boto:path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.710 DEBUG:boto:auth_path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.711 DEBUG:boto:Method: HEAD 2022-09-03T05:25:58.711 DEBUG:boto:Path: /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.712 DEBUG:boto:Data: 2022-09-03T05:25:58.712 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.713 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:58.714 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.714 DEBUG:boto:Params: {} 2022-09-03T05:25:58.715 DEBUG:boto:Token: None 2022-09-03T05:25:58.715 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.716 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rrjNjxC5sJoe7NgTs4hqHp+lj6I= 2022-09-03T05:25:58.717 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rrjNjxC5sJoe7NgTs4hqHp+lj6I=', 'Content-Length': '0'} 2022-09-03T05:25:58.719 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007f457d8d59577e31-006312e566-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.720 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:25:58.720 DEBUG:boto:path=/dsgvcy-14/myobj 2022-09-03T05:25:58.721 DEBUG:boto:auth_path=/dsgvcy-14/myobj 2022-09-03T05:25:58.722 DEBUG:boto:path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.722 DEBUG:boto:auth_path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.723 DEBUG:boto:Method: GET 2022-09-03T05:25:58.723 DEBUG:boto:Path: /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.724 DEBUG:boto:Data: 2022-09-03T05:25:58.724 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.725 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:25:58.725 DEBUG:boto:Port: 8001 2022-09-03T05:25:58.726 DEBUG:boto:Params: {} 2022-09-03T05:25:58.727 DEBUG:boto:Token: None 2022-09-03T05:25:58.727 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.728 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vTZZ1osZ5MUcfEur6EfStmaywaA= 2022-09-03T05:25:58.729 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vTZZ1osZ5MUcfEur6EfStmaywaA=', 'Content-Length': '0'} 2022-09-03T05:25:58.730 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000013d8528beec5d679-006312e566-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.731 DEBUG:boto:path=/dsgvcy-14/myobj 2022-09-03T05:25:58.732 DEBUG:boto:auth_path=/dsgvcy-14/myobj 2022-09-03T05:25:58.732 DEBUG:boto:path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.733 DEBUG:boto:auth_path=/dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.734 DEBUG:boto:Method: GET 2022-09-03T05:25:58.734 DEBUG:boto:Path: /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.735 DEBUG:boto:Data: 2022-09-03T05:25:58.735 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.736 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:58.737 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.737 DEBUG:boto:Params: {} 2022-09-03T05:25:58.738 DEBUG:boto:Token: None 2022-09-03T05:25:58.738 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/myobj?versionId=null 2022-09-03T05:25:58.739 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vTZZ1osZ5MUcfEur6EfStmaywaA= 2022-09-03T05:25:58.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vTZZ1osZ5MUcfEur6EfStmaywaA=', 'Content-Length': '0'} 2022-09-03T05:25:58.742 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007023d7bb41c309f8-006312e566-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.782 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:25:58.783 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:25:58.784 DEBUG:boto:path=/dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.785 DEBUG:boto:auth_path=/dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.786 DEBUG:boto:Method: GET 2022-09-03T05:25:58.787 DEBUG:boto:Path: /dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.788 DEBUG:boto:Data: 2022-09-03T05:25:58.788 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.789 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:25:58.791 DEBUG:boto:Port: 8001 2022-09-03T05:25:58.792 DEBUG:boto:Params: {} 2022-09-03T05:25:58.793 DEBUG:boto:Token: None 2022-09-03T05:25:58.794 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/ 2022-09-03T05:25:58.796 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkLcY0gPOIyCdu8LQhjVPhOmVus= 2022-09-03T05:25:58.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkLcY0gPOIyCdu8LQhjVPhOmVus=', 'Content-Length': '0'} 2022-09-03T05:25:58.799 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e266ee1a1d9bdbe1-006312e566-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.801 DEBUG:boto:b'dsgvcy-14myobj1000falsemyobj2022-09-03T05:25:46.137Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:58.802 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:25:58.802 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:25:58.803 DEBUG:boto:path=/dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.803 DEBUG:boto:auth_path=/dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.804 DEBUG:boto:Method: GET 2022-09-03T05:25:58.804 DEBUG:boto:Path: /dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.805 DEBUG:boto:Data: 2022-09-03T05:25:58.805 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.806 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:58.806 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.807 DEBUG:boto:Params: {} 2022-09-03T05:25:58.808 DEBUG:boto:Token: None 2022-09-03T05:25:58.809 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/ 2022-09-03T05:25:58.810 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkLcY0gPOIyCdu8LQhjVPhOmVus= 2022-09-03T05:25:58.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkLcY0gPOIyCdu8LQhjVPhOmVus=', 'Content-Length': '0'} 2022-09-03T05:25:58.813 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a6dcae57a903e423-006312e566-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.814 DEBUG:boto:b'dsgvcy-14myobj1000falsemyobj2022-09-03T05:25:46.137Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:58.816 DEBUG:boto:path=/dsgvcy-14/myobj 2022-09-03T05:25:58.816 DEBUG:boto:auth_path=/dsgvcy-14/myobj 2022-09-03T05:25:58.817 DEBUG:boto:Method: HEAD 2022-09-03T05:25:58.818 DEBUG:boto:Path: /dsgvcy-14/myobj 2022-09-03T05:25:58.819 DEBUG:boto:Data: 2022-09-03T05:25:58.819 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.820 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:25:58.821 DEBUG:boto:Port: 8001 2022-09-03T05:25:58.821 DEBUG:boto:Params: {} 2022-09-03T05:25:58.822 DEBUG:boto:Token: None 2022-09-03T05:25:58.823 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/myobj 2022-09-03T05:25:58.824 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YmwtxgihhbS2tNSfiMFH5EjE9gQ= 2022-09-03T05:25:58.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YmwtxgihhbS2tNSfiMFH5EjE9gQ=', 'Content-Length': '0'} 2022-09-03T05:25:58.828 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b51c8e8c7735e113-006312e566-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.829 DEBUG:boto:path=/dsgvcy-14/myobj 2022-09-03T05:25:58.830 DEBUG:boto:auth_path=/dsgvcy-14/myobj 2022-09-03T05:25:58.831 DEBUG:boto:Method: HEAD 2022-09-03T05:25:58.832 DEBUG:boto:Path: /dsgvcy-14/myobj 2022-09-03T05:25:58.832 DEBUG:boto:Data: 2022-09-03T05:25:58.833 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.834 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:58.835 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.836 DEBUG:boto:Params: {} 2022-09-03T05:25:58.837 DEBUG:boto:Token: None 2022-09-03T05:25:58.838 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/myobj 2022-09-03T05:25:58.839 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YmwtxgihhbS2tNSfiMFH5EjE9gQ= 2022-09-03T05:25:58.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YmwtxgihhbS2tNSfiMFH5EjE9gQ=', 'Content-Length': '0'} 2022-09-03T05:25:58.842 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f9a7d2128b060310-006312e566-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.843 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:25:58.844 DEBUG:boto:path=/dsgvcy-14/myobj 2022-09-03T05:25:58.845 DEBUG:boto:auth_path=/dsgvcy-14/myobj 2022-09-03T05:25:58.845 DEBUG:boto:Method: GET 2022-09-03T05:25:58.846 DEBUG:boto:Path: /dsgvcy-14/myobj 2022-09-03T05:25:58.847 DEBUG:boto:Data: 2022-09-03T05:25:58.847 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.848 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:25:58.848 DEBUG:boto:Port: 8001 2022-09-03T05:25:58.849 DEBUG:boto:Params: {} 2022-09-03T05:25:58.850 DEBUG:boto:Token: None 2022-09-03T05:25:58.851 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/myobj 2022-09-03T05:25:58.852 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Pjk5KiRi09fzYU6JvYXFHXU/pPY= 2022-09-03T05:25:58.853 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Pjk5KiRi09fzYU6JvYXFHXU/pPY=', 'Content-Length': '0'} 2022-09-03T05:25:58.855 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000374fe7d6e0bcc52e-006312e566-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.856 DEBUG:boto:path=/dsgvcy-14/myobj 2022-09-03T05:25:58.857 DEBUG:boto:auth_path=/dsgvcy-14/myobj 2022-09-03T05:25:58.858 DEBUG:boto:Method: GET 2022-09-03T05:25:58.859 DEBUG:boto:Path: /dsgvcy-14/myobj 2022-09-03T05:25:58.860 DEBUG:boto:Data: 2022-09-03T05:25:58.861 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.862 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:58.863 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.864 DEBUG:boto:Params: {} 2022-09-03T05:25:58.865 DEBUG:boto:Token: None 2022-09-03T05:25:58.866 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/myobj 2022-09-03T05:25:58.867 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Pjk5KiRi09fzYU6JvYXFHXU/pPY= 2022-09-03T05:25:58.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Pjk5KiRi09fzYU6JvYXFHXU/pPY=', 'Content-Length': '0'} 2022-09-03T05:25:58.871 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000014fbfb178253f277-006312e566-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.873 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:25:58.874 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:25:58.874 DEBUG:boto:path=/dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.875 DEBUG:boto:auth_path=/dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.876 DEBUG:boto:Method: GET 2022-09-03T05:25:58.877 DEBUG:boto:Path: /dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.878 DEBUG:boto:Data: 2022-09-03T05:25:58.878 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.879 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:25:58.880 DEBUG:boto:Port: 8001 2022-09-03T05:25:58.880 DEBUG:boto:Params: {} 2022-09-03T05:25:58.881 DEBUG:boto:Token: None 2022-09-03T05:25:58.882 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/ 2022-09-03T05:25:58.883 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkLcY0gPOIyCdu8LQhjVPhOmVus= 2022-09-03T05:25:58.883 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkLcY0gPOIyCdu8LQhjVPhOmVus=', 'Content-Length': '0'} 2022-09-03T05:25:58.886 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dd1894427cfc55ff-006312e566-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.886 DEBUG:boto:b'dsgvcy-14myobj1000falsemyobj2022-09-03T05:25:46.137Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:58.887 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:25:58.888 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:25:58.888 DEBUG:boto:path=/dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.889 DEBUG:boto:auth_path=/dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.889 DEBUG:boto:Method: GET 2022-09-03T05:25:58.890 DEBUG:boto:Path: /dsgvcy-14/?prefix=myobj 2022-09-03T05:25:58.890 DEBUG:boto:Data: 2022-09-03T05:25:58.891 DEBUG:boto:Headers: {} 2022-09-03T05:25:58.891 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:25:58.892 DEBUG:boto:Port: 8000 2022-09-03T05:25:58.892 DEBUG:boto:Params: {} 2022-09-03T05:25:58.893 DEBUG:boto:Token: None 2022-09-03T05:25:58.894 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:25:58 GMT /dsgvcy-14/ 2022-09-03T05:25:58.894 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkLcY0gPOIyCdu8LQhjVPhOmVus= 2022-09-03T05:25:58.895 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:25:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkLcY0gPOIyCdu8LQhjVPhOmVus=', 'Content-Length': '0'} 2022-09-03T05:25:58.898 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000050205819b66fdca7-006312e566-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:25:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:25:58.898 DEBUG:boto:b'dsgvcy-14myobj1000falsemyobj2022-09-03T05:25:46.137Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:25:58.899 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-14 zones={test-zone3, test-zone2} 2022-09-03T05:25:58.900 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-15', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:25:58.900 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-15', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:25:58.901 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-15 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:25:58.958 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:25:58.958 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:25:58.991 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:25:58.993+0000 7f28c326e600 1 waiting to reach incremental sync.. 2022-09-03T05:26:03.992 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:26:03.994+0000 7f28c326e600 1 waiting to reach incremental sync.. 2022-09-03T05:26:08.994 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:26:08.997+0000 7f28c326e600 1 bucket sync caught up with source: 2022-09-03T05:26:08.994 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.39.5, , , , , , , , , ] 2022-09-03T05:26:08.995 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.39.5, , , , , , , , , ] 2022-09-03T05:26:08.995 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:26:08.997+0000 7f28c326e600 0 bucket checkpoint complete 2022-09-03T05:26:09.009 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-15 zones={test-zone1, test-zone3} 2022-09-03T05:26:09.010 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:26:09.010 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:26:09.011 DEBUG:boto:Method: HEAD 2022-09-03T05:26:09.012 DEBUG:boto:Path: /dsgvcy-15/ 2022-09-03T05:26:09.012 DEBUG:boto:Data: 2022-09-03T05:26:09.013 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.013 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:26:09.014 DEBUG:boto:Port: 8000 2022-09-03T05:26:09.014 DEBUG:boto:Params: {} 2022-09-03T05:26:09.015 DEBUG:boto:Token: None 2022-09-03T05:26:09.015 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/ 2022-09-03T05:26:09.016 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:APYzeuRQ8oz70oPXYPsoCLKxkVg= 2022-09-03T05:26:09.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:APYzeuRQ8oz70oPXYPsoCLKxkVg=', 'Content-Length': '0'} 2022-09-03T05:26:09.020 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000055034307be3fd61c-006312e571-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.021 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:26:09.021 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:26:09.022 DEBUG:boto:Method: HEAD 2022-09-03T05:26:09.022 DEBUG:boto:Path: /dsgvcy-15/ 2022-09-03T05:26:09.023 DEBUG:boto:Data: 2022-09-03T05:26:09.023 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.024 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:26:09.024 DEBUG:boto:Port: 8001 2022-09-03T05:26:09.025 DEBUG:boto:Params: {} 2022-09-03T05:26:09.026 DEBUG:boto:Token: None 2022-09-03T05:26:09.026 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/ 2022-09-03T05:26:09.027 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:APYzeuRQ8oz70oPXYPsoCLKxkVg= 2022-09-03T05:26:09.027 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:APYzeuRQ8oz70oPXYPsoCLKxkVg=', 'Content-Length': '0'} 2022-09-03T05:26:09.031 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ea25d9ce0d6921db-006312e571-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.032 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:26:09.033 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:26:09.034 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:26:09.035 DEBUG:boto:path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.036 DEBUG:boto:auth_path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.037 DEBUG:boto:Method: GET 2022-09-03T05:26:09.037 DEBUG:boto:Path: /dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.038 DEBUG:boto:Data: 2022-09-03T05:26:09.039 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.040 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:26:09.041 DEBUG:boto:Port: 8000 2022-09-03T05:26:09.042 DEBUG:boto:Params: {} 2022-09-03T05:26:09.044 DEBUG:boto:Token: None 2022-09-03T05:26:09.045 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/?versions 2022-09-03T05:26:09.046 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nt6wNnFVUa4MR/rrl4SLCClwx7Y= 2022-09-03T05:26:09.047 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nt6wNnFVUa4MR/rrl4SLCClwx7Y=', 'Content-Length': '0'} 2022-09-03T05:26:09.050 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000392a6ffcbd939411-006312e571-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.051 DEBUG:boto:b'dsgvcy-15999falsemyobjnulltrue2022-09-03T05:25:46.165Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:26:09.052 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2022-09-03T05:26:09.052 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:26:09.053 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:26:09.053 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:26:09.054 DEBUG:boto:path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.054 DEBUG:boto:auth_path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.055 DEBUG:boto:Method: GET 2022-09-03T05:26:09.056 DEBUG:boto:Path: /dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.056 DEBUG:boto:Data: 2022-09-03T05:26:09.057 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.057 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:26:09.058 DEBUG:boto:Port: 8001 2022-09-03T05:26:09.058 DEBUG:boto:Params: {} 2022-09-03T05:26:09.059 DEBUG:boto:Token: None 2022-09-03T05:26:09.059 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/?versions 2022-09-03T05:26:09.060 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nt6wNnFVUa4MR/rrl4SLCClwx7Y= 2022-09-03T05:26:09.061 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nt6wNnFVUa4MR/rrl4SLCClwx7Y=', 'Content-Length': '0'} 2022-09-03T05:26:09.063 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002178e0dc88027ade-006312e571-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.064 DEBUG:boto:b'dsgvcy-15999falsemyobjnulltrue2022-09-03T05:25:46.165Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:26:09.065 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2022-09-03T05:26:09.066 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:26:09.067 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:26:09.067 DEBUG:boto:path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.068 DEBUG:boto:auth_path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.069 DEBUG:boto:Method: GET 2022-09-03T05:26:09.069 DEBUG:boto:Path: /dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.070 DEBUG:boto:Data: 2022-09-03T05:26:09.070 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.071 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:26:09.071 DEBUG:boto:Port: 8000 2022-09-03T05:26:09.072 DEBUG:boto:Params: {} 2022-09-03T05:26:09.073 DEBUG:boto:Token: None 2022-09-03T05:26:09.073 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/?versions 2022-09-03T05:26:09.074 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nt6wNnFVUa4MR/rrl4SLCClwx7Y= 2022-09-03T05:26:09.075 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nt6wNnFVUa4MR/rrl4SLCClwx7Y=', 'Content-Length': '0'} 2022-09-03T05:26:09.078 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002774b1a99ea5c343-006312e571-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.123 DEBUG:boto:b'dsgvcy-15999falsemyobjnulltrue2022-09-03T05:25:46.165Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:26:09.125 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:26:09.127 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:26:09.128 DEBUG:boto:path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.129 DEBUG:boto:auth_path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.130 DEBUG:boto:Method: GET 2022-09-03T05:26:09.131 DEBUG:boto:Path: /dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.132 DEBUG:boto:Data: 2022-09-03T05:26:09.133 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.133 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:26:09.136 DEBUG:boto:Port: 8001 2022-09-03T05:26:09.137 DEBUG:boto:Params: {} 2022-09-03T05:26:09.138 DEBUG:boto:Token: None 2022-09-03T05:26:09.139 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/?versions 2022-09-03T05:26:09.140 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nt6wNnFVUa4MR/rrl4SLCClwx7Y= 2022-09-03T05:26:09.143 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nt6wNnFVUa4MR/rrl4SLCClwx7Y=', 'Content-Length': '0'} 2022-09-03T05:26:09.147 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000db0029dee37ca896-006312e571-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.149 DEBUG:boto:b'dsgvcy-15999falsemyobjnulltrue2022-09-03T05:25:46.165Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:26:09.150 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:26:09.151 DEBUG:boto:path=/dsgvcy-15/myobj 2022-09-03T05:26:09.152 DEBUG:boto:auth_path=/dsgvcy-15/myobj 2022-09-03T05:26:09.153 DEBUG:boto:path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.153 DEBUG:boto:auth_path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.154 DEBUG:boto:Method: GET 2022-09-03T05:26:09.157 DEBUG:boto:Path: /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.158 DEBUG:boto:Data: 2022-09-03T05:26:09.159 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.160 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:26:09.160 DEBUG:boto:Port: 8000 2022-09-03T05:26:09.161 DEBUG:boto:Params: {} 2022-09-03T05:26:09.163 DEBUG:boto:Token: None 2022-09-03T05:26:09.164 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.165 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eMbYdqlOlbt8O1d5IV3CCHiDFOw= 2022-09-03T05:26:09.167 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eMbYdqlOlbt8O1d5IV3CCHiDFOw=', 'Content-Length': '0'} 2022-09-03T05:26:09.170 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000051af6ed7626fab2d-006312e571-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.172 DEBUG:boto:path=/dsgvcy-15/myobj 2022-09-03T05:26:09.173 DEBUG:boto:auth_path=/dsgvcy-15/myobj 2022-09-03T05:26:09.174 DEBUG:boto:path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.175 DEBUG:boto:auth_path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.177 DEBUG:boto:Method: GET 2022-09-03T05:26:09.178 DEBUG:boto:Path: /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.179 DEBUG:boto:Data: 2022-09-03T05:26:09.180 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.181 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:26:09.182 DEBUG:boto:Port: 8001 2022-09-03T05:26:09.183 DEBUG:boto:Params: {} 2022-09-03T05:26:09.185 DEBUG:boto:Token: None 2022-09-03T05:26:09.185 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.186 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eMbYdqlOlbt8O1d5IV3CCHiDFOw= 2022-09-03T05:26:09.187 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eMbYdqlOlbt8O1d5IV3CCHiDFOw=', 'Content-Length': '0'} 2022-09-03T05:26:09.189 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000f8eb29bf4a5ae151-006312e571-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.190 DEBUG:boto:path=/dsgvcy-15/myobj 2022-09-03T05:26:09.191 DEBUG:boto:auth_path=/dsgvcy-15/myobj 2022-09-03T05:26:09.191 DEBUG:boto:path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.192 DEBUG:boto:auth_path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.192 DEBUG:boto:Method: HEAD 2022-09-03T05:26:09.193 DEBUG:boto:Path: /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.194 DEBUG:boto:Data: 2022-09-03T05:26:09.194 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.195 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:26:09.195 DEBUG:boto:Port: 8000 2022-09-03T05:26:09.196 DEBUG:boto:Params: {} 2022-09-03T05:26:09.196 DEBUG:boto:Token: None 2022-09-03T05:26:09.197 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.198 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ctnb7N/c9v4VDGylSEo32gEIN/U= 2022-09-03T05:26:09.198 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ctnb7N/c9v4VDGylSEo32gEIN/U=', 'Content-Length': '0'} 2022-09-03T05:26:09.201 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000b942dbaec3c0d8ce-006312e571-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.202 DEBUG:boto:path=/dsgvcy-15/myobj 2022-09-03T05:26:09.202 DEBUG:boto:auth_path=/dsgvcy-15/myobj 2022-09-03T05:26:09.203 DEBUG:boto:path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.204 DEBUG:boto:auth_path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.205 DEBUG:boto:Method: HEAD 2022-09-03T05:26:09.205 DEBUG:boto:Path: /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.206 DEBUG:boto:Data: 2022-09-03T05:26:09.207 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.207 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:26:09.208 DEBUG:boto:Port: 8001 2022-09-03T05:26:09.209 DEBUG:boto:Params: {} 2022-09-03T05:26:09.209 DEBUG:boto:Token: None 2022-09-03T05:26:09.210 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.211 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ctnb7N/c9v4VDGylSEo32gEIN/U= 2022-09-03T05:26:09.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ctnb7N/c9v4VDGylSEo32gEIN/U=', 'Content-Length': '0'} 2022-09-03T05:26:09.215 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004b4a62dc5418bd10-006312e571-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.216 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:26:09.217 DEBUG:boto:path=/dsgvcy-15/myobj 2022-09-03T05:26:09.219 DEBUG:boto:auth_path=/dsgvcy-15/myobj 2022-09-03T05:26:09.220 DEBUG:boto:path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.221 DEBUG:boto:auth_path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.221 DEBUG:boto:Method: GET 2022-09-03T05:26:09.222 DEBUG:boto:Path: /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.223 DEBUG:boto:Data: 2022-09-03T05:26:09.224 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.225 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:26:09.226 DEBUG:boto:Port: 8000 2022-09-03T05:26:09.229 DEBUG:boto:Params: {} 2022-09-03T05:26:09.230 DEBUG:boto:Token: None 2022-09-03T05:26:09.232 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.233 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eMbYdqlOlbt8O1d5IV3CCHiDFOw= 2022-09-03T05:26:09.234 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eMbYdqlOlbt8O1d5IV3CCHiDFOw=', 'Content-Length': '0'} 2022-09-03T05:26:09.237 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000001ce5a9f1605a1399-006312e571-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.239 DEBUG:boto:path=/dsgvcy-15/myobj 2022-09-03T05:26:09.240 DEBUG:boto:auth_path=/dsgvcy-15/myobj 2022-09-03T05:26:09.241 DEBUG:boto:path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.242 DEBUG:boto:auth_path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.243 DEBUG:boto:Method: GET 2022-09-03T05:26:09.244 DEBUG:boto:Path: /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.245 DEBUG:boto:Data: 2022-09-03T05:26:09.246 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.247 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:26:09.248 DEBUG:boto:Port: 8001 2022-09-03T05:26:09.249 DEBUG:boto:Params: {} 2022-09-03T05:26:09.251 DEBUG:boto:Token: None 2022-09-03T05:26:09.252 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.253 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eMbYdqlOlbt8O1d5IV3CCHiDFOw= 2022-09-03T05:26:09.254 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eMbYdqlOlbt8O1d5IV3CCHiDFOw=', 'Content-Length': '0'} 2022-09-03T05:26:09.257 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000006072b3ce132c0eb8-006312e571-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.298 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:26:09.299 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:26:09.301 DEBUG:boto:path=/dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.302 DEBUG:boto:auth_path=/dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.303 DEBUG:boto:Method: GET 2022-09-03T05:26:09.304 DEBUG:boto:Path: /dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.305 DEBUG:boto:Data: 2022-09-03T05:26:09.306 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.308 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:26:09.310 DEBUG:boto:Port: 8000 2022-09-03T05:26:09.311 DEBUG:boto:Params: {} 2022-09-03T05:26:09.312 DEBUG:boto:Token: None 2022-09-03T05:26:09.313 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/ 2022-09-03T05:26:09.314 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pdOEBqkdpxMiZmY05F4B2MFKBZY= 2022-09-03T05:26:09.316 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pdOEBqkdpxMiZmY05F4B2MFKBZY=', 'Content-Length': '0'} 2022-09-03T05:26:09.319 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c5ae98a27c96a1bb-006312e571-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.321 DEBUG:boto:b'dsgvcy-15myobj1000falsemyobj2022-09-03T05:25:46.165Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:26:09.322 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:26:09.323 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:26:09.324 DEBUG:boto:path=/dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.325 DEBUG:boto:auth_path=/dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.326 DEBUG:boto:Method: GET 2022-09-03T05:26:09.327 DEBUG:boto:Path: /dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.330 DEBUG:boto:Data: 2022-09-03T05:26:09.331 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.332 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:26:09.332 DEBUG:boto:Port: 8001 2022-09-03T05:26:09.333 DEBUG:boto:Params: {} 2022-09-03T05:26:09.333 DEBUG:boto:Token: None 2022-09-03T05:26:09.334 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/ 2022-09-03T05:26:09.335 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pdOEBqkdpxMiZmY05F4B2MFKBZY= 2022-09-03T05:26:09.335 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pdOEBqkdpxMiZmY05F4B2MFKBZY=', 'Content-Length': '0'} 2022-09-03T05:26:09.350 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007167260f904cab8b-006312e571-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.351 DEBUG:boto:b'dsgvcy-15myobj1000falsemyobj2022-09-03T05:25:46.165Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:26:09.352 DEBUG:boto:path=/dsgvcy-15/myobj 2022-09-03T05:26:09.352 DEBUG:boto:auth_path=/dsgvcy-15/myobj 2022-09-03T05:26:09.353 DEBUG:boto:Method: HEAD 2022-09-03T05:26:09.354 DEBUG:boto:Path: /dsgvcy-15/myobj 2022-09-03T05:26:09.356 DEBUG:boto:Data: 2022-09-03T05:26:09.357 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.357 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:26:09.358 DEBUG:boto:Port: 8000 2022-09-03T05:26:09.359 DEBUG:boto:Params: {} 2022-09-03T05:26:09.361 DEBUG:boto:Token: None 2022-09-03T05:26:09.362 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/myobj 2022-09-03T05:26:09.363 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YDGOy4MqdDjnjUu9fROiUhIOm28= 2022-09-03T05:26:09.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YDGOy4MqdDjnjUu9fROiUhIOm28=', 'Content-Length': '0'} 2022-09-03T05:26:09.367 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000239e9914f42ea6e7-006312e571-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.368 DEBUG:boto:path=/dsgvcy-15/myobj 2022-09-03T05:26:09.369 DEBUG:boto:auth_path=/dsgvcy-15/myobj 2022-09-03T05:26:09.370 DEBUG:boto:Method: HEAD 2022-09-03T05:26:09.370 DEBUG:boto:Path: /dsgvcy-15/myobj 2022-09-03T05:26:09.371 DEBUG:boto:Data: 2022-09-03T05:26:09.371 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.372 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:26:09.372 DEBUG:boto:Port: 8001 2022-09-03T05:26:09.373 DEBUG:boto:Params: {} 2022-09-03T05:26:09.377 DEBUG:boto:Token: None 2022-09-03T05:26:09.378 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/myobj 2022-09-03T05:26:09.379 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YDGOy4MqdDjnjUu9fROiUhIOm28= 2022-09-03T05:26:09.379 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YDGOy4MqdDjnjUu9fROiUhIOm28=', 'Content-Length': '0'} 2022-09-03T05:26:09.381 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000aa87b037ba6812a6-006312e571-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.382 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:26:09.383 DEBUG:boto:path=/dsgvcy-15/myobj 2022-09-03T05:26:09.383 DEBUG:boto:auth_path=/dsgvcy-15/myobj 2022-09-03T05:26:09.384 DEBUG:boto:Method: GET 2022-09-03T05:26:09.384 DEBUG:boto:Path: /dsgvcy-15/myobj 2022-09-03T05:26:09.385 DEBUG:boto:Data: 2022-09-03T05:26:09.385 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.386 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:26:09.386 DEBUG:boto:Port: 8000 2022-09-03T05:26:09.387 DEBUG:boto:Params: {} 2022-09-03T05:26:09.388 DEBUG:boto:Token: None 2022-09-03T05:26:09.388 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/myobj 2022-09-03T05:26:09.389 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CvkWTh892WxFasmogwwYnLf83EQ= 2022-09-03T05:26:09.389 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CvkWTh892WxFasmogwwYnLf83EQ=', 'Content-Length': '0'} 2022-09-03T05:26:09.396 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c8c0872a0cd4ee8c-006312e571-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.398 DEBUG:boto:path=/dsgvcy-15/myobj 2022-09-03T05:26:09.398 DEBUG:boto:auth_path=/dsgvcy-15/myobj 2022-09-03T05:26:09.399 DEBUG:boto:Method: GET 2022-09-03T05:26:09.400 DEBUG:boto:Path: /dsgvcy-15/myobj 2022-09-03T05:26:09.400 DEBUG:boto:Data: 2022-09-03T05:26:09.401 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.401 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:26:09.402 DEBUG:boto:Port: 8001 2022-09-03T05:26:09.402 DEBUG:boto:Params: {} 2022-09-03T05:26:09.403 DEBUG:boto:Token: None 2022-09-03T05:26:09.406 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/myobj 2022-09-03T05:26:09.407 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CvkWTh892WxFasmogwwYnLf83EQ= 2022-09-03T05:26:09.408 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CvkWTh892WxFasmogwwYnLf83EQ=', 'Content-Length': '0'} 2022-09-03T05:26:09.414 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000ed25486ba0d9b6f5-006312e571-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.415 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:26:09.417 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:26:09.417 DEBUG:boto:path=/dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.418 DEBUG:boto:auth_path=/dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.419 DEBUG:boto:Method: GET 2022-09-03T05:26:09.420 DEBUG:boto:Path: /dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.421 DEBUG:boto:Data: 2022-09-03T05:26:09.422 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.422 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:26:09.423 DEBUG:boto:Port: 8000 2022-09-03T05:26:09.423 DEBUG:boto:Params: {} 2022-09-03T05:26:09.424 DEBUG:boto:Token: None 2022-09-03T05:26:09.425 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/ 2022-09-03T05:26:09.426 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pdOEBqkdpxMiZmY05F4B2MFKBZY= 2022-09-03T05:26:09.427 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pdOEBqkdpxMiZmY05F4B2MFKBZY=', 'Content-Length': '0'} 2022-09-03T05:26:09.431 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002e065e6890277636-006312e571-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.474 DEBUG:boto:b'dsgvcy-15myobj1000falsemyobj2022-09-03T05:25:46.165Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:26:09.476 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:26:09.477 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:26:09.478 DEBUG:boto:path=/dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.479 DEBUG:boto:auth_path=/dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.479 DEBUG:boto:Method: GET 2022-09-03T05:26:09.480 DEBUG:boto:Path: /dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.480 DEBUG:boto:Data: 2022-09-03T05:26:09.481 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.481 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:26:09.482 DEBUG:boto:Port: 8001 2022-09-03T05:26:09.482 DEBUG:boto:Params: {} 2022-09-03T05:26:09.483 DEBUG:boto:Token: None 2022-09-03T05:26:09.484 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/ 2022-09-03T05:26:09.484 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pdOEBqkdpxMiZmY05F4B2MFKBZY= 2022-09-03T05:26:09.485 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pdOEBqkdpxMiZmY05F4B2MFKBZY=', 'Content-Length': '0'} 2022-09-03T05:26:09.488 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000070b550bedcbcd010-006312e571-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.489 DEBUG:boto:b'dsgvcy-15myobj1000falsemyobj2022-09-03T05:25:46.165Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:26:09.490 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-15 zones={test-zone1, test-zone3} 2022-09-03T05:26:09.491 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-15', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:26:09.491 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-15', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:26:09.492 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-15 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:26:09.556 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:26:09.557 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:26:09.592 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:26:09.594+0000 7f43c742b600 1 bucket sync caught up with source: 2022-09-03T05:26:09.592 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.39.5, , , , , , , , , ] 2022-09-03T05:26:09.593 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.39.5, , , , , , , , , ] 2022-09-03T05:26:09.593 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:26:09.594+0000 7f43c742b600 0 bucket checkpoint complete 2022-09-03T05:26:09.604 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-15 zones={test-zone2, test-zone3} 2022-09-03T05:26:09.605 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:26:09.605 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:26:09.606 DEBUG:boto:Method: HEAD 2022-09-03T05:26:09.608 DEBUG:boto:Path: /dsgvcy-15/ 2022-09-03T05:26:09.609 DEBUG:boto:Data: 2022-09-03T05:26:09.610 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.611 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:26:09.611 DEBUG:boto:Port: 8000 2022-09-03T05:26:09.612 DEBUG:boto:Params: {} 2022-09-03T05:26:09.612 DEBUG:boto:Token: None 2022-09-03T05:26:09.613 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/ 2022-09-03T05:26:09.614 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:APYzeuRQ8oz70oPXYPsoCLKxkVg= 2022-09-03T05:26:09.614 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:APYzeuRQ8oz70oPXYPsoCLKxkVg=', 'Content-Length': '0'} 2022-09-03T05:26:09.617 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000de64788a32bd2c24-006312e571-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.618 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:26:09.619 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:26:09.619 DEBUG:boto:Method: HEAD 2022-09-03T05:26:09.620 DEBUG:boto:Path: /dsgvcy-15/ 2022-09-03T05:26:09.621 DEBUG:boto:Data: 2022-09-03T05:26:09.622 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.623 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:26:09.624 DEBUG:boto:Port: 8001 2022-09-03T05:26:09.625 DEBUG:boto:Params: {} 2022-09-03T05:26:09.626 DEBUG:boto:Token: None 2022-09-03T05:26:09.627 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/ 2022-09-03T05:26:09.628 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:APYzeuRQ8oz70oPXYPsoCLKxkVg= 2022-09-03T05:26:09.629 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:APYzeuRQ8oz70oPXYPsoCLKxkVg=', 'Content-Length': '0'} 2022-09-03T05:26:09.632 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b9c3b6bcd918555e-006312e571-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.633 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:26:09.633 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:26:09.634 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:26:09.634 DEBUG:boto:path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.635 DEBUG:boto:auth_path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.635 DEBUG:boto:Method: GET 2022-09-03T05:26:09.636 DEBUG:boto:Path: /dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.636 DEBUG:boto:Data: 2022-09-03T05:26:09.637 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.637 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:26:09.638 DEBUG:boto:Port: 8000 2022-09-03T05:26:09.639 DEBUG:boto:Params: {} 2022-09-03T05:26:09.639 DEBUG:boto:Token: None 2022-09-03T05:26:09.640 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/?versions 2022-09-03T05:26:09.641 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nt6wNnFVUa4MR/rrl4SLCClwx7Y= 2022-09-03T05:26:09.641 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nt6wNnFVUa4MR/rrl4SLCClwx7Y=', 'Content-Length': '0'} 2022-09-03T05:26:09.645 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b9823740c1ed0743-006312e571-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.645 DEBUG:boto:b'dsgvcy-15999falsemyobjnulltrue2022-09-03T05:25:46.165Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:26:09.646 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2022-09-03T05:26:09.647 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:26:09.647 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:26:09.648 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:26:09.648 DEBUG:boto:path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.649 DEBUG:boto:auth_path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.649 DEBUG:boto:Method: GET 2022-09-03T05:26:09.650 DEBUG:boto:Path: /dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.651 DEBUG:boto:Data: 2022-09-03T05:26:09.651 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.652 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:26:09.652 DEBUG:boto:Port: 8001 2022-09-03T05:26:09.653 DEBUG:boto:Params: {} 2022-09-03T05:26:09.653 DEBUG:boto:Token: None 2022-09-03T05:26:09.655 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/?versions 2022-09-03T05:26:09.655 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nt6wNnFVUa4MR/rrl4SLCClwx7Y= 2022-09-03T05:26:09.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nt6wNnFVUa4MR/rrl4SLCClwx7Y=', 'Content-Length': '0'} 2022-09-03T05:26:09.658 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000037734070814aee31-006312e571-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.659 DEBUG:boto:b'dsgvcy-15999falsemyobjnulltrue2022-09-03T05:25:46.165Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:26:09.660 DEBUG:tasks.rgw_multi.zone_rados:o=myobj 2022-09-03T05:26:09.661 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:26:09.661 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:26:09.662 DEBUG:boto:path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.662 DEBUG:boto:auth_path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.663 DEBUG:boto:Method: GET 2022-09-03T05:26:09.663 DEBUG:boto:Path: /dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.664 DEBUG:boto:Data: 2022-09-03T05:26:09.665 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.665 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:26:09.666 DEBUG:boto:Port: 8000 2022-09-03T05:26:09.666 DEBUG:boto:Params: {} 2022-09-03T05:26:09.667 DEBUG:boto:Token: None 2022-09-03T05:26:09.667 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/?versions 2022-09-03T05:26:09.668 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nt6wNnFVUa4MR/rrl4SLCClwx7Y= 2022-09-03T05:26:09.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nt6wNnFVUa4MR/rrl4SLCClwx7Y=', 'Content-Length': '0'} 2022-09-03T05:26:09.671 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f97348d8199d6871-006312e571-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.672 DEBUG:boto:b'dsgvcy-15999falsemyobjnulltrue2022-09-03T05:25:46.165Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:26:09.673 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:26:09.674 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:26:09.674 DEBUG:boto:path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.675 DEBUG:boto:auth_path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.676 DEBUG:boto:Method: GET 2022-09-03T05:26:09.676 DEBUG:boto:Path: /dsgvcy-15/?versions&max-keys=999 2022-09-03T05:26:09.677 DEBUG:boto:Data: 2022-09-03T05:26:09.677 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.678 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:26:09.678 DEBUG:boto:Port: 8001 2022-09-03T05:26:09.679 DEBUG:boto:Params: {} 2022-09-03T05:26:09.679 DEBUG:boto:Token: None 2022-09-03T05:26:09.680 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/?versions 2022-09-03T05:26:09.681 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nt6wNnFVUa4MR/rrl4SLCClwx7Y= 2022-09-03T05:26:09.681 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nt6wNnFVUa4MR/rrl4SLCClwx7Y=', 'Content-Length': '0'} 2022-09-03T05:26:09.685 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000659bc9bc1745f9c-006312e571-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.730 DEBUG:boto:b'dsgvcy-15999falsemyobjnulltrue2022-09-03T05:25:46.165Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:26:09.732 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:26:09.732 DEBUG:boto:path=/dsgvcy-15/myobj 2022-09-03T05:26:09.733 DEBUG:boto:auth_path=/dsgvcy-15/myobj 2022-09-03T05:26:09.733 DEBUG:boto:path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.734 DEBUG:boto:auth_path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.735 DEBUG:boto:Method: GET 2022-09-03T05:26:09.736 DEBUG:boto:Path: /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.737 DEBUG:boto:Data: 2022-09-03T05:26:09.737 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.738 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:26:09.738 DEBUG:boto:Port: 8000 2022-09-03T05:26:09.739 DEBUG:boto:Params: {} 2022-09-03T05:26:09.740 DEBUG:boto:Token: None 2022-09-03T05:26:09.740 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.741 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eMbYdqlOlbt8O1d5IV3CCHiDFOw= 2022-09-03T05:26:09.742 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eMbYdqlOlbt8O1d5IV3CCHiDFOw=', 'Content-Length': '0'} 2022-09-03T05:26:09.744 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000002440a1480d371f4b-006312e571-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.745 DEBUG:boto:path=/dsgvcy-15/myobj 2022-09-03T05:26:09.745 DEBUG:boto:auth_path=/dsgvcy-15/myobj 2022-09-03T05:26:09.746 DEBUG:boto:path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.746 DEBUG:boto:auth_path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.747 DEBUG:boto:Method: GET 2022-09-03T05:26:09.748 DEBUG:boto:Path: /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.749 DEBUG:boto:Data: 2022-09-03T05:26:09.750 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.750 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:26:09.751 DEBUG:boto:Port: 8001 2022-09-03T05:26:09.752 DEBUG:boto:Params: {} 2022-09-03T05:26:09.754 DEBUG:boto:Token: None 2022-09-03T05:26:09.755 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.756 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eMbYdqlOlbt8O1d5IV3CCHiDFOw= 2022-09-03T05:26:09.757 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eMbYdqlOlbt8O1d5IV3CCHiDFOw=', 'Content-Length': '0'} 2022-09-03T05:26:09.760 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000009b27f3d74b562d7c-006312e571-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.762 DEBUG:boto:path=/dsgvcy-15/myobj 2022-09-03T05:26:09.763 DEBUG:boto:auth_path=/dsgvcy-15/myobj 2022-09-03T05:26:09.763 DEBUG:boto:path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.767 DEBUG:boto:auth_path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.768 DEBUG:boto:Method: HEAD 2022-09-03T05:26:09.770 DEBUG:boto:Path: /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.771 DEBUG:boto:Data: 2022-09-03T05:26:09.772 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.773 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:26:09.774 DEBUG:boto:Port: 8000 2022-09-03T05:26:09.775 DEBUG:boto:Params: {} 2022-09-03T05:26:09.775 DEBUG:boto:Token: None 2022-09-03T05:26:09.776 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.777 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ctnb7N/c9v4VDGylSEo32gEIN/U= 2022-09-03T05:26:09.778 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ctnb7N/c9v4VDGylSEo32gEIN/U=', 'Content-Length': '0'} 2022-09-03T05:26:09.780 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000faaa8742b886359f-006312e571-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.781 DEBUG:boto:path=/dsgvcy-15/myobj 2022-09-03T05:26:09.782 DEBUG:boto:auth_path=/dsgvcy-15/myobj 2022-09-03T05:26:09.783 DEBUG:boto:path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.783 DEBUG:boto:auth_path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.784 DEBUG:boto:Method: HEAD 2022-09-03T05:26:09.784 DEBUG:boto:Path: /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.785 DEBUG:boto:Data: 2022-09-03T05:26:09.785 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.786 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:26:09.786 DEBUG:boto:Port: 8001 2022-09-03T05:26:09.787 DEBUG:boto:Params: {} 2022-09-03T05:26:09.788 DEBUG:boto:Token: None 2022-09-03T05:26:09.788 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.789 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ctnb7N/c9v4VDGylSEo32gEIN/U= 2022-09-03T05:26:09.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ctnb7N/c9v4VDGylSEo32gEIN/U=', 'Content-Length': '0'} 2022-09-03T05:26:09.792 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000016bdb73da79c0ae6-006312e571-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.793 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:26:09.794 DEBUG:boto:path=/dsgvcy-15/myobj 2022-09-03T05:26:09.795 DEBUG:boto:auth_path=/dsgvcy-15/myobj 2022-09-03T05:26:09.795 DEBUG:boto:path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.796 DEBUG:boto:auth_path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.796 DEBUG:boto:Method: GET 2022-09-03T05:26:09.797 DEBUG:boto:Path: /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.797 DEBUG:boto:Data: 2022-09-03T05:26:09.799 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.800 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:26:09.801 DEBUG:boto:Port: 8000 2022-09-03T05:26:09.802 DEBUG:boto:Params: {} 2022-09-03T05:26:09.803 DEBUG:boto:Token: None 2022-09-03T05:26:09.810 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.811 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eMbYdqlOlbt8O1d5IV3CCHiDFOw= 2022-09-03T05:26:09.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eMbYdqlOlbt8O1d5IV3CCHiDFOw=', 'Content-Length': '0'} 2022-09-03T05:26:09.814 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000bbd1df933015e8fb-006312e571-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.816 DEBUG:boto:path=/dsgvcy-15/myobj 2022-09-03T05:26:09.816 DEBUG:boto:auth_path=/dsgvcy-15/myobj 2022-09-03T05:26:09.817 DEBUG:boto:path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.817 DEBUG:boto:auth_path=/dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.818 DEBUG:boto:Method: GET 2022-09-03T05:26:09.819 DEBUG:boto:Path: /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.820 DEBUG:boto:Data: 2022-09-03T05:26:09.820 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.821 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:26:09.821 DEBUG:boto:Port: 8001 2022-09-03T05:26:09.822 DEBUG:boto:Params: {} 2022-09-03T05:26:09.822 DEBUG:boto:Token: None 2022-09-03T05:26:09.823 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/myobj?versionId=null 2022-09-03T05:26:09.824 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eMbYdqlOlbt8O1d5IV3CCHiDFOw= 2022-09-03T05:26:09.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eMbYdqlOlbt8O1d5IV3CCHiDFOw=', 'Content-Length': '0'} 2022-09-03T05:26:09.827 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000009fee5fae35e06388-006312e571-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.829 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:26:09.830 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:26:09.831 DEBUG:boto:path=/dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.832 DEBUG:boto:auth_path=/dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.833 DEBUG:boto:Method: GET 2022-09-03T05:26:09.833 DEBUG:boto:Path: /dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.834 DEBUG:boto:Data: 2022-09-03T05:26:09.835 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.836 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:26:09.837 DEBUG:boto:Port: 8000 2022-09-03T05:26:09.837 DEBUG:boto:Params: {} 2022-09-03T05:26:09.838 DEBUG:boto:Token: None 2022-09-03T05:26:09.839 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/ 2022-09-03T05:26:09.840 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pdOEBqkdpxMiZmY05F4B2MFKBZY= 2022-09-03T05:26:09.841 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pdOEBqkdpxMiZmY05F4B2MFKBZY=', 'Content-Length': '0'} 2022-09-03T05:26:09.844 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000197de76d4bd249f3-006312e571-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.846 DEBUG:boto:b'dsgvcy-15myobj1000falsemyobj2022-09-03T05:25:46.165Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:26:09.848 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:26:09.848 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:26:09.849 DEBUG:boto:path=/dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.850 DEBUG:boto:auth_path=/dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.851 DEBUG:boto:Method: GET 2022-09-03T05:26:09.852 DEBUG:boto:Path: /dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.853 DEBUG:boto:Data: 2022-09-03T05:26:09.854 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.855 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:26:09.856 DEBUG:boto:Port: 8001 2022-09-03T05:26:09.857 DEBUG:boto:Params: {} 2022-09-03T05:26:09.858 DEBUG:boto:Token: None 2022-09-03T05:26:09.858 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/ 2022-09-03T05:26:09.859 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pdOEBqkdpxMiZmY05F4B2MFKBZY= 2022-09-03T05:26:09.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pdOEBqkdpxMiZmY05F4B2MFKBZY=', 'Content-Length': '0'} 2022-09-03T05:26:09.863 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003494b4d2cb84f739-006312e571-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.863 DEBUG:boto:b'dsgvcy-15myobj1000falsemyobj2022-09-03T05:25:46.165Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:26:09.865 DEBUG:boto:path=/dsgvcy-15/myobj 2022-09-03T05:26:09.866 DEBUG:boto:auth_path=/dsgvcy-15/myobj 2022-09-03T05:26:09.867 DEBUG:boto:Method: HEAD 2022-09-03T05:26:09.868 DEBUG:boto:Path: /dsgvcy-15/myobj 2022-09-03T05:26:09.869 DEBUG:boto:Data: 2022-09-03T05:26:09.869 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.870 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:26:09.871 DEBUG:boto:Port: 8000 2022-09-03T05:26:09.872 DEBUG:boto:Params: {} 2022-09-03T05:26:09.873 DEBUG:boto:Token: None 2022-09-03T05:26:09.873 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/myobj 2022-09-03T05:26:09.874 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YDGOy4MqdDjnjUu9fROiUhIOm28= 2022-09-03T05:26:09.875 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YDGOy4MqdDjnjUu9fROiUhIOm28=', 'Content-Length': '0'} 2022-09-03T05:26:09.877 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000003ebafd94d1d14fb6-006312e571-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.879 DEBUG:boto:path=/dsgvcy-15/myobj 2022-09-03T05:26:09.880 DEBUG:boto:auth_path=/dsgvcy-15/myobj 2022-09-03T05:26:09.881 DEBUG:boto:Method: HEAD 2022-09-03T05:26:09.882 DEBUG:boto:Path: /dsgvcy-15/myobj 2022-09-03T05:26:09.883 DEBUG:boto:Data: 2022-09-03T05:26:09.883 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.884 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:26:09.885 DEBUG:boto:Port: 8001 2022-09-03T05:26:09.886 DEBUG:boto:Params: {} 2022-09-03T05:26:09.887 DEBUG:boto:Token: None 2022-09-03T05:26:09.888 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/myobj 2022-09-03T05:26:09.889 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YDGOy4MqdDjnjUu9fROiUhIOm28= 2022-09-03T05:26:09.890 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YDGOy4MqdDjnjUu9fROiUhIOm28=', 'Content-Length': '0'} 2022-09-03T05:26:09.892 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000c81c893a727bf829-006312e571-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.892 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=myobj 2022-09-03T05:26:09.893 DEBUG:boto:path=/dsgvcy-15/myobj 2022-09-03T05:26:09.894 DEBUG:boto:auth_path=/dsgvcy-15/myobj 2022-09-03T05:26:09.894 DEBUG:boto:Method: GET 2022-09-03T05:26:09.895 DEBUG:boto:Path: /dsgvcy-15/myobj 2022-09-03T05:26:09.895 DEBUG:boto:Data: 2022-09-03T05:26:09.896 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.896 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:26:09.897 DEBUG:boto:Port: 8000 2022-09-03T05:26:09.897 DEBUG:boto:Params: {} 2022-09-03T05:26:09.898 DEBUG:boto:Token: None 2022-09-03T05:26:09.899 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/myobj 2022-09-03T05:26:09.899 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CvkWTh892WxFasmogwwYnLf83EQ= 2022-09-03T05:26:09.900 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CvkWTh892WxFasmogwwYnLf83EQ=', 'Content-Length': '0'} 2022-09-03T05:26:09.902 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000089c615203013d41c-006312e571-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.903 DEBUG:boto:path=/dsgvcy-15/myobj 2022-09-03T05:26:09.903 DEBUG:boto:auth_path=/dsgvcy-15/myobj 2022-09-03T05:26:09.904 DEBUG:boto:Method: GET 2022-09-03T05:26:09.904 DEBUG:boto:Path: /dsgvcy-15/myobj 2022-09-03T05:26:09.905 DEBUG:boto:Data: 2022-09-03T05:26:09.905 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.906 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:26:09.906 DEBUG:boto:Port: 8001 2022-09-03T05:26:09.907 DEBUG:boto:Params: {} 2022-09-03T05:26:09.908 DEBUG:boto:Token: None 2022-09-03T05:26:09.908 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/myobj 2022-09-03T05:26:09.909 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CvkWTh892WxFasmogwwYnLf83EQ= 2022-09-03T05:26:09.909 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CvkWTh892WxFasmogwwYnLf83EQ=', 'Content-Length': '0'} 2022-09-03T05:26:09.911 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx0000044d3962cab037b8e-006312e571-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.954 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:26:09.955 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:26:09.956 DEBUG:boto:path=/dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.956 DEBUG:boto:auth_path=/dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.957 DEBUG:boto:Method: GET 2022-09-03T05:26:09.958 DEBUG:boto:Path: /dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.958 DEBUG:boto:Data: 2022-09-03T05:26:09.959 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.960 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:26:09.961 DEBUG:boto:Port: 8000 2022-09-03T05:26:09.962 DEBUG:boto:Params: {} 2022-09-03T05:26:09.964 DEBUG:boto:Token: None 2022-09-03T05:26:09.965 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/ 2022-09-03T05:26:09.966 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pdOEBqkdpxMiZmY05F4B2MFKBZY= 2022-09-03T05:26:09.967 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pdOEBqkdpxMiZmY05F4B2MFKBZY=', 'Content-Length': '0'} 2022-09-03T05:26:09.970 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001d7e3d071fb8da92-006312e571-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.971 DEBUG:boto:b'dsgvcy-15myobj1000falsemyobj2022-09-03T05:25:46.165Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:26:09.973 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:26:09.974 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:26:09.974 DEBUG:boto:path=/dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.975 DEBUG:boto:auth_path=/dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.975 DEBUG:boto:Method: GET 2022-09-03T05:26:09.976 DEBUG:boto:Path: /dsgvcy-15/?prefix=myobj 2022-09-03T05:26:09.977 DEBUG:boto:Data: 2022-09-03T05:26:09.977 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.978 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:26:09.979 DEBUG:boto:Port: 8001 2022-09-03T05:26:09.980 DEBUG:boto:Params: {} 2022-09-03T05:26:09.981 DEBUG:boto:Token: None 2022-09-03T05:26:09.981 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-15/ 2022-09-03T05:26:09.982 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pdOEBqkdpxMiZmY05F4B2MFKBZY= 2022-09-03T05:26:09.982 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pdOEBqkdpxMiZmY05F4B2MFKBZY=', 'Content-Length': '0'} 2022-09-03T05:26:09.985 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000683a14eb59800014-006312e571-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.986 DEBUG:boto:b'dsgvcy-15myobj1000falsemyobj2022-09-03T05:25:46.165Z"a8f5f167f44f4964e6c998dee827110c"6STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:26:09.987 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-15 zones={test-zone2, test-zone3} 2022-09-03T05:26:09.988 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:26:09.988 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:26:09.989 DEBUG:boto:Method: HEAD 2022-09-03T05:26:09.990 DEBUG:boto:Path: /dsgvcy-13/ 2022-09-03T05:26:09.990 DEBUG:boto:Data: 2022-09-03T05:26:09.991 DEBUG:boto:Headers: {} 2022-09-03T05:26:09.992 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:26:09.992 DEBUG:boto:Port: 8000 2022-09-03T05:26:09.993 DEBUG:boto:Params: {} 2022-09-03T05:26:09.993 DEBUG:boto:Token: None 2022-09-03T05:26:09.994 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:26:09 GMT /dsgvcy-13/ 2022-09-03T05:26:09.995 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:23b2ws754ouRb6/mfPmj6DYCh54= 2022-09-03T05:26:09.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:23b2ws754ouRb6/mfPmj6DYCh54=', 'Content-Length': '0'} 2022-09-03T05:26:09.998 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003494c01b514e53ae-006312e572-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:26:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:09.999 DEBUG:boto:path=/dsgvcy-13/myobj 2022-09-03T05:26:10.000 DEBUG:boto:auth_path=/dsgvcy-13/myobj 2022-09-03T05:26:10.000 DEBUG:boto:Method: HEAD 2022-09-03T05:26:10.001 DEBUG:boto:Path: /dsgvcy-13/myobj 2022-09-03T05:26:10.001 DEBUG:boto:Data: 2022-09-03T05:26:10.002 DEBUG:boto:Headers: {} 2022-09-03T05:26:10.002 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:26:10.003 DEBUG:boto:Port: 8000 2022-09-03T05:26:10.003 DEBUG:boto:Params: {} 2022-09-03T05:26:10.004 DEBUG:boto:Token: None 2022-09-03T05:26:10.005 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:26:10 GMT /dsgvcy-13/myobj 2022-09-03T05:26:10.005 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4msZ+PH06jbrFYeRFuXWls4lZqM= 2022-09-03T05:26:10.006 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4msZ+PH06jbrFYeRFuXWls4lZqM=', 'Content-Length': '0'} 2022-09-03T05:26:10.009 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000007a9aeaa1b6da7ed5-006312e572-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:26:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:10.010 DEBUG:boto:path=/dsgvcy-13/myobj 2022-09-03T05:26:10.010 DEBUG:boto:auth_path=/dsgvcy-13/myobj 2022-09-03T05:26:10.011 DEBUG:boto:Method: DELETE 2022-09-03T05:26:10.012 DEBUG:boto:Path: /dsgvcy-13/myobj 2022-09-03T05:26:10.013 DEBUG:boto:Data: 2022-09-03T05:26:10.013 DEBUG:boto:Headers: {} 2022-09-03T05:26:10.014 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:26:10.014 DEBUG:boto:Port: 8000 2022-09-03T05:26:10.015 DEBUG:boto:Params: {} 2022-09-03T05:26:10.015 DEBUG:boto:Token: None 2022-09-03T05:26:10.016 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:26:10 GMT /dsgvcy-13/myobj 2022-09-03T05:26:10.016 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zXDGKeTXT0CeJcJkBQcRUAjoWZk= 2022-09-03T05:26:10.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zXDGKeTXT0CeJcJkBQcRUAjoWZk=', 'Content-Length': '0'} 2022-09-03T05:26:10.022 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000e1816f2ceb51a1ce-006312e572-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:26:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:10.023 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-13', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:26:10.023 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-13', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:26:10.024 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-13 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:26:10.084 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:26:10.084 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:26:10.124 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:26:10.125+0000 7ff58051e600 1 waiting for incremental sync to catch up: 2022-09-03T05:26:10.125 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.6.5, , , , , , , , , ] 2022-09-03T05:26:10.125 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000002.8.5, , , , , , , , , ] 2022-09-03T05:26:15.125 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:26:15.127+0000 7ff58051e600 1 waiting for incremental sync to catch up: 2022-09-03T05:26:15.126 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.6.5, , , , , , , , , ] 2022-09-03T05:26:15.126 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000002.8.5, , , , , , , , , ] 2022-09-03T05:26:20.126 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:26:20.128+0000 7ff58051e600 1 waiting for incremental sync to catch up: 2022-09-03T05:26:20.126 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.6.5, , , , , , , , , ] 2022-09-03T05:26:20.127 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000002.8.5, , , , , , , , , ] 2022-09-03T05:26:25.127 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:26:25.130+0000 7ff58051e600 1 waiting for incremental sync to catch up: 2022-09-03T05:26:25.128 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.6.5, , , , , , , , , ] 2022-09-03T05:26:25.128 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000002.8.5, , , , , , , , , ] 2022-09-03T05:26:30.129 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:26:30.131+0000 7ff58051e600 1 waiting for incremental sync to catch up: 2022-09-03T05:26:30.131 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.6.5, , , , , , , , , ] 2022-09-03T05:26:30.132 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000002.8.5, , , , , , , , , ] 2022-09-03T05:26:35.130 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:26:35.132+0000 7ff58051e600 1 waiting for incremental sync to catch up: 2022-09-03T05:26:35.131 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.6.5, , , , , , , , , ] 2022-09-03T05:26:35.131 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000002.8.5, , , , , , , , , ] 2022-09-03T05:26:40.132 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:26:40.134+0000 7ff58051e600 1 waiting for incremental sync to catch up: 2022-09-03T05:26:40.133 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.6.5, , , , , , , , , ] 2022-09-03T05:26:40.133 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000002.8.5, , , , , , , , , ] 2022-09-03T05:26:45.133 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:26:45.136+0000 7ff58051e600 1 waiting for incremental sync to catch up: 2022-09-03T05:26:45.134 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.6.5, , , , , , , , , ] 2022-09-03T05:26:45.135 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000002.8.5, , , , , , , , , ] 2022-09-03T05:26:50.135 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:26:50.137+0000 7ff58051e600 1 bucket sync caught up with source: 2022-09-03T05:26:50.135 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000002.8.5, , , , , , , , , ] 2022-09-03T05:26:50.136 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000002.8.5, , , , , , , , , ] 2022-09-03T05:26:50.136 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:26:50.137+0000 7ff58051e600 0 bucket checkpoint complete 2022-09-03T05:26:50.153 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-13 zones={test-zone2, test-zone1} 2022-09-03T05:26:50.155 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:26:50.156 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:26:50.157 DEBUG:boto:Method: HEAD 2022-09-03T05:26:50.158 DEBUG:boto:Path: /dsgvcy-13/ 2022-09-03T05:26:50.158 DEBUG:boto:Data: 2022-09-03T05:26:50.159 DEBUG:boto:Headers: {} 2022-09-03T05:26:50.160 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:26:50.161 DEBUG:boto:Port: 8000 2022-09-03T05:26:50.162 DEBUG:boto:Params: {} 2022-09-03T05:26:50.163 DEBUG:boto:Token: None 2022-09-03T05:26:50.163 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:26:50 GMT /dsgvcy-13/ 2022-09-03T05:26:50.164 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fQI9lHP0IRoXElMVXitVwnJnRcY= 2022-09-03T05:26:50.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fQI9lHP0IRoXElMVXitVwnJnRcY=', 'Content-Length': '0'} 2022-09-03T05:26:50.169 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000094b5753bd773fb6c-006312e59a-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:26:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:50.169 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:26:50.170 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:26:50.171 DEBUG:boto:Method: HEAD 2022-09-03T05:26:50.171 DEBUG:boto:Path: /dsgvcy-13/ 2022-09-03T05:26:50.172 DEBUG:boto:Data: 2022-09-03T05:26:50.172 DEBUG:boto:Headers: {} 2022-09-03T05:26:50.173 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:26:50.173 DEBUG:boto:Port: 8000 2022-09-03T05:26:50.174 DEBUG:boto:Params: {} 2022-09-03T05:26:50.175 DEBUG:boto:Token: None 2022-09-03T05:26:50.175 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:26:50 GMT /dsgvcy-13/ 2022-09-03T05:26:50.177 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fQI9lHP0IRoXElMVXitVwnJnRcY= 2022-09-03T05:26:50.178 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fQI9lHP0IRoXElMVXitVwnJnRcY=', 'Content-Length': '0'} 2022-09-03T05:26:50.181 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000701e9b34924675ec-006312e59a-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:26:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:50.182 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:26:50.183 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:26:50.184 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:26:50.185 DEBUG:boto:path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.185 DEBUG:boto:auth_path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.186 DEBUG:boto:Method: GET 2022-09-03T05:26:50.186 DEBUG:boto:Path: /dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.187 DEBUG:boto:Data: 2022-09-03T05:26:50.187 DEBUG:boto:Headers: {} 2022-09-03T05:26:50.188 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:26:50.188 DEBUG:boto:Port: 8000 2022-09-03T05:26:50.189 DEBUG:boto:Params: {} 2022-09-03T05:26:50.189 DEBUG:boto:Token: None 2022-09-03T05:26:50.190 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:50 GMT /dsgvcy-13/?versions 2022-09-03T05:26:50.191 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YakzUhKipop6ZEknwmGFkObLXU8= 2022-09-03T05:26:50.191 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YakzUhKipop6ZEknwmGFkObLXU8=', 'Content-Length': '0'} 2022-09-03T05:26:50.194 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006b25f40ce4dce8e8-006312e59a-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:50.195 DEBUG:boto:b'dsgvcy-13999false' 2022-09-03T05:26:50.196 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:26:50.196 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:26:50.197 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:26:50.198 DEBUG:boto:path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.198 DEBUG:boto:auth_path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.199 DEBUG:boto:Method: GET 2022-09-03T05:26:50.199 DEBUG:boto:Path: /dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.200 DEBUG:boto:Data: 2022-09-03T05:26:50.200 DEBUG:boto:Headers: {} 2022-09-03T05:26:50.201 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:26:50.201 DEBUG:boto:Port: 8000 2022-09-03T05:26:50.202 DEBUG:boto:Params: {} 2022-09-03T05:26:50.202 DEBUG:boto:Token: None 2022-09-03T05:26:50.203 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:50 GMT /dsgvcy-13/?versions 2022-09-03T05:26:50.203 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YakzUhKipop6ZEknwmGFkObLXU8= 2022-09-03T05:26:50.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YakzUhKipop6ZEknwmGFkObLXU8=', 'Content-Length': '0'} 2022-09-03T05:26:50.207 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f33aeb00f54d3cea-006312e59a-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:50.208 DEBUG:boto:b'dsgvcy-13999false' 2022-09-03T05:26:50.208 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:26:50.209 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:26:50.209 DEBUG:boto:path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.210 DEBUG:boto:auth_path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.211 DEBUG:boto:Method: GET 2022-09-03T05:26:50.211 DEBUG:boto:Path: /dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.212 DEBUG:boto:Data: 2022-09-03T05:26:50.212 DEBUG:boto:Headers: {} 2022-09-03T05:26:50.213 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:26:50.213 DEBUG:boto:Port: 8000 2022-09-03T05:26:50.214 DEBUG:boto:Params: {} 2022-09-03T05:26:50.215 DEBUG:boto:Token: None 2022-09-03T05:26:50.215 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:50 GMT /dsgvcy-13/?versions 2022-09-03T05:26:50.216 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YakzUhKipop6ZEknwmGFkObLXU8= 2022-09-03T05:26:50.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YakzUhKipop6ZEknwmGFkObLXU8=', 'Content-Length': '0'} 2022-09-03T05:26:50.219 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009088f9925f507ba5-006312e59a-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:50.262 DEBUG:boto:b'dsgvcy-13999false' 2022-09-03T05:26:50.264 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:26:50.264 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:26:50.265 DEBUG:boto:path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.265 DEBUG:boto:auth_path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.266 DEBUG:boto:Method: GET 2022-09-03T05:26:50.267 DEBUG:boto:Path: /dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.268 DEBUG:boto:Data: 2022-09-03T05:26:50.269 DEBUG:boto:Headers: {} 2022-09-03T05:26:50.270 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:26:50.271 DEBUG:boto:Port: 8000 2022-09-03T05:26:50.272 DEBUG:boto:Params: {} 2022-09-03T05:26:50.273 DEBUG:boto:Token: None 2022-09-03T05:26:50.274 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:50 GMT /dsgvcy-13/?versions 2022-09-03T05:26:50.275 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YakzUhKipop6ZEknwmGFkObLXU8= 2022-09-03T05:26:50.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YakzUhKipop6ZEknwmGFkObLXU8=', 'Content-Length': '0'} 2022-09-03T05:26:50.287 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000096d60f2ca3a68e9f-006312e59a-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:50.288 DEBUG:boto:b'dsgvcy-13999false' 2022-09-03T05:26:50.289 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-13 zones={test-zone2, test-zone1} 2022-09-03T05:26:50.289 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-13', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:26:50.291 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-13', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:26:50.292 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-13 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:26:50.359 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:26:50.359 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:26:50.410 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:26:50.412+0000 7fe950786600 1 bucket sync caught up with source: 2022-09-03T05:26:50.411 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.8.5, , , , , , , , , ] 2022-09-03T05:26:50.411 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.8.5, , , , , , , , , ] 2022-09-03T05:26:50.412 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:26:50.412+0000 7fe950786600 0 bucket checkpoint complete 2022-09-03T05:26:50.435 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-13 zones={test-zone3, test-zone1} 2022-09-03T05:26:50.437 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:26:50.438 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:26:50.440 DEBUG:boto:Method: HEAD 2022-09-03T05:26:50.441 DEBUG:boto:Path: /dsgvcy-13/ 2022-09-03T05:26:50.443 DEBUG:boto:Data: 2022-09-03T05:26:50.444 DEBUG:boto:Headers: {} 2022-09-03T05:26:50.445 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:26:50.446 DEBUG:boto:Port: 8001 2022-09-03T05:26:50.447 DEBUG:boto:Params: {} 2022-09-03T05:26:50.448 DEBUG:boto:Token: None 2022-09-03T05:26:50.448 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:26:50 GMT /dsgvcy-13/ 2022-09-03T05:26:50.449 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fQI9lHP0IRoXElMVXitVwnJnRcY= 2022-09-03T05:26:50.450 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fQI9lHP0IRoXElMVXitVwnJnRcY=', 'Content-Length': '0'} 2022-09-03T05:26:50.458 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e293a7098d2bdfca-006312e59a-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:26:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:50.460 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:26:50.461 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:26:50.462 DEBUG:boto:Method: HEAD 2022-09-03T05:26:50.463 DEBUG:boto:Path: /dsgvcy-13/ 2022-09-03T05:26:50.464 DEBUG:boto:Data: 2022-09-03T05:26:50.465 DEBUG:boto:Headers: {} 2022-09-03T05:26:50.465 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:26:50.466 DEBUG:boto:Port: 8000 2022-09-03T05:26:50.466 DEBUG:boto:Params: {} 2022-09-03T05:26:50.467 DEBUG:boto:Token: None 2022-09-03T05:26:50.467 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:26:50 GMT /dsgvcy-13/ 2022-09-03T05:26:50.468 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fQI9lHP0IRoXElMVXitVwnJnRcY= 2022-09-03T05:26:50.469 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fQI9lHP0IRoXElMVXitVwnJnRcY=', 'Content-Length': '0'} 2022-09-03T05:26:50.471 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a73b7784455bd0e1-006312e59a-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:26:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:50.471 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:26:50.472 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:26:50.473 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:26:50.473 DEBUG:boto:path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.474 DEBUG:boto:auth_path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.474 DEBUG:boto:Method: GET 2022-09-03T05:26:50.475 DEBUG:boto:Path: /dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.475 DEBUG:boto:Data: 2022-09-03T05:26:50.476 DEBUG:boto:Headers: {} 2022-09-03T05:26:50.476 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:26:50.477 DEBUG:boto:Port: 8001 2022-09-03T05:26:50.478 DEBUG:boto:Params: {} 2022-09-03T05:26:50.479 DEBUG:boto:Token: None 2022-09-03T05:26:50.479 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:50 GMT /dsgvcy-13/?versions 2022-09-03T05:26:50.480 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YakzUhKipop6ZEknwmGFkObLXU8= 2022-09-03T05:26:50.481 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YakzUhKipop6ZEknwmGFkObLXU8=', 'Content-Length': '0'} 2022-09-03T05:26:50.485 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c874fd28cf60454f-006312e59a-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:50.486 DEBUG:boto:b'dsgvcy-13999false' 2022-09-03T05:26:50.487 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:26:50.488 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:26:50.489 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:26:50.489 DEBUG:boto:path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.490 DEBUG:boto:auth_path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.491 DEBUG:boto:Method: GET 2022-09-03T05:26:50.492 DEBUG:boto:Path: /dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.492 DEBUG:boto:Data: 2022-09-03T05:26:50.493 DEBUG:boto:Headers: {} 2022-09-03T05:26:50.494 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:26:50.494 DEBUG:boto:Port: 8000 2022-09-03T05:26:50.495 DEBUG:boto:Params: {} 2022-09-03T05:26:50.495 DEBUG:boto:Token: None 2022-09-03T05:26:50.496 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:50 GMT /dsgvcy-13/?versions 2022-09-03T05:26:50.496 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YakzUhKipop6ZEknwmGFkObLXU8= 2022-09-03T05:26:50.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YakzUhKipop6ZEknwmGFkObLXU8=', 'Content-Length': '0'} 2022-09-03T05:26:50.499 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005a753fe82599d617-006312e59a-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:50.500 DEBUG:boto:b'dsgvcy-13999false' 2022-09-03T05:26:50.502 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:26:50.503 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:26:50.503 DEBUG:boto:path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.504 DEBUG:boto:auth_path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.505 DEBUG:boto:Method: GET 2022-09-03T05:26:50.506 DEBUG:boto:Path: /dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.507 DEBUG:boto:Data: 2022-09-03T05:26:50.508 DEBUG:boto:Headers: {} 2022-09-03T05:26:50.519 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:26:50.521 DEBUG:boto:Port: 8001 2022-09-03T05:26:50.521 DEBUG:boto:Params: {} 2022-09-03T05:26:50.522 DEBUG:boto:Token: None 2022-09-03T05:26:50.523 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:50 GMT /dsgvcy-13/?versions 2022-09-03T05:26:50.523 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YakzUhKipop6ZEknwmGFkObLXU8= 2022-09-03T05:26:50.524 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YakzUhKipop6ZEknwmGFkObLXU8=', 'Content-Length': '0'} 2022-09-03T05:26:50.527 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d3a9400dcab717a5-006312e59a-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:50.572 DEBUG:boto:b'dsgvcy-13999false' 2022-09-03T05:26:50.573 DEBUG:boto:path=/dsgvcy-13/ 2022-09-03T05:26:50.574 DEBUG:boto:auth_path=/dsgvcy-13/ 2022-09-03T05:26:50.575 DEBUG:boto:path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.575 DEBUG:boto:auth_path=/dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.576 DEBUG:boto:Method: GET 2022-09-03T05:26:50.576 DEBUG:boto:Path: /dsgvcy-13/?versions&max-keys=999 2022-09-03T05:26:50.577 DEBUG:boto:Data: 2022-09-03T05:26:50.577 DEBUG:boto:Headers: {} 2022-09-03T05:26:50.578 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:26:50.578 DEBUG:boto:Port: 8000 2022-09-03T05:26:50.579 DEBUG:boto:Params: {} 2022-09-03T05:26:50.580 DEBUG:boto:Token: None 2022-09-03T05:26:50.580 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:26:50 GMT /dsgvcy-13/?versions 2022-09-03T05:26:50.582 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YakzUhKipop6ZEknwmGFkObLXU8= 2022-09-03T05:26:50.583 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YakzUhKipop6ZEknwmGFkObLXU8=', 'Content-Length': '0'} 2022-09-03T05:26:50.585 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cc7ce33c8465073a-006312e59a-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:26:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:50.586 DEBUG:boto:b'dsgvcy-13999false' 2022-09-03T05:26:50.586 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-13 zones={test-zone3, test-zone1} 2022-09-03T05:26:50.587 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-13', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:26:50.588 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-13', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:26:50.588 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-13 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:26:50.654 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:26:50.654 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:26:50.699 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:26:50.701+0000 7f5dfb9ea600 1 bucket sync caught up with source: 2022-09-03T05:26:50.699 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000002.8.5, , , , , , , , , ] 2022-09-03T05:26:50.700 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000002.8.5, , , , , , , , , ] 2022-09-03T05:26:50.700 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:26:50.701+0000 7f5dfb9ea600 0 bucket checkpoint complete 2022-09-03T05:26:50.712 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:26:50.713 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:26:50.714 DEBUG:boto:Method: HEAD 2022-09-03T05:26:50.715 DEBUG:boto:Path: /dsgvcy-14/ 2022-09-03T05:26:50.716 DEBUG:boto:Data: 2022-09-03T05:26:50.717 DEBUG:boto:Headers: {} 2022-09-03T05:26:50.719 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:26:50.719 DEBUG:boto:Port: 8000 2022-09-03T05:26:50.720 DEBUG:boto:Params: {} 2022-09-03T05:26:50.720 DEBUG:boto:Token: None 2022-09-03T05:26:50.721 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:26:50 GMT /dsgvcy-14/ 2022-09-03T05:26:50.722 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vd8v42IVHKtbmT2K3fo1gi/JHiY= 2022-09-03T05:26:50.722 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vd8v42IVHKtbmT2K3fo1gi/JHiY=', 'Content-Length': '0'} 2022-09-03T05:26:50.725 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000955a6ce5f1df9139-006312e59a-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:26:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:50.726 DEBUG:boto:path=/dsgvcy-14/myobj 2022-09-03T05:26:50.727 DEBUG:boto:auth_path=/dsgvcy-14/myobj 2022-09-03T05:26:50.728 DEBUG:boto:Method: HEAD 2022-09-03T05:26:50.729 DEBUG:boto:Path: /dsgvcy-14/myobj 2022-09-03T05:26:50.730 DEBUG:boto:Data: 2022-09-03T05:26:50.730 DEBUG:boto:Headers: {} 2022-09-03T05:26:50.731 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:26:50.731 DEBUG:boto:Port: 8000 2022-09-03T05:26:50.732 DEBUG:boto:Params: {} 2022-09-03T05:26:50.733 DEBUG:boto:Token: None 2022-09-03T05:26:50.734 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:26:50 GMT /dsgvcy-14/myobj 2022-09-03T05:26:50.736 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NfHpraVnU5PAuZ3t6anpvJoBrDY= 2022-09-03T05:26:50.737 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NfHpraVnU5PAuZ3t6anpvJoBrDY=', 'Content-Length': '0'} 2022-09-03T05:26:50.739 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx000004cd6e015aac91c20-006312e59a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:26:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:50.741 DEBUG:boto:path=/dsgvcy-14/myobj 2022-09-03T05:26:50.742 DEBUG:boto:auth_path=/dsgvcy-14/myobj 2022-09-03T05:26:50.743 DEBUG:boto:Method: DELETE 2022-09-03T05:26:50.744 DEBUG:boto:Path: /dsgvcy-14/myobj 2022-09-03T05:26:50.745 DEBUG:boto:Data: 2022-09-03T05:26:50.746 DEBUG:boto:Headers: {} 2022-09-03T05:26:50.747 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:26:50.748 DEBUG:boto:Port: 8000 2022-09-03T05:26:50.749 DEBUG:boto:Params: {} 2022-09-03T05:26:50.750 DEBUG:boto:Token: None 2022-09-03T05:26:50.751 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:26:50 GMT /dsgvcy-14/myobj 2022-09-03T05:26:50.752 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9BqyD12HsW+8i1lfxRR1xiE60Vw= 2022-09-03T05:26:50.753 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:26:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9BqyD12HsW+8i1lfxRR1xiE60Vw=', 'Content-Length': '0'} 2022-09-03T05:26:50.759 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000f6dde9a6eab9cea1-006312e59a-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:26:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:26:50.760 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-14', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:26:50.761 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-14', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:26:50.762 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-14 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:26:50.821 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:26:50.821 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:26:50.883 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:26:50.883+0000 7fce4c1a8600 1 waiting for incremental sync to catch up: 2022-09-03T05:26:50.884 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.6.5, , , , , , , , , ] 2022-09-03T05:26:50.885 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.9.5, , , , , , , , , ] 2022-09-03T05:26:55.883 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:26:55.886+0000 7fce4c1a8600 1 waiting for incremental sync to catch up: 2022-09-03T05:26:55.883 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.6.5, , , , , , , , , ] 2022-09-03T05:26:55.884 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.9.5, , , , , , , , , ] 2022-09-03T05:27:00.884 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:27:00.887+0000 7fce4c1a8600 1 waiting for incremental sync to catch up: 2022-09-03T05:27:00.885 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.6.5, , , , , , , , , ] 2022-09-03T05:27:00.885 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.9.5, , , , , , , , , ] 2022-09-03T05:27:05.887 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:27:05.889+0000 7fce4c1a8600 1 waiting for incremental sync to catch up: 2022-09-03T05:27:05.888 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.6.5, , , , , , , , , ] 2022-09-03T05:27:05.888 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.9.5, , , , , , , , , ] 2022-09-03T05:27:10.893 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:27:10.891+0000 7fce4c1a8600 1 bucket sync caught up with source: 2022-09-03T05:27:10.894 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.9.5, , , , , , , , , ] 2022-09-03T05:27:10.894 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.9.5, , , , , , , , , ] 2022-09-03T05:27:10.895 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:27:10.891+0000 7fce4c1a8600 0 bucket checkpoint complete 2022-09-03T05:27:10.909 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-14 zones={test-zone1, test-zone2} 2022-09-03T05:27:10.910 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:27:10.911 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:27:10.912 DEBUG:boto:Method: HEAD 2022-09-03T05:27:10.913 DEBUG:boto:Path: /dsgvcy-14/ 2022-09-03T05:27:10.914 DEBUG:boto:Data: 2022-09-03T05:27:10.915 DEBUG:boto:Headers: {} 2022-09-03T05:27:10.916 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:27:10.917 DEBUG:boto:Port: 8000 2022-09-03T05:27:10.918 DEBUG:boto:Params: {} 2022-09-03T05:27:10.919 DEBUG:boto:Token: None 2022-09-03T05:27:10.920 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:27:10 GMT /dsgvcy-14/ 2022-09-03T05:27:10.921 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q8vxxuBPQ42VwlNRh6hJqe65ZGQ= 2022-09-03T05:27:10.925 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q8vxxuBPQ42VwlNRh6hJqe65ZGQ=', 'Content-Length': '0'} 2022-09-03T05:27:10.935 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000508d8201b26fb227-006312e5ae-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:27:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:10.936 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:27:10.936 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:27:10.937 DEBUG:boto:Method: HEAD 2022-09-03T05:27:10.937 DEBUG:boto:Path: /dsgvcy-14/ 2022-09-03T05:27:10.941 DEBUG:boto:Data: 2022-09-03T05:27:10.942 DEBUG:boto:Headers: {} 2022-09-03T05:27:10.943 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:27:10.944 DEBUG:boto:Port: 8000 2022-09-03T05:27:10.945 DEBUG:boto:Params: {} 2022-09-03T05:27:10.946 DEBUG:boto:Token: None 2022-09-03T05:27:10.946 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:27:10 GMT /dsgvcy-14/ 2022-09-03T05:27:10.947 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q8vxxuBPQ42VwlNRh6hJqe65ZGQ= 2022-09-03T05:27:10.947 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q8vxxuBPQ42VwlNRh6hJqe65ZGQ=', 'Content-Length': '0'} 2022-09-03T05:27:10.950 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005f482716f6dd79fa-006312e5ae-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:27:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:10.951 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:27:10.952 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:27:10.952 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:27:10.953 DEBUG:boto:path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:10.954 DEBUG:boto:auth_path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:10.955 DEBUG:boto:Method: GET 2022-09-03T05:27:10.955 DEBUG:boto:Path: /dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:10.956 DEBUG:boto:Data: 2022-09-03T05:27:10.956 DEBUG:boto:Headers: {} 2022-09-03T05:27:10.957 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:27:10.957 DEBUG:boto:Port: 8000 2022-09-03T05:27:10.958 DEBUG:boto:Params: {} 2022-09-03T05:27:10.959 DEBUG:boto:Token: None 2022-09-03T05:27:10.959 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:27:10 GMT /dsgvcy-14/?versions 2022-09-03T05:27:10.960 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+f/NkEkqtYJB5ZP5PzU64Ow7A8c= 2022-09-03T05:27:10.961 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+f/NkEkqtYJB5ZP5PzU64Ow7A8c=', 'Content-Length': '0'} 2022-09-03T05:27:10.964 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000052bfeca82198cfd-006312e5ae-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:27:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:10.965 DEBUG:boto:b'dsgvcy-14999false' 2022-09-03T05:27:10.966 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:27:10.966 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:27:10.967 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:27:10.967 DEBUG:boto:path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:10.968 DEBUG:boto:auth_path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:10.968 DEBUG:boto:Method: GET 2022-09-03T05:27:10.969 DEBUG:boto:Path: /dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:10.970 DEBUG:boto:Data: 2022-09-03T05:27:10.971 DEBUG:boto:Headers: {} 2022-09-03T05:27:10.972 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:27:10.973 DEBUG:boto:Port: 8000 2022-09-03T05:27:10.974 DEBUG:boto:Params: {} 2022-09-03T05:27:10.976 DEBUG:boto:Token: None 2022-09-03T05:27:10.977 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:27:10 GMT /dsgvcy-14/?versions 2022-09-03T05:27:10.979 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+f/NkEkqtYJB5ZP5PzU64Ow7A8c= 2022-09-03T05:27:10.980 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+f/NkEkqtYJB5ZP5PzU64Ow7A8c=', 'Content-Length': '0'} 2022-09-03T05:27:10.988 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b75b7667ce0a3b54-006312e5ae-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:27:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:10.989 DEBUG:boto:b'dsgvcy-14999false' 2022-09-03T05:27:10.990 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:27:10.991 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:27:10.992 DEBUG:boto:path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:10.993 DEBUG:boto:auth_path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:10.993 DEBUG:boto:Method: GET 2022-09-03T05:27:10.994 DEBUG:boto:Path: /dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:10.995 DEBUG:boto:Data: 2022-09-03T05:27:10.995 DEBUG:boto:Headers: {} 2022-09-03T05:27:10.996 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:27:10.996 DEBUG:boto:Port: 8000 2022-09-03T05:27:10.997 DEBUG:boto:Params: {} 2022-09-03T05:27:10.997 DEBUG:boto:Token: None 2022-09-03T05:27:10.998 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:27:10 GMT /dsgvcy-14/?versions 2022-09-03T05:27:10.998 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+f/NkEkqtYJB5ZP5PzU64Ow7A8c= 2022-09-03T05:27:10.999 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+f/NkEkqtYJB5ZP5PzU64Ow7A8c=', 'Content-Length': '0'} 2022-09-03T05:27:11.004 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000047588e77f081e677-006312e5af-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:27:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:11.050 DEBUG:boto:b'dsgvcy-14999false' 2022-09-03T05:27:11.052 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:27:11.052 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:27:11.053 DEBUG:boto:path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:11.054 DEBUG:boto:auth_path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:11.055 DEBUG:boto:Method: GET 2022-09-03T05:27:11.056 DEBUG:boto:Path: /dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:11.058 DEBUG:boto:Data: 2022-09-03T05:27:11.058 DEBUG:boto:Headers: {} 2022-09-03T05:27:11.059 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:27:11.059 DEBUG:boto:Port: 8000 2022-09-03T05:27:11.060 DEBUG:boto:Params: {} 2022-09-03T05:27:11.060 DEBUG:boto:Token: None 2022-09-03T05:27:11.061 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:27:11 GMT /dsgvcy-14/?versions 2022-09-03T05:27:11.062 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KM1Xo/emUDlpBSofWXy2wTS/Q9A= 2022-09-03T05:27:11.062 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KM1Xo/emUDlpBSofWXy2wTS/Q9A=', 'Content-Length': '0'} 2022-09-03T05:27:11.065 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007b4ab0eb5c216d48-006312e5af-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:27:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:11.067 DEBUG:boto:b'dsgvcy-14999false' 2022-09-03T05:27:11.068 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-14 zones={test-zone1, test-zone2} 2022-09-03T05:27:11.068 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-14', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:27:11.069 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-14', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:27:11.069 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-14 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:27:11.146 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:27:11.147 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:27:11.197 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:27:11.200+0000 7f0828118600 1 bucket sync caught up with source: 2022-09-03T05:27:11.197 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.9.5, , , , , , , , , ] 2022-09-03T05:27:11.198 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.9.5, , , , , , , , , ] 2022-09-03T05:27:11.198 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:27:11.200+0000 7f0828118600 0 bucket checkpoint complete 2022-09-03T05:27:11.212 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-14 zones={test-zone3, test-zone2} 2022-09-03T05:27:11.213 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:27:11.213 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:27:11.214 DEBUG:boto:Method: HEAD 2022-09-03T05:27:11.214 DEBUG:boto:Path: /dsgvcy-14/ 2022-09-03T05:27:11.215 DEBUG:boto:Data: 2022-09-03T05:27:11.216 DEBUG:boto:Headers: {} 2022-09-03T05:27:11.216 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:27:11.217 DEBUG:boto:Port: 8001 2022-09-03T05:27:11.217 DEBUG:boto:Params: {} 2022-09-03T05:27:11.218 DEBUG:boto:Token: None 2022-09-03T05:27:11.219 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:27:11 GMT /dsgvcy-14/ 2022-09-03T05:27:11.219 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BhN+vZOOrUmydOb6FfoiprFi8wY= 2022-09-03T05:27:11.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BhN+vZOOrUmydOb6FfoiprFi8wY=', 'Content-Length': '0'} 2022-09-03T05:27:11.223 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000defa903e2c4122c6-006312e5af-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:27:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:11.224 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:27:11.225 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:27:11.225 DEBUG:boto:Method: HEAD 2022-09-03T05:27:11.226 DEBUG:boto:Path: /dsgvcy-14/ 2022-09-03T05:27:11.227 DEBUG:boto:Data: 2022-09-03T05:27:11.227 DEBUG:boto:Headers: {} 2022-09-03T05:27:11.228 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:27:11.228 DEBUG:boto:Port: 8000 2022-09-03T05:27:11.229 DEBUG:boto:Params: {} 2022-09-03T05:27:11.229 DEBUG:boto:Token: None 2022-09-03T05:27:11.230 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:27:11 GMT /dsgvcy-14/ 2022-09-03T05:27:11.231 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BhN+vZOOrUmydOb6FfoiprFi8wY= 2022-09-03T05:27:11.231 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BhN+vZOOrUmydOb6FfoiprFi8wY=', 'Content-Length': '0'} 2022-09-03T05:27:11.235 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000043f456432cc684aa-006312e5af-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:27:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:11.237 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:27:11.237 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:27:11.238 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:27:11.239 DEBUG:boto:path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:11.239 DEBUG:boto:auth_path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:11.240 DEBUG:boto:Method: GET 2022-09-03T05:27:11.241 DEBUG:boto:Path: /dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:11.241 DEBUG:boto:Data: 2022-09-03T05:27:11.242 DEBUG:boto:Headers: {} 2022-09-03T05:27:11.243 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:27:11.244 DEBUG:boto:Port: 8001 2022-09-03T05:27:11.246 DEBUG:boto:Params: {} 2022-09-03T05:27:11.247 DEBUG:boto:Token: None 2022-09-03T05:27:11.247 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:27:11 GMT /dsgvcy-14/?versions 2022-09-03T05:27:11.248 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KM1Xo/emUDlpBSofWXy2wTS/Q9A= 2022-09-03T05:27:11.249 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KM1Xo/emUDlpBSofWXy2wTS/Q9A=', 'Content-Length': '0'} 2022-09-03T05:27:11.254 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000431642a6488b8164-006312e5af-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:27:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:11.255 DEBUG:boto:b'dsgvcy-14999false' 2022-09-03T05:27:11.256 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:27:11.256 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:27:11.257 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:27:11.257 DEBUG:boto:path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:11.258 DEBUG:boto:auth_path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:11.259 DEBUG:boto:Method: GET 2022-09-03T05:27:11.259 DEBUG:boto:Path: /dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:11.260 DEBUG:boto:Data: 2022-09-03T05:27:11.260 DEBUG:boto:Headers: {} 2022-09-03T05:27:11.261 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:27:11.261 DEBUG:boto:Port: 8000 2022-09-03T05:27:11.262 DEBUG:boto:Params: {} 2022-09-03T05:27:11.263 DEBUG:boto:Token: None 2022-09-03T05:27:11.264 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:27:11 GMT /dsgvcy-14/?versions 2022-09-03T05:27:11.264 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KM1Xo/emUDlpBSofWXy2wTS/Q9A= 2022-09-03T05:27:11.265 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KM1Xo/emUDlpBSofWXy2wTS/Q9A=', 'Content-Length': '0'} 2022-09-03T05:27:11.268 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000defb2d7afce48bc3-006312e5af-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:27:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:11.268 DEBUG:boto:b'dsgvcy-14999false' 2022-09-03T05:27:11.269 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:27:11.270 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:27:11.270 DEBUG:boto:path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:11.271 DEBUG:boto:auth_path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:11.271 DEBUG:boto:Method: GET 2022-09-03T05:27:11.272 DEBUG:boto:Path: /dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:11.273 DEBUG:boto:Data: 2022-09-03T05:27:11.273 DEBUG:boto:Headers: {} 2022-09-03T05:27:11.274 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:27:11.274 DEBUG:boto:Port: 8001 2022-09-03T05:27:11.275 DEBUG:boto:Params: {} 2022-09-03T05:27:11.275 DEBUG:boto:Token: None 2022-09-03T05:27:11.276 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:27:11 GMT /dsgvcy-14/?versions 2022-09-03T05:27:11.277 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KM1Xo/emUDlpBSofWXy2wTS/Q9A= 2022-09-03T05:27:11.277 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KM1Xo/emUDlpBSofWXy2wTS/Q9A=', 'Content-Length': '0'} 2022-09-03T05:27:11.280 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000079cf829565d004e7-006312e5af-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:27:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:11.280 DEBUG:boto:b'dsgvcy-14999false' 2022-09-03T05:27:11.281 DEBUG:boto:path=/dsgvcy-14/ 2022-09-03T05:27:11.282 DEBUG:boto:auth_path=/dsgvcy-14/ 2022-09-03T05:27:11.282 DEBUG:boto:path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:11.283 DEBUG:boto:auth_path=/dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:11.284 DEBUG:boto:Method: GET 2022-09-03T05:27:11.284 DEBUG:boto:Path: /dsgvcy-14/?versions&max-keys=999 2022-09-03T05:27:11.285 DEBUG:boto:Data: 2022-09-03T05:27:11.285 DEBUG:boto:Headers: {} 2022-09-03T05:27:11.286 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:27:11.286 DEBUG:boto:Port: 8000 2022-09-03T05:27:11.287 DEBUG:boto:Params: {} 2022-09-03T05:27:11.287 DEBUG:boto:Token: None 2022-09-03T05:27:11.288 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:27:11 GMT /dsgvcy-14/?versions 2022-09-03T05:27:11.289 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KM1Xo/emUDlpBSofWXy2wTS/Q9A= 2022-09-03T05:27:11.289 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KM1Xo/emUDlpBSofWXy2wTS/Q9A=', 'Content-Length': '0'} 2022-09-03T05:27:11.292 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000005fa43744adc234f-006312e5af-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:27:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:11.293 DEBUG:boto:b'dsgvcy-14999false' 2022-09-03T05:27:11.294 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-14 zones={test-zone3, test-zone2} 2022-09-03T05:27:11.294 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:27:11.295 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:27:11.295 DEBUG:boto:Method: HEAD 2022-09-03T05:27:11.296 DEBUG:boto:Path: /dsgvcy-15/ 2022-09-03T05:27:11.296 DEBUG:boto:Data: 2022-09-03T05:27:11.297 DEBUG:boto:Headers: {} 2022-09-03T05:27:11.297 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:27:11.298 DEBUG:boto:Port: 8001 2022-09-03T05:27:11.298 DEBUG:boto:Params: {} 2022-09-03T05:27:11.299 DEBUG:boto:Token: None 2022-09-03T05:27:11.300 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:27:11 GMT /dsgvcy-15/ 2022-09-03T05:27:11.301 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DwoAZTe5f9ei/glwREzu4C3d3dk= 2022-09-03T05:27:11.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DwoAZTe5f9ei/glwREzu4C3d3dk=', 'Content-Length': '0'} 2022-09-03T05:27:11.306 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000548f95ec6b3ab932-006312e5af-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:27:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:11.307 DEBUG:boto:path=/dsgvcy-15/myobj 2022-09-03T05:27:11.307 DEBUG:boto:auth_path=/dsgvcy-15/myobj 2022-09-03T05:27:11.308 DEBUG:boto:Method: HEAD 2022-09-03T05:27:11.308 DEBUG:boto:Path: /dsgvcy-15/myobj 2022-09-03T05:27:11.309 DEBUG:boto:Data: 2022-09-03T05:27:11.309 DEBUG:boto:Headers: {} 2022-09-03T05:27:11.310 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:27:11.310 DEBUG:boto:Port: 8001 2022-09-03T05:27:11.311 DEBUG:boto:Params: {} 2022-09-03T05:27:11.311 DEBUG:boto:Token: None 2022-09-03T05:27:11.312 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:27:11 GMT /dsgvcy-15/myobj 2022-09-03T05:27:11.312 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4Myee3gfPy1W5QqnBppxVI2XAiY= 2022-09-03T05:27:11.313 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4Myee3gfPy1W5QqnBppxVI2XAiY=', 'Content-Length': '0'} 2022-09-03T05:27:11.321 DEBUG:boto:Response headers: [('Content-Length', '6'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:25:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('x-amz-request-id', 'tx00000e11c3bc184c10379-006312e5af-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:27:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:11.322 DEBUG:boto:path=/dsgvcy-15/myobj 2022-09-03T05:27:11.323 DEBUG:boto:auth_path=/dsgvcy-15/myobj 2022-09-03T05:27:11.324 DEBUG:boto:Method: DELETE 2022-09-03T05:27:11.324 DEBUG:boto:Path: /dsgvcy-15/myobj 2022-09-03T05:27:11.325 DEBUG:boto:Data: 2022-09-03T05:27:11.325 DEBUG:boto:Headers: {} 2022-09-03T05:27:11.326 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:27:11.326 DEBUG:boto:Port: 8001 2022-09-03T05:27:11.327 DEBUG:boto:Params: {} 2022-09-03T05:27:11.327 DEBUG:boto:Token: None 2022-09-03T05:27:11.328 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:27:11 GMT /dsgvcy-15/myobj 2022-09-03T05:27:11.328 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QmrUhPqNUmOgURzwJp2kVZbV31M= 2022-09-03T05:27:11.329 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QmrUhPqNUmOgURzwJp2kVZbV31M=', 'Content-Length': '0'} 2022-09-03T05:27:11.335 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000004b924fa9b712667-006312e5af-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:27:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:11.337 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-15', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:27:11.338 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-15', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:27:11.338 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-15 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:27:11.395 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:27:11.396 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:27:11.430 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:27:11.431+0000 7f55afa01600 1 waiting for incremental sync to catch up: 2022-09-03T05:27:11.431 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.39.5, , , , , , , , , ] 2022-09-03T05:27:11.431 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.49.5, , , , , , , , , ] 2022-09-03T05:27:16.429 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:27:16.432+0000 7f55afa01600 1 waiting for incremental sync to catch up: 2022-09-03T05:27:16.430 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.39.5, , , , , , , , , ] 2022-09-03T05:27:16.431 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.49.5, , , , , , , , , ] 2022-09-03T05:27:21.431 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:27:21.433+0000 7f55afa01600 1 waiting for incremental sync to catch up: 2022-09-03T05:27:21.432 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.39.5, , , , , , , , , ] 2022-09-03T05:27:21.432 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.49.5, , , , , , , , , ] 2022-09-03T05:27:26.432 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:27:26.434+0000 7f55afa01600 1 bucket sync caught up with source: 2022-09-03T05:27:26.433 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.49.5, , , , , , , , , ] 2022-09-03T05:27:26.433 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.49.5, , , , , , , , , ] 2022-09-03T05:27:26.434 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:27:26.434+0000 7f55afa01600 0 bucket checkpoint complete 2022-09-03T05:27:26.455 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-15 zones={test-zone1, test-zone3} 2022-09-03T05:27:26.456 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:27:26.456 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:27:26.457 DEBUG:boto:Method: HEAD 2022-09-03T05:27:26.457 DEBUG:boto:Path: /dsgvcy-15/ 2022-09-03T05:27:26.458 DEBUG:boto:Data: 2022-09-03T05:27:26.459 DEBUG:boto:Headers: {} 2022-09-03T05:27:26.460 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:27:26.461 DEBUG:boto:Port: 8000 2022-09-03T05:27:26.461 DEBUG:boto:Params: {} 2022-09-03T05:27:26.462 DEBUG:boto:Token: None 2022-09-03T05:27:26.463 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:27:26 GMT /dsgvcy-15/ 2022-09-03T05:27:26.463 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:745lw8HybUXvQGLf/yBKq9nyj+4= 2022-09-03T05:27:26.465 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:745lw8HybUXvQGLf/yBKq9nyj+4=', 'Content-Length': '0'} 2022-09-03T05:27:26.469 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d3faac270c912232-006312e5be-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:27:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:26.470 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:27:26.471 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:27:26.472 DEBUG:boto:Method: HEAD 2022-09-03T05:27:26.472 DEBUG:boto:Path: /dsgvcy-15/ 2022-09-03T05:27:26.473 DEBUG:boto:Data: 2022-09-03T05:27:26.473 DEBUG:boto:Headers: {} 2022-09-03T05:27:26.474 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:27:26.475 DEBUG:boto:Port: 8001 2022-09-03T05:27:26.475 DEBUG:boto:Params: {} 2022-09-03T05:27:26.476 DEBUG:boto:Token: None 2022-09-03T05:27:26.477 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:27:26 GMT /dsgvcy-15/ 2022-09-03T05:27:26.477 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:745lw8HybUXvQGLf/yBKq9nyj+4= 2022-09-03T05:27:26.478 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:745lw8HybUXvQGLf/yBKq9nyj+4=', 'Content-Length': '0'} 2022-09-03T05:27:26.481 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f0c95841a260d08d-006312e5be-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:27:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:26.482 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:27:26.483 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:27:26.484 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:27:26.485 DEBUG:boto:path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:26.486 DEBUG:boto:auth_path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:26.487 DEBUG:boto:Method: GET 2022-09-03T05:27:26.487 DEBUG:boto:Path: /dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:26.488 DEBUG:boto:Data: 2022-09-03T05:27:26.489 DEBUG:boto:Headers: {} 2022-09-03T05:27:26.490 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:27:26.491 DEBUG:boto:Port: 8000 2022-09-03T05:27:26.492 DEBUG:boto:Params: {} 2022-09-03T05:27:26.493 DEBUG:boto:Token: None 2022-09-03T05:27:26.494 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:27:26 GMT /dsgvcy-15/?versions 2022-09-03T05:27:26.495 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V0IyK6YncgchUn5wfTxhuNpwgIk= 2022-09-03T05:27:26.496 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V0IyK6YncgchUn5wfTxhuNpwgIk=', 'Content-Length': '0'} 2022-09-03T05:27:26.500 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f848b4807965bf74-006312e5be-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:27:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:26.501 DEBUG:boto:b'dsgvcy-15999false' 2022-09-03T05:27:26.502 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:27:26.503 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:27:26.504 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:27:26.505 DEBUG:boto:path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:26.506 DEBUG:boto:auth_path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:26.507 DEBUG:boto:Method: GET 2022-09-03T05:27:26.508 DEBUG:boto:Path: /dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:26.509 DEBUG:boto:Data: 2022-09-03T05:27:26.509 DEBUG:boto:Headers: {} 2022-09-03T05:27:26.510 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:27:26.511 DEBUG:boto:Port: 8001 2022-09-03T05:27:26.512 DEBUG:boto:Params: {} 2022-09-03T05:27:26.513 DEBUG:boto:Token: None 2022-09-03T05:27:26.514 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:27:26 GMT /dsgvcy-15/?versions 2022-09-03T05:27:26.515 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V0IyK6YncgchUn5wfTxhuNpwgIk= 2022-09-03T05:27:26.515 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V0IyK6YncgchUn5wfTxhuNpwgIk=', 'Content-Length': '0'} 2022-09-03T05:27:26.518 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c854473b08d32607-006312e5be-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:27:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:26.519 DEBUG:boto:b'dsgvcy-15999false' 2022-09-03T05:27:26.520 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:27:26.520 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:27:26.521 DEBUG:boto:path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:26.521 DEBUG:boto:auth_path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:26.522 DEBUG:boto:Method: GET 2022-09-03T05:27:26.522 DEBUG:boto:Path: /dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:26.523 DEBUG:boto:Data: 2022-09-03T05:27:26.524 DEBUG:boto:Headers: {} 2022-09-03T05:27:26.524 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:27:26.525 DEBUG:boto:Port: 8000 2022-09-03T05:27:26.525 DEBUG:boto:Params: {} 2022-09-03T05:27:26.526 DEBUG:boto:Token: None 2022-09-03T05:27:26.527 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:27:26 GMT /dsgvcy-15/?versions 2022-09-03T05:27:26.527 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V0IyK6YncgchUn5wfTxhuNpwgIk= 2022-09-03T05:27:26.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V0IyK6YncgchUn5wfTxhuNpwgIk=', 'Content-Length': '0'} 2022-09-03T05:27:26.530 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e61139f5d1a71f96-006312e5be-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:27:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:26.532 DEBUG:boto:b'dsgvcy-15999false' 2022-09-03T05:27:26.533 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:27:26.533 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:27:26.534 DEBUG:boto:path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:26.534 DEBUG:boto:auth_path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:26.535 DEBUG:boto:Method: GET 2022-09-03T05:27:26.535 DEBUG:boto:Path: /dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:26.536 DEBUG:boto:Data: 2022-09-03T05:27:26.536 DEBUG:boto:Headers: {} 2022-09-03T05:27:26.537 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:27:26.537 DEBUG:boto:Port: 8001 2022-09-03T05:27:26.538 DEBUG:boto:Params: {} 2022-09-03T05:27:26.538 DEBUG:boto:Token: None 2022-09-03T05:27:26.539 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:27:26 GMT /dsgvcy-15/?versions 2022-09-03T05:27:26.540 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V0IyK6YncgchUn5wfTxhuNpwgIk= 2022-09-03T05:27:26.541 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V0IyK6YncgchUn5wfTxhuNpwgIk=', 'Content-Length': '0'} 2022-09-03T05:27:26.543 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ac45d80f09c2df3b-006312e5be-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:27:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:26.544 DEBUG:boto:b'dsgvcy-15999false' 2022-09-03T05:27:26.545 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-15 zones={test-zone1, test-zone3} 2022-09-03T05:27:26.546 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-15', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:27:26.546 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-15', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:27:26.547 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-15 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:27:26.610 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:27:26.611 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:27:26.654 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:27:26.656+0000 7ff0867e7600 1 waiting for incremental sync to catch up: 2022-09-03T05:27:26.655 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.39.5, , , , , , , , , ] 2022-09-03T05:27:26.655 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000002.49.5, , , , , , , , , ] 2022-09-03T05:27:31.657 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:27:31.659+0000 7ff0867e7600 1 bucket sync caught up with source: 2022-09-03T05:27:31.659 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000002.49.5, , , , , , , , , ] 2022-09-03T05:27:31.659 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000002.49.5, , , , , , , , , ] 2022-09-03T05:27:31.660 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:27:31.659+0000 7ff0867e7600 0 bucket checkpoint complete 2022-09-03T05:27:31.676 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-15 zones={test-zone2, test-zone3} 2022-09-03T05:27:31.676 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:27:31.677 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:27:31.679 DEBUG:boto:Method: HEAD 2022-09-03T05:27:31.680 DEBUG:boto:Path: /dsgvcy-15/ 2022-09-03T05:27:31.681 DEBUG:boto:Data: 2022-09-03T05:27:31.682 DEBUG:boto:Headers: {} 2022-09-03T05:27:31.683 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:27:31.684 DEBUG:boto:Port: 8000 2022-09-03T05:27:31.684 DEBUG:boto:Params: {} 2022-09-03T05:27:31.685 DEBUG:boto:Token: None 2022-09-03T05:27:31.686 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:27:31 GMT /dsgvcy-15/ 2022-09-03T05:27:31.686 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9MdXm3Z3Hwy6fYm5nTmgjcW/4MA= 2022-09-03T05:27:31.687 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9MdXm3Z3Hwy6fYm5nTmgjcW/4MA=', 'Content-Length': '0'} 2022-09-03T05:27:31.691 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000004c4552858545c19-006312e5c3-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:27:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:31.692 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:27:31.693 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:27:31.694 DEBUG:boto:Method: HEAD 2022-09-03T05:27:31.695 DEBUG:boto:Path: /dsgvcy-15/ 2022-09-03T05:27:31.695 DEBUG:boto:Data: 2022-09-03T05:27:31.696 DEBUG:boto:Headers: {} 2022-09-03T05:27:31.697 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:27:31.697 DEBUG:boto:Port: 8001 2022-09-03T05:27:31.698 DEBUG:boto:Params: {} 2022-09-03T05:27:31.699 DEBUG:boto:Token: None 2022-09-03T05:27:31.701 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:27:31 GMT /dsgvcy-15/ 2022-09-03T05:27:31.701 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9MdXm3Z3Hwy6fYm5nTmgjcW/4MA= 2022-09-03T05:27:31.702 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9MdXm3Z3Hwy6fYm5nTmgjcW/4MA=', 'Content-Length': '0'} 2022-09-03T05:27:31.706 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007b62049b52b2ae40-006312e5c3-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:27:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:31.708 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:27:31.709 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:27:31.711 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:27:31.712 DEBUG:boto:path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:31.713 DEBUG:boto:auth_path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:31.714 DEBUG:boto:Method: GET 2022-09-03T05:27:31.715 DEBUG:boto:Path: /dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:31.716 DEBUG:boto:Data: 2022-09-03T05:27:31.717 DEBUG:boto:Headers: {} 2022-09-03T05:27:31.719 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:27:31.720 DEBUG:boto:Port: 8000 2022-09-03T05:27:31.721 DEBUG:boto:Params: {} 2022-09-03T05:27:31.722 DEBUG:boto:Token: None 2022-09-03T05:27:31.723 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:27:31 GMT /dsgvcy-15/?versions 2022-09-03T05:27:31.724 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rgGIf/dyBKSq0kEZmyMXVnEzvuw= 2022-09-03T05:27:31.726 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rgGIf/dyBKSq0kEZmyMXVnEzvuw=', 'Content-Length': '0'} 2022-09-03T05:27:31.729 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003f7742badf9fbf73-006312e5c3-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:27:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:31.775 DEBUG:boto:b'dsgvcy-15999false' 2022-09-03T05:27:31.777 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:27:31.777 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:27:31.778 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:27:31.779 DEBUG:boto:path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:31.779 DEBUG:boto:auth_path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:31.780 DEBUG:boto:Method: GET 2022-09-03T05:27:31.780 DEBUG:boto:Path: /dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:31.781 DEBUG:boto:Data: 2022-09-03T05:27:31.781 DEBUG:boto:Headers: {} 2022-09-03T05:27:31.782 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:27:31.783 DEBUG:boto:Port: 8001 2022-09-03T05:27:31.784 DEBUG:boto:Params: {} 2022-09-03T05:27:31.784 DEBUG:boto:Token: None 2022-09-03T05:27:31.785 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:27:31 GMT /dsgvcy-15/?versions 2022-09-03T05:27:31.786 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rgGIf/dyBKSq0kEZmyMXVnEzvuw= 2022-09-03T05:27:31.787 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rgGIf/dyBKSq0kEZmyMXVnEzvuw=', 'Content-Length': '0'} 2022-09-03T05:27:31.795 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001fdc162454f37661-006312e5c3-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:27:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:31.797 DEBUG:boto:b'dsgvcy-15999false' 2022-09-03T05:27:31.798 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:27:31.799 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:27:31.800 DEBUG:boto:path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:31.800 DEBUG:boto:auth_path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:31.801 DEBUG:boto:Method: GET 2022-09-03T05:27:31.802 DEBUG:boto:Path: /dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:31.802 DEBUG:boto:Data: 2022-09-03T05:27:31.803 DEBUG:boto:Headers: {} 2022-09-03T05:27:31.803 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:27:31.804 DEBUG:boto:Port: 8000 2022-09-03T05:27:31.805 DEBUG:boto:Params: {} 2022-09-03T05:27:31.806 DEBUG:boto:Token: None 2022-09-03T05:27:31.806 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:27:31 GMT /dsgvcy-15/?versions 2022-09-03T05:27:31.807 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rgGIf/dyBKSq0kEZmyMXVnEzvuw= 2022-09-03T05:27:31.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rgGIf/dyBKSq0kEZmyMXVnEzvuw=', 'Content-Length': '0'} 2022-09-03T05:27:31.812 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001da9e62003df4553-006312e5c3-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:27:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:31.813 DEBUG:boto:b'dsgvcy-15999false' 2022-09-03T05:27:31.814 DEBUG:boto:path=/dsgvcy-15/ 2022-09-03T05:27:31.814 DEBUG:boto:auth_path=/dsgvcy-15/ 2022-09-03T05:27:31.815 DEBUG:boto:path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:31.815 DEBUG:boto:auth_path=/dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:31.817 DEBUG:boto:Method: GET 2022-09-03T05:27:31.818 DEBUG:boto:Path: /dsgvcy-15/?versions&max-keys=999 2022-09-03T05:27:31.819 DEBUG:boto:Data: 2022-09-03T05:27:31.820 DEBUG:boto:Headers: {} 2022-09-03T05:27:31.820 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:27:31.821 DEBUG:boto:Port: 8001 2022-09-03T05:27:31.822 DEBUG:boto:Params: {} 2022-09-03T05:27:31.823 DEBUG:boto:Token: None 2022-09-03T05:27:31.824 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:27:31 GMT /dsgvcy-15/?versions 2022-09-03T05:27:31.825 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rgGIf/dyBKSq0kEZmyMXVnEzvuw= 2022-09-03T05:27:31.826 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rgGIf/dyBKSq0kEZmyMXVnEzvuw=', 'Content-Length': '0'} 2022-09-03T05:27:31.829 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000030da75986953bcfd-006312e5c3-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:27:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:31.870 DEBUG:boto:b'dsgvcy-15999false' 2022-09-03T05:27:31.871 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-15 zones={test-zone2, test-zone3} 2022-09-03T05:27:31.872 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_object_delete ... ok 2022-09-03T05:27:31.874 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-16 2022-09-03T05:27:31.875 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:27:31.877 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:27:31.878 DEBUG:boto:Method: PUT 2022-09-03T05:27:31.879 DEBUG:boto:Path: /dsgvcy-16/ 2022-09-03T05:27:31.880 DEBUG:boto:Data: 2022-09-03T05:27:31.881 DEBUG:boto:Headers: {} 2022-09-03T05:27:31.882 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:27:31.884 DEBUG:boto:Port: 8000 2022-09-03T05:27:31.885 DEBUG:boto:Params: {} 2022-09-03T05:27:31.886 DEBUG:boto:Token: None 2022-09-03T05:27:31.888 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:27:31 GMT /dsgvcy-16/ 2022-09-03T05:27:31.889 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tOQ/OE+y0JyVw5QQidT9qCr5MhA= 2022-09-03T05:27:31.890 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tOQ/OE+y0JyVw5QQidT9qCr5MhA=', 'Content-Length': '0'} 2022-09-03T05:27:31.914 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000e068e6cd0c715694-006312e5c3-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:27:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:31.915 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=dsgvcy-17 2022-09-03T05:27:31.916 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:27:31.918 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:27:31.919 DEBUG:boto:Method: PUT 2022-09-03T05:27:31.920 DEBUG:boto:Path: /dsgvcy-17/ 2022-09-03T05:27:31.921 DEBUG:boto:Data: 2022-09-03T05:27:31.922 DEBUG:boto:Headers: {} 2022-09-03T05:27:31.923 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:27:31.924 DEBUG:boto:Port: 8000 2022-09-03T05:27:31.925 DEBUG:boto:Params: {} 2022-09-03T05:27:31.926 DEBUG:boto:Token: None 2022-09-03T05:27:31.928 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:27:31 GMT /dsgvcy-17/ 2022-09-03T05:27:31.929 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cx9sWU+i2+2Vg01zgCl96u13lps= 2022-09-03T05:27:31.930 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cx9sWU+i2+2Vg01zgCl96u13lps=', 'Content-Length': '0'} 2022-09-03T05:27:31.967 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000006b87230ebc02443a-006312e5c3-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:27:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:31.968 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=dsgvcy-18 2022-09-03T05:27:31.968 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:27:31.969 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:27:31.969 DEBUG:boto:Method: PUT 2022-09-03T05:27:31.972 DEBUG:boto:Path: /dsgvcy-18/ 2022-09-03T05:27:31.973 DEBUG:boto:Data: 2022-09-03T05:27:31.974 DEBUG:boto:Headers: {} 2022-09-03T05:27:31.976 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:27:31.977 DEBUG:boto:Port: 8001 2022-09-03T05:27:31.977 DEBUG:boto:Params: {} 2022-09-03T05:27:31.978 DEBUG:boto:Token: None 2022-09-03T05:27:31.979 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:27:31 GMT /dsgvcy-18/ 2022-09-03T05:27:31.980 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VVsyu8oqbMrF5sVGroqxu3IhdHo= 2022-09-03T05:27:31.981 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VVsyu8oqbMrF5sVGroqxu3IhdHo=', 'Content-Length': '0'} 2022-09-03T05:27:32.046 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000049fe882ca34374db-006312e5c3-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:27:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:32.048 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:27:32.048 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:27:32.049 DEBUG:boto:path=/dsgvcy-16/?versioning 2022-09-03T05:27:32.049 DEBUG:boto:auth_path=/dsgvcy-16/?versioning 2022-09-03T05:27:32.050 DEBUG:boto:Method: PUT 2022-09-03T05:27:32.050 DEBUG:boto:Path: /dsgvcy-16/?versioning 2022-09-03T05:27:32.054 DEBUG:boto:Data: Enabled Disabled 2022-09-03T05:27:32.055 DEBUG:boto:Headers: {} 2022-09-03T05:27:32.056 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:27:32.057 DEBUG:boto:Port: 8000 2022-09-03T05:27:32.059 DEBUG:boto:Params: {} 2022-09-03T05:27:32.060 DEBUG:boto:Token: None 2022-09-03T05:27:32.061 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:27:32 GMT /dsgvcy-16/?versioning 2022-09-03T05:27:32.062 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9n3HkZw7wq6MZr/GP0Z+otBKhyw= 2022-09-03T05:27:32.063 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9n3HkZw7wq6MZr/GP0Z+otBKhyw=', 'Content-Length': '228'} 2022-09-03T05:27:32.077 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000004e91f99fe8f05f70-006312e5c4-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:27:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:32.078 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:27:32.079 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:27:32.080 DEBUG:boto:path=/dsgvcy-17/?versioning 2022-09-03T05:27:32.081 DEBUG:boto:auth_path=/dsgvcy-17/?versioning 2022-09-03T05:27:32.081 DEBUG:boto:Method: PUT 2022-09-03T05:27:32.082 DEBUG:boto:Path: /dsgvcy-17/?versioning 2022-09-03T05:27:32.082 DEBUG:boto:Data: Enabled Disabled 2022-09-03T05:27:32.083 DEBUG:boto:Headers: {} 2022-09-03T05:27:32.083 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:27:32.084 DEBUG:boto:Port: 8000 2022-09-03T05:27:32.084 DEBUG:boto:Params: {} 2022-09-03T05:27:32.085 DEBUG:boto:Token: None 2022-09-03T05:27:32.085 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:27:32 GMT /dsgvcy-17/?versioning 2022-09-03T05:27:32.086 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6toAno/pMUi4Z68azZQv1pfDKM4= 2022-09-03T05:27:32.087 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6toAno/pMUi4Z68azZQv1pfDKM4=', 'Content-Length': '228'} 2022-09-03T05:27:32.099 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000007fa5dee1c701de81-006312e5c4-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:27:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:32.100 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:27:32.101 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:27:32.102 DEBUG:boto:path=/dsgvcy-18/?versioning 2022-09-03T05:27:32.103 DEBUG:boto:auth_path=/dsgvcy-18/?versioning 2022-09-03T05:27:32.104 DEBUG:boto:Method: PUT 2022-09-03T05:27:32.105 DEBUG:boto:Path: /dsgvcy-18/?versioning 2022-09-03T05:27:32.106 DEBUG:boto:Data: Enabled Disabled 2022-09-03T05:27:32.107 DEBUG:boto:Headers: {} 2022-09-03T05:27:32.108 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:27:32.109 DEBUG:boto:Port: 8001 2022-09-03T05:27:32.110 DEBUG:boto:Params: {} 2022-09-03T05:27:32.111 DEBUG:boto:Token: None 2022-09-03T05:27:32.112 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:27:32 GMT /dsgvcy-18/?versioning 2022-09-03T05:27:32.113 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dEOL50ozslIgvqODrC4bVSmSYVU= 2022-09-03T05:27:32.114 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dEOL50ozslIgvqODrC4bVSmSYVU=', 'Content-Length': '228'} 2022-09-03T05:27:32.124 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000277fd94d5727e311-006312e5c4-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:27:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:32.125 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:27:32.128 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:27:32.129 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:27:32.190 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:27:32.190 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:27:32.243 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=41ecd2a5-afc0-4eae-9349-7dfe39baa1cb 2022-09-03T05:27:32.256 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:27:32.256 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:27:32.257 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:27:32.258 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:18:18.906852Z" 2022-09-03T05:27:32.258 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:27:32.259 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:27:32.260 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:27:32.260 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:27:32.260 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:27:32.261 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:27:32.262 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182852.124949_139.1", 2022-09-03T05:27:32.263 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:27:32.124949Z" 2022-09-03T05:27:32.268 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:27:32.268 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:27:32.268 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:27:32.269 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:20:30.928498Z" 2022-09-03T05:27:32.269 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:27:32.269 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:27:32.271 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182852.124949_139.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:27:32.272 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:27:32.273 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:27:32.273 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:27:32.275 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:27:32.327 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:27:32.328 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:27:32.358 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:27:32.361+0000 7fb870ffb600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:27:32.359 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:27:32.362+0000 7fb870ffb600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:27:32.371 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:27:32.371 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:27:32.371 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:27:32.372 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:27:32.372 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:27:32.372 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:27:32.372 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:27:32.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:27:32.373 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:27:32.373 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:27:32.374 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:27:32.374 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:27:32.374 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:27:32.374 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:27:32.375 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:27:32.375 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:27:32.375 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:27:32.376 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:27:32.376 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:27:32.377 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:27:32.377 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:27:32.377 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:27:32.378 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:27:32.378 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:27:32.379 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:27:32.380 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:27:32.380 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:27:32.380 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:27:32.380 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:27:32.381 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:27:32.381 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:27:32.381 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:27:32.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:27:32.382 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:27:32.382 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:27:32.382 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:27:32.383 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:27:32.383 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182852.069466_134.1", 2022-09-03T05:27:32.384 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:27:32.388 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:27:32.388 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:27:32.388 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:32.069466Z", 2022-09-03T05:27:32.389 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:27:32.389 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:27:32.390 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:27:32.390 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:27:32.391 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:27:32.391 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:27:32.392 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:27:32.392 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:27:32.393 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:27:32.393 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:27:32.394 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:27:32.394 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:27:32.395 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:27:32.395 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:27:32.396 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:27:32.396 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:27:32.397 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:27:32.397 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:27:32.397 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:27:32.398 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:27:32.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:27:32.399 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:27:32Z" 2022-09-03T05:27:32.400 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:27:32.401 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662182852.069466_134.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:27:32.069466Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:27:32Z" } 2022-09-03T05:27:32.402 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662182852.069466_134.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:27:32.069466Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:27:32.403 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182852.124949_139.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:27:32.404 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182852.069466_134.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:27:32.404 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662182852.124949_139.1 target=1_1662182852.069466_134.1 2022-09-03T05:27:37.406 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:27:37.407 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:27:37.408 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:27:37.465 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:27:37.466 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:27:37.498 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:27:37.501+0000 7f1ea96bf600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:27:37.499 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:27:37.502+0000 7f1ea96bf600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:27:37.512 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:27:37.512 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:27:37.513 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:27:37.513 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:27:37.513 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:27:37.514 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:27:37.514 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:27:37.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:27:37.514 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:27:37.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:27:37.515 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:27:37.515 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:27:37.515 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:27:37.516 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:27:37.516 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:27:37.516 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:27:37.516 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:27:37.517 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:27:37.517 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:27:37.517 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:27:37.517 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:27:37.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:27:37.518 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:27:37.518 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:27:37.519 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:27:37.519 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:27:37.519 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:27:37.519 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:27:37.520 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:27:37.520 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:27:37.520 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:27:37.520 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:27:37.521 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:27:37.521 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:27:37.521 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:27:37.521 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:27:37.522 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:27:37.522 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182852.124949_139.1", 2022-09-03T05:27:37.522 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:27:37.522 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:27:37.523 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:27:37.523 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:32.124949Z", 2022-09-03T05:27:37.523 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:27:37.523 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:27:37.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:27:37.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:27:37.524 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:27:37.525 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:27:37.525 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:27:37.525 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:27:37.526 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:27:37.526 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:27:37.526 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:27:37.526 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:27:37.527 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:27:37.527 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:27:37.527 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:27:37.527 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:27:37.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:27:37.528 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:27:37.528 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:27:37.528 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:27:37.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:27:37.529 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:27:37Z" 2022-09-03T05:27:37.529 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:27:37.530 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662182852.124949_139.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:27:32.124949Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:27:37Z" } 2022-09-03T05:27:37.531 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662182852.124949_139.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:27:32.124949Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:27:37.532 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182852.124949_139.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:27:37.532 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182852.124949_139.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:27:37.532 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:27:37.533 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:27:37.533 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:27:37.534 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:27:37.535 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:27:37.593 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:27:37.593 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:27:37.628 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:27:37.630+0000 7f1dd424b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:27:37.630 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:27:37.631+0000 7f1dd424b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:27:37.649 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:27:37.651 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:27:37.651 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:27:37.652 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:27:37.652 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:27:37.653 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:27:37.653 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:27:37.654 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:27:37.654 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:27:37.655 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:27:37.655 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:27:37.656 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:27:37.656 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:27:37.657 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:27:37.657 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:27:37.658 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:27:37.659 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:27:37.659 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:27:37.660 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:27:37.660 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:27:37.661 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:27:37.661 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:27:37.662 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:27:37.662 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:27:37.663 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:27:37.663 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:27:37.664 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:27:37.664 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:27:37.665 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:27:37.666 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:27:37.666 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:27:37.667 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:27:37.667 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:27:37.668 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:27:37.668 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:27:37.669 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:27:37.669 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:27:37.670 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182852.124949_139.1", 2022-09-03T05:27:37.670 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:27:37.671 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:27:37.671 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:27:37.671 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:32.124949Z", 2022-09-03T05:27:37.671 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:27:37.672 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:27:37.672 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:27:37.672 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:27:37.672 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:27:37.673 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:27:37.673 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:27:37.674 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:27:37.674 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:27:37.675 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:27:37.675 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:27:37.676 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:27:37.676 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:27:37.677 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:27:37.677 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:27:37.678 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:27:37.678 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:27:37.679 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:27:37.679 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:27:37.680 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:27:37.680 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:27:37.680 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:27:37Z" 2022-09-03T05:27:37.680 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:27:37.682 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662182852.124949_139.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:27:32.124949Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:27:37Z" } 2022-09-03T05:27:37.682 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662182852.124949_139.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:27:32.124949Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:27:37.683 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182852.124949_139.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:27:37.683 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182852.124949_139.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:27:37.684 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:27:37.684 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:27:37.685 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:27:37.686 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:27:37.686 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:27:37.738 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:27:37.738 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:27:37.764 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:27:37.765+0000 7f841f3d2600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:27:37.765 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:27:37.767+0000 7f841f3d2600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:27:37.774 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:27:37.775 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:27:37.775 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:27:37.776 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:27:37.776 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:27:37.777 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:27:37.777 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:27:37.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:27:37.778 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:27:37.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:27:37.779 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:27:37.780 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:27:37.780 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:27:37.780 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:27:37.780 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:27:37.781 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:27:37.781 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:27:37.781 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:27:37.781 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:27:37.782 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:27:37.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:27:37.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:27:37.783 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:27:37.783 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:27:37.783 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:27:37.784 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:27:37.784 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:27:37.784 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:27:37.785 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:27:37.785 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:27:37.785 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:27:37.786 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:27:37.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:27:37.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:27:37.786 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:27:37.787 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:27:37.787 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:27:37.787 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182852.124949_139.1", 2022-09-03T05:27:37.788 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:27:37.788 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:27:37.788 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:27:37.788 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:32.124949Z", 2022-09-03T05:27:37.789 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:27:37.789 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:27:37.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:27:37.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:27:37.790 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:27:37.790 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:27:37.791 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:27:37.791 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:27:37.792 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:27:37.792 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:27:37.793 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:27:37.793 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:27:37.794 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:27:37.794 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:27:37.795 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:27:37.795 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:27:37.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:27:37.796 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:27:37.797 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:27:37.797 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:27:37.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:27:37.798 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:27:37Z" 2022-09-03T05:27:37.799 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:27:37.801 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662182852.124949_139.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:27:32.124949Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:27:37Z" } 2022-09-03T05:27:37.802 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662182852.124949_139.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:27:32.124949Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:27:37.803 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182852.124949_139.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:27:37.804 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662182852.124949_139.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:27:37.805 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:27:37.806 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:27:37.806 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:27:37.807 DEBUG:boto:Method: HEAD 2022-09-03T05:27:37.807 DEBUG:boto:Path: /dsgvcy-16/ 2022-09-03T05:27:37.808 DEBUG:boto:Data: 2022-09-03T05:27:37.808 DEBUG:boto:Headers: {} 2022-09-03T05:27:37.810 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:27:37.810 DEBUG:boto:Port: 8000 2022-09-03T05:27:37.811 DEBUG:boto:Params: {} 2022-09-03T05:27:37.812 DEBUG:boto:Token: None 2022-09-03T05:27:37.813 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:27:37 GMT /dsgvcy-16/ 2022-09-03T05:27:37.813 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/4XMg66l1rJqs90xXqKqN6uZNkg= 2022-09-03T05:27:37.814 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/4XMg66l1rJqs90xXqKqN6uZNkg=', 'Content-Length': '0'} 2022-09-03T05:27:37.818 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e3ae30a773438298-006312e5c9-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:37.819 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:27:37.819 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:27:37.820 DEBUG:boto:Method: PUT 2022-09-03T05:27:37.821 DEBUG:boto:Path: /dsgvcy-16/dummy 2022-09-03T05:27:37.822 DEBUG:boto:Data: 2022-09-03T05:27:37.822 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue'} 2022-09-03T05:27:37.823 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:27:37.823 DEBUG:boto:Port: 8000 2022-09-03T05:27:37.824 DEBUG:boto:Params: {} 2022-09-03T05:27:37.825 DEBUG:boto:Token: None 2022-09-03T05:27:37.825 DEBUG:boto:StringToSign: PUT 1B2M2Y8AsgTpgAmY7PhCfg== application/octet-stream Sat, 03 Sep 2022 05:27:37 GMT /dsgvcy-16/dummy 2022-09-03T05:27:37.826 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CbAcPe6wA5fFbY6cw8amztBrosU= 2022-09-03T05:27:37.827 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:27:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CbAcPe6wA5fFbY6cw8amztBrosU='} 2022-09-03T05:27:37.850 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-amz-request-id', 'tx0000019bceae79cd0fb1d-006312e5c9-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:37.851 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:27:37.852 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:27:37.854 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-16 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:27:37.911 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:27:37.911 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:27:37.947 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:27:37.949+0000 7fbd41155600 1 waiting to reach incremental sync.. 2022-09-03T05:27:42.947 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:27:42.950+0000 7fbd41155600 1 waiting to reach incremental sync.. 2022-09-03T05:27:47.950 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:27:47.952+0000 7fbd41155600 1 bucket sync caught up with source: 2022-09-03T05:27:47.950 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000002.9.10, , , , , ] 2022-09-03T05:27:47.951 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000002.9.10, , , , , ] 2022-09-03T05:27:47.952 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:27:47.952+0000 7fbd41155600 0 bucket checkpoint complete 2022-09-03T05:27:47.974 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:27:47.975 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:27:47.977 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-16 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:27:48.042 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:27:48.042 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:27:48.103 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:27:48.105+0000 7fbd24f91600 1 bucket sync caught up with source: 2022-09-03T05:27:48.104 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000002.9.10, , , , , ] 2022-09-03T05:27:48.104 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000002.9.10, , , , , ] 2022-09-03T05:27:48.105 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:27:48.105+0000 7fbd24f91600 0 bucket checkpoint complete 2022-09-03T05:27:48.130 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:27:48.131 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:27:48.132 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-16 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:27:48.201 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:27:48.202 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:27:48.235 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:27:48.238+0000 7f6e92698600 1 waiting to reach incremental sync.. 2022-09-03T05:27:53.236 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:27:53.238+0000 7f6e92698600 1 waiting to reach incremental sync.. 2022-09-03T05:27:58.238 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:27:58.240+0000 7f6e92698600 1 bucket sync caught up with source: 2022-09-03T05:27:58.238 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000002.9.10, , , , , ] 2022-09-03T05:27:58.239 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000002.9.10, , , , , ] 2022-09-03T05:27:58.239 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:27:58.240+0000 7f6e92698600 0 bucket checkpoint complete 2022-09-03T05:27:58.252 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:27:58.253 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:27:58.254 DEBUG:boto:Method: HEAD 2022-09-03T05:27:58.254 DEBUG:boto:Path: /dsgvcy-17/ 2022-09-03T05:27:58.255 DEBUG:boto:Data: 2022-09-03T05:27:58.255 DEBUG:boto:Headers: {} 2022-09-03T05:27:58.256 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:27:58.256 DEBUG:boto:Port: 8000 2022-09-03T05:27:58.257 DEBUG:boto:Params: {} 2022-09-03T05:27:58.257 DEBUG:boto:Token: None 2022-09-03T05:27:58.258 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:27:58 GMT /dsgvcy-17/ 2022-09-03T05:27:58.259 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oQvU/0SVCWdG9UG0rMdNB3v8ZAs= 2022-09-03T05:27:58.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:27:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oQvU/0SVCWdG9UG0rMdNB3v8ZAs=', 'Content-Length': '0'} 2022-09-03T05:27:58.263 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e40d92dade15feb7-006312e5de-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:58.265 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:27:58.266 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:27:58.267 DEBUG:boto:Method: PUT 2022-09-03T05:27:58.268 DEBUG:boto:Path: /dsgvcy-17/dummy 2022-09-03T05:27:58.269 DEBUG:boto:Data: 2022-09-03T05:27:58.270 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue'} 2022-09-03T05:27:58.271 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:27:58.271 DEBUG:boto:Port: 8000 2022-09-03T05:27:58.272 DEBUG:boto:Params: {} 2022-09-03T05:27:58.274 DEBUG:boto:Token: None 2022-09-03T05:27:58.275 DEBUG:boto:StringToSign: PUT 1B2M2Y8AsgTpgAmY7PhCfg== application/octet-stream Sat, 03 Sep 2022 05:27:58 GMT /dsgvcy-17/dummy 2022-09-03T05:27:58.276 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KT3sloKG8cf8ACchBk/C6k4WNdk= 2022-09-03T05:27:58.277 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:27:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KT3sloKG8cf8ACchBk/C6k4WNdk='} 2022-09-03T05:27:58.290 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-amz-request-id', 'tx000004ec5259c273d0a7c-006312e5de-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:27:58.291 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:27:58.291 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:27:58.292 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-17 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:27:58.358 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:27:58.359 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:27:58.392 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:27:58.395+0000 7f1a5a087600 1 waiting to reach incremental sync.. 2022-09-03T05:28:03.393 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:28:03.396+0000 7f1a5a087600 1 waiting to reach incremental sync.. 2022-09-03T05:28:08.395 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:28:08.398+0000 7f1a5a087600 1 bucket sync caught up with source: 2022-09-03T05:28:08.395 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000002.25.10, , , , , ] 2022-09-03T05:28:08.396 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000002.25.10, , , , , ] 2022-09-03T05:28:08.396 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:28:08.398+0000 7f1a5a087600 0 bucket checkpoint complete 2022-09-03T05:28:08.410 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:28:08.411 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:28:08.412 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-17 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:28:08.478 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:28:08.479 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:28:08.516 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:28:08.513+0000 7f470b3b4600 1 waiting to reach incremental sync.. 2022-09-03T05:28:13.512 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:28:13.513+0000 7f470b3b4600 1 waiting to reach incremental sync.. 2022-09-03T05:28:18.511 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:28:18.513+0000 7f470b3b4600 1 waiting to reach incremental sync.. 2022-09-03T05:28:23.513 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:28:23.514+0000 7f470b3b4600 1 waiting to reach incremental sync.. 2022-09-03T05:28:28.518 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:28:28.516+0000 7f470b3b4600 1 bucket sync caught up with source: 2022-09-03T05:28:28.519 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.27.9, , , , , ] 2022-09-03T05:28:28.519 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000002.25.10, , , , , ] 2022-09-03T05:28:28.520 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:28:28.516+0000 7f470b3b4600 0 bucket checkpoint complete 2022-09-03T05:28:28.532 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:28:28.533 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:28:28.534 DEBUG:boto:Method: HEAD 2022-09-03T05:28:28.534 DEBUG:boto:Path: /dsgvcy-18/ 2022-09-03T05:28:28.535 DEBUG:boto:Data: 2022-09-03T05:28:28.535 DEBUG:boto:Headers: {} 2022-09-03T05:28:28.536 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:28:28.536 DEBUG:boto:Port: 8001 2022-09-03T05:28:28.537 DEBUG:boto:Params: {} 2022-09-03T05:28:28.538 DEBUG:boto:Token: None 2022-09-03T05:28:28.538 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:28:28 GMT /dsgvcy-18/ 2022-09-03T05:28:28.539 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7tXxkSNjQ3lS+VRXRMTr4cR0jHU= 2022-09-03T05:28:28.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:28:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7tXxkSNjQ3lS+VRXRMTr4cR0jHU=', 'Content-Length': '0'} 2022-09-03T05:28:28.543 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000cc569e9be4370f2e-006312e5fc-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:28:28.544 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:28:28.545 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:28:28.545 DEBUG:boto:Method: PUT 2022-09-03T05:28:28.546 DEBUG:boto:Path: /dsgvcy-18/dummy 2022-09-03T05:28:28.546 DEBUG:boto:Data: 2022-09-03T05:28:28.547 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue'} 2022-09-03T05:28:28.547 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:28:28.548 DEBUG:boto:Port: 8001 2022-09-03T05:28:28.548 DEBUG:boto:Params: {} 2022-09-03T05:28:28.549 DEBUG:boto:Token: None 2022-09-03T05:28:28.550 DEBUG:boto:StringToSign: PUT 1B2M2Y8AsgTpgAmY7PhCfg== application/octet-stream Sat, 03 Sep 2022 05:28:28 GMT /dsgvcy-18/dummy 2022-09-03T05:28:28.552 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jX5x1PGDaff3SiBWyj159aBkpAM= 2022-09-03T05:28:28.553 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:28:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jX5x1PGDaff3SiBWyj159aBkpAM='} 2022-09-03T05:28:28.566 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-amz-request-id', 'tx00000aa44330e1dd9a7e2-006312e5fc-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:28:28.570 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:28:28.571 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:28:28.573 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-18 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:28:28.637 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:28:28.637 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:28:28.670 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:28:28.672+0000 7f2ca9cef600 1 waiting to reach incremental sync.. 2022-09-03T05:28:33.670 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:28:33.672+0000 7f2ca9cef600 1 waiting to reach incremental sync.. 2022-09-03T05:28:38.671 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:28:38.673+0000 7f2ca9cef600 1 waiting to reach incremental sync.. 2022-09-03T05:28:43.671 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:28:43.674+0000 7f2ca9cef600 1 waiting to reach incremental sync.. 2022-09-03T05:28:48.673 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:28:48.676+0000 7f2ca9cef600 1 bucket sync caught up with source: 2022-09-03T05:28:48.674 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000002.18.10, , , , , ] 2022-09-03T05:28:48.675 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000002.18.10, , , , , ] 2022-09-03T05:28:48.675 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:28:48.676+0000 7f2ca9cef600 0 bucket checkpoint complete 2022-09-03T05:28:48.698 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:28:48.699 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:28:48.701 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-18 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:28:48.765 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:28:48.766 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:28:48.809 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:28:48.811+0000 7fa9cd69e600 1 waiting for incremental sync to catch up: 2022-09-03T05:28:48.809 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000002.18.10, , , , , ] 2022-09-03T05:28:48.809 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000004.21.9, , , , , ] 2022-09-03T05:28:53.810 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:28:53.812+0000 7fa9cd69e600 1 waiting for incremental sync to catch up: 2022-09-03T05:28:53.810 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000002.18.10, , , , , ] 2022-09-03T05:28:53.810 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000004.21.9, , , , , ] 2022-09-03T05:28:58.811 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:28:58.814+0000 7fa9cd69e600 1 waiting for incremental sync to catch up: 2022-09-03T05:28:58.812 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000002.18.10, , , , , ] 2022-09-03T05:28:58.812 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000004.21.9, , , , , ] 2022-09-03T05:29:03.812 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:29:03.815+0000 7fa9cd69e600 1 waiting for incremental sync to catch up: 2022-09-03T05:29:03.813 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000002.18.10, , , , , ] 2022-09-03T05:29:03.813 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000004.21.9, , , , , ] 2022-09-03T05:29:08.814 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:29:08.816+0000 7fa9cd69e600 1 bucket sync caught up with source: 2022-09-03T05:29:08.815 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000005.24.9, , , , , ] 2022-09-03T05:29:08.815 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000004.21.9, , , , , ] 2022-09-03T05:29:08.816 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:29:08.816+0000 7fa9cd69e600 0 bucket checkpoint complete 2022-09-03T05:29:08.831 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:08.832 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:08.833 DEBUG:boto:Method: HEAD 2022-09-03T05:29:08.833 DEBUG:boto:Path: /dsgvcy-16/ 2022-09-03T05:29:08.834 DEBUG:boto:Data: 2022-09-03T05:29:08.835 DEBUG:boto:Headers: {} 2022-09-03T05:29:08.835 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:08.836 DEBUG:boto:Port: 8000 2022-09-03T05:29:08.836 DEBUG:boto:Params: {} 2022-09-03T05:29:08.837 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:29:08.837 DEBUG:boto:Token: None 2022-09-03T05:29:08.838 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:08 GMT /dsgvcy-16/ 2022-09-03T05:29:08.839 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qSg8AtPJ/ak7yxOFIUcvs07+ywE= 2022-09-03T05:29:08.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qSg8AtPJ/ak7yxOFIUcvs07+ywE=', 'Content-Length': '0'} 2022-09-03T05:29:08.845 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c5ba231667c5c1a9-006312e624-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:08.847 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:08.848 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:08.849 DEBUG:boto:Method: PUT 2022-09-03T05:29:08.850 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1 2022-09-03T05:29:08.852 DEBUG:boto:Data: 2022-09-03T05:29:08.853 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:08.854 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:08.855 DEBUG:boto:Port: 8000 2022-09-03T05:29:08.856 DEBUG:boto:Params: {} 2022-09-03T05:29:08.857 DEBUG:boto:Token: None 2022-09-03T05:29:08.858 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:29:08 GMT /dsgvcy-16/obj-test-zone1 2022-09-03T05:29:08.860 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:achz2kzTQCaGL2lC9z92dRFpQxE= 2022-09-03T05:29:08.861 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:achz2kzTQCaGL2lC9z92dRFpQxE='} 2022-09-03T05:29:08.879 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-amz-request-id', 'tx00000365ce2e23ef43808-006312e624-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:08.880 DEBUG:rgw_multi.tests:version1 id=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:08.881 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:08.882 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:08.882 DEBUG:boto:Method: PUT 2022-09-03T05:29:08.883 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1 2022-09-03T05:29:08.883 DEBUG:boto:Data: 2022-09-03T05:29:08.884 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:08.885 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:08.885 DEBUG:boto:Port: 8000 2022-09-03T05:29:08.886 DEBUG:boto:Params: {} 2022-09-03T05:29:08.887 DEBUG:boto:Token: None 2022-09-03T05:29:08.887 DEBUG:boto:StringToSign: PUT Lg6VKF8IoH3qF+fuERshyA== application/octet-stream Sat, 03 Sep 2022 05:29:08 GMT /dsgvcy-16/obj-test-zone1 2022-09-03T05:29:08.888 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QkDhIjT1XTckt0PEEynSX5eTOjE= 2022-09-03T05:29:08.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QkDhIjT1XTckt0PEEynSX5eTOjE='} 2022-09-03T05:29:08.907 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"2e0e95285f08a07dea17e7ee111b21c8"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '-Voazy-.ahz7Y17jeL5WAJngkDDUnCH'), ('x-amz-request-id', 'tx000004bce824c165fe252-006312e624-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:08.908 DEBUG:rgw_multi.tests:version2 id=-Voazy-.ahz7Y17jeL5WAJngkDDUnCH 2022-09-03T05:29:08.908 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:08.909 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:08.910 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=-Voazy-.ahz7Y17jeL5WAJngkDDUnCH 2022-09-03T05:29:08.911 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=-Voazy-.ahz7Y17jeL5WAJngkDDUnCH 2022-09-03T05:29:08.912 DEBUG:boto:Method: DELETE 2022-09-03T05:29:08.913 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=-Voazy-.ahz7Y17jeL5WAJngkDDUnCH 2022-09-03T05:29:08.914 DEBUG:boto:Data: 2022-09-03T05:29:08.915 DEBUG:boto:Headers: {} 2022-09-03T05:29:08.915 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:08.916 DEBUG:boto:Port: 8000 2022-09-03T05:29:08.917 DEBUG:boto:Params: {} 2022-09-03T05:29:08.918 DEBUG:boto:Token: None 2022-09-03T05:29:08.919 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:29:08 GMT /dsgvcy-16/obj-test-zone1?versionId=-Voazy-.ahz7Y17jeL5WAJngkDDUnCH 2022-09-03T05:29:08.921 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lk8yEDtuCACR+j2cKKSgIEhaebA= 2022-09-03T05:29:08.922 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lk8yEDtuCACR+j2cKKSgIEhaebA=', 'Content-Length': '0'} 2022-09-03T05:29:08.942 DEBUG:boto:Response headers: [('x-amz-version-id', '-Voazy-.ahz7Y17jeL5WAJngkDDUnCH'), ('x-amz-request-id', 'tx00000092377a54a7a4fb7-006312e624-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:08.943 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:08.943 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:08.944 DEBUG:boto:Method: PUT 2022-09-03T05:29:08.945 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1 2022-09-03T05:29:08.945 DEBUG:boto:Data: 2022-09-03T05:29:08.946 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:08.946 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:08.947 DEBUG:boto:Port: 8000 2022-09-03T05:29:08.947 DEBUG:boto:Params: {} 2022-09-03T05:29:08.948 DEBUG:boto:Token: None 2022-09-03T05:29:08.948 DEBUG:boto:StringToSign: PUT HiNA5/HU7CwjRIhUZJ8d7Q== application/octet-stream Sat, 03 Sep 2022 05:29:08 GMT /dsgvcy-16/obj-test-zone1 2022-09-03T05:29:08.949 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tPqTpAAmrsCaS9V+mheFrBEA/vw= 2022-09-03T05:29:08.950 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tPqTpAAmrsCaS9V+mheFrBEA/vw='} 2022-09-03T05:29:08.995 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"1e2340e7f1d4ec2c23448854649f1ded"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'hoCmJIptunm.mKroxLIWTK6H4z4ZPUl'), ('x-amz-request-id', 'tx00000645da2696d2e5c8a-006312e624-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:08.996 DEBUG:rgw_multi.tests:version3 id=hoCmJIptunm.mKroxLIWTK6H4z4ZPUl 2022-09-03T05:29:08.997 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:08.997 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:08.998 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=hoCmJIptunm.mKroxLIWTK6H4z4ZPUl 2022-09-03T05:29:08.998 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=hoCmJIptunm.mKroxLIWTK6H4z4ZPUl 2022-09-03T05:29:08.999 DEBUG:boto:Method: DELETE 2022-09-03T05:29:08.999 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=hoCmJIptunm.mKroxLIWTK6H4z4ZPUl 2022-09-03T05:29:09.000 DEBUG:boto:Data: 2022-09-03T05:29:09.000 DEBUG:boto:Headers: {} 2022-09-03T05:29:09.001 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:09.001 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.002 DEBUG:boto:Params: {} 2022-09-03T05:29:09.003 DEBUG:boto:Token: None 2022-09-03T05:29:09.004 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-16/obj-test-zone1?versionId=hoCmJIptunm.mKroxLIWTK6H4z4ZPUl 2022-09-03T05:29:09.005 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hvYNI2LlN6pBFSm0Qo6ihJ1BGvQ= 2022-09-03T05:29:09.005 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hvYNI2LlN6pBFSm0Qo6ihJ1BGvQ=', 'Content-Length': '0'} 2022-09-03T05:29:09.018 DEBUG:boto:Response headers: [('x-amz-version-id', 'hoCmJIptunm.mKroxLIWTK6H4z4ZPUl'), ('x-amz-request-id', 'tx000000187e4cffb5ed758-006312e625-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.019 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:09.020 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:09.021 DEBUG:boto:Method: HEAD 2022-09-03T05:29:09.022 DEBUG:boto:Path: /dsgvcy-16/ 2022-09-03T05:29:09.023 DEBUG:boto:Data: 2022-09-03T05:29:09.024 DEBUG:boto:Headers: {} 2022-09-03T05:29:09.024 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:09.025 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.026 DEBUG:boto:Params: {} 2022-09-03T05:29:09.027 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:29:09.027 DEBUG:boto:Token: None 2022-09-03T05:29:09.028 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-16/ 2022-09-03T05:29:09.029 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oneXQoTnkLcPQseGzuE9r3vgQ7A= 2022-09-03T05:29:09.030 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oneXQoTnkLcPQseGzuE9r3vgQ7A=', 'Content-Length': '0'} 2022-09-03T05:29:09.034 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e32b5138d5191efb-006312e625-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.034 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:09.035 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:09.035 DEBUG:boto:Method: PUT 2022-09-03T05:29:09.036 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2 2022-09-03T05:29:09.036 DEBUG:boto:Data: 2022-09-03T05:29:09.037 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:09.037 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:09.038 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.039 DEBUG:boto:Params: {} 2022-09-03T05:29:09.039 DEBUG:boto:Token: None 2022-09-03T05:29:09.040 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-16/obj-test-zone2 2022-09-03T05:29:09.040 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OmgLYi5qT9Pjkmr82uuFAPlOOkc= 2022-09-03T05:29:09.041 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OmgLYi5qT9Pjkmr82uuFAPlOOkc='} 2022-09-03T05:29:09.055 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-amz-request-id', 'tx0000009593696dab957e3-006312e625-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.056 DEBUG:rgw_multi.tests:version1 id=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:09.057 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:09.058 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:09.058 DEBUG:boto:Method: PUT 2022-09-03T05:29:09.059 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2 2022-09-03T05:29:09.060 DEBUG:boto:Data: 2022-09-03T05:29:09.061 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:09.062 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:09.062 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.063 DEBUG:boto:Params: {} 2022-09-03T05:29:09.064 DEBUG:boto:Token: None 2022-09-03T05:29:09.065 DEBUG:boto:StringToSign: PUT Lg6VKF8IoH3qF+fuERshyA== application/octet-stream Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-16/obj-test-zone2 2022-09-03T05:29:09.066 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mFy3k0sbID9ajU2Z6zIqK0c68FE= 2022-09-03T05:29:09.067 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mFy3k0sbID9ajU2Z6zIqK0c68FE='} 2022-09-03T05:29:09.079 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"2e0e95285f08a07dea17e7ee111b21c8"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'riEhyO8w-14UXhh8pnbsSy1nm0ZEG4R'), ('x-amz-request-id', 'tx00000f89eea202fc2200d-006312e625-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.080 DEBUG:rgw_multi.tests:version2 id=riEhyO8w-14UXhh8pnbsSy1nm0ZEG4R 2022-09-03T05:29:09.080 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:09.081 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:09.082 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=riEhyO8w-14UXhh8pnbsSy1nm0ZEG4R 2022-09-03T05:29:09.082 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=riEhyO8w-14UXhh8pnbsSy1nm0ZEG4R 2022-09-03T05:29:09.083 DEBUG:boto:Method: DELETE 2022-09-03T05:29:09.083 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=riEhyO8w-14UXhh8pnbsSy1nm0ZEG4R 2022-09-03T05:29:09.084 DEBUG:boto:Data: 2022-09-03T05:29:09.084 DEBUG:boto:Headers: {} 2022-09-03T05:29:09.085 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:09.085 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.086 DEBUG:boto:Params: {} 2022-09-03T05:29:09.086 DEBUG:boto:Token: None 2022-09-03T05:29:09.087 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-16/obj-test-zone2?versionId=riEhyO8w-14UXhh8pnbsSy1nm0ZEG4R 2022-09-03T05:29:09.088 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n3YYUEkPPKoOWGzPA7STs6OfDg0= 2022-09-03T05:29:09.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n3YYUEkPPKoOWGzPA7STs6OfDg0=', 'Content-Length': '0'} 2022-09-03T05:29:09.100 DEBUG:boto:Response headers: [('x-amz-version-id', 'riEhyO8w-14UXhh8pnbsSy1nm0ZEG4R'), ('x-amz-request-id', 'tx0000017adb94c3e0a0f8d-006312e625-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.101 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:09.101 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:09.102 DEBUG:boto:Method: PUT 2022-09-03T05:29:09.102 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2 2022-09-03T05:29:09.103 DEBUG:boto:Data: 2022-09-03T05:29:09.103 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:09.104 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:09.104 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.105 DEBUG:boto:Params: {} 2022-09-03T05:29:09.106 DEBUG:boto:Token: None 2022-09-03T05:29:09.107 DEBUG:boto:StringToSign: PUT HiNA5/HU7CwjRIhUZJ8d7Q== application/octet-stream Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-16/obj-test-zone2 2022-09-03T05:29:09.107 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iwY3hfy5OJkh8vcqxOhZuGkL+bU= 2022-09-03T05:29:09.108 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iwY3hfy5OJkh8vcqxOhZuGkL+bU='} 2022-09-03T05:29:09.150 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"1e2340e7f1d4ec2c23448854649f1ded"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'J-Iz6VPK-ai2riUjBGnf2dcb7-z7rJ5'), ('x-amz-request-id', 'tx0000012a5ea324acddee0-006312e625-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.151 DEBUG:rgw_multi.tests:version3 id=J-Iz6VPK-ai2riUjBGnf2dcb7-z7rJ5 2022-09-03T05:29:09.152 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:09.152 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:09.153 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=J-Iz6VPK-ai2riUjBGnf2dcb7-z7rJ5 2022-09-03T05:29:09.153 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=J-Iz6VPK-ai2riUjBGnf2dcb7-z7rJ5 2022-09-03T05:29:09.154 DEBUG:boto:Method: DELETE 2022-09-03T05:29:09.154 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=J-Iz6VPK-ai2riUjBGnf2dcb7-z7rJ5 2022-09-03T05:29:09.155 DEBUG:boto:Data: 2022-09-03T05:29:09.155 DEBUG:boto:Headers: {} 2022-09-03T05:29:09.156 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:09.156 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.157 DEBUG:boto:Params: {} 2022-09-03T05:29:09.158 DEBUG:boto:Token: None 2022-09-03T05:29:09.158 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-16/obj-test-zone2?versionId=J-Iz6VPK-ai2riUjBGnf2dcb7-z7rJ5 2022-09-03T05:29:09.159 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O4FO66A2+Kwj/Fj8k3ijzsUX72A= 2022-09-03T05:29:09.159 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O4FO66A2+Kwj/Fj8k3ijzsUX72A=', 'Content-Length': '0'} 2022-09-03T05:29:09.178 DEBUG:boto:Response headers: [('x-amz-version-id', 'J-Iz6VPK-ai2riUjBGnf2dcb7-z7rJ5'), ('x-amz-request-id', 'tx00000153f843c250744c9-006312e625-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.179 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:09.180 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:09.180 DEBUG:boto:Method: HEAD 2022-09-03T05:29:09.181 DEBUG:boto:Path: /dsgvcy-16/ 2022-09-03T05:29:09.182 DEBUG:boto:Data: 2022-09-03T05:29:09.183 DEBUG:boto:Headers: {} 2022-09-03T05:29:09.183 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:09.184 DEBUG:boto:Port: 8001 2022-09-03T05:29:09.184 DEBUG:boto:Params: {} 2022-09-03T05:29:09.185 DEBUG:boto:Token: None 2022-09-03T05:29:09.186 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-16/ 2022-09-03T05:29:09.186 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oneXQoTnkLcPQseGzuE9r3vgQ7A= 2022-09-03T05:29:09.187 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oneXQoTnkLcPQseGzuE9r3vgQ7A=', 'Content-Length': '0'} 2022-09-03T05:29:09.190 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000df8c9fed94807a97-006312e625-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.191 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:09.191 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:09.192 DEBUG:boto:Method: PUT 2022-09-03T05:29:09.192 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3 2022-09-03T05:29:09.193 DEBUG:boto:Data: 2022-09-03T05:29:09.193 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:09.194 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:09.195 DEBUG:boto:Port: 8001 2022-09-03T05:29:09.195 DEBUG:boto:Params: {} 2022-09-03T05:29:09.196 DEBUG:boto:Token: None 2022-09-03T05:29:09.196 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-16/obj-test-zone3 2022-09-03T05:29:09.197 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yZh1FS8mOhmvStyf4Kk7AE9k05c= 2022-09-03T05:29:09.198 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yZh1FS8mOhmvStyf4Kk7AE9k05c='} 2022-09-03T05:29:09.208 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-amz-request-id', 'tx00000fb320f2e3a4102bd-006312e625-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.209 DEBUG:rgw_multi.tests:version1 id=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:09.209 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:09.210 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:09.210 DEBUG:boto:Method: PUT 2022-09-03T05:29:09.211 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3 2022-09-03T05:29:09.212 DEBUG:boto:Data: 2022-09-03T05:29:09.212 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:09.213 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:09.214 DEBUG:boto:Port: 8001 2022-09-03T05:29:09.215 DEBUG:boto:Params: {} 2022-09-03T05:29:09.215 DEBUG:boto:Token: None 2022-09-03T05:29:09.216 DEBUG:boto:StringToSign: PUT Lg6VKF8IoH3qF+fuERshyA== application/octet-stream Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-16/obj-test-zone3 2022-09-03T05:29:09.217 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BCg3kyTcuJv/4oZgtQBS+Lny0P4= 2022-09-03T05:29:09.217 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BCg3kyTcuJv/4oZgtQBS+Lny0P4='} 2022-09-03T05:29:09.229 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"2e0e95285f08a07dea17e7ee111b21c8"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'hvfvnv5A1HNLDf--ZLOrIXlK79w-Cxo'), ('x-amz-request-id', 'tx00000937252401f24e948-006312e625-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.230 DEBUG:rgw_multi.tests:version2 id=hvfvnv5A1HNLDf--ZLOrIXlK79w-Cxo 2022-09-03T05:29:09.231 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:09.231 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:09.232 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=hvfvnv5A1HNLDf--ZLOrIXlK79w-Cxo 2022-09-03T05:29:09.233 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=hvfvnv5A1HNLDf--ZLOrIXlK79w-Cxo 2022-09-03T05:29:09.233 DEBUG:boto:Method: DELETE 2022-09-03T05:29:09.234 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=hvfvnv5A1HNLDf--ZLOrIXlK79w-Cxo 2022-09-03T05:29:09.234 DEBUG:boto:Data: 2022-09-03T05:29:09.235 DEBUG:boto:Headers: {} 2022-09-03T05:29:09.236 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:09.237 DEBUG:boto:Port: 8001 2022-09-03T05:29:09.237 DEBUG:boto:Params: {} 2022-09-03T05:29:09.238 DEBUG:boto:Token: None 2022-09-03T05:29:09.239 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-16/obj-test-zone3?versionId=hvfvnv5A1HNLDf--ZLOrIXlK79w-Cxo 2022-09-03T05:29:09.240 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HR0FA5ZKF/wZKYd33NiGCp9Rbi4= 2022-09-03T05:29:09.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HR0FA5ZKF/wZKYd33NiGCp9Rbi4=', 'Content-Length': '0'} 2022-09-03T05:29:09.251 DEBUG:boto:Response headers: [('x-amz-version-id', 'hvfvnv5A1HNLDf--ZLOrIXlK79w-Cxo'), ('x-amz-request-id', 'tx000001ca69cab0a552977-006312e625-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.252 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:09.252 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:09.253 DEBUG:boto:Method: PUT 2022-09-03T05:29:09.253 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3 2022-09-03T05:29:09.254 DEBUG:boto:Data: 2022-09-03T05:29:09.255 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:09.255 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:09.256 DEBUG:boto:Port: 8001 2022-09-03T05:29:09.256 DEBUG:boto:Params: {} 2022-09-03T05:29:09.257 DEBUG:boto:Token: None 2022-09-03T05:29:09.257 DEBUG:boto:StringToSign: PUT HiNA5/HU7CwjRIhUZJ8d7Q== application/octet-stream Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-16/obj-test-zone3 2022-09-03T05:29:09.258 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kRaU0ZIZxIKAwVyq5OxTeajRk+4= 2022-09-03T05:29:09.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kRaU0ZIZxIKAwVyq5OxTeajRk+4='} 2022-09-03T05:29:09.302 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"1e2340e7f1d4ec2c23448854649f1ded"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'jgBUNZ3sI-4hwk0d2T6U7Vk38jXlI3Q'), ('x-amz-request-id', 'tx0000098d53486c784ac55-006312e625-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.304 DEBUG:rgw_multi.tests:version3 id=jgBUNZ3sI-4hwk0d2T6U7Vk38jXlI3Q 2022-09-03T05:29:09.305 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:09.305 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:09.306 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=jgBUNZ3sI-4hwk0d2T6U7Vk38jXlI3Q 2022-09-03T05:29:09.307 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=jgBUNZ3sI-4hwk0d2T6U7Vk38jXlI3Q 2022-09-03T05:29:09.308 DEBUG:boto:Method: DELETE 2022-09-03T05:29:09.309 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=jgBUNZ3sI-4hwk0d2T6U7Vk38jXlI3Q 2022-09-03T05:29:09.309 DEBUG:boto:Data: 2022-09-03T05:29:09.310 DEBUG:boto:Headers: {} 2022-09-03T05:29:09.310 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:09.311 DEBUG:boto:Port: 8001 2022-09-03T05:29:09.311 DEBUG:boto:Params: {} 2022-09-03T05:29:09.312 DEBUG:boto:Token: None 2022-09-03T05:29:09.314 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-16/obj-test-zone3?versionId=jgBUNZ3sI-4hwk0d2T6U7Vk38jXlI3Q 2022-09-03T05:29:09.315 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QKtBsa3PEBLVZUrg9bObGqPPynk= 2022-09-03T05:29:09.316 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QKtBsa3PEBLVZUrg9bObGqPPynk=', 'Content-Length': '0'} 2022-09-03T05:29:09.326 DEBUG:boto:Response headers: [('x-amz-version-id', 'jgBUNZ3sI-4hwk0d2T6U7Vk38jXlI3Q'), ('x-amz-request-id', 'tx00000c3d4dd1008aa6ffc-006312e625-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.327 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:29:09.328 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:29:09.328 DEBUG:boto:Method: HEAD 2022-09-03T05:29:09.329 DEBUG:boto:Path: /dsgvcy-17/ 2022-09-03T05:29:09.329 DEBUG:boto:Data: 2022-09-03T05:29:09.330 DEBUG:boto:Headers: {} 2022-09-03T05:29:09.331 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:09.331 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.332 DEBUG:boto:Params: {} 2022-09-03T05:29:09.332 DEBUG:boto:Token: None 2022-09-03T05:29:09.333 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-17/ 2022-09-03T05:29:09.334 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UkQIFBflqf08XT4pgAmc4UoViDU= 2022-09-03T05:29:09.335 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UkQIFBflqf08XT4pgAmc4UoViDU=', 'Content-Length': '0'} 2022-09-03T05:29:09.337 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000326ad7e7bf5ed813-006312e625-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.338 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:29:09.339 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:29:09.339 DEBUG:boto:Method: PUT 2022-09-03T05:29:09.340 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1 2022-09-03T05:29:09.341 DEBUG:boto:Data: 2022-09-03T05:29:09.341 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:09.342 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:09.342 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.343 DEBUG:boto:Params: {} 2022-09-03T05:29:09.343 DEBUG:boto:Token: None 2022-09-03T05:29:09.344 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-17/obj-test-zone1 2022-09-03T05:29:09.345 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eBGmTcXcAyy4Riu35Kq1q5i7S5U= 2022-09-03T05:29:09.345 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eBGmTcXcAyy4Riu35Kq1q5i7S5U='} 2022-09-03T05:29:09.363 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-amz-request-id', 'tx000008171b19c18c27b69-006312e625-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.365 DEBUG:rgw_multi.tests:version1 id=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:29:09.366 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:29:09.367 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:29:09.368 DEBUG:boto:Method: PUT 2022-09-03T05:29:09.368 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1 2022-09-03T05:29:09.369 DEBUG:boto:Data: 2022-09-03T05:29:09.370 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:09.371 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:09.371 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.372 DEBUG:boto:Params: {} 2022-09-03T05:29:09.373 DEBUG:boto:Token: None 2022-09-03T05:29:09.373 DEBUG:boto:StringToSign: PUT Lg6VKF8IoH3qF+fuERshyA== application/octet-stream Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-17/obj-test-zone1 2022-09-03T05:29:09.374 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+NHdeGx9EmnfQ8fXvut57nVcRQE= 2022-09-03T05:29:09.374 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+NHdeGx9EmnfQ8fXvut57nVcRQE='} 2022-09-03T05:29:09.388 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"2e0e95285f08a07dea17e7ee111b21c8"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '6ReZk8rcIxKcbtdG9110M0aCBVu-TYm'), ('x-amz-request-id', 'tx00000855968c2cc76240b-006312e625-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.389 DEBUG:rgw_multi.tests:version2 id=6ReZk8rcIxKcbtdG9110M0aCBVu-TYm 2022-09-03T05:29:09.389 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:29:09.390 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:29:09.390 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=6ReZk8rcIxKcbtdG9110M0aCBVu-TYm 2022-09-03T05:29:09.391 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=6ReZk8rcIxKcbtdG9110M0aCBVu-TYm 2022-09-03T05:29:09.391 DEBUG:boto:Method: DELETE 2022-09-03T05:29:09.392 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=6ReZk8rcIxKcbtdG9110M0aCBVu-TYm 2022-09-03T05:29:09.392 DEBUG:boto:Data: 2022-09-03T05:29:09.393 DEBUG:boto:Headers: {} 2022-09-03T05:29:09.393 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:09.394 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.394 DEBUG:boto:Params: {} 2022-09-03T05:29:09.395 DEBUG:boto:Token: None 2022-09-03T05:29:09.395 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-17/obj-test-zone1?versionId=6ReZk8rcIxKcbtdG9110M0aCBVu-TYm 2022-09-03T05:29:09.396 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:es3nkgK3RHDpXnxoAtB4iMstGTQ= 2022-09-03T05:29:09.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:es3nkgK3RHDpXnxoAtB4iMstGTQ=', 'Content-Length': '0'} 2022-09-03T05:29:09.409 DEBUG:boto:Response headers: [('x-amz-version-id', '6ReZk8rcIxKcbtdG9110M0aCBVu-TYm'), ('x-amz-request-id', 'tx0000016e51146d60eaca8-006312e625-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.411 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:29:09.411 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:29:09.412 DEBUG:boto:Method: PUT 2022-09-03T05:29:09.413 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1 2022-09-03T05:29:09.414 DEBUG:boto:Data: 2022-09-03T05:29:09.415 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:09.416 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:09.417 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.417 DEBUG:boto:Params: {} 2022-09-03T05:29:09.418 DEBUG:boto:Token: None 2022-09-03T05:29:09.419 DEBUG:boto:StringToSign: PUT HiNA5/HU7CwjRIhUZJ8d7Q== application/octet-stream Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-17/obj-test-zone1 2022-09-03T05:29:09.419 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WX0rQdJ5yJxUWIUu+i5m7bPisEQ= 2022-09-03T05:29:09.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WX0rQdJ5yJxUWIUu+i5m7bPisEQ='} 2022-09-03T05:29:09.462 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"1e2340e7f1d4ec2c23448854649f1ded"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'N.Yeu6qWy1agl0rtaUPgQceciedQmQp'), ('x-amz-request-id', 'tx00000deaf5cef1b42e0a3-006312e625-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.463 DEBUG:rgw_multi.tests:version3 id=N.Yeu6qWy1agl0rtaUPgQceciedQmQp 2022-09-03T05:29:09.465 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:29:09.466 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:29:09.467 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=N.Yeu6qWy1agl0rtaUPgQceciedQmQp 2022-09-03T05:29:09.467 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=N.Yeu6qWy1agl0rtaUPgQceciedQmQp 2022-09-03T05:29:09.468 DEBUG:boto:Method: DELETE 2022-09-03T05:29:09.469 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=N.Yeu6qWy1agl0rtaUPgQceciedQmQp 2022-09-03T05:29:09.470 DEBUG:boto:Data: 2022-09-03T05:29:09.471 DEBUG:boto:Headers: {} 2022-09-03T05:29:09.472 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:09.473 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.473 DEBUG:boto:Params: {} 2022-09-03T05:29:09.475 DEBUG:boto:Token: None 2022-09-03T05:29:09.476 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-17/obj-test-zone1?versionId=N.Yeu6qWy1agl0rtaUPgQceciedQmQp 2022-09-03T05:29:09.477 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/ZqFUEO6jyQCJY6Eq8Z94MhjLlQ= 2022-09-03T05:29:09.478 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/ZqFUEO6jyQCJY6Eq8Z94MhjLlQ=', 'Content-Length': '0'} 2022-09-03T05:29:09.499 DEBUG:boto:Response headers: [('x-amz-version-id', 'N.Yeu6qWy1agl0rtaUPgQceciedQmQp'), ('x-amz-request-id', 'tx000007dda0106ef8caa5b-006312e625-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.501 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:29:09.501 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:29:09.503 DEBUG:boto:Method: HEAD 2022-09-03T05:29:09.503 DEBUG:boto:Path: /dsgvcy-17/ 2022-09-03T05:29:09.504 DEBUG:boto:Data: 2022-09-03T05:29:09.505 DEBUG:boto:Headers: {} 2022-09-03T05:29:09.506 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:09.506 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.507 DEBUG:boto:Params: {} 2022-09-03T05:29:09.507 DEBUG:boto:Token: None 2022-09-03T05:29:09.508 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-17/ 2022-09-03T05:29:09.509 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UkQIFBflqf08XT4pgAmc4UoViDU= 2022-09-03T05:29:09.509 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UkQIFBflqf08XT4pgAmc4UoViDU=', 'Content-Length': '0'} 2022-09-03T05:29:09.513 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b4875885f16eef19-006312e625-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.515 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:29:09.516 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:29:09.517 DEBUG:boto:Method: PUT 2022-09-03T05:29:09.518 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2 2022-09-03T05:29:09.519 DEBUG:boto:Data: 2022-09-03T05:29:09.519 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:09.521 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:09.521 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.522 DEBUG:boto:Params: {} 2022-09-03T05:29:09.523 DEBUG:boto:Token: None 2022-09-03T05:29:09.524 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-17/obj-test-zone2 2022-09-03T05:29:09.524 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L3g0wmUEiQl8ti/esMutObNVUl4= 2022-09-03T05:29:09.525 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L3g0wmUEiQl8ti/esMutObNVUl4='} 2022-09-03T05:29:09.543 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-amz-request-id', 'tx000002d098d28e7ff7a58-006312e625-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.544 DEBUG:rgw_multi.tests:version1 id=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:29:09.545 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:29:09.546 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:29:09.547 DEBUG:boto:Method: PUT 2022-09-03T05:29:09.547 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2 2022-09-03T05:29:09.548 DEBUG:boto:Data: 2022-09-03T05:29:09.549 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:09.549 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:09.550 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.550 DEBUG:boto:Params: {} 2022-09-03T05:29:09.551 DEBUG:boto:Token: None 2022-09-03T05:29:09.551 DEBUG:boto:StringToSign: PUT Lg6VKF8IoH3qF+fuERshyA== application/octet-stream Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-17/obj-test-zone2 2022-09-03T05:29:09.552 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2w42cCAFbKxmjnbez0UGNlpSqh8= 2022-09-03T05:29:09.553 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2w42cCAFbKxmjnbez0UGNlpSqh8='} 2022-09-03T05:29:09.565 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"2e0e95285f08a07dea17e7ee111b21c8"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '5N4.OApC-HLvDa4wdhIagDBeASNpXif'), ('x-amz-request-id', 'tx000004481fe60d2c03642-006312e625-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.566 DEBUG:rgw_multi.tests:version2 id=5N4.OApC-HLvDa4wdhIagDBeASNpXif 2022-09-03T05:29:09.567 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:29:09.568 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:29:09.568 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=5N4.OApC-HLvDa4wdhIagDBeASNpXif 2022-09-03T05:29:09.569 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=5N4.OApC-HLvDa4wdhIagDBeASNpXif 2022-09-03T05:29:09.570 DEBUG:boto:Method: DELETE 2022-09-03T05:29:09.571 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=5N4.OApC-HLvDa4wdhIagDBeASNpXif 2022-09-03T05:29:09.572 DEBUG:boto:Data: 2022-09-03T05:29:09.573 DEBUG:boto:Headers: {} 2022-09-03T05:29:09.573 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:09.574 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.574 DEBUG:boto:Params: {} 2022-09-03T05:29:09.575 DEBUG:boto:Token: None 2022-09-03T05:29:09.576 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-17/obj-test-zone2?versionId=5N4.OApC-HLvDa4wdhIagDBeASNpXif 2022-09-03T05:29:09.576 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vq+lcaSugEib4on0Je0dgoKAr2k= 2022-09-03T05:29:09.577 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vq+lcaSugEib4on0Je0dgoKAr2k=', 'Content-Length': '0'} 2022-09-03T05:29:09.594 DEBUG:boto:Response headers: [('x-amz-version-id', '5N4.OApC-HLvDa4wdhIagDBeASNpXif'), ('x-amz-request-id', 'tx0000023557c14753502f7-006312e625-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.595 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:29:09.596 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:29:09.596 DEBUG:boto:Method: PUT 2022-09-03T05:29:09.597 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2 2022-09-03T05:29:09.597 DEBUG:boto:Data: 2022-09-03T05:29:09.598 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:09.599 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:09.599 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.600 DEBUG:boto:Params: {} 2022-09-03T05:29:09.600 DEBUG:boto:Token: None 2022-09-03T05:29:09.601 DEBUG:boto:StringToSign: PUT HiNA5/HU7CwjRIhUZJ8d7Q== application/octet-stream Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-17/obj-test-zone2 2022-09-03T05:29:09.602 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:55MeBElizZuEyIsHRXOmQSAtTWI= 2022-09-03T05:29:09.602 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:55MeBElizZuEyIsHRXOmQSAtTWI='} 2022-09-03T05:29:09.651 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"1e2340e7f1d4ec2c23448854649f1ded"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'fA.ii8asU1hE12tiEbJoR7-dzA1Q9Bv'), ('x-amz-request-id', 'tx00000de54d6b32e675eee-006312e625-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.652 DEBUG:rgw_multi.tests:version3 id=fA.ii8asU1hE12tiEbJoR7-dzA1Q9Bv 2022-09-03T05:29:09.653 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:29:09.653 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:29:09.654 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=fA.ii8asU1hE12tiEbJoR7-dzA1Q9Bv 2022-09-03T05:29:09.654 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=fA.ii8asU1hE12tiEbJoR7-dzA1Q9Bv 2022-09-03T05:29:09.655 DEBUG:boto:Method: DELETE 2022-09-03T05:29:09.655 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=fA.ii8asU1hE12tiEbJoR7-dzA1Q9Bv 2022-09-03T05:29:09.656 DEBUG:boto:Data: 2022-09-03T05:29:09.657 DEBUG:boto:Headers: {} 2022-09-03T05:29:09.658 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:09.658 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.659 DEBUG:boto:Params: {} 2022-09-03T05:29:09.660 DEBUG:boto:Token: None 2022-09-03T05:29:09.661 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-17/obj-test-zone2?versionId=fA.ii8asU1hE12tiEbJoR7-dzA1Q9Bv 2022-09-03T05:29:09.662 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BPN5yMiTLnxWe9V/edPkwWv9Vh0= 2022-09-03T05:29:09.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BPN5yMiTLnxWe9V/edPkwWv9Vh0=', 'Content-Length': '0'} 2022-09-03T05:29:09.677 DEBUG:boto:Response headers: [('x-amz-version-id', 'fA.ii8asU1hE12tiEbJoR7-dzA1Q9Bv'), ('x-amz-request-id', 'tx0000059bfcc3e96f66838-006312e625-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.678 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:29:09.679 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:29:09.679 DEBUG:boto:Method: HEAD 2022-09-03T05:29:09.680 DEBUG:boto:Path: /dsgvcy-17/ 2022-09-03T05:29:09.680 DEBUG:boto:Data: 2022-09-03T05:29:09.681 DEBUG:boto:Headers: {} 2022-09-03T05:29:09.681 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:09.682 DEBUG:boto:Port: 8001 2022-09-03T05:29:09.682 DEBUG:boto:Params: {} 2022-09-03T05:29:09.683 DEBUG:boto:Token: None 2022-09-03T05:29:09.684 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-17/ 2022-09-03T05:29:09.684 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UkQIFBflqf08XT4pgAmc4UoViDU= 2022-09-03T05:29:09.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UkQIFBflqf08XT4pgAmc4UoViDU=', 'Content-Length': '0'} 2022-09-03T05:29:09.687 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c9da4ffe100d78f5-006312e625-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.688 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:29:09.689 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:29:09.689 DEBUG:boto:Method: PUT 2022-09-03T05:29:09.690 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3 2022-09-03T05:29:09.691 DEBUG:boto:Data: 2022-09-03T05:29:09.692 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:09.692 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:09.693 DEBUG:boto:Port: 8001 2022-09-03T05:29:09.693 DEBUG:boto:Params: {} 2022-09-03T05:29:09.694 DEBUG:boto:Token: None 2022-09-03T05:29:09.695 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-17/obj-test-zone3 2022-09-03T05:29:09.696 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k6U7312c+b2mMBCMUphVfQ5o7pg= 2022-09-03T05:29:09.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k6U7312c+b2mMBCMUphVfQ5o7pg='} 2022-09-03T05:29:09.717 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-amz-request-id', 'tx00000921537761585cb7e-006312e625-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.719 DEBUG:rgw_multi.tests:version1 id=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:29:09.720 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:29:09.721 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:29:09.722 DEBUG:boto:Method: PUT 2022-09-03T05:29:09.723 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3 2022-09-03T05:29:09.724 DEBUG:boto:Data: 2022-09-03T05:29:09.725 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:09.725 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:09.727 DEBUG:boto:Port: 8001 2022-09-03T05:29:09.728 DEBUG:boto:Params: {} 2022-09-03T05:29:09.729 DEBUG:boto:Token: None 2022-09-03T05:29:09.730 DEBUG:boto:StringToSign: PUT Lg6VKF8IoH3qF+fuERshyA== application/octet-stream Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-17/obj-test-zone3 2022-09-03T05:29:09.731 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rqLr/EAtDuGH7Jv1uH4/N327e2g= 2022-09-03T05:29:09.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rqLr/EAtDuGH7Jv1uH4/N327e2g='} 2022-09-03T05:29:09.783 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"2e0e95285f08a07dea17e7ee111b21c8"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'Ukw-g-78mciY2cqu65V945jkJwhn-9O'), ('x-amz-request-id', 'tx00000267cef000e6ac0c6-006312e625-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.784 DEBUG:rgw_multi.tests:version2 id=Ukw-g-78mciY2cqu65V945jkJwhn-9O 2022-09-03T05:29:09.784 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:29:09.785 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:29:09.785 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=Ukw-g-78mciY2cqu65V945jkJwhn-9O 2022-09-03T05:29:09.786 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=Ukw-g-78mciY2cqu65V945jkJwhn-9O 2022-09-03T05:29:09.786 DEBUG:boto:Method: DELETE 2022-09-03T05:29:09.787 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=Ukw-g-78mciY2cqu65V945jkJwhn-9O 2022-09-03T05:29:09.788 DEBUG:boto:Data: 2022-09-03T05:29:09.789 DEBUG:boto:Headers: {} 2022-09-03T05:29:09.789 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:09.790 DEBUG:boto:Port: 8001 2022-09-03T05:29:09.791 DEBUG:boto:Params: {} 2022-09-03T05:29:09.791 DEBUG:boto:Token: None 2022-09-03T05:29:09.792 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-17/obj-test-zone3?versionId=Ukw-g-78mciY2cqu65V945jkJwhn-9O 2022-09-03T05:29:09.792 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zLJmu6ok+jnRcPV5v5le6ys+Vls= 2022-09-03T05:29:09.794 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zLJmu6ok+jnRcPV5v5le6ys+Vls=', 'Content-Length': '0'} 2022-09-03T05:29:09.814 DEBUG:boto:Response headers: [('x-amz-version-id', 'Ukw-g-78mciY2cqu65V945jkJwhn-9O'), ('x-amz-request-id', 'tx00000d64e4b69033bf3a9-006312e625-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.815 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:29:09.816 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:29:09.816 DEBUG:boto:Method: PUT 2022-09-03T05:29:09.817 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3 2022-09-03T05:29:09.817 DEBUG:boto:Data: 2022-09-03T05:29:09.818 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:09.818 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:09.819 DEBUG:boto:Port: 8001 2022-09-03T05:29:09.819 DEBUG:boto:Params: {} 2022-09-03T05:29:09.820 DEBUG:boto:Token: None 2022-09-03T05:29:09.821 DEBUG:boto:StringToSign: PUT HiNA5/HU7CwjRIhUZJ8d7Q== application/octet-stream Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-17/obj-test-zone3 2022-09-03T05:29:09.821 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wpqTljXtJn0DjqxfzAxCH18pniI= 2022-09-03T05:29:09.822 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wpqTljXtJn0DjqxfzAxCH18pniI='} 2022-09-03T05:29:09.839 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"1e2340e7f1d4ec2c23448854649f1ded"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'QJ3iopXCxVJG3eC7-Ds8VpJzT2adSlB'), ('x-amz-request-id', 'tx000008567417a665cafa4-006312e625-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.840 DEBUG:rgw_multi.tests:version3 id=QJ3iopXCxVJG3eC7-Ds8VpJzT2adSlB 2022-09-03T05:29:09.841 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:29:09.841 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:29:09.842 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=QJ3iopXCxVJG3eC7-Ds8VpJzT2adSlB 2022-09-03T05:29:09.843 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=QJ3iopXCxVJG3eC7-Ds8VpJzT2adSlB 2022-09-03T05:29:09.843 DEBUG:boto:Method: DELETE 2022-09-03T05:29:09.844 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=QJ3iopXCxVJG3eC7-Ds8VpJzT2adSlB 2022-09-03T05:29:09.844 DEBUG:boto:Data: 2022-09-03T05:29:09.845 DEBUG:boto:Headers: {} 2022-09-03T05:29:09.845 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:09.846 DEBUG:boto:Port: 8001 2022-09-03T05:29:09.848 DEBUG:boto:Params: {} 2022-09-03T05:29:09.848 DEBUG:boto:Token: None 2022-09-03T05:29:09.849 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-17/obj-test-zone3?versionId=QJ3iopXCxVJG3eC7-Ds8VpJzT2adSlB 2022-09-03T05:29:09.850 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YK81GfgGtw79NlQlY7dPz2SkhTI= 2022-09-03T05:29:09.850 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YK81GfgGtw79NlQlY7dPz2SkhTI=', 'Content-Length': '0'} 2022-09-03T05:29:09.862 DEBUG:boto:Response headers: [('x-amz-version-id', 'QJ3iopXCxVJG3eC7-Ds8VpJzT2adSlB'), ('x-amz-request-id', 'tx00000ac0a4d12006d3adc-006312e625-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.863 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:29:09.864 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:29:09.865 DEBUG:boto:Method: HEAD 2022-09-03T05:29:09.866 DEBUG:boto:Path: /dsgvcy-18/ 2022-09-03T05:29:09.866 DEBUG:boto:Data: 2022-09-03T05:29:09.867 DEBUG:boto:Headers: {} 2022-09-03T05:29:09.868 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:09.868 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.869 DEBUG:boto:Params: {} 2022-09-03T05:29:09.869 DEBUG:boto:Token: None 2022-09-03T05:29:09.870 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-18/ 2022-09-03T05:29:09.871 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6Q5fXeth4pJYq9AbInl49Kj29gA= 2022-09-03T05:29:09.871 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6Q5fXeth4pJYq9AbInl49Kj29gA=', 'Content-Length': '0'} 2022-09-03T05:29:09.874 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000074945e341f1996da-006312e625-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.875 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:29:09.875 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:29:09.876 DEBUG:boto:Method: PUT 2022-09-03T05:29:09.877 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1 2022-09-03T05:29:09.877 DEBUG:boto:Data: 2022-09-03T05:29:09.878 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:09.878 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:09.879 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.879 DEBUG:boto:Params: {} 2022-09-03T05:29:09.880 DEBUG:boto:Token: None 2022-09-03T05:29:09.881 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-18/obj-test-zone1 2022-09-03T05:29:09.881 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4RsQVGQ6F4jxoxWXi1hZOGljFWk= 2022-09-03T05:29:09.882 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4RsQVGQ6F4jxoxWXi1hZOGljFWk='} 2022-09-03T05:29:09.893 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-amz-request-id', 'tx00000dc839c91972d6ae2-006312e625-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.894 DEBUG:rgw_multi.tests:version1 id=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:29:09.895 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:29:09.896 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:29:09.897 DEBUG:boto:Method: PUT 2022-09-03T05:29:09.898 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1 2022-09-03T05:29:09.899 DEBUG:boto:Data: 2022-09-03T05:29:09.904 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:09.904 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:09.905 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.905 DEBUG:boto:Params: {} 2022-09-03T05:29:09.907 DEBUG:boto:Token: None 2022-09-03T05:29:09.907 DEBUG:boto:StringToSign: PUT Lg6VKF8IoH3qF+fuERshyA== application/octet-stream Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-18/obj-test-zone1 2022-09-03T05:29:09.908 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0ZcbdSSuVRZENsJXLiEEJv3Lu0Y= 2022-09-03T05:29:09.909 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0ZcbdSSuVRZENsJXLiEEJv3Lu0Y='} 2022-09-03T05:29:09.954 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"2e0e95285f08a07dea17e7ee111b21c8"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'zQGPR0qbBAUXrZ27O-EIV0lBBR8EC3g'), ('x-amz-request-id', 'tx000002a6f9101db749f67-006312e625-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.955 DEBUG:rgw_multi.tests:version2 id=zQGPR0qbBAUXrZ27O-EIV0lBBR8EC3g 2022-09-03T05:29:09.956 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:29:09.956 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:29:09.957 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=zQGPR0qbBAUXrZ27O-EIV0lBBR8EC3g 2022-09-03T05:29:09.958 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=zQGPR0qbBAUXrZ27O-EIV0lBBR8EC3g 2022-09-03T05:29:09.958 DEBUG:boto:Method: DELETE 2022-09-03T05:29:09.959 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=zQGPR0qbBAUXrZ27O-EIV0lBBR8EC3g 2022-09-03T05:29:09.959 DEBUG:boto:Data: 2022-09-03T05:29:09.960 DEBUG:boto:Headers: {} 2022-09-03T05:29:09.960 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:09.961 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.961 DEBUG:boto:Params: {} 2022-09-03T05:29:09.962 DEBUG:boto:Token: None 2022-09-03T05:29:09.963 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-18/obj-test-zone1?versionId=zQGPR0qbBAUXrZ27O-EIV0lBBR8EC3g 2022-09-03T05:29:09.963 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ltyn7P5tWDsgB4PhmMkdFaVkk80= 2022-09-03T05:29:09.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ltyn7P5tWDsgB4PhmMkdFaVkk80=', 'Content-Length': '0'} 2022-09-03T05:29:09.976 DEBUG:boto:Response headers: [('x-amz-version-id', 'zQGPR0qbBAUXrZ27O-EIV0lBBR8EC3g'), ('x-amz-request-id', 'tx000003632e9f5b8713e00-006312e625-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.977 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:29:09.978 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:29:09.978 DEBUG:boto:Method: PUT 2022-09-03T05:29:09.979 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1 2022-09-03T05:29:09.979 DEBUG:boto:Data: 2022-09-03T05:29:09.980 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:09.980 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:09.981 DEBUG:boto:Port: 8000 2022-09-03T05:29:09.981 DEBUG:boto:Params: {} 2022-09-03T05:29:09.982 DEBUG:boto:Token: None 2022-09-03T05:29:09.983 DEBUG:boto:StringToSign: PUT HiNA5/HU7CwjRIhUZJ8d7Q== application/octet-stream Sat, 03 Sep 2022 05:29:09 GMT /dsgvcy-18/obj-test-zone1 2022-09-03T05:29:09.984 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r1qf0VTlmdfPe5ZsRX2Uy94toLc= 2022-09-03T05:29:09.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r1qf0VTlmdfPe5ZsRX2Uy94toLc='} 2022-09-03T05:29:09.994 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"1e2340e7f1d4ec2c23448854649f1ded"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'k0VFpA0EN0z5QRIQDthJCNn.dqPQkHr'), ('x-amz-request-id', 'tx00000d3bc78efa660ed67-006312e625-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:09.995 DEBUG:rgw_multi.tests:version3 id=k0VFpA0EN0z5QRIQDthJCNn.dqPQkHr 2022-09-03T05:29:09.996 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:29:09.996 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:29:09.996 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=k0VFpA0EN0z5QRIQDthJCNn.dqPQkHr 2022-09-03T05:29:09.997 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=k0VFpA0EN0z5QRIQDthJCNn.dqPQkHr 2022-09-03T05:29:09.998 DEBUG:boto:Method: DELETE 2022-09-03T05:29:09.998 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=k0VFpA0EN0z5QRIQDthJCNn.dqPQkHr 2022-09-03T05:29:09.999 DEBUG:boto:Data: 2022-09-03T05:29:10.000 DEBUG:boto:Headers: {} 2022-09-03T05:29:10.002 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:10.003 DEBUG:boto:Port: 8000 2022-09-03T05:29:10.004 DEBUG:boto:Params: {} 2022-09-03T05:29:10.005 DEBUG:boto:Token: None 2022-09-03T05:29:10.006 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:29:10 GMT /dsgvcy-18/obj-test-zone1?versionId=k0VFpA0EN0z5QRIQDthJCNn.dqPQkHr 2022-09-03T05:29:10.007 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FQFs8g9KRDMw8LQ8iW1Fa3kPMuY= 2022-09-03T05:29:10.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FQFs8g9KRDMw8LQ8iW1Fa3kPMuY=', 'Content-Length': '0'} 2022-09-03T05:29:10.022 DEBUG:boto:Response headers: [('x-amz-version-id', 'k0VFpA0EN0z5QRIQDthJCNn.dqPQkHr'), ('x-amz-request-id', 'tx00000712f31b75e1701e2-006312e626-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:10.023 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:29:10.024 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:29:10.025 DEBUG:boto:Method: HEAD 2022-09-03T05:29:10.026 DEBUG:boto:Path: /dsgvcy-18/ 2022-09-03T05:29:10.026 DEBUG:boto:Data: 2022-09-03T05:29:10.027 DEBUG:boto:Headers: {} 2022-09-03T05:29:10.027 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:10.028 DEBUG:boto:Port: 8000 2022-09-03T05:29:10.028 DEBUG:boto:Params: {} 2022-09-03T05:29:10.029 DEBUG:boto:Token: None 2022-09-03T05:29:10.029 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:10 GMT /dsgvcy-18/ 2022-09-03T05:29:10.031 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9zB6f1E2TKTe9aavvU32xBZGKEw= 2022-09-03T05:29:10.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9zB6f1E2TKTe9aavvU32xBZGKEw=', 'Content-Length': '0'} 2022-09-03T05:29:10.035 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fae53b9ed3c96c73-006312e626-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:10.037 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:29:10.038 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:29:10.038 DEBUG:boto:Method: PUT 2022-09-03T05:29:10.039 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2 2022-09-03T05:29:10.039 DEBUG:boto:Data: 2022-09-03T05:29:10.040 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:10.040 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:10.041 DEBUG:boto:Port: 8000 2022-09-03T05:29:10.041 DEBUG:boto:Params: {} 2022-09-03T05:29:10.042 DEBUG:boto:Token: None 2022-09-03T05:29:10.043 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:29:10 GMT /dsgvcy-18/obj-test-zone2 2022-09-03T05:29:10.043 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:khQfwIXGjBeY1Fg0D5U/yIraFR0= 2022-09-03T05:29:10.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:khQfwIXGjBeY1Fg0D5U/yIraFR0='} 2022-09-03T05:29:10.067 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-amz-request-id', 'tx000008e226014ad1ca0c2-006312e626-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:10.068 DEBUG:rgw_multi.tests:version1 id=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:29:10.069 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:29:10.070 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:29:10.070 DEBUG:boto:Method: PUT 2022-09-03T05:29:10.071 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2 2022-09-03T05:29:10.071 DEBUG:boto:Data: 2022-09-03T05:29:10.072 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:10.072 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:10.073 DEBUG:boto:Port: 8000 2022-09-03T05:29:10.074 DEBUG:boto:Params: {} 2022-09-03T05:29:10.075 DEBUG:boto:Token: None 2022-09-03T05:29:10.077 DEBUG:boto:StringToSign: PUT Lg6VKF8IoH3qF+fuERshyA== application/octet-stream Sat, 03 Sep 2022 05:29:10 GMT /dsgvcy-18/obj-test-zone2 2022-09-03T05:29:10.078 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t4qDqDsKTALRWX1slef8CyJdtZQ= 2022-09-03T05:29:10.079 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t4qDqDsKTALRWX1slef8CyJdtZQ='} 2022-09-03T05:29:10.126 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"2e0e95285f08a07dea17e7ee111b21c8"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '1pRUeBKGkRflg6eSU-e9skOubVi0msT'), ('x-amz-request-id', 'tx000003fe6942c767a5f34-006312e626-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:10.127 DEBUG:rgw_multi.tests:version2 id=1pRUeBKGkRflg6eSU-e9skOubVi0msT 2022-09-03T05:29:10.128 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:29:10.128 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:29:10.129 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=1pRUeBKGkRflg6eSU-e9skOubVi0msT 2022-09-03T05:29:10.129 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=1pRUeBKGkRflg6eSU-e9skOubVi0msT 2022-09-03T05:29:10.129 DEBUG:boto:Method: DELETE 2022-09-03T05:29:10.130 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=1pRUeBKGkRflg6eSU-e9skOubVi0msT 2022-09-03T05:29:10.130 DEBUG:boto:Data: 2022-09-03T05:29:10.131 DEBUG:boto:Headers: {} 2022-09-03T05:29:10.131 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:10.132 DEBUG:boto:Port: 8000 2022-09-03T05:29:10.132 DEBUG:boto:Params: {} 2022-09-03T05:29:10.133 DEBUG:boto:Token: None 2022-09-03T05:29:10.133 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:29:10 GMT /dsgvcy-18/obj-test-zone2?versionId=1pRUeBKGkRflg6eSU-e9skOubVi0msT 2022-09-03T05:29:10.134 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D+Gn5FhKSfE9Jn3tE4vVQGRvvm8= 2022-09-03T05:29:10.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D+Gn5FhKSfE9Jn3tE4vVQGRvvm8=', 'Content-Length': '0'} 2022-09-03T05:29:10.152 DEBUG:boto:Response headers: [('x-amz-version-id', '1pRUeBKGkRflg6eSU-e9skOubVi0msT'), ('x-amz-request-id', 'tx000006edaf74917d7900d-006312e626-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:10.153 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:29:10.154 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:29:10.155 DEBUG:boto:Method: PUT 2022-09-03T05:29:10.155 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2 2022-09-03T05:29:10.156 DEBUG:boto:Data: 2022-09-03T05:29:10.156 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:10.157 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:10.157 DEBUG:boto:Port: 8000 2022-09-03T05:29:10.158 DEBUG:boto:Params: {} 2022-09-03T05:29:10.158 DEBUG:boto:Token: None 2022-09-03T05:29:10.159 DEBUG:boto:StringToSign: PUT HiNA5/HU7CwjRIhUZJ8d7Q== application/octet-stream Sat, 03 Sep 2022 05:29:10 GMT /dsgvcy-18/obj-test-zone2 2022-09-03T05:29:10.160 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:66etyPeklX55x6zMqC9GZIp+mvM= 2022-09-03T05:29:10.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:66etyPeklX55x6zMqC9GZIp+mvM='} 2022-09-03T05:29:10.172 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"1e2340e7f1d4ec2c23448854649f1ded"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '9ynIP0PGCb3AZGRxK7tEuRvAzBjg2MS'), ('x-amz-request-id', 'tx000000dd8fec08b1674ff-006312e626-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:10.173 DEBUG:rgw_multi.tests:version3 id=9ynIP0PGCb3AZGRxK7tEuRvAzBjg2MS 2022-09-03T05:29:10.174 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:29:10.174 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:29:10.175 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=9ynIP0PGCb3AZGRxK7tEuRvAzBjg2MS 2022-09-03T05:29:10.175 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=9ynIP0PGCb3AZGRxK7tEuRvAzBjg2MS 2022-09-03T05:29:10.176 DEBUG:boto:Method: DELETE 2022-09-03T05:29:10.176 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=9ynIP0PGCb3AZGRxK7tEuRvAzBjg2MS 2022-09-03T05:29:10.177 DEBUG:boto:Data: 2022-09-03T05:29:10.177 DEBUG:boto:Headers: {} 2022-09-03T05:29:10.178 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:10.179 DEBUG:boto:Port: 8000 2022-09-03T05:29:10.179 DEBUG:boto:Params: {} 2022-09-03T05:29:10.180 DEBUG:boto:Token: None 2022-09-03T05:29:10.180 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:29:10 GMT /dsgvcy-18/obj-test-zone2?versionId=9ynIP0PGCb3AZGRxK7tEuRvAzBjg2MS 2022-09-03T05:29:10.181 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4zWHeZQKWEjkPFYuIWfBWsTObG4= 2022-09-03T05:29:10.181 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4zWHeZQKWEjkPFYuIWfBWsTObG4=', 'Content-Length': '0'} 2022-09-03T05:29:10.192 DEBUG:boto:Response headers: [('x-amz-version-id', '9ynIP0PGCb3AZGRxK7tEuRvAzBjg2MS'), ('x-amz-request-id', 'tx00000cc76591029cc50d6-006312e626-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:10.193 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:29:10.193 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:29:10.194 DEBUG:boto:Method: HEAD 2022-09-03T05:29:10.194 DEBUG:boto:Path: /dsgvcy-18/ 2022-09-03T05:29:10.195 DEBUG:boto:Data: 2022-09-03T05:29:10.195 DEBUG:boto:Headers: {} 2022-09-03T05:29:10.196 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:10.197 DEBUG:boto:Port: 8001 2022-09-03T05:29:10.197 DEBUG:boto:Params: {} 2022-09-03T05:29:10.198 DEBUG:boto:Token: None 2022-09-03T05:29:10.198 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:10 GMT /dsgvcy-18/ 2022-09-03T05:29:10.199 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9zB6f1E2TKTe9aavvU32xBZGKEw= 2022-09-03T05:29:10.199 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9zB6f1E2TKTe9aavvU32xBZGKEw=', 'Content-Length': '0'} 2022-09-03T05:29:10.202 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f13baa3a435eedfd-006312e626-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:10.203 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:29:10.204 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:29:10.205 DEBUG:boto:Method: PUT 2022-09-03T05:29:10.206 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3 2022-09-03T05:29:10.207 DEBUG:boto:Data: 2022-09-03T05:29:10.208 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:10.209 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:10.210 DEBUG:boto:Port: 8001 2022-09-03T05:29:10.211 DEBUG:boto:Params: {} 2022-09-03T05:29:10.211 DEBUG:boto:Token: None 2022-09-03T05:29:10.212 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:29:10 GMT /dsgvcy-18/obj-test-zone3 2022-09-03T05:29:10.213 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qBlYzSrNDqtgyQmYIigCc3uwjNo= 2022-09-03T05:29:10.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qBlYzSrNDqtgyQmYIigCc3uwjNo='} 2022-09-03T05:29:10.230 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-amz-request-id', 'tx00000dba290f42d2e75f4-006312e626-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:10.230 DEBUG:rgw_multi.tests:version1 id=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:29:10.231 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:29:10.232 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:29:10.232 DEBUG:boto:Method: PUT 2022-09-03T05:29:10.233 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3 2022-09-03T05:29:10.234 DEBUG:boto:Data: 2022-09-03T05:29:10.234 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:10.235 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:10.236 DEBUG:boto:Port: 8001 2022-09-03T05:29:10.236 DEBUG:boto:Params: {} 2022-09-03T05:29:10.237 DEBUG:boto:Token: None 2022-09-03T05:29:10.238 DEBUG:boto:StringToSign: PUT Lg6VKF8IoH3qF+fuERshyA== application/octet-stream Sat, 03 Sep 2022 05:29:10 GMT /dsgvcy-18/obj-test-zone3 2022-09-03T05:29:10.239 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Pg8pybjFct2XBgZbEkAoqv5Aumo= 2022-09-03T05:29:10.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'Lg6VKF8IoH3qF+fuERshyA==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Pg8pybjFct2XBgZbEkAoqv5Aumo='} 2022-09-03T05:29:10.252 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"2e0e95285f08a07dea17e7ee111b21c8"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'pyA6BQi5zhbrJHy4S9gUBe0CMxC6mek'), ('x-amz-request-id', 'tx000001e985ca7cf752198-006312e626-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:10.252 DEBUG:rgw_multi.tests:version2 id=pyA6BQi5zhbrJHy4S9gUBe0CMxC6mek 2022-09-03T05:29:10.253 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:29:10.253 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:29:10.254 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=pyA6BQi5zhbrJHy4S9gUBe0CMxC6mek 2022-09-03T05:29:10.254 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=pyA6BQi5zhbrJHy4S9gUBe0CMxC6mek 2022-09-03T05:29:10.255 DEBUG:boto:Method: DELETE 2022-09-03T05:29:10.255 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=pyA6BQi5zhbrJHy4S9gUBe0CMxC6mek 2022-09-03T05:29:10.256 DEBUG:boto:Data: 2022-09-03T05:29:10.256 DEBUG:boto:Headers: {} 2022-09-03T05:29:10.257 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:10.257 DEBUG:boto:Port: 8001 2022-09-03T05:29:10.258 DEBUG:boto:Params: {} 2022-09-03T05:29:10.259 DEBUG:boto:Token: None 2022-09-03T05:29:10.259 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:29:10 GMT /dsgvcy-18/obj-test-zone3?versionId=pyA6BQi5zhbrJHy4S9gUBe0CMxC6mek 2022-09-03T05:29:10.260 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YJl7tsxOVCGFfp1UKcUa94nak94= 2022-09-03T05:29:10.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YJl7tsxOVCGFfp1UKcUa94nak94=', 'Content-Length': '0'} 2022-09-03T05:29:10.274 DEBUG:boto:Response headers: [('x-amz-version-id', 'pyA6BQi5zhbrJHy4S9gUBe0CMxC6mek'), ('x-amz-request-id', 'tx0000021a02f56308756a5-006312e626-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:10.275 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:29:10.276 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:29:10.277 DEBUG:boto:Method: PUT 2022-09-03T05:29:10.277 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3 2022-09-03T05:29:10.278 DEBUG:boto:Data: 2022-09-03T05:29:10.279 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:29:10.279 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:10.280 DEBUG:boto:Port: 8001 2022-09-03T05:29:10.280 DEBUG:boto:Params: {} 2022-09-03T05:29:10.281 DEBUG:boto:Token: None 2022-09-03T05:29:10.281 DEBUG:boto:StringToSign: PUT HiNA5/HU7CwjRIhUZJ8d7Q== application/octet-stream Sat, 03 Sep 2022 05:29:10 GMT /dsgvcy-18/obj-test-zone3 2022-09-03T05:29:10.282 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UEMLngRbMWe+Y4LtwFJFpWD1TNY= 2022-09-03T05:29:10.282 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'HiNA5/HU7CwjRIhUZJ8d7Q==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:29:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UEMLngRbMWe+Y4LtwFJFpWD1TNY='} 2022-09-03T05:29:10.330 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"1e2340e7f1d4ec2c23448854649f1ded"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '9cxt5CghhbLnpkIhWGaJ.FMswZGV0TN'), ('x-amz-request-id', 'tx000001b673ee14105ef71-006312e626-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:10.331 DEBUG:rgw_multi.tests:version3 id=9cxt5CghhbLnpkIhWGaJ.FMswZGV0TN 2022-09-03T05:29:10.332 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:29:10.332 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:29:10.333 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=9cxt5CghhbLnpkIhWGaJ.FMswZGV0TN 2022-09-03T05:29:10.333 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=9cxt5CghhbLnpkIhWGaJ.FMswZGV0TN 2022-09-03T05:29:10.334 DEBUG:boto:Method: DELETE 2022-09-03T05:29:10.335 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=9cxt5CghhbLnpkIhWGaJ.FMswZGV0TN 2022-09-03T05:29:10.335 DEBUG:boto:Data: 2022-09-03T05:29:10.336 DEBUG:boto:Headers: {} 2022-09-03T05:29:10.336 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:10.337 DEBUG:boto:Port: 8001 2022-09-03T05:29:10.337 DEBUG:boto:Params: {} 2022-09-03T05:29:10.338 DEBUG:boto:Token: None 2022-09-03T05:29:10.339 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:29:10 GMT /dsgvcy-18/obj-test-zone3?versionId=9cxt5CghhbLnpkIhWGaJ.FMswZGV0TN 2022-09-03T05:29:10.340 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DYuhbfWTvPCkpoTASuosspdNtT4= 2022-09-03T05:29:10.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DYuhbfWTvPCkpoTASuosspdNtT4=', 'Content-Length': '0'} 2022-09-03T05:29:10.357 DEBUG:boto:Response headers: [('x-amz-version-id', '9cxt5CghhbLnpkIhWGaJ.FMswZGV0TN'), ('x-amz-request-id', 'tx00000bb4bb5096188fe8d-006312e626-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:10.358 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-16 2022-09-03T05:29:10.358 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:29:10.359 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:29:10.360 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-16 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:29:10.420 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:29:10.421 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:29:10.454 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:29:10.456+0000 7f347da57600 1 waiting for incremental sync to catch up: 2022-09-03T05:29:10.454 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000004.13.9, , , , , ] 2022-09-03T05:29:10.455 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000010.42.5, , , , 00000000004.13.9, , , , , ] 2022-09-03T05:29:15.455 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:29:15.458+0000 7f347da57600 1 waiting for incremental sync to catch up: 2022-09-03T05:29:15.456 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000004.13.9, , , , , ] 2022-09-03T05:29:15.457 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000010.42.5, , , , 00000000004.13.9, , , , , ] 2022-09-03T05:29:20.457 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:29:20.460+0000 7f347da57600 1 waiting for incremental sync to catch up: 2022-09-03T05:29:20.457 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000004.13.9, , , , , ] 2022-09-03T05:29:20.458 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000010.42.5, , , , 00000000004.13.9, , , , , ] 2022-09-03T05:29:25.460 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:29:25.461+0000 7f347da57600 1 waiting for incremental sync to catch up: 2022-09-03T05:29:25.461 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000004.13.9, , , , , ] 2022-09-03T05:29:25.461 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000010.42.5, , , , 00000000004.13.9, , , , , ] 2022-09-03T05:29:30.461 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:29:30.463+0000 7f347da57600 1 bucket sync caught up with source: 2022-09-03T05:29:30.462 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000010.42.5, , 00000000002.6.10, , 00000000004.13.9, , , , , ] 2022-09-03T05:29:30.462 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000010.42.5, , , , 00000000004.13.9, , , , , ] 2022-09-03T05:29:30.463 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:29:30.463+0000 7f347da57600 0 bucket checkpoint complete 2022-09-03T05:29:30.476 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-16 2022-09-03T05:29:30.477 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:29:30.478 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:29:30.479 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-16 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:29:30.557 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:29:30.558 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:29:30.590 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:29:30.593+0000 7f11e8d79600 1 bucket sync caught up with source: 2022-09-03T05:29:30.590 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000010.42.5, , 00000000002.6.10, , 00000000004.13.9, , , , , ] 2022-09-03T05:29:30.590 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000010.42.5, , 00000000002.6.10, , 00000000004.13.9, , , , , ] 2022-09-03T05:29:30.591 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:29:30.593+0000 7f11e8d79600 0 bucket checkpoint complete 2022-09-03T05:29:30.603 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-16 2022-09-03T05:29:30.604 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:29:30.604 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:29:30.605 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-16 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:29:30.670 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:29:30.671 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:29:30.715 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:29:30.718+0000 7ff9a72f0600 1 waiting for incremental sync to catch up: 2022-09-03T05:29:30.716 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000010.42.5, , , , 00000000004.13.9, , , , , ] 2022-09-03T05:29:30.716 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000010.42.5, , 00000000002.6.10, , 00000000004.13.9, , , , , ] 2022-09-03T05:29:35.717 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:29:35.720+0000 7ff9a72f0600 1 waiting for incremental sync to catch up: 2022-09-03T05:29:35.718 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000010.42.5, , , , 00000000004.13.9, , , , , ] 2022-09-03T05:29:35.718 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000010.42.5, , 00000000002.6.10, , 00000000004.13.9, , , , , ] 2022-09-03T05:29:40.719 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:29:40.721+0000 7ff9a72f0600 1 bucket sync caught up with source: 2022-09-03T05:29:40.719 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000010.42.5, , 00000000002.6.10, , 00000000004.13.9, , , , , ] 2022-09-03T05:29:40.720 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000010.42.5, , 00000000002.6.10, , 00000000004.13.9, , , , , ] 2022-09-03T05:29:40.720 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:29:40.721+0000 7ff9a72f0600 0 bucket checkpoint complete 2022-09-03T05:29:40.740 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-16 2022-09-03T05:29:40.741 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:29:40.742 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:29:40.743 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-16 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:29:40.806 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:29:40.807 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:29:40.857 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:29:40.860+0000 7fc32f90d600 1 bucket sync caught up with source: 2022-09-03T05:29:40.858 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.68.10, , 00000000010.37.5, , 00000000004.28.9, , , , , ] 2022-09-03T05:29:40.858 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.68.10, , 00000000010.37.5, , 00000000004.28.9, , , , , ] 2022-09-03T05:29:40.858 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:29:40.860+0000 7fc32f90d600 0 bucket checkpoint complete 2022-09-03T05:29:40.871 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-16 2022-09-03T05:29:40.871 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:29:40.872 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:29:40.873 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-16 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:29:40.930 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:29:40.931 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:29:40.969 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:29:40.971+0000 7fd2be418600 1 bucket sync caught up with source: 2022-09-03T05:29:40.969 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.68.10, , 00000000010.37.5, , 00000000004.28.9, , , , , ] 2022-09-03T05:29:40.970 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.68.10, , 00000000010.37.5, , 00000000004.28.9, , , , , ] 2022-09-03T05:29:40.970 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:29:40.971+0000 7fd2be418600 0 bucket checkpoint complete 2022-09-03T05:29:40.983 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-16 2022-09-03T05:29:40.983 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-16 2022-09-03T05:29:40.985 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:29:40.986 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:29:40.987 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-16 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:29:41.044 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:29:41.044 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:29:41.096 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:29:41.099+0000 7f349a941600 1 waiting for incremental sync to catch up: 2022-09-03T05:29:41.096 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000004.17.9, , , , , ] 2022-09-03T05:29:41.097 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.62.9, , 00000000003.15.9, , 00000000014.37.5, , , , , ] 2022-09-03T05:29:46.097 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:29:46.100+0000 7f349a941600 1 bucket sync caught up with source: 2022-09-03T05:29:46.098 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.62.9, , 00000000003.15.9, , 00000000014.37.5, , , , , ] 2022-09-03T05:29:46.098 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.62.9, , 00000000003.15.9, , 00000000014.37.5, , , , , ] 2022-09-03T05:29:46.099 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:29:46.100+0000 7f349a941600 0 bucket checkpoint complete 2022-09-03T05:29:46.112 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-16 2022-09-03T05:29:46.112 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:29:46.113 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:29:46.114 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-16 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:29:46.178 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:29:46.179 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:29:46.229 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:29:46.232+0000 7fd841acd600 1 waiting for incremental sync to catch up: 2022-09-03T05:29:46.230 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , 00000000003.15.9, , 00000000004.17.9, , , , , ] 2022-09-03T05:29:46.230 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000004.62.9, , 00000000003.15.9, , 00000000014.37.5, , , , , ] 2022-09-03T05:29:51.231 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:29:51.233+0000 7fd841acd600 1 bucket sync caught up with source: 2022-09-03T05:29:51.232 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000004.62.9, , 00000000003.15.9, , 00000000014.37.5, , , , , ] 2022-09-03T05:29:51.232 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000004.62.9, , 00000000003.15.9, , 00000000014.37.5, , , , , ] 2022-09-03T05:29:51.232 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:29:51.233+0000 7fd841acd600 0 bucket checkpoint complete 2022-09-03T05:29:51.250 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-16 2022-09-03T05:29:51.251 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-16 zones={test-zone2, test-zone1} 2022-09-03T05:29:51.251 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:51.252 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:51.253 DEBUG:boto:Method: HEAD 2022-09-03T05:29:51.253 DEBUG:boto:Path: /dsgvcy-16/ 2022-09-03T05:29:51.254 DEBUG:boto:Data: 2022-09-03T05:29:51.254 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.255 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:51.256 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.257 DEBUG:boto:Params: {} 2022-09-03T05:29:51.257 DEBUG:boto:Token: None 2022-09-03T05:29:51.258 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/ 2022-09-03T05:29:51.259 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uwD35Hs7ue47yTBj5I02lSpPpp8= 2022-09-03T05:29:51.259 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uwD35Hs7ue47yTBj5I02lSpPpp8=', 'Content-Length': '0'} 2022-09-03T05:29:51.262 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e03f962e5c3eab73-006312e64f-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.263 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:51.264 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:51.264 DEBUG:boto:Method: HEAD 2022-09-03T05:29:51.265 DEBUG:boto:Path: /dsgvcy-16/ 2022-09-03T05:29:51.265 DEBUG:boto:Data: 2022-09-03T05:29:51.266 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.266 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:51.267 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.267 DEBUG:boto:Params: {} 2022-09-03T05:29:51.268 DEBUG:boto:Token: None 2022-09-03T05:29:51.268 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/ 2022-09-03T05:29:51.269 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uwD35Hs7ue47yTBj5I02lSpPpp8= 2022-09-03T05:29:51.270 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uwD35Hs7ue47yTBj5I02lSpPpp8=', 'Content-Length': '0'} 2022-09-03T05:29:51.272 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000343646f95e02e979-006312e64f-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.273 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:29:51.274 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:51.274 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:51.275 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:51.275 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:51.276 DEBUG:boto:Method: GET 2022-09-03T05:29:51.276 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:51.277 DEBUG:boto:Data: 2022-09-03T05:29:51.277 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.278 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:51.278 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.279 DEBUG:boto:Params: {} 2022-09-03T05:29:51.279 DEBUG:boto:Token: None 2022-09-03T05:29:51.280 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/?versions 2022-09-03T05:29:51.280 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:d/sH0Zk7m3RLamtuL7OQUACY4gw= 2022-09-03T05:29:51.281 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:d/sH0Zk7m3RLamtuL7OQUACY4gw=', 'Content-Length': '0'} 2022-09-03T05:29:51.284 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a307290cca15f786-006312e64f-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.284 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:51.286 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:29:51.287 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:29:51.287 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:29:51.287 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:29:51.288 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:29:51.288 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:51.289 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:51.289 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:51.290 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:51.290 DEBUG:boto:Method: GET 2022-09-03T05:29:51.291 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:51.291 DEBUG:boto:Data: 2022-09-03T05:29:51.292 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.292 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:51.293 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.293 DEBUG:boto:Params: {} 2022-09-03T05:29:51.294 DEBUG:boto:Token: None 2022-09-03T05:29:51.295 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/?versions 2022-09-03T05:29:51.295 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:d/sH0Zk7m3RLamtuL7OQUACY4gw= 2022-09-03T05:29:51.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:d/sH0Zk7m3RLamtuL7OQUACY4gw=', 'Content-Length': '0'} 2022-09-03T05:29:51.298 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cf4f09635eaa0665-006312e64f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.299 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:51.300 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:29:51.300 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:29:51.301 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:29:51.301 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:29:51.302 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:51.302 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:51.303 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:51.303 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:51.304 DEBUG:boto:Method: GET 2022-09-03T05:29:51.304 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:51.305 DEBUG:boto:Data: 2022-09-03T05:29:51.305 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.306 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:51.306 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.307 DEBUG:boto:Params: {} 2022-09-03T05:29:51.308 DEBUG:boto:Token: None 2022-09-03T05:29:51.309 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/?versions 2022-09-03T05:29:51.309 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:d/sH0Zk7m3RLamtuL7OQUACY4gw= 2022-09-03T05:29:51.310 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:d/sH0Zk7m3RLamtuL7OQUACY4gw=', 'Content-Length': '0'} 2022-09-03T05:29:51.313 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000069f5764abc7e05f6-006312e64f-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.313 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:51.314 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:51.315 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:51.316 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:51.317 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:51.318 DEBUG:boto:Method: GET 2022-09-03T05:29:51.319 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:51.319 DEBUG:boto:Data: 2022-09-03T05:29:51.320 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.321 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:51.322 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.323 DEBUG:boto:Params: {} 2022-09-03T05:29:51.324 DEBUG:boto:Token: None 2022-09-03T05:29:51.325 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/?versions 2022-09-03T05:29:51.325 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:d/sH0Zk7m3RLamtuL7OQUACY4gw= 2022-09-03T05:29:51.326 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:d/sH0Zk7m3RLamtuL7OQUACY4gw=', 'Content-Length': '0'} 2022-09-03T05:29:51.329 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000046cdaa96b5bfa239-006312e64f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.329 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:51.330 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:29:51.331 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:51.332 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:51.332 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.333 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.333 DEBUG:boto:Method: GET 2022-09-03T05:29:51.340 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.341 DEBUG:boto:Data: 2022-09-03T05:29:51.341 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.342 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:51.342 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.343 DEBUG:boto:Params: {} 2022-09-03T05:29:51.344 DEBUG:boto:Token: None 2022-09-03T05:29:51.344 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.345 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WZMYFy6kMrJuaL/fh5ePaSLgeJs= 2022-09-03T05:29:51.346 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WZMYFy6kMrJuaL/fh5ePaSLgeJs=', 'Content-Length': '0'} 2022-09-03T05:29:51.348 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000bfd4021bf05fc20c-006312e64f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.349 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:51.349 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:51.350 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.351 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.352 DEBUG:boto:Method: GET 2022-09-03T05:29:51.352 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.353 DEBUG:boto:Data: 2022-09-03T05:29:51.353 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.354 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:51.354 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.355 DEBUG:boto:Params: {} 2022-09-03T05:29:51.356 DEBUG:boto:Token: None 2022-09-03T05:29:51.356 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.357 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WZMYFy6kMrJuaL/fh5ePaSLgeJs= 2022-09-03T05:29:51.358 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WZMYFy6kMrJuaL/fh5ePaSLgeJs=', 'Content-Length': '0'} 2022-09-03T05:29:51.360 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000092b25e2cadc26060-006312e64f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.361 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:51.361 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:51.362 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.362 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.363 DEBUG:boto:Method: HEAD 2022-09-03T05:29:51.364 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.364 DEBUG:boto:Data: 2022-09-03T05:29:51.365 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.366 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:51.366 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.367 DEBUG:boto:Params: {} 2022-09-03T05:29:51.367 DEBUG:boto:Token: None 2022-09-03T05:29:51.368 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.369 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Pt8oHtieW+ypBR0lPJnqlCYWPRQ= 2022-09-03T05:29:51.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Pt8oHtieW+ypBR0lPJnqlCYWPRQ=', 'Content-Length': '0'} 2022-09-03T05:29:51.371 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000ea70cdea7dfe47e5-006312e64f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.372 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:51.373 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:51.373 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.374 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.375 DEBUG:boto:Method: HEAD 2022-09-03T05:29:51.375 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.376 DEBUG:boto:Data: 2022-09-03T05:29:51.377 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.377 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:51.378 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.379 DEBUG:boto:Params: {} 2022-09-03T05:29:51.379 DEBUG:boto:Token: None 2022-09-03T05:29:51.380 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.381 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Pt8oHtieW+ypBR0lPJnqlCYWPRQ= 2022-09-03T05:29:51.381 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Pt8oHtieW+ypBR0lPJnqlCYWPRQ=', 'Content-Length': '0'} 2022-09-03T05:29:51.384 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000bed96a1a6c270c1c-006312e64f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.385 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:29:51.385 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:51.386 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:51.387 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.388 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.389 DEBUG:boto:Method: GET 2022-09-03T05:29:51.390 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.391 DEBUG:boto:Data: 2022-09-03T05:29:51.392 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.392 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:51.393 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.393 DEBUG:boto:Params: {} 2022-09-03T05:29:51.395 DEBUG:boto:Token: None 2022-09-03T05:29:51.395 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.396 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WZMYFy6kMrJuaL/fh5ePaSLgeJs= 2022-09-03T05:29:51.397 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WZMYFy6kMrJuaL/fh5ePaSLgeJs=', 'Content-Length': '0'} 2022-09-03T05:29:51.399 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000d7d284da1cd962e1-006312e64f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.400 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:51.400 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:51.401 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.401 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.402 DEBUG:boto:Method: GET 2022-09-03T05:29:51.402 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.403 DEBUG:boto:Data: 2022-09-03T05:29:51.404 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.404 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:51.405 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.405 DEBUG:boto:Params: {} 2022-09-03T05:29:51.406 DEBUG:boto:Token: None 2022-09-03T05:29:51.407 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.407 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WZMYFy6kMrJuaL/fh5ePaSLgeJs= 2022-09-03T05:29:51.408 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WZMYFy6kMrJuaL/fh5ePaSLgeJs=', 'Content-Length': '0'} 2022-09-03T05:29:51.410 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000002cb6794cccd1e6c5-006312e64f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.411 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:51.412 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:51.413 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:51.413 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:51.414 DEBUG:boto:Method: GET 2022-09-03T05:29:51.414 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:29:51.415 DEBUG:boto:Data: 2022-09-03T05:29:51.415 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.416 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:51.417 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.418 DEBUG:boto:Params: {} 2022-09-03T05:29:51.419 DEBUG:boto:Token: None 2022-09-03T05:29:51.419 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/ 2022-09-03T05:29:51.420 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NXT5fudwcUEryIWszGqcgysBTGk= 2022-09-03T05:29:51.421 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NXT5fudwcUEryIWszGqcgysBTGk=', 'Content-Length': '0'} 2022-09-03T05:29:51.423 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000487ad1b54cd18e42-006312e64f-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.424 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:51.425 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:51.426 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:51.426 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:51.427 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:51.427 DEBUG:boto:Method: GET 2022-09-03T05:29:51.428 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:29:51.428 DEBUG:boto:Data: 2022-09-03T05:29:51.429 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.430 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:51.430 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.431 DEBUG:boto:Params: {} 2022-09-03T05:29:51.431 DEBUG:boto:Token: None 2022-09-03T05:29:51.432 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/ 2022-09-03T05:29:51.433 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NXT5fudwcUEryIWszGqcgysBTGk= 2022-09-03T05:29:51.433 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NXT5fudwcUEryIWszGqcgysBTGk=', 'Content-Length': '0'} 2022-09-03T05:29:51.436 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000090eb09b3a9ea0893-006312e64f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.436 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:51.437 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:51.438 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:51.438 DEBUG:boto:Method: HEAD 2022-09-03T05:29:51.439 DEBUG:boto:Path: /dsgvcy-16/dummy 2022-09-03T05:29:51.440 DEBUG:boto:Data: 2022-09-03T05:29:51.440 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.441 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:51.441 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.441 DEBUG:boto:Params: {} 2022-09-03T05:29:51.442 DEBUG:boto:Token: None 2022-09-03T05:29:51.443 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/dummy 2022-09-03T05:29:51.443 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sXaFgtgmqFyHWhvwBOSoT3ZzfCE= 2022-09-03T05:29:51.444 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sXaFgtgmqFyHWhvwBOSoT3ZzfCE=', 'Content-Length': '0'} 2022-09-03T05:29:51.446 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000fb74d41d78543d17-006312e64f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.448 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:51.448 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:51.449 DEBUG:boto:Method: HEAD 2022-09-03T05:29:51.449 DEBUG:boto:Path: /dsgvcy-16/dummy 2022-09-03T05:29:51.450 DEBUG:boto:Data: 2022-09-03T05:29:51.451 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.451 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:51.452 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.452 DEBUG:boto:Params: {} 2022-09-03T05:29:51.453 DEBUG:boto:Token: None 2022-09-03T05:29:51.453 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/dummy 2022-09-03T05:29:51.454 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sXaFgtgmqFyHWhvwBOSoT3ZzfCE= 2022-09-03T05:29:51.455 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sXaFgtgmqFyHWhvwBOSoT3ZzfCE=', 'Content-Length': '0'} 2022-09-03T05:29:51.459 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000001f6a3f33f30e42e7-006312e64f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.460 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:29:51.461 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:51.462 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:51.463 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.464 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.465 DEBUG:boto:Method: GET 2022-09-03T05:29:51.466 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.467 DEBUG:boto:Data: 2022-09-03T05:29:51.468 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.469 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:51.469 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.470 DEBUG:boto:Params: {} 2022-09-03T05:29:51.470 DEBUG:boto:Token: None 2022-09-03T05:29:51.471 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.472 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WZMYFy6kMrJuaL/fh5ePaSLgeJs= 2022-09-03T05:29:51.473 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WZMYFy6kMrJuaL/fh5ePaSLgeJs=', 'Content-Length': '0'} 2022-09-03T05:29:51.480 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000069270eb461d251c7-006312e64f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.481 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:51.481 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:51.482 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.483 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.483 DEBUG:boto:Method: GET 2022-09-03T05:29:51.484 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.484 DEBUG:boto:Data: 2022-09-03T05:29:51.485 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.485 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:51.486 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.487 DEBUG:boto:Params: {} 2022-09-03T05:29:51.487 DEBUG:boto:Token: None 2022-09-03T05:29:51.488 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:51.489 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WZMYFy6kMrJuaL/fh5ePaSLgeJs= 2022-09-03T05:29:51.490 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WZMYFy6kMrJuaL/fh5ePaSLgeJs=', 'Content-Length': '0'} 2022-09-03T05:29:51.492 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f8ba6bd84b16c759-006312e64f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.493 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:51.494 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:51.494 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:51.495 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:51.495 DEBUG:boto:Method: GET 2022-09-03T05:29:51.496 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:29:51.496 DEBUG:boto:Data: 2022-09-03T05:29:51.497 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.497 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:51.498 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.498 DEBUG:boto:Params: {} 2022-09-03T05:29:51.499 DEBUG:boto:Token: None 2022-09-03T05:29:51.500 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/ 2022-09-03T05:29:51.500 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NXT5fudwcUEryIWszGqcgysBTGk= 2022-09-03T05:29:51.501 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NXT5fudwcUEryIWszGqcgysBTGk=', 'Content-Length': '0'} 2022-09-03T05:29:51.505 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000160a0ca80361f8d1-006312e64f-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.546 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:51.548 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:51.548 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:51.549 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:51.550 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:51.551 DEBUG:boto:Method: GET 2022-09-03T05:29:51.552 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:29:51.552 DEBUG:boto:Data: 2022-09-03T05:29:51.553 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.553 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:51.554 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.555 DEBUG:boto:Params: {} 2022-09-03T05:29:51.556 DEBUG:boto:Token: None 2022-09-03T05:29:51.557 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/ 2022-09-03T05:29:51.558 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NXT5fudwcUEryIWszGqcgysBTGk= 2022-09-03T05:29:51.559 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NXT5fudwcUEryIWszGqcgysBTGk=', 'Content-Length': '0'} 2022-09-03T05:29:51.562 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000097f1dd0130040e55-006312e64f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.564 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:51.564 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:29:51.566 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.566 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.567 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.567 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.568 DEBUG:boto:Method: GET 2022-09-03T05:29:51.568 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.569 DEBUG:boto:Data: 2022-09-03T05:29:51.571 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.572 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:51.572 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.573 DEBUG:boto:Params: {} 2022-09-03T05:29:51.574 DEBUG:boto:Token: None 2022-09-03T05:29:51.574 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.576 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LZAxRkW9/VhAA5mxVePvyXBkhsA= 2022-09-03T05:29:51.577 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LZAxRkW9/VhAA5mxVePvyXBkhsA=', 'Content-Length': '0'} 2022-09-03T05:29:51.580 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008b5c57798976d01c-006312e64f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.581 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.582 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.583 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.584 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.585 DEBUG:boto:Method: GET 2022-09-03T05:29:51.585 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.587 DEBUG:boto:Data: 2022-09-03T05:29:51.588 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.589 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:51.589 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.590 DEBUG:boto:Params: {} 2022-09-03T05:29:51.591 DEBUG:boto:Token: None 2022-09-03T05:29:51.592 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.593 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LZAxRkW9/VhAA5mxVePvyXBkhsA= 2022-09-03T05:29:51.594 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LZAxRkW9/VhAA5mxVePvyXBkhsA=', 'Content-Length': '0'} 2022-09-03T05:29:51.597 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000017d2d88dc7c39d22-006312e64f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.599 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.600 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.600 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.601 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.602 DEBUG:boto:Method: HEAD 2022-09-03T05:29:51.603 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.604 DEBUG:boto:Data: 2022-09-03T05:29:51.605 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.606 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:51.607 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.608 DEBUG:boto:Params: {} 2022-09-03T05:29:51.609 DEBUG:boto:Token: None 2022-09-03T05:29:51.610 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.611 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SriyTSNvyYeFbm2OJVZetELOkxs= 2022-09-03T05:29:51.613 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SriyTSNvyYeFbm2OJVZetELOkxs=', 'Content-Length': '0'} 2022-09-03T05:29:51.615 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009cd63f7ff86f476a-006312e64f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.616 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.617 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.617 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.618 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.619 DEBUG:boto:Method: HEAD 2022-09-03T05:29:51.619 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.620 DEBUG:boto:Data: 2022-09-03T05:29:51.620 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.621 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:51.621 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.622 DEBUG:boto:Params: {} 2022-09-03T05:29:51.623 DEBUG:boto:Token: None 2022-09-03T05:29:51.624 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.625 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SriyTSNvyYeFbm2OJVZetELOkxs= 2022-09-03T05:29:51.627 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SriyTSNvyYeFbm2OJVZetELOkxs=', 'Content-Length': '0'} 2022-09-03T05:29:51.629 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000477f6d38dfbacbc3-006312e64f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.631 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:29:51.631 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.632 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.632 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.633 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.634 DEBUG:boto:Method: GET 2022-09-03T05:29:51.635 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.636 DEBUG:boto:Data: 2022-09-03T05:29:51.637 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.638 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:51.639 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.639 DEBUG:boto:Params: {} 2022-09-03T05:29:51.640 DEBUG:boto:Token: None 2022-09-03T05:29:51.641 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.643 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LZAxRkW9/VhAA5mxVePvyXBkhsA= 2022-09-03T05:29:51.643 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LZAxRkW9/VhAA5mxVePvyXBkhsA=', 'Content-Length': '0'} 2022-09-03T05:29:51.646 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006b8a89977a7602d8-006312e64f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.647 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.648 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.648 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.649 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.650 DEBUG:boto:Method: GET 2022-09-03T05:29:51.651 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.652 DEBUG:boto:Data: 2022-09-03T05:29:51.653 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.654 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:51.655 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.656 DEBUG:boto:Params: {} 2022-09-03T05:29:51.657 DEBUG:boto:Token: None 2022-09-03T05:29:51.658 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.659 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LZAxRkW9/VhAA5mxVePvyXBkhsA= 2022-09-03T05:29:51.660 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LZAxRkW9/VhAA5mxVePvyXBkhsA=', 'Content-Length': '0'} 2022-09-03T05:29:51.662 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000003dca6d40726fae4-006312e64f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.663 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:51.664 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:51.664 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:51.665 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:51.666 DEBUG:boto:Method: GET 2022-09-03T05:29:51.666 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:51.667 DEBUG:boto:Data: 2022-09-03T05:29:51.667 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.668 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:51.668 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.669 DEBUG:boto:Params: {} 2022-09-03T05:29:51.670 DEBUG:boto:Token: None 2022-09-03T05:29:51.670 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/ 2022-09-03T05:29:51.671 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NXT5fudwcUEryIWszGqcgysBTGk= 2022-09-03T05:29:51.671 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NXT5fudwcUEryIWszGqcgysBTGk=', 'Content-Length': '0'} 2022-09-03T05:29:51.674 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006f42c87e55e86a6f-006312e64f-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.675 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:51.676 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:51.676 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:51.677 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:51.677 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:51.678 DEBUG:boto:Method: GET 2022-09-03T05:29:51.678 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:51.679 DEBUG:boto:Data: 2022-09-03T05:29:51.679 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.680 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:51.680 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.681 DEBUG:boto:Params: {} 2022-09-03T05:29:51.682 DEBUG:boto:Token: None 2022-09-03T05:29:51.682 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/ 2022-09-03T05:29:51.683 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NXT5fudwcUEryIWszGqcgysBTGk= 2022-09-03T05:29:51.683 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NXT5fudwcUEryIWszGqcgysBTGk=', 'Content-Length': '0'} 2022-09-03T05:29:51.686 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000042be742bc0812221-006312e64f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.687 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:51.688 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.688 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.689 DEBUG:boto:Method: HEAD 2022-09-03T05:29:51.689 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.690 DEBUG:boto:Data: 2022-09-03T05:29:51.691 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.691 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:51.692 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.692 DEBUG:boto:Params: {} 2022-09-03T05:29:51.693 DEBUG:boto:Token: None 2022-09-03T05:29:51.693 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.694 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pQ2dXc5DC2hJfOSmHN5Sre3eOR0= 2022-09-03T05:29:51.695 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pQ2dXc5DC2hJfOSmHN5Sre3eOR0=', 'Content-Length': '0'} 2022-09-03T05:29:51.697 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000cb54b67f1fe52bf-006312e64f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.698 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.698 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.699 DEBUG:boto:Method: HEAD 2022-09-03T05:29:51.699 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.700 DEBUG:boto:Data: 2022-09-03T05:29:51.700 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.701 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:51.701 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.702 DEBUG:boto:Params: {} 2022-09-03T05:29:51.703 DEBUG:boto:Token: None 2022-09-03T05:29:51.703 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.704 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pQ2dXc5DC2hJfOSmHN5Sre3eOR0= 2022-09-03T05:29:51.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pQ2dXc5DC2hJfOSmHN5Sre3eOR0=', 'Content-Length': '0'} 2022-09-03T05:29:51.707 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004caf6592ebaeeabf-006312e64f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.708 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:29:51.708 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.709 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.709 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.710 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.710 DEBUG:boto:Method: GET 2022-09-03T05:29:51.711 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.713 DEBUG:boto:Data: 2022-09-03T05:29:51.713 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.714 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:51.715 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.716 DEBUG:boto:Params: {} 2022-09-03T05:29:51.718 DEBUG:boto:Token: None 2022-09-03T05:29:51.719 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.720 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LZAxRkW9/VhAA5mxVePvyXBkhsA= 2022-09-03T05:29:51.721 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LZAxRkW9/VhAA5mxVePvyXBkhsA=', 'Content-Length': '0'} 2022-09-03T05:29:51.723 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000066d6da262bbe32b6-006312e64f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.724 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.724 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:51.725 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.725 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.726 DEBUG:boto:Method: GET 2022-09-03T05:29:51.726 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.727 DEBUG:boto:Data: 2022-09-03T05:29:51.727 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.728 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:51.728 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.729 DEBUG:boto:Params: {} 2022-09-03T05:29:51.729 DEBUG:boto:Token: None 2022-09-03T05:29:51.730 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:51.731 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LZAxRkW9/VhAA5mxVePvyXBkhsA= 2022-09-03T05:29:51.731 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LZAxRkW9/VhAA5mxVePvyXBkhsA=', 'Content-Length': '0'} 2022-09-03T05:29:51.733 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000689cab3e68ec2f1d-006312e64f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.778 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:51.779 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:51.780 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:51.781 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:51.783 DEBUG:boto:Method: GET 2022-09-03T05:29:51.785 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:51.785 DEBUG:boto:Data: 2022-09-03T05:29:51.788 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.789 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:51.789 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.790 DEBUG:boto:Params: {} 2022-09-03T05:29:51.790 DEBUG:boto:Token: None 2022-09-03T05:29:51.791 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/ 2022-09-03T05:29:51.792 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NXT5fudwcUEryIWszGqcgysBTGk= 2022-09-03T05:29:51.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NXT5fudwcUEryIWszGqcgysBTGk=', 'Content-Length': '0'} 2022-09-03T05:29:51.795 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b61c163883801dd1-006312e64f-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.796 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:51.796 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:51.797 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:51.798 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:51.798 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:51.799 DEBUG:boto:Method: GET 2022-09-03T05:29:51.799 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:51.800 DEBUG:boto:Data: 2022-09-03T05:29:51.800 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.801 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:51.802 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.802 DEBUG:boto:Params: {} 2022-09-03T05:29:51.803 DEBUG:boto:Token: None 2022-09-03T05:29:51.804 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/ 2022-09-03T05:29:51.804 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NXT5fudwcUEryIWszGqcgysBTGk= 2022-09-03T05:29:51.805 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NXT5fudwcUEryIWszGqcgysBTGk=', 'Content-Length': '0'} 2022-09-03T05:29:51.809 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004191202cb76b5c36-006312e64f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.810 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:51.811 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:29:51.812 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:51.813 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:51.814 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.815 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.816 DEBUG:boto:Method: GET 2022-09-03T05:29:51.816 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.817 DEBUG:boto:Data: 2022-09-03T05:29:51.818 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.819 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:51.820 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.820 DEBUG:boto:Params: {} 2022-09-03T05:29:51.821 DEBUG:boto:Token: None 2022-09-03T05:29:51.821 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.822 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rMIvs+VOYybbhZ3JMs2Y3wmHVJs= 2022-09-03T05:29:51.823 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rMIvs+VOYybbhZ3JMs2Y3wmHVJs=', 'Content-Length': '0'} 2022-09-03T05:29:51.826 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000033b93b40bcde1c24-006312e64f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.866 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:51.867 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:51.868 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.868 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.869 DEBUG:boto:Method: GET 2022-09-03T05:29:51.869 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.870 DEBUG:boto:Data: 2022-09-03T05:29:51.870 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.871 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:51.871 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.872 DEBUG:boto:Params: {} 2022-09-03T05:29:51.873 DEBUG:boto:Token: None 2022-09-03T05:29:51.873 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.874 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rMIvs+VOYybbhZ3JMs2Y3wmHVJs= 2022-09-03T05:29:51.875 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rMIvs+VOYybbhZ3JMs2Y3wmHVJs=', 'Content-Length': '0'} 2022-09-03T05:29:51.877 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000048598c763cc44cea-006312e64f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.878 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:51.878 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:51.879 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.879 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.880 DEBUG:boto:Method: HEAD 2022-09-03T05:29:51.880 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.881 DEBUG:boto:Data: 2022-09-03T05:29:51.881 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.882 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:51.882 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.883 DEBUG:boto:Params: {} 2022-09-03T05:29:51.884 DEBUG:boto:Token: None 2022-09-03T05:29:51.885 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.885 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lqH/2CZzDDX6M4budPL+MBI4D9g= 2022-09-03T05:29:51.886 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lqH/2CZzDDX6M4budPL+MBI4D9g=', 'Content-Length': '0'} 2022-09-03T05:29:51.888 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000513104c88d9f5555-006312e64f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.889 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:51.889 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:51.890 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.890 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.891 DEBUG:boto:Method: HEAD 2022-09-03T05:29:51.891 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.892 DEBUG:boto:Data: 2022-09-03T05:29:51.892 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.893 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:51.893 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.894 DEBUG:boto:Params: {} 2022-09-03T05:29:51.895 DEBUG:boto:Token: None 2022-09-03T05:29:51.895 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.896 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lqH/2CZzDDX6M4budPL+MBI4D9g= 2022-09-03T05:29:51.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lqH/2CZzDDX6M4budPL+MBI4D9g=', 'Content-Length': '0'} 2022-09-03T05:29:51.902 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a10ec2301ba56444-006312e64f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.903 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:29:51.904 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:51.904 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:51.905 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.906 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.906 DEBUG:boto:Method: GET 2022-09-03T05:29:51.907 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.907 DEBUG:boto:Data: 2022-09-03T05:29:51.908 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.908 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:51.909 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.910 DEBUG:boto:Params: {} 2022-09-03T05:29:51.910 DEBUG:boto:Token: None 2022-09-03T05:29:51.911 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.911 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rMIvs+VOYybbhZ3JMs2Y3wmHVJs= 2022-09-03T05:29:51.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rMIvs+VOYybbhZ3JMs2Y3wmHVJs=', 'Content-Length': '0'} 2022-09-03T05:29:51.914 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007ce849217893108c-006312e64f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.915 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:51.916 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:51.916 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.917 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.917 DEBUG:boto:Method: GET 2022-09-03T05:29:51.918 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.918 DEBUG:boto:Data: 2022-09-03T05:29:51.919 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.920 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:51.920 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.921 DEBUG:boto:Params: {} 2022-09-03T05:29:51.921 DEBUG:boto:Token: None 2022-09-03T05:29:51.922 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.923 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rMIvs+VOYybbhZ3JMs2Y3wmHVJs= 2022-09-03T05:29:51.923 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rMIvs+VOYybbhZ3JMs2Y3wmHVJs=', 'Content-Length': '0'} 2022-09-03T05:29:51.925 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000080725d0f26afe2a1-006312e64f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.926 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:51.927 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:51.927 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:51.928 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:51.928 DEBUG:boto:Method: GET 2022-09-03T05:29:51.929 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:51.929 DEBUG:boto:Data: 2022-09-03T05:29:51.930 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.930 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:51.931 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.931 DEBUG:boto:Params: {} 2022-09-03T05:29:51.932 DEBUG:boto:Token: None 2022-09-03T05:29:51.933 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/ 2022-09-03T05:29:51.933 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NXT5fudwcUEryIWszGqcgysBTGk= 2022-09-03T05:29:51.934 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NXT5fudwcUEryIWszGqcgysBTGk=', 'Content-Length': '0'} 2022-09-03T05:29:51.936 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007634af1b20d7304b-006312e64f-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.937 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:51.938 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:51.939 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:51.939 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:51.940 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:51.940 DEBUG:boto:Method: GET 2022-09-03T05:29:51.941 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:51.941 DEBUG:boto:Data: 2022-09-03T05:29:51.942 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.942 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:51.943 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.943 DEBUG:boto:Params: {} 2022-09-03T05:29:51.944 DEBUG:boto:Token: None 2022-09-03T05:29:51.945 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/ 2022-09-03T05:29:51.946 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NXT5fudwcUEryIWszGqcgysBTGk= 2022-09-03T05:29:51.946 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NXT5fudwcUEryIWszGqcgysBTGk=', 'Content-Length': '0'} 2022-09-03T05:29:51.950 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007a6b1ffd3457bd97-006312e64f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.951 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:51.953 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:51.954 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:51.955 DEBUG:boto:Method: HEAD 2022-09-03T05:29:51.956 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2 2022-09-03T05:29:51.957 DEBUG:boto:Data: 2022-09-03T05:29:51.957 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.958 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:51.959 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.960 DEBUG:boto:Params: {} 2022-09-03T05:29:51.961 DEBUG:boto:Token: None 2022-09-03T05:29:51.962 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/obj-test-zone2 2022-09-03T05:29:51.963 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2TDaB1KcbrboLgP6kWEzAFAC6Ko= 2022-09-03T05:29:51.963 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2TDaB1KcbrboLgP6kWEzAFAC6Ko=', 'Content-Length': '0'} 2022-09-03T05:29:51.966 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d281c320c05fb2f4-006312e64f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.967 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:51.968 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:51.969 DEBUG:boto:Method: HEAD 2022-09-03T05:29:51.969 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2 2022-09-03T05:29:51.970 DEBUG:boto:Data: 2022-09-03T05:29:51.971 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.971 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:51.972 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.972 DEBUG:boto:Params: {} 2022-09-03T05:29:51.973 DEBUG:boto:Token: None 2022-09-03T05:29:51.974 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/obj-test-zone2 2022-09-03T05:29:51.974 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2TDaB1KcbrboLgP6kWEzAFAC6Ko= 2022-09-03T05:29:51.975 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2TDaB1KcbrboLgP6kWEzAFAC6Ko=', 'Content-Length': '0'} 2022-09-03T05:29:51.978 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000837bd1c05e2233e8-006312e64f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.979 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:29:51.980 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:51.981 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:51.982 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.983 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.984 DEBUG:boto:Method: GET 2022-09-03T05:29:51.985 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.986 DEBUG:boto:Data: 2022-09-03T05:29:51.987 DEBUG:boto:Headers: {} 2022-09-03T05:29:51.988 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:51.989 DEBUG:boto:Port: 8000 2022-09-03T05:29:51.990 DEBUG:boto:Params: {} 2022-09-03T05:29:51.991 DEBUG:boto:Token: None 2022-09-03T05:29:51.992 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:51 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:51.994 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rMIvs+VOYybbhZ3JMs2Y3wmHVJs= 2022-09-03T05:29:51.995 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rMIvs+VOYybbhZ3JMs2Y3wmHVJs=', 'Content-Length': '0'} 2022-09-03T05:29:51.998 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000089fc7b9c163ee82c-006312e650-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:51.999 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:52.000 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:52.001 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:52.002 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:52.003 DEBUG:boto:Method: GET 2022-09-03T05:29:52.004 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:52.005 DEBUG:boto:Data: 2022-09-03T05:29:52.005 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.006 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.007 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.008 DEBUG:boto:Params: {} 2022-09-03T05:29:52.009 DEBUG:boto:Token: None 2022-09-03T05:29:52.010 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:52.010 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:69BqvT7gH06f1oQrhpc2lYx7LPA= 2022-09-03T05:29:52.011 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:69BqvT7gH06f1oQrhpc2lYx7LPA=', 'Content-Length': '0'} 2022-09-03T05:29:52.014 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e23853eeb73f089e-006312e650-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.054 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:52.056 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:52.056 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:52.057 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:52.058 DEBUG:boto:Method: GET 2022-09-03T05:29:52.059 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:52.060 DEBUG:boto:Data: 2022-09-03T05:29:52.061 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.061 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:52.062 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.062 DEBUG:boto:Params: {} 2022-09-03T05:29:52.063 DEBUG:boto:Token: None 2022-09-03T05:29:52.063 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/ 2022-09-03T05:29:52.064 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM= 2022-09-03T05:29:52.065 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM=', 'Content-Length': '0'} 2022-09-03T05:29:52.069 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007546c8f11f4b2933-006312e650-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.070 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:52.072 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:52.072 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:52.073 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:52.073 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:52.074 DEBUG:boto:Method: GET 2022-09-03T05:29:52.074 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:52.075 DEBUG:boto:Data: 2022-09-03T05:29:52.076 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.076 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.077 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.078 DEBUG:boto:Params: {} 2022-09-03T05:29:52.079 DEBUG:boto:Token: None 2022-09-03T05:29:52.080 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/ 2022-09-03T05:29:52.081 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM= 2022-09-03T05:29:52.083 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM=', 'Content-Length': '0'} 2022-09-03T05:29:52.086 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000010f8a8c5a63e2ca5-006312e650-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.087 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:52.089 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:29:52.090 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.091 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.092 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.093 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.094 DEBUG:boto:Method: GET 2022-09-03T05:29:52.095 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.096 DEBUG:boto:Data: 2022-09-03T05:29:52.097 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.099 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:52.099 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.100 DEBUG:boto:Params: {} 2022-09-03T05:29:52.101 DEBUG:boto:Token: None 2022-09-03T05:29:52.102 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.102 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1I20iLJzkZ7l+MmFVUwLIDHIpN8= 2022-09-03T05:29:52.103 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1I20iLJzkZ7l+MmFVUwLIDHIpN8=', 'Content-Length': '0'} 2022-09-03T05:29:52.105 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e509b2ca2b4ec975-006312e650-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.107 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.108 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.109 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.110 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.111 DEBUG:boto:Method: GET 2022-09-03T05:29:52.112 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.113 DEBUG:boto:Data: 2022-09-03T05:29:52.114 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.115 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.116 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.116 DEBUG:boto:Params: {} 2022-09-03T05:29:52.117 DEBUG:boto:Token: None 2022-09-03T05:29:52.118 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.119 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1I20iLJzkZ7l+MmFVUwLIDHIpN8= 2022-09-03T05:29:52.120 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1I20iLJzkZ7l+MmFVUwLIDHIpN8=', 'Content-Length': '0'} 2022-09-03T05:29:52.123 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000958606fcce53e10c-006312e650-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.124 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.125 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.126 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.127 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.128 DEBUG:boto:Method: HEAD 2022-09-03T05:29:52.128 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.129 DEBUG:boto:Data: 2022-09-03T05:29:52.129 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.129 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:52.130 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.131 DEBUG:boto:Params: {} 2022-09-03T05:29:52.131 DEBUG:boto:Token: None 2022-09-03T05:29:52.132 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.133 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hO+5CWCX/2sxVgC/xBc0Rj55/d8= 2022-09-03T05:29:52.134 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hO+5CWCX/2sxVgC/xBc0Rj55/d8=', 'Content-Length': '0'} 2022-09-03T05:29:52.137 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bde3613d3e7f643e-006312e650-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.138 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.139 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.140 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.141 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.142 DEBUG:boto:Method: HEAD 2022-09-03T05:29:52.143 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.144 DEBUG:boto:Data: 2022-09-03T05:29:52.144 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.145 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.145 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.146 DEBUG:boto:Params: {} 2022-09-03T05:29:52.146 DEBUG:boto:Token: None 2022-09-03T05:29:52.147 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.148 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hO+5CWCX/2sxVgC/xBc0Rj55/d8= 2022-09-03T05:29:52.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hO+5CWCX/2sxVgC/xBc0Rj55/d8=', 'Content-Length': '0'} 2022-09-03T05:29:52.150 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a3e3087a0cf9968f-006312e650-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.151 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:29:52.152 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.152 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.153 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.153 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.154 DEBUG:boto:Method: GET 2022-09-03T05:29:52.155 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.155 DEBUG:boto:Data: 2022-09-03T05:29:52.156 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.156 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:52.157 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.157 DEBUG:boto:Params: {} 2022-09-03T05:29:52.158 DEBUG:boto:Token: None 2022-09-03T05:29:52.159 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.159 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1I20iLJzkZ7l+MmFVUwLIDHIpN8= 2022-09-03T05:29:52.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1I20iLJzkZ7l+MmFVUwLIDHIpN8=', 'Content-Length': '0'} 2022-09-03T05:29:52.163 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000433349e62ba55a02-006312e650-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.164 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.165 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.165 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.166 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.166 DEBUG:boto:Method: GET 2022-09-03T05:29:52.167 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.167 DEBUG:boto:Data: 2022-09-03T05:29:52.168 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.169 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.169 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.170 DEBUG:boto:Params: {} 2022-09-03T05:29:52.170 DEBUG:boto:Token: None 2022-09-03T05:29:52.171 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.172 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1I20iLJzkZ7l+MmFVUwLIDHIpN8= 2022-09-03T05:29:52.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1I20iLJzkZ7l+MmFVUwLIDHIpN8=', 'Content-Length': '0'} 2022-09-03T05:29:52.174 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b7e06b78fde0c5c6-006312e650-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.222 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:52.224 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:52.224 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:52.225 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:52.225 DEBUG:boto:Method: GET 2022-09-03T05:29:52.226 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:52.226 DEBUG:boto:Data: 2022-09-03T05:29:52.227 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.228 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:52.229 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.230 DEBUG:boto:Params: {} 2022-09-03T05:29:52.230 DEBUG:boto:Token: None 2022-09-03T05:29:52.231 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/ 2022-09-03T05:29:52.231 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM= 2022-09-03T05:29:52.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM=', 'Content-Length': '0'} 2022-09-03T05:29:52.235 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000008d8e9fb0e6f5e47-006312e650-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.236 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:52.236 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:52.237 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:52.238 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:52.238 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:52.239 DEBUG:boto:Method: GET 2022-09-03T05:29:52.239 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:52.240 DEBUG:boto:Data: 2022-09-03T05:29:52.240 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.241 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.241 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.242 DEBUG:boto:Params: {} 2022-09-03T05:29:52.243 DEBUG:boto:Token: None 2022-09-03T05:29:52.243 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/ 2022-09-03T05:29:52.244 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM= 2022-09-03T05:29:52.245 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM=', 'Content-Length': '0'} 2022-09-03T05:29:52.248 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000919b3c7206d02fd9-006312e650-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.249 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:52.249 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.250 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.251 DEBUG:boto:Method: HEAD 2022-09-03T05:29:52.251 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.252 DEBUG:boto:Data: 2022-09-03T05:29:52.252 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.253 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:52.253 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.254 DEBUG:boto:Params: {} 2022-09-03T05:29:52.254 DEBUG:boto:Token: None 2022-09-03T05:29:52.255 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.255 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:em/OFqX6cjKTrrGbte9DjCnpwoM= 2022-09-03T05:29:52.256 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:em/OFqX6cjKTrrGbte9DjCnpwoM=', 'Content-Length': '0'} 2022-09-03T05:29:52.258 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dc6a85b9393c06b3-006312e650-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.259 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.260 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.260 DEBUG:boto:Method: HEAD 2022-09-03T05:29:52.261 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.261 DEBUG:boto:Data: 2022-09-03T05:29:52.262 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.262 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.263 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.263 DEBUG:boto:Params: {} 2022-09-03T05:29:52.264 DEBUG:boto:Token: None 2022-09-03T05:29:52.264 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.265 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:em/OFqX6cjKTrrGbte9DjCnpwoM= 2022-09-03T05:29:52.266 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:em/OFqX6cjKTrrGbte9DjCnpwoM=', 'Content-Length': '0'} 2022-09-03T05:29:52.268 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d95d0244ed0f6664-006312e650-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.269 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:29:52.270 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.270 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.271 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.272 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.272 DEBUG:boto:Method: GET 2022-09-03T05:29:52.273 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.273 DEBUG:boto:Data: 2022-09-03T05:29:52.274 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.274 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:52.275 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.275 DEBUG:boto:Params: {} 2022-09-03T05:29:52.276 DEBUG:boto:Token: None 2022-09-03T05:29:52.277 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.277 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1I20iLJzkZ7l+MmFVUwLIDHIpN8= 2022-09-03T05:29:52.278 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1I20iLJzkZ7l+MmFVUwLIDHIpN8=', 'Content-Length': '0'} 2022-09-03T05:29:52.280 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000023180f7874dfc9a0-006312e650-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.323 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.324 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:52.325 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.326 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.327 DEBUG:boto:Method: GET 2022-09-03T05:29:52.328 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.329 DEBUG:boto:Data: 2022-09-03T05:29:52.330 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.331 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.333 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.334 DEBUG:boto:Params: {} 2022-09-03T05:29:52.335 DEBUG:boto:Token: None 2022-09-03T05:29:52.336 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:52.337 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1I20iLJzkZ7l+MmFVUwLIDHIpN8= 2022-09-03T05:29:52.338 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1I20iLJzkZ7l+MmFVUwLIDHIpN8=', 'Content-Length': '0'} 2022-09-03T05:29:52.341 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009a7d08b6a39c93e8-006312e650-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.342 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:52.343 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:52.344 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:52.346 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:52.347 DEBUG:boto:Method: GET 2022-09-03T05:29:52.348 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:52.349 DEBUG:boto:Data: 2022-09-03T05:29:52.350 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.351 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:52.352 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.352 DEBUG:boto:Params: {} 2022-09-03T05:29:52.353 DEBUG:boto:Token: None 2022-09-03T05:29:52.354 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/ 2022-09-03T05:29:52.356 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM= 2022-09-03T05:29:52.357 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM=', 'Content-Length': '0'} 2022-09-03T05:29:52.361 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ba00c15527296f32-006312e650-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.363 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:52.365 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:52.366 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:52.367 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:52.368 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:52.369 DEBUG:boto:Method: GET 2022-09-03T05:29:52.370 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:52.371 DEBUG:boto:Data: 2022-09-03T05:29:52.372 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.373 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.373 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.374 DEBUG:boto:Params: {} 2022-09-03T05:29:52.375 DEBUG:boto:Token: None 2022-09-03T05:29:52.375 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/ 2022-09-03T05:29:52.376 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM= 2022-09-03T05:29:52.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM=', 'Content-Length': '0'} 2022-09-03T05:29:52.380 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b2a6d38602797b2c-006312e650-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.381 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:52.381 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-16 zones={test-zone2, test-zone1} 2022-09-03T05:29:52.382 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-16 zones={test-zone3, test-zone1} 2022-09-03T05:29:52.383 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:52.383 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:52.384 DEBUG:boto:Method: HEAD 2022-09-03T05:29:52.384 DEBUG:boto:Path: /dsgvcy-16/ 2022-09-03T05:29:52.385 DEBUG:boto:Data: 2022-09-03T05:29:52.385 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.386 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:52.386 DEBUG:boto:Port: 8001 2022-09-03T05:29:52.387 DEBUG:boto:Params: {} 2022-09-03T05:29:52.387 DEBUG:boto:Token: None 2022-09-03T05:29:52.388 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/ 2022-09-03T05:29:52.389 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VmAwWUjkKzMamKmY+BgJTfszzIo= 2022-09-03T05:29:52.389 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VmAwWUjkKzMamKmY+BgJTfszzIo=', 'Content-Length': '0'} 2022-09-03T05:29:52.392 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007c01975628e8c9f2-006312e650-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.393 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:52.394 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:52.394 DEBUG:boto:Method: HEAD 2022-09-03T05:29:52.395 DEBUG:boto:Path: /dsgvcy-16/ 2022-09-03T05:29:52.395 DEBUG:boto:Data: 2022-09-03T05:29:52.396 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.396 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.397 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.398 DEBUG:boto:Params: {} 2022-09-03T05:29:52.398 DEBUG:boto:Token: None 2022-09-03T05:29:52.399 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/ 2022-09-03T05:29:52.399 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VmAwWUjkKzMamKmY+BgJTfszzIo= 2022-09-03T05:29:52.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VmAwWUjkKzMamKmY+BgJTfszzIo=', 'Content-Length': '0'} 2022-09-03T05:29:52.403 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000076d96cb11d79c050-006312e650-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.403 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:29:52.404 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:52.404 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:52.405 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:52.405 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:52.406 DEBUG:boto:Method: GET 2022-09-03T05:29:52.406 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:52.407 DEBUG:boto:Data: 2022-09-03T05:29:52.407 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.408 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:52.408 DEBUG:boto:Port: 8001 2022-09-03T05:29:52.409 DEBUG:boto:Params: {} 2022-09-03T05:29:52.410 DEBUG:boto:Token: None 2022-09-03T05:29:52.410 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/?versions 2022-09-03T05:29:52.411 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N6XnKz2TSEz7WbkYVabCjhHC3Eo= 2022-09-03T05:29:52.411 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N6XnKz2TSEz7WbkYVabCjhHC3Eo=', 'Content-Length': '0'} 2022-09-03T05:29:52.415 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007bccafff073c23f0-006312e650-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.416 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:52.418 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:29:52.419 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:29:52.419 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:29:52.420 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:29:52.421 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:29:52.422 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:52.423 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:52.423 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:52.424 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:52.425 DEBUG:boto:Method: GET 2022-09-03T05:29:52.426 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:52.427 DEBUG:boto:Data: 2022-09-03T05:29:52.428 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.428 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.429 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.429 DEBUG:boto:Params: {} 2022-09-03T05:29:52.430 DEBUG:boto:Token: None 2022-09-03T05:29:52.431 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/?versions 2022-09-03T05:29:52.432 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N6XnKz2TSEz7WbkYVabCjhHC3Eo= 2022-09-03T05:29:52.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N6XnKz2TSEz7WbkYVabCjhHC3Eo=', 'Content-Length': '0'} 2022-09-03T05:29:52.436 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000526fb2eb7cd995ba-006312e650-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.437 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:52.439 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:29:52.439 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:29:52.440 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:29:52.441 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:29:52.442 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:52.443 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:52.443 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:52.444 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:52.444 DEBUG:boto:Method: GET 2022-09-03T05:29:52.445 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:52.445 DEBUG:boto:Data: 2022-09-03T05:29:52.446 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.446 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:52.447 DEBUG:boto:Port: 8001 2022-09-03T05:29:52.448 DEBUG:boto:Params: {} 2022-09-03T05:29:52.449 DEBUG:boto:Token: None 2022-09-03T05:29:52.450 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/?versions 2022-09-03T05:29:52.451 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N6XnKz2TSEz7WbkYVabCjhHC3Eo= 2022-09-03T05:29:52.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N6XnKz2TSEz7WbkYVabCjhHC3Eo=', 'Content-Length': '0'} 2022-09-03T05:29:52.455 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000eebe0d4043bca3c-006312e650-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.456 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:52.458 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:52.459 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:52.461 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:52.461 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:52.462 DEBUG:boto:Method: GET 2022-09-03T05:29:52.464 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:52.464 DEBUG:boto:Data: 2022-09-03T05:29:52.466 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.466 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.467 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.467 DEBUG:boto:Params: {} 2022-09-03T05:29:52.468 DEBUG:boto:Token: None 2022-09-03T05:29:52.469 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/?versions 2022-09-03T05:29:52.469 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N6XnKz2TSEz7WbkYVabCjhHC3Eo= 2022-09-03T05:29:52.470 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N6XnKz2TSEz7WbkYVabCjhHC3Eo=', 'Content-Length': '0'} 2022-09-03T05:29:52.473 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000331c4ef9c29979fd-006312e650-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.474 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:52.475 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:29:52.476 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:52.476 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:52.477 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.477 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.478 DEBUG:boto:Method: GET 2022-09-03T05:29:52.478 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.479 DEBUG:boto:Data: 2022-09-03T05:29:52.480 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.480 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:52.481 DEBUG:boto:Port: 8001 2022-09-03T05:29:52.482 DEBUG:boto:Params: {} 2022-09-03T05:29:52.485 DEBUG:boto:Token: None 2022-09-03T05:29:52.487 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.488 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:P2HvUV1SI941YwLzoxJjavGqL98= 2022-09-03T05:29:52.489 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:P2HvUV1SI941YwLzoxJjavGqL98=', 'Content-Length': '0'} 2022-09-03T05:29:52.491 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000db4a4068626f8921-006312e650-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.492 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:52.492 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:52.493 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.494 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.494 DEBUG:boto:Method: GET 2022-09-03T05:29:52.495 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.495 DEBUG:boto:Data: 2022-09-03T05:29:52.496 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.497 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.497 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.498 DEBUG:boto:Params: {} 2022-09-03T05:29:52.499 DEBUG:boto:Token: None 2022-09-03T05:29:52.500 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.501 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:P2HvUV1SI941YwLzoxJjavGqL98= 2022-09-03T05:29:52.502 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:P2HvUV1SI941YwLzoxJjavGqL98=', 'Content-Length': '0'} 2022-09-03T05:29:52.504 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000d448cfa69f51a3dd-006312e650-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.506 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:52.507 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:52.507 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.508 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.508 DEBUG:boto:Method: HEAD 2022-09-03T05:29:52.509 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.509 DEBUG:boto:Data: 2022-09-03T05:29:52.510 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.510 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:52.511 DEBUG:boto:Port: 8001 2022-09-03T05:29:52.512 DEBUG:boto:Params: {} 2022-09-03T05:29:52.512 DEBUG:boto:Token: None 2022-09-03T05:29:52.513 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.514 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:si5T1ietwgkO0EuUdl+c0XJwpNg= 2022-09-03T05:29:52.514 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:si5T1ietwgkO0EuUdl+c0XJwpNg=', 'Content-Length': '0'} 2022-09-03T05:29:52.516 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000006f0552ad50da68c6-006312e650-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.517 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:52.518 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:52.518 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.519 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.519 DEBUG:boto:Method: HEAD 2022-09-03T05:29:52.520 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.520 DEBUG:boto:Data: 2022-09-03T05:29:52.521 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.522 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.522 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.523 DEBUG:boto:Params: {} 2022-09-03T05:29:52.523 DEBUG:boto:Token: None 2022-09-03T05:29:52.524 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.525 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:si5T1ietwgkO0EuUdl+c0XJwpNg= 2022-09-03T05:29:52.525 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:si5T1ietwgkO0EuUdl+c0XJwpNg=', 'Content-Length': '0'} 2022-09-03T05:29:52.527 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000eba1fb050049e247-006312e650-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.528 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:29:52.529 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:52.529 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:52.530 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.530 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.531 DEBUG:boto:Method: GET 2022-09-03T05:29:52.531 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.532 DEBUG:boto:Data: 2022-09-03T05:29:52.533 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.534 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:52.535 DEBUG:boto:Port: 8001 2022-09-03T05:29:52.536 DEBUG:boto:Params: {} 2022-09-03T05:29:52.537 DEBUG:boto:Token: None 2022-09-03T05:29:52.538 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.539 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:P2HvUV1SI941YwLzoxJjavGqL98= 2022-09-03T05:29:52.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:P2HvUV1SI941YwLzoxJjavGqL98=', 'Content-Length': '0'} 2022-09-03T05:29:52.543 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000003cd2f141066ec179-006312e650-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.543 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:52.544 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:52.544 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.545 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.545 DEBUG:boto:Method: GET 2022-09-03T05:29:52.546 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.547 DEBUG:boto:Data: 2022-09-03T05:29:52.547 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.548 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.548 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.549 DEBUG:boto:Params: {} 2022-09-03T05:29:52.549 DEBUG:boto:Token: None 2022-09-03T05:29:52.550 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.551 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:P2HvUV1SI941YwLzoxJjavGqL98= 2022-09-03T05:29:52.551 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:P2HvUV1SI941YwLzoxJjavGqL98=', 'Content-Length': '0'} 2022-09-03T05:29:52.553 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000006a84b4ccb2e29e1d-006312e650-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.554 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:52.555 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:52.556 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:52.557 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:52.558 DEBUG:boto:Method: GET 2022-09-03T05:29:52.559 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:29:52.560 DEBUG:boto:Data: 2022-09-03T05:29:52.561 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.562 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:52.563 DEBUG:boto:Port: 8001 2022-09-03T05:29:52.564 DEBUG:boto:Params: {} 2022-09-03T05:29:52.565 DEBUG:boto:Token: None 2022-09-03T05:29:52.566 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/ 2022-09-03T05:29:52.566 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM= 2022-09-03T05:29:52.567 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM=', 'Content-Length': '0'} 2022-09-03T05:29:52.570 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b40e06bab98268ef-006312e650-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.611 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:52.612 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:52.612 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:52.613 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:52.613 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:52.614 DEBUG:boto:Method: GET 2022-09-03T05:29:52.615 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:29:52.615 DEBUG:boto:Data: 2022-09-03T05:29:52.616 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.616 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.617 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.617 DEBUG:boto:Params: {} 2022-09-03T05:29:52.618 DEBUG:boto:Token: None 2022-09-03T05:29:52.619 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/ 2022-09-03T05:29:52.620 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM= 2022-09-03T05:29:52.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM=', 'Content-Length': '0'} 2022-09-03T05:29:52.623 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000458e6fd07e6551b6-006312e650-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.624 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:52.625 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:52.625 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:52.626 DEBUG:boto:Method: HEAD 2022-09-03T05:29:52.626 DEBUG:boto:Path: /dsgvcy-16/dummy 2022-09-03T05:29:52.627 DEBUG:boto:Data: 2022-09-03T05:29:52.627 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.628 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:52.628 DEBUG:boto:Port: 8001 2022-09-03T05:29:52.629 DEBUG:boto:Params: {} 2022-09-03T05:29:52.630 DEBUG:boto:Token: None 2022-09-03T05:29:52.630 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/dummy 2022-09-03T05:29:52.631 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0VcaFzJXFv/hkfcIXc2NGLfQGWQ= 2022-09-03T05:29:52.631 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0VcaFzJXFv/hkfcIXc2NGLfQGWQ=', 'Content-Length': '0'} 2022-09-03T05:29:52.634 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000e3b8368a172c1d9b-006312e650-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.635 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:52.635 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:52.636 DEBUG:boto:Method: HEAD 2022-09-03T05:29:52.636 DEBUG:boto:Path: /dsgvcy-16/dummy 2022-09-03T05:29:52.637 DEBUG:boto:Data: 2022-09-03T05:29:52.638 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.638 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.639 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.639 DEBUG:boto:Params: {} 2022-09-03T05:29:52.640 DEBUG:boto:Token: None 2022-09-03T05:29:52.641 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/dummy 2022-09-03T05:29:52.641 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0VcaFzJXFv/hkfcIXc2NGLfQGWQ= 2022-09-03T05:29:52.642 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0VcaFzJXFv/hkfcIXc2NGLfQGWQ=', 'Content-Length': '0'} 2022-09-03T05:29:52.644 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000c05b7695e8221d82-006312e650-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.645 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:29:52.646 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:52.646 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:52.647 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.647 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.648 DEBUG:boto:Method: GET 2022-09-03T05:29:52.649 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.649 DEBUG:boto:Data: 2022-09-03T05:29:52.650 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.650 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:52.651 DEBUG:boto:Port: 8001 2022-09-03T05:29:52.652 DEBUG:boto:Params: {} 2022-09-03T05:29:52.652 DEBUG:boto:Token: None 2022-09-03T05:29:52.653 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.654 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:P2HvUV1SI941YwLzoxJjavGqL98= 2022-09-03T05:29:52.654 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:P2HvUV1SI941YwLzoxJjavGqL98=', 'Content-Length': '0'} 2022-09-03T05:29:52.656 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000fcfa93ed5a2520c7-006312e650-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.657 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:52.658 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:52.658 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.659 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.660 DEBUG:boto:Method: GET 2022-09-03T05:29:52.660 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.661 DEBUG:boto:Data: 2022-09-03T05:29:52.661 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.662 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.663 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.663 DEBUG:boto:Params: {} 2022-09-03T05:29:52.664 DEBUG:boto:Token: None 2022-09-03T05:29:52.665 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:52.665 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:P2HvUV1SI941YwLzoxJjavGqL98= 2022-09-03T05:29:52.666 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:P2HvUV1SI941YwLzoxJjavGqL98=', 'Content-Length': '0'} 2022-09-03T05:29:52.668 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000005440f40217a4eb98-006312e650-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.669 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:52.670 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:52.671 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:52.671 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:52.672 DEBUG:boto:Method: GET 2022-09-03T05:29:52.672 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:29:52.673 DEBUG:boto:Data: 2022-09-03T05:29:52.674 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.674 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:52.675 DEBUG:boto:Port: 8001 2022-09-03T05:29:52.676 DEBUG:boto:Params: {} 2022-09-03T05:29:52.676 DEBUG:boto:Token: None 2022-09-03T05:29:52.677 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/ 2022-09-03T05:29:52.677 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM= 2022-09-03T05:29:52.678 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM=', 'Content-Length': '0'} 2022-09-03T05:29:52.681 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a13b28a260e75765-006312e650-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.682 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:52.682 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:52.683 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:52.684 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:52.684 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:52.685 DEBUG:boto:Method: GET 2022-09-03T05:29:52.686 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:29:52.686 DEBUG:boto:Data: 2022-09-03T05:29:52.687 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.687 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.688 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.689 DEBUG:boto:Params: {} 2022-09-03T05:29:52.689 DEBUG:boto:Token: None 2022-09-03T05:29:52.690 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/ 2022-09-03T05:29:52.691 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM= 2022-09-03T05:29:52.691 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM=', 'Content-Length': '0'} 2022-09-03T05:29:52.694 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005402185fdb158884-006312e650-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.695 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:52.697 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:29:52.698 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.699 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.700 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.700 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.701 DEBUG:boto:Method: GET 2022-09-03T05:29:52.702 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.703 DEBUG:boto:Data: 2022-09-03T05:29:52.704 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.705 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:52.706 DEBUG:boto:Port: 8001 2022-09-03T05:29:52.706 DEBUG:boto:Params: {} 2022-09-03T05:29:52.707 DEBUG:boto:Token: None 2022-09-03T05:29:52.708 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.708 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pUfJ3/ioYHKMUB2I/9znQW7i8iE= 2022-09-03T05:29:52.709 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pUfJ3/ioYHKMUB2I/9znQW7i8iE=', 'Content-Length': '0'} 2022-09-03T05:29:52.711 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000082ca6598608077a2-006312e650-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.712 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.713 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.713 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.714 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.714 DEBUG:boto:Method: GET 2022-09-03T05:29:52.715 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.715 DEBUG:boto:Data: 2022-09-03T05:29:52.716 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.716 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.717 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.717 DEBUG:boto:Params: {} 2022-09-03T05:29:52.718 DEBUG:boto:Token: None 2022-09-03T05:29:52.719 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.719 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pUfJ3/ioYHKMUB2I/9znQW7i8iE= 2022-09-03T05:29:52.720 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pUfJ3/ioYHKMUB2I/9znQW7i8iE=', 'Content-Length': '0'} 2022-09-03T05:29:52.722 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ba7bb63f3607f954-006312e650-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.723 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.723 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.724 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.724 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.725 DEBUG:boto:Method: HEAD 2022-09-03T05:29:52.725 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.726 DEBUG:boto:Data: 2022-09-03T05:29:52.726 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.727 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:52.727 DEBUG:boto:Port: 8001 2022-09-03T05:29:52.728 DEBUG:boto:Params: {} 2022-09-03T05:29:52.728 DEBUG:boto:Token: None 2022-09-03T05:29:52.729 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.730 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sWufjOn/YRZ8rUbsiTcD3MzRyM0= 2022-09-03T05:29:52.730 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sWufjOn/YRZ8rUbsiTcD3MzRyM0=', 'Content-Length': '0'} 2022-09-03T05:29:52.732 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008e598fcffa7d5fae-006312e650-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.733 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.734 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.734 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.735 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.735 DEBUG:boto:Method: HEAD 2022-09-03T05:29:52.736 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.736 DEBUG:boto:Data: 2022-09-03T05:29:52.737 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.737 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.738 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.738 DEBUG:boto:Params: {} 2022-09-03T05:29:52.739 DEBUG:boto:Token: None 2022-09-03T05:29:52.739 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.740 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sWufjOn/YRZ8rUbsiTcD3MzRyM0= 2022-09-03T05:29:52.741 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sWufjOn/YRZ8rUbsiTcD3MzRyM0=', 'Content-Length': '0'} 2022-09-03T05:29:52.743 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bb6ed58c65536d8f-006312e650-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.744 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:29:52.744 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.745 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.745 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.746 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.747 DEBUG:boto:Method: GET 2022-09-03T05:29:52.748 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.749 DEBUG:boto:Data: 2022-09-03T05:29:52.749 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.750 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:52.750 DEBUG:boto:Port: 8001 2022-09-03T05:29:52.751 DEBUG:boto:Params: {} 2022-09-03T05:29:52.752 DEBUG:boto:Token: None 2022-09-03T05:29:52.752 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.753 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pUfJ3/ioYHKMUB2I/9znQW7i8iE= 2022-09-03T05:29:52.754 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pUfJ3/ioYHKMUB2I/9znQW7i8iE=', 'Content-Length': '0'} 2022-09-03T05:29:52.756 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e5e0396830348dfd-006312e650-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.757 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.757 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.758 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.759 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.759 DEBUG:boto:Method: GET 2022-09-03T05:29:52.760 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.761 DEBUG:boto:Data: 2022-09-03T05:29:52.762 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.763 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.763 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.764 DEBUG:boto:Params: {} 2022-09-03T05:29:52.765 DEBUG:boto:Token: None 2022-09-03T05:29:52.766 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.767 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pUfJ3/ioYHKMUB2I/9znQW7i8iE= 2022-09-03T05:29:52.767 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pUfJ3/ioYHKMUB2I/9znQW7i8iE=', 'Content-Length': '0'} 2022-09-03T05:29:52.770 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001da47c20bd112498-006312e650-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.814 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:52.815 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:52.815 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:52.816 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:52.817 DEBUG:boto:Method: GET 2022-09-03T05:29:52.817 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:52.818 DEBUG:boto:Data: 2022-09-03T05:29:52.818 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.819 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:52.819 DEBUG:boto:Port: 8001 2022-09-03T05:29:52.820 DEBUG:boto:Params: {} 2022-09-03T05:29:52.821 DEBUG:boto:Token: None 2022-09-03T05:29:52.821 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/ 2022-09-03T05:29:52.822 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM= 2022-09-03T05:29:52.823 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM=', 'Content-Length': '0'} 2022-09-03T05:29:52.826 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004f32747e36337e40-006312e650-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.826 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:52.827 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:52.828 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:52.828 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:52.829 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:52.829 DEBUG:boto:Method: GET 2022-09-03T05:29:52.830 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:52.831 DEBUG:boto:Data: 2022-09-03T05:29:52.831 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.832 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.832 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.833 DEBUG:boto:Params: {} 2022-09-03T05:29:52.834 DEBUG:boto:Token: None 2022-09-03T05:29:52.834 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/ 2022-09-03T05:29:52.835 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM= 2022-09-03T05:29:52.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM=', 'Content-Length': '0'} 2022-09-03T05:29:52.840 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008b30d6f63a205f6c-006312e650-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.840 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:52.841 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.842 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.842 DEBUG:boto:Method: HEAD 2022-09-03T05:29:52.843 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.843 DEBUG:boto:Data: 2022-09-03T05:29:52.844 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.844 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:52.845 DEBUG:boto:Port: 8001 2022-09-03T05:29:52.845 DEBUG:boto:Params: {} 2022-09-03T05:29:52.846 DEBUG:boto:Token: None 2022-09-03T05:29:52.847 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.847 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ypZzciUoOUEqV+iSGgAbgBXJPOU= 2022-09-03T05:29:52.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ypZzciUoOUEqV+iSGgAbgBXJPOU=', 'Content-Length': '0'} 2022-09-03T05:29:52.850 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009129f1136c7f0a94-006312e650-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.851 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.852 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.852 DEBUG:boto:Method: HEAD 2022-09-03T05:29:52.853 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.853 DEBUG:boto:Data: 2022-09-03T05:29:52.854 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.854 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.855 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.855 DEBUG:boto:Params: {} 2022-09-03T05:29:52.856 DEBUG:boto:Token: None 2022-09-03T05:29:52.857 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.857 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ypZzciUoOUEqV+iSGgAbgBXJPOU= 2022-09-03T05:29:52.858 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ypZzciUoOUEqV+iSGgAbgBXJPOU=', 'Content-Length': '0'} 2022-09-03T05:29:52.860 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000093fb8984fe1732c7-006312e650-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.861 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:29:52.862 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.862 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.863 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.863 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.864 DEBUG:boto:Method: GET 2022-09-03T05:29:52.864 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.865 DEBUG:boto:Data: 2022-09-03T05:29:52.865 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.866 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:52.866 DEBUG:boto:Port: 8001 2022-09-03T05:29:52.867 DEBUG:boto:Params: {} 2022-09-03T05:29:52.868 DEBUG:boto:Token: None 2022-09-03T05:29:52.868 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.869 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pUfJ3/ioYHKMUB2I/9znQW7i8iE= 2022-09-03T05:29:52.869 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pUfJ3/ioYHKMUB2I/9znQW7i8iE=', 'Content-Length': '0'} 2022-09-03T05:29:52.872 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c825d4a536c3447b-006312e650-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.914 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.915 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:52.916 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.917 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.917 DEBUG:boto:Method: GET 2022-09-03T05:29:52.918 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.919 DEBUG:boto:Data: 2022-09-03T05:29:52.920 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.920 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.921 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.922 DEBUG:boto:Params: {} 2022-09-03T05:29:52.922 DEBUG:boto:Token: None 2022-09-03T05:29:52.923 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:52.924 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pUfJ3/ioYHKMUB2I/9znQW7i8iE= 2022-09-03T05:29:52.924 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pUfJ3/ioYHKMUB2I/9znQW7i8iE=', 'Content-Length': '0'} 2022-09-03T05:29:52.927 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007e42a2de989cfb37-006312e650-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.928 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:52.928 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:52.929 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:52.929 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:52.930 DEBUG:boto:Method: GET 2022-09-03T05:29:52.931 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:52.931 DEBUG:boto:Data: 2022-09-03T05:29:52.932 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.932 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:52.933 DEBUG:boto:Port: 8001 2022-09-03T05:29:52.933 DEBUG:boto:Params: {} 2022-09-03T05:29:52.934 DEBUG:boto:Token: None 2022-09-03T05:29:52.935 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/ 2022-09-03T05:29:52.935 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM= 2022-09-03T05:29:52.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM=', 'Content-Length': '0'} 2022-09-03T05:29:52.940 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d315a6e898aea1b6-006312e650-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.941 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:52.942 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:52.943 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:52.943 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:52.944 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:52.945 DEBUG:boto:Method: GET 2022-09-03T05:29:52.945 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:52.946 DEBUG:boto:Data: 2022-09-03T05:29:52.946 DEBUG:boto:Headers: {} 2022-09-03T05:29:52.947 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:52.948 DEBUG:boto:Port: 8000 2022-09-03T05:29:52.948 DEBUG:boto:Params: {} 2022-09-03T05:29:52.949 DEBUG:boto:Token: None 2022-09-03T05:29:52.950 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:52 GMT /dsgvcy-16/ 2022-09-03T05:29:52.950 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM= 2022-09-03T05:29:52.951 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DE5VEtAHtzcOO65/9e2Qze+1rTM=', 'Content-Length': '0'} 2022-09-03T05:29:52.954 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007dd6cecdcbecfade-006312e650-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:52.998 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:52.999 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:29:53.000 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.000 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.001 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.001 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.002 DEBUG:boto:Method: GET 2022-09-03T05:29:53.003 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.004 DEBUG:boto:Data: 2022-09-03T05:29:53.005 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.005 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.006 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.007 DEBUG:boto:Params: {} 2022-09-03T05:29:53.008 DEBUG:boto:Token: None 2022-09-03T05:29:53.009 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.010 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EH8BXO5gV+pCkfosNTf7Wc4uJzU= 2022-09-03T05:29:53.011 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EH8BXO5gV+pCkfosNTf7Wc4uJzU=', 'Content-Length': '0'} 2022-09-03T05:29:53.015 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000154ee6a608d5f57d-006312e651-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.016 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.017 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.018 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.019 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.020 DEBUG:boto:Method: GET 2022-09-03T05:29:53.021 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.022 DEBUG:boto:Data: 2022-09-03T05:29:53.023 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.024 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:53.025 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.026 DEBUG:boto:Params: {} 2022-09-03T05:29:53.027 DEBUG:boto:Token: None 2022-09-03T05:29:53.028 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.029 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EH8BXO5gV+pCkfosNTf7Wc4uJzU= 2022-09-03T05:29:53.030 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EH8BXO5gV+pCkfosNTf7Wc4uJzU=', 'Content-Length': '0'} 2022-09-03T05:29:53.033 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dd55217d8ac29bf1-006312e651-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.034 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.035 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.036 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.037 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.038 DEBUG:boto:Method: HEAD 2022-09-03T05:29:53.039 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.040 DEBUG:boto:Data: 2022-09-03T05:29:53.041 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.041 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.042 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.043 DEBUG:boto:Params: {} 2022-09-03T05:29:53.044 DEBUG:boto:Token: None 2022-09-03T05:29:53.045 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.046 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sb/crV8tOG9VyEghdaqeHzwWXKQ= 2022-09-03T05:29:53.047 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sb/crV8tOG9VyEghdaqeHzwWXKQ=', 'Content-Length': '0'} 2022-09-03T05:29:53.050 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003e90deb1b7d638ed-006312e651-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.051 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.052 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.053 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.054 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.055 DEBUG:boto:Method: HEAD 2022-09-03T05:29:53.056 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.057 DEBUG:boto:Data: 2022-09-03T05:29:53.058 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.059 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:53.060 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.061 DEBUG:boto:Params: {} 2022-09-03T05:29:53.062 DEBUG:boto:Token: None 2022-09-03T05:29:53.063 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.064 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sb/crV8tOG9VyEghdaqeHzwWXKQ= 2022-09-03T05:29:53.065 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sb/crV8tOG9VyEghdaqeHzwWXKQ=', 'Content-Length': '0'} 2022-09-03T05:29:53.067 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c2dad137cc5338e6-006312e651-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.068 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:29:53.069 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.070 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.070 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.071 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.072 DEBUG:boto:Method: GET 2022-09-03T05:29:53.072 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.073 DEBUG:boto:Data: 2022-09-03T05:29:53.074 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.075 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.076 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.076 DEBUG:boto:Params: {} 2022-09-03T05:29:53.077 DEBUG:boto:Token: None 2022-09-03T05:29:53.078 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.079 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EH8BXO5gV+pCkfosNTf7Wc4uJzU= 2022-09-03T05:29:53.080 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EH8BXO5gV+pCkfosNTf7Wc4uJzU=', 'Content-Length': '0'} 2022-09-03T05:29:53.083 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c7f618d83bdd1408-006312e651-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.084 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.085 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.086 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.094 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.095 DEBUG:boto:Method: GET 2022-09-03T05:29:53.096 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.097 DEBUG:boto:Data: 2022-09-03T05:29:53.098 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.099 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:53.099 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.100 DEBUG:boto:Params: {} 2022-09-03T05:29:53.101 DEBUG:boto:Token: None 2022-09-03T05:29:53.102 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.103 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EH8BXO5gV+pCkfosNTf7Wc4uJzU= 2022-09-03T05:29:53.105 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EH8BXO5gV+pCkfosNTf7Wc4uJzU=', 'Content-Length': '0'} 2022-09-03T05:29:53.108 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c8a6056bf4c73385-006312e651-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.109 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:53.110 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:53.111 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:53.112 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:53.113 DEBUG:boto:Method: GET 2022-09-03T05:29:53.114 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:53.115 DEBUG:boto:Data: 2022-09-03T05:29:53.115 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.116 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.117 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.117 DEBUG:boto:Params: {} 2022-09-03T05:29:53.119 DEBUG:boto:Token: None 2022-09-03T05:29:53.119 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/ 2022-09-03T05:29:53.120 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY= 2022-09-03T05:29:53.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY=', 'Content-Length': '0'} 2022-09-03T05:29:53.124 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ee00e58fc1be903e-006312e651-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.166 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:53.167 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:53.168 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:53.169 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:53.170 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:53.170 DEBUG:boto:Method: GET 2022-09-03T05:29:53.171 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:53.172 DEBUG:boto:Data: 2022-09-03T05:29:53.172 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.173 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:53.173 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.174 DEBUG:boto:Params: {} 2022-09-03T05:29:53.175 DEBUG:boto:Token: None 2022-09-03T05:29:53.175 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/ 2022-09-03T05:29:53.176 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY= 2022-09-03T05:29:53.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY=', 'Content-Length': '0'} 2022-09-03T05:29:53.180 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000039cb1d38a4ab5c0e-006312e651-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.181 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:53.182 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.183 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.183 DEBUG:boto:Method: HEAD 2022-09-03T05:29:53.184 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.184 DEBUG:boto:Data: 2022-09-03T05:29:53.185 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.186 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.186 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.187 DEBUG:boto:Params: {} 2022-09-03T05:29:53.190 DEBUG:boto:Token: None 2022-09-03T05:29:53.190 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.191 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MbBUqE7h+re5vcRyH/hiVDAixQo= 2022-09-03T05:29:53.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MbBUqE7h+re5vcRyH/hiVDAixQo=', 'Content-Length': '0'} 2022-09-03T05:29:53.195 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ecc022b1c14ea0c9-006312e651-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.196 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.197 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.198 DEBUG:boto:Method: HEAD 2022-09-03T05:29:53.198 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.199 DEBUG:boto:Data: 2022-09-03T05:29:53.200 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.200 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:53.201 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.202 DEBUG:boto:Params: {} 2022-09-03T05:29:53.202 DEBUG:boto:Token: None 2022-09-03T05:29:53.203 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.204 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MbBUqE7h+re5vcRyH/hiVDAixQo= 2022-09-03T05:29:53.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MbBUqE7h+re5vcRyH/hiVDAixQo=', 'Content-Length': '0'} 2022-09-03T05:29:53.207 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000001a0f56ad61cb39a-006312e651-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.208 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:29:53.208 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.209 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.210 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.210 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.211 DEBUG:boto:Method: GET 2022-09-03T05:29:53.211 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.212 DEBUG:boto:Data: 2022-09-03T05:29:53.213 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.213 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.214 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.215 DEBUG:boto:Params: {} 2022-09-03T05:29:53.215 DEBUG:boto:Token: None 2022-09-03T05:29:53.216 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.217 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EH8BXO5gV+pCkfosNTf7Wc4uJzU= 2022-09-03T05:29:53.218 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EH8BXO5gV+pCkfosNTf7Wc4uJzU=', 'Content-Length': '0'} 2022-09-03T05:29:53.220 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000916281816ec1149a-006312e651-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.221 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.222 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:53.222 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.223 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.223 DEBUG:boto:Method: GET 2022-09-03T05:29:53.224 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.225 DEBUG:boto:Data: 2022-09-03T05:29:53.225 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.226 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:53.227 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.227 DEBUG:boto:Params: {} 2022-09-03T05:29:53.228 DEBUG:boto:Token: None 2022-09-03T05:29:53.229 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:53.229 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EH8BXO5gV+pCkfosNTf7Wc4uJzU= 2022-09-03T05:29:53.230 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EH8BXO5gV+pCkfosNTf7Wc4uJzU=', 'Content-Length': '0'} 2022-09-03T05:29:53.233 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fd2ee529aed691fa-006312e651-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.234 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:53.235 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:53.236 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:53.237 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:53.238 DEBUG:boto:Method: GET 2022-09-03T05:29:53.239 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:53.239 DEBUG:boto:Data: 2022-09-03T05:29:53.240 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.241 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.241 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.242 DEBUG:boto:Params: {} 2022-09-03T05:29:53.243 DEBUG:boto:Token: None 2022-09-03T05:29:53.244 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/ 2022-09-03T05:29:53.245 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY= 2022-09-03T05:29:53.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY=', 'Content-Length': '0'} 2022-09-03T05:29:53.249 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000040fc36bed8ec97d1-006312e651-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.250 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:53.251 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:53.252 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:53.253 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:53.254 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:53.255 DEBUG:boto:Method: GET 2022-09-03T05:29:53.255 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:53.256 DEBUG:boto:Data: 2022-09-03T05:29:53.257 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.258 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:53.259 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.260 DEBUG:boto:Params: {} 2022-09-03T05:29:53.261 DEBUG:boto:Token: None 2022-09-03T05:29:53.262 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/ 2022-09-03T05:29:53.262 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY= 2022-09-03T05:29:53.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY=', 'Content-Length': '0'} 2022-09-03T05:29:53.267 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000b330a775b1c7a2e-006312e651-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.268 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:53.269 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:29:53.270 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.271 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.272 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.272 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.273 DEBUG:boto:Method: GET 2022-09-03T05:29:53.274 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.274 DEBUG:boto:Data: 2022-09-03T05:29:53.275 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.276 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.276 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.277 DEBUG:boto:Params: {} 2022-09-03T05:29:53.278 DEBUG:boto:Token: None 2022-09-03T05:29:53.279 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.279 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:azZ4BlAw3MUPDaqAi2Zm0B+JGlU= 2022-09-03T05:29:53.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:azZ4BlAw3MUPDaqAi2Zm0B+JGlU=', 'Content-Length': '0'} 2022-09-03T05:29:53.282 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008b8393e1a8b77e54-006312e651-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.283 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.284 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.285 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.285 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.286 DEBUG:boto:Method: GET 2022-09-03T05:29:53.287 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.287 DEBUG:boto:Data: 2022-09-03T05:29:53.288 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.288 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:53.289 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.290 DEBUG:boto:Params: {} 2022-09-03T05:29:53.291 DEBUG:boto:Token: None 2022-09-03T05:29:53.291 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.292 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:azZ4BlAw3MUPDaqAi2Zm0B+JGlU= 2022-09-03T05:29:53.293 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:azZ4BlAw3MUPDaqAi2Zm0B+JGlU=', 'Content-Length': '0'} 2022-09-03T05:29:53.295 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009cef35d045b1ac49-006312e651-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.338 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.339 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.340 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.340 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.341 DEBUG:boto:Method: HEAD 2022-09-03T05:29:53.341 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.342 DEBUG:boto:Data: 2022-09-03T05:29:53.342 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.343 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.343 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.344 DEBUG:boto:Params: {} 2022-09-03T05:29:53.345 DEBUG:boto:Token: None 2022-09-03T05:29:53.346 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.346 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/C9aNrC7KwIV77YTslX1S0Zsufs= 2022-09-03T05:29:53.347 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/C9aNrC7KwIV77YTslX1S0Zsufs=', 'Content-Length': '0'} 2022-09-03T05:29:53.349 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000029ce3ee89fd32c5-006312e651-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.350 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.351 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.351 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.352 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.352 DEBUG:boto:Method: HEAD 2022-09-03T05:29:53.353 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.353 DEBUG:boto:Data: 2022-09-03T05:29:53.354 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.354 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:53.355 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.355 DEBUG:boto:Params: {} 2022-09-03T05:29:53.356 DEBUG:boto:Token: None 2022-09-03T05:29:53.357 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.357 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/C9aNrC7KwIV77YTslX1S0Zsufs= 2022-09-03T05:29:53.358 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/C9aNrC7KwIV77YTslX1S0Zsufs=', 'Content-Length': '0'} 2022-09-03T05:29:53.360 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b9fe344ed84257ce-006312e651-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.361 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:29:53.362 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.362 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.363 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.363 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.364 DEBUG:boto:Method: GET 2022-09-03T05:29:53.364 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.365 DEBUG:boto:Data: 2022-09-03T05:29:53.365 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.366 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.366 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.367 DEBUG:boto:Params: {} 2022-09-03T05:29:53.368 DEBUG:boto:Token: None 2022-09-03T05:29:53.368 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.369 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:azZ4BlAw3MUPDaqAi2Zm0B+JGlU= 2022-09-03T05:29:53.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:azZ4BlAw3MUPDaqAi2Zm0B+JGlU=', 'Content-Length': '0'} 2022-09-03T05:29:53.372 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003a3b4e402e305f2c-006312e651-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.372 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.373 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.374 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.374 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.375 DEBUG:boto:Method: GET 2022-09-03T05:29:53.375 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.376 DEBUG:boto:Data: 2022-09-03T05:29:53.376 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.377 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:53.377 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.378 DEBUG:boto:Params: {} 2022-09-03T05:29:53.379 DEBUG:boto:Token: None 2022-09-03T05:29:53.379 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.380 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:azZ4BlAw3MUPDaqAi2Zm0B+JGlU= 2022-09-03T05:29:53.381 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:azZ4BlAw3MUPDaqAi2Zm0B+JGlU=', 'Content-Length': '0'} 2022-09-03T05:29:53.383 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006f3f821127ec0644-006312e651-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.384 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:53.384 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:53.385 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:53.385 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:53.386 DEBUG:boto:Method: GET 2022-09-03T05:29:53.387 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:53.387 DEBUG:boto:Data: 2022-09-03T05:29:53.388 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.388 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.389 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.389 DEBUG:boto:Params: {} 2022-09-03T05:29:53.390 DEBUG:boto:Token: None 2022-09-03T05:29:53.390 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/ 2022-09-03T05:29:53.391 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY= 2022-09-03T05:29:53.392 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY=', 'Content-Length': '0'} 2022-09-03T05:29:53.394 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aa961e7ba100be5d-006312e651-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.395 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:53.396 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:53.397 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:53.397 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:53.398 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:53.398 DEBUG:boto:Method: GET 2022-09-03T05:29:53.399 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:53.399 DEBUG:boto:Data: 2022-09-03T05:29:53.400 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.400 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:53.401 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.401 DEBUG:boto:Params: {} 2022-09-03T05:29:53.402 DEBUG:boto:Token: None 2022-09-03T05:29:53.403 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/ 2022-09-03T05:29:53.403 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY= 2022-09-03T05:29:53.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY=', 'Content-Length': '0'} 2022-09-03T05:29:53.407 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d37901fc0d11d67-006312e651-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.407 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:53.408 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.409 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.409 DEBUG:boto:Method: HEAD 2022-09-03T05:29:53.410 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.410 DEBUG:boto:Data: 2022-09-03T05:29:53.411 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.411 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.412 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.412 DEBUG:boto:Params: {} 2022-09-03T05:29:53.413 DEBUG:boto:Token: None 2022-09-03T05:29:53.414 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.414 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2fcb5ag07Sl/ItVRs7zQFFp8kD0= 2022-09-03T05:29:53.415 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2fcb5ag07Sl/ItVRs7zQFFp8kD0=', 'Content-Length': '0'} 2022-09-03T05:29:53.417 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f0e2938b9302d20b-006312e651-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.418 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.418 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.419 DEBUG:boto:Method: HEAD 2022-09-03T05:29:53.419 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.420 DEBUG:boto:Data: 2022-09-03T05:29:53.420 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.421 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:53.421 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.422 DEBUG:boto:Params: {} 2022-09-03T05:29:53.422 DEBUG:boto:Token: None 2022-09-03T05:29:53.423 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.424 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2fcb5ag07Sl/ItVRs7zQFFp8kD0= 2022-09-03T05:29:53.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2fcb5ag07Sl/ItVRs7zQFFp8kD0=', 'Content-Length': '0'} 2022-09-03T05:29:53.426 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000319248e12f94ea2e-006312e651-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.427 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:29:53.428 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.428 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.429 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.429 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.430 DEBUG:boto:Method: GET 2022-09-03T05:29:53.431 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.431 DEBUG:boto:Data: 2022-09-03T05:29:53.432 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.432 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.433 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.433 DEBUG:boto:Params: {} 2022-09-03T05:29:53.434 DEBUG:boto:Token: None 2022-09-03T05:29:53.435 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.435 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:azZ4BlAw3MUPDaqAi2Zm0B+JGlU= 2022-09-03T05:29:53.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:azZ4BlAw3MUPDaqAi2Zm0B+JGlU=', 'Content-Length': '0'} 2022-09-03T05:29:53.438 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bd075aace7898c17-006312e651-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.482 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.483 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:53.484 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.484 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.485 DEBUG:boto:Method: GET 2022-09-03T05:29:53.485 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.486 DEBUG:boto:Data: 2022-09-03T05:29:53.487 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.487 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:53.488 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.488 DEBUG:boto:Params: {} 2022-09-03T05:29:53.489 DEBUG:boto:Token: None 2022-09-03T05:29:53.490 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:53.491 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:azZ4BlAw3MUPDaqAi2Zm0B+JGlU= 2022-09-03T05:29:53.491 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:azZ4BlAw3MUPDaqAi2Zm0B+JGlU=', 'Content-Length': '0'} 2022-09-03T05:29:53.494 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f86bf38c8cd91d10-006312e651-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.495 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:53.495 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:53.496 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:53.496 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:53.497 DEBUG:boto:Method: GET 2022-09-03T05:29:53.498 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:53.498 DEBUG:boto:Data: 2022-09-03T05:29:53.499 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.499 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.500 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.500 DEBUG:boto:Params: {} 2022-09-03T05:29:53.501 DEBUG:boto:Token: None 2022-09-03T05:29:53.502 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/ 2022-09-03T05:29:53.503 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY= 2022-09-03T05:29:53.503 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY=', 'Content-Length': '0'} 2022-09-03T05:29:53.506 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bd8f57dca5684c83-006312e651-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.507 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:53.508 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:53.508 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:53.509 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:53.510 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:53.510 DEBUG:boto:Method: GET 2022-09-03T05:29:53.511 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:53.512 DEBUG:boto:Data: 2022-09-03T05:29:53.512 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.513 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:29:53.513 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.514 DEBUG:boto:Params: {} 2022-09-03T05:29:53.515 DEBUG:boto:Token: None 2022-09-03T05:29:53.515 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/ 2022-09-03T05:29:53.516 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY= 2022-09-03T05:29:53.517 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY=', 'Content-Length': '0'} 2022-09-03T05:29:53.520 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001cb7ad87cae5206b-006312e651-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.521 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:53.523 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-16 zones={test-zone3, test-zone1} 2022-09-03T05:29:53.524 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-16 zones={test-zone3, test-zone2} 2022-09-03T05:29:53.525 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:53.526 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:53.527 DEBUG:boto:Method: HEAD 2022-09-03T05:29:53.528 DEBUG:boto:Path: /dsgvcy-16/ 2022-09-03T05:29:53.529 DEBUG:boto:Data: 2022-09-03T05:29:53.529 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.530 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.531 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.531 DEBUG:boto:Params: {} 2022-09-03T05:29:53.532 DEBUG:boto:Token: None 2022-09-03T05:29:53.532 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/ 2022-09-03T05:29:53.533 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+AY3hUuJm2ne6lJ1tquWj3YxauU= 2022-09-03T05:29:53.534 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+AY3hUuJm2ne6lJ1tquWj3YxauU=', 'Content-Length': '0'} 2022-09-03T05:29:53.536 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ff159b014a946ab8-006312e651-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.537 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:53.538 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:53.539 DEBUG:boto:Method: HEAD 2022-09-03T05:29:53.540 DEBUG:boto:Path: /dsgvcy-16/ 2022-09-03T05:29:53.541 DEBUG:boto:Data: 2022-09-03T05:29:53.541 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.542 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:53.542 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.543 DEBUG:boto:Params: {} 2022-09-03T05:29:53.543 DEBUG:boto:Token: None 2022-09-03T05:29:53.545 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/ 2022-09-03T05:29:53.545 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+AY3hUuJm2ne6lJ1tquWj3YxauU= 2022-09-03T05:29:53.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+AY3hUuJm2ne6lJ1tquWj3YxauU=', 'Content-Length': '0'} 2022-09-03T05:29:53.549 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c92ccc1c1368b2ef-006312e651-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.550 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:29:53.550 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:53.551 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:53.551 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:53.552 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:53.552 DEBUG:boto:Method: GET 2022-09-03T05:29:53.553 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:53.553 DEBUG:boto:Data: 2022-09-03T05:29:53.554 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.555 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.555 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.556 DEBUG:boto:Params: {} 2022-09-03T05:29:53.556 DEBUG:boto:Token: None 2022-09-03T05:29:53.557 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/?versions 2022-09-03T05:29:53.558 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a/JVOOIezhDfAu9klUNwd9YeY3o= 2022-09-03T05:29:53.558 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a/JVOOIezhDfAu9klUNwd9YeY3o=', 'Content-Length': '0'} 2022-09-03T05:29:53.561 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000052e645694c1668a0-006312e651-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.562 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:53.563 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:29:53.563 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:29:53.564 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:29:53.564 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:29:53.564 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:29:53.565 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:53.566 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:53.566 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:53.567 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:53.567 DEBUG:boto:Method: GET 2022-09-03T05:29:53.568 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:53.568 DEBUG:boto:Data: 2022-09-03T05:29:53.569 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.569 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:53.570 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.570 DEBUG:boto:Params: {} 2022-09-03T05:29:53.571 DEBUG:boto:Token: None 2022-09-03T05:29:53.572 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/?versions 2022-09-03T05:29:53.572 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a/JVOOIezhDfAu9klUNwd9YeY3o= 2022-09-03T05:29:53.573 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a/JVOOIezhDfAu9klUNwd9YeY3o=', 'Content-Length': '0'} 2022-09-03T05:29:53.575 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d22b301ada1d20df-006312e651-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.576 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:53.577 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:29:53.578 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:29:53.579 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:29:53.580 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:29:53.580 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:53.581 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:53.582 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:53.583 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:53.584 DEBUG:boto:Method: GET 2022-09-03T05:29:53.585 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:53.586 DEBUG:boto:Data: 2022-09-03T05:29:53.586 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.587 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.587 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.588 DEBUG:boto:Params: {} 2022-09-03T05:29:53.589 DEBUG:boto:Token: None 2022-09-03T05:29:53.589 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/?versions 2022-09-03T05:29:53.590 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a/JVOOIezhDfAu9klUNwd9YeY3o= 2022-09-03T05:29:53.591 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a/JVOOIezhDfAu9klUNwd9YeY3o=', 'Content-Length': '0'} 2022-09-03T05:29:53.594 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e451dd71e3fed3a3-006312e651-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.595 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:53.596 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:53.597 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:53.598 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:53.599 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:53.600 DEBUG:boto:Method: GET 2022-09-03T05:29:53.601 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:29:53.601 DEBUG:boto:Data: 2022-09-03T05:29:53.602 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.603 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:53.603 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.604 DEBUG:boto:Params: {} 2022-09-03T05:29:53.605 DEBUG:boto:Token: None 2022-09-03T05:29:53.605 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/?versions 2022-09-03T05:29:53.606 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a/JVOOIezhDfAu9klUNwd9YeY3o= 2022-09-03T05:29:53.607 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a/JVOOIezhDfAu9klUNwd9YeY3o=', 'Content-Length': '0'} 2022-09-03T05:29:53.609 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000062ec5b7e0b259a22-006312e651-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.610 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:53.611 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:29:53.612 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:53.613 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:53.614 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.615 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.616 DEBUG:boto:Method: GET 2022-09-03T05:29:53.617 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.618 DEBUG:boto:Data: 2022-09-03T05:29:53.619 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.620 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.622 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.623 DEBUG:boto:Params: {} 2022-09-03T05:29:53.624 DEBUG:boto:Token: None 2022-09-03T05:29:53.625 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.626 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zdc4iL8iQi0OunNvMtZKheLXPuc= 2022-09-03T05:29:53.627 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zdc4iL8iQi0OunNvMtZKheLXPuc=', 'Content-Length': '0'} 2022-09-03T05:29:53.630 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000065949bd4e8b4795c-006312e651-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.631 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:53.632 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:53.633 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.634 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.635 DEBUG:boto:Method: GET 2022-09-03T05:29:53.637 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.638 DEBUG:boto:Data: 2022-09-03T05:29:53.639 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.640 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:53.640 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.641 DEBUG:boto:Params: {} 2022-09-03T05:29:53.642 DEBUG:boto:Token: None 2022-09-03T05:29:53.643 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.644 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zdc4iL8iQi0OunNvMtZKheLXPuc= 2022-09-03T05:29:53.645 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zdc4iL8iQi0OunNvMtZKheLXPuc=', 'Content-Length': '0'} 2022-09-03T05:29:53.647 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f8ca8f1591c92e0b-006312e651-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.648 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:53.649 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:53.649 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.650 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.650 DEBUG:boto:Method: HEAD 2022-09-03T05:29:53.651 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.651 DEBUG:boto:Data: 2022-09-03T05:29:53.652 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.652 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.653 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.654 DEBUG:boto:Params: {} 2022-09-03T05:29:53.654 DEBUG:boto:Token: None 2022-09-03T05:29:53.655 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.656 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XxNjXOT6ZKRoxNRyXCu2FeOxhE0= 2022-09-03T05:29:53.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XxNjXOT6ZKRoxNRyXCu2FeOxhE0=', 'Content-Length': '0'} 2022-09-03T05:29:53.659 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000005140f13eb83a1181-006312e651-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.659 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:53.660 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:53.661 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.661 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.662 DEBUG:boto:Method: HEAD 2022-09-03T05:29:53.662 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.663 DEBUG:boto:Data: 2022-09-03T05:29:53.663 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.664 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:53.664 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.665 DEBUG:boto:Params: {} 2022-09-03T05:29:53.665 DEBUG:boto:Token: None 2022-09-03T05:29:53.666 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.667 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XxNjXOT6ZKRoxNRyXCu2FeOxhE0= 2022-09-03T05:29:53.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XxNjXOT6ZKRoxNRyXCu2FeOxhE0=', 'Content-Length': '0'} 2022-09-03T05:29:53.672 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000e61b9ce9a54a9629-006312e651-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.673 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:29:53.674 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:53.675 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:53.676 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.677 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.678 DEBUG:boto:Method: GET 2022-09-03T05:29:53.678 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.679 DEBUG:boto:Data: 2022-09-03T05:29:53.680 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.680 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.681 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.681 DEBUG:boto:Params: {} 2022-09-03T05:29:53.682 DEBUG:boto:Token: None 2022-09-03T05:29:53.683 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.684 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zdc4iL8iQi0OunNvMtZKheLXPuc= 2022-09-03T05:29:53.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zdc4iL8iQi0OunNvMtZKheLXPuc=', 'Content-Length': '0'} 2022-09-03T05:29:53.686 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000090751e58b0646ecc-006312e651-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.687 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:53.689 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:53.689 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.690 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.691 DEBUG:boto:Method: GET 2022-09-03T05:29:53.692 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.693 DEBUG:boto:Data: 2022-09-03T05:29:53.695 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.695 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:53.696 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.697 DEBUG:boto:Params: {} 2022-09-03T05:29:53.698 DEBUG:boto:Token: None 2022-09-03T05:29:53.698 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.699 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zdc4iL8iQi0OunNvMtZKheLXPuc= 2022-09-03T05:29:53.700 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zdc4iL8iQi0OunNvMtZKheLXPuc=', 'Content-Length': '0'} 2022-09-03T05:29:53.702 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000000c034b5ca8846c35-006312e651-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.703 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:53.704 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:53.704 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:53.705 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:53.705 DEBUG:boto:Method: GET 2022-09-03T05:29:53.706 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:29:53.707 DEBUG:boto:Data: 2022-09-03T05:29:53.707 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.708 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.708 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.709 DEBUG:boto:Params: {} 2022-09-03T05:29:53.710 DEBUG:boto:Token: None 2022-09-03T05:29:53.710 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/ 2022-09-03T05:29:53.711 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY= 2022-09-03T05:29:53.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY=', 'Content-Length': '0'} 2022-09-03T05:29:53.715 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c71216daccefc757-006312e651-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.766 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:53.767 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:53.768 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:53.769 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:53.769 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:53.770 DEBUG:boto:Method: GET 2022-09-03T05:29:53.771 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:29:53.772 DEBUG:boto:Data: 2022-09-03T05:29:53.772 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.773 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:53.773 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.774 DEBUG:boto:Params: {} 2022-09-03T05:29:53.775 DEBUG:boto:Token: None 2022-09-03T05:29:53.775 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/ 2022-09-03T05:29:53.776 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY= 2022-09-03T05:29:53.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY=', 'Content-Length': '0'} 2022-09-03T05:29:53.780 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d9d6ae46533fe3d0-006312e651-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.780 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:53.781 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:53.782 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:53.782 DEBUG:boto:Method: HEAD 2022-09-03T05:29:53.783 DEBUG:boto:Path: /dsgvcy-16/dummy 2022-09-03T05:29:53.783 DEBUG:boto:Data: 2022-09-03T05:29:53.784 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.784 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.785 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.785 DEBUG:boto:Params: {} 2022-09-03T05:29:53.786 DEBUG:boto:Token: None 2022-09-03T05:29:53.787 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/dummy 2022-09-03T05:29:53.787 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:em87mEcucstU0uYilMqpT5y34ps= 2022-09-03T05:29:53.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:em87mEcucstU0uYilMqpT5y34ps=', 'Content-Length': '0'} 2022-09-03T05:29:53.791 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000ba9e4ee3f4244684-006312e651-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.792 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:53.793 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:53.793 DEBUG:boto:Method: HEAD 2022-09-03T05:29:53.794 DEBUG:boto:Path: /dsgvcy-16/dummy 2022-09-03T05:29:53.794 DEBUG:boto:Data: 2022-09-03T05:29:53.795 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.795 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:53.796 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.796 DEBUG:boto:Params: {} 2022-09-03T05:29:53.797 DEBUG:boto:Token: None 2022-09-03T05:29:53.797 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/dummy 2022-09-03T05:29:53.798 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:em87mEcucstU0uYilMqpT5y34ps= 2022-09-03T05:29:53.799 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:em87mEcucstU0uYilMqpT5y34ps=', 'Content-Length': '0'} 2022-09-03T05:29:53.802 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000d630a83cd02b44d3-006312e651-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.803 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:29:53.805 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:53.806 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:53.807 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.808 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.809 DEBUG:boto:Method: GET 2022-09-03T05:29:53.810 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.811 DEBUG:boto:Data: 2022-09-03T05:29:53.812 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.813 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.813 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.814 DEBUG:boto:Params: {} 2022-09-03T05:29:53.815 DEBUG:boto:Token: None 2022-09-03T05:29:53.816 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.816 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zdc4iL8iQi0OunNvMtZKheLXPuc= 2022-09-03T05:29:53.818 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zdc4iL8iQi0OunNvMtZKheLXPuc=', 'Content-Length': '0'} 2022-09-03T05:29:53.820 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000798c1d72d5942129-006312e651-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.821 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:29:53.822 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:29:53.823 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.824 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.825 DEBUG:boto:Method: GET 2022-09-03T05:29:53.825 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.826 DEBUG:boto:Data: 2022-09-03T05:29:53.827 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.828 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:53.829 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.830 DEBUG:boto:Params: {} 2022-09-03T05:29:53.830 DEBUG:boto:Token: None 2022-09-03T05:29:53.832 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:29:53.833 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zdc4iL8iQi0OunNvMtZKheLXPuc= 2022-09-03T05:29:53.833 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zdc4iL8iQi0OunNvMtZKheLXPuc=', 'Content-Length': '0'} 2022-09-03T05:29:53.836 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000565aaefc5dc329a9-006312e651-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.837 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:53.838 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:53.839 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:53.840 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:53.841 DEBUG:boto:Method: GET 2022-09-03T05:29:53.841 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:29:53.842 DEBUG:boto:Data: 2022-09-03T05:29:53.843 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.843 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.844 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.844 DEBUG:boto:Params: {} 2022-09-03T05:29:53.845 DEBUG:boto:Token: None 2022-09-03T05:29:53.846 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/ 2022-09-03T05:29:53.846 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY= 2022-09-03T05:29:53.847 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY=', 'Content-Length': '0'} 2022-09-03T05:29:53.850 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002198131107b7dbab-006312e651-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.851 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:53.852 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:53.853 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:53.853 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:53.854 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:29:53.854 DEBUG:boto:Method: GET 2022-09-03T05:29:53.855 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:29:53.856 DEBUG:boto:Data: 2022-09-03T05:29:53.856 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.857 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:53.857 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.858 DEBUG:boto:Params: {} 2022-09-03T05:29:53.858 DEBUG:boto:Token: None 2022-09-03T05:29:53.859 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/ 2022-09-03T05:29:53.860 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY= 2022-09-03T05:29:53.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vo2YggKu7AXbHOgC3yWIwv3uoBY=', 'Content-Length': '0'} 2022-09-03T05:29:53.863 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000044982c3d32a14d96-006312e651-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.864 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:53.865 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:29:53.866 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:53.866 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:53.867 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.868 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.868 DEBUG:boto:Method: GET 2022-09-03T05:29:53.869 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.870 DEBUG:boto:Data: 2022-09-03T05:29:53.871 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.872 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.873 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.874 DEBUG:boto:Params: {} 2022-09-03T05:29:53.875 DEBUG:boto:Token: None 2022-09-03T05:29:53.877 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.878 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FvhSrJ8kxICG5LbTZNFA3JgEYRg= 2022-09-03T05:29:53.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FvhSrJ8kxICG5LbTZNFA3JgEYRg=', 'Content-Length': '0'} 2022-09-03T05:29:53.882 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000868328b7ffc242b4-006312e651-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.931 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:53.932 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:53.933 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.935 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.936 DEBUG:boto:Method: GET 2022-09-03T05:29:53.938 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.939 DEBUG:boto:Data: 2022-09-03T05:29:53.940 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.940 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:53.941 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.942 DEBUG:boto:Params: {} 2022-09-03T05:29:53.942 DEBUG:boto:Token: None 2022-09-03T05:29:53.943 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.944 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FvhSrJ8kxICG5LbTZNFA3JgEYRg= 2022-09-03T05:29:53.945 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FvhSrJ8kxICG5LbTZNFA3JgEYRg=', 'Content-Length': '0'} 2022-09-03T05:29:53.948 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c3fe58c44ef54e2e-006312e651-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.949 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:53.950 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:53.950 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.951 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.951 DEBUG:boto:Method: HEAD 2022-09-03T05:29:53.952 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.952 DEBUG:boto:Data: 2022-09-03T05:29:53.953 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.953 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.954 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.954 DEBUG:boto:Params: {} 2022-09-03T05:29:53.955 DEBUG:boto:Token: None 2022-09-03T05:29:53.955 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.956 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gpH/wVQgkdHVcVxnrV7NQRxSJNY= 2022-09-03T05:29:53.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gpH/wVQgkdHVcVxnrV7NQRxSJNY=', 'Content-Length': '0'} 2022-09-03T05:29:53.959 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000db9d06ea55f1710a-006312e651-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.960 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:53.960 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:53.961 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.961 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.962 DEBUG:boto:Method: HEAD 2022-09-03T05:29:53.962 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.963 DEBUG:boto:Data: 2022-09-03T05:29:53.963 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.964 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:53.964 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.965 DEBUG:boto:Params: {} 2022-09-03T05:29:53.965 DEBUG:boto:Token: None 2022-09-03T05:29:53.966 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.967 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gpH/wVQgkdHVcVxnrV7NQRxSJNY= 2022-09-03T05:29:53.967 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gpH/wVQgkdHVcVxnrV7NQRxSJNY=', 'Content-Length': '0'} 2022-09-03T05:29:53.970 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008d2a81598e7c8cb6-006312e651-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.972 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:29:53.973 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:53.973 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:53.974 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.975 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.975 DEBUG:boto:Method: GET 2022-09-03T05:29:53.976 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.976 DEBUG:boto:Data: 2022-09-03T05:29:53.977 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.977 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:53.978 DEBUG:boto:Port: 8001 2022-09-03T05:29:53.978 DEBUG:boto:Params: {} 2022-09-03T05:29:53.979 DEBUG:boto:Token: None 2022-09-03T05:29:53.980 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.981 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FvhSrJ8kxICG5LbTZNFA3JgEYRg= 2022-09-03T05:29:53.981 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FvhSrJ8kxICG5LbTZNFA3JgEYRg=', 'Content-Length': '0'} 2022-09-03T05:29:53.984 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007ad92eeca5cc9dc3-006312e651-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.984 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:53.985 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:53.986 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.986 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.987 DEBUG:boto:Method: GET 2022-09-03T05:29:53.987 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.988 DEBUG:boto:Data: 2022-09-03T05:29:53.988 DEBUG:boto:Headers: {} 2022-09-03T05:29:53.989 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:53.989 DEBUG:boto:Port: 8000 2022-09-03T05:29:53.990 DEBUG:boto:Params: {} 2022-09-03T05:29:53.991 DEBUG:boto:Token: None 2022-09-03T05:29:53.991 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:53 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:53.992 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FvhSrJ8kxICG5LbTZNFA3JgEYRg= 2022-09-03T05:29:53.993 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FvhSrJ8kxICG5LbTZNFA3JgEYRg=', 'Content-Length': '0'} 2022-09-03T05:29:53.995 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d717eb526a58aa77-006312e651-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:53.996 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:53.997 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:53.997 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:53.998 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:53.998 DEBUG:boto:Method: GET 2022-09-03T05:29:53.999 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:54.000 DEBUG:boto:Data: 2022-09-03T05:29:54.000 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.001 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:54.001 DEBUG:boto:Port: 8001 2022-09-03T05:29:54.002 DEBUG:boto:Params: {} 2022-09-03T05:29:54.002 DEBUG:boto:Token: None 2022-09-03T05:29:54.003 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/ 2022-09-03T05:29:54.004 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo= 2022-09-03T05:29:54.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo=', 'Content-Length': '0'} 2022-09-03T05:29:54.008 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000060f1c1ea3c1e7752-006312e652-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.009 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:54.010 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:54.010 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:54.011 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:54.011 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:54.012 DEBUG:boto:Method: GET 2022-09-03T05:29:54.013 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:54.013 DEBUG:boto:Data: 2022-09-03T05:29:54.014 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.014 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:54.015 DEBUG:boto:Port: 8000 2022-09-03T05:29:54.015 DEBUG:boto:Params: {} 2022-09-03T05:29:54.016 DEBUG:boto:Token: None 2022-09-03T05:29:54.016 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/ 2022-09-03T05:29:54.017 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo= 2022-09-03T05:29:54.018 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo=', 'Content-Length': '0'} 2022-09-03T05:29:54.020 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000a90f03d38d4f51e-006312e652-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.021 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:54.022 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:54.022 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:54.023 DEBUG:boto:Method: HEAD 2022-09-03T05:29:54.023 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1 2022-09-03T05:29:54.024 DEBUG:boto:Data: 2022-09-03T05:29:54.025 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.025 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:54.026 DEBUG:boto:Port: 8001 2022-09-03T05:29:54.026 DEBUG:boto:Params: {} 2022-09-03T05:29:54.027 DEBUG:boto:Token: None 2022-09-03T05:29:54.027 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone1 2022-09-03T05:29:54.028 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TNsDjbu3SIBhzYTMwYc7OsTOVyI= 2022-09-03T05:29:54.028 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TNsDjbu3SIBhzYTMwYc7OsTOVyI=', 'Content-Length': '0'} 2022-09-03T05:29:54.031 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c62dfa24d04e57d1-006312e652-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.032 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:54.033 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:54.033 DEBUG:boto:Method: HEAD 2022-09-03T05:29:54.034 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1 2022-09-03T05:29:54.035 DEBUG:boto:Data: 2022-09-03T05:29:54.036 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.036 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:54.037 DEBUG:boto:Port: 8000 2022-09-03T05:29:54.037 DEBUG:boto:Params: {} 2022-09-03T05:29:54.038 DEBUG:boto:Token: None 2022-09-03T05:29:54.038 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone1 2022-09-03T05:29:54.039 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TNsDjbu3SIBhzYTMwYc7OsTOVyI= 2022-09-03T05:29:54.039 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TNsDjbu3SIBhzYTMwYc7OsTOVyI=', 'Content-Length': '0'} 2022-09-03T05:29:54.042 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ce5bacd88ff6243d-006312e652-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.043 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:29:54.043 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:54.044 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:54.044 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:54.045 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:54.046 DEBUG:boto:Method: GET 2022-09-03T05:29:54.046 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:54.047 DEBUG:boto:Data: 2022-09-03T05:29:54.047 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.048 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:54.048 DEBUG:boto:Port: 8001 2022-09-03T05:29:54.049 DEBUG:boto:Params: {} 2022-09-03T05:29:54.050 DEBUG:boto:Token: None 2022-09-03T05:29:54.050 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:54.051 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:76Jhk9HolVdv3jie5r6rwTbuRPE= 2022-09-03T05:29:54.051 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:76Jhk9HolVdv3jie5r6rwTbuRPE=', 'Content-Length': '0'} 2022-09-03T05:29:54.053 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005fb993a52233c720-006312e652-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.098 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:54.099 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:29:54.099 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:54.100 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:54.101 DEBUG:boto:Method: GET 2022-09-03T05:29:54.101 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:54.102 DEBUG:boto:Data: 2022-09-03T05:29:54.103 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.104 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:54.105 DEBUG:boto:Port: 8000 2022-09-03T05:29:54.106 DEBUG:boto:Params: {} 2022-09-03T05:29:54.107 DEBUG:boto:Token: None 2022-09-03T05:29:54.108 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:29:54.108 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:76Jhk9HolVdv3jie5r6rwTbuRPE= 2022-09-03T05:29:54.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:76Jhk9HolVdv3jie5r6rwTbuRPE=', 'Content-Length': '0'} 2022-09-03T05:29:54.112 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:08 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000003ab55583c32f8d9-006312e652-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.112 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:54.113 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:54.114 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:54.114 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:54.115 DEBUG:boto:Method: GET 2022-09-03T05:29:54.115 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:54.116 DEBUG:boto:Data: 2022-09-03T05:29:54.116 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.117 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:54.118 DEBUG:boto:Port: 8001 2022-09-03T05:29:54.119 DEBUG:boto:Params: {} 2022-09-03T05:29:54.120 DEBUG:boto:Token: None 2022-09-03T05:29:54.122 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/ 2022-09-03T05:29:54.123 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo= 2022-09-03T05:29:54.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo=', 'Content-Length': '0'} 2022-09-03T05:29:54.127 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ac516715f74acc4d-006312e652-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.129 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:54.130 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:54.130 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:54.131 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:54.132 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:54.132 DEBUG:boto:Method: GET 2022-09-03T05:29:54.133 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:29:54.134 DEBUG:boto:Data: 2022-09-03T05:29:54.134 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.135 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:54.136 DEBUG:boto:Port: 8000 2022-09-03T05:29:54.137 DEBUG:boto:Params: {} 2022-09-03T05:29:54.138 DEBUG:boto:Token: None 2022-09-03T05:29:54.139 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/ 2022-09-03T05:29:54.140 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo= 2022-09-03T05:29:54.141 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo=', 'Content-Length': '0'} 2022-09-03T05:29:54.144 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cb3d9224c1984d02-006312e652-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.145 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:54.146 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:29:54.147 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.147 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.148 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.148 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.149 DEBUG:boto:Method: GET 2022-09-03T05:29:54.149 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.150 DEBUG:boto:Data: 2022-09-03T05:29:54.151 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.152 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:54.152 DEBUG:boto:Port: 8001 2022-09-03T05:29:54.153 DEBUG:boto:Params: {} 2022-09-03T05:29:54.153 DEBUG:boto:Token: None 2022-09-03T05:29:54.155 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.156 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jXYPO6yKUTIlEfpWZ0KIPqbOAck= 2022-09-03T05:29:54.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jXYPO6yKUTIlEfpWZ0KIPqbOAck=', 'Content-Length': '0'} 2022-09-03T05:29:54.158 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003d5ed3234374f502-006312e652-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.159 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.160 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.160 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.161 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.161 DEBUG:boto:Method: GET 2022-09-03T05:29:54.162 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.162 DEBUG:boto:Data: 2022-09-03T05:29:54.163 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.163 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:54.164 DEBUG:boto:Port: 8000 2022-09-03T05:29:54.164 DEBUG:boto:Params: {} 2022-09-03T05:29:54.165 DEBUG:boto:Token: None 2022-09-03T05:29:54.166 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.166 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jXYPO6yKUTIlEfpWZ0KIPqbOAck= 2022-09-03T05:29:54.167 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jXYPO6yKUTIlEfpWZ0KIPqbOAck=', 'Content-Length': '0'} 2022-09-03T05:29:54.169 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d59977be46a68427-006312e652-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.170 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.170 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.171 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.172 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.172 DEBUG:boto:Method: HEAD 2022-09-03T05:29:54.173 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.174 DEBUG:boto:Data: 2022-09-03T05:29:54.174 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.175 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:54.175 DEBUG:boto:Port: 8001 2022-09-03T05:29:54.176 DEBUG:boto:Params: {} 2022-09-03T05:29:54.176 DEBUG:boto:Token: None 2022-09-03T05:29:54.177 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.178 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BrNGtX/5tJ6OrfYfwJRYBd3yysY= 2022-09-03T05:29:54.178 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BrNGtX/5tJ6OrfYfwJRYBd3yysY=', 'Content-Length': '0'} 2022-09-03T05:29:54.180 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008f9c68230c8261f7-006312e652-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.181 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.182 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.182 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.183 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.183 DEBUG:boto:Method: HEAD 2022-09-03T05:29:54.184 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.184 DEBUG:boto:Data: 2022-09-03T05:29:54.185 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.185 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:54.186 DEBUG:boto:Port: 8000 2022-09-03T05:29:54.186 DEBUG:boto:Params: {} 2022-09-03T05:29:54.187 DEBUG:boto:Token: None 2022-09-03T05:29:54.187 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.188 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BrNGtX/5tJ6OrfYfwJRYBd3yysY= 2022-09-03T05:29:54.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BrNGtX/5tJ6OrfYfwJRYBd3yysY=', 'Content-Length': '0'} 2022-09-03T05:29:54.193 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004669240827d5d121-006312e652-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.193 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:29:54.194 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.195 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.195 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.196 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.196 DEBUG:boto:Method: GET 2022-09-03T05:29:54.197 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.197 DEBUG:boto:Data: 2022-09-03T05:29:54.198 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.199 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:54.199 DEBUG:boto:Port: 8001 2022-09-03T05:29:54.200 DEBUG:boto:Params: {} 2022-09-03T05:29:54.200 DEBUG:boto:Token: None 2022-09-03T05:29:54.201 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.202 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jXYPO6yKUTIlEfpWZ0KIPqbOAck= 2022-09-03T05:29:54.202 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jXYPO6yKUTIlEfpWZ0KIPqbOAck=', 'Content-Length': '0'} 2022-09-03T05:29:54.204 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007a197cfa56541565-006312e652-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.250 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.251 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.252 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.253 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.254 DEBUG:boto:Method: GET 2022-09-03T05:29:54.255 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.255 DEBUG:boto:Data: 2022-09-03T05:29:54.256 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.256 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:54.257 DEBUG:boto:Port: 8000 2022-09-03T05:29:54.257 DEBUG:boto:Params: {} 2022-09-03T05:29:54.258 DEBUG:boto:Token: None 2022-09-03T05:29:54.258 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.259 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jXYPO6yKUTIlEfpWZ0KIPqbOAck= 2022-09-03T05:29:54.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jXYPO6yKUTIlEfpWZ0KIPqbOAck=', 'Content-Length': '0'} 2022-09-03T05:29:54.262 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000256fe27c933dd72f-006312e652-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.263 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:54.264 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:54.264 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:54.265 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:54.265 DEBUG:boto:Method: GET 2022-09-03T05:29:54.266 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:54.267 DEBUG:boto:Data: 2022-09-03T05:29:54.267 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.268 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:54.268 DEBUG:boto:Port: 8001 2022-09-03T05:29:54.269 DEBUG:boto:Params: {} 2022-09-03T05:29:54.269 DEBUG:boto:Token: None 2022-09-03T05:29:54.270 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/ 2022-09-03T05:29:54.271 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo= 2022-09-03T05:29:54.271 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo=', 'Content-Length': '0'} 2022-09-03T05:29:54.274 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cc4fd263e7bf5ef7-006312e652-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.275 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:54.276 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:54.276 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:54.277 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:54.278 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:54.278 DEBUG:boto:Method: GET 2022-09-03T05:29:54.279 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:54.280 DEBUG:boto:Data: 2022-09-03T05:29:54.281 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.282 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:54.283 DEBUG:boto:Port: 8000 2022-09-03T05:29:54.284 DEBUG:boto:Params: {} 2022-09-03T05:29:54.285 DEBUG:boto:Token: None 2022-09-03T05:29:54.286 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/ 2022-09-03T05:29:54.287 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo= 2022-09-03T05:29:54.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo=', 'Content-Length': '0'} 2022-09-03T05:29:54.291 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c214b425b22e9ae7-006312e652-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.334 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:54.335 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.336 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.336 DEBUG:boto:Method: HEAD 2022-09-03T05:29:54.337 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.337 DEBUG:boto:Data: 2022-09-03T05:29:54.338 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.338 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:54.339 DEBUG:boto:Port: 8001 2022-09-03T05:29:54.340 DEBUG:boto:Params: {} 2022-09-03T05:29:54.340 DEBUG:boto:Token: None 2022-09-03T05:29:54.341 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.342 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5+X3HNSyoOmosjN8FVF0vHekpXM= 2022-09-03T05:29:54.342 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5+X3HNSyoOmosjN8FVF0vHekpXM=', 'Content-Length': '0'} 2022-09-03T05:29:54.345 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b89a78b0686bff10-006312e652-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.345 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.346 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.347 DEBUG:boto:Method: HEAD 2022-09-03T05:29:54.347 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.348 DEBUG:boto:Data: 2022-09-03T05:29:54.348 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.348 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:54.349 DEBUG:boto:Port: 8000 2022-09-03T05:29:54.349 DEBUG:boto:Params: {} 2022-09-03T05:29:54.351 DEBUG:boto:Token: None 2022-09-03T05:29:54.351 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.352 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5+X3HNSyoOmosjN8FVF0vHekpXM= 2022-09-03T05:29:54.353 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5+X3HNSyoOmosjN8FVF0vHekpXM=', 'Content-Length': '0'} 2022-09-03T05:29:54.355 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d92851bd677f0883-006312e652-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.356 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:29:54.356 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.357 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.358 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.358 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.359 DEBUG:boto:Method: GET 2022-09-03T05:29:54.360 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.362 DEBUG:boto:Data: 2022-09-03T05:29:54.363 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.363 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:54.364 DEBUG:boto:Port: 8001 2022-09-03T05:29:54.365 DEBUG:boto:Params: {} 2022-09-03T05:29:54.365 DEBUG:boto:Token: None 2022-09-03T05:29:54.366 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.367 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jXYPO6yKUTIlEfpWZ0KIPqbOAck= 2022-09-03T05:29:54.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jXYPO6yKUTIlEfpWZ0KIPqbOAck=', 'Content-Length': '0'} 2022-09-03T05:29:54.370 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005e795cb4e986d2d2-006312e652-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.371 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.372 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:29:54.373 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.373 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.374 DEBUG:boto:Method: GET 2022-09-03T05:29:54.375 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.375 DEBUG:boto:Data: 2022-09-03T05:29:54.376 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.376 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:54.377 DEBUG:boto:Port: 8000 2022-09-03T05:29:54.377 DEBUG:boto:Params: {} 2022-09-03T05:29:54.378 DEBUG:boto:Token: None 2022-09-03T05:29:54.379 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:29:54.379 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jXYPO6yKUTIlEfpWZ0KIPqbOAck= 2022-09-03T05:29:54.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jXYPO6yKUTIlEfpWZ0KIPqbOAck=', 'Content-Length': '0'} 2022-09-03T05:29:54.382 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000581178e265ec057-006312e652-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.384 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:54.385 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:54.386 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:54.387 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:54.388 DEBUG:boto:Method: GET 2022-09-03T05:29:54.389 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:54.390 DEBUG:boto:Data: 2022-09-03T05:29:54.391 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.392 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:54.392 DEBUG:boto:Port: 8001 2022-09-03T05:29:54.393 DEBUG:boto:Params: {} 2022-09-03T05:29:54.394 DEBUG:boto:Token: None 2022-09-03T05:29:54.396 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/ 2022-09-03T05:29:54.397 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo= 2022-09-03T05:29:54.398 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo=', 'Content-Length': '0'} 2022-09-03T05:29:54.401 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002ae39cf45f1d2b77-006312e652-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.402 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:54.403 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:54.404 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:54.404 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:54.405 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:54.405 DEBUG:boto:Method: GET 2022-09-03T05:29:54.406 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:29:54.407 DEBUG:boto:Data: 2022-09-03T05:29:54.407 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.408 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:54.408 DEBUG:boto:Port: 8000 2022-09-03T05:29:54.409 DEBUG:boto:Params: {} 2022-09-03T05:29:54.410 DEBUG:boto:Token: None 2022-09-03T05:29:54.410 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/ 2022-09-03T05:29:54.411 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo= 2022-09-03T05:29:54.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo=', 'Content-Length': '0'} 2022-09-03T05:29:54.414 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000051282f2b05c1fd45-006312e652-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.458 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:54.459 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:29:54.460 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.460 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.461 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.461 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.462 DEBUG:boto:Method: GET 2022-09-03T05:29:54.463 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.463 DEBUG:boto:Data: 2022-09-03T05:29:54.464 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.464 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:54.465 DEBUG:boto:Port: 8001 2022-09-03T05:29:54.466 DEBUG:boto:Params: {} 2022-09-03T05:29:54.466 DEBUG:boto:Token: None 2022-09-03T05:29:54.467 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.468 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p+73wh6eIQzWWiZB/JbsmfZfnos= 2022-09-03T05:29:54.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p+73wh6eIQzWWiZB/JbsmfZfnos=', 'Content-Length': '0'} 2022-09-03T05:29:54.470 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008c068b6bd33eec64-006312e652-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.472 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.473 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.474 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.475 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.476 DEBUG:boto:Method: GET 2022-09-03T05:29:54.477 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.477 DEBUG:boto:Data: 2022-09-03T05:29:54.478 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.479 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:54.480 DEBUG:boto:Port: 8000 2022-09-03T05:29:54.481 DEBUG:boto:Params: {} 2022-09-03T05:29:54.481 DEBUG:boto:Token: None 2022-09-03T05:29:54.482 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p+73wh6eIQzWWiZB/JbsmfZfnos= 2022-09-03T05:29:54.483 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p+73wh6eIQzWWiZB/JbsmfZfnos=', 'Content-Length': '0'} 2022-09-03T05:29:54.486 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000787d73a18670d504-006312e652-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.488 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.489 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.490 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.490 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.491 DEBUG:boto:Method: HEAD 2022-09-03T05:29:54.491 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.492 DEBUG:boto:Data: 2022-09-03T05:29:54.492 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.493 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:54.493 DEBUG:boto:Port: 8001 2022-09-03T05:29:54.494 DEBUG:boto:Params: {} 2022-09-03T05:29:54.494 DEBUG:boto:Token: None 2022-09-03T05:29:54.495 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.496 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rAxL66N2SuYPDt2xo7Ke7qfT1ms= 2022-09-03T05:29:54.496 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rAxL66N2SuYPDt2xo7Ke7qfT1ms=', 'Content-Length': '0'} 2022-09-03T05:29:54.498 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000058f508381a0a6a07-006312e652-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.499 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.500 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.500 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.501 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.501 DEBUG:boto:Method: HEAD 2022-09-03T05:29:54.502 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.502 DEBUG:boto:Data: 2022-09-03T05:29:54.503 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.503 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:54.504 DEBUG:boto:Port: 8000 2022-09-03T05:29:54.504 DEBUG:boto:Params: {} 2022-09-03T05:29:54.505 DEBUG:boto:Token: None 2022-09-03T05:29:54.506 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.506 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rAxL66N2SuYPDt2xo7Ke7qfT1ms= 2022-09-03T05:29:54.507 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rAxL66N2SuYPDt2xo7Ke7qfT1ms=', 'Content-Length': '0'} 2022-09-03T05:29:54.509 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000565ed0c57c3ec8c7-006312e652-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.509 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:29:54.510 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.511 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.511 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.512 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.512 DEBUG:boto:Method: GET 2022-09-03T05:29:54.513 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.513 DEBUG:boto:Data: 2022-09-03T05:29:54.514 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.514 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:54.515 DEBUG:boto:Port: 8001 2022-09-03T05:29:54.516 DEBUG:boto:Params: {} 2022-09-03T05:29:54.516 DEBUG:boto:Token: None 2022-09-03T05:29:54.517 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.518 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p+73wh6eIQzWWiZB/JbsmfZfnos= 2022-09-03T05:29:54.518 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p+73wh6eIQzWWiZB/JbsmfZfnos=', 'Content-Length': '0'} 2022-09-03T05:29:54.520 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000dc62ec9e53b342b-006312e652-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.566 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.567 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.568 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.568 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.569 DEBUG:boto:Method: GET 2022-09-03T05:29:54.569 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.570 DEBUG:boto:Data: 2022-09-03T05:29:54.571 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.571 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:54.572 DEBUG:boto:Port: 8000 2022-09-03T05:29:54.572 DEBUG:boto:Params: {} 2022-09-03T05:29:54.573 DEBUG:boto:Token: None 2022-09-03T05:29:54.574 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.574 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p+73wh6eIQzWWiZB/JbsmfZfnos= 2022-09-03T05:29:54.575 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p+73wh6eIQzWWiZB/JbsmfZfnos=', 'Content-Length': '0'} 2022-09-03T05:29:54.578 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000094439bf9f45e67e3-006312e652-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.580 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:54.581 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:54.582 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:54.583 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:54.583 DEBUG:boto:Method: GET 2022-09-03T05:29:54.584 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:54.585 DEBUG:boto:Data: 2022-09-03T05:29:54.586 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.587 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:54.587 DEBUG:boto:Port: 8001 2022-09-03T05:29:54.588 DEBUG:boto:Params: {} 2022-09-03T05:29:54.588 DEBUG:boto:Token: None 2022-09-03T05:29:54.589 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/ 2022-09-03T05:29:54.590 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo= 2022-09-03T05:29:54.591 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo=', 'Content-Length': '0'} 2022-09-03T05:29:54.594 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c8ec01f8593dd844-006312e652-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.595 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:54.597 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:54.598 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:54.598 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:54.599 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:54.599 DEBUG:boto:Method: GET 2022-09-03T05:29:54.600 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:54.601 DEBUG:boto:Data: 2022-09-03T05:29:54.602 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.602 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:54.603 DEBUG:boto:Port: 8000 2022-09-03T05:29:54.603 DEBUG:boto:Params: {} 2022-09-03T05:29:54.604 DEBUG:boto:Token: None 2022-09-03T05:29:54.605 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/ 2022-09-03T05:29:54.605 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo= 2022-09-03T05:29:54.606 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo=', 'Content-Length': '0'} 2022-09-03T05:29:54.609 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002392343298f995e1-006312e652-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.650 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:54.651 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.652 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.652 DEBUG:boto:Method: HEAD 2022-09-03T05:29:54.653 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.653 DEBUG:boto:Data: 2022-09-03T05:29:54.654 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.654 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:54.655 DEBUG:boto:Port: 8001 2022-09-03T05:29:54.655 DEBUG:boto:Params: {} 2022-09-03T05:29:54.656 DEBUG:boto:Token: None 2022-09-03T05:29:54.657 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.657 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f0NSqug8mYWhp2Mla6W/+tdBWoY= 2022-09-03T05:29:54.658 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f0NSqug8mYWhp2Mla6W/+tdBWoY=', 'Content-Length': '0'} 2022-09-03T05:29:54.660 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000065ce10ed603bdc2c-006312e652-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.661 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.662 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.662 DEBUG:boto:Method: HEAD 2022-09-03T05:29:54.663 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.663 DEBUG:boto:Data: 2022-09-03T05:29:54.664 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.664 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:54.665 DEBUG:boto:Port: 8000 2022-09-03T05:29:54.665 DEBUG:boto:Params: {} 2022-09-03T05:29:54.666 DEBUG:boto:Token: None 2022-09-03T05:29:54.667 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.667 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f0NSqug8mYWhp2Mla6W/+tdBWoY= 2022-09-03T05:29:54.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f0NSqug8mYWhp2Mla6W/+tdBWoY=', 'Content-Length': '0'} 2022-09-03T05:29:54.671 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000025501ed0c8ee4356-006312e652-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.672 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:29:54.672 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.673 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.674 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.674 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.675 DEBUG:boto:Method: GET 2022-09-03T05:29:54.675 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.676 DEBUG:boto:Data: 2022-09-03T05:29:54.676 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.677 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:54.678 DEBUG:boto:Port: 8001 2022-09-03T05:29:54.678 DEBUG:boto:Params: {} 2022-09-03T05:29:54.679 DEBUG:boto:Token: None 2022-09-03T05:29:54.679 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.680 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p+73wh6eIQzWWiZB/JbsmfZfnos= 2022-09-03T05:29:54.681 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p+73wh6eIQzWWiZB/JbsmfZfnos=', 'Content-Length': '0'} 2022-09-03T05:29:54.683 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c001969450869837-006312e652-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.684 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.685 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:29:54.685 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.686 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.686 DEBUG:boto:Method: GET 2022-09-03T05:29:54.687 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.687 DEBUG:boto:Data: 2022-09-03T05:29:54.688 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.689 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:54.689 DEBUG:boto:Port: 8000 2022-09-03T05:29:54.690 DEBUG:boto:Params: {} 2022-09-03T05:29:54.690 DEBUG:boto:Token: None 2022-09-03T05:29:54.691 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:29:54.692 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p+73wh6eIQzWWiZB/JbsmfZfnos= 2022-09-03T05:29:54.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p+73wh6eIQzWWiZB/JbsmfZfnos=', 'Content-Length': '0'} 2022-09-03T05:29:54.695 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ee66e2ab8fbf0ebf-006312e652-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.696 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:54.696 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:54.697 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:54.698 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:54.698 DEBUG:boto:Method: GET 2022-09-03T05:29:54.699 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:54.699 DEBUG:boto:Data: 2022-09-03T05:29:54.700 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.700 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:29:54.701 DEBUG:boto:Port: 8001 2022-09-03T05:29:54.702 DEBUG:boto:Params: {} 2022-09-03T05:29:54.702 DEBUG:boto:Token: None 2022-09-03T05:29:54.703 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/ 2022-09-03T05:29:54.704 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo= 2022-09-03T05:29:54.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo=', 'Content-Length': '0'} 2022-09-03T05:29:54.707 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003d4d1ec8af0c8347-006312e652-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.708 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:54.709 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:29:54.709 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:29:54.710 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:54.711 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:54.711 DEBUG:boto:Method: GET 2022-09-03T05:29:54.712 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:29:54.712 DEBUG:boto:Data: 2022-09-03T05:29:54.713 DEBUG:boto:Headers: {} 2022-09-03T05:29:54.713 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:29:54.714 DEBUG:boto:Port: 8000 2022-09-03T05:29:54.714 DEBUG:boto:Params: {} 2022-09-03T05:29:54.715 DEBUG:boto:Token: None 2022-09-03T05:29:54.716 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:29:54 GMT /dsgvcy-16/ 2022-09-03T05:29:54.716 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo= 2022-09-03T05:29:54.717 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:29:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o3heX4doJ1e7RzeF/+dwXxVXPwo=', 'Content-Length': '0'} 2022-09-03T05:29:54.720 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000027a6dcc7d9369422-006312e652-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:29:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:29:54.766 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:29:54.767 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-16 zones={test-zone3, test-zone2} 2022-09-03T05:29:54.768 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-17 2022-09-03T05:29:54.769 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:29:54.769 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:29:54.770 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-17 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:29:54.834 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:29:54.835 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:29:54.884 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:29:54.887+0000 7fe2cb615600 1 bucket sync caught up with source: 2022-09-03T05:29:54.885 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000010.30.5, , 00000000003.13.9, , 00000000007.18.9, , , , , ] 2022-09-03T05:29:54.885 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000010.30.5, , 00000000003.13.9, , 00000000007.18.9, , , , , ] 2022-09-03T05:29:54.885 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:29:54.887+0000 7fe2cb615600 0 bucket checkpoint complete 2022-09-03T05:29:54.902 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-17 2022-09-03T05:29:54.903 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:29:54.903 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:29:54.904 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-17 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:29:54.966 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:29:54.966 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:29:55.000 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:29:55.002+0000 7f73b176c600 1 bucket sync caught up with source: 2022-09-03T05:29:55.001 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000010.30.5, , 00000000003.13.9, , 00000000007.18.9, , , , , ] 2022-09-03T05:29:55.001 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000010.30.5, , 00000000003.13.9, , 00000000007.18.9, , , , , ] 2022-09-03T05:29:55.001 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:29:55.002+0000 7f73b176c600 0 bucket checkpoint complete 2022-09-03T05:29:55.013 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-17 2022-09-03T05:29:55.013 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:29:55.014 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:29:55.015 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-17 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:29:55.073 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:29:55.074 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:29:55.109 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:29:55.109+0000 7f89dc124600 1 waiting for incremental sync to catch up: 2022-09-03T05:29:55.110 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000010.30.5, , , , 00000000005.15.10, , , , , ] 2022-09-03T05:29:55.110 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000010.30.5, , 00000000003.13.9, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:00.108 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:30:00.109+0000 7f89dc124600 1 bucket sync caught up with source: 2022-09-03T05:30:00.108 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000010.30.5, , 00000000003.13.9, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:00.109 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000010.30.5, , 00000000003.13.9, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:00.109 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:30:00.109+0000 7f89dc124600 0 bucket checkpoint complete 2022-09-03T05:30:00.128 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-17 2022-09-03T05:30:00.129 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:30:00.129 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:30:00.130 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-17 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:30:00.194 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:30:00.194 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:30:00.255 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:30:00.257+0000 7f5457c81600 1 waiting for incremental sync to catch up: 2022-09-03T05:30:00.256 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.31.10, , 00000000010.64.5, , 00000000008.36.9, , , , , ] 2022-09-03T05:30:00.256 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000003.33.9, , 00000000010.64.5, , 00000000009.39.9, , , , , ] 2022-09-03T05:30:05.260 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:30:05.259+0000 7f5457c81600 1 waiting for incremental sync to catch up: 2022-09-03T05:30:05.261 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.31.10, , 00000000010.64.5, , 00000000008.36.9, , , , , ] 2022-09-03T05:30:05.261 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000003.33.9, , 00000000010.64.5, , 00000000009.39.9, , , , , ] 2022-09-03T05:30:10.259 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:30:10.260+0000 7f5457c81600 1 bucket sync caught up with source: 2022-09-03T05:30:10.260 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000003.33.9, , 00000000010.64.5, , 00000000009.39.9, , , , , ] 2022-09-03T05:30:10.260 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000003.33.9, , 00000000010.64.5, , 00000000009.39.9, , , , , ] 2022-09-03T05:30:10.261 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:30:10.260+0000 7f5457c81600 0 bucket checkpoint complete 2022-09-03T05:30:10.284 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-17 2022-09-03T05:30:10.285 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:30:10.286 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:30:10.287 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-17 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:30:10.349 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:30:10.349 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:30:10.405 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:30:10.404+0000 7fb55c5dd600 1 bucket sync caught up with source: 2022-09-03T05:30:10.406 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000003.33.9, , 00000000010.64.5, , 00000000009.39.9, , , , , ] 2022-09-03T05:30:10.406 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000003.33.9, , 00000000010.64.5, , 00000000009.39.9, , , , , ] 2022-09-03T05:30:10.407 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:30:10.404+0000 7fb55c5dd600 0 bucket checkpoint complete 2022-09-03T05:30:10.422 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-17 2022-09-03T05:30:10.423 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-17 2022-09-03T05:30:10.424 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:30:10.425 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:30:10.427 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-17 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:30:10.484 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:30:10.485 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:30:10.542 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:30:10.544+0000 7f313f99d600 1 bucket sync caught up with source: 2022-09-03T05:30:10.543 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000003.29.9, , 00000000003.36.9, , 00000000014.51.5, , , , , ] 2022-09-03T05:30:10.544 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000003.29.9, , 00000000003.36.9, , 00000000014.51.5, , , , , ] 2022-09-03T05:30:10.544 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:30:10.544+0000 7f313f99d600 0 bucket checkpoint complete 2022-09-03T05:30:10.558 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-17 2022-09-03T05:30:10.559 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:30:10.560 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:30:10.560 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-17 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:30:10.624 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:30:10.624 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:30:10.660 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:30:10.662+0000 7fdcbb434600 1 bucket sync caught up with source: 2022-09-03T05:30:10.660 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000003.29.9, , 00000000003.36.9, , 00000000014.51.5, , , , , ] 2022-09-03T05:30:10.660 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000003.29.9, , 00000000003.36.9, , 00000000014.51.5, , , , , ] 2022-09-03T05:30:10.661 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:30:10.662+0000 7fdcbb434600 0 bucket checkpoint complete 2022-09-03T05:30:10.672 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-17 2022-09-03T05:30:10.673 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-17 zones={test-zone2, test-zone1} 2022-09-03T05:30:10.673 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:10.674 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:10.675 DEBUG:boto:Method: HEAD 2022-09-03T05:30:10.675 DEBUG:boto:Path: /dsgvcy-17/ 2022-09-03T05:30:10.676 DEBUG:boto:Data: 2022-09-03T05:30:10.676 DEBUG:boto:Headers: {} 2022-09-03T05:30:10.677 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:10.678 DEBUG:boto:Port: 8000 2022-09-03T05:30:10.678 DEBUG:boto:Params: {} 2022-09-03T05:30:10.679 DEBUG:boto:Token: None 2022-09-03T05:30:10.680 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:10 GMT /dsgvcy-17/ 2022-09-03T05:30:10.681 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zP3j4OGUjb4Iu2hGTMPFIbv1yOk= 2022-09-03T05:30:10.681 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zP3j4OGUjb4Iu2hGTMPFIbv1yOk=', 'Content-Length': '0'} 2022-09-03T05:30:10.685 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f866dc8df09c3bfa-006312e662-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:30:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:10.685 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:10.686 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:10.687 DEBUG:boto:Method: HEAD 2022-09-03T05:30:10.687 DEBUG:boto:Path: /dsgvcy-17/ 2022-09-03T05:30:10.688 DEBUG:boto:Data: 2022-09-03T05:30:10.689 DEBUG:boto:Headers: {} 2022-09-03T05:30:10.690 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:10.691 DEBUG:boto:Port: 8000 2022-09-03T05:30:10.691 DEBUG:boto:Params: {} 2022-09-03T05:30:10.692 DEBUG:boto:Token: None 2022-09-03T05:30:10.693 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:10 GMT /dsgvcy-17/ 2022-09-03T05:30:10.694 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zP3j4OGUjb4Iu2hGTMPFIbv1yOk= 2022-09-03T05:30:10.695 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zP3j4OGUjb4Iu2hGTMPFIbv1yOk=', 'Content-Length': '0'} 2022-09-03T05:30:10.698 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b6316809f287f8f6-006312e662-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:30:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:10.699 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:30:10.700 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:10.701 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:10.701 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:10.702 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:10.703 DEBUG:boto:Method: GET 2022-09-03T05:30:10.703 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:10.704 DEBUG:boto:Data: 2022-09-03T05:30:10.705 DEBUG:boto:Headers: {} 2022-09-03T05:30:10.705 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:10.706 DEBUG:boto:Port: 8000 2022-09-03T05:30:10.706 DEBUG:boto:Params: {} 2022-09-03T05:30:10.707 DEBUG:boto:Token: None 2022-09-03T05:30:10.708 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:10 GMT /dsgvcy-17/?versions 2022-09-03T05:30:10.708 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b/wuB2/BWWx5M0yQ4tTPIn6oooE= 2022-09-03T05:30:10.709 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b/wuB2/BWWx5M0yQ4tTPIn6oooE=', 'Content-Length': '0'} 2022-09-03T05:30:10.712 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007ad868473fec816b-006312e662-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:10.716 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:10.718 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:30:10.719 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:30:10.721 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:30:10.721 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:30:10.722 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:30:10.723 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:10.724 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:10.725 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:10.725 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:10.726 DEBUG:boto:Method: GET 2022-09-03T05:30:10.726 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:10.727 DEBUG:boto:Data: 2022-09-03T05:30:10.728 DEBUG:boto:Headers: {} 2022-09-03T05:30:10.728 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:10.729 DEBUG:boto:Port: 8000 2022-09-03T05:30:10.729 DEBUG:boto:Params: {} 2022-09-03T05:30:10.730 DEBUG:boto:Token: None 2022-09-03T05:30:10.730 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:10 GMT /dsgvcy-17/?versions 2022-09-03T05:30:10.731 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b/wuB2/BWWx5M0yQ4tTPIn6oooE= 2022-09-03T05:30:10.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b/wuB2/BWWx5M0yQ4tTPIn6oooE=', 'Content-Length': '0'} 2022-09-03T05:30:10.734 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e07181d8f6450a6f-006312e662-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:10.735 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:10.737 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:30:10.737 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:30:10.738 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:30:10.739 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:30:10.740 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:10.741 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:10.742 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:10.742 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:10.743 DEBUG:boto:Method: GET 2022-09-03T05:30:10.743 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:10.744 DEBUG:boto:Data: 2022-09-03T05:30:10.744 DEBUG:boto:Headers: {} 2022-09-03T05:30:10.745 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:10.745 DEBUG:boto:Port: 8000 2022-09-03T05:30:10.746 DEBUG:boto:Params: {} 2022-09-03T05:30:10.746 DEBUG:boto:Token: None 2022-09-03T05:30:10.747 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:10 GMT /dsgvcy-17/?versions 2022-09-03T05:30:10.747 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b/wuB2/BWWx5M0yQ4tTPIn6oooE= 2022-09-03T05:30:10.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b/wuB2/BWWx5M0yQ4tTPIn6oooE=', 'Content-Length': '0'} 2022-09-03T05:30:10.750 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009c591a52ca3c80e8-006312e662-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:10.751 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:10.752 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:10.753 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:10.753 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:10.754 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:10.754 DEBUG:boto:Method: GET 2022-09-03T05:30:10.755 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:10.756 DEBUG:boto:Data: 2022-09-03T05:30:10.757 DEBUG:boto:Headers: {} 2022-09-03T05:30:10.757 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:10.758 DEBUG:boto:Port: 8000 2022-09-03T05:30:10.759 DEBUG:boto:Params: {} 2022-09-03T05:30:10.760 DEBUG:boto:Token: None 2022-09-03T05:30:10.761 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:10 GMT /dsgvcy-17/?versions 2022-09-03T05:30:10.762 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b/wuB2/BWWx5M0yQ4tTPIn6oooE= 2022-09-03T05:30:10.763 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b/wuB2/BWWx5M0yQ4tTPIn6oooE=', 'Content-Length': '0'} 2022-09-03T05:30:10.766 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b3439ada69cb57ba-006312e662-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:10.767 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:10.768 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:30:10.769 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:10.769 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:10.770 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.770 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.771 DEBUG:boto:Method: GET 2022-09-03T05:30:10.771 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.772 DEBUG:boto:Data: 2022-09-03T05:30:10.772 DEBUG:boto:Headers: {} 2022-09-03T05:30:10.773 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:10.774 DEBUG:boto:Port: 8000 2022-09-03T05:30:10.774 DEBUG:boto:Params: {} 2022-09-03T05:30:10.775 DEBUG:boto:Token: None 2022-09-03T05:30:10.776 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:10 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.776 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wk4xtv6XufUP5puLH4QJA52x7jI= 2022-09-03T05:30:10.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wk4xtv6XufUP5puLH4QJA52x7jI=', 'Content-Length': '0'} 2022-09-03T05:30:10.779 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000004c491f1f7841ea-006312e662-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:10.780 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:10.781 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:10.782 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.783 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.784 DEBUG:boto:Method: GET 2022-09-03T05:30:10.785 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.786 DEBUG:boto:Data: 2022-09-03T05:30:10.787 DEBUG:boto:Headers: {} 2022-09-03T05:30:10.788 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:10.789 DEBUG:boto:Port: 8000 2022-09-03T05:30:10.790 DEBUG:boto:Params: {} 2022-09-03T05:30:10.791 DEBUG:boto:Token: None 2022-09-03T05:30:10.791 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:10 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.792 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wk4xtv6XufUP5puLH4QJA52x7jI= 2022-09-03T05:30:10.793 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wk4xtv6XufUP5puLH4QJA52x7jI=', 'Content-Length': '0'} 2022-09-03T05:30:10.795 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000744879bdb3412a71-006312e662-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:10.796 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:10.796 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:10.797 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.797 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.798 DEBUG:boto:Method: HEAD 2022-09-03T05:30:10.798 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.799 DEBUG:boto:Data: 2022-09-03T05:30:10.799 DEBUG:boto:Headers: {} 2022-09-03T05:30:10.800 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:10.800 DEBUG:boto:Port: 8000 2022-09-03T05:30:10.801 DEBUG:boto:Params: {} 2022-09-03T05:30:10.802 DEBUG:boto:Token: None 2022-09-03T05:30:10.802 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:10 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.803 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2JTetKSjFDHS4iaIytoWJhbXk00= 2022-09-03T05:30:10.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2JTetKSjFDHS4iaIytoWJhbXk00=', 'Content-Length': '0'} 2022-09-03T05:30:10.805 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f47379ffc0d310e1-006312e662-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:10.807 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:10.808 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:10.809 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.810 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.811 DEBUG:boto:Method: HEAD 2022-09-03T05:30:10.811 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.812 DEBUG:boto:Data: 2022-09-03T05:30:10.812 DEBUG:boto:Headers: {} 2022-09-03T05:30:10.813 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:10.813 DEBUG:boto:Port: 8000 2022-09-03T05:30:10.814 DEBUG:boto:Params: {} 2022-09-03T05:30:10.814 DEBUG:boto:Token: None 2022-09-03T05:30:10.815 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:10 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.816 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2JTetKSjFDHS4iaIytoWJhbXk00= 2022-09-03T05:30:10.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2JTetKSjFDHS4iaIytoWJhbXk00=', 'Content-Length': '0'} 2022-09-03T05:30:10.819 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000038107e5ca56edab0-006312e662-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:10.819 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:30:10.820 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:10.821 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:10.821 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.822 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.822 DEBUG:boto:Method: GET 2022-09-03T05:30:10.823 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.823 DEBUG:boto:Data: 2022-09-03T05:30:10.824 DEBUG:boto:Headers: {} 2022-09-03T05:30:10.824 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:10.825 DEBUG:boto:Port: 8000 2022-09-03T05:30:10.825 DEBUG:boto:Params: {} 2022-09-03T05:30:10.826 DEBUG:boto:Token: None 2022-09-03T05:30:10.827 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:10 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.827 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wk4xtv6XufUP5puLH4QJA52x7jI= 2022-09-03T05:30:10.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wk4xtv6XufUP5puLH4QJA52x7jI=', 'Content-Length': '0'} 2022-09-03T05:30:10.830 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a9326551b26cb8cc-006312e662-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:10.831 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:10.832 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:10.833 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.834 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.834 DEBUG:boto:Method: GET 2022-09-03T05:30:10.835 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.835 DEBUG:boto:Data: 2022-09-03T05:30:10.836 DEBUG:boto:Headers: {} 2022-09-03T05:30:10.837 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:10.837 DEBUG:boto:Port: 8000 2022-09-03T05:30:10.838 DEBUG:boto:Params: {} 2022-09-03T05:30:10.838 DEBUG:boto:Token: None 2022-09-03T05:30:10.839 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:10 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.840 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wk4xtv6XufUP5puLH4QJA52x7jI= 2022-09-03T05:30:10.841 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wk4xtv6XufUP5puLH4QJA52x7jI=', 'Content-Length': '0'} 2022-09-03T05:30:10.843 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000724403465557c354-006312e662-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:10.844 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:10.844 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:10.845 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:10.846 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:10.846 DEBUG:boto:Method: GET 2022-09-03T05:30:10.847 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:30:10.847 DEBUG:boto:Data: 2022-09-03T05:30:10.848 DEBUG:boto:Headers: {} 2022-09-03T05:30:10.849 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:10.849 DEBUG:boto:Port: 8000 2022-09-03T05:30:10.850 DEBUG:boto:Params: {} 2022-09-03T05:30:10.851 DEBUG:boto:Token: None 2022-09-03T05:30:10.851 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:10 GMT /dsgvcy-17/ 2022-09-03T05:30:10.852 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RPolxKWlDmkxZSE7/1OjJ5N5L+I= 2022-09-03T05:30:10.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RPolxKWlDmkxZSE7/1OjJ5N5L+I=', 'Content-Length': '0'} 2022-09-03T05:30:10.855 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000054ab89aa4109683b-006312e662-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:10.856 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:10.857 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:10.857 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:10.858 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:10.858 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:10.859 DEBUG:boto:Method: GET 2022-09-03T05:30:10.859 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:30:10.860 DEBUG:boto:Data: 2022-09-03T05:30:10.860 DEBUG:boto:Headers: {} 2022-09-03T05:30:10.861 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:10.862 DEBUG:boto:Port: 8000 2022-09-03T05:30:10.862 DEBUG:boto:Params: {} 2022-09-03T05:30:10.863 DEBUG:boto:Token: None 2022-09-03T05:30:10.863 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:10 GMT /dsgvcy-17/ 2022-09-03T05:30:10.864 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RPolxKWlDmkxZSE7/1OjJ5N5L+I= 2022-09-03T05:30:10.865 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RPolxKWlDmkxZSE7/1OjJ5N5L+I=', 'Content-Length': '0'} 2022-09-03T05:30:10.869 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000db2a9653a53fe194-006312e662-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:10.910 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:10.911 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:10.912 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:10.912 DEBUG:boto:Method: HEAD 2022-09-03T05:30:10.913 DEBUG:boto:Path: /dsgvcy-17/dummy 2022-09-03T05:30:10.913 DEBUG:boto:Data: 2022-09-03T05:30:10.914 DEBUG:boto:Headers: {} 2022-09-03T05:30:10.914 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:10.915 DEBUG:boto:Port: 8000 2022-09-03T05:30:10.915 DEBUG:boto:Params: {} 2022-09-03T05:30:10.916 DEBUG:boto:Token: None 2022-09-03T05:30:10.916 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:10 GMT /dsgvcy-17/dummy 2022-09-03T05:30:10.917 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r+9iepktIUsaxStz2A8LJFoha7k= 2022-09-03T05:30:10.917 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r+9iepktIUsaxStz2A8LJFoha7k=', 'Content-Length': '0'} 2022-09-03T05:30:10.920 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000d78f6453afcd69e1-006312e662-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:10.921 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:10.922 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:10.923 DEBUG:boto:Method: HEAD 2022-09-03T05:30:10.924 DEBUG:boto:Path: /dsgvcy-17/dummy 2022-09-03T05:30:10.925 DEBUG:boto:Data: 2022-09-03T05:30:10.926 DEBUG:boto:Headers: {} 2022-09-03T05:30:10.926 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:10.927 DEBUG:boto:Port: 8000 2022-09-03T05:30:10.928 DEBUG:boto:Params: {} 2022-09-03T05:30:10.929 DEBUG:boto:Token: None 2022-09-03T05:30:10.930 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:10 GMT /dsgvcy-17/dummy 2022-09-03T05:30:10.931 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r+9iepktIUsaxStz2A8LJFoha7k= 2022-09-03T05:30:10.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r+9iepktIUsaxStz2A8LJFoha7k=', 'Content-Length': '0'} 2022-09-03T05:30:10.934 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000e4bbc7efb59b167c-006312e662-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:10.935 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:30:10.936 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:10.937 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:10.937 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.938 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.939 DEBUG:boto:Method: GET 2022-09-03T05:30:10.939 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.940 DEBUG:boto:Data: 2022-09-03T05:30:10.940 DEBUG:boto:Headers: {} 2022-09-03T05:30:10.941 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:10.941 DEBUG:boto:Port: 8000 2022-09-03T05:30:10.942 DEBUG:boto:Params: {} 2022-09-03T05:30:10.942 DEBUG:boto:Token: None 2022-09-03T05:30:10.943 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:10 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.944 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wk4xtv6XufUP5puLH4QJA52x7jI= 2022-09-03T05:30:10.945 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wk4xtv6XufUP5puLH4QJA52x7jI=', 'Content-Length': '0'} 2022-09-03T05:30:10.947 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000ddd562805119aead-006312e662-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:10.947 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:10.948 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:10.949 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.949 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.950 DEBUG:boto:Method: GET 2022-09-03T05:30:10.950 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.951 DEBUG:boto:Data: 2022-09-03T05:30:10.951 DEBUG:boto:Headers: {} 2022-09-03T05:30:10.952 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:10.953 DEBUG:boto:Port: 8000 2022-09-03T05:30:10.953 DEBUG:boto:Params: {} 2022-09-03T05:30:10.954 DEBUG:boto:Token: None 2022-09-03T05:30:10.955 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:10 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:10.955 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wk4xtv6XufUP5puLH4QJA52x7jI= 2022-09-03T05:30:10.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wk4xtv6XufUP5puLH4QJA52x7jI=', 'Content-Length': '0'} 2022-09-03T05:30:10.958 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000038cd15f65d863a50-006312e662-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:10.959 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:10.959 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:10.960 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:10.960 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:10.961 DEBUG:boto:Method: GET 2022-09-03T05:30:10.961 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:30:10.962 DEBUG:boto:Data: 2022-09-03T05:30:10.962 DEBUG:boto:Headers: {} 2022-09-03T05:30:10.963 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:10.963 DEBUG:boto:Port: 8000 2022-09-03T05:30:10.964 DEBUG:boto:Params: {} 2022-09-03T05:30:10.965 DEBUG:boto:Token: None 2022-09-03T05:30:10.965 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:10 GMT /dsgvcy-17/ 2022-09-03T05:30:10.966 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RPolxKWlDmkxZSE7/1OjJ5N5L+I= 2022-09-03T05:30:10.966 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RPolxKWlDmkxZSE7/1OjJ5N5L+I=', 'Content-Length': '0'} 2022-09-03T05:30:10.969 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000833154c2aaa459d5-006312e662-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.010 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:11.011 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:11.012 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:11.013 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:11.014 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:11.015 DEBUG:boto:Method: GET 2022-09-03T05:30:11.016 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:30:11.017 DEBUG:boto:Data: 2022-09-03T05:30:11.017 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.018 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.019 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.020 DEBUG:boto:Params: {} 2022-09-03T05:30:11.021 DEBUG:boto:Token: None 2022-09-03T05:30:11.022 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/ 2022-09-03T05:30:11.022 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw= 2022-09-03T05:30:11.023 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw=', 'Content-Length': '0'} 2022-09-03T05:30:11.025 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000935fdb70a70262ba-006312e663-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.026 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:11.027 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:30:11.027 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.028 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.028 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.029 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.030 DEBUG:boto:Method: GET 2022-09-03T05:30:11.031 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.031 DEBUG:boto:Data: 2022-09-03T05:30:11.032 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.032 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:11.033 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.033 DEBUG:boto:Params: {} 2022-09-03T05:30:11.034 DEBUG:boto:Token: None 2022-09-03T05:30:11.035 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.036 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uEAAq1UEMmEYQWdkKR9KXpiOWu0= 2022-09-03T05:30:11.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uEAAq1UEMmEYQWdkKR9KXpiOWu0=', 'Content-Length': '0'} 2022-09-03T05:30:11.038 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000085546e643ee76935-006312e663-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.039 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.040 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.040 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.041 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.042 DEBUG:boto:Method: GET 2022-09-03T05:30:11.042 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.043 DEBUG:boto:Data: 2022-09-03T05:30:11.043 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.044 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.044 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.045 DEBUG:boto:Params: {} 2022-09-03T05:30:11.046 DEBUG:boto:Token: None 2022-09-03T05:30:11.047 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.047 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uEAAq1UEMmEYQWdkKR9KXpiOWu0= 2022-09-03T05:30:11.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uEAAq1UEMmEYQWdkKR9KXpiOWu0=', 'Content-Length': '0'} 2022-09-03T05:30:11.050 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000073a25e6c83428fb7-006312e663-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.051 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.052 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.052 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.053 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.053 DEBUG:boto:Method: HEAD 2022-09-03T05:30:11.054 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.054 DEBUG:boto:Data: 2022-09-03T05:30:11.055 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.055 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:11.056 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.056 DEBUG:boto:Params: {} 2022-09-03T05:30:11.057 DEBUG:boto:Token: None 2022-09-03T05:30:11.057 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.058 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LT+I8dtbOwp7pq5eeo9wEb74BX8= 2022-09-03T05:30:11.059 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LT+I8dtbOwp7pq5eeo9wEb74BX8=', 'Content-Length': '0'} 2022-09-03T05:30:11.061 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000213fd19144b06c50-006312e663-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.062 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.062 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.063 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.063 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.064 DEBUG:boto:Method: HEAD 2022-09-03T05:30:11.064 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.065 DEBUG:boto:Data: 2022-09-03T05:30:11.065 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.066 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.066 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.067 DEBUG:boto:Params: {} 2022-09-03T05:30:11.067 DEBUG:boto:Token: None 2022-09-03T05:30:11.068 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.069 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LT+I8dtbOwp7pq5eeo9wEb74BX8= 2022-09-03T05:30:11.069 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LT+I8dtbOwp7pq5eeo9wEb74BX8=', 'Content-Length': '0'} 2022-09-03T05:30:11.074 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fc41c2db11bc7ad6-006312e663-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.075 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:30:11.076 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.077 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.078 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.079 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.080 DEBUG:boto:Method: GET 2022-09-03T05:30:11.081 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.082 DEBUG:boto:Data: 2022-09-03T05:30:11.084 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.085 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:11.086 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.086 DEBUG:boto:Params: {} 2022-09-03T05:30:11.087 DEBUG:boto:Token: None 2022-09-03T05:30:11.088 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.089 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uEAAq1UEMmEYQWdkKR9KXpiOWu0= 2022-09-03T05:30:11.089 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uEAAq1UEMmEYQWdkKR9KXpiOWu0=', 'Content-Length': '0'} 2022-09-03T05:30:11.091 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000daaaf1ad2f191570-006312e663-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.092 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.093 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.094 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.095 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.096 DEBUG:boto:Method: GET 2022-09-03T05:30:11.097 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.098 DEBUG:boto:Data: 2022-09-03T05:30:11.099 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.100 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.101 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.102 DEBUG:boto:Params: {} 2022-09-03T05:30:11.102 DEBUG:boto:Token: None 2022-09-03T05:30:11.103 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.104 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uEAAq1UEMmEYQWdkKR9KXpiOWu0= 2022-09-03T05:30:11.105 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uEAAq1UEMmEYQWdkKR9KXpiOWu0=', 'Content-Length': '0'} 2022-09-03T05:30:11.107 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009c6b1fee8850b26d-006312e663-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.108 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:11.108 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:11.109 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:11.110 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:11.110 DEBUG:boto:Method: GET 2022-09-03T05:30:11.111 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:11.111 DEBUG:boto:Data: 2022-09-03T05:30:11.112 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.112 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:11.113 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.114 DEBUG:boto:Params: {} 2022-09-03T05:30:11.115 DEBUG:boto:Token: None 2022-09-03T05:30:11.116 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/ 2022-09-03T05:30:11.117 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw= 2022-09-03T05:30:11.118 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw=', 'Content-Length': '0'} 2022-09-03T05:30:11.120 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c8d93e5b39b7f5cf-006312e663-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.121 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:11.123 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:11.124 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:11.125 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:11.125 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:11.126 DEBUG:boto:Method: GET 2022-09-03T05:30:11.127 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:11.128 DEBUG:boto:Data: 2022-09-03T05:30:11.129 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.130 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.131 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.132 DEBUG:boto:Params: {} 2022-09-03T05:30:11.133 DEBUG:boto:Token: None 2022-09-03T05:30:11.134 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/ 2022-09-03T05:30:11.135 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw= 2022-09-03T05:30:11.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw=', 'Content-Length': '0'} 2022-09-03T05:30:11.138 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000037203b28ed97d0b1-006312e663-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.139 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:11.140 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.141 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.141 DEBUG:boto:Method: HEAD 2022-09-03T05:30:11.142 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.142 DEBUG:boto:Data: 2022-09-03T05:30:11.143 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.143 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:11.144 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.144 DEBUG:boto:Params: {} 2022-09-03T05:30:11.145 DEBUG:boto:Token: None 2022-09-03T05:30:11.145 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.146 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oW+4iyDDnT3vpc389ee24kK0K70= 2022-09-03T05:30:11.146 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oW+4iyDDnT3vpc389ee24kK0K70=', 'Content-Length': '0'} 2022-09-03T05:30:11.149 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dd7f56462ff87791-006312e663-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.150 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.150 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.151 DEBUG:boto:Method: HEAD 2022-09-03T05:30:11.151 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.152 DEBUG:boto:Data: 2022-09-03T05:30:11.152 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.153 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.153 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.154 DEBUG:boto:Params: {} 2022-09-03T05:30:11.155 DEBUG:boto:Token: None 2022-09-03T05:30:11.155 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.156 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oW+4iyDDnT3vpc389ee24kK0K70= 2022-09-03T05:30:11.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oW+4iyDDnT3vpc389ee24kK0K70=', 'Content-Length': '0'} 2022-09-03T05:30:11.159 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000423544b6e5908ac2-006312e663-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.160 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:30:11.161 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.162 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.163 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.165 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.166 DEBUG:boto:Method: GET 2022-09-03T05:30:11.166 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.167 DEBUG:boto:Data: 2022-09-03T05:30:11.168 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.169 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:11.170 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.171 DEBUG:boto:Params: {} 2022-09-03T05:30:11.172 DEBUG:boto:Token: None 2022-09-03T05:30:11.173 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.174 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uEAAq1UEMmEYQWdkKR9KXpiOWu0= 2022-09-03T05:30:11.175 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uEAAq1UEMmEYQWdkKR9KXpiOWu0=', 'Content-Length': '0'} 2022-09-03T05:30:11.178 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000102e54a062b2ea24-006312e663-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.179 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.180 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:11.181 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.182 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.183 DEBUG:boto:Method: GET 2022-09-03T05:30:11.185 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.186 DEBUG:boto:Data: 2022-09-03T05:30:11.186 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.187 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.188 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.188 DEBUG:boto:Params: {} 2022-09-03T05:30:11.189 DEBUG:boto:Token: None 2022-09-03T05:30:11.190 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:11.190 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uEAAq1UEMmEYQWdkKR9KXpiOWu0= 2022-09-03T05:30:11.191 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uEAAq1UEMmEYQWdkKR9KXpiOWu0=', 'Content-Length': '0'} 2022-09-03T05:30:11.193 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001253f6c6438d3d54-006312e663-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.234 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:11.235 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:11.235 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:11.236 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:11.237 DEBUG:boto:Method: GET 2022-09-03T05:30:11.237 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:11.238 DEBUG:boto:Data: 2022-09-03T05:30:11.239 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.240 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:11.241 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.242 DEBUG:boto:Params: {} 2022-09-03T05:30:11.243 DEBUG:boto:Token: None 2022-09-03T05:30:11.244 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/ 2022-09-03T05:30:11.245 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw= 2022-09-03T05:30:11.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw=', 'Content-Length': '0'} 2022-09-03T05:30:11.249 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007951ba696405a3ae-006312e663-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.250 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:11.251 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:11.251 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:11.252 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:11.252 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:11.253 DEBUG:boto:Method: GET 2022-09-03T05:30:11.254 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:11.255 DEBUG:boto:Data: 2022-09-03T05:30:11.255 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.256 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.257 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.257 DEBUG:boto:Params: {} 2022-09-03T05:30:11.258 DEBUG:boto:Token: None 2022-09-03T05:30:11.259 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/ 2022-09-03T05:30:11.259 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw= 2022-09-03T05:30:11.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw=', 'Content-Length': '0'} 2022-09-03T05:30:11.273 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000036c6b6ea050b591f-006312e663-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.274 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:11.276 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:30:11.277 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.277 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.278 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.279 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.280 DEBUG:boto:Method: GET 2022-09-03T05:30:11.281 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.282 DEBUG:boto:Data: 2022-09-03T05:30:11.283 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.283 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:11.284 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.284 DEBUG:boto:Params: {} 2022-09-03T05:30:11.285 DEBUG:boto:Token: None 2022-09-03T05:30:11.286 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.287 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jgp90S0xq7yzQYa0v9eTtw3n9wk= 2022-09-03T05:30:11.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jgp90S0xq7yzQYa0v9eTtw3n9wk=', 'Content-Length': '0'} 2022-09-03T05:30:11.290 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000071a6e9db43eca8c5-006312e663-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.291 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.292 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.293 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.294 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.295 DEBUG:boto:Method: GET 2022-09-03T05:30:11.296 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.296 DEBUG:boto:Data: 2022-09-03T05:30:11.297 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.298 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.299 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.300 DEBUG:boto:Params: {} 2022-09-03T05:30:11.301 DEBUG:boto:Token: None 2022-09-03T05:30:11.302 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.303 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jgp90S0xq7yzQYa0v9eTtw3n9wk= 2022-09-03T05:30:11.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jgp90S0xq7yzQYa0v9eTtw3n9wk=', 'Content-Length': '0'} 2022-09-03T05:30:11.307 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000570bfa124a6f20bb-006312e663-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.309 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.310 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.311 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.312 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.313 DEBUG:boto:Method: HEAD 2022-09-03T05:30:11.314 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.315 DEBUG:boto:Data: 2022-09-03T05:30:11.316 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.317 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:11.317 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.318 DEBUG:boto:Params: {} 2022-09-03T05:30:11.318 DEBUG:boto:Token: None 2022-09-03T05:30:11.319 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.319 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OTRby3+C542kBrvnCIh2QvOYGv4= 2022-09-03T05:30:11.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OTRby3+C542kBrvnCIh2QvOYGv4=', 'Content-Length': '0'} 2022-09-03T05:30:11.322 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d62f570f233f4385-006312e663-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.323 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.323 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.324 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.324 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.325 DEBUG:boto:Method: HEAD 2022-09-03T05:30:11.325 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.326 DEBUG:boto:Data: 2022-09-03T05:30:11.326 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.327 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.328 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.328 DEBUG:boto:Params: {} 2022-09-03T05:30:11.329 DEBUG:boto:Token: None 2022-09-03T05:30:11.329 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.330 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OTRby3+C542kBrvnCIh2QvOYGv4= 2022-09-03T05:30:11.331 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OTRby3+C542kBrvnCIh2QvOYGv4=', 'Content-Length': '0'} 2022-09-03T05:30:11.332 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008b9548b4bd5dbc64-006312e663-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.333 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:30:11.334 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.334 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.335 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.335 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.336 DEBUG:boto:Method: GET 2022-09-03T05:30:11.337 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.337 DEBUG:boto:Data: 2022-09-03T05:30:11.338 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.338 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:11.339 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.339 DEBUG:boto:Params: {} 2022-09-03T05:30:11.340 DEBUG:boto:Token: None 2022-09-03T05:30:11.341 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.341 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jgp90S0xq7yzQYa0v9eTtw3n9wk= 2022-09-03T05:30:11.342 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jgp90S0xq7yzQYa0v9eTtw3n9wk=', 'Content-Length': '0'} 2022-09-03T05:30:11.344 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ea1f88694feeeba2-006312e663-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.386 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.387 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.388 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.389 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.390 DEBUG:boto:Method: GET 2022-09-03T05:30:11.391 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.392 DEBUG:boto:Data: 2022-09-03T05:30:11.392 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.393 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.393 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.394 DEBUG:boto:Params: {} 2022-09-03T05:30:11.395 DEBUG:boto:Token: None 2022-09-03T05:30:11.395 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.396 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jgp90S0xq7yzQYa0v9eTtw3n9wk= 2022-09-03T05:30:11.397 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jgp90S0xq7yzQYa0v9eTtw3n9wk=', 'Content-Length': '0'} 2022-09-03T05:30:11.399 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000051a5658cbcc7dc1a-006312e663-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.401 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:11.402 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:11.403 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:11.404 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:11.405 DEBUG:boto:Method: GET 2022-09-03T05:30:11.405 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:11.406 DEBUG:boto:Data: 2022-09-03T05:30:11.407 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.408 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:11.409 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.410 DEBUG:boto:Params: {} 2022-09-03T05:30:11.410 DEBUG:boto:Token: None 2022-09-03T05:30:11.411 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/ 2022-09-03T05:30:11.411 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw= 2022-09-03T05:30:11.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw=', 'Content-Length': '0'} 2022-09-03T05:30:11.415 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ba72aadb92968262-006312e663-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.416 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:11.417 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:11.418 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:11.419 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:11.419 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:11.420 DEBUG:boto:Method: GET 2022-09-03T05:30:11.420 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:11.421 DEBUG:boto:Data: 2022-09-03T05:30:11.421 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.422 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.422 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.423 DEBUG:boto:Params: {} 2022-09-03T05:30:11.424 DEBUG:boto:Token: None 2022-09-03T05:30:11.424 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/ 2022-09-03T05:30:11.425 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw= 2022-09-03T05:30:11.426 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw=', 'Content-Length': '0'} 2022-09-03T05:30:11.429 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cc98fc9fed82489a-006312e663-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.430 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:11.431 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.432 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.433 DEBUG:boto:Method: HEAD 2022-09-03T05:30:11.434 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.435 DEBUG:boto:Data: 2022-09-03T05:30:11.436 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.436 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:11.437 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.438 DEBUG:boto:Params: {} 2022-09-03T05:30:11.439 DEBUG:boto:Token: None 2022-09-03T05:30:11.440 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.441 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8ogYHkriY3D33Xasp+ISynqikEY= 2022-09-03T05:30:11.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8ogYHkriY3D33Xasp+ISynqikEY=', 'Content-Length': '0'} 2022-09-03T05:30:11.448 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000072a585d55756ae0e-006312e663-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.449 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.450 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.450 DEBUG:boto:Method: HEAD 2022-09-03T05:30:11.451 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.452 DEBUG:boto:Data: 2022-09-03T05:30:11.454 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.454 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.455 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.456 DEBUG:boto:Params: {} 2022-09-03T05:30:11.458 DEBUG:boto:Token: None 2022-09-03T05:30:11.459 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.460 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8ogYHkriY3D33Xasp+ISynqikEY= 2022-09-03T05:30:11.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8ogYHkriY3D33Xasp+ISynqikEY=', 'Content-Length': '0'} 2022-09-03T05:30:11.464 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e086eee6f344ba91-006312e663-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.465 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:30:11.466 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.466 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.467 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.467 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.468 DEBUG:boto:Method: GET 2022-09-03T05:30:11.469 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.469 DEBUG:boto:Data: 2022-09-03T05:30:11.470 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.470 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:11.471 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.471 DEBUG:boto:Params: {} 2022-09-03T05:30:11.472 DEBUG:boto:Token: None 2022-09-03T05:30:11.473 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.473 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jgp90S0xq7yzQYa0v9eTtw3n9wk= 2022-09-03T05:30:11.474 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jgp90S0xq7yzQYa0v9eTtw3n9wk=', 'Content-Length': '0'} 2022-09-03T05:30:11.476 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b246dee04c8c1273-006312e663-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.518 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.519 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:11.520 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.521 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.522 DEBUG:boto:Method: GET 2022-09-03T05:30:11.522 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.523 DEBUG:boto:Data: 2022-09-03T05:30:11.523 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.524 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.524 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.525 DEBUG:boto:Params: {} 2022-09-03T05:30:11.526 DEBUG:boto:Token: None 2022-09-03T05:30:11.526 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:11.527 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jgp90S0xq7yzQYa0v9eTtw3n9wk= 2022-09-03T05:30:11.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jgp90S0xq7yzQYa0v9eTtw3n9wk=', 'Content-Length': '0'} 2022-09-03T05:30:11.530 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d32df1020d9f19ae-006312e663-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.531 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:11.532 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:11.532 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:11.533 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:11.533 DEBUG:boto:Method: GET 2022-09-03T05:30:11.534 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:11.535 DEBUG:boto:Data: 2022-09-03T05:30:11.535 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.536 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:11.537 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.537 DEBUG:boto:Params: {} 2022-09-03T05:30:11.538 DEBUG:boto:Token: None 2022-09-03T05:30:11.538 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/ 2022-09-03T05:30:11.539 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw= 2022-09-03T05:30:11.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw=', 'Content-Length': '0'} 2022-09-03T05:30:11.543 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ea09a9b318a568f0-006312e663-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.544 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:11.546 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:11.547 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:11.548 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:11.549 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:11.550 DEBUG:boto:Method: GET 2022-09-03T05:30:11.550 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:11.551 DEBUG:boto:Data: 2022-09-03T05:30:11.552 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.552 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.553 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.554 DEBUG:boto:Params: {} 2022-09-03T05:30:11.554 DEBUG:boto:Token: None 2022-09-03T05:30:11.555 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/ 2022-09-03T05:30:11.556 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw= 2022-09-03T05:30:11.556 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw=', 'Content-Length': '0'} 2022-09-03T05:30:11.559 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000caefe5d86bf99abb-006312e663-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.560 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:11.561 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:30:11.562 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.563 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.564 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.564 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.565 DEBUG:boto:Method: GET 2022-09-03T05:30:11.566 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.567 DEBUG:boto:Data: 2022-09-03T05:30:11.568 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.570 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:11.571 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.572 DEBUG:boto:Params: {} 2022-09-03T05:30:11.573 DEBUG:boto:Token: None 2022-09-03T05:30:11.574 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.575 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:09uVUH8JTrLXFs34KjQ/1vHsbSM= 2022-09-03T05:30:11.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:09uVUH8JTrLXFs34KjQ/1vHsbSM=', 'Content-Length': '0'} 2022-09-03T05:30:11.580 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004811214e18ffdb10-006312e663-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.581 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.582 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.583 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.584 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.585 DEBUG:boto:Method: GET 2022-09-03T05:30:11.586 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.587 DEBUG:boto:Data: 2022-09-03T05:30:11.588 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.589 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.590 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.591 DEBUG:boto:Params: {} 2022-09-03T05:30:11.591 DEBUG:boto:Token: None 2022-09-03T05:30:11.592 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.593 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:09uVUH8JTrLXFs34KjQ/1vHsbSM= 2022-09-03T05:30:11.593 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:09uVUH8JTrLXFs34KjQ/1vHsbSM=', 'Content-Length': '0'} 2022-09-03T05:30:11.596 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bcdbbc4d08e2a42a-006312e663-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.597 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.598 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.599 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.600 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.601 DEBUG:boto:Method: HEAD 2022-09-03T05:30:11.602 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.603 DEBUG:boto:Data: 2022-09-03T05:30:11.604 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.605 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:11.605 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.606 DEBUG:boto:Params: {} 2022-09-03T05:30:11.606 DEBUG:boto:Token: None 2022-09-03T05:30:11.607 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.608 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tjovzBKDZJfht2yS6lsuEZU47i8= 2022-09-03T05:30:11.608 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tjovzBKDZJfht2yS6lsuEZU47i8=', 'Content-Length': '0'} 2022-09-03T05:30:11.610 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004fc0741f090ec65c-006312e663-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.611 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.612 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.612 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.613 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.613 DEBUG:boto:Method: HEAD 2022-09-03T05:30:11.614 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.614 DEBUG:boto:Data: 2022-09-03T05:30:11.615 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.616 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.616 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.617 DEBUG:boto:Params: {} 2022-09-03T05:30:11.617 DEBUG:boto:Token: None 2022-09-03T05:30:11.618 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.619 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tjovzBKDZJfht2yS6lsuEZU47i8= 2022-09-03T05:30:11.619 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tjovzBKDZJfht2yS6lsuEZU47i8=', 'Content-Length': '0'} 2022-09-03T05:30:11.621 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dd3862c54cdb10fe-006312e663-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.622 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:30:11.623 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.623 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.624 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.625 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.625 DEBUG:boto:Method: GET 2022-09-03T05:30:11.626 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.626 DEBUG:boto:Data: 2022-09-03T05:30:11.627 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.627 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:11.628 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.628 DEBUG:boto:Params: {} 2022-09-03T05:30:11.629 DEBUG:boto:Token: None 2022-09-03T05:30:11.630 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.630 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:09uVUH8JTrLXFs34KjQ/1vHsbSM= 2022-09-03T05:30:11.631 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:09uVUH8JTrLXFs34KjQ/1vHsbSM=', 'Content-Length': '0'} 2022-09-03T05:30:11.633 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000015128ce695770564-006312e663-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.678 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.679 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.680 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.680 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.681 DEBUG:boto:Method: GET 2022-09-03T05:30:11.681 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.682 DEBUG:boto:Data: 2022-09-03T05:30:11.683 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.683 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.684 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.684 DEBUG:boto:Params: {} 2022-09-03T05:30:11.685 DEBUG:boto:Token: None 2022-09-03T05:30:11.685 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.686 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:09uVUH8JTrLXFs34KjQ/1vHsbSM= 2022-09-03T05:30:11.687 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:09uVUH8JTrLXFs34KjQ/1vHsbSM=', 'Content-Length': '0'} 2022-09-03T05:30:11.689 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006cf29ca9a44dbd7d-006312e663-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.690 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:11.690 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:11.691 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:11.691 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:11.692 DEBUG:boto:Method: GET 2022-09-03T05:30:11.692 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:11.693 DEBUG:boto:Data: 2022-09-03T05:30:11.693 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.694 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:11.694 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.695 DEBUG:boto:Params: {} 2022-09-03T05:30:11.696 DEBUG:boto:Token: None 2022-09-03T05:30:11.696 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/ 2022-09-03T05:30:11.697 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw= 2022-09-03T05:30:11.698 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw=', 'Content-Length': '0'} 2022-09-03T05:30:11.700 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c730fc4e2548355a-006312e663-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.701 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:11.702 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:11.703 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:11.703 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:11.704 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:11.704 DEBUG:boto:Method: GET 2022-09-03T05:30:11.705 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:11.705 DEBUG:boto:Data: 2022-09-03T05:30:11.706 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.707 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.707 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.708 DEBUG:boto:Params: {} 2022-09-03T05:30:11.708 DEBUG:boto:Token: None 2022-09-03T05:30:11.709 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/ 2022-09-03T05:30:11.709 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw= 2022-09-03T05:30:11.710 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw=', 'Content-Length': '0'} 2022-09-03T05:30:11.712 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007ace22747b395962-006312e663-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.713 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:11.714 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.714 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.715 DEBUG:boto:Method: HEAD 2022-09-03T05:30:11.715 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.716 DEBUG:boto:Data: 2022-09-03T05:30:11.716 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.717 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:11.717 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.718 DEBUG:boto:Params: {} 2022-09-03T05:30:11.719 DEBUG:boto:Token: None 2022-09-03T05:30:11.719 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.720 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yCAv0qA2JNku536ZVVhynaCWqqc= 2022-09-03T05:30:11.721 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yCAv0qA2JNku536ZVVhynaCWqqc=', 'Content-Length': '0'} 2022-09-03T05:30:11.723 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b7f64bef36104b9e-006312e663-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.724 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.725 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.725 DEBUG:boto:Method: HEAD 2022-09-03T05:30:11.726 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.726 DEBUG:boto:Data: 2022-09-03T05:30:11.727 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.727 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.728 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.728 DEBUG:boto:Params: {} 2022-09-03T05:30:11.729 DEBUG:boto:Token: None 2022-09-03T05:30:11.729 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.730 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yCAv0qA2JNku536ZVVhynaCWqqc= 2022-09-03T05:30:11.730 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yCAv0qA2JNku536ZVVhynaCWqqc=', 'Content-Length': '0'} 2022-09-03T05:30:11.733 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c4e1752ac823f838-006312e663-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.734 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:30:11.734 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.735 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.735 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.736 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.736 DEBUG:boto:Method: GET 2022-09-03T05:30:11.737 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.737 DEBUG:boto:Data: 2022-09-03T05:30:11.738 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.738 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:11.739 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.740 DEBUG:boto:Params: {} 2022-09-03T05:30:11.740 DEBUG:boto:Token: None 2022-09-03T05:30:11.741 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.742 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:09uVUH8JTrLXFs34KjQ/1vHsbSM= 2022-09-03T05:30:11.742 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:09uVUH8JTrLXFs34KjQ/1vHsbSM=', 'Content-Length': '0'} 2022-09-03T05:30:11.745 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e972a96e27dc0563-006312e663-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.746 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.746 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:11.747 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.748 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.749 DEBUG:boto:Method: GET 2022-09-03T05:30:11.750 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.751 DEBUG:boto:Data: 2022-09-03T05:30:11.752 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.753 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.754 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.755 DEBUG:boto:Params: {} 2022-09-03T05:30:11.757 DEBUG:boto:Token: None 2022-09-03T05:30:11.757 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:11.758 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:09uVUH8JTrLXFs34KjQ/1vHsbSM= 2022-09-03T05:30:11.759 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:09uVUH8JTrLXFs34KjQ/1vHsbSM=', 'Content-Length': '0'} 2022-09-03T05:30:11.761 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b00fe29f0591a0be-006312e663-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.762 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:11.763 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:11.764 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:11.765 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:11.765 DEBUG:boto:Method: GET 2022-09-03T05:30:11.766 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:11.767 DEBUG:boto:Data: 2022-09-03T05:30:11.768 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.768 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:11.769 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.770 DEBUG:boto:Params: {} 2022-09-03T05:30:11.770 DEBUG:boto:Token: None 2022-09-03T05:30:11.771 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/ 2022-09-03T05:30:11.772 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw= 2022-09-03T05:30:11.773 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw=', 'Content-Length': '0'} 2022-09-03T05:30:11.776 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000041f117486e9bdd0b-006312e663-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.777 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:11.777 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:11.778 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:11.779 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:11.779 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:11.780 DEBUG:boto:Method: GET 2022-09-03T05:30:11.780 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:11.781 DEBUG:boto:Data: 2022-09-03T05:30:11.781 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.782 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.782 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.783 DEBUG:boto:Params: {} 2022-09-03T05:30:11.784 DEBUG:boto:Token: None 2022-09-03T05:30:11.784 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/ 2022-09-03T05:30:11.785 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw= 2022-09-03T05:30:11.785 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw=', 'Content-Length': '0'} 2022-09-03T05:30:11.788 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001be9bd052d4ad25f-006312e663-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.789 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:11.790 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-17 zones={test-zone2, test-zone1} 2022-09-03T05:30:11.791 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-17 zones={test-zone3, test-zone1} 2022-09-03T05:30:11.791 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:11.792 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:11.793 DEBUG:boto:Method: HEAD 2022-09-03T05:30:11.793 DEBUG:boto:Path: /dsgvcy-17/ 2022-09-03T05:30:11.794 DEBUG:boto:Data: 2022-09-03T05:30:11.795 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.796 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:11.797 DEBUG:boto:Port: 8001 2022-09-03T05:30:11.798 DEBUG:boto:Params: {} 2022-09-03T05:30:11.799 DEBUG:boto:Token: None 2022-09-03T05:30:11.800 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/ 2022-09-03T05:30:11.801 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K7AFnNgzK1++M7k+JL4sY7oUXZQ= 2022-09-03T05:30:11.802 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K7AFnNgzK1++M7k+JL4sY7oUXZQ=', 'Content-Length': '0'} 2022-09-03T05:30:11.805 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000659338c2329b5e82-006312e663-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.806 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:11.806 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:11.807 DEBUG:boto:Method: HEAD 2022-09-03T05:30:11.807 DEBUG:boto:Path: /dsgvcy-17/ 2022-09-03T05:30:11.808 DEBUG:boto:Data: 2022-09-03T05:30:11.809 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.809 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.810 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.810 DEBUG:boto:Params: {} 2022-09-03T05:30:11.811 DEBUG:boto:Token: None 2022-09-03T05:30:11.811 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/ 2022-09-03T05:30:11.812 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K7AFnNgzK1++M7k+JL4sY7oUXZQ= 2022-09-03T05:30:11.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K7AFnNgzK1++M7k+JL4sY7oUXZQ=', 'Content-Length': '0'} 2022-09-03T05:30:11.815 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000070300b8383c959d4-006312e663-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.815 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:30:11.816 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:11.817 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:11.817 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:11.818 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:11.818 DEBUG:boto:Method: GET 2022-09-03T05:30:11.819 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:11.819 DEBUG:boto:Data: 2022-09-03T05:30:11.820 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.820 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:11.821 DEBUG:boto:Port: 8001 2022-09-03T05:30:11.821 DEBUG:boto:Params: {} 2022-09-03T05:30:11.822 DEBUG:boto:Token: None 2022-09-03T05:30:11.822 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/?versions 2022-09-03T05:30:11.823 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v9hziwvL3l0SCOADvpaiEZOe2x4= 2022-09-03T05:30:11.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v9hziwvL3l0SCOADvpaiEZOe2x4=', 'Content-Length': '0'} 2022-09-03T05:30:11.826 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000060b9dfc8133fb345-006312e663-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.827 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:11.828 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:30:11.829 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:30:11.829 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:30:11.830 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:30:11.830 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:30:11.831 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:11.831 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:11.832 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:11.832 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:11.833 DEBUG:boto:Method: GET 2022-09-03T05:30:11.833 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:11.834 DEBUG:boto:Data: 2022-09-03T05:30:11.834 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.835 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.835 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.836 DEBUG:boto:Params: {} 2022-09-03T05:30:11.836 DEBUG:boto:Token: None 2022-09-03T05:30:11.837 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/?versions 2022-09-03T05:30:11.838 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v9hziwvL3l0SCOADvpaiEZOe2x4= 2022-09-03T05:30:11.838 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v9hziwvL3l0SCOADvpaiEZOe2x4=', 'Content-Length': '0'} 2022-09-03T05:30:11.841 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bd24973965f8f252-006312e663-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.841 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:11.842 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:30:11.843 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:30:11.843 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:30:11.844 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:30:11.844 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:11.845 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:11.845 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:11.846 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:11.846 DEBUG:boto:Method: GET 2022-09-03T05:30:11.847 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:11.847 DEBUG:boto:Data: 2022-09-03T05:30:11.848 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.849 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:11.849 DEBUG:boto:Port: 8001 2022-09-03T05:30:11.850 DEBUG:boto:Params: {} 2022-09-03T05:30:11.851 DEBUG:boto:Token: None 2022-09-03T05:30:11.851 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/?versions 2022-09-03T05:30:11.852 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v9hziwvL3l0SCOADvpaiEZOe2x4= 2022-09-03T05:30:11.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v9hziwvL3l0SCOADvpaiEZOe2x4=', 'Content-Length': '0'} 2022-09-03T05:30:11.855 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bddf19c90cb260b2-006312e663-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.857 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:11.858 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:11.859 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:11.860 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:11.861 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:11.863 DEBUG:boto:Method: GET 2022-09-03T05:30:11.864 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:11.865 DEBUG:boto:Data: 2022-09-03T05:30:11.866 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.866 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.868 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.869 DEBUG:boto:Params: {} 2022-09-03T05:30:11.870 DEBUG:boto:Token: None 2022-09-03T05:30:11.871 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/?versions 2022-09-03T05:30:11.872 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v9hziwvL3l0SCOADvpaiEZOe2x4= 2022-09-03T05:30:11.873 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v9hziwvL3l0SCOADvpaiEZOe2x4=', 'Content-Length': '0'} 2022-09-03T05:30:11.876 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000081ca7b102183134c-006312e663-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.879 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:11.880 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:30:11.881 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:11.882 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:11.882 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.883 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.884 DEBUG:boto:Method: GET 2022-09-03T05:30:11.884 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.885 DEBUG:boto:Data: 2022-09-03T05:30:11.885 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.886 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:11.887 DEBUG:boto:Port: 8001 2022-09-03T05:30:11.887 DEBUG:boto:Params: {} 2022-09-03T05:30:11.888 DEBUG:boto:Token: None 2022-09-03T05:30:11.888 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.889 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/fPit03R0cTztQXGXnUYl0hEopU= 2022-09-03T05:30:11.890 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/fPit03R0cTztQXGXnUYl0hEopU=', 'Content-Length': '0'} 2022-09-03T05:30:11.892 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000c0e0cc9dc9b3145b-006312e663-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.893 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:11.894 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:11.894 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.895 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.895 DEBUG:boto:Method: GET 2022-09-03T05:30:11.896 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.896 DEBUG:boto:Data: 2022-09-03T05:30:11.897 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.897 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.898 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.898 DEBUG:boto:Params: {} 2022-09-03T05:30:11.899 DEBUG:boto:Token: None 2022-09-03T05:30:11.900 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.900 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/fPit03R0cTztQXGXnUYl0hEopU= 2022-09-03T05:30:11.901 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/fPit03R0cTztQXGXnUYl0hEopU=', 'Content-Length': '0'} 2022-09-03T05:30:11.903 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000958fff996b6dd3d3-006312e663-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.904 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:11.904 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:11.905 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.905 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.906 DEBUG:boto:Method: HEAD 2022-09-03T05:30:11.906 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.907 DEBUG:boto:Data: 2022-09-03T05:30:11.907 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.908 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:11.908 DEBUG:boto:Port: 8001 2022-09-03T05:30:11.909 DEBUG:boto:Params: {} 2022-09-03T05:30:11.909 DEBUG:boto:Token: None 2022-09-03T05:30:11.910 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.911 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JxAqeqRM6e0xa/f0lFg6lBkDfYA= 2022-09-03T05:30:11.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JxAqeqRM6e0xa/f0lFg6lBkDfYA=', 'Content-Length': '0'} 2022-09-03T05:30:11.913 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000004badb410c35bd27d-006312e663-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.914 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:11.915 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:11.916 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.917 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.918 DEBUG:boto:Method: HEAD 2022-09-03T05:30:11.919 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.920 DEBUG:boto:Data: 2022-09-03T05:30:11.921 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.922 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.923 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.923 DEBUG:boto:Params: {} 2022-09-03T05:30:11.925 DEBUG:boto:Token: None 2022-09-03T05:30:11.926 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.927 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JxAqeqRM6e0xa/f0lFg6lBkDfYA= 2022-09-03T05:30:11.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JxAqeqRM6e0xa/f0lFg6lBkDfYA=', 'Content-Length': '0'} 2022-09-03T05:30:11.930 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000dea2cf5c2b2fbb18-006312e663-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.931 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:30:11.932 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:11.932 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:11.933 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.933 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.934 DEBUG:boto:Method: GET 2022-09-03T05:30:11.935 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.935 DEBUG:boto:Data: 2022-09-03T05:30:11.936 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.936 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:11.937 DEBUG:boto:Port: 8001 2022-09-03T05:30:11.937 DEBUG:boto:Params: {} 2022-09-03T05:30:11.938 DEBUG:boto:Token: None 2022-09-03T05:30:11.939 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.939 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/fPit03R0cTztQXGXnUYl0hEopU= 2022-09-03T05:30:11.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/fPit03R0cTztQXGXnUYl0hEopU=', 'Content-Length': '0'} 2022-09-03T05:30:11.942 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000d960aefacf807504-006312e663-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.943 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:11.944 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:11.944 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.945 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.945 DEBUG:boto:Method: GET 2022-09-03T05:30:11.946 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.946 DEBUG:boto:Data: 2022-09-03T05:30:11.947 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.947 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:11.948 DEBUG:boto:Port: 8000 2022-09-03T05:30:11.948 DEBUG:boto:Params: {} 2022-09-03T05:30:11.949 DEBUG:boto:Token: None 2022-09-03T05:30:11.950 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:11.950 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/fPit03R0cTztQXGXnUYl0hEopU= 2022-09-03T05:30:11.951 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/fPit03R0cTztQXGXnUYl0hEopU=', 'Content-Length': '0'} 2022-09-03T05:30:11.953 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000005f4de46168635109-006312e663-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:11.954 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:11.955 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:11.955 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:11.956 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:11.956 DEBUG:boto:Method: GET 2022-09-03T05:30:11.957 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:30:11.957 DEBUG:boto:Data: 2022-09-03T05:30:11.958 DEBUG:boto:Headers: {} 2022-09-03T05:30:11.958 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:11.959 DEBUG:boto:Port: 8001 2022-09-03T05:30:11.959 DEBUG:boto:Params: {} 2022-09-03T05:30:11.960 DEBUG:boto:Token: None 2022-09-03T05:30:11.961 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:11 GMT /dsgvcy-17/ 2022-09-03T05:30:11.961 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw= 2022-09-03T05:30:11.962 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eI1P6+LTp+M6yG18nx3jv33tOJw=', 'Content-Length': '0'} 2022-09-03T05:30:11.964 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000474f6f6942fa3f17-006312e663-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.006 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:12.008 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:12.009 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:12.010 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:12.011 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:12.012 DEBUG:boto:Method: GET 2022-09-03T05:30:12.013 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:30:12.014 DEBUG:boto:Data: 2022-09-03T05:30:12.015 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.016 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.016 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.017 DEBUG:boto:Params: {} 2022-09-03T05:30:12.018 DEBUG:boto:Token: None 2022-09-03T05:30:12.019 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/ 2022-09-03T05:30:12.020 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE= 2022-09-03T05:30:12.021 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE=', 'Content-Length': '0'} 2022-09-03T05:30:12.024 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d0e48b89558c477-006312e664-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.025 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:12.026 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:12.026 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:12.027 DEBUG:boto:Method: HEAD 2022-09-03T05:30:12.027 DEBUG:boto:Path: /dsgvcy-17/dummy 2022-09-03T05:30:12.028 DEBUG:boto:Data: 2022-09-03T05:30:12.028 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.029 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.029 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.030 DEBUG:boto:Params: {} 2022-09-03T05:30:12.030 DEBUG:boto:Token: None 2022-09-03T05:30:12.031 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/dummy 2022-09-03T05:30:12.031 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ULvS4iHVr1dfdPJq80pg+hlpOL0= 2022-09-03T05:30:12.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ULvS4iHVr1dfdPJq80pg+hlpOL0=', 'Content-Length': '0'} 2022-09-03T05:30:12.034 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000051b267eee8c2b27b-006312e664-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.035 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:12.036 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:12.037 DEBUG:boto:Method: HEAD 2022-09-03T05:30:12.037 DEBUG:boto:Path: /dsgvcy-17/dummy 2022-09-03T05:30:12.038 DEBUG:boto:Data: 2022-09-03T05:30:12.038 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.039 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.039 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.040 DEBUG:boto:Params: {} 2022-09-03T05:30:12.040 DEBUG:boto:Token: None 2022-09-03T05:30:12.041 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/dummy 2022-09-03T05:30:12.041 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ULvS4iHVr1dfdPJq80pg+hlpOL0= 2022-09-03T05:30:12.042 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ULvS4iHVr1dfdPJq80pg+hlpOL0=', 'Content-Length': '0'} 2022-09-03T05:30:12.044 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000933fd53f3a958570-006312e664-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.045 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:30:12.046 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:12.047 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:12.047 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.048 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.049 DEBUG:boto:Method: GET 2022-09-03T05:30:12.049 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.050 DEBUG:boto:Data: 2022-09-03T05:30:12.050 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.051 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.051 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.052 DEBUG:boto:Params: {} 2022-09-03T05:30:12.052 DEBUG:boto:Token: None 2022-09-03T05:30:12.053 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.054 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8m6NMRdP9Uhem8cmc8cJJQwJCDU= 2022-09-03T05:30:12.055 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8m6NMRdP9Uhem8cmc8cJJQwJCDU=', 'Content-Length': '0'} 2022-09-03T05:30:12.057 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000020ab6c8c2a19a946-006312e664-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.058 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:12.058 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:12.059 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.059 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.060 DEBUG:boto:Method: GET 2022-09-03T05:30:12.060 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.061 DEBUG:boto:Data: 2022-09-03T05:30:12.061 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.062 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.062 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.063 DEBUG:boto:Params: {} 2022-09-03T05:30:12.064 DEBUG:boto:Token: None 2022-09-03T05:30:12.064 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.065 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8m6NMRdP9Uhem8cmc8cJJQwJCDU= 2022-09-03T05:30:12.066 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8m6NMRdP9Uhem8cmc8cJJQwJCDU=', 'Content-Length': '0'} 2022-09-03T05:30:12.068 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a164cb393b239c36-006312e664-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.069 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:12.069 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:12.070 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:12.070 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:12.071 DEBUG:boto:Method: GET 2022-09-03T05:30:12.071 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:30:12.072 DEBUG:boto:Data: 2022-09-03T05:30:12.072 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.073 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.073 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.074 DEBUG:boto:Params: {} 2022-09-03T05:30:12.075 DEBUG:boto:Token: None 2022-09-03T05:30:12.075 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/ 2022-09-03T05:30:12.076 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE= 2022-09-03T05:30:12.077 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE=', 'Content-Length': '0'} 2022-09-03T05:30:12.080 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000250d59278a7ba538-006312e664-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.080 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:12.081 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:12.082 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:12.082 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:12.083 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:12.083 DEBUG:boto:Method: GET 2022-09-03T05:30:12.084 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:30:12.084 DEBUG:boto:Data: 2022-09-03T05:30:12.085 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.085 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.086 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.087 DEBUG:boto:Params: {} 2022-09-03T05:30:12.087 DEBUG:boto:Token: None 2022-09-03T05:30:12.088 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/ 2022-09-03T05:30:12.089 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE= 2022-09-03T05:30:12.089 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE=', 'Content-Length': '0'} 2022-09-03T05:30:12.092 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a2f6ce82e7072d91-006312e664-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.093 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:12.093 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:30:12.094 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.095 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.095 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.096 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.096 DEBUG:boto:Method: GET 2022-09-03T05:30:12.097 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.097 DEBUG:boto:Data: 2022-09-03T05:30:12.098 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.098 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.099 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.099 DEBUG:boto:Params: {} 2022-09-03T05:30:12.100 DEBUG:boto:Token: None 2022-09-03T05:30:12.100 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.101 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yb12P3SsglRc0E+FdWP/1pnu0ck= 2022-09-03T05:30:12.102 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yb12P3SsglRc0E+FdWP/1pnu0ck=', 'Content-Length': '0'} 2022-09-03T05:30:12.104 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c5d3fa6c286f02a0-006312e664-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.104 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.105 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.105 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.106 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.106 DEBUG:boto:Method: GET 2022-09-03T05:30:12.107 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.108 DEBUG:boto:Data: 2022-09-03T05:30:12.108 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.109 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.109 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.110 DEBUG:boto:Params: {} 2022-09-03T05:30:12.110 DEBUG:boto:Token: None 2022-09-03T05:30:12.111 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.112 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yb12P3SsglRc0E+FdWP/1pnu0ck= 2022-09-03T05:30:12.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yb12P3SsglRc0E+FdWP/1pnu0ck=', 'Content-Length': '0'} 2022-09-03T05:30:12.114 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ce5c97aa0863394d-006312e664-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.115 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.116 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.116 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.117 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.117 DEBUG:boto:Method: HEAD 2022-09-03T05:30:12.118 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.118 DEBUG:boto:Data: 2022-09-03T05:30:12.119 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.119 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.120 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.120 DEBUG:boto:Params: {} 2022-09-03T05:30:12.121 DEBUG:boto:Token: None 2022-09-03T05:30:12.121 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.122 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TQJWD2kR+u80t82C6gEknzZfzy0= 2022-09-03T05:30:12.122 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TQJWD2kR+u80t82C6gEknzZfzy0=', 'Content-Length': '0'} 2022-09-03T05:30:12.124 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e3db38228b1b9d54-006312e664-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.125 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.126 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.126 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.127 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.127 DEBUG:boto:Method: HEAD 2022-09-03T05:30:12.128 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.128 DEBUG:boto:Data: 2022-09-03T05:30:12.129 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.129 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.130 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.130 DEBUG:boto:Params: {} 2022-09-03T05:30:12.131 DEBUG:boto:Token: None 2022-09-03T05:30:12.132 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.132 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TQJWD2kR+u80t82C6gEknzZfzy0= 2022-09-03T05:30:12.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TQJWD2kR+u80t82C6gEknzZfzy0=', 'Content-Length': '0'} 2022-09-03T05:30:12.135 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000567eed98744716e6-006312e664-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.136 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:30:12.136 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.137 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.137 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.138 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.138 DEBUG:boto:Method: GET 2022-09-03T05:30:12.139 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.139 DEBUG:boto:Data: 2022-09-03T05:30:12.140 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.140 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.141 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.141 DEBUG:boto:Params: {} 2022-09-03T05:30:12.142 DEBUG:boto:Token: None 2022-09-03T05:30:12.143 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.143 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yb12P3SsglRc0E+FdWP/1pnu0ck= 2022-09-03T05:30:12.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yb12P3SsglRc0E+FdWP/1pnu0ck=', 'Content-Length': '0'} 2022-09-03T05:30:12.146 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dfc02077fe2a40af-006312e664-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.147 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.148 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.148 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.149 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.150 DEBUG:boto:Method: GET 2022-09-03T05:30:12.151 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.152 DEBUG:boto:Data: 2022-09-03T05:30:12.152 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.153 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.153 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.154 DEBUG:boto:Params: {} 2022-09-03T05:30:12.155 DEBUG:boto:Token: None 2022-09-03T05:30:12.155 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.156 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yb12P3SsglRc0E+FdWP/1pnu0ck= 2022-09-03T05:30:12.157 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yb12P3SsglRc0E+FdWP/1pnu0ck=', 'Content-Length': '0'} 2022-09-03T05:30:12.159 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000045edfc64fef475d-006312e664-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.160 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:12.161 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:12.162 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:12.162 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:12.163 DEBUG:boto:Method: GET 2022-09-03T05:30:12.164 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:12.165 DEBUG:boto:Data: 2022-09-03T05:30:12.166 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.167 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.168 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.168 DEBUG:boto:Params: {} 2022-09-03T05:30:12.169 DEBUG:boto:Token: None 2022-09-03T05:30:12.170 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/ 2022-09-03T05:30:12.171 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE= 2022-09-03T05:30:12.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE=', 'Content-Length': '0'} 2022-09-03T05:30:12.175 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ff10cdedd4402890-006312e664-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.218 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:12.220 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:12.221 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:12.222 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:12.223 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:12.224 DEBUG:boto:Method: GET 2022-09-03T05:30:12.224 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:12.225 DEBUG:boto:Data: 2022-09-03T05:30:12.226 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.226 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.227 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.228 DEBUG:boto:Params: {} 2022-09-03T05:30:12.229 DEBUG:boto:Token: None 2022-09-03T05:30:12.229 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/ 2022-09-03T05:30:12.230 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE= 2022-09-03T05:30:12.231 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE=', 'Content-Length': '0'} 2022-09-03T05:30:12.233 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fce65ad67b15db0c-006312e664-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.235 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:12.236 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.237 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.238 DEBUG:boto:Method: HEAD 2022-09-03T05:30:12.239 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.239 DEBUG:boto:Data: 2022-09-03T05:30:12.240 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.241 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.242 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.243 DEBUG:boto:Params: {} 2022-09-03T05:30:12.243 DEBUG:boto:Token: None 2022-09-03T05:30:12.245 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.246 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wj49oVtZHw+/hAVNWmz+nHKi0no= 2022-09-03T05:30:12.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wj49oVtZHw+/hAVNWmz+nHKi0no=', 'Content-Length': '0'} 2022-09-03T05:30:12.254 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fa5cc026d5275337-006312e664-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.256 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.256 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.257 DEBUG:boto:Method: HEAD 2022-09-03T05:30:12.258 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.258 DEBUG:boto:Data: 2022-09-03T05:30:12.259 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.260 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.260 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.261 DEBUG:boto:Params: {} 2022-09-03T05:30:12.261 DEBUG:boto:Token: None 2022-09-03T05:30:12.262 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.263 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wj49oVtZHw+/hAVNWmz+nHKi0no= 2022-09-03T05:30:12.263 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wj49oVtZHw+/hAVNWmz+nHKi0no=', 'Content-Length': '0'} 2022-09-03T05:30:12.267 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002820fc4ffaa68b95-006312e664-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.268 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:30:12.268 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.269 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.269 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.270 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.271 DEBUG:boto:Method: GET 2022-09-03T05:30:12.271 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.272 DEBUG:boto:Data: 2022-09-03T05:30:12.272 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.273 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.274 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.274 DEBUG:boto:Params: {} 2022-09-03T05:30:12.275 DEBUG:boto:Token: None 2022-09-03T05:30:12.276 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.277 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yb12P3SsglRc0E+FdWP/1pnu0ck= 2022-09-03T05:30:12.277 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yb12P3SsglRc0E+FdWP/1pnu0ck=', 'Content-Length': '0'} 2022-09-03T05:30:12.280 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002065a8bc62bb9ede-006312e664-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.281 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.282 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:12.282 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.283 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.283 DEBUG:boto:Method: GET 2022-09-03T05:30:12.284 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.284 DEBUG:boto:Data: 2022-09-03T05:30:12.285 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.285 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.286 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.287 DEBUG:boto:Params: {} 2022-09-03T05:30:12.288 DEBUG:boto:Token: None 2022-09-03T05:30:12.289 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:12.289 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yb12P3SsglRc0E+FdWP/1pnu0ck= 2022-09-03T05:30:12.290 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yb12P3SsglRc0E+FdWP/1pnu0ck=', 'Content-Length': '0'} 2022-09-03T05:30:12.292 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000cdd357bea5038b0-006312e664-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.294 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:12.294 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:12.295 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:12.295 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:12.296 DEBUG:boto:Method: GET 2022-09-03T05:30:12.297 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:12.297 DEBUG:boto:Data: 2022-09-03T05:30:12.298 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.298 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.299 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.299 DEBUG:boto:Params: {} 2022-09-03T05:30:12.301 DEBUG:boto:Token: None 2022-09-03T05:30:12.301 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/ 2022-09-03T05:30:12.303 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE= 2022-09-03T05:30:12.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE=', 'Content-Length': '0'} 2022-09-03T05:30:12.307 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002d1433235e02a44c-006312e664-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.308 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:12.310 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:12.311 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:12.312 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:12.313 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:12.313 DEBUG:boto:Method: GET 2022-09-03T05:30:12.314 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:12.317 DEBUG:boto:Data: 2022-09-03T05:30:12.318 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.319 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.320 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.320 DEBUG:boto:Params: {} 2022-09-03T05:30:12.321 DEBUG:boto:Token: None 2022-09-03T05:30:12.322 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/ 2022-09-03T05:30:12.323 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE= 2022-09-03T05:30:12.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE=', 'Content-Length': '0'} 2022-09-03T05:30:12.328 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000078a4a2cc1ee7789b-006312e664-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.328 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:12.329 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:30:12.330 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.330 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.331 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.332 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.332 DEBUG:boto:Method: GET 2022-09-03T05:30:12.333 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.333 DEBUG:boto:Data: 2022-09-03T05:30:12.334 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.335 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.335 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.336 DEBUG:boto:Params: {} 2022-09-03T05:30:12.338 DEBUG:boto:Token: None 2022-09-03T05:30:12.339 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.339 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lK+hoA6uwQeg9VP4bf2cCWgNvuA= 2022-09-03T05:30:12.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lK+hoA6uwQeg9VP4bf2cCWgNvuA=', 'Content-Length': '0'} 2022-09-03T05:30:12.342 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b04f2021f45371dd-006312e664-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.386 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.387 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.388 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.389 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.390 DEBUG:boto:Method: GET 2022-09-03T05:30:12.392 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.393 DEBUG:boto:Data: 2022-09-03T05:30:12.394 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.394 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.395 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.395 DEBUG:boto:Params: {} 2022-09-03T05:30:12.396 DEBUG:boto:Token: None 2022-09-03T05:30:12.397 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.397 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lK+hoA6uwQeg9VP4bf2cCWgNvuA= 2022-09-03T05:30:12.398 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lK+hoA6uwQeg9VP4bf2cCWgNvuA=', 'Content-Length': '0'} 2022-09-03T05:30:12.402 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c5559ff05b9b6646-006312e664-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.403 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.403 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.404 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.404 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.405 DEBUG:boto:Method: HEAD 2022-09-03T05:30:12.406 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.406 DEBUG:boto:Data: 2022-09-03T05:30:12.407 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.407 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.408 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.409 DEBUG:boto:Params: {} 2022-09-03T05:30:12.409 DEBUG:boto:Token: None 2022-09-03T05:30:12.410 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.411 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bax2BRnew9U55ESq3PL4W8GHCPM= 2022-09-03T05:30:12.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bax2BRnew9U55ESq3PL4W8GHCPM=', 'Content-Length': '0'} 2022-09-03T05:30:12.415 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000965588eab9cc3aad-006312e664-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.415 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.416 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.417 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.418 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.418 DEBUG:boto:Method: HEAD 2022-09-03T05:30:12.419 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.420 DEBUG:boto:Data: 2022-09-03T05:30:12.420 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.421 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.421 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.422 DEBUG:boto:Params: {} 2022-09-03T05:30:12.422 DEBUG:boto:Token: None 2022-09-03T05:30:12.423 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.423 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bax2BRnew9U55ESq3PL4W8GHCPM= 2022-09-03T05:30:12.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bax2BRnew9U55ESq3PL4W8GHCPM=', 'Content-Length': '0'} 2022-09-03T05:30:12.426 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005d61cedf0b23917d-006312e664-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.427 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:30:12.427 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.428 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.428 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.429 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.429 DEBUG:boto:Method: GET 2022-09-03T05:30:12.430 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.430 DEBUG:boto:Data: 2022-09-03T05:30:12.431 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.431 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.432 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.433 DEBUG:boto:Params: {} 2022-09-03T05:30:12.433 DEBUG:boto:Token: None 2022-09-03T05:30:12.434 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.434 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lK+hoA6uwQeg9VP4bf2cCWgNvuA= 2022-09-03T05:30:12.435 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lK+hoA6uwQeg9VP4bf2cCWgNvuA=', 'Content-Length': '0'} 2022-09-03T05:30:12.437 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000085541955134825df-006312e664-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.438 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.439 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.439 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.440 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.440 DEBUG:boto:Method: GET 2022-09-03T05:30:12.441 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.441 DEBUG:boto:Data: 2022-09-03T05:30:12.442 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.442 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.443 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.444 DEBUG:boto:Params: {} 2022-09-03T05:30:12.444 DEBUG:boto:Token: None 2022-09-03T05:30:12.445 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.446 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lK+hoA6uwQeg9VP4bf2cCWgNvuA= 2022-09-03T05:30:12.446 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lK+hoA6uwQeg9VP4bf2cCWgNvuA=', 'Content-Length': '0'} 2022-09-03T05:30:12.449 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003df7dc3cbf59a37e-006312e664-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.451 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:12.452 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:12.453 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:12.454 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:12.455 DEBUG:boto:Method: GET 2022-09-03T05:30:12.456 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:12.457 DEBUG:boto:Data: 2022-09-03T05:30:12.457 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.458 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.459 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.459 DEBUG:boto:Params: {} 2022-09-03T05:30:12.460 DEBUG:boto:Token: None 2022-09-03T05:30:12.461 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/ 2022-09-03T05:30:12.462 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE= 2022-09-03T05:30:12.463 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE=', 'Content-Length': '0'} 2022-09-03T05:30:12.466 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000f2442da6eb173d7-006312e664-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.467 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:12.468 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:12.468 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:12.469 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:12.469 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:12.470 DEBUG:boto:Method: GET 2022-09-03T05:30:12.471 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:12.471 DEBUG:boto:Data: 2022-09-03T05:30:12.472 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.472 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.473 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.474 DEBUG:boto:Params: {} 2022-09-03T05:30:12.474 DEBUG:boto:Token: None 2022-09-03T05:30:12.475 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/ 2022-09-03T05:30:12.475 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE= 2022-09-03T05:30:12.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE=', 'Content-Length': '0'} 2022-09-03T05:30:12.479 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006dc0cb47e1c253a2-006312e664-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.480 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:12.481 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.481 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.482 DEBUG:boto:Method: HEAD 2022-09-03T05:30:12.483 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.484 DEBUG:boto:Data: 2022-09-03T05:30:12.484 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.485 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.486 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.486 DEBUG:boto:Params: {} 2022-09-03T05:30:12.487 DEBUG:boto:Token: None 2022-09-03T05:30:12.487 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.488 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZpufcfGYmjaLDnHO4MLHFqAy/fI= 2022-09-03T05:30:12.489 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZpufcfGYmjaLDnHO4MLHFqAy/fI=', 'Content-Length': '0'} 2022-09-03T05:30:12.491 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f7e25ae0fbf33870-006312e664-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.492 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.492 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.493 DEBUG:boto:Method: HEAD 2022-09-03T05:30:12.493 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.494 DEBUG:boto:Data: 2022-09-03T05:30:12.494 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.495 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.495 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.496 DEBUG:boto:Params: {} 2022-09-03T05:30:12.497 DEBUG:boto:Token: None 2022-09-03T05:30:12.497 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.498 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZpufcfGYmjaLDnHO4MLHFqAy/fI= 2022-09-03T05:30:12.498 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZpufcfGYmjaLDnHO4MLHFqAy/fI=', 'Content-Length': '0'} 2022-09-03T05:30:12.501 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c255351546ce6ee4-006312e664-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.502 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:30:12.502 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.503 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.503 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.504 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.504 DEBUG:boto:Method: GET 2022-09-03T05:30:12.505 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.505 DEBUG:boto:Data: 2022-09-03T05:30:12.506 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.506 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.507 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.508 DEBUG:boto:Params: {} 2022-09-03T05:30:12.508 DEBUG:boto:Token: None 2022-09-03T05:30:12.509 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.509 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lK+hoA6uwQeg9VP4bf2cCWgNvuA= 2022-09-03T05:30:12.510 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lK+hoA6uwQeg9VP4bf2cCWgNvuA=', 'Content-Length': '0'} 2022-09-03T05:30:12.512 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007348c07f0d8a6034-006312e664-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.554 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.555 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:12.556 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.556 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.557 DEBUG:boto:Method: GET 2022-09-03T05:30:12.558 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.558 DEBUG:boto:Data: 2022-09-03T05:30:12.559 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.559 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.560 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.561 DEBUG:boto:Params: {} 2022-09-03T05:30:12.562 DEBUG:boto:Token: None 2022-09-03T05:30:12.563 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:12.563 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lK+hoA6uwQeg9VP4bf2cCWgNvuA= 2022-09-03T05:30:12.564 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lK+hoA6uwQeg9VP4bf2cCWgNvuA=', 'Content-Length': '0'} 2022-09-03T05:30:12.566 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000286468a00e90cafa-006312e664-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.567 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:12.567 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:12.568 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:12.569 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:12.570 DEBUG:boto:Method: GET 2022-09-03T05:30:12.570 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:12.571 DEBUG:boto:Data: 2022-09-03T05:30:12.571 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.572 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.572 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.573 DEBUG:boto:Params: {} 2022-09-03T05:30:12.573 DEBUG:boto:Token: None 2022-09-03T05:30:12.574 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/ 2022-09-03T05:30:12.575 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE= 2022-09-03T05:30:12.575 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE=', 'Content-Length': '0'} 2022-09-03T05:30:12.578 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000db2e12a36ede7f12-006312e664-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.579 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:12.580 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:12.580 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:12.581 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:12.582 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:12.582 DEBUG:boto:Method: GET 2022-09-03T05:30:12.583 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:12.584 DEBUG:boto:Data: 2022-09-03T05:30:12.585 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.586 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.586 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.587 DEBUG:boto:Params: {} 2022-09-03T05:30:12.587 DEBUG:boto:Token: None 2022-09-03T05:30:12.588 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/ 2022-09-03T05:30:12.589 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE= 2022-09-03T05:30:12.589 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE=', 'Content-Length': '0'} 2022-09-03T05:30:12.592 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002ab4569620707875-006312e664-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.593 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:12.594 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:30:12.594 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.595 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.596 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.596 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.597 DEBUG:boto:Method: GET 2022-09-03T05:30:12.597 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.598 DEBUG:boto:Data: 2022-09-03T05:30:12.599 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.599 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.600 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.601 DEBUG:boto:Params: {} 2022-09-03T05:30:12.601 DEBUG:boto:Token: None 2022-09-03T05:30:12.602 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.602 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:umNKRNNTgowgc5TBScilnvfF2GQ= 2022-09-03T05:30:12.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:umNKRNNTgowgc5TBScilnvfF2GQ=', 'Content-Length': '0'} 2022-09-03T05:30:12.605 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d12c970f48340007-006312e664-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.606 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.607 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.607 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.608 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.609 DEBUG:boto:Method: GET 2022-09-03T05:30:12.609 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.610 DEBUG:boto:Data: 2022-09-03T05:30:12.610 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.611 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.611 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.612 DEBUG:boto:Params: {} 2022-09-03T05:30:12.613 DEBUG:boto:Token: None 2022-09-03T05:30:12.613 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.614 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:umNKRNNTgowgc5TBScilnvfF2GQ= 2022-09-03T05:30:12.615 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:umNKRNNTgowgc5TBScilnvfF2GQ=', 'Content-Length': '0'} 2022-09-03T05:30:12.617 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000985fde131a5c5254-006312e664-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.618 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.619 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.619 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.620 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.620 DEBUG:boto:Method: HEAD 2022-09-03T05:30:12.621 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.621 DEBUG:boto:Data: 2022-09-03T05:30:12.622 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.622 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.623 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.623 DEBUG:boto:Params: {} 2022-09-03T05:30:12.624 DEBUG:boto:Token: None 2022-09-03T05:30:12.625 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.625 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7qZ8w/JKDFEWY0Saz74ofIAk+GA= 2022-09-03T05:30:12.626 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7qZ8w/JKDFEWY0Saz74ofIAk+GA=', 'Content-Length': '0'} 2022-09-03T05:30:12.628 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f17f1d7ddad9ca45-006312e664-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.629 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.630 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.630 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.631 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.632 DEBUG:boto:Method: HEAD 2022-09-03T05:30:12.632 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.633 DEBUG:boto:Data: 2022-09-03T05:30:12.635 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.635 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.636 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.637 DEBUG:boto:Params: {} 2022-09-03T05:30:12.639 DEBUG:boto:Token: None 2022-09-03T05:30:12.640 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.641 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7qZ8w/JKDFEWY0Saz74ofIAk+GA= 2022-09-03T05:30:12.642 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7qZ8w/JKDFEWY0Saz74ofIAk+GA=', 'Content-Length': '0'} 2022-09-03T05:30:12.644 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e60e787ee7db8fd3-006312e664-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.645 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:30:12.646 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.646 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.647 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.648 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.649 DEBUG:boto:Method: GET 2022-09-03T05:30:12.649 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.650 DEBUG:boto:Data: 2022-09-03T05:30:12.651 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.651 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.652 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.653 DEBUG:boto:Params: {} 2022-09-03T05:30:12.653 DEBUG:boto:Token: None 2022-09-03T05:30:12.654 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.655 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:umNKRNNTgowgc5TBScilnvfF2GQ= 2022-09-03T05:30:12.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:umNKRNNTgowgc5TBScilnvfF2GQ=', 'Content-Length': '0'} 2022-09-03T05:30:12.660 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000217a531445dfe56a-006312e664-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.661 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.662 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.663 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.664 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.665 DEBUG:boto:Method: GET 2022-09-03T05:30:12.666 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.667 DEBUG:boto:Data: 2022-09-03T05:30:12.668 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.669 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.671 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.672 DEBUG:boto:Params: {} 2022-09-03T05:30:12.673 DEBUG:boto:Token: None 2022-09-03T05:30:12.674 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.676 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:umNKRNNTgowgc5TBScilnvfF2GQ= 2022-09-03T05:30:12.677 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:umNKRNNTgowgc5TBScilnvfF2GQ=', 'Content-Length': '0'} 2022-09-03T05:30:12.682 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000abec7505323e1ffb-006312e664-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.683 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:12.685 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:12.686 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:12.687 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:12.688 DEBUG:boto:Method: GET 2022-09-03T05:30:12.689 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:12.690 DEBUG:boto:Data: 2022-09-03T05:30:12.691 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.692 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.693 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.694 DEBUG:boto:Params: {} 2022-09-03T05:30:12.695 DEBUG:boto:Token: None 2022-09-03T05:30:12.696 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/ 2022-09-03T05:30:12.697 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE= 2022-09-03T05:30:12.699 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE=', 'Content-Length': '0'} 2022-09-03T05:30:12.702 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c0156aa27bc50e57-006312e664-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.704 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:12.704 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:12.705 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:12.705 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:12.706 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:12.707 DEBUG:boto:Method: GET 2022-09-03T05:30:12.707 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:12.708 DEBUG:boto:Data: 2022-09-03T05:30:12.708 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.709 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.709 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.710 DEBUG:boto:Params: {} 2022-09-03T05:30:12.710 DEBUG:boto:Token: None 2022-09-03T05:30:12.711 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/ 2022-09-03T05:30:12.712 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE= 2022-09-03T05:30:12.713 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE=', 'Content-Length': '0'} 2022-09-03T05:30:12.715 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000290f7d60cba34cce-006312e664-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.717 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:12.718 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.719 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.719 DEBUG:boto:Method: HEAD 2022-09-03T05:30:12.720 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.720 DEBUG:boto:Data: 2022-09-03T05:30:12.721 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.722 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.723 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.724 DEBUG:boto:Params: {} 2022-09-03T05:30:12.724 DEBUG:boto:Token: None 2022-09-03T05:30:12.726 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.727 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5eNLcwozTMzsrVDDhc9RYHdqLRQ= 2022-09-03T05:30:12.728 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5eNLcwozTMzsrVDDhc9RYHdqLRQ=', 'Content-Length': '0'} 2022-09-03T05:30:12.731 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003c26c8b8bbed7179-006312e664-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.732 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.733 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.733 DEBUG:boto:Method: HEAD 2022-09-03T05:30:12.734 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.735 DEBUG:boto:Data: 2022-09-03T05:30:12.736 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.736 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.737 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.738 DEBUG:boto:Params: {} 2022-09-03T05:30:12.739 DEBUG:boto:Token: None 2022-09-03T05:30:12.740 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.741 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5eNLcwozTMzsrVDDhc9RYHdqLRQ= 2022-09-03T05:30:12.742 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5eNLcwozTMzsrVDDhc9RYHdqLRQ=', 'Content-Length': '0'} 2022-09-03T05:30:12.745 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000576a90244c91b850-006312e664-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.747 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:30:12.748 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.749 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.750 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.750 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.751 DEBUG:boto:Method: GET 2022-09-03T05:30:12.752 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.753 DEBUG:boto:Data: 2022-09-03T05:30:12.754 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.755 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.756 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.757 DEBUG:boto:Params: {} 2022-09-03T05:30:12.759 DEBUG:boto:Token: None 2022-09-03T05:30:12.760 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.761 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:umNKRNNTgowgc5TBScilnvfF2GQ= 2022-09-03T05:30:12.762 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:umNKRNNTgowgc5TBScilnvfF2GQ=', 'Content-Length': '0'} 2022-09-03T05:30:12.765 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000db58055af2328ec8-006312e664-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.806 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.807 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:12.808 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.809 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.811 DEBUG:boto:Method: GET 2022-09-03T05:30:12.812 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.813 DEBUG:boto:Data: 2022-09-03T05:30:12.814 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.815 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.816 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.817 DEBUG:boto:Params: {} 2022-09-03T05:30:12.818 DEBUG:boto:Token: None 2022-09-03T05:30:12.819 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:12.820 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:umNKRNNTgowgc5TBScilnvfF2GQ= 2022-09-03T05:30:12.821 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:umNKRNNTgowgc5TBScilnvfF2GQ=', 'Content-Length': '0'} 2022-09-03T05:30:12.824 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000056ffee389ce325aa-006312e664-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.826 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:12.827 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:12.828 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:12.829 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:12.830 DEBUG:boto:Method: GET 2022-09-03T05:30:12.831 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:12.832 DEBUG:boto:Data: 2022-09-03T05:30:12.833 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.834 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.835 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.836 DEBUG:boto:Params: {} 2022-09-03T05:30:12.837 DEBUG:boto:Token: None 2022-09-03T05:30:12.838 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/ 2022-09-03T05:30:12.839 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE= 2022-09-03T05:30:12.841 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE=', 'Content-Length': '0'} 2022-09-03T05:30:12.844 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000072bb4a3a0120e0b1-006312e664-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.845 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:12.846 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:12.847 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:12.848 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:12.849 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:12.850 DEBUG:boto:Method: GET 2022-09-03T05:30:12.851 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:12.852 DEBUG:boto:Data: 2022-09-03T05:30:12.853 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.854 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:12.855 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.856 DEBUG:boto:Params: {} 2022-09-03T05:30:12.858 DEBUG:boto:Token: None 2022-09-03T05:30:12.859 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/ 2022-09-03T05:30:12.860 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE= 2022-09-03T05:30:12.861 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3+/OodQtDnhW4A8jL4xbA/X+KyE=', 'Content-Length': '0'} 2022-09-03T05:30:12.864 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002957e07e6b484dc5-006312e664-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.865 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:12.866 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-17 zones={test-zone3, test-zone1} 2022-09-03T05:30:12.867 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-17 zones={test-zone3, test-zone2} 2022-09-03T05:30:12.867 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:12.868 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:12.869 DEBUG:boto:Method: HEAD 2022-09-03T05:30:12.870 DEBUG:boto:Path: /dsgvcy-17/ 2022-09-03T05:30:12.870 DEBUG:boto:Data: 2022-09-03T05:30:12.871 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.872 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.873 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.873 DEBUG:boto:Params: {} 2022-09-03T05:30:12.874 DEBUG:boto:Token: None 2022-09-03T05:30:12.874 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/ 2022-09-03T05:30:12.875 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5AQB1VBp+q7RU6lSQQ7ZUfdXrcc= 2022-09-03T05:30:12.876 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5AQB1VBp+q7RU6lSQQ7ZUfdXrcc=', 'Content-Length': '0'} 2022-09-03T05:30:12.878 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c50ffdaa2570abc3-006312e664-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.879 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:12.879 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:12.880 DEBUG:boto:Method: HEAD 2022-09-03T05:30:12.880 DEBUG:boto:Path: /dsgvcy-17/ 2022-09-03T05:30:12.881 DEBUG:boto:Data: 2022-09-03T05:30:12.881 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.882 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:12.882 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.883 DEBUG:boto:Params: {} 2022-09-03T05:30:12.883 DEBUG:boto:Token: None 2022-09-03T05:30:12.884 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/ 2022-09-03T05:30:12.884 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5AQB1VBp+q7RU6lSQQ7ZUfdXrcc= 2022-09-03T05:30:12.885 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5AQB1VBp+q7RU6lSQQ7ZUfdXrcc=', 'Content-Length': '0'} 2022-09-03T05:30:12.888 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000007a35ab2cca1d894-006312e664-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.888 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:30:12.889 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:12.889 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:12.890 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:12.891 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:12.891 DEBUG:boto:Method: GET 2022-09-03T05:30:12.892 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:12.892 DEBUG:boto:Data: 2022-09-03T05:30:12.893 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.894 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.894 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.895 DEBUG:boto:Params: {} 2022-09-03T05:30:12.896 DEBUG:boto:Token: None 2022-09-03T05:30:12.896 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/?versions 2022-09-03T05:30:12.897 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:78PGJBQrTMbnmu4K6nK5GlEWYUs= 2022-09-03T05:30:12.897 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:78PGJBQrTMbnmu4K6nK5GlEWYUs=', 'Content-Length': '0'} 2022-09-03T05:30:12.900 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000064e8e685594a6131-006312e664-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.901 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:12.902 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:30:12.903 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:30:12.904 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:30:12.905 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:30:12.906 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:30:12.907 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:12.907 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:12.908 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:12.908 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:12.909 DEBUG:boto:Method: GET 2022-09-03T05:30:12.909 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:12.910 DEBUG:boto:Data: 2022-09-03T05:30:12.911 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.911 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:12.912 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.912 DEBUG:boto:Params: {} 2022-09-03T05:30:12.913 DEBUG:boto:Token: None 2022-09-03T05:30:12.913 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/?versions 2022-09-03T05:30:12.914 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:78PGJBQrTMbnmu4K6nK5GlEWYUs= 2022-09-03T05:30:12.914 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:78PGJBQrTMbnmu4K6nK5GlEWYUs=', 'Content-Length': '0'} 2022-09-03T05:30:12.917 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003455ec3098b3d07f-006312e664-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.918 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:12.919 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:30:12.919 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:30:12.920 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:30:12.921 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:30:12.921 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:12.922 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:12.922 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:12.923 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:12.923 DEBUG:boto:Method: GET 2022-09-03T05:30:12.924 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:12.924 DEBUG:boto:Data: 2022-09-03T05:30:12.925 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.925 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.926 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.926 DEBUG:boto:Params: {} 2022-09-03T05:30:12.927 DEBUG:boto:Token: None 2022-09-03T05:30:12.928 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/?versions 2022-09-03T05:30:12.929 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:78PGJBQrTMbnmu4K6nK5GlEWYUs= 2022-09-03T05:30:12.930 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:78PGJBQrTMbnmu4K6nK5GlEWYUs=', 'Content-Length': '0'} 2022-09-03T05:30:12.932 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cc9ec7db662e7a53-006312e664-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.933 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:12.935 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:12.935 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:12.936 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:12.936 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:12.937 DEBUG:boto:Method: GET 2022-09-03T05:30:12.937 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:30:12.938 DEBUG:boto:Data: 2022-09-03T05:30:12.938 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.939 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:12.939 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.940 DEBUG:boto:Params: {} 2022-09-03T05:30:12.941 DEBUG:boto:Token: None 2022-09-03T05:30:12.941 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/?versions 2022-09-03T05:30:12.942 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:78PGJBQrTMbnmu4K6nK5GlEWYUs= 2022-09-03T05:30:12.942 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:78PGJBQrTMbnmu4K6nK5GlEWYUs=', 'Content-Length': '0'} 2022-09-03T05:30:12.945 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000066af91e08667bdf7-006312e664-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.946 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:12.947 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:30:12.948 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:12.949 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:12.949 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.950 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.950 DEBUG:boto:Method: GET 2022-09-03T05:30:12.951 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.952 DEBUG:boto:Data: 2022-09-03T05:30:12.952 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.953 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.953 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.954 DEBUG:boto:Params: {} 2022-09-03T05:30:12.954 DEBUG:boto:Token: None 2022-09-03T05:30:12.955 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.956 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8m6NMRdP9Uhem8cmc8cJJQwJCDU= 2022-09-03T05:30:12.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8m6NMRdP9Uhem8cmc8cJJQwJCDU=', 'Content-Length': '0'} 2022-09-03T05:30:12.958 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000006b89c4a11572f016-006312e664-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.959 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:12.960 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:12.960 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.961 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.961 DEBUG:boto:Method: GET 2022-09-03T05:30:12.962 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.963 DEBUG:boto:Data: 2022-09-03T05:30:12.964 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.964 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:12.965 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.965 DEBUG:boto:Params: {} 2022-09-03T05:30:12.966 DEBUG:boto:Token: None 2022-09-03T05:30:12.967 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.967 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8m6NMRdP9Uhem8cmc8cJJQwJCDU= 2022-09-03T05:30:12.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8m6NMRdP9Uhem8cmc8cJJQwJCDU=', 'Content-Length': '0'} 2022-09-03T05:30:12.970 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000d921104b2ee8c788-006312e664-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.971 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:12.972 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:12.972 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.973 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.973 DEBUG:boto:Method: HEAD 2022-09-03T05:30:12.974 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.974 DEBUG:boto:Data: 2022-09-03T05:30:12.975 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.975 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:12.976 DEBUG:boto:Port: 8001 2022-09-03T05:30:12.976 DEBUG:boto:Params: {} 2022-09-03T05:30:12.977 DEBUG:boto:Token: None 2022-09-03T05:30:12.977 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.978 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7kMaH6lP9P1dqeEHt75H9929wQc= 2022-09-03T05:30:12.979 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7kMaH6lP9P1dqeEHt75H9929wQc=', 'Content-Length': '0'} 2022-09-03T05:30:12.981 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f1aa4f4d81d35eec-006312e664-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.982 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:12.983 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:12.983 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.984 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.984 DEBUG:boto:Method: HEAD 2022-09-03T05:30:12.985 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.985 DEBUG:boto:Data: 2022-09-03T05:30:12.986 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.986 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:12.987 DEBUG:boto:Port: 8000 2022-09-03T05:30:12.987 DEBUG:boto:Params: {} 2022-09-03T05:30:12.988 DEBUG:boto:Token: None 2022-09-03T05:30:12.989 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:12 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.990 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7kMaH6lP9P1dqeEHt75H9929wQc= 2022-09-03T05:30:12.990 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7kMaH6lP9P1dqeEHt75H9929wQc=', 'Content-Length': '0'} 2022-09-03T05:30:12.993 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000027b1605fbd97661d-006312e664-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:12.994 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:30:12.995 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:12.995 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:12.996 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.996 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.997 DEBUG:boto:Method: GET 2022-09-03T05:30:12.997 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:12.998 DEBUG:boto:Data: 2022-09-03T05:30:12.998 DEBUG:boto:Headers: {} 2022-09-03T05:30:12.999 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.000 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.001 DEBUG:boto:Params: {} 2022-09-03T05:30:13.001 DEBUG:boto:Token: None 2022-09-03T05:30:13.002 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:13.003 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0L84srA8qXStR4Dfsmg/8tgpy+o= 2022-09-03T05:30:13.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0L84srA8qXStR4Dfsmg/8tgpy+o=', 'Content-Length': '0'} 2022-09-03T05:30:13.005 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f57e773cad967c95-006312e665-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.006 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:13.007 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:13.008 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:13.008 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:13.009 DEBUG:boto:Method: GET 2022-09-03T05:30:13.009 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:13.010 DEBUG:boto:Data: 2022-09-03T05:30:13.010 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.011 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.011 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.012 DEBUG:boto:Params: {} 2022-09-03T05:30:13.013 DEBUG:boto:Token: None 2022-09-03T05:30:13.013 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:13.014 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0L84srA8qXStR4Dfsmg/8tgpy+o= 2022-09-03T05:30:13.014 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0L84srA8qXStR4Dfsmg/8tgpy+o=', 'Content-Length': '0'} 2022-09-03T05:30:13.017 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000007efd20093c91e9bf-006312e665-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.017 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:13.018 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:13.019 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:13.019 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:13.020 DEBUG:boto:Method: GET 2022-09-03T05:30:13.020 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:30:13.021 DEBUG:boto:Data: 2022-09-03T05:30:13.021 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.022 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.022 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.023 DEBUG:boto:Params: {} 2022-09-03T05:30:13.025 DEBUG:boto:Token: None 2022-09-03T05:30:13.026 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/ 2022-09-03T05:30:13.027 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8= 2022-09-03T05:30:13.028 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8=', 'Content-Length': '0'} 2022-09-03T05:30:13.030 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000034384756e4c6dd02-006312e665-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.031 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:13.032 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:13.033 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:13.034 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:13.034 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:13.035 DEBUG:boto:Method: GET 2022-09-03T05:30:13.035 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:30:13.036 DEBUG:boto:Data: 2022-09-03T05:30:13.036 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.037 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.038 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.038 DEBUG:boto:Params: {} 2022-09-03T05:30:13.039 DEBUG:boto:Token: None 2022-09-03T05:30:13.039 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/ 2022-09-03T05:30:13.040 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8= 2022-09-03T05:30:13.041 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8=', 'Content-Length': '0'} 2022-09-03T05:30:13.043 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000381ce6708246eadf-006312e665-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.044 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:13.045 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:13.046 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:13.046 DEBUG:boto:Method: HEAD 2022-09-03T05:30:13.047 DEBUG:boto:Path: /dsgvcy-17/dummy 2022-09-03T05:30:13.047 DEBUG:boto:Data: 2022-09-03T05:30:13.048 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.048 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.049 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.049 DEBUG:boto:Params: {} 2022-09-03T05:30:13.050 DEBUG:boto:Token: None 2022-09-03T05:30:13.051 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/dummy 2022-09-03T05:30:13.051 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zx/F6tyg2STjPn6pwNFz58upq5M= 2022-09-03T05:30:13.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zx/F6tyg2STjPn6pwNFz58upq5M=', 'Content-Length': '0'} 2022-09-03T05:30:13.054 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a9207fa884badae0-006312e665-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.055 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:13.055 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:13.056 DEBUG:boto:Method: HEAD 2022-09-03T05:30:13.056 DEBUG:boto:Path: /dsgvcy-17/dummy 2022-09-03T05:30:13.057 DEBUG:boto:Data: 2022-09-03T05:30:13.057 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.058 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.058 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.059 DEBUG:boto:Params: {} 2022-09-03T05:30:13.059 DEBUG:boto:Token: None 2022-09-03T05:30:13.060 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/dummy 2022-09-03T05:30:13.061 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zx/F6tyg2STjPn6pwNFz58upq5M= 2022-09-03T05:30:13.063 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zx/F6tyg2STjPn6pwNFz58upq5M=', 'Content-Length': '0'} 2022-09-03T05:30:13.065 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000002b31032210c5fd6d-006312e665-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.066 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:30:13.066 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:13.067 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:13.067 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:13.068 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:13.069 DEBUG:boto:Method: GET 2022-09-03T05:30:13.069 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:13.070 DEBUG:boto:Data: 2022-09-03T05:30:13.070 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.071 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.072 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.072 DEBUG:boto:Params: {} 2022-09-03T05:30:13.073 DEBUG:boto:Token: None 2022-09-03T05:30:13.074 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:13.074 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0L84srA8qXStR4Dfsmg/8tgpy+o= 2022-09-03T05:30:13.075 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0L84srA8qXStR4Dfsmg/8tgpy+o=', 'Content-Length': '0'} 2022-09-03T05:30:13.078 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000afb82ded8ac09095-006312e665-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.079 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:30:13.080 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:30:13.080 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:13.081 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:13.081 DEBUG:boto:Method: GET 2022-09-03T05:30:13.082 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:13.082 DEBUG:boto:Data: 2022-09-03T05:30:13.083 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.084 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.084 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.085 DEBUG:boto:Params: {} 2022-09-03T05:30:13.085 DEBUG:boto:Token: None 2022-09-03T05:30:13.086 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:30:13.086 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0L84srA8qXStR4Dfsmg/8tgpy+o= 2022-09-03T05:30:13.087 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0L84srA8qXStR4Dfsmg/8tgpy+o=', 'Content-Length': '0'} 2022-09-03T05:30:13.089 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000009f6f1b51f11b4ac8-006312e665-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.090 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:13.091 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:13.091 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:13.092 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:13.092 DEBUG:boto:Method: GET 2022-09-03T05:30:13.093 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:30:13.093 DEBUG:boto:Data: 2022-09-03T05:30:13.094 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.095 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.095 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.096 DEBUG:boto:Params: {} 2022-09-03T05:30:13.096 DEBUG:boto:Token: None 2022-09-03T05:30:13.097 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/ 2022-09-03T05:30:13.097 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8= 2022-09-03T05:30:13.098 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8=', 'Content-Length': '0'} 2022-09-03T05:30:13.101 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002bb2723e720bb1af-006312e665-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.142 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:13.143 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:13.144 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:13.144 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:13.145 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:30:13.145 DEBUG:boto:Method: GET 2022-09-03T05:30:13.146 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:30:13.146 DEBUG:boto:Data: 2022-09-03T05:30:13.147 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.147 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.148 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.148 DEBUG:boto:Params: {} 2022-09-03T05:30:13.149 DEBUG:boto:Token: None 2022-09-03T05:30:13.149 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/ 2022-09-03T05:30:13.150 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8= 2022-09-03T05:30:13.151 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8=', 'Content-Length': '0'} 2022-09-03T05:30:13.153 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003761c9f68c9e40db-006312e665-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.155 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:13.156 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:30:13.156 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.157 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.157 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.158 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.158 DEBUG:boto:Method: GET 2022-09-03T05:30:13.159 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.159 DEBUG:boto:Data: 2022-09-03T05:30:13.160 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.160 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.161 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.162 DEBUG:boto:Params: {} 2022-09-03T05:30:13.162 DEBUG:boto:Token: None 2022-09-03T05:30:13.163 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.163 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iELBBrXkgFM7yuq4pCxSHZXI56M= 2022-09-03T05:30:13.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iELBBrXkgFM7yuq4pCxSHZXI56M=', 'Content-Length': '0'} 2022-09-03T05:30:13.166 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001dffaee102661075-006312e665-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.167 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.167 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.168 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.168 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.169 DEBUG:boto:Method: GET 2022-09-03T05:30:13.170 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.170 DEBUG:boto:Data: 2022-09-03T05:30:13.171 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.171 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.172 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.172 DEBUG:boto:Params: {} 2022-09-03T05:30:13.173 DEBUG:boto:Token: None 2022-09-03T05:30:13.173 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.174 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iELBBrXkgFM7yuq4pCxSHZXI56M= 2022-09-03T05:30:13.175 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iELBBrXkgFM7yuq4pCxSHZXI56M=', 'Content-Length': '0'} 2022-09-03T05:30:13.177 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e33c7a5222170cec-006312e665-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.178 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.178 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.179 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.179 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.180 DEBUG:boto:Method: HEAD 2022-09-03T05:30:13.180 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.181 DEBUG:boto:Data: 2022-09-03T05:30:13.181 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.182 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.182 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.183 DEBUG:boto:Params: {} 2022-09-03T05:30:13.183 DEBUG:boto:Token: None 2022-09-03T05:30:13.184 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.185 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7zvaUF3IOvX8E/+Ykl9rB2JFeZI= 2022-09-03T05:30:13.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7zvaUF3IOvX8E/+Ykl9rB2JFeZI=', 'Content-Length': '0'} 2022-09-03T05:30:13.187 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009b5c81e4bf382619-006312e665-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.188 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.189 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.189 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.190 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.191 DEBUG:boto:Method: HEAD 2022-09-03T05:30:13.191 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.192 DEBUG:boto:Data: 2022-09-03T05:30:13.192 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.193 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.194 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.195 DEBUG:boto:Params: {} 2022-09-03T05:30:13.196 DEBUG:boto:Token: None 2022-09-03T05:30:13.196 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.197 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7zvaUF3IOvX8E/+Ykl9rB2JFeZI= 2022-09-03T05:30:13.198 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7zvaUF3IOvX8E/+Ykl9rB2JFeZI=', 'Content-Length': '0'} 2022-09-03T05:30:13.200 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000eed0e5ce6dfee04a-006312e665-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.201 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:30:13.202 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.203 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.204 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.205 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.206 DEBUG:boto:Method: GET 2022-09-03T05:30:13.207 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.207 DEBUG:boto:Data: 2022-09-03T05:30:13.208 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.208 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.209 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.209 DEBUG:boto:Params: {} 2022-09-03T05:30:13.210 DEBUG:boto:Token: None 2022-09-03T05:30:13.211 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.211 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iELBBrXkgFM7yuq4pCxSHZXI56M= 2022-09-03T05:30:13.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iELBBrXkgFM7yuq4pCxSHZXI56M=', 'Content-Length': '0'} 2022-09-03T05:30:13.214 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004f72a607221aeb1c-006312e665-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.215 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.216 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.216 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.217 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.217 DEBUG:boto:Method: GET 2022-09-03T05:30:13.218 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.218 DEBUG:boto:Data: 2022-09-03T05:30:13.219 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.219 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.220 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.220 DEBUG:boto:Params: {} 2022-09-03T05:30:13.221 DEBUG:boto:Token: None 2022-09-03T05:30:13.222 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.222 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iELBBrXkgFM7yuq4pCxSHZXI56M= 2022-09-03T05:30:13.223 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iELBBrXkgFM7yuq4pCxSHZXI56M=', 'Content-Length': '0'} 2022-09-03T05:30:13.225 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003d4fc32e09efb77b-006312e665-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.226 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:13.228 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:13.228 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:13.229 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:13.230 DEBUG:boto:Method: GET 2022-09-03T05:30:13.231 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:13.232 DEBUG:boto:Data: 2022-09-03T05:30:13.232 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.233 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.234 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.235 DEBUG:boto:Params: {} 2022-09-03T05:30:13.236 DEBUG:boto:Token: None 2022-09-03T05:30:13.238 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/ 2022-09-03T05:30:13.239 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8= 2022-09-03T05:30:13.241 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8=', 'Content-Length': '0'} 2022-09-03T05:30:13.244 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f3d2eec56b2be1d3-006312e665-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.290 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:13.291 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:13.292 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:13.293 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:13.294 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:13.295 DEBUG:boto:Method: GET 2022-09-03T05:30:13.296 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:13.297 DEBUG:boto:Data: 2022-09-03T05:30:13.298 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.299 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.300 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.300 DEBUG:boto:Params: {} 2022-09-03T05:30:13.302 DEBUG:boto:Token: None 2022-09-03T05:30:13.303 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/ 2022-09-03T05:30:13.304 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8= 2022-09-03T05:30:13.305 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8=', 'Content-Length': '0'} 2022-09-03T05:30:13.307 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000699ec13944c5ba0e-006312e665-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.308 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:13.309 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.309 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.310 DEBUG:boto:Method: HEAD 2022-09-03T05:30:13.310 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.311 DEBUG:boto:Data: 2022-09-03T05:30:13.311 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.312 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.312 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.313 DEBUG:boto:Params: {} 2022-09-03T05:30:13.313 DEBUG:boto:Token: None 2022-09-03T05:30:13.314 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.315 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lIXQ5ABanFGyIlswb2pl2FLhMKo= 2022-09-03T05:30:13.315 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lIXQ5ABanFGyIlswb2pl2FLhMKo=', 'Content-Length': '0'} 2022-09-03T05:30:13.318 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000384bac027ee5bd43-006312e665-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.319 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.319 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.320 DEBUG:boto:Method: HEAD 2022-09-03T05:30:13.320 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.321 DEBUG:boto:Data: 2022-09-03T05:30:13.321 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.322 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.322 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.323 DEBUG:boto:Params: {} 2022-09-03T05:30:13.323 DEBUG:boto:Token: None 2022-09-03T05:30:13.324 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.324 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lIXQ5ABanFGyIlswb2pl2FLhMKo= 2022-09-03T05:30:13.325 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lIXQ5ABanFGyIlswb2pl2FLhMKo=', 'Content-Length': '0'} 2022-09-03T05:30:13.328 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000eb2d18d78ec33828-006312e665-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.329 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:30:13.329 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.330 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.331 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.331 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.332 DEBUG:boto:Method: GET 2022-09-03T05:30:13.333 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.333 DEBUG:boto:Data: 2022-09-03T05:30:13.334 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.334 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.335 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.336 DEBUG:boto:Params: {} 2022-09-03T05:30:13.337 DEBUG:boto:Token: None 2022-09-03T05:30:13.338 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.339 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iELBBrXkgFM7yuq4pCxSHZXI56M= 2022-09-03T05:30:13.339 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iELBBrXkgFM7yuq4pCxSHZXI56M=', 'Content-Length': '0'} 2022-09-03T05:30:13.342 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000aeb3e62f8ecd989e-006312e665-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.343 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.343 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:30:13.344 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.344 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.345 DEBUG:boto:Method: GET 2022-09-03T05:30:13.345 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.346 DEBUG:boto:Data: 2022-09-03T05:30:13.346 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.347 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.347 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.348 DEBUG:boto:Params: {} 2022-09-03T05:30:13.348 DEBUG:boto:Token: None 2022-09-03T05:30:13.349 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:30:13.350 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iELBBrXkgFM7yuq4pCxSHZXI56M= 2022-09-03T05:30:13.350 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iELBBrXkgFM7yuq4pCxSHZXI56M=', 'Content-Length': '0'} 2022-09-03T05:30:13.352 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005a102e6ed0719c10-006312e665-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.353 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:13.354 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:13.354 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:13.355 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:13.355 DEBUG:boto:Method: GET 2022-09-03T05:30:13.356 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:13.356 DEBUG:boto:Data: 2022-09-03T05:30:13.357 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.358 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.358 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.359 DEBUG:boto:Params: {} 2022-09-03T05:30:13.359 DEBUG:boto:Token: None 2022-09-03T05:30:13.360 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/ 2022-09-03T05:30:13.360 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8= 2022-09-03T05:30:13.361 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8=', 'Content-Length': '0'} 2022-09-03T05:30:13.364 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001ee25fe344818397-006312e665-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.364 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:13.365 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:13.366 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:13.366 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:13.367 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:13.367 DEBUG:boto:Method: GET 2022-09-03T05:30:13.368 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:30:13.369 DEBUG:boto:Data: 2022-09-03T05:30:13.369 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.370 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.370 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.371 DEBUG:boto:Params: {} 2022-09-03T05:30:13.371 DEBUG:boto:Token: None 2022-09-03T05:30:13.372 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/ 2022-09-03T05:30:13.372 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8= 2022-09-03T05:30:13.373 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8=', 'Content-Length': '0'} 2022-09-03T05:30:13.376 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000005696f07844b35de-006312e665-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.376 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:13.377 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:30:13.378 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.378 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.379 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.379 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.380 DEBUG:boto:Method: GET 2022-09-03T05:30:13.380 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.381 DEBUG:boto:Data: 2022-09-03T05:30:13.381 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.382 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.383 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.383 DEBUG:boto:Params: {} 2022-09-03T05:30:13.384 DEBUG:boto:Token: None 2022-09-03T05:30:13.384 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.385 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6yedv6QRp6Yb93709MB9VbijCCk= 2022-09-03T05:30:13.386 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6yedv6QRp6Yb93709MB9VbijCCk=', 'Content-Length': '0'} 2022-09-03T05:30:13.388 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000035eb09ea5c0497e3-006312e665-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.389 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.389 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.390 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.390 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.391 DEBUG:boto:Method: GET 2022-09-03T05:30:13.392 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.392 DEBUG:boto:Data: 2022-09-03T05:30:13.393 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.393 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.394 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.394 DEBUG:boto:Params: {} 2022-09-03T05:30:13.395 DEBUG:boto:Token: None 2022-09-03T05:30:13.395 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.396 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6yedv6QRp6Yb93709MB9VbijCCk= 2022-09-03T05:30:13.397 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6yedv6QRp6Yb93709MB9VbijCCk=', 'Content-Length': '0'} 2022-09-03T05:30:13.399 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004164d8be02112a98-006312e665-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.400 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.400 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.401 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.402 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.402 DEBUG:boto:Method: HEAD 2022-09-03T05:30:13.403 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.403 DEBUG:boto:Data: 2022-09-03T05:30:13.404 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.404 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.405 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.405 DEBUG:boto:Params: {} 2022-09-03T05:30:13.406 DEBUG:boto:Token: None 2022-09-03T05:30:13.407 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.408 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UgBvvgg2zTkKNtf6YK1XYkB9Vv8= 2022-09-03T05:30:13.408 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UgBvvgg2zTkKNtf6YK1XYkB9Vv8=', 'Content-Length': '0'} 2022-09-03T05:30:13.410 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000072ed029980d90aff-006312e665-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.411 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.412 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.412 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.413 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.414 DEBUG:boto:Method: HEAD 2022-09-03T05:30:13.414 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.415 DEBUG:boto:Data: 2022-09-03T05:30:13.416 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.416 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.417 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.417 DEBUG:boto:Params: {} 2022-09-03T05:30:13.418 DEBUG:boto:Token: None 2022-09-03T05:30:13.419 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.419 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UgBvvgg2zTkKNtf6YK1XYkB9Vv8= 2022-09-03T05:30:13.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UgBvvgg2zTkKNtf6YK1XYkB9Vv8=', 'Content-Length': '0'} 2022-09-03T05:30:13.422 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008c940a2063fbeaa2-006312e665-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.423 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:30:13.424 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.424 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.425 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.426 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.426 DEBUG:boto:Method: GET 2022-09-03T05:30:13.427 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.427 DEBUG:boto:Data: 2022-09-03T05:30:13.428 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.429 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.429 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.430 DEBUG:boto:Params: {} 2022-09-03T05:30:13.431 DEBUG:boto:Token: None 2022-09-03T05:30:13.431 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.432 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6yedv6QRp6Yb93709MB9VbijCCk= 2022-09-03T05:30:13.433 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6yedv6QRp6Yb93709MB9VbijCCk=', 'Content-Length': '0'} 2022-09-03T05:30:13.436 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006f868fd1987f81b2-006312e665-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.437 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.438 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.439 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.439 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.440 DEBUG:boto:Method: GET 2022-09-03T05:30:13.441 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.442 DEBUG:boto:Data: 2022-09-03T05:30:13.443 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.443 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.444 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.444 DEBUG:boto:Params: {} 2022-09-03T05:30:13.446 DEBUG:boto:Token: None 2022-09-03T05:30:13.447 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.448 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6yedv6QRp6Yb93709MB9VbijCCk= 2022-09-03T05:30:13.449 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6yedv6QRp6Yb93709MB9VbijCCk=', 'Content-Length': '0'} 2022-09-03T05:30:13.452 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005ac38f23cec8301a-006312e665-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.497 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:13.497 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:13.498 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:13.499 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:13.499 DEBUG:boto:Method: GET 2022-09-03T05:30:13.500 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:13.500 DEBUG:boto:Data: 2022-09-03T05:30:13.501 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.501 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.502 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.503 DEBUG:boto:Params: {} 2022-09-03T05:30:13.503 DEBUG:boto:Token: None 2022-09-03T05:30:13.504 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/ 2022-09-03T05:30:13.504 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8= 2022-09-03T05:30:13.505 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8=', 'Content-Length': '0'} 2022-09-03T05:30:13.508 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004f907b556ee1bf6c-006312e665-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.509 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:13.509 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:13.510 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:13.511 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:13.511 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:13.512 DEBUG:boto:Method: GET 2022-09-03T05:30:13.513 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:13.513 DEBUG:boto:Data: 2022-09-03T05:30:13.514 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.514 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.515 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.515 DEBUG:boto:Params: {} 2022-09-03T05:30:13.516 DEBUG:boto:Token: None 2022-09-03T05:30:13.516 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/ 2022-09-03T05:30:13.517 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8= 2022-09-03T05:30:13.518 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8=', 'Content-Length': '0'} 2022-09-03T05:30:13.520 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009f160cf84505580e-006312e665-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.521 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:13.522 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.522 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.523 DEBUG:boto:Method: HEAD 2022-09-03T05:30:13.523 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.524 DEBUG:boto:Data: 2022-09-03T05:30:13.524 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.525 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.525 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.526 DEBUG:boto:Params: {} 2022-09-03T05:30:13.527 DEBUG:boto:Token: None 2022-09-03T05:30:13.527 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.528 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UwjOHMN1S4IvNfJh6yqwzhtH+kE= 2022-09-03T05:30:13.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UwjOHMN1S4IvNfJh6yqwzhtH+kE=', 'Content-Length': '0'} 2022-09-03T05:30:13.531 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000068d139e5dd1d223e-006312e665-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.532 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.532 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.533 DEBUG:boto:Method: HEAD 2022-09-03T05:30:13.534 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.534 DEBUG:boto:Data: 2022-09-03T05:30:13.535 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.535 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.536 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.536 DEBUG:boto:Params: {} 2022-09-03T05:30:13.537 DEBUG:boto:Token: None 2022-09-03T05:30:13.539 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.540 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UwjOHMN1S4IvNfJh6yqwzhtH+kE= 2022-09-03T05:30:13.541 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UwjOHMN1S4IvNfJh6yqwzhtH+kE=', 'Content-Length': '0'} 2022-09-03T05:30:13.545 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008293face78207cac-006312e665-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.545 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:30:13.546 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.547 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.547 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.548 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.548 DEBUG:boto:Method: GET 2022-09-03T05:30:13.549 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.549 DEBUG:boto:Data: 2022-09-03T05:30:13.550 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.551 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.552 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.553 DEBUG:boto:Params: {} 2022-09-03T05:30:13.554 DEBUG:boto:Token: None 2022-09-03T05:30:13.555 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.556 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6yedv6QRp6Yb93709MB9VbijCCk= 2022-09-03T05:30:13.557 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6yedv6QRp6Yb93709MB9VbijCCk=', 'Content-Length': '0'} 2022-09-03T05:30:13.560 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a396f62a899d9cc6-006312e665-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.602 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.603 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:30:13.603 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.604 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.604 DEBUG:boto:Method: GET 2022-09-03T05:30:13.605 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.605 DEBUG:boto:Data: 2022-09-03T05:30:13.606 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.607 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.607 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.608 DEBUG:boto:Params: {} 2022-09-03T05:30:13.608 DEBUG:boto:Token: None 2022-09-03T05:30:13.609 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:30:13.610 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6yedv6QRp6Yb93709MB9VbijCCk= 2022-09-03T05:30:13.610 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6yedv6QRp6Yb93709MB9VbijCCk=', 'Content-Length': '0'} 2022-09-03T05:30:13.613 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000012575ad500bb9508-006312e665-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.614 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:13.615 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:13.616 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:13.616 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:13.618 DEBUG:boto:Method: GET 2022-09-03T05:30:13.619 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:13.619 DEBUG:boto:Data: 2022-09-03T05:30:13.620 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.621 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.621 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.622 DEBUG:boto:Params: {} 2022-09-03T05:30:13.622 DEBUG:boto:Token: None 2022-09-03T05:30:13.623 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/ 2022-09-03T05:30:13.623 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8= 2022-09-03T05:30:13.624 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8=', 'Content-Length': '0'} 2022-09-03T05:30:13.627 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000041c6dca4c6653f1d-006312e665-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.628 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:13.629 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:13.629 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:13.630 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:13.630 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:13.631 DEBUG:boto:Method: GET 2022-09-03T05:30:13.631 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:30:13.632 DEBUG:boto:Data: 2022-09-03T05:30:13.632 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.633 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.633 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.634 DEBUG:boto:Params: {} 2022-09-03T05:30:13.635 DEBUG:boto:Token: None 2022-09-03T05:30:13.636 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/ 2022-09-03T05:30:13.636 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8= 2022-09-03T05:30:13.637 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8=', 'Content-Length': '0'} 2022-09-03T05:30:13.640 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c333c88f6c9c7f0c-006312e665-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.640 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:13.641 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:30:13.642 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.642 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.643 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.643 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.644 DEBUG:boto:Method: GET 2022-09-03T05:30:13.644 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.645 DEBUG:boto:Data: 2022-09-03T05:30:13.645 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.646 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.647 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.647 DEBUG:boto:Params: {} 2022-09-03T05:30:13.648 DEBUG:boto:Token: None 2022-09-03T05:30:13.648 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.649 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bPNWVgmSNj7e2kZxjhJ4c6CW6gc= 2022-09-03T05:30:13.650 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bPNWVgmSNj7e2kZxjhJ4c6CW6gc=', 'Content-Length': '0'} 2022-09-03T05:30:13.652 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d5fa2c315ef186c8-006312e665-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.654 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.654 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.655 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.655 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.656 DEBUG:boto:Method: GET 2022-09-03T05:30:13.656 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.657 DEBUG:boto:Data: 2022-09-03T05:30:13.657 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.658 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.658 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.659 DEBUG:boto:Params: {} 2022-09-03T05:30:13.660 DEBUG:boto:Token: None 2022-09-03T05:30:13.660 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.661 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bPNWVgmSNj7e2kZxjhJ4c6CW6gc= 2022-09-03T05:30:13.661 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bPNWVgmSNj7e2kZxjhJ4c6CW6gc=', 'Content-Length': '0'} 2022-09-03T05:30:13.663 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000037a6bd4969101020-006312e665-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.664 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.665 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.665 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.666 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.667 DEBUG:boto:Method: HEAD 2022-09-03T05:30:13.668 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.668 DEBUG:boto:Data: 2022-09-03T05:30:13.669 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.669 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.670 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.670 DEBUG:boto:Params: {} 2022-09-03T05:30:13.671 DEBUG:boto:Token: None 2022-09-03T05:30:13.672 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.673 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ksSLb8nBH/Fnvjou4QjK3RJcHx4= 2022-09-03T05:30:13.674 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ksSLb8nBH/Fnvjou4QjK3RJcHx4=', 'Content-Length': '0'} 2022-09-03T05:30:13.682 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b2e9aac4c9f0f0a7-006312e665-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.683 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.684 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.684 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.685 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.686 DEBUG:boto:Method: HEAD 2022-09-03T05:30:13.687 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.687 DEBUG:boto:Data: 2022-09-03T05:30:13.688 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.689 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.689 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.690 DEBUG:boto:Params: {} 2022-09-03T05:30:13.691 DEBUG:boto:Token: None 2022-09-03T05:30:13.691 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.692 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ksSLb8nBH/Fnvjou4QjK3RJcHx4= 2022-09-03T05:30:13.693 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ksSLb8nBH/Fnvjou4QjK3RJcHx4=', 'Content-Length': '0'} 2022-09-03T05:30:13.696 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007dfd3629fdeaef9d-006312e665-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.697 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:30:13.698 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.699 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.699 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.700 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.701 DEBUG:boto:Method: GET 2022-09-03T05:30:13.701 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.702 DEBUG:boto:Data: 2022-09-03T05:30:13.702 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.703 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.704 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.705 DEBUG:boto:Params: {} 2022-09-03T05:30:13.706 DEBUG:boto:Token: None 2022-09-03T05:30:13.707 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.708 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bPNWVgmSNj7e2kZxjhJ4c6CW6gc= 2022-09-03T05:30:13.709 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bPNWVgmSNj7e2kZxjhJ4c6CW6gc=', 'Content-Length': '0'} 2022-09-03T05:30:13.712 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009ab7f7655cc8de15-006312e665-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.754 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.755 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.755 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.756 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.757 DEBUG:boto:Method: GET 2022-09-03T05:30:13.757 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.758 DEBUG:boto:Data: 2022-09-03T05:30:13.759 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.759 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.760 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.761 DEBUG:boto:Params: {} 2022-09-03T05:30:13.762 DEBUG:boto:Token: None 2022-09-03T05:30:13.763 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.764 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bPNWVgmSNj7e2kZxjhJ4c6CW6gc= 2022-09-03T05:30:13.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bPNWVgmSNj7e2kZxjhJ4c6CW6gc=', 'Content-Length': '0'} 2022-09-03T05:30:13.766 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008b2931f04e7899a8-006312e665-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.767 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:13.768 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:13.768 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:13.769 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:13.769 DEBUG:boto:Method: GET 2022-09-03T05:30:13.770 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:13.770 DEBUG:boto:Data: 2022-09-03T05:30:13.771 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.772 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.772 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.773 DEBUG:boto:Params: {} 2022-09-03T05:30:13.773 DEBUG:boto:Token: None 2022-09-03T05:30:13.774 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/ 2022-09-03T05:30:13.774 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8= 2022-09-03T05:30:13.775 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8=', 'Content-Length': '0'} 2022-09-03T05:30:13.778 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000084abaf9bb47fa913-006312e665-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.778 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:13.779 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:13.780 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:13.780 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:13.781 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:13.781 DEBUG:boto:Method: GET 2022-09-03T05:30:13.782 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:13.783 DEBUG:boto:Data: 2022-09-03T05:30:13.783 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.784 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.784 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.785 DEBUG:boto:Params: {} 2022-09-03T05:30:13.785 DEBUG:boto:Token: None 2022-09-03T05:30:13.786 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/ 2022-09-03T05:30:13.786 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8= 2022-09-03T05:30:13.787 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8=', 'Content-Length': '0'} 2022-09-03T05:30:13.790 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e0c70da8d58ae4e7-006312e665-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.830 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:13.831 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.832 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.833 DEBUG:boto:Method: HEAD 2022-09-03T05:30:13.834 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.835 DEBUG:boto:Data: 2022-09-03T05:30:13.836 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.837 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.837 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.838 DEBUG:boto:Params: {} 2022-09-03T05:30:13.839 DEBUG:boto:Token: None 2022-09-03T05:30:13.839 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.840 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V2WsU3EDIordMiQ2EitKh8rkQfE= 2022-09-03T05:30:13.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V2WsU3EDIordMiQ2EitKh8rkQfE=', 'Content-Length': '0'} 2022-09-03T05:30:13.843 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007a9d32fa6288b1ec-006312e665-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.844 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.845 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.845 DEBUG:boto:Method: HEAD 2022-09-03T05:30:13.846 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.846 DEBUG:boto:Data: 2022-09-03T05:30:13.847 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.847 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.848 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.848 DEBUG:boto:Params: {} 2022-09-03T05:30:13.849 DEBUG:boto:Token: None 2022-09-03T05:30:13.849 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.850 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V2WsU3EDIordMiQ2EitKh8rkQfE= 2022-09-03T05:30:13.851 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V2WsU3EDIordMiQ2EitKh8rkQfE=', 'Content-Length': '0'} 2022-09-03T05:30:13.853 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000068c1fbb90be4222c-006312e665-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.855 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:30:13.856 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.857 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.857 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.858 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.859 DEBUG:boto:Method: GET 2022-09-03T05:30:13.859 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.860 DEBUG:boto:Data: 2022-09-03T05:30:13.860 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.861 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.862 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.862 DEBUG:boto:Params: {} 2022-09-03T05:30:13.863 DEBUG:boto:Token: None 2022-09-03T05:30:13.864 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.864 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bPNWVgmSNj7e2kZxjhJ4c6CW6gc= 2022-09-03T05:30:13.865 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bPNWVgmSNj7e2kZxjhJ4c6CW6gc=', 'Content-Length': '0'} 2022-09-03T05:30:13.867 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000042107ec489a47ca9-006312e665-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.868 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.869 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:30:13.869 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.870 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.870 DEBUG:boto:Method: GET 2022-09-03T05:30:13.871 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.872 DEBUG:boto:Data: 2022-09-03T05:30:13.872 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.873 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.873 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.874 DEBUG:boto:Params: {} 2022-09-03T05:30:13.874 DEBUG:boto:Token: None 2022-09-03T05:30:13.875 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:30:13.876 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bPNWVgmSNj7e2kZxjhJ4c6CW6gc= 2022-09-03T05:30:13.877 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bPNWVgmSNj7e2kZxjhJ4c6CW6gc=', 'Content-Length': '0'} 2022-09-03T05:30:13.879 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006da0c286f40508b7-006312e665-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.880 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:13.880 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:13.881 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:13.881 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:13.882 DEBUG:boto:Method: GET 2022-09-03T05:30:13.883 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:13.883 DEBUG:boto:Data: 2022-09-03T05:30:13.884 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.884 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:30:13.885 DEBUG:boto:Port: 8001 2022-09-03T05:30:13.885 DEBUG:boto:Params: {} 2022-09-03T05:30:13.886 DEBUG:boto:Token: None 2022-09-03T05:30:13.886 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/ 2022-09-03T05:30:13.887 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8= 2022-09-03T05:30:13.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8=', 'Content-Length': '0'} 2022-09-03T05:30:13.890 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008f5b6f0f3a4525ec-006312e665-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.891 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:13.892 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:30:13.893 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:30:13.893 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:13.894 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:13.894 DEBUG:boto:Method: GET 2022-09-03T05:30:13.895 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:30:13.896 DEBUG:boto:Data: 2022-09-03T05:30:13.896 DEBUG:boto:Headers: {} 2022-09-03T05:30:13.897 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:13.898 DEBUG:boto:Port: 8000 2022-09-03T05:30:13.899 DEBUG:boto:Params: {} 2022-09-03T05:30:13.900 DEBUG:boto:Token: None 2022-09-03T05:30:13.901 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:13 GMT /dsgvcy-17/ 2022-09-03T05:30:13.901 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8= 2022-09-03T05:30:13.902 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:na1AlrcQC/fZ7ft00XdNGuYp9d8=', 'Content-Length': '0'} 2022-09-03T05:30:13.905 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000058af74736b516db-006312e665-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:13.907 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:13.908 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-17 zones={test-zone3, test-zone2} 2022-09-03T05:30:13.909 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-18 2022-09-03T05:30:13.910 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:30:13.911 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:30:13.912 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-18 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:30:13.981 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:30:13.981 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:30:14.031 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:30:14.033+0000 7f4088ca5600 1 waiting for incremental sync to catch up: 2022-09-03T05:30:14.032 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000010.24.5, , 00000000003.13.9, , 00000000006.15.10, , , , , ] 2022-09-03T05:30:14.032 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000010.24.5, , 00000000003.13.9, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:19.033 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:30:19.035+0000 7f4088ca5600 1 waiting for incremental sync to catch up: 2022-09-03T05:30:19.033 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000010.24.5, , 00000000003.13.9, , 00000000006.15.10, , , , , ] 2022-09-03T05:30:19.033 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000010.24.5, , 00000000003.13.9, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:24.034 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:30:24.036+0000 7f4088ca5600 1 waiting for incremental sync to catch up: 2022-09-03T05:30:24.035 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000010.24.5, , 00000000003.13.9, , 00000000006.15.10, , , , , ] 2022-09-03T05:30:24.035 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000010.24.5, , 00000000003.13.9, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:29.036 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:30:29.038+0000 7f4088ca5600 1 bucket sync caught up with source: 2022-09-03T05:30:29.036 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000010.24.5, , 00000000003.13.9, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:29.036 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000010.24.5, , 00000000003.13.9, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:29.037 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:30:29.038+0000 7f4088ca5600 0 bucket checkpoint complete 2022-09-03T05:30:29.051 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-18 2022-09-03T05:30:29.052 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:30:29.053 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:30:29.055 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-18 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:30:29.119 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:30:29.120 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:30:29.182 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:30:29.182+0000 7f9f9dd25600 1 bucket sync caught up with source: 2022-09-03T05:30:29.183 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000010.24.5, , 00000000003.13.9, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:29.183 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000010.24.5, , 00000000003.13.9, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:29.184 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:30:29.182+0000 7f9f9dd25600 0 bucket checkpoint complete 2022-09-03T05:30:29.200 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-18 2022-09-03T05:30:29.201 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:30:29.201 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:30:29.202 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-18 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:30:29.265 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:30:29.266 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:30:29.302 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:30:29.304+0000 7f3283b4c600 1 waiting for incremental sync to catch up: 2022-09-03T05:30:29.302 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000010.24.5, , 00000000001.10.5, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:29.303 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000010.24.5, , 00000000003.13.9, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:34.304 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:30:34.306+0000 7f3283b4c600 1 waiting for incremental sync to catch up: 2022-09-03T05:30:34.304 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000010.24.5, , 00000000001.10.5, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:34.305 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000010.24.5, , 00000000003.13.9, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:39.305 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:30:39.307+0000 7f3283b4c600 1 waiting for incremental sync to catch up: 2022-09-03T05:30:39.306 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000010.24.5, , 00000000001.10.5, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:39.306 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000010.24.5, , 00000000003.13.9, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:44.307 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:30:44.309+0000 7f3283b4c600 1 waiting for incremental sync to catch up: 2022-09-03T05:30:44.307 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000010.24.5, , 00000000001.10.5, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:44.307 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000010.24.5, , 00000000003.13.9, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:49.312 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:30:49.309+0000 7f3283b4c600 1 waiting for incremental sync to catch up: 2022-09-03T05:30:49.312 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000010.24.5, , 00000000001.10.5, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:49.313 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000010.24.5, , 00000000003.13.9, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:54.310 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:30:54.312+0000 7f3283b4c600 1 waiting for incremental sync to catch up: 2022-09-03T05:30:54.311 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000010.24.5, , 00000000001.10.5, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:54.311 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000010.24.5, , 00000000003.13.9, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:59.311 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:30:59.313+0000 7f3283b4c600 1 bucket sync caught up with source: 2022-09-03T05:30:59.312 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000010.24.5, , 00000000003.13.9, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:59.312 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000010.24.5, , 00000000003.13.9, , 00000000007.18.9, , , , , ] 2022-09-03T05:30:59.313 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:30:59.313+0000 7f3283b4c600 0 bucket checkpoint complete 2022-09-03T05:30:59.327 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-18 2022-09-03T05:30:59.328 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:30:59.329 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:30:59.330 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-18 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:30:59.393 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:30:59.394 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:30:59.454 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:30:59.456+0000 7fc87dee2600 1 bucket sync caught up with source: 2022-09-03T05:30:59.455 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000003.24.9, , 00000000010.43.5, , 00000000007.31.9, , , , , ] 2022-09-03T05:30:59.455 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000003.24.9, , 00000000010.43.5, , 00000000007.31.9, , , , , ] 2022-09-03T05:30:59.456 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:30:59.456+0000 7fc87dee2600 0 bucket checkpoint complete 2022-09-03T05:30:59.478 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-18 2022-09-03T05:30:59.479 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:30:59.480 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:30:59.480 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-18 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:30:59.537 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:30:59.537 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:30:59.587 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:30:59.589+0000 7fc75f025600 1 bucket sync caught up with source: 2022-09-03T05:30:59.588 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000003.24.9, , 00000000010.43.5, , 00000000007.31.9, , , , , ] 2022-09-03T05:30:59.588 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000003.24.9, , 00000000010.43.5, , 00000000007.31.9, , , , , ] 2022-09-03T05:30:59.588 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:30:59.589+0000 7fc75f025600 0 bucket checkpoint complete 2022-09-03T05:30:59.599 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-18 2022-09-03T05:30:59.600 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-18 2022-09-03T05:30:59.601 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:30:59.601 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:30:59.602 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-18 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:30:59.658 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:30:59.659 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:30:59.692 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:30:59.693+0000 7f46f9994600 1 bucket sync caught up with source: 2022-09-03T05:30:59.692 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.17.9, , 00000000003.34.9, , 00000000015.44.5, , , , , ] 2022-09-03T05:30:59.693 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.17.9, , 00000000003.34.9, , 00000000015.44.5, , , , , ] 2022-09-03T05:30:59.693 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:30:59.693+0000 7f46f9994600 0 bucket checkpoint complete 2022-09-03T05:30:59.704 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-18 2022-09-03T05:30:59.706 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:30:59.715 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:30:59.716 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-18 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:30:59.783 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:30:59.784 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:30:59.818 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:30:59.820+0000 7f6443a6a600 1 bucket sync caught up with source: 2022-09-03T05:30:59.819 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000004.17.9, , 00000000003.34.9, , 00000000015.44.5, , , , , ] 2022-09-03T05:30:59.819 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000004.17.9, , 00000000003.34.9, , 00000000015.44.5, , , , , ] 2022-09-03T05:30:59.820 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:30:59.820+0000 7f6443a6a600 0 bucket checkpoint complete 2022-09-03T05:30:59.836 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-18 2022-09-03T05:30:59.836 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-18 zones={test-zone2, test-zone1} 2022-09-03T05:30:59.837 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:30:59.838 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:30:59.838 DEBUG:boto:Method: HEAD 2022-09-03T05:30:59.839 DEBUG:boto:Path: /dsgvcy-18/ 2022-09-03T05:30:59.839 DEBUG:boto:Data: 2022-09-03T05:30:59.840 DEBUG:boto:Headers: {} 2022-09-03T05:30:59.840 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:59.841 DEBUG:boto:Port: 8000 2022-09-03T05:30:59.842 DEBUG:boto:Params: {} 2022-09-03T05:30:59.843 DEBUG:boto:Token: None 2022-09-03T05:30:59.845 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:59 GMT /dsgvcy-18/ 2022-09-03T05:30:59.845 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GzI/FBZ+HPRpYqaNlrtNPFxTFZ8= 2022-09-03T05:30:59.846 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GzI/FBZ+HPRpYqaNlrtNPFxTFZ8=', 'Content-Length': '0'} 2022-09-03T05:30:59.849 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000404475809bec0129-006312e693-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:30:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:59.851 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:30:59.852 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:30:59.853 DEBUG:boto:Method: HEAD 2022-09-03T05:30:59.854 DEBUG:boto:Path: /dsgvcy-18/ 2022-09-03T05:30:59.855 DEBUG:boto:Data: 2022-09-03T05:30:59.855 DEBUG:boto:Headers: {} 2022-09-03T05:30:59.856 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:59.857 DEBUG:boto:Port: 8000 2022-09-03T05:30:59.858 DEBUG:boto:Params: {} 2022-09-03T05:30:59.858 DEBUG:boto:Token: None 2022-09-03T05:30:59.859 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:59 GMT /dsgvcy-18/ 2022-09-03T05:30:59.860 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GzI/FBZ+HPRpYqaNlrtNPFxTFZ8= 2022-09-03T05:30:59.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GzI/FBZ+HPRpYqaNlrtNPFxTFZ8=', 'Content-Length': '0'} 2022-09-03T05:30:59.863 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004e22f4cc7960bf5b-006312e693-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:30:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:59.864 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:30:59.865 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:30:59.865 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:30:59.866 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:30:59.867 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:30:59.868 DEBUG:boto:Method: GET 2022-09-03T05:30:59.868 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:30:59.869 DEBUG:boto:Data: 2022-09-03T05:30:59.869 DEBUG:boto:Headers: {} 2022-09-03T05:30:59.870 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:59.870 DEBUG:boto:Port: 8000 2022-09-03T05:30:59.871 DEBUG:boto:Params: {} 2022-09-03T05:30:59.872 DEBUG:boto:Token: None 2022-09-03T05:30:59.872 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:59 GMT /dsgvcy-18/?versions 2022-09-03T05:30:59.873 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:np6oerIeZ2piVAeckPQjOlYvZxM= 2022-09-03T05:30:59.874 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:np6oerIeZ2piVAeckPQjOlYvZxM=', 'Content-Length': '0'} 2022-09-03T05:30:59.877 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000401200203b45b190-006312e693-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:59.878 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:59.880 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:30:59.880 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:30:59.881 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:30:59.881 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:30:59.882 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:30:59.883 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:30:59.883 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:30:59.884 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:30:59.885 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:30:59.886 DEBUG:boto:Method: GET 2022-09-03T05:30:59.887 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:30:59.888 DEBUG:boto:Data: 2022-09-03T05:30:59.888 DEBUG:boto:Headers: {} 2022-09-03T05:30:59.889 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:59.890 DEBUG:boto:Port: 8000 2022-09-03T05:30:59.891 DEBUG:boto:Params: {} 2022-09-03T05:30:59.892 DEBUG:boto:Token: None 2022-09-03T05:30:59.893 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:59 GMT /dsgvcy-18/?versions 2022-09-03T05:30:59.894 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:np6oerIeZ2piVAeckPQjOlYvZxM= 2022-09-03T05:30:59.895 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:np6oerIeZ2piVAeckPQjOlYvZxM=', 'Content-Length': '0'} 2022-09-03T05:30:59.898 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008c10ba078c09f7e1-006312e693-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:59.899 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:59.900 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:30:59.901 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:30:59.901 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:30:59.902 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:30:59.902 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:30:59.903 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:30:59.903 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:30:59.904 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:30:59.904 DEBUG:boto:Method: GET 2022-09-03T05:30:59.905 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:30:59.906 DEBUG:boto:Data: 2022-09-03T05:30:59.906 DEBUG:boto:Headers: {} 2022-09-03T05:30:59.907 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:59.907 DEBUG:boto:Port: 8000 2022-09-03T05:30:59.908 DEBUG:boto:Params: {} 2022-09-03T05:30:59.909 DEBUG:boto:Token: None 2022-09-03T05:30:59.909 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:59 GMT /dsgvcy-18/?versions 2022-09-03T05:30:59.910 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:np6oerIeZ2piVAeckPQjOlYvZxM= 2022-09-03T05:30:59.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:np6oerIeZ2piVAeckPQjOlYvZxM=', 'Content-Length': '0'} 2022-09-03T05:30:59.919 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001b5adfa5e8b6f5b1-006312e693-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:59.921 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:59.922 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:30:59.923 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:30:59.923 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:30:59.924 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:30:59.924 DEBUG:boto:Method: GET 2022-09-03T05:30:59.925 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:30:59.925 DEBUG:boto:Data: 2022-09-03T05:30:59.926 DEBUG:boto:Headers: {} 2022-09-03T05:30:59.926 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:59.927 DEBUG:boto:Port: 8000 2022-09-03T05:30:59.927 DEBUG:boto:Params: {} 2022-09-03T05:30:59.928 DEBUG:boto:Token: None 2022-09-03T05:30:59.928 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:59 GMT /dsgvcy-18/?versions 2022-09-03T05:30:59.929 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:np6oerIeZ2piVAeckPQjOlYvZxM= 2022-09-03T05:30:59.930 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:np6oerIeZ2piVAeckPQjOlYvZxM=', 'Content-Length': '0'} 2022-09-03T05:30:59.934 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e5606cf4a5a45997-006312e693-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:30:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:59.935 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:30:59.936 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:30:59.937 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:30:59.938 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:30:59.939 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:30:59.940 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:30:59.940 DEBUG:boto:Method: GET 2022-09-03T05:30:59.941 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:30:59.942 DEBUG:boto:Data: 2022-09-03T05:30:59.942 DEBUG:boto:Headers: {} 2022-09-03T05:30:59.943 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:59.943 DEBUG:boto:Port: 8000 2022-09-03T05:30:59.944 DEBUG:boto:Params: {} 2022-09-03T05:30:59.945 DEBUG:boto:Token: None 2022-09-03T05:30:59.945 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:59 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:30:59.946 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zDMVAkmEAEvo3gM3cm6duo4a2eQ= 2022-09-03T05:30:59.947 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zDMVAkmEAEvo3gM3cm6duo4a2eQ=', 'Content-Length': '0'} 2022-09-03T05:30:59.949 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000779dcbb052d9f575-006312e693-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:59.950 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:30:59.951 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:30:59.951 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:30:59.952 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:30:59.952 DEBUG:boto:Method: GET 2022-09-03T05:30:59.953 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:30:59.954 DEBUG:boto:Data: 2022-09-03T05:30:59.954 DEBUG:boto:Headers: {} 2022-09-03T05:30:59.955 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:59.955 DEBUG:boto:Port: 8000 2022-09-03T05:30:59.956 DEBUG:boto:Params: {} 2022-09-03T05:30:59.956 DEBUG:boto:Token: None 2022-09-03T05:30:59.957 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:59 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:30:59.958 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zDMVAkmEAEvo3gM3cm6duo4a2eQ= 2022-09-03T05:30:59.958 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zDMVAkmEAEvo3gM3cm6duo4a2eQ=', 'Content-Length': '0'} 2022-09-03T05:30:59.961 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000008286828d0fb2352f-006312e693-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:59.962 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:30:59.965 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:30:59.965 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:30:59.966 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:30:59.966 DEBUG:boto:Method: HEAD 2022-09-03T05:30:59.967 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:30:59.967 DEBUG:boto:Data: 2022-09-03T05:30:59.968 DEBUG:boto:Headers: {} 2022-09-03T05:30:59.968 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:59.969 DEBUG:boto:Port: 8000 2022-09-03T05:30:59.969 DEBUG:boto:Params: {} 2022-09-03T05:30:59.970 DEBUG:boto:Token: None 2022-09-03T05:30:59.971 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:59 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:30:59.972 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Dcp3kCORwuUYYxM1bgQlQodl0y0= 2022-09-03T05:30:59.972 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Dcp3kCORwuUYYxM1bgQlQodl0y0=', 'Content-Length': '0'} 2022-09-03T05:30:59.974 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000002e9c90b99751753a-006312e693-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:59.975 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:30:59.975 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:30:59.976 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:30:59.976 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:30:59.977 DEBUG:boto:Method: HEAD 2022-09-03T05:30:59.977 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:30:59.978 DEBUG:boto:Data: 2022-09-03T05:30:59.978 DEBUG:boto:Headers: {} 2022-09-03T05:30:59.979 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:30:59.979 DEBUG:boto:Port: 8000 2022-09-03T05:30:59.980 DEBUG:boto:Params: {} 2022-09-03T05:30:59.981 DEBUG:boto:Token: None 2022-09-03T05:30:59.981 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:30:59 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:30:59.982 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Dcp3kCORwuUYYxM1bgQlQodl0y0= 2022-09-03T05:30:59.982 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Dcp3kCORwuUYYxM1bgQlQodl0y0=', 'Content-Length': '0'} 2022-09-03T05:30:59.985 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000fe5fda421468f5c8-006312e693-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:30:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:30:59.986 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:30:59.986 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:30:59.987 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:30:59.987 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:30:59.988 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:30:59.988 DEBUG:boto:Method: GET 2022-09-03T05:30:59.989 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:30:59.989 DEBUG:boto:Data: 2022-09-03T05:30:59.990 DEBUG:boto:Headers: {} 2022-09-03T05:30:59.991 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:30:59.992 DEBUG:boto:Port: 8000 2022-09-03T05:30:59.993 DEBUG:boto:Params: {} 2022-09-03T05:30:59.993 DEBUG:boto:Token: None 2022-09-03T05:30:59.994 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:30:59 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:30:59.996 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zDMVAkmEAEvo3gM3cm6duo4a2eQ= 2022-09-03T05:30:59.997 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:30:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zDMVAkmEAEvo3gM3cm6duo4a2eQ=', 'Content-Length': '0'} 2022-09-03T05:30:59.999 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000494ee9689241437f-006312e694-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.000 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:00.003 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:00.003 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:00.004 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:00.004 DEBUG:boto:Method: GET 2022-09-03T05:31:00.005 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:00.005 DEBUG:boto:Data: 2022-09-03T05:31:00.006 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.006 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.007 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.007 DEBUG:boto:Params: {} 2022-09-03T05:31:00.008 DEBUG:boto:Token: None 2022-09-03T05:31:00.009 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:00.009 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fTEnU3BHfvRuqkayWbq4Ot58ulM= 2022-09-03T05:31:00.010 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fTEnU3BHfvRuqkayWbq4Ot58ulM=', 'Content-Length': '0'} 2022-09-03T05:31:00.012 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000b599ddfa3b142160-006312e694-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.013 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:00.014 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:00.015 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:00.016 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:00.016 DEBUG:boto:Method: GET 2022-09-03T05:31:00.017 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:31:00.017 DEBUG:boto:Data: 2022-09-03T05:31:00.018 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.019 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.019 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.020 DEBUG:boto:Params: {} 2022-09-03T05:31:00.020 DEBUG:boto:Token: None 2022-09-03T05:31:00.021 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/ 2022-09-03T05:31:00.022 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE= 2022-09-03T05:31:00.023 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE=', 'Content-Length': '0'} 2022-09-03T05:31:00.025 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000210ada121abd74e-006312e694-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.070 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:00.072 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:00.073 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:00.074 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:00.075 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:00.075 DEBUG:boto:Method: GET 2022-09-03T05:31:00.076 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:31:00.076 DEBUG:boto:Data: 2022-09-03T05:31:00.077 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.077 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.078 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.079 DEBUG:boto:Params: {} 2022-09-03T05:31:00.079 DEBUG:boto:Token: None 2022-09-03T05:31:00.080 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/ 2022-09-03T05:31:00.082 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE= 2022-09-03T05:31:00.083 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE=', 'Content-Length': '0'} 2022-09-03T05:31:00.087 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000002c55f7bccec1de-006312e694-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.088 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:00.090 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:00.091 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:00.092 DEBUG:boto:Method: HEAD 2022-09-03T05:31:00.093 DEBUG:boto:Path: /dsgvcy-18/dummy 2022-09-03T05:31:00.093 DEBUG:boto:Data: 2022-09-03T05:31:00.094 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.095 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.096 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.097 DEBUG:boto:Params: {} 2022-09-03T05:31:00.098 DEBUG:boto:Token: None 2022-09-03T05:31:00.100 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/dummy 2022-09-03T05:31:00.101 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kTPAXLrSiPxYIWB1mkLdm9zkOAg= 2022-09-03T05:31:00.102 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kTPAXLrSiPxYIWB1mkLdm9zkOAg=', 'Content-Length': '0'} 2022-09-03T05:31:00.105 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000001e223a1e39a2c8e0-006312e694-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.107 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:00.108 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:00.109 DEBUG:boto:Method: HEAD 2022-09-03T05:31:00.112 DEBUG:boto:Path: /dsgvcy-18/dummy 2022-09-03T05:31:00.113 DEBUG:boto:Data: 2022-09-03T05:31:00.113 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.114 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.115 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.115 DEBUG:boto:Params: {} 2022-09-03T05:31:00.116 DEBUG:boto:Token: None 2022-09-03T05:31:00.117 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/dummy 2022-09-03T05:31:00.119 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kTPAXLrSiPxYIWB1mkLdm9zkOAg= 2022-09-03T05:31:00.120 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kTPAXLrSiPxYIWB1mkLdm9zkOAg=', 'Content-Length': '0'} 2022-09-03T05:31:00.123 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000760c72097df59493-006312e694-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.124 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:31:00.125 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:00.126 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:00.126 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:00.127 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:00.128 DEBUG:boto:Method: GET 2022-09-03T05:31:00.128 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:00.129 DEBUG:boto:Data: 2022-09-03T05:31:00.129 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.130 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.131 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.132 DEBUG:boto:Params: {} 2022-09-03T05:31:00.133 DEBUG:boto:Token: None 2022-09-03T05:31:00.134 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:00.135 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fTEnU3BHfvRuqkayWbq4Ot58ulM= 2022-09-03T05:31:00.137 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fTEnU3BHfvRuqkayWbq4Ot58ulM=', 'Content-Length': '0'} 2022-09-03T05:31:00.142 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000049dfbc27d7f5ec49-006312e694-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.143 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:00.144 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:00.145 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:00.146 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:00.147 DEBUG:boto:Method: GET 2022-09-03T05:31:00.147 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:00.148 DEBUG:boto:Data: 2022-09-03T05:31:00.149 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.150 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.151 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.151 DEBUG:boto:Params: {} 2022-09-03T05:31:00.153 DEBUG:boto:Token: None 2022-09-03T05:31:00.154 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:00.155 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fTEnU3BHfvRuqkayWbq4Ot58ulM= 2022-09-03T05:31:00.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fTEnU3BHfvRuqkayWbq4Ot58ulM=', 'Content-Length': '0'} 2022-09-03T05:31:00.159 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000d33c2e755b2ef7a0-006312e694-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.161 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:00.161 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:00.162 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:00.163 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:00.163 DEBUG:boto:Method: GET 2022-09-03T05:31:00.164 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:31:00.164 DEBUG:boto:Data: 2022-09-03T05:31:00.165 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.166 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.166 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.167 DEBUG:boto:Params: {} 2022-09-03T05:31:00.168 DEBUG:boto:Token: None 2022-09-03T05:31:00.169 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/ 2022-09-03T05:31:00.170 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE= 2022-09-03T05:31:00.170 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE=', 'Content-Length': '0'} 2022-09-03T05:31:00.173 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000214a6f863696742d-006312e694-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.174 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:00.176 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:00.177 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:00.178 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:00.179 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:00.180 DEBUG:boto:Method: GET 2022-09-03T05:31:00.181 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:31:00.182 DEBUG:boto:Data: 2022-09-03T05:31:00.183 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.184 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.185 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.187 DEBUG:boto:Params: {} 2022-09-03T05:31:00.188 DEBUG:boto:Token: None 2022-09-03T05:31:00.188 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/ 2022-09-03T05:31:00.189 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE= 2022-09-03T05:31:00.190 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE=', 'Content-Length': '0'} 2022-09-03T05:31:00.193 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000092b34ca3280df222-006312e694-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.238 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:00.240 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:31:00.240 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.241 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.242 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.243 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.244 DEBUG:boto:Method: GET 2022-09-03T05:31:00.245 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.246 DEBUG:boto:Data: 2022-09-03T05:31:00.247 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.249 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.250 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.251 DEBUG:boto:Params: {} 2022-09-03T05:31:00.252 DEBUG:boto:Token: None 2022-09-03T05:31:00.253 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.254 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tWOTOVXQDUSAkZEmGyjeYhCeWF0= 2022-09-03T05:31:00.255 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tWOTOVXQDUSAkZEmGyjeYhCeWF0=', 'Content-Length': '0'} 2022-09-03T05:31:00.259 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006b988aaca91011f8-006312e694-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.260 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.264 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.265 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.266 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.267 DEBUG:boto:Method: GET 2022-09-03T05:31:00.268 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.269 DEBUG:boto:Data: 2022-09-03T05:31:00.270 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.271 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.273 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.274 DEBUG:boto:Params: {} 2022-09-03T05:31:00.275 DEBUG:boto:Token: None 2022-09-03T05:31:00.276 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.277 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tWOTOVXQDUSAkZEmGyjeYhCeWF0= 2022-09-03T05:31:00.278 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tWOTOVXQDUSAkZEmGyjeYhCeWF0=', 'Content-Length': '0'} 2022-09-03T05:31:00.281 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000084b61be78cf06b9-006312e694-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.282 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.283 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.284 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.285 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.286 DEBUG:boto:Method: HEAD 2022-09-03T05:31:00.287 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.288 DEBUG:boto:Data: 2022-09-03T05:31:00.289 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.290 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.291 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.292 DEBUG:boto:Params: {} 2022-09-03T05:31:00.292 DEBUG:boto:Token: None 2022-09-03T05:31:00.293 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.294 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pv7nsWC6W2TGa0Li2DK6CA8+F5s= 2022-09-03T05:31:00.295 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pv7nsWC6W2TGa0Li2DK6CA8+F5s=', 'Content-Length': '0'} 2022-09-03T05:31:00.297 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000019df8275da5e744c-006312e694-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.298 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.298 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.299 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.299 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.300 DEBUG:boto:Method: HEAD 2022-09-03T05:31:00.300 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.301 DEBUG:boto:Data: 2022-09-03T05:31:00.301 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.302 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.303 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.303 DEBUG:boto:Params: {} 2022-09-03T05:31:00.304 DEBUG:boto:Token: None 2022-09-03T05:31:00.304 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.305 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pv7nsWC6W2TGa0Li2DK6CA8+F5s= 2022-09-03T05:31:00.305 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pv7nsWC6W2TGa0Li2DK6CA8+F5s=', 'Content-Length': '0'} 2022-09-03T05:31:00.308 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004366866b726d294e-006312e694-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.308 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:31:00.309 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.310 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.310 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.311 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.311 DEBUG:boto:Method: GET 2022-09-03T05:31:00.312 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.312 DEBUG:boto:Data: 2022-09-03T05:31:00.313 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.313 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.314 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.315 DEBUG:boto:Params: {} 2022-09-03T05:31:00.315 DEBUG:boto:Token: None 2022-09-03T05:31:00.316 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.317 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tWOTOVXQDUSAkZEmGyjeYhCeWF0= 2022-09-03T05:31:00.317 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tWOTOVXQDUSAkZEmGyjeYhCeWF0=', 'Content-Length': '0'} 2022-09-03T05:31:00.319 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000093ad07ce3b055036-006312e694-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.320 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.321 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.321 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.322 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.322 DEBUG:boto:Method: GET 2022-09-03T05:31:00.323 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.324 DEBUG:boto:Data: 2022-09-03T05:31:00.324 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.325 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.325 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.326 DEBUG:boto:Params: {} 2022-09-03T05:31:00.326 DEBUG:boto:Token: None 2022-09-03T05:31:00.327 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.328 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tWOTOVXQDUSAkZEmGyjeYhCeWF0= 2022-09-03T05:31:00.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tWOTOVXQDUSAkZEmGyjeYhCeWF0=', 'Content-Length': '0'} 2022-09-03T05:31:00.330 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fb0cbb54158e3793-006312e694-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.331 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:00.332 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:00.332 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:00.333 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:00.333 DEBUG:boto:Method: GET 2022-09-03T05:31:00.334 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:00.334 DEBUG:boto:Data: 2022-09-03T05:31:00.335 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.336 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.336 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.337 DEBUG:boto:Params: {} 2022-09-03T05:31:00.337 DEBUG:boto:Token: None 2022-09-03T05:31:00.338 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/ 2022-09-03T05:31:00.339 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE= 2022-09-03T05:31:00.339 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE=', 'Content-Length': '0'} 2022-09-03T05:31:00.342 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000081dd5e7a51580e03-006312e694-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.343 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:00.344 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:00.345 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:00.346 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:00.346 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:00.347 DEBUG:boto:Method: GET 2022-09-03T05:31:00.347 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:00.348 DEBUG:boto:Data: 2022-09-03T05:31:00.348 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.349 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.349 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.350 DEBUG:boto:Params: {} 2022-09-03T05:31:00.351 DEBUG:boto:Token: None 2022-09-03T05:31:00.351 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/ 2022-09-03T05:31:00.352 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE= 2022-09-03T05:31:00.353 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE=', 'Content-Length': '0'} 2022-09-03T05:31:00.355 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000097da4297952d2d98-006312e694-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.356 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:00.357 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.357 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.358 DEBUG:boto:Method: HEAD 2022-09-03T05:31:00.359 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.359 DEBUG:boto:Data: 2022-09-03T05:31:00.360 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.360 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.361 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.361 DEBUG:boto:Params: {} 2022-09-03T05:31:00.362 DEBUG:boto:Token: None 2022-09-03T05:31:00.363 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.364 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NNrJOAvps7JdbS2JSvTra3+lMEw= 2022-09-03T05:31:00.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NNrJOAvps7JdbS2JSvTra3+lMEw=', 'Content-Length': '0'} 2022-09-03T05:31:00.374 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000032074f44b3adade9-006312e694-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.376 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.377 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.378 DEBUG:boto:Method: HEAD 2022-09-03T05:31:00.379 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.380 DEBUG:boto:Data: 2022-09-03T05:31:00.381 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.382 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.383 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.384 DEBUG:boto:Params: {} 2022-09-03T05:31:00.385 DEBUG:boto:Token: None 2022-09-03T05:31:00.387 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.388 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NNrJOAvps7JdbS2JSvTra3+lMEw= 2022-09-03T05:31:00.389 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NNrJOAvps7JdbS2JSvTra3+lMEw=', 'Content-Length': '0'} 2022-09-03T05:31:00.391 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ef72e59082b31504-006312e694-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.393 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:31:00.393 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.394 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.394 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.395 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.396 DEBUG:boto:Method: GET 2022-09-03T05:31:00.396 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.397 DEBUG:boto:Data: 2022-09-03T05:31:00.397 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.398 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.399 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.399 DEBUG:boto:Params: {} 2022-09-03T05:31:00.400 DEBUG:boto:Token: None 2022-09-03T05:31:00.401 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.402 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tWOTOVXQDUSAkZEmGyjeYhCeWF0= 2022-09-03T05:31:00.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tWOTOVXQDUSAkZEmGyjeYhCeWF0=', 'Content-Length': '0'} 2022-09-03T05:31:00.407 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a4862af136c4729c-006312e694-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.454 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.455 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:00.456 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.457 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.459 DEBUG:boto:Method: GET 2022-09-03T05:31:00.460 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.461 DEBUG:boto:Data: 2022-09-03T05:31:00.462 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.463 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.464 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.465 DEBUG:boto:Params: {} 2022-09-03T05:31:00.466 DEBUG:boto:Token: None 2022-09-03T05:31:00.467 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:00.469 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tWOTOVXQDUSAkZEmGyjeYhCeWF0= 2022-09-03T05:31:00.470 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tWOTOVXQDUSAkZEmGyjeYhCeWF0=', 'Content-Length': '0'} 2022-09-03T05:31:00.472 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a57b96aa13d54212-006312e694-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.473 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:00.474 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:00.475 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:00.477 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:00.478 DEBUG:boto:Method: GET 2022-09-03T05:31:00.478 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:00.479 DEBUG:boto:Data: 2022-09-03T05:31:00.479 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.480 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.480 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.481 DEBUG:boto:Params: {} 2022-09-03T05:31:00.481 DEBUG:boto:Token: None 2022-09-03T05:31:00.482 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/ 2022-09-03T05:31:00.482 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE= 2022-09-03T05:31:00.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE=', 'Content-Length': '0'} 2022-09-03T05:31:00.487 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000723494bd6b51b4aa-006312e694-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.489 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:00.490 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:00.491 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:00.492 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:00.493 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:00.494 DEBUG:boto:Method: GET 2022-09-03T05:31:00.495 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:00.496 DEBUG:boto:Data: 2022-09-03T05:31:00.497 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.498 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.499 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.500 DEBUG:boto:Params: {} 2022-09-03T05:31:00.501 DEBUG:boto:Token: None 2022-09-03T05:31:00.502 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/ 2022-09-03T05:31:00.503 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE= 2022-09-03T05:31:00.504 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE=', 'Content-Length': '0'} 2022-09-03T05:31:00.507 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004e160f9a5c5de8da-006312e694-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.508 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:00.509 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:31:00.509 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.510 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.510 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.511 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.512 DEBUG:boto:Method: GET 2022-09-03T05:31:00.512 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.513 DEBUG:boto:Data: 2022-09-03T05:31:00.514 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.514 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.515 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.516 DEBUG:boto:Params: {} 2022-09-03T05:31:00.516 DEBUG:boto:Token: None 2022-09-03T05:31:00.517 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.517 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QU2dA2GjVoAUNAAzJ+jxVAYcaNw= 2022-09-03T05:31:00.518 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QU2dA2GjVoAUNAAzJ+jxVAYcaNw=', 'Content-Length': '0'} 2022-09-03T05:31:00.520 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000072e79071b5dbd50f-006312e694-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.521 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.521 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.522 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.523 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.523 DEBUG:boto:Method: GET 2022-09-03T05:31:00.524 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.524 DEBUG:boto:Data: 2022-09-03T05:31:00.525 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.525 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.526 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.526 DEBUG:boto:Params: {} 2022-09-03T05:31:00.527 DEBUG:boto:Token: None 2022-09-03T05:31:00.528 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.528 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QU2dA2GjVoAUNAAzJ+jxVAYcaNw= 2022-09-03T05:31:00.529 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QU2dA2GjVoAUNAAzJ+jxVAYcaNw=', 'Content-Length': '0'} 2022-09-03T05:31:00.531 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000718596782ca1758-006312e694-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.578 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.579 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.580 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.581 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.582 DEBUG:boto:Method: HEAD 2022-09-03T05:31:00.583 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.584 DEBUG:boto:Data: 2022-09-03T05:31:00.584 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.585 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.586 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.587 DEBUG:boto:Params: {} 2022-09-03T05:31:00.588 DEBUG:boto:Token: None 2022-09-03T05:31:00.588 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.589 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5myN/ULqI/nr+5pjrAbHjiE8sRY= 2022-09-03T05:31:00.590 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5myN/ULqI/nr+5pjrAbHjiE8sRY=', 'Content-Length': '0'} 2022-09-03T05:31:00.592 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000052530530c7a8ee34-006312e694-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.593 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.593 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.594 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.594 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.595 DEBUG:boto:Method: HEAD 2022-09-03T05:31:00.595 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.596 DEBUG:boto:Data: 2022-09-03T05:31:00.597 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.597 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.598 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.598 DEBUG:boto:Params: {} 2022-09-03T05:31:00.599 DEBUG:boto:Token: None 2022-09-03T05:31:00.600 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.600 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5myN/ULqI/nr+5pjrAbHjiE8sRY= 2022-09-03T05:31:00.601 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5myN/ULqI/nr+5pjrAbHjiE8sRY=', 'Content-Length': '0'} 2022-09-03T05:31:00.603 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000222a732e19367ba3-006312e694-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.604 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:31:00.605 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.605 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.606 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.606 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.607 DEBUG:boto:Method: GET 2022-09-03T05:31:00.607 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.608 DEBUG:boto:Data: 2022-09-03T05:31:00.608 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.609 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.610 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.611 DEBUG:boto:Params: {} 2022-09-03T05:31:00.612 DEBUG:boto:Token: None 2022-09-03T05:31:00.612 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.613 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QU2dA2GjVoAUNAAzJ+jxVAYcaNw= 2022-09-03T05:31:00.614 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QU2dA2GjVoAUNAAzJ+jxVAYcaNw=', 'Content-Length': '0'} 2022-09-03T05:31:00.616 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000caa11941828131c4-006312e694-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.617 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.617 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.618 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.619 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.619 DEBUG:boto:Method: GET 2022-09-03T05:31:00.620 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.621 DEBUG:boto:Data: 2022-09-03T05:31:00.622 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.623 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.624 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.625 DEBUG:boto:Params: {} 2022-09-03T05:31:00.626 DEBUG:boto:Token: None 2022-09-03T05:31:00.627 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.628 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QU2dA2GjVoAUNAAzJ+jxVAYcaNw= 2022-09-03T05:31:00.630 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QU2dA2GjVoAUNAAzJ+jxVAYcaNw=', 'Content-Length': '0'} 2022-09-03T05:31:00.632 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f43aff35c9aa3443-006312e694-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.634 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:00.635 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:00.636 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:00.637 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:00.638 DEBUG:boto:Method: GET 2022-09-03T05:31:00.639 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:00.640 DEBUG:boto:Data: 2022-09-03T05:31:00.641 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.641 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.642 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.642 DEBUG:boto:Params: {} 2022-09-03T05:31:00.643 DEBUG:boto:Token: None 2022-09-03T05:31:00.644 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/ 2022-09-03T05:31:00.644 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE= 2022-09-03T05:31:00.645 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE=', 'Content-Length': '0'} 2022-09-03T05:31:00.647 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d78949fc0e8ec46a-006312e694-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.648 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:00.649 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:00.650 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:00.650 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:00.651 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:00.651 DEBUG:boto:Method: GET 2022-09-03T05:31:00.652 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:00.652 DEBUG:boto:Data: 2022-09-03T05:31:00.653 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.653 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.654 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.654 DEBUG:boto:Params: {} 2022-09-03T05:31:00.655 DEBUG:boto:Token: None 2022-09-03T05:31:00.656 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/ 2022-09-03T05:31:00.656 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE= 2022-09-03T05:31:00.657 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE=', 'Content-Length': '0'} 2022-09-03T05:31:00.659 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000084fd25137b280617-006312e694-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.660 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:00.661 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.661 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.662 DEBUG:boto:Method: HEAD 2022-09-03T05:31:00.662 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.663 DEBUG:boto:Data: 2022-09-03T05:31:00.663 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.664 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.664 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.665 DEBUG:boto:Params: {} 2022-09-03T05:31:00.665 DEBUG:boto:Token: None 2022-09-03T05:31:00.666 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.667 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hdyYMgX7j47ZrGVntZbtXZirqec= 2022-09-03T05:31:00.667 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hdyYMgX7j47ZrGVntZbtXZirqec=', 'Content-Length': '0'} 2022-09-03T05:31:00.669 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d699b2ceaef10973-006312e694-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.670 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.671 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.671 DEBUG:boto:Method: HEAD 2022-09-03T05:31:00.672 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.672 DEBUG:boto:Data: 2022-09-03T05:31:00.673 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.673 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.674 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.675 DEBUG:boto:Params: {} 2022-09-03T05:31:00.675 DEBUG:boto:Token: None 2022-09-03T05:31:00.676 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.676 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hdyYMgX7j47ZrGVntZbtXZirqec= 2022-09-03T05:31:00.677 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hdyYMgX7j47ZrGVntZbtXZirqec=', 'Content-Length': '0'} 2022-09-03T05:31:00.679 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ce0794abef48618a-006312e694-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.680 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:31:00.681 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.681 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.682 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.683 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.683 DEBUG:boto:Method: GET 2022-09-03T05:31:00.684 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.684 DEBUG:boto:Data: 2022-09-03T05:31:00.685 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.685 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.686 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.687 DEBUG:boto:Params: {} 2022-09-03T05:31:00.687 DEBUG:boto:Token: None 2022-09-03T05:31:00.688 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.688 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QU2dA2GjVoAUNAAzJ+jxVAYcaNw= 2022-09-03T05:31:00.689 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QU2dA2GjVoAUNAAzJ+jxVAYcaNw=', 'Content-Length': '0'} 2022-09-03T05:31:00.691 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003e41c225a55c5424-006312e694-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.738 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.739 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:00.742 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.743 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.744 DEBUG:boto:Method: GET 2022-09-03T05:31:00.744 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.745 DEBUG:boto:Data: 2022-09-03T05:31:00.745 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.746 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.747 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.747 DEBUG:boto:Params: {} 2022-09-03T05:31:00.748 DEBUG:boto:Token: None 2022-09-03T05:31:00.749 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:00.750 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QU2dA2GjVoAUNAAzJ+jxVAYcaNw= 2022-09-03T05:31:00.751 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QU2dA2GjVoAUNAAzJ+jxVAYcaNw=', 'Content-Length': '0'} 2022-09-03T05:31:00.754 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007f5ccd8f4e05a4dd-006312e694-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.755 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:00.756 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:00.757 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:00.758 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:00.759 DEBUG:boto:Method: GET 2022-09-03T05:31:00.760 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:00.761 DEBUG:boto:Data: 2022-09-03T05:31:00.762 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.763 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.764 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.765 DEBUG:boto:Params: {} 2022-09-03T05:31:00.766 DEBUG:boto:Token: None 2022-09-03T05:31:00.767 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/ 2022-09-03T05:31:00.768 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE= 2022-09-03T05:31:00.769 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE=', 'Content-Length': '0'} 2022-09-03T05:31:00.772 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003c0370c19bb29eef-006312e694-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.773 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:00.775 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:00.776 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:00.777 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:00.778 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:00.778 DEBUG:boto:Method: GET 2022-09-03T05:31:00.779 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:00.780 DEBUG:boto:Data: 2022-09-03T05:31:00.781 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.782 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.782 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.783 DEBUG:boto:Params: {} 2022-09-03T05:31:00.784 DEBUG:boto:Token: None 2022-09-03T05:31:00.785 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/ 2022-09-03T05:31:00.786 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE= 2022-09-03T05:31:00.787 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE=', 'Content-Length': '0'} 2022-09-03T05:31:00.790 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002e48db1d292fe027-006312e694-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.791 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:00.793 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:31:00.794 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.795 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.796 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.796 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.797 DEBUG:boto:Method: GET 2022-09-03T05:31:00.798 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.798 DEBUG:boto:Data: 2022-09-03T05:31:00.799 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.800 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.800 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.801 DEBUG:boto:Params: {} 2022-09-03T05:31:00.801 DEBUG:boto:Token: None 2022-09-03T05:31:00.802 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.806 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E/YCyAE81Pxa+L8klzoFc87Km3g= 2022-09-03T05:31:00.807 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E/YCyAE81Pxa+L8klzoFc87Km3g=', 'Content-Length': '0'} 2022-09-03T05:31:00.809 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cef61b92434312fa-006312e694-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.810 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.810 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.811 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.811 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.812 DEBUG:boto:Method: GET 2022-09-03T05:31:00.813 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.813 DEBUG:boto:Data: 2022-09-03T05:31:00.814 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.814 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.815 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.815 DEBUG:boto:Params: {} 2022-09-03T05:31:00.816 DEBUG:boto:Token: None 2022-09-03T05:31:00.816 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.817 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E/YCyAE81Pxa+L8klzoFc87Km3g= 2022-09-03T05:31:00.818 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E/YCyAE81Pxa+L8klzoFc87Km3g=', 'Content-Length': '0'} 2022-09-03T05:31:00.820 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a0238a82fa2982ab-006312e694-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.820 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.821 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.822 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.822 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.823 DEBUG:boto:Method: HEAD 2022-09-03T05:31:00.823 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.824 DEBUG:boto:Data: 2022-09-03T05:31:00.824 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.825 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.826 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.827 DEBUG:boto:Params: {} 2022-09-03T05:31:00.828 DEBUG:boto:Token: None 2022-09-03T05:31:00.829 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.830 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EEQQROAOS9vEPRxH67kVWlIjTDE= 2022-09-03T05:31:00.830 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EEQQROAOS9vEPRxH67kVWlIjTDE=', 'Content-Length': '0'} 2022-09-03T05:31:00.833 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009270e25b9878a2b3-006312e694-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.834 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.834 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.835 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.835 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.836 DEBUG:boto:Method: HEAD 2022-09-03T05:31:00.837 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.837 DEBUG:boto:Data: 2022-09-03T05:31:00.838 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.838 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.839 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.840 DEBUG:boto:Params: {} 2022-09-03T05:31:00.841 DEBUG:boto:Token: None 2022-09-03T05:31:00.841 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.842 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EEQQROAOS9vEPRxH67kVWlIjTDE= 2022-09-03T05:31:00.843 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EEQQROAOS9vEPRxH67kVWlIjTDE=', 'Content-Length': '0'} 2022-09-03T05:31:00.845 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000400db56107c85956-006312e694-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.845 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:31:00.846 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.847 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.847 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.848 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.848 DEBUG:boto:Method: GET 2022-09-03T05:31:00.849 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.849 DEBUG:boto:Data: 2022-09-03T05:31:00.850 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.851 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.852 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.853 DEBUG:boto:Params: {} 2022-09-03T05:31:00.854 DEBUG:boto:Token: None 2022-09-03T05:31:00.855 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.856 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E/YCyAE81Pxa+L8klzoFc87Km3g= 2022-09-03T05:31:00.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E/YCyAE81Pxa+L8klzoFc87Km3g=', 'Content-Length': '0'} 2022-09-03T05:31:00.859 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000b01053733f6f3d5-006312e694-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.860 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.860 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.861 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.861 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.862 DEBUG:boto:Method: GET 2022-09-03T05:31:00.863 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.864 DEBUG:boto:Data: 2022-09-03T05:31:00.865 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.866 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.867 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.867 DEBUG:boto:Params: {} 2022-09-03T05:31:00.868 DEBUG:boto:Token: None 2022-09-03T05:31:00.868 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.869 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E/YCyAE81Pxa+L8klzoFc87Km3g= 2022-09-03T05:31:00.871 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E/YCyAE81Pxa+L8klzoFc87Km3g=', 'Content-Length': '0'} 2022-09-03T05:31:00.873 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001fb5fdd94b820f8c-006312e694-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.875 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:00.876 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:00.877 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:00.877 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:00.878 DEBUG:boto:Method: GET 2022-09-03T05:31:00.879 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:00.879 DEBUG:boto:Data: 2022-09-03T05:31:00.880 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.880 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.881 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.881 DEBUG:boto:Params: {} 2022-09-03T05:31:00.882 DEBUG:boto:Token: None 2022-09-03T05:31:00.883 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/ 2022-09-03T05:31:00.884 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE= 2022-09-03T05:31:00.884 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE=', 'Content-Length': '0'} 2022-09-03T05:31:00.887 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fe5f0752283b2c67-006312e694-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.888 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:00.888 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:00.889 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:00.890 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:00.890 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:00.891 DEBUG:boto:Method: GET 2022-09-03T05:31:00.891 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:00.892 DEBUG:boto:Data: 2022-09-03T05:31:00.893 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.893 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.894 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.894 DEBUG:boto:Params: {} 2022-09-03T05:31:00.895 DEBUG:boto:Token: None 2022-09-03T05:31:00.895 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/ 2022-09-03T05:31:00.896 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE= 2022-09-03T05:31:00.897 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RKY0aFdLCD1R+dM1sXKA+ngWrpE=', 'Content-Length': '0'} 2022-09-03T05:31:00.900 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000056e0ba9d9c75e9bc-006312e694-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.900 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:00.901 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.902 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.903 DEBUG:boto:Method: HEAD 2022-09-03T05:31:00.903 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.904 DEBUG:boto:Data: 2022-09-03T05:31:00.904 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.905 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.906 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.906 DEBUG:boto:Params: {} 2022-09-03T05:31:00.907 DEBUG:boto:Token: None 2022-09-03T05:31:00.908 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.909 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mb7VCPP9J1HS+ehrx7rot7hOgH4= 2022-09-03T05:31:00.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mb7VCPP9J1HS+ehrx7rot7hOgH4=', 'Content-Length': '0'} 2022-09-03T05:31:00.913 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000073a49a8928aa8576-006312e694-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.915 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.916 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.917 DEBUG:boto:Method: HEAD 2022-09-03T05:31:00.917 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.918 DEBUG:boto:Data: 2022-09-03T05:31:00.919 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.920 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.921 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.921 DEBUG:boto:Params: {} 2022-09-03T05:31:00.923 DEBUG:boto:Token: None 2022-09-03T05:31:00.924 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.924 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mb7VCPP9J1HS+ehrx7rot7hOgH4= 2022-09-03T05:31:00.925 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mb7VCPP9J1HS+ehrx7rot7hOgH4=', 'Content-Length': '0'} 2022-09-03T05:31:00.928 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d059bf49a6d74df2-006312e694-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.928 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:31:00.929 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.929 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.930 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.931 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.931 DEBUG:boto:Method: GET 2022-09-03T05:31:00.932 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.932 DEBUG:boto:Data: 2022-09-03T05:31:00.933 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.933 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:00.934 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.935 DEBUG:boto:Params: {} 2022-09-03T05:31:00.935 DEBUG:boto:Token: None 2022-09-03T05:31:00.936 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.936 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E/YCyAE81Pxa+L8klzoFc87Km3g= 2022-09-03T05:31:00.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E/YCyAE81Pxa+L8klzoFc87Km3g=', 'Content-Length': '0'} 2022-09-03T05:31:00.939 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006945fd2c3eca66f1-006312e694-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.986 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.987 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:00.987 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.988 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.988 DEBUG:boto:Method: GET 2022-09-03T05:31:00.989 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.990 DEBUG:boto:Data: 2022-09-03T05:31:00.990 DEBUG:boto:Headers: {} 2022-09-03T05:31:00.991 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:00.991 DEBUG:boto:Port: 8000 2022-09-03T05:31:00.992 DEBUG:boto:Params: {} 2022-09-03T05:31:00.992 DEBUG:boto:Token: None 2022-09-03T05:31:00.993 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:00 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:00.994 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E/YCyAE81Pxa+L8klzoFc87Km3g= 2022-09-03T05:31:00.994 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E/YCyAE81Pxa+L8klzoFc87Km3g=', 'Content-Length': '0'} 2022-09-03T05:31:00.997 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000007a813533879587e-006312e694-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:00.998 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:00.998 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:00.999 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:00.999 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:01.000 DEBUG:boto:Method: GET 2022-09-03T05:31:01.000 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:01.001 DEBUG:boto:Data: 2022-09-03T05:31:01.001 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.002 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:01.002 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.003 DEBUG:boto:Params: {} 2022-09-03T05:31:01.004 DEBUG:boto:Token: None 2022-09-03T05:31:01.004 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/ 2022-09-03T05:31:01.005 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM= 2022-09-03T05:31:01.005 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM=', 'Content-Length': '0'} 2022-09-03T05:31:01.008 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007640b7ddcccd81b6-006312e695-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.009 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:01.010 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:01.010 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:01.011 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:01.011 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:01.012 DEBUG:boto:Method: GET 2022-09-03T05:31:01.012 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:01.013 DEBUG:boto:Data: 2022-09-03T05:31:01.013 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.014 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.015 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.015 DEBUG:boto:Params: {} 2022-09-03T05:31:01.016 DEBUG:boto:Token: None 2022-09-03T05:31:01.016 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/ 2022-09-03T05:31:01.017 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM= 2022-09-03T05:31:01.018 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM=', 'Content-Length': '0'} 2022-09-03T05:31:01.020 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000038190b2a87f4b69f-006312e695-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.021 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:01.022 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-18 zones={test-zone2, test-zone1} 2022-09-03T05:31:01.023 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-18 zones={test-zone3, test-zone1} 2022-09-03T05:31:01.023 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:01.024 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:01.024 DEBUG:boto:Method: HEAD 2022-09-03T05:31:01.025 DEBUG:boto:Path: /dsgvcy-18/ 2022-09-03T05:31:01.025 DEBUG:boto:Data: 2022-09-03T05:31:01.026 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.026 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.027 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.027 DEBUG:boto:Params: {} 2022-09-03T05:31:01.029 DEBUG:boto:Token: None 2022-09-03T05:31:01.030 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/ 2022-09-03T05:31:01.030 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jgGqShDunI7eVGwg1TqBpsZ0wgk= 2022-09-03T05:31:01.031 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jgGqShDunI7eVGwg1TqBpsZ0wgk=', 'Content-Length': '0'} 2022-09-03T05:31:01.033 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004540776301bb793f-006312e695-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.034 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:01.035 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:01.035 DEBUG:boto:Method: HEAD 2022-09-03T05:31:01.036 DEBUG:boto:Path: /dsgvcy-18/ 2022-09-03T05:31:01.036 DEBUG:boto:Data: 2022-09-03T05:31:01.037 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.038 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.038 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.039 DEBUG:boto:Params: {} 2022-09-03T05:31:01.040 DEBUG:boto:Token: None 2022-09-03T05:31:01.041 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/ 2022-09-03T05:31:01.041 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jgGqShDunI7eVGwg1TqBpsZ0wgk= 2022-09-03T05:31:01.042 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jgGqShDunI7eVGwg1TqBpsZ0wgk=', 'Content-Length': '0'} 2022-09-03T05:31:01.044 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005ec8b28c1962e9ef-006312e695-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.045 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:31:01.046 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:01.046 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:01.047 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:01.048 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:01.049 DEBUG:boto:Method: GET 2022-09-03T05:31:01.053 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:01.054 DEBUG:boto:Data: 2022-09-03T05:31:01.055 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.056 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.057 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.059 DEBUG:boto:Params: {} 2022-09-03T05:31:01.059 DEBUG:boto:Token: None 2022-09-03T05:31:01.060 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/?versions 2022-09-03T05:31:01.060 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EESqYjD90bTU6w7MoeYk5FCkzAY= 2022-09-03T05:31:01.061 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EESqYjD90bTU6w7MoeYk5FCkzAY=', 'Content-Length': '0'} 2022-09-03T05:31:01.064 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000b84efb9db19c73b-006312e695-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.064 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:01.065 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:31:01.066 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:31:01.067 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:31:01.067 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:31:01.068 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:31:01.069 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:01.069 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:01.070 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:01.071 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:01.071 DEBUG:boto:Method: GET 2022-09-03T05:31:01.072 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:01.072 DEBUG:boto:Data: 2022-09-03T05:31:01.073 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.074 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.075 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.076 DEBUG:boto:Params: {} 2022-09-03T05:31:01.077 DEBUG:boto:Token: None 2022-09-03T05:31:01.078 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/?versions 2022-09-03T05:31:01.079 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EESqYjD90bTU6w7MoeYk5FCkzAY= 2022-09-03T05:31:01.080 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EESqYjD90bTU6w7MoeYk5FCkzAY=', 'Content-Length': '0'} 2022-09-03T05:31:01.084 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d804e3b871f4aae6-006312e695-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.085 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:01.087 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:31:01.088 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:31:01.088 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:31:01.089 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:31:01.090 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:01.091 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:01.092 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:01.093 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:01.094 DEBUG:boto:Method: GET 2022-09-03T05:31:01.095 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:01.096 DEBUG:boto:Data: 2022-09-03T05:31:01.097 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.098 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.099 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.100 DEBUG:boto:Params: {} 2022-09-03T05:31:01.101 DEBUG:boto:Token: None 2022-09-03T05:31:01.102 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/?versions 2022-09-03T05:31:01.103 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EESqYjD90bTU6w7MoeYk5FCkzAY= 2022-09-03T05:31:01.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EESqYjD90bTU6w7MoeYk5FCkzAY=', 'Content-Length': '0'} 2022-09-03T05:31:01.107 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aa07ac3c135d38f0-006312e695-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.108 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:01.109 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:01.110 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:01.110 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:01.111 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:01.111 DEBUG:boto:Method: GET 2022-09-03T05:31:01.112 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:01.112 DEBUG:boto:Data: 2022-09-03T05:31:01.113 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.113 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.114 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.115 DEBUG:boto:Params: {} 2022-09-03T05:31:01.115 DEBUG:boto:Token: None 2022-09-03T05:31:01.116 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/?versions 2022-09-03T05:31:01.117 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EESqYjD90bTU6w7MoeYk5FCkzAY= 2022-09-03T05:31:01.117 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EESqYjD90bTU6w7MoeYk5FCkzAY=', 'Content-Length': '0'} 2022-09-03T05:31:01.120 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e128dcddfed92b43-006312e695-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.121 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:01.122 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:31:01.122 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:01.123 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:01.124 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.124 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.125 DEBUG:boto:Method: GET 2022-09-03T05:31:01.125 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.126 DEBUG:boto:Data: 2022-09-03T05:31:01.127 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.127 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.128 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.129 DEBUG:boto:Params: {} 2022-09-03T05:31:01.130 DEBUG:boto:Token: None 2022-09-03T05:31:01.131 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.133 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wQ+564Ldtg76Sglr+YHu/3VMZ0M= 2022-09-03T05:31:01.134 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wQ+564Ldtg76Sglr+YHu/3VMZ0M=', 'Content-Length': '0'} 2022-09-03T05:31:01.137 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000011fe22fe3f5ead2c-006312e695-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.138 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:01.139 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:01.140 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.141 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.142 DEBUG:boto:Method: GET 2022-09-03T05:31:01.143 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.144 DEBUG:boto:Data: 2022-09-03T05:31:01.145 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.148 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.149 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.150 DEBUG:boto:Params: {} 2022-09-03T05:31:01.151 DEBUG:boto:Token: None 2022-09-03T05:31:01.152 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.153 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wQ+564Ldtg76Sglr+YHu/3VMZ0M= 2022-09-03T05:31:01.154 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wQ+564Ldtg76Sglr+YHu/3VMZ0M=', 'Content-Length': '0'} 2022-09-03T05:31:01.156 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000b6607c501783a762-006312e695-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.157 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:01.158 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:01.158 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.159 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.159 DEBUG:boto:Method: HEAD 2022-09-03T05:31:01.160 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.161 DEBUG:boto:Data: 2022-09-03T05:31:01.161 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.162 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.163 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.164 DEBUG:boto:Params: {} 2022-09-03T05:31:01.165 DEBUG:boto:Token: None 2022-09-03T05:31:01.166 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.167 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QZwwvq6SYLiCBXOpQOAUjS0pDVQ= 2022-09-03T05:31:01.167 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QZwwvq6SYLiCBXOpQOAUjS0pDVQ=', 'Content-Length': '0'} 2022-09-03T05:31:01.169 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000006a4655783612d1f0-006312e695-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.170 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:01.170 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:01.171 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.171 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.172 DEBUG:boto:Method: HEAD 2022-09-03T05:31:01.172 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.173 DEBUG:boto:Data: 2022-09-03T05:31:01.174 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.174 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.175 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.175 DEBUG:boto:Params: {} 2022-09-03T05:31:01.176 DEBUG:boto:Token: None 2022-09-03T05:31:01.176 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.177 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QZwwvq6SYLiCBXOpQOAUjS0pDVQ= 2022-09-03T05:31:01.178 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QZwwvq6SYLiCBXOpQOAUjS0pDVQ=', 'Content-Length': '0'} 2022-09-03T05:31:01.180 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000555b79bf7bbf8e01-006312e695-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.181 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:31:01.183 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:01.184 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:01.185 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.186 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.187 DEBUG:boto:Method: GET 2022-09-03T05:31:01.188 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.189 DEBUG:boto:Data: 2022-09-03T05:31:01.190 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.191 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.192 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.193 DEBUG:boto:Params: {} 2022-09-03T05:31:01.195 DEBUG:boto:Token: None 2022-09-03T05:31:01.196 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.197 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wQ+564Ldtg76Sglr+YHu/3VMZ0M= 2022-09-03T05:31:01.198 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wQ+564Ldtg76Sglr+YHu/3VMZ0M=', 'Content-Length': '0'} 2022-09-03T05:31:01.209 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000d2ad0f61f4e900ea-006312e695-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.210 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:01.211 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:01.211 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.212 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.212 DEBUG:boto:Method: GET 2022-09-03T05:31:01.213 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.214 DEBUG:boto:Data: 2022-09-03T05:31:01.214 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.215 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.215 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.216 DEBUG:boto:Params: {} 2022-09-03T05:31:01.217 DEBUG:boto:Token: None 2022-09-03T05:31:01.217 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.218 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wQ+564Ldtg76Sglr+YHu/3VMZ0M= 2022-09-03T05:31:01.219 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wQ+564Ldtg76Sglr+YHu/3VMZ0M=', 'Content-Length': '0'} 2022-09-03T05:31:01.221 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000e3e43a34a9c56c91-006312e695-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.222 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:01.223 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:01.223 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:01.224 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:01.225 DEBUG:boto:Method: GET 2022-09-03T05:31:01.225 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:31:01.226 DEBUG:boto:Data: 2022-09-03T05:31:01.226 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.227 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.228 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.229 DEBUG:boto:Params: {} 2022-09-03T05:31:01.230 DEBUG:boto:Token: None 2022-09-03T05:31:01.231 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/ 2022-09-03T05:31:01.232 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM= 2022-09-03T05:31:01.233 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM=', 'Content-Length': '0'} 2022-09-03T05:31:01.241 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003d316626f78e3e56-006312e695-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.280 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:01.282 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:01.283 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:01.284 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:01.284 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:01.285 DEBUG:boto:Method: GET 2022-09-03T05:31:01.285 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:31:01.286 DEBUG:boto:Data: 2022-09-03T05:31:01.287 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.287 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.288 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.289 DEBUG:boto:Params: {} 2022-09-03T05:31:01.290 DEBUG:boto:Token: None 2022-09-03T05:31:01.291 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/ 2022-09-03T05:31:01.293 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM= 2022-09-03T05:31:01.294 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM=', 'Content-Length': '0'} 2022-09-03T05:31:01.298 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006272600b7c37530c-006312e695-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.299 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:01.300 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:01.301 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:01.302 DEBUG:boto:Method: HEAD 2022-09-03T05:31:01.303 DEBUG:boto:Path: /dsgvcy-18/dummy 2022-09-03T05:31:01.304 DEBUG:boto:Data: 2022-09-03T05:31:01.305 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.305 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.307 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.307 DEBUG:boto:Params: {} 2022-09-03T05:31:01.309 DEBUG:boto:Token: None 2022-09-03T05:31:01.310 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/dummy 2022-09-03T05:31:01.311 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:haEJrGKaKqxofUlCpGu+fCwX0ck= 2022-09-03T05:31:01.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:haEJrGKaKqxofUlCpGu+fCwX0ck=', 'Content-Length': '0'} 2022-09-03T05:31:01.320 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000ab88eaffa5d7697e-006312e695-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.321 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:01.322 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:01.323 DEBUG:boto:Method: HEAD 2022-09-03T05:31:01.324 DEBUG:boto:Path: /dsgvcy-18/dummy 2022-09-03T05:31:01.325 DEBUG:boto:Data: 2022-09-03T05:31:01.326 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.327 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.328 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.329 DEBUG:boto:Params: {} 2022-09-03T05:31:01.330 DEBUG:boto:Token: None 2022-09-03T05:31:01.331 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/dummy 2022-09-03T05:31:01.332 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:haEJrGKaKqxofUlCpGu+fCwX0ck= 2022-09-03T05:31:01.333 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:haEJrGKaKqxofUlCpGu+fCwX0ck=', 'Content-Length': '0'} 2022-09-03T05:31:01.335 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000090036cf8e1c34a3d-006312e695-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.336 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:31:01.337 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:01.338 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:01.339 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.340 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.341 DEBUG:boto:Method: GET 2022-09-03T05:31:01.342 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.343 DEBUG:boto:Data: 2022-09-03T05:31:01.344 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.345 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.346 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.346 DEBUG:boto:Params: {} 2022-09-03T05:31:01.347 DEBUG:boto:Token: None 2022-09-03T05:31:01.348 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.348 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wQ+564Ldtg76Sglr+YHu/3VMZ0M= 2022-09-03T05:31:01.349 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wQ+564Ldtg76Sglr+YHu/3VMZ0M=', 'Content-Length': '0'} 2022-09-03T05:31:01.351 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000082e79511d83b5d5f-006312e695-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.352 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:01.353 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:01.353 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.354 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.354 DEBUG:boto:Method: GET 2022-09-03T05:31:01.355 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.355 DEBUG:boto:Data: 2022-09-03T05:31:01.356 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.357 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.357 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.358 DEBUG:boto:Params: {} 2022-09-03T05:31:01.359 DEBUG:boto:Token: None 2022-09-03T05:31:01.360 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:01.361 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wQ+564Ldtg76Sglr+YHu/3VMZ0M= 2022-09-03T05:31:01.363 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wQ+564Ldtg76Sglr+YHu/3VMZ0M=', 'Content-Length': '0'} 2022-09-03T05:31:01.366 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a9816f94bb85f97b-006312e695-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.368 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:01.369 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:01.370 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:01.371 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:01.372 DEBUG:boto:Method: GET 2022-09-03T05:31:01.374 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:31:01.375 DEBUG:boto:Data: 2022-09-03T05:31:01.376 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.377 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.378 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.378 DEBUG:boto:Params: {} 2022-09-03T05:31:01.379 DEBUG:boto:Token: None 2022-09-03T05:31:01.380 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/ 2022-09-03T05:31:01.381 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM= 2022-09-03T05:31:01.382 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM=', 'Content-Length': '0'} 2022-09-03T05:31:01.385 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f2cc1dc922424248-006312e695-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.386 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:01.387 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:01.388 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:01.388 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:01.389 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:01.389 DEBUG:boto:Method: GET 2022-09-03T05:31:01.390 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:31:01.390 DEBUG:boto:Data: 2022-09-03T05:31:01.391 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.391 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.392 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.392 DEBUG:boto:Params: {} 2022-09-03T05:31:01.393 DEBUG:boto:Token: None 2022-09-03T05:31:01.394 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/ 2022-09-03T05:31:01.394 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM= 2022-09-03T05:31:01.395 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM=', 'Content-Length': '0'} 2022-09-03T05:31:01.398 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c0f815adbad16313-006312e695-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.438 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:01.439 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:31:01.440 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.441 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.441 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.442 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.443 DEBUG:boto:Method: GET 2022-09-03T05:31:01.443 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.444 DEBUG:boto:Data: 2022-09-03T05:31:01.445 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.445 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.457 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.458 DEBUG:boto:Params: {} 2022-09-03T05:31:01.459 DEBUG:boto:Token: None 2022-09-03T05:31:01.460 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.460 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uueZTOyfWLGpmpVgJqCfCNMyhQ8= 2022-09-03T05:31:01.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uueZTOyfWLGpmpVgJqCfCNMyhQ8=', 'Content-Length': '0'} 2022-09-03T05:31:01.463 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002170433d1cdc4c3a-006312e695-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.464 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.465 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.465 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.466 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.467 DEBUG:boto:Method: GET 2022-09-03T05:31:01.468 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.468 DEBUG:boto:Data: 2022-09-03T05:31:01.469 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.469 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.470 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.471 DEBUG:boto:Params: {} 2022-09-03T05:31:01.472 DEBUG:boto:Token: None 2022-09-03T05:31:01.472 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.473 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uueZTOyfWLGpmpVgJqCfCNMyhQ8= 2022-09-03T05:31:01.474 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uueZTOyfWLGpmpVgJqCfCNMyhQ8=', 'Content-Length': '0'} 2022-09-03T05:31:01.476 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003367b895ef70d562-006312e695-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.478 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.479 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.480 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.481 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.482 DEBUG:boto:Method: HEAD 2022-09-03T05:31:01.483 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.484 DEBUG:boto:Data: 2022-09-03T05:31:01.485 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.486 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.487 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.488 DEBUG:boto:Params: {} 2022-09-03T05:31:01.489 DEBUG:boto:Token: None 2022-09-03T05:31:01.490 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.491 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mQUU+LVFZrxC1af0nSjIVJEkfgw= 2022-09-03T05:31:01.492 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mQUU+LVFZrxC1af0nSjIVJEkfgw=', 'Content-Length': '0'} 2022-09-03T05:31:01.495 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000021abe1a23025c4a9-006312e695-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.496 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.497 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.498 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.499 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.500 DEBUG:boto:Method: HEAD 2022-09-03T05:31:01.500 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.501 DEBUG:boto:Data: 2022-09-03T05:31:01.501 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.502 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.502 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.503 DEBUG:boto:Params: {} 2022-09-03T05:31:01.503 DEBUG:boto:Token: None 2022-09-03T05:31:01.504 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.505 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mQUU+LVFZrxC1af0nSjIVJEkfgw= 2022-09-03T05:31:01.505 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mQUU+LVFZrxC1af0nSjIVJEkfgw=', 'Content-Length': '0'} 2022-09-03T05:31:01.507 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000aa49d8ed035cb75e-006312e695-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.508 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:31:01.509 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.509 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.510 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.510 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.511 DEBUG:boto:Method: GET 2022-09-03T05:31:01.511 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.512 DEBUG:boto:Data: 2022-09-03T05:31:01.513 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.513 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.514 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.514 DEBUG:boto:Params: {} 2022-09-03T05:31:01.515 DEBUG:boto:Token: None 2022-09-03T05:31:01.516 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.516 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uueZTOyfWLGpmpVgJqCfCNMyhQ8= 2022-09-03T05:31:01.517 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uueZTOyfWLGpmpVgJqCfCNMyhQ8=', 'Content-Length': '0'} 2022-09-03T05:31:01.519 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006576006c36551388-006312e695-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.520 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.521 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.521 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.522 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.522 DEBUG:boto:Method: GET 2022-09-03T05:31:01.523 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.523 DEBUG:boto:Data: 2022-09-03T05:31:01.524 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.525 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.525 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.526 DEBUG:boto:Params: {} 2022-09-03T05:31:01.526 DEBUG:boto:Token: None 2022-09-03T05:31:01.527 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.527 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uueZTOyfWLGpmpVgJqCfCNMyhQ8= 2022-09-03T05:31:01.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uueZTOyfWLGpmpVgJqCfCNMyhQ8=', 'Content-Length': '0'} 2022-09-03T05:31:01.530 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f772180d694dac5a-006312e695-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.531 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:01.532 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:01.533 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:01.533 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:01.534 DEBUG:boto:Method: GET 2022-09-03T05:31:01.535 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:01.535 DEBUG:boto:Data: 2022-09-03T05:31:01.536 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.536 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.537 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.537 DEBUG:boto:Params: {} 2022-09-03T05:31:01.538 DEBUG:boto:Token: None 2022-09-03T05:31:01.538 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/ 2022-09-03T05:31:01.539 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM= 2022-09-03T05:31:01.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM=', 'Content-Length': '0'} 2022-09-03T05:31:01.542 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f01ea6eb184a495d-006312e695-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.543 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:01.544 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:01.545 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:01.545 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:01.546 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:01.547 DEBUG:boto:Method: GET 2022-09-03T05:31:01.547 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:01.548 DEBUG:boto:Data: 2022-09-03T05:31:01.548 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.549 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.549 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.550 DEBUG:boto:Params: {} 2022-09-03T05:31:01.550 DEBUG:boto:Token: None 2022-09-03T05:31:01.551 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/ 2022-09-03T05:31:01.552 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM= 2022-09-03T05:31:01.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM=', 'Content-Length': '0'} 2022-09-03T05:31:01.555 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fa7a972422cb311d-006312e695-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.555 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:01.556 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.557 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.558 DEBUG:boto:Method: HEAD 2022-09-03T05:31:01.558 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.559 DEBUG:boto:Data: 2022-09-03T05:31:01.559 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.560 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.560 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.561 DEBUG:boto:Params: {} 2022-09-03T05:31:01.561 DEBUG:boto:Token: None 2022-09-03T05:31:01.562 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.562 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z9ktIPiIp6oNDRU9ft0c6FRFIt8= 2022-09-03T05:31:01.563 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z9ktIPiIp6oNDRU9ft0c6FRFIt8=', 'Content-Length': '0'} 2022-09-03T05:31:01.565 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e0e33fffcb73f72a-006312e695-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.566 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.567 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.567 DEBUG:boto:Method: HEAD 2022-09-03T05:31:01.568 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.568 DEBUG:boto:Data: 2022-09-03T05:31:01.569 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.569 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.570 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.570 DEBUG:boto:Params: {} 2022-09-03T05:31:01.571 DEBUG:boto:Token: None 2022-09-03T05:31:01.572 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.572 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z9ktIPiIp6oNDRU9ft0c6FRFIt8= 2022-09-03T05:31:01.573 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z9ktIPiIp6oNDRU9ft0c6FRFIt8=', 'Content-Length': '0'} 2022-09-03T05:31:01.576 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e4e6845f20629d63-006312e695-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.577 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:31:01.578 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.579 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.579 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.580 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.581 DEBUG:boto:Method: GET 2022-09-03T05:31:01.582 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.583 DEBUG:boto:Data: 2022-09-03T05:31:01.584 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.586 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.587 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.588 DEBUG:boto:Params: {} 2022-09-03T05:31:01.589 DEBUG:boto:Token: None 2022-09-03T05:31:01.590 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.591 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uueZTOyfWLGpmpVgJqCfCNMyhQ8= 2022-09-03T05:31:01.593 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uueZTOyfWLGpmpVgJqCfCNMyhQ8=', 'Content-Length': '0'} 2022-09-03T05:31:01.595 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000056991c5a61c5e115-006312e695-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.642 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.643 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:01.644 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.645 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.645 DEBUG:boto:Method: GET 2022-09-03T05:31:01.646 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.646 DEBUG:boto:Data: 2022-09-03T05:31:01.647 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.647 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.648 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.648 DEBUG:boto:Params: {} 2022-09-03T05:31:01.649 DEBUG:boto:Token: None 2022-09-03T05:31:01.649 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:01.650 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uueZTOyfWLGpmpVgJqCfCNMyhQ8= 2022-09-03T05:31:01.651 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uueZTOyfWLGpmpVgJqCfCNMyhQ8=', 'Content-Length': '0'} 2022-09-03T05:31:01.653 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ffc670fbfb317556-006312e695-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.654 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:01.655 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:01.656 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:01.657 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:01.658 DEBUG:boto:Method: GET 2022-09-03T05:31:01.659 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:01.660 DEBUG:boto:Data: 2022-09-03T05:31:01.661 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.662 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.663 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.663 DEBUG:boto:Params: {} 2022-09-03T05:31:01.664 DEBUG:boto:Token: None 2022-09-03T05:31:01.664 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/ 2022-09-03T05:31:01.665 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM= 2022-09-03T05:31:01.666 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM=', 'Content-Length': '0'} 2022-09-03T05:31:01.669 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eec02ed7db89d978-006312e695-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.670 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:01.671 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:01.673 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:01.674 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:01.674 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:01.675 DEBUG:boto:Method: GET 2022-09-03T05:31:01.676 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:01.676 DEBUG:boto:Data: 2022-09-03T05:31:01.677 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.678 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.679 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.679 DEBUG:boto:Params: {} 2022-09-03T05:31:01.680 DEBUG:boto:Token: None 2022-09-03T05:31:01.681 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/ 2022-09-03T05:31:01.682 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM= 2022-09-03T05:31:01.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM=', 'Content-Length': '0'} 2022-09-03T05:31:01.686 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000af604d00794044b9-006312e695-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.687 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:01.689 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:31:01.690 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.690 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.693 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.694 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.695 DEBUG:boto:Method: GET 2022-09-03T05:31:01.696 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.697 DEBUG:boto:Data: 2022-09-03T05:31:01.697 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.698 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.699 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.699 DEBUG:boto:Params: {} 2022-09-03T05:31:01.700 DEBUG:boto:Token: None 2022-09-03T05:31:01.701 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.701 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n9jn6mlIxKSu3JsvN5AUyLqQ1sY= 2022-09-03T05:31:01.702 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n9jn6mlIxKSu3JsvN5AUyLqQ1sY=', 'Content-Length': '0'} 2022-09-03T05:31:01.705 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000011a2a5cc53dbe65a-006312e695-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.707 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.707 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.708 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.709 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.710 DEBUG:boto:Method: GET 2022-09-03T05:31:01.711 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.711 DEBUG:boto:Data: 2022-09-03T05:31:01.712 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.713 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.714 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.715 DEBUG:boto:Params: {} 2022-09-03T05:31:01.716 DEBUG:boto:Token: None 2022-09-03T05:31:01.717 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.718 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n9jn6mlIxKSu3JsvN5AUyLqQ1sY= 2022-09-03T05:31:01.719 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n9jn6mlIxKSu3JsvN5AUyLqQ1sY=', 'Content-Length': '0'} 2022-09-03T05:31:01.722 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007c2b49480b04ce6b-006312e695-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.732 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.734 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.735 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.736 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.737 DEBUG:boto:Method: HEAD 2022-09-03T05:31:01.738 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.739 DEBUG:boto:Data: 2022-09-03T05:31:01.740 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.740 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.741 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.742 DEBUG:boto:Params: {} 2022-09-03T05:31:01.745 DEBUG:boto:Token: None 2022-09-03T05:31:01.745 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.746 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h6DFGsv/2JOrD4GvrLDGEWwFrXU= 2022-09-03T05:31:01.747 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h6DFGsv/2JOrD4GvrLDGEWwFrXU=', 'Content-Length': '0'} 2022-09-03T05:31:01.749 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004cb27e36022f72e0-006312e695-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.749 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.750 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.751 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.751 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.752 DEBUG:boto:Method: HEAD 2022-09-03T05:31:01.752 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.753 DEBUG:boto:Data: 2022-09-03T05:31:01.755 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.756 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.757 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.758 DEBUG:boto:Params: {} 2022-09-03T05:31:01.759 DEBUG:boto:Token: None 2022-09-03T05:31:01.760 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.762 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h6DFGsv/2JOrD4GvrLDGEWwFrXU= 2022-09-03T05:31:01.763 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h6DFGsv/2JOrD4GvrLDGEWwFrXU=', 'Content-Length': '0'} 2022-09-03T05:31:01.767 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e456c0d1e5c2ad68-006312e695-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.768 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:31:01.769 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.770 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.771 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.772 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.773 DEBUG:boto:Method: GET 2022-09-03T05:31:01.774 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.776 DEBUG:boto:Data: 2022-09-03T05:31:01.777 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.778 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.779 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.780 DEBUG:boto:Params: {} 2022-09-03T05:31:01.780 DEBUG:boto:Token: None 2022-09-03T05:31:01.781 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.782 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n9jn6mlIxKSu3JsvN5AUyLqQ1sY= 2022-09-03T05:31:01.783 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n9jn6mlIxKSu3JsvN5AUyLqQ1sY=', 'Content-Length': '0'} 2022-09-03T05:31:01.785 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000055daea1a571499d2-006312e695-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.787 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.788 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.789 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.790 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.791 DEBUG:boto:Method: GET 2022-09-03T05:31:01.792 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.793 DEBUG:boto:Data: 2022-09-03T05:31:01.794 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.795 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.796 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.797 DEBUG:boto:Params: {} 2022-09-03T05:31:01.798 DEBUG:boto:Token: None 2022-09-03T05:31:01.799 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.799 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n9jn6mlIxKSu3JsvN5AUyLqQ1sY= 2022-09-03T05:31:01.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n9jn6mlIxKSu3JsvN5AUyLqQ1sY=', 'Content-Length': '0'} 2022-09-03T05:31:01.802 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008cb57e7f8c342b3a-006312e695-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.804 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:01.804 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:01.805 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:01.805 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:01.806 DEBUG:boto:Method: GET 2022-09-03T05:31:01.807 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:01.808 DEBUG:boto:Data: 2022-09-03T05:31:01.809 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.811 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.811 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.813 DEBUG:boto:Params: {} 2022-09-03T05:31:01.814 DEBUG:boto:Token: None 2022-09-03T05:31:01.815 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/ 2022-09-03T05:31:01.816 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM= 2022-09-03T05:31:01.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM=', 'Content-Length': '0'} 2022-09-03T05:31:01.820 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e4deee623c70cb39-006312e695-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.821 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:01.822 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:01.822 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:01.823 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:01.823 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:01.824 DEBUG:boto:Method: GET 2022-09-03T05:31:01.825 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:01.825 DEBUG:boto:Data: 2022-09-03T05:31:01.826 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.827 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.828 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.829 DEBUG:boto:Params: {} 2022-09-03T05:31:01.829 DEBUG:boto:Token: None 2022-09-03T05:31:01.830 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/ 2022-09-03T05:31:01.831 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM= 2022-09-03T05:31:01.831 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM=', 'Content-Length': '0'} 2022-09-03T05:31:01.835 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000031003953c76743d7-006312e695-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.878 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:01.880 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.881 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.881 DEBUG:boto:Method: HEAD 2022-09-03T05:31:01.882 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.882 DEBUG:boto:Data: 2022-09-03T05:31:01.883 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.884 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.884 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.885 DEBUG:boto:Params: {} 2022-09-03T05:31:01.885 DEBUG:boto:Token: None 2022-09-03T05:31:01.886 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.887 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IfV1CeiE59qPKiUtyTlFwFLBFG0= 2022-09-03T05:31:01.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IfV1CeiE59qPKiUtyTlFwFLBFG0=', 'Content-Length': '0'} 2022-09-03T05:31:01.890 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000036f12664485b4234-006312e695-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.891 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.892 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.893 DEBUG:boto:Method: HEAD 2022-09-03T05:31:01.894 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.895 DEBUG:boto:Data: 2022-09-03T05:31:01.896 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.897 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.897 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.898 DEBUG:boto:Params: {} 2022-09-03T05:31:01.899 DEBUG:boto:Token: None 2022-09-03T05:31:01.899 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.900 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IfV1CeiE59qPKiUtyTlFwFLBFG0= 2022-09-03T05:31:01.901 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IfV1CeiE59qPKiUtyTlFwFLBFG0=', 'Content-Length': '0'} 2022-09-03T05:31:01.903 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f27701d17c5885e6-006312e695-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.904 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:31:01.904 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.905 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.905 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.906 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.907 DEBUG:boto:Method: GET 2022-09-03T05:31:01.907 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.908 DEBUG:boto:Data: 2022-09-03T05:31:01.908 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.910 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.910 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.911 DEBUG:boto:Params: {} 2022-09-03T05:31:01.912 DEBUG:boto:Token: None 2022-09-03T05:31:01.913 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.913 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n9jn6mlIxKSu3JsvN5AUyLqQ1sY= 2022-09-03T05:31:01.914 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n9jn6mlIxKSu3JsvN5AUyLqQ1sY=', 'Content-Length': '0'} 2022-09-03T05:31:01.916 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000636845f797f38e40-006312e695-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.917 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.918 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:01.918 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.919 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.920 DEBUG:boto:Method: GET 2022-09-03T05:31:01.920 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.921 DEBUG:boto:Data: 2022-09-03T05:31:01.921 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.922 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.922 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.923 DEBUG:boto:Params: {} 2022-09-03T05:31:01.924 DEBUG:boto:Token: None 2022-09-03T05:31:01.924 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:01.925 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n9jn6mlIxKSu3JsvN5AUyLqQ1sY= 2022-09-03T05:31:01.926 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n9jn6mlIxKSu3JsvN5AUyLqQ1sY=', 'Content-Length': '0'} 2022-09-03T05:31:01.929 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008e501b3121144cae-006312e695-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.930 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:01.930 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:01.931 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:01.931 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:01.932 DEBUG:boto:Method: GET 2022-09-03T05:31:01.932 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:01.933 DEBUG:boto:Data: 2022-09-03T05:31:01.933 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.934 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.934 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.935 DEBUG:boto:Params: {} 2022-09-03T05:31:01.935 DEBUG:boto:Token: None 2022-09-03T05:31:01.936 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/ 2022-09-03T05:31:01.937 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM= 2022-09-03T05:31:01.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM=', 'Content-Length': '0'} 2022-09-03T05:31:01.940 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b6356b3b815339c9-006312e695-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.940 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:01.941 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:01.942 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:01.942 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:01.943 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:01.943 DEBUG:boto:Method: GET 2022-09-03T05:31:01.944 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:01.944 DEBUG:boto:Data: 2022-09-03T05:31:01.945 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.945 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.946 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.946 DEBUG:boto:Params: {} 2022-09-03T05:31:01.947 DEBUG:boto:Token: None 2022-09-03T05:31:01.948 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/ 2022-09-03T05:31:01.948 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM= 2022-09-03T05:31:01.949 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf7ee+fvfRFQkzepGJsabVvF9hM=', 'Content-Length': '0'} 2022-09-03T05:31:01.951 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eb3dab780e95bc1e-006312e695-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.952 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:01.953 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:31:01.953 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:01.954 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:01.954 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:01.955 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:01.955 DEBUG:boto:Method: GET 2022-09-03T05:31:01.956 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:01.957 DEBUG:boto:Data: 2022-09-03T05:31:01.957 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.958 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:01.958 DEBUG:boto:Port: 8001 2022-09-03T05:31:01.959 DEBUG:boto:Params: {} 2022-09-03T05:31:01.959 DEBUG:boto:Token: None 2022-09-03T05:31:01.961 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:01.961 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bEbdiyBMqzaM59Agu+oleqJOD+E= 2022-09-03T05:31:01.962 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bEbdiyBMqzaM59Agu+oleqJOD+E=', 'Content-Length': '0'} 2022-09-03T05:31:01.964 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000513bb49508abcbfd-006312e695-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:01.965 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:01.966 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:01.966 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:01.967 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:01.968 DEBUG:boto:Method: GET 2022-09-03T05:31:01.968 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:01.969 DEBUG:boto:Data: 2022-09-03T05:31:01.969 DEBUG:boto:Headers: {} 2022-09-03T05:31:01.970 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:01.971 DEBUG:boto:Port: 8000 2022-09-03T05:31:01.971 DEBUG:boto:Params: {} 2022-09-03T05:31:01.972 DEBUG:boto:Token: None 2022-09-03T05:31:01.973 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:01 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:01.974 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bEbdiyBMqzaM59Agu+oleqJOD+E= 2022-09-03T05:31:01.975 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bEbdiyBMqzaM59Agu+oleqJOD+E=', 'Content-Length': '0'} 2022-09-03T05:31:01.977 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000056bfa57c2515eb11-006312e695-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.018 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:02.019 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:02.020 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.021 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.022 DEBUG:boto:Method: HEAD 2022-09-03T05:31:02.023 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.024 DEBUG:boto:Data: 2022-09-03T05:31:02.024 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.025 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.026 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.027 DEBUG:boto:Params: {} 2022-09-03T05:31:02.028 DEBUG:boto:Token: None 2022-09-03T05:31:02.029 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.030 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yaoSUgFU/TaNQIa1Ni22Wya8Wrc= 2022-09-03T05:31:02.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yaoSUgFU/TaNQIa1Ni22Wya8Wrc=', 'Content-Length': '0'} 2022-09-03T05:31:02.034 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007a7854bbfa267835-006312e696-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.036 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:02.037 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:02.038 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.039 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.040 DEBUG:boto:Method: HEAD 2022-09-03T05:31:02.041 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.042 DEBUG:boto:Data: 2022-09-03T05:31:02.043 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.044 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:02.045 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.046 DEBUG:boto:Params: {} 2022-09-03T05:31:02.047 DEBUG:boto:Token: None 2022-09-03T05:31:02.048 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.049 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yaoSUgFU/TaNQIa1Ni22Wya8Wrc= 2022-09-03T05:31:02.050 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yaoSUgFU/TaNQIa1Ni22Wya8Wrc=', 'Content-Length': '0'} 2022-09-03T05:31:02.053 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b24510af5f836547-006312e696-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.054 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:31:02.055 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:02.056 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:02.057 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.058 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.059 DEBUG:boto:Method: GET 2022-09-03T05:31:02.060 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.061 DEBUG:boto:Data: 2022-09-03T05:31:02.062 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.063 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.064 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.065 DEBUG:boto:Params: {} 2022-09-03T05:31:02.066 DEBUG:boto:Token: None 2022-09-03T05:31:02.067 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.068 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7NebVAyL3F8+q2QUEiSGXP/nmc4= 2022-09-03T05:31:02.069 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7NebVAyL3F8+q2QUEiSGXP/nmc4=', 'Content-Length': '0'} 2022-09-03T05:31:02.071 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005951c7932ac58d5a-006312e696-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.114 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:02.115 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:02.116 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.116 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.117 DEBUG:boto:Method: GET 2022-09-03T05:31:02.117 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.118 DEBUG:boto:Data: 2022-09-03T05:31:02.118 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.119 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:02.119 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.120 DEBUG:boto:Params: {} 2022-09-03T05:31:02.121 DEBUG:boto:Token: None 2022-09-03T05:31:02.121 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.123 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7NebVAyL3F8+q2QUEiSGXP/nmc4= 2022-09-03T05:31:02.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7NebVAyL3F8+q2QUEiSGXP/nmc4=', 'Content-Length': '0'} 2022-09-03T05:31:02.126 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fc75c2fdbcd62c84-006312e696-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.127 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:02.128 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:02.128 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:02.129 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:02.129 DEBUG:boto:Method: GET 2022-09-03T05:31:02.130 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:02.131 DEBUG:boto:Data: 2022-09-03T05:31:02.131 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.132 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.132 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.133 DEBUG:boto:Params: {} 2022-09-03T05:31:02.133 DEBUG:boto:Token: None 2022-09-03T05:31:02.134 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/ 2022-09-03T05:31:02.135 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk= 2022-09-03T05:31:02.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk=', 'Content-Length': '0'} 2022-09-03T05:31:02.138 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000438412ba19c3e39d-006312e696-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.139 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:02.140 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:02.140 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:02.141 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:02.141 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:02.142 DEBUG:boto:Method: GET 2022-09-03T05:31:02.143 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:02.144 DEBUG:boto:Data: 2022-09-03T05:31:02.145 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.146 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:02.146 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.147 DEBUG:boto:Params: {} 2022-09-03T05:31:02.148 DEBUG:boto:Token: None 2022-09-03T05:31:02.148 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/ 2022-09-03T05:31:02.149 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk= 2022-09-03T05:31:02.149 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk=', 'Content-Length': '0'} 2022-09-03T05:31:02.152 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c0081fe5aaaefe4d-006312e696-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.153 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:02.153 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:02.154 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:02.155 DEBUG:boto:Method: HEAD 2022-09-03T05:31:02.155 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3 2022-09-03T05:31:02.156 DEBUG:boto:Data: 2022-09-03T05:31:02.156 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.157 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.157 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.158 DEBUG:boto:Params: {} 2022-09-03T05:31:02.158 DEBUG:boto:Token: None 2022-09-03T05:31:02.159 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone3 2022-09-03T05:31:02.160 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PGnXnAyV3pQems9Sm4/muFP5Lyk= 2022-09-03T05:31:02.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PGnXnAyV3pQems9Sm4/muFP5Lyk=', 'Content-Length': '0'} 2022-09-03T05:31:02.163 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000038d67b062ef4e2a-006312e696-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.164 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:02.164 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:02.165 DEBUG:boto:Method: HEAD 2022-09-03T05:31:02.166 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3 2022-09-03T05:31:02.167 DEBUG:boto:Data: 2022-09-03T05:31:02.167 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.168 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:02.168 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.169 DEBUG:boto:Params: {} 2022-09-03T05:31:02.169 DEBUG:boto:Token: None 2022-09-03T05:31:02.170 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone3 2022-09-03T05:31:02.170 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PGnXnAyV3pQems9Sm4/muFP5Lyk= 2022-09-03T05:31:02.171 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PGnXnAyV3pQems9Sm4/muFP5Lyk=', 'Content-Length': '0'} 2022-09-03T05:31:02.173 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007c896368cd973f56-006312e696-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.174 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:31:02.175 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:02.176 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:02.176 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.177 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.177 DEBUG:boto:Method: GET 2022-09-03T05:31:02.178 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.178 DEBUG:boto:Data: 2022-09-03T05:31:02.179 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.179 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.180 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.180 DEBUG:boto:Params: {} 2022-09-03T05:31:02.181 DEBUG:boto:Token: None 2022-09-03T05:31:02.182 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.182 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7NebVAyL3F8+q2QUEiSGXP/nmc4= 2022-09-03T05:31:02.183 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7NebVAyL3F8+q2QUEiSGXP/nmc4=', 'Content-Length': '0'} 2022-09-03T05:31:02.185 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000086c4605bf215445f-006312e696-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.226 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:02.227 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:02.227 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.229 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.230 DEBUG:boto:Method: GET 2022-09-03T05:31:02.230 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.231 DEBUG:boto:Data: 2022-09-03T05:31:02.231 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.232 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:02.233 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.233 DEBUG:boto:Params: {} 2022-09-03T05:31:02.234 DEBUG:boto:Token: None 2022-09-03T05:31:02.235 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:02.235 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7NebVAyL3F8+q2QUEiSGXP/nmc4= 2022-09-03T05:31:02.238 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7NebVAyL3F8+q2QUEiSGXP/nmc4=', 'Content-Length': '0'} 2022-09-03T05:31:02.241 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000046b31303127f56fb-006312e696-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.242 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:02.243 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:02.244 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:02.246 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:02.247 DEBUG:boto:Method: GET 2022-09-03T05:31:02.248 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:02.249 DEBUG:boto:Data: 2022-09-03T05:31:02.250 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.251 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.252 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.253 DEBUG:boto:Params: {} 2022-09-03T05:31:02.255 DEBUG:boto:Token: None 2022-09-03T05:31:02.256 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/ 2022-09-03T05:31:02.257 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk= 2022-09-03T05:31:02.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk=', 'Content-Length': '0'} 2022-09-03T05:31:02.261 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000055ac7cf007cc4bd9-006312e696-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.262 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:02.264 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:02.265 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:02.266 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:02.267 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:02.268 DEBUG:boto:Method: GET 2022-09-03T05:31:02.269 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:02.270 DEBUG:boto:Data: 2022-09-03T05:31:02.271 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.272 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:02.273 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.274 DEBUG:boto:Params: {} 2022-09-03T05:31:02.275 DEBUG:boto:Token: None 2022-09-03T05:31:02.277 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/ 2022-09-03T05:31:02.278 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk= 2022-09-03T05:31:02.279 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk=', 'Content-Length': '0'} 2022-09-03T05:31:02.282 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000077815fd3bf70f21b-006312e696-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.283 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:02.284 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-18 zones={test-zone3, test-zone1} 2022-09-03T05:31:02.285 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-18 zones={test-zone3, test-zone2} 2022-09-03T05:31:02.285 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:02.286 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:02.286 DEBUG:boto:Method: HEAD 2022-09-03T05:31:02.287 DEBUG:boto:Path: /dsgvcy-18/ 2022-09-03T05:31:02.287 DEBUG:boto:Data: 2022-09-03T05:31:02.288 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.288 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.289 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.290 DEBUG:boto:Params: {} 2022-09-03T05:31:02.290 DEBUG:boto:Token: None 2022-09-03T05:31:02.291 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/ 2022-09-03T05:31:02.291 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gWr1HLAijMggcy66g3KqsAAo/8I= 2022-09-03T05:31:02.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gWr1HLAijMggcy66g3KqsAAo/8I=', 'Content-Length': '0'} 2022-09-03T05:31:02.294 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f412bcc924093997-006312e696-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.295 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:02.296 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:02.296 DEBUG:boto:Method: HEAD 2022-09-03T05:31:02.297 DEBUG:boto:Path: /dsgvcy-18/ 2022-09-03T05:31:02.297 DEBUG:boto:Data: 2022-09-03T05:31:02.298 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.298 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:02.299 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.299 DEBUG:boto:Params: {} 2022-09-03T05:31:02.300 DEBUG:boto:Token: None 2022-09-03T05:31:02.301 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/ 2022-09-03T05:31:02.301 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gWr1HLAijMggcy66g3KqsAAo/8I= 2022-09-03T05:31:02.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gWr1HLAijMggcy66g3KqsAAo/8I=', 'Content-Length': '0'} 2022-09-03T05:31:02.305 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b2b11391c43622af-006312e696-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.306 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:31:02.306 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:02.307 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:02.307 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:02.308 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:02.308 DEBUG:boto:Method: GET 2022-09-03T05:31:02.309 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:02.309 DEBUG:boto:Data: 2022-09-03T05:31:02.310 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.310 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.311 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.311 DEBUG:boto:Params: {} 2022-09-03T05:31:02.312 DEBUG:boto:Token: None 2022-09-03T05:31:02.313 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/?versions 2022-09-03T05:31:02.313 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WrfK9jGcF1pLpJjxLRGJJnYqork= 2022-09-03T05:31:02.314 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WrfK9jGcF1pLpJjxLRGJJnYqork=', 'Content-Length': '0'} 2022-09-03T05:31:02.316 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000678c76cf4f67f74a-006312e696-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.317 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:02.319 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:31:02.319 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:31:02.320 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:31:02.320 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:31:02.320 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:31:02.321 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:02.321 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:02.322 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:02.322 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:02.323 DEBUG:boto:Method: GET 2022-09-03T05:31:02.324 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:02.324 DEBUG:boto:Data: 2022-09-03T05:31:02.325 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.325 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:02.326 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.326 DEBUG:boto:Params: {} 2022-09-03T05:31:02.327 DEBUG:boto:Token: None 2022-09-03T05:31:02.328 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/?versions 2022-09-03T05:31:02.328 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WrfK9jGcF1pLpJjxLRGJJnYqork= 2022-09-03T05:31:02.329 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WrfK9jGcF1pLpJjxLRGJJnYqork=', 'Content-Length': '0'} 2022-09-03T05:31:02.332 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000051ca6e30bae847a8-006312e696-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.333 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:02.334 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:31:02.335 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:31:02.335 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:31:02.336 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:31:02.336 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:02.337 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:02.338 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:02.338 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:02.339 DEBUG:boto:Method: GET 2022-09-03T05:31:02.339 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:02.340 DEBUG:boto:Data: 2022-09-03T05:31:02.340 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.341 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.342 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.342 DEBUG:boto:Params: {} 2022-09-03T05:31:02.343 DEBUG:boto:Token: None 2022-09-03T05:31:02.344 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/?versions 2022-09-03T05:31:02.344 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WrfK9jGcF1pLpJjxLRGJJnYqork= 2022-09-03T05:31:02.345 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WrfK9jGcF1pLpJjxLRGJJnYqork=', 'Content-Length': '0'} 2022-09-03T05:31:02.347 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009712d4de4e61ca52-006312e696-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.348 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:02.349 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:02.349 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:02.350 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:02.350 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:02.351 DEBUG:boto:Method: GET 2022-09-03T05:31:02.351 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:31:02.352 DEBUG:boto:Data: 2022-09-03T05:31:02.353 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.354 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:02.354 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.355 DEBUG:boto:Params: {} 2022-09-03T05:31:02.355 DEBUG:boto:Token: None 2022-09-03T05:31:02.356 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/?versions 2022-09-03T05:31:02.356 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WrfK9jGcF1pLpJjxLRGJJnYqork= 2022-09-03T05:31:02.357 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WrfK9jGcF1pLpJjxLRGJJnYqork=', 'Content-Length': '0'} 2022-09-03T05:31:02.360 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000abc3d9bc5f0b235d-006312e696-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.360 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:02.362 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:31:02.362 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:02.364 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:02.364 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.365 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.365 DEBUG:boto:Method: GET 2022-09-03T05:31:02.366 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.367 DEBUG:boto:Data: 2022-09-03T05:31:02.367 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.368 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.368 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.369 DEBUG:boto:Params: {} 2022-09-03T05:31:02.370 DEBUG:boto:Token: None 2022-09-03T05:31:02.371 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.371 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mBnIzX2jR0pYQRlXVZ9kYZB0cp8= 2022-09-03T05:31:02.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mBnIzX2jR0pYQRlXVZ9kYZB0cp8=', 'Content-Length': '0'} 2022-09-03T05:31:02.374 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000ff9a40ba51b9fdba-006312e696-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.375 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:02.376 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:02.377 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.378 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.378 DEBUG:boto:Method: GET 2022-09-03T05:31:02.379 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.380 DEBUG:boto:Data: 2022-09-03T05:31:02.381 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.382 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:02.383 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.383 DEBUG:boto:Params: {} 2022-09-03T05:31:02.384 DEBUG:boto:Token: None 2022-09-03T05:31:02.386 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.387 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mBnIzX2jR0pYQRlXVZ9kYZB0cp8= 2022-09-03T05:31:02.388 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mBnIzX2jR0pYQRlXVZ9kYZB0cp8=', 'Content-Length': '0'} 2022-09-03T05:31:02.391 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f75a57f6bb24a241-006312e696-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.393 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:02.393 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:02.394 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.395 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.395 DEBUG:boto:Method: HEAD 2022-09-03T05:31:02.396 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.396 DEBUG:boto:Data: 2022-09-03T05:31:02.397 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.397 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.398 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.399 DEBUG:boto:Params: {} 2022-09-03T05:31:02.399 DEBUG:boto:Token: None 2022-09-03T05:31:02.400 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.402 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p1DMZeOtIZhquKQG8o9MBf0usrs= 2022-09-03T05:31:02.402 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p1DMZeOtIZhquKQG8o9MBf0usrs=', 'Content-Length': '0'} 2022-09-03T05:31:02.404 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000e6e4ab8bc1e5b7cb-006312e696-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.405 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:02.406 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:02.407 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.408 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.408 DEBUG:boto:Method: HEAD 2022-09-03T05:31:02.409 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.409 DEBUG:boto:Data: 2022-09-03T05:31:02.410 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.410 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:02.411 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.412 DEBUG:boto:Params: {} 2022-09-03T05:31:02.413 DEBUG:boto:Token: None 2022-09-03T05:31:02.413 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.414 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p1DMZeOtIZhquKQG8o9MBf0usrs= 2022-09-03T05:31:02.415 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p1DMZeOtIZhquKQG8o9MBf0usrs=', 'Content-Length': '0'} 2022-09-03T05:31:02.417 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000000efd953d18ababa8-006312e696-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.417 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:31:02.418 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:02.419 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:02.420 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.420 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.421 DEBUG:boto:Method: GET 2022-09-03T05:31:02.422 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.423 DEBUG:boto:Data: 2022-09-03T05:31:02.423 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.424 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.425 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.426 DEBUG:boto:Params: {} 2022-09-03T05:31:02.427 DEBUG:boto:Token: None 2022-09-03T05:31:02.428 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.429 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mBnIzX2jR0pYQRlXVZ9kYZB0cp8= 2022-09-03T05:31:02.430 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mBnIzX2jR0pYQRlXVZ9kYZB0cp8=', 'Content-Length': '0'} 2022-09-03T05:31:02.434 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000008cadca1818a90ec7-006312e696-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.435 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:02.436 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:02.437 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.437 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.438 DEBUG:boto:Method: GET 2022-09-03T05:31:02.439 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.439 DEBUG:boto:Data: 2022-09-03T05:31:02.440 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.441 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:02.442 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.444 DEBUG:boto:Params: {} 2022-09-03T05:31:02.444 DEBUG:boto:Token: None 2022-09-03T05:31:02.445 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.446 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mBnIzX2jR0pYQRlXVZ9kYZB0cp8= 2022-09-03T05:31:02.446 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mBnIzX2jR0pYQRlXVZ9kYZB0cp8=', 'Content-Length': '0'} 2022-09-03T05:31:02.448 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000063859a9e22bf39ff-006312e696-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.449 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:02.450 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:02.450 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:02.451 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:02.452 DEBUG:boto:Method: GET 2022-09-03T05:31:02.452 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:31:02.453 DEBUG:boto:Data: 2022-09-03T05:31:02.454 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.454 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.455 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.455 DEBUG:boto:Params: {} 2022-09-03T05:31:02.456 DEBUG:boto:Token: None 2022-09-03T05:31:02.457 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/ 2022-09-03T05:31:02.457 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk= 2022-09-03T05:31:02.458 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk=', 'Content-Length': '0'} 2022-09-03T05:31:02.461 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000002bfe242fc3ff4d6-006312e696-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.506 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:02.507 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:02.508 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:02.509 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:02.510 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:02.511 DEBUG:boto:Method: GET 2022-09-03T05:31:02.512 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:31:02.513 DEBUG:boto:Data: 2022-09-03T05:31:02.514 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.515 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:02.516 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.516 DEBUG:boto:Params: {} 2022-09-03T05:31:02.517 DEBUG:boto:Token: None 2022-09-03T05:31:02.517 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/ 2022-09-03T05:31:02.519 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk= 2022-09-03T05:31:02.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk=', 'Content-Length': '0'} 2022-09-03T05:31:02.523 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003cca879fb5f2d5e7-006312e696-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.524 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:02.525 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:02.526 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:02.527 DEBUG:boto:Method: HEAD 2022-09-03T05:31:02.528 DEBUG:boto:Path: /dsgvcy-18/dummy 2022-09-03T05:31:02.528 DEBUG:boto:Data: 2022-09-03T05:31:02.529 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.530 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.530 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.531 DEBUG:boto:Params: {} 2022-09-03T05:31:02.531 DEBUG:boto:Token: None 2022-09-03T05:31:02.532 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/dummy 2022-09-03T05:31:02.532 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PfHW0f7HSlwYIm87W3Iqodo8beo= 2022-09-03T05:31:02.533 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PfHW0f7HSlwYIm87W3Iqodo8beo=', 'Content-Length': '0'} 2022-09-03T05:31:02.536 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000cecc9a387a12e271-006312e696-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.536 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:02.537 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:02.537 DEBUG:boto:Method: HEAD 2022-09-03T05:31:02.538 DEBUG:boto:Path: /dsgvcy-18/dummy 2022-09-03T05:31:02.539 DEBUG:boto:Data: 2022-09-03T05:31:02.539 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.540 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:02.540 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.541 DEBUG:boto:Params: {} 2022-09-03T05:31:02.541 DEBUG:boto:Token: None 2022-09-03T05:31:02.542 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/dummy 2022-09-03T05:31:02.543 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PfHW0f7HSlwYIm87W3Iqodo8beo= 2022-09-03T05:31:02.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PfHW0f7HSlwYIm87W3Iqodo8beo=', 'Content-Length': '0'} 2022-09-03T05:31:02.546 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000026bb9b4432af58ae-006312e696-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.547 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:31:02.548 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:02.548 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:02.549 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.549 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.550 DEBUG:boto:Method: GET 2022-09-03T05:31:02.551 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.551 DEBUG:boto:Data: 2022-09-03T05:31:02.552 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.553 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.553 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.554 DEBUG:boto:Params: {} 2022-09-03T05:31:02.554 DEBUG:boto:Token: None 2022-09-03T05:31:02.555 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.556 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mBnIzX2jR0pYQRlXVZ9kYZB0cp8= 2022-09-03T05:31:02.557 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mBnIzX2jR0pYQRlXVZ9kYZB0cp8=', 'Content-Length': '0'} 2022-09-03T05:31:02.558 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000008680ecda5a7a5a85-006312e696-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.559 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:31:02.560 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:31:02.560 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.561 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.562 DEBUG:boto:Method: GET 2022-09-03T05:31:02.562 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.563 DEBUG:boto:Data: 2022-09-03T05:31:02.564 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.565 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:02.566 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.566 DEBUG:boto:Params: {} 2022-09-03T05:31:02.567 DEBUG:boto:Token: None 2022-09-03T05:31:02.568 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:31:02.569 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mBnIzX2jR0pYQRlXVZ9kYZB0cp8= 2022-09-03T05:31:02.570 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mBnIzX2jR0pYQRlXVZ9kYZB0cp8=', 'Content-Length': '0'} 2022-09-03T05:31:02.572 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000c5727e6b29cca6f2-006312e696-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.573 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:02.575 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:02.576 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:02.577 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:02.578 DEBUG:boto:Method: GET 2022-09-03T05:31:02.579 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:31:02.580 DEBUG:boto:Data: 2022-09-03T05:31:02.581 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.582 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.583 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.584 DEBUG:boto:Params: {} 2022-09-03T05:31:02.585 DEBUG:boto:Token: None 2022-09-03T05:31:02.586 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/ 2022-09-03T05:31:02.587 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk= 2022-09-03T05:31:02.588 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk=', 'Content-Length': '0'} 2022-09-03T05:31:02.590 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006f2a006a205c16e8-006312e696-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.591 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:02.592 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:02.593 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:02.593 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:02.594 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:31:02.594 DEBUG:boto:Method: GET 2022-09-03T05:31:02.595 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:31:02.596 DEBUG:boto:Data: 2022-09-03T05:31:02.596 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.597 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:02.597 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.598 DEBUG:boto:Params: {} 2022-09-03T05:31:02.599 DEBUG:boto:Token: None 2022-09-03T05:31:02.600 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/ 2022-09-03T05:31:02.600 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk= 2022-09-03T05:31:02.601 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk=', 'Content-Length': '0'} 2022-09-03T05:31:02.604 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000081140b3d118dc693-006312e696-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.605 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:02.607 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:31:02.607 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.608 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.608 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.609 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.610 DEBUG:boto:Method: GET 2022-09-03T05:31:02.610 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.611 DEBUG:boto:Data: 2022-09-03T05:31:02.612 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.612 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.613 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.613 DEBUG:boto:Params: {} 2022-09-03T05:31:02.614 DEBUG:boto:Token: None 2022-09-03T05:31:02.615 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.616 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eVmf6jtNC/xvIiJhprPoXuwfaCI= 2022-09-03T05:31:02.616 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eVmf6jtNC/xvIiJhprPoXuwfaCI=', 'Content-Length': '0'} 2022-09-03T05:31:02.619 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006f0aa4440b2401cc-006312e696-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.666 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.667 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.667 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.668 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.668 DEBUG:boto:Method: GET 2022-09-03T05:31:02.669 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.669 DEBUG:boto:Data: 2022-09-03T05:31:02.670 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.671 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:02.671 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.672 DEBUG:boto:Params: {} 2022-09-03T05:31:02.672 DEBUG:boto:Token: None 2022-09-03T05:31:02.673 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.673 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eVmf6jtNC/xvIiJhprPoXuwfaCI= 2022-09-03T05:31:02.674 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eVmf6jtNC/xvIiJhprPoXuwfaCI=', 'Content-Length': '0'} 2022-09-03T05:31:02.677 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009ec22321bc1afc37-006312e696-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.677 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.678 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.679 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.680 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.681 DEBUG:boto:Method: HEAD 2022-09-03T05:31:02.682 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.683 DEBUG:boto:Data: 2022-09-03T05:31:02.683 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.684 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.685 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.686 DEBUG:boto:Params: {} 2022-09-03T05:31:02.687 DEBUG:boto:Token: None 2022-09-03T05:31:02.688 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.689 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fXl3Rh17EsPt2lF5PMjHzdOElUI= 2022-09-03T05:31:02.690 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fXl3Rh17EsPt2lF5PMjHzdOElUI=', 'Content-Length': '0'} 2022-09-03T05:31:02.692 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005b9a4994ee41e8a0-006312e696-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.693 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.693 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.694 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.694 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.695 DEBUG:boto:Method: HEAD 2022-09-03T05:31:02.695 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.696 DEBUG:boto:Data: 2022-09-03T05:31:02.696 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.697 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:02.697 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.698 DEBUG:boto:Params: {} 2022-09-03T05:31:02.699 DEBUG:boto:Token: None 2022-09-03T05:31:02.699 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.700 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fXl3Rh17EsPt2lF5PMjHzdOElUI= 2022-09-03T05:31:02.701 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fXl3Rh17EsPt2lF5PMjHzdOElUI=', 'Content-Length': '0'} 2022-09-03T05:31:02.703 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007cec3185f99fab1d-006312e696-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.704 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:31:02.705 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.706 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.707 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.708 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.709 DEBUG:boto:Method: GET 2022-09-03T05:31:02.710 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.711 DEBUG:boto:Data: 2022-09-03T05:31:02.712 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.712 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.713 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.713 DEBUG:boto:Params: {} 2022-09-03T05:31:02.714 DEBUG:boto:Token: None 2022-09-03T05:31:02.715 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.715 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eVmf6jtNC/xvIiJhprPoXuwfaCI= 2022-09-03T05:31:02.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eVmf6jtNC/xvIiJhprPoXuwfaCI=', 'Content-Length': '0'} 2022-09-03T05:31:02.718 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bfa7856f1fb77282-006312e696-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.719 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.719 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.720 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.721 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.721 DEBUG:boto:Method: GET 2022-09-03T05:31:02.722 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.722 DEBUG:boto:Data: 2022-09-03T05:31:02.723 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.723 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:02.724 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.724 DEBUG:boto:Params: {} 2022-09-03T05:31:02.725 DEBUG:boto:Token: None 2022-09-03T05:31:02.726 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.726 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eVmf6jtNC/xvIiJhprPoXuwfaCI= 2022-09-03T05:31:02.727 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eVmf6jtNC/xvIiJhprPoXuwfaCI=', 'Content-Length': '0'} 2022-09-03T05:31:02.729 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000044601ffc7428c138-006312e696-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.730 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:02.730 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:02.731 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:02.732 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:02.732 DEBUG:boto:Method: GET 2022-09-03T05:31:02.733 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:02.733 DEBUG:boto:Data: 2022-09-03T05:31:02.734 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.734 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.735 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.735 DEBUG:boto:Params: {} 2022-09-03T05:31:02.736 DEBUG:boto:Token: None 2022-09-03T05:31:02.736 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/ 2022-09-03T05:31:02.737 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk= 2022-09-03T05:31:02.738 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk=', 'Content-Length': '0'} 2022-09-03T05:31:02.740 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000012d451f497502323-006312e696-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.741 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:02.742 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:02.743 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:02.744 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:02.745 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:02.745 DEBUG:boto:Method: GET 2022-09-03T05:31:02.746 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:02.747 DEBUG:boto:Data: 2022-09-03T05:31:02.748 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.748 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:02.749 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.751 DEBUG:boto:Params: {} 2022-09-03T05:31:02.752 DEBUG:boto:Token: None 2022-09-03T05:31:02.752 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/ 2022-09-03T05:31:02.753 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk= 2022-09-03T05:31:02.754 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk=', 'Content-Length': '0'} 2022-09-03T05:31:02.758 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a97e88e63ea3ff87-006312e696-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.759 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:02.760 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.760 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.761 DEBUG:boto:Method: HEAD 2022-09-03T05:31:02.762 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.762 DEBUG:boto:Data: 2022-09-03T05:31:02.763 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.763 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.764 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.765 DEBUG:boto:Params: {} 2022-09-03T05:31:02.765 DEBUG:boto:Token: None 2022-09-03T05:31:02.766 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.767 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YJ5aQN97m/VSB8cUz7BKUazKnmE= 2022-09-03T05:31:02.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YJ5aQN97m/VSB8cUz7BKUazKnmE=', 'Content-Length': '0'} 2022-09-03T05:31:02.770 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009cf6959f8dda8ccc-006312e696-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.771 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.772 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.772 DEBUG:boto:Method: HEAD 2022-09-03T05:31:02.773 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.773 DEBUG:boto:Data: 2022-09-03T05:31:02.774 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.774 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:02.775 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.776 DEBUG:boto:Params: {} 2022-09-03T05:31:02.776 DEBUG:boto:Token: None 2022-09-03T05:31:02.777 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.778 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YJ5aQN97m/VSB8cUz7BKUazKnmE= 2022-09-03T05:31:02.778 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YJ5aQN97m/VSB8cUz7BKUazKnmE=', 'Content-Length': '0'} 2022-09-03T05:31:02.781 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000969d0050e08df072-006312e696-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.782 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:31:02.782 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.783 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.783 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.784 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.784 DEBUG:boto:Method: GET 2022-09-03T05:31:02.785 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.785 DEBUG:boto:Data: 2022-09-03T05:31:02.786 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.787 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.787 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.788 DEBUG:boto:Params: {} 2022-09-03T05:31:02.788 DEBUG:boto:Token: None 2022-09-03T05:31:02.789 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.789 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eVmf6jtNC/xvIiJhprPoXuwfaCI= 2022-09-03T05:31:02.790 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eVmf6jtNC/xvIiJhprPoXuwfaCI=', 'Content-Length': '0'} 2022-09-03T05:31:02.792 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b1778ef07ab7263e-006312e696-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.834 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.835 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:02.835 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.836 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.836 DEBUG:boto:Method: GET 2022-09-03T05:31:02.837 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.837 DEBUG:boto:Data: 2022-09-03T05:31:02.838 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.839 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:02.840 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.841 DEBUG:boto:Params: {} 2022-09-03T05:31:02.841 DEBUG:boto:Token: None 2022-09-03T05:31:02.842 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:31:02.843 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eVmf6jtNC/xvIiJhprPoXuwfaCI= 2022-09-03T05:31:02.843 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eVmf6jtNC/xvIiJhprPoXuwfaCI=', 'Content-Length': '0'} 2022-09-03T05:31:02.845 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:09 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c6115d5aa3cda750-006312e696-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.846 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:02.847 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:02.848 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:02.848 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:02.849 DEBUG:boto:Method: GET 2022-09-03T05:31:02.850 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:02.850 DEBUG:boto:Data: 2022-09-03T05:31:02.852 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.852 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.853 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.853 DEBUG:boto:Params: {} 2022-09-03T05:31:02.854 DEBUG:boto:Token: None 2022-09-03T05:31:02.855 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/ 2022-09-03T05:31:02.856 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk= 2022-09-03T05:31:02.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk=', 'Content-Length': '0'} 2022-09-03T05:31:02.859 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ee13243e1d17fa42-006312e696-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.860 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:02.861 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:02.862 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:02.862 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:02.863 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:02.863 DEBUG:boto:Method: GET 2022-09-03T05:31:02.864 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:31:02.864 DEBUG:boto:Data: 2022-09-03T05:31:02.865 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.865 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:02.866 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.866 DEBUG:boto:Params: {} 2022-09-03T05:31:02.867 DEBUG:boto:Token: None 2022-09-03T05:31:02.868 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/ 2022-09-03T05:31:02.868 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk= 2022-09-03T05:31:02.869 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:78EDPknLydQMwSCahYIO0fCsyDk=', 'Content-Length': '0'} 2022-09-03T05:31:02.873 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007de29edadacdc5ac-006312e696-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.874 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:02.875 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:31:02.875 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:02.876 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:02.876 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.877 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.877 DEBUG:boto:Method: GET 2022-09-03T05:31:02.878 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.878 DEBUG:boto:Data: 2022-09-03T05:31:02.879 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.879 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.880 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.880 DEBUG:boto:Params: {} 2022-09-03T05:31:02.881 DEBUG:boto:Token: None 2022-09-03T05:31:02.882 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.882 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HJOXK6KsBS9kJebGqNFQjk+/Hxs= 2022-09-03T05:31:02.883 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HJOXK6KsBS9kJebGqNFQjk+/Hxs=', 'Content-Length': '0'} 2022-09-03T05:31:02.893 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000070113ada361a4c35-006312e696-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.894 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:02.896 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:02.897 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.897 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.898 DEBUG:boto:Method: GET 2022-09-03T05:31:02.898 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.899 DEBUG:boto:Data: 2022-09-03T05:31:02.899 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.900 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:02.901 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.902 DEBUG:boto:Params: {} 2022-09-03T05:31:02.902 DEBUG:boto:Token: None 2022-09-03T05:31:02.903 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.904 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HJOXK6KsBS9kJebGqNFQjk+/Hxs= 2022-09-03T05:31:02.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HJOXK6KsBS9kJebGqNFQjk+/Hxs=', 'Content-Length': '0'} 2022-09-03T05:31:02.906 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000082cf756bf91086f0-006312e696-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.907 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:02.908 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:02.908 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.909 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.909 DEBUG:boto:Method: HEAD 2022-09-03T05:31:02.910 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.911 DEBUG:boto:Data: 2022-09-03T05:31:02.911 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.912 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.912 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.913 DEBUG:boto:Params: {} 2022-09-03T05:31:02.913 DEBUG:boto:Token: None 2022-09-03T05:31:02.914 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.914 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MW/Iu02vyLxyFl9OJi6AeTSKqcI= 2022-09-03T05:31:02.915 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MW/Iu02vyLxyFl9OJi6AeTSKqcI=', 'Content-Length': '0'} 2022-09-03T05:31:02.917 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f98b8dc00be40b9a-006312e696-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.918 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:02.918 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:02.919 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.919 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.920 DEBUG:boto:Method: HEAD 2022-09-03T05:31:02.920 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.921 DEBUG:boto:Data: 2022-09-03T05:31:02.921 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.922 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:02.922 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.923 DEBUG:boto:Params: {} 2022-09-03T05:31:02.924 DEBUG:boto:Token: None 2022-09-03T05:31:02.924 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.925 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MW/Iu02vyLxyFl9OJi6AeTSKqcI= 2022-09-03T05:31:02.926 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MW/Iu02vyLxyFl9OJi6AeTSKqcI=', 'Content-Length': '0'} 2022-09-03T05:31:02.928 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c198c8eda4fef438-006312e696-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.929 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:31:02.929 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:02.930 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:02.930 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.931 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.931 DEBUG:boto:Method: GET 2022-09-03T05:31:02.932 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.933 DEBUG:boto:Data: 2022-09-03T05:31:02.933 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.934 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:02.934 DEBUG:boto:Port: 8001 2022-09-03T05:31:02.935 DEBUG:boto:Params: {} 2022-09-03T05:31:02.935 DEBUG:boto:Token: None 2022-09-03T05:31:02.936 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.936 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HJOXK6KsBS9kJebGqNFQjk+/Hxs= 2022-09-03T05:31:02.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HJOXK6KsBS9kJebGqNFQjk+/Hxs=', 'Content-Length': '0'} 2022-09-03T05:31:02.940 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c603ef4dc09b3080-006312e696-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.982 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:02.984 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:02.985 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.986 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.987 DEBUG:boto:Method: GET 2022-09-03T05:31:02.987 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.988 DEBUG:boto:Data: 2022-09-03T05:31:02.989 DEBUG:boto:Headers: {} 2022-09-03T05:31:02.990 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:02.991 DEBUG:boto:Port: 8000 2022-09-03T05:31:02.992 DEBUG:boto:Params: {} 2022-09-03T05:31:02.993 DEBUG:boto:Token: None 2022-09-03T05:31:02.993 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:02 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:02.994 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HJOXK6KsBS9kJebGqNFQjk+/Hxs= 2022-09-03T05:31:02.995 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HJOXK6KsBS9kJebGqNFQjk+/Hxs=', 'Content-Length': '0'} 2022-09-03T05:31:02.997 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003252d38eee340b40-006312e696-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:02.998 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:02.999 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:03.000 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:03.001 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:03.002 DEBUG:boto:Method: GET 2022-09-03T05:31:03.003 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:03.004 DEBUG:boto:Data: 2022-09-03T05:31:03.005 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.006 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:03.006 DEBUG:boto:Port: 8001 2022-09-03T05:31:03.007 DEBUG:boto:Params: {} 2022-09-03T05:31:03.008 DEBUG:boto:Token: None 2022-09-03T05:31:03.008 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/ 2022-09-03T05:31:03.009 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xMesId9zJjQmjhYd22Gt6bf9RRY= 2022-09-03T05:31:03.010 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xMesId9zJjQmjhYd22Gt6bf9RRY=', 'Content-Length': '0'} 2022-09-03T05:31:03.014 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b89f61a16d0aa7bf-006312e697-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.014 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:03.015 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:03.016 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:03.016 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:03.017 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:03.018 DEBUG:boto:Method: GET 2022-09-03T05:31:03.018 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:03.019 DEBUG:boto:Data: 2022-09-03T05:31:03.019 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.020 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:03.020 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.021 DEBUG:boto:Params: {} 2022-09-03T05:31:03.021 DEBUG:boto:Token: None 2022-09-03T05:31:03.022 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/ 2022-09-03T05:31:03.023 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xMesId9zJjQmjhYd22Gt6bf9RRY= 2022-09-03T05:31:03.023 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xMesId9zJjQmjhYd22Gt6bf9RRY=', 'Content-Length': '0'} 2022-09-03T05:31:03.026 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009edb495321eb5b20-006312e697-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.027 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:03.027 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:03.028 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:03.028 DEBUG:boto:Method: HEAD 2022-09-03T05:31:03.029 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2 2022-09-03T05:31:03.029 DEBUG:boto:Data: 2022-09-03T05:31:03.030 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.030 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:03.031 DEBUG:boto:Port: 8001 2022-09-03T05:31:03.031 DEBUG:boto:Params: {} 2022-09-03T05:31:03.032 DEBUG:boto:Token: None 2022-09-03T05:31:03.033 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/obj-test-zone2 2022-09-03T05:31:03.033 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SoyJ2lxq2XDu0aUhexO5LuuJxe8= 2022-09-03T05:31:03.034 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SoyJ2lxq2XDu0aUhexO5LuuJxe8=', 'Content-Length': '0'} 2022-09-03T05:31:03.036 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009a9c807a46fe3d0b-006312e697-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.037 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:03.038 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:03.039 DEBUG:boto:Method: HEAD 2022-09-03T05:31:03.039 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2 2022-09-03T05:31:03.040 DEBUG:boto:Data: 2022-09-03T05:31:03.040 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.041 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:03.041 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.042 DEBUG:boto:Params: {} 2022-09-03T05:31:03.042 DEBUG:boto:Token: None 2022-09-03T05:31:03.043 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/obj-test-zone2 2022-09-03T05:31:03.043 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SoyJ2lxq2XDu0aUhexO5LuuJxe8= 2022-09-03T05:31:03.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SoyJ2lxq2XDu0aUhexO5LuuJxe8=', 'Content-Length': '0'} 2022-09-03T05:31:03.046 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008270fa7b04d7a9e5-006312e697-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.047 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:31:03.048 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:03.049 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:03.049 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:03.050 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:03.050 DEBUG:boto:Method: GET 2022-09-03T05:31:03.051 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:03.051 DEBUG:boto:Data: 2022-09-03T05:31:03.052 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.052 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:03.053 DEBUG:boto:Port: 8001 2022-09-03T05:31:03.053 DEBUG:boto:Params: {} 2022-09-03T05:31:03.054 DEBUG:boto:Token: None 2022-09-03T05:31:03.055 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:03.055 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gbj7TntUPbW+YCSOu42h3+Ut3M0= 2022-09-03T05:31:03.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gbj7TntUPbW+YCSOu42h3+Ut3M0=', 'Content-Length': '0'} 2022-09-03T05:31:03.058 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a57abd86247911a9-006312e697-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.059 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:03.059 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:03.060 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:03.061 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:03.061 DEBUG:boto:Method: GET 2022-09-03T05:31:03.062 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:03.062 DEBUG:boto:Data: 2022-09-03T05:31:03.063 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.063 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:03.064 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.064 DEBUG:boto:Params: {} 2022-09-03T05:31:03.065 DEBUG:boto:Token: None 2022-09-03T05:31:03.066 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:31:03.066 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gbj7TntUPbW+YCSOu42h3+Ut3M0= 2022-09-03T05:31:03.067 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gbj7TntUPbW+YCSOu42h3+Ut3M0=', 'Content-Length': '0'} 2022-09-03T05:31:03.069 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003c7dcb695b748a57-006312e697-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.070 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:03.070 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:03.071 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:03.071 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:03.072 DEBUG:boto:Method: GET 2022-09-03T05:31:03.072 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:03.073 DEBUG:boto:Data: 2022-09-03T05:31:03.073 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.074 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:03.075 DEBUG:boto:Port: 8001 2022-09-03T05:31:03.075 DEBUG:boto:Params: {} 2022-09-03T05:31:03.076 DEBUG:boto:Token: None 2022-09-03T05:31:03.076 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/ 2022-09-03T05:31:03.077 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xMesId9zJjQmjhYd22Gt6bf9RRY= 2022-09-03T05:31:03.078 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xMesId9zJjQmjhYd22Gt6bf9RRY=', 'Content-Length': '0'} 2022-09-03T05:31:03.080 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a6d2b2e98a77ec4d-006312e697-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.081 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:03.082 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:03.082 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:03.083 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:03.083 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:03.084 DEBUG:boto:Method: GET 2022-09-03T05:31:03.085 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:31:03.085 DEBUG:boto:Data: 2022-09-03T05:31:03.086 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.086 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:03.087 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.087 DEBUG:boto:Params: {} 2022-09-03T05:31:03.088 DEBUG:boto:Token: None 2022-09-03T05:31:03.088 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/ 2022-09-03T05:31:03.089 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xMesId9zJjQmjhYd22Gt6bf9RRY= 2022-09-03T05:31:03.090 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xMesId9zJjQmjhYd22Gt6bf9RRY=', 'Content-Length': '0'} 2022-09-03T05:31:03.092 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007afebdd8e8cab221-006312e697-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.093 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:03.093 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:31:03.094 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.095 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.095 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.096 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.096 DEBUG:boto:Method: GET 2022-09-03T05:31:03.097 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.097 DEBUG:boto:Data: 2022-09-03T05:31:03.098 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.098 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:03.099 DEBUG:boto:Port: 8001 2022-09-03T05:31:03.099 DEBUG:boto:Params: {} 2022-09-03T05:31:03.100 DEBUG:boto:Token: None 2022-09-03T05:31:03.101 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.101 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xQ2pyyrSdGMwu2rnArB2oJ/eOUU= 2022-09-03T05:31:03.102 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xQ2pyyrSdGMwu2rnArB2oJ/eOUU=', 'Content-Length': '0'} 2022-09-03T05:31:03.104 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000070a551a228913a9d-006312e697-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.105 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.106 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.106 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.107 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.107 DEBUG:boto:Method: GET 2022-09-03T05:31:03.108 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.108 DEBUG:boto:Data: 2022-09-03T05:31:03.109 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.109 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:03.110 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.111 DEBUG:boto:Params: {} 2022-09-03T05:31:03.111 DEBUG:boto:Token: None 2022-09-03T05:31:03.112 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.112 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xQ2pyyrSdGMwu2rnArB2oJ/eOUU= 2022-09-03T05:31:03.113 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xQ2pyyrSdGMwu2rnArB2oJ/eOUU=', 'Content-Length': '0'} 2022-09-03T05:31:03.115 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000015e8ee448743cf5-006312e697-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.116 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.116 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.117 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.117 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.118 DEBUG:boto:Method: HEAD 2022-09-03T05:31:03.118 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.119 DEBUG:boto:Data: 2022-09-03T05:31:03.119 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.120 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:03.120 DEBUG:boto:Port: 8001 2022-09-03T05:31:03.121 DEBUG:boto:Params: {} 2022-09-03T05:31:03.122 DEBUG:boto:Token: None 2022-09-03T05:31:03.122 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.123 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BHLhRCSymo/hEy9PRAQ8nbpVO1c= 2022-09-03T05:31:03.123 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BHLhRCSymo/hEy9PRAQ8nbpVO1c=', 'Content-Length': '0'} 2022-09-03T05:31:03.126 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000095a8f9d1bab111c2-006312e697-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.126 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.127 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.127 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.128 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.128 DEBUG:boto:Method: HEAD 2022-09-03T05:31:03.129 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.129 DEBUG:boto:Data: 2022-09-03T05:31:03.130 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.130 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:03.131 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.131 DEBUG:boto:Params: {} 2022-09-03T05:31:03.132 DEBUG:boto:Token: None 2022-09-03T05:31:03.133 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.133 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BHLhRCSymo/hEy9PRAQ8nbpVO1c= 2022-09-03T05:31:03.134 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BHLhRCSymo/hEy9PRAQ8nbpVO1c=', 'Content-Length': '0'} 2022-09-03T05:31:03.136 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005d9e940035947045-006312e697-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.136 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:31:03.137 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.138 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.138 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.139 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.139 DEBUG:boto:Method: GET 2022-09-03T05:31:03.140 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.140 DEBUG:boto:Data: 2022-09-03T05:31:03.141 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.141 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:03.142 DEBUG:boto:Port: 8001 2022-09-03T05:31:03.142 DEBUG:boto:Params: {} 2022-09-03T05:31:03.143 DEBUG:boto:Token: None 2022-09-03T05:31:03.144 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.144 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xQ2pyyrSdGMwu2rnArB2oJ/eOUU= 2022-09-03T05:31:03.145 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xQ2pyyrSdGMwu2rnArB2oJ/eOUU=', 'Content-Length': '0'} 2022-09-03T05:31:03.147 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dcc15ff34402c9d0-006312e697-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.148 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.148 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.149 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.149 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.150 DEBUG:boto:Method: GET 2022-09-03T05:31:03.150 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.151 DEBUG:boto:Data: 2022-09-03T05:31:03.151 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.152 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:03.153 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.153 DEBUG:boto:Params: {} 2022-09-03T05:31:03.154 DEBUG:boto:Token: None 2022-09-03T05:31:03.154 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.155 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xQ2pyyrSdGMwu2rnArB2oJ/eOUU= 2022-09-03T05:31:03.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xQ2pyyrSdGMwu2rnArB2oJ/eOUU=', 'Content-Length': '0'} 2022-09-03T05:31:03.158 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009d3d5e9e5b2ae225-006312e697-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.159 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:03.159 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:03.160 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:03.161 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:03.161 DEBUG:boto:Method: GET 2022-09-03T05:31:03.162 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:03.162 DEBUG:boto:Data: 2022-09-03T05:31:03.163 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.163 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:03.164 DEBUG:boto:Port: 8001 2022-09-03T05:31:03.164 DEBUG:boto:Params: {} 2022-09-03T05:31:03.165 DEBUG:boto:Token: None 2022-09-03T05:31:03.166 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/ 2022-09-03T05:31:03.166 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xMesId9zJjQmjhYd22Gt6bf9RRY= 2022-09-03T05:31:03.167 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xMesId9zJjQmjhYd22Gt6bf9RRY=', 'Content-Length': '0'} 2022-09-03T05:31:03.170 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008bd9cc9b60ab1271-006312e697-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.211 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:03.212 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:03.212 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:03.213 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:03.214 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:03.214 DEBUG:boto:Method: GET 2022-09-03T05:31:03.215 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:03.216 DEBUG:boto:Data: 2022-09-03T05:31:03.216 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.217 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:03.217 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.218 DEBUG:boto:Params: {} 2022-09-03T05:31:03.219 DEBUG:boto:Token: None 2022-09-03T05:31:03.220 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/ 2022-09-03T05:31:03.220 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xMesId9zJjQmjhYd22Gt6bf9RRY= 2022-09-03T05:31:03.221 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xMesId9zJjQmjhYd22Gt6bf9RRY=', 'Content-Length': '0'} 2022-09-03T05:31:03.224 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004a668ea0fca2f073-006312e697-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.225 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:03.226 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.226 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.227 DEBUG:boto:Method: HEAD 2022-09-03T05:31:03.227 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.228 DEBUG:boto:Data: 2022-09-03T05:31:03.229 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.230 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:03.230 DEBUG:boto:Port: 8001 2022-09-03T05:31:03.231 DEBUG:boto:Params: {} 2022-09-03T05:31:03.231 DEBUG:boto:Token: None 2022-09-03T05:31:03.232 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.233 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kxvzu33XxwFCemOMiC5JNWzRIy4= 2022-09-03T05:31:03.234 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kxvzu33XxwFCemOMiC5JNWzRIy4=', 'Content-Length': '0'} 2022-09-03T05:31:03.236 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d96aeb878f71db2b-006312e697-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.237 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.239 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.240 DEBUG:boto:Method: HEAD 2022-09-03T05:31:03.241 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.242 DEBUG:boto:Data: 2022-09-03T05:31:03.243 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.244 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:03.245 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.246 DEBUG:boto:Params: {} 2022-09-03T05:31:03.247 DEBUG:boto:Token: None 2022-09-03T05:31:03.248 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.249 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kxvzu33XxwFCemOMiC5JNWzRIy4= 2022-09-03T05:31:03.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kxvzu33XxwFCemOMiC5JNWzRIy4=', 'Content-Length': '0'} 2022-09-03T05:31:03.255 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002736d8dde6d4f5b1-006312e697-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.257 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:31:03.257 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.259 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.259 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.260 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.261 DEBUG:boto:Method: GET 2022-09-03T05:31:03.261 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.262 DEBUG:boto:Data: 2022-09-03T05:31:03.262 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.263 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:03.263 DEBUG:boto:Port: 8001 2022-09-03T05:31:03.264 DEBUG:boto:Params: {} 2022-09-03T05:31:03.264 DEBUG:boto:Token: None 2022-09-03T05:31:03.265 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.266 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xQ2pyyrSdGMwu2rnArB2oJ/eOUU= 2022-09-03T05:31:03.266 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xQ2pyyrSdGMwu2rnArB2oJ/eOUU=', 'Content-Length': '0'} 2022-09-03T05:31:03.268 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008e08b31a022c70bf-006312e697-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.269 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.270 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.270 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.271 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.271 DEBUG:boto:Method: GET 2022-09-03T05:31:03.272 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.273 DEBUG:boto:Data: 2022-09-03T05:31:03.273 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.274 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:03.274 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.275 DEBUG:boto:Params: {} 2022-09-03T05:31:03.275 DEBUG:boto:Token: None 2022-09-03T05:31:03.276 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:31:03.277 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xQ2pyyrSdGMwu2rnArB2oJ/eOUU= 2022-09-03T05:31:03.277 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xQ2pyyrSdGMwu2rnArB2oJ/eOUU=', 'Content-Length': '0'} 2022-09-03T05:31:03.280 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:29:10 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cff32715894cb281-006312e697-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.324 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:03.325 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:03.326 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:03.327 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:03.328 DEBUG:boto:Method: GET 2022-09-03T05:31:03.330 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:03.331 DEBUG:boto:Data: 2022-09-03T05:31:03.332 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.333 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:03.334 DEBUG:boto:Port: 8001 2022-09-03T05:31:03.335 DEBUG:boto:Params: {} 2022-09-03T05:31:03.336 DEBUG:boto:Token: None 2022-09-03T05:31:03.337 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/ 2022-09-03T05:31:03.338 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xMesId9zJjQmjhYd22Gt6bf9RRY= 2022-09-03T05:31:03.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xMesId9zJjQmjhYd22Gt6bf9RRY=', 'Content-Length': '0'} 2022-09-03T05:31:03.347 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000402716160b9aecc2-006312e697-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.349 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:03.351 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:03.352 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:03.353 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:03.354 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:03.355 DEBUG:boto:Method: GET 2022-09-03T05:31:03.356 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:31:03.357 DEBUG:boto:Data: 2022-09-03T05:31:03.358 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.359 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:03.360 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.361 DEBUG:boto:Params: {} 2022-09-03T05:31:03.362 DEBUG:boto:Token: None 2022-09-03T05:31:03.363 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/ 2022-09-03T05:31:03.364 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xMesId9zJjQmjhYd22Gt6bf9RRY= 2022-09-03T05:31:03.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xMesId9zJjQmjhYd22Gt6bf9RRY=', 'Content-Length': '0'} 2022-09-03T05:31:03.368 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007d9cda0512e99f2e-006312e697-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.368 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:03.369 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-18 zones={test-zone3, test-zone2} 2022-09-03T05:31:03.370 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:31:03.370 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:31:03.371 DEBUG:boto:Method: HEAD 2022-09-03T05:31:03.371 DEBUG:boto:Path: /dsgvcy-16/ 2022-09-03T05:31:03.372 DEBUG:boto:Data: 2022-09-03T05:31:03.372 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.373 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:03.374 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.374 DEBUG:boto:Params: {} 2022-09-03T05:31:03.375 DEBUG:boto:Token: None 2022-09-03T05:31:03.375 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-16/ 2022-09-03T05:31:03.376 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pcaQDC2Qs4hobq9UyRTIQSoh0VM= 2022-09-03T05:31:03.376 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pcaQDC2Qs4hobq9UyRTIQSoh0VM=', 'Content-Length': '0'} 2022-09-03T05:31:03.380 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008a3b11382ee4fb84-006312e697-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.381 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:31:03.382 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:31:03.383 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999&prefix=obj-test-zone1 2022-09-03T05:31:03.384 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999&prefix=obj-test-zone1 2022-09-03T05:31:03.385 DEBUG:boto:Method: GET 2022-09-03T05:31:03.386 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999&prefix=obj-test-zone1 2022-09-03T05:31:03.387 DEBUG:boto:Data: 2022-09-03T05:31:03.388 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.388 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:03.389 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.389 DEBUG:boto:Params: {} 2022-09-03T05:31:03.390 DEBUG:boto:Token: None 2022-09-03T05:31:03.391 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-16/?versions 2022-09-03T05:31:03.391 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:br7MAa8r7WDtFgltjoErMFnq+0Y= 2022-09-03T05:31:03.392 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:br7MAa8r7WDtFgltjoErMFnq+0Y=', 'Content-Length': '0'} 2022-09-03T05:31:03.395 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000507c18ee44a29998-006312e697-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.442 DEBUG:boto:b'dsgvcy-16obj-test-zone1999falseobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:03.443 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:31:03.444 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:31:03.444 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?acl 2022-09-03T05:31:03.445 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?acl 2022-09-03T05:31:03.445 DEBUG:boto:Method: PUT 2022-09-03T05:31:03.446 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?acl 2022-09-03T05:31:03.446 DEBUG:boto:Data: 2022-09-03T05:31:03.447 DEBUG:boto:Headers: {'x-amz-acl': 'public-read'} 2022-09-03T05:31:03.447 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:03.448 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.448 DEBUG:boto:Params: {} 2022-09-03T05:31:03.449 DEBUG:boto:Token: None 2022-09-03T05:31:03.450 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:31:03 GMT x-amz-acl:public-read /dsgvcy-16/obj-test-zone1?acl 2022-09-03T05:31:03.450 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ygZcsZ4/mCKsJw//LbA2mT54JXI= 2022-09-03T05:31:03.451 DEBUG:boto:Final headers: {'x-amz-acl': 'public-read', 'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ygZcsZ4/mCKsJw//LbA2mT54JXI=', 'Content-Length': '0'} 2022-09-03T05:31:03.460 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000003a1edf0e1911a49b-006312e697-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.462 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:31:03.463 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:31:03.464 DEBUG:boto:Method: HEAD 2022-09-03T05:31:03.465 DEBUG:boto:Path: /dsgvcy-16/ 2022-09-03T05:31:03.466 DEBUG:boto:Data: 2022-09-03T05:31:03.467 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.467 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:03.468 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.468 DEBUG:boto:Params: {} 2022-09-03T05:31:03.469 DEBUG:boto:Token: None 2022-09-03T05:31:03.470 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-16/ 2022-09-03T05:31:03.470 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pcaQDC2Qs4hobq9UyRTIQSoh0VM= 2022-09-03T05:31:03.471 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pcaQDC2Qs4hobq9UyRTIQSoh0VM=', 'Content-Length': '0'} 2022-09-03T05:31:03.473 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000bb57ccbc9528b2fd-006312e697-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.474 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:31:03.475 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:31:03.475 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999&prefix=obj-test-zone2 2022-09-03T05:31:03.476 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999&prefix=obj-test-zone2 2022-09-03T05:31:03.476 DEBUG:boto:Method: GET 2022-09-03T05:31:03.477 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999&prefix=obj-test-zone2 2022-09-03T05:31:03.477 DEBUG:boto:Data: 2022-09-03T05:31:03.478 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.478 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:03.479 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.480 DEBUG:boto:Params: {} 2022-09-03T05:31:03.481 DEBUG:boto:Token: None 2022-09-03T05:31:03.482 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-16/?versions 2022-09-03T05:31:03.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:br7MAa8r7WDtFgltjoErMFnq+0Y= 2022-09-03T05:31:03.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:br7MAa8r7WDtFgltjoErMFnq+0Y=', 'Content-Length': '0'} 2022-09-03T05:31:03.487 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000094f343be89fe74e4-006312e697-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.488 DEBUG:boto:b'dsgvcy-16obj-test-zone2999falseobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:03.488 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:31:03.489 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:31:03.489 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?acl 2022-09-03T05:31:03.490 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?acl 2022-09-03T05:31:03.490 DEBUG:boto:Method: PUT 2022-09-03T05:31:03.491 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?acl 2022-09-03T05:31:03.491 DEBUG:boto:Data: 2022-09-03T05:31:03.492 DEBUG:boto:Headers: {'x-amz-acl': 'public-read'} 2022-09-03T05:31:03.492 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:03.493 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.493 DEBUG:boto:Params: {} 2022-09-03T05:31:03.494 DEBUG:boto:Token: None 2022-09-03T05:31:03.495 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:31:03 GMT x-amz-acl:public-read /dsgvcy-16/obj-test-zone2?acl 2022-09-03T05:31:03.496 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XptTk81S1mfxmBnXXoRjhTcLfD0= 2022-09-03T05:31:03.497 DEBUG:boto:Final headers: {'x-amz-acl': 'public-read', 'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XptTk81S1mfxmBnXXoRjhTcLfD0=', 'Content-Length': '0'} 2022-09-03T05:31:03.510 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000007b45784daea6773c-006312e697-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.511 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:31:03.511 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:31:03.512 DEBUG:boto:Method: HEAD 2022-09-03T05:31:03.513 DEBUG:boto:Path: /dsgvcy-16/ 2022-09-03T05:31:03.513 DEBUG:boto:Data: 2022-09-03T05:31:03.514 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.514 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:03.515 DEBUG:boto:Port: 8001 2022-09-03T05:31:03.516 DEBUG:boto:Params: {} 2022-09-03T05:31:03.517 DEBUG:boto:Token: None 2022-09-03T05:31:03.517 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-16/ 2022-09-03T05:31:03.518 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pcaQDC2Qs4hobq9UyRTIQSoh0VM= 2022-09-03T05:31:03.518 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pcaQDC2Qs4hobq9UyRTIQSoh0VM=', 'Content-Length': '0'} 2022-09-03T05:31:03.521 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000078eec7ec25559427-006312e697-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.522 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:31:03.523 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:31:03.523 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999&prefix=obj-test-zone3 2022-09-03T05:31:03.523 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999&prefix=obj-test-zone3 2022-09-03T05:31:03.524 DEBUG:boto:Method: GET 2022-09-03T05:31:03.525 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999&prefix=obj-test-zone3 2022-09-03T05:31:03.525 DEBUG:boto:Data: 2022-09-03T05:31:03.526 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.527 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:03.527 DEBUG:boto:Port: 8001 2022-09-03T05:31:03.528 DEBUG:boto:Params: {} 2022-09-03T05:31:03.528 DEBUG:boto:Token: None 2022-09-03T05:31:03.529 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-16/?versions 2022-09-03T05:31:03.530 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:br7MAa8r7WDtFgltjoErMFnq+0Y= 2022-09-03T05:31:03.530 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:br7MAa8r7WDtFgltjoErMFnq+0Y=', 'Content-Length': '0'} 2022-09-03T05:31:03.533 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001ade004d5e541c1c-006312e697-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.534 DEBUG:boto:b'dsgvcy-16obj-test-zone3999falseobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:03.535 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:31:03.535 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:31:03.536 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?acl 2022-09-03T05:31:03.537 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?acl 2022-09-03T05:31:03.537 DEBUG:boto:Method: PUT 2022-09-03T05:31:03.538 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?acl 2022-09-03T05:31:03.538 DEBUG:boto:Data: 2022-09-03T05:31:03.539 DEBUG:boto:Headers: {'x-amz-acl': 'public-read'} 2022-09-03T05:31:03.539 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:03.540 DEBUG:boto:Port: 8001 2022-09-03T05:31:03.540 DEBUG:boto:Params: {} 2022-09-03T05:31:03.541 DEBUG:boto:Token: None 2022-09-03T05:31:03.541 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:31:03 GMT x-amz-acl:public-read /dsgvcy-16/obj-test-zone3?acl 2022-09-03T05:31:03.542 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a1q32z6RT4QIwB+qwoPRFM6DE3A= 2022-09-03T05:31:03.543 DEBUG:boto:Final headers: {'x-amz-acl': 'public-read', 'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a1q32z6RT4QIwB+qwoPRFM6DE3A=', 'Content-Length': '0'} 2022-09-03T05:31:03.555 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000007e9f2e888ed558ba-006312e697-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.556 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:31:03.556 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:31:03.557 DEBUG:boto:Method: HEAD 2022-09-03T05:31:03.558 DEBUG:boto:Path: /dsgvcy-17/ 2022-09-03T05:31:03.558 DEBUG:boto:Data: 2022-09-03T05:31:03.559 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.559 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:03.560 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.560 DEBUG:boto:Params: {} 2022-09-03T05:31:03.561 DEBUG:boto:Token: None 2022-09-03T05:31:03.561 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-17/ 2022-09-03T05:31:03.562 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2R0Q+De5qPplqkg+oliY2JFwdWQ= 2022-09-03T05:31:03.563 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2R0Q+De5qPplqkg+oliY2JFwdWQ=', 'Content-Length': '0'} 2022-09-03T05:31:03.565 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006c559c839fd83fe0-006312e697-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.566 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:31:03.566 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:31:03.567 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999&prefix=obj-test-zone1 2022-09-03T05:31:03.567 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999&prefix=obj-test-zone1 2022-09-03T05:31:03.568 DEBUG:boto:Method: GET 2022-09-03T05:31:03.568 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999&prefix=obj-test-zone1 2022-09-03T05:31:03.569 DEBUG:boto:Data: 2022-09-03T05:31:03.569 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.570 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:03.570 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.571 DEBUG:boto:Params: {} 2022-09-03T05:31:03.571 DEBUG:boto:Token: None 2022-09-03T05:31:03.572 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-17/?versions 2022-09-03T05:31:03.573 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KojEBqMU1H0rRwkBtvO5QECSr5Y= 2022-09-03T05:31:03.573 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KojEBqMU1H0rRwkBtvO5QECSr5Y=', 'Content-Length': '0'} 2022-09-03T05:31:03.576 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005aa0d2dbfe64eb26-006312e697-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.577 DEBUG:boto:b'dsgvcy-17obj-test-zone1999falseobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:03.577 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:31:03.578 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:31:03.578 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?acl 2022-09-03T05:31:03.579 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?acl 2022-09-03T05:31:03.579 DEBUG:boto:Method: PUT 2022-09-03T05:31:03.580 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?acl 2022-09-03T05:31:03.580 DEBUG:boto:Data: 2022-09-03T05:31:03.581 DEBUG:boto:Headers: {'x-amz-acl': 'public-read'} 2022-09-03T05:31:03.581 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:03.582 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.582 DEBUG:boto:Params: {} 2022-09-03T05:31:03.583 DEBUG:boto:Token: None 2022-09-03T05:31:03.584 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:31:03 GMT x-amz-acl:public-read /dsgvcy-17/obj-test-zone1?acl 2022-09-03T05:31:03.584 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1k8bAcehCCjgdqw6dnG+aMf3WXU= 2022-09-03T05:31:03.585 DEBUG:boto:Final headers: {'x-amz-acl': 'public-read', 'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1k8bAcehCCjgdqw6dnG+aMf3WXU=', 'Content-Length': '0'} 2022-09-03T05:31:03.598 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000c16d96b27dce3dfe-006312e697-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.599 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:31:03.600 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:31:03.600 DEBUG:boto:Method: HEAD 2022-09-03T05:31:03.601 DEBUG:boto:Path: /dsgvcy-17/ 2022-09-03T05:31:03.601 DEBUG:boto:Data: 2022-09-03T05:31:03.602 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.602 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:03.603 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.604 DEBUG:boto:Params: {} 2022-09-03T05:31:03.604 DEBUG:boto:Token: None 2022-09-03T05:31:03.605 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-17/ 2022-09-03T05:31:03.605 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2R0Q+De5qPplqkg+oliY2JFwdWQ= 2022-09-03T05:31:03.606 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2R0Q+De5qPplqkg+oliY2JFwdWQ=', 'Content-Length': '0'} 2022-09-03T05:31:03.609 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000125aeb44d4aca56c-006312e697-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.610 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:31:03.610 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:31:03.611 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999&prefix=obj-test-zone2 2022-09-03T05:31:03.611 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999&prefix=obj-test-zone2 2022-09-03T05:31:03.612 DEBUG:boto:Method: GET 2022-09-03T05:31:03.612 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999&prefix=obj-test-zone2 2022-09-03T05:31:03.613 DEBUG:boto:Data: 2022-09-03T05:31:03.613 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.614 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:03.614 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.615 DEBUG:boto:Params: {} 2022-09-03T05:31:03.615 DEBUG:boto:Token: None 2022-09-03T05:31:03.616 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-17/?versions 2022-09-03T05:31:03.617 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KojEBqMU1H0rRwkBtvO5QECSr5Y= 2022-09-03T05:31:03.617 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KojEBqMU1H0rRwkBtvO5QECSr5Y=', 'Content-Length': '0'} 2022-09-03T05:31:03.620 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bd7154e4c13688b4-006312e697-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.621 DEBUG:boto:b'dsgvcy-17obj-test-zone2999falseobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:03.621 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:31:03.622 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:31:03.623 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?acl 2022-09-03T05:31:03.623 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?acl 2022-09-03T05:31:03.624 DEBUG:boto:Method: PUT 2022-09-03T05:31:03.624 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?acl 2022-09-03T05:31:03.625 DEBUG:boto:Data: 2022-09-03T05:31:03.626 DEBUG:boto:Headers: {'x-amz-acl': 'public-read'} 2022-09-03T05:31:03.627 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:03.627 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.628 DEBUG:boto:Params: {} 2022-09-03T05:31:03.628 DEBUG:boto:Token: None 2022-09-03T05:31:03.629 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:31:03 GMT x-amz-acl:public-read /dsgvcy-17/obj-test-zone2?acl 2022-09-03T05:31:03.630 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kyeAGNKFZjarnJZZc86lsbdD3EE= 2022-09-03T05:31:03.630 DEBUG:boto:Final headers: {'x-amz-acl': 'public-read', 'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kyeAGNKFZjarnJZZc86lsbdD3EE=', 'Content-Length': '0'} 2022-09-03T05:31:03.645 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000007a89513e30463d80-006312e697-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.646 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:31:03.647 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:31:03.648 DEBUG:boto:Method: HEAD 2022-09-03T05:31:03.649 DEBUG:boto:Path: /dsgvcy-17/ 2022-09-03T05:31:03.650 DEBUG:boto:Data: 2022-09-03T05:31:03.651 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.652 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:03.652 DEBUG:boto:Port: 8001 2022-09-03T05:31:03.653 DEBUG:boto:Params: {} 2022-09-03T05:31:03.653 DEBUG:boto:Token: None 2022-09-03T05:31:03.654 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-17/ 2022-09-03T05:31:03.655 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2R0Q+De5qPplqkg+oliY2JFwdWQ= 2022-09-03T05:31:03.655 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2R0Q+De5qPplqkg+oliY2JFwdWQ=', 'Content-Length': '0'} 2022-09-03T05:31:03.658 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000d1460a035561d50-006312e697-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.659 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:31:03.660 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:31:03.660 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999&prefix=obj-test-zone3 2022-09-03T05:31:03.661 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999&prefix=obj-test-zone3 2022-09-03T05:31:03.661 DEBUG:boto:Method: GET 2022-09-03T05:31:03.662 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999&prefix=obj-test-zone3 2022-09-03T05:31:03.662 DEBUG:boto:Data: 2022-09-03T05:31:03.663 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.663 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:03.664 DEBUG:boto:Port: 8001 2022-09-03T05:31:03.664 DEBUG:boto:Params: {} 2022-09-03T05:31:03.665 DEBUG:boto:Token: None 2022-09-03T05:31:03.665 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-17/?versions 2022-09-03T05:31:03.666 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KojEBqMU1H0rRwkBtvO5QECSr5Y= 2022-09-03T05:31:03.666 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KojEBqMU1H0rRwkBtvO5QECSr5Y=', 'Content-Length': '0'} 2022-09-03T05:31:03.669 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a7d2e838efcf04cf-006312e697-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.670 DEBUG:boto:b'dsgvcy-17obj-test-zone3999falseobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:03.671 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:31:03.671 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:31:03.672 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?acl 2022-09-03T05:31:03.672 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?acl 2022-09-03T05:31:03.673 DEBUG:boto:Method: PUT 2022-09-03T05:31:03.673 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?acl 2022-09-03T05:31:03.674 DEBUG:boto:Data: 2022-09-03T05:31:03.674 DEBUG:boto:Headers: {'x-amz-acl': 'public-read'} 2022-09-03T05:31:03.675 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:03.675 DEBUG:boto:Port: 8001 2022-09-03T05:31:03.676 DEBUG:boto:Params: {} 2022-09-03T05:31:03.676 DEBUG:boto:Token: None 2022-09-03T05:31:03.677 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:31:03 GMT x-amz-acl:public-read /dsgvcy-17/obj-test-zone3?acl 2022-09-03T05:31:03.678 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m6vwcMNDifcd4jo56qsJz2gzdsA= 2022-09-03T05:31:03.679 DEBUG:boto:Final headers: {'x-amz-acl': 'public-read', 'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m6vwcMNDifcd4jo56qsJz2gzdsA=', 'Content-Length': '0'} 2022-09-03T05:31:03.691 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000037d7dfa29bd7890c-006312e697-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.692 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:03.693 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:03.693 DEBUG:boto:Method: HEAD 2022-09-03T05:31:03.694 DEBUG:boto:Path: /dsgvcy-18/ 2022-09-03T05:31:03.694 DEBUG:boto:Data: 2022-09-03T05:31:03.695 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.695 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:03.696 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.696 DEBUG:boto:Params: {} 2022-09-03T05:31:03.697 DEBUG:boto:Token: None 2022-09-03T05:31:03.697 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/ 2022-09-03T05:31:03.698 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jooLDUwjoYSzjMjfj870NwS1Dig= 2022-09-03T05:31:03.699 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jooLDUwjoYSzjMjfj870NwS1Dig=', 'Content-Length': '0'} 2022-09-03T05:31:03.702 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004e58bcf50e5954c7-006312e697-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.703 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:03.704 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:03.704 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999&prefix=obj-test-zone1 2022-09-03T05:31:03.705 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999&prefix=obj-test-zone1 2022-09-03T05:31:03.706 DEBUG:boto:Method: GET 2022-09-03T05:31:03.706 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999&prefix=obj-test-zone1 2022-09-03T05:31:03.707 DEBUG:boto:Data: 2022-09-03T05:31:03.707 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.708 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:03.709 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.709 DEBUG:boto:Params: {} 2022-09-03T05:31:03.710 DEBUG:boto:Token: None 2022-09-03T05:31:03.710 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/?versions 2022-09-03T05:31:03.711 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VOZVOIdQh4cI7YDv4Ks2AbVZPnY= 2022-09-03T05:31:03.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VOZVOIdQh4cI7YDv4Ks2AbVZPnY=', 'Content-Length': '0'} 2022-09-03T05:31:03.714 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004a8d385850a8a480-006312e697-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.715 DEBUG:boto:b'dsgvcy-18obj-test-zone1999falseobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:03.716 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:03.716 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:31:03.717 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?acl 2022-09-03T05:31:03.717 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?acl 2022-09-03T05:31:03.718 DEBUG:boto:Method: PUT 2022-09-03T05:31:03.718 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?acl 2022-09-03T05:31:03.719 DEBUG:boto:Data: 2022-09-03T05:31:03.719 DEBUG:boto:Headers: {'x-amz-acl': 'public-read'} 2022-09-03T05:31:03.720 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:31:03.720 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.721 DEBUG:boto:Params: {} 2022-09-03T05:31:03.721 DEBUG:boto:Token: None 2022-09-03T05:31:03.722 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:31:03 GMT x-amz-acl:public-read /dsgvcy-18/obj-test-zone1?acl 2022-09-03T05:31:03.722 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7i7TXzj7RPO699okjg18dtPlyi0= 2022-09-03T05:31:03.723 DEBUG:boto:Final headers: {'x-amz-acl': 'public-read', 'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7i7TXzj7RPO699okjg18dtPlyi0=', 'Content-Length': '0'} 2022-09-03T05:31:03.735 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000db1c5ad258af6996-006312e697-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.736 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:03.737 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:03.737 DEBUG:boto:Method: HEAD 2022-09-03T05:31:03.738 DEBUG:boto:Path: /dsgvcy-18/ 2022-09-03T05:31:03.739 DEBUG:boto:Data: 2022-09-03T05:31:03.739 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.740 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:03.741 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.741 DEBUG:boto:Params: {} 2022-09-03T05:31:03.752 DEBUG:boto:Token: None 2022-09-03T05:31:03.756 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/ 2022-09-03T05:31:03.758 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jooLDUwjoYSzjMjfj870NwS1Dig= 2022-09-03T05:31:03.759 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jooLDUwjoYSzjMjfj870NwS1Dig=', 'Content-Length': '0'} 2022-09-03T05:31:03.763 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008bb028e385da9716-006312e697-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.765 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:03.766 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:03.766 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999&prefix=obj-test-zone2 2022-09-03T05:31:03.767 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999&prefix=obj-test-zone2 2022-09-03T05:31:03.768 DEBUG:boto:Method: GET 2022-09-03T05:31:03.769 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999&prefix=obj-test-zone2 2022-09-03T05:31:03.770 DEBUG:boto:Data: 2022-09-03T05:31:03.770 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.771 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:03.771 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.772 DEBUG:boto:Params: {} 2022-09-03T05:31:03.773 DEBUG:boto:Token: None 2022-09-03T05:31:03.775 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/?versions 2022-09-03T05:31:03.775 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VOZVOIdQh4cI7YDv4Ks2AbVZPnY= 2022-09-03T05:31:03.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VOZVOIdQh4cI7YDv4Ks2AbVZPnY=', 'Content-Length': '0'} 2022-09-03T05:31:03.778 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eec55e8d84738474-006312e697-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.779 DEBUG:boto:b'dsgvcy-18obj-test-zone2999falseobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:03.780 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:03.781 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:31:03.781 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?acl 2022-09-03T05:31:03.782 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?acl 2022-09-03T05:31:03.782 DEBUG:boto:Method: PUT 2022-09-03T05:31:03.783 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?acl 2022-09-03T05:31:03.783 DEBUG:boto:Data: 2022-09-03T05:31:03.784 DEBUG:boto:Headers: {'x-amz-acl': 'public-read'} 2022-09-03T05:31:03.784 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:31:03.785 DEBUG:boto:Port: 8000 2022-09-03T05:31:03.786 DEBUG:boto:Params: {} 2022-09-03T05:31:03.786 DEBUG:boto:Token: None 2022-09-03T05:31:03.787 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:31:03 GMT x-amz-acl:public-read /dsgvcy-18/obj-test-zone2?acl 2022-09-03T05:31:03.788 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/kYvvInAX4BmTLIWyS6Rt0d+zdk= 2022-09-03T05:31:03.788 DEBUG:boto:Final headers: {'x-amz-acl': 'public-read', 'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/kYvvInAX4BmTLIWyS6Rt0d+zdk=', 'Content-Length': '0'} 2022-09-03T05:31:03.799 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000036f36783f05533d1-006312e697-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.801 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:03.802 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:03.803 DEBUG:boto:Method: HEAD 2022-09-03T05:31:03.803 DEBUG:boto:Path: /dsgvcy-18/ 2022-09-03T05:31:03.805 DEBUG:boto:Data: 2022-09-03T05:31:03.806 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.807 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:03.808 DEBUG:boto:Port: 8001 2022-09-03T05:31:03.809 DEBUG:boto:Params: {} 2022-09-03T05:31:03.810 DEBUG:boto:Token: None 2022-09-03T05:31:03.811 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/ 2022-09-03T05:31:03.811 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jooLDUwjoYSzjMjfj870NwS1Dig= 2022-09-03T05:31:03.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jooLDUwjoYSzjMjfj870NwS1Dig=', 'Content-Length': '0'} 2022-09-03T05:31:03.814 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000020bf82c87748cab2-006312e697-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.815 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:31:03.815 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:31:03.816 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999&prefix=obj-test-zone3 2022-09-03T05:31:03.817 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999&prefix=obj-test-zone3 2022-09-03T05:31:03.817 DEBUG:boto:Method: GET 2022-09-03T05:31:03.818 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999&prefix=obj-test-zone3 2022-09-03T05:31:03.819 DEBUG:boto:Data: 2022-09-03T05:31:03.819 DEBUG:boto:Headers: {} 2022-09-03T05:31:03.820 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:03.820 DEBUG:boto:Port: 8001 2022-09-03T05:31:03.821 DEBUG:boto:Params: {} 2022-09-03T05:31:03.821 DEBUG:boto:Token: None 2022-09-03T05:31:03.822 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:31:03 GMT /dsgvcy-18/?versions 2022-09-03T05:31:03.822 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VOZVOIdQh4cI7YDv4Ks2AbVZPnY= 2022-09-03T05:31:03.823 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VOZVOIdQh4cI7YDv4Ks2AbVZPnY=', 'Content-Length': '0'} 2022-09-03T05:31:03.830 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000392f9cbbe4f9ca3-006312e697-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.870 DEBUG:boto:b'dsgvcy-18obj-test-zone3999falseobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:31:03.872 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.872 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:31:03.873 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?acl 2022-09-03T05:31:03.873 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?acl 2022-09-03T05:31:03.874 DEBUG:boto:Method: PUT 2022-09-03T05:31:03.875 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?acl 2022-09-03T05:31:03.876 DEBUG:boto:Data: 2022-09-03T05:31:03.877 DEBUG:boto:Headers: {'x-amz-acl': 'public-read'} 2022-09-03T05:31:03.877 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:31:03.879 DEBUG:boto:Port: 8001 2022-09-03T05:31:03.880 DEBUG:boto:Params: {} 2022-09-03T05:31:03.882 DEBUG:boto:Token: None 2022-09-03T05:31:03.883 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:31:03 GMT x-amz-acl:public-read /dsgvcy-18/obj-test-zone3?acl 2022-09-03T05:31:03.884 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zp/dkktzz3/qQcaJl31qQpXdkMY= 2022-09-03T05:31:03.885 DEBUG:boto:Final headers: {'x-amz-acl': 'public-read', 'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:31:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zp/dkktzz3/qQcaJl31qQpXdkMY=', 'Content-Length': '0'} 2022-09-03T05:31:03.897 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000d9a3c42cb6bd2bbd-006312e697-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:31:03.898 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-16 2022-09-03T05:31:03.898 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:31:03.900 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:31:03.901 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-16 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:31:03.968 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:31:03.968 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:31:04.009 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:31:04.011+0000 7f2501f99600 1 waiting for incremental sync to catch up: 2022-09-03T05:31:04.010 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000010.42.5, , 00000000003.8.9, , 00000000007.19.9, , , , , ] 2022-09-03T05:31:04.010 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000011.45.5, , 00000000003.8.9, , 00000000007.19.9, , , , , ] 2022-09-03T05:31:09.011 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:31:09.013+0000 7f2501f99600 1 bucket sync caught up with source: 2022-09-03T05:31:09.011 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000011.45.5, , 00000000003.8.9, , 00000000008.22.5, , , , , ] 2022-09-03T05:31:09.012 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000011.45.5, , 00000000003.8.9, , 00000000007.19.9, , , , , ] 2022-09-03T05:31:09.012 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:31:09.013+0000 7f2501f99600 0 bucket checkpoint complete 2022-09-03T05:31:09.029 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-16 2022-09-03T05:31:09.030 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:31:09.031 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:31:09.032 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-16 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:31:09.097 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:31:09.097 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:31:09.141 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:31:09.143+0000 7fb5d21e4600 1 bucket sync caught up with source: 2022-09-03T05:31:09.142 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000011.45.5, , 00000000003.8.9, , 00000000008.22.5, , , , , ] 2022-09-03T05:31:09.142 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000011.45.5, , 00000000003.8.9, , 00000000008.22.5, , , , , ] 2022-09-03T05:31:09.143 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:31:09.143+0000 7fb5d21e4600 0 bucket checkpoint complete 2022-09-03T05:31:09.163 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-16 2022-09-03T05:31:09.164 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:31:09.165 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:31:09.166 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-16 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:31:09.232 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:31:09.232 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:31:09.279 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:31:09.281+0000 7f6657798600 1 waiting for incremental sync to catch up: 2022-09-03T05:31:09.280 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000010.42.5, , 00000000003.8.9, , 00000000007.19.9, , , , , ] 2022-09-03T05:31:09.280 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000011.45.5, , 00000000003.8.9, , 00000000008.22.5, , , , , ] 2022-09-03T05:31:14.281 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:31:14.283+0000 7f6657798600 1 waiting for incremental sync to catch up: 2022-09-03T05:31:14.282 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000010.42.5, , 00000000003.8.9, , 00000000007.19.9, , , , , ] 2022-09-03T05:31:14.282 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000011.45.5, , 00000000003.8.9, , 00000000008.22.5, , , , , ] 2022-09-03T05:31:19.282 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:31:19.284+0000 7f6657798600 1 bucket sync caught up with source: 2022-09-03T05:31:19.283 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000011.45.5, , 00000000003.8.9, , 00000000008.22.5, , , , , ] 2022-09-03T05:31:19.284 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000011.45.5, , 00000000003.8.9, , 00000000008.22.5, , , , , ] 2022-09-03T05:31:19.284 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:31:19.284+0000 7f6657798600 0 bucket checkpoint complete 2022-09-03T05:31:19.298 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-16 2022-09-03T05:31:19.299 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:31:19.300 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:31:19.300 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-16 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:31:19.364 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:31:19.364 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:31:19.400 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:31:19.402+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:31:19.401 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:31:19.401 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:31:24.402 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:31:24.404+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:31:24.403 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:31:24.403 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:31:29.404 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:31:29.404+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:31:29.404 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:31:29.407 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:31:34.405 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:31:34.406+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:31:34.405 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:31:34.406 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:31:39.406 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:31:39.408+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:31:39.407 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:31:39.407 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:31:44.408 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:31:44.409+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:31:44.409 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:31:44.410 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:31:49.412 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:31:49.411+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:31:49.412 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:31:49.413 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:31:54.411 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:31:54.413+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:31:54.412 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:31:54.412 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:31:59.413 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:31:59.415+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:31:59.414 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:31:59.414 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:04.414 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:32:04.416+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:32:04.415 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:04.415 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:09.415 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:32:09.418+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:32:09.416 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:09.416 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:14.417 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:32:14.419+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:32:14.418 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:14.418 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:19.418 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:32:19.419+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:32:19.419 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:19.419 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:24.425 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:32:24.421+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:32:24.425 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:24.426 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:29.422 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:32:29.422+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:32:29.422 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:29.422 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:34.424 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:32:34.424+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:32:34.425 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:34.425 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:39.424 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:32:39.426+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:32:39.425 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:39.426 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:44.426 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:32:44.428+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:32:44.426 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:44.427 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:49.427 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:32:49.429+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:32:49.428 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:49.428 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:54.433 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:32:54.430+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:32:54.434 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:54.434 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:59.430 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:32:59.432+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:32:59.431 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:32:59.431 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:33:04.432 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:04.434+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:33:04.432 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:33:04.433 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:33:09.434 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:09.434+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:33:09.434 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:33:09.435 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:33:14.435 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:14.435+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:33:14.435 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:33:14.435 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:33:19.436 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:19.437+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:33:19.437 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:33:19.437 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:33:24.437 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:24.438+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:33:24.438 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:33:24.438 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:33:29.439 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:29.440+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:33:29.439 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:33:29.440 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:33:34.441 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:34.442+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:33:34.442 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:33:34.442 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:33:39.442 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:39.443+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:33:39.443 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:33:39.444 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:33:44.444 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:44.445+0000 7f65a0d3a600 1 waiting for incremental sync to catch up: 2022-09-03T05:33:44.445 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000010.37.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:33:44.445 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:33:49.445 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:49.446+0000 7f65a0d3a600 1 bucket sync caught up with source: 2022-09-03T05:33:49.446 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:33:49.446 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:33:49.446 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:49.446+0000 7f65a0d3a600 0 bucket checkpoint complete 2022-09-03T05:33:49.468 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-16 2022-09-03T05:33:49.469 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:49.470 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:49.471 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-16 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:33:49.535 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:33:49.535 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:33:49.583 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:49.585+0000 7f2e2823b600 1 bucket sync caught up with source: 2022-09-03T05:33:49.584 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:33:49.585 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.73.5, , 00000000011.40.5, , 00000000008.37.5, , , , , ] 2022-09-03T05:33:49.585 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:49.585+0000 7f2e2823b600 0 bucket checkpoint complete 2022-09-03T05:33:49.609 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-16 2022-09-03T05:33:49.610 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-16 2022-09-03T05:33:49.611 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:49.612 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:49.613 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-16 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:33:49.676 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:33:49.677 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:33:49.728 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:49.730+0000 7f20be3e9600 1 bucket sync caught up with source: 2022-09-03T05:33:49.729 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000005.65.5, , 00000000004.18.5, , 00000000015.40.5, , , , , ] 2022-09-03T05:33:49.729 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000005.65.5, , 00000000004.18.5, , 00000000015.40.5, , , , , ] 2022-09-03T05:33:49.730 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:49.730+0000 7f20be3e9600 0 bucket checkpoint complete 2022-09-03T05:33:49.742 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-16 2022-09-03T05:33:49.743 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:49.743 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-16', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:49.744 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-16 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:33:49.808 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:33:49.809 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:33:49.857 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:33:49.858+0000 7fb6da072600 1 bucket sync caught up with source: 2022-09-03T05:33:49.857 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000005.65.5, , 00000000004.18.5, , 00000000015.40.5, , , , , ] 2022-09-03T05:33:49.858 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000005.65.5, , 00000000004.18.5, , 00000000015.40.5, , , , , ] 2022-09-03T05:33:49.858 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:33:49.858+0000 7fb6da072600 0 bucket checkpoint complete 2022-09-03T05:33:49.871 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-16 2022-09-03T05:33:49.872 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-16 zones={test-zone2, test-zone1} 2022-09-03T05:33:49.874 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:49.875 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:49.876 DEBUG:boto:Method: HEAD 2022-09-03T05:33:49.877 DEBUG:boto:Path: /dsgvcy-16/ 2022-09-03T05:33:49.878 DEBUG:boto:Data: 2022-09-03T05:33:49.879 DEBUG:boto:Headers: {} 2022-09-03T05:33:49.880 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:49.881 DEBUG:boto:Port: 8000 2022-09-03T05:33:49.882 DEBUG:boto:Params: {} 2022-09-03T05:33:49.884 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:33:49.884 DEBUG:boto:Token: None 2022-09-03T05:33:49.885 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:49 GMT /dsgvcy-16/ 2022-09-03T05:33:49.886 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hdA5yWpumdSsXA68KPfFCkG4DyU= 2022-09-03T05:33:49.887 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hdA5yWpumdSsXA68KPfFCkG4DyU=', 'Content-Length': '0'} 2022-09-03T05:33:49.891 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000001d7462fb124dbc4-006312e73d-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:33:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:49.892 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:49.893 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:49.893 DEBUG:boto:Method: HEAD 2022-09-03T05:33:49.894 DEBUG:boto:Path: /dsgvcy-16/ 2022-09-03T05:33:49.895 DEBUG:boto:Data: 2022-09-03T05:33:49.895 DEBUG:boto:Headers: {} 2022-09-03T05:33:49.896 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:49.897 DEBUG:boto:Port: 8000 2022-09-03T05:33:49.897 DEBUG:boto:Params: {} 2022-09-03T05:33:49.898 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:33:49.898 DEBUG:boto:Token: None 2022-09-03T05:33:49.899 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:49 GMT /dsgvcy-16/ 2022-09-03T05:33:49.900 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hdA5yWpumdSsXA68KPfFCkG4DyU= 2022-09-03T05:33:49.900 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hdA5yWpumdSsXA68KPfFCkG4DyU=', 'Content-Length': '0'} 2022-09-03T05:33:49.904 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000409ad41ce74d2e99-006312e73d-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:33:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:49.905 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:33:49.906 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:49.906 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:49.907 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:49.907 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:49.908 DEBUG:boto:Method: GET 2022-09-03T05:33:49.908 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:49.909 DEBUG:boto:Data: 2022-09-03T05:33:49.909 DEBUG:boto:Headers: {} 2022-09-03T05:33:49.910 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:49.910 DEBUG:boto:Port: 8000 2022-09-03T05:33:49.911 DEBUG:boto:Params: {} 2022-09-03T05:33:49.911 DEBUG:boto:Token: None 2022-09-03T05:33:49.912 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:49 GMT /dsgvcy-16/?versions 2022-09-03T05:33:49.913 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bou8m7YC76aQZOme5JaSr4xof+U= 2022-09-03T05:33:49.913 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bou8m7YC76aQZOme5JaSr4xof+U=', 'Content-Length': '0'} 2022-09-03T05:33:49.916 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000f2af33bdce2548d-006312e73d-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:49.916 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:49.918 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:33:49.919 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:33:49.919 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:33:49.920 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:33:49.920 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:33:49.921 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:49.921 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:49.922 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:49.922 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:49.923 DEBUG:boto:Method: GET 2022-09-03T05:33:49.923 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:49.924 DEBUG:boto:Data: 2022-09-03T05:33:49.924 DEBUG:boto:Headers: {} 2022-09-03T05:33:49.925 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:49.925 DEBUG:boto:Port: 8000 2022-09-03T05:33:49.926 DEBUG:boto:Params: {} 2022-09-03T05:33:49.926 DEBUG:boto:Token: None 2022-09-03T05:33:49.927 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:49 GMT /dsgvcy-16/?versions 2022-09-03T05:33:49.928 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bou8m7YC76aQZOme5JaSr4xof+U= 2022-09-03T05:33:49.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bou8m7YC76aQZOme5JaSr4xof+U=', 'Content-Length': '0'} 2022-09-03T05:33:49.931 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e7c67c5e1107cd0c-006312e73d-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:49.932 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:49.933 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:33:49.933 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:33:49.934 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:33:49.934 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:33:49.935 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:49.935 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:49.936 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:49.936 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:49.937 DEBUG:boto:Method: GET 2022-09-03T05:33:49.937 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:49.938 DEBUG:boto:Data: 2022-09-03T05:33:49.939 DEBUG:boto:Headers: {} 2022-09-03T05:33:49.939 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:49.940 DEBUG:boto:Port: 8000 2022-09-03T05:33:49.940 DEBUG:boto:Params: {} 2022-09-03T05:33:49.941 DEBUG:boto:Token: None 2022-09-03T05:33:49.941 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:49 GMT /dsgvcy-16/?versions 2022-09-03T05:33:49.942 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bou8m7YC76aQZOme5JaSr4xof+U= 2022-09-03T05:33:49.943 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bou8m7YC76aQZOme5JaSr4xof+U=', 'Content-Length': '0'} 2022-09-03T05:33:49.945 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000491b3b8e115a32a1-006312e73d-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:49.946 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:49.947 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:49.947 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:49.948 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:49.948 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:49.949 DEBUG:boto:Method: GET 2022-09-03T05:33:49.949 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:49.950 DEBUG:boto:Data: 2022-09-03T05:33:49.950 DEBUG:boto:Headers: {} 2022-09-03T05:33:49.951 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:49.951 DEBUG:boto:Port: 8000 2022-09-03T05:33:49.952 DEBUG:boto:Params: {} 2022-09-03T05:33:49.952 DEBUG:boto:Token: None 2022-09-03T05:33:49.953 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:49 GMT /dsgvcy-16/?versions 2022-09-03T05:33:49.954 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bou8m7YC76aQZOme5JaSr4xof+U= 2022-09-03T05:33:49.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bou8m7YC76aQZOme5JaSr4xof+U=', 'Content-Length': '0'} 2022-09-03T05:33:49.957 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007aca3f79fcf38d75-006312e73d-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:49.958 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:49.959 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:33:49.960 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:49.961 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:49.961 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:49.962 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:49.962 DEBUG:boto:Method: GET 2022-09-03T05:33:49.963 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:49.964 DEBUG:boto:Data: 2022-09-03T05:33:49.964 DEBUG:boto:Headers: {} 2022-09-03T05:33:49.965 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:49.966 DEBUG:boto:Port: 8000 2022-09-03T05:33:49.967 DEBUG:boto:Params: {} 2022-09-03T05:33:49.967 DEBUG:boto:Token: None 2022-09-03T05:33:49.968 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:49 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:49.968 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:P2zqpI9tDvvLVVmJD7t3HLiQRpw= 2022-09-03T05:33:49.969 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:P2zqpI9tDvvLVVmJD7t3HLiQRpw=', 'Content-Length': '0'} 2022-09-03T05:33:49.971 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000009de75bcc08dc9c20-006312e73d-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:49.973 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:49.974 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:49.975 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:49.976 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:49.977 DEBUG:boto:Method: GET 2022-09-03T05:33:49.978 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:49.979 DEBUG:boto:Data: 2022-09-03T05:33:49.980 DEBUG:boto:Headers: {} 2022-09-03T05:33:49.981 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:49.982 DEBUG:boto:Port: 8000 2022-09-03T05:33:49.983 DEBUG:boto:Params: {} 2022-09-03T05:33:49.984 DEBUG:boto:Token: None 2022-09-03T05:33:49.985 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:49 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:49.986 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:P2zqpI9tDvvLVVmJD7t3HLiQRpw= 2022-09-03T05:33:49.987 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:P2zqpI9tDvvLVVmJD7t3HLiQRpw=', 'Content-Length': '0'} 2022-09-03T05:33:49.990 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000defd76b2a9055df6-006312e73d-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:49.991 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:49.992 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:49.992 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:49.993 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:49.993 DEBUG:boto:Method: HEAD 2022-09-03T05:33:49.994 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:49.994 DEBUG:boto:Data: 2022-09-03T05:33:49.995 DEBUG:boto:Headers: {} 2022-09-03T05:33:49.995 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:49.996 DEBUG:boto:Port: 8000 2022-09-03T05:33:49.997 DEBUG:boto:Params: {} 2022-09-03T05:33:49.998 DEBUG:boto:Token: None 2022-09-03T05:33:49.999 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:49 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:50.000 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3ncagSrMN6PuYIE2PzDGsAqS5Ew= 2022-09-03T05:33:50.001 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3ncagSrMN6PuYIE2PzDGsAqS5Ew=', 'Content-Length': '0'} 2022-09-03T05:33:50.004 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000005cd6b3be340bd664-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.005 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:50.006 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:50.007 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:50.009 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:50.010 DEBUG:boto:Method: HEAD 2022-09-03T05:33:50.011 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:50.011 DEBUG:boto:Data: 2022-09-03T05:33:50.012 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.013 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.014 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.015 DEBUG:boto:Params: {} 2022-09-03T05:33:50.016 DEBUG:boto:Token: None 2022-09-03T05:33:50.016 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:50.017 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q0p0Wq0Po6ZOVOPtvfAnlS+yu38= 2022-09-03T05:33:50.018 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q0p0Wq0Po6ZOVOPtvfAnlS+yu38=', 'Content-Length': '0'} 2022-09-03T05:33:50.020 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000002f3bd339c4627083-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.021 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:33:50.022 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:50.023 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:50.025 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:50.025 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:50.026 DEBUG:boto:Method: GET 2022-09-03T05:33:50.027 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:50.028 DEBUG:boto:Data: 2022-09-03T05:33:50.029 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.030 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.031 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.032 DEBUG:boto:Params: {} 2022-09-03T05:33:50.033 DEBUG:boto:Token: None 2022-09-03T05:33:50.035 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:50.036 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hVuyeG+26CYnV3Y5LB1dORHZxlA= 2022-09-03T05:33:50.037 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hVuyeG+26CYnV3Y5LB1dORHZxlA=', 'Content-Length': '0'} 2022-09-03T05:33:50.039 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a88d94422bcd5b28-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.040 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:50.041 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:50.041 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:50.042 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:50.043 DEBUG:boto:Method: GET 2022-09-03T05:33:50.044 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:50.045 DEBUG:boto:Data: 2022-09-03T05:33:50.046 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.046 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.047 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.047 DEBUG:boto:Params: {} 2022-09-03T05:33:50.048 DEBUG:boto:Token: None 2022-09-03T05:33:50.048 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:50.049 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hVuyeG+26CYnV3Y5LB1dORHZxlA= 2022-09-03T05:33:50.050 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hVuyeG+26CYnV3Y5LB1dORHZxlA=', 'Content-Length': '0'} 2022-09-03T05:33:50.052 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000057889314b0bfa374-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.052 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:50.053 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:50.054 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:50.054 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:50.055 DEBUG:boto:Method: GET 2022-09-03T05:33:50.055 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:33:50.056 DEBUG:boto:Data: 2022-09-03T05:33:50.056 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.057 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.057 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.058 DEBUG:boto:Params: {} 2022-09-03T05:33:50.059 DEBUG:boto:Token: None 2022-09-03T05:33:50.059 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/ 2022-09-03T05:33:50.060 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g= 2022-09-03T05:33:50.061 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g=', 'Content-Length': '0'} 2022-09-03T05:33:50.063 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007d3ac6d446bdff9f-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.064 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:50.065 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:50.065 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:50.066 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:50.066 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:50.067 DEBUG:boto:Method: GET 2022-09-03T05:33:50.067 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:33:50.068 DEBUG:boto:Data: 2022-09-03T05:33:50.068 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.069 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.070 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.070 DEBUG:boto:Params: {} 2022-09-03T05:33:50.071 DEBUG:boto:Token: None 2022-09-03T05:33:50.071 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/ 2022-09-03T05:33:50.072 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g= 2022-09-03T05:33:50.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g=', 'Content-Length': '0'} 2022-09-03T05:33:50.075 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000082884a987993eb04-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.076 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:50.077 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:50.077 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:50.078 DEBUG:boto:Method: HEAD 2022-09-03T05:33:50.078 DEBUG:boto:Path: /dsgvcy-16/dummy 2022-09-03T05:33:50.079 DEBUG:boto:Data: 2022-09-03T05:33:50.079 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.080 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.080 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.081 DEBUG:boto:Params: {} 2022-09-03T05:33:50.081 DEBUG:boto:Token: None 2022-09-03T05:33:50.082 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/dummy 2022-09-03T05:33:50.082 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZSdnfjs49Pf7/wJDf2PgXVjjE0c= 2022-09-03T05:33:50.083 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZSdnfjs49Pf7/wJDf2PgXVjjE0c=', 'Content-Length': '0'} 2022-09-03T05:33:50.085 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000d7b50231fa75a633-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.086 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:50.087 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:50.087 DEBUG:boto:Method: HEAD 2022-09-03T05:33:50.088 DEBUG:boto:Path: /dsgvcy-16/dummy 2022-09-03T05:33:50.088 DEBUG:boto:Data: 2022-09-03T05:33:50.089 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.089 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.090 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.090 DEBUG:boto:Params: {} 2022-09-03T05:33:50.091 DEBUG:boto:Token: None 2022-09-03T05:33:50.091 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/dummy 2022-09-03T05:33:50.092 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZSdnfjs49Pf7/wJDf2PgXVjjE0c= 2022-09-03T05:33:50.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZSdnfjs49Pf7/wJDf2PgXVjjE0c=', 'Content-Length': '0'} 2022-09-03T05:33:50.095 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a1145b5afaa46a12-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.096 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:33:50.096 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:50.097 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:50.097 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:50.098 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:50.098 DEBUG:boto:Method: GET 2022-09-03T05:33:50.099 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:50.099 DEBUG:boto:Data: 2022-09-03T05:33:50.100 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.100 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.101 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.101 DEBUG:boto:Params: {} 2022-09-03T05:33:50.102 DEBUG:boto:Token: None 2022-09-03T05:33:50.103 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:50.103 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hVuyeG+26CYnV3Y5LB1dORHZxlA= 2022-09-03T05:33:50.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hVuyeG+26CYnV3Y5LB1dORHZxlA=', 'Content-Length': '0'} 2022-09-03T05:33:50.107 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000d5301278b95cad24-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.108 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:50.109 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:50.110 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:50.111 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:50.111 DEBUG:boto:Method: GET 2022-09-03T05:33:50.112 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:50.112 DEBUG:boto:Data: 2022-09-03T05:33:50.113 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.113 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.114 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.115 DEBUG:boto:Params: {} 2022-09-03T05:33:50.115 DEBUG:boto:Token: None 2022-09-03T05:33:50.116 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:50.117 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hVuyeG+26CYnV3Y5LB1dORHZxlA= 2022-09-03T05:33:50.117 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hVuyeG+26CYnV3Y5LB1dORHZxlA=', 'Content-Length': '0'} 2022-09-03T05:33:50.119 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000447e36b5627bbc2b-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.120 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:50.121 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:50.121 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:50.122 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:50.123 DEBUG:boto:Method: GET 2022-09-03T05:33:50.123 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:33:50.124 DEBUG:boto:Data: 2022-09-03T05:33:50.124 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.125 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.126 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.127 DEBUG:boto:Params: {} 2022-09-03T05:33:50.128 DEBUG:boto:Token: None 2022-09-03T05:33:50.129 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/ 2022-09-03T05:33:50.130 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g= 2022-09-03T05:33:50.131 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g=', 'Content-Length': '0'} 2022-09-03T05:33:50.133 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000081ce85f4b44126dd-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.174 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:50.175 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:50.176 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:50.177 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:50.177 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:50.179 DEBUG:boto:Method: GET 2022-09-03T05:33:50.180 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:33:50.181 DEBUG:boto:Data: 2022-09-03T05:33:50.181 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.182 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.182 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.183 DEBUG:boto:Params: {} 2022-09-03T05:33:50.183 DEBUG:boto:Token: None 2022-09-03T05:33:50.184 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/ 2022-09-03T05:33:50.185 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g= 2022-09-03T05:33:50.186 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g=', 'Content-Length': '0'} 2022-09-03T05:33:50.188 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d5e69d2757180fd0-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.190 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:50.191 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:33:50.192 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.193 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.194 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.195 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.196 DEBUG:boto:Method: GET 2022-09-03T05:33:50.197 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.198 DEBUG:boto:Data: 2022-09-03T05:33:50.199 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.199 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.200 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.200 DEBUG:boto:Params: {} 2022-09-03T05:33:50.201 DEBUG:boto:Token: None 2022-09-03T05:33:50.202 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.202 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y3TTkuHXT1gFo9nXUdDQSLApQwA= 2022-09-03T05:33:50.203 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y3TTkuHXT1gFo9nXUdDQSLApQwA=', 'Content-Length': '0'} 2022-09-03T05:33:50.205 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b0dc4a1cd99d05fc-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.206 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.207 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.207 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.208 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.208 DEBUG:boto:Method: GET 2022-09-03T05:33:50.209 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.209 DEBUG:boto:Data: 2022-09-03T05:33:50.210 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.210 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.211 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.211 DEBUG:boto:Params: {} 2022-09-03T05:33:50.212 DEBUG:boto:Token: None 2022-09-03T05:33:50.213 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.213 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y3TTkuHXT1gFo9nXUdDQSLApQwA= 2022-09-03T05:33:50.214 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y3TTkuHXT1gFo9nXUdDQSLApQwA=', 'Content-Length': '0'} 2022-09-03T05:33:50.216 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001e9cbe840ecd40da-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.218 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.219 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.219 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.220 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.220 DEBUG:boto:Method: HEAD 2022-09-03T05:33:50.221 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.221 DEBUG:boto:Data: 2022-09-03T05:33:50.222 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.222 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.223 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.223 DEBUG:boto:Params: {} 2022-09-03T05:33:50.224 DEBUG:boto:Token: None 2022-09-03T05:33:50.225 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.226 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Iq/ofqnDNR+ATdDVlZGktynRFeo= 2022-09-03T05:33:50.227 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Iq/ofqnDNR+ATdDVlZGktynRFeo=', 'Content-Length': '0'} 2022-09-03T05:33:50.229 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d94b806cf28e55e2-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.229 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.230 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.231 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.231 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.232 DEBUG:boto:Method: HEAD 2022-09-03T05:33:50.232 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.233 DEBUG:boto:Data: 2022-09-03T05:33:50.234 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.235 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.235 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.236 DEBUG:boto:Params: {} 2022-09-03T05:33:50.236 DEBUG:boto:Token: None 2022-09-03T05:33:50.237 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.237 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Iq/ofqnDNR+ATdDVlZGktynRFeo= 2022-09-03T05:33:50.238 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Iq/ofqnDNR+ATdDVlZGktynRFeo=', 'Content-Length': '0'} 2022-09-03T05:33:50.241 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e63587d770004910-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.242 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:33:50.242 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.243 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.243 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.244 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.245 DEBUG:boto:Method: GET 2022-09-03T05:33:50.245 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.246 DEBUG:boto:Data: 2022-09-03T05:33:50.246 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.247 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.247 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.248 DEBUG:boto:Params: {} 2022-09-03T05:33:50.249 DEBUG:boto:Token: None 2022-09-03T05:33:50.249 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.250 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y3TTkuHXT1gFo9nXUdDQSLApQwA= 2022-09-03T05:33:50.250 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y3TTkuHXT1gFo9nXUdDQSLApQwA=', 'Content-Length': '0'} 2022-09-03T05:33:50.252 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d223a15879e0551e-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.253 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.254 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.254 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.255 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.256 DEBUG:boto:Method: GET 2022-09-03T05:33:50.256 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.257 DEBUG:boto:Data: 2022-09-03T05:33:50.257 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.258 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.259 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.260 DEBUG:boto:Params: {} 2022-09-03T05:33:50.260 DEBUG:boto:Token: None 2022-09-03T05:33:50.261 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.262 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y3TTkuHXT1gFo9nXUdDQSLApQwA= 2022-09-03T05:33:50.262 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y3TTkuHXT1gFo9nXUdDQSLApQwA=', 'Content-Length': '0'} 2022-09-03T05:33:50.264 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e7ea69effdfc2f65-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.265 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:50.266 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:50.267 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:50.267 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:50.268 DEBUG:boto:Method: GET 2022-09-03T05:33:50.268 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:50.269 DEBUG:boto:Data: 2022-09-03T05:33:50.269 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.270 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.270 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.271 DEBUG:boto:Params: {} 2022-09-03T05:33:50.272 DEBUG:boto:Token: None 2022-09-03T05:33:50.273 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/ 2022-09-03T05:33:50.275 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g= 2022-09-03T05:33:50.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g=', 'Content-Length': '0'} 2022-09-03T05:33:50.279 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000822eca1912f3874d-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.280 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:50.282 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:50.283 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:50.284 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:50.285 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:50.285 DEBUG:boto:Method: GET 2022-09-03T05:33:50.286 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:50.287 DEBUG:boto:Data: 2022-09-03T05:33:50.288 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.289 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.290 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.291 DEBUG:boto:Params: {} 2022-09-03T05:33:50.292 DEBUG:boto:Token: None 2022-09-03T05:33:50.293 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/ 2022-09-03T05:33:50.294 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g= 2022-09-03T05:33:50.295 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g=', 'Content-Length': '0'} 2022-09-03T05:33:50.298 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000073fe233e1380a924-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.299 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:50.300 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.301 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.301 DEBUG:boto:Method: HEAD 2022-09-03T05:33:50.302 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.302 DEBUG:boto:Data: 2022-09-03T05:33:50.303 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.303 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.304 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.304 DEBUG:boto:Params: {} 2022-09-03T05:33:50.305 DEBUG:boto:Token: None 2022-09-03T05:33:50.306 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.306 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:u0DNj7wThfWEWByXUpHQ/chsEvE= 2022-09-03T05:33:50.307 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:u0DNj7wThfWEWByXUpHQ/chsEvE=', 'Content-Length': '0'} 2022-09-03T05:33:50.310 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009ef87b329ae12459-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.311 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.313 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.314 DEBUG:boto:Method: HEAD 2022-09-03T05:33:50.315 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.316 DEBUG:boto:Data: 2022-09-03T05:33:50.317 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.318 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.318 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.319 DEBUG:boto:Params: {} 2022-09-03T05:33:50.320 DEBUG:boto:Token: None 2022-09-03T05:33:50.320 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.321 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:u0DNj7wThfWEWByXUpHQ/chsEvE= 2022-09-03T05:33:50.322 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:u0DNj7wThfWEWByXUpHQ/chsEvE=', 'Content-Length': '0'} 2022-09-03T05:33:50.324 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d00cdf3439f386ec-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.325 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:33:50.325 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.326 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.326 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.327 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.327 DEBUG:boto:Method: GET 2022-09-03T05:33:50.328 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.329 DEBUG:boto:Data: 2022-09-03T05:33:50.329 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.330 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.330 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.331 DEBUG:boto:Params: {} 2022-09-03T05:33:50.331 DEBUG:boto:Token: None 2022-09-03T05:33:50.332 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.333 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y3TTkuHXT1gFo9nXUdDQSLApQwA= 2022-09-03T05:33:50.333 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y3TTkuHXT1gFo9nXUdDQSLApQwA=', 'Content-Length': '0'} 2022-09-03T05:33:50.335 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000018859970742346d2-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.336 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.337 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:50.337 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.338 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.338 DEBUG:boto:Method: GET 2022-09-03T05:33:50.339 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.339 DEBUG:boto:Data: 2022-09-03T05:33:50.340 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.340 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.341 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.341 DEBUG:boto:Params: {} 2022-09-03T05:33:50.342 DEBUG:boto:Token: None 2022-09-03T05:33:50.343 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:50.343 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y3TTkuHXT1gFo9nXUdDQSLApQwA= 2022-09-03T05:33:50.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y3TTkuHXT1gFo9nXUdDQSLApQwA=', 'Content-Length': '0'} 2022-09-03T05:33:50.346 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000026fc5100e26356ed-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.347 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:50.348 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:50.348 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:50.349 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:50.350 DEBUG:boto:Method: GET 2022-09-03T05:33:50.350 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:50.351 DEBUG:boto:Data: 2022-09-03T05:33:50.351 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.352 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.352 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.353 DEBUG:boto:Params: {} 2022-09-03T05:33:50.354 DEBUG:boto:Token: None 2022-09-03T05:33:50.354 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/ 2022-09-03T05:33:50.355 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g= 2022-09-03T05:33:50.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g=', 'Content-Length': '0'} 2022-09-03T05:33:50.358 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001c3439ece0bd6416-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.359 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:50.360 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:50.360 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:50.361 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:50.361 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:50.362 DEBUG:boto:Method: GET 2022-09-03T05:33:50.362 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:50.363 DEBUG:boto:Data: 2022-09-03T05:33:50.364 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.364 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.365 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.365 DEBUG:boto:Params: {} 2022-09-03T05:33:50.366 DEBUG:boto:Token: None 2022-09-03T05:33:50.367 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/ 2022-09-03T05:33:50.367 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g= 2022-09-03T05:33:50.368 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g=', 'Content-Length': '0'} 2022-09-03T05:33:50.370 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005423102cc36b10e8-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.371 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:50.372 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:33:50.373 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.373 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.374 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.374 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.375 DEBUG:boto:Method: GET 2022-09-03T05:33:50.376 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.376 DEBUG:boto:Data: 2022-09-03T05:33:50.377 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.377 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.378 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.378 DEBUG:boto:Params: {} 2022-09-03T05:33:50.379 DEBUG:boto:Token: None 2022-09-03T05:33:50.380 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.380 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rl7nQdAHZbKGKHHYOQgrC19EO00= 2022-09-03T05:33:50.381 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rl7nQdAHZbKGKHHYOQgrC19EO00=', 'Content-Length': '0'} 2022-09-03T05:33:50.383 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000071ecd935219ea56b-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.430 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.431 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.432 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.432 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.433 DEBUG:boto:Method: GET 2022-09-03T05:33:50.433 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.434 DEBUG:boto:Data: 2022-09-03T05:33:50.435 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.435 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.436 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.436 DEBUG:boto:Params: {} 2022-09-03T05:33:50.437 DEBUG:boto:Token: None 2022-09-03T05:33:50.438 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.438 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rl7nQdAHZbKGKHHYOQgrC19EO00= 2022-09-03T05:33:50.439 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rl7nQdAHZbKGKHHYOQgrC19EO00=', 'Content-Length': '0'} 2022-09-03T05:33:50.441 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000534c95e844daaabf-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.442 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.443 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.443 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.444 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.444 DEBUG:boto:Method: HEAD 2022-09-03T05:33:50.445 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.445 DEBUG:boto:Data: 2022-09-03T05:33:50.446 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.447 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.447 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.448 DEBUG:boto:Params: {} 2022-09-03T05:33:50.449 DEBUG:boto:Token: None 2022-09-03T05:33:50.449 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.450 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AMVQHEWdOfI+rgp62cq4mJ4/m08= 2022-09-03T05:33:50.450 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AMVQHEWdOfI+rgp62cq4mJ4/m08=', 'Content-Length': '0'} 2022-09-03T05:33:50.453 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000016f9d1857a6e3a4e-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.453 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.454 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.455 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.455 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.456 DEBUG:boto:Method: HEAD 2022-09-03T05:33:50.456 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.457 DEBUG:boto:Data: 2022-09-03T05:33:50.457 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.458 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.459 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.459 DEBUG:boto:Params: {} 2022-09-03T05:33:50.460 DEBUG:boto:Token: None 2022-09-03T05:33:50.461 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.461 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AMVQHEWdOfI+rgp62cq4mJ4/m08= 2022-09-03T05:33:50.462 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AMVQHEWdOfI+rgp62cq4mJ4/m08=', 'Content-Length': '0'} 2022-09-03T05:33:50.464 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f0cad4ee659b6191-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.465 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:33:50.466 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.467 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.467 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.468 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.468 DEBUG:boto:Method: GET 2022-09-03T05:33:50.469 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.470 DEBUG:boto:Data: 2022-09-03T05:33:50.470 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.471 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.471 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.472 DEBUG:boto:Params: {} 2022-09-03T05:33:50.473 DEBUG:boto:Token: None 2022-09-03T05:33:50.473 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.474 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rl7nQdAHZbKGKHHYOQgrC19EO00= 2022-09-03T05:33:50.475 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rl7nQdAHZbKGKHHYOQgrC19EO00=', 'Content-Length': '0'} 2022-09-03T05:33:50.478 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008296a28c54baf1f5-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.480 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.481 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.482 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.483 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.484 DEBUG:boto:Method: GET 2022-09-03T05:33:50.485 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.485 DEBUG:boto:Data: 2022-09-03T05:33:50.487 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.488 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.488 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.489 DEBUG:boto:Params: {} 2022-09-03T05:33:50.489 DEBUG:boto:Token: None 2022-09-03T05:33:50.490 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.492 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rl7nQdAHZbKGKHHYOQgrC19EO00= 2022-09-03T05:33:50.493 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rl7nQdAHZbKGKHHYOQgrC19EO00=', 'Content-Length': '0'} 2022-09-03T05:33:50.496 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000f1b3777cb40c1d5-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.497 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:50.497 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:50.498 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:50.499 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:50.499 DEBUG:boto:Method: GET 2022-09-03T05:33:50.500 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:50.500 DEBUG:boto:Data: 2022-09-03T05:33:50.501 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.501 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.502 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.502 DEBUG:boto:Params: {} 2022-09-03T05:33:50.503 DEBUG:boto:Token: None 2022-09-03T05:33:50.504 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/ 2022-09-03T05:33:50.504 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g= 2022-09-03T05:33:50.505 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g=', 'Content-Length': '0'} 2022-09-03T05:33:50.509 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000970d3dee708758ec-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.510 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:50.511 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:50.512 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:50.512 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:50.513 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:50.513 DEBUG:boto:Method: GET 2022-09-03T05:33:50.514 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:50.515 DEBUG:boto:Data: 2022-09-03T05:33:50.515 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.516 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.516 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.517 DEBUG:boto:Params: {} 2022-09-03T05:33:50.518 DEBUG:boto:Token: None 2022-09-03T05:33:50.518 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/ 2022-09-03T05:33:50.519 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g= 2022-09-03T05:33:50.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g=', 'Content-Length': '0'} 2022-09-03T05:33:50.523 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e1800438b12ddb45-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.527 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:50.528 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.529 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.530 DEBUG:boto:Method: HEAD 2022-09-03T05:33:50.531 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.531 DEBUG:boto:Data: 2022-09-03T05:33:50.532 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.532 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.533 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.533 DEBUG:boto:Params: {} 2022-09-03T05:33:50.534 DEBUG:boto:Token: None 2022-09-03T05:33:50.535 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.536 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8v30DoDshvln2+IlzE5ETzYSB0o= 2022-09-03T05:33:50.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8v30DoDshvln2+IlzE5ETzYSB0o=', 'Content-Length': '0'} 2022-09-03T05:33:50.539 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007cc6fa6ee3acf951-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.539 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.540 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.541 DEBUG:boto:Method: HEAD 2022-09-03T05:33:50.541 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.542 DEBUG:boto:Data: 2022-09-03T05:33:50.542 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.543 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.543 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.544 DEBUG:boto:Params: {} 2022-09-03T05:33:50.544 DEBUG:boto:Token: None 2022-09-03T05:33:50.545 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.545 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8v30DoDshvln2+IlzE5ETzYSB0o= 2022-09-03T05:33:50.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8v30DoDshvln2+IlzE5ETzYSB0o=', 'Content-Length': '0'} 2022-09-03T05:33:50.548 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000029b348074baf1517-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.549 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:33:50.550 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.550 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.551 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.551 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.552 DEBUG:boto:Method: GET 2022-09-03T05:33:50.552 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.553 DEBUG:boto:Data: 2022-09-03T05:33:50.553 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.554 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.554 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.555 DEBUG:boto:Params: {} 2022-09-03T05:33:50.556 DEBUG:boto:Token: None 2022-09-03T05:33:50.556 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.557 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rl7nQdAHZbKGKHHYOQgrC19EO00= 2022-09-03T05:33:50.557 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rl7nQdAHZbKGKHHYOQgrC19EO00=', 'Content-Length': '0'} 2022-09-03T05:33:50.560 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000fb8cf7c9d990143-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.602 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.603 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:50.604 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.604 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.605 DEBUG:boto:Method: GET 2022-09-03T05:33:50.605 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.606 DEBUG:boto:Data: 2022-09-03T05:33:50.607 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.607 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.608 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.608 DEBUG:boto:Params: {} 2022-09-03T05:33:50.609 DEBUG:boto:Token: None 2022-09-03T05:33:50.610 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:50.611 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rl7nQdAHZbKGKHHYOQgrC19EO00= 2022-09-03T05:33:50.612 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rl7nQdAHZbKGKHHYOQgrC19EO00=', 'Content-Length': '0'} 2022-09-03T05:33:50.626 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000545be094667c458d-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.641 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:50.643 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:50.644 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:50.645 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:50.646 DEBUG:boto:Method: GET 2022-09-03T05:33:50.647 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:50.648 DEBUG:boto:Data: 2022-09-03T05:33:50.649 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.650 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.651 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.652 DEBUG:boto:Params: {} 2022-09-03T05:33:50.653 DEBUG:boto:Token: None 2022-09-03T05:33:50.654 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/ 2022-09-03T05:33:50.655 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g= 2022-09-03T05:33:50.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g=', 'Content-Length': '0'} 2022-09-03T05:33:50.659 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bd33897c104342c0-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.660 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:50.662 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:50.663 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:50.664 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:50.666 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:50.667 DEBUG:boto:Method: GET 2022-09-03T05:33:50.667 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:50.668 DEBUG:boto:Data: 2022-09-03T05:33:50.669 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.670 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.671 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.672 DEBUG:boto:Params: {} 2022-09-03T05:33:50.673 DEBUG:boto:Token: None 2022-09-03T05:33:50.674 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/ 2022-09-03T05:33:50.675 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g= 2022-09-03T05:33:50.676 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g=', 'Content-Length': '0'} 2022-09-03T05:33:50.684 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c2de3ad88c5d2474-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.686 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:50.688 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:33:50.689 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.690 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.691 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.692 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.693 DEBUG:boto:Method: GET 2022-09-03T05:33:50.694 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.695 DEBUG:boto:Data: 2022-09-03T05:33:50.696 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.697 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.697 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.699 DEBUG:boto:Params: {} 2022-09-03T05:33:50.700 DEBUG:boto:Token: None 2022-09-03T05:33:50.701 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.702 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MVX/PyZCUIShCxtMxuZfWsnZfkc= 2022-09-03T05:33:50.703 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MVX/PyZCUIShCxtMxuZfWsnZfkc=', 'Content-Length': '0'} 2022-09-03T05:33:50.706 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005a6e0c91743f79e9-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.707 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.707 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.708 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.708 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.709 DEBUG:boto:Method: GET 2022-09-03T05:33:50.710 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.710 DEBUG:boto:Data: 2022-09-03T05:33:50.711 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.711 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.712 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.712 DEBUG:boto:Params: {} 2022-09-03T05:33:50.713 DEBUG:boto:Token: None 2022-09-03T05:33:50.713 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.714 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MVX/PyZCUIShCxtMxuZfWsnZfkc= 2022-09-03T05:33:50.715 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MVX/PyZCUIShCxtMxuZfWsnZfkc=', 'Content-Length': '0'} 2022-09-03T05:33:50.717 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000076ba8ea4c8ed6cb9-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.718 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.719 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.720 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.720 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.721 DEBUG:boto:Method: HEAD 2022-09-03T05:33:50.722 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.722 DEBUG:boto:Data: 2022-09-03T05:33:50.723 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.724 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.725 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.725 DEBUG:boto:Params: {} 2022-09-03T05:33:50.726 DEBUG:boto:Token: None 2022-09-03T05:33:50.727 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.727 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mb0aTEFqmQB/UyDvlUk/OX9hMOA= 2022-09-03T05:33:50.728 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mb0aTEFqmQB/UyDvlUk/OX9hMOA=', 'Content-Length': '0'} 2022-09-03T05:33:50.730 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003462d28ab64eb636-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.731 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.732 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.732 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.733 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.733 DEBUG:boto:Method: HEAD 2022-09-03T05:33:50.734 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.735 DEBUG:boto:Data: 2022-09-03T05:33:50.736 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.737 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.738 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.739 DEBUG:boto:Params: {} 2022-09-03T05:33:50.740 DEBUG:boto:Token: None 2022-09-03T05:33:50.741 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.742 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mb0aTEFqmQB/UyDvlUk/OX9hMOA= 2022-09-03T05:33:50.743 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mb0aTEFqmQB/UyDvlUk/OX9hMOA=', 'Content-Length': '0'} 2022-09-03T05:33:50.751 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000037e6273bee9bc7fa-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.753 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:33:50.755 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.756 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.756 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.758 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.759 DEBUG:boto:Method: GET 2022-09-03T05:33:50.760 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.761 DEBUG:boto:Data: 2022-09-03T05:33:50.762 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.763 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.764 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.765 DEBUG:boto:Params: {} 2022-09-03T05:33:50.766 DEBUG:boto:Token: None 2022-09-03T05:33:50.767 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.768 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MVX/PyZCUIShCxtMxuZfWsnZfkc= 2022-09-03T05:33:50.769 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MVX/PyZCUIShCxtMxuZfWsnZfkc=', 'Content-Length': '0'} 2022-09-03T05:33:50.772 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009a0ae1a5989f9589-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.773 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.774 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.775 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.776 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.777 DEBUG:boto:Method: GET 2022-09-03T05:33:50.778 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.779 DEBUG:boto:Data: 2022-09-03T05:33:50.779 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.780 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.781 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.782 DEBUG:boto:Params: {} 2022-09-03T05:33:50.783 DEBUG:boto:Token: None 2022-09-03T05:33:50.785 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.786 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MVX/PyZCUIShCxtMxuZfWsnZfkc= 2022-09-03T05:33:50.787 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MVX/PyZCUIShCxtMxuZfWsnZfkc=', 'Content-Length': '0'} 2022-09-03T05:33:50.791 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ae1897cce8cc8806-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.792 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:50.793 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:50.794 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:50.795 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:50.796 DEBUG:boto:Method: GET 2022-09-03T05:33:50.796 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:50.797 DEBUG:boto:Data: 2022-09-03T05:33:50.798 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.800 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.801 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.801 DEBUG:boto:Params: {} 2022-09-03T05:33:50.802 DEBUG:boto:Token: None 2022-09-03T05:33:50.803 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/ 2022-09-03T05:33:50.804 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g= 2022-09-03T05:33:50.805 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g=', 'Content-Length': '0'} 2022-09-03T05:33:50.808 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000009368b8293286652-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.809 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:50.811 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:50.812 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:50.812 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:50.813 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:50.814 DEBUG:boto:Method: GET 2022-09-03T05:33:50.815 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:50.816 DEBUG:boto:Data: 2022-09-03T05:33:50.817 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.818 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.819 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.820 DEBUG:boto:Params: {} 2022-09-03T05:33:50.820 DEBUG:boto:Token: None 2022-09-03T05:33:50.821 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/ 2022-09-03T05:33:50.822 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g= 2022-09-03T05:33:50.823 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g=', 'Content-Length': '0'} 2022-09-03T05:33:50.831 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004aa890e0d6de5737-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.870 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:50.871 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.872 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.872 DEBUG:boto:Method: HEAD 2022-09-03T05:33:50.873 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.873 DEBUG:boto:Data: 2022-09-03T05:33:50.874 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.874 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.875 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.875 DEBUG:boto:Params: {} 2022-09-03T05:33:50.876 DEBUG:boto:Token: None 2022-09-03T05:33:50.876 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.877 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rPERqk0tDPxhPCfxZCR9dTMu7KQ= 2022-09-03T05:33:50.878 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rPERqk0tDPxhPCfxZCR9dTMu7KQ=', 'Content-Length': '0'} 2022-09-03T05:33:50.880 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008e6f9aabc86b175d-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.881 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.881 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.882 DEBUG:boto:Method: HEAD 2022-09-03T05:33:50.882 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.883 DEBUG:boto:Data: 2022-09-03T05:33:50.884 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.885 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.885 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.886 DEBUG:boto:Params: {} 2022-09-03T05:33:50.887 DEBUG:boto:Token: None 2022-09-03T05:33:50.888 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.889 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rPERqk0tDPxhPCfxZCR9dTMu7KQ= 2022-09-03T05:33:50.890 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rPERqk0tDPxhPCfxZCR9dTMu7KQ=', 'Content-Length': '0'} 2022-09-03T05:33:50.893 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000175deabf50463379-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.894 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:33:50.895 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.895 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.896 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.897 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.897 DEBUG:boto:Method: GET 2022-09-03T05:33:50.898 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.898 DEBUG:boto:Data: 2022-09-03T05:33:50.899 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.899 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.900 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.901 DEBUG:boto:Params: {} 2022-09-03T05:33:50.901 DEBUG:boto:Token: None 2022-09-03T05:33:50.902 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.902 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MVX/PyZCUIShCxtMxuZfWsnZfkc= 2022-09-03T05:33:50.903 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MVX/PyZCUIShCxtMxuZfWsnZfkc=', 'Content-Length': '0'} 2022-09-03T05:33:50.905 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c0dcb051ae08025f-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.907 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.907 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:50.908 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.909 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.910 DEBUG:boto:Method: GET 2022-09-03T05:33:50.910 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.912 DEBUG:boto:Data: 2022-09-03T05:33:50.912 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.913 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.914 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.915 DEBUG:boto:Params: {} 2022-09-03T05:33:50.916 DEBUG:boto:Token: None 2022-09-03T05:33:50.917 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:50.918 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MVX/PyZCUIShCxtMxuZfWsnZfkc= 2022-09-03T05:33:50.919 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MVX/PyZCUIShCxtMxuZfWsnZfkc=', 'Content-Length': '0'} 2022-09-03T05:33:50.921 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000574dab5cfb258007-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.922 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:50.923 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:50.924 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:50.925 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:50.925 DEBUG:boto:Method: GET 2022-09-03T05:33:50.926 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:50.927 DEBUG:boto:Data: 2022-09-03T05:33:50.927 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.928 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:50.929 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.930 DEBUG:boto:Params: {} 2022-09-03T05:33:50.931 DEBUG:boto:Token: None 2022-09-03T05:33:50.931 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/ 2022-09-03T05:33:50.932 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g= 2022-09-03T05:33:50.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g=', 'Content-Length': '0'} 2022-09-03T05:33:50.935 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002bb0d4bf9dfc2df7-006312e73e-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.936 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:50.937 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:50.938 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:50.938 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:50.939 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:50.939 DEBUG:boto:Method: GET 2022-09-03T05:33:50.940 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:50.941 DEBUG:boto:Data: 2022-09-03T05:33:50.942 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.942 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.943 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.944 DEBUG:boto:Params: {} 2022-09-03T05:33:50.944 DEBUG:boto:Token: None 2022-09-03T05:33:50.946 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/ 2022-09-03T05:33:50.946 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g= 2022-09-03T05:33:50.947 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YD6Of1OxcSgjQ378WFNJxhjKg2g=', 'Content-Length': '0'} 2022-09-03T05:33:50.950 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000021ff18d55d72d2bc-006312e73e-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.950 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:50.951 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-16 zones={test-zone2, test-zone1} 2022-09-03T05:33:50.952 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-16 zones={test-zone3, test-zone1} 2022-09-03T05:33:50.952 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:50.953 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:50.953 DEBUG:boto:Method: HEAD 2022-09-03T05:33:50.954 DEBUG:boto:Path: /dsgvcy-16/ 2022-09-03T05:33:50.954 DEBUG:boto:Data: 2022-09-03T05:33:50.955 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.956 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:50.956 DEBUG:boto:Port: 8001 2022-09-03T05:33:50.957 DEBUG:boto:Params: {} 2022-09-03T05:33:50.957 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T05:33:50.958 DEBUG:boto:Token: None 2022-09-03T05:33:50.958 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/ 2022-09-03T05:33:50.959 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v07xWeIJQSxpQ2vGP+X9IP8+0wY= 2022-09-03T05:33:50.960 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v07xWeIJQSxpQ2vGP+X9IP8+0wY=', 'Content-Length': '0'} 2022-09-03T05:33:50.963 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000594a19dc3aa8b16c-006312e73e-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.964 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:50.964 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:50.965 DEBUG:boto:Method: HEAD 2022-09-03T05:33:50.965 DEBUG:boto:Path: /dsgvcy-16/ 2022-09-03T05:33:50.966 DEBUG:boto:Data: 2022-09-03T05:33:50.967 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.967 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:50.968 DEBUG:boto:Port: 8000 2022-09-03T05:33:50.968 DEBUG:boto:Params: {} 2022-09-03T05:33:50.969 DEBUG:boto:Token: None 2022-09-03T05:33:50.969 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/ 2022-09-03T05:33:50.970 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v07xWeIJQSxpQ2vGP+X9IP8+0wY= 2022-09-03T05:33:50.971 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v07xWeIJQSxpQ2vGP+X9IP8+0wY=', 'Content-Length': '0'} 2022-09-03T05:33:50.973 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009ea6e97ec5bde9fd-006312e73e-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.974 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:33:50.975 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:50.977 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:50.978 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:50.978 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:50.979 DEBUG:boto:Method: GET 2022-09-03T05:33:50.980 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:50.980 DEBUG:boto:Data: 2022-09-03T05:33:50.981 DEBUG:boto:Headers: {} 2022-09-03T05:33:50.981 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:50.982 DEBUG:boto:Port: 8001 2022-09-03T05:33:50.983 DEBUG:boto:Params: {} 2022-09-03T05:33:50.983 DEBUG:boto:Token: None 2022-09-03T05:33:50.984 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:50 GMT /dsgvcy-16/?versions 2022-09-03T05:33:50.985 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:W5ml4EdNNw8SiZnf5lpw1uDcIpM= 2022-09-03T05:33:50.985 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:W5ml4EdNNw8SiZnf5lpw1uDcIpM=', 'Content-Length': '0'} 2022-09-03T05:33:50.989 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a671c59955032124-006312e73e-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:50.990 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:50.991 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:33:50.992 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:33:50.993 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:33:50.994 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:33:50.995 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:33:50.996 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:50.997 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:50.998 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:50.999 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:51.000 DEBUG:boto:Method: GET 2022-09-03T05:33:51.000 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:51.001 DEBUG:boto:Data: 2022-09-03T05:33:51.002 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.003 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.004 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.005 DEBUG:boto:Params: {} 2022-09-03T05:33:51.006 DEBUG:boto:Token: None 2022-09-03T05:33:51.007 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/?versions 2022-09-03T05:33:51.008 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H2dnfBy4COws1Qn1ZAR1dW+pFeM= 2022-09-03T05:33:51.009 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H2dnfBy4COws1Qn1ZAR1dW+pFeM=', 'Content-Length': '0'} 2022-09-03T05:33:51.012 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bd53a33b86b77116-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.014 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:51.015 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:33:51.016 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:33:51.017 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:33:51.018 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:33:51.019 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:51.020 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:51.021 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:51.021 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:51.022 DEBUG:boto:Method: GET 2022-09-03T05:33:51.023 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:51.024 DEBUG:boto:Data: 2022-09-03T05:33:51.025 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.026 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.027 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.028 DEBUG:boto:Params: {} 2022-09-03T05:33:51.029 DEBUG:boto:Token: None 2022-09-03T05:33:51.029 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/?versions 2022-09-03T05:33:51.032 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H2dnfBy4COws1Qn1ZAR1dW+pFeM= 2022-09-03T05:33:51.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H2dnfBy4COws1Qn1ZAR1dW+pFeM=', 'Content-Length': '0'} 2022-09-03T05:33:51.035 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003073484bff527ef6-006312e73f-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.036 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:51.037 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:51.037 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:51.038 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:51.039 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:51.039 DEBUG:boto:Method: GET 2022-09-03T05:33:51.040 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:51.040 DEBUG:boto:Data: 2022-09-03T05:33:51.041 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.041 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.042 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.042 DEBUG:boto:Params: {} 2022-09-03T05:33:51.043 DEBUG:boto:Token: None 2022-09-03T05:33:51.043 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/?versions 2022-09-03T05:33:51.044 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H2dnfBy4COws1Qn1ZAR1dW+pFeM= 2022-09-03T05:33:51.045 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H2dnfBy4COws1Qn1ZAR1dW+pFeM=', 'Content-Length': '0'} 2022-09-03T05:33:51.047 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005f9aeccbe9f63fa1-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.047 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:51.048 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:33:51.049 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:51.050 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:51.050 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.051 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.051 DEBUG:boto:Method: GET 2022-09-03T05:33:51.052 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.052 DEBUG:boto:Data: 2022-09-03T05:33:51.053 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.053 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.054 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.054 DEBUG:boto:Params: {} 2022-09-03T05:33:51.055 DEBUG:boto:Token: None 2022-09-03T05:33:51.056 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.056 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+d8FKzgtSy0LsjrEJjdfqkO6xWw= 2022-09-03T05:33:51.057 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+d8FKzgtSy0LsjrEJjdfqkO6xWw=', 'Content-Length': '0'} 2022-09-03T05:33:51.059 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000006aa836436865efa8-006312e73f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.060 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:51.060 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:51.061 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.061 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.062 DEBUG:boto:Method: GET 2022-09-03T05:33:51.062 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.063 DEBUG:boto:Data: 2022-09-03T05:33:51.063 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.064 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.065 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.065 DEBUG:boto:Params: {} 2022-09-03T05:33:51.066 DEBUG:boto:Token: None 2022-09-03T05:33:51.066 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.067 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+d8FKzgtSy0LsjrEJjdfqkO6xWw= 2022-09-03T05:33:51.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+d8FKzgtSy0LsjrEJjdfqkO6xWw=', 'Content-Length': '0'} 2022-09-03T05:33:51.070 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000001df683904e9559a7-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.070 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:51.071 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:51.072 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.072 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.073 DEBUG:boto:Method: HEAD 2022-09-03T05:33:51.073 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.074 DEBUG:boto:Data: 2022-09-03T05:33:51.074 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.075 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.075 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.076 DEBUG:boto:Params: {} 2022-09-03T05:33:51.076 DEBUG:boto:Token: None 2022-09-03T05:33:51.077 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.078 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AIzhodRkBPRFyHlTsw/eZpeKLaY= 2022-09-03T05:33:51.078 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AIzhodRkBPRFyHlTsw/eZpeKLaY=', 'Content-Length': '0'} 2022-09-03T05:33:51.080 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000b62684dd5d22888e-006312e73f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.081 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:51.082 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:51.082 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.083 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.083 DEBUG:boto:Method: HEAD 2022-09-03T05:33:51.084 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.084 DEBUG:boto:Data: 2022-09-03T05:33:51.085 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.085 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.086 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.086 DEBUG:boto:Params: {} 2022-09-03T05:33:51.087 DEBUG:boto:Token: None 2022-09-03T05:33:51.088 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.088 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AIzhodRkBPRFyHlTsw/eZpeKLaY= 2022-09-03T05:33:51.089 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AIzhodRkBPRFyHlTsw/eZpeKLaY=', 'Content-Length': '0'} 2022-09-03T05:33:51.090 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000ae7eb56dc77a7bf9-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.091 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:33:51.092 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:51.092 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:51.093 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.093 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.094 DEBUG:boto:Method: GET 2022-09-03T05:33:51.095 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.095 DEBUG:boto:Data: 2022-09-03T05:33:51.096 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.096 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.097 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.097 DEBUG:boto:Params: {} 2022-09-03T05:33:51.098 DEBUG:boto:Token: None 2022-09-03T05:33:51.098 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.099 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+d8FKzgtSy0LsjrEJjdfqkO6xWw= 2022-09-03T05:33:51.100 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+d8FKzgtSy0LsjrEJjdfqkO6xWw=', 'Content-Length': '0'} 2022-09-03T05:33:51.102 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000004e7192dbcafc5ea7-006312e73f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.103 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:51.103 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:51.104 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.104 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.105 DEBUG:boto:Method: GET 2022-09-03T05:33:51.105 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.106 DEBUG:boto:Data: 2022-09-03T05:33:51.107 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.107 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.108 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.108 DEBUG:boto:Params: {} 2022-09-03T05:33:51.109 DEBUG:boto:Token: None 2022-09-03T05:33:51.109 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.110 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+d8FKzgtSy0LsjrEJjdfqkO6xWw= 2022-09-03T05:33:51.111 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+d8FKzgtSy0LsjrEJjdfqkO6xWw=', 'Content-Length': '0'} 2022-09-03T05:33:51.113 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000866b73f8d76a04c0-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.114 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:51.114 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:51.115 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:51.115 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:51.116 DEBUG:boto:Method: GET 2022-09-03T05:33:51.116 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:33:51.117 DEBUG:boto:Data: 2022-09-03T05:33:51.117 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.118 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.118 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.119 DEBUG:boto:Params: {} 2022-09-03T05:33:51.119 DEBUG:boto:Token: None 2022-09-03T05:33:51.120 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/ 2022-09-03T05:33:51.121 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k= 2022-09-03T05:33:51.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k=', 'Content-Length': '0'} 2022-09-03T05:33:51.124 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000652a6d41f42fd6a-006312e73f-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.174 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:51.176 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:51.177 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:51.178 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:51.179 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:51.180 DEBUG:boto:Method: GET 2022-09-03T05:33:51.180 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:33:51.181 DEBUG:boto:Data: 2022-09-03T05:33:51.182 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.183 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.184 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.185 DEBUG:boto:Params: {} 2022-09-03T05:33:51.185 DEBUG:boto:Token: None 2022-09-03T05:33:51.186 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/ 2022-09-03T05:33:51.187 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k= 2022-09-03T05:33:51.187 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k=', 'Content-Length': '0'} 2022-09-03T05:33:51.190 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007e6c4bc6f6168671-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.190 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:51.191 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:51.192 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:51.192 DEBUG:boto:Method: HEAD 2022-09-03T05:33:51.193 DEBUG:boto:Path: /dsgvcy-16/dummy 2022-09-03T05:33:51.193 DEBUG:boto:Data: 2022-09-03T05:33:51.194 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.194 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.195 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.195 DEBUG:boto:Params: {} 2022-09-03T05:33:51.196 DEBUG:boto:Token: None 2022-09-03T05:33:51.196 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/dummy 2022-09-03T05:33:51.197 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OFX2dXTYl8DO8+WHKzVPUMjNBJQ= 2022-09-03T05:33:51.197 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OFX2dXTYl8DO8+WHKzVPUMjNBJQ=', 'Content-Length': '0'} 2022-09-03T05:33:51.200 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f27a393922a08054-006312e73f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.201 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:51.201 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:51.202 DEBUG:boto:Method: HEAD 2022-09-03T05:33:51.202 DEBUG:boto:Path: /dsgvcy-16/dummy 2022-09-03T05:33:51.203 DEBUG:boto:Data: 2022-09-03T05:33:51.203 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.204 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.204 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.205 DEBUG:boto:Params: {} 2022-09-03T05:33:51.205 DEBUG:boto:Token: None 2022-09-03T05:33:51.206 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/dummy 2022-09-03T05:33:51.206 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OFX2dXTYl8DO8+WHKzVPUMjNBJQ= 2022-09-03T05:33:51.207 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OFX2dXTYl8DO8+WHKzVPUMjNBJQ=', 'Content-Length': '0'} 2022-09-03T05:33:51.209 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000002c4590f5654a382c-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.210 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:33:51.211 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:51.211 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:51.212 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.212 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.213 DEBUG:boto:Method: GET 2022-09-03T05:33:51.213 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.214 DEBUG:boto:Data: 2022-09-03T05:33:51.214 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.215 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.215 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.216 DEBUG:boto:Params: {} 2022-09-03T05:33:51.216 DEBUG:boto:Token: None 2022-09-03T05:33:51.217 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.218 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+d8FKzgtSy0LsjrEJjdfqkO6xWw= 2022-09-03T05:33:51.218 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+d8FKzgtSy0LsjrEJjdfqkO6xWw=', 'Content-Length': '0'} 2022-09-03T05:33:51.220 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000eafb2a2a7331b2f3-006312e73f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.221 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:51.222 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:51.222 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.223 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.223 DEBUG:boto:Method: GET 2022-09-03T05:33:51.224 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.225 DEBUG:boto:Data: 2022-09-03T05:33:51.225 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.226 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.226 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.227 DEBUG:boto:Params: {} 2022-09-03T05:33:51.227 DEBUG:boto:Token: None 2022-09-03T05:33:51.228 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:51.229 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+d8FKzgtSy0LsjrEJjdfqkO6xWw= 2022-09-03T05:33:51.229 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+d8FKzgtSy0LsjrEJjdfqkO6xWw=', 'Content-Length': '0'} 2022-09-03T05:33:51.231 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000094e37fe674385064-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.232 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:51.233 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:51.233 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:51.234 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:51.234 DEBUG:boto:Method: GET 2022-09-03T05:33:51.235 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:33:51.235 DEBUG:boto:Data: 2022-09-03T05:33:51.236 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.236 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.237 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.237 DEBUG:boto:Params: {} 2022-09-03T05:33:51.238 DEBUG:boto:Token: None 2022-09-03T05:33:51.239 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/ 2022-09-03T05:33:51.239 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k= 2022-09-03T05:33:51.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k=', 'Content-Length': '0'} 2022-09-03T05:33:51.243 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a441b5099aa4e6eb-006312e73f-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.243 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:51.244 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:51.245 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:51.245 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:51.246 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:51.247 DEBUG:boto:Method: GET 2022-09-03T05:33:51.247 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:33:51.248 DEBUG:boto:Data: 2022-09-03T05:33:51.248 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.249 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.249 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.250 DEBUG:boto:Params: {} 2022-09-03T05:33:51.251 DEBUG:boto:Token: None 2022-09-03T05:33:51.251 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/ 2022-09-03T05:33:51.252 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k= 2022-09-03T05:33:51.253 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k=', 'Content-Length': '0'} 2022-09-03T05:33:51.255 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000748b9ca5a185b820-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.256 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:51.257 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:33:51.257 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.258 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.258 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.267 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.267 DEBUG:boto:Method: GET 2022-09-03T05:33:51.268 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.268 DEBUG:boto:Data: 2022-09-03T05:33:51.269 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.269 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.270 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.270 DEBUG:boto:Params: {} 2022-09-03T05:33:51.271 DEBUG:boto:Token: None 2022-09-03T05:33:51.272 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.272 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iLeVqo8Ia+5bk06XWk21VKeDgjE= 2022-09-03T05:33:51.273 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iLeVqo8Ia+5bk06XWk21VKeDgjE=', 'Content-Length': '0'} 2022-09-03T05:33:51.275 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000534ff9ab81e24a20-006312e73f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.276 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.277 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.277 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.278 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.278 DEBUG:boto:Method: GET 2022-09-03T05:33:51.279 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.279 DEBUG:boto:Data: 2022-09-03T05:33:51.280 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.280 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.281 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.281 DEBUG:boto:Params: {} 2022-09-03T05:33:51.282 DEBUG:boto:Token: None 2022-09-03T05:33:51.283 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.283 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iLeVqo8Ia+5bk06XWk21VKeDgjE= 2022-09-03T05:33:51.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iLeVqo8Ia+5bk06XWk21VKeDgjE=', 'Content-Length': '0'} 2022-09-03T05:33:51.286 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009f940ff864fb368f-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.287 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.287 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.288 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.288 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.289 DEBUG:boto:Method: HEAD 2022-09-03T05:33:51.289 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.290 DEBUG:boto:Data: 2022-09-03T05:33:51.290 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.291 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.291 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.292 DEBUG:boto:Params: {} 2022-09-03T05:33:51.293 DEBUG:boto:Token: None 2022-09-03T05:33:51.293 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.294 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fh4B4ew+4WJp3hAeg21emDU0QRA= 2022-09-03T05:33:51.294 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fh4B4ew+4WJp3hAeg21emDU0QRA=', 'Content-Length': '0'} 2022-09-03T05:33:51.296 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009abb9341b695c156-006312e73f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.297 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.298 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.298 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.299 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.299 DEBUG:boto:Method: HEAD 2022-09-03T05:33:51.300 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.300 DEBUG:boto:Data: 2022-09-03T05:33:51.301 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.301 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.302 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.302 DEBUG:boto:Params: {} 2022-09-03T05:33:51.303 DEBUG:boto:Token: None 2022-09-03T05:33:51.303 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.304 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fh4B4ew+4WJp3hAeg21emDU0QRA= 2022-09-03T05:33:51.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fh4B4ew+4WJp3hAeg21emDU0QRA=', 'Content-Length': '0'} 2022-09-03T05:33:51.306 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b50e2d87176804ed-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.307 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:33:51.308 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.308 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.309 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.309 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.310 DEBUG:boto:Method: GET 2022-09-03T05:33:51.310 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.311 DEBUG:boto:Data: 2022-09-03T05:33:51.311 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.312 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.312 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.313 DEBUG:boto:Params: {} 2022-09-03T05:33:51.313 DEBUG:boto:Token: None 2022-09-03T05:33:51.314 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.315 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iLeVqo8Ia+5bk06XWk21VKeDgjE= 2022-09-03T05:33:51.315 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iLeVqo8Ia+5bk06XWk21VKeDgjE=', 'Content-Length': '0'} 2022-09-03T05:33:51.317 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ea7446fec5f483b0-006312e73f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.362 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.363 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.363 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.364 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.365 DEBUG:boto:Method: GET 2022-09-03T05:33:51.365 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.366 DEBUG:boto:Data: 2022-09-03T05:33:51.367 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.367 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.368 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.368 DEBUG:boto:Params: {} 2022-09-03T05:33:51.369 DEBUG:boto:Token: None 2022-09-03T05:33:51.370 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.371 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iLeVqo8Ia+5bk06XWk21VKeDgjE= 2022-09-03T05:33:51.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iLeVqo8Ia+5bk06XWk21VKeDgjE=', 'Content-Length': '0'} 2022-09-03T05:33:51.375 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009c8ab63f3b5d98f8-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.376 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:51.377 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:51.377 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:51.378 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:51.378 DEBUG:boto:Method: GET 2022-09-03T05:33:51.379 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:51.379 DEBUG:boto:Data: 2022-09-03T05:33:51.380 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.381 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.381 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.382 DEBUG:boto:Params: {} 2022-09-03T05:33:51.382 DEBUG:boto:Token: None 2022-09-03T05:33:51.383 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/ 2022-09-03T05:33:51.383 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k= 2022-09-03T05:33:51.385 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k=', 'Content-Length': '0'} 2022-09-03T05:33:51.387 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004ac451391250f8c3-006312e73f-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.388 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:51.389 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:51.390 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:51.390 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:51.391 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:51.391 DEBUG:boto:Method: GET 2022-09-03T05:33:51.392 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:51.392 DEBUG:boto:Data: 2022-09-03T05:33:51.393 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.394 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.394 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.395 DEBUG:boto:Params: {} 2022-09-03T05:33:51.395 DEBUG:boto:Token: None 2022-09-03T05:33:51.396 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/ 2022-09-03T05:33:51.397 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k= 2022-09-03T05:33:51.397 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k=', 'Content-Length': '0'} 2022-09-03T05:33:51.400 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a41de76d3e146209-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.401 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:51.401 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.402 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.402 DEBUG:boto:Method: HEAD 2022-09-03T05:33:51.403 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.403 DEBUG:boto:Data: 2022-09-03T05:33:51.404 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.404 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.405 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.406 DEBUG:boto:Params: {} 2022-09-03T05:33:51.406 DEBUG:boto:Token: None 2022-09-03T05:33:51.407 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.407 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RGzmph0XUz5tt3E6TtDaRt4akhM= 2022-09-03T05:33:51.408 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RGzmph0XUz5tt3E6TtDaRt4akhM=', 'Content-Length': '0'} 2022-09-03T05:33:51.410 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b9ae27094a53f0e5-006312e73f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.411 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.412 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.412 DEBUG:boto:Method: HEAD 2022-09-03T05:33:51.413 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.413 DEBUG:boto:Data: 2022-09-03T05:33:51.414 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.414 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.415 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.415 DEBUG:boto:Params: {} 2022-09-03T05:33:51.416 DEBUG:boto:Token: None 2022-09-03T05:33:51.417 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.417 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RGzmph0XUz5tt3E6TtDaRt4akhM= 2022-09-03T05:33:51.418 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RGzmph0XUz5tt3E6TtDaRt4akhM=', 'Content-Length': '0'} 2022-09-03T05:33:51.420 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000da793fc9b2b43efa-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.421 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:33:51.421 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.422 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.422 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.423 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.424 DEBUG:boto:Method: GET 2022-09-03T05:33:51.424 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.425 DEBUG:boto:Data: 2022-09-03T05:33:51.425 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.426 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.426 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.427 DEBUG:boto:Params: {} 2022-09-03T05:33:51.427 DEBUG:boto:Token: None 2022-09-03T05:33:51.428 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.429 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iLeVqo8Ia+5bk06XWk21VKeDgjE= 2022-09-03T05:33:51.429 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iLeVqo8Ia+5bk06XWk21VKeDgjE=', 'Content-Length': '0'} 2022-09-03T05:33:51.431 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f51514c921beaaa0-006312e73f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.476 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.477 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:51.477 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.478 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.478 DEBUG:boto:Method: GET 2022-09-03T05:33:51.479 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.479 DEBUG:boto:Data: 2022-09-03T05:33:51.480 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.480 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.481 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.481 DEBUG:boto:Params: {} 2022-09-03T05:33:51.482 DEBUG:boto:Token: None 2022-09-03T05:33:51.483 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:51.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iLeVqo8Ia+5bk06XWk21VKeDgjE= 2022-09-03T05:33:51.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iLeVqo8Ia+5bk06XWk21VKeDgjE=', 'Content-Length': '0'} 2022-09-03T05:33:51.486 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006debac505f4eb9c3-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.487 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:51.487 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:51.488 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:51.489 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:51.489 DEBUG:boto:Method: GET 2022-09-03T05:33:51.490 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:51.490 DEBUG:boto:Data: 2022-09-03T05:33:51.491 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.491 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.492 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.492 DEBUG:boto:Params: {} 2022-09-03T05:33:51.493 DEBUG:boto:Token: None 2022-09-03T05:33:51.494 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/ 2022-09-03T05:33:51.494 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k= 2022-09-03T05:33:51.495 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k=', 'Content-Length': '0'} 2022-09-03T05:33:51.498 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dabb7dcfab973efa-006312e73f-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.499 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:51.499 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:51.500 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:51.501 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:51.501 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:51.504 DEBUG:boto:Method: GET 2022-09-03T05:33:51.505 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:51.505 DEBUG:boto:Data: 2022-09-03T05:33:51.506 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.506 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.507 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.507 DEBUG:boto:Params: {} 2022-09-03T05:33:51.508 DEBUG:boto:Token: None 2022-09-03T05:33:51.508 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/ 2022-09-03T05:33:51.509 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k= 2022-09-03T05:33:51.510 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k=', 'Content-Length': '0'} 2022-09-03T05:33:51.513 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000467d2c02edb590c4-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.513 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:51.514 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:33:51.515 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.516 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.516 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.517 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.517 DEBUG:boto:Method: GET 2022-09-03T05:33:51.518 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.518 DEBUG:boto:Data: 2022-09-03T05:33:51.519 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.519 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.520 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.520 DEBUG:boto:Params: {} 2022-09-03T05:33:51.521 DEBUG:boto:Token: None 2022-09-03T05:33:51.522 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.523 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:toCqHMi7dqMMc4wkXRwHi+qG8c4= 2022-09-03T05:33:51.523 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:toCqHMi7dqMMc4wkXRwHi+qG8c4=', 'Content-Length': '0'} 2022-09-03T05:33:51.530 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002afd9563b0bfc8a7-006312e73f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.531 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.532 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.532 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.533 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.533 DEBUG:boto:Method: GET 2022-09-03T05:33:51.534 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.535 DEBUG:boto:Data: 2022-09-03T05:33:51.535 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.536 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.536 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.537 DEBUG:boto:Params: {} 2022-09-03T05:33:51.537 DEBUG:boto:Token: None 2022-09-03T05:33:51.538 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.538 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:toCqHMi7dqMMc4wkXRwHi+qG8c4= 2022-09-03T05:33:51.539 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:toCqHMi7dqMMc4wkXRwHi+qG8c4=', 'Content-Length': '0'} 2022-09-03T05:33:51.541 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004d3ea0d3e0684f1c-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.542 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.543 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.543 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.543 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.544 DEBUG:boto:Method: HEAD 2022-09-03T05:33:51.544 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.545 DEBUG:boto:Data: 2022-09-03T05:33:51.546 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.546 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.547 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.547 DEBUG:boto:Params: {} 2022-09-03T05:33:51.548 DEBUG:boto:Token: None 2022-09-03T05:33:51.548 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.549 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vfj6WebtmjvIXU9UGC3f0zzej60= 2022-09-03T05:33:51.549 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vfj6WebtmjvIXU9UGC3f0zzej60=', 'Content-Length': '0'} 2022-09-03T05:33:51.551 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000014b89bfe7530a9f8-006312e73f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.552 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.553 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.553 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.554 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.555 DEBUG:boto:Method: HEAD 2022-09-03T05:33:51.556 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.556 DEBUG:boto:Data: 2022-09-03T05:33:51.557 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.558 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.559 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.560 DEBUG:boto:Params: {} 2022-09-03T05:33:51.561 DEBUG:boto:Token: None 2022-09-03T05:33:51.562 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.563 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vfj6WebtmjvIXU9UGC3f0zzej60= 2022-09-03T05:33:51.564 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vfj6WebtmjvIXU9UGC3f0zzej60=', 'Content-Length': '0'} 2022-09-03T05:33:51.567 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003157452c6ba07a45-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.568 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:33:51.569 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.570 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.571 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.572 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.573 DEBUG:boto:Method: GET 2022-09-03T05:33:51.574 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.575 DEBUG:boto:Data: 2022-09-03T05:33:51.576 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.577 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.577 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.578 DEBUG:boto:Params: {} 2022-09-03T05:33:51.579 DEBUG:boto:Token: None 2022-09-03T05:33:51.580 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.581 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:toCqHMi7dqMMc4wkXRwHi+qG8c4= 2022-09-03T05:33:51.581 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:toCqHMi7dqMMc4wkXRwHi+qG8c4=', 'Content-Length': '0'} 2022-09-03T05:33:51.583 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000037b3f2e72f5169da-006312e73f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.631 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.632 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.633 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.633 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.634 DEBUG:boto:Method: GET 2022-09-03T05:33:51.635 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.635 DEBUG:boto:Data: 2022-09-03T05:33:51.636 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.636 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.637 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.638 DEBUG:boto:Params: {} 2022-09-03T05:33:51.639 DEBUG:boto:Token: None 2022-09-03T05:33:51.639 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.640 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:toCqHMi7dqMMc4wkXRwHi+qG8c4= 2022-09-03T05:33:51.641 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:toCqHMi7dqMMc4wkXRwHi+qG8c4=', 'Content-Length': '0'} 2022-09-03T05:33:51.644 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bea09cf84ac35a0e-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.646 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:51.647 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:51.648 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:51.649 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:51.651 DEBUG:boto:Method: GET 2022-09-03T05:33:51.652 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:51.653 DEBUG:boto:Data: 2022-09-03T05:33:51.654 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.654 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.655 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.655 DEBUG:boto:Params: {} 2022-09-03T05:33:51.656 DEBUG:boto:Token: None 2022-09-03T05:33:51.657 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/ 2022-09-03T05:33:51.658 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k= 2022-09-03T05:33:51.659 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k=', 'Content-Length': '0'} 2022-09-03T05:33:51.662 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000058bef5405eb447c5-006312e73f-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.664 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:51.666 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:51.667 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:51.668 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:51.669 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:51.670 DEBUG:boto:Method: GET 2022-09-03T05:33:51.671 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:51.671 DEBUG:boto:Data: 2022-09-03T05:33:51.672 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.673 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.673 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.674 DEBUG:boto:Params: {} 2022-09-03T05:33:51.674 DEBUG:boto:Token: None 2022-09-03T05:33:51.675 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/ 2022-09-03T05:33:51.676 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k= 2022-09-03T05:33:51.676 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k=', 'Content-Length': '0'} 2022-09-03T05:33:51.680 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b464f368cdbfdd94-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.680 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:51.681 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.682 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.683 DEBUG:boto:Method: HEAD 2022-09-03T05:33:51.683 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.684 DEBUG:boto:Data: 2022-09-03T05:33:51.685 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.686 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.686 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.687 DEBUG:boto:Params: {} 2022-09-03T05:33:51.688 DEBUG:boto:Token: None 2022-09-03T05:33:51.688 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.689 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NgAIMam95s1loelb1Zk6YHzYmW4= 2022-09-03T05:33:51.690 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NgAIMam95s1loelb1Zk6YHzYmW4=', 'Content-Length': '0'} 2022-09-03T05:33:51.693 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fac1cbb227450c72-006312e73f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.693 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.694 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.695 DEBUG:boto:Method: HEAD 2022-09-03T05:33:51.695 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.696 DEBUG:boto:Data: 2022-09-03T05:33:51.697 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.697 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.698 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.698 DEBUG:boto:Params: {} 2022-09-03T05:33:51.699 DEBUG:boto:Token: None 2022-09-03T05:33:51.700 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.700 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NgAIMam95s1loelb1Zk6YHzYmW4= 2022-09-03T05:33:51.701 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NgAIMam95s1loelb1Zk6YHzYmW4=', 'Content-Length': '0'} 2022-09-03T05:33:51.703 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b1e8da3c14fe8d05-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.704 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:33:51.705 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.705 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.706 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.707 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.708 DEBUG:boto:Method: GET 2022-09-03T05:33:51.708 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.709 DEBUG:boto:Data: 2022-09-03T05:33:51.709 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.710 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.710 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.711 DEBUG:boto:Params: {} 2022-09-03T05:33:51.711 DEBUG:boto:Token: None 2022-09-03T05:33:51.712 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.713 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:toCqHMi7dqMMc4wkXRwHi+qG8c4= 2022-09-03T05:33:51.713 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:toCqHMi7dqMMc4wkXRwHi+qG8c4=', 'Content-Length': '0'} 2022-09-03T05:33:51.716 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003c0ef824854f28f3-006312e73f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.716 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.718 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:51.718 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.719 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.719 DEBUG:boto:Method: GET 2022-09-03T05:33:51.720 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.720 DEBUG:boto:Data: 2022-09-03T05:33:51.721 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.721 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.722 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.723 DEBUG:boto:Params: {} 2022-09-03T05:33:51.723 DEBUG:boto:Token: None 2022-09-03T05:33:51.724 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:51.725 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:toCqHMi7dqMMc4wkXRwHi+qG8c4= 2022-09-03T05:33:51.725 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:toCqHMi7dqMMc4wkXRwHi+qG8c4=', 'Content-Length': '0'} 2022-09-03T05:33:51.728 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cdf34f2309d72cdb-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.770 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:51.771 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:51.772 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:51.772 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:51.773 DEBUG:boto:Method: GET 2022-09-03T05:33:51.773 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:51.774 DEBUG:boto:Data: 2022-09-03T05:33:51.774 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.775 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.775 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.776 DEBUG:boto:Params: {} 2022-09-03T05:33:51.777 DEBUG:boto:Token: None 2022-09-03T05:33:51.777 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/ 2022-09-03T05:33:51.778 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k= 2022-09-03T05:33:51.778 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k=', 'Content-Length': '0'} 2022-09-03T05:33:51.781 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004abc3e2d6e3c94e8-006312e73f-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.782 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:51.783 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:51.783 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:51.784 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:51.785 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:51.785 DEBUG:boto:Method: GET 2022-09-03T05:33:51.786 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:51.786 DEBUG:boto:Data: 2022-09-03T05:33:51.787 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.787 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.788 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.788 DEBUG:boto:Params: {} 2022-09-03T05:33:51.789 DEBUG:boto:Token: None 2022-09-03T05:33:51.790 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/ 2022-09-03T05:33:51.791 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k= 2022-09-03T05:33:51.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k=', 'Content-Length': '0'} 2022-09-03T05:33:51.795 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003650c80b09b82f30-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.795 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:51.796 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:33:51.797 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:51.797 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:51.798 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.799 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.799 DEBUG:boto:Method: GET 2022-09-03T05:33:51.800 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.800 DEBUG:boto:Data: 2022-09-03T05:33:51.801 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.802 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.802 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.803 DEBUG:boto:Params: {} 2022-09-03T05:33:51.804 DEBUG:boto:Token: None 2022-09-03T05:33:51.805 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.805 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FoqJC/sFND6git9NX6KgKi6rDDw= 2022-09-03T05:33:51.806 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FoqJC/sFND6git9NX6KgKi6rDDw=', 'Content-Length': '0'} 2022-09-03T05:33:51.809 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001705c6f09c78ec48-006312e73f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.809 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:51.810 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:51.810 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.811 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.812 DEBUG:boto:Method: GET 2022-09-03T05:33:51.812 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.813 DEBUG:boto:Data: 2022-09-03T05:33:51.813 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.814 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.814 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.815 DEBUG:boto:Params: {} 2022-09-03T05:33:51.815 DEBUG:boto:Token: None 2022-09-03T05:33:51.816 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.817 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FoqJC/sFND6git9NX6KgKi6rDDw= 2022-09-03T05:33:51.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FoqJC/sFND6git9NX6KgKi6rDDw=', 'Content-Length': '0'} 2022-09-03T05:33:51.819 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000073bbb9820432e488-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.820 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:51.821 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:51.821 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.822 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.822 DEBUG:boto:Method: HEAD 2022-09-03T05:33:51.823 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.823 DEBUG:boto:Data: 2022-09-03T05:33:51.824 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.824 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.825 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.825 DEBUG:boto:Params: {} 2022-09-03T05:33:51.826 DEBUG:boto:Token: None 2022-09-03T05:33:51.826 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.827 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e1lYLvwmxH0lagtabn6sZ3fNGPo= 2022-09-03T05:33:51.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e1lYLvwmxH0lagtabn6sZ3fNGPo=', 'Content-Length': '0'} 2022-09-03T05:33:51.830 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c39cd1684a801932-006312e73f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.830 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:51.831 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:51.832 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.832 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.834 DEBUG:boto:Method: HEAD 2022-09-03T05:33:51.835 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.835 DEBUG:boto:Data: 2022-09-03T05:33:51.836 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.837 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.837 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.838 DEBUG:boto:Params: {} 2022-09-03T05:33:51.838 DEBUG:boto:Token: None 2022-09-03T05:33:51.839 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.840 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e1lYLvwmxH0lagtabn6sZ3fNGPo= 2022-09-03T05:33:51.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e1lYLvwmxH0lagtabn6sZ3fNGPo=', 'Content-Length': '0'} 2022-09-03T05:33:51.842 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000eb13570859244538-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.843 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:33:51.843 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:51.844 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:51.845 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.845 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.846 DEBUG:boto:Method: GET 2022-09-03T05:33:51.846 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.847 DEBUG:boto:Data: 2022-09-03T05:33:51.847 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.848 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.848 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.849 DEBUG:boto:Params: {} 2022-09-03T05:33:51.850 DEBUG:boto:Token: None 2022-09-03T05:33:51.850 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.851 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FoqJC/sFND6git9NX6KgKi6rDDw= 2022-09-03T05:33:51.851 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FoqJC/sFND6git9NX6KgKi6rDDw=', 'Content-Length': '0'} 2022-09-03T05:33:51.854 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007f5636cdecf90c67-006312e73f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.855 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:51.855 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:51.856 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.856 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.857 DEBUG:boto:Method: GET 2022-09-03T05:33:51.857 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.858 DEBUG:boto:Data: 2022-09-03T05:33:51.859 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.859 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.860 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.860 DEBUG:boto:Params: {} 2022-09-03T05:33:51.861 DEBUG:boto:Token: None 2022-09-03T05:33:51.862 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.862 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FoqJC/sFND6git9NX6KgKi6rDDw= 2022-09-03T05:33:51.863 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FoqJC/sFND6git9NX6KgKi6rDDw=', 'Content-Length': '0'} 2022-09-03T05:33:51.865 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d0ee06e232570a1f-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.910 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:51.911 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:51.911 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:51.912 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:51.912 DEBUG:boto:Method: GET 2022-09-03T05:33:51.913 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:51.913 DEBUG:boto:Data: 2022-09-03T05:33:51.914 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.914 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.915 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.916 DEBUG:boto:Params: {} 2022-09-03T05:33:51.916 DEBUG:boto:Token: None 2022-09-03T05:33:51.917 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/ 2022-09-03T05:33:51.918 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k= 2022-09-03T05:33:51.918 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k=', 'Content-Length': '0'} 2022-09-03T05:33:51.921 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f0385a8746a14a35-006312e73f-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.922 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:51.923 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:51.924 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:51.924 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:51.925 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:51.926 DEBUG:boto:Method: GET 2022-09-03T05:33:51.926 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:51.927 DEBUG:boto:Data: 2022-09-03T05:33:51.927 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.928 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.928 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.929 DEBUG:boto:Params: {} 2022-09-03T05:33:51.930 DEBUG:boto:Token: None 2022-09-03T05:33:51.930 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/ 2022-09-03T05:33:51.931 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k= 2022-09-03T05:33:51.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CTlV4F3f4xjyP+5YsW2/ewKSp3k=', 'Content-Length': '0'} 2022-09-03T05:33:51.935 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009a30b7f6003a78cf-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.936 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:51.937 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:51.938 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:51.938 DEBUG:boto:Method: HEAD 2022-09-03T05:33:51.939 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3 2022-09-03T05:33:51.940 DEBUG:boto:Data: 2022-09-03T05:33:51.940 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.941 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.941 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.942 DEBUG:boto:Params: {} 2022-09-03T05:33:51.943 DEBUG:boto:Token: None 2022-09-03T05:33:51.943 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone3 2022-09-03T05:33:51.944 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0Jrukw5icAy8gkQofb++XL++bgA= 2022-09-03T05:33:51.945 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0Jrukw5icAy8gkQofb++XL++bgA=', 'Content-Length': '0'} 2022-09-03T05:33:51.947 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e6c6b54e643b1165-006312e73f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.948 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:51.949 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:51.949 DEBUG:boto:Method: HEAD 2022-09-03T05:33:51.950 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3 2022-09-03T05:33:51.951 DEBUG:boto:Data: 2022-09-03T05:33:51.951 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.952 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:51.952 DEBUG:boto:Port: 8000 2022-09-03T05:33:51.953 DEBUG:boto:Params: {} 2022-09-03T05:33:51.953 DEBUG:boto:Token: None 2022-09-03T05:33:51.954 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone3 2022-09-03T05:33:51.955 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0Jrukw5icAy8gkQofb++XL++bgA= 2022-09-03T05:33:51.955 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0Jrukw5icAy8gkQofb++XL++bgA=', 'Content-Length': '0'} 2022-09-03T05:33:51.958 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a20a7a9833c5eb56-006312e73f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:51.959 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:33:51.959 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:51.960 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:51.960 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.961 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.961 DEBUG:boto:Method: GET 2022-09-03T05:33:51.962 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.963 DEBUG:boto:Data: 2022-09-03T05:33:51.963 DEBUG:boto:Headers: {} 2022-09-03T05:33:51.964 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:51.964 DEBUG:boto:Port: 8001 2022-09-03T05:33:51.965 DEBUG:boto:Params: {} 2022-09-03T05:33:51.965 DEBUG:boto:Token: None 2022-09-03T05:33:51.966 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:51 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:51.967 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FoqJC/sFND6git9NX6KgKi6rDDw= 2022-09-03T05:33:51.967 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FoqJC/sFND6git9NX6KgKi6rDDw=', 'Content-Length': '0'} 2022-09-03T05:33:51.970 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ecf5699f1400aa27-006312e73f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.010 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:52.011 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:52.011 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:52.012 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:52.013 DEBUG:boto:Method: GET 2022-09-03T05:33:52.013 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:52.014 DEBUG:boto:Data: 2022-09-03T05:33:52.014 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.015 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:52.016 DEBUG:boto:Port: 8000 2022-09-03T05:33:52.016 DEBUG:boto:Params: {} 2022-09-03T05:33:52.017 DEBUG:boto:Token: None 2022-09-03T05:33:52.017 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:52.018 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n8E8eX79CXnnYOtfYOmjwF4qK18= 2022-09-03T05:33:52.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n8E8eX79CXnnYOtfYOmjwF4qK18=', 'Content-Length': '0'} 2022-09-03T05:33:52.022 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000079ad8cdd7b41d594-006312e740-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.023 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:52.025 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:52.026 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:52.027 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:52.028 DEBUG:boto:Method: GET 2022-09-03T05:33:52.029 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:52.030 DEBUG:boto:Data: 2022-09-03T05:33:52.031 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.032 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:52.032 DEBUG:boto:Port: 8001 2022-09-03T05:33:52.033 DEBUG:boto:Params: {} 2022-09-03T05:33:52.034 DEBUG:boto:Token: None 2022-09-03T05:33:52.035 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/ 2022-09-03T05:33:52.036 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s= 2022-09-03T05:33:52.037 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s=', 'Content-Length': '0'} 2022-09-03T05:33:52.040 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000034c2aa44dd67405c-006312e740-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.041 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:52.042 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:52.043 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:52.043 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:52.044 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:52.045 DEBUG:boto:Method: GET 2022-09-03T05:33:52.045 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:52.046 DEBUG:boto:Data: 2022-09-03T05:33:52.047 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.047 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:52.048 DEBUG:boto:Port: 8000 2022-09-03T05:33:52.048 DEBUG:boto:Params: {} 2022-09-03T05:33:52.049 DEBUG:boto:Token: None 2022-09-03T05:33:52.051 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/ 2022-09-03T05:33:52.052 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s= 2022-09-03T05:33:52.053 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s=', 'Content-Length': '0'} 2022-09-03T05:33:52.058 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b924065973ac6d77-006312e740-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.106 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:52.107 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-16 zones={test-zone3, test-zone1} 2022-09-03T05:33:52.108 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-16 zones={test-zone3, test-zone2} 2022-09-03T05:33:52.108 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:52.109 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:52.109 DEBUG:boto:Method: HEAD 2022-09-03T05:33:52.110 DEBUG:boto:Path: /dsgvcy-16/ 2022-09-03T05:33:52.111 DEBUG:boto:Data: 2022-09-03T05:33:52.111 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.112 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:52.113 DEBUG:boto:Port: 8001 2022-09-03T05:33:52.114 DEBUG:boto:Params: {} 2022-09-03T05:33:52.115 DEBUG:boto:Token: None 2022-09-03T05:33:52.115 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/ 2022-09-03T05:33:52.116 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ia9Ir9PH5pvJfkNESCJY8i3/dFc= 2022-09-03T05:33:52.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ia9Ir9PH5pvJfkNESCJY8i3/dFc=', 'Content-Length': '0'} 2022-09-03T05:33:52.119 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004447f1c98d874d61-006312e740-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.120 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:52.120 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:52.121 DEBUG:boto:Method: HEAD 2022-09-03T05:33:52.121 DEBUG:boto:Path: /dsgvcy-16/ 2022-09-03T05:33:52.122 DEBUG:boto:Data: 2022-09-03T05:33:52.123 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.124 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:52.125 DEBUG:boto:Port: 8000 2022-09-03T05:33:52.125 DEBUG:boto:Params: {} 2022-09-03T05:33:52.126 DEBUG:boto:Token: None 2022-09-03T05:33:52.126 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/ 2022-09-03T05:33:52.127 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ia9Ir9PH5pvJfkNESCJY8i3/dFc= 2022-09-03T05:33:52.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ia9Ir9PH5pvJfkNESCJY8i3/dFc=', 'Content-Length': '0'} 2022-09-03T05:33:52.130 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fb28458bb53e923a-006312e740-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.131 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:33:52.131 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:52.132 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:52.132 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:52.133 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:52.135 DEBUG:boto:Method: GET 2022-09-03T05:33:52.135 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:52.136 DEBUG:boto:Data: 2022-09-03T05:33:52.137 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.138 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:52.138 DEBUG:boto:Port: 8001 2022-09-03T05:33:52.139 DEBUG:boto:Params: {} 2022-09-03T05:33:52.140 DEBUG:boto:Token: None 2022-09-03T05:33:52.140 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/?versions 2022-09-03T05:33:52.141 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bNoewJVRin1vEp0MGgZ7iSjshy0= 2022-09-03T05:33:52.141 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bNoewJVRin1vEp0MGgZ7iSjshy0=', 'Content-Length': '0'} 2022-09-03T05:33:52.144 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d46fcfb19f9d4833-006312e740-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.146 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:52.147 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:33:52.148 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:33:52.149 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:33:52.150 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:33:52.151 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:33:52.151 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:52.152 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:52.153 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:52.153 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:52.154 DEBUG:boto:Method: GET 2022-09-03T05:33:52.155 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:52.156 DEBUG:boto:Data: 2022-09-03T05:33:52.157 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.157 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:52.158 DEBUG:boto:Port: 8000 2022-09-03T05:33:52.158 DEBUG:boto:Params: {} 2022-09-03T05:33:52.159 DEBUG:boto:Token: None 2022-09-03T05:33:52.160 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/?versions 2022-09-03T05:33:52.161 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bNoewJVRin1vEp0MGgZ7iSjshy0= 2022-09-03T05:33:52.162 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bNoewJVRin1vEp0MGgZ7iSjshy0=', 'Content-Length': '0'} 2022-09-03T05:33:52.164 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006f16f467dbb59841-006312e740-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.165 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:52.167 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:33:52.168 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:33:52.169 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:33:52.170 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:33:52.171 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:52.171 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:52.172 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:52.173 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:52.174 DEBUG:boto:Method: GET 2022-09-03T05:33:52.175 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:52.176 DEBUG:boto:Data: 2022-09-03T05:33:52.177 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.178 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:52.179 DEBUG:boto:Port: 8001 2022-09-03T05:33:52.180 DEBUG:boto:Params: {} 2022-09-03T05:33:52.181 DEBUG:boto:Token: None 2022-09-03T05:33:52.182 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/?versions 2022-09-03T05:33:52.182 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bNoewJVRin1vEp0MGgZ7iSjshy0= 2022-09-03T05:33:52.183 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bNoewJVRin1vEp0MGgZ7iSjshy0=', 'Content-Length': '0'} 2022-09-03T05:33:52.185 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009559cdac2036dc74-006312e740-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.186 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:52.187 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:52.188 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:52.188 DEBUG:boto:path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:52.189 DEBUG:boto:auth_path=/dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:52.189 DEBUG:boto:Method: GET 2022-09-03T05:33:52.190 DEBUG:boto:Path: /dsgvcy-16/?versions&max-keys=999 2022-09-03T05:33:52.191 DEBUG:boto:Data: 2022-09-03T05:33:52.191 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.192 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:52.192 DEBUG:boto:Port: 8000 2022-09-03T05:33:52.193 DEBUG:boto:Params: {} 2022-09-03T05:33:52.193 DEBUG:boto:Token: None 2022-09-03T05:33:52.194 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/?versions 2022-09-03T05:33:52.195 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bNoewJVRin1vEp0MGgZ7iSjshy0= 2022-09-03T05:33:52.196 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bNoewJVRin1vEp0MGgZ7iSjshy0=', 'Content-Length': '0'} 2022-09-03T05:33:52.199 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000efa6ea9f3052f6ea-006312e740-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.199 DEBUG:boto:b'dsgvcy-16999falsedummyiJxQIs7jrKchIKwzf3kPx8w51PQyH2Mtrue2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1Ypw07nUES8aCnyjd6kp6dRzyDQhB0VHtrue2022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dowtrue2022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFWtrue2022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:52.201 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:33:52.201 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:52.203 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:52.203 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.204 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.205 DEBUG:boto:Method: GET 2022-09-03T05:33:52.206 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.207 DEBUG:boto:Data: 2022-09-03T05:33:52.207 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.208 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:52.209 DEBUG:boto:Port: 8001 2022-09-03T05:33:52.209 DEBUG:boto:Params: {} 2022-09-03T05:33:52.210 DEBUG:boto:Token: None 2022-09-03T05:33:52.211 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.211 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hubxnre7BSFevkqcEdYg16CJWV4= 2022-09-03T05:33:52.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hubxnre7BSFevkqcEdYg16CJWV4=', 'Content-Length': '0'} 2022-09-03T05:33:52.214 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000b404171da1058cdd-006312e740-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.215 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:52.215 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:52.216 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.217 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.218 DEBUG:boto:Method: GET 2022-09-03T05:33:52.219 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.220 DEBUG:boto:Data: 2022-09-03T05:33:52.221 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.222 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:52.223 DEBUG:boto:Port: 8000 2022-09-03T05:33:52.224 DEBUG:boto:Params: {} 2022-09-03T05:33:52.225 DEBUG:boto:Token: None 2022-09-03T05:33:52.226 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.227 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hubxnre7BSFevkqcEdYg16CJWV4= 2022-09-03T05:33:52.228 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hubxnre7BSFevkqcEdYg16CJWV4=', 'Content-Length': '0'} 2022-09-03T05:33:52.230 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000003cc60af268da033b-006312e740-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.231 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:52.231 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:52.232 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.232 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.233 DEBUG:boto:Method: HEAD 2022-09-03T05:33:52.233 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.234 DEBUG:boto:Data: 2022-09-03T05:33:52.234 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.235 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:52.235 DEBUG:boto:Port: 8001 2022-09-03T05:33:52.236 DEBUG:boto:Params: {} 2022-09-03T05:33:52.237 DEBUG:boto:Token: None 2022-09-03T05:33:52.237 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.238 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yWMq+EuXFzbdbmoVL6A9UK1aVSU= 2022-09-03T05:33:52.239 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yWMq+EuXFzbdbmoVL6A9UK1aVSU=', 'Content-Length': '0'} 2022-09-03T05:33:52.241 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000089a7506fe888e91d-006312e740-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.242 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:52.243 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:52.243 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.244 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.245 DEBUG:boto:Method: HEAD 2022-09-03T05:33:52.246 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.247 DEBUG:boto:Data: 2022-09-03T05:33:52.248 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.249 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:52.250 DEBUG:boto:Port: 8000 2022-09-03T05:33:52.251 DEBUG:boto:Params: {} 2022-09-03T05:33:52.252 DEBUG:boto:Token: None 2022-09-03T05:33:52.253 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.254 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yWMq+EuXFzbdbmoVL6A9UK1aVSU= 2022-09-03T05:33:52.255 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yWMq+EuXFzbdbmoVL6A9UK1aVSU=', 'Content-Length': '0'} 2022-09-03T05:33:52.257 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000223712401242ace4-006312e740-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.259 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:33:52.259 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:52.260 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:52.261 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.262 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.263 DEBUG:boto:Method: GET 2022-09-03T05:33:52.264 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.264 DEBUG:boto:Data: 2022-09-03T05:33:52.265 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.266 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:52.266 DEBUG:boto:Port: 8001 2022-09-03T05:33:52.267 DEBUG:boto:Params: {} 2022-09-03T05:33:52.268 DEBUG:boto:Token: None 2022-09-03T05:33:52.268 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.269 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hubxnre7BSFevkqcEdYg16CJWV4= 2022-09-03T05:33:52.270 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hubxnre7BSFevkqcEdYg16CJWV4=', 'Content-Length': '0'} 2022-09-03T05:33:52.272 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000fbed1afa275bd075-006312e740-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.273 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:52.274 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:52.275 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.276 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.277 DEBUG:boto:Method: GET 2022-09-03T05:33:52.278 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.278 DEBUG:boto:Data: 2022-09-03T05:33:52.279 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.279 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:52.280 DEBUG:boto:Port: 8000 2022-09-03T05:33:52.280 DEBUG:boto:Params: {} 2022-09-03T05:33:52.281 DEBUG:boto:Token: None 2022-09-03T05:33:52.282 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.282 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hubxnre7BSFevkqcEdYg16CJWV4= 2022-09-03T05:33:52.283 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hubxnre7BSFevkqcEdYg16CJWV4=', 'Content-Length': '0'} 2022-09-03T05:33:52.285 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000d34c9c35c81832d6-006312e740-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.286 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:52.287 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:52.288 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:52.289 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:52.289 DEBUG:boto:Method: GET 2022-09-03T05:33:52.290 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:33:52.291 DEBUG:boto:Data: 2022-09-03T05:33:52.291 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.292 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:52.292 DEBUG:boto:Port: 8001 2022-09-03T05:33:52.293 DEBUG:boto:Params: {} 2022-09-03T05:33:52.293 DEBUG:boto:Token: None 2022-09-03T05:33:52.294 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/ 2022-09-03T05:33:52.295 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s= 2022-09-03T05:33:52.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s=', 'Content-Length': '0'} 2022-09-03T05:33:52.298 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000035ad5e4a3b5e1b64-006312e740-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.346 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:52.347 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:52.348 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:52.348 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:52.349 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:52.349 DEBUG:boto:Method: GET 2022-09-03T05:33:52.350 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:33:52.351 DEBUG:boto:Data: 2022-09-03T05:33:52.352 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.353 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:52.353 DEBUG:boto:Port: 8000 2022-09-03T05:33:52.354 DEBUG:boto:Params: {} 2022-09-03T05:33:52.354 DEBUG:boto:Token: None 2022-09-03T05:33:52.355 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/ 2022-09-03T05:33:52.356 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s= 2022-09-03T05:33:52.357 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s=', 'Content-Length': '0'} 2022-09-03T05:33:52.359 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e06d641059d6d4f6-006312e740-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.360 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:52.361 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:52.361 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:52.362 DEBUG:boto:Method: HEAD 2022-09-03T05:33:52.362 DEBUG:boto:Path: /dsgvcy-16/dummy 2022-09-03T05:33:52.363 DEBUG:boto:Data: 2022-09-03T05:33:52.363 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.364 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:52.364 DEBUG:boto:Port: 8001 2022-09-03T05:33:52.365 DEBUG:boto:Params: {} 2022-09-03T05:33:52.365 DEBUG:boto:Token: None 2022-09-03T05:33:52.366 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/dummy 2022-09-03T05:33:52.367 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t2GiUuZDZlcFXgm8n7b3FyL28iQ= 2022-09-03T05:33:52.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t2GiUuZDZlcFXgm8n7b3FyL28iQ=', 'Content-Length': '0'} 2022-09-03T05:33:52.369 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000009dd65408ccf7530b-006312e740-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.370 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:52.371 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:52.371 DEBUG:boto:Method: HEAD 2022-09-03T05:33:52.372 DEBUG:boto:Path: /dsgvcy-16/dummy 2022-09-03T05:33:52.372 DEBUG:boto:Data: 2022-09-03T05:33:52.373 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.373 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:52.374 DEBUG:boto:Port: 8000 2022-09-03T05:33:52.374 DEBUG:boto:Params: {} 2022-09-03T05:33:52.375 DEBUG:boto:Token: None 2022-09-03T05:33:52.375 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/dummy 2022-09-03T05:33:52.376 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t2GiUuZDZlcFXgm8n7b3FyL28iQ= 2022-09-03T05:33:52.376 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t2GiUuZDZlcFXgm8n7b3FyL28iQ=', 'Content-Length': '0'} 2022-09-03T05:33:52.379 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000b24d0febf96f224c-006312e740-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.380 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:33:52.381 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:52.381 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:52.382 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.382 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.383 DEBUG:boto:Method: GET 2022-09-03T05:33:52.383 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.384 DEBUG:boto:Data: 2022-09-03T05:33:52.384 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.385 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:52.385 DEBUG:boto:Port: 8001 2022-09-03T05:33:52.386 DEBUG:boto:Params: {} 2022-09-03T05:33:52.387 DEBUG:boto:Token: None 2022-09-03T05:33:52.387 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.388 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hubxnre7BSFevkqcEdYg16CJWV4= 2022-09-03T05:33:52.389 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hubxnre7BSFevkqcEdYg16CJWV4=', 'Content-Length': '0'} 2022-09-03T05:33:52.391 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000939574cda3fb4ed3-006312e740-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.391 DEBUG:boto:path=/dsgvcy-16/dummy 2022-09-03T05:33:52.392 DEBUG:boto:auth_path=/dsgvcy-16/dummy 2022-09-03T05:33:52.392 DEBUG:boto:path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.393 DEBUG:boto:auth_path=/dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.393 DEBUG:boto:Method: GET 2022-09-03T05:33:52.394 DEBUG:boto:Path: /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.394 DEBUG:boto:Data: 2022-09-03T05:33:52.395 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.395 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:52.396 DEBUG:boto:Port: 8000 2022-09-03T05:33:52.397 DEBUG:boto:Params: {} 2022-09-03T05:33:52.397 DEBUG:boto:Token: None 2022-09-03T05:33:52.398 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/dummy?versionId=iJxQIs7jrKchIKwzf3kPx8w51PQyH2M 2022-09-03T05:33:52.398 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hubxnre7BSFevkqcEdYg16CJWV4= 2022-09-03T05:33:52.399 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hubxnre7BSFevkqcEdYg16CJWV4=', 'Content-Length': '0'} 2022-09-03T05:33:52.401 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:37 GMT'), ('x-amz-version-id', 'iJxQIs7jrKchIKwzf3kPx8w51PQyH2M'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000c4fce099580a91c5-006312e740-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.402 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:52.402 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:52.403 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:52.403 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:52.404 DEBUG:boto:Method: GET 2022-09-03T05:33:52.405 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:33:52.405 DEBUG:boto:Data: 2022-09-03T05:33:52.406 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.406 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:52.407 DEBUG:boto:Port: 8001 2022-09-03T05:33:52.407 DEBUG:boto:Params: {} 2022-09-03T05:33:52.408 DEBUG:boto:Token: None 2022-09-03T05:33:52.408 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/ 2022-09-03T05:33:52.409 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s= 2022-09-03T05:33:52.410 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s=', 'Content-Length': '0'} 2022-09-03T05:33:52.413 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fc27605089aedd8c-006312e740-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.415 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:52.417 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:52.417 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:52.418 DEBUG:boto:path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:52.418 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=dummy 2022-09-03T05:33:52.419 DEBUG:boto:Method: GET 2022-09-03T05:33:52.419 DEBUG:boto:Path: /dsgvcy-16/?prefix=dummy 2022-09-03T05:33:52.420 DEBUG:boto:Data: 2022-09-03T05:33:52.421 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.421 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:52.422 DEBUG:boto:Port: 8000 2022-09-03T05:33:52.423 DEBUG:boto:Params: {} 2022-09-03T05:33:52.424 DEBUG:boto:Token: None 2022-09-03T05:33:52.425 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/ 2022-09-03T05:33:52.426 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s= 2022-09-03T05:33:52.427 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s=', 'Content-Length': '0'} 2022-09-03T05:33:52.431 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006809f3672a6fc3bb-006312e740-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.432 DEBUG:boto:b'dsgvcy-16dummy1000falsedummy2022-09-03T05:27:37.833Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:52.433 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:33:52.434 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.435 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.436 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.437 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.438 DEBUG:boto:Method: GET 2022-09-03T05:33:52.438 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.439 DEBUG:boto:Data: 2022-09-03T05:33:52.439 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.440 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:52.440 DEBUG:boto:Port: 8001 2022-09-03T05:33:52.442 DEBUG:boto:Params: {} 2022-09-03T05:33:52.443 DEBUG:boto:Token: None 2022-09-03T05:33:52.444 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.444 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0rWT5OkNlinB6+bDYW2JLvoGDBA= 2022-09-03T05:33:52.445 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0rWT5OkNlinB6+bDYW2JLvoGDBA=', 'Content-Length': '0'} 2022-09-03T05:33:52.448 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000042590db6bfc3634-006312e740-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.492 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.494 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.494 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.495 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.496 DEBUG:boto:Method: GET 2022-09-03T05:33:52.497 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.498 DEBUG:boto:Data: 2022-09-03T05:33:52.499 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.501 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:52.502 DEBUG:boto:Port: 8000 2022-09-03T05:33:52.503 DEBUG:boto:Params: {} 2022-09-03T05:33:52.505 DEBUG:boto:Token: None 2022-09-03T05:33:52.506 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.507 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0rWT5OkNlinB6+bDYW2JLvoGDBA= 2022-09-03T05:33:52.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0rWT5OkNlinB6+bDYW2JLvoGDBA=', 'Content-Length': '0'} 2022-09-03T05:33:52.512 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000958ca83db4c2c0dd-006312e740-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.514 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.514 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.515 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.516 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.517 DEBUG:boto:Method: HEAD 2022-09-03T05:33:52.518 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.520 DEBUG:boto:Data: 2022-09-03T05:33:52.521 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.521 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:52.522 DEBUG:boto:Port: 8001 2022-09-03T05:33:52.523 DEBUG:boto:Params: {} 2022-09-03T05:33:52.524 DEBUG:boto:Token: None 2022-09-03T05:33:52.525 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.525 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vv20WtSciV9th3noT81q5g9ytqE= 2022-09-03T05:33:52.526 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vv20WtSciV9th3noT81q5g9ytqE=', 'Content-Length': '0'} 2022-09-03T05:33:52.528 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b852d7492c1fa201-006312e740-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.529 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.530 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.531 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.532 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.534 DEBUG:boto:Method: HEAD 2022-09-03T05:33:52.535 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.536 DEBUG:boto:Data: 2022-09-03T05:33:52.537 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.537 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:52.538 DEBUG:boto:Port: 8000 2022-09-03T05:33:52.540 DEBUG:boto:Params: {} 2022-09-03T05:33:52.541 DEBUG:boto:Token: None 2022-09-03T05:33:52.542 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.543 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vv20WtSciV9th3noT81q5g9ytqE= 2022-09-03T05:33:52.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vv20WtSciV9th3noT81q5g9ytqE=', 'Content-Length': '0'} 2022-09-03T05:33:52.546 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007adc653acb843a52-006312e740-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.547 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:33:52.548 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.549 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.550 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.550 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.551 DEBUG:boto:Method: GET 2022-09-03T05:33:52.552 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.553 DEBUG:boto:Data: 2022-09-03T05:33:52.554 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.555 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:52.555 DEBUG:boto:Port: 8001 2022-09-03T05:33:52.556 DEBUG:boto:Params: {} 2022-09-03T05:33:52.557 DEBUG:boto:Token: None 2022-09-03T05:33:52.558 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.559 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0rWT5OkNlinB6+bDYW2JLvoGDBA= 2022-09-03T05:33:52.560 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0rWT5OkNlinB6+bDYW2JLvoGDBA=', 'Content-Length': '0'} 2022-09-03T05:33:52.563 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d1f195eb4f904069-006312e740-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.565 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.566 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.567 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.568 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.569 DEBUG:boto:Method: GET 2022-09-03T05:33:52.570 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.571 DEBUG:boto:Data: 2022-09-03T05:33:52.572 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.573 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:52.574 DEBUG:boto:Port: 8000 2022-09-03T05:33:52.575 DEBUG:boto:Params: {} 2022-09-03T05:33:52.576 DEBUG:boto:Token: None 2022-09-03T05:33:52.577 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.578 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0rWT5OkNlinB6+bDYW2JLvoGDBA= 2022-09-03T05:33:52.579 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0rWT5OkNlinB6+bDYW2JLvoGDBA=', 'Content-Length': '0'} 2022-09-03T05:33:52.582 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003b624488737a692a-006312e740-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.626 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:52.627 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:52.628 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:52.629 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:52.630 DEBUG:boto:Method: GET 2022-09-03T05:33:52.631 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:52.632 DEBUG:boto:Data: 2022-09-03T05:33:52.633 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.633 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:52.635 DEBUG:boto:Port: 8001 2022-09-03T05:33:52.635 DEBUG:boto:Params: {} 2022-09-03T05:33:52.636 DEBUG:boto:Token: None 2022-09-03T05:33:52.637 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/ 2022-09-03T05:33:52.639 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s= 2022-09-03T05:33:52.640 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s=', 'Content-Length': '0'} 2022-09-03T05:33:52.643 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000074b24181f30e3585-006312e740-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.644 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:52.645 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:52.646 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:52.646 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:52.647 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:52.648 DEBUG:boto:Method: GET 2022-09-03T05:33:52.649 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:52.649 DEBUG:boto:Data: 2022-09-03T05:33:52.651 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.651 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:52.652 DEBUG:boto:Port: 8000 2022-09-03T05:33:52.653 DEBUG:boto:Params: {} 2022-09-03T05:33:52.654 DEBUG:boto:Token: None 2022-09-03T05:33:52.654 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/ 2022-09-03T05:33:52.655 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s= 2022-09-03T05:33:52.655 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s=', 'Content-Length': '0'} 2022-09-03T05:33:52.658 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000afc88f66715da8a0-006312e740-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.659 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:52.661 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.662 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.663 DEBUG:boto:Method: HEAD 2022-09-03T05:33:52.664 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.665 DEBUG:boto:Data: 2022-09-03T05:33:52.666 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.667 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:52.667 DEBUG:boto:Port: 8001 2022-09-03T05:33:52.668 DEBUG:boto:Params: {} 2022-09-03T05:33:52.668 DEBUG:boto:Token: None 2022-09-03T05:33:52.670 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.671 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:epDtjDZGBzcdPz3wPMP/1bDYjVQ= 2022-09-03T05:33:52.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:epDtjDZGBzcdPz3wPMP/1bDYjVQ=', 'Content-Length': '0'} 2022-09-03T05:33:52.675 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000068a5bc0e0f2539a1-006312e740-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.676 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.677 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.677 DEBUG:boto:Method: HEAD 2022-09-03T05:33:52.678 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.679 DEBUG:boto:Data: 2022-09-03T05:33:52.680 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.681 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:52.682 DEBUG:boto:Port: 8000 2022-09-03T05:33:52.682 DEBUG:boto:Params: {} 2022-09-03T05:33:52.683 DEBUG:boto:Token: None 2022-09-03T05:33:52.684 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.685 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:epDtjDZGBzcdPz3wPMP/1bDYjVQ= 2022-09-03T05:33:52.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:epDtjDZGBzcdPz3wPMP/1bDYjVQ=', 'Content-Length': '0'} 2022-09-03T05:33:52.688 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007b19375ffca1e842-006312e740-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.689 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:33:52.689 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.690 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.690 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.691 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.692 DEBUG:boto:Method: GET 2022-09-03T05:33:52.692 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.693 DEBUG:boto:Data: 2022-09-03T05:33:52.693 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.694 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:52.694 DEBUG:boto:Port: 8001 2022-09-03T05:33:52.696 DEBUG:boto:Params: {} 2022-09-03T05:33:52.697 DEBUG:boto:Token: None 2022-09-03T05:33:52.697 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.698 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0rWT5OkNlinB6+bDYW2JLvoGDBA= 2022-09-03T05:33:52.698 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0rWT5OkNlinB6+bDYW2JLvoGDBA=', 'Content-Length': '0'} 2022-09-03T05:33:52.701 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000813104daeda986b7-006312e740-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.742 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.743 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1 2022-09-03T05:33:52.743 DEBUG:boto:path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.744 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.745 DEBUG:boto:Method: GET 2022-09-03T05:33:52.745 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.746 DEBUG:boto:Data: 2022-09-03T05:33:52.746 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.747 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:52.748 DEBUG:boto:Port: 8000 2022-09-03T05:33:52.748 DEBUG:boto:Params: {} 2022-09-03T05:33:52.749 DEBUG:boto:Token: None 2022-09-03T05:33:52.750 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/obj-test-zone1?versionId=Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH 2022-09-03T05:33:52.750 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0rWT5OkNlinB6+bDYW2JLvoGDBA= 2022-09-03T05:33:52.751 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0rWT5OkNlinB6+bDYW2JLvoGDBA=', 'Content-Length': '0'} 2022-09-03T05:33:52.754 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'Ypw07nUES8aCnyjd6kp6dRzyDQhB0VH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d1d1ac88f1e9624f-006312e740-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.755 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:52.755 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:52.756 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:52.756 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:52.757 DEBUG:boto:Method: GET 2022-09-03T05:33:52.758 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:52.759 DEBUG:boto:Data: 2022-09-03T05:33:52.760 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.761 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:52.762 DEBUG:boto:Port: 8001 2022-09-03T05:33:52.763 DEBUG:boto:Params: {} 2022-09-03T05:33:52.764 DEBUG:boto:Token: None 2022-09-03T05:33:52.765 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/ 2022-09-03T05:33:52.766 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s= 2022-09-03T05:33:52.767 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s=', 'Content-Length': '0'} 2022-09-03T05:33:52.770 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ac032336d3f01087-006312e740-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.772 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:52.773 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:52.774 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:52.775 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:52.776 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:52.777 DEBUG:boto:Method: GET 2022-09-03T05:33:52.778 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone1 2022-09-03T05:33:52.779 DEBUG:boto:Data: 2022-09-03T05:33:52.780 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.781 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:52.782 DEBUG:boto:Port: 8000 2022-09-03T05:33:52.783 DEBUG:boto:Params: {} 2022-09-03T05:33:52.784 DEBUG:boto:Token: None 2022-09-03T05:33:52.785 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/ 2022-09-03T05:33:52.785 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s= 2022-09-03T05:33:52.786 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s=', 'Content-Length': '0'} 2022-09-03T05:33:52.789 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008241a0123be3cc5a-006312e740-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.834 DEBUG:boto:b'dsgvcy-16obj-test-zone11000falseobj-test-zone12022-09-03T05:29:08.869Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:52.835 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:33:52.835 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:52.836 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:52.837 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.837 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.838 DEBUG:boto:Method: GET 2022-09-03T05:33:52.839 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.839 DEBUG:boto:Data: 2022-09-03T05:33:52.840 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.840 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:52.841 DEBUG:boto:Port: 8001 2022-09-03T05:33:52.841 DEBUG:boto:Params: {} 2022-09-03T05:33:52.842 DEBUG:boto:Token: None 2022-09-03T05:33:52.843 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.843 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XaYo4n9pX75xCkq/6fhnlcKnPrM= 2022-09-03T05:33:52.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XaYo4n9pX75xCkq/6fhnlcKnPrM=', 'Content-Length': '0'} 2022-09-03T05:33:52.846 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f148c38f7b1ebf9c-006312e740-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.847 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:52.848 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:52.848 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.849 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.849 DEBUG:boto:Method: GET 2022-09-03T05:33:52.850 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.850 DEBUG:boto:Data: 2022-09-03T05:33:52.851 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.852 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:52.852 DEBUG:boto:Port: 8000 2022-09-03T05:33:52.853 DEBUG:boto:Params: {} 2022-09-03T05:33:52.853 DEBUG:boto:Token: None 2022-09-03T05:33:52.854 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.855 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XaYo4n9pX75xCkq/6fhnlcKnPrM= 2022-09-03T05:33:52.855 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XaYo4n9pX75xCkq/6fhnlcKnPrM=', 'Content-Length': '0'} 2022-09-03T05:33:52.857 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000feb330c551c500fb-006312e740-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.858 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:52.859 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:52.860 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.860 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.861 DEBUG:boto:Method: HEAD 2022-09-03T05:33:52.862 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.863 DEBUG:boto:Data: 2022-09-03T05:33:52.864 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.865 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:52.866 DEBUG:boto:Port: 8001 2022-09-03T05:33:52.866 DEBUG:boto:Params: {} 2022-09-03T05:33:52.867 DEBUG:boto:Token: None 2022-09-03T05:33:52.867 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.868 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZaLuWXq91+GmpULDE5ZygDTw1GY= 2022-09-03T05:33:52.869 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZaLuWXq91+GmpULDE5ZygDTw1GY=', 'Content-Length': '0'} 2022-09-03T05:33:52.871 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b1aa84881d1eaabd-006312e740-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.872 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:52.872 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:52.873 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.873 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.874 DEBUG:boto:Method: HEAD 2022-09-03T05:33:52.874 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.875 DEBUG:boto:Data: 2022-09-03T05:33:52.875 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.876 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:52.876 DEBUG:boto:Port: 8000 2022-09-03T05:33:52.877 DEBUG:boto:Params: {} 2022-09-03T05:33:52.878 DEBUG:boto:Token: None 2022-09-03T05:33:52.878 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.879 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZaLuWXq91+GmpULDE5ZygDTw1GY= 2022-09-03T05:33:52.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZaLuWXq91+GmpULDE5ZygDTw1GY=', 'Content-Length': '0'} 2022-09-03T05:33:52.881 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000928a4e94a2331647-006312e740-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.882 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:33:52.883 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:52.884 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:52.885 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.885 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.886 DEBUG:boto:Method: GET 2022-09-03T05:33:52.887 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.887 DEBUG:boto:Data: 2022-09-03T05:33:52.888 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.888 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:52.889 DEBUG:boto:Port: 8001 2022-09-03T05:33:52.889 DEBUG:boto:Params: {} 2022-09-03T05:33:52.890 DEBUG:boto:Token: None 2022-09-03T05:33:52.891 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.891 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XaYo4n9pX75xCkq/6fhnlcKnPrM= 2022-09-03T05:33:52.893 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XaYo4n9pX75xCkq/6fhnlcKnPrM=', 'Content-Length': '0'} 2022-09-03T05:33:52.896 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000095a35495558158bf-006312e740-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.942 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:52.943 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:52.944 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.944 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.945 DEBUG:boto:Method: GET 2022-09-03T05:33:52.946 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.946 DEBUG:boto:Data: 2022-09-03T05:33:52.947 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.947 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:52.948 DEBUG:boto:Port: 8000 2022-09-03T05:33:52.948 DEBUG:boto:Params: {} 2022-09-03T05:33:52.949 DEBUG:boto:Token: None 2022-09-03T05:33:52.950 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:52.950 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XaYo4n9pX75xCkq/6fhnlcKnPrM= 2022-09-03T05:33:52.951 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XaYo4n9pX75xCkq/6fhnlcKnPrM=', 'Content-Length': '0'} 2022-09-03T05:33:52.953 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007217c0612250fc4f-006312e740-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.954 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:52.955 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:52.955 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:52.956 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:52.956 DEBUG:boto:Method: GET 2022-09-03T05:33:52.957 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:52.957 DEBUG:boto:Data: 2022-09-03T05:33:52.958 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.958 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:52.959 DEBUG:boto:Port: 8001 2022-09-03T05:33:52.960 DEBUG:boto:Params: {} 2022-09-03T05:33:52.960 DEBUG:boto:Token: None 2022-09-03T05:33:52.961 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/ 2022-09-03T05:33:52.961 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s= 2022-09-03T05:33:52.962 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s=', 'Content-Length': '0'} 2022-09-03T05:33:52.965 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000065a094aa8c7fe945-006312e740-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.966 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:52.967 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:52.968 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:52.968 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:52.969 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:52.970 DEBUG:boto:Method: GET 2022-09-03T05:33:52.971 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:52.972 DEBUG:boto:Data: 2022-09-03T05:33:52.973 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.973 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:52.974 DEBUG:boto:Port: 8000 2022-09-03T05:33:52.975 DEBUG:boto:Params: {} 2022-09-03T05:33:52.976 DEBUG:boto:Token: None 2022-09-03T05:33:52.978 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/ 2022-09-03T05:33:52.979 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s= 2022-09-03T05:33:52.980 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jNfsa2A5J9PUAxcasxHN2cWj64s=', 'Content-Length': '0'} 2022-09-03T05:33:52.983 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009985c3ca53dda10d-006312e740-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.984 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:52.985 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:52.985 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:52.986 DEBUG:boto:Method: HEAD 2022-09-03T05:33:52.986 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2 2022-09-03T05:33:52.987 DEBUG:boto:Data: 2022-09-03T05:33:52.987 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.988 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:52.988 DEBUG:boto:Port: 8001 2022-09-03T05:33:52.989 DEBUG:boto:Params: {} 2022-09-03T05:33:52.990 DEBUG:boto:Token: None 2022-09-03T05:33:52.990 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:52 GMT /dsgvcy-16/obj-test-zone2 2022-09-03T05:33:52.991 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R4pidlWNlgZRsPi18zMtptgsX4E= 2022-09-03T05:33:52.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R4pidlWNlgZRsPi18zMtptgsX4E=', 'Content-Length': '0'} 2022-09-03T05:33:52.994 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b19a50e24e020c56-006312e740-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:52.996 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:52.996 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:52.997 DEBUG:boto:Method: HEAD 2022-09-03T05:33:52.997 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2 2022-09-03T05:33:52.998 DEBUG:boto:Data: 2022-09-03T05:33:52.998 DEBUG:boto:Headers: {} 2022-09-03T05:33:52.999 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:53.000 DEBUG:boto:Port: 8000 2022-09-03T05:33:53.000 DEBUG:boto:Params: {} 2022-09-03T05:33:53.001 DEBUG:boto:Token: None 2022-09-03T05:33:53.001 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:53 GMT /dsgvcy-16/obj-test-zone2 2022-09-03T05:33:53.002 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iYzOBqz5oIHF8BLzPekI6WlxXrM= 2022-09-03T05:33:53.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iYzOBqz5oIHF8BLzPekI6WlxXrM=', 'Content-Length': '0'} 2022-09-03T05:33:53.005 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000048fe3a8931725811-006312e741-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:53.006 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:33:53.007 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:53.007 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:53.008 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:53.008 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:53.009 DEBUG:boto:Method: GET 2022-09-03T05:33:53.009 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:53.010 DEBUG:boto:Data: 2022-09-03T05:33:53.011 DEBUG:boto:Headers: {} 2022-09-03T05:33:53.011 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:53.012 DEBUG:boto:Port: 8001 2022-09-03T05:33:53.013 DEBUG:boto:Params: {} 2022-09-03T05:33:53.014 DEBUG:boto:Token: None 2022-09-03T05:33:53.015 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:53 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:53.016 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NpVbuoa3bbpYZixuBX8HC1lFTMs= 2022-09-03T05:33:53.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NpVbuoa3bbpYZixuBX8HC1lFTMs=', 'Content-Length': '0'} 2022-09-03T05:33:53.020 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000057204442b2436455-006312e741-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:53.021 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:53.021 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2 2022-09-03T05:33:53.022 DEBUG:boto:path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:53.022 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:53.023 DEBUG:boto:Method: GET 2022-09-03T05:33:53.024 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:53.024 DEBUG:boto:Data: 2022-09-03T05:33:53.025 DEBUG:boto:Headers: {} 2022-09-03T05:33:53.025 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:53.026 DEBUG:boto:Port: 8000 2022-09-03T05:33:53.026 DEBUG:boto:Params: {} 2022-09-03T05:33:53.027 DEBUG:boto:Token: None 2022-09-03T05:33:53.027 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:53 GMT /dsgvcy-16/obj-test-zone2?versionId=soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow 2022-09-03T05:33:53.028 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NpVbuoa3bbpYZixuBX8HC1lFTMs= 2022-09-03T05:33:53.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NpVbuoa3bbpYZixuBX8HC1lFTMs=', 'Content-Length': '0'} 2022-09-03T05:33:53.032 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'soEESbBjrx2kX.Mbf5eKFZ9TCEg1Dow'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f681e9a7bc22518d-006312e741-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:53.032 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:53.033 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:53.034 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:53.034 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:53.035 DEBUG:boto:Method: GET 2022-09-03T05:33:53.035 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:53.036 DEBUG:boto:Data: 2022-09-03T05:33:53.037 DEBUG:boto:Headers: {} 2022-09-03T05:33:53.037 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:53.038 DEBUG:boto:Port: 8001 2022-09-03T05:33:53.038 DEBUG:boto:Params: {} 2022-09-03T05:33:53.039 DEBUG:boto:Token: None 2022-09-03T05:33:53.039 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:53 GMT /dsgvcy-16/ 2022-09-03T05:33:53.040 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aNiHWqQ3/0xFazBuRF4ppq2Lh+8= 2022-09-03T05:33:53.041 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aNiHWqQ3/0xFazBuRF4ppq2Lh+8=', 'Content-Length': '0'} 2022-09-03T05:33:53.043 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006e932da546e89c01-006312e741-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:53.044 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:53.045 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:53.046 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:53.046 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:53.047 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:53.047 DEBUG:boto:Method: GET 2022-09-03T05:33:53.048 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone2 2022-09-03T05:33:53.048 DEBUG:boto:Data: 2022-09-03T05:33:53.049 DEBUG:boto:Headers: {} 2022-09-03T05:33:53.049 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:53.050 DEBUG:boto:Port: 8000 2022-09-03T05:33:53.050 DEBUG:boto:Params: {} 2022-09-03T05:33:53.051 DEBUG:boto:Token: None 2022-09-03T05:33:53.051 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:53 GMT /dsgvcy-16/ 2022-09-03T05:33:53.052 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aNiHWqQ3/0xFazBuRF4ppq2Lh+8= 2022-09-03T05:33:53.053 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aNiHWqQ3/0xFazBuRF4ppq2Lh+8=', 'Content-Length': '0'} 2022-09-03T05:33:53.055 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001897ecd95a1e1e3f-006312e741-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:53.098 DEBUG:boto:b'dsgvcy-16obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.047Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:53.100 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:33:53.100 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.101 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.102 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.103 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.104 DEBUG:boto:Method: GET 2022-09-03T05:33:53.104 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.105 DEBUG:boto:Data: 2022-09-03T05:33:53.106 DEBUG:boto:Headers: {} 2022-09-03T05:33:53.106 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:53.107 DEBUG:boto:Port: 8001 2022-09-03T05:33:53.107 DEBUG:boto:Params: {} 2022-09-03T05:33:53.108 DEBUG:boto:Token: None 2022-09-03T05:33:53.109 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:53 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.109 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AzkpHP7IY+qSoIiAyweHxHKWwgQ= 2022-09-03T05:33:53.110 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AzkpHP7IY+qSoIiAyweHxHKWwgQ=', 'Content-Length': '0'} 2022-09-03T05:33:53.113 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bf881d72846d2de5-006312e741-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:53.113 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.114 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.115 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.116 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.116 DEBUG:boto:Method: GET 2022-09-03T05:33:53.117 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.117 DEBUG:boto:Data: 2022-09-03T05:33:53.118 DEBUG:boto:Headers: {} 2022-09-03T05:33:53.119 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:53.120 DEBUG:boto:Port: 8000 2022-09-03T05:33:53.121 DEBUG:boto:Params: {} 2022-09-03T05:33:53.121 DEBUG:boto:Token: None 2022-09-03T05:33:53.122 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:53 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.123 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AzkpHP7IY+qSoIiAyweHxHKWwgQ= 2022-09-03T05:33:53.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AzkpHP7IY+qSoIiAyweHxHKWwgQ=', 'Content-Length': '0'} 2022-09-03T05:33:53.127 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000006d45599212e51e1-006312e741-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:53.128 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.129 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.130 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.130 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.131 DEBUG:boto:Method: HEAD 2022-09-03T05:33:53.131 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.132 DEBUG:boto:Data: 2022-09-03T05:33:53.132 DEBUG:boto:Headers: {} 2022-09-03T05:33:53.133 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:53.133 DEBUG:boto:Port: 8001 2022-09-03T05:33:53.134 DEBUG:boto:Params: {} 2022-09-03T05:33:53.134 DEBUG:boto:Token: None 2022-09-03T05:33:53.135 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:53 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.136 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bRtytA3T9XG+xSMBiiHy59eCO9Y= 2022-09-03T05:33:53.137 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bRtytA3T9XG+xSMBiiHy59eCO9Y=', 'Content-Length': '0'} 2022-09-03T05:33:53.139 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f3f2f54057beb282-006312e741-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:53.141 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.142 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.142 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.143 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.144 DEBUG:boto:Method: HEAD 2022-09-03T05:33:53.145 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.146 DEBUG:boto:Data: 2022-09-03T05:33:53.146 DEBUG:boto:Headers: {} 2022-09-03T05:33:53.147 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:53.148 DEBUG:boto:Port: 8000 2022-09-03T05:33:53.149 DEBUG:boto:Params: {} 2022-09-03T05:33:53.150 DEBUG:boto:Token: None 2022-09-03T05:33:53.151 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:53 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.152 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bRtytA3T9XG+xSMBiiHy59eCO9Y= 2022-09-03T05:33:53.153 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bRtytA3T9XG+xSMBiiHy59eCO9Y=', 'Content-Length': '0'} 2022-09-03T05:33:53.155 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000632e6316e81ed5fa-006312e741-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:53.156 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:33:53.157 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.158 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.158 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.159 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.159 DEBUG:boto:Method: GET 2022-09-03T05:33:53.160 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.160 DEBUG:boto:Data: 2022-09-03T05:33:53.161 DEBUG:boto:Headers: {} 2022-09-03T05:33:53.162 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:53.163 DEBUG:boto:Port: 8001 2022-09-03T05:33:53.164 DEBUG:boto:Params: {} 2022-09-03T05:33:53.165 DEBUG:boto:Token: None 2022-09-03T05:33:53.166 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:53 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.167 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AzkpHP7IY+qSoIiAyweHxHKWwgQ= 2022-09-03T05:33:53.169 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AzkpHP7IY+qSoIiAyweHxHKWwgQ=', 'Content-Length': '0'} 2022-09-03T05:33:53.171 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b6ea378e3a24c381-006312e741-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:53.172 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.172 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.173 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.174 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.174 DEBUG:boto:Method: GET 2022-09-03T05:33:53.175 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.175 DEBUG:boto:Data: 2022-09-03T05:33:53.176 DEBUG:boto:Headers: {} 2022-09-03T05:33:53.176 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:53.177 DEBUG:boto:Port: 8000 2022-09-03T05:33:53.178 DEBUG:boto:Params: {} 2022-09-03T05:33:53.178 DEBUG:boto:Token: None 2022-09-03T05:33:53.179 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:53 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.180 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AzkpHP7IY+qSoIiAyweHxHKWwgQ= 2022-09-03T05:33:53.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AzkpHP7IY+qSoIiAyweHxHKWwgQ=', 'Content-Length': '0'} 2022-09-03T05:33:53.183 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b76c43b66bf4bf96-006312e741-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:53.185 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:53.185 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:53.186 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:53.186 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:53.187 DEBUG:boto:Method: GET 2022-09-03T05:33:53.188 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:53.188 DEBUG:boto:Data: 2022-09-03T05:33:53.189 DEBUG:boto:Headers: {} 2022-09-03T05:33:53.189 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:53.190 DEBUG:boto:Port: 8001 2022-09-03T05:33:53.190 DEBUG:boto:Params: {} 2022-09-03T05:33:53.191 DEBUG:boto:Token: None 2022-09-03T05:33:53.192 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:53 GMT /dsgvcy-16/ 2022-09-03T05:33:53.192 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aNiHWqQ3/0xFazBuRF4ppq2Lh+8= 2022-09-03T05:33:53.193 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aNiHWqQ3/0xFazBuRF4ppq2Lh+8=', 'Content-Length': '0'} 2022-09-03T05:33:53.196 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e4853e24bd546096-006312e741-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:53.242 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:53.244 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:53.245 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:53.246 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:53.247 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:53.247 DEBUG:boto:Method: GET 2022-09-03T05:33:53.248 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:53.249 DEBUG:boto:Data: 2022-09-03T05:33:53.250 DEBUG:boto:Headers: {} 2022-09-03T05:33:53.251 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:53.252 DEBUG:boto:Port: 8000 2022-09-03T05:33:53.253 DEBUG:boto:Params: {} 2022-09-03T05:33:53.254 DEBUG:boto:Token: None 2022-09-03T05:33:53.255 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:53 GMT /dsgvcy-16/ 2022-09-03T05:33:53.256 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aNiHWqQ3/0xFazBuRF4ppq2Lh+8= 2022-09-03T05:33:53.256 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aNiHWqQ3/0xFazBuRF4ppq2Lh+8=', 'Content-Length': '0'} 2022-09-03T05:33:53.259 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009adb40f2f59f89a0-006312e741-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:53.260 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:53.261 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.263 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.264 DEBUG:boto:Method: HEAD 2022-09-03T05:33:53.264 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.265 DEBUG:boto:Data: 2022-09-03T05:33:53.265 DEBUG:boto:Headers: {} 2022-09-03T05:33:53.266 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:53.267 DEBUG:boto:Port: 8001 2022-09-03T05:33:53.267 DEBUG:boto:Params: {} 2022-09-03T05:33:53.268 DEBUG:boto:Token: None 2022-09-03T05:33:53.268 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:53 GMT /dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.269 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tpqHULjPlm3RUFktx0xjfsTGeHc= 2022-09-03T05:33:53.271 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tpqHULjPlm3RUFktx0xjfsTGeHc=', 'Content-Length': '0'} 2022-09-03T05:33:53.274 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b3a3b476b2add4da-006312e741-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:53.285 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.285 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.286 DEBUG:boto:Method: HEAD 2022-09-03T05:33:53.286 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.287 DEBUG:boto:Data: 2022-09-03T05:33:53.288 DEBUG:boto:Headers: {} 2022-09-03T05:33:53.288 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:53.289 DEBUG:boto:Port: 8000 2022-09-03T05:33:53.289 DEBUG:boto:Params: {} 2022-09-03T05:33:53.290 DEBUG:boto:Token: None 2022-09-03T05:33:53.291 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:53 GMT /dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.291 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tpqHULjPlm3RUFktx0xjfsTGeHc= 2022-09-03T05:33:53.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tpqHULjPlm3RUFktx0xjfsTGeHc=', 'Content-Length': '0'} 2022-09-03T05:33:53.295 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000759eecaa530d69ad-006312e741-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:53.296 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:33:53.296 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.297 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.298 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.298 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.299 DEBUG:boto:Method: GET 2022-09-03T05:33:53.299 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.300 DEBUG:boto:Data: 2022-09-03T05:33:53.301 DEBUG:boto:Headers: {} 2022-09-03T05:33:53.301 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:53.302 DEBUG:boto:Port: 8001 2022-09-03T05:33:53.302 DEBUG:boto:Params: {} 2022-09-03T05:33:53.303 DEBUG:boto:Token: None 2022-09-03T05:33:53.304 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:53 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.305 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AzkpHP7IY+qSoIiAyweHxHKWwgQ= 2022-09-03T05:33:53.305 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AzkpHP7IY+qSoIiAyweHxHKWwgQ=', 'Content-Length': '0'} 2022-09-03T05:33:53.308 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000031281b423b8b9e71-006312e741-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:53.309 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.309 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3 2022-09-03T05:33:53.310 DEBUG:boto:path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.310 DEBUG:boto:auth_path=/dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.311 DEBUG:boto:Method: GET 2022-09-03T05:33:53.312 DEBUG:boto:Path: /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.312 DEBUG:boto:Data: 2022-09-03T05:33:53.313 DEBUG:boto:Headers: {} 2022-09-03T05:33:53.313 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:53.314 DEBUG:boto:Port: 8000 2022-09-03T05:33:53.314 DEBUG:boto:Params: {} 2022-09-03T05:33:53.315 DEBUG:boto:Token: None 2022-09-03T05:33:53.316 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:53 GMT /dsgvcy-16/obj-test-zone3?versionId=dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW 2022-09-03T05:33:53.317 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AzkpHP7IY+qSoIiAyweHxHKWwgQ= 2022-09-03T05:33:53.317 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AzkpHP7IY+qSoIiAyweHxHKWwgQ=', 'Content-Length': '0'} 2022-09-03T05:33:53.319 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'dUn8Ra0ctzrQABA.gxa8sYuyxMYvtFW'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002b342272aab33c35-006312e741-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:53.362 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:53.363 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:53.364 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:53.364 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:53.365 DEBUG:boto:Method: GET 2022-09-03T05:33:53.365 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:53.366 DEBUG:boto:Data: 2022-09-03T05:33:53.367 DEBUG:boto:Headers: {} 2022-09-03T05:33:53.367 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:53.368 DEBUG:boto:Port: 8001 2022-09-03T05:33:53.369 DEBUG:boto:Params: {} 2022-09-03T05:33:53.371 DEBUG:boto:Token: None 2022-09-03T05:33:53.372 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:53 GMT /dsgvcy-16/ 2022-09-03T05:33:53.373 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aNiHWqQ3/0xFazBuRF4ppq2Lh+8= 2022-09-03T05:33:53.375 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aNiHWqQ3/0xFazBuRF4ppq2Lh+8=', 'Content-Length': '0'} 2022-09-03T05:33:53.378 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e958531c26131b94-006312e741-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:53.379 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:53.380 DEBUG:boto:path=/dsgvcy-16/ 2022-09-03T05:33:53.381 DEBUG:boto:auth_path=/dsgvcy-16/ 2022-09-03T05:33:53.382 DEBUG:boto:path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:53.382 DEBUG:boto:auth_path=/dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:53.383 DEBUG:boto:Method: GET 2022-09-03T05:33:53.383 DEBUG:boto:Path: /dsgvcy-16/?prefix=obj-test-zone3 2022-09-03T05:33:53.384 DEBUG:boto:Data: 2022-09-03T05:33:53.385 DEBUG:boto:Headers: {} 2022-09-03T05:33:53.386 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:53.387 DEBUG:boto:Port: 8000 2022-09-03T05:33:53.387 DEBUG:boto:Params: {} 2022-09-03T05:33:53.388 DEBUG:boto:Token: None 2022-09-03T05:33:53.388 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:53 GMT /dsgvcy-16/ 2022-09-03T05:33:53.389 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aNiHWqQ3/0xFazBuRF4ppq2Lh+8= 2022-09-03T05:33:53.390 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aNiHWqQ3/0xFazBuRF4ppq2Lh+8=', 'Content-Length': '0'} 2022-09-03T05:33:53.393 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fe599d93618dde2b-006312e741-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:53.393 DEBUG:boto:b'dsgvcy-16obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.204Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:53.394 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-16 zones={test-zone3, test-zone2} 2022-09-03T05:33:53.395 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-17 2022-09-03T05:33:53.396 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:53.396 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:53.397 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-17 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:33:53.461 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:33:53.461 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:33:53.506 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:33:53.507+0000 7f6774d16600 1 bucket sync caught up with source: 2022-09-03T05:33:53.506 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000011.34.5, , 00000000004.16.5, , 00000000009.24.5, , , , , ] 2022-09-03T05:33:53.507 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000011.34.5, , 00000000004.16.5, , 00000000009.24.5, , , , , ] 2022-09-03T05:33:53.507 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:33:53.507+0000 7f6774d16600 0 bucket checkpoint complete 2022-09-03T05:33:53.519 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-17 2022-09-03T05:33:53.520 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:53.521 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:53.523 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-17 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:33:53.588 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:33:53.589 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:33:53.630 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:53.631+0000 7f0aa5ec7600 1 bucket sync caught up with source: 2022-09-03T05:33:53.630 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000011.34.5, , 00000000004.16.5, , 00000000009.24.5, , , , , ] 2022-09-03T05:33:53.631 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000011.34.5, , 00000000004.16.5, , 00000000009.24.5, , , , , ] 2022-09-03T05:33:53.631 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:53.631+0000 7f0aa5ec7600 0 bucket checkpoint complete 2022-09-03T05:33:53.653 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-17 2022-09-03T05:33:53.653 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:53.654 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:53.655 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-17 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:33:53.712 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:33:53.712 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:33:53.744 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:33:53.745+0000 7f1979f79600 1 bucket sync caught up with source: 2022-09-03T05:33:53.744 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000011.34.5, , 00000000004.16.5, , 00000000009.24.5, , , , , ] 2022-09-03T05:33:53.745 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000011.34.5, , 00000000004.16.5, , 00000000009.24.5, , , , , ] 2022-09-03T05:33:53.745 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:33:53.745+0000 7f1979f79600 0 bucket checkpoint complete 2022-09-03T05:33:53.755 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-17 2022-09-03T05:33:53.756 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:53.756 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:53.757 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-17 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:33:53.816 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:33:53.816 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:33:53.862 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:53.864+0000 7f00070e4600 1 bucket sync caught up with source: 2022-09-03T05:33:53.863 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.36.5, , 00000000011.68.5, , 00000000010.42.5, , , , , ] 2022-09-03T05:33:53.863 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.36.5, , 00000000011.68.5, , 00000000010.42.5, , , , , ] 2022-09-03T05:33:53.863 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:53.864+0000 7f00070e4600 0 bucket checkpoint complete 2022-09-03T05:33:53.880 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-17 2022-09-03T05:33:53.881 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:53.882 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:53.882 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-17 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:33:53.939 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:33:53.940 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:33:53.987 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:53.989+0000 7f6437f6b600 1 bucket sync caught up with source: 2022-09-03T05:33:53.988 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.36.5, , 00000000011.68.5, , 00000000010.42.5, , , , , ] 2022-09-03T05:33:53.989 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.36.5, , 00000000011.68.5, , 00000000010.42.5, , , , , ] 2022-09-03T05:33:53.989 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:53.989+0000 7f6437f6b600 0 bucket checkpoint complete 2022-09-03T05:33:54.003 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-17 2022-09-03T05:33:54.004 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-17 2022-09-03T05:33:54.005 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:54.006 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:54.007 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-17 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:33:54.064 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:33:54.065 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:33:54.101 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:54.101+0000 7fa618748600 1 bucket sync caught up with source: 2022-09-03T05:33:54.103 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.32.5, , 00000000004.39.5, , 00000000015.54.5, , , , , ] 2022-09-03T05:33:54.104 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.32.5, , 00000000004.39.5, , 00000000015.54.5, , , , , ] 2022-09-03T05:33:54.105 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:54.101+0000 7fa618748600 0 bucket checkpoint complete 2022-09-03T05:33:54.122 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-17 2022-09-03T05:33:54.123 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:54.124 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-17', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:54.124 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-17 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:33:54.181 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:33:54.181 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:33:54.215 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:33:54.216+0000 7f4a492af600 1 bucket sync caught up with source: 2022-09-03T05:33:54.216 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000004.32.5, , 00000000004.39.5, , 00000000015.54.5, , , , , ] 2022-09-03T05:33:54.216 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000004.32.5, , 00000000004.39.5, , 00000000015.54.5, , , , , ] 2022-09-03T05:33:54.216 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:33:54.216+0000 7f4a492af600 0 bucket checkpoint complete 2022-09-03T05:33:54.227 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-17 2022-09-03T05:33:54.227 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-17 zones={test-zone2, test-zone1} 2022-09-03T05:33:54.228 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:54.229 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:54.230 DEBUG:boto:Method: HEAD 2022-09-03T05:33:54.230 DEBUG:boto:Path: /dsgvcy-17/ 2022-09-03T05:33:54.231 DEBUG:boto:Data: 2022-09-03T05:33:54.231 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.232 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:54.232 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.233 DEBUG:boto:Params: {} 2022-09-03T05:33:54.234 DEBUG:boto:Token: None 2022-09-03T05:33:54.234 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/ 2022-09-03T05:33:54.235 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QcmMdv0qGXoyrv4O1syUJXzbkb4= 2022-09-03T05:33:54.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QcmMdv0qGXoyrv4O1syUJXzbkb4=', 'Content-Length': '0'} 2022-09-03T05:33:54.238 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000410b2bfb84bdb283-006312e742-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.239 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:54.239 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:54.240 DEBUG:boto:Method: HEAD 2022-09-03T05:33:54.240 DEBUG:boto:Path: /dsgvcy-17/ 2022-09-03T05:33:54.241 DEBUG:boto:Data: 2022-09-03T05:33:54.241 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.242 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:54.243 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.243 DEBUG:boto:Params: {} 2022-09-03T05:33:54.244 DEBUG:boto:Token: None 2022-09-03T05:33:54.244 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/ 2022-09-03T05:33:54.245 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QcmMdv0qGXoyrv4O1syUJXzbkb4= 2022-09-03T05:33:54.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QcmMdv0qGXoyrv4O1syUJXzbkb4=', 'Content-Length': '0'} 2022-09-03T05:33:54.248 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000334db8ce1a098d77-006312e742-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.249 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:33:54.249 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:54.250 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:54.250 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:54.251 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:54.252 DEBUG:boto:Method: GET 2022-09-03T05:33:54.252 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:54.253 DEBUG:boto:Data: 2022-09-03T05:33:54.253 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.254 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:54.254 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.256 DEBUG:boto:Params: {} 2022-09-03T05:33:54.256 DEBUG:boto:Token: None 2022-09-03T05:33:54.257 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/?versions 2022-09-03T05:33:54.258 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6j34FxLz2PYhe8xCfQOD/50ZnV0= 2022-09-03T05:33:54.259 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6j34FxLz2PYhe8xCfQOD/50ZnV0=', 'Content-Length': '0'} 2022-09-03T05:33:54.262 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008c0561a6ba88c8cb-006312e742-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.263 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:54.264 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:33:54.265 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:33:54.266 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:33:54.267 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:33:54.268 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:33:54.270 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:54.271 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:54.272 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:54.273 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:54.274 DEBUG:boto:Method: GET 2022-09-03T05:33:54.275 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:54.276 DEBUG:boto:Data: 2022-09-03T05:33:54.277 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.278 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:54.279 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.281 DEBUG:boto:Params: {} 2022-09-03T05:33:54.282 DEBUG:boto:Token: None 2022-09-03T05:33:54.283 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/?versions 2022-09-03T05:33:54.284 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6j34FxLz2PYhe8xCfQOD/50ZnV0= 2022-09-03T05:33:54.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6j34FxLz2PYhe8xCfQOD/50ZnV0=', 'Content-Length': '0'} 2022-09-03T05:33:54.287 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005e452f32a7282509-006312e742-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.287 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:54.288 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:33:54.289 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:33:54.289 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:33:54.290 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:33:54.290 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:54.291 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:54.292 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:54.293 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:54.294 DEBUG:boto:Method: GET 2022-09-03T05:33:54.294 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:54.295 DEBUG:boto:Data: 2022-09-03T05:33:54.296 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.298 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:54.299 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.299 DEBUG:boto:Params: {} 2022-09-03T05:33:54.300 DEBUG:boto:Token: None 2022-09-03T05:33:54.301 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/?versions 2022-09-03T05:33:54.302 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6j34FxLz2PYhe8xCfQOD/50ZnV0= 2022-09-03T05:33:54.303 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6j34FxLz2PYhe8xCfQOD/50ZnV0=', 'Content-Length': '0'} 2022-09-03T05:33:54.306 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009714142972217701-006312e742-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.307 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:54.309 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:54.309 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:54.310 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:54.311 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:54.312 DEBUG:boto:Method: GET 2022-09-03T05:33:54.312 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:54.313 DEBUG:boto:Data: 2022-09-03T05:33:54.314 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.315 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:54.316 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.316 DEBUG:boto:Params: {} 2022-09-03T05:33:54.317 DEBUG:boto:Token: None 2022-09-03T05:33:54.318 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/?versions 2022-09-03T05:33:54.319 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6j34FxLz2PYhe8xCfQOD/50ZnV0= 2022-09-03T05:33:54.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6j34FxLz2PYhe8xCfQOD/50ZnV0=', 'Content-Length': '0'} 2022-09-03T05:33:54.323 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000014a42428f5f3c5ac-006312e742-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.323 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:54.325 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:33:54.326 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:54.327 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:54.327 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.328 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.329 DEBUG:boto:Method: GET 2022-09-03T05:33:54.330 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.331 DEBUG:boto:Data: 2022-09-03T05:33:54.332 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.332 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:54.333 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.333 DEBUG:boto:Params: {} 2022-09-03T05:33:54.335 DEBUG:boto:Token: None 2022-09-03T05:33:54.337 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.337 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZlVYy5a4bwmbpFbZi0LB6cKl+sc= 2022-09-03T05:33:54.338 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZlVYy5a4bwmbpFbZi0LB6cKl+sc=', 'Content-Length': '0'} 2022-09-03T05:33:54.340 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000006c0abbdf393f2286-006312e742-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.341 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:54.341 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:54.342 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.343 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.345 DEBUG:boto:Method: GET 2022-09-03T05:33:54.345 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.346 DEBUG:boto:Data: 2022-09-03T05:33:54.347 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.347 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:54.349 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.349 DEBUG:boto:Params: {} 2022-09-03T05:33:54.350 DEBUG:boto:Token: None 2022-09-03T05:33:54.350 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.351 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZlVYy5a4bwmbpFbZi0LB6cKl+sc= 2022-09-03T05:33:54.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZlVYy5a4bwmbpFbZi0LB6cKl+sc=', 'Content-Length': '0'} 2022-09-03T05:33:54.354 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000004a9b32ef7bee9ada-006312e742-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.354 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:54.355 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:54.355 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.356 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.356 DEBUG:boto:Method: HEAD 2022-09-03T05:33:54.357 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.357 DEBUG:boto:Data: 2022-09-03T05:33:54.358 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.358 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:54.359 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.359 DEBUG:boto:Params: {} 2022-09-03T05:33:54.360 DEBUG:boto:Token: None 2022-09-03T05:33:54.361 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.361 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZVmCN8z18sgh/Q3967si/gKhANc= 2022-09-03T05:33:54.362 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZVmCN8z18sgh/Q3967si/gKhANc=', 'Content-Length': '0'} 2022-09-03T05:33:54.364 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f409c3ac89f7a1d1-006312e742-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.364 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:54.365 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:54.365 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.366 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.367 DEBUG:boto:Method: HEAD 2022-09-03T05:33:54.367 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.368 DEBUG:boto:Data: 2022-09-03T05:33:54.368 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.369 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:54.369 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.370 DEBUG:boto:Params: {} 2022-09-03T05:33:54.370 DEBUG:boto:Token: None 2022-09-03T05:33:54.371 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.372 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZVmCN8z18sgh/Q3967si/gKhANc= 2022-09-03T05:33:54.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZVmCN8z18sgh/Q3967si/gKhANc=', 'Content-Length': '0'} 2022-09-03T05:33:54.374 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000067c6a89a27b53d34-006312e742-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.375 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:33:54.375 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:54.376 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:54.376 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.377 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.377 DEBUG:boto:Method: GET 2022-09-03T05:33:54.378 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.379 DEBUG:boto:Data: 2022-09-03T05:33:54.379 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.380 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:54.380 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.381 DEBUG:boto:Params: {} 2022-09-03T05:33:54.381 DEBUG:boto:Token: None 2022-09-03T05:33:54.382 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.382 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZlVYy5a4bwmbpFbZi0LB6cKl+sc= 2022-09-03T05:33:54.383 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZlVYy5a4bwmbpFbZi0LB6cKl+sc=', 'Content-Length': '0'} 2022-09-03T05:33:54.386 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f238e3cd21078ad9-006312e742-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.387 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:54.388 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:54.389 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.390 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.391 DEBUG:boto:Method: GET 2022-09-03T05:33:54.392 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.392 DEBUG:boto:Data: 2022-09-03T05:33:54.393 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.394 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:54.394 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.395 DEBUG:boto:Params: {} 2022-09-03T05:33:54.395 DEBUG:boto:Token: None 2022-09-03T05:33:54.396 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.397 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZlVYy5a4bwmbpFbZi0LB6cKl+sc= 2022-09-03T05:33:54.398 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZlVYy5a4bwmbpFbZi0LB6cKl+sc=', 'Content-Length': '0'} 2022-09-03T05:33:54.400 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000000ee77274be7a8e0-006312e742-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.401 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:54.401 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:54.402 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:54.403 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:54.404 DEBUG:boto:Method: GET 2022-09-03T05:33:54.405 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:33:54.406 DEBUG:boto:Data: 2022-09-03T05:33:54.407 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.408 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:54.409 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.410 DEBUG:boto:Params: {} 2022-09-03T05:33:54.411 DEBUG:boto:Token: None 2022-09-03T05:33:54.412 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/ 2022-09-03T05:33:54.413 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fvc8zBqTOKS+NqM5IkAHStxGl/Y= 2022-09-03T05:33:54.414 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fvc8zBqTOKS+NqM5IkAHStxGl/Y=', 'Content-Length': '0'} 2022-09-03T05:33:54.416 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d3f4f72c1f4e609e-006312e742-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.458 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:54.459 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:54.460 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:54.461 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:54.462 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:54.462 DEBUG:boto:Method: GET 2022-09-03T05:33:54.463 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:33:54.464 DEBUG:boto:Data: 2022-09-03T05:33:54.464 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.465 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:54.465 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.466 DEBUG:boto:Params: {} 2022-09-03T05:33:54.467 DEBUG:boto:Token: None 2022-09-03T05:33:54.468 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/ 2022-09-03T05:33:54.469 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fvc8zBqTOKS+NqM5IkAHStxGl/Y= 2022-09-03T05:33:54.470 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fvc8zBqTOKS+NqM5IkAHStxGl/Y=', 'Content-Length': '0'} 2022-09-03T05:33:54.473 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000985694e4573a5f5a-006312e742-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.474 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:54.475 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:54.475 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:54.476 DEBUG:boto:Method: HEAD 2022-09-03T05:33:54.477 DEBUG:boto:Path: /dsgvcy-17/dummy 2022-09-03T05:33:54.478 DEBUG:boto:Data: 2022-09-03T05:33:54.478 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.479 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:54.479 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.480 DEBUG:boto:Params: {} 2022-09-03T05:33:54.480 DEBUG:boto:Token: None 2022-09-03T05:33:54.481 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/dummy 2022-09-03T05:33:54.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:g7vIVYqx3LCGzx3x+By692eQID8= 2022-09-03T05:33:54.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:g7vIVYqx3LCGzx3x+By692eQID8=', 'Content-Length': '0'} 2022-09-03T05:33:54.486 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000ea65f5bd6e3cc3d8-006312e742-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.487 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:54.488 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:54.488 DEBUG:boto:Method: HEAD 2022-09-03T05:33:54.489 DEBUG:boto:Path: /dsgvcy-17/dummy 2022-09-03T05:33:54.489 DEBUG:boto:Data: 2022-09-03T05:33:54.490 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.490 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:54.491 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.491 DEBUG:boto:Params: {} 2022-09-03T05:33:54.492 DEBUG:boto:Token: None 2022-09-03T05:33:54.493 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/dummy 2022-09-03T05:33:54.494 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:g7vIVYqx3LCGzx3x+By692eQID8= 2022-09-03T05:33:54.495 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:g7vIVYqx3LCGzx3x+By692eQID8=', 'Content-Length': '0'} 2022-09-03T05:33:54.498 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000001f71bb2dc9a90733-006312e742-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.499 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:33:54.500 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:54.500 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:54.501 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.501 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.502 DEBUG:boto:Method: GET 2022-09-03T05:33:54.503 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.503 DEBUG:boto:Data: 2022-09-03T05:33:54.504 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.504 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:54.505 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.506 DEBUG:boto:Params: {} 2022-09-03T05:33:54.506 DEBUG:boto:Token: None 2022-09-03T05:33:54.507 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.508 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZlVYy5a4bwmbpFbZi0LB6cKl+sc= 2022-09-03T05:33:54.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZlVYy5a4bwmbpFbZi0LB6cKl+sc=', 'Content-Length': '0'} 2022-09-03T05:33:54.510 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000004ce8c3b3d76f0384-006312e742-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.511 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:54.512 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:54.512 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.513 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.513 DEBUG:boto:Method: GET 2022-09-03T05:33:54.514 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.514 DEBUG:boto:Data: 2022-09-03T05:33:54.515 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.515 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:54.516 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.517 DEBUG:boto:Params: {} 2022-09-03T05:33:54.517 DEBUG:boto:Token: None 2022-09-03T05:33:54.518 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:54.519 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZlVYy5a4bwmbpFbZi0LB6cKl+sc= 2022-09-03T05:33:54.519 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZlVYy5a4bwmbpFbZi0LB6cKl+sc=', 'Content-Length': '0'} 2022-09-03T05:33:54.521 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000fa292e0ce27bc97e-006312e742-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.522 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:54.523 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:54.523 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:54.524 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:54.524 DEBUG:boto:Method: GET 2022-09-03T05:33:54.525 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:33:54.525 DEBUG:boto:Data: 2022-09-03T05:33:54.526 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.526 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:54.527 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.527 DEBUG:boto:Params: {} 2022-09-03T05:33:54.528 DEBUG:boto:Token: None 2022-09-03T05:33:54.529 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/ 2022-09-03T05:33:54.530 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fvc8zBqTOKS+NqM5IkAHStxGl/Y= 2022-09-03T05:33:54.530 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fvc8zBqTOKS+NqM5IkAHStxGl/Y=', 'Content-Length': '0'} 2022-09-03T05:33:54.533 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000069808fe6c8ce4096-006312e742-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.576 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:54.578 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:54.579 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:54.580 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:54.581 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:54.582 DEBUG:boto:Method: GET 2022-09-03T05:33:54.583 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:33:54.584 DEBUG:boto:Data: 2022-09-03T05:33:54.585 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.585 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:54.586 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.587 DEBUG:boto:Params: {} 2022-09-03T05:33:54.589 DEBUG:boto:Token: None 2022-09-03T05:33:54.590 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/ 2022-09-03T05:33:54.591 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fvc8zBqTOKS+NqM5IkAHStxGl/Y= 2022-09-03T05:33:54.592 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fvc8zBqTOKS+NqM5IkAHStxGl/Y=', 'Content-Length': '0'} 2022-09-03T05:33:54.595 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000891982e32f5c56c9-006312e742-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.595 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:54.597 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:33:54.598 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.598 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.599 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.599 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.600 DEBUG:boto:Method: GET 2022-09-03T05:33:54.600 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.601 DEBUG:boto:Data: 2022-09-03T05:33:54.602 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.602 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:54.603 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.603 DEBUG:boto:Params: {} 2022-09-03T05:33:54.604 DEBUG:boto:Token: None 2022-09-03T05:33:54.605 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.605 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2X9LDaYp9WWQyR3OanGW+Ct/5rw= 2022-09-03T05:33:54.606 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2X9LDaYp9WWQyR3OanGW+Ct/5rw=', 'Content-Length': '0'} 2022-09-03T05:33:54.608 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000080af67c7f4d165b4-006312e742-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.609 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.610 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.610 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.611 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.611 DEBUG:boto:Method: GET 2022-09-03T05:33:54.612 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.613 DEBUG:boto:Data: 2022-09-03T05:33:54.613 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.614 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:54.615 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.615 DEBUG:boto:Params: {} 2022-09-03T05:33:54.616 DEBUG:boto:Token: None 2022-09-03T05:33:54.617 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.619 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2X9LDaYp9WWQyR3OanGW+Ct/5rw= 2022-09-03T05:33:54.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2X9LDaYp9WWQyR3OanGW+Ct/5rw=', 'Content-Length': '0'} 2022-09-03T05:33:54.622 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000ee6f1b37c710952-006312e742-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.623 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.623 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.624 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.624 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.625 DEBUG:boto:Method: HEAD 2022-09-03T05:33:54.625 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.626 DEBUG:boto:Data: 2022-09-03T05:33:54.626 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.627 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:54.627 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.628 DEBUG:boto:Params: {} 2022-09-03T05:33:54.629 DEBUG:boto:Token: None 2022-09-03T05:33:54.630 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.631 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zid4NrS5wbQacjLOhiyQWiA1888= 2022-09-03T05:33:54.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zid4NrS5wbQacjLOhiyQWiA1888=', 'Content-Length': '0'} 2022-09-03T05:33:54.634 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e02c03f8d2c50768-006312e742-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.636 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.637 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.637 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.638 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.639 DEBUG:boto:Method: HEAD 2022-09-03T05:33:54.640 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.641 DEBUG:boto:Data: 2022-09-03T05:33:54.641 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.642 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:54.642 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.643 DEBUG:boto:Params: {} 2022-09-03T05:33:54.643 DEBUG:boto:Token: None 2022-09-03T05:33:54.644 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.645 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zid4NrS5wbQacjLOhiyQWiA1888= 2022-09-03T05:33:54.645 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zid4NrS5wbQacjLOhiyQWiA1888=', 'Content-Length': '0'} 2022-09-03T05:33:54.647 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008e97926f47c1c089-006312e742-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.648 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:33:54.649 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.650 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.650 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.651 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.651 DEBUG:boto:Method: GET 2022-09-03T05:33:54.652 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.652 DEBUG:boto:Data: 2022-09-03T05:33:54.653 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.654 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:54.654 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.655 DEBUG:boto:Params: {} 2022-09-03T05:33:54.655 DEBUG:boto:Token: None 2022-09-03T05:33:54.656 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.657 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2X9LDaYp9WWQyR3OanGW+Ct/5rw= 2022-09-03T05:33:54.657 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2X9LDaYp9WWQyR3OanGW+Ct/5rw=', 'Content-Length': '0'} 2022-09-03T05:33:54.659 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000594bc75416e8fc78-006312e742-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.660 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.661 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.661 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.662 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.663 DEBUG:boto:Method: GET 2022-09-03T05:33:54.663 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.664 DEBUG:boto:Data: 2022-09-03T05:33:54.664 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.665 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:54.666 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.666 DEBUG:boto:Params: {} 2022-09-03T05:33:54.667 DEBUG:boto:Token: None 2022-09-03T05:33:54.667 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.668 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2X9LDaYp9WWQyR3OanGW+Ct/5rw= 2022-09-03T05:33:54.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2X9LDaYp9WWQyR3OanGW+Ct/5rw=', 'Content-Length': '0'} 2022-09-03T05:33:54.671 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b98deefb445492b8-006312e742-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.672 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:54.673 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:54.673 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:54.674 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:54.674 DEBUG:boto:Method: GET 2022-09-03T05:33:54.675 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:54.675 DEBUG:boto:Data: 2022-09-03T05:33:54.676 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.676 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:54.677 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.677 DEBUG:boto:Params: {} 2022-09-03T05:33:54.678 DEBUG:boto:Token: None 2022-09-03T05:33:54.679 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/ 2022-09-03T05:33:54.679 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fvc8zBqTOKS+NqM5IkAHStxGl/Y= 2022-09-03T05:33:54.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fvc8zBqTOKS+NqM5IkAHStxGl/Y=', 'Content-Length': '0'} 2022-09-03T05:33:54.682 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000027788392142a69fc-006312e742-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.683 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:54.684 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:54.685 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:54.686 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:54.686 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:54.687 DEBUG:boto:Method: GET 2022-09-03T05:33:54.687 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:54.688 DEBUG:boto:Data: 2022-09-03T05:33:54.689 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.689 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:54.690 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.691 DEBUG:boto:Params: {} 2022-09-03T05:33:54.691 DEBUG:boto:Token: None 2022-09-03T05:33:54.692 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/ 2022-09-03T05:33:54.693 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fvc8zBqTOKS+NqM5IkAHStxGl/Y= 2022-09-03T05:33:54.694 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fvc8zBqTOKS+NqM5IkAHStxGl/Y=', 'Content-Length': '0'} 2022-09-03T05:33:54.696 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007cb29bb8715234fb-006312e742-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.697 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:54.698 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.699 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.700 DEBUG:boto:Method: HEAD 2022-09-03T05:33:54.700 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.701 DEBUG:boto:Data: 2022-09-03T05:33:54.702 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.703 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:54.703 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.704 DEBUG:boto:Params: {} 2022-09-03T05:33:54.704 DEBUG:boto:Token: None 2022-09-03T05:33:54.705 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.706 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eE/U7mxp3O7V6H0oq8BH8QESKt0= 2022-09-03T05:33:54.707 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eE/U7mxp3O7V6H0oq8BH8QESKt0=', 'Content-Length': '0'} 2022-09-03T05:33:54.709 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000315f8361a5d7949f-006312e742-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.710 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.711 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.712 DEBUG:boto:Method: HEAD 2022-09-03T05:33:54.712 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.713 DEBUG:boto:Data: 2022-09-03T05:33:54.713 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.714 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:54.714 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.715 DEBUG:boto:Params: {} 2022-09-03T05:33:54.716 DEBUG:boto:Token: None 2022-09-03T05:33:54.716 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.718 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eE/U7mxp3O7V6H0oq8BH8QESKt0= 2022-09-03T05:33:54.719 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eE/U7mxp3O7V6H0oq8BH8QESKt0=', 'Content-Length': '0'} 2022-09-03T05:33:54.722 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f1b81ab694e6e654-006312e742-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.723 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:33:54.723 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.724 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.725 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.725 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.726 DEBUG:boto:Method: GET 2022-09-03T05:33:54.726 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.727 DEBUG:boto:Data: 2022-09-03T05:33:54.728 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.729 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:54.730 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.731 DEBUG:boto:Params: {} 2022-09-03T05:33:54.732 DEBUG:boto:Token: None 2022-09-03T05:33:54.733 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.734 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2X9LDaYp9WWQyR3OanGW+Ct/5rw= 2022-09-03T05:33:54.735 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2X9LDaYp9WWQyR3OanGW+Ct/5rw=', 'Content-Length': '0'} 2022-09-03T05:33:54.736 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fca9d537f1371f7d-006312e742-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.778 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.779 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:54.780 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.782 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.783 DEBUG:boto:Method: GET 2022-09-03T05:33:54.783 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.784 DEBUG:boto:Data: 2022-09-03T05:33:54.785 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.786 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:54.788 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.789 DEBUG:boto:Params: {} 2022-09-03T05:33:54.790 DEBUG:boto:Token: None 2022-09-03T05:33:54.791 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:54.792 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2X9LDaYp9WWQyR3OanGW+Ct/5rw= 2022-09-03T05:33:54.793 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2X9LDaYp9WWQyR3OanGW+Ct/5rw=', 'Content-Length': '0'} 2022-09-03T05:33:54.795 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003606b2e80b7d29c6-006312e742-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.796 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:54.796 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:54.797 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:54.797 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:54.798 DEBUG:boto:Method: GET 2022-09-03T05:33:54.798 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:54.799 DEBUG:boto:Data: 2022-09-03T05:33:54.799 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.800 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:54.800 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.801 DEBUG:boto:Params: {} 2022-09-03T05:33:54.802 DEBUG:boto:Token: None 2022-09-03T05:33:54.802 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/ 2022-09-03T05:33:54.803 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fvc8zBqTOKS+NqM5IkAHStxGl/Y= 2022-09-03T05:33:54.803 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fvc8zBqTOKS+NqM5IkAHStxGl/Y=', 'Content-Length': '0'} 2022-09-03T05:33:54.805 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000911e4b3ef3455d9b-006312e742-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.806 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:54.807 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:54.808 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:54.808 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:54.809 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:54.809 DEBUG:boto:Method: GET 2022-09-03T05:33:54.810 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:54.810 DEBUG:boto:Data: 2022-09-03T05:33:54.811 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.811 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:54.812 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.812 DEBUG:boto:Params: {} 2022-09-03T05:33:54.813 DEBUG:boto:Token: None 2022-09-03T05:33:54.813 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/ 2022-09-03T05:33:54.814 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fvc8zBqTOKS+NqM5IkAHStxGl/Y= 2022-09-03T05:33:54.815 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fvc8zBqTOKS+NqM5IkAHStxGl/Y=', 'Content-Length': '0'} 2022-09-03T05:33:54.817 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ddf4af11de54455c-006312e742-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.818 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:54.819 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:33:54.820 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:54.821 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:54.822 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.823 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.824 DEBUG:boto:Method: GET 2022-09-03T05:33:54.825 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.826 DEBUG:boto:Data: 2022-09-03T05:33:54.827 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.828 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:54.828 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.829 DEBUG:boto:Params: {} 2022-09-03T05:33:54.829 DEBUG:boto:Token: None 2022-09-03T05:33:54.830 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.831 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xXgIwFtinKfkwSS2svqXgSwWbhM= 2022-09-03T05:33:54.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xXgIwFtinKfkwSS2svqXgSwWbhM=', 'Content-Length': '0'} 2022-09-03T05:33:54.834 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004679f4c047804e62-006312e742-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.834 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:54.835 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:54.836 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.836 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.837 DEBUG:boto:Method: GET 2022-09-03T05:33:54.837 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.838 DEBUG:boto:Data: 2022-09-03T05:33:54.838 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.839 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:54.839 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.840 DEBUG:boto:Params: {} 2022-09-03T05:33:54.841 DEBUG:boto:Token: None 2022-09-03T05:33:54.842 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.842 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xXgIwFtinKfkwSS2svqXgSwWbhM= 2022-09-03T05:33:54.843 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xXgIwFtinKfkwSS2svqXgSwWbhM=', 'Content-Length': '0'} 2022-09-03T05:33:54.846 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003a49b1908f2b045a-006312e742-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.847 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:54.848 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:54.849 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.850 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.851 DEBUG:boto:Method: HEAD 2022-09-03T05:33:54.852 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.853 DEBUG:boto:Data: 2022-09-03T05:33:54.854 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.855 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:54.856 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.857 DEBUG:boto:Params: {} 2022-09-03T05:33:54.858 DEBUG:boto:Token: None 2022-09-03T05:33:54.859 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.860 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PYVgJIyuSASP+2R1uceTuha56xE= 2022-09-03T05:33:54.861 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PYVgJIyuSASP+2R1uceTuha56xE=', 'Content-Length': '0'} 2022-09-03T05:33:54.867 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001e39040ce98e3368-006312e742-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.868 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:54.869 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:54.870 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.871 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.872 DEBUG:boto:Method: HEAD 2022-09-03T05:33:54.873 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.874 DEBUG:boto:Data: 2022-09-03T05:33:54.875 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.876 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:54.877 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.878 DEBUG:boto:Params: {} 2022-09-03T05:33:54.879 DEBUG:boto:Token: None 2022-09-03T05:33:54.880 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.881 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PYVgJIyuSASP+2R1uceTuha56xE= 2022-09-03T05:33:54.882 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PYVgJIyuSASP+2R1uceTuha56xE=', 'Content-Length': '0'} 2022-09-03T05:33:54.884 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f6fe11f968a7ae1b-006312e742-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.886 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:33:54.887 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:54.888 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:54.889 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.890 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.891 DEBUG:boto:Method: GET 2022-09-03T05:33:54.892 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.893 DEBUG:boto:Data: 2022-09-03T05:33:54.894 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.895 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:54.896 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.897 DEBUG:boto:Params: {} 2022-09-03T05:33:54.899 DEBUG:boto:Token: None 2022-09-03T05:33:54.900 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.901 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xXgIwFtinKfkwSS2svqXgSwWbhM= 2022-09-03T05:33:54.902 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xXgIwFtinKfkwSS2svqXgSwWbhM=', 'Content-Length': '0'} 2022-09-03T05:33:54.904 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007e37e88298260894-006312e742-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.905 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:54.906 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:54.907 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.908 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.909 DEBUG:boto:Method: GET 2022-09-03T05:33:54.909 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.910 DEBUG:boto:Data: 2022-09-03T05:33:54.911 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.912 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:54.913 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.914 DEBUG:boto:Params: {} 2022-09-03T05:33:54.915 DEBUG:boto:Token: None 2022-09-03T05:33:54.916 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.918 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xXgIwFtinKfkwSS2svqXgSwWbhM= 2022-09-03T05:33:54.919 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xXgIwFtinKfkwSS2svqXgSwWbhM=', 'Content-Length': '0'} 2022-09-03T05:33:54.921 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000149c2fd4e1a0295c-006312e742-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.922 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:54.923 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:54.924 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:54.925 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:54.925 DEBUG:boto:Method: GET 2022-09-03T05:33:54.926 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:54.927 DEBUG:boto:Data: 2022-09-03T05:33:54.927 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.928 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:54.929 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.929 DEBUG:boto:Params: {} 2022-09-03T05:33:54.930 DEBUG:boto:Token: None 2022-09-03T05:33:54.931 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/ 2022-09-03T05:33:54.932 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fvc8zBqTOKS+NqM5IkAHStxGl/Y= 2022-09-03T05:33:54.933 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fvc8zBqTOKS+NqM5IkAHStxGl/Y=', 'Content-Length': '0'} 2022-09-03T05:33:54.935 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c0eadeb973e062d8-006312e742-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.936 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:54.937 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:54.937 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:54.938 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:54.939 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:54.939 DEBUG:boto:Method: GET 2022-09-03T05:33:54.940 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:54.940 DEBUG:boto:Data: 2022-09-03T05:33:54.941 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.941 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:54.942 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.942 DEBUG:boto:Params: {} 2022-09-03T05:33:54.943 DEBUG:boto:Token: None 2022-09-03T05:33:54.944 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/ 2022-09-03T05:33:54.944 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fvc8zBqTOKS+NqM5IkAHStxGl/Y= 2022-09-03T05:33:54.945 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fvc8zBqTOKS+NqM5IkAHStxGl/Y=', 'Content-Length': '0'} 2022-09-03T05:33:54.948 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000069ae228e28065bb0-006312e742-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.949 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:54.949 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:54.950 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:54.951 DEBUG:boto:Method: HEAD 2022-09-03T05:33:54.951 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2 2022-09-03T05:33:54.952 DEBUG:boto:Data: 2022-09-03T05:33:54.952 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.953 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:54.953 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.954 DEBUG:boto:Params: {} 2022-09-03T05:33:54.954 DEBUG:boto:Token: None 2022-09-03T05:33:54.955 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/obj-test-zone2 2022-09-03T05:33:54.956 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oFKx5bkCtqtVy21NXjQByFMPWAU= 2022-09-03T05:33:54.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oFKx5bkCtqtVy21NXjQByFMPWAU=', 'Content-Length': '0'} 2022-09-03T05:33:54.960 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004a808012caafcfdc-006312e742-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.961 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:54.961 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:54.962 DEBUG:boto:Method: HEAD 2022-09-03T05:33:54.962 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2 2022-09-03T05:33:54.963 DEBUG:boto:Data: 2022-09-03T05:33:54.963 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.964 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:54.964 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.965 DEBUG:boto:Params: {} 2022-09-03T05:33:54.965 DEBUG:boto:Token: None 2022-09-03T05:33:54.966 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/obj-test-zone2 2022-09-03T05:33:54.967 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oFKx5bkCtqtVy21NXjQByFMPWAU= 2022-09-03T05:33:54.967 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oFKx5bkCtqtVy21NXjQByFMPWAU=', 'Content-Length': '0'} 2022-09-03T05:33:54.970 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fb043481b6198731-006312e742-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:54.971 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:33:54.972 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:54.972 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:54.973 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.973 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.974 DEBUG:boto:Method: GET 2022-09-03T05:33:54.974 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.975 DEBUG:boto:Data: 2022-09-03T05:33:54.975 DEBUG:boto:Headers: {} 2022-09-03T05:33:54.976 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:54.976 DEBUG:boto:Port: 8000 2022-09-03T05:33:54.977 DEBUG:boto:Params: {} 2022-09-03T05:33:54.978 DEBUG:boto:Token: None 2022-09-03T05:33:54.978 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:54 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:54.979 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xXgIwFtinKfkwSS2svqXgSwWbhM= 2022-09-03T05:33:54.980 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xXgIwFtinKfkwSS2svqXgSwWbhM=', 'Content-Length': '0'} 2022-09-03T05:33:54.982 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dab6f4a658175c76-006312e742-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.026 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:55.027 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:55.028 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:55.029 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:55.030 DEBUG:boto:Method: GET 2022-09-03T05:33:55.031 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:55.032 DEBUG:boto:Data: 2022-09-03T05:33:55.033 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.034 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.035 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.036 DEBUG:boto:Params: {} 2022-09-03T05:33:55.037 DEBUG:boto:Token: None 2022-09-03T05:33:55.038 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:55.039 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qJqPqSrDVbSp6H+OQBe2oYi0lGU= 2022-09-03T05:33:55.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qJqPqSrDVbSp6H+OQBe2oYi0lGU=', 'Content-Length': '0'} 2022-09-03T05:33:55.042 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000810ac30fceefa0e2-006312e743-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.044 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:55.045 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:55.046 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:55.046 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:55.047 DEBUG:boto:Method: GET 2022-09-03T05:33:55.048 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:55.049 DEBUG:boto:Data: 2022-09-03T05:33:55.050 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.050 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:55.051 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.052 DEBUG:boto:Params: {} 2022-09-03T05:33:55.052 DEBUG:boto:Token: None 2022-09-03T05:33:55.053 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/ 2022-09-03T05:33:55.054 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E= 2022-09-03T05:33:55.054 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E=', 'Content-Length': '0'} 2022-09-03T05:33:55.057 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c512210bbe29ca74-006312e743-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.057 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:55.059 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:55.059 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:55.060 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:55.061 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:55.061 DEBUG:boto:Method: GET 2022-09-03T05:33:55.062 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:55.062 DEBUG:boto:Data: 2022-09-03T05:33:55.063 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.064 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.064 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.065 DEBUG:boto:Params: {} 2022-09-03T05:33:55.066 DEBUG:boto:Token: None 2022-09-03T05:33:55.066 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/ 2022-09-03T05:33:55.067 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E= 2022-09-03T05:33:55.067 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E=', 'Content-Length': '0'} 2022-09-03T05:33:55.070 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000080f81c083e128224-006312e743-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.114 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:55.115 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:33:55.116 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.117 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.117 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.118 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.118 DEBUG:boto:Method: GET 2022-09-03T05:33:55.119 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.120 DEBUG:boto:Data: 2022-09-03T05:33:55.120 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.121 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:55.121 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.122 DEBUG:boto:Params: {} 2022-09-03T05:33:55.123 DEBUG:boto:Token: None 2022-09-03T05:33:55.123 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.124 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b2Uwlsc+V9jGneEsKPRJqW7fFJ0= 2022-09-03T05:33:55.125 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b2Uwlsc+V9jGneEsKPRJqW7fFJ0=', 'Content-Length': '0'} 2022-09-03T05:33:55.127 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a8443beb8e658e5c-006312e743-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.128 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.129 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.129 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.130 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.131 DEBUG:boto:Method: GET 2022-09-03T05:33:55.131 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.132 DEBUG:boto:Data: 2022-09-03T05:33:55.133 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.133 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.134 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.134 DEBUG:boto:Params: {} 2022-09-03T05:33:55.135 DEBUG:boto:Token: None 2022-09-03T05:33:55.136 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.136 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b2Uwlsc+V9jGneEsKPRJqW7fFJ0= 2022-09-03T05:33:55.137 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b2Uwlsc+V9jGneEsKPRJqW7fFJ0=', 'Content-Length': '0'} 2022-09-03T05:33:55.140 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001b59f2b2ef321e72-006312e743-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.141 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.141 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.142 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.142 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.143 DEBUG:boto:Method: HEAD 2022-09-03T05:33:55.143 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.144 DEBUG:boto:Data: 2022-09-03T05:33:55.144 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.145 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:55.145 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.146 DEBUG:boto:Params: {} 2022-09-03T05:33:55.146 DEBUG:boto:Token: None 2022-09-03T05:33:55.147 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.147 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dzQd+VaKenKrIvvNHgM0JvkdhyM= 2022-09-03T05:33:55.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dzQd+VaKenKrIvvNHgM0JvkdhyM=', 'Content-Length': '0'} 2022-09-03T05:33:55.150 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006d0aa91f79a1b9b2-006312e743-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.151 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.151 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.152 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.152 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.153 DEBUG:boto:Method: HEAD 2022-09-03T05:33:55.154 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.154 DEBUG:boto:Data: 2022-09-03T05:33:55.155 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.155 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.156 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.156 DEBUG:boto:Params: {} 2022-09-03T05:33:55.157 DEBUG:boto:Token: None 2022-09-03T05:33:55.158 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.158 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dzQd+VaKenKrIvvNHgM0JvkdhyM= 2022-09-03T05:33:55.159 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dzQd+VaKenKrIvvNHgM0JvkdhyM=', 'Content-Length': '0'} 2022-09-03T05:33:55.161 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003828bde094f1814b-006312e743-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.162 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:33:55.163 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.163 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.164 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.164 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.165 DEBUG:boto:Method: GET 2022-09-03T05:33:55.165 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.166 DEBUG:boto:Data: 2022-09-03T05:33:55.167 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.167 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:55.168 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.168 DEBUG:boto:Params: {} 2022-09-03T05:33:55.169 DEBUG:boto:Token: None 2022-09-03T05:33:55.170 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.171 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b2Uwlsc+V9jGneEsKPRJqW7fFJ0= 2022-09-03T05:33:55.171 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b2Uwlsc+V9jGneEsKPRJqW7fFJ0=', 'Content-Length': '0'} 2022-09-03T05:33:55.174 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000025554409ec77727b-006312e743-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.218 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.221 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.225 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.226 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.227 DEBUG:boto:Method: GET 2022-09-03T05:33:55.228 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.229 DEBUG:boto:Data: 2022-09-03T05:33:55.230 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.230 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.231 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.231 DEBUG:boto:Params: {} 2022-09-03T05:33:55.232 DEBUG:boto:Token: None 2022-09-03T05:33:55.233 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.234 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b2Uwlsc+V9jGneEsKPRJqW7fFJ0= 2022-09-03T05:33:55.234 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b2Uwlsc+V9jGneEsKPRJqW7fFJ0=', 'Content-Length': '0'} 2022-09-03T05:33:55.237 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000da5c6a5b6927016b-006312e743-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.238 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:55.238 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:55.239 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:55.239 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:55.240 DEBUG:boto:Method: GET 2022-09-03T05:33:55.240 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:55.241 DEBUG:boto:Data: 2022-09-03T05:33:55.241 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.242 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:55.242 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.243 DEBUG:boto:Params: {} 2022-09-03T05:33:55.244 DEBUG:boto:Token: None 2022-09-03T05:33:55.245 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/ 2022-09-03T05:33:55.245 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E= 2022-09-03T05:33:55.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E=', 'Content-Length': '0'} 2022-09-03T05:33:55.249 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000004f3f0de42c580e6-006312e743-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.249 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:55.250 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:55.251 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:55.251 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:55.252 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:55.252 DEBUG:boto:Method: GET 2022-09-03T05:33:55.253 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:55.254 DEBUG:boto:Data: 2022-09-03T05:33:55.254 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.255 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.255 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.256 DEBUG:boto:Params: {} 2022-09-03T05:33:55.256 DEBUG:boto:Token: None 2022-09-03T05:33:55.257 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/ 2022-09-03T05:33:55.257 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E= 2022-09-03T05:33:55.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E=', 'Content-Length': '0'} 2022-09-03T05:33:55.261 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000034783365ad0c84f4-006312e743-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.261 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:55.262 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.263 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.263 DEBUG:boto:Method: HEAD 2022-09-03T05:33:55.264 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.264 DEBUG:boto:Data: 2022-09-03T05:33:55.265 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.265 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:55.266 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.266 DEBUG:boto:Params: {} 2022-09-03T05:33:55.267 DEBUG:boto:Token: None 2022-09-03T05:33:55.267 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.268 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IFVzDPmOzqZ8orgp3R2KfFB+8NA= 2022-09-03T05:33:55.269 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IFVzDPmOzqZ8orgp3R2KfFB+8NA=', 'Content-Length': '0'} 2022-09-03T05:33:55.271 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004956ca349c90a27f-006312e743-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.272 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.272 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.273 DEBUG:boto:Method: HEAD 2022-09-03T05:33:55.273 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.274 DEBUG:boto:Data: 2022-09-03T05:33:55.274 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.275 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.275 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.276 DEBUG:boto:Params: {} 2022-09-03T05:33:55.276 DEBUG:boto:Token: None 2022-09-03T05:33:55.277 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.278 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IFVzDPmOzqZ8orgp3R2KfFB+8NA= 2022-09-03T05:33:55.279 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IFVzDPmOzqZ8orgp3R2KfFB+8NA=', 'Content-Length': '0'} 2022-09-03T05:33:55.281 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000019ad82ef3628c3b8-006312e743-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.282 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:33:55.282 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.283 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.283 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.284 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.285 DEBUG:boto:Method: GET 2022-09-03T05:33:55.285 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.286 DEBUG:boto:Data: 2022-09-03T05:33:55.286 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.287 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:55.287 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.288 DEBUG:boto:Params: {} 2022-09-03T05:33:55.288 DEBUG:boto:Token: None 2022-09-03T05:33:55.289 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.290 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b2Uwlsc+V9jGneEsKPRJqW7fFJ0= 2022-09-03T05:33:55.290 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b2Uwlsc+V9jGneEsKPRJqW7fFJ0=', 'Content-Length': '0'} 2022-09-03T05:33:55.292 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007c5f2ef6288d7e92-006312e743-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.293 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.300 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:55.302 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.303 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.304 DEBUG:boto:Method: GET 2022-09-03T05:33:55.305 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.306 DEBUG:boto:Data: 2022-09-03T05:33:55.307 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.307 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.308 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.309 DEBUG:boto:Params: {} 2022-09-03T05:33:55.309 DEBUG:boto:Token: None 2022-09-03T05:33:55.310 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:55.311 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b2Uwlsc+V9jGneEsKPRJqW7fFJ0= 2022-09-03T05:33:55.311 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b2Uwlsc+V9jGneEsKPRJqW7fFJ0=', 'Content-Length': '0'} 2022-09-03T05:33:55.313 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000b31ef1749c82ec7-006312e743-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.314 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:55.315 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:55.315 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:55.316 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:55.317 DEBUG:boto:Method: GET 2022-09-03T05:33:55.317 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:55.318 DEBUG:boto:Data: 2022-09-03T05:33:55.318 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.319 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:55.320 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.320 DEBUG:boto:Params: {} 2022-09-03T05:33:55.321 DEBUG:boto:Token: None 2022-09-03T05:33:55.322 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/ 2022-09-03T05:33:55.322 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E= 2022-09-03T05:33:55.323 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E=', 'Content-Length': '0'} 2022-09-03T05:33:55.326 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005881fa2b1897b2cf-006312e743-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.326 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:55.327 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:55.328 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:55.328 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:55.329 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:55.329 DEBUG:boto:Method: GET 2022-09-03T05:33:55.330 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:55.330 DEBUG:boto:Data: 2022-09-03T05:33:55.331 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.331 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.332 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.333 DEBUG:boto:Params: {} 2022-09-03T05:33:55.334 DEBUG:boto:Token: None 2022-09-03T05:33:55.334 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/ 2022-09-03T05:33:55.335 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E= 2022-09-03T05:33:55.336 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E=', 'Content-Length': '0'} 2022-09-03T05:33:55.338 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c691f634973164d8-006312e743-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.339 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:55.340 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-17 zones={test-zone2, test-zone1} 2022-09-03T05:33:55.340 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-17 zones={test-zone3, test-zone1} 2022-09-03T05:33:55.341 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:55.341 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:55.342 DEBUG:boto:Method: HEAD 2022-09-03T05:33:55.343 DEBUG:boto:Path: /dsgvcy-17/ 2022-09-03T05:33:55.343 DEBUG:boto:Data: 2022-09-03T05:33:55.344 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.344 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:55.345 DEBUG:boto:Port: 8001 2022-09-03T05:33:55.345 DEBUG:boto:Params: {} 2022-09-03T05:33:55.346 DEBUG:boto:Token: None 2022-09-03T05:33:55.346 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/ 2022-09-03T05:33:55.347 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1u9bajq+RJIAtn/84z3Z/AelAMo= 2022-09-03T05:33:55.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1u9bajq+RJIAtn/84z3Z/AelAMo=', 'Content-Length': '0'} 2022-09-03T05:33:55.355 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000023de81a13aa783a0-006312e743-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.356 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:55.357 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:55.358 DEBUG:boto:Method: HEAD 2022-09-03T05:33:55.359 DEBUG:boto:Path: /dsgvcy-17/ 2022-09-03T05:33:55.360 DEBUG:boto:Data: 2022-09-03T05:33:55.360 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.361 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.362 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.363 DEBUG:boto:Params: {} 2022-09-03T05:33:55.364 DEBUG:boto:Token: None 2022-09-03T05:33:55.365 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/ 2022-09-03T05:33:55.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1u9bajq+RJIAtn/84z3Z/AelAMo= 2022-09-03T05:33:55.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1u9bajq+RJIAtn/84z3Z/AelAMo=', 'Content-Length': '0'} 2022-09-03T05:33:55.370 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004ec4d83fff9099e4-006312e743-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.371 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:33:55.372 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:55.374 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:55.374 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:55.375 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:55.376 DEBUG:boto:Method: GET 2022-09-03T05:33:55.377 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:55.378 DEBUG:boto:Data: 2022-09-03T05:33:55.379 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.380 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:55.381 DEBUG:boto:Port: 8001 2022-09-03T05:33:55.382 DEBUG:boto:Params: {} 2022-09-03T05:33:55.383 DEBUG:boto:Token: None 2022-09-03T05:33:55.384 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/?versions 2022-09-03T05:33:55.384 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OfpJ8UqJzvcAwbgmryhymV7f2y4= 2022-09-03T05:33:55.385 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OfpJ8UqJzvcAwbgmryhymV7f2y4=', 'Content-Length': '0'} 2022-09-03T05:33:55.388 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aca688d02028ba68-006312e743-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.390 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:55.391 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:33:55.392 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:33:55.393 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:33:55.394 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:33:55.395 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:33:55.395 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:55.396 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:55.400 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:55.401 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:55.403 DEBUG:boto:Method: GET 2022-09-03T05:33:55.404 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:55.405 DEBUG:boto:Data: 2022-09-03T05:33:55.406 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.406 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.408 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.410 DEBUG:boto:Params: {} 2022-09-03T05:33:55.411 DEBUG:boto:Token: None 2022-09-03T05:33:55.411 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/?versions 2022-09-03T05:33:55.412 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OfpJ8UqJzvcAwbgmryhymV7f2y4= 2022-09-03T05:33:55.413 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OfpJ8UqJzvcAwbgmryhymV7f2y4=', 'Content-Length': '0'} 2022-09-03T05:33:55.415 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000074a76b6de46017b2-006312e743-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.416 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:55.417 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:33:55.417 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:33:55.418 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:33:55.421 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:33:55.422 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:55.423 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:55.423 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:55.424 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:55.425 DEBUG:boto:Method: GET 2022-09-03T05:33:55.426 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:55.427 DEBUG:boto:Data: 2022-09-03T05:33:55.427 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.428 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:55.430 DEBUG:boto:Port: 8001 2022-09-03T05:33:55.431 DEBUG:boto:Params: {} 2022-09-03T05:33:55.432 DEBUG:boto:Token: None 2022-09-03T05:33:55.432 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/?versions 2022-09-03T05:33:55.433 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OfpJ8UqJzvcAwbgmryhymV7f2y4= 2022-09-03T05:33:55.434 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OfpJ8UqJzvcAwbgmryhymV7f2y4=', 'Content-Length': '0'} 2022-09-03T05:33:55.437 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cc3cc00a1fd802df-006312e743-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.438 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:55.439 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:55.439 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:55.440 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:55.441 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:55.442 DEBUG:boto:Method: GET 2022-09-03T05:33:55.442 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:55.443 DEBUG:boto:Data: 2022-09-03T05:33:55.443 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.444 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.445 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.445 DEBUG:boto:Params: {} 2022-09-03T05:33:55.446 DEBUG:boto:Token: None 2022-09-03T05:33:55.447 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/?versions 2022-09-03T05:33:55.448 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OfpJ8UqJzvcAwbgmryhymV7f2y4= 2022-09-03T05:33:55.449 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OfpJ8UqJzvcAwbgmryhymV7f2y4=', 'Content-Length': '0'} 2022-09-03T05:33:55.452 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c7b08d2711471f97-006312e743-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.452 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:55.453 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:33:55.454 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:55.454 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:55.455 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.455 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.456 DEBUG:boto:Method: GET 2022-09-03T05:33:55.457 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.457 DEBUG:boto:Data: 2022-09-03T05:33:55.458 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.458 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:55.459 DEBUG:boto:Port: 8001 2022-09-03T05:33:55.459 DEBUG:boto:Params: {} 2022-09-03T05:33:55.460 DEBUG:boto:Token: None 2022-09-03T05:33:55.460 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.461 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y9SbnnTqCU1+X8fh1WxChXFcDKE= 2022-09-03T05:33:55.462 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y9SbnnTqCU1+X8fh1WxChXFcDKE=', 'Content-Length': '0'} 2022-09-03T05:33:55.464 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000025ce0fe35fb26ab0-006312e743-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.465 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:55.465 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:55.466 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.467 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.467 DEBUG:boto:Method: GET 2022-09-03T05:33:55.468 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.468 DEBUG:boto:Data: 2022-09-03T05:33:55.469 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.469 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.470 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.470 DEBUG:boto:Params: {} 2022-09-03T05:33:55.471 DEBUG:boto:Token: None 2022-09-03T05:33:55.472 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.472 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y9SbnnTqCU1+X8fh1WxChXFcDKE= 2022-09-03T05:33:55.473 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y9SbnnTqCU1+X8fh1WxChXFcDKE=', 'Content-Length': '0'} 2022-09-03T05:33:55.476 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000021c59f97e3cb4e3a-006312e743-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.478 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:55.478 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:55.479 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.481 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.482 DEBUG:boto:Method: HEAD 2022-09-03T05:33:55.484 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.485 DEBUG:boto:Data: 2022-09-03T05:33:55.486 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.487 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:55.488 DEBUG:boto:Port: 8001 2022-09-03T05:33:55.489 DEBUG:boto:Params: {} 2022-09-03T05:33:55.490 DEBUG:boto:Token: None 2022-09-03T05:33:55.491 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.492 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:apAPhv/ATeLXILYtlzmrpAXm1Wk= 2022-09-03T05:33:55.493 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:apAPhv/ATeLXILYtlzmrpAXm1Wk=', 'Content-Length': '0'} 2022-09-03T05:33:55.495 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000002c5e2cc884243c43-006312e743-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.496 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:55.497 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:55.498 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.498 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.499 DEBUG:boto:Method: HEAD 2022-09-03T05:33:55.499 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.500 DEBUG:boto:Data: 2022-09-03T05:33:55.501 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.501 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.502 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.503 DEBUG:boto:Params: {} 2022-09-03T05:33:55.504 DEBUG:boto:Token: None 2022-09-03T05:33:55.505 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.506 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:apAPhv/ATeLXILYtlzmrpAXm1Wk= 2022-09-03T05:33:55.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:apAPhv/ATeLXILYtlzmrpAXm1Wk=', 'Content-Length': '0'} 2022-09-03T05:33:55.511 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000008814b6d2ef8d291e-006312e743-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.513 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:33:55.514 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:55.514 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:55.515 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.516 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.517 DEBUG:boto:Method: GET 2022-09-03T05:33:55.518 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.519 DEBUG:boto:Data: 2022-09-03T05:33:55.520 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.520 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:55.521 DEBUG:boto:Port: 8001 2022-09-03T05:33:55.522 DEBUG:boto:Params: {} 2022-09-03T05:33:55.523 DEBUG:boto:Token: None 2022-09-03T05:33:55.524 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.525 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y9SbnnTqCU1+X8fh1WxChXFcDKE= 2022-09-03T05:33:55.526 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y9SbnnTqCU1+X8fh1WxChXFcDKE=', 'Content-Length': '0'} 2022-09-03T05:33:55.529 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000001526d0c8fa0f8c72-006312e743-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.530 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:55.532 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:55.533 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.534 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.535 DEBUG:boto:Method: GET 2022-09-03T05:33:55.535 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.537 DEBUG:boto:Data: 2022-09-03T05:33:55.538 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.539 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.540 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.541 DEBUG:boto:Params: {} 2022-09-03T05:33:55.542 DEBUG:boto:Token: None 2022-09-03T05:33:55.543 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.544 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y9SbnnTqCU1+X8fh1WxChXFcDKE= 2022-09-03T05:33:55.545 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y9SbnnTqCU1+X8fh1WxChXFcDKE=', 'Content-Length': '0'} 2022-09-03T05:33:55.547 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000006b2dd04f6cdbaf7-006312e743-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.548 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:55.549 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:55.549 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:55.550 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:55.550 DEBUG:boto:Method: GET 2022-09-03T05:33:55.551 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:33:55.551 DEBUG:boto:Data: 2022-09-03T05:33:55.552 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.552 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:55.553 DEBUG:boto:Port: 8001 2022-09-03T05:33:55.553 DEBUG:boto:Params: {} 2022-09-03T05:33:55.554 DEBUG:boto:Token: None 2022-09-03T05:33:55.555 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/ 2022-09-03T05:33:55.555 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E= 2022-09-03T05:33:55.556 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E=', 'Content-Length': '0'} 2022-09-03T05:33:55.559 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008b47ee0d919e12cf-006312e743-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.560 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:55.560 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:55.561 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:55.561 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:55.562 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:55.563 DEBUG:boto:Method: GET 2022-09-03T05:33:55.563 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:33:55.564 DEBUG:boto:Data: 2022-09-03T05:33:55.564 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.565 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.566 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.566 DEBUG:boto:Params: {} 2022-09-03T05:33:55.567 DEBUG:boto:Token: None 2022-09-03T05:33:55.567 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/ 2022-09-03T05:33:55.568 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E= 2022-09-03T05:33:55.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E=', 'Content-Length': '0'} 2022-09-03T05:33:55.571 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000094af3f97a8354c58-006312e743-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.614 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:55.616 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:55.617 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:55.618 DEBUG:boto:Method: HEAD 2022-09-03T05:33:55.619 DEBUG:boto:Path: /dsgvcy-17/dummy 2022-09-03T05:33:55.620 DEBUG:boto:Data: 2022-09-03T05:33:55.621 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.623 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:55.624 DEBUG:boto:Port: 8001 2022-09-03T05:33:55.624 DEBUG:boto:Params: {} 2022-09-03T05:33:55.626 DEBUG:boto:Token: None 2022-09-03T05:33:55.627 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/dummy 2022-09-03T05:33:55.628 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DoN2QaYeYCqvHKiS/Kog3bSCqEc= 2022-09-03T05:33:55.629 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DoN2QaYeYCqvHKiS/Kog3bSCqEc=', 'Content-Length': '0'} 2022-09-03T05:33:55.633 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000004de5840b76eb6452-006312e743-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.634 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:55.634 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:55.635 DEBUG:boto:Method: HEAD 2022-09-03T05:33:55.636 DEBUG:boto:Path: /dsgvcy-17/dummy 2022-09-03T05:33:55.636 DEBUG:boto:Data: 2022-09-03T05:33:55.637 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.638 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.638 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.639 DEBUG:boto:Params: {} 2022-09-03T05:33:55.639 DEBUG:boto:Token: None 2022-09-03T05:33:55.640 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/dummy 2022-09-03T05:33:55.641 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DoN2QaYeYCqvHKiS/Kog3bSCqEc= 2022-09-03T05:33:55.641 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DoN2QaYeYCqvHKiS/Kog3bSCqEc=', 'Content-Length': '0'} 2022-09-03T05:33:55.644 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000003cd8a155f12e6691-006312e743-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.645 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:33:55.645 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:55.646 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:55.651 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.652 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.652 DEBUG:boto:Method: GET 2022-09-03T05:33:55.653 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.655 DEBUG:boto:Data: 2022-09-03T05:33:55.655 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.656 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:55.657 DEBUG:boto:Port: 8001 2022-09-03T05:33:55.658 DEBUG:boto:Params: {} 2022-09-03T05:33:55.659 DEBUG:boto:Token: None 2022-09-03T05:33:55.660 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.660 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y9SbnnTqCU1+X8fh1WxChXFcDKE= 2022-09-03T05:33:55.661 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y9SbnnTqCU1+X8fh1WxChXFcDKE=', 'Content-Length': '0'} 2022-09-03T05:33:55.663 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000016834f8e33ab436f-006312e743-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.664 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:55.665 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:55.666 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.667 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.667 DEBUG:boto:Method: GET 2022-09-03T05:33:55.668 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.669 DEBUG:boto:Data: 2022-09-03T05:33:55.670 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.671 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.672 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.672 DEBUG:boto:Params: {} 2022-09-03T05:33:55.673 DEBUG:boto:Token: None 2022-09-03T05:33:55.674 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:55.676 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y9SbnnTqCU1+X8fh1WxChXFcDKE= 2022-09-03T05:33:55.676 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y9SbnnTqCU1+X8fh1WxChXFcDKE=', 'Content-Length': '0'} 2022-09-03T05:33:55.679 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000dae3054af15859b0-006312e743-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.681 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:55.683 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:55.683 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:55.684 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:55.685 DEBUG:boto:Method: GET 2022-09-03T05:33:55.685 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:33:55.686 DEBUG:boto:Data: 2022-09-03T05:33:55.686 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.687 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:55.688 DEBUG:boto:Port: 8001 2022-09-03T05:33:55.688 DEBUG:boto:Params: {} 2022-09-03T05:33:55.689 DEBUG:boto:Token: None 2022-09-03T05:33:55.690 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/ 2022-09-03T05:33:55.690 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E= 2022-09-03T05:33:55.691 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E=', 'Content-Length': '0'} 2022-09-03T05:33:55.694 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a8cc8e25f7e14351-006312e743-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.738 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:55.740 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:55.741 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:55.742 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:55.742 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:55.743 DEBUG:boto:Method: GET 2022-09-03T05:33:55.744 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:33:55.744 DEBUG:boto:Data: 2022-09-03T05:33:55.745 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.745 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.746 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.747 DEBUG:boto:Params: {} 2022-09-03T05:33:55.747 DEBUG:boto:Token: None 2022-09-03T05:33:55.748 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/ 2022-09-03T05:33:55.749 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E= 2022-09-03T05:33:55.750 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E=', 'Content-Length': '0'} 2022-09-03T05:33:55.753 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000105c6b6a05ae9d12-006312e743-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.754 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:55.755 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:33:55.755 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.756 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.756 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.757 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.757 DEBUG:boto:Method: GET 2022-09-03T05:33:55.758 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.759 DEBUG:boto:Data: 2022-09-03T05:33:55.759 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.760 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:55.760 DEBUG:boto:Port: 8001 2022-09-03T05:33:55.761 DEBUG:boto:Params: {} 2022-09-03T05:33:55.762 DEBUG:boto:Token: None 2022-09-03T05:33:55.762 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.763 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5WhFrcRA6YZaDOiOfMy17GMjiRE= 2022-09-03T05:33:55.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5WhFrcRA6YZaDOiOfMy17GMjiRE=', 'Content-Length': '0'} 2022-09-03T05:33:55.767 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001c49ce77f1fd755a-006312e743-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.769 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.770 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.771 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.772 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.773 DEBUG:boto:Method: GET 2022-09-03T05:33:55.774 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.775 DEBUG:boto:Data: 2022-09-03T05:33:55.776 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.777 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.778 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.779 DEBUG:boto:Params: {} 2022-09-03T05:33:55.780 DEBUG:boto:Token: None 2022-09-03T05:33:55.781 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.782 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5WhFrcRA6YZaDOiOfMy17GMjiRE= 2022-09-03T05:33:55.783 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5WhFrcRA6YZaDOiOfMy17GMjiRE=', 'Content-Length': '0'} 2022-09-03T05:33:55.786 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000056c6bc21c7b9276f-006312e743-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.788 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.790 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.791 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.791 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.792 DEBUG:boto:Method: HEAD 2022-09-03T05:33:55.792 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.793 DEBUG:boto:Data: 2022-09-03T05:33:55.793 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.794 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:55.794 DEBUG:boto:Port: 8001 2022-09-03T05:33:55.795 DEBUG:boto:Params: {} 2022-09-03T05:33:55.795 DEBUG:boto:Token: None 2022-09-03T05:33:55.796 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.797 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:55EdDhlt0nZIuh9lsfZUWSFoPdo= 2022-09-03T05:33:55.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:55EdDhlt0nZIuh9lsfZUWSFoPdo=', 'Content-Length': '0'} 2022-09-03T05:33:55.800 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a6a4a8806dd52ec4-006312e743-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.800 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.801 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.801 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.802 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.802 DEBUG:boto:Method: HEAD 2022-09-03T05:33:55.803 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.803 DEBUG:boto:Data: 2022-09-03T05:33:55.804 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.805 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.805 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.806 DEBUG:boto:Params: {} 2022-09-03T05:33:55.806 DEBUG:boto:Token: None 2022-09-03T05:33:55.807 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.808 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:55EdDhlt0nZIuh9lsfZUWSFoPdo= 2022-09-03T05:33:55.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:55EdDhlt0nZIuh9lsfZUWSFoPdo=', 'Content-Length': '0'} 2022-09-03T05:33:55.810 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f045f8754261bb0f-006312e743-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.811 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:33:55.812 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.812 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.813 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.813 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.814 DEBUG:boto:Method: GET 2022-09-03T05:33:55.815 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.815 DEBUG:boto:Data: 2022-09-03T05:33:55.816 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.816 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:55.817 DEBUG:boto:Port: 8001 2022-09-03T05:33:55.817 DEBUG:boto:Params: {} 2022-09-03T05:33:55.818 DEBUG:boto:Token: None 2022-09-03T05:33:55.819 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.819 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5WhFrcRA6YZaDOiOfMy17GMjiRE= 2022-09-03T05:33:55.820 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5WhFrcRA6YZaDOiOfMy17GMjiRE=', 'Content-Length': '0'} 2022-09-03T05:33:55.822 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005c2b6f0cde6325e5-006312e743-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.823 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.823 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.824 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.824 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.825 DEBUG:boto:Method: GET 2022-09-03T05:33:55.826 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.826 DEBUG:boto:Data: 2022-09-03T05:33:55.827 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.827 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.828 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.828 DEBUG:boto:Params: {} 2022-09-03T05:33:55.830 DEBUG:boto:Token: None 2022-09-03T05:33:55.831 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.831 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5WhFrcRA6YZaDOiOfMy17GMjiRE= 2022-09-03T05:33:55.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5WhFrcRA6YZaDOiOfMy17GMjiRE=', 'Content-Length': '0'} 2022-09-03T05:33:55.834 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000aa97bb6974c5e52b-006312e743-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.882 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:55.883 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:55.884 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:55.884 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:55.885 DEBUG:boto:Method: GET 2022-09-03T05:33:55.885 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:55.886 DEBUG:boto:Data: 2022-09-03T05:33:55.886 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.887 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:55.887 DEBUG:boto:Port: 8001 2022-09-03T05:33:55.888 DEBUG:boto:Params: {} 2022-09-03T05:33:55.888 DEBUG:boto:Token: None 2022-09-03T05:33:55.889 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/ 2022-09-03T05:33:55.890 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E= 2022-09-03T05:33:55.891 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E=', 'Content-Length': '0'} 2022-09-03T05:33:55.895 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000063cdf8933fbec21b-006312e743-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.896 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:55.897 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:55.898 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:55.899 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:55.900 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:55.901 DEBUG:boto:Method: GET 2022-09-03T05:33:55.902 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:55.903 DEBUG:boto:Data: 2022-09-03T05:33:55.904 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.906 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.907 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.908 DEBUG:boto:Params: {} 2022-09-03T05:33:55.909 DEBUG:boto:Token: None 2022-09-03T05:33:55.910 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/ 2022-09-03T05:33:55.911 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E= 2022-09-03T05:33:55.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uqc12YISS9dIaRKJhBmMVgIMt4E=', 'Content-Length': '0'} 2022-09-03T05:33:55.915 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000abb635a0601f5f37-006312e743-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.916 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:55.917 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.918 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.919 DEBUG:boto:Method: HEAD 2022-09-03T05:33:55.920 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.920 DEBUG:boto:Data: 2022-09-03T05:33:55.921 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.921 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:55.922 DEBUG:boto:Port: 8001 2022-09-03T05:33:55.923 DEBUG:boto:Params: {} 2022-09-03T05:33:55.924 DEBUG:boto:Token: None 2022-09-03T05:33:55.924 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.925 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NY9rVdoMW3LqUFZrRGaBLr3URag= 2022-09-03T05:33:55.925 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NY9rVdoMW3LqUFZrRGaBLr3URag=', 'Content-Length': '0'} 2022-09-03T05:33:55.928 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009d406d33ec8399df-006312e743-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.929 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.930 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.931 DEBUG:boto:Method: HEAD 2022-09-03T05:33:55.931 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.932 DEBUG:boto:Data: 2022-09-03T05:33:55.933 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.934 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.935 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.936 DEBUG:boto:Params: {} 2022-09-03T05:33:55.937 DEBUG:boto:Token: None 2022-09-03T05:33:55.938 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.939 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NY9rVdoMW3LqUFZrRGaBLr3URag= 2022-09-03T05:33:55.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NY9rVdoMW3LqUFZrRGaBLr3URag=', 'Content-Length': '0'} 2022-09-03T05:33:55.944 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000088dba8cff6e5e2e-006312e743-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.945 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:33:55.946 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.947 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.947 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.948 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.949 DEBUG:boto:Method: GET 2022-09-03T05:33:55.951 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.951 DEBUG:boto:Data: 2022-09-03T05:33:55.952 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.953 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:55.954 DEBUG:boto:Port: 8001 2022-09-03T05:33:55.955 DEBUG:boto:Params: {} 2022-09-03T05:33:55.956 DEBUG:boto:Token: None 2022-09-03T05:33:55.957 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.958 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5WhFrcRA6YZaDOiOfMy17GMjiRE= 2022-09-03T05:33:55.959 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5WhFrcRA6YZaDOiOfMy17GMjiRE=', 'Content-Length': '0'} 2022-09-03T05:33:55.961 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a33a22b1ee91906c-006312e743-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:55.963 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.963 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:55.964 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.965 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.966 DEBUG:boto:Method: GET 2022-09-03T05:33:55.968 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.969 DEBUG:boto:Data: 2022-09-03T05:33:55.970 DEBUG:boto:Headers: {} 2022-09-03T05:33:55.971 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:55.972 DEBUG:boto:Port: 8000 2022-09-03T05:33:55.973 DEBUG:boto:Params: {} 2022-09-03T05:33:55.974 DEBUG:boto:Token: None 2022-09-03T05:33:55.975 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:55 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:55.976 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5WhFrcRA6YZaDOiOfMy17GMjiRE= 2022-09-03T05:33:55.977 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5WhFrcRA6YZaDOiOfMy17GMjiRE=', 'Content-Length': '0'} 2022-09-03T05:33:55.979 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001dc7f793ef7eacfa-006312e743-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.022 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:56.024 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:56.025 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:56.026 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:56.027 DEBUG:boto:Method: GET 2022-09-03T05:33:56.028 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:56.029 DEBUG:boto:Data: 2022-09-03T05:33:56.030 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.032 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.033 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.034 DEBUG:boto:Params: {} 2022-09-03T05:33:56.035 DEBUG:boto:Token: None 2022-09-03T05:33:56.036 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/ 2022-09-03T05:33:56.036 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g= 2022-09-03T05:33:56.038 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g=', 'Content-Length': '0'} 2022-09-03T05:33:56.041 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000820f4828ea59e627-006312e744-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.043 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:56.043 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:56.044 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:56.045 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:56.045 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:56.046 DEBUG:boto:Method: GET 2022-09-03T05:33:56.046 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:56.047 DEBUG:boto:Data: 2022-09-03T05:33:56.048 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.048 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:56.049 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.050 DEBUG:boto:Params: {} 2022-09-03T05:33:56.050 DEBUG:boto:Token: None 2022-09-03T05:33:56.051 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/ 2022-09-03T05:33:56.052 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g= 2022-09-03T05:33:56.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g=', 'Content-Length': '0'} 2022-09-03T05:33:56.055 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dd14c38808129092-006312e744-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.055 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:56.056 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:33:56.057 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.059 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.059 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.060 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.060 DEBUG:boto:Method: GET 2022-09-03T05:33:56.061 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.061 DEBUG:boto:Data: 2022-09-03T05:33:56.062 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.062 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.063 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.063 DEBUG:boto:Params: {} 2022-09-03T05:33:56.064 DEBUG:boto:Token: None 2022-09-03T05:33:56.065 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.065 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cWf7SgrbGMo5zItbNZ1SCJSujjY= 2022-09-03T05:33:56.066 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cWf7SgrbGMo5zItbNZ1SCJSujjY=', 'Content-Length': '0'} 2022-09-03T05:33:56.068 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f7f9ffa81c1aaf0e-006312e744-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.069 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.070 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.071 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.071 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.072 DEBUG:boto:Method: GET 2022-09-03T05:33:56.072 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.073 DEBUG:boto:Data: 2022-09-03T05:33:56.073 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.074 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:56.075 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.075 DEBUG:boto:Params: {} 2022-09-03T05:33:56.076 DEBUG:boto:Token: None 2022-09-03T05:33:56.076 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.077 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cWf7SgrbGMo5zItbNZ1SCJSujjY= 2022-09-03T05:33:56.078 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cWf7SgrbGMo5zItbNZ1SCJSujjY=', 'Content-Length': '0'} 2022-09-03T05:33:56.080 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004fe26b8074772d2e-006312e744-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.081 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.082 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.082 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.083 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.084 DEBUG:boto:Method: HEAD 2022-09-03T05:33:56.084 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.085 DEBUG:boto:Data: 2022-09-03T05:33:56.085 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.086 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.087 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.087 DEBUG:boto:Params: {} 2022-09-03T05:33:56.088 DEBUG:boto:Token: None 2022-09-03T05:33:56.088 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.089 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MUCqDbQfyX64RDHpWgxe5a1JoDE= 2022-09-03T05:33:56.090 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MUCqDbQfyX64RDHpWgxe5a1JoDE=', 'Content-Length': '0'} 2022-09-03T05:33:56.093 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b5a5b82a4f27c34c-006312e744-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.093 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.094 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.095 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.095 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.096 DEBUG:boto:Method: HEAD 2022-09-03T05:33:56.096 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.097 DEBUG:boto:Data: 2022-09-03T05:33:56.097 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.098 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:56.098 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.099 DEBUG:boto:Params: {} 2022-09-03T05:33:56.100 DEBUG:boto:Token: None 2022-09-03T05:33:56.100 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.101 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MUCqDbQfyX64RDHpWgxe5a1JoDE= 2022-09-03T05:33:56.102 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MUCqDbQfyX64RDHpWgxe5a1JoDE=', 'Content-Length': '0'} 2022-09-03T05:33:56.104 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c4a7d75a3bbaa838-006312e744-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.105 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:33:56.105 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.106 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.107 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.107 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.108 DEBUG:boto:Method: GET 2022-09-03T05:33:56.108 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.109 DEBUG:boto:Data: 2022-09-03T05:33:56.109 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.110 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.111 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.111 DEBUG:boto:Params: {} 2022-09-03T05:33:56.112 DEBUG:boto:Token: None 2022-09-03T05:33:56.112 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.113 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cWf7SgrbGMo5zItbNZ1SCJSujjY= 2022-09-03T05:33:56.114 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cWf7SgrbGMo5zItbNZ1SCJSujjY=', 'Content-Length': '0'} 2022-09-03T05:33:56.115 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000778e9b0a8890f369-006312e744-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.116 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.117 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.117 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.118 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.118 DEBUG:boto:Method: GET 2022-09-03T05:33:56.119 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.119 DEBUG:boto:Data: 2022-09-03T05:33:56.120 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.121 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:56.121 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.122 DEBUG:boto:Params: {} 2022-09-03T05:33:56.122 DEBUG:boto:Token: None 2022-09-03T05:33:56.123 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.124 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cWf7SgrbGMo5zItbNZ1SCJSujjY= 2022-09-03T05:33:56.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cWf7SgrbGMo5zItbNZ1SCJSujjY=', 'Content-Length': '0'} 2022-09-03T05:33:56.126 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001799babd7cff4d22-006312e744-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.127 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:56.128 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:56.128 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:56.129 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:56.129 DEBUG:boto:Method: GET 2022-09-03T05:33:56.130 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:56.130 DEBUG:boto:Data: 2022-09-03T05:33:56.131 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.132 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.132 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.133 DEBUG:boto:Params: {} 2022-09-03T05:33:56.133 DEBUG:boto:Token: None 2022-09-03T05:33:56.134 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/ 2022-09-03T05:33:56.134 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g= 2022-09-03T05:33:56.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g=', 'Content-Length': '0'} 2022-09-03T05:33:56.138 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000087a95e98625fb847-006312e744-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.138 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:56.139 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:56.140 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:56.140 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:56.141 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:56.141 DEBUG:boto:Method: GET 2022-09-03T05:33:56.142 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:56.143 DEBUG:boto:Data: 2022-09-03T05:33:56.143 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.144 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:56.144 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.145 DEBUG:boto:Params: {} 2022-09-03T05:33:56.145 DEBUG:boto:Token: None 2022-09-03T05:33:56.146 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/ 2022-09-03T05:33:56.146 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g= 2022-09-03T05:33:56.147 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g=', 'Content-Length': '0'} 2022-09-03T05:33:56.149 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002b4eb285ca1a9ace-006312e744-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.190 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:56.191 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.192 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.193 DEBUG:boto:Method: HEAD 2022-09-03T05:33:56.195 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.196 DEBUG:boto:Data: 2022-09-03T05:33:56.196 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.198 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.200 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.201 DEBUG:boto:Params: {} 2022-09-03T05:33:56.201 DEBUG:boto:Token: None 2022-09-03T05:33:56.202 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.202 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eoX2WWCzwHSjPlak3yj5NJSJUzE= 2022-09-03T05:33:56.203 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eoX2WWCzwHSjPlak3yj5NJSJUzE=', 'Content-Length': '0'} 2022-09-03T05:33:56.206 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b2dae494b64116b8-006312e744-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.206 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.207 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.208 DEBUG:boto:Method: HEAD 2022-09-03T05:33:56.208 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.208 DEBUG:boto:Data: 2022-09-03T05:33:56.209 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.209 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:56.210 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.210 DEBUG:boto:Params: {} 2022-09-03T05:33:56.211 DEBUG:boto:Token: None 2022-09-03T05:33:56.212 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.212 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eoX2WWCzwHSjPlak3yj5NJSJUzE= 2022-09-03T05:33:56.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eoX2WWCzwHSjPlak3yj5NJSJUzE=', 'Content-Length': '0'} 2022-09-03T05:33:56.215 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000082543f1848a5d629-006312e744-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.216 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:33:56.217 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.217 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.218 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.219 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.219 DEBUG:boto:Method: GET 2022-09-03T05:33:56.220 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.220 DEBUG:boto:Data: 2022-09-03T05:33:56.221 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.221 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.222 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.222 DEBUG:boto:Params: {} 2022-09-03T05:33:56.223 DEBUG:boto:Token: None 2022-09-03T05:33:56.224 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.224 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cWf7SgrbGMo5zItbNZ1SCJSujjY= 2022-09-03T05:33:56.225 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cWf7SgrbGMo5zItbNZ1SCJSujjY=', 'Content-Length': '0'} 2022-09-03T05:33:56.227 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000075861325948f7a2-006312e744-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.228 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.228 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:56.229 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.230 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.230 DEBUG:boto:Method: GET 2022-09-03T05:33:56.231 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.231 DEBUG:boto:Data: 2022-09-03T05:33:56.232 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.232 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:56.233 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.234 DEBUG:boto:Params: {} 2022-09-03T05:33:56.234 DEBUG:boto:Token: None 2022-09-03T05:33:56.235 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:56.235 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cWf7SgrbGMo5zItbNZ1SCJSujjY= 2022-09-03T05:33:56.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cWf7SgrbGMo5zItbNZ1SCJSujjY=', 'Content-Length': '0'} 2022-09-03T05:33:56.238 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000efbd8bb54b4c0b03-006312e744-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.240 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:56.240 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:56.241 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:56.241 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:56.242 DEBUG:boto:Method: GET 2022-09-03T05:33:56.243 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:56.243 DEBUG:boto:Data: 2022-09-03T05:33:56.244 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.244 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.245 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.245 DEBUG:boto:Params: {} 2022-09-03T05:33:56.246 DEBUG:boto:Token: None 2022-09-03T05:33:56.247 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/ 2022-09-03T05:33:56.247 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g= 2022-09-03T05:33:56.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g=', 'Content-Length': '0'} 2022-09-03T05:33:56.259 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000294af650cbb7126b-006312e744-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.260 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:56.261 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:56.262 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:56.262 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:56.263 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:56.263 DEBUG:boto:Method: GET 2022-09-03T05:33:56.264 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:56.264 DEBUG:boto:Data: 2022-09-03T05:33:56.265 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.266 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:56.266 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.267 DEBUG:boto:Params: {} 2022-09-03T05:33:56.268 DEBUG:boto:Token: None 2022-09-03T05:33:56.268 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/ 2022-09-03T05:33:56.269 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g= 2022-09-03T05:33:56.270 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g=', 'Content-Length': '0'} 2022-09-03T05:33:56.273 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bab9a86afadc3ca6-006312e744-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.322 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:56.324 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:33:56.326 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.328 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.328 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.329 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.330 DEBUG:boto:Method: GET 2022-09-03T05:33:56.330 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.331 DEBUG:boto:Data: 2022-09-03T05:33:56.332 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.334 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.334 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.335 DEBUG:boto:Params: {} 2022-09-03T05:33:56.337 DEBUG:boto:Token: None 2022-09-03T05:33:56.338 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.339 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lb5BiAw0WEqoStQnFzKb4prozLE= 2022-09-03T05:33:56.341 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lb5BiAw0WEqoStQnFzKb4prozLE=', 'Content-Length': '0'} 2022-09-03T05:33:56.344 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d61d5ea1ebedacf1-006312e744-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.345 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.346 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.347 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.347 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.348 DEBUG:boto:Method: GET 2022-09-03T05:33:56.349 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.350 DEBUG:boto:Data: 2022-09-03T05:33:56.350 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.351 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:56.351 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.352 DEBUG:boto:Params: {} 2022-09-03T05:33:56.353 DEBUG:boto:Token: None 2022-09-03T05:33:56.353 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.354 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lb5BiAw0WEqoStQnFzKb4prozLE= 2022-09-03T05:33:56.355 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lb5BiAw0WEqoStQnFzKb4prozLE=', 'Content-Length': '0'} 2022-09-03T05:33:56.357 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000083a7d27def3b635c-006312e744-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.358 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.359 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.359 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.360 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.360 DEBUG:boto:Method: HEAD 2022-09-03T05:33:56.361 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.362 DEBUG:boto:Data: 2022-09-03T05:33:56.363 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.363 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.365 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.366 DEBUG:boto:Params: {} 2022-09-03T05:33:56.367 DEBUG:boto:Token: None 2022-09-03T05:33:56.368 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.369 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0ZfatNFnPoQq17u2uVgXeI+vQlY= 2022-09-03T05:33:56.370 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0ZfatNFnPoQq17u2uVgXeI+vQlY=', 'Content-Length': '0'} 2022-09-03T05:33:56.373 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000067b00b4a0eabd3fd-006312e744-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.375 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.376 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.377 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.378 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.379 DEBUG:boto:Method: HEAD 2022-09-03T05:33:56.380 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.380 DEBUG:boto:Data: 2022-09-03T05:33:56.381 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.382 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:56.383 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.384 DEBUG:boto:Params: {} 2022-09-03T05:33:56.385 DEBUG:boto:Token: None 2022-09-03T05:33:56.386 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.387 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0ZfatNFnPoQq17u2uVgXeI+vQlY= 2022-09-03T05:33:56.389 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0ZfatNFnPoQq17u2uVgXeI+vQlY=', 'Content-Length': '0'} 2022-09-03T05:33:56.391 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c0b9515b8a36ad1f-006312e744-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.392 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:33:56.393 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.394 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.394 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.395 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.395 DEBUG:boto:Method: GET 2022-09-03T05:33:56.396 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.396 DEBUG:boto:Data: 2022-09-03T05:33:56.397 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.398 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.399 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.400 DEBUG:boto:Params: {} 2022-09-03T05:33:56.401 DEBUG:boto:Token: None 2022-09-03T05:33:56.402 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.403 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lb5BiAw0WEqoStQnFzKb4prozLE= 2022-09-03T05:33:56.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lb5BiAw0WEqoStQnFzKb4prozLE=', 'Content-Length': '0'} 2022-09-03T05:33:56.407 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d7fbedbe597ea299-006312e744-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.409 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.410 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.410 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.411 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.411 DEBUG:boto:Method: GET 2022-09-03T05:33:56.412 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.412 DEBUG:boto:Data: 2022-09-03T05:33:56.413 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.414 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:56.415 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.416 DEBUG:boto:Params: {} 2022-09-03T05:33:56.417 DEBUG:boto:Token: None 2022-09-03T05:33:56.419 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.420 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lb5BiAw0WEqoStQnFzKb4prozLE= 2022-09-03T05:33:56.421 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lb5BiAw0WEqoStQnFzKb4prozLE=', 'Content-Length': '0'} 2022-09-03T05:33:56.424 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ca92b545212e1538-006312e744-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.425 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:56.426 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:56.426 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:56.427 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:56.427 DEBUG:boto:Method: GET 2022-09-03T05:33:56.428 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:56.428 DEBUG:boto:Data: 2022-09-03T05:33:56.429 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.430 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.431 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.432 DEBUG:boto:Params: {} 2022-09-03T05:33:56.433 DEBUG:boto:Token: None 2022-09-03T05:33:56.434 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/ 2022-09-03T05:33:56.435 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g= 2022-09-03T05:33:56.435 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g=', 'Content-Length': '0'} 2022-09-03T05:33:56.448 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003015932570896df5-006312e744-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.454 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:56.456 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:56.457 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:56.458 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:56.459 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:56.460 DEBUG:boto:Method: GET 2022-09-03T05:33:56.461 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:56.462 DEBUG:boto:Data: 2022-09-03T05:33:56.463 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.464 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:56.465 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.466 DEBUG:boto:Params: {} 2022-09-03T05:33:56.467 DEBUG:boto:Token: None 2022-09-03T05:33:56.468 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/ 2022-09-03T05:33:56.470 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g= 2022-09-03T05:33:56.471 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g=', 'Content-Length': '0'} 2022-09-03T05:33:56.474 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000004ed8a0d7a4f171b-006312e744-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.475 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:56.476 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.476 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.477 DEBUG:boto:Method: HEAD 2022-09-03T05:33:56.477 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.478 DEBUG:boto:Data: 2022-09-03T05:33:56.479 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.479 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.480 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.481 DEBUG:boto:Params: {} 2022-09-03T05:33:56.481 DEBUG:boto:Token: None 2022-09-03T05:33:56.482 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6Wez1bhVSZhCGSRZnSYEozLPXgc= 2022-09-03T05:33:56.483 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6Wez1bhVSZhCGSRZnSYEozLPXgc=', 'Content-Length': '0'} 2022-09-03T05:33:56.487 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009cfe6771107b67e3-006312e744-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.488 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.489 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.490 DEBUG:boto:Method: HEAD 2022-09-03T05:33:56.491 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.492 DEBUG:boto:Data: 2022-09-03T05:33:56.493 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.494 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:56.495 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.496 DEBUG:boto:Params: {} 2022-09-03T05:33:56.497 DEBUG:boto:Token: None 2022-09-03T05:33:56.498 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.499 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6Wez1bhVSZhCGSRZnSYEozLPXgc= 2022-09-03T05:33:56.500 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6Wez1bhVSZhCGSRZnSYEozLPXgc=', 'Content-Length': '0'} 2022-09-03T05:33:56.503 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f128371ec9d89576-006312e744-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.505 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:33:56.506 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.507 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.508 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.509 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.510 DEBUG:boto:Method: GET 2022-09-03T05:33:56.511 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.511 DEBUG:boto:Data: 2022-09-03T05:33:56.512 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.513 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.513 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.514 DEBUG:boto:Params: {} 2022-09-03T05:33:56.514 DEBUG:boto:Token: None 2022-09-03T05:33:56.515 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.516 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lb5BiAw0WEqoStQnFzKb4prozLE= 2022-09-03T05:33:56.517 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lb5BiAw0WEqoStQnFzKb4prozLE=', 'Content-Length': '0'} 2022-09-03T05:33:56.519 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008ccd23922c825a2b-006312e744-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.566 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.567 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:56.567 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.568 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.568 DEBUG:boto:Method: GET 2022-09-03T05:33:56.569 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.570 DEBUG:boto:Data: 2022-09-03T05:33:56.571 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.571 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:56.572 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.573 DEBUG:boto:Params: {} 2022-09-03T05:33:56.574 DEBUG:boto:Token: None 2022-09-03T05:33:56.575 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:56.576 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lb5BiAw0WEqoStQnFzKb4prozLE= 2022-09-03T05:33:56.577 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lb5BiAw0WEqoStQnFzKb4prozLE=', 'Content-Length': '0'} 2022-09-03T05:33:56.580 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000edd71c30d39eff43-006312e744-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.581 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:56.582 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:56.583 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:56.584 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:56.585 DEBUG:boto:Method: GET 2022-09-03T05:33:56.586 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:56.587 DEBUG:boto:Data: 2022-09-03T05:33:56.588 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.589 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.590 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.590 DEBUG:boto:Params: {} 2022-09-03T05:33:56.591 DEBUG:boto:Token: None 2022-09-03T05:33:56.592 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/ 2022-09-03T05:33:56.592 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g= 2022-09-03T05:33:56.593 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g=', 'Content-Length': '0'} 2022-09-03T05:33:56.597 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000684910a008a26176-006312e744-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.598 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:56.599 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:56.599 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:56.600 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:56.600 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:56.601 DEBUG:boto:Method: GET 2022-09-03T05:33:56.601 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:56.602 DEBUG:boto:Data: 2022-09-03T05:33:56.603 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.603 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:56.604 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.605 DEBUG:boto:Params: {} 2022-09-03T05:33:56.605 DEBUG:boto:Token: None 2022-09-03T05:33:56.606 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/ 2022-09-03T05:33:56.606 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g= 2022-09-03T05:33:56.607 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g=', 'Content-Length': '0'} 2022-09-03T05:33:56.610 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e8916fca26426f14-006312e744-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.654 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:56.655 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-17 zones={test-zone3, test-zone1} 2022-09-03T05:33:56.656 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-17 zones={test-zone3, test-zone2} 2022-09-03T05:33:56.656 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:56.657 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:56.657 DEBUG:boto:Method: HEAD 2022-09-03T05:33:56.658 DEBUG:boto:Path: /dsgvcy-17/ 2022-09-03T05:33:56.659 DEBUG:boto:Data: 2022-09-03T05:33:56.659 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.660 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.661 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.661 DEBUG:boto:Params: {} 2022-09-03T05:33:56.662 DEBUG:boto:Token: None 2022-09-03T05:33:56.663 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/ 2022-09-03T05:33:56.663 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/EMVhhVP3kx0FzOo2r3iPZvnot4= 2022-09-03T05:33:56.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/EMVhhVP3kx0FzOo2r3iPZvnot4=', 'Content-Length': '0'} 2022-09-03T05:33:56.666 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c8af6bbec6dafee7-006312e744-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.667 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:56.668 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:56.668 DEBUG:boto:Method: HEAD 2022-09-03T05:33:56.669 DEBUG:boto:Path: /dsgvcy-17/ 2022-09-03T05:33:56.669 DEBUG:boto:Data: 2022-09-03T05:33:56.670 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.670 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:56.671 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.672 DEBUG:boto:Params: {} 2022-09-03T05:33:56.672 DEBUG:boto:Token: None 2022-09-03T05:33:56.673 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/ 2022-09-03T05:33:56.674 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/EMVhhVP3kx0FzOo2r3iPZvnot4= 2022-09-03T05:33:56.675 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/EMVhhVP3kx0FzOo2r3iPZvnot4=', 'Content-Length': '0'} 2022-09-03T05:33:56.683 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e056d78fc96c763a-006312e744-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.684 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:33:56.684 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:56.685 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:56.685 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:56.686 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:56.686 DEBUG:boto:Method: GET 2022-09-03T05:33:56.687 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:56.687 DEBUG:boto:Data: 2022-09-03T05:33:56.688 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.689 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.689 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.690 DEBUG:boto:Params: {} 2022-09-03T05:33:56.690 DEBUG:boto:Token: None 2022-09-03T05:33:56.691 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/?versions 2022-09-03T05:33:56.692 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n7yBmn2sHv3xNpUqecrWVpaT0i4= 2022-09-03T05:33:56.693 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n7yBmn2sHv3xNpUqecrWVpaT0i4=', 'Content-Length': '0'} 2022-09-03T05:33:56.696 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002dd151335b0f85ae-006312e744-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.697 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:56.699 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:33:56.699 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:33:56.700 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:33:56.701 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:33:56.702 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:33:56.702 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:56.703 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:56.705 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:56.706 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:56.706 DEBUG:boto:Method: GET 2022-09-03T05:33:56.707 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:56.708 DEBUG:boto:Data: 2022-09-03T05:33:56.708 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.709 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:56.709 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.710 DEBUG:boto:Params: {} 2022-09-03T05:33:56.710 DEBUG:boto:Token: None 2022-09-03T05:33:56.711 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/?versions 2022-09-03T05:33:56.712 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n7yBmn2sHv3xNpUqecrWVpaT0i4= 2022-09-03T05:33:56.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n7yBmn2sHv3xNpUqecrWVpaT0i4=', 'Content-Length': '0'} 2022-09-03T05:33:56.715 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002726f0663e465af0-006312e744-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.716 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:56.717 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:33:56.717 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:33:56.718 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:33:56.720 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:33:56.721 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:56.722 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:56.722 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:56.723 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:56.723 DEBUG:boto:Method: GET 2022-09-03T05:33:56.724 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:56.725 DEBUG:boto:Data: 2022-09-03T05:33:56.725 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.726 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.726 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.727 DEBUG:boto:Params: {} 2022-09-03T05:33:56.728 DEBUG:boto:Token: None 2022-09-03T05:33:56.729 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/?versions 2022-09-03T05:33:56.730 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n7yBmn2sHv3xNpUqecrWVpaT0i4= 2022-09-03T05:33:56.731 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n7yBmn2sHv3xNpUqecrWVpaT0i4=', 'Content-Length': '0'} 2022-09-03T05:33:56.734 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002ad8daff5ac0bb9a-006312e744-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.735 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:56.737 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:56.738 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:56.739 DEBUG:boto:path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:56.739 DEBUG:boto:auth_path=/dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:56.740 DEBUG:boto:Method: GET 2022-09-03T05:33:56.741 DEBUG:boto:Path: /dsgvcy-17/?versions&max-keys=999 2022-09-03T05:33:56.742 DEBUG:boto:Data: 2022-09-03T05:33:56.743 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.744 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:56.745 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.746 DEBUG:boto:Params: {} 2022-09-03T05:33:56.747 DEBUG:boto:Token: None 2022-09-03T05:33:56.748 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/?versions 2022-09-03T05:33:56.749 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n7yBmn2sHv3xNpUqecrWVpaT0i4= 2022-09-03T05:33:56.750 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n7yBmn2sHv3xNpUqecrWVpaT0i4=', 'Content-Length': '0'} 2022-09-03T05:33:56.766 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a3ccae27964b431f-006312e744-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.768 DEBUG:boto:b'dsgvcy-17999falsedummy460bjTYBQ4BMNIYlxvH9mOOJmXfY4qxtrue2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone1yx2lTqvongZDmkvI6tLywhzAB9azvA0true2022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone2qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1Dtrue2022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone34hufCTEaqzXM3cUjSTjtsbNbxySDq.vtrue2022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:56.770 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:33:56.771 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:56.772 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:56.772 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.773 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.774 DEBUG:boto:Method: GET 2022-09-03T05:33:56.775 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.776 DEBUG:boto:Data: 2022-09-03T05:33:56.777 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.778 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.779 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.780 DEBUG:boto:Params: {} 2022-09-03T05:33:56.781 DEBUG:boto:Token: None 2022-09-03T05:33:56.782 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.783 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zfr8T5d3/s/3hsKOTRXlRedLkrw= 2022-09-03T05:33:56.783 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zfr8T5d3/s/3hsKOTRXlRedLkrw=', 'Content-Length': '0'} 2022-09-03T05:33:56.787 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000debc99bde5341fe1-006312e744-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.788 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:56.789 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:56.789 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.790 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.790 DEBUG:boto:Method: GET 2022-09-03T05:33:56.791 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.791 DEBUG:boto:Data: 2022-09-03T05:33:56.792 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.793 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:56.793 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.794 DEBUG:boto:Params: {} 2022-09-03T05:33:56.794 DEBUG:boto:Token: None 2022-09-03T05:33:56.795 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.796 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zfr8T5d3/s/3hsKOTRXlRedLkrw= 2022-09-03T05:33:56.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zfr8T5d3/s/3hsKOTRXlRedLkrw=', 'Content-Length': '0'} 2022-09-03T05:33:56.799 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000024b7341d5ca09056-006312e744-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.801 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:56.802 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:56.803 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.804 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.804 DEBUG:boto:Method: HEAD 2022-09-03T05:33:56.805 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.806 DEBUG:boto:Data: 2022-09-03T05:33:56.807 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.808 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.809 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.811 DEBUG:boto:Params: {} 2022-09-03T05:33:56.812 DEBUG:boto:Token: None 2022-09-03T05:33:56.813 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.814 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bNXELBSdCRF7woKZFsMTUNeabL8= 2022-09-03T05:33:56.816 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bNXELBSdCRF7woKZFsMTUNeabL8=', 'Content-Length': '0'} 2022-09-03T05:33:56.819 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000034efebce13054d24-006312e744-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.821 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:56.822 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:56.824 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.825 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.826 DEBUG:boto:Method: HEAD 2022-09-03T05:33:56.828 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.828 DEBUG:boto:Data: 2022-09-03T05:33:56.829 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.829 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:56.830 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.831 DEBUG:boto:Params: {} 2022-09-03T05:33:56.831 DEBUG:boto:Token: None 2022-09-03T05:33:56.832 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.833 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bNXELBSdCRF7woKZFsMTUNeabL8= 2022-09-03T05:33:56.834 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bNXELBSdCRF7woKZFsMTUNeabL8=', 'Content-Length': '0'} 2022-09-03T05:33:56.837 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000050382826ecc8e927-006312e744-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.839 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:33:56.840 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:56.841 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:56.843 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.843 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.844 DEBUG:boto:Method: GET 2022-09-03T05:33:56.845 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.846 DEBUG:boto:Data: 2022-09-03T05:33:56.846 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.847 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.848 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.849 DEBUG:boto:Params: {} 2022-09-03T05:33:56.849 DEBUG:boto:Token: None 2022-09-03T05:33:56.850 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.851 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zfr8T5d3/s/3hsKOTRXlRedLkrw= 2022-09-03T05:33:56.851 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zfr8T5d3/s/3hsKOTRXlRedLkrw=', 'Content-Length': '0'} 2022-09-03T05:33:56.854 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000005273513dd7e99117-006312e744-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.854 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:56.855 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:56.856 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.856 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.857 DEBUG:boto:Method: GET 2022-09-03T05:33:56.857 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.858 DEBUG:boto:Data: 2022-09-03T05:33:56.858 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.859 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:56.859 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.860 DEBUG:boto:Params: {} 2022-09-03T05:33:56.861 DEBUG:boto:Token: None 2022-09-03T05:33:56.861 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.862 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zfr8T5d3/s/3hsKOTRXlRedLkrw= 2022-09-03T05:33:56.863 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zfr8T5d3/s/3hsKOTRXlRedLkrw=', 'Content-Length': '0'} 2022-09-03T05:33:56.865 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000d9d45a62a1a6398b-006312e744-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.867 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:56.868 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:56.869 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:56.869 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:56.870 DEBUG:boto:Method: GET 2022-09-03T05:33:56.871 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:33:56.872 DEBUG:boto:Data: 2022-09-03T05:33:56.873 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.874 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.875 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.876 DEBUG:boto:Params: {} 2022-09-03T05:33:56.877 DEBUG:boto:Token: None 2022-09-03T05:33:56.878 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/ 2022-09-03T05:33:56.880 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g= 2022-09-03T05:33:56.881 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g=', 'Content-Length': '0'} 2022-09-03T05:33:56.884 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e01ce2e59078bedb-006312e744-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.885 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:56.886 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:56.887 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:56.887 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:56.888 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:56.889 DEBUG:boto:Method: GET 2022-09-03T05:33:56.889 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:33:56.890 DEBUG:boto:Data: 2022-09-03T05:33:56.891 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.892 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:56.893 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.894 DEBUG:boto:Params: {} 2022-09-03T05:33:56.895 DEBUG:boto:Token: None 2022-09-03T05:33:56.896 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/ 2022-09-03T05:33:56.897 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g= 2022-09-03T05:33:56.897 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7W1gHvIc50aPOqkCSLu6KM9fT5g=', 'Content-Length': '0'} 2022-09-03T05:33:56.900 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000168b31bc3eb02ec3-006312e744-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.942 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:56.943 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:56.944 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:56.945 DEBUG:boto:Method: HEAD 2022-09-03T05:33:56.945 DEBUG:boto:Path: /dsgvcy-17/dummy 2022-09-03T05:33:56.946 DEBUG:boto:Data: 2022-09-03T05:33:56.947 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.947 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.948 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.948 DEBUG:boto:Params: {} 2022-09-03T05:33:56.949 DEBUG:boto:Token: None 2022-09-03T05:33:56.950 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/dummy 2022-09-03T05:33:56.950 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uM7LM8Ri+ZWTqqhKwkR+jOenRrY= 2022-09-03T05:33:56.951 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uM7LM8Ri+ZWTqqhKwkR+jOenRrY=', 'Content-Length': '0'} 2022-09-03T05:33:56.954 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000038095e50fdf97330-006312e744-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.957 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:56.957 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:56.958 DEBUG:boto:Method: HEAD 2022-09-03T05:33:56.959 DEBUG:boto:Path: /dsgvcy-17/dummy 2022-09-03T05:33:56.960 DEBUG:boto:Data: 2022-09-03T05:33:56.961 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.962 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:56.963 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.964 DEBUG:boto:Params: {} 2022-09-03T05:33:56.965 DEBUG:boto:Token: None 2022-09-03T05:33:56.966 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/dummy 2022-09-03T05:33:56.967 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uM7LM8Ri+ZWTqqhKwkR+jOenRrY= 2022-09-03T05:33:56.967 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uM7LM8Ri+ZWTqqhKwkR+jOenRrY=', 'Content-Length': '0'} 2022-09-03T05:33:56.970 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000c515054039c69f24-006312e744-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.971 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:33:56.971 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:56.972 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:56.972 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.973 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.974 DEBUG:boto:Method: GET 2022-09-03T05:33:56.974 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.975 DEBUG:boto:Data: 2022-09-03T05:33:56.975 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.976 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:56.977 DEBUG:boto:Port: 8001 2022-09-03T05:33:56.978 DEBUG:boto:Params: {} 2022-09-03T05:33:56.979 DEBUG:boto:Token: None 2022-09-03T05:33:56.980 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:56 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.982 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zfr8T5d3/s/3hsKOTRXlRedLkrw= 2022-09-03T05:33:56.983 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zfr8T5d3/s/3hsKOTRXlRedLkrw=', 'Content-Length': '0'} 2022-09-03T05:33:56.986 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000004cfa78856c147ca5-006312e744-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:56.988 DEBUG:boto:path=/dsgvcy-17/dummy 2022-09-03T05:33:56.989 DEBUG:boto:auth_path=/dsgvcy-17/dummy 2022-09-03T05:33:56.990 DEBUG:boto:path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.991 DEBUG:boto:auth_path=/dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.992 DEBUG:boto:Method: GET 2022-09-03T05:33:56.993 DEBUG:boto:Path: /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:56.994 DEBUG:boto:Data: 2022-09-03T05:33:56.995 DEBUG:boto:Headers: {} 2022-09-03T05:33:56.996 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:56.997 DEBUG:boto:Port: 8000 2022-09-03T05:33:56.998 DEBUG:boto:Params: {} 2022-09-03T05:33:56.999 DEBUG:boto:Token: None 2022-09-03T05:33:57.001 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/dummy?versionId=460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx 2022-09-03T05:33:57.002 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s6zTo7ZAFV8Lvfxq6LHWB1/wwDs= 2022-09-03T05:33:57.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s6zTo7ZAFV8Lvfxq6LHWB1/wwDs=', 'Content-Length': '0'} 2022-09-03T05:33:57.006 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:27:58 GMT'), ('x-amz-version-id', '460bjTYBQ4BMNIYlxvH9mOOJmXfY4qx'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000eef1ca834c899bbe-006312e745-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.007 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:57.008 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:57.009 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:57.009 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:57.010 DEBUG:boto:Method: GET 2022-09-03T05:33:57.011 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:33:57.012 DEBUG:boto:Data: 2022-09-03T05:33:57.013 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.014 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:57.014 DEBUG:boto:Port: 8001 2022-09-03T05:33:57.015 DEBUG:boto:Params: {} 2022-09-03T05:33:57.016 DEBUG:boto:Token: None 2022-09-03T05:33:57.017 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/ 2022-09-03T05:33:57.018 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg= 2022-09-03T05:33:57.019 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg=', 'Content-Length': '0'} 2022-09-03T05:33:57.023 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002764ed8e88bd4265-006312e745-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.070 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:57.072 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:57.072 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:57.074 DEBUG:boto:path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:57.075 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=dummy 2022-09-03T05:33:57.075 DEBUG:boto:Method: GET 2022-09-03T05:33:57.076 DEBUG:boto:Path: /dsgvcy-17/?prefix=dummy 2022-09-03T05:33:57.076 DEBUG:boto:Data: 2022-09-03T05:33:57.077 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.077 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:57.078 DEBUG:boto:Port: 8000 2022-09-03T05:33:57.078 DEBUG:boto:Params: {} 2022-09-03T05:33:57.079 DEBUG:boto:Token: None 2022-09-03T05:33:57.080 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/ 2022-09-03T05:33:57.080 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg= 2022-09-03T05:33:57.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg=', 'Content-Length': '0'} 2022-09-03T05:33:57.084 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003ea04b93210316ac-006312e745-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.084 DEBUG:boto:b'dsgvcy-17dummy1000falsedummy2022-09-03T05:27:58.283Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:57.085 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:33:57.086 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.086 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.087 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.088 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.088 DEBUG:boto:Method: GET 2022-09-03T05:33:57.089 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.089 DEBUG:boto:Data: 2022-09-03T05:33:57.090 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.090 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:57.091 DEBUG:boto:Port: 8001 2022-09-03T05:33:57.091 DEBUG:boto:Params: {} 2022-09-03T05:33:57.092 DEBUG:boto:Token: None 2022-09-03T05:33:57.093 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.094 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JbANO73DzM9CnvVemVcSHh9+WJg= 2022-09-03T05:33:57.094 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JbANO73DzM9CnvVemVcSHh9+WJg=', 'Content-Length': '0'} 2022-09-03T05:33:57.096 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005e0b066d537b4b36-006312e745-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.097 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.098 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.098 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.099 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.099 DEBUG:boto:Method: GET 2022-09-03T05:33:57.100 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.100 DEBUG:boto:Data: 2022-09-03T05:33:57.101 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.101 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:57.102 DEBUG:boto:Port: 8000 2022-09-03T05:33:57.103 DEBUG:boto:Params: {} 2022-09-03T05:33:57.104 DEBUG:boto:Token: None 2022-09-03T05:33:57.104 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.105 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JbANO73DzM9CnvVemVcSHh9+WJg= 2022-09-03T05:33:57.106 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JbANO73DzM9CnvVemVcSHh9+WJg=', 'Content-Length': '0'} 2022-09-03T05:33:57.109 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003c38fb65111e5ce9-006312e745-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.111 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.112 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.113 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.113 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.115 DEBUG:boto:Method: HEAD 2022-09-03T05:33:57.116 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.117 DEBUG:boto:Data: 2022-09-03T05:33:57.118 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.119 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:57.120 DEBUG:boto:Port: 8001 2022-09-03T05:33:57.120 DEBUG:boto:Params: {} 2022-09-03T05:33:57.121 DEBUG:boto:Token: None 2022-09-03T05:33:57.122 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.123 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iesTf70gb5ZA1ffgIKwJOrXHBI0= 2022-09-03T05:33:57.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iesTf70gb5ZA1ffgIKwJOrXHBI0=', 'Content-Length': '0'} 2022-09-03T05:33:57.127 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000017ae9d9b33207b64-006312e745-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.128 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.128 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.129 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.129 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.130 DEBUG:boto:Method: HEAD 2022-09-03T05:33:57.131 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.132 DEBUG:boto:Data: 2022-09-03T05:33:57.132 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.133 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:57.134 DEBUG:boto:Port: 8000 2022-09-03T05:33:57.135 DEBUG:boto:Params: {} 2022-09-03T05:33:57.135 DEBUG:boto:Token: None 2022-09-03T05:33:57.136 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.137 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iesTf70gb5ZA1ffgIKwJOrXHBI0= 2022-09-03T05:33:57.138 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iesTf70gb5ZA1ffgIKwJOrXHBI0=', 'Content-Length': '0'} 2022-09-03T05:33:57.142 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005a210b65f663e087-006312e745-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.143 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:33:57.144 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.145 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.145 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.146 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.146 DEBUG:boto:Method: GET 2022-09-03T05:33:57.147 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.148 DEBUG:boto:Data: 2022-09-03T05:33:57.148 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.149 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:57.149 DEBUG:boto:Port: 8001 2022-09-03T05:33:57.150 DEBUG:boto:Params: {} 2022-09-03T05:33:57.151 DEBUG:boto:Token: None 2022-09-03T05:33:57.152 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.152 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JbANO73DzM9CnvVemVcSHh9+WJg= 2022-09-03T05:33:57.153 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JbANO73DzM9CnvVemVcSHh9+WJg=', 'Content-Length': '0'} 2022-09-03T05:33:57.158 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003b1ea84bc6fc49e8-006312e745-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.159 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.160 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.160 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.161 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.161 DEBUG:boto:Method: GET 2022-09-03T05:33:57.162 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.162 DEBUG:boto:Data: 2022-09-03T05:33:57.163 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.164 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:57.164 DEBUG:boto:Port: 8000 2022-09-03T05:33:57.165 DEBUG:boto:Params: {} 2022-09-03T05:33:57.166 DEBUG:boto:Token: None 2022-09-03T05:33:57.167 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.168 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JbANO73DzM9CnvVemVcSHh9+WJg= 2022-09-03T05:33:57.169 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JbANO73DzM9CnvVemVcSHh9+WJg=', 'Content-Length': '0'} 2022-09-03T05:33:57.174 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000091c1b09c3f5f83ca-006312e745-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.175 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:57.175 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:57.176 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:57.177 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:57.177 DEBUG:boto:Method: GET 2022-09-03T05:33:57.178 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:57.178 DEBUG:boto:Data: 2022-09-03T05:33:57.179 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.179 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:57.180 DEBUG:boto:Port: 8001 2022-09-03T05:33:57.180 DEBUG:boto:Params: {} 2022-09-03T05:33:57.181 DEBUG:boto:Token: None 2022-09-03T05:33:57.182 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/ 2022-09-03T05:33:57.184 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg= 2022-09-03T05:33:57.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg=', 'Content-Length': '0'} 2022-09-03T05:33:57.188 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000014ee7d2d96fecec5-006312e745-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.234 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:57.236 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:57.236 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:57.237 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:57.238 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:57.238 DEBUG:boto:Method: GET 2022-09-03T05:33:57.239 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:57.240 DEBUG:boto:Data: 2022-09-03T05:33:57.241 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.242 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:57.243 DEBUG:boto:Port: 8000 2022-09-03T05:33:57.244 DEBUG:boto:Params: {} 2022-09-03T05:33:57.245 DEBUG:boto:Token: None 2022-09-03T05:33:57.247 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/ 2022-09-03T05:33:57.249 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg= 2022-09-03T05:33:57.250 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg=', 'Content-Length': '0'} 2022-09-03T05:33:57.253 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000051970487dcfd556-006312e745-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.254 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:57.255 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.256 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.256 DEBUG:boto:Method: HEAD 2022-09-03T05:33:57.257 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.258 DEBUG:boto:Data: 2022-09-03T05:33:57.259 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.260 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:57.260 DEBUG:boto:Port: 8001 2022-09-03T05:33:57.261 DEBUG:boto:Params: {} 2022-09-03T05:33:57.262 DEBUG:boto:Token: None 2022-09-03T05:33:57.263 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.264 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cNN+kUDoX692zWxjRmnIZpk/4vE= 2022-09-03T05:33:57.265 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cNN+kUDoX692zWxjRmnIZpk/4vE=', 'Content-Length': '0'} 2022-09-03T05:33:57.267 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000033ba8c37e9ade88b-006312e745-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.268 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.269 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.269 DEBUG:boto:Method: HEAD 2022-09-03T05:33:57.270 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.270 DEBUG:boto:Data: 2022-09-03T05:33:57.271 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.271 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:57.272 DEBUG:boto:Port: 8000 2022-09-03T05:33:57.273 DEBUG:boto:Params: {} 2022-09-03T05:33:57.273 DEBUG:boto:Token: None 2022-09-03T05:33:57.274 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.275 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cNN+kUDoX692zWxjRmnIZpk/4vE= 2022-09-03T05:33:57.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cNN+kUDoX692zWxjRmnIZpk/4vE=', 'Content-Length': '0'} 2022-09-03T05:33:57.280 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d6b00e4c2638b451-006312e745-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.281 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:33:57.282 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.283 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.284 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.285 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.285 DEBUG:boto:Method: GET 2022-09-03T05:33:57.286 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.287 DEBUG:boto:Data: 2022-09-03T05:33:57.287 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.289 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:57.290 DEBUG:boto:Port: 8001 2022-09-03T05:33:57.291 DEBUG:boto:Params: {} 2022-09-03T05:33:57.292 DEBUG:boto:Token: None 2022-09-03T05:33:57.293 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.294 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JbANO73DzM9CnvVemVcSHh9+WJg= 2022-09-03T05:33:57.295 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JbANO73DzM9CnvVemVcSHh9+WJg=', 'Content-Length': '0'} 2022-09-03T05:33:57.298 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001a29b070e5573026-006312e745-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.299 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.300 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1 2022-09-03T05:33:57.301 DEBUG:boto:path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.302 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.303 DEBUG:boto:Method: GET 2022-09-03T05:33:57.304 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.304 DEBUG:boto:Data: 2022-09-03T05:33:57.305 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.306 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:57.317 DEBUG:boto:Port: 8000 2022-09-03T05:33:57.318 DEBUG:boto:Params: {} 2022-09-03T05:33:57.319 DEBUG:boto:Token: None 2022-09-03T05:33:57.320 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone1?versionId=yx2lTqvongZDmkvI6tLywhzAB9azvA0 2022-09-03T05:33:57.321 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JbANO73DzM9CnvVemVcSHh9+WJg= 2022-09-03T05:33:57.322 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JbANO73DzM9CnvVemVcSHh9+WJg=', 'Content-Length': '0'} 2022-09-03T05:33:57.324 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'yx2lTqvongZDmkvI6tLywhzAB9azvA0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cc3d1bf14ef9c8a5-006312e745-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.326 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:57.327 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:57.328 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:57.328 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:57.329 DEBUG:boto:Method: GET 2022-09-03T05:33:57.330 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:57.331 DEBUG:boto:Data: 2022-09-03T05:33:57.332 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.333 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:57.334 DEBUG:boto:Port: 8001 2022-09-03T05:33:57.335 DEBUG:boto:Params: {} 2022-09-03T05:33:57.336 DEBUG:boto:Token: None 2022-09-03T05:33:57.337 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/ 2022-09-03T05:33:57.338 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg= 2022-09-03T05:33:57.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg=', 'Content-Length': '0'} 2022-09-03T05:33:57.343 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000315e7b381e5dfc1f-006312e745-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.344 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:57.345 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:57.346 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:57.347 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:57.347 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:57.348 DEBUG:boto:Method: GET 2022-09-03T05:33:57.349 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone1 2022-09-03T05:33:57.350 DEBUG:boto:Data: 2022-09-03T05:33:57.351 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.351 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:57.352 DEBUG:boto:Port: 8000 2022-09-03T05:33:57.352 DEBUG:boto:Params: {} 2022-09-03T05:33:57.353 DEBUG:boto:Token: None 2022-09-03T05:33:57.353 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/ 2022-09-03T05:33:57.354 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg= 2022-09-03T05:33:57.355 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg=', 'Content-Length': '0'} 2022-09-03T05:33:57.357 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008e34e7b92de103e2-006312e745-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.358 DEBUG:boto:b'dsgvcy-17obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.351Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:57.358 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:33:57.359 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.360 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.360 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.361 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.362 DEBUG:boto:Method: GET 2022-09-03T05:33:57.363 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.363 DEBUG:boto:Data: 2022-09-03T05:33:57.364 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.364 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:57.365 DEBUG:boto:Port: 8001 2022-09-03T05:33:57.365 DEBUG:boto:Params: {} 2022-09-03T05:33:57.366 DEBUG:boto:Token: None 2022-09-03T05:33:57.366 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.367 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lcZFdOyswBqNP6rxQbPdBxvAfzQ= 2022-09-03T05:33:57.368 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lcZFdOyswBqNP6rxQbPdBxvAfzQ=', 'Content-Length': '0'} 2022-09-03T05:33:57.370 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005d0781bdbb9120f1-006312e745-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.414 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.415 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.415 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.416 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.417 DEBUG:boto:Method: GET 2022-09-03T05:33:57.417 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.418 DEBUG:boto:Data: 2022-09-03T05:33:57.418 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.419 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:57.419 DEBUG:boto:Port: 8000 2022-09-03T05:33:57.420 DEBUG:boto:Params: {} 2022-09-03T05:33:57.420 DEBUG:boto:Token: None 2022-09-03T05:33:57.421 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.421 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lcZFdOyswBqNP6rxQbPdBxvAfzQ= 2022-09-03T05:33:57.422 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lcZFdOyswBqNP6rxQbPdBxvAfzQ=', 'Content-Length': '0'} 2022-09-03T05:33:57.424 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001b13418e2e1d3c1d-006312e745-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.425 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.426 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.426 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.427 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.427 DEBUG:boto:Method: HEAD 2022-09-03T05:33:57.428 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.428 DEBUG:boto:Data: 2022-09-03T05:33:57.429 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.429 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:57.430 DEBUG:boto:Port: 8001 2022-09-03T05:33:57.430 DEBUG:boto:Params: {} 2022-09-03T05:33:57.431 DEBUG:boto:Token: None 2022-09-03T05:33:57.431 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.432 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r9dLHNl81mfDzhUz3gV59tPZ7fY= 2022-09-03T05:33:57.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r9dLHNl81mfDzhUz3gV59tPZ7fY=', 'Content-Length': '0'} 2022-09-03T05:33:57.435 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b6a98db8908b2962-006312e745-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.435 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.436 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.436 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.437 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.437 DEBUG:boto:Method: HEAD 2022-09-03T05:33:57.438 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.439 DEBUG:boto:Data: 2022-09-03T05:33:57.439 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.440 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:57.440 DEBUG:boto:Port: 8000 2022-09-03T05:33:57.441 DEBUG:boto:Params: {} 2022-09-03T05:33:57.441 DEBUG:boto:Token: None 2022-09-03T05:33:57.442 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.443 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r9dLHNl81mfDzhUz3gV59tPZ7fY= 2022-09-03T05:33:57.444 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r9dLHNl81mfDzhUz3gV59tPZ7fY=', 'Content-Length': '0'} 2022-09-03T05:33:57.447 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ffbd6096552ec776-006312e745-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.448 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:33:57.449 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.450 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.451 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.452 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.453 DEBUG:boto:Method: GET 2022-09-03T05:33:57.454 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.455 DEBUG:boto:Data: 2022-09-03T05:33:57.457 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.458 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:57.459 DEBUG:boto:Port: 8001 2022-09-03T05:33:57.460 DEBUG:boto:Params: {} 2022-09-03T05:33:57.461 DEBUG:boto:Token: None 2022-09-03T05:33:57.462 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.463 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lcZFdOyswBqNP6rxQbPdBxvAfzQ= 2022-09-03T05:33:57.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lcZFdOyswBqNP6rxQbPdBxvAfzQ=', 'Content-Length': '0'} 2022-09-03T05:33:57.470 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b59483647c2d2ffe-006312e745-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.472 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.473 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.474 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.474 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.475 DEBUG:boto:Method: GET 2022-09-03T05:33:57.476 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.477 DEBUG:boto:Data: 2022-09-03T05:33:57.478 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.479 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:57.480 DEBUG:boto:Port: 8000 2022-09-03T05:33:57.480 DEBUG:boto:Params: {} 2022-09-03T05:33:57.481 DEBUG:boto:Token: None 2022-09-03T05:33:57.482 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lcZFdOyswBqNP6rxQbPdBxvAfzQ= 2022-09-03T05:33:57.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lcZFdOyswBqNP6rxQbPdBxvAfzQ=', 'Content-Length': '0'} 2022-09-03T05:33:57.487 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d762d2a754600856-006312e745-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.493 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:57.493 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:57.494 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:57.494 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:57.495 DEBUG:boto:Method: GET 2022-09-03T05:33:57.496 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:57.496 DEBUG:boto:Data: 2022-09-03T05:33:57.497 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.497 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:57.498 DEBUG:boto:Port: 8001 2022-09-03T05:33:57.500 DEBUG:boto:Params: {} 2022-09-03T05:33:57.500 DEBUG:boto:Token: None 2022-09-03T05:33:57.501 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/ 2022-09-03T05:33:57.501 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg= 2022-09-03T05:33:57.502 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg=', 'Content-Length': '0'} 2022-09-03T05:33:57.505 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000031e25d7641435e38-006312e745-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.546 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:57.547 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:57.548 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:57.549 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:57.549 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:57.550 DEBUG:boto:Method: GET 2022-09-03T05:33:57.550 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:57.551 DEBUG:boto:Data: 2022-09-03T05:33:57.551 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.552 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:57.552 DEBUG:boto:Port: 8000 2022-09-03T05:33:57.553 DEBUG:boto:Params: {} 2022-09-03T05:33:57.553 DEBUG:boto:Token: None 2022-09-03T05:33:57.554 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/ 2022-09-03T05:33:57.554 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg= 2022-09-03T05:33:57.555 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg=', 'Content-Length': '0'} 2022-09-03T05:33:57.558 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003d622d9cfbb948a4-006312e745-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.559 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:57.560 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.560 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.561 DEBUG:boto:Method: HEAD 2022-09-03T05:33:57.562 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.563 DEBUG:boto:Data: 2022-09-03T05:33:57.563 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.564 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:57.564 DEBUG:boto:Port: 8001 2022-09-03T05:33:57.565 DEBUG:boto:Params: {} 2022-09-03T05:33:57.565 DEBUG:boto:Token: None 2022-09-03T05:33:57.566 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.566 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bFyX96DOQ+j/4Q/e5bdMRZj6RzM= 2022-09-03T05:33:57.567 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bFyX96DOQ+j/4Q/e5bdMRZj6RzM=', 'Content-Length': '0'} 2022-09-03T05:33:57.570 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dd8b096bb00670f4-006312e745-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.570 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.571 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.572 DEBUG:boto:Method: HEAD 2022-09-03T05:33:57.572 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.573 DEBUG:boto:Data: 2022-09-03T05:33:57.573 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.574 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:57.574 DEBUG:boto:Port: 8000 2022-09-03T05:33:57.575 DEBUG:boto:Params: {} 2022-09-03T05:33:57.576 DEBUG:boto:Token: None 2022-09-03T05:33:57.576 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.577 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bFyX96DOQ+j/4Q/e5bdMRZj6RzM= 2022-09-03T05:33:57.578 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bFyX96DOQ+j/4Q/e5bdMRZj6RzM=', 'Content-Length': '0'} 2022-09-03T05:33:57.581 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000865fc291d84291d-006312e745-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.582 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:33:57.582 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.583 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.584 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.585 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.585 DEBUG:boto:Method: GET 2022-09-03T05:33:57.586 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.586 DEBUG:boto:Data: 2022-09-03T05:33:57.587 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.587 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:57.588 DEBUG:boto:Port: 8001 2022-09-03T05:33:57.588 DEBUG:boto:Params: {} 2022-09-03T05:33:57.589 DEBUG:boto:Token: None 2022-09-03T05:33:57.590 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.590 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lcZFdOyswBqNP6rxQbPdBxvAfzQ= 2022-09-03T05:33:57.592 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lcZFdOyswBqNP6rxQbPdBxvAfzQ=', 'Content-Length': '0'} 2022-09-03T05:33:57.595 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000033a8a367ba5e2c59-006312e745-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.596 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.597 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2 2022-09-03T05:33:57.597 DEBUG:boto:path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.598 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.598 DEBUG:boto:Method: GET 2022-09-03T05:33:57.599 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.599 DEBUG:boto:Data: 2022-09-03T05:33:57.600 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.600 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:57.601 DEBUG:boto:Port: 8000 2022-09-03T05:33:57.601 DEBUG:boto:Params: {} 2022-09-03T05:33:57.602 DEBUG:boto:Token: None 2022-09-03T05:33:57.603 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone2?versionId=qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D 2022-09-03T05:33:57.603 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lcZFdOyswBqNP6rxQbPdBxvAfzQ= 2022-09-03T05:33:57.604 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lcZFdOyswBqNP6rxQbPdBxvAfzQ=', 'Content-Length': '0'} 2022-09-03T05:33:57.606 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'qJZTeQzADwbtR5SJZIh1QJ7FEtHiz1D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f1b1271c043c6c24-006312e745-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.607 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:57.609 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:57.609 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:57.610 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:57.611 DEBUG:boto:Method: GET 2022-09-03T05:33:57.611 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:57.612 DEBUG:boto:Data: 2022-09-03T05:33:57.612 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.613 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:57.614 DEBUG:boto:Port: 8001 2022-09-03T05:33:57.614 DEBUG:boto:Params: {} 2022-09-03T05:33:57.615 DEBUG:boto:Token: None 2022-09-03T05:33:57.615 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/ 2022-09-03T05:33:57.616 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg= 2022-09-03T05:33:57.617 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg=', 'Content-Length': '0'} 2022-09-03T05:33:57.619 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f0f410d60b2e2926-006312e745-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.620 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:57.621 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:57.621 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:57.622 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:57.623 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:57.623 DEBUG:boto:Method: GET 2022-09-03T05:33:57.624 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone2 2022-09-03T05:33:57.624 DEBUG:boto:Data: 2022-09-03T05:33:57.625 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.625 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:57.626 DEBUG:boto:Port: 8000 2022-09-03T05:33:57.627 DEBUG:boto:Params: {} 2022-09-03T05:33:57.627 DEBUG:boto:Token: None 2022-09-03T05:33:57.628 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/ 2022-09-03T05:33:57.629 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg= 2022-09-03T05:33:57.629 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg=', 'Content-Length': '0'} 2022-09-03T05:33:57.632 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002d5ded9dadbef9db-006312e745-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.674 DEBUG:boto:b'dsgvcy-17obj-test-zone21000falseobj-test-zone22022-09-03T05:29:09.531Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:57.675 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:33:57.676 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.676 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.677 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.677 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.678 DEBUG:boto:Method: GET 2022-09-03T05:33:57.678 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.679 DEBUG:boto:Data: 2022-09-03T05:33:57.679 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.680 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:57.680 DEBUG:boto:Port: 8001 2022-09-03T05:33:57.681 DEBUG:boto:Params: {} 2022-09-03T05:33:57.681 DEBUG:boto:Token: None 2022-09-03T05:33:57.682 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.683 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:haVNUlqoanr+NGNyzOdQumqwsnQ= 2022-09-03T05:33:57.683 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:haVNUlqoanr+NGNyzOdQumqwsnQ=', 'Content-Length': '0'} 2022-09-03T05:33:57.685 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000acc86bca011a44cf-006312e745-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.686 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.687 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.688 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.688 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.689 DEBUG:boto:Method: GET 2022-09-03T05:33:57.689 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.690 DEBUG:boto:Data: 2022-09-03T05:33:57.690 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.691 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:57.692 DEBUG:boto:Port: 8000 2022-09-03T05:33:57.692 DEBUG:boto:Params: {} 2022-09-03T05:33:57.693 DEBUG:boto:Token: None 2022-09-03T05:33:57.693 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.694 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:haVNUlqoanr+NGNyzOdQumqwsnQ= 2022-09-03T05:33:57.695 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:haVNUlqoanr+NGNyzOdQumqwsnQ=', 'Content-Length': '0'} 2022-09-03T05:33:57.697 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dbd15d02ad20ddd5-006312e745-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.698 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.698 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.700 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.700 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.701 DEBUG:boto:Method: HEAD 2022-09-03T05:33:57.701 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.702 DEBUG:boto:Data: 2022-09-03T05:33:57.702 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.703 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:57.703 DEBUG:boto:Port: 8001 2022-09-03T05:33:57.704 DEBUG:boto:Params: {} 2022-09-03T05:33:57.704 DEBUG:boto:Token: None 2022-09-03T05:33:57.705 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.705 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PRMG8eBaDb81XVvxOPtUrycuM60= 2022-09-03T05:33:57.706 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PRMG8eBaDb81XVvxOPtUrycuM60=', 'Content-Length': '0'} 2022-09-03T05:33:57.708 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d55c10cac9382705-006312e745-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.709 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.711 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.712 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.713 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.715 DEBUG:boto:Method: HEAD 2022-09-03T05:33:57.716 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.717 DEBUG:boto:Data: 2022-09-03T05:33:57.718 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.718 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:57.719 DEBUG:boto:Port: 8000 2022-09-03T05:33:57.720 DEBUG:boto:Params: {} 2022-09-03T05:33:57.722 DEBUG:boto:Token: None 2022-09-03T05:33:57.723 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.724 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PRMG8eBaDb81XVvxOPtUrycuM60= 2022-09-03T05:33:57.725 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PRMG8eBaDb81XVvxOPtUrycuM60=', 'Content-Length': '0'} 2022-09-03T05:33:57.727 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d396397608d767a1-006312e745-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.729 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:33:57.730 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.732 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.733 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.734 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.736 DEBUG:boto:Method: GET 2022-09-03T05:33:57.737 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.738 DEBUG:boto:Data: 2022-09-03T05:33:57.739 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.740 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:57.741 DEBUG:boto:Port: 8001 2022-09-03T05:33:57.742 DEBUG:boto:Params: {} 2022-09-03T05:33:57.744 DEBUG:boto:Token: None 2022-09-03T05:33:57.745 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.747 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:haVNUlqoanr+NGNyzOdQumqwsnQ= 2022-09-03T05:33:57.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:haVNUlqoanr+NGNyzOdQumqwsnQ=', 'Content-Length': '0'} 2022-09-03T05:33:57.751 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009efe4dff976efaa1-006312e745-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.752 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.752 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.753 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.753 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.754 DEBUG:boto:Method: GET 2022-09-03T05:33:57.754 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.755 DEBUG:boto:Data: 2022-09-03T05:33:57.755 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.756 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:57.757 DEBUG:boto:Port: 8000 2022-09-03T05:33:57.758 DEBUG:boto:Params: {} 2022-09-03T05:33:57.758 DEBUG:boto:Token: None 2022-09-03T05:33:57.759 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.760 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:haVNUlqoanr+NGNyzOdQumqwsnQ= 2022-09-03T05:33:57.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:haVNUlqoanr+NGNyzOdQumqwsnQ=', 'Content-Length': '0'} 2022-09-03T05:33:57.763 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009df158b0a828a3b7-006312e745-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.764 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:57.764 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:57.765 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:57.765 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:57.766 DEBUG:boto:Method: GET 2022-09-03T05:33:57.767 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:57.768 DEBUG:boto:Data: 2022-09-03T05:33:57.768 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.769 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:57.769 DEBUG:boto:Port: 8001 2022-09-03T05:33:57.770 DEBUG:boto:Params: {} 2022-09-03T05:33:57.771 DEBUG:boto:Token: None 2022-09-03T05:33:57.771 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/ 2022-09-03T05:33:57.772 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg= 2022-09-03T05:33:57.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg=', 'Content-Length': '0'} 2022-09-03T05:33:57.775 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e4168add7797389f-006312e745-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.818 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:57.819 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:57.820 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:57.820 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:57.821 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:57.821 DEBUG:boto:Method: GET 2022-09-03T05:33:57.822 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:57.823 DEBUG:boto:Data: 2022-09-03T05:33:57.823 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.824 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:57.824 DEBUG:boto:Port: 8000 2022-09-03T05:33:57.825 DEBUG:boto:Params: {} 2022-09-03T05:33:57.825 DEBUG:boto:Token: None 2022-09-03T05:33:57.826 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/ 2022-09-03T05:33:57.826 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg= 2022-09-03T05:33:57.827 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg=', 'Content-Length': '0'} 2022-09-03T05:33:57.830 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c3829a3958af1881-006312e745-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.831 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:57.833 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.834 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.835 DEBUG:boto:Method: HEAD 2022-09-03T05:33:57.835 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.836 DEBUG:boto:Data: 2022-09-03T05:33:57.836 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.837 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:57.837 DEBUG:boto:Port: 8001 2022-09-03T05:33:57.838 DEBUG:boto:Params: {} 2022-09-03T05:33:57.838 DEBUG:boto:Token: None 2022-09-03T05:33:57.839 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.839 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kUqwDFGWJcQ2SvJXdx58VdHGSJg= 2022-09-03T05:33:57.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kUqwDFGWJcQ2SvJXdx58VdHGSJg=', 'Content-Length': '0'} 2022-09-03T05:33:57.842 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000070f216e807b51328-006312e745-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.843 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.844 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.844 DEBUG:boto:Method: HEAD 2022-09-03T05:33:57.845 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.845 DEBUG:boto:Data: 2022-09-03T05:33:57.846 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.846 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:57.847 DEBUG:boto:Port: 8000 2022-09-03T05:33:57.847 DEBUG:boto:Params: {} 2022-09-03T05:33:57.848 DEBUG:boto:Token: None 2022-09-03T05:33:57.849 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.849 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kUqwDFGWJcQ2SvJXdx58VdHGSJg= 2022-09-03T05:33:57.850 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kUqwDFGWJcQ2SvJXdx58VdHGSJg=', 'Content-Length': '0'} 2022-09-03T05:33:57.852 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000091b207ab3765196a-006312e745-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.853 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:33:57.854 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.854 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.855 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.856 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.856 DEBUG:boto:Method: GET 2022-09-03T05:33:57.857 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.857 DEBUG:boto:Data: 2022-09-03T05:33:57.858 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.858 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:57.859 DEBUG:boto:Port: 8001 2022-09-03T05:33:57.859 DEBUG:boto:Params: {} 2022-09-03T05:33:57.860 DEBUG:boto:Token: None 2022-09-03T05:33:57.861 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.861 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:haVNUlqoanr+NGNyzOdQumqwsnQ= 2022-09-03T05:33:57.862 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:haVNUlqoanr+NGNyzOdQumqwsnQ=', 'Content-Length': '0'} 2022-09-03T05:33:57.865 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000212729381f6a377d-006312e745-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.867 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.867 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3 2022-09-03T05:33:57.868 DEBUG:boto:path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.868 DEBUG:boto:auth_path=/dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.869 DEBUG:boto:Method: GET 2022-09-03T05:33:57.869 DEBUG:boto:Path: /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.870 DEBUG:boto:Data: 2022-09-03T05:33:57.871 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.871 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:57.872 DEBUG:boto:Port: 8000 2022-09-03T05:33:57.872 DEBUG:boto:Params: {} 2022-09-03T05:33:57.873 DEBUG:boto:Token: None 2022-09-03T05:33:57.874 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/obj-test-zone3?versionId=4hufCTEaqzXM3cUjSTjtsbNbxySDq.v 2022-09-03T05:33:57.875 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:haVNUlqoanr+NGNyzOdQumqwsnQ= 2022-09-03T05:33:57.875 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:haVNUlqoanr+NGNyzOdQumqwsnQ=', 'Content-Length': '0'} 2022-09-03T05:33:57.878 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4hufCTEaqzXM3cUjSTjtsbNbxySDq.v'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001d1cb8521c9fdedc-006312e745-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.879 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:57.879 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:57.880 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:57.880 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:57.881 DEBUG:boto:Method: GET 2022-09-03T05:33:57.882 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:57.883 DEBUG:boto:Data: 2022-09-03T05:33:57.883 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.884 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:57.885 DEBUG:boto:Port: 8001 2022-09-03T05:33:57.886 DEBUG:boto:Params: {} 2022-09-03T05:33:57.887 DEBUG:boto:Token: None 2022-09-03T05:33:57.888 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/ 2022-09-03T05:33:57.890 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg= 2022-09-03T05:33:57.891 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg=', 'Content-Length': '0'} 2022-09-03T05:33:57.894 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e5133a418e62188a-006312e745-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.895 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:57.896 DEBUG:boto:path=/dsgvcy-17/ 2022-09-03T05:33:57.897 DEBUG:boto:auth_path=/dsgvcy-17/ 2022-09-03T05:33:57.897 DEBUG:boto:path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:57.898 DEBUG:boto:auth_path=/dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:57.899 DEBUG:boto:Method: GET 2022-09-03T05:33:57.899 DEBUG:boto:Path: /dsgvcy-17/?prefix=obj-test-zone3 2022-09-03T05:33:57.900 DEBUG:boto:Data: 2022-09-03T05:33:57.900 DEBUG:boto:Headers: {} 2022-09-03T05:33:57.901 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:57.901 DEBUG:boto:Port: 8000 2022-09-03T05:33:57.902 DEBUG:boto:Params: {} 2022-09-03T05:33:57.903 DEBUG:boto:Token: None 2022-09-03T05:33:57.903 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:57 GMT /dsgvcy-17/ 2022-09-03T05:33:57.904 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg= 2022-09-03T05:33:57.905 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0gCj/s2pbpU+I1E2KBb5jGTrfdg=', 'Content-Length': '0'} 2022-09-03T05:33:57.908 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d1d7a6b482a7f568-006312e745-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:57.909 DEBUG:boto:b'dsgvcy-17obj-test-zone31000falseobj-test-zone32022-09-03T05:29:09.702Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:57.911 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-17 zones={test-zone3, test-zone2} 2022-09-03T05:33:57.912 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-18 2022-09-03T05:33:57.913 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:57.913 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:57.914 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-18 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:33:57.989 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:33:57.990 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:33:58.038 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:33:58.039+0000 7faf14d80600 1 bucket sync caught up with source: 2022-09-03T05:33:58.039 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000011.27.5, , 00000000004.16.5, , 00000000008.21.5, , , , , ] 2022-09-03T05:33:58.040 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000011.27.5, , 00000000004.16.5, , 00000000008.21.5, , , , , ] 2022-09-03T05:33:58.040 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:33:58.039+0000 7faf14d80600 0 bucket checkpoint complete 2022-09-03T05:33:58.052 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-18 2022-09-03T05:33:58.053 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:58.054 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:58.055 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-18 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:33:58.120 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:33:58.120 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:33:58.179 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:58.180+0000 7fe793749600 1 bucket sync caught up with source: 2022-09-03T05:33:58.180 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000011.27.5, , 00000000004.16.5, , 00000000008.21.5, , , , , ] 2022-09-03T05:33:58.180 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000011.27.5, , 00000000004.16.5, , 00000000008.21.5, , , , , ] 2022-09-03T05:33:58.180 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:58.180+0000 7fe793749600 0 bucket checkpoint complete 2022-09-03T05:33:58.198 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-18 2022-09-03T05:33:58.199 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:58.200 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:58.200 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-18 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:33:58.260 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:33:58.261 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:33:58.294 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:33:58.295+0000 7f33ca45a600 1 bucket sync caught up with source: 2022-09-03T05:33:58.295 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000011.27.5, , 00000000004.16.5, , 00000000008.21.5, , , , , ] 2022-09-03T05:33:58.295 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000011.27.5, , 00000000004.16.5, , 00000000008.21.5, , , , , ] 2022-09-03T05:33:58.296 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:33:58.295+0000 7f33ca45a600 0 bucket checkpoint complete 2022-09-03T05:33:58.306 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-18 2022-09-03T05:33:58.307 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:58.307 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:58.308 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-18 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:33:58.368 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:33:58.368 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:33:58.402 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:58.402+0000 7f8cc1b1e600 1 bucket sync caught up with source: 2022-09-03T05:33:58.403 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.27.5, , 00000000011.46.5, , 00000000008.34.5, , , , , ] 2022-09-03T05:33:58.403 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.27.5, , 00000000011.46.5, , 00000000008.34.5, , , , , ] 2022-09-03T05:33:58.403 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:58.402+0000 7f8cc1b1e600 0 bucket checkpoint complete 2022-09-03T05:33:58.418 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-18 2022-09-03T05:33:58.419 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:58.419 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:58.420 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-18 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:33:58.476 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:33:58.477 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:33:58.526 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:58.526+0000 7f12397a5600 1 bucket sync caught up with source: 2022-09-03T05:33:58.527 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000004.27.5, , 00000000011.46.5, , 00000000008.34.5, , , , , ] 2022-09-03T05:33:58.527 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000004.27.5, , 00000000011.46.5, , 00000000008.34.5, , , , , ] 2022-09-03T05:33:58.527 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:58.526+0000 7f12397a5600 0 bucket checkpoint complete 2022-09-03T05:33:58.551 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-18 2022-09-03T05:33:58.552 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-18 2022-09-03T05:33:58.552 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:58.553 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:58.553 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-18 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:33:58.611 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:33:58.612 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:33:58.660 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:58.660+0000 7f2493447600 1 bucket sync caught up with source: 2022-09-03T05:33:58.660 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000005.20.5, , 00000000004.37.5, , 00000000016.47.5, , , , , ] 2022-09-03T05:33:58.661 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000005.20.5, , 00000000004.37.5, , 00000000016.47.5, , , , , ] 2022-09-03T05:33:58.661 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:33:58.660+0000 7f2493447600 0 bucket checkpoint complete 2022-09-03T05:33:58.672 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-18 2022-09-03T05:33:58.673 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:58.673 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-18', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:33:58.674 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-18 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:33:58.731 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:33:58.732 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:33:58.779 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:33:58.780+0000 7fe4db9d1600 1 bucket sync caught up with source: 2022-09-03T05:33:58.779 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000005.20.5, , 00000000004.37.5, , 00000000016.47.5, , , , , ] 2022-09-03T05:33:58.780 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000005.20.5, , 00000000004.37.5, , 00000000016.47.5, , , , , ] 2022-09-03T05:33:58.780 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:33:58.780+0000 7fe4db9d1600 0 bucket checkpoint complete 2022-09-03T05:33:58.796 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-18 2022-09-03T05:33:58.797 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-18 zones={test-zone2, test-zone1} 2022-09-03T05:33:58.797 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:58.799 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:58.799 DEBUG:boto:Method: HEAD 2022-09-03T05:33:58.800 DEBUG:boto:Path: /dsgvcy-18/ 2022-09-03T05:33:58.800 DEBUG:boto:Data: 2022-09-03T05:33:58.801 DEBUG:boto:Headers: {} 2022-09-03T05:33:58.801 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:58.802 DEBUG:boto:Port: 8000 2022-09-03T05:33:58.803 DEBUG:boto:Params: {} 2022-09-03T05:33:58.803 DEBUG:boto:Token: None 2022-09-03T05:33:58.804 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:58 GMT /dsgvcy-18/ 2022-09-03T05:33:58.805 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YKSJgDv9riJt6fKH8q61Sn21Wqw= 2022-09-03T05:33:58.805 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YKSJgDv9riJt6fKH8q61Sn21Wqw=', 'Content-Length': '0'} 2022-09-03T05:33:58.808 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a92c69ce20fdba43-006312e746-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:33:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:58.808 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:58.809 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:58.810 DEBUG:boto:Method: HEAD 2022-09-03T05:33:58.810 DEBUG:boto:Path: /dsgvcy-18/ 2022-09-03T05:33:58.811 DEBUG:boto:Data: 2022-09-03T05:33:58.811 DEBUG:boto:Headers: {} 2022-09-03T05:33:58.812 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:58.812 DEBUG:boto:Port: 8000 2022-09-03T05:33:58.813 DEBUG:boto:Params: {} 2022-09-03T05:33:58.813 DEBUG:boto:Token: None 2022-09-03T05:33:58.814 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:58 GMT /dsgvcy-18/ 2022-09-03T05:33:58.814 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YKSJgDv9riJt6fKH8q61Sn21Wqw= 2022-09-03T05:33:58.815 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YKSJgDv9riJt6fKH8q61Sn21Wqw=', 'Content-Length': '0'} 2022-09-03T05:33:58.817 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000bd1a605828eff23e-006312e746-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:33:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:58.818 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:33:58.819 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:58.819 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:58.820 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:58.821 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:58.821 DEBUG:boto:Method: GET 2022-09-03T05:33:58.822 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:58.822 DEBUG:boto:Data: 2022-09-03T05:33:58.823 DEBUG:boto:Headers: {} 2022-09-03T05:33:58.823 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:58.824 DEBUG:boto:Port: 8000 2022-09-03T05:33:58.825 DEBUG:boto:Params: {} 2022-09-03T05:33:58.825 DEBUG:boto:Token: None 2022-09-03T05:33:58.826 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:58 GMT /dsgvcy-18/?versions 2022-09-03T05:33:58.827 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WzkeuTYfjXgldRCQUL4q0uFIDcU= 2022-09-03T05:33:58.827 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WzkeuTYfjXgldRCQUL4q0uFIDcU=', 'Content-Length': '0'} 2022-09-03T05:33:58.830 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009cb066558f5e0320-006312e746-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:58.830 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:58.831 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:33:58.832 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:33:58.832 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:33:58.833 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:33:58.833 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:33:58.834 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:58.834 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:58.835 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:58.835 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:58.836 DEBUG:boto:Method: GET 2022-09-03T05:33:58.836 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:58.837 DEBUG:boto:Data: 2022-09-03T05:33:58.837 DEBUG:boto:Headers: {} 2022-09-03T05:33:58.838 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:58.839 DEBUG:boto:Port: 8000 2022-09-03T05:33:58.839 DEBUG:boto:Params: {} 2022-09-03T05:33:58.840 DEBUG:boto:Token: None 2022-09-03T05:33:58.840 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:58 GMT /dsgvcy-18/?versions 2022-09-03T05:33:58.841 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WzkeuTYfjXgldRCQUL4q0uFIDcU= 2022-09-03T05:33:58.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WzkeuTYfjXgldRCQUL4q0uFIDcU=', 'Content-Length': '0'} 2022-09-03T05:33:58.844 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003d2b6eede8529cd6-006312e746-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:58.845 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:58.846 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:33:58.846 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:33:58.847 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:33:58.847 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:33:58.848 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:58.848 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:58.849 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:58.849 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:58.850 DEBUG:boto:Method: GET 2022-09-03T05:33:58.850 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:58.851 DEBUG:boto:Data: 2022-09-03T05:33:58.851 DEBUG:boto:Headers: {} 2022-09-03T05:33:58.852 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:58.852 DEBUG:boto:Port: 8000 2022-09-03T05:33:58.853 DEBUG:boto:Params: {} 2022-09-03T05:33:58.854 DEBUG:boto:Token: None 2022-09-03T05:33:58.854 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:58 GMT /dsgvcy-18/?versions 2022-09-03T05:33:58.855 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WzkeuTYfjXgldRCQUL4q0uFIDcU= 2022-09-03T05:33:58.855 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WzkeuTYfjXgldRCQUL4q0uFIDcU=', 'Content-Length': '0'} 2022-09-03T05:33:58.858 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007f78fe69c32b93a3-006312e746-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:58.859 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:58.861 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:58.863 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:58.863 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:58.864 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:58.865 DEBUG:boto:Method: GET 2022-09-03T05:33:58.867 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:58.868 DEBUG:boto:Data: 2022-09-03T05:33:58.869 DEBUG:boto:Headers: {} 2022-09-03T05:33:58.870 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:58.871 DEBUG:boto:Port: 8000 2022-09-03T05:33:58.872 DEBUG:boto:Params: {} 2022-09-03T05:33:58.873 DEBUG:boto:Token: None 2022-09-03T05:33:58.874 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:58 GMT /dsgvcy-18/?versions 2022-09-03T05:33:58.875 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WzkeuTYfjXgldRCQUL4q0uFIDcU= 2022-09-03T05:33:58.876 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WzkeuTYfjXgldRCQUL4q0uFIDcU=', 'Content-Length': '0'} 2022-09-03T05:33:58.879 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000093c27a61bad9e96f-006312e746-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:58.880 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:58.881 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:33:58.881 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:33:58.882 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:33:58.882 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.883 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.883 DEBUG:boto:Method: GET 2022-09-03T05:33:58.884 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.885 DEBUG:boto:Data: 2022-09-03T05:33:58.885 DEBUG:boto:Headers: {} 2022-09-03T05:33:58.886 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:58.886 DEBUG:boto:Port: 8000 2022-09-03T05:33:58.887 DEBUG:boto:Params: {} 2022-09-03T05:33:58.888 DEBUG:boto:Token: None 2022-09-03T05:33:58.888 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:58 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.889 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oInJjZM3oHo4E5nTwTz+CpuoEvI= 2022-09-03T05:33:58.890 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oInJjZM3oHo4E5nTwTz+CpuoEvI=', 'Content-Length': '0'} 2022-09-03T05:33:58.892 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000405b1857b7cb1866-006312e746-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:58.892 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:33:58.893 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:33:58.894 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.894 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.895 DEBUG:boto:Method: GET 2022-09-03T05:33:58.895 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.896 DEBUG:boto:Data: 2022-09-03T05:33:58.897 DEBUG:boto:Headers: {} 2022-09-03T05:33:58.897 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:58.898 DEBUG:boto:Port: 8000 2022-09-03T05:33:58.898 DEBUG:boto:Params: {} 2022-09-03T05:33:58.899 DEBUG:boto:Token: None 2022-09-03T05:33:58.900 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:58 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.900 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oInJjZM3oHo4E5nTwTz+CpuoEvI= 2022-09-03T05:33:58.901 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oInJjZM3oHo4E5nTwTz+CpuoEvI=', 'Content-Length': '0'} 2022-09-03T05:33:58.903 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000d1fac923876eb6a2-006312e746-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:58.905 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:33:58.906 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:33:58.906 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.907 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.908 DEBUG:boto:Method: HEAD 2022-09-03T05:33:58.910 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.910 DEBUG:boto:Data: 2022-09-03T05:33:58.911 DEBUG:boto:Headers: {} 2022-09-03T05:33:58.912 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:58.913 DEBUG:boto:Port: 8000 2022-09-03T05:33:58.913 DEBUG:boto:Params: {} 2022-09-03T05:33:58.914 DEBUG:boto:Token: None 2022-09-03T05:33:58.915 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:58 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.915 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M0W15ZDvDvtr6uPV86AWnzFwdF4= 2022-09-03T05:33:58.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M0W15ZDvDvtr6uPV86AWnzFwdF4=', 'Content-Length': '0'} 2022-09-03T05:33:58.918 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000b3ed1fbcd9cba961-006312e746-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:58.919 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:33:58.919 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:33:58.920 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.920 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.921 DEBUG:boto:Method: HEAD 2022-09-03T05:33:58.921 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.922 DEBUG:boto:Data: 2022-09-03T05:33:58.923 DEBUG:boto:Headers: {} 2022-09-03T05:33:58.923 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:58.924 DEBUG:boto:Port: 8000 2022-09-03T05:33:58.924 DEBUG:boto:Params: {} 2022-09-03T05:33:58.925 DEBUG:boto:Token: None 2022-09-03T05:33:58.925 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:58 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.926 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M0W15ZDvDvtr6uPV86AWnzFwdF4= 2022-09-03T05:33:58.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M0W15ZDvDvtr6uPV86AWnzFwdF4=', 'Content-Length': '0'} 2022-09-03T05:33:58.931 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000383bda4661ffe8db-006312e746-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:58.932 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:33:58.933 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:33:58.934 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:33:58.935 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.936 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.936 DEBUG:boto:Method: GET 2022-09-03T05:33:58.938 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.938 DEBUG:boto:Data: 2022-09-03T05:33:58.939 DEBUG:boto:Headers: {} 2022-09-03T05:33:58.939 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:58.940 DEBUG:boto:Port: 8000 2022-09-03T05:33:58.940 DEBUG:boto:Params: {} 2022-09-03T05:33:58.941 DEBUG:boto:Token: None 2022-09-03T05:33:58.942 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:58 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.943 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oInJjZM3oHo4E5nTwTz+CpuoEvI= 2022-09-03T05:33:58.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oInJjZM3oHo4E5nTwTz+CpuoEvI=', 'Content-Length': '0'} 2022-09-03T05:33:58.945 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000008c49ff366aff04b2-006312e746-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:58.947 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:33:58.947 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:33:58.948 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.949 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.950 DEBUG:boto:Method: GET 2022-09-03T05:33:58.951 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.952 DEBUG:boto:Data: 2022-09-03T05:33:58.953 DEBUG:boto:Headers: {} 2022-09-03T05:33:58.954 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:58.955 DEBUG:boto:Port: 8000 2022-09-03T05:33:58.956 DEBUG:boto:Params: {} 2022-09-03T05:33:58.957 DEBUG:boto:Token: None 2022-09-03T05:33:58.958 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:58 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:58.959 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oInJjZM3oHo4E5nTwTz+CpuoEvI= 2022-09-03T05:33:58.961 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oInJjZM3oHo4E5nTwTz+CpuoEvI=', 'Content-Length': '0'} 2022-09-03T05:33:58.963 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000e7d55a60b0ef4154-006312e746-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:58.964 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:58.965 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:58.966 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:33:58.967 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:33:58.968 DEBUG:boto:Method: GET 2022-09-03T05:33:58.969 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:33:58.970 DEBUG:boto:Data: 2022-09-03T05:33:58.971 DEBUG:boto:Headers: {} 2022-09-03T05:33:58.972 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:58.973 DEBUG:boto:Port: 8000 2022-09-03T05:33:58.974 DEBUG:boto:Params: {} 2022-09-03T05:33:58.975 DEBUG:boto:Token: None 2022-09-03T05:33:58.975 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:58 GMT /dsgvcy-18/ 2022-09-03T05:33:58.976 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IPlsp8jjdmELTuhGxrbJZxEwOqM= 2022-09-03T05:33:58.977 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IPlsp8jjdmELTuhGxrbJZxEwOqM=', 'Content-Length': '0'} 2022-09-03T05:33:58.979 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000494554a8a85e03f5-006312e746-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:58.981 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:58.982 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:58.983 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:58.983 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:33:58.985 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:33:58.985 DEBUG:boto:Method: GET 2022-09-03T05:33:58.986 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:33:58.988 DEBUG:boto:Data: 2022-09-03T05:33:58.988 DEBUG:boto:Headers: {} 2022-09-03T05:33:58.989 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:58.990 DEBUG:boto:Port: 8000 2022-09-03T05:33:58.991 DEBUG:boto:Params: {} 2022-09-03T05:33:58.992 DEBUG:boto:Token: None 2022-09-03T05:33:58.994 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:58 GMT /dsgvcy-18/ 2022-09-03T05:33:58.995 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IPlsp8jjdmELTuhGxrbJZxEwOqM= 2022-09-03T05:33:58.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IPlsp8jjdmELTuhGxrbJZxEwOqM=', 'Content-Length': '0'} 2022-09-03T05:33:58.999 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009374766538428ca2-006312e746-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.000 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:59.001 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:33:59.003 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:33:59.004 DEBUG:boto:Method: HEAD 2022-09-03T05:33:59.005 DEBUG:boto:Path: /dsgvcy-18/dummy 2022-09-03T05:33:59.006 DEBUG:boto:Data: 2022-09-03T05:33:59.006 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.007 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.008 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.009 DEBUG:boto:Params: {} 2022-09-03T05:33:59.010 DEBUG:boto:Token: None 2022-09-03T05:33:59.011 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/dummy 2022-09-03T05:33:59.012 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4+TghE51WeQa+gQ+jI4Y4h8dkJg= 2022-09-03T05:33:59.012 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4+TghE51WeQa+gQ+jI4Y4h8dkJg=', 'Content-Length': '0'} 2022-09-03T05:33:59.015 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f749c2e33a10900d-006312e747-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.016 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:33:59.016 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:33:59.017 DEBUG:boto:Method: HEAD 2022-09-03T05:33:59.018 DEBUG:boto:Path: /dsgvcy-18/dummy 2022-09-03T05:33:59.019 DEBUG:boto:Data: 2022-09-03T05:33:59.020 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.021 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.022 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.023 DEBUG:boto:Params: {} 2022-09-03T05:33:59.024 DEBUG:boto:Token: None 2022-09-03T05:33:59.025 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/dummy 2022-09-03T05:33:59.026 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4+TghE51WeQa+gQ+jI4Y4h8dkJg= 2022-09-03T05:33:59.027 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4+TghE51WeQa+gQ+jI4Y4h8dkJg=', 'Content-Length': '0'} 2022-09-03T05:33:59.030 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000007fe1e05b0d1b785e-006312e747-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.031 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:33:59.031 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:33:59.032 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:33:59.032 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.033 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.033 DEBUG:boto:Method: GET 2022-09-03T05:33:59.034 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.035 DEBUG:boto:Data: 2022-09-03T05:33:59.035 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.036 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.036 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.037 DEBUG:boto:Params: {} 2022-09-03T05:33:59.038 DEBUG:boto:Token: None 2022-09-03T05:33:59.038 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.039 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1Q2wPCaU5vRiaPx28/SqRYZ4Yo0= 2022-09-03T05:33:59.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1Q2wPCaU5vRiaPx28/SqRYZ4Yo0=', 'Content-Length': '0'} 2022-09-03T05:33:59.041 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000bb9148d190450ea4-006312e747-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.042 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:33:59.043 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:33:59.043 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.044 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.044 DEBUG:boto:Method: GET 2022-09-03T05:33:59.045 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.045 DEBUG:boto:Data: 2022-09-03T05:33:59.046 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.047 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.047 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.048 DEBUG:boto:Params: {} 2022-09-03T05:33:59.048 DEBUG:boto:Token: None 2022-09-03T05:33:59.049 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.050 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1Q2wPCaU5vRiaPx28/SqRYZ4Yo0= 2022-09-03T05:33:59.050 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1Q2wPCaU5vRiaPx28/SqRYZ4Yo0=', 'Content-Length': '0'} 2022-09-03T05:33:59.052 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000078efa068145b5a5a-006312e747-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.053 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:59.054 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:59.054 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:33:59.055 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:33:59.055 DEBUG:boto:Method: GET 2022-09-03T05:33:59.056 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:33:59.057 DEBUG:boto:Data: 2022-09-03T05:33:59.057 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.058 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.058 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.059 DEBUG:boto:Params: {} 2022-09-03T05:33:59.060 DEBUG:boto:Token: None 2022-09-03T05:33:59.060 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/ 2022-09-03T05:33:59.061 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4= 2022-09-03T05:33:59.062 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4=', 'Content-Length': '0'} 2022-09-03T05:33:59.064 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000df8305d227dddc1a-006312e747-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.110 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:59.111 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:59.112 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:59.112 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:33:59.113 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:33:59.114 DEBUG:boto:Method: GET 2022-09-03T05:33:59.115 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:33:59.116 DEBUG:boto:Data: 2022-09-03T05:33:59.117 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.117 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.118 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.119 DEBUG:boto:Params: {} 2022-09-03T05:33:59.120 DEBUG:boto:Token: None 2022-09-03T05:33:59.120 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/ 2022-09-03T05:33:59.121 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4= 2022-09-03T05:33:59.122 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4=', 'Content-Length': '0'} 2022-09-03T05:33:59.124 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fdcfa7ae20d89a6c-006312e747-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.125 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:59.126 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:33:59.127 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.127 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.128 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.129 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.130 DEBUG:boto:Method: GET 2022-09-03T05:33:59.130 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.131 DEBUG:boto:Data: 2022-09-03T05:33:59.131 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.132 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.133 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.133 DEBUG:boto:Params: {} 2022-09-03T05:33:59.134 DEBUG:boto:Token: None 2022-09-03T05:33:59.135 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.135 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LkPOizJHcLOS/IyoKkKwXbOT5W0= 2022-09-03T05:33:59.136 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LkPOizJHcLOS/IyoKkKwXbOT5W0=', 'Content-Length': '0'} 2022-09-03T05:33:59.138 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000032b702e24f61744a-006312e747-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.139 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.140 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.140 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.141 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.141 DEBUG:boto:Method: GET 2022-09-03T05:33:59.142 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.143 DEBUG:boto:Data: 2022-09-03T05:33:59.144 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.145 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.146 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.147 DEBUG:boto:Params: {} 2022-09-03T05:33:59.148 DEBUG:boto:Token: None 2022-09-03T05:33:59.149 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.151 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LkPOizJHcLOS/IyoKkKwXbOT5W0= 2022-09-03T05:33:59.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LkPOizJHcLOS/IyoKkKwXbOT5W0=', 'Content-Length': '0'} 2022-09-03T05:33:59.154 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c59d5ac4a4f2257d-006312e747-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.155 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.155 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.156 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.157 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.157 DEBUG:boto:Method: HEAD 2022-09-03T05:33:59.158 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.158 DEBUG:boto:Data: 2022-09-03T05:33:59.159 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.159 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.160 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.161 DEBUG:boto:Params: {} 2022-09-03T05:33:59.162 DEBUG:boto:Token: None 2022-09-03T05:33:59.162 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.163 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A7UsWDJbOxFeTqxvjKoZIC4hZeo= 2022-09-03T05:33:59.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A7UsWDJbOxFeTqxvjKoZIC4hZeo=', 'Content-Length': '0'} 2022-09-03T05:33:59.166 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001f0fe46e1c7cf4f4-006312e747-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.167 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.167 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.168 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.168 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.169 DEBUG:boto:Method: HEAD 2022-09-03T05:33:59.169 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.170 DEBUG:boto:Data: 2022-09-03T05:33:59.170 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.171 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.171 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.172 DEBUG:boto:Params: {} 2022-09-03T05:33:59.172 DEBUG:boto:Token: None 2022-09-03T05:33:59.173 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.174 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A7UsWDJbOxFeTqxvjKoZIC4hZeo= 2022-09-03T05:33:59.174 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A7UsWDJbOxFeTqxvjKoZIC4hZeo=', 'Content-Length': '0'} 2022-09-03T05:33:59.176 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000060c9f997d51c57d0-006312e747-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.177 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:33:59.177 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.178 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.179 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.179 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.180 DEBUG:boto:Method: GET 2022-09-03T05:33:59.180 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.181 DEBUG:boto:Data: 2022-09-03T05:33:59.181 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.182 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.182 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.183 DEBUG:boto:Params: {} 2022-09-03T05:33:59.184 DEBUG:boto:Token: None 2022-09-03T05:33:59.184 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.185 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LkPOizJHcLOS/IyoKkKwXbOT5W0= 2022-09-03T05:33:59.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LkPOizJHcLOS/IyoKkKwXbOT5W0=', 'Content-Length': '0'} 2022-09-03T05:33:59.188 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002c620a107f7e0b17-006312e747-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.189 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.189 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.190 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.191 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.191 DEBUG:boto:Method: GET 2022-09-03T05:33:59.192 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.192 DEBUG:boto:Data: 2022-09-03T05:33:59.193 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.193 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.194 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.195 DEBUG:boto:Params: {} 2022-09-03T05:33:59.195 DEBUG:boto:Token: None 2022-09-03T05:33:59.196 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.197 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LkPOizJHcLOS/IyoKkKwXbOT5W0= 2022-09-03T05:33:59.197 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LkPOizJHcLOS/IyoKkKwXbOT5W0=', 'Content-Length': '0'} 2022-09-03T05:33:59.199 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dcfb3554eceb5cba-006312e747-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.200 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:59.201 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:59.201 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:33:59.202 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:33:59.203 DEBUG:boto:Method: GET 2022-09-03T05:33:59.203 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:33:59.204 DEBUG:boto:Data: 2022-09-03T05:33:59.204 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.205 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.205 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.206 DEBUG:boto:Params: {} 2022-09-03T05:33:59.206 DEBUG:boto:Token: None 2022-09-03T05:33:59.207 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/ 2022-09-03T05:33:59.208 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4= 2022-09-03T05:33:59.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4=', 'Content-Length': '0'} 2022-09-03T05:33:59.211 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000010df2eea1585a098-006312e747-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.211 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:59.212 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:59.213 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:59.214 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:33:59.214 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:33:59.215 DEBUG:boto:Method: GET 2022-09-03T05:33:59.215 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:33:59.216 DEBUG:boto:Data: 2022-09-03T05:33:59.216 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.217 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.218 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.218 DEBUG:boto:Params: {} 2022-09-03T05:33:59.219 DEBUG:boto:Token: None 2022-09-03T05:33:59.220 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/ 2022-09-03T05:33:59.220 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4= 2022-09-03T05:33:59.221 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4=', 'Content-Length': '0'} 2022-09-03T05:33:59.223 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d38cbceeab6011ec-006312e747-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.224 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:59.225 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.225 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.226 DEBUG:boto:Method: HEAD 2022-09-03T05:33:59.227 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.227 DEBUG:boto:Data: 2022-09-03T05:33:59.228 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.229 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.229 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.230 DEBUG:boto:Params: {} 2022-09-03T05:33:59.230 DEBUG:boto:Token: None 2022-09-03T05:33:59.231 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.232 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zW4hBX4oJiiz4Akj/uFAvFc3PWo= 2022-09-03T05:33:59.233 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zW4hBX4oJiiz4Akj/uFAvFc3PWo=', 'Content-Length': '0'} 2022-09-03T05:33:59.236 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004f4791618a164299-006312e747-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.237 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.238 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.238 DEBUG:boto:Method: HEAD 2022-09-03T05:33:59.239 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.239 DEBUG:boto:Data: 2022-09-03T05:33:59.240 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.241 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.242 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.242 DEBUG:boto:Params: {} 2022-09-03T05:33:59.243 DEBUG:boto:Token: None 2022-09-03T05:33:59.244 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.244 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zW4hBX4oJiiz4Akj/uFAvFc3PWo= 2022-09-03T05:33:59.245 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zW4hBX4oJiiz4Akj/uFAvFc3PWo=', 'Content-Length': '0'} 2022-09-03T05:33:59.247 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000002cc6047e458afaf-006312e747-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.248 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:33:59.248 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.249 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.249 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.250 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.251 DEBUG:boto:Method: GET 2022-09-03T05:33:59.251 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.252 DEBUG:boto:Data: 2022-09-03T05:33:59.252 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.253 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.253 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.254 DEBUG:boto:Params: {} 2022-09-03T05:33:59.255 DEBUG:boto:Token: None 2022-09-03T05:33:59.255 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.256 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LkPOizJHcLOS/IyoKkKwXbOT5W0= 2022-09-03T05:33:59.257 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LkPOizJHcLOS/IyoKkKwXbOT5W0=', 'Content-Length': '0'} 2022-09-03T05:33:59.259 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e29e7c4c9b6ece33-006312e747-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.307 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.308 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:33:59.308 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.309 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.310 DEBUG:boto:Method: GET 2022-09-03T05:33:59.311 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.312 DEBUG:boto:Data: 2022-09-03T05:33:59.314 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.315 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.316 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.317 DEBUG:boto:Params: {} 2022-09-03T05:33:59.318 DEBUG:boto:Token: None 2022-09-03T05:33:59.319 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:33:59.321 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LkPOizJHcLOS/IyoKkKwXbOT5W0= 2022-09-03T05:33:59.323 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LkPOizJHcLOS/IyoKkKwXbOT5W0=', 'Content-Length': '0'} 2022-09-03T05:33:59.327 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ca908cea85e81b8d-006312e747-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.341 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:59.342 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:59.343 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:33:59.344 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:33:59.345 DEBUG:boto:Method: GET 2022-09-03T05:33:59.346 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:33:59.347 DEBUG:boto:Data: 2022-09-03T05:33:59.347 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.348 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.348 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.349 DEBUG:boto:Params: {} 2022-09-03T05:33:59.350 DEBUG:boto:Token: None 2022-09-03T05:33:59.351 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/ 2022-09-03T05:33:59.352 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4= 2022-09-03T05:33:59.354 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4=', 'Content-Length': '0'} 2022-09-03T05:33:59.357 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c5d51f5bbeaa3e5c-006312e747-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.358 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:59.359 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:59.360 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:59.362 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:33:59.363 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:33:59.364 DEBUG:boto:Method: GET 2022-09-03T05:33:59.365 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:33:59.366 DEBUG:boto:Data: 2022-09-03T05:33:59.367 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.367 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.368 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.369 DEBUG:boto:Params: {} 2022-09-03T05:33:59.370 DEBUG:boto:Token: None 2022-09-03T05:33:59.371 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/ 2022-09-03T05:33:59.372 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4= 2022-09-03T05:33:59.373 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4=', 'Content-Length': '0'} 2022-09-03T05:33:59.376 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c192b9a5096c6fbd-006312e747-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.377 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:59.378 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:33:59.379 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.379 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.380 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.381 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.381 DEBUG:boto:Method: GET 2022-09-03T05:33:59.382 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.382 DEBUG:boto:Data: 2022-09-03T05:33:59.383 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.383 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.384 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.385 DEBUG:boto:Params: {} 2022-09-03T05:33:59.385 DEBUG:boto:Token: None 2022-09-03T05:33:59.386 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.387 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:937qf5B14jtiCAQFQ5ExXUj7HmM= 2022-09-03T05:33:59.387 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:937qf5B14jtiCAQFQ5ExXUj7HmM=', 'Content-Length': '0'} 2022-09-03T05:33:59.389 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b28a27d14e527b62-006312e747-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.390 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.391 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.391 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.392 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.392 DEBUG:boto:Method: GET 2022-09-03T05:33:59.393 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.394 DEBUG:boto:Data: 2022-09-03T05:33:59.394 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.395 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.396 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.397 DEBUG:boto:Params: {} 2022-09-03T05:33:59.398 DEBUG:boto:Token: None 2022-09-03T05:33:59.398 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.399 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:937qf5B14jtiCAQFQ5ExXUj7HmM= 2022-09-03T05:33:59.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:937qf5B14jtiCAQFQ5ExXUj7HmM=', 'Content-Length': '0'} 2022-09-03T05:33:59.401 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001795be7df1c3e443-006312e747-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.402 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.403 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.403 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.404 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.404 DEBUG:boto:Method: HEAD 2022-09-03T05:33:59.405 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.405 DEBUG:boto:Data: 2022-09-03T05:33:59.406 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.406 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.407 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.407 DEBUG:boto:Params: {} 2022-09-03T05:33:59.408 DEBUG:boto:Token: None 2022-09-03T05:33:59.408 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.409 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y79cSaWGRm/8gJuprmDKxeff65E= 2022-09-03T05:33:59.410 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y79cSaWGRm/8gJuprmDKxeff65E=', 'Content-Length': '0'} 2022-09-03T05:33:59.411 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005808e9f7024e674b-006312e747-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.412 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.413 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.413 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.414 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.414 DEBUG:boto:Method: HEAD 2022-09-03T05:33:59.415 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.415 DEBUG:boto:Data: 2022-09-03T05:33:59.416 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.416 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.417 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.417 DEBUG:boto:Params: {} 2022-09-03T05:33:59.418 DEBUG:boto:Token: None 2022-09-03T05:33:59.419 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.419 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y79cSaWGRm/8gJuprmDKxeff65E= 2022-09-03T05:33:59.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y79cSaWGRm/8gJuprmDKxeff65E=', 'Content-Length': '0'} 2022-09-03T05:33:59.422 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cb776ad6ec344857-006312e747-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.423 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:33:59.424 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.424 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.425 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.425 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.426 DEBUG:boto:Method: GET 2022-09-03T05:33:59.426 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.427 DEBUG:boto:Data: 2022-09-03T05:33:59.427 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.428 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.428 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.429 DEBUG:boto:Params: {} 2022-09-03T05:33:59.429 DEBUG:boto:Token: None 2022-09-03T05:33:59.430 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.431 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:937qf5B14jtiCAQFQ5ExXUj7HmM= 2022-09-03T05:33:59.431 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:937qf5B14jtiCAQFQ5ExXUj7HmM=', 'Content-Length': '0'} 2022-09-03T05:33:59.433 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b292c2e0b8283c66-006312e747-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.434 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.435 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.435 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.436 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.436 DEBUG:boto:Method: GET 2022-09-03T05:33:59.437 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.437 DEBUG:boto:Data: 2022-09-03T05:33:59.438 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.439 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.439 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.440 DEBUG:boto:Params: {} 2022-09-03T05:33:59.440 DEBUG:boto:Token: None 2022-09-03T05:33:59.441 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.442 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:937qf5B14jtiCAQFQ5ExXUj7HmM= 2022-09-03T05:33:59.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:937qf5B14jtiCAQFQ5ExXUj7HmM=', 'Content-Length': '0'} 2022-09-03T05:33:59.444 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006ba970f6d3cac313-006312e747-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.445 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:59.446 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:59.446 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:33:59.447 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:33:59.447 DEBUG:boto:Method: GET 2022-09-03T05:33:59.448 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:33:59.448 DEBUG:boto:Data: 2022-09-03T05:33:59.449 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.450 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.450 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.451 DEBUG:boto:Params: {} 2022-09-03T05:33:59.451 DEBUG:boto:Token: None 2022-09-03T05:33:59.452 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/ 2022-09-03T05:33:59.453 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4= 2022-09-03T05:33:59.453 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4=', 'Content-Length': '0'} 2022-09-03T05:33:59.459 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000014802fac8573640a-006312e747-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.461 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:59.462 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:59.463 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:59.463 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:33:59.464 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:33:59.465 DEBUG:boto:Method: GET 2022-09-03T05:33:59.465 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:33:59.466 DEBUG:boto:Data: 2022-09-03T05:33:59.466 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.467 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.467 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.468 DEBUG:boto:Params: {} 2022-09-03T05:33:59.469 DEBUG:boto:Token: None 2022-09-03T05:33:59.470 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/ 2022-09-03T05:33:59.471 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4= 2022-09-03T05:33:59.472 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4=', 'Content-Length': '0'} 2022-09-03T05:33:59.474 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000241f0e58d552176d-006312e747-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.475 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:59.476 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.477 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.478 DEBUG:boto:Method: HEAD 2022-09-03T05:33:59.479 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.480 DEBUG:boto:Data: 2022-09-03T05:33:59.481 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.482 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.483 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.484 DEBUG:boto:Params: {} 2022-09-03T05:33:59.486 DEBUG:boto:Token: None 2022-09-03T05:33:59.487 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.488 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1rRd+KjpFuLXTDw1aBXfFacc57U= 2022-09-03T05:33:59.489 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1rRd+KjpFuLXTDw1aBXfFacc57U=', 'Content-Length': '0'} 2022-09-03T05:33:59.506 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001b544d4989eb516d-006312e747-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.507 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.508 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.508 DEBUG:boto:Method: HEAD 2022-09-03T05:33:59.509 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.509 DEBUG:boto:Data: 2022-09-03T05:33:59.511 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.511 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.512 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.513 DEBUG:boto:Params: {} 2022-09-03T05:33:59.513 DEBUG:boto:Token: None 2022-09-03T05:33:59.514 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.515 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1rRd+KjpFuLXTDw1aBXfFacc57U= 2022-09-03T05:33:59.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1rRd+KjpFuLXTDw1aBXfFacc57U=', 'Content-Length': '0'} 2022-09-03T05:33:59.519 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e6e93d9619f34e59-006312e747-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.520 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:33:59.521 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.522 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.523 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.524 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.524 DEBUG:boto:Method: GET 2022-09-03T05:33:59.525 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.525 DEBUG:boto:Data: 2022-09-03T05:33:59.526 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.527 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.527 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.528 DEBUG:boto:Params: {} 2022-09-03T05:33:59.529 DEBUG:boto:Token: None 2022-09-03T05:33:59.530 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.531 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:937qf5B14jtiCAQFQ5ExXUj7HmM= 2022-09-03T05:33:59.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:937qf5B14jtiCAQFQ5ExXUj7HmM=', 'Content-Length': '0'} 2022-09-03T05:33:59.534 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005d309e59d1173a33-006312e747-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.538 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.539 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:33:59.539 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.540 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.540 DEBUG:boto:Method: GET 2022-09-03T05:33:59.541 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.542 DEBUG:boto:Data: 2022-09-03T05:33:59.543 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.544 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.545 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.546 DEBUG:boto:Params: {} 2022-09-03T05:33:59.547 DEBUG:boto:Token: None 2022-09-03T05:33:59.548 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:33:59.549 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:937qf5B14jtiCAQFQ5ExXUj7HmM= 2022-09-03T05:33:59.550 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:937qf5B14jtiCAQFQ5ExXUj7HmM=', 'Content-Length': '0'} 2022-09-03T05:33:59.552 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000967d6ed5affbf1c4-006312e747-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.553 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:59.553 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:59.554 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:33:59.555 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:33:59.556 DEBUG:boto:Method: GET 2022-09-03T05:33:59.557 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:33:59.558 DEBUG:boto:Data: 2022-09-03T05:33:59.559 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.560 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.561 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.562 DEBUG:boto:Params: {} 2022-09-03T05:33:59.563 DEBUG:boto:Token: None 2022-09-03T05:33:59.564 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/ 2022-09-03T05:33:59.565 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4= 2022-09-03T05:33:59.565 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4=', 'Content-Length': '0'} 2022-09-03T05:33:59.568 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001a51110597255a61-006312e747-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.569 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:59.570 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:59.570 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:59.571 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:33:59.572 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:33:59.572 DEBUG:boto:Method: GET 2022-09-03T05:33:59.573 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:33:59.573 DEBUG:boto:Data: 2022-09-03T05:33:59.574 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.575 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.575 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.576 DEBUG:boto:Params: {} 2022-09-03T05:33:59.576 DEBUG:boto:Token: None 2022-09-03T05:33:59.577 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/ 2022-09-03T05:33:59.578 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4= 2022-09-03T05:33:59.579 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4=', 'Content-Length': '0'} 2022-09-03T05:33:59.582 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000012977ba76410c9a3-006312e747-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.583 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:59.585 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:33:59.586 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.587 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.588 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.588 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.589 DEBUG:boto:Method: GET 2022-09-03T05:33:59.589 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.590 DEBUG:boto:Data: 2022-09-03T05:33:59.590 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.591 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.592 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.592 DEBUG:boto:Params: {} 2022-09-03T05:33:59.593 DEBUG:boto:Token: None 2022-09-03T05:33:59.593 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.594 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t3YouFwhlfkZxKKA5wMgruHdXQo= 2022-09-03T05:33:59.595 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t3YouFwhlfkZxKKA5wMgruHdXQo=', 'Content-Length': '0'} 2022-09-03T05:33:59.597 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000020fae8576d6c5345-006312e747-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.598 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.598 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.599 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.599 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.600 DEBUG:boto:Method: GET 2022-09-03T05:33:59.600 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.601 DEBUG:boto:Data: 2022-09-03T05:33:59.602 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.602 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.603 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.603 DEBUG:boto:Params: {} 2022-09-03T05:33:59.604 DEBUG:boto:Token: None 2022-09-03T05:33:59.605 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.605 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t3YouFwhlfkZxKKA5wMgruHdXQo= 2022-09-03T05:33:59.606 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t3YouFwhlfkZxKKA5wMgruHdXQo=', 'Content-Length': '0'} 2022-09-03T05:33:59.608 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000072e3555d0a6701f8-006312e747-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.650 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.651 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.652 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.652 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.653 DEBUG:boto:Method: HEAD 2022-09-03T05:33:59.653 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.654 DEBUG:boto:Data: 2022-09-03T05:33:59.654 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.655 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.655 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.656 DEBUG:boto:Params: {} 2022-09-03T05:33:59.656 DEBUG:boto:Token: None 2022-09-03T05:33:59.657 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.657 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MqMH+RjhxtJJpEjagg4seolRpWk= 2022-09-03T05:33:59.658 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MqMH+RjhxtJJpEjagg4seolRpWk=', 'Content-Length': '0'} 2022-09-03T05:33:59.660 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000950cede0337908d3-006312e747-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.661 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.662 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.663 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.663 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.664 DEBUG:boto:Method: HEAD 2022-09-03T05:33:59.665 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.665 DEBUG:boto:Data: 2022-09-03T05:33:59.666 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.666 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.667 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.668 DEBUG:boto:Params: {} 2022-09-03T05:33:59.668 DEBUG:boto:Token: None 2022-09-03T05:33:59.669 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.670 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MqMH+RjhxtJJpEjagg4seolRpWk= 2022-09-03T05:33:59.670 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MqMH+RjhxtJJpEjagg4seolRpWk=', 'Content-Length': '0'} 2022-09-03T05:33:59.672 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dbe9550877f96004-006312e747-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.673 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:33:59.674 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.675 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.675 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.676 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.677 DEBUG:boto:Method: GET 2022-09-03T05:33:59.677 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.678 DEBUG:boto:Data: 2022-09-03T05:33:59.678 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.679 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.679 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.680 DEBUG:boto:Params: {} 2022-09-03T05:33:59.681 DEBUG:boto:Token: None 2022-09-03T05:33:59.682 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.683 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t3YouFwhlfkZxKKA5wMgruHdXQo= 2022-09-03T05:33:59.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t3YouFwhlfkZxKKA5wMgruHdXQo=', 'Content-Length': '0'} 2022-09-03T05:33:59.686 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000005d34abdce52dae6-006312e747-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.688 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.689 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.690 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.690 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.691 DEBUG:boto:Method: GET 2022-09-03T05:33:59.692 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.693 DEBUG:boto:Data: 2022-09-03T05:33:59.694 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.694 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.695 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.696 DEBUG:boto:Params: {} 2022-09-03T05:33:59.696 DEBUG:boto:Token: None 2022-09-03T05:33:59.697 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.698 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t3YouFwhlfkZxKKA5wMgruHdXQo= 2022-09-03T05:33:59.698 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t3YouFwhlfkZxKKA5wMgruHdXQo=', 'Content-Length': '0'} 2022-09-03T05:33:59.700 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000641ce35f38b50564-006312e747-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.701 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:59.702 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:59.702 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:33:59.703 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:33:59.703 DEBUG:boto:Method: GET 2022-09-03T05:33:59.704 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:33:59.704 DEBUG:boto:Data: 2022-09-03T05:33:59.705 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.705 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.706 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.707 DEBUG:boto:Params: {} 2022-09-03T05:33:59.707 DEBUG:boto:Token: None 2022-09-03T05:33:59.708 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/ 2022-09-03T05:33:59.708 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4= 2022-09-03T05:33:59.709 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4=', 'Content-Length': '0'} 2022-09-03T05:33:59.711 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001169a58a183fcb66-006312e747-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.754 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:59.756 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:59.757 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:59.758 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:33:59.759 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:33:59.759 DEBUG:boto:Method: GET 2022-09-03T05:33:59.760 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:33:59.761 DEBUG:boto:Data: 2022-09-03T05:33:59.761 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.762 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.763 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.763 DEBUG:boto:Params: {} 2022-09-03T05:33:59.764 DEBUG:boto:Token: None 2022-09-03T05:33:59.764 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/ 2022-09-03T05:33:59.766 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4= 2022-09-03T05:33:59.767 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4=', 'Content-Length': '0'} 2022-09-03T05:33:59.769 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d5ddb056436760b-006312e747-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.770 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:59.771 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.772 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.772 DEBUG:boto:Method: HEAD 2022-09-03T05:33:59.773 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.773 DEBUG:boto:Data: 2022-09-03T05:33:59.774 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.774 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.775 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.775 DEBUG:boto:Params: {} 2022-09-03T05:33:59.776 DEBUG:boto:Token: None 2022-09-03T05:33:59.777 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.777 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8kS1gjY6Lo2HnXpGvhUIFDC66w8= 2022-09-03T05:33:59.778 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8kS1gjY6Lo2HnXpGvhUIFDC66w8=', 'Content-Length': '0'} 2022-09-03T05:33:59.781 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000b7dd4bfb63dcb4a-006312e747-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.782 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.783 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.783 DEBUG:boto:Method: HEAD 2022-09-03T05:33:59.784 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.784 DEBUG:boto:Data: 2022-09-03T05:33:59.785 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.785 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.786 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.786 DEBUG:boto:Params: {} 2022-09-03T05:33:59.787 DEBUG:boto:Token: None 2022-09-03T05:33:59.787 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.788 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8kS1gjY6Lo2HnXpGvhUIFDC66w8= 2022-09-03T05:33:59.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8kS1gjY6Lo2HnXpGvhUIFDC66w8=', 'Content-Length': '0'} 2022-09-03T05:33:59.791 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000b35f6d15ae6da35-006312e747-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.792 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:33:59.793 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.794 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.794 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.795 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.796 DEBUG:boto:Method: GET 2022-09-03T05:33:59.796 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.797 DEBUG:boto:Data: 2022-09-03T05:33:59.797 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.798 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.798 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.799 DEBUG:boto:Params: {} 2022-09-03T05:33:59.800 DEBUG:boto:Token: None 2022-09-03T05:33:59.800 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.801 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t3YouFwhlfkZxKKA5wMgruHdXQo= 2022-09-03T05:33:59.802 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t3YouFwhlfkZxKKA5wMgruHdXQo=', 'Content-Length': '0'} 2022-09-03T05:33:59.805 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000179564c5c3c0cd13-006312e747-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.806 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.807 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:33:59.808 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.809 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.810 DEBUG:boto:Method: GET 2022-09-03T05:33:59.811 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.812 DEBUG:boto:Data: 2022-09-03T05:33:59.813 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.813 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.814 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.815 DEBUG:boto:Params: {} 2022-09-03T05:33:59.815 DEBUG:boto:Token: None 2022-09-03T05:33:59.816 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:33:59.817 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t3YouFwhlfkZxKKA5wMgruHdXQo= 2022-09-03T05:33:59.818 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t3YouFwhlfkZxKKA5wMgruHdXQo=', 'Content-Length': '0'} 2022-09-03T05:33:59.820 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000715b2b3a6b7cb4d-006312e747-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.821 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:59.821 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:59.822 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:33:59.823 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:33:59.823 DEBUG:boto:Method: GET 2022-09-03T05:33:59.824 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:33:59.825 DEBUG:boto:Data: 2022-09-03T05:33:59.825 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.826 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:33:59.826 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.827 DEBUG:boto:Params: {} 2022-09-03T05:33:59.828 DEBUG:boto:Token: None 2022-09-03T05:33:59.828 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/ 2022-09-03T05:33:59.829 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4= 2022-09-03T05:33:59.829 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4=', 'Content-Length': '0'} 2022-09-03T05:33:59.832 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000839fd28bd4f29160-006312e747-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.833 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:59.833 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:59.834 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:59.835 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:33:59.836 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:33:59.836 DEBUG:boto:Method: GET 2022-09-03T05:33:59.837 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:33:59.838 DEBUG:boto:Data: 2022-09-03T05:33:59.838 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.839 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.840 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.840 DEBUG:boto:Params: {} 2022-09-03T05:33:59.841 DEBUG:boto:Token: None 2022-09-03T05:33:59.842 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/ 2022-09-03T05:33:59.843 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4= 2022-09-03T05:33:59.843 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SNWcyKMGolt/D9zHI2aXoSEaRd4=', 'Content-Length': '0'} 2022-09-03T05:33:59.846 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000021fc8b690e57ad82-006312e747-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.847 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:59.848 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-18 zones={test-zone2, test-zone1} 2022-09-03T05:33:59.849 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-18 zones={test-zone3, test-zone1} 2022-09-03T05:33:59.850 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:59.851 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:59.851 DEBUG:boto:Method: HEAD 2022-09-03T05:33:59.852 DEBUG:boto:Path: /dsgvcy-18/ 2022-09-03T05:33:59.852 DEBUG:boto:Data: 2022-09-03T05:33:59.853 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.853 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:59.854 DEBUG:boto:Port: 8001 2022-09-03T05:33:59.854 DEBUG:boto:Params: {} 2022-09-03T05:33:59.855 DEBUG:boto:Token: None 2022-09-03T05:33:59.856 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/ 2022-09-03T05:33:59.856 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:643FKp4uC2K0HBZJ63nHnaKFiTw= 2022-09-03T05:33:59.857 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:643FKp4uC2K0HBZJ63nHnaKFiTw=', 'Content-Length': '0'} 2022-09-03T05:33:59.859 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fc7519af66021052-006312e747-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.860 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:59.861 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:59.861 DEBUG:boto:Method: HEAD 2022-09-03T05:33:59.862 DEBUG:boto:Path: /dsgvcy-18/ 2022-09-03T05:33:59.863 DEBUG:boto:Data: 2022-09-03T05:33:59.863 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.864 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.864 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.864 DEBUG:boto:Params: {} 2022-09-03T05:33:59.865 DEBUG:boto:Token: None 2022-09-03T05:33:59.866 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/ 2022-09-03T05:33:59.866 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:643FKp4uC2K0HBZJ63nHnaKFiTw= 2022-09-03T05:33:59.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:643FKp4uC2K0HBZJ63nHnaKFiTw=', 'Content-Length': '0'} 2022-09-03T05:33:59.869 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c95e8f6718ba87d0-006312e747-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.870 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:33:59.871 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:59.872 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:59.873 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:59.874 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:59.875 DEBUG:boto:Method: GET 2022-09-03T05:33:59.875 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:59.876 DEBUG:boto:Data: 2022-09-03T05:33:59.877 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.878 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:59.878 DEBUG:boto:Port: 8001 2022-09-03T05:33:59.879 DEBUG:boto:Params: {} 2022-09-03T05:33:59.880 DEBUG:boto:Token: None 2022-09-03T05:33:59.881 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/?versions 2022-09-03T05:33:59.882 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qXsdYnNql5ZuRiutPoywRWvJiZg= 2022-09-03T05:33:59.882 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qXsdYnNql5ZuRiutPoywRWvJiZg=', 'Content-Length': '0'} 2022-09-03T05:33:59.885 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b69672abc42df232-006312e747-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.885 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:59.886 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:33:59.887 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:33:59.887 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:33:59.888 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:33:59.888 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:33:59.889 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:59.890 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:59.890 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:59.891 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:59.892 DEBUG:boto:Method: GET 2022-09-03T05:33:59.892 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:59.893 DEBUG:boto:Data: 2022-09-03T05:33:59.893 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.894 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.895 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.895 DEBUG:boto:Params: {} 2022-09-03T05:33:59.896 DEBUG:boto:Token: None 2022-09-03T05:33:59.897 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/?versions 2022-09-03T05:33:59.898 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qXsdYnNql5ZuRiutPoywRWvJiZg= 2022-09-03T05:33:59.899 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qXsdYnNql5ZuRiutPoywRWvJiZg=', 'Content-Length': '0'} 2022-09-03T05:33:59.902 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000044800c15533389ed-006312e747-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.903 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:59.904 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:33:59.905 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:33:59.905 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:33:59.906 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:33:59.906 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:59.907 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:59.907 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:59.908 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:59.908 DEBUG:boto:Method: GET 2022-09-03T05:33:59.909 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:59.909 DEBUG:boto:Data: 2022-09-03T05:33:59.910 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.910 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:59.911 DEBUG:boto:Port: 8001 2022-09-03T05:33:59.912 DEBUG:boto:Params: {} 2022-09-03T05:33:59.913 DEBUG:boto:Token: None 2022-09-03T05:33:59.914 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/?versions 2022-09-03T05:33:59.914 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qXsdYnNql5ZuRiutPoywRWvJiZg= 2022-09-03T05:33:59.915 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qXsdYnNql5ZuRiutPoywRWvJiZg=', 'Content-Length': '0'} 2022-09-03T05:33:59.917 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c9694a4e99790546-006312e747-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.918 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:59.919 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:33:59.920 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:33:59.921 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:59.921 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:59.922 DEBUG:boto:Method: GET 2022-09-03T05:33:59.922 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:33:59.923 DEBUG:boto:Data: 2022-09-03T05:33:59.923 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.924 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.925 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.926 DEBUG:boto:Params: {} 2022-09-03T05:33:59.927 DEBUG:boto:Token: None 2022-09-03T05:33:59.928 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/?versions 2022-09-03T05:33:59.929 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qXsdYnNql5ZuRiutPoywRWvJiZg= 2022-09-03T05:33:59.930 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qXsdYnNql5ZuRiutPoywRWvJiZg=', 'Content-Length': '0'} 2022-09-03T05:33:59.933 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000016c7c2e8ce7beb5e-006312e747-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.934 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:33:59.935 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:33:59.936 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:33:59.936 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:33:59.937 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.937 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.938 DEBUG:boto:Method: GET 2022-09-03T05:33:59.939 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.939 DEBUG:boto:Data: 2022-09-03T05:33:59.940 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.941 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:59.942 DEBUG:boto:Port: 8001 2022-09-03T05:33:59.943 DEBUG:boto:Params: {} 2022-09-03T05:33:59.944 DEBUG:boto:Token: None 2022-09-03T05:33:59.944 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.945 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1Q2wPCaU5vRiaPx28/SqRYZ4Yo0= 2022-09-03T05:33:59.947 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1Q2wPCaU5vRiaPx28/SqRYZ4Yo0=', 'Content-Length': '0'} 2022-09-03T05:33:59.949 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000243c8f4a76d53380-006312e747-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.951 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:33:59.952 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:33:59.953 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.954 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.954 DEBUG:boto:Method: GET 2022-09-03T05:33:59.955 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.955 DEBUG:boto:Data: 2022-09-03T05:33:59.956 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.957 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.957 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.958 DEBUG:boto:Params: {} 2022-09-03T05:33:59.959 DEBUG:boto:Token: None 2022-09-03T05:33:59.959 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.960 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1Q2wPCaU5vRiaPx28/SqRYZ4Yo0= 2022-09-03T05:33:59.961 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1Q2wPCaU5vRiaPx28/SqRYZ4Yo0=', 'Content-Length': '0'} 2022-09-03T05:33:59.964 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000059045d23f6edba4f-006312e747-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.965 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:33:59.965 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:33:59.966 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.966 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.967 DEBUG:boto:Method: HEAD 2022-09-03T05:33:59.967 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.968 DEBUG:boto:Data: 2022-09-03T05:33:59.969 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.969 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:59.970 DEBUG:boto:Port: 8001 2022-09-03T05:33:59.970 DEBUG:boto:Params: {} 2022-09-03T05:33:59.971 DEBUG:boto:Token: None 2022-09-03T05:33:59.972 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.973 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7klbYJQS2aBQ2wuI4gYA2sX7aYU= 2022-09-03T05:33:59.974 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7klbYJQS2aBQ2wuI4gYA2sX7aYU=', 'Content-Length': '0'} 2022-09-03T05:33:59.976 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000001fa7130f250cd319-006312e747-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.976 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:33:59.977 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:33:59.977 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.978 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.979 DEBUG:boto:Method: HEAD 2022-09-03T05:33:59.979 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.980 DEBUG:boto:Data: 2022-09-03T05:33:59.980 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.981 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:33:59.981 DEBUG:boto:Port: 8000 2022-09-03T05:33:59.982 DEBUG:boto:Params: {} 2022-09-03T05:33:59.982 DEBUG:boto:Token: None 2022-09-03T05:33:59.983 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.983 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7klbYJQS2aBQ2wuI4gYA2sX7aYU= 2022-09-03T05:33:59.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7klbYJQS2aBQ2wuI4gYA2sX7aYU=', 'Content-Length': '0'} 2022-09-03T05:33:59.986 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000001261e135668ad51-006312e747-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.987 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:33:59.987 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:33:59.988 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:33:59.988 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.989 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.990 DEBUG:boto:Method: GET 2022-09-03T05:33:59.990 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.991 DEBUG:boto:Data: 2022-09-03T05:33:59.991 DEBUG:boto:Headers: {} 2022-09-03T05:33:59.992 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:33:59.992 DEBUG:boto:Port: 8001 2022-09-03T05:33:59.993 DEBUG:boto:Params: {} 2022-09-03T05:33:59.993 DEBUG:boto:Token: None 2022-09-03T05:33:59.994 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:33:59 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:33:59.995 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1Q2wPCaU5vRiaPx28/SqRYZ4Yo0= 2022-09-03T05:33:59.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:33:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1Q2wPCaU5vRiaPx28/SqRYZ4Yo0=', 'Content-Length': '0'} 2022-09-03T05:33:59.998 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000355e016e07e86dd5-006312e747-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:33:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:33:59.998 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:33:59.999 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:34:00.000 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:00.000 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:00.001 DEBUG:boto:Method: GET 2022-09-03T05:34:00.001 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:00.002 DEBUG:boto:Data: 2022-09-03T05:34:00.002 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.003 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.003 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.004 DEBUG:boto:Params: {} 2022-09-03T05:34:00.005 DEBUG:boto:Token: None 2022-09-03T05:34:00.005 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:00.006 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M2qL0eIGYTv8UI9auOHlaXI1wp4= 2022-09-03T05:34:00.007 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M2qL0eIGYTv8UI9auOHlaXI1wp4=', 'Content-Length': '0'} 2022-09-03T05:34:00.008 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a04c7b62f941c395-006312e748-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.009 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:00.010 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:00.010 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:34:00.011 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:34:00.011 DEBUG:boto:Method: GET 2022-09-03T05:34:00.012 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:34:00.012 DEBUG:boto:Data: 2022-09-03T05:34:00.013 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.014 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.014 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.015 DEBUG:boto:Params: {} 2022-09-03T05:34:00.015 DEBUG:boto:Token: None 2022-09-03T05:34:00.016 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/ 2022-09-03T05:34:00.017 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA= 2022-09-03T05:34:00.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA=', 'Content-Length': '0'} 2022-09-03T05:34:00.020 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e9bed8b7ba144d70-006312e748-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.066 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:00.067 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:00.068 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:00.068 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:34:00.069 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:34:00.070 DEBUG:boto:Method: GET 2022-09-03T05:34:00.070 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:34:00.071 DEBUG:boto:Data: 2022-09-03T05:34:00.072 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.072 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.073 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.073 DEBUG:boto:Params: {} 2022-09-03T05:34:00.074 DEBUG:boto:Token: None 2022-09-03T05:34:00.075 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/ 2022-09-03T05:34:00.075 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA= 2022-09-03T05:34:00.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA=', 'Content-Length': '0'} 2022-09-03T05:34:00.079 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001070b92def5863cf-006312e748-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.079 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:00.080 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:34:00.081 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:34:00.081 DEBUG:boto:Method: HEAD 2022-09-03T05:34:00.082 DEBUG:boto:Path: /dsgvcy-18/dummy 2022-09-03T05:34:00.082 DEBUG:boto:Data: 2022-09-03T05:34:00.083 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.083 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.084 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.084 DEBUG:boto:Params: {} 2022-09-03T05:34:00.085 DEBUG:boto:Token: None 2022-09-03T05:34:00.086 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/dummy 2022-09-03T05:34:00.086 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nNoWrUBYeJw2+dKsjKKDqbGTRes= 2022-09-03T05:34:00.087 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nNoWrUBYeJw2+dKsjKKDqbGTRes=', 'Content-Length': '0'} 2022-09-03T05:34:00.089 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a0d18f0138c38b6f-006312e748-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.090 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:34:00.091 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:34:00.091 DEBUG:boto:Method: HEAD 2022-09-03T05:34:00.092 DEBUG:boto:Path: /dsgvcy-18/dummy 2022-09-03T05:34:00.092 DEBUG:boto:Data: 2022-09-03T05:34:00.093 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.093 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.094 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.095 DEBUG:boto:Params: {} 2022-09-03T05:34:00.095 DEBUG:boto:Token: None 2022-09-03T05:34:00.096 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/dummy 2022-09-03T05:34:00.097 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nNoWrUBYeJw2+dKsjKKDqbGTRes= 2022-09-03T05:34:00.097 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nNoWrUBYeJw2+dKsjKKDqbGTRes=', 'Content-Length': '0'} 2022-09-03T05:34:00.100 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000082d325a4dd4e60c8-006312e748-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.100 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:34:00.101 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:34:00.102 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:34:00.102 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:00.103 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:00.103 DEBUG:boto:Method: GET 2022-09-03T05:34:00.104 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:00.105 DEBUG:boto:Data: 2022-09-03T05:34:00.105 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.106 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.106 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.107 DEBUG:boto:Params: {} 2022-09-03T05:34:00.108 DEBUG:boto:Token: None 2022-09-03T05:34:00.108 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:00.109 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M2qL0eIGYTv8UI9auOHlaXI1wp4= 2022-09-03T05:34:00.110 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M2qL0eIGYTv8UI9auOHlaXI1wp4=', 'Content-Length': '0'} 2022-09-03T05:34:00.111 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000048b42ebb38008439-006312e748-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.112 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:34:00.113 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:34:00.113 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:00.114 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:00.115 DEBUG:boto:Method: GET 2022-09-03T05:34:00.115 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:00.116 DEBUG:boto:Data: 2022-09-03T05:34:00.116 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.117 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.117 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.118 DEBUG:boto:Params: {} 2022-09-03T05:34:00.119 DEBUG:boto:Token: None 2022-09-03T05:34:00.119 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:00.120 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M2qL0eIGYTv8UI9auOHlaXI1wp4= 2022-09-03T05:34:00.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M2qL0eIGYTv8UI9auOHlaXI1wp4=', 'Content-Length': '0'} 2022-09-03T05:34:00.123 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000092f6361378d9fe6c-006312e748-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.124 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:00.124 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:00.125 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:34:00.125 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:34:00.126 DEBUG:boto:Method: GET 2022-09-03T05:34:00.126 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:34:00.127 DEBUG:boto:Data: 2022-09-03T05:34:00.128 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.128 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.129 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.129 DEBUG:boto:Params: {} 2022-09-03T05:34:00.130 DEBUG:boto:Token: None 2022-09-03T05:34:00.131 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/ 2022-09-03T05:34:00.131 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA= 2022-09-03T05:34:00.132 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA=', 'Content-Length': '0'} 2022-09-03T05:34:00.134 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000076e91bd328a107da-006312e748-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.135 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:00.137 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:00.138 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:00.139 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:34:00.140 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:34:00.141 DEBUG:boto:Method: GET 2022-09-03T05:34:00.142 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:34:00.143 DEBUG:boto:Data: 2022-09-03T05:34:00.145 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.146 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.147 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.148 DEBUG:boto:Params: {} 2022-09-03T05:34:00.148 DEBUG:boto:Token: None 2022-09-03T05:34:00.149 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/ 2022-09-03T05:34:00.150 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA= 2022-09-03T05:34:00.150 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA=', 'Content-Length': '0'} 2022-09-03T05:34:00.153 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004fc92891ffe9c8a3-006312e748-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.154 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:00.155 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:34:00.155 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.156 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.156 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.157 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.158 DEBUG:boto:Method: GET 2022-09-03T05:34:00.158 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.159 DEBUG:boto:Data: 2022-09-03T05:34:00.159 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.160 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.161 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.161 DEBUG:boto:Params: {} 2022-09-03T05:34:00.162 DEBUG:boto:Token: None 2022-09-03T05:34:00.162 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.163 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0C3jPTBmyByTnaHkW3Jopyy/2/Y= 2022-09-03T05:34:00.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0C3jPTBmyByTnaHkW3Jopyy/2/Y=', 'Content-Length': '0'} 2022-09-03T05:34:00.166 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009470b6d01623abed-006312e748-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.167 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.168 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.169 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.170 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.170 DEBUG:boto:Method: GET 2022-09-03T05:34:00.171 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.172 DEBUG:boto:Data: 2022-09-03T05:34:00.172 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.173 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.174 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.174 DEBUG:boto:Params: {} 2022-09-03T05:34:00.175 DEBUG:boto:Token: None 2022-09-03T05:34:00.176 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.176 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0C3jPTBmyByTnaHkW3Jopyy/2/Y= 2022-09-03T05:34:00.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0C3jPTBmyByTnaHkW3Jopyy/2/Y=', 'Content-Length': '0'} 2022-09-03T05:34:00.179 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f0b12343b8438508-006312e748-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.180 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.181 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.181 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.182 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.182 DEBUG:boto:Method: HEAD 2022-09-03T05:34:00.183 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.183 DEBUG:boto:Data: 2022-09-03T05:34:00.184 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.184 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.185 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.185 DEBUG:boto:Params: {} 2022-09-03T05:34:00.186 DEBUG:boto:Token: None 2022-09-03T05:34:00.186 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.187 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O1jqTmCgPeqrLFxuGqXK591V7pU= 2022-09-03T05:34:00.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O1jqTmCgPeqrLFxuGqXK591V7pU=', 'Content-Length': '0'} 2022-09-03T05:34:00.190 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005db0add4fe07a544-006312e748-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.191 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.191 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.192 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.192 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.193 DEBUG:boto:Method: HEAD 2022-09-03T05:34:00.193 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.194 DEBUG:boto:Data: 2022-09-03T05:34:00.195 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.196 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.197 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.198 DEBUG:boto:Params: {} 2022-09-03T05:34:00.199 DEBUG:boto:Token: None 2022-09-03T05:34:00.200 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.201 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O1jqTmCgPeqrLFxuGqXK591V7pU= 2022-09-03T05:34:00.202 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O1jqTmCgPeqrLFxuGqXK591V7pU=', 'Content-Length': '0'} 2022-09-03T05:34:00.203 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000903c570a91fde806-006312e748-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.204 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:34:00.205 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.205 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.206 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.206 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.207 DEBUG:boto:Method: GET 2022-09-03T05:34:00.207 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.208 DEBUG:boto:Data: 2022-09-03T05:34:00.209 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.209 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.210 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.210 DEBUG:boto:Params: {} 2022-09-03T05:34:00.211 DEBUG:boto:Token: None 2022-09-03T05:34:00.211 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.212 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0C3jPTBmyByTnaHkW3Jopyy/2/Y= 2022-09-03T05:34:00.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0C3jPTBmyByTnaHkW3Jopyy/2/Y=', 'Content-Length': '0'} 2022-09-03T05:34:00.215 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006428398d091578bc-006312e748-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.258 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.259 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.260 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.260 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.261 DEBUG:boto:Method: GET 2022-09-03T05:34:00.262 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.263 DEBUG:boto:Data: 2022-09-03T05:34:00.263 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.264 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.264 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.265 DEBUG:boto:Params: {} 2022-09-03T05:34:00.266 DEBUG:boto:Token: None 2022-09-03T05:34:00.266 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.267 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0C3jPTBmyByTnaHkW3Jopyy/2/Y= 2022-09-03T05:34:00.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0C3jPTBmyByTnaHkW3Jopyy/2/Y=', 'Content-Length': '0'} 2022-09-03T05:34:00.270 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000003fb48553ca6ad4d-006312e748-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.271 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:00.272 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:00.272 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:00.273 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:00.274 DEBUG:boto:Method: GET 2022-09-03T05:34:00.274 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:00.275 DEBUG:boto:Data: 2022-09-03T05:34:00.276 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.276 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.277 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.277 DEBUG:boto:Params: {} 2022-09-03T05:34:00.278 DEBUG:boto:Token: None 2022-09-03T05:34:00.279 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/ 2022-09-03T05:34:00.279 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA= 2022-09-03T05:34:00.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA=', 'Content-Length': '0'} 2022-09-03T05:34:00.283 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fc0ebe6c67b8658b-006312e748-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.284 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:00.286 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:00.287 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:00.288 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:00.290 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:00.291 DEBUG:boto:Method: GET 2022-09-03T05:34:00.292 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:00.293 DEBUG:boto:Data: 2022-09-03T05:34:00.294 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.295 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.296 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.296 DEBUG:boto:Params: {} 2022-09-03T05:34:00.297 DEBUG:boto:Token: None 2022-09-03T05:34:00.298 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/ 2022-09-03T05:34:00.299 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA= 2022-09-03T05:34:00.299 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA=', 'Content-Length': '0'} 2022-09-03T05:34:00.302 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000c4d423c173a279c-006312e748-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.303 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:00.303 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.304 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.304 DEBUG:boto:Method: HEAD 2022-09-03T05:34:00.305 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.306 DEBUG:boto:Data: 2022-09-03T05:34:00.306 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.307 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.307 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.308 DEBUG:boto:Params: {} 2022-09-03T05:34:00.308 DEBUG:boto:Token: None 2022-09-03T05:34:00.309 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.310 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oapYKXtcCZ7CKjde1e3/V2V7SGw= 2022-09-03T05:34:00.311 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oapYKXtcCZ7CKjde1e3/V2V7SGw=', 'Content-Length': '0'} 2022-09-03T05:34:00.313 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000030eb3fc885a4beee-006312e748-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.314 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.314 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.315 DEBUG:boto:Method: HEAD 2022-09-03T05:34:00.315 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.316 DEBUG:boto:Data: 2022-09-03T05:34:00.316 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.317 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.317 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.318 DEBUG:boto:Params: {} 2022-09-03T05:34:00.319 DEBUG:boto:Token: None 2022-09-03T05:34:00.319 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.320 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oapYKXtcCZ7CKjde1e3/V2V7SGw= 2022-09-03T05:34:00.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oapYKXtcCZ7CKjde1e3/V2V7SGw=', 'Content-Length': '0'} 2022-09-03T05:34:00.323 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000088badcbea4fba25b-006312e748-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.324 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:34:00.325 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.325 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.326 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.326 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.327 DEBUG:boto:Method: GET 2022-09-03T05:34:00.328 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.328 DEBUG:boto:Data: 2022-09-03T05:34:00.329 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.329 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.330 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.330 DEBUG:boto:Params: {} 2022-09-03T05:34:00.331 DEBUG:boto:Token: None 2022-09-03T05:34:00.332 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.332 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0C3jPTBmyByTnaHkW3Jopyy/2/Y= 2022-09-03T05:34:00.333 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0C3jPTBmyByTnaHkW3Jopyy/2/Y=', 'Content-Length': '0'} 2022-09-03T05:34:00.335 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000be2d2580134068e8-006312e748-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.336 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.337 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:00.337 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.338 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.339 DEBUG:boto:Method: GET 2022-09-03T05:34:00.339 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.340 DEBUG:boto:Data: 2022-09-03T05:34:00.341 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.342 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.343 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.344 DEBUG:boto:Params: {} 2022-09-03T05:34:00.345 DEBUG:boto:Token: None 2022-09-03T05:34:00.345 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:00.346 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0C3jPTBmyByTnaHkW3Jopyy/2/Y= 2022-09-03T05:34:00.347 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0C3jPTBmyByTnaHkW3Jopyy/2/Y=', 'Content-Length': '0'} 2022-09-03T05:34:00.350 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a90f50fab4e05282-006312e748-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.394 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:00.396 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:00.397 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:00.398 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:00.399 DEBUG:boto:Method: GET 2022-09-03T05:34:00.400 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:00.400 DEBUG:boto:Data: 2022-09-03T05:34:00.401 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.401 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.402 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.403 DEBUG:boto:Params: {} 2022-09-03T05:34:00.404 DEBUG:boto:Token: None 2022-09-03T05:34:00.405 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/ 2022-09-03T05:34:00.406 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA= 2022-09-03T05:34:00.406 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA=', 'Content-Length': '0'} 2022-09-03T05:34:00.409 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000016253396f0278703-006312e748-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.409 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:00.411 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:00.411 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:00.412 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:00.412 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:00.413 DEBUG:boto:Method: GET 2022-09-03T05:34:00.414 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:00.414 DEBUG:boto:Data: 2022-09-03T05:34:00.415 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.416 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.416 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.417 DEBUG:boto:Params: {} 2022-09-03T05:34:00.418 DEBUG:boto:Token: None 2022-09-03T05:34:00.418 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/ 2022-09-03T05:34:00.419 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA= 2022-09-03T05:34:00.419 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA=', 'Content-Length': '0'} 2022-09-03T05:34:00.422 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dd54691eac34b37a-006312e748-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.423 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:00.423 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:34:00.424 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.425 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.425 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.426 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.426 DEBUG:boto:Method: GET 2022-09-03T05:34:00.427 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.428 DEBUG:boto:Data: 2022-09-03T05:34:00.428 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.429 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.430 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.430 DEBUG:boto:Params: {} 2022-09-03T05:34:00.431 DEBUG:boto:Token: None 2022-09-03T05:34:00.431 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.433 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TGX68AkoPUVSYALSJ4aQQ7DOLPo= 2022-09-03T05:34:00.433 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TGX68AkoPUVSYALSJ4aQQ7DOLPo=', 'Content-Length': '0'} 2022-09-03T05:34:00.439 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000a9e98d23a0bef3e-006312e748-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.444 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.445 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.447 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.448 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.449 DEBUG:boto:Method: GET 2022-09-03T05:34:00.450 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.451 DEBUG:boto:Data: 2022-09-03T05:34:00.452 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.452 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.453 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.454 DEBUG:boto:Params: {} 2022-09-03T05:34:00.455 DEBUG:boto:Token: None 2022-09-03T05:34:00.456 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.456 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TGX68AkoPUVSYALSJ4aQQ7DOLPo= 2022-09-03T05:34:00.458 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TGX68AkoPUVSYALSJ4aQQ7DOLPo=', 'Content-Length': '0'} 2022-09-03T05:34:00.461 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000031eda0b387ed04e5-006312e748-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.462 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.463 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.463 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.464 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.464 DEBUG:boto:Method: HEAD 2022-09-03T05:34:00.465 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.465 DEBUG:boto:Data: 2022-09-03T05:34:00.466 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.467 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.467 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.467 DEBUG:boto:Params: {} 2022-09-03T05:34:00.468 DEBUG:boto:Token: None 2022-09-03T05:34:00.469 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.470 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2IBiguYgtG/sXSEvWjrb8l0yR2Q= 2022-09-03T05:34:00.470 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2IBiguYgtG/sXSEvWjrb8l0yR2Q=', 'Content-Length': '0'} 2022-09-03T05:34:00.472 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000394c526e1e3f1d83-006312e748-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.473 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.474 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.474 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.475 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.476 DEBUG:boto:Method: HEAD 2022-09-03T05:34:00.476 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.477 DEBUG:boto:Data: 2022-09-03T05:34:00.477 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.478 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.479 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.480 DEBUG:boto:Params: {} 2022-09-03T05:34:00.481 DEBUG:boto:Token: None 2022-09-03T05:34:00.482 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2IBiguYgtG/sXSEvWjrb8l0yR2Q= 2022-09-03T05:34:00.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2IBiguYgtG/sXSEvWjrb8l0yR2Q=', 'Content-Length': '0'} 2022-09-03T05:34:00.487 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a84a55642f3194b7-006312e748-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.489 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:34:00.490 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.490 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.491 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.492 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.493 DEBUG:boto:Method: GET 2022-09-03T05:34:00.494 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.495 DEBUG:boto:Data: 2022-09-03T05:34:00.496 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.497 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.498 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.499 DEBUG:boto:Params: {} 2022-09-03T05:34:00.500 DEBUG:boto:Token: None 2022-09-03T05:34:00.501 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.502 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TGX68AkoPUVSYALSJ4aQQ7DOLPo= 2022-09-03T05:34:00.503 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TGX68AkoPUVSYALSJ4aQQ7DOLPo=', 'Content-Length': '0'} 2022-09-03T05:34:00.506 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000006d14c0392660c27-006312e748-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.550 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.551 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.552 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.552 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.553 DEBUG:boto:Method: GET 2022-09-03T05:34:00.553 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.554 DEBUG:boto:Data: 2022-09-03T05:34:00.555 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.555 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.556 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.556 DEBUG:boto:Params: {} 2022-09-03T05:34:00.557 DEBUG:boto:Token: None 2022-09-03T05:34:00.557 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.558 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TGX68AkoPUVSYALSJ4aQQ7DOLPo= 2022-09-03T05:34:00.559 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TGX68AkoPUVSYALSJ4aQQ7DOLPo=', 'Content-Length': '0'} 2022-09-03T05:34:00.561 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000075609e91df144cd8-006312e748-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.562 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:00.562 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:00.563 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:00.563 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:00.564 DEBUG:boto:Method: GET 2022-09-03T05:34:00.564 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:00.565 DEBUG:boto:Data: 2022-09-03T05:34:00.565 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.566 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.567 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.567 DEBUG:boto:Params: {} 2022-09-03T05:34:00.568 DEBUG:boto:Token: None 2022-09-03T05:34:00.568 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/ 2022-09-03T05:34:00.569 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA= 2022-09-03T05:34:00.570 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA=', 'Content-Length': '0'} 2022-09-03T05:34:00.572 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002ebfededba3c0266-006312e748-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.573 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:00.574 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:00.574 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:00.575 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:00.576 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:00.576 DEBUG:boto:Method: GET 2022-09-03T05:34:00.577 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:00.577 DEBUG:boto:Data: 2022-09-03T05:34:00.578 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.578 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.579 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.579 DEBUG:boto:Params: {} 2022-09-03T05:34:00.580 DEBUG:boto:Token: None 2022-09-03T05:34:00.581 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/ 2022-09-03T05:34:00.581 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA= 2022-09-03T05:34:00.582 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA=', 'Content-Length': '0'} 2022-09-03T05:34:00.584 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d0d09f0f1db60e03-006312e748-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.585 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:00.586 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.587 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.587 DEBUG:boto:Method: HEAD 2022-09-03T05:34:00.588 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.588 DEBUG:boto:Data: 2022-09-03T05:34:00.589 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.589 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.590 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.590 DEBUG:boto:Params: {} 2022-09-03T05:34:00.591 DEBUG:boto:Token: None 2022-09-03T05:34:00.591 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.592 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:accJFKnqhFfgtSnpH+Ey5jQvz+Y= 2022-09-03T05:34:00.592 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:accJFKnqhFfgtSnpH+Ey5jQvz+Y=', 'Content-Length': '0'} 2022-09-03T05:34:00.595 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000667d41d438e634a6-006312e748-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.596 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.596 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.597 DEBUG:boto:Method: HEAD 2022-09-03T05:34:00.597 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.598 DEBUG:boto:Data: 2022-09-03T05:34:00.599 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.599 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.600 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.600 DEBUG:boto:Params: {} 2022-09-03T05:34:00.601 DEBUG:boto:Token: None 2022-09-03T05:34:00.602 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.602 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:accJFKnqhFfgtSnpH+Ey5jQvz+Y= 2022-09-03T05:34:00.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:accJFKnqhFfgtSnpH+Ey5jQvz+Y=', 'Content-Length': '0'} 2022-09-03T05:34:00.605 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008f6ed9682f4d6eba-006312e748-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.606 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:34:00.607 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.608 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.609 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.609 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.610 DEBUG:boto:Method: GET 2022-09-03T05:34:00.610 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.611 DEBUG:boto:Data: 2022-09-03T05:34:00.611 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.612 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.612 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.613 DEBUG:boto:Params: {} 2022-09-03T05:34:00.614 DEBUG:boto:Token: None 2022-09-03T05:34:00.614 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.615 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TGX68AkoPUVSYALSJ4aQQ7DOLPo= 2022-09-03T05:34:00.616 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TGX68AkoPUVSYALSJ4aQQ7DOLPo=', 'Content-Length': '0'} 2022-09-03T05:34:00.618 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ea1d37db200af894-006312e748-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.619 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.619 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:00.620 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.620 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.621 DEBUG:boto:Method: GET 2022-09-03T05:34:00.621 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.622 DEBUG:boto:Data: 2022-09-03T05:34:00.623 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.623 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.624 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.624 DEBUG:boto:Params: {} 2022-09-03T05:34:00.625 DEBUG:boto:Token: None 2022-09-03T05:34:00.625 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:00.626 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TGX68AkoPUVSYALSJ4aQQ7DOLPo= 2022-09-03T05:34:00.627 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TGX68AkoPUVSYALSJ4aQQ7DOLPo=', 'Content-Length': '0'} 2022-09-03T05:34:00.629 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000aaacd855c8ad0578-006312e748-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.630 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:00.630 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:00.631 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:00.631 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:00.632 DEBUG:boto:Method: GET 2022-09-03T05:34:00.633 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:00.633 DEBUG:boto:Data: 2022-09-03T05:34:00.634 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.635 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.635 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.636 DEBUG:boto:Params: {} 2022-09-03T05:34:00.636 DEBUG:boto:Token: None 2022-09-03T05:34:00.637 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/ 2022-09-03T05:34:00.637 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA= 2022-09-03T05:34:00.638 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA=', 'Content-Length': '0'} 2022-09-03T05:34:00.641 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000679239074374c380-006312e748-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.641 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:00.643 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:00.644 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:00.645 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:00.645 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:00.646 DEBUG:boto:Method: GET 2022-09-03T05:34:00.647 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:00.648 DEBUG:boto:Data: 2022-09-03T05:34:00.649 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.650 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.650 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.651 DEBUG:boto:Params: {} 2022-09-03T05:34:00.652 DEBUG:boto:Token: None 2022-09-03T05:34:00.653 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/ 2022-09-03T05:34:00.653 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA= 2022-09-03T05:34:00.654 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA=', 'Content-Length': '0'} 2022-09-03T05:34:00.656 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e2e124a74ad9c820-006312e748-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.657 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:00.658 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:34:00.659 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.659 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.660 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.660 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.661 DEBUG:boto:Method: GET 2022-09-03T05:34:00.661 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.662 DEBUG:boto:Data: 2022-09-03T05:34:00.662 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.663 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.663 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.664 DEBUG:boto:Params: {} 2022-09-03T05:34:00.665 DEBUG:boto:Token: None 2022-09-03T05:34:00.665 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.666 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:woNPDBUKLS0OXkc0dlapDk0Yw/o= 2022-09-03T05:34:00.666 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:woNPDBUKLS0OXkc0dlapDk0Yw/o=', 'Content-Length': '0'} 2022-09-03T05:34:00.669 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006893989341bf0700-006312e748-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.669 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.670 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.671 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.672 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.672 DEBUG:boto:Method: GET 2022-09-03T05:34:00.673 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.673 DEBUG:boto:Data: 2022-09-03T05:34:00.674 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.675 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.675 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.676 DEBUG:boto:Params: {} 2022-09-03T05:34:00.678 DEBUG:boto:Token: None 2022-09-03T05:34:00.680 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.681 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:woNPDBUKLS0OXkc0dlapDk0Yw/o= 2022-09-03T05:34:00.682 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:woNPDBUKLS0OXkc0dlapDk0Yw/o=', 'Content-Length': '0'} 2022-09-03T05:34:00.684 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001f68d70cba5a32a0-006312e748-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.685 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.686 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.686 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.687 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.687 DEBUG:boto:Method: HEAD 2022-09-03T05:34:00.688 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.688 DEBUG:boto:Data: 2022-09-03T05:34:00.689 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.689 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.690 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.690 DEBUG:boto:Params: {} 2022-09-03T05:34:00.691 DEBUG:boto:Token: None 2022-09-03T05:34:00.692 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.692 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JO3QMzmc1JIgGdT3+fZ9oiDTn8k= 2022-09-03T05:34:00.693 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JO3QMzmc1JIgGdT3+fZ9oiDTn8k=', 'Content-Length': '0'} 2022-09-03T05:34:00.695 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000012c063dadeb64330-006312e748-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.696 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.696 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.697 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.697 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.698 DEBUG:boto:Method: HEAD 2022-09-03T05:34:00.802 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.804 DEBUG:boto:Data: 2022-09-03T05:34:00.805 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.805 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.806 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.807 DEBUG:boto:Params: {} 2022-09-03T05:34:00.808 DEBUG:boto:Token: None 2022-09-03T05:34:00.809 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.810 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JO3QMzmc1JIgGdT3+fZ9oiDTn8k= 2022-09-03T05:34:00.811 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JO3QMzmc1JIgGdT3+fZ9oiDTn8k=', 'Content-Length': '0'} 2022-09-03T05:34:00.815 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000039e4920cb2cf9d70-006312e748-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.816 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:34:00.817 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.817 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.818 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.818 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.819 DEBUG:boto:Method: GET 2022-09-03T05:34:00.820 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.821 DEBUG:boto:Data: 2022-09-03T05:34:00.822 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.824 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.825 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.826 DEBUG:boto:Params: {} 2022-09-03T05:34:00.827 DEBUG:boto:Token: None 2022-09-03T05:34:00.828 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.828 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:woNPDBUKLS0OXkc0dlapDk0Yw/o= 2022-09-03T05:34:00.829 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:woNPDBUKLS0OXkc0dlapDk0Yw/o=', 'Content-Length': '0'} 2022-09-03T05:34:00.832 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006dcfbb720af93dd5-006312e748-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.833 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.834 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.834 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.835 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.836 DEBUG:boto:Method: GET 2022-09-03T05:34:00.837 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.838 DEBUG:boto:Data: 2022-09-03T05:34:00.839 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.839 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.840 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.841 DEBUG:boto:Params: {} 2022-09-03T05:34:00.842 DEBUG:boto:Token: None 2022-09-03T05:34:00.843 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.843 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:woNPDBUKLS0OXkc0dlapDk0Yw/o= 2022-09-03T05:34:00.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:woNPDBUKLS0OXkc0dlapDk0Yw/o=', 'Content-Length': '0'} 2022-09-03T05:34:00.847 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000142b3cedf4adbd22-006312e748-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.890 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:00.891 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:00.892 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:00.893 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:00.894 DEBUG:boto:Method: GET 2022-09-03T05:34:00.895 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:00.895 DEBUG:boto:Data: 2022-09-03T05:34:00.896 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.897 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.898 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.899 DEBUG:boto:Params: {} 2022-09-03T05:34:00.900 DEBUG:boto:Token: None 2022-09-03T05:34:00.901 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/ 2022-09-03T05:34:00.902 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA= 2022-09-03T05:34:00.903 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA=', 'Content-Length': '0'} 2022-09-03T05:34:00.906 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b1eb82fe721edcc1-006312e748-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.906 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:00.907 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:00.908 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:00.908 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:00.909 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:00.910 DEBUG:boto:Method: GET 2022-09-03T05:34:00.910 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:00.911 DEBUG:boto:Data: 2022-09-03T05:34:00.911 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.912 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.912 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.913 DEBUG:boto:Params: {} 2022-09-03T05:34:00.914 DEBUG:boto:Token: None 2022-09-03T05:34:00.915 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/ 2022-09-03T05:34:00.915 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA= 2022-09-03T05:34:00.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GO0eL1FUm2XZmAh3UbQND7UUPkA=', 'Content-Length': '0'} 2022-09-03T05:34:00.919 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dc73edb28c287fb6-006312e748-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.920 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:00.921 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.921 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.922 DEBUG:boto:Method: HEAD 2022-09-03T05:34:00.922 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.923 DEBUG:boto:Data: 2022-09-03T05:34:00.924 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.924 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.925 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.925 DEBUG:boto:Params: {} 2022-09-03T05:34:00.926 DEBUG:boto:Token: None 2022-09-03T05:34:00.927 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.927 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CQrzz2yD+awcrZ19NePr3+/3yV8= 2022-09-03T05:34:00.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CQrzz2yD+awcrZ19NePr3+/3yV8=', 'Content-Length': '0'} 2022-09-03T05:34:00.931 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ae83263e0e7d881d-006312e748-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.932 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.933 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.933 DEBUG:boto:Method: HEAD 2022-09-03T05:34:00.934 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.934 DEBUG:boto:Data: 2022-09-03T05:34:00.935 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.935 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.936 DEBUG:boto:Port: 8000 2022-09-03T05:34:00.936 DEBUG:boto:Params: {} 2022-09-03T05:34:00.937 DEBUG:boto:Token: None 2022-09-03T05:34:00.937 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.938 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CQrzz2yD+awcrZ19NePr3+/3yV8= 2022-09-03T05:34:00.939 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CQrzz2yD+awcrZ19NePr3+/3yV8=', 'Content-Length': '0'} 2022-09-03T05:34:00.941 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b5052a1b780d0551-006312e748-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.942 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:34:00.942 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.943 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.944 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.944 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.945 DEBUG:boto:Method: GET 2022-09-03T05:34:00.946 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.946 DEBUG:boto:Data: 2022-09-03T05:34:00.947 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.948 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:00.948 DEBUG:boto:Port: 8001 2022-09-03T05:34:00.949 DEBUG:boto:Params: {} 2022-09-03T05:34:00.950 DEBUG:boto:Token: None 2022-09-03T05:34:00.950 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:00 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.951 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:woNPDBUKLS0OXkc0dlapDk0Yw/o= 2022-09-03T05:34:00.952 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:woNPDBUKLS0OXkc0dlapDk0Yw/o=', 'Content-Length': '0'} 2022-09-03T05:34:00.954 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004d83214a01f1d37e-006312e748-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:00.994 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.995 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:00.995 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.996 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.997 DEBUG:boto:Method: GET 2022-09-03T05:34:00.997 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:00.998 DEBUG:boto:Data: 2022-09-03T05:34:00.998 DEBUG:boto:Headers: {} 2022-09-03T05:34:00.999 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:00.999 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.000 DEBUG:boto:Params: {} 2022-09-03T05:34:01.001 DEBUG:boto:Token: None 2022-09-03T05:34:01.001 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:01.002 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4GpegQEITrQxsjuWQkNX+xxlP8s= 2022-09-03T05:34:01.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4GpegQEITrQxsjuWQkNX+xxlP8s=', 'Content-Length': '0'} 2022-09-03T05:34:01.005 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000037f7fff995ad6753-006312e749-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.007 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:01.007 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:01.008 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:01.010 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:01.011 DEBUG:boto:Method: GET 2022-09-03T05:34:01.012 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:01.013 DEBUG:boto:Data: 2022-09-03T05:34:01.014 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.015 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.016 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.016 DEBUG:boto:Params: {} 2022-09-03T05:34:01.017 DEBUG:boto:Token: None 2022-09-03T05:34:01.018 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/ 2022-09-03T05:34:01.018 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw= 2022-09-03T05:34:01.019 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw=', 'Content-Length': '0'} 2022-09-03T05:34:01.022 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ec041a84d2c57dac-006312e749-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.022 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:01.023 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:01.024 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:01.024 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:01.025 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:01.025 DEBUG:boto:Method: GET 2022-09-03T05:34:01.026 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:01.027 DEBUG:boto:Data: 2022-09-03T05:34:01.027 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.028 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:01.028 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.029 DEBUG:boto:Params: {} 2022-09-03T05:34:01.030 DEBUG:boto:Token: None 2022-09-03T05:34:01.030 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/ 2022-09-03T05:34:01.031 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw= 2022-09-03T05:34:01.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw=', 'Content-Length': '0'} 2022-09-03T05:34:01.035 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000f76ed7a53e6a66c-006312e749-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.035 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:01.036 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-18 zones={test-zone3, test-zone1} 2022-09-03T05:34:01.037 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-18 zones={test-zone3, test-zone2} 2022-09-03T05:34:01.037 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:01.038 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:01.038 DEBUG:boto:Method: HEAD 2022-09-03T05:34:01.039 DEBUG:boto:Path: /dsgvcy-18/ 2022-09-03T05:34:01.039 DEBUG:boto:Data: 2022-09-03T05:34:01.040 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.040 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.041 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.042 DEBUG:boto:Params: {} 2022-09-03T05:34:01.042 DEBUG:boto:Token: None 2022-09-03T05:34:01.043 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/ 2022-09-03T05:34:01.044 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XscHBJfQd9ht5vky2vph73kPcr0= 2022-09-03T05:34:01.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XscHBJfQd9ht5vky2vph73kPcr0=', 'Content-Length': '0'} 2022-09-03T05:34:01.048 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006fa6fdc63a236800-006312e749-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.048 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:01.049 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:01.049 DEBUG:boto:Method: HEAD 2022-09-03T05:34:01.050 DEBUG:boto:Path: /dsgvcy-18/ 2022-09-03T05:34:01.050 DEBUG:boto:Data: 2022-09-03T05:34:01.051 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.051 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.052 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.052 DEBUG:boto:Params: {} 2022-09-03T05:34:01.053 DEBUG:boto:Token: None 2022-09-03T05:34:01.053 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/ 2022-09-03T05:34:01.054 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XscHBJfQd9ht5vky2vph73kPcr0= 2022-09-03T05:34:01.055 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XscHBJfQd9ht5vky2vph73kPcr0=', 'Content-Length': '0'} 2022-09-03T05:34:01.058 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b5ff23342ac6d28c-006312e749-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.058 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:34:01.059 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:01.059 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:01.060 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:34:01.060 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:34:01.061 DEBUG:boto:Method: GET 2022-09-03T05:34:01.061 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:34:01.062 DEBUG:boto:Data: 2022-09-03T05:34:01.062 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.063 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.063 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.064 DEBUG:boto:Params: {} 2022-09-03T05:34:01.064 DEBUG:boto:Token: None 2022-09-03T05:34:01.065 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/?versions 2022-09-03T05:34:01.066 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a5+nh9MNIb8ue33DVJG6HOwPins= 2022-09-03T05:34:01.066 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a5+nh9MNIb8ue33DVJG6HOwPins=', 'Content-Length': '0'} 2022-09-03T05:34:01.069 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003195e1fd87f8c1ef-006312e749-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.070 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:01.071 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:34:01.071 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:34:01.072 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:34:01.072 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:34:01.073 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:34:01.073 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:01.074 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:01.074 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:34:01.075 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:34:01.075 DEBUG:boto:Method: GET 2022-09-03T05:34:01.076 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:34:01.076 DEBUG:boto:Data: 2022-09-03T05:34:01.077 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.077 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.078 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.079 DEBUG:boto:Params: {} 2022-09-03T05:34:01.079 DEBUG:boto:Token: None 2022-09-03T05:34:01.080 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/?versions 2022-09-03T05:34:01.080 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a5+nh9MNIb8ue33DVJG6HOwPins= 2022-09-03T05:34:01.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a5+nh9MNIb8ue33DVJG6HOwPins=', 'Content-Length': '0'} 2022-09-03T05:34:01.084 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000be62fff3b19aa90-006312e749-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.085 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:01.086 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:34:01.086 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone1 2022-09-03T05:34:01.087 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone2 2022-09-03T05:34:01.087 DEBUG:tasks.rgw_multi.zone_rados:o=obj-test-zone3 2022-09-03T05:34:01.088 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:01.088 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:01.089 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:34:01.089 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:34:01.090 DEBUG:boto:Method: GET 2022-09-03T05:34:01.090 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:34:01.091 DEBUG:boto:Data: 2022-09-03T05:34:01.091 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.092 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.092 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.093 DEBUG:boto:Params: {} 2022-09-03T05:34:01.094 DEBUG:boto:Token: None 2022-09-03T05:34:01.094 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/?versions 2022-09-03T05:34:01.095 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a5+nh9MNIb8ue33DVJG6HOwPins= 2022-09-03T05:34:01.095 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a5+nh9MNIb8ue33DVJG6HOwPins=', 'Content-Length': '0'} 2022-09-03T05:34:01.098 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000024bcb8c03118507a-006312e749-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.099 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:01.100 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:01.101 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:01.101 DEBUG:boto:path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:34:01.102 DEBUG:boto:auth_path=/dsgvcy-18/?versions&max-keys=999 2022-09-03T05:34:01.103 DEBUG:boto:Method: GET 2022-09-03T05:34:01.103 DEBUG:boto:Path: /dsgvcy-18/?versions&max-keys=999 2022-09-03T05:34:01.104 DEBUG:boto:Data: 2022-09-03T05:34:01.105 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.105 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.106 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.106 DEBUG:boto:Params: {} 2022-09-03T05:34:01.107 DEBUG:boto:Token: None 2022-09-03T05:34:01.107 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/?versions 2022-09-03T05:34:01.108 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a5+nh9MNIb8ue33DVJG6HOwPins= 2022-09-03T05:34:01.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a5+nh9MNIb8ue33DVJG6HOwPins=', 'Content-Length': '0'} 2022-09-03T05:34:01.112 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004e0e67b2ef3be967-006312e749-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.113 DEBUG:boto:b'dsgvcy-18999falsedummyoGSUnGWMT-ds3ddOT-KcIQfbrHE7lZatrue2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone14KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0true2022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone26Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-true2022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj-test-zone3OvqhLUtjFelftEABGk77o1bs.QvCmgbtrue2022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:01.114 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:34:01.115 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:34:01.115 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:34:01.116 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.116 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.117 DEBUG:boto:Method: GET 2022-09-03T05:34:01.117 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.118 DEBUG:boto:Data: 2022-09-03T05:34:01.118 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.119 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.119 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.120 DEBUG:boto:Params: {} 2022-09-03T05:34:01.121 DEBUG:boto:Token: None 2022-09-03T05:34:01.121 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.122 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AOP2MxWMW1+FYNebTL+fmyuzQxU= 2022-09-03T05:34:01.123 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AOP2MxWMW1+FYNebTL+fmyuzQxU=', 'Content-Length': '0'} 2022-09-03T05:34:01.125 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000d41becdb9a4a3401-006312e749-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.126 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:34:01.126 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:34:01.127 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.127 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.128 DEBUG:boto:Method: GET 2022-09-03T05:34:01.129 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.129 DEBUG:boto:Data: 2022-09-03T05:34:01.130 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.130 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.131 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.131 DEBUG:boto:Params: {} 2022-09-03T05:34:01.132 DEBUG:boto:Token: None 2022-09-03T05:34:01.133 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.133 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AOP2MxWMW1+FYNebTL+fmyuzQxU= 2022-09-03T05:34:01.134 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AOP2MxWMW1+FYNebTL+fmyuzQxU=', 'Content-Length': '0'} 2022-09-03T05:34:01.136 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000084c9120bc65d6d91-006312e749-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.137 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:34:01.138 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:34:01.138 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.139 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.139 DEBUG:boto:Method: HEAD 2022-09-03T05:34:01.140 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.140 DEBUG:boto:Data: 2022-09-03T05:34:01.141 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.142 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.142 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.143 DEBUG:boto:Params: {} 2022-09-03T05:34:01.143 DEBUG:boto:Token: None 2022-09-03T05:34:01.144 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.144 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4yA8xJM50tLpQ/y1IYwhu2XXqvc= 2022-09-03T05:34:01.145 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4yA8xJM50tLpQ/y1IYwhu2XXqvc=', 'Content-Length': '0'} 2022-09-03T05:34:01.147 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000090a83707e4cb603c-006312e749-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.148 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:34:01.149 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:34:01.149 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.150 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.150 DEBUG:boto:Method: HEAD 2022-09-03T05:34:01.151 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.151 DEBUG:boto:Data: 2022-09-03T05:34:01.152 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.152 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.153 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.153 DEBUG:boto:Params: {} 2022-09-03T05:34:01.154 DEBUG:boto:Token: None 2022-09-03T05:34:01.155 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.155 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4yA8xJM50tLpQ/y1IYwhu2XXqvc= 2022-09-03T05:34:01.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4yA8xJM50tLpQ/y1IYwhu2XXqvc=', 'Content-Length': '0'} 2022-09-03T05:34:01.158 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000000909380dd4596751-006312e749-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.159 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:34:01.160 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:34:01.161 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:34:01.162 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.162 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.163 DEBUG:boto:Method: GET 2022-09-03T05:34:01.163 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.164 DEBUG:boto:Data: 2022-09-03T05:34:01.164 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.165 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.165 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.166 DEBUG:boto:Params: {} 2022-09-03T05:34:01.167 DEBUG:boto:Token: None 2022-09-03T05:34:01.167 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.168 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AOP2MxWMW1+FYNebTL+fmyuzQxU= 2022-09-03T05:34:01.169 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AOP2MxWMW1+FYNebTL+fmyuzQxU=', 'Content-Length': '0'} 2022-09-03T05:34:01.171 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a4741e0674f0b741-006312e749-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.173 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:34:01.174 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:34:01.175 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.176 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.177 DEBUG:boto:Method: GET 2022-09-03T05:34:01.178 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.178 DEBUG:boto:Data: 2022-09-03T05:34:01.179 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.179 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.180 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.181 DEBUG:boto:Params: {} 2022-09-03T05:34:01.181 DEBUG:boto:Token: None 2022-09-03T05:34:01.182 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.183 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AOP2MxWMW1+FYNebTL+fmyuzQxU= 2022-09-03T05:34:01.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AOP2MxWMW1+FYNebTL+fmyuzQxU=', 'Content-Length': '0'} 2022-09-03T05:34:01.186 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000007ed9f51ab36f752a-006312e749-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.187 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:01.187 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:01.188 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:34:01.188 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:34:01.189 DEBUG:boto:Method: GET 2022-09-03T05:34:01.189 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:34:01.190 DEBUG:boto:Data: 2022-09-03T05:34:01.191 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.191 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.192 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.192 DEBUG:boto:Params: {} 2022-09-03T05:34:01.193 DEBUG:boto:Token: None 2022-09-03T05:34:01.194 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/ 2022-09-03T05:34:01.194 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw= 2022-09-03T05:34:01.195 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw=', 'Content-Length': '0'} 2022-09-03T05:34:01.198 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009d24c4d13d5a85ca-006312e749-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.242 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:01.243 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:01.244 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:01.244 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:34:01.245 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:34:01.245 DEBUG:boto:Method: GET 2022-09-03T05:34:01.246 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:34:01.246 DEBUG:boto:Data: 2022-09-03T05:34:01.247 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.247 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.248 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.249 DEBUG:boto:Params: {} 2022-09-03T05:34:01.249 DEBUG:boto:Token: None 2022-09-03T05:34:01.250 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/ 2022-09-03T05:34:01.251 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw= 2022-09-03T05:34:01.251 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw=', 'Content-Length': '0'} 2022-09-03T05:34:01.254 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009293e0860aa5daa0-006312e749-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.255 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:01.256 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:34:01.256 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:34:01.257 DEBUG:boto:Method: HEAD 2022-09-03T05:34:01.257 DEBUG:boto:Path: /dsgvcy-18/dummy 2022-09-03T05:34:01.258 DEBUG:boto:Data: 2022-09-03T05:34:01.258 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.259 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.259 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.260 DEBUG:boto:Params: {} 2022-09-03T05:34:01.261 DEBUG:boto:Token: None 2022-09-03T05:34:01.261 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/dummy 2022-09-03T05:34:01.262 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GhCJ/RdASUu5cdCoukDp3PixX8w= 2022-09-03T05:34:01.263 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GhCJ/RdASUu5cdCoukDp3PixX8w=', 'Content-Length': '0'} 2022-09-03T05:34:01.266 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000001ac1254391e6ad1a-006312e749-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.267 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:34:01.267 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:34:01.268 DEBUG:boto:Method: HEAD 2022-09-03T05:34:01.268 DEBUG:boto:Path: /dsgvcy-18/dummy 2022-09-03T05:34:01.269 DEBUG:boto:Data: 2022-09-03T05:34:01.270 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.271 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.271 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.272 DEBUG:boto:Params: {} 2022-09-03T05:34:01.272 DEBUG:boto:Token: None 2022-09-03T05:34:01.273 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/dummy 2022-09-03T05:34:01.274 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GhCJ/RdASUu5cdCoukDp3PixX8w= 2022-09-03T05:34:01.275 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GhCJ/RdASUu5cdCoukDp3PixX8w=', 'Content-Length': '0'} 2022-09-03T05:34:01.278 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000eac323acb7ad584d-006312e749-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.279 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:34:01.280 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:34:01.281 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:34:01.281 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.282 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.282 DEBUG:boto:Method: GET 2022-09-03T05:34:01.283 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.283 DEBUG:boto:Data: 2022-09-03T05:34:01.284 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.286 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.287 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.288 DEBUG:boto:Params: {} 2022-09-03T05:34:01.289 DEBUG:boto:Token: None 2022-09-03T05:34:01.290 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.291 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AOP2MxWMW1+FYNebTL+fmyuzQxU= 2022-09-03T05:34:01.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AOP2MxWMW1+FYNebTL+fmyuzQxU=', 'Content-Length': '0'} 2022-09-03T05:34:01.296 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000d1c86cc8339d317b-006312e749-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.297 DEBUG:boto:path=/dsgvcy-18/dummy 2022-09-03T05:34:01.297 DEBUG:boto:auth_path=/dsgvcy-18/dummy 2022-09-03T05:34:01.298 DEBUG:boto:path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.298 DEBUG:boto:auth_path=/dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.299 DEBUG:boto:Method: GET 2022-09-03T05:34:01.299 DEBUG:boto:Path: /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.300 DEBUG:boto:Data: 2022-09-03T05:34:01.300 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.301 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.301 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.302 DEBUG:boto:Params: {} 2022-09-03T05:34:01.303 DEBUG:boto:Token: None 2022-09-03T05:34:01.303 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/dummy?versionId=oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa 2022-09-03T05:34:01.304 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AOP2MxWMW1+FYNebTL+fmyuzQxU= 2022-09-03T05:34:01.305 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AOP2MxWMW1+FYNebTL+fmyuzQxU=', 'Content-Length': '0'} 2022-09-03T05:34:01.307 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:28:28 GMT'), ('x-amz-version-id', 'oGSUnGWMT-ds3ddOT-KcIQfbrHE7lZa'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000007dcf9172156f6be4-006312e749-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.308 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:01.309 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:01.309 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:34:01.310 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:34:01.311 DEBUG:boto:Method: GET 2022-09-03T05:34:01.311 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:34:01.312 DEBUG:boto:Data: 2022-09-03T05:34:01.312 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.313 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.313 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.314 DEBUG:boto:Params: {} 2022-09-03T05:34:01.314 DEBUG:boto:Token: None 2022-09-03T05:34:01.315 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/ 2022-09-03T05:34:01.316 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw= 2022-09-03T05:34:01.316 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw=', 'Content-Length': '0'} 2022-09-03T05:34:01.319 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fd547d7976a5479b-006312e749-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.320 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:01.321 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:01.321 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:01.322 DEBUG:boto:path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:34:01.322 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=dummy 2022-09-03T05:34:01.323 DEBUG:boto:Method: GET 2022-09-03T05:34:01.323 DEBUG:boto:Path: /dsgvcy-18/?prefix=dummy 2022-09-03T05:34:01.324 DEBUG:boto:Data: 2022-09-03T05:34:01.324 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.325 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.325 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.326 DEBUG:boto:Params: {} 2022-09-03T05:34:01.326 DEBUG:boto:Token: None 2022-09-03T05:34:01.327 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/ 2022-09-03T05:34:01.328 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw= 2022-09-03T05:34:01.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw=', 'Content-Length': '0'} 2022-09-03T05:34:01.331 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b50b047a787cc1cd-006312e749-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.377 DEBUG:boto:b'dsgvcy-18dummy1000falsedummy2022-09-03T05:28:28.560Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:01.378 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:34:01.380 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.380 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.381 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.382 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.383 DEBUG:boto:Method: GET 2022-09-03T05:34:01.384 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.385 DEBUG:boto:Data: 2022-09-03T05:34:01.386 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.387 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.388 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.389 DEBUG:boto:Params: {} 2022-09-03T05:34:01.390 DEBUG:boto:Token: None 2022-09-03T05:34:01.391 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.392 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qE1cWLqTzsl9tNt+1aStACiAJIc= 2022-09-03T05:34:01.393 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qE1cWLqTzsl9tNt+1aStACiAJIc=', 'Content-Length': '0'} 2022-09-03T05:34:01.395 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000050a9202c5c5b6bb2-006312e749-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.396 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.397 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.397 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.398 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.398 DEBUG:boto:Method: GET 2022-09-03T05:34:01.399 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.400 DEBUG:boto:Data: 2022-09-03T05:34:01.400 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.401 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.401 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.402 DEBUG:boto:Params: {} 2022-09-03T05:34:01.402 DEBUG:boto:Token: None 2022-09-03T05:34:01.403 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.404 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qE1cWLqTzsl9tNt+1aStACiAJIc= 2022-09-03T05:34:01.405 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qE1cWLqTzsl9tNt+1aStACiAJIc=', 'Content-Length': '0'} 2022-09-03T05:34:01.407 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ab32413633e83866-006312e749-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.408 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.408 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.409 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.409 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.410 DEBUG:boto:Method: HEAD 2022-09-03T05:34:01.410 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.411 DEBUG:boto:Data: 2022-09-03T05:34:01.411 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.412 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.412 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.413 DEBUG:boto:Params: {} 2022-09-03T05:34:01.414 DEBUG:boto:Token: None 2022-09-03T05:34:01.414 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.415 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PPTv2GMQDgHm0eXw4ILrdIfYHVw= 2022-09-03T05:34:01.415 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PPTv2GMQDgHm0eXw4ILrdIfYHVw=', 'Content-Length': '0'} 2022-09-03T05:34:01.417 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000eaf6ee064731a2e1-006312e749-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.418 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.419 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.420 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.420 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.421 DEBUG:boto:Method: HEAD 2022-09-03T05:34:01.421 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.422 DEBUG:boto:Data: 2022-09-03T05:34:01.422 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.423 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.423 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.424 DEBUG:boto:Params: {} 2022-09-03T05:34:01.424 DEBUG:boto:Token: None 2022-09-03T05:34:01.425 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.426 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PPTv2GMQDgHm0eXw4ILrdIfYHVw= 2022-09-03T05:34:01.426 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PPTv2GMQDgHm0eXw4ILrdIfYHVw=', 'Content-Length': '0'} 2022-09-03T05:34:01.428 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b8e5d7ee0ff1ffa4-006312e749-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.429 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:34:01.430 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.430 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.431 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.431 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.432 DEBUG:boto:Method: GET 2022-09-03T05:34:01.433 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.433 DEBUG:boto:Data: 2022-09-03T05:34:01.434 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.434 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.435 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.436 DEBUG:boto:Params: {} 2022-09-03T05:34:01.436 DEBUG:boto:Token: None 2022-09-03T05:34:01.437 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.438 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qE1cWLqTzsl9tNt+1aStACiAJIc= 2022-09-03T05:34:01.438 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qE1cWLqTzsl9tNt+1aStACiAJIc=', 'Content-Length': '0'} 2022-09-03T05:34:01.441 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005b7077857bad417a-006312e749-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.441 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.442 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.443 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.443 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.444 DEBUG:boto:Method: GET 2022-09-03T05:34:01.445 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.445 DEBUG:boto:Data: 2022-09-03T05:34:01.446 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.446 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.447 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.447 DEBUG:boto:Params: {} 2022-09-03T05:34:01.448 DEBUG:boto:Token: None 2022-09-03T05:34:01.449 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.449 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qE1cWLqTzsl9tNt+1aStACiAJIc= 2022-09-03T05:34:01.450 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qE1cWLqTzsl9tNt+1aStACiAJIc=', 'Content-Length': '0'} 2022-09-03T05:34:01.452 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b38d8cd0d5fc6cc7-006312e749-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.453 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:01.454 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:01.454 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:01.455 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:01.455 DEBUG:boto:Method: GET 2022-09-03T05:34:01.456 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:01.456 DEBUG:boto:Data: 2022-09-03T05:34:01.457 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.457 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.458 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.458 DEBUG:boto:Params: {} 2022-09-03T05:34:01.459 DEBUG:boto:Token: None 2022-09-03T05:34:01.460 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/ 2022-09-03T05:34:01.460 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw= 2022-09-03T05:34:01.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw=', 'Content-Length': '0'} 2022-09-03T05:34:01.464 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000193511d2717639da-006312e749-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.464 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:01.465 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:01.466 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:01.467 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:01.467 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:01.468 DEBUG:boto:Method: GET 2022-09-03T05:34:01.468 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:01.469 DEBUG:boto:Data: 2022-09-03T05:34:01.470 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.470 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.471 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.471 DEBUG:boto:Params: {} 2022-09-03T05:34:01.472 DEBUG:boto:Token: None 2022-09-03T05:34:01.473 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/ 2022-09-03T05:34:01.473 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw= 2022-09-03T05:34:01.474 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw=', 'Content-Length': '0'} 2022-09-03T05:34:01.476 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e1137b4807bb33d1-006312e749-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.477 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:01.478 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.479 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.479 DEBUG:boto:Method: HEAD 2022-09-03T05:34:01.480 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.480 DEBUG:boto:Data: 2022-09-03T05:34:01.481 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.481 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.482 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.482 DEBUG:boto:Params: {} 2022-09-03T05:34:01.483 DEBUG:boto:Token: None 2022-09-03T05:34:01.483 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.484 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3lqNuNt4I5RAyAff1rUwYqDOfXY= 2022-09-03T05:34:01.485 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3lqNuNt4I5RAyAff1rUwYqDOfXY=', 'Content-Length': '0'} 2022-09-03T05:34:01.487 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006474f728e44ada7d-006312e749-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.488 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.488 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.489 DEBUG:boto:Method: HEAD 2022-09-03T05:34:01.490 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.490 DEBUG:boto:Data: 2022-09-03T05:34:01.491 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.491 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.492 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.493 DEBUG:boto:Params: {} 2022-09-03T05:34:01.493 DEBUG:boto:Token: None 2022-09-03T05:34:01.494 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.495 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3lqNuNt4I5RAyAff1rUwYqDOfXY= 2022-09-03T05:34:01.495 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3lqNuNt4I5RAyAff1rUwYqDOfXY=', 'Content-Length': '0'} 2022-09-03T05:34:01.498 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a93c5335ca0728be-006312e749-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.499 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone1 2022-09-03T05:34:01.499 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.500 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.501 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.502 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.503 DEBUG:boto:Method: GET 2022-09-03T05:34:01.504 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.505 DEBUG:boto:Data: 2022-09-03T05:34:01.505 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.506 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.509 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.510 DEBUG:boto:Params: {} 2022-09-03T05:34:01.511 DEBUG:boto:Token: None 2022-09-03T05:34:01.512 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.513 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qE1cWLqTzsl9tNt+1aStACiAJIc= 2022-09-03T05:34:01.515 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qE1cWLqTzsl9tNt+1aStACiAJIc=', 'Content-Length': '0'} 2022-09-03T05:34:01.518 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000499840c42f2856bd-006312e749-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.558 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.559 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1 2022-09-03T05:34:01.560 DEBUG:boto:path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.560 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.561 DEBUG:boto:Method: GET 2022-09-03T05:34:01.562 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.563 DEBUG:boto:Data: 2022-09-03T05:34:01.564 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.565 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.566 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.566 DEBUG:boto:Params: {} 2022-09-03T05:34:01.568 DEBUG:boto:Token: None 2022-09-03T05:34:01.569 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/obj-test-zone1?versionId=4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0 2022-09-03T05:34:01.570 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qE1cWLqTzsl9tNt+1aStACiAJIc= 2022-09-03T05:34:01.571 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qE1cWLqTzsl9tNt+1aStACiAJIc=', 'Content-Length': '0'} 2022-09-03T05:34:01.574 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '4KZP4Pv9tMSxD67TLb4wgPcOlCvx4E0'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003a6eaf3a87aa55a4-006312e749-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.577 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:01.579 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:01.580 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:01.581 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:01.582 DEBUG:boto:Method: GET 2022-09-03T05:34:01.583 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:01.584 DEBUG:boto:Data: 2022-09-03T05:34:01.585 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.586 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.588 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.589 DEBUG:boto:Params: {} 2022-09-03T05:34:01.590 DEBUG:boto:Token: None 2022-09-03T05:34:01.591 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/ 2022-09-03T05:34:01.592 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw= 2022-09-03T05:34:01.593 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw=', 'Content-Length': '0'} 2022-09-03T05:34:01.597 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f431c3d58d9ca8d4-006312e749-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.599 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:01.600 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:01.601 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:01.602 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:01.603 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:01.605 DEBUG:boto:Method: GET 2022-09-03T05:34:01.606 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone1 2022-09-03T05:34:01.607 DEBUG:boto:Data: 2022-09-03T05:34:01.608 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.609 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.610 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.611 DEBUG:boto:Params: {} 2022-09-03T05:34:01.612 DEBUG:boto:Token: None 2022-09-03T05:34:01.613 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/ 2022-09-03T05:34:01.614 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw= 2022-09-03T05:34:01.615 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw=', 'Content-Length': '0'} 2022-09-03T05:34:01.618 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ef134d52fc299a5b-006312e749-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.619 DEBUG:boto:b'dsgvcy-18obj-test-zone11000falseobj-test-zone12022-09-03T05:29:09.888Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:01.620 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:34:01.621 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.622 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.623 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.624 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.625 DEBUG:boto:Method: GET 2022-09-03T05:34:01.625 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.626 DEBUG:boto:Data: 2022-09-03T05:34:01.628 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.628 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.629 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.629 DEBUG:boto:Params: {} 2022-09-03T05:34:01.630 DEBUG:boto:Token: None 2022-09-03T05:34:01.630 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.631 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:886PjfGgKuwLwTiRVoHch8WIks0= 2022-09-03T05:34:01.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:886PjfGgKuwLwTiRVoHch8WIks0=', 'Content-Length': '0'} 2022-09-03T05:34:01.634 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000df3ca5a9105114f8-006312e749-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.635 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.635 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.636 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.636 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.637 DEBUG:boto:Method: GET 2022-09-03T05:34:01.637 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.638 DEBUG:boto:Data: 2022-09-03T05:34:01.639 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.639 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.640 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.640 DEBUG:boto:Params: {} 2022-09-03T05:34:01.641 DEBUG:boto:Token: None 2022-09-03T05:34:01.642 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.642 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:886PjfGgKuwLwTiRVoHch8WIks0= 2022-09-03T05:34:01.643 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:886PjfGgKuwLwTiRVoHch8WIks0=', 'Content-Length': '0'} 2022-09-03T05:34:01.646 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000b8628657b73eb4f-006312e749-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.648 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.649 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.650 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.650 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.651 DEBUG:boto:Method: HEAD 2022-09-03T05:34:01.652 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.653 DEBUG:boto:Data: 2022-09-03T05:34:01.654 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.656 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.656 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.657 DEBUG:boto:Params: {} 2022-09-03T05:34:01.658 DEBUG:boto:Token: None 2022-09-03T05:34:01.660 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.661 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JIDOJRXop47CSOWnPd2ojdfiKAE= 2022-09-03T05:34:01.662 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JIDOJRXop47CSOWnPd2ojdfiKAE=', 'Content-Length': '0'} 2022-09-03T05:34:01.665 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fa045e7c01a0966e-006312e749-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.667 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.667 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.668 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.668 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.669 DEBUG:boto:Method: HEAD 2022-09-03T05:34:01.670 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.670 DEBUG:boto:Data: 2022-09-03T05:34:01.671 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.671 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.672 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.672 DEBUG:boto:Params: {} 2022-09-03T05:34:01.673 DEBUG:boto:Token: None 2022-09-03T05:34:01.674 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.674 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JIDOJRXop47CSOWnPd2ojdfiKAE= 2022-09-03T05:34:01.676 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JIDOJRXop47CSOWnPd2ojdfiKAE=', 'Content-Length': '0'} 2022-09-03T05:34:01.679 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000071a657e1790f8880-006312e749-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.681 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:34:01.682 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.683 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.684 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.685 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.685 DEBUG:boto:Method: GET 2022-09-03T05:34:01.687 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.688 DEBUG:boto:Data: 2022-09-03T05:34:01.690 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.692 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.692 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.693 DEBUG:boto:Params: {} 2022-09-03T05:34:01.694 DEBUG:boto:Token: None 2022-09-03T05:34:01.695 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.697 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:886PjfGgKuwLwTiRVoHch8WIks0= 2022-09-03T05:34:01.699 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:886PjfGgKuwLwTiRVoHch8WIks0=', 'Content-Length': '0'} 2022-09-03T05:34:01.701 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f837649e1883c069-006312e749-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.702 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.703 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.703 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.704 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.704 DEBUG:boto:Method: GET 2022-09-03T05:34:01.705 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.705 DEBUG:boto:Data: 2022-09-03T05:34:01.706 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.707 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.707 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.708 DEBUG:boto:Params: {} 2022-09-03T05:34:01.708 DEBUG:boto:Token: None 2022-09-03T05:34:01.709 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.710 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:886PjfGgKuwLwTiRVoHch8WIks0= 2022-09-03T05:34:01.710 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:886PjfGgKuwLwTiRVoHch8WIks0=', 'Content-Length': '0'} 2022-09-03T05:34:01.713 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009d0327a80036db28-006312e749-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.760 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:01.762 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:01.762 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:01.763 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:01.763 DEBUG:boto:Method: GET 2022-09-03T05:34:01.764 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:01.764 DEBUG:boto:Data: 2022-09-03T05:34:01.765 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.772 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.773 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.774 DEBUG:boto:Params: {} 2022-09-03T05:34:01.774 DEBUG:boto:Token: None 2022-09-03T05:34:01.775 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/ 2022-09-03T05:34:01.776 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw= 2022-09-03T05:34:01.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw=', 'Content-Length': '0'} 2022-09-03T05:34:01.779 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009f7f7fa0592c750a-006312e749-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.780 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:01.782 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:01.783 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:01.783 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:01.785 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:01.786 DEBUG:boto:Method: GET 2022-09-03T05:34:01.787 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:01.787 DEBUG:boto:Data: 2022-09-03T05:34:01.788 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.789 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.791 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.792 DEBUG:boto:Params: {} 2022-09-03T05:34:01.794 DEBUG:boto:Token: None 2022-09-03T05:34:01.794 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/ 2022-09-03T05:34:01.795 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw= 2022-09-03T05:34:01.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw=', 'Content-Length': '0'} 2022-09-03T05:34:01.799 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c9c57b46612e1555-006312e749-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.800 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:01.801 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.802 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.803 DEBUG:boto:Method: HEAD 2022-09-03T05:34:01.804 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.804 DEBUG:boto:Data: 2022-09-03T05:34:01.805 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.806 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.807 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.808 DEBUG:boto:Params: {} 2022-09-03T05:34:01.809 DEBUG:boto:Token: None 2022-09-03T05:34:01.810 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.811 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rlI5y9uzOZ3ww2IDZzhCB/G6Moc= 2022-09-03T05:34:01.813 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rlI5y9uzOZ3ww2IDZzhCB/G6Moc=', 'Content-Length': '0'} 2022-09-03T05:34:01.816 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a8e4c5ef636d8029-006312e749-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.817 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.817 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.818 DEBUG:boto:Method: HEAD 2022-09-03T05:34:01.818 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.819 DEBUG:boto:Data: 2022-09-03T05:34:01.820 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.820 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.821 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.821 DEBUG:boto:Params: {} 2022-09-03T05:34:01.822 DEBUG:boto:Token: None 2022-09-03T05:34:01.822 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.823 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rlI5y9uzOZ3ww2IDZzhCB/G6Moc= 2022-09-03T05:34:01.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rlI5y9uzOZ3ww2IDZzhCB/G6Moc=', 'Content-Length': '0'} 2022-09-03T05:34:01.826 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007a82afbbc9b5c640-006312e749-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.827 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone2 2022-09-03T05:34:01.828 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.828 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.829 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.829 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.830 DEBUG:boto:Method: GET 2022-09-03T05:34:01.831 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.831 DEBUG:boto:Data: 2022-09-03T05:34:01.832 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.832 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.833 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.833 DEBUG:boto:Params: {} 2022-09-03T05:34:01.834 DEBUG:boto:Token: None 2022-09-03T05:34:01.835 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.835 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:886PjfGgKuwLwTiRVoHch8WIks0= 2022-09-03T05:34:01.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:886PjfGgKuwLwTiRVoHch8WIks0=', 'Content-Length': '0'} 2022-09-03T05:34:01.839 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009ac69302daa3722b-006312e749-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.886 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.887 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2 2022-09-03T05:34:01.888 DEBUG:boto:path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.889 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.890 DEBUG:boto:Method: GET 2022-09-03T05:34:01.891 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.892 DEBUG:boto:Data: 2022-09-03T05:34:01.893 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.893 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.894 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.896 DEBUG:boto:Params: {} 2022-09-03T05:34:01.897 DEBUG:boto:Token: None 2022-09-03T05:34:01.898 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/obj-test-zone2?versionId=6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw- 2022-09-03T05:34:01.899 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:886PjfGgKuwLwTiRVoHch8WIks0= 2022-09-03T05:34:01.900 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:886PjfGgKuwLwTiRVoHch8WIks0=', 'Content-Length': '0'} 2022-09-03T05:34:01.903 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', '6Zw.9tP40IVs1D1lYjOYat5Z.6VIYw-'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000096775d9e936b9132-006312e749-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.904 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:01.905 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:01.907 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:01.907 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:01.909 DEBUG:boto:Method: GET 2022-09-03T05:34:01.909 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:01.910 DEBUG:boto:Data: 2022-09-03T05:34:01.911 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.911 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.912 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.912 DEBUG:boto:Params: {} 2022-09-03T05:34:01.913 DEBUG:boto:Token: None 2022-09-03T05:34:01.914 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/ 2022-09-03T05:34:01.914 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw= 2022-09-03T05:34:01.915 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw=', 'Content-Length': '0'} 2022-09-03T05:34:01.918 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bdb84430a22b98f5-006312e749-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.919 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:01.919 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:01.920 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:01.920 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:01.921 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:01.922 DEBUG:boto:Method: GET 2022-09-03T05:34:01.923 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone2 2022-09-03T05:34:01.923 DEBUG:boto:Data: 2022-09-03T05:34:01.924 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.924 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.925 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.925 DEBUG:boto:Params: {} 2022-09-03T05:34:01.926 DEBUG:boto:Token: None 2022-09-03T05:34:01.927 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/ 2022-09-03T05:34:01.927 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw= 2022-09-03T05:34:01.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i7odtzFntmhbDcNXyFBCC+WM9Qw=', 'Content-Length': '0'} 2022-09-03T05:34:01.931 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006482f4f11133d001-006312e749-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.974 DEBUG:boto:b'dsgvcy-18obj-test-zone21000falseobj-test-zone22022-09-03T05:29:10.050Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:01.975 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:34:01.976 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:01.977 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:01.977 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:01.978 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:01.978 DEBUG:boto:Method: GET 2022-09-03T05:34:01.979 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:01.979 DEBUG:boto:Data: 2022-09-03T05:34:01.980 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.981 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:01.981 DEBUG:boto:Port: 8001 2022-09-03T05:34:01.982 DEBUG:boto:Params: {} 2022-09-03T05:34:01.983 DEBUG:boto:Token: None 2022-09-03T05:34:01.984 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:01.985 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4GpegQEITrQxsjuWQkNX+xxlP8s= 2022-09-03T05:34:01.986 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4GpegQEITrQxsjuWQkNX+xxlP8s=', 'Content-Length': '0'} 2022-09-03T05:34:01.990 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008da63bfb02c12464-006312e749-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:01.991 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:01.992 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:01.992 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:01.993 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:01.993 DEBUG:boto:Method: GET 2022-09-03T05:34:01.994 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:01.994 DEBUG:boto:Data: 2022-09-03T05:34:01.995 DEBUG:boto:Headers: {} 2022-09-03T05:34:01.996 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:01.996 DEBUG:boto:Port: 8000 2022-09-03T05:34:01.997 DEBUG:boto:Params: {} 2022-09-03T05:34:01.997 DEBUG:boto:Token: None 2022-09-03T05:34:01.998 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:01 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:01.999 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4GpegQEITrQxsjuWQkNX+xxlP8s= 2022-09-03T05:34:01.999 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4GpegQEITrQxsjuWQkNX+xxlP8s=', 'Content-Length': '0'} 2022-09-03T05:34:02.002 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000039d20b87b29eabeb-006312e74a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:02.002 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:02.003 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:02.003 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.004 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.005 DEBUG:boto:Method: HEAD 2022-09-03T05:34:02.005 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.006 DEBUG:boto:Data: 2022-09-03T05:34:02.006 DEBUG:boto:Headers: {} 2022-09-03T05:34:02.007 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:02.007 DEBUG:boto:Port: 8001 2022-09-03T05:34:02.008 DEBUG:boto:Params: {} 2022-09-03T05:34:02.008 DEBUG:boto:Token: None 2022-09-03T05:34:02.009 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:02 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.009 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cAktH4jSMqjYgC0PBukH1Al0rxs= 2022-09-03T05:34:02.010 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cAktH4jSMqjYgC0PBukH1Al0rxs=', 'Content-Length': '0'} 2022-09-03T05:34:02.013 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000013826a77e02e8459-006312e74a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:02.014 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:02.015 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:02.015 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.016 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.016 DEBUG:boto:Method: HEAD 2022-09-03T05:34:02.017 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.017 DEBUG:boto:Data: 2022-09-03T05:34:02.018 DEBUG:boto:Headers: {} 2022-09-03T05:34:02.018 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:02.019 DEBUG:boto:Port: 8000 2022-09-03T05:34:02.019 DEBUG:boto:Params: {} 2022-09-03T05:34:02.020 DEBUG:boto:Token: None 2022-09-03T05:34:02.020 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:02 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.021 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cAktH4jSMqjYgC0PBukH1Al0rxs= 2022-09-03T05:34:02.022 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cAktH4jSMqjYgC0PBukH1Al0rxs=', 'Content-Length': '0'} 2022-09-03T05:34:02.024 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ae032e4b6d9ec401-006312e74a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:02.025 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:34:02.025 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:02.026 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:02.026 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.027 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.028 DEBUG:boto:Method: GET 2022-09-03T05:34:02.028 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.029 DEBUG:boto:Data: 2022-09-03T05:34:02.029 DEBUG:boto:Headers: {} 2022-09-03T05:34:02.030 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:02.030 DEBUG:boto:Port: 8001 2022-09-03T05:34:02.031 DEBUG:boto:Params: {} 2022-09-03T05:34:02.032 DEBUG:boto:Token: None 2022-09-03T05:34:02.032 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:02 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.033 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X9uSp1494zX5K6pM9OkuF138CL0= 2022-09-03T05:34:02.033 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X9uSp1494zX5K6pM9OkuF138CL0=', 'Content-Length': '0'} 2022-09-03T05:34:02.036 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f438e24db4432131-006312e74a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:02.037 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:02.037 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:02.038 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.039 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.041 DEBUG:boto:Method: GET 2022-09-03T05:34:02.042 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.043 DEBUG:boto:Data: 2022-09-03T05:34:02.043 DEBUG:boto:Headers: {} 2022-09-03T05:34:02.044 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:02.044 DEBUG:boto:Port: 8000 2022-09-03T05:34:02.045 DEBUG:boto:Params: {} 2022-09-03T05:34:02.045 DEBUG:boto:Token: None 2022-09-03T05:34:02.046 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:02 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.047 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X9uSp1494zX5K6pM9OkuF138CL0= 2022-09-03T05:34:02.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X9uSp1494zX5K6pM9OkuF138CL0=', 'Content-Length': '0'} 2022-09-03T05:34:02.050 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000532dbc03f26fc112-006312e74a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:02.051 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:02.051 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:02.052 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:02.052 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:02.053 DEBUG:boto:Method: GET 2022-09-03T05:34:02.053 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:02.054 DEBUG:boto:Data: 2022-09-03T05:34:02.055 DEBUG:boto:Headers: {} 2022-09-03T05:34:02.055 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:02.056 DEBUG:boto:Port: 8001 2022-09-03T05:34:02.056 DEBUG:boto:Params: {} 2022-09-03T05:34:02.057 DEBUG:boto:Token: None 2022-09-03T05:34:02.057 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:02 GMT /dsgvcy-18/ 2022-09-03T05:34:02.058 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rHqg599ERuctiA3tIMRt+/1e5Tw= 2022-09-03T05:34:02.059 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rHqg599ERuctiA3tIMRt+/1e5Tw=', 'Content-Length': '0'} 2022-09-03T05:34:02.061 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000b0bc2bd360d4e9c-006312e74a-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:02.062 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:02.063 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:02.064 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:02.064 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:02.065 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:02.065 DEBUG:boto:Method: GET 2022-09-03T05:34:02.066 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:02.066 DEBUG:boto:Data: 2022-09-03T05:34:02.067 DEBUG:boto:Headers: {} 2022-09-03T05:34:02.067 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:02.068 DEBUG:boto:Port: 8000 2022-09-03T05:34:02.068 DEBUG:boto:Params: {} 2022-09-03T05:34:02.069 DEBUG:boto:Token: None 2022-09-03T05:34:02.070 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:02 GMT /dsgvcy-18/ 2022-09-03T05:34:02.071 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rHqg599ERuctiA3tIMRt+/1e5Tw= 2022-09-03T05:34:02.072 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rHqg599ERuctiA3tIMRt+/1e5Tw=', 'Content-Length': '0'} 2022-09-03T05:34:02.074 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ed4604b6eaaac76a-006312e74a-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:02.075 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:02.076 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:02.076 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:02.077 DEBUG:boto:Method: HEAD 2022-09-03T05:34:02.077 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3 2022-09-03T05:34:02.078 DEBUG:boto:Data: 2022-09-03T05:34:02.078 DEBUG:boto:Headers: {} 2022-09-03T05:34:02.079 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:02.079 DEBUG:boto:Port: 8001 2022-09-03T05:34:02.080 DEBUG:boto:Params: {} 2022-09-03T05:34:02.081 DEBUG:boto:Token: None 2022-09-03T05:34:02.082 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:02 GMT /dsgvcy-18/obj-test-zone3 2022-09-03T05:34:02.083 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y/pnPSINWJudYLcpcYOsqsZ+gwg= 2022-09-03T05:34:02.083 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y/pnPSINWJudYLcpcYOsqsZ+gwg=', 'Content-Length': '0'} 2022-09-03T05:34:02.086 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c479279aca4cdc2d-006312e74a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:02.088 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:02.089 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:02.091 DEBUG:boto:Method: HEAD 2022-09-03T05:34:02.091 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3 2022-09-03T05:34:02.092 DEBUG:boto:Data: 2022-09-03T05:34:02.092 DEBUG:boto:Headers: {} 2022-09-03T05:34:02.093 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:02.093 DEBUG:boto:Port: 8000 2022-09-03T05:34:02.094 DEBUG:boto:Params: {} 2022-09-03T05:34:02.094 DEBUG:boto:Token: None 2022-09-03T05:34:02.095 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:02 GMT /dsgvcy-18/obj-test-zone3 2022-09-03T05:34:02.096 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y/pnPSINWJudYLcpcYOsqsZ+gwg= 2022-09-03T05:34:02.097 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y/pnPSINWJudYLcpcYOsqsZ+gwg=', 'Content-Length': '0'} 2022-09-03T05:34:02.103 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000f5c7ebd4597ecaf-006312e74a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:02.105 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj-test-zone3 2022-09-03T05:34:02.106 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:02.106 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:02.107 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.109 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.110 DEBUG:boto:Method: GET 2022-09-03T05:34:02.111 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.112 DEBUG:boto:Data: 2022-09-03T05:34:02.112 DEBUG:boto:Headers: {} 2022-09-03T05:34:02.113 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:02.113 DEBUG:boto:Port: 8001 2022-09-03T05:34:02.114 DEBUG:boto:Params: {} 2022-09-03T05:34:02.115 DEBUG:boto:Token: None 2022-09-03T05:34:02.116 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:02 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.117 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X9uSp1494zX5K6pM9OkuF138CL0= 2022-09-03T05:34:02.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X9uSp1494zX5K6pM9OkuF138CL0=', 'Content-Length': '0'} 2022-09-03T05:34:02.121 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dca54a23dcbb4d38-006312e74a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:02.122 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:02.123 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3 2022-09-03T05:34:02.123 DEBUG:boto:path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.124 DEBUG:boto:auth_path=/dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.125 DEBUG:boto:Method: GET 2022-09-03T05:34:02.125 DEBUG:boto:Path: /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.126 DEBUG:boto:Data: 2022-09-03T05:34:02.126 DEBUG:boto:Headers: {} 2022-09-03T05:34:02.127 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:02.127 DEBUG:boto:Port: 8000 2022-09-03T05:34:02.128 DEBUG:boto:Params: {} 2022-09-03T05:34:02.129 DEBUG:boto:Token: None 2022-09-03T05:34:02.130 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:02 GMT /dsgvcy-18/obj-test-zone3?versionId=OvqhLUtjFelftEABGk77o1bs.QvCmgb 2022-09-03T05:34:02.131 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X9uSp1494zX5K6pM9OkuF138CL0= 2022-09-03T05:34:02.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X9uSp1494zX5K6pM9OkuF138CL0=', 'Content-Length': '0'} 2022-09-03T05:34:02.136 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:31:03 GMT'), ('x-amz-version-id', 'OvqhLUtjFelftEABGk77o1bs.QvCmgb'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d27ce8d04341662a-006312e74a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:02.182 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:02.184 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:02.185 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:02.186 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:02.187 DEBUG:boto:Method: GET 2022-09-03T05:34:02.187 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:02.188 DEBUG:boto:Data: 2022-09-03T05:34:02.188 DEBUG:boto:Headers: {} 2022-09-03T05:34:02.189 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:02.189 DEBUG:boto:Port: 8001 2022-09-03T05:34:02.190 DEBUG:boto:Params: {} 2022-09-03T05:34:02.191 DEBUG:boto:Token: None 2022-09-03T05:34:02.191 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:02 GMT /dsgvcy-18/ 2022-09-03T05:34:02.192 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rHqg599ERuctiA3tIMRt+/1e5Tw= 2022-09-03T05:34:02.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rHqg599ERuctiA3tIMRt+/1e5Tw=', 'Content-Length': '0'} 2022-09-03T05:34:02.196 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ed449633aeb38ae5-006312e74a-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:02.197 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:02.199 DEBUG:boto:path=/dsgvcy-18/ 2022-09-03T05:34:02.200 DEBUG:boto:auth_path=/dsgvcy-18/ 2022-09-03T05:34:02.201 DEBUG:boto:path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:02.202 DEBUG:boto:auth_path=/dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:02.203 DEBUG:boto:Method: GET 2022-09-03T05:34:02.204 DEBUG:boto:Path: /dsgvcy-18/?prefix=obj-test-zone3 2022-09-03T05:34:02.206 DEBUG:boto:Data: 2022-09-03T05:34:02.206 DEBUG:boto:Headers: {} 2022-09-03T05:34:02.207 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:02.207 DEBUG:boto:Port: 8000 2022-09-03T05:34:02.208 DEBUG:boto:Params: {} 2022-09-03T05:34:02.208 DEBUG:boto:Token: None 2022-09-03T05:34:02.209 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:02 GMT /dsgvcy-18/ 2022-09-03T05:34:02.210 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rHqg599ERuctiA3tIMRt+/1e5Tw= 2022-09-03T05:34:02.210 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rHqg599ERuctiA3tIMRt+/1e5Tw=', 'Content-Length': '0'} 2022-09-03T05:34:02.213 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009f690e9ad6d9f8dc-006312e74a-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:02.214 DEBUG:boto:b'dsgvcy-18obj-test-zone31000falseobj-test-zone32022-09-03T05:29:10.219Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:02.215 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-18 zones={test-zone3, test-zone2} 2022-09-03T05:34:02.216 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_versioned_object_incremental_sync ... ok 2022-09-03T05:34:02.220 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:02.221 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:02.222 DEBUG:boto:Method: PUT 2022-09-03T05:34:02.222 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:02.223 DEBUG:boto:Data: 2022-09-03T05:34:02.224 DEBUG:boto:Headers: {} 2022-09-03T05:34:02.225 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:02.225 DEBUG:boto:Port: 8000 2022-09-03T05:34:02.226 DEBUG:boto:Params: {} 2022-09-03T05:34:02.226 DEBUG:boto:Token: None 2022-09-03T05:34:02.227 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:34:02 GMT /dsgvcy-19/ 2022-09-03T05:34:02.228 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cMX99CGFt+r9bY1t8psbXhWptN0= 2022-09-03T05:34:02.228 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cMX99CGFt+r9bY1t8psbXhWptN0=', 'Content-Length': '0'} 2022-09-03T05:34:02.247 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000d0bd1295172d04ab-006312e74a-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:02.248 DEBUG:rgw_multi.tests:created bucket=dsgvcy-19 2022-09-03T05:34:02.249 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:02.250 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:02.252 DEBUG:boto:path=/dsgvcy-19/?versioning 2022-09-03T05:34:02.253 DEBUG:boto:auth_path=/dsgvcy-19/?versioning 2022-09-03T05:34:02.253 DEBUG:boto:Method: PUT 2022-09-03T05:34:02.255 DEBUG:boto:Path: /dsgvcy-19/?versioning 2022-09-03T05:34:02.256 DEBUG:boto:Data: Enabled Disabled 2022-09-03T05:34:02.257 DEBUG:boto:Headers: {} 2022-09-03T05:34:02.257 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:02.258 DEBUG:boto:Port: 8000 2022-09-03T05:34:02.258 DEBUG:boto:Params: {} 2022-09-03T05:34:02.259 DEBUG:boto:Token: None 2022-09-03T05:34:02.260 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:34:02 GMT /dsgvcy-19/?versioning 2022-09-03T05:34:02.260 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NZbQaZ7noIDGMMoeJffILFnu9Zw= 2022-09-03T05:34:02.261 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NZbQaZ7noIDGMMoeJffILFnu9Zw=', 'Content-Length': '228'} 2022-09-03T05:34:02.268 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000008e6307a2d58afbb6-006312e74a-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:02.269 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:34:02.270 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:34:02.271 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:34:02.323 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:34:02.323 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:34:02.374 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=41ecd2a5-afc0-4eae-9349-7dfe39baa1cb 2022-09-03T05:34:02.398 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:34:02.398 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:34:02.399 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:34:02.399 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:18:18.906852Z" 2022-09-03T05:34:02.399 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:34:02.400 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:34:02.400 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:34:02.401 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:34:02.401 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:34:02.402 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:34:02.402 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662183242.267929_162.1", 2022-09-03T05:34:02.403 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:34:02.267929Z" 2022-09-03T05:34:02.403 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:34:02.404 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:34:02.404 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:34:02.405 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:20:30.928498Z" 2022-09-03T05:34:02.405 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:34:02.406 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:34:02.408 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183242.267929_162.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:34:02.409 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:34:02.409 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:34:02.410 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:34:02.410 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:34:02.471 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:34:02.472 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:34:02.503 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:34:02.504+0000 7fe96d171600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:34:02.504 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:34:02.505+0000 7fe96d171600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:34:02.517 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:34:02.518 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:34:02.518 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:34:02.519 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:34:02.519 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:34:02.520 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:34:02.520 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:34:02.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:34:02.520 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:34:02.521 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:34:02.521 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:34:02.521 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:34:02.522 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:34:02.522 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:34:02.522 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:34:02.522 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:34:02.523 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:34:02.523 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:34:02.523 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:34:02.523 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:34:02.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:34:02.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:34:02.524 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:34:02.524 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:34:02.525 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:34:02.525 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:34:02.525 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:34:02.525 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:34:02.526 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:34:02.526 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:34:02.526 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:34:02.526 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:34:02.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:34:02.527 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:34:02.527 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:34:02.528 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:34:02.528 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:34:02.528 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183242.267929_162.1", 2022-09-03T05:34:02.528 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:34:02.529 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:34:02.529 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:34:02.529 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:34:02.267929Z", 2022-09-03T05:34:02.529 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:34:02.530 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:34:02.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:34:02.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:34:02.530 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:34:02.531 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:34:02.531 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:34:02.531 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:34:02.531 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:34:02.532 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:34:02.532 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:34:02.532 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:34:02.532 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:34:02.533 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:34:02.533 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:34:02.533 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:34:02.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:34:02.534 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:34:02.534 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:34:02.534 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:34:02.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:34:02.535 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:34:02Z" 2022-09-03T05:34:02.535 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:34:02.536 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183242.267929_162.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:34:02.267929Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:34:02Z" } 2022-09-03T05:34:02.537 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183242.267929_162.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:34:02.267929Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:34:02.537 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183242.267929_162.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:34:02.538 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183242.267929_162.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:34:02.538 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:34:02.539 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:34:02.539 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:34:02.540 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:34:02.540 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:34:02.590 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:34:02.591 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:34:02.636 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:34:02.637+0000 7f9c525a9600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:34:02.637 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:34:02.638+0000 7f9c525a9600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:34:02.659 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:34:02.660 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:34:02.660 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:34:02.661 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:34:02.661 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:34:02.662 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:34:02.662 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:34:02.663 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:34:02.663 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:34:02.664 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:34:02.664 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:34:02.665 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:34:02.665 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:34:02.666 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:34:02.667 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:34:02.668 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:34:02.668 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:34:02.668 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:34:02.668 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:34:02.669 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:34:02.669 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:34:02.669 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:34:02.669 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:34:02.670 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:34:02.670 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:34:02.670 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:34:02.671 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:34:02.671 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:34:02.671 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:34:02.671 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:34:02.672 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:34:02.672 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:34:02.672 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:34:02.673 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:34:02.673 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:34:02.673 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:34:02.673 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:34:02.674 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662183242.267929_162.1", 2022-09-03T05:34:02.674 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:34:02.675 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:34:02.675 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:34:02.676 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:34:02.267929Z", 2022-09-03T05:34:02.676 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:34:02.676 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:34:02.676 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:34:02.677 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:34:02.677 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:34:02.678 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:34:02.678 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:34:02.679 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:34:02.679 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:34:02.680 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:34:02.680 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:34:02.681 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:34:02.681 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:34:02.682 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:34:02.683 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:34:02.683 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:34:02.684 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:34:02.684 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:34:02.684 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:34:02.685 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:34:02.685 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:34:02.686 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:34:02Z" 2022-09-03T05:34:02.686 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:34:02.687 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183242.267929_162.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:34:02.267929Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:34:02Z" } 2022-09-03T05:34:02.688 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183242.267929_162.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:34:02.267929Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:34:02.689 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183242.267929_162.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:34:02.689 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183242.267929_162.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:34:02.691 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:34:02.691 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:34:02.692 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:34:02.692 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:34:02.693 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:34:02.744 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:34:02.745 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:34:02.774 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:34:02.775+0000 7f61e487b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:34:02.775 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:34:02.776+0000 7f61e487b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:34:02.786 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:34:02.786 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:34:02.787 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:34:02.789 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:34:02.789 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:34:02.789 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:34:02.790 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:34:02.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:34:02.790 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:34:02.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:34:02.791 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:34:02.791 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:34:02.791 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:34:02.792 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:34:02.792 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:34:02.792 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:34:02.792 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:34:02.793 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:34:02.793 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:34:02.793 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:34:02.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:34:02.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:34:02.794 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:34:02.794 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:34:02.795 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:34:02.795 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:34:02.795 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:34:02.795 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:34:02.796 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:34:02.796 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:34:02.796 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:34:02.796 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:34:02.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:34:02.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:34:02.797 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:34:02.798 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:34:02.798 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:34:02.798 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183242.267929_162.1", 2022-09-03T05:34:02.799 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:34:02.799 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:34:02.799 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:34:02.799 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:34:02.267929Z", 2022-09-03T05:34:02.800 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:34:02.800 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:34:02.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:34:02.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:34:02.801 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:34:02.801 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:34:02.801 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:34:02.802 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:34:02.802 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:34:02.802 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:34:02.802 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:34:02.803 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:34:02.803 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:34:02.803 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:34:02.804 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:34:02.804 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:34:02.804 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:34:02.804 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:34:02.805 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:34:02.805 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:34:02.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:34:02.806 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:34:02Z" 2022-09-03T05:34:02.806 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:34:02.807 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183242.267929_162.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:34:02.267929Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:34:02Z" } 2022-09-03T05:34:02.808 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183242.267929_162.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:34:02.267929Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:34:02.809 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183242.267929_162.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:34:02.810 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183242.267929_162.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:34:02.810 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:34:02.811 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:02.811 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:02.812 DEBUG:boto:Method: HEAD 2022-09-03T05:34:02.812 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:02.813 DEBUG:boto:Data: 2022-09-03T05:34:02.814 DEBUG:boto:Headers: {} 2022-09-03T05:34:02.814 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:02.815 DEBUG:boto:Port: 8000 2022-09-03T05:34:02.815 DEBUG:boto:Params: {} 2022-09-03T05:34:02.817 DEBUG:boto:Token: None 2022-09-03T05:34:02.817 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:02 GMT /dsgvcy-19/ 2022-09-03T05:34:02.818 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5rMGI/9kSlBZ7reN16xkVLBcMW4= 2022-09-03T05:34:02.819 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5rMGI/9kSlBZ7reN16xkVLBcMW4=', 'Content-Length': '0'} 2022-09-03T05:34:02.821 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006f8eeb95e1df9608-006312e74a-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:02.822 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:02.823 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:02.824 DEBUG:boto:Method: PUT 2022-09-03T05:34:02.824 DEBUG:boto:Path: /dsgvcy-19/dummy 2022-09-03T05:34:02.825 DEBUG:boto:Data: 2022-09-03T05:34:02.825 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue'} 2022-09-03T05:34:02.826 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:02.827 DEBUG:boto:Port: 8000 2022-09-03T05:34:02.827 DEBUG:boto:Params: {} 2022-09-03T05:34:02.828 DEBUG:boto:Token: None 2022-09-03T05:34:02.829 DEBUG:boto:StringToSign: PUT 1B2M2Y8AsgTpgAmY7PhCfg== application/octet-stream Sat, 03 Sep 2022 05:34:02 GMT /dsgvcy-19/dummy 2022-09-03T05:34:02.829 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:83te5R2soXoRo82Oa+fRsk2fVnI= 2022-09-03T05:34:02.831 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:83te5R2soXoRo82Oa+fRsk2fVnI='} 2022-09-03T05:34:02.844 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-amz-request-id', 'tx00000b6bf4c77377009af-006312e74a-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:02.845 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-19 2022-09-03T05:34:02.846 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:34:02.847 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:34:02.847 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-19 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:34:02.903 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:34:02.904 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:34:02.935 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:34:02.936+0000 7f40756ab600 1 waiting to reach incremental sync.. 2022-09-03T05:34:07.937 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:34:07.938+0000 7f40756ab600 1 bucket sync caught up with source: 2022-09-03T05:34:07.938 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000002.32.10, , , , , ] 2022-09-03T05:34:07.938 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000002.32.10, , , , , ] 2022-09-03T05:34:07.938 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:34:07.938+0000 7f40756ab600 0 bucket checkpoint complete 2022-09-03T05:34:07.955 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-19 2022-09-03T05:34:07.957 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:34:07.958 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:34:07.959 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-19 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:34:08.025 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:34:08.025 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:34:08.067 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:34:08.067+0000 7f31b208e600 1 bucket sync caught up with source: 2022-09-03T05:34:08.068 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000002.32.10, , , , , ] 2022-09-03T05:34:08.068 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000002.32.10, , , , , ] 2022-09-03T05:34:08.069 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:34:08.067+0000 7f31b208e600 0 bucket checkpoint complete 2022-09-03T05:34:08.085 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-19 2022-09-03T05:34:08.086 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:34:08.086 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:34:08.087 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-19 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:34:08.151 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:34:08.151 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:34:08.201 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:34:08.202+0000 7fbdedd77600 1 waiting to reach incremental sync.. 2022-09-03T05:34:13.201 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:34:13.202+0000 7fbdedd77600 1 waiting to reach incremental sync.. 2022-09-03T05:34:18.207 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:34:18.203+0000 7fbdedd77600 1 bucket sync caught up with source: 2022-09-03T05:34:18.207 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000002.32.10, , , , , ] 2022-09-03T05:34:18.208 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000002.32.10, , , , , ] 2022-09-03T05:34:18.208 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:34:18.203+0000 7fbdedd77600 0 bucket checkpoint complete 2022-09-03T05:34:18.220 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-19 2022-09-03T05:34:18.221 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:34:18.223 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:34:18.225 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-19 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:34:18.297 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:34:18.297 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:34:18.358 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:34:18.359+0000 7f2222451600 1 waiting to reach incremental sync.. 2022-09-03T05:34:23.358 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:34:23.360+0000 7f2222451600 1 waiting to reach incremental sync.. 2022-09-03T05:34:28.360 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:34:28.362+0000 7f2222451600 1 bucket sync caught up with source: 2022-09-03T05:34:28.361 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.36.9, , , , , ] 2022-09-03T05:34:28.362 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000002.34.10, , , , , ] 2022-09-03T05:34:28.363 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:34:28.362+0000 7f2222451600 0 bucket checkpoint complete 2022-09-03T05:34:28.385 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-19 2022-09-03T05:34:28.387 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:34:28.388 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:34:28.389 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-19 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:34:28.451 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:34:28.452 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:34:28.506 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:34:28.507+0000 7f0cf0fcc600 1 bucket sync caught up with source: 2022-09-03T05:34:28.506 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.36.9, , , , , ] 2022-09-03T05:34:28.507 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000003.36.9, , , , , ] 2022-09-03T05:34:28.507 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:34:28.507+0000 7f0cf0fcc600 0 bucket checkpoint complete 2022-09-03T05:34:28.524 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-19 2022-09-03T05:34:28.525 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-19 2022-09-03T05:34:28.525 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:34:28.526 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:34:28.527 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-19 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:34:28.594 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:34:28.594 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:34:28.629 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:34:28.630+0000 7fd672879600 1 waiting for incremental sync to catch up: 2022-09-03T05:34:28.630 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.29.9, , , , , ] 2022-09-03T05:34:28.631 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.31.9, , , , , ] 2022-09-03T05:34:33.635 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:34:33.632+0000 7fd672879600 1 waiting for incremental sync to catch up: 2022-09-03T05:34:33.636 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.29.9, , , , , ] 2022-09-03T05:34:33.637 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.31.9, , , , , ] 2022-09-03T05:34:38.632 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:34:38.633+0000 7fd672879600 1 waiting for incremental sync to catch up: 2022-09-03T05:34:38.632 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.29.9, , , , , ] 2022-09-03T05:34:38.633 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.31.9, , , , , ] 2022-09-03T05:34:43.633 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:34:43.633+0000 7fd672879600 1 waiting for incremental sync to catch up: 2022-09-03T05:34:43.634 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.29.9, , , , , ] 2022-09-03T05:34:43.634 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.31.9, , , , , ] 2022-09-03T05:34:48.635 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:34:48.635+0000 7fd672879600 1 bucket sync caught up with source: 2022-09-03T05:34:48.636 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000004.31.9, , , , , ] 2022-09-03T05:34:48.636 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.31.9, , , , , ] 2022-09-03T05:34:48.637 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:34:48.635+0000 7fd672879600 0 bucket checkpoint complete 2022-09-03T05:34:48.653 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-19 2022-09-03T05:34:48.655 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:34:48.656 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:34:48.657 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-19 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:34:48.723 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:34:48.723 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:34:48.771 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:34:48.772+0000 7ff330d0b600 1 bucket sync caught up with source: 2022-09-03T05:34:48.771 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000004.31.9, , , , , ] 2022-09-03T05:34:48.772 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000004.31.9, , , , , ] 2022-09-03T05:34:48.773 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:34:48.772+0000 7ff330d0b600 0 bucket checkpoint complete 2022-09-03T05:34:48.785 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-19 2022-09-03T05:34:48.786 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-19 zones={test-zone2, test-zone1} 2022-09-03T05:34:48.787 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:48.787 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:48.788 DEBUG:boto:Method: HEAD 2022-09-03T05:34:48.791 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:48.792 DEBUG:boto:Data: 2022-09-03T05:34:48.793 DEBUG:boto:Headers: {} 2022-09-03T05:34:48.794 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:48.794 DEBUG:boto:Port: 8000 2022-09-03T05:34:48.795 DEBUG:boto:Params: {} 2022-09-03T05:34:48.796 DEBUG:boto:Token: None 2022-09-03T05:34:48.796 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:48 GMT /dsgvcy-19/ 2022-09-03T05:34:48.798 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nCDxPFaLyMaskuMvp/qlX3yTA14= 2022-09-03T05:34:48.799 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nCDxPFaLyMaskuMvp/qlX3yTA14=', 'Content-Length': '0'} 2022-09-03T05:34:48.803 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004d74d9752f0f10dd-006312e778-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:48.804 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:48.804 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:48.805 DEBUG:boto:Method: HEAD 2022-09-03T05:34:48.805 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:48.806 DEBUG:boto:Data: 2022-09-03T05:34:48.806 DEBUG:boto:Headers: {} 2022-09-03T05:34:48.807 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:48.807 DEBUG:boto:Port: 8000 2022-09-03T05:34:48.808 DEBUG:boto:Params: {} 2022-09-03T05:34:48.809 DEBUG:boto:Token: None 2022-09-03T05:34:48.811 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:48 GMT /dsgvcy-19/ 2022-09-03T05:34:48.812 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nCDxPFaLyMaskuMvp/qlX3yTA14= 2022-09-03T05:34:48.813 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nCDxPFaLyMaskuMvp/qlX3yTA14=', 'Content-Length': '0'} 2022-09-03T05:34:48.818 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b9b934f575a5bb28-006312e778-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:48.819 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:34:48.820 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:48.821 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:48.822 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:48.823 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:48.824 DEBUG:boto:Method: GET 2022-09-03T05:34:48.825 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:48.826 DEBUG:boto:Data: 2022-09-03T05:34:48.827 DEBUG:boto:Headers: {} 2022-09-03T05:34:48.828 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:48.829 DEBUG:boto:Port: 8000 2022-09-03T05:34:48.830 DEBUG:boto:Params: {} 2022-09-03T05:34:48.831 DEBUG:boto:Token: None 2022-09-03T05:34:48.832 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:48 GMT /dsgvcy-19/?versions 2022-09-03T05:34:48.833 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NW/djbB4tqu5sGPfkcKz21ckGtE= 2022-09-03T05:34:48.834 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NW/djbB4tqu5sGPfkcKz21ckGtE=', 'Content-Length': '0'} 2022-09-03T05:34:48.837 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004163ef2a8a472b74-006312e778-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:48.838 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:48.840 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:34:48.842 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:34:48.843 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:48.844 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:48.844 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:48.845 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:48.846 DEBUG:boto:Method: GET 2022-09-03T05:34:48.848 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:48.849 DEBUG:boto:Data: 2022-09-03T05:34:48.851 DEBUG:boto:Headers: {} 2022-09-03T05:34:48.852 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:48.853 DEBUG:boto:Port: 8000 2022-09-03T05:34:48.853 DEBUG:boto:Params: {} 2022-09-03T05:34:48.855 DEBUG:boto:Token: None 2022-09-03T05:34:48.856 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:48 GMT /dsgvcy-19/?versions 2022-09-03T05:34:48.857 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NW/djbB4tqu5sGPfkcKz21ckGtE= 2022-09-03T05:34:48.858 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NW/djbB4tqu5sGPfkcKz21ckGtE=', 'Content-Length': '0'} 2022-09-03T05:34:48.861 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a401f21a6775feee-006312e778-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:48.863 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:48.864 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:34:48.866 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:48.867 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:48.868 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:48.869 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:48.870 DEBUG:boto:Method: GET 2022-09-03T05:34:48.871 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:48.871 DEBUG:boto:Data: 2022-09-03T05:34:48.872 DEBUG:boto:Headers: {} 2022-09-03T05:34:48.873 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:48.873 DEBUG:boto:Port: 8000 2022-09-03T05:34:48.874 DEBUG:boto:Params: {} 2022-09-03T05:34:48.875 DEBUG:boto:Token: None 2022-09-03T05:34:48.875 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:48 GMT /dsgvcy-19/?versions 2022-09-03T05:34:48.876 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NW/djbB4tqu5sGPfkcKz21ckGtE= 2022-09-03T05:34:48.877 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NW/djbB4tqu5sGPfkcKz21ckGtE=', 'Content-Length': '0'} 2022-09-03T05:34:48.879 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c2a23e5548cedbb3-006312e778-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:48.880 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:48.881 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:48.881 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:48.882 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:48.882 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:48.883 DEBUG:boto:Method: GET 2022-09-03T05:34:48.884 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:48.885 DEBUG:boto:Data: 2022-09-03T05:34:48.886 DEBUG:boto:Headers: {} 2022-09-03T05:34:48.886 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:48.887 DEBUG:boto:Port: 8000 2022-09-03T05:34:48.887 DEBUG:boto:Params: {} 2022-09-03T05:34:48.888 DEBUG:boto:Token: None 2022-09-03T05:34:48.889 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:48 GMT /dsgvcy-19/?versions 2022-09-03T05:34:48.890 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NW/djbB4tqu5sGPfkcKz21ckGtE= 2022-09-03T05:34:48.891 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NW/djbB4tqu5sGPfkcKz21ckGtE=', 'Content-Length': '0'} 2022-09-03T05:34:48.895 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f9f4e65512abe618-006312e778-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:48.896 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:48.897 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:34:48.899 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:48.900 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:48.901 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:48.901 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:48.902 DEBUG:boto:Method: GET 2022-09-03T05:34:48.903 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:48.904 DEBUG:boto:Data: 2022-09-03T05:34:48.905 DEBUG:boto:Headers: {} 2022-09-03T05:34:48.906 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:48.907 DEBUG:boto:Port: 8000 2022-09-03T05:34:48.908 DEBUG:boto:Params: {} 2022-09-03T05:34:48.910 DEBUG:boto:Token: None 2022-09-03T05:34:48.911 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:48 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:48.912 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8HI24V0BPM6EmKDT/hxdbvvgc/g= 2022-09-03T05:34:48.913 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8HI24V0BPM6EmKDT/hxdbvvgc/g=', 'Content-Length': '0'} 2022-09-03T05:34:48.916 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000c8b958265db28d02-006312e778-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:48.917 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:48.918 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:48.919 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:48.920 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:48.920 DEBUG:boto:Method: GET 2022-09-03T05:34:48.921 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:48.922 DEBUG:boto:Data: 2022-09-03T05:34:48.923 DEBUG:boto:Headers: {} 2022-09-03T05:34:48.924 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:48.925 DEBUG:boto:Port: 8000 2022-09-03T05:34:48.926 DEBUG:boto:Params: {} 2022-09-03T05:34:48.927 DEBUG:boto:Token: None 2022-09-03T05:34:48.928 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:48 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:48.929 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8HI24V0BPM6EmKDT/hxdbvvgc/g= 2022-09-03T05:34:48.930 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8HI24V0BPM6EmKDT/hxdbvvgc/g=', 'Content-Length': '0'} 2022-09-03T05:34:48.933 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000cf80d945ea710591-006312e778-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:48.934 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:48.934 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:48.935 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:48.935 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:48.936 DEBUG:boto:Method: HEAD 2022-09-03T05:34:48.937 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:48.938 DEBUG:boto:Data: 2022-09-03T05:34:48.939 DEBUG:boto:Headers: {} 2022-09-03T05:34:48.940 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:48.941 DEBUG:boto:Port: 8000 2022-09-03T05:34:48.942 DEBUG:boto:Params: {} 2022-09-03T05:34:48.943 DEBUG:boto:Token: None 2022-09-03T05:34:48.944 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:48 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:48.945 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mNWrhLjUaILteTAiaPaqnOEk3p4= 2022-09-03T05:34:48.946 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mNWrhLjUaILteTAiaPaqnOEk3p4=', 'Content-Length': '0'} 2022-09-03T05:34:48.948 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000088041bd86b722d24-006312e778-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:48.949 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:48.950 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:48.952 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:48.952 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:48.953 DEBUG:boto:Method: HEAD 2022-09-03T05:34:48.954 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:48.958 DEBUG:boto:Data: 2022-09-03T05:34:48.959 DEBUG:boto:Headers: {} 2022-09-03T05:34:48.960 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:48.962 DEBUG:boto:Port: 8000 2022-09-03T05:34:48.963 DEBUG:boto:Params: {} 2022-09-03T05:34:48.964 DEBUG:boto:Token: None 2022-09-03T05:34:48.965 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:48 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:48.966 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mNWrhLjUaILteTAiaPaqnOEk3p4= 2022-09-03T05:34:48.967 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mNWrhLjUaILteTAiaPaqnOEk3p4=', 'Content-Length': '0'} 2022-09-03T05:34:48.969 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000408193fcf430453f-006312e778-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:48.970 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:34:48.971 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:48.972 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:48.972 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:48.973 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:48.974 DEBUG:boto:Method: GET 2022-09-03T05:34:48.975 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:48.976 DEBUG:boto:Data: 2022-09-03T05:34:48.977 DEBUG:boto:Headers: {} 2022-09-03T05:34:48.978 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:48.979 DEBUG:boto:Port: 8000 2022-09-03T05:34:48.980 DEBUG:boto:Params: {} 2022-09-03T05:34:48.981 DEBUG:boto:Token: None 2022-09-03T05:34:48.982 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:48 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:48.983 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8HI24V0BPM6EmKDT/hxdbvvgc/g= 2022-09-03T05:34:48.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8HI24V0BPM6EmKDT/hxdbvvgc/g=', 'Content-Length': '0'} 2022-09-03T05:34:48.987 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000005cd344d4714c08bc-006312e778-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:48.988 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:48.990 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:48.991 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:48.992 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:48.993 DEBUG:boto:Method: GET 2022-09-03T05:34:48.994 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:48.994 DEBUG:boto:Data: 2022-09-03T05:34:48.995 DEBUG:boto:Headers: {} 2022-09-03T05:34:48.996 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:48.997 DEBUG:boto:Port: 8000 2022-09-03T05:34:48.998 DEBUG:boto:Params: {} 2022-09-03T05:34:48.999 DEBUG:boto:Token: None 2022-09-03T05:34:49.001 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.002 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c= 2022-09-03T05:34:49.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c=', 'Content-Length': '0'} 2022-09-03T05:34:49.006 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000057bad0eff4ec95f5-006312e779-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.007 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.008 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.009 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.011 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.012 DEBUG:boto:Method: GET 2022-09-03T05:34:49.012 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.013 DEBUG:boto:Data: 2022-09-03T05:34:49.013 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.014 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:49.015 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.016 DEBUG:boto:Params: {} 2022-09-03T05:34:49.016 DEBUG:boto:Token: None 2022-09-03T05:34:49.017 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/ 2022-09-03T05:34:49.017 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QkieLvGLyVS84BMqA1sPgfzustQ= 2022-09-03T05:34:49.018 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QkieLvGLyVS84BMqA1sPgfzustQ=', 'Content-Length': '0'} 2022-09-03T05:34:49.024 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000798f34696b38e7ed-006312e779-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.062 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:49.064 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.065 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.066 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.069 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.070 DEBUG:boto:Method: GET 2022-09-03T05:34:49.070 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.071 DEBUG:boto:Data: 2022-09-03T05:34:49.072 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.072 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:49.073 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.073 DEBUG:boto:Params: {} 2022-09-03T05:34:49.074 DEBUG:boto:Token: None 2022-09-03T05:34:49.075 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/ 2022-09-03T05:34:49.075 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QkieLvGLyVS84BMqA1sPgfzustQ= 2022-09-03T05:34:49.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QkieLvGLyVS84BMqA1sPgfzustQ=', 'Content-Length': '0'} 2022-09-03T05:34:49.079 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000061ac15e44af3ac55-006312e779-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.080 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:49.081 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.082 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.083 DEBUG:boto:Method: HEAD 2022-09-03T05:34:49.083 DEBUG:boto:Path: /dsgvcy-19/dummy 2022-09-03T05:34:49.084 DEBUG:boto:Data: 2022-09-03T05:34:49.085 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.086 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:49.087 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.088 DEBUG:boto:Params: {} 2022-09-03T05:34:49.089 DEBUG:boto:Token: None 2022-09-03T05:34:49.090 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy 2022-09-03T05:34:49.091 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vc1IeqoOBRXNNqQ054AXL/bYNhA= 2022-09-03T05:34:49.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vc1IeqoOBRXNNqQ054AXL/bYNhA=', 'Content-Length': '0'} 2022-09-03T05:34:49.095 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000be9950750c803329-006312e779-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.097 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.098 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.100 DEBUG:boto:Method: HEAD 2022-09-03T05:34:49.101 DEBUG:boto:Path: /dsgvcy-19/dummy 2022-09-03T05:34:49.101 DEBUG:boto:Data: 2022-09-03T05:34:49.102 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.103 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:49.105 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.106 DEBUG:boto:Params: {} 2022-09-03T05:34:49.107 DEBUG:boto:Token: None 2022-09-03T05:34:49.109 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy 2022-09-03T05:34:49.110 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vc1IeqoOBRXNNqQ054AXL/bYNhA= 2022-09-03T05:34:49.111 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vc1IeqoOBRXNNqQ054AXL/bYNhA=', 'Content-Length': '0'} 2022-09-03T05:34:49.116 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000000c51471826e02338-006312e779-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.117 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:34:49.118 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.119 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.120 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.121 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.122 DEBUG:boto:Method: GET 2022-09-03T05:34:49.124 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.126 DEBUG:boto:Data: 2022-09-03T05:34:49.127 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.128 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:49.129 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.129 DEBUG:boto:Params: {} 2022-09-03T05:34:49.130 DEBUG:boto:Token: None 2022-09-03T05:34:49.131 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.132 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c= 2022-09-03T05:34:49.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c=', 'Content-Length': '0'} 2022-09-03T05:34:49.135 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000233c11cf6c7aba45-006312e779-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.136 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.137 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.137 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.138 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.139 DEBUG:boto:Method: GET 2022-09-03T05:34:49.140 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.141 DEBUG:boto:Data: 2022-09-03T05:34:49.142 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.143 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:49.144 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.145 DEBUG:boto:Params: {} 2022-09-03T05:34:49.147 DEBUG:boto:Token: None 2022-09-03T05:34:49.148 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.150 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c= 2022-09-03T05:34:49.151 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c=', 'Content-Length': '0'} 2022-09-03T05:34:49.154 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000065e0e08385e5e141-006312e779-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.155 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.156 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.157 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.158 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.159 DEBUG:boto:Method: GET 2022-09-03T05:34:49.159 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.160 DEBUG:boto:Data: 2022-09-03T05:34:49.161 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.161 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:49.162 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.163 DEBUG:boto:Params: {} 2022-09-03T05:34:49.163 DEBUG:boto:Token: None 2022-09-03T05:34:49.164 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/ 2022-09-03T05:34:49.164 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QkieLvGLyVS84BMqA1sPgfzustQ= 2022-09-03T05:34:49.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QkieLvGLyVS84BMqA1sPgfzustQ=', 'Content-Length': '0'} 2022-09-03T05:34:49.168 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000168842df3d9c0bde-006312e779-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.168 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:49.169 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.170 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.171 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.172 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.172 DEBUG:boto:Method: GET 2022-09-03T05:34:49.173 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.173 DEBUG:boto:Data: 2022-09-03T05:34:49.174 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.174 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:49.175 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.176 DEBUG:boto:Params: {} 2022-09-03T05:34:49.177 DEBUG:boto:Token: None 2022-09-03T05:34:49.178 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/ 2022-09-03T05:34:49.178 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QkieLvGLyVS84BMqA1sPgfzustQ= 2022-09-03T05:34:49.179 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QkieLvGLyVS84BMqA1sPgfzustQ=', 'Content-Length': '0'} 2022-09-03T05:34:49.182 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d00ac86ba7b05775-006312e779-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.226 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:49.229 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-19 zones={test-zone2, test-zone1} 2022-09-03T05:34:49.230 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-19 zones={test-zone3, test-zone1} 2022-09-03T05:34:49.231 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.233 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.234 DEBUG:boto:Method: HEAD 2022-09-03T05:34:49.235 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:49.236 DEBUG:boto:Data: 2022-09-03T05:34:49.237 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.238 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:49.238 DEBUG:boto:Port: 8001 2022-09-03T05:34:49.239 DEBUG:boto:Params: {} 2022-09-03T05:34:49.240 DEBUG:boto:Token: None 2022-09-03T05:34:49.241 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/ 2022-09-03T05:34:49.242 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3hjBZNzZVW6bOzCAQn8/Hau/iwo= 2022-09-03T05:34:49.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3hjBZNzZVW6bOzCAQn8/Hau/iwo=', 'Content-Length': '0'} 2022-09-03T05:34:49.251 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003fd202e0ea2ebccc-006312e779-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.252 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.252 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.253 DEBUG:boto:Method: HEAD 2022-09-03T05:34:49.253 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:49.254 DEBUG:boto:Data: 2022-09-03T05:34:49.255 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.256 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:49.257 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.258 DEBUG:boto:Params: {} 2022-09-03T05:34:49.261 DEBUG:boto:Token: None 2022-09-03T05:34:49.262 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/ 2022-09-03T05:34:49.262 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3hjBZNzZVW6bOzCAQn8/Hau/iwo= 2022-09-03T05:34:49.263 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3hjBZNzZVW6bOzCAQn8/Hau/iwo=', 'Content-Length': '0'} 2022-09-03T05:34:49.266 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e0cf1749f4cb1079-006312e779-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.267 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:34:49.269 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.269 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.270 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.270 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.271 DEBUG:boto:Method: GET 2022-09-03T05:34:49.271 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.272 DEBUG:boto:Data: 2022-09-03T05:34:49.275 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.275 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:49.276 DEBUG:boto:Port: 8001 2022-09-03T05:34:49.276 DEBUG:boto:Params: {} 2022-09-03T05:34:49.277 DEBUG:boto:Token: None 2022-09-03T05:34:49.278 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/?versions 2022-09-03T05:34:49.279 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iMUj6O3klgTaG7Vn0bwuZBTv3rs= 2022-09-03T05:34:49.279 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iMUj6O3klgTaG7Vn0bwuZBTv3rs=', 'Content-Length': '0'} 2022-09-03T05:34:49.284 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001720b1af6a60849a-006312e779-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.330 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:49.331 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:34:49.332 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:34:49.332 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.333 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.333 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.334 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.334 DEBUG:boto:Method: GET 2022-09-03T05:34:49.335 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.335 DEBUG:boto:Data: 2022-09-03T05:34:49.336 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.336 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:49.337 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.337 DEBUG:boto:Params: {} 2022-09-03T05:34:49.338 DEBUG:boto:Token: None 2022-09-03T05:34:49.339 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/?versions 2022-09-03T05:34:49.339 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iMUj6O3klgTaG7Vn0bwuZBTv3rs= 2022-09-03T05:34:49.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iMUj6O3klgTaG7Vn0bwuZBTv3rs=', 'Content-Length': '0'} 2022-09-03T05:34:49.343 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e18b14af5dcd4111-006312e779-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.344 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:49.345 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:34:49.345 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.346 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.347 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.347 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.348 DEBUG:boto:Method: GET 2022-09-03T05:34:49.348 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.349 DEBUG:boto:Data: 2022-09-03T05:34:49.350 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.350 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:49.351 DEBUG:boto:Port: 8001 2022-09-03T05:34:49.352 DEBUG:boto:Params: {} 2022-09-03T05:34:49.352 DEBUG:boto:Token: None 2022-09-03T05:34:49.353 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/?versions 2022-09-03T05:34:49.353 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iMUj6O3klgTaG7Vn0bwuZBTv3rs= 2022-09-03T05:34:49.354 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iMUj6O3klgTaG7Vn0bwuZBTv3rs=', 'Content-Length': '0'} 2022-09-03T05:34:49.357 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000053b8444e7e3946a9-006312e779-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.358 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:49.359 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.360 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.360 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.361 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.361 DEBUG:boto:Method: GET 2022-09-03T05:34:49.362 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.362 DEBUG:boto:Data: 2022-09-03T05:34:49.363 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.363 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:49.364 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.364 DEBUG:boto:Params: {} 2022-09-03T05:34:49.365 DEBUG:boto:Token: None 2022-09-03T05:34:49.366 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/?versions 2022-09-03T05:34:49.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iMUj6O3klgTaG7Vn0bwuZBTv3rs= 2022-09-03T05:34:49.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iMUj6O3klgTaG7Vn0bwuZBTv3rs=', 'Content-Length': '0'} 2022-09-03T05:34:49.370 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000028b5d1cdceeb7af4-006312e779-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.371 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:49.372 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:34:49.372 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.373 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.373 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.374 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.374 DEBUG:boto:Method: GET 2022-09-03T05:34:49.375 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.375 DEBUG:boto:Data: 2022-09-03T05:34:49.376 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.376 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:49.377 DEBUG:boto:Port: 8001 2022-09-03T05:34:49.378 DEBUG:boto:Params: {} 2022-09-03T05:34:49.378 DEBUG:boto:Token: None 2022-09-03T05:34:49.379 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.380 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c= 2022-09-03T05:34:49.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c=', 'Content-Length': '0'} 2022-09-03T05:34:49.382 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000001c40fb92eb85501f-006312e779-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.383 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.384 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.384 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.385 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.386 DEBUG:boto:Method: GET 2022-09-03T05:34:49.386 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.387 DEBUG:boto:Data: 2022-09-03T05:34:49.387 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.388 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:49.388 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.389 DEBUG:boto:Params: {} 2022-09-03T05:34:49.390 DEBUG:boto:Token: None 2022-09-03T05:34:49.391 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.391 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c= 2022-09-03T05:34:49.392 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c=', 'Content-Length': '0'} 2022-09-03T05:34:49.394 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000700194d8afa18b36-006312e779-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.395 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.396 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.396 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.397 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.397 DEBUG:boto:Method: HEAD 2022-09-03T05:34:49.398 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.398 DEBUG:boto:Data: 2022-09-03T05:34:49.399 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.400 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:49.401 DEBUG:boto:Port: 8001 2022-09-03T05:34:49.402 DEBUG:boto:Params: {} 2022-09-03T05:34:49.403 DEBUG:boto:Token: None 2022-09-03T05:34:49.404 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.405 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:80UcD3+8TqKiSP39+Nc/gpsSLeI= 2022-09-03T05:34:49.406 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:80UcD3+8TqKiSP39+Nc/gpsSLeI=', 'Content-Length': '0'} 2022-09-03T05:34:49.409 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000007165bc21d6af8463-006312e779-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.410 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.411 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.412 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.413 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.414 DEBUG:boto:Method: HEAD 2022-09-03T05:34:49.415 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.416 DEBUG:boto:Data: 2022-09-03T05:34:49.417 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.418 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:49.419 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.420 DEBUG:boto:Params: {} 2022-09-03T05:34:49.421 DEBUG:boto:Token: None 2022-09-03T05:34:49.421 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.422 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:80UcD3+8TqKiSP39+Nc/gpsSLeI= 2022-09-03T05:34:49.423 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:80UcD3+8TqKiSP39+Nc/gpsSLeI=', 'Content-Length': '0'} 2022-09-03T05:34:49.425 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f76ece92b9c0bf6e-006312e779-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.431 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:34:49.432 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.433 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.434 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.434 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.435 DEBUG:boto:Method: GET 2022-09-03T05:34:49.436 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.437 DEBUG:boto:Data: 2022-09-03T05:34:49.438 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.439 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:49.440 DEBUG:boto:Port: 8001 2022-09-03T05:34:49.441 DEBUG:boto:Params: {} 2022-09-03T05:34:49.441 DEBUG:boto:Token: None 2022-09-03T05:34:49.442 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.443 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c= 2022-09-03T05:34:49.443 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c=', 'Content-Length': '0'} 2022-09-03T05:34:49.446 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000911789a17c3dcd8d-006312e779-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.447 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.448 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.449 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.450 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.451 DEBUG:boto:Method: GET 2022-09-03T05:34:49.452 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.453 DEBUG:boto:Data: 2022-09-03T05:34:49.453 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.454 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:49.455 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.455 DEBUG:boto:Params: {} 2022-09-03T05:34:49.456 DEBUG:boto:Token: None 2022-09-03T05:34:49.457 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.457 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c= 2022-09-03T05:34:49.458 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c=', 'Content-Length': '0'} 2022-09-03T05:34:49.460 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000004e2557dab2c18b69-006312e779-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.461 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.462 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.462 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.463 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.464 DEBUG:boto:Method: GET 2022-09-03T05:34:49.465 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.466 DEBUG:boto:Data: 2022-09-03T05:34:49.467 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.468 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:49.469 DEBUG:boto:Port: 8001 2022-09-03T05:34:49.470 DEBUG:boto:Params: {} 2022-09-03T05:34:49.471 DEBUG:boto:Token: None 2022-09-03T05:34:49.472 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/ 2022-09-03T05:34:49.473 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QkieLvGLyVS84BMqA1sPgfzustQ= 2022-09-03T05:34:49.474 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QkieLvGLyVS84BMqA1sPgfzustQ=', 'Content-Length': '0'} 2022-09-03T05:34:49.478 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008413a232b9af0d1d-006312e779-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.479 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:49.480 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.481 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.482 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.483 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.484 DEBUG:boto:Method: GET 2022-09-03T05:34:49.485 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.486 DEBUG:boto:Data: 2022-09-03T05:34:49.487 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.488 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:49.489 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.490 DEBUG:boto:Params: {} 2022-09-03T05:34:49.491 DEBUG:boto:Token: None 2022-09-03T05:34:49.491 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/ 2022-09-03T05:34:49.492 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QkieLvGLyVS84BMqA1sPgfzustQ= 2022-09-03T05:34:49.493 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QkieLvGLyVS84BMqA1sPgfzustQ=', 'Content-Length': '0'} 2022-09-03T05:34:49.495 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000995d3c1a0ab7165e-006312e779-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.539 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:49.540 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.541 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.542 DEBUG:boto:Method: HEAD 2022-09-03T05:34:49.543 DEBUG:boto:Path: /dsgvcy-19/dummy 2022-09-03T05:34:49.544 DEBUG:boto:Data: 2022-09-03T05:34:49.545 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.546 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:49.547 DEBUG:boto:Port: 8001 2022-09-03T05:34:49.548 DEBUG:boto:Params: {} 2022-09-03T05:34:49.549 DEBUG:boto:Token: None 2022-09-03T05:34:49.550 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy 2022-09-03T05:34:49.551 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vc1IeqoOBRXNNqQ054AXL/bYNhA= 2022-09-03T05:34:49.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vc1IeqoOBRXNNqQ054AXL/bYNhA=', 'Content-Length': '0'} 2022-09-03T05:34:49.554 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000002c585e9309f14046-006312e779-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.555 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.556 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.556 DEBUG:boto:Method: HEAD 2022-09-03T05:34:49.557 DEBUG:boto:Path: /dsgvcy-19/dummy 2022-09-03T05:34:49.557 DEBUG:boto:Data: 2022-09-03T05:34:49.558 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.559 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:49.560 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.561 DEBUG:boto:Params: {} 2022-09-03T05:34:49.562 DEBUG:boto:Token: None 2022-09-03T05:34:49.563 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy 2022-09-03T05:34:49.564 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vc1IeqoOBRXNNqQ054AXL/bYNhA= 2022-09-03T05:34:49.566 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vc1IeqoOBRXNNqQ054AXL/bYNhA=', 'Content-Length': '0'} 2022-09-03T05:34:49.570 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000e4a3e56075c46c0e-006312e779-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.572 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:34:49.573 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.574 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.575 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.576 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.577 DEBUG:boto:Method: GET 2022-09-03T05:34:49.578 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.579 DEBUG:boto:Data: 2022-09-03T05:34:49.580 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.581 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:49.581 DEBUG:boto:Port: 8001 2022-09-03T05:34:49.582 DEBUG:boto:Params: {} 2022-09-03T05:34:49.583 DEBUG:boto:Token: None 2022-09-03T05:34:49.583 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.584 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c= 2022-09-03T05:34:49.585 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c=', 'Content-Length': '0'} 2022-09-03T05:34:49.587 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000072e96db22a9d8990-006312e779-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.588 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.589 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.589 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.590 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.590 DEBUG:boto:Method: GET 2022-09-03T05:34:49.591 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.591 DEBUG:boto:Data: 2022-09-03T05:34:49.592 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.593 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:49.593 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.594 DEBUG:boto:Params: {} 2022-09-03T05:34:49.595 DEBUG:boto:Token: None 2022-09-03T05:34:49.596 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.597 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c= 2022-09-03T05:34:49.599 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c=', 'Content-Length': '0'} 2022-09-03T05:34:49.602 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000ee1b4e29cce7dc83-006312e779-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.603 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.604 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.605 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.606 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.607 DEBUG:boto:Method: GET 2022-09-03T05:34:49.608 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.609 DEBUG:boto:Data: 2022-09-03T05:34:49.610 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.611 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:49.612 DEBUG:boto:Port: 8001 2022-09-03T05:34:49.612 DEBUG:boto:Params: {} 2022-09-03T05:34:49.614 DEBUG:boto:Token: None 2022-09-03T05:34:49.615 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/ 2022-09-03T05:34:49.616 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QkieLvGLyVS84BMqA1sPgfzustQ= 2022-09-03T05:34:49.617 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QkieLvGLyVS84BMqA1sPgfzustQ=', 'Content-Length': '0'} 2022-09-03T05:34:49.621 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000026bc58c493dee154-006312e779-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.669 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:49.670 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.670 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.671 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.671 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.672 DEBUG:boto:Method: GET 2022-09-03T05:34:49.673 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.673 DEBUG:boto:Data: 2022-09-03T05:34:49.674 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.674 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:49.675 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.675 DEBUG:boto:Params: {} 2022-09-03T05:34:49.676 DEBUG:boto:Token: None 2022-09-03T05:34:49.677 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/ 2022-09-03T05:34:49.677 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QkieLvGLyVS84BMqA1sPgfzustQ= 2022-09-03T05:34:49.678 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QkieLvGLyVS84BMqA1sPgfzustQ=', 'Content-Length': '0'} 2022-09-03T05:34:49.681 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008391d64828d11b4f-006312e779-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.682 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:49.683 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-19 zones={test-zone3, test-zone1} 2022-09-03T05:34:49.683 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-19 zones={test-zone3, test-zone2} 2022-09-03T05:34:49.684 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.684 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.685 DEBUG:boto:Method: HEAD 2022-09-03T05:34:49.685 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:49.686 DEBUG:boto:Data: 2022-09-03T05:34:49.686 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.687 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:49.687 DEBUG:boto:Port: 8001 2022-09-03T05:34:49.688 DEBUG:boto:Params: {} 2022-09-03T05:34:49.688 DEBUG:boto:Token: None 2022-09-03T05:34:49.689 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/ 2022-09-03T05:34:49.690 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3hjBZNzZVW6bOzCAQn8/Hau/iwo= 2022-09-03T05:34:49.690 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3hjBZNzZVW6bOzCAQn8/Hau/iwo=', 'Content-Length': '0'} 2022-09-03T05:34:49.693 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000087a6670c5cd74426-006312e779-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.694 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.695 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.696 DEBUG:boto:Method: HEAD 2022-09-03T05:34:49.696 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:49.697 DEBUG:boto:Data: 2022-09-03T05:34:49.698 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.698 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:49.699 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.699 DEBUG:boto:Params: {} 2022-09-03T05:34:49.700 DEBUG:boto:Token: None 2022-09-03T05:34:49.700 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/ 2022-09-03T05:34:49.701 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3hjBZNzZVW6bOzCAQn8/Hau/iwo= 2022-09-03T05:34:49.702 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3hjBZNzZVW6bOzCAQn8/Hau/iwo=', 'Content-Length': '0'} 2022-09-03T05:34:49.705 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004fdb1305b010a898-006312e779-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.706 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:34:49.706 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.707 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.708 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.708 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.709 DEBUG:boto:Method: GET 2022-09-03T05:34:49.710 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.711 DEBUG:boto:Data: 2022-09-03T05:34:49.711 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.712 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:49.712 DEBUG:boto:Port: 8001 2022-09-03T05:34:49.713 DEBUG:boto:Params: {} 2022-09-03T05:34:49.714 DEBUG:boto:Token: None 2022-09-03T05:34:49.715 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/?versions 2022-09-03T05:34:49.716 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iMUj6O3klgTaG7Vn0bwuZBTv3rs= 2022-09-03T05:34:49.717 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iMUj6O3klgTaG7Vn0bwuZBTv3rs=', 'Content-Length': '0'} 2022-09-03T05:34:49.721 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d81ac7e22c0a1d21-006312e779-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.722 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:49.724 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:34:49.725 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:34:49.726 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.726 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.727 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.727 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.728 DEBUG:boto:Method: GET 2022-09-03T05:34:49.729 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.729 DEBUG:boto:Data: 2022-09-03T05:34:49.730 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.730 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:49.731 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.731 DEBUG:boto:Params: {} 2022-09-03T05:34:49.732 DEBUG:boto:Token: None 2022-09-03T05:34:49.733 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/?versions 2022-09-03T05:34:49.733 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iMUj6O3klgTaG7Vn0bwuZBTv3rs= 2022-09-03T05:34:49.734 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iMUj6O3klgTaG7Vn0bwuZBTv3rs=', 'Content-Length': '0'} 2022-09-03T05:34:49.737 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b96d03bce26f9705-006312e779-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.737 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:49.738 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:34:49.739 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.739 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.740 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.740 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.741 DEBUG:boto:Method: GET 2022-09-03T05:34:49.742 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.743 DEBUG:boto:Data: 2022-09-03T05:34:49.743 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.744 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:49.745 DEBUG:boto:Port: 8001 2022-09-03T05:34:49.745 DEBUG:boto:Params: {} 2022-09-03T05:34:49.746 DEBUG:boto:Token: None 2022-09-03T05:34:49.747 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/?versions 2022-09-03T05:34:49.747 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iMUj6O3klgTaG7Vn0bwuZBTv3rs= 2022-09-03T05:34:49.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iMUj6O3klgTaG7Vn0bwuZBTv3rs=', 'Content-Length': '0'} 2022-09-03T05:34:49.751 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005ff6fea221b42b81-006312e779-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.752 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:49.752 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.753 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.754 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.754 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.755 DEBUG:boto:Method: GET 2022-09-03T05:34:49.755 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:34:49.756 DEBUG:boto:Data: 2022-09-03T05:34:49.757 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.757 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:49.758 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.759 DEBUG:boto:Params: {} 2022-09-03T05:34:49.759 DEBUG:boto:Token: None 2022-09-03T05:34:49.760 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/?versions 2022-09-03T05:34:49.760 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iMUj6O3klgTaG7Vn0bwuZBTv3rs= 2022-09-03T05:34:49.762 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iMUj6O3klgTaG7Vn0bwuZBTv3rs=', 'Content-Length': '0'} 2022-09-03T05:34:49.765 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007cb1192bab1492dd-006312e779-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.767 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:49.768 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:34:49.770 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.771 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.771 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.772 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.774 DEBUG:boto:Method: GET 2022-09-03T05:34:49.775 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.775 DEBUG:boto:Data: 2022-09-03T05:34:49.776 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.776 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:49.777 DEBUG:boto:Port: 8001 2022-09-03T05:34:49.778 DEBUG:boto:Params: {} 2022-09-03T05:34:49.780 DEBUG:boto:Token: None 2022-09-03T05:34:49.780 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.781 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c= 2022-09-03T05:34:49.782 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c=', 'Content-Length': '0'} 2022-09-03T05:34:49.784 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000007cc55ee890a07294-006312e779-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.785 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.785 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.786 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.786 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.787 DEBUG:boto:Method: GET 2022-09-03T05:34:49.787 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.788 DEBUG:boto:Data: 2022-09-03T05:34:49.788 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.789 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:49.790 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.791 DEBUG:boto:Params: {} 2022-09-03T05:34:49.791 DEBUG:boto:Token: None 2022-09-03T05:34:49.792 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.793 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c= 2022-09-03T05:34:49.793 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c=', 'Content-Length': '0'} 2022-09-03T05:34:49.795 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000b8c06b21de742886-006312e779-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.796 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.798 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.799 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.800 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.800 DEBUG:boto:Method: HEAD 2022-09-03T05:34:49.801 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.802 DEBUG:boto:Data: 2022-09-03T05:34:49.803 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.804 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:49.805 DEBUG:boto:Port: 8001 2022-09-03T05:34:49.806 DEBUG:boto:Params: {} 2022-09-03T05:34:49.807 DEBUG:boto:Token: None 2022-09-03T05:34:49.808 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.809 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:80UcD3+8TqKiSP39+Nc/gpsSLeI= 2022-09-03T05:34:49.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:80UcD3+8TqKiSP39+Nc/gpsSLeI=', 'Content-Length': '0'} 2022-09-03T05:34:49.812 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000257d19d6da62e540-006312e779-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.812 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.813 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.813 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.814 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.815 DEBUG:boto:Method: HEAD 2022-09-03T05:34:49.815 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.816 DEBUG:boto:Data: 2022-09-03T05:34:49.817 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.818 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:49.819 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.819 DEBUG:boto:Params: {} 2022-09-03T05:34:49.820 DEBUG:boto:Token: None 2022-09-03T05:34:49.821 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.821 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:80UcD3+8TqKiSP39+Nc/gpsSLeI= 2022-09-03T05:34:49.822 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:80UcD3+8TqKiSP39+Nc/gpsSLeI=', 'Content-Length': '0'} 2022-09-03T05:34:49.834 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000445b7ae89257cd6b-006312e779-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.835 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:34:49.836 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.836 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.837 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.837 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.838 DEBUG:boto:Method: GET 2022-09-03T05:34:49.839 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.839 DEBUG:boto:Data: 2022-09-03T05:34:49.840 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.840 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:49.841 DEBUG:boto:Port: 8001 2022-09-03T05:34:49.841 DEBUG:boto:Params: {} 2022-09-03T05:34:49.842 DEBUG:boto:Token: None 2022-09-03T05:34:49.843 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.844 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c= 2022-09-03T05:34:49.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c=', 'Content-Length': '0'} 2022-09-03T05:34:49.846 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000b6ae38b1f5624206-006312e779-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.847 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.848 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.848 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.849 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.849 DEBUG:boto:Method: GET 2022-09-03T05:34:49.850 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.850 DEBUG:boto:Data: 2022-09-03T05:34:49.851 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.852 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:49.852 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.853 DEBUG:boto:Params: {} 2022-09-03T05:34:49.854 DEBUG:boto:Token: None 2022-09-03T05:34:49.855 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.856 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c= 2022-09-03T05:34:49.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c=', 'Content-Length': '0'} 2022-09-03T05:34:49.859 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000eb5945f6b69f204b-006312e779-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.860 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.860 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.861 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.861 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.862 DEBUG:boto:Method: GET 2022-09-03T05:34:49.862 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.863 DEBUG:boto:Data: 2022-09-03T05:34:49.864 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.865 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:49.866 DEBUG:boto:Port: 8001 2022-09-03T05:34:49.867 DEBUG:boto:Params: {} 2022-09-03T05:34:49.868 DEBUG:boto:Token: None 2022-09-03T05:34:49.869 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/ 2022-09-03T05:34:49.870 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QkieLvGLyVS84BMqA1sPgfzustQ= 2022-09-03T05:34:49.871 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QkieLvGLyVS84BMqA1sPgfzustQ=', 'Content-Length': '0'} 2022-09-03T05:34:49.874 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000376073785cf8c66e-006312e779-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.914 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:49.916 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.917 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.918 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.918 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.919 DEBUG:boto:Method: GET 2022-09-03T05:34:49.919 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.920 DEBUG:boto:Data: 2022-09-03T05:34:49.920 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.921 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:49.922 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.922 DEBUG:boto:Params: {} 2022-09-03T05:34:49.923 DEBUG:boto:Token: None 2022-09-03T05:34:49.923 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/ 2022-09-03T05:34:49.924 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QkieLvGLyVS84BMqA1sPgfzustQ= 2022-09-03T05:34:49.925 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QkieLvGLyVS84BMqA1sPgfzustQ=', 'Content-Length': '0'} 2022-09-03T05:34:49.927 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aa4d42d14388ff49-006312e779-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.928 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:49.929 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.929 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.930 DEBUG:boto:Method: HEAD 2022-09-03T05:34:49.930 DEBUG:boto:Path: /dsgvcy-19/dummy 2022-09-03T05:34:49.931 DEBUG:boto:Data: 2022-09-03T05:34:49.932 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.932 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:49.933 DEBUG:boto:Port: 8001 2022-09-03T05:34:49.933 DEBUG:boto:Params: {} 2022-09-03T05:34:49.934 DEBUG:boto:Token: None 2022-09-03T05:34:49.934 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy 2022-09-03T05:34:49.935 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vc1IeqoOBRXNNqQ054AXL/bYNhA= 2022-09-03T05:34:49.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vc1IeqoOBRXNNqQ054AXL/bYNhA=', 'Content-Length': '0'} 2022-09-03T05:34:49.938 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000279be5f639be0708-006312e779-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.939 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.940 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.940 DEBUG:boto:Method: HEAD 2022-09-03T05:34:49.941 DEBUG:boto:Path: /dsgvcy-19/dummy 2022-09-03T05:34:49.941 DEBUG:boto:Data: 2022-09-03T05:34:49.942 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.942 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:49.943 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.943 DEBUG:boto:Params: {} 2022-09-03T05:34:49.944 DEBUG:boto:Token: None 2022-09-03T05:34:49.945 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy 2022-09-03T05:34:49.945 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vc1IeqoOBRXNNqQ054AXL/bYNhA= 2022-09-03T05:34:49.946 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vc1IeqoOBRXNNqQ054AXL/bYNhA=', 'Content-Length': '0'} 2022-09-03T05:34:49.949 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000096ae59b2464eab33-006312e779-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.949 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:34:49.950 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.951 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.951 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.952 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.953 DEBUG:boto:Method: GET 2022-09-03T05:34:49.953 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.954 DEBUG:boto:Data: 2022-09-03T05:34:49.954 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.955 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:49.955 DEBUG:boto:Port: 8001 2022-09-03T05:34:49.956 DEBUG:boto:Params: {} 2022-09-03T05:34:49.956 DEBUG:boto:Token: None 2022-09-03T05:34:49.957 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.958 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c= 2022-09-03T05:34:49.958 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c=', 'Content-Length': '0'} 2022-09-03T05:34:49.960 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000008d78496bf259d9df-006312e779-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.961 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:34:49.962 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:34:49.963 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.963 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.964 DEBUG:boto:Method: GET 2022-09-03T05:34:49.964 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.965 DEBUG:boto:Data: 2022-09-03T05:34:49.966 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.966 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:49.967 DEBUG:boto:Port: 8000 2022-09-03T05:34:49.967 DEBUG:boto:Params: {} 2022-09-03T05:34:49.969 DEBUG:boto:Token: None 2022-09-03T05:34:49.970 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:34:49.971 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c= 2022-09-03T05:34:49.972 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ly6seE7N2h99ou0fVP6BcsJnO4c=', 'Content-Length': '0'} 2022-09-03T05:34:49.976 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000c6bbc87e09e3cac2-006312e779-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.978 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.979 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.980 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.981 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.982 DEBUG:boto:Method: GET 2022-09-03T05:34:49.983 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.984 DEBUG:boto:Data: 2022-09-03T05:34:49.984 DEBUG:boto:Headers: {} 2022-09-03T05:34:49.985 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:49.986 DEBUG:boto:Port: 8001 2022-09-03T05:34:49.987 DEBUG:boto:Params: {} 2022-09-03T05:34:49.988 DEBUG:boto:Token: None 2022-09-03T05:34:49.988 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:49 GMT /dsgvcy-19/ 2022-09-03T05:34:49.989 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QkieLvGLyVS84BMqA1sPgfzustQ= 2022-09-03T05:34:49.990 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QkieLvGLyVS84BMqA1sPgfzustQ=', 'Content-Length': '0'} 2022-09-03T05:34:49.993 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c2cf54dc685e73e7-006312e779-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:49.994 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:49.996 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:49.997 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:49.997 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.998 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.998 DEBUG:boto:Method: GET 2022-09-03T05:34:49.999 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:34:49.999 DEBUG:boto:Data: 2022-09-03T05:34:50.000 DEBUG:boto:Headers: {} 2022-09-03T05:34:50.000 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:50.001 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.001 DEBUG:boto:Params: {} 2022-09-03T05:34:50.002 DEBUG:boto:Token: None 2022-09-03T05:34:50.003 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/ 2022-09-03T05:34:50.003 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6QCziKQlhM1BjXo8/id4HBH8zA8= 2022-09-03T05:34:50.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6QCziKQlhM1BjXo8/id4HBH8zA8=', 'Content-Length': '0'} 2022-09-03T05:34:50.007 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d615062024399154-006312e77a-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.009 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:34:50.011 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-19 zones={test-zone3, test-zone2} 2022-09-03T05:34:50.012 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:50.014 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:50.015 DEBUG:boto:Method: HEAD 2022-09-03T05:34:50.017 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:50.018 DEBUG:boto:Data: 2022-09-03T05:34:50.018 DEBUG:boto:Headers: {} 2022-09-03T05:34:50.019 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:50.020 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.020 DEBUG:boto:Params: {} 2022-09-03T05:34:50.021 DEBUG:boto:Token: None 2022-09-03T05:34:50.021 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/ 2022-09-03T05:34:50.022 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA= 2022-09-03T05:34:50.023 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA=', 'Content-Length': '0'} 2022-09-03T05:34:50.028 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000016c79cc3baebf5ac-006312e77a-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.029 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:50.030 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:50.030 DEBUG:boto:Method: PUT 2022-09-03T05:34:50.031 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:50.031 DEBUG:boto:Data: 2022-09-03T05:34:50.032 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:50.032 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:50.033 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.033 DEBUG:boto:Params: {} 2022-09-03T05:34:50.034 DEBUG:boto:Token: None 2022-09-03T05:34:50.034 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/obj 2022-09-03T05:34:50.035 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E= 2022-09-03T05:34:50.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E='} 2022-09-03T05:34:50.078 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY'), ('x-amz-request-id', 'tx00000bb328900f8b9ea7e-006312e77a-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.080 DEBUG:rgw_multi.tests:zone=test-zone1 version=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:34:50.081 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:50.081 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:50.083 DEBUG:boto:Method: HEAD 2022-09-03T05:34:50.084 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:50.084 DEBUG:boto:Data: 2022-09-03T05:34:50.085 DEBUG:boto:Headers: {} 2022-09-03T05:34:50.085 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:50.086 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.086 DEBUG:boto:Params: {} 2022-09-03T05:34:50.087 DEBUG:boto:Token: None 2022-09-03T05:34:50.088 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/ 2022-09-03T05:34:50.089 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA= 2022-09-03T05:34:50.089 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA=', 'Content-Length': '0'} 2022-09-03T05:34:50.092 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007eb6e8c0b9fbd502-006312e77a-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.094 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:50.096 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:50.097 DEBUG:boto:Method: PUT 2022-09-03T05:34:50.098 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:50.099 DEBUG:boto:Data: 2022-09-03T05:34:50.100 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:50.101 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:50.102 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.103 DEBUG:boto:Params: {} 2022-09-03T05:34:50.104 DEBUG:boto:Token: None 2022-09-03T05:34:50.105 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/obj 2022-09-03T05:34:50.106 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E= 2022-09-03T05:34:50.107 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E='} 2022-09-03T05:34:50.121 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1'), ('x-amz-request-id', 'tx00000a7b2b9246e69eb41-006312e77a-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.122 DEBUG:rgw_multi.tests:zone=test-zone2 version=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:34:50.123 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:50.124 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:50.126 DEBUG:boto:Method: HEAD 2022-09-03T05:34:50.127 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:50.127 DEBUG:boto:Data: 2022-09-03T05:34:50.128 DEBUG:boto:Headers: {} 2022-09-03T05:34:50.130 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:50.131 DEBUG:boto:Port: 8001 2022-09-03T05:34:50.131 DEBUG:boto:Params: {} 2022-09-03T05:34:50.132 DEBUG:boto:Token: None 2022-09-03T05:34:50.133 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/ 2022-09-03T05:34:50.133 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA= 2022-09-03T05:34:50.134 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA=', 'Content-Length': '0'} 2022-09-03T05:34:50.137 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000be9c3cfc0c61a732-006312e77a-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.138 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:50.139 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:50.139 DEBUG:boto:Method: PUT 2022-09-03T05:34:50.140 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:50.141 DEBUG:boto:Data: 2022-09-03T05:34:50.141 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:50.142 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:50.142 DEBUG:boto:Port: 8001 2022-09-03T05:34:50.143 DEBUG:boto:Params: {} 2022-09-03T05:34:50.143 DEBUG:boto:Token: None 2022-09-03T05:34:50.144 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/obj 2022-09-03T05:34:50.144 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E= 2022-09-03T05:34:50.145 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E='} 2022-09-03T05:34:50.162 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'CFKxqazR5g.k3.oWs62bQgs9moHAimI'), ('x-amz-request-id', 'tx000003b3ea770afcbf9bb-006312e77a-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.163 DEBUG:rgw_multi.tests:zone=test-zone3 version=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:34:50.164 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:50.164 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:50.165 DEBUG:boto:Method: HEAD 2022-09-03T05:34:50.165 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:50.166 DEBUG:boto:Data: 2022-09-03T05:34:50.166 DEBUG:boto:Headers: {} 2022-09-03T05:34:50.168 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:50.169 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.170 DEBUG:boto:Params: {} 2022-09-03T05:34:50.171 DEBUG:boto:Token: None 2022-09-03T05:34:50.171 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/ 2022-09-03T05:34:50.172 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA= 2022-09-03T05:34:50.174 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA=', 'Content-Length': '0'} 2022-09-03T05:34:50.177 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '8'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000953abd7131ee3bec-006312e77a-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.178 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:50.178 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:50.179 DEBUG:boto:Method: PUT 2022-09-03T05:34:50.179 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:50.180 DEBUG:boto:Data: 2022-09-03T05:34:50.180 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:50.181 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:50.181 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.182 DEBUG:boto:Params: {} 2022-09-03T05:34:50.183 DEBUG:boto:Token: None 2022-09-03T05:34:50.183 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/obj 2022-09-03T05:34:50.184 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E= 2022-09-03T05:34:50.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E='} 2022-09-03T05:34:50.197 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC'), ('x-amz-request-id', 'tx00000fdb97a1d411e3b6b-006312e77a-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.197 DEBUG:rgw_multi.tests:zone=test-zone1 version=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:34:50.198 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:50.198 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:50.199 DEBUG:boto:Method: HEAD 2022-09-03T05:34:50.199 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:50.200 DEBUG:boto:Data: 2022-09-03T05:34:50.200 DEBUG:boto:Headers: {} 2022-09-03T05:34:50.201 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:50.201 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.202 DEBUG:boto:Params: {} 2022-09-03T05:34:50.203 DEBUG:boto:Token: None 2022-09-03T05:34:50.203 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/ 2022-09-03T05:34:50.204 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA= 2022-09-03T05:34:50.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA=', 'Content-Length': '0'} 2022-09-03T05:34:50.211 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '8'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006ba274593e80e35f-006312e77a-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.213 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:50.215 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:50.216 DEBUG:boto:Method: PUT 2022-09-03T05:34:50.217 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:50.219 DEBUG:boto:Data: 2022-09-03T05:34:50.220 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:50.221 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:50.224 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.225 DEBUG:boto:Params: {} 2022-09-03T05:34:50.226 DEBUG:boto:Token: None 2022-09-03T05:34:50.226 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/obj 2022-09-03T05:34:50.227 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E= 2022-09-03T05:34:50.228 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E='} 2022-09-03T05:34:50.239 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'oYY-icmxg74gZ4carPTk38at9fTxjJi'), ('x-amz-request-id', 'tx00000eb5ab0aef0a46322-006312e77a-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.240 DEBUG:rgw_multi.tests:zone=test-zone2 version=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:34:50.241 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:50.242 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:50.242 DEBUG:boto:Method: HEAD 2022-09-03T05:34:50.243 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:50.244 DEBUG:boto:Data: 2022-09-03T05:34:50.245 DEBUG:boto:Headers: {} 2022-09-03T05:34:50.245 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:50.246 DEBUG:boto:Port: 8001 2022-09-03T05:34:50.246 DEBUG:boto:Params: {} 2022-09-03T05:34:50.247 DEBUG:boto:Token: None 2022-09-03T05:34:50.248 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/ 2022-09-03T05:34:50.249 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA= 2022-09-03T05:34:50.250 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA=', 'Content-Length': '0'} 2022-09-03T05:34:50.256 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '8'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001767c34f5cf07d23-006312e77a-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.257 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:50.258 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:50.259 DEBUG:boto:Method: PUT 2022-09-03T05:34:50.260 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:50.261 DEBUG:boto:Data: 2022-09-03T05:34:50.262 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:50.263 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:50.264 DEBUG:boto:Port: 8001 2022-09-03T05:34:50.265 DEBUG:boto:Params: {} 2022-09-03T05:34:50.266 DEBUG:boto:Token: None 2022-09-03T05:34:50.267 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/obj 2022-09-03T05:34:50.268 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E= 2022-09-03T05:34:50.269 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E='} 2022-09-03T05:34:50.315 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A'), ('x-amz-request-id', 'tx000006a314204e707599d-006312e77a-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.316 DEBUG:rgw_multi.tests:zone=test-zone3 version=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:34:50.317 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:50.317 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:50.318 DEBUG:boto:Method: HEAD 2022-09-03T05:34:50.319 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:50.319 DEBUG:boto:Data: 2022-09-03T05:34:50.320 DEBUG:boto:Headers: {} 2022-09-03T05:34:50.320 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:50.321 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.321 DEBUG:boto:Params: {} 2022-09-03T05:34:50.322 DEBUG:boto:Token: None 2022-09-03T05:34:50.323 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/ 2022-09-03T05:34:50.324 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA= 2022-09-03T05:34:50.325 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA=', 'Content-Length': '0'} 2022-09-03T05:34:50.327 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '16'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003e7696b7bd687c8e-006312e77a-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.329 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:50.330 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:50.330 DEBUG:boto:Method: PUT 2022-09-03T05:34:50.331 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:50.331 DEBUG:boto:Data: 2022-09-03T05:34:50.332 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:50.332 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:50.333 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.334 DEBUG:boto:Params: {} 2022-09-03T05:34:50.335 DEBUG:boto:Token: None 2022-09-03T05:34:50.336 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/obj 2022-09-03T05:34:50.337 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E= 2022-09-03T05:34:50.338 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E='} 2022-09-03T05:34:50.350 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4'), ('x-amz-request-id', 'tx0000091d6111c32a4a739-006312e77a-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.351 DEBUG:rgw_multi.tests:zone=test-zone1 version=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:34:50.352 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:50.353 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:50.354 DEBUG:boto:Method: HEAD 2022-09-03T05:34:50.355 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:50.356 DEBUG:boto:Data: 2022-09-03T05:34:50.357 DEBUG:boto:Headers: {} 2022-09-03T05:34:50.358 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:50.359 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.360 DEBUG:boto:Params: {} 2022-09-03T05:34:50.361 DEBUG:boto:Token: None 2022-09-03T05:34:50.362 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/ 2022-09-03T05:34:50.363 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA= 2022-09-03T05:34:50.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA=', 'Content-Length': '0'} 2022-09-03T05:34:50.367 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '16'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c51ea8a3ab2bc471-006312e77a-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.369 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:50.370 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:50.371 DEBUG:boto:Method: PUT 2022-09-03T05:34:50.372 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:50.373 DEBUG:boto:Data: 2022-09-03T05:34:50.374 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:50.375 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:50.376 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.377 DEBUG:boto:Params: {} 2022-09-03T05:34:50.378 DEBUG:boto:Token: None 2022-09-03T05:34:50.379 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/obj 2022-09-03T05:34:50.380 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E= 2022-09-03T05:34:50.381 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E='} 2022-09-03T05:34:50.431 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'vauOlKlWVHmDxq4ZzMBF97KP66F89Pc'), ('x-amz-request-id', 'tx00000244f103ff42c1b0a-006312e77a-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.431 DEBUG:rgw_multi.tests:zone=test-zone2 version=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:34:50.432 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:50.433 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:50.434 DEBUG:boto:Method: HEAD 2022-09-03T05:34:50.434 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:50.435 DEBUG:boto:Data: 2022-09-03T05:34:50.436 DEBUG:boto:Headers: {} 2022-09-03T05:34:50.437 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:50.438 DEBUG:boto:Port: 8001 2022-09-03T05:34:50.441 DEBUG:boto:Params: {} 2022-09-03T05:34:50.442 DEBUG:boto:Token: None 2022-09-03T05:34:50.443 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/ 2022-09-03T05:34:50.444 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA= 2022-09-03T05:34:50.445 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA=', 'Content-Length': '0'} 2022-09-03T05:34:50.449 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '16'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fb36fa09bfdb38b5-006312e77a-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.451 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:50.452 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:50.453 DEBUG:boto:Method: PUT 2022-09-03T05:34:50.454 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:50.454 DEBUG:boto:Data: 2022-09-03T05:34:50.455 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:50.456 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:50.457 DEBUG:boto:Port: 8001 2022-09-03T05:34:50.458 DEBUG:boto:Params: {} 2022-09-03T05:34:50.459 DEBUG:boto:Token: None 2022-09-03T05:34:50.461 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/obj 2022-09-03T05:34:50.462 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E= 2022-09-03T05:34:50.462 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E='} 2022-09-03T05:34:50.474 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL'), ('x-amz-request-id', 'tx000004db3a194e19d4a5e-006312e77a-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.476 DEBUG:rgw_multi.tests:zone=test-zone3 version=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:34:50.476 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:50.479 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:50.480 DEBUG:boto:Method: HEAD 2022-09-03T05:34:50.481 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:50.481 DEBUG:boto:Data: 2022-09-03T05:34:50.482 DEBUG:boto:Headers: {} 2022-09-03T05:34:50.483 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:50.484 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.485 DEBUG:boto:Params: {} 2022-09-03T05:34:50.490 DEBUG:boto:Token: None 2022-09-03T05:34:50.491 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/ 2022-09-03T05:34:50.492 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA= 2022-09-03T05:34:50.493 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA=', 'Content-Length': '0'} 2022-09-03T05:34:50.501 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000832a023ee94cf3a5-006312e77a-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.503 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:50.503 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:50.504 DEBUG:boto:Method: PUT 2022-09-03T05:34:50.504 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:50.505 DEBUG:boto:Data: 2022-09-03T05:34:50.506 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:50.506 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:50.507 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.508 DEBUG:boto:Params: {} 2022-09-03T05:34:50.509 DEBUG:boto:Token: None 2022-09-03T05:34:50.510 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/obj 2022-09-03T05:34:50.512 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E= 2022-09-03T05:34:50.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E='} 2022-09-03T05:34:50.525 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98'), ('x-amz-request-id', 'tx00000f289658ceab24cac-006312e77a-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.527 DEBUG:rgw_multi.tests:zone=test-zone1 version=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:34:50.528 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:50.529 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:50.530 DEBUG:boto:Method: HEAD 2022-09-03T05:34:50.530 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:50.531 DEBUG:boto:Data: 2022-09-03T05:34:50.532 DEBUG:boto:Headers: {} 2022-09-03T05:34:50.533 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:50.534 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.535 DEBUG:boto:Params: {} 2022-09-03T05:34:50.535 DEBUG:boto:Token: None 2022-09-03T05:34:50.536 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/ 2022-09-03T05:34:50.537 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA= 2022-09-03T05:34:50.537 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA=', 'Content-Length': '0'} 2022-09-03T05:34:50.542 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002fe2ed477704e195-006312e77a-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.543 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:50.544 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:50.545 DEBUG:boto:Method: PUT 2022-09-03T05:34:50.545 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:50.546 DEBUG:boto:Data: 2022-09-03T05:34:50.546 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:50.547 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:50.547 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.548 DEBUG:boto:Params: {} 2022-09-03T05:34:50.549 DEBUG:boto:Token: None 2022-09-03T05:34:50.549 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/obj 2022-09-03T05:34:50.550 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E= 2022-09-03T05:34:50.551 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E='} 2022-09-03T05:34:50.562 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i'), ('x-amz-request-id', 'tx00000498d1e977973b055-006312e77a-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.564 DEBUG:rgw_multi.tests:zone=test-zone2 version=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:34:50.565 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:50.565 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:50.566 DEBUG:boto:Method: HEAD 2022-09-03T05:34:50.567 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:50.568 DEBUG:boto:Data: 2022-09-03T05:34:50.569 DEBUG:boto:Headers: {} 2022-09-03T05:34:50.570 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:50.571 DEBUG:boto:Port: 8001 2022-09-03T05:34:50.572 DEBUG:boto:Params: {} 2022-09-03T05:34:50.573 DEBUG:boto:Token: None 2022-09-03T05:34:50.573 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/ 2022-09-03T05:34:50.574 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA= 2022-09-03T05:34:50.575 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA=', 'Content-Length': '0'} 2022-09-03T05:34:50.578 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c94c6cc04bc10e77-006312e77a-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.579 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:50.580 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:50.580 DEBUG:boto:Method: PUT 2022-09-03T05:34:50.581 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:50.581 DEBUG:boto:Data: 2022-09-03T05:34:50.582 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:50.583 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:50.584 DEBUG:boto:Port: 8001 2022-09-03T05:34:50.585 DEBUG:boto:Params: {} 2022-09-03T05:34:50.586 DEBUG:boto:Token: None 2022-09-03T05:34:50.587 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/obj 2022-09-03T05:34:50.588 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E= 2022-09-03T05:34:50.590 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E='} 2022-09-03T05:34:50.603 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB'), ('x-amz-request-id', 'tx00000904afb2f14380822-006312e77a-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.605 DEBUG:rgw_multi.tests:zone=test-zone3 version=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:34:50.607 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:50.611 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:50.612 DEBUG:boto:Method: HEAD 2022-09-03T05:34:50.612 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:50.613 DEBUG:boto:Data: 2022-09-03T05:34:50.613 DEBUG:boto:Headers: {} 2022-09-03T05:34:50.614 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:50.614 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.615 DEBUG:boto:Params: {} 2022-09-03T05:34:50.616 DEBUG:boto:Token: None 2022-09-03T05:34:50.616 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/ 2022-09-03T05:34:50.617 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA= 2022-09-03T05:34:50.618 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA=', 'Content-Length': '0'} 2022-09-03T05:34:50.621 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '32'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004e24bf33e47837c8-006312e77a-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.622 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:50.624 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:50.624 DEBUG:boto:Method: PUT 2022-09-03T05:34:50.625 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:50.626 DEBUG:boto:Data: 2022-09-03T05:34:50.626 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:50.627 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:50.627 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.628 DEBUG:boto:Params: {} 2022-09-03T05:34:50.628 DEBUG:boto:Token: None 2022-09-03T05:34:50.629 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/obj 2022-09-03T05:34:50.629 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E= 2022-09-03T05:34:50.630 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E='} 2022-09-03T05:34:50.642 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'kUiw73n7LyoUiDcS9JXEfdlCebChUCE'), ('x-amz-request-id', 'tx000006f3dc6b3c48b4661-006312e77a-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.644 DEBUG:rgw_multi.tests:zone=test-zone1 version=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:34:50.645 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:50.646 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:50.647 DEBUG:boto:Method: HEAD 2022-09-03T05:34:50.648 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:50.649 DEBUG:boto:Data: 2022-09-03T05:34:50.650 DEBUG:boto:Headers: {} 2022-09-03T05:34:50.651 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:50.654 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.655 DEBUG:boto:Params: {} 2022-09-03T05:34:50.656 DEBUG:boto:Token: None 2022-09-03T05:34:50.657 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/ 2022-09-03T05:34:50.658 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA= 2022-09-03T05:34:50.659 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA=', 'Content-Length': '0'} 2022-09-03T05:34:50.662 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '32'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000278be4548447e9c9-006312e77a-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.663 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:50.664 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:50.664 DEBUG:boto:Method: PUT 2022-09-03T05:34:50.665 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:50.666 DEBUG:boto:Data: 2022-09-03T05:34:50.667 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:50.668 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:50.668 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.669 DEBUG:boto:Params: {} 2022-09-03T05:34:50.670 DEBUG:boto:Token: None 2022-09-03T05:34:50.671 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/obj 2022-09-03T05:34:50.672 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E= 2022-09-03T05:34:50.673 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E='} 2022-09-03T05:34:50.684 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '94bPO72IhlWQEGwklU15Pvpr-qRv.lm'), ('x-amz-request-id', 'tx000000ac1f132da7d1e6d-006312e77a-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.686 DEBUG:rgw_multi.tests:zone=test-zone2 version=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:34:50.686 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:50.687 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:50.688 DEBUG:boto:Method: HEAD 2022-09-03T05:34:50.689 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:50.690 DEBUG:boto:Data: 2022-09-03T05:34:50.691 DEBUG:boto:Headers: {} 2022-09-03T05:34:50.692 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:50.693 DEBUG:boto:Port: 8001 2022-09-03T05:34:50.694 DEBUG:boto:Params: {} 2022-09-03T05:34:50.695 DEBUG:boto:Token: None 2022-09-03T05:34:50.696 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/ 2022-09-03T05:34:50.697 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA= 2022-09-03T05:34:50.698 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA=', 'Content-Length': '0'} 2022-09-03T05:34:50.701 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '32'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000361360650bee46f5-006312e77a-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.703 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:50.704 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:50.704 DEBUG:boto:Method: PUT 2022-09-03T05:34:50.705 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:50.706 DEBUG:boto:Data: 2022-09-03T05:34:50.707 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:50.708 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:50.709 DEBUG:boto:Port: 8001 2022-09-03T05:34:50.710 DEBUG:boto:Params: {} 2022-09-03T05:34:50.711 DEBUG:boto:Token: None 2022-09-03T05:34:50.712 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/obj 2022-09-03T05:34:50.713 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E= 2022-09-03T05:34:50.714 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E='} 2022-09-03T05:34:50.725 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'ORmfT2fVQ5EFIFI89EGuLigbPyfeObT'), ('x-amz-request-id', 'tx00000e2806f4bb21788ca-006312e77a-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.727 DEBUG:rgw_multi.tests:zone=test-zone3 version=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:34:50.730 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:50.730 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:50.731 DEBUG:boto:Method: HEAD 2022-09-03T05:34:50.732 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:50.733 DEBUG:boto:Data: 2022-09-03T05:34:50.733 DEBUG:boto:Headers: {} 2022-09-03T05:34:50.734 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:50.735 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.735 DEBUG:boto:Params: {} 2022-09-03T05:34:50.736 DEBUG:boto:Token: None 2022-09-03T05:34:50.738 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/ 2022-09-03T05:34:50.739 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA= 2022-09-03T05:34:50.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA=', 'Content-Length': '0'} 2022-09-03T05:34:50.744 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '40'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f4e7697db5e3c5c4-006312e77a-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.746 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:50.747 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:50.749 DEBUG:boto:Method: PUT 2022-09-03T05:34:50.749 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:50.750 DEBUG:boto:Data: 2022-09-03T05:34:50.751 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:50.752 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:50.753 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.754 DEBUG:boto:Params: {} 2022-09-03T05:34:50.755 DEBUG:boto:Token: None 2022-09-03T05:34:50.756 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/obj 2022-09-03T05:34:50.757 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E= 2022-09-03T05:34:50.758 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E='} 2022-09-03T05:34:50.771 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U'), ('x-amz-request-id', 'tx00000318c0eaa7dfe32e7-006312e77a-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.771 DEBUG:rgw_multi.tests:zone=test-zone1 version=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:34:50.772 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:50.772 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:50.773 DEBUG:boto:Method: HEAD 2022-09-03T05:34:50.773 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:50.774 DEBUG:boto:Data: 2022-09-03T05:34:50.774 DEBUG:boto:Headers: {} 2022-09-03T05:34:50.775 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:50.775 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.776 DEBUG:boto:Params: {} 2022-09-03T05:34:50.777 DEBUG:boto:Token: None 2022-09-03T05:34:50.777 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/ 2022-09-03T05:34:50.778 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA= 2022-09-03T05:34:50.779 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA=', 'Content-Length': '0'} 2022-09-03T05:34:50.782 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '40'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000957618d222d9078d-006312e77a-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.783 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:50.784 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:50.785 DEBUG:boto:Method: PUT 2022-09-03T05:34:50.785 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:50.786 DEBUG:boto:Data: 2022-09-03T05:34:50.786 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:50.787 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:50.787 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.788 DEBUG:boto:Params: {} 2022-09-03T05:34:50.788 DEBUG:boto:Token: None 2022-09-03T05:34:50.789 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/obj 2022-09-03T05:34:50.789 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E= 2022-09-03T05:34:50.790 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E='} 2022-09-03T05:34:50.802 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'QfRyggjm1gZ7H.LeMCey0hHybmVWONG'), ('x-amz-request-id', 'tx00000d6706f5005798992-006312e77a-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.803 DEBUG:rgw_multi.tests:zone=test-zone2 version=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:34:50.803 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:50.804 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:50.804 DEBUG:boto:Method: HEAD 2022-09-03T05:34:50.805 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:50.805 DEBUG:boto:Data: 2022-09-03T05:34:50.806 DEBUG:boto:Headers: {} 2022-09-03T05:34:50.806 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:50.807 DEBUG:boto:Port: 8001 2022-09-03T05:34:50.807 DEBUG:boto:Params: {} 2022-09-03T05:34:50.808 DEBUG:boto:Token: None 2022-09-03T05:34:50.809 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/ 2022-09-03T05:34:50.809 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA= 2022-09-03T05:34:50.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA=', 'Content-Length': '0'} 2022-09-03T05:34:50.812 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '40'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002ea52604fe323f96-006312e77a-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.813 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:50.814 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:50.814 DEBUG:boto:Method: PUT 2022-09-03T05:34:50.815 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:50.815 DEBUG:boto:Data: 2022-09-03T05:34:50.816 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:50.817 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:50.817 DEBUG:boto:Port: 8001 2022-09-03T05:34:50.818 DEBUG:boto:Params: {} 2022-09-03T05:34:50.818 DEBUG:boto:Token: None 2022-09-03T05:34:50.819 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/obj 2022-09-03T05:34:50.820 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E= 2022-09-03T05:34:50.820 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E='} 2022-09-03T05:34:50.831 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt'), ('x-amz-request-id', 'tx00000566bdc87ad08b68e-006312e77a-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.831 DEBUG:rgw_multi.tests:zone=test-zone3 version=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:34:50.832 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:50.832 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:50.834 DEBUG:boto:Method: HEAD 2022-09-03T05:34:50.835 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:50.838 DEBUG:boto:Data: 2022-09-03T05:34:50.839 DEBUG:boto:Headers: {} 2022-09-03T05:34:50.840 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:50.840 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.841 DEBUG:boto:Params: {} 2022-09-03T05:34:50.841 DEBUG:boto:Token: None 2022-09-03T05:34:50.842 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/ 2022-09-03T05:34:50.843 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA= 2022-09-03T05:34:50.843 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA=', 'Content-Length': '0'} 2022-09-03T05:34:50.847 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '48'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c6102ca78789fa91-006312e77a-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.847 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:50.848 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:50.849 DEBUG:boto:Method: PUT 2022-09-03T05:34:50.849 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:50.850 DEBUG:boto:Data: 2022-09-03T05:34:50.850 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:50.851 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:50.851 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.852 DEBUG:boto:Params: {} 2022-09-03T05:34:50.853 DEBUG:boto:Token: None 2022-09-03T05:34:50.853 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/obj 2022-09-03T05:34:50.856 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E= 2022-09-03T05:34:50.857 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E='} 2022-09-03T05:34:50.868 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj'), ('x-amz-request-id', 'tx0000090266f42560d9020-006312e77a-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.869 DEBUG:rgw_multi.tests:zone=test-zone1 version=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:34:50.870 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:50.871 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:50.872 DEBUG:boto:Method: HEAD 2022-09-03T05:34:50.873 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:50.874 DEBUG:boto:Data: 2022-09-03T05:34:50.875 DEBUG:boto:Headers: {} 2022-09-03T05:34:50.876 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:50.877 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.878 DEBUG:boto:Params: {} 2022-09-03T05:34:50.879 DEBUG:boto:Token: None 2022-09-03T05:34:50.880 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/ 2022-09-03T05:34:50.881 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA= 2022-09-03T05:34:50.882 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA=', 'Content-Length': '0'} 2022-09-03T05:34:50.886 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '48'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e161b75ba7168c44-006312e77a-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.887 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:50.888 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:50.889 DEBUG:boto:Method: PUT 2022-09-03T05:34:50.893 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:50.894 DEBUG:boto:Data: 2022-09-03T05:34:50.895 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:50.895 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:50.896 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.897 DEBUG:boto:Params: {} 2022-09-03T05:34:50.898 DEBUG:boto:Token: None 2022-09-03T05:34:50.898 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/obj 2022-09-03T05:34:50.899 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E= 2022-09-03T05:34:50.899 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E='} 2022-09-03T05:34:50.915 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'yL2uOIeC0l.C5YXdYoqQBvDIly3idxO'), ('x-amz-request-id', 'tx000003c17963f70b4adca-006312e77a-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.916 DEBUG:rgw_multi.tests:zone=test-zone2 version=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:34:50.917 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:50.918 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:50.918 DEBUG:boto:Method: HEAD 2022-09-03T05:34:50.919 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:50.920 DEBUG:boto:Data: 2022-09-03T05:34:50.921 DEBUG:boto:Headers: {} 2022-09-03T05:34:50.922 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:50.923 DEBUG:boto:Port: 8001 2022-09-03T05:34:50.923 DEBUG:boto:Params: {} 2022-09-03T05:34:50.924 DEBUG:boto:Token: None 2022-09-03T05:34:50.925 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/ 2022-09-03T05:34:50.926 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA= 2022-09-03T05:34:50.927 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA=', 'Content-Length': '0'} 2022-09-03T05:34:50.930 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '48'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002553d98eb2917846-006312e77a-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.932 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:50.933 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:50.934 DEBUG:boto:Method: PUT 2022-09-03T05:34:50.936 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:50.937 DEBUG:boto:Data: 2022-09-03T05:34:50.939 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:50.940 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:50.940 DEBUG:boto:Port: 8001 2022-09-03T05:34:50.941 DEBUG:boto:Params: {} 2022-09-03T05:34:50.942 DEBUG:boto:Token: None 2022-09-03T05:34:50.943 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/obj 2022-09-03T05:34:50.944 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E= 2022-09-03T05:34:50.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E='} 2022-09-03T05:34:50.957 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'EFeprdWgJo5gbXM16hy6HNBwhGTxK7e'), ('x-amz-request-id', 'tx00000cad13f47b3e141b1-006312e77a-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.958 DEBUG:rgw_multi.tests:zone=test-zone3 version=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:34:50.959 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:50.960 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:50.961 DEBUG:boto:Method: HEAD 2022-09-03T05:34:50.962 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:50.964 DEBUG:boto:Data: 2022-09-03T05:34:50.965 DEBUG:boto:Headers: {} 2022-09-03T05:34:50.966 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:50.967 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.968 DEBUG:boto:Params: {} 2022-09-03T05:34:50.969 DEBUG:boto:Token: None 2022-09-03T05:34:50.970 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/ 2022-09-03T05:34:50.971 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA= 2022-09-03T05:34:50.973 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfwnL6fMFQDdJX9usVXdRCRIxAA=', 'Content-Length': '0'} 2022-09-03T05:34:50.976 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '56'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f329b79623049eea-006312e77a-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:50.977 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:50.978 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:50.979 DEBUG:boto:Method: PUT 2022-09-03T05:34:50.980 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:50.981 DEBUG:boto:Data: 2022-09-03T05:34:50.982 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:50.983 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:50.984 DEBUG:boto:Port: 8000 2022-09-03T05:34:50.985 DEBUG:boto:Params: {} 2022-09-03T05:34:50.986 DEBUG:boto:Token: None 2022-09-03T05:34:50.987 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:50 GMT /dsgvcy-19/obj 2022-09-03T05:34:50.988 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E= 2022-09-03T05:34:50.989 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yK6tyC3+LRgHJojyOtYKc/O/e/E='} 2022-09-03T05:34:51.010 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2'), ('x-amz-request-id', 'tx00000c68f30adab5036cf-006312e77a-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:51.011 DEBUG:rgw_multi.tests:zone=test-zone1 version=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:34:51.012 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:51.012 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:51.013 DEBUG:boto:Method: HEAD 2022-09-03T05:34:51.013 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:51.014 DEBUG:boto:Data: 2022-09-03T05:34:51.015 DEBUG:boto:Headers: {} 2022-09-03T05:34:51.016 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:51.016 DEBUG:boto:Port: 8000 2022-09-03T05:34:51.017 DEBUG:boto:Params: {} 2022-09-03T05:34:51.018 DEBUG:boto:Token: None 2022-09-03T05:34:51.019 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:51 GMT /dsgvcy-19/ 2022-09-03T05:34:51.019 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EzVkUtsMCik+CDux37oLyKAd3n0= 2022-09-03T05:34:51.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EzVkUtsMCik+CDux37oLyKAd3n0=', 'Content-Length': '0'} 2022-09-03T05:34:51.025 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '56'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005181f01c8d6f57f0-006312e77b-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:51.030 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:51.031 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:51.031 DEBUG:boto:Method: PUT 2022-09-03T05:34:51.032 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:51.033 DEBUG:boto:Data: 2022-09-03T05:34:51.033 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:51.034 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:51.035 DEBUG:boto:Port: 8000 2022-09-03T05:34:51.035 DEBUG:boto:Params: {} 2022-09-03T05:34:51.036 DEBUG:boto:Token: None 2022-09-03T05:34:51.037 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:51 GMT /dsgvcy-19/obj 2022-09-03T05:34:51.037 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:McCWnPdZfh54fv6KAZ7v/h5Dtfg= 2022-09-03T05:34:51.038 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:McCWnPdZfh54fv6KAZ7v/h5Dtfg='} 2022-09-03T05:34:51.086 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '0PhwqB536OEXdH1QOmPxhlUfX3fwPvS'), ('x-amz-request-id', 'tx00000af7dc7bd179434d4-006312e77b-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:51.088 DEBUG:rgw_multi.tests:zone=test-zone2 version=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:34:51.089 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:51.089 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:51.091 DEBUG:boto:Method: HEAD 2022-09-03T05:34:51.092 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:51.093 DEBUG:boto:Data: 2022-09-03T05:34:51.093 DEBUG:boto:Headers: {} 2022-09-03T05:34:51.102 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:51.103 DEBUG:boto:Port: 8001 2022-09-03T05:34:51.104 DEBUG:boto:Params: {} 2022-09-03T05:34:51.106 DEBUG:boto:Token: None 2022-09-03T05:34:51.107 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:51 GMT /dsgvcy-19/ 2022-09-03T05:34:51.108 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EzVkUtsMCik+CDux37oLyKAd3n0= 2022-09-03T05:34:51.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EzVkUtsMCik+CDux37oLyKAd3n0=', 'Content-Length': '0'} 2022-09-03T05:34:51.114 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '56'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fccd905ec4502de8-006312e77b-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:51.115 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:51.116 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:51.116 DEBUG:boto:Method: PUT 2022-09-03T05:34:51.117 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:51.117 DEBUG:boto:Data: 2022-09-03T05:34:51.120 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:51.121 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:51.122 DEBUG:boto:Port: 8001 2022-09-03T05:34:51.122 DEBUG:boto:Params: {} 2022-09-03T05:34:51.123 DEBUG:boto:Token: None 2022-09-03T05:34:51.124 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:51 GMT /dsgvcy-19/obj 2022-09-03T05:34:51.124 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:McCWnPdZfh54fv6KAZ7v/h5Dtfg= 2022-09-03T05:34:51.125 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:McCWnPdZfh54fv6KAZ7v/h5Dtfg='} 2022-09-03T05:34:51.137 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi'), ('x-amz-request-id', 'tx00000866a528bc2fb943d-006312e77b-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:51.138 DEBUG:rgw_multi.tests:zone=test-zone3 version=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:34:51.139 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:51.139 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:51.140 DEBUG:boto:Method: HEAD 2022-09-03T05:34:51.142 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:51.143 DEBUG:boto:Data: 2022-09-03T05:34:51.143 DEBUG:boto:Headers: {} 2022-09-03T05:34:51.144 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:51.145 DEBUG:boto:Port: 8000 2022-09-03T05:34:51.146 DEBUG:boto:Params: {} 2022-09-03T05:34:51.147 DEBUG:boto:Token: None 2022-09-03T05:34:51.148 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:51 GMT /dsgvcy-19/ 2022-09-03T05:34:51.149 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EzVkUtsMCik+CDux37oLyKAd3n0= 2022-09-03T05:34:51.150 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EzVkUtsMCik+CDux37oLyKAd3n0=', 'Content-Length': '0'} 2022-09-03T05:34:51.154 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '9'), ('X-RGW-Bytes-Used', '64'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fd3cfe3be9be073c-006312e77b-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:51.155 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:51.156 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:51.157 DEBUG:boto:Method: PUT 2022-09-03T05:34:51.158 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:51.159 DEBUG:boto:Data: 2022-09-03T05:34:51.160 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:51.163 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:51.163 DEBUG:boto:Port: 8000 2022-09-03T05:34:51.164 DEBUG:boto:Params: {} 2022-09-03T05:34:51.165 DEBUG:boto:Token: None 2022-09-03T05:34:51.167 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:51 GMT /dsgvcy-19/obj 2022-09-03T05:34:51.168 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:McCWnPdZfh54fv6KAZ7v/h5Dtfg= 2022-09-03T05:34:51.169 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:McCWnPdZfh54fv6KAZ7v/h5Dtfg='} 2022-09-03T05:34:51.180 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf'), ('x-amz-request-id', 'tx00000c326f8545a5a1cf1-006312e77b-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:51.181 DEBUG:rgw_multi.tests:zone=test-zone1 version=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:34:51.182 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:51.182 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:51.183 DEBUG:boto:Method: HEAD 2022-09-03T05:34:51.183 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:51.184 DEBUG:boto:Data: 2022-09-03T05:34:51.184 DEBUG:boto:Headers: {} 2022-09-03T05:34:51.185 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:51.185 DEBUG:boto:Port: 8000 2022-09-03T05:34:51.186 DEBUG:boto:Params: {} 2022-09-03T05:34:51.187 DEBUG:boto:Token: None 2022-09-03T05:34:51.187 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:51 GMT /dsgvcy-19/ 2022-09-03T05:34:51.188 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EzVkUtsMCik+CDux37oLyKAd3n0= 2022-09-03T05:34:51.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EzVkUtsMCik+CDux37oLyKAd3n0=', 'Content-Length': '0'} 2022-09-03T05:34:51.191 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '9'), ('X-RGW-Bytes-Used', '64'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007b2a5a49c673bdf8-006312e77b-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:51.193 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:51.193 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:51.194 DEBUG:boto:Method: PUT 2022-09-03T05:34:51.195 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:51.195 DEBUG:boto:Data: 2022-09-03T05:34:51.196 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:51.197 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:51.198 DEBUG:boto:Port: 8000 2022-09-03T05:34:51.198 DEBUG:boto:Params: {} 2022-09-03T05:34:51.199 DEBUG:boto:Token: None 2022-09-03T05:34:51.200 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:51 GMT /dsgvcy-19/obj 2022-09-03T05:34:51.201 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:McCWnPdZfh54fv6KAZ7v/h5Dtfg= 2022-09-03T05:34:51.201 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:McCWnPdZfh54fv6KAZ7v/h5Dtfg='} 2022-09-03T05:34:51.217 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D'), ('x-amz-request-id', 'tx000008ae38a361581ef9d-006312e77b-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:51.218 DEBUG:rgw_multi.tests:zone=test-zone2 version=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:34:51.218 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:51.219 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:51.219 DEBUG:boto:Method: HEAD 2022-09-03T05:34:51.220 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:51.220 DEBUG:boto:Data: 2022-09-03T05:34:51.221 DEBUG:boto:Headers: {} 2022-09-03T05:34:51.221 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:51.222 DEBUG:boto:Port: 8001 2022-09-03T05:34:51.222 DEBUG:boto:Params: {} 2022-09-03T05:34:51.223 DEBUG:boto:Token: None 2022-09-03T05:34:51.224 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:51 GMT /dsgvcy-19/ 2022-09-03T05:34:51.224 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EzVkUtsMCik+CDux37oLyKAd3n0= 2022-09-03T05:34:51.225 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EzVkUtsMCik+CDux37oLyKAd3n0=', 'Content-Length': '0'} 2022-09-03T05:34:51.227 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '9'), ('X-RGW-Bytes-Used', '64'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d77c24ebd0caddbd-006312e77b-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:51.228 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:51.229 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:51.230 DEBUG:boto:Method: PUT 2022-09-03T05:34:51.230 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:51.231 DEBUG:boto:Data: 2022-09-03T05:34:51.231 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:51.232 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:51.232 DEBUG:boto:Port: 8001 2022-09-03T05:34:51.233 DEBUG:boto:Params: {} 2022-09-03T05:34:51.234 DEBUG:boto:Token: None 2022-09-03T05:34:51.235 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:51 GMT /dsgvcy-19/obj 2022-09-03T05:34:51.236 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:McCWnPdZfh54fv6KAZ7v/h5Dtfg= 2022-09-03T05:34:51.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:McCWnPdZfh54fv6KAZ7v/h5Dtfg='} 2022-09-03T05:34:51.287 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i'), ('x-amz-request-id', 'tx000002edcbe4c01309955-006312e77b-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:51.290 DEBUG:rgw_multi.tests:zone=test-zone3 version=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:34:51.291 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:51.292 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:51.293 DEBUG:boto:Method: HEAD 2022-09-03T05:34:51.293 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:51.294 DEBUG:boto:Data: 2022-09-03T05:34:51.295 DEBUG:boto:Headers: {} 2022-09-03T05:34:51.296 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:51.297 DEBUG:boto:Port: 8000 2022-09-03T05:34:51.298 DEBUG:boto:Params: {} 2022-09-03T05:34:51.299 DEBUG:boto:Token: None 2022-09-03T05:34:51.300 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:51 GMT /dsgvcy-19/ 2022-09-03T05:34:51.301 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EzVkUtsMCik+CDux37oLyKAd3n0= 2022-09-03T05:34:51.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EzVkUtsMCik+CDux37oLyKAd3n0=', 'Content-Length': '0'} 2022-09-03T05:34:51.306 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '10'), ('X-RGW-Bytes-Used', '72'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000062960eb548e88caf-006312e77b-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:51.307 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:51.308 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:51.309 DEBUG:boto:Method: PUT 2022-09-03T05:34:51.310 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:51.311 DEBUG:boto:Data: 2022-09-03T05:34:51.312 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:51.313 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:34:51.314 DEBUG:boto:Port: 8000 2022-09-03T05:34:51.315 DEBUG:boto:Params: {} 2022-09-03T05:34:51.316 DEBUG:boto:Token: None 2022-09-03T05:34:51.318 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:51 GMT /dsgvcy-19/obj 2022-09-03T05:34:51.319 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:McCWnPdZfh54fv6KAZ7v/h5Dtfg= 2022-09-03T05:34:51.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:McCWnPdZfh54fv6KAZ7v/h5Dtfg='} 2022-09-03T05:34:51.331 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'hqLZJOZHmRi2EtNletjE-T3uOA2T8pT'), ('x-amz-request-id', 'tx0000030d3b2917192c7f0-006312e77b-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:51.332 DEBUG:rgw_multi.tests:zone=test-zone1 version=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:34:51.333 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:51.334 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:51.335 DEBUG:boto:Method: HEAD 2022-09-03T05:34:51.336 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:51.336 DEBUG:boto:Data: 2022-09-03T05:34:51.337 DEBUG:boto:Headers: {} 2022-09-03T05:34:51.339 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:51.340 DEBUG:boto:Port: 8000 2022-09-03T05:34:51.341 DEBUG:boto:Params: {} 2022-09-03T05:34:51.342 DEBUG:boto:Token: None 2022-09-03T05:34:51.343 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:51 GMT /dsgvcy-19/ 2022-09-03T05:34:51.344 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EzVkUtsMCik+CDux37oLyKAd3n0= 2022-09-03T05:34:51.345 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EzVkUtsMCik+CDux37oLyKAd3n0=', 'Content-Length': '0'} 2022-09-03T05:34:51.348 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '10'), ('X-RGW-Bytes-Used', '72'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006506dc62b6c3cbd7-006312e77b-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:51.349 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:51.350 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:51.351 DEBUG:boto:Method: PUT 2022-09-03T05:34:51.352 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:51.353 DEBUG:boto:Data: 2022-09-03T05:34:51.354 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:51.355 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:34:51.356 DEBUG:boto:Port: 8000 2022-09-03T05:34:51.357 DEBUG:boto:Params: {} 2022-09-03T05:34:51.358 DEBUG:boto:Token: None 2022-09-03T05:34:51.359 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:51 GMT /dsgvcy-19/obj 2022-09-03T05:34:51.359 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:McCWnPdZfh54fv6KAZ7v/h5Dtfg= 2022-09-03T05:34:51.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:McCWnPdZfh54fv6KAZ7v/h5Dtfg='} 2022-09-03T05:34:51.375 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'mBYgYYD3drKvdzws.hh3-36xKvL9eTH'), ('x-amz-request-id', 'tx000009f852ca2c258bf49-006312e77b-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:51.376 DEBUG:rgw_multi.tests:zone=test-zone2 version=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:34:51.377 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:34:51.379 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:34:51.381 DEBUG:boto:Method: HEAD 2022-09-03T05:34:51.381 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:34:51.382 DEBUG:boto:Data: 2022-09-03T05:34:51.383 DEBUG:boto:Headers: {} 2022-09-03T05:34:51.384 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:51.385 DEBUG:boto:Port: 8001 2022-09-03T05:34:51.386 DEBUG:boto:Params: {} 2022-09-03T05:34:51.387 DEBUG:boto:Token: None 2022-09-03T05:34:51.388 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:34:51 GMT /dsgvcy-19/ 2022-09-03T05:34:51.390 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EzVkUtsMCik+CDux37oLyKAd3n0= 2022-09-03T05:34:51.391 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:34:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EzVkUtsMCik+CDux37oLyKAd3n0=', 'Content-Length': '0'} 2022-09-03T05:34:51.396 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '10'), ('X-RGW-Bytes-Used', '72'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002ed4ab6a5d31bf9e-006312e77b-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:51.398 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:34:51.399 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:34:51.400 DEBUG:boto:Method: PUT 2022-09-03T05:34:51.400 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:34:51.401 DEBUG:boto:Data: 2022-09-03T05:34:51.402 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue'} 2022-09-03T05:34:51.403 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:34:51.404 DEBUG:boto:Port: 8001 2022-09-03T05:34:51.404 DEBUG:boto:Params: {} 2022-09-03T05:34:51.405 DEBUG:boto:Token: None 2022-09-03T05:34:51.407 DEBUG:boto:StringToSign: PUT lmY06/L8E1cH1nU2kr9LHg== application/octet-stream Sat, 03 Sep 2022 05:34:51 GMT /dsgvcy-19/obj 2022-09-03T05:34:51.408 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:McCWnPdZfh54fv6KAZ7v/h5Dtfg= 2022-09-03T05:34:51.409 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'lmY06/L8E1cH1nU2kr9LHg==', 'Content-Length': '8', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:34:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:McCWnPdZfh54fv6KAZ7v/h5Dtfg='} 2022-09-03T05:34:51.425 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-amz-request-id', 'tx0000087b3352f6e0704f7-006312e77b-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:34:51.426 DEBUG:rgw_multi.tests:zone=test-zone3 version=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:34:51.427 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-19 2022-09-03T05:34:51.427 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:34:51.428 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:34:51.428 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-19 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:34:51.493 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:34:51.494 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:34:51.539 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:34:51.540+0000 7f1ef4028600 1 waiting for incremental sync to catch up: 2022-09-03T05:34:51.540 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000004.36.9, , , , , ] 2022-09-03T05:34:51.540 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000004.36.9, , , 00000000020.46.11, , ] 2022-09-03T05:34:56.541 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:34:56.542+0000 7f1ef4028600 1 waiting for incremental sync to catch up: 2022-09-03T05:34:56.542 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000004.36.9, , , , , ] 2022-09-03T05:34:56.542 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000004.36.9, , , 00000000020.46.11, , ] 2022-09-03T05:35:01.543 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:35:01.543+0000 7f1ef4028600 1 waiting for incremental sync to catch up: 2022-09-03T05:35:01.544 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000004.36.9, , , , , ] 2022-09-03T05:35:01.544 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000004.36.9, , , 00000000020.46.11, , ] 2022-09-03T05:35:06.546 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:35:06.543+0000 7f1ef4028600 1 waiting for incremental sync to catch up: 2022-09-03T05:35:06.547 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000004.36.9, , , , , ] 2022-09-03T05:35:06.547 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000004.36.9, , , 00000000020.46.11, , ] 2022-09-03T05:35:11.545 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:35:11.546+0000 7f1ef4028600 1 waiting for incremental sync to catch up: 2022-09-03T05:35:11.546 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000004.36.9, , , 00000000019.45.5, , ] 2022-09-03T05:35:11.547 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000004.36.9, , , 00000000020.46.11, , ] 2022-09-03T05:35:16.547 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:35:16.547+0000 7f1ef4028600 1 waiting for incremental sync to catch up: 2022-09-03T05:35:16.548 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000004.36.9, , , 00000000019.45.5, , ] 2022-09-03T05:35:16.548 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000004.36.9, , , 00000000020.46.11, , ] 2022-09-03T05:35:21.548 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:35:21.549+0000 7f1ef4028600 1 waiting for incremental sync to catch up: 2022-09-03T05:35:21.549 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000004.36.9, , , 00000000019.45.5, , ] 2022-09-03T05:35:21.549 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000004.36.9, , , 00000000020.46.11, , ] 2022-09-03T05:35:26.550 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:35:26.550+0000 7f1ef4028600 1 waiting for incremental sync to catch up: 2022-09-03T05:35:26.550 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000004.36.9, , , 00000000019.45.5, , ] 2022-09-03T05:35:26.551 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000004.36.9, , , 00000000020.46.11, , ] 2022-09-03T05:35:31.551 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:35:31.551+0000 7f1ef4028600 1 bucket sync caught up with source: 2022-09-03T05:35:31.551 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000004.36.9, , , 00000000042.99.5, , ] 2022-09-03T05:35:31.552 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000004.36.9, , , 00000000020.46.11, , ] 2022-09-03T05:35:31.552 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:35:31.551+0000 7f1ef4028600 0 bucket checkpoint complete 2022-09-03T05:35:31.566 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-19 2022-09-03T05:35:31.567 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:35:31.568 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:35:31.569 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-19 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:35:31.633 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:35:31.633 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:35:31.676 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:35:31.677+0000 7f40f4bfc600 1 bucket sync caught up with source: 2022-09-03T05:35:31.677 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000004.36.9, , , 00000000042.99.5, , ] 2022-09-03T05:35:31.677 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.36.9, , , 00000000042.99.5, , ] 2022-09-03T05:35:31.677 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:35:31.677+0000 7f40f4bfc600 0 bucket checkpoint complete 2022-09-03T05:35:31.691 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-19 2022-09-03T05:35:31.692 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:35:31.693 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:35:31.694 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-19 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:35:31.758 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:35:31.759 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:35:31.810 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:35:31.810+0000 7ff3819ee600 1 waiting for incremental sync to catch up: 2022-09-03T05:35:31.811 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000004.36.9, , , 00000000031.77.11, , ] 2022-09-03T05:35:31.812 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000004.36.9, , , 00000000042.99.5, , ] 2022-09-03T05:35:36.815 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:35:36.812+0000 7ff3819ee600 1 waiting for incremental sync to catch up: 2022-09-03T05:35:36.816 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000004.36.9, , , 00000000031.77.11, , ] 2022-09-03T05:35:36.817 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000004.36.9, , , 00000000042.99.5, , ] 2022-09-03T05:35:41.813 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:35:41.813+0000 7ff3819ee600 1 bucket sync caught up with source: 2022-09-03T05:35:41.814 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000004.36.9, , , 00000000042.99.5, , ] 2022-09-03T05:35:41.814 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000004.36.9, , , 00000000042.99.5, , ] 2022-09-03T05:35:41.815 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:35:41.813+0000 7ff3819ee600 0 bucket checkpoint complete 2022-09-03T05:35:41.829 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-19 2022-09-03T05:35:41.830 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:35:41.831 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:35:41.832 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-19 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:35:41.898 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:35:41.898 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:35:41.956 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:35:41.953+0000 7f5244bb1600 1 waiting for incremental sync to catch up: 2022-09-03T05:35:41.956 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.36.9, , , 00000000013.55.5, , ] 2022-09-03T05:35:41.957 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.38.9, , , 00000000043.128.11, , ] 2022-09-03T05:35:46.955 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:35:46.955+0000 7f5244bb1600 1 waiting for incremental sync to catch up: 2022-09-03T05:35:46.956 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.36.9, , , 00000000043.128.11, , ] 2022-09-03T05:35:46.956 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.38.9, , , 00000000043.128.11, , ] 2022-09-03T05:35:51.956 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:35:51.956+0000 7f5244bb1600 1 waiting for incremental sync to catch up: 2022-09-03T05:35:51.957 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.36.9, , , 00000000043.128.11, , ] 2022-09-03T05:35:51.957 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.38.9, , , 00000000043.128.11, , ] 2022-09-03T05:35:56.960 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:35:56.958+0000 7f5244bb1600 1 waiting for incremental sync to catch up: 2022-09-03T05:35:56.961 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.36.9, , , 00000000043.128.11, , ] 2022-09-03T05:35:56.961 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.38.9, , , 00000000043.128.11, , ] 2022-09-03T05:36:01.959 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:36:01.960+0000 7f5244bb1600 1 waiting for incremental sync to catch up: 2022-09-03T05:36:01.968 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.36.9, , , 00000000043.128.11, , ] 2022-09-03T05:36:01.968 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.38.9, , , 00000000043.128.11, , ] 2022-09-03T05:36:06.961 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:36:06.961+0000 7f5244bb1600 1 waiting for incremental sync to catch up: 2022-09-03T05:36:06.961 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.36.9, , , 00000000045.137.9, , ] 2022-09-03T05:36:06.962 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.38.9, , , 00000000043.128.11, , ] 2022-09-03T05:36:11.962 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:36:11.962+0000 7f5244bb1600 1 waiting for incremental sync to catch up: 2022-09-03T05:36:11.963 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.36.9, , , 00000000045.137.9, , ] 2022-09-03T05:36:11.963 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.38.9, , , 00000000043.128.11, , ] 2022-09-03T05:36:16.964 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:36:16.964+0000 7f5244bb1600 1 waiting for incremental sync to catch up: 2022-09-03T05:36:16.964 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.36.9, , , 00000000045.137.9, , ] 2022-09-03T05:36:16.965 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.38.9, , , 00000000043.128.11, , ] 2022-09-03T05:36:21.964 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:36:21.964+0000 7f5244bb1600 1 waiting for incremental sync to catch up: 2022-09-03T05:36:21.965 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.36.9, , , 00000000045.137.9, , ] 2022-09-03T05:36:21.965 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.38.9, , , 00000000043.128.11, , ] 2022-09-03T05:36:26.967 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:36:26.966+0000 7f5244bb1600 1 waiting for incremental sync to catch up: 2022-09-03T05:36:26.968 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.36.9, , , 00000000045.137.9, , ] 2022-09-03T05:36:26.968 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.38.9, , , 00000000043.128.11, , ] 2022-09-03T05:36:31.968 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:36:31.967+0000 7f5244bb1600 1 waiting for incremental sync to catch up: 2022-09-03T05:36:31.968 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.36.9, , , 00000000045.137.9, , ] 2022-09-03T05:36:31.969 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.38.9, , , 00000000043.128.11, , ] 2022-09-03T05:36:36.969 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:36:36.969+0000 7f5244bb1600 1 waiting for incremental sync to catch up: 2022-09-03T05:36:36.970 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.36.9, , , 00000000045.137.9, , ] 2022-09-03T05:36:36.971 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.38.9, , , 00000000043.128.11, , ] 2022-09-03T05:36:41.971 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:36:41.970+0000 7f5244bb1600 1 waiting for incremental sync to catch up: 2022-09-03T05:36:41.971 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.36.9, , , 00000000045.137.9, , ] 2022-09-03T05:36:41.972 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.38.9, , , 00000000043.128.11, , ] 2022-09-03T05:36:46.972 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:36:46.973+0000 7f5244bb1600 1 waiting for incremental sync to catch up: 2022-09-03T05:36:46.973 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.36.9, , , 00000000045.137.9, , ] 2022-09-03T05:36:46.974 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.38.9, , , 00000000043.128.11, , ] 2022-09-03T05:36:51.974 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:36:51.974+0000 7f5244bb1600 1 waiting for incremental sync to catch up: 2022-09-03T05:36:51.975 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.36.9, , , 00000000045.137.9, , ] 2022-09-03T05:36:51.975 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.38.9, , , 00000000043.128.11, , ] 2022-09-03T05:36:56.975 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:36:56.976+0000 7f5244bb1600 1 waiting for incremental sync to catch up: 2022-09-03T05:36:56.976 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.36.9, , , 00000000045.137.9, , ] 2022-09-03T05:36:56.976 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.38.9, , , 00000000043.128.11, , ] 2022-09-03T05:37:01.978 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:37:01.978+0000 7f5244bb1600 1 waiting for incremental sync to catch up: 2022-09-03T05:37:01.978 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000003.36.9, , , 00000000045.137.9, , ] 2022-09-03T05:37:01.979 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.38.9, , , 00000000043.128.11, , ] 2022-09-03T05:37:06.979 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:37:06.979+0000 7f5244bb1600 1 bucket sync caught up with source: 2022-09-03T05:37:06.980 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000004.38.9, , , 00000000045.137.9, , ] 2022-09-03T05:37:06.980 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.38.9, , , 00000000043.128.11, , ] 2022-09-03T05:37:06.981 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:37:06.979+0000 7f5244bb1600 0 bucket checkpoint complete 2022-09-03T05:37:06.996 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-19 2022-09-03T05:37:06.998 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:37:06.998 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:37:06.999 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-19 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:37:07.057 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:37:07.058 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:37:07.100 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:37:07.100+0000 7f7b88af4600 1 bucket sync caught up with source: 2022-09-03T05:37:07.100 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000004.38.9, , , 00000000045.137.9, , ] 2022-09-03T05:37:07.101 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.38.9, , , 00000000045.137.9, , ] 2022-09-03T05:37:07.101 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:37:07.100+0000 7f7b88af4600 0 bucket checkpoint complete 2022-09-03T05:37:07.111 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-19 2022-09-03T05:37:07.112 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-19 2022-09-03T05:37:07.112 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:37:07.113 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:37:07.114 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-19 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:37:07.173 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:37:07.173 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:37:07.219 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:37:07.219+0000 7f0a4f3b4600 1 bucket sync caught up with source: 2022-09-03T05:37:07.219 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000004.31.9, , , 00000000047.130.9, , ] 2022-09-03T05:37:07.220 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000004.31.9, , , 00000000047.130.9, , ] 2022-09-03T05:37:07.220 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:37:07.219+0000 7f0a4f3b4600 0 bucket checkpoint complete 2022-09-03T05:37:07.236 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-19 2022-09-03T05:37:07.237 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:37:07.237 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-19', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:37:07.238 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-19 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:37:07.304 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:37:07.304 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:37:07.352 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:37:07.352+0000 7fde9af57600 1 bucket sync caught up with source: 2022-09-03T05:37:07.352 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000004.31.9, , , 00000000047.130.9, , ] 2022-09-03T05:37:07.353 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000004.31.9, , , 00000000047.130.9, , ] 2022-09-03T05:37:07.353 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:37:07.352+0000 7fde9af57600 0 bucket checkpoint complete 2022-09-03T05:37:07.368 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-19 2022-09-03T05:37:07.369 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-19 zones={test-zone2, test-zone1} 2022-09-03T05:37:07.370 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:07.371 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:07.372 DEBUG:boto:Method: HEAD 2022-09-03T05:37:07.372 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:37:07.373 DEBUG:boto:Data: 2022-09-03T05:37:07.374 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.375 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:07.376 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.377 DEBUG:boto:Params: {} 2022-09-03T05:37:07.378 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:37:07.379 DEBUG:boto:Token: None 2022-09-03T05:37:07.380 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/ 2022-09-03T05:37:07.381 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Sv9O7EtWsBcTWMmxf9t1c/Cl8oQ= 2022-09-03T05:37:07.382 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Sv9O7EtWsBcTWMmxf9t1c/Cl8oQ=', 'Content-Length': '0'} 2022-09-03T05:37:07.389 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '31'), ('X-RGW-Bytes-Used', '240'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007c6a05b9b7959da0-006312e803-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.390 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:07.390 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:07.391 DEBUG:boto:Method: HEAD 2022-09-03T05:37:07.391 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:37:07.392 DEBUG:boto:Data: 2022-09-03T05:37:07.394 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.394 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:07.395 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.396 DEBUG:boto:Params: {} 2022-09-03T05:37:07.397 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:37:07.397 DEBUG:boto:Token: None 2022-09-03T05:37:07.398 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/ 2022-09-03T05:37:07.399 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Sv9O7EtWsBcTWMmxf9t1c/Cl8oQ= 2022-09-03T05:37:07.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Sv9O7EtWsBcTWMmxf9t1c/Cl8oQ=', 'Content-Length': '0'} 2022-09-03T05:37:07.405 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '31'), ('X-RGW-Bytes-Used', '240'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009f3008ee18770f45-006312e803-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.407 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:37:07.408 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:07.409 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:07.410 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:07.411 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:07.412 DEBUG:boto:Method: GET 2022-09-03T05:37:07.413 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:07.414 DEBUG:boto:Data: 2022-09-03T05:37:07.415 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.416 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:07.417 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.418 DEBUG:boto:Params: {} 2022-09-03T05:37:07.419 DEBUG:boto:Token: None 2022-09-03T05:37:07.420 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/?versions 2022-09-03T05:37:07.421 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JdN8PRCBASjZcBoBHT6zb+2MaQE= 2022-09-03T05:37:07.422 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JdN8PRCBASjZcBoBHT6zb+2MaQE=', 'Content-Length': '0'} 2022-09-03T05:37:07.426 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000014213d1253dc7525-006312e803-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.427 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjFV.ndVyKCalSXA6cZK3aQWN-LzBZK3Rtrue2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjhqLZJOZHmRi2EtNletjE-T3uOA2T8pTfalse2022-09-03T05:34:51.324Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjmBYgYYD3drKvdzws.hh3-36xKvL9eTHfalse2022-09-03T05:34:51.365Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjMnuLKlk.Rq8n.YuO6g0DugPfRiHStyffalse2022-09-03T05:34:51.173Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjN.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1ifalse2022-09-03T05:34:51.242Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjgFHndXVQlMCeVf-4Dc4FENAmsrgKp-Dfalse2022-09-03T05:34:51.206Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj0PhwqB536OEXdH1QOmPxhlUfX3fwPvSfalse2022-09-03T05:34:51.042Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjGM1mLEe4fDwiUzp2k3PtPzJLy-tnWKifalse2022-09-03T05:34:51.129Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjKl9OW7HSpMTjjnmOMkqEktWm9H3uMr2false2022-09-03T05:34:50.995Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjEFeprdWgJo5gbXM16hy6HNBwhGTxK7efalse2022-09-03T05:34:50.949Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjJID7ijdm1B3ZxtamFDxAcL6LJfFKoNjfalse2022-09-03T05:34:50.861Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjyL2uOIeC0l.C5YXdYoqQBvDIly3idxOfalse2022-09-03T05:34:50.905Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.lPacc7Eu6BJ5IOsSyLccq2.KRUK24Ufalse2022-09-03T05:34:50.763Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjNkMDAxRk7LkNPj1mtl1YEsdjIaTowHtfalse2022-09-03T05:34:50.824Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQfRyggjm1gZ7H.LeMCey0hHybmVWONGfalse2022-09-03T05:34:50.794Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj94bPO72IhlWQEGwklU15Pvpr-qRv.lmfalse2022-09-03T05:34:50.677Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjORmfT2fVQ5EFIFI89EGuLigbPyfeObTfalse2022-09-03T05:34:50.719Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkUiw73n7LyoUiDcS9JXEfdlCebChUCEfalse2022-09-03T05:34:50.635Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjBZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98false2022-09-03T05:34:50.517Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjIoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SBfalse2022-09-03T05:34:50.595Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjrHbEnZ16TQjAC.xp1wNbn0Ce5Vc948ifalse2022-09-03T05:34:50.555Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjG4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXLfalse2022-09-03T05:34:50.467Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQl7wbqZZa4psf4mYXa6Hq8W4qHqEHV4false2022-09-03T05:34:50.343Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjvauOlKlWVHmDxq4ZzMBF97KP66F89Pcfalse2022-09-03T05:34:50.391Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKCfalse2022-09-03T05:34:50.189Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjoYY-icmxg74gZ4carPTk38at9fTxjJifalse2022-09-03T05:34:50.232Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjxYnysJgMUeKh.Mew5S-gBJZsJgD4g.Afalse2022-09-03T05:34:50.274Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvYfalse2022-09-03T05:34:50.041Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjCFKxqazR5g.k3.oWs62bQgs9moHAimIfalse2022-09-03T05:34:50.150Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkJSzTg.tDVQ-nL1nX84.W7ifJxIseT1false2022-09-03T05:34:50.112Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:07.430 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:37:07.433 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.433 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.434 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.434 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.435 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.435 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.436 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.436 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.437 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.437 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.439 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.439 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.440 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.440 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.441 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.441 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.442 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.442 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.443 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.443 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.444 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.444 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.445 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.445 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.446 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.448 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.449 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.450 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.450 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.451 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.451 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:37:07.452 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:07.453 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:07.453 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:07.453 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:07.454 DEBUG:boto:Method: GET 2022-09-03T05:37:07.455 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:07.455 DEBUG:boto:Data: 2022-09-03T05:37:07.456 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.456 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:07.457 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.457 DEBUG:boto:Params: {} 2022-09-03T05:37:07.458 DEBUG:boto:Token: None 2022-09-03T05:37:07.458 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/?versions 2022-09-03T05:37:07.459 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JdN8PRCBASjZcBoBHT6zb+2MaQE= 2022-09-03T05:37:07.459 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JdN8PRCBASjZcBoBHT6zb+2MaQE=', 'Content-Length': '0'} 2022-09-03T05:37:07.463 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007edeb0df20e3497c-006312e803-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.463 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjFV.ndVyKCalSXA6cZK3aQWN-LzBZK3Rtrue2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjhqLZJOZHmRi2EtNletjE-T3uOA2T8pTfalse2022-09-03T05:34:51.324Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjmBYgYYD3drKvdzws.hh3-36xKvL9eTHfalse2022-09-03T05:34:51.365Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjMnuLKlk.Rq8n.YuO6g0DugPfRiHStyffalse2022-09-03T05:34:51.173Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjN.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1ifalse2022-09-03T05:34:51.242Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjgFHndXVQlMCeVf-4Dc4FENAmsrgKp-Dfalse2022-09-03T05:34:51.206Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj0PhwqB536OEXdH1QOmPxhlUfX3fwPvSfalse2022-09-03T05:34:51.042Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjGM1mLEe4fDwiUzp2k3PtPzJLy-tnWKifalse2022-09-03T05:34:51.129Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjKl9OW7HSpMTjjnmOMkqEktWm9H3uMr2false2022-09-03T05:34:50.995Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjEFeprdWgJo5gbXM16hy6HNBwhGTxK7efalse2022-09-03T05:34:50.949Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjJID7ijdm1B3ZxtamFDxAcL6LJfFKoNjfalse2022-09-03T05:34:50.861Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjyL2uOIeC0l.C5YXdYoqQBvDIly3idxOfalse2022-09-03T05:34:50.905Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.lPacc7Eu6BJ5IOsSyLccq2.KRUK24Ufalse2022-09-03T05:34:50.763Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjNkMDAxRk7LkNPj1mtl1YEsdjIaTowHtfalse2022-09-03T05:34:50.824Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQfRyggjm1gZ7H.LeMCey0hHybmVWONGfalse2022-09-03T05:34:50.794Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj94bPO72IhlWQEGwklU15Pvpr-qRv.lmfalse2022-09-03T05:34:50.677Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjORmfT2fVQ5EFIFI89EGuLigbPyfeObTfalse2022-09-03T05:34:50.719Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkUiw73n7LyoUiDcS9JXEfdlCebChUCEfalse2022-09-03T05:34:50.635Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjBZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98false2022-09-03T05:34:50.517Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjIoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SBfalse2022-09-03T05:34:50.595Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjrHbEnZ16TQjAC.xp1wNbn0Ce5Vc948ifalse2022-09-03T05:34:50.555Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjG4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXLfalse2022-09-03T05:34:50.467Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQl7wbqZZa4psf4mYXa6Hq8W4qHqEHV4false2022-09-03T05:34:50.343Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjvauOlKlWVHmDxq4ZzMBF97KP66F89Pcfalse2022-09-03T05:34:50.391Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKCfalse2022-09-03T05:34:50.189Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjoYY-icmxg74gZ4carPTk38at9fTxjJifalse2022-09-03T05:34:50.232Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjxYnysJgMUeKh.Mew5S-gBJZsJgD4g.Afalse2022-09-03T05:34:50.274Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvYfalse2022-09-03T05:34:50.041Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjCFKxqazR5g.k3.oWs62bQgs9moHAimIfalse2022-09-03T05:34:50.150Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkJSzTg.tDVQ-nL1nX84.W7ifJxIseT1false2022-09-03T05:34:50.112Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:07.466 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:37:07.466 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.467 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.467 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.468 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.468 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.469 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.469 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.470 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.470 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.471 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.471 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.472 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.472 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.473 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.474 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.474 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.475 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.475 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.476 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.476 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.477 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.477 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.478 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.478 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.479 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.479 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.479 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.480 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.480 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.481 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:07.481 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:07.482 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:07.483 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:07.483 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:07.484 DEBUG:boto:Method: GET 2022-09-03T05:37:07.484 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:07.485 DEBUG:boto:Data: 2022-09-03T05:37:07.485 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.486 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:07.486 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.487 DEBUG:boto:Params: {} 2022-09-03T05:37:07.487 DEBUG:boto:Token: None 2022-09-03T05:37:07.488 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/?versions 2022-09-03T05:37:07.488 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JdN8PRCBASjZcBoBHT6zb+2MaQE= 2022-09-03T05:37:07.489 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JdN8PRCBASjZcBoBHT6zb+2MaQE=', 'Content-Length': '0'} 2022-09-03T05:37:07.492 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007e76a426c44d323c-006312e803-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.493 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjFV.ndVyKCalSXA6cZK3aQWN-LzBZK3Rtrue2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjhqLZJOZHmRi2EtNletjE-T3uOA2T8pTfalse2022-09-03T05:34:51.324Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjmBYgYYD3drKvdzws.hh3-36xKvL9eTHfalse2022-09-03T05:34:51.365Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjMnuLKlk.Rq8n.YuO6g0DugPfRiHStyffalse2022-09-03T05:34:51.173Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjN.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1ifalse2022-09-03T05:34:51.242Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjgFHndXVQlMCeVf-4Dc4FENAmsrgKp-Dfalse2022-09-03T05:34:51.206Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj0PhwqB536OEXdH1QOmPxhlUfX3fwPvSfalse2022-09-03T05:34:51.042Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjGM1mLEe4fDwiUzp2k3PtPzJLy-tnWKifalse2022-09-03T05:34:51.129Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjKl9OW7HSpMTjjnmOMkqEktWm9H3uMr2false2022-09-03T05:34:50.995Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjEFeprdWgJo5gbXM16hy6HNBwhGTxK7efalse2022-09-03T05:34:50.949Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjJID7ijdm1B3ZxtamFDxAcL6LJfFKoNjfalse2022-09-03T05:34:50.861Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjyL2uOIeC0l.C5YXdYoqQBvDIly3idxOfalse2022-09-03T05:34:50.905Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.lPacc7Eu6BJ5IOsSyLccq2.KRUK24Ufalse2022-09-03T05:34:50.763Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjNkMDAxRk7LkNPj1mtl1YEsdjIaTowHtfalse2022-09-03T05:34:50.824Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQfRyggjm1gZ7H.LeMCey0hHybmVWONGfalse2022-09-03T05:34:50.794Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj94bPO72IhlWQEGwklU15Pvpr-qRv.lmfalse2022-09-03T05:34:50.677Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjORmfT2fVQ5EFIFI89EGuLigbPyfeObTfalse2022-09-03T05:34:50.719Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkUiw73n7LyoUiDcS9JXEfdlCebChUCEfalse2022-09-03T05:34:50.635Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjBZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98false2022-09-03T05:34:50.517Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjIoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SBfalse2022-09-03T05:34:50.595Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjrHbEnZ16TQjAC.xp1wNbn0Ce5Vc948ifalse2022-09-03T05:34:50.555Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjG4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXLfalse2022-09-03T05:34:50.467Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQl7wbqZZa4psf4mYXa6Hq8W4qHqEHV4false2022-09-03T05:34:50.343Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjvauOlKlWVHmDxq4ZzMBF97KP66F89Pcfalse2022-09-03T05:34:50.391Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKCfalse2022-09-03T05:34:50.189Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjoYY-icmxg74gZ4carPTk38at9fTxjJifalse2022-09-03T05:34:50.232Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjxYnysJgMUeKh.Mew5S-gBJZsJgD4g.Afalse2022-09-03T05:34:50.274Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvYfalse2022-09-03T05:34:50.041Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjCFKxqazR5g.k3.oWs62bQgs9moHAimIfalse2022-09-03T05:34:50.150Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkJSzTg.tDVQ-nL1nX84.W7ifJxIseT1false2022-09-03T05:34:50.112Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:07.495 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:07.496 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:07.496 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:07.497 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:07.497 DEBUG:boto:Method: GET 2022-09-03T05:37:07.498 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:07.499 DEBUG:boto:Data: 2022-09-03T05:37:07.500 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.501 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:07.502 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.503 DEBUG:boto:Params: {} 2022-09-03T05:37:07.504 DEBUG:boto:Token: None 2022-09-03T05:37:07.505 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/?versions 2022-09-03T05:37:07.506 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JdN8PRCBASjZcBoBHT6zb+2MaQE= 2022-09-03T05:37:07.507 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JdN8PRCBASjZcBoBHT6zb+2MaQE=', 'Content-Length': '0'} 2022-09-03T05:37:07.511 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000047c478cfc89e6ab0-006312e803-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.512 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjFV.ndVyKCalSXA6cZK3aQWN-LzBZK3Rtrue2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjhqLZJOZHmRi2EtNletjE-T3uOA2T8pTfalse2022-09-03T05:34:51.324Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjmBYgYYD3drKvdzws.hh3-36xKvL9eTHfalse2022-09-03T05:34:51.365Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjMnuLKlk.Rq8n.YuO6g0DugPfRiHStyffalse2022-09-03T05:34:51.173Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjN.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1ifalse2022-09-03T05:34:51.242Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjgFHndXVQlMCeVf-4Dc4FENAmsrgKp-Dfalse2022-09-03T05:34:51.206Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj0PhwqB536OEXdH1QOmPxhlUfX3fwPvSfalse2022-09-03T05:34:51.042Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjGM1mLEe4fDwiUzp2k3PtPzJLy-tnWKifalse2022-09-03T05:34:51.129Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjKl9OW7HSpMTjjnmOMkqEktWm9H3uMr2false2022-09-03T05:34:50.995Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjEFeprdWgJo5gbXM16hy6HNBwhGTxK7efalse2022-09-03T05:34:50.949Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjJID7ijdm1B3ZxtamFDxAcL6LJfFKoNjfalse2022-09-03T05:34:50.861Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjyL2uOIeC0l.C5YXdYoqQBvDIly3idxOfalse2022-09-03T05:34:50.905Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.lPacc7Eu6BJ5IOsSyLccq2.KRUK24Ufalse2022-09-03T05:34:50.763Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjNkMDAxRk7LkNPj1mtl1YEsdjIaTowHtfalse2022-09-03T05:34:50.824Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQfRyggjm1gZ7H.LeMCey0hHybmVWONGfalse2022-09-03T05:34:50.794Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj94bPO72IhlWQEGwklU15Pvpr-qRv.lmfalse2022-09-03T05:34:50.677Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjORmfT2fVQ5EFIFI89EGuLigbPyfeObTfalse2022-09-03T05:34:50.719Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkUiw73n7LyoUiDcS9JXEfdlCebChUCEfalse2022-09-03T05:34:50.635Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjBZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98false2022-09-03T05:34:50.517Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjIoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SBfalse2022-09-03T05:34:50.595Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjrHbEnZ16TQjAC.xp1wNbn0Ce5Vc948ifalse2022-09-03T05:34:50.555Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjG4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXLfalse2022-09-03T05:34:50.467Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQl7wbqZZa4psf4mYXa6Hq8W4qHqEHV4false2022-09-03T05:34:50.343Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjvauOlKlWVHmDxq4ZzMBF97KP66F89Pcfalse2022-09-03T05:34:50.391Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKCfalse2022-09-03T05:34:50.189Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjoYY-icmxg74gZ4carPTk38at9fTxjJifalse2022-09-03T05:34:50.232Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjxYnysJgMUeKh.Mew5S-gBJZsJgD4g.Afalse2022-09-03T05:34:50.274Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvYfalse2022-09-03T05:34:50.041Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjCFKxqazR5g.k3.oWs62bQgs9moHAimIfalse2022-09-03T05:34:50.150Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkJSzTg.tDVQ-nL1nX84.W7ifJxIseT1false2022-09-03T05:34:50.112Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:07.516 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:37:07.517 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:07.519 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:07.519 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.520 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.521 DEBUG:boto:Method: GET 2022-09-03T05:37:07.522 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.523 DEBUG:boto:Data: 2022-09-03T05:37:07.523 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.524 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:07.524 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.525 DEBUG:boto:Params: {} 2022-09-03T05:37:07.525 DEBUG:boto:Token: None 2022-09-03T05:37:07.526 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.527 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yErQR20CGmL5r8UbI3ktz79Ss6s= 2022-09-03T05:37:07.527 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yErQR20CGmL5r8UbI3ktz79Ss6s=', 'Content-Length': '0'} 2022-09-03T05:37:07.529 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000fb683aac822b0f60-006312e803-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.530 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:07.531 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:07.531 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.532 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.532 DEBUG:boto:Method: GET 2022-09-03T05:37:07.533 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.533 DEBUG:boto:Data: 2022-09-03T05:37:07.534 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.534 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:07.535 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.536 DEBUG:boto:Params: {} 2022-09-03T05:37:07.536 DEBUG:boto:Token: None 2022-09-03T05:37:07.537 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.537 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yErQR20CGmL5r8UbI3ktz79Ss6s= 2022-09-03T05:37:07.538 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yErQR20CGmL5r8UbI3ktz79Ss6s=', 'Content-Length': '0'} 2022-09-03T05:37:07.540 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000ec30c691bbb99ef6-006312e803-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.541 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:07.542 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:07.542 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.543 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.544 DEBUG:boto:Method: HEAD 2022-09-03T05:37:07.544 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.545 DEBUG:boto:Data: 2022-09-03T05:37:07.545 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.546 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:07.546 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.547 DEBUG:boto:Params: {} 2022-09-03T05:37:07.547 DEBUG:boto:Token: None 2022-09-03T05:37:07.548 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.549 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H4AH15qyi/Gntu9IqYAD78idIOI= 2022-09-03T05:37:07.549 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H4AH15qyi/Gntu9IqYAD78idIOI=', 'Content-Length': '0'} 2022-09-03T05:37:07.553 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000eecddaf25e8f28af-006312e803-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.555 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:07.555 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:07.556 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.558 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.559 DEBUG:boto:Method: HEAD 2022-09-03T05:37:07.559 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.560 DEBUG:boto:Data: 2022-09-03T05:37:07.561 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.562 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:07.563 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.564 DEBUG:boto:Params: {} 2022-09-03T05:37:07.565 DEBUG:boto:Token: None 2022-09-03T05:37:07.566 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.567 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H4AH15qyi/Gntu9IqYAD78idIOI= 2022-09-03T05:37:07.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H4AH15qyi/Gntu9IqYAD78idIOI=', 'Content-Length': '0'} 2022-09-03T05:37:07.570 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000001dbb3b59677e5484-006312e803-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.571 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:37:07.572 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:07.573 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:07.573 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.574 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.575 DEBUG:boto:Method: GET 2022-09-03T05:37:07.576 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.577 DEBUG:boto:Data: 2022-09-03T05:37:07.577 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.578 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:07.579 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.579 DEBUG:boto:Params: {} 2022-09-03T05:37:07.580 DEBUG:boto:Token: None 2022-09-03T05:37:07.580 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.581 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yErQR20CGmL5r8UbI3ktz79Ss6s= 2022-09-03T05:37:07.582 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yErQR20CGmL5r8UbI3ktz79Ss6s=', 'Content-Length': '0'} 2022-09-03T05:37:07.584 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000023fd048ee078e4e6-006312e803-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.585 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:07.586 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:07.586 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.587 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.587 DEBUG:boto:Method: GET 2022-09-03T05:37:07.588 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.589 DEBUG:boto:Data: 2022-09-03T05:37:07.590 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.590 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:07.591 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.592 DEBUG:boto:Params: {} 2022-09-03T05:37:07.593 DEBUG:boto:Token: None 2022-09-03T05:37:07.594 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.595 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yErQR20CGmL5r8UbI3ktz79Ss6s= 2022-09-03T05:37:07.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yErQR20CGmL5r8UbI3ktz79Ss6s=', 'Content-Length': '0'} 2022-09-03T05:37:07.598 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000006acbb2c91b603ee3-006312e803-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.600 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:07.601 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:07.602 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:07.602 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:07.603 DEBUG:boto:Method: GET 2022-09-03T05:37:07.605 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:37:07.606 DEBUG:boto:Data: 2022-09-03T05:37:07.607 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.608 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:07.609 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.610 DEBUG:boto:Params: {} 2022-09-03T05:37:07.611 DEBUG:boto:Token: None 2022-09-03T05:37:07.612 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/ 2022-09-03T05:37:07.613 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aXenryEtqGjbMcMpfYD3d/SLCgs= 2022-09-03T05:37:07.614 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aXenryEtqGjbMcMpfYD3d/SLCgs=', 'Content-Length': '0'} 2022-09-03T05:37:07.617 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000004c4b4148902d1f7-006312e803-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.618 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:07.620 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:07.621 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:07.622 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:07.622 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:07.623 DEBUG:boto:Method: GET 2022-09-03T05:37:07.623 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:37:07.624 DEBUG:boto:Data: 2022-09-03T05:37:07.624 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.625 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:07.625 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.626 DEBUG:boto:Params: {} 2022-09-03T05:37:07.627 DEBUG:boto:Token: None 2022-09-03T05:37:07.627 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/ 2022-09-03T05:37:07.628 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aXenryEtqGjbMcMpfYD3d/SLCgs= 2022-09-03T05:37:07.629 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aXenryEtqGjbMcMpfYD3d/SLCgs=', 'Content-Length': '0'} 2022-09-03T05:37:07.631 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e92de50ae37c977f-006312e803-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.678 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:07.680 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:07.681 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:07.682 DEBUG:boto:Method: HEAD 2022-09-03T05:37:07.682 DEBUG:boto:Path: /dsgvcy-19/dummy 2022-09-03T05:37:07.683 DEBUG:boto:Data: 2022-09-03T05:37:07.684 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.684 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:07.685 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.686 DEBUG:boto:Params: {} 2022-09-03T05:37:07.686 DEBUG:boto:Token: None 2022-09-03T05:37:07.688 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/dummy 2022-09-03T05:37:07.688 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dw5ALYn2m+Apn5ENUev8q5+LoF4= 2022-09-03T05:37:07.690 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dw5ALYn2m+Apn5ENUev8q5+LoF4=', 'Content-Length': '0'} 2022-09-03T05:37:07.692 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000004f05ad483f514676-006312e803-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.693 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:07.693 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:07.694 DEBUG:boto:Method: HEAD 2022-09-03T05:37:07.694 DEBUG:boto:Path: /dsgvcy-19/dummy 2022-09-03T05:37:07.695 DEBUG:boto:Data: 2022-09-03T05:37:07.695 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.696 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:07.696 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.697 DEBUG:boto:Params: {} 2022-09-03T05:37:07.698 DEBUG:boto:Token: None 2022-09-03T05:37:07.698 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/dummy 2022-09-03T05:37:07.699 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dw5ALYn2m+Apn5ENUev8q5+LoF4= 2022-09-03T05:37:07.699 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dw5ALYn2m+Apn5ENUev8q5+LoF4=', 'Content-Length': '0'} 2022-09-03T05:37:07.702 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000bd2448953da44aea-006312e803-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.703 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:37:07.703 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:07.704 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:07.704 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.705 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.705 DEBUG:boto:Method: GET 2022-09-03T05:37:07.706 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.706 DEBUG:boto:Data: 2022-09-03T05:37:07.707 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.707 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:07.708 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.709 DEBUG:boto:Params: {} 2022-09-03T05:37:07.709 DEBUG:boto:Token: None 2022-09-03T05:37:07.710 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.710 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yErQR20CGmL5r8UbI3ktz79Ss6s= 2022-09-03T05:37:07.711 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yErQR20CGmL5r8UbI3ktz79Ss6s=', 'Content-Length': '0'} 2022-09-03T05:37:07.713 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000037fb086aa3dfeca6-006312e803-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.714 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:07.714 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:07.715 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.716 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.716 DEBUG:boto:Method: GET 2022-09-03T05:37:07.717 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.717 DEBUG:boto:Data: 2022-09-03T05:37:07.718 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.718 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:07.719 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.720 DEBUG:boto:Params: {} 2022-09-03T05:37:07.720 DEBUG:boto:Token: None 2022-09-03T05:37:07.721 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:07.722 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yErQR20CGmL5r8UbI3ktz79Ss6s= 2022-09-03T05:37:07.722 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yErQR20CGmL5r8UbI3ktz79Ss6s=', 'Content-Length': '0'} 2022-09-03T05:37:07.725 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a429f49aec4e79bc-006312e803-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.725 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:07.726 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:07.727 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:07.727 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:07.728 DEBUG:boto:Method: GET 2022-09-03T05:37:07.728 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:37:07.729 DEBUG:boto:Data: 2022-09-03T05:37:07.729 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.730 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:07.730 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.731 DEBUG:boto:Params: {} 2022-09-03T05:37:07.731 DEBUG:boto:Token: None 2022-09-03T05:37:07.732 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/ 2022-09-03T05:37:07.732 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aXenryEtqGjbMcMpfYD3d/SLCgs= 2022-09-03T05:37:07.733 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aXenryEtqGjbMcMpfYD3d/SLCgs=', 'Content-Length': '0'} 2022-09-03T05:37:07.736 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b4cd951176b7d234-006312e803-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.737 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:07.739 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:07.739 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:07.740 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:07.741 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:07.742 DEBUG:boto:Method: GET 2022-09-03T05:37:07.743 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:37:07.744 DEBUG:boto:Data: 2022-09-03T05:37:07.745 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.746 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:07.747 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.748 DEBUG:boto:Params: {} 2022-09-03T05:37:07.749 DEBUG:boto:Token: None 2022-09-03T05:37:07.750 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/ 2022-09-03T05:37:07.751 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aXenryEtqGjbMcMpfYD3d/SLCgs= 2022-09-03T05:37:07.752 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aXenryEtqGjbMcMpfYD3d/SLCgs=', 'Content-Length': '0'} 2022-09-03T05:37:07.757 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a5452c1f2b9554ac-006312e803-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.758 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:07.760 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:07.760 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:07.761 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:07.763 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.764 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.765 DEBUG:boto:Method: GET 2022-09-03T05:37:07.765 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.767 DEBUG:boto:Data: 2022-09-03T05:37:07.768 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.769 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:07.769 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.770 DEBUG:boto:Params: {} 2022-09-03T05:37:07.773 DEBUG:boto:Token: None 2022-09-03T05:37:07.775 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.776 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IeRljyLjQo3dziyIuid3yQWFR2E= 2022-09-03T05:37:07.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IeRljyLjQo3dziyIuid3yQWFR2E=', 'Content-Length': '0'} 2022-09-03T05:37:07.780 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cb268859f0ecd8e2-006312e803-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.781 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:07.782 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:07.783 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.784 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.785 DEBUG:boto:Method: GET 2022-09-03T05:37:07.786 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.786 DEBUG:boto:Data: 2022-09-03T05:37:07.787 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.788 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:07.788 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.789 DEBUG:boto:Params: {} 2022-09-03T05:37:07.790 DEBUG:boto:Token: None 2022-09-03T05:37:07.790 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.791 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IeRljyLjQo3dziyIuid3yQWFR2E= 2022-09-03T05:37:07.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IeRljyLjQo3dziyIuid3yQWFR2E=', 'Content-Length': '0'} 2022-09-03T05:37:07.794 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008f615095be602a53-006312e803-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.795 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:07.796 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:07.796 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.797 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.797 DEBUG:boto:Method: HEAD 2022-09-03T05:37:07.798 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.798 DEBUG:boto:Data: 2022-09-03T05:37:07.799 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.799 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:07.800 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.801 DEBUG:boto:Params: {} 2022-09-03T05:37:07.802 DEBUG:boto:Token: None 2022-09-03T05:37:07.802 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.803 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B94b64hnCWZWJif3qrhiLKUOlwQ= 2022-09-03T05:37:07.803 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B94b64hnCWZWJif3qrhiLKUOlwQ=', 'Content-Length': '0'} 2022-09-03T05:37:07.805 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b7b3347ccdabecc0-006312e803-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.806 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:07.807 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:07.807 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.808 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.808 DEBUG:boto:Method: HEAD 2022-09-03T05:37:07.809 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.809 DEBUG:boto:Data: 2022-09-03T05:37:07.810 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.812 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:07.812 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.813 DEBUG:boto:Params: {} 2022-09-03T05:37:07.814 DEBUG:boto:Token: None 2022-09-03T05:37:07.814 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.815 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B94b64hnCWZWJif3qrhiLKUOlwQ= 2022-09-03T05:37:07.815 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B94b64hnCWZWJif3qrhiLKUOlwQ=', 'Content-Length': '0'} 2022-09-03T05:37:07.818 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000917f225a9d569e71-006312e803-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.819 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:07.820 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:07.821 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:07.822 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.823 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.825 DEBUG:boto:Method: GET 2022-09-03T05:37:07.826 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.827 DEBUG:boto:Data: 2022-09-03T05:37:07.828 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.829 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:07.830 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.831 DEBUG:boto:Params: {} 2022-09-03T05:37:07.832 DEBUG:boto:Token: None 2022-09-03T05:37:07.833 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.834 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IeRljyLjQo3dziyIuid3yQWFR2E= 2022-09-03T05:37:07.835 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IeRljyLjQo3dziyIuid3yQWFR2E=', 'Content-Length': '0'} 2022-09-03T05:37:07.838 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002968f20e2c33cfd1-006312e803-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.840 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:07.842 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:07.842 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.843 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.843 DEBUG:boto:Method: GET 2022-09-03T05:37:07.845 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.845 DEBUG:boto:Data: 2022-09-03T05:37:07.846 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.846 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:07.847 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.848 DEBUG:boto:Params: {} 2022-09-03T05:37:07.848 DEBUG:boto:Token: None 2022-09-03T05:37:07.849 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.849 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IeRljyLjQo3dziyIuid3yQWFR2E= 2022-09-03T05:37:07.850 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IeRljyLjQo3dziyIuid3yQWFR2E=', 'Content-Length': '0'} 2022-09-03T05:37:07.854 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007af3846da72a5828-006312e803-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.855 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:07.856 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:07.856 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:07.857 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:07.857 DEBUG:boto:Method: GET 2022-09-03T05:37:07.858 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:07.859 DEBUG:boto:Data: 2022-09-03T05:37:07.859 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.860 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:07.860 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.861 DEBUG:boto:Params: {} 2022-09-03T05:37:07.861 DEBUG:boto:Token: None 2022-09-03T05:37:07.862 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/ 2022-09-03T05:37:07.863 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aXenryEtqGjbMcMpfYD3d/SLCgs= 2022-09-03T05:37:07.864 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aXenryEtqGjbMcMpfYD3d/SLCgs=', 'Content-Length': '0'} 2022-09-03T05:37:07.867 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001b92b4628ab6e9e7-006312e803-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.915 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:07.916 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:07.917 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:07.918 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:07.920 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:07.921 DEBUG:boto:Method: GET 2022-09-03T05:37:07.922 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:07.923 DEBUG:boto:Data: 2022-09-03T05:37:07.924 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.925 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:07.926 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.927 DEBUG:boto:Params: {} 2022-09-03T05:37:07.932 DEBUG:boto:Token: None 2022-09-03T05:37:07.933 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/ 2022-09-03T05:37:07.934 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aXenryEtqGjbMcMpfYD3d/SLCgs= 2022-09-03T05:37:07.935 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aXenryEtqGjbMcMpfYD3d/SLCgs=', 'Content-Length': '0'} 2022-09-03T05:37:07.939 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000392c27831efc7474-006312e803-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.941 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:07.942 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:07.942 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:07.943 DEBUG:boto:Method: HEAD 2022-09-03T05:37:07.943 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:07.944 DEBUG:boto:Data: 2022-09-03T05:37:07.944 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.945 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:07.946 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.946 DEBUG:boto:Params: {} 2022-09-03T05:37:07.947 DEBUG:boto:Token: None 2022-09-03T05:37:07.947 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/obj 2022-09-03T05:37:07.948 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I3LMNPZfFBaoYOOSPPxpZWa2mW0= 2022-09-03T05:37:07.949 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I3LMNPZfFBaoYOOSPPxpZWa2mW0=', 'Content-Length': '0'} 2022-09-03T05:37:07.953 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000381a59583356d9ed-006312e803-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.954 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:07.955 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:07.955 DEBUG:boto:Method: HEAD 2022-09-03T05:37:07.956 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:07.956 DEBUG:boto:Data: 2022-09-03T05:37:07.958 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.960 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:07.960 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.961 DEBUG:boto:Params: {} 2022-09-03T05:37:07.962 DEBUG:boto:Token: None 2022-09-03T05:37:07.962 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/obj 2022-09-03T05:37:07.963 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I3LMNPZfFBaoYOOSPPxpZWa2mW0= 2022-09-03T05:37:07.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I3LMNPZfFBaoYOOSPPxpZWa2mW0=', 'Content-Length': '0'} 2022-09-03T05:37:07.968 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008542ec59444c30f5-006312e803-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.969 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:07.969 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:07.970 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:07.971 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.972 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.972 DEBUG:boto:Method: GET 2022-09-03T05:37:07.973 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.974 DEBUG:boto:Data: 2022-09-03T05:37:07.975 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.975 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:07.976 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.977 DEBUG:boto:Params: {} 2022-09-03T05:37:07.978 DEBUG:boto:Token: None 2022-09-03T05:37:07.979 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.980 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IeRljyLjQo3dziyIuid3yQWFR2E= 2022-09-03T05:37:07.982 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IeRljyLjQo3dziyIuid3yQWFR2E=', 'Content-Length': '0'} 2022-09-03T05:37:07.984 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000078971aa92e54356f-006312e803-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:07.986 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:07.987 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:07.988 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.989 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.990 DEBUG:boto:Method: GET 2022-09-03T05:37:07.991 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.992 DEBUG:boto:Data: 2022-09-03T05:37:07.993 DEBUG:boto:Headers: {} 2022-09-03T05:37:07.994 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:07.994 DEBUG:boto:Port: 8000 2022-09-03T05:37:07.995 DEBUG:boto:Params: {} 2022-09-03T05:37:07.996 DEBUG:boto:Token: None 2022-09-03T05:37:07.997 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:07 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:07.998 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IeRljyLjQo3dziyIuid3yQWFR2E= 2022-09-03T05:37:07.999 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IeRljyLjQo3dziyIuid3yQWFR2E=', 'Content-Length': '0'} 2022-09-03T05:37:08.001 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005fc0f76fec673f30-006312e804-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.046 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:08.048 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:08.049 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.050 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.050 DEBUG:boto:Method: GET 2022-09-03T05:37:08.051 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.052 DEBUG:boto:Data: 2022-09-03T05:37:08.053 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.055 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:08.055 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.056 DEBUG:boto:Params: {} 2022-09-03T05:37:08.057 DEBUG:boto:Token: None 2022-09-03T05:37:08.058 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/ 2022-09-03T05:37:08.059 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA= 2022-09-03T05:37:08.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA=', 'Content-Length': '0'} 2022-09-03T05:37:08.063 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000408e487778372035-006312e804-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.064 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:08.066 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:08.067 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:08.068 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.069 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.070 DEBUG:boto:Method: GET 2022-09-03T05:37:08.071 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.071 DEBUG:boto:Data: 2022-09-03T05:37:08.072 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.072 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:08.073 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.073 DEBUG:boto:Params: {} 2022-09-03T05:37:08.074 DEBUG:boto:Token: None 2022-09-03T05:37:08.075 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/ 2022-09-03T05:37:08.075 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA= 2022-09-03T05:37:08.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA=', 'Content-Length': '0'} 2022-09-03T05:37:08.079 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006cacebbd669fd4fe-006312e804-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.081 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:08.082 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:08.083 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.084 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.085 DEBUG:boto:path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.086 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.087 DEBUG:boto:Method: GET 2022-09-03T05:37:08.088 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.089 DEBUG:boto:Data: 2022-09-03T05:37:08.091 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.092 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:08.093 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.094 DEBUG:boto:Params: {} 2022-09-03T05:37:08.095 DEBUG:boto:Token: None 2022-09-03T05:37:08.096 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.097 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0oBSo7OEq4ftLwhPbIEPIeJRVWQ= 2022-09-03T05:37:08.098 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0oBSo7OEq4ftLwhPbIEPIeJRVWQ=', 'Content-Length': '0'} 2022-09-03T05:37:08.101 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'hqLZJOZHmRi2EtNletjE-T3uOA2T8pT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000004d934598c9334fe-006312e804-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.103 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.104 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.105 DEBUG:boto:path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.106 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.107 DEBUG:boto:Method: GET 2022-09-03T05:37:08.108 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.109 DEBUG:boto:Data: 2022-09-03T05:37:08.110 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.111 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:08.112 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.113 DEBUG:boto:Params: {} 2022-09-03T05:37:08.114 DEBUG:boto:Token: None 2022-09-03T05:37:08.115 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.116 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0oBSo7OEq4ftLwhPbIEPIeJRVWQ= 2022-09-03T05:37:08.118 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0oBSo7OEq4ftLwhPbIEPIeJRVWQ=', 'Content-Length': '0'} 2022-09-03T05:37:08.120 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'hqLZJOZHmRi2EtNletjE-T3uOA2T8pT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001d7482a7a28a1c6d-006312e804-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.122 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.123 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.123 DEBUG:boto:path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.124 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.125 DEBUG:boto:Method: HEAD 2022-09-03T05:37:08.125 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.126 DEBUG:boto:Data: 2022-09-03T05:37:08.126 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.127 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:08.127 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.128 DEBUG:boto:Params: {} 2022-09-03T05:37:08.128 DEBUG:boto:Token: None 2022-09-03T05:37:08.129 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.130 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a5brE8XIB2x4AqxrILcEdsnPfy0= 2022-09-03T05:37:08.130 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a5brE8XIB2x4AqxrILcEdsnPfy0=', 'Content-Length': '0'} 2022-09-03T05:37:08.132 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'hqLZJOZHmRi2EtNletjE-T3uOA2T8pT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000bbbcfaa7f8e9478-006312e804-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.133 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.134 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.134 DEBUG:boto:path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.135 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.135 DEBUG:boto:Method: HEAD 2022-09-03T05:37:08.136 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.136 DEBUG:boto:Data: 2022-09-03T05:37:08.137 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.137 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:08.138 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.138 DEBUG:boto:Params: {} 2022-09-03T05:37:08.139 DEBUG:boto:Token: None 2022-09-03T05:37:08.139 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.140 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a5brE8XIB2x4AqxrILcEdsnPfy0= 2022-09-03T05:37:08.141 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a5brE8XIB2x4AqxrILcEdsnPfy0=', 'Content-Length': '0'} 2022-09-03T05:37:08.143 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'hqLZJOZHmRi2EtNletjE-T3uOA2T8pT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008bcfdf2049463a33-006312e804-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.144 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:08.144 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.145 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.145 DEBUG:boto:path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.146 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.147 DEBUG:boto:Method: GET 2022-09-03T05:37:08.147 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.148 DEBUG:boto:Data: 2022-09-03T05:37:08.148 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.149 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:08.149 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.150 DEBUG:boto:Params: {} 2022-09-03T05:37:08.150 DEBUG:boto:Token: None 2022-09-03T05:37:08.151 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.152 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0oBSo7OEq4ftLwhPbIEPIeJRVWQ= 2022-09-03T05:37:08.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0oBSo7OEq4ftLwhPbIEPIeJRVWQ=', 'Content-Length': '0'} 2022-09-03T05:37:08.158 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'hqLZJOZHmRi2EtNletjE-T3uOA2T8pT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000de4789c6dc321d03-006312e804-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.159 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.160 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.160 DEBUG:boto:path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.161 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.161 DEBUG:boto:Method: GET 2022-09-03T05:37:08.162 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.163 DEBUG:boto:Data: 2022-09-03T05:37:08.163 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.164 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:08.164 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.165 DEBUG:boto:Params: {} 2022-09-03T05:37:08.166 DEBUG:boto:Token: None 2022-09-03T05:37:08.166 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:08.167 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0oBSo7OEq4ftLwhPbIEPIeJRVWQ= 2022-09-03T05:37:08.167 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0oBSo7OEq4ftLwhPbIEPIeJRVWQ=', 'Content-Length': '0'} 2022-09-03T05:37:08.169 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'hqLZJOZHmRi2EtNletjE-T3uOA2T8pT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000616c1ba4fb6718de-006312e804-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.210 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:08.211 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:08.211 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.212 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.213 DEBUG:boto:Method: GET 2022-09-03T05:37:08.213 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.214 DEBUG:boto:Data: 2022-09-03T05:37:08.215 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.216 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:08.218 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.219 DEBUG:boto:Params: {} 2022-09-03T05:37:08.220 DEBUG:boto:Token: None 2022-09-03T05:37:08.221 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/ 2022-09-03T05:37:08.222 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA= 2022-09-03T05:37:08.223 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA=', 'Content-Length': '0'} 2022-09-03T05:37:08.226 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d8689475923678ee-006312e804-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.227 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:08.229 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:08.229 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:08.231 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.231 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.232 DEBUG:boto:Method: GET 2022-09-03T05:37:08.233 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.234 DEBUG:boto:Data: 2022-09-03T05:37:08.235 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.235 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:08.236 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.237 DEBUG:boto:Params: {} 2022-09-03T05:37:08.238 DEBUG:boto:Token: None 2022-09-03T05:37:08.239 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/ 2022-09-03T05:37:08.240 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA= 2022-09-03T05:37:08.241 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA=', 'Content-Length': '0'} 2022-09-03T05:37:08.244 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000079e3968cb1d57107-006312e804-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.245 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:08.246 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.247 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.248 DEBUG:boto:Method: HEAD 2022-09-03T05:37:08.249 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:08.250 DEBUG:boto:Data: 2022-09-03T05:37:08.251 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.252 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:08.253 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.254 DEBUG:boto:Params: {} 2022-09-03T05:37:08.255 DEBUG:boto:Token: None 2022-09-03T05:37:08.256 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj 2022-09-03T05:37:08.257 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ruC3EJ3J/pFKOOCeHQIWnWX7znI= 2022-09-03T05:37:08.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ruC3EJ3J/pFKOOCeHQIWnWX7znI=', 'Content-Length': '0'} 2022-09-03T05:37:08.263 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000746410a3d9727b09-006312e804-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.264 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.265 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.266 DEBUG:boto:Method: HEAD 2022-09-03T05:37:08.268 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:08.269 DEBUG:boto:Data: 2022-09-03T05:37:08.269 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.270 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:08.270 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.271 DEBUG:boto:Params: {} 2022-09-03T05:37:08.271 DEBUG:boto:Token: None 2022-09-03T05:37:08.272 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj 2022-09-03T05:37:08.273 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ruC3EJ3J/pFKOOCeHQIWnWX7znI= 2022-09-03T05:37:08.273 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ruC3EJ3J/pFKOOCeHQIWnWX7znI=', 'Content-Length': '0'} 2022-09-03T05:37:08.277 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cb61577a19ea185e-006312e804-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.277 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:08.278 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.279 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.279 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.280 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.280 DEBUG:boto:Method: GET 2022-09-03T05:37:08.281 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.282 DEBUG:boto:Data: 2022-09-03T05:37:08.282 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.283 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:08.283 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.284 DEBUG:boto:Params: {} 2022-09-03T05:37:08.285 DEBUG:boto:Token: None 2022-09-03T05:37:08.286 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.287 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hbZkM+jcjabOc9f+QRW7FLm6qn4= 2022-09-03T05:37:08.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hbZkM+jcjabOc9f+QRW7FLm6qn4=', 'Content-Length': '0'} 2022-09-03T05:37:08.291 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006f35f988444cb546-006312e804-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.334 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.336 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.337 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.338 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.339 DEBUG:boto:Method: GET 2022-09-03T05:37:08.340 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.341 DEBUG:boto:Data: 2022-09-03T05:37:08.342 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.343 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:08.344 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.345 DEBUG:boto:Params: {} 2022-09-03T05:37:08.346 DEBUG:boto:Token: None 2022-09-03T05:37:08.347 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.348 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hbZkM+jcjabOc9f+QRW7FLm6qn4= 2022-09-03T05:37:08.349 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hbZkM+jcjabOc9f+QRW7FLm6qn4=', 'Content-Length': '0'} 2022-09-03T05:37:08.352 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fb9ea52eea66a0d1-006312e804-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.353 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:08.354 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:08.355 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.356 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.357 DEBUG:boto:Method: GET 2022-09-03T05:37:08.358 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.359 DEBUG:boto:Data: 2022-09-03T05:37:08.360 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.360 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:08.361 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.361 DEBUG:boto:Params: {} 2022-09-03T05:37:08.362 DEBUG:boto:Token: None 2022-09-03T05:37:08.363 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/ 2022-09-03T05:37:08.363 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA= 2022-09-03T05:37:08.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA=', 'Content-Length': '0'} 2022-09-03T05:37:08.367 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007097897317254ff9-006312e804-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.368 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:08.369 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:08.369 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:08.370 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.370 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.371 DEBUG:boto:Method: GET 2022-09-03T05:37:08.371 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.372 DEBUG:boto:Data: 2022-09-03T05:37:08.373 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.373 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:08.374 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.374 DEBUG:boto:Params: {} 2022-09-03T05:37:08.375 DEBUG:boto:Token: None 2022-09-03T05:37:08.375 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/ 2022-09-03T05:37:08.376 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA= 2022-09-03T05:37:08.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA=', 'Content-Length': '0'} 2022-09-03T05:37:08.384 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a4d8303ac72e96af-006312e804-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.385 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:08.386 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:08.386 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.387 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.388 DEBUG:boto:path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.388 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.389 DEBUG:boto:Method: GET 2022-09-03T05:37:08.390 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.391 DEBUG:boto:Data: 2022-09-03T05:37:08.392 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.393 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:08.394 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.395 DEBUG:boto:Params: {} 2022-09-03T05:37:08.396 DEBUG:boto:Token: None 2022-09-03T05:37:08.397 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.398 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MI0rM+CXGwDvkfAe4nTm55qvIDM= 2022-09-03T05:37:08.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MI0rM+CXGwDvkfAe4nTm55qvIDM=', 'Content-Length': '0'} 2022-09-03T05:37:08.402 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'mBYgYYD3drKvdzws.hh3-36xKvL9eTH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000092f123525ec9cf9d-006312e804-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.403 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.404 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.405 DEBUG:boto:path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.405 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.407 DEBUG:boto:Method: GET 2022-09-03T05:37:08.408 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.409 DEBUG:boto:Data: 2022-09-03T05:37:08.409 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.410 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:08.412 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.413 DEBUG:boto:Params: {} 2022-09-03T05:37:08.414 DEBUG:boto:Token: None 2022-09-03T05:37:08.415 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.416 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MI0rM+CXGwDvkfAe4nTm55qvIDM= 2022-09-03T05:37:08.417 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MI0rM+CXGwDvkfAe4nTm55qvIDM=', 'Content-Length': '0'} 2022-09-03T05:37:08.419 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'mBYgYYD3drKvdzws.hh3-36xKvL9eTH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b15f11e4eba61e52-006312e804-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.420 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.421 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.421 DEBUG:boto:path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.422 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.423 DEBUG:boto:Method: HEAD 2022-09-03T05:37:08.424 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.425 DEBUG:boto:Data: 2022-09-03T05:37:08.425 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.426 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:08.427 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.428 DEBUG:boto:Params: {} 2022-09-03T05:37:08.428 DEBUG:boto:Token: None 2022-09-03T05:37:08.430 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.430 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yyj+Lzg1fbVIszp9J+zVsd1xma8= 2022-09-03T05:37:08.431 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yyj+Lzg1fbVIszp9J+zVsd1xma8=', 'Content-Length': '0'} 2022-09-03T05:37:08.434 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'mBYgYYD3drKvdzws.hh3-36xKvL9eTH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e68933b3d9218afd-006312e804-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.435 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.435 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.436 DEBUG:boto:path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.436 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.437 DEBUG:boto:Method: HEAD 2022-09-03T05:37:08.437 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.438 DEBUG:boto:Data: 2022-09-03T05:37:08.438 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.439 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:08.440 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.440 DEBUG:boto:Params: {} 2022-09-03T05:37:08.441 DEBUG:boto:Token: None 2022-09-03T05:37:08.441 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.442 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yyj+Lzg1fbVIszp9J+zVsd1xma8= 2022-09-03T05:37:08.443 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yyj+Lzg1fbVIszp9J+zVsd1xma8=', 'Content-Length': '0'} 2022-09-03T05:37:08.445 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'mBYgYYD3drKvdzws.hh3-36xKvL9eTH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005e1cf7790703db1b-006312e804-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.446 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:08.446 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.447 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.447 DEBUG:boto:path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.448 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.448 DEBUG:boto:Method: GET 2022-09-03T05:37:08.449 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.450 DEBUG:boto:Data: 2022-09-03T05:37:08.450 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.451 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:08.451 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.452 DEBUG:boto:Params: {} 2022-09-03T05:37:08.452 DEBUG:boto:Token: None 2022-09-03T05:37:08.453 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.454 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MI0rM+CXGwDvkfAe4nTm55qvIDM= 2022-09-03T05:37:08.454 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MI0rM+CXGwDvkfAe4nTm55qvIDM=', 'Content-Length': '0'} 2022-09-03T05:37:08.457 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'mBYgYYD3drKvdzws.hh3-36xKvL9eTH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bf3bf89c7790c415-006312e804-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.502 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.503 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.504 DEBUG:boto:path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.504 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.505 DEBUG:boto:Method: GET 2022-09-03T05:37:08.506 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.507 DEBUG:boto:Data: 2022-09-03T05:37:08.507 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.508 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:08.509 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.510 DEBUG:boto:Params: {} 2022-09-03T05:37:08.511 DEBUG:boto:Token: None 2022-09-03T05:37:08.512 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:08.512 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MI0rM+CXGwDvkfAe4nTm55qvIDM= 2022-09-03T05:37:08.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MI0rM+CXGwDvkfAe4nTm55qvIDM=', 'Content-Length': '0'} 2022-09-03T05:37:08.516 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'mBYgYYD3drKvdzws.hh3-36xKvL9eTH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008c945b338d901320-006312e804-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.516 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:08.517 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:08.517 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.518 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.519 DEBUG:boto:Method: GET 2022-09-03T05:37:08.519 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.520 DEBUG:boto:Data: 2022-09-03T05:37:08.520 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.521 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:08.521 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.522 DEBUG:boto:Params: {} 2022-09-03T05:37:08.523 DEBUG:boto:Token: None 2022-09-03T05:37:08.524 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/ 2022-09-03T05:37:08.525 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA= 2022-09-03T05:37:08.527 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA=', 'Content-Length': '0'} 2022-09-03T05:37:08.531 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006b726ce91cd06cce-006312e804-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.533 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:08.534 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:08.535 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:08.535 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.536 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.537 DEBUG:boto:Method: GET 2022-09-03T05:37:08.537 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.538 DEBUG:boto:Data: 2022-09-03T05:37:08.538 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.540 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:08.540 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.541 DEBUG:boto:Params: {} 2022-09-03T05:37:08.541 DEBUG:boto:Token: None 2022-09-03T05:37:08.542 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/ 2022-09-03T05:37:08.543 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA= 2022-09-03T05:37:08.543 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA=', 'Content-Length': '0'} 2022-09-03T05:37:08.547 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e785f5ee7995e20f-006312e804-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.548 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:08.548 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.549 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.550 DEBUG:boto:Method: HEAD 2022-09-03T05:37:08.550 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:08.551 DEBUG:boto:Data: 2022-09-03T05:37:08.551 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.552 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:08.552 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.553 DEBUG:boto:Params: {} 2022-09-03T05:37:08.553 DEBUG:boto:Token: None 2022-09-03T05:37:08.554 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj 2022-09-03T05:37:08.555 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ruC3EJ3J/pFKOOCeHQIWnWX7znI= 2022-09-03T05:37:08.555 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ruC3EJ3J/pFKOOCeHQIWnWX7znI=', 'Content-Length': '0'} 2022-09-03T05:37:08.559 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006e84017b7eec59cd-006312e804-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.560 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.561 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.561 DEBUG:boto:Method: HEAD 2022-09-03T05:37:08.562 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:08.562 DEBUG:boto:Data: 2022-09-03T05:37:08.563 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.563 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:08.564 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.564 DEBUG:boto:Params: {} 2022-09-03T05:37:08.565 DEBUG:boto:Token: None 2022-09-03T05:37:08.566 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj 2022-09-03T05:37:08.566 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ruC3EJ3J/pFKOOCeHQIWnWX7znI= 2022-09-03T05:37:08.569 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ruC3EJ3J/pFKOOCeHQIWnWX7znI=', 'Content-Length': '0'} 2022-09-03T05:37:08.572 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002eb2d7b50414ac84-006312e804-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.573 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:08.574 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.574 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.575 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.575 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.576 DEBUG:boto:Method: GET 2022-09-03T05:37:08.576 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.577 DEBUG:boto:Data: 2022-09-03T05:37:08.578 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.579 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:08.580 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.580 DEBUG:boto:Params: {} 2022-09-03T05:37:08.581 DEBUG:boto:Token: None 2022-09-03T05:37:08.582 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.582 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hbZkM+jcjabOc9f+QRW7FLm6qn4= 2022-09-03T05:37:08.583 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hbZkM+jcjabOc9f+QRW7FLm6qn4=', 'Content-Length': '0'} 2022-09-03T05:37:08.585 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fb928c4eb62cd005-006312e804-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.626 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.627 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.628 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.628 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.629 DEBUG:boto:Method: GET 2022-09-03T05:37:08.629 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.630 DEBUG:boto:Data: 2022-09-03T05:37:08.630 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.631 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:08.631 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.632 DEBUG:boto:Params: {} 2022-09-03T05:37:08.633 DEBUG:boto:Token: None 2022-09-03T05:37:08.633 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.634 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hbZkM+jcjabOc9f+QRW7FLm6qn4= 2022-09-03T05:37:08.634 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hbZkM+jcjabOc9f+QRW7FLm6qn4=', 'Content-Length': '0'} 2022-09-03T05:37:08.637 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c090c5a7bf012eed-006312e804-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.638 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:08.639 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:08.640 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.640 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.641 DEBUG:boto:Method: GET 2022-09-03T05:37:08.642 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.642 DEBUG:boto:Data: 2022-09-03T05:37:08.643 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.643 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:08.644 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.644 DEBUG:boto:Params: {} 2022-09-03T05:37:08.645 DEBUG:boto:Token: None 2022-09-03T05:37:08.646 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/ 2022-09-03T05:37:08.647 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA= 2022-09-03T05:37:08.648 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA=', 'Content-Length': '0'} 2022-09-03T05:37:08.651 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f62884d763b47b6c-006312e804-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.652 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:08.652 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:08.653 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:08.654 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.654 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.655 DEBUG:boto:Method: GET 2022-09-03T05:37:08.655 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.656 DEBUG:boto:Data: 2022-09-03T05:37:08.656 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.657 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:08.658 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.658 DEBUG:boto:Params: {} 2022-09-03T05:37:08.659 DEBUG:boto:Token: None 2022-09-03T05:37:08.659 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/ 2022-09-03T05:37:08.660 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA= 2022-09-03T05:37:08.661 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA=', 'Content-Length': '0'} 2022-09-03T05:37:08.664 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cec884e6e1fc045e-006312e804-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.664 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:08.665 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:08.666 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.666 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.667 DEBUG:boto:path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.667 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.668 DEBUG:boto:Method: GET 2022-09-03T05:37:08.668 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.669 DEBUG:boto:Data: 2022-09-03T05:37:08.669 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.670 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:08.671 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.671 DEBUG:boto:Params: {} 2022-09-03T05:37:08.672 DEBUG:boto:Token: None 2022-09-03T05:37:08.672 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.673 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:khjJVoCWTcOYp735hnedZPxXJvo= 2022-09-03T05:37:08.674 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:khjJVoCWTcOYp735hnedZPxXJvo=', 'Content-Length': '0'} 2022-09-03T05:37:08.676 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ae9c11e9d64aabef-006312e804-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.677 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.677 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.678 DEBUG:boto:path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.678 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.679 DEBUG:boto:Method: GET 2022-09-03T05:37:08.679 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.680 DEBUG:boto:Data: 2022-09-03T05:37:08.680 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.681 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:08.681 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.682 DEBUG:boto:Params: {} 2022-09-03T05:37:08.684 DEBUG:boto:Token: None 2022-09-03T05:37:08.685 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.686 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:khjJVoCWTcOYp735hnedZPxXJvo= 2022-09-03T05:37:08.686 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:khjJVoCWTcOYp735hnedZPxXJvo=', 'Content-Length': '0'} 2022-09-03T05:37:08.689 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006d7880584e841b9d-006312e804-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.690 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.691 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.691 DEBUG:boto:path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.692 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.692 DEBUG:boto:Method: HEAD 2022-09-03T05:37:08.693 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.694 DEBUG:boto:Data: 2022-09-03T05:37:08.695 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.696 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:08.696 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.697 DEBUG:boto:Params: {} 2022-09-03T05:37:08.698 DEBUG:boto:Token: None 2022-09-03T05:37:08.699 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.700 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k0cOMgjhsuTFt4DHEyqLF9DLqSw= 2022-09-03T05:37:08.701 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k0cOMgjhsuTFt4DHEyqLF9DLqSw=', 'Content-Length': '0'} 2022-09-03T05:37:08.703 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e4a6289d9ed412b2-006312e804-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.704 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.705 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.705 DEBUG:boto:path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.706 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.707 DEBUG:boto:Method: HEAD 2022-09-03T05:37:08.708 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.709 DEBUG:boto:Data: 2022-09-03T05:37:08.710 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.712 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:08.712 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.713 DEBUG:boto:Params: {} 2022-09-03T05:37:08.714 DEBUG:boto:Token: None 2022-09-03T05:37:08.715 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.716 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k0cOMgjhsuTFt4DHEyqLF9DLqSw= 2022-09-03T05:37:08.717 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k0cOMgjhsuTFt4DHEyqLF9DLqSw=', 'Content-Length': '0'} 2022-09-03T05:37:08.720 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000043bc032bc9f97925-006312e804-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.721 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:08.722 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.723 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.724 DEBUG:boto:path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.725 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.726 DEBUG:boto:Method: GET 2022-09-03T05:37:08.727 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.728 DEBUG:boto:Data: 2022-09-03T05:37:08.728 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.729 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:08.729 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.730 DEBUG:boto:Params: {} 2022-09-03T05:37:08.730 DEBUG:boto:Token: None 2022-09-03T05:37:08.731 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.732 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:khjJVoCWTcOYp735hnedZPxXJvo= 2022-09-03T05:37:08.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:khjJVoCWTcOYp735hnedZPxXJvo=', 'Content-Length': '0'} 2022-09-03T05:37:08.735 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d765d7dae0e8e3bc-006312e804-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.736 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.736 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.737 DEBUG:boto:path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.737 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.738 DEBUG:boto:Method: GET 2022-09-03T05:37:08.739 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.740 DEBUG:boto:Data: 2022-09-03T05:37:08.741 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.742 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:08.743 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.743 DEBUG:boto:Params: {} 2022-09-03T05:37:08.744 DEBUG:boto:Token: None 2022-09-03T05:37:08.745 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:08.745 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:khjJVoCWTcOYp735hnedZPxXJvo= 2022-09-03T05:37:08.746 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:khjJVoCWTcOYp735hnedZPxXJvo=', 'Content-Length': '0'} 2022-09-03T05:37:08.748 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000029c0b3fe2dcfaf8b-006312e804-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.749 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:08.749 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:08.750 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.751 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.751 DEBUG:boto:Method: GET 2022-09-03T05:37:08.752 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.752 DEBUG:boto:Data: 2022-09-03T05:37:08.753 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.753 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:08.754 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.754 DEBUG:boto:Params: {} 2022-09-03T05:37:08.755 DEBUG:boto:Token: None 2022-09-03T05:37:08.755 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/ 2022-09-03T05:37:08.756 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA= 2022-09-03T05:37:08.757 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA=', 'Content-Length': '0'} 2022-09-03T05:37:08.760 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003f18c7a2eb52e222-006312e804-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.802 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:08.804 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:08.805 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:08.808 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.809 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.810 DEBUG:boto:Method: GET 2022-09-03T05:37:08.811 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.811 DEBUG:boto:Data: 2022-09-03T05:37:08.812 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.813 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:08.813 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.814 DEBUG:boto:Params: {} 2022-09-03T05:37:08.815 DEBUG:boto:Token: None 2022-09-03T05:37:08.816 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/ 2022-09-03T05:37:08.817 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA= 2022-09-03T05:37:08.818 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA=', 'Content-Length': '0'} 2022-09-03T05:37:08.822 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006b7bd4c7efaf904e-006312e804-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.823 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:08.825 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.826 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.827 DEBUG:boto:Method: HEAD 2022-09-03T05:37:08.828 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:08.829 DEBUG:boto:Data: 2022-09-03T05:37:08.829 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.830 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:08.831 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.832 DEBUG:boto:Params: {} 2022-09-03T05:37:08.833 DEBUG:boto:Token: None 2022-09-03T05:37:08.834 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj 2022-09-03T05:37:08.835 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ruC3EJ3J/pFKOOCeHQIWnWX7znI= 2022-09-03T05:37:08.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ruC3EJ3J/pFKOOCeHQIWnWX7znI=', 'Content-Length': '0'} 2022-09-03T05:37:08.843 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000b16de4fe85f8026-006312e804-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.845 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.846 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.847 DEBUG:boto:Method: HEAD 2022-09-03T05:37:08.848 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:08.850 DEBUG:boto:Data: 2022-09-03T05:37:08.851 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.855 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:08.856 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.857 DEBUG:boto:Params: {} 2022-09-03T05:37:08.858 DEBUG:boto:Token: None 2022-09-03T05:37:08.859 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj 2022-09-03T05:37:08.860 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ruC3EJ3J/pFKOOCeHQIWnWX7znI= 2022-09-03T05:37:08.861 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ruC3EJ3J/pFKOOCeHQIWnWX7znI=', 'Content-Length': '0'} 2022-09-03T05:37:08.865 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000eee7e29db801e137-006312e804-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.867 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:08.868 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.868 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.870 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.871 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.872 DEBUG:boto:Method: GET 2022-09-03T05:37:08.873 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.875 DEBUG:boto:Data: 2022-09-03T05:37:08.876 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.877 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:08.878 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.879 DEBUG:boto:Params: {} 2022-09-03T05:37:08.880 DEBUG:boto:Token: None 2022-09-03T05:37:08.881 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.882 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hbZkM+jcjabOc9f+QRW7FLm6qn4= 2022-09-03T05:37:08.883 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hbZkM+jcjabOc9f+QRW7FLm6qn4=', 'Content-Length': '0'} 2022-09-03T05:37:08.887 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000034b26c7b091705e7-006312e804-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.888 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.889 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.890 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.890 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.891 DEBUG:boto:Method: GET 2022-09-03T05:37:08.893 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.894 DEBUG:boto:Data: 2022-09-03T05:37:08.896 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.896 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:08.897 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.897 DEBUG:boto:Params: {} 2022-09-03T05:37:08.898 DEBUG:boto:Token: None 2022-09-03T05:37:08.898 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:08.899 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hbZkM+jcjabOc9f+QRW7FLm6qn4= 2022-09-03T05:37:08.900 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hbZkM+jcjabOc9f+QRW7FLm6qn4=', 'Content-Length': '0'} 2022-09-03T05:37:08.903 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001aee2e01aa64afda-006312e804-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.904 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:08.904 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:08.905 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.905 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.906 DEBUG:boto:Method: GET 2022-09-03T05:37:08.906 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.907 DEBUG:boto:Data: 2022-09-03T05:37:08.907 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.908 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:08.908 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.909 DEBUG:boto:Params: {} 2022-09-03T05:37:08.910 DEBUG:boto:Token: None 2022-09-03T05:37:08.910 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/ 2022-09-03T05:37:08.911 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA= 2022-09-03T05:37:08.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA=', 'Content-Length': '0'} 2022-09-03T05:37:08.914 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ff23faea29d05f21-006312e804-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.915 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:08.916 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:08.917 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:08.917 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.918 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.918 DEBUG:boto:Method: GET 2022-09-03T05:37:08.919 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:08.919 DEBUG:boto:Data: 2022-09-03T05:37:08.920 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.920 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:08.921 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.921 DEBUG:boto:Params: {} 2022-09-03T05:37:08.922 DEBUG:boto:Token: None 2022-09-03T05:37:08.923 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/ 2022-09-03T05:37:08.923 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA= 2022-09-03T05:37:08.924 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fjD++c9hfRE9tDT+UnWaMIaKWEA=', 'Content-Length': '0'} 2022-09-03T05:37:08.927 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000034937d242bac51a1-006312e804-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.928 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:08.929 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:08.929 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.930 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.931 DEBUG:boto:path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:08.931 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:08.932 DEBUG:boto:Method: GET 2022-09-03T05:37:08.932 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:08.933 DEBUG:boto:Data: 2022-09-03T05:37:08.934 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.934 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:08.935 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.936 DEBUG:boto:Params: {} 2022-09-03T05:37:08.937 DEBUG:boto:Token: None 2022-09-03T05:37:08.938 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:08.939 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZRmCrmAItYyTSOrjBv733ZfDMBM= 2022-09-03T05:37:08.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZRmCrmAItYyTSOrjBv733ZfDMBM=', 'Content-Length': '0'} 2022-09-03T05:37:08.942 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004275efb78fc37c4f-006312e804-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.986 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.987 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.987 DEBUG:boto:path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:08.988 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:08.988 DEBUG:boto:Method: GET 2022-09-03T05:37:08.989 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:08.990 DEBUG:boto:Data: 2022-09-03T05:37:08.990 DEBUG:boto:Headers: {} 2022-09-03T05:37:08.991 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:08.991 DEBUG:boto:Port: 8000 2022-09-03T05:37:08.992 DEBUG:boto:Params: {} 2022-09-03T05:37:08.992 DEBUG:boto:Token: None 2022-09-03T05:37:08.993 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:08 GMT /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:08.994 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZRmCrmAItYyTSOrjBv733ZfDMBM= 2022-09-03T05:37:08.994 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZRmCrmAItYyTSOrjBv733ZfDMBM=', 'Content-Length': '0'} 2022-09-03T05:37:08.997 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000b3fbc01dda1abc4-006312e804-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:08.998 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:08.998 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:08.999 DEBUG:boto:path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:08.999 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:09.000 DEBUG:boto:Method: HEAD 2022-09-03T05:37:09.000 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:09.001 DEBUG:boto:Data: 2022-09-03T05:37:09.001 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.002 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.002 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.003 DEBUG:boto:Params: {} 2022-09-03T05:37:09.003 DEBUG:boto:Token: None 2022-09-03T05:37:09.004 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:09.005 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:12IytkEksTLad1pqLL8q3e7QJsY= 2022-09-03T05:37:09.005 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:12IytkEksTLad1pqLL8q3e7QJsY=', 'Content-Length': '0'} 2022-09-03T05:37:09.008 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000008f3bb314f3e79fe-006312e805-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.009 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.009 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.010 DEBUG:boto:path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:09.010 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:09.011 DEBUG:boto:Method: HEAD 2022-09-03T05:37:09.012 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:09.012 DEBUG:boto:Data: 2022-09-03T05:37:09.013 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.013 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:09.014 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.014 DEBUG:boto:Params: {} 2022-09-03T05:37:09.015 DEBUG:boto:Token: None 2022-09-03T05:37:09.016 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:09.017 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:12IytkEksTLad1pqLL8q3e7QJsY= 2022-09-03T05:37:09.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:12IytkEksTLad1pqLL8q3e7QJsY=', 'Content-Length': '0'} 2022-09-03T05:37:09.019 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000006f7bc0dc8d78d32-006312e805-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.020 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:09.021 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.021 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.022 DEBUG:boto:path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:09.022 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:09.023 DEBUG:boto:Method: GET 2022-09-03T05:37:09.024 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:09.024 DEBUG:boto:Data: 2022-09-03T05:37:09.025 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.025 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.026 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.026 DEBUG:boto:Params: {} 2022-09-03T05:37:09.027 DEBUG:boto:Token: None 2022-09-03T05:37:09.028 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:09.028 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:38tzn/+5mAn1seg+3dxzqeeXb8o= 2022-09-03T05:37:09.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:38tzn/+5mAn1seg+3dxzqeeXb8o=', 'Content-Length': '0'} 2022-09-03T05:37:09.031 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000537dd1bd84b3f583-006312e805-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.032 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.033 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.033 DEBUG:boto:path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:09.034 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:09.034 DEBUG:boto:Method: GET 2022-09-03T05:37:09.035 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:09.035 DEBUG:boto:Data: 2022-09-03T05:37:09.036 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.036 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:09.037 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.037 DEBUG:boto:Params: {} 2022-09-03T05:37:09.038 DEBUG:boto:Token: None 2022-09-03T05:37:09.039 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:09.039 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:38tzn/+5mAn1seg+3dxzqeeXb8o= 2022-09-03T05:37:09.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:38tzn/+5mAn1seg+3dxzqeeXb8o=', 'Content-Length': '0'} 2022-09-03T05:37:09.042 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000054fded9d81ffa5cd-006312e805-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.043 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:09.044 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:09.044 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.045 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.046 DEBUG:boto:Method: GET 2022-09-03T05:37:09.046 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.047 DEBUG:boto:Data: 2022-09-03T05:37:09.047 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.048 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.049 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.049 DEBUG:boto:Params: {} 2022-09-03T05:37:09.050 DEBUG:boto:Token: None 2022-09-03T05:37:09.051 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/ 2022-09-03T05:37:09.051 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis= 2022-09-03T05:37:09.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis=', 'Content-Length': '0'} 2022-09-03T05:37:09.055 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001e4ba5a1779fe5e8-006312e805-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.056 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:09.057 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:09.058 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:09.058 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.059 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.059 DEBUG:boto:Method: GET 2022-09-03T05:37:09.060 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.061 DEBUG:boto:Data: 2022-09-03T05:37:09.061 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.062 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:09.062 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.063 DEBUG:boto:Params: {} 2022-09-03T05:37:09.063 DEBUG:boto:Token: None 2022-09-03T05:37:09.064 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/ 2022-09-03T05:37:09.065 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis= 2022-09-03T05:37:09.065 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis=', 'Content-Length': '0'} 2022-09-03T05:37:09.069 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004af215120ac3a0ac-006312e805-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.114 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:09.116 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.117 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.118 DEBUG:boto:Method: HEAD 2022-09-03T05:37:09.119 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:09.119 DEBUG:boto:Data: 2022-09-03T05:37:09.120 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.121 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.122 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.123 DEBUG:boto:Params: {} 2022-09-03T05:37:09.123 DEBUG:boto:Token: None 2022-09-03T05:37:09.124 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj 2022-09-03T05:37:09.125 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:W7AvTHuRhu5a94Cy4gAKUb7njjY= 2022-09-03T05:37:09.125 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:W7AvTHuRhu5a94Cy4gAKUb7njjY=', 'Content-Length': '0'} 2022-09-03T05:37:09.129 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000409c7b22403f46b2-006312e805-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.130 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.131 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.131 DEBUG:boto:Method: HEAD 2022-09-03T05:37:09.132 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:09.132 DEBUG:boto:Data: 2022-09-03T05:37:09.133 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.134 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:09.135 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.136 DEBUG:boto:Params: {} 2022-09-03T05:37:09.136 DEBUG:boto:Token: None 2022-09-03T05:37:09.137 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj 2022-09-03T05:37:09.138 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:W7AvTHuRhu5a94Cy4gAKUb7njjY= 2022-09-03T05:37:09.138 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:W7AvTHuRhu5a94Cy4gAKUb7njjY=', 'Content-Length': '0'} 2022-09-03T05:37:09.142 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000835590d03fd89e61-006312e805-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.143 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:09.144 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.144 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.145 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.145 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.146 DEBUG:boto:Method: GET 2022-09-03T05:37:09.147 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.147 DEBUG:boto:Data: 2022-09-03T05:37:09.148 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.148 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.149 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.150 DEBUG:boto:Params: {} 2022-09-03T05:37:09.150 DEBUG:boto:Token: None 2022-09-03T05:37:09.151 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.152 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZLr1jzrHxun9Aiv86b4FVuCV47Y= 2022-09-03T05:37:09.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZLr1jzrHxun9Aiv86b4FVuCV47Y=', 'Content-Length': '0'} 2022-09-03T05:37:09.155 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007ffb45cc135b9e22-006312e805-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.156 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.156 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.157 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.158 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.159 DEBUG:boto:Method: GET 2022-09-03T05:37:09.159 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.160 DEBUG:boto:Data: 2022-09-03T05:37:09.160 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.161 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:09.161 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.162 DEBUG:boto:Params: {} 2022-09-03T05:37:09.163 DEBUG:boto:Token: None 2022-09-03T05:37:09.163 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.164 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZLr1jzrHxun9Aiv86b4FVuCV47Y= 2022-09-03T05:37:09.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZLr1jzrHxun9Aiv86b4FVuCV47Y=', 'Content-Length': '0'} 2022-09-03T05:37:09.167 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000501f068d09536489-006312e805-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.168 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:09.169 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:09.169 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.170 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.171 DEBUG:boto:Method: GET 2022-09-03T05:37:09.171 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.172 DEBUG:boto:Data: 2022-09-03T05:37:09.173 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.173 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.174 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.174 DEBUG:boto:Params: {} 2022-09-03T05:37:09.175 DEBUG:boto:Token: None 2022-09-03T05:37:09.175 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/ 2022-09-03T05:37:09.176 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis= 2022-09-03T05:37:09.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis=', 'Content-Length': '0'} 2022-09-03T05:37:09.189 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000917bee1b78ad4738-006312e805-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.231 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:09.232 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:09.233 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:09.233 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.234 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.234 DEBUG:boto:Method: GET 2022-09-03T05:37:09.235 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.236 DEBUG:boto:Data: 2022-09-03T05:37:09.236 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.237 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:09.237 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.238 DEBUG:boto:Params: {} 2022-09-03T05:37:09.239 DEBUG:boto:Token: None 2022-09-03T05:37:09.239 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/ 2022-09-03T05:37:09.240 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis= 2022-09-03T05:37:09.241 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis=', 'Content-Length': '0'} 2022-09-03T05:37:09.245 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000a14465149d81266-006312e805-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.246 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:09.247 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:09.247 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.248 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.249 DEBUG:boto:path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.249 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.250 DEBUG:boto:Method: GET 2022-09-03T05:37:09.250 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.251 DEBUG:boto:Data: 2022-09-03T05:37:09.251 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.252 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.252 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.253 DEBUG:boto:Params: {} 2022-09-03T05:37:09.254 DEBUG:boto:Token: None 2022-09-03T05:37:09.254 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.255 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RL7zySqfi5+fDzUZ2pXlxxvmgS4= 2022-09-03T05:37:09.256 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RL7zySqfi5+fDzUZ2pXlxxvmgS4=', 'Content-Length': '0'} 2022-09-03T05:37:09.258 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fe7d210e981a1283-006312e805-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.259 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.260 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.260 DEBUG:boto:path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.261 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.261 DEBUG:boto:Method: GET 2022-09-03T05:37:09.262 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.263 DEBUG:boto:Data: 2022-09-03T05:37:09.263 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.264 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:09.264 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.265 DEBUG:boto:Params: {} 2022-09-03T05:37:09.266 DEBUG:boto:Token: None 2022-09-03T05:37:09.266 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.267 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RL7zySqfi5+fDzUZ2pXlxxvmgS4= 2022-09-03T05:37:09.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RL7zySqfi5+fDzUZ2pXlxxvmgS4=', 'Content-Length': '0'} 2022-09-03T05:37:09.270 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fd80690b7ce315fc-006312e805-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.271 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.272 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.272 DEBUG:boto:path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.273 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.273 DEBUG:boto:Method: HEAD 2022-09-03T05:37:09.274 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.274 DEBUG:boto:Data: 2022-09-03T05:37:09.275 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.275 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.276 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.276 DEBUG:boto:Params: {} 2022-09-03T05:37:09.277 DEBUG:boto:Token: None 2022-09-03T05:37:09.277 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.278 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JWJ+BappDzb/P2FI5wOh4h5CHhA= 2022-09-03T05:37:09.279 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JWJ+BappDzb/P2FI5wOh4h5CHhA=', 'Content-Length': '0'} 2022-09-03T05:37:09.281 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f8c01298310d3adf-006312e805-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.282 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.283 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.283 DEBUG:boto:path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.284 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.284 DEBUG:boto:Method: HEAD 2022-09-03T05:37:09.284 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.285 DEBUG:boto:Data: 2022-09-03T05:37:09.285 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.286 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:09.286 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.287 DEBUG:boto:Params: {} 2022-09-03T05:37:09.288 DEBUG:boto:Token: None 2022-09-03T05:37:09.288 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.289 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JWJ+BappDzb/P2FI5wOh4h5CHhA= 2022-09-03T05:37:09.289 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JWJ+BappDzb/P2FI5wOh4h5CHhA=', 'Content-Length': '0'} 2022-09-03T05:37:09.292 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003b86bac6a68692e1-006312e805-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.292 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:09.293 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.293 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.294 DEBUG:boto:path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.295 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.295 DEBUG:boto:Method: GET 2022-09-03T05:37:09.296 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.296 DEBUG:boto:Data: 2022-09-03T05:37:09.297 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.298 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.298 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.300 DEBUG:boto:Params: {} 2022-09-03T05:37:09.300 DEBUG:boto:Token: None 2022-09-03T05:37:09.301 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.302 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RL7zySqfi5+fDzUZ2pXlxxvmgS4= 2022-09-03T05:37:09.303 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RL7zySqfi5+fDzUZ2pXlxxvmgS4=', 'Content-Length': '0'} 2022-09-03T05:37:09.305 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003e599430dcbaac6a-006312e805-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.306 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.306 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.307 DEBUG:boto:path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.307 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.308 DEBUG:boto:Method: GET 2022-09-03T05:37:09.308 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.309 DEBUG:boto:Data: 2022-09-03T05:37:09.309 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.310 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:09.310 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.311 DEBUG:boto:Params: {} 2022-09-03T05:37:09.312 DEBUG:boto:Token: None 2022-09-03T05:37:09.312 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:09.313 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RL7zySqfi5+fDzUZ2pXlxxvmgS4= 2022-09-03T05:37:09.313 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RL7zySqfi5+fDzUZ2pXlxxvmgS4=', 'Content-Length': '0'} 2022-09-03T05:37:09.316 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009c717d824c0a7b3c-006312e805-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.361 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:09.362 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:09.363 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.364 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.366 DEBUG:boto:Method: GET 2022-09-03T05:37:09.367 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.368 DEBUG:boto:Data: 2022-09-03T05:37:09.369 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.370 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.371 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.371 DEBUG:boto:Params: {} 2022-09-03T05:37:09.372 DEBUG:boto:Token: None 2022-09-03T05:37:09.373 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/ 2022-09-03T05:37:09.373 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis= 2022-09-03T05:37:09.375 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis=', 'Content-Length': '0'} 2022-09-03T05:37:09.379 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000005153d5ad6895285-006312e805-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.380 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:09.381 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:09.381 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:09.382 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.382 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.383 DEBUG:boto:Method: GET 2022-09-03T05:37:09.384 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.384 DEBUG:boto:Data: 2022-09-03T05:37:09.385 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.385 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:09.386 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.386 DEBUG:boto:Params: {} 2022-09-03T05:37:09.387 DEBUG:boto:Token: None 2022-09-03T05:37:09.388 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/ 2022-09-03T05:37:09.388 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis= 2022-09-03T05:37:09.389 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis=', 'Content-Length': '0'} 2022-09-03T05:37:09.393 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000003d502399bb656b3-006312e805-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.394 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:09.395 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.396 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.396 DEBUG:boto:Method: HEAD 2022-09-03T05:37:09.397 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:09.397 DEBUG:boto:Data: 2022-09-03T05:37:09.398 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.398 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.399 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.399 DEBUG:boto:Params: {} 2022-09-03T05:37:09.400 DEBUG:boto:Token: None 2022-09-03T05:37:09.401 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj 2022-09-03T05:37:09.403 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:W7AvTHuRhu5a94Cy4gAKUb7njjY= 2022-09-03T05:37:09.403 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:W7AvTHuRhu5a94Cy4gAKUb7njjY=', 'Content-Length': '0'} 2022-09-03T05:37:09.407 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c569cc21340d3f85-006312e805-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.408 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.409 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.410 DEBUG:boto:Method: HEAD 2022-09-03T05:37:09.410 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:09.411 DEBUG:boto:Data: 2022-09-03T05:37:09.412 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.413 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:09.414 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.414 DEBUG:boto:Params: {} 2022-09-03T05:37:09.415 DEBUG:boto:Token: None 2022-09-03T05:37:09.416 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj 2022-09-03T05:37:09.416 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:W7AvTHuRhu5a94Cy4gAKUb7njjY= 2022-09-03T05:37:09.417 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:W7AvTHuRhu5a94Cy4gAKUb7njjY=', 'Content-Length': '0'} 2022-09-03T05:37:09.421 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000df1669217449fd80-006312e805-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.422 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:09.424 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.425 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.426 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.427 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.428 DEBUG:boto:Method: GET 2022-09-03T05:37:09.429 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.430 DEBUG:boto:Data: 2022-09-03T05:37:09.431 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.432 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.433 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.434 DEBUG:boto:Params: {} 2022-09-03T05:37:09.435 DEBUG:boto:Token: None 2022-09-03T05:37:09.436 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.437 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZLr1jzrHxun9Aiv86b4FVuCV47Y= 2022-09-03T05:37:09.438 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZLr1jzrHxun9Aiv86b4FVuCV47Y=', 'Content-Length': '0'} 2022-09-03T05:37:09.441 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d37aed3f08f33a9f-006312e805-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.486 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.487 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.488 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.488 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.489 DEBUG:boto:Method: GET 2022-09-03T05:37:09.490 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.491 DEBUG:boto:Data: 2022-09-03T05:37:09.492 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.493 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:09.494 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.495 DEBUG:boto:Params: {} 2022-09-03T05:37:09.496 DEBUG:boto:Token: None 2022-09-03T05:37:09.497 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.498 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZLr1jzrHxun9Aiv86b4FVuCV47Y= 2022-09-03T05:37:09.500 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZLr1jzrHxun9Aiv86b4FVuCV47Y=', 'Content-Length': '0'} 2022-09-03T05:37:09.502 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000071d1e75f50147d19-006312e805-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.504 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:09.504 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:09.505 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.505 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.506 DEBUG:boto:Method: GET 2022-09-03T05:37:09.506 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.507 DEBUG:boto:Data: 2022-09-03T05:37:09.507 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.508 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.510 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.510 DEBUG:boto:Params: {} 2022-09-03T05:37:09.511 DEBUG:boto:Token: None 2022-09-03T05:37:09.512 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/ 2022-09-03T05:37:09.512 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis= 2022-09-03T05:37:09.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis=', 'Content-Length': '0'} 2022-09-03T05:37:09.519 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007e30a10fdc7b884e-006312e805-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.521 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:09.522 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:09.523 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:09.524 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.525 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.525 DEBUG:boto:Method: GET 2022-09-03T05:37:09.526 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.527 DEBUG:boto:Data: 2022-09-03T05:37:09.528 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.528 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:09.529 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.529 DEBUG:boto:Params: {} 2022-09-03T05:37:09.530 DEBUG:boto:Token: None 2022-09-03T05:37:09.531 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/ 2022-09-03T05:37:09.534 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis= 2022-09-03T05:37:09.537 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis=', 'Content-Length': '0'} 2022-09-03T05:37:09.542 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006b777075af1458f0-006312e805-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.543 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:09.544 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:09.545 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.547 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.548 DEBUG:boto:path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.548 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.549 DEBUG:boto:Method: GET 2022-09-03T05:37:09.550 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.550 DEBUG:boto:Data: 2022-09-03T05:37:09.551 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.551 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.552 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.552 DEBUG:boto:Params: {} 2022-09-03T05:37:09.553 DEBUG:boto:Token: None 2022-09-03T05:37:09.554 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.554 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+sCkOPKoO8N9mOoIGiPOKF50eEo= 2022-09-03T05:37:09.555 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+sCkOPKoO8N9mOoIGiPOKF50eEo=', 'Content-Length': '0'} 2022-09-03T05:37:09.558 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', '0PhwqB536OEXdH1QOmPxhlUfX3fwPvS'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000016a9d5c07579a40a-006312e805-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.561 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.563 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.564 DEBUG:boto:path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.564 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.565 DEBUG:boto:Method: GET 2022-09-03T05:37:09.566 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.568 DEBUG:boto:Data: 2022-09-03T05:37:09.569 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.574 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:09.575 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.575 DEBUG:boto:Params: {} 2022-09-03T05:37:09.576 DEBUG:boto:Token: None 2022-09-03T05:37:09.577 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.578 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+sCkOPKoO8N9mOoIGiPOKF50eEo= 2022-09-03T05:37:09.578 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+sCkOPKoO8N9mOoIGiPOKF50eEo=', 'Content-Length': '0'} 2022-09-03T05:37:09.581 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', '0PhwqB536OEXdH1QOmPxhlUfX3fwPvS'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000028852dbfdce8e4b6-006312e805-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.622 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.623 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.624 DEBUG:boto:path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.624 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.625 DEBUG:boto:Method: HEAD 2022-09-03T05:37:09.626 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.626 DEBUG:boto:Data: 2022-09-03T05:37:09.627 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.627 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.628 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.629 DEBUG:boto:Params: {} 2022-09-03T05:37:09.630 DEBUG:boto:Token: None 2022-09-03T05:37:09.632 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.632 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xylle5CKmsX6vQG8KGZdNxgppf8= 2022-09-03T05:37:09.633 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xylle5CKmsX6vQG8KGZdNxgppf8=', 'Content-Length': '0'} 2022-09-03T05:37:09.635 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', '0PhwqB536OEXdH1QOmPxhlUfX3fwPvS'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b7564d362877fe80-006312e805-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.636 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.637 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.637 DEBUG:boto:path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.638 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.638 DEBUG:boto:Method: HEAD 2022-09-03T05:37:09.639 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.640 DEBUG:boto:Data: 2022-09-03T05:37:09.641 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.642 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:09.643 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.643 DEBUG:boto:Params: {} 2022-09-03T05:37:09.644 DEBUG:boto:Token: None 2022-09-03T05:37:09.645 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.645 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xylle5CKmsX6vQG8KGZdNxgppf8= 2022-09-03T05:37:09.646 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xylle5CKmsX6vQG8KGZdNxgppf8=', 'Content-Length': '0'} 2022-09-03T05:37:09.648 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', '0PhwqB536OEXdH1QOmPxhlUfX3fwPvS'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000055ee3b413f31042c-006312e805-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.649 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:09.649 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.650 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.651 DEBUG:boto:path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.651 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.652 DEBUG:boto:Method: GET 2022-09-03T05:37:09.652 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.653 DEBUG:boto:Data: 2022-09-03T05:37:09.654 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.654 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.655 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.655 DEBUG:boto:Params: {} 2022-09-03T05:37:09.656 DEBUG:boto:Token: None 2022-09-03T05:37:09.657 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.657 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+sCkOPKoO8N9mOoIGiPOKF50eEo= 2022-09-03T05:37:09.658 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+sCkOPKoO8N9mOoIGiPOKF50eEo=', 'Content-Length': '0'} 2022-09-03T05:37:09.660 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', '0PhwqB536OEXdH1QOmPxhlUfX3fwPvS'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003382cc1cd2428f84-006312e805-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.661 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.662 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.662 DEBUG:boto:path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.663 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.663 DEBUG:boto:Method: GET 2022-09-03T05:37:09.664 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.665 DEBUG:boto:Data: 2022-09-03T05:37:09.666 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.666 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:09.667 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.667 DEBUG:boto:Params: {} 2022-09-03T05:37:09.668 DEBUG:boto:Token: None 2022-09-03T05:37:09.669 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:09.669 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+sCkOPKoO8N9mOoIGiPOKF50eEo= 2022-09-03T05:37:09.670 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+sCkOPKoO8N9mOoIGiPOKF50eEo=', 'Content-Length': '0'} 2022-09-03T05:37:09.673 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', '0PhwqB536OEXdH1QOmPxhlUfX3fwPvS'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002a340889d36982d4-006312e805-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.674 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:09.675 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:09.676 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.677 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.678 DEBUG:boto:Method: GET 2022-09-03T05:37:09.679 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.680 DEBUG:boto:Data: 2022-09-03T05:37:09.680 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.681 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.682 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.683 DEBUG:boto:Params: {} 2022-09-03T05:37:09.683 DEBUG:boto:Token: None 2022-09-03T05:37:09.684 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/ 2022-09-03T05:37:09.684 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis= 2022-09-03T05:37:09.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis=', 'Content-Length': '0'} 2022-09-03T05:37:09.688 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000858b0698be92461a-006312e805-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.689 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:09.690 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:09.691 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:09.691 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.692 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.692 DEBUG:boto:Method: GET 2022-09-03T05:37:09.693 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.693 DEBUG:boto:Data: 2022-09-03T05:37:09.694 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.695 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:09.695 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.696 DEBUG:boto:Params: {} 2022-09-03T05:37:09.696 DEBUG:boto:Token: None 2022-09-03T05:37:09.697 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/ 2022-09-03T05:37:09.698 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis= 2022-09-03T05:37:09.698 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis=', 'Content-Length': '0'} 2022-09-03T05:37:09.703 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000020cfed34e27d13a0-006312e805-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.703 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:09.704 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.705 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.706 DEBUG:boto:Method: HEAD 2022-09-03T05:37:09.706 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:09.708 DEBUG:boto:Data: 2022-09-03T05:37:09.708 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.709 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.710 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.711 DEBUG:boto:Params: {} 2022-09-03T05:37:09.712 DEBUG:boto:Token: None 2022-09-03T05:37:09.712 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj 2022-09-03T05:37:09.713 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:W7AvTHuRhu5a94Cy4gAKUb7njjY= 2022-09-03T05:37:09.714 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:W7AvTHuRhu5a94Cy4gAKUb7njjY=', 'Content-Length': '0'} 2022-09-03T05:37:09.718 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004d69f31a1807e984-006312e805-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.720 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.721 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.722 DEBUG:boto:Method: HEAD 2022-09-03T05:37:09.723 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:09.724 DEBUG:boto:Data: 2022-09-03T05:37:09.724 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.725 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:09.726 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.727 DEBUG:boto:Params: {} 2022-09-03T05:37:09.727 DEBUG:boto:Token: None 2022-09-03T05:37:09.728 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj 2022-09-03T05:37:09.729 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:W7AvTHuRhu5a94Cy4gAKUb7njjY= 2022-09-03T05:37:09.731 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:W7AvTHuRhu5a94Cy4gAKUb7njjY=', 'Content-Length': '0'} 2022-09-03T05:37:09.737 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f5885d32411f9616-006312e805-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.738 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:09.739 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.740 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.741 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.742 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.743 DEBUG:boto:Method: GET 2022-09-03T05:37:09.744 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.745 DEBUG:boto:Data: 2022-09-03T05:37:09.746 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.747 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.748 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.749 DEBUG:boto:Params: {} 2022-09-03T05:37:09.750 DEBUG:boto:Token: None 2022-09-03T05:37:09.751 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.752 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZLr1jzrHxun9Aiv86b4FVuCV47Y= 2022-09-03T05:37:09.753 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZLr1jzrHxun9Aiv86b4FVuCV47Y=', 'Content-Length': '0'} 2022-09-03T05:37:09.757 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ed059b12e1fd51c0-006312e805-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.799 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.800 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.801 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.802 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.803 DEBUG:boto:Method: GET 2022-09-03T05:37:09.804 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.805 DEBUG:boto:Data: 2022-09-03T05:37:09.806 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.808 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:09.809 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.810 DEBUG:boto:Params: {} 2022-09-03T05:37:09.811 DEBUG:boto:Token: None 2022-09-03T05:37:09.812 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:09.813 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZLr1jzrHxun9Aiv86b4FVuCV47Y= 2022-09-03T05:37:09.815 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZLr1jzrHxun9Aiv86b4FVuCV47Y=', 'Content-Length': '0'} 2022-09-03T05:37:09.818 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006500f2bcae7e4fd5-006312e805-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.820 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:09.821 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:09.822 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.822 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.823 DEBUG:boto:Method: GET 2022-09-03T05:37:09.824 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.824 DEBUG:boto:Data: 2022-09-03T05:37:09.825 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.825 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.827 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.827 DEBUG:boto:Params: {} 2022-09-03T05:37:09.828 DEBUG:boto:Token: None 2022-09-03T05:37:09.828 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/ 2022-09-03T05:37:09.829 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis= 2022-09-03T05:37:09.830 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis=', 'Content-Length': '0'} 2022-09-03T05:37:09.834 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000046dbc825f2f49c43-006312e805-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.834 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:09.835 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:09.836 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:09.836 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.837 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.837 DEBUG:boto:Method: GET 2022-09-03T05:37:09.838 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.839 DEBUG:boto:Data: 2022-09-03T05:37:09.839 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.840 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:09.840 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.841 DEBUG:boto:Params: {} 2022-09-03T05:37:09.842 DEBUG:boto:Token: None 2022-09-03T05:37:09.842 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/ 2022-09-03T05:37:09.843 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis= 2022-09-03T05:37:09.843 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis=', 'Content-Length': '0'} 2022-09-03T05:37:09.847 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007b64134860cd7344-006312e805-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.890 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:09.891 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:09.892 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.892 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.893 DEBUG:boto:path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.893 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.894 DEBUG:boto:Method: GET 2022-09-03T05:37:09.894 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.895 DEBUG:boto:Data: 2022-09-03T05:37:09.895 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.896 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.896 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.897 DEBUG:boto:Params: {} 2022-09-03T05:37:09.897 DEBUG:boto:Token: None 2022-09-03T05:37:09.898 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.899 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4x482D9HyM2zwFu/b7RhJ2MK7LA= 2022-09-03T05:37:09.899 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4x482D9HyM2zwFu/b7RhJ2MK7LA=', 'Content-Length': '0'} 2022-09-03T05:37:09.902 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000368957710b72d4e6-006312e805-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.903 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.903 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.904 DEBUG:boto:path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.904 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.905 DEBUG:boto:Method: GET 2022-09-03T05:37:09.905 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.906 DEBUG:boto:Data: 2022-09-03T05:37:09.906 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.907 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:09.907 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.908 DEBUG:boto:Params: {} 2022-09-03T05:37:09.909 DEBUG:boto:Token: None 2022-09-03T05:37:09.910 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.910 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4x482D9HyM2zwFu/b7RhJ2MK7LA= 2022-09-03T05:37:09.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4x482D9HyM2zwFu/b7RhJ2MK7LA=', 'Content-Length': '0'} 2022-09-03T05:37:09.913 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c30cf04b00b72f3d-006312e805-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.914 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.914 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.915 DEBUG:boto:path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.915 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.916 DEBUG:boto:Method: HEAD 2022-09-03T05:37:09.916 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.917 DEBUG:boto:Data: 2022-09-03T05:37:09.918 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.919 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.919 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.920 DEBUG:boto:Params: {} 2022-09-03T05:37:09.920 DEBUG:boto:Token: None 2022-09-03T05:37:09.921 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.921 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mguSdbYZLDlFcSOEyoaie033Kq0= 2022-09-03T05:37:09.922 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mguSdbYZLDlFcSOEyoaie033Kq0=', 'Content-Length': '0'} 2022-09-03T05:37:09.924 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000137f6c04d71c1e18-006312e805-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.926 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.927 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.928 DEBUG:boto:path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.929 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.929 DEBUG:boto:Method: HEAD 2022-09-03T05:37:09.931 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.932 DEBUG:boto:Data: 2022-09-03T05:37:09.932 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.933 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:09.934 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.935 DEBUG:boto:Params: {} 2022-09-03T05:37:09.936 DEBUG:boto:Token: None 2022-09-03T05:37:09.937 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.938 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mguSdbYZLDlFcSOEyoaie033Kq0= 2022-09-03T05:37:09.939 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mguSdbYZLDlFcSOEyoaie033Kq0=', 'Content-Length': '0'} 2022-09-03T05:37:09.942 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000660f219e3e954299-006312e805-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.944 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:09.945 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.946 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.947 DEBUG:boto:path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.948 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.949 DEBUG:boto:Method: GET 2022-09-03T05:37:09.950 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.951 DEBUG:boto:Data: 2022-09-03T05:37:09.952 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.952 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.953 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.953 DEBUG:boto:Params: {} 2022-09-03T05:37:09.954 DEBUG:boto:Token: None 2022-09-03T05:37:09.955 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.955 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4x482D9HyM2zwFu/b7RhJ2MK7LA= 2022-09-03T05:37:09.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4x482D9HyM2zwFu/b7RhJ2MK7LA=', 'Content-Length': '0'} 2022-09-03T05:37:09.959 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000099eb73f87a6a18c1-006312e805-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.961 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:09.962 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:09.962 DEBUG:boto:path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.963 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.964 DEBUG:boto:Method: GET 2022-09-03T05:37:09.965 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.966 DEBUG:boto:Data: 2022-09-03T05:37:09.967 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.968 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:09.969 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.970 DEBUG:boto:Params: {} 2022-09-03T05:37:09.971 DEBUG:boto:Token: None 2022-09-03T05:37:09.972 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:09.973 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4x482D9HyM2zwFu/b7RhJ2MK7LA= 2022-09-03T05:37:09.974 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4x482D9HyM2zwFu/b7RhJ2MK7LA=', 'Content-Length': '0'} 2022-09-03T05:37:09.977 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fbe52d41166b6df3-006312e805-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:09.978 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:09.979 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:09.980 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.981 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.981 DEBUG:boto:Method: GET 2022-09-03T05:37:09.982 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:09.983 DEBUG:boto:Data: 2022-09-03T05:37:09.984 DEBUG:boto:Headers: {} 2022-09-03T05:37:09.985 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:09.985 DEBUG:boto:Port: 8000 2022-09-03T05:37:09.986 DEBUG:boto:Params: {} 2022-09-03T05:37:09.987 DEBUG:boto:Token: None 2022-09-03T05:37:09.988 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:09 GMT /dsgvcy-19/ 2022-09-03T05:37:09.989 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis= 2022-09-03T05:37:09.989 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hFRYJSaDnRRq0KU7kJIboPqYBis=', 'Content-Length': '0'} 2022-09-03T05:37:09.993 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008f5f42013051f5e4-006312e805-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.034 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:10.036 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:10.037 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:10.038 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.039 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.041 DEBUG:boto:Method: GET 2022-09-03T05:37:10.041 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.043 DEBUG:boto:Data: 2022-09-03T05:37:10.043 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.044 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:10.045 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.046 DEBUG:boto:Params: {} 2022-09-03T05:37:10.047 DEBUG:boto:Token: None 2022-09-03T05:37:10.048 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/ 2022-09-03T05:37:10.049 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw= 2022-09-03T05:37:10.050 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw=', 'Content-Length': '0'} 2022-09-03T05:37:10.056 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c06f0e49f46d139e-006312e806-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.057 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:10.059 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.059 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.060 DEBUG:boto:Method: HEAD 2022-09-03T05:37:10.060 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:10.061 DEBUG:boto:Data: 2022-09-03T05:37:10.062 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.062 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:10.063 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.063 DEBUG:boto:Params: {} 2022-09-03T05:37:10.064 DEBUG:boto:Token: None 2022-09-03T05:37:10.065 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj 2022-09-03T05:37:10.065 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YUhhcqHw+9fs4kEZhkrEPKjeQa4= 2022-09-03T05:37:10.066 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YUhhcqHw+9fs4kEZhkrEPKjeQa4=', 'Content-Length': '0'} 2022-09-03T05:37:10.069 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002f216d4cba987140-006312e806-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.070 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.071 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.072 DEBUG:boto:Method: HEAD 2022-09-03T05:37:10.072 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:10.073 DEBUG:boto:Data: 2022-09-03T05:37:10.073 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.074 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:10.075 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.075 DEBUG:boto:Params: {} 2022-09-03T05:37:10.076 DEBUG:boto:Token: None 2022-09-03T05:37:10.076 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj 2022-09-03T05:37:10.077 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YUhhcqHw+9fs4kEZhkrEPKjeQa4= 2022-09-03T05:37:10.078 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YUhhcqHw+9fs4kEZhkrEPKjeQa4=', 'Content-Length': '0'} 2022-09-03T05:37:10.083 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000817f2ca7829eef4a-006312e806-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.084 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:10.085 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.085 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.086 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.086 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.087 DEBUG:boto:Method: GET 2022-09-03T05:37:10.087 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.088 DEBUG:boto:Data: 2022-09-03T05:37:10.088 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.089 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:10.089 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.090 DEBUG:boto:Params: {} 2022-09-03T05:37:10.091 DEBUG:boto:Token: None 2022-09-03T05:37:10.091 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.092 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5teNsOAFFRZhu6gJI2H17GB++aI= 2022-09-03T05:37:10.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5teNsOAFFRZhu6gJI2H17GB++aI=', 'Content-Length': '0'} 2022-09-03T05:37:10.096 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ef4ad1b7f4f58ec6-006312e806-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.097 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.098 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.098 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.099 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.100 DEBUG:boto:Method: GET 2022-09-03T05:37:10.101 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.102 DEBUG:boto:Data: 2022-09-03T05:37:10.102 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.103 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:10.103 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.105 DEBUG:boto:Params: {} 2022-09-03T05:37:10.106 DEBUG:boto:Token: None 2022-09-03T05:37:10.107 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.108 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5teNsOAFFRZhu6gJI2H17GB++aI= 2022-09-03T05:37:10.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5teNsOAFFRZhu6gJI2H17GB++aI=', 'Content-Length': '0'} 2022-09-03T05:37:10.114 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000505161fbdcab9fd3-006312e806-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.154 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:10.156 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:10.157 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.158 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.158 DEBUG:boto:Method: GET 2022-09-03T05:37:10.159 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.159 DEBUG:boto:Data: 2022-09-03T05:37:10.160 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.161 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:10.161 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.162 DEBUG:boto:Params: {} 2022-09-03T05:37:10.163 DEBUG:boto:Token: None 2022-09-03T05:37:10.163 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/ 2022-09-03T05:37:10.164 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw= 2022-09-03T05:37:10.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw=', 'Content-Length': '0'} 2022-09-03T05:37:10.169 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ba2ed9f4cb162bfc-006312e806-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.170 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:10.172 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:10.173 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:10.174 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.175 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.178 DEBUG:boto:Method: GET 2022-09-03T05:37:10.179 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.179 DEBUG:boto:Data: 2022-09-03T05:37:10.180 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.180 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:10.181 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.182 DEBUG:boto:Params: {} 2022-09-03T05:37:10.183 DEBUG:boto:Token: None 2022-09-03T05:37:10.183 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/ 2022-09-03T05:37:10.184 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw= 2022-09-03T05:37:10.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw=', 'Content-Length': '0'} 2022-09-03T05:37:10.189 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004ea09c026a43b778-006312e806-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.190 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:10.192 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:10.193 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.194 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.195 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.195 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.196 DEBUG:boto:Method: GET 2022-09-03T05:37:10.197 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.200 DEBUG:boto:Data: 2022-09-03T05:37:10.201 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.201 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:10.202 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.203 DEBUG:boto:Params: {} 2022-09-03T05:37:10.204 DEBUG:boto:Token: None 2022-09-03T05:37:10.204 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.205 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jTrvQ6V2hlrYh09wzGsRiKuwVwI= 2022-09-03T05:37:10.206 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jTrvQ6V2hlrYh09wzGsRiKuwVwI=', 'Content-Length': '0'} 2022-09-03T05:37:10.208 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000adfc1343c8bc1d4f-006312e806-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.209 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.210 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.211 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.212 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.212 DEBUG:boto:Method: GET 2022-09-03T05:37:10.213 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.214 DEBUG:boto:Data: 2022-09-03T05:37:10.215 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.217 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:10.218 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.219 DEBUG:boto:Params: {} 2022-09-03T05:37:10.219 DEBUG:boto:Token: None 2022-09-03T05:37:10.220 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.221 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jTrvQ6V2hlrYh09wzGsRiKuwVwI= 2022-09-03T05:37:10.222 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jTrvQ6V2hlrYh09wzGsRiKuwVwI=', 'Content-Length': '0'} 2022-09-03T05:37:10.226 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001ef6fa59d580cd6d-006312e806-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.227 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.228 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.228 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.229 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.229 DEBUG:boto:Method: HEAD 2022-09-03T05:37:10.230 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.230 DEBUG:boto:Data: 2022-09-03T05:37:10.231 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.231 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:10.232 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.232 DEBUG:boto:Params: {} 2022-09-03T05:37:10.233 DEBUG:boto:Token: None 2022-09-03T05:37:10.234 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.234 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8XpHhewzZF4BQea/i1PAlYcNufA= 2022-09-03T05:37:10.235 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8XpHhewzZF4BQea/i1PAlYcNufA=', 'Content-Length': '0'} 2022-09-03T05:37:10.237 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000058a8d185840542a0-006312e806-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.238 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.238 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.239 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.240 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.241 DEBUG:boto:Method: HEAD 2022-09-03T05:37:10.242 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.242 DEBUG:boto:Data: 2022-09-03T05:37:10.243 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.244 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:10.244 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.245 DEBUG:boto:Params: {} 2022-09-03T05:37:10.245 DEBUG:boto:Token: None 2022-09-03T05:37:10.246 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.247 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8XpHhewzZF4BQea/i1PAlYcNufA= 2022-09-03T05:37:10.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8XpHhewzZF4BQea/i1PAlYcNufA=', 'Content-Length': '0'} 2022-09-03T05:37:10.250 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002925066f56ff5695-006312e806-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.251 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:10.252 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.252 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.253 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.253 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.254 DEBUG:boto:Method: GET 2022-09-03T05:37:10.255 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.256 DEBUG:boto:Data: 2022-09-03T05:37:10.257 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.258 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:10.259 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.260 DEBUG:boto:Params: {} 2022-09-03T05:37:10.261 DEBUG:boto:Token: None 2022-09-03T05:37:10.262 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.263 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jTrvQ6V2hlrYh09wzGsRiKuwVwI= 2022-09-03T05:37:10.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jTrvQ6V2hlrYh09wzGsRiKuwVwI=', 'Content-Length': '0'} 2022-09-03T05:37:10.267 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000060ee2ae564cd40dc-006312e806-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.269 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.270 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.271 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.271 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.272 DEBUG:boto:Method: GET 2022-09-03T05:37:10.273 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.274 DEBUG:boto:Data: 2022-09-03T05:37:10.275 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.275 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:10.276 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.276 DEBUG:boto:Params: {} 2022-09-03T05:37:10.277 DEBUG:boto:Token: None 2022-09-03T05:37:10.278 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:10.279 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jTrvQ6V2hlrYh09wzGsRiKuwVwI= 2022-09-03T05:37:10.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jTrvQ6V2hlrYh09wzGsRiKuwVwI=', 'Content-Length': '0'} 2022-09-03T05:37:10.283 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fbffe97a4c770188-006312e806-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.330 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:10.331 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:10.332 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.332 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.333 DEBUG:boto:Method: GET 2022-09-03T05:37:10.333 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.334 DEBUG:boto:Data: 2022-09-03T05:37:10.334 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.335 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:10.336 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.336 DEBUG:boto:Params: {} 2022-09-03T05:37:10.337 DEBUG:boto:Token: None 2022-09-03T05:37:10.338 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/ 2022-09-03T05:37:10.339 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw= 2022-09-03T05:37:10.339 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw=', 'Content-Length': '0'} 2022-09-03T05:37:10.343 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006950d0eecc4824b5-006312e806-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.344 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:10.345 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:10.345 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:10.346 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.346 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.347 DEBUG:boto:Method: GET 2022-09-03T05:37:10.347 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.348 DEBUG:boto:Data: 2022-09-03T05:37:10.349 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.349 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:10.350 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.350 DEBUG:boto:Params: {} 2022-09-03T05:37:10.351 DEBUG:boto:Token: None 2022-09-03T05:37:10.352 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/ 2022-09-03T05:37:10.352 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw= 2022-09-03T05:37:10.353 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw=', 'Content-Length': '0'} 2022-09-03T05:37:10.357 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000432edb5565c93113-006312e806-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.359 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:10.360 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.361 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.362 DEBUG:boto:Method: HEAD 2022-09-03T05:37:10.363 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:10.364 DEBUG:boto:Data: 2022-09-03T05:37:10.365 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.365 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:10.366 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.367 DEBUG:boto:Params: {} 2022-09-03T05:37:10.368 DEBUG:boto:Token: None 2022-09-03T05:37:10.370 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj 2022-09-03T05:37:10.371 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YUhhcqHw+9fs4kEZhkrEPKjeQa4= 2022-09-03T05:37:10.373 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YUhhcqHw+9fs4kEZhkrEPKjeQa4=', 'Content-Length': '0'} 2022-09-03T05:37:10.378 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b0433211560e2f8c-006312e806-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.379 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.380 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.381 DEBUG:boto:Method: HEAD 2022-09-03T05:37:10.383 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:10.383 DEBUG:boto:Data: 2022-09-03T05:37:10.384 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.385 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:10.386 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.388 DEBUG:boto:Params: {} 2022-09-03T05:37:10.388 DEBUG:boto:Token: None 2022-09-03T05:37:10.389 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj 2022-09-03T05:37:10.390 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YUhhcqHw+9fs4kEZhkrEPKjeQa4= 2022-09-03T05:37:10.390 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YUhhcqHw+9fs4kEZhkrEPKjeQa4=', 'Content-Length': '0'} 2022-09-03T05:37:10.394 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a64ed837eb71dff7-006312e806-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.395 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:10.396 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.397 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.398 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.399 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.400 DEBUG:boto:Method: GET 2022-09-03T05:37:10.401 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.402 DEBUG:boto:Data: 2022-09-03T05:37:10.403 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.403 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:10.404 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.408 DEBUG:boto:Params: {} 2022-09-03T05:37:10.409 DEBUG:boto:Token: None 2022-09-03T05:37:10.410 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.412 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5teNsOAFFRZhu6gJI2H17GB++aI= 2022-09-03T05:37:10.413 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5teNsOAFFRZhu6gJI2H17GB++aI=', 'Content-Length': '0'} 2022-09-03T05:37:10.416 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000227fe3e7ea97fe19-006312e806-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.417 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.418 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.419 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.420 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.421 DEBUG:boto:Method: GET 2022-09-03T05:37:10.422 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.423 DEBUG:boto:Data: 2022-09-03T05:37:10.424 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.425 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:10.425 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.426 DEBUG:boto:Params: {} 2022-09-03T05:37:10.427 DEBUG:boto:Token: None 2022-09-03T05:37:10.427 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.428 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5teNsOAFFRZhu6gJI2H17GB++aI= 2022-09-03T05:37:10.429 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5teNsOAFFRZhu6gJI2H17GB++aI=', 'Content-Length': '0'} 2022-09-03T05:37:10.431 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ad338d5b723f6dcc-006312e806-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.433 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:10.433 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:10.434 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.435 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.435 DEBUG:boto:Method: GET 2022-09-03T05:37:10.436 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.436 DEBUG:boto:Data: 2022-09-03T05:37:10.437 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.438 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:10.438 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.439 DEBUG:boto:Params: {} 2022-09-03T05:37:10.439 DEBUG:boto:Token: None 2022-09-03T05:37:10.440 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/ 2022-09-03T05:37:10.440 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw= 2022-09-03T05:37:10.441 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw=', 'Content-Length': '0'} 2022-09-03T05:37:10.444 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ff738ac2f76bbd87-006312e806-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.446 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:10.448 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:10.449 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:10.450 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.451 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.452 DEBUG:boto:Method: GET 2022-09-03T05:37:10.453 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.454 DEBUG:boto:Data: 2022-09-03T05:37:10.456 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.457 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:10.458 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.459 DEBUG:boto:Params: {} 2022-09-03T05:37:10.460 DEBUG:boto:Token: None 2022-09-03T05:37:10.462 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/ 2022-09-03T05:37:10.463 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw= 2022-09-03T05:37:10.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw=', 'Content-Length': '0'} 2022-09-03T05:37:10.468 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000cde315f6b7e72f4-006312e806-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.518 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:10.519 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:10.520 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.521 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.521 DEBUG:boto:path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.522 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.522 DEBUG:boto:Method: GET 2022-09-03T05:37:10.523 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.524 DEBUG:boto:Data: 2022-09-03T05:37:10.525 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.526 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:10.526 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.527 DEBUG:boto:Params: {} 2022-09-03T05:37:10.527 DEBUG:boto:Token: None 2022-09-03T05:37:10.528 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.529 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O2+0TTZEIm4NE1jnEMrRobdvT/g= 2022-09-03T05:37:10.529 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O2+0TTZEIm4NE1jnEMrRobdvT/g=', 'Content-Length': '0'} 2022-09-03T05:37:10.533 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'EFeprdWgJo5gbXM16hy6HNBwhGTxK7e'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000070fe3a3041455faa-006312e806-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.535 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.537 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.539 DEBUG:boto:path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.539 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.540 DEBUG:boto:Method: GET 2022-09-03T05:37:10.541 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.542 DEBUG:boto:Data: 2022-09-03T05:37:10.542 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.543 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:10.543 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.544 DEBUG:boto:Params: {} 2022-09-03T05:37:10.544 DEBUG:boto:Token: None 2022-09-03T05:37:10.545 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.546 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O2+0TTZEIm4NE1jnEMrRobdvT/g= 2022-09-03T05:37:10.548 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O2+0TTZEIm4NE1jnEMrRobdvT/g=', 'Content-Length': '0'} 2022-09-03T05:37:10.551 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'EFeprdWgJo5gbXM16hy6HNBwhGTxK7e'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003400b05ae27243f1-006312e806-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.553 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.553 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.554 DEBUG:boto:path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.555 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.556 DEBUG:boto:Method: HEAD 2022-09-03T05:37:10.556 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.557 DEBUG:boto:Data: 2022-09-03T05:37:10.558 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.559 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:10.560 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.561 DEBUG:boto:Params: {} 2022-09-03T05:37:10.562 DEBUG:boto:Token: None 2022-09-03T05:37:10.563 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.564 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+z2YXiwHD2vVkkETKBgKXjkEIxc= 2022-09-03T05:37:10.565 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+z2YXiwHD2vVkkETKBgKXjkEIxc=', 'Content-Length': '0'} 2022-09-03T05:37:10.567 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'EFeprdWgJo5gbXM16hy6HNBwhGTxK7e'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000031aa520d033914ff-006312e806-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.569 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.570 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.572 DEBUG:boto:path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.572 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.573 DEBUG:boto:Method: HEAD 2022-09-03T05:37:10.574 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.574 DEBUG:boto:Data: 2022-09-03T05:37:10.575 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.575 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:10.576 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.576 DEBUG:boto:Params: {} 2022-09-03T05:37:10.577 DEBUG:boto:Token: None 2022-09-03T05:37:10.578 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.578 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+z2YXiwHD2vVkkETKBgKXjkEIxc= 2022-09-03T05:37:10.579 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+z2YXiwHD2vVkkETKBgKXjkEIxc=', 'Content-Length': '0'} 2022-09-03T05:37:10.581 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'EFeprdWgJo5gbXM16hy6HNBwhGTxK7e'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000674831b6cc50746f-006312e806-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.582 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:10.583 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.583 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.584 DEBUG:boto:path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.585 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.586 DEBUG:boto:Method: GET 2022-09-03T05:37:10.589 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.591 DEBUG:boto:Data: 2022-09-03T05:37:10.591 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.592 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:10.592 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.594 DEBUG:boto:Params: {} 2022-09-03T05:37:10.595 DEBUG:boto:Token: None 2022-09-03T05:37:10.595 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.596 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O2+0TTZEIm4NE1jnEMrRobdvT/g= 2022-09-03T05:37:10.597 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O2+0TTZEIm4NE1jnEMrRobdvT/g=', 'Content-Length': '0'} 2022-09-03T05:37:10.599 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'EFeprdWgJo5gbXM16hy6HNBwhGTxK7e'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000092c633c7d9ccb1e8-006312e806-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.600 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.601 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.601 DEBUG:boto:path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.602 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.602 DEBUG:boto:Method: GET 2022-09-03T05:37:10.603 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.603 DEBUG:boto:Data: 2022-09-03T05:37:10.604 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.605 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:10.605 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.606 DEBUG:boto:Params: {} 2022-09-03T05:37:10.606 DEBUG:boto:Token: None 2022-09-03T05:37:10.607 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:10.608 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O2+0TTZEIm4NE1jnEMrRobdvT/g= 2022-09-03T05:37:10.608 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O2+0TTZEIm4NE1jnEMrRobdvT/g=', 'Content-Length': '0'} 2022-09-03T05:37:10.610 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'EFeprdWgJo5gbXM16hy6HNBwhGTxK7e'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000067bb97ac7ad8d002-006312e806-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.611 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:10.612 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:10.612 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.613 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.613 DEBUG:boto:Method: GET 2022-09-03T05:37:10.614 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.615 DEBUG:boto:Data: 2022-09-03T05:37:10.616 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.616 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:10.617 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.617 DEBUG:boto:Params: {} 2022-09-03T05:37:10.618 DEBUG:boto:Token: None 2022-09-03T05:37:10.619 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/ 2022-09-03T05:37:10.619 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw= 2022-09-03T05:37:10.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw=', 'Content-Length': '0'} 2022-09-03T05:37:10.624 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003400221c197ee8ba-006312e806-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.626 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:10.627 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:10.628 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:10.629 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.630 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.634 DEBUG:boto:Method: GET 2022-09-03T05:37:10.635 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.635 DEBUG:boto:Data: 2022-09-03T05:37:10.636 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.636 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:10.637 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.638 DEBUG:boto:Params: {} 2022-09-03T05:37:10.638 DEBUG:boto:Token: None 2022-09-03T05:37:10.639 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/ 2022-09-03T05:37:10.640 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw= 2022-09-03T05:37:10.641 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw=', 'Content-Length': '0'} 2022-09-03T05:37:10.647 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000006d756922b8ed70-006312e806-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.694 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:10.695 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.696 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.697 DEBUG:boto:Method: HEAD 2022-09-03T05:37:10.697 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:10.698 DEBUG:boto:Data: 2022-09-03T05:37:10.698 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.699 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:10.699 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.700 DEBUG:boto:Params: {} 2022-09-03T05:37:10.700 DEBUG:boto:Token: None 2022-09-03T05:37:10.701 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj 2022-09-03T05:37:10.702 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YUhhcqHw+9fs4kEZhkrEPKjeQa4= 2022-09-03T05:37:10.702 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YUhhcqHw+9fs4kEZhkrEPKjeQa4=', 'Content-Length': '0'} 2022-09-03T05:37:10.706 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005850b034d17f6f81-006312e806-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.707 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.708 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.709 DEBUG:boto:Method: HEAD 2022-09-03T05:37:10.710 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:10.712 DEBUG:boto:Data: 2022-09-03T05:37:10.713 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.714 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:10.716 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.716 DEBUG:boto:Params: {} 2022-09-03T05:37:10.717 DEBUG:boto:Token: None 2022-09-03T05:37:10.718 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj 2022-09-03T05:37:10.718 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YUhhcqHw+9fs4kEZhkrEPKjeQa4= 2022-09-03T05:37:10.719 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YUhhcqHw+9fs4kEZhkrEPKjeQa4=', 'Content-Length': '0'} 2022-09-03T05:37:10.722 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000086d091b432db8fdb-006312e806-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.723 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:10.724 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.725 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.725 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.726 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.727 DEBUG:boto:Method: GET 2022-09-03T05:37:10.727 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.728 DEBUG:boto:Data: 2022-09-03T05:37:10.728 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.729 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:10.729 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.730 DEBUG:boto:Params: {} 2022-09-03T05:37:10.731 DEBUG:boto:Token: None 2022-09-03T05:37:10.731 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.733 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5teNsOAFFRZhu6gJI2H17GB++aI= 2022-09-03T05:37:10.733 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5teNsOAFFRZhu6gJI2H17GB++aI=', 'Content-Length': '0'} 2022-09-03T05:37:10.735 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000e820360b94fabec-006312e806-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.782 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.783 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.784 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.785 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.786 DEBUG:boto:Method: GET 2022-09-03T05:37:10.787 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.789 DEBUG:boto:Data: 2022-09-03T05:37:10.790 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.791 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:10.792 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.792 DEBUG:boto:Params: {} 2022-09-03T05:37:10.793 DEBUG:boto:Token: None 2022-09-03T05:37:10.795 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:10.796 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5teNsOAFFRZhu6gJI2H17GB++aI= 2022-09-03T05:37:10.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5teNsOAFFRZhu6gJI2H17GB++aI=', 'Content-Length': '0'} 2022-09-03T05:37:10.800 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004ba564c2005fba44-006312e806-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.802 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:10.803 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:10.804 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.805 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.805 DEBUG:boto:Method: GET 2022-09-03T05:37:10.806 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.807 DEBUG:boto:Data: 2022-09-03T05:37:10.807 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.808 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:10.809 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.810 DEBUG:boto:Params: {} 2022-09-03T05:37:10.811 DEBUG:boto:Token: None 2022-09-03T05:37:10.812 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/ 2022-09-03T05:37:10.813 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw= 2022-09-03T05:37:10.815 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw=', 'Content-Length': '0'} 2022-09-03T05:37:10.818 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c06e6d4c21dcd1d7-006312e806-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.819 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:10.820 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:10.821 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:10.822 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.823 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.824 DEBUG:boto:Method: GET 2022-09-03T05:37:10.826 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.827 DEBUG:boto:Data: 2022-09-03T05:37:10.828 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.829 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:10.830 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.831 DEBUG:boto:Params: {} 2022-09-03T05:37:10.832 DEBUG:boto:Token: None 2022-09-03T05:37:10.834 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/ 2022-09-03T05:37:10.835 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw= 2022-09-03T05:37:10.835 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw=', 'Content-Length': '0'} 2022-09-03T05:37:10.839 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000076dd2da30a4b5c48-006312e806-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.839 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:10.841 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:10.841 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.842 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.842 DEBUG:boto:path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.843 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.843 DEBUG:boto:Method: GET 2022-09-03T05:37:10.844 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.845 DEBUG:boto:Data: 2022-09-03T05:37:10.845 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.846 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:10.846 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.847 DEBUG:boto:Params: {} 2022-09-03T05:37:10.848 DEBUG:boto:Token: None 2022-09-03T05:37:10.848 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.849 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BGQmwIHV+h/hG3koDAbTevpSjhM= 2022-09-03T05:37:10.849 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BGQmwIHV+h/hG3koDAbTevpSjhM=', 'Content-Length': '0'} 2022-09-03T05:37:10.852 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005fc2a6dc49a87a0f-006312e806-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.853 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.853 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.854 DEBUG:boto:path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.854 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.855 DEBUG:boto:Method: GET 2022-09-03T05:37:10.855 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.856 DEBUG:boto:Data: 2022-09-03T05:37:10.856 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.857 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:10.858 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.859 DEBUG:boto:Params: {} 2022-09-03T05:37:10.860 DEBUG:boto:Token: None 2022-09-03T05:37:10.861 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.861 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BGQmwIHV+h/hG3koDAbTevpSjhM= 2022-09-03T05:37:10.862 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BGQmwIHV+h/hG3koDAbTevpSjhM=', 'Content-Length': '0'} 2022-09-03T05:37:10.864 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000881d58055c88579a-006312e806-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.865 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.866 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.866 DEBUG:boto:path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.867 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.868 DEBUG:boto:Method: HEAD 2022-09-03T05:37:10.868 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.869 DEBUG:boto:Data: 2022-09-03T05:37:10.869 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.870 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:10.870 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.871 DEBUG:boto:Params: {} 2022-09-03T05:37:10.871 DEBUG:boto:Token: None 2022-09-03T05:37:10.872 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.873 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0eoA7scNAKG2bBu/MYJ3u5wl+NQ= 2022-09-03T05:37:10.873 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0eoA7scNAKG2bBu/MYJ3u5wl+NQ=', 'Content-Length': '0'} 2022-09-03T05:37:10.876 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c8602946a0ee5fe6-006312e806-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.877 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.877 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.878 DEBUG:boto:path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.879 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.879 DEBUG:boto:Method: HEAD 2022-09-03T05:37:10.880 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.880 DEBUG:boto:Data: 2022-09-03T05:37:10.881 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.881 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:10.882 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.882 DEBUG:boto:Params: {} 2022-09-03T05:37:10.883 DEBUG:boto:Token: None 2022-09-03T05:37:10.883 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.884 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0eoA7scNAKG2bBu/MYJ3u5wl+NQ= 2022-09-03T05:37:10.885 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0eoA7scNAKG2bBu/MYJ3u5wl+NQ=', 'Content-Length': '0'} 2022-09-03T05:37:10.887 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ad5f5ac16903ff75-006312e806-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.888 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:10.888 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.889 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.889 DEBUG:boto:path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.890 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.891 DEBUG:boto:Method: GET 2022-09-03T05:37:10.891 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.892 DEBUG:boto:Data: 2022-09-03T05:37:10.892 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.893 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:10.893 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.894 DEBUG:boto:Params: {} 2022-09-03T05:37:10.894 DEBUG:boto:Token: None 2022-09-03T05:37:10.895 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.896 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BGQmwIHV+h/hG3koDAbTevpSjhM= 2022-09-03T05:37:10.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BGQmwIHV+h/hG3koDAbTevpSjhM=', 'Content-Length': '0'} 2022-09-03T05:37:10.898 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002197d1ba66357384-006312e806-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.900 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.900 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.901 DEBUG:boto:path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.901 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.902 DEBUG:boto:Method: GET 2022-09-03T05:37:10.903 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.903 DEBUG:boto:Data: 2022-09-03T05:37:10.904 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.904 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:10.905 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.905 DEBUG:boto:Params: {} 2022-09-03T05:37:10.906 DEBUG:boto:Token: None 2022-09-03T05:37:10.907 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:10.907 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BGQmwIHV+h/hG3koDAbTevpSjhM= 2022-09-03T05:37:10.908 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BGQmwIHV+h/hG3koDAbTevpSjhM=', 'Content-Length': '0'} 2022-09-03T05:37:10.910 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000004cc20efd7dc5a4c-006312e806-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.958 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:10.960 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:10.961 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.961 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.963 DEBUG:boto:Method: GET 2022-09-03T05:37:10.965 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.966 DEBUG:boto:Data: 2022-09-03T05:37:10.967 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.968 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:10.969 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.970 DEBUG:boto:Params: {} 2022-09-03T05:37:10.971 DEBUG:boto:Token: None 2022-09-03T05:37:10.972 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/ 2022-09-03T05:37:10.973 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw= 2022-09-03T05:37:10.973 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw=', 'Content-Length': '0'} 2022-09-03T05:37:10.977 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000014f9787538c149a1-006312e806-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.978 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:10.978 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:10.979 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:10.980 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.981 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.981 DEBUG:boto:Method: GET 2022-09-03T05:37:10.982 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:10.983 DEBUG:boto:Data: 2022-09-03T05:37:10.983 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.984 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:10.984 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.985 DEBUG:boto:Params: {} 2022-09-03T05:37:10.986 DEBUG:boto:Token: None 2022-09-03T05:37:10.986 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/ 2022-09-03T05:37:10.987 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw= 2022-09-03T05:37:10.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iZzV12ridsQ4q/g5JwlPBrHsMAw=', 'Content-Length': '0'} 2022-09-03T05:37:10.991 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000933988737f76ee71-006312e806-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:10.992 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:10.993 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:10.993 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:10.994 DEBUG:boto:Method: HEAD 2022-09-03T05:37:10.995 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:10.995 DEBUG:boto:Data: 2022-09-03T05:37:10.996 DEBUG:boto:Headers: {} 2022-09-03T05:37:10.996 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:10.997 DEBUG:boto:Port: 8000 2022-09-03T05:37:10.998 DEBUG:boto:Params: {} 2022-09-03T05:37:10.998 DEBUG:boto:Token: None 2022-09-03T05:37:10.999 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:10 GMT /dsgvcy-19/obj 2022-09-03T05:37:11.000 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YUhhcqHw+9fs4kEZhkrEPKjeQa4= 2022-09-03T05:37:11.001 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YUhhcqHw+9fs4kEZhkrEPKjeQa4=', 'Content-Length': '0'} 2022-09-03T05:37:11.005 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000db95d65025fe967f-006312e807-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.006 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.006 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.007 DEBUG:boto:Method: HEAD 2022-09-03T05:37:11.007 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:11.010 DEBUG:boto:Data: 2022-09-03T05:37:11.011 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.012 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:11.014 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.016 DEBUG:boto:Params: {} 2022-09-03T05:37:11.017 DEBUG:boto:Token: None 2022-09-03T05:37:11.018 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj 2022-09-03T05:37:11.019 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eInQB5yh+8azRKjgQlJf7C6Lepo= 2022-09-03T05:37:11.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eInQB5yh+8azRKjgQlJf7C6Lepo=', 'Content-Length': '0'} 2022-09-03T05:37:11.025 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000048691367b34ee349-006312e807-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.028 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:11.029 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.030 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.031 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.032 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.032 DEBUG:boto:Method: GET 2022-09-03T05:37:11.033 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.034 DEBUG:boto:Data: 2022-09-03T05:37:11.037 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.038 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:11.039 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.040 DEBUG:boto:Params: {} 2022-09-03T05:37:11.041 DEBUG:boto:Token: None 2022-09-03T05:37:11.042 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.044 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfsMmm+ZqMzko7vFZT9zq95WvMw= 2022-09-03T05:37:11.045 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfsMmm+ZqMzko7vFZT9zq95WvMw=', 'Content-Length': '0'} 2022-09-03T05:37:11.048 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000069bdc47aac672f9d-006312e807-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.050 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.051 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.052 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.054 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.055 DEBUG:boto:Method: GET 2022-09-03T05:37:11.055 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.056 DEBUG:boto:Data: 2022-09-03T05:37:11.057 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.057 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:11.058 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.058 DEBUG:boto:Params: {} 2022-09-03T05:37:11.059 DEBUG:boto:Token: None 2022-09-03T05:37:11.060 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.060 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfsMmm+ZqMzko7vFZT9zq95WvMw= 2022-09-03T05:37:11.061 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfsMmm+ZqMzko7vFZT9zq95WvMw=', 'Content-Length': '0'} 2022-09-03T05:37:11.063 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007b5945885664b9b9-006312e807-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.064 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:11.065 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:11.065 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.066 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.066 DEBUG:boto:Method: GET 2022-09-03T05:37:11.067 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.068 DEBUG:boto:Data: 2022-09-03T05:37:11.068 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.069 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:11.069 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.070 DEBUG:boto:Params: {} 2022-09-03T05:37:11.070 DEBUG:boto:Token: None 2022-09-03T05:37:11.071 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/ 2022-09-03T05:37:11.072 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU= 2022-09-03T05:37:11.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU=', 'Content-Length': '0'} 2022-09-03T05:37:11.077 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000010c75767db59ceec-006312e807-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.079 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:11.080 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:11.081 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:11.084 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.085 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.086 DEBUG:boto:Method: GET 2022-09-03T05:37:11.087 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.088 DEBUG:boto:Data: 2022-09-03T05:37:11.089 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.092 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:11.093 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.094 DEBUG:boto:Params: {} 2022-09-03T05:37:11.095 DEBUG:boto:Token: None 2022-09-03T05:37:11.097 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/ 2022-09-03T05:37:11.098 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU= 2022-09-03T05:37:11.100 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU=', 'Content-Length': '0'} 2022-09-03T05:37:11.104 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e5ab661118c9f1a1-006312e807-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.105 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:11.106 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:11.107 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.107 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.108 DEBUG:boto:path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.109 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.109 DEBUG:boto:Method: GET 2022-09-03T05:37:11.110 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.111 DEBUG:boto:Data: 2022-09-03T05:37:11.111 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.112 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:11.113 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.113 DEBUG:boto:Params: {} 2022-09-03T05:37:11.114 DEBUG:boto:Token: None 2022-09-03T05:37:11.114 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.115 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QfXlV1/pXPDj9QiY6pV93cOQHoM= 2022-09-03T05:37:11.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QfXlV1/pXPDj9QiY6pV93cOQHoM=', 'Content-Length': '0'} 2022-09-03T05:37:11.118 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'yL2uOIeC0l.C5YXdYoqQBvDIly3idxO'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000db0e5c5e2b6e8000-006312e807-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.119 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.120 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.121 DEBUG:boto:path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.121 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.122 DEBUG:boto:Method: GET 2022-09-03T05:37:11.122 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.123 DEBUG:boto:Data: 2022-09-03T05:37:11.124 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.125 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:11.125 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.126 DEBUG:boto:Params: {} 2022-09-03T05:37:11.126 DEBUG:boto:Token: None 2022-09-03T05:37:11.127 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.128 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QfXlV1/pXPDj9QiY6pV93cOQHoM= 2022-09-03T05:37:11.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QfXlV1/pXPDj9QiY6pV93cOQHoM=', 'Content-Length': '0'} 2022-09-03T05:37:11.131 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'yL2uOIeC0l.C5YXdYoqQBvDIly3idxO'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008dea273157c005eb-006312e807-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.174 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.175 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.176 DEBUG:boto:path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.177 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.178 DEBUG:boto:Method: HEAD 2022-09-03T05:37:11.179 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.180 DEBUG:boto:Data: 2022-09-03T05:37:11.181 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.182 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:11.183 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.185 DEBUG:boto:Params: {} 2022-09-03T05:37:11.186 DEBUG:boto:Token: None 2022-09-03T05:37:11.186 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.187 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RD72/La52/uTb4H2q4AGbJTtyUo= 2022-09-03T05:37:11.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RD72/La52/uTb4H2q4AGbJTtyUo=', 'Content-Length': '0'} 2022-09-03T05:37:11.190 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'yL2uOIeC0l.C5YXdYoqQBvDIly3idxO'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000449dc4a3913bcebd-006312e807-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.191 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.192 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.192 DEBUG:boto:path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.193 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.193 DEBUG:boto:Method: HEAD 2022-09-03T05:37:11.194 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.194 DEBUG:boto:Data: 2022-09-03T05:37:11.206 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.207 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:11.208 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.208 DEBUG:boto:Params: {} 2022-09-03T05:37:11.209 DEBUG:boto:Token: None 2022-09-03T05:37:11.210 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.211 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RD72/La52/uTb4H2q4AGbJTtyUo= 2022-09-03T05:37:11.211 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RD72/La52/uTb4H2q4AGbJTtyUo=', 'Content-Length': '0'} 2022-09-03T05:37:11.214 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'yL2uOIeC0l.C5YXdYoqQBvDIly3idxO'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008b42ff910a276d4d-006312e807-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.215 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:11.216 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.217 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.217 DEBUG:boto:path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.218 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.219 DEBUG:boto:Method: GET 2022-09-03T05:37:11.220 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.221 DEBUG:boto:Data: 2022-09-03T05:37:11.222 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.223 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:11.224 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.224 DEBUG:boto:Params: {} 2022-09-03T05:37:11.226 DEBUG:boto:Token: None 2022-09-03T05:37:11.227 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.228 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QfXlV1/pXPDj9QiY6pV93cOQHoM= 2022-09-03T05:37:11.229 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QfXlV1/pXPDj9QiY6pV93cOQHoM=', 'Content-Length': '0'} 2022-09-03T05:37:11.231 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'yL2uOIeC0l.C5YXdYoqQBvDIly3idxO'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009b5a0678f95834de-006312e807-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.232 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.232 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.233 DEBUG:boto:path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.233 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.234 DEBUG:boto:Method: GET 2022-09-03T05:37:11.235 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.236 DEBUG:boto:Data: 2022-09-03T05:37:11.237 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.238 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:11.239 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.240 DEBUG:boto:Params: {} 2022-09-03T05:37:11.241 DEBUG:boto:Token: None 2022-09-03T05:37:11.242 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:11.243 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QfXlV1/pXPDj9QiY6pV93cOQHoM= 2022-09-03T05:37:11.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QfXlV1/pXPDj9QiY6pV93cOQHoM=', 'Content-Length': '0'} 2022-09-03T05:37:11.247 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'yL2uOIeC0l.C5YXdYoqQBvDIly3idxO'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008498aeb3758aadf0-006312e807-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.249 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:11.249 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:11.250 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.250 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.251 DEBUG:boto:Method: GET 2022-09-03T05:37:11.251 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.252 DEBUG:boto:Data: 2022-09-03T05:37:11.252 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.253 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:11.254 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.254 DEBUG:boto:Params: {} 2022-09-03T05:37:11.255 DEBUG:boto:Token: None 2022-09-03T05:37:11.256 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/ 2022-09-03T05:37:11.256 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU= 2022-09-03T05:37:11.257 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU=', 'Content-Length': '0'} 2022-09-03T05:37:11.260 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002440cf0024a97e7b-006312e807-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.261 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:11.262 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:11.263 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:11.263 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.264 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.264 DEBUG:boto:Method: GET 2022-09-03T05:37:11.265 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.266 DEBUG:boto:Data: 2022-09-03T05:37:11.266 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.267 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:11.268 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.268 DEBUG:boto:Params: {} 2022-09-03T05:37:11.269 DEBUG:boto:Token: None 2022-09-03T05:37:11.270 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/ 2022-09-03T05:37:11.270 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU= 2022-09-03T05:37:11.271 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU=', 'Content-Length': '0'} 2022-09-03T05:37:11.274 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000079bc7a48a3107937-006312e807-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.275 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:11.276 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.276 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.277 DEBUG:boto:Method: HEAD 2022-09-03T05:37:11.277 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:11.278 DEBUG:boto:Data: 2022-09-03T05:37:11.278 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.279 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:11.280 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.280 DEBUG:boto:Params: {} 2022-09-03T05:37:11.281 DEBUG:boto:Token: None 2022-09-03T05:37:11.281 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj 2022-09-03T05:37:11.282 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eInQB5yh+8azRKjgQlJf7C6Lepo= 2022-09-03T05:37:11.283 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eInQB5yh+8azRKjgQlJf7C6Lepo=', 'Content-Length': '0'} 2022-09-03T05:37:11.286 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ac2bdcc336a85728-006312e807-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.287 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.288 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.288 DEBUG:boto:Method: HEAD 2022-09-03T05:37:11.289 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:11.289 DEBUG:boto:Data: 2022-09-03T05:37:11.290 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.290 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:11.291 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.291 DEBUG:boto:Params: {} 2022-09-03T05:37:11.292 DEBUG:boto:Token: None 2022-09-03T05:37:11.292 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj 2022-09-03T05:37:11.293 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eInQB5yh+8azRKjgQlJf7C6Lepo= 2022-09-03T05:37:11.294 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eInQB5yh+8azRKjgQlJf7C6Lepo=', 'Content-Length': '0'} 2022-09-03T05:37:11.298 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d53889ba10d568f6-006312e807-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.298 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:11.299 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.300 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.300 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.301 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.301 DEBUG:boto:Method: GET 2022-09-03T05:37:11.302 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.302 DEBUG:boto:Data: 2022-09-03T05:37:11.303 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.304 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:11.304 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.305 DEBUG:boto:Params: {} 2022-09-03T05:37:11.305 DEBUG:boto:Token: None 2022-09-03T05:37:11.306 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.307 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfsMmm+ZqMzko7vFZT9zq95WvMw= 2022-09-03T05:37:11.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfsMmm+ZqMzko7vFZT9zq95WvMw=', 'Content-Length': '0'} 2022-09-03T05:37:11.310 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000de153bfe8fda4900-006312e807-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.350 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.351 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.351 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.352 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.353 DEBUG:boto:Method: GET 2022-09-03T05:37:11.353 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.354 DEBUG:boto:Data: 2022-09-03T05:37:11.355 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.356 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:11.357 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.358 DEBUG:boto:Params: {} 2022-09-03T05:37:11.359 DEBUG:boto:Token: None 2022-09-03T05:37:11.360 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.361 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfsMmm+ZqMzko7vFZT9zq95WvMw= 2022-09-03T05:37:11.363 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfsMmm+ZqMzko7vFZT9zq95WvMw=', 'Content-Length': '0'} 2022-09-03T05:37:11.365 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bd0662206811cdf9-006312e807-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.367 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:11.368 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:11.369 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.370 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.371 DEBUG:boto:Method: GET 2022-09-03T05:37:11.372 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.372 DEBUG:boto:Data: 2022-09-03T05:37:11.373 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.373 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:11.374 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.374 DEBUG:boto:Params: {} 2022-09-03T05:37:11.375 DEBUG:boto:Token: None 2022-09-03T05:37:11.376 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/ 2022-09-03T05:37:11.376 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU= 2022-09-03T05:37:11.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU=', 'Content-Length': '0'} 2022-09-03T05:37:11.380 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008f30575ca0aff4a7-006312e807-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.381 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:11.382 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:11.383 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:11.383 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.384 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.384 DEBUG:boto:Method: GET 2022-09-03T05:37:11.385 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.386 DEBUG:boto:Data: 2022-09-03T05:37:11.387 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.388 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:11.389 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.389 DEBUG:boto:Params: {} 2022-09-03T05:37:11.390 DEBUG:boto:Token: None 2022-09-03T05:37:11.391 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/ 2022-09-03T05:37:11.392 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU= 2022-09-03T05:37:11.392 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU=', 'Content-Length': '0'} 2022-09-03T05:37:11.395 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000023b2e74ede14b01a-006312e807-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.396 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:11.397 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:11.398 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.399 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.399 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.400 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.400 DEBUG:boto:Method: GET 2022-09-03T05:37:11.401 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.401 DEBUG:boto:Data: 2022-09-03T05:37:11.402 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.403 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:11.403 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.404 DEBUG:boto:Params: {} 2022-09-03T05:37:11.404 DEBUG:boto:Token: None 2022-09-03T05:37:11.405 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.406 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:d5dXhTuao/xh+yJZ1Tt7L5UK0zk= 2022-09-03T05:37:11.406 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:d5dXhTuao/xh+yJZ1Tt7L5UK0zk=', 'Content-Length': '0'} 2022-09-03T05:37:11.408 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ec9feb403b9f8559-006312e807-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.410 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.410 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.411 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.411 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.412 DEBUG:boto:Method: GET 2022-09-03T05:37:11.412 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.413 DEBUG:boto:Data: 2022-09-03T05:37:11.414 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.414 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:11.415 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.416 DEBUG:boto:Params: {} 2022-09-03T05:37:11.416 DEBUG:boto:Token: None 2022-09-03T05:37:11.417 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.417 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:d5dXhTuao/xh+yJZ1Tt7L5UK0zk= 2022-09-03T05:37:11.418 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:d5dXhTuao/xh+yJZ1Tt7L5UK0zk=', 'Content-Length': '0'} 2022-09-03T05:37:11.421 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003943c036b8f272ef-006312e807-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.423 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.423 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.424 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.425 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.426 DEBUG:boto:Method: HEAD 2022-09-03T05:37:11.427 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.428 DEBUG:boto:Data: 2022-09-03T05:37:11.428 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.429 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:11.430 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.431 DEBUG:boto:Params: {} 2022-09-03T05:37:11.432 DEBUG:boto:Token: None 2022-09-03T05:37:11.433 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.435 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2VStbf/m7Z/Vi9Z68tGFg/vIPrM= 2022-09-03T05:37:11.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2VStbf/m7Z/Vi9Z68tGFg/vIPrM=', 'Content-Length': '0'} 2022-09-03T05:37:11.440 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002174d9bdf7338ee3-006312e807-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.440 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.441 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.442 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.442 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.443 DEBUG:boto:Method: HEAD 2022-09-03T05:37:11.443 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.444 DEBUG:boto:Data: 2022-09-03T05:37:11.444 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.445 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:11.445 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.446 DEBUG:boto:Params: {} 2022-09-03T05:37:11.446 DEBUG:boto:Token: None 2022-09-03T05:37:11.447 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.448 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2VStbf/m7Z/Vi9Z68tGFg/vIPrM= 2022-09-03T05:37:11.449 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2VStbf/m7Z/Vi9Z68tGFg/vIPrM=', 'Content-Length': '0'} 2022-09-03T05:37:11.451 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005c6bf24cf432d6a7-006312e807-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.451 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:11.452 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.453 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.454 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.454 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.455 DEBUG:boto:Method: GET 2022-09-03T05:37:11.456 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.457 DEBUG:boto:Data: 2022-09-03T05:37:11.457 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.458 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:11.459 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.459 DEBUG:boto:Params: {} 2022-09-03T05:37:11.460 DEBUG:boto:Token: None 2022-09-03T05:37:11.460 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.461 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:d5dXhTuao/xh+yJZ1Tt7L5UK0zk= 2022-09-03T05:37:11.462 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:d5dXhTuao/xh+yJZ1Tt7L5UK0zk=', 'Content-Length': '0'} 2022-09-03T05:37:11.466 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000606c58b450b76b2d-006312e807-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.467 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.468 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.470 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.471 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.472 DEBUG:boto:Method: GET 2022-09-03T05:37:11.473 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.474 DEBUG:boto:Data: 2022-09-03T05:37:11.476 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.477 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:11.478 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.479 DEBUG:boto:Params: {} 2022-09-03T05:37:11.480 DEBUG:boto:Token: None 2022-09-03T05:37:11.481 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:11.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:d5dXhTuao/xh+yJZ1Tt7L5UK0zk= 2022-09-03T05:37:11.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:d5dXhTuao/xh+yJZ1Tt7L5UK0zk=', 'Content-Length': '0'} 2022-09-03T05:37:11.487 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006a280447a9c3884c-006312e807-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.488 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:11.489 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:11.490 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.491 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.492 DEBUG:boto:Method: GET 2022-09-03T05:37:11.493 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.494 DEBUG:boto:Data: 2022-09-03T05:37:11.495 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.496 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:11.498 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.498 DEBUG:boto:Params: {} 2022-09-03T05:37:11.499 DEBUG:boto:Token: None 2022-09-03T05:37:11.500 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/ 2022-09-03T05:37:11.501 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU= 2022-09-03T05:37:11.502 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU=', 'Content-Length': '0'} 2022-09-03T05:37:11.515 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000df89305fd16c9187-006312e807-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.516 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:11.517 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:11.518 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:11.519 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.519 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.520 DEBUG:boto:Method: GET 2022-09-03T05:37:11.520 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.522 DEBUG:boto:Data: 2022-09-03T05:37:11.522 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.523 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:11.523 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.524 DEBUG:boto:Params: {} 2022-09-03T05:37:11.525 DEBUG:boto:Token: None 2022-09-03T05:37:11.525 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/ 2022-09-03T05:37:11.526 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU= 2022-09-03T05:37:11.527 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU=', 'Content-Length': '0'} 2022-09-03T05:37:11.530 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d16f3e54f710eae2-006312e807-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.532 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:11.533 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.535 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.535 DEBUG:boto:Method: HEAD 2022-09-03T05:37:11.536 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:11.536 DEBUG:boto:Data: 2022-09-03T05:37:11.537 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.537 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:11.538 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.538 DEBUG:boto:Params: {} 2022-09-03T05:37:11.539 DEBUG:boto:Token: None 2022-09-03T05:37:11.540 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj 2022-09-03T05:37:11.540 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eInQB5yh+8azRKjgQlJf7C6Lepo= 2022-09-03T05:37:11.541 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eInQB5yh+8azRKjgQlJf7C6Lepo=', 'Content-Length': '0'} 2022-09-03T05:37:11.545 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d7040363b4e893f2-006312e807-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.547 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.548 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.549 DEBUG:boto:Method: HEAD 2022-09-03T05:37:11.550 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:11.552 DEBUG:boto:Data: 2022-09-03T05:37:11.553 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.554 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:11.555 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.555 DEBUG:boto:Params: {} 2022-09-03T05:37:11.556 DEBUG:boto:Token: None 2022-09-03T05:37:11.557 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj 2022-09-03T05:37:11.557 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eInQB5yh+8azRKjgQlJf7C6Lepo= 2022-09-03T05:37:11.558 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eInQB5yh+8azRKjgQlJf7C6Lepo=', 'Content-Length': '0'} 2022-09-03T05:37:11.562 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e75771ef1e97fb84-006312e807-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.564 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:11.565 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.566 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.567 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.568 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.569 DEBUG:boto:Method: GET 2022-09-03T05:37:11.570 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.571 DEBUG:boto:Data: 2022-09-03T05:37:11.571 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.572 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:11.573 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.573 DEBUG:boto:Params: {} 2022-09-03T05:37:11.574 DEBUG:boto:Token: None 2022-09-03T05:37:11.574 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.576 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfsMmm+ZqMzko7vFZT9zq95WvMw= 2022-09-03T05:37:11.577 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfsMmm+ZqMzko7vFZT9zq95WvMw=', 'Content-Length': '0'} 2022-09-03T05:37:11.580 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003236f85324f6f7cb-006312e807-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.623 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.624 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.625 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.626 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.627 DEBUG:boto:Method: GET 2022-09-03T05:37:11.628 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.629 DEBUG:boto:Data: 2022-09-03T05:37:11.630 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.630 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:11.631 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.632 DEBUG:boto:Params: {} 2022-09-03T05:37:11.633 DEBUG:boto:Token: None 2022-09-03T05:37:11.634 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.635 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfsMmm+ZqMzko7vFZT9zq95WvMw= 2022-09-03T05:37:11.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfsMmm+ZqMzko7vFZT9zq95WvMw=', 'Content-Length': '0'} 2022-09-03T05:37:11.639 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009034c41c2f9d638d-006312e807-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.640 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:11.641 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:11.642 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.643 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.644 DEBUG:boto:Method: GET 2022-09-03T05:37:11.645 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.646 DEBUG:boto:Data: 2022-09-03T05:37:11.647 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.648 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:11.648 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.649 DEBUG:boto:Params: {} 2022-09-03T05:37:11.649 DEBUG:boto:Token: None 2022-09-03T05:37:11.650 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/ 2022-09-03T05:37:11.651 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU= 2022-09-03T05:37:11.651 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU=', 'Content-Length': '0'} 2022-09-03T05:37:11.654 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008c759a3747eb1483-006312e807-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.655 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:11.656 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:11.656 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:11.657 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.657 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.658 DEBUG:boto:Method: GET 2022-09-03T05:37:11.658 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.659 DEBUG:boto:Data: 2022-09-03T05:37:11.659 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.660 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:11.660 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.661 DEBUG:boto:Params: {} 2022-09-03T05:37:11.662 DEBUG:boto:Token: None 2022-09-03T05:37:11.662 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/ 2022-09-03T05:37:11.663 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU= 2022-09-03T05:37:11.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU=', 'Content-Length': '0'} 2022-09-03T05:37:11.666 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007a2520afce774868-006312e807-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.714 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:11.716 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:11.721 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.722 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.727 DEBUG:boto:path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.731 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.732 DEBUG:boto:Method: GET 2022-09-03T05:37:11.734 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.734 DEBUG:boto:Data: 2022-09-03T05:37:11.735 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.736 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:11.738 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.738 DEBUG:boto:Params: {} 2022-09-03T05:37:11.739 DEBUG:boto:Token: None 2022-09-03T05:37:11.740 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.740 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bqZBpV0r9Ll4KPIQ/NGOy2BojWE= 2022-09-03T05:37:11.741 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bqZBpV0r9Ll4KPIQ/NGOy2BojWE=', 'Content-Length': '0'} 2022-09-03T05:37:11.744 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000041f856a1261eb7cd-006312e807-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.745 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.746 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.747 DEBUG:boto:path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.748 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.749 DEBUG:boto:Method: GET 2022-09-03T05:37:11.751 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.751 DEBUG:boto:Data: 2022-09-03T05:37:11.752 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.754 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:11.755 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.755 DEBUG:boto:Params: {} 2022-09-03T05:37:11.756 DEBUG:boto:Token: None 2022-09-03T05:37:11.756 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.757 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bqZBpV0r9Ll4KPIQ/NGOy2BojWE= 2022-09-03T05:37:11.758 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bqZBpV0r9Ll4KPIQ/NGOy2BojWE=', 'Content-Length': '0'} 2022-09-03T05:37:11.764 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000542c29066dc38c50-006312e807-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.765 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.766 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.767 DEBUG:boto:path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.768 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.769 DEBUG:boto:Method: HEAD 2022-09-03T05:37:11.770 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.771 DEBUG:boto:Data: 2022-09-03T05:37:11.772 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.773 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:11.773 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.774 DEBUG:boto:Params: {} 2022-09-03T05:37:11.774 DEBUG:boto:Token: None 2022-09-03T05:37:11.775 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.775 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H2yu8eYxWeQv7198E0oYMmVguUE= 2022-09-03T05:37:11.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H2yu8eYxWeQv7198E0oYMmVguUE=', 'Content-Length': '0'} 2022-09-03T05:37:11.778 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000079b55046f67f4e9b-006312e807-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.779 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.779 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.780 DEBUG:boto:path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.780 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.781 DEBUG:boto:Method: HEAD 2022-09-03T05:37:11.781 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.782 DEBUG:boto:Data: 2022-09-03T05:37:11.782 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.783 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:11.784 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.784 DEBUG:boto:Params: {} 2022-09-03T05:37:11.785 DEBUG:boto:Token: None 2022-09-03T05:37:11.785 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.786 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H2yu8eYxWeQv7198E0oYMmVguUE= 2022-09-03T05:37:11.787 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H2yu8eYxWeQv7198E0oYMmVguUE=', 'Content-Length': '0'} 2022-09-03T05:37:11.791 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c0c46558b8a4eb4c-006312e807-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.791 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:11.792 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.793 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.793 DEBUG:boto:path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.794 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.794 DEBUG:boto:Method: GET 2022-09-03T05:37:11.795 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.795 DEBUG:boto:Data: 2022-09-03T05:37:11.796 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.796 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:11.797 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.797 DEBUG:boto:Params: {} 2022-09-03T05:37:11.798 DEBUG:boto:Token: None 2022-09-03T05:37:11.799 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.800 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bqZBpV0r9Ll4KPIQ/NGOy2BojWE= 2022-09-03T05:37:11.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bqZBpV0r9Ll4KPIQ/NGOy2BojWE=', 'Content-Length': '0'} 2022-09-03T05:37:11.803 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008f9764cefa5b6faa-006312e807-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.803 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.804 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.805 DEBUG:boto:path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.805 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.806 DEBUG:boto:Method: GET 2022-09-03T05:37:11.806 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.807 DEBUG:boto:Data: 2022-09-03T05:37:11.807 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.808 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:11.809 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.809 DEBUG:boto:Params: {} 2022-09-03T05:37:11.810 DEBUG:boto:Token: None 2022-09-03T05:37:11.811 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:11.811 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bqZBpV0r9Ll4KPIQ/NGOy2BojWE= 2022-09-03T05:37:11.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bqZBpV0r9Ll4KPIQ/NGOy2BojWE=', 'Content-Length': '0'} 2022-09-03T05:37:11.814 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c0da02e04e80cb0d-006312e807-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.815 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:11.816 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:11.816 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.817 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.817 DEBUG:boto:Method: GET 2022-09-03T05:37:11.818 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.819 DEBUG:boto:Data: 2022-09-03T05:37:11.819 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.820 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:11.820 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.821 DEBUG:boto:Params: {} 2022-09-03T05:37:11.821 DEBUG:boto:Token: None 2022-09-03T05:37:11.822 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/ 2022-09-03T05:37:11.823 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU= 2022-09-03T05:37:11.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU=', 'Content-Length': '0'} 2022-09-03T05:37:11.827 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a785734b36956db3-006312e807-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.874 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:11.876 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:11.877 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:11.878 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.879 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.880 DEBUG:boto:Method: GET 2022-09-03T05:37:11.881 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.882 DEBUG:boto:Data: 2022-09-03T05:37:11.882 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.883 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:11.884 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.885 DEBUG:boto:Params: {} 2022-09-03T05:37:11.885 DEBUG:boto:Token: None 2022-09-03T05:37:11.887 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/ 2022-09-03T05:37:11.888 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU= 2022-09-03T05:37:11.889 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU=', 'Content-Length': '0'} 2022-09-03T05:37:11.893 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000896f86124bda4fb7-006312e807-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.895 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:11.896 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.897 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.897 DEBUG:boto:Method: HEAD 2022-09-03T05:37:11.898 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:11.899 DEBUG:boto:Data: 2022-09-03T05:37:11.899 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.900 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:11.900 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.901 DEBUG:boto:Params: {} 2022-09-03T05:37:11.901 DEBUG:boto:Token: None 2022-09-03T05:37:11.902 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj 2022-09-03T05:37:11.902 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eInQB5yh+8azRKjgQlJf7C6Lepo= 2022-09-03T05:37:11.903 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eInQB5yh+8azRKjgQlJf7C6Lepo=', 'Content-Length': '0'} 2022-09-03T05:37:11.911 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000051332efda9d8a7a9-006312e807-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.913 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.914 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.917 DEBUG:boto:Method: HEAD 2022-09-03T05:37:11.919 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:11.920 DEBUG:boto:Data: 2022-09-03T05:37:11.921 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.922 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:11.923 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.924 DEBUG:boto:Params: {} 2022-09-03T05:37:11.926 DEBUG:boto:Token: None 2022-09-03T05:37:11.927 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj 2022-09-03T05:37:11.928 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eInQB5yh+8azRKjgQlJf7C6Lepo= 2022-09-03T05:37:11.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eInQB5yh+8azRKjgQlJf7C6Lepo=', 'Content-Length': '0'} 2022-09-03T05:37:11.933 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008a8955f552b19176-006312e807-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.934 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:11.935 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.936 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.937 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.938 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.940 DEBUG:boto:Method: GET 2022-09-03T05:37:11.943 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.944 DEBUG:boto:Data: 2022-09-03T05:37:11.945 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.946 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:11.947 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.947 DEBUG:boto:Params: {} 2022-09-03T05:37:11.949 DEBUG:boto:Token: None 2022-09-03T05:37:11.950 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.951 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfsMmm+ZqMzko7vFZT9zq95WvMw= 2022-09-03T05:37:11.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfsMmm+ZqMzko7vFZT9zq95WvMw=', 'Content-Length': '0'} 2022-09-03T05:37:11.957 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000574c6b9c572990e1-006312e807-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.959 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:11.960 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:11.960 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.961 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.961 DEBUG:boto:Method: GET 2022-09-03T05:37:11.962 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.962 DEBUG:boto:Data: 2022-09-03T05:37:11.963 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.964 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:11.964 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.965 DEBUG:boto:Params: {} 2022-09-03T05:37:11.965 DEBUG:boto:Token: None 2022-09-03T05:37:11.966 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:11.967 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tfsMmm+ZqMzko7vFZT9zq95WvMw= 2022-09-03T05:37:11.967 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tfsMmm+ZqMzko7vFZT9zq95WvMw=', 'Content-Length': '0'} 2022-09-03T05:37:11.972 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000023ff0befead5a9f2-006312e807-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:11.983 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:11.984 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:11.986 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.987 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.988 DEBUG:boto:Method: GET 2022-09-03T05:37:11.989 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:11.990 DEBUG:boto:Data: 2022-09-03T05:37:11.991 DEBUG:boto:Headers: {} 2022-09-03T05:37:11.992 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:11.993 DEBUG:boto:Port: 8000 2022-09-03T05:37:11.994 DEBUG:boto:Params: {} 2022-09-03T05:37:11.996 DEBUG:boto:Token: None 2022-09-03T05:37:11.997 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:11 GMT /dsgvcy-19/ 2022-09-03T05:37:11.997 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU= 2022-09-03T05:37:11.998 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zi8HPp2wv+uGLB9Icd/PtgOUXlU=', 'Content-Length': '0'} 2022-09-03T05:37:12.011 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bc98de8a3e14022d-006312e808-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.013 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:12.015 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:12.016 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:12.017 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.018 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.019 DEBUG:boto:Method: GET 2022-09-03T05:37:12.021 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.021 DEBUG:boto:Data: 2022-09-03T05:37:12.022 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.023 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:12.023 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.024 DEBUG:boto:Params: {} 2022-09-03T05:37:12.024 DEBUG:boto:Token: None 2022-09-03T05:37:12.025 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/ 2022-09-03T05:37:12.026 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk= 2022-09-03T05:37:12.026 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk=', 'Content-Length': '0'} 2022-09-03T05:37:12.030 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003744cbe6ae3ae1c3-006312e808-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.031 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:12.032 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:12.033 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.033 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.034 DEBUG:boto:path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.035 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.035 DEBUG:boto:Method: GET 2022-09-03T05:37:12.036 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.037 DEBUG:boto:Data: 2022-09-03T05:37:12.038 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.039 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:12.040 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.040 DEBUG:boto:Params: {} 2022-09-03T05:37:12.042 DEBUG:boto:Token: None 2022-09-03T05:37:12.043 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.044 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2iV1hV8lUChstQ8WPsR3dhrDYrg= 2022-09-03T05:37:12.046 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2iV1hV8lUChstQ8WPsR3dhrDYrg=', 'Content-Length': '0'} 2022-09-03T05:37:12.053 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'QfRyggjm1gZ7H.LeMCey0hHybmVWONG'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002bb3fe433d24787d-006312e808-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.055 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.056 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.057 DEBUG:boto:path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.058 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.059 DEBUG:boto:Method: GET 2022-09-03T05:37:12.059 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.060 DEBUG:boto:Data: 2022-09-03T05:37:12.061 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.061 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:12.064 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.064 DEBUG:boto:Params: {} 2022-09-03T05:37:12.065 DEBUG:boto:Token: None 2022-09-03T05:37:12.066 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.067 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2iV1hV8lUChstQ8WPsR3dhrDYrg= 2022-09-03T05:37:12.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2iV1hV8lUChstQ8WPsR3dhrDYrg=', 'Content-Length': '0'} 2022-09-03T05:37:12.078 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'QfRyggjm1gZ7H.LeMCey0hHybmVWONG'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000550da1348f6fd483-006312e808-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.115 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.116 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.117 DEBUG:boto:path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.118 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.119 DEBUG:boto:Method: HEAD 2022-09-03T05:37:12.120 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.121 DEBUG:boto:Data: 2022-09-03T05:37:12.125 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.126 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:12.127 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.127 DEBUG:boto:Params: {} 2022-09-03T05:37:12.128 DEBUG:boto:Token: None 2022-09-03T05:37:12.128 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.129 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5V9y7mEL3xMdh+TbRHsBGtgCSmk= 2022-09-03T05:37:12.130 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5V9y7mEL3xMdh+TbRHsBGtgCSmk=', 'Content-Length': '0'} 2022-09-03T05:37:12.134 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'QfRyggjm1gZ7H.LeMCey0hHybmVWONG'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000be29091a77880167-006312e808-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.136 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.136 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.137 DEBUG:boto:path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.138 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.139 DEBUG:boto:Method: HEAD 2022-09-03T05:37:12.139 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.140 DEBUG:boto:Data: 2022-09-03T05:37:12.140 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.141 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:12.141 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.142 DEBUG:boto:Params: {} 2022-09-03T05:37:12.143 DEBUG:boto:Token: None 2022-09-03T05:37:12.145 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.145 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5V9y7mEL3xMdh+TbRHsBGtgCSmk= 2022-09-03T05:37:12.147 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5V9y7mEL3xMdh+TbRHsBGtgCSmk=', 'Content-Length': '0'} 2022-09-03T05:37:12.149 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'QfRyggjm1gZ7H.LeMCey0hHybmVWONG'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fd0ea8f9a6b89bf9-006312e808-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.151 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:12.152 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.153 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.153 DEBUG:boto:path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.155 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.156 DEBUG:boto:Method: GET 2022-09-03T05:37:12.157 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.159 DEBUG:boto:Data: 2022-09-03T05:37:12.161 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.162 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:12.163 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.164 DEBUG:boto:Params: {} 2022-09-03T05:37:12.165 DEBUG:boto:Token: None 2022-09-03T05:37:12.168 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.169 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2iV1hV8lUChstQ8WPsR3dhrDYrg= 2022-09-03T05:37:12.169 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2iV1hV8lUChstQ8WPsR3dhrDYrg=', 'Content-Length': '0'} 2022-09-03T05:37:12.172 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'QfRyggjm1gZ7H.LeMCey0hHybmVWONG'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000062000a8ad969984c-006312e808-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.173 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.173 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.174 DEBUG:boto:path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.174 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.175 DEBUG:boto:Method: GET 2022-09-03T05:37:12.176 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.177 DEBUG:boto:Data: 2022-09-03T05:37:12.178 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.179 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:12.180 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.181 DEBUG:boto:Params: {} 2022-09-03T05:37:12.182 DEBUG:boto:Token: None 2022-09-03T05:37:12.183 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:12.185 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2iV1hV8lUChstQ8WPsR3dhrDYrg= 2022-09-03T05:37:12.186 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2iV1hV8lUChstQ8WPsR3dhrDYrg=', 'Content-Length': '0'} 2022-09-03T05:37:12.194 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'QfRyggjm1gZ7H.LeMCey0hHybmVWONG'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000055915bd51e03e8ef-006312e808-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.196 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:12.198 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:12.199 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.200 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.201 DEBUG:boto:Method: GET 2022-09-03T05:37:12.202 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.203 DEBUG:boto:Data: 2022-09-03T05:37:12.204 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.205 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:12.206 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.207 DEBUG:boto:Params: {} 2022-09-03T05:37:12.208 DEBUG:boto:Token: None 2022-09-03T05:37:12.209 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/ 2022-09-03T05:37:12.210 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk= 2022-09-03T05:37:12.211 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk=', 'Content-Length': '0'} 2022-09-03T05:37:12.214 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000080f4ec9d9be34dd8-006312e808-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.216 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:12.218 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:12.218 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:12.219 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.219 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.220 DEBUG:boto:Method: GET 2022-09-03T05:37:12.221 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.221 DEBUG:boto:Data: 2022-09-03T05:37:12.222 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.222 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:12.223 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.223 DEBUG:boto:Params: {} 2022-09-03T05:37:12.224 DEBUG:boto:Token: None 2022-09-03T05:37:12.225 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/ 2022-09-03T05:37:12.225 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk= 2022-09-03T05:37:12.226 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk=', 'Content-Length': '0'} 2022-09-03T05:37:12.229 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000039dc8441916c7e14-006312e808-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.230 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:12.231 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.231 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.232 DEBUG:boto:Method: HEAD 2022-09-03T05:37:12.232 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:12.233 DEBUG:boto:Data: 2022-09-03T05:37:12.233 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.234 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:12.234 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.235 DEBUG:boto:Params: {} 2022-09-03T05:37:12.236 DEBUG:boto:Token: None 2022-09-03T05:37:12.237 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj 2022-09-03T05:37:12.237 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1y36wu2Pkct48+yW/9iwLXmeezc= 2022-09-03T05:37:12.238 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1y36wu2Pkct48+yW/9iwLXmeezc=', 'Content-Length': '0'} 2022-09-03T05:37:12.245 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f19ee668a7d9ddae-006312e808-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.246 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.247 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.248 DEBUG:boto:Method: HEAD 2022-09-03T05:37:12.249 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:12.249 DEBUG:boto:Data: 2022-09-03T05:37:12.250 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.251 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:12.251 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.252 DEBUG:boto:Params: {} 2022-09-03T05:37:12.252 DEBUG:boto:Token: None 2022-09-03T05:37:12.253 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj 2022-09-03T05:37:12.253 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1y36wu2Pkct48+yW/9iwLXmeezc= 2022-09-03T05:37:12.254 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1y36wu2Pkct48+yW/9iwLXmeezc=', 'Content-Length': '0'} 2022-09-03T05:37:12.258 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001a4826ba04acbc5f-006312e808-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.258 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:12.259 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.260 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.260 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.261 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.261 DEBUG:boto:Method: GET 2022-09-03T05:37:12.262 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.262 DEBUG:boto:Data: 2022-09-03T05:37:12.263 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.263 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:12.264 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.264 DEBUG:boto:Params: {} 2022-09-03T05:37:12.265 DEBUG:boto:Token: None 2022-09-03T05:37:12.266 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.267 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BKz/vWcFkHXz8VwBJxwcKxZb+go= 2022-09-03T05:37:12.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BKz/vWcFkHXz8VwBJxwcKxZb+go=', 'Content-Length': '0'} 2022-09-03T05:37:12.270 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cb697c9d4745bd84-006312e808-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.318 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.319 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.319 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.320 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.321 DEBUG:boto:Method: GET 2022-09-03T05:37:12.321 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.322 DEBUG:boto:Data: 2022-09-03T05:37:12.323 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.323 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:12.324 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.325 DEBUG:boto:Params: {} 2022-09-03T05:37:12.326 DEBUG:boto:Token: None 2022-09-03T05:37:12.326 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.327 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BKz/vWcFkHXz8VwBJxwcKxZb+go= 2022-09-03T05:37:12.327 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BKz/vWcFkHXz8VwBJxwcKxZb+go=', 'Content-Length': '0'} 2022-09-03T05:37:12.330 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000746ec1b65529b90-006312e808-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.330 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:12.331 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:12.332 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.332 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.333 DEBUG:boto:Method: GET 2022-09-03T05:37:12.333 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.334 DEBUG:boto:Data: 2022-09-03T05:37:12.335 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.335 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:12.336 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.336 DEBUG:boto:Params: {} 2022-09-03T05:37:12.337 DEBUG:boto:Token: None 2022-09-03T05:37:12.337 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/ 2022-09-03T05:37:12.338 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk= 2022-09-03T05:37:12.339 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk=', 'Content-Length': '0'} 2022-09-03T05:37:12.342 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bb8fa8ef3afc314f-006312e808-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.344 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:12.345 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:12.346 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:12.346 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.347 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.348 DEBUG:boto:Method: GET 2022-09-03T05:37:12.349 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.350 DEBUG:boto:Data: 2022-09-03T05:37:12.351 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.352 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:12.353 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.354 DEBUG:boto:Params: {} 2022-09-03T05:37:12.355 DEBUG:boto:Token: None 2022-09-03T05:37:12.356 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/ 2022-09-03T05:37:12.357 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk= 2022-09-03T05:37:12.358 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk=', 'Content-Length': '0'} 2022-09-03T05:37:12.362 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c0758755c63d351d-006312e808-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.364 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:12.365 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:12.366 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.367 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.368 DEBUG:boto:path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.371 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.372 DEBUG:boto:Method: GET 2022-09-03T05:37:12.373 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.374 DEBUG:boto:Data: 2022-09-03T05:37:12.375 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.376 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:12.378 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.379 DEBUG:boto:Params: {} 2022-09-03T05:37:12.379 DEBUG:boto:Token: None 2022-09-03T05:37:12.380 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.381 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bKdFNoK6F6homzfZfcU3uQphAVs= 2022-09-03T05:37:12.381 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bKdFNoK6F6homzfZfcU3uQphAVs=', 'Content-Length': '0'} 2022-09-03T05:37:12.384 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '94bPO72IhlWQEGwklU15Pvpr-qRv.lm'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b89d94160056daea-006312e808-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.385 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.386 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.386 DEBUG:boto:path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.387 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.387 DEBUG:boto:Method: GET 2022-09-03T05:37:12.388 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.388 DEBUG:boto:Data: 2022-09-03T05:37:12.389 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.390 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:12.390 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.392 DEBUG:boto:Params: {} 2022-09-03T05:37:12.392 DEBUG:boto:Token: None 2022-09-03T05:37:12.393 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.395 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bKdFNoK6F6homzfZfcU3uQphAVs= 2022-09-03T05:37:12.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bKdFNoK6F6homzfZfcU3uQphAVs=', 'Content-Length': '0'} 2022-09-03T05:37:12.398 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '94bPO72IhlWQEGwklU15Pvpr-qRv.lm'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008243bc2950dfb002-006312e808-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.400 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.400 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.401 DEBUG:boto:path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.401 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.402 DEBUG:boto:Method: HEAD 2022-09-03T05:37:12.403 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.404 DEBUG:boto:Data: 2022-09-03T05:37:12.405 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.406 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:12.406 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.407 DEBUG:boto:Params: {} 2022-09-03T05:37:12.407 DEBUG:boto:Token: None 2022-09-03T05:37:12.408 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.408 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OYT9VBdBKXniU4tqBaGczjto6hE= 2022-09-03T05:37:12.409 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OYT9VBdBKXniU4tqBaGczjto6hE=', 'Content-Length': '0'} 2022-09-03T05:37:12.411 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '94bPO72IhlWQEGwklU15Pvpr-qRv.lm'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009ddf6fdaf8d5d3c8-006312e808-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.412 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.413 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.413 DEBUG:boto:path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.414 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.415 DEBUG:boto:Method: HEAD 2022-09-03T05:37:12.415 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.416 DEBUG:boto:Data: 2022-09-03T05:37:12.416 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.417 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:12.417 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.418 DEBUG:boto:Params: {} 2022-09-03T05:37:12.418 DEBUG:boto:Token: None 2022-09-03T05:37:12.419 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.419 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OYT9VBdBKXniU4tqBaGczjto6hE= 2022-09-03T05:37:12.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OYT9VBdBKXniU4tqBaGczjto6hE=', 'Content-Length': '0'} 2022-09-03T05:37:12.422 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '94bPO72IhlWQEGwklU15Pvpr-qRv.lm'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000005dfce9ab96b0fd5-006312e808-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.423 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:12.424 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.424 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.425 DEBUG:boto:path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.425 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.426 DEBUG:boto:Method: GET 2022-09-03T05:37:12.426 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.427 DEBUG:boto:Data: 2022-09-03T05:37:12.428 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.428 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:12.429 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.429 DEBUG:boto:Params: {} 2022-09-03T05:37:12.430 DEBUG:boto:Token: None 2022-09-03T05:37:12.431 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.432 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bKdFNoK6F6homzfZfcU3uQphAVs= 2022-09-03T05:37:12.434 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bKdFNoK6F6homzfZfcU3uQphAVs=', 'Content-Length': '0'} 2022-09-03T05:37:12.436 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '94bPO72IhlWQEGwklU15Pvpr-qRv.lm'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007aa3384b78feae5c-006312e808-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.437 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.437 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.438 DEBUG:boto:path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.439 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.439 DEBUG:boto:Method: GET 2022-09-03T05:37:12.440 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.440 DEBUG:boto:Data: 2022-09-03T05:37:12.441 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.441 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:12.442 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.442 DEBUG:boto:Params: {} 2022-09-03T05:37:12.443 DEBUG:boto:Token: None 2022-09-03T05:37:12.445 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:12.447 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bKdFNoK6F6homzfZfcU3uQphAVs= 2022-09-03T05:37:12.448 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bKdFNoK6F6homzfZfcU3uQphAVs=', 'Content-Length': '0'} 2022-09-03T05:37:12.451 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '94bPO72IhlWQEGwklU15Pvpr-qRv.lm'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007839ff7fee841e90-006312e808-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.497 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:12.498 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:12.498 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.499 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.501 DEBUG:boto:Method: GET 2022-09-03T05:37:12.502 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.502 DEBUG:boto:Data: 2022-09-03T05:37:12.503 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.503 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:12.504 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.504 DEBUG:boto:Params: {} 2022-09-03T05:37:12.505 DEBUG:boto:Token: None 2022-09-03T05:37:12.505 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/ 2022-09-03T05:37:12.506 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk= 2022-09-03T05:37:12.507 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk=', 'Content-Length': '0'} 2022-09-03T05:37:12.515 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005abb00ba6b6f0912-006312e808-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.516 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:12.518 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:12.518 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:12.519 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.520 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.520 DEBUG:boto:Method: GET 2022-09-03T05:37:12.521 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.521 DEBUG:boto:Data: 2022-09-03T05:37:12.522 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.524 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:12.524 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.525 DEBUG:boto:Params: {} 2022-09-03T05:37:12.526 DEBUG:boto:Token: None 2022-09-03T05:37:12.529 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/ 2022-09-03T05:37:12.530 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk= 2022-09-03T05:37:12.530 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk=', 'Content-Length': '0'} 2022-09-03T05:37:12.534 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000556ee598f18d0774-006312e808-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.536 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:12.537 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.538 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.540 DEBUG:boto:Method: HEAD 2022-09-03T05:37:12.541 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:12.541 DEBUG:boto:Data: 2022-09-03T05:37:12.542 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.542 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:12.543 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.543 DEBUG:boto:Params: {} 2022-09-03T05:37:12.544 DEBUG:boto:Token: None 2022-09-03T05:37:12.544 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj 2022-09-03T05:37:12.545 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1y36wu2Pkct48+yW/9iwLXmeezc= 2022-09-03T05:37:12.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1y36wu2Pkct48+yW/9iwLXmeezc=', 'Content-Length': '0'} 2022-09-03T05:37:12.550 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000010b55dfd2385f4eb-006312e808-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.551 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.553 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.554 DEBUG:boto:Method: HEAD 2022-09-03T05:37:12.555 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:12.555 DEBUG:boto:Data: 2022-09-03T05:37:12.556 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.557 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:12.558 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.559 DEBUG:boto:Params: {} 2022-09-03T05:37:12.561 DEBUG:boto:Token: None 2022-09-03T05:37:12.562 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj 2022-09-03T05:37:12.563 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1y36wu2Pkct48+yW/9iwLXmeezc= 2022-09-03T05:37:12.564 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1y36wu2Pkct48+yW/9iwLXmeezc=', 'Content-Length': '0'} 2022-09-03T05:37:12.568 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000049eef09ea433fbbf-006312e808-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.569 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:12.570 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.571 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.572 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.573 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.575 DEBUG:boto:Method: GET 2022-09-03T05:37:12.576 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.576 DEBUG:boto:Data: 2022-09-03T05:37:12.577 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.578 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:12.579 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.580 DEBUG:boto:Params: {} 2022-09-03T05:37:12.581 DEBUG:boto:Token: None 2022-09-03T05:37:12.583 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.586 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BKz/vWcFkHXz8VwBJxwcKxZb+go= 2022-09-03T05:37:12.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BKz/vWcFkHXz8VwBJxwcKxZb+go=', 'Content-Length': '0'} 2022-09-03T05:37:12.590 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008a67a3a70ab61cc8-006312e808-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.591 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.592 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.592 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.593 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.593 DEBUG:boto:Method: GET 2022-09-03T05:37:12.594 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.595 DEBUG:boto:Data: 2022-09-03T05:37:12.595 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.596 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:12.596 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.597 DEBUG:boto:Params: {} 2022-09-03T05:37:12.597 DEBUG:boto:Token: None 2022-09-03T05:37:12.598 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.599 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BKz/vWcFkHXz8VwBJxwcKxZb+go= 2022-09-03T05:37:12.599 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BKz/vWcFkHXz8VwBJxwcKxZb+go=', 'Content-Length': '0'} 2022-09-03T05:37:12.602 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000001f4c218d012e83c-006312e808-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.603 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:12.604 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:12.604 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.605 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.605 DEBUG:boto:Method: GET 2022-09-03T05:37:12.606 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.606 DEBUG:boto:Data: 2022-09-03T05:37:12.607 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.607 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:12.608 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.609 DEBUG:boto:Params: {} 2022-09-03T05:37:12.609 DEBUG:boto:Token: None 2022-09-03T05:37:12.610 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/ 2022-09-03T05:37:12.610 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk= 2022-09-03T05:37:12.611 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk=', 'Content-Length': '0'} 2022-09-03T05:37:12.614 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000815eaa9ed6aebd8e-006312e808-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.662 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:12.663 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:12.664 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:12.664 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.665 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.665 DEBUG:boto:Method: GET 2022-09-03T05:37:12.666 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.667 DEBUG:boto:Data: 2022-09-03T05:37:12.667 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.668 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:12.668 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.670 DEBUG:boto:Params: {} 2022-09-03T05:37:12.670 DEBUG:boto:Token: None 2022-09-03T05:37:12.671 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/ 2022-09-03T05:37:12.672 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk= 2022-09-03T05:37:12.673 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk=', 'Content-Length': '0'} 2022-09-03T05:37:12.677 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000773b04f92a9cdf2a-006312e808-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.678 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:12.679 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:12.680 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.681 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.681 DEBUG:boto:path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.682 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.682 DEBUG:boto:Method: GET 2022-09-03T05:37:12.683 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.684 DEBUG:boto:Data: 2022-09-03T05:37:12.685 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.685 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:12.686 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.686 DEBUG:boto:Params: {} 2022-09-03T05:37:12.687 DEBUG:boto:Token: None 2022-09-03T05:37:12.687 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.688 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IbjaaOXBLA5IS3GJrRB3UQcfoN4= 2022-09-03T05:37:12.689 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IbjaaOXBLA5IS3GJrRB3UQcfoN4=', 'Content-Length': '0'} 2022-09-03T05:37:12.692 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'ORmfT2fVQ5EFIFI89EGuLigbPyfeObT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b1f0a95a963d075e-006312e808-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.693 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.693 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.694 DEBUG:boto:path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.694 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.695 DEBUG:boto:Method: GET 2022-09-03T05:37:12.696 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.696 DEBUG:boto:Data: 2022-09-03T05:37:12.697 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.697 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:12.698 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.698 DEBUG:boto:Params: {} 2022-09-03T05:37:12.699 DEBUG:boto:Token: None 2022-09-03T05:37:12.699 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.700 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IbjaaOXBLA5IS3GJrRB3UQcfoN4= 2022-09-03T05:37:12.701 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IbjaaOXBLA5IS3GJrRB3UQcfoN4=', 'Content-Length': '0'} 2022-09-03T05:37:12.703 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'ORmfT2fVQ5EFIFI89EGuLigbPyfeObT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005c873325ee42a07d-006312e808-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.750 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.751 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.752 DEBUG:boto:path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.753 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.754 DEBUG:boto:Method: HEAD 2022-09-03T05:37:12.755 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.755 DEBUG:boto:Data: 2022-09-03T05:37:12.756 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.757 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:12.757 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.758 DEBUG:boto:Params: {} 2022-09-03T05:37:12.758 DEBUG:boto:Token: None 2022-09-03T05:37:12.759 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.759 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H8xD99N8xfyIn6/4C11VL5NPkwI= 2022-09-03T05:37:12.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H8xD99N8xfyIn6/4C11VL5NPkwI=', 'Content-Length': '0'} 2022-09-03T05:37:12.762 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'ORmfT2fVQ5EFIFI89EGuLigbPyfeObT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000074e9545d319ffbb1-006312e808-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.763 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.764 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.764 DEBUG:boto:path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.765 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.765 DEBUG:boto:Method: HEAD 2022-09-03T05:37:12.766 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.766 DEBUG:boto:Data: 2022-09-03T05:37:12.767 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.768 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:12.768 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.769 DEBUG:boto:Params: {} 2022-09-03T05:37:12.769 DEBUG:boto:Token: None 2022-09-03T05:37:12.771 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.773 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H8xD99N8xfyIn6/4C11VL5NPkwI= 2022-09-03T05:37:12.774 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H8xD99N8xfyIn6/4C11VL5NPkwI=', 'Content-Length': '0'} 2022-09-03T05:37:12.777 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'ORmfT2fVQ5EFIFI89EGuLigbPyfeObT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cfd776e756243b19-006312e808-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.779 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:12.780 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.781 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.782 DEBUG:boto:path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.783 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.784 DEBUG:boto:Method: GET 2022-09-03T05:37:12.785 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.786 DEBUG:boto:Data: 2022-09-03T05:37:12.787 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.787 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:12.788 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.788 DEBUG:boto:Params: {} 2022-09-03T05:37:12.789 DEBUG:boto:Token: None 2022-09-03T05:37:12.790 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.791 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IbjaaOXBLA5IS3GJrRB3UQcfoN4= 2022-09-03T05:37:12.791 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IbjaaOXBLA5IS3GJrRB3UQcfoN4=', 'Content-Length': '0'} 2022-09-03T05:37:12.794 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'ORmfT2fVQ5EFIFI89EGuLigbPyfeObT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e7743070dc635190-006312e808-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.796 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.797 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.798 DEBUG:boto:path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.799 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.800 DEBUG:boto:Method: GET 2022-09-03T05:37:12.801 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.801 DEBUG:boto:Data: 2022-09-03T05:37:12.802 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.803 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:12.804 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.805 DEBUG:boto:Params: {} 2022-09-03T05:37:12.806 DEBUG:boto:Token: None 2022-09-03T05:37:12.807 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:12.808 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IbjaaOXBLA5IS3GJrRB3UQcfoN4= 2022-09-03T05:37:12.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IbjaaOXBLA5IS3GJrRB3UQcfoN4=', 'Content-Length': '0'} 2022-09-03T05:37:12.813 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'ORmfT2fVQ5EFIFI89EGuLigbPyfeObT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000027d6c0856898a340-006312e808-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.814 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:12.815 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:12.815 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.816 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.816 DEBUG:boto:Method: GET 2022-09-03T05:37:12.817 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.818 DEBUG:boto:Data: 2022-09-03T05:37:12.818 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.819 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:12.819 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.820 DEBUG:boto:Params: {} 2022-09-03T05:37:12.821 DEBUG:boto:Token: None 2022-09-03T05:37:12.823 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/ 2022-09-03T05:37:12.824 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk= 2022-09-03T05:37:12.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk=', 'Content-Length': '0'} 2022-09-03T05:37:12.828 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c9daf59764fae3d5-006312e808-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.873 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:12.875 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:12.876 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:12.876 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.877 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.877 DEBUG:boto:Method: GET 2022-09-03T05:37:12.878 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.879 DEBUG:boto:Data: 2022-09-03T05:37:12.879 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.880 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:12.880 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.881 DEBUG:boto:Params: {} 2022-09-03T05:37:12.882 DEBUG:boto:Token: None 2022-09-03T05:37:12.882 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/ 2022-09-03T05:37:12.883 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk= 2022-09-03T05:37:12.883 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk=', 'Content-Length': '0'} 2022-09-03T05:37:12.889 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bcf9bcf06be27e0c-006312e808-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.890 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:12.892 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.893 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.894 DEBUG:boto:Method: HEAD 2022-09-03T05:37:12.895 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:12.897 DEBUG:boto:Data: 2022-09-03T05:37:12.898 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.898 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:12.899 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.899 DEBUG:boto:Params: {} 2022-09-03T05:37:12.900 DEBUG:boto:Token: None 2022-09-03T05:37:12.901 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj 2022-09-03T05:37:12.901 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1y36wu2Pkct48+yW/9iwLXmeezc= 2022-09-03T05:37:12.902 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1y36wu2Pkct48+yW/9iwLXmeezc=', 'Content-Length': '0'} 2022-09-03T05:37:12.906 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005c9788a2ceba4db0-006312e808-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.908 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.909 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.910 DEBUG:boto:Method: HEAD 2022-09-03T05:37:12.911 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:12.912 DEBUG:boto:Data: 2022-09-03T05:37:12.913 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.914 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:12.915 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.916 DEBUG:boto:Params: {} 2022-09-03T05:37:12.916 DEBUG:boto:Token: None 2022-09-03T05:37:12.917 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj 2022-09-03T05:37:12.918 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1y36wu2Pkct48+yW/9iwLXmeezc= 2022-09-03T05:37:12.919 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1y36wu2Pkct48+yW/9iwLXmeezc=', 'Content-Length': '0'} 2022-09-03T05:37:12.923 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001ef3b1b0265f0feb-006312e808-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.924 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:12.926 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.927 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.933 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.934 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.934 DEBUG:boto:Method: GET 2022-09-03T05:37:12.935 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.935 DEBUG:boto:Data: 2022-09-03T05:37:12.936 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.937 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:12.937 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.938 DEBUG:boto:Params: {} 2022-09-03T05:37:12.939 DEBUG:boto:Token: None 2022-09-03T05:37:12.939 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.940 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BKz/vWcFkHXz8VwBJxwcKxZb+go= 2022-09-03T05:37:12.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BKz/vWcFkHXz8VwBJxwcKxZb+go=', 'Content-Length': '0'} 2022-09-03T05:37:12.948 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e4223d54f74bf5b6-006312e808-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.949 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:12.951 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:12.952 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.953 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.954 DEBUG:boto:Method: GET 2022-09-03T05:37:12.955 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.956 DEBUG:boto:Data: 2022-09-03T05:37:12.957 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.964 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:12.965 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.966 DEBUG:boto:Params: {} 2022-09-03T05:37:12.968 DEBUG:boto:Token: None 2022-09-03T05:37:12.969 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:12.970 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BKz/vWcFkHXz8VwBJxwcKxZb+go= 2022-09-03T05:37:12.971 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BKz/vWcFkHXz8VwBJxwcKxZb+go=', 'Content-Length': '0'} 2022-09-03T05:37:12.974 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000074e9e2e27ab73d23-006312e808-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.976 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:12.977 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:12.979 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.979 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.980 DEBUG:boto:Method: GET 2022-09-03T05:37:12.981 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.981 DEBUG:boto:Data: 2022-09-03T05:37:12.982 DEBUG:boto:Headers: {} 2022-09-03T05:37:12.982 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:12.983 DEBUG:boto:Port: 8000 2022-09-03T05:37:12.983 DEBUG:boto:Params: {} 2022-09-03T05:37:12.984 DEBUG:boto:Token: None 2022-09-03T05:37:12.984 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:12 GMT /dsgvcy-19/ 2022-09-03T05:37:12.985 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk= 2022-09-03T05:37:12.986 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UxKPm19c1hhM6puyb/+oj+DIUjk=', 'Content-Length': '0'} 2022-09-03T05:37:12.992 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bda827fe5f159acb-006312e808-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:12.993 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:12.995 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:12.995 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:12.996 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.997 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.998 DEBUG:boto:Method: GET 2022-09-03T05:37:12.999 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:12.999 DEBUG:boto:Data: 2022-09-03T05:37:13.001 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.001 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.002 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.003 DEBUG:boto:Params: {} 2022-09-03T05:37:13.004 DEBUG:boto:Token: None 2022-09-03T05:37:13.005 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/ 2022-09-03T05:37:13.007 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA= 2022-09-03T05:37:13.007 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA=', 'Content-Length': '0'} 2022-09-03T05:37:13.011 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b9a17d6a4bde8be2-006312e809-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.054 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:13.055 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:13.056 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.057 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.057 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.058 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.059 DEBUG:boto:Method: GET 2022-09-03T05:37:13.060 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.061 DEBUG:boto:Data: 2022-09-03T05:37:13.062 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.063 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.065 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.066 DEBUG:boto:Params: {} 2022-09-03T05:37:13.067 DEBUG:boto:Token: None 2022-09-03T05:37:13.069 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.070 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q2zq860qQP27sFEQMrRfo6VZZOY= 2022-09-03T05:37:13.071 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q2zq860qQP27sFEQMrRfo6VZZOY=', 'Content-Length': '0'} 2022-09-03T05:37:13.080 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kUiw73n7LyoUiDcS9JXEfdlCebChUCE'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d5c8655857cd00db-006312e809-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.082 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.083 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.084 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.085 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.086 DEBUG:boto:Method: GET 2022-09-03T05:37:13.087 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.089 DEBUG:boto:Data: 2022-09-03T05:37:13.090 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.091 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.092 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.093 DEBUG:boto:Params: {} 2022-09-03T05:37:13.095 DEBUG:boto:Token: None 2022-09-03T05:37:13.096 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.097 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q2zq860qQP27sFEQMrRfo6VZZOY= 2022-09-03T05:37:13.098 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q2zq860qQP27sFEQMrRfo6VZZOY=', 'Content-Length': '0'} 2022-09-03T05:37:13.100 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kUiw73n7LyoUiDcS9JXEfdlCebChUCE'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000321dbd9d34a0dd15-006312e809-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.102 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.103 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.103 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.104 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.105 DEBUG:boto:Method: HEAD 2022-09-03T05:37:13.106 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.107 DEBUG:boto:Data: 2022-09-03T05:37:13.108 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.109 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.110 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.111 DEBUG:boto:Params: {} 2022-09-03T05:37:13.111 DEBUG:boto:Token: None 2022-09-03T05:37:13.112 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.113 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HIYtmWg+FNvJqavsasW4HOZtvj8= 2022-09-03T05:37:13.114 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HIYtmWg+FNvJqavsasW4HOZtvj8=', 'Content-Length': '0'} 2022-09-03T05:37:13.116 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kUiw73n7LyoUiDcS9JXEfdlCebChUCE'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000936e0811ebf1edf8-006312e809-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.117 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.118 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.120 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.120 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.121 DEBUG:boto:Method: HEAD 2022-09-03T05:37:13.122 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.123 DEBUG:boto:Data: 2022-09-03T05:37:13.124 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.125 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.126 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.126 DEBUG:boto:Params: {} 2022-09-03T05:37:13.127 DEBUG:boto:Token: None 2022-09-03T05:37:13.128 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.129 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HIYtmWg+FNvJqavsasW4HOZtvj8= 2022-09-03T05:37:13.130 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HIYtmWg+FNvJqavsasW4HOZtvj8=', 'Content-Length': '0'} 2022-09-03T05:37:13.133 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kUiw73n7LyoUiDcS9JXEfdlCebChUCE'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000004865ce590609121-006312e809-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.134 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:13.134 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.135 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.135 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.136 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.136 DEBUG:boto:Method: GET 2022-09-03T05:37:13.137 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.137 DEBUG:boto:Data: 2022-09-03T05:37:13.138 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.138 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.139 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.139 DEBUG:boto:Params: {} 2022-09-03T05:37:13.140 DEBUG:boto:Token: None 2022-09-03T05:37:13.141 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.141 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q2zq860qQP27sFEQMrRfo6VZZOY= 2022-09-03T05:37:13.142 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q2zq860qQP27sFEQMrRfo6VZZOY=', 'Content-Length': '0'} 2022-09-03T05:37:13.144 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kUiw73n7LyoUiDcS9JXEfdlCebChUCE'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000056c2f8c9d9aca7fb-006312e809-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.145 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.146 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.146 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.147 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.147 DEBUG:boto:Method: GET 2022-09-03T05:37:13.148 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.148 DEBUG:boto:Data: 2022-09-03T05:37:13.149 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.149 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.150 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.150 DEBUG:boto:Params: {} 2022-09-03T05:37:13.151 DEBUG:boto:Token: None 2022-09-03T05:37:13.151 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:13.152 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q2zq860qQP27sFEQMrRfo6VZZOY= 2022-09-03T05:37:13.153 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q2zq860qQP27sFEQMrRfo6VZZOY=', 'Content-Length': '0'} 2022-09-03T05:37:13.156 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kUiw73n7LyoUiDcS9JXEfdlCebChUCE'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002a396558937443b0-006312e809-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.158 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:13.159 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:13.160 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.161 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.162 DEBUG:boto:Method: GET 2022-09-03T05:37:13.163 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.164 DEBUG:boto:Data: 2022-09-03T05:37:13.164 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.165 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.166 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.167 DEBUG:boto:Params: {} 2022-09-03T05:37:13.168 DEBUG:boto:Token: None 2022-09-03T05:37:13.169 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/ 2022-09-03T05:37:13.171 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA= 2022-09-03T05:37:13.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA=', 'Content-Length': '0'} 2022-09-03T05:37:13.177 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000027f85e57a612b816-006312e809-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.179 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:13.181 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:13.182 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:13.183 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.184 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.185 DEBUG:boto:Method: GET 2022-09-03T05:37:13.186 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.187 DEBUG:boto:Data: 2022-09-03T05:37:13.189 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.190 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.190 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.191 DEBUG:boto:Params: {} 2022-09-03T05:37:13.192 DEBUG:boto:Token: None 2022-09-03T05:37:13.193 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/ 2022-09-03T05:37:13.194 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA= 2022-09-03T05:37:13.195 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA=', 'Content-Length': '0'} 2022-09-03T05:37:13.199 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cf61959f61c943d0-006312e809-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.200 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:13.201 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.202 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.202 DEBUG:boto:Method: HEAD 2022-09-03T05:37:13.203 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:13.203 DEBUG:boto:Data: 2022-09-03T05:37:13.204 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.204 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.205 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.205 DEBUG:boto:Params: {} 2022-09-03T05:37:13.206 DEBUG:boto:Token: None 2022-09-03T05:37:13.207 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj 2022-09-03T05:37:13.208 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VCWJ2ebSEf2tpnhhEaF7tveqRTc= 2022-09-03T05:37:13.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VCWJ2ebSEf2tpnhhEaF7tveqRTc=', 'Content-Length': '0'} 2022-09-03T05:37:13.221 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002f013690b5466ac7-006312e809-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.222 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.223 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.223 DEBUG:boto:Method: HEAD 2022-09-03T05:37:13.224 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:13.224 DEBUG:boto:Data: 2022-09-03T05:37:13.225 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.225 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.226 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.226 DEBUG:boto:Params: {} 2022-09-03T05:37:13.227 DEBUG:boto:Token: None 2022-09-03T05:37:13.228 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj 2022-09-03T05:37:13.228 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VCWJ2ebSEf2tpnhhEaF7tveqRTc= 2022-09-03T05:37:13.229 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VCWJ2ebSEf2tpnhhEaF7tveqRTc=', 'Content-Length': '0'} 2022-09-03T05:37:13.244 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000309bfaa05e57490d-006312e809-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.246 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:13.247 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.248 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.249 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.249 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.250 DEBUG:boto:Method: GET 2022-09-03T05:37:13.251 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.251 DEBUG:boto:Data: 2022-09-03T05:37:13.252 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.252 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.253 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.253 DEBUG:boto:Params: {} 2022-09-03T05:37:13.254 DEBUG:boto:Token: None 2022-09-03T05:37:13.255 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.256 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gph23U+XA9gkqYZtwU1zGU/d4xM= 2022-09-03T05:37:13.257 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gph23U+XA9gkqYZtwU1zGU/d4xM=', 'Content-Length': '0'} 2022-09-03T05:37:13.260 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bf769fa10bfc9ac4-006312e809-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.262 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.263 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.264 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.265 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.266 DEBUG:boto:Method: GET 2022-09-03T05:37:13.267 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.268 DEBUG:boto:Data: 2022-09-03T05:37:13.269 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.270 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.271 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.272 DEBUG:boto:Params: {} 2022-09-03T05:37:13.273 DEBUG:boto:Token: None 2022-09-03T05:37:13.274 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.275 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gph23U+XA9gkqYZtwU1zGU/d4xM= 2022-09-03T05:37:13.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gph23U+XA9gkqYZtwU1zGU/d4xM=', 'Content-Length': '0'} 2022-09-03T05:37:13.279 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cc95cace4a176be2-006312e809-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.281 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:13.282 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:13.283 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.284 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.285 DEBUG:boto:Method: GET 2022-09-03T05:37:13.286 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.287 DEBUG:boto:Data: 2022-09-03T05:37:13.288 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.289 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.290 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.291 DEBUG:boto:Params: {} 2022-09-03T05:37:13.292 DEBUG:boto:Token: None 2022-09-03T05:37:13.293 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/ 2022-09-03T05:37:13.294 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA= 2022-09-03T05:37:13.295 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA=', 'Content-Length': '0'} 2022-09-03T05:37:13.300 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000090deb708580ca219-006312e809-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.301 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:13.302 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:13.302 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:13.303 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.303 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.304 DEBUG:boto:Method: GET 2022-09-03T05:37:13.305 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.305 DEBUG:boto:Data: 2022-09-03T05:37:13.306 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.306 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.307 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.308 DEBUG:boto:Params: {} 2022-09-03T05:37:13.308 DEBUG:boto:Token: None 2022-09-03T05:37:13.309 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/ 2022-09-03T05:37:13.309 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA= 2022-09-03T05:37:13.310 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA=', 'Content-Length': '0'} 2022-09-03T05:37:13.314 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c13321a483023db6-006312e809-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.358 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:13.360 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:13.361 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.362 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.363 DEBUG:boto:path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.365 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.366 DEBUG:boto:Method: GET 2022-09-03T05:37:13.367 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.368 DEBUG:boto:Data: 2022-09-03T05:37:13.370 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.370 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.371 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.372 DEBUG:boto:Params: {} 2022-09-03T05:37:13.372 DEBUG:boto:Token: None 2022-09-03T05:37:13.373 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.374 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fe1g52Eks/8ZhZ/h0gZ1YJ+BmQ0= 2022-09-03T05:37:13.375 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fe1g52Eks/8ZhZ/h0gZ1YJ+BmQ0=', 'Content-Length': '0'} 2022-09-03T05:37:13.378 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000edbe64daa6a8bca8-006312e809-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.379 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.380 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.381 DEBUG:boto:path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.381 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.382 DEBUG:boto:Method: GET 2022-09-03T05:37:13.383 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.384 DEBUG:boto:Data: 2022-09-03T05:37:13.385 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.386 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.387 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.388 DEBUG:boto:Params: {} 2022-09-03T05:37:13.390 DEBUG:boto:Token: None 2022-09-03T05:37:13.391 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.392 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fe1g52Eks/8ZhZ/h0gZ1YJ+BmQ0= 2022-09-03T05:37:13.393 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fe1g52Eks/8ZhZ/h0gZ1YJ+BmQ0=', 'Content-Length': '0'} 2022-09-03T05:37:13.396 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a8f85ca86f37bcb7-006312e809-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.398 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.399 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.400 DEBUG:boto:path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.400 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.401 DEBUG:boto:Method: HEAD 2022-09-03T05:37:13.402 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.403 DEBUG:boto:Data: 2022-09-03T05:37:13.403 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.404 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.405 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.406 DEBUG:boto:Params: {} 2022-09-03T05:37:13.407 DEBUG:boto:Token: None 2022-09-03T05:37:13.407 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.408 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N6UJBZNfHycdWvWhY82Nf4pVMFs= 2022-09-03T05:37:13.409 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N6UJBZNfHycdWvWhY82Nf4pVMFs=', 'Content-Length': '0'} 2022-09-03T05:37:13.411 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006a1bbeb35e9caf78-006312e809-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.412 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.412 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.413 DEBUG:boto:path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.413 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.414 DEBUG:boto:Method: HEAD 2022-09-03T05:37:13.414 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.415 DEBUG:boto:Data: 2022-09-03T05:37:13.415 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.416 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.416 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.417 DEBUG:boto:Params: {} 2022-09-03T05:37:13.417 DEBUG:boto:Token: None 2022-09-03T05:37:13.418 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.419 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N6UJBZNfHycdWvWhY82Nf4pVMFs= 2022-09-03T05:37:13.419 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N6UJBZNfHycdWvWhY82Nf4pVMFs=', 'Content-Length': '0'} 2022-09-03T05:37:13.421 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bbc23ed32c4f316b-006312e809-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.422 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:13.423 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.423 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.424 DEBUG:boto:path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.424 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.425 DEBUG:boto:Method: GET 2022-09-03T05:37:13.425 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.426 DEBUG:boto:Data: 2022-09-03T05:37:13.426 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.427 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.427 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.428 DEBUG:boto:Params: {} 2022-09-03T05:37:13.429 DEBUG:boto:Token: None 2022-09-03T05:37:13.429 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.430 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fe1g52Eks/8ZhZ/h0gZ1YJ+BmQ0= 2022-09-03T05:37:13.431 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fe1g52Eks/8ZhZ/h0gZ1YJ+BmQ0=', 'Content-Length': '0'} 2022-09-03T05:37:13.434 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000011e1e8e045788663-006312e809-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.474 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.475 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.477 DEBUG:boto:path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.477 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.478 DEBUG:boto:Method: GET 2022-09-03T05:37:13.478 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.479 DEBUG:boto:Data: 2022-09-03T05:37:13.480 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.480 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.481 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.482 DEBUG:boto:Params: {} 2022-09-03T05:37:13.483 DEBUG:boto:Token: None 2022-09-03T05:37:13.483 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:13.484 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fe1g52Eks/8ZhZ/h0gZ1YJ+BmQ0= 2022-09-03T05:37:13.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fe1g52Eks/8ZhZ/h0gZ1YJ+BmQ0=', 'Content-Length': '0'} 2022-09-03T05:37:13.487 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004ecabff186d567b9-006312e809-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.488 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:13.488 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:13.489 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.489 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.490 DEBUG:boto:Method: GET 2022-09-03T05:37:13.491 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.491 DEBUG:boto:Data: 2022-09-03T05:37:13.492 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.492 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.493 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.493 DEBUG:boto:Params: {} 2022-09-03T05:37:13.494 DEBUG:boto:Token: None 2022-09-03T05:37:13.496 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/ 2022-09-03T05:37:13.497 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA= 2022-09-03T05:37:13.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA=', 'Content-Length': '0'} 2022-09-03T05:37:13.500 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cd209e79fc89a029-006312e809-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.501 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:13.502 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:13.503 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:13.503 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.504 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.505 DEBUG:boto:Method: GET 2022-09-03T05:37:13.505 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.506 DEBUG:boto:Data: 2022-09-03T05:37:13.506 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.507 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.507 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.508 DEBUG:boto:Params: {} 2022-09-03T05:37:13.509 DEBUG:boto:Token: None 2022-09-03T05:37:13.509 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/ 2022-09-03T05:37:13.510 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA= 2022-09-03T05:37:13.511 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA=', 'Content-Length': '0'} 2022-09-03T05:37:13.514 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000c09e09608c7fb5b-006312e809-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.514 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:13.515 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.516 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.516 DEBUG:boto:Method: HEAD 2022-09-03T05:37:13.517 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:13.517 DEBUG:boto:Data: 2022-09-03T05:37:13.518 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.519 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.519 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.520 DEBUG:boto:Params: {} 2022-09-03T05:37:13.520 DEBUG:boto:Token: None 2022-09-03T05:37:13.521 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj 2022-09-03T05:37:13.522 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VCWJ2ebSEf2tpnhhEaF7tveqRTc= 2022-09-03T05:37:13.522 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VCWJ2ebSEf2tpnhhEaF7tveqRTc=', 'Content-Length': '0'} 2022-09-03T05:37:13.526 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d141cda22942bdf5-006312e809-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.527 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.527 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.528 DEBUG:boto:Method: HEAD 2022-09-03T05:37:13.529 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:13.529 DEBUG:boto:Data: 2022-09-03T05:37:13.530 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.531 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.531 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.532 DEBUG:boto:Params: {} 2022-09-03T05:37:13.532 DEBUG:boto:Token: None 2022-09-03T05:37:13.533 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj 2022-09-03T05:37:13.533 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VCWJ2ebSEf2tpnhhEaF7tveqRTc= 2022-09-03T05:37:13.534 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VCWJ2ebSEf2tpnhhEaF7tveqRTc=', 'Content-Length': '0'} 2022-09-03T05:37:13.539 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001d766a501ca24ca4-006312e809-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.540 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:13.541 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.541 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.542 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.543 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.544 DEBUG:boto:Method: GET 2022-09-03T05:37:13.544 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.545 DEBUG:boto:Data: 2022-09-03T05:37:13.546 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.546 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.547 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.547 DEBUG:boto:Params: {} 2022-09-03T05:37:13.548 DEBUG:boto:Token: None 2022-09-03T05:37:13.549 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.549 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gph23U+XA9gkqYZtwU1zGU/d4xM= 2022-09-03T05:37:13.550 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gph23U+XA9gkqYZtwU1zGU/d4xM=', 'Content-Length': '0'} 2022-09-03T05:37:13.555 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007f0177b176ae3853-006312e809-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.557 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.558 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.560 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.561 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.561 DEBUG:boto:Method: GET 2022-09-03T05:37:13.562 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.562 DEBUG:boto:Data: 2022-09-03T05:37:13.563 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.563 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.564 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.564 DEBUG:boto:Params: {} 2022-09-03T05:37:13.565 DEBUG:boto:Token: None 2022-09-03T05:37:13.566 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.569 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gph23U+XA9gkqYZtwU1zGU/d4xM= 2022-09-03T05:37:13.571 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gph23U+XA9gkqYZtwU1zGU/d4xM=', 'Content-Length': '0'} 2022-09-03T05:37:13.574 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000096daa5977df2ca92-006312e809-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.627 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:13.628 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:13.629 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.629 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.630 DEBUG:boto:Method: GET 2022-09-03T05:37:13.630 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.631 DEBUG:boto:Data: 2022-09-03T05:37:13.632 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.632 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.633 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.634 DEBUG:boto:Params: {} 2022-09-03T05:37:13.634 DEBUG:boto:Token: None 2022-09-03T05:37:13.636 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/ 2022-09-03T05:37:13.637 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA= 2022-09-03T05:37:13.638 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA=', 'Content-Length': '0'} 2022-09-03T05:37:13.641 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003dcfddf13ca97f02-006312e809-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.642 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:13.643 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:13.644 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:13.645 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.645 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.646 DEBUG:boto:Method: GET 2022-09-03T05:37:13.647 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.647 DEBUG:boto:Data: 2022-09-03T05:37:13.653 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.654 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.654 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.655 DEBUG:boto:Params: {} 2022-09-03T05:37:13.656 DEBUG:boto:Token: None 2022-09-03T05:37:13.656 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/ 2022-09-03T05:37:13.657 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA= 2022-09-03T05:37:13.657 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA=', 'Content-Length': '0'} 2022-09-03T05:37:13.661 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a68340254bc631ae-006312e809-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.661 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:13.662 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:13.665 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.665 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.666 DEBUG:boto:path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.667 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.667 DEBUG:boto:Method: GET 2022-09-03T05:37:13.668 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.669 DEBUG:boto:Data: 2022-09-03T05:37:13.669 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.670 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.671 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.671 DEBUG:boto:Params: {} 2022-09-03T05:37:13.672 DEBUG:boto:Token: None 2022-09-03T05:37:13.672 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.673 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3TJFOgyS2cb5c7R3qMEg9EUtQZk= 2022-09-03T05:37:13.675 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3TJFOgyS2cb5c7R3qMEg9EUtQZk=', 'Content-Length': '0'} 2022-09-03T05:37:13.678 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009a2458d127f74ab6-006312e809-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.680 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.680 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.681 DEBUG:boto:path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.681 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.682 DEBUG:boto:Method: GET 2022-09-03T05:37:13.683 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.683 DEBUG:boto:Data: 2022-09-03T05:37:13.684 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.685 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.685 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.686 DEBUG:boto:Params: {} 2022-09-03T05:37:13.687 DEBUG:boto:Token: None 2022-09-03T05:37:13.688 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.689 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3TJFOgyS2cb5c7R3qMEg9EUtQZk= 2022-09-03T05:37:13.689 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3TJFOgyS2cb5c7R3qMEg9EUtQZk=', 'Content-Length': '0'} 2022-09-03T05:37:13.692 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000021796892cd9bd77a-006312e809-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.693 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.693 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.694 DEBUG:boto:path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.694 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.695 DEBUG:boto:Method: HEAD 2022-09-03T05:37:13.695 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.696 DEBUG:boto:Data: 2022-09-03T05:37:13.697 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.698 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.698 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.699 DEBUG:boto:Params: {} 2022-09-03T05:37:13.699 DEBUG:boto:Token: None 2022-09-03T05:37:13.700 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.701 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kyIXXIBLwoenqh2JfOVtSxdX1v0= 2022-09-03T05:37:13.702 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kyIXXIBLwoenqh2JfOVtSxdX1v0=', 'Content-Length': '0'} 2022-09-03T05:37:13.704 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000cce9ed8ef034483-006312e809-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.705 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.706 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.706 DEBUG:boto:path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.707 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.707 DEBUG:boto:Method: HEAD 2022-09-03T05:37:13.708 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.708 DEBUG:boto:Data: 2022-09-03T05:37:13.709 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.710 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.711 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.711 DEBUG:boto:Params: {} 2022-09-03T05:37:13.712 DEBUG:boto:Token: None 2022-09-03T05:37:13.713 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.713 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kyIXXIBLwoenqh2JfOVtSxdX1v0= 2022-09-03T05:37:13.714 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kyIXXIBLwoenqh2JfOVtSxdX1v0=', 'Content-Length': '0'} 2022-09-03T05:37:13.716 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003b64e29b4b1fd3ab-006312e809-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.717 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:13.717 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.718 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.719 DEBUG:boto:path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.719 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.720 DEBUG:boto:Method: GET 2022-09-03T05:37:13.720 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.721 DEBUG:boto:Data: 2022-09-03T05:37:13.721 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.722 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.722 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.723 DEBUG:boto:Params: {} 2022-09-03T05:37:13.723 DEBUG:boto:Token: None 2022-09-03T05:37:13.724 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.725 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3TJFOgyS2cb5c7R3qMEg9EUtQZk= 2022-09-03T05:37:13.725 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3TJFOgyS2cb5c7R3qMEg9EUtQZk=', 'Content-Length': '0'} 2022-09-03T05:37:13.727 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007215fcae3aaa1335-006312e809-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.728 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.729 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.729 DEBUG:boto:path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.730 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.731 DEBUG:boto:Method: GET 2022-09-03T05:37:13.732 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.732 DEBUG:boto:Data: 2022-09-03T05:37:13.733 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.733 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.734 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.734 DEBUG:boto:Params: {} 2022-09-03T05:37:13.735 DEBUG:boto:Token: None 2022-09-03T05:37:13.735 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:13.736 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3TJFOgyS2cb5c7R3qMEg9EUtQZk= 2022-09-03T05:37:13.737 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3TJFOgyS2cb5c7R3qMEg9EUtQZk=', 'Content-Length': '0'} 2022-09-03T05:37:13.739 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000089ade5c3c8d9cdc6-006312e809-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.740 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:13.740 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:13.741 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.741 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.742 DEBUG:boto:Method: GET 2022-09-03T05:37:13.742 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.743 DEBUG:boto:Data: 2022-09-03T05:37:13.743 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.744 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.745 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.745 DEBUG:boto:Params: {} 2022-09-03T05:37:13.746 DEBUG:boto:Token: None 2022-09-03T05:37:13.746 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/ 2022-09-03T05:37:13.747 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA= 2022-09-03T05:37:13.747 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA=', 'Content-Length': '0'} 2022-09-03T05:37:13.751 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000594aa6f68305c8f0-006312e809-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.751 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:13.752 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:13.753 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:13.754 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.754 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.755 DEBUG:boto:Method: GET 2022-09-03T05:37:13.755 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.756 DEBUG:boto:Data: 2022-09-03T05:37:13.756 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.757 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.757 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.758 DEBUG:boto:Params: {} 2022-09-03T05:37:13.759 DEBUG:boto:Token: None 2022-09-03T05:37:13.759 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/ 2022-09-03T05:37:13.760 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA= 2022-09-03T05:37:13.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA=', 'Content-Length': '0'} 2022-09-03T05:37:13.763 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000012aea51044f37d15-006312e809-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.764 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:13.765 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.766 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.766 DEBUG:boto:Method: HEAD 2022-09-03T05:37:13.767 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:13.767 DEBUG:boto:Data: 2022-09-03T05:37:13.768 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.768 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.769 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.769 DEBUG:boto:Params: {} 2022-09-03T05:37:13.770 DEBUG:boto:Token: None 2022-09-03T05:37:13.770 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj 2022-09-03T05:37:13.771 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VCWJ2ebSEf2tpnhhEaF7tveqRTc= 2022-09-03T05:37:13.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VCWJ2ebSEf2tpnhhEaF7tveqRTc=', 'Content-Length': '0'} 2022-09-03T05:37:13.775 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000719127d71b5a45e1-006312e809-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.776 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.776 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.777 DEBUG:boto:Method: HEAD 2022-09-03T05:37:13.777 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:13.778 DEBUG:boto:Data: 2022-09-03T05:37:13.778 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.779 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.780 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.780 DEBUG:boto:Params: {} 2022-09-03T05:37:13.781 DEBUG:boto:Token: None 2022-09-03T05:37:13.781 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj 2022-09-03T05:37:13.782 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VCWJ2ebSEf2tpnhhEaF7tveqRTc= 2022-09-03T05:37:13.783 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VCWJ2ebSEf2tpnhhEaF7tveqRTc=', 'Content-Length': '0'} 2022-09-03T05:37:13.786 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003427c9966fa901cc-006312e809-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.787 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:13.788 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.788 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.789 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.789 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.790 DEBUG:boto:Method: GET 2022-09-03T05:37:13.790 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.791 DEBUG:boto:Data: 2022-09-03T05:37:13.791 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.792 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.793 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.793 DEBUG:boto:Params: {} 2022-09-03T05:37:13.794 DEBUG:boto:Token: None 2022-09-03T05:37:13.794 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.795 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gph23U+XA9gkqYZtwU1zGU/d4xM= 2022-09-03T05:37:13.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gph23U+XA9gkqYZtwU1zGU/d4xM=', 'Content-Length': '0'} 2022-09-03T05:37:13.798 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001fa105a58a6ff614-006312e809-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.842 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.843 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.843 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.844 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.845 DEBUG:boto:Method: GET 2022-09-03T05:37:13.845 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.846 DEBUG:boto:Data: 2022-09-03T05:37:13.846 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.847 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.847 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.848 DEBUG:boto:Params: {} 2022-09-03T05:37:13.848 DEBUG:boto:Token: None 2022-09-03T05:37:13.849 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:13.850 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gph23U+XA9gkqYZtwU1zGU/d4xM= 2022-09-03T05:37:13.851 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gph23U+XA9gkqYZtwU1zGU/d4xM=', 'Content-Length': '0'} 2022-09-03T05:37:13.853 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000090db61b9226e4a9a-006312e809-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.854 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:13.855 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:13.855 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.856 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.857 DEBUG:boto:Method: GET 2022-09-03T05:37:13.857 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.858 DEBUG:boto:Data: 2022-09-03T05:37:13.858 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.859 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.859 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.860 DEBUG:boto:Params: {} 2022-09-03T05:37:13.860 DEBUG:boto:Token: None 2022-09-03T05:37:13.861 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/ 2022-09-03T05:37:13.861 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA= 2022-09-03T05:37:13.862 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA=', 'Content-Length': '0'} 2022-09-03T05:37:13.866 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002062f6f700fc6cea-006312e809-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.866 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:13.867 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:13.868 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:13.868 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.869 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.869 DEBUG:boto:Method: GET 2022-09-03T05:37:13.870 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:13.871 DEBUG:boto:Data: 2022-09-03T05:37:13.871 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.872 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.873 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.874 DEBUG:boto:Params: {} 2022-09-03T05:37:13.875 DEBUG:boto:Token: None 2022-09-03T05:37:13.877 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/ 2022-09-03T05:37:13.878 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA= 2022-09-03T05:37:13.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NRiCOVnuHCd1WGFFtcSBDIjiuJA=', 'Content-Length': '0'} 2022-09-03T05:37:13.883 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004684d183599be7cb-006312e809-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.884 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:13.885 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:13.886 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.887 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.888 DEBUG:boto:path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.889 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.890 DEBUG:boto:Method: GET 2022-09-03T05:37:13.891 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.892 DEBUG:boto:Data: 2022-09-03T05:37:13.893 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.893 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.894 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.895 DEBUG:boto:Params: {} 2022-09-03T05:37:13.896 DEBUG:boto:Token: None 2022-09-03T05:37:13.897 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.897 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IMx28ckprXewMt/wnKbm8n6TFbc= 2022-09-03T05:37:13.898 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IMx28ckprXewMt/wnKbm8n6TFbc=', 'Content-Length': '0'} 2022-09-03T05:37:13.900 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d258207fe61e4d3f-006312e809-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.902 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.903 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.903 DEBUG:boto:path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.904 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.904 DEBUG:boto:Method: GET 2022-09-03T05:37:13.905 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.906 DEBUG:boto:Data: 2022-09-03T05:37:13.906 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.907 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.908 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.909 DEBUG:boto:Params: {} 2022-09-03T05:37:13.910 DEBUG:boto:Token: None 2022-09-03T05:37:13.911 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.911 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IMx28ckprXewMt/wnKbm8n6TFbc= 2022-09-03T05:37:13.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IMx28ckprXewMt/wnKbm8n6TFbc=', 'Content-Length': '0'} 2022-09-03T05:37:13.915 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000080f58616ea9895d2-006312e809-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.918 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.919 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.920 DEBUG:boto:path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.920 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.921 DEBUG:boto:Method: HEAD 2022-09-03T05:37:13.922 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.922 DEBUG:boto:Data: 2022-09-03T05:37:13.923 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.923 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.924 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.924 DEBUG:boto:Params: {} 2022-09-03T05:37:13.925 DEBUG:boto:Token: None 2022-09-03T05:37:13.926 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.926 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q3XTkM4LXHu1Zk1tJ9xbwNIQiis= 2022-09-03T05:37:13.927 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q3XTkM4LXHu1Zk1tJ9xbwNIQiis=', 'Content-Length': '0'} 2022-09-03T05:37:13.929 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006b4768308cf6a5cd-006312e809-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.930 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.931 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.931 DEBUG:boto:path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.932 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.933 DEBUG:boto:Method: HEAD 2022-09-03T05:37:13.933 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.934 DEBUG:boto:Data: 2022-09-03T05:37:13.935 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.935 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.936 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.937 DEBUG:boto:Params: {} 2022-09-03T05:37:13.938 DEBUG:boto:Token: None 2022-09-03T05:37:13.939 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.940 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q3XTkM4LXHu1Zk1tJ9xbwNIQiis= 2022-09-03T05:37:13.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q3XTkM4LXHu1Zk1tJ9xbwNIQiis=', 'Content-Length': '0'} 2022-09-03T05:37:13.942 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000003eaf8ef5f4e87c0-006312e809-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.943 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:13.944 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.944 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.945 DEBUG:boto:path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.946 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.946 DEBUG:boto:Method: GET 2022-09-03T05:37:13.947 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.947 DEBUG:boto:Data: 2022-09-03T05:37:13.948 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.949 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:13.950 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.951 DEBUG:boto:Params: {} 2022-09-03T05:37:13.952 DEBUG:boto:Token: None 2022-09-03T05:37:13.952 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.953 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IMx28ckprXewMt/wnKbm8n6TFbc= 2022-09-03T05:37:13.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IMx28ckprXewMt/wnKbm8n6TFbc=', 'Content-Length': '0'} 2022-09-03T05:37:13.957 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000591e61ed4ee63929-006312e809-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:13.958 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:13.959 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:13.959 DEBUG:boto:path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.961 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.962 DEBUG:boto:Method: GET 2022-09-03T05:37:13.963 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.963 DEBUG:boto:Data: 2022-09-03T05:37:13.964 DEBUG:boto:Headers: {} 2022-09-03T05:37:13.964 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:13.965 DEBUG:boto:Port: 8000 2022-09-03T05:37:13.965 DEBUG:boto:Params: {} 2022-09-03T05:37:13.966 DEBUG:boto:Token: None 2022-09-03T05:37:13.967 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:13 GMT /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:13.968 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IMx28ckprXewMt/wnKbm8n6TFbc= 2022-09-03T05:37:13.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IMx28ckprXewMt/wnKbm8n6TFbc=', 'Content-Length': '0'} 2022-09-03T05:37:13.971 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000099004680de2accc5-006312e809-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.014 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:14.016 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:14.016 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.017 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.018 DEBUG:boto:Method: GET 2022-09-03T05:37:14.018 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.019 DEBUG:boto:Data: 2022-09-03T05:37:14.019 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.020 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:14.021 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.021 DEBUG:boto:Params: {} 2022-09-03T05:37:14.022 DEBUG:boto:Token: None 2022-09-03T05:37:14.023 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/ 2022-09-03T05:37:14.024 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI= 2022-09-03T05:37:14.026 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI=', 'Content-Length': '0'} 2022-09-03T05:37:14.032 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009633a829d9fe7fc0-006312e80a-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.034 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:14.035 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:14.036 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:14.037 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.039 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.040 DEBUG:boto:Method: GET 2022-09-03T05:37:14.041 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.042 DEBUG:boto:Data: 2022-09-03T05:37:14.043 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.044 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:14.045 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.045 DEBUG:boto:Params: {} 2022-09-03T05:37:14.047 DEBUG:boto:Token: None 2022-09-03T05:37:14.048 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/ 2022-09-03T05:37:14.049 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI= 2022-09-03T05:37:14.051 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI=', 'Content-Length': '0'} 2022-09-03T05:37:14.055 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007e840ea2a5ab6e22-006312e80a-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.057 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:14.058 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.059 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.061 DEBUG:boto:Method: HEAD 2022-09-03T05:37:14.062 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:14.063 DEBUG:boto:Data: 2022-09-03T05:37:14.063 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.064 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:14.064 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.065 DEBUG:boto:Params: {} 2022-09-03T05:37:14.065 DEBUG:boto:Token: None 2022-09-03T05:37:14.066 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj 2022-09-03T05:37:14.067 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eTVKf4Y5YXm4aXrxETQjqe2rQfQ= 2022-09-03T05:37:14.067 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eTVKf4Y5YXm4aXrxETQjqe2rQfQ=', 'Content-Length': '0'} 2022-09-03T05:37:14.071 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d828042875b55b6a-006312e80a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.073 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.074 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.075 DEBUG:boto:Method: HEAD 2022-09-03T05:37:14.076 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:14.077 DEBUG:boto:Data: 2022-09-03T05:37:14.078 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.079 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:14.079 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.081 DEBUG:boto:Params: {} 2022-09-03T05:37:14.083 DEBUG:boto:Token: None 2022-09-03T05:37:14.084 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj 2022-09-03T05:37:14.085 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eTVKf4Y5YXm4aXrxETQjqe2rQfQ= 2022-09-03T05:37:14.086 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eTVKf4Y5YXm4aXrxETQjqe2rQfQ=', 'Content-Length': '0'} 2022-09-03T05:37:14.090 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008eed323db1485c6c-006312e80a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.092 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:14.093 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.093 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.094 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.095 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.096 DEBUG:boto:Method: GET 2022-09-03T05:37:14.097 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.097 DEBUG:boto:Data: 2022-09-03T05:37:14.098 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.098 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:14.099 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.100 DEBUG:boto:Params: {} 2022-09-03T05:37:14.101 DEBUG:boto:Token: None 2022-09-03T05:37:14.101 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.102 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TZDqBWkhdhgy5Mg5GvdX85+GBMQ= 2022-09-03T05:37:14.102 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TZDqBWkhdhgy5Mg5GvdX85+GBMQ=', 'Content-Length': '0'} 2022-09-03T05:37:14.105 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a589a80acacaa985-006312e80a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.150 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.152 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.153 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.154 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.155 DEBUG:boto:Method: GET 2022-09-03T05:37:14.155 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.156 DEBUG:boto:Data: 2022-09-03T05:37:14.156 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.157 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:14.157 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.158 DEBUG:boto:Params: {} 2022-09-03T05:37:14.159 DEBUG:boto:Token: None 2022-09-03T05:37:14.159 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.160 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TZDqBWkhdhgy5Mg5GvdX85+GBMQ= 2022-09-03T05:37:14.161 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TZDqBWkhdhgy5Mg5GvdX85+GBMQ=', 'Content-Length': '0'} 2022-09-03T05:37:14.165 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000097b5fb1efa1ddb1e-006312e80a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.167 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:14.168 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:14.169 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.170 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.171 DEBUG:boto:Method: GET 2022-09-03T05:37:14.173 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.174 DEBUG:boto:Data: 2022-09-03T05:37:14.175 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.176 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:14.177 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.178 DEBUG:boto:Params: {} 2022-09-03T05:37:14.179 DEBUG:boto:Token: None 2022-09-03T05:37:14.180 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/ 2022-09-03T05:37:14.181 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI= 2022-09-03T05:37:14.182 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI=', 'Content-Length': '0'} 2022-09-03T05:37:14.192 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000031054f4be31d858c-006312e80a-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.200 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:14.202 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:14.203 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:14.203 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.204 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.205 DEBUG:boto:Method: GET 2022-09-03T05:37:14.205 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.206 DEBUG:boto:Data: 2022-09-03T05:37:14.207 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.207 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:14.208 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.208 DEBUG:boto:Params: {} 2022-09-03T05:37:14.209 DEBUG:boto:Token: None 2022-09-03T05:37:14.212 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/ 2022-09-03T05:37:14.213 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI= 2022-09-03T05:37:14.214 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI=', 'Content-Length': '0'} 2022-09-03T05:37:14.219 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000011b937c56714836b-006312e80a-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.221 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:14.222 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:14.223 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.224 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.225 DEBUG:boto:path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.225 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.227 DEBUG:boto:Method: GET 2022-09-03T05:37:14.228 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.228 DEBUG:boto:Data: 2022-09-03T05:37:14.229 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.230 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:14.231 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.233 DEBUG:boto:Params: {} 2022-09-03T05:37:14.234 DEBUG:boto:Token: None 2022-09-03T05:37:14.235 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.236 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mic9Q7YStLLGatwV9ZoVZXmlQjo= 2022-09-03T05:37:14.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mic9Q7YStLLGatwV9ZoVZXmlQjo=', 'Content-Length': '0'} 2022-09-03T05:37:14.242 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f093137ccd806209-006312e80a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.246 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.247 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.248 DEBUG:boto:path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.249 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.250 DEBUG:boto:Method: GET 2022-09-03T05:37:14.251 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.252 DEBUG:boto:Data: 2022-09-03T05:37:14.253 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.254 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:14.255 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.256 DEBUG:boto:Params: {} 2022-09-03T05:37:14.257 DEBUG:boto:Token: None 2022-09-03T05:37:14.258 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.259 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mic9Q7YStLLGatwV9ZoVZXmlQjo= 2022-09-03T05:37:14.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mic9Q7YStLLGatwV9ZoVZXmlQjo=', 'Content-Length': '0'} 2022-09-03T05:37:14.263 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002b82e38d0e479daf-006312e80a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.265 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.267 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.270 DEBUG:boto:path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.270 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.271 DEBUG:boto:Method: HEAD 2022-09-03T05:37:14.272 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.272 DEBUG:boto:Data: 2022-09-03T05:37:14.273 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.273 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:14.274 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.274 DEBUG:boto:Params: {} 2022-09-03T05:37:14.275 DEBUG:boto:Token: None 2022-09-03T05:37:14.276 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.276 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V3HC3oyx4/gz7j0ItcH/x69mr8Y= 2022-09-03T05:37:14.277 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V3HC3oyx4/gz7j0ItcH/x69mr8Y=', 'Content-Length': '0'} 2022-09-03T05:37:14.279 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e9517e93255790f3-006312e80a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.280 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.280 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.281 DEBUG:boto:path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.282 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.284 DEBUG:boto:Method: HEAD 2022-09-03T05:37:14.285 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.286 DEBUG:boto:Data: 2022-09-03T05:37:14.288 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.289 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:14.289 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.290 DEBUG:boto:Params: {} 2022-09-03T05:37:14.291 DEBUG:boto:Token: None 2022-09-03T05:37:14.299 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.300 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V3HC3oyx4/gz7j0ItcH/x69mr8Y= 2022-09-03T05:37:14.301 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V3HC3oyx4/gz7j0ItcH/x69mr8Y=', 'Content-Length': '0'} 2022-09-03T05:37:14.321 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000002ab625678dcf4e4-006312e80a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.323 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:14.325 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.326 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.327 DEBUG:boto:path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.328 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.329 DEBUG:boto:Method: GET 2022-09-03T05:37:14.332 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.333 DEBUG:boto:Data: 2022-09-03T05:37:14.334 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.335 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:14.336 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.338 DEBUG:boto:Params: {} 2022-09-03T05:37:14.339 DEBUG:boto:Token: None 2022-09-03T05:37:14.340 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.341 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mic9Q7YStLLGatwV9ZoVZXmlQjo= 2022-09-03T05:37:14.342 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mic9Q7YStLLGatwV9ZoVZXmlQjo=', 'Content-Length': '0'} 2022-09-03T05:37:14.346 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008f91f8d8b28b16e3-006312e80a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.347 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.347 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.348 DEBUG:boto:path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.348 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.349 DEBUG:boto:Method: GET 2022-09-03T05:37:14.349 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.350 DEBUG:boto:Data: 2022-09-03T05:37:14.351 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.352 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:14.352 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.353 DEBUG:boto:Params: {} 2022-09-03T05:37:14.354 DEBUG:boto:Token: None 2022-09-03T05:37:14.355 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:14.362 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mic9Q7YStLLGatwV9ZoVZXmlQjo= 2022-09-03T05:37:14.363 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mic9Q7YStLLGatwV9ZoVZXmlQjo=', 'Content-Length': '0'} 2022-09-03T05:37:14.368 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001f449609229a9cf8-006312e80a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.371 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:14.372 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:14.372 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.373 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.374 DEBUG:boto:Method: GET 2022-09-03T05:37:14.380 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.382 DEBUG:boto:Data: 2022-09-03T05:37:14.383 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.384 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:14.384 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.385 DEBUG:boto:Params: {} 2022-09-03T05:37:14.386 DEBUG:boto:Token: None 2022-09-03T05:37:14.390 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/ 2022-09-03T05:37:14.391 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI= 2022-09-03T05:37:14.392 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI=', 'Content-Length': '0'} 2022-09-03T05:37:14.397 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009095e1be7dda1c6a-006312e80a-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.400 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:14.401 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:14.404 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:14.405 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.408 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.409 DEBUG:boto:Method: GET 2022-09-03T05:37:14.410 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.411 DEBUG:boto:Data: 2022-09-03T05:37:14.412 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.413 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:14.414 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.415 DEBUG:boto:Params: {} 2022-09-03T05:37:14.416 DEBUG:boto:Token: None 2022-09-03T05:37:14.417 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/ 2022-09-03T05:37:14.417 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI= 2022-09-03T05:37:14.418 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI=', 'Content-Length': '0'} 2022-09-03T05:37:14.422 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c3f550bfad3eaf58-006312e80a-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.423 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:14.424 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.425 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.427 DEBUG:boto:Method: HEAD 2022-09-03T05:37:14.428 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:14.429 DEBUG:boto:Data: 2022-09-03T05:37:14.430 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.431 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:14.431 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.432 DEBUG:boto:Params: {} 2022-09-03T05:37:14.433 DEBUG:boto:Token: None 2022-09-03T05:37:14.434 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj 2022-09-03T05:37:14.435 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eTVKf4Y5YXm4aXrxETQjqe2rQfQ= 2022-09-03T05:37:14.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eTVKf4Y5YXm4aXrxETQjqe2rQfQ=', 'Content-Length': '0'} 2022-09-03T05:37:14.440 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000605ca3d3c29e40b8-006312e80a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.441 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.442 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.443 DEBUG:boto:Method: HEAD 2022-09-03T05:37:14.444 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:14.444 DEBUG:boto:Data: 2022-09-03T05:37:14.445 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.445 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:14.446 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.447 DEBUG:boto:Params: {} 2022-09-03T05:37:14.447 DEBUG:boto:Token: None 2022-09-03T05:37:14.448 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj 2022-09-03T05:37:14.449 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eTVKf4Y5YXm4aXrxETQjqe2rQfQ= 2022-09-03T05:37:14.450 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eTVKf4Y5YXm4aXrxETQjqe2rQfQ=', 'Content-Length': '0'} 2022-09-03T05:37:14.455 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dfc3e848ee81f61d-006312e80a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.457 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:14.457 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.458 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.459 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.460 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.460 DEBUG:boto:Method: GET 2022-09-03T05:37:14.461 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.462 DEBUG:boto:Data: 2022-09-03T05:37:14.463 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.464 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:14.465 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.465 DEBUG:boto:Params: {} 2022-09-03T05:37:14.467 DEBUG:boto:Token: None 2022-09-03T05:37:14.468 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.469 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TZDqBWkhdhgy5Mg5GvdX85+GBMQ= 2022-09-03T05:37:14.469 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TZDqBWkhdhgy5Mg5GvdX85+GBMQ=', 'Content-Length': '0'} 2022-09-03T05:37:14.478 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c1ca417e03fdbfb0-006312e80a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.526 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.527 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.527 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.528 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.528 DEBUG:boto:Method: GET 2022-09-03T05:37:14.538 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.539 DEBUG:boto:Data: 2022-09-03T05:37:14.539 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.540 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:14.541 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.541 DEBUG:boto:Params: {} 2022-09-03T05:37:14.542 DEBUG:boto:Token: None 2022-09-03T05:37:14.543 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.543 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TZDqBWkhdhgy5Mg5GvdX85+GBMQ= 2022-09-03T05:37:14.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TZDqBWkhdhgy5Mg5GvdX85+GBMQ=', 'Content-Length': '0'} 2022-09-03T05:37:14.546 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e8a21c16b33f62c2-006312e80a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.548 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:14.549 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:14.550 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.551 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.551 DEBUG:boto:Method: GET 2022-09-03T05:37:14.552 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.553 DEBUG:boto:Data: 2022-09-03T05:37:14.553 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.558 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:14.559 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.559 DEBUG:boto:Params: {} 2022-09-03T05:37:14.560 DEBUG:boto:Token: None 2022-09-03T05:37:14.561 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/ 2022-09-03T05:37:14.561 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI= 2022-09-03T05:37:14.562 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI=', 'Content-Length': '0'} 2022-09-03T05:37:14.566 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004434bdb9b45610ad-006312e80a-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.567 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:14.568 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:14.569 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:14.570 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.570 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.571 DEBUG:boto:Method: GET 2022-09-03T05:37:14.572 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.572 DEBUG:boto:Data: 2022-09-03T05:37:14.573 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.573 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:14.574 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.576 DEBUG:boto:Params: {} 2022-09-03T05:37:14.577 DEBUG:boto:Token: None 2022-09-03T05:37:14.579 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/ 2022-09-03T05:37:14.587 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI= 2022-09-03T05:37:14.588 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI=', 'Content-Length': '0'} 2022-09-03T05:37:14.602 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004a0ad274f0d4075d-006312e80a-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.604 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:14.605 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:14.606 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.607 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.607 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.608 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.608 DEBUG:boto:Method: GET 2022-09-03T05:37:14.609 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.610 DEBUG:boto:Data: 2022-09-03T05:37:14.610 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.611 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:14.611 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.612 DEBUG:boto:Params: {} 2022-09-03T05:37:14.612 DEBUG:boto:Token: None 2022-09-03T05:37:14.613 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.614 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MpWiYks+SEj4YzIgsVkrUmyxU04= 2022-09-03T05:37:14.614 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MpWiYks+SEj4YzIgsVkrUmyxU04=', 'Content-Length': '0'} 2022-09-03T05:37:14.617 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001ddfe4be34c5fadc-006312e80a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.617 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.618 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.619 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.619 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.620 DEBUG:boto:Method: GET 2022-09-03T05:37:14.620 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.621 DEBUG:boto:Data: 2022-09-03T05:37:14.621 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.622 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:14.622 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.623 DEBUG:boto:Params: {} 2022-09-03T05:37:14.623 DEBUG:boto:Token: None 2022-09-03T05:37:14.624 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.625 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MpWiYks+SEj4YzIgsVkrUmyxU04= 2022-09-03T05:37:14.625 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MpWiYks+SEj4YzIgsVkrUmyxU04=', 'Content-Length': '0'} 2022-09-03T05:37:14.627 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000082c2ce147bc66118-006312e80a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.670 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.671 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.671 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.672 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.672 DEBUG:boto:Method: HEAD 2022-09-03T05:37:14.673 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.673 DEBUG:boto:Data: 2022-09-03T05:37:14.674 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.674 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:14.675 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.675 DEBUG:boto:Params: {} 2022-09-03T05:37:14.676 DEBUG:boto:Token: None 2022-09-03T05:37:14.677 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.677 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E7CNV9oun6+k4ku8n3ikWFs84zw= 2022-09-03T05:37:14.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E7CNV9oun6+k4ku8n3ikWFs84zw=', 'Content-Length': '0'} 2022-09-03T05:37:14.686 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000eb151701bee22e33-006312e80a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.687 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.688 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.688 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.688 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.689 DEBUG:boto:Method: HEAD 2022-09-03T05:37:14.689 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.690 DEBUG:boto:Data: 2022-09-03T05:37:14.690 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.691 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:14.691 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.692 DEBUG:boto:Params: {} 2022-09-03T05:37:14.693 DEBUG:boto:Token: None 2022-09-03T05:37:14.693 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.694 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E7CNV9oun6+k4ku8n3ikWFs84zw= 2022-09-03T05:37:14.695 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E7CNV9oun6+k4ku8n3ikWFs84zw=', 'Content-Length': '0'} 2022-09-03T05:37:14.697 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003d46fedf0af2cd42-006312e80a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.698 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:14.699 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.699 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.700 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.700 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.701 DEBUG:boto:Method: GET 2022-09-03T05:37:14.702 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.703 DEBUG:boto:Data: 2022-09-03T05:37:14.704 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.704 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:14.705 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.705 DEBUG:boto:Params: {} 2022-09-03T05:37:14.706 DEBUG:boto:Token: None 2022-09-03T05:37:14.707 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.708 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MpWiYks+SEj4YzIgsVkrUmyxU04= 2022-09-03T05:37:14.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MpWiYks+SEj4YzIgsVkrUmyxU04=', 'Content-Length': '0'} 2022-09-03T05:37:14.711 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ce3ddf3b47215dbd-006312e80a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.712 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.712 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.713 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.714 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.714 DEBUG:boto:Method: GET 2022-09-03T05:37:14.715 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.715 DEBUG:boto:Data: 2022-09-03T05:37:14.716 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.716 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:14.717 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.717 DEBUG:boto:Params: {} 2022-09-03T05:37:14.718 DEBUG:boto:Token: None 2022-09-03T05:37:14.719 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:14.720 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MpWiYks+SEj4YzIgsVkrUmyxU04= 2022-09-03T05:37:14.721 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MpWiYks+SEj4YzIgsVkrUmyxU04=', 'Content-Length': '0'} 2022-09-03T05:37:14.723 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000098c88f354b442e39-006312e80a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.724 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:14.725 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:14.725 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.726 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.726 DEBUG:boto:Method: GET 2022-09-03T05:37:14.727 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.728 DEBUG:boto:Data: 2022-09-03T05:37:14.728 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.729 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:14.729 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.730 DEBUG:boto:Params: {} 2022-09-03T05:37:14.730 DEBUG:boto:Token: None 2022-09-03T05:37:14.731 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/ 2022-09-03T05:37:14.732 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI= 2022-09-03T05:37:14.733 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI=', 'Content-Length': '0'} 2022-09-03T05:37:14.740 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000313ab438c531f945-006312e80a-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.743 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:14.747 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:14.748 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:14.749 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.749 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.750 DEBUG:boto:Method: GET 2022-09-03T05:37:14.750 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.751 DEBUG:boto:Data: 2022-09-03T05:37:14.752 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.752 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:14.753 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.753 DEBUG:boto:Params: {} 2022-09-03T05:37:14.754 DEBUG:boto:Token: None 2022-09-03T05:37:14.755 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/ 2022-09-03T05:37:14.755 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI= 2022-09-03T05:37:14.756 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI=', 'Content-Length': '0'} 2022-09-03T05:37:14.760 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000762fe23e2a9f230a-006312e80a-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.762 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:14.764 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.765 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.766 DEBUG:boto:Method: HEAD 2022-09-03T05:37:14.767 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:14.767 DEBUG:boto:Data: 2022-09-03T05:37:14.768 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.769 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:14.770 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.772 DEBUG:boto:Params: {} 2022-09-03T05:37:14.773 DEBUG:boto:Token: None 2022-09-03T05:37:14.774 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj 2022-09-03T05:37:14.775 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eTVKf4Y5YXm4aXrxETQjqe2rQfQ= 2022-09-03T05:37:14.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eTVKf4Y5YXm4aXrxETQjqe2rQfQ=', 'Content-Length': '0'} 2022-09-03T05:37:14.782 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fe54c46ae5075ef1-006312e80a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.783 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.784 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.785 DEBUG:boto:Method: HEAD 2022-09-03T05:37:14.786 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:14.787 DEBUG:boto:Data: 2022-09-03T05:37:14.788 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.789 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:14.790 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.791 DEBUG:boto:Params: {} 2022-09-03T05:37:14.792 DEBUG:boto:Token: None 2022-09-03T05:37:14.793 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj 2022-09-03T05:37:14.793 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eTVKf4Y5YXm4aXrxETQjqe2rQfQ= 2022-09-03T05:37:14.794 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eTVKf4Y5YXm4aXrxETQjqe2rQfQ=', 'Content-Length': '0'} 2022-09-03T05:37:14.808 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004a655316dad01c8e-006312e80a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.809 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:14.810 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.810 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.811 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.811 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.812 DEBUG:boto:Method: GET 2022-09-03T05:37:14.813 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.813 DEBUG:boto:Data: 2022-09-03T05:37:14.814 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.814 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:14.815 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.815 DEBUG:boto:Params: {} 2022-09-03T05:37:14.816 DEBUG:boto:Token: None 2022-09-03T05:37:14.817 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.818 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TZDqBWkhdhgy5Mg5GvdX85+GBMQ= 2022-09-03T05:37:14.818 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TZDqBWkhdhgy5Mg5GvdX85+GBMQ=', 'Content-Length': '0'} 2022-09-03T05:37:14.821 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a66f68e91ee64642-006312e80a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.863 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.864 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.865 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.865 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.866 DEBUG:boto:Method: GET 2022-09-03T05:37:14.866 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.867 DEBUG:boto:Data: 2022-09-03T05:37:14.867 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.869 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:14.869 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.870 DEBUG:boto:Params: {} 2022-09-03T05:37:14.871 DEBUG:boto:Token: None 2022-09-03T05:37:14.873 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:14.874 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TZDqBWkhdhgy5Mg5GvdX85+GBMQ= 2022-09-03T05:37:14.875 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TZDqBWkhdhgy5Mg5GvdX85+GBMQ=', 'Content-Length': '0'} 2022-09-03T05:37:14.879 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bcff6fead81e69fb-006312e80a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.880 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:14.881 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:14.882 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.884 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.885 DEBUG:boto:Method: GET 2022-09-03T05:37:14.885 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.886 DEBUG:boto:Data: 2022-09-03T05:37:14.887 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.887 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:14.888 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.888 DEBUG:boto:Params: {} 2022-09-03T05:37:14.889 DEBUG:boto:Token: None 2022-09-03T05:37:14.889 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/ 2022-09-03T05:37:14.890 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI= 2022-09-03T05:37:14.891 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI=', 'Content-Length': '0'} 2022-09-03T05:37:14.900 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d0161ec4f82719e0-006312e80a-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.901 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:14.903 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:14.904 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:14.905 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.906 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.909 DEBUG:boto:Method: GET 2022-09-03T05:37:14.910 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:14.911 DEBUG:boto:Data: 2022-09-03T05:37:14.912 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.913 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:14.914 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.914 DEBUG:boto:Params: {} 2022-09-03T05:37:14.915 DEBUG:boto:Token: None 2022-09-03T05:37:14.916 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/ 2022-09-03T05:37:14.917 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI= 2022-09-03T05:37:14.918 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FWI78OZmzH5UNn3TLLy6+B7tiSI=', 'Content-Length': '0'} 2022-09-03T05:37:14.922 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000027901975acaf2857-006312e80a-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.923 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:14.924 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:14.924 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.925 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.925 DEBUG:boto:path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:14.926 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:14.927 DEBUG:boto:Method: GET 2022-09-03T05:37:14.928 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:14.929 DEBUG:boto:Data: 2022-09-03T05:37:14.930 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.931 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:14.932 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.932 DEBUG:boto:Params: {} 2022-09-03T05:37:14.934 DEBUG:boto:Token: None 2022-09-03T05:37:14.935 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:14.936 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VZlP23PgtgQtJ0IC6ly/LufoNPc= 2022-09-03T05:37:14.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VZlP23PgtgQtJ0IC6ly/LufoNPc=', 'Content-Length': '0'} 2022-09-03T05:37:14.940 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'vauOlKlWVHmDxq4ZzMBF97KP66F89Pc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007ec7fd2bfc5e7d01-006312e80a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.940 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.941 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.942 DEBUG:boto:path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:14.942 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:14.943 DEBUG:boto:Method: GET 2022-09-03T05:37:14.943 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:14.944 DEBUG:boto:Data: 2022-09-03T05:37:14.944 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.945 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:14.945 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.946 DEBUG:boto:Params: {} 2022-09-03T05:37:14.946 DEBUG:boto:Token: None 2022-09-03T05:37:14.947 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:14.948 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VZlP23PgtgQtJ0IC6ly/LufoNPc= 2022-09-03T05:37:14.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VZlP23PgtgQtJ0IC6ly/LufoNPc=', 'Content-Length': '0'} 2022-09-03T05:37:14.956 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'vauOlKlWVHmDxq4ZzMBF97KP66F89Pc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a2dff5c82f851811-006312e80a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.957 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.958 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.959 DEBUG:boto:path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:14.959 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:14.960 DEBUG:boto:Method: HEAD 2022-09-03T05:37:14.961 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:14.961 DEBUG:boto:Data: 2022-09-03T05:37:14.962 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.963 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:14.963 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.964 DEBUG:boto:Params: {} 2022-09-03T05:37:14.964 DEBUG:boto:Token: None 2022-09-03T05:37:14.965 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:14.966 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WE8OibSYhM9Oxe7+MF9Ld9Dfj7I= 2022-09-03T05:37:14.967 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WE8OibSYhM9Oxe7+MF9Ld9Dfj7I=', 'Content-Length': '0'} 2022-09-03T05:37:14.969 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'vauOlKlWVHmDxq4ZzMBF97KP66F89Pc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004771d54b670e9a34-006312e80a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.970 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.971 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.972 DEBUG:boto:path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:14.973 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:14.974 DEBUG:boto:Method: HEAD 2022-09-03T05:37:14.975 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:14.976 DEBUG:boto:Data: 2022-09-03T05:37:14.978 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.979 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:14.979 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.980 DEBUG:boto:Params: {} 2022-09-03T05:37:14.982 DEBUG:boto:Token: None 2022-09-03T05:37:14.983 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:14 GMT /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:14.984 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WE8OibSYhM9Oxe7+MF9Ld9Dfj7I= 2022-09-03T05:37:14.986 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WE8OibSYhM9Oxe7+MF9Ld9Dfj7I=', 'Content-Length': '0'} 2022-09-03T05:37:14.988 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'vauOlKlWVHmDxq4ZzMBF97KP66F89Pc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c06a8e45259dc4cd-006312e80a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:14.989 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:14.990 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:14.991 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:14.992 DEBUG:boto:path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:14.993 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:14.994 DEBUG:boto:Method: GET 2022-09-03T05:37:14.995 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:14.996 DEBUG:boto:Data: 2022-09-03T05:37:14.997 DEBUG:boto:Headers: {} 2022-09-03T05:37:14.997 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:14.998 DEBUG:boto:Port: 8000 2022-09-03T05:37:14.999 DEBUG:boto:Params: {} 2022-09-03T05:37:15.001 DEBUG:boto:Token: None 2022-09-03T05:37:15.002 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:15.003 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AgQAi6QcO3mzt0TwfQ0OO1vEna8= 2022-09-03T05:37:15.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AgQAi6QcO3mzt0TwfQ0OO1vEna8=', 'Content-Length': '0'} 2022-09-03T05:37:15.009 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'vauOlKlWVHmDxq4ZzMBF97KP66F89Pc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000009086028e5a111c4-006312e80b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.014 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.015 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.016 DEBUG:boto:path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:15.017 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:15.018 DEBUG:boto:Method: GET 2022-09-03T05:37:15.019 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:15.020 DEBUG:boto:Data: 2022-09-03T05:37:15.021 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.022 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:15.023 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.024 DEBUG:boto:Params: {} 2022-09-03T05:37:15.025 DEBUG:boto:Token: None 2022-09-03T05:37:15.026 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:15.027 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AgQAi6QcO3mzt0TwfQ0OO1vEna8= 2022-09-03T05:37:15.028 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AgQAi6QcO3mzt0TwfQ0OO1vEna8=', 'Content-Length': '0'} 2022-09-03T05:37:15.037 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'vauOlKlWVHmDxq4ZzMBF97KP66F89Pc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000613d8cfb9af11f2f-006312e80b-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.039 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:15.040 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:15.042 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.043 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.044 DEBUG:boto:Method: GET 2022-09-03T05:37:15.044 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.045 DEBUG:boto:Data: 2022-09-03T05:37:15.046 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.046 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:15.047 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.047 DEBUG:boto:Params: {} 2022-09-03T05:37:15.048 DEBUG:boto:Token: None 2022-09-03T05:37:15.049 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/ 2022-09-03T05:37:15.049 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA= 2022-09-03T05:37:15.050 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA=', 'Content-Length': '0'} 2022-09-03T05:37:15.056 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003766df7e281b1231-006312e80b-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.057 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:15.058 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:15.059 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:15.059 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.064 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.065 DEBUG:boto:Method: GET 2022-09-03T05:37:15.065 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.066 DEBUG:boto:Data: 2022-09-03T05:37:15.067 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.068 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:15.069 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.070 DEBUG:boto:Params: {} 2022-09-03T05:37:15.071 DEBUG:boto:Token: None 2022-09-03T05:37:15.072 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/ 2022-09-03T05:37:15.073 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA= 2022-09-03T05:37:15.074 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA=', 'Content-Length': '0'} 2022-09-03T05:37:15.078 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005dcb1a4d0b2ca9b5-006312e80b-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.079 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:15.082 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.083 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.085 DEBUG:boto:Method: HEAD 2022-09-03T05:37:15.086 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:15.087 DEBUG:boto:Data: 2022-09-03T05:37:15.088 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.089 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:15.090 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.092 DEBUG:boto:Params: {} 2022-09-03T05:37:15.093 DEBUG:boto:Token: None 2022-09-03T05:37:15.094 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj 2022-09-03T05:37:15.095 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kClmXDbin3uuBBpz8LC2Fa7lyIM= 2022-09-03T05:37:15.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kClmXDbin3uuBBpz8LC2Fa7lyIM=', 'Content-Length': '0'} 2022-09-03T05:37:15.100 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005a515abd3cf49684-006312e80b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.101 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.102 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.103 DEBUG:boto:Method: HEAD 2022-09-03T05:37:15.104 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:15.104 DEBUG:boto:Data: 2022-09-03T05:37:15.105 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.105 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:15.106 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.106 DEBUG:boto:Params: {} 2022-09-03T05:37:15.108 DEBUG:boto:Token: None 2022-09-03T05:37:15.108 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj 2022-09-03T05:37:15.109 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kClmXDbin3uuBBpz8LC2Fa7lyIM= 2022-09-03T05:37:15.110 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kClmXDbin3uuBBpz8LC2Fa7lyIM=', 'Content-Length': '0'} 2022-09-03T05:37:15.114 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000c0d55c6cebface3-006312e80b-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.115 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:15.115 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.116 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.116 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.117 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.117 DEBUG:boto:Method: GET 2022-09-03T05:37:15.118 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.118 DEBUG:boto:Data: 2022-09-03T05:37:15.119 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.120 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:15.120 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.121 DEBUG:boto:Params: {} 2022-09-03T05:37:15.121 DEBUG:boto:Token: None 2022-09-03T05:37:15.125 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.126 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5k+aZ/tXeBLBTBsWTXv+n0nVdO4= 2022-09-03T05:37:15.126 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5k+aZ/tXeBLBTBsWTXv+n0nVdO4=', 'Content-Length': '0'} 2022-09-03T05:37:15.128 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a27eb6f3d04230c0-006312e80b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.134 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.137 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.138 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.138 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.139 DEBUG:boto:Method: GET 2022-09-03T05:37:15.140 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.140 DEBUG:boto:Data: 2022-09-03T05:37:15.141 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.141 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:15.142 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.143 DEBUG:boto:Params: {} 2022-09-03T05:37:15.144 DEBUG:boto:Token: None 2022-09-03T05:37:15.144 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.145 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5k+aZ/tXeBLBTBsWTXv+n0nVdO4= 2022-09-03T05:37:15.147 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5k+aZ/tXeBLBTBsWTXv+n0nVdO4=', 'Content-Length': '0'} 2022-09-03T05:37:15.149 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001be400673c1d72ad-006312e80b-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.150 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:15.151 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:15.152 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.152 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.153 DEBUG:boto:Method: GET 2022-09-03T05:37:15.153 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.154 DEBUG:boto:Data: 2022-09-03T05:37:15.155 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.155 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:15.156 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.157 DEBUG:boto:Params: {} 2022-09-03T05:37:15.158 DEBUG:boto:Token: None 2022-09-03T05:37:15.159 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/ 2022-09-03T05:37:15.161 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA= 2022-09-03T05:37:15.161 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA=', 'Content-Length': '0'} 2022-09-03T05:37:15.165 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000050999758e4d9a3ff-006312e80b-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.167 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:15.168 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:15.170 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:15.170 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.171 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.172 DEBUG:boto:Method: GET 2022-09-03T05:37:15.172 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.173 DEBUG:boto:Data: 2022-09-03T05:37:15.174 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.174 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:15.175 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.175 DEBUG:boto:Params: {} 2022-09-03T05:37:15.176 DEBUG:boto:Token: None 2022-09-03T05:37:15.177 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/ 2022-09-03T05:37:15.178 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA= 2022-09-03T05:37:15.178 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA=', 'Content-Length': '0'} 2022-09-03T05:37:15.190 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000269fa0275918eb2f-006312e80b-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.192 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:15.193 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:15.194 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.195 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.196 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.197 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.198 DEBUG:boto:Method: GET 2022-09-03T05:37:15.198 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.199 DEBUG:boto:Data: 2022-09-03T05:37:15.200 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.201 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:15.202 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.203 DEBUG:boto:Params: {} 2022-09-03T05:37:15.204 DEBUG:boto:Token: None 2022-09-03T05:37:15.205 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.207 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NpFE3qxCm8jeWe/JOJ34j31of2A= 2022-09-03T05:37:15.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NpFE3qxCm8jeWe/JOJ34j31of2A=', 'Content-Length': '0'} 2022-09-03T05:37:15.212 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006953780da89150ae-006312e80b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.214 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.215 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.216 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.217 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.218 DEBUG:boto:Method: GET 2022-09-03T05:37:15.219 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.220 DEBUG:boto:Data: 2022-09-03T05:37:15.223 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.225 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:15.237 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.239 DEBUG:boto:Params: {} 2022-09-03T05:37:15.240 DEBUG:boto:Token: None 2022-09-03T05:37:15.241 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.242 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NpFE3qxCm8jeWe/JOJ34j31of2A= 2022-09-03T05:37:15.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NpFE3qxCm8jeWe/JOJ34j31of2A=', 'Content-Length': '0'} 2022-09-03T05:37:15.250 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dec71c04c8e81d72-006312e80b-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.251 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.252 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.253 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.254 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.255 DEBUG:boto:Method: HEAD 2022-09-03T05:37:15.256 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.257 DEBUG:boto:Data: 2022-09-03T05:37:15.258 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.260 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:15.261 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.262 DEBUG:boto:Params: {} 2022-09-03T05:37:15.263 DEBUG:boto:Token: None 2022-09-03T05:37:15.264 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.265 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mkLlT5oxLiIngMXLUqFJqGYxKT0= 2022-09-03T05:37:15.266 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mkLlT5oxLiIngMXLUqFJqGYxKT0=', 'Content-Length': '0'} 2022-09-03T05:37:15.269 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000876eef12ea6a2af4-006312e80b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.270 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.271 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.272 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.272 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.273 DEBUG:boto:Method: HEAD 2022-09-03T05:37:15.274 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.274 DEBUG:boto:Data: 2022-09-03T05:37:15.275 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.275 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:15.276 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.276 DEBUG:boto:Params: {} 2022-09-03T05:37:15.277 DEBUG:boto:Token: None 2022-09-03T05:37:15.277 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.283 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mkLlT5oxLiIngMXLUqFJqGYxKT0= 2022-09-03T05:37:15.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mkLlT5oxLiIngMXLUqFJqGYxKT0=', 'Content-Length': '0'} 2022-09-03T05:37:15.287 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e82df70822b6409a-006312e80b-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.289 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:15.290 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.291 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.296 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.297 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.299 DEBUG:boto:Method: GET 2022-09-03T05:37:15.300 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.300 DEBUG:boto:Data: 2022-09-03T05:37:15.301 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.302 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:15.302 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.303 DEBUG:boto:Params: {} 2022-09-03T05:37:15.304 DEBUG:boto:Token: None 2022-09-03T05:37:15.305 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.305 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NpFE3qxCm8jeWe/JOJ34j31of2A= 2022-09-03T05:37:15.306 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NpFE3qxCm8jeWe/JOJ34j31of2A=', 'Content-Length': '0'} 2022-09-03T05:37:15.309 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fe93223d6111358a-006312e80b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.310 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.311 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.312 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.313 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.314 DEBUG:boto:Method: GET 2022-09-03T05:37:15.315 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.316 DEBUG:boto:Data: 2022-09-03T05:37:15.316 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.318 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:15.319 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.320 DEBUG:boto:Params: {} 2022-09-03T05:37:15.321 DEBUG:boto:Token: None 2022-09-03T05:37:15.322 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:15.323 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NpFE3qxCm8jeWe/JOJ34j31of2A= 2022-09-03T05:37:15.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NpFE3qxCm8jeWe/JOJ34j31of2A=', 'Content-Length': '0'} 2022-09-03T05:37:15.327 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000086a99b62378c65ff-006312e80b-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.328 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:15.329 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:15.329 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.330 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.330 DEBUG:boto:Method: GET 2022-09-03T05:37:15.331 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.331 DEBUG:boto:Data: 2022-09-03T05:37:15.332 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.332 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:15.334 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.334 DEBUG:boto:Params: {} 2022-09-03T05:37:15.336 DEBUG:boto:Token: None 2022-09-03T05:37:15.336 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/ 2022-09-03T05:37:15.338 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA= 2022-09-03T05:37:15.338 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA=', 'Content-Length': '0'} 2022-09-03T05:37:15.342 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c6f72d037f7f2a16-006312e80b-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.343 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:15.345 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:15.347 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:15.348 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.349 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.350 DEBUG:boto:Method: GET 2022-09-03T05:37:15.351 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.352 DEBUG:boto:Data: 2022-09-03T05:37:15.353 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.354 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:15.355 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.356 DEBUG:boto:Params: {} 2022-09-03T05:37:15.357 DEBUG:boto:Token: None 2022-09-03T05:37:15.358 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/ 2022-09-03T05:37:15.360 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA= 2022-09-03T05:37:15.361 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA=', 'Content-Length': '0'} 2022-09-03T05:37:15.365 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000111b7feb8468cfc5-006312e80b-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.366 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:15.368 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.370 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.370 DEBUG:boto:Method: HEAD 2022-09-03T05:37:15.371 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:15.371 DEBUG:boto:Data: 2022-09-03T05:37:15.372 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.372 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:15.373 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.373 DEBUG:boto:Params: {} 2022-09-03T05:37:15.374 DEBUG:boto:Token: None 2022-09-03T05:37:15.375 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj 2022-09-03T05:37:15.376 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kClmXDbin3uuBBpz8LC2Fa7lyIM= 2022-09-03T05:37:15.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kClmXDbin3uuBBpz8LC2Fa7lyIM=', 'Content-Length': '0'} 2022-09-03T05:37:15.381 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000601462c734720d51-006312e80b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.382 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.383 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.384 DEBUG:boto:Method: HEAD 2022-09-03T05:37:15.384 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:15.385 DEBUG:boto:Data: 2022-09-03T05:37:15.386 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.386 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:15.387 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.387 DEBUG:boto:Params: {} 2022-09-03T05:37:15.388 DEBUG:boto:Token: None 2022-09-03T05:37:15.389 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj 2022-09-03T05:37:15.390 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kClmXDbin3uuBBpz8LC2Fa7lyIM= 2022-09-03T05:37:15.391 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kClmXDbin3uuBBpz8LC2Fa7lyIM=', 'Content-Length': '0'} 2022-09-03T05:37:15.396 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002bbb6dca0d5cca8b-006312e80b-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.397 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:15.398 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.398 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.399 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.399 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.400 DEBUG:boto:Method: GET 2022-09-03T05:37:15.400 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.401 DEBUG:boto:Data: 2022-09-03T05:37:15.401 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.402 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:15.403 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.403 DEBUG:boto:Params: {} 2022-09-03T05:37:15.404 DEBUG:boto:Token: None 2022-09-03T05:37:15.406 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.407 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5k+aZ/tXeBLBTBsWTXv+n0nVdO4= 2022-09-03T05:37:15.408 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5k+aZ/tXeBLBTBsWTXv+n0nVdO4=', 'Content-Length': '0'} 2022-09-03T05:37:15.410 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f66648461090851c-006312e80b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.454 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.455 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.456 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.456 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.457 DEBUG:boto:Method: GET 2022-09-03T05:37:15.457 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.458 DEBUG:boto:Data: 2022-09-03T05:37:15.458 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.459 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:15.460 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.461 DEBUG:boto:Params: {} 2022-09-03T05:37:15.462 DEBUG:boto:Token: None 2022-09-03T05:37:15.463 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.463 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5k+aZ/tXeBLBTBsWTXv+n0nVdO4= 2022-09-03T05:37:15.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5k+aZ/tXeBLBTBsWTXv+n0nVdO4=', 'Content-Length': '0'} 2022-09-03T05:37:15.466 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003a9c0dd34c2ff90d-006312e80b-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.467 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:15.468 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:15.469 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.469 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.470 DEBUG:boto:Method: GET 2022-09-03T05:37:15.470 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.471 DEBUG:boto:Data: 2022-09-03T05:37:15.472 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.472 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:15.474 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.474 DEBUG:boto:Params: {} 2022-09-03T05:37:15.475 DEBUG:boto:Token: None 2022-09-03T05:37:15.475 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/ 2022-09-03T05:37:15.476 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA= 2022-09-03T05:37:15.477 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA=', 'Content-Length': '0'} 2022-09-03T05:37:15.480 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000080022bf91d236fd2-006312e80b-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.481 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:15.482 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:15.483 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:15.484 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.484 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.485 DEBUG:boto:Method: GET 2022-09-03T05:37:15.485 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.486 DEBUG:boto:Data: 2022-09-03T05:37:15.487 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.489 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:15.490 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.491 DEBUG:boto:Params: {} 2022-09-03T05:37:15.492 DEBUG:boto:Token: None 2022-09-03T05:37:15.493 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/ 2022-09-03T05:37:15.494 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA= 2022-09-03T05:37:15.496 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA=', 'Content-Length': '0'} 2022-09-03T05:37:15.501 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002af4de97b8b08da6-006312e80b-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.502 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:15.503 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:15.504 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.504 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.505 DEBUG:boto:path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.505 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.506 DEBUG:boto:Method: GET 2022-09-03T05:37:15.507 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.507 DEBUG:boto:Data: 2022-09-03T05:37:15.508 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.509 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:15.509 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.510 DEBUG:boto:Params: {} 2022-09-03T05:37:15.511 DEBUG:boto:Token: None 2022-09-03T05:37:15.512 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.512 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oHvStl8ulUYSWfDKkt94WzftTns= 2022-09-03T05:37:15.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oHvStl8ulUYSWfDKkt94WzftTns=', 'Content-Length': '0'} 2022-09-03T05:37:15.516 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'oYY-icmxg74gZ4carPTk38at9fTxjJi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c861b9e26e620100-006312e80b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.517 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.518 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.518 DEBUG:boto:path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.519 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.519 DEBUG:boto:Method: GET 2022-09-03T05:37:15.520 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.520 DEBUG:boto:Data: 2022-09-03T05:37:15.521 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.521 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:15.522 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.523 DEBUG:boto:Params: {} 2022-09-03T05:37:15.523 DEBUG:boto:Token: None 2022-09-03T05:37:15.524 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.525 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oHvStl8ulUYSWfDKkt94WzftTns= 2022-09-03T05:37:15.525 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oHvStl8ulUYSWfDKkt94WzftTns=', 'Content-Length': '0'} 2022-09-03T05:37:15.528 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'oYY-icmxg74gZ4carPTk38at9fTxjJi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000082a208b9ad4fc3a8-006312e80b-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.570 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.572 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.574 DEBUG:boto:path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.575 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.575 DEBUG:boto:Method: HEAD 2022-09-03T05:37:15.576 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.577 DEBUG:boto:Data: 2022-09-03T05:37:15.579 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.580 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:15.581 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.582 DEBUG:boto:Params: {} 2022-09-03T05:37:15.584 DEBUG:boto:Token: None 2022-09-03T05:37:15.585 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.586 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6SWt0LER2PUnLKt/bdYUOjOlRXc= 2022-09-03T05:37:15.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6SWt0LER2PUnLKt/bdYUOjOlRXc=', 'Content-Length': '0'} 2022-09-03T05:37:15.589 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'oYY-icmxg74gZ4carPTk38at9fTxjJi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e7ea935e933bb4d6-006312e80b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.591 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.592 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.593 DEBUG:boto:path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.593 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.594 DEBUG:boto:Method: HEAD 2022-09-03T05:37:15.595 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.596 DEBUG:boto:Data: 2022-09-03T05:37:15.597 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.598 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:15.599 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.600 DEBUG:boto:Params: {} 2022-09-03T05:37:15.602 DEBUG:boto:Token: None 2022-09-03T05:37:15.605 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.605 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6SWt0LER2PUnLKt/bdYUOjOlRXc= 2022-09-03T05:37:15.607 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6SWt0LER2PUnLKt/bdYUOjOlRXc=', 'Content-Length': '0'} 2022-09-03T05:37:15.609 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'oYY-icmxg74gZ4carPTk38at9fTxjJi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000027370e10821d2dca-006312e80b-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.611 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:15.612 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.613 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.614 DEBUG:boto:path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.615 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.616 DEBUG:boto:Method: GET 2022-09-03T05:37:15.617 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.618 DEBUG:boto:Data: 2022-09-03T05:37:15.619 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.620 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:15.621 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.622 DEBUG:boto:Params: {} 2022-09-03T05:37:15.623 DEBUG:boto:Token: None 2022-09-03T05:37:15.624 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.625 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oHvStl8ulUYSWfDKkt94WzftTns= 2022-09-03T05:37:15.626 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oHvStl8ulUYSWfDKkt94WzftTns=', 'Content-Length': '0'} 2022-09-03T05:37:15.630 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'oYY-icmxg74gZ4carPTk38at9fTxjJi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000032eed78ef15933e0-006312e80b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.632 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.633 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.633 DEBUG:boto:path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.634 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.635 DEBUG:boto:Method: GET 2022-09-03T05:37:15.635 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.636 DEBUG:boto:Data: 2022-09-03T05:37:15.637 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.638 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:15.638 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.639 DEBUG:boto:Params: {} 2022-09-03T05:37:15.639 DEBUG:boto:Token: None 2022-09-03T05:37:15.640 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:15.642 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oHvStl8ulUYSWfDKkt94WzftTns= 2022-09-03T05:37:15.645 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oHvStl8ulUYSWfDKkt94WzftTns=', 'Content-Length': '0'} 2022-09-03T05:37:15.648 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'oYY-icmxg74gZ4carPTk38at9fTxjJi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dce85636bf4a83fb-006312e80b-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.650 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:15.651 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:15.651 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.652 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.653 DEBUG:boto:Method: GET 2022-09-03T05:37:15.656 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.657 DEBUG:boto:Data: 2022-09-03T05:37:15.658 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.659 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:15.660 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.661 DEBUG:boto:Params: {} 2022-09-03T05:37:15.662 DEBUG:boto:Token: None 2022-09-03T05:37:15.663 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/ 2022-09-03T05:37:15.665 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA= 2022-09-03T05:37:15.667 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA=', 'Content-Length': '0'} 2022-09-03T05:37:15.675 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000debd417ebb3c60ac-006312e80b-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.678 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:15.681 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:15.682 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:15.683 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.683 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.684 DEBUG:boto:Method: GET 2022-09-03T05:37:15.684 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.685 DEBUG:boto:Data: 2022-09-03T05:37:15.685 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.686 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:15.688 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.689 DEBUG:boto:Params: {} 2022-09-03T05:37:15.690 DEBUG:boto:Token: None 2022-09-03T05:37:15.691 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/ 2022-09-03T05:37:15.692 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA= 2022-09-03T05:37:15.694 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA=', 'Content-Length': '0'} 2022-09-03T05:37:15.698 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b7f8c64efb6689bb-006312e80b-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.712 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:15.714 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.716 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.717 DEBUG:boto:Method: HEAD 2022-09-03T05:37:15.724 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:15.724 DEBUG:boto:Data: 2022-09-03T05:37:15.725 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.725 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:15.726 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.727 DEBUG:boto:Params: {} 2022-09-03T05:37:15.735 DEBUG:boto:Token: None 2022-09-03T05:37:15.736 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj 2022-09-03T05:37:15.737 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kClmXDbin3uuBBpz8LC2Fa7lyIM= 2022-09-03T05:37:15.737 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kClmXDbin3uuBBpz8LC2Fa7lyIM=', 'Content-Length': '0'} 2022-09-03T05:37:15.744 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000197b0eab56145bdc-006312e80b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.745 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.746 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.747 DEBUG:boto:Method: HEAD 2022-09-03T05:37:15.747 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:15.748 DEBUG:boto:Data: 2022-09-03T05:37:15.749 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.750 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:15.751 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.751 DEBUG:boto:Params: {} 2022-09-03T05:37:15.752 DEBUG:boto:Token: None 2022-09-03T05:37:15.752 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj 2022-09-03T05:37:15.753 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kClmXDbin3uuBBpz8LC2Fa7lyIM= 2022-09-03T05:37:15.754 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kClmXDbin3uuBBpz8LC2Fa7lyIM=', 'Content-Length': '0'} 2022-09-03T05:37:15.758 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000063c1a22a6bf60cb4-006312e80b-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.759 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:15.761 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.762 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.763 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.763 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.764 DEBUG:boto:Method: GET 2022-09-03T05:37:15.765 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.766 DEBUG:boto:Data: 2022-09-03T05:37:15.768 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.769 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:15.770 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.771 DEBUG:boto:Params: {} 2022-09-03T05:37:15.772 DEBUG:boto:Token: None 2022-09-03T05:37:15.773 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.774 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5k+aZ/tXeBLBTBsWTXv+n0nVdO4= 2022-09-03T05:37:15.775 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5k+aZ/tXeBLBTBsWTXv+n0nVdO4=', 'Content-Length': '0'} 2022-09-03T05:37:15.778 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000065982f103e147b13-006312e80b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.779 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.780 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.781 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.782 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.783 DEBUG:boto:Method: GET 2022-09-03T05:37:15.785 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.795 DEBUG:boto:Data: 2022-09-03T05:37:15.796 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.797 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:15.798 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.798 DEBUG:boto:Params: {} 2022-09-03T05:37:15.799 DEBUG:boto:Token: None 2022-09-03T05:37:15.800 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:15.800 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5k+aZ/tXeBLBTBsWTXv+n0nVdO4= 2022-09-03T05:37:15.802 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5k+aZ/tXeBLBTBsWTXv+n0nVdO4=', 'Content-Length': '0'} 2022-09-03T05:37:15.805 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dad34fc962ec9db1-006312e80b-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.807 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:15.808 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:15.809 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.809 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.810 DEBUG:boto:Method: GET 2022-09-03T05:37:15.811 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.812 DEBUG:boto:Data: 2022-09-03T05:37:15.813 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.818 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:15.819 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.820 DEBUG:boto:Params: {} 2022-09-03T05:37:15.820 DEBUG:boto:Token: None 2022-09-03T05:37:15.821 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/ 2022-09-03T05:37:15.822 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA= 2022-09-03T05:37:15.822 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA=', 'Content-Length': '0'} 2022-09-03T05:37:15.825 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eac2f0509d566bb5-006312e80b-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.826 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:15.827 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:15.828 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:15.828 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.829 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.829 DEBUG:boto:Method: GET 2022-09-03T05:37:15.830 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.830 DEBUG:boto:Data: 2022-09-03T05:37:15.831 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.831 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:15.832 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.832 DEBUG:boto:Params: {} 2022-09-03T05:37:15.833 DEBUG:boto:Token: None 2022-09-03T05:37:15.834 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/ 2022-09-03T05:37:15.834 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA= 2022-09-03T05:37:15.835 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA=', 'Content-Length': '0'} 2022-09-03T05:37:15.841 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000487a5d6e912fde61-006312e80b-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.842 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:15.843 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:15.843 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.844 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.845 DEBUG:boto:path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.845 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.846 DEBUG:boto:Method: GET 2022-09-03T05:37:15.846 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.847 DEBUG:boto:Data: 2022-09-03T05:37:15.847 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.848 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:15.848 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.849 DEBUG:boto:Params: {} 2022-09-03T05:37:15.850 DEBUG:boto:Token: None 2022-09-03T05:37:15.850 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.851 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o1g3mt8KFbHPNMugI2Ez/uiCuMw= 2022-09-03T05:37:15.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o1g3mt8KFbHPNMugI2Ez/uiCuMw=', 'Content-Length': '0'} 2022-09-03T05:37:15.854 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002e92771969492cf0-006312e80b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.855 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.856 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.856 DEBUG:boto:path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.857 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.857 DEBUG:boto:Method: GET 2022-09-03T05:37:15.858 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.859 DEBUG:boto:Data: 2022-09-03T05:37:15.860 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.862 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:15.863 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.864 DEBUG:boto:Params: {} 2022-09-03T05:37:15.865 DEBUG:boto:Token: None 2022-09-03T05:37:15.865 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.866 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o1g3mt8KFbHPNMugI2Ez/uiCuMw= 2022-09-03T05:37:15.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o1g3mt8KFbHPNMugI2Ez/uiCuMw=', 'Content-Length': '0'} 2022-09-03T05:37:15.871 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009742f531fcd5dd26-006312e80b-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.872 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.872 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.873 DEBUG:boto:path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.873 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.874 DEBUG:boto:Method: HEAD 2022-09-03T05:37:15.874 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.875 DEBUG:boto:Data: 2022-09-03T05:37:15.875 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.876 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:15.876 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.877 DEBUG:boto:Params: {} 2022-09-03T05:37:15.877 DEBUG:boto:Token: None 2022-09-03T05:37:15.878 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.879 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Lq7B5IKy8Gzx2PSlVV4G13RYEyU= 2022-09-03T05:37:15.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Lq7B5IKy8Gzx2PSlVV4G13RYEyU=', 'Content-Length': '0'} 2022-09-03T05:37:15.882 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000f1096abd7e5e2d2-006312e80b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.884 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.884 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.890 DEBUG:boto:path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.895 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.896 DEBUG:boto:Method: HEAD 2022-09-03T05:37:15.896 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.897 DEBUG:boto:Data: 2022-09-03T05:37:15.897 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.898 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:15.898 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.899 DEBUG:boto:Params: {} 2022-09-03T05:37:15.900 DEBUG:boto:Token: None 2022-09-03T05:37:15.901 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.902 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Lq7B5IKy8Gzx2PSlVV4G13RYEyU= 2022-09-03T05:37:15.903 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Lq7B5IKy8Gzx2PSlVV4G13RYEyU=', 'Content-Length': '0'} 2022-09-03T05:37:15.906 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007eeb9f9042c4dbb3-006312e80b-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.908 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:15.908 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.909 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.910 DEBUG:boto:path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.912 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.913 DEBUG:boto:Method: GET 2022-09-03T05:37:15.914 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.915 DEBUG:boto:Data: 2022-09-03T05:37:15.916 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.917 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:15.918 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.918 DEBUG:boto:Params: {} 2022-09-03T05:37:15.920 DEBUG:boto:Token: None 2022-09-03T05:37:15.921 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.922 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o1g3mt8KFbHPNMugI2Ez/uiCuMw= 2022-09-03T05:37:15.924 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o1g3mt8KFbHPNMugI2Ez/uiCuMw=', 'Content-Length': '0'} 2022-09-03T05:37:15.926 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004faac25168cd385f-006312e80b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.970 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:15.971 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:15.972 DEBUG:boto:path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.972 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.973 DEBUG:boto:Method: GET 2022-09-03T05:37:15.974 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.974 DEBUG:boto:Data: 2022-09-03T05:37:15.975 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.975 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:15.981 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.981 DEBUG:boto:Params: {} 2022-09-03T05:37:15.982 DEBUG:boto:Token: None 2022-09-03T05:37:15.983 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:15.983 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o1g3mt8KFbHPNMugI2Ez/uiCuMw= 2022-09-03T05:37:15.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o1g3mt8KFbHPNMugI2Ez/uiCuMw=', 'Content-Length': '0'} 2022-09-03T05:37:15.990 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f7a3f7d9fd3e032a-006312e80b-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:15.991 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:15.991 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:15.992 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.993 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.993 DEBUG:boto:Method: GET 2022-09-03T05:37:15.994 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:15.994 DEBUG:boto:Data: 2022-09-03T05:37:15.995 DEBUG:boto:Headers: {} 2022-09-03T05:37:15.996 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:15.997 DEBUG:boto:Port: 8000 2022-09-03T05:37:15.998 DEBUG:boto:Params: {} 2022-09-03T05:37:15.999 DEBUG:boto:Token: None 2022-09-03T05:37:15.999 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:15 GMT /dsgvcy-19/ 2022-09-03T05:37:16.000 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA= 2022-09-03T05:37:16.000 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+54vwRywpCZvzVZX9X4cqwLEpFA=', 'Content-Length': '0'} 2022-09-03T05:37:16.004 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007ee7981a31c95193-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.005 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:16.006 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:16.006 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:16.007 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.007 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.008 DEBUG:boto:Method: GET 2022-09-03T05:37:16.008 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.009 DEBUG:boto:Data: 2022-09-03T05:37:16.010 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.010 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.011 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.012 DEBUG:boto:Params: {} 2022-09-03T05:37:16.013 DEBUG:boto:Token: None 2022-09-03T05:37:16.014 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/ 2022-09-03T05:37:16.015 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI= 2022-09-03T05:37:16.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI=', 'Content-Length': '0'} 2022-09-03T05:37:16.030 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009d43eb99373413ba-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.031 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:16.032 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.033 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.033 DEBUG:boto:Method: HEAD 2022-09-03T05:37:16.034 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:16.034 DEBUG:boto:Data: 2022-09-03T05:37:16.035 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.035 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.036 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.037 DEBUG:boto:Params: {} 2022-09-03T05:37:16.037 DEBUG:boto:Token: None 2022-09-03T05:37:16.038 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj 2022-09-03T05:37:16.038 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4N8E3REQprlZYJTC8yiSyg9T40I= 2022-09-03T05:37:16.039 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4N8E3REQprlZYJTC8yiSyg9T40I=', 'Content-Length': '0'} 2022-09-03T05:37:16.043 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000091c068fc1759ade3-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.045 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.046 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.047 DEBUG:boto:Method: HEAD 2022-09-03T05:37:16.049 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:16.051 DEBUG:boto:Data: 2022-09-03T05:37:16.052 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.053 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.054 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.055 DEBUG:boto:Params: {} 2022-09-03T05:37:16.056 DEBUG:boto:Token: None 2022-09-03T05:37:16.057 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj 2022-09-03T05:37:16.069 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4N8E3REQprlZYJTC8yiSyg9T40I= 2022-09-03T05:37:16.069 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4N8E3REQprlZYJTC8yiSyg9T40I=', 'Content-Length': '0'} 2022-09-03T05:37:16.073 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dfaeb2e40e0e197b-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.074 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:16.075 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.075 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.076 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.076 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.077 DEBUG:boto:Method: GET 2022-09-03T05:37:16.077 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.078 DEBUG:boto:Data: 2022-09-03T05:37:16.079 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.079 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.081 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.082 DEBUG:boto:Params: {} 2022-09-03T05:37:16.082 DEBUG:boto:Token: None 2022-09-03T05:37:16.084 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.084 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xot1jZa7b4/yUSjAvYOTjYePM2k= 2022-09-03T05:37:16.085 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xot1jZa7b4/yUSjAvYOTjYePM2k=', 'Content-Length': '0'} 2022-09-03T05:37:16.088 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c46073f21f39b3df-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.089 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.090 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.091 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.092 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.093 DEBUG:boto:Method: GET 2022-09-03T05:37:16.093 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.094 DEBUG:boto:Data: 2022-09-03T05:37:16.094 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.095 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.095 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.098 DEBUG:boto:Params: {} 2022-09-03T05:37:16.098 DEBUG:boto:Token: None 2022-09-03T05:37:16.099 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.100 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xot1jZa7b4/yUSjAvYOTjYePM2k= 2022-09-03T05:37:16.102 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xot1jZa7b4/yUSjAvYOTjYePM2k=', 'Content-Length': '0'} 2022-09-03T05:37:16.104 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006da62f61bc4a4f65-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.105 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:16.106 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:16.107 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.108 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.108 DEBUG:boto:Method: GET 2022-09-03T05:37:16.109 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.109 DEBUG:boto:Data: 2022-09-03T05:37:16.110 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.111 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.111 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.112 DEBUG:boto:Params: {} 2022-09-03T05:37:16.114 DEBUG:boto:Token: None 2022-09-03T05:37:16.115 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/ 2022-09-03T05:37:16.115 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI= 2022-09-03T05:37:16.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI=', 'Content-Length': '0'} 2022-09-03T05:37:16.125 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ba658672552840fb-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.126 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:16.127 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:16.128 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:16.129 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.129 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.130 DEBUG:boto:Method: GET 2022-09-03T05:37:16.132 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.133 DEBUG:boto:Data: 2022-09-03T05:37:16.133 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.134 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.135 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.135 DEBUG:boto:Params: {} 2022-09-03T05:37:16.136 DEBUG:boto:Token: None 2022-09-03T05:37:16.137 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/ 2022-09-03T05:37:16.137 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI= 2022-09-03T05:37:16.138 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI=', 'Content-Length': '0'} 2022-09-03T05:37:16.142 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d8da51e8670915a1-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.143 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:16.143 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:16.144 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.145 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.145 DEBUG:boto:path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.146 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.146 DEBUG:boto:Method: GET 2022-09-03T05:37:16.147 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.147 DEBUG:boto:Data: 2022-09-03T05:37:16.148 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.148 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.150 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.151 DEBUG:boto:Params: {} 2022-09-03T05:37:16.151 DEBUG:boto:Token: None 2022-09-03T05:37:16.152 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.153 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9Gson1Otnk/Mu8WJjkXylT+YkWI= 2022-09-03T05:37:16.153 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9Gson1Otnk/Mu8WJjkXylT+YkWI=', 'Content-Length': '0'} 2022-09-03T05:37:16.155 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e08be3edfe166542-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.199 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.200 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.201 DEBUG:boto:path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.207 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.208 DEBUG:boto:Method: GET 2022-09-03T05:37:16.209 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.215 DEBUG:boto:Data: 2022-09-03T05:37:16.217 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.218 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.219 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.220 DEBUG:boto:Params: {} 2022-09-03T05:37:16.221 DEBUG:boto:Token: None 2022-09-03T05:37:16.222 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.223 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9Gson1Otnk/Mu8WJjkXylT+YkWI= 2022-09-03T05:37:16.225 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9Gson1Otnk/Mu8WJjkXylT+YkWI=', 'Content-Length': '0'} 2022-09-03T05:37:16.228 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000035b878caed238614-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.229 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.231 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.231 DEBUG:boto:path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.232 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.232 DEBUG:boto:Method: HEAD 2022-09-03T05:37:16.233 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.233 DEBUG:boto:Data: 2022-09-03T05:37:16.234 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.234 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.235 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.235 DEBUG:boto:Params: {} 2022-09-03T05:37:16.236 DEBUG:boto:Token: None 2022-09-03T05:37:16.237 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.238 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vY1MFRpCeLTdzZFh9A3ed64++QI= 2022-09-03T05:37:16.238 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vY1MFRpCeLTdzZFh9A3ed64++QI=', 'Content-Length': '0'} 2022-09-03T05:37:16.241 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004febb1a5360a2144-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.241 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.242 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.243 DEBUG:boto:path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.243 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.244 DEBUG:boto:Method: HEAD 2022-09-03T05:37:16.244 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.245 DEBUG:boto:Data: 2022-09-03T05:37:16.245 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.246 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.247 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.247 DEBUG:boto:Params: {} 2022-09-03T05:37:16.248 DEBUG:boto:Token: None 2022-09-03T05:37:16.249 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.250 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vY1MFRpCeLTdzZFh9A3ed64++QI= 2022-09-03T05:37:16.250 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vY1MFRpCeLTdzZFh9A3ed64++QI=', 'Content-Length': '0'} 2022-09-03T05:37:16.258 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000027cd7c78a8185074-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.259 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:16.259 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.260 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.261 DEBUG:boto:path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.261 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.263 DEBUG:boto:Method: GET 2022-09-03T05:37:16.264 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.264 DEBUG:boto:Data: 2022-09-03T05:37:16.265 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.266 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.266 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.267 DEBUG:boto:Params: {} 2022-09-03T05:37:16.268 DEBUG:boto:Token: None 2022-09-03T05:37:16.268 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.269 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9Gson1Otnk/Mu8WJjkXylT+YkWI= 2022-09-03T05:37:16.270 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9Gson1Otnk/Mu8WJjkXylT+YkWI=', 'Content-Length': '0'} 2022-09-03T05:37:16.272 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000081210b17bc3c8b49-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.273 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.273 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.274 DEBUG:boto:path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.274 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.275 DEBUG:boto:Method: GET 2022-09-03T05:37:16.276 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.276 DEBUG:boto:Data: 2022-09-03T05:37:16.277 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.278 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.279 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.279 DEBUG:boto:Params: {} 2022-09-03T05:37:16.280 DEBUG:boto:Token: None 2022-09-03T05:37:16.281 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:16.281 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9Gson1Otnk/Mu8WJjkXylT+YkWI= 2022-09-03T05:37:16.282 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9Gson1Otnk/Mu8WJjkXylT+YkWI=', 'Content-Length': '0'} 2022-09-03T05:37:16.284 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a43601371a3e3486-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.285 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:16.286 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:16.287 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.288 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.290 DEBUG:boto:Method: GET 2022-09-03T05:37:16.290 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.291 DEBUG:boto:Data: 2022-09-03T05:37:16.292 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.292 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.293 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.294 DEBUG:boto:Params: {} 2022-09-03T05:37:16.295 DEBUG:boto:Token: None 2022-09-03T05:37:16.296 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/ 2022-09-03T05:37:16.297 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI= 2022-09-03T05:37:16.298 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI=', 'Content-Length': '0'} 2022-09-03T05:37:16.301 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000077985b6208cf533e-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.302 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:16.303 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:16.304 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:16.304 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.305 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.305 DEBUG:boto:Method: GET 2022-09-03T05:37:16.306 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.306 DEBUG:boto:Data: 2022-09-03T05:37:16.307 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.307 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.308 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.309 DEBUG:boto:Params: {} 2022-09-03T05:37:16.309 DEBUG:boto:Token: None 2022-09-03T05:37:16.310 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/ 2022-09-03T05:37:16.311 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI= 2022-09-03T05:37:16.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI=', 'Content-Length': '0'} 2022-09-03T05:37:16.315 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005a234af71d91b519-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.316 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:16.317 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.318 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.318 DEBUG:boto:Method: HEAD 2022-09-03T05:37:16.319 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:16.320 DEBUG:boto:Data: 2022-09-03T05:37:16.321 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.321 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.322 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.323 DEBUG:boto:Params: {} 2022-09-03T05:37:16.325 DEBUG:boto:Token: None 2022-09-03T05:37:16.326 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj 2022-09-03T05:37:16.327 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4N8E3REQprlZYJTC8yiSyg9T40I= 2022-09-03T05:37:16.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4N8E3REQprlZYJTC8yiSyg9T40I=', 'Content-Length': '0'} 2022-09-03T05:37:16.331 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d54794ea47e8768b-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.332 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.333 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.333 DEBUG:boto:Method: HEAD 2022-09-03T05:37:16.334 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:16.334 DEBUG:boto:Data: 2022-09-03T05:37:16.335 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.335 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.337 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.337 DEBUG:boto:Params: {} 2022-09-03T05:37:16.338 DEBUG:boto:Token: None 2022-09-03T05:37:16.338 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj 2022-09-03T05:37:16.339 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4N8E3REQprlZYJTC8yiSyg9T40I= 2022-09-03T05:37:16.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4N8E3REQprlZYJTC8yiSyg9T40I=', 'Content-Length': '0'} 2022-09-03T05:37:16.343 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000089f7c6c0ac564e34-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.344 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:16.345 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.346 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.347 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.347 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.348 DEBUG:boto:Method: GET 2022-09-03T05:37:16.348 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.349 DEBUG:boto:Data: 2022-09-03T05:37:16.349 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.350 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.351 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.351 DEBUG:boto:Params: {} 2022-09-03T05:37:16.352 DEBUG:boto:Token: None 2022-09-03T05:37:16.352 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.353 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xot1jZa7b4/yUSjAvYOTjYePM2k= 2022-09-03T05:37:16.354 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xot1jZa7b4/yUSjAvYOTjYePM2k=', 'Content-Length': '0'} 2022-09-03T05:37:16.356 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005bbca34c8c18be4e-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.357 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.358 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.358 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.359 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.360 DEBUG:boto:Method: GET 2022-09-03T05:37:16.361 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.362 DEBUG:boto:Data: 2022-09-03T05:37:16.362 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.363 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.363 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.364 DEBUG:boto:Params: {} 2022-09-03T05:37:16.365 DEBUG:boto:Token: None 2022-09-03T05:37:16.365 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xot1jZa7b4/yUSjAvYOTjYePM2k= 2022-09-03T05:37:16.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xot1jZa7b4/yUSjAvYOTjYePM2k=', 'Content-Length': '0'} 2022-09-03T05:37:16.370 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000072e70db972d1a239-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.411 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:16.413 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:16.414 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.415 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.416 DEBUG:boto:Method: GET 2022-09-03T05:37:16.417 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.420 DEBUG:boto:Data: 2022-09-03T05:37:16.421 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.423 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.423 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.424 DEBUG:boto:Params: {} 2022-09-03T05:37:16.425 DEBUG:boto:Token: None 2022-09-03T05:37:16.426 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/ 2022-09-03T05:37:16.426 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI= 2022-09-03T05:37:16.427 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI=', 'Content-Length': '0'} 2022-09-03T05:37:16.432 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c7665f2086487dea-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.432 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:16.433 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:16.434 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:16.434 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.435 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.435 DEBUG:boto:Method: GET 2022-09-03T05:37:16.436 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.436 DEBUG:boto:Data: 2022-09-03T05:37:16.437 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.437 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.438 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.439 DEBUG:boto:Params: {} 2022-09-03T05:37:16.439 DEBUG:boto:Token: None 2022-09-03T05:37:16.440 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/ 2022-09-03T05:37:16.450 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI= 2022-09-03T05:37:16.451 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI=', 'Content-Length': '0'} 2022-09-03T05:37:16.454 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a058d40c4e322121-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.455 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:16.456 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:16.457 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.457 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.458 DEBUG:boto:path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.459 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.460 DEBUG:boto:Method: GET 2022-09-03T05:37:16.461 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.461 DEBUG:boto:Data: 2022-09-03T05:37:16.462 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.463 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.463 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.464 DEBUG:boto:Params: {} 2022-09-03T05:37:16.465 DEBUG:boto:Token: None 2022-09-03T05:37:16.465 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.466 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TP4pOi2QovpNN7Mch2JBC5Ptj6Y= 2022-09-03T05:37:16.467 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TP4pOi2QovpNN7Mch2JBC5Ptj6Y=', 'Content-Length': '0'} 2022-09-03T05:37:16.469 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'CFKxqazR5g.k3.oWs62bQgs9moHAimI'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000095d83d6077ec84a3-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.470 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.470 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.471 DEBUG:boto:path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.471 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.472 DEBUG:boto:Method: GET 2022-09-03T05:37:16.472 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.473 DEBUG:boto:Data: 2022-09-03T05:37:16.474 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.474 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.475 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.476 DEBUG:boto:Params: {} 2022-09-03T05:37:16.476 DEBUG:boto:Token: None 2022-09-03T05:37:16.477 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.478 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TP4pOi2QovpNN7Mch2JBC5Ptj6Y= 2022-09-03T05:37:16.478 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TP4pOi2QovpNN7Mch2JBC5Ptj6Y=', 'Content-Length': '0'} 2022-09-03T05:37:16.481 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'CFKxqazR5g.k3.oWs62bQgs9moHAimI'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c98c6a7a9d16d622-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.482 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.482 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.483 DEBUG:boto:path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.483 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.484 DEBUG:boto:Method: HEAD 2022-09-03T05:37:16.484 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.485 DEBUG:boto:Data: 2022-09-03T05:37:16.485 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.486 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.486 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.487 DEBUG:boto:Params: {} 2022-09-03T05:37:16.487 DEBUG:boto:Token: None 2022-09-03T05:37:16.488 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.489 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:niNqHsr0rqlr56wX7tY8XOc9IEI= 2022-09-03T05:37:16.489 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:niNqHsr0rqlr56wX7tY8XOc9IEI=', 'Content-Length': '0'} 2022-09-03T05:37:16.491 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'CFKxqazR5g.k3.oWs62bQgs9moHAimI'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006b7ea2efdffd9fe1-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.492 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.493 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.494 DEBUG:boto:path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.495 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.496 DEBUG:boto:Method: HEAD 2022-09-03T05:37:16.497 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.497 DEBUG:boto:Data: 2022-09-03T05:37:16.498 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.499 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.500 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.500 DEBUG:boto:Params: {} 2022-09-03T05:37:16.501 DEBUG:boto:Token: None 2022-09-03T05:37:16.501 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.502 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:niNqHsr0rqlr56wX7tY8XOc9IEI= 2022-09-03T05:37:16.503 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:niNqHsr0rqlr56wX7tY8XOc9IEI=', 'Content-Length': '0'} 2022-09-03T05:37:16.505 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'CFKxqazR5g.k3.oWs62bQgs9moHAimI'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000815c48e59dfcd312-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.505 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:16.506 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.506 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.507 DEBUG:boto:path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.508 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.509 DEBUG:boto:Method: GET 2022-09-03T05:37:16.509 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.510 DEBUG:boto:Data: 2022-09-03T05:37:16.510 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.511 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.511 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.512 DEBUG:boto:Params: {} 2022-09-03T05:37:16.512 DEBUG:boto:Token: None 2022-09-03T05:37:16.513 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.514 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TP4pOi2QovpNN7Mch2JBC5Ptj6Y= 2022-09-03T05:37:16.514 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TP4pOi2QovpNN7Mch2JBC5Ptj6Y=', 'Content-Length': '0'} 2022-09-03T05:37:16.517 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'CFKxqazR5g.k3.oWs62bQgs9moHAimI'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000050293aa544eefd29-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.518 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.519 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.520 DEBUG:boto:path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.521 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.522 DEBUG:boto:Method: GET 2022-09-03T05:37:16.523 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.524 DEBUG:boto:Data: 2022-09-03T05:37:16.525 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.526 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.527 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.528 DEBUG:boto:Params: {} 2022-09-03T05:37:16.529 DEBUG:boto:Token: None 2022-09-03T05:37:16.530 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:16.531 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TP4pOi2QovpNN7Mch2JBC5Ptj6Y= 2022-09-03T05:37:16.531 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TP4pOi2QovpNN7Mch2JBC5Ptj6Y=', 'Content-Length': '0'} 2022-09-03T05:37:16.534 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'CFKxqazR5g.k3.oWs62bQgs9moHAimI'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008535f8339f9e6d91-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.535 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:16.535 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:16.536 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.536 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.537 DEBUG:boto:Method: GET 2022-09-03T05:37:16.537 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.538 DEBUG:boto:Data: 2022-09-03T05:37:16.539 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.539 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.540 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.540 DEBUG:boto:Params: {} 2022-09-03T05:37:16.541 DEBUG:boto:Token: None 2022-09-03T05:37:16.541 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/ 2022-09-03T05:37:16.542 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI= 2022-09-03T05:37:16.543 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI=', 'Content-Length': '0'} 2022-09-03T05:37:16.546 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006f885204ff61ff17-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.547 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:16.548 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:16.549 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:16.550 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.551 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.552 DEBUG:boto:Method: GET 2022-09-03T05:37:16.553 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.554 DEBUG:boto:Data: 2022-09-03T05:37:16.555 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.556 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.557 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.558 DEBUG:boto:Params: {} 2022-09-03T05:37:16.559 DEBUG:boto:Token: None 2022-09-03T05:37:16.560 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/ 2022-09-03T05:37:16.560 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI= 2022-09-03T05:37:16.561 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI=', 'Content-Length': '0'} 2022-09-03T05:37:16.565 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bc3ab34ff2bc9363-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.566 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:16.567 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.568 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.569 DEBUG:boto:Method: HEAD 2022-09-03T05:37:16.569 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:16.570 DEBUG:boto:Data: 2022-09-03T05:37:16.570 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.571 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.571 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.572 DEBUG:boto:Params: {} 2022-09-03T05:37:16.572 DEBUG:boto:Token: None 2022-09-03T05:37:16.573 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj 2022-09-03T05:37:16.573 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4N8E3REQprlZYJTC8yiSyg9T40I= 2022-09-03T05:37:16.574 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4N8E3REQprlZYJTC8yiSyg9T40I=', 'Content-Length': '0'} 2022-09-03T05:37:16.578 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b47be8e213ffbda5-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.578 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.579 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.579 DEBUG:boto:Method: HEAD 2022-09-03T05:37:16.580 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:16.580 DEBUG:boto:Data: 2022-09-03T05:37:16.581 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.581 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.582 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.582 DEBUG:boto:Params: {} 2022-09-03T05:37:16.583 DEBUG:boto:Token: None 2022-09-03T05:37:16.584 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj 2022-09-03T05:37:16.584 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4N8E3REQprlZYJTC8yiSyg9T40I= 2022-09-03T05:37:16.585 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4N8E3REQprlZYJTC8yiSyg9T40I=', 'Content-Length': '0'} 2022-09-03T05:37:16.589 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000611ff5f1ee19582c-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.589 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:16.590 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.591 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.591 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.592 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.593 DEBUG:boto:Method: GET 2022-09-03T05:37:16.593 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.594 DEBUG:boto:Data: 2022-09-03T05:37:16.594 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.595 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.595 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.596 DEBUG:boto:Params: {} 2022-09-03T05:37:16.596 DEBUG:boto:Token: None 2022-09-03T05:37:16.597 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.598 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xot1jZa7b4/yUSjAvYOTjYePM2k= 2022-09-03T05:37:16.599 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xot1jZa7b4/yUSjAvYOTjYePM2k=', 'Content-Length': '0'} 2022-09-03T05:37:16.601 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001ecefb52d270803e-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.643 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.644 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.645 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.645 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.646 DEBUG:boto:Method: GET 2022-09-03T05:37:16.646 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.647 DEBUG:boto:Data: 2022-09-03T05:37:16.647 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.648 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.648 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.649 DEBUG:boto:Params: {} 2022-09-03T05:37:16.650 DEBUG:boto:Token: None 2022-09-03T05:37:16.650 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.651 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xot1jZa7b4/yUSjAvYOTjYePM2k= 2022-09-03T05:37:16.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xot1jZa7b4/yUSjAvYOTjYePM2k=', 'Content-Length': '0'} 2022-09-03T05:37:16.654 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000033b58035d31369f2-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.655 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:16.655 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:16.656 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.656 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.657 DEBUG:boto:Method: GET 2022-09-03T05:37:16.658 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.658 DEBUG:boto:Data: 2022-09-03T05:37:16.659 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.659 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.660 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.660 DEBUG:boto:Params: {} 2022-09-03T05:37:16.661 DEBUG:boto:Token: None 2022-09-03T05:37:16.661 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/ 2022-09-03T05:37:16.662 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI= 2022-09-03T05:37:16.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI=', 'Content-Length': '0'} 2022-09-03T05:37:16.666 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f4f2766be0b34c9b-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.667 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:16.669 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:16.670 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:16.671 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.672 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.673 DEBUG:boto:Method: GET 2022-09-03T05:37:16.673 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.674 DEBUG:boto:Data: 2022-09-03T05:37:16.675 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.675 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.676 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.677 DEBUG:boto:Params: {} 2022-09-03T05:37:16.677 DEBUG:boto:Token: None 2022-09-03T05:37:16.678 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/ 2022-09-03T05:37:16.679 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI= 2022-09-03T05:37:16.679 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI=', 'Content-Length': '0'} 2022-09-03T05:37:16.683 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000001a60c60f25538bb-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.684 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:16.686 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:16.687 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.688 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.689 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.690 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.691 DEBUG:boto:Method: GET 2022-09-03T05:37:16.692 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.693 DEBUG:boto:Data: 2022-09-03T05:37:16.694 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.695 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.695 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.696 DEBUG:boto:Params: {} 2022-09-03T05:37:16.697 DEBUG:boto:Token: None 2022-09-03T05:37:16.698 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.699 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l70J4f2HviVRppsBvjQJvviEfCE= 2022-09-03T05:37:16.700 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l70J4f2HviVRppsBvjQJvviEfCE=', 'Content-Length': '0'} 2022-09-03T05:37:16.703 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000207dda134af9671d-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.704 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.705 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.705 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.706 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.706 DEBUG:boto:Method: GET 2022-09-03T05:37:16.707 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.707 DEBUG:boto:Data: 2022-09-03T05:37:16.708 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.708 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.709 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.709 DEBUG:boto:Params: {} 2022-09-03T05:37:16.710 DEBUG:boto:Token: None 2022-09-03T05:37:16.711 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.711 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l70J4f2HviVRppsBvjQJvviEfCE= 2022-09-03T05:37:16.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l70J4f2HviVRppsBvjQJvviEfCE=', 'Content-Length': '0'} 2022-09-03T05:37:16.715 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ef94fe1dff1e968e-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.759 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.760 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.761 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.762 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.762 DEBUG:boto:Method: HEAD 2022-09-03T05:37:16.763 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.764 DEBUG:boto:Data: 2022-09-03T05:37:16.765 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.766 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.767 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.768 DEBUG:boto:Params: {} 2022-09-03T05:37:16.769 DEBUG:boto:Token: None 2022-09-03T05:37:16.770 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.771 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h7u6pqI3r3rd9GbPvIHEeqmapvw= 2022-09-03T05:37:16.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h7u6pqI3r3rd9GbPvIHEeqmapvw=', 'Content-Length': '0'} 2022-09-03T05:37:16.775 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b2b50dface80400d-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.776 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.777 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.778 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.779 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.780 DEBUG:boto:Method: HEAD 2022-09-03T05:37:16.781 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.781 DEBUG:boto:Data: 2022-09-03T05:37:16.783 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.783 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.784 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.785 DEBUG:boto:Params: {} 2022-09-03T05:37:16.786 DEBUG:boto:Token: None 2022-09-03T05:37:16.787 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.788 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h7u6pqI3r3rd9GbPvIHEeqmapvw= 2022-09-03T05:37:16.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h7u6pqI3r3rd9GbPvIHEeqmapvw=', 'Content-Length': '0'} 2022-09-03T05:37:16.792 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a64700106590ec5e-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.793 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:16.794 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.794 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.795 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.795 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.796 DEBUG:boto:Method: GET 2022-09-03T05:37:16.796 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.797 DEBUG:boto:Data: 2022-09-03T05:37:16.798 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.799 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.799 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.800 DEBUG:boto:Params: {} 2022-09-03T05:37:16.800 DEBUG:boto:Token: None 2022-09-03T05:37:16.801 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.802 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l70J4f2HviVRppsBvjQJvviEfCE= 2022-09-03T05:37:16.803 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l70J4f2HviVRppsBvjQJvviEfCE=', 'Content-Length': '0'} 2022-09-03T05:37:16.805 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008c0625b09f1291b0-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.846 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.847 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.848 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.848 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.849 DEBUG:boto:Method: GET 2022-09-03T05:37:16.849 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.850 DEBUG:boto:Data: 2022-09-03T05:37:16.851 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.851 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.852 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.852 DEBUG:boto:Params: {} 2022-09-03T05:37:16.853 DEBUG:boto:Token: None 2022-09-03T05:37:16.854 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:16.854 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l70J4f2HviVRppsBvjQJvviEfCE= 2022-09-03T05:37:16.855 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l70J4f2HviVRppsBvjQJvviEfCE=', 'Content-Length': '0'} 2022-09-03T05:37:16.857 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000003243ef174b00b2e-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.858 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:16.859 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:16.859 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.860 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.860 DEBUG:boto:Method: GET 2022-09-03T05:37:16.861 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.862 DEBUG:boto:Data: 2022-09-03T05:37:16.862 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.863 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.863 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.864 DEBUG:boto:Params: {} 2022-09-03T05:37:16.865 DEBUG:boto:Token: None 2022-09-03T05:37:16.865 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/ 2022-09-03T05:37:16.866 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI= 2022-09-03T05:37:16.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI=', 'Content-Length': '0'} 2022-09-03T05:37:16.870 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008dca4ab600b71797-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.871 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:16.872 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:16.872 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:16.873 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.873 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.874 DEBUG:boto:Method: GET 2022-09-03T05:37:16.874 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.875 DEBUG:boto:Data: 2022-09-03T05:37:16.875 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.876 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.877 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.877 DEBUG:boto:Params: {} 2022-09-03T05:37:16.878 DEBUG:boto:Token: None 2022-09-03T05:37:16.878 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/ 2022-09-03T05:37:16.879 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI= 2022-09-03T05:37:16.881 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI=', 'Content-Length': '0'} 2022-09-03T05:37:16.884 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fa36661f29ff7665-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.885 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:16.886 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.887 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.887 DEBUG:boto:Method: HEAD 2022-09-03T05:37:16.888 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:16.888 DEBUG:boto:Data: 2022-09-03T05:37:16.889 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.889 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.890 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.890 DEBUG:boto:Params: {} 2022-09-03T05:37:16.891 DEBUG:boto:Token: None 2022-09-03T05:37:16.892 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj 2022-09-03T05:37:16.892 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4N8E3REQprlZYJTC8yiSyg9T40I= 2022-09-03T05:37:16.893 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4N8E3REQprlZYJTC8yiSyg9T40I=', 'Content-Length': '0'} 2022-09-03T05:37:16.896 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a58711f8f74053f4-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.897 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.898 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.899 DEBUG:boto:Method: HEAD 2022-09-03T05:37:16.899 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:16.900 DEBUG:boto:Data: 2022-09-03T05:37:16.901 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.902 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.903 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.903 DEBUG:boto:Params: {} 2022-09-03T05:37:16.905 DEBUG:boto:Token: None 2022-09-03T05:37:16.905 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj 2022-09-03T05:37:16.910 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4N8E3REQprlZYJTC8yiSyg9T40I= 2022-09-03T05:37:16.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4N8E3REQprlZYJTC8yiSyg9T40I=', 'Content-Length': '0'} 2022-09-03T05:37:16.917 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004ddaaa43e6575a1c-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.919 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:16.920 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.922 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.923 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.925 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.926 DEBUG:boto:Method: GET 2022-09-03T05:37:16.927 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.928 DEBUG:boto:Data: 2022-09-03T05:37:16.928 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.929 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.930 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.931 DEBUG:boto:Params: {} 2022-09-03T05:37:16.932 DEBUG:boto:Token: None 2022-09-03T05:37:16.932 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.933 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xot1jZa7b4/yUSjAvYOTjYePM2k= 2022-09-03T05:37:16.933 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xot1jZa7b4/yUSjAvYOTjYePM2k=', 'Content-Length': '0'} 2022-09-03T05:37:16.936 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000034b3d899c3d423df-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.937 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:16.937 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:16.938 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.939 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.939 DEBUG:boto:Method: GET 2022-09-03T05:37:16.940 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.940 DEBUG:boto:Data: 2022-09-03T05:37:16.941 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.942 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.943 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.943 DEBUG:boto:Params: {} 2022-09-03T05:37:16.944 DEBUG:boto:Token: None 2022-09-03T05:37:16.945 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:16.945 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xot1jZa7b4/yUSjAvYOTjYePM2k= 2022-09-03T05:37:16.946 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xot1jZa7b4/yUSjAvYOTjYePM2k=', 'Content-Length': '0'} 2022-09-03T05:37:16.949 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ee8044cd144c9108-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.951 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:16.953 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:16.954 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.955 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.956 DEBUG:boto:Method: GET 2022-09-03T05:37:16.957 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.958 DEBUG:boto:Data: 2022-09-03T05:37:16.959 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.959 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:16.961 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.962 DEBUG:boto:Params: {} 2022-09-03T05:37:16.963 DEBUG:boto:Token: None 2022-09-03T05:37:16.964 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/ 2022-09-03T05:37:16.964 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI= 2022-09-03T05:37:16.965 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI=', 'Content-Length': '0'} 2022-09-03T05:37:16.969 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000047c46dffc312da90-006312e80c-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.969 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:16.970 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:16.971 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:16.972 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.973 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.974 DEBUG:boto:Method: GET 2022-09-03T05:37:16.974 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:16.975 DEBUG:boto:Data: 2022-09-03T05:37:16.976 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.977 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:16.978 DEBUG:boto:Port: 8000 2022-09-03T05:37:16.979 DEBUG:boto:Params: {} 2022-09-03T05:37:16.979 DEBUG:boto:Token: None 2022-09-03T05:37:16.980 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:16 GMT /dsgvcy-19/ 2022-09-03T05:37:16.981 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI= 2022-09-03T05:37:16.982 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yi0QHJUXrnZNrkB9ncJf6Xsj7uI=', 'Content-Length': '0'} 2022-09-03T05:37:16.986 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004ade360459e13e31-006312e80c-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:16.988 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:16.990 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-19 zones={test-zone2, test-zone1} 2022-09-03T05:37:16.991 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-19 zones={test-zone3, test-zone1} 2022-09-03T05:37:16.992 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:16.994 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:16.995 DEBUG:boto:Method: HEAD 2022-09-03T05:37:16.995 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:37:16.996 DEBUG:boto:Data: 2022-09-03T05:37:16.997 DEBUG:boto:Headers: {} 2022-09-03T05:37:16.998 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:16.999 DEBUG:boto:Port: 8001 2022-09-03T05:37:17.000 DEBUG:boto:Params: {} 2022-09-03T05:37:17.001 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T05:37:17.002 DEBUG:boto:Token: None 2022-09-03T05:37:17.003 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/ 2022-09-03T05:37:17.004 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n6qx7+Itax6I+fSLUbooWHgTys4= 2022-09-03T05:37:17.005 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n6qx7+Itax6I+fSLUbooWHgTys4=', 'Content-Length': '0'} 2022-09-03T05:37:17.010 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '31'), ('X-RGW-Bytes-Used', '240'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000af529378ef584d00-006312e80d-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.011 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:17.012 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:17.013 DEBUG:boto:Method: HEAD 2022-09-03T05:37:17.013 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:37:17.014 DEBUG:boto:Data: 2022-09-03T05:37:17.015 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.015 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:17.016 DEBUG:boto:Port: 8000 2022-09-03T05:37:17.016 DEBUG:boto:Params: {} 2022-09-03T05:37:17.017 DEBUG:boto:Token: None 2022-09-03T05:37:17.017 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/ 2022-09-03T05:37:17.019 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n6qx7+Itax6I+fSLUbooWHgTys4= 2022-09-03T05:37:17.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n6qx7+Itax6I+fSLUbooWHgTys4=', 'Content-Length': '0'} 2022-09-03T05:37:17.024 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '31'), ('X-RGW-Bytes-Used', '240'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c735f370312e0b33-006312e80d-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.025 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:37:17.026 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:17.027 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:17.028 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:17.028 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:17.029 DEBUG:boto:Method: GET 2022-09-03T05:37:17.030 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:17.030 DEBUG:boto:Data: 2022-09-03T05:37:17.031 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.031 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:17.032 DEBUG:boto:Port: 8001 2022-09-03T05:37:17.032 DEBUG:boto:Params: {} 2022-09-03T05:37:17.033 DEBUG:boto:Token: None 2022-09-03T05:37:17.034 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/?versions 2022-09-03T05:37:17.035 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sas/N/06vG/6VglyLJpoeSiNmgs= 2022-09-03T05:37:17.035 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sas/N/06vG/6VglyLJpoeSiNmgs=', 'Content-Length': '0'} 2022-09-03T05:37:17.040 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006d35df0722b4c549-006312e80d-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.041 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjFV.ndVyKCalSXA6cZK3aQWN-LzBZK3Rtrue2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjhqLZJOZHmRi2EtNletjE-T3uOA2T8pTfalse2022-09-03T05:34:51.324Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjmBYgYYD3drKvdzws.hh3-36xKvL9eTHfalse2022-09-03T05:34:51.365Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjMnuLKlk.Rq8n.YuO6g0DugPfRiHStyffalse2022-09-03T05:34:51.173Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjN.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1ifalse2022-09-03T05:34:51.242Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjgFHndXVQlMCeVf-4Dc4FENAmsrgKp-Dfalse2022-09-03T05:34:51.206Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj0PhwqB536OEXdH1QOmPxhlUfX3fwPvSfalse2022-09-03T05:34:51.042Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjGM1mLEe4fDwiUzp2k3PtPzJLy-tnWKifalse2022-09-03T05:34:51.129Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjKl9OW7HSpMTjjnmOMkqEktWm9H3uMr2false2022-09-03T05:34:50.995Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjEFeprdWgJo5gbXM16hy6HNBwhGTxK7efalse2022-09-03T05:34:50.949Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjJID7ijdm1B3ZxtamFDxAcL6LJfFKoNjfalse2022-09-03T05:34:50.861Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjyL2uOIeC0l.C5YXdYoqQBvDIly3idxOfalse2022-09-03T05:34:50.905Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.lPacc7Eu6BJ5IOsSyLccq2.KRUK24Ufalse2022-09-03T05:34:50.763Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjNkMDAxRk7LkNPj1mtl1YEsdjIaTowHtfalse2022-09-03T05:34:50.824Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQfRyggjm1gZ7H.LeMCey0hHybmVWONGfalse2022-09-03T05:34:50.794Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj94bPO72IhlWQEGwklU15Pvpr-qRv.lmfalse2022-09-03T05:34:50.677Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjORmfT2fVQ5EFIFI89EGuLigbPyfeObTfalse2022-09-03T05:34:50.719Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkUiw73n7LyoUiDcS9JXEfdlCebChUCEfalse2022-09-03T05:34:50.635Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjBZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98false2022-09-03T05:34:50.517Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjIoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SBfalse2022-09-03T05:34:50.595Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjrHbEnZ16TQjAC.xp1wNbn0Ce5Vc948ifalse2022-09-03T05:34:50.555Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjG4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXLfalse2022-09-03T05:34:50.467Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQl7wbqZZa4psf4mYXa6Hq8W4qHqEHV4false2022-09-03T05:34:50.343Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjvauOlKlWVHmDxq4ZzMBF97KP66F89Pcfalse2022-09-03T05:34:50.391Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKCfalse2022-09-03T05:34:50.189Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjoYY-icmxg74gZ4carPTk38at9fTxjJifalse2022-09-03T05:34:50.232Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjxYnysJgMUeKh.Mew5S-gBJZsJgD4g.Afalse2022-09-03T05:34:50.274Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvYfalse2022-09-03T05:34:50.041Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjCFKxqazR5g.k3.oWs62bQgs9moHAimIfalse2022-09-03T05:34:50.150Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkJSzTg.tDVQ-nL1nX84.W7ifJxIseT1false2022-09-03T05:34:50.112Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:17.045 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:37:17.045 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.046 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.046 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.047 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.047 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.048 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.048 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.049 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.049 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.049 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.050 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.051 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.052 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.053 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.053 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.054 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.055 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.056 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.057 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.058 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.058 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.059 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.060 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.061 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.061 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.062 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.062 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.063 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.063 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.064 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.064 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:37:17.065 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:17.065 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:17.066 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:17.066 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:17.067 DEBUG:boto:Method: GET 2022-09-03T05:37:17.068 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:17.068 DEBUG:boto:Data: 2022-09-03T05:37:17.069 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.069 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:17.070 DEBUG:boto:Port: 8000 2022-09-03T05:37:17.070 DEBUG:boto:Params: {} 2022-09-03T05:37:17.071 DEBUG:boto:Token: None 2022-09-03T05:37:17.072 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/?versions 2022-09-03T05:37:17.072 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sas/N/06vG/6VglyLJpoeSiNmgs= 2022-09-03T05:37:17.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sas/N/06vG/6VglyLJpoeSiNmgs=', 'Content-Length': '0'} 2022-09-03T05:37:17.086 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b7691c85cdfd9a0e-006312e80d-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.087 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjFV.ndVyKCalSXA6cZK3aQWN-LzBZK3Rtrue2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjhqLZJOZHmRi2EtNletjE-T3uOA2T8pTfalse2022-09-03T05:34:51.324Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjmBYgYYD3drKvdzws.hh3-36xKvL9eTHfalse2022-09-03T05:34:51.365Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjMnuLKlk.Rq8n.YuO6g0DugPfRiHStyffalse2022-09-03T05:34:51.173Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjN.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1ifalse2022-09-03T05:34:51.242Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjgFHndXVQlMCeVf-4Dc4FENAmsrgKp-Dfalse2022-09-03T05:34:51.206Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj0PhwqB536OEXdH1QOmPxhlUfX3fwPvSfalse2022-09-03T05:34:51.042Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjGM1mLEe4fDwiUzp2k3PtPzJLy-tnWKifalse2022-09-03T05:34:51.129Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjKl9OW7HSpMTjjnmOMkqEktWm9H3uMr2false2022-09-03T05:34:50.995Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjEFeprdWgJo5gbXM16hy6HNBwhGTxK7efalse2022-09-03T05:34:50.949Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjJID7ijdm1B3ZxtamFDxAcL6LJfFKoNjfalse2022-09-03T05:34:50.861Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjyL2uOIeC0l.C5YXdYoqQBvDIly3idxOfalse2022-09-03T05:34:50.905Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.lPacc7Eu6BJ5IOsSyLccq2.KRUK24Ufalse2022-09-03T05:34:50.763Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjNkMDAxRk7LkNPj1mtl1YEsdjIaTowHtfalse2022-09-03T05:34:50.824Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQfRyggjm1gZ7H.LeMCey0hHybmVWONGfalse2022-09-03T05:34:50.794Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj94bPO72IhlWQEGwklU15Pvpr-qRv.lmfalse2022-09-03T05:34:50.677Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjORmfT2fVQ5EFIFI89EGuLigbPyfeObTfalse2022-09-03T05:34:50.719Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkUiw73n7LyoUiDcS9JXEfdlCebChUCEfalse2022-09-03T05:34:50.635Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjBZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98false2022-09-03T05:34:50.517Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjIoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SBfalse2022-09-03T05:34:50.595Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjrHbEnZ16TQjAC.xp1wNbn0Ce5Vc948ifalse2022-09-03T05:34:50.555Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjG4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXLfalse2022-09-03T05:34:50.467Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQl7wbqZZa4psf4mYXa6Hq8W4qHqEHV4false2022-09-03T05:34:50.343Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjvauOlKlWVHmDxq4ZzMBF97KP66F89Pcfalse2022-09-03T05:34:50.391Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKCfalse2022-09-03T05:34:50.189Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjoYY-icmxg74gZ4carPTk38at9fTxjJifalse2022-09-03T05:34:50.232Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjxYnysJgMUeKh.Mew5S-gBJZsJgD4g.Afalse2022-09-03T05:34:50.274Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvYfalse2022-09-03T05:34:50.041Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjCFKxqazR5g.k3.oWs62bQgs9moHAimIfalse2022-09-03T05:34:50.150Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkJSzTg.tDVQ-nL1nX84.W7ifJxIseT1false2022-09-03T05:34:50.112Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:17.089 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:37:17.089 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.090 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.090 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.091 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.091 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.092 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.092 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.093 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.093 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.094 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.094 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.095 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.095 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.096 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.096 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.097 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.097 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.097 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.098 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.098 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.099 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.099 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.100 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.100 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.101 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.101 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.102 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.102 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.103 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.103 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:17.104 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:17.104 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:17.105 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:17.105 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:17.106 DEBUG:boto:Method: GET 2022-09-03T05:37:17.106 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:17.107 DEBUG:boto:Data: 2022-09-03T05:37:17.107 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.108 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:17.108 DEBUG:boto:Port: 8001 2022-09-03T05:37:17.109 DEBUG:boto:Params: {} 2022-09-03T05:37:17.110 DEBUG:boto:Token: None 2022-09-03T05:37:17.110 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/?versions 2022-09-03T05:37:17.111 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sas/N/06vG/6VglyLJpoeSiNmgs= 2022-09-03T05:37:17.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sas/N/06vG/6VglyLJpoeSiNmgs=', 'Content-Length': '0'} 2022-09-03T05:37:17.115 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000078afa8814602bdf8-006312e80d-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.116 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjFV.ndVyKCalSXA6cZK3aQWN-LzBZK3Rtrue2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjhqLZJOZHmRi2EtNletjE-T3uOA2T8pTfalse2022-09-03T05:34:51.324Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjmBYgYYD3drKvdzws.hh3-36xKvL9eTHfalse2022-09-03T05:34:51.365Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjMnuLKlk.Rq8n.YuO6g0DugPfRiHStyffalse2022-09-03T05:34:51.173Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjN.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1ifalse2022-09-03T05:34:51.242Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjgFHndXVQlMCeVf-4Dc4FENAmsrgKp-Dfalse2022-09-03T05:34:51.206Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj0PhwqB536OEXdH1QOmPxhlUfX3fwPvSfalse2022-09-03T05:34:51.042Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjGM1mLEe4fDwiUzp2k3PtPzJLy-tnWKifalse2022-09-03T05:34:51.129Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjKl9OW7HSpMTjjnmOMkqEktWm9H3uMr2false2022-09-03T05:34:50.995Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjEFeprdWgJo5gbXM16hy6HNBwhGTxK7efalse2022-09-03T05:34:50.949Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjJID7ijdm1B3ZxtamFDxAcL6LJfFKoNjfalse2022-09-03T05:34:50.861Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjyL2uOIeC0l.C5YXdYoqQBvDIly3idxOfalse2022-09-03T05:34:50.905Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.lPacc7Eu6BJ5IOsSyLccq2.KRUK24Ufalse2022-09-03T05:34:50.763Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjNkMDAxRk7LkNPj1mtl1YEsdjIaTowHtfalse2022-09-03T05:34:50.824Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQfRyggjm1gZ7H.LeMCey0hHybmVWONGfalse2022-09-03T05:34:50.794Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj94bPO72IhlWQEGwklU15Pvpr-qRv.lmfalse2022-09-03T05:34:50.677Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjORmfT2fVQ5EFIFI89EGuLigbPyfeObTfalse2022-09-03T05:34:50.719Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkUiw73n7LyoUiDcS9JXEfdlCebChUCEfalse2022-09-03T05:34:50.635Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjBZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98false2022-09-03T05:34:50.517Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjIoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SBfalse2022-09-03T05:34:50.595Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjrHbEnZ16TQjAC.xp1wNbn0Ce5Vc948ifalse2022-09-03T05:34:50.555Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjG4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXLfalse2022-09-03T05:34:50.467Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQl7wbqZZa4psf4mYXa6Hq8W4qHqEHV4false2022-09-03T05:34:50.343Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjvauOlKlWVHmDxq4ZzMBF97KP66F89Pcfalse2022-09-03T05:34:50.391Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKCfalse2022-09-03T05:34:50.189Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjoYY-icmxg74gZ4carPTk38at9fTxjJifalse2022-09-03T05:34:50.232Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjxYnysJgMUeKh.Mew5S-gBJZsJgD4g.Afalse2022-09-03T05:34:50.274Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvYfalse2022-09-03T05:34:50.041Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjCFKxqazR5g.k3.oWs62bQgs9moHAimIfalse2022-09-03T05:34:50.150Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkJSzTg.tDVQ-nL1nX84.W7ifJxIseT1false2022-09-03T05:34:50.112Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:17.118 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:17.119 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:17.119 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:17.120 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:17.121 DEBUG:boto:Method: GET 2022-09-03T05:37:17.121 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:17.122 DEBUG:boto:Data: 2022-09-03T05:37:17.122 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.123 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:17.124 DEBUG:boto:Port: 8000 2022-09-03T05:37:17.124 DEBUG:boto:Params: {} 2022-09-03T05:37:17.125 DEBUG:boto:Token: None 2022-09-03T05:37:17.126 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/?versions 2022-09-03T05:37:17.126 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sas/N/06vG/6VglyLJpoeSiNmgs= 2022-09-03T05:37:17.127 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sas/N/06vG/6VglyLJpoeSiNmgs=', 'Content-Length': '0'} 2022-09-03T05:37:17.131 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c7c6b0f4ebffe33a-006312e80d-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.132 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjFV.ndVyKCalSXA6cZK3aQWN-LzBZK3Rtrue2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjhqLZJOZHmRi2EtNletjE-T3uOA2T8pTfalse2022-09-03T05:34:51.324Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjmBYgYYD3drKvdzws.hh3-36xKvL9eTHfalse2022-09-03T05:34:51.365Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjMnuLKlk.Rq8n.YuO6g0DugPfRiHStyffalse2022-09-03T05:34:51.173Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjN.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1ifalse2022-09-03T05:34:51.242Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjgFHndXVQlMCeVf-4Dc4FENAmsrgKp-Dfalse2022-09-03T05:34:51.206Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj0PhwqB536OEXdH1QOmPxhlUfX3fwPvSfalse2022-09-03T05:34:51.042Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjGM1mLEe4fDwiUzp2k3PtPzJLy-tnWKifalse2022-09-03T05:34:51.129Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjKl9OW7HSpMTjjnmOMkqEktWm9H3uMr2false2022-09-03T05:34:50.995Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjEFeprdWgJo5gbXM16hy6HNBwhGTxK7efalse2022-09-03T05:34:50.949Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjJID7ijdm1B3ZxtamFDxAcL6LJfFKoNjfalse2022-09-03T05:34:50.861Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjyL2uOIeC0l.C5YXdYoqQBvDIly3idxOfalse2022-09-03T05:34:50.905Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.lPacc7Eu6BJ5IOsSyLccq2.KRUK24Ufalse2022-09-03T05:34:50.763Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjNkMDAxRk7LkNPj1mtl1YEsdjIaTowHtfalse2022-09-03T05:34:50.824Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQfRyggjm1gZ7H.LeMCey0hHybmVWONGfalse2022-09-03T05:34:50.794Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj94bPO72IhlWQEGwklU15Pvpr-qRv.lmfalse2022-09-03T05:34:50.677Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjORmfT2fVQ5EFIFI89EGuLigbPyfeObTfalse2022-09-03T05:34:50.719Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkUiw73n7LyoUiDcS9JXEfdlCebChUCEfalse2022-09-03T05:34:50.635Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjBZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98false2022-09-03T05:34:50.517Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjIoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SBfalse2022-09-03T05:34:50.595Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjrHbEnZ16TQjAC.xp1wNbn0Ce5Vc948ifalse2022-09-03T05:34:50.555Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjG4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXLfalse2022-09-03T05:34:50.467Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQl7wbqZZa4psf4mYXa6Hq8W4qHqEHV4false2022-09-03T05:34:50.343Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjvauOlKlWVHmDxq4ZzMBF97KP66F89Pcfalse2022-09-03T05:34:50.391Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKCfalse2022-09-03T05:34:50.189Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjoYY-icmxg74gZ4carPTk38at9fTxjJifalse2022-09-03T05:34:50.232Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjxYnysJgMUeKh.Mew5S-gBJZsJgD4g.Afalse2022-09-03T05:34:50.274Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvYfalse2022-09-03T05:34:50.041Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjCFKxqazR5g.k3.oWs62bQgs9moHAimIfalse2022-09-03T05:34:50.150Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkJSzTg.tDVQ-nL1nX84.W7ifJxIseT1false2022-09-03T05:34:50.112Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:17.134 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:37:17.134 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:17.135 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:17.135 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.136 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.137 DEBUG:boto:Method: GET 2022-09-03T05:37:17.137 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.138 DEBUG:boto:Data: 2022-09-03T05:37:17.138 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.139 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:17.139 DEBUG:boto:Port: 8001 2022-09-03T05:37:17.140 DEBUG:boto:Params: {} 2022-09-03T05:37:17.141 DEBUG:boto:Token: None 2022-09-03T05:37:17.142 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.142 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:juZermKK83QsqpofF+V0MLtx6ko= 2022-09-03T05:37:17.143 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:juZermKK83QsqpofF+V0MLtx6ko=', 'Content-Length': '0'} 2022-09-03T05:37:17.145 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000081f61bf781760ce4-006312e80d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.146 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:17.147 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:17.148 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.148 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.149 DEBUG:boto:Method: GET 2022-09-03T05:37:17.149 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.150 DEBUG:boto:Data: 2022-09-03T05:37:17.150 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.151 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:17.151 DEBUG:boto:Port: 8000 2022-09-03T05:37:17.152 DEBUG:boto:Params: {} 2022-09-03T05:37:17.152 DEBUG:boto:Token: None 2022-09-03T05:37:17.153 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.154 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:juZermKK83QsqpofF+V0MLtx6ko= 2022-09-03T05:37:17.154 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:juZermKK83QsqpofF+V0MLtx6ko=', 'Content-Length': '0'} 2022-09-03T05:37:17.156 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000087d4b9df9e9d6ff9-006312e80d-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.157 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:17.158 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:17.158 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.159 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.159 DEBUG:boto:Method: HEAD 2022-09-03T05:37:17.160 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.161 DEBUG:boto:Data: 2022-09-03T05:37:17.161 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.162 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:17.162 DEBUG:boto:Port: 8001 2022-09-03T05:37:17.163 DEBUG:boto:Params: {} 2022-09-03T05:37:17.163 DEBUG:boto:Token: None 2022-09-03T05:37:17.164 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.164 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wxtn0gmXVwCSfzn8+CGrN21Z/9w= 2022-09-03T05:37:17.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wxtn0gmXVwCSfzn8+CGrN21Z/9w=', 'Content-Length': '0'} 2022-09-03T05:37:17.167 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000005bb385acdf4d8faa-006312e80d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.168 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:17.169 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:17.169 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.170 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.170 DEBUG:boto:Method: HEAD 2022-09-03T05:37:17.171 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.171 DEBUG:boto:Data: 2022-09-03T05:37:17.172 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.172 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:17.173 DEBUG:boto:Port: 8000 2022-09-03T05:37:17.173 DEBUG:boto:Params: {} 2022-09-03T05:37:17.174 DEBUG:boto:Token: None 2022-09-03T05:37:17.174 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.175 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wxtn0gmXVwCSfzn8+CGrN21Z/9w= 2022-09-03T05:37:17.176 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wxtn0gmXVwCSfzn8+CGrN21Z/9w=', 'Content-Length': '0'} 2022-09-03T05:37:17.178 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000bcccc86addc37ec7-006312e80d-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.179 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:37:17.180 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:17.181 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:17.182 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.182 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.183 DEBUG:boto:Method: GET 2022-09-03T05:37:17.184 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.185 DEBUG:boto:Data: 2022-09-03T05:37:17.186 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.187 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:17.188 DEBUG:boto:Port: 8001 2022-09-03T05:37:17.189 DEBUG:boto:Params: {} 2022-09-03T05:37:17.190 DEBUG:boto:Token: None 2022-09-03T05:37:17.191 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.192 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:juZermKK83QsqpofF+V0MLtx6ko= 2022-09-03T05:37:17.193 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:juZermKK83QsqpofF+V0MLtx6ko=', 'Content-Length': '0'} 2022-09-03T05:37:17.195 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f807e2f4bb6eca5f-006312e80d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.196 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:17.197 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:17.197 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.198 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.199 DEBUG:boto:Method: GET 2022-09-03T05:37:17.199 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.200 DEBUG:boto:Data: 2022-09-03T05:37:17.200 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.201 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:17.202 DEBUG:boto:Port: 8000 2022-09-03T05:37:17.202 DEBUG:boto:Params: {} 2022-09-03T05:37:17.203 DEBUG:boto:Token: None 2022-09-03T05:37:17.203 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.204 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:juZermKK83QsqpofF+V0MLtx6ko= 2022-09-03T05:37:17.205 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:juZermKK83QsqpofF+V0MLtx6ko=', 'Content-Length': '0'} 2022-09-03T05:37:17.207 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000058bb092f5bc557d7-006312e80d-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.208 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:17.209 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:17.210 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:17.210 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:17.211 DEBUG:boto:Method: GET 2022-09-03T05:37:17.211 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:37:17.212 DEBUG:boto:Data: 2022-09-03T05:37:17.212 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.213 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:17.213 DEBUG:boto:Port: 8001 2022-09-03T05:37:17.214 DEBUG:boto:Params: {} 2022-09-03T05:37:17.214 DEBUG:boto:Token: None 2022-09-03T05:37:17.215 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/ 2022-09-03T05:37:17.216 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:drxH8uNd9ALBcDqTSqIV6N+XPmQ= 2022-09-03T05:37:17.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:drxH8uNd9ALBcDqTSqIV6N+XPmQ=', 'Content-Length': '0'} 2022-09-03T05:37:17.221 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f6a4e0932a983444-006312e80d-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.222 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:17.224 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:17.225 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:17.225 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:17.226 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:17.227 DEBUG:boto:Method: GET 2022-09-03T05:37:17.229 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:37:17.230 DEBUG:boto:Data: 2022-09-03T05:37:17.231 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.232 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:17.232 DEBUG:boto:Port: 8000 2022-09-03T05:37:17.233 DEBUG:boto:Params: {} 2022-09-03T05:37:17.234 DEBUG:boto:Token: None 2022-09-03T05:37:17.237 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/ 2022-09-03T05:37:17.239 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:drxH8uNd9ALBcDqTSqIV6N+XPmQ= 2022-09-03T05:37:17.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:drxH8uNd9ALBcDqTSqIV6N+XPmQ=', 'Content-Length': '0'} 2022-09-03T05:37:17.250 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000311b73583df656c0-006312e80d-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.252 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:17.253 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:17.254 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:17.255 DEBUG:boto:Method: HEAD 2022-09-03T05:37:17.256 DEBUG:boto:Path: /dsgvcy-19/dummy 2022-09-03T05:37:17.257 DEBUG:boto:Data: 2022-09-03T05:37:17.258 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.259 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:17.260 DEBUG:boto:Port: 8001 2022-09-03T05:37:17.261 DEBUG:boto:Params: {} 2022-09-03T05:37:17.262 DEBUG:boto:Token: None 2022-09-03T05:37:17.263 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/dummy 2022-09-03T05:37:17.264 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rse9lxiKA44OczzNbRKxp8IzSz4= 2022-09-03T05:37:17.265 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rse9lxiKA44OczzNbRKxp8IzSz4=', 'Content-Length': '0'} 2022-09-03T05:37:17.270 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000060bc47994f11a8b1-006312e80d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.272 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:17.273 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:17.274 DEBUG:boto:Method: HEAD 2022-09-03T05:37:17.275 DEBUG:boto:Path: /dsgvcy-19/dummy 2022-09-03T05:37:17.276 DEBUG:boto:Data: 2022-09-03T05:37:17.277 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.279 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:17.280 DEBUG:boto:Port: 8000 2022-09-03T05:37:17.281 DEBUG:boto:Params: {} 2022-09-03T05:37:17.282 DEBUG:boto:Token: None 2022-09-03T05:37:17.283 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/dummy 2022-09-03T05:37:17.284 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rse9lxiKA44OczzNbRKxp8IzSz4= 2022-09-03T05:37:17.285 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rse9lxiKA44OczzNbRKxp8IzSz4=', 'Content-Length': '0'} 2022-09-03T05:37:17.288 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000066b0a801150ac4dd-006312e80d-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.290 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:37:17.291 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:17.292 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:17.293 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.294 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.295 DEBUG:boto:Method: GET 2022-09-03T05:37:17.296 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.298 DEBUG:boto:Data: 2022-09-03T05:37:17.299 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.300 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:17.301 DEBUG:boto:Port: 8001 2022-09-03T05:37:17.303 DEBUG:boto:Params: {} 2022-09-03T05:37:17.304 DEBUG:boto:Token: None 2022-09-03T05:37:17.305 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.307 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:juZermKK83QsqpofF+V0MLtx6ko= 2022-09-03T05:37:17.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:juZermKK83QsqpofF+V0MLtx6ko=', 'Content-Length': '0'} 2022-09-03T05:37:17.311 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000000e38ad4e9d11946d-006312e80d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.312 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:17.313 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:17.313 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.314 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.315 DEBUG:boto:Method: GET 2022-09-03T05:37:17.315 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.316 DEBUG:boto:Data: 2022-09-03T05:37:17.317 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.317 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:17.318 DEBUG:boto:Port: 8000 2022-09-03T05:37:17.318 DEBUG:boto:Params: {} 2022-09-03T05:37:17.319 DEBUG:boto:Token: None 2022-09-03T05:37:17.320 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:17.320 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:juZermKK83QsqpofF+V0MLtx6ko= 2022-09-03T05:37:17.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:juZermKK83QsqpofF+V0MLtx6ko=', 'Content-Length': '0'} 2022-09-03T05:37:17.323 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000077476a7d062481b9-006312e80d-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.324 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:17.325 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:17.325 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:17.326 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:17.326 DEBUG:boto:Method: GET 2022-09-03T05:37:17.327 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:37:17.328 DEBUG:boto:Data: 2022-09-03T05:37:17.328 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.329 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:17.329 DEBUG:boto:Port: 8001 2022-09-03T05:37:17.330 DEBUG:boto:Params: {} 2022-09-03T05:37:17.330 DEBUG:boto:Token: None 2022-09-03T05:37:17.331 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/ 2022-09-03T05:37:17.332 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:drxH8uNd9ALBcDqTSqIV6N+XPmQ= 2022-09-03T05:37:17.332 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:drxH8uNd9ALBcDqTSqIV6N+XPmQ=', 'Content-Length': '0'} 2022-09-03T05:37:17.335 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dec4a4b50523f1f5-006312e80d-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.336 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:17.341 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:17.342 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:17.343 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:17.343 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:17.344 DEBUG:boto:Method: GET 2022-09-03T05:37:17.344 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:37:17.345 DEBUG:boto:Data: 2022-09-03T05:37:17.345 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.346 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:17.346 DEBUG:boto:Port: 8000 2022-09-03T05:37:17.347 DEBUG:boto:Params: {} 2022-09-03T05:37:17.347 DEBUG:boto:Token: None 2022-09-03T05:37:17.348 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/ 2022-09-03T05:37:17.349 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:drxH8uNd9ALBcDqTSqIV6N+XPmQ= 2022-09-03T05:37:17.349 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:drxH8uNd9ALBcDqTSqIV6N+XPmQ=', 'Content-Length': '0'} 2022-09-03T05:37:17.353 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a0436bc2d3a45f4c-006312e80d-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.355 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:17.356 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:17.357 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:17.358 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:17.359 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.360 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.361 DEBUG:boto:Method: GET 2022-09-03T05:37:17.362 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.363 DEBUG:boto:Data: 2022-09-03T05:37:17.364 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.365 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:17.365 DEBUG:boto:Port: 8001 2022-09-03T05:37:17.366 DEBUG:boto:Params: {} 2022-09-03T05:37:17.368 DEBUG:boto:Token: None 2022-09-03T05:37:17.368 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.369 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kh4PXJ8eBAWQkg5iuXHJcIXud7o= 2022-09-03T05:37:17.370 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kh4PXJ8eBAWQkg5iuXHJcIXud7o=', 'Content-Length': '0'} 2022-09-03T05:37:17.375 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000020d671e33a44b998-006312e80d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.418 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:17.420 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:17.421 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.422 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.423 DEBUG:boto:Method: GET 2022-09-03T05:37:17.423 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.424 DEBUG:boto:Data: 2022-09-03T05:37:17.424 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.425 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:17.426 DEBUG:boto:Port: 8000 2022-09-03T05:37:17.426 DEBUG:boto:Params: {} 2022-09-03T05:37:17.427 DEBUG:boto:Token: None 2022-09-03T05:37:17.428 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.428 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kh4PXJ8eBAWQkg5iuXHJcIXud7o= 2022-09-03T05:37:17.429 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kh4PXJ8eBAWQkg5iuXHJcIXud7o=', 'Content-Length': '0'} 2022-09-03T05:37:17.432 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002cd0fcf965dd7de3-006312e80d-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.433 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:17.434 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:17.437 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.437 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.439 DEBUG:boto:Method: HEAD 2022-09-03T05:37:17.440 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.441 DEBUG:boto:Data: 2022-09-03T05:37:17.442 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.443 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:17.444 DEBUG:boto:Port: 8001 2022-09-03T05:37:17.447 DEBUG:boto:Params: {} 2022-09-03T05:37:17.448 DEBUG:boto:Token: None 2022-09-03T05:37:17.448 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.449 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:T0DAzZtJi/Op/B+4OZDIFh2rESE= 2022-09-03T05:37:17.451 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:T0DAzZtJi/Op/B+4OZDIFh2rESE=', 'Content-Length': '0'} 2022-09-03T05:37:17.453 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000797490ecc1dcf055-006312e80d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.454 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:17.455 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:17.455 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.456 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.456 DEBUG:boto:Method: HEAD 2022-09-03T05:37:17.457 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.457 DEBUG:boto:Data: 2022-09-03T05:37:17.458 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.458 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:17.459 DEBUG:boto:Port: 8000 2022-09-03T05:37:17.459 DEBUG:boto:Params: {} 2022-09-03T05:37:17.460 DEBUG:boto:Token: None 2022-09-03T05:37:17.461 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.461 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:T0DAzZtJi/Op/B+4OZDIFh2rESE= 2022-09-03T05:37:17.462 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:T0DAzZtJi/Op/B+4OZDIFh2rESE=', 'Content-Length': '0'} 2022-09-03T05:37:17.466 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d6abc706f48389b5-006312e80d-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.466 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:17.467 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:17.468 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:17.468 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.469 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.469 DEBUG:boto:Method: GET 2022-09-03T05:37:17.470 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.470 DEBUG:boto:Data: 2022-09-03T05:37:17.471 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.471 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:17.472 DEBUG:boto:Port: 8001 2022-09-03T05:37:17.472 DEBUG:boto:Params: {} 2022-09-03T05:37:17.473 DEBUG:boto:Token: None 2022-09-03T05:37:17.474 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.475 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kh4PXJ8eBAWQkg5iuXHJcIXud7o= 2022-09-03T05:37:17.475 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kh4PXJ8eBAWQkg5iuXHJcIXud7o=', 'Content-Length': '0'} 2022-09-03T05:37:17.478 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a4544d89240e3a59-006312e80d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.480 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:17.481 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:17.481 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.482 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.482 DEBUG:boto:Method: GET 2022-09-03T05:37:17.483 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.483 DEBUG:boto:Data: 2022-09-03T05:37:17.484 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.484 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:17.485 DEBUG:boto:Port: 8000 2022-09-03T05:37:17.485 DEBUG:boto:Params: {} 2022-09-03T05:37:17.486 DEBUG:boto:Token: None 2022-09-03T05:37:17.487 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.487 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kh4PXJ8eBAWQkg5iuXHJcIXud7o= 2022-09-03T05:37:17.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kh4PXJ8eBAWQkg5iuXHJcIXud7o=', 'Content-Length': '0'} 2022-09-03T05:37:17.491 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000050c2e17791b6ac7a-006312e80d-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.492 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:17.493 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:17.494 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:17.495 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:17.496 DEBUG:boto:Method: GET 2022-09-03T05:37:17.497 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:17.498 DEBUG:boto:Data: 2022-09-03T05:37:17.499 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.499 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:17.500 DEBUG:boto:Port: 8001 2022-09-03T05:37:17.500 DEBUG:boto:Params: {} 2022-09-03T05:37:17.501 DEBUG:boto:Token: None 2022-09-03T05:37:17.501 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/ 2022-09-03T05:37:17.502 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:drxH8uNd9ALBcDqTSqIV6N+XPmQ= 2022-09-03T05:37:17.503 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:drxH8uNd9ALBcDqTSqIV6N+XPmQ=', 'Content-Length': '0'} 2022-09-03T05:37:17.508 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002291b6fe0ec3862d-006312e80d-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.509 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:17.511 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:17.512 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:17.513 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:17.514 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:17.515 DEBUG:boto:Method: GET 2022-09-03T05:37:17.519 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:17.526 DEBUG:boto:Data: 2022-09-03T05:37:17.528 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.529 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:17.531 DEBUG:boto:Port: 8000 2022-09-03T05:37:17.532 DEBUG:boto:Params: {} 2022-09-03T05:37:17.533 DEBUG:boto:Token: None 2022-09-03T05:37:17.534 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/ 2022-09-03T05:37:17.535 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:drxH8uNd9ALBcDqTSqIV6N+XPmQ= 2022-09-03T05:37:17.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:drxH8uNd9ALBcDqTSqIV6N+XPmQ=', 'Content-Length': '0'} 2022-09-03T05:37:17.548 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008c1502983e68e32a-006312e80d-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.550 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:17.551 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:17.552 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:17.553 DEBUG:boto:Method: HEAD 2022-09-03T05:37:17.555 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:17.557 DEBUG:boto:Data: 2022-09-03T05:37:17.557 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.558 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:17.558 DEBUG:boto:Port: 8001 2022-09-03T05:37:17.559 DEBUG:boto:Params: {} 2022-09-03T05:37:17.560 DEBUG:boto:Token: None 2022-09-03T05:37:17.560 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/obj 2022-09-03T05:37:17.561 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5UJC6NROTd09jozIcBhuJobvrCs= 2022-09-03T05:37:17.561 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5UJC6NROTd09jozIcBhuJobvrCs=', 'Content-Length': '0'} 2022-09-03T05:37:17.565 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008d1fd03c92c38349-006312e80d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.567 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:17.568 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:17.569 DEBUG:boto:Method: HEAD 2022-09-03T05:37:17.570 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:17.571 DEBUG:boto:Data: 2022-09-03T05:37:17.573 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.574 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:17.575 DEBUG:boto:Port: 8000 2022-09-03T05:37:17.576 DEBUG:boto:Params: {} 2022-09-03T05:37:17.577 DEBUG:boto:Token: None 2022-09-03T05:37:17.578 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/obj 2022-09-03T05:37:17.582 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5UJC6NROTd09jozIcBhuJobvrCs= 2022-09-03T05:37:17.583 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5UJC6NROTd09jozIcBhuJobvrCs=', 'Content-Length': '0'} 2022-09-03T05:37:17.586 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001b3ee4b60f379c34-006312e80d-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.587 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:17.588 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:17.588 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:17.590 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.592 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.598 DEBUG:boto:Method: GET 2022-09-03T05:37:17.599 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.600 DEBUG:boto:Data: 2022-09-03T05:37:17.600 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.601 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:17.601 DEBUG:boto:Port: 8001 2022-09-03T05:37:17.602 DEBUG:boto:Params: {} 2022-09-03T05:37:17.603 DEBUG:boto:Token: None 2022-09-03T05:37:17.604 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.605 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kh4PXJ8eBAWQkg5iuXHJcIXud7o= 2022-09-03T05:37:17.608 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kh4PXJ8eBAWQkg5iuXHJcIXud7o=', 'Content-Length': '0'} 2022-09-03T05:37:17.611 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b0e4672902d28f7c-006312e80d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.613 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:17.614 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:17.620 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.621 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.621 DEBUG:boto:Method: GET 2022-09-03T05:37:17.623 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.624 DEBUG:boto:Data: 2022-09-03T05:37:17.625 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.626 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:17.629 DEBUG:boto:Port: 8000 2022-09-03T05:37:17.630 DEBUG:boto:Params: {} 2022-09-03T05:37:17.632 DEBUG:boto:Token: None 2022-09-03T05:37:17.633 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.634 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kh4PXJ8eBAWQkg5iuXHJcIXud7o= 2022-09-03T05:37:17.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kh4PXJ8eBAWQkg5iuXHJcIXud7o=', 'Content-Length': '0'} 2022-09-03T05:37:17.639 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000af180a6d6a4bccf1-006312e80d-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.641 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:17.643 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:17.644 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:17.645 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:17.646 DEBUG:boto:Method: GET 2022-09-03T05:37:17.647 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:17.655 DEBUG:boto:Data: 2022-09-03T05:37:17.655 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.656 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:17.656 DEBUG:boto:Port: 8001 2022-09-03T05:37:17.657 DEBUG:boto:Params: {} 2022-09-03T05:37:17.658 DEBUG:boto:Token: None 2022-09-03T05:37:17.658 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/ 2022-09-03T05:37:17.659 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:drxH8uNd9ALBcDqTSqIV6N+XPmQ= 2022-09-03T05:37:17.659 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:drxH8uNd9ALBcDqTSqIV6N+XPmQ=', 'Content-Length': '0'} 2022-09-03T05:37:17.663 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000090ae14dae7013d3-006312e80d-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.663 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:17.664 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:17.665 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:17.665 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:17.666 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:17.666 DEBUG:boto:Method: GET 2022-09-03T05:37:17.667 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:17.667 DEBUG:boto:Data: 2022-09-03T05:37:17.668 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.668 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:17.669 DEBUG:boto:Port: 8000 2022-09-03T05:37:17.670 DEBUG:boto:Params: {} 2022-09-03T05:37:17.670 DEBUG:boto:Token: None 2022-09-03T05:37:17.671 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/ 2022-09-03T05:37:17.671 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:drxH8uNd9ALBcDqTSqIV6N+XPmQ= 2022-09-03T05:37:17.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:drxH8uNd9ALBcDqTSqIV6N+XPmQ=', 'Content-Length': '0'} 2022-09-03T05:37:17.675 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007cc7f5ef64a86df7-006312e80d-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.676 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:17.678 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:17.678 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:17.679 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:17.680 DEBUG:boto:path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.681 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.682 DEBUG:boto:Method: GET 2022-09-03T05:37:17.685 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.686 DEBUG:boto:Data: 2022-09-03T05:37:17.688 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.689 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:17.690 DEBUG:boto:Port: 8001 2022-09-03T05:37:17.691 DEBUG:boto:Params: {} 2022-09-03T05:37:17.692 DEBUG:boto:Token: None 2022-09-03T05:37:17.693 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.695 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CY9jX/jnes5D0zqlxio4tSUpPG0= 2022-09-03T05:37:17.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CY9jX/jnes5D0zqlxio4tSUpPG0=', 'Content-Length': '0'} 2022-09-03T05:37:17.699 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'hqLZJOZHmRi2EtNletjE-T3uOA2T8pT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000077c69f57c2f5da8a-006312e80d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.700 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:17.702 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:17.703 DEBUG:boto:path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.704 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.705 DEBUG:boto:Method: GET 2022-09-03T05:37:17.706 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.708 DEBUG:boto:Data: 2022-09-03T05:37:17.709 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.710 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:17.711 DEBUG:boto:Port: 8000 2022-09-03T05:37:17.712 DEBUG:boto:Params: {} 2022-09-03T05:37:17.713 DEBUG:boto:Token: None 2022-09-03T05:37:17.714 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.715 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CY9jX/jnes5D0zqlxio4tSUpPG0= 2022-09-03T05:37:17.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CY9jX/jnes5D0zqlxio4tSUpPG0=', 'Content-Length': '0'} 2022-09-03T05:37:17.719 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'hqLZJOZHmRi2EtNletjE-T3uOA2T8pT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004329a0ed376316fe-006312e80d-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.720 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:17.721 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:17.722 DEBUG:boto:path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.723 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.724 DEBUG:boto:Method: HEAD 2022-09-03T05:37:17.725 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.727 DEBUG:boto:Data: 2022-09-03T05:37:17.728 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.729 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:17.729 DEBUG:boto:Port: 8001 2022-09-03T05:37:17.730 DEBUG:boto:Params: {} 2022-09-03T05:37:17.732 DEBUG:boto:Token: None 2022-09-03T05:37:17.733 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.734 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DOy+KysD49Ds0+krXRIFyYVUs6Y= 2022-09-03T05:37:17.735 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DOy+KysD49Ds0+krXRIFyYVUs6Y=', 'Content-Length': '0'} 2022-09-03T05:37:17.740 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'hqLZJOZHmRi2EtNletjE-T3uOA2T8pT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000109c1625ac73ea06-006312e80d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.741 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:17.742 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:17.743 DEBUG:boto:path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.743 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.744 DEBUG:boto:Method: HEAD 2022-09-03T05:37:17.744 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.745 DEBUG:boto:Data: 2022-09-03T05:37:17.745 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.746 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:17.746 DEBUG:boto:Port: 8000 2022-09-03T05:37:17.747 DEBUG:boto:Params: {} 2022-09-03T05:37:17.747 DEBUG:boto:Token: None 2022-09-03T05:37:17.748 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.748 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DOy+KysD49Ds0+krXRIFyYVUs6Y= 2022-09-03T05:37:17.749 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DOy+KysD49Ds0+krXRIFyYVUs6Y=', 'Content-Length': '0'} 2022-09-03T05:37:17.752 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'hqLZJOZHmRi2EtNletjE-T3uOA2T8pT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002cd4403912386cc1-006312e80d-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.753 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:17.754 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:17.755 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:17.755 DEBUG:boto:path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.756 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.757 DEBUG:boto:Method: GET 2022-09-03T05:37:17.758 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.760 DEBUG:boto:Data: 2022-09-03T05:37:17.762 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.762 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:17.763 DEBUG:boto:Port: 8001 2022-09-03T05:37:17.764 DEBUG:boto:Params: {} 2022-09-03T05:37:17.764 DEBUG:boto:Token: None 2022-09-03T05:37:17.765 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.766 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CY9jX/jnes5D0zqlxio4tSUpPG0= 2022-09-03T05:37:17.766 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CY9jX/jnes5D0zqlxio4tSUpPG0=', 'Content-Length': '0'} 2022-09-03T05:37:17.771 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'hqLZJOZHmRi2EtNletjE-T3uOA2T8pT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009d59fb9407c1e1b9-006312e80d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.810 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:17.811 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:17.812 DEBUG:boto:path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.812 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.813 DEBUG:boto:Method: GET 2022-09-03T05:37:17.813 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.814 DEBUG:boto:Data: 2022-09-03T05:37:17.814 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.820 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:17.821 DEBUG:boto:Port: 8000 2022-09-03T05:37:17.822 DEBUG:boto:Params: {} 2022-09-03T05:37:17.823 DEBUG:boto:Token: None 2022-09-03T05:37:17.824 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:17.824 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CY9jX/jnes5D0zqlxio4tSUpPG0= 2022-09-03T05:37:17.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CY9jX/jnes5D0zqlxio4tSUpPG0=', 'Content-Length': '0'} 2022-09-03T05:37:17.830 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'hqLZJOZHmRi2EtNletjE-T3uOA2T8pT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000051c355159701f32d-006312e80d-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.831 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:17.832 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:17.834 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:17.835 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:17.836 DEBUG:boto:Method: GET 2022-09-03T05:37:17.837 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:17.838 DEBUG:boto:Data: 2022-09-03T05:37:17.840 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.841 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:17.842 DEBUG:boto:Port: 8001 2022-09-03T05:37:17.844 DEBUG:boto:Params: {} 2022-09-03T05:37:17.851 DEBUG:boto:Token: None 2022-09-03T05:37:17.854 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/ 2022-09-03T05:37:17.855 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:drxH8uNd9ALBcDqTSqIV6N+XPmQ= 2022-09-03T05:37:17.855 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:drxH8uNd9ALBcDqTSqIV6N+XPmQ=', 'Content-Length': '0'} 2022-09-03T05:37:17.859 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fc3a723d515e5322-006312e80d-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.861 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:17.862 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:17.863 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:17.865 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:17.866 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:17.871 DEBUG:boto:Method: GET 2022-09-03T05:37:17.872 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:17.873 DEBUG:boto:Data: 2022-09-03T05:37:17.873 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.881 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:17.882 DEBUG:boto:Port: 8000 2022-09-03T05:37:17.883 DEBUG:boto:Params: {} 2022-09-03T05:37:17.884 DEBUG:boto:Token: None 2022-09-03T05:37:17.885 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/ 2022-09-03T05:37:17.886 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:drxH8uNd9ALBcDqTSqIV6N+XPmQ= 2022-09-03T05:37:17.887 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:drxH8uNd9ALBcDqTSqIV6N+XPmQ=', 'Content-Length': '0'} 2022-09-03T05:37:17.891 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000806741c4f482df8c-006312e80d-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.892 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:17.894 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:17.895 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:17.896 DEBUG:boto:Method: HEAD 2022-09-03T05:37:17.897 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:17.902 DEBUG:boto:Data: 2022-09-03T05:37:17.904 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.905 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:17.906 DEBUG:boto:Port: 8001 2022-09-03T05:37:17.907 DEBUG:boto:Params: {} 2022-09-03T05:37:17.908 DEBUG:boto:Token: None 2022-09-03T05:37:17.909 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/obj 2022-09-03T05:37:17.910 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5UJC6NROTd09jozIcBhuJobvrCs= 2022-09-03T05:37:17.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5UJC6NROTd09jozIcBhuJobvrCs=', 'Content-Length': '0'} 2022-09-03T05:37:17.916 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ea6278402403450d-006312e80d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.919 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:17.919 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:17.920 DEBUG:boto:Method: HEAD 2022-09-03T05:37:17.921 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:17.922 DEBUG:boto:Data: 2022-09-03T05:37:17.923 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.924 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:17.925 DEBUG:boto:Port: 8000 2022-09-03T05:37:17.925 DEBUG:boto:Params: {} 2022-09-03T05:37:17.927 DEBUG:boto:Token: None 2022-09-03T05:37:17.929 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/obj 2022-09-03T05:37:17.930 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5UJC6NROTd09jozIcBhuJobvrCs= 2022-09-03T05:37:17.931 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5UJC6NROTd09jozIcBhuJobvrCs=', 'Content-Length': '0'} 2022-09-03T05:37:17.937 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001a353f57d2ceb3f8-006312e80d-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.938 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:17.940 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:17.941 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:17.943 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.944 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.945 DEBUG:boto:Method: GET 2022-09-03T05:37:17.946 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.948 DEBUG:boto:Data: 2022-09-03T05:37:17.950 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.951 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:17.951 DEBUG:boto:Port: 8001 2022-09-03T05:37:17.952 DEBUG:boto:Params: {} 2022-09-03T05:37:17.953 DEBUG:boto:Token: None 2022-09-03T05:37:17.955 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.956 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kh4PXJ8eBAWQkg5iuXHJcIXud7o= 2022-09-03T05:37:17.957 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kh4PXJ8eBAWQkg5iuXHJcIXud7o=', 'Content-Length': '0'} 2022-09-03T05:37:17.966 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000833280a5c4703274-006312e80d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.968 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:17.969 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:17.970 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.971 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.972 DEBUG:boto:Method: GET 2022-09-03T05:37:17.973 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.975 DEBUG:boto:Data: 2022-09-03T05:37:17.976 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.978 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:17.979 DEBUG:boto:Port: 8000 2022-09-03T05:37:17.980 DEBUG:boto:Params: {} 2022-09-03T05:37:17.982 DEBUG:boto:Token: None 2022-09-03T05:37:17.983 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:17 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:17.984 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kh4PXJ8eBAWQkg5iuXHJcIXud7o= 2022-09-03T05:37:17.985 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kh4PXJ8eBAWQkg5iuXHJcIXud7o=', 'Content-Length': '0'} 2022-09-03T05:37:17.988 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002eb835484483f207-006312e80d-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:17.990 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:17.991 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:17.993 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:17.994 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:17.995 DEBUG:boto:Method: GET 2022-09-03T05:37:17.995 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:17.996 DEBUG:boto:Data: 2022-09-03T05:37:17.997 DEBUG:boto:Headers: {} 2022-09-03T05:37:17.998 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:17.999 DEBUG:boto:Port: 8001 2022-09-03T05:37:18.000 DEBUG:boto:Params: {} 2022-09-03T05:37:18.002 DEBUG:boto:Token: None 2022-09-03T05:37:18.003 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/ 2022-09-03T05:37:18.004 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos= 2022-09-03T05:37:18.006 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos=', 'Content-Length': '0'} 2022-09-03T05:37:18.010 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a8544ad5caab9c10-006312e80e-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.011 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:18.012 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:18.012 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:18.013 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.013 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.014 DEBUG:boto:Method: GET 2022-09-03T05:37:18.014 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.015 DEBUG:boto:Data: 2022-09-03T05:37:18.015 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.018 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:18.019 DEBUG:boto:Port: 8000 2022-09-03T05:37:18.020 DEBUG:boto:Params: {} 2022-09-03T05:37:18.021 DEBUG:boto:Token: None 2022-09-03T05:37:18.023 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/ 2022-09-03T05:37:18.024 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos= 2022-09-03T05:37:18.025 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos=', 'Content-Length': '0'} 2022-09-03T05:37:18.028 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000006e3cbcb0f221265-006312e80e-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.029 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:18.031 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:18.032 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.033 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.034 DEBUG:boto:path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.035 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.036 DEBUG:boto:Method: GET 2022-09-03T05:37:18.037 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.038 DEBUG:boto:Data: 2022-09-03T05:37:18.039 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.040 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:18.040 DEBUG:boto:Port: 8001 2022-09-03T05:37:18.041 DEBUG:boto:Params: {} 2022-09-03T05:37:18.041 DEBUG:boto:Token: None 2022-09-03T05:37:18.043 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.043 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gZh52EC24invZrza8ZRs17P+/7I= 2022-09-03T05:37:18.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gZh52EC24invZrza8ZRs17P+/7I=', 'Content-Length': '0'} 2022-09-03T05:37:18.046 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'mBYgYYD3drKvdzws.hh3-36xKvL9eTH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009f78dc24f19900d0-006312e80e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.047 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.048 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.050 DEBUG:boto:path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.051 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.052 DEBUG:boto:Method: GET 2022-09-03T05:37:18.053 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.053 DEBUG:boto:Data: 2022-09-03T05:37:18.054 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.055 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:18.056 DEBUG:boto:Port: 8000 2022-09-03T05:37:18.057 DEBUG:boto:Params: {} 2022-09-03T05:37:18.058 DEBUG:boto:Token: None 2022-09-03T05:37:18.059 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.060 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gZh52EC24invZrza8ZRs17P+/7I= 2022-09-03T05:37:18.061 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gZh52EC24invZrza8ZRs17P+/7I=', 'Content-Length': '0'} 2022-09-03T05:37:18.067 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'mBYgYYD3drKvdzws.hh3-36xKvL9eTH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000092ebb9e9184a262-006312e80e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.110 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.111 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.112 DEBUG:boto:path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.113 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.114 DEBUG:boto:Method: HEAD 2022-09-03T05:37:18.116 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.118 DEBUG:boto:Data: 2022-09-03T05:37:18.119 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.120 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:18.122 DEBUG:boto:Port: 8001 2022-09-03T05:37:18.123 DEBUG:boto:Params: {} 2022-09-03T05:37:18.124 DEBUG:boto:Token: None 2022-09-03T05:37:18.124 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.125 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gkJ80TYHspl6v3P4Xg4XVQEhhnA= 2022-09-03T05:37:18.126 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gkJ80TYHspl6v3P4Xg4XVQEhhnA=', 'Content-Length': '0'} 2022-09-03T05:37:18.128 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'mBYgYYD3drKvdzws.hh3-36xKvL9eTH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b4603d5163353444-006312e80e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.129 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.129 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.130 DEBUG:boto:path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.131 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.131 DEBUG:boto:Method: HEAD 2022-09-03T05:37:18.133 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.134 DEBUG:boto:Data: 2022-09-03T05:37:18.135 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.136 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:18.137 DEBUG:boto:Port: 8000 2022-09-03T05:37:18.138 DEBUG:boto:Params: {} 2022-09-03T05:37:18.139 DEBUG:boto:Token: None 2022-09-03T05:37:18.139 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.141 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gkJ80TYHspl6v3P4Xg4XVQEhhnA= 2022-09-03T05:37:18.142 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gkJ80TYHspl6v3P4Xg4XVQEhhnA=', 'Content-Length': '0'} 2022-09-03T05:37:18.145 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'mBYgYYD3drKvdzws.hh3-36xKvL9eTH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000064180ab7bc70ae74-006312e80e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.146 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:18.146 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.147 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.148 DEBUG:boto:path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.148 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.149 DEBUG:boto:Method: GET 2022-09-03T05:37:18.149 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.150 DEBUG:boto:Data: 2022-09-03T05:37:18.150 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.151 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:18.151 DEBUG:boto:Port: 8001 2022-09-03T05:37:18.155 DEBUG:boto:Params: {} 2022-09-03T05:37:18.157 DEBUG:boto:Token: None 2022-09-03T05:37:18.158 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.159 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gZh52EC24invZrza8ZRs17P+/7I= 2022-09-03T05:37:18.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gZh52EC24invZrza8ZRs17P+/7I=', 'Content-Length': '0'} 2022-09-03T05:37:18.163 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'mBYgYYD3drKvdzws.hh3-36xKvL9eTH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fe96cec4c36a6b39-006312e80e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.165 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.166 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.167 DEBUG:boto:path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.168 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.169 DEBUG:boto:Method: GET 2022-09-03T05:37:18.170 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.171 DEBUG:boto:Data: 2022-09-03T05:37:18.172 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.173 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:18.174 DEBUG:boto:Port: 8000 2022-09-03T05:37:18.175 DEBUG:boto:Params: {} 2022-09-03T05:37:18.176 DEBUG:boto:Token: None 2022-09-03T05:37:18.177 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:18.179 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gZh52EC24invZrza8ZRs17P+/7I= 2022-09-03T05:37:18.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gZh52EC24invZrza8ZRs17P+/7I=', 'Content-Length': '0'} 2022-09-03T05:37:18.192 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'mBYgYYD3drKvdzws.hh3-36xKvL9eTH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005397b82acd1a99c5-006312e80e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.193 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:18.194 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:18.195 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.195 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.196 DEBUG:boto:Method: GET 2022-09-03T05:37:18.200 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.201 DEBUG:boto:Data: 2022-09-03T05:37:18.201 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.203 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:18.203 DEBUG:boto:Port: 8001 2022-09-03T05:37:18.204 DEBUG:boto:Params: {} 2022-09-03T05:37:18.205 DEBUG:boto:Token: None 2022-09-03T05:37:18.205 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/ 2022-09-03T05:37:18.206 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos= 2022-09-03T05:37:18.207 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos=', 'Content-Length': '0'} 2022-09-03T05:37:18.210 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002cf75f7b155e3bf9-006312e80e-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.211 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:18.212 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:18.213 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:18.213 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.214 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.214 DEBUG:boto:Method: GET 2022-09-03T05:37:18.215 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.216 DEBUG:boto:Data: 2022-09-03T05:37:18.216 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.217 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:18.217 DEBUG:boto:Port: 8000 2022-09-03T05:37:18.218 DEBUG:boto:Params: {} 2022-09-03T05:37:18.219 DEBUG:boto:Token: None 2022-09-03T05:37:18.219 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/ 2022-09-03T05:37:18.220 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos= 2022-09-03T05:37:18.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos=', 'Content-Length': '0'} 2022-09-03T05:37:18.227 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000031924e80b27728c9-006312e80e-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.228 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:18.230 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.232 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.233 DEBUG:boto:Method: HEAD 2022-09-03T05:37:18.235 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:18.236 DEBUG:boto:Data: 2022-09-03T05:37:18.238 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.238 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:18.239 DEBUG:boto:Port: 8001 2022-09-03T05:37:18.240 DEBUG:boto:Params: {} 2022-09-03T05:37:18.241 DEBUG:boto:Token: None 2022-09-03T05:37:18.242 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj 2022-09-03T05:37:18.242 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aiFTt3tdgNfMhqep4M+oUOgBRmc= 2022-09-03T05:37:18.243 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aiFTt3tdgNfMhqep4M+oUOgBRmc=', 'Content-Length': '0'} 2022-09-03T05:37:18.247 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003e19359c59b957bb-006312e80e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.249 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.250 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.251 DEBUG:boto:Method: HEAD 2022-09-03T05:37:18.253 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:18.254 DEBUG:boto:Data: 2022-09-03T05:37:18.255 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.256 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:18.257 DEBUG:boto:Port: 8000 2022-09-03T05:37:18.258 DEBUG:boto:Params: {} 2022-09-03T05:37:18.261 DEBUG:boto:Token: None 2022-09-03T05:37:18.262 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj 2022-09-03T05:37:18.263 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aiFTt3tdgNfMhqep4M+oUOgBRmc= 2022-09-03T05:37:18.263 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aiFTt3tdgNfMhqep4M+oUOgBRmc=', 'Content-Length': '0'} 2022-09-03T05:37:18.270 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000095f4b1ef8fcb6316-006312e80e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.271 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:18.272 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.272 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.273 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:18.274 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:18.275 DEBUG:boto:Method: GET 2022-09-03T05:37:18.275 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:18.276 DEBUG:boto:Data: 2022-09-03T05:37:18.277 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.279 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:18.280 DEBUG:boto:Port: 8001 2022-09-03T05:37:18.281 DEBUG:boto:Params: {} 2022-09-03T05:37:18.281 DEBUG:boto:Token: None 2022-09-03T05:37:18.282 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:18.283 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6hcLKVT89Bz7Ajrq9Xrh3K9PO48= 2022-09-03T05:37:18.283 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6hcLKVT89Bz7Ajrq9Xrh3K9PO48=', 'Content-Length': '0'} 2022-09-03T05:37:18.285 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000c3ea42a181d4059-006312e80e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.326 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.327 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.328 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:18.328 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:18.329 DEBUG:boto:Method: GET 2022-09-03T05:37:18.329 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:18.330 DEBUG:boto:Data: 2022-09-03T05:37:18.330 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.331 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:18.331 DEBUG:boto:Port: 8000 2022-09-03T05:37:18.332 DEBUG:boto:Params: {} 2022-09-03T05:37:18.333 DEBUG:boto:Token: None 2022-09-03T05:37:18.333 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:18.334 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6hcLKVT89Bz7Ajrq9Xrh3K9PO48= 2022-09-03T05:37:18.335 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6hcLKVT89Bz7Ajrq9Xrh3K9PO48=', 'Content-Length': '0'} 2022-09-03T05:37:18.338 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000473f4c81fc76007-006312e80e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.339 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:18.340 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:18.340 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.341 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.341 DEBUG:boto:Method: GET 2022-09-03T05:37:18.342 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.342 DEBUG:boto:Data: 2022-09-03T05:37:18.343 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.344 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:18.344 DEBUG:boto:Port: 8001 2022-09-03T05:37:18.345 DEBUG:boto:Params: {} 2022-09-03T05:37:18.345 DEBUG:boto:Token: None 2022-09-03T05:37:18.346 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/ 2022-09-03T05:37:18.346 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos= 2022-09-03T05:37:18.347 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos=', 'Content-Length': '0'} 2022-09-03T05:37:18.350 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009594e192ec79d277-006312e80e-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.351 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:18.352 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:18.353 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:18.354 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.354 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.355 DEBUG:boto:Method: GET 2022-09-03T05:37:18.355 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.356 DEBUG:boto:Data: 2022-09-03T05:37:18.356 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.357 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:18.358 DEBUG:boto:Port: 8000 2022-09-03T05:37:18.358 DEBUG:boto:Params: {} 2022-09-03T05:37:18.359 DEBUG:boto:Token: None 2022-09-03T05:37:18.360 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/ 2022-09-03T05:37:18.360 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos= 2022-09-03T05:37:18.361 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos=', 'Content-Length': '0'} 2022-09-03T05:37:18.364 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005bc647657ddaf892-006312e80e-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.409 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:18.411 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:18.412 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.413 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.414 DEBUG:boto:path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.415 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.416 DEBUG:boto:Method: GET 2022-09-03T05:37:18.417 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.418 DEBUG:boto:Data: 2022-09-03T05:37:18.419 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.420 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:18.421 DEBUG:boto:Port: 8001 2022-09-03T05:37:18.422 DEBUG:boto:Params: {} 2022-09-03T05:37:18.424 DEBUG:boto:Token: None 2022-09-03T05:37:18.425 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.426 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ciVXhWk1EzBmBI+DHwdBDhfmGNg= 2022-09-03T05:37:18.427 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ciVXhWk1EzBmBI+DHwdBDhfmGNg=', 'Content-Length': '0'} 2022-09-03T05:37:18.431 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000706e4e77f20bd241-006312e80e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.433 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.434 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.435 DEBUG:boto:path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.436 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.437 DEBUG:boto:Method: GET 2022-09-03T05:37:18.438 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.439 DEBUG:boto:Data: 2022-09-03T05:37:18.440 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.441 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:18.442 DEBUG:boto:Port: 8000 2022-09-03T05:37:18.443 DEBUG:boto:Params: {} 2022-09-03T05:37:18.445 DEBUG:boto:Token: None 2022-09-03T05:37:18.446 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.446 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ciVXhWk1EzBmBI+DHwdBDhfmGNg= 2022-09-03T05:37:18.447 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ciVXhWk1EzBmBI+DHwdBDhfmGNg=', 'Content-Length': '0'} 2022-09-03T05:37:18.450 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006981665c36ca5f0e-006312e80e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.452 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.453 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.454 DEBUG:boto:path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.455 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.456 DEBUG:boto:Method: HEAD 2022-09-03T05:37:18.457 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.458 DEBUG:boto:Data: 2022-09-03T05:37:18.459 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.461 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:18.463 DEBUG:boto:Port: 8001 2022-09-03T05:37:18.464 DEBUG:boto:Params: {} 2022-09-03T05:37:18.465 DEBUG:boto:Token: None 2022-09-03T05:37:18.466 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.466 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5QdAUzKF4hnD0lqCNkvn4PIcmHQ= 2022-09-03T05:37:18.467 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5QdAUzKF4hnD0lqCNkvn4PIcmHQ=', 'Content-Length': '0'} 2022-09-03T05:37:18.470 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008440ce57e454befd-006312e80e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.471 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.472 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.473 DEBUG:boto:path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.473 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.474 DEBUG:boto:Method: HEAD 2022-09-03T05:37:18.475 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.476 DEBUG:boto:Data: 2022-09-03T05:37:18.476 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.477 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:18.478 DEBUG:boto:Port: 8000 2022-09-03T05:37:18.479 DEBUG:boto:Params: {} 2022-09-03T05:37:18.481 DEBUG:boto:Token: None 2022-09-03T05:37:18.482 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5QdAUzKF4hnD0lqCNkvn4PIcmHQ= 2022-09-03T05:37:18.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5QdAUzKF4hnD0lqCNkvn4PIcmHQ=', 'Content-Length': '0'} 2022-09-03T05:37:18.490 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f6c0980361798ac0-006312e80e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.492 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:18.493 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.493 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.494 DEBUG:boto:path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.494 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.495 DEBUG:boto:Method: GET 2022-09-03T05:37:18.496 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.497 DEBUG:boto:Data: 2022-09-03T05:37:18.498 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.499 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:18.499 DEBUG:boto:Port: 8001 2022-09-03T05:37:18.500 DEBUG:boto:Params: {} 2022-09-03T05:37:18.501 DEBUG:boto:Token: None 2022-09-03T05:37:18.502 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.502 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ciVXhWk1EzBmBI+DHwdBDhfmGNg= 2022-09-03T05:37:18.503 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ciVXhWk1EzBmBI+DHwdBDhfmGNg=', 'Content-Length': '0'} 2022-09-03T05:37:18.506 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fd47248a9e47871d-006312e80e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.507 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.508 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.509 DEBUG:boto:path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.510 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.511 DEBUG:boto:Method: GET 2022-09-03T05:37:18.512 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.513 DEBUG:boto:Data: 2022-09-03T05:37:18.513 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.514 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:18.514 DEBUG:boto:Port: 8000 2022-09-03T05:37:18.515 DEBUG:boto:Params: {} 2022-09-03T05:37:18.516 DEBUG:boto:Token: None 2022-09-03T05:37:18.516 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:18.517 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ciVXhWk1EzBmBI+DHwdBDhfmGNg= 2022-09-03T05:37:18.518 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ciVXhWk1EzBmBI+DHwdBDhfmGNg=', 'Content-Length': '0'} 2022-09-03T05:37:18.520 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005b9f14c963a08352-006312e80e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.521 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:18.522 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:18.523 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.523 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.524 DEBUG:boto:Method: GET 2022-09-03T05:37:18.525 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.526 DEBUG:boto:Data: 2022-09-03T05:37:18.526 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.527 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:18.527 DEBUG:boto:Port: 8001 2022-09-03T05:37:18.528 DEBUG:boto:Params: {} 2022-09-03T05:37:18.529 DEBUG:boto:Token: None 2022-09-03T05:37:18.529 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/ 2022-09-03T05:37:18.530 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos= 2022-09-03T05:37:18.531 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos=', 'Content-Length': '0'} 2022-09-03T05:37:18.534 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000097a267106fb1a359-006312e80e-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.575 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:18.577 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:18.579 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:18.580 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.581 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.581 DEBUG:boto:Method: GET 2022-09-03T05:37:18.582 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.583 DEBUG:boto:Data: 2022-09-03T05:37:18.583 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.584 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:18.584 DEBUG:boto:Port: 8000 2022-09-03T05:37:18.585 DEBUG:boto:Params: {} 2022-09-03T05:37:18.586 DEBUG:boto:Token: None 2022-09-03T05:37:18.586 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/ 2022-09-03T05:37:18.587 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos= 2022-09-03T05:37:18.588 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos=', 'Content-Length': '0'} 2022-09-03T05:37:18.598 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007a4dfeebc40039ea-006312e80e-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.600 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:18.600 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.601 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.602 DEBUG:boto:Method: HEAD 2022-09-03T05:37:18.602 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:18.604 DEBUG:boto:Data: 2022-09-03T05:37:18.605 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.606 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:18.607 DEBUG:boto:Port: 8001 2022-09-03T05:37:18.608 DEBUG:boto:Params: {} 2022-09-03T05:37:18.609 DEBUG:boto:Token: None 2022-09-03T05:37:18.610 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj 2022-09-03T05:37:18.611 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aiFTt3tdgNfMhqep4M+oUOgBRmc= 2022-09-03T05:37:18.612 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aiFTt3tdgNfMhqep4M+oUOgBRmc=', 'Content-Length': '0'} 2022-09-03T05:37:18.617 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000028428168ab728e4c-006312e80e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.625 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.627 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.627 DEBUG:boto:Method: HEAD 2022-09-03T05:37:18.628 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:18.629 DEBUG:boto:Data: 2022-09-03T05:37:18.630 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.630 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:18.631 DEBUG:boto:Port: 8000 2022-09-03T05:37:18.631 DEBUG:boto:Params: {} 2022-09-03T05:37:18.632 DEBUG:boto:Token: None 2022-09-03T05:37:18.633 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj 2022-09-03T05:37:18.633 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aiFTt3tdgNfMhqep4M+oUOgBRmc= 2022-09-03T05:37:18.634 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aiFTt3tdgNfMhqep4M+oUOgBRmc=', 'Content-Length': '0'} 2022-09-03T05:37:18.638 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000a2712466da5a7f1-006312e80e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.639 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:18.640 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.641 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.642 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:18.643 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:18.644 DEBUG:boto:Method: GET 2022-09-03T05:37:18.645 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:18.646 DEBUG:boto:Data: 2022-09-03T05:37:18.647 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.648 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:18.649 DEBUG:boto:Port: 8001 2022-09-03T05:37:18.650 DEBUG:boto:Params: {} 2022-09-03T05:37:18.651 DEBUG:boto:Token: None 2022-09-03T05:37:18.652 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:18.653 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6hcLKVT89Bz7Ajrq9Xrh3K9PO48= 2022-09-03T05:37:18.654 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6hcLKVT89Bz7Ajrq9Xrh3K9PO48=', 'Content-Length': '0'} 2022-09-03T05:37:18.662 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d5cf1c61c967c621-006312e80e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.664 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.665 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.666 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:18.667 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:18.668 DEBUG:boto:Method: GET 2022-09-03T05:37:18.669 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:18.671 DEBUG:boto:Data: 2022-09-03T05:37:18.671 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.673 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:18.674 DEBUG:boto:Port: 8000 2022-09-03T05:37:18.675 DEBUG:boto:Params: {} 2022-09-03T05:37:18.676 DEBUG:boto:Token: None 2022-09-03T05:37:18.677 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:18.678 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6hcLKVT89Bz7Ajrq9Xrh3K9PO48= 2022-09-03T05:37:18.681 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6hcLKVT89Bz7Ajrq9Xrh3K9PO48=', 'Content-Length': '0'} 2022-09-03T05:37:18.684 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d43322f901b0f120-006312e80e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.685 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:18.686 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:18.686 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.687 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.688 DEBUG:boto:Method: GET 2022-09-03T05:37:18.689 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.690 DEBUG:boto:Data: 2022-09-03T05:37:18.691 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.692 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:18.693 DEBUG:boto:Port: 8001 2022-09-03T05:37:18.693 DEBUG:boto:Params: {} 2022-09-03T05:37:18.694 DEBUG:boto:Token: None 2022-09-03T05:37:18.695 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/ 2022-09-03T05:37:18.696 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos= 2022-09-03T05:37:18.698 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos=', 'Content-Length': '0'} 2022-09-03T05:37:18.702 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008c847a90e905fc97-006312e80e-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.704 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:18.705 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:18.706 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:18.707 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.708 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.709 DEBUG:boto:Method: GET 2022-09-03T05:37:18.711 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.712 DEBUG:boto:Data: 2022-09-03T05:37:18.712 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.713 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:18.714 DEBUG:boto:Port: 8000 2022-09-03T05:37:18.714 DEBUG:boto:Params: {} 2022-09-03T05:37:18.715 DEBUG:boto:Token: None 2022-09-03T05:37:18.716 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/ 2022-09-03T05:37:18.716 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos= 2022-09-03T05:37:18.717 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos=', 'Content-Length': '0'} 2022-09-03T05:37:18.720 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008fa543f0178bd242-006312e80e-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.721 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:18.722 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:18.722 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.723 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.723 DEBUG:boto:path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.724 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.724 DEBUG:boto:Method: GET 2022-09-03T05:37:18.725 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.725 DEBUG:boto:Data: 2022-09-03T05:37:18.726 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.727 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:18.727 DEBUG:boto:Port: 8001 2022-09-03T05:37:18.728 DEBUG:boto:Params: {} 2022-09-03T05:37:18.728 DEBUG:boto:Token: None 2022-09-03T05:37:18.731 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.734 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nuaCU8LmY2AESYDPpllBTHjE0PU= 2022-09-03T05:37:18.735 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nuaCU8LmY2AESYDPpllBTHjE0PU=', 'Content-Length': '0'} 2022-09-03T05:37:18.737 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007c70fae6490e4204-006312e80e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.778 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.779 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.780 DEBUG:boto:path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.781 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.782 DEBUG:boto:Method: GET 2022-09-03T05:37:18.783 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.784 DEBUG:boto:Data: 2022-09-03T05:37:18.785 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.786 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:18.787 DEBUG:boto:Port: 8000 2022-09-03T05:37:18.787 DEBUG:boto:Params: {} 2022-09-03T05:37:18.789 DEBUG:boto:Token: None 2022-09-03T05:37:18.790 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.791 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nuaCU8LmY2AESYDPpllBTHjE0PU= 2022-09-03T05:37:18.791 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nuaCU8LmY2AESYDPpllBTHjE0PU=', 'Content-Length': '0'} 2022-09-03T05:37:18.794 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008d890a3a09daff7c-006312e80e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.796 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.797 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.798 DEBUG:boto:path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.798 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.799 DEBUG:boto:Method: HEAD 2022-09-03T05:37:18.801 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.802 DEBUG:boto:Data: 2022-09-03T05:37:18.802 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.803 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:18.805 DEBUG:boto:Port: 8001 2022-09-03T05:37:18.806 DEBUG:boto:Params: {} 2022-09-03T05:37:18.807 DEBUG:boto:Token: None 2022-09-03T05:37:18.808 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.809 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ND/FUy33X8bqpTYvsiyA4dK69M4= 2022-09-03T05:37:18.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ND/FUy33X8bqpTYvsiyA4dK69M4=', 'Content-Length': '0'} 2022-09-03T05:37:18.812 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000878107a5b2383cdf-006312e80e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.813 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.814 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.815 DEBUG:boto:path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.815 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.816 DEBUG:boto:Method: HEAD 2022-09-03T05:37:18.816 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.817 DEBUG:boto:Data: 2022-09-03T05:37:18.817 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.818 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:18.819 DEBUG:boto:Port: 8000 2022-09-03T05:37:18.820 DEBUG:boto:Params: {} 2022-09-03T05:37:18.821 DEBUG:boto:Token: None 2022-09-03T05:37:18.822 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.823 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ND/FUy33X8bqpTYvsiyA4dK69M4= 2022-09-03T05:37:18.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ND/FUy33X8bqpTYvsiyA4dK69M4=', 'Content-Length': '0'} 2022-09-03T05:37:18.835 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000257bdbe9b756b7bd-006312e80e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.839 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:18.840 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.841 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.842 DEBUG:boto:path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.843 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.844 DEBUG:boto:Method: GET 2022-09-03T05:37:18.845 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.846 DEBUG:boto:Data: 2022-09-03T05:37:18.848 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.849 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:18.850 DEBUG:boto:Port: 8001 2022-09-03T05:37:18.852 DEBUG:boto:Params: {} 2022-09-03T05:37:18.853 DEBUG:boto:Token: None 2022-09-03T05:37:18.855 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.856 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nuaCU8LmY2AESYDPpllBTHjE0PU= 2022-09-03T05:37:18.857 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nuaCU8LmY2AESYDPpllBTHjE0PU=', 'Content-Length': '0'} 2022-09-03T05:37:18.861 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004912109e5dbb3add-006312e80e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.862 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.863 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.864 DEBUG:boto:path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.865 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.866 DEBUG:boto:Method: GET 2022-09-03T05:37:18.867 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.868 DEBUG:boto:Data: 2022-09-03T05:37:18.869 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.871 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:18.872 DEBUG:boto:Port: 8000 2022-09-03T05:37:18.873 DEBUG:boto:Params: {} 2022-09-03T05:37:18.875 DEBUG:boto:Token: None 2022-09-03T05:37:18.876 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:18.877 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nuaCU8LmY2AESYDPpllBTHjE0PU= 2022-09-03T05:37:18.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nuaCU8LmY2AESYDPpllBTHjE0PU=', 'Content-Length': '0'} 2022-09-03T05:37:18.882 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c362dd9c11fe3bff-006312e80e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.884 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:18.885 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:18.886 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.887 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.888 DEBUG:boto:Method: GET 2022-09-03T05:37:18.889 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.890 DEBUG:boto:Data: 2022-09-03T05:37:18.891 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.892 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:18.893 DEBUG:boto:Port: 8001 2022-09-03T05:37:18.894 DEBUG:boto:Params: {} 2022-09-03T05:37:18.895 DEBUG:boto:Token: None 2022-09-03T05:37:18.896 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/ 2022-09-03T05:37:18.897 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos= 2022-09-03T05:37:18.898 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos=', 'Content-Length': '0'} 2022-09-03T05:37:18.903 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e2d09dfd5129dceb-006312e80e-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.904 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:18.906 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:18.907 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:18.908 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.910 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.911 DEBUG:boto:Method: GET 2022-09-03T05:37:18.912 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:18.913 DEBUG:boto:Data: 2022-09-03T05:37:18.914 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.916 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:18.917 DEBUG:boto:Port: 8000 2022-09-03T05:37:18.918 DEBUG:boto:Params: {} 2022-09-03T05:37:18.919 DEBUG:boto:Token: None 2022-09-03T05:37:18.920 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/ 2022-09-03T05:37:18.922 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos= 2022-09-03T05:37:18.923 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XLFpBAPalo4o30Lbp7ZZX94nEos=', 'Content-Length': '0'} 2022-09-03T05:37:18.928 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b3cbab09b1685481-006312e80e-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.929 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:18.931 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.932 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.932 DEBUG:boto:Method: HEAD 2022-09-03T05:37:18.933 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:18.934 DEBUG:boto:Data: 2022-09-03T05:37:18.939 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.940 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:18.940 DEBUG:boto:Port: 8001 2022-09-03T05:37:18.941 DEBUG:boto:Params: {} 2022-09-03T05:37:18.941 DEBUG:boto:Token: None 2022-09-03T05:37:18.942 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj 2022-09-03T05:37:18.943 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aiFTt3tdgNfMhqep4M+oUOgBRmc= 2022-09-03T05:37:18.943 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aiFTt3tdgNfMhqep4M+oUOgBRmc=', 'Content-Length': '0'} 2022-09-03T05:37:18.949 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e2816909322bb212-006312e80e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.951 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.952 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.953 DEBUG:boto:Method: HEAD 2022-09-03T05:37:18.954 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:18.955 DEBUG:boto:Data: 2022-09-03T05:37:18.955 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.956 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:18.956 DEBUG:boto:Port: 8000 2022-09-03T05:37:18.957 DEBUG:boto:Params: {} 2022-09-03T05:37:18.958 DEBUG:boto:Token: None 2022-09-03T05:37:18.958 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj 2022-09-03T05:37:18.959 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aiFTt3tdgNfMhqep4M+oUOgBRmc= 2022-09-03T05:37:18.960 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aiFTt3tdgNfMhqep4M+oUOgBRmc=', 'Content-Length': '0'} 2022-09-03T05:37:18.970 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000725a176328042927-006312e80e-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:18.972 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:18.973 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:18.974 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:18.974 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:18.975 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:18.975 DEBUG:boto:Method: GET 2022-09-03T05:37:18.976 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:18.976 DEBUG:boto:Data: 2022-09-03T05:37:18.977 DEBUG:boto:Headers: {} 2022-09-03T05:37:18.978 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:18.978 DEBUG:boto:Port: 8001 2022-09-03T05:37:18.979 DEBUG:boto:Params: {} 2022-09-03T05:37:18.979 DEBUG:boto:Token: None 2022-09-03T05:37:18.980 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:18 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:18.980 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6hcLKVT89Bz7Ajrq9Xrh3K9PO48= 2022-09-03T05:37:18.981 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6hcLKVT89Bz7Ajrq9Xrh3K9PO48=', 'Content-Length': '0'} 2022-09-03T05:37:18.984 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000068431164754b27bd-006312e80e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.026 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.027 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.028 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:19.029 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:19.029 DEBUG:boto:Method: GET 2022-09-03T05:37:19.030 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:19.031 DEBUG:boto:Data: 2022-09-03T05:37:19.032 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.033 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:19.034 DEBUG:boto:Port: 8000 2022-09-03T05:37:19.035 DEBUG:boto:Params: {} 2022-09-03T05:37:19.036 DEBUG:boto:Token: None 2022-09-03T05:37:19.038 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:19.039 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YpSOmvHx2UlScSKc4sk6eDWUkBM= 2022-09-03T05:37:19.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YpSOmvHx2UlScSKc4sk6eDWUkBM=', 'Content-Length': '0'} 2022-09-03T05:37:19.043 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000180c84e8dba200ab-006312e80f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.044 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:19.045 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:19.046 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.047 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.048 DEBUG:boto:Method: GET 2022-09-03T05:37:19.050 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.051 DEBUG:boto:Data: 2022-09-03T05:37:19.052 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.061 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:19.062 DEBUG:boto:Port: 8001 2022-09-03T05:37:19.063 DEBUG:boto:Params: {} 2022-09-03T05:37:19.064 DEBUG:boto:Token: None 2022-09-03T05:37:19.066 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/ 2022-09-03T05:37:19.067 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A= 2022-09-03T05:37:19.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A=', 'Content-Length': '0'} 2022-09-03T05:37:19.075 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007d9063f3aee48992-006312e80f-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.077 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:19.077 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:19.078 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:19.079 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.079 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.080 DEBUG:boto:Method: GET 2022-09-03T05:37:19.080 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.081 DEBUG:boto:Data: 2022-09-03T05:37:19.082 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.082 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:19.083 DEBUG:boto:Port: 8000 2022-09-03T05:37:19.086 DEBUG:boto:Params: {} 2022-09-03T05:37:19.087 DEBUG:boto:Token: None 2022-09-03T05:37:19.089 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/ 2022-09-03T05:37:19.091 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A= 2022-09-03T05:37:19.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A=', 'Content-Length': '0'} 2022-09-03T05:37:19.107 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004dbb80606ae5d3b9-006312e80f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.108 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:19.110 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:19.111 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.112 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.113 DEBUG:boto:path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.114 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.115 DEBUG:boto:Method: GET 2022-09-03T05:37:19.116 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.117 DEBUG:boto:Data: 2022-09-03T05:37:19.118 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.119 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:19.120 DEBUG:boto:Port: 8001 2022-09-03T05:37:19.121 DEBUG:boto:Params: {} 2022-09-03T05:37:19.122 DEBUG:boto:Token: None 2022-09-03T05:37:19.123 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.124 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:udqXOX0n82LighSEhXxd5Pptxto= 2022-09-03T05:37:19.125 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:udqXOX0n82LighSEhXxd5Pptxto=', 'Content-Length': '0'} 2022-09-03T05:37:19.127 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000012dbe36878ee8ab3-006312e80f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.128 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.129 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.129 DEBUG:boto:path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.130 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.130 DEBUG:boto:Method: GET 2022-09-03T05:37:19.131 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.132 DEBUG:boto:Data: 2022-09-03T05:37:19.132 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.133 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:19.133 DEBUG:boto:Port: 8000 2022-09-03T05:37:19.134 DEBUG:boto:Params: {} 2022-09-03T05:37:19.137 DEBUG:boto:Token: None 2022-09-03T05:37:19.138 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.138 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:udqXOX0n82LighSEhXxd5Pptxto= 2022-09-03T05:37:19.139 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:udqXOX0n82LighSEhXxd5Pptxto=', 'Content-Length': '0'} 2022-09-03T05:37:19.141 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005bc73ee57cc90393-006312e80f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.142 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.143 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.143 DEBUG:boto:path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.144 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.145 DEBUG:boto:Method: HEAD 2022-09-03T05:37:19.145 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.146 DEBUG:boto:Data: 2022-09-03T05:37:19.146 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.147 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:19.147 DEBUG:boto:Port: 8001 2022-09-03T05:37:19.148 DEBUG:boto:Params: {} 2022-09-03T05:37:19.148 DEBUG:boto:Token: None 2022-09-03T05:37:19.150 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.151 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GrO7g8UISOfKFML3j8695oH/7fk= 2022-09-03T05:37:19.151 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GrO7g8UISOfKFML3j8695oH/7fk=', 'Content-Length': '0'} 2022-09-03T05:37:19.157 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000657b386ce37f8532-006312e80f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.157 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.158 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.159 DEBUG:boto:path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.159 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.160 DEBUG:boto:Method: HEAD 2022-09-03T05:37:19.160 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.161 DEBUG:boto:Data: 2022-09-03T05:37:19.161 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.162 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:19.162 DEBUG:boto:Port: 8000 2022-09-03T05:37:19.163 DEBUG:boto:Params: {} 2022-09-03T05:37:19.164 DEBUG:boto:Token: None 2022-09-03T05:37:19.164 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.165 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GrO7g8UISOfKFML3j8695oH/7fk= 2022-09-03T05:37:19.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GrO7g8UISOfKFML3j8695oH/7fk=', 'Content-Length': '0'} 2022-09-03T05:37:19.168 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e0608a17c295a994-006312e80f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.168 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:19.169 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.170 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.170 DEBUG:boto:path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.171 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.171 DEBUG:boto:Method: GET 2022-09-03T05:37:19.172 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.172 DEBUG:boto:Data: 2022-09-03T05:37:19.173 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.173 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:19.174 DEBUG:boto:Port: 8001 2022-09-03T05:37:19.174 DEBUG:boto:Params: {} 2022-09-03T05:37:19.175 DEBUG:boto:Token: None 2022-09-03T05:37:19.176 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.176 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:udqXOX0n82LighSEhXxd5Pptxto= 2022-09-03T05:37:19.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:udqXOX0n82LighSEhXxd5Pptxto=', 'Content-Length': '0'} 2022-09-03T05:37:19.179 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f0980fe1dc190640-006312e80f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.181 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.183 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.184 DEBUG:boto:path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.186 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.187 DEBUG:boto:Method: GET 2022-09-03T05:37:19.187 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.189 DEBUG:boto:Data: 2022-09-03T05:37:19.191 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.192 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:19.193 DEBUG:boto:Port: 8000 2022-09-03T05:37:19.194 DEBUG:boto:Params: {} 2022-09-03T05:37:19.195 DEBUG:boto:Token: None 2022-09-03T05:37:19.196 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:19.197 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:udqXOX0n82LighSEhXxd5Pptxto= 2022-09-03T05:37:19.198 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:udqXOX0n82LighSEhXxd5Pptxto=', 'Content-Length': '0'} 2022-09-03T05:37:19.203 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003742bf0c210f1972-006312e80f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.204 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:19.205 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:19.205 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.206 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.206 DEBUG:boto:Method: GET 2022-09-03T05:37:19.207 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.207 DEBUG:boto:Data: 2022-09-03T05:37:19.208 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.208 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:19.210 DEBUG:boto:Port: 8001 2022-09-03T05:37:19.211 DEBUG:boto:Params: {} 2022-09-03T05:37:19.212 DEBUG:boto:Token: None 2022-09-03T05:37:19.212 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/ 2022-09-03T05:37:19.213 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A= 2022-09-03T05:37:19.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A=', 'Content-Length': '0'} 2022-09-03T05:37:19.217 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009bd7350c2c5d2682-006312e80f-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.258 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:19.262 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:19.273 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:19.274 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.276 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.277 DEBUG:boto:Method: GET 2022-09-03T05:37:19.278 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.279 DEBUG:boto:Data: 2022-09-03T05:37:19.280 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.281 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:19.282 DEBUG:boto:Port: 8000 2022-09-03T05:37:19.284 DEBUG:boto:Params: {} 2022-09-03T05:37:19.286 DEBUG:boto:Token: None 2022-09-03T05:37:19.287 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/ 2022-09-03T05:37:19.288 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A= 2022-09-03T05:37:19.290 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A=', 'Content-Length': '0'} 2022-09-03T05:37:19.293 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000011b4808b7bf4dea2-006312e80f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.295 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:19.295 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.296 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.297 DEBUG:boto:Method: HEAD 2022-09-03T05:37:19.297 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:19.298 DEBUG:boto:Data: 2022-09-03T05:37:19.298 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.299 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:19.299 DEBUG:boto:Port: 8001 2022-09-03T05:37:19.300 DEBUG:boto:Params: {} 2022-09-03T05:37:19.301 DEBUG:boto:Token: None 2022-09-03T05:37:19.301 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj 2022-09-03T05:37:19.302 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dj25ZAStyYQbr6xH4Y3OZo+Hg54= 2022-09-03T05:37:19.303 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dj25ZAStyYQbr6xH4Y3OZo+Hg54=', 'Content-Length': '0'} 2022-09-03T05:37:19.313 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d084a135a4d86056-006312e80f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.317 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.318 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.319 DEBUG:boto:Method: HEAD 2022-09-03T05:37:19.320 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:19.325 DEBUG:boto:Data: 2022-09-03T05:37:19.326 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.327 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:19.328 DEBUG:boto:Port: 8000 2022-09-03T05:37:19.329 DEBUG:boto:Params: {} 2022-09-03T05:37:19.330 DEBUG:boto:Token: None 2022-09-03T05:37:19.330 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj 2022-09-03T05:37:19.332 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dj25ZAStyYQbr6xH4Y3OZo+Hg54= 2022-09-03T05:37:19.332 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dj25ZAStyYQbr6xH4Y3OZo+Hg54=', 'Content-Length': '0'} 2022-09-03T05:37:19.336 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000058f916d12eaf2eeb-006312e80f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.337 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:19.338 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.339 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.340 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:19.341 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:19.342 DEBUG:boto:Method: GET 2022-09-03T05:37:19.343 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:19.346 DEBUG:boto:Data: 2022-09-03T05:37:19.347 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.348 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:19.349 DEBUG:boto:Port: 8001 2022-09-03T05:37:19.352 DEBUG:boto:Params: {} 2022-09-03T05:37:19.354 DEBUG:boto:Token: None 2022-09-03T05:37:19.355 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:19.356 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YpSOmvHx2UlScSKc4sk6eDWUkBM= 2022-09-03T05:37:19.357 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YpSOmvHx2UlScSKc4sk6eDWUkBM=', 'Content-Length': '0'} 2022-09-03T05:37:19.363 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003bf080dee5781e3d-006312e80f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.365 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.367 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.368 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:19.369 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:19.371 DEBUG:boto:Method: GET 2022-09-03T05:37:19.372 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:19.373 DEBUG:boto:Data: 2022-09-03T05:37:19.374 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.374 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:19.375 DEBUG:boto:Port: 8000 2022-09-03T05:37:19.375 DEBUG:boto:Params: {} 2022-09-03T05:37:19.376 DEBUG:boto:Token: None 2022-09-03T05:37:19.376 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:19.377 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YpSOmvHx2UlScSKc4sk6eDWUkBM= 2022-09-03T05:37:19.378 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YpSOmvHx2UlScSKc4sk6eDWUkBM=', 'Content-Length': '0'} 2022-09-03T05:37:19.382 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008782015332443187-006312e80f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.384 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:19.385 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:19.386 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.387 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.388 DEBUG:boto:Method: GET 2022-09-03T05:37:19.389 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.391 DEBUG:boto:Data: 2022-09-03T05:37:19.392 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.392 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:19.393 DEBUG:boto:Port: 8001 2022-09-03T05:37:19.394 DEBUG:boto:Params: {} 2022-09-03T05:37:19.395 DEBUG:boto:Token: None 2022-09-03T05:37:19.395 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/ 2022-09-03T05:37:19.396 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A= 2022-09-03T05:37:19.397 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A=', 'Content-Length': '0'} 2022-09-03T05:37:19.400 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000046725927e0ae4ac5-006312e80f-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.401 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:19.402 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:19.403 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:19.404 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.405 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.406 DEBUG:boto:Method: GET 2022-09-03T05:37:19.407 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.408 DEBUG:boto:Data: 2022-09-03T05:37:19.409 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.410 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:19.411 DEBUG:boto:Port: 8000 2022-09-03T05:37:19.412 DEBUG:boto:Params: {} 2022-09-03T05:37:19.413 DEBUG:boto:Token: None 2022-09-03T05:37:19.415 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/ 2022-09-03T05:37:19.416 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A= 2022-09-03T05:37:19.418 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A=', 'Content-Length': '0'} 2022-09-03T05:37:19.423 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000633dc19049e851a9-006312e80f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.424 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:19.425 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:19.427 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.428 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.428 DEBUG:boto:path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.430 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.431 DEBUG:boto:Method: GET 2022-09-03T05:37:19.432 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.433 DEBUG:boto:Data: 2022-09-03T05:37:19.434 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.435 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:19.436 DEBUG:boto:Port: 8001 2022-09-03T05:37:19.437 DEBUG:boto:Params: {} 2022-09-03T05:37:19.438 DEBUG:boto:Token: None 2022-09-03T05:37:19.439 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.440 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a1PHS12DiAmNW17OfZ+mo6bTDP0= 2022-09-03T05:37:19.441 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a1PHS12DiAmNW17OfZ+mo6bTDP0=', 'Content-Length': '0'} 2022-09-03T05:37:19.444 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', '0PhwqB536OEXdH1QOmPxhlUfX3fwPvS'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000079950202651ba12d-006312e80f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.444 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.445 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.445 DEBUG:boto:path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.446 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.447 DEBUG:boto:Method: GET 2022-09-03T05:37:19.447 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.448 DEBUG:boto:Data: 2022-09-03T05:37:19.448 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.449 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:19.449 DEBUG:boto:Port: 8000 2022-09-03T05:37:19.450 DEBUG:boto:Params: {} 2022-09-03T05:37:19.451 DEBUG:boto:Token: None 2022-09-03T05:37:19.452 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.453 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a1PHS12DiAmNW17OfZ+mo6bTDP0= 2022-09-03T05:37:19.454 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a1PHS12DiAmNW17OfZ+mo6bTDP0=', 'Content-Length': '0'} 2022-09-03T05:37:19.459 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', '0PhwqB536OEXdH1QOmPxhlUfX3fwPvS'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000009c5d7931083240-006312e80f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.502 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.503 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.504 DEBUG:boto:path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.508 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.509 DEBUG:boto:Method: HEAD 2022-09-03T05:37:19.510 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.511 DEBUG:boto:Data: 2022-09-03T05:37:19.512 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.513 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:19.513 DEBUG:boto:Port: 8001 2022-09-03T05:37:19.514 DEBUG:boto:Params: {} 2022-09-03T05:37:19.515 DEBUG:boto:Token: None 2022-09-03T05:37:19.516 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.517 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:U/aYsK+nwL1fi5SNB7jKRGdDp2w= 2022-09-03T05:37:19.518 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:U/aYsK+nwL1fi5SNB7jKRGdDp2w=', 'Content-Length': '0'} 2022-09-03T05:37:19.522 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', '0PhwqB536OEXdH1QOmPxhlUfX3fwPvS'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b3c18496fccc5e38-006312e80f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.523 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.524 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.524 DEBUG:boto:path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.525 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.525 DEBUG:boto:Method: HEAD 2022-09-03T05:37:19.526 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.526 DEBUG:boto:Data: 2022-09-03T05:37:19.527 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.528 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:19.528 DEBUG:boto:Port: 8000 2022-09-03T05:37:19.529 DEBUG:boto:Params: {} 2022-09-03T05:37:19.530 DEBUG:boto:Token: None 2022-09-03T05:37:19.530 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.531 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:U/aYsK+nwL1fi5SNB7jKRGdDp2w= 2022-09-03T05:37:19.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:U/aYsK+nwL1fi5SNB7jKRGdDp2w=', 'Content-Length': '0'} 2022-09-03T05:37:19.534 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', '0PhwqB536OEXdH1QOmPxhlUfX3fwPvS'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a55678850aad9c3a-006312e80f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.536 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:19.537 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.538 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.539 DEBUG:boto:path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.540 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.541 DEBUG:boto:Method: GET 2022-09-03T05:37:19.541 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.542 DEBUG:boto:Data: 2022-09-03T05:37:19.543 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.543 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:19.544 DEBUG:boto:Port: 8001 2022-09-03T05:37:19.544 DEBUG:boto:Params: {} 2022-09-03T05:37:19.545 DEBUG:boto:Token: None 2022-09-03T05:37:19.546 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.546 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a1PHS12DiAmNW17OfZ+mo6bTDP0= 2022-09-03T05:37:19.547 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a1PHS12DiAmNW17OfZ+mo6bTDP0=', 'Content-Length': '0'} 2022-09-03T05:37:19.549 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', '0PhwqB536OEXdH1QOmPxhlUfX3fwPvS'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000460247c46763590f-006312e80f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.550 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.551 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.551 DEBUG:boto:path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.552 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.552 DEBUG:boto:Method: GET 2022-09-03T05:37:19.553 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.553 DEBUG:boto:Data: 2022-09-03T05:37:19.554 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.554 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:19.556 DEBUG:boto:Port: 8000 2022-09-03T05:37:19.557 DEBUG:boto:Params: {} 2022-09-03T05:37:19.558 DEBUG:boto:Token: None 2022-09-03T05:37:19.559 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:19.559 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a1PHS12DiAmNW17OfZ+mo6bTDP0= 2022-09-03T05:37:19.561 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a1PHS12DiAmNW17OfZ+mo6bTDP0=', 'Content-Length': '0'} 2022-09-03T05:37:19.572 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', '0PhwqB536OEXdH1QOmPxhlUfX3fwPvS'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000443f55c6233469d5-006312e80f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.574 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:19.575 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:19.576 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.577 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.578 DEBUG:boto:Method: GET 2022-09-03T05:37:19.579 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.580 DEBUG:boto:Data: 2022-09-03T05:37:19.581 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.582 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:19.583 DEBUG:boto:Port: 8001 2022-09-03T05:37:19.584 DEBUG:boto:Params: {} 2022-09-03T05:37:19.585 DEBUG:boto:Token: None 2022-09-03T05:37:19.586 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/ 2022-09-03T05:37:19.587 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A= 2022-09-03T05:37:19.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A=', 'Content-Length': '0'} 2022-09-03T05:37:19.591 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000088f5b24fd1469181-006312e80f-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.592 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:19.592 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:19.593 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:19.594 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.595 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.595 DEBUG:boto:Method: GET 2022-09-03T05:37:19.596 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.597 DEBUG:boto:Data: 2022-09-03T05:37:19.598 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.598 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:19.599 DEBUG:boto:Port: 8000 2022-09-03T05:37:19.600 DEBUG:boto:Params: {} 2022-09-03T05:37:19.600 DEBUG:boto:Token: None 2022-09-03T05:37:19.601 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/ 2022-09-03T05:37:19.602 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A= 2022-09-03T05:37:19.604 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A=', 'Content-Length': '0'} 2022-09-03T05:37:19.607 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fb3377e7f34034dd-006312e80f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.654 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:19.655 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.656 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.656 DEBUG:boto:Method: HEAD 2022-09-03T05:37:19.657 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:19.657 DEBUG:boto:Data: 2022-09-03T05:37:19.658 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.658 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:19.659 DEBUG:boto:Port: 8001 2022-09-03T05:37:19.660 DEBUG:boto:Params: {} 2022-09-03T05:37:19.660 DEBUG:boto:Token: None 2022-09-03T05:37:19.661 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj 2022-09-03T05:37:19.662 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dj25ZAStyYQbr6xH4Y3OZo+Hg54= 2022-09-03T05:37:19.662 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dj25ZAStyYQbr6xH4Y3OZo+Hg54=', 'Content-Length': '0'} 2022-09-03T05:37:19.666 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b9fe7659fa210ffc-006312e80f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.667 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.668 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.668 DEBUG:boto:Method: HEAD 2022-09-03T05:37:19.669 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:19.670 DEBUG:boto:Data: 2022-09-03T05:37:19.670 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.671 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:19.672 DEBUG:boto:Port: 8000 2022-09-03T05:37:19.672 DEBUG:boto:Params: {} 2022-09-03T05:37:19.673 DEBUG:boto:Token: None 2022-09-03T05:37:19.674 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj 2022-09-03T05:37:19.675 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dj25ZAStyYQbr6xH4Y3OZo+Hg54= 2022-09-03T05:37:19.675 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dj25ZAStyYQbr6xH4Y3OZo+Hg54=', 'Content-Length': '0'} 2022-09-03T05:37:19.679 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000034efd04304e535f6-006312e80f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.680 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:19.681 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.682 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.683 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:19.684 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:19.685 DEBUG:boto:Method: GET 2022-09-03T05:37:19.685 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:19.686 DEBUG:boto:Data: 2022-09-03T05:37:19.687 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.688 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:19.689 DEBUG:boto:Port: 8001 2022-09-03T05:37:19.690 DEBUG:boto:Params: {} 2022-09-03T05:37:19.691 DEBUG:boto:Token: None 2022-09-03T05:37:19.692 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:19.693 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YpSOmvHx2UlScSKc4sk6eDWUkBM= 2022-09-03T05:37:19.695 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YpSOmvHx2UlScSKc4sk6eDWUkBM=', 'Content-Length': '0'} 2022-09-03T05:37:19.698 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002564b66a5875a24a-006312e80f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.699 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.700 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.700 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:19.701 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:19.701 DEBUG:boto:Method: GET 2022-09-03T05:37:19.702 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:19.702 DEBUG:boto:Data: 2022-09-03T05:37:19.703 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.703 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:19.704 DEBUG:boto:Port: 8000 2022-09-03T05:37:19.705 DEBUG:boto:Params: {} 2022-09-03T05:37:19.706 DEBUG:boto:Token: None 2022-09-03T05:37:19.706 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:19.707 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YpSOmvHx2UlScSKc4sk6eDWUkBM= 2022-09-03T05:37:19.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YpSOmvHx2UlScSKc4sk6eDWUkBM=', 'Content-Length': '0'} 2022-09-03T05:37:19.710 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000d0b56f812c85ab7-006312e80f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.711 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:19.711 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:19.712 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.713 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.713 DEBUG:boto:Method: GET 2022-09-03T05:37:19.714 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.715 DEBUG:boto:Data: 2022-09-03T05:37:19.715 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.716 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:19.716 DEBUG:boto:Port: 8001 2022-09-03T05:37:19.717 DEBUG:boto:Params: {} 2022-09-03T05:37:19.718 DEBUG:boto:Token: None 2022-09-03T05:37:19.718 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/ 2022-09-03T05:37:19.719 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A= 2022-09-03T05:37:19.719 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A=', 'Content-Length': '0'} 2022-09-03T05:37:19.723 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ad0a5595cecb3e0b-006312e80f-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.771 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:19.774 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:19.775 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:19.776 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.776 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.777 DEBUG:boto:Method: GET 2022-09-03T05:37:19.778 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.779 DEBUG:boto:Data: 2022-09-03T05:37:19.780 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.781 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:19.782 DEBUG:boto:Port: 8000 2022-09-03T05:37:19.783 DEBUG:boto:Params: {} 2022-09-03T05:37:19.784 DEBUG:boto:Token: None 2022-09-03T05:37:19.786 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/ 2022-09-03T05:37:19.787 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A= 2022-09-03T05:37:19.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A=', 'Content-Length': '0'} 2022-09-03T05:37:19.792 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fd4942e1b46ad145-006312e80f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.792 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:19.793 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:19.794 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.795 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.795 DEBUG:boto:path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.796 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.796 DEBUG:boto:Method: GET 2022-09-03T05:37:19.800 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.800 DEBUG:boto:Data: 2022-09-03T05:37:19.801 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.801 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:19.802 DEBUG:boto:Port: 8001 2022-09-03T05:37:19.802 DEBUG:boto:Params: {} 2022-09-03T05:37:19.803 DEBUG:boto:Token: None 2022-09-03T05:37:19.804 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.804 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:haoOdxS6MTg/9E5IHlrO/0gNDcI= 2022-09-03T05:37:19.805 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:haoOdxS6MTg/9E5IHlrO/0gNDcI=', 'Content-Length': '0'} 2022-09-03T05:37:19.811 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cf4d03f3d54c65a8-006312e80f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.813 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.814 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.815 DEBUG:boto:path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.816 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.817 DEBUG:boto:Method: GET 2022-09-03T05:37:19.818 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.819 DEBUG:boto:Data: 2022-09-03T05:37:19.820 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.821 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:19.822 DEBUG:boto:Port: 8000 2022-09-03T05:37:19.823 DEBUG:boto:Params: {} 2022-09-03T05:37:19.824 DEBUG:boto:Token: None 2022-09-03T05:37:19.826 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.827 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:haoOdxS6MTg/9E5IHlrO/0gNDcI= 2022-09-03T05:37:19.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:haoOdxS6MTg/9E5IHlrO/0gNDcI=', 'Content-Length': '0'} 2022-09-03T05:37:19.831 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000df16d12729a3213d-006312e80f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.874 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.875 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.876 DEBUG:boto:path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.877 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.878 DEBUG:boto:Method: HEAD 2022-09-03T05:37:19.879 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.880 DEBUG:boto:Data: 2022-09-03T05:37:19.881 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.882 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:19.883 DEBUG:boto:Port: 8001 2022-09-03T05:37:19.884 DEBUG:boto:Params: {} 2022-09-03T05:37:19.885 DEBUG:boto:Token: None 2022-09-03T05:37:19.886 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.887 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E7VYIzjjPKeVKxV2u+7OcOGYcvQ= 2022-09-03T05:37:19.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E7VYIzjjPKeVKxV2u+7OcOGYcvQ=', 'Content-Length': '0'} 2022-09-03T05:37:19.890 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e6727f6db4519caa-006312e80f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.891 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.892 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.893 DEBUG:boto:path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.894 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.895 DEBUG:boto:Method: HEAD 2022-09-03T05:37:19.896 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.896 DEBUG:boto:Data: 2022-09-03T05:37:19.897 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.898 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:19.898 DEBUG:boto:Port: 8000 2022-09-03T05:37:19.899 DEBUG:boto:Params: {} 2022-09-03T05:37:19.901 DEBUG:boto:Token: None 2022-09-03T05:37:19.902 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.902 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E7VYIzjjPKeVKxV2u+7OcOGYcvQ= 2022-09-03T05:37:19.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E7VYIzjjPKeVKxV2u+7OcOGYcvQ=', 'Content-Length': '0'} 2022-09-03T05:37:19.906 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bcf2e5c769ce512b-006312e80f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.908 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:19.909 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.909 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.910 DEBUG:boto:path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.911 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.912 DEBUG:boto:Method: GET 2022-09-03T05:37:19.913 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.914 DEBUG:boto:Data: 2022-09-03T05:37:19.915 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.916 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:19.917 DEBUG:boto:Port: 8001 2022-09-03T05:37:19.918 DEBUG:boto:Params: {} 2022-09-03T05:37:19.919 DEBUG:boto:Token: None 2022-09-03T05:37:19.920 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.921 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:haoOdxS6MTg/9E5IHlrO/0gNDcI= 2022-09-03T05:37:19.923 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:haoOdxS6MTg/9E5IHlrO/0gNDcI=', 'Content-Length': '0'} 2022-09-03T05:37:19.925 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c514f2fbb06849d7-006312e80f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.927 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.928 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.929 DEBUG:boto:path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.930 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.931 DEBUG:boto:Method: GET 2022-09-03T05:37:19.932 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.933 DEBUG:boto:Data: 2022-09-03T05:37:19.933 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.934 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:19.935 DEBUG:boto:Port: 8000 2022-09-03T05:37:19.936 DEBUG:boto:Params: {} 2022-09-03T05:37:19.937 DEBUG:boto:Token: None 2022-09-03T05:37:19.938 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:19.939 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:haoOdxS6MTg/9E5IHlrO/0gNDcI= 2022-09-03T05:37:19.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:haoOdxS6MTg/9E5IHlrO/0gNDcI=', 'Content-Length': '0'} 2022-09-03T05:37:19.942 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a666ed4dc531f8c6-006312e80f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.943 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:19.943 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:19.944 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.945 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.946 DEBUG:boto:Method: GET 2022-09-03T05:37:19.947 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.948 DEBUG:boto:Data: 2022-09-03T05:37:19.949 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.949 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:19.950 DEBUG:boto:Port: 8001 2022-09-03T05:37:19.951 DEBUG:boto:Params: {} 2022-09-03T05:37:19.952 DEBUG:boto:Token: None 2022-09-03T05:37:19.953 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/ 2022-09-03T05:37:19.955 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A= 2022-09-03T05:37:19.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A=', 'Content-Length': '0'} 2022-09-03T05:37:19.960 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000081979eaa8c076d41-006312e80f-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.961 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:19.962 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:19.963 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:19.963 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.964 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.964 DEBUG:boto:Method: GET 2022-09-03T05:37:19.965 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:19.965 DEBUG:boto:Data: 2022-09-03T05:37:19.966 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.966 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:19.967 DEBUG:boto:Port: 8000 2022-09-03T05:37:19.968 DEBUG:boto:Params: {} 2022-09-03T05:37:19.968 DEBUG:boto:Token: None 2022-09-03T05:37:19.969 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/ 2022-09-03T05:37:19.970 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A= 2022-09-03T05:37:19.971 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XR/z4rHF/XlRr/z6hYZxK3uic9A=', 'Content-Length': '0'} 2022-09-03T05:37:19.974 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e63fc16bae6ba9e2-006312e80f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.975 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:19.976 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.976 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.977 DEBUG:boto:Method: HEAD 2022-09-03T05:37:19.977 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:19.978 DEBUG:boto:Data: 2022-09-03T05:37:19.978 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.979 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:19.979 DEBUG:boto:Port: 8001 2022-09-03T05:37:19.980 DEBUG:boto:Params: {} 2022-09-03T05:37:19.981 DEBUG:boto:Token: None 2022-09-03T05:37:19.982 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj 2022-09-03T05:37:19.983 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dj25ZAStyYQbr6xH4Y3OZo+Hg54= 2022-09-03T05:37:19.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dj25ZAStyYQbr6xH4Y3OZo+Hg54=', 'Content-Length': '0'} 2022-09-03T05:37:19.987 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000863ab33056abb119-006312e80f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:19.988 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:19.989 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:19.989 DEBUG:boto:Method: HEAD 2022-09-03T05:37:19.990 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:19.992 DEBUG:boto:Data: 2022-09-03T05:37:19.993 DEBUG:boto:Headers: {} 2022-09-03T05:37:19.994 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:19.995 DEBUG:boto:Port: 8000 2022-09-03T05:37:19.995 DEBUG:boto:Params: {} 2022-09-03T05:37:19.996 DEBUG:boto:Token: None 2022-09-03T05:37:19.997 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:19 GMT /dsgvcy-19/obj 2022-09-03T05:37:19.997 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dj25ZAStyYQbr6xH4Y3OZo+Hg54= 2022-09-03T05:37:19.998 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dj25ZAStyYQbr6xH4Y3OZo+Hg54=', 'Content-Length': '0'} 2022-09-03T05:37:20.001 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cc168f0a09bad8b8-006312e80f-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.002 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:20.003 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.003 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.004 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.005 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.006 DEBUG:boto:Method: GET 2022-09-03T05:37:20.006 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.007 DEBUG:boto:Data: 2022-09-03T05:37:20.008 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.008 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:20.009 DEBUG:boto:Port: 8001 2022-09-03T05:37:20.009 DEBUG:boto:Params: {} 2022-09-03T05:37:20.010 DEBUG:boto:Token: None 2022-09-03T05:37:20.011 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.012 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Es2XdQLMQg93ab60hXPzKW8omXw= 2022-09-03T05:37:20.014 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Es2XdQLMQg93ab60hXPzKW8omXw=', 'Content-Length': '0'} 2022-09-03T05:37:20.016 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cf3085d6388696de-006312e810-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.062 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.063 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.064 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.064 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.065 DEBUG:boto:Method: GET 2022-09-03T05:37:20.067 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.068 DEBUG:boto:Data: 2022-09-03T05:37:20.069 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.070 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:20.071 DEBUG:boto:Port: 8000 2022-09-03T05:37:20.072 DEBUG:boto:Params: {} 2022-09-03T05:37:20.073 DEBUG:boto:Token: None 2022-09-03T05:37:20.075 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.076 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Es2XdQLMQg93ab60hXPzKW8omXw= 2022-09-03T05:37:20.077 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Es2XdQLMQg93ab60hXPzKW8omXw=', 'Content-Length': '0'} 2022-09-03T05:37:20.081 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000026c8ce267789f5b5-006312e810-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.082 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:20.083 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:20.083 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.084 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.084 DEBUG:boto:Method: GET 2022-09-03T05:37:20.085 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.086 DEBUG:boto:Data: 2022-09-03T05:37:20.087 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.088 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:20.089 DEBUG:boto:Port: 8001 2022-09-03T05:37:20.089 DEBUG:boto:Params: {} 2022-09-03T05:37:20.091 DEBUG:boto:Token: None 2022-09-03T05:37:20.092 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/ 2022-09-03T05:37:20.092 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ= 2022-09-03T05:37:20.093 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ=', 'Content-Length': '0'} 2022-09-03T05:37:20.096 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bd51f5bdaa74124e-006312e810-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.101 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:20.102 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:20.103 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:20.105 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.106 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.107 DEBUG:boto:Method: GET 2022-09-03T05:37:20.108 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.109 DEBUG:boto:Data: 2022-09-03T05:37:20.110 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.111 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:20.112 DEBUG:boto:Port: 8000 2022-09-03T05:37:20.113 DEBUG:boto:Params: {} 2022-09-03T05:37:20.116 DEBUG:boto:Token: None 2022-09-03T05:37:20.117 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/ 2022-09-03T05:37:20.118 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ= 2022-09-03T05:37:20.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ=', 'Content-Length': '0'} 2022-09-03T05:37:20.124 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008415a1953ff24718-006312e810-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.125 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:20.128 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:20.129 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.130 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.131 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.131 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.133 DEBUG:boto:Method: GET 2022-09-03T05:37:20.134 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.135 DEBUG:boto:Data: 2022-09-03T05:37:20.136 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.137 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:20.138 DEBUG:boto:Port: 8001 2022-09-03T05:37:20.139 DEBUG:boto:Params: {} 2022-09-03T05:37:20.139 DEBUG:boto:Token: None 2022-09-03T05:37:20.140 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.141 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0Gm74HfrDe/9If46sLyicCxSN6c= 2022-09-03T05:37:20.141 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0Gm74HfrDe/9If46sLyicCxSN6c=', 'Content-Length': '0'} 2022-09-03T05:37:20.145 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000962f6fa92fdc6917-006312e810-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.148 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.150 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.151 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.152 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.153 DEBUG:boto:Method: GET 2022-09-03T05:37:20.155 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.156 DEBUG:boto:Data: 2022-09-03T05:37:20.157 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.158 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:20.159 DEBUG:boto:Port: 8000 2022-09-03T05:37:20.160 DEBUG:boto:Params: {} 2022-09-03T05:37:20.161 DEBUG:boto:Token: None 2022-09-03T05:37:20.161 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.162 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0Gm74HfrDe/9If46sLyicCxSN6c= 2022-09-03T05:37:20.163 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0Gm74HfrDe/9If46sLyicCxSN6c=', 'Content-Length': '0'} 2022-09-03T05:37:20.165 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bbec90bb607496a6-006312e810-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.166 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.166 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.167 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.167 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.168 DEBUG:boto:Method: HEAD 2022-09-03T05:37:20.168 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.169 DEBUG:boto:Data: 2022-09-03T05:37:20.170 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.170 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:20.171 DEBUG:boto:Port: 8001 2022-09-03T05:37:20.171 DEBUG:boto:Params: {} 2022-09-03T05:37:20.172 DEBUG:boto:Token: None 2022-09-03T05:37:20.172 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.173 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nA5ttAwshLDBqlQfP2r61a7ELzY= 2022-09-03T05:37:20.174 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nA5ttAwshLDBqlQfP2r61a7ELzY=', 'Content-Length': '0'} 2022-09-03T05:37:20.176 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000aa75bffe874b771a-006312e810-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.177 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.177 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.178 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.178 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.179 DEBUG:boto:Method: HEAD 2022-09-03T05:37:20.179 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.180 DEBUG:boto:Data: 2022-09-03T05:37:20.181 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.181 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:20.182 DEBUG:boto:Port: 8000 2022-09-03T05:37:20.182 DEBUG:boto:Params: {} 2022-09-03T05:37:20.183 DEBUG:boto:Token: None 2022-09-03T05:37:20.183 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.184 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nA5ttAwshLDBqlQfP2r61a7ELzY= 2022-09-03T05:37:20.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nA5ttAwshLDBqlQfP2r61a7ELzY=', 'Content-Length': '0'} 2022-09-03T05:37:20.187 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005b12d94378a1bc06-006312e810-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.188 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:20.189 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.190 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.190 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.191 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.192 DEBUG:boto:Method: GET 2022-09-03T05:37:20.192 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.193 DEBUG:boto:Data: 2022-09-03T05:37:20.193 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.194 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:20.194 DEBUG:boto:Port: 8001 2022-09-03T05:37:20.195 DEBUG:boto:Params: {} 2022-09-03T05:37:20.196 DEBUG:boto:Token: None 2022-09-03T05:37:20.196 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.198 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0Gm74HfrDe/9If46sLyicCxSN6c= 2022-09-03T05:37:20.198 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0Gm74HfrDe/9If46sLyicCxSN6c=', 'Content-Length': '0'} 2022-09-03T05:37:20.200 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000dd0c102511c620b-006312e810-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.201 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.202 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.203 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.203 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.204 DEBUG:boto:Method: GET 2022-09-03T05:37:20.204 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.205 DEBUG:boto:Data: 2022-09-03T05:37:20.205 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.206 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:20.207 DEBUG:boto:Port: 8000 2022-09-03T05:37:20.207 DEBUG:boto:Params: {} 2022-09-03T05:37:20.208 DEBUG:boto:Token: None 2022-09-03T05:37:20.209 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:20.210 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0Gm74HfrDe/9If46sLyicCxSN6c= 2022-09-03T05:37:20.210 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0Gm74HfrDe/9If46sLyicCxSN6c=', 'Content-Length': '0'} 2022-09-03T05:37:20.212 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000008bd3012d649a775-006312e810-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.215 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:20.216 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:20.217 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.218 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.219 DEBUG:boto:Method: GET 2022-09-03T05:37:20.220 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.221 DEBUG:boto:Data: 2022-09-03T05:37:20.222 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.223 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:20.224 DEBUG:boto:Port: 8001 2022-09-03T05:37:20.225 DEBUG:boto:Params: {} 2022-09-03T05:37:20.226 DEBUG:boto:Token: None 2022-09-03T05:37:20.227 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/ 2022-09-03T05:37:20.228 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ= 2022-09-03T05:37:20.229 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ=', 'Content-Length': '0'} 2022-09-03T05:37:20.234 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007659667df54bdc0f-006312e810-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.235 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:20.237 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:20.238 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:20.239 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.239 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.240 DEBUG:boto:Method: GET 2022-09-03T05:37:20.241 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.242 DEBUG:boto:Data: 2022-09-03T05:37:20.243 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.244 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:20.244 DEBUG:boto:Port: 8000 2022-09-03T05:37:20.245 DEBUG:boto:Params: {} 2022-09-03T05:37:20.245 DEBUG:boto:Token: None 2022-09-03T05:37:20.246 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/ 2022-09-03T05:37:20.247 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ= 2022-09-03T05:37:20.247 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ=', 'Content-Length': '0'} 2022-09-03T05:37:20.251 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d01e9a8bd2019ca-006312e810-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.251 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:20.252 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.253 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.253 DEBUG:boto:Method: HEAD 2022-09-03T05:37:20.254 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:20.254 DEBUG:boto:Data: 2022-09-03T05:37:20.255 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.255 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:20.256 DEBUG:boto:Port: 8001 2022-09-03T05:37:20.256 DEBUG:boto:Params: {} 2022-09-03T05:37:20.257 DEBUG:boto:Token: None 2022-09-03T05:37:20.258 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj 2022-09-03T05:37:20.258 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MbDfQe30zMfqXJcHZIbEiaMF20w= 2022-09-03T05:37:20.259 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MbDfQe30zMfqXJcHZIbEiaMF20w=', 'Content-Length': '0'} 2022-09-03T05:37:20.262 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001de153f17117389b-006312e810-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.263 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.264 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.264 DEBUG:boto:Method: HEAD 2022-09-03T05:37:20.265 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:20.265 DEBUG:boto:Data: 2022-09-03T05:37:20.266 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.266 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:20.267 DEBUG:boto:Port: 8000 2022-09-03T05:37:20.267 DEBUG:boto:Params: {} 2022-09-03T05:37:20.268 DEBUG:boto:Token: None 2022-09-03T05:37:20.268 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj 2022-09-03T05:37:20.270 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MbDfQe30zMfqXJcHZIbEiaMF20w= 2022-09-03T05:37:20.270 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MbDfQe30zMfqXJcHZIbEiaMF20w=', 'Content-Length': '0'} 2022-09-03T05:37:20.274 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000065caeac0c2ca0dcc-006312e810-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.276 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:20.277 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.278 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.279 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.279 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.280 DEBUG:boto:Method: GET 2022-09-03T05:37:20.281 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.282 DEBUG:boto:Data: 2022-09-03T05:37:20.283 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.284 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:20.285 DEBUG:boto:Port: 8001 2022-09-03T05:37:20.286 DEBUG:boto:Params: {} 2022-09-03T05:37:20.288 DEBUG:boto:Token: None 2022-09-03T05:37:20.289 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.290 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Es2XdQLMQg93ab60hXPzKW8omXw= 2022-09-03T05:37:20.291 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Es2XdQLMQg93ab60hXPzKW8omXw=', 'Content-Length': '0'} 2022-09-03T05:37:20.294 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e66eafccbff8351c-006312e810-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.337 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.338 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.339 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.340 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.340 DEBUG:boto:Method: GET 2022-09-03T05:37:20.342 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.342 DEBUG:boto:Data: 2022-09-03T05:37:20.343 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.343 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:20.344 DEBUG:boto:Port: 8000 2022-09-03T05:37:20.345 DEBUG:boto:Params: {} 2022-09-03T05:37:20.345 DEBUG:boto:Token: None 2022-09-03T05:37:20.346 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.347 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Es2XdQLMQg93ab60hXPzKW8omXw= 2022-09-03T05:37:20.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Es2XdQLMQg93ab60hXPzKW8omXw=', 'Content-Length': '0'} 2022-09-03T05:37:20.350 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000088430cf9b3094dd-006312e810-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.352 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:20.353 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:20.354 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.354 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.355 DEBUG:boto:Method: GET 2022-09-03T05:37:20.356 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.356 DEBUG:boto:Data: 2022-09-03T05:37:20.357 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.357 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:20.358 DEBUG:boto:Port: 8001 2022-09-03T05:37:20.359 DEBUG:boto:Params: {} 2022-09-03T05:37:20.359 DEBUG:boto:Token: None 2022-09-03T05:37:20.360 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/ 2022-09-03T05:37:20.361 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ= 2022-09-03T05:37:20.362 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ=', 'Content-Length': '0'} 2022-09-03T05:37:20.365 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a9e12c53d4c84613-006312e810-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.366 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:20.367 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:20.367 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:20.368 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.368 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.369 DEBUG:boto:Method: GET 2022-09-03T05:37:20.369 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.370 DEBUG:boto:Data: 2022-09-03T05:37:20.371 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.371 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:20.372 DEBUG:boto:Port: 8000 2022-09-03T05:37:20.373 DEBUG:boto:Params: {} 2022-09-03T05:37:20.373 DEBUG:boto:Token: None 2022-09-03T05:37:20.374 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/ 2022-09-03T05:37:20.374 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ= 2022-09-03T05:37:20.375 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ=', 'Content-Length': '0'} 2022-09-03T05:37:20.378 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c2f1757de8ee8739-006312e810-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.422 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:20.424 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:20.424 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.425 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.425 DEBUG:boto:path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.427 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.428 DEBUG:boto:Method: GET 2022-09-03T05:37:20.428 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.429 DEBUG:boto:Data: 2022-09-03T05:37:20.432 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.433 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:20.434 DEBUG:boto:Port: 8001 2022-09-03T05:37:20.435 DEBUG:boto:Params: {} 2022-09-03T05:37:20.436 DEBUG:boto:Token: None 2022-09-03T05:37:20.437 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.438 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ejWe23EtdiNVWvQCvr/1KPFAAQM= 2022-09-03T05:37:20.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ejWe23EtdiNVWvQCvr/1KPFAAQM=', 'Content-Length': '0'} 2022-09-03T05:37:20.444 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'EFeprdWgJo5gbXM16hy6HNBwhGTxK7e'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008207dfe09dd8e950-006312e810-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.446 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.447 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.448 DEBUG:boto:path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.449 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.450 DEBUG:boto:Method: GET 2022-09-03T05:37:20.451 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.452 DEBUG:boto:Data: 2022-09-03T05:37:20.454 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.454 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:20.455 DEBUG:boto:Port: 8000 2022-09-03T05:37:20.456 DEBUG:boto:Params: {} 2022-09-03T05:37:20.457 DEBUG:boto:Token: None 2022-09-03T05:37:20.459 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.460 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ejWe23EtdiNVWvQCvr/1KPFAAQM= 2022-09-03T05:37:20.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ejWe23EtdiNVWvQCvr/1KPFAAQM=', 'Content-Length': '0'} 2022-09-03T05:37:20.464 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'EFeprdWgJo5gbXM16hy6HNBwhGTxK7e'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000046592672f47344ff-006312e810-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.465 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.465 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.466 DEBUG:boto:path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.467 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.470 DEBUG:boto:Method: HEAD 2022-09-03T05:37:20.470 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.471 DEBUG:boto:Data: 2022-09-03T05:37:20.471 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.472 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:20.472 DEBUG:boto:Port: 8001 2022-09-03T05:37:20.473 DEBUG:boto:Params: {} 2022-09-03T05:37:20.474 DEBUG:boto:Token: None 2022-09-03T05:37:20.474 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.475 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkmFI7nDEGPF62PuBhfaQ15z484= 2022-09-03T05:37:20.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkmFI7nDEGPF62PuBhfaQ15z484=', 'Content-Length': '0'} 2022-09-03T05:37:20.478 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'EFeprdWgJo5gbXM16hy6HNBwhGTxK7e'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000478641bbe07342a8-006312e810-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.479 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.480 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.481 DEBUG:boto:path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.482 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.482 DEBUG:boto:Method: HEAD 2022-09-03T05:37:20.483 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.484 DEBUG:boto:Data: 2022-09-03T05:37:20.485 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.486 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:20.486 DEBUG:boto:Port: 8000 2022-09-03T05:37:20.487 DEBUG:boto:Params: {} 2022-09-03T05:37:20.487 DEBUG:boto:Token: None 2022-09-03T05:37:20.488 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.488 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkmFI7nDEGPF62PuBhfaQ15z484= 2022-09-03T05:37:20.489 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkmFI7nDEGPF62PuBhfaQ15z484=', 'Content-Length': '0'} 2022-09-03T05:37:20.501 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'EFeprdWgJo5gbXM16hy6HNBwhGTxK7e'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000035b37184d8f699d3-006312e810-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.502 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:20.502 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.503 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.504 DEBUG:boto:path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.504 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.505 DEBUG:boto:Method: GET 2022-09-03T05:37:20.506 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.506 DEBUG:boto:Data: 2022-09-03T05:37:20.507 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.507 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:20.508 DEBUG:boto:Port: 8001 2022-09-03T05:37:20.509 DEBUG:boto:Params: {} 2022-09-03T05:37:20.509 DEBUG:boto:Token: None 2022-09-03T05:37:20.510 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.511 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ejWe23EtdiNVWvQCvr/1KPFAAQM= 2022-09-03T05:37:20.511 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ejWe23EtdiNVWvQCvr/1KPFAAQM=', 'Content-Length': '0'} 2022-09-03T05:37:20.514 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'EFeprdWgJo5gbXM16hy6HNBwhGTxK7e'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000717bf3123b38777f-006312e810-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.515 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.515 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.516 DEBUG:boto:path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.516 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.517 DEBUG:boto:Method: GET 2022-09-03T05:37:20.517 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.521 DEBUG:boto:Data: 2022-09-03T05:37:20.522 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.523 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:20.523 DEBUG:boto:Port: 8000 2022-09-03T05:37:20.524 DEBUG:boto:Params: {} 2022-09-03T05:37:20.524 DEBUG:boto:Token: None 2022-09-03T05:37:20.525 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:20.526 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ejWe23EtdiNVWvQCvr/1KPFAAQM= 2022-09-03T05:37:20.526 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ejWe23EtdiNVWvQCvr/1KPFAAQM=', 'Content-Length': '0'} 2022-09-03T05:37:20.530 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'EFeprdWgJo5gbXM16hy6HNBwhGTxK7e'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000053f4728edb49a70c-006312e810-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.531 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:20.532 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:20.532 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.533 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.533 DEBUG:boto:Method: GET 2022-09-03T05:37:20.534 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.534 DEBUG:boto:Data: 2022-09-03T05:37:20.535 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.535 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:20.536 DEBUG:boto:Port: 8001 2022-09-03T05:37:20.536 DEBUG:boto:Params: {} 2022-09-03T05:37:20.537 DEBUG:boto:Token: None 2022-09-03T05:37:20.538 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/ 2022-09-03T05:37:20.538 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ= 2022-09-03T05:37:20.539 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ=', 'Content-Length': '0'} 2022-09-03T05:37:20.543 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a688f8273a966848-006312e810-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.544 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:20.546 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:20.546 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:20.547 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.547 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.548 DEBUG:boto:Method: GET 2022-09-03T05:37:20.548 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.549 DEBUG:boto:Data: 2022-09-03T05:37:20.550 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.550 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:20.551 DEBUG:boto:Port: 8000 2022-09-03T05:37:20.551 DEBUG:boto:Params: {} 2022-09-03T05:37:20.552 DEBUG:boto:Token: None 2022-09-03T05:37:20.553 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/ 2022-09-03T05:37:20.554 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ= 2022-09-03T05:37:20.554 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ=', 'Content-Length': '0'} 2022-09-03T05:37:20.558 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008eb171b70c7de1ce-006312e810-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.558 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:20.559 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.561 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.562 DEBUG:boto:Method: HEAD 2022-09-03T05:37:20.562 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:20.563 DEBUG:boto:Data: 2022-09-03T05:37:20.564 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.564 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:20.565 DEBUG:boto:Port: 8001 2022-09-03T05:37:20.565 DEBUG:boto:Params: {} 2022-09-03T05:37:20.566 DEBUG:boto:Token: None 2022-09-03T05:37:20.568 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj 2022-09-03T05:37:20.568 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MbDfQe30zMfqXJcHZIbEiaMF20w= 2022-09-03T05:37:20.571 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MbDfQe30zMfqXJcHZIbEiaMF20w=', 'Content-Length': '0'} 2022-09-03T05:37:20.575 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002d151b6a55221dd3-006312e810-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.576 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.576 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.577 DEBUG:boto:Method: HEAD 2022-09-03T05:37:20.577 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:20.578 DEBUG:boto:Data: 2022-09-03T05:37:20.578 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.579 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:20.579 DEBUG:boto:Port: 8000 2022-09-03T05:37:20.581 DEBUG:boto:Params: {} 2022-09-03T05:37:20.582 DEBUG:boto:Token: None 2022-09-03T05:37:20.582 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj 2022-09-03T05:37:20.583 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MbDfQe30zMfqXJcHZIbEiaMF20w= 2022-09-03T05:37:20.583 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MbDfQe30zMfqXJcHZIbEiaMF20w=', 'Content-Length': '0'} 2022-09-03T05:37:20.588 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000322c5b0e336b32ce-006312e810-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.589 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:20.589 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.590 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.591 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.591 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.592 DEBUG:boto:Method: GET 2022-09-03T05:37:20.592 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.593 DEBUG:boto:Data: 2022-09-03T05:37:20.593 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.594 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:20.595 DEBUG:boto:Port: 8001 2022-09-03T05:37:20.595 DEBUG:boto:Params: {} 2022-09-03T05:37:20.596 DEBUG:boto:Token: None 2022-09-03T05:37:20.597 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.597 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Es2XdQLMQg93ab60hXPzKW8omXw= 2022-09-03T05:37:20.598 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Es2XdQLMQg93ab60hXPzKW8omXw=', 'Content-Length': '0'} 2022-09-03T05:37:20.601 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ce1f5abc2081d242-006312e810-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.642 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.643 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.643 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.644 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.644 DEBUG:boto:Method: GET 2022-09-03T05:37:20.645 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.646 DEBUG:boto:Data: 2022-09-03T05:37:20.646 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.647 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:20.647 DEBUG:boto:Port: 8000 2022-09-03T05:37:20.648 DEBUG:boto:Params: {} 2022-09-03T05:37:20.649 DEBUG:boto:Token: None 2022-09-03T05:37:20.649 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.650 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Es2XdQLMQg93ab60hXPzKW8omXw= 2022-09-03T05:37:20.651 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Es2XdQLMQg93ab60hXPzKW8omXw=', 'Content-Length': '0'} 2022-09-03T05:37:20.653 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000089c7126731f85fa3-006312e810-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.655 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:20.656 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:20.657 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.658 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.659 DEBUG:boto:Method: GET 2022-09-03T05:37:20.660 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.661 DEBUG:boto:Data: 2022-09-03T05:37:20.662 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.663 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:20.664 DEBUG:boto:Port: 8001 2022-09-03T05:37:20.664 DEBUG:boto:Params: {} 2022-09-03T05:37:20.666 DEBUG:boto:Token: None 2022-09-03T05:37:20.667 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/ 2022-09-03T05:37:20.667 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ= 2022-09-03T05:37:20.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ=', 'Content-Length': '0'} 2022-09-03T05:37:20.672 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bdd389901e8d852c-006312e810-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.673 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:20.674 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:20.675 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:20.675 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.676 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.676 DEBUG:boto:Method: GET 2022-09-03T05:37:20.677 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.677 DEBUG:boto:Data: 2022-09-03T05:37:20.679 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.680 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:20.680 DEBUG:boto:Port: 8000 2022-09-03T05:37:20.681 DEBUG:boto:Params: {} 2022-09-03T05:37:20.683 DEBUG:boto:Token: None 2022-09-03T05:37:20.684 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/ 2022-09-03T05:37:20.685 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ= 2022-09-03T05:37:20.686 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ=', 'Content-Length': '0'} 2022-09-03T05:37:20.689 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006ac1ff7021cbdc1f-006312e810-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.690 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:20.691 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:20.692 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.692 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.693 DEBUG:boto:path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.693 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.694 DEBUG:boto:Method: GET 2022-09-03T05:37:20.695 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.695 DEBUG:boto:Data: 2022-09-03T05:37:20.696 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.696 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:20.697 DEBUG:boto:Port: 8001 2022-09-03T05:37:20.697 DEBUG:boto:Params: {} 2022-09-03T05:37:20.699 DEBUG:boto:Token: None 2022-09-03T05:37:20.699 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.700 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:55F/QdEeyHApOYOTz3fjmdX4n0c= 2022-09-03T05:37:20.701 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:55F/QdEeyHApOYOTz3fjmdX4n0c=', 'Content-Length': '0'} 2022-09-03T05:37:20.703 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f5d54395b13f19a8-006312e810-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.704 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.705 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.706 DEBUG:boto:path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.707 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.707 DEBUG:boto:Method: GET 2022-09-03T05:37:20.708 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.709 DEBUG:boto:Data: 2022-09-03T05:37:20.711 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.712 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:20.715 DEBUG:boto:Port: 8000 2022-09-03T05:37:20.716 DEBUG:boto:Params: {} 2022-09-03T05:37:20.716 DEBUG:boto:Token: None 2022-09-03T05:37:20.717 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.718 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:55F/QdEeyHApOYOTz3fjmdX4n0c= 2022-09-03T05:37:20.718 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:55F/QdEeyHApOYOTz3fjmdX4n0c=', 'Content-Length': '0'} 2022-09-03T05:37:20.720 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000455cfe65231085f0-006312e810-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.721 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.722 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.722 DEBUG:boto:path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.723 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.723 DEBUG:boto:Method: HEAD 2022-09-03T05:37:20.724 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.725 DEBUG:boto:Data: 2022-09-03T05:37:20.725 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.726 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:20.726 DEBUG:boto:Port: 8001 2022-09-03T05:37:20.727 DEBUG:boto:Params: {} 2022-09-03T05:37:20.727 DEBUG:boto:Token: None 2022-09-03T05:37:20.728 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.729 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BxAkKmsEkm6Hff/sRm0zrcgIrMM= 2022-09-03T05:37:20.729 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BxAkKmsEkm6Hff/sRm0zrcgIrMM=', 'Content-Length': '0'} 2022-09-03T05:37:20.738 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008d3becd27d8488c2-006312e810-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.740 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.740 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.741 DEBUG:boto:path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.742 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.746 DEBUG:boto:Method: HEAD 2022-09-03T05:37:20.747 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.748 DEBUG:boto:Data: 2022-09-03T05:37:20.749 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.749 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:20.750 DEBUG:boto:Port: 8000 2022-09-03T05:37:20.751 DEBUG:boto:Params: {} 2022-09-03T05:37:20.751 DEBUG:boto:Token: None 2022-09-03T05:37:20.752 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.752 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BxAkKmsEkm6Hff/sRm0zrcgIrMM= 2022-09-03T05:37:20.753 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BxAkKmsEkm6Hff/sRm0zrcgIrMM=', 'Content-Length': '0'} 2022-09-03T05:37:20.756 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000eac49c0d78b3c759-006312e810-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.758 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:20.759 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.760 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.761 DEBUG:boto:path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.762 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.763 DEBUG:boto:Method: GET 2022-09-03T05:37:20.764 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.765 DEBUG:boto:Data: 2022-09-03T05:37:20.766 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.767 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:20.768 DEBUG:boto:Port: 8001 2022-09-03T05:37:20.769 DEBUG:boto:Params: {} 2022-09-03T05:37:20.770 DEBUG:boto:Token: None 2022-09-03T05:37:20.771 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.772 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:55F/QdEeyHApOYOTz3fjmdX4n0c= 2022-09-03T05:37:20.773 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:55F/QdEeyHApOYOTz3fjmdX4n0c=', 'Content-Length': '0'} 2022-09-03T05:37:20.775 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005b3a33eba7662a64-006312e810-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.818 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.819 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.820 DEBUG:boto:path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.820 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.821 DEBUG:boto:Method: GET 2022-09-03T05:37:20.821 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.822 DEBUG:boto:Data: 2022-09-03T05:37:20.822 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.823 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:20.823 DEBUG:boto:Port: 8000 2022-09-03T05:37:20.824 DEBUG:boto:Params: {} 2022-09-03T05:37:20.825 DEBUG:boto:Token: None 2022-09-03T05:37:20.825 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:20.826 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:55F/QdEeyHApOYOTz3fjmdX4n0c= 2022-09-03T05:37:20.827 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:55F/QdEeyHApOYOTz3fjmdX4n0c=', 'Content-Length': '0'} 2022-09-03T05:37:20.835 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000023238e1eda0157a7-006312e810-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.837 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:20.838 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:20.838 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.839 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.839 DEBUG:boto:Method: GET 2022-09-03T05:37:20.840 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.841 DEBUG:boto:Data: 2022-09-03T05:37:20.841 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.842 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:20.843 DEBUG:boto:Port: 8001 2022-09-03T05:37:20.844 DEBUG:boto:Params: {} 2022-09-03T05:37:20.844 DEBUG:boto:Token: None 2022-09-03T05:37:20.845 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/ 2022-09-03T05:37:20.846 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ= 2022-09-03T05:37:20.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ=', 'Content-Length': '0'} 2022-09-03T05:37:20.852 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f36a06eb9e344e63-006312e810-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.853 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:20.854 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:20.855 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:20.856 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.857 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.857 DEBUG:boto:Method: GET 2022-09-03T05:37:20.858 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.858 DEBUG:boto:Data: 2022-09-03T05:37:20.859 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.859 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:20.860 DEBUG:boto:Port: 8000 2022-09-03T05:37:20.860 DEBUG:boto:Params: {} 2022-09-03T05:37:20.861 DEBUG:boto:Token: None 2022-09-03T05:37:20.862 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/ 2022-09-03T05:37:20.862 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ= 2022-09-03T05:37:20.863 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ=', 'Content-Length': '0'} 2022-09-03T05:37:20.866 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000049ee83b26c57bb87-006312e810-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.867 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:20.868 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.869 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.869 DEBUG:boto:Method: HEAD 2022-09-03T05:37:20.870 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:20.870 DEBUG:boto:Data: 2022-09-03T05:37:20.871 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.871 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:20.872 DEBUG:boto:Port: 8001 2022-09-03T05:37:20.872 DEBUG:boto:Params: {} 2022-09-03T05:37:20.873 DEBUG:boto:Token: None 2022-09-03T05:37:20.873 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj 2022-09-03T05:37:20.874 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MbDfQe30zMfqXJcHZIbEiaMF20w= 2022-09-03T05:37:20.874 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MbDfQe30zMfqXJcHZIbEiaMF20w=', 'Content-Length': '0'} 2022-09-03T05:37:20.878 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000008e8af8e2518b3d5-006312e810-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.883 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.884 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.884 DEBUG:boto:Method: HEAD 2022-09-03T05:37:20.885 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:20.885 DEBUG:boto:Data: 2022-09-03T05:37:20.887 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.887 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:20.888 DEBUG:boto:Port: 8000 2022-09-03T05:37:20.888 DEBUG:boto:Params: {} 2022-09-03T05:37:20.889 DEBUG:boto:Token: None 2022-09-03T05:37:20.890 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj 2022-09-03T05:37:20.895 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MbDfQe30zMfqXJcHZIbEiaMF20w= 2022-09-03T05:37:20.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MbDfQe30zMfqXJcHZIbEiaMF20w=', 'Content-Length': '0'} 2022-09-03T05:37:20.900 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f1c8401adcb422f2-006312e810-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.901 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:20.901 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.902 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.902 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.903 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.903 DEBUG:boto:Method: GET 2022-09-03T05:37:20.904 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.905 DEBUG:boto:Data: 2022-09-03T05:37:20.905 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.909 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:20.909 DEBUG:boto:Port: 8001 2022-09-03T05:37:20.910 DEBUG:boto:Params: {} 2022-09-03T05:37:20.911 DEBUG:boto:Token: None 2022-09-03T05:37:20.911 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.913 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Es2XdQLMQg93ab60hXPzKW8omXw= 2022-09-03T05:37:20.914 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Es2XdQLMQg93ab60hXPzKW8omXw=', 'Content-Length': '0'} 2022-09-03T05:37:20.916 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c31bfe1eb92468ef-006312e810-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.962 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:20.963 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:20.964 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.964 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.965 DEBUG:boto:Method: GET 2022-09-03T05:37:20.965 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.966 DEBUG:boto:Data: 2022-09-03T05:37:20.967 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.967 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:20.968 DEBUG:boto:Port: 8000 2022-09-03T05:37:20.968 DEBUG:boto:Params: {} 2022-09-03T05:37:20.969 DEBUG:boto:Token: None 2022-09-03T05:37:20.969 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:20.970 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Es2XdQLMQg93ab60hXPzKW8omXw= 2022-09-03T05:37:20.971 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Es2XdQLMQg93ab60hXPzKW8omXw=', 'Content-Length': '0'} 2022-09-03T05:37:20.974 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bb6fb366571a1c94-006312e810-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.975 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:20.975 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:20.976 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.977 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.977 DEBUG:boto:Method: GET 2022-09-03T05:37:20.978 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.979 DEBUG:boto:Data: 2022-09-03T05:37:20.980 DEBUG:boto:Headers: {} 2022-09-03T05:37:20.980 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:20.981 DEBUG:boto:Port: 8001 2022-09-03T05:37:20.982 DEBUG:boto:Params: {} 2022-09-03T05:37:20.982 DEBUG:boto:Token: None 2022-09-03T05:37:20.983 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:20 GMT /dsgvcy-19/ 2022-09-03T05:37:20.984 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ= 2022-09-03T05:37:20.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FEaMh0jQawqW3nSt/svj9p+3AGQ=', 'Content-Length': '0'} 2022-09-03T05:37:20.994 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c0ec53a93323003a-006312e810-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:20.994 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:20.995 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:20.996 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:20.996 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.997 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:20.997 DEBUG:boto:Method: GET 2022-09-03T05:37:21.001 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.001 DEBUG:boto:Data: 2022-09-03T05:37:21.002 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.002 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:21.003 DEBUG:boto:Port: 8000 2022-09-03T05:37:21.003 DEBUG:boto:Params: {} 2022-09-03T05:37:21.004 DEBUG:boto:Token: None 2022-09-03T05:37:21.006 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/ 2022-09-03T05:37:21.007 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk= 2022-09-03T05:37:21.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk=', 'Content-Length': '0'} 2022-09-03T05:37:21.012 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006b453eeaf1382734-006312e811-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.013 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:21.015 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:21.016 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.016 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.017 DEBUG:boto:path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.017 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.018 DEBUG:boto:Method: GET 2022-09-03T05:37:21.019 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.019 DEBUG:boto:Data: 2022-09-03T05:37:21.020 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.020 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:21.021 DEBUG:boto:Port: 8001 2022-09-03T05:37:21.021 DEBUG:boto:Params: {} 2022-09-03T05:37:21.022 DEBUG:boto:Token: None 2022-09-03T05:37:21.024 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.024 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EPqnZBGMIPZA6GhrIl8QpryQS6A= 2022-09-03T05:37:21.025 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EPqnZBGMIPZA6GhrIl8QpryQS6A=', 'Content-Length': '0'} 2022-09-03T05:37:21.034 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'yL2uOIeC0l.C5YXdYoqQBvDIly3idxO'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002351129831f3cf75-006312e811-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.035 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.036 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.036 DEBUG:boto:path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.037 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.037 DEBUG:boto:Method: GET 2022-09-03T05:37:21.038 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.038 DEBUG:boto:Data: 2022-09-03T05:37:21.039 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.040 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:21.040 DEBUG:boto:Port: 8000 2022-09-03T05:37:21.044 DEBUG:boto:Params: {} 2022-09-03T05:37:21.045 DEBUG:boto:Token: None 2022-09-03T05:37:21.046 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.047 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EPqnZBGMIPZA6GhrIl8QpryQS6A= 2022-09-03T05:37:21.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EPqnZBGMIPZA6GhrIl8QpryQS6A=', 'Content-Length': '0'} 2022-09-03T05:37:21.053 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'yL2uOIeC0l.C5YXdYoqQBvDIly3idxO'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c579ead4f86c3b9e-006312e811-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.054 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.055 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.056 DEBUG:boto:path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.056 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.057 DEBUG:boto:Method: HEAD 2022-09-03T05:37:21.058 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.058 DEBUG:boto:Data: 2022-09-03T05:37:21.059 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.060 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:21.060 DEBUG:boto:Port: 8001 2022-09-03T05:37:21.061 DEBUG:boto:Params: {} 2022-09-03T05:37:21.061 DEBUG:boto:Token: None 2022-09-03T05:37:21.062 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.063 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:46JKQ9+3vg9+2AvVSur8f6extls= 2022-09-03T05:37:21.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:46JKQ9+3vg9+2AvVSur8f6extls=', 'Content-Length': '0'} 2022-09-03T05:37:21.066 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'yL2uOIeC0l.C5YXdYoqQBvDIly3idxO'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007cb26c25d8ec7d22-006312e811-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.068 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.068 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.069 DEBUG:boto:path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.070 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.071 DEBUG:boto:Method: HEAD 2022-09-03T05:37:21.072 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.073 DEBUG:boto:Data: 2022-09-03T05:37:21.073 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.074 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:21.075 DEBUG:boto:Port: 8000 2022-09-03T05:37:21.076 DEBUG:boto:Params: {} 2022-09-03T05:37:21.077 DEBUG:boto:Token: None 2022-09-03T05:37:21.078 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.079 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:46JKQ9+3vg9+2AvVSur8f6extls= 2022-09-03T05:37:21.080 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:46JKQ9+3vg9+2AvVSur8f6extls=', 'Content-Length': '0'} 2022-09-03T05:37:21.085 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'yL2uOIeC0l.C5YXdYoqQBvDIly3idxO'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d0e5af70eb2ef06d-006312e811-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.086 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:21.087 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.088 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.089 DEBUG:boto:path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.089 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.090 DEBUG:boto:Method: GET 2022-09-03T05:37:21.091 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.091 DEBUG:boto:Data: 2022-09-03T05:37:21.092 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.092 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:21.093 DEBUG:boto:Port: 8001 2022-09-03T05:37:21.093 DEBUG:boto:Params: {} 2022-09-03T05:37:21.094 DEBUG:boto:Token: None 2022-09-03T05:37:21.095 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.095 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EPqnZBGMIPZA6GhrIl8QpryQS6A= 2022-09-03T05:37:21.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EPqnZBGMIPZA6GhrIl8QpryQS6A=', 'Content-Length': '0'} 2022-09-03T05:37:21.099 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'yL2uOIeC0l.C5YXdYoqQBvDIly3idxO'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000e4c3cb8a51c4725-006312e811-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.101 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.102 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.103 DEBUG:boto:path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.104 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.105 DEBUG:boto:Method: GET 2022-09-03T05:37:21.107 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.109 DEBUG:boto:Data: 2022-09-03T05:37:21.110 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.111 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:21.112 DEBUG:boto:Port: 8000 2022-09-03T05:37:21.113 DEBUG:boto:Params: {} 2022-09-03T05:37:21.114 DEBUG:boto:Token: None 2022-09-03T05:37:21.116 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:21.117 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EPqnZBGMIPZA6GhrIl8QpryQS6A= 2022-09-03T05:37:21.120 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EPqnZBGMIPZA6GhrIl8QpryQS6A=', 'Content-Length': '0'} 2022-09-03T05:37:21.124 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'yL2uOIeC0l.C5YXdYoqQBvDIly3idxO'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000007b2134d595a44cc-006312e811-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.126 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:21.127 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:21.128 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.131 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.135 DEBUG:boto:Method: GET 2022-09-03T05:37:21.136 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.137 DEBUG:boto:Data: 2022-09-03T05:37:21.138 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.139 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:21.140 DEBUG:boto:Port: 8001 2022-09-03T05:37:21.141 DEBUG:boto:Params: {} 2022-09-03T05:37:21.142 DEBUG:boto:Token: None 2022-09-03T05:37:21.143 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/ 2022-09-03T05:37:21.144 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk= 2022-09-03T05:37:21.146 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk=', 'Content-Length': '0'} 2022-09-03T05:37:21.151 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a230f0cbed6070e0-006312e811-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.152 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:21.154 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:21.156 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:21.156 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.157 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.158 DEBUG:boto:Method: GET 2022-09-03T05:37:21.160 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.160 DEBUG:boto:Data: 2022-09-03T05:37:21.161 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.162 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:21.163 DEBUG:boto:Port: 8000 2022-09-03T05:37:21.164 DEBUG:boto:Params: {} 2022-09-03T05:37:21.165 DEBUG:boto:Token: None 2022-09-03T05:37:21.165 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/ 2022-09-03T05:37:21.166 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk= 2022-09-03T05:37:21.167 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk=', 'Content-Length': '0'} 2022-09-03T05:37:21.171 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000049f40ee6442d52ef-006312e811-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.173 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:21.174 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.175 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.176 DEBUG:boto:Method: HEAD 2022-09-03T05:37:21.177 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:21.178 DEBUG:boto:Data: 2022-09-03T05:37:21.179 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.180 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:21.180 DEBUG:boto:Port: 8001 2022-09-03T05:37:21.181 DEBUG:boto:Params: {} 2022-09-03T05:37:21.182 DEBUG:boto:Token: None 2022-09-03T05:37:21.183 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj 2022-09-03T05:37:21.184 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/b/qNfzjBCiKFNIsfbBrjNyXoss= 2022-09-03T05:37:21.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/b/qNfzjBCiKFNIsfbBrjNyXoss=', 'Content-Length': '0'} 2022-09-03T05:37:21.190 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003d2bd4e61c9e402c-006312e811-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.191 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.192 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.193 DEBUG:boto:Method: HEAD 2022-09-03T05:37:21.194 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:21.195 DEBUG:boto:Data: 2022-09-03T05:37:21.196 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.197 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:21.198 DEBUG:boto:Port: 8000 2022-09-03T05:37:21.198 DEBUG:boto:Params: {} 2022-09-03T05:37:21.199 DEBUG:boto:Token: None 2022-09-03T05:37:21.200 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj 2022-09-03T05:37:21.200 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/b/qNfzjBCiKFNIsfbBrjNyXoss= 2022-09-03T05:37:21.201 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/b/qNfzjBCiKFNIsfbBrjNyXoss=', 'Content-Length': '0'} 2022-09-03T05:37:21.206 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007a0de20fe2b34e10-006312e811-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.208 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:21.209 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.210 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.211 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.212 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.213 DEBUG:boto:Method: GET 2022-09-03T05:37:21.214 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.215 DEBUG:boto:Data: 2022-09-03T05:37:21.216 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.217 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:21.217 DEBUG:boto:Port: 8001 2022-09-03T05:37:21.218 DEBUG:boto:Params: {} 2022-09-03T05:37:21.218 DEBUG:boto:Token: None 2022-09-03T05:37:21.219 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.220 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HRcgasCv6K0crTWnZbVC6QyzUqY= 2022-09-03T05:37:21.221 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HRcgasCv6K0crTWnZbVC6QyzUqY=', 'Content-Length': '0'} 2022-09-03T05:37:21.223 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000099dbf30e06f6d568-006312e811-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.224 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.225 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.225 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.226 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.227 DEBUG:boto:Method: GET 2022-09-03T05:37:21.227 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.228 DEBUG:boto:Data: 2022-09-03T05:37:21.228 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.229 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:21.229 DEBUG:boto:Port: 8000 2022-09-03T05:37:21.230 DEBUG:boto:Params: {} 2022-09-03T05:37:21.230 DEBUG:boto:Token: None 2022-09-03T05:37:21.231 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.232 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HRcgasCv6K0crTWnZbVC6QyzUqY= 2022-09-03T05:37:21.233 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HRcgasCv6K0crTWnZbVC6QyzUqY=', 'Content-Length': '0'} 2022-09-03T05:37:21.236 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bcf51aa98b814081-006312e811-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.286 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:21.287 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:21.289 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.290 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.291 DEBUG:boto:Method: GET 2022-09-03T05:37:21.291 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.292 DEBUG:boto:Data: 2022-09-03T05:37:21.293 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.294 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:21.295 DEBUG:boto:Port: 8001 2022-09-03T05:37:21.296 DEBUG:boto:Params: {} 2022-09-03T05:37:21.298 DEBUG:boto:Token: None 2022-09-03T05:37:21.299 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/ 2022-09-03T05:37:21.300 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk= 2022-09-03T05:37:21.301 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk=', 'Content-Length': '0'} 2022-09-03T05:37:21.304 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000033b6cfc4ff2a9649-006312e811-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.305 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:21.312 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:21.313 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:21.314 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.314 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.315 DEBUG:boto:Method: GET 2022-09-03T05:37:21.316 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.316 DEBUG:boto:Data: 2022-09-03T05:37:21.317 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.318 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:21.318 DEBUG:boto:Port: 8000 2022-09-03T05:37:21.319 DEBUG:boto:Params: {} 2022-09-03T05:37:21.319 DEBUG:boto:Token: None 2022-09-03T05:37:21.320 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/ 2022-09-03T05:37:21.321 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk= 2022-09-03T05:37:21.322 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk=', 'Content-Length': '0'} 2022-09-03T05:37:21.325 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000f7d4f6099d7a4ee-006312e811-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.327 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:21.328 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:21.328 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.329 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.330 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.330 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.332 DEBUG:boto:Method: GET 2022-09-03T05:37:21.332 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.333 DEBUG:boto:Data: 2022-09-03T05:37:21.333 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.334 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:21.335 DEBUG:boto:Port: 8001 2022-09-03T05:37:21.336 DEBUG:boto:Params: {} 2022-09-03T05:37:21.336 DEBUG:boto:Token: None 2022-09-03T05:37:21.337 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.338 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mSWOty7gkZQ/QAk8d475fSQOkrw= 2022-09-03T05:37:21.339 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mSWOty7gkZQ/QAk8d475fSQOkrw=', 'Content-Length': '0'} 2022-09-03T05:37:21.342 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000016b91e8a4c240c7e-006312e811-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.391 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.392 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.393 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.395 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.396 DEBUG:boto:Method: GET 2022-09-03T05:37:21.397 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.398 DEBUG:boto:Data: 2022-09-03T05:37:21.400 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.401 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:21.402 DEBUG:boto:Port: 8000 2022-09-03T05:37:21.402 DEBUG:boto:Params: {} 2022-09-03T05:37:21.403 DEBUG:boto:Token: None 2022-09-03T05:37:21.404 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.404 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mSWOty7gkZQ/QAk8d475fSQOkrw= 2022-09-03T05:37:21.405 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mSWOty7gkZQ/QAk8d475fSQOkrw=', 'Content-Length': '0'} 2022-09-03T05:37:21.407 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007e77ded465aef0e4-006312e811-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.409 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.410 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.411 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.412 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.413 DEBUG:boto:Method: HEAD 2022-09-03T05:37:21.413 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.414 DEBUG:boto:Data: 2022-09-03T05:37:21.415 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.416 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:21.417 DEBUG:boto:Port: 8001 2022-09-03T05:37:21.418 DEBUG:boto:Params: {} 2022-09-03T05:37:21.419 DEBUG:boto:Token: None 2022-09-03T05:37:21.422 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.423 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B23ufhsH5AQpQP/zsIw+iR2pKaU= 2022-09-03T05:37:21.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B23ufhsH5AQpQP/zsIw+iR2pKaU=', 'Content-Length': '0'} 2022-09-03T05:37:21.427 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b893be48f517269e-006312e811-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.427 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.428 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.428 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.429 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.429 DEBUG:boto:Method: HEAD 2022-09-03T05:37:21.430 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.430 DEBUG:boto:Data: 2022-09-03T05:37:21.431 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.432 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:21.433 DEBUG:boto:Port: 8000 2022-09-03T05:37:21.433 DEBUG:boto:Params: {} 2022-09-03T05:37:21.434 DEBUG:boto:Token: None 2022-09-03T05:37:21.434 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.435 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B23ufhsH5AQpQP/zsIw+iR2pKaU= 2022-09-03T05:37:21.435 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B23ufhsH5AQpQP/zsIw+iR2pKaU=', 'Content-Length': '0'} 2022-09-03T05:37:21.438 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000036298e646be15167-006312e811-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.438 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:21.439 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.440 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.440 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.441 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.441 DEBUG:boto:Method: GET 2022-09-03T05:37:21.442 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.442 DEBUG:boto:Data: 2022-09-03T05:37:21.443 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.443 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:21.444 DEBUG:boto:Port: 8001 2022-09-03T05:37:21.444 DEBUG:boto:Params: {} 2022-09-03T05:37:21.445 DEBUG:boto:Token: None 2022-09-03T05:37:21.446 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.446 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mSWOty7gkZQ/QAk8d475fSQOkrw= 2022-09-03T05:37:21.447 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mSWOty7gkZQ/QAk8d475fSQOkrw=', 'Content-Length': '0'} 2022-09-03T05:37:21.450 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ab2028b06c20d13d-006312e811-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.451 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.452 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.453 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.454 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.455 DEBUG:boto:Method: GET 2022-09-03T05:37:21.456 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.457 DEBUG:boto:Data: 2022-09-03T05:37:21.457 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.458 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:21.459 DEBUG:boto:Port: 8000 2022-09-03T05:37:21.460 DEBUG:boto:Params: {} 2022-09-03T05:37:21.461 DEBUG:boto:Token: None 2022-09-03T05:37:21.463 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:21.464 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mSWOty7gkZQ/QAk8d475fSQOkrw= 2022-09-03T05:37:21.465 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mSWOty7gkZQ/QAk8d475fSQOkrw=', 'Content-Length': '0'} 2022-09-03T05:37:21.468 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e6c9be91f2d98b11-006312e811-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.470 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:21.471 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:21.472 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.473 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.474 DEBUG:boto:Method: GET 2022-09-03T05:37:21.474 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.475 DEBUG:boto:Data: 2022-09-03T05:37:21.476 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.478 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:21.479 DEBUG:boto:Port: 8001 2022-09-03T05:37:21.480 DEBUG:boto:Params: {} 2022-09-03T05:37:21.481 DEBUG:boto:Token: None 2022-09-03T05:37:21.482 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/ 2022-09-03T05:37:21.482 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk= 2022-09-03T05:37:21.483 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk=', 'Content-Length': '0'} 2022-09-03T05:37:21.486 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000902e49865d0373d0-006312e811-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.487 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:21.488 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:21.489 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:21.489 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.490 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.490 DEBUG:boto:Method: GET 2022-09-03T05:37:21.491 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.491 DEBUG:boto:Data: 2022-09-03T05:37:21.492 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.492 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:21.493 DEBUG:boto:Port: 8000 2022-09-03T05:37:21.494 DEBUG:boto:Params: {} 2022-09-03T05:37:21.494 DEBUG:boto:Token: None 2022-09-03T05:37:21.495 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/ 2022-09-03T05:37:21.496 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk= 2022-09-03T05:37:21.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk=', 'Content-Length': '0'} 2022-09-03T05:37:21.502 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000627beabbe0f179f5-006312e811-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.503 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:21.503 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.504 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.505 DEBUG:boto:Method: HEAD 2022-09-03T05:37:21.505 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:21.506 DEBUG:boto:Data: 2022-09-03T05:37:21.507 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.507 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:21.508 DEBUG:boto:Port: 8001 2022-09-03T05:37:21.509 DEBUG:boto:Params: {} 2022-09-03T05:37:21.510 DEBUG:boto:Token: None 2022-09-03T05:37:21.511 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj 2022-09-03T05:37:21.512 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/b/qNfzjBCiKFNIsfbBrjNyXoss= 2022-09-03T05:37:21.512 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/b/qNfzjBCiKFNIsfbBrjNyXoss=', 'Content-Length': '0'} 2022-09-03T05:37:21.516 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fd6b8698c6f9094e-006312e811-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.517 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.518 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.519 DEBUG:boto:Method: HEAD 2022-09-03T05:37:21.519 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:21.520 DEBUG:boto:Data: 2022-09-03T05:37:21.520 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.521 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:21.521 DEBUG:boto:Port: 8000 2022-09-03T05:37:21.522 DEBUG:boto:Params: {} 2022-09-03T05:37:21.522 DEBUG:boto:Token: None 2022-09-03T05:37:21.523 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj 2022-09-03T05:37:21.524 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/b/qNfzjBCiKFNIsfbBrjNyXoss= 2022-09-03T05:37:21.524 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/b/qNfzjBCiKFNIsfbBrjNyXoss=', 'Content-Length': '0'} 2022-09-03T05:37:21.528 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c59a4f33152558eb-006312e811-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.529 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:21.529 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.530 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.530 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.531 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.531 DEBUG:boto:Method: GET 2022-09-03T05:37:21.532 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.533 DEBUG:boto:Data: 2022-09-03T05:37:21.533 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.534 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:21.534 DEBUG:boto:Port: 8001 2022-09-03T05:37:21.535 DEBUG:boto:Params: {} 2022-09-03T05:37:21.535 DEBUG:boto:Token: None 2022-09-03T05:37:21.536 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.537 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HRcgasCv6K0crTWnZbVC6QyzUqY= 2022-09-03T05:37:21.537 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HRcgasCv6K0crTWnZbVC6QyzUqY=', 'Content-Length': '0'} 2022-09-03T05:37:21.540 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f2de2149df4248a5-006312e811-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.582 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.583 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.583 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.584 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.584 DEBUG:boto:Method: GET 2022-09-03T05:37:21.585 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.586 DEBUG:boto:Data: 2022-09-03T05:37:21.586 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.587 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:21.587 DEBUG:boto:Port: 8000 2022-09-03T05:37:21.588 DEBUG:boto:Params: {} 2022-09-03T05:37:21.589 DEBUG:boto:Token: None 2022-09-03T05:37:21.589 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.590 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HRcgasCv6K0crTWnZbVC6QyzUqY= 2022-09-03T05:37:21.592 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HRcgasCv6K0crTWnZbVC6QyzUqY=', 'Content-Length': '0'} 2022-09-03T05:37:21.594 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000098b0113d3bb9dc70-006312e811-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.596 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:21.597 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:21.598 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.599 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.600 DEBUG:boto:Method: GET 2022-09-03T05:37:21.601 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.602 DEBUG:boto:Data: 2022-09-03T05:37:21.603 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.604 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:21.604 DEBUG:boto:Port: 8001 2022-09-03T05:37:21.605 DEBUG:boto:Params: {} 2022-09-03T05:37:21.607 DEBUG:boto:Token: None 2022-09-03T05:37:21.607 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/ 2022-09-03T05:37:21.608 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk= 2022-09-03T05:37:21.609 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk=', 'Content-Length': '0'} 2022-09-03T05:37:21.612 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001042ddb7eac99184-006312e811-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.613 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:21.614 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:21.615 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:21.615 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.616 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.616 DEBUG:boto:Method: GET 2022-09-03T05:37:21.617 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.617 DEBUG:boto:Data: 2022-09-03T05:37:21.618 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.619 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:21.619 DEBUG:boto:Port: 8000 2022-09-03T05:37:21.620 DEBUG:boto:Params: {} 2022-09-03T05:37:21.620 DEBUG:boto:Token: None 2022-09-03T05:37:21.621 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/ 2022-09-03T05:37:21.621 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk= 2022-09-03T05:37:21.622 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk=', 'Content-Length': '0'} 2022-09-03T05:37:21.629 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000017e218bf9a05800-006312e811-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.666 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:21.668 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:21.669 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.671 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.672 DEBUG:boto:path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.673 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.674 DEBUG:boto:Method: GET 2022-09-03T05:37:21.675 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.676 DEBUG:boto:Data: 2022-09-03T05:37:21.677 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.678 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:21.679 DEBUG:boto:Port: 8001 2022-09-03T05:37:21.680 DEBUG:boto:Params: {} 2022-09-03T05:37:21.681 DEBUG:boto:Token: None 2022-09-03T05:37:21.682 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.683 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5J9Ntx4vc5pe6G2Py/iuHJcHtEM= 2022-09-03T05:37:21.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5J9Ntx4vc5pe6G2Py/iuHJcHtEM=', 'Content-Length': '0'} 2022-09-03T05:37:21.689 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ece5ba4a08bf7c85-006312e811-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.690 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.692 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.693 DEBUG:boto:path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.694 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.695 DEBUG:boto:Method: GET 2022-09-03T05:37:21.696 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.697 DEBUG:boto:Data: 2022-09-03T05:37:21.698 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.699 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:21.700 DEBUG:boto:Port: 8000 2022-09-03T05:37:21.701 DEBUG:boto:Params: {} 2022-09-03T05:37:21.703 DEBUG:boto:Token: None 2022-09-03T05:37:21.703 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.704 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5J9Ntx4vc5pe6G2Py/iuHJcHtEM= 2022-09-03T05:37:21.705 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5J9Ntx4vc5pe6G2Py/iuHJcHtEM=', 'Content-Length': '0'} 2022-09-03T05:37:21.707 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006cdb85904ccf452c-006312e811-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.708 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.709 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.709 DEBUG:boto:path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.710 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.711 DEBUG:boto:Method: HEAD 2022-09-03T05:37:21.711 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.712 DEBUG:boto:Data: 2022-09-03T05:37:21.712 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.713 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:21.714 DEBUG:boto:Port: 8001 2022-09-03T05:37:21.714 DEBUG:boto:Params: {} 2022-09-03T05:37:21.715 DEBUG:boto:Token: None 2022-09-03T05:37:21.715 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.716 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DuyRd/NYmDNtcJTso/0KjcNgizA= 2022-09-03T05:37:21.717 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DuyRd/NYmDNtcJTso/0KjcNgizA=', 'Content-Length': '0'} 2022-09-03T05:37:21.722 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006ef1cd64ce01c83d-006312e811-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.723 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.723 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.724 DEBUG:boto:path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.724 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.725 DEBUG:boto:Method: HEAD 2022-09-03T05:37:21.725 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.726 DEBUG:boto:Data: 2022-09-03T05:37:21.726 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.727 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:21.727 DEBUG:boto:Port: 8000 2022-09-03T05:37:21.728 DEBUG:boto:Params: {} 2022-09-03T05:37:21.728 DEBUG:boto:Token: None 2022-09-03T05:37:21.729 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.730 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DuyRd/NYmDNtcJTso/0KjcNgizA= 2022-09-03T05:37:21.730 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DuyRd/NYmDNtcJTso/0KjcNgizA=', 'Content-Length': '0'} 2022-09-03T05:37:21.733 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000033c1bfd31b2b2460-006312e811-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.733 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:21.734 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.735 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.735 DEBUG:boto:path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.736 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.736 DEBUG:boto:Method: GET 2022-09-03T05:37:21.737 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.737 DEBUG:boto:Data: 2022-09-03T05:37:21.738 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.738 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:21.739 DEBUG:boto:Port: 8001 2022-09-03T05:37:21.740 DEBUG:boto:Params: {} 2022-09-03T05:37:21.740 DEBUG:boto:Token: None 2022-09-03T05:37:21.741 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.741 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5J9Ntx4vc5pe6G2Py/iuHJcHtEM= 2022-09-03T05:37:21.743 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5J9Ntx4vc5pe6G2Py/iuHJcHtEM=', 'Content-Length': '0'} 2022-09-03T05:37:21.746 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fb94652b28ae5544-006312e811-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.748 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.749 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.749 DEBUG:boto:path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.750 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.750 DEBUG:boto:Method: GET 2022-09-03T05:37:21.751 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.751 DEBUG:boto:Data: 2022-09-03T05:37:21.752 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.753 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:21.754 DEBUG:boto:Port: 8000 2022-09-03T05:37:21.754 DEBUG:boto:Params: {} 2022-09-03T05:37:21.755 DEBUG:boto:Token: None 2022-09-03T05:37:21.755 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:21.756 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5J9Ntx4vc5pe6G2Py/iuHJcHtEM= 2022-09-03T05:37:21.757 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5J9Ntx4vc5pe6G2Py/iuHJcHtEM=', 'Content-Length': '0'} 2022-09-03T05:37:21.759 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003466a6fdad20b6d2-006312e811-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.761 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:21.761 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:21.762 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.762 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.764 DEBUG:boto:Method: GET 2022-09-03T05:37:21.765 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.766 DEBUG:boto:Data: 2022-09-03T05:37:21.767 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.768 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:21.769 DEBUG:boto:Port: 8001 2022-09-03T05:37:21.770 DEBUG:boto:Params: {} 2022-09-03T05:37:21.771 DEBUG:boto:Token: None 2022-09-03T05:37:21.772 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/ 2022-09-03T05:37:21.773 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk= 2022-09-03T05:37:21.773 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk=', 'Content-Length': '0'} 2022-09-03T05:37:21.778 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bda49227885f45a1-006312e811-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.779 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:21.780 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:21.781 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:21.781 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.782 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.782 DEBUG:boto:Method: GET 2022-09-03T05:37:21.783 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.783 DEBUG:boto:Data: 2022-09-03T05:37:21.784 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.785 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:21.786 DEBUG:boto:Port: 8000 2022-09-03T05:37:21.787 DEBUG:boto:Params: {} 2022-09-03T05:37:21.788 DEBUG:boto:Token: None 2022-09-03T05:37:21.789 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/ 2022-09-03T05:37:21.790 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk= 2022-09-03T05:37:21.791 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk=', 'Content-Length': '0'} 2022-09-03T05:37:21.799 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000011bb9afda08ad99d-006312e811-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.801 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:21.802 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.803 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.804 DEBUG:boto:Method: HEAD 2022-09-03T05:37:21.805 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:21.809 DEBUG:boto:Data: 2022-09-03T05:37:21.810 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.811 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:21.812 DEBUG:boto:Port: 8001 2022-09-03T05:37:21.813 DEBUG:boto:Params: {} 2022-09-03T05:37:21.814 DEBUG:boto:Token: None 2022-09-03T05:37:21.815 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj 2022-09-03T05:37:21.816 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/b/qNfzjBCiKFNIsfbBrjNyXoss= 2022-09-03T05:37:21.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/b/qNfzjBCiKFNIsfbBrjNyXoss=', 'Content-Length': '0'} 2022-09-03T05:37:21.825 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007266c6e9298077f0-006312e811-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.827 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.828 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.829 DEBUG:boto:Method: HEAD 2022-09-03T05:37:21.830 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:21.831 DEBUG:boto:Data: 2022-09-03T05:37:21.832 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.835 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:21.836 DEBUG:boto:Port: 8000 2022-09-03T05:37:21.837 DEBUG:boto:Params: {} 2022-09-03T05:37:21.838 DEBUG:boto:Token: None 2022-09-03T05:37:21.839 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj 2022-09-03T05:37:21.840 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/b/qNfzjBCiKFNIsfbBrjNyXoss= 2022-09-03T05:37:21.841 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/b/qNfzjBCiKFNIsfbBrjNyXoss=', 'Content-Length': '0'} 2022-09-03T05:37:21.850 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a81e465196bec1b1-006312e811-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.852 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:21.853 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.854 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.855 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.856 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.857 DEBUG:boto:Method: GET 2022-09-03T05:37:21.858 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.859 DEBUG:boto:Data: 2022-09-03T05:37:21.860 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.861 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:21.862 DEBUG:boto:Port: 8001 2022-09-03T05:37:21.863 DEBUG:boto:Params: {} 2022-09-03T05:37:21.864 DEBUG:boto:Token: None 2022-09-03T05:37:21.865 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.866 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HRcgasCv6K0crTWnZbVC6QyzUqY= 2022-09-03T05:37:21.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HRcgasCv6K0crTWnZbVC6QyzUqY=', 'Content-Length': '0'} 2022-09-03T05:37:21.870 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008471c66f8170be20-006312e811-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.872 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.872 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.873 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.874 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.875 DEBUG:boto:Method: GET 2022-09-03T05:37:21.876 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.876 DEBUG:boto:Data: 2022-09-03T05:37:21.877 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.878 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:21.879 DEBUG:boto:Port: 8000 2022-09-03T05:37:21.880 DEBUG:boto:Params: {} 2022-09-03T05:37:21.881 DEBUG:boto:Token: None 2022-09-03T05:37:21.883 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:21.884 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HRcgasCv6K0crTWnZbVC6QyzUqY= 2022-09-03T05:37:21.885 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HRcgasCv6K0crTWnZbVC6QyzUqY=', 'Content-Length': '0'} 2022-09-03T05:37:21.888 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004eec272da850f333-006312e811-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.891 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:21.891 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:21.892 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.893 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.894 DEBUG:boto:Method: GET 2022-09-03T05:37:21.894 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.896 DEBUG:boto:Data: 2022-09-03T05:37:21.896 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.897 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:21.897 DEBUG:boto:Port: 8001 2022-09-03T05:37:21.899 DEBUG:boto:Params: {} 2022-09-03T05:37:21.899 DEBUG:boto:Token: None 2022-09-03T05:37:21.901 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/ 2022-09-03T05:37:21.903 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk= 2022-09-03T05:37:21.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk=', 'Content-Length': '0'} 2022-09-03T05:37:21.908 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000037df6c07fade0eeb-006312e811-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.911 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:21.912 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:21.913 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:21.918 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.919 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.920 DEBUG:boto:Method: GET 2022-09-03T05:37:21.920 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:21.921 DEBUG:boto:Data: 2022-09-03T05:37:21.922 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.923 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:21.924 DEBUG:boto:Port: 8000 2022-09-03T05:37:21.924 DEBUG:boto:Params: {} 2022-09-03T05:37:21.925 DEBUG:boto:Token: None 2022-09-03T05:37:21.927 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/ 2022-09-03T05:37:21.928 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk= 2022-09-03T05:37:21.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HubF5/TkrasiNnc2nwNzbRlpkUk=', 'Content-Length': '0'} 2022-09-03T05:37:21.935 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ce41f1745da1cd3b-006312e811-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.936 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:21.938 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:21.939 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.942 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.943 DEBUG:boto:path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:21.944 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:21.945 DEBUG:boto:Method: GET 2022-09-03T05:37:21.946 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:21.948 DEBUG:boto:Data: 2022-09-03T05:37:21.949 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.950 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:21.951 DEBUG:boto:Port: 8001 2022-09-03T05:37:21.952 DEBUG:boto:Params: {} 2022-09-03T05:37:21.953 DEBUG:boto:Token: None 2022-09-03T05:37:21.954 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:21.955 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MPin/zUwd15OI/vA+6Yn+hNuXfc= 2022-09-03T05:37:21.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MPin/zUwd15OI/vA+6Yn+hNuXfc=', 'Content-Length': '0'} 2022-09-03T05:37:21.960 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'QfRyggjm1gZ7H.LeMCey0hHybmVWONG'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008986524efcf49071-006312e811-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.961 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.962 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.963 DEBUG:boto:path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:21.964 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:21.965 DEBUG:boto:Method: GET 2022-09-03T05:37:21.966 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:21.967 DEBUG:boto:Data: 2022-09-03T05:37:21.968 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.969 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:21.970 DEBUG:boto:Port: 8000 2022-09-03T05:37:21.971 DEBUG:boto:Params: {} 2022-09-03T05:37:21.972 DEBUG:boto:Token: None 2022-09-03T05:37:21.973 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:21.974 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MPin/zUwd15OI/vA+6Yn+hNuXfc= 2022-09-03T05:37:21.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MPin/zUwd15OI/vA+6Yn+hNuXfc=', 'Content-Length': '0'} 2022-09-03T05:37:21.980 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'QfRyggjm1gZ7H.LeMCey0hHybmVWONG'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ad533e653300772b-006312e811-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:21.981 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:21.986 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:21.987 DEBUG:boto:path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:21.988 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:21.989 DEBUG:boto:Method: HEAD 2022-09-03T05:37:21.990 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:21.991 DEBUG:boto:Data: 2022-09-03T05:37:21.991 DEBUG:boto:Headers: {} 2022-09-03T05:37:21.992 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:21.993 DEBUG:boto:Port: 8001 2022-09-03T05:37:21.994 DEBUG:boto:Params: {} 2022-09-03T05:37:21.995 DEBUG:boto:Token: None 2022-09-03T05:37:21.996 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:21 GMT /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:21.996 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Iq3aPLA/QSmCUaDAoN9iXHMGUsk= 2022-09-03T05:37:21.997 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Iq3aPLA/QSmCUaDAoN9iXHMGUsk=', 'Content-Length': '0'} 2022-09-03T05:37:22.002 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'QfRyggjm1gZ7H.LeMCey0hHybmVWONG'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ed653760ea9c6aae-006312e811-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.003 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.004 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.005 DEBUG:boto:path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:22.006 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:22.011 DEBUG:boto:Method: HEAD 2022-09-03T05:37:22.012 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:22.013 DEBUG:boto:Data: 2022-09-03T05:37:22.014 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.015 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:22.016 DEBUG:boto:Port: 8000 2022-09-03T05:37:22.017 DEBUG:boto:Params: {} 2022-09-03T05:37:22.018 DEBUG:boto:Token: None 2022-09-03T05:37:22.019 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:22.019 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z/YqJEnb/fbFezPGhTy2Pe7hHEs= 2022-09-03T05:37:22.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z/YqJEnb/fbFezPGhTy2Pe7hHEs=', 'Content-Length': '0'} 2022-09-03T05:37:22.025 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'QfRyggjm1gZ7H.LeMCey0hHybmVWONG'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003c2443e6713b551d-006312e812-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.027 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:22.028 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.029 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.030 DEBUG:boto:path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:22.030 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:22.031 DEBUG:boto:Method: GET 2022-09-03T05:37:22.032 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:22.033 DEBUG:boto:Data: 2022-09-03T05:37:22.034 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.035 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:22.036 DEBUG:boto:Port: 8001 2022-09-03T05:37:22.037 DEBUG:boto:Params: {} 2022-09-03T05:37:22.038 DEBUG:boto:Token: None 2022-09-03T05:37:22.040 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:22.042 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fAiOHoV5LvRTXt2kGTX+jJUvSr8= 2022-09-03T05:37:22.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fAiOHoV5LvRTXt2kGTX+jJUvSr8=', 'Content-Length': '0'} 2022-09-03T05:37:22.046 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'QfRyggjm1gZ7H.LeMCey0hHybmVWONG'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000020467f52e788a9bf-006312e812-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.054 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.055 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.056 DEBUG:boto:path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:22.058 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:22.059 DEBUG:boto:Method: GET 2022-09-03T05:37:22.060 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:22.062 DEBUG:boto:Data: 2022-09-03T05:37:22.063 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.064 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:22.065 DEBUG:boto:Port: 8000 2022-09-03T05:37:22.066 DEBUG:boto:Params: {} 2022-09-03T05:37:22.067 DEBUG:boto:Token: None 2022-09-03T05:37:22.068 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:22.069 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fAiOHoV5LvRTXt2kGTX+jJUvSr8= 2022-09-03T05:37:22.070 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fAiOHoV5LvRTXt2kGTX+jJUvSr8=', 'Content-Length': '0'} 2022-09-03T05:37:22.073 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'QfRyggjm1gZ7H.LeMCey0hHybmVWONG'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000083edb17d448a97ff-006312e812-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.121 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:22.122 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:22.122 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.123 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.123 DEBUG:boto:Method: GET 2022-09-03T05:37:22.124 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.124 DEBUG:boto:Data: 2022-09-03T05:37:22.125 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.126 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:22.126 DEBUG:boto:Port: 8001 2022-09-03T05:37:22.127 DEBUG:boto:Params: {} 2022-09-03T05:37:22.128 DEBUG:boto:Token: None 2022-09-03T05:37:22.128 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/ 2022-09-03T05:37:22.129 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU= 2022-09-03T05:37:22.130 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU=', 'Content-Length': '0'} 2022-09-03T05:37:22.134 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f624ace3eb167485-006312e812-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.139 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:22.141 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:22.143 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:22.144 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.144 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.145 DEBUG:boto:Method: GET 2022-09-03T05:37:22.146 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.146 DEBUG:boto:Data: 2022-09-03T05:37:22.148 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.151 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:22.151 DEBUG:boto:Port: 8000 2022-09-03T05:37:22.152 DEBUG:boto:Params: {} 2022-09-03T05:37:22.153 DEBUG:boto:Token: None 2022-09-03T05:37:22.154 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/ 2022-09-03T05:37:22.154 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU= 2022-09-03T05:37:22.155 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU=', 'Content-Length': '0'} 2022-09-03T05:37:22.159 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000089bbca69e8df1976-006312e812-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.160 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:22.160 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.161 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.162 DEBUG:boto:Method: HEAD 2022-09-03T05:37:22.163 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:22.164 DEBUG:boto:Data: 2022-09-03T05:37:22.166 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.167 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:22.169 DEBUG:boto:Port: 8001 2022-09-03T05:37:22.169 DEBUG:boto:Params: {} 2022-09-03T05:37:22.170 DEBUG:boto:Token: None 2022-09-03T05:37:22.171 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj 2022-09-03T05:37:22.173 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hNlt/Jrz29gaEGbWE+uHjDJ6wuI= 2022-09-03T05:37:22.174 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hNlt/Jrz29gaEGbWE+uHjDJ6wuI=', 'Content-Length': '0'} 2022-09-03T05:37:22.179 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000827ee0d225bac419-006312e812-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.181 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.182 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.184 DEBUG:boto:Method: HEAD 2022-09-03T05:37:22.185 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:22.186 DEBUG:boto:Data: 2022-09-03T05:37:22.186 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.189 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:22.190 DEBUG:boto:Port: 8000 2022-09-03T05:37:22.190 DEBUG:boto:Params: {} 2022-09-03T05:37:22.191 DEBUG:boto:Token: None 2022-09-03T05:37:22.192 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj 2022-09-03T05:37:22.193 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hNlt/Jrz29gaEGbWE+uHjDJ6wuI= 2022-09-03T05:37:22.194 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hNlt/Jrz29gaEGbWE+uHjDJ6wuI=', 'Content-Length': '0'} 2022-09-03T05:37:22.199 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d845c3c7375d1c7f-006312e812-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.200 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:22.201 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.202 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.203 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.204 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.205 DEBUG:boto:Method: GET 2022-09-03T05:37:22.205 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.206 DEBUG:boto:Data: 2022-09-03T05:37:22.207 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.208 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:22.208 DEBUG:boto:Port: 8001 2022-09-03T05:37:22.209 DEBUG:boto:Params: {} 2022-09-03T05:37:22.210 DEBUG:boto:Token: None 2022-09-03T05:37:22.210 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.211 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1eU5YFpNREQtkoHIsDx2O7fDBv4= 2022-09-03T05:37:22.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1eU5YFpNREQtkoHIsDx2O7fDBv4=', 'Content-Length': '0'} 2022-09-03T05:37:22.214 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a6b08f1b0846e6f9-006312e812-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.254 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.256 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.257 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.258 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.259 DEBUG:boto:Method: GET 2022-09-03T05:37:22.260 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.260 DEBUG:boto:Data: 2022-09-03T05:37:22.261 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.261 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:22.262 DEBUG:boto:Port: 8000 2022-09-03T05:37:22.263 DEBUG:boto:Params: {} 2022-09-03T05:37:22.263 DEBUG:boto:Token: None 2022-09-03T05:37:22.264 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.265 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1eU5YFpNREQtkoHIsDx2O7fDBv4= 2022-09-03T05:37:22.266 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1eU5YFpNREQtkoHIsDx2O7fDBv4=', 'Content-Length': '0'} 2022-09-03T05:37:22.268 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d9f6b1cebe4fc2e6-006312e812-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.269 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:22.269 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:22.270 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.271 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.271 DEBUG:boto:Method: GET 2022-09-03T05:37:22.272 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.273 DEBUG:boto:Data: 2022-09-03T05:37:22.274 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.275 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:22.276 DEBUG:boto:Port: 8001 2022-09-03T05:37:22.276 DEBUG:boto:Params: {} 2022-09-03T05:37:22.277 DEBUG:boto:Token: None 2022-09-03T05:37:22.279 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/ 2022-09-03T05:37:22.279 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU= 2022-09-03T05:37:22.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU=', 'Content-Length': '0'} 2022-09-03T05:37:22.283 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000071c206962b766649-006312e812-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.284 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:22.285 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:22.285 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:22.286 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.286 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.287 DEBUG:boto:Method: GET 2022-09-03T05:37:22.287 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.288 DEBUG:boto:Data: 2022-09-03T05:37:22.288 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.289 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:22.289 DEBUG:boto:Port: 8000 2022-09-03T05:37:22.290 DEBUG:boto:Params: {} 2022-09-03T05:37:22.290 DEBUG:boto:Token: None 2022-09-03T05:37:22.291 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/ 2022-09-03T05:37:22.292 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU= 2022-09-03T05:37:22.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU=', 'Content-Length': '0'} 2022-09-03T05:37:22.295 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e7b1b54be9280bc3-006312e812-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.296 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:22.297 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:22.298 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.299 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.299 DEBUG:boto:path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.300 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.300 DEBUG:boto:Method: GET 2022-09-03T05:37:22.301 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.302 DEBUG:boto:Data: 2022-09-03T05:37:22.302 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.303 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:22.303 DEBUG:boto:Port: 8001 2022-09-03T05:37:22.304 DEBUG:boto:Params: {} 2022-09-03T05:37:22.305 DEBUG:boto:Token: None 2022-09-03T05:37:22.305 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.306 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A0bCKTbtVfw9iDjwWD8qXBYvObk= 2022-09-03T05:37:22.306 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A0bCKTbtVfw9iDjwWD8qXBYvObk=', 'Content-Length': '0'} 2022-09-03T05:37:22.309 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '94bPO72IhlWQEGwklU15Pvpr-qRv.lm'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000e3078d567d87e6b-006312e812-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.310 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.310 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.311 DEBUG:boto:path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.311 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.312 DEBUG:boto:Method: GET 2022-09-03T05:37:22.313 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.313 DEBUG:boto:Data: 2022-09-03T05:37:22.314 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.314 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:22.315 DEBUG:boto:Port: 8000 2022-09-03T05:37:22.315 DEBUG:boto:Params: {} 2022-09-03T05:37:22.316 DEBUG:boto:Token: None 2022-09-03T05:37:22.317 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.317 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A0bCKTbtVfw9iDjwWD8qXBYvObk= 2022-09-03T05:37:22.318 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A0bCKTbtVfw9iDjwWD8qXBYvObk=', 'Content-Length': '0'} 2022-09-03T05:37:22.321 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '94bPO72IhlWQEGwklU15Pvpr-qRv.lm'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fd04ff96b23ffc51-006312e812-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.362 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.365 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.366 DEBUG:boto:path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.367 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.368 DEBUG:boto:Method: HEAD 2022-09-03T05:37:22.369 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.370 DEBUG:boto:Data: 2022-09-03T05:37:22.371 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.371 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:22.373 DEBUG:boto:Port: 8001 2022-09-03T05:37:22.374 DEBUG:boto:Params: {} 2022-09-03T05:37:22.376 DEBUG:boto:Token: None 2022-09-03T05:37:22.377 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.378 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BlL1jPU7WPTUVDqk2tcYE1c3L+k= 2022-09-03T05:37:22.379 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BlL1jPU7WPTUVDqk2tcYE1c3L+k=', 'Content-Length': '0'} 2022-09-03T05:37:22.382 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '94bPO72IhlWQEGwklU15Pvpr-qRv.lm'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a031bf19c32dfb60-006312e812-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.384 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.385 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.386 DEBUG:boto:path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.387 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.388 DEBUG:boto:Method: HEAD 2022-09-03T05:37:22.389 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.390 DEBUG:boto:Data: 2022-09-03T05:37:22.391 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.391 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:22.392 DEBUG:boto:Port: 8000 2022-09-03T05:37:22.392 DEBUG:boto:Params: {} 2022-09-03T05:37:22.393 DEBUG:boto:Token: None 2022-09-03T05:37:22.393 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.395 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BlL1jPU7WPTUVDqk2tcYE1c3L+k= 2022-09-03T05:37:22.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BlL1jPU7WPTUVDqk2tcYE1c3L+k=', 'Content-Length': '0'} 2022-09-03T05:37:22.399 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '94bPO72IhlWQEGwklU15Pvpr-qRv.lm'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000027430cc2a8ec322-006312e812-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.401 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:22.402 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.403 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.403 DEBUG:boto:path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.404 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.404 DEBUG:boto:Method: GET 2022-09-03T05:37:22.406 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.407 DEBUG:boto:Data: 2022-09-03T05:37:22.408 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.409 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:22.410 DEBUG:boto:Port: 8001 2022-09-03T05:37:22.411 DEBUG:boto:Params: {} 2022-09-03T05:37:22.412 DEBUG:boto:Token: None 2022-09-03T05:37:22.413 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.414 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A0bCKTbtVfw9iDjwWD8qXBYvObk= 2022-09-03T05:37:22.416 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A0bCKTbtVfw9iDjwWD8qXBYvObk=', 'Content-Length': '0'} 2022-09-03T05:37:22.420 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '94bPO72IhlWQEGwklU15Pvpr-qRv.lm'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ede2c26dcd0e43f1-006312e812-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.421 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.421 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.422 DEBUG:boto:path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.423 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.424 DEBUG:boto:Method: GET 2022-09-03T05:37:22.425 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.426 DEBUG:boto:Data: 2022-09-03T05:37:22.427 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.428 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:22.428 DEBUG:boto:Port: 8000 2022-09-03T05:37:22.429 DEBUG:boto:Params: {} 2022-09-03T05:37:22.431 DEBUG:boto:Token: None 2022-09-03T05:37:22.432 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:22.433 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A0bCKTbtVfw9iDjwWD8qXBYvObk= 2022-09-03T05:37:22.434 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A0bCKTbtVfw9iDjwWD8qXBYvObk=', 'Content-Length': '0'} 2022-09-03T05:37:22.442 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '94bPO72IhlWQEGwklU15Pvpr-qRv.lm'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000033f87d68ae01724f-006312e812-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.443 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:22.444 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:22.444 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.445 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.445 DEBUG:boto:Method: GET 2022-09-03T05:37:22.446 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.447 DEBUG:boto:Data: 2022-09-03T05:37:22.447 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.448 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:22.448 DEBUG:boto:Port: 8001 2022-09-03T05:37:22.449 DEBUG:boto:Params: {} 2022-09-03T05:37:22.449 DEBUG:boto:Token: None 2022-09-03T05:37:22.450 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/ 2022-09-03T05:37:22.450 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU= 2022-09-03T05:37:22.451 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU=', 'Content-Length': '0'} 2022-09-03T05:37:22.454 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000022b9c73f9aa9bdc5-006312e812-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.455 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:22.456 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:22.457 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:22.457 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.458 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.458 DEBUG:boto:Method: GET 2022-09-03T05:37:22.459 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.460 DEBUG:boto:Data: 2022-09-03T05:37:22.460 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.461 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:22.461 DEBUG:boto:Port: 8000 2022-09-03T05:37:22.462 DEBUG:boto:Params: {} 2022-09-03T05:37:22.462 DEBUG:boto:Token: None 2022-09-03T05:37:22.463 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/ 2022-09-03T05:37:22.463 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU= 2022-09-03T05:37:22.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU=', 'Content-Length': '0'} 2022-09-03T05:37:22.467 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009d720f5160d18f19-006312e812-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.510 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:22.513 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.513 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.514 DEBUG:boto:Method: HEAD 2022-09-03T05:37:22.515 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:22.515 DEBUG:boto:Data: 2022-09-03T05:37:22.516 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.516 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:22.517 DEBUG:boto:Port: 8001 2022-09-03T05:37:22.517 DEBUG:boto:Params: {} 2022-09-03T05:37:22.518 DEBUG:boto:Token: None 2022-09-03T05:37:22.518 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj 2022-09-03T05:37:22.519 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hNlt/Jrz29gaEGbWE+uHjDJ6wuI= 2022-09-03T05:37:22.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hNlt/Jrz29gaEGbWE+uHjDJ6wuI=', 'Content-Length': '0'} 2022-09-03T05:37:22.526 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006bbdc18fbee818bc-006312e812-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.527 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.528 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.528 DEBUG:boto:Method: HEAD 2022-09-03T05:37:22.529 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:22.529 DEBUG:boto:Data: 2022-09-03T05:37:22.530 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.531 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:22.532 DEBUG:boto:Port: 8000 2022-09-03T05:37:22.533 DEBUG:boto:Params: {} 2022-09-03T05:37:22.534 DEBUG:boto:Token: None 2022-09-03T05:37:22.534 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj 2022-09-03T05:37:22.535 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hNlt/Jrz29gaEGbWE+uHjDJ6wuI= 2022-09-03T05:37:22.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hNlt/Jrz29gaEGbWE+uHjDJ6wuI=', 'Content-Length': '0'} 2022-09-03T05:37:22.540 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007d929426ce309152-006312e812-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.541 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:22.541 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.542 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.543 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.544 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.545 DEBUG:boto:Method: GET 2022-09-03T05:37:22.546 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.547 DEBUG:boto:Data: 2022-09-03T05:37:22.548 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.550 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:22.551 DEBUG:boto:Port: 8001 2022-09-03T05:37:22.552 DEBUG:boto:Params: {} 2022-09-03T05:37:22.553 DEBUG:boto:Token: None 2022-09-03T05:37:22.554 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.555 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1eU5YFpNREQtkoHIsDx2O7fDBv4= 2022-09-03T05:37:22.555 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1eU5YFpNREQtkoHIsDx2O7fDBv4=', 'Content-Length': '0'} 2022-09-03T05:37:22.558 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000059f5994fae73a3ff-006312e812-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.598 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.599 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.599 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.600 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.601 DEBUG:boto:Method: GET 2022-09-03T05:37:22.601 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.602 DEBUG:boto:Data: 2022-09-03T05:37:22.602 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.603 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:22.603 DEBUG:boto:Port: 8000 2022-09-03T05:37:22.606 DEBUG:boto:Params: {} 2022-09-03T05:37:22.607 DEBUG:boto:Token: None 2022-09-03T05:37:22.611 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.612 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1eU5YFpNREQtkoHIsDx2O7fDBv4= 2022-09-03T05:37:22.613 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1eU5YFpNREQtkoHIsDx2O7fDBv4=', 'Content-Length': '0'} 2022-09-03T05:37:22.616 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fd08bf779e85965f-006312e812-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.618 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:22.619 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:22.620 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.621 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.623 DEBUG:boto:Method: GET 2022-09-03T05:37:22.624 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.625 DEBUG:boto:Data: 2022-09-03T05:37:22.626 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.627 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:22.628 DEBUG:boto:Port: 8001 2022-09-03T05:37:22.629 DEBUG:boto:Params: {} 2022-09-03T05:37:22.630 DEBUG:boto:Token: None 2022-09-03T05:37:22.631 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/ 2022-09-03T05:37:22.632 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU= 2022-09-03T05:37:22.633 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU=', 'Content-Length': '0'} 2022-09-03T05:37:22.637 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006ca6cde04392da70-006312e812-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.639 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:22.640 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:22.641 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:22.642 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.643 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.644 DEBUG:boto:Method: GET 2022-09-03T05:37:22.645 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.646 DEBUG:boto:Data: 2022-09-03T05:37:22.646 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.647 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:22.647 DEBUG:boto:Port: 8000 2022-09-03T05:37:22.648 DEBUG:boto:Params: {} 2022-09-03T05:37:22.649 DEBUG:boto:Token: None 2022-09-03T05:37:22.649 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/ 2022-09-03T05:37:22.650 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU= 2022-09-03T05:37:22.651 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU=', 'Content-Length': '0'} 2022-09-03T05:37:22.654 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007b05079f225c9909-006312e812-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.655 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:22.655 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:22.656 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.657 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.658 DEBUG:boto:path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.659 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.659 DEBUG:boto:Method: GET 2022-09-03T05:37:22.660 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.660 DEBUG:boto:Data: 2022-09-03T05:37:22.661 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.661 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:22.663 DEBUG:boto:Port: 8001 2022-09-03T05:37:22.664 DEBUG:boto:Params: {} 2022-09-03T05:37:22.665 DEBUG:boto:Token: None 2022-09-03T05:37:22.666 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.667 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SWgNfP1Q10sp527oTj1dk9RDRNc= 2022-09-03T05:37:22.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SWgNfP1Q10sp527oTj1dk9RDRNc=', 'Content-Length': '0'} 2022-09-03T05:37:22.671 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'ORmfT2fVQ5EFIFI89EGuLigbPyfeObT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000adc41a249412fb7a-006312e812-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.672 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.673 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.674 DEBUG:boto:path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.674 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.675 DEBUG:boto:Method: GET 2022-09-03T05:37:22.676 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.676 DEBUG:boto:Data: 2022-09-03T05:37:22.677 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.678 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:22.678 DEBUG:boto:Port: 8000 2022-09-03T05:37:22.679 DEBUG:boto:Params: {} 2022-09-03T05:37:22.680 DEBUG:boto:Token: None 2022-09-03T05:37:22.681 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.683 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SWgNfP1Q10sp527oTj1dk9RDRNc= 2022-09-03T05:37:22.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SWgNfP1Q10sp527oTj1dk9RDRNc=', 'Content-Length': '0'} 2022-09-03T05:37:22.687 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'ORmfT2fVQ5EFIFI89EGuLigbPyfeObT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000be9edbf8e47ed1b1-006312e812-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.688 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.688 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.689 DEBUG:boto:path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.689 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.690 DEBUG:boto:Method: HEAD 2022-09-03T05:37:22.691 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.691 DEBUG:boto:Data: 2022-09-03T05:37:22.692 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.692 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:22.693 DEBUG:boto:Port: 8001 2022-09-03T05:37:22.694 DEBUG:boto:Params: {} 2022-09-03T05:37:22.695 DEBUG:boto:Token: None 2022-09-03T05:37:22.696 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.697 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Nu2lvBP7aVO9BYaqU0x3sU5UsME= 2022-09-03T05:37:22.698 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Nu2lvBP7aVO9BYaqU0x3sU5UsME=', 'Content-Length': '0'} 2022-09-03T05:37:22.701 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'ORmfT2fVQ5EFIFI89EGuLigbPyfeObT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a1267b6d464b26fe-006312e812-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.702 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.703 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.703 DEBUG:boto:path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.704 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.704 DEBUG:boto:Method: HEAD 2022-09-03T05:37:22.705 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.706 DEBUG:boto:Data: 2022-09-03T05:37:22.706 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.707 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:22.708 DEBUG:boto:Port: 8000 2022-09-03T05:37:22.708 DEBUG:boto:Params: {} 2022-09-03T05:37:22.709 DEBUG:boto:Token: None 2022-09-03T05:37:22.710 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.711 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Nu2lvBP7aVO9BYaqU0x3sU5UsME= 2022-09-03T05:37:22.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Nu2lvBP7aVO9BYaqU0x3sU5UsME=', 'Content-Length': '0'} 2022-09-03T05:37:22.715 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'ORmfT2fVQ5EFIFI89EGuLigbPyfeObT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d907679306279898-006312e812-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.716 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:22.717 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.718 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.719 DEBUG:boto:path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.720 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.721 DEBUG:boto:Method: GET 2022-09-03T05:37:22.722 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.723 DEBUG:boto:Data: 2022-09-03T05:37:22.724 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.725 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:22.726 DEBUG:boto:Port: 8001 2022-09-03T05:37:22.727 DEBUG:boto:Params: {} 2022-09-03T05:37:22.728 DEBUG:boto:Token: None 2022-09-03T05:37:22.729 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.730 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SWgNfP1Q10sp527oTj1dk9RDRNc= 2022-09-03T05:37:22.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SWgNfP1Q10sp527oTj1dk9RDRNc=', 'Content-Length': '0'} 2022-09-03T05:37:22.734 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'ORmfT2fVQ5EFIFI89EGuLigbPyfeObT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007cbee0d53c3c066a-006312e812-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.736 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.737 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.738 DEBUG:boto:path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.739 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.740 DEBUG:boto:Method: GET 2022-09-03T05:37:22.741 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.742 DEBUG:boto:Data: 2022-09-03T05:37:22.743 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.744 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:22.745 DEBUG:boto:Port: 8000 2022-09-03T05:37:22.746 DEBUG:boto:Params: {} 2022-09-03T05:37:22.747 DEBUG:boto:Token: None 2022-09-03T05:37:22.748 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:22.749 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SWgNfP1Q10sp527oTj1dk9RDRNc= 2022-09-03T05:37:22.750 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SWgNfP1Q10sp527oTj1dk9RDRNc=', 'Content-Length': '0'} 2022-09-03T05:37:22.753 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'ORmfT2fVQ5EFIFI89EGuLigbPyfeObT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000068bd20fd0e8eb3cf-006312e812-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.794 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:22.795 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:22.795 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.796 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.796 DEBUG:boto:Method: GET 2022-09-03T05:37:22.797 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.798 DEBUG:boto:Data: 2022-09-03T05:37:22.799 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.799 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:22.800 DEBUG:boto:Port: 8001 2022-09-03T05:37:22.801 DEBUG:boto:Params: {} 2022-09-03T05:37:22.802 DEBUG:boto:Token: None 2022-09-03T05:37:22.803 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/ 2022-09-03T05:37:22.804 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU= 2022-09-03T05:37:22.806 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU=', 'Content-Length': '0'} 2022-09-03T05:37:22.809 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c5c0cac9d1831613-006312e812-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.811 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:22.812 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:22.813 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:22.814 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.815 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.816 DEBUG:boto:Method: GET 2022-09-03T05:37:22.816 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.817 DEBUG:boto:Data: 2022-09-03T05:37:22.817 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.818 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:22.819 DEBUG:boto:Port: 8000 2022-09-03T05:37:22.820 DEBUG:boto:Params: {} 2022-09-03T05:37:22.821 DEBUG:boto:Token: None 2022-09-03T05:37:22.821 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/ 2022-09-03T05:37:22.822 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU= 2022-09-03T05:37:22.823 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU=', 'Content-Length': '0'} 2022-09-03T05:37:22.827 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cf77e0a8d0a0d126-006312e812-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.829 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:22.830 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.830 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.831 DEBUG:boto:Method: HEAD 2022-09-03T05:37:22.831 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:22.832 DEBUG:boto:Data: 2022-09-03T05:37:22.833 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.834 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:22.835 DEBUG:boto:Port: 8001 2022-09-03T05:37:22.836 DEBUG:boto:Params: {} 2022-09-03T05:37:22.837 DEBUG:boto:Token: None 2022-09-03T05:37:22.838 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj 2022-09-03T05:37:22.839 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hNlt/Jrz29gaEGbWE+uHjDJ6wuI= 2022-09-03T05:37:22.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hNlt/Jrz29gaEGbWE+uHjDJ6wuI=', 'Content-Length': '0'} 2022-09-03T05:37:22.847 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000108b61a51b41fb50-006312e812-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.848 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.849 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.850 DEBUG:boto:Method: HEAD 2022-09-03T05:37:22.851 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:22.852 DEBUG:boto:Data: 2022-09-03T05:37:22.853 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.855 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:22.856 DEBUG:boto:Port: 8000 2022-09-03T05:37:22.858 DEBUG:boto:Params: {} 2022-09-03T05:37:22.859 DEBUG:boto:Token: None 2022-09-03T05:37:22.860 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj 2022-09-03T05:37:22.862 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hNlt/Jrz29gaEGbWE+uHjDJ6wuI= 2022-09-03T05:37:22.864 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hNlt/Jrz29gaEGbWE+uHjDJ6wuI=', 'Content-Length': '0'} 2022-09-03T05:37:22.870 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000026f3d2fd87736107-006312e812-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.872 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:22.873 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.874 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.875 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.876 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.877 DEBUG:boto:Method: GET 2022-09-03T05:37:22.878 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.879 DEBUG:boto:Data: 2022-09-03T05:37:22.880 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.881 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:22.882 DEBUG:boto:Port: 8001 2022-09-03T05:37:22.883 DEBUG:boto:Params: {} 2022-09-03T05:37:22.884 DEBUG:boto:Token: None 2022-09-03T05:37:22.885 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.886 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1eU5YFpNREQtkoHIsDx2O7fDBv4= 2022-09-03T05:37:22.887 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1eU5YFpNREQtkoHIsDx2O7fDBv4=', 'Content-Length': '0'} 2022-09-03T05:37:22.890 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006640d6eb3cba120e-006312e812-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.892 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.893 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.893 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.894 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.895 DEBUG:boto:Method: GET 2022-09-03T05:37:22.898 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.899 DEBUG:boto:Data: 2022-09-03T05:37:22.899 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.900 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:22.900 DEBUG:boto:Port: 8000 2022-09-03T05:37:22.901 DEBUG:boto:Params: {} 2022-09-03T05:37:22.902 DEBUG:boto:Token: None 2022-09-03T05:37:22.902 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:22.903 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1eU5YFpNREQtkoHIsDx2O7fDBv4= 2022-09-03T05:37:22.903 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1eU5YFpNREQtkoHIsDx2O7fDBv4=', 'Content-Length': '0'} 2022-09-03T05:37:22.905 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d300a174d0d0f3be-006312e812-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.946 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:22.947 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:22.948 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.949 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.950 DEBUG:boto:Method: GET 2022-09-03T05:37:22.951 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.952 DEBUG:boto:Data: 2022-09-03T05:37:22.952 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.953 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:22.954 DEBUG:boto:Port: 8001 2022-09-03T05:37:22.955 DEBUG:boto:Params: {} 2022-09-03T05:37:22.956 DEBUG:boto:Token: None 2022-09-03T05:37:22.957 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/ 2022-09-03T05:37:22.958 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU= 2022-09-03T05:37:22.959 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU=', 'Content-Length': '0'} 2022-09-03T05:37:22.963 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000813feb4291df58d7-006312e812-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.963 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:22.964 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:22.965 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:22.966 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.966 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.967 DEBUG:boto:Method: GET 2022-09-03T05:37:22.968 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:22.968 DEBUG:boto:Data: 2022-09-03T05:37:22.969 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.970 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:22.970 DEBUG:boto:Port: 8000 2022-09-03T05:37:22.971 DEBUG:boto:Params: {} 2022-09-03T05:37:22.972 DEBUG:boto:Token: None 2022-09-03T05:37:22.973 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/ 2022-09-03T05:37:22.973 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU= 2022-09-03T05:37:22.974 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lw/YMiW+qOjB3d7jBfR4gXJsSEU=', 'Content-Length': '0'} 2022-09-03T05:37:22.978 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005f8ca43896b20c15-006312e812-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:22.980 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:22.981 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:22.982 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:22.983 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:22.983 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:22.984 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:22.985 DEBUG:boto:Method: GET 2022-09-03T05:37:22.985 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:22.986 DEBUG:boto:Data: 2022-09-03T05:37:22.986 DEBUG:boto:Headers: {} 2022-09-03T05:37:22.987 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:22.988 DEBUG:boto:Port: 8001 2022-09-03T05:37:22.988 DEBUG:boto:Params: {} 2022-09-03T05:37:22.989 DEBUG:boto:Token: None 2022-09-03T05:37:22.990 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:22 GMT /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:22.991 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yk9phWKOroqzPpVfZMQxQnU5CkU= 2022-09-03T05:37:22.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yk9phWKOroqzPpVfZMQxQnU5CkU=', 'Content-Length': '0'} 2022-09-03T05:37:22.995 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kUiw73n7LyoUiDcS9JXEfdlCebChUCE'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000daf1529fc70d3351-006312e812-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.038 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.039 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.039 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:23.040 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:23.041 DEBUG:boto:Method: GET 2022-09-03T05:37:23.041 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:23.042 DEBUG:boto:Data: 2022-09-03T05:37:23.042 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.043 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:23.044 DEBUG:boto:Port: 8000 2022-09-03T05:37:23.044 DEBUG:boto:Params: {} 2022-09-03T05:37:23.045 DEBUG:boto:Token: None 2022-09-03T05:37:23.046 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:23.046 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z/LtpbLY5APhXaD8TXCUZYBIdVM= 2022-09-03T05:37:23.047 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z/LtpbLY5APhXaD8TXCUZYBIdVM=', 'Content-Length': '0'} 2022-09-03T05:37:23.049 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kUiw73n7LyoUiDcS9JXEfdlCebChUCE'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009c02b558fb97624b-006312e813-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.050 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.051 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.051 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:23.052 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:23.052 DEBUG:boto:Method: HEAD 2022-09-03T05:37:23.053 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:23.053 DEBUG:boto:Data: 2022-09-03T05:37:23.054 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.054 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:23.055 DEBUG:boto:Port: 8001 2022-09-03T05:37:23.055 DEBUG:boto:Params: {} 2022-09-03T05:37:23.056 DEBUG:boto:Token: None 2022-09-03T05:37:23.056 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:23.057 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oDwOzLPdMUZ4QXAvXTIfTPh6rbQ= 2022-09-03T05:37:23.058 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oDwOzLPdMUZ4QXAvXTIfTPh6rbQ=', 'Content-Length': '0'} 2022-09-03T05:37:23.060 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kUiw73n7LyoUiDcS9JXEfdlCebChUCE'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006e9ddbbec115ba45-006312e813-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.060 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.061 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.062 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:23.062 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:23.063 DEBUG:boto:Method: HEAD 2022-09-03T05:37:23.063 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:23.064 DEBUG:boto:Data: 2022-09-03T05:37:23.064 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.065 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:23.065 DEBUG:boto:Port: 8000 2022-09-03T05:37:23.066 DEBUG:boto:Params: {} 2022-09-03T05:37:23.066 DEBUG:boto:Token: None 2022-09-03T05:37:23.067 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:23.068 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oDwOzLPdMUZ4QXAvXTIfTPh6rbQ= 2022-09-03T05:37:23.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oDwOzLPdMUZ4QXAvXTIfTPh6rbQ=', 'Content-Length': '0'} 2022-09-03T05:37:23.070 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kUiw73n7LyoUiDcS9JXEfdlCebChUCE'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008bf9e547d69279ef-006312e813-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.071 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:23.071 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.072 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.072 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:23.073 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:23.073 DEBUG:boto:Method: GET 2022-09-03T05:37:23.074 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:23.075 DEBUG:boto:Data: 2022-09-03T05:37:23.075 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.076 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:23.076 DEBUG:boto:Port: 8001 2022-09-03T05:37:23.077 DEBUG:boto:Params: {} 2022-09-03T05:37:23.078 DEBUG:boto:Token: None 2022-09-03T05:37:23.079 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:23.080 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z/LtpbLY5APhXaD8TXCUZYBIdVM= 2022-09-03T05:37:23.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z/LtpbLY5APhXaD8TXCUZYBIdVM=', 'Content-Length': '0'} 2022-09-03T05:37:23.084 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kUiw73n7LyoUiDcS9JXEfdlCebChUCE'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f635af29242a6b88-006312e813-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.086 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.087 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.088 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:23.089 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:23.090 DEBUG:boto:Method: GET 2022-09-03T05:37:23.091 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:23.092 DEBUG:boto:Data: 2022-09-03T05:37:23.093 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.094 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:23.094 DEBUG:boto:Port: 8000 2022-09-03T05:37:23.095 DEBUG:boto:Params: {} 2022-09-03T05:37:23.095 DEBUG:boto:Token: None 2022-09-03T05:37:23.096 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:23.097 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z/LtpbLY5APhXaD8TXCUZYBIdVM= 2022-09-03T05:37:23.097 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z/LtpbLY5APhXaD8TXCUZYBIdVM=', 'Content-Length': '0'} 2022-09-03T05:37:23.100 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kUiw73n7LyoUiDcS9JXEfdlCebChUCE'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ff2bcc602c424d45-006312e813-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.101 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:23.102 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:23.102 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.103 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.104 DEBUG:boto:Method: GET 2022-09-03T05:37:23.104 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.105 DEBUG:boto:Data: 2022-09-03T05:37:23.105 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.106 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:23.106 DEBUG:boto:Port: 8001 2022-09-03T05:37:23.107 DEBUG:boto:Params: {} 2022-09-03T05:37:23.107 DEBUG:boto:Token: None 2022-09-03T05:37:23.108 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/ 2022-09-03T05:37:23.109 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40= 2022-09-03T05:37:23.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40=', 'Content-Length': '0'} 2022-09-03T05:37:23.123 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000047be30cb100d74-006312e813-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.124 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:23.125 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:23.125 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:23.126 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.126 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.127 DEBUG:boto:Method: GET 2022-09-03T05:37:23.128 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.128 DEBUG:boto:Data: 2022-09-03T05:37:23.129 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.129 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:23.130 DEBUG:boto:Port: 8000 2022-09-03T05:37:23.131 DEBUG:boto:Params: {} 2022-09-03T05:37:23.132 DEBUG:boto:Token: None 2022-09-03T05:37:23.132 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/ 2022-09-03T05:37:23.133 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40= 2022-09-03T05:37:23.134 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40=', 'Content-Length': '0'} 2022-09-03T05:37:23.137 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001514fd9855d6aedb-006312e813-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.138 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:23.139 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.139 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.140 DEBUG:boto:Method: HEAD 2022-09-03T05:37:23.140 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:23.141 DEBUG:boto:Data: 2022-09-03T05:37:23.141 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.142 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:23.142 DEBUG:boto:Port: 8001 2022-09-03T05:37:23.143 DEBUG:boto:Params: {} 2022-09-03T05:37:23.144 DEBUG:boto:Token: None 2022-09-03T05:37:23.144 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj 2022-09-03T05:37:23.145 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HtMD+5y+L/7uB9/QxRqhiP8sexM= 2022-09-03T05:37:23.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HtMD+5y+L/7uB9/QxRqhiP8sexM=', 'Content-Length': '0'} 2022-09-03T05:37:23.152 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c76dfd1eee556503-006312e813-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.153 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.154 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.155 DEBUG:boto:Method: HEAD 2022-09-03T05:37:23.156 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:23.157 DEBUG:boto:Data: 2022-09-03T05:37:23.158 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.160 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:23.161 DEBUG:boto:Port: 8000 2022-09-03T05:37:23.162 DEBUG:boto:Params: {} 2022-09-03T05:37:23.163 DEBUG:boto:Token: None 2022-09-03T05:37:23.164 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj 2022-09-03T05:37:23.165 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HtMD+5y+L/7uB9/QxRqhiP8sexM= 2022-09-03T05:37:23.166 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HtMD+5y+L/7uB9/QxRqhiP8sexM=', 'Content-Length': '0'} 2022-09-03T05:37:23.170 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000069c80e43f322a067-006312e813-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.171 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:23.171 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.172 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.172 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.173 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.174 DEBUG:boto:Method: GET 2022-09-03T05:37:23.174 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.175 DEBUG:boto:Data: 2022-09-03T05:37:23.175 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.176 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:23.176 DEBUG:boto:Port: 8001 2022-09-03T05:37:23.177 DEBUG:boto:Params: {} 2022-09-03T05:37:23.178 DEBUG:boto:Token: None 2022-09-03T05:37:23.179 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.180 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MXlwoaVi5g/MkiJrKV1CecGjRjg= 2022-09-03T05:37:23.181 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MXlwoaVi5g/MkiJrKV1CecGjRjg=', 'Content-Length': '0'} 2022-09-03T05:37:23.183 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f5336bdd3113542d-006312e813-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.230 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.231 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.231 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.232 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.233 DEBUG:boto:Method: GET 2022-09-03T05:37:23.233 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.234 DEBUG:boto:Data: 2022-09-03T05:37:23.234 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.235 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:23.235 DEBUG:boto:Port: 8000 2022-09-03T05:37:23.236 DEBUG:boto:Params: {} 2022-09-03T05:37:23.237 DEBUG:boto:Token: None 2022-09-03T05:37:23.238 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.239 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MXlwoaVi5g/MkiJrKV1CecGjRjg= 2022-09-03T05:37:23.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MXlwoaVi5g/MkiJrKV1CecGjRjg=', 'Content-Length': '0'} 2022-09-03T05:37:23.245 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000094cb5774908a4957-006312e813-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.247 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:23.247 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:23.249 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.249 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.250 DEBUG:boto:Method: GET 2022-09-03T05:37:23.251 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.251 DEBUG:boto:Data: 2022-09-03T05:37:23.252 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.252 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:23.253 DEBUG:boto:Port: 8001 2022-09-03T05:37:23.253 DEBUG:boto:Params: {} 2022-09-03T05:37:23.254 DEBUG:boto:Token: None 2022-09-03T05:37:23.255 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/ 2022-09-03T05:37:23.256 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40= 2022-09-03T05:37:23.257 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40=', 'Content-Length': '0'} 2022-09-03T05:37:23.262 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001d4613e852721a62-006312e813-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.263 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:23.264 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:23.264 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:23.265 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.265 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.266 DEBUG:boto:Method: GET 2022-09-03T05:37:23.266 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.267 DEBUG:boto:Data: 2022-09-03T05:37:23.268 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.268 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:23.269 DEBUG:boto:Port: 8000 2022-09-03T05:37:23.269 DEBUG:boto:Params: {} 2022-09-03T05:37:23.270 DEBUG:boto:Token: None 2022-09-03T05:37:23.270 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/ 2022-09-03T05:37:23.271 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40= 2022-09-03T05:37:23.272 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40=', 'Content-Length': '0'} 2022-09-03T05:37:23.275 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000452f59e4bab1371c-006312e813-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.276 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:23.277 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:23.278 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.278 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.279 DEBUG:boto:path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.279 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.282 DEBUG:boto:Method: GET 2022-09-03T05:37:23.282 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.283 DEBUG:boto:Data: 2022-09-03T05:37:23.284 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.284 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:23.285 DEBUG:boto:Port: 8001 2022-09-03T05:37:23.286 DEBUG:boto:Params: {} 2022-09-03T05:37:23.287 DEBUG:boto:Token: None 2022-09-03T05:37:23.287 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.288 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nfEdKqF1Ld0nuBaQRoLKlwAJsjk= 2022-09-03T05:37:23.289 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nfEdKqF1Ld0nuBaQRoLKlwAJsjk=', 'Content-Length': '0'} 2022-09-03T05:37:23.291 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b52de3465a938ce9-006312e813-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.292 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.293 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.293 DEBUG:boto:path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.294 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.295 DEBUG:boto:Method: GET 2022-09-03T05:37:23.296 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.297 DEBUG:boto:Data: 2022-09-03T05:37:23.298 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.299 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:23.299 DEBUG:boto:Port: 8000 2022-09-03T05:37:23.300 DEBUG:boto:Params: {} 2022-09-03T05:37:23.301 DEBUG:boto:Token: None 2022-09-03T05:37:23.302 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.303 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nfEdKqF1Ld0nuBaQRoLKlwAJsjk= 2022-09-03T05:37:23.303 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nfEdKqF1Ld0nuBaQRoLKlwAJsjk=', 'Content-Length': '0'} 2022-09-03T05:37:23.306 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cf57accb4cc77a7e-006312e813-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.356 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.356 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.357 DEBUG:boto:path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.358 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.358 DEBUG:boto:Method: HEAD 2022-09-03T05:37:23.359 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.359 DEBUG:boto:Data: 2022-09-03T05:37:23.360 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.360 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:23.361 DEBUG:boto:Port: 8001 2022-09-03T05:37:23.361 DEBUG:boto:Params: {} 2022-09-03T05:37:23.362 DEBUG:boto:Token: None 2022-09-03T05:37:23.363 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.364 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eugm70dlEmWtwbNmdZQKinIzehw= 2022-09-03T05:37:23.365 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eugm70dlEmWtwbNmdZQKinIzehw=', 'Content-Length': '0'} 2022-09-03T05:37:23.367 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005d43af0e9826ad99-006312e813-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.368 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.368 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.369 DEBUG:boto:path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.369 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.370 DEBUG:boto:Method: HEAD 2022-09-03T05:37:23.371 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.371 DEBUG:boto:Data: 2022-09-03T05:37:23.372 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.372 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:23.373 DEBUG:boto:Port: 8000 2022-09-03T05:37:23.374 DEBUG:boto:Params: {} 2022-09-03T05:37:23.374 DEBUG:boto:Token: None 2022-09-03T05:37:23.375 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.376 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eugm70dlEmWtwbNmdZQKinIzehw= 2022-09-03T05:37:23.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eugm70dlEmWtwbNmdZQKinIzehw=', 'Content-Length': '0'} 2022-09-03T05:37:23.380 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001c0d290557cc0a83-006312e813-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.381 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:23.383 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.384 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.385 DEBUG:boto:path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.387 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.388 DEBUG:boto:Method: GET 2022-09-03T05:37:23.389 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.390 DEBUG:boto:Data: 2022-09-03T05:37:23.391 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.391 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:23.392 DEBUG:boto:Port: 8001 2022-09-03T05:37:23.393 DEBUG:boto:Params: {} 2022-09-03T05:37:23.394 DEBUG:boto:Token: None 2022-09-03T05:37:23.394 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.395 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nfEdKqF1Ld0nuBaQRoLKlwAJsjk= 2022-09-03T05:37:23.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nfEdKqF1Ld0nuBaQRoLKlwAJsjk=', 'Content-Length': '0'} 2022-09-03T05:37:23.398 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f3f27042138a0570-006312e813-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.438 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.439 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.440 DEBUG:boto:path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.440 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.447 DEBUG:boto:Method: GET 2022-09-03T05:37:23.448 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.448 DEBUG:boto:Data: 2022-09-03T05:37:23.449 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.449 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:23.450 DEBUG:boto:Port: 8000 2022-09-03T05:37:23.450 DEBUG:boto:Params: {} 2022-09-03T05:37:23.451 DEBUG:boto:Token: None 2022-09-03T05:37:23.451 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:23.452 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nfEdKqF1Ld0nuBaQRoLKlwAJsjk= 2022-09-03T05:37:23.453 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nfEdKqF1Ld0nuBaQRoLKlwAJsjk=', 'Content-Length': '0'} 2022-09-03T05:37:23.455 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005412768bc001454d-006312e813-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.456 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:23.457 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:23.457 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.458 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.458 DEBUG:boto:Method: GET 2022-09-03T05:37:23.459 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.460 DEBUG:boto:Data: 2022-09-03T05:37:23.461 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.461 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:23.462 DEBUG:boto:Port: 8001 2022-09-03T05:37:23.462 DEBUG:boto:Params: {} 2022-09-03T05:37:23.463 DEBUG:boto:Token: None 2022-09-03T05:37:23.464 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/ 2022-09-03T05:37:23.464 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40= 2022-09-03T05:37:23.465 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40=', 'Content-Length': '0'} 2022-09-03T05:37:23.470 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000385eae56be95a744-006312e813-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.471 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:23.473 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:23.474 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:23.475 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.475 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.476 DEBUG:boto:Method: GET 2022-09-03T05:37:23.477 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.478 DEBUG:boto:Data: 2022-09-03T05:37:23.479 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.480 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:23.481 DEBUG:boto:Port: 8000 2022-09-03T05:37:23.481 DEBUG:boto:Params: {} 2022-09-03T05:37:23.482 DEBUG:boto:Token: None 2022-09-03T05:37:23.483 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/ 2022-09-03T05:37:23.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40= 2022-09-03T05:37:23.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40=', 'Content-Length': '0'} 2022-09-03T05:37:23.487 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000039d3cf0b9a2dbc15-006312e813-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.488 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:23.489 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.489 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.490 DEBUG:boto:Method: HEAD 2022-09-03T05:37:23.491 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:23.491 DEBUG:boto:Data: 2022-09-03T05:37:23.492 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.492 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:23.493 DEBUG:boto:Port: 8001 2022-09-03T05:37:23.493 DEBUG:boto:Params: {} 2022-09-03T05:37:23.494 DEBUG:boto:Token: None 2022-09-03T05:37:23.495 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj 2022-09-03T05:37:23.495 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HtMD+5y+L/7uB9/QxRqhiP8sexM= 2022-09-03T05:37:23.496 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HtMD+5y+L/7uB9/QxRqhiP8sexM=', 'Content-Length': '0'} 2022-09-03T05:37:23.501 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cef22161b915ef71-006312e813-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.502 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.503 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.503 DEBUG:boto:Method: HEAD 2022-09-03T05:37:23.504 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:23.504 DEBUG:boto:Data: 2022-09-03T05:37:23.505 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.505 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:23.506 DEBUG:boto:Port: 8000 2022-09-03T05:37:23.506 DEBUG:boto:Params: {} 2022-09-03T05:37:23.507 DEBUG:boto:Token: None 2022-09-03T05:37:23.508 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj 2022-09-03T05:37:23.508 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HtMD+5y+L/7uB9/QxRqhiP8sexM= 2022-09-03T05:37:23.509 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HtMD+5y+L/7uB9/QxRqhiP8sexM=', 'Content-Length': '0'} 2022-09-03T05:37:23.512 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001b8289a9402de55d-006312e813-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.513 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:23.514 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.515 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.515 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.516 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.516 DEBUG:boto:Method: GET 2022-09-03T05:37:23.517 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.517 DEBUG:boto:Data: 2022-09-03T05:37:23.518 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.519 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:23.520 DEBUG:boto:Port: 8001 2022-09-03T05:37:23.520 DEBUG:boto:Params: {} 2022-09-03T05:37:23.521 DEBUG:boto:Token: None 2022-09-03T05:37:23.522 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.522 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MXlwoaVi5g/MkiJrKV1CecGjRjg= 2022-09-03T05:37:23.523 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MXlwoaVi5g/MkiJrKV1CecGjRjg=', 'Content-Length': '0'} 2022-09-03T05:37:23.525 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d985ff33f256d0a4-006312e813-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.526 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.527 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.527 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.528 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.528 DEBUG:boto:Method: GET 2022-09-03T05:37:23.530 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.531 DEBUG:boto:Data: 2022-09-03T05:37:23.531 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.532 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:23.533 DEBUG:boto:Port: 8000 2022-09-03T05:37:23.535 DEBUG:boto:Params: {} 2022-09-03T05:37:23.537 DEBUG:boto:Token: None 2022-09-03T05:37:23.538 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.540 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MXlwoaVi5g/MkiJrKV1CecGjRjg= 2022-09-03T05:37:23.541 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MXlwoaVi5g/MkiJrKV1CecGjRjg=', 'Content-Length': '0'} 2022-09-03T05:37:23.544 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003ea93d2b3739b4a5-006312e813-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.590 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:23.592 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:23.593 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.594 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.595 DEBUG:boto:Method: GET 2022-09-03T05:37:23.596 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.597 DEBUG:boto:Data: 2022-09-03T05:37:23.598 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.599 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:23.599 DEBUG:boto:Port: 8001 2022-09-03T05:37:23.600 DEBUG:boto:Params: {} 2022-09-03T05:37:23.601 DEBUG:boto:Token: None 2022-09-03T05:37:23.602 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/ 2022-09-03T05:37:23.602 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40= 2022-09-03T05:37:23.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40=', 'Content-Length': '0'} 2022-09-03T05:37:23.607 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000064b1339748a0de2c-006312e813-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.607 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:23.609 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:23.609 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:23.610 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.610 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.611 DEBUG:boto:Method: GET 2022-09-03T05:37:23.612 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.612 DEBUG:boto:Data: 2022-09-03T05:37:23.613 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.613 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:23.615 DEBUG:boto:Port: 8000 2022-09-03T05:37:23.615 DEBUG:boto:Params: {} 2022-09-03T05:37:23.616 DEBUG:boto:Token: None 2022-09-03T05:37:23.616 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/ 2022-09-03T05:37:23.617 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40= 2022-09-03T05:37:23.618 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40=', 'Content-Length': '0'} 2022-09-03T05:37:23.622 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ff591474653b5f3b-006312e813-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.622 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:23.623 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:23.624 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.624 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.625 DEBUG:boto:path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.625 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.626 DEBUG:boto:Method: GET 2022-09-03T05:37:23.626 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.627 DEBUG:boto:Data: 2022-09-03T05:37:23.628 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.628 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:23.629 DEBUG:boto:Port: 8001 2022-09-03T05:37:23.629 DEBUG:boto:Params: {} 2022-09-03T05:37:23.630 DEBUG:boto:Token: None 2022-09-03T05:37:23.630 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.631 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z/L6KYhkZHYD1EXCNzMEdYsJDCM= 2022-09-03T05:37:23.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z/L6KYhkZHYD1EXCNzMEdYsJDCM=', 'Content-Length': '0'} 2022-09-03T05:37:23.634 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000054e2cf21198f2034-006312e813-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.674 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.675 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.675 DEBUG:boto:path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.676 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.676 DEBUG:boto:Method: GET 2022-09-03T05:37:23.677 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.678 DEBUG:boto:Data: 2022-09-03T05:37:23.678 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.679 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:23.679 DEBUG:boto:Port: 8000 2022-09-03T05:37:23.680 DEBUG:boto:Params: {} 2022-09-03T05:37:23.681 DEBUG:boto:Token: None 2022-09-03T05:37:23.682 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.682 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z/L6KYhkZHYD1EXCNzMEdYsJDCM= 2022-09-03T05:37:23.683 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z/L6KYhkZHYD1EXCNzMEdYsJDCM=', 'Content-Length': '0'} 2022-09-03T05:37:23.685 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ecf00e2dd4a96878-006312e813-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.686 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.687 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.687 DEBUG:boto:path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.688 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.688 DEBUG:boto:Method: HEAD 2022-09-03T05:37:23.689 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.689 DEBUG:boto:Data: 2022-09-03T05:37:23.690 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.692 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:23.693 DEBUG:boto:Port: 8001 2022-09-03T05:37:23.694 DEBUG:boto:Params: {} 2022-09-03T05:37:23.695 DEBUG:boto:Token: None 2022-09-03T05:37:23.696 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.697 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6UnR5dkPGBMA5uL8pM8R95G3aeA= 2022-09-03T05:37:23.698 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6UnR5dkPGBMA5uL8pM8R95G3aeA=', 'Content-Length': '0'} 2022-09-03T05:37:23.700 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000033f0f496d0d0b139-006312e813-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.702 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.702 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.703 DEBUG:boto:path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.704 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.704 DEBUG:boto:Method: HEAD 2022-09-03T05:37:23.705 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.706 DEBUG:boto:Data: 2022-09-03T05:37:23.707 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.708 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:23.709 DEBUG:boto:Port: 8000 2022-09-03T05:37:23.709 DEBUG:boto:Params: {} 2022-09-03T05:37:23.710 DEBUG:boto:Token: None 2022-09-03T05:37:23.711 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.712 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6UnR5dkPGBMA5uL8pM8R95G3aeA= 2022-09-03T05:37:23.713 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6UnR5dkPGBMA5uL8pM8R95G3aeA=', 'Content-Length': '0'} 2022-09-03T05:37:23.715 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000007d10c2ebba99a9b-006312e813-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.716 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:23.716 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.717 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.717 DEBUG:boto:path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.718 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.719 DEBUG:boto:Method: GET 2022-09-03T05:37:23.719 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.720 DEBUG:boto:Data: 2022-09-03T05:37:23.720 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.721 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:23.721 DEBUG:boto:Port: 8001 2022-09-03T05:37:23.722 DEBUG:boto:Params: {} 2022-09-03T05:37:23.722 DEBUG:boto:Token: None 2022-09-03T05:37:23.723 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.723 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z/L6KYhkZHYD1EXCNzMEdYsJDCM= 2022-09-03T05:37:23.724 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z/L6KYhkZHYD1EXCNzMEdYsJDCM=', 'Content-Length': '0'} 2022-09-03T05:37:23.727 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000967de47b6c51a41b-006312e813-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.728 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.728 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.729 DEBUG:boto:path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.729 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.730 DEBUG:boto:Method: GET 2022-09-03T05:37:23.731 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.732 DEBUG:boto:Data: 2022-09-03T05:37:23.733 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.733 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:23.734 DEBUG:boto:Port: 8000 2022-09-03T05:37:23.735 DEBUG:boto:Params: {} 2022-09-03T05:37:23.736 DEBUG:boto:Token: None 2022-09-03T05:37:23.738 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:23.739 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z/L6KYhkZHYD1EXCNzMEdYsJDCM= 2022-09-03T05:37:23.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z/L6KYhkZHYD1EXCNzMEdYsJDCM=', 'Content-Length': '0'} 2022-09-03T05:37:23.743 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006720dbfe20144372-006312e813-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.745 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:23.746 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:23.746 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.747 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.748 DEBUG:boto:Method: GET 2022-09-03T05:37:23.749 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.749 DEBUG:boto:Data: 2022-09-03T05:37:23.750 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.751 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:23.752 DEBUG:boto:Port: 8001 2022-09-03T05:37:23.752 DEBUG:boto:Params: {} 2022-09-03T05:37:23.754 DEBUG:boto:Token: None 2022-09-03T05:37:23.755 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/ 2022-09-03T05:37:23.755 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40= 2022-09-03T05:37:23.756 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40=', 'Content-Length': '0'} 2022-09-03T05:37:23.761 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000010eac4978676ad4b-006312e813-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.762 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:23.763 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:23.763 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:23.764 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.765 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.765 DEBUG:boto:Method: GET 2022-09-03T05:37:23.766 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.766 DEBUG:boto:Data: 2022-09-03T05:37:23.767 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.768 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:23.768 DEBUG:boto:Port: 8000 2022-09-03T05:37:23.769 DEBUG:boto:Params: {} 2022-09-03T05:37:23.770 DEBUG:boto:Token: None 2022-09-03T05:37:23.772 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/ 2022-09-03T05:37:23.773 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40= 2022-09-03T05:37:23.774 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40=', 'Content-Length': '0'} 2022-09-03T05:37:23.778 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006d5d4b96daa7d712-006312e813-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.779 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:23.780 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.780 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.781 DEBUG:boto:Method: HEAD 2022-09-03T05:37:23.781 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:23.782 DEBUG:boto:Data: 2022-09-03T05:37:23.783 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.784 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:23.785 DEBUG:boto:Port: 8001 2022-09-03T05:37:23.785 DEBUG:boto:Params: {} 2022-09-03T05:37:23.786 DEBUG:boto:Token: None 2022-09-03T05:37:23.788 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj 2022-09-03T05:37:23.790 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HtMD+5y+L/7uB9/QxRqhiP8sexM= 2022-09-03T05:37:23.791 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HtMD+5y+L/7uB9/QxRqhiP8sexM=', 'Content-Length': '0'} 2022-09-03T05:37:23.795 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000800363a9bcfcda58-006312e813-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.796 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.797 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.798 DEBUG:boto:Method: HEAD 2022-09-03T05:37:23.799 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:23.800 DEBUG:boto:Data: 2022-09-03T05:37:23.801 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.802 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:23.803 DEBUG:boto:Port: 8000 2022-09-03T05:37:23.804 DEBUG:boto:Params: {} 2022-09-03T05:37:23.805 DEBUG:boto:Token: None 2022-09-03T05:37:23.806 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj 2022-09-03T05:37:23.807 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HtMD+5y+L/7uB9/QxRqhiP8sexM= 2022-09-03T05:37:23.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HtMD+5y+L/7uB9/QxRqhiP8sexM=', 'Content-Length': '0'} 2022-09-03T05:37:23.812 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fa34c942d6434926-006312e813-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.813 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:23.813 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.814 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.815 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.815 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.816 DEBUG:boto:Method: GET 2022-09-03T05:37:23.816 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.817 DEBUG:boto:Data: 2022-09-03T05:37:23.817 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.818 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:23.818 DEBUG:boto:Port: 8001 2022-09-03T05:37:23.819 DEBUG:boto:Params: {} 2022-09-03T05:37:23.819 DEBUG:boto:Token: None 2022-09-03T05:37:23.820 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.821 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MXlwoaVi5g/MkiJrKV1CecGjRjg= 2022-09-03T05:37:23.821 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MXlwoaVi5g/MkiJrKV1CecGjRjg=', 'Content-Length': '0'} 2022-09-03T05:37:23.824 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ea35212fe35aea3f-006312e813-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.867 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.868 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.869 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.870 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.871 DEBUG:boto:Method: GET 2022-09-03T05:37:23.872 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.873 DEBUG:boto:Data: 2022-09-03T05:37:23.874 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.875 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:23.876 DEBUG:boto:Port: 8000 2022-09-03T05:37:23.876 DEBUG:boto:Params: {} 2022-09-03T05:37:23.877 DEBUG:boto:Token: None 2022-09-03T05:37:23.878 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:23.879 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MXlwoaVi5g/MkiJrKV1CecGjRjg= 2022-09-03T05:37:23.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MXlwoaVi5g/MkiJrKV1CecGjRjg=', 'Content-Length': '0'} 2022-09-03T05:37:23.881 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003a67a3206a6f8819-006312e813-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.883 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:23.884 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:23.884 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.885 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.885 DEBUG:boto:Method: GET 2022-09-03T05:37:23.886 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.886 DEBUG:boto:Data: 2022-09-03T05:37:23.887 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.888 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:23.888 DEBUG:boto:Port: 8001 2022-09-03T05:37:23.889 DEBUG:boto:Params: {} 2022-09-03T05:37:23.889 DEBUG:boto:Token: None 2022-09-03T05:37:23.890 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/ 2022-09-03T05:37:23.891 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40= 2022-09-03T05:37:23.891 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40=', 'Content-Length': '0'} 2022-09-03T05:37:23.895 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000095b33425797ea86f-006312e813-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.896 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:23.897 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:23.897 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:23.898 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.898 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.899 DEBUG:boto:Method: GET 2022-09-03T05:37:23.899 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:23.900 DEBUG:boto:Data: 2022-09-03T05:37:23.900 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.901 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:23.901 DEBUG:boto:Port: 8000 2022-09-03T05:37:23.902 DEBUG:boto:Params: {} 2022-09-03T05:37:23.903 DEBUG:boto:Token: None 2022-09-03T05:37:23.904 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/ 2022-09-03T05:37:23.904 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40= 2022-09-03T05:37:23.905 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v7ItdcLyZsJ9bV9CihZlWaMqr40=', 'Content-Length': '0'} 2022-09-03T05:37:23.912 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000998498dcba8fafe5-006312e813-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.913 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:23.913 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:23.914 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.915 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.915 DEBUG:boto:path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:23.916 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:23.916 DEBUG:boto:Method: GET 2022-09-03T05:37:23.917 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:23.917 DEBUG:boto:Data: 2022-09-03T05:37:23.918 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.919 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:23.919 DEBUG:boto:Port: 8001 2022-09-03T05:37:23.920 DEBUG:boto:Params: {} 2022-09-03T05:37:23.921 DEBUG:boto:Token: None 2022-09-03T05:37:23.922 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:23.924 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QHoVGGUKhrAqUFR52dbtz2ntGV0= 2022-09-03T05:37:23.924 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QHoVGGUKhrAqUFR52dbtz2ntGV0=', 'Content-Length': '0'} 2022-09-03T05:37:23.927 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002cbde06998b98c2f-006312e813-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.929 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.930 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.931 DEBUG:boto:path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:23.932 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:23.933 DEBUG:boto:Method: GET 2022-09-03T05:37:23.934 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:23.935 DEBUG:boto:Data: 2022-09-03T05:37:23.936 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.937 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:23.938 DEBUG:boto:Port: 8000 2022-09-03T05:37:23.938 DEBUG:boto:Params: {} 2022-09-03T05:37:23.940 DEBUG:boto:Token: None 2022-09-03T05:37:23.941 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:23.941 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QHoVGGUKhrAqUFR52dbtz2ntGV0= 2022-09-03T05:37:23.942 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QHoVGGUKhrAqUFR52dbtz2ntGV0=', 'Content-Length': '0'} 2022-09-03T05:37:23.944 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000066449d9aac7ed93-006312e813-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:23.986 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:23.987 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:23.990 DEBUG:boto:path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:23.990 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:23.991 DEBUG:boto:Method: HEAD 2022-09-03T05:37:23.991 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:23.992 DEBUG:boto:Data: 2022-09-03T05:37:23.992 DEBUG:boto:Headers: {} 2022-09-03T05:37:23.993 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:23.994 DEBUG:boto:Port: 8001 2022-09-03T05:37:23.995 DEBUG:boto:Params: {} 2022-09-03T05:37:23.996 DEBUG:boto:Token: None 2022-09-03T05:37:23.996 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:23 GMT /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:23.997 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5UK9S/kliNQXfF9/yYCmazUVSz4= 2022-09-03T05:37:23.998 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5UK9S/kliNQXfF9/yYCmazUVSz4=', 'Content-Length': '0'} 2022-09-03T05:37:24.000 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a8cb0c6db2105a7a-006312e813-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.001 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.001 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.002 DEBUG:boto:path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:24.003 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:24.004 DEBUG:boto:Method: HEAD 2022-09-03T05:37:24.004 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:24.005 DEBUG:boto:Data: 2022-09-03T05:37:24.006 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.006 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.007 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.008 DEBUG:boto:Params: {} 2022-09-03T05:37:24.009 DEBUG:boto:Token: None 2022-09-03T05:37:24.010 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:24.010 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kNqsqJ6Y+i0udGXwkMY98kqJnsg= 2022-09-03T05:37:24.011 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kNqsqJ6Y+i0udGXwkMY98kqJnsg=', 'Content-Length': '0'} 2022-09-03T05:37:24.013 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cc136d71eea25ac5-006312e814-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.014 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:24.015 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.015 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.016 DEBUG:boto:path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:24.017 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:24.017 DEBUG:boto:Method: GET 2022-09-03T05:37:24.018 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:24.019 DEBUG:boto:Data: 2022-09-03T05:37:24.020 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.020 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.021 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.021 DEBUG:boto:Params: {} 2022-09-03T05:37:24.022 DEBUG:boto:Token: None 2022-09-03T05:37:24.024 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:24.025 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MrlOCMNLKQSNqEB/2sDAowjBQuU= 2022-09-03T05:37:24.026 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MrlOCMNLKQSNqEB/2sDAowjBQuU=', 'Content-Length': '0'} 2022-09-03T05:37:24.029 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000039f301bb9a227315-006312e814-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.031 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.032 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.033 DEBUG:boto:path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:24.035 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:24.036 DEBUG:boto:Method: GET 2022-09-03T05:37:24.037 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:24.038 DEBUG:boto:Data: 2022-09-03T05:37:24.038 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.039 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.040 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.040 DEBUG:boto:Params: {} 2022-09-03T05:37:24.041 DEBUG:boto:Token: None 2022-09-03T05:37:24.042 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:24.042 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MrlOCMNLKQSNqEB/2sDAowjBQuU= 2022-09-03T05:37:24.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MrlOCMNLKQSNqEB/2sDAowjBQuU=', 'Content-Length': '0'} 2022-09-03T05:37:24.045 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f743fb544346b3b1-006312e814-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.047 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:24.048 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:24.049 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.050 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.051 DEBUG:boto:Method: GET 2022-09-03T05:37:24.052 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.053 DEBUG:boto:Data: 2022-09-03T05:37:24.054 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.055 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.055 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.056 DEBUG:boto:Params: {} 2022-09-03T05:37:24.057 DEBUG:boto:Token: None 2022-09-03T05:37:24.058 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/ 2022-09-03T05:37:24.059 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0= 2022-09-03T05:37:24.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0=', 'Content-Length': '0'} 2022-09-03T05:37:24.064 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ddaf8d72ea24301f-006312e814-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.065 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:24.066 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:24.067 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:24.067 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.068 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.068 DEBUG:boto:Method: GET 2022-09-03T05:37:24.069 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.071 DEBUG:boto:Data: 2022-09-03T05:37:24.072 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.073 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.074 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.075 DEBUG:boto:Params: {} 2022-09-03T05:37:24.076 DEBUG:boto:Token: None 2022-09-03T05:37:24.077 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/ 2022-09-03T05:37:24.078 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0= 2022-09-03T05:37:24.079 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0=', 'Content-Length': '0'} 2022-09-03T05:37:24.083 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000044d4f7196002856c-006312e814-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.084 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:24.085 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.086 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.087 DEBUG:boto:Method: HEAD 2022-09-03T05:37:24.088 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:24.089 DEBUG:boto:Data: 2022-09-03T05:37:24.090 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.090 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.091 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.092 DEBUG:boto:Params: {} 2022-09-03T05:37:24.093 DEBUG:boto:Token: None 2022-09-03T05:37:24.094 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj 2022-09-03T05:37:24.095 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zd0PtizSmexVr0kLqECxaDkBdWs= 2022-09-03T05:37:24.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zd0PtizSmexVr0kLqECxaDkBdWs=', 'Content-Length': '0'} 2022-09-03T05:37:24.101 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008f75379e735a8e02-006312e814-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.102 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.103 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.104 DEBUG:boto:Method: HEAD 2022-09-03T05:37:24.105 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:24.106 DEBUG:boto:Data: 2022-09-03T05:37:24.106 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.107 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.108 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.109 DEBUG:boto:Params: {} 2022-09-03T05:37:24.110 DEBUG:boto:Token: None 2022-09-03T05:37:24.111 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj 2022-09-03T05:37:24.112 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zd0PtizSmexVr0kLqECxaDkBdWs= 2022-09-03T05:37:24.113 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zd0PtizSmexVr0kLqECxaDkBdWs=', 'Content-Length': '0'} 2022-09-03T05:37:24.117 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f902f6d77f2f7259-006312e814-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.119 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:24.119 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.120 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.121 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.121 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.122 DEBUG:boto:Method: GET 2022-09-03T05:37:24.122 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.123 DEBUG:boto:Data: 2022-09-03T05:37:24.123 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.124 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.124 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.125 DEBUG:boto:Params: {} 2022-09-03T05:37:24.126 DEBUG:boto:Token: None 2022-09-03T05:37:24.127 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.128 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:okojhEHBNQnCsTVy+dWZUR07sOA= 2022-09-03T05:37:24.129 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:okojhEHBNQnCsTVy+dWZUR07sOA=', 'Content-Length': '0'} 2022-09-03T05:37:24.132 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bd9f5d81dced5d45-006312e814-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.174 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.175 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.176 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.177 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.178 DEBUG:boto:Method: GET 2022-09-03T05:37:24.179 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.179 DEBUG:boto:Data: 2022-09-03T05:37:24.180 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.181 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.181 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.182 DEBUG:boto:Params: {} 2022-09-03T05:37:24.183 DEBUG:boto:Token: None 2022-09-03T05:37:24.183 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.184 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:okojhEHBNQnCsTVy+dWZUR07sOA= 2022-09-03T05:37:24.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:okojhEHBNQnCsTVy+dWZUR07sOA=', 'Content-Length': '0'} 2022-09-03T05:37:24.187 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b526677d8411e9c7-006312e814-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.188 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:24.188 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:24.189 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.190 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.190 DEBUG:boto:Method: GET 2022-09-03T05:37:24.191 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.191 DEBUG:boto:Data: 2022-09-03T05:37:24.192 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.193 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.193 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.194 DEBUG:boto:Params: {} 2022-09-03T05:37:24.194 DEBUG:boto:Token: None 2022-09-03T05:37:24.195 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/ 2022-09-03T05:37:24.196 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0= 2022-09-03T05:37:24.197 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0=', 'Content-Length': '0'} 2022-09-03T05:37:24.200 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b8ab2efb7a8911cf-006312e814-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.200 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:24.201 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:24.202 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:24.203 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.203 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.204 DEBUG:boto:Method: GET 2022-09-03T05:37:24.204 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.205 DEBUG:boto:Data: 2022-09-03T05:37:24.205 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.206 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.206 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.207 DEBUG:boto:Params: {} 2022-09-03T05:37:24.208 DEBUG:boto:Token: None 2022-09-03T05:37:24.209 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/ 2022-09-03T05:37:24.209 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0= 2022-09-03T05:37:24.211 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0=', 'Content-Length': '0'} 2022-09-03T05:37:24.216 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002168aab8b79e6865-006312e814-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.258 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:24.259 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:24.260 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.262 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.263 DEBUG:boto:path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.263 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.264 DEBUG:boto:Method: GET 2022-09-03T05:37:24.266 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.267 DEBUG:boto:Data: 2022-09-03T05:37:24.270 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.270 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.271 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.271 DEBUG:boto:Params: {} 2022-09-03T05:37:24.272 DEBUG:boto:Token: None 2022-09-03T05:37:24.273 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.273 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:doTQUYv64FlrpQvzVzsrl80AJvg= 2022-09-03T05:37:24.274 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:doTQUYv64FlrpQvzVzsrl80AJvg=', 'Content-Length': '0'} 2022-09-03T05:37:24.278 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000466bb3b6f819e6ba-006312e814-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.280 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.281 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.282 DEBUG:boto:path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.282 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.283 DEBUG:boto:Method: GET 2022-09-03T05:37:24.284 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.285 DEBUG:boto:Data: 2022-09-03T05:37:24.285 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.286 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.287 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.287 DEBUG:boto:Params: {} 2022-09-03T05:37:24.288 DEBUG:boto:Token: None 2022-09-03T05:37:24.289 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.290 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:doTQUYv64FlrpQvzVzsrl80AJvg= 2022-09-03T05:37:24.290 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:doTQUYv64FlrpQvzVzsrl80AJvg=', 'Content-Length': '0'} 2022-09-03T05:37:24.293 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b7dbd2df22761ce3-006312e814-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.295 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.296 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.297 DEBUG:boto:path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.298 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.299 DEBUG:boto:Method: HEAD 2022-09-03T05:37:24.300 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.301 DEBUG:boto:Data: 2022-09-03T05:37:24.302 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.303 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.304 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.305 DEBUG:boto:Params: {} 2022-09-03T05:37:24.306 DEBUG:boto:Token: None 2022-09-03T05:37:24.307 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.308 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ro22TlsZBAG/2bGBe25wHlt7c6k= 2022-09-03T05:37:24.311 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ro22TlsZBAG/2bGBe25wHlt7c6k=', 'Content-Length': '0'} 2022-09-03T05:37:24.313 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007af5a1143c6fd3cb-006312e814-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.314 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.315 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.316 DEBUG:boto:path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.316 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.317 DEBUG:boto:Method: HEAD 2022-09-03T05:37:24.317 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.318 DEBUG:boto:Data: 2022-09-03T05:37:24.318 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.319 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.320 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.321 DEBUG:boto:Params: {} 2022-09-03T05:37:24.322 DEBUG:boto:Token: None 2022-09-03T05:37:24.323 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.324 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ro22TlsZBAG/2bGBe25wHlt7c6k= 2022-09-03T05:37:24.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ro22TlsZBAG/2bGBe25wHlt7c6k=', 'Content-Length': '0'} 2022-09-03T05:37:24.326 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d2862143fdced216-006312e814-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.327 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:24.328 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.328 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.329 DEBUG:boto:path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.330 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.331 DEBUG:boto:Method: GET 2022-09-03T05:37:24.332 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.332 DEBUG:boto:Data: 2022-09-03T05:37:24.333 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.334 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.335 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.336 DEBUG:boto:Params: {} 2022-09-03T05:37:24.337 DEBUG:boto:Token: None 2022-09-03T05:37:24.338 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.340 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:doTQUYv64FlrpQvzVzsrl80AJvg= 2022-09-03T05:37:24.341 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:doTQUYv64FlrpQvzVzsrl80AJvg=', 'Content-Length': '0'} 2022-09-03T05:37:24.344 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002e2aaa6ead8c8217-006312e814-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.345 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.346 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.347 DEBUG:boto:path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.348 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.349 DEBUG:boto:Method: GET 2022-09-03T05:37:24.350 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.351 DEBUG:boto:Data: 2022-09-03T05:37:24.352 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.353 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.354 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.355 DEBUG:boto:Params: {} 2022-09-03T05:37:24.355 DEBUG:boto:Token: None 2022-09-03T05:37:24.356 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:24.357 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:doTQUYv64FlrpQvzVzsrl80AJvg= 2022-09-03T05:37:24.358 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:doTQUYv64FlrpQvzVzsrl80AJvg=', 'Content-Length': '0'} 2022-09-03T05:37:24.360 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000089fdcec3442de673-006312e814-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.362 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:24.363 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:24.364 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.364 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.365 DEBUG:boto:Method: GET 2022-09-03T05:37:24.365 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.366 DEBUG:boto:Data: 2022-09-03T05:37:24.367 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.368 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.369 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.370 DEBUG:boto:Params: {} 2022-09-03T05:37:24.371 DEBUG:boto:Token: None 2022-09-03T05:37:24.373 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/ 2022-09-03T05:37:24.374 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0= 2022-09-03T05:37:24.375 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0=', 'Content-Length': '0'} 2022-09-03T05:37:24.385 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001ecaa4a718d5b027-006312e814-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.386 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:24.387 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:24.387 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:24.388 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.389 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.389 DEBUG:boto:Method: GET 2022-09-03T05:37:24.390 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.390 DEBUG:boto:Data: 2022-09-03T05:37:24.391 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.392 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.392 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.393 DEBUG:boto:Params: {} 2022-09-03T05:37:24.393 DEBUG:boto:Token: None 2022-09-03T05:37:24.395 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/ 2022-09-03T05:37:24.395 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0= 2022-09-03T05:37:24.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0=', 'Content-Length': '0'} 2022-09-03T05:37:24.399 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005ce216be0aff32ca-006312e814-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.400 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:24.401 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.401 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.402 DEBUG:boto:Method: HEAD 2022-09-03T05:37:24.402 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:24.403 DEBUG:boto:Data: 2022-09-03T05:37:24.404 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.404 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.405 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.405 DEBUG:boto:Params: {} 2022-09-03T05:37:24.406 DEBUG:boto:Token: None 2022-09-03T05:37:24.406 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj 2022-09-03T05:37:24.407 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zd0PtizSmexVr0kLqECxaDkBdWs= 2022-09-03T05:37:24.408 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zd0PtizSmexVr0kLqECxaDkBdWs=', 'Content-Length': '0'} 2022-09-03T05:37:24.412 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009363fc02c4dec498-006312e814-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.412 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.413 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.413 DEBUG:boto:Method: HEAD 2022-09-03T05:37:24.414 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:24.415 DEBUG:boto:Data: 2022-09-03T05:37:24.415 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.416 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.417 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.417 DEBUG:boto:Params: {} 2022-09-03T05:37:24.418 DEBUG:boto:Token: None 2022-09-03T05:37:24.419 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj 2022-09-03T05:37:24.419 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zd0PtizSmexVr0kLqECxaDkBdWs= 2022-09-03T05:37:24.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zd0PtizSmexVr0kLqECxaDkBdWs=', 'Content-Length': '0'} 2022-09-03T05:37:24.423 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000083bbf66c4370d1e2-006312e814-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.424 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:24.424 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.425 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.426 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.427 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.428 DEBUG:boto:Method: GET 2022-09-03T05:37:24.429 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.430 DEBUG:boto:Data: 2022-09-03T05:37:24.431 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.432 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.433 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.435 DEBUG:boto:Params: {} 2022-09-03T05:37:24.436 DEBUG:boto:Token: None 2022-09-03T05:37:24.437 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.439 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:okojhEHBNQnCsTVy+dWZUR07sOA= 2022-09-03T05:37:24.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:okojhEHBNQnCsTVy+dWZUR07sOA=', 'Content-Length': '0'} 2022-09-03T05:37:24.442 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dacbce0289584523-006312e814-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.486 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.488 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.489 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.489 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.491 DEBUG:boto:Method: GET 2022-09-03T05:37:24.491 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.492 DEBUG:boto:Data: 2022-09-03T05:37:24.493 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.493 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.494 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.494 DEBUG:boto:Params: {} 2022-09-03T05:37:24.495 DEBUG:boto:Token: None 2022-09-03T05:37:24.496 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.496 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:okojhEHBNQnCsTVy+dWZUR07sOA= 2022-09-03T05:37:24.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:okojhEHBNQnCsTVy+dWZUR07sOA=', 'Content-Length': '0'} 2022-09-03T05:37:24.499 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000e3d676109a6d54b-006312e814-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.500 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:24.501 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:24.502 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.502 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.503 DEBUG:boto:Method: GET 2022-09-03T05:37:24.503 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.504 DEBUG:boto:Data: 2022-09-03T05:37:24.504 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.505 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.505 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.506 DEBUG:boto:Params: {} 2022-09-03T05:37:24.507 DEBUG:boto:Token: None 2022-09-03T05:37:24.508 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/ 2022-09-03T05:37:24.508 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0= 2022-09-03T05:37:24.509 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0=', 'Content-Length': '0'} 2022-09-03T05:37:24.512 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000083e2cfb12fd9eefc-006312e814-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.512 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:24.513 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:24.514 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:24.515 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.515 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.516 DEBUG:boto:Method: GET 2022-09-03T05:37:24.516 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.518 DEBUG:boto:Data: 2022-09-03T05:37:24.519 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.520 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.521 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.522 DEBUG:boto:Params: {} 2022-09-03T05:37:24.524 DEBUG:boto:Token: None 2022-09-03T05:37:24.525 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/ 2022-09-03T05:37:24.527 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0= 2022-09-03T05:37:24.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0=', 'Content-Length': '0'} 2022-09-03T05:37:24.531 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002c4557afc7ead7ef-006312e814-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.532 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:24.534 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:24.535 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.535 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.536 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.536 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.537 DEBUG:boto:Method: GET 2022-09-03T05:37:24.538 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.539 DEBUG:boto:Data: 2022-09-03T05:37:24.540 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.541 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.542 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.543 DEBUG:boto:Params: {} 2022-09-03T05:37:24.543 DEBUG:boto:Token: None 2022-09-03T05:37:24.544 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.545 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RHA17BsmgG6i3gc83aH7LV/wOpM= 2022-09-03T05:37:24.545 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RHA17BsmgG6i3gc83aH7LV/wOpM=', 'Content-Length': '0'} 2022-09-03T05:37:24.553 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a705f557d4476a4d-006312e814-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.554 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.554 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.555 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.555 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.556 DEBUG:boto:Method: GET 2022-09-03T05:37:24.556 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.557 DEBUG:boto:Data: 2022-09-03T05:37:24.557 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.558 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.558 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.559 DEBUG:boto:Params: {} 2022-09-03T05:37:24.560 DEBUG:boto:Token: None 2022-09-03T05:37:24.560 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.561 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RHA17BsmgG6i3gc83aH7LV/wOpM= 2022-09-03T05:37:24.562 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RHA17BsmgG6i3gc83aH7LV/wOpM=', 'Content-Length': '0'} 2022-09-03T05:37:24.564 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000050ed169e371cbd46-006312e814-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.565 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.566 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.566 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.567 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.567 DEBUG:boto:Method: HEAD 2022-09-03T05:37:24.568 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.568 DEBUG:boto:Data: 2022-09-03T05:37:24.569 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.569 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.569 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.570 DEBUG:boto:Params: {} 2022-09-03T05:37:24.571 DEBUG:boto:Token: None 2022-09-03T05:37:24.571 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.572 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4GhDwrCHCJDSa35JnN91lFI2cNI= 2022-09-03T05:37:24.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4GhDwrCHCJDSa35JnN91lFI2cNI=', 'Content-Length': '0'} 2022-09-03T05:37:24.574 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000388e808944fb13b-006312e814-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.575 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.576 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.576 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.577 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.577 DEBUG:boto:Method: HEAD 2022-09-03T05:37:24.578 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.578 DEBUG:boto:Data: 2022-09-03T05:37:24.579 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.579 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.580 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.580 DEBUG:boto:Params: {} 2022-09-03T05:37:24.581 DEBUG:boto:Token: None 2022-09-03T05:37:24.581 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.582 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4GhDwrCHCJDSa35JnN91lFI2cNI= 2022-09-03T05:37:24.582 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4GhDwrCHCJDSa35JnN91lFI2cNI=', 'Content-Length': '0'} 2022-09-03T05:37:24.584 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007b12ba0ea8a13bd2-006312e814-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.585 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:24.586 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.586 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.587 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.587 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.588 DEBUG:boto:Method: GET 2022-09-03T05:37:24.588 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.589 DEBUG:boto:Data: 2022-09-03T05:37:24.590 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.590 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.591 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.591 DEBUG:boto:Params: {} 2022-09-03T05:37:24.592 DEBUG:boto:Token: None 2022-09-03T05:37:24.592 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.593 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RHA17BsmgG6i3gc83aH7LV/wOpM= 2022-09-03T05:37:24.594 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RHA17BsmgG6i3gc83aH7LV/wOpM=', 'Content-Length': '0'} 2022-09-03T05:37:24.596 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004c9286bf9d5046fe-006312e814-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.638 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.639 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.639 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.640 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.641 DEBUG:boto:Method: GET 2022-09-03T05:37:24.641 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.643 DEBUG:boto:Data: 2022-09-03T05:37:24.644 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.645 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.645 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.646 DEBUG:boto:Params: {} 2022-09-03T05:37:24.647 DEBUG:boto:Token: None 2022-09-03T05:37:24.648 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:24.650 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RHA17BsmgG6i3gc83aH7LV/wOpM= 2022-09-03T05:37:24.651 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RHA17BsmgG6i3gc83aH7LV/wOpM=', 'Content-Length': '0'} 2022-09-03T05:37:24.655 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b468c311421d1dcd-006312e814-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.657 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:24.658 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:24.659 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.660 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.661 DEBUG:boto:Method: GET 2022-09-03T05:37:24.662 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.663 DEBUG:boto:Data: 2022-09-03T05:37:24.665 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.665 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.666 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.666 DEBUG:boto:Params: {} 2022-09-03T05:37:24.667 DEBUG:boto:Token: None 2022-09-03T05:37:24.668 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/ 2022-09-03T05:37:24.668 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0= 2022-09-03T05:37:24.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0=', 'Content-Length': '0'} 2022-09-03T05:37:24.672 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000050469a1cd546bc52-006312e814-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.672 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:24.673 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:24.674 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:24.674 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.675 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.676 DEBUG:boto:Method: GET 2022-09-03T05:37:24.677 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.678 DEBUG:boto:Data: 2022-09-03T05:37:24.678 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.679 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.680 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.681 DEBUG:boto:Params: {} 2022-09-03T05:37:24.683 DEBUG:boto:Token: None 2022-09-03T05:37:24.684 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/ 2022-09-03T05:37:24.685 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0= 2022-09-03T05:37:24.686 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0=', 'Content-Length': '0'} 2022-09-03T05:37:24.690 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000038fd794cae583503-006312e814-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.691 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:24.692 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.692 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.693 DEBUG:boto:Method: HEAD 2022-09-03T05:37:24.693 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:24.694 DEBUG:boto:Data: 2022-09-03T05:37:24.694 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.695 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.696 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.697 DEBUG:boto:Params: {} 2022-09-03T05:37:24.698 DEBUG:boto:Token: None 2022-09-03T05:37:24.700 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj 2022-09-03T05:37:24.701 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zd0PtizSmexVr0kLqECxaDkBdWs= 2022-09-03T05:37:24.702 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zd0PtizSmexVr0kLqECxaDkBdWs=', 'Content-Length': '0'} 2022-09-03T05:37:24.705 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bc8d58ae553fbc89-006312e814-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.706 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.707 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.707 DEBUG:boto:Method: HEAD 2022-09-03T05:37:24.708 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:24.708 DEBUG:boto:Data: 2022-09-03T05:37:24.709 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.709 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.710 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.710 DEBUG:boto:Params: {} 2022-09-03T05:37:24.711 DEBUG:boto:Token: None 2022-09-03T05:37:24.712 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj 2022-09-03T05:37:24.712 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zd0PtizSmexVr0kLqECxaDkBdWs= 2022-09-03T05:37:24.713 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zd0PtizSmexVr0kLqECxaDkBdWs=', 'Content-Length': '0'} 2022-09-03T05:37:24.717 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000efb2398c7cfb501d-006312e814-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.718 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:24.718 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.719 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.720 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.720 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.721 DEBUG:boto:Method: GET 2022-09-03T05:37:24.722 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.722 DEBUG:boto:Data: 2022-09-03T05:37:24.723 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.723 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.725 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.726 DEBUG:boto:Params: {} 2022-09-03T05:37:24.726 DEBUG:boto:Token: None 2022-09-03T05:37:24.727 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.728 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:okojhEHBNQnCsTVy+dWZUR07sOA= 2022-09-03T05:37:24.728 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:okojhEHBNQnCsTVy+dWZUR07sOA=', 'Content-Length': '0'} 2022-09-03T05:37:24.731 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000787941fe5fd412e2-006312e814-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.732 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.733 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.734 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.735 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.736 DEBUG:boto:Method: GET 2022-09-03T05:37:24.737 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.737 DEBUG:boto:Data: 2022-09-03T05:37:24.738 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.738 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.739 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.739 DEBUG:boto:Params: {} 2022-09-03T05:37:24.740 DEBUG:boto:Token: None 2022-09-03T05:37:24.740 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:24.741 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:okojhEHBNQnCsTVy+dWZUR07sOA= 2022-09-03T05:37:24.742 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:okojhEHBNQnCsTVy+dWZUR07sOA=', 'Content-Length': '0'} 2022-09-03T05:37:24.745 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006ae2ef900631a581-006312e814-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.786 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:24.787 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:24.787 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.788 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.788 DEBUG:boto:Method: GET 2022-09-03T05:37:24.789 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.790 DEBUG:boto:Data: 2022-09-03T05:37:24.790 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.791 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.791 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.792 DEBUG:boto:Params: {} 2022-09-03T05:37:24.792 DEBUG:boto:Token: None 2022-09-03T05:37:24.793 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/ 2022-09-03T05:37:24.795 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0= 2022-09-03T05:37:24.795 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0=', 'Content-Length': '0'} 2022-09-03T05:37:24.798 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000088c0480f6c771acb-006312e814-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.800 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:24.801 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:24.802 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:24.803 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.804 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.805 DEBUG:boto:Method: GET 2022-09-03T05:37:24.806 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.807 DEBUG:boto:Data: 2022-09-03T05:37:24.808 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.808 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.809 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.809 DEBUG:boto:Params: {} 2022-09-03T05:37:24.810 DEBUG:boto:Token: None 2022-09-03T05:37:24.811 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/ 2022-09-03T05:37:24.811 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0= 2022-09-03T05:37:24.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0=', 'Content-Length': '0'} 2022-09-03T05:37:24.815 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c9dfc0cbda597e87-006312e814-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.817 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:24.817 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:24.818 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.819 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.819 DEBUG:boto:path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.820 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.820 DEBUG:boto:Method: GET 2022-09-03T05:37:24.821 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.821 DEBUG:boto:Data: 2022-09-03T05:37:24.822 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.823 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.823 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.824 DEBUG:boto:Params: {} 2022-09-03T05:37:24.824 DEBUG:boto:Token: None 2022-09-03T05:37:24.825 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.826 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G/M72RNHLbapTs7PPpA++DkETX4= 2022-09-03T05:37:24.827 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G/M72RNHLbapTs7PPpA++DkETX4=', 'Content-Length': '0'} 2022-09-03T05:37:24.829 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'vauOlKlWVHmDxq4ZzMBF97KP66F89Pc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009f7361022200c152-006312e814-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.830 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.831 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.831 DEBUG:boto:path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.832 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.832 DEBUG:boto:Method: GET 2022-09-03T05:37:24.833 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.833 DEBUG:boto:Data: 2022-09-03T05:37:24.834 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.835 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.835 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.836 DEBUG:boto:Params: {} 2022-09-03T05:37:24.836 DEBUG:boto:Token: None 2022-09-03T05:37:24.837 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.838 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G/M72RNHLbapTs7PPpA++DkETX4= 2022-09-03T05:37:24.839 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G/M72RNHLbapTs7PPpA++DkETX4=', 'Content-Length': '0'} 2022-09-03T05:37:24.841 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'vauOlKlWVHmDxq4ZzMBF97KP66F89Pc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000db774ce37cd815dd-006312e814-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.842 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.843 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.843 DEBUG:boto:path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.844 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.844 DEBUG:boto:Method: HEAD 2022-09-03T05:37:24.845 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.845 DEBUG:boto:Data: 2022-09-03T05:37:24.846 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.846 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.849 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.849 DEBUG:boto:Params: {} 2022-09-03T05:37:24.850 DEBUG:boto:Token: None 2022-09-03T05:37:24.851 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.853 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ileh7v3oJbssJvjmK/JG0d16NZ0= 2022-09-03T05:37:24.853 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ileh7v3oJbssJvjmK/JG0d16NZ0=', 'Content-Length': '0'} 2022-09-03T05:37:24.856 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'vauOlKlWVHmDxq4ZzMBF97KP66F89Pc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001f9b396c114180ef-006312e814-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.857 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.858 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.859 DEBUG:boto:path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.860 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.861 DEBUG:boto:Method: HEAD 2022-09-03T05:37:24.862 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.863 DEBUG:boto:Data: 2022-09-03T05:37:24.863 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.864 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.865 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.866 DEBUG:boto:Params: {} 2022-09-03T05:37:24.867 DEBUG:boto:Token: None 2022-09-03T05:37:24.868 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.869 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ileh7v3oJbssJvjmK/JG0d16NZ0= 2022-09-03T05:37:24.870 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ileh7v3oJbssJvjmK/JG0d16NZ0=', 'Content-Length': '0'} 2022-09-03T05:37:24.873 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'vauOlKlWVHmDxq4ZzMBF97KP66F89Pc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007783a1cfe7dbf8f5-006312e814-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.874 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:24.875 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.877 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.878 DEBUG:boto:path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.879 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.880 DEBUG:boto:Method: GET 2022-09-03T05:37:24.881 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.882 DEBUG:boto:Data: 2022-09-03T05:37:24.883 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.884 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.885 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.886 DEBUG:boto:Params: {} 2022-09-03T05:37:24.887 DEBUG:boto:Token: None 2022-09-03T05:37:24.888 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.889 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G/M72RNHLbapTs7PPpA++DkETX4= 2022-09-03T05:37:24.891 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G/M72RNHLbapTs7PPpA++DkETX4=', 'Content-Length': '0'} 2022-09-03T05:37:24.894 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'vauOlKlWVHmDxq4ZzMBF97KP66F89Pc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f39370485b175a70-006312e814-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.896 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.897 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.897 DEBUG:boto:path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.898 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.899 DEBUG:boto:Method: GET 2022-09-03T05:37:24.900 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.901 DEBUG:boto:Data: 2022-09-03T05:37:24.902 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.903 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.904 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.905 DEBUG:boto:Params: {} 2022-09-03T05:37:24.906 DEBUG:boto:Token: None 2022-09-03T05:37:24.908 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:24.909 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G/M72RNHLbapTs7PPpA++DkETX4= 2022-09-03T05:37:24.910 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G/M72RNHLbapTs7PPpA++DkETX4=', 'Content-Length': '0'} 2022-09-03T05:37:24.914 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'vauOlKlWVHmDxq4ZzMBF97KP66F89Pc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006d38ad891330fce0-006312e814-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.915 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:24.916 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:24.918 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.918 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.919 DEBUG:boto:Method: GET 2022-09-03T05:37:24.919 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.920 DEBUG:boto:Data: 2022-09-03T05:37:24.920 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.921 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.921 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.922 DEBUG:boto:Params: {} 2022-09-03T05:37:24.922 DEBUG:boto:Token: None 2022-09-03T05:37:24.923 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/ 2022-09-03T05:37:24.924 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0= 2022-09-03T05:37:24.925 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0=', 'Content-Length': '0'} 2022-09-03T05:37:24.928 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004522fb21e2a009b1-006312e814-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.970 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:24.972 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:24.972 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:24.973 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.974 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.974 DEBUG:boto:Method: GET 2022-09-03T05:37:24.975 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:24.975 DEBUG:boto:Data: 2022-09-03T05:37:24.976 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.976 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:24.977 DEBUG:boto:Port: 8000 2022-09-03T05:37:24.978 DEBUG:boto:Params: {} 2022-09-03T05:37:24.979 DEBUG:boto:Token: None 2022-09-03T05:37:24.980 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/ 2022-09-03T05:37:24.981 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0= 2022-09-03T05:37:24.982 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xmndqFiNn97EHlFhDBNuVfOtvb0=', 'Content-Length': '0'} 2022-09-03T05:37:24.986 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b463b55c2aa346d0-006312e814-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:24.987 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:24.988 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:24.989 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:24.989 DEBUG:boto:Method: HEAD 2022-09-03T05:37:24.990 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:24.991 DEBUG:boto:Data: 2022-09-03T05:37:24.991 DEBUG:boto:Headers: {} 2022-09-03T05:37:24.992 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:24.993 DEBUG:boto:Port: 8001 2022-09-03T05:37:24.993 DEBUG:boto:Params: {} 2022-09-03T05:37:24.994 DEBUG:boto:Token: None 2022-09-03T05:37:24.995 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:24 GMT /dsgvcy-19/obj 2022-09-03T05:37:24.997 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zd0PtizSmexVr0kLqECxaDkBdWs= 2022-09-03T05:37:24.998 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zd0PtizSmexVr0kLqECxaDkBdWs=', 'Content-Length': '0'} 2022-09-03T05:37:25.002 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000063d10485cfbd4db7-006312e815-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.003 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.004 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.005 DEBUG:boto:Method: HEAD 2022-09-03T05:37:25.005 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:25.005 DEBUG:boto:Data: 2022-09-03T05:37:25.006 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.009 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:25.009 DEBUG:boto:Port: 8000 2022-09-03T05:37:25.010 DEBUG:boto:Params: {} 2022-09-03T05:37:25.011 DEBUG:boto:Token: None 2022-09-03T05:37:25.011 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj 2022-09-03T05:37:25.012 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5IFWKkxEO7nGaY8aewaXmaI4X3s= 2022-09-03T05:37:25.012 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5IFWKkxEO7nGaY8aewaXmaI4X3s=', 'Content-Length': '0'} 2022-09-03T05:37:25.018 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000456defa0d367b3b4-006312e815-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.018 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:25.019 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.020 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.020 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.021 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.021 DEBUG:boto:Method: GET 2022-09-03T05:37:25.022 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.022 DEBUG:boto:Data: 2022-09-03T05:37:25.023 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.024 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:25.024 DEBUG:boto:Port: 8001 2022-09-03T05:37:25.025 DEBUG:boto:Params: {} 2022-09-03T05:37:25.026 DEBUG:boto:Token: None 2022-09-03T05:37:25.027 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.028 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LbrSPdOAUya8XjxNvWub+OTuv9M= 2022-09-03T05:37:25.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LbrSPdOAUya8XjxNvWub+OTuv9M=', 'Content-Length': '0'} 2022-09-03T05:37:25.033 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006610d3009cd7f094-006312e815-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.034 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.035 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.036 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.037 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.038 DEBUG:boto:Method: GET 2022-09-03T05:37:25.039 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.040 DEBUG:boto:Data: 2022-09-03T05:37:25.040 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.041 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:25.041 DEBUG:boto:Port: 8000 2022-09-03T05:37:25.042 DEBUG:boto:Params: {} 2022-09-03T05:37:25.043 DEBUG:boto:Token: None 2022-09-03T05:37:25.043 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.044 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LbrSPdOAUya8XjxNvWub+OTuv9M= 2022-09-03T05:37:25.045 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LbrSPdOAUya8XjxNvWub+OTuv9M=', 'Content-Length': '0'} 2022-09-03T05:37:25.052 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004c92f116d3d9caca-006312e815-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.094 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:25.095 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:25.097 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.098 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.099 DEBUG:boto:Method: GET 2022-09-03T05:37:25.099 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.100 DEBUG:boto:Data: 2022-09-03T05:37:25.101 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.102 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:25.104 DEBUG:boto:Port: 8001 2022-09-03T05:37:25.105 DEBUG:boto:Params: {} 2022-09-03T05:37:25.106 DEBUG:boto:Token: None 2022-09-03T05:37:25.107 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/ 2022-09-03T05:37:25.108 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE= 2022-09-03T05:37:25.108 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE=', 'Content-Length': '0'} 2022-09-03T05:37:25.112 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000011f4deb5febfd6ad-006312e815-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.115 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:25.120 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:25.121 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:25.129 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.131 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.132 DEBUG:boto:Method: GET 2022-09-03T05:37:25.133 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.134 DEBUG:boto:Data: 2022-09-03T05:37:25.135 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.136 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:25.137 DEBUG:boto:Port: 8000 2022-09-03T05:37:25.138 DEBUG:boto:Params: {} 2022-09-03T05:37:25.139 DEBUG:boto:Token: None 2022-09-03T05:37:25.140 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/ 2022-09-03T05:37:25.141 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE= 2022-09-03T05:37:25.143 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE=', 'Content-Length': '0'} 2022-09-03T05:37:25.146 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c1312285fce1e363-006312e815-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.147 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:25.148 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:25.149 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.150 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.151 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.152 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.153 DEBUG:boto:Method: GET 2022-09-03T05:37:25.154 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.155 DEBUG:boto:Data: 2022-09-03T05:37:25.156 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.159 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:25.161 DEBUG:boto:Port: 8001 2022-09-03T05:37:25.162 DEBUG:boto:Params: {} 2022-09-03T05:37:25.163 DEBUG:boto:Token: None 2022-09-03T05:37:25.164 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.165 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/ao/9ayt/LrEvnexz8Zbzkiha7E= 2022-09-03T05:37:25.167 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/ao/9ayt/LrEvnexz8Zbzkiha7E=', 'Content-Length': '0'} 2022-09-03T05:37:25.169 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000661ca182aaee2834-006312e815-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.171 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.171 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.172 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.172 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.174 DEBUG:boto:Method: GET 2022-09-03T05:37:25.175 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.176 DEBUG:boto:Data: 2022-09-03T05:37:25.177 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.178 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:25.179 DEBUG:boto:Port: 8000 2022-09-03T05:37:25.180 DEBUG:boto:Params: {} 2022-09-03T05:37:25.181 DEBUG:boto:Token: None 2022-09-03T05:37:25.183 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.184 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/ao/9ayt/LrEvnexz8Zbzkiha7E= 2022-09-03T05:37:25.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/ao/9ayt/LrEvnexz8Zbzkiha7E=', 'Content-Length': '0'} 2022-09-03T05:37:25.187 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000f936ec0f24b186a-006312e815-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.188 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.189 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.190 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.191 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.192 DEBUG:boto:Method: HEAD 2022-09-03T05:37:25.193 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.194 DEBUG:boto:Data: 2022-09-03T05:37:25.194 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.195 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:25.195 DEBUG:boto:Port: 8001 2022-09-03T05:37:25.196 DEBUG:boto:Params: {} 2022-09-03T05:37:25.197 DEBUG:boto:Token: None 2022-09-03T05:37:25.198 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.199 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TbD26q0IZafStOmY3LFdfQQI+eg= 2022-09-03T05:37:25.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TbD26q0IZafStOmY3LFdfQQI+eg=', 'Content-Length': '0'} 2022-09-03T05:37:25.202 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009ab02ef8def96930-006312e815-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.203 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.204 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.205 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.206 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.207 DEBUG:boto:Method: HEAD 2022-09-03T05:37:25.208 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.208 DEBUG:boto:Data: 2022-09-03T05:37:25.209 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.210 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:25.211 DEBUG:boto:Port: 8000 2022-09-03T05:37:25.212 DEBUG:boto:Params: {} 2022-09-03T05:37:25.213 DEBUG:boto:Token: None 2022-09-03T05:37:25.214 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.215 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TbD26q0IZafStOmY3LFdfQQI+eg= 2022-09-03T05:37:25.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TbD26q0IZafStOmY3LFdfQQI+eg=', 'Content-Length': '0'} 2022-09-03T05:37:25.222 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009b085ece18f17dc2-006312e815-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.223 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:25.223 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.224 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.225 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.225 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.226 DEBUG:boto:Method: GET 2022-09-03T05:37:25.226 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.227 DEBUG:boto:Data: 2022-09-03T05:37:25.227 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.228 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:25.229 DEBUG:boto:Port: 8001 2022-09-03T05:37:25.229 DEBUG:boto:Params: {} 2022-09-03T05:37:25.230 DEBUG:boto:Token: None 2022-09-03T05:37:25.230 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.231 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/ao/9ayt/LrEvnexz8Zbzkiha7E= 2022-09-03T05:37:25.233 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/ao/9ayt/LrEvnexz8Zbzkiha7E=', 'Content-Length': '0'} 2022-09-03T05:37:25.235 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000045d6591e5f916ae6-006312e815-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.236 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.236 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.237 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.238 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.238 DEBUG:boto:Method: GET 2022-09-03T05:37:25.239 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.239 DEBUG:boto:Data: 2022-09-03T05:37:25.240 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.240 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:25.241 DEBUG:boto:Port: 8000 2022-09-03T05:37:25.242 DEBUG:boto:Params: {} 2022-09-03T05:37:25.243 DEBUG:boto:Token: None 2022-09-03T05:37:25.243 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:25.244 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/ao/9ayt/LrEvnexz8Zbzkiha7E= 2022-09-03T05:37:25.245 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/ao/9ayt/LrEvnexz8Zbzkiha7E=', 'Content-Length': '0'} 2022-09-03T05:37:25.247 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e0fe572bb0242dfe-006312e815-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.248 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:25.249 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:25.249 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.250 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.251 DEBUG:boto:Method: GET 2022-09-03T05:37:25.251 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.252 DEBUG:boto:Data: 2022-09-03T05:37:25.252 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.253 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:25.254 DEBUG:boto:Port: 8001 2022-09-03T05:37:25.254 DEBUG:boto:Params: {} 2022-09-03T05:37:25.255 DEBUG:boto:Token: None 2022-09-03T05:37:25.256 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/ 2022-09-03T05:37:25.256 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE= 2022-09-03T05:37:25.257 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE=', 'Content-Length': '0'} 2022-09-03T05:37:25.260 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000843a541791e6a877-006312e815-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.302 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:25.303 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:25.304 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:25.304 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.305 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.305 DEBUG:boto:Method: GET 2022-09-03T05:37:25.306 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.307 DEBUG:boto:Data: 2022-09-03T05:37:25.307 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.308 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:25.309 DEBUG:boto:Port: 8000 2022-09-03T05:37:25.309 DEBUG:boto:Params: {} 2022-09-03T05:37:25.310 DEBUG:boto:Token: None 2022-09-03T05:37:25.311 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/ 2022-09-03T05:37:25.311 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE= 2022-09-03T05:37:25.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE=', 'Content-Length': '0'} 2022-09-03T05:37:25.315 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b0bf0ce0dad39e86-006312e815-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.316 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:25.317 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.318 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.319 DEBUG:boto:Method: HEAD 2022-09-03T05:37:25.319 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:25.320 DEBUG:boto:Data: 2022-09-03T05:37:25.320 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.321 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:25.321 DEBUG:boto:Port: 8001 2022-09-03T05:37:25.322 DEBUG:boto:Params: {} 2022-09-03T05:37:25.322 DEBUG:boto:Token: None 2022-09-03T05:37:25.323 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj 2022-09-03T05:37:25.324 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5IFWKkxEO7nGaY8aewaXmaI4X3s= 2022-09-03T05:37:25.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5IFWKkxEO7nGaY8aewaXmaI4X3s=', 'Content-Length': '0'} 2022-09-03T05:37:25.328 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000042ec30f23555546-006312e815-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.329 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.330 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.331 DEBUG:boto:Method: HEAD 2022-09-03T05:37:25.331 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:25.332 DEBUG:boto:Data: 2022-09-03T05:37:25.332 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.333 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:25.333 DEBUG:boto:Port: 8000 2022-09-03T05:37:25.334 DEBUG:boto:Params: {} 2022-09-03T05:37:25.334 DEBUG:boto:Token: None 2022-09-03T05:37:25.335 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj 2022-09-03T05:37:25.336 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5IFWKkxEO7nGaY8aewaXmaI4X3s= 2022-09-03T05:37:25.336 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5IFWKkxEO7nGaY8aewaXmaI4X3s=', 'Content-Length': '0'} 2022-09-03T05:37:25.349 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fed675cb684906db-006312e815-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.350 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:25.350 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.351 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.351 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.352 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.352 DEBUG:boto:Method: GET 2022-09-03T05:37:25.353 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.353 DEBUG:boto:Data: 2022-09-03T05:37:25.354 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.355 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:25.356 DEBUG:boto:Port: 8001 2022-09-03T05:37:25.357 DEBUG:boto:Params: {} 2022-09-03T05:37:25.359 DEBUG:boto:Token: None 2022-09-03T05:37:25.360 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.361 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LbrSPdOAUya8XjxNvWub+OTuv9M= 2022-09-03T05:37:25.363 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LbrSPdOAUya8XjxNvWub+OTuv9M=', 'Content-Length': '0'} 2022-09-03T05:37:25.367 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008775c9e26e5d9976-006312e815-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.368 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.369 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.369 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.370 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.370 DEBUG:boto:Method: GET 2022-09-03T05:37:25.371 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.371 DEBUG:boto:Data: 2022-09-03T05:37:25.372 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.372 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:25.373 DEBUG:boto:Port: 8000 2022-09-03T05:37:25.373 DEBUG:boto:Params: {} 2022-09-03T05:37:25.374 DEBUG:boto:Token: None 2022-09-03T05:37:25.375 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.375 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LbrSPdOAUya8XjxNvWub+OTuv9M= 2022-09-03T05:37:25.376 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LbrSPdOAUya8XjxNvWub+OTuv9M=', 'Content-Length': '0'} 2022-09-03T05:37:25.380 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000086720aacb86fb0cf-006312e815-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.426 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:25.428 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:25.429 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.429 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.432 DEBUG:boto:Method: GET 2022-09-03T05:37:25.435 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.436 DEBUG:boto:Data: 2022-09-03T05:37:25.437 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.438 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:25.439 DEBUG:boto:Port: 8001 2022-09-03T05:37:25.440 DEBUG:boto:Params: {} 2022-09-03T05:37:25.442 DEBUG:boto:Token: None 2022-09-03T05:37:25.443 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/ 2022-09-03T05:37:25.444 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE= 2022-09-03T05:37:25.445 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE=', 'Content-Length': '0'} 2022-09-03T05:37:25.449 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000013744f55a43804d1-006312e815-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.450 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:25.451 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:25.452 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:25.452 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.453 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.454 DEBUG:boto:Method: GET 2022-09-03T05:37:25.454 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.455 DEBUG:boto:Data: 2022-09-03T05:37:25.456 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.457 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:25.457 DEBUG:boto:Port: 8000 2022-09-03T05:37:25.458 DEBUG:boto:Params: {} 2022-09-03T05:37:25.458 DEBUG:boto:Token: None 2022-09-03T05:37:25.459 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/ 2022-09-03T05:37:25.460 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE= 2022-09-03T05:37:25.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE=', 'Content-Length': '0'} 2022-09-03T05:37:25.463 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002ae570a9cef9fcca-006312e815-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.464 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:25.469 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:25.471 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.472 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.473 DEBUG:boto:path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.474 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.475 DEBUG:boto:Method: GET 2022-09-03T05:37:25.476 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.477 DEBUG:boto:Data: 2022-09-03T05:37:25.477 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.479 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:25.479 DEBUG:boto:Port: 8001 2022-09-03T05:37:25.480 DEBUG:boto:Params: {} 2022-09-03T05:37:25.480 DEBUG:boto:Token: None 2022-09-03T05:37:25.482 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YRPOsTsHXlkwF3Bw94aMhOkyVrE= 2022-09-03T05:37:25.483 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YRPOsTsHXlkwF3Bw94aMhOkyVrE=', 'Content-Length': '0'} 2022-09-03T05:37:25.486 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'oYY-icmxg74gZ4carPTk38at9fTxjJi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003a90ed64175fcbec-006312e815-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.487 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.488 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.489 DEBUG:boto:path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.489 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.490 DEBUG:boto:Method: GET 2022-09-03T05:37:25.491 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.491 DEBUG:boto:Data: 2022-09-03T05:37:25.492 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.492 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:25.493 DEBUG:boto:Port: 8000 2022-09-03T05:37:25.493 DEBUG:boto:Params: {} 2022-09-03T05:37:25.494 DEBUG:boto:Token: None 2022-09-03T05:37:25.496 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.497 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YRPOsTsHXlkwF3Bw94aMhOkyVrE= 2022-09-03T05:37:25.498 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YRPOsTsHXlkwF3Bw94aMhOkyVrE=', 'Content-Length': '0'} 2022-09-03T05:37:25.502 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'oYY-icmxg74gZ4carPTk38at9fTxjJi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d63ddc03957acfc1-006312e815-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.506 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.508 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.509 DEBUG:boto:path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.510 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.510 DEBUG:boto:Method: HEAD 2022-09-03T05:37:25.511 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.511 DEBUG:boto:Data: 2022-09-03T05:37:25.512 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.512 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:25.513 DEBUG:boto:Port: 8001 2022-09-03T05:37:25.513 DEBUG:boto:Params: {} 2022-09-03T05:37:25.514 DEBUG:boto:Token: None 2022-09-03T05:37:25.515 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.516 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XWI4+vaRsYqhMkP8ndIrU+U6y8Y= 2022-09-03T05:37:25.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XWI4+vaRsYqhMkP8ndIrU+U6y8Y=', 'Content-Length': '0'} 2022-09-03T05:37:25.520 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'oYY-icmxg74gZ4carPTk38at9fTxjJi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007d1a5cec272daa6e-006312e815-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.522 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.523 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.524 DEBUG:boto:path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.524 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.525 DEBUG:boto:Method: HEAD 2022-09-03T05:37:25.526 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.527 DEBUG:boto:Data: 2022-09-03T05:37:25.527 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.529 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:25.531 DEBUG:boto:Port: 8000 2022-09-03T05:37:25.532 DEBUG:boto:Params: {} 2022-09-03T05:37:25.533 DEBUG:boto:Token: None 2022-09-03T05:37:25.534 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.535 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XWI4+vaRsYqhMkP8ndIrU+U6y8Y= 2022-09-03T05:37:25.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XWI4+vaRsYqhMkP8ndIrU+U6y8Y=', 'Content-Length': '0'} 2022-09-03T05:37:25.539 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'oYY-icmxg74gZ4carPTk38at9fTxjJi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000e58c0de1059aa89-006312e815-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.541 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:25.541 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.542 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.542 DEBUG:boto:path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.543 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.544 DEBUG:boto:Method: GET 2022-09-03T05:37:25.544 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.545 DEBUG:boto:Data: 2022-09-03T05:37:25.546 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.546 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:25.547 DEBUG:boto:Port: 8001 2022-09-03T05:37:25.550 DEBUG:boto:Params: {} 2022-09-03T05:37:25.552 DEBUG:boto:Token: None 2022-09-03T05:37:25.553 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.554 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YRPOsTsHXlkwF3Bw94aMhOkyVrE= 2022-09-03T05:37:25.556 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YRPOsTsHXlkwF3Bw94aMhOkyVrE=', 'Content-Length': '0'} 2022-09-03T05:37:25.563 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'oYY-icmxg74gZ4carPTk38at9fTxjJi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f6028a8769a33025-006312e815-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.610 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.612 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.613 DEBUG:boto:path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.614 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.615 DEBUG:boto:Method: GET 2022-09-03T05:37:25.616 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.617 DEBUG:boto:Data: 2022-09-03T05:37:25.618 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.619 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:25.620 DEBUG:boto:Port: 8000 2022-09-03T05:37:25.624 DEBUG:boto:Params: {} 2022-09-03T05:37:25.626 DEBUG:boto:Token: None 2022-09-03T05:37:25.627 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:25.628 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YRPOsTsHXlkwF3Bw94aMhOkyVrE= 2022-09-03T05:37:25.629 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YRPOsTsHXlkwF3Bw94aMhOkyVrE=', 'Content-Length': '0'} 2022-09-03T05:37:25.633 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'oYY-icmxg74gZ4carPTk38at9fTxjJi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c1980e9a45705d94-006312e815-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.634 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:25.635 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:25.637 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.637 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.638 DEBUG:boto:Method: GET 2022-09-03T05:37:25.639 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.640 DEBUG:boto:Data: 2022-09-03T05:37:25.641 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.643 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:25.643 DEBUG:boto:Port: 8001 2022-09-03T05:37:25.644 DEBUG:boto:Params: {} 2022-09-03T05:37:25.645 DEBUG:boto:Token: None 2022-09-03T05:37:25.645 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/ 2022-09-03T05:37:25.646 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE= 2022-09-03T05:37:25.647 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE=', 'Content-Length': '0'} 2022-09-03T05:37:25.650 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000089b4ff246ce3c6d8-006312e815-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.652 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:25.653 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:25.655 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:25.656 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.656 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.657 DEBUG:boto:Method: GET 2022-09-03T05:37:25.658 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.659 DEBUG:boto:Data: 2022-09-03T05:37:25.660 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.661 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:25.663 DEBUG:boto:Port: 8000 2022-09-03T05:37:25.664 DEBUG:boto:Params: {} 2022-09-03T05:37:25.665 DEBUG:boto:Token: None 2022-09-03T05:37:25.666 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/ 2022-09-03T05:37:25.667 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE= 2022-09-03T05:37:25.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE=', 'Content-Length': '0'} 2022-09-03T05:37:25.672 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e731c350cb58660c-006312e815-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.673 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:25.674 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.675 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.676 DEBUG:boto:Method: HEAD 2022-09-03T05:37:25.677 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:25.678 DEBUG:boto:Data: 2022-09-03T05:37:25.679 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.679 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:25.680 DEBUG:boto:Port: 8001 2022-09-03T05:37:25.681 DEBUG:boto:Params: {} 2022-09-03T05:37:25.681 DEBUG:boto:Token: None 2022-09-03T05:37:25.682 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj 2022-09-03T05:37:25.682 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5IFWKkxEO7nGaY8aewaXmaI4X3s= 2022-09-03T05:37:25.683 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5IFWKkxEO7nGaY8aewaXmaI4X3s=', 'Content-Length': '0'} 2022-09-03T05:37:25.687 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009ec7d52ca7dc146b-006312e815-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.688 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.689 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.690 DEBUG:boto:Method: HEAD 2022-09-03T05:37:25.691 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:25.692 DEBUG:boto:Data: 2022-09-03T05:37:25.693 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.694 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:25.695 DEBUG:boto:Port: 8000 2022-09-03T05:37:25.696 DEBUG:boto:Params: {} 2022-09-03T05:37:25.697 DEBUG:boto:Token: None 2022-09-03T05:37:25.698 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj 2022-09-03T05:37:25.699 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5IFWKkxEO7nGaY8aewaXmaI4X3s= 2022-09-03T05:37:25.699 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5IFWKkxEO7nGaY8aewaXmaI4X3s=', 'Content-Length': '0'} 2022-09-03T05:37:25.704 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000506a1b2e8a960005-006312e815-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.706 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:25.707 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.708 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.709 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.710 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.711 DEBUG:boto:Method: GET 2022-09-03T05:37:25.712 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.713 DEBUG:boto:Data: 2022-09-03T05:37:25.714 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.716 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:25.717 DEBUG:boto:Port: 8001 2022-09-03T05:37:25.718 DEBUG:boto:Params: {} 2022-09-03T05:37:25.719 DEBUG:boto:Token: None 2022-09-03T05:37:25.720 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.721 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LbrSPdOAUya8XjxNvWub+OTuv9M= 2022-09-03T05:37:25.722 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LbrSPdOAUya8XjxNvWub+OTuv9M=', 'Content-Length': '0'} 2022-09-03T05:37:25.725 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000adaf6e407a239f7d-006312e815-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.766 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.767 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.768 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.769 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.769 DEBUG:boto:Method: GET 2022-09-03T05:37:25.770 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.770 DEBUG:boto:Data: 2022-09-03T05:37:25.771 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.771 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:25.772 DEBUG:boto:Port: 8000 2022-09-03T05:37:25.773 DEBUG:boto:Params: {} 2022-09-03T05:37:25.774 DEBUG:boto:Token: None 2022-09-03T05:37:25.774 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.775 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LbrSPdOAUya8XjxNvWub+OTuv9M= 2022-09-03T05:37:25.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LbrSPdOAUya8XjxNvWub+OTuv9M=', 'Content-Length': '0'} 2022-09-03T05:37:25.778 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f98f80c792652249-006312e815-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.780 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:25.781 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:25.782 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.783 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.784 DEBUG:boto:Method: GET 2022-09-03T05:37:25.786 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.787 DEBUG:boto:Data: 2022-09-03T05:37:25.788 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.790 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:25.790 DEBUG:boto:Port: 8001 2022-09-03T05:37:25.791 DEBUG:boto:Params: {} 2022-09-03T05:37:25.792 DEBUG:boto:Token: None 2022-09-03T05:37:25.792 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/ 2022-09-03T05:37:25.793 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE= 2022-09-03T05:37:25.793 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE=', 'Content-Length': '0'} 2022-09-03T05:37:25.797 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002913f913183b65ec-006312e815-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.799 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:25.799 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:25.800 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:25.801 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.801 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.802 DEBUG:boto:Method: GET 2022-09-03T05:37:25.802 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.803 DEBUG:boto:Data: 2022-09-03T05:37:25.803 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.804 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:25.804 DEBUG:boto:Port: 8000 2022-09-03T05:37:25.805 DEBUG:boto:Params: {} 2022-09-03T05:37:25.805 DEBUG:boto:Token: None 2022-09-03T05:37:25.806 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/ 2022-09-03T05:37:25.807 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE= 2022-09-03T05:37:25.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE=', 'Content-Length': '0'} 2022-09-03T05:37:25.811 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ad212fb107021af0-006312e815-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.812 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:25.813 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:25.813 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.814 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.814 DEBUG:boto:path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.815 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.815 DEBUG:boto:Method: GET 2022-09-03T05:37:25.816 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.817 DEBUG:boto:Data: 2022-09-03T05:37:25.817 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.818 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:25.819 DEBUG:boto:Port: 8001 2022-09-03T05:37:25.819 DEBUG:boto:Params: {} 2022-09-03T05:37:25.820 DEBUG:boto:Token: None 2022-09-03T05:37:25.821 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.821 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UC7RAkhZ9GZVWl0l6Z1W52FVnxI= 2022-09-03T05:37:25.822 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UC7RAkhZ9GZVWl0l6Z1W52FVnxI=', 'Content-Length': '0'} 2022-09-03T05:37:25.824 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000709623295f662239-006312e815-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.825 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.826 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.826 DEBUG:boto:path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.827 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.827 DEBUG:boto:Method: GET 2022-09-03T05:37:25.828 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.828 DEBUG:boto:Data: 2022-09-03T05:37:25.829 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.829 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:25.830 DEBUG:boto:Port: 8000 2022-09-03T05:37:25.831 DEBUG:boto:Params: {} 2022-09-03T05:37:25.831 DEBUG:boto:Token: None 2022-09-03T05:37:25.832 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.833 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UC7RAkhZ9GZVWl0l6Z1W52FVnxI= 2022-09-03T05:37:25.833 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UC7RAkhZ9GZVWl0l6Z1W52FVnxI=', 'Content-Length': '0'} 2022-09-03T05:37:25.835 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000af118383915f0f46-006312e815-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.836 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.837 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.837 DEBUG:boto:path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.838 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.838 DEBUG:boto:Method: HEAD 2022-09-03T05:37:25.839 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.839 DEBUG:boto:Data: 2022-09-03T05:37:25.840 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.840 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:25.841 DEBUG:boto:Port: 8001 2022-09-03T05:37:25.841 DEBUG:boto:Params: {} 2022-09-03T05:37:25.842 DEBUG:boto:Token: None 2022-09-03T05:37:25.842 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.843 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G+MUIo0DFBOo0Dcn5uGnM44qzMA= 2022-09-03T05:37:25.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G+MUIo0DFBOo0Dcn5uGnM44qzMA=', 'Content-Length': '0'} 2022-09-03T05:37:25.846 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b4e6c28be044a737-006312e815-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.847 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.847 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.848 DEBUG:boto:path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.848 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.849 DEBUG:boto:Method: HEAD 2022-09-03T05:37:25.850 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.851 DEBUG:boto:Data: 2022-09-03T05:37:25.852 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.852 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:25.853 DEBUG:boto:Port: 8000 2022-09-03T05:37:25.854 DEBUG:boto:Params: {} 2022-09-03T05:37:25.857 DEBUG:boto:Token: None 2022-09-03T05:37:25.858 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.858 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G+MUIo0DFBOo0Dcn5uGnM44qzMA= 2022-09-03T05:37:25.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G+MUIo0DFBOo0Dcn5uGnM44qzMA=', 'Content-Length': '0'} 2022-09-03T05:37:25.862 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000469ac9381db72134-006312e815-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.863 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:25.864 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.865 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.866 DEBUG:boto:path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.867 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.868 DEBUG:boto:Method: GET 2022-09-03T05:37:25.868 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.869 DEBUG:boto:Data: 2022-09-03T05:37:25.870 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.871 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:25.873 DEBUG:boto:Port: 8001 2022-09-03T05:37:25.874 DEBUG:boto:Params: {} 2022-09-03T05:37:25.875 DEBUG:boto:Token: None 2022-09-03T05:37:25.876 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.878 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UC7RAkhZ9GZVWl0l6Z1W52FVnxI= 2022-09-03T05:37:25.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UC7RAkhZ9GZVWl0l6Z1W52FVnxI=', 'Content-Length': '0'} 2022-09-03T05:37:25.882 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cb57cfe036c517d0-006312e815-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.883 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.884 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.884 DEBUG:boto:path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.885 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.885 DEBUG:boto:Method: GET 2022-09-03T05:37:25.886 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.887 DEBUG:boto:Data: 2022-09-03T05:37:25.888 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.889 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:25.890 DEBUG:boto:Port: 8000 2022-09-03T05:37:25.891 DEBUG:boto:Params: {} 2022-09-03T05:37:25.892 DEBUG:boto:Token: None 2022-09-03T05:37:25.893 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:25.894 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UC7RAkhZ9GZVWl0l6Z1W52FVnxI= 2022-09-03T05:37:25.895 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UC7RAkhZ9GZVWl0l6Z1W52FVnxI=', 'Content-Length': '0'} 2022-09-03T05:37:25.898 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b45e933508ad7285-006312e815-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.900 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:25.901 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:25.902 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.902 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.903 DEBUG:boto:Method: GET 2022-09-03T05:37:25.904 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.904 DEBUG:boto:Data: 2022-09-03T05:37:25.905 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.906 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:25.907 DEBUG:boto:Port: 8001 2022-09-03T05:37:25.907 DEBUG:boto:Params: {} 2022-09-03T05:37:25.908 DEBUG:boto:Token: None 2022-09-03T05:37:25.908 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/ 2022-09-03T05:37:25.909 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE= 2022-09-03T05:37:25.910 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE=', 'Content-Length': '0'} 2022-09-03T05:37:25.913 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b06ebb2aa9cb9bf2-006312e815-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.914 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:25.915 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:25.915 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:25.916 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.916 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.917 DEBUG:boto:Method: GET 2022-09-03T05:37:25.917 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:25.918 DEBUG:boto:Data: 2022-09-03T05:37:25.919 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.919 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:25.920 DEBUG:boto:Port: 8000 2022-09-03T05:37:25.920 DEBUG:boto:Params: {} 2022-09-03T05:37:25.921 DEBUG:boto:Token: None 2022-09-03T05:37:25.922 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/ 2022-09-03T05:37:25.923 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE= 2022-09-03T05:37:25.923 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YQcA3PWQ7p+bGJDJEndTUbNJCpE=', 'Content-Length': '0'} 2022-09-03T05:37:25.926 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005eb6d4e06fc9d224-006312e815-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.927 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:25.928 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.929 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.929 DEBUG:boto:Method: HEAD 2022-09-03T05:37:25.930 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:25.930 DEBUG:boto:Data: 2022-09-03T05:37:25.931 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.931 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:25.932 DEBUG:boto:Port: 8001 2022-09-03T05:37:25.932 DEBUG:boto:Params: {} 2022-09-03T05:37:25.933 DEBUG:boto:Token: None 2022-09-03T05:37:25.934 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj 2022-09-03T05:37:25.935 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5IFWKkxEO7nGaY8aewaXmaI4X3s= 2022-09-03T05:37:25.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5IFWKkxEO7nGaY8aewaXmaI4X3s=', 'Content-Length': '0'} 2022-09-03T05:37:25.940 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007e1af71dc0419244-006312e815-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.941 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.942 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.943 DEBUG:boto:Method: HEAD 2022-09-03T05:37:25.944 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:25.945 DEBUG:boto:Data: 2022-09-03T05:37:25.946 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.946 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:25.947 DEBUG:boto:Port: 8000 2022-09-03T05:37:25.947 DEBUG:boto:Params: {} 2022-09-03T05:37:25.948 DEBUG:boto:Token: None 2022-09-03T05:37:25.949 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj 2022-09-03T05:37:25.950 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5IFWKkxEO7nGaY8aewaXmaI4X3s= 2022-09-03T05:37:25.951 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5IFWKkxEO7nGaY8aewaXmaI4X3s=', 'Content-Length': '0'} 2022-09-03T05:37:25.955 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c2f68d19057becb2-006312e815-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.956 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:25.957 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.957 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.958 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.958 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.959 DEBUG:boto:Method: GET 2022-09-03T05:37:25.959 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.960 DEBUG:boto:Data: 2022-09-03T05:37:25.961 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.961 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:25.962 DEBUG:boto:Port: 8001 2022-09-03T05:37:25.962 DEBUG:boto:Params: {} 2022-09-03T05:37:25.963 DEBUG:boto:Token: None 2022-09-03T05:37:25.963 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.964 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LbrSPdOAUya8XjxNvWub+OTuv9M= 2022-09-03T05:37:25.965 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LbrSPdOAUya8XjxNvWub+OTuv9M=', 'Content-Length': '0'} 2022-09-03T05:37:25.967 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000671e5762557fe5e-006312e815-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:25.968 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:25.968 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:25.969 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.969 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.970 DEBUG:boto:Method: GET 2022-09-03T05:37:25.971 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.971 DEBUG:boto:Data: 2022-09-03T05:37:25.972 DEBUG:boto:Headers: {} 2022-09-03T05:37:25.972 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:25.973 DEBUG:boto:Port: 8000 2022-09-03T05:37:25.973 DEBUG:boto:Params: {} 2022-09-03T05:37:25.974 DEBUG:boto:Token: None 2022-09-03T05:37:25.975 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:25 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:25.975 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LbrSPdOAUya8XjxNvWub+OTuv9M= 2022-09-03T05:37:25.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LbrSPdOAUya8XjxNvWub+OTuv9M=', 'Content-Length': '0'} 2022-09-03T05:37:25.978 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001ded68b7a336cd89-006312e815-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.026 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:26.027 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:26.028 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.028 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.029 DEBUG:boto:Method: GET 2022-09-03T05:37:26.029 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.030 DEBUG:boto:Data: 2022-09-03T05:37:26.031 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.032 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:26.033 DEBUG:boto:Port: 8001 2022-09-03T05:37:26.034 DEBUG:boto:Params: {} 2022-09-03T05:37:26.036 DEBUG:boto:Token: None 2022-09-03T05:37:26.037 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/ 2022-09-03T05:37:26.038 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw= 2022-09-03T05:37:26.039 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw=', 'Content-Length': '0'} 2022-09-03T05:37:26.042 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000909aa116ed23b584-006312e816-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.044 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:26.045 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:26.046 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:26.047 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.048 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.048 DEBUG:boto:Method: GET 2022-09-03T05:37:26.049 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.049 DEBUG:boto:Data: 2022-09-03T05:37:26.050 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.050 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:26.051 DEBUG:boto:Port: 8000 2022-09-03T05:37:26.051 DEBUG:boto:Params: {} 2022-09-03T05:37:26.052 DEBUG:boto:Token: None 2022-09-03T05:37:26.053 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/ 2022-09-03T05:37:26.053 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw= 2022-09-03T05:37:26.054 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw=', 'Content-Length': '0'} 2022-09-03T05:37:26.057 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005889251ac5a2ec37-006312e816-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.059 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:26.060 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:26.061 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.062 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.063 DEBUG:boto:path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.064 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.065 DEBUG:boto:Method: GET 2022-09-03T05:37:26.065 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.066 DEBUG:boto:Data: 2022-09-03T05:37:26.067 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.067 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:26.068 DEBUG:boto:Port: 8001 2022-09-03T05:37:26.068 DEBUG:boto:Params: {} 2022-09-03T05:37:26.069 DEBUG:boto:Token: None 2022-09-03T05:37:26.070 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.071 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9vG9DHTkz3NYK0lohOENky300hA= 2022-09-03T05:37:26.071 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9vG9DHTkz3NYK0lohOENky300hA=', 'Content-Length': '0'} 2022-09-03T05:37:26.074 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004cf0c557f29668f4-006312e816-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.116 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.116 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.117 DEBUG:boto:path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.118 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.118 DEBUG:boto:Method: GET 2022-09-03T05:37:26.119 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.119 DEBUG:boto:Data: 2022-09-03T05:37:26.120 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.120 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:26.121 DEBUG:boto:Port: 8000 2022-09-03T05:37:26.122 DEBUG:boto:Params: {} 2022-09-03T05:37:26.122 DEBUG:boto:Token: None 2022-09-03T05:37:26.123 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.123 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9vG9DHTkz3NYK0lohOENky300hA= 2022-09-03T05:37:26.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9vG9DHTkz3NYK0lohOENky300hA=', 'Content-Length': '0'} 2022-09-03T05:37:26.127 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000058180f5bc6d1e58d-006312e816-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.127 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.128 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.129 DEBUG:boto:path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.129 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.130 DEBUG:boto:Method: HEAD 2022-09-03T05:37:26.130 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.131 DEBUG:boto:Data: 2022-09-03T05:37:26.131 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.132 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:26.132 DEBUG:boto:Port: 8001 2022-09-03T05:37:26.133 DEBUG:boto:Params: {} 2022-09-03T05:37:26.133 DEBUG:boto:Token: None 2022-09-03T05:37:26.134 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.134 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rShUQADu/FJH0bjlCPB2FIUB6BA= 2022-09-03T05:37:26.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rShUQADu/FJH0bjlCPB2FIUB6BA=', 'Content-Length': '0'} 2022-09-03T05:37:26.140 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000765a43d266cf9de-006312e816-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.141 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.142 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.142 DEBUG:boto:path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.143 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.143 DEBUG:boto:Method: HEAD 2022-09-03T05:37:26.144 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.144 DEBUG:boto:Data: 2022-09-03T05:37:26.145 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.146 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:26.150 DEBUG:boto:Port: 8000 2022-09-03T05:37:26.151 DEBUG:boto:Params: {} 2022-09-03T05:37:26.151 DEBUG:boto:Token: None 2022-09-03T05:37:26.152 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.153 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rShUQADu/FJH0bjlCPB2FIUB6BA= 2022-09-03T05:37:26.154 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rShUQADu/FJH0bjlCPB2FIUB6BA=', 'Content-Length': '0'} 2022-09-03T05:37:26.156 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f856253d5df90d24-006312e816-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.157 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:26.157 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.158 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.158 DEBUG:boto:path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.159 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.159 DEBUG:boto:Method: GET 2022-09-03T05:37:26.160 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.160 DEBUG:boto:Data: 2022-09-03T05:37:26.161 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.161 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:26.162 DEBUG:boto:Port: 8001 2022-09-03T05:37:26.163 DEBUG:boto:Params: {} 2022-09-03T05:37:26.163 DEBUG:boto:Token: None 2022-09-03T05:37:26.164 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.165 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9vG9DHTkz3NYK0lohOENky300hA= 2022-09-03T05:37:26.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9vG9DHTkz3NYK0lohOENky300hA=', 'Content-Length': '0'} 2022-09-03T05:37:26.170 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000050abcc393b2b8cd1-006312e816-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.172 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.173 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.173 DEBUG:boto:path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.174 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.175 DEBUG:boto:Method: GET 2022-09-03T05:37:26.177 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.178 DEBUG:boto:Data: 2022-09-03T05:37:26.179 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.180 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:26.182 DEBUG:boto:Port: 8000 2022-09-03T05:37:26.183 DEBUG:boto:Params: {} 2022-09-03T05:37:26.184 DEBUG:boto:Token: None 2022-09-03T05:37:26.185 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:26.186 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9vG9DHTkz3NYK0lohOENky300hA= 2022-09-03T05:37:26.187 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9vG9DHTkz3NYK0lohOENky300hA=', 'Content-Length': '0'} 2022-09-03T05:37:26.189 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000036982ae625f1156d-006312e816-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.190 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:26.192 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:26.193 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.193 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.194 DEBUG:boto:Method: GET 2022-09-03T05:37:26.195 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.196 DEBUG:boto:Data: 2022-09-03T05:37:26.197 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.198 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:26.199 DEBUG:boto:Port: 8001 2022-09-03T05:37:26.200 DEBUG:boto:Params: {} 2022-09-03T05:37:26.201 DEBUG:boto:Token: None 2022-09-03T05:37:26.202 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/ 2022-09-03T05:37:26.203 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw= 2022-09-03T05:37:26.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw=', 'Content-Length': '0'} 2022-09-03T05:37:26.208 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bcc7b47ec319f88a-006312e816-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.258 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:26.259 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:26.259 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:26.260 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.261 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.261 DEBUG:boto:Method: GET 2022-09-03T05:37:26.262 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.262 DEBUG:boto:Data: 2022-09-03T05:37:26.263 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.263 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:26.264 DEBUG:boto:Port: 8000 2022-09-03T05:37:26.264 DEBUG:boto:Params: {} 2022-09-03T05:37:26.265 DEBUG:boto:Token: None 2022-09-03T05:37:26.265 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/ 2022-09-03T05:37:26.266 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw= 2022-09-03T05:37:26.267 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw=', 'Content-Length': '0'} 2022-09-03T05:37:26.271 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005c50f3e29c350de8-006312e816-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.273 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:26.274 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.274 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.275 DEBUG:boto:Method: HEAD 2022-09-03T05:37:26.275 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:26.276 DEBUG:boto:Data: 2022-09-03T05:37:26.277 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.278 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:26.279 DEBUG:boto:Port: 8001 2022-09-03T05:37:26.280 DEBUG:boto:Params: {} 2022-09-03T05:37:26.281 DEBUG:boto:Token: None 2022-09-03T05:37:26.282 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj 2022-09-03T05:37:26.283 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2QXGvMbOwXuBXbIcPQxrAkjvJgc= 2022-09-03T05:37:26.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2QXGvMbOwXuBXbIcPQxrAkjvJgc=', 'Content-Length': '0'} 2022-09-03T05:37:26.288 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003f27eb4faef4b934-006312e816-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.290 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.290 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.291 DEBUG:boto:Method: HEAD 2022-09-03T05:37:26.292 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:26.292 DEBUG:boto:Data: 2022-09-03T05:37:26.293 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.293 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:26.294 DEBUG:boto:Port: 8000 2022-09-03T05:37:26.294 DEBUG:boto:Params: {} 2022-09-03T05:37:26.295 DEBUG:boto:Token: None 2022-09-03T05:37:26.296 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj 2022-09-03T05:37:26.296 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2QXGvMbOwXuBXbIcPQxrAkjvJgc= 2022-09-03T05:37:26.297 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2QXGvMbOwXuBXbIcPQxrAkjvJgc=', 'Content-Length': '0'} 2022-09-03T05:37:26.301 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004c376b3263e128df-006312e816-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.303 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:26.303 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.304 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.305 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.307 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.308 DEBUG:boto:Method: GET 2022-09-03T05:37:26.309 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.310 DEBUG:boto:Data: 2022-09-03T05:37:26.311 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.312 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:26.313 DEBUG:boto:Port: 8001 2022-09-03T05:37:26.313 DEBUG:boto:Params: {} 2022-09-03T05:37:26.315 DEBUG:boto:Token: None 2022-09-03T05:37:26.316 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.316 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5+25tEUuBdxcBdrxSvFjALRwDIs= 2022-09-03T05:37:26.317 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5+25tEUuBdxcBdrxSvFjALRwDIs=', 'Content-Length': '0'} 2022-09-03T05:37:26.320 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000ac7e1d266564b0e-006312e816-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.321 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.322 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.322 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.323 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.323 DEBUG:boto:Method: GET 2022-09-03T05:37:26.324 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.324 DEBUG:boto:Data: 2022-09-03T05:37:26.325 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.327 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:26.327 DEBUG:boto:Port: 8000 2022-09-03T05:37:26.328 DEBUG:boto:Params: {} 2022-09-03T05:37:26.329 DEBUG:boto:Token: None 2022-09-03T05:37:26.330 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.331 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5+25tEUuBdxcBdrxSvFjALRwDIs= 2022-09-03T05:37:26.331 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5+25tEUuBdxcBdrxSvFjALRwDIs=', 'Content-Length': '0'} 2022-09-03T05:37:26.333 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000026ceec909adefe5f-006312e816-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.374 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:26.375 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:26.376 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.376 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.377 DEBUG:boto:Method: GET 2022-09-03T05:37:26.377 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.378 DEBUG:boto:Data: 2022-09-03T05:37:26.378 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.379 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:26.379 DEBUG:boto:Port: 8001 2022-09-03T05:37:26.380 DEBUG:boto:Params: {} 2022-09-03T05:37:26.381 DEBUG:boto:Token: None 2022-09-03T05:37:26.382 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/ 2022-09-03T05:37:26.382 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw= 2022-09-03T05:37:26.383 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw=', 'Content-Length': '0'} 2022-09-03T05:37:26.386 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000058eec220e25c625b-006312e816-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.387 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:26.388 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:26.391 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:26.392 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.393 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.394 DEBUG:boto:Method: GET 2022-09-03T05:37:26.395 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.396 DEBUG:boto:Data: 2022-09-03T05:37:26.397 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.397 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:26.399 DEBUG:boto:Port: 8000 2022-09-03T05:37:26.400 DEBUG:boto:Params: {} 2022-09-03T05:37:26.401 DEBUG:boto:Token: None 2022-09-03T05:37:26.402 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/ 2022-09-03T05:37:26.403 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw= 2022-09-03T05:37:26.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw=', 'Content-Length': '0'} 2022-09-03T05:37:26.407 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a1022e322847a2cb-006312e816-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.408 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:26.409 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:26.410 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.410 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.411 DEBUG:boto:path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.412 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.412 DEBUG:boto:Method: GET 2022-09-03T05:37:26.413 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.413 DEBUG:boto:Data: 2022-09-03T05:37:26.414 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.414 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:26.415 DEBUG:boto:Port: 8001 2022-09-03T05:37:26.415 DEBUG:boto:Params: {} 2022-09-03T05:37:26.416 DEBUG:boto:Token: None 2022-09-03T05:37:26.418 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.418 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VXN+E52+EacJkf3bgZV9QLaPGxo= 2022-09-03T05:37:26.419 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VXN+E52+EacJkf3bgZV9QLaPGxo=', 'Content-Length': '0'} 2022-09-03T05:37:26.422 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'CFKxqazR5g.k3.oWs62bQgs9moHAimI'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f60678a76da06569-006312e816-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.466 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.468 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.469 DEBUG:boto:path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.471 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.472 DEBUG:boto:Method: GET 2022-09-03T05:37:26.473 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.475 DEBUG:boto:Data: 2022-09-03T05:37:26.477 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.479 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:26.479 DEBUG:boto:Port: 8000 2022-09-03T05:37:26.480 DEBUG:boto:Params: {} 2022-09-03T05:37:26.482 DEBUG:boto:Token: None 2022-09-03T05:37:26.483 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.485 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VXN+E52+EacJkf3bgZV9QLaPGxo= 2022-09-03T05:37:26.486 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VXN+E52+EacJkf3bgZV9QLaPGxo=', 'Content-Length': '0'} 2022-09-03T05:37:26.489 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'CFKxqazR5g.k3.oWs62bQgs9moHAimI'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003624d3d7ff0d7f45-006312e816-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.491 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.491 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.492 DEBUG:boto:path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.492 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.493 DEBUG:boto:Method: HEAD 2022-09-03T05:37:26.494 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.494 DEBUG:boto:Data: 2022-09-03T05:37:26.495 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.496 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:26.497 DEBUG:boto:Port: 8001 2022-09-03T05:37:26.499 DEBUG:boto:Params: {} 2022-09-03T05:37:26.500 DEBUG:boto:Token: None 2022-09-03T05:37:26.501 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.503 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j7+eTGCYUtHpVbK8txjnLcJIk5w= 2022-09-03T05:37:26.504 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j7+eTGCYUtHpVbK8txjnLcJIk5w=', 'Content-Length': '0'} 2022-09-03T05:37:26.508 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'CFKxqazR5g.k3.oWs62bQgs9moHAimI'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000579403500e7c458d-006312e816-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.510 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.511 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.512 DEBUG:boto:path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.512 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.513 DEBUG:boto:Method: HEAD 2022-09-03T05:37:26.514 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.515 DEBUG:boto:Data: 2022-09-03T05:37:26.516 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.517 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:26.518 DEBUG:boto:Port: 8000 2022-09-03T05:37:26.519 DEBUG:boto:Params: {} 2022-09-03T05:37:26.520 DEBUG:boto:Token: None 2022-09-03T05:37:26.521 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.522 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j7+eTGCYUtHpVbK8txjnLcJIk5w= 2022-09-03T05:37:26.523 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j7+eTGCYUtHpVbK8txjnLcJIk5w=', 'Content-Length': '0'} 2022-09-03T05:37:26.526 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'CFKxqazR5g.k3.oWs62bQgs9moHAimI'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004edbc63d41f9dc1d-006312e816-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.527 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:26.527 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.528 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.529 DEBUG:boto:path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.529 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.530 DEBUG:boto:Method: GET 2022-09-03T05:37:26.531 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.531 DEBUG:boto:Data: 2022-09-03T05:37:26.532 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.533 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:26.533 DEBUG:boto:Port: 8001 2022-09-03T05:37:26.534 DEBUG:boto:Params: {} 2022-09-03T05:37:26.535 DEBUG:boto:Token: None 2022-09-03T05:37:26.535 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.537 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VXN+E52+EacJkf3bgZV9QLaPGxo= 2022-09-03T05:37:26.537 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VXN+E52+EacJkf3bgZV9QLaPGxo=', 'Content-Length': '0'} 2022-09-03T05:37:26.545 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'CFKxqazR5g.k3.oWs62bQgs9moHAimI'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001db480c97414b6dd-006312e816-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.547 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.547 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.548 DEBUG:boto:path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.550 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.550 DEBUG:boto:Method: GET 2022-09-03T05:37:26.551 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.552 DEBUG:boto:Data: 2022-09-03T05:37:26.552 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.553 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:26.554 DEBUG:boto:Port: 8000 2022-09-03T05:37:26.555 DEBUG:boto:Params: {} 2022-09-03T05:37:26.556 DEBUG:boto:Token: None 2022-09-03T05:37:26.556 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:26.557 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VXN+E52+EacJkf3bgZV9QLaPGxo= 2022-09-03T05:37:26.558 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VXN+E52+EacJkf3bgZV9QLaPGxo=', 'Content-Length': '0'} 2022-09-03T05:37:26.560 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'CFKxqazR5g.k3.oWs62bQgs9moHAimI'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000009e039c09862160-006312e816-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.561 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:26.562 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:26.563 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.563 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.565 DEBUG:boto:Method: GET 2022-09-03T05:37:26.565 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.566 DEBUG:boto:Data: 2022-09-03T05:37:26.566 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.567 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:26.567 DEBUG:boto:Port: 8001 2022-09-03T05:37:26.568 DEBUG:boto:Params: {} 2022-09-03T05:37:26.569 DEBUG:boto:Token: None 2022-09-03T05:37:26.569 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/ 2022-09-03T05:37:26.570 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw= 2022-09-03T05:37:26.571 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw=', 'Content-Length': '0'} 2022-09-03T05:37:26.574 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000707595fcadbfcf6-006312e816-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.575 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:26.577 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:26.578 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:26.579 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.579 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.580 DEBUG:boto:Method: GET 2022-09-03T05:37:26.581 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.582 DEBUG:boto:Data: 2022-09-03T05:37:26.583 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.584 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:26.584 DEBUG:boto:Port: 8000 2022-09-03T05:37:26.585 DEBUG:boto:Params: {} 2022-09-03T05:37:26.586 DEBUG:boto:Token: None 2022-09-03T05:37:26.587 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/ 2022-09-03T05:37:26.588 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw= 2022-09-03T05:37:26.589 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw=', 'Content-Length': '0'} 2022-09-03T05:37:26.593 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e285eb2c02113935-006312e816-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.594 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:26.595 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.596 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.597 DEBUG:boto:Method: HEAD 2022-09-03T05:37:26.598 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:26.599 DEBUG:boto:Data: 2022-09-03T05:37:26.599 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.600 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:26.600 DEBUG:boto:Port: 8001 2022-09-03T05:37:26.601 DEBUG:boto:Params: {} 2022-09-03T05:37:26.602 DEBUG:boto:Token: None 2022-09-03T05:37:26.603 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj 2022-09-03T05:37:26.604 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2QXGvMbOwXuBXbIcPQxrAkjvJgc= 2022-09-03T05:37:26.605 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2QXGvMbOwXuBXbIcPQxrAkjvJgc=', 'Content-Length': '0'} 2022-09-03T05:37:26.609 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bab1e596ffeb6f56-006312e816-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.611 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.612 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.613 DEBUG:boto:Method: HEAD 2022-09-03T05:37:26.613 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:26.614 DEBUG:boto:Data: 2022-09-03T05:37:26.615 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.615 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:26.616 DEBUG:boto:Port: 8000 2022-09-03T05:37:26.617 DEBUG:boto:Params: {} 2022-09-03T05:37:26.619 DEBUG:boto:Token: None 2022-09-03T05:37:26.620 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj 2022-09-03T05:37:26.621 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2QXGvMbOwXuBXbIcPQxrAkjvJgc= 2022-09-03T05:37:26.622 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2QXGvMbOwXuBXbIcPQxrAkjvJgc=', 'Content-Length': '0'} 2022-09-03T05:37:26.627 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000686c1b53813d53fd-006312e816-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.628 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:26.629 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.630 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.630 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.631 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.631 DEBUG:boto:Method: GET 2022-09-03T05:37:26.632 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.632 DEBUG:boto:Data: 2022-09-03T05:37:26.633 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.634 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:26.635 DEBUG:boto:Port: 8001 2022-09-03T05:37:26.636 DEBUG:boto:Params: {} 2022-09-03T05:37:26.637 DEBUG:boto:Token: None 2022-09-03T05:37:26.638 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.639 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5+25tEUuBdxcBdrxSvFjALRwDIs= 2022-09-03T05:37:26.639 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5+25tEUuBdxcBdrxSvFjALRwDIs=', 'Content-Length': '0'} 2022-09-03T05:37:26.642 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000078ec81f6fae0baad-006312e816-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.643 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.645 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.646 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.647 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.648 DEBUG:boto:Method: GET 2022-09-03T05:37:26.649 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.649 DEBUG:boto:Data: 2022-09-03T05:37:26.650 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.651 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:26.652 DEBUG:boto:Port: 8000 2022-09-03T05:37:26.652 DEBUG:boto:Params: {} 2022-09-03T05:37:26.653 DEBUG:boto:Token: None 2022-09-03T05:37:26.654 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.654 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5+25tEUuBdxcBdrxSvFjALRwDIs= 2022-09-03T05:37:26.655 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5+25tEUuBdxcBdrxSvFjALRwDIs=', 'Content-Length': '0'} 2022-09-03T05:37:26.658 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000033f53e1703aa703d-006312e816-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.659 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:26.659 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:26.660 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.660 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.661 DEBUG:boto:Method: GET 2022-09-03T05:37:26.661 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.662 DEBUG:boto:Data: 2022-09-03T05:37:26.663 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.663 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:26.664 DEBUG:boto:Port: 8001 2022-09-03T05:37:26.664 DEBUG:boto:Params: {} 2022-09-03T05:37:26.665 DEBUG:boto:Token: None 2022-09-03T05:37:26.665 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/ 2022-09-03T05:37:26.666 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw= 2022-09-03T05:37:26.667 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw=', 'Content-Length': '0'} 2022-09-03T05:37:26.670 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000084369d31aa3c17f7-006312e816-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.714 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:26.715 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:26.716 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:26.717 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.717 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.718 DEBUG:boto:Method: GET 2022-09-03T05:37:26.718 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.719 DEBUG:boto:Data: 2022-09-03T05:37:26.719 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.720 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:26.720 DEBUG:boto:Port: 8000 2022-09-03T05:37:26.721 DEBUG:boto:Params: {} 2022-09-03T05:37:26.721 DEBUG:boto:Token: None 2022-09-03T05:37:26.722 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/ 2022-09-03T05:37:26.723 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw= 2022-09-03T05:37:26.723 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw=', 'Content-Length': '0'} 2022-09-03T05:37:26.726 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a972621e034871ab-006312e816-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.727 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:26.728 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:26.729 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.730 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.730 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.731 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.732 DEBUG:boto:Method: GET 2022-09-03T05:37:26.733 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.734 DEBUG:boto:Data: 2022-09-03T05:37:26.735 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.737 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:26.738 DEBUG:boto:Port: 8001 2022-09-03T05:37:26.739 DEBUG:boto:Params: {} 2022-09-03T05:37:26.740 DEBUG:boto:Token: None 2022-09-03T05:37:26.741 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.742 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jUBQH6rTXdso4Gg+cRDcozbaB1g= 2022-09-03T05:37:26.743 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jUBQH6rTXdso4Gg+cRDcozbaB1g=', 'Content-Length': '0'} 2022-09-03T05:37:26.745 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009abb3697070f76e7-006312e816-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.747 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.747 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.748 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.748 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.749 DEBUG:boto:Method: GET 2022-09-03T05:37:26.749 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.750 DEBUG:boto:Data: 2022-09-03T05:37:26.750 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.751 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:26.752 DEBUG:boto:Port: 8000 2022-09-03T05:37:26.752 DEBUG:boto:Params: {} 2022-09-03T05:37:26.753 DEBUG:boto:Token: None 2022-09-03T05:37:26.754 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.755 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jUBQH6rTXdso4Gg+cRDcozbaB1g= 2022-09-03T05:37:26.756 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jUBQH6rTXdso4Gg+cRDcozbaB1g=', 'Content-Length': '0'} 2022-09-03T05:37:26.758 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000018f360dc40eb307b-006312e816-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.802 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.804 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.804 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.806 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.807 DEBUG:boto:Method: HEAD 2022-09-03T05:37:26.809 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.809 DEBUG:boto:Data: 2022-09-03T05:37:26.810 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.811 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:26.812 DEBUG:boto:Port: 8001 2022-09-03T05:37:26.813 DEBUG:boto:Params: {} 2022-09-03T05:37:26.814 DEBUG:boto:Token: None 2022-09-03T05:37:26.815 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.817 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qWlgGdkcoZVjs4dwcPyPR1XAa8I= 2022-09-03T05:37:26.818 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qWlgGdkcoZVjs4dwcPyPR1XAa8I=', 'Content-Length': '0'} 2022-09-03T05:37:26.822 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b1397db925f6c573-006312e816-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.823 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.824 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.825 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.826 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.827 DEBUG:boto:Method: HEAD 2022-09-03T05:37:26.828 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.829 DEBUG:boto:Data: 2022-09-03T05:37:26.829 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.830 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:26.830 DEBUG:boto:Port: 8000 2022-09-03T05:37:26.831 DEBUG:boto:Params: {} 2022-09-03T05:37:26.832 DEBUG:boto:Token: None 2022-09-03T05:37:26.833 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.833 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qWlgGdkcoZVjs4dwcPyPR1XAa8I= 2022-09-03T05:37:26.834 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qWlgGdkcoZVjs4dwcPyPR1XAa8I=', 'Content-Length': '0'} 2022-09-03T05:37:26.836 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000487ecd5257b9a0a8-006312e816-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.837 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:26.838 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.838 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.839 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.840 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.840 DEBUG:boto:Method: GET 2022-09-03T05:37:26.841 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.841 DEBUG:boto:Data: 2022-09-03T05:37:26.842 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.843 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:26.843 DEBUG:boto:Port: 8001 2022-09-03T05:37:26.844 DEBUG:boto:Params: {} 2022-09-03T05:37:26.844 DEBUG:boto:Token: None 2022-09-03T05:37:26.845 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.846 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jUBQH6rTXdso4Gg+cRDcozbaB1g= 2022-09-03T05:37:26.847 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jUBQH6rTXdso4Gg+cRDcozbaB1g=', 'Content-Length': '0'} 2022-09-03T05:37:26.850 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000058e058607e144f27-006312e816-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.851 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.852 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.854 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.855 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.857 DEBUG:boto:Method: GET 2022-09-03T05:37:26.858 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.859 DEBUG:boto:Data: 2022-09-03T05:37:26.861 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.861 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:26.863 DEBUG:boto:Port: 8000 2022-09-03T05:37:26.863 DEBUG:boto:Params: {} 2022-09-03T05:37:26.864 DEBUG:boto:Token: None 2022-09-03T05:37:26.865 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:26.865 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jUBQH6rTXdso4Gg+cRDcozbaB1g= 2022-09-03T05:37:26.866 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jUBQH6rTXdso4Gg+cRDcozbaB1g=', 'Content-Length': '0'} 2022-09-03T05:37:26.868 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dd43523b688dab25-006312e816-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.869 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:26.870 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:26.870 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.871 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.872 DEBUG:boto:Method: GET 2022-09-03T05:37:26.872 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.873 DEBUG:boto:Data: 2022-09-03T05:37:26.874 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.874 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:26.875 DEBUG:boto:Port: 8001 2022-09-03T05:37:26.875 DEBUG:boto:Params: {} 2022-09-03T05:37:26.876 DEBUG:boto:Token: None 2022-09-03T05:37:26.876 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/ 2022-09-03T05:37:26.877 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw= 2022-09-03T05:37:26.878 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw=', 'Content-Length': '0'} 2022-09-03T05:37:26.881 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002155f0d49d15006a-006312e816-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.882 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:26.883 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:26.883 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:26.884 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.884 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.886 DEBUG:boto:Method: GET 2022-09-03T05:37:26.886 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.887 DEBUG:boto:Data: 2022-09-03T05:37:26.887 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.888 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:26.889 DEBUG:boto:Port: 8000 2022-09-03T05:37:26.889 DEBUG:boto:Params: {} 2022-09-03T05:37:26.890 DEBUG:boto:Token: None 2022-09-03T05:37:26.891 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/ 2022-09-03T05:37:26.892 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw= 2022-09-03T05:37:26.893 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSQA6wbwMPrkCJ1uErL+ikc7uLw=', 'Content-Length': '0'} 2022-09-03T05:37:26.896 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003b85a10c960af51c-006312e816-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.896 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:26.897 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.898 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.899 DEBUG:boto:Method: HEAD 2022-09-03T05:37:26.899 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:26.900 DEBUG:boto:Data: 2022-09-03T05:37:26.900 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.901 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:26.901 DEBUG:boto:Port: 8001 2022-09-03T05:37:26.902 DEBUG:boto:Params: {} 2022-09-03T05:37:26.902 DEBUG:boto:Token: None 2022-09-03T05:37:26.903 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj 2022-09-03T05:37:26.903 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2QXGvMbOwXuBXbIcPQxrAkjvJgc= 2022-09-03T05:37:26.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2QXGvMbOwXuBXbIcPQxrAkjvJgc=', 'Content-Length': '0'} 2022-09-03T05:37:26.907 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000057f5ac37a425d102-006312e816-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.908 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.909 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.909 DEBUG:boto:Method: HEAD 2022-09-03T05:37:26.910 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:26.910 DEBUG:boto:Data: 2022-09-03T05:37:26.911 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.911 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:26.912 DEBUG:boto:Port: 8000 2022-09-03T05:37:26.912 DEBUG:boto:Params: {} 2022-09-03T05:37:26.913 DEBUG:boto:Token: None 2022-09-03T05:37:26.914 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj 2022-09-03T05:37:26.915 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2QXGvMbOwXuBXbIcPQxrAkjvJgc= 2022-09-03T05:37:26.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2QXGvMbOwXuBXbIcPQxrAkjvJgc=', 'Content-Length': '0'} 2022-09-03T05:37:26.920 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a9bf536df902231a-006312e816-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.921 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:26.921 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.922 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.922 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.923 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.924 DEBUG:boto:Method: GET 2022-09-03T05:37:26.924 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.925 DEBUG:boto:Data: 2022-09-03T05:37:26.925 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.926 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:26.927 DEBUG:boto:Port: 8001 2022-09-03T05:37:26.927 DEBUG:boto:Params: {} 2022-09-03T05:37:26.928 DEBUG:boto:Token: None 2022-09-03T05:37:26.929 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.929 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5+25tEUuBdxcBdrxSvFjALRwDIs= 2022-09-03T05:37:26.930 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5+25tEUuBdxcBdrxSvFjALRwDIs=', 'Content-Length': '0'} 2022-09-03T05:37:26.933 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ef592b0050cb774a-006312e816-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.938 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:26.938 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:26.939 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.939 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.940 DEBUG:boto:Method: GET 2022-09-03T05:37:26.941 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.941 DEBUG:boto:Data: 2022-09-03T05:37:26.942 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.942 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:26.943 DEBUG:boto:Port: 8000 2022-09-03T05:37:26.943 DEBUG:boto:Params: {} 2022-09-03T05:37:26.944 DEBUG:boto:Token: None 2022-09-03T05:37:26.945 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:26 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:26.946 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5+25tEUuBdxcBdrxSvFjALRwDIs= 2022-09-03T05:37:26.946 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5+25tEUuBdxcBdrxSvFjALRwDIs=', 'Content-Length': '0'} 2022-09-03T05:37:26.949 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000acfdfad678c90423-006312e816-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:26.994 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:26.994 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:26.995 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.995 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.996 DEBUG:boto:Method: GET 2022-09-03T05:37:26.997 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:26.997 DEBUG:boto:Data: 2022-09-03T05:37:26.998 DEBUG:boto:Headers: {} 2022-09-03T05:37:26.998 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:26.999 DEBUG:boto:Port: 8001 2022-09-03T05:37:26.999 DEBUG:boto:Params: {} 2022-09-03T05:37:27.000 DEBUG:boto:Token: None 2022-09-03T05:37:27.000 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/ 2022-09-03T05:37:27.001 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk= 2022-09-03T05:37:27.002 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk=', 'Content-Length': '0'} 2022-09-03T05:37:27.005 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000018c95a273356a0fe-006312e817-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.006 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:27.007 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:27.007 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:27.008 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:27.008 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:27.009 DEBUG:boto:Method: GET 2022-09-03T05:37:27.009 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:27.010 DEBUG:boto:Data: 2022-09-03T05:37:27.010 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.011 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:27.011 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.012 DEBUG:boto:Params: {} 2022-09-03T05:37:27.013 DEBUG:boto:Token: None 2022-09-03T05:37:27.013 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/ 2022-09-03T05:37:27.014 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk= 2022-09-03T05:37:27.015 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk=', 'Content-Length': '0'} 2022-09-03T05:37:27.018 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002cb7192c1ecf7be1-006312e817-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.019 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:27.021 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-19 zones={test-zone3, test-zone1} 2022-09-03T05:37:27.022 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-19 zones={test-zone3, test-zone2} 2022-09-03T05:37:27.023 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:27.024 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:27.025 DEBUG:boto:Method: HEAD 2022-09-03T05:37:27.025 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:37:27.026 DEBUG:boto:Data: 2022-09-03T05:37:27.027 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.028 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:27.029 DEBUG:boto:Port: 8001 2022-09-03T05:37:27.029 DEBUG:boto:Params: {} 2022-09-03T05:37:27.030 DEBUG:boto:Token: None 2022-09-03T05:37:27.031 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/ 2022-09-03T05:37:27.031 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bj6RuQpXXVub/0PDLl+pLsHDNYY= 2022-09-03T05:37:27.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bj6RuQpXXVub/0PDLl+pLsHDNYY=', 'Content-Length': '0'} 2022-09-03T05:37:27.035 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '31'), ('X-RGW-Bytes-Used', '240'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007d238b02dc6a8078-006312e817-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.036 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:27.036 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:27.037 DEBUG:boto:Method: HEAD 2022-09-03T05:37:27.038 DEBUG:boto:Path: /dsgvcy-19/ 2022-09-03T05:37:27.038 DEBUG:boto:Data: 2022-09-03T05:37:27.039 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.039 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:27.040 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.040 DEBUG:boto:Params: {} 2022-09-03T05:37:27.041 DEBUG:boto:Token: None 2022-09-03T05:37:27.041 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/ 2022-09-03T05:37:27.042 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bj6RuQpXXVub/0PDLl+pLsHDNYY= 2022-09-03T05:37:27.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bj6RuQpXXVub/0PDLl+pLsHDNYY=', 'Content-Length': '0'} 2022-09-03T05:37:27.046 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '31'), ('X-RGW-Bytes-Used', '240'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006af32657d6e9d362-006312e817-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.047 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:37:27.048 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:27.049 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:27.049 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:27.050 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:27.051 DEBUG:boto:Method: GET 2022-09-03T05:37:27.051 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:27.052 DEBUG:boto:Data: 2022-09-03T05:37:27.052 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.053 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:27.053 DEBUG:boto:Port: 8001 2022-09-03T05:37:27.054 DEBUG:boto:Params: {} 2022-09-03T05:37:27.055 DEBUG:boto:Token: None 2022-09-03T05:37:27.055 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/?versions 2022-09-03T05:37:27.056 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nuubkOlMdRyjQfXs0HWlbFZ7yHc= 2022-09-03T05:37:27.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nuubkOlMdRyjQfXs0HWlbFZ7yHc=', 'Content-Length': '0'} 2022-09-03T05:37:27.060 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005d6dd50235c140a7-006312e817-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.061 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjFV.ndVyKCalSXA6cZK3aQWN-LzBZK3Rtrue2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjhqLZJOZHmRi2EtNletjE-T3uOA2T8pTfalse2022-09-03T05:34:51.324Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjmBYgYYD3drKvdzws.hh3-36xKvL9eTHfalse2022-09-03T05:34:51.365Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjMnuLKlk.Rq8n.YuO6g0DugPfRiHStyffalse2022-09-03T05:34:51.173Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjN.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1ifalse2022-09-03T05:34:51.242Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjgFHndXVQlMCeVf-4Dc4FENAmsrgKp-Dfalse2022-09-03T05:34:51.206Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj0PhwqB536OEXdH1QOmPxhlUfX3fwPvSfalse2022-09-03T05:34:51.042Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjGM1mLEe4fDwiUzp2k3PtPzJLy-tnWKifalse2022-09-03T05:34:51.129Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjKl9OW7HSpMTjjnmOMkqEktWm9H3uMr2false2022-09-03T05:34:50.995Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjEFeprdWgJo5gbXM16hy6HNBwhGTxK7efalse2022-09-03T05:34:50.949Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjJID7ijdm1B3ZxtamFDxAcL6LJfFKoNjfalse2022-09-03T05:34:50.861Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjyL2uOIeC0l.C5YXdYoqQBvDIly3idxOfalse2022-09-03T05:34:50.905Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.lPacc7Eu6BJ5IOsSyLccq2.KRUK24Ufalse2022-09-03T05:34:50.763Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjNkMDAxRk7LkNPj1mtl1YEsdjIaTowHtfalse2022-09-03T05:34:50.824Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQfRyggjm1gZ7H.LeMCey0hHybmVWONGfalse2022-09-03T05:34:50.794Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj94bPO72IhlWQEGwklU15Pvpr-qRv.lmfalse2022-09-03T05:34:50.677Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjORmfT2fVQ5EFIFI89EGuLigbPyfeObTfalse2022-09-03T05:34:50.719Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkUiw73n7LyoUiDcS9JXEfdlCebChUCEfalse2022-09-03T05:34:50.635Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjBZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98false2022-09-03T05:34:50.517Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjIoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SBfalse2022-09-03T05:34:50.595Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjrHbEnZ16TQjAC.xp1wNbn0Ce5Vc948ifalse2022-09-03T05:34:50.555Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjG4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXLfalse2022-09-03T05:34:50.467Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQl7wbqZZa4psf4mYXa6Hq8W4qHqEHV4false2022-09-03T05:34:50.343Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjvauOlKlWVHmDxq4ZzMBF97KP66F89Pcfalse2022-09-03T05:34:50.391Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKCfalse2022-09-03T05:34:50.189Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjoYY-icmxg74gZ4carPTk38at9fTxjJifalse2022-09-03T05:34:50.232Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjxYnysJgMUeKh.Mew5S-gBJZsJgD4g.Afalse2022-09-03T05:34:50.274Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvYfalse2022-09-03T05:34:50.041Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjCFKxqazR5g.k3.oWs62bQgs9moHAimIfalse2022-09-03T05:34:50.150Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkJSzTg.tDVQ-nL1nX84.W7ifJxIseT1false2022-09-03T05:34:50.112Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:27.063 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:37:27.064 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.064 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.065 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.065 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.066 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.066 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.067 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.067 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.068 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.068 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.069 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.069 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.070 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.070 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.071 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.071 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.072 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.072 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.073 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.073 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.074 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.074 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.075 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.075 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.076 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.076 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.077 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.077 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.078 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.078 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.079 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:37:27.079 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:27.080 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:27.081 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:27.081 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:27.082 DEBUG:boto:Method: GET 2022-09-03T05:37:27.082 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:27.083 DEBUG:boto:Data: 2022-09-03T05:37:27.083 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.084 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:27.084 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.085 DEBUG:boto:Params: {} 2022-09-03T05:37:27.086 DEBUG:boto:Token: None 2022-09-03T05:37:27.086 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/?versions 2022-09-03T05:37:27.087 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nuubkOlMdRyjQfXs0HWlbFZ7yHc= 2022-09-03T05:37:27.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nuubkOlMdRyjQfXs0HWlbFZ7yHc=', 'Content-Length': '0'} 2022-09-03T05:37:27.091 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007341f4de116020a3-006312e817-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.092 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjFV.ndVyKCalSXA6cZK3aQWN-LzBZK3Rtrue2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjhqLZJOZHmRi2EtNletjE-T3uOA2T8pTfalse2022-09-03T05:34:51.324Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjmBYgYYD3drKvdzws.hh3-36xKvL9eTHfalse2022-09-03T05:34:51.365Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjMnuLKlk.Rq8n.YuO6g0DugPfRiHStyffalse2022-09-03T05:34:51.173Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjN.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1ifalse2022-09-03T05:34:51.242Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjgFHndXVQlMCeVf-4Dc4FENAmsrgKp-Dfalse2022-09-03T05:34:51.206Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj0PhwqB536OEXdH1QOmPxhlUfX3fwPvSfalse2022-09-03T05:34:51.042Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjGM1mLEe4fDwiUzp2k3PtPzJLy-tnWKifalse2022-09-03T05:34:51.129Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjKl9OW7HSpMTjjnmOMkqEktWm9H3uMr2false2022-09-03T05:34:50.995Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjEFeprdWgJo5gbXM16hy6HNBwhGTxK7efalse2022-09-03T05:34:50.949Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjJID7ijdm1B3ZxtamFDxAcL6LJfFKoNjfalse2022-09-03T05:34:50.861Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjyL2uOIeC0l.C5YXdYoqQBvDIly3idxOfalse2022-09-03T05:34:50.905Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.lPacc7Eu6BJ5IOsSyLccq2.KRUK24Ufalse2022-09-03T05:34:50.763Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjNkMDAxRk7LkNPj1mtl1YEsdjIaTowHtfalse2022-09-03T05:34:50.824Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQfRyggjm1gZ7H.LeMCey0hHybmVWONGfalse2022-09-03T05:34:50.794Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj94bPO72IhlWQEGwklU15Pvpr-qRv.lmfalse2022-09-03T05:34:50.677Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjORmfT2fVQ5EFIFI89EGuLigbPyfeObTfalse2022-09-03T05:34:50.719Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkUiw73n7LyoUiDcS9JXEfdlCebChUCEfalse2022-09-03T05:34:50.635Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjBZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98false2022-09-03T05:34:50.517Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjIoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SBfalse2022-09-03T05:34:50.595Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjrHbEnZ16TQjAC.xp1wNbn0Ce5Vc948ifalse2022-09-03T05:34:50.555Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjG4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXLfalse2022-09-03T05:34:50.467Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQl7wbqZZa4psf4mYXa6Hq8W4qHqEHV4false2022-09-03T05:34:50.343Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjvauOlKlWVHmDxq4ZzMBF97KP66F89Pcfalse2022-09-03T05:34:50.391Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKCfalse2022-09-03T05:34:50.189Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjoYY-icmxg74gZ4carPTk38at9fTxjJifalse2022-09-03T05:34:50.232Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjxYnysJgMUeKh.Mew5S-gBJZsJgD4g.Afalse2022-09-03T05:34:50.274Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvYfalse2022-09-03T05:34:50.041Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjCFKxqazR5g.k3.oWs62bQgs9moHAimIfalse2022-09-03T05:34:50.150Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkJSzTg.tDVQ-nL1nX84.W7ifJxIseT1false2022-09-03T05:34:50.112Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:27.103 DEBUG:tasks.rgw_multi.zone_rados:o=dummy 2022-09-03T05:37:27.103 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.104 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.104 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.105 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.105 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.106 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.106 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.107 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.108 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.108 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.109 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.109 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.110 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.110 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.111 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.111 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.112 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.112 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.113 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.113 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.114 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.114 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.115 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.115 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.115 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.116 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.116 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.117 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.117 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.118 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:37:27.119 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:27.119 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:27.120 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:27.120 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:27.121 DEBUG:boto:Method: GET 2022-09-03T05:37:27.121 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:27.122 DEBUG:boto:Data: 2022-09-03T05:37:27.123 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.123 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:27.124 DEBUG:boto:Port: 8001 2022-09-03T05:37:27.124 DEBUG:boto:Params: {} 2022-09-03T05:37:27.125 DEBUG:boto:Token: None 2022-09-03T05:37:27.125 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/?versions 2022-09-03T05:37:27.126 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nuubkOlMdRyjQfXs0HWlbFZ7yHc= 2022-09-03T05:37:27.127 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nuubkOlMdRyjQfXs0HWlbFZ7yHc=', 'Content-Length': '0'} 2022-09-03T05:37:27.130 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b6d6d91d4b37a3c3-006312e817-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.131 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjFV.ndVyKCalSXA6cZK3aQWN-LzBZK3Rtrue2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjhqLZJOZHmRi2EtNletjE-T3uOA2T8pTfalse2022-09-03T05:34:51.324Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjmBYgYYD3drKvdzws.hh3-36xKvL9eTHfalse2022-09-03T05:34:51.365Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjMnuLKlk.Rq8n.YuO6g0DugPfRiHStyffalse2022-09-03T05:34:51.173Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjN.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1ifalse2022-09-03T05:34:51.242Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjgFHndXVQlMCeVf-4Dc4FENAmsrgKp-Dfalse2022-09-03T05:34:51.206Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj0PhwqB536OEXdH1QOmPxhlUfX3fwPvSfalse2022-09-03T05:34:51.042Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjGM1mLEe4fDwiUzp2k3PtPzJLy-tnWKifalse2022-09-03T05:34:51.129Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjKl9OW7HSpMTjjnmOMkqEktWm9H3uMr2false2022-09-03T05:34:50.995Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjEFeprdWgJo5gbXM16hy6HNBwhGTxK7efalse2022-09-03T05:34:50.949Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjJID7ijdm1B3ZxtamFDxAcL6LJfFKoNjfalse2022-09-03T05:34:50.861Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjyL2uOIeC0l.C5YXdYoqQBvDIly3idxOfalse2022-09-03T05:34:50.905Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.lPacc7Eu6BJ5IOsSyLccq2.KRUK24Ufalse2022-09-03T05:34:50.763Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjNkMDAxRk7LkNPj1mtl1YEsdjIaTowHtfalse2022-09-03T05:34:50.824Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQfRyggjm1gZ7H.LeMCey0hHybmVWONGfalse2022-09-03T05:34:50.794Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj94bPO72IhlWQEGwklU15Pvpr-qRv.lmfalse2022-09-03T05:34:50.677Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjORmfT2fVQ5EFIFI89EGuLigbPyfeObTfalse2022-09-03T05:34:50.719Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkUiw73n7LyoUiDcS9JXEfdlCebChUCEfalse2022-09-03T05:34:50.635Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjBZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98false2022-09-03T05:34:50.517Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjIoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SBfalse2022-09-03T05:34:50.595Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjrHbEnZ16TQjAC.xp1wNbn0Ce5Vc948ifalse2022-09-03T05:34:50.555Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjG4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXLfalse2022-09-03T05:34:50.467Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQl7wbqZZa4psf4mYXa6Hq8W4qHqEHV4false2022-09-03T05:34:50.343Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjvauOlKlWVHmDxq4ZzMBF97KP66F89Pcfalse2022-09-03T05:34:50.391Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKCfalse2022-09-03T05:34:50.189Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjoYY-icmxg74gZ4carPTk38at9fTxjJifalse2022-09-03T05:34:50.232Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjxYnysJgMUeKh.Mew5S-gBJZsJgD4g.Afalse2022-09-03T05:34:50.274Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvYfalse2022-09-03T05:34:50.041Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjCFKxqazR5g.k3.oWs62bQgs9moHAimIfalse2022-09-03T05:34:50.150Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkJSzTg.tDVQ-nL1nX84.W7ifJxIseT1false2022-09-03T05:34:50.112Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:27.134 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:27.134 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:27.135 DEBUG:boto:path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:27.135 DEBUG:boto:auth_path=/dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:27.136 DEBUG:boto:Method: GET 2022-09-03T05:37:27.138 DEBUG:boto:Path: /dsgvcy-19/?versions&max-keys=999 2022-09-03T05:37:27.138 DEBUG:boto:Data: 2022-09-03T05:37:27.139 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.139 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:27.140 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.140 DEBUG:boto:Params: {} 2022-09-03T05:37:27.141 DEBUG:boto:Token: None 2022-09-03T05:37:27.142 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/?versions 2022-09-03T05:37:27.142 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nuubkOlMdRyjQfXs0HWlbFZ7yHc= 2022-09-03T05:37:27.143 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nuubkOlMdRyjQfXs0HWlbFZ7yHc=', 'Content-Length': '0'} 2022-09-03T05:37:27.146 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000372e00847d91f7f-006312e817-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.147 DEBUG:boto:b'dsgvcy-19999falsedummyLD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3true2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjFV.ndVyKCalSXA6cZK3aQWN-LzBZK3Rtrue2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjhqLZJOZHmRi2EtNletjE-T3uOA2T8pTfalse2022-09-03T05:34:51.324Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjmBYgYYD3drKvdzws.hh3-36xKvL9eTHfalse2022-09-03T05:34:51.365Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjMnuLKlk.Rq8n.YuO6g0DugPfRiHStyffalse2022-09-03T05:34:51.173Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjN.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1ifalse2022-09-03T05:34:51.242Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjgFHndXVQlMCeVf-4Dc4FENAmsrgKp-Dfalse2022-09-03T05:34:51.206Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj0PhwqB536OEXdH1QOmPxhlUfX3fwPvSfalse2022-09-03T05:34:51.042Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjGM1mLEe4fDwiUzp2k3PtPzJLy-tnWKifalse2022-09-03T05:34:51.129Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjKl9OW7HSpMTjjnmOMkqEktWm9H3uMr2false2022-09-03T05:34:50.995Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjEFeprdWgJo5gbXM16hy6HNBwhGTxK7efalse2022-09-03T05:34:50.949Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjJID7ijdm1B3ZxtamFDxAcL6LJfFKoNjfalse2022-09-03T05:34:50.861Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjyL2uOIeC0l.C5YXdYoqQBvDIly3idxOfalse2022-09-03T05:34:50.905Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.lPacc7Eu6BJ5IOsSyLccq2.KRUK24Ufalse2022-09-03T05:34:50.763Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjNkMDAxRk7LkNPj1mtl1YEsdjIaTowHtfalse2022-09-03T05:34:50.824Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQfRyggjm1gZ7H.LeMCey0hHybmVWONGfalse2022-09-03T05:34:50.794Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj94bPO72IhlWQEGwklU15Pvpr-qRv.lmfalse2022-09-03T05:34:50.677Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjORmfT2fVQ5EFIFI89EGuLigbPyfeObTfalse2022-09-03T05:34:50.719Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkUiw73n7LyoUiDcS9JXEfdlCebChUCEfalse2022-09-03T05:34:50.635Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjBZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98false2022-09-03T05:34:50.517Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjIoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SBfalse2022-09-03T05:34:50.595Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjrHbEnZ16TQjAC.xp1wNbn0Ce5Vc948ifalse2022-09-03T05:34:50.555Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjG4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXLfalse2022-09-03T05:34:50.467Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjQl7wbqZZa4psf4mYXa6Hq8W4qHqEHV4false2022-09-03T05:34:50.343Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjvauOlKlWVHmDxq4ZzMBF97KP66F89Pcfalse2022-09-03T05:34:50.391Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKCfalse2022-09-03T05:34:50.189Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjoYY-icmxg74gZ4carPTk38at9fTxjJifalse2022-09-03T05:34:50.232Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjxYnysJgMUeKh.Mew5S-gBJZsJgD4g.Afalse2022-09-03T05:34:50.274Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobj4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvYfalse2022-09-03T05:34:50.041Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjCFKxqazR5g.k3.oWs62bQgs9moHAimIfalse2022-09-03T05:34:50.150Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormalobjkJSzTg.tDVQ-nL1nX84.W7ifJxIseT1false2022-09-03T05:34:50.112Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:27.149 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:37:27.150 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:27.151 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:27.151 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.152 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.152 DEBUG:boto:Method: GET 2022-09-03T05:37:27.153 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.153 DEBUG:boto:Data: 2022-09-03T05:37:27.154 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.155 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:27.155 DEBUG:boto:Port: 8001 2022-09-03T05:37:27.156 DEBUG:boto:Params: {} 2022-09-03T05:37:27.157 DEBUG:boto:Token: None 2022-09-03T05:37:27.157 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.158 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XpYPi32nBMHRICNCegH68wvZqlc= 2022-09-03T05:37:27.159 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XpYPi32nBMHRICNCegH68wvZqlc=', 'Content-Length': '0'} 2022-09-03T05:37:27.161 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000fed21fbf2f00035f-006312e817-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.162 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:27.163 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:27.165 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.166 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.166 DEBUG:boto:Method: GET 2022-09-03T05:37:27.167 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.168 DEBUG:boto:Data: 2022-09-03T05:37:27.169 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.170 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:27.171 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.172 DEBUG:boto:Params: {} 2022-09-03T05:37:27.172 DEBUG:boto:Token: None 2022-09-03T05:37:27.173 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.174 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XpYPi32nBMHRICNCegH68wvZqlc= 2022-09-03T05:37:27.175 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XpYPi32nBMHRICNCegH68wvZqlc=', 'Content-Length': '0'} 2022-09-03T05:37:27.181 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f00ee73f04a7dd95-006312e817-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.182 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:27.184 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:27.185 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.185 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.186 DEBUG:boto:Method: HEAD 2022-09-03T05:37:27.187 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.188 DEBUG:boto:Data: 2022-09-03T05:37:27.189 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.189 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:27.190 DEBUG:boto:Port: 8001 2022-09-03T05:37:27.190 DEBUG:boto:Params: {} 2022-09-03T05:37:27.191 DEBUG:boto:Token: None 2022-09-03T05:37:27.192 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.192 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:g5Ftrw+7d4SmlBgujOdbCn2Urbo= 2022-09-03T05:37:27.194 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:g5Ftrw+7d4SmlBgujOdbCn2Urbo=', 'Content-Length': '0'} 2022-09-03T05:37:27.196 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000068e7ab09df7f42bd-006312e817-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.197 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:27.198 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:27.199 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.199 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.200 DEBUG:boto:Method: HEAD 2022-09-03T05:37:27.200 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.201 DEBUG:boto:Data: 2022-09-03T05:37:27.201 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.202 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:27.203 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.203 DEBUG:boto:Params: {} 2022-09-03T05:37:27.204 DEBUG:boto:Token: None 2022-09-03T05:37:27.205 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.206 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:g5Ftrw+7d4SmlBgujOdbCn2Urbo= 2022-09-03T05:37:27.206 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:g5Ftrw+7d4SmlBgujOdbCn2Urbo=', 'Content-Length': '0'} 2022-09-03T05:37:27.209 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000001a4ac7c7ddd9746e-006312e817-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.210 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:37:27.211 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:27.212 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:27.213 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.214 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.215 DEBUG:boto:Method: GET 2022-09-03T05:37:27.216 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.216 DEBUG:boto:Data: 2022-09-03T05:37:27.217 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.217 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:27.218 DEBUG:boto:Port: 8001 2022-09-03T05:37:27.219 DEBUG:boto:Params: {} 2022-09-03T05:37:27.220 DEBUG:boto:Token: None 2022-09-03T05:37:27.221 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.221 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XpYPi32nBMHRICNCegH68wvZqlc= 2022-09-03T05:37:27.222 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XpYPi32nBMHRICNCegH68wvZqlc=', 'Content-Length': '0'} 2022-09-03T05:37:27.224 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000abad7e04e7a5b9bb-006312e817-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.225 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:27.226 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:27.227 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.227 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.228 DEBUG:boto:Method: GET 2022-09-03T05:37:27.229 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.230 DEBUG:boto:Data: 2022-09-03T05:37:27.231 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.232 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:27.233 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.234 DEBUG:boto:Params: {} 2022-09-03T05:37:27.235 DEBUG:boto:Token: None 2022-09-03T05:37:27.235 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.236 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XpYPi32nBMHRICNCegH68wvZqlc= 2022-09-03T05:37:27.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XpYPi32nBMHRICNCegH68wvZqlc=', 'Content-Length': '0'} 2022-09-03T05:37:27.239 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000039ee024189f4ea26-006312e817-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.239 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:27.240 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:27.241 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:27.241 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:27.242 DEBUG:boto:Method: GET 2022-09-03T05:37:27.243 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:37:27.243 DEBUG:boto:Data: 2022-09-03T05:37:27.244 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.244 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:27.245 DEBUG:boto:Port: 8001 2022-09-03T05:37:27.245 DEBUG:boto:Params: {} 2022-09-03T05:37:27.246 DEBUG:boto:Token: None 2022-09-03T05:37:27.247 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/ 2022-09-03T05:37:27.248 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk= 2022-09-03T05:37:27.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk=', 'Content-Length': '0'} 2022-09-03T05:37:27.251 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a8122defe2c5d0f6-006312e817-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.252 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:27.253 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:27.254 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:27.255 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:27.255 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:27.256 DEBUG:boto:Method: GET 2022-09-03T05:37:27.256 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:37:27.257 DEBUG:boto:Data: 2022-09-03T05:37:27.258 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.259 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:27.260 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.261 DEBUG:boto:Params: {} 2022-09-03T05:37:27.262 DEBUG:boto:Token: None 2022-09-03T05:37:27.263 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/ 2022-09-03T05:37:27.264 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk= 2022-09-03T05:37:27.265 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk=', 'Content-Length': '0'} 2022-09-03T05:37:27.268 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bb74eca084e5555d-006312e817-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.314 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:27.316 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:27.317 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:27.318 DEBUG:boto:Method: HEAD 2022-09-03T05:37:27.320 DEBUG:boto:Path: /dsgvcy-19/dummy 2022-09-03T05:37:27.321 DEBUG:boto:Data: 2022-09-03T05:37:27.323 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.324 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:27.325 DEBUG:boto:Port: 8001 2022-09-03T05:37:27.327 DEBUG:boto:Params: {} 2022-09-03T05:37:27.328 DEBUG:boto:Token: None 2022-09-03T05:37:27.329 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/dummy 2022-09-03T05:37:27.330 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Xvca3V+iNOZMp7cmLqKvJMrIes4= 2022-09-03T05:37:27.331 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Xvca3V+iNOZMp7cmLqKvJMrIes4=', 'Content-Length': '0'} 2022-09-03T05:37:27.334 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000539e6968bbbc8fb8-006312e817-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.335 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:27.336 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:27.337 DEBUG:boto:Method: HEAD 2022-09-03T05:37:27.337 DEBUG:boto:Path: /dsgvcy-19/dummy 2022-09-03T05:37:27.338 DEBUG:boto:Data: 2022-09-03T05:37:27.338 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.339 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:27.339 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.340 DEBUG:boto:Params: {} 2022-09-03T05:37:27.340 DEBUG:boto:Token: None 2022-09-03T05:37:27.341 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/dummy 2022-09-03T05:37:27.341 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Xvca3V+iNOZMp7cmLqKvJMrIes4= 2022-09-03T05:37:27.342 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Xvca3V+iNOZMp7cmLqKvJMrIes4=', 'Content-Length': '0'} 2022-09-03T05:37:27.345 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000174faa8201482923-006312e817-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.345 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=dummy 2022-09-03T05:37:27.346 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:27.347 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:27.347 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.348 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.348 DEBUG:boto:Method: GET 2022-09-03T05:37:27.349 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.349 DEBUG:boto:Data: 2022-09-03T05:37:27.350 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.350 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:27.351 DEBUG:boto:Port: 8001 2022-09-03T05:37:27.351 DEBUG:boto:Params: {} 2022-09-03T05:37:27.352 DEBUG:boto:Token: None 2022-09-03T05:37:27.353 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.353 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XpYPi32nBMHRICNCegH68wvZqlc= 2022-09-03T05:37:27.354 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XpYPi32nBMHRICNCegH68wvZqlc=', 'Content-Length': '0'} 2022-09-03T05:37:27.356 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000002b37f5da20fc4ea4-006312e817-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.357 DEBUG:boto:path=/dsgvcy-19/dummy 2022-09-03T05:37:27.358 DEBUG:boto:auth_path=/dsgvcy-19/dummy 2022-09-03T05:37:27.367 DEBUG:boto:path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.368 DEBUG:boto:auth_path=/dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.368 DEBUG:boto:Method: GET 2022-09-03T05:37:27.369 DEBUG:boto:Path: /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.369 DEBUG:boto:Data: 2022-09-03T05:37:27.370 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.371 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:27.371 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.372 DEBUG:boto:Params: {} 2022-09-03T05:37:27.372 DEBUG:boto:Token: None 2022-09-03T05:37:27.373 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/dummy?versionId=LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3 2022-09-03T05:37:27.374 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XpYPi32nBMHRICNCegH68wvZqlc= 2022-09-03T05:37:27.374 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XpYPi32nBMHRICNCegH68wvZqlc=', 'Content-Length': '0'} 2022-09-03T05:37:27.377 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:02 GMT'), ('x-amz-version-id', 'LD8GHmfwb0Q5CvFo-7oH1to0ZrM21A3'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000dde5f6105167065e-006312e817-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.378 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:27.379 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:27.380 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:27.381 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:27.382 DEBUG:boto:Method: GET 2022-09-03T05:37:27.383 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:37:27.384 DEBUG:boto:Data: 2022-09-03T05:37:27.385 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.386 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:27.387 DEBUG:boto:Port: 8001 2022-09-03T05:37:27.388 DEBUG:boto:Params: {} 2022-09-03T05:37:27.389 DEBUG:boto:Token: None 2022-09-03T05:37:27.390 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/ 2022-09-03T05:37:27.390 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk= 2022-09-03T05:37:27.391 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk=', 'Content-Length': '0'} 2022-09-03T05:37:27.395 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c22501510e622c0b-006312e817-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.396 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:27.397 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:27.398 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:27.399 DEBUG:boto:path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:27.399 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=dummy 2022-09-03T05:37:27.400 DEBUG:boto:Method: GET 2022-09-03T05:37:27.401 DEBUG:boto:Path: /dsgvcy-19/?prefix=dummy 2022-09-03T05:37:27.402 DEBUG:boto:Data: 2022-09-03T05:37:27.402 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.403 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:27.403 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.404 DEBUG:boto:Params: {} 2022-09-03T05:37:27.405 DEBUG:boto:Token: None 2022-09-03T05:37:27.405 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/ 2022-09-03T05:37:27.406 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk= 2022-09-03T05:37:27.407 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk=', 'Content-Length': '0'} 2022-09-03T05:37:27.410 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b71b1c83cd70932a-006312e817-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.411 DEBUG:boto:b'dsgvcy-19dummy1000falsedummy2022-09-03T05:34:02.835Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:27.413 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:27.414 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:27.414 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:27.415 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.416 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.417 DEBUG:boto:Method: GET 2022-09-03T05:37:27.418 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.419 DEBUG:boto:Data: 2022-09-03T05:37:27.420 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.421 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:27.422 DEBUG:boto:Port: 8001 2022-09-03T05:37:27.423 DEBUG:boto:Params: {} 2022-09-03T05:37:27.424 DEBUG:boto:Token: None 2022-09-03T05:37:27.425 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.426 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xWIUiGwsZ8t/WavHjgr95gecTyc= 2022-09-03T05:37:27.427 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xWIUiGwsZ8t/WavHjgr95gecTyc=', 'Content-Length': '0'} 2022-09-03T05:37:27.430 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000932b4cdc32db2e17-006312e817-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.474 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:27.475 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:27.476 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.477 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.479 DEBUG:boto:Method: GET 2022-09-03T05:37:27.480 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.481 DEBUG:boto:Data: 2022-09-03T05:37:27.482 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.483 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:27.484 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.485 DEBUG:boto:Params: {} 2022-09-03T05:37:27.487 DEBUG:boto:Token: None 2022-09-03T05:37:27.488 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.489 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xWIUiGwsZ8t/WavHjgr95gecTyc= 2022-09-03T05:37:27.491 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xWIUiGwsZ8t/WavHjgr95gecTyc=', 'Content-Length': '0'} 2022-09-03T05:37:27.494 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fbb52189f2892436-006312e817-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.495 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:27.496 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:27.497 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.498 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.499 DEBUG:boto:Method: HEAD 2022-09-03T05:37:27.500 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.501 DEBUG:boto:Data: 2022-09-03T05:37:27.502 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.502 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:27.503 DEBUG:boto:Port: 8001 2022-09-03T05:37:27.504 DEBUG:boto:Params: {} 2022-09-03T05:37:27.504 DEBUG:boto:Token: None 2022-09-03T05:37:27.505 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.506 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MparQyfnad182HmXNA33jtaPSSo= 2022-09-03T05:37:27.506 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MparQyfnad182HmXNA33jtaPSSo=', 'Content-Length': '0'} 2022-09-03T05:37:27.509 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a5a7b5a8347231ab-006312e817-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.510 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:27.511 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:27.512 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.512 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.513 DEBUG:boto:Method: HEAD 2022-09-03T05:37:27.513 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.514 DEBUG:boto:Data: 2022-09-03T05:37:27.514 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.515 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:27.515 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.516 DEBUG:boto:Params: {} 2022-09-03T05:37:27.516 DEBUG:boto:Token: None 2022-09-03T05:37:27.517 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.518 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MparQyfnad182HmXNA33jtaPSSo= 2022-09-03T05:37:27.518 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MparQyfnad182HmXNA33jtaPSSo=', 'Content-Length': '0'} 2022-09-03T05:37:27.521 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000361eb9d85dfc3ce4-006312e817-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.523 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:27.523 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:27.524 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:27.524 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.525 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.525 DEBUG:boto:Method: GET 2022-09-03T05:37:27.526 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.526 DEBUG:boto:Data: 2022-09-03T05:37:27.527 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.527 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:27.528 DEBUG:boto:Port: 8001 2022-09-03T05:37:27.528 DEBUG:boto:Params: {} 2022-09-03T05:37:27.529 DEBUG:boto:Token: None 2022-09-03T05:37:27.530 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.530 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xWIUiGwsZ8t/WavHjgr95gecTyc= 2022-09-03T05:37:27.531 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xWIUiGwsZ8t/WavHjgr95gecTyc=', 'Content-Length': '0'} 2022-09-03T05:37:27.535 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008ad5c90e87b0e3a4-006312e817-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.536 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:27.537 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:27.537 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.538 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.539 DEBUG:boto:Method: GET 2022-09-03T05:37:27.540 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.540 DEBUG:boto:Data: 2022-09-03T05:37:27.541 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.543 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:27.544 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.545 DEBUG:boto:Params: {} 2022-09-03T05:37:27.546 DEBUG:boto:Token: None 2022-09-03T05:37:27.547 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.548 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xWIUiGwsZ8t/WavHjgr95gecTyc= 2022-09-03T05:37:27.549 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xWIUiGwsZ8t/WavHjgr95gecTyc=', 'Content-Length': '0'} 2022-09-03T05:37:27.552 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d292e3a108f49805-006312e817-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.553 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:27.554 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:27.555 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:27.556 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:27.557 DEBUG:boto:Method: GET 2022-09-03T05:37:27.558 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:27.559 DEBUG:boto:Data: 2022-09-03T05:37:27.560 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.560 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:27.561 DEBUG:boto:Port: 8001 2022-09-03T05:37:27.562 DEBUG:boto:Params: {} 2022-09-03T05:37:27.563 DEBUG:boto:Token: None 2022-09-03T05:37:27.564 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/ 2022-09-03T05:37:27.565 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk= 2022-09-03T05:37:27.566 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk=', 'Content-Length': '0'} 2022-09-03T05:37:27.570 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005bf6ce2860d7b622-006312e817-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.571 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:27.572 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:27.573 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:27.573 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:27.574 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:27.575 DEBUG:boto:Method: GET 2022-09-03T05:37:27.575 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:27.576 DEBUG:boto:Data: 2022-09-03T05:37:27.577 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.577 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:27.578 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.579 DEBUG:boto:Params: {} 2022-09-03T05:37:27.579 DEBUG:boto:Token: None 2022-09-03T05:37:27.580 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/ 2022-09-03T05:37:27.581 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk= 2022-09-03T05:37:27.583 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk=', 'Content-Length': '0'} 2022-09-03T05:37:27.586 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d032761b81099c7e-006312e817-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.587 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:27.588 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:27.588 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:27.589 DEBUG:boto:Method: HEAD 2022-09-03T05:37:27.589 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:27.590 DEBUG:boto:Data: 2022-09-03T05:37:27.591 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.591 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:27.592 DEBUG:boto:Port: 8001 2022-09-03T05:37:27.592 DEBUG:boto:Params: {} 2022-09-03T05:37:27.593 DEBUG:boto:Token: None 2022-09-03T05:37:27.594 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/obj 2022-09-03T05:37:27.594 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gPiIZKSezChIxdF1CesFHm9U5Yg= 2022-09-03T05:37:27.595 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gPiIZKSezChIxdF1CesFHm9U5Yg=', 'Content-Length': '0'} 2022-09-03T05:37:27.599 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d6c1a039a8f96b8a-006312e817-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.600 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:27.600 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:27.601 DEBUG:boto:Method: HEAD 2022-09-03T05:37:27.601 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:27.602 DEBUG:boto:Data: 2022-09-03T05:37:27.603 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.603 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:27.604 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.605 DEBUG:boto:Params: {} 2022-09-03T05:37:27.606 DEBUG:boto:Token: None 2022-09-03T05:37:27.606 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/obj 2022-09-03T05:37:27.607 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gPiIZKSezChIxdF1CesFHm9U5Yg= 2022-09-03T05:37:27.608 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gPiIZKSezChIxdF1CesFHm9U5Yg=', 'Content-Length': '0'} 2022-09-03T05:37:27.611 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000134ec91ee5b1556a-006312e817-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.613 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:27.614 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:27.615 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:27.616 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.616 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.617 DEBUG:boto:Method: GET 2022-09-03T05:37:27.618 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.618 DEBUG:boto:Data: 2022-09-03T05:37:27.619 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.619 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:27.620 DEBUG:boto:Port: 8001 2022-09-03T05:37:27.621 DEBUG:boto:Params: {} 2022-09-03T05:37:27.623 DEBUG:boto:Token: None 2022-09-03T05:37:27.623 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.624 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xWIUiGwsZ8t/WavHjgr95gecTyc= 2022-09-03T05:37:27.625 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xWIUiGwsZ8t/WavHjgr95gecTyc=', 'Content-Length': '0'} 2022-09-03T05:37:27.627 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a6e5ed7e67011a73-006312e817-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.674 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:27.675 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:27.676 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.677 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.678 DEBUG:boto:Method: GET 2022-09-03T05:37:27.679 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.680 DEBUG:boto:Data: 2022-09-03T05:37:27.681 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.681 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:27.682 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.683 DEBUG:boto:Params: {} 2022-09-03T05:37:27.684 DEBUG:boto:Token: None 2022-09-03T05:37:27.685 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.685 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xWIUiGwsZ8t/WavHjgr95gecTyc= 2022-09-03T05:37:27.686 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xWIUiGwsZ8t/WavHjgr95gecTyc=', 'Content-Length': '0'} 2022-09-03T05:37:27.688 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000004a826f7e48c3077-006312e817-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.689 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:27.690 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:27.690 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:27.691 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:27.692 DEBUG:boto:Method: GET 2022-09-03T05:37:27.692 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:27.693 DEBUG:boto:Data: 2022-09-03T05:37:27.694 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.695 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:27.696 DEBUG:boto:Port: 8001 2022-09-03T05:37:27.697 DEBUG:boto:Params: {} 2022-09-03T05:37:27.698 DEBUG:boto:Token: None 2022-09-03T05:37:27.699 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/ 2022-09-03T05:37:27.701 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk= 2022-09-03T05:37:27.702 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk=', 'Content-Length': '0'} 2022-09-03T05:37:27.706 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009146c385b48435a5-006312e817-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.707 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:27.709 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:27.709 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:27.710 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:27.711 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:27.712 DEBUG:boto:Method: GET 2022-09-03T05:37:27.713 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:27.713 DEBUG:boto:Data: 2022-09-03T05:37:27.715 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.716 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:27.716 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.717 DEBUG:boto:Params: {} 2022-09-03T05:37:27.717 DEBUG:boto:Token: None 2022-09-03T05:37:27.718 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/ 2022-09-03T05:37:27.719 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk= 2022-09-03T05:37:27.719 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk=', 'Content-Length': '0'} 2022-09-03T05:37:27.723 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d1354f44c0fca007-006312e817-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.724 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:27.725 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:27.726 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:27.727 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:27.727 DEBUG:boto:path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.728 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.728 DEBUG:boto:Method: GET 2022-09-03T05:37:27.729 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.729 DEBUG:boto:Data: 2022-09-03T05:37:27.730 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.731 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:27.732 DEBUG:boto:Port: 8001 2022-09-03T05:37:27.733 DEBUG:boto:Params: {} 2022-09-03T05:37:27.734 DEBUG:boto:Token: None 2022-09-03T05:37:27.734 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.735 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OXnzSNsgzlW2w+vo3tM2tuaRh6Q= 2022-09-03T05:37:27.736 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OXnzSNsgzlW2w+vo3tM2tuaRh6Q=', 'Content-Length': '0'} 2022-09-03T05:37:27.738 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'hqLZJOZHmRi2EtNletjE-T3uOA2T8pT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000955b568de9b413ca-006312e817-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.739 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:27.739 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:27.740 DEBUG:boto:path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.740 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.741 DEBUG:boto:Method: GET 2022-09-03T05:37:27.741 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.742 DEBUG:boto:Data: 2022-09-03T05:37:27.743 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.743 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:27.744 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.744 DEBUG:boto:Params: {} 2022-09-03T05:37:27.745 DEBUG:boto:Token: None 2022-09-03T05:37:27.745 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.746 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OXnzSNsgzlW2w+vo3tM2tuaRh6Q= 2022-09-03T05:37:27.747 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OXnzSNsgzlW2w+vo3tM2tuaRh6Q=', 'Content-Length': '0'} 2022-09-03T05:37:27.749 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'hqLZJOZHmRi2EtNletjE-T3uOA2T8pT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009f6a9f36a44d99f6-006312e817-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.790 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:27.792 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:27.792 DEBUG:boto:path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.793 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.793 DEBUG:boto:Method: HEAD 2022-09-03T05:37:27.794 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.794 DEBUG:boto:Data: 2022-09-03T05:37:27.795 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.795 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:27.796 DEBUG:boto:Port: 8001 2022-09-03T05:37:27.796 DEBUG:boto:Params: {} 2022-09-03T05:37:27.797 DEBUG:boto:Token: None 2022-09-03T05:37:27.798 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.799 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v05dSfj6Ea6UOQQhtWA0URJt/aw= 2022-09-03T05:37:27.799 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v05dSfj6Ea6UOQQhtWA0URJt/aw=', 'Content-Length': '0'} 2022-09-03T05:37:27.802 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'hqLZJOZHmRi2EtNletjE-T3uOA2T8pT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006012397e98b204da-006312e817-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.804 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:27.805 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:27.806 DEBUG:boto:path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.807 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.808 DEBUG:boto:Method: HEAD 2022-09-03T05:37:27.809 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.810 DEBUG:boto:Data: 2022-09-03T05:37:27.811 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.812 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:27.813 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.814 DEBUG:boto:Params: {} 2022-09-03T05:37:27.816 DEBUG:boto:Token: None 2022-09-03T05:37:27.817 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.818 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v05dSfj6Ea6UOQQhtWA0URJt/aw= 2022-09-03T05:37:27.819 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v05dSfj6Ea6UOQQhtWA0URJt/aw=', 'Content-Length': '0'} 2022-09-03T05:37:27.823 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'hqLZJOZHmRi2EtNletjE-T3uOA2T8pT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000047f8f9ebb5dd10fe-006312e817-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.824 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:27.825 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:27.825 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:27.826 DEBUG:boto:path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.827 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.827 DEBUG:boto:Method: GET 2022-09-03T05:37:27.828 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.830 DEBUG:boto:Data: 2022-09-03T05:37:27.831 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.832 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:27.833 DEBUG:boto:Port: 8001 2022-09-03T05:37:27.834 DEBUG:boto:Params: {} 2022-09-03T05:37:27.835 DEBUG:boto:Token: None 2022-09-03T05:37:27.837 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.838 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OXnzSNsgzlW2w+vo3tM2tuaRh6Q= 2022-09-03T05:37:27.839 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OXnzSNsgzlW2w+vo3tM2tuaRh6Q=', 'Content-Length': '0'} 2022-09-03T05:37:27.850 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'hqLZJOZHmRi2EtNletjE-T3uOA2T8pT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000697dbd54563c6aba-006312e817-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.852 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:27.853 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:27.854 DEBUG:boto:path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.855 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.856 DEBUG:boto:Method: GET 2022-09-03T05:37:27.857 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.858 DEBUG:boto:Data: 2022-09-03T05:37:27.860 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.862 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:27.863 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.864 DEBUG:boto:Params: {} 2022-09-03T05:37:27.865 DEBUG:boto:Token: None 2022-09-03T05:37:27.866 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/obj?versionId=hqLZJOZHmRi2EtNletjE-T3uOA2T8pT 2022-09-03T05:37:27.868 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OXnzSNsgzlW2w+vo3tM2tuaRh6Q= 2022-09-03T05:37:27.869 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OXnzSNsgzlW2w+vo3tM2tuaRh6Q=', 'Content-Length': '0'} 2022-09-03T05:37:27.873 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'hqLZJOZHmRi2EtNletjE-T3uOA2T8pT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003a09664d1611b815-006312e817-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.875 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:27.876 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:27.877 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:27.878 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:27.879 DEBUG:boto:Method: GET 2022-09-03T05:37:27.880 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:27.881 DEBUG:boto:Data: 2022-09-03T05:37:27.882 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.882 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:27.883 DEBUG:boto:Port: 8001 2022-09-03T05:37:27.883 DEBUG:boto:Params: {} 2022-09-03T05:37:27.884 DEBUG:boto:Token: None 2022-09-03T05:37:27.885 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/ 2022-09-03T05:37:27.885 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk= 2022-09-03T05:37:27.886 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk=', 'Content-Length': '0'} 2022-09-03T05:37:27.889 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000683529b453b6dd29-006312e817-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.891 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:27.892 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:27.893 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:27.893 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:27.896 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:27.897 DEBUG:boto:Method: GET 2022-09-03T05:37:27.897 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:27.898 DEBUG:boto:Data: 2022-09-03T05:37:27.899 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.899 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:27.900 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.901 DEBUG:boto:Params: {} 2022-09-03T05:37:27.902 DEBUG:boto:Token: None 2022-09-03T05:37:27.903 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/ 2022-09-03T05:37:27.904 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk= 2022-09-03T05:37:27.905 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lnsrMmjOei98C8OpWyL8ZyAt+Qk=', 'Content-Length': '0'} 2022-09-03T05:37:27.909 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ca191939432cd543-006312e817-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.910 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:27.911 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:27.913 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:27.913 DEBUG:boto:Method: HEAD 2022-09-03T05:37:27.914 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:27.915 DEBUG:boto:Data: 2022-09-03T05:37:27.915 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.916 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:27.916 DEBUG:boto:Port: 8001 2022-09-03T05:37:27.917 DEBUG:boto:Params: {} 2022-09-03T05:37:27.917 DEBUG:boto:Token: None 2022-09-03T05:37:27.918 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/obj 2022-09-03T05:37:27.919 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gPiIZKSezChIxdF1CesFHm9U5Yg= 2022-09-03T05:37:27.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gPiIZKSezChIxdF1CesFHm9U5Yg=', 'Content-Length': '0'} 2022-09-03T05:37:27.923 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b20a61f5c723f6ec-006312e817-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.924 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:27.925 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:27.925 DEBUG:boto:Method: HEAD 2022-09-03T05:37:27.926 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:27.926 DEBUG:boto:Data: 2022-09-03T05:37:27.927 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.927 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:27.928 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.928 DEBUG:boto:Params: {} 2022-09-03T05:37:27.929 DEBUG:boto:Token: None 2022-09-03T05:37:27.930 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/obj 2022-09-03T05:37:27.930 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gPiIZKSezChIxdF1CesFHm9U5Yg= 2022-09-03T05:37:27.931 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gPiIZKSezChIxdF1CesFHm9U5Yg=', 'Content-Length': '0'} 2022-09-03T05:37:27.935 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001aa0f0e50dcb2e28-006312e817-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.936 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:27.937 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:27.937 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:27.938 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.938 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.939 DEBUG:boto:Method: GET 2022-09-03T05:37:27.939 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.940 DEBUG:boto:Data: 2022-09-03T05:37:27.940 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.941 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:27.942 DEBUG:boto:Port: 8001 2022-09-03T05:37:27.943 DEBUG:boto:Params: {} 2022-09-03T05:37:27.944 DEBUG:boto:Token: None 2022-09-03T05:37:27.944 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.945 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xWIUiGwsZ8t/WavHjgr95gecTyc= 2022-09-03T05:37:27.946 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xWIUiGwsZ8t/WavHjgr95gecTyc=', 'Content-Length': '0'} 2022-09-03T05:37:27.948 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000038729078710c2987-006312e817-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:27.949 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:27.950 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:27.950 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.951 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.952 DEBUG:boto:Method: GET 2022-09-03T05:37:27.953 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.954 DEBUG:boto:Data: 2022-09-03T05:37:27.955 DEBUG:boto:Headers: {} 2022-09-03T05:37:27.956 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:27.957 DEBUG:boto:Port: 8000 2022-09-03T05:37:27.959 DEBUG:boto:Params: {} 2022-09-03T05:37:27.960 DEBUG:boto:Token: None 2022-09-03T05:37:27.961 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:27 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:27.962 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xWIUiGwsZ8t/WavHjgr95gecTyc= 2022-09-03T05:37:27.962 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xWIUiGwsZ8t/WavHjgr95gecTyc=', 'Content-Length': '0'} 2022-09-03T05:37:27.967 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000c94cccec7d12dbe-006312e817-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.006 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:28.007 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:28.007 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.008 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.008 DEBUG:boto:Method: GET 2022-09-03T05:37:28.009 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.009 DEBUG:boto:Data: 2022-09-03T05:37:28.010 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.011 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:28.011 DEBUG:boto:Port: 8001 2022-09-03T05:37:28.012 DEBUG:boto:Params: {} 2022-09-03T05:37:28.012 DEBUG:boto:Token: None 2022-09-03T05:37:28.013 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/ 2022-09-03T05:37:28.013 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U= 2022-09-03T05:37:28.014 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U=', 'Content-Length': '0'} 2022-09-03T05:37:28.018 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000032d09bbaeffe6c79-006312e818-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.018 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:28.019 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:28.020 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:28.020 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.021 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.021 DEBUG:boto:Method: GET 2022-09-03T05:37:28.022 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.023 DEBUG:boto:Data: 2022-09-03T05:37:28.023 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.024 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:28.024 DEBUG:boto:Port: 8000 2022-09-03T05:37:28.025 DEBUG:boto:Params: {} 2022-09-03T05:37:28.025 DEBUG:boto:Token: None 2022-09-03T05:37:28.026 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/ 2022-09-03T05:37:28.027 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U= 2022-09-03T05:37:28.027 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U=', 'Content-Length': '0'} 2022-09-03T05:37:28.030 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e86edf20ec3e2317-006312e818-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.031 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:28.032 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:28.033 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.033 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.034 DEBUG:boto:path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.034 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.035 DEBUG:boto:Method: GET 2022-09-03T05:37:28.035 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.036 DEBUG:boto:Data: 2022-09-03T05:37:28.037 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.037 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:28.038 DEBUG:boto:Port: 8001 2022-09-03T05:37:28.039 DEBUG:boto:Params: {} 2022-09-03T05:37:28.039 DEBUG:boto:Token: None 2022-09-03T05:37:28.040 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.040 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:voOXZwLEppk5hbpFZTj1DaBGLZw= 2022-09-03T05:37:28.041 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:voOXZwLEppk5hbpFZTj1DaBGLZw=', 'Content-Length': '0'} 2022-09-03T05:37:28.043 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'mBYgYYD3drKvdzws.hh3-36xKvL9eTH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c870163f4bcc094b-006312e818-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.044 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.045 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.045 DEBUG:boto:path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.046 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.046 DEBUG:boto:Method: GET 2022-09-03T05:37:28.047 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.047 DEBUG:boto:Data: 2022-09-03T05:37:28.048 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.049 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:28.049 DEBUG:boto:Port: 8000 2022-09-03T05:37:28.050 DEBUG:boto:Params: {} 2022-09-03T05:37:28.050 DEBUG:boto:Token: None 2022-09-03T05:37:28.051 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.051 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:voOXZwLEppk5hbpFZTj1DaBGLZw= 2022-09-03T05:37:28.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:voOXZwLEppk5hbpFZTj1DaBGLZw=', 'Content-Length': '0'} 2022-09-03T05:37:28.055 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'mBYgYYD3drKvdzws.hh3-36xKvL9eTH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f859cd2ebdf82d77-006312e818-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.056 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.056 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.057 DEBUG:boto:path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.057 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.058 DEBUG:boto:Method: HEAD 2022-09-03T05:37:28.059 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.059 DEBUG:boto:Data: 2022-09-03T05:37:28.060 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.061 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:28.062 DEBUG:boto:Port: 8001 2022-09-03T05:37:28.063 DEBUG:boto:Params: {} 2022-09-03T05:37:28.064 DEBUG:boto:Token: None 2022-09-03T05:37:28.065 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.066 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f7/qSLbNoMlg9XIFqclsIJLT1nk= 2022-09-03T05:37:28.066 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f7/qSLbNoMlg9XIFqclsIJLT1nk=', 'Content-Length': '0'} 2022-09-03T05:37:28.068 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'mBYgYYD3drKvdzws.hh3-36xKvL9eTH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004ef6db34a5889244-006312e818-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.069 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.070 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.070 DEBUG:boto:path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.071 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.071 DEBUG:boto:Method: HEAD 2022-09-03T05:37:28.072 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.072 DEBUG:boto:Data: 2022-09-03T05:37:28.073 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.073 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:28.074 DEBUG:boto:Port: 8000 2022-09-03T05:37:28.075 DEBUG:boto:Params: {} 2022-09-03T05:37:28.076 DEBUG:boto:Token: None 2022-09-03T05:37:28.077 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.078 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f7/qSLbNoMlg9XIFqclsIJLT1nk= 2022-09-03T05:37:28.080 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f7/qSLbNoMlg9XIFqclsIJLT1nk=', 'Content-Length': '0'} 2022-09-03T05:37:28.082 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'mBYgYYD3drKvdzws.hh3-36xKvL9eTH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000278942acaacc3736-006312e818-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.084 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:28.084 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.085 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.086 DEBUG:boto:path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.087 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.088 DEBUG:boto:Method: GET 2022-09-03T05:37:28.089 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.090 DEBUG:boto:Data: 2022-09-03T05:37:28.091 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.092 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:28.093 DEBUG:boto:Port: 8001 2022-09-03T05:37:28.094 DEBUG:boto:Params: {} 2022-09-03T05:37:28.095 DEBUG:boto:Token: None 2022-09-03T05:37:28.096 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.097 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:voOXZwLEppk5hbpFZTj1DaBGLZw= 2022-09-03T05:37:28.098 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:voOXZwLEppk5hbpFZTj1DaBGLZw=', 'Content-Length': '0'} 2022-09-03T05:37:28.102 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'mBYgYYD3drKvdzws.hh3-36xKvL9eTH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007bfd38fc6b5d75e9-006312e818-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.103 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.104 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.105 DEBUG:boto:path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.105 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.106 DEBUG:boto:Method: GET 2022-09-03T05:37:28.107 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.107 DEBUG:boto:Data: 2022-09-03T05:37:28.108 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.108 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:28.109 DEBUG:boto:Port: 8000 2022-09-03T05:37:28.109 DEBUG:boto:Params: {} 2022-09-03T05:37:28.110 DEBUG:boto:Token: None 2022-09-03T05:37:28.111 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=mBYgYYD3drKvdzws.hh3-36xKvL9eTH 2022-09-03T05:37:28.111 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:voOXZwLEppk5hbpFZTj1DaBGLZw= 2022-09-03T05:37:28.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:voOXZwLEppk5hbpFZTj1DaBGLZw=', 'Content-Length': '0'} 2022-09-03T05:37:28.114 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'mBYgYYD3drKvdzws.hh3-36xKvL9eTH'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000667231c90c7df06d-006312e818-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.159 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:28.161 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:28.162 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.163 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.164 DEBUG:boto:Method: GET 2022-09-03T05:37:28.165 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.165 DEBUG:boto:Data: 2022-09-03T05:37:28.167 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.168 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:28.169 DEBUG:boto:Port: 8001 2022-09-03T05:37:28.169 DEBUG:boto:Params: {} 2022-09-03T05:37:28.171 DEBUG:boto:Token: None 2022-09-03T05:37:28.172 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/ 2022-09-03T05:37:28.173 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U= 2022-09-03T05:37:28.174 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U=', 'Content-Length': '0'} 2022-09-03T05:37:28.181 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000094e734635aa6738a-006312e818-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.183 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:28.184 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:28.184 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:28.185 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.185 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.186 DEBUG:boto:Method: GET 2022-09-03T05:37:28.187 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.187 DEBUG:boto:Data: 2022-09-03T05:37:28.188 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.188 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:28.189 DEBUG:boto:Port: 8000 2022-09-03T05:37:28.189 DEBUG:boto:Params: {} 2022-09-03T05:37:28.190 DEBUG:boto:Token: None 2022-09-03T05:37:28.191 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/ 2022-09-03T05:37:28.191 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U= 2022-09-03T05:37:28.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U=', 'Content-Length': '0'} 2022-09-03T05:37:28.195 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000174bd49d09b825ed-006312e818-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.196 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:28.197 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.198 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.199 DEBUG:boto:Method: HEAD 2022-09-03T05:37:28.200 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:28.201 DEBUG:boto:Data: 2022-09-03T05:37:28.202 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.203 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:28.204 DEBUG:boto:Port: 8001 2022-09-03T05:37:28.205 DEBUG:boto:Params: {} 2022-09-03T05:37:28.206 DEBUG:boto:Token: None 2022-09-03T05:37:28.207 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj 2022-09-03T05:37:28.207 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f7rXvMzzERLvsnhpCo4YbapcDCE= 2022-09-03T05:37:28.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f7rXvMzzERLvsnhpCo4YbapcDCE=', 'Content-Length': '0'} 2022-09-03T05:37:28.221 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cd67daa284e9bda7-006312e818-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.223 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.224 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.225 DEBUG:boto:Method: HEAD 2022-09-03T05:37:28.226 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:28.227 DEBUG:boto:Data: 2022-09-03T05:37:28.228 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.230 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:28.231 DEBUG:boto:Port: 8000 2022-09-03T05:37:28.231 DEBUG:boto:Params: {} 2022-09-03T05:37:28.232 DEBUG:boto:Token: None 2022-09-03T05:37:28.233 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj 2022-09-03T05:37:28.234 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f7rXvMzzERLvsnhpCo4YbapcDCE= 2022-09-03T05:37:28.235 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f7rXvMzzERLvsnhpCo4YbapcDCE=', 'Content-Length': '0'} 2022-09-03T05:37:28.241 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000041b71f843410f228-006312e818-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.243 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:28.244 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.245 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.245 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.246 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.247 DEBUG:boto:Method: GET 2022-09-03T05:37:28.248 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.249 DEBUG:boto:Data: 2022-09-03T05:37:28.250 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.251 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:28.252 DEBUG:boto:Port: 8001 2022-09-03T05:37:28.252 DEBUG:boto:Params: {} 2022-09-03T05:37:28.254 DEBUG:boto:Token: None 2022-09-03T05:37:28.255 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.256 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MeQQYw9QjJf2RcLsPd6zt4G0nHA= 2022-09-03T05:37:28.257 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MeQQYw9QjJf2RcLsPd6zt4G0nHA=', 'Content-Length': '0'} 2022-09-03T05:37:28.260 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000efdb7f37a78ff4b5-006312e818-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.302 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.303 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.304 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.305 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.305 DEBUG:boto:Method: GET 2022-09-03T05:37:28.306 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.307 DEBUG:boto:Data: 2022-09-03T05:37:28.307 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.308 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:28.309 DEBUG:boto:Port: 8000 2022-09-03T05:37:28.309 DEBUG:boto:Params: {} 2022-09-03T05:37:28.310 DEBUG:boto:Token: None 2022-09-03T05:37:28.311 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.312 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MeQQYw9QjJf2RcLsPd6zt4G0nHA= 2022-09-03T05:37:28.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MeQQYw9QjJf2RcLsPd6zt4G0nHA=', 'Content-Length': '0'} 2022-09-03T05:37:28.315 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e61333c2d50d3bda-006312e818-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.318 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:28.319 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:28.319 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.320 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.321 DEBUG:boto:Method: GET 2022-09-03T05:37:28.321 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.322 DEBUG:boto:Data: 2022-09-03T05:37:28.323 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.323 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:28.324 DEBUG:boto:Port: 8001 2022-09-03T05:37:28.324 DEBUG:boto:Params: {} 2022-09-03T05:37:28.325 DEBUG:boto:Token: None 2022-09-03T05:37:28.326 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/ 2022-09-03T05:37:28.327 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U= 2022-09-03T05:37:28.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U=', 'Content-Length': '0'} 2022-09-03T05:37:28.334 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006518fc7f826feb4c-006312e818-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.336 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:28.337 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:28.339 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:28.340 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.341 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.342 DEBUG:boto:Method: GET 2022-09-03T05:37:28.343 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.344 DEBUG:boto:Data: 2022-09-03T05:37:28.345 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.346 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:28.347 DEBUG:boto:Port: 8000 2022-09-03T05:37:28.348 DEBUG:boto:Params: {} 2022-09-03T05:37:28.349 DEBUG:boto:Token: None 2022-09-03T05:37:28.350 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/ 2022-09-03T05:37:28.351 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U= 2022-09-03T05:37:28.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U=', 'Content-Length': '0'} 2022-09-03T05:37:28.357 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a200940a1fee9c40-006312e818-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.358 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:28.359 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:28.360 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.361 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.362 DEBUG:boto:path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.363 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.364 DEBUG:boto:Method: GET 2022-09-03T05:37:28.365 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.366 DEBUG:boto:Data: 2022-09-03T05:37:28.366 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.367 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:28.368 DEBUG:boto:Port: 8001 2022-09-03T05:37:28.368 DEBUG:boto:Params: {} 2022-09-03T05:37:28.369 DEBUG:boto:Token: None 2022-09-03T05:37:28.369 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.370 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qRWwu6MgG31dwuw5t8PEat03tuY= 2022-09-03T05:37:28.371 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qRWwu6MgG31dwuw5t8PEat03tuY=', 'Content-Length': '0'} 2022-09-03T05:37:28.373 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003c7e32241023ed66-006312e818-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.374 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.375 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.375 DEBUG:boto:path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.376 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.376 DEBUG:boto:Method: GET 2022-09-03T05:37:28.377 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.377 DEBUG:boto:Data: 2022-09-03T05:37:28.378 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.378 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:28.379 DEBUG:boto:Port: 8000 2022-09-03T05:37:28.379 DEBUG:boto:Params: {} 2022-09-03T05:37:28.380 DEBUG:boto:Token: None 2022-09-03T05:37:28.381 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.381 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qRWwu6MgG31dwuw5t8PEat03tuY= 2022-09-03T05:37:28.382 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qRWwu6MgG31dwuw5t8PEat03tuY=', 'Content-Length': '0'} 2022-09-03T05:37:28.384 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000073701fcbbeb0539f-006312e818-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.439 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.439 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.440 DEBUG:boto:path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.440 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.441 DEBUG:boto:Method: HEAD 2022-09-03T05:37:28.442 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.442 DEBUG:boto:Data: 2022-09-03T05:37:28.443 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.443 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:28.444 DEBUG:boto:Port: 8001 2022-09-03T05:37:28.444 DEBUG:boto:Params: {} 2022-09-03T05:37:28.445 DEBUG:boto:Token: None 2022-09-03T05:37:28.445 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.446 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VCN6WLRqu16bXY85szuPAbjzCdw= 2022-09-03T05:37:28.447 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VCN6WLRqu16bXY85szuPAbjzCdw=', 'Content-Length': '0'} 2022-09-03T05:37:28.449 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e7ea8dc6105d0d41-006312e818-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.451 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.452 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.453 DEBUG:boto:path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.454 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.455 DEBUG:boto:Method: HEAD 2022-09-03T05:37:28.456 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.457 DEBUG:boto:Data: 2022-09-03T05:37:28.458 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.458 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:28.459 DEBUG:boto:Port: 8000 2022-09-03T05:37:28.460 DEBUG:boto:Params: {} 2022-09-03T05:37:28.461 DEBUG:boto:Token: None 2022-09-03T05:37:28.462 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.463 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VCN6WLRqu16bXY85szuPAbjzCdw= 2022-09-03T05:37:28.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VCN6WLRqu16bXY85szuPAbjzCdw=', 'Content-Length': '0'} 2022-09-03T05:37:28.467 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008dbfa95a8d12a099-006312e818-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.468 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:28.469 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.470 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.471 DEBUG:boto:path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.472 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.472 DEBUG:boto:Method: GET 2022-09-03T05:37:28.473 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.475 DEBUG:boto:Data: 2022-09-03T05:37:28.475 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.476 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:28.476 DEBUG:boto:Port: 8001 2022-09-03T05:37:28.477 DEBUG:boto:Params: {} 2022-09-03T05:37:28.478 DEBUG:boto:Token: None 2022-09-03T05:37:28.478 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.479 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qRWwu6MgG31dwuw5t8PEat03tuY= 2022-09-03T05:37:28.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qRWwu6MgG31dwuw5t8PEat03tuY=', 'Content-Length': '0'} 2022-09-03T05:37:28.484 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000008d5a60a61b9a93f-006312e818-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.486 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.487 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.488 DEBUG:boto:path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.489 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.490 DEBUG:boto:Method: GET 2022-09-03T05:37:28.491 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.493 DEBUG:boto:Data: 2022-09-03T05:37:28.493 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.494 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:28.495 DEBUG:boto:Port: 8000 2022-09-03T05:37:28.496 DEBUG:boto:Params: {} 2022-09-03T05:37:28.496 DEBUG:boto:Token: None 2022-09-03T05:37:28.497 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf 2022-09-03T05:37:28.498 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qRWwu6MgG31dwuw5t8PEat03tuY= 2022-09-03T05:37:28.499 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qRWwu6MgG31dwuw5t8PEat03tuY=', 'Content-Length': '0'} 2022-09-03T05:37:28.514 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'MnuLKlk.Rq8n.YuO6g0DugPfRiHStyf'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000052b6864b4f7ccfc3-006312e818-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.518 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:28.519 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:28.520 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.521 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.522 DEBUG:boto:Method: GET 2022-09-03T05:37:28.527 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.529 DEBUG:boto:Data: 2022-09-03T05:37:28.530 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.530 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:28.531 DEBUG:boto:Port: 8001 2022-09-03T05:37:28.532 DEBUG:boto:Params: {} 2022-09-03T05:37:28.533 DEBUG:boto:Token: None 2022-09-03T05:37:28.534 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/ 2022-09-03T05:37:28.535 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U= 2022-09-03T05:37:28.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U=', 'Content-Length': '0'} 2022-09-03T05:37:28.540 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000291f00164001f5d5-006312e818-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.541 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:28.541 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:28.544 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:28.545 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.546 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.547 DEBUG:boto:Method: GET 2022-09-03T05:37:28.548 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.549 DEBUG:boto:Data: 2022-09-03T05:37:28.550 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.551 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:28.552 DEBUG:boto:Port: 8000 2022-09-03T05:37:28.553 DEBUG:boto:Params: {} 2022-09-03T05:37:28.554 DEBUG:boto:Token: None 2022-09-03T05:37:28.555 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/ 2022-09-03T05:37:28.556 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U= 2022-09-03T05:37:28.557 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U=', 'Content-Length': '0'} 2022-09-03T05:37:28.561 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b73ea74607b7f562-006312e818-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.562 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:28.563 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.564 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.564 DEBUG:boto:Method: HEAD 2022-09-03T05:37:28.565 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:28.566 DEBUG:boto:Data: 2022-09-03T05:37:28.566 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.567 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:28.567 DEBUG:boto:Port: 8001 2022-09-03T05:37:28.568 DEBUG:boto:Params: {} 2022-09-03T05:37:28.569 DEBUG:boto:Token: None 2022-09-03T05:37:28.570 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj 2022-09-03T05:37:28.571 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f7rXvMzzERLvsnhpCo4YbapcDCE= 2022-09-03T05:37:28.573 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f7rXvMzzERLvsnhpCo4YbapcDCE=', 'Content-Length': '0'} 2022-09-03T05:37:28.584 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e1c95814cc49ad28-006312e818-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.586 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.587 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.587 DEBUG:boto:Method: HEAD 2022-09-03T05:37:28.588 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:28.589 DEBUG:boto:Data: 2022-09-03T05:37:28.589 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.590 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:28.590 DEBUG:boto:Port: 8000 2022-09-03T05:37:28.591 DEBUG:boto:Params: {} 2022-09-03T05:37:28.592 DEBUG:boto:Token: None 2022-09-03T05:37:28.593 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj 2022-09-03T05:37:28.594 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f7rXvMzzERLvsnhpCo4YbapcDCE= 2022-09-03T05:37:28.595 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f7rXvMzzERLvsnhpCo4YbapcDCE=', 'Content-Length': '0'} 2022-09-03T05:37:28.598 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003034a25bde4cb756-006312e818-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.600 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:28.601 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.603 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.604 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.605 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.606 DEBUG:boto:Method: GET 2022-09-03T05:37:28.607 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.607 DEBUG:boto:Data: 2022-09-03T05:37:28.608 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.608 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:28.609 DEBUG:boto:Port: 8001 2022-09-03T05:37:28.610 DEBUG:boto:Params: {} 2022-09-03T05:37:28.610 DEBUG:boto:Token: None 2022-09-03T05:37:28.611 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.611 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MeQQYw9QjJf2RcLsPd6zt4G0nHA= 2022-09-03T05:37:28.612 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MeQQYw9QjJf2RcLsPd6zt4G0nHA=', 'Content-Length': '0'} 2022-09-03T05:37:28.614 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003dd69e2fb2fd0c9a-006312e818-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.659 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.660 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.661 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.662 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.662 DEBUG:boto:Method: GET 2022-09-03T05:37:28.663 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.663 DEBUG:boto:Data: 2022-09-03T05:37:28.664 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.665 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:28.666 DEBUG:boto:Port: 8000 2022-09-03T05:37:28.667 DEBUG:boto:Params: {} 2022-09-03T05:37:28.668 DEBUG:boto:Token: None 2022-09-03T05:37:28.669 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.670 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MeQQYw9QjJf2RcLsPd6zt4G0nHA= 2022-09-03T05:37:28.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MeQQYw9QjJf2RcLsPd6zt4G0nHA=', 'Content-Length': '0'} 2022-09-03T05:37:28.675 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000046ecf7964332a2b4-006312e818-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.677 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:28.678 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:28.679 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.679 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.680 DEBUG:boto:Method: GET 2022-09-03T05:37:28.681 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.682 DEBUG:boto:Data: 2022-09-03T05:37:28.683 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.684 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:28.685 DEBUG:boto:Port: 8001 2022-09-03T05:37:28.686 DEBUG:boto:Params: {} 2022-09-03T05:37:28.687 DEBUG:boto:Token: None 2022-09-03T05:37:28.688 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/ 2022-09-03T05:37:28.689 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U= 2022-09-03T05:37:28.689 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U=', 'Content-Length': '0'} 2022-09-03T05:37:28.692 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003ac0a34a8f3f1298-006312e818-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.693 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:28.694 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:28.695 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:28.696 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.697 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.698 DEBUG:boto:Method: GET 2022-09-03T05:37:28.699 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.700 DEBUG:boto:Data: 2022-09-03T05:37:28.700 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.701 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:28.702 DEBUG:boto:Port: 8000 2022-09-03T05:37:28.703 DEBUG:boto:Params: {} 2022-09-03T05:37:28.704 DEBUG:boto:Token: None 2022-09-03T05:37:28.705 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/ 2022-09-03T05:37:28.705 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U= 2022-09-03T05:37:28.706 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U=', 'Content-Length': '0'} 2022-09-03T05:37:28.709 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006c5136e45e9282e6-006312e818-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.750 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:28.752 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:28.753 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.754 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.755 DEBUG:boto:path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.755 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.756 DEBUG:boto:Method: GET 2022-09-03T05:37:28.756 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.757 DEBUG:boto:Data: 2022-09-03T05:37:28.757 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.758 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:28.759 DEBUG:boto:Port: 8001 2022-09-03T05:37:28.761 DEBUG:boto:Params: {} 2022-09-03T05:37:28.762 DEBUG:boto:Token: None 2022-09-03T05:37:28.763 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.764 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CpIOaT6d9+px7uwZp7av2/Rg1UQ= 2022-09-03T05:37:28.765 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CpIOaT6d9+px7uwZp7av2/Rg1UQ=', 'Content-Length': '0'} 2022-09-03T05:37:28.768 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008b07c339a9273545-006312e818-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.769 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.769 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.770 DEBUG:boto:path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.770 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.771 DEBUG:boto:Method: GET 2022-09-03T05:37:28.772 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.772 DEBUG:boto:Data: 2022-09-03T05:37:28.773 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.773 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:28.774 DEBUG:boto:Port: 8000 2022-09-03T05:37:28.775 DEBUG:boto:Params: {} 2022-09-03T05:37:28.775 DEBUG:boto:Token: None 2022-09-03T05:37:28.776 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.777 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CpIOaT6d9+px7uwZp7av2/Rg1UQ= 2022-09-03T05:37:28.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CpIOaT6d9+px7uwZp7av2/Rg1UQ=', 'Content-Length': '0'} 2022-09-03T05:37:28.780 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001f06e9c1019db643-006312e818-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.780 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.781 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.782 DEBUG:boto:path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.782 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.783 DEBUG:boto:Method: HEAD 2022-09-03T05:37:28.783 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.784 DEBUG:boto:Data: 2022-09-03T05:37:28.784 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.785 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:28.786 DEBUG:boto:Port: 8001 2022-09-03T05:37:28.787 DEBUG:boto:Params: {} 2022-09-03T05:37:28.787 DEBUG:boto:Token: None 2022-09-03T05:37:28.788 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.789 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NR7w+YCtCVieZjfJPEa0D/aUo/s= 2022-09-03T05:37:28.790 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NR7w+YCtCVieZjfJPEa0D/aUo/s=', 'Content-Length': '0'} 2022-09-03T05:37:28.792 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005c150ba8630dfd41-006312e818-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.793 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.793 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.794 DEBUG:boto:path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.794 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.795 DEBUG:boto:Method: HEAD 2022-09-03T05:37:28.796 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.797 DEBUG:boto:Data: 2022-09-03T05:37:28.798 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.799 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:28.800 DEBUG:boto:Port: 8000 2022-09-03T05:37:28.800 DEBUG:boto:Params: {} 2022-09-03T05:37:28.801 DEBUG:boto:Token: None 2022-09-03T05:37:28.801 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.802 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NR7w+YCtCVieZjfJPEa0D/aUo/s= 2022-09-03T05:37:28.803 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NR7w+YCtCVieZjfJPEa0D/aUo/s=', 'Content-Length': '0'} 2022-09-03T05:37:28.805 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009f12daf0a17f0b60-006312e818-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.807 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:28.808 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.808 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.809 DEBUG:boto:path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.810 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.810 DEBUG:boto:Method: GET 2022-09-03T05:37:28.811 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.812 DEBUG:boto:Data: 2022-09-03T05:37:28.812 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.813 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:28.813 DEBUG:boto:Port: 8001 2022-09-03T05:37:28.814 DEBUG:boto:Params: {} 2022-09-03T05:37:28.815 DEBUG:boto:Token: None 2022-09-03T05:37:28.816 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.816 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CpIOaT6d9+px7uwZp7av2/Rg1UQ= 2022-09-03T05:37:28.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CpIOaT6d9+px7uwZp7av2/Rg1UQ=', 'Content-Length': '0'} 2022-09-03T05:37:28.819 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a23aff02158e702c-006312e818-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.820 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.821 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.821 DEBUG:boto:path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.822 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.822 DEBUG:boto:Method: GET 2022-09-03T05:37:28.823 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.823 DEBUG:boto:Data: 2022-09-03T05:37:28.824 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.825 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:28.825 DEBUG:boto:Port: 8000 2022-09-03T05:37:28.826 DEBUG:boto:Params: {} 2022-09-03T05:37:28.826 DEBUG:boto:Token: None 2022-09-03T05:37:28.827 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i 2022-09-03T05:37:28.828 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CpIOaT6d9+px7uwZp7av2/Rg1UQ= 2022-09-03T05:37:28.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CpIOaT6d9+px7uwZp7av2/Rg1UQ=', 'Content-Length': '0'} 2022-09-03T05:37:28.830 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'N.S3SKS1.MUy4HLF8hJ6Gl6IrHYQV1i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000077dcf1c43597337e-006312e818-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.831 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:28.832 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:28.832 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.833 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.833 DEBUG:boto:Method: GET 2022-09-03T05:37:28.834 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.834 DEBUG:boto:Data: 2022-09-03T05:37:28.835 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.836 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:28.836 DEBUG:boto:Port: 8001 2022-09-03T05:37:28.837 DEBUG:boto:Params: {} 2022-09-03T05:37:28.837 DEBUG:boto:Token: None 2022-09-03T05:37:28.838 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/ 2022-09-03T05:37:28.838 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U= 2022-09-03T05:37:28.839 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U=', 'Content-Length': '0'} 2022-09-03T05:37:28.843 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003776710b24164c03-006312e818-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.843 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:28.844 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:28.845 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:28.846 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.846 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.847 DEBUG:boto:Method: GET 2022-09-03T05:37:28.848 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.848 DEBUG:boto:Data: 2022-09-03T05:37:28.849 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.849 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:28.850 DEBUG:boto:Port: 8000 2022-09-03T05:37:28.851 DEBUG:boto:Params: {} 2022-09-03T05:37:28.852 DEBUG:boto:Token: None 2022-09-03T05:37:28.852 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/ 2022-09-03T05:37:28.853 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U= 2022-09-03T05:37:28.854 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U=', 'Content-Length': '0'} 2022-09-03T05:37:28.857 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a09a421d5c6ea54e-006312e818-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.857 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:28.858 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.859 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.859 DEBUG:boto:Method: HEAD 2022-09-03T05:37:28.860 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:28.861 DEBUG:boto:Data: 2022-09-03T05:37:28.861 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.862 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:28.862 DEBUG:boto:Port: 8001 2022-09-03T05:37:28.863 DEBUG:boto:Params: {} 2022-09-03T05:37:28.864 DEBUG:boto:Token: None 2022-09-03T05:37:28.864 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj 2022-09-03T05:37:28.865 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f7rXvMzzERLvsnhpCo4YbapcDCE= 2022-09-03T05:37:28.865 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f7rXvMzzERLvsnhpCo4YbapcDCE=', 'Content-Length': '0'} 2022-09-03T05:37:28.869 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a7b883cd66cafb21-006312e818-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.869 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.870 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.870 DEBUG:boto:Method: HEAD 2022-09-03T05:37:28.871 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:28.872 DEBUG:boto:Data: 2022-09-03T05:37:28.872 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.873 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:28.874 DEBUG:boto:Port: 8000 2022-09-03T05:37:28.874 DEBUG:boto:Params: {} 2022-09-03T05:37:28.875 DEBUG:boto:Token: None 2022-09-03T05:37:28.875 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj 2022-09-03T05:37:28.876 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f7rXvMzzERLvsnhpCo4YbapcDCE= 2022-09-03T05:37:28.876 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f7rXvMzzERLvsnhpCo4YbapcDCE=', 'Content-Length': '0'} 2022-09-03T05:37:28.880 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b05796204b26c8f7-006312e818-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.881 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:28.881 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.883 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.883 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.884 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.885 DEBUG:boto:Method: GET 2022-09-03T05:37:28.886 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.887 DEBUG:boto:Data: 2022-09-03T05:37:28.888 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.889 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:28.890 DEBUG:boto:Port: 8001 2022-09-03T05:37:28.891 DEBUG:boto:Params: {} 2022-09-03T05:37:28.893 DEBUG:boto:Token: None 2022-09-03T05:37:28.894 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.895 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MeQQYw9QjJf2RcLsPd6zt4G0nHA= 2022-09-03T05:37:28.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MeQQYw9QjJf2RcLsPd6zt4G0nHA=', 'Content-Length': '0'} 2022-09-03T05:37:28.899 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000028baef636e97e5-006312e818-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.946 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.947 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.948 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.949 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.950 DEBUG:boto:Method: GET 2022-09-03T05:37:28.951 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.952 DEBUG:boto:Data: 2022-09-03T05:37:28.953 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.953 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:28.954 DEBUG:boto:Port: 8000 2022-09-03T05:37:28.956 DEBUG:boto:Params: {} 2022-09-03T05:37:28.957 DEBUG:boto:Token: None 2022-09-03T05:37:28.958 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:28.958 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MeQQYw9QjJf2RcLsPd6zt4G0nHA= 2022-09-03T05:37:28.959 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MeQQYw9QjJf2RcLsPd6zt4G0nHA=', 'Content-Length': '0'} 2022-09-03T05:37:28.961 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000039c2578224f5d1c4-006312e818-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.962 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:28.963 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:28.964 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.964 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.965 DEBUG:boto:Method: GET 2022-09-03T05:37:28.965 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.966 DEBUG:boto:Data: 2022-09-03T05:37:28.967 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.968 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:28.969 DEBUG:boto:Port: 8001 2022-09-03T05:37:28.969 DEBUG:boto:Params: {} 2022-09-03T05:37:28.970 DEBUG:boto:Token: None 2022-09-03T05:37:28.971 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/ 2022-09-03T05:37:28.972 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U= 2022-09-03T05:37:28.973 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U=', 'Content-Length': '0'} 2022-09-03T05:37:28.976 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005c177dada41bc35b-006312e818-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.977 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:28.978 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:28.978 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:28.979 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.979 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.980 DEBUG:boto:Method: GET 2022-09-03T05:37:28.980 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:28.981 DEBUG:boto:Data: 2022-09-03T05:37:28.982 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.982 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:28.983 DEBUG:boto:Port: 8000 2022-09-03T05:37:28.983 DEBUG:boto:Params: {} 2022-09-03T05:37:28.984 DEBUG:boto:Token: None 2022-09-03T05:37:28.984 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:28 GMT /dsgvcy-19/ 2022-09-03T05:37:28.985 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U= 2022-09-03T05:37:28.986 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SLGREVZ7He+AKSxre2EBWzGB73U=', 'Content-Length': '0'} 2022-09-03T05:37:28.989 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a00ed1a1d990ee8c-006312e818-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:28.990 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:28.991 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:28.991 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:28.992 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:28.993 DEBUG:boto:path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:28.993 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:28.994 DEBUG:boto:Method: GET 2022-09-03T05:37:28.995 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:28.995 DEBUG:boto:Data: 2022-09-03T05:37:28.996 DEBUG:boto:Headers: {} 2022-09-03T05:37:28.997 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:28.998 DEBUG:boto:Port: 8001 2022-09-03T05:37:28.999 DEBUG:boto:Params: {} 2022-09-03T05:37:29.000 DEBUG:boto:Token: None 2022-09-03T05:37:29.001 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:29.002 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rstp1X3Rk8GjHSvdPaGzglbNBMo= 2022-09-03T05:37:29.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rstp1X3Rk8GjHSvdPaGzglbNBMo=', 'Content-Length': '0'} 2022-09-03T05:37:29.005 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000532062c299e76f95-006312e819-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.006 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.007 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.007 DEBUG:boto:path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:29.008 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:29.008 DEBUG:boto:Method: GET 2022-09-03T05:37:29.009 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:29.009 DEBUG:boto:Data: 2022-09-03T05:37:29.010 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.010 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.011 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.011 DEBUG:boto:Params: {} 2022-09-03T05:37:29.012 DEBUG:boto:Token: None 2022-09-03T05:37:29.012 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:29.013 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rstp1X3Rk8GjHSvdPaGzglbNBMo= 2022-09-03T05:37:29.014 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rstp1X3Rk8GjHSvdPaGzglbNBMo=', 'Content-Length': '0'} 2022-09-03T05:37:29.016 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f28452f150b55934-006312e819-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.017 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.017 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.018 DEBUG:boto:path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:29.019 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:29.019 DEBUG:boto:Method: HEAD 2022-09-03T05:37:29.020 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:29.021 DEBUG:boto:Data: 2022-09-03T05:37:29.021 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.022 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:29.023 DEBUG:boto:Port: 8001 2022-09-03T05:37:29.023 DEBUG:boto:Params: {} 2022-09-03T05:37:29.024 DEBUG:boto:Token: None 2022-09-03T05:37:29.025 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:29.026 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0xlR+a0Cqfc1JRKaXN02yDIEw3c= 2022-09-03T05:37:29.026 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0xlR+a0Cqfc1JRKaXN02yDIEw3c=', 'Content-Length': '0'} 2022-09-03T05:37:29.029 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000064e4ac6b2e5647a9-006312e819-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.030 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.030 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.031 DEBUG:boto:path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:29.032 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:29.033 DEBUG:boto:Method: HEAD 2022-09-03T05:37:29.033 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:29.034 DEBUG:boto:Data: 2022-09-03T05:37:29.035 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.035 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.036 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.037 DEBUG:boto:Params: {} 2022-09-03T05:37:29.038 DEBUG:boto:Token: None 2022-09-03T05:37:29.040 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:29.041 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0xlR+a0Cqfc1JRKaXN02yDIEw3c= 2022-09-03T05:37:29.042 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0xlR+a0Cqfc1JRKaXN02yDIEw3c=', 'Content-Length': '0'} 2022-09-03T05:37:29.045 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c72099663deee639-006312e819-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.046 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:29.046 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.047 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.048 DEBUG:boto:path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:29.048 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:29.049 DEBUG:boto:Method: GET 2022-09-03T05:37:29.049 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:29.050 DEBUG:boto:Data: 2022-09-03T05:37:29.051 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.051 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:29.052 DEBUG:boto:Port: 8001 2022-09-03T05:37:29.052 DEBUG:boto:Params: {} 2022-09-03T05:37:29.053 DEBUG:boto:Token: None 2022-09-03T05:37:29.054 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:29.055 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rstp1X3Rk8GjHSvdPaGzglbNBMo= 2022-09-03T05:37:29.055 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rstp1X3Rk8GjHSvdPaGzglbNBMo=', 'Content-Length': '0'} 2022-09-03T05:37:29.058 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000471ec5c14e183c55-006312e819-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.059 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.059 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.060 DEBUG:boto:path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:29.060 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:29.061 DEBUG:boto:Method: GET 2022-09-03T05:37:29.062 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:29.062 DEBUG:boto:Data: 2022-09-03T05:37:29.063 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.063 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.064 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.065 DEBUG:boto:Params: {} 2022-09-03T05:37:29.065 DEBUG:boto:Token: None 2022-09-03T05:37:29.066 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D 2022-09-03T05:37:29.067 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rstp1X3Rk8GjHSvdPaGzglbNBMo= 2022-09-03T05:37:29.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rstp1X3Rk8GjHSvdPaGzglbNBMo=', 'Content-Length': '0'} 2022-09-03T05:37:29.070 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'gFHndXVQlMCeVf-4Dc4FENAmsrgKp-D'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006f6a21010392c7d0-006312e819-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.071 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:29.071 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:29.072 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.072 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.073 DEBUG:boto:Method: GET 2022-09-03T05:37:29.074 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.074 DEBUG:boto:Data: 2022-09-03T05:37:29.075 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.075 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:29.076 DEBUG:boto:Port: 8001 2022-09-03T05:37:29.076 DEBUG:boto:Params: {} 2022-09-03T05:37:29.077 DEBUG:boto:Token: None 2022-09-03T05:37:29.078 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/ 2022-09-03T05:37:29.078 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE= 2022-09-03T05:37:29.079 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE=', 'Content-Length': '0'} 2022-09-03T05:37:29.082 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005092e0daf386698c-006312e819-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.083 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:29.084 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:29.084 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:29.085 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.085 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.086 DEBUG:boto:Method: GET 2022-09-03T05:37:29.087 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.087 DEBUG:boto:Data: 2022-09-03T05:37:29.088 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.088 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.089 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.089 DEBUG:boto:Params: {} 2022-09-03T05:37:29.090 DEBUG:boto:Token: None 2022-09-03T05:37:29.091 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/ 2022-09-03T05:37:29.091 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE= 2022-09-03T05:37:29.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE=', 'Content-Length': '0'} 2022-09-03T05:37:29.095 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000080f591ad7089f2d3-006312e819-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.138 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:29.139 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.140 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.140 DEBUG:boto:Method: HEAD 2022-09-03T05:37:29.141 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:29.142 DEBUG:boto:Data: 2022-09-03T05:37:29.142 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.143 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:29.143 DEBUG:boto:Port: 8001 2022-09-03T05:37:29.144 DEBUG:boto:Params: {} 2022-09-03T05:37:29.144 DEBUG:boto:Token: None 2022-09-03T05:37:29.145 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj 2022-09-03T05:37:29.146 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/cZx0m0iJZNmGU56yG2A8HT2ydc= 2022-09-03T05:37:29.146 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/cZx0m0iJZNmGU56yG2A8HT2ydc=', 'Content-Length': '0'} 2022-09-03T05:37:29.150 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005d130a96a5edbab7-006312e819-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.151 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.151 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.152 DEBUG:boto:Method: HEAD 2022-09-03T05:37:29.152 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:29.153 DEBUG:boto:Data: 2022-09-03T05:37:29.154 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.155 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.155 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.156 DEBUG:boto:Params: {} 2022-09-03T05:37:29.156 DEBUG:boto:Token: None 2022-09-03T05:37:29.157 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj 2022-09-03T05:37:29.158 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/cZx0m0iJZNmGU56yG2A8HT2ydc= 2022-09-03T05:37:29.158 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/cZx0m0iJZNmGU56yG2A8HT2ydc=', 'Content-Length': '0'} 2022-09-03T05:37:29.163 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009a63a8d8203801fd-006312e819-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.165 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:29.166 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.167 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.168 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.169 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.170 DEBUG:boto:Method: GET 2022-09-03T05:37:29.172 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.172 DEBUG:boto:Data: 2022-09-03T05:37:29.173 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.174 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:29.175 DEBUG:boto:Port: 8001 2022-09-03T05:37:29.176 DEBUG:boto:Params: {} 2022-09-03T05:37:29.177 DEBUG:boto:Token: None 2022-09-03T05:37:29.178 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.179 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r+jlFfDDPGGOApygy3BPwJT7kOI= 2022-09-03T05:37:29.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r+jlFfDDPGGOApygy3BPwJT7kOI=', 'Content-Length': '0'} 2022-09-03T05:37:29.183 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009b6899ab72e19906-006312e819-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.184 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.185 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.185 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.186 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.186 DEBUG:boto:Method: GET 2022-09-03T05:37:29.187 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.188 DEBUG:boto:Data: 2022-09-03T05:37:29.188 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.189 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.190 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.191 DEBUG:boto:Params: {} 2022-09-03T05:37:29.191 DEBUG:boto:Token: None 2022-09-03T05:37:29.192 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.193 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r+jlFfDDPGGOApygy3BPwJT7kOI= 2022-09-03T05:37:29.193 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r+jlFfDDPGGOApygy3BPwJT7kOI=', 'Content-Length': '0'} 2022-09-03T05:37:29.196 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000020735483e5965d17-006312e819-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.197 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:29.198 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:29.198 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.199 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.200 DEBUG:boto:Method: GET 2022-09-03T05:37:29.201 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.201 DEBUG:boto:Data: 2022-09-03T05:37:29.202 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.202 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:29.203 DEBUG:boto:Port: 8001 2022-09-03T05:37:29.204 DEBUG:boto:Params: {} 2022-09-03T05:37:29.204 DEBUG:boto:Token: None 2022-09-03T05:37:29.205 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/ 2022-09-03T05:37:29.206 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE= 2022-09-03T05:37:29.207 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE=', 'Content-Length': '0'} 2022-09-03T05:37:29.210 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000031224180bb6c6656-006312e819-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.250 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:29.251 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:29.252 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:29.252 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.253 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.254 DEBUG:boto:Method: GET 2022-09-03T05:37:29.254 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.255 DEBUG:boto:Data: 2022-09-03T05:37:29.255 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.256 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.256 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.257 DEBUG:boto:Params: {} 2022-09-03T05:37:29.258 DEBUG:boto:Token: None 2022-09-03T05:37:29.259 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/ 2022-09-03T05:37:29.260 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE= 2022-09-03T05:37:29.261 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE=', 'Content-Length': '0'} 2022-09-03T05:37:29.264 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005cfff8b55e442af4-006312e819-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.266 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:29.267 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:29.268 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.269 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.270 DEBUG:boto:path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.271 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.272 DEBUG:boto:Method: GET 2022-09-03T05:37:29.273 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.275 DEBUG:boto:Data: 2022-09-03T05:37:29.276 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.276 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:29.277 DEBUG:boto:Port: 8001 2022-09-03T05:37:29.279 DEBUG:boto:Params: {} 2022-09-03T05:37:29.280 DEBUG:boto:Token: None 2022-09-03T05:37:29.281 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.283 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eaXSa6dJma/0W9fXrwgByudGM+s= 2022-09-03T05:37:29.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eaXSa6dJma/0W9fXrwgByudGM+s=', 'Content-Length': '0'} 2022-09-03T05:37:29.287 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', '0PhwqB536OEXdH1QOmPxhlUfX3fwPvS'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000079ef54ec37a5572e-006312e819-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.290 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.291 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.292 DEBUG:boto:path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.293 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.294 DEBUG:boto:Method: GET 2022-09-03T05:37:29.295 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.295 DEBUG:boto:Data: 2022-09-03T05:37:29.296 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.296 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.297 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.297 DEBUG:boto:Params: {} 2022-09-03T05:37:29.298 DEBUG:boto:Token: None 2022-09-03T05:37:29.299 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.300 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eaXSa6dJma/0W9fXrwgByudGM+s= 2022-09-03T05:37:29.301 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eaXSa6dJma/0W9fXrwgByudGM+s=', 'Content-Length': '0'} 2022-09-03T05:37:29.304 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', '0PhwqB536OEXdH1QOmPxhlUfX3fwPvS'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000713146086f16544e-006312e819-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.306 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.307 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.308 DEBUG:boto:path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.309 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.310 DEBUG:boto:Method: HEAD 2022-09-03T05:37:29.310 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.311 DEBUG:boto:Data: 2022-09-03T05:37:29.312 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.313 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:29.313 DEBUG:boto:Port: 8001 2022-09-03T05:37:29.314 DEBUG:boto:Params: {} 2022-09-03T05:37:29.315 DEBUG:boto:Token: None 2022-09-03T05:37:29.315 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.316 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uNAJQ87HhgA3jaHQ0AWsS86z6XU= 2022-09-03T05:37:29.317 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uNAJQ87HhgA3jaHQ0AWsS86z6XU=', 'Content-Length': '0'} 2022-09-03T05:37:29.319 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', '0PhwqB536OEXdH1QOmPxhlUfX3fwPvS'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ec8af2bdfa6529b0-006312e819-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.320 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.320 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.321 DEBUG:boto:path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.321 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.322 DEBUG:boto:Method: HEAD 2022-09-03T05:37:29.322 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.323 DEBUG:boto:Data: 2022-09-03T05:37:29.323 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.324 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.324 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.325 DEBUG:boto:Params: {} 2022-09-03T05:37:29.325 DEBUG:boto:Token: None 2022-09-03T05:37:29.326 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.327 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uNAJQ87HhgA3jaHQ0AWsS86z6XU= 2022-09-03T05:37:29.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uNAJQ87HhgA3jaHQ0AWsS86z6XU=', 'Content-Length': '0'} 2022-09-03T05:37:29.331 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', '0PhwqB536OEXdH1QOmPxhlUfX3fwPvS'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d8a70d4c1c1b2aaf-006312e819-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.331 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:29.332 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.333 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.333 DEBUG:boto:path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.334 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.335 DEBUG:boto:Method: GET 2022-09-03T05:37:29.335 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.336 DEBUG:boto:Data: 2022-09-03T05:37:29.336 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.337 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:29.337 DEBUG:boto:Port: 8001 2022-09-03T05:37:29.338 DEBUG:boto:Params: {} 2022-09-03T05:37:29.338 DEBUG:boto:Token: None 2022-09-03T05:37:29.339 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.340 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eaXSa6dJma/0W9fXrwgByudGM+s= 2022-09-03T05:37:29.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eaXSa6dJma/0W9fXrwgByudGM+s=', 'Content-Length': '0'} 2022-09-03T05:37:29.342 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', '0PhwqB536OEXdH1QOmPxhlUfX3fwPvS'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d0e55a003f5dac68-006312e819-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.388 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.394 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.395 DEBUG:boto:path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.396 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.397 DEBUG:boto:Method: GET 2022-09-03T05:37:29.398 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.399 DEBUG:boto:Data: 2022-09-03T05:37:29.399 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.400 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.400 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.401 DEBUG:boto:Params: {} 2022-09-03T05:37:29.402 DEBUG:boto:Token: None 2022-09-03T05:37:29.403 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=0PhwqB536OEXdH1QOmPxhlUfX3fwPvS 2022-09-03T05:37:29.403 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eaXSa6dJma/0W9fXrwgByudGM+s= 2022-09-03T05:37:29.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eaXSa6dJma/0W9fXrwgByudGM+s=', 'Content-Length': '0'} 2022-09-03T05:37:29.406 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', '0PhwqB536OEXdH1QOmPxhlUfX3fwPvS'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e248ffccc40350db-006312e819-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.407 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:29.408 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:29.408 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.409 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.410 DEBUG:boto:Method: GET 2022-09-03T05:37:29.411 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.411 DEBUG:boto:Data: 2022-09-03T05:37:29.413 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.414 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:29.415 DEBUG:boto:Port: 8001 2022-09-03T05:37:29.416 DEBUG:boto:Params: {} 2022-09-03T05:37:29.417 DEBUG:boto:Token: None 2022-09-03T05:37:29.418 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/ 2022-09-03T05:37:29.419 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE= 2022-09-03T05:37:29.419 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE=', 'Content-Length': '0'} 2022-09-03T05:37:29.424 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a31cbd805b49fb7f-006312e819-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.425 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:29.427 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:29.428 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:29.429 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.430 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.431 DEBUG:boto:Method: GET 2022-09-03T05:37:29.433 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.434 DEBUG:boto:Data: 2022-09-03T05:37:29.435 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.436 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.437 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.437 DEBUG:boto:Params: {} 2022-09-03T05:37:29.438 DEBUG:boto:Token: None 2022-09-03T05:37:29.439 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/ 2022-09-03T05:37:29.440 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE= 2022-09-03T05:37:29.441 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE=', 'Content-Length': '0'} 2022-09-03T05:37:29.444 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000052237f35e85cc275-006312e819-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.445 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:29.446 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.447 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.447 DEBUG:boto:Method: HEAD 2022-09-03T05:37:29.448 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:29.448 DEBUG:boto:Data: 2022-09-03T05:37:29.449 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.449 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:29.450 DEBUG:boto:Port: 8001 2022-09-03T05:37:29.450 DEBUG:boto:Params: {} 2022-09-03T05:37:29.451 DEBUG:boto:Token: None 2022-09-03T05:37:29.452 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj 2022-09-03T05:37:29.452 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/cZx0m0iJZNmGU56yG2A8HT2ydc= 2022-09-03T05:37:29.453 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/cZx0m0iJZNmGU56yG2A8HT2ydc=', 'Content-Length': '0'} 2022-09-03T05:37:29.459 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000aca1656c7d909520-006312e819-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.461 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.462 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.463 DEBUG:boto:Method: HEAD 2022-09-03T05:37:29.464 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:29.465 DEBUG:boto:Data: 2022-09-03T05:37:29.466 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.467 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.468 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.469 DEBUG:boto:Params: {} 2022-09-03T05:37:29.470 DEBUG:boto:Token: None 2022-09-03T05:37:29.471 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj 2022-09-03T05:37:29.472 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/cZx0m0iJZNmGU56yG2A8HT2ydc= 2022-09-03T05:37:29.473 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/cZx0m0iJZNmGU56yG2A8HT2ydc=', 'Content-Length': '0'} 2022-09-03T05:37:29.483 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000062861de01c0ff00d-006312e819-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.484 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:29.485 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.486 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.487 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.488 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.489 DEBUG:boto:Method: GET 2022-09-03T05:37:29.490 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.491 DEBUG:boto:Data: 2022-09-03T05:37:29.492 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.493 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:29.493 DEBUG:boto:Port: 8001 2022-09-03T05:37:29.495 DEBUG:boto:Params: {} 2022-09-03T05:37:29.496 DEBUG:boto:Token: None 2022-09-03T05:37:29.497 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.499 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r+jlFfDDPGGOApygy3BPwJT7kOI= 2022-09-03T05:37:29.500 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r+jlFfDDPGGOApygy3BPwJT7kOI=', 'Content-Length': '0'} 2022-09-03T05:37:29.503 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005395e1675bac7ac0-006312e819-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.505 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.507 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.508 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.509 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.510 DEBUG:boto:Method: GET 2022-09-03T05:37:29.511 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.512 DEBUG:boto:Data: 2022-09-03T05:37:29.513 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.514 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.515 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.517 DEBUG:boto:Params: {} 2022-09-03T05:37:29.518 DEBUG:boto:Token: None 2022-09-03T05:37:29.519 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.520 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r+jlFfDDPGGOApygy3BPwJT7kOI= 2022-09-03T05:37:29.521 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r+jlFfDDPGGOApygy3BPwJT7kOI=', 'Content-Length': '0'} 2022-09-03T05:37:29.524 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cecdb6000b9c009d-006312e819-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.526 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:29.527 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:29.529 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.530 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.531 DEBUG:boto:Method: GET 2022-09-03T05:37:29.532 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.533 DEBUG:boto:Data: 2022-09-03T05:37:29.534 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.535 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:29.537 DEBUG:boto:Port: 8001 2022-09-03T05:37:29.538 DEBUG:boto:Params: {} 2022-09-03T05:37:29.539 DEBUG:boto:Token: None 2022-09-03T05:37:29.540 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/ 2022-09-03T05:37:29.542 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE= 2022-09-03T05:37:29.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE=', 'Content-Length': '0'} 2022-09-03T05:37:29.549 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004e1bcdf14e6d7306-006312e819-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.550 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:29.551 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:29.552 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:29.553 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.553 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.554 DEBUG:boto:Method: GET 2022-09-03T05:37:29.554 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.555 DEBUG:boto:Data: 2022-09-03T05:37:29.555 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.556 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.557 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.557 DEBUG:boto:Params: {} 2022-09-03T05:37:29.558 DEBUG:boto:Token: None 2022-09-03T05:37:29.559 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/ 2022-09-03T05:37:29.560 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE= 2022-09-03T05:37:29.561 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE=', 'Content-Length': '0'} 2022-09-03T05:37:29.565 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009adb66a808f60ef2-006312e819-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.567 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:29.569 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:29.570 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.571 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.572 DEBUG:boto:path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.573 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.574 DEBUG:boto:Method: GET 2022-09-03T05:37:29.575 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.576 DEBUG:boto:Data: 2022-09-03T05:37:29.577 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.578 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:29.579 DEBUG:boto:Port: 8001 2022-09-03T05:37:29.580 DEBUG:boto:Params: {} 2022-09-03T05:37:29.581 DEBUG:boto:Token: None 2022-09-03T05:37:29.583 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.584 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fMTbc+OAm6Tlg/SCjYyb6sGZPmM= 2022-09-03T05:37:29.585 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fMTbc+OAm6Tlg/SCjYyb6sGZPmM=', 'Content-Length': '0'} 2022-09-03T05:37:29.588 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000029e5ae0d2475a507-006312e819-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.589 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.591 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.592 DEBUG:boto:path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.593 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.594 DEBUG:boto:Method: GET 2022-09-03T05:37:29.595 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.596 DEBUG:boto:Data: 2022-09-03T05:37:29.597 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.598 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.599 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.600 DEBUG:boto:Params: {} 2022-09-03T05:37:29.600 DEBUG:boto:Token: None 2022-09-03T05:37:29.601 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.602 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fMTbc+OAm6Tlg/SCjYyb6sGZPmM= 2022-09-03T05:37:29.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fMTbc+OAm6Tlg/SCjYyb6sGZPmM=', 'Content-Length': '0'} 2022-09-03T05:37:29.605 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a737fca27c78e72f-006312e819-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.606 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.607 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.607 DEBUG:boto:path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.608 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.608 DEBUG:boto:Method: HEAD 2022-09-03T05:37:29.610 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.611 DEBUG:boto:Data: 2022-09-03T05:37:29.611 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.612 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:29.612 DEBUG:boto:Port: 8001 2022-09-03T05:37:29.613 DEBUG:boto:Params: {} 2022-09-03T05:37:29.614 DEBUG:boto:Token: None 2022-09-03T05:37:29.614 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.615 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k8gjQGjRQ6XXLAdqJwGz7LYqOTk= 2022-09-03T05:37:29.616 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k8gjQGjRQ6XXLAdqJwGz7LYqOTk=', 'Content-Length': '0'} 2022-09-03T05:37:29.618 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005dfecc1b8e1abb1b-006312e819-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.620 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.621 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.622 DEBUG:boto:path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.623 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.624 DEBUG:boto:Method: HEAD 2022-09-03T05:37:29.625 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.626 DEBUG:boto:Data: 2022-09-03T05:37:29.626 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.627 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.627 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.629 DEBUG:boto:Params: {} 2022-09-03T05:37:29.630 DEBUG:boto:Token: None 2022-09-03T05:37:29.630 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.631 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k8gjQGjRQ6XXLAdqJwGz7LYqOTk= 2022-09-03T05:37:29.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k8gjQGjRQ6XXLAdqJwGz7LYqOTk=', 'Content-Length': '0'} 2022-09-03T05:37:29.636 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000779494dc19215e7e-006312e819-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.637 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:29.637 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.638 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.639 DEBUG:boto:path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.640 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.640 DEBUG:boto:Method: GET 2022-09-03T05:37:29.641 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.641 DEBUG:boto:Data: 2022-09-03T05:37:29.642 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.642 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:29.643 DEBUG:boto:Port: 8001 2022-09-03T05:37:29.643 DEBUG:boto:Params: {} 2022-09-03T05:37:29.644 DEBUG:boto:Token: None 2022-09-03T05:37:29.645 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.646 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fMTbc+OAm6Tlg/SCjYyb6sGZPmM= 2022-09-03T05:37:29.646 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fMTbc+OAm6Tlg/SCjYyb6sGZPmM=', 'Content-Length': '0'} 2022-09-03T05:37:29.649 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f2c126cd02b3e965-006312e819-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.690 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.691 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.691 DEBUG:boto:path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.692 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.693 DEBUG:boto:Method: GET 2022-09-03T05:37:29.693 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.694 DEBUG:boto:Data: 2022-09-03T05:37:29.694 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.695 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.695 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.696 DEBUG:boto:Params: {} 2022-09-03T05:37:29.696 DEBUG:boto:Token: None 2022-09-03T05:37:29.697 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi 2022-09-03T05:37:29.698 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fMTbc+OAm6Tlg/SCjYyb6sGZPmM= 2022-09-03T05:37:29.698 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fMTbc+OAm6Tlg/SCjYyb6sGZPmM=', 'Content-Length': '0'} 2022-09-03T05:37:29.701 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'GM1mLEe4fDwiUzp2k3PtPzJLy-tnWKi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005146659c04598d0e-006312e819-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.701 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:29.702 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:29.703 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.703 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.704 DEBUG:boto:Method: GET 2022-09-03T05:37:29.704 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.705 DEBUG:boto:Data: 2022-09-03T05:37:29.705 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.706 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:29.706 DEBUG:boto:Port: 8001 2022-09-03T05:37:29.707 DEBUG:boto:Params: {} 2022-09-03T05:37:29.708 DEBUG:boto:Token: None 2022-09-03T05:37:29.708 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/ 2022-09-03T05:37:29.709 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE= 2022-09-03T05:37:29.710 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE=', 'Content-Length': '0'} 2022-09-03T05:37:29.714 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001a1b4af80340f01d-006312e819-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.714 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:29.715 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:29.716 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:29.717 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.717 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.718 DEBUG:boto:Method: GET 2022-09-03T05:37:29.718 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.719 DEBUG:boto:Data: 2022-09-03T05:37:29.719 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.720 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.720 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.721 DEBUG:boto:Params: {} 2022-09-03T05:37:29.722 DEBUG:boto:Token: None 2022-09-03T05:37:29.723 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/ 2022-09-03T05:37:29.723 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE= 2022-09-03T05:37:29.724 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE=', 'Content-Length': '0'} 2022-09-03T05:37:29.727 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000db61310ca6354646-006312e819-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.728 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:29.730 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.731 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.731 DEBUG:boto:Method: HEAD 2022-09-03T05:37:29.733 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:29.734 DEBUG:boto:Data: 2022-09-03T05:37:29.736 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.737 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:29.738 DEBUG:boto:Port: 8001 2022-09-03T05:37:29.739 DEBUG:boto:Params: {} 2022-09-03T05:37:29.740 DEBUG:boto:Token: None 2022-09-03T05:37:29.741 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj 2022-09-03T05:37:29.741 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/cZx0m0iJZNmGU56yG2A8HT2ydc= 2022-09-03T05:37:29.743 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/cZx0m0iJZNmGU56yG2A8HT2ydc=', 'Content-Length': '0'} 2022-09-03T05:37:29.746 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000026c9cde8fab46b6f-006312e819-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.747 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.747 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.749 DEBUG:boto:Method: HEAD 2022-09-03T05:37:29.749 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:29.750 DEBUG:boto:Data: 2022-09-03T05:37:29.751 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.751 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.752 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.752 DEBUG:boto:Params: {} 2022-09-03T05:37:29.753 DEBUG:boto:Token: None 2022-09-03T05:37:29.754 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj 2022-09-03T05:37:29.754 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/cZx0m0iJZNmGU56yG2A8HT2ydc= 2022-09-03T05:37:29.755 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/cZx0m0iJZNmGU56yG2A8HT2ydc=', 'Content-Length': '0'} 2022-09-03T05:37:29.761 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009592aaa649baec98-006312e819-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.762 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:29.764 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.765 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.766 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.767 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.768 DEBUG:boto:Method: GET 2022-09-03T05:37:29.769 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.770 DEBUG:boto:Data: 2022-09-03T05:37:29.771 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.772 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:29.773 DEBUG:boto:Port: 8001 2022-09-03T05:37:29.774 DEBUG:boto:Params: {} 2022-09-03T05:37:29.775 DEBUG:boto:Token: None 2022-09-03T05:37:29.776 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.778 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r+jlFfDDPGGOApygy3BPwJT7kOI= 2022-09-03T05:37:29.779 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r+jlFfDDPGGOApygy3BPwJT7kOI=', 'Content-Length': '0'} 2022-09-03T05:37:29.782 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009e3d13d6fed12f00-006312e819-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.783 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.784 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.785 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.785 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.786 DEBUG:boto:Method: GET 2022-09-03T05:37:29.787 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.787 DEBUG:boto:Data: 2022-09-03T05:37:29.788 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.789 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.789 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.790 DEBUG:boto:Params: {} 2022-09-03T05:37:29.791 DEBUG:boto:Token: None 2022-09-03T05:37:29.792 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:29.793 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r+jlFfDDPGGOApygy3BPwJT7kOI= 2022-09-03T05:37:29.794 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r+jlFfDDPGGOApygy3BPwJT7kOI=', 'Content-Length': '0'} 2022-09-03T05:37:29.797 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002e1f493dd7869887-006312e819-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.842 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:29.843 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:29.844 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.844 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.845 DEBUG:boto:Method: GET 2022-09-03T05:37:29.846 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.846 DEBUG:boto:Data: 2022-09-03T05:37:29.847 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.847 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:29.848 DEBUG:boto:Port: 8001 2022-09-03T05:37:29.849 DEBUG:boto:Params: {} 2022-09-03T05:37:29.850 DEBUG:boto:Token: None 2022-09-03T05:37:29.851 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/ 2022-09-03T05:37:29.851 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE= 2022-09-03T05:37:29.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE=', 'Content-Length': '0'} 2022-09-03T05:37:29.859 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c9f53b655e4b7b7c-006312e819-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.860 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:29.867 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:29.868 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:29.869 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.886 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.887 DEBUG:boto:Method: GET 2022-09-03T05:37:29.888 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:29.889 DEBUG:boto:Data: 2022-09-03T05:37:29.895 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.897 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.898 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.899 DEBUG:boto:Params: {} 2022-09-03T05:37:29.900 DEBUG:boto:Token: None 2022-09-03T05:37:29.901 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/ 2022-09-03T05:37:29.904 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE= 2022-09-03T05:37:29.905 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EvWg0ELAlmHwrHVwaotWA6Uh2KE=', 'Content-Length': '0'} 2022-09-03T05:37:29.911 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000033701deabdb89efd-006312e819-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.912 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:29.914 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:29.915 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.916 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.917 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:29.918 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:29.919 DEBUG:boto:Method: GET 2022-09-03T05:37:29.920 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:29.921 DEBUG:boto:Data: 2022-09-03T05:37:29.921 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.922 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:29.922 DEBUG:boto:Port: 8001 2022-09-03T05:37:29.923 DEBUG:boto:Params: {} 2022-09-03T05:37:29.924 DEBUG:boto:Token: None 2022-09-03T05:37:29.924 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:29.925 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tKresLlOD50nHf7v0fGywZEyW7U= 2022-09-03T05:37:29.926 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tKresLlOD50nHf7v0fGywZEyW7U=', 'Content-Length': '0'} 2022-09-03T05:37:29.928 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000265efa3ab180ef10-006312e819-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.929 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.930 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.931 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:29.931 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:29.932 DEBUG:boto:Method: GET 2022-09-03T05:37:29.932 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:29.933 DEBUG:boto:Data: 2022-09-03T05:37:29.933 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.934 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.934 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.935 DEBUG:boto:Params: {} 2022-09-03T05:37:29.936 DEBUG:boto:Token: None 2022-09-03T05:37:29.936 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:29.937 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tKresLlOD50nHf7v0fGywZEyW7U= 2022-09-03T05:37:29.938 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tKresLlOD50nHf7v0fGywZEyW7U=', 'Content-Length': '0'} 2022-09-03T05:37:29.940 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000654eec052b98789a-006312e819-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.942 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.943 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.944 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:29.944 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:29.945 DEBUG:boto:Method: HEAD 2022-09-03T05:37:29.947 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:29.947 DEBUG:boto:Data: 2022-09-03T05:37:29.948 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.949 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:29.950 DEBUG:boto:Port: 8001 2022-09-03T05:37:29.951 DEBUG:boto:Params: {} 2022-09-03T05:37:29.952 DEBUG:boto:Token: None 2022-09-03T05:37:29.953 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:29.954 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GCxIBhX3AB23it8QE5FPiral7nM= 2022-09-03T05:37:29.955 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GCxIBhX3AB23it8QE5FPiral7nM=', 'Content-Length': '0'} 2022-09-03T05:37:29.958 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fdaae470c64fb1ff-006312e819-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.959 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.960 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.961 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:29.962 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:29.962 DEBUG:boto:Method: HEAD 2022-09-03T05:37:29.963 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:29.963 DEBUG:boto:Data: 2022-09-03T05:37:29.964 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.964 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.965 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.965 DEBUG:boto:Params: {} 2022-09-03T05:37:29.966 DEBUG:boto:Token: None 2022-09-03T05:37:29.967 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:29.968 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GCxIBhX3AB23it8QE5FPiral7nM= 2022-09-03T05:37:29.969 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GCxIBhX3AB23it8QE5FPiral7nM=', 'Content-Length': '0'} 2022-09-03T05:37:29.973 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006b56151893af58ec-006312e819-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.974 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:29.975 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.976 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.977 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:29.978 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:29.979 DEBUG:boto:Method: GET 2022-09-03T05:37:29.980 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:29.981 DEBUG:boto:Data: 2022-09-03T05:37:29.981 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.982 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:29.983 DEBUG:boto:Port: 8001 2022-09-03T05:37:29.984 DEBUG:boto:Params: {} 2022-09-03T05:37:29.985 DEBUG:boto:Token: None 2022-09-03T05:37:29.986 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:29 GMT /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:29.987 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tKresLlOD50nHf7v0fGywZEyW7U= 2022-09-03T05:37:29.989 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tKresLlOD50nHf7v0fGywZEyW7U=', 'Content-Length': '0'} 2022-09-03T05:37:29.991 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a5f22663e1a6baaa-006312e819-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:29.993 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:29.993 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:29.994 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:29.995 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:29.995 DEBUG:boto:Method: GET 2022-09-03T05:37:29.996 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:29.996 DEBUG:boto:Data: 2022-09-03T05:37:29.997 DEBUG:boto:Headers: {} 2022-09-03T05:37:29.997 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:29.998 DEBUG:boto:Port: 8000 2022-09-03T05:37:29.999 DEBUG:boto:Params: {} 2022-09-03T05:37:29.999 DEBUG:boto:Token: None 2022-09-03T05:37:30.000 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2 2022-09-03T05:37:30.001 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CTXmnEYSvcHpCj+SH6NyO2CitCg= 2022-09-03T05:37:30.001 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CTXmnEYSvcHpCj+SH6NyO2CitCg=', 'Content-Length': '0'} 2022-09-03T05:37:30.004 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Kl9OW7HSpMTjjnmOMkqEktWm9H3uMr2'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e660c2a21b94e745-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.005 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:30.005 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:30.006 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.006 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.007 DEBUG:boto:Method: GET 2022-09-03T05:37:30.008 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.008 DEBUG:boto:Data: 2022-09-03T05:37:30.009 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.009 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.010 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.011 DEBUG:boto:Params: {} 2022-09-03T05:37:30.011 DEBUG:boto:Token: None 2022-09-03T05:37:30.012 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/ 2022-09-03T05:37:30.013 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI= 2022-09-03T05:37:30.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI=', 'Content-Length': '0'} 2022-09-03T05:37:30.017 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000041f52b618612b8ef-006312e81a-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.018 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:30.019 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:30.020 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:30.020 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.021 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.022 DEBUG:boto:Method: GET 2022-09-03T05:37:30.023 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.024 DEBUG:boto:Data: 2022-09-03T05:37:30.025 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.026 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:30.027 DEBUG:boto:Port: 8000 2022-09-03T05:37:30.028 DEBUG:boto:Params: {} 2022-09-03T05:37:30.029 DEBUG:boto:Token: None 2022-09-03T05:37:30.031 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/ 2022-09-03T05:37:30.032 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI= 2022-09-03T05:37:30.033 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI=', 'Content-Length': '0'} 2022-09-03T05:37:30.037 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000007f41612b3da22a7-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.038 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:30.039 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.039 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.040 DEBUG:boto:Method: HEAD 2022-09-03T05:37:30.041 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:30.041 DEBUG:boto:Data: 2022-09-03T05:37:30.042 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.042 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.043 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.043 DEBUG:boto:Params: {} 2022-09-03T05:37:30.044 DEBUG:boto:Token: None 2022-09-03T05:37:30.045 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj 2022-09-03T05:37:30.045 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cwg/mcrsNu8VG6QqEXFUZA3iKkI= 2022-09-03T05:37:30.046 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cwg/mcrsNu8VG6QqEXFUZA3iKkI=', 'Content-Length': '0'} 2022-09-03T05:37:30.050 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000794b4b12172faddc-006312e81a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.051 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.052 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.053 DEBUG:boto:Method: HEAD 2022-09-03T05:37:30.054 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:30.055 DEBUG:boto:Data: 2022-09-03T05:37:30.055 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.056 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:30.056 DEBUG:boto:Port: 8000 2022-09-03T05:37:30.057 DEBUG:boto:Params: {} 2022-09-03T05:37:30.058 DEBUG:boto:Token: None 2022-09-03T05:37:30.058 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj 2022-09-03T05:37:30.059 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cwg/mcrsNu8VG6QqEXFUZA3iKkI= 2022-09-03T05:37:30.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cwg/mcrsNu8VG6QqEXFUZA3iKkI=', 'Content-Length': '0'} 2022-09-03T05:37:30.063 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000306f583e7a58328b-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.064 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:30.065 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.066 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.066 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.067 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.067 DEBUG:boto:Method: GET 2022-09-03T05:37:30.068 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.069 DEBUG:boto:Data: 2022-09-03T05:37:30.069 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.070 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.070 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.071 DEBUG:boto:Params: {} 2022-09-03T05:37:30.072 DEBUG:boto:Token: None 2022-09-03T05:37:30.072 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.073 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2rh2LAu+PM4HCNfxBJOBXHeSJqU= 2022-09-03T05:37:30.074 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2rh2LAu+PM4HCNfxBJOBXHeSJqU=', 'Content-Length': '0'} 2022-09-03T05:37:30.076 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ec65c78a6a8b5227-006312e81a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.122 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.123 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.123 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.124 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.125 DEBUG:boto:Method: GET 2022-09-03T05:37:30.125 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.126 DEBUG:boto:Data: 2022-09-03T05:37:30.127 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.127 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:30.128 DEBUG:boto:Port: 8000 2022-09-03T05:37:30.128 DEBUG:boto:Params: {} 2022-09-03T05:37:30.129 DEBUG:boto:Token: None 2022-09-03T05:37:30.131 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.132 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2rh2LAu+PM4HCNfxBJOBXHeSJqU= 2022-09-03T05:37:30.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2rh2LAu+PM4HCNfxBJOBXHeSJqU=', 'Content-Length': '0'} 2022-09-03T05:37:30.139 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ac3d2c859779ecf3-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.140 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:30.142 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:30.142 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.143 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.143 DEBUG:boto:Method: GET 2022-09-03T05:37:30.144 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.144 DEBUG:boto:Data: 2022-09-03T05:37:30.145 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.146 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.146 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.147 DEBUG:boto:Params: {} 2022-09-03T05:37:30.147 DEBUG:boto:Token: None 2022-09-03T05:37:30.148 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/ 2022-09-03T05:37:30.149 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI= 2022-09-03T05:37:30.149 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI=', 'Content-Length': '0'} 2022-09-03T05:37:30.153 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008a28bd69c8b61640-006312e81a-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.154 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:30.155 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:30.155 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:30.156 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.156 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.157 DEBUG:boto:Method: GET 2022-09-03T05:37:30.157 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.158 DEBUG:boto:Data: 2022-09-03T05:37:30.159 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.160 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:30.160 DEBUG:boto:Port: 8000 2022-09-03T05:37:30.161 DEBUG:boto:Params: {} 2022-09-03T05:37:30.163 DEBUG:boto:Token: None 2022-09-03T05:37:30.163 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/ 2022-09-03T05:37:30.164 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI= 2022-09-03T05:37:30.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI=', 'Content-Length': '0'} 2022-09-03T05:37:30.168 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c1d0df6168b28c54-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.169 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:30.170 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:30.170 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.171 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.172 DEBUG:boto:path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.173 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.174 DEBUG:boto:Method: GET 2022-09-03T05:37:30.175 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.175 DEBUG:boto:Data: 2022-09-03T05:37:30.176 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.176 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.177 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.178 DEBUG:boto:Params: {} 2022-09-03T05:37:30.178 DEBUG:boto:Token: None 2022-09-03T05:37:30.179 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.180 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:afADjcVlxqNe58G21bG0bP8jPfs= 2022-09-03T05:37:30.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:afADjcVlxqNe58G21bG0bP8jPfs=', 'Content-Length': '0'} 2022-09-03T05:37:30.184 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'EFeprdWgJo5gbXM16hy6HNBwhGTxK7e'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d413a15c59426443-006312e81a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.185 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.185 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.186 DEBUG:boto:path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.186 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.187 DEBUG:boto:Method: GET 2022-09-03T05:37:30.188 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.188 DEBUG:boto:Data: 2022-09-03T05:37:30.189 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.189 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:30.190 DEBUG:boto:Port: 8000 2022-09-03T05:37:30.191 DEBUG:boto:Params: {} 2022-09-03T05:37:30.191 DEBUG:boto:Token: None 2022-09-03T05:37:30.192 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.193 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:afADjcVlxqNe58G21bG0bP8jPfs= 2022-09-03T05:37:30.194 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:afADjcVlxqNe58G21bG0bP8jPfs=', 'Content-Length': '0'} 2022-09-03T05:37:30.196 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'EFeprdWgJo5gbXM16hy6HNBwhGTxK7e'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000426109c5d80e2e45-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.197 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.197 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.198 DEBUG:boto:path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.198 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.199 DEBUG:boto:Method: HEAD 2022-09-03T05:37:30.199 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.200 DEBUG:boto:Data: 2022-09-03T05:37:30.200 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.201 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.201 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.202 DEBUG:boto:Params: {} 2022-09-03T05:37:30.203 DEBUG:boto:Token: None 2022-09-03T05:37:30.203 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.204 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IPgMPmqSYOvbepORIWZ7saXd3b8= 2022-09-03T05:37:30.205 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IPgMPmqSYOvbepORIWZ7saXd3b8=', 'Content-Length': '0'} 2022-09-03T05:37:30.207 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'EFeprdWgJo5gbXM16hy6HNBwhGTxK7e'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000aad9b2d9c975bbce-006312e81a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.208 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.208 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.209 DEBUG:boto:path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.209 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.210 DEBUG:boto:Method: HEAD 2022-09-03T05:37:30.210 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.211 DEBUG:boto:Data: 2022-09-03T05:37:30.211 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.212 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:30.212 DEBUG:boto:Port: 8000 2022-09-03T05:37:30.213 DEBUG:boto:Params: {} 2022-09-03T05:37:30.213 DEBUG:boto:Token: None 2022-09-03T05:37:30.214 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.215 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IPgMPmqSYOvbepORIWZ7saXd3b8= 2022-09-03T05:37:30.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IPgMPmqSYOvbepORIWZ7saXd3b8=', 'Content-Length': '0'} 2022-09-03T05:37:30.218 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'EFeprdWgJo5gbXM16hy6HNBwhGTxK7e'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bdd49e08ac571982-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.218 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:30.219 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.220 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.220 DEBUG:boto:path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.221 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.221 DEBUG:boto:Method: GET 2022-09-03T05:37:30.222 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.222 DEBUG:boto:Data: 2022-09-03T05:37:30.223 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.224 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.224 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.225 DEBUG:boto:Params: {} 2022-09-03T05:37:30.226 DEBUG:boto:Token: None 2022-09-03T05:37:30.227 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.227 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:afADjcVlxqNe58G21bG0bP8jPfs= 2022-09-03T05:37:30.228 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:afADjcVlxqNe58G21bG0bP8jPfs=', 'Content-Length': '0'} 2022-09-03T05:37:30.230 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'EFeprdWgJo5gbXM16hy6HNBwhGTxK7e'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c4aa91fa22047a7b-006312e81a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.274 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.275 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.275 DEBUG:boto:path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.276 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.277 DEBUG:boto:Method: GET 2022-09-03T05:37:30.277 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.278 DEBUG:boto:Data: 2022-09-03T05:37:30.279 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.281 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:30.282 DEBUG:boto:Port: 8000 2022-09-03T05:37:30.282 DEBUG:boto:Params: {} 2022-09-03T05:37:30.283 DEBUG:boto:Token: None 2022-09-03T05:37:30.284 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=EFeprdWgJo5gbXM16hy6HNBwhGTxK7e 2022-09-03T05:37:30.284 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:afADjcVlxqNe58G21bG0bP8jPfs= 2022-09-03T05:37:30.285 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:afADjcVlxqNe58G21bG0bP8jPfs=', 'Content-Length': '0'} 2022-09-03T05:37:30.287 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'EFeprdWgJo5gbXM16hy6HNBwhGTxK7e'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e98b124aac932ab9-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.288 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:30.289 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:30.289 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.290 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.291 DEBUG:boto:Method: GET 2022-09-03T05:37:30.292 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.293 DEBUG:boto:Data: 2022-09-03T05:37:30.294 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.295 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.296 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.297 DEBUG:boto:Params: {} 2022-09-03T05:37:30.299 DEBUG:boto:Token: None 2022-09-03T05:37:30.300 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/ 2022-09-03T05:37:30.301 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI= 2022-09-03T05:37:30.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI=', 'Content-Length': '0'} 2022-09-03T05:37:30.313 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006f05d55e93e0c2f0-006312e81a-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.314 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:30.316 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:30.316 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:30.317 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.318 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.319 DEBUG:boto:Method: GET 2022-09-03T05:37:30.320 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.321 DEBUG:boto:Data: 2022-09-03T05:37:30.322 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.323 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:30.324 DEBUG:boto:Port: 8000 2022-09-03T05:37:30.325 DEBUG:boto:Params: {} 2022-09-03T05:37:30.326 DEBUG:boto:Token: None 2022-09-03T05:37:30.327 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/ 2022-09-03T05:37:30.328 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI= 2022-09-03T05:37:30.329 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI=', 'Content-Length': '0'} 2022-09-03T05:37:30.333 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000baabd003e11ef9d-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.335 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:30.336 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.337 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.338 DEBUG:boto:Method: HEAD 2022-09-03T05:37:30.340 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:30.341 DEBUG:boto:Data: 2022-09-03T05:37:30.342 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.343 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.344 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.345 DEBUG:boto:Params: {} 2022-09-03T05:37:30.347 DEBUG:boto:Token: None 2022-09-03T05:37:30.348 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj 2022-09-03T05:37:30.349 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cwg/mcrsNu8VG6QqEXFUZA3iKkI= 2022-09-03T05:37:30.350 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cwg/mcrsNu8VG6QqEXFUZA3iKkI=', 'Content-Length': '0'} 2022-09-03T05:37:30.354 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006190747e1f5fd2bd-006312e81a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.356 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.357 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.357 DEBUG:boto:Method: HEAD 2022-09-03T05:37:30.358 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:30.359 DEBUG:boto:Data: 2022-09-03T05:37:30.360 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.361 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:30.362 DEBUG:boto:Port: 8000 2022-09-03T05:37:30.363 DEBUG:boto:Params: {} 2022-09-03T05:37:30.364 DEBUG:boto:Token: None 2022-09-03T05:37:30.365 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj 2022-09-03T05:37:30.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cwg/mcrsNu8VG6QqEXFUZA3iKkI= 2022-09-03T05:37:30.368 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cwg/mcrsNu8VG6QqEXFUZA3iKkI=', 'Content-Length': '0'} 2022-09-03T05:37:30.371 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003c5de6aa002e6c5b-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.373 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:30.374 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.374 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.375 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.376 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.377 DEBUG:boto:Method: GET 2022-09-03T05:37:30.378 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.379 DEBUG:boto:Data: 2022-09-03T05:37:30.380 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.381 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.382 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.383 DEBUG:boto:Params: {} 2022-09-03T05:37:30.384 DEBUG:boto:Token: None 2022-09-03T05:37:30.384 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.385 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2rh2LAu+PM4HCNfxBJOBXHeSJqU= 2022-09-03T05:37:30.385 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2rh2LAu+PM4HCNfxBJOBXHeSJqU=', 'Content-Length': '0'} 2022-09-03T05:37:30.388 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004b6e8ba1df6996cc-006312e81a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.389 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.390 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.391 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.391 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.392 DEBUG:boto:Method: GET 2022-09-03T05:37:30.392 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.393 DEBUG:boto:Data: 2022-09-03T05:37:30.394 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.394 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:30.395 DEBUG:boto:Port: 8000 2022-09-03T05:37:30.395 DEBUG:boto:Params: {} 2022-09-03T05:37:30.396 DEBUG:boto:Token: None 2022-09-03T05:37:30.397 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.397 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2rh2LAu+PM4HCNfxBJOBXHeSJqU= 2022-09-03T05:37:30.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2rh2LAu+PM4HCNfxBJOBXHeSJqU=', 'Content-Length': '0'} 2022-09-03T05:37:30.402 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002077076acb3448e1-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.446 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:30.447 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:30.448 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.448 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.449 DEBUG:boto:Method: GET 2022-09-03T05:37:30.450 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.450 DEBUG:boto:Data: 2022-09-03T05:37:30.451 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.452 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.452 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.453 DEBUG:boto:Params: {} 2022-09-03T05:37:30.453 DEBUG:boto:Token: None 2022-09-03T05:37:30.454 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/ 2022-09-03T05:37:30.455 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI= 2022-09-03T05:37:30.455 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI=', 'Content-Length': '0'} 2022-09-03T05:37:30.459 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e83fd24e4b1474e8-006312e81a-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.460 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:30.462 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:30.462 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:30.463 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.463 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.464 DEBUG:boto:Method: GET 2022-09-03T05:37:30.464 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.465 DEBUG:boto:Data: 2022-09-03T05:37:30.465 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.466 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:30.467 DEBUG:boto:Port: 8000 2022-09-03T05:37:30.468 DEBUG:boto:Params: {} 2022-09-03T05:37:30.469 DEBUG:boto:Token: None 2022-09-03T05:37:30.469 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/ 2022-09-03T05:37:30.470 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI= 2022-09-03T05:37:30.471 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI=', 'Content-Length': '0'} 2022-09-03T05:37:30.474 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001d5fd9910a0a76b1-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.475 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:30.476 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:30.477 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.478 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.479 DEBUG:boto:path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.479 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.480 DEBUG:boto:Method: GET 2022-09-03T05:37:30.480 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.481 DEBUG:boto:Data: 2022-09-03T05:37:30.482 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.483 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.484 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.485 DEBUG:boto:Params: {} 2022-09-03T05:37:30.486 DEBUG:boto:Token: None 2022-09-03T05:37:30.487 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.487 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4GclDUxtnV3X+N5ngcmsDO2B/v4= 2022-09-03T05:37:30.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4GclDUxtnV3X+N5ngcmsDO2B/v4=', 'Content-Length': '0'} 2022-09-03T05:37:30.491 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f3c9bba1b357ef07-006312e81a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.534 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.535 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.536 DEBUG:boto:path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.537 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.538 DEBUG:boto:Method: GET 2022-09-03T05:37:30.539 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.540 DEBUG:boto:Data: 2022-09-03T05:37:30.541 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.542 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:30.542 DEBUG:boto:Port: 8000 2022-09-03T05:37:30.543 DEBUG:boto:Params: {} 2022-09-03T05:37:30.544 DEBUG:boto:Token: None 2022-09-03T05:37:30.545 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.545 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4GclDUxtnV3X+N5ngcmsDO2B/v4= 2022-09-03T05:37:30.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4GclDUxtnV3X+N5ngcmsDO2B/v4=', 'Content-Length': '0'} 2022-09-03T05:37:30.549 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000feb6488467d09b34-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.550 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.551 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.552 DEBUG:boto:path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.553 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.554 DEBUG:boto:Method: HEAD 2022-09-03T05:37:30.555 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.555 DEBUG:boto:Data: 2022-09-03T05:37:30.556 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.557 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.557 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.558 DEBUG:boto:Params: {} 2022-09-03T05:37:30.559 DEBUG:boto:Token: None 2022-09-03T05:37:30.559 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.560 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jPfrclKrzfeweDCbb3XnppmMJCU= 2022-09-03T05:37:30.560 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jPfrclKrzfeweDCbb3XnppmMJCU=', 'Content-Length': '0'} 2022-09-03T05:37:30.563 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f0ea8b0528f387b2-006312e81a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.563 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.564 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.565 DEBUG:boto:path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.565 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.566 DEBUG:boto:Method: HEAD 2022-09-03T05:37:30.567 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.568 DEBUG:boto:Data: 2022-09-03T05:37:30.568 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.569 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:30.570 DEBUG:boto:Port: 8000 2022-09-03T05:37:30.571 DEBUG:boto:Params: {} 2022-09-03T05:37:30.572 DEBUG:boto:Token: None 2022-09-03T05:37:30.573 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.574 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jPfrclKrzfeweDCbb3XnppmMJCU= 2022-09-03T05:37:30.575 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jPfrclKrzfeweDCbb3XnppmMJCU=', 'Content-Length': '0'} 2022-09-03T05:37:30.577 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000836a711274a37930-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.578 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:30.579 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.580 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.580 DEBUG:boto:path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.581 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.582 DEBUG:boto:Method: GET 2022-09-03T05:37:30.583 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.584 DEBUG:boto:Data: 2022-09-03T05:37:30.585 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.586 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.587 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.587 DEBUG:boto:Params: {} 2022-09-03T05:37:30.588 DEBUG:boto:Token: None 2022-09-03T05:37:30.589 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.590 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4GclDUxtnV3X+N5ngcmsDO2B/v4= 2022-09-03T05:37:30.591 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4GclDUxtnV3X+N5ngcmsDO2B/v4=', 'Content-Length': '0'} 2022-09-03T05:37:30.593 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000725d8cc4ee153d7f-006312e81a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.593 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.594 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.595 DEBUG:boto:path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.595 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.596 DEBUG:boto:Method: GET 2022-09-03T05:37:30.597 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.597 DEBUG:boto:Data: 2022-09-03T05:37:30.598 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.598 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:30.599 DEBUG:boto:Port: 8000 2022-09-03T05:37:30.600 DEBUG:boto:Params: {} 2022-09-03T05:37:30.600 DEBUG:boto:Token: None 2022-09-03T05:37:30.601 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj 2022-09-03T05:37:30.602 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4GclDUxtnV3X+N5ngcmsDO2B/v4= 2022-09-03T05:37:30.602 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4GclDUxtnV3X+N5ngcmsDO2B/v4=', 'Content-Length': '0'} 2022-09-03T05:37:30.605 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'JID7ijdm1B3ZxtamFDxAcL6LJfFKoNj'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000740f87b07eee49cf-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.606 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:30.607 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:30.607 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.608 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.608 DEBUG:boto:Method: GET 2022-09-03T05:37:30.609 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.610 DEBUG:boto:Data: 2022-09-03T05:37:30.610 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.611 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.611 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.612 DEBUG:boto:Params: {} 2022-09-03T05:37:30.613 DEBUG:boto:Token: None 2022-09-03T05:37:30.613 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/ 2022-09-03T05:37:30.614 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI= 2022-09-03T05:37:30.615 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI=', 'Content-Length': '0'} 2022-09-03T05:37:30.618 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000256481153d1cbf5-006312e81a-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.619 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:30.620 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:30.621 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:30.621 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.622 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.623 DEBUG:boto:Method: GET 2022-09-03T05:37:30.623 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.624 DEBUG:boto:Data: 2022-09-03T05:37:30.624 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.625 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:30.625 DEBUG:boto:Port: 8000 2022-09-03T05:37:30.626 DEBUG:boto:Params: {} 2022-09-03T05:37:30.627 DEBUG:boto:Token: None 2022-09-03T05:37:30.628 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/ 2022-09-03T05:37:30.629 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI= 2022-09-03T05:37:30.630 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI=', 'Content-Length': '0'} 2022-09-03T05:37:30.633 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004b461f56a44fa64f-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.634 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:30.635 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.636 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.637 DEBUG:boto:Method: HEAD 2022-09-03T05:37:30.637 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:30.638 DEBUG:boto:Data: 2022-09-03T05:37:30.639 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.640 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.641 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.642 DEBUG:boto:Params: {} 2022-09-03T05:37:30.642 DEBUG:boto:Token: None 2022-09-03T05:37:30.643 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj 2022-09-03T05:37:30.644 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cwg/mcrsNu8VG6QqEXFUZA3iKkI= 2022-09-03T05:37:30.645 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cwg/mcrsNu8VG6QqEXFUZA3iKkI=', 'Content-Length': '0'} 2022-09-03T05:37:30.648 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000133060f24d9872ee-006312e81a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.649 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.649 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.650 DEBUG:boto:Method: HEAD 2022-09-03T05:37:30.651 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:30.651 DEBUG:boto:Data: 2022-09-03T05:37:30.652 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.653 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:30.654 DEBUG:boto:Port: 8000 2022-09-03T05:37:30.655 DEBUG:boto:Params: {} 2022-09-03T05:37:30.655 DEBUG:boto:Token: None 2022-09-03T05:37:30.656 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj 2022-09-03T05:37:30.656 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cwg/mcrsNu8VG6QqEXFUZA3iKkI= 2022-09-03T05:37:30.657 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cwg/mcrsNu8VG6QqEXFUZA3iKkI=', 'Content-Length': '0'} 2022-09-03T05:37:30.661 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005609f7f878c7362e-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.662 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:30.662 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.663 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.664 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.664 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.665 DEBUG:boto:Method: GET 2022-09-03T05:37:30.665 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.666 DEBUG:boto:Data: 2022-09-03T05:37:30.667 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.667 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.668 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.668 DEBUG:boto:Params: {} 2022-09-03T05:37:30.669 DEBUG:boto:Token: None 2022-09-03T05:37:30.671 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.672 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2rh2LAu+PM4HCNfxBJOBXHeSJqU= 2022-09-03T05:37:30.673 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2rh2LAu+PM4HCNfxBJOBXHeSJqU=', 'Content-Length': '0'} 2022-09-03T05:37:30.676 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a8b48c6ea80fddad-006312e81a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.718 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.719 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.720 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.720 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.721 DEBUG:boto:Method: GET 2022-09-03T05:37:30.721 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.722 DEBUG:boto:Data: 2022-09-03T05:37:30.722 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.723 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:30.723 DEBUG:boto:Port: 8000 2022-09-03T05:37:30.724 DEBUG:boto:Params: {} 2022-09-03T05:37:30.725 DEBUG:boto:Token: None 2022-09-03T05:37:30.725 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.726 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2rh2LAu+PM4HCNfxBJOBXHeSJqU= 2022-09-03T05:37:30.727 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2rh2LAu+PM4HCNfxBJOBXHeSJqU=', 'Content-Length': '0'} 2022-09-03T05:37:30.729 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000e1c8f9f31c03115-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.731 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:30.732 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:30.733 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.734 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.735 DEBUG:boto:Method: GET 2022-09-03T05:37:30.736 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.737 DEBUG:boto:Data: 2022-09-03T05:37:30.738 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.739 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.740 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.741 DEBUG:boto:Params: {} 2022-09-03T05:37:30.742 DEBUG:boto:Token: None 2022-09-03T05:37:30.743 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/ 2022-09-03T05:37:30.745 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI= 2022-09-03T05:37:30.746 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI=', 'Content-Length': '0'} 2022-09-03T05:37:30.749 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000068534db848db5270-006312e81a-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.750 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:30.752 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:30.753 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:30.754 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.755 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.756 DEBUG:boto:Method: GET 2022-09-03T05:37:30.757 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.758 DEBUG:boto:Data: 2022-09-03T05:37:30.758 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.759 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:30.760 DEBUG:boto:Port: 8000 2022-09-03T05:37:30.762 DEBUG:boto:Params: {} 2022-09-03T05:37:30.763 DEBUG:boto:Token: None 2022-09-03T05:37:30.764 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/ 2022-09-03T05:37:30.766 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI= 2022-09-03T05:37:30.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI=', 'Content-Length': '0'} 2022-09-03T05:37:30.771 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000072e0b332b0a8622c-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.773 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:30.774 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:30.775 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.776 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.777 DEBUG:boto:path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.778 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.779 DEBUG:boto:Method: GET 2022-09-03T05:37:30.780 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.781 DEBUG:boto:Data: 2022-09-03T05:37:30.782 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.783 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.784 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.785 DEBUG:boto:Params: {} 2022-09-03T05:37:30.787 DEBUG:boto:Token: None 2022-09-03T05:37:30.788 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.789 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ywoHsjwKnnQpAMZyDv0Qe/Yxx5k= 2022-09-03T05:37:30.790 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ywoHsjwKnnQpAMZyDv0Qe/Yxx5k=', 'Content-Length': '0'} 2022-09-03T05:37:30.798 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'yL2uOIeC0l.C5YXdYoqQBvDIly3idxO'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000198cb4b9632785fa-006312e81a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.799 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.800 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.800 DEBUG:boto:path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.801 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.801 DEBUG:boto:Method: GET 2022-09-03T05:37:30.802 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.803 DEBUG:boto:Data: 2022-09-03T05:37:30.803 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.804 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:30.805 DEBUG:boto:Port: 8000 2022-09-03T05:37:30.805 DEBUG:boto:Params: {} 2022-09-03T05:37:30.806 DEBUG:boto:Token: None 2022-09-03T05:37:30.807 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.807 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ywoHsjwKnnQpAMZyDv0Qe/Yxx5k= 2022-09-03T05:37:30.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ywoHsjwKnnQpAMZyDv0Qe/Yxx5k=', 'Content-Length': '0'} 2022-09-03T05:37:30.810 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'yL2uOIeC0l.C5YXdYoqQBvDIly3idxO'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000032ae5c1d5f8a704f-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.812 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.813 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.815 DEBUG:boto:path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.816 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.816 DEBUG:boto:Method: HEAD 2022-09-03T05:37:30.817 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.818 DEBUG:boto:Data: 2022-09-03T05:37:30.820 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.821 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.822 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.823 DEBUG:boto:Params: {} 2022-09-03T05:37:30.824 DEBUG:boto:Token: None 2022-09-03T05:37:30.824 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.825 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ktUSq2flB5GolQZj2LlgA9dJ3tA= 2022-09-03T05:37:30.826 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ktUSq2flB5GolQZj2LlgA9dJ3tA=', 'Content-Length': '0'} 2022-09-03T05:37:30.829 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'yL2uOIeC0l.C5YXdYoqQBvDIly3idxO'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000019f5af10399a688c-006312e81a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.831 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.831 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.832 DEBUG:boto:path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.833 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.834 DEBUG:boto:Method: HEAD 2022-09-03T05:37:30.835 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.835 DEBUG:boto:Data: 2022-09-03T05:37:30.836 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.837 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:30.838 DEBUG:boto:Port: 8000 2022-09-03T05:37:30.839 DEBUG:boto:Params: {} 2022-09-03T05:37:30.839 DEBUG:boto:Token: None 2022-09-03T05:37:30.840 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.841 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ktUSq2flB5GolQZj2LlgA9dJ3tA= 2022-09-03T05:37:30.841 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ktUSq2flB5GolQZj2LlgA9dJ3tA=', 'Content-Length': '0'} 2022-09-03T05:37:30.843 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'yL2uOIeC0l.C5YXdYoqQBvDIly3idxO'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000593b11a8851db654-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.844 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:30.845 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.845 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.846 DEBUG:boto:path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.846 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.847 DEBUG:boto:Method: GET 2022-09-03T05:37:30.847 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.848 DEBUG:boto:Data: 2022-09-03T05:37:30.848 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.849 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.849 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.850 DEBUG:boto:Params: {} 2022-09-03T05:37:30.851 DEBUG:boto:Token: None 2022-09-03T05:37:30.851 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.852 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ywoHsjwKnnQpAMZyDv0Qe/Yxx5k= 2022-09-03T05:37:30.853 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ywoHsjwKnnQpAMZyDv0Qe/Yxx5k=', 'Content-Length': '0'} 2022-09-03T05:37:30.855 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'yL2uOIeC0l.C5YXdYoqQBvDIly3idxO'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000367d344f51f688ab-006312e81a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.856 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.857 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.857 DEBUG:boto:path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.858 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.858 DEBUG:boto:Method: GET 2022-09-03T05:37:30.859 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.859 DEBUG:boto:Data: 2022-09-03T05:37:30.860 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.861 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:30.861 DEBUG:boto:Port: 8000 2022-09-03T05:37:30.862 DEBUG:boto:Params: {} 2022-09-03T05:37:30.862 DEBUG:boto:Token: None 2022-09-03T05:37:30.863 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=yL2uOIeC0l.C5YXdYoqQBvDIly3idxO 2022-09-03T05:37:30.863 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ywoHsjwKnnQpAMZyDv0Qe/Yxx5k= 2022-09-03T05:37:30.864 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ywoHsjwKnnQpAMZyDv0Qe/Yxx5k=', 'Content-Length': '0'} 2022-09-03T05:37:30.866 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'yL2uOIeC0l.C5YXdYoqQBvDIly3idxO'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000125e44968c592b7e-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.910 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:30.911 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:30.912 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.913 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.914 DEBUG:boto:Method: GET 2022-09-03T05:37:30.914 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.915 DEBUG:boto:Data: 2022-09-03T05:37:30.915 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.916 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.916 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.917 DEBUG:boto:Params: {} 2022-09-03T05:37:30.918 DEBUG:boto:Token: None 2022-09-03T05:37:30.919 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/ 2022-09-03T05:37:30.919 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI= 2022-09-03T05:37:30.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI=', 'Content-Length': '0'} 2022-09-03T05:37:30.923 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000096c39fc02f4e1555-006312e81a-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.924 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:30.925 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:30.926 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:30.927 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.927 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.928 DEBUG:boto:Method: GET 2022-09-03T05:37:30.928 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:30.929 DEBUG:boto:Data: 2022-09-03T05:37:30.930 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.930 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:30.932 DEBUG:boto:Port: 8000 2022-09-03T05:37:30.933 DEBUG:boto:Params: {} 2022-09-03T05:37:30.934 DEBUG:boto:Token: None 2022-09-03T05:37:30.935 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/ 2022-09-03T05:37:30.936 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI= 2022-09-03T05:37:30.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wj4K3yg3h4kyu8tO+vpifl9xAXI=', 'Content-Length': '0'} 2022-09-03T05:37:30.941 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006b11b4a6829121b2-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.943 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:30.944 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.945 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.945 DEBUG:boto:Method: HEAD 2022-09-03T05:37:30.946 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:30.946 DEBUG:boto:Data: 2022-09-03T05:37:30.947 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.947 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.948 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.949 DEBUG:boto:Params: {} 2022-09-03T05:37:30.949 DEBUG:boto:Token: None 2022-09-03T05:37:30.950 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj 2022-09-03T05:37:30.951 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cwg/mcrsNu8VG6QqEXFUZA3iKkI= 2022-09-03T05:37:30.951 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cwg/mcrsNu8VG6QqEXFUZA3iKkI=', 'Content-Length': '0'} 2022-09-03T05:37:30.955 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007da2714270eb8dd5-006312e81a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.956 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.956 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.957 DEBUG:boto:Method: HEAD 2022-09-03T05:37:30.957 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:30.958 DEBUG:boto:Data: 2022-09-03T05:37:30.958 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.959 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:30.959 DEBUG:boto:Port: 8000 2022-09-03T05:37:30.960 DEBUG:boto:Params: {} 2022-09-03T05:37:30.961 DEBUG:boto:Token: None 2022-09-03T05:37:30.961 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj 2022-09-03T05:37:30.962 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cwg/mcrsNu8VG6QqEXFUZA3iKkI= 2022-09-03T05:37:30.962 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cwg/mcrsNu8VG6QqEXFUZA3iKkI=', 'Content-Length': '0'} 2022-09-03T05:37:30.966 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000001409c6eecba065f-006312e81a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:30.967 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:30.967 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:30.968 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:30.968 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.969 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.969 DEBUG:boto:Method: GET 2022-09-03T05:37:30.970 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.970 DEBUG:boto:Data: 2022-09-03T05:37:30.971 DEBUG:boto:Headers: {} 2022-09-03T05:37:30.971 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:30.972 DEBUG:boto:Port: 8001 2022-09-03T05:37:30.973 DEBUG:boto:Params: {} 2022-09-03T05:37:30.973 DEBUG:boto:Token: None 2022-09-03T05:37:30.974 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:30 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:30.975 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2rh2LAu+PM4HCNfxBJOBXHeSJqU= 2022-09-03T05:37:30.975 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2rh2LAu+PM4HCNfxBJOBXHeSJqU=', 'Content-Length': '0'} 2022-09-03T05:37:30.977 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dfbd9dc1348d146c-006312e81a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.018 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.019 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.020 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.020 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.021 DEBUG:boto:Method: GET 2022-09-03T05:37:31.022 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.023 DEBUG:boto:Data: 2022-09-03T05:37:31.024 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.025 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.026 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.027 DEBUG:boto:Params: {} 2022-09-03T05:37:31.027 DEBUG:boto:Token: None 2022-09-03T05:37:31.028 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.029 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HuDtub/lTnUkNs5n6K1xW/t5kUA= 2022-09-03T05:37:31.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HuDtub/lTnUkNs5n6K1xW/t5kUA=', 'Content-Length': '0'} 2022-09-03T05:37:31.032 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f03b560d4e3390f4-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.033 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:31.034 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:31.035 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.035 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.036 DEBUG:boto:Method: GET 2022-09-03T05:37:31.037 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.037 DEBUG:boto:Data: 2022-09-03T05:37:31.038 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.038 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.039 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.039 DEBUG:boto:Params: {} 2022-09-03T05:37:31.040 DEBUG:boto:Token: None 2022-09-03T05:37:31.040 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/ 2022-09-03T05:37:31.042 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k= 2022-09-03T05:37:31.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k=', 'Content-Length': '0'} 2022-09-03T05:37:31.046 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f2278255b72137bc-006312e81b-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.047 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:31.048 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:31.049 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:31.049 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.050 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.051 DEBUG:boto:Method: GET 2022-09-03T05:37:31.051 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.052 DEBUG:boto:Data: 2022-09-03T05:37:31.053 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.053 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.054 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.055 DEBUG:boto:Params: {} 2022-09-03T05:37:31.056 DEBUG:boto:Token: None 2022-09-03T05:37:31.056 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/ 2022-09-03T05:37:31.057 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k= 2022-09-03T05:37:31.058 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k=', 'Content-Length': '0'} 2022-09-03T05:37:31.061 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000af5baf2f9b28b34a-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.062 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:31.062 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:31.063 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.064 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.065 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.066 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.066 DEBUG:boto:Method: GET 2022-09-03T05:37:31.067 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.067 DEBUG:boto:Data: 2022-09-03T05:37:31.068 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.068 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.069 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.070 DEBUG:boto:Params: {} 2022-09-03T05:37:31.071 DEBUG:boto:Token: None 2022-09-03T05:37:31.072 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.072 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EtCksftSm0brQ06Rxhl+9y8pIOs= 2022-09-03T05:37:31.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EtCksftSm0brQ06Rxhl+9y8pIOs=', 'Content-Length': '0'} 2022-09-03T05:37:31.076 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bee26e538bebb145-006312e81b-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.078 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.079 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.080 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.081 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.082 DEBUG:boto:Method: GET 2022-09-03T05:37:31.083 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.084 DEBUG:boto:Data: 2022-09-03T05:37:31.085 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.086 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.087 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.088 DEBUG:boto:Params: {} 2022-09-03T05:37:31.089 DEBUG:boto:Token: None 2022-09-03T05:37:31.090 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.090 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EtCksftSm0brQ06Rxhl+9y8pIOs= 2022-09-03T05:37:31.091 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EtCksftSm0brQ06Rxhl+9y8pIOs=', 'Content-Length': '0'} 2022-09-03T05:37:31.093 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000058b488908612ccda-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.094 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.095 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.095 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.096 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.096 DEBUG:boto:Method: HEAD 2022-09-03T05:37:31.097 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.097 DEBUG:boto:Data: 2022-09-03T05:37:31.098 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.099 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.099 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.100 DEBUG:boto:Params: {} 2022-09-03T05:37:31.100 DEBUG:boto:Token: None 2022-09-03T05:37:31.101 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.102 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:26HxmJRvikhoPVxP8eKxe+cjaOU= 2022-09-03T05:37:31.102 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:26HxmJRvikhoPVxP8eKxe+cjaOU=', 'Content-Length': '0'} 2022-09-03T05:37:31.105 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004832834eb7cc70d7-006312e81b-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.105 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.106 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.107 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.107 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.108 DEBUG:boto:Method: HEAD 2022-09-03T05:37:31.108 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.109 DEBUG:boto:Data: 2022-09-03T05:37:31.109 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.110 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.110 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.111 DEBUG:boto:Params: {} 2022-09-03T05:37:31.111 DEBUG:boto:Token: None 2022-09-03T05:37:31.112 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.113 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:26HxmJRvikhoPVxP8eKxe+cjaOU= 2022-09-03T05:37:31.113 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:26HxmJRvikhoPVxP8eKxe+cjaOU=', 'Content-Length': '0'} 2022-09-03T05:37:31.116 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007cd44ee5b9c048c5-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.117 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:31.118 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.118 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.119 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.120 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.120 DEBUG:boto:Method: GET 2022-09-03T05:37:31.121 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.121 DEBUG:boto:Data: 2022-09-03T05:37:31.122 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.123 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.123 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.124 DEBUG:boto:Params: {} 2022-09-03T05:37:31.124 DEBUG:boto:Token: None 2022-09-03T05:37:31.125 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.126 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EtCksftSm0brQ06Rxhl+9y8pIOs= 2022-09-03T05:37:31.126 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EtCksftSm0brQ06Rxhl+9y8pIOs=', 'Content-Length': '0'} 2022-09-03T05:37:31.128 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fa4b160545fd00f6-006312e81b-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.170 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.171 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.172 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.173 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.173 DEBUG:boto:Method: GET 2022-09-03T05:37:31.174 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.174 DEBUG:boto:Data: 2022-09-03T05:37:31.175 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.175 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.176 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.177 DEBUG:boto:Params: {} 2022-09-03T05:37:31.177 DEBUG:boto:Token: None 2022-09-03T05:37:31.178 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U 2022-09-03T05:37:31.179 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EtCksftSm0brQ06Rxhl+9y8pIOs= 2022-09-03T05:37:31.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EtCksftSm0brQ06Rxhl+9y8pIOs=', 'Content-Length': '0'} 2022-09-03T05:37:31.182 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.lPacc7Eu6BJ5IOsSyLccq2.KRUK24U'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b64c51bf3fe76809-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.183 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:31.184 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:31.184 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.185 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.186 DEBUG:boto:Method: GET 2022-09-03T05:37:31.186 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.187 DEBUG:boto:Data: 2022-09-03T05:37:31.188 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.189 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.189 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.190 DEBUG:boto:Params: {} 2022-09-03T05:37:31.190 DEBUG:boto:Token: None 2022-09-03T05:37:31.191 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/ 2022-09-03T05:37:31.192 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k= 2022-09-03T05:37:31.193 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k=', 'Content-Length': '0'} 2022-09-03T05:37:31.196 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eb0d85ffdd7e7f3d-006312e81b-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.198 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:31.199 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:31.199 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:31.200 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.201 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.201 DEBUG:boto:Method: GET 2022-09-03T05:37:31.202 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.203 DEBUG:boto:Data: 2022-09-03T05:37:31.203 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.204 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.204 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.205 DEBUG:boto:Params: {} 2022-09-03T05:37:31.206 DEBUG:boto:Token: None 2022-09-03T05:37:31.207 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/ 2022-09-03T05:37:31.207 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k= 2022-09-03T05:37:31.209 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k=', 'Content-Length': '0'} 2022-09-03T05:37:31.212 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000015e4c5f965703eba-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.213 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:31.214 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.214 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.215 DEBUG:boto:Method: HEAD 2022-09-03T05:37:31.216 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:31.216 DEBUG:boto:Data: 2022-09-03T05:37:31.217 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.217 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.218 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.219 DEBUG:boto:Params: {} 2022-09-03T05:37:31.219 DEBUG:boto:Token: None 2022-09-03T05:37:31.220 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj 2022-09-03T05:37:31.220 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TftGgiPWD4YKEEh7bb9vcEd6iM8= 2022-09-03T05:37:31.221 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TftGgiPWD4YKEEh7bb9vcEd6iM8=', 'Content-Length': '0'} 2022-09-03T05:37:31.224 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001109f996079a3a7c-006312e81b-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.225 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.226 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.227 DEBUG:boto:Method: HEAD 2022-09-03T05:37:31.227 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:31.228 DEBUG:boto:Data: 2022-09-03T05:37:31.229 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.230 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.230 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.231 DEBUG:boto:Params: {} 2022-09-03T05:37:31.232 DEBUG:boto:Token: None 2022-09-03T05:37:31.232 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj 2022-09-03T05:37:31.233 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TftGgiPWD4YKEEh7bb9vcEd6iM8= 2022-09-03T05:37:31.233 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TftGgiPWD4YKEEh7bb9vcEd6iM8=', 'Content-Length': '0'} 2022-09-03T05:37:31.237 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003dc51a2bb75ed760-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.238 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:31.239 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.240 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.240 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.241 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.241 DEBUG:boto:Method: GET 2022-09-03T05:37:31.242 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.242 DEBUG:boto:Data: 2022-09-03T05:37:31.243 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.243 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.244 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.245 DEBUG:boto:Params: {} 2022-09-03T05:37:31.246 DEBUG:boto:Token: None 2022-09-03T05:37:31.246 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.247 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HuDtub/lTnUkNs5n6K1xW/t5kUA= 2022-09-03T05:37:31.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HuDtub/lTnUkNs5n6K1xW/t5kUA=', 'Content-Length': '0'} 2022-09-03T05:37:31.250 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a6cb60f3c9695929-006312e81b-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.294 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.295 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.296 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.296 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.297 DEBUG:boto:Method: GET 2022-09-03T05:37:31.297 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.298 DEBUG:boto:Data: 2022-09-03T05:37:31.298 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.299 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.299 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.300 DEBUG:boto:Params: {} 2022-09-03T05:37:31.301 DEBUG:boto:Token: None 2022-09-03T05:37:31.301 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.302 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HuDtub/lTnUkNs5n6K1xW/t5kUA= 2022-09-03T05:37:31.303 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HuDtub/lTnUkNs5n6K1xW/t5kUA=', 'Content-Length': '0'} 2022-09-03T05:37:31.305 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005b50ccfcbe421c65-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.306 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:31.307 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:31.307 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.308 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.308 DEBUG:boto:Method: GET 2022-09-03T05:37:31.309 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.309 DEBUG:boto:Data: 2022-09-03T05:37:31.310 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.310 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.311 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.311 DEBUG:boto:Params: {} 2022-09-03T05:37:31.312 DEBUG:boto:Token: None 2022-09-03T05:37:31.313 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/ 2022-09-03T05:37:31.314 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k= 2022-09-03T05:37:31.315 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k=', 'Content-Length': '0'} 2022-09-03T05:37:31.319 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008ff3de05e71d9eef-006312e81b-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.319 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:31.320 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:31.321 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:31.322 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.322 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.323 DEBUG:boto:Method: GET 2022-09-03T05:37:31.324 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.325 DEBUG:boto:Data: 2022-09-03T05:37:31.325 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.326 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.326 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.327 DEBUG:boto:Params: {} 2022-09-03T05:37:31.328 DEBUG:boto:Token: None 2022-09-03T05:37:31.328 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/ 2022-09-03T05:37:31.329 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k= 2022-09-03T05:37:31.330 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k=', 'Content-Length': '0'} 2022-09-03T05:37:31.334 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001da4a34dd1287e9a-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.334 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:31.335 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:31.336 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.336 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.337 DEBUG:boto:path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.337 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.339 DEBUG:boto:Method: GET 2022-09-03T05:37:31.340 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.341 DEBUG:boto:Data: 2022-09-03T05:37:31.341 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.342 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.343 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.344 DEBUG:boto:Params: {} 2022-09-03T05:37:31.344 DEBUG:boto:Token: None 2022-09-03T05:37:31.345 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.346 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XWQAR1rkKHUXIAuupqdmKgpLsf4= 2022-09-03T05:37:31.347 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XWQAR1rkKHUXIAuupqdmKgpLsf4=', 'Content-Length': '0'} 2022-09-03T05:37:31.350 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000017bffe491c9f361c-006312e81b-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.351 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.352 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.353 DEBUG:boto:path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.353 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.354 DEBUG:boto:Method: GET 2022-09-03T05:37:31.355 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.355 DEBUG:boto:Data: 2022-09-03T05:37:31.356 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.356 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.357 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.358 DEBUG:boto:Params: {} 2022-09-03T05:37:31.358 DEBUG:boto:Token: None 2022-09-03T05:37:31.359 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.360 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XWQAR1rkKHUXIAuupqdmKgpLsf4= 2022-09-03T05:37:31.361 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XWQAR1rkKHUXIAuupqdmKgpLsf4=', 'Content-Length': '0'} 2022-09-03T05:37:31.365 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006986f6349101bd29-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.366 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.367 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.368 DEBUG:boto:path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.368 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.369 DEBUG:boto:Method: HEAD 2022-09-03T05:37:31.370 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.371 DEBUG:boto:Data: 2022-09-03T05:37:31.372 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.373 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.373 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.374 DEBUG:boto:Params: {} 2022-09-03T05:37:31.375 DEBUG:boto:Token: None 2022-09-03T05:37:31.375 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.376 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FQC7hyA945o8VSEpPeeZDq+50+E= 2022-09-03T05:37:31.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FQC7hyA945o8VSEpPeeZDq+50+E=', 'Content-Length': '0'} 2022-09-03T05:37:31.379 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a2dc75c3e3ffc940-006312e81b-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.380 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.381 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.382 DEBUG:boto:path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.382 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.383 DEBUG:boto:Method: HEAD 2022-09-03T05:37:31.384 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.385 DEBUG:boto:Data: 2022-09-03T05:37:31.386 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.395 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.395 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.396 DEBUG:boto:Params: {} 2022-09-03T05:37:31.396 DEBUG:boto:Token: None 2022-09-03T05:37:31.397 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.398 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FQC7hyA945o8VSEpPeeZDq+50+E= 2022-09-03T05:37:31.398 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FQC7hyA945o8VSEpPeeZDq+50+E=', 'Content-Length': '0'} 2022-09-03T05:37:31.400 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000edfde9d9ba13ab9f-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.401 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:31.402 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.402 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.403 DEBUG:boto:path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.403 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.404 DEBUG:boto:Method: GET 2022-09-03T05:37:31.404 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.405 DEBUG:boto:Data: 2022-09-03T05:37:31.405 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.406 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.407 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.407 DEBUG:boto:Params: {} 2022-09-03T05:37:31.408 DEBUG:boto:Token: None 2022-09-03T05:37:31.408 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.409 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XWQAR1rkKHUXIAuupqdmKgpLsf4= 2022-09-03T05:37:31.410 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XWQAR1rkKHUXIAuupqdmKgpLsf4=', 'Content-Length': '0'} 2022-09-03T05:37:31.412 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bb0214721f134d9c-006312e81b-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.412 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.413 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.414 DEBUG:boto:path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.414 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.415 DEBUG:boto:Method: GET 2022-09-03T05:37:31.415 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.416 DEBUG:boto:Data: 2022-09-03T05:37:31.416 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.417 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.417 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.418 DEBUG:boto:Params: {} 2022-09-03T05:37:31.418 DEBUG:boto:Token: None 2022-09-03T05:37:31.419 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt 2022-09-03T05:37:31.420 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XWQAR1rkKHUXIAuupqdmKgpLsf4= 2022-09-03T05:37:31.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XWQAR1rkKHUXIAuupqdmKgpLsf4=', 'Content-Length': '0'} 2022-09-03T05:37:31.423 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'NkMDAxRk7LkNPj1mtl1YEsdjIaTowHt'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000050498fa059c3fed8-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.423 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:31.424 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:31.425 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.426 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.426 DEBUG:boto:Method: GET 2022-09-03T05:37:31.427 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.427 DEBUG:boto:Data: 2022-09-03T05:37:31.428 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.428 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.429 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.430 DEBUG:boto:Params: {} 2022-09-03T05:37:31.430 DEBUG:boto:Token: None 2022-09-03T05:37:31.431 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/ 2022-09-03T05:37:31.431 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k= 2022-09-03T05:37:31.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k=', 'Content-Length': '0'} 2022-09-03T05:37:31.435 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000db20bfd299a3dd40-006312e81b-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.435 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:31.436 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:31.437 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:31.438 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.438 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.439 DEBUG:boto:Method: GET 2022-09-03T05:37:31.439 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.440 DEBUG:boto:Data: 2022-09-03T05:37:31.440 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.441 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.441 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.442 DEBUG:boto:Params: {} 2022-09-03T05:37:31.443 DEBUG:boto:Token: None 2022-09-03T05:37:31.443 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/ 2022-09-03T05:37:31.444 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k= 2022-09-03T05:37:31.445 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k=', 'Content-Length': '0'} 2022-09-03T05:37:31.448 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fa4d6d2e79442dd2-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.490 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:31.491 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.492 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.492 DEBUG:boto:Method: HEAD 2022-09-03T05:37:31.493 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:31.494 DEBUG:boto:Data: 2022-09-03T05:37:31.495 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.495 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.496 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.497 DEBUG:boto:Params: {} 2022-09-03T05:37:31.497 DEBUG:boto:Token: None 2022-09-03T05:37:31.498 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj 2022-09-03T05:37:31.499 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TftGgiPWD4YKEEh7bb9vcEd6iM8= 2022-09-03T05:37:31.499 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TftGgiPWD4YKEEh7bb9vcEd6iM8=', 'Content-Length': '0'} 2022-09-03T05:37:31.503 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000474cf25f7c463dd6-006312e81b-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.504 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.505 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.505 DEBUG:boto:Method: HEAD 2022-09-03T05:37:31.506 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:31.507 DEBUG:boto:Data: 2022-09-03T05:37:31.507 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.508 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.508 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.509 DEBUG:boto:Params: {} 2022-09-03T05:37:31.510 DEBUG:boto:Token: None 2022-09-03T05:37:31.510 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj 2022-09-03T05:37:31.512 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TftGgiPWD4YKEEh7bb9vcEd6iM8= 2022-09-03T05:37:31.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TftGgiPWD4YKEEh7bb9vcEd6iM8=', 'Content-Length': '0'} 2022-09-03T05:37:31.517 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009be3ef0f333a746d-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.518 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:31.519 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.519 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.520 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.520 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.521 DEBUG:boto:Method: GET 2022-09-03T05:37:31.522 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.523 DEBUG:boto:Data: 2022-09-03T05:37:31.523 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.524 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.525 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.526 DEBUG:boto:Params: {} 2022-09-03T05:37:31.528 DEBUG:boto:Token: None 2022-09-03T05:37:31.529 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.530 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HuDtub/lTnUkNs5n6K1xW/t5kUA= 2022-09-03T05:37:31.531 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HuDtub/lTnUkNs5n6K1xW/t5kUA=', 'Content-Length': '0'} 2022-09-03T05:37:31.535 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000af99d4fe7a788e60-006312e81b-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.578 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.580 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.581 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.582 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.583 DEBUG:boto:Method: GET 2022-09-03T05:37:31.583 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.584 DEBUG:boto:Data: 2022-09-03T05:37:31.585 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.586 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.587 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.588 DEBUG:boto:Params: {} 2022-09-03T05:37:31.588 DEBUG:boto:Token: None 2022-09-03T05:37:31.589 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.590 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HuDtub/lTnUkNs5n6K1xW/t5kUA= 2022-09-03T05:37:31.591 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HuDtub/lTnUkNs5n6K1xW/t5kUA=', 'Content-Length': '0'} 2022-09-03T05:37:31.593 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000496ecc976cb9fc36-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.594 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:31.594 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:31.595 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.595 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.596 DEBUG:boto:Method: GET 2022-09-03T05:37:31.596 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.597 DEBUG:boto:Data: 2022-09-03T05:37:31.597 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.598 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.599 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.599 DEBUG:boto:Params: {} 2022-09-03T05:37:31.600 DEBUG:boto:Token: None 2022-09-03T05:37:31.600 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/ 2022-09-03T05:37:31.601 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k= 2022-09-03T05:37:31.602 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k=', 'Content-Length': '0'} 2022-09-03T05:37:31.605 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008b284396543be749-006312e81b-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.606 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:31.608 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:31.609 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:31.610 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.611 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.612 DEBUG:boto:Method: GET 2022-09-03T05:37:31.613 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.614 DEBUG:boto:Data: 2022-09-03T05:37:31.614 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.615 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.615 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.616 DEBUG:boto:Params: {} 2022-09-03T05:37:31.617 DEBUG:boto:Token: None 2022-09-03T05:37:31.617 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/ 2022-09-03T05:37:31.618 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k= 2022-09-03T05:37:31.619 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k=', 'Content-Length': '0'} 2022-09-03T05:37:31.622 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002d6b54278d173bcd-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.623 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:31.624 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:31.625 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.626 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.627 DEBUG:boto:path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.628 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.629 DEBUG:boto:Method: GET 2022-09-03T05:37:31.630 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.631 DEBUG:boto:Data: 2022-09-03T05:37:31.632 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.633 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.634 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.635 DEBUG:boto:Params: {} 2022-09-03T05:37:31.636 DEBUG:boto:Token: None 2022-09-03T05:37:31.637 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.638 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OypZfXhypgv1yvpd1knFw5/XKlU= 2022-09-03T05:37:31.639 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OypZfXhypgv1yvpd1knFw5/XKlU=', 'Content-Length': '0'} 2022-09-03T05:37:31.642 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'QfRyggjm1gZ7H.LeMCey0hHybmVWONG'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000077ab08c61cef9e39-006312e81b-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.643 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.643 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.644 DEBUG:boto:path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.645 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.646 DEBUG:boto:Method: GET 2022-09-03T05:37:31.647 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.648 DEBUG:boto:Data: 2022-09-03T05:37:31.649 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.650 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.651 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.652 DEBUG:boto:Params: {} 2022-09-03T05:37:31.653 DEBUG:boto:Token: None 2022-09-03T05:37:31.654 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.655 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OypZfXhypgv1yvpd1knFw5/XKlU= 2022-09-03T05:37:31.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OypZfXhypgv1yvpd1knFw5/XKlU=', 'Content-Length': '0'} 2022-09-03T05:37:31.658 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'QfRyggjm1gZ7H.LeMCey0hHybmVWONG'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d186e3b617875c04-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.659 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.660 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.661 DEBUG:boto:path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.662 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.663 DEBUG:boto:Method: HEAD 2022-09-03T05:37:31.664 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.665 DEBUG:boto:Data: 2022-09-03T05:37:31.666 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.667 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.668 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.669 DEBUG:boto:Params: {} 2022-09-03T05:37:31.670 DEBUG:boto:Token: None 2022-09-03T05:37:31.671 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.672 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D2VJ3mmpTzo9EfVpib78GY00AYs= 2022-09-03T05:37:31.673 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D2VJ3mmpTzo9EfVpib78GY00AYs=', 'Content-Length': '0'} 2022-09-03T05:37:31.676 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'QfRyggjm1gZ7H.LeMCey0hHybmVWONG'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dc1e413b9c24019e-006312e81b-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.677 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.678 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.679 DEBUG:boto:path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.680 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.681 DEBUG:boto:Method: HEAD 2022-09-03T05:37:31.682 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.682 DEBUG:boto:Data: 2022-09-03T05:37:31.683 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.683 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.684 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.684 DEBUG:boto:Params: {} 2022-09-03T05:37:31.685 DEBUG:boto:Token: None 2022-09-03T05:37:31.686 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.686 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D2VJ3mmpTzo9EfVpib78GY00AYs= 2022-09-03T05:37:31.687 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D2VJ3mmpTzo9EfVpib78GY00AYs=', 'Content-Length': '0'} 2022-09-03T05:37:31.689 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'QfRyggjm1gZ7H.LeMCey0hHybmVWONG'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000025f34c80f87aca83-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.690 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:31.690 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.691 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.691 DEBUG:boto:path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.692 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.692 DEBUG:boto:Method: GET 2022-09-03T05:37:31.693 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.694 DEBUG:boto:Data: 2022-09-03T05:37:31.694 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.695 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.695 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.696 DEBUG:boto:Params: {} 2022-09-03T05:37:31.696 DEBUG:boto:Token: None 2022-09-03T05:37:31.697 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.698 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OypZfXhypgv1yvpd1knFw5/XKlU= 2022-09-03T05:37:31.698 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OypZfXhypgv1yvpd1knFw5/XKlU=', 'Content-Length': '0'} 2022-09-03T05:37:31.701 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'QfRyggjm1gZ7H.LeMCey0hHybmVWONG'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002b96eccfd4cf9422-006312e81b-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.702 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.702 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.703 DEBUG:boto:path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.703 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.704 DEBUG:boto:Method: GET 2022-09-03T05:37:31.704 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.705 DEBUG:boto:Data: 2022-09-03T05:37:31.705 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.706 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.707 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.707 DEBUG:boto:Params: {} 2022-09-03T05:37:31.708 DEBUG:boto:Token: None 2022-09-03T05:37:31.708 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=QfRyggjm1gZ7H.LeMCey0hHybmVWONG 2022-09-03T05:37:31.709 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OypZfXhypgv1yvpd1knFw5/XKlU= 2022-09-03T05:37:31.710 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OypZfXhypgv1yvpd1knFw5/XKlU=', 'Content-Length': '0'} 2022-09-03T05:37:31.712 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'QfRyggjm1gZ7H.LeMCey0hHybmVWONG'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b831b6372e9fdcab-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.758 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:31.760 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:31.761 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.762 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.762 DEBUG:boto:Method: GET 2022-09-03T05:37:31.763 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.764 DEBUG:boto:Data: 2022-09-03T05:37:31.765 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.766 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.767 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.767 DEBUG:boto:Params: {} 2022-09-03T05:37:31.768 DEBUG:boto:Token: None 2022-09-03T05:37:31.769 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/ 2022-09-03T05:37:31.771 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k= 2022-09-03T05:37:31.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k=', 'Content-Length': '0'} 2022-09-03T05:37:31.776 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002cb0a839d126ffe7-006312e81b-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.777 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:31.779 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:31.780 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:31.782 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.783 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.783 DEBUG:boto:Method: GET 2022-09-03T05:37:31.784 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.784 DEBUG:boto:Data: 2022-09-03T05:37:31.785 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.786 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.786 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.787 DEBUG:boto:Params: {} 2022-09-03T05:37:31.788 DEBUG:boto:Token: None 2022-09-03T05:37:31.789 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/ 2022-09-03T05:37:31.791 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k= 2022-09-03T05:37:31.793 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k=', 'Content-Length': '0'} 2022-09-03T05:37:31.796 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000389eca971f92d3ab-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.798 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:31.799 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.800 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.800 DEBUG:boto:Method: HEAD 2022-09-03T05:37:31.801 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:31.802 DEBUG:boto:Data: 2022-09-03T05:37:31.803 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.804 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.804 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.805 DEBUG:boto:Params: {} 2022-09-03T05:37:31.805 DEBUG:boto:Token: None 2022-09-03T05:37:31.806 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj 2022-09-03T05:37:31.807 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TftGgiPWD4YKEEh7bb9vcEd6iM8= 2022-09-03T05:37:31.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TftGgiPWD4YKEEh7bb9vcEd6iM8=', 'Content-Length': '0'} 2022-09-03T05:37:31.811 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ff348b33286f6f4c-006312e81b-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.812 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.813 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.814 DEBUG:boto:Method: HEAD 2022-09-03T05:37:31.814 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:31.815 DEBUG:boto:Data: 2022-09-03T05:37:31.816 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.816 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.817 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.817 DEBUG:boto:Params: {} 2022-09-03T05:37:31.818 DEBUG:boto:Token: None 2022-09-03T05:37:31.819 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj 2022-09-03T05:37:31.819 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TftGgiPWD4YKEEh7bb9vcEd6iM8= 2022-09-03T05:37:31.820 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TftGgiPWD4YKEEh7bb9vcEd6iM8=', 'Content-Length': '0'} 2022-09-03T05:37:31.824 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a645b761bd009e6c-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.825 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:31.825 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.826 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.827 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.827 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.828 DEBUG:boto:Method: GET 2022-09-03T05:37:31.828 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.829 DEBUG:boto:Data: 2022-09-03T05:37:31.829 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.830 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.830 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.831 DEBUG:boto:Params: {} 2022-09-03T05:37:31.832 DEBUG:boto:Token: None 2022-09-03T05:37:31.832 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.833 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HuDtub/lTnUkNs5n6K1xW/t5kUA= 2022-09-03T05:37:31.834 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HuDtub/lTnUkNs5n6K1xW/t5kUA=', 'Content-Length': '0'} 2022-09-03T05:37:31.836 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f7bccd6539b22017-006312e81b-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.878 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.879 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.880 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.880 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.881 DEBUG:boto:Method: GET 2022-09-03T05:37:31.881 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.882 DEBUG:boto:Data: 2022-09-03T05:37:31.882 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.883 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.884 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.884 DEBUG:boto:Params: {} 2022-09-03T05:37:31.885 DEBUG:boto:Token: None 2022-09-03T05:37:31.885 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:31.886 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HuDtub/lTnUkNs5n6K1xW/t5kUA= 2022-09-03T05:37:31.887 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HuDtub/lTnUkNs5n6K1xW/t5kUA=', 'Content-Length': '0'} 2022-09-03T05:37:31.889 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000774a96e1a6c705ef-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.892 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:31.893 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:31.893 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.894 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.895 DEBUG:boto:Method: GET 2022-09-03T05:37:31.895 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.896 DEBUG:boto:Data: 2022-09-03T05:37:31.896 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.897 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.898 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.898 DEBUG:boto:Params: {} 2022-09-03T05:37:31.899 DEBUG:boto:Token: None 2022-09-03T05:37:31.900 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/ 2022-09-03T05:37:31.901 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k= 2022-09-03T05:37:31.901 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k=', 'Content-Length': '0'} 2022-09-03T05:37:31.904 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000698f613be23583dd-006312e81b-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.905 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:31.906 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:31.907 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:31.907 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.908 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.909 DEBUG:boto:Method: GET 2022-09-03T05:37:31.910 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:31.910 DEBUG:boto:Data: 2022-09-03T05:37:31.911 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.911 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.912 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.913 DEBUG:boto:Params: {} 2022-09-03T05:37:31.914 DEBUG:boto:Token: None 2022-09-03T05:37:31.914 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/ 2022-09-03T05:37:31.915 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k= 2022-09-03T05:37:31.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:95VJH7dDIZotn8/LdDTnnjrf25k=', 'Content-Length': '0'} 2022-09-03T05:37:31.920 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001f17f8ebc14f19c3-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.921 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:31.922 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:31.923 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.924 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.924 DEBUG:boto:path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:31.925 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:31.926 DEBUG:boto:Method: GET 2022-09-03T05:37:31.926 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:31.927 DEBUG:boto:Data: 2022-09-03T05:37:31.928 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.928 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.929 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.929 DEBUG:boto:Params: {} 2022-09-03T05:37:31.931 DEBUG:boto:Token: None 2022-09-03T05:37:31.932 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:31.932 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yHhTQSxI+CihwOjuJJYLTnuHvyg= 2022-09-03T05:37:31.933 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yHhTQSxI+CihwOjuJJYLTnuHvyg=', 'Content-Length': '0'} 2022-09-03T05:37:31.935 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '94bPO72IhlWQEGwklU15Pvpr-qRv.lm'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003b86811d08f3e9bc-006312e81b-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.936 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.937 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.938 DEBUG:boto:path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:31.938 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:31.939 DEBUG:boto:Method: GET 2022-09-03T05:37:31.940 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:31.940 DEBUG:boto:Data: 2022-09-03T05:37:31.941 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.942 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.942 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.943 DEBUG:boto:Params: {} 2022-09-03T05:37:31.943 DEBUG:boto:Token: None 2022-09-03T05:37:31.944 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:31.945 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yHhTQSxI+CihwOjuJJYLTnuHvyg= 2022-09-03T05:37:31.946 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yHhTQSxI+CihwOjuJJYLTnuHvyg=', 'Content-Length': '0'} 2022-09-03T05:37:31.948 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '94bPO72IhlWQEGwklU15Pvpr-qRv.lm'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fb939eb2c2e54cee-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.950 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.950 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.951 DEBUG:boto:path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:31.952 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:31.952 DEBUG:boto:Method: HEAD 2022-09-03T05:37:31.953 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:31.954 DEBUG:boto:Data: 2022-09-03T05:37:31.954 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.955 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:31.955 DEBUG:boto:Port: 8001 2022-09-03T05:37:31.956 DEBUG:boto:Params: {} 2022-09-03T05:37:31.957 DEBUG:boto:Token: None 2022-09-03T05:37:31.957 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:31.958 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wbKdJ8V9h0elgwL7bWoMQxYxQpg= 2022-09-03T05:37:31.959 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wbKdJ8V9h0elgwL7bWoMQxYxQpg=', 'Content-Length': '0'} 2022-09-03T05:37:31.961 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '94bPO72IhlWQEGwklU15Pvpr-qRv.lm'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000095149ad6a332e9f6-006312e81b-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.963 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.964 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.965 DEBUG:boto:path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:31.966 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:31.967 DEBUG:boto:Method: HEAD 2022-09-03T05:37:31.969 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:31.970 DEBUG:boto:Data: 2022-09-03T05:37:31.971 DEBUG:boto:Headers: {} 2022-09-03T05:37:31.973 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:31.974 DEBUG:boto:Port: 8000 2022-09-03T05:37:31.976 DEBUG:boto:Params: {} 2022-09-03T05:37:31.977 DEBUG:boto:Token: None 2022-09-03T05:37:31.979 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:31 GMT /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:31.982 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wbKdJ8V9h0elgwL7bWoMQxYxQpg= 2022-09-03T05:37:31.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wbKdJ8V9h0elgwL7bWoMQxYxQpg=', 'Content-Length': '0'} 2022-09-03T05:37:31.986 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '94bPO72IhlWQEGwklU15Pvpr-qRv.lm'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000010dc09e0f8a950fb-006312e81b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:31.990 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:31.991 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:31.992 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:31.998 DEBUG:boto:path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:31.999 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:32.000 DEBUG:boto:Method: GET 2022-09-03T05:37:32.001 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:32.002 DEBUG:boto:Data: 2022-09-03T05:37:32.009 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.011 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:32.011 DEBUG:boto:Port: 8001 2022-09-03T05:37:32.012 DEBUG:boto:Params: {} 2022-09-03T05:37:32.013 DEBUG:boto:Token: None 2022-09-03T05:37:32.027 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:32.028 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QfJuq6GVrnkOJbyZLRNUdWNMMgk= 2022-09-03T05:37:32.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QfJuq6GVrnkOJbyZLRNUdWNMMgk=', 'Content-Length': '0'} 2022-09-03T05:37:32.032 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '94bPO72IhlWQEGwklU15Pvpr-qRv.lm'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007e85569ecde030fb-006312e81c-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.033 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.034 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.035 DEBUG:boto:path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:32.035 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:32.036 DEBUG:boto:Method: GET 2022-09-03T05:37:32.037 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:32.042 DEBUG:boto:Data: 2022-09-03T05:37:32.046 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.047 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:32.048 DEBUG:boto:Port: 8000 2022-09-03T05:37:32.049 DEBUG:boto:Params: {} 2022-09-03T05:37:32.052 DEBUG:boto:Token: None 2022-09-03T05:37:32.054 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=94bPO72IhlWQEGwklU15Pvpr-qRv.lm 2022-09-03T05:37:32.055 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QfJuq6GVrnkOJbyZLRNUdWNMMgk= 2022-09-03T05:37:32.057 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QfJuq6GVrnkOJbyZLRNUdWNMMgk=', 'Content-Length': '0'} 2022-09-03T05:37:32.063 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '94bPO72IhlWQEGwklU15Pvpr-qRv.lm'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f0bf4c257b46fc6b-006312e81c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.065 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:32.066 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:32.067 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.068 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.069 DEBUG:boto:Method: GET 2022-09-03T05:37:32.070 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.071 DEBUG:boto:Data: 2022-09-03T05:37:32.072 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.073 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:32.074 DEBUG:boto:Port: 8001 2022-09-03T05:37:32.075 DEBUG:boto:Params: {} 2022-09-03T05:37:32.076 DEBUG:boto:Token: None 2022-09-03T05:37:32.078 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/ 2022-09-03T05:37:32.079 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc= 2022-09-03T05:37:32.080 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc=', 'Content-Length': '0'} 2022-09-03T05:37:32.084 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000753fb202fc82be60-006312e81c-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.085 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:32.087 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:32.087 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:32.088 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.088 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.089 DEBUG:boto:Method: GET 2022-09-03T05:37:32.090 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.091 DEBUG:boto:Data: 2022-09-03T05:37:32.092 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.093 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:32.094 DEBUG:boto:Port: 8000 2022-09-03T05:37:32.095 DEBUG:boto:Params: {} 2022-09-03T05:37:32.096 DEBUG:boto:Token: None 2022-09-03T05:37:32.097 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/ 2022-09-03T05:37:32.098 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc= 2022-09-03T05:37:32.099 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc=', 'Content-Length': '0'} 2022-09-03T05:37:32.103 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ab036da4b01b9ba7-006312e81c-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.114 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:32.116 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.117 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.121 DEBUG:boto:Method: HEAD 2022-09-03T05:37:32.122 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:32.123 DEBUG:boto:Data: 2022-09-03T05:37:32.125 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.126 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:32.127 DEBUG:boto:Port: 8001 2022-09-03T05:37:32.128 DEBUG:boto:Params: {} 2022-09-03T05:37:32.129 DEBUG:boto:Token: None 2022-09-03T05:37:32.133 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj 2022-09-03T05:37:32.135 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QJW5UOc94YcPdLn9jBfvJw4zbaI= 2022-09-03T05:37:32.138 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QJW5UOc94YcPdLn9jBfvJw4zbaI=', 'Content-Length': '0'} 2022-09-03T05:37:32.145 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dac881b55858cfde-006312e81c-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.147 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.149 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.150 DEBUG:boto:Method: HEAD 2022-09-03T05:37:32.150 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:32.151 DEBUG:boto:Data: 2022-09-03T05:37:32.152 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.153 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:32.154 DEBUG:boto:Port: 8000 2022-09-03T05:37:32.155 DEBUG:boto:Params: {} 2022-09-03T05:37:32.156 DEBUG:boto:Token: None 2022-09-03T05:37:32.157 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj 2022-09-03T05:37:32.159 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QJW5UOc94YcPdLn9jBfvJw4zbaI= 2022-09-03T05:37:32.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QJW5UOc94YcPdLn9jBfvJw4zbaI=', 'Content-Length': '0'} 2022-09-03T05:37:32.165 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000047207e3e2fff8779-006312e81c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.166 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:32.170 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.171 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.172 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.172 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.173 DEBUG:boto:Method: GET 2022-09-03T05:37:32.174 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.175 DEBUG:boto:Data: 2022-09-03T05:37:32.180 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.181 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:32.181 DEBUG:boto:Port: 8001 2022-09-03T05:37:32.182 DEBUG:boto:Params: {} 2022-09-03T05:37:32.183 DEBUG:boto:Token: None 2022-09-03T05:37:32.184 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.184 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WaBDyA+MT6ZgK6emBPsjQNI+dzs= 2022-09-03T05:37:32.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WaBDyA+MT6ZgK6emBPsjQNI+dzs=', 'Content-Length': '0'} 2022-09-03T05:37:32.187 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008d0ee298e4442775-006312e81c-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.189 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.189 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.195 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.196 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.197 DEBUG:boto:Method: GET 2022-09-03T05:37:32.197 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.198 DEBUG:boto:Data: 2022-09-03T05:37:32.198 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.199 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:32.200 DEBUG:boto:Port: 8000 2022-09-03T05:37:32.200 DEBUG:boto:Params: {} 2022-09-03T05:37:32.202 DEBUG:boto:Token: None 2022-09-03T05:37:32.204 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.207 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WaBDyA+MT6ZgK6emBPsjQNI+dzs= 2022-09-03T05:37:32.209 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WaBDyA+MT6ZgK6emBPsjQNI+dzs=', 'Content-Length': '0'} 2022-09-03T05:37:32.213 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c236abfb1dd27f41-006312e81c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.254 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:32.256 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:32.257 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.258 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.259 DEBUG:boto:Method: GET 2022-09-03T05:37:32.260 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.262 DEBUG:boto:Data: 2022-09-03T05:37:32.263 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.263 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:32.264 DEBUG:boto:Port: 8001 2022-09-03T05:37:32.265 DEBUG:boto:Params: {} 2022-09-03T05:37:32.266 DEBUG:boto:Token: None 2022-09-03T05:37:32.267 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/ 2022-09-03T05:37:32.268 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc= 2022-09-03T05:37:32.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc=', 'Content-Length': '0'} 2022-09-03T05:37:32.272 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002661b7a49938edb7-006312e81c-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.273 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:32.274 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:32.276 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:32.277 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.278 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.278 DEBUG:boto:Method: GET 2022-09-03T05:37:32.279 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.280 DEBUG:boto:Data: 2022-09-03T05:37:32.280 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.281 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:32.281 DEBUG:boto:Port: 8000 2022-09-03T05:37:32.282 DEBUG:boto:Params: {} 2022-09-03T05:37:32.283 DEBUG:boto:Token: None 2022-09-03T05:37:32.284 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/ 2022-09-03T05:37:32.285 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc= 2022-09-03T05:37:32.285 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc=', 'Content-Length': '0'} 2022-09-03T05:37:32.289 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000035afdcb602c7a8f9-006312e81c-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.289 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:32.290 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:32.291 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.292 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.292 DEBUG:boto:path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.293 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.294 DEBUG:boto:Method: GET 2022-09-03T05:37:32.295 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.295 DEBUG:boto:Data: 2022-09-03T05:37:32.296 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.296 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:32.297 DEBUG:boto:Port: 8001 2022-09-03T05:37:32.299 DEBUG:boto:Params: {} 2022-09-03T05:37:32.299 DEBUG:boto:Token: None 2022-09-03T05:37:32.300 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.301 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pEF3Is7uBLagS1iB3AERRw41e+I= 2022-09-03T05:37:32.301 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pEF3Is7uBLagS1iB3AERRw41e+I=', 'Content-Length': '0'} 2022-09-03T05:37:32.304 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'ORmfT2fVQ5EFIFI89EGuLigbPyfeObT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a0903811b663db71-006312e81c-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.305 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.306 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.307 DEBUG:boto:path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.308 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.308 DEBUG:boto:Method: GET 2022-09-03T05:37:32.309 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.310 DEBUG:boto:Data: 2022-09-03T05:37:32.310 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.311 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:32.311 DEBUG:boto:Port: 8000 2022-09-03T05:37:32.312 DEBUG:boto:Params: {} 2022-09-03T05:37:32.312 DEBUG:boto:Token: None 2022-09-03T05:37:32.314 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.315 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pEF3Is7uBLagS1iB3AERRw41e+I= 2022-09-03T05:37:32.316 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pEF3Is7uBLagS1iB3AERRw41e+I=', 'Content-Length': '0'} 2022-09-03T05:37:32.319 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'ORmfT2fVQ5EFIFI89EGuLigbPyfeObT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e6c180463896a510-006312e81c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.320 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.323 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.324 DEBUG:boto:path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.324 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.325 DEBUG:boto:Method: HEAD 2022-09-03T05:37:32.325 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.326 DEBUG:boto:Data: 2022-09-03T05:37:32.326 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.327 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:32.327 DEBUG:boto:Port: 8001 2022-09-03T05:37:32.327 DEBUG:boto:Params: {} 2022-09-03T05:37:32.328 DEBUG:boto:Token: None 2022-09-03T05:37:32.329 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.329 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JjEEFL1kpFEO4SyaFLbAlfo6gvE= 2022-09-03T05:37:32.330 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JjEEFL1kpFEO4SyaFLbAlfo6gvE=', 'Content-Length': '0'} 2022-09-03T05:37:32.332 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'ORmfT2fVQ5EFIFI89EGuLigbPyfeObT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000009f488dd02b8f2be-006312e81c-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.333 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.334 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.334 DEBUG:boto:path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.335 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.335 DEBUG:boto:Method: HEAD 2022-09-03T05:37:32.336 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.336 DEBUG:boto:Data: 2022-09-03T05:37:32.337 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.337 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:32.338 DEBUG:boto:Port: 8000 2022-09-03T05:37:32.338 DEBUG:boto:Params: {} 2022-09-03T05:37:32.339 DEBUG:boto:Token: None 2022-09-03T05:37:32.339 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.340 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JjEEFL1kpFEO4SyaFLbAlfo6gvE= 2022-09-03T05:37:32.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JjEEFL1kpFEO4SyaFLbAlfo6gvE=', 'Content-Length': '0'} 2022-09-03T05:37:32.343 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'ORmfT2fVQ5EFIFI89EGuLigbPyfeObT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005ba81b1a8477c9c5-006312e81c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.343 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:32.344 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.344 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.345 DEBUG:boto:path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.346 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.346 DEBUG:boto:Method: GET 2022-09-03T05:37:32.347 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.347 DEBUG:boto:Data: 2022-09-03T05:37:32.348 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.348 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:32.349 DEBUG:boto:Port: 8001 2022-09-03T05:37:32.349 DEBUG:boto:Params: {} 2022-09-03T05:37:32.350 DEBUG:boto:Token: None 2022-09-03T05:37:32.351 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.351 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pEF3Is7uBLagS1iB3AERRw41e+I= 2022-09-03T05:37:32.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pEF3Is7uBLagS1iB3AERRw41e+I=', 'Content-Length': '0'} 2022-09-03T05:37:32.359 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'ORmfT2fVQ5EFIFI89EGuLigbPyfeObT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c5229186ac793f13-006312e81c-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.360 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.361 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.361 DEBUG:boto:path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.362 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.362 DEBUG:boto:Method: GET 2022-09-03T05:37:32.363 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.363 DEBUG:boto:Data: 2022-09-03T05:37:32.364 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.364 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:32.366 DEBUG:boto:Port: 8000 2022-09-03T05:37:32.367 DEBUG:boto:Params: {} 2022-09-03T05:37:32.367 DEBUG:boto:Token: None 2022-09-03T05:37:32.368 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=ORmfT2fVQ5EFIFI89EGuLigbPyfeObT 2022-09-03T05:37:32.369 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pEF3Is7uBLagS1iB3AERRw41e+I= 2022-09-03T05:37:32.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pEF3Is7uBLagS1iB3AERRw41e+I=', 'Content-Length': '0'} 2022-09-03T05:37:32.371 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'ORmfT2fVQ5EFIFI89EGuLigbPyfeObT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000092f98bd72eb3d372-006312e81c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.416 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:32.417 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:32.418 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.419 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.420 DEBUG:boto:Method: GET 2022-09-03T05:37:32.420 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.421 DEBUG:boto:Data: 2022-09-03T05:37:32.422 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.423 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:32.424 DEBUG:boto:Port: 8001 2022-09-03T05:37:32.425 DEBUG:boto:Params: {} 2022-09-03T05:37:32.427 DEBUG:boto:Token: None 2022-09-03T05:37:32.428 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/ 2022-09-03T05:37:32.429 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc= 2022-09-03T05:37:32.431 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc=', 'Content-Length': '0'} 2022-09-03T05:37:32.438 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000022279b68a3fa0c09-006312e81c-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.439 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:32.440 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:32.440 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:32.441 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.441 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.442 DEBUG:boto:Method: GET 2022-09-03T05:37:32.442 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.444 DEBUG:boto:Data: 2022-09-03T05:37:32.444 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.445 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:32.445 DEBUG:boto:Port: 8000 2022-09-03T05:37:32.446 DEBUG:boto:Params: {} 2022-09-03T05:37:32.446 DEBUG:boto:Token: None 2022-09-03T05:37:32.447 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/ 2022-09-03T05:37:32.448 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc= 2022-09-03T05:37:32.448 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc=', 'Content-Length': '0'} 2022-09-03T05:37:32.452 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eee2d701219b7075-006312e81c-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.453 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:32.455 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.456 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.457 DEBUG:boto:Method: HEAD 2022-09-03T05:37:32.458 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:32.459 DEBUG:boto:Data: 2022-09-03T05:37:32.459 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.460 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:32.462 DEBUG:boto:Port: 8001 2022-09-03T05:37:32.463 DEBUG:boto:Params: {} 2022-09-03T05:37:32.464 DEBUG:boto:Token: None 2022-09-03T05:37:32.465 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj 2022-09-03T05:37:32.466 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QJW5UOc94YcPdLn9jBfvJw4zbaI= 2022-09-03T05:37:32.467 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QJW5UOc94YcPdLn9jBfvJw4zbaI=', 'Content-Length': '0'} 2022-09-03T05:37:32.472 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d07c96d985384db6-006312e81c-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.473 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.476 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.477 DEBUG:boto:Method: HEAD 2022-09-03T05:37:32.478 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:32.479 DEBUG:boto:Data: 2022-09-03T05:37:32.479 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.480 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:32.481 DEBUG:boto:Port: 8000 2022-09-03T05:37:32.481 DEBUG:boto:Params: {} 2022-09-03T05:37:32.482 DEBUG:boto:Token: None 2022-09-03T05:37:32.482 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj 2022-09-03T05:37:32.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QJW5UOc94YcPdLn9jBfvJw4zbaI= 2022-09-03T05:37:32.485 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QJW5UOc94YcPdLn9jBfvJw4zbaI=', 'Content-Length': '0'} 2022-09-03T05:37:32.489 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b340f2108ca72f53-006312e81c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.491 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:32.491 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.492 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.493 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.493 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.494 DEBUG:boto:Method: GET 2022-09-03T05:37:32.495 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.496 DEBUG:boto:Data: 2022-09-03T05:37:32.497 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.498 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:32.498 DEBUG:boto:Port: 8001 2022-09-03T05:37:32.499 DEBUG:boto:Params: {} 2022-09-03T05:37:32.500 DEBUG:boto:Token: None 2022-09-03T05:37:32.500 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.501 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WaBDyA+MT6ZgK6emBPsjQNI+dzs= 2022-09-03T05:37:32.502 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WaBDyA+MT6ZgK6emBPsjQNI+dzs=', 'Content-Length': '0'} 2022-09-03T05:37:32.504 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006268d8501901244f-006312e81c-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.548 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.552 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.553 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.554 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.556 DEBUG:boto:Method: GET 2022-09-03T05:37:32.557 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.558 DEBUG:boto:Data: 2022-09-03T05:37:32.560 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.561 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:32.562 DEBUG:boto:Port: 8000 2022-09-03T05:37:32.564 DEBUG:boto:Params: {} 2022-09-03T05:37:32.565 DEBUG:boto:Token: None 2022-09-03T05:37:32.565 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.566 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WaBDyA+MT6ZgK6emBPsjQNI+dzs= 2022-09-03T05:37:32.567 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WaBDyA+MT6ZgK6emBPsjQNI+dzs=', 'Content-Length': '0'} 2022-09-03T05:37:32.569 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d63bafc0f833fd06-006312e81c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.570 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:32.571 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:32.571 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.572 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.572 DEBUG:boto:Method: GET 2022-09-03T05:37:32.573 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.573 DEBUG:boto:Data: 2022-09-03T05:37:32.574 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.575 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:32.576 DEBUG:boto:Port: 8001 2022-09-03T05:37:32.577 DEBUG:boto:Params: {} 2022-09-03T05:37:32.578 DEBUG:boto:Token: None 2022-09-03T05:37:32.579 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/ 2022-09-03T05:37:32.580 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc= 2022-09-03T05:37:32.581 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc=', 'Content-Length': '0'} 2022-09-03T05:37:32.585 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ec9d1411c6b08665-006312e81c-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.586 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:32.587 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:32.588 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:32.588 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.589 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.593 DEBUG:boto:Method: GET 2022-09-03T05:37:32.593 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.594 DEBUG:boto:Data: 2022-09-03T05:37:32.595 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.596 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:32.597 DEBUG:boto:Port: 8000 2022-09-03T05:37:32.598 DEBUG:boto:Params: {} 2022-09-03T05:37:32.598 DEBUG:boto:Token: None 2022-09-03T05:37:32.599 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/ 2022-09-03T05:37:32.600 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc= 2022-09-03T05:37:32.602 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc=', 'Content-Length': '0'} 2022-09-03T05:37:32.605 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008b580a4154b4d5e8-006312e81c-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.607 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:32.609 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:32.609 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.610 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.611 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.612 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.612 DEBUG:boto:Method: GET 2022-09-03T05:37:32.613 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.613 DEBUG:boto:Data: 2022-09-03T05:37:32.614 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.615 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:32.615 DEBUG:boto:Port: 8001 2022-09-03T05:37:32.616 DEBUG:boto:Params: {} 2022-09-03T05:37:32.617 DEBUG:boto:Token: None 2022-09-03T05:37:32.618 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.618 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hn6E5Py0paITTkNkeXHrwbmg8b0= 2022-09-03T05:37:32.619 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hn6E5Py0paITTkNkeXHrwbmg8b0=', 'Content-Length': '0'} 2022-09-03T05:37:32.621 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kUiw73n7LyoUiDcS9JXEfdlCebChUCE'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000004f3c347ab644fe5-006312e81c-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.623 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.624 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.625 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.626 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.627 DEBUG:boto:Method: GET 2022-09-03T05:37:32.628 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.629 DEBUG:boto:Data: 2022-09-03T05:37:32.630 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.631 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:32.632 DEBUG:boto:Port: 8000 2022-09-03T05:37:32.633 DEBUG:boto:Params: {} 2022-09-03T05:37:32.635 DEBUG:boto:Token: None 2022-09-03T05:37:32.636 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.637 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hn6E5Py0paITTkNkeXHrwbmg8b0= 2022-09-03T05:37:32.639 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hn6E5Py0paITTkNkeXHrwbmg8b0=', 'Content-Length': '0'} 2022-09-03T05:37:32.642 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kUiw73n7LyoUiDcS9JXEfdlCebChUCE'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ed7679bb448da6c3-006312e81c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.643 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.644 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.645 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.646 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.647 DEBUG:boto:Method: HEAD 2022-09-03T05:37:32.648 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.650 DEBUG:boto:Data: 2022-09-03T05:37:32.651 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.652 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:32.652 DEBUG:boto:Port: 8001 2022-09-03T05:37:32.653 DEBUG:boto:Params: {} 2022-09-03T05:37:32.655 DEBUG:boto:Token: None 2022-09-03T05:37:32.656 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.657 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OCbxZ7vInh7Rv3DxhIYHczdM2eU= 2022-09-03T05:37:32.658 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OCbxZ7vInh7Rv3DxhIYHczdM2eU=', 'Content-Length': '0'} 2022-09-03T05:37:32.661 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kUiw73n7LyoUiDcS9JXEfdlCebChUCE'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000030685a42645ad3ac-006312e81c-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.663 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.663 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.664 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.664 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.665 DEBUG:boto:Method: HEAD 2022-09-03T05:37:32.665 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.666 DEBUG:boto:Data: 2022-09-03T05:37:32.666 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.667 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:32.668 DEBUG:boto:Port: 8000 2022-09-03T05:37:32.669 DEBUG:boto:Params: {} 2022-09-03T05:37:32.670 DEBUG:boto:Token: None 2022-09-03T05:37:32.670 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.671 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OCbxZ7vInh7Rv3DxhIYHczdM2eU= 2022-09-03T05:37:32.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OCbxZ7vInh7Rv3DxhIYHczdM2eU=', 'Content-Length': '0'} 2022-09-03T05:37:32.674 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kUiw73n7LyoUiDcS9JXEfdlCebChUCE'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000af903fc010546ec5-006312e81c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.675 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:32.676 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.676 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.677 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.677 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.678 DEBUG:boto:Method: GET 2022-09-03T05:37:32.679 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.679 DEBUG:boto:Data: 2022-09-03T05:37:32.680 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.680 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:32.681 DEBUG:boto:Port: 8001 2022-09-03T05:37:32.681 DEBUG:boto:Params: {} 2022-09-03T05:37:32.682 DEBUG:boto:Token: None 2022-09-03T05:37:32.683 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.683 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hn6E5Py0paITTkNkeXHrwbmg8b0= 2022-09-03T05:37:32.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hn6E5Py0paITTkNkeXHrwbmg8b0=', 'Content-Length': '0'} 2022-09-03T05:37:32.687 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kUiw73n7LyoUiDcS9JXEfdlCebChUCE'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f17a308054279079-006312e81c-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.688 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.688 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.689 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.689 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.690 DEBUG:boto:Method: GET 2022-09-03T05:37:32.690 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.691 DEBUG:boto:Data: 2022-09-03T05:37:32.691 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.692 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:32.693 DEBUG:boto:Port: 8000 2022-09-03T05:37:32.693 DEBUG:boto:Params: {} 2022-09-03T05:37:32.694 DEBUG:boto:Token: None 2022-09-03T05:37:32.694 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=kUiw73n7LyoUiDcS9JXEfdlCebChUCE 2022-09-03T05:37:32.695 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hn6E5Py0paITTkNkeXHrwbmg8b0= 2022-09-03T05:37:32.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hn6E5Py0paITTkNkeXHrwbmg8b0=', 'Content-Length': '0'} 2022-09-03T05:37:32.698 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kUiw73n7LyoUiDcS9JXEfdlCebChUCE'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fd4c07fea015a2aa-006312e81c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.742 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:32.743 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:32.744 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.744 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.745 DEBUG:boto:Method: GET 2022-09-03T05:37:32.746 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.747 DEBUG:boto:Data: 2022-09-03T05:37:32.748 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.749 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:32.750 DEBUG:boto:Port: 8001 2022-09-03T05:37:32.751 DEBUG:boto:Params: {} 2022-09-03T05:37:32.752 DEBUG:boto:Token: None 2022-09-03T05:37:32.755 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/ 2022-09-03T05:37:32.756 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc= 2022-09-03T05:37:32.758 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc=', 'Content-Length': '0'} 2022-09-03T05:37:32.766 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000080fb4f4de23be4be-006312e81c-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.769 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:32.770 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:32.772 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:32.772 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.773 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.774 DEBUG:boto:Method: GET 2022-09-03T05:37:32.775 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.776 DEBUG:boto:Data: 2022-09-03T05:37:32.777 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.778 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:32.779 DEBUG:boto:Port: 8000 2022-09-03T05:37:32.780 DEBUG:boto:Params: {} 2022-09-03T05:37:32.781 DEBUG:boto:Token: None 2022-09-03T05:37:32.782 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/ 2022-09-03T05:37:32.782 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc= 2022-09-03T05:37:32.783 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc=', 'Content-Length': '0'} 2022-09-03T05:37:32.786 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009334d31a154db60d-006312e81c-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.787 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:32.788 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.789 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.790 DEBUG:boto:Method: HEAD 2022-09-03T05:37:32.790 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:32.791 DEBUG:boto:Data: 2022-09-03T05:37:32.791 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.792 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:32.793 DEBUG:boto:Port: 8001 2022-09-03T05:37:32.793 DEBUG:boto:Params: {} 2022-09-03T05:37:32.794 DEBUG:boto:Token: None 2022-09-03T05:37:32.795 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj 2022-09-03T05:37:32.796 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QJW5UOc94YcPdLn9jBfvJw4zbaI= 2022-09-03T05:37:32.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QJW5UOc94YcPdLn9jBfvJw4zbaI=', 'Content-Length': '0'} 2022-09-03T05:37:32.801 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c2b2f5005b1d2a32-006312e81c-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.802 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.802 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.803 DEBUG:boto:Method: HEAD 2022-09-03T05:37:32.803 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:32.804 DEBUG:boto:Data: 2022-09-03T05:37:32.805 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.805 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:32.806 DEBUG:boto:Port: 8000 2022-09-03T05:37:32.807 DEBUG:boto:Params: {} 2022-09-03T05:37:32.808 DEBUG:boto:Token: None 2022-09-03T05:37:32.808 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj 2022-09-03T05:37:32.809 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QJW5UOc94YcPdLn9jBfvJw4zbaI= 2022-09-03T05:37:32.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QJW5UOc94YcPdLn9jBfvJw4zbaI=', 'Content-Length': '0'} 2022-09-03T05:37:32.815 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000ae67c2e982bfdc5-006312e81c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.816 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:32.818 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.819 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.820 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.821 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.822 DEBUG:boto:Method: GET 2022-09-03T05:37:32.823 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.823 DEBUG:boto:Data: 2022-09-03T05:37:32.824 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.824 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:32.825 DEBUG:boto:Port: 8001 2022-09-03T05:37:32.825 DEBUG:boto:Params: {} 2022-09-03T05:37:32.826 DEBUG:boto:Token: None 2022-09-03T05:37:32.827 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.828 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WaBDyA+MT6ZgK6emBPsjQNI+dzs= 2022-09-03T05:37:32.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WaBDyA+MT6ZgK6emBPsjQNI+dzs=', 'Content-Length': '0'} 2022-09-03T05:37:32.831 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000c19a9ffa169306a-006312e81c-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.874 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.875 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.876 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.876 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.877 DEBUG:boto:Method: GET 2022-09-03T05:37:32.877 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.878 DEBUG:boto:Data: 2022-09-03T05:37:32.879 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.879 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:32.880 DEBUG:boto:Port: 8000 2022-09-03T05:37:32.880 DEBUG:boto:Params: {} 2022-09-03T05:37:32.881 DEBUG:boto:Token: None 2022-09-03T05:37:32.882 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:32.882 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WaBDyA+MT6ZgK6emBPsjQNI+dzs= 2022-09-03T05:37:32.883 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WaBDyA+MT6ZgK6emBPsjQNI+dzs=', 'Content-Length': '0'} 2022-09-03T05:37:32.885 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ef11c735b417bd52-006312e81c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.886 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:32.888 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:32.889 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.890 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.891 DEBUG:boto:Method: GET 2022-09-03T05:37:32.892 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.893 DEBUG:boto:Data: 2022-09-03T05:37:32.894 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.895 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:32.896 DEBUG:boto:Port: 8001 2022-09-03T05:37:32.897 DEBUG:boto:Params: {} 2022-09-03T05:37:32.898 DEBUG:boto:Token: None 2022-09-03T05:37:32.899 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/ 2022-09-03T05:37:32.899 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc= 2022-09-03T05:37:32.900 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc=', 'Content-Length': '0'} 2022-09-03T05:37:32.904 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006234537029d41b59-006312e81c-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.904 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:32.905 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:32.906 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:32.906 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.907 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.907 DEBUG:boto:Method: GET 2022-09-03T05:37:32.909 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:32.909 DEBUG:boto:Data: 2022-09-03T05:37:32.910 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.911 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:32.911 DEBUG:boto:Port: 8000 2022-09-03T05:37:32.912 DEBUG:boto:Params: {} 2022-09-03T05:37:32.913 DEBUG:boto:Token: None 2022-09-03T05:37:32.913 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/ 2022-09-03T05:37:32.914 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc= 2022-09-03T05:37:32.915 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ICIsVKoXppkN5NlI7ar39WM3utc=', 'Content-Length': '0'} 2022-09-03T05:37:32.918 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f8b70f7e91dc0751-006312e81c-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.920 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:32.921 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:32.921 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.922 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.923 DEBUG:boto:path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:32.924 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:32.925 DEBUG:boto:Method: GET 2022-09-03T05:37:32.926 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:32.927 DEBUG:boto:Data: 2022-09-03T05:37:32.927 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.928 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:32.929 DEBUG:boto:Port: 8001 2022-09-03T05:37:32.930 DEBUG:boto:Params: {} 2022-09-03T05:37:32.931 DEBUG:boto:Token: None 2022-09-03T05:37:32.932 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:32.933 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RXFHcjDscoYn0gPx9+cfefmtqko= 2022-09-03T05:37:32.933 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RXFHcjDscoYn0gPx9+cfefmtqko=', 'Content-Length': '0'} 2022-09-03T05:37:32.936 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c84d4dcdf52c457e-006312e81c-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.937 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.938 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.939 DEBUG:boto:path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:32.940 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:32.940 DEBUG:boto:Method: GET 2022-09-03T05:37:32.941 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:32.942 DEBUG:boto:Data: 2022-09-03T05:37:32.942 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.943 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:32.944 DEBUG:boto:Port: 8000 2022-09-03T05:37:32.945 DEBUG:boto:Params: {} 2022-09-03T05:37:32.946 DEBUG:boto:Token: None 2022-09-03T05:37:32.948 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:32.948 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RXFHcjDscoYn0gPx9+cfefmtqko= 2022-09-03T05:37:32.950 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RXFHcjDscoYn0gPx9+cfefmtqko=', 'Content-Length': '0'} 2022-09-03T05:37:32.952 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000148ee1d48edd5ba9-006312e81c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.953 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.954 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.955 DEBUG:boto:path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:32.956 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:32.956 DEBUG:boto:Method: HEAD 2022-09-03T05:37:32.957 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:32.957 DEBUG:boto:Data: 2022-09-03T05:37:32.958 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.958 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:32.959 DEBUG:boto:Port: 8001 2022-09-03T05:37:32.959 DEBUG:boto:Params: {} 2022-09-03T05:37:32.960 DEBUG:boto:Token: None 2022-09-03T05:37:32.961 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:32.962 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uE4jlDyRsEcXRzkhpVSJJa1iHB0= 2022-09-03T05:37:32.963 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uE4jlDyRsEcXRzkhpVSJJa1iHB0=', 'Content-Length': '0'} 2022-09-03T05:37:32.966 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ec8f711f485d54be-006312e81c-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.968 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.969 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.970 DEBUG:boto:path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:32.971 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:32.972 DEBUG:boto:Method: HEAD 2022-09-03T05:37:32.974 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:32.975 DEBUG:boto:Data: 2022-09-03T05:37:32.976 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.977 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:32.978 DEBUG:boto:Port: 8000 2022-09-03T05:37:32.979 DEBUG:boto:Params: {} 2022-09-03T05:37:32.980 DEBUG:boto:Token: None 2022-09-03T05:37:32.981 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:32.982 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uE4jlDyRsEcXRzkhpVSJJa1iHB0= 2022-09-03T05:37:32.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uE4jlDyRsEcXRzkhpVSJJa1iHB0=', 'Content-Length': '0'} 2022-09-03T05:37:32.987 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f63ff53a68fda122-006312e81c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:32.988 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:32.989 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:32.990 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:32.990 DEBUG:boto:path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:32.991 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:32.991 DEBUG:boto:Method: GET 2022-09-03T05:37:32.992 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:32.993 DEBUG:boto:Data: 2022-09-03T05:37:32.994 DEBUG:boto:Headers: {} 2022-09-03T05:37:32.995 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:32.995 DEBUG:boto:Port: 8001 2022-09-03T05:37:32.996 DEBUG:boto:Params: {} 2022-09-03T05:37:32.996 DEBUG:boto:Token: None 2022-09-03T05:37:32.997 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:32 GMT /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:32.998 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RXFHcjDscoYn0gPx9+cfefmtqko= 2022-09-03T05:37:32.999 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RXFHcjDscoYn0gPx9+cfefmtqko=', 'Content-Length': '0'} 2022-09-03T05:37:33.001 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d34a7c11b1083ac6-006312e81d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.046 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.048 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.049 DEBUG:boto:path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:33.050 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:33.051 DEBUG:boto:Method: GET 2022-09-03T05:37:33.052 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:33.053 DEBUG:boto:Data: 2022-09-03T05:37:33.054 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.055 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:33.056 DEBUG:boto:Port: 8000 2022-09-03T05:37:33.057 DEBUG:boto:Params: {} 2022-09-03T05:37:33.060 DEBUG:boto:Token: None 2022-09-03T05:37:33.061 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98 2022-09-03T05:37:33.062 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:daMq5RwfeKcMjjmml6YI5pE0LZQ= 2022-09-03T05:37:33.063 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:daMq5RwfeKcMjjmml6YI5pE0LZQ=', 'Content-Length': '0'} 2022-09-03T05:37:33.066 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'BZ8GlCm1r6O5tgvp21X3Pzi1IzK6E98'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d4601de513183d86-006312e81d-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.068 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:33.069 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:33.070 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.071 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.072 DEBUG:boto:Method: GET 2022-09-03T05:37:33.073 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.074 DEBUG:boto:Data: 2022-09-03T05:37:33.075 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.076 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:33.077 DEBUG:boto:Port: 8001 2022-09-03T05:37:33.078 DEBUG:boto:Params: {} 2022-09-03T05:37:33.079 DEBUG:boto:Token: None 2022-09-03T05:37:33.080 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/ 2022-09-03T05:37:33.081 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q= 2022-09-03T05:37:33.082 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q=', 'Content-Length': '0'} 2022-09-03T05:37:33.085 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e9365993398fded7-006312e81d-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.086 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:33.087 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:33.088 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:33.089 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.090 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.092 DEBUG:boto:Method: GET 2022-09-03T05:37:33.092 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.093 DEBUG:boto:Data: 2022-09-03T05:37:33.094 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.095 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:33.096 DEBUG:boto:Port: 8000 2022-09-03T05:37:33.097 DEBUG:boto:Params: {} 2022-09-03T05:37:33.097 DEBUG:boto:Token: None 2022-09-03T05:37:33.100 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/ 2022-09-03T05:37:33.100 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q= 2022-09-03T05:37:33.101 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q=', 'Content-Length': '0'} 2022-09-03T05:37:33.105 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002e59b91d4faa4dde-006312e81d-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.106 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:33.108 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.108 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.109 DEBUG:boto:Method: HEAD 2022-09-03T05:37:33.111 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:33.112 DEBUG:boto:Data: 2022-09-03T05:37:33.113 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.114 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:33.115 DEBUG:boto:Port: 8001 2022-09-03T05:37:33.115 DEBUG:boto:Params: {} 2022-09-03T05:37:33.116 DEBUG:boto:Token: None 2022-09-03T05:37:33.117 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj 2022-09-03T05:37:33.118 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dLT9ybjpN7eX0yIr9jzUyz3cpAI= 2022-09-03T05:37:33.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dLT9ybjpN7eX0yIr9jzUyz3cpAI=', 'Content-Length': '0'} 2022-09-03T05:37:33.123 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005bf31d7d7c5f23b0-006312e81d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.124 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.125 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.126 DEBUG:boto:Method: HEAD 2022-09-03T05:37:33.127 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:33.128 DEBUG:boto:Data: 2022-09-03T05:37:33.129 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.130 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:33.130 DEBUG:boto:Port: 8000 2022-09-03T05:37:33.131 DEBUG:boto:Params: {} 2022-09-03T05:37:33.132 DEBUG:boto:Token: None 2022-09-03T05:37:33.133 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj 2022-09-03T05:37:33.134 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dLT9ybjpN7eX0yIr9jzUyz3cpAI= 2022-09-03T05:37:33.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dLT9ybjpN7eX0yIr9jzUyz3cpAI=', 'Content-Length': '0'} 2022-09-03T05:37:33.141 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003cfc30d7ade5684a-006312e81d-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.142 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:33.143 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.144 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.145 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.147 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.148 DEBUG:boto:Method: GET 2022-09-03T05:37:33.149 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.150 DEBUG:boto:Data: 2022-09-03T05:37:33.151 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.152 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:33.153 DEBUG:boto:Port: 8001 2022-09-03T05:37:33.154 DEBUG:boto:Params: {} 2022-09-03T05:37:33.156 DEBUG:boto:Token: None 2022-09-03T05:37:33.157 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.158 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZqN0cGqeAJIMbORV3c5cf2C7iyc= 2022-09-03T05:37:33.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZqN0cGqeAJIMbORV3c5cf2C7iyc=', 'Content-Length': '0'} 2022-09-03T05:37:33.164 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000037f8a5e9eac77059-006312e81d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.165 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.165 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.167 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.167 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.168 DEBUG:boto:Method: GET 2022-09-03T05:37:33.169 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.170 DEBUG:boto:Data: 2022-09-03T05:37:33.170 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.171 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:33.171 DEBUG:boto:Port: 8000 2022-09-03T05:37:33.172 DEBUG:boto:Params: {} 2022-09-03T05:37:33.173 DEBUG:boto:Token: None 2022-09-03T05:37:33.174 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.174 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZqN0cGqeAJIMbORV3c5cf2C7iyc= 2022-09-03T05:37:33.175 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZqN0cGqeAJIMbORV3c5cf2C7iyc=', 'Content-Length': '0'} 2022-09-03T05:37:33.180 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b74bfb02dd6be112-006312e81d-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.218 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:33.219 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:33.219 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.220 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.221 DEBUG:boto:Method: GET 2022-09-03T05:37:33.222 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.223 DEBUG:boto:Data: 2022-09-03T05:37:33.224 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.225 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:33.226 DEBUG:boto:Port: 8001 2022-09-03T05:37:33.227 DEBUG:boto:Params: {} 2022-09-03T05:37:33.228 DEBUG:boto:Token: None 2022-09-03T05:37:33.230 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/ 2022-09-03T05:37:33.231 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q= 2022-09-03T05:37:33.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q=', 'Content-Length': '0'} 2022-09-03T05:37:33.236 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e630c5897f32347f-006312e81d-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.237 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:33.238 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:33.239 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:33.239 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.240 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.240 DEBUG:boto:Method: GET 2022-09-03T05:37:33.241 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.242 DEBUG:boto:Data: 2022-09-03T05:37:33.242 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.243 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:33.243 DEBUG:boto:Port: 8000 2022-09-03T05:37:33.244 DEBUG:boto:Params: {} 2022-09-03T05:37:33.244 DEBUG:boto:Token: None 2022-09-03T05:37:33.245 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/ 2022-09-03T05:37:33.246 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q= 2022-09-03T05:37:33.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q=', 'Content-Length': '0'} 2022-09-03T05:37:33.249 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000090bf01eb95347096-006312e81d-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.250 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:33.251 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:33.252 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.253 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.253 DEBUG:boto:path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.254 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.255 DEBUG:boto:Method: GET 2022-09-03T05:37:33.255 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.256 DEBUG:boto:Data: 2022-09-03T05:37:33.256 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.257 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:33.257 DEBUG:boto:Port: 8001 2022-09-03T05:37:33.258 DEBUG:boto:Params: {} 2022-09-03T05:37:33.258 DEBUG:boto:Token: None 2022-09-03T05:37:33.259 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.260 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zpAeWYagHeCssJkPq1JdIBm4s4I= 2022-09-03T05:37:33.261 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zpAeWYagHeCssJkPq1JdIBm4s4I=', 'Content-Length': '0'} 2022-09-03T05:37:33.263 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000023afcc2346479304-006312e81d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.264 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.265 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.266 DEBUG:boto:path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.266 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.267 DEBUG:boto:Method: GET 2022-09-03T05:37:33.267 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.268 DEBUG:boto:Data: 2022-09-03T05:37:33.269 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.269 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:33.270 DEBUG:boto:Port: 8000 2022-09-03T05:37:33.270 DEBUG:boto:Params: {} 2022-09-03T05:37:33.271 DEBUG:boto:Token: None 2022-09-03T05:37:33.272 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.272 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zpAeWYagHeCssJkPq1JdIBm4s4I= 2022-09-03T05:37:33.273 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zpAeWYagHeCssJkPq1JdIBm4s4I=', 'Content-Length': '0'} 2022-09-03T05:37:33.275 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003798a2752d41d2ca-006312e81d-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.276 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.276 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.277 DEBUG:boto:path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.278 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.278 DEBUG:boto:Method: HEAD 2022-09-03T05:37:33.279 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.279 DEBUG:boto:Data: 2022-09-03T05:37:33.280 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.280 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:33.281 DEBUG:boto:Port: 8001 2022-09-03T05:37:33.281 DEBUG:boto:Params: {} 2022-09-03T05:37:33.282 DEBUG:boto:Token: None 2022-09-03T05:37:33.282 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.283 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:foTG/kLqw+QJ9ST9US2iKUxNPm0= 2022-09-03T05:37:33.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:foTG/kLqw+QJ9ST9US2iKUxNPm0=', 'Content-Length': '0'} 2022-09-03T05:37:33.286 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004b1a56242fb2a609-006312e81d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.287 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.287 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.288 DEBUG:boto:path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.288 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.289 DEBUG:boto:Method: HEAD 2022-09-03T05:37:33.289 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.290 DEBUG:boto:Data: 2022-09-03T05:37:33.291 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.291 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:33.292 DEBUG:boto:Port: 8000 2022-09-03T05:37:33.292 DEBUG:boto:Params: {} 2022-09-03T05:37:33.293 DEBUG:boto:Token: None 2022-09-03T05:37:33.294 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.294 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:foTG/kLqw+QJ9ST9US2iKUxNPm0= 2022-09-03T05:37:33.295 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:foTG/kLqw+QJ9ST9US2iKUxNPm0=', 'Content-Length': '0'} 2022-09-03T05:37:33.297 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c4a7cf16377d7890-006312e81d-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.299 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:33.300 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.301 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.302 DEBUG:boto:path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.303 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.304 DEBUG:boto:Method: GET 2022-09-03T05:37:33.305 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.305 DEBUG:boto:Data: 2022-09-03T05:37:33.306 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.306 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:33.307 DEBUG:boto:Port: 8001 2022-09-03T05:37:33.307 DEBUG:boto:Params: {} 2022-09-03T05:37:33.308 DEBUG:boto:Token: None 2022-09-03T05:37:33.309 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.310 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zpAeWYagHeCssJkPq1JdIBm4s4I= 2022-09-03T05:37:33.311 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zpAeWYagHeCssJkPq1JdIBm4s4I=', 'Content-Length': '0'} 2022-09-03T05:37:33.313 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000bc6f27b35975648-006312e81d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.358 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.359 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.359 DEBUG:boto:path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.360 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.361 DEBUG:boto:Method: GET 2022-09-03T05:37:33.361 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.370 DEBUG:boto:Data: 2022-09-03T05:37:33.371 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.372 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:33.373 DEBUG:boto:Port: 8000 2022-09-03T05:37:33.375 DEBUG:boto:Params: {} 2022-09-03T05:37:33.376 DEBUG:boto:Token: None 2022-09-03T05:37:33.378 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB 2022-09-03T05:37:33.379 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zpAeWYagHeCssJkPq1JdIBm4s4I= 2022-09-03T05:37:33.379 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zpAeWYagHeCssJkPq1JdIBm4s4I=', 'Content-Length': '0'} 2022-09-03T05:37:33.382 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'IoWdEYmmGoqkLlDY9VTnAjqaP2JQ5SB'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002a97ab63ec8c65e4-006312e81d-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.383 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:33.384 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:33.384 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.385 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.386 DEBUG:boto:Method: GET 2022-09-03T05:37:33.386 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.387 DEBUG:boto:Data: 2022-09-03T05:37:33.387 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.388 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:33.388 DEBUG:boto:Port: 8001 2022-09-03T05:37:33.389 DEBUG:boto:Params: {} 2022-09-03T05:37:33.390 DEBUG:boto:Token: None 2022-09-03T05:37:33.391 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/ 2022-09-03T05:37:33.394 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q= 2022-09-03T05:37:33.395 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q=', 'Content-Length': '0'} 2022-09-03T05:37:33.399 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d774fd21255cbdad-006312e81d-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.401 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:33.402 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:33.413 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:33.414 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.415 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.416 DEBUG:boto:Method: GET 2022-09-03T05:37:33.417 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.418 DEBUG:boto:Data: 2022-09-03T05:37:33.419 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.420 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:33.420 DEBUG:boto:Port: 8000 2022-09-03T05:37:33.421 DEBUG:boto:Params: {} 2022-09-03T05:37:33.422 DEBUG:boto:Token: None 2022-09-03T05:37:33.423 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/ 2022-09-03T05:37:33.424 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q= 2022-09-03T05:37:33.425 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q=', 'Content-Length': '0'} 2022-09-03T05:37:33.429 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b98a22c9d1be37e5-006312e81d-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.431 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:33.432 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.433 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.434 DEBUG:boto:Method: HEAD 2022-09-03T05:37:33.434 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:33.435 DEBUG:boto:Data: 2022-09-03T05:37:33.436 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.437 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:33.438 DEBUG:boto:Port: 8001 2022-09-03T05:37:33.439 DEBUG:boto:Params: {} 2022-09-03T05:37:33.439 DEBUG:boto:Token: None 2022-09-03T05:37:33.440 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj 2022-09-03T05:37:33.440 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dLT9ybjpN7eX0yIr9jzUyz3cpAI= 2022-09-03T05:37:33.441 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dLT9ybjpN7eX0yIr9jzUyz3cpAI=', 'Content-Length': '0'} 2022-09-03T05:37:33.445 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f89f6e12736d1a01-006312e81d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.446 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.447 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.447 DEBUG:boto:Method: HEAD 2022-09-03T05:37:33.448 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:33.448 DEBUG:boto:Data: 2022-09-03T05:37:33.450 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.451 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:33.452 DEBUG:boto:Port: 8000 2022-09-03T05:37:33.453 DEBUG:boto:Params: {} 2022-09-03T05:37:33.453 DEBUG:boto:Token: None 2022-09-03T05:37:33.454 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj 2022-09-03T05:37:33.455 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dLT9ybjpN7eX0yIr9jzUyz3cpAI= 2022-09-03T05:37:33.455 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dLT9ybjpN7eX0yIr9jzUyz3cpAI=', 'Content-Length': '0'} 2022-09-03T05:37:33.460 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a72702f447cc6f2c-006312e81d-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.462 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:33.463 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.463 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.464 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.464 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.465 DEBUG:boto:Method: GET 2022-09-03T05:37:33.466 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.466 DEBUG:boto:Data: 2022-09-03T05:37:33.467 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.467 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:33.468 DEBUG:boto:Port: 8001 2022-09-03T05:37:33.468 DEBUG:boto:Params: {} 2022-09-03T05:37:33.469 DEBUG:boto:Token: None 2022-09-03T05:37:33.470 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.470 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZqN0cGqeAJIMbORV3c5cf2C7iyc= 2022-09-03T05:37:33.471 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZqN0cGqeAJIMbORV3c5cf2C7iyc=', 'Content-Length': '0'} 2022-09-03T05:37:33.473 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cd667604ce060f9f-006312e81d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.474 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.475 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.476 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.476 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.477 DEBUG:boto:Method: GET 2022-09-03T05:37:33.477 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.478 DEBUG:boto:Data: 2022-09-03T05:37:33.478 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.479 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:33.480 DEBUG:boto:Port: 8000 2022-09-03T05:37:33.481 DEBUG:boto:Params: {} 2022-09-03T05:37:33.482 DEBUG:boto:Token: None 2022-09-03T05:37:33.482 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZqN0cGqeAJIMbORV3c5cf2C7iyc= 2022-09-03T05:37:33.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZqN0cGqeAJIMbORV3c5cf2C7iyc=', 'Content-Length': '0'} 2022-09-03T05:37:33.487 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000012eb7e28f9a1c7e8-006312e81d-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.531 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:33.533 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:33.534 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.535 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.536 DEBUG:boto:Method: GET 2022-09-03T05:37:33.537 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.538 DEBUG:boto:Data: 2022-09-03T05:37:33.538 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.539 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:33.540 DEBUG:boto:Port: 8001 2022-09-03T05:37:33.540 DEBUG:boto:Params: {} 2022-09-03T05:37:33.541 DEBUG:boto:Token: None 2022-09-03T05:37:33.542 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/ 2022-09-03T05:37:33.543 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q= 2022-09-03T05:37:33.543 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q=', 'Content-Length': '0'} 2022-09-03T05:37:33.546 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000074fcdc4435d0798b-006312e81d-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.547 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:33.548 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:33.549 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:33.549 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.550 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.551 DEBUG:boto:Method: GET 2022-09-03T05:37:33.551 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.552 DEBUG:boto:Data: 2022-09-03T05:37:33.552 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.553 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:33.553 DEBUG:boto:Port: 8000 2022-09-03T05:37:33.554 DEBUG:boto:Params: {} 2022-09-03T05:37:33.554 DEBUG:boto:Token: None 2022-09-03T05:37:33.555 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/ 2022-09-03T05:37:33.557 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q= 2022-09-03T05:37:33.557 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q=', 'Content-Length': '0'} 2022-09-03T05:37:33.561 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dc574d335dc11de3-006312e81d-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.561 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:33.562 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:33.563 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.564 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.564 DEBUG:boto:path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.565 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.565 DEBUG:boto:Method: GET 2022-09-03T05:37:33.566 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.566 DEBUG:boto:Data: 2022-09-03T05:37:33.567 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.568 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:33.568 DEBUG:boto:Port: 8001 2022-09-03T05:37:33.569 DEBUG:boto:Params: {} 2022-09-03T05:37:33.570 DEBUG:boto:Token: None 2022-09-03T05:37:33.571 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.572 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Tq309A0m8WWQUV/t9F/N64VR/uk= 2022-09-03T05:37:33.573 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Tq309A0m8WWQUV/t9F/N64VR/uk=', 'Content-Length': '0'} 2022-09-03T05:37:33.575 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e06312110ee5a436-006312e81d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.577 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.577 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.578 DEBUG:boto:path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.578 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.579 DEBUG:boto:Method: GET 2022-09-03T05:37:33.580 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.580 DEBUG:boto:Data: 2022-09-03T05:37:33.581 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.581 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:33.582 DEBUG:boto:Port: 8000 2022-09-03T05:37:33.583 DEBUG:boto:Params: {} 2022-09-03T05:37:33.583 DEBUG:boto:Token: None 2022-09-03T05:37:33.584 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.585 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Tq309A0m8WWQUV/t9F/N64VR/uk= 2022-09-03T05:37:33.585 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Tq309A0m8WWQUV/t9F/N64VR/uk=', 'Content-Length': '0'} 2022-09-03T05:37:33.588 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f41ac95c44cc56c3-006312e81d-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.589 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.590 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.590 DEBUG:boto:path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.591 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.591 DEBUG:boto:Method: HEAD 2022-09-03T05:37:33.592 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.593 DEBUG:boto:Data: 2022-09-03T05:37:33.593 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.594 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:33.594 DEBUG:boto:Port: 8001 2022-09-03T05:37:33.595 DEBUG:boto:Params: {} 2022-09-03T05:37:33.596 DEBUG:boto:Token: None 2022-09-03T05:37:33.596 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.597 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZvH4j/1kp3nutgtYFYSlEPEOnCU= 2022-09-03T05:37:33.597 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZvH4j/1kp3nutgtYFYSlEPEOnCU=', 'Content-Length': '0'} 2022-09-03T05:37:33.599 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000de6163429df56ef6-006312e81d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.600 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.601 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.601 DEBUG:boto:path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.602 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.603 DEBUG:boto:Method: HEAD 2022-09-03T05:37:33.603 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.604 DEBUG:boto:Data: 2022-09-03T05:37:33.604 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.605 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:33.605 DEBUG:boto:Port: 8000 2022-09-03T05:37:33.606 DEBUG:boto:Params: {} 2022-09-03T05:37:33.606 DEBUG:boto:Token: None 2022-09-03T05:37:33.607 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.608 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZvH4j/1kp3nutgtYFYSlEPEOnCU= 2022-09-03T05:37:33.608 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZvH4j/1kp3nutgtYFYSlEPEOnCU=', 'Content-Length': '0'} 2022-09-03T05:37:33.610 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000285addcaa9c368b9-006312e81d-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.612 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:33.612 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.613 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.614 DEBUG:boto:path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.614 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.615 DEBUG:boto:Method: GET 2022-09-03T05:37:33.616 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.616 DEBUG:boto:Data: 2022-09-03T05:37:33.617 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.617 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:33.618 DEBUG:boto:Port: 8001 2022-09-03T05:37:33.618 DEBUG:boto:Params: {} 2022-09-03T05:37:33.619 DEBUG:boto:Token: None 2022-09-03T05:37:33.620 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.620 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Tq309A0m8WWQUV/t9F/N64VR/uk= 2022-09-03T05:37:33.621 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Tq309A0m8WWQUV/t9F/N64VR/uk=', 'Content-Length': '0'} 2022-09-03T05:37:33.623 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000008ba438de6694a698-006312e81d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.671 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.672 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.673 DEBUG:boto:path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.673 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.674 DEBUG:boto:Method: GET 2022-09-03T05:37:33.675 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.675 DEBUG:boto:Data: 2022-09-03T05:37:33.676 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.676 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:33.677 DEBUG:boto:Port: 8000 2022-09-03T05:37:33.677 DEBUG:boto:Params: {} 2022-09-03T05:37:33.678 DEBUG:boto:Token: None 2022-09-03T05:37:33.679 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i 2022-09-03T05:37:33.679 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Tq309A0m8WWQUV/t9F/N64VR/uk= 2022-09-03T05:37:33.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Tq309A0m8WWQUV/t9F/N64VR/uk=', 'Content-Length': '0'} 2022-09-03T05:37:33.682 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'rHbEnZ16TQjAC.xp1wNbn0Ce5Vc948i'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a549fa50a27c4380-006312e81d-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.684 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:33.685 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:33.686 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.687 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.687 DEBUG:boto:Method: GET 2022-09-03T05:37:33.688 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.688 DEBUG:boto:Data: 2022-09-03T05:37:33.690 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.690 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:33.691 DEBUG:boto:Port: 8001 2022-09-03T05:37:33.692 DEBUG:boto:Params: {} 2022-09-03T05:37:33.692 DEBUG:boto:Token: None 2022-09-03T05:37:33.693 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/ 2022-09-03T05:37:33.694 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q= 2022-09-03T05:37:33.695 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q=', 'Content-Length': '0'} 2022-09-03T05:37:33.707 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000055174fbc068b9e38-006312e81d-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.711 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:33.712 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:33.714 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:33.715 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.716 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.718 DEBUG:boto:Method: GET 2022-09-03T05:37:33.720 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.721 DEBUG:boto:Data: 2022-09-03T05:37:33.722 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.723 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:33.724 DEBUG:boto:Port: 8000 2022-09-03T05:37:33.725 DEBUG:boto:Params: {} 2022-09-03T05:37:33.727 DEBUG:boto:Token: None 2022-09-03T05:37:33.730 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/ 2022-09-03T05:37:33.731 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q= 2022-09-03T05:37:33.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q=', 'Content-Length': '0'} 2022-09-03T05:37:33.735 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bea2d50b6e19488e-006312e81d-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.736 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:33.737 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.738 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.738 DEBUG:boto:Method: HEAD 2022-09-03T05:37:33.739 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:33.739 DEBUG:boto:Data: 2022-09-03T05:37:33.740 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.740 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:33.741 DEBUG:boto:Port: 8001 2022-09-03T05:37:33.741 DEBUG:boto:Params: {} 2022-09-03T05:37:33.742 DEBUG:boto:Token: None 2022-09-03T05:37:33.743 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj 2022-09-03T05:37:33.743 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dLT9ybjpN7eX0yIr9jzUyz3cpAI= 2022-09-03T05:37:33.744 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dLT9ybjpN7eX0yIr9jzUyz3cpAI=', 'Content-Length': '0'} 2022-09-03T05:37:33.748 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000007e0707d8070d95f3-006312e81d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.749 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.749 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.750 DEBUG:boto:Method: HEAD 2022-09-03T05:37:33.750 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:33.751 DEBUG:boto:Data: 2022-09-03T05:37:33.751 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.752 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:33.752 DEBUG:boto:Port: 8000 2022-09-03T05:37:33.753 DEBUG:boto:Params: {} 2022-09-03T05:37:33.754 DEBUG:boto:Token: None 2022-09-03T05:37:33.754 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj 2022-09-03T05:37:33.755 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dLT9ybjpN7eX0yIr9jzUyz3cpAI= 2022-09-03T05:37:33.756 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dLT9ybjpN7eX0yIr9jzUyz3cpAI=', 'Content-Length': '0'} 2022-09-03T05:37:33.759 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009690852679eb969c-006312e81d-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.760 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:33.761 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.762 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.762 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.763 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.763 DEBUG:boto:Method: GET 2022-09-03T05:37:33.764 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.765 DEBUG:boto:Data: 2022-09-03T05:37:33.765 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.766 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:33.767 DEBUG:boto:Port: 8001 2022-09-03T05:37:33.767 DEBUG:boto:Params: {} 2022-09-03T05:37:33.768 DEBUG:boto:Token: None 2022-09-03T05:37:33.769 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.770 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZqN0cGqeAJIMbORV3c5cf2C7iyc= 2022-09-03T05:37:33.771 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZqN0cGqeAJIMbORV3c5cf2C7iyc=', 'Content-Length': '0'} 2022-09-03T05:37:33.774 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a17f86bcf0d21ad4-006312e81d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.814 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.815 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.816 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.816 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.817 DEBUG:boto:Method: GET 2022-09-03T05:37:33.817 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.818 DEBUG:boto:Data: 2022-09-03T05:37:33.818 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.819 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:33.820 DEBUG:boto:Port: 8000 2022-09-03T05:37:33.820 DEBUG:boto:Params: {} 2022-09-03T05:37:33.821 DEBUG:boto:Token: None 2022-09-03T05:37:33.821 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:33.822 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZqN0cGqeAJIMbORV3c5cf2C7iyc= 2022-09-03T05:37:33.823 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZqN0cGqeAJIMbORV3c5cf2C7iyc=', 'Content-Length': '0'} 2022-09-03T05:37:33.825 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000025947879569167d1-006312e81d-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.826 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:33.827 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:33.828 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.828 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.829 DEBUG:boto:Method: GET 2022-09-03T05:37:33.830 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.830 DEBUG:boto:Data: 2022-09-03T05:37:33.831 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.831 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:33.832 DEBUG:boto:Port: 8001 2022-09-03T05:37:33.832 DEBUG:boto:Params: {} 2022-09-03T05:37:33.833 DEBUG:boto:Token: None 2022-09-03T05:37:33.834 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/ 2022-09-03T05:37:33.834 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q= 2022-09-03T05:37:33.835 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q=', 'Content-Length': '0'} 2022-09-03T05:37:33.839 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a04c535664b7ca78-006312e81d-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.840 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:33.842 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:33.843 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:33.844 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.845 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.845 DEBUG:boto:Method: GET 2022-09-03T05:37:33.846 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.847 DEBUG:boto:Data: 2022-09-03T05:37:33.847 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.848 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:33.848 DEBUG:boto:Port: 8000 2022-09-03T05:37:33.849 DEBUG:boto:Params: {} 2022-09-03T05:37:33.849 DEBUG:boto:Token: None 2022-09-03T05:37:33.850 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/ 2022-09-03T05:37:33.851 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q= 2022-09-03T05:37:33.851 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q=', 'Content-Length': '0'} 2022-09-03T05:37:33.855 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003aa44b8b8626f845-006312e81d-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.856 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:33.857 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:33.858 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.859 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.859 DEBUG:boto:path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.860 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.862 DEBUG:boto:Method: GET 2022-09-03T05:37:33.863 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.864 DEBUG:boto:Data: 2022-09-03T05:37:33.865 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.866 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:33.867 DEBUG:boto:Port: 8001 2022-09-03T05:37:33.868 DEBUG:boto:Params: {} 2022-09-03T05:37:33.870 DEBUG:boto:Token: None 2022-09-03T05:37:33.871 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.872 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h7LZhxtWis3elasKDW5lFygfcGg= 2022-09-03T05:37:33.873 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h7LZhxtWis3elasKDW5lFygfcGg=', 'Content-Length': '0'} 2022-09-03T05:37:33.876 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000040f7d0ba1772cd00-006312e81d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.877 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.877 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.878 DEBUG:boto:path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.879 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.879 DEBUG:boto:Method: GET 2022-09-03T05:37:33.880 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.880 DEBUG:boto:Data: 2022-09-03T05:37:33.881 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.881 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:33.882 DEBUG:boto:Port: 8000 2022-09-03T05:37:33.883 DEBUG:boto:Params: {} 2022-09-03T05:37:33.884 DEBUG:boto:Token: None 2022-09-03T05:37:33.885 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.886 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h7LZhxtWis3elasKDW5lFygfcGg= 2022-09-03T05:37:33.887 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h7LZhxtWis3elasKDW5lFygfcGg=', 'Content-Length': '0'} 2022-09-03T05:37:33.889 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000068498ae7c75850a1-006312e81d-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.934 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.935 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.935 DEBUG:boto:path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.936 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.937 DEBUG:boto:Method: HEAD 2022-09-03T05:37:33.937 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.938 DEBUG:boto:Data: 2022-09-03T05:37:33.938 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.939 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:33.939 DEBUG:boto:Port: 8001 2022-09-03T05:37:33.940 DEBUG:boto:Params: {} 2022-09-03T05:37:33.941 DEBUG:boto:Token: None 2022-09-03T05:37:33.941 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.942 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t3myjvHX066McupGAt4cHUKShPA= 2022-09-03T05:37:33.943 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t3myjvHX066McupGAt4cHUKShPA=', 'Content-Length': '0'} 2022-09-03T05:37:33.945 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000178add95dd882928-006312e81d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.946 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.946 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.947 DEBUG:boto:path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.947 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.948 DEBUG:boto:Method: HEAD 2022-09-03T05:37:33.948 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.949 DEBUG:boto:Data: 2022-09-03T05:37:33.949 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.950 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:33.950 DEBUG:boto:Port: 8000 2022-09-03T05:37:33.951 DEBUG:boto:Params: {} 2022-09-03T05:37:33.952 DEBUG:boto:Token: None 2022-09-03T05:37:33.952 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.953 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t3myjvHX066McupGAt4cHUKShPA= 2022-09-03T05:37:33.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t3myjvHX066McupGAt4cHUKShPA=', 'Content-Length': '0'} 2022-09-03T05:37:33.956 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d47180a17da53d95-006312e81d-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.956 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:33.957 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.957 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.958 DEBUG:boto:path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.958 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.959 DEBUG:boto:Method: GET 2022-09-03T05:37:33.960 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.960 DEBUG:boto:Data: 2022-09-03T05:37:33.961 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.961 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:33.962 DEBUG:boto:Port: 8001 2022-09-03T05:37:33.962 DEBUG:boto:Params: {} 2022-09-03T05:37:33.963 DEBUG:boto:Token: None 2022-09-03T05:37:33.963 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.964 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h7LZhxtWis3elasKDW5lFygfcGg= 2022-09-03T05:37:33.965 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h7LZhxtWis3elasKDW5lFygfcGg=', 'Content-Length': '0'} 2022-09-03T05:37:33.967 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000074da47e4f9e4b74c-006312e81d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.968 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:33.968 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:33.969 DEBUG:boto:path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.970 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.970 DEBUG:boto:Method: GET 2022-09-03T05:37:33.971 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.972 DEBUG:boto:Data: 2022-09-03T05:37:33.972 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.973 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:33.973 DEBUG:boto:Port: 8000 2022-09-03T05:37:33.974 DEBUG:boto:Params: {} 2022-09-03T05:37:33.975 DEBUG:boto:Token: None 2022-09-03T05:37:33.975 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/obj?versionId=G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL 2022-09-03T05:37:33.976 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h7LZhxtWis3elasKDW5lFygfcGg= 2022-09-03T05:37:33.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h7LZhxtWis3elasKDW5lFygfcGg=', 'Content-Length': '0'} 2022-09-03T05:37:33.979 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'G4CvLAzjxT7MhR0RJSAzrP2ZFjtqGXL'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004e5e0d4dc8001987-006312e81d-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:33.980 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:33.980 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:33.981 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.981 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.982 DEBUG:boto:Method: GET 2022-09-03T05:37:33.983 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:33.983 DEBUG:boto:Data: 2022-09-03T05:37:33.984 DEBUG:boto:Headers: {} 2022-09-03T05:37:33.984 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:33.985 DEBUG:boto:Port: 8001 2022-09-03T05:37:33.985 DEBUG:boto:Params: {} 2022-09-03T05:37:33.986 DEBUG:boto:Token: None 2022-09-03T05:37:33.986 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:33 GMT /dsgvcy-19/ 2022-09-03T05:37:33.987 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q= 2022-09-03T05:37:33.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tNYR2fXR8nKvNvmOxw6zIke3+4Q=', 'Content-Length': '0'} 2022-09-03T05:37:33.991 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000be1eaf2b2884b547-006312e81d-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.038 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:34.039 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:34.040 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:34.040 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.041 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.042 DEBUG:boto:Method: GET 2022-09-03T05:37:34.042 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.043 DEBUG:boto:Data: 2022-09-03T05:37:34.043 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.044 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:34.044 DEBUG:boto:Port: 8000 2022-09-03T05:37:34.045 DEBUG:boto:Params: {} 2022-09-03T05:37:34.046 DEBUG:boto:Token: None 2022-09-03T05:37:34.047 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/ 2022-09-03T05:37:34.047 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8= 2022-09-03T05:37:34.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8=', 'Content-Length': '0'} 2022-09-03T05:37:34.051 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000091a86f0fcb87ab93-006312e81e-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.052 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:34.053 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.054 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.054 DEBUG:boto:Method: HEAD 2022-09-03T05:37:34.055 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:34.055 DEBUG:boto:Data: 2022-09-03T05:37:34.056 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.057 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:34.058 DEBUG:boto:Port: 8001 2022-09-03T05:37:34.059 DEBUG:boto:Params: {} 2022-09-03T05:37:34.060 DEBUG:boto:Token: None 2022-09-03T05:37:34.060 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj 2022-09-03T05:37:34.061 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+6yL13+ho9qLQbsdnUpvTDWXisI= 2022-09-03T05:37:34.062 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+6yL13+ho9qLQbsdnUpvTDWXisI=', 'Content-Length': '0'} 2022-09-03T05:37:34.065 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001ae414cefc944e11-006312e81e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.067 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.068 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.068 DEBUG:boto:Method: HEAD 2022-09-03T05:37:34.069 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:34.069 DEBUG:boto:Data: 2022-09-03T05:37:34.070 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.070 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:34.071 DEBUG:boto:Port: 8000 2022-09-03T05:37:34.071 DEBUG:boto:Params: {} 2022-09-03T05:37:34.072 DEBUG:boto:Token: None 2022-09-03T05:37:34.073 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj 2022-09-03T05:37:34.073 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+6yL13+ho9qLQbsdnUpvTDWXisI= 2022-09-03T05:37:34.074 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+6yL13+ho9qLQbsdnUpvTDWXisI=', 'Content-Length': '0'} 2022-09-03T05:37:34.078 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005df5e8a47f22d406-006312e81e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.079 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:34.080 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.080 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.081 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.081 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.082 DEBUG:boto:Method: GET 2022-09-03T05:37:34.082 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.083 DEBUG:boto:Data: 2022-09-03T05:37:34.084 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.084 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:34.085 DEBUG:boto:Port: 8001 2022-09-03T05:37:34.085 DEBUG:boto:Params: {} 2022-09-03T05:37:34.086 DEBUG:boto:Token: None 2022-09-03T05:37:34.086 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.088 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xO+YLUQxjUNaC5s3Yj7/I6vCfIU= 2022-09-03T05:37:34.089 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xO+YLUQxjUNaC5s3Yj7/I6vCfIU=', 'Content-Length': '0'} 2022-09-03T05:37:34.091 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003ea86276c7101ef7-006312e81e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.093 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.095 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.096 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.096 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.097 DEBUG:boto:Method: GET 2022-09-03T05:37:34.098 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.098 DEBUG:boto:Data: 2022-09-03T05:37:34.099 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.099 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:34.100 DEBUG:boto:Port: 8000 2022-09-03T05:37:34.101 DEBUG:boto:Params: {} 2022-09-03T05:37:34.101 DEBUG:boto:Token: None 2022-09-03T05:37:34.103 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.103 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xO+YLUQxjUNaC5s3Yj7/I6vCfIU= 2022-09-03T05:37:34.105 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xO+YLUQxjUNaC5s3Yj7/I6vCfIU=', 'Content-Length': '0'} 2022-09-03T05:37:34.107 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000856b71d68c51ffe6-006312e81e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.154 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:34.155 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:34.156 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.156 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.157 DEBUG:boto:Method: GET 2022-09-03T05:37:34.157 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.158 DEBUG:boto:Data: 2022-09-03T05:37:34.159 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.159 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:34.160 DEBUG:boto:Port: 8001 2022-09-03T05:37:34.160 DEBUG:boto:Params: {} 2022-09-03T05:37:34.161 DEBUG:boto:Token: None 2022-09-03T05:37:34.161 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/ 2022-09-03T05:37:34.162 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8= 2022-09-03T05:37:34.163 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8=', 'Content-Length': '0'} 2022-09-03T05:37:34.166 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d0f2dfe8aeddcb4-006312e81e-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.168 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:34.169 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:34.170 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:34.170 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.171 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.172 DEBUG:boto:Method: GET 2022-09-03T05:37:34.173 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.173 DEBUG:boto:Data: 2022-09-03T05:37:34.174 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.174 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:34.175 DEBUG:boto:Port: 8000 2022-09-03T05:37:34.176 DEBUG:boto:Params: {} 2022-09-03T05:37:34.177 DEBUG:boto:Token: None 2022-09-03T05:37:34.177 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/ 2022-09-03T05:37:34.178 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8= 2022-09-03T05:37:34.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8=', 'Content-Length': '0'} 2022-09-03T05:37:34.185 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000080cce75e95a28c79-006312e81e-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.188 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:34.191 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:34.194 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.196 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.197 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.201 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.202 DEBUG:boto:Method: GET 2022-09-03T05:37:34.203 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.203 DEBUG:boto:Data: 2022-09-03T05:37:34.204 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.205 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:34.205 DEBUG:boto:Port: 8001 2022-09-03T05:37:34.206 DEBUG:boto:Params: {} 2022-09-03T05:37:34.206 DEBUG:boto:Token: None 2022-09-03T05:37:34.207 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.208 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OAaAXSY8l9qmjV1QZUWmMe8/5+M= 2022-09-03T05:37:34.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OAaAXSY8l9qmjV1QZUWmMe8/5+M=', 'Content-Length': '0'} 2022-09-03T05:37:34.211 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000034ae02644586d7e2-006312e81e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.213 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.213 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.214 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.214 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.215 DEBUG:boto:Method: GET 2022-09-03T05:37:34.216 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.216 DEBUG:boto:Data: 2022-09-03T05:37:34.217 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.217 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:34.218 DEBUG:boto:Port: 8000 2022-09-03T05:37:34.219 DEBUG:boto:Params: {} 2022-09-03T05:37:34.219 DEBUG:boto:Token: None 2022-09-03T05:37:34.221 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.222 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OAaAXSY8l9qmjV1QZUWmMe8/5+M= 2022-09-03T05:37:34.223 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OAaAXSY8l9qmjV1QZUWmMe8/5+M=', 'Content-Length': '0'} 2022-09-03T05:37:34.226 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cebe0579ffbe67d1-006312e81e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.228 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.229 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.231 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.232 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.233 DEBUG:boto:Method: HEAD 2022-09-03T05:37:34.234 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.235 DEBUG:boto:Data: 2022-09-03T05:37:34.236 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.237 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:34.238 DEBUG:boto:Port: 8001 2022-09-03T05:37:34.239 DEBUG:boto:Params: {} 2022-09-03T05:37:34.239 DEBUG:boto:Token: None 2022-09-03T05:37:34.240 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.241 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/pJHuPB9YrzsB7Teww5Tu9WwOOc= 2022-09-03T05:37:34.241 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/pJHuPB9YrzsB7Teww5Tu9WwOOc=', 'Content-Length': '0'} 2022-09-03T05:37:34.244 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000cf61fe72178b4829-006312e81e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.245 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.246 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.246 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.247 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.248 DEBUG:boto:Method: HEAD 2022-09-03T05:37:34.248 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.249 DEBUG:boto:Data: 2022-09-03T05:37:34.249 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.250 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:34.250 DEBUG:boto:Port: 8000 2022-09-03T05:37:34.251 DEBUG:boto:Params: {} 2022-09-03T05:37:34.252 DEBUG:boto:Token: None 2022-09-03T05:37:34.252 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.253 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/pJHuPB9YrzsB7Teww5Tu9WwOOc= 2022-09-03T05:37:34.253 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/pJHuPB9YrzsB7Teww5Tu9WwOOc=', 'Content-Length': '0'} 2022-09-03T05:37:34.261 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004e179ae6a71810d3-006312e81e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.263 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:34.264 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.265 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.266 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.267 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.268 DEBUG:boto:Method: GET 2022-09-03T05:37:34.269 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.270 DEBUG:boto:Data: 2022-09-03T05:37:34.271 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.272 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:34.273 DEBUG:boto:Port: 8001 2022-09-03T05:37:34.274 DEBUG:boto:Params: {} 2022-09-03T05:37:34.275 DEBUG:boto:Token: None 2022-09-03T05:37:34.276 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.278 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OAaAXSY8l9qmjV1QZUWmMe8/5+M= 2022-09-03T05:37:34.279 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OAaAXSY8l9qmjV1QZUWmMe8/5+M=', 'Content-Length': '0'} 2022-09-03T05:37:34.281 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000024d0ad7e6d1a8b20-006312e81e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.283 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.284 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.285 DEBUG:boto:path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.286 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.287 DEBUG:boto:Method: GET 2022-09-03T05:37:34.288 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.288 DEBUG:boto:Data: 2022-09-03T05:37:34.289 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.289 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:34.290 DEBUG:boto:Port: 8000 2022-09-03T05:37:34.291 DEBUG:boto:Params: {} 2022-09-03T05:37:34.292 DEBUG:boto:Token: None 2022-09-03T05:37:34.292 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4 2022-09-03T05:37:34.293 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OAaAXSY8l9qmjV1QZUWmMe8/5+M= 2022-09-03T05:37:34.293 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OAaAXSY8l9qmjV1QZUWmMe8/5+M=', 'Content-Length': '0'} 2022-09-03T05:37:34.296 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'Ql7wbqZZa4psf4mYXa6Hq8W4qHqEHV4'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f462ce3eb41da806-006312e81e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.338 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:34.339 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:34.340 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.341 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.342 DEBUG:boto:Method: GET 2022-09-03T05:37:34.344 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.345 DEBUG:boto:Data: 2022-09-03T05:37:34.346 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.348 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:34.349 DEBUG:boto:Port: 8001 2022-09-03T05:37:34.350 DEBUG:boto:Params: {} 2022-09-03T05:37:34.352 DEBUG:boto:Token: None 2022-09-03T05:37:34.353 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/ 2022-09-03T05:37:34.354 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8= 2022-09-03T05:37:34.355 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8=', 'Content-Length': '0'} 2022-09-03T05:37:34.358 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008409d14130ed12c4-006312e81e-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.359 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:34.360 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:34.361 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:34.362 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.363 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.363 DEBUG:boto:Method: GET 2022-09-03T05:37:34.364 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.365 DEBUG:boto:Data: 2022-09-03T05:37:34.366 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.366 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:34.367 DEBUG:boto:Port: 8000 2022-09-03T05:37:34.368 DEBUG:boto:Params: {} 2022-09-03T05:37:34.369 DEBUG:boto:Token: None 2022-09-03T05:37:34.370 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/ 2022-09-03T05:37:34.370 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8= 2022-09-03T05:37:34.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8=', 'Content-Length': '0'} 2022-09-03T05:37:34.386 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000069006cef11df7e08-006312e81e-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.388 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:34.390 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.391 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.392 DEBUG:boto:Method: HEAD 2022-09-03T05:37:34.393 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:34.393 DEBUG:boto:Data: 2022-09-03T05:37:34.394 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.395 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:34.396 DEBUG:boto:Port: 8001 2022-09-03T05:37:34.397 DEBUG:boto:Params: {} 2022-09-03T05:37:34.398 DEBUG:boto:Token: None 2022-09-03T05:37:34.399 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj 2022-09-03T05:37:34.400 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+6yL13+ho9qLQbsdnUpvTDWXisI= 2022-09-03T05:37:34.401 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+6yL13+ho9qLQbsdnUpvTDWXisI=', 'Content-Length': '0'} 2022-09-03T05:37:34.406 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002132e976a62fc51a-006312e81e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.409 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.410 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.411 DEBUG:boto:Method: HEAD 2022-09-03T05:37:34.412 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:34.414 DEBUG:boto:Data: 2022-09-03T05:37:34.416 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.417 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:34.421 DEBUG:boto:Port: 8000 2022-09-03T05:37:34.422 DEBUG:boto:Params: {} 2022-09-03T05:37:34.422 DEBUG:boto:Token: None 2022-09-03T05:37:34.423 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj 2022-09-03T05:37:34.424 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+6yL13+ho9qLQbsdnUpvTDWXisI= 2022-09-03T05:37:34.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+6yL13+ho9qLQbsdnUpvTDWXisI=', 'Content-Length': '0'} 2022-09-03T05:37:34.428 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000932496ce41c4b97-006312e81e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.429 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:34.430 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.431 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.432 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.434 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.435 DEBUG:boto:Method: GET 2022-09-03T05:37:34.437 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.438 DEBUG:boto:Data: 2022-09-03T05:37:34.439 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.440 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:34.441 DEBUG:boto:Port: 8001 2022-09-03T05:37:34.442 DEBUG:boto:Params: {} 2022-09-03T05:37:34.444 DEBUG:boto:Token: None 2022-09-03T05:37:34.445 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.447 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xO+YLUQxjUNaC5s3Yj7/I6vCfIU= 2022-09-03T05:37:34.448 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xO+YLUQxjUNaC5s3Yj7/I6vCfIU=', 'Content-Length': '0'} 2022-09-03T05:37:34.451 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000552b07aca35edb51-006312e81e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.453 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.455 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.456 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.457 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.458 DEBUG:boto:Method: GET 2022-09-03T05:37:34.459 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.460 DEBUG:boto:Data: 2022-09-03T05:37:34.461 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.462 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:34.463 DEBUG:boto:Port: 8000 2022-09-03T05:37:34.464 DEBUG:boto:Params: {} 2022-09-03T05:37:34.465 DEBUG:boto:Token: None 2022-09-03T05:37:34.466 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.467 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xO+YLUQxjUNaC5s3Yj7/I6vCfIU= 2022-09-03T05:37:34.469 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xO+YLUQxjUNaC5s3Yj7/I6vCfIU=', 'Content-Length': '0'} 2022-09-03T05:37:34.472 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000074c576e2cced8ee-006312e81e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.474 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:34.475 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:34.476 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.476 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.477 DEBUG:boto:Method: GET 2022-09-03T05:37:34.478 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.479 DEBUG:boto:Data: 2022-09-03T05:37:34.479 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.480 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:34.482 DEBUG:boto:Port: 8001 2022-09-03T05:37:34.483 DEBUG:boto:Params: {} 2022-09-03T05:37:34.484 DEBUG:boto:Token: None 2022-09-03T05:37:34.485 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/ 2022-09-03T05:37:34.486 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8= 2022-09-03T05:37:34.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8=', 'Content-Length': '0'} 2022-09-03T05:37:34.492 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009396ee51e5973ca2-006312e81e-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.541 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:34.543 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:34.543 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:34.546 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.547 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.548 DEBUG:boto:Method: GET 2022-09-03T05:37:34.549 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.549 DEBUG:boto:Data: 2022-09-03T05:37:34.550 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.551 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:34.551 DEBUG:boto:Port: 8000 2022-09-03T05:37:34.552 DEBUG:boto:Params: {} 2022-09-03T05:37:34.552 DEBUG:boto:Token: None 2022-09-03T05:37:34.553 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/ 2022-09-03T05:37:34.555 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8= 2022-09-03T05:37:34.556 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8=', 'Content-Length': '0'} 2022-09-03T05:37:34.559 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f697dbc8fd61322d-006312e81e-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.560 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:34.561 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:34.561 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.562 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.563 DEBUG:boto:path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.563 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.564 DEBUG:boto:Method: GET 2022-09-03T05:37:34.564 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.565 DEBUG:boto:Data: 2022-09-03T05:37:34.565 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.566 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:34.566 DEBUG:boto:Port: 8001 2022-09-03T05:37:34.567 DEBUG:boto:Params: {} 2022-09-03T05:37:34.568 DEBUG:boto:Token: None 2022-09-03T05:37:34.568 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.569 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q9V/apFp+gj/CGDjQv0dZxj2wW0= 2022-09-03T05:37:34.570 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q9V/apFp+gj/CGDjQv0dZxj2wW0=', 'Content-Length': '0'} 2022-09-03T05:37:34.572 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'vauOlKlWVHmDxq4ZzMBF97KP66F89Pc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000034cee9352527486-006312e81e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.573 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.574 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.574 DEBUG:boto:path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.575 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.576 DEBUG:boto:Method: GET 2022-09-03T05:37:34.576 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.577 DEBUG:boto:Data: 2022-09-03T05:37:34.577 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.578 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:34.578 DEBUG:boto:Port: 8000 2022-09-03T05:37:34.579 DEBUG:boto:Params: {} 2022-09-03T05:37:34.580 DEBUG:boto:Token: None 2022-09-03T05:37:34.580 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.581 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q9V/apFp+gj/CGDjQv0dZxj2wW0= 2022-09-03T05:37:34.581 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q9V/apFp+gj/CGDjQv0dZxj2wW0=', 'Content-Length': '0'} 2022-09-03T05:37:34.584 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'vauOlKlWVHmDxq4ZzMBF97KP66F89Pc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c1342bd00b8370ca-006312e81e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.585 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.586 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.587 DEBUG:boto:path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.588 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.588 DEBUG:boto:Method: HEAD 2022-09-03T05:37:34.589 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.589 DEBUG:boto:Data: 2022-09-03T05:37:34.590 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.590 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:34.591 DEBUG:boto:Port: 8001 2022-09-03T05:37:34.592 DEBUG:boto:Params: {} 2022-09-03T05:37:34.592 DEBUG:boto:Token: None 2022-09-03T05:37:34.593 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.594 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wpIp/H+72Jf3d00ha1z5yH/YHLA= 2022-09-03T05:37:34.594 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wpIp/H+72Jf3d00ha1z5yH/YHLA=', 'Content-Length': '0'} 2022-09-03T05:37:34.597 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'vauOlKlWVHmDxq4ZzMBF97KP66F89Pc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000065ae8d1ab6ab09b3-006312e81e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.598 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.599 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.599 DEBUG:boto:path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.600 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.600 DEBUG:boto:Method: HEAD 2022-09-03T05:37:34.601 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.602 DEBUG:boto:Data: 2022-09-03T05:37:34.603 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.603 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:34.604 DEBUG:boto:Port: 8000 2022-09-03T05:37:34.605 DEBUG:boto:Params: {} 2022-09-03T05:37:34.606 DEBUG:boto:Token: None 2022-09-03T05:37:34.607 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.608 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wpIp/H+72Jf3d00ha1z5yH/YHLA= 2022-09-03T05:37:34.609 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wpIp/H+72Jf3d00ha1z5yH/YHLA=', 'Content-Length': '0'} 2022-09-03T05:37:34.611 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'vauOlKlWVHmDxq4ZzMBF97KP66F89Pc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000026e835461262e370-006312e81e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.611 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:34.612 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.613 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.613 DEBUG:boto:path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.614 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.614 DEBUG:boto:Method: GET 2022-09-03T05:37:34.615 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.615 DEBUG:boto:Data: 2022-09-03T05:37:34.616 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.616 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:34.617 DEBUG:boto:Port: 8001 2022-09-03T05:37:34.617 DEBUG:boto:Params: {} 2022-09-03T05:37:34.618 DEBUG:boto:Token: None 2022-09-03T05:37:34.619 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.620 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q9V/apFp+gj/CGDjQv0dZxj2wW0= 2022-09-03T05:37:34.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q9V/apFp+gj/CGDjQv0dZxj2wW0=', 'Content-Length': '0'} 2022-09-03T05:37:34.622 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'vauOlKlWVHmDxq4ZzMBF97KP66F89Pc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d236d5b553390f14-006312e81e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.623 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.624 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.624 DEBUG:boto:path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.625 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.626 DEBUG:boto:Method: GET 2022-09-03T05:37:34.626 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.627 DEBUG:boto:Data: 2022-09-03T05:37:34.627 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.628 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:34.628 DEBUG:boto:Port: 8000 2022-09-03T05:37:34.629 DEBUG:boto:Params: {} 2022-09-03T05:37:34.629 DEBUG:boto:Token: None 2022-09-03T05:37:34.630 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=vauOlKlWVHmDxq4ZzMBF97KP66F89Pc 2022-09-03T05:37:34.631 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q9V/apFp+gj/CGDjQv0dZxj2wW0= 2022-09-03T05:37:34.631 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q9V/apFp+gj/CGDjQv0dZxj2wW0=', 'Content-Length': '0'} 2022-09-03T05:37:34.633 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'vauOlKlWVHmDxq4ZzMBF97KP66F89Pc'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000024c0bbbd3fc8a4a0-006312e81e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.674 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:34.675 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:34.676 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.677 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.679 DEBUG:boto:Method: GET 2022-09-03T05:37:34.680 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.681 DEBUG:boto:Data: 2022-09-03T05:37:34.682 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.683 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:34.684 DEBUG:boto:Port: 8001 2022-09-03T05:37:34.684 DEBUG:boto:Params: {} 2022-09-03T05:37:34.686 DEBUG:boto:Token: None 2022-09-03T05:37:34.687 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/ 2022-09-03T05:37:34.688 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8= 2022-09-03T05:37:34.689 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8=', 'Content-Length': '0'} 2022-09-03T05:37:34.694 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000a53c115d8045e9a-006312e81e-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.696 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:34.697 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:34.698 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:34.699 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.700 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.701 DEBUG:boto:Method: GET 2022-09-03T05:37:34.701 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.702 DEBUG:boto:Data: 2022-09-03T05:37:34.703 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.704 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:34.704 DEBUG:boto:Port: 8000 2022-09-03T05:37:34.705 DEBUG:boto:Params: {} 2022-09-03T05:37:34.706 DEBUG:boto:Token: None 2022-09-03T05:37:34.707 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/ 2022-09-03T05:37:34.708 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8= 2022-09-03T05:37:34.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8=', 'Content-Length': '0'} 2022-09-03T05:37:34.712 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b9a1b830ea031cc3-006312e81e-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.714 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:34.715 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.716 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.716 DEBUG:boto:Method: HEAD 2022-09-03T05:37:34.717 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:34.717 DEBUG:boto:Data: 2022-09-03T05:37:34.718 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.718 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:34.719 DEBUG:boto:Port: 8001 2022-09-03T05:37:34.720 DEBUG:boto:Params: {} 2022-09-03T05:37:34.721 DEBUG:boto:Token: None 2022-09-03T05:37:34.722 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj 2022-09-03T05:37:34.723 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+6yL13+ho9qLQbsdnUpvTDWXisI= 2022-09-03T05:37:34.724 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+6yL13+ho9qLQbsdnUpvTDWXisI=', 'Content-Length': '0'} 2022-09-03T05:37:34.728 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000030d937c54654e4fe-006312e81e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.729 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.730 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.730 DEBUG:boto:Method: HEAD 2022-09-03T05:37:34.732 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:34.732 DEBUG:boto:Data: 2022-09-03T05:37:34.733 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.733 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:34.734 DEBUG:boto:Port: 8000 2022-09-03T05:37:34.734 DEBUG:boto:Params: {} 2022-09-03T05:37:34.735 DEBUG:boto:Token: None 2022-09-03T05:37:34.735 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj 2022-09-03T05:37:34.736 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+6yL13+ho9qLQbsdnUpvTDWXisI= 2022-09-03T05:37:34.737 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+6yL13+ho9qLQbsdnUpvTDWXisI=', 'Content-Length': '0'} 2022-09-03T05:37:34.740 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000a41f4faf76d68cc4-006312e81e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.741 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:34.742 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.743 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.744 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.744 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.745 DEBUG:boto:Method: GET 2022-09-03T05:37:34.745 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.746 DEBUG:boto:Data: 2022-09-03T05:37:34.746 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.748 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:34.748 DEBUG:boto:Port: 8001 2022-09-03T05:37:34.749 DEBUG:boto:Params: {} 2022-09-03T05:37:34.751 DEBUG:boto:Token: None 2022-09-03T05:37:34.752 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.752 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xO+YLUQxjUNaC5s3Yj7/I6vCfIU= 2022-09-03T05:37:34.753 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xO+YLUQxjUNaC5s3Yj7/I6vCfIU=', 'Content-Length': '0'} 2022-09-03T05:37:34.755 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000af001d3aa500231a-006312e81e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.756 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.757 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.757 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.758 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.759 DEBUG:boto:Method: GET 2022-09-03T05:37:34.759 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.760 DEBUG:boto:Data: 2022-09-03T05:37:34.760 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.761 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:34.761 DEBUG:boto:Port: 8000 2022-09-03T05:37:34.762 DEBUG:boto:Params: {} 2022-09-03T05:37:34.763 DEBUG:boto:Token: None 2022-09-03T05:37:34.763 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:34.764 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xO+YLUQxjUNaC5s3Yj7/I6vCfIU= 2022-09-03T05:37:34.765 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xO+YLUQxjUNaC5s3Yj7/I6vCfIU=', 'Content-Length': '0'} 2022-09-03T05:37:34.767 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000601c7a7d5aeb09f5-006312e81e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.810 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:34.811 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:34.812 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.813 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.813 DEBUG:boto:Method: GET 2022-09-03T05:37:34.814 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.814 DEBUG:boto:Data: 2022-09-03T05:37:34.815 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.815 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:34.816 DEBUG:boto:Port: 8001 2022-09-03T05:37:34.817 DEBUG:boto:Params: {} 2022-09-03T05:37:34.818 DEBUG:boto:Token: None 2022-09-03T05:37:34.818 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/ 2022-09-03T05:37:34.819 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8= 2022-09-03T05:37:34.820 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8=', 'Content-Length': '0'} 2022-09-03T05:37:34.824 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007b00be10ef7b6a53-006312e81e-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.827 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:34.828 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:34.829 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:34.829 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.830 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.830 DEBUG:boto:Method: GET 2022-09-03T05:37:34.831 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.831 DEBUG:boto:Data: 2022-09-03T05:37:34.832 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.833 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:34.833 DEBUG:boto:Port: 8000 2022-09-03T05:37:34.834 DEBUG:boto:Params: {} 2022-09-03T05:37:34.834 DEBUG:boto:Token: None 2022-09-03T05:37:34.835 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/ 2022-09-03T05:37:34.836 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8= 2022-09-03T05:37:34.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8=', 'Content-Length': '0'} 2022-09-03T05:37:34.840 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009f321f93aaa84b9e-006312e81e-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.841 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:34.841 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:34.842 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.843 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.843 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.844 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.845 DEBUG:boto:Method: GET 2022-09-03T05:37:34.845 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.846 DEBUG:boto:Data: 2022-09-03T05:37:34.847 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.847 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:34.848 DEBUG:boto:Port: 8001 2022-09-03T05:37:34.848 DEBUG:boto:Params: {} 2022-09-03T05:37:34.849 DEBUG:boto:Token: None 2022-09-03T05:37:34.850 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.850 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9FhTiFJWul9TFjo28njR8c8PZeo= 2022-09-03T05:37:34.851 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9FhTiFJWul9TFjo28njR8c8PZeo=', 'Content-Length': '0'} 2022-09-03T05:37:34.854 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006f6d50bfa5ed1064-006312e81e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.894 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.895 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.896 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.896 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.897 DEBUG:boto:Method: GET 2022-09-03T05:37:34.897 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.898 DEBUG:boto:Data: 2022-09-03T05:37:34.898 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.899 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:34.900 DEBUG:boto:Port: 8000 2022-09-03T05:37:34.900 DEBUG:boto:Params: {} 2022-09-03T05:37:34.901 DEBUG:boto:Token: None 2022-09-03T05:37:34.902 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.902 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9FhTiFJWul9TFjo28njR8c8PZeo= 2022-09-03T05:37:34.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9FhTiFJWul9TFjo28njR8c8PZeo=', 'Content-Length': '0'} 2022-09-03T05:37:34.906 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001623f1c1cb1a8070-006312e81e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.908 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.909 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.910 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.910 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.911 DEBUG:boto:Method: HEAD 2022-09-03T05:37:34.912 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.913 DEBUG:boto:Data: 2022-09-03T05:37:34.914 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.915 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:34.915 DEBUG:boto:Port: 8001 2022-09-03T05:37:34.916 DEBUG:boto:Params: {} 2022-09-03T05:37:34.917 DEBUG:boto:Token: None 2022-09-03T05:37:34.918 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.919 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KUgNqkU0R3zyAibqjlwsp+Y4v+E= 2022-09-03T05:37:34.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KUgNqkU0R3zyAibqjlwsp+Y4v+E=', 'Content-Length': '0'} 2022-09-03T05:37:34.922 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000049e80f969d0fd768-006312e81e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.923 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.924 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.925 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.926 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.928 DEBUG:boto:Method: HEAD 2022-09-03T05:37:34.929 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.929 DEBUG:boto:Data: 2022-09-03T05:37:34.930 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.930 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:34.933 DEBUG:boto:Port: 8000 2022-09-03T05:37:34.934 DEBUG:boto:Params: {} 2022-09-03T05:37:34.935 DEBUG:boto:Token: None 2022-09-03T05:37:34.936 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.937 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KUgNqkU0R3zyAibqjlwsp+Y4v+E= 2022-09-03T05:37:34.938 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KUgNqkU0R3zyAibqjlwsp+Y4v+E=', 'Content-Length': '0'} 2022-09-03T05:37:34.941 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f950b7d95f6d433d-006312e81e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.942 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:34.943 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.943 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.944 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.945 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.945 DEBUG:boto:Method: GET 2022-09-03T05:37:34.946 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.947 DEBUG:boto:Data: 2022-09-03T05:37:34.947 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.948 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:34.949 DEBUG:boto:Port: 8001 2022-09-03T05:37:34.950 DEBUG:boto:Params: {} 2022-09-03T05:37:34.951 DEBUG:boto:Token: None 2022-09-03T05:37:34.952 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.952 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9FhTiFJWul9TFjo28njR8c8PZeo= 2022-09-03T05:37:34.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9FhTiFJWul9TFjo28njR8c8PZeo=', 'Content-Length': '0'} 2022-09-03T05:37:34.957 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002cd061c496448190-006312e81e-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.959 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:34.961 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:34.961 DEBUG:boto:path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.962 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.963 DEBUG:boto:Method: GET 2022-09-03T05:37:34.964 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.965 DEBUG:boto:Data: 2022-09-03T05:37:34.966 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.967 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:34.968 DEBUG:boto:Port: 8000 2022-09-03T05:37:34.969 DEBUG:boto:Params: {} 2022-09-03T05:37:34.970 DEBUG:boto:Token: None 2022-09-03T05:37:34.971 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/obj?versionId=.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC 2022-09-03T05:37:34.972 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9FhTiFJWul9TFjo28njR8c8PZeo= 2022-09-03T05:37:34.973 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9FhTiFJWul9TFjo28njR8c8PZeo=', 'Content-Length': '0'} 2022-09-03T05:37:34.976 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '.IP2uEcJZCZoYPscJ3WXfeeiA-zoPKC'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000529e56b06822bbdc-006312e81e-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.977 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:34.978 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:34.979 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.979 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.980 DEBUG:boto:Method: GET 2022-09-03T05:37:34.980 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.981 DEBUG:boto:Data: 2022-09-03T05:37:34.981 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.982 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:34.983 DEBUG:boto:Port: 8001 2022-09-03T05:37:34.983 DEBUG:boto:Params: {} 2022-09-03T05:37:34.984 DEBUG:boto:Token: None 2022-09-03T05:37:34.984 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/ 2022-09-03T05:37:34.985 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8= 2022-09-03T05:37:34.986 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8=', 'Content-Length': '0'} 2022-09-03T05:37:34.989 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000371c3a75e01403ef-006312e81e-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:34.991 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:34.992 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:34.993 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:34.994 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.994 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.995 DEBUG:boto:Method: GET 2022-09-03T05:37:34.995 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:34.996 DEBUG:boto:Data: 2022-09-03T05:37:34.996 DEBUG:boto:Headers: {} 2022-09-03T05:37:34.997 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:34.998 DEBUG:boto:Port: 8000 2022-09-03T05:37:34.998 DEBUG:boto:Params: {} 2022-09-03T05:37:34.999 DEBUG:boto:Token: None 2022-09-03T05:37:35.000 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:34 GMT /dsgvcy-19/ 2022-09-03T05:37:35.000 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8= 2022-09-03T05:37:35.001 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nj/An7PPGae6FbgyEXoR6q0TZP8=', 'Content-Length': '0'} 2022-09-03T05:37:35.005 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002c6313cd87c692ba-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.006 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:35.008 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.009 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.011 DEBUG:boto:Method: HEAD 2022-09-03T05:37:35.012 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:35.013 DEBUG:boto:Data: 2022-09-03T05:37:35.014 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.015 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.016 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.017 DEBUG:boto:Params: {} 2022-09-03T05:37:35.018 DEBUG:boto:Token: None 2022-09-03T05:37:35.020 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj 2022-09-03T05:37:35.021 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tLT2Yz6BIo9+jfdq5NwIKc9ejHE= 2022-09-03T05:37:35.022 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tLT2Yz6BIo9+jfdq5NwIKc9ejHE=', 'Content-Length': '0'} 2022-09-03T05:37:35.026 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c8c064337ae6afcb-006312e81f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.028 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.029 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.030 DEBUG:boto:Method: HEAD 2022-09-03T05:37:35.031 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:35.031 DEBUG:boto:Data: 2022-09-03T05:37:35.032 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.032 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:35.033 DEBUG:boto:Port: 8000 2022-09-03T05:37:35.034 DEBUG:boto:Params: {} 2022-09-03T05:37:35.034 DEBUG:boto:Token: None 2022-09-03T05:37:35.035 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj 2022-09-03T05:37:35.036 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tLT2Yz6BIo9+jfdq5NwIKc9ejHE= 2022-09-03T05:37:35.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tLT2Yz6BIo9+jfdq5NwIKc9ejHE=', 'Content-Length': '0'} 2022-09-03T05:37:35.040 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d7bee8aa0e41aced-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.042 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:35.044 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.045 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.047 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.048 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.049 DEBUG:boto:Method: GET 2022-09-03T05:37:35.050 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.051 DEBUG:boto:Data: 2022-09-03T05:37:35.052 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.053 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.054 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.055 DEBUG:boto:Params: {} 2022-09-03T05:37:35.057 DEBUG:boto:Token: None 2022-09-03T05:37:35.058 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.059 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LFfbsZBheRU0XBXqPpzKPB4CqR8= 2022-09-03T05:37:35.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LFfbsZBheRU0XBXqPpzKPB4CqR8=', 'Content-Length': '0'} 2022-09-03T05:37:35.064 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000473cb40d27b0026c-006312e81f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.065 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.065 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.066 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.067 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.067 DEBUG:boto:Method: GET 2022-09-03T05:37:35.068 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.069 DEBUG:boto:Data: 2022-09-03T05:37:35.069 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.070 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:35.070 DEBUG:boto:Port: 8000 2022-09-03T05:37:35.071 DEBUG:boto:Params: {} 2022-09-03T05:37:35.072 DEBUG:boto:Token: None 2022-09-03T05:37:35.072 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.073 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LFfbsZBheRU0XBXqPpzKPB4CqR8= 2022-09-03T05:37:35.074 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LFfbsZBheRU0XBXqPpzKPB4CqR8=', 'Content-Length': '0'} 2022-09-03T05:37:35.076 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000035bddc233088872f-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.119 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:35.120 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:35.121 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.122 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.123 DEBUG:boto:Method: GET 2022-09-03T05:37:35.124 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.125 DEBUG:boto:Data: 2022-09-03T05:37:35.126 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.127 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.128 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.130 DEBUG:boto:Params: {} 2022-09-03T05:37:35.131 DEBUG:boto:Token: None 2022-09-03T05:37:35.132 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/ 2022-09-03T05:37:35.133 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY= 2022-09-03T05:37:35.134 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY=', 'Content-Length': '0'} 2022-09-03T05:37:35.139 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000384a8cee6babc72-006312e81f-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.140 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:35.142 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:35.143 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:35.144 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.145 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.146 DEBUG:boto:Method: GET 2022-09-03T05:37:35.147 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.148 DEBUG:boto:Data: 2022-09-03T05:37:35.149 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.150 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:35.151 DEBUG:boto:Port: 8000 2022-09-03T05:37:35.152 DEBUG:boto:Params: {} 2022-09-03T05:37:35.154 DEBUG:boto:Token: None 2022-09-03T05:37:35.159 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/ 2022-09-03T05:37:35.160 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY= 2022-09-03T05:37:35.161 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY=', 'Content-Length': '0'} 2022-09-03T05:37:35.170 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000042ea73f1511b21e5-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.171 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:35.172 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:35.173 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.173 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.174 DEBUG:boto:path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.174 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.175 DEBUG:boto:Method: GET 2022-09-03T05:37:35.176 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.177 DEBUG:boto:Data: 2022-09-03T05:37:35.177 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.178 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.179 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.179 DEBUG:boto:Params: {} 2022-09-03T05:37:35.180 DEBUG:boto:Token: None 2022-09-03T05:37:35.181 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.182 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OQ6jIEgdyymEesZher1V+vQGSXM= 2022-09-03T05:37:35.183 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OQ6jIEgdyymEesZher1V+vQGSXM=', 'Content-Length': '0'} 2022-09-03T05:37:35.185 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'oYY-icmxg74gZ4carPTk38at9fTxjJi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000099e6c53a13a41a2b-006312e81f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.187 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.187 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.188 DEBUG:boto:path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.189 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.189 DEBUG:boto:Method: GET 2022-09-03T05:37:35.190 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.191 DEBUG:boto:Data: 2022-09-03T05:37:35.191 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.192 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:35.193 DEBUG:boto:Port: 8000 2022-09-03T05:37:35.193 DEBUG:boto:Params: {} 2022-09-03T05:37:35.194 DEBUG:boto:Token: None 2022-09-03T05:37:35.196 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.197 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OQ6jIEgdyymEesZher1V+vQGSXM= 2022-09-03T05:37:35.198 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OQ6jIEgdyymEesZher1V+vQGSXM=', 'Content-Length': '0'} 2022-09-03T05:37:35.202 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'oYY-icmxg74gZ4carPTk38at9fTxjJi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e3bffbce135cef4b-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.204 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.205 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.206 DEBUG:boto:path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.208 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.209 DEBUG:boto:Method: HEAD 2022-09-03T05:37:35.209 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.210 DEBUG:boto:Data: 2022-09-03T05:37:35.212 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.213 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.214 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.215 DEBUG:boto:Params: {} 2022-09-03T05:37:35.216 DEBUG:boto:Token: None 2022-09-03T05:37:35.217 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.218 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WTjgF/UW1022LCx9gaUomDUoyg0= 2022-09-03T05:37:35.219 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WTjgF/UW1022LCx9gaUomDUoyg0=', 'Content-Length': '0'} 2022-09-03T05:37:35.225 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'oYY-icmxg74gZ4carPTk38at9fTxjJi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000164cf5ae53f7a877-006312e81f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.226 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.227 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.228 DEBUG:boto:path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.229 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.230 DEBUG:boto:Method: HEAD 2022-09-03T05:37:35.232 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.233 DEBUG:boto:Data: 2022-09-03T05:37:35.234 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.238 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:35.239 DEBUG:boto:Port: 8000 2022-09-03T05:37:35.239 DEBUG:boto:Params: {} 2022-09-03T05:37:35.240 DEBUG:boto:Token: None 2022-09-03T05:37:35.240 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.241 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WTjgF/UW1022LCx9gaUomDUoyg0= 2022-09-03T05:37:35.242 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WTjgF/UW1022LCx9gaUomDUoyg0=', 'Content-Length': '0'} 2022-09-03T05:37:35.245 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'oYY-icmxg74gZ4carPTk38at9fTxjJi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009d3af3e19dd99226-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.245 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:35.246 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.247 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.247 DEBUG:boto:path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.248 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.249 DEBUG:boto:Method: GET 2022-09-03T05:37:35.249 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.250 DEBUG:boto:Data: 2022-09-03T05:37:35.250 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.251 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.251 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.252 DEBUG:boto:Params: {} 2022-09-03T05:37:35.252 DEBUG:boto:Token: None 2022-09-03T05:37:35.253 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.254 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OQ6jIEgdyymEesZher1V+vQGSXM= 2022-09-03T05:37:35.254 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OQ6jIEgdyymEesZher1V+vQGSXM=', 'Content-Length': '0'} 2022-09-03T05:37:35.256 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'oYY-icmxg74gZ4carPTk38at9fTxjJi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005028a6af790d560c-006312e81f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.257 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.258 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.258 DEBUG:boto:path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.259 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.260 DEBUG:boto:Method: GET 2022-09-03T05:37:35.260 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.261 DEBUG:boto:Data: 2022-09-03T05:37:35.261 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.262 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:35.262 DEBUG:boto:Port: 8000 2022-09-03T05:37:35.263 DEBUG:boto:Params: {} 2022-09-03T05:37:35.263 DEBUG:boto:Token: None 2022-09-03T05:37:35.264 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=oYY-icmxg74gZ4carPTk38at9fTxjJi 2022-09-03T05:37:35.264 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OQ6jIEgdyymEesZher1V+vQGSXM= 2022-09-03T05:37:35.265 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OQ6jIEgdyymEesZher1V+vQGSXM=', 'Content-Length': '0'} 2022-09-03T05:37:35.267 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'oYY-icmxg74gZ4carPTk38at9fTxjJi'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000047e97a43430ef0b9-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.268 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:35.269 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:35.269 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.270 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.270 DEBUG:boto:Method: GET 2022-09-03T05:37:35.271 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.272 DEBUG:boto:Data: 2022-09-03T05:37:35.272 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.273 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.273 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.274 DEBUG:boto:Params: {} 2022-09-03T05:37:35.274 DEBUG:boto:Token: None 2022-09-03T05:37:35.275 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/ 2022-09-03T05:37:35.275 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY= 2022-09-03T05:37:35.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY=', 'Content-Length': '0'} 2022-09-03T05:37:35.279 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ec9981f6f1e4bb39-006312e81f-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.280 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:35.281 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:35.282 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:35.282 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.283 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.283 DEBUG:boto:Method: GET 2022-09-03T05:37:35.284 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.284 DEBUG:boto:Data: 2022-09-03T05:37:35.285 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.285 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:35.286 DEBUG:boto:Port: 8000 2022-09-03T05:37:35.286 DEBUG:boto:Params: {} 2022-09-03T05:37:35.287 DEBUG:boto:Token: None 2022-09-03T05:37:35.288 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/ 2022-09-03T05:37:35.288 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY= 2022-09-03T05:37:35.289 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY=', 'Content-Length': '0'} 2022-09-03T05:37:35.292 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f884ee24dc4d96bf-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.293 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:35.294 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.295 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.295 DEBUG:boto:Method: HEAD 2022-09-03T05:37:35.296 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:35.297 DEBUG:boto:Data: 2022-09-03T05:37:35.297 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.298 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.298 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.299 DEBUG:boto:Params: {} 2022-09-03T05:37:35.300 DEBUG:boto:Token: None 2022-09-03T05:37:35.300 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj 2022-09-03T05:37:35.301 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tLT2Yz6BIo9+jfdq5NwIKc9ejHE= 2022-09-03T05:37:35.301 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tLT2Yz6BIo9+jfdq5NwIKc9ejHE=', 'Content-Length': '0'} 2022-09-03T05:37:35.305 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000613664d5ef54122b-006312e81f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.306 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.307 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.307 DEBUG:boto:Method: HEAD 2022-09-03T05:37:35.308 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:35.308 DEBUG:boto:Data: 2022-09-03T05:37:35.309 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.309 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:35.310 DEBUG:boto:Port: 8000 2022-09-03T05:37:35.310 DEBUG:boto:Params: {} 2022-09-03T05:37:35.311 DEBUG:boto:Token: None 2022-09-03T05:37:35.312 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj 2022-09-03T05:37:35.312 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tLT2Yz6BIo9+jfdq5NwIKc9ejHE= 2022-09-03T05:37:35.313 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tLT2Yz6BIo9+jfdq5NwIKc9ejHE=', 'Content-Length': '0'} 2022-09-03T05:37:35.316 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000715032cc4b2f1116-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.317 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:35.318 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.318 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.319 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.319 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.320 DEBUG:boto:Method: GET 2022-09-03T05:37:35.320 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.321 DEBUG:boto:Data: 2022-09-03T05:37:35.321 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.322 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.323 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.323 DEBUG:boto:Params: {} 2022-09-03T05:37:35.324 DEBUG:boto:Token: None 2022-09-03T05:37:35.324 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.325 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LFfbsZBheRU0XBXqPpzKPB4CqR8= 2022-09-03T05:37:35.326 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LFfbsZBheRU0XBXqPpzKPB4CqR8=', 'Content-Length': '0'} 2022-09-03T05:37:35.328 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e7d0f581700bfbda-006312e81f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.377 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.378 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.379 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.379 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.380 DEBUG:boto:Method: GET 2022-09-03T05:37:35.380 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.381 DEBUG:boto:Data: 2022-09-03T05:37:35.381 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.382 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:35.382 DEBUG:boto:Port: 8000 2022-09-03T05:37:35.383 DEBUG:boto:Params: {} 2022-09-03T05:37:35.384 DEBUG:boto:Token: None 2022-09-03T05:37:35.384 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.385 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LFfbsZBheRU0XBXqPpzKPB4CqR8= 2022-09-03T05:37:35.385 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LFfbsZBheRU0XBXqPpzKPB4CqR8=', 'Content-Length': '0'} 2022-09-03T05:37:35.387 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001561f0e8cc9735ef-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.388 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:35.389 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:35.390 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.391 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.392 DEBUG:boto:Method: GET 2022-09-03T05:37:35.393 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.394 DEBUG:boto:Data: 2022-09-03T05:37:35.395 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.396 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.396 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.397 DEBUG:boto:Params: {} 2022-09-03T05:37:35.398 DEBUG:boto:Token: None 2022-09-03T05:37:35.400 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/ 2022-09-03T05:37:35.401 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY= 2022-09-03T05:37:35.401 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY=', 'Content-Length': '0'} 2022-09-03T05:37:35.405 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003b519ccde8d57db0-006312e81f-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.406 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:35.407 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:35.408 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:35.408 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.409 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.409 DEBUG:boto:Method: GET 2022-09-03T05:37:35.410 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.411 DEBUG:boto:Data: 2022-09-03T05:37:35.411 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.412 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:35.413 DEBUG:boto:Port: 8000 2022-09-03T05:37:35.413 DEBUG:boto:Params: {} 2022-09-03T05:37:35.414 DEBUG:boto:Token: None 2022-09-03T05:37:35.415 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/ 2022-09-03T05:37:35.415 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY= 2022-09-03T05:37:35.416 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY=', 'Content-Length': '0'} 2022-09-03T05:37:35.419 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002fcd7e3a78ff20bd-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.429 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:35.431 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:35.432 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.433 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.434 DEBUG:boto:path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.435 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.436 DEBUG:boto:Method: GET 2022-09-03T05:37:35.437 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.438 DEBUG:boto:Data: 2022-09-03T05:37:35.440 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.441 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.442 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.443 DEBUG:boto:Params: {} 2022-09-03T05:37:35.445 DEBUG:boto:Token: None 2022-09-03T05:37:35.446 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.448 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LTCmu7rxNe13FBjtNebAdwipl9A= 2022-09-03T05:37:35.449 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LTCmu7rxNe13FBjtNebAdwipl9A=', 'Content-Length': '0'} 2022-09-03T05:37:35.457 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bb718d2dca1426c1-006312e81f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.459 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.460 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.461 DEBUG:boto:path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.462 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.463 DEBUG:boto:Method: GET 2022-09-03T05:37:35.465 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.466 DEBUG:boto:Data: 2022-09-03T05:37:35.467 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.468 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:35.469 DEBUG:boto:Port: 8000 2022-09-03T05:37:35.470 DEBUG:boto:Params: {} 2022-09-03T05:37:35.471 DEBUG:boto:Token: None 2022-09-03T05:37:35.473 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.475 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LTCmu7rxNe13FBjtNebAdwipl9A= 2022-09-03T05:37:35.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LTCmu7rxNe13FBjtNebAdwipl9A=', 'Content-Length': '0'} 2022-09-03T05:37:35.479 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004e6e8be9bd8e450a-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.482 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.483 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.484 DEBUG:boto:path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.485 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.486 DEBUG:boto:Method: HEAD 2022-09-03T05:37:35.487 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.488 DEBUG:boto:Data: 2022-09-03T05:37:35.489 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.490 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.491 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.492 DEBUG:boto:Params: {} 2022-09-03T05:37:35.493 DEBUG:boto:Token: None 2022-09-03T05:37:35.494 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.496 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ExmtJJMuoPNL/WURg3EPw8ml3Jg= 2022-09-03T05:37:35.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ExmtJJMuoPNL/WURg3EPw8ml3Jg=', 'Content-Length': '0'} 2022-09-03T05:37:35.500 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000dd8e41a456b99acc-006312e81f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.502 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.503 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.504 DEBUG:boto:path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.505 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.506 DEBUG:boto:Method: HEAD 2022-09-03T05:37:35.508 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.509 DEBUG:boto:Data: 2022-09-03T05:37:35.510 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.511 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:35.511 DEBUG:boto:Port: 8000 2022-09-03T05:37:35.512 DEBUG:boto:Params: {} 2022-09-03T05:37:35.513 DEBUG:boto:Token: None 2022-09-03T05:37:35.514 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.515 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ExmtJJMuoPNL/WURg3EPw8ml3Jg= 2022-09-03T05:37:35.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ExmtJJMuoPNL/WURg3EPw8ml3Jg=', 'Content-Length': '0'} 2022-09-03T05:37:35.519 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000f697c6b8f293eb8d-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.521 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:35.522 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.523 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.524 DEBUG:boto:path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.525 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.526 DEBUG:boto:Method: GET 2022-09-03T05:37:35.527 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.527 DEBUG:boto:Data: 2022-09-03T05:37:35.528 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.528 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.529 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.530 DEBUG:boto:Params: {} 2022-09-03T05:37:35.531 DEBUG:boto:Token: None 2022-09-03T05:37:35.532 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.534 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LTCmu7rxNe13FBjtNebAdwipl9A= 2022-09-03T05:37:35.535 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LTCmu7rxNe13FBjtNebAdwipl9A=', 'Content-Length': '0'} 2022-09-03T05:37:35.543 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ee43e0779b7fcf93-006312e81f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.545 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.546 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.546 DEBUG:boto:path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.548 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.549 DEBUG:boto:Method: GET 2022-09-03T05:37:35.552 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.553 DEBUG:boto:Data: 2022-09-03T05:37:35.554 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.555 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:35.556 DEBUG:boto:Port: 8000 2022-09-03T05:37:35.557 DEBUG:boto:Params: {} 2022-09-03T05:37:35.559 DEBUG:boto:Token: None 2022-09-03T05:37:35.560 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A 2022-09-03T05:37:35.561 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LTCmu7rxNe13FBjtNebAdwipl9A= 2022-09-03T05:37:35.564 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LTCmu7rxNe13FBjtNebAdwipl9A=', 'Content-Length': '0'} 2022-09-03T05:37:35.572 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'xYnysJgMUeKh.Mew5S-gBJZsJgD4g.A'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003fc3b52914475f33-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.573 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:35.574 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:35.574 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.575 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.575 DEBUG:boto:Method: GET 2022-09-03T05:37:35.576 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.576 DEBUG:boto:Data: 2022-09-03T05:37:35.577 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.577 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.578 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.579 DEBUG:boto:Params: {} 2022-09-03T05:37:35.579 DEBUG:boto:Token: None 2022-09-03T05:37:35.580 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/ 2022-09-03T05:37:35.580 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY= 2022-09-03T05:37:35.581 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY=', 'Content-Length': '0'} 2022-09-03T05:37:35.585 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006a1405c868c0fa82-006312e81f-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.586 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:35.587 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:35.587 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:35.588 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.588 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.589 DEBUG:boto:Method: GET 2022-09-03T05:37:35.590 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.590 DEBUG:boto:Data: 2022-09-03T05:37:35.592 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.593 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:35.593 DEBUG:boto:Port: 8000 2022-09-03T05:37:35.594 DEBUG:boto:Params: {} 2022-09-03T05:37:35.595 DEBUG:boto:Token: None 2022-09-03T05:37:35.596 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/ 2022-09-03T05:37:35.596 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY= 2022-09-03T05:37:35.597 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY=', 'Content-Length': '0'} 2022-09-03T05:37:35.601 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ab27e30662b6b021-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.602 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:35.603 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.603 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.604 DEBUG:boto:Method: HEAD 2022-09-03T05:37:35.604 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:35.605 DEBUG:boto:Data: 2022-09-03T05:37:35.605 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.606 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.607 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.607 DEBUG:boto:Params: {} 2022-09-03T05:37:35.608 DEBUG:boto:Token: None 2022-09-03T05:37:35.608 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj 2022-09-03T05:37:35.609 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tLT2Yz6BIo9+jfdq5NwIKc9ejHE= 2022-09-03T05:37:35.610 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tLT2Yz6BIo9+jfdq5NwIKc9ejHE=', 'Content-Length': '0'} 2022-09-03T05:37:35.614 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001d867d2f5aefde2a-006312e81f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.615 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.615 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.616 DEBUG:boto:Method: HEAD 2022-09-03T05:37:35.616 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:35.617 DEBUG:boto:Data: 2022-09-03T05:37:35.617 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.618 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:35.619 DEBUG:boto:Port: 8000 2022-09-03T05:37:35.619 DEBUG:boto:Params: {} 2022-09-03T05:37:35.620 DEBUG:boto:Token: None 2022-09-03T05:37:35.620 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj 2022-09-03T05:37:35.621 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tLT2Yz6BIo9+jfdq5NwIKc9ejHE= 2022-09-03T05:37:35.622 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tLT2Yz6BIo9+jfdq5NwIKc9ejHE=', 'Content-Length': '0'} 2022-09-03T05:37:35.625 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d4d05ba2a3220f40-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.626 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:35.627 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.628 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.629 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.630 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.631 DEBUG:boto:Method: GET 2022-09-03T05:37:35.632 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.632 DEBUG:boto:Data: 2022-09-03T05:37:35.633 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.633 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.634 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.634 DEBUG:boto:Params: {} 2022-09-03T05:37:35.635 DEBUG:boto:Token: None 2022-09-03T05:37:35.636 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.636 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LFfbsZBheRU0XBXqPpzKPB4CqR8= 2022-09-03T05:37:35.638 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LFfbsZBheRU0XBXqPpzKPB4CqR8=', 'Content-Length': '0'} 2022-09-03T05:37:35.640 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000c726f8038230d847-006312e81f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.641 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.641 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.642 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.643 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.643 DEBUG:boto:Method: GET 2022-09-03T05:37:35.644 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.644 DEBUG:boto:Data: 2022-09-03T05:37:35.645 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.645 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:35.646 DEBUG:boto:Port: 8000 2022-09-03T05:37:35.646 DEBUG:boto:Params: {} 2022-09-03T05:37:35.647 DEBUG:boto:Token: None 2022-09-03T05:37:35.648 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.649 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LFfbsZBheRU0XBXqPpzKPB4CqR8= 2022-09-03T05:37:35.649 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LFfbsZBheRU0XBXqPpzKPB4CqR8=', 'Content-Length': '0'} 2022-09-03T05:37:35.652 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000082e612a9121031a9-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.652 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:35.653 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:35.654 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.654 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.655 DEBUG:boto:Method: GET 2022-09-03T05:37:35.655 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.656 DEBUG:boto:Data: 2022-09-03T05:37:35.657 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.657 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.658 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.658 DEBUG:boto:Params: {} 2022-09-03T05:37:35.659 DEBUG:boto:Token: None 2022-09-03T05:37:35.660 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/ 2022-09-03T05:37:35.660 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY= 2022-09-03T05:37:35.661 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY=', 'Content-Length': '0'} 2022-09-03T05:37:35.664 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c7d6e4693b031ed2-006312e81f-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.665 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:35.666 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:35.667 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:35.667 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.668 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.668 DEBUG:boto:Method: GET 2022-09-03T05:37:35.669 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.669 DEBUG:boto:Data: 2022-09-03T05:37:35.670 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.670 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:35.671 DEBUG:boto:Port: 8000 2022-09-03T05:37:35.671 DEBUG:boto:Params: {} 2022-09-03T05:37:35.672 DEBUG:boto:Token: None 2022-09-03T05:37:35.673 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/ 2022-09-03T05:37:35.673 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY= 2022-09-03T05:37:35.674 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY=', 'Content-Length': '0'} 2022-09-03T05:37:35.678 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000047e0eeea61485f9c-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.679 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:35.680 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:35.680 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.681 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.681 DEBUG:boto:path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.682 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.682 DEBUG:boto:Method: GET 2022-09-03T05:37:35.683 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.683 DEBUG:boto:Data: 2022-09-03T05:37:35.684 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.684 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.685 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.685 DEBUG:boto:Params: {} 2022-09-03T05:37:35.687 DEBUG:boto:Token: None 2022-09-03T05:37:35.687 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.690 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FNAEB3EmeZqugCRUL+Bse3JkuFw= 2022-09-03T05:37:35.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FNAEB3EmeZqugCRUL+Bse3JkuFw=', 'Content-Length': '0'} 2022-09-03T05:37:35.695 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001b22dc602ed972e1-006312e81f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.698 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.699 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.700 DEBUG:boto:path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.700 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.701 DEBUG:boto:Method: GET 2022-09-03T05:37:35.702 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.702 DEBUG:boto:Data: 2022-09-03T05:37:35.703 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.704 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:35.705 DEBUG:boto:Port: 8000 2022-09-03T05:37:35.706 DEBUG:boto:Params: {} 2022-09-03T05:37:35.708 DEBUG:boto:Token: None 2022-09-03T05:37:35.709 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.710 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FNAEB3EmeZqugCRUL+Bse3JkuFw= 2022-09-03T05:37:35.711 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FNAEB3EmeZqugCRUL+Bse3JkuFw=', 'Content-Length': '0'} 2022-09-03T05:37:35.714 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b5b73a8c49ef99b2-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.716 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.717 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.717 DEBUG:boto:path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.719 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.720 DEBUG:boto:Method: HEAD 2022-09-03T05:37:35.721 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.722 DEBUG:boto:Data: 2022-09-03T05:37:35.723 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.725 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.725 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.727 DEBUG:boto:Params: {} 2022-09-03T05:37:35.728 DEBUG:boto:Token: None 2022-09-03T05:37:35.729 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.730 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K0iZm/Y2hyvca/XfYcdOyWnqxGY= 2022-09-03T05:37:35.731 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K0iZm/Y2hyvca/XfYcdOyWnqxGY=', 'Content-Length': '0'} 2022-09-03T05:37:35.736 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000bd344c49f42abaa7-006312e81f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.737 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.738 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.739 DEBUG:boto:path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.739 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.741 DEBUG:boto:Method: HEAD 2022-09-03T05:37:35.742 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.743 DEBUG:boto:Data: 2022-09-03T05:37:35.744 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.746 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:35.747 DEBUG:boto:Port: 8000 2022-09-03T05:37:35.749 DEBUG:boto:Params: {} 2022-09-03T05:37:35.752 DEBUG:boto:Token: None 2022-09-03T05:37:35.753 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.754 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K0iZm/Y2hyvca/XfYcdOyWnqxGY= 2022-09-03T05:37:35.754 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K0iZm/Y2hyvca/XfYcdOyWnqxGY=', 'Content-Length': '0'} 2022-09-03T05:37:35.757 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009848dfbecf91d1f5-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.758 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:35.759 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.760 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.760 DEBUG:boto:path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.762 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.763 DEBUG:boto:Method: GET 2022-09-03T05:37:35.763 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.764 DEBUG:boto:Data: 2022-09-03T05:37:35.764 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.765 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.765 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.766 DEBUG:boto:Params: {} 2022-09-03T05:37:35.767 DEBUG:boto:Token: None 2022-09-03T05:37:35.768 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.769 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FNAEB3EmeZqugCRUL+Bse3JkuFw= 2022-09-03T05:37:35.770 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FNAEB3EmeZqugCRUL+Bse3JkuFw=', 'Content-Length': '0'} 2022-09-03T05:37:35.773 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000fd22a879e83ef70d-006312e81f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.819 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.820 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.821 DEBUG:boto:path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.821 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.822 DEBUG:boto:Method: GET 2022-09-03T05:37:35.823 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.824 DEBUG:boto:Data: 2022-09-03T05:37:35.825 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.826 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:35.828 DEBUG:boto:Port: 8000 2022-09-03T05:37:35.829 DEBUG:boto:Params: {} 2022-09-03T05:37:35.830 DEBUG:boto:Token: None 2022-09-03T05:37:35.832 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY 2022-09-03T05:37:35.834 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FNAEB3EmeZqugCRUL+Bse3JkuFw= 2022-09-03T05:37:35.838 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FNAEB3EmeZqugCRUL+Bse3JkuFw=', 'Content-Length': '0'} 2022-09-03T05:37:35.841 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', '4h2LKmX5r3UtRMZOhJ3OXaoQCm4-zvY'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000752af9fb5c59b5a0-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.843 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:35.844 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:35.845 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.846 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.847 DEBUG:boto:Method: GET 2022-09-03T05:37:35.849 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.850 DEBUG:boto:Data: 2022-09-03T05:37:35.851 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.852 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.853 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.854 DEBUG:boto:Params: {} 2022-09-03T05:37:35.855 DEBUG:boto:Token: None 2022-09-03T05:37:35.856 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/ 2022-09-03T05:37:35.857 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY= 2022-09-03T05:37:35.858 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY=', 'Content-Length': '0'} 2022-09-03T05:37:35.862 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000888abfbef83c81cf-006312e81f-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.863 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:35.864 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:35.865 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:35.865 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.866 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.867 DEBUG:boto:Method: GET 2022-09-03T05:37:35.867 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.868 DEBUG:boto:Data: 2022-09-03T05:37:35.868 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.869 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:35.869 DEBUG:boto:Port: 8000 2022-09-03T05:37:35.870 DEBUG:boto:Params: {} 2022-09-03T05:37:35.871 DEBUG:boto:Token: None 2022-09-03T05:37:35.871 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/ 2022-09-03T05:37:35.872 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY= 2022-09-03T05:37:35.873 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY=', 'Content-Length': '0'} 2022-09-03T05:37:35.876 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dafd2d425b5b8dd6-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.877 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:35.878 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.879 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.879 DEBUG:boto:Method: HEAD 2022-09-03T05:37:35.880 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:35.880 DEBUG:boto:Data: 2022-09-03T05:37:35.881 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.882 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.882 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.883 DEBUG:boto:Params: {} 2022-09-03T05:37:35.884 DEBUG:boto:Token: None 2022-09-03T05:37:35.884 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj 2022-09-03T05:37:35.885 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tLT2Yz6BIo9+jfdq5NwIKc9ejHE= 2022-09-03T05:37:35.886 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tLT2Yz6BIo9+jfdq5NwIKc9ejHE=', 'Content-Length': '0'} 2022-09-03T05:37:35.892 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000e9727ba1105fa61-006312e81f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.894 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.895 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.896 DEBUG:boto:Method: HEAD 2022-09-03T05:37:35.897 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:35.898 DEBUG:boto:Data: 2022-09-03T05:37:35.899 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.900 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:35.901 DEBUG:boto:Port: 8000 2022-09-03T05:37:35.902 DEBUG:boto:Params: {} 2022-09-03T05:37:35.903 DEBUG:boto:Token: None 2022-09-03T05:37:35.904 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj 2022-09-03T05:37:35.905 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tLT2Yz6BIo9+jfdq5NwIKc9ejHE= 2022-09-03T05:37:35.907 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tLT2Yz6BIo9+jfdq5NwIKc9ejHE=', 'Content-Length': '0'} 2022-09-03T05:37:35.911 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009f5b99a8f01e54da-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.913 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:35.914 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.915 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.916 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.917 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.919 DEBUG:boto:Method: GET 2022-09-03T05:37:35.920 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.921 DEBUG:boto:Data: 2022-09-03T05:37:35.922 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.922 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.923 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.923 DEBUG:boto:Params: {} 2022-09-03T05:37:35.924 DEBUG:boto:Token: None 2022-09-03T05:37:35.925 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.926 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LFfbsZBheRU0XBXqPpzKPB4CqR8= 2022-09-03T05:37:35.927 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LFfbsZBheRU0XBXqPpzKPB4CqR8=', 'Content-Length': '0'} 2022-09-03T05:37:35.931 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000002aa798ab1792517a-006312e81f-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.933 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:35.934 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:35.934 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.935 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.936 DEBUG:boto:Method: GET 2022-09-03T05:37:35.936 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.937 DEBUG:boto:Data: 2022-09-03T05:37:35.937 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.938 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:35.939 DEBUG:boto:Port: 8000 2022-09-03T05:37:35.940 DEBUG:boto:Params: {} 2022-09-03T05:37:35.940 DEBUG:boto:Token: None 2022-09-03T05:37:35.941 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:35.942 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LFfbsZBheRU0XBXqPpzKPB4CqR8= 2022-09-03T05:37:35.942 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LFfbsZBheRU0XBXqPpzKPB4CqR8=', 'Content-Length': '0'} 2022-09-03T05:37:35.945 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003d7ed6dfae9f7f20-006312e81f-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:35.987 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:35.987 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:35.988 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.989 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.989 DEBUG:boto:Method: GET 2022-09-03T05:37:35.990 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:35.991 DEBUG:boto:Data: 2022-09-03T05:37:35.991 DEBUG:boto:Headers: {} 2022-09-03T05:37:35.993 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:35.993 DEBUG:boto:Port: 8001 2022-09-03T05:37:35.994 DEBUG:boto:Params: {} 2022-09-03T05:37:35.995 DEBUG:boto:Token: None 2022-09-03T05:37:35.996 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:35 GMT /dsgvcy-19/ 2022-09-03T05:37:35.997 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY= 2022-09-03T05:37:35.997 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xcsXRAmjqTWP80/OPJbygpfZXLY=', 'Content-Length': '0'} 2022-09-03T05:37:36.002 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000154689893ec575b8-006312e820-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.003 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:36.004 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:36.004 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:36.005 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.006 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.006 DEBUG:boto:Method: GET 2022-09-03T05:37:36.007 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.008 DEBUG:boto:Data: 2022-09-03T05:37:36.008 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.009 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:36.009 DEBUG:boto:Port: 8000 2022-09-03T05:37:36.010 DEBUG:boto:Params: {} 2022-09-03T05:37:36.011 DEBUG:boto:Token: None 2022-09-03T05:37:36.011 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/ 2022-09-03T05:37:36.012 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HCutlhmE9eNk9C3uFch0YT1sTv8= 2022-09-03T05:37:36.012 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HCutlhmE9eNk9C3uFch0YT1sTv8=', 'Content-Length': '0'} 2022-09-03T05:37:36.016 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b2dcb493a43671ca-006312e820-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.017 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:36.018 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:36.018 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:36.019 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:36.020 DEBUG:boto:path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.020 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.021 DEBUG:boto:Method: GET 2022-09-03T05:37:36.023 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.024 DEBUG:boto:Data: 2022-09-03T05:37:36.025 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.025 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:36.026 DEBUG:boto:Port: 8001 2022-09-03T05:37:36.027 DEBUG:boto:Params: {} 2022-09-03T05:37:36.028 DEBUG:boto:Token: None 2022-09-03T05:37:36.028 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.029 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vGS9fuTSsLVUl3bC2gJtS5MPrZo= 2022-09-03T05:37:36.030 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vGS9fuTSsLVUl3bC2gJtS5MPrZo=', 'Content-Length': '0'} 2022-09-03T05:37:36.032 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'CFKxqazR5g.k3.oWs62bQgs9moHAimI'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000d80e01cf1efce0f0-006312e820-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.033 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:36.034 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:36.035 DEBUG:boto:path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.035 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.036 DEBUG:boto:Method: GET 2022-09-03T05:37:36.036 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.037 DEBUG:boto:Data: 2022-09-03T05:37:36.038 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.038 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:36.039 DEBUG:boto:Port: 8000 2022-09-03T05:37:36.039 DEBUG:boto:Params: {} 2022-09-03T05:37:36.040 DEBUG:boto:Token: None 2022-09-03T05:37:36.041 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.041 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vGS9fuTSsLVUl3bC2gJtS5MPrZo= 2022-09-03T05:37:36.042 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vGS9fuTSsLVUl3bC2gJtS5MPrZo=', 'Content-Length': '0'} 2022-09-03T05:37:36.044 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'CFKxqazR5g.k3.oWs62bQgs9moHAimI'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000000cb2d63a2f51e970-006312e820-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.045 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:36.046 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:36.047 DEBUG:boto:path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.047 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.048 DEBUG:boto:Method: HEAD 2022-09-03T05:37:36.048 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.049 DEBUG:boto:Data: 2022-09-03T05:37:36.049 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.050 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:36.050 DEBUG:boto:Port: 8001 2022-09-03T05:37:36.051 DEBUG:boto:Params: {} 2022-09-03T05:37:36.052 DEBUG:boto:Token: None 2022-09-03T05:37:36.053 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.053 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ijzqy4Hdb4fYVV6FMHpCwalKjkQ= 2022-09-03T05:37:36.054 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ijzqy4Hdb4fYVV6FMHpCwalKjkQ=', 'Content-Length': '0'} 2022-09-03T05:37:36.056 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'CFKxqazR5g.k3.oWs62bQgs9moHAimI'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b6862524e7cb741b-006312e820-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.057 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:36.058 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:36.058 DEBUG:boto:path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.059 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.060 DEBUG:boto:Method: HEAD 2022-09-03T05:37:36.061 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.062 DEBUG:boto:Data: 2022-09-03T05:37:36.062 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.063 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:36.063 DEBUG:boto:Port: 8000 2022-09-03T05:37:36.064 DEBUG:boto:Params: {} 2022-09-03T05:37:36.064 DEBUG:boto:Token: None 2022-09-03T05:37:36.065 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.066 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ijzqy4Hdb4fYVV6FMHpCwalKjkQ= 2022-09-03T05:37:36.067 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ijzqy4Hdb4fYVV6FMHpCwalKjkQ=', 'Content-Length': '0'} 2022-09-03T05:37:36.069 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'CFKxqazR5g.k3.oWs62bQgs9moHAimI'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000ee24ef27bd121e53-006312e820-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.070 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:36.070 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:36.071 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:36.072 DEBUG:boto:path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.072 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.073 DEBUG:boto:Method: GET 2022-09-03T05:37:36.074 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.074 DEBUG:boto:Data: 2022-09-03T05:37:36.075 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.076 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:36.076 DEBUG:boto:Port: 8001 2022-09-03T05:37:36.077 DEBUG:boto:Params: {} 2022-09-03T05:37:36.078 DEBUG:boto:Token: None 2022-09-03T05:37:36.079 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.080 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vGS9fuTSsLVUl3bC2gJtS5MPrZo= 2022-09-03T05:37:36.080 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vGS9fuTSsLVUl3bC2gJtS5MPrZo=', 'Content-Length': '0'} 2022-09-03T05:37:36.083 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'CFKxqazR5g.k3.oWs62bQgs9moHAimI'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000088a454c896e57328-006312e820-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.084 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:36.085 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:36.086 DEBUG:boto:path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.087 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.088 DEBUG:boto:Method: GET 2022-09-03T05:37:36.090 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.091 DEBUG:boto:Data: 2022-09-03T05:37:36.092 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.092 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:36.093 DEBUG:boto:Port: 8000 2022-09-03T05:37:36.094 DEBUG:boto:Params: {} 2022-09-03T05:37:36.094 DEBUG:boto:Token: None 2022-09-03T05:37:36.095 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/obj?versionId=CFKxqazR5g.k3.oWs62bQgs9moHAimI 2022-09-03T05:37:36.095 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vGS9fuTSsLVUl3bC2gJtS5MPrZo= 2022-09-03T05:37:36.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vGS9fuTSsLVUl3bC2gJtS5MPrZo=', 'Content-Length': '0'} 2022-09-03T05:37:36.098 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'CFKxqazR5g.k3.oWs62bQgs9moHAimI'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000649a643495e9841c-006312e820-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.099 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:36.100 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:36.100 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.101 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.101 DEBUG:boto:Method: GET 2022-09-03T05:37:36.102 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.102 DEBUG:boto:Data: 2022-09-03T05:37:36.103 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.104 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:36.104 DEBUG:boto:Port: 8001 2022-09-03T05:37:36.105 DEBUG:boto:Params: {} 2022-09-03T05:37:36.105 DEBUG:boto:Token: None 2022-09-03T05:37:36.106 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/ 2022-09-03T05:37:36.107 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HCutlhmE9eNk9C3uFch0YT1sTv8= 2022-09-03T05:37:36.107 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HCutlhmE9eNk9C3uFch0YT1sTv8=', 'Content-Length': '0'} 2022-09-03T05:37:36.110 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e052bd3b2ec39c49-006312e820-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.160 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:36.162 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:36.164 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:36.165 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.166 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.167 DEBUG:boto:Method: GET 2022-09-03T05:37:36.168 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.169 DEBUG:boto:Data: 2022-09-03T05:37:36.170 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.172 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:36.173 DEBUG:boto:Port: 8000 2022-09-03T05:37:36.174 DEBUG:boto:Params: {} 2022-09-03T05:37:36.176 DEBUG:boto:Token: None 2022-09-03T05:37:36.177 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/ 2022-09-03T05:37:36.177 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HCutlhmE9eNk9C3uFch0YT1sTv8= 2022-09-03T05:37:36.178 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HCutlhmE9eNk9C3uFch0YT1sTv8=', 'Content-Length': '0'} 2022-09-03T05:37:36.182 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ec59221eee3d7e87-006312e820-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.183 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:36.184 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:36.184 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:36.185 DEBUG:boto:Method: HEAD 2022-09-03T05:37:36.185 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:36.186 DEBUG:boto:Data: 2022-09-03T05:37:36.187 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.187 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:36.188 DEBUG:boto:Port: 8001 2022-09-03T05:37:36.188 DEBUG:boto:Params: {} 2022-09-03T05:37:36.189 DEBUG:boto:Token: None 2022-09-03T05:37:36.189 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/obj 2022-09-03T05:37:36.190 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kS3YnCAqGCcho5tPtR27KBlJ3kY= 2022-09-03T05:37:36.191 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kS3YnCAqGCcho5tPtR27KBlJ3kY=', 'Content-Length': '0'} 2022-09-03T05:37:36.196 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000115bcaaf7d40b8af-006312e820-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.197 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:36.197 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:36.198 DEBUG:boto:Method: HEAD 2022-09-03T05:37:36.198 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:36.199 DEBUG:boto:Data: 2022-09-03T05:37:36.200 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.200 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:36.201 DEBUG:boto:Port: 8000 2022-09-03T05:37:36.201 DEBUG:boto:Params: {} 2022-09-03T05:37:36.202 DEBUG:boto:Token: None 2022-09-03T05:37:36.202 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/obj 2022-09-03T05:37:36.203 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kS3YnCAqGCcho5tPtR27KBlJ3kY= 2022-09-03T05:37:36.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kS3YnCAqGCcho5tPtR27KBlJ3kY=', 'Content-Length': '0'} 2022-09-03T05:37:36.207 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000092c0b83189f8f4e1-006312e820-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.208 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:36.209 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:36.210 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:36.210 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:36.211 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:36.211 DEBUG:boto:Method: GET 2022-09-03T05:37:36.212 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:36.212 DEBUG:boto:Data: 2022-09-03T05:37:36.213 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.213 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:36.214 DEBUG:boto:Port: 8001 2022-09-03T05:37:36.215 DEBUG:boto:Params: {} 2022-09-03T05:37:36.215 DEBUG:boto:Token: None 2022-09-03T05:37:36.216 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:36.217 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CI7JBds+IVUfGWV9lPUU+PJIgG8= 2022-09-03T05:37:36.217 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CI7JBds+IVUfGWV9lPUU+PJIgG8=', 'Content-Length': '0'} 2022-09-03T05:37:36.220 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000006525512855529d8d-006312e820-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.221 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:36.222 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:36.223 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:36.224 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:36.225 DEBUG:boto:Method: GET 2022-09-03T05:37:36.226 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:36.227 DEBUG:boto:Data: 2022-09-03T05:37:36.228 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.229 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:36.230 DEBUG:boto:Port: 8000 2022-09-03T05:37:36.231 DEBUG:boto:Params: {} 2022-09-03T05:37:36.233 DEBUG:boto:Token: None 2022-09-03T05:37:36.235 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:36.236 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CI7JBds+IVUfGWV9lPUU+PJIgG8= 2022-09-03T05:37:36.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CI7JBds+IVUfGWV9lPUU+PJIgG8=', 'Content-Length': '0'} 2022-09-03T05:37:36.240 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000005cc2600bc7574d36-006312e820-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.286 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:36.287 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:36.289 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.290 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.293 DEBUG:boto:Method: GET 2022-09-03T05:37:36.294 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.295 DEBUG:boto:Data: 2022-09-03T05:37:36.295 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.296 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:36.297 DEBUG:boto:Port: 8001 2022-09-03T05:37:36.297 DEBUG:boto:Params: {} 2022-09-03T05:37:36.298 DEBUG:boto:Token: None 2022-09-03T05:37:36.299 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/ 2022-09-03T05:37:36.299 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HCutlhmE9eNk9C3uFch0YT1sTv8= 2022-09-03T05:37:36.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HCutlhmE9eNk9C3uFch0YT1sTv8=', 'Content-Length': '0'} 2022-09-03T05:37:36.304 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ca1b3237d6924a56-006312e820-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.305 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:36.307 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:36.309 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:36.309 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.310 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.313 DEBUG:boto:Method: GET 2022-09-03T05:37:36.314 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.315 DEBUG:boto:Data: 2022-09-03T05:37:36.316 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.317 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:36.318 DEBUG:boto:Port: 8000 2022-09-03T05:37:36.319 DEBUG:boto:Params: {} 2022-09-03T05:37:36.320 DEBUG:boto:Token: None 2022-09-03T05:37:36.321 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/ 2022-09-03T05:37:36.322 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HCutlhmE9eNk9C3uFch0YT1sTv8= 2022-09-03T05:37:36.323 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HCutlhmE9eNk9C3uFch0YT1sTv8=', 'Content-Length': '0'} 2022-09-03T05:37:36.327 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001e0b7aad0f189f1f-006312e820-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.328 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:36.330 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:36.332 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:36.333 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:36.334 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.336 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.337 DEBUG:boto:Method: GET 2022-09-03T05:37:36.339 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.340 DEBUG:boto:Data: 2022-09-03T05:37:36.342 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.343 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:36.344 DEBUG:boto:Port: 8001 2022-09-03T05:37:36.345 DEBUG:boto:Params: {} 2022-09-03T05:37:36.346 DEBUG:boto:Token: None 2022-09-03T05:37:36.346 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.347 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ex8amEoSPVpv/IHTdkTyWF/EO/k= 2022-09-03T05:37:36.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ex8amEoSPVpv/IHTdkTyWF/EO/k=', 'Content-Length': '0'} 2022-09-03T05:37:36.351 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000078083f3eeac14893-006312e820-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.352 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:36.353 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:36.353 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.354 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.355 DEBUG:boto:Method: GET 2022-09-03T05:37:36.355 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.356 DEBUG:boto:Data: 2022-09-03T05:37:36.356 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.357 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:36.360 DEBUG:boto:Port: 8000 2022-09-03T05:37:36.362 DEBUG:boto:Params: {} 2022-09-03T05:37:36.362 DEBUG:boto:Token: None 2022-09-03T05:37:36.363 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.364 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ex8amEoSPVpv/IHTdkTyWF/EO/k= 2022-09-03T05:37:36.365 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ex8amEoSPVpv/IHTdkTyWF/EO/k=', 'Content-Length': '0'} 2022-09-03T05:37:36.367 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000004a1a650e6d58f44c-006312e820-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.369 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:36.370 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:36.371 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.372 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.372 DEBUG:boto:Method: HEAD 2022-09-03T05:37:36.373 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.373 DEBUG:boto:Data: 2022-09-03T05:37:36.374 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.374 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:36.375 DEBUG:boto:Port: 8001 2022-09-03T05:37:36.375 DEBUG:boto:Params: {} 2022-09-03T05:37:36.376 DEBUG:boto:Token: None 2022-09-03T05:37:36.377 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.377 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RKdgcIk2DyIpYj7m+9v5FIUPtTc= 2022-09-03T05:37:36.378 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RKdgcIk2DyIpYj7m+9v5FIUPtTc=', 'Content-Length': '0'} 2022-09-03T05:37:36.380 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000057d90dc076c8dbd9-006312e820-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.381 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:36.382 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:36.383 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.383 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.384 DEBUG:boto:Method: HEAD 2022-09-03T05:37:36.384 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.385 DEBUG:boto:Data: 2022-09-03T05:37:36.385 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.386 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:36.386 DEBUG:boto:Port: 8000 2022-09-03T05:37:36.387 DEBUG:boto:Params: {} 2022-09-03T05:37:36.388 DEBUG:boto:Token: None 2022-09-03T05:37:36.388 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.389 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RKdgcIk2DyIpYj7m+9v5FIUPtTc= 2022-09-03T05:37:36.389 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RKdgcIk2DyIpYj7m+9v5FIUPtTc=', 'Content-Length': '0'} 2022-09-03T05:37:36.392 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000b9befa8feb6ee957-006312e820-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.393 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:36.394 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:36.394 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:36.395 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.396 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.397 DEBUG:boto:Method: GET 2022-09-03T05:37:36.397 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.398 DEBUG:boto:Data: 2022-09-03T05:37:36.399 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.400 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:36.400 DEBUG:boto:Port: 8001 2022-09-03T05:37:36.401 DEBUG:boto:Params: {} 2022-09-03T05:37:36.402 DEBUG:boto:Token: None 2022-09-03T05:37:36.403 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.404 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ex8amEoSPVpv/IHTdkTyWF/EO/k= 2022-09-03T05:37:36.405 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ex8amEoSPVpv/IHTdkTyWF/EO/k=', 'Content-Length': '0'} 2022-09-03T05:37:36.407 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000eea038e97773b6e6-006312e820-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.451 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:36.452 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:36.452 DEBUG:boto:path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.453 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.454 DEBUG:boto:Method: GET 2022-09-03T05:37:36.454 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.455 DEBUG:boto:Data: 2022-09-03T05:37:36.455 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.456 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:36.456 DEBUG:boto:Port: 8000 2022-09-03T05:37:36.457 DEBUG:boto:Params: {} 2022-09-03T05:37:36.458 DEBUG:boto:Token: None 2022-09-03T05:37:36.458 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/obj?versionId=kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1 2022-09-03T05:37:36.459 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ex8amEoSPVpv/IHTdkTyWF/EO/k= 2022-09-03T05:37:36.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ex8amEoSPVpv/IHTdkTyWF/EO/k=', 'Content-Length': '0'} 2022-09-03T05:37:36.462 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:50 GMT'), ('x-amz-version-id', 'kJSzTg.tDVQ-nL1nX84.W7ifJxIseT1'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000009a8141034da3ff89-006312e820-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.463 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:36.464 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:36.464 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.465 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.466 DEBUG:boto:Method: GET 2022-09-03T05:37:36.466 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.467 DEBUG:boto:Data: 2022-09-03T05:37:36.467 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.468 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:36.469 DEBUG:boto:Port: 8001 2022-09-03T05:37:36.469 DEBUG:boto:Params: {} 2022-09-03T05:37:36.470 DEBUG:boto:Token: None 2022-09-03T05:37:36.470 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/ 2022-09-03T05:37:36.471 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HCutlhmE9eNk9C3uFch0YT1sTv8= 2022-09-03T05:37:36.472 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HCutlhmE9eNk9C3uFch0YT1sTv8=', 'Content-Length': '0'} 2022-09-03T05:37:36.475 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000077dd1380c6e6b85-006312e820-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.476 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:36.477 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:36.477 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:36.478 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.479 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.479 DEBUG:boto:Method: GET 2022-09-03T05:37:36.480 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.481 DEBUG:boto:Data: 2022-09-03T05:37:36.481 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.482 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:36.482 DEBUG:boto:Port: 8000 2022-09-03T05:37:36.483 DEBUG:boto:Params: {} 2022-09-03T05:37:36.484 DEBUG:boto:Token: None 2022-09-03T05:37:36.484 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/ 2022-09-03T05:37:36.485 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HCutlhmE9eNk9C3uFch0YT1sTv8= 2022-09-03T05:37:36.486 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HCutlhmE9eNk9C3uFch0YT1sTv8=', 'Content-Length': '0'} 2022-09-03T05:37:36.489 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007a11365117c72d83-006312e820-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.490 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:36.491 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:36.492 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:36.492 DEBUG:boto:Method: HEAD 2022-09-03T05:37:36.493 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:36.494 DEBUG:boto:Data: 2022-09-03T05:37:36.494 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.495 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:36.495 DEBUG:boto:Port: 8001 2022-09-03T05:37:36.496 DEBUG:boto:Params: {} 2022-09-03T05:37:36.496 DEBUG:boto:Token: None 2022-09-03T05:37:36.497 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/obj 2022-09-03T05:37:36.498 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kS3YnCAqGCcho5tPtR27KBlJ3kY= 2022-09-03T05:37:36.498 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kS3YnCAqGCcho5tPtR27KBlJ3kY=', 'Content-Length': '0'} 2022-09-03T05:37:36.503 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx00000e486ba878068fe36-006312e820-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.504 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:36.505 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:36.505 DEBUG:boto:Method: HEAD 2022-09-03T05:37:36.506 DEBUG:boto:Path: /dsgvcy-19/obj 2022-09-03T05:37:36.506 DEBUG:boto:Data: 2022-09-03T05:37:36.507 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.508 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:36.508 DEBUG:boto:Port: 8000 2022-09-03T05:37:36.509 DEBUG:boto:Params: {} 2022-09-03T05:37:36.509 DEBUG:boto:Token: None 2022-09-03T05:37:36.510 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/obj 2022-09-03T05:37:36.511 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kS3YnCAqGCcho5tPtR27KBlJ3kY= 2022-09-03T05:37:36.511 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kS3YnCAqGCcho5tPtR27KBlJ3kY=', 'Content-Length': '0'} 2022-09-03T05:37:36.517 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000003fac2cb082a442c6-006312e820-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.518 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:37:36.519 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:36.520 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:36.521 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:36.521 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:36.523 DEBUG:boto:Method: GET 2022-09-03T05:37:36.523 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:36.524 DEBUG:boto:Data: 2022-09-03T05:37:36.525 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.525 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:36.526 DEBUG:boto:Port: 8001 2022-09-03T05:37:36.527 DEBUG:boto:Params: {} 2022-09-03T05:37:36.528 DEBUG:boto:Token: None 2022-09-03T05:37:36.528 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:36.529 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CI7JBds+IVUfGWV9lPUU+PJIgG8= 2022-09-03T05:37:36.530 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CI7JBds+IVUfGWV9lPUU+PJIgG8=', 'Content-Length': '0'} 2022-09-03T05:37:36.532 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx000001ce205d402518a8e-006312e820-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.578 DEBUG:boto:path=/dsgvcy-19/obj 2022-09-03T05:37:36.579 DEBUG:boto:auth_path=/dsgvcy-19/obj 2022-09-03T05:37:36.580 DEBUG:boto:path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:36.581 DEBUG:boto:auth_path=/dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:36.582 DEBUG:boto:Method: GET 2022-09-03T05:37:36.583 DEBUG:boto:Path: /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:36.584 DEBUG:boto:Data: 2022-09-03T05:37:36.585 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.586 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:36.588 DEBUG:boto:Port: 8000 2022-09-03T05:37:36.589 DEBUG:boto:Params: {} 2022-09-03T05:37:36.590 DEBUG:boto:Token: None 2022-09-03T05:37:36.592 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/obj?versionId=FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R 2022-09-03T05:37:36.593 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CI7JBds+IVUfGWV9lPUU+PJIgG8= 2022-09-03T05:37:36.594 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CI7JBds+IVUfGWV9lPUU+PJIgG8=', 'Content-Length': '0'} 2022-09-03T05:37:36.598 DEBUG:boto:Response headers: [('Content-Length', '8'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:34:51 GMT'), ('x-amz-version-id', 'FV.ndVyKCalSXA6cZK3aQWN-LzBZK3R'), ('x-rgw-object-type', 'Normal'), ('ETag', '"966634ebf2fc135707d6753692bf4b1e"'), ('x-amz-request-id', 'tx0000042b841f693057fc9-006312e820-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.600 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:36.601 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:36.602 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.603 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.604 DEBUG:boto:Method: GET 2022-09-03T05:37:36.605 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.606 DEBUG:boto:Data: 2022-09-03T05:37:36.608 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.609 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:37:36.611 DEBUG:boto:Port: 8001 2022-09-03T05:37:36.612 DEBUG:boto:Params: {} 2022-09-03T05:37:36.613 DEBUG:boto:Token: None 2022-09-03T05:37:36.615 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/ 2022-09-03T05:37:36.616 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HCutlhmE9eNk9C3uFch0YT1sTv8= 2022-09-03T05:37:36.617 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HCutlhmE9eNk9C3uFch0YT1sTv8=', 'Content-Length': '0'} 2022-09-03T05:37:36.622 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000002abd9dfb62a9d7e-006312e820-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.623 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:36.625 DEBUG:boto:path=/dsgvcy-19/ 2022-09-03T05:37:36.626 DEBUG:boto:auth_path=/dsgvcy-19/ 2022-09-03T05:37:36.627 DEBUG:boto:path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.628 DEBUG:boto:auth_path=/dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.629 DEBUG:boto:Method: GET 2022-09-03T05:37:36.630 DEBUG:boto:Path: /dsgvcy-19/?prefix=obj 2022-09-03T05:37:36.631 DEBUG:boto:Data: 2022-09-03T05:37:36.632 DEBUG:boto:Headers: {} 2022-09-03T05:37:36.632 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:37:36.633 DEBUG:boto:Port: 8000 2022-09-03T05:37:36.633 DEBUG:boto:Params: {} 2022-09-03T05:37:36.634 DEBUG:boto:Token: None 2022-09-03T05:37:36.635 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:37:36 GMT /dsgvcy-19/ 2022-09-03T05:37:36.635 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HCutlhmE9eNk9C3uFch0YT1sTv8= 2022-09-03T05:37:36.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HCutlhmE9eNk9C3uFch0YT1sTv8=', 'Content-Length': '0'} 2022-09-03T05:37:36.639 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000019eba1e991f7a63b-006312e820-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:37:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:36.642 DEBUG:boto:b'dsgvcy-19obj1000falseobj2022-09-03T05:34:51.414Z"966634ebf2fc135707d6753692bf4b1e"8STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:37:36.644 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-19 zones={test-zone3, test-zone2} 2022-09-03T05:37:36.645 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone1 target=test-zone2 2022-09-03T05:37:36.646 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:36.647 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:36.649 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:37:36.710 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:37:36.710 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:37:36.772 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:37:36.773 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:36.773 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002768", 2022-09-03T05:37:36.773 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:31:26.029653Z" 2022-09-03T05:37:36.774 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:36.774 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:36.775 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000001640", 2022-09-03T05:37:36.775 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:36:25.524733Z" 2022-09-03T05:37:36.776 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:36.776 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:36.777 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:37:36.777 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:37:06.825525Z" 2022-09-03T05:37:36.778 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:36.778 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:36.779 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:37:36.779 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:37:36.781 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:36.782 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:37:36.784 DEBUG:rgw_multi.tests:data markers for zone=test-zone1 markers={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000001640', 2: '00000000000000000000:00000000000000006434', 3: ''} 2022-09-03T05:37:36.785 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2022-09-03T05:37:36.786 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:36.786 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:36.787 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:37:36.848 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:37:36.849 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:37:36.891 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:37:36.891 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:37:36.892 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:37:36.893 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:37:36.894 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:37:36.895 INFO:teuthology.orchestra.run.smithi189.stdout: "instance_id": 4093329069989467412 2022-09-03T05:37:36.896 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:36.896 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:37:36.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:36.898 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:37:36.898 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:36.899 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:37:36.899 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000002768", 2022-09-03T05:37:36.900 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:36.900 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:37:36.901 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:36.901 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:31:26.029653Z" 2022-09-03T05:37:36.902 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:36.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:36.903 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:36.903 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:37:36.904 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:36.904 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:37:36.905 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000001640", 2022-09-03T05:37:36.906 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:36.907 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:37:36.907 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:36.907 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:36:25.524733Z" 2022-09-03T05:37:36.908 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:36.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:36.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:36.908 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:37:36.909 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:36.909 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:37:36.910 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:37:36.910 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:36.911 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:37:36.911 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:36.912 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:37:06.825525Z" 2022-09-03T05:37:36.912 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:36.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:36.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:36.917 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:37:36.918 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:36.918 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:37:36.919 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:37:36.919 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:36.920 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:37:36.920 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:36.921 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000" 2022-09-03T05:37:36.921 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:36.922 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:36.922 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:37:36.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:36.923 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:37:36.923 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 0, 2022-09-03T05:37:36.923 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 0 2022-09-03T05:37:36.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:36.924 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:37:36Z" 2022-09-03T05:37:36.924 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:37:36.926 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 4093329069989467412 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002768", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:26.029653Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000001640", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:36:25.524733Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000006434", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:37:06.825525Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:37:36Z" } 2022-09-03T05:37:36.930 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002768', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:26.029653Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000001640', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:36:25.524733Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006434', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:37:06.825525Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000'}}] 2022-09-03T05:37:36.931 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000001640', 2: '00000000000000000000:00000000000000006434', 3: ''} 2022-09-03T05:37:36.932 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000001640', 2: '00000000000000000000:00000000000000006434', 3: ''} 2022-09-03T05:37:36.935 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2022-09-03T05:37:36.939 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone1 target=test-zone3 2022-09-03T05:37:36.940 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:36.941 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:36.942 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:37:37.009 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:37:37.009 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:37:37.069 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:37:37.070 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:37.070 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002768", 2022-09-03T05:37:37.071 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:31:26.029653Z" 2022-09-03T05:37:37.071 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:37.071 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:37.072 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000001640", 2022-09-03T05:37:37.072 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:36:25.524733Z" 2022-09-03T05:37:37.073 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:37.073 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:37.074 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:37:37.075 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:37:06.825525Z" 2022-09-03T05:37:37.076 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:37.076 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:37.077 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:37:37.077 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:37:37.078 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:37.078 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:37:37.081 DEBUG:rgw_multi.tests:data markers for zone=test-zone1 markers={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000001640', 2: '00000000000000000000:00000000000000006434', 3: ''} 2022-09-03T05:37:37.082 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2022-09-03T05:37:37.088 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:37.089 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:37.091 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:37:37.144 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:37:37.145 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:37:37.190 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:37:37.191 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:37:37.191 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:37:37.192 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:37:37.192 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:37:37.193 INFO:teuthology.orchestra.run.smithi181.stdout: "instance_id": 16186606431531372785 2022-09-03T05:37:37.193 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:37.194 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:37:37.194 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:37.195 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:37:37.195 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:37:37.196 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:37:37.196 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002768", 2022-09-03T05:37:37.196 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:37:37.196 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:37:37.197 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:37:37.197 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:31:26.029653Z" 2022-09-03T05:37:37.197 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:37.198 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:37.198 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:37.199 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:37:37.199 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:37:37.199 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:37:37.200 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000001640", 2022-09-03T05:37:37.200 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:37:37.201 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:37:37.201 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:37:37.202 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:36:25.524733Z" 2022-09-03T05:37:37.202 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:37.203 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:37.203 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:37.204 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:37:37.205 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:37:37.205 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:37:37.206 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:37:37.206 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:37:37.207 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:37:37.207 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:37:37.208 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:37:06.825525Z" 2022-09-03T05:37:37.208 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:37.209 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:37.209 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:37.209 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:37:37.210 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:37:37.211 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:37:37.211 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:37:37.212 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:37:37.212 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:37:37.212 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:37:37.213 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000" 2022-09-03T05:37:37.213 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:37.214 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:37.214 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:37:37.215 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:37.215 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:37:37.216 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 0, 2022-09-03T05:37:37.216 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 0 2022-09-03T05:37:37.217 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:37.217 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:37:37Z" 2022-09-03T05:37:37.218 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:37:37.220 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 16186606431531372785 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002768", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:26.029653Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000001640", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:36:25.524733Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000006434", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:37:06.825525Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:37:37Z" } 2022-09-03T05:37:37.221 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002768', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:26.029653Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000001640', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:36:25.524733Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006434', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:37:06.825525Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000'}}] 2022-09-03T05:37:37.222 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000001640', 2: '00000000000000000000:00000000000000006434', 3: ''} 2022-09-03T05:37:37.223 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000001640', 2: '00000000000000000000:00000000000000006434', 3: ''} 2022-09-03T05:37:37.224 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2022-09-03T05:37:37.225 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone1 target=test-zone4 2022-09-03T05:37:37.226 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:37.227 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:37.228 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:37:37.290 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:37:37.290 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:37:37.349 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:37:37.350 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:37.351 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002768", 2022-09-03T05:37:37.351 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:31:26.029653Z" 2022-09-03T05:37:37.352 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:37.352 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:37.352 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000001640", 2022-09-03T05:37:37.353 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:36:25.524733Z" 2022-09-03T05:37:37.353 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:37.354 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:37.354 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:37:37.355 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:37:06.825525Z" 2022-09-03T05:37:37.355 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:37.355 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:37.356 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:37:37.356 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:37:37.357 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:37.357 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:37:37.362 DEBUG:rgw_multi.tests:data markers for zone=test-zone1 markers={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000001640', 2: '00000000000000000000:00000000000000006434', 3: ''} 2022-09-03T05:37:37.363 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone4 source_zone=test-zone1 2022-09-03T05:37:37.365 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:37.366 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:37.367 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:37:37.424 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:37:37.425 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:37:37.478 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:37:37.478 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:37:37.479 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:37:37.479 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:37:37.480 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:37:37.480 INFO:teuthology.orchestra.run.smithi189.stdout: "instance_id": 13887610924008747321 2022-09-03T05:37:37.481 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:37.482 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:37:37.482 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:37.483 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:37:37.483 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:37.484 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:37:37.484 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000002768", 2022-09-03T05:37:37.485 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:37.485 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:37:37.486 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:37.486 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:31:26.029653Z" 2022-09-03T05:37:37.487 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:37.487 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:37.487 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:37.488 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:37:37.488 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:37.489 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:37:37.489 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000001640", 2022-09-03T05:37:37.490 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:37.490 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:37:37.491 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:37.495 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:36:25.524733Z" 2022-09-03T05:37:37.496 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:37.496 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:37.497 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:37.497 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:37:37.500 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:37.500 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:37:37.501 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:37:37.501 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:37.502 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:37:37.502 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:37.503 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:37:06.825525Z" 2022-09-03T05:37:37.503 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:37.505 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:37.505 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:37.506 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:37:37.506 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:37.507 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:37:37.507 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:37:37.508 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:37.509 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:37:37.510 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:37.510 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000" 2022-09-03T05:37:37.511 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:37.511 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:37.512 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:37:37.512 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:37.513 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:37:37.513 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 0, 2022-09-03T05:37:37.514 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 0 2022-09-03T05:37:37.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:37.515 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:37:37Z" 2022-09-03T05:37:37.516 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:37:37.518 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 13887610924008747321 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002768", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:26.029653Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000001640", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:36:25.524733Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000006434", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:37:06.825525Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:37:37Z" } 2022-09-03T05:37:37.520 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002768', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:26.029653Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000001640', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:36:25.524733Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006434', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:37:06.825525Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000'}}] 2022-09-03T05:37:37.521 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000001640', 2: '00000000000000000000:00000000000000006434', 3: ''} 2022-09-03T05:37:37.522 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000001640', 2: '00000000000000000000:00000000000000006434', 3: ''} 2022-09-03T05:37:37.523 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone4 source_zone=test-zone1 2022-09-03T05:37:37.524 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone2 target=test-zone1 2022-09-03T05:37:37.525 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:37.526 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:37.527 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:37:37.579 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:37:37.579 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:37:37.621 INFO:teuthology.orchestra.run.smithi189.stdout:[ 2022-09-03T05:37:37.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:37.622 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:37:37.623 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:31:03.799590Z" 2022-09-03T05:37:37.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:37.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:37.625 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000001499", 2022-09-03T05:37:37.625 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:36:03.583371Z" 2022-09-03T05:37:37.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:37.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:37.627 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:37:37.627 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:34:57.577753Z" 2022-09-03T05:37:37.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:37.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:37.628 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:37:37.628 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "0.000000" 2022-09-03T05:37:37.629 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:37.629 INFO:teuthology.orchestra.run.smithi189.stdout:] 2022-09-03T05:37:37.631 DEBUG:rgw_multi.tests:data markers for zone=test-zone2 markers={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000001499', 2: '00000000000000000000:00000000000000006434', 3: ''} 2022-09-03T05:37:37.632 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2022-09-03T05:37:37.633 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:37.634 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:37.636 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:37:37.693 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:37:37.694 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:37:37.769 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:37:37.773 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:37:37.773 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:37:37.774 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:37:37.775 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:37:37.775 INFO:teuthology.orchestra.run.smithi181.stdout: "instance_id": 5551903584477280978 2022-09-03T05:37:37.776 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:37.776 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:37:37.777 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:37.777 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:37:37.778 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:37:37.778 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:37:37.779 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:37:37.779 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:37:37.780 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:37:37.780 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:37:37.781 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:31:03.799590Z" 2022-09-03T05:37:37.781 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:37.782 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:37.782 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:37.783 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:37:37.783 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:37:37.784 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:37:37.784 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000001499", 2022-09-03T05:37:37.785 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:37:37.785 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:37:37.786 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:37:37.787 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:36:03.583371Z" 2022-09-03T05:37:37.788 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:37.788 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:37.789 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:37.790 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:37:37.791 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:37:37.791 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:37:37.792 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:37:37.792 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:37:37.793 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:37:37.793 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:37:37.794 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:34:57.577753Z" 2022-09-03T05:37:37.794 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:37.795 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:37.796 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:37.797 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:37:37.797 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:37:37.798 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:37:37.798 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:37:37.799 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:37:37.800 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:37:37.800 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:37:37.801 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000" 2022-09-03T05:37:37.801 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:37.802 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:37.803 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:37:37.803 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:37.804 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:37:37.804 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 0, 2022-09-03T05:37:37.805 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 0 2022-09-03T05:37:37.805 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:37.806 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:37:37Z" 2022-09-03T05:37:37.807 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:37:37.812 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 5551903584477280978 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002627", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:03.799590Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000001499", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:36:03.583371Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000006434", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:34:57.577753Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:37:37Z" } 2022-09-03T05:37:37.814 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002627', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:03.799590Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000001499', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:36:03.583371Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006434', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:34:57.577753Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000'}}] 2022-09-03T05:37:37.815 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000001499', 2: '00000000000000000000:00000000000000006434', 3: ''} 2022-09-03T05:37:37.815 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000001499', 2: '00000000000000000000:00000000000000006434', 3: ''} 2022-09-03T05:37:37.816 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2022-09-03T05:37:37.816 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone2 target=test-zone3 2022-09-03T05:37:37.817 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:37.818 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:37.820 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:37:37.874 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:37:37.877 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:37:37.937 INFO:teuthology.orchestra.run.smithi189.stdout:[ 2022-09-03T05:37:37.938 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:37.938 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:37:37.939 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:31:03.799590Z" 2022-09-03T05:37:37.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:37.940 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:37.940 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000001499", 2022-09-03T05:37:37.941 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:36:03.583371Z" 2022-09-03T05:37:37.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:37.943 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:37.944 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:37:37.945 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:34:57.577753Z" 2022-09-03T05:37:37.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:37.946 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:37.947 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:37:37.947 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "0.000000" 2022-09-03T05:37:37.948 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:37.949 INFO:teuthology.orchestra.run.smithi189.stdout:] 2022-09-03T05:37:37.951 DEBUG:rgw_multi.tests:data markers for zone=test-zone2 markers={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000001499', 2: '00000000000000000000:00000000000000006434', 3: ''} 2022-09-03T05:37:37.951 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone3 source_zone=test-zone2 2022-09-03T05:37:37.952 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:37.953 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:37.954 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:37:38.010 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:37:38.010 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:37:38.066 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:37:38.067 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:37:38.067 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:37:38.068 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:37:38.068 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:37:38.069 INFO:teuthology.orchestra.run.smithi181.stdout: "instance_id": 14474342186618979890 2022-09-03T05:37:38.070 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:38.070 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:37:38.071 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:38.071 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:37:38.072 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:37:38.072 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:37:38.073 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:37:38.073 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:37:38.073 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:37:38.074 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:37:38.074 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:31:03.799590Z" 2022-09-03T05:37:38.075 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:38.075 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:38.075 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:38.076 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:37:38.076 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:37:38.076 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:37:38.077 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000001499", 2022-09-03T05:37:38.077 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:37:38.077 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:37:38.077 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:37:38.078 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:36:03.583371Z" 2022-09-03T05:37:38.078 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:38.078 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:38.079 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:38.079 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:37:38.080 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:37:38.081 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:37:38.081 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:37:38.081 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:37:38.082 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:37:38.083 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:37:38.083 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:34:57.577753Z" 2022-09-03T05:37:38.084 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:38.084 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:38.085 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:38.085 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:37:38.087 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:37:38.088 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:37:38.088 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:37:38.089 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:37:38.089 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:37:38.090 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:37:38.090 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000" 2022-09-03T05:37:38.091 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:38.091 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:38.092 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:37:38.092 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:38.093 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:37:38.093 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 0, 2022-09-03T05:37:38.093 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 0 2022-09-03T05:37:38.094 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:38.094 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:37:38Z" 2022-09-03T05:37:38.095 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:37:38.097 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 14474342186618979890 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002627", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:03.799590Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000001499", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:36:03.583371Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000006434", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:34:57.577753Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:37:38Z" } 2022-09-03T05:37:38.099 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002627', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:03.799590Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000001499', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:36:03.583371Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006434', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:34:57.577753Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000'}}] 2022-09-03T05:37:38.100 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000001499', 2: '00000000000000000000:00000000000000006434', 3: ''} 2022-09-03T05:37:38.101 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000001499', 2: '00000000000000000000:00000000000000006434', 3: ''} 2022-09-03T05:37:38.102 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone3 source_zone=test-zone2 2022-09-03T05:37:38.103 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone2 target=test-zone4 2022-09-03T05:37:38.104 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone3 target=test-zone1 2022-09-03T05:37:38.105 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:38.105 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:38.106 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:37:38.156 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:37:38.157 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:37:38.215 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:37:38.216 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:38.216 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:37:38.217 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:31:06.854659Z" 2022-09-03T05:37:38.218 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:38.219 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:38.220 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000001499", 2022-09-03T05:37:38.220 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:36:03.665960Z" 2022-09-03T05:37:38.221 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:38.222 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:38.222 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000005588", 2022-09-03T05:37:38.222 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:34:35.658751Z" 2022-09-03T05:37:38.223 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:38.224 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:38.224 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:37:38.225 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:37:38.225 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:38.226 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:37:38.228 DEBUG:rgw_multi.tests:data markers for zone=test-zone3 markers={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000001499', 2: '00000000000000000000:00000000000000005588', 3: ''} 2022-09-03T05:37:38.229 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone1 source_zone=test-zone3 2022-09-03T05:37:38.230 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:38.231 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:38.231 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone3 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:37:38.288 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:37:38.289 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:37:38.340 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:37:38.341 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:37:38.341 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:37:38.341 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:37:38.341 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:37:38.342 INFO:teuthology.orchestra.run.smithi181.stdout: "instance_id": 4175403477120006881 2022-09-03T05:37:38.342 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:38.342 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:37:38.343 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:38.343 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:37:38.343 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:37:38.343 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:37:38.344 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:37:38.344 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:37:38.344 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:37:38.344 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:37:38.345 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:31:06.854659Z" 2022-09-03T05:37:38.346 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:38.346 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:38.347 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:38.347 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:37:38.348 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:37:38.349 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:37:38.349 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000001499", 2022-09-03T05:37:38.350 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:37:38.351 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:37:38.351 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:37:38.352 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:36:03.665960Z" 2022-09-03T05:37:38.353 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:38.353 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:38.354 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:38.354 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:37:38.355 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:37:38.355 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:37:38.356 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000005588", 2022-09-03T05:37:38.356 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:37:38.357 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:37:38.357 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:37:38.358 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:34:35.658751Z" 2022-09-03T05:37:38.358 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:38.359 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:38.359 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:38.359 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:37:38.359 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:37:38.360 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:37:38.360 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:37:38.362 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:37:38.363 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:37:38.363 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:37:38.364 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000" 2022-09-03T05:37:38.364 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:38.365 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:38.365 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:37:38.366 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:38.367 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:37:38.367 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 0, 2022-09-03T05:37:38.368 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 0 2022-09-03T05:37:38.368 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:38.369 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:37:38Z" 2022-09-03T05:37:38.374 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:37:38.377 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 4175403477120006881 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002627", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:06.854659Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000001499", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:36:03.665960Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000005588", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:34:35.658751Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:37:38Z" } 2022-09-03T05:37:38.378 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002627', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:06.854659Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000001499', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:36:03.665960Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000005588', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:34:35.658751Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000'}}] 2022-09-03T05:37:38.379 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000001499', 2: '00000000000000000000:00000000000000005588', 3: ''} 2022-09-03T05:37:38.380 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000001499', 2: '00000000000000000000:00000000000000005588', 3: ''} 2022-09-03T05:37:38.381 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone1 source_zone=test-zone3 2022-09-03T05:37:38.383 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone3 target=test-zone2 2022-09-03T05:37:38.384 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:38.385 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:38.386 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:37:38.437 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:37:38.441 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:37:38.501 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:37:38.501 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:38.502 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:37:38.502 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:31:06.854659Z" 2022-09-03T05:37:38.502 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:38.503 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:38.503 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000001499", 2022-09-03T05:37:38.503 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:36:03.665960Z" 2022-09-03T05:37:38.503 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:38.505 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:38.505 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000005588", 2022-09-03T05:37:38.506 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:34:35.658751Z" 2022-09-03T05:37:38.506 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:38.507 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:38.507 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:37:38.507 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:37:38.507 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:38.508 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:37:38.509 DEBUG:rgw_multi.tests:data markers for zone=test-zone3 markers={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000001499', 2: '00000000000000000000:00000000000000005588', 3: ''} 2022-09-03T05:37:38.510 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone2 source_zone=test-zone3 2022-09-03T05:37:38.511 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:38.511 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:38.512 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone3 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:37:38.569 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:37:38.570 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:37:38.618 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:37:38.619 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:37:38.620 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:37:38.620 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:37:38.621 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:37:38.621 INFO:teuthology.orchestra.run.smithi189.stdout: "instance_id": 1043408315071002039 2022-09-03T05:37:38.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:38.622 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:37:38.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:38.623 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:37:38.624 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:38.624 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:37:38.625 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:37:38.625 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:38.626 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:37:38.627 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:38.627 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:31:06.854659Z" 2022-09-03T05:37:38.628 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:38.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:38.629 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:38.630 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:37:38.631 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:38.631 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:37:38.632 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000001499", 2022-09-03T05:37:38.632 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:38.633 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:37:38.633 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:38.634 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:36:03.665960Z" 2022-09-03T05:37:38.634 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:38.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:38.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:38.636 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:37:38.636 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:38.636 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:37:38.636 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000005588", 2022-09-03T05:37:38.637 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:38.637 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:37:38.637 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:38.638 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:34:35.658751Z" 2022-09-03T05:37:38.638 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:38.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:38.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:38.639 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:37:38.640 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:38.640 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:37:38.640 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:37:38.640 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:38.641 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:37:38.641 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:38.642 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000" 2022-09-03T05:37:38.642 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:38.642 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:38.643 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:37:38.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:38.644 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:37:38.644 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 0, 2022-09-03T05:37:38.650 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 0 2022-09-03T05:37:38.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:38.651 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:37:38Z" 2022-09-03T05:37:38.651 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:37:38.652 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 1043408315071002039 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002627", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:06.854659Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000001499", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:36:03.665960Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000005588", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:34:35.658751Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:37:38Z" } 2022-09-03T05:37:38.653 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002627', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:06.854659Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000001499', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:36:03.665960Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000005588', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:34:35.658751Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000'}}] 2022-09-03T05:37:38.654 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000001499', 2: '00000000000000000000:00000000000000005588', 3: ''} 2022-09-03T05:37:38.654 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000001499', 2: '00000000000000000000:00000000000000005588', 3: ''} 2022-09-03T05:37:38.655 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone2 source_zone=test-zone3 2022-09-03T05:37:38.655 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone3 target=test-zone4 2022-09-03T05:37:38.656 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_concurrent_versioned_object_incremental_sync ... ok 2022-09-03T05:37:38.660 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:37:38.661 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:37:38.661 DEBUG:boto:Method: PUT 2022-09-03T05:37:38.662 DEBUG:boto:Path: /dsgvcy-20/ 2022-09-03T05:37:38.662 DEBUG:boto:Data: 2022-09-03T05:37:38.663 DEBUG:boto:Headers: {} 2022-09-03T05:37:38.663 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:38.664 DEBUG:boto:Port: 8000 2022-09-03T05:37:38.664 DEBUG:boto:Params: {} 2022-09-03T05:37:38.665 DEBUG:boto:Token: None 2022-09-03T05:37:38.666 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:37:38 GMT /dsgvcy-20/ 2022-09-03T05:37:38.666 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O+0nl/j799tNNJNjiv5R7WBWcwQ= 2022-09-03T05:37:38.667 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O+0nl/j799tNNJNjiv5R7WBWcwQ=', 'Content-Length': '0'} 2022-09-03T05:37:38.697 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000d141f92abf966e90-006312e822-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:37:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:38.698 DEBUG:rgw_multi.tests:created bucket=dsgvcy-20 2022-09-03T05:37:38.699 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:38.699 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:38.700 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:37:38.752 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:37:38.753 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:37:38.786 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=41ecd2a5-afc0-4eae-9349-7dfe39baa1cb 2022-09-03T05:37:38.802 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:37:38.802 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:38.803 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:37:38.804 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:18:18.906852Z" 2022-09-03T05:37:38.804 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:38.804 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:38.805 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:37:38.805 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:37:38.805 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:38.806 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:38.807 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662183458.694130_166.1", 2022-09-03T05:37:38.808 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:37:38.694130Z" 2022-09-03T05:37:38.808 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:38.808 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:38.809 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:37:38.810 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:20:30.928498Z" 2022-09-03T05:37:38.810 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:38.811 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:37:38.811 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183458.694130_166.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:37:38.812 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:37:38.813 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:38.814 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:38.815 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:37:38.870 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:37:38.870 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:37:38.900 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:37:38.898+0000 7efdb2473600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:37:38.901 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:37:38.899+0000 7efdb2473600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:37:38.916 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:37:38.917 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:37:38.918 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:37:38.918 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:37:38.919 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:37:38.921 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:37:38.922 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:37:38.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:38.922 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:37:38.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:38.923 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:37:38.924 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:38.924 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:37:38.924 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:37:38.925 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:38.928 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:37:38.928 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:38.929 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:37:38.929 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:37:38.929 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:38.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:38.930 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:38.930 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:37:38.930 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:38.931 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:37:38.931 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:37:38.931 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:38.931 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:37:38.932 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:38.932 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:37:38.932 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:37:38.932 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:38.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:38.933 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:38.934 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:37:38.934 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:38.934 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:37:38.934 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183458.677214_163.1", 2022-09-03T05:37:38.935 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:38.935 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:37:38.935 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:38.936 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:37:38.677214Z", 2022-09-03T05:37:38.936 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:37:38.936 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:38.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:38.937 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:38.937 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:37:38.937 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:38.937 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:37:38.938 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:37:38.938 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:38.938 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:37:38.940 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:38.940 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:37:38.941 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:37:38.941 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:38.941 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:38.941 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:37:38.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:38.942 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:37:38.942 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:37:38.943 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:37:38.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:38.943 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:37:38Z" 2022-09-03T05:37:38.943 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:37:38.944 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183458.677214_163.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:37:38.677214Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:37:38Z" } 2022-09-03T05:37:38.944 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183458.677214_163.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:37:38.677214Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:37:38.948 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183458.694130_166.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:37:38.949 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183458.677214_163.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:37:38.951 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662183458.694130_166.1 target=1_1662183458.677214_163.1 2022-09-03T05:37:43.954 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:43.956 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:43.957 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:37:44.015 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:37:44.016 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:37:44.058 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:37:44.058+0000 7f0e489b3600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:37:44.059 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:37:44.059+0000 7f0e489b3600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:37:44.077 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:37:44.078 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:37:44.079 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:37:44.079 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:37:44.079 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:37:44.080 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:37:44.080 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:37:44.081 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:44.082 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:37:44.082 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:44.083 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:37:44.083 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:44.084 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:37:44.084 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:37:44.085 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:44.085 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:37:44.085 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:44.085 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:37:44.086 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:37:44.086 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:44.086 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:44.086 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:44.087 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:37:44.087 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:44.087 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:37:44.088 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:37:44.088 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:44.088 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:37:44.088 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:44.089 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:37:44.089 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:37:44.089 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:44.089 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:44.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:44.090 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:37:44.090 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:44.091 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:37:44.091 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183458.694130_166.1", 2022-09-03T05:37:44.091 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:44.092 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:37:44.092 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:44.093 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:37:38.694130Z", 2022-09-03T05:37:44.093 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:37:44.093 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:44.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:44.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:44.094 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:37:44.095 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:44.095 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:37:44.095 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:37:44.096 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:44.096 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:37:44.096 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:44.096 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:37:44.097 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:37:44.097 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:44.097 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:44.097 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:37:44.097 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:44.098 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:37:44.098 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:37:44.098 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:37:44.099 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:44.099 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:37:44Z" 2022-09-03T05:37:44.099 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:37:44.099 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183458.694130_166.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:37:38.694130Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:37:44Z" } 2022-09-03T05:37:44.100 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183458.694130_166.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:37:38.694130Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:37:44.101 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183458.694130_166.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:37:44.101 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183458.694130_166.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:37:44.102 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:37:44.104 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:37:44.105 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:44.106 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:44.107 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:37:44.165 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:37:44.165 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:37:44.214 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:37:44.215+0000 7f498385d600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:37:44.216 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:37:44.216+0000 7f498385d600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:37:44.233 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:37:44.234 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:37:44.234 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:37:44.235 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:37:44.235 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:37:44.235 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:37:44.235 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:37:44.236 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:44.236 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:37:44.236 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:44.236 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:37:44.237 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:37:44.237 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:37:44.237 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:37:44.237 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:37:44.238 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:37:44.238 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:37:44.238 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:37:44.239 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:37:44.239 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:44.239 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:44.239 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:44.240 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:37:44.240 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:37:44.240 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:37:44.240 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:37:44.241 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:37:44.241 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:37:44.241 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:37:44.241 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:37:44.242 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:37:44.243 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:44.243 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:44.243 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:44.244 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:37:44.244 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:37:44.244 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:37:44.244 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662183458.694130_166.1", 2022-09-03T05:37:44.245 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:37:44.245 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:37:44.245 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:37:44.245 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:37:38.694130Z", 2022-09-03T05:37:44.246 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:37:44.246 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:44.246 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:44.246 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:37:44.247 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:37:44.247 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:37:44.247 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:37:44.247 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:37:44.248 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:37:44.248 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:37:44.248 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:37:44.248 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:37:44.249 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:37:44.249 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:44.249 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:37:44.249 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:37:44.250 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:44.250 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:37:44.250 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:37:44.250 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:37:44.251 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:37:44.251 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:37:44Z" 2022-09-03T05:37:44.251 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:37:44.252 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183458.694130_166.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:37:38.694130Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:37:44Z" } 2022-09-03T05:37:44.252 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183458.694130_166.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:37:38.694130Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:37:44.254 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183458.694130_166.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:37:44.254 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183458.694130_166.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:37:44.255 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:37:44.255 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:37:44.256 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:44.256 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:37:44.257 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:37:44.312 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:37:44.313 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:37:44.345 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:37:44.345+0000 7f50d9513600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:37:44.346 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:37:44.346+0000 7f50d9513600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:37:44.359 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:37:44.359 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:37:44.360 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:37:44.360 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:37:44.360 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:37:44.360 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:37:44.361 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:37:44.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:44.361 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:37:44.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:44.362 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:37:44.362 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:44.363 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:37:44.363 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:37:44.363 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:44.363 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:37:44.364 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:44.364 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:37:44.364 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:37:44.364 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:44.365 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:44.365 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:44.366 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:37:44.366 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:44.366 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:37:44.366 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:37:44.367 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:44.367 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:37:44.367 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:44.368 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:37:44.368 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:37:44.368 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:44.369 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:44.369 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:44.369 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:37:44.369 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:44.370 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:37:44.370 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183458.694130_166.1", 2022-09-03T05:37:44.370 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:44.371 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:37:44.371 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:44.371 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:37:38.694130Z", 2022-09-03T05:37:44.371 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:37:44.372 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:44.372 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:44.372 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:37:44.372 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:37:44.373 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:37:44.373 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:37:44.373 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:37:44.374 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:37:44.374 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:37:44.374 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:37:44.374 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:37:44.375 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:37:44.375 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:44.375 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:37:44.376 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:37:44.376 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:44.376 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:37:44.376 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:37:44.377 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:37:44.377 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:37:44.377 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:37:44Z" 2022-09-03T05:37:44.378 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:37:44.379 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183458.694130_166.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:37:38.694130Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:37:44Z" } 2022-09-03T05:37:44.380 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183458.694130_166.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:37:38.694130Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:37:44.380 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183458.694130_166.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:37:44.381 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183458.694130_166.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:37:44.381 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:37:44.382 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:37:44.382 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:37:44.383 DEBUG:boto:Method: HEAD 2022-09-03T05:37:44.384 DEBUG:boto:Path: /dsgvcy-20/ 2022-09-03T05:37:44.384 DEBUG:boto:Data: 2022-09-03T05:37:44.384 DEBUG:boto:Headers: {} 2022-09-03T05:37:44.385 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:44.386 DEBUG:boto:Port: 8000 2022-09-03T05:37:44.386 DEBUG:boto:Params: {} 2022-09-03T05:37:44.387 DEBUG:boto:Token: None 2022-09-03T05:37:44.387 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:37:44 GMT /dsgvcy-20/ 2022-09-03T05:37:44.388 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nNYaOQQdkMHAlcc36z+aPr6yaZU= 2022-09-03T05:37:44.389 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:37:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nNYaOQQdkMHAlcc36z+aPr6yaZU=', 'Content-Length': '0'} 2022-09-03T05:37:44.392 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000056440b8cc195dcd7-006312e828-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:44.393 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:37:44.394 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:37:44.394 DEBUG:boto:Method: PUT 2022-09-03T05:37:44.395 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:37:44.395 DEBUG:boto:Data: 2022-09-03T05:37:44.396 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue'} 2022-09-03T05:37:44.397 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:37:44.397 DEBUG:boto:Port: 8000 2022-09-03T05:37:44.398 DEBUG:boto:Params: {} 2022-09-03T05:37:44.398 DEBUG:boto:Token: None 2022-09-03T05:37:44.399 DEBUG:boto:StringToSign: PUT 1B2M2Y8AsgTpgAmY7PhCfg== application/octet-stream Sat, 03 Sep 2022 05:37:44 GMT /dsgvcy-20/obj 2022-09-03T05:37:44.399 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WFzMi7oXYfeXELluFVFQCsJjm9A= 2022-09-03T05:37:44.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:37:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WFzMi7oXYfeXELluFVFQCsJjm9A='} 2022-09-03T05:37:44.408 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d6b9b2bb1826c7cb-006312e828-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:37:44.409 DEBUG:rgw_multi.tests:created initial version id=None 2022-09-03T05:37:44.411 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-20 2022-09-03T05:37:44.413 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:37:44.414 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:37:44.415 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-20 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:37:44.471 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:37:44.471 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:37:44.503 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:37:44.503+0000 7fa06328a600 1 waiting to reach incremental sync.. 2022-09-03T05:37:49.503 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:37:49.504+0000 7fa06328a600 1 waiting to reach incremental sync.. 2022-09-03T05:37:54.505 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:37:54.504+0000 7fa06328a600 1 waiting to reach incremental sync.. 2022-09-03T05:37:59.505 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:37:59.505+0000 7fa06328a600 1 waiting to reach incremental sync.. 2022-09-03T05:38:04.506 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:38:04.506+0000 7fa06328a600 1 waiting to reach incremental sync.. 2022-09-03T05:38:09.507 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:38:09.506+0000 7fa06328a600 1 waiting to reach incremental sync.. 2022-09-03T05:38:14.507 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:38:14.507+0000 7fa06328a600 1 waiting to reach incremental sync.. 2022-09-03T05:38:19.508 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:38:19.508+0000 7fa06328a600 1 waiting to reach incremental sync.. 2022-09-03T05:38:24.509 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:38:24.509+0000 7fa06328a600 1 waiting to reach incremental sync.. 2022-09-03T05:38:29.509 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:38:29.509+0000 7fa06328a600 1 waiting to reach incremental sync.. 2022-09-03T05:38:34.510 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:38:34.509+0000 7fa06328a600 1 waiting to reach incremental sync.. 2022-09-03T05:38:39.511 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:38:39.511+0000 7fa06328a600 1 waiting to reach incremental sync.. 2022-09-03T05:38:44.511 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:38:44.511+0000 7fa06328a600 1 waiting to reach incremental sync.. 2022-09-03T05:38:49.513 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:38:49.513+0000 7fa06328a600 1 bucket sync caught up with source: 2022-09-03T05:38:49.514 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.6.5, , ] 2022-09-03T05:38:49.514 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000001.6.5, , ] 2022-09-03T05:38:49.515 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:38:49.513+0000 7fa06328a600 0 bucket checkpoint complete 2022-09-03T05:38:49.530 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-20 2022-09-03T05:38:49.531 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:38:49.531 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:38:49.532 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-20 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:38:49.597 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:38:49.598 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:38:49.654 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:38:49.653+0000 7f673a0dc600 1 bucket sync caught up with source: 2022-09-03T05:38:49.654 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000001.6.5, , ] 2022-09-03T05:38:49.655 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000001.6.5, , ] 2022-09-03T05:38:49.655 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:38:49.653+0000 7f673a0dc600 0 bucket checkpoint complete 2022-09-03T05:38:49.674 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-20 2022-09-03T05:38:49.674 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:38:49.675 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:38:49.676 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-20 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:38:49.739 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:38:49.740 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:38:49.780 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:38:49.780+0000 7f1bf4ac7600 1 bucket sync caught up with source: 2022-09-03T05:38:49.781 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.6.5, , ] 2022-09-03T05:38:49.782 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000001.6.5, , ] 2022-09-03T05:38:49.782 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:38:49.780+0000 7f1bf4ac7600 0 bucket checkpoint complete 2022-09-03T05:38:49.801 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-20 2022-09-03T05:38:49.802 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:38:49.803 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:38:49.804 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-20 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:38:49.863 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:38:49.863 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:38:49.902 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:38:49.901+0000 7f095cbfc600 1 bucket sync caught up with source: 2022-09-03T05:38:49.902 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000001.21.5, , ] 2022-09-03T05:38:49.903 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000001.21.5, , ] 2022-09-03T05:38:49.903 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:38:49.901+0000 7f095cbfc600 0 bucket checkpoint complete 2022-09-03T05:38:49.913 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-20 2022-09-03T05:38:49.914 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:38:49.915 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:38:49.915 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-20 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:38:49.972 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:38:49.973 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:38:50.018 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:38:50.010+0000 7f3310bd7600 1 bucket sync caught up with source: 2022-09-03T05:38:50.018 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000001.21.5, , ] 2022-09-03T05:38:50.019 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000001.21.5, , ] 2022-09-03T05:38:50.019 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:38:50.010+0000 7f3310bd7600 0 bucket checkpoint complete 2022-09-03T05:38:50.034 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-20 2022-09-03T05:38:50.034 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-20 2022-09-03T05:38:50.035 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:38:50.035 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:38:50.036 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-20 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:38:50.093 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:38:50.094 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:38:50.142 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:38:50.142+0000 7f78070af600 1 bucket sync caught up with source: 2022-09-03T05:38:50.143 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000001.19.5, , ] 2022-09-03T05:38:50.143 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000001.19.5, , ] 2022-09-03T05:38:50.144 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:38:50.142+0000 7f78070af600 0 bucket checkpoint complete 2022-09-03T05:38:50.161 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-20 2022-09-03T05:38:50.162 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:38:50.162 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:38:50.163 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-20 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:38:50.222 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:38:50.223 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:38:50.263 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:38:50.263+0000 7f608688e600 1 bucket sync caught up with source: 2022-09-03T05:38:50.263 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.19.5, , ] 2022-09-03T05:38:50.263 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000001.19.5, , ] 2022-09-03T05:38:50.264 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:38:50.263+0000 7f608688e600 0 bucket checkpoint complete 2022-09-03T05:38:50.279 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-20 2022-09-03T05:38:50.280 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-20 zones={test-zone2, test-zone1} 2022-09-03T05:38:50.281 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:50.281 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:50.282 DEBUG:boto:Method: HEAD 2022-09-03T05:38:50.283 DEBUG:boto:Path: /dsgvcy-20/ 2022-09-03T05:38:50.284 DEBUG:boto:Data: 2022-09-03T05:38:50.284 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.285 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:38:50.285 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.287 DEBUG:boto:Params: {} 2022-09-03T05:38:50.288 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:38:50.289 DEBUG:boto:Token: None 2022-09-03T05:38:50.290 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/ 2022-09-03T05:38:50.294 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8oYmlXUnMKpZJ+IDyjZaqe8KQMk= 2022-09-03T05:38:50.295 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8oYmlXUnMKpZJ+IDyjZaqe8KQMk=', 'Content-Length': '0'} 2022-09-03T05:38:50.299 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a2a77d68b9996dbc-006312e86a-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.300 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:50.300 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:50.301 DEBUG:boto:Method: HEAD 2022-09-03T05:38:50.301 DEBUG:boto:Path: /dsgvcy-20/ 2022-09-03T05:38:50.302 DEBUG:boto:Data: 2022-09-03T05:38:50.304 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.304 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:38:50.306 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.306 DEBUG:boto:Params: {} 2022-09-03T05:38:50.307 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:38:50.309 DEBUG:boto:Token: None 2022-09-03T05:38:50.310 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/ 2022-09-03T05:38:50.310 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8oYmlXUnMKpZJ+IDyjZaqe8KQMk= 2022-09-03T05:38:50.311 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8oYmlXUnMKpZJ+IDyjZaqe8KQMk=', 'Content-Length': '0'} 2022-09-03T05:38:50.315 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f8a47ca48b58063a-006312e86a-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.316 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:38:50.316 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:50.317 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:50.318 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.318 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.319 DEBUG:boto:Method: GET 2022-09-03T05:38:50.319 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.320 DEBUG:boto:Data: 2022-09-03T05:38:50.320 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.321 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:38:50.321 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.322 DEBUG:boto:Params: {} 2022-09-03T05:38:50.322 DEBUG:boto:Token: None 2022-09-03T05:38:50.323 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/?versions 2022-09-03T05:38:50.324 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D0iIEVXg/T33fkpnB2f2xaT0Kns= 2022-09-03T05:38:50.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D0iIEVXg/T33fkpnB2f2xaT0Kns=', 'Content-Length': '0'} 2022-09-03T05:38:50.327 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000057af41b08747af0e-006312e86a-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.328 DEBUG:boto:b'dsgvcy-20999falseobjnulltrue2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:50.329 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:38:50.330 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:38:50.332 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:50.333 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:50.333 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.334 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.335 DEBUG:boto:Method: GET 2022-09-03T05:38:50.335 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.336 DEBUG:boto:Data: 2022-09-03T05:38:50.336 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.337 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:38:50.337 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.338 DEBUG:boto:Params: {} 2022-09-03T05:38:50.338 DEBUG:boto:Token: None 2022-09-03T05:38:50.339 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/?versions 2022-09-03T05:38:50.339 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D0iIEVXg/T33fkpnB2f2xaT0Kns= 2022-09-03T05:38:50.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D0iIEVXg/T33fkpnB2f2xaT0Kns=', 'Content-Length': '0'} 2022-09-03T05:38:50.343 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007cd3aed367a88beb-006312e86a-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.344 DEBUG:boto:b'dsgvcy-20999falseobjnulltrue2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:50.344 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:38:50.345 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:50.345 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:50.346 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.347 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.347 DEBUG:boto:Method: GET 2022-09-03T05:38:50.348 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.348 DEBUG:boto:Data: 2022-09-03T05:38:50.349 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.349 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:38:50.350 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.350 DEBUG:boto:Params: {} 2022-09-03T05:38:50.351 DEBUG:boto:Token: None 2022-09-03T05:38:50.352 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/?versions 2022-09-03T05:38:50.352 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D0iIEVXg/T33fkpnB2f2xaT0Kns= 2022-09-03T05:38:50.353 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D0iIEVXg/T33fkpnB2f2xaT0Kns=', 'Content-Length': '0'} 2022-09-03T05:38:50.356 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000045a854eec61fe67c-006312e86a-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.357 DEBUG:boto:b'dsgvcy-20999falseobjnulltrue2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:50.359 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:50.359 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:50.360 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.360 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.361 DEBUG:boto:Method: GET 2022-09-03T05:38:50.361 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.362 DEBUG:boto:Data: 2022-09-03T05:38:50.363 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.364 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:38:50.364 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.365 DEBUG:boto:Params: {} 2022-09-03T05:38:50.365 DEBUG:boto:Token: None 2022-09-03T05:38:50.366 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/?versions 2022-09-03T05:38:50.367 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D0iIEVXg/T33fkpnB2f2xaT0Kns= 2022-09-03T05:38:50.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D0iIEVXg/T33fkpnB2f2xaT0Kns=', 'Content-Length': '0'} 2022-09-03T05:38:50.370 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e22d0dac6f08b1d9-006312e86a-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.371 DEBUG:boto:b'dsgvcy-20999falseobjnulltrue2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:50.373 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:38:50.374 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:50.374 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:50.375 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.376 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.377 DEBUG:boto:Method: GET 2022-09-03T05:38:50.378 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.379 DEBUG:boto:Data: 2022-09-03T05:38:50.381 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.381 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:38:50.382 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.383 DEBUG:boto:Params: {} 2022-09-03T05:38:50.384 DEBUG:boto:Token: None 2022-09-03T05:38:50.385 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.386 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c1lzM4UO/lAx/bGqA+t9mhjQyMI= 2022-09-03T05:38:50.387 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c1lzM4UO/lAx/bGqA+t9mhjQyMI=', 'Content-Length': '0'} 2022-09-03T05:38:50.390 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000faa31fba6c676ea2-006312e86a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.392 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:50.392 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:50.393 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.394 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.394 DEBUG:boto:Method: GET 2022-09-03T05:38:50.395 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.396 DEBUG:boto:Data: 2022-09-03T05:38:50.396 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.397 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:38:50.397 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.398 DEBUG:boto:Params: {} 2022-09-03T05:38:50.399 DEBUG:boto:Token: None 2022-09-03T05:38:50.399 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.400 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c1lzM4UO/lAx/bGqA+t9mhjQyMI= 2022-09-03T05:38:50.401 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c1lzM4UO/lAx/bGqA+t9mhjQyMI=', 'Content-Length': '0'} 2022-09-03T05:38:50.404 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000073f8827eb09e74e9-006312e86a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.406 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:50.407 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:50.407 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.408 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.409 DEBUG:boto:Method: HEAD 2022-09-03T05:38:50.409 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.410 DEBUG:boto:Data: 2022-09-03T05:38:50.411 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.411 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:38:50.413 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.414 DEBUG:boto:Params: {} 2022-09-03T05:38:50.415 DEBUG:boto:Token: None 2022-09-03T05:38:50.416 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.417 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nAHYAklw8I7xug2b/+vDkueXKSQ= 2022-09-03T05:38:50.418 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nAHYAklw8I7xug2b/+vDkueXKSQ=', 'Content-Length': '0'} 2022-09-03T05:38:50.420 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f0ee8015b07fcffb-006312e86a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.421 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:50.422 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:50.423 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.424 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.425 DEBUG:boto:Method: HEAD 2022-09-03T05:38:50.425 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.426 DEBUG:boto:Data: 2022-09-03T05:38:50.427 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.428 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:38:50.429 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.430 DEBUG:boto:Params: {} 2022-09-03T05:38:50.430 DEBUG:boto:Token: None 2022-09-03T05:38:50.431 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.432 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nAHYAklw8I7xug2b/+vDkueXKSQ= 2022-09-03T05:38:50.433 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nAHYAklw8I7xug2b/+vDkueXKSQ=', 'Content-Length': '0'} 2022-09-03T05:38:50.434 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000099896793b1bb15d0-006312e86a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.435 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:38:50.436 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:50.436 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:50.437 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.438 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.438 DEBUG:boto:Method: GET 2022-09-03T05:38:50.439 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.439 DEBUG:boto:Data: 2022-09-03T05:38:50.440 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.441 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:38:50.441 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.442 DEBUG:boto:Params: {} 2022-09-03T05:38:50.443 DEBUG:boto:Token: None 2022-09-03T05:38:50.443 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.444 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c1lzM4UO/lAx/bGqA+t9mhjQyMI= 2022-09-03T05:38:50.445 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c1lzM4UO/lAx/bGqA+t9mhjQyMI=', 'Content-Length': '0'} 2022-09-03T05:38:50.447 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a7e51539421706e1-006312e86a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.448 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:50.448 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:50.449 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.449 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.450 DEBUG:boto:Method: GET 2022-09-03T05:38:50.450 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.451 DEBUG:boto:Data: 2022-09-03T05:38:50.451 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.452 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:38:50.453 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.453 DEBUG:boto:Params: {} 2022-09-03T05:38:50.454 DEBUG:boto:Token: None 2022-09-03T05:38:50.454 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.455 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c1lzM4UO/lAx/bGqA+t9mhjQyMI= 2022-09-03T05:38:50.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c1lzM4UO/lAx/bGqA+t9mhjQyMI=', 'Content-Length': '0'} 2022-09-03T05:38:50.458 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000008dd808e68d02e32a-006312e86a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.459 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:50.459 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:50.460 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.460 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.461 DEBUG:boto:Method: GET 2022-09-03T05:38:50.461 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.462 DEBUG:boto:Data: 2022-09-03T05:38:50.462 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.463 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:38:50.464 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.464 DEBUG:boto:Params: {} 2022-09-03T05:38:50.465 DEBUG:boto:Token: None 2022-09-03T05:38:50.466 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/ 2022-09-03T05:38:50.466 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X7aqvIlE77WMGEqlaXSvZ0uyiYg= 2022-09-03T05:38:50.467 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X7aqvIlE77WMGEqlaXSvZ0uyiYg=', 'Content-Length': '0'} 2022-09-03T05:38:50.470 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c980e82b36147c9b-006312e86a-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.471 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:50.472 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:50.472 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:50.473 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.474 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.475 DEBUG:boto:Method: GET 2022-09-03T05:38:50.475 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.476 DEBUG:boto:Data: 2022-09-03T05:38:50.476 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.477 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:38:50.477 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.478 DEBUG:boto:Params: {} 2022-09-03T05:38:50.478 DEBUG:boto:Token: None 2022-09-03T05:38:50.479 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/ 2022-09-03T05:38:50.480 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X7aqvIlE77WMGEqlaXSvZ0uyiYg= 2022-09-03T05:38:50.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X7aqvIlE77WMGEqlaXSvZ0uyiYg=', 'Content-Length': '0'} 2022-09-03T05:38:50.483 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000774a690e8f468c4b-006312e86a-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.526 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:50.528 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:50.529 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:50.530 DEBUG:boto:Method: HEAD 2022-09-03T05:38:50.531 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:38:50.531 DEBUG:boto:Data: 2022-09-03T05:38:50.532 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.533 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:38:50.534 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.535 DEBUG:boto:Params: {} 2022-09-03T05:38:50.537 DEBUG:boto:Token: None 2022-09-03T05:38:50.537 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/obj 2022-09-03T05:38:50.538 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p5Qo5kD6ejbINIi/5xX1ZIH28wY= 2022-09-03T05:38:50.539 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p5Qo5kD6ejbINIi/5xX1ZIH28wY=', 'Content-Length': '0'} 2022-09-03T05:38:50.541 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000005d9ef521676466cf-006312e86a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.542 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:50.543 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:50.543 DEBUG:boto:Method: HEAD 2022-09-03T05:38:50.544 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:38:50.545 DEBUG:boto:Data: 2022-09-03T05:38:50.546 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.547 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:38:50.547 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.548 DEBUG:boto:Params: {} 2022-09-03T05:38:50.549 DEBUG:boto:Token: None 2022-09-03T05:38:50.549 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/obj 2022-09-03T05:38:50.550 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p5Qo5kD6ejbINIi/5xX1ZIH28wY= 2022-09-03T05:38:50.551 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p5Qo5kD6ejbINIi/5xX1ZIH28wY=', 'Content-Length': '0'} 2022-09-03T05:38:50.552 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000274bb55a19b0259e-006312e86a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.554 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:38:50.555 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:50.556 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:50.557 DEBUG:boto:Method: GET 2022-09-03T05:38:50.558 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:38:50.559 DEBUG:boto:Data: 2022-09-03T05:38:50.560 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.561 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:38:50.561 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.562 DEBUG:boto:Params: {} 2022-09-03T05:38:50.563 DEBUG:boto:Token: None 2022-09-03T05:38:50.564 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/obj 2022-09-03T05:38:50.564 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yq1ltO+x2y6PRCLmPD4xg8V2gc4= 2022-09-03T05:38:50.565 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yq1ltO+x2y6PRCLmPD4xg8V2gc4=', 'Content-Length': '0'} 2022-09-03T05:38:50.568 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f429bdcb7033cbe8-006312e86a-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.569 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:50.570 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:50.570 DEBUG:boto:Method: GET 2022-09-03T05:38:50.571 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:38:50.571 DEBUG:boto:Data: 2022-09-03T05:38:50.572 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.573 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:38:50.573 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.574 DEBUG:boto:Params: {} 2022-09-03T05:38:50.574 DEBUG:boto:Token: None 2022-09-03T05:38:50.575 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/obj 2022-09-03T05:38:50.576 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yq1ltO+x2y6PRCLmPD4xg8V2gc4= 2022-09-03T05:38:50.577 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yq1ltO+x2y6PRCLmPD4xg8V2gc4=', 'Content-Length': '0'} 2022-09-03T05:38:50.579 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000009bdf0109ac00d651-006312e86a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.581 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:50.581 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:50.582 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.583 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.583 DEBUG:boto:Method: GET 2022-09-03T05:38:50.584 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.584 DEBUG:boto:Data: 2022-09-03T05:38:50.585 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.586 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:38:50.586 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.587 DEBUG:boto:Params: {} 2022-09-03T05:38:50.588 DEBUG:boto:Token: None 2022-09-03T05:38:50.589 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/ 2022-09-03T05:38:50.590 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X7aqvIlE77WMGEqlaXSvZ0uyiYg= 2022-09-03T05:38:50.591 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X7aqvIlE77WMGEqlaXSvZ0uyiYg=', 'Content-Length': '0'} 2022-09-03T05:38:50.593 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007889418e143a60ed-006312e86a-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.594 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:50.595 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:50.596 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:50.596 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.597 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.598 DEBUG:boto:Method: GET 2022-09-03T05:38:50.599 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.599 DEBUG:boto:Data: 2022-09-03T05:38:50.600 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.601 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:38:50.601 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.602 DEBUG:boto:Params: {} 2022-09-03T05:38:50.603 DEBUG:boto:Token: None 2022-09-03T05:38:50.604 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/ 2022-09-03T05:38:50.605 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X7aqvIlE77WMGEqlaXSvZ0uyiYg= 2022-09-03T05:38:50.606 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X7aqvIlE77WMGEqlaXSvZ0uyiYg=', 'Content-Length': '0'} 2022-09-03T05:38:50.609 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000abe998a6baa61127-006312e86a-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.610 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:50.611 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-20 zones={test-zone2, test-zone1} 2022-09-03T05:38:50.612 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-20 zones={test-zone3, test-zone1} 2022-09-03T05:38:50.612 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:50.613 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:50.613 DEBUG:boto:Method: HEAD 2022-09-03T05:38:50.614 DEBUG:boto:Path: /dsgvcy-20/ 2022-09-03T05:38:50.615 DEBUG:boto:Data: 2022-09-03T05:38:50.615 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.616 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:38:50.616 DEBUG:boto:Port: 8001 2022-09-03T05:38:50.617 DEBUG:boto:Params: {} 2022-09-03T05:38:50.618 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T05:38:50.619 DEBUG:boto:Token: None 2022-09-03T05:38:50.619 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/ 2022-09-03T05:38:50.620 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8oYmlXUnMKpZJ+IDyjZaqe8KQMk= 2022-09-03T05:38:50.621 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8oYmlXUnMKpZJ+IDyjZaqe8KQMk=', 'Content-Length': '0'} 2022-09-03T05:38:50.624 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001eea77796aea16fb-006312e86a-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.625 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:50.626 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:50.626 DEBUG:boto:Method: HEAD 2022-09-03T05:38:50.627 DEBUG:boto:Path: /dsgvcy-20/ 2022-09-03T05:38:50.628 DEBUG:boto:Data: 2022-09-03T05:38:50.628 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.629 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:38:50.629 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.630 DEBUG:boto:Params: {} 2022-09-03T05:38:50.631 DEBUG:boto:Token: None 2022-09-03T05:38:50.631 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/ 2022-09-03T05:38:50.632 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8oYmlXUnMKpZJ+IDyjZaqe8KQMk= 2022-09-03T05:38:50.633 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8oYmlXUnMKpZJ+IDyjZaqe8KQMk=', 'Content-Length': '0'} 2022-09-03T05:38:50.635 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ecf689f427df81ba-006312e86a-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.636 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:38:50.637 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:50.638 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:50.639 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.640 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.641 DEBUG:boto:Method: GET 2022-09-03T05:38:50.642 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.643 DEBUG:boto:Data: 2022-09-03T05:38:50.644 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.645 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:38:50.646 DEBUG:boto:Port: 8001 2022-09-03T05:38:50.647 DEBUG:boto:Params: {} 2022-09-03T05:38:50.648 DEBUG:boto:Token: None 2022-09-03T05:38:50.649 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/?versions 2022-09-03T05:38:50.651 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D0iIEVXg/T33fkpnB2f2xaT0Kns= 2022-09-03T05:38:50.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D0iIEVXg/T33fkpnB2f2xaT0Kns=', 'Content-Length': '0'} 2022-09-03T05:38:50.654 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000073b4982f5c160b28-006312e86a-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.655 DEBUG:boto:b'dsgvcy-20999falseobjnulltrue2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:50.655 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:38:50.656 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:38:50.657 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:50.657 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:50.658 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.658 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.659 DEBUG:boto:Method: GET 2022-09-03T05:38:50.659 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.660 DEBUG:boto:Data: 2022-09-03T05:38:50.660 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.661 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:38:50.661 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.662 DEBUG:boto:Params: {} 2022-09-03T05:38:50.662 DEBUG:boto:Token: None 2022-09-03T05:38:50.663 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/?versions 2022-09-03T05:38:50.664 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D0iIEVXg/T33fkpnB2f2xaT0Kns= 2022-09-03T05:38:50.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D0iIEVXg/T33fkpnB2f2xaT0Kns=', 'Content-Length': '0'} 2022-09-03T05:38:50.667 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fc5556789beebef1-006312e86a-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.667 DEBUG:boto:b'dsgvcy-20999falseobjnulltrue2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:50.668 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:38:50.669 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:50.669 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:50.670 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.671 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.671 DEBUG:boto:Method: GET 2022-09-03T05:38:50.672 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.672 DEBUG:boto:Data: 2022-09-03T05:38:50.673 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.673 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:38:50.674 DEBUG:boto:Port: 8001 2022-09-03T05:38:50.674 DEBUG:boto:Params: {} 2022-09-03T05:38:50.675 DEBUG:boto:Token: None 2022-09-03T05:38:50.676 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/?versions 2022-09-03T05:38:50.676 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D0iIEVXg/T33fkpnB2f2xaT0Kns= 2022-09-03T05:38:50.677 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D0iIEVXg/T33fkpnB2f2xaT0Kns=', 'Content-Length': '0'} 2022-09-03T05:38:50.679 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a2457b4b762fb850-006312e86a-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.680 DEBUG:boto:b'dsgvcy-20999falseobjnulltrue2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:50.681 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:50.681 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:50.682 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.682 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.683 DEBUG:boto:Method: GET 2022-09-03T05:38:50.683 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:50.684 DEBUG:boto:Data: 2022-09-03T05:38:50.684 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.685 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:38:50.686 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.686 DEBUG:boto:Params: {} 2022-09-03T05:38:50.687 DEBUG:boto:Token: None 2022-09-03T05:38:50.687 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/?versions 2022-09-03T05:38:50.688 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D0iIEVXg/T33fkpnB2f2xaT0Kns= 2022-09-03T05:38:50.689 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D0iIEVXg/T33fkpnB2f2xaT0Kns=', 'Content-Length': '0'} 2022-09-03T05:38:50.691 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d38e25e6e49209f-006312e86a-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.734 DEBUG:boto:b'dsgvcy-20999falseobjnulltrue2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:50.736 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:38:50.737 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:50.738 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:50.739 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.740 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.741 DEBUG:boto:Method: GET 2022-09-03T05:38:50.742 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.744 DEBUG:boto:Data: 2022-09-03T05:38:50.745 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.746 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:38:50.747 DEBUG:boto:Port: 8001 2022-09-03T05:38:50.748 DEBUG:boto:Params: {} 2022-09-03T05:38:50.749 DEBUG:boto:Token: None 2022-09-03T05:38:50.750 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.751 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c1lzM4UO/lAx/bGqA+t9mhjQyMI= 2022-09-03T05:38:50.751 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c1lzM4UO/lAx/bGqA+t9mhjQyMI=', 'Content-Length': '0'} 2022-09-03T05:38:50.753 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000002a82549cfafa81e-006312e86a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.754 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:50.755 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:50.756 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.756 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.757 DEBUG:boto:Method: GET 2022-09-03T05:38:50.758 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.758 DEBUG:boto:Data: 2022-09-03T05:38:50.759 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.759 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:38:50.760 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.760 DEBUG:boto:Params: {} 2022-09-03T05:38:50.761 DEBUG:boto:Token: None 2022-09-03T05:38:50.762 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.763 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c1lzM4UO/lAx/bGqA+t9mhjQyMI= 2022-09-03T05:38:50.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c1lzM4UO/lAx/bGqA+t9mhjQyMI=', 'Content-Length': '0'} 2022-09-03T05:38:50.766 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f832ec254d934410-006312e86a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.768 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:50.769 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:50.769 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.770 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.771 DEBUG:boto:Method: HEAD 2022-09-03T05:38:50.771 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.773 DEBUG:boto:Data: 2022-09-03T05:38:50.774 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.775 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:38:50.776 DEBUG:boto:Port: 8001 2022-09-03T05:38:50.777 DEBUG:boto:Params: {} 2022-09-03T05:38:50.778 DEBUG:boto:Token: None 2022-09-03T05:38:50.779 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.780 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nAHYAklw8I7xug2b/+vDkueXKSQ= 2022-09-03T05:38:50.781 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nAHYAklw8I7xug2b/+vDkueXKSQ=', 'Content-Length': '0'} 2022-09-03T05:38:50.783 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000e68682fc77ecd115-006312e86a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.784 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:50.785 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:50.786 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.787 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.788 DEBUG:boto:Method: HEAD 2022-09-03T05:38:50.789 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.790 DEBUG:boto:Data: 2022-09-03T05:38:50.791 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.792 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:38:50.793 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.793 DEBUG:boto:Params: {} 2022-09-03T05:38:50.794 DEBUG:boto:Token: None 2022-09-03T05:38:50.795 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.796 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nAHYAklw8I7xug2b/+vDkueXKSQ= 2022-09-03T05:38:50.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nAHYAklw8I7xug2b/+vDkueXKSQ=', 'Content-Length': '0'} 2022-09-03T05:38:50.799 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000964ee0894fb5a9f7-006312e86a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.800 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:38:50.801 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:50.801 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:50.802 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.802 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.803 DEBUG:boto:Method: GET 2022-09-03T05:38:50.804 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.804 DEBUG:boto:Data: 2022-09-03T05:38:50.805 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.805 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:38:50.806 DEBUG:boto:Port: 8001 2022-09-03T05:38:50.807 DEBUG:boto:Params: {} 2022-09-03T05:38:50.807 DEBUG:boto:Token: None 2022-09-03T05:38:50.808 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.809 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c1lzM4UO/lAx/bGqA+t9mhjQyMI= 2022-09-03T05:38:50.809 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c1lzM4UO/lAx/bGqA+t9mhjQyMI=', 'Content-Length': '0'} 2022-09-03T05:38:50.811 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000066e891f341eb3b0a-006312e86a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.812 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:50.813 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:50.813 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.814 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.814 DEBUG:boto:Method: GET 2022-09-03T05:38:50.815 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.815 DEBUG:boto:Data: 2022-09-03T05:38:50.816 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.817 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:38:50.817 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.818 DEBUG:boto:Params: {} 2022-09-03T05:38:50.820 DEBUG:boto:Token: None 2022-09-03T05:38:50.820 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:50.821 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c1lzM4UO/lAx/bGqA+t9mhjQyMI= 2022-09-03T05:38:50.822 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c1lzM4UO/lAx/bGqA+t9mhjQyMI=', 'Content-Length': '0'} 2022-09-03T05:38:50.824 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000003321d64a18708722-006312e86a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.825 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:50.825 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:50.826 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.828 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.829 DEBUG:boto:Method: GET 2022-09-03T05:38:50.830 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.831 DEBUG:boto:Data: 2022-09-03T05:38:50.831 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.832 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:38:50.832 DEBUG:boto:Port: 8001 2022-09-03T05:38:50.833 DEBUG:boto:Params: {} 2022-09-03T05:38:50.833 DEBUG:boto:Token: None 2022-09-03T05:38:50.834 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/ 2022-09-03T05:38:50.835 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X7aqvIlE77WMGEqlaXSvZ0uyiYg= 2022-09-03T05:38:50.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X7aqvIlE77WMGEqlaXSvZ0uyiYg=', 'Content-Length': '0'} 2022-09-03T05:38:50.840 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000021b4f44f5753b1b4-006312e86a-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.841 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:50.842 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:50.842 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:50.843 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.843 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.844 DEBUG:boto:Method: GET 2022-09-03T05:38:50.845 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.845 DEBUG:boto:Data: 2022-09-03T05:38:50.846 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.846 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:38:50.847 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.847 DEBUG:boto:Params: {} 2022-09-03T05:38:50.848 DEBUG:boto:Token: None 2022-09-03T05:38:50.849 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/ 2022-09-03T05:38:50.849 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X7aqvIlE77WMGEqlaXSvZ0uyiYg= 2022-09-03T05:38:50.850 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X7aqvIlE77WMGEqlaXSvZ0uyiYg=', 'Content-Length': '0'} 2022-09-03T05:38:50.853 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006f8c412b83e5d3bc-006312e86a-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.854 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:50.855 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:50.855 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:50.856 DEBUG:boto:Method: HEAD 2022-09-03T05:38:50.856 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:38:50.857 DEBUG:boto:Data: 2022-09-03T05:38:50.857 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.858 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:38:50.858 DEBUG:boto:Port: 8001 2022-09-03T05:38:50.859 DEBUG:boto:Params: {} 2022-09-03T05:38:50.860 DEBUG:boto:Token: None 2022-09-03T05:38:50.860 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/obj 2022-09-03T05:38:50.861 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p5Qo5kD6ejbINIi/5xX1ZIH28wY= 2022-09-03T05:38:50.862 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p5Qo5kD6ejbINIi/5xX1ZIH28wY=', 'Content-Length': '0'} 2022-09-03T05:38:50.864 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000492731cfb3d2e61c-006312e86a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.865 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:50.865 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:50.866 DEBUG:boto:Method: HEAD 2022-09-03T05:38:50.866 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:38:50.867 DEBUG:boto:Data: 2022-09-03T05:38:50.867 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.868 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:38:50.868 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.869 DEBUG:boto:Params: {} 2022-09-03T05:38:50.869 DEBUG:boto:Token: None 2022-09-03T05:38:50.870 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/obj 2022-09-03T05:38:50.871 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p5Qo5kD6ejbINIi/5xX1ZIH28wY= 2022-09-03T05:38:50.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p5Qo5kD6ejbINIi/5xX1ZIH28wY=', 'Content-Length': '0'} 2022-09-03T05:38:50.874 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000056c0e17a47243c7e-006312e86a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.875 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:38:50.876 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:50.876 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:50.877 DEBUG:boto:Method: GET 2022-09-03T05:38:50.877 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:38:50.878 DEBUG:boto:Data: 2022-09-03T05:38:50.879 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.879 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:38:50.880 DEBUG:boto:Port: 8001 2022-09-03T05:38:50.880 DEBUG:boto:Params: {} 2022-09-03T05:38:50.881 DEBUG:boto:Token: None 2022-09-03T05:38:50.881 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/obj 2022-09-03T05:38:50.882 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yq1ltO+x2y6PRCLmPD4xg8V2gc4= 2022-09-03T05:38:50.883 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yq1ltO+x2y6PRCLmPD4xg8V2gc4=', 'Content-Length': '0'} 2022-09-03T05:38:50.885 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000059c5a92918b44387-006312e86a-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.886 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:50.887 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:50.887 DEBUG:boto:Method: GET 2022-09-03T05:38:50.888 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:38:50.889 DEBUG:boto:Data: 2022-09-03T05:38:50.889 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.890 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:38:50.891 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.891 DEBUG:boto:Params: {} 2022-09-03T05:38:50.892 DEBUG:boto:Token: None 2022-09-03T05:38:50.893 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/obj 2022-09-03T05:38:50.893 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yq1ltO+x2y6PRCLmPD4xg8V2gc4= 2022-09-03T05:38:50.894 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yq1ltO+x2y6PRCLmPD4xg8V2gc4=', 'Content-Length': '0'} 2022-09-03T05:38:50.896 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000000b820987de77a6c4-006312e86a-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.897 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:50.897 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:50.898 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.898 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.899 DEBUG:boto:Method: GET 2022-09-03T05:38:50.900 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.900 DEBUG:boto:Data: 2022-09-03T05:38:50.901 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.902 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:38:50.903 DEBUG:boto:Port: 8001 2022-09-03T05:38:50.903 DEBUG:boto:Params: {} 2022-09-03T05:38:50.904 DEBUG:boto:Token: None 2022-09-03T05:38:50.905 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/ 2022-09-03T05:38:50.905 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X7aqvIlE77WMGEqlaXSvZ0uyiYg= 2022-09-03T05:38:50.906 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X7aqvIlE77WMGEqlaXSvZ0uyiYg=', 'Content-Length': '0'} 2022-09-03T05:38:50.909 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008fb5945df56b5c1f-006312e86a-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.910 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:50.911 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:50.911 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:50.912 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.913 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.913 DEBUG:boto:Method: GET 2022-09-03T05:38:50.914 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:38:50.915 DEBUG:boto:Data: 2022-09-03T05:38:50.915 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.916 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:38:50.916 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.917 DEBUG:boto:Params: {} 2022-09-03T05:38:50.918 DEBUG:boto:Token: None 2022-09-03T05:38:50.919 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/ 2022-09-03T05:38:50.919 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X7aqvIlE77WMGEqlaXSvZ0uyiYg= 2022-09-03T05:38:50.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X7aqvIlE77WMGEqlaXSvZ0uyiYg=', 'Content-Length': '0'} 2022-09-03T05:38:50.923 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000098fb7eb8ea8db4f0-006312e86a-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.970 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:50.972 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-20 zones={test-zone3, test-zone1} 2022-09-03T05:38:50.973 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-20 zones={test-zone3, test-zone2} 2022-09-03T05:38:50.974 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:50.975 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:50.976 DEBUG:boto:Method: HEAD 2022-09-03T05:38:50.976 DEBUG:boto:Path: /dsgvcy-20/ 2022-09-03T05:38:50.977 DEBUG:boto:Data: 2022-09-03T05:38:50.978 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.979 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:38:50.980 DEBUG:boto:Port: 8001 2022-09-03T05:38:50.980 DEBUG:boto:Params: {} 2022-09-03T05:38:50.981 DEBUG:boto:Token: None 2022-09-03T05:38:50.982 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/ 2022-09-03T05:38:50.982 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8oYmlXUnMKpZJ+IDyjZaqe8KQMk= 2022-09-03T05:38:50.983 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8oYmlXUnMKpZJ+IDyjZaqe8KQMk=', 'Content-Length': '0'} 2022-09-03T05:38:50.985 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007f32f2a1d6e4d38b-006312e86a-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:38:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:50.986 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:50.987 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:50.988 DEBUG:boto:Method: HEAD 2022-09-03T05:38:50.989 DEBUG:boto:Path: /dsgvcy-20/ 2022-09-03T05:38:50.990 DEBUG:boto:Data: 2022-09-03T05:38:50.991 DEBUG:boto:Headers: {} 2022-09-03T05:38:50.992 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:38:50.993 DEBUG:boto:Port: 8000 2022-09-03T05:38:50.994 DEBUG:boto:Params: {} 2022-09-03T05:38:50.995 DEBUG:boto:Token: None 2022-09-03T05:38:50.996 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:38:50 GMT /dsgvcy-20/ 2022-09-03T05:38:50.997 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8oYmlXUnMKpZJ+IDyjZaqe8KQMk= 2022-09-03T05:38:50.998 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8oYmlXUnMKpZJ+IDyjZaqe8KQMk=', 'Content-Length': '0'} 2022-09-03T05:38:51.001 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000050bc8ffad43068f7-006312e86b-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:38:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:51.002 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:38:51.003 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:51.003 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:51.004 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:51.004 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:51.005 DEBUG:boto:Method: GET 2022-09-03T05:38:51.005 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:51.006 DEBUG:boto:Data: 2022-09-03T05:38:51.006 DEBUG:boto:Headers: {} 2022-09-03T05:38:51.007 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:38:51.007 DEBUG:boto:Port: 8001 2022-09-03T05:38:51.008 DEBUG:boto:Params: {} 2022-09-03T05:38:51.008 DEBUG:boto:Token: None 2022-09-03T05:38:51.009 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:51 GMT /dsgvcy-20/?versions 2022-09-03T05:38:51.010 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ENq5IBBEDK1I1NQZJBK3bezaMM0= 2022-09-03T05:38:51.010 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ENq5IBBEDK1I1NQZJBK3bezaMM0=', 'Content-Length': '0'} 2022-09-03T05:38:51.013 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004f941538c7aba5b3-006312e86b-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:51.013 DEBUG:boto:b'dsgvcy-20999falseobjnulltrue2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:51.014 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:38:51.015 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:38:51.015 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:51.016 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:51.016 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:51.017 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:51.017 DEBUG:boto:Method: GET 2022-09-03T05:38:51.018 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:51.018 DEBUG:boto:Data: 2022-09-03T05:38:51.019 DEBUG:boto:Headers: {} 2022-09-03T05:38:51.019 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:38:51.020 DEBUG:boto:Port: 8000 2022-09-03T05:38:51.020 DEBUG:boto:Params: {} 2022-09-03T05:38:51.021 DEBUG:boto:Token: None 2022-09-03T05:38:51.022 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:51 GMT /dsgvcy-20/?versions 2022-09-03T05:38:51.022 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ENq5IBBEDK1I1NQZJBK3bezaMM0= 2022-09-03T05:38:51.023 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ENq5IBBEDK1I1NQZJBK3bezaMM0=', 'Content-Length': '0'} 2022-09-03T05:38:51.026 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008026a73952c0d077-006312e86b-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:51.027 DEBUG:boto:b'dsgvcy-20999falseobjnulltrue2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:51.028 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:38:51.029 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:51.030 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:51.031 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:51.032 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:51.033 DEBUG:boto:Method: GET 2022-09-03T05:38:51.035 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:51.036 DEBUG:boto:Data: 2022-09-03T05:38:51.037 DEBUG:boto:Headers: {} 2022-09-03T05:38:51.038 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:38:51.039 DEBUG:boto:Port: 8001 2022-09-03T05:38:51.040 DEBUG:boto:Params: {} 2022-09-03T05:38:51.041 DEBUG:boto:Token: None 2022-09-03T05:38:51.041 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:51 GMT /dsgvcy-20/?versions 2022-09-03T05:38:51.042 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ENq5IBBEDK1I1NQZJBK3bezaMM0= 2022-09-03T05:38:51.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ENq5IBBEDK1I1NQZJBK3bezaMM0=', 'Content-Length': '0'} 2022-09-03T05:38:51.045 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c2ae18b25cf991a1-006312e86b-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:51.046 DEBUG:boto:b'dsgvcy-20999falseobjnulltrue2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:51.047 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:51.047 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:51.048 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:51.048 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:51.049 DEBUG:boto:Method: GET 2022-09-03T05:38:51.049 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:38:51.050 DEBUG:boto:Data: 2022-09-03T05:38:51.051 DEBUG:boto:Headers: {} 2022-09-03T05:38:51.051 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:38:51.052 DEBUG:boto:Port: 8000 2022-09-03T05:38:51.052 DEBUG:boto:Params: {} 2022-09-03T05:38:51.053 DEBUG:boto:Token: None 2022-09-03T05:38:51.054 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:51 GMT /dsgvcy-20/?versions 2022-09-03T05:38:51.055 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ENq5IBBEDK1I1NQZJBK3bezaMM0= 2022-09-03T05:38:51.055 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ENq5IBBEDK1I1NQZJBK3bezaMM0=', 'Content-Length': '0'} 2022-09-03T05:38:51.058 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000310e28e8ed07c3f7-006312e86b-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:51.059 DEBUG:boto:b'dsgvcy-20999falseobjnulltrue2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:51.060 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:38:51.061 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:51.061 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:51.062 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.063 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.063 DEBUG:boto:Method: GET 2022-09-03T05:38:51.064 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.065 DEBUG:boto:Data: 2022-09-03T05:38:51.066 DEBUG:boto:Headers: {} 2022-09-03T05:38:51.067 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:38:51.068 DEBUG:boto:Port: 8001 2022-09-03T05:38:51.069 DEBUG:boto:Params: {} 2022-09-03T05:38:51.070 DEBUG:boto:Token: None 2022-09-03T05:38:51.071 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:51 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.072 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jwkahQD5FFx7CwPk8bVMzkMEJcM= 2022-09-03T05:38:51.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jwkahQD5FFx7CwPk8bVMzkMEJcM=', 'Content-Length': '0'} 2022-09-03T05:38:51.076 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000500e26dd61fbed3d-006312e86b-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:51.077 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:51.078 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:51.079 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.080 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.081 DEBUG:boto:Method: GET 2022-09-03T05:38:51.082 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.082 DEBUG:boto:Data: 2022-09-03T05:38:51.083 DEBUG:boto:Headers: {} 2022-09-03T05:38:51.084 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:38:51.085 DEBUG:boto:Port: 8000 2022-09-03T05:38:51.086 DEBUG:boto:Params: {} 2022-09-03T05:38:51.087 DEBUG:boto:Token: None 2022-09-03T05:38:51.088 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:51 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.088 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jwkahQD5FFx7CwPk8bVMzkMEJcM= 2022-09-03T05:38:51.090 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jwkahQD5FFx7CwPk8bVMzkMEJcM=', 'Content-Length': '0'} 2022-09-03T05:38:51.097 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000c04ea48bd4f24891-006312e86b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:51.098 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:51.099 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:51.100 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.100 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.101 DEBUG:boto:Method: HEAD 2022-09-03T05:38:51.101 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.101 DEBUG:boto:Data: 2022-09-03T05:38:51.102 DEBUG:boto:Headers: {} 2022-09-03T05:38:51.103 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:38:51.103 DEBUG:boto:Port: 8001 2022-09-03T05:38:51.103 DEBUG:boto:Params: {} 2022-09-03T05:38:51.104 DEBUG:boto:Token: None 2022-09-03T05:38:51.105 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:38:51 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.106 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QgENEiiOqZm4VFaYQI64/zh5/mI= 2022-09-03T05:38:51.107 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QgENEiiOqZm4VFaYQI64/zh5/mI=', 'Content-Length': '0'} 2022-09-03T05:38:51.109 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000794e8d8d43261813-006312e86b-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:51.111 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:51.112 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:51.113 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.114 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.115 DEBUG:boto:Method: HEAD 2022-09-03T05:38:51.116 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.116 DEBUG:boto:Data: 2022-09-03T05:38:51.117 DEBUG:boto:Headers: {} 2022-09-03T05:38:51.118 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:38:51.119 DEBUG:boto:Port: 8000 2022-09-03T05:38:51.120 DEBUG:boto:Params: {} 2022-09-03T05:38:51.121 DEBUG:boto:Token: None 2022-09-03T05:38:51.122 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:38:51 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.123 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QgENEiiOqZm4VFaYQI64/zh5/mI= 2022-09-03T05:38:51.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QgENEiiOqZm4VFaYQI64/zh5/mI=', 'Content-Length': '0'} 2022-09-03T05:38:51.127 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000eef0209f721ed3d4-006312e86b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:51.128 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:38:51.129 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:51.130 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:51.131 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.132 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.133 DEBUG:boto:Method: GET 2022-09-03T05:38:51.134 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.135 DEBUG:boto:Data: 2022-09-03T05:38:51.136 DEBUG:boto:Headers: {} 2022-09-03T05:38:51.137 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:38:51.138 DEBUG:boto:Port: 8001 2022-09-03T05:38:51.139 DEBUG:boto:Params: {} 2022-09-03T05:38:51.140 DEBUG:boto:Token: None 2022-09-03T05:38:51.140 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:51 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.141 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jwkahQD5FFx7CwPk8bVMzkMEJcM= 2022-09-03T05:38:51.142 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jwkahQD5FFx7CwPk8bVMzkMEJcM=', 'Content-Length': '0'} 2022-09-03T05:38:51.144 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000008c371cd93bb3bd17-006312e86b-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:51.145 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:51.146 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:51.148 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.149 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.149 DEBUG:boto:Method: GET 2022-09-03T05:38:51.150 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.151 DEBUG:boto:Data: 2022-09-03T05:38:51.152 DEBUG:boto:Headers: {} 2022-09-03T05:38:51.153 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:38:51.153 DEBUG:boto:Port: 8000 2022-09-03T05:38:51.154 DEBUG:boto:Params: {} 2022-09-03T05:38:51.155 DEBUG:boto:Token: None 2022-09-03T05:38:51.155 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:51 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:38:51.156 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jwkahQD5FFx7CwPk8bVMzkMEJcM= 2022-09-03T05:38:51.157 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jwkahQD5FFx7CwPk8bVMzkMEJcM=', 'Content-Length': '0'} 2022-09-03T05:38:51.159 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000527e3a42f04e7b94-006312e86b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:51.160 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:51.160 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:51.161 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:51.162 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:51.162 DEBUG:boto:Method: GET 2022-09-03T05:38:51.163 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:38:51.163 DEBUG:boto:Data: 2022-09-03T05:38:51.164 DEBUG:boto:Headers: {} 2022-09-03T05:38:51.164 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:38:51.165 DEBUG:boto:Port: 8001 2022-09-03T05:38:51.165 DEBUG:boto:Params: {} 2022-09-03T05:38:51.166 DEBUG:boto:Token: None 2022-09-03T05:38:51.167 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:51 GMT /dsgvcy-20/ 2022-09-03T05:38:51.167 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GhKsRnhxMQrxLcxklQh2tMGarwY= 2022-09-03T05:38:51.168 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GhKsRnhxMQrxLcxklQh2tMGarwY=', 'Content-Length': '0'} 2022-09-03T05:38:51.170 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007c6370bd5771571c-006312e86b-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:51.173 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:51.174 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:51.175 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:51.176 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:51.177 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:51.178 DEBUG:boto:Method: GET 2022-09-03T05:38:51.180 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:38:51.180 DEBUG:boto:Data: 2022-09-03T05:38:51.181 DEBUG:boto:Headers: {} 2022-09-03T05:38:51.181 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:38:51.182 DEBUG:boto:Port: 8000 2022-09-03T05:38:51.183 DEBUG:boto:Params: {} 2022-09-03T05:38:51.183 DEBUG:boto:Token: None 2022-09-03T05:38:51.184 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:51 GMT /dsgvcy-20/ 2022-09-03T05:38:51.185 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GhKsRnhxMQrxLcxklQh2tMGarwY= 2022-09-03T05:38:51.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GhKsRnhxMQrxLcxklQh2tMGarwY=', 'Content-Length': '0'} 2022-09-03T05:38:51.188 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003bc75d384c559c72-006312e86b-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:51.230 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:51.231 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:51.232 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:51.232 DEBUG:boto:Method: HEAD 2022-09-03T05:38:51.233 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:38:51.233 DEBUG:boto:Data: 2022-09-03T05:38:51.234 DEBUG:boto:Headers: {} 2022-09-03T05:38:51.235 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:38:51.235 DEBUG:boto:Port: 8001 2022-09-03T05:38:51.236 DEBUG:boto:Params: {} 2022-09-03T05:38:51.237 DEBUG:boto:Token: None 2022-09-03T05:38:51.237 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:38:51 GMT /dsgvcy-20/obj 2022-09-03T05:38:51.238 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r84GUrrkin0pdplNpAyIqysmMp4= 2022-09-03T05:38:51.239 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r84GUrrkin0pdplNpAyIqysmMp4=', 'Content-Length': '0'} 2022-09-03T05:38:51.241 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000009dda1c0c3d68ee9-006312e86b-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:51.241 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:51.242 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:51.242 DEBUG:boto:Method: HEAD 2022-09-03T05:38:51.243 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:38:51.243 DEBUG:boto:Data: 2022-09-03T05:38:51.244 DEBUG:boto:Headers: {} 2022-09-03T05:38:51.244 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:38:51.245 DEBUG:boto:Port: 8000 2022-09-03T05:38:51.245 DEBUG:boto:Params: {} 2022-09-03T05:38:51.246 DEBUG:boto:Token: None 2022-09-03T05:38:51.247 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:38:51 GMT /dsgvcy-20/obj 2022-09-03T05:38:51.247 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r84GUrrkin0pdplNpAyIqysmMp4= 2022-09-03T05:38:51.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r84GUrrkin0pdplNpAyIqysmMp4=', 'Content-Length': '0'} 2022-09-03T05:38:51.250 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000b69ddd1711c77627-006312e86b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:51.251 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:38:51.252 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:51.253 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:51.253 DEBUG:boto:Method: GET 2022-09-03T05:38:51.254 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:38:51.254 DEBUG:boto:Data: 2022-09-03T05:38:51.255 DEBUG:boto:Headers: {} 2022-09-03T05:38:51.255 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:38:51.256 DEBUG:boto:Port: 8001 2022-09-03T05:38:51.256 DEBUG:boto:Params: {} 2022-09-03T05:38:51.257 DEBUG:boto:Token: None 2022-09-03T05:38:51.258 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:51 GMT /dsgvcy-20/obj 2022-09-03T05:38:51.259 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UIfAsBjX59IlIa1VCoQf0TPoG8w= 2022-09-03T05:38:51.259 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UIfAsBjX59IlIa1VCoQf0TPoG8w=', 'Content-Length': '0'} 2022-09-03T05:38:51.261 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000940bfd4dee1885f2-006312e86b-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:51.262 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:51.262 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:51.263 DEBUG:boto:Method: GET 2022-09-03T05:38:51.264 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:38:51.264 DEBUG:boto:Data: 2022-09-03T05:38:51.265 DEBUG:boto:Headers: {} 2022-09-03T05:38:51.265 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:38:51.266 DEBUG:boto:Port: 8000 2022-09-03T05:38:51.266 DEBUG:boto:Params: {} 2022-09-03T05:38:51.267 DEBUG:boto:Token: None 2022-09-03T05:38:51.267 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:51 GMT /dsgvcy-20/obj 2022-09-03T05:38:51.268 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UIfAsBjX59IlIa1VCoQf0TPoG8w= 2022-09-03T05:38:51.269 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UIfAsBjX59IlIa1VCoQf0TPoG8w=', 'Content-Length': '0'} 2022-09-03T05:38:51.271 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000d48b2408a3ac2e75-006312e86b-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:38:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:51.271 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:51.272 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:51.273 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:51.273 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:51.274 DEBUG:boto:Method: GET 2022-09-03T05:38:51.275 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:38:51.275 DEBUG:boto:Data: 2022-09-03T05:38:51.276 DEBUG:boto:Headers: {} 2022-09-03T05:38:51.276 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:38:51.277 DEBUG:boto:Port: 8001 2022-09-03T05:38:51.278 DEBUG:boto:Params: {} 2022-09-03T05:38:51.279 DEBUG:boto:Token: None 2022-09-03T05:38:51.280 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:51 GMT /dsgvcy-20/ 2022-09-03T05:38:51.281 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GhKsRnhxMQrxLcxklQh2tMGarwY= 2022-09-03T05:38:51.282 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GhKsRnhxMQrxLcxklQh2tMGarwY=', 'Content-Length': '0'} 2022-09-03T05:38:51.284 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d2f130366579f47-006312e86b-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:51.285 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:51.286 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:51.286 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:51.287 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:51.287 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:38:51.288 DEBUG:boto:Method: GET 2022-09-03T05:38:51.288 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:38:51.289 DEBUG:boto:Data: 2022-09-03T05:38:51.290 DEBUG:boto:Headers: {} 2022-09-03T05:38:51.290 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:38:51.291 DEBUG:boto:Port: 8000 2022-09-03T05:38:51.291 DEBUG:boto:Params: {} 2022-09-03T05:38:51.292 DEBUG:boto:Token: None 2022-09-03T05:38:51.292 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:38:51 GMT /dsgvcy-20/ 2022-09-03T05:38:51.293 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GhKsRnhxMQrxLcxklQh2tMGarwY= 2022-09-03T05:38:51.294 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GhKsRnhxMQrxLcxklQh2tMGarwY=', 'Content-Length': '0'} 2022-09-03T05:38:51.296 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f60c1795cf9461e0-006312e86b-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:38:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:51.338 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:38:51.339 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-20 zones={test-zone3, test-zone2} 2022-09-03T05:38:51.340 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:51.340 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:51.341 DEBUG:boto:path=/dsgvcy-20/?versioning 2022-09-03T05:38:51.342 DEBUG:boto:auth_path=/dsgvcy-20/?versioning 2022-09-03T05:38:51.343 DEBUG:boto:Method: PUT 2022-09-03T05:38:51.343 DEBUG:boto:Path: /dsgvcy-20/?versioning 2022-09-03T05:38:51.344 DEBUG:boto:Data: Enabled Disabled 2022-09-03T05:38:51.344 DEBUG:boto:Headers: {} 2022-09-03T05:38:51.345 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:38:51.345 DEBUG:boto:Port: 8000 2022-09-03T05:38:51.346 DEBUG:boto:Params: {} 2022-09-03T05:38:51.346 DEBUG:boto:Token: None 2022-09-03T05:38:51.348 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:38:51 GMT /dsgvcy-20/?versioning 2022-09-03T05:38:51.348 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gryJMj4G399eJaFtqu37usDdzig= 2022-09-03T05:38:51.349 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gryJMj4G399eJaFtqu37usDdzig=', 'Content-Length': '228'} 2022-09-03T05:38:51.356 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000014609d33545f21f0-006312e86b-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:38:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:51.357 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:38:51.358 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:38:51.359 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:38:51.412 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:38:51.413 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:38:51.454 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=41ecd2a5-afc0-4eae-9349-7dfe39baa1cb 2022-09-03T05:38:51.478 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:38:51.479 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:38:51.479 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:38:51.480 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:18:18.906852Z" 2022-09-03T05:38:51.480 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:38:51.481 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:38:51.481 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:38:51.482 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:38:51.482 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:38:51.483 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:38:51.483 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662183531.355467_184.1", 2022-09-03T05:38:51.483 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:38:51.355467Z" 2022-09-03T05:38:51.484 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:38:51.484 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:38:51.485 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:38:51.485 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:20:30.928498Z" 2022-09-03T05:38:51.485 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:38:51.486 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:38:51.488 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183531.355467_184.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:38:51.488 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:38:51.489 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:38:51.490 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:38:51.490 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:38:51.549 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:38:51.550 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:38:51.603 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:38:51.603+0000 7fb96e6d4600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:38:51.604 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:38:51.604+0000 7fb96e6d4600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:38:51.630 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:38:51.631 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:38:51.631 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:38:51.631 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:38:51.632 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:38:51.632 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:38:51.633 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:38:51.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:38:51.633 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:38:51.634 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:38:51.634 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:38:51.634 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:38:51.634 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:38:51.635 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:38:51.635 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:38:51.635 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:38:51.636 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:38:51.636 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:38:51.636 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:38:51.636 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:38:51.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:38:51.637 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:38:51.637 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:38:51.637 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:38:51.638 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:38:51.638 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:38:51.638 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:38:51.638 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:38:51.639 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:38:51.639 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:38:51.639 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:38:51.640 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:38:51.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:38:51.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:38:51.641 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:38:51.641 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:38:51.641 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:38:51.641 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183531.352224_183.1", 2022-09-03T05:38:51.642 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:38:51.642 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:38:51.643 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:38:51.643 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:38:51.352224Z", 2022-09-03T05:38:51.643 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:38:51.643 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:38:51.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:38:51.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:38:51.644 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:38:51.645 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:38:51.645 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:38:51.645 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:38:51.645 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:38:51.646 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:38:51.646 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:38:51.646 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:38:51.646 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:38:51.647 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:38:51.647 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:38:51.647 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:38:51.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:38:51.648 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:38:51.648 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:38:51.648 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:38:51.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:38:51.649 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:38:51Z" 2022-09-03T05:38:51.649 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:38:51.651 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183531.352224_183.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:38:51.352224Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:38:51Z" } 2022-09-03T05:38:51.651 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183531.352224_183.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:38:51.352224Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:38:51.653 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183531.355467_184.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:38:51.653 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183531.352224_183.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:38:51.654 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662183531.355467_184.1 target=1_1662183531.352224_183.1 2022-09-03T05:38:56.656 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:38:56.657 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:38:56.659 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:38:56.719 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:38:56.720 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:38:56.750 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:38:56.749+0000 7fa5f5ea1600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:38:56.751 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:38:56.751+0000 7fa5f5ea1600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:38:56.765 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:38:56.765 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:38:56.766 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:38:56.766 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:38:56.766 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:38:56.766 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:38:56.768 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:38:56.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:38:56.769 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:38:56.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:38:56.770 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:38:56.770 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:38:56.771 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:38:56.775 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:38:56.775 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:38:56.776 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:38:56.776 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:38:56.777 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:38:56.778 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:38:56.778 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:38:56.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:38:56.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:38:56.780 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:38:56.780 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:38:56.780 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:38:56.781 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:38:56.781 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:38:56.782 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:38:56.782 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:38:56.783 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:38:56.783 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:38:56.784 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:38:56.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:38:56.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:38:56.785 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:38:56.786 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:38:56.786 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:38:56.787 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183531.355467_184.1", 2022-09-03T05:38:56.787 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:38:56.788 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:38:56.788 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:38:56.788 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:38:51.355467Z", 2022-09-03T05:38:56.789 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:38:56.789 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:38:56.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:38:56.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:38:56.790 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:38:56.790 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:38:56.790 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:38:56.791 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:38:56.791 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:38:56.791 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:38:56.791 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:38:56.792 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:38:56.792 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:38:56.793 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:38:56.793 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:38:56.794 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:38:56.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:38:56.795 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:38:56.796 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:38:56.796 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:38:56.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:38:56.797 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:38:56Z" 2022-09-03T05:38:56.798 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:38:56.798 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183531.355467_184.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:38:51.355467Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:38:56Z" } 2022-09-03T05:38:56.799 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183531.355467_184.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:38:51.355467Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:38:56.800 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183531.355467_184.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:38:56.801 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183531.355467_184.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:38:56.802 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:38:56.803 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:38:56.804 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:38:56.805 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:38:56.806 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:38:56.867 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:38:56.868 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:38:56.897 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:38:56.896+0000 7f245d575600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:38:56.899 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:38:56.898+0000 7f245d575600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:38:56.916 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:38:56.917 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:38:56.917 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:38:56.917 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:38:56.918 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:38:56.918 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:38:56.919 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:38:56.919 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:38:56.920 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:38:56.920 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:38:56.922 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:38:56.922 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:38:56.923 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:38:56.923 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:38:56.923 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:38:56.923 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:38:56.924 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:38:56.924 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:38:56.924 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:38:56.924 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:38:56.925 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:38:56.925 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:38:56.925 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:38:56.925 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:38:56.926 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:38:56.926 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:38:56.926 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:38:56.927 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:38:56.927 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:38:56.927 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:38:56.927 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:38:56.928 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:38:56.928 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:38:56.928 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:38:56.928 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:38:56.929 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:38:56.929 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:38:56.929 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662183531.355467_184.1", 2022-09-03T05:38:56.929 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:38:56.930 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:38:56.930 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:38:56.930 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:38:51.355467Z", 2022-09-03T05:38:56.931 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:38:56.931 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:38:56.931 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:38:56.931 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:38:56.932 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:38:56.932 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:38:56.932 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:38:56.932 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:38:56.933 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:38:56.933 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:38:56.933 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:38:56.933 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:38:56.934 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:38:56.934 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:38:56.934 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:38:56.934 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:38:56.935 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:38:56.935 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:38:56.935 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:38:56.935 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:38:56.936 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:38:56.936 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:38:56Z" 2022-09-03T05:38:56.936 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:38:56.938 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183531.355467_184.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:38:51.355467Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:38:56Z" } 2022-09-03T05:38:56.939 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183531.355467_184.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:38:51.355467Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:38:56.939 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183531.355467_184.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:38:56.941 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183531.355467_184.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:38:56.941 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:38:56.942 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:38:56.942 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:38:56.943 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:38:56.943 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:38:56.996 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:38:56.997 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:38:57.048 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:38:57.038+0000 7f0681151600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:38:57.057 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:38:57.039+0000 7f0681151600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:38:57.058 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:38:57.059 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:38:57.059 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:38:57.059 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:38:57.060 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:38:57.060 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:38:57.061 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:38:57.061 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:38:57.062 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:38:57.063 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:38:57.063 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:38:57.064 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:38:57.064 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:38:57.065 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:38:57.065 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:38:57.066 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:38:57.066 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:38:57.067 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:38:57.067 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:38:57.068 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:38:57.068 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:38:57.069 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:38:57.070 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:38:57.070 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:38:57.071 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:38:57.071 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:38:57.072 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:38:57.072 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:38:57.072 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:38:57.073 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:38:57.073 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:38:57.073 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:38:57.073 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:38:57.074 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:38:57.074 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:38:57.075 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:38:57.075 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:38:57.075 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183531.355467_184.1", 2022-09-03T05:38:57.076 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:38:57.076 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:38:57.076 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:38:57.076 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:38:51.355467Z", 2022-09-03T05:38:57.077 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:38:57.077 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:38:57.078 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:38:57.078 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:38:57.078 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:38:57.079 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:38:57.079 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:38:57.080 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:38:57.081 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:38:57.081 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:38:57.082 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:38:57.082 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:38:57.083 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:38:57.084 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:38:57.084 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:38:57.085 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:38:57.085 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:38:57.086 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:38:57.086 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:38:57.087 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:38:57.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:38:57.088 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:38:57Z" 2022-09-03T05:38:57.088 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:38:57.090 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183531.355467_184.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:38:51.355467Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:38:57Z" } 2022-09-03T05:38:57.091 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183531.355467_184.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:38:51.355467Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:38:57.092 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183531.355467_184.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:38:57.096 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183531.355467_184.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:38:57.097 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:38:57.098 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:38:57.099 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:38:57.100 DEBUG:boto:Method: HEAD 2022-09-03T05:38:57.102 DEBUG:boto:Path: /dsgvcy-20/ 2022-09-03T05:38:57.106 DEBUG:boto:Data: 2022-09-03T05:38:57.107 DEBUG:boto:Headers: {} 2022-09-03T05:38:57.108 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:38:57.111 DEBUG:boto:Port: 8000 2022-09-03T05:38:57.114 DEBUG:boto:Params: {} 2022-09-03T05:38:57.115 DEBUG:boto:Token: None 2022-09-03T05:38:57.116 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:38:57 GMT /dsgvcy-20/ 2022-09-03T05:38:57.117 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zXXSUQDqhH2g5ZZw0g1zIU7tiek= 2022-09-03T05:38:57.123 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:38:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zXXSUQDqhH2g5ZZw0g1zIU7tiek=', 'Content-Length': '0'} 2022-09-03T05:38:57.127 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000047b2859b13fa3ff1-006312e871-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:57.128 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:38:57.129 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:38:57.130 DEBUG:boto:Method: PUT 2022-09-03T05:38:57.130 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:38:57.131 DEBUG:boto:Data: 2022-09-03T05:38:57.131 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue'} 2022-09-03T05:38:57.132 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:38:57.136 DEBUG:boto:Port: 8000 2022-09-03T05:38:57.137 DEBUG:boto:Params: {} 2022-09-03T05:38:57.138 DEBUG:boto:Token: None 2022-09-03T05:38:57.139 DEBUG:boto:StringToSign: PUT 1B2M2Y8AsgTpgAmY7PhCfg== application/octet-stream Sat, 03 Sep 2022 05:38:57 GMT /dsgvcy-20/obj 2022-09-03T05:38:57.140 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9cUer6JT1F5l65M8326joVB4v3Y= 2022-09-03T05:38:57.141 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:38:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9cUer6JT1F5l65M8326joVB4v3Y='} 2022-09-03T05:38:57.167 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-amz-request-id', 'tx00000ac7bbb0bac7db1ef-006312e871-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:38:57.169 DEBUG:rgw_multi.tests:created new version id=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:38:57.174 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-20 2022-09-03T05:38:57.174 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:38:57.175 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:38:57.176 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-20 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:38:57.241 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:38:57.241 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:38:57.287 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:38:57.284+0000 7fe81c05d600 1 waiting for incremental sync to catch up: 2022-09-03T05:38:57.288 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.6.5, , ] 2022-09-03T05:38:57.288 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000003.9.12, , ] 2022-09-03T05:39:02.286 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:39:02.285+0000 7fe81c05d600 1 waiting for incremental sync to catch up: 2022-09-03T05:39:02.286 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.6.5, , ] 2022-09-03T05:39:02.287 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000003.9.12, , ] 2022-09-03T05:39:07.290 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:39:07.287+0000 7fe81c05d600 1 bucket sync caught up with source: 2022-09-03T05:39:07.291 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000003.9.12, , ] 2022-09-03T05:39:07.293 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000003.9.12, , ] 2022-09-03T05:39:07.294 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:39:07.287+0000 7fe81c05d600 0 bucket checkpoint complete 2022-09-03T05:39:07.308 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-20 2022-09-03T05:39:07.309 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:39:07.310 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:39:07.311 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-20 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:39:07.374 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:39:07.375 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:39:07.434 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:39:07.431+0000 7fee4ddc5600 1 bucket sync caught up with source: 2022-09-03T05:39:07.435 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000003.9.12, , ] 2022-09-03T05:39:07.435 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000003.9.12, , ] 2022-09-03T05:39:07.436 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:39:07.431+0000 7fee4ddc5600 0 bucket checkpoint complete 2022-09-03T05:39:07.459 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-20 2022-09-03T05:39:07.460 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:39:07.460 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:39:07.461 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-20 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:39:07.528 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:39:07.530 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:39:07.579 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:39:07.579+0000 7f4ab0021600 1 waiting for incremental sync to catch up: 2022-09-03T05:39:07.579 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.6.5, , ] 2022-09-03T05:39:07.579 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000003.9.12, , ] 2022-09-03T05:39:12.580 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:39:12.580+0000 7f4ab0021600 1 waiting for incremental sync to catch up: 2022-09-03T05:39:12.581 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.6.5, , ] 2022-09-03T05:39:12.588 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000003.9.12, , ] 2022-09-03T05:39:17.581 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:39:17.581+0000 7f4ab0021600 1 bucket sync caught up with source: 2022-09-03T05:39:17.582 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000003.9.12, , ] 2022-09-03T05:39:17.582 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000003.9.12, , ] 2022-09-03T05:39:17.583 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:39:17.581+0000 7f4ab0021600 0 bucket checkpoint complete 2022-09-03T05:39:17.596 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-20 2022-09-03T05:39:17.597 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:39:17.597 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:39:17.598 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-20 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:39:17.663 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:39:17.663 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:39:17.710 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:39:17.711+0000 7f08b61fb600 1 waiting for incremental sync to catch up: 2022-09-03T05:39:17.711 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000001.21.5, , ] 2022-09-03T05:39:17.711 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000003.26.12, , ] 2022-09-03T05:39:22.712 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:39:22.712+0000 7f08b61fb600 1 waiting for incremental sync to catch up: 2022-09-03T05:39:22.712 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000001.21.5, , ] 2022-09-03T05:39:22.713 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000003.26.12, , ] 2022-09-03T05:39:27.714 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:39:27.714+0000 7f08b61fb600 1 bucket sync caught up with source: 2022-09-03T05:39:27.714 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000003.26.12, , ] 2022-09-03T05:39:27.715 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000003.26.12, , ] 2022-09-03T05:39:27.715 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:39:27.714+0000 7f08b61fb600 0 bucket checkpoint complete 2022-09-03T05:39:27.736 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-20 2022-09-03T05:39:27.737 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:39:27.738 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:39:27.739 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-20 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:39:27.800 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:39:27.800 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:39:27.860 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:39:27.860+0000 7f596f2f3600 1 bucket sync caught up with source: 2022-09-03T05:39:27.860 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000003.26.12, , ] 2022-09-03T05:39:27.861 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000003.26.12, , ] 2022-09-03T05:39:27.861 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:39:27.860+0000 7f596f2f3600 0 bucket checkpoint complete 2022-09-03T05:39:27.877 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-20 2022-09-03T05:39:27.878 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-20 2022-09-03T05:39:27.879 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:39:27.879 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:39:27.880 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-20 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:39:27.937 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:39:27.938 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:39:27.990 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:39:27.990+0000 7f1dde157600 1 bucket sync caught up with source: 2022-09-03T05:39:27.990 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000004.26.9, , ] 2022-09-03T05:39:27.991 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000004.26.9, , ] 2022-09-03T05:39:27.991 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:39:27.990+0000 7f1dde157600 0 bucket checkpoint complete 2022-09-03T05:39:28.011 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-20 2022-09-03T05:39:28.012 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:39:28.021 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:39:28.022 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-20 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:39:28.091 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:39:28.093 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:39:28.123 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:39:28.122+0000 7fc2fec9f600 1 bucket sync caught up with source: 2022-09-03T05:39:28.124 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000004.26.9, , ] 2022-09-03T05:39:28.124 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000004.26.9, , ] 2022-09-03T05:39:28.125 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:39:28.122+0000 7fc2fec9f600 0 bucket checkpoint complete 2022-09-03T05:39:28.138 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-20 2022-09-03T05:39:28.139 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-20 zones={test-zone2, test-zone1} 2022-09-03T05:39:28.140 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:28.140 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:28.141 DEBUG:boto:Method: HEAD 2022-09-03T05:39:28.141 DEBUG:boto:Path: /dsgvcy-20/ 2022-09-03T05:39:28.143 DEBUG:boto:Data: 2022-09-03T05:39:28.144 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.144 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:28.146 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.146 DEBUG:boto:Params: {} 2022-09-03T05:39:28.147 DEBUG:boto:Token: None 2022-09-03T05:39:28.148 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/ 2022-09-03T05:39:28.148 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KtlToRonrRlmuFGibZcyIwd4YyA= 2022-09-03T05:39:28.150 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KtlToRonrRlmuFGibZcyIwd4YyA=', 'Content-Length': '0'} 2022-09-03T05:39:28.154 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b876a38987fd6ca5-006312e890-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.155 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:28.156 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:28.156 DEBUG:boto:Method: HEAD 2022-09-03T05:39:28.157 DEBUG:boto:Path: /dsgvcy-20/ 2022-09-03T05:39:28.157 DEBUG:boto:Data: 2022-09-03T05:39:28.158 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.158 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:28.159 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.161 DEBUG:boto:Params: {} 2022-09-03T05:39:28.163 DEBUG:boto:Token: None 2022-09-03T05:39:28.167 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/ 2022-09-03T05:39:28.168 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KtlToRonrRlmuFGibZcyIwd4YyA= 2022-09-03T05:39:28.171 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KtlToRonrRlmuFGibZcyIwd4YyA=', 'Content-Length': '0'} 2022-09-03T05:39:28.178 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000acef97c697ef3174-006312e890-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.180 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:39:28.183 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:28.184 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:28.185 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.186 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.187 DEBUG:boto:Method: GET 2022-09-03T05:39:28.187 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.188 DEBUG:boto:Data: 2022-09-03T05:39:28.188 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.189 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:28.189 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.191 DEBUG:boto:Params: {} 2022-09-03T05:39:28.191 DEBUG:boto:Token: None 2022-09-03T05:39:28.192 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/?versions 2022-09-03T05:39:28.192 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f9z5dNkrVOD8yzn2UkjHDH5/8pI= 2022-09-03T05:39:28.193 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f9z5dNkrVOD8yzn2UkjHDH5/8pI=', 'Content-Length': '0'} 2022-09-03T05:39:28.197 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006f1eff697baa8b38-006312e890-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.198 DEBUG:boto:b'dsgvcy-20999falseobj3gHYK5tcVyHVd8hBlbGdESZfgPef8xAtrue2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjnullfalse2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:28.199 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:39:28.200 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:39:28.200 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:39:28.201 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:28.201 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:28.202 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.202 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.203 DEBUG:boto:Method: GET 2022-09-03T05:39:28.204 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.204 DEBUG:boto:Data: 2022-09-03T05:39:28.205 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.205 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:28.206 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.206 DEBUG:boto:Params: {} 2022-09-03T05:39:28.207 DEBUG:boto:Token: None 2022-09-03T05:39:28.207 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/?versions 2022-09-03T05:39:28.208 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f9z5dNkrVOD8yzn2UkjHDH5/8pI= 2022-09-03T05:39:28.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f9z5dNkrVOD8yzn2UkjHDH5/8pI=', 'Content-Length': '0'} 2022-09-03T05:39:28.211 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004be9974af9f27124-006312e890-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.212 DEBUG:boto:b'dsgvcy-20999falseobj3gHYK5tcVyHVd8hBlbGdESZfgPef8xAtrue2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjnullfalse2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:28.213 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:39:28.213 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:39:28.214 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:28.215 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:28.215 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.216 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.216 DEBUG:boto:Method: GET 2022-09-03T05:39:28.217 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.217 DEBUG:boto:Data: 2022-09-03T05:39:28.218 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.218 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:28.219 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.220 DEBUG:boto:Params: {} 2022-09-03T05:39:28.220 DEBUG:boto:Token: None 2022-09-03T05:39:28.221 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/?versions 2022-09-03T05:39:28.222 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f9z5dNkrVOD8yzn2UkjHDH5/8pI= 2022-09-03T05:39:28.222 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f9z5dNkrVOD8yzn2UkjHDH5/8pI=', 'Content-Length': '0'} 2022-09-03T05:39:28.225 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d98d9db4daa6a4f6-006312e890-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.225 DEBUG:boto:b'dsgvcy-20999falseobj3gHYK5tcVyHVd8hBlbGdESZfgPef8xAtrue2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjnullfalse2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:28.227 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:28.229 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:28.229 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.230 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.230 DEBUG:boto:Method: GET 2022-09-03T05:39:28.233 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.233 DEBUG:boto:Data: 2022-09-03T05:39:28.234 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.235 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:28.236 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.237 DEBUG:boto:Params: {} 2022-09-03T05:39:28.237 DEBUG:boto:Token: None 2022-09-03T05:39:28.239 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/?versions 2022-09-03T05:39:28.240 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f9z5dNkrVOD8yzn2UkjHDH5/8pI= 2022-09-03T05:39:28.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f9z5dNkrVOD8yzn2UkjHDH5/8pI=', 'Content-Length': '0'} 2022-09-03T05:39:28.243 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000df854b6f5225527e-006312e890-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.244 DEBUG:boto:b'dsgvcy-20999falseobj3gHYK5tcVyHVd8hBlbGdESZfgPef8xAtrue2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjnullfalse2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:28.246 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:39:28.247 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:28.247 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:28.248 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.250 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.251 DEBUG:boto:Method: GET 2022-09-03T05:39:28.252 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.253 DEBUG:boto:Data: 2022-09-03T05:39:28.254 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.255 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:28.256 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.257 DEBUG:boto:Params: {} 2022-09-03T05:39:28.258 DEBUG:boto:Token: None 2022-09-03T05:39:28.259 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.260 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+Oe7EHR8cMugVWr+sQPzq8L4JzM= 2022-09-03T05:39:28.261 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+Oe7EHR8cMugVWr+sQPzq8L4JzM=', 'Content-Length': '0'} 2022-09-03T05:39:28.265 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000007eb18c8bfa400eaa-006312e890-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.267 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:28.268 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:28.269 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.270 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.273 DEBUG:boto:Method: GET 2022-09-03T05:39:28.274 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.275 DEBUG:boto:Data: 2022-09-03T05:39:28.275 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.276 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:28.277 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.277 DEBUG:boto:Params: {} 2022-09-03T05:39:28.279 DEBUG:boto:Token: None 2022-09-03T05:39:28.280 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.281 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+Oe7EHR8cMugVWr+sQPzq8L4JzM= 2022-09-03T05:39:28.281 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+Oe7EHR8cMugVWr+sQPzq8L4JzM=', 'Content-Length': '0'} 2022-09-03T05:39:28.283 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000776c8dac91e1fc1a-006312e890-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.285 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:28.285 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:28.286 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.287 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.287 DEBUG:boto:Method: HEAD 2022-09-03T05:39:28.288 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.288 DEBUG:boto:Data: 2022-09-03T05:39:28.289 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.289 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:28.290 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.290 DEBUG:boto:Params: {} 2022-09-03T05:39:28.291 DEBUG:boto:Token: None 2022-09-03T05:39:28.292 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.292 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yb7sXiWBeGXMW7BjO1nqt+Vas7c= 2022-09-03T05:39:28.293 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yb7sXiWBeGXMW7BjO1nqt+Vas7c=', 'Content-Length': '0'} 2022-09-03T05:39:28.295 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000006965b70d841486c1-006312e890-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.297 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:28.298 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:28.299 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.300 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.301 DEBUG:boto:Method: HEAD 2022-09-03T05:39:28.301 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.302 DEBUG:boto:Data: 2022-09-03T05:39:28.305 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.305 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:28.306 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.306 DEBUG:boto:Params: {} 2022-09-03T05:39:28.307 DEBUG:boto:Token: None 2022-09-03T05:39:28.307 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.308 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yb7sXiWBeGXMW7BjO1nqt+Vas7c= 2022-09-03T05:39:28.310 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yb7sXiWBeGXMW7BjO1nqt+Vas7c=', 'Content-Length': '0'} 2022-09-03T05:39:28.312 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000004945e706f36eaec7-006312e890-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.314 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:39:28.315 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:28.316 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:28.317 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.318 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.319 DEBUG:boto:Method: GET 2022-09-03T05:39:28.320 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.320 DEBUG:boto:Data: 2022-09-03T05:39:28.321 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.321 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:28.322 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.323 DEBUG:boto:Params: {} 2022-09-03T05:39:28.323 DEBUG:boto:Token: None 2022-09-03T05:39:28.324 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.325 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+Oe7EHR8cMugVWr+sQPzq8L4JzM= 2022-09-03T05:39:28.326 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+Oe7EHR8cMugVWr+sQPzq8L4JzM=', 'Content-Length': '0'} 2022-09-03T05:39:28.329 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000773d9e78ffe97c87-006312e890-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.331 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:28.332 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:28.333 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.334 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.335 DEBUG:boto:Method: GET 2022-09-03T05:39:28.336 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.337 DEBUG:boto:Data: 2022-09-03T05:39:28.337 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.338 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:28.339 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.340 DEBUG:boto:Params: {} 2022-09-03T05:39:28.341 DEBUG:boto:Token: None 2022-09-03T05:39:28.341 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.342 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+Oe7EHR8cMugVWr+sQPzq8L4JzM= 2022-09-03T05:39:28.343 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+Oe7EHR8cMugVWr+sQPzq8L4JzM=', 'Content-Length': '0'} 2022-09-03T05:39:28.345 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000004f00fefa58c53b8b-006312e890-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.346 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:28.347 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:28.347 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.348 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.348 DEBUG:boto:Method: GET 2022-09-03T05:39:28.349 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.349 DEBUG:boto:Data: 2022-09-03T05:39:28.350 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.350 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:28.351 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.351 DEBUG:boto:Params: {} 2022-09-03T05:39:28.352 DEBUG:boto:Token: None 2022-09-03T05:39:28.352 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/ 2022-09-03T05:39:28.353 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rdOzQzXQ4dtow2UQDW/lkMzvoIM= 2022-09-03T05:39:28.354 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rdOzQzXQ4dtow2UQDW/lkMzvoIM=', 'Content-Length': '0'} 2022-09-03T05:39:28.357 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fb49b49adabd59b2-006312e890-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.357 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:28.358 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:28.359 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:28.359 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.360 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.360 DEBUG:boto:Method: GET 2022-09-03T05:39:28.361 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.361 DEBUG:boto:Data: 2022-09-03T05:39:28.362 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.362 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:28.363 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.364 DEBUG:boto:Params: {} 2022-09-03T05:39:28.365 DEBUG:boto:Token: None 2022-09-03T05:39:28.366 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/ 2022-09-03T05:39:28.367 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rdOzQzXQ4dtow2UQDW/lkMzvoIM= 2022-09-03T05:39:28.368 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rdOzQzXQ4dtow2UQDW/lkMzvoIM=', 'Content-Length': '0'} 2022-09-03T05:39:28.371 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007da12df0cb1b63c5-006312e890-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.414 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:28.416 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:28.416 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:28.417 DEBUG:boto:Method: HEAD 2022-09-03T05:39:28.417 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:39:28.418 DEBUG:boto:Data: 2022-09-03T05:39:28.418 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.420 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:28.422 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.424 DEBUG:boto:Params: {} 2022-09-03T05:39:28.425 DEBUG:boto:Token: None 2022-09-03T05:39:28.426 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/obj 2022-09-03T05:39:28.427 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y8DZSBvVz2aSFxhTAhAxhtGvZQc= 2022-09-03T05:39:28.429 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y8DZSBvVz2aSFxhTAhAxhtGvZQc=', 'Content-Length': '0'} 2022-09-03T05:39:28.432 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000066ba7af11ed28e47-006312e890-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.434 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:28.435 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:28.436 DEBUG:boto:Method: HEAD 2022-09-03T05:39:28.437 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:39:28.438 DEBUG:boto:Data: 2022-09-03T05:39:28.439 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.440 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:28.441 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.442 DEBUG:boto:Params: {} 2022-09-03T05:39:28.443 DEBUG:boto:Token: None 2022-09-03T05:39:28.444 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/obj 2022-09-03T05:39:28.446 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y8DZSBvVz2aSFxhTAhAxhtGvZQc= 2022-09-03T05:39:28.454 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y8DZSBvVz2aSFxhTAhAxhtGvZQc=', 'Content-Length': '0'} 2022-09-03T05:39:28.457 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000006451d93761d19e0a-006312e890-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.458 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:39:28.459 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:28.460 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:28.460 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.461 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.461 DEBUG:boto:Method: GET 2022-09-03T05:39:28.462 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.463 DEBUG:boto:Data: 2022-09-03T05:39:28.463 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.464 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:28.464 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.465 DEBUG:boto:Params: {} 2022-09-03T05:39:28.466 DEBUG:boto:Token: None 2022-09-03T05:39:28.466 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.468 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+Oe7EHR8cMugVWr+sQPzq8L4JzM= 2022-09-03T05:39:28.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+Oe7EHR8cMugVWr+sQPzq8L4JzM=', 'Content-Length': '0'} 2022-09-03T05:39:28.470 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a38f6fcf2daff3cb-006312e890-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.471 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:28.472 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:28.472 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.473 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.473 DEBUG:boto:Method: GET 2022-09-03T05:39:28.476 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.477 DEBUG:boto:Data: 2022-09-03T05:39:28.478 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.479 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:28.480 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.481 DEBUG:boto:Params: {} 2022-09-03T05:39:28.482 DEBUG:boto:Token: None 2022-09-03T05:39:28.483 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.484 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+Oe7EHR8cMugVWr+sQPzq8L4JzM= 2022-09-03T05:39:28.485 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+Oe7EHR8cMugVWr+sQPzq8L4JzM=', 'Content-Length': '0'} 2022-09-03T05:39:28.492 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000006f056c065dda4780-006312e890-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.493 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:28.494 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:28.495 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.495 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.496 DEBUG:boto:Method: GET 2022-09-03T05:39:28.497 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.499 DEBUG:boto:Data: 2022-09-03T05:39:28.503 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.504 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:28.505 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.506 DEBUG:boto:Params: {} 2022-09-03T05:39:28.510 DEBUG:boto:Token: None 2022-09-03T05:39:28.511 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/ 2022-09-03T05:39:28.513 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rdOzQzXQ4dtow2UQDW/lkMzvoIM= 2022-09-03T05:39:28.514 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rdOzQzXQ4dtow2UQDW/lkMzvoIM=', 'Content-Length': '0'} 2022-09-03T05:39:28.518 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d3f3799c5b9fa6a2-006312e890-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.519 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:28.521 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:28.521 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:28.522 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.523 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.524 DEBUG:boto:Method: GET 2022-09-03T05:39:28.525 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.526 DEBUG:boto:Data: 2022-09-03T05:39:28.527 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.528 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:28.529 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.529 DEBUG:boto:Params: {} 2022-09-03T05:39:28.530 DEBUG:boto:Token: None 2022-09-03T05:39:28.531 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/ 2022-09-03T05:39:28.532 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rdOzQzXQ4dtow2UQDW/lkMzvoIM= 2022-09-03T05:39:28.533 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rdOzQzXQ4dtow2UQDW/lkMzvoIM=', 'Content-Length': '0'} 2022-09-03T05:39:28.536 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002264dfcb0fae3841-006312e890-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.538 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:28.539 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:39:28.540 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:28.541 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:28.542 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.543 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.544 DEBUG:boto:Method: GET 2022-09-03T05:39:28.545 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.546 DEBUG:boto:Data: 2022-09-03T05:39:28.547 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.548 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:28.549 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.551 DEBUG:boto:Params: {} 2022-09-03T05:39:28.552 DEBUG:boto:Token: None 2022-09-03T05:39:28.553 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.554 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pFlNiPqjJTO2k2gAzcqWYtwngZ4= 2022-09-03T05:39:28.556 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pFlNiPqjJTO2k2gAzcqWYtwngZ4=', 'Content-Length': '0'} 2022-09-03T05:39:28.558 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000b46d77b2abe7b75b-006312e890-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.559 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:28.560 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:28.561 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.562 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.563 DEBUG:boto:Method: GET 2022-09-03T05:39:28.564 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.565 DEBUG:boto:Data: 2022-09-03T05:39:28.566 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.567 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:28.568 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.570 DEBUG:boto:Params: {} 2022-09-03T05:39:28.571 DEBUG:boto:Token: None 2022-09-03T05:39:28.572 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.573 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pFlNiPqjJTO2k2gAzcqWYtwngZ4= 2022-09-03T05:39:28.574 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pFlNiPqjJTO2k2gAzcqWYtwngZ4=', 'Content-Length': '0'} 2022-09-03T05:39:28.577 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000980a3d25b042867b-006312e890-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.578 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:28.579 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:28.580 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.581 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.582 DEBUG:boto:Method: HEAD 2022-09-03T05:39:28.583 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.584 DEBUG:boto:Data: 2022-09-03T05:39:28.585 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.586 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:28.588 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.588 DEBUG:boto:Params: {} 2022-09-03T05:39:28.589 DEBUG:boto:Token: None 2022-09-03T05:39:28.590 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.590 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iXSVDS1gPFEDfddWxVJxB5gU83g= 2022-09-03T05:39:28.591 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iXSVDS1gPFEDfddWxVJxB5gU83g=', 'Content-Length': '0'} 2022-09-03T05:39:28.595 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000004d327da426e4932f-006312e890-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.597 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:28.598 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:28.599 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.600 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.601 DEBUG:boto:Method: HEAD 2022-09-03T05:39:28.603 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.604 DEBUG:boto:Data: 2022-09-03T05:39:28.607 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.608 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:28.609 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.609 DEBUG:boto:Params: {} 2022-09-03T05:39:28.611 DEBUG:boto:Token: None 2022-09-03T05:39:28.612 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.613 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iXSVDS1gPFEDfddWxVJxB5gU83g= 2022-09-03T05:39:28.614 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iXSVDS1gPFEDfddWxVJxB5gU83g=', 'Content-Length': '0'} 2022-09-03T05:39:28.617 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000009050290e3c30679d-006312e890-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.618 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:39:28.619 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:28.620 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:28.620 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.621 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.621 DEBUG:boto:Method: GET 2022-09-03T05:39:28.622 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.623 DEBUG:boto:Data: 2022-09-03T05:39:28.623 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.624 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:28.624 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.625 DEBUG:boto:Params: {} 2022-09-03T05:39:28.626 DEBUG:boto:Token: None 2022-09-03T05:39:28.626 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.627 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pFlNiPqjJTO2k2gAzcqWYtwngZ4= 2022-09-03T05:39:28.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pFlNiPqjJTO2k2gAzcqWYtwngZ4=', 'Content-Length': '0'} 2022-09-03T05:39:28.630 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000008661ce7c64f17939-006312e890-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.631 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:28.635 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:28.636 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.637 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.638 DEBUG:boto:Method: GET 2022-09-03T05:39:28.639 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.640 DEBUG:boto:Data: 2022-09-03T05:39:28.642 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.643 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:28.646 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.647 DEBUG:boto:Params: {} 2022-09-03T05:39:28.648 DEBUG:boto:Token: None 2022-09-03T05:39:28.649 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:28.650 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pFlNiPqjJTO2k2gAzcqWYtwngZ4= 2022-09-03T05:39:28.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pFlNiPqjJTO2k2gAzcqWYtwngZ4=', 'Content-Length': '0'} 2022-09-03T05:39:28.654 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000cfd5adcc6bb74849-006312e890-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.655 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:28.656 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:28.659 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.661 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.662 DEBUG:boto:Method: GET 2022-09-03T05:39:28.663 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.664 DEBUG:boto:Data: 2022-09-03T05:39:28.665 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.666 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:28.667 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.668 DEBUG:boto:Params: {} 2022-09-03T05:39:28.669 DEBUG:boto:Token: None 2022-09-03T05:39:28.670 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/ 2022-09-03T05:39:28.672 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rdOzQzXQ4dtow2UQDW/lkMzvoIM= 2022-09-03T05:39:28.673 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rdOzQzXQ4dtow2UQDW/lkMzvoIM=', 'Content-Length': '0'} 2022-09-03T05:39:28.676 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000824cde7e80155d43-006312e890-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.677 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:28.678 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:28.678 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:28.679 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.680 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.680 DEBUG:boto:Method: GET 2022-09-03T05:39:28.681 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.681 DEBUG:boto:Data: 2022-09-03T05:39:28.682 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.683 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:28.684 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.685 DEBUG:boto:Params: {} 2022-09-03T05:39:28.686 DEBUG:boto:Token: None 2022-09-03T05:39:28.687 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/ 2022-09-03T05:39:28.688 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rdOzQzXQ4dtow2UQDW/lkMzvoIM= 2022-09-03T05:39:28.689 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rdOzQzXQ4dtow2UQDW/lkMzvoIM=', 'Content-Length': '0'} 2022-09-03T05:39:28.697 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c81e6d84eada1534-006312e890-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.698 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:28.699 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:28.700 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:28.700 DEBUG:boto:Method: HEAD 2022-09-03T05:39:28.701 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:39:28.701 DEBUG:boto:Data: 2022-09-03T05:39:28.702 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.703 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:28.704 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.705 DEBUG:boto:Params: {} 2022-09-03T05:39:28.706 DEBUG:boto:Token: None 2022-09-03T05:39:28.707 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/obj 2022-09-03T05:39:28.707 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y8DZSBvVz2aSFxhTAhAxhtGvZQc= 2022-09-03T05:39:28.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y8DZSBvVz2aSFxhTAhAxhtGvZQc=', 'Content-Length': '0'} 2022-09-03T05:39:28.710 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000633b0c17e6433351-006312e890-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.711 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:28.711 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:28.712 DEBUG:boto:Method: HEAD 2022-09-03T05:39:28.712 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:39:28.713 DEBUG:boto:Data: 2022-09-03T05:39:28.713 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.715 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:28.718 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.719 DEBUG:boto:Params: {} 2022-09-03T05:39:28.719 DEBUG:boto:Token: None 2022-09-03T05:39:28.720 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/obj 2022-09-03T05:39:28.720 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y8DZSBvVz2aSFxhTAhAxhtGvZQc= 2022-09-03T05:39:28.721 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y8DZSBvVz2aSFxhTAhAxhtGvZQc=', 'Content-Length': '0'} 2022-09-03T05:39:28.729 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000001c58c69e9f4d25b0-006312e890-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.730 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:39:28.731 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:28.731 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:28.732 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.733 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.733 DEBUG:boto:Method: GET 2022-09-03T05:39:28.734 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.734 DEBUG:boto:Data: 2022-09-03T05:39:28.735 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.735 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:28.736 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.736 DEBUG:boto:Params: {} 2022-09-03T05:39:28.737 DEBUG:boto:Token: None 2022-09-03T05:39:28.738 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.741 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+Oe7EHR8cMugVWr+sQPzq8L4JzM= 2022-09-03T05:39:28.743 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+Oe7EHR8cMugVWr+sQPzq8L4JzM=', 'Content-Length': '0'} 2022-09-03T05:39:28.746 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000aeb73bd9449a1820-006312e890-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.747 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:28.748 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:28.748 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.749 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.753 DEBUG:boto:Method: GET 2022-09-03T05:39:28.753 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.754 DEBUG:boto:Data: 2022-09-03T05:39:28.755 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.755 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:28.756 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.756 DEBUG:boto:Params: {} 2022-09-03T05:39:28.759 DEBUG:boto:Token: None 2022-09-03T05:39:28.760 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.761 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+Oe7EHR8cMugVWr+sQPzq8L4JzM= 2022-09-03T05:39:28.762 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+Oe7EHR8cMugVWr+sQPzq8L4JzM=', 'Content-Length': '0'} 2022-09-03T05:39:28.765 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000008d6e271e3fde0548-006312e890-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.766 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:28.767 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:28.768 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.769 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.770 DEBUG:boto:Method: GET 2022-09-03T05:39:28.771 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.772 DEBUG:boto:Data: 2022-09-03T05:39:28.773 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.774 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:28.776 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.777 DEBUG:boto:Params: {} 2022-09-03T05:39:28.778 DEBUG:boto:Token: None 2022-09-03T05:39:28.779 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/ 2022-09-03T05:39:28.780 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rdOzQzXQ4dtow2UQDW/lkMzvoIM= 2022-09-03T05:39:28.782 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rdOzQzXQ4dtow2UQDW/lkMzvoIM=', 'Content-Length': '0'} 2022-09-03T05:39:28.786 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002863436af831eaa7-006312e890-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.787 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:28.788 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:28.788 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:28.789 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.790 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.790 DEBUG:boto:Method: GET 2022-09-03T05:39:28.791 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:28.791 DEBUG:boto:Data: 2022-09-03T05:39:28.792 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.793 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:28.793 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.794 DEBUG:boto:Params: {} 2022-09-03T05:39:28.799 DEBUG:boto:Token: None 2022-09-03T05:39:28.800 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/ 2022-09-03T05:39:28.802 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rdOzQzXQ4dtow2UQDW/lkMzvoIM= 2022-09-03T05:39:28.803 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rdOzQzXQ4dtow2UQDW/lkMzvoIM=', 'Content-Length': '0'} 2022-09-03T05:39:28.815 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c28b6e8895e124ff-006312e890-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.816 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:28.817 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-20 zones={test-zone2, test-zone1} 2022-09-03T05:39:28.819 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-20 zones={test-zone3, test-zone1} 2022-09-03T05:39:28.820 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:28.821 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:28.822 DEBUG:boto:Method: HEAD 2022-09-03T05:39:28.823 DEBUG:boto:Path: /dsgvcy-20/ 2022-09-03T05:39:28.824 DEBUG:boto:Data: 2022-09-03T05:39:28.825 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.826 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:28.827 DEBUG:boto:Port: 8001 2022-09-03T05:39:28.828 DEBUG:boto:Params: {} 2022-09-03T05:39:28.830 DEBUG:boto:Token: None 2022-09-03T05:39:28.831 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/ 2022-09-03T05:39:28.832 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KtlToRonrRlmuFGibZcyIwd4YyA= 2022-09-03T05:39:28.834 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KtlToRonrRlmuFGibZcyIwd4YyA=', 'Content-Length': '0'} 2022-09-03T05:39:28.838 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000cc29fa5aff425136-006312e890-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.840 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:28.840 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:28.841 DEBUG:boto:Method: HEAD 2022-09-03T05:39:28.842 DEBUG:boto:Path: /dsgvcy-20/ 2022-09-03T05:39:28.842 DEBUG:boto:Data: 2022-09-03T05:39:28.843 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.843 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:28.844 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.845 DEBUG:boto:Params: {} 2022-09-03T05:39:28.846 DEBUG:boto:Token: None 2022-09-03T05:39:28.847 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/ 2022-09-03T05:39:28.847 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KtlToRonrRlmuFGibZcyIwd4YyA= 2022-09-03T05:39:28.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KtlToRonrRlmuFGibZcyIwd4YyA=', 'Content-Length': '0'} 2022-09-03T05:39:28.851 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000066bd612d1b79cfa2-006312e890-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.851 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:39:28.852 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:28.852 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:28.853 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.853 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.854 DEBUG:boto:Method: GET 2022-09-03T05:39:28.855 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.856 DEBUG:boto:Data: 2022-09-03T05:39:28.857 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.858 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:28.859 DEBUG:boto:Port: 8001 2022-09-03T05:39:28.860 DEBUG:boto:Params: {} 2022-09-03T05:39:28.861 DEBUG:boto:Token: None 2022-09-03T05:39:28.862 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/?versions 2022-09-03T05:39:28.863 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f9z5dNkrVOD8yzn2UkjHDH5/8pI= 2022-09-03T05:39:28.864 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f9z5dNkrVOD8yzn2UkjHDH5/8pI=', 'Content-Length': '0'} 2022-09-03T05:39:28.867 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001f8e963c1618c502-006312e890-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.868 DEBUG:boto:b'dsgvcy-20999falseobj3gHYK5tcVyHVd8hBlbGdESZfgPef8xAtrue2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjnullfalse2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:28.869 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:39:28.870 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:39:28.871 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:39:28.872 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:28.873 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:28.874 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.875 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.876 DEBUG:boto:Method: GET 2022-09-03T05:39:28.877 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.878 DEBUG:boto:Data: 2022-09-03T05:39:28.879 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.880 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:28.881 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.881 DEBUG:boto:Params: {} 2022-09-03T05:39:28.883 DEBUG:boto:Token: None 2022-09-03T05:39:28.884 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/?versions 2022-09-03T05:39:28.885 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f9z5dNkrVOD8yzn2UkjHDH5/8pI= 2022-09-03T05:39:28.887 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f9z5dNkrVOD8yzn2UkjHDH5/8pI=', 'Content-Length': '0'} 2022-09-03T05:39:28.890 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000476848a881548a0c-006312e890-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.934 DEBUG:boto:b'dsgvcy-20999falseobj3gHYK5tcVyHVd8hBlbGdESZfgPef8xAtrue2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjnullfalse2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:28.935 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:39:28.936 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:39:28.936 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:28.937 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:28.938 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.939 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.940 DEBUG:boto:Method: GET 2022-09-03T05:39:28.940 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.941 DEBUG:boto:Data: 2022-09-03T05:39:28.943 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.944 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:28.945 DEBUG:boto:Port: 8001 2022-09-03T05:39:28.946 DEBUG:boto:Params: {} 2022-09-03T05:39:28.947 DEBUG:boto:Token: None 2022-09-03T05:39:28.949 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/?versions 2022-09-03T05:39:28.950 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f9z5dNkrVOD8yzn2UkjHDH5/8pI= 2022-09-03T05:39:28.951 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f9z5dNkrVOD8yzn2UkjHDH5/8pI=', 'Content-Length': '0'} 2022-09-03T05:39:28.954 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ba147be14552ffbe-006312e890-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.955 DEBUG:boto:b'dsgvcy-20999falseobj3gHYK5tcVyHVd8hBlbGdESZfgPef8xAtrue2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjnullfalse2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:28.956 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:28.956 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:28.957 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.958 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.959 DEBUG:boto:Method: GET 2022-09-03T05:39:28.960 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:28.960 DEBUG:boto:Data: 2022-09-03T05:39:28.961 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.962 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:28.962 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.963 DEBUG:boto:Params: {} 2022-09-03T05:39:28.964 DEBUG:boto:Token: None 2022-09-03T05:39:28.964 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/?versions 2022-09-03T05:39:28.965 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f9z5dNkrVOD8yzn2UkjHDH5/8pI= 2022-09-03T05:39:28.966 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f9z5dNkrVOD8yzn2UkjHDH5/8pI=', 'Content-Length': '0'} 2022-09-03T05:39:28.969 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005cfaff6a5142ba41-006312e890-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.970 DEBUG:boto:b'dsgvcy-20999falseobj3gHYK5tcVyHVd8hBlbGdESZfgPef8xAtrue2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjnullfalse2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:28.972 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:39:28.972 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:28.973 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:28.974 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.974 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.976 DEBUG:boto:Method: GET 2022-09-03T05:39:28.976 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.977 DEBUG:boto:Data: 2022-09-03T05:39:28.977 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.978 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:28.979 DEBUG:boto:Port: 8001 2022-09-03T05:39:28.980 DEBUG:boto:Params: {} 2022-09-03T05:39:28.981 DEBUG:boto:Token: None 2022-09-03T05:39:28.981 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:28 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.982 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+Oe7EHR8cMugVWr+sQPzq8L4JzM= 2022-09-03T05:39:28.983 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+Oe7EHR8cMugVWr+sQPzq8L4JzM=', 'Content-Length': '0'} 2022-09-03T05:39:28.985 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000dad4bfed1fb9741b-006312e890-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:28.986 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:28.987 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:28.987 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.988 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.988 DEBUG:boto:Method: GET 2022-09-03T05:39:28.990 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:28.991 DEBUG:boto:Data: 2022-09-03T05:39:28.991 DEBUG:boto:Headers: {} 2022-09-03T05:39:28.992 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:28.996 DEBUG:boto:Port: 8000 2022-09-03T05:39:28.997 DEBUG:boto:Params: {} 2022-09-03T05:39:29.003 DEBUG:boto:Token: None 2022-09-03T05:39:29.005 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.007 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kOcOUybqx2yl4o9uP0y7/TO1ZL4= 2022-09-03T05:39:29.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kOcOUybqx2yl4o9uP0y7/TO1ZL4=', 'Content-Length': '0'} 2022-09-03T05:39:29.011 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a1a41432afc66116-006312e891-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.014 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.016 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.017 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.017 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.018 DEBUG:boto:Method: HEAD 2022-09-03T05:39:29.018 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.022 DEBUG:boto:Data: 2022-09-03T05:39:29.022 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.023 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:29.023 DEBUG:boto:Port: 8001 2022-09-03T05:39:29.024 DEBUG:boto:Params: {} 2022-09-03T05:39:29.025 DEBUG:boto:Token: None 2022-09-03T05:39:29.025 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.026 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kB8edVj/f8If2au6jG5/aMZdpvE= 2022-09-03T05:39:29.026 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kB8edVj/f8If2au6jG5/aMZdpvE=', 'Content-Length': '0'} 2022-09-03T05:39:29.029 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000219ef3350e67121a-006312e891-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.030 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.031 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.032 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.033 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.034 DEBUG:boto:Method: HEAD 2022-09-03T05:39:29.035 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.036 DEBUG:boto:Data: 2022-09-03T05:39:29.037 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.038 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:29.039 DEBUG:boto:Port: 8000 2022-09-03T05:39:29.040 DEBUG:boto:Params: {} 2022-09-03T05:39:29.041 DEBUG:boto:Token: None 2022-09-03T05:39:29.042 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.042 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kB8edVj/f8If2au6jG5/aMZdpvE= 2022-09-03T05:39:29.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kB8edVj/f8If2au6jG5/aMZdpvE=', 'Content-Length': '0'} 2022-09-03T05:39:29.048 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000ceada44119422246-006312e891-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.049 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:39:29.050 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.051 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.052 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.053 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.054 DEBUG:boto:Method: GET 2022-09-03T05:39:29.055 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.057 DEBUG:boto:Data: 2022-09-03T05:39:29.058 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.059 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:29.060 DEBUG:boto:Port: 8001 2022-09-03T05:39:29.061 DEBUG:boto:Params: {} 2022-09-03T05:39:29.062 DEBUG:boto:Token: None 2022-09-03T05:39:29.064 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.065 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kOcOUybqx2yl4o9uP0y7/TO1ZL4= 2022-09-03T05:39:29.066 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kOcOUybqx2yl4o9uP0y7/TO1ZL4=', 'Content-Length': '0'} 2022-09-03T05:39:29.074 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000002ac6ddd0a0fe7ef1-006312e891-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.075 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.076 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.076 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.077 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.078 DEBUG:boto:Method: GET 2022-09-03T05:39:29.078 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.079 DEBUG:boto:Data: 2022-09-03T05:39:29.079 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.080 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:29.080 DEBUG:boto:Port: 8000 2022-09-03T05:39:29.081 DEBUG:boto:Params: {} 2022-09-03T05:39:29.082 DEBUG:boto:Token: None 2022-09-03T05:39:29.083 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.084 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kOcOUybqx2yl4o9uP0y7/TO1ZL4= 2022-09-03T05:39:29.085 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kOcOUybqx2yl4o9uP0y7/TO1ZL4=', 'Content-Length': '0'} 2022-09-03T05:39:29.088 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000000bcd23b662930402-006312e891-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.089 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:29.090 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:29.091 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.091 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.092 DEBUG:boto:Method: GET 2022-09-03T05:39:29.093 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.095 DEBUG:boto:Data: 2022-09-03T05:39:29.096 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.097 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:29.098 DEBUG:boto:Port: 8001 2022-09-03T05:39:29.098 DEBUG:boto:Params: {} 2022-09-03T05:39:29.100 DEBUG:boto:Token: None 2022-09-03T05:39:29.101 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/ 2022-09-03T05:39:29.118 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MChLAbpkSF/sA6HPt7wBgUa04Yw= 2022-09-03T05:39:29.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MChLAbpkSF/sA6HPt7wBgUa04Yw=', 'Content-Length': '0'} 2022-09-03T05:39:29.123 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003138217cf35d5aea-006312e891-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.128 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:29.129 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:29.130 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:29.131 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.135 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.136 DEBUG:boto:Method: GET 2022-09-03T05:39:29.137 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.139 DEBUG:boto:Data: 2022-09-03T05:39:29.140 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.141 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:29.142 DEBUG:boto:Port: 8000 2022-09-03T05:39:29.143 DEBUG:boto:Params: {} 2022-09-03T05:39:29.145 DEBUG:boto:Token: None 2022-09-03T05:39:29.146 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/ 2022-09-03T05:39:29.147 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MChLAbpkSF/sA6HPt7wBgUa04Yw= 2022-09-03T05:39:29.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MChLAbpkSF/sA6HPt7wBgUa04Yw=', 'Content-Length': '0'} 2022-09-03T05:39:29.152 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003da2fc3cb830e327-006312e891-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.155 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:29.156 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.157 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.158 DEBUG:boto:Method: HEAD 2022-09-03T05:39:29.159 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:39:29.160 DEBUG:boto:Data: 2022-09-03T05:39:29.161 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.162 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:29.165 DEBUG:boto:Port: 8001 2022-09-03T05:39:29.165 DEBUG:boto:Params: {} 2022-09-03T05:39:29.166 DEBUG:boto:Token: None 2022-09-03T05:39:29.170 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj 2022-09-03T05:39:29.171 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3FzaPDH1s5iF26yIuo/bSpUv3HU= 2022-09-03T05:39:29.173 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3FzaPDH1s5iF26yIuo/bSpUv3HU=', 'Content-Length': '0'} 2022-09-03T05:39:29.176 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000061517ec1d56d547f-006312e891-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.179 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.180 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.181 DEBUG:boto:Method: HEAD 2022-09-03T05:39:29.182 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:39:29.183 DEBUG:boto:Data: 2022-09-03T05:39:29.183 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.184 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:29.186 DEBUG:boto:Port: 8000 2022-09-03T05:39:29.187 DEBUG:boto:Params: {} 2022-09-03T05:39:29.188 DEBUG:boto:Token: None 2022-09-03T05:39:29.189 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj 2022-09-03T05:39:29.190 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3FzaPDH1s5iF26yIuo/bSpUv3HU= 2022-09-03T05:39:29.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3FzaPDH1s5iF26yIuo/bSpUv3HU=', 'Content-Length': '0'} 2022-09-03T05:39:29.195 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000002e676692d3619263-006312e891-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.198 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:39:29.199 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.200 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.201 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.203 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.205 DEBUG:boto:Method: GET 2022-09-03T05:39:29.207 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.208 DEBUG:boto:Data: 2022-09-03T05:39:29.209 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.210 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:29.211 DEBUG:boto:Port: 8001 2022-09-03T05:39:29.218 DEBUG:boto:Params: {} 2022-09-03T05:39:29.220 DEBUG:boto:Token: None 2022-09-03T05:39:29.221 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.221 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kOcOUybqx2yl4o9uP0y7/TO1ZL4= 2022-09-03T05:39:29.222 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kOcOUybqx2yl4o9uP0y7/TO1ZL4=', 'Content-Length': '0'} 2022-09-03T05:39:29.224 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000009fa0c6eb6a4671da-006312e891-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.225 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.226 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.227 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.228 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.232 DEBUG:boto:Method: GET 2022-09-03T05:39:29.233 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.233 DEBUG:boto:Data: 2022-09-03T05:39:29.234 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.235 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:29.235 DEBUG:boto:Port: 8000 2022-09-03T05:39:29.236 DEBUG:boto:Params: {} 2022-09-03T05:39:29.236 DEBUG:boto:Token: None 2022-09-03T05:39:29.237 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.238 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kOcOUybqx2yl4o9uP0y7/TO1ZL4= 2022-09-03T05:39:29.238 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kOcOUybqx2yl4o9uP0y7/TO1ZL4=', 'Content-Length': '0'} 2022-09-03T05:39:29.240 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f39d531b6d04e478-006312e891-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.241 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:29.242 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:29.242 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.243 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.244 DEBUG:boto:Method: GET 2022-09-03T05:39:29.245 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.245 DEBUG:boto:Data: 2022-09-03T05:39:29.246 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.246 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:29.247 DEBUG:boto:Port: 8001 2022-09-03T05:39:29.248 DEBUG:boto:Params: {} 2022-09-03T05:39:29.248 DEBUG:boto:Token: None 2022-09-03T05:39:29.249 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/ 2022-09-03T05:39:29.250 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MChLAbpkSF/sA6HPt7wBgUa04Yw= 2022-09-03T05:39:29.250 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MChLAbpkSF/sA6HPt7wBgUa04Yw=', 'Content-Length': '0'} 2022-09-03T05:39:29.256 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dc4c878ed780cecb-006312e891-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.257 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:29.259 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:29.260 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:29.261 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.262 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.266 DEBUG:boto:Method: GET 2022-09-03T05:39:29.268 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.269 DEBUG:boto:Data: 2022-09-03T05:39:29.270 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.271 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:29.272 DEBUG:boto:Port: 8000 2022-09-03T05:39:29.273 DEBUG:boto:Params: {} 2022-09-03T05:39:29.274 DEBUG:boto:Token: None 2022-09-03T05:39:29.276 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/ 2022-09-03T05:39:29.283 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MChLAbpkSF/sA6HPt7wBgUa04Yw= 2022-09-03T05:39:29.285 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MChLAbpkSF/sA6HPt7wBgUa04Yw=', 'Content-Length': '0'} 2022-09-03T05:39:29.288 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ca417511f5d76d27-006312e891-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.290 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:29.295 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:39:29.296 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.297 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.298 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.299 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.300 DEBUG:boto:Method: GET 2022-09-03T05:39:29.301 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.302 DEBUG:boto:Data: 2022-09-03T05:39:29.303 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.311 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:29.312 DEBUG:boto:Port: 8001 2022-09-03T05:39:29.313 DEBUG:boto:Params: {} 2022-09-03T05:39:29.317 DEBUG:boto:Token: None 2022-09-03T05:39:29.317 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.318 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fj0Pi2pglQYLiGqXHKoXbZawYbI= 2022-09-03T05:39:29.319 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fj0Pi2pglQYLiGqXHKoXbZawYbI=', 'Content-Length': '0'} 2022-09-03T05:39:29.321 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000054a5b120418a3251-006312e891-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.322 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.323 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.323 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.324 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.324 DEBUG:boto:Method: GET 2022-09-03T05:39:29.325 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.325 DEBUG:boto:Data: 2022-09-03T05:39:29.326 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.327 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:29.328 DEBUG:boto:Port: 8000 2022-09-03T05:39:29.328 DEBUG:boto:Params: {} 2022-09-03T05:39:29.329 DEBUG:boto:Token: None 2022-09-03T05:39:29.329 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.330 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fj0Pi2pglQYLiGqXHKoXbZawYbI= 2022-09-03T05:39:29.331 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fj0Pi2pglQYLiGqXHKoXbZawYbI=', 'Content-Length': '0'} 2022-09-03T05:39:29.334 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000b290f4f2cb41eb98-006312e891-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.336 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.338 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.340 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.342 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.346 DEBUG:boto:Method: HEAD 2022-09-03T05:39:29.347 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.348 DEBUG:boto:Data: 2022-09-03T05:39:29.349 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.350 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:29.351 DEBUG:boto:Port: 8001 2022-09-03T05:39:29.352 DEBUG:boto:Params: {} 2022-09-03T05:39:29.353 DEBUG:boto:Token: None 2022-09-03T05:39:29.353 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.354 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VEO8VY3aolq5yIUTdX5R4lolFjg= 2022-09-03T05:39:29.355 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VEO8VY3aolq5yIUTdX5R4lolFjg=', 'Content-Length': '0'} 2022-09-03T05:39:29.357 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000001f41a3459f8665f2-006312e891-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.358 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.359 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.360 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.361 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.362 DEBUG:boto:Method: HEAD 2022-09-03T05:39:29.363 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.364 DEBUG:boto:Data: 2022-09-03T05:39:29.366 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.367 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:29.368 DEBUG:boto:Port: 8000 2022-09-03T05:39:29.368 DEBUG:boto:Params: {} 2022-09-03T05:39:29.369 DEBUG:boto:Token: None 2022-09-03T05:39:29.371 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.371 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VEO8VY3aolq5yIUTdX5R4lolFjg= 2022-09-03T05:39:29.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VEO8VY3aolq5yIUTdX5R4lolFjg=', 'Content-Length': '0'} 2022-09-03T05:39:29.375 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000d6c54354067cdf11-006312e891-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.377 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:39:29.378 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.379 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.380 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.381 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.385 DEBUG:boto:Method: GET 2022-09-03T05:39:29.387 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.388 DEBUG:boto:Data: 2022-09-03T05:39:29.388 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.389 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:29.389 DEBUG:boto:Port: 8001 2022-09-03T05:39:29.390 DEBUG:boto:Params: {} 2022-09-03T05:39:29.391 DEBUG:boto:Token: None 2022-09-03T05:39:29.392 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.393 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fj0Pi2pglQYLiGqXHKoXbZawYbI= 2022-09-03T05:39:29.394 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fj0Pi2pglQYLiGqXHKoXbZawYbI=', 'Content-Length': '0'} 2022-09-03T05:39:29.397 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000008a5bea268765bc17-006312e891-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.399 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.400 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.401 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.402 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.403 DEBUG:boto:Method: GET 2022-09-03T05:39:29.405 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.406 DEBUG:boto:Data: 2022-09-03T05:39:29.407 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.408 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:29.409 DEBUG:boto:Port: 8000 2022-09-03T05:39:29.409 DEBUG:boto:Params: {} 2022-09-03T05:39:29.410 DEBUG:boto:Token: None 2022-09-03T05:39:29.411 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:29.411 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fj0Pi2pglQYLiGqXHKoXbZawYbI= 2022-09-03T05:39:29.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fj0Pi2pglQYLiGqXHKoXbZawYbI=', 'Content-Length': '0'} 2022-09-03T05:39:29.415 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000009c84835c8d88ba53-006312e891-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.417 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:29.418 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:29.419 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.420 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.421 DEBUG:boto:Method: GET 2022-09-03T05:39:29.422 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.431 DEBUG:boto:Data: 2022-09-03T05:39:29.432 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.433 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:29.433 DEBUG:boto:Port: 8001 2022-09-03T05:39:29.434 DEBUG:boto:Params: {} 2022-09-03T05:39:29.435 DEBUG:boto:Token: None 2022-09-03T05:39:29.435 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/ 2022-09-03T05:39:29.436 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MChLAbpkSF/sA6HPt7wBgUa04Yw= 2022-09-03T05:39:29.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MChLAbpkSF/sA6HPt7wBgUa04Yw=', 'Content-Length': '0'} 2022-09-03T05:39:29.439 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000018970564e36a9911-006312e891-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.440 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:29.441 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:29.442 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:29.442 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.443 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.443 DEBUG:boto:Method: GET 2022-09-03T05:39:29.444 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.445 DEBUG:boto:Data: 2022-09-03T05:39:29.446 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.446 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:29.447 DEBUG:boto:Port: 8000 2022-09-03T05:39:29.447 DEBUG:boto:Params: {} 2022-09-03T05:39:29.448 DEBUG:boto:Token: None 2022-09-03T05:39:29.449 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/ 2022-09-03T05:39:29.449 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MChLAbpkSF/sA6HPt7wBgUa04Yw= 2022-09-03T05:39:29.450 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MChLAbpkSF/sA6HPt7wBgUa04Yw=', 'Content-Length': '0'} 2022-09-03T05:39:29.453 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000473ac84fab7fe523-006312e891-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.454 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:29.456 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.457 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.457 DEBUG:boto:Method: HEAD 2022-09-03T05:39:29.458 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:39:29.459 DEBUG:boto:Data: 2022-09-03T05:39:29.461 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.462 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:29.463 DEBUG:boto:Port: 8001 2022-09-03T05:39:29.464 DEBUG:boto:Params: {} 2022-09-03T05:39:29.465 DEBUG:boto:Token: None 2022-09-03T05:39:29.466 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj 2022-09-03T05:39:29.467 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3FzaPDH1s5iF26yIuo/bSpUv3HU= 2022-09-03T05:39:29.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3FzaPDH1s5iF26yIuo/bSpUv3HU=', 'Content-Length': '0'} 2022-09-03T05:39:29.478 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000048cc2059ff90b948-006312e891-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.480 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.481 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.482 DEBUG:boto:Method: HEAD 2022-09-03T05:39:29.483 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:39:29.484 DEBUG:boto:Data: 2022-09-03T05:39:29.486 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.487 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:29.488 DEBUG:boto:Port: 8000 2022-09-03T05:39:29.489 DEBUG:boto:Params: {} 2022-09-03T05:39:29.490 DEBUG:boto:Token: None 2022-09-03T05:39:29.491 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj 2022-09-03T05:39:29.493 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3FzaPDH1s5iF26yIuo/bSpUv3HU= 2022-09-03T05:39:29.494 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3FzaPDH1s5iF26yIuo/bSpUv3HU=', 'Content-Length': '0'} 2022-09-03T05:39:29.500 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000e7058d1b1ed359e8-006312e891-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.501 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:39:29.506 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.507 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.508 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.510 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.510 DEBUG:boto:Method: GET 2022-09-03T05:39:29.511 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.512 DEBUG:boto:Data: 2022-09-03T05:39:29.513 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.514 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:29.515 DEBUG:boto:Port: 8001 2022-09-03T05:39:29.516 DEBUG:boto:Params: {} 2022-09-03T05:39:29.518 DEBUG:boto:Token: None 2022-09-03T05:39:29.519 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.520 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kOcOUybqx2yl4o9uP0y7/TO1ZL4= 2022-09-03T05:39:29.521 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kOcOUybqx2yl4o9uP0y7/TO1ZL4=', 'Content-Length': '0'} 2022-09-03T05:39:29.524 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a1515b0c431e2820-006312e891-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.525 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.526 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.527 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.528 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.529 DEBUG:boto:Method: GET 2022-09-03T05:39:29.530 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.531 DEBUG:boto:Data: 2022-09-03T05:39:29.532 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.533 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:29.542 DEBUG:boto:Port: 8000 2022-09-03T05:39:29.543 DEBUG:boto:Params: {} 2022-09-03T05:39:29.544 DEBUG:boto:Token: None 2022-09-03T05:39:29.545 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.546 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kOcOUybqx2yl4o9uP0y7/TO1ZL4= 2022-09-03T05:39:29.547 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kOcOUybqx2yl4o9uP0y7/TO1ZL4=', 'Content-Length': '0'} 2022-09-03T05:39:29.549 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000cacf0abc925af081-006312e891-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.551 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:29.552 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:29.552 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.553 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.553 DEBUG:boto:Method: GET 2022-09-03T05:39:29.554 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.554 DEBUG:boto:Data: 2022-09-03T05:39:29.555 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.555 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:29.556 DEBUG:boto:Port: 8001 2022-09-03T05:39:29.556 DEBUG:boto:Params: {} 2022-09-03T05:39:29.557 DEBUG:boto:Token: None 2022-09-03T05:39:29.558 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/ 2022-09-03T05:39:29.558 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MChLAbpkSF/sA6HPt7wBgUa04Yw= 2022-09-03T05:39:29.559 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MChLAbpkSF/sA6HPt7wBgUa04Yw=', 'Content-Length': '0'} 2022-09-03T05:39:29.562 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006a93c7ca6df61599-006312e891-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.562 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:29.564 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:29.564 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:29.565 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.565 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.566 DEBUG:boto:Method: GET 2022-09-03T05:39:29.566 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.567 DEBUG:boto:Data: 2022-09-03T05:39:29.567 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.568 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:29.570 DEBUG:boto:Port: 8000 2022-09-03T05:39:29.572 DEBUG:boto:Params: {} 2022-09-03T05:39:29.573 DEBUG:boto:Token: None 2022-09-03T05:39:29.574 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/ 2022-09-03T05:39:29.574 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MChLAbpkSF/sA6HPt7wBgUa04Yw= 2022-09-03T05:39:29.575 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MChLAbpkSF/sA6HPt7wBgUa04Yw=', 'Content-Length': '0'} 2022-09-03T05:39:29.578 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007f8752409c36d4d8-006312e891-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.580 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:29.581 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-20 zones={test-zone3, test-zone1} 2022-09-03T05:39:29.583 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-20 zones={test-zone3, test-zone2} 2022-09-03T05:39:29.584 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:29.585 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:29.586 DEBUG:boto:Method: HEAD 2022-09-03T05:39:29.586 DEBUG:boto:Path: /dsgvcy-20/ 2022-09-03T05:39:29.587 DEBUG:boto:Data: 2022-09-03T05:39:29.587 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.588 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:29.588 DEBUG:boto:Port: 8001 2022-09-03T05:39:29.589 DEBUG:boto:Params: {} 2022-09-03T05:39:29.591 DEBUG:boto:Token: None 2022-09-03T05:39:29.592 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/ 2022-09-03T05:39:29.593 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fTMAicy9x9u5jcv+nDjmA/1tzow= 2022-09-03T05:39:29.594 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fTMAicy9x9u5jcv+nDjmA/1tzow=', 'Content-Length': '0'} 2022-09-03T05:39:29.597 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009037c83bbb98b32b-006312e891-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.599 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:29.600 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:29.600 DEBUG:boto:Method: HEAD 2022-09-03T05:39:29.601 DEBUG:boto:Path: /dsgvcy-20/ 2022-09-03T05:39:29.602 DEBUG:boto:Data: 2022-09-03T05:39:29.603 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.605 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:29.606 DEBUG:boto:Port: 8000 2022-09-03T05:39:29.607 DEBUG:boto:Params: {} 2022-09-03T05:39:29.607 DEBUG:boto:Token: None 2022-09-03T05:39:29.608 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/ 2022-09-03T05:39:29.609 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fTMAicy9x9u5jcv+nDjmA/1tzow= 2022-09-03T05:39:29.610 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fTMAicy9x9u5jcv+nDjmA/1tzow=', 'Content-Length': '0'} 2022-09-03T05:39:29.614 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000da2e65e34f904e8c-006312e891-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.615 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:39:29.615 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:29.616 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:29.617 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:29.619 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:29.619 DEBUG:boto:Method: GET 2022-09-03T05:39:29.620 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:29.621 DEBUG:boto:Data: 2022-09-03T05:39:29.622 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.623 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:29.624 DEBUG:boto:Port: 8001 2022-09-03T05:39:29.625 DEBUG:boto:Params: {} 2022-09-03T05:39:29.626 DEBUG:boto:Token: None 2022-09-03T05:39:29.627 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/?versions 2022-09-03T05:39:29.628 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1lNCeSi4LCb39wNggpIOYLBTGXQ= 2022-09-03T05:39:29.629 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1lNCeSi4LCb39wNggpIOYLBTGXQ=', 'Content-Length': '0'} 2022-09-03T05:39:29.633 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f9f161ccc4d5fec5-006312e891-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.677 DEBUG:boto:b'dsgvcy-20999falseobj3gHYK5tcVyHVd8hBlbGdESZfgPef8xAtrue2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjnullfalse2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:29.678 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:39:29.680 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:39:29.680 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:39:29.681 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:29.682 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:29.683 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:29.685 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:29.686 DEBUG:boto:Method: GET 2022-09-03T05:39:29.687 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:29.688 DEBUG:boto:Data: 2022-09-03T05:39:29.689 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.691 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:29.692 DEBUG:boto:Port: 8000 2022-09-03T05:39:29.694 DEBUG:boto:Params: {} 2022-09-03T05:39:29.695 DEBUG:boto:Token: None 2022-09-03T05:39:29.696 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/?versions 2022-09-03T05:39:29.697 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1lNCeSi4LCb39wNggpIOYLBTGXQ= 2022-09-03T05:39:29.699 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1lNCeSi4LCb39wNggpIOYLBTGXQ=', 'Content-Length': '0'} 2022-09-03T05:39:29.702 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fc4d19b8e276ef68-006312e891-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.704 DEBUG:boto:b'dsgvcy-20999falseobj3gHYK5tcVyHVd8hBlbGdESZfgPef8xAtrue2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjnullfalse2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:29.706 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:39:29.706 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:39:29.707 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:29.708 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:29.708 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:29.709 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:29.710 DEBUG:boto:Method: GET 2022-09-03T05:39:29.710 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:29.712 DEBUG:boto:Data: 2022-09-03T05:39:29.712 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.713 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:29.714 DEBUG:boto:Port: 8001 2022-09-03T05:39:29.714 DEBUG:boto:Params: {} 2022-09-03T05:39:29.715 DEBUG:boto:Token: None 2022-09-03T05:39:29.716 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/?versions 2022-09-03T05:39:29.718 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1lNCeSi4LCb39wNggpIOYLBTGXQ= 2022-09-03T05:39:29.719 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1lNCeSi4LCb39wNggpIOYLBTGXQ=', 'Content-Length': '0'} 2022-09-03T05:39:29.722 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a4bafa51cc26c637-006312e891-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.723 DEBUG:boto:b'dsgvcy-20999falseobj3gHYK5tcVyHVd8hBlbGdESZfgPef8xAtrue2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjnullfalse2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:29.725 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:29.725 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:29.726 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:29.727 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:29.727 DEBUG:boto:Method: GET 2022-09-03T05:39:29.728 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:39:29.728 DEBUG:boto:Data: 2022-09-03T05:39:29.729 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.730 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:29.731 DEBUG:boto:Port: 8000 2022-09-03T05:39:29.732 DEBUG:boto:Params: {} 2022-09-03T05:39:29.733 DEBUG:boto:Token: None 2022-09-03T05:39:29.734 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/?versions 2022-09-03T05:39:29.735 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1lNCeSi4LCb39wNggpIOYLBTGXQ= 2022-09-03T05:39:29.736 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1lNCeSi4LCb39wNggpIOYLBTGXQ=', 'Content-Length': '0'} 2022-09-03T05:39:29.740 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003daad6783f226ae2-006312e891-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.784 DEBUG:boto:b'dsgvcy-20999falseobj3gHYK5tcVyHVd8hBlbGdESZfgPef8xAtrue2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobjnullfalse2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:29.786 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:39:29.787 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.788 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.789 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.790 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.791 DEBUG:boto:Method: GET 2022-09-03T05:39:29.792 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.793 DEBUG:boto:Data: 2022-09-03T05:39:29.794 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.795 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:29.796 DEBUG:boto:Port: 8001 2022-09-03T05:39:29.797 DEBUG:boto:Params: {} 2022-09-03T05:39:29.798 DEBUG:boto:Token: None 2022-09-03T05:39:29.799 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.801 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kOcOUybqx2yl4o9uP0y7/TO1ZL4= 2022-09-03T05:39:29.802 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kOcOUybqx2yl4o9uP0y7/TO1ZL4=', 'Content-Length': '0'} 2022-09-03T05:39:29.805 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000004bc21a45b98e7ce4-006312e891-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.806 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.806 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.807 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.807 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.808 DEBUG:boto:Method: GET 2022-09-03T05:39:29.808 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.809 DEBUG:boto:Data: 2022-09-03T05:39:29.809 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.810 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:29.811 DEBUG:boto:Port: 8000 2022-09-03T05:39:29.811 DEBUG:boto:Params: {} 2022-09-03T05:39:29.812 DEBUG:boto:Token: None 2022-09-03T05:39:29.813 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.813 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kOcOUybqx2yl4o9uP0y7/TO1ZL4= 2022-09-03T05:39:29.814 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kOcOUybqx2yl4o9uP0y7/TO1ZL4=', 'Content-Length': '0'} 2022-09-03T05:39:29.816 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000002f00a24d693d4669-006312e891-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.817 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.818 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.818 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.819 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.819 DEBUG:boto:Method: HEAD 2022-09-03T05:39:29.821 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.821 DEBUG:boto:Data: 2022-09-03T05:39:29.822 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.822 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:29.823 DEBUG:boto:Port: 8001 2022-09-03T05:39:29.824 DEBUG:boto:Params: {} 2022-09-03T05:39:29.824 DEBUG:boto:Token: None 2022-09-03T05:39:29.825 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.826 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kB8edVj/f8If2au6jG5/aMZdpvE= 2022-09-03T05:39:29.826 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kB8edVj/f8If2au6jG5/aMZdpvE=', 'Content-Length': '0'} 2022-09-03T05:39:29.828 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a269d5021a9a4c5c-006312e891-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.829 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.830 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.830 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.831 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.831 DEBUG:boto:Method: HEAD 2022-09-03T05:39:29.832 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.832 DEBUG:boto:Data: 2022-09-03T05:39:29.833 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.833 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:29.834 DEBUG:boto:Port: 8000 2022-09-03T05:39:29.834 DEBUG:boto:Params: {} 2022-09-03T05:39:29.835 DEBUG:boto:Token: None 2022-09-03T05:39:29.836 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.836 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kB8edVj/f8If2au6jG5/aMZdpvE= 2022-09-03T05:39:29.837 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kB8edVj/f8If2au6jG5/aMZdpvE=', 'Content-Length': '0'} 2022-09-03T05:39:29.839 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000d1871a17b36af8b3-006312e891-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.840 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:39:29.841 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.842 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.843 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.843 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.844 DEBUG:boto:Method: GET 2022-09-03T05:39:29.844 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.845 DEBUG:boto:Data: 2022-09-03T05:39:29.850 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.851 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:29.851 DEBUG:boto:Port: 8001 2022-09-03T05:39:29.853 DEBUG:boto:Params: {} 2022-09-03T05:39:29.854 DEBUG:boto:Token: None 2022-09-03T05:39:29.854 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.855 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kOcOUybqx2yl4o9uP0y7/TO1ZL4= 2022-09-03T05:39:29.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kOcOUybqx2yl4o9uP0y7/TO1ZL4=', 'Content-Length': '0'} 2022-09-03T05:39:29.859 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000fe22dcae4d04ecc1-006312e891-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.860 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.861 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.861 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.862 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.862 DEBUG:boto:Method: GET 2022-09-03T05:39:29.863 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.863 DEBUG:boto:Data: 2022-09-03T05:39:29.864 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.865 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:29.865 DEBUG:boto:Port: 8000 2022-09-03T05:39:29.866 DEBUG:boto:Params: {} 2022-09-03T05:39:29.867 DEBUG:boto:Token: None 2022-09-03T05:39:29.868 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.868 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kOcOUybqx2yl4o9uP0y7/TO1ZL4= 2022-09-03T05:39:29.869 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kOcOUybqx2yl4o9uP0y7/TO1ZL4=', 'Content-Length': '0'} 2022-09-03T05:39:29.877 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000cbc992a30e512da3-006312e891-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.878 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:29.878 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:29.879 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.880 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.880 DEBUG:boto:Method: GET 2022-09-03T05:39:29.881 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.881 DEBUG:boto:Data: 2022-09-03T05:39:29.882 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.882 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:29.883 DEBUG:boto:Port: 8001 2022-09-03T05:39:29.884 DEBUG:boto:Params: {} 2022-09-03T05:39:29.885 DEBUG:boto:Token: None 2022-09-03T05:39:29.886 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/ 2022-09-03T05:39:29.887 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MChLAbpkSF/sA6HPt7wBgUa04Yw= 2022-09-03T05:39:29.889 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MChLAbpkSF/sA6HPt7wBgUa04Yw=', 'Content-Length': '0'} 2022-09-03T05:39:29.893 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000047e02da07e27a237-006312e891-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.894 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:29.895 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:29.895 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:29.897 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.897 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.898 DEBUG:boto:Method: GET 2022-09-03T05:39:29.898 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:29.899 DEBUG:boto:Data: 2022-09-03T05:39:29.899 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.900 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:29.900 DEBUG:boto:Port: 8000 2022-09-03T05:39:29.901 DEBUG:boto:Params: {} 2022-09-03T05:39:29.902 DEBUG:boto:Token: None 2022-09-03T05:39:29.903 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/ 2022-09-03T05:39:29.903 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MChLAbpkSF/sA6HPt7wBgUa04Yw= 2022-09-03T05:39:29.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MChLAbpkSF/sA6HPt7wBgUa04Yw=', 'Content-Length': '0'} 2022-09-03T05:39:29.912 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a73e5fb3c3033d1f-006312e891-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.954 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:29.956 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.957 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.958 DEBUG:boto:Method: HEAD 2022-09-03T05:39:29.959 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:39:29.960 DEBUG:boto:Data: 2022-09-03T05:39:29.961 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.962 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:29.963 DEBUG:boto:Port: 8001 2022-09-03T05:39:29.965 DEBUG:boto:Params: {} 2022-09-03T05:39:29.966 DEBUG:boto:Token: None 2022-09-03T05:39:29.967 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj 2022-09-03T05:39:29.967 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3FzaPDH1s5iF26yIuo/bSpUv3HU= 2022-09-03T05:39:29.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3FzaPDH1s5iF26yIuo/bSpUv3HU=', 'Content-Length': '0'} 2022-09-03T05:39:29.970 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000ab7822e58a9843e9-006312e891-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.971 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.972 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.972 DEBUG:boto:Method: HEAD 2022-09-03T05:39:29.973 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:39:29.973 DEBUG:boto:Data: 2022-09-03T05:39:29.974 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.974 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:29.975 DEBUG:boto:Port: 8000 2022-09-03T05:39:29.976 DEBUG:boto:Params: {} 2022-09-03T05:39:29.976 DEBUG:boto:Token: None 2022-09-03T05:39:29.977 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:29 GMT /dsgvcy-20/obj 2022-09-03T05:39:29.977 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3FzaPDH1s5iF26yIuo/bSpUv3HU= 2022-09-03T05:39:29.978 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3FzaPDH1s5iF26yIuo/bSpUv3HU=', 'Content-Length': '0'} 2022-09-03T05:39:29.982 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000c8a939b699767091-006312e891-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:29.984 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:39:29.985 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:29.985 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:29.986 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.987 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.988 DEBUG:boto:Method: GET 2022-09-03T05:39:29.990 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:29.996 DEBUG:boto:Data: 2022-09-03T05:39:29.997 DEBUG:boto:Headers: {} 2022-09-03T05:39:29.998 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:29.998 DEBUG:boto:Port: 8001 2022-09-03T05:39:29.999 DEBUG:boto:Params: {} 2022-09-03T05:39:30.000 DEBUG:boto:Token: None 2022-09-03T05:39:30.000 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:30 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:30.001 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s1RW+7J0R4xQJNwJzjpmiRGzw/I= 2022-09-03T05:39:30.002 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s1RW+7J0R4xQJNwJzjpmiRGzw/I=', 'Content-Length': '0'} 2022-09-03T05:39:30.004 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000b1220a509f20efa6-006312e892-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:30.005 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:30.006 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:30.006 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:30.007 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:30.008 DEBUG:boto:Method: GET 2022-09-03T05:39:30.009 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:30.009 DEBUG:boto:Data: 2022-09-03T05:39:30.010 DEBUG:boto:Headers: {} 2022-09-03T05:39:30.010 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:30.011 DEBUG:boto:Port: 8000 2022-09-03T05:39:30.011 DEBUG:boto:Params: {} 2022-09-03T05:39:30.012 DEBUG:boto:Token: None 2022-09-03T05:39:30.013 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:30 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:30.014 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s1RW+7J0R4xQJNwJzjpmiRGzw/I= 2022-09-03T05:39:30.014 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s1RW+7J0R4xQJNwJzjpmiRGzw/I=', 'Content-Length': '0'} 2022-09-03T05:39:30.016 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000005ed6a1ee3ee94251-006312e892-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:30.017 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:30.018 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:30.019 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:30.019 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:30.020 DEBUG:boto:Method: GET 2022-09-03T05:39:30.021 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:30.021 DEBUG:boto:Data: 2022-09-03T05:39:30.022 DEBUG:boto:Headers: {} 2022-09-03T05:39:30.022 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:30.023 DEBUG:boto:Port: 8001 2022-09-03T05:39:30.024 DEBUG:boto:Params: {} 2022-09-03T05:39:30.025 DEBUG:boto:Token: None 2022-09-03T05:39:30.025 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:30 GMT /dsgvcy-20/ 2022-09-03T05:39:30.026 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xzxH/FVrI3v2/PnYsEEoXiC5Ee8= 2022-09-03T05:39:30.026 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xzxH/FVrI3v2/PnYsEEoXiC5Ee8=', 'Content-Length': '0'} 2022-09-03T05:39:30.033 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003e679d4a0f715a58-006312e892-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:30.035 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:30.040 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:30.041 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:30.041 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:30.042 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:30.043 DEBUG:boto:Method: GET 2022-09-03T05:39:30.043 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:30.044 DEBUG:boto:Data: 2022-09-03T05:39:30.044 DEBUG:boto:Headers: {} 2022-09-03T05:39:30.045 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:30.045 DEBUG:boto:Port: 8000 2022-09-03T05:39:30.046 DEBUG:boto:Params: {} 2022-09-03T05:39:30.046 DEBUG:boto:Token: None 2022-09-03T05:39:30.047 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:30 GMT /dsgvcy-20/ 2022-09-03T05:39:30.048 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xzxH/FVrI3v2/PnYsEEoXiC5Ee8= 2022-09-03T05:39:30.049 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xzxH/FVrI3v2/PnYsEEoXiC5Ee8=', 'Content-Length': '0'} 2022-09-03T05:39:30.052 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000b6628e512754e89-006312e892-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:30.054 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:30.055 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:39:30.057 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:30.058 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:30.058 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.059 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.060 DEBUG:boto:Method: GET 2022-09-03T05:39:30.062 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.062 DEBUG:boto:Data: 2022-09-03T05:39:30.063 DEBUG:boto:Headers: {} 2022-09-03T05:39:30.064 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:30.064 DEBUG:boto:Port: 8001 2022-09-03T05:39:30.065 DEBUG:boto:Params: {} 2022-09-03T05:39:30.065 DEBUG:boto:Token: None 2022-09-03T05:39:30.066 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:30 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.067 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jsolNtpy95NyckPYLmqjqfmiGgo= 2022-09-03T05:39:30.067 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jsolNtpy95NyckPYLmqjqfmiGgo=', 'Content-Length': '0'} 2022-09-03T05:39:30.069 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000006d760db90d275aff-006312e892-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:30.070 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:30.071 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:30.071 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.072 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.072 DEBUG:boto:Method: GET 2022-09-03T05:39:30.073 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.073 DEBUG:boto:Data: 2022-09-03T05:39:30.074 DEBUG:boto:Headers: {} 2022-09-03T05:39:30.074 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:30.075 DEBUG:boto:Port: 8000 2022-09-03T05:39:30.075 DEBUG:boto:Params: {} 2022-09-03T05:39:30.076 DEBUG:boto:Token: None 2022-09-03T05:39:30.077 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:30 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.077 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jsolNtpy95NyckPYLmqjqfmiGgo= 2022-09-03T05:39:30.078 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jsolNtpy95NyckPYLmqjqfmiGgo=', 'Content-Length': '0'} 2022-09-03T05:39:30.080 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000002d2a6f79c550173e-006312e892-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:30.081 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:30.081 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:30.082 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.083 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.083 DEBUG:boto:Method: HEAD 2022-09-03T05:39:30.084 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.084 DEBUG:boto:Data: 2022-09-03T05:39:30.085 DEBUG:boto:Headers: {} 2022-09-03T05:39:30.085 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:30.086 DEBUG:boto:Port: 8001 2022-09-03T05:39:30.086 DEBUG:boto:Params: {} 2022-09-03T05:39:30.087 DEBUG:boto:Token: None 2022-09-03T05:39:30.087 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:30 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.088 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QRMImgR6bRV88KBUV6Rw4uIdVls= 2022-09-03T05:39:30.089 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QRMImgR6bRV88KBUV6Rw4uIdVls=', 'Content-Length': '0'} 2022-09-03T05:39:30.092 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000958fbe1dc9d8b3a0-006312e892-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:30.093 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:30.093 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:30.094 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.094 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.095 DEBUG:boto:Method: HEAD 2022-09-03T05:39:30.095 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.096 DEBUG:boto:Data: 2022-09-03T05:39:30.096 DEBUG:boto:Headers: {} 2022-09-03T05:39:30.097 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:30.098 DEBUG:boto:Port: 8000 2022-09-03T05:39:30.098 DEBUG:boto:Params: {} 2022-09-03T05:39:30.099 DEBUG:boto:Token: None 2022-09-03T05:39:30.099 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:30 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.100 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QRMImgR6bRV88KBUV6Rw4uIdVls= 2022-09-03T05:39:30.101 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QRMImgR6bRV88KBUV6Rw4uIdVls=', 'Content-Length': '0'} 2022-09-03T05:39:30.103 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a11f6e682897d485-006312e892-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:30.104 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:39:30.104 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:30.105 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:30.105 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.106 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.106 DEBUG:boto:Method: GET 2022-09-03T05:39:30.107 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.108 DEBUG:boto:Data: 2022-09-03T05:39:30.108 DEBUG:boto:Headers: {} 2022-09-03T05:39:30.109 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:30.109 DEBUG:boto:Port: 8001 2022-09-03T05:39:30.110 DEBUG:boto:Params: {} 2022-09-03T05:39:30.111 DEBUG:boto:Token: None 2022-09-03T05:39:30.111 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:30 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.112 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jsolNtpy95NyckPYLmqjqfmiGgo= 2022-09-03T05:39:30.113 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jsolNtpy95NyckPYLmqjqfmiGgo=', 'Content-Length': '0'} 2022-09-03T05:39:30.115 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000b6c18e605fd1520b-006312e892-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:30.116 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:30.117 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:30.117 DEBUG:boto:path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.118 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.119 DEBUG:boto:Method: GET 2022-09-03T05:39:30.119 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.120 DEBUG:boto:Data: 2022-09-03T05:39:30.121 DEBUG:boto:Headers: {} 2022-09-03T05:39:30.122 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:30.122 DEBUG:boto:Port: 8000 2022-09-03T05:39:30.123 DEBUG:boto:Params: {} 2022-09-03T05:39:30.123 DEBUG:boto:Token: None 2022-09-03T05:39:30.124 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:30 GMT /dsgvcy-20/obj?versionId=null 2022-09-03T05:39:30.125 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jsolNtpy95NyckPYLmqjqfmiGgo= 2022-09-03T05:39:30.126 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jsolNtpy95NyckPYLmqjqfmiGgo=', 'Content-Length': '0'} 2022-09-03T05:39:30.130 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:37:44 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f6d02478dad84547-006312e892-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:30.131 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:30.132 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:30.133 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:30.135 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:30.135 DEBUG:boto:Method: GET 2022-09-03T05:39:30.136 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:30.137 DEBUG:boto:Data: 2022-09-03T05:39:30.138 DEBUG:boto:Headers: {} 2022-09-03T05:39:30.139 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:30.140 DEBUG:boto:Port: 8001 2022-09-03T05:39:30.141 DEBUG:boto:Params: {} 2022-09-03T05:39:30.142 DEBUG:boto:Token: None 2022-09-03T05:39:30.143 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:30 GMT /dsgvcy-20/ 2022-09-03T05:39:30.144 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xzxH/FVrI3v2/PnYsEEoXiC5Ee8= 2022-09-03T05:39:30.145 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xzxH/FVrI3v2/PnYsEEoXiC5Ee8=', 'Content-Length': '0'} 2022-09-03T05:39:30.152 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007fbaf97a99b7d9d1-006312e892-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:30.194 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:30.196 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:30.197 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:30.198 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:30.199 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:30.200 DEBUG:boto:Method: GET 2022-09-03T05:39:30.201 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:30.202 DEBUG:boto:Data: 2022-09-03T05:39:30.203 DEBUG:boto:Headers: {} 2022-09-03T05:39:30.204 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:30.207 DEBUG:boto:Port: 8000 2022-09-03T05:39:30.208 DEBUG:boto:Params: {} 2022-09-03T05:39:30.209 DEBUG:boto:Token: None 2022-09-03T05:39:30.210 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:30 GMT /dsgvcy-20/ 2022-09-03T05:39:30.212 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xzxH/FVrI3v2/PnYsEEoXiC5Ee8= 2022-09-03T05:39:30.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xzxH/FVrI3v2/PnYsEEoXiC5Ee8=', 'Content-Length': '0'} 2022-09-03T05:39:30.216 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000053e11869ee90a946-006312e892-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:30.218 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:30.220 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:30.221 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:30.222 DEBUG:boto:Method: HEAD 2022-09-03T05:39:30.223 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:39:30.223 DEBUG:boto:Data: 2022-09-03T05:39:30.224 DEBUG:boto:Headers: {} 2022-09-03T05:39:30.225 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:30.226 DEBUG:boto:Port: 8001 2022-09-03T05:39:30.227 DEBUG:boto:Params: {} 2022-09-03T05:39:30.227 DEBUG:boto:Token: None 2022-09-03T05:39:30.228 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:30 GMT /dsgvcy-20/obj 2022-09-03T05:39:30.228 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Nj0Mw+Fu8qYuciJEtmygGh5oF+M= 2022-09-03T05:39:30.230 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Nj0Mw+Fu8qYuciJEtmygGh5oF+M=', 'Content-Length': '0'} 2022-09-03T05:39:30.233 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000073b3eba283980365-006312e892-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:30.235 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:30.236 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:30.237 DEBUG:boto:Method: HEAD 2022-09-03T05:39:30.238 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:39:30.239 DEBUG:boto:Data: 2022-09-03T05:39:30.240 DEBUG:boto:Headers: {} 2022-09-03T05:39:30.246 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:30.247 DEBUG:boto:Port: 8000 2022-09-03T05:39:30.254 DEBUG:boto:Params: {} 2022-09-03T05:39:30.256 DEBUG:boto:Token: None 2022-09-03T05:39:30.257 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:30 GMT /dsgvcy-20/obj 2022-09-03T05:39:30.259 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Nj0Mw+Fu8qYuciJEtmygGh5oF+M= 2022-09-03T05:39:30.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Nj0Mw+Fu8qYuciJEtmygGh5oF+M=', 'Content-Length': '0'} 2022-09-03T05:39:30.263 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000004f5f1bfd1cd1e624-006312e892-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:30.264 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:39:30.265 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:30.267 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:30.267 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:30.268 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:30.271 DEBUG:boto:Method: GET 2022-09-03T05:39:30.272 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:30.273 DEBUG:boto:Data: 2022-09-03T05:39:30.274 DEBUG:boto:Headers: {} 2022-09-03T05:39:30.275 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:30.276 DEBUG:boto:Port: 8001 2022-09-03T05:39:30.276 DEBUG:boto:Params: {} 2022-09-03T05:39:30.277 DEBUG:boto:Token: None 2022-09-03T05:39:30.278 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:30 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:30.279 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s1RW+7J0R4xQJNwJzjpmiRGzw/I= 2022-09-03T05:39:30.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s1RW+7J0R4xQJNwJzjpmiRGzw/I=', 'Content-Length': '0'} 2022-09-03T05:39:30.283 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000bc2b0d26ec2f9079-006312e892-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:30.284 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:30.284 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:30.285 DEBUG:boto:path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:30.286 DEBUG:boto:auth_path=/dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:30.287 DEBUG:boto:Method: GET 2022-09-03T05:39:30.287 DEBUG:boto:Path: /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:30.288 DEBUG:boto:Data: 2022-09-03T05:39:30.289 DEBUG:boto:Headers: {} 2022-09-03T05:39:30.290 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:30.291 DEBUG:boto:Port: 8000 2022-09-03T05:39:30.292 DEBUG:boto:Params: {} 2022-09-03T05:39:30.293 DEBUG:boto:Token: None 2022-09-03T05:39:30.294 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:30 GMT /dsgvcy-20/obj?versionId=3gHYK5tcVyHVd8hBlbGdESZfgPef8xA 2022-09-03T05:39:30.295 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s1RW+7J0R4xQJNwJzjpmiRGzw/I= 2022-09-03T05:39:30.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s1RW+7J0R4xQJNwJzjpmiRGzw/I=', 'Content-Length': '0'} 2022-09-03T05:39:30.303 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:38:57 GMT'), ('x-amz-version-id', '3gHYK5tcVyHVd8hBlbGdESZfgPef8xA'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000006b3a5403d7dc0b63-006312e892-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:39:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:30.307 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:30.308 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:30.308 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:30.309 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:30.310 DEBUG:boto:Method: GET 2022-09-03T05:39:30.310 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:30.311 DEBUG:boto:Data: 2022-09-03T05:39:30.312 DEBUG:boto:Headers: {} 2022-09-03T05:39:30.312 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:39:30.313 DEBUG:boto:Port: 8001 2022-09-03T05:39:30.313 DEBUG:boto:Params: {} 2022-09-03T05:39:30.314 DEBUG:boto:Token: None 2022-09-03T05:39:30.315 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:30 GMT /dsgvcy-20/ 2022-09-03T05:39:30.316 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xzxH/FVrI3v2/PnYsEEoXiC5Ee8= 2022-09-03T05:39:30.318 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xzxH/FVrI3v2/PnYsEEoXiC5Ee8=', 'Content-Length': '0'} 2022-09-03T05:39:30.321 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f7be454c7a79fdde-006312e892-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:30.322 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:30.324 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:30.325 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:30.326 DEBUG:boto:path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:30.327 DEBUG:boto:auth_path=/dsgvcy-20/?prefix=obj 2022-09-03T05:39:30.328 DEBUG:boto:Method: GET 2022-09-03T05:39:30.329 DEBUG:boto:Path: /dsgvcy-20/?prefix=obj 2022-09-03T05:39:30.330 DEBUG:boto:Data: 2022-09-03T05:39:30.331 DEBUG:boto:Headers: {} 2022-09-03T05:39:30.332 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:39:30.333 DEBUG:boto:Port: 8000 2022-09-03T05:39:30.334 DEBUG:boto:Params: {} 2022-09-03T05:39:30.335 DEBUG:boto:Token: None 2022-09-03T05:39:30.337 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:39:30 GMT /dsgvcy-20/ 2022-09-03T05:39:30.338 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xzxH/FVrI3v2/PnYsEEoXiC5Ee8= 2022-09-03T05:39:30.339 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xzxH/FVrI3v2/PnYsEEoXiC5Ee8=', 'Content-Length': '0'} 2022-09-03T05:39:30.343 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b99dbff7e0d83145-006312e892-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:39:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:30.345 DEBUG:boto:b'dsgvcy-20obj1000falseobj2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:39:30.347 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-20 zones={test-zone3, test-zone2} 2022-09-03T05:39:30.347 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:30.348 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:30.349 DEBUG:boto:path=/dsgvcy-20/?versioning 2022-09-03T05:39:30.350 DEBUG:boto:auth_path=/dsgvcy-20/?versioning 2022-09-03T05:39:30.353 DEBUG:boto:Method: PUT 2022-09-03T05:39:30.353 DEBUG:boto:Path: /dsgvcy-20/?versioning 2022-09-03T05:39:30.354 DEBUG:boto:Data: Suspended Disabled 2022-09-03T05:39:30.355 DEBUG:boto:Headers: {} 2022-09-03T05:39:30.355 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:30.356 DEBUG:boto:Port: 8000 2022-09-03T05:39:30.357 DEBUG:boto:Params: {} 2022-09-03T05:39:30.358 DEBUG:boto:Token: None 2022-09-03T05:39:30.359 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:39:30 GMT /dsgvcy-20/?versioning 2022-09-03T05:39:30.360 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zFck1OyFSpK92hGIw61RmXXOXk0= 2022-09-03T05:39:30.361 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zFck1OyFSpK92hGIw61RmXXOXk0=', 'Content-Length': '230'} 2022-09-03T05:39:30.369 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000009c4d529b3fa1df51-006312e892-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:39:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:30.371 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:39:30.372 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:39:30.374 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:39:30.430 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:39:30.431 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:39:30.476 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=41ecd2a5-afc0-4eae-9349-7dfe39baa1cb 2022-09-03T05:39:30.491 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:39:30.492 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:39:30.493 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:39:30.494 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:18:18.906852Z" 2022-09-03T05:39:30.495 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:39:30.495 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:39:30.496 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:39:30.497 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:39:30.497 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:39:30.498 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:39:30.498 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662183570.368454_186.1", 2022-09-03T05:39:30.499 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:39:30.368454Z" 2022-09-03T05:39:30.499 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:39:30.500 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:39:30.500 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:39:30.500 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:20:30.928498Z" 2022-09-03T05:39:30.502 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:39:30.502 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:39:30.506 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183570.368454_186.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:39:30.507 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:39:30.508 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:39:30.508 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:39:30.509 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:39:30.568 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:39:30.569 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:39:30.621 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:39:30.621+0000 7f712972e600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:39:30.623 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:39:30.623+0000 7f712972e600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:39:30.642 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:39:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:39:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:39:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:39:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:39:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:39:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:39:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:39:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:39:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:39:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:39:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:39:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:39:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:39:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:39:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:39:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:39:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:39:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:39:30.653 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:39:30.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:39:30.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:39:30.655 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:39:30.656 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:39:30.657 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:39:30.658 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:39:30.658 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:39:30.659 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:39:30.660 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:39:30.660 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:39:30.661 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:39:30.661 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:39:30.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:39:30.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:39:30.662 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:39:30.663 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:39:30.663 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:39:30.664 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183570.368454_186.1", 2022-09-03T05:39:30.664 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:39:30.666 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:39:30.667 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:39:30.667 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:39:30.368454Z", 2022-09-03T05:39:30.668 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:39:30.668 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:39:30.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:39:30.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:39:30.670 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:39:30.670 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:39:30.671 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:39:30.671 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:39:30.672 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:39:30.672 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:39:30.673 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:39:30.673 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:39:30.674 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:39:30.676 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:39:30.676 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:39:30.677 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:39:30.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:39:30.677 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:39:30.677 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:39:30.678 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:39:30.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:39:30.678 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:39:30Z" 2022-09-03T05:39:30.679 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:39:30.680 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183570.368454_186.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:39:30.368454Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:39:30Z" } 2022-09-03T05:39:30.681 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183570.368454_186.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:39:30.368454Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:39:30.681 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183570.368454_186.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:39:30.682 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183570.368454_186.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:39:30.683 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:39:30.683 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:39:30.684 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:39:30.684 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:39:30.685 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:39:30.746 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:39:30.746 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:39:30.780 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:39:30.774+0000 7fbe38191600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:39:30.781 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:39:30.776+0000 7fbe38191600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:39:30.789 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:39:30.789 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:39:30.790 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:39:30.792 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:39:30.792 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:39:30.793 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:39:30.793 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:39:30.794 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:39:30.795 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:39:30.795 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:39:30.796 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:39:30.796 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:39:30.796 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:39:30.797 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:39:30.797 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:39:30.797 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:39:30.798 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:39:30.798 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:39:30.799 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:39:30.799 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:39:30.799 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:39:30.800 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:39:30.800 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:39:30.800 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:39:30.800 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:39:30.801 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:39:30.801 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:39:30.801 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:39:30.801 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:39:30.802 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:39:30.802 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:39:30.802 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:39:30.803 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:39:30.803 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:39:30.803 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:39:30.804 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:39:30.804 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:39:30.805 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662183570.365072_185.1", 2022-09-03T05:39:30.806 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:39:30.806 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:39:30.809 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:39:30.810 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:39:30.365072Z", 2022-09-03T05:39:30.810 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:39:30.811 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:39:30.811 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:39:30.812 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:39:30.812 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:39:30.813 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:39:30.813 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:39:30.815 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:39:30.816 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:39:30.816 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:39:30.816 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:39:30.816 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:39:30.817 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:39:30.817 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:39:30.817 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:39:30.817 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:39:30.818 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:39:30.818 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:39:30.818 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:39:30.819 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:39:30.819 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:39:30.819 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:39:30Z" 2022-09-03T05:39:30.819 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:39:30.821 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183570.365072_185.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:39:30.365072Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:39:30Z" } 2022-09-03T05:39:30.822 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183570.365072_185.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:39:30.365072Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:39:30.822 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183570.368454_186.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:39:30.823 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183570.365072_185.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:39:30.823 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662183570.368454_186.1 target=1_1662183570.365072_185.1 2022-09-03T05:39:35.824 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:39:35.825 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:39:35.826 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:39:35.884 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:39:35.885 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:39:35.928 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:39:35.928+0000 7f4fda7d4600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:39:35.929 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:39:35.929+0000 7f4fda7d4600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:39:35.941 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:39:35.941 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:39:35.942 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:39:35.942 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:39:35.942 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:39:35.943 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:39:35.943 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:39:35.944 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:39:35.944 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:39:35.945 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:39:35.945 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:39:35.946 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:39:35.946 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:39:35.947 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:39:35.947 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:39:35.948 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:39:35.948 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:39:35.948 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:39:35.949 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:39:35.949 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:39:35.949 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:39:35.950 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:39:35.950 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:39:35.950 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:39:35.950 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:39:35.951 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:39:35.951 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:39:35.951 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:39:35.951 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:39:35.952 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:39:35.952 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:39:35.952 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:39:35.952 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:39:35.953 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:39:35.953 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:39:35.953 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:39:35.953 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:39:35.954 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662183570.368454_186.1", 2022-09-03T05:39:35.954 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:39:35.954 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:39:35.955 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:39:35.955 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:39:30.368454Z", 2022-09-03T05:39:35.955 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:39:35.955 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:39:35.956 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:39:35.956 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:39:35.956 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:39:35.956 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:39:35.957 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:39:35.957 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:39:35.957 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:39:35.957 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:39:35.958 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:39:35.958 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:39:35.958 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:39:35.958 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:39:35.959 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:39:35.959 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:39:35.959 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:39:35.959 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:39:35.960 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:39:35.960 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:39:35.960 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:39:35.960 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:39:35Z" 2022-09-03T05:39:35.961 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:39:35.962 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183570.368454_186.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:39:30.368454Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:39:35Z" } 2022-09-03T05:39:35.963 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183570.368454_186.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:39:30.368454Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:39:35.963 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183570.368454_186.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:39:35.964 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183570.368454_186.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:39:35.964 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:39:35.965 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:39:35.965 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:39:35.966 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:39:35.967 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:39:36.026 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:39:36.027 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:39:36.055 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:39:36.055+0000 7f7a25345600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:39:36.056 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:39:36.056+0000 7f7a25345600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:39:36.067 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:39:36.068 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:39:36.068 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:39:36.068 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:39:36.069 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:39:36.069 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:39:36.069 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:39:36.069 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:39:36.070 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:39:36.070 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:39:36.071 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:39:36.071 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:39:36.071 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:39:36.072 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:39:36.072 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:39:36.072 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:39:36.073 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:39:36.073 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:39:36.073 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:39:36.073 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:39:36.074 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:39:36.074 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:39:36.074 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:39:36.075 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:39:36.075 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:39:36.075 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:39:36.075 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:39:36.076 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:39:36.076 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:39:36.076 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:39:36.076 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:39:36.077 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:39:36.077 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:39:36.077 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:39:36.078 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:39:36.078 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:39:36.078 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:39:36.078 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183570.368454_186.1", 2022-09-03T05:39:36.079 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:39:36.079 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:39:36.079 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:39:36.080 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:39:30.368454Z", 2022-09-03T05:39:36.080 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:39:36.081 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:39:36.081 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:39:36.081 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:39:36.082 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:39:36.082 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:39:36.083 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:39:36.083 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:39:36.083 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:39:36.084 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:39:36.084 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:39:36.085 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:39:36.085 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:39:36.085 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:39:36.086 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:39:36.086 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:39:36.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:39:36.087 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:39:36.087 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:39:36.088 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:39:36.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:39:36.089 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:39:36Z" 2022-09-03T05:39:36.089 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:39:36.091 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183570.368454_186.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:39:30.368454Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:39:36Z" } 2022-09-03T05:39:36.092 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183570.368454_186.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:39:30.368454Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:39:36.093 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183570.368454_186.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:39:36.093 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183570.368454_186.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:39:36.094 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:39:36.094 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:39:36.095 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:39:36.096 DEBUG:boto:Method: HEAD 2022-09-03T05:39:36.097 DEBUG:boto:Path: /dsgvcy-20/ 2022-09-03T05:39:36.098 DEBUG:boto:Data: 2022-09-03T05:39:36.099 DEBUG:boto:Headers: {} 2022-09-03T05:39:36.100 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:36.100 DEBUG:boto:Port: 8000 2022-09-03T05:39:36.101 DEBUG:boto:Params: {} 2022-09-03T05:39:36.102 DEBUG:boto:Token: None 2022-09-03T05:39:36.104 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:39:36 GMT /dsgvcy-20/ 2022-09-03T05:39:36.105 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9Y1VfFNd8DohRFP1uNAdHCD+YN8= 2022-09-03T05:39:36.106 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:39:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9Y1VfFNd8DohRFP1uNAdHCD+YN8=', 'Content-Length': '0'} 2022-09-03T05:39:36.109 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005008c876a11fee71-006312e898-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:39:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:36.111 DEBUG:boto:path=/dsgvcy-20/obj 2022-09-03T05:39:36.112 DEBUG:boto:auth_path=/dsgvcy-20/obj 2022-09-03T05:39:36.113 DEBUG:boto:Method: PUT 2022-09-03T05:39:36.114 DEBUG:boto:Path: /dsgvcy-20/obj 2022-09-03T05:39:36.114 DEBUG:boto:Data: 2022-09-03T05:39:36.115 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue'} 2022-09-03T05:39:36.116 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:39:36.117 DEBUG:boto:Port: 8000 2022-09-03T05:39:36.117 DEBUG:boto:Params: {} 2022-09-03T05:39:36.118 DEBUG:boto:Token: None 2022-09-03T05:39:36.119 DEBUG:boto:StringToSign: PUT 1B2M2Y8AsgTpgAmY7PhCfg== application/octet-stream Sat, 03 Sep 2022 05:39:36 GMT /dsgvcy-20/obj 2022-09-03T05:39:36.119 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yrrFyz9xbw6miVPyrvyphou4DGg= 2022-09-03T05:39:36.120 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:39:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yrrFyz9xbw6miVPyrvyphou4DGg='} 2022-09-03T05:39:36.131 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f7acc91702dcb5d5-006312e898-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:39:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:39:36.132 DEBUG:rgw_multi.tests:created null version id=None 2022-09-03T05:39:36.132 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-20 2022-09-03T05:39:36.133 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:39:36.133 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:39:36.134 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-20 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:39:36.192 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:39:36.193 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:39:36.227 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:39:36.225+0000 7f45f0923600 1 waiting for incremental sync to catch up: 2022-09-03T05:39:36.228 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000003.9.12, , ] 2022-09-03T05:39:36.228 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000005.13.17, , ] 2022-09-03T05:39:41.227 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:39:41.227+0000 7f45f0923600 1 waiting for incremental sync to catch up: 2022-09-03T05:39:41.228 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000003.9.12, , ] 2022-09-03T05:39:41.228 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000005.13.17, , ] 2022-09-03T05:39:46.229 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:39:46.228+0000 7f45f0923600 1 waiting for incremental sync to catch up: 2022-09-03T05:39:46.229 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000003.9.12, , ] 2022-09-03T05:39:46.230 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000005.13.17, , ] 2022-09-03T05:39:51.230 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:39:51.230+0000 7f45f0923600 1 bucket sync caught up with source: 2022-09-03T05:39:51.231 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000005.13.17, , ] 2022-09-03T05:39:51.231 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000005.13.17, , ] 2022-09-03T05:39:51.231 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:39:51.230+0000 7f45f0923600 0 bucket checkpoint complete 2022-09-03T05:39:51.248 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-20 2022-09-03T05:39:51.249 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:39:51.250 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:39:51.252 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-20 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:39:51.316 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:39:51.316 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:39:51.359 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:39:51.358+0000 7f520647d600 1 bucket sync caught up with source: 2022-09-03T05:39:51.359 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000005.13.17, , ] 2022-09-03T05:39:51.360 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000005.13.17, , ] 2022-09-03T05:39:51.360 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:39:51.358+0000 7f520647d600 0 bucket checkpoint complete 2022-09-03T05:39:51.372 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-20 2022-09-03T05:39:51.373 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:39:51.374 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:39:51.374 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-20 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:39:51.438 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:39:51.438 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:39:51.485 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:39:51.485+0000 7f089d33b600 1 bucket sync caught up with source: 2022-09-03T05:39:51.486 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000005.13.17, , ] 2022-09-03T05:39:51.486 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000005.13.17, , ] 2022-09-03T05:39:51.487 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:39:51.485+0000 7f089d33b600 0 bucket checkpoint complete 2022-09-03T05:39:51.502 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-20 2022-09-03T05:39:51.503 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:39:51.504 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:39:51.506 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-20 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:39:51.565 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:39:51.566 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:39:51.599 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:39:51.598+0000 7f14d153e600 1 waiting for incremental sync to catch up: 2022-09-03T05:39:51.600 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000004.28.9, , ] 2022-09-03T05:39:51.603 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000005.30.17, , ] 2022-09-03T05:39:56.601 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:39:56.599+0000 7f14d153e600 1 waiting for incremental sync to catch up: 2022-09-03T05:39:56.602 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000004.28.9, , ] 2022-09-03T05:39:56.603 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000005.30.17, , ] 2022-09-03T05:40:01.602 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:40:01.602+0000 7f14d153e600 1 waiting for incremental sync to catch up: 2022-09-03T05:40:01.603 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000004.28.9, , ] 2022-09-03T05:40:01.603 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000005.30.17, , ] 2022-09-03T05:40:06.603 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:40:06.603+0000 7f14d153e600 1 waiting for incremental sync to catch up: 2022-09-03T05:40:06.604 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000004.28.9, , ] 2022-09-03T05:40:06.604 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000005.30.17, , ] 2022-09-03T05:40:11.605 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:40:11.605+0000 7f14d153e600 1 bucket sync caught up with source: 2022-09-03T05:40:11.606 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000005.30.17, , ] 2022-09-03T05:40:11.606 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000005.30.17, , ] 2022-09-03T05:40:11.606 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:40:11.605+0000 7f14d153e600 0 bucket checkpoint complete 2022-09-03T05:40:11.623 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-20 2022-09-03T05:40:11.624 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:40:11.625 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:40:11.625 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-20 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:40:11.686 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:40:11.687 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:40:11.740 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:40:11.740+0000 7f770af37600 1 waiting for incremental sync to catch up: 2022-09-03T05:40:11.740 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000005.30.17, , ] 2022-09-03T05:40:11.741 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000006.34.14, , ] 2022-09-03T05:40:16.742 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:40:16.741+0000 7f770af37600 1 waiting for incremental sync to catch up: 2022-09-03T05:40:16.743 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000005.30.17, , ] 2022-09-03T05:40:16.743 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000006.34.14, , ] 2022-09-03T05:40:21.742 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:40:21.742+0000 7f770af37600 1 waiting for incremental sync to catch up: 2022-09-03T05:40:21.743 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000005.30.17, , ] 2022-09-03T05:40:21.743 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000006.34.14, , ] 2022-09-03T05:40:26.744 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:40:26.743+0000 7f770af37600 1 waiting for incremental sync to catch up: 2022-09-03T05:40:26.745 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000005.30.17, , ] 2022-09-03T05:40:26.745 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000006.34.14, , ] 2022-09-03T05:40:31.751 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:40:31.744+0000 7f770af37600 1 waiting for incremental sync to catch up: 2022-09-03T05:40:31.752 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000005.30.17, , ] 2022-09-03T05:40:31.752 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000006.34.14, , ] 2022-09-03T05:40:36.749 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:40:36.746+0000 7f770af37600 1 waiting for incremental sync to catch up: 2022-09-03T05:40:36.750 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000005.30.17, , ] 2022-09-03T05:40:36.751 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000006.34.14, , ] 2022-09-03T05:40:41.748 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:40:41.747+0000 7f770af37600 1 waiting for incremental sync to catch up: 2022-09-03T05:40:41.749 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000005.30.17, , ] 2022-09-03T05:40:41.750 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000006.34.14, , ] 2022-09-03T05:40:46.750 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:40:46.749+0000 7f770af37600 1 waiting for incremental sync to catch up: 2022-09-03T05:40:46.751 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000005.30.17, , ] 2022-09-03T05:40:46.751 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000006.34.14, , ] 2022-09-03T05:40:51.751 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:40:51.751+0000 7f770af37600 1 bucket sync caught up with source: 2022-09-03T05:40:51.752 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000006.34.14, , ] 2022-09-03T05:40:51.752 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000006.34.14, , ] 2022-09-03T05:40:51.753 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:40:51.751+0000 7f770af37600 0 bucket checkpoint complete 2022-09-03T05:40:51.769 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-20 2022-09-03T05:40:51.770 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-20 2022-09-03T05:40:51.771 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:40:51.771 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:40:51.772 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-20 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:40:51.833 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:40:51.833 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:40:51.883 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:40:51.883+0000 7f470ae26600 1 bucket sync caught up with source: 2022-09-03T05:40:51.884 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000006.32.14, , ] 2022-09-03T05:40:51.884 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000006.32.14, , ] 2022-09-03T05:40:51.884 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:40:51.883+0000 7f470ae26600 0 bucket checkpoint complete 2022-09-03T05:40:51.902 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-20 2022-09-03T05:40:51.903 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:40:51.903 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-20', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:40:51.904 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-20 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:40:51.967 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:40:51.968 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:40:52.011 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:40:52.011+0000 7f409325c600 1 bucket sync caught up with source: 2022-09-03T05:40:52.012 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000006.32.14, , ] 2022-09-03T05:40:52.012 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000006.32.14, , ] 2022-09-03T05:40:52.013 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:40:52.011+0000 7f409325c600 0 bucket checkpoint complete 2022-09-03T05:40:52.026 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-20 2022-09-03T05:40:52.027 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-20 zones={test-zone2, test-zone1} 2022-09-03T05:40:52.028 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:40:52.028 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:40:52.029 DEBUG:boto:Method: HEAD 2022-09-03T05:40:52.030 DEBUG:boto:Path: /dsgvcy-20/ 2022-09-03T05:40:52.030 DEBUG:boto:Data: 2022-09-03T05:40:52.031 DEBUG:boto:Headers: {} 2022-09-03T05:40:52.031 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:40:52.032 DEBUG:boto:Port: 8000 2022-09-03T05:40:52.032 DEBUG:boto:Params: {} 2022-09-03T05:40:52.033 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:40:52.034 DEBUG:boto:Token: None 2022-09-03T05:40:52.035 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:40:52 GMT /dsgvcy-20/ 2022-09-03T05:40:52.035 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oG4Sl8FiTr5L9VeRr//r9wriCNE= 2022-09-03T05:40:52.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oG4Sl8FiTr5L9VeRr//r9wriCNE=', 'Content-Length': '0'} 2022-09-03T05:40:52.040 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d0b84d1324ba6712-006312e8e4-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:40:52.040 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:40:52.041 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:40:52.041 DEBUG:boto:Method: HEAD 2022-09-03T05:40:52.042 DEBUG:boto:Path: /dsgvcy-20/ 2022-09-03T05:40:52.043 DEBUG:boto:Data: 2022-09-03T05:40:52.043 DEBUG:boto:Headers: {} 2022-09-03T05:40:52.044 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:40:52.044 DEBUG:boto:Port: 8000 2022-09-03T05:40:52.045 DEBUG:boto:Params: {} 2022-09-03T05:40:52.045 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:40:52.046 DEBUG:boto:Token: None 2022-09-03T05:40:52.046 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:40:52 GMT /dsgvcy-20/ 2022-09-03T05:40:52.047 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oG4Sl8FiTr5L9VeRr//r9wriCNE= 2022-09-03T05:40:52.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oG4Sl8FiTr5L9VeRr//r9wriCNE=', 'Content-Length': '0'} 2022-09-03T05:40:52.051 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a6143171d72616d9-006312e8e4-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:40:52.052 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:40:52.052 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:40:52.053 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:40:52.053 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:40:52.054 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:40:52.054 DEBUG:boto:Method: GET 2022-09-03T05:40:52.055 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:40:52.055 DEBUG:boto:Data: 2022-09-03T05:40:52.056 DEBUG:boto:Headers: {} 2022-09-03T05:40:52.056 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:40:52.057 DEBUG:boto:Port: 8000 2022-09-03T05:40:52.057 DEBUG:boto:Params: {} 2022-09-03T05:40:52.058 DEBUG:boto:Token: None 2022-09-03T05:40:52.059 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:40:52 GMT /dsgvcy-20/?versions 2022-09-03T05:40:52.059 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FtNsTyGHbBV+h5opljy8jrR4FSo= 2022-09-03T05:40:52.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FtNsTyGHbBV+h5opljy8jrR4FSo=', 'Content-Length': '0'} 2022-09-03T05:40:52.062 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008b10c21bc62e88a2-006312e8e4-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:40:52.063 DEBUG:boto:b'dsgvcy-20999falseobjnulltrue2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj3gHYK5tcVyHVd8hBlbGdESZfgPef8xAfalse2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:40:52.064 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:40:52.065 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:40:52.065 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:40:52.066 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:40:52.066 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:40:52.067 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:40:52.067 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:40:52.068 DEBUG:boto:Method: GET 2022-09-03T05:40:52.068 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:40:52.069 DEBUG:boto:Data: 2022-09-03T05:40:52.070 DEBUG:boto:Headers: {} 2022-09-03T05:40:52.070 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:40:52.071 DEBUG:boto:Port: 8000 2022-09-03T05:40:52.071 DEBUG:boto:Params: {} 2022-09-03T05:40:52.072 DEBUG:boto:Token: None 2022-09-03T05:40:52.072 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:40:52 GMT /dsgvcy-20/?versions 2022-09-03T05:40:52.073 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FtNsTyGHbBV+h5opljy8jrR4FSo= 2022-09-03T05:40:52.074 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FtNsTyGHbBV+h5opljy8jrR4FSo=', 'Content-Length': '0'} 2022-09-03T05:40:52.076 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000be2e1d6b9265957c-006312e8e4-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:40:52.077 DEBUG:boto:b'dsgvcy-20999falseobjnulltrue2022-09-03T05:39:36.123Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj3gHYK5tcVyHVd8hBlbGdESZfgPef8xAfalse2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:40:52.077 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:40:52.078 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:40:52.079 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:40:52.079 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:40:52.080 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:40:52.080 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:40:52.081 DEBUG:boto:Method: GET 2022-09-03T05:40:52.081 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:40:52.082 DEBUG:boto:Data: 2022-09-03T05:40:52.082 DEBUG:boto:Headers: {} 2022-09-03T05:40:52.083 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:40:52.083 DEBUG:boto:Port: 8000 2022-09-03T05:40:52.084 DEBUG:boto:Params: {} 2022-09-03T05:40:52.084 DEBUG:boto:Token: None 2022-09-03T05:40:52.085 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:40:52 GMT /dsgvcy-20/?versions 2022-09-03T05:40:52.086 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FtNsTyGHbBV+h5opljy8jrR4FSo= 2022-09-03T05:40:52.086 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FtNsTyGHbBV+h5opljy8jrR4FSo=', 'Content-Length': '0'} 2022-09-03T05:40:52.089 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000108a628098688d42-006312e8e4-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:40:52.089 DEBUG:boto:b'dsgvcy-20999falseobjnulltrue2022-09-03T05:37:44.404Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj3gHYK5tcVyHVd8hBlbGdESZfgPef8xAfalse2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:40:52.090 DEBUG:boto:path=/dsgvcy-20/ 2022-09-03T05:40:52.091 DEBUG:boto:auth_path=/dsgvcy-20/ 2022-09-03T05:40:52.091 DEBUG:boto:path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:40:52.092 DEBUG:boto:auth_path=/dsgvcy-20/?versions&max-keys=999 2022-09-03T05:40:52.092 DEBUG:boto:Method: GET 2022-09-03T05:40:52.093 DEBUG:boto:Path: /dsgvcy-20/?versions&max-keys=999 2022-09-03T05:40:52.093 DEBUG:boto:Data: 2022-09-03T05:40:52.094 DEBUG:boto:Headers: {} 2022-09-03T05:40:52.094 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:40:52.095 DEBUG:boto:Port: 8000 2022-09-03T05:40:52.095 DEBUG:boto:Params: {} 2022-09-03T05:40:52.096 DEBUG:boto:Token: None 2022-09-03T05:40:52.097 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:40:52 GMT /dsgvcy-20/?versions 2022-09-03T05:40:52.097 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FtNsTyGHbBV+h5opljy8jrR4FSo= 2022-09-03T05:40:52.098 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FtNsTyGHbBV+h5opljy8jrR4FSo=', 'Content-Length': '0'} 2022-09-03T05:40:52.100 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e00d431a8d2fad1d-006312e8e4-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:40:52.101 DEBUG:boto:b'dsgvcy-20999falseobjnulltrue2022-09-03T05:39:36.123Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormalobj3gHYK5tcVyHVd8hBlbGdESZfgPef8xAfalse2022-09-03T05:38:57.145Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:40:52.101 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:40:52.107 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_version_suspended_incremental_sync ... FAIL 2022-09-03T05:40:52.108 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-21 2022-09-03T05:40:52.108 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:40:52.109 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:40:52.109 DEBUG:boto:Method: PUT 2022-09-03T05:40:52.110 DEBUG:boto:Path: /dsgvcy-21/ 2022-09-03T05:40:52.110 DEBUG:boto:Data: 2022-09-03T05:40:52.111 DEBUG:boto:Headers: {} 2022-09-03T05:40:52.111 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:40:52.112 DEBUG:boto:Port: 8000 2022-09-03T05:40:52.112 DEBUG:boto:Params: {} 2022-09-03T05:40:52.113 DEBUG:boto:Token: None 2022-09-03T05:40:52.114 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:40:52 GMT /dsgvcy-21/ 2022-09-03T05:40:52.114 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jSGcc51+XKMy+rnU7CjPXDby+/8= 2022-09-03T05:40:52.115 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jSGcc51+XKMy+rnU7CjPXDby+/8=', 'Content-Length': '0'} 2022-09-03T05:40:52.142 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000004d9251a44c141d37-006312e8e4-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:40:52.143 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=dsgvcy-22 2022-09-03T05:40:52.144 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:40:52.144 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:40:52.145 DEBUG:boto:Method: PUT 2022-09-03T05:40:52.145 DEBUG:boto:Path: /dsgvcy-22/ 2022-09-03T05:40:52.146 DEBUG:boto:Data: 2022-09-03T05:40:52.146 DEBUG:boto:Headers: {} 2022-09-03T05:40:52.147 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:40:52.147 DEBUG:boto:Port: 8000 2022-09-03T05:40:52.148 DEBUG:boto:Params: {} 2022-09-03T05:40:52.148 DEBUG:boto:Token: None 2022-09-03T05:40:52.149 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:40:52 GMT /dsgvcy-22/ 2022-09-03T05:40:52.149 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:01NBF4d+XR+MT6i+KMEbm6O57oE= 2022-09-03T05:40:52.150 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:01NBF4d+XR+MT6i+KMEbm6O57oE=', 'Content-Length': '0'} 2022-09-03T05:40:53.192 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000007394381bc1904ae4-006312e8e4-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:40:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:40:53.193 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=dsgvcy-23 2022-09-03T05:40:53.194 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:40:53.195 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:40:53.196 DEBUG:boto:Method: PUT 2022-09-03T05:40:53.197 DEBUG:boto:Path: /dsgvcy-23/ 2022-09-03T05:40:53.198 DEBUG:boto:Data: 2022-09-03T05:40:53.198 DEBUG:boto:Headers: {} 2022-09-03T05:40:53.200 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:40:53.201 DEBUG:boto:Port: 8001 2022-09-03T05:40:53.202 DEBUG:boto:Params: {} 2022-09-03T05:40:53.203 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T05:40:53.205 DEBUG:boto:Token: None 2022-09-03T05:40:53.206 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:40:53 GMT /dsgvcy-23/ 2022-09-03T05:40:53.207 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PEmeNOzzdJJqtpuYDmsDliNvEnA= 2022-09-03T05:40:53.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:40:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PEmeNOzzdJJqtpuYDmsDliNvEnA=', 'Content-Length': '0'} 2022-09-03T05:40:53.256 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000088299930256c2f15-006312e8e5-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:40:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:40:53.257 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:40:53.258 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:40:53.258 DEBUG:boto:path=/dsgvcy-21/?versioning 2022-09-03T05:40:53.259 DEBUG:boto:auth_path=/dsgvcy-21/?versioning 2022-09-03T05:40:53.259 DEBUG:boto:Method: PUT 2022-09-03T05:40:53.260 DEBUG:boto:Path: /dsgvcy-21/?versioning 2022-09-03T05:40:53.260 DEBUG:boto:Data: Enabled Disabled 2022-09-03T05:40:53.261 DEBUG:boto:Headers: {} 2022-09-03T05:40:53.261 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:40:53.262 DEBUG:boto:Port: 8000 2022-09-03T05:40:53.262 DEBUG:boto:Params: {} 2022-09-03T05:40:53.263 DEBUG:boto:Token: None 2022-09-03T05:40:53.264 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:40:53 GMT /dsgvcy-21/?versioning 2022-09-03T05:40:53.264 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wapGzi9p99fc3uj0OfK/PU1Hvmc= 2022-09-03T05:40:53.265 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:40:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wapGzi9p99fc3uj0OfK/PU1Hvmc=', 'Content-Length': '228'} 2022-09-03T05:40:53.271 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000009b725dbc7508f8fa-006312e8e5-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:40:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:40:53.272 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:40:53.273 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:40:53.273 DEBUG:boto:path=/dsgvcy-22/?versioning 2022-09-03T05:40:53.274 DEBUG:boto:auth_path=/dsgvcy-22/?versioning 2022-09-03T05:40:53.275 DEBUG:boto:Method: PUT 2022-09-03T05:40:53.275 DEBUG:boto:Path: /dsgvcy-22/?versioning 2022-09-03T05:40:53.276 DEBUG:boto:Data: Enabled Disabled 2022-09-03T05:40:53.276 DEBUG:boto:Headers: {} 2022-09-03T05:40:53.277 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:40:53.277 DEBUG:boto:Port: 8000 2022-09-03T05:40:53.278 DEBUG:boto:Params: {} 2022-09-03T05:40:53.278 DEBUG:boto:Token: None 2022-09-03T05:40:53.279 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:40:53 GMT /dsgvcy-22/?versioning 2022-09-03T05:40:53.279 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hITyKwEAftmEDvZeeAmnmrsfo70= 2022-09-03T05:40:53.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:40:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hITyKwEAftmEDvZeeAmnmrsfo70=', 'Content-Length': '228'} 2022-09-03T05:40:53.291 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000009252c981a90de590-006312e8e5-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:40:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:40:53.292 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:40:53.292 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:40:53.293 DEBUG:boto:path=/dsgvcy-23/?versioning 2022-09-03T05:40:53.294 DEBUG:boto:auth_path=/dsgvcy-23/?versioning 2022-09-03T05:40:53.294 DEBUG:boto:Method: PUT 2022-09-03T05:40:53.295 DEBUG:boto:Path: /dsgvcy-23/?versioning 2022-09-03T05:40:53.295 DEBUG:boto:Data: Enabled Disabled 2022-09-03T05:40:53.296 DEBUG:boto:Headers: {} 2022-09-03T05:40:53.297 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:40:53.297 DEBUG:boto:Port: 8001 2022-09-03T05:40:53.298 DEBUG:boto:Params: {} 2022-09-03T05:40:53.298 DEBUG:boto:Token: None 2022-09-03T05:40:53.299 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:40:53 GMT /dsgvcy-23/?versioning 2022-09-03T05:40:53.300 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hVx6u7a7RXEwj+lRveWoaOJ7gQE= 2022-09-03T05:40:53.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:40:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hVx6u7a7RXEwj+lRveWoaOJ7gQE=', 'Content-Length': '228'} 2022-09-03T05:40:53.313 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000811e5b235b6b9119-006312e8e5-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:40:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:40:53.314 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:40:53.314 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:40:53.315 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:40:53.364 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:40:53.365 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:40:53.418 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=41ecd2a5-afc0-4eae-9349-7dfe39baa1cb 2022-09-03T05:40:53.429 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:40:53.430 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:40:53.431 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:40:53.431 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:18:18.906852Z" 2022-09-03T05:40:53.432 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:40:53.432 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:40:53.433 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:40:53.433 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:40:53.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:40:53.434 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:40:53.435 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662183653.308405_204.1", 2022-09-03T05:40:53.435 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:40:53.308405Z" 2022-09-03T05:40:53.436 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:40:53.436 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:40:53.437 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:40:53.437 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:20:30.928498Z" 2022-09-03T05:40:53.438 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:40:53.439 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:40:53.441 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183653.308405_204.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:40:53.442 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:40:53.443 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:40:53.444 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:40:53.445 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:40:53.501 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:40:53.502 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:40:53.543 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:40:53.537+0000 7f74ced67600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:40:53.543 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:40:53.538+0000 7f74ced67600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:40:53.552 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:40:53.552 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:40:53.553 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:40:53.553 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:40:53.554 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:40:53.554 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:40:53.554 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:40:53.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:40:53.556 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:40:53.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:40:53.557 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:40:53.557 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:40:53.558 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:40:53.558 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:40:53.558 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:40:53.559 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:40:53.559 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:40:53.559 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:40:53.559 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:40:53.560 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:40:53.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:40:53.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:40:53.560 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:40:53.561 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:40:53.561 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:40:53.561 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:40:53.561 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:40:53.562 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:40:53.562 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:40:53.562 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:40:53.563 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:40:53.563 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:40:53.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:40:53.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:40:53.564 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:40:53.564 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:40:53.564 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:40:53.564 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183653.308405_204.1", 2022-09-03T05:40:53.565 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:40:53.565 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:40:53.566 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:40:53.566 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.308405Z", 2022-09-03T05:40:53.567 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:40:53.567 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:40:53.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:40:53.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:40:53.568 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:40:53.569 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:40:53.569 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:40:53.570 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:40:53.570 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:40:53.571 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:40:53.571 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:40:53.572 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:40:53.572 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:40:53.573 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:40:53.573 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:40:53.574 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:40:53.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:40:53.575 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:40:53.575 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:40:53.576 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:40:53.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:40:53.576 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:40:53Z" 2022-09-03T05:40:53.577 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:40:53.579 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183653.308405_204.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:40:53.308405Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:40:53Z" } 2022-09-03T05:40:53.580 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183653.308405_204.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:40:53.308405Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:40:53.581 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183653.308405_204.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:40:53.582 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183653.308405_204.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:40:53.582 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:40:53.583 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:40:53.583 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:40:53.584 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:40:53.585 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:40:53.635 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:40:53.635 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:40:53.679 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:40:53.678+0000 7fc654069600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:40:53.680 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:40:53.679+0000 7fc654069600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:40:53.701 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:40:53.702 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:40:53.702 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:40:53.702 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:40:53.703 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:40:53.703 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:40:53.704 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:40:53.704 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:40:53.704 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:40:53.705 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:40:53.705 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:40:53.705 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:40:53.705 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:40:53.706 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:40:53.706 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:40:53.706 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:40:53.707 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:40:53.707 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:40:53.707 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:40:53.708 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:40:53.708 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:40:53.708 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:40:53.709 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:40:53.709 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:40:53.709 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:40:53.709 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:40:53.710 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:40:53.710 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:40:53.710 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:40:53.711 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:40:53.711 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:40:53.711 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:40:53.711 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:40:53.712 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:40:53.712 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:40:53.712 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:40:53.712 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:40:53.713 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662183653.220411_195.1", 2022-09-03T05:40:53.713 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:40:53.713 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:40:53.713 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:40:53.714 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.220411Z", 2022-09-03T05:40:53.714 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:40:53.715 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:40:53.715 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:40:53.716 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:40:53.716 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:40:53.716 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:40:53.717 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:40:53.717 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:40:53.717 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:40:53.717 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:40:53.718 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:40:53.718 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:40:53.718 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:40:53.718 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:40:53.719 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:40:53.719 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:40:53.719 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:40:53.719 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:40:53.720 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:40:53.720 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:40:53.720 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:40:53.720 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:40:53Z" 2022-09-03T05:40:53.721 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:40:53.722 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183653.220411_195.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:40:53.220411Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:40:53Z" } 2022-09-03T05:40:53.723 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183653.220411_195.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:40:53.220411Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:40:53.723 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183653.308405_204.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:40:53.724 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183653.220411_195.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:40:53.725 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662183653.308405_204.1 target=1_1662183653.220411_195.1 2022-09-03T05:40:58.726 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:40:58.727 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:40:58.727 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:40:58.784 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:40:58.785 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:40:58.828 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:40:58.828+0000 7fa42157c600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:40:58.828 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:40:58.829+0000 7fa42157c600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:40:58.843 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:40:58.843 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:40:58.844 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:40:58.844 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:40:58.844 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:40:58.844 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:40:58.845 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:40:58.845 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:40:58.846 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:40:58.846 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:40:58.846 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:40:58.847 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:40:58.847 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:40:58.848 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:40:58.848 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:40:58.848 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:40:58.848 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:40:58.849 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:40:58.849 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:40:58.849 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:40:58.849 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:40:58.850 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:40:58.850 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:40:58.850 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:40:58.851 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:40:58.851 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:40:58.852 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:40:58.853 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:40:58.853 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:40:58.854 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:40:58.854 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:40:58.854 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:40:58.855 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:40:58.855 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:40:58.856 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:40:58.856 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:40:58.857 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:40:58.857 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662183653.162940_191.1", 2022-09-03T05:40:58.858 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:40:58.858 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:40:58.858 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:40:58.859 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.162940Z", 2022-09-03T05:40:58.859 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:40:58.859 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:40:58.859 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:40:58.860 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:40:58.860 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:40:58.860 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:40:58.861 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:40:58.861 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:40:58.863 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:40:58.863 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:40:58.863 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:40:58.864 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:40:58.864 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:40:58.864 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:40:58.865 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:40:58.865 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:40:58.866 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:40:58.866 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:40:58.866 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:40:58.867 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:40:58.867 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:40:58.868 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:40:58Z" 2022-09-03T05:40:58.868 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:40:58.870 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183653.162940_191.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:40:53.162940Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:40:58Z" } 2022-09-03T05:40:58.871 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183653.162940_191.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:40:53.162940Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:40:58.872 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183653.308405_204.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:40:58.872 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183653.162940_191.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:40:58.873 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662183653.308405_204.1 target=1_1662183653.162940_191.1 2022-09-03T05:41:03.874 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:41:03.875 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:41:03.876 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:41:03.934 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:03.935 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:03.977 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:03.976+0000 7f689570c600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:41:03.978 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:03.977+0000 7f689570c600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:41:03.990 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:41:03.990 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:41:03.991 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:41:03.991 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:41:03.991 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:41:03.992 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:41:03.992 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:41:03.992 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:41:03.992 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:41:03.992 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:41:03.993 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:41:03.993 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:41:03.993 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:41:03.993 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:41:03.994 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:41:03.994 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:41:03.994 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:41:03.995 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:41:03.995 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:41:03.995 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:41:03.996 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:41:03.997 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:41:03.997 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:41:03.997 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:41:03.998 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:41:03.998 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:41:03.999 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:41:03.999 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:41:03.999 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:41:03.999 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:41:04.000 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:41:04.000 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:41:04.000 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:41:04.001 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:41:04.001 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:41:04.001 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:41:04.002 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:41:04.002 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662183653.308405_204.1", 2022-09-03T05:41:04.003 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:41:04.003 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:41:04.004 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:41:04.004 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.308405Z", 2022-09-03T05:41:04.004 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:41:04.005 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:41:04.005 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:41:04.005 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:41:04.006 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:41:04.006 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:41:04.006 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:41:04.007 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:41:04.007 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:41:04.007 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:41:04.008 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:41:04.008 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:41:04.008 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:41:04.009 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:41:04.009 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:41:04.009 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:41:04.009 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:41:04.010 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:41:04.010 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:41:04.010 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:41:04.010 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:41:04.011 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:41:03Z" 2022-09-03T05:41:04.011 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:41:04.012 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183653.308405_204.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:40:53.308405Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:41:03Z" } 2022-09-03T05:41:04.013 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183653.308405_204.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:40:53.308405Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:41:04.013 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183653.308405_204.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:41:04.014 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183653.308405_204.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:41:04.014 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:41:04.015 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:41:04.015 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:41:04.016 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:41:04.017 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:41:04.073 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:04.074 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:04.101 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:04.100+0000 7fb280191600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:41:04.102 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:04.101+0000 7fb280191600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:41:04.114 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:41:04.115 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:41:04.115 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:41:04.115 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:41:04.116 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:41:04.116 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:41:04.117 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:41:04.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:04.118 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:41:04.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:41:04.119 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:41:04.119 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:41:04.120 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:41:04.121 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:41:04.121 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:41:04.121 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:41:04.122 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:41:04.122 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:41:04.123 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:41:04.123 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:41:04.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:04.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:41:04.124 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:41:04.125 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:41:04.125 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:41:04.126 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:41:04.126 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:41:04.127 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:41:04.127 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:41:04.128 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:41:04.128 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:41:04.129 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:41:04.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:04.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:41:04.130 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:41:04.130 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:41:04.130 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:41:04.131 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183653.308405_204.1", 2022-09-03T05:41:04.131 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:41:04.131 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:41:04.132 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:41:04.132 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.308405Z", 2022-09-03T05:41:04.133 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:41:04.133 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:41:04.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:04.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:41:04.134 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:41:04.135 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:41:04.135 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:41:04.136 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:41:04.136 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:41:04.137 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:41:04.137 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:41:04.137 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:41:04.137 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:41:04.138 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:41:04.138 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:41:04.138 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:41:04.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:04.139 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:41:04.139 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:41:04.139 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:41:04.139 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:04.140 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:41:04Z" 2022-09-03T05:41:04.140 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:41:04.141 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183653.308405_204.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:40:53.308405Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:41:04Z" } 2022-09-03T05:41:04.142 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183653.308405_204.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:40:53.308405Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:41:04.143 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183653.308405_204.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:41:04.143 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183653.308405_204.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:41:04.144 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:41:04.144 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:04.145 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:04.145 DEBUG:boto:Method: HEAD 2022-09-03T05:41:04.146 DEBUG:boto:Path: /dsgvcy-21/ 2022-09-03T05:41:04.146 DEBUG:boto:Data: 2022-09-03T05:41:04.147 DEBUG:boto:Headers: {} 2022-09-03T05:41:04.147 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:04.148 DEBUG:boto:Port: 8000 2022-09-03T05:41:04.148 DEBUG:boto:Params: {} 2022-09-03T05:41:04.149 DEBUG:boto:Token: None 2022-09-03T05:41:04.149 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:04 GMT /dsgvcy-21/ 2022-09-03T05:41:04.150 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o2rMkqgfx4WneG+IhXYyaRx32tc= 2022-09-03T05:41:04.151 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o2rMkqgfx4WneG+IhXYyaRx32tc=', 'Content-Length': '0'} 2022-09-03T05:41:04.154 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000071c6c301993fdfe1-006312e8f0-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:04.155 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:04.155 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:04.156 DEBUG:boto:Method: PUT 2022-09-03T05:41:04.157 DEBUG:boto:Path: /dsgvcy-21/obj 2022-09-03T05:41:04.157 DEBUG:boto:Data: 2022-09-03T05:41:04.158 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue'} 2022-09-03T05:41:04.159 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:04.160 DEBUG:boto:Port: 8000 2022-09-03T05:41:04.161 DEBUG:boto:Params: {} 2022-09-03T05:41:04.162 DEBUG:boto:Token: None 2022-09-03T05:41:04.163 DEBUG:boto:StringToSign: PUT 1B2M2Y8AsgTpgAmY7PhCfg== application/octet-stream Sat, 03 Sep 2022 05:41:04 GMT /dsgvcy-21/obj 2022-09-03T05:41:04.164 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BD0dsOiCOElHK/hBgJiVvbf/jyU= 2022-09-03T05:41:04.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:41:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BD0dsOiCOElHK/hBgJiVvbf/jyU='} 2022-09-03T05:41:04.177 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'hsp-uXpwzRjamHyM57a9u6NdISYw8SR'), ('x-amz-request-id', 'tx00000c00d72fee5bece8c-006312e8f0-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:04.178 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:04.179 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:04.179 DEBUG:boto:Method: HEAD 2022-09-03T05:41:04.180 DEBUG:boto:Path: /dsgvcy-21/ 2022-09-03T05:41:04.180 DEBUG:boto:Data: 2022-09-03T05:41:04.181 DEBUG:boto:Headers: {} 2022-09-03T05:41:04.181 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:04.182 DEBUG:boto:Port: 8000 2022-09-03T05:41:04.182 DEBUG:boto:Params: {} 2022-09-03T05:41:04.183 DEBUG:boto:Token: None 2022-09-03T05:41:04.184 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:04 GMT /dsgvcy-21/ 2022-09-03T05:41:04.184 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o2rMkqgfx4WneG+IhXYyaRx32tc= 2022-09-03T05:41:04.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o2rMkqgfx4WneG+IhXYyaRx32tc=', 'Content-Length': '0'} 2022-09-03T05:41:04.188 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007c21a8ec9b2de314-006312e8f0-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:04.189 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:04.190 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:04.191 DEBUG:boto:Method: DELETE 2022-09-03T05:41:04.192 DEBUG:boto:Path: /dsgvcy-21/obj 2022-09-03T05:41:04.192 DEBUG:boto:Data: 2022-09-03T05:41:04.193 DEBUG:boto:Headers: {} 2022-09-03T05:41:04.194 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:04.195 DEBUG:boto:Port: 8000 2022-09-03T05:41:04.196 DEBUG:boto:Params: {} 2022-09-03T05:41:04.197 DEBUG:boto:Token: None 2022-09-03T05:41:04.198 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:41:04 GMT /dsgvcy-21/obj 2022-09-03T05:41:04.199 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VKxxABVAFQ3O7boJ0wmsZzHtZAs= 2022-09-03T05:41:04.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VKxxABVAFQ3O7boJ0wmsZzHtZAs=', 'Content-Length': '0'} 2022-09-03T05:41:04.207 DEBUG:boto:Response headers: [('x-amz-version-id', 'WOsbc4Am7smafmwB2ISQ9XrKfB7fKjg'), ('x-amz-delete-marker', 'true'), ('x-amz-request-id', 'tx000009813fc8230002cc2-006312e8f0-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:04.208 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:04.209 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:04.210 DEBUG:boto:Method: HEAD 2022-09-03T05:41:04.210 DEBUG:boto:Path: /dsgvcy-22/ 2022-09-03T05:41:04.211 DEBUG:boto:Data: 2022-09-03T05:41:04.212 DEBUG:boto:Headers: {} 2022-09-03T05:41:04.213 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:04.214 DEBUG:boto:Port: 8000 2022-09-03T05:41:04.215 DEBUG:boto:Params: {} 2022-09-03T05:41:04.215 DEBUG:boto:Token: None 2022-09-03T05:41:04.216 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:04 GMT /dsgvcy-22/ 2022-09-03T05:41:04.217 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SQTS4E+k9pHvqt51KTkRcEEh14s= 2022-09-03T05:41:04.217 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SQTS4E+k9pHvqt51KTkRcEEh14s=', 'Content-Length': '0'} 2022-09-03T05:41:04.220 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003238f067277ae995-006312e8f0-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:04.221 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:04.222 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:04.222 DEBUG:boto:Method: PUT 2022-09-03T05:41:04.223 DEBUG:boto:Path: /dsgvcy-22/obj 2022-09-03T05:41:04.223 DEBUG:boto:Data: 2022-09-03T05:41:04.224 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue'} 2022-09-03T05:41:04.224 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:04.225 DEBUG:boto:Port: 8000 2022-09-03T05:41:04.225 DEBUG:boto:Params: {} 2022-09-03T05:41:04.226 DEBUG:boto:Token: None 2022-09-03T05:41:04.227 DEBUG:boto:StringToSign: PUT 1B2M2Y8AsgTpgAmY7PhCfg== application/octet-stream Sat, 03 Sep 2022 05:41:04 GMT /dsgvcy-22/obj 2022-09-03T05:41:04.227 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cKK+3VtcF/jLqb55/FBmr4OokuA= 2022-09-03T05:41:04.228 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:41:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cKK+3VtcF/jLqb55/FBmr4OokuA='} 2022-09-03T05:41:04.239 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c'), ('x-amz-request-id', 'tx00000badce60b0d94ed0a-006312e8f0-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:04.240 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:04.241 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:04.241 DEBUG:boto:Method: HEAD 2022-09-03T05:41:04.242 DEBUG:boto:Path: /dsgvcy-22/ 2022-09-03T05:41:04.242 DEBUG:boto:Data: 2022-09-03T05:41:04.244 DEBUG:boto:Headers: {} 2022-09-03T05:41:04.245 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:04.245 DEBUG:boto:Port: 8000 2022-09-03T05:41:04.246 DEBUG:boto:Params: {} 2022-09-03T05:41:04.246 DEBUG:boto:Token: None 2022-09-03T05:41:04.247 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:04 GMT /dsgvcy-22/ 2022-09-03T05:41:04.248 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SQTS4E+k9pHvqt51KTkRcEEh14s= 2022-09-03T05:41:04.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SQTS4E+k9pHvqt51KTkRcEEh14s=', 'Content-Length': '0'} 2022-09-03T05:41:04.251 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000075c99de8b817a16a-006312e8f0-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:04.251 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:04.252 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:04.253 DEBUG:boto:Method: DELETE 2022-09-03T05:41:04.253 DEBUG:boto:Path: /dsgvcy-22/obj 2022-09-03T05:41:04.254 DEBUG:boto:Data: 2022-09-03T05:41:04.254 DEBUG:boto:Headers: {} 2022-09-03T05:41:04.255 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:04.255 DEBUG:boto:Port: 8000 2022-09-03T05:41:04.256 DEBUG:boto:Params: {} 2022-09-03T05:41:04.257 DEBUG:boto:Token: None 2022-09-03T05:41:04.257 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:41:04 GMT /dsgvcy-22/obj 2022-09-03T05:41:04.258 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zbgWEk3M/APad22Sj6veKAZQs7Q= 2022-09-03T05:41:04.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zbgWEk3M/APad22Sj6veKAZQs7Q=', 'Content-Length': '0'} 2022-09-03T05:41:04.265 DEBUG:boto:Response headers: [('x-amz-version-id', 'ME-STHbynUNWJeX4LBwGonFwf1Mv1Cc'), ('x-amz-delete-marker', 'true'), ('x-amz-request-id', 'tx00000975fdb1fced91e10-006312e8f0-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:04.266 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:04.267 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:04.267 DEBUG:boto:Method: HEAD 2022-09-03T05:41:04.268 DEBUG:boto:Path: /dsgvcy-23/ 2022-09-03T05:41:04.268 DEBUG:boto:Data: 2022-09-03T05:41:04.269 DEBUG:boto:Headers: {} 2022-09-03T05:41:04.269 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:04.270 DEBUG:boto:Port: 8001 2022-09-03T05:41:04.270 DEBUG:boto:Params: {} 2022-09-03T05:41:04.271 DEBUG:boto:Token: None 2022-09-03T05:41:04.271 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:04 GMT /dsgvcy-23/ 2022-09-03T05:41:04.272 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GeAYnufwJ+r/3NKY+jG4QwnJOw0= 2022-09-03T05:41:04.272 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GeAYnufwJ+r/3NKY+jG4QwnJOw0=', 'Content-Length': '0'} 2022-09-03T05:41:04.275 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000264dc3b5dda11d8c-006312e8f0-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:04.275 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:04.276 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:04.277 DEBUG:boto:Method: PUT 2022-09-03T05:41:04.277 DEBUG:boto:Path: /dsgvcy-23/obj 2022-09-03T05:41:04.278 DEBUG:boto:Data: 2022-09-03T05:41:04.278 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue'} 2022-09-03T05:41:04.279 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:04.279 DEBUG:boto:Port: 8001 2022-09-03T05:41:04.280 DEBUG:boto:Params: {} 2022-09-03T05:41:04.280 DEBUG:boto:Token: None 2022-09-03T05:41:04.281 DEBUG:boto:StringToSign: PUT 1B2M2Y8AsgTpgAmY7PhCfg== application/octet-stream Sat, 03 Sep 2022 05:41:04 GMT /dsgvcy-23/obj 2022-09-03T05:41:04.282 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MMvRc0+sW1cJdS0fXfk3VmM4Vxs= 2022-09-03T05:41:04.282 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:41:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MMvRc0+sW1cJdS0fXfk3VmM4Vxs='} 2022-09-03T05:41:04.293 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('x-amz-version-id', 'OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ'), ('x-amz-request-id', 'tx00000475a353c49526720-006312e8f0-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:04.294 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:04.295 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:04.296 DEBUG:boto:Method: HEAD 2022-09-03T05:41:04.297 DEBUG:boto:Path: /dsgvcy-23/ 2022-09-03T05:41:04.297 DEBUG:boto:Data: 2022-09-03T05:41:04.298 DEBUG:boto:Headers: {} 2022-09-03T05:41:04.298 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:04.299 DEBUG:boto:Port: 8001 2022-09-03T05:41:04.299 DEBUG:boto:Params: {} 2022-09-03T05:41:04.300 DEBUG:boto:Token: None 2022-09-03T05:41:04.300 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:04 GMT /dsgvcy-23/ 2022-09-03T05:41:04.301 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GeAYnufwJ+r/3NKY+jG4QwnJOw0= 2022-09-03T05:41:04.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GeAYnufwJ+r/3NKY+jG4QwnJOw0=', 'Content-Length': '0'} 2022-09-03T05:41:04.304 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b0ef617024113d0d-006312e8f0-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:04.305 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:04.305 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:04.306 DEBUG:boto:Method: DELETE 2022-09-03T05:41:04.306 DEBUG:boto:Path: /dsgvcy-23/obj 2022-09-03T05:41:04.307 DEBUG:boto:Data: 2022-09-03T05:41:04.307 DEBUG:boto:Headers: {} 2022-09-03T05:41:04.308 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:04.308 DEBUG:boto:Port: 8001 2022-09-03T05:41:04.309 DEBUG:boto:Params: {} 2022-09-03T05:41:04.309 DEBUG:boto:Token: None 2022-09-03T05:41:04.310 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:41:04 GMT /dsgvcy-23/obj 2022-09-03T05:41:04.310 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3t/u07O1T/Rk5UqtmmhA+VJD0zs= 2022-09-03T05:41:04.311 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3t/u07O1T/Rk5UqtmmhA+VJD0zs=', 'Content-Length': '0'} 2022-09-03T05:41:04.318 DEBUG:boto:Response headers: [('x-amz-version-id', 'V4KwdIyXLOdi9kXyJFsArpACSrMHdqm'), ('x-amz-delete-marker', 'true'), ('x-amz-request-id', 'tx000001128a28840eb2835-006312e8f0-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:04.319 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-21 2022-09-03T05:41:04.319 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-21', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:04.320 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-21', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:04.320 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-21 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:41:04.380 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:04.380 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:04.410 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:04.409+0000 7f323c0e7600 1 waiting to reach incremental sync.. 2022-09-03T05:41:09.412 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:09.412+0000 7f323c0e7600 1 bucket sync caught up with source: 2022-09-03T05:41:09.413 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000003.17.11, , ] 2022-09-03T05:41:09.413 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000003.17.11, , ] 2022-09-03T05:41:09.414 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:09.412+0000 7f323c0e7600 0 bucket checkpoint complete 2022-09-03T05:41:09.428 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-21 2022-09-03T05:41:09.429 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-21', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:09.429 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-21', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:09.430 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-21 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:41:09.494 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:09.495 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:09.542 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:09.542+0000 7f92d8792600 1 bucket sync caught up with source: 2022-09-03T05:41:09.543 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000003.17.11, , ] 2022-09-03T05:41:09.543 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000003.17.11, , ] 2022-09-03T05:41:09.544 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:09.542+0000 7f92d8792600 0 bucket checkpoint complete 2022-09-03T05:41:09.560 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-21 2022-09-03T05:41:09.561 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-21', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:09.562 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-21', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:09.562 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-21 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:41:09.626 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:09.627 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:09.660 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:09.660+0000 7ff75766f600 1 waiting to reach incremental sync.. 2022-09-03T05:41:14.661 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:14.660+0000 7ff75766f600 1 waiting to reach incremental sync.. 2022-09-03T05:41:19.663 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:19.662+0000 7ff75766f600 1 waiting for incremental sync to catch up: 2022-09-03T05:41:19.663 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , ] 2022-09-03T05:41:19.663 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000003.17.11, , ] 2022-09-03T05:41:24.665 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:24.664+0000 7ff75766f600 1 waiting for incremental sync to catch up: 2022-09-03T05:41:24.665 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , ] 2022-09-03T05:41:24.666 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000003.17.11, , ] 2022-09-03T05:41:29.666 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:29.665+0000 7ff75766f600 1 waiting for incremental sync to catch up: 2022-09-03T05:41:29.667 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , ] 2022-09-03T05:41:29.668 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000003.17.11, , ] 2022-09-03T05:41:34.667 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:34.667+0000 7ff75766f600 1 waiting for incremental sync to catch up: 2022-09-03T05:41:34.668 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , ] 2022-09-03T05:41:34.668 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000003.17.11, , ] 2022-09-03T05:41:39.669 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:39.668+0000 7ff75766f600 1 bucket sync caught up with source: 2022-09-03T05:41:39.670 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000003.17.11, , ] 2022-09-03T05:41:39.670 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000003.17.11, , ] 2022-09-03T05:41:39.671 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:39.668+0000 7ff75766f600 0 bucket checkpoint complete 2022-09-03T05:41:39.688 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-21 2022-09-03T05:41:39.689 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-21', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:39.690 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-21', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:39.691 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-21 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:41:39.755 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:39.756 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:39.809 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:39.809+0000 7fb2b8625600 1 bucket sync caught up with source: 2022-09-03T05:41:39.809 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000002.47.5, , ] 2022-09-03T05:41:39.810 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000002.47.5, , ] 2022-09-03T05:41:39.810 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:39.809+0000 7fb2b8625600 0 bucket checkpoint complete 2022-09-03T05:41:39.823 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-21 2022-09-03T05:41:39.823 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-21', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:39.824 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-21', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:39.825 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-21 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:41:39.882 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:39.883 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:39.923 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:39.922+0000 7fb170e77600 1 bucket sync caught up with source: 2022-09-03T05:41:39.924 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000002.47.5, , ] 2022-09-03T05:41:39.924 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000002.47.5, , ] 2022-09-03T05:41:39.924 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:39.922+0000 7fb170e77600 0 bucket checkpoint complete 2022-09-03T05:41:39.938 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-21 2022-09-03T05:41:39.939 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-21 2022-09-03T05:41:39.940 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-21', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:39.941 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-21', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:39.942 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-21 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:41:40.000 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:40.001 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:40.050 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:40.049+0000 7fbfd3c6f600 1 bucket sync caught up with source: 2022-09-03T05:41:40.051 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000003.98.11, , ] 2022-09-03T05:41:40.051 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000003.98.11, , ] 2022-09-03T05:41:40.052 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:40.049+0000 7fbfd3c6f600 0 bucket checkpoint complete 2022-09-03T05:41:40.073 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-21 2022-09-03T05:41:40.074 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-21', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:40.075 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-21', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:40.076 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-21 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:41:40.141 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:40.141 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:40.182 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:40.181+0000 7f16033d0600 1 bucket sync caught up with source: 2022-09-03T05:41:40.183 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000003.98.11, , ] 2022-09-03T05:41:40.183 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000003.98.11, , ] 2022-09-03T05:41:40.183 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:40.181+0000 7f16033d0600 0 bucket checkpoint complete 2022-09-03T05:41:40.195 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-21 2022-09-03T05:41:40.196 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-21 zones={test-zone2, test-zone1} 2022-09-03T05:41:40.197 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.197 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.198 DEBUG:boto:Method: HEAD 2022-09-03T05:41:40.198 DEBUG:boto:Path: /dsgvcy-21/ 2022-09-03T05:41:40.199 DEBUG:boto:Data: 2022-09-03T05:41:40.200 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.200 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:40.201 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.202 DEBUG:boto:Params: {} 2022-09-03T05:41:40.203 DEBUG:boto:Token: None 2022-09-03T05:41:40.203 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/ 2022-09-03T05:41:40.204 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a1RAUyKYZgLVrJQjGMW5xjayvEY= 2022-09-03T05:41:40.205 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a1RAUyKYZgLVrJQjGMW5xjayvEY=', 'Content-Length': '0'} 2022-09-03T05:41:40.208 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000019f6fbc26439aa01-006312e914-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.210 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.210 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.211 DEBUG:boto:Method: HEAD 2022-09-03T05:41:40.212 DEBUG:boto:Path: /dsgvcy-21/ 2022-09-03T05:41:40.213 DEBUG:boto:Data: 2022-09-03T05:41:40.214 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.215 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:40.216 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.217 DEBUG:boto:Params: {} 2022-09-03T05:41:40.218 DEBUG:boto:Token: None 2022-09-03T05:41:40.219 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/ 2022-09-03T05:41:40.220 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a1RAUyKYZgLVrJQjGMW5xjayvEY= 2022-09-03T05:41:40.221 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a1RAUyKYZgLVrJQjGMW5xjayvEY=', 'Content-Length': '0'} 2022-09-03T05:41:40.224 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006741a2326efa6c41-006312e914-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.224 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:41:40.225 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.225 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.226 DEBUG:boto:path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.226 DEBUG:boto:auth_path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.227 DEBUG:boto:Method: GET 2022-09-03T05:41:40.227 DEBUG:boto:Path: /dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.228 DEBUG:boto:Data: 2022-09-03T05:41:40.228 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.229 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:40.229 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.230 DEBUG:boto:Params: {} 2022-09-03T05:41:40.231 DEBUG:boto:Token: None 2022-09-03T05:41:40.231 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/?versions 2022-09-03T05:41:40.232 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s= 2022-09-03T05:41:40.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s=', 'Content-Length': '0'} 2022-09-03T05:41:40.235 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e5db66c937084f02-006312e914-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.235 DEBUG:boto:b'dsgvcy-21999falseobjWOsbc4Am7smafmwB2ISQ9XrKfB7fKjgtrue2022-09-03T05:41:04.203Zrgw-multisite-test-userMultisite Test UserNormalobjhsp-uXpwzRjamHyM57a9u6NdISYw8SRfalse2022-09-03T05:41:04.168Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:40.237 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:40.237 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:40.238 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:41:40.238 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.239 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.239 DEBUG:boto:path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.240 DEBUG:boto:auth_path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.240 DEBUG:boto:Method: GET 2022-09-03T05:41:40.241 DEBUG:boto:Path: /dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.241 DEBUG:boto:Data: 2022-09-03T05:41:40.242 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.242 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:40.243 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.243 DEBUG:boto:Params: {} 2022-09-03T05:41:40.244 DEBUG:boto:Token: None 2022-09-03T05:41:40.244 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/?versions 2022-09-03T05:41:40.245 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s= 2022-09-03T05:41:40.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s=', 'Content-Length': '0'} 2022-09-03T05:41:40.248 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000087b939e5042d41f4-006312e914-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.250 DEBUG:boto:b'dsgvcy-21999falseobjWOsbc4Am7smafmwB2ISQ9XrKfB7fKjgtrue2022-09-03T05:41:04.203Zrgw-multisite-test-userMultisite Test UserNormalobjhsp-uXpwzRjamHyM57a9u6NdISYw8SRfalse2022-09-03T05:41:04.168Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:40.251 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:40.252 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:40.253 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.254 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.254 DEBUG:boto:path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.255 DEBUG:boto:auth_path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.255 DEBUG:boto:Method: GET 2022-09-03T05:41:40.256 DEBUG:boto:Path: /dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.257 DEBUG:boto:Data: 2022-09-03T05:41:40.257 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.259 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:40.260 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.260 DEBUG:boto:Params: {} 2022-09-03T05:41:40.262 DEBUG:boto:Token: None 2022-09-03T05:41:40.262 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/?versions 2022-09-03T05:41:40.263 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s= 2022-09-03T05:41:40.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s=', 'Content-Length': '0'} 2022-09-03T05:41:40.266 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000033732c17cedbbf02-006312e914-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.267 DEBUG:boto:b'dsgvcy-21999falseobjWOsbc4Am7smafmwB2ISQ9XrKfB7fKjgtrue2022-09-03T05:41:04.203Zrgw-multisite-test-userMultisite Test UserNormalobjhsp-uXpwzRjamHyM57a9u6NdISYw8SRfalse2022-09-03T05:41:04.168Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:40.268 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.268 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.269 DEBUG:boto:path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.269 DEBUG:boto:auth_path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.270 DEBUG:boto:Method: GET 2022-09-03T05:41:40.270 DEBUG:boto:Path: /dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.271 DEBUG:boto:Data: 2022-09-03T05:41:40.271 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.272 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:40.272 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.273 DEBUG:boto:Params: {} 2022-09-03T05:41:40.273 DEBUG:boto:Token: None 2022-09-03T05:41:40.274 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/?versions 2022-09-03T05:41:40.275 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s= 2022-09-03T05:41:40.277 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s=', 'Content-Length': '0'} 2022-09-03T05:41:40.279 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009c08e8f129c955e7-006312e914-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.280 DEBUG:boto:b'dsgvcy-21999falseobjWOsbc4Am7smafmwB2ISQ9XrKfB7fKjgtrue2022-09-03T05:41:04.203Zrgw-multisite-test-userMultisite Test UserNormalobjhsp-uXpwzRjamHyM57a9u6NdISYw8SRfalse2022-09-03T05:41:04.168Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:40.282 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:40.282 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.283 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.283 DEBUG:boto:Method: HEAD 2022-09-03T05:41:40.284 DEBUG:boto:Path: /dsgvcy-21/obj 2022-09-03T05:41:40.284 DEBUG:boto:Data: 2022-09-03T05:41:40.285 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.285 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:40.286 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.286 DEBUG:boto:Params: {} 2022-09-03T05:41:40.287 DEBUG:boto:Token: None 2022-09-03T05:41:40.288 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj 2022-09-03T05:41:40.288 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bFVeKM0LNS5OGWK/y9rnXD/FVIU= 2022-09-03T05:41:40.289 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bFVeKM0LNS5OGWK/y9rnXD/FVIU=', 'Content-Length': '0'} 2022-09-03T05:41:40.291 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx0000028cbafde05e01d50-006312e914-10ea-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.292 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.292 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.293 DEBUG:boto:Method: HEAD 2022-09-03T05:41:40.294 DEBUG:boto:Path: /dsgvcy-21/obj 2022-09-03T05:41:40.294 DEBUG:boto:Data: 2022-09-03T05:41:40.295 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.295 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:40.296 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.296 DEBUG:boto:Params: {} 2022-09-03T05:41:40.297 DEBUG:boto:Token: None 2022-09-03T05:41:40.297 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj 2022-09-03T05:41:40.298 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bFVeKM0LNS5OGWK/y9rnXD/FVIU= 2022-09-03T05:41:40.299 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bFVeKM0LNS5OGWK/y9rnXD/FVIU=', 'Content-Length': '0'} 2022-09-03T05:41:40.303 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx000009dd8c6c20e61692d-006312e914-10ff-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.304 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:40.305 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.305 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.306 DEBUG:boto:path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.307 DEBUG:boto:auth_path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.307 DEBUG:boto:Method: GET 2022-09-03T05:41:40.308 DEBUG:boto:Path: /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.308 DEBUG:boto:Data: 2022-09-03T05:41:40.309 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.309 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:40.310 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.310 DEBUG:boto:Params: {} 2022-09-03T05:41:40.311 DEBUG:boto:Token: None 2022-09-03T05:41:40.312 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.312 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs= 2022-09-03T05:41:40.313 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs=', 'Content-Length': '0'} 2022-09-03T05:41:40.315 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'hsp-uXpwzRjamHyM57a9u6NdISYw8SR'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000fce32cd6d9a0d0ae-006312e914-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.316 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.317 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.317 DEBUG:boto:path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.318 DEBUG:boto:auth_path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.318 DEBUG:boto:Method: GET 2022-09-03T05:41:40.319 DEBUG:boto:Path: /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.319 DEBUG:boto:Data: 2022-09-03T05:41:40.320 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.320 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:40.321 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.322 DEBUG:boto:Params: {} 2022-09-03T05:41:40.322 DEBUG:boto:Token: None 2022-09-03T05:41:40.323 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.324 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs= 2022-09-03T05:41:40.326 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs=', 'Content-Length': '0'} 2022-09-03T05:41:40.329 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'hsp-uXpwzRjamHyM57a9u6NdISYw8SR'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000001b3a1a269a2de1da-006312e914-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.330 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.331 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.332 DEBUG:boto:path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.332 DEBUG:boto:auth_path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.333 DEBUG:boto:Method: HEAD 2022-09-03T05:41:40.333 DEBUG:boto:Path: /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.334 DEBUG:boto:Data: 2022-09-03T05:41:40.334 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.335 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:40.336 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.337 DEBUG:boto:Params: {} 2022-09-03T05:41:40.338 DEBUG:boto:Token: None 2022-09-03T05:41:40.339 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.339 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:T+3vs9GWiJ5zbA4MYFNDDH3ok7c= 2022-09-03T05:41:40.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:T+3vs9GWiJ5zbA4MYFNDDH3ok7c=', 'Content-Length': '0'} 2022-09-03T05:41:40.342 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'hsp-uXpwzRjamHyM57a9u6NdISYw8SR'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000053610cc2c3985f6-006312e914-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.343 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.344 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.345 DEBUG:boto:path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.346 DEBUG:boto:auth_path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.347 DEBUG:boto:Method: HEAD 2022-09-03T05:41:40.348 DEBUG:boto:Path: /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.349 DEBUG:boto:Data: 2022-09-03T05:41:40.350 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.351 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:40.351 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.352 DEBUG:boto:Params: {} 2022-09-03T05:41:40.353 DEBUG:boto:Token: None 2022-09-03T05:41:40.354 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.355 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:T+3vs9GWiJ5zbA4MYFNDDH3ok7c= 2022-09-03T05:41:40.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:T+3vs9GWiJ5zbA4MYFNDDH3ok7c=', 'Content-Length': '0'} 2022-09-03T05:41:40.358 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'hsp-uXpwzRjamHyM57a9u6NdISYw8SR'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000004f4f3a5277c06b4b-006312e914-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.360 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:40.360 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.361 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.361 DEBUG:boto:path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.362 DEBUG:boto:auth_path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.363 DEBUG:boto:Method: GET 2022-09-03T05:41:40.363 DEBUG:boto:Path: /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.364 DEBUG:boto:Data: 2022-09-03T05:41:40.364 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.365 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:40.366 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.366 DEBUG:boto:Params: {} 2022-09-03T05:41:40.367 DEBUG:boto:Token: None 2022-09-03T05:41:40.368 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.368 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs= 2022-09-03T05:41:40.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs=', 'Content-Length': '0'} 2022-09-03T05:41:40.371 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'hsp-uXpwzRjamHyM57a9u6NdISYw8SR'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000006b160376b82459d6-006312e914-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.372 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.372 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.373 DEBUG:boto:path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.373 DEBUG:boto:auth_path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.374 DEBUG:boto:Method: GET 2022-09-03T05:41:40.375 DEBUG:boto:Path: /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.375 DEBUG:boto:Data: 2022-09-03T05:41:40.376 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.376 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:40.377 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.377 DEBUG:boto:Params: {} 2022-09-03T05:41:40.378 DEBUG:boto:Token: None 2022-09-03T05:41:40.378 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.379 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs= 2022-09-03T05:41:40.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs=', 'Content-Length': '0'} 2022-09-03T05:41:40.382 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'hsp-uXpwzRjamHyM57a9u6NdISYw8SR'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000025b56218b7a18e8e-006312e914-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.382 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.383 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.384 DEBUG:boto:path=/dsgvcy-21/?prefix=obj 2022-09-03T05:41:40.384 DEBUG:boto:auth_path=/dsgvcy-21/?prefix=obj 2022-09-03T05:41:40.385 DEBUG:boto:Method: GET 2022-09-03T05:41:40.385 DEBUG:boto:Path: /dsgvcy-21/?prefix=obj 2022-09-03T05:41:40.386 DEBUG:boto:Data: 2022-09-03T05:41:40.386 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.387 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:40.387 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.388 DEBUG:boto:Params: {} 2022-09-03T05:41:40.389 DEBUG:boto:Token: None 2022-09-03T05:41:40.389 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/ 2022-09-03T05:41:40.390 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TXmg0PIrCLVq1qCya00o2gaEeEY= 2022-09-03T05:41:40.391 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TXmg0PIrCLVq1qCya00o2gaEeEY=', 'Content-Length': '0'} 2022-09-03T05:41:40.393 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d65131d1fffa4906-006312e914-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.438 DEBUG:boto:b'dsgvcy-21obj1000false' 2022-09-03T05:41:40.439 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.440 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.440 DEBUG:boto:path=/dsgvcy-21/?prefix=obj 2022-09-03T05:41:40.441 DEBUG:boto:auth_path=/dsgvcy-21/?prefix=obj 2022-09-03T05:41:40.441 DEBUG:boto:Method: GET 2022-09-03T05:41:40.442 DEBUG:boto:Path: /dsgvcy-21/?prefix=obj 2022-09-03T05:41:40.442 DEBUG:boto:Data: 2022-09-03T05:41:40.443 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.443 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:40.444 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.444 DEBUG:boto:Params: {} 2022-09-03T05:41:40.445 DEBUG:boto:Token: None 2022-09-03T05:41:40.446 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/ 2022-09-03T05:41:40.446 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TXmg0PIrCLVq1qCya00o2gaEeEY= 2022-09-03T05:41:40.447 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TXmg0PIrCLVq1qCya00o2gaEeEY=', 'Content-Length': '0'} 2022-09-03T05:41:40.450 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d3c144a41c012d19-006312e914-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.452 DEBUG:boto:b'dsgvcy-21obj1000false' 2022-09-03T05:41:40.453 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.455 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.456 DEBUG:boto:Method: HEAD 2022-09-03T05:41:40.457 DEBUG:boto:Path: /dsgvcy-21/obj 2022-09-03T05:41:40.457 DEBUG:boto:Data: 2022-09-03T05:41:40.458 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.458 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:40.459 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.460 DEBUG:boto:Params: {} 2022-09-03T05:41:40.461 DEBUG:boto:Token: None 2022-09-03T05:41:40.462 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj 2022-09-03T05:41:40.463 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bFVeKM0LNS5OGWK/y9rnXD/FVIU= 2022-09-03T05:41:40.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bFVeKM0LNS5OGWK/y9rnXD/FVIU=', 'Content-Length': '0'} 2022-09-03T05:41:40.467 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx00000b5c54ed10260830d-006312e914-10ea-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.468 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.469 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.469 DEBUG:boto:Method: HEAD 2022-09-03T05:41:40.470 DEBUG:boto:Path: /dsgvcy-21/obj 2022-09-03T05:41:40.471 DEBUG:boto:Data: 2022-09-03T05:41:40.471 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.472 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:40.472 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.473 DEBUG:boto:Params: {} 2022-09-03T05:41:40.473 DEBUG:boto:Token: None 2022-09-03T05:41:40.474 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj 2022-09-03T05:41:40.475 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bFVeKM0LNS5OGWK/y9rnXD/FVIU= 2022-09-03T05:41:40.475 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bFVeKM0LNS5OGWK/y9rnXD/FVIU=', 'Content-Length': '0'} 2022-09-03T05:41:40.478 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx000005fde57bb7d2bb7a1-006312e914-10ff-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.479 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-21 zones={test-zone2, test-zone1} 2022-09-03T05:41:40.479 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-21 zones={test-zone3, test-zone1} 2022-09-03T05:41:40.480 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.480 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.481 DEBUG:boto:Method: HEAD 2022-09-03T05:41:40.481 DEBUG:boto:Path: /dsgvcy-21/ 2022-09-03T05:41:40.482 DEBUG:boto:Data: 2022-09-03T05:41:40.482 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.483 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:40.484 DEBUG:boto:Port: 8001 2022-09-03T05:41:40.484 DEBUG:boto:Params: {} 2022-09-03T05:41:40.485 DEBUG:boto:Token: None 2022-09-03T05:41:40.485 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/ 2022-09-03T05:41:40.486 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a1RAUyKYZgLVrJQjGMW5xjayvEY= 2022-09-03T05:41:40.487 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a1RAUyKYZgLVrJQjGMW5xjayvEY=', 'Content-Length': '0'} 2022-09-03T05:41:40.493 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008246c1faddb99314-006312e914-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.494 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.495 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.496 DEBUG:boto:Method: HEAD 2022-09-03T05:41:40.496 DEBUG:boto:Path: /dsgvcy-21/ 2022-09-03T05:41:40.497 DEBUG:boto:Data: 2022-09-03T05:41:40.497 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.498 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:40.498 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.499 DEBUG:boto:Params: {} 2022-09-03T05:41:40.500 DEBUG:boto:Token: None 2022-09-03T05:41:40.500 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/ 2022-09-03T05:41:40.501 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a1RAUyKYZgLVrJQjGMW5xjayvEY= 2022-09-03T05:41:40.502 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a1RAUyKYZgLVrJQjGMW5xjayvEY=', 'Content-Length': '0'} 2022-09-03T05:41:40.504 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c172b31a1cd16a6e-006312e914-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.505 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:41:40.505 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.506 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.506 DEBUG:boto:path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.507 DEBUG:boto:auth_path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.507 DEBUG:boto:Method: GET 2022-09-03T05:41:40.508 DEBUG:boto:Path: /dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.508 DEBUG:boto:Data: 2022-09-03T05:41:40.509 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.510 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:40.510 DEBUG:boto:Port: 8001 2022-09-03T05:41:40.511 DEBUG:boto:Params: {} 2022-09-03T05:41:40.511 DEBUG:boto:Token: None 2022-09-03T05:41:40.512 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/?versions 2022-09-03T05:41:40.513 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s= 2022-09-03T05:41:40.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s=', 'Content-Length': '0'} 2022-09-03T05:41:40.516 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a3e4feea8ea66496-006312e914-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.562 DEBUG:boto:b'dsgvcy-21999falseobjWOsbc4Am7smafmwB2ISQ9XrKfB7fKjgtrue2022-09-03T05:41:04.203Zrgw-multisite-test-userMultisite Test UserNormalobjhsp-uXpwzRjamHyM57a9u6NdISYw8SRfalse2022-09-03T05:41:04.168Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:40.563 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:40.564 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:40.564 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:41:40.565 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.566 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.566 DEBUG:boto:path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.567 DEBUG:boto:auth_path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.568 DEBUG:boto:Method: GET 2022-09-03T05:41:40.569 DEBUG:boto:Path: /dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.570 DEBUG:boto:Data: 2022-09-03T05:41:40.571 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.572 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:40.573 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.573 DEBUG:boto:Params: {} 2022-09-03T05:41:40.575 DEBUG:boto:Token: None 2022-09-03T05:41:40.575 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/?versions 2022-09-03T05:41:40.576 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s= 2022-09-03T05:41:40.577 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s=', 'Content-Length': '0'} 2022-09-03T05:41:40.580 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a7aad537627fc1b7-006312e914-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.581 DEBUG:boto:b'dsgvcy-21999falseobjWOsbc4Am7smafmwB2ISQ9XrKfB7fKjgtrue2022-09-03T05:41:04.203Zrgw-multisite-test-userMultisite Test UserNormalobjhsp-uXpwzRjamHyM57a9u6NdISYw8SRfalse2022-09-03T05:41:04.168Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:40.582 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:40.583 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:40.583 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.584 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.584 DEBUG:boto:path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.585 DEBUG:boto:auth_path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.585 DEBUG:boto:Method: GET 2022-09-03T05:41:40.586 DEBUG:boto:Path: /dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.587 DEBUG:boto:Data: 2022-09-03T05:41:40.587 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.588 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:40.588 DEBUG:boto:Port: 8001 2022-09-03T05:41:40.589 DEBUG:boto:Params: {} 2022-09-03T05:41:40.589 DEBUG:boto:Token: None 2022-09-03T05:41:40.590 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/?versions 2022-09-03T05:41:40.591 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s= 2022-09-03T05:41:40.592 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s=', 'Content-Length': '0'} 2022-09-03T05:41:40.596 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006001a7c3995533bb-006312e914-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.597 DEBUG:boto:b'dsgvcy-21999falseobjWOsbc4Am7smafmwB2ISQ9XrKfB7fKjgtrue2022-09-03T05:41:04.203Zrgw-multisite-test-userMultisite Test UserNormalobjhsp-uXpwzRjamHyM57a9u6NdISYw8SRfalse2022-09-03T05:41:04.168Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:40.599 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.600 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.601 DEBUG:boto:path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.602 DEBUG:boto:auth_path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.603 DEBUG:boto:Method: GET 2022-09-03T05:41:40.604 DEBUG:boto:Path: /dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.605 DEBUG:boto:Data: 2022-09-03T05:41:40.606 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.607 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:40.608 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.609 DEBUG:boto:Params: {} 2022-09-03T05:41:40.610 DEBUG:boto:Token: None 2022-09-03T05:41:40.611 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/?versions 2022-09-03T05:41:40.612 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s= 2022-09-03T05:41:40.613 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s=', 'Content-Length': '0'} 2022-09-03T05:41:40.615 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000026fccbe32168c3e7-006312e914-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.616 DEBUG:boto:b'dsgvcy-21999falseobjWOsbc4Am7smafmwB2ISQ9XrKfB7fKjgtrue2022-09-03T05:41:04.203Zrgw-multisite-test-userMultisite Test UserNormalobjhsp-uXpwzRjamHyM57a9u6NdISYw8SRfalse2022-09-03T05:41:04.168Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:40.617 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:40.617 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.618 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.618 DEBUG:boto:Method: HEAD 2022-09-03T05:41:40.619 DEBUG:boto:Path: /dsgvcy-21/obj 2022-09-03T05:41:40.619 DEBUG:boto:Data: 2022-09-03T05:41:40.620 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.620 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:40.621 DEBUG:boto:Port: 8001 2022-09-03T05:41:40.621 DEBUG:boto:Params: {} 2022-09-03T05:41:40.622 DEBUG:boto:Token: None 2022-09-03T05:41:40.623 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj 2022-09-03T05:41:40.623 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bFVeKM0LNS5OGWK/y9rnXD/FVIU= 2022-09-03T05:41:40.624 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bFVeKM0LNS5OGWK/y9rnXD/FVIU=', 'Content-Length': '0'} 2022-09-03T05:41:40.626 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx000004cd725afce7b4a4a-006312e914-1103-test-zone3'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.627 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.628 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.628 DEBUG:boto:Method: HEAD 2022-09-03T05:41:40.629 DEBUG:boto:Path: /dsgvcy-21/obj 2022-09-03T05:41:40.629 DEBUG:boto:Data: 2022-09-03T05:41:40.630 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.631 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:40.632 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.633 DEBUG:boto:Params: {} 2022-09-03T05:41:40.633 DEBUG:boto:Token: None 2022-09-03T05:41:40.634 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj 2022-09-03T05:41:40.635 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bFVeKM0LNS5OGWK/y9rnXD/FVIU= 2022-09-03T05:41:40.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bFVeKM0LNS5OGWK/y9rnXD/FVIU=', 'Content-Length': '0'} 2022-09-03T05:41:40.639 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx00000201ef5a5a0a75695-006312e914-10ff-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.640 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:40.641 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.642 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.643 DEBUG:boto:path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.644 DEBUG:boto:auth_path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.645 DEBUG:boto:Method: GET 2022-09-03T05:41:40.646 DEBUG:boto:Path: /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.647 DEBUG:boto:Data: 2022-09-03T05:41:40.648 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.649 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:40.650 DEBUG:boto:Port: 8001 2022-09-03T05:41:40.651 DEBUG:boto:Params: {} 2022-09-03T05:41:40.652 DEBUG:boto:Token: None 2022-09-03T05:41:40.653 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.654 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs= 2022-09-03T05:41:40.654 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs=', 'Content-Length': '0'} 2022-09-03T05:41:40.657 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'hsp-uXpwzRjamHyM57a9u6NdISYw8SR'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000c521c80cc52593ce-006312e914-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.657 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.658 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.659 DEBUG:boto:path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.659 DEBUG:boto:auth_path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.660 DEBUG:boto:Method: GET 2022-09-03T05:41:40.660 DEBUG:boto:Path: /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.661 DEBUG:boto:Data: 2022-09-03T05:41:40.661 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.662 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:40.663 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.663 DEBUG:boto:Params: {} 2022-09-03T05:41:40.664 DEBUG:boto:Token: None 2022-09-03T05:41:40.665 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.666 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs= 2022-09-03T05:41:40.667 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs=', 'Content-Length': '0'} 2022-09-03T05:41:40.669 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'hsp-uXpwzRjamHyM57a9u6NdISYw8SR'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000001e9d0ccb36f4439-006312e914-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.671 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.672 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.673 DEBUG:boto:path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.674 DEBUG:boto:auth_path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.675 DEBUG:boto:Method: HEAD 2022-09-03T05:41:40.675 DEBUG:boto:Path: /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.676 DEBUG:boto:Data: 2022-09-03T05:41:40.677 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.678 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:40.679 DEBUG:boto:Port: 8001 2022-09-03T05:41:40.680 DEBUG:boto:Params: {} 2022-09-03T05:41:40.681 DEBUG:boto:Token: None 2022-09-03T05:41:40.682 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.683 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:T+3vs9GWiJ5zbA4MYFNDDH3ok7c= 2022-09-03T05:41:40.683 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:T+3vs9GWiJ5zbA4MYFNDDH3ok7c=', 'Content-Length': '0'} 2022-09-03T05:41:40.686 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'hsp-uXpwzRjamHyM57a9u6NdISYw8SR'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000007940184bd8f7ae95-006312e914-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.687 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.688 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.688 DEBUG:boto:path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.689 DEBUG:boto:auth_path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.690 DEBUG:boto:Method: HEAD 2022-09-03T05:41:40.691 DEBUG:boto:Path: /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.691 DEBUG:boto:Data: 2022-09-03T05:41:40.692 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.692 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:40.693 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.693 DEBUG:boto:Params: {} 2022-09-03T05:41:40.694 DEBUG:boto:Token: None 2022-09-03T05:41:40.695 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.695 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:T+3vs9GWiJ5zbA4MYFNDDH3ok7c= 2022-09-03T05:41:40.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:T+3vs9GWiJ5zbA4MYFNDDH3ok7c=', 'Content-Length': '0'} 2022-09-03T05:41:40.698 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'hsp-uXpwzRjamHyM57a9u6NdISYw8SR'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000989c7f34b2574c67-006312e914-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.699 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:40.700 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.701 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.702 DEBUG:boto:path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.703 DEBUG:boto:auth_path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.704 DEBUG:boto:Method: GET 2022-09-03T05:41:40.704 DEBUG:boto:Path: /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.705 DEBUG:boto:Data: 2022-09-03T05:41:40.706 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.707 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:40.707 DEBUG:boto:Port: 8001 2022-09-03T05:41:40.708 DEBUG:boto:Params: {} 2022-09-03T05:41:40.709 DEBUG:boto:Token: None 2022-09-03T05:41:40.710 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.711 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs= 2022-09-03T05:41:40.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs=', 'Content-Length': '0'} 2022-09-03T05:41:40.714 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'hsp-uXpwzRjamHyM57a9u6NdISYw8SR'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000008000f99280006ab0-006312e914-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.715 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.716 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.717 DEBUG:boto:path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.718 DEBUG:boto:auth_path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.719 DEBUG:boto:Method: GET 2022-09-03T05:41:40.720 DEBUG:boto:Path: /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.721 DEBUG:boto:Data: 2022-09-03T05:41:40.722 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.723 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:40.724 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.725 DEBUG:boto:Params: {} 2022-09-03T05:41:40.726 DEBUG:boto:Token: None 2022-09-03T05:41:40.728 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.729 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs= 2022-09-03T05:41:40.730 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs=', 'Content-Length': '0'} 2022-09-03T05:41:40.732 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'hsp-uXpwzRjamHyM57a9u6NdISYw8SR'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000001397448691caf81f-006312e914-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.733 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.734 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.735 DEBUG:boto:path=/dsgvcy-21/?prefix=obj 2022-09-03T05:41:40.735 DEBUG:boto:auth_path=/dsgvcy-21/?prefix=obj 2022-09-03T05:41:40.736 DEBUG:boto:Method: GET 2022-09-03T05:41:40.736 DEBUG:boto:Path: /dsgvcy-21/?prefix=obj 2022-09-03T05:41:40.737 DEBUG:boto:Data: 2022-09-03T05:41:40.737 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.738 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:40.738 DEBUG:boto:Port: 8001 2022-09-03T05:41:40.739 DEBUG:boto:Params: {} 2022-09-03T05:41:40.739 DEBUG:boto:Token: None 2022-09-03T05:41:40.740 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/ 2022-09-03T05:41:40.741 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TXmg0PIrCLVq1qCya00o2gaEeEY= 2022-09-03T05:41:40.741 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TXmg0PIrCLVq1qCya00o2gaEeEY=', 'Content-Length': '0'} 2022-09-03T05:41:40.743 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000293edbf316b99ad8-006312e914-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.744 DEBUG:boto:b'dsgvcy-21obj1000false' 2022-09-03T05:41:40.745 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.746 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.746 DEBUG:boto:path=/dsgvcy-21/?prefix=obj 2022-09-03T05:41:40.747 DEBUG:boto:auth_path=/dsgvcy-21/?prefix=obj 2022-09-03T05:41:40.747 DEBUG:boto:Method: GET 2022-09-03T05:41:40.748 DEBUG:boto:Path: /dsgvcy-21/?prefix=obj 2022-09-03T05:41:40.748 DEBUG:boto:Data: 2022-09-03T05:41:40.749 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.749 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:40.750 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.750 DEBUG:boto:Params: {} 2022-09-03T05:41:40.751 DEBUG:boto:Token: None 2022-09-03T05:41:40.752 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/ 2022-09-03T05:41:40.752 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TXmg0PIrCLVq1qCya00o2gaEeEY= 2022-09-03T05:41:40.753 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TXmg0PIrCLVq1qCya00o2gaEeEY=', 'Content-Length': '0'} 2022-09-03T05:41:40.755 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c1f21daff530baa7-006312e914-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.756 DEBUG:boto:b'dsgvcy-21obj1000false' 2022-09-03T05:41:40.757 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.757 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.758 DEBUG:boto:Method: HEAD 2022-09-03T05:41:40.759 DEBUG:boto:Path: /dsgvcy-21/obj 2022-09-03T05:41:40.759 DEBUG:boto:Data: 2022-09-03T05:41:40.760 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.760 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:40.761 DEBUG:boto:Port: 8001 2022-09-03T05:41:40.761 DEBUG:boto:Params: {} 2022-09-03T05:41:40.762 DEBUG:boto:Token: None 2022-09-03T05:41:40.762 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj 2022-09-03T05:41:40.763 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bFVeKM0LNS5OGWK/y9rnXD/FVIU= 2022-09-03T05:41:40.763 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bFVeKM0LNS5OGWK/y9rnXD/FVIU=', 'Content-Length': '0'} 2022-09-03T05:41:40.765 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx000005e22c1e4ff45a259-006312e914-1103-test-zone3'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.766 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.767 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.767 DEBUG:boto:Method: HEAD 2022-09-03T05:41:40.768 DEBUG:boto:Path: /dsgvcy-21/obj 2022-09-03T05:41:40.768 DEBUG:boto:Data: 2022-09-03T05:41:40.769 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.769 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:40.770 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.770 DEBUG:boto:Params: {} 2022-09-03T05:41:40.771 DEBUG:boto:Token: None 2022-09-03T05:41:40.771 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj 2022-09-03T05:41:40.772 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bFVeKM0LNS5OGWK/y9rnXD/FVIU= 2022-09-03T05:41:40.773 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bFVeKM0LNS5OGWK/y9rnXD/FVIU=', 'Content-Length': '0'} 2022-09-03T05:41:40.775 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx00000d378734643d1cf48-006312e914-10ff-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.775 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-21 zones={test-zone3, test-zone1} 2022-09-03T05:41:40.776 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-21 zones={test-zone3, test-zone2} 2022-09-03T05:41:40.776 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.777 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.777 DEBUG:boto:Method: HEAD 2022-09-03T05:41:40.778 DEBUG:boto:Path: /dsgvcy-21/ 2022-09-03T05:41:40.779 DEBUG:boto:Data: 2022-09-03T05:41:40.779 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.780 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:40.780 DEBUG:boto:Port: 8001 2022-09-03T05:41:40.781 DEBUG:boto:Params: {} 2022-09-03T05:41:40.781 DEBUG:boto:Token: None 2022-09-03T05:41:40.782 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/ 2022-09-03T05:41:40.782 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a1RAUyKYZgLVrJQjGMW5xjayvEY= 2022-09-03T05:41:40.783 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a1RAUyKYZgLVrJQjGMW5xjayvEY=', 'Content-Length': '0'} 2022-09-03T05:41:40.785 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004e0073f547793a6c-006312e914-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.786 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.787 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.787 DEBUG:boto:Method: HEAD 2022-09-03T05:41:40.788 DEBUG:boto:Path: /dsgvcy-21/ 2022-09-03T05:41:40.788 DEBUG:boto:Data: 2022-09-03T05:41:40.789 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.789 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:40.790 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.790 DEBUG:boto:Params: {} 2022-09-03T05:41:40.791 DEBUG:boto:Token: None 2022-09-03T05:41:40.791 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/ 2022-09-03T05:41:40.792 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a1RAUyKYZgLVrJQjGMW5xjayvEY= 2022-09-03T05:41:40.793 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a1RAUyKYZgLVrJQjGMW5xjayvEY=', 'Content-Length': '0'} 2022-09-03T05:41:40.795 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ce9c0b3a12dd512e-006312e914-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.796 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:41:40.796 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.797 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.797 DEBUG:boto:path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.798 DEBUG:boto:auth_path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.798 DEBUG:boto:Method: GET 2022-09-03T05:41:40.799 DEBUG:boto:Path: /dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.799 DEBUG:boto:Data: 2022-09-03T05:41:40.800 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.800 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:40.801 DEBUG:boto:Port: 8001 2022-09-03T05:41:40.801 DEBUG:boto:Params: {} 2022-09-03T05:41:40.802 DEBUG:boto:Token: None 2022-09-03T05:41:40.803 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/?versions 2022-09-03T05:41:40.803 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s= 2022-09-03T05:41:40.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s=', 'Content-Length': '0'} 2022-09-03T05:41:40.806 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f2cfe83a13f299f6-006312e914-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.808 DEBUG:boto:b'dsgvcy-21999falseobjWOsbc4Am7smafmwB2ISQ9XrKfB7fKjgtrue2022-09-03T05:41:04.203Zrgw-multisite-test-userMultisite Test UserNormalobjhsp-uXpwzRjamHyM57a9u6NdISYw8SRfalse2022-09-03T05:41:04.168Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:40.809 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:40.810 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:40.811 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:41:40.812 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.812 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.813 DEBUG:boto:path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.813 DEBUG:boto:auth_path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.814 DEBUG:boto:Method: GET 2022-09-03T05:41:40.814 DEBUG:boto:Path: /dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.815 DEBUG:boto:Data: 2022-09-03T05:41:40.815 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.816 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:40.817 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.817 DEBUG:boto:Params: {} 2022-09-03T05:41:40.819 DEBUG:boto:Token: None 2022-09-03T05:41:40.819 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/?versions 2022-09-03T05:41:40.820 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s= 2022-09-03T05:41:40.821 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s=', 'Content-Length': '0'} 2022-09-03T05:41:40.823 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f03e20a33caca0ad-006312e914-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.824 DEBUG:boto:b'dsgvcy-21999falseobjWOsbc4Am7smafmwB2ISQ9XrKfB7fKjgtrue2022-09-03T05:41:04.203Zrgw-multisite-test-userMultisite Test UserNormalobjhsp-uXpwzRjamHyM57a9u6NdISYw8SRfalse2022-09-03T05:41:04.168Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:40.825 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:40.825 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:40.826 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.826 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.827 DEBUG:boto:path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.828 DEBUG:boto:auth_path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.828 DEBUG:boto:Method: GET 2022-09-03T05:41:40.829 DEBUG:boto:Path: /dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.829 DEBUG:boto:Data: 2022-09-03T05:41:40.830 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.830 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:40.831 DEBUG:boto:Port: 8001 2022-09-03T05:41:40.831 DEBUG:boto:Params: {} 2022-09-03T05:41:40.832 DEBUG:boto:Token: None 2022-09-03T05:41:40.832 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/?versions 2022-09-03T05:41:40.833 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s= 2022-09-03T05:41:40.834 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s=', 'Content-Length': '0'} 2022-09-03T05:41:40.836 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000795ee487a4616c0d-006312e914-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.878 DEBUG:boto:b'dsgvcy-21999falseobjWOsbc4Am7smafmwB2ISQ9XrKfB7fKjgtrue2022-09-03T05:41:04.203Zrgw-multisite-test-userMultisite Test UserNormalobjhsp-uXpwzRjamHyM57a9u6NdISYw8SRfalse2022-09-03T05:41:04.168Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:40.879 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.880 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.880 DEBUG:boto:path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.881 DEBUG:boto:auth_path=/dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.882 DEBUG:boto:Method: GET 2022-09-03T05:41:40.882 DEBUG:boto:Path: /dsgvcy-21/?versions&max-keys=999 2022-09-03T05:41:40.883 DEBUG:boto:Data: 2022-09-03T05:41:40.884 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.884 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:40.885 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.885 DEBUG:boto:Params: {} 2022-09-03T05:41:40.886 DEBUG:boto:Token: None 2022-09-03T05:41:40.887 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/?versions 2022-09-03T05:41:40.887 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s= 2022-09-03T05:41:40.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1KEx3jbAPvXTRtvmz3fweNsFp4s=', 'Content-Length': '0'} 2022-09-03T05:41:40.890 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000014814d2197d8c6dd-006312e914-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.891 DEBUG:boto:b'dsgvcy-21999falseobjWOsbc4Am7smafmwB2ISQ9XrKfB7fKjgtrue2022-09-03T05:41:04.203Zrgw-multisite-test-userMultisite Test UserNormalobjhsp-uXpwzRjamHyM57a9u6NdISYw8SRfalse2022-09-03T05:41:04.168Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:40.892 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:40.892 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.893 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.893 DEBUG:boto:Method: HEAD 2022-09-03T05:41:40.894 DEBUG:boto:Path: /dsgvcy-21/obj 2022-09-03T05:41:40.895 DEBUG:boto:Data: 2022-09-03T05:41:40.895 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.896 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:40.896 DEBUG:boto:Port: 8001 2022-09-03T05:41:40.896 DEBUG:boto:Params: {} 2022-09-03T05:41:40.897 DEBUG:boto:Token: None 2022-09-03T05:41:40.898 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj 2022-09-03T05:41:40.898 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bFVeKM0LNS5OGWK/y9rnXD/FVIU= 2022-09-03T05:41:40.899 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bFVeKM0LNS5OGWK/y9rnXD/FVIU=', 'Content-Length': '0'} 2022-09-03T05:41:40.901 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx000006cdf9999e43dec6f-006312e914-1103-test-zone3'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.902 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.903 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.903 DEBUG:boto:Method: HEAD 2022-09-03T05:41:40.904 DEBUG:boto:Path: /dsgvcy-21/obj 2022-09-03T05:41:40.904 DEBUG:boto:Data: 2022-09-03T05:41:40.905 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.905 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:40.906 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.906 DEBUG:boto:Params: {} 2022-09-03T05:41:40.907 DEBUG:boto:Token: None 2022-09-03T05:41:40.908 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj 2022-09-03T05:41:40.908 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bFVeKM0LNS5OGWK/y9rnXD/FVIU= 2022-09-03T05:41:40.909 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bFVeKM0LNS5OGWK/y9rnXD/FVIU=', 'Content-Length': '0'} 2022-09-03T05:41:40.912 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx00000ee8ce8a68fcc3a74-006312e914-10ea-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.912 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:40.913 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.914 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.914 DEBUG:boto:path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.915 DEBUG:boto:auth_path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.916 DEBUG:boto:Method: GET 2022-09-03T05:41:40.916 DEBUG:boto:Path: /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.917 DEBUG:boto:Data: 2022-09-03T05:41:40.917 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.918 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:40.919 DEBUG:boto:Port: 8001 2022-09-03T05:41:40.919 DEBUG:boto:Params: {} 2022-09-03T05:41:40.920 DEBUG:boto:Token: None 2022-09-03T05:41:40.920 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.921 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs= 2022-09-03T05:41:40.922 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs=', 'Content-Length': '0'} 2022-09-03T05:41:40.924 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'hsp-uXpwzRjamHyM57a9u6NdISYw8SR'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a9a5d433e588ae3c-006312e914-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.924 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.925 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.926 DEBUG:boto:path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.927 DEBUG:boto:auth_path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.927 DEBUG:boto:Method: GET 2022-09-03T05:41:40.928 DEBUG:boto:Path: /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.929 DEBUG:boto:Data: 2022-09-03T05:41:40.929 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.930 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:40.931 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.931 DEBUG:boto:Params: {} 2022-09-03T05:41:40.932 DEBUG:boto:Token: None 2022-09-03T05:41:40.932 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.933 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs= 2022-09-03T05:41:40.934 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs=', 'Content-Length': '0'} 2022-09-03T05:41:40.935 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'hsp-uXpwzRjamHyM57a9u6NdISYw8SR'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000029f362a5abd5e2aa-006312e914-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.936 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.937 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.938 DEBUG:boto:path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.938 DEBUG:boto:auth_path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.939 DEBUG:boto:Method: HEAD 2022-09-03T05:41:40.939 DEBUG:boto:Path: /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.940 DEBUG:boto:Data: 2022-09-03T05:41:40.940 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.941 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:40.941 DEBUG:boto:Port: 8001 2022-09-03T05:41:40.942 DEBUG:boto:Params: {} 2022-09-03T05:41:40.943 DEBUG:boto:Token: None 2022-09-03T05:41:40.943 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.944 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:T+3vs9GWiJ5zbA4MYFNDDH3ok7c= 2022-09-03T05:41:40.945 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:T+3vs9GWiJ5zbA4MYFNDDH3ok7c=', 'Content-Length': '0'} 2022-09-03T05:41:40.946 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'hsp-uXpwzRjamHyM57a9u6NdISYw8SR'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f3fce5bef0aef29d-006312e914-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.947 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.948 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.949 DEBUG:boto:path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.950 DEBUG:boto:auth_path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.950 DEBUG:boto:Method: HEAD 2022-09-03T05:41:40.951 DEBUG:boto:Path: /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.952 DEBUG:boto:Data: 2022-09-03T05:41:40.952 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.953 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:40.953 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.954 DEBUG:boto:Params: {} 2022-09-03T05:41:40.954 DEBUG:boto:Token: None 2022-09-03T05:41:40.955 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.956 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:T+3vs9GWiJ5zbA4MYFNDDH3ok7c= 2022-09-03T05:41:40.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:T+3vs9GWiJ5zbA4MYFNDDH3ok7c=', 'Content-Length': '0'} 2022-09-03T05:41:40.959 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'hsp-uXpwzRjamHyM57a9u6NdISYw8SR'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000006bc6bb35d9407667-006312e914-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.960 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:40.961 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.962 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.963 DEBUG:boto:path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.963 DEBUG:boto:auth_path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.964 DEBUG:boto:Method: GET 2022-09-03T05:41:40.965 DEBUG:boto:Path: /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.966 DEBUG:boto:Data: 2022-09-03T05:41:40.966 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.967 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:40.967 DEBUG:boto:Port: 8001 2022-09-03T05:41:40.968 DEBUG:boto:Params: {} 2022-09-03T05:41:40.968 DEBUG:boto:Token: None 2022-09-03T05:41:40.969 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.970 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs= 2022-09-03T05:41:40.970 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs=', 'Content-Length': '0'} 2022-09-03T05:41:40.972 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'hsp-uXpwzRjamHyM57a9u6NdISYw8SR'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000eb87692fdffbbb9a-006312e914-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.973 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:40.974 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:40.974 DEBUG:boto:path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.975 DEBUG:boto:auth_path=/dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.975 DEBUG:boto:Method: GET 2022-09-03T05:41:40.976 DEBUG:boto:Path: /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.977 DEBUG:boto:Data: 2022-09-03T05:41:40.977 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.978 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:40.978 DEBUG:boto:Port: 8000 2022-09-03T05:41:40.979 DEBUG:boto:Params: {} 2022-09-03T05:41:40.979 DEBUG:boto:Token: None 2022-09-03T05:41:40.980 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/obj?versionId=hsp-uXpwzRjamHyM57a9u6NdISYw8SR 2022-09-03T05:41:40.981 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs= 2022-09-03T05:41:40.981 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hC/LR1hg1MZx0audMwXE2dT9ZXs=', 'Content-Length': '0'} 2022-09-03T05:41:40.983 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'hsp-uXpwzRjamHyM57a9u6NdISYw8SR'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000e53389afa05e5877-006312e914-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.984 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.985 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.985 DEBUG:boto:path=/dsgvcy-21/?prefix=obj 2022-09-03T05:41:40.986 DEBUG:boto:auth_path=/dsgvcy-21/?prefix=obj 2022-09-03T05:41:40.986 DEBUG:boto:Method: GET 2022-09-03T05:41:40.987 DEBUG:boto:Path: /dsgvcy-21/?prefix=obj 2022-09-03T05:41:40.987 DEBUG:boto:Data: 2022-09-03T05:41:40.988 DEBUG:boto:Headers: {} 2022-09-03T05:41:40.989 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:40.989 DEBUG:boto:Port: 8001 2022-09-03T05:41:40.990 DEBUG:boto:Params: {} 2022-09-03T05:41:40.990 DEBUG:boto:Token: None 2022-09-03T05:41:40.991 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:40 GMT /dsgvcy-21/ 2022-09-03T05:41:40.992 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TXmg0PIrCLVq1qCya00o2gaEeEY= 2022-09-03T05:41:40.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TXmg0PIrCLVq1qCya00o2gaEeEY=', 'Content-Length': '0'} 2022-09-03T05:41:40.995 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000038850a5beac2cfdd-006312e914-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:40.996 DEBUG:boto:b'dsgvcy-21obj1000false' 2022-09-03T05:41:40.996 DEBUG:boto:path=/dsgvcy-21/ 2022-09-03T05:41:40.997 DEBUG:boto:auth_path=/dsgvcy-21/ 2022-09-03T05:41:40.997 DEBUG:boto:path=/dsgvcy-21/?prefix=obj 2022-09-03T05:41:40.998 DEBUG:boto:auth_path=/dsgvcy-21/?prefix=obj 2022-09-03T05:41:40.998 DEBUG:boto:Method: GET 2022-09-03T05:41:40.999 DEBUG:boto:Path: /dsgvcy-21/?prefix=obj 2022-09-03T05:41:41.000 DEBUG:boto:Data: 2022-09-03T05:41:41.000 DEBUG:boto:Headers: {} 2022-09-03T05:41:41.001 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:41.001 DEBUG:boto:Port: 8000 2022-09-03T05:41:41.002 DEBUG:boto:Params: {} 2022-09-03T05:41:41.003 DEBUG:boto:Token: None 2022-09-03T05:41:41.003 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:41 GMT /dsgvcy-21/ 2022-09-03T05:41:41.004 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e2rUBksDIZiePFbFzGAZ+xI7hjU= 2022-09-03T05:41:41.005 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e2rUBksDIZiePFbFzGAZ+xI7hjU=', 'Content-Length': '0'} 2022-09-03T05:41:41.007 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c5d4073c2f0126c0-006312e915-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:41.060 DEBUG:boto:b'dsgvcy-21obj1000false' 2022-09-03T05:41:41.062 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:41.063 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:41.064 DEBUG:boto:Method: HEAD 2022-09-03T05:41:41.065 DEBUG:boto:Path: /dsgvcy-21/obj 2022-09-03T05:41:41.066 DEBUG:boto:Data: 2022-09-03T05:41:41.067 DEBUG:boto:Headers: {} 2022-09-03T05:41:41.068 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:41.069 DEBUG:boto:Port: 8001 2022-09-03T05:41:41.070 DEBUG:boto:Params: {} 2022-09-03T05:41:41.071 DEBUG:boto:Token: None 2022-09-03T05:41:41.072 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:41 GMT /dsgvcy-21/obj 2022-09-03T05:41:41.073 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nk2qL5krPjU7uDS2xMiTQ/9qQTw= 2022-09-03T05:41:41.074 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nk2qL5krPjU7uDS2xMiTQ/9qQTw=', 'Content-Length': '0'} 2022-09-03T05:41:41.081 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx000007bd47eab083fe5b0-006312e915-1103-test-zone3'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:41.082 DEBUG:boto:path=/dsgvcy-21/obj 2022-09-03T05:41:41.083 DEBUG:boto:auth_path=/dsgvcy-21/obj 2022-09-03T05:41:41.083 DEBUG:boto:Method: HEAD 2022-09-03T05:41:41.084 DEBUG:boto:Path: /dsgvcy-21/obj 2022-09-03T05:41:41.085 DEBUG:boto:Data: 2022-09-03T05:41:41.086 DEBUG:boto:Headers: {} 2022-09-03T05:41:41.087 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:41.088 DEBUG:boto:Port: 8000 2022-09-03T05:41:41.088 DEBUG:boto:Params: {} 2022-09-03T05:41:41.089 DEBUG:boto:Token: None 2022-09-03T05:41:41.091 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:41 GMT /dsgvcy-21/obj 2022-09-03T05:41:41.092 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nk2qL5krPjU7uDS2xMiTQ/9qQTw= 2022-09-03T05:41:41.093 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nk2qL5krPjU7uDS2xMiTQ/9qQTw=', 'Content-Length': '0'} 2022-09-03T05:41:41.096 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx000001124e96ad81d8d2a-006312e915-10ea-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:41.097 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-21 zones={test-zone3, test-zone2} 2022-09-03T05:41:41.098 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-22 2022-09-03T05:41:41.098 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-22', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:41.099 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-22', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:41.100 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-22 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:41:41.160 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:41.161 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:41.195 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:41.194+0000 7fa43591a600 1 bucket sync caught up with source: 2022-09-03T05:41:41.195 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000002.21.5, , ] 2022-09-03T05:41:41.196 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000002.21.5, , ] 2022-09-03T05:41:41.196 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:41.194+0000 7fa43591a600 0 bucket checkpoint complete 2022-09-03T05:41:41.213 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-22 2022-09-03T05:41:41.214 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-22', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:41.214 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-22', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:41.215 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-22 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:41:41.275 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:41.275 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:41.318 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:41.317+0000 7feb553b2600 1 bucket sync caught up with source: 2022-09-03T05:41:41.318 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000002.21.5, , ] 2022-09-03T05:41:41.319 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000002.21.5, , ] 2022-09-03T05:41:41.319 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:41.317+0000 7feb553b2600 0 bucket checkpoint complete 2022-09-03T05:41:41.330 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-22 2022-09-03T05:41:41.331 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-22', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:41.332 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-22', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:41.333 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-22 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:41:41.389 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:41.390 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:41.440 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:41.440+0000 7f60c4438600 1 bucket sync caught up with source: 2022-09-03T05:41:41.441 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000002.21.5, , ] 2022-09-03T05:41:41.441 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000002.21.5, , ] 2022-09-03T05:41:41.442 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:41.440+0000 7f60c4438600 0 bucket checkpoint complete 2022-09-03T05:41:41.457 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-22 2022-09-03T05:41:41.458 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-22', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:41.459 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-22', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:41.460 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-22 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:41:41.523 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:41.524 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:41.564 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:41.564+0000 7f650a315600 1 bucket sync caught up with source: 2022-09-03T05:41:41.564 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000003.43.11, , ] 2022-09-03T05:41:41.565 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000003.43.11, , ] 2022-09-03T05:41:41.565 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:41.564+0000 7f650a315600 0 bucket checkpoint complete 2022-09-03T05:41:41.585 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-22 2022-09-03T05:41:41.585 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-22', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:41.586 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-22', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:41.587 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-22 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:41:41.643 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:41.644 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:41.690 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:41.690+0000 7f5cbeeb2600 1 bucket sync caught up with source: 2022-09-03T05:41:41.691 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000003.43.11, , ] 2022-09-03T05:41:41.691 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000003.43.11, , ] 2022-09-03T05:41:41.691 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:41.690+0000 7f5cbeeb2600 0 bucket checkpoint complete 2022-09-03T05:41:41.708 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-22 2022-09-03T05:41:41.709 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-22 2022-09-03T05:41:41.710 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-22', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:41.711 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-22', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:41.712 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-22 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:41:41.775 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:41.775 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:41.804 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:41.803+0000 7f46bc151600 1 bucket sync caught up with source: 2022-09-03T05:41:41.804 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000002.94.5, , ] 2022-09-03T05:41:41.805 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000002.94.5, , ] 2022-09-03T05:41:41.805 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:41.803+0000 7f46bc151600 0 bucket checkpoint complete 2022-09-03T05:41:41.816 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-22 2022-09-03T05:41:41.818 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-22', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:41.819 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-22', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:41.820 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-22 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:41:41.879 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:41.880 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:41.920 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:41.920+0000 7f95c3463600 1 bucket sync caught up with source: 2022-09-03T05:41:41.921 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000002.94.5, , ] 2022-09-03T05:41:41.921 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000002.94.5, , ] 2022-09-03T05:41:41.921 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:41.920+0000 7f95c3463600 0 bucket checkpoint complete 2022-09-03T05:41:41.935 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-22 2022-09-03T05:41:41.935 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-22 zones={test-zone2, test-zone1} 2022-09-03T05:41:41.936 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:41.937 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:41.937 DEBUG:boto:Method: HEAD 2022-09-03T05:41:41.938 DEBUG:boto:Path: /dsgvcy-22/ 2022-09-03T05:41:41.938 DEBUG:boto:Data: 2022-09-03T05:41:41.939 DEBUG:boto:Headers: {} 2022-09-03T05:41:41.939 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:41.940 DEBUG:boto:Port: 8000 2022-09-03T05:41:41.940 DEBUG:boto:Params: {} 2022-09-03T05:41:41.941 DEBUG:boto:Token: None 2022-09-03T05:41:41.942 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:41 GMT /dsgvcy-22/ 2022-09-03T05:41:41.942 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B1VT6mSyIR7OyMkOynu3GTTmb/k= 2022-09-03T05:41:41.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B1VT6mSyIR7OyMkOynu3GTTmb/k=', 'Content-Length': '0'} 2022-09-03T05:41:41.946 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c09b2e1b9576bc80-006312e915-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:41.947 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:41.947 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:41.948 DEBUG:boto:Method: HEAD 2022-09-03T05:41:41.949 DEBUG:boto:Path: /dsgvcy-22/ 2022-09-03T05:41:41.949 DEBUG:boto:Data: 2022-09-03T05:41:41.950 DEBUG:boto:Headers: {} 2022-09-03T05:41:41.950 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:41.951 DEBUG:boto:Port: 8000 2022-09-03T05:41:41.951 DEBUG:boto:Params: {} 2022-09-03T05:41:41.952 DEBUG:boto:Token: None 2022-09-03T05:41:41.952 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:41 GMT /dsgvcy-22/ 2022-09-03T05:41:41.953 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B1VT6mSyIR7OyMkOynu3GTTmb/k= 2022-09-03T05:41:41.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B1VT6mSyIR7OyMkOynu3GTTmb/k=', 'Content-Length': '0'} 2022-09-03T05:41:41.956 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000041c5851896354ca4-006312e915-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:41.957 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:41:41.958 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:41.958 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:41.959 DEBUG:boto:path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:41.960 DEBUG:boto:auth_path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:41.961 DEBUG:boto:Method: GET 2022-09-03T05:41:41.961 DEBUG:boto:Path: /dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:41.962 DEBUG:boto:Data: 2022-09-03T05:41:41.962 DEBUG:boto:Headers: {} 2022-09-03T05:41:41.963 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:41.963 DEBUG:boto:Port: 8000 2022-09-03T05:41:41.964 DEBUG:boto:Params: {} 2022-09-03T05:41:41.964 DEBUG:boto:Token: None 2022-09-03T05:41:41.965 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:41 GMT /dsgvcy-22/?versions 2022-09-03T05:41:41.966 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VVhPd8yI8TuC+3SMeBKPjkToCRM= 2022-09-03T05:41:41.967 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VVhPd8yI8TuC+3SMeBKPjkToCRM=', 'Content-Length': '0'} 2022-09-03T05:41:41.969 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000762d7126ea0b6bbc-006312e915-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:41.970 DEBUG:boto:b'dsgvcy-22999falseobjME-STHbynUNWJeX4LBwGonFwf1Mv1Cctrue2022-09-03T05:41:04.261Zrgw-multisite-test-userMultisite Test UserNormalobjZoRNNCWwWzoLPTocBQP3pl6jO5yZh1cfalse2022-09-03T05:41:04.231Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:41.971 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:41.971 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:41.972 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:41:41.972 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:41.973 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:41.973 DEBUG:boto:path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:41.974 DEBUG:boto:auth_path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:41.974 DEBUG:boto:Method: GET 2022-09-03T05:41:41.975 DEBUG:boto:Path: /dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:41.975 DEBUG:boto:Data: 2022-09-03T05:41:41.976 DEBUG:boto:Headers: {} 2022-09-03T05:41:41.976 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:41.977 DEBUG:boto:Port: 8000 2022-09-03T05:41:41.977 DEBUG:boto:Params: {} 2022-09-03T05:41:41.978 DEBUG:boto:Token: None 2022-09-03T05:41:41.979 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:41 GMT /dsgvcy-22/?versions 2022-09-03T05:41:41.979 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VVhPd8yI8TuC+3SMeBKPjkToCRM= 2022-09-03T05:41:41.980 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VVhPd8yI8TuC+3SMeBKPjkToCRM=', 'Content-Length': '0'} 2022-09-03T05:41:41.983 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000533e4edff260c8bb-006312e915-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:41.984 DEBUG:boto:b'dsgvcy-22999falseobjME-STHbynUNWJeX4LBwGonFwf1Mv1Cctrue2022-09-03T05:41:04.261Zrgw-multisite-test-userMultisite Test UserNormalobjZoRNNCWwWzoLPTocBQP3pl6jO5yZh1cfalse2022-09-03T05:41:04.231Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:41.985 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:41.985 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:41.986 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:41.986 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:41.987 DEBUG:boto:path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:41.987 DEBUG:boto:auth_path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:41.988 DEBUG:boto:Method: GET 2022-09-03T05:41:41.988 DEBUG:boto:Path: /dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:41.989 DEBUG:boto:Data: 2022-09-03T05:41:41.989 DEBUG:boto:Headers: {} 2022-09-03T05:41:41.990 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:41.991 DEBUG:boto:Port: 8000 2022-09-03T05:41:41.991 DEBUG:boto:Params: {} 2022-09-03T05:41:41.992 DEBUG:boto:Token: None 2022-09-03T05:41:41.993 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:41 GMT /dsgvcy-22/?versions 2022-09-03T05:41:41.994 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VVhPd8yI8TuC+3SMeBKPjkToCRM= 2022-09-03T05:41:41.995 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VVhPd8yI8TuC+3SMeBKPjkToCRM=', 'Content-Length': '0'} 2022-09-03T05:41:41.998 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007339b80dd7e01c7b-006312e915-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:41.999 DEBUG:boto:b'dsgvcy-22999falseobjME-STHbynUNWJeX4LBwGonFwf1Mv1Cctrue2022-09-03T05:41:04.261Zrgw-multisite-test-userMultisite Test UserNormalobjZoRNNCWwWzoLPTocBQP3pl6jO5yZh1cfalse2022-09-03T05:41:04.231Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:42.001 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:42.002 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:42.003 DEBUG:boto:path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.004 DEBUG:boto:auth_path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.005 DEBUG:boto:Method: GET 2022-09-03T05:41:42.006 DEBUG:boto:Path: /dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.006 DEBUG:boto:Data: 2022-09-03T05:41:42.007 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.008 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:42.008 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.009 DEBUG:boto:Params: {} 2022-09-03T05:41:42.010 DEBUG:boto:Token: None 2022-09-03T05:41:42.010 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/?versions 2022-09-03T05:41:42.012 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Sfkqy/IWcINN7cyDyPdnC4Xz/2Q= 2022-09-03T05:41:42.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Sfkqy/IWcINN7cyDyPdnC4Xz/2Q=', 'Content-Length': '0'} 2022-09-03T05:41:42.016 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c7a19ffe699ad853-006312e916-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.062 DEBUG:boto:b'dsgvcy-22999falseobjME-STHbynUNWJeX4LBwGonFwf1Mv1Cctrue2022-09-03T05:41:04.261Zrgw-multisite-test-userMultisite Test UserNormalobjZoRNNCWwWzoLPTocBQP3pl6jO5yZh1cfalse2022-09-03T05:41:04.231Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:42.064 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:42.065 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.067 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.068 DEBUG:boto:Method: HEAD 2022-09-03T05:41:42.069 DEBUG:boto:Path: /dsgvcy-22/obj 2022-09-03T05:41:42.069 DEBUG:boto:Data: 2022-09-03T05:41:42.070 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.071 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:42.073 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.074 DEBUG:boto:Params: {} 2022-09-03T05:41:42.075 DEBUG:boto:Token: None 2022-09-03T05:41:42.076 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj 2022-09-03T05:41:42.079 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY= 2022-09-03T05:41:42.080 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY=', 'Content-Length': '0'} 2022-09-03T05:41:42.086 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx00000b7d964c1e3692fd9-006312e916-10ea-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.088 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.088 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.089 DEBUG:boto:Method: HEAD 2022-09-03T05:41:42.090 DEBUG:boto:Path: /dsgvcy-22/obj 2022-09-03T05:41:42.091 DEBUG:boto:Data: 2022-09-03T05:41:42.091 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.092 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:42.093 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.093 DEBUG:boto:Params: {} 2022-09-03T05:41:42.094 DEBUG:boto:Token: None 2022-09-03T05:41:42.095 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj 2022-09-03T05:41:42.096 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY= 2022-09-03T05:41:42.097 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY=', 'Content-Length': '0'} 2022-09-03T05:41:42.100 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx00000da3163d7d861d4bc-006312e916-10ff-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.101 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:42.102 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.103 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.104 DEBUG:boto:path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.105 DEBUG:boto:auth_path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.106 DEBUG:boto:Method: GET 2022-09-03T05:41:42.107 DEBUG:boto:Path: /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.108 DEBUG:boto:Data: 2022-09-03T05:41:42.109 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.110 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:42.111 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.112 DEBUG:boto:Params: {} 2022-09-03T05:41:42.113 DEBUG:boto:Token: None 2022-09-03T05:41:42.114 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.115 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M= 2022-09-03T05:41:42.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M=', 'Content-Length': '0'} 2022-09-03T05:41:42.120 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000005ae12f138031423-006312e916-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.121 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.122 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.122 DEBUG:boto:path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.123 DEBUG:boto:auth_path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.124 DEBUG:boto:Method: GET 2022-09-03T05:41:42.124 DEBUG:boto:Path: /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.125 DEBUG:boto:Data: 2022-09-03T05:41:42.125 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.126 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:42.126 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.127 DEBUG:boto:Params: {} 2022-09-03T05:41:42.127 DEBUG:boto:Token: None 2022-09-03T05:41:42.128 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.129 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M= 2022-09-03T05:41:42.129 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M=', 'Content-Length': '0'} 2022-09-03T05:41:42.131 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000038d83e3c965bb097-006312e916-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.132 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.133 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.133 DEBUG:boto:path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.134 DEBUG:boto:auth_path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.134 DEBUG:boto:Method: HEAD 2022-09-03T05:41:42.135 DEBUG:boto:Path: /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.135 DEBUG:boto:Data: 2022-09-03T05:41:42.136 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.136 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:42.137 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.137 DEBUG:boto:Params: {} 2022-09-03T05:41:42.138 DEBUG:boto:Token: None 2022-09-03T05:41:42.138 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.139 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dyJMQOF1FGUk+Pc721Tc6D+r9f8= 2022-09-03T05:41:42.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dyJMQOF1FGUk+Pc721Tc6D+r9f8=', 'Content-Length': '0'} 2022-09-03T05:41:42.141 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000040e70eab7f4d493e-006312e916-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.142 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.143 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.143 DEBUG:boto:path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.144 DEBUG:boto:auth_path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.144 DEBUG:boto:Method: HEAD 2022-09-03T05:41:42.145 DEBUG:boto:Path: /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.146 DEBUG:boto:Data: 2022-09-03T05:41:42.146 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.147 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:42.148 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.148 DEBUG:boto:Params: {} 2022-09-03T05:41:42.149 DEBUG:boto:Token: None 2022-09-03T05:41:42.149 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.150 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dyJMQOF1FGUk+Pc721Tc6D+r9f8= 2022-09-03T05:41:42.151 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dyJMQOF1FGUk+Pc721Tc6D+r9f8=', 'Content-Length': '0'} 2022-09-03T05:41:42.153 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000e8dbbd6dc1778b23-006312e916-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.154 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:42.155 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.156 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.157 DEBUG:boto:path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.158 DEBUG:boto:auth_path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.160 DEBUG:boto:Method: GET 2022-09-03T05:41:42.161 DEBUG:boto:Path: /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.162 DEBUG:boto:Data: 2022-09-03T05:41:42.162 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.164 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:42.165 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.166 DEBUG:boto:Params: {} 2022-09-03T05:41:42.167 DEBUG:boto:Token: None 2022-09-03T05:41:42.168 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.170 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M= 2022-09-03T05:41:42.171 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M=', 'Content-Length': '0'} 2022-09-03T05:41:42.173 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000259050a534fc6885-006312e916-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.175 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.175 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.176 DEBUG:boto:path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.177 DEBUG:boto:auth_path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.177 DEBUG:boto:Method: GET 2022-09-03T05:41:42.178 DEBUG:boto:Path: /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.179 DEBUG:boto:Data: 2022-09-03T05:41:42.180 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.181 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:42.182 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.183 DEBUG:boto:Params: {} 2022-09-03T05:41:42.184 DEBUG:boto:Token: None 2022-09-03T05:41:42.185 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.186 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M= 2022-09-03T05:41:42.187 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M=', 'Content-Length': '0'} 2022-09-03T05:41:42.189 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000009b0293ce2063cfbc-006312e916-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.191 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:42.192 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:42.193 DEBUG:boto:path=/dsgvcy-22/?prefix=obj 2022-09-03T05:41:42.194 DEBUG:boto:auth_path=/dsgvcy-22/?prefix=obj 2022-09-03T05:41:42.195 DEBUG:boto:Method: GET 2022-09-03T05:41:42.196 DEBUG:boto:Path: /dsgvcy-22/?prefix=obj 2022-09-03T05:41:42.197 DEBUG:boto:Data: 2022-09-03T05:41:42.198 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.199 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:42.200 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.201 DEBUG:boto:Params: {} 2022-09-03T05:41:42.202 DEBUG:boto:Token: None 2022-09-03T05:41:42.203 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/ 2022-09-03T05:41:42.204 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nZdolWpiqQ0tyL4zMpRxabZux9A= 2022-09-03T05:41:42.205 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nZdolWpiqQ0tyL4zMpRxabZux9A=', 'Content-Length': '0'} 2022-09-03T05:41:42.209 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000391649d8683bdfef-006312e916-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.250 DEBUG:boto:b'dsgvcy-22obj1000false' 2022-09-03T05:41:42.251 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:42.252 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:42.253 DEBUG:boto:path=/dsgvcy-22/?prefix=obj 2022-09-03T05:41:42.253 DEBUG:boto:auth_path=/dsgvcy-22/?prefix=obj 2022-09-03T05:41:42.254 DEBUG:boto:Method: GET 2022-09-03T05:41:42.255 DEBUG:boto:Path: /dsgvcy-22/?prefix=obj 2022-09-03T05:41:42.256 DEBUG:boto:Data: 2022-09-03T05:41:42.257 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.259 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:42.260 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.261 DEBUG:boto:Params: {} 2022-09-03T05:41:42.261 DEBUG:boto:Token: None 2022-09-03T05:41:42.262 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/ 2022-09-03T05:41:42.263 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nZdolWpiqQ0tyL4zMpRxabZux9A= 2022-09-03T05:41:42.263 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nZdolWpiqQ0tyL4zMpRxabZux9A=', 'Content-Length': '0'} 2022-09-03T05:41:42.266 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b9e86a132b797005-006312e916-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.267 DEBUG:boto:b'dsgvcy-22obj1000false' 2022-09-03T05:41:42.267 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.268 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.268 DEBUG:boto:Method: HEAD 2022-09-03T05:41:42.269 DEBUG:boto:Path: /dsgvcy-22/obj 2022-09-03T05:41:42.269 DEBUG:boto:Data: 2022-09-03T05:41:42.270 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.271 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:42.271 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.272 DEBUG:boto:Params: {} 2022-09-03T05:41:42.272 DEBUG:boto:Token: None 2022-09-03T05:41:42.273 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj 2022-09-03T05:41:42.274 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY= 2022-09-03T05:41:42.275 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY=', 'Content-Length': '0'} 2022-09-03T05:41:42.277 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx00000ea6d315723d38abe-006312e916-10ea-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.278 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.279 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.279 DEBUG:boto:Method: HEAD 2022-09-03T05:41:42.280 DEBUG:boto:Path: /dsgvcy-22/obj 2022-09-03T05:41:42.281 DEBUG:boto:Data: 2022-09-03T05:41:42.281 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.282 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:42.282 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.283 DEBUG:boto:Params: {} 2022-09-03T05:41:42.283 DEBUG:boto:Token: None 2022-09-03T05:41:42.284 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj 2022-09-03T05:41:42.285 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY= 2022-09-03T05:41:42.285 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY=', 'Content-Length': '0'} 2022-09-03T05:41:42.288 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx000009ceb735c581a1abd-006312e916-10ff-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.288 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-22 zones={test-zone2, test-zone1} 2022-09-03T05:41:42.289 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-22 zones={test-zone3, test-zone1} 2022-09-03T05:41:42.289 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:42.290 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:42.290 DEBUG:boto:Method: HEAD 2022-09-03T05:41:42.291 DEBUG:boto:Path: /dsgvcy-22/ 2022-09-03T05:41:42.291 DEBUG:boto:Data: 2022-09-03T05:41:42.292 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.292 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:42.293 DEBUG:boto:Port: 8001 2022-09-03T05:41:42.294 DEBUG:boto:Params: {} 2022-09-03T05:41:42.295 DEBUG:boto:Token: None 2022-09-03T05:41:42.295 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/ 2022-09-03T05:41:42.296 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KoRh6/xuswcswaiS0VSvwdMXlNY= 2022-09-03T05:41:42.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KoRh6/xuswcswaiS0VSvwdMXlNY=', 'Content-Length': '0'} 2022-09-03T05:41:42.299 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000063ab74600c1e723a-006312e916-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.299 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:42.300 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:42.300 DEBUG:boto:Method: HEAD 2022-09-03T05:41:42.301 DEBUG:boto:Path: /dsgvcy-22/ 2022-09-03T05:41:42.301 DEBUG:boto:Data: 2022-09-03T05:41:42.302 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.303 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:42.303 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.303 DEBUG:boto:Params: {} 2022-09-03T05:41:42.304 DEBUG:boto:Token: None 2022-09-03T05:41:42.305 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/ 2022-09-03T05:41:42.305 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KoRh6/xuswcswaiS0VSvwdMXlNY= 2022-09-03T05:41:42.306 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KoRh6/xuswcswaiS0VSvwdMXlNY=', 'Content-Length': '0'} 2022-09-03T05:41:42.308 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006a42ce02e6bccb1b-006312e916-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.309 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:41:42.309 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:42.310 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:42.311 DEBUG:boto:path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.311 DEBUG:boto:auth_path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.312 DEBUG:boto:Method: GET 2022-09-03T05:41:42.312 DEBUG:boto:Path: /dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.313 DEBUG:boto:Data: 2022-09-03T05:41:42.313 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.314 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:42.314 DEBUG:boto:Port: 8001 2022-09-03T05:41:42.315 DEBUG:boto:Params: {} 2022-09-03T05:41:42.315 DEBUG:boto:Token: None 2022-09-03T05:41:42.316 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/?versions 2022-09-03T05:41:42.317 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Sfkqy/IWcINN7cyDyPdnC4Xz/2Q= 2022-09-03T05:41:42.317 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Sfkqy/IWcINN7cyDyPdnC4Xz/2Q=', 'Content-Length': '0'} 2022-09-03T05:41:42.320 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cc0935ca31334c2a-006312e916-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.362 DEBUG:boto:b'dsgvcy-22999falseobjME-STHbynUNWJeX4LBwGonFwf1Mv1Cctrue2022-09-03T05:41:04.261Zrgw-multisite-test-userMultisite Test UserNormalobjZoRNNCWwWzoLPTocBQP3pl6jO5yZh1cfalse2022-09-03T05:41:04.231Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:42.363 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:42.364 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:42.365 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:41:42.366 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:42.367 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:42.367 DEBUG:boto:path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.368 DEBUG:boto:auth_path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.368 DEBUG:boto:Method: GET 2022-09-03T05:41:42.369 DEBUG:boto:Path: /dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.369 DEBUG:boto:Data: 2022-09-03T05:41:42.370 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.370 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:42.371 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.371 DEBUG:boto:Params: {} 2022-09-03T05:41:42.372 DEBUG:boto:Token: None 2022-09-03T05:41:42.373 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/?versions 2022-09-03T05:41:42.373 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Sfkqy/IWcINN7cyDyPdnC4Xz/2Q= 2022-09-03T05:41:42.375 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Sfkqy/IWcINN7cyDyPdnC4Xz/2Q=', 'Content-Length': '0'} 2022-09-03T05:41:42.378 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003ee10af5d5a9d0e7-006312e916-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.379 DEBUG:boto:b'dsgvcy-22999falseobjME-STHbynUNWJeX4LBwGonFwf1Mv1Cctrue2022-09-03T05:41:04.261Zrgw-multisite-test-userMultisite Test UserNormalobjZoRNNCWwWzoLPTocBQP3pl6jO5yZh1cfalse2022-09-03T05:41:04.231Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:42.380 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:42.380 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:42.381 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:42.382 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:42.383 DEBUG:boto:path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.383 DEBUG:boto:auth_path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.384 DEBUG:boto:Method: GET 2022-09-03T05:41:42.386 DEBUG:boto:Path: /dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.386 DEBUG:boto:Data: 2022-09-03T05:41:42.387 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.387 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:42.388 DEBUG:boto:Port: 8001 2022-09-03T05:41:42.389 DEBUG:boto:Params: {} 2022-09-03T05:41:42.390 DEBUG:boto:Token: None 2022-09-03T05:41:42.391 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/?versions 2022-09-03T05:41:42.392 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Sfkqy/IWcINN7cyDyPdnC4Xz/2Q= 2022-09-03T05:41:42.392 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Sfkqy/IWcINN7cyDyPdnC4Xz/2Q=', 'Content-Length': '0'} 2022-09-03T05:41:42.395 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000017d5a9d6b9ac773f-006312e916-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.396 DEBUG:boto:b'dsgvcy-22999falseobjME-STHbynUNWJeX4LBwGonFwf1Mv1Cctrue2022-09-03T05:41:04.261Zrgw-multisite-test-userMultisite Test UserNormalobjZoRNNCWwWzoLPTocBQP3pl6jO5yZh1cfalse2022-09-03T05:41:04.231Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:42.398 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:42.399 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:42.400 DEBUG:boto:path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.401 DEBUG:boto:auth_path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.402 DEBUG:boto:Method: GET 2022-09-03T05:41:42.402 DEBUG:boto:Path: /dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.403 DEBUG:boto:Data: 2022-09-03T05:41:42.404 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.405 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:42.407 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.408 DEBUG:boto:Params: {} 2022-09-03T05:41:42.410 DEBUG:boto:Token: None 2022-09-03T05:41:42.410 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/?versions 2022-09-03T05:41:42.411 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Sfkqy/IWcINN7cyDyPdnC4Xz/2Q= 2022-09-03T05:41:42.411 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Sfkqy/IWcINN7cyDyPdnC4Xz/2Q=', 'Content-Length': '0'} 2022-09-03T05:41:42.414 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000486278e2d05429d0-006312e916-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.415 DEBUG:boto:b'dsgvcy-22999falseobjME-STHbynUNWJeX4LBwGonFwf1Mv1Cctrue2022-09-03T05:41:04.261Zrgw-multisite-test-userMultisite Test UserNormalobjZoRNNCWwWzoLPTocBQP3pl6jO5yZh1cfalse2022-09-03T05:41:04.231Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:42.416 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:42.416 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.417 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.417 DEBUG:boto:Method: HEAD 2022-09-03T05:41:42.418 DEBUG:boto:Path: /dsgvcy-22/obj 2022-09-03T05:41:42.418 DEBUG:boto:Data: 2022-09-03T05:41:42.419 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.419 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:42.420 DEBUG:boto:Port: 8001 2022-09-03T05:41:42.422 DEBUG:boto:Params: {} 2022-09-03T05:41:42.422 DEBUG:boto:Token: None 2022-09-03T05:41:42.423 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj 2022-09-03T05:41:42.424 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY= 2022-09-03T05:41:42.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY=', 'Content-Length': '0'} 2022-09-03T05:41:42.427 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx00000e26a09a435996fc8-006312e916-1103-test-zone3'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.428 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.428 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.429 DEBUG:boto:Method: HEAD 2022-09-03T05:41:42.430 DEBUG:boto:Path: /dsgvcy-22/obj 2022-09-03T05:41:42.430 DEBUG:boto:Data: 2022-09-03T05:41:42.431 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.431 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:42.432 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.432 DEBUG:boto:Params: {} 2022-09-03T05:41:42.433 DEBUG:boto:Token: None 2022-09-03T05:41:42.433 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj 2022-09-03T05:41:42.434 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY= 2022-09-03T05:41:42.435 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY=', 'Content-Length': '0'} 2022-09-03T05:41:42.437 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx0000069f9af3966d8c811-006312e916-10ff-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.438 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:42.438 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.439 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.439 DEBUG:boto:path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.440 DEBUG:boto:auth_path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.440 DEBUG:boto:Method: GET 2022-09-03T05:41:42.441 DEBUG:boto:Path: /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.442 DEBUG:boto:Data: 2022-09-03T05:41:42.442 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.443 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:42.443 DEBUG:boto:Port: 8001 2022-09-03T05:41:42.444 DEBUG:boto:Params: {} 2022-09-03T05:41:42.445 DEBUG:boto:Token: None 2022-09-03T05:41:42.446 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.446 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M= 2022-09-03T05:41:42.447 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M=', 'Content-Length': '0'} 2022-09-03T05:41:42.449 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000d11e9ba01c7f433d-006312e916-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.450 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.450 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.451 DEBUG:boto:path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.451 DEBUG:boto:auth_path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.452 DEBUG:boto:Method: GET 2022-09-03T05:41:42.453 DEBUG:boto:Path: /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.453 DEBUG:boto:Data: 2022-09-03T05:41:42.454 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.454 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:42.455 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.455 DEBUG:boto:Params: {} 2022-09-03T05:41:42.456 DEBUG:boto:Token: None 2022-09-03T05:41:42.457 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.458 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M= 2022-09-03T05:41:42.458 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M=', 'Content-Length': '0'} 2022-09-03T05:41:42.460 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000e4b1e0322128212b-006312e916-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.463 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.465 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.468 DEBUG:boto:path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.468 DEBUG:boto:auth_path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.469 DEBUG:boto:Method: HEAD 2022-09-03T05:41:42.469 DEBUG:boto:Path: /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.470 DEBUG:boto:Data: 2022-09-03T05:41:42.470 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.471 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:42.471 DEBUG:boto:Port: 8001 2022-09-03T05:41:42.472 DEBUG:boto:Params: {} 2022-09-03T05:41:42.472 DEBUG:boto:Token: None 2022-09-03T05:41:42.473 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.474 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dyJMQOF1FGUk+Pc721Tc6D+r9f8= 2022-09-03T05:41:42.475 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dyJMQOF1FGUk+Pc721Tc6D+r9f8=', 'Content-Length': '0'} 2022-09-03T05:41:42.476 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000e097dcb721db2a68-006312e916-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.477 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.478 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.478 DEBUG:boto:path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.479 DEBUG:boto:auth_path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.480 DEBUG:boto:Method: HEAD 2022-09-03T05:41:42.480 DEBUG:boto:Path: /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.481 DEBUG:boto:Data: 2022-09-03T05:41:42.481 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.482 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:42.482 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.483 DEBUG:boto:Params: {} 2022-09-03T05:41:42.484 DEBUG:boto:Token: None 2022-09-03T05:41:42.485 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.485 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dyJMQOF1FGUk+Pc721Tc6D+r9f8= 2022-09-03T05:41:42.486 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dyJMQOF1FGUk+Pc721Tc6D+r9f8=', 'Content-Length': '0'} 2022-09-03T05:41:42.488 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000df2e6d4a455c2c73-006312e916-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.489 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:42.490 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.491 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.492 DEBUG:boto:path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.492 DEBUG:boto:auth_path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.493 DEBUG:boto:Method: GET 2022-09-03T05:41:42.493 DEBUG:boto:Path: /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.494 DEBUG:boto:Data: 2022-09-03T05:41:42.494 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.495 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:42.495 DEBUG:boto:Port: 8001 2022-09-03T05:41:42.497 DEBUG:boto:Params: {} 2022-09-03T05:41:42.497 DEBUG:boto:Token: None 2022-09-03T05:41:42.498 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.499 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M= 2022-09-03T05:41:42.499 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M=', 'Content-Length': '0'} 2022-09-03T05:41:42.501 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000d6254f737e747e08-006312e916-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.502 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.503 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.503 DEBUG:boto:path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.504 DEBUG:boto:auth_path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.504 DEBUG:boto:Method: GET 2022-09-03T05:41:42.505 DEBUG:boto:Path: /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.505 DEBUG:boto:Data: 2022-09-03T05:41:42.506 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.507 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:42.507 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.508 DEBUG:boto:Params: {} 2022-09-03T05:41:42.508 DEBUG:boto:Token: None 2022-09-03T05:41:42.509 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.510 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M= 2022-09-03T05:41:42.511 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M=', 'Content-Length': '0'} 2022-09-03T05:41:42.512 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000036cb20b948771d9f-006312e916-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.513 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:42.514 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:42.514 DEBUG:boto:path=/dsgvcy-22/?prefix=obj 2022-09-03T05:41:42.515 DEBUG:boto:auth_path=/dsgvcy-22/?prefix=obj 2022-09-03T05:41:42.515 DEBUG:boto:Method: GET 2022-09-03T05:41:42.516 DEBUG:boto:Path: /dsgvcy-22/?prefix=obj 2022-09-03T05:41:42.517 DEBUG:boto:Data: 2022-09-03T05:41:42.517 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.518 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:42.518 DEBUG:boto:Port: 8001 2022-09-03T05:41:42.519 DEBUG:boto:Params: {} 2022-09-03T05:41:42.519 DEBUG:boto:Token: None 2022-09-03T05:41:42.520 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/ 2022-09-03T05:41:42.520 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nZdolWpiqQ0tyL4zMpRxabZux9A= 2022-09-03T05:41:42.521 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nZdolWpiqQ0tyL4zMpRxabZux9A=', 'Content-Length': '0'} 2022-09-03T05:41:42.531 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fa8a57c5b6eb845c-006312e916-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.574 DEBUG:boto:b'dsgvcy-22obj1000false' 2022-09-03T05:41:42.575 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:42.576 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:42.576 DEBUG:boto:path=/dsgvcy-22/?prefix=obj 2022-09-03T05:41:42.577 DEBUG:boto:auth_path=/dsgvcy-22/?prefix=obj 2022-09-03T05:41:42.578 DEBUG:boto:Method: GET 2022-09-03T05:41:42.579 DEBUG:boto:Path: /dsgvcy-22/?prefix=obj 2022-09-03T05:41:42.579 DEBUG:boto:Data: 2022-09-03T05:41:42.580 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.580 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:42.581 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.581 DEBUG:boto:Params: {} 2022-09-03T05:41:42.582 DEBUG:boto:Token: None 2022-09-03T05:41:42.583 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/ 2022-09-03T05:41:42.584 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nZdolWpiqQ0tyL4zMpRxabZux9A= 2022-09-03T05:41:42.585 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nZdolWpiqQ0tyL4zMpRxabZux9A=', 'Content-Length': '0'} 2022-09-03T05:41:42.587 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000052a4e450c480cdc8-006312e916-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.588 DEBUG:boto:b'dsgvcy-22obj1000false' 2022-09-03T05:41:42.589 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.589 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.590 DEBUG:boto:Method: HEAD 2022-09-03T05:41:42.591 DEBUG:boto:Path: /dsgvcy-22/obj 2022-09-03T05:41:42.591 DEBUG:boto:Data: 2022-09-03T05:41:42.592 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.592 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:42.593 DEBUG:boto:Port: 8001 2022-09-03T05:41:42.594 DEBUG:boto:Params: {} 2022-09-03T05:41:42.595 DEBUG:boto:Token: None 2022-09-03T05:41:42.596 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj 2022-09-03T05:41:42.598 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY= 2022-09-03T05:41:42.599 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY=', 'Content-Length': '0'} 2022-09-03T05:41:42.602 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx0000095f08ad66e964ad8-006312e916-1103-test-zone3'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.603 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.604 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.605 DEBUG:boto:Method: HEAD 2022-09-03T05:41:42.606 DEBUG:boto:Path: /dsgvcy-22/obj 2022-09-03T05:41:42.607 DEBUG:boto:Data: 2022-09-03T05:41:42.608 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.609 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:42.610 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.611 DEBUG:boto:Params: {} 2022-09-03T05:41:42.612 DEBUG:boto:Token: None 2022-09-03T05:41:42.612 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj 2022-09-03T05:41:42.613 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY= 2022-09-03T05:41:42.614 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY=', 'Content-Length': '0'} 2022-09-03T05:41:42.616 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx00000b8e70d7e2eb46ceb-006312e916-10ff-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.617 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-22 zones={test-zone3, test-zone1} 2022-09-03T05:41:42.618 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-22 zones={test-zone3, test-zone2} 2022-09-03T05:41:42.619 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:42.620 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:42.621 DEBUG:boto:Method: HEAD 2022-09-03T05:41:42.621 DEBUG:boto:Path: /dsgvcy-22/ 2022-09-03T05:41:42.622 DEBUG:boto:Data: 2022-09-03T05:41:42.623 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.624 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:42.625 DEBUG:boto:Port: 8001 2022-09-03T05:41:42.626 DEBUG:boto:Params: {} 2022-09-03T05:41:42.627 DEBUG:boto:Token: None 2022-09-03T05:41:42.628 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/ 2022-09-03T05:41:42.628 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KoRh6/xuswcswaiS0VSvwdMXlNY= 2022-09-03T05:41:42.629 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KoRh6/xuswcswaiS0VSvwdMXlNY=', 'Content-Length': '0'} 2022-09-03T05:41:42.632 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002c4bbf2590b62212-006312e916-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.633 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:42.633 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:42.634 DEBUG:boto:Method: HEAD 2022-09-03T05:41:42.634 DEBUG:boto:Path: /dsgvcy-22/ 2022-09-03T05:41:42.636 DEBUG:boto:Data: 2022-09-03T05:41:42.636 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.637 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:42.638 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.638 DEBUG:boto:Params: {} 2022-09-03T05:41:42.639 DEBUG:boto:Token: None 2022-09-03T05:41:42.639 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/ 2022-09-03T05:41:42.640 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KoRh6/xuswcswaiS0VSvwdMXlNY= 2022-09-03T05:41:42.641 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KoRh6/xuswcswaiS0VSvwdMXlNY=', 'Content-Length': '0'} 2022-09-03T05:41:42.643 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000519fcf255f336ba9-006312e916-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.644 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:41:42.645 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:42.645 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:42.646 DEBUG:boto:path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.646 DEBUG:boto:auth_path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.647 DEBUG:boto:Method: GET 2022-09-03T05:41:42.648 DEBUG:boto:Path: /dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.649 DEBUG:boto:Data: 2022-09-03T05:41:42.650 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.650 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:42.651 DEBUG:boto:Port: 8001 2022-09-03T05:41:42.652 DEBUG:boto:Params: {} 2022-09-03T05:41:42.653 DEBUG:boto:Token: None 2022-09-03T05:41:42.654 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/?versions 2022-09-03T05:41:42.655 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Sfkqy/IWcINN7cyDyPdnC4Xz/2Q= 2022-09-03T05:41:42.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Sfkqy/IWcINN7cyDyPdnC4Xz/2Q=', 'Content-Length': '0'} 2022-09-03T05:41:42.659 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000bbc1c3ba6d2e962-006312e916-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.660 DEBUG:boto:b'dsgvcy-22999falseobjME-STHbynUNWJeX4LBwGonFwf1Mv1Cctrue2022-09-03T05:41:04.261Zrgw-multisite-test-userMultisite Test UserNormalobjZoRNNCWwWzoLPTocBQP3pl6jO5yZh1cfalse2022-09-03T05:41:04.231Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:42.661 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:42.662 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:42.663 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:41:42.663 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:42.664 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:42.665 DEBUG:boto:path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.665 DEBUG:boto:auth_path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.666 DEBUG:boto:Method: GET 2022-09-03T05:41:42.666 DEBUG:boto:Path: /dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.667 DEBUG:boto:Data: 2022-09-03T05:41:42.667 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.668 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:42.668 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.669 DEBUG:boto:Params: {} 2022-09-03T05:41:42.669 DEBUG:boto:Token: None 2022-09-03T05:41:42.670 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/?versions 2022-09-03T05:41:42.670 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Sfkqy/IWcINN7cyDyPdnC4Xz/2Q= 2022-09-03T05:41:42.671 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Sfkqy/IWcINN7cyDyPdnC4Xz/2Q=', 'Content-Length': '0'} 2022-09-03T05:41:42.673 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d60c4e4a2ba22d15-006312e916-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.674 DEBUG:boto:b'dsgvcy-22999falseobjME-STHbynUNWJeX4LBwGonFwf1Mv1Cctrue2022-09-03T05:41:04.261Zrgw-multisite-test-userMultisite Test UserNormalobjZoRNNCWwWzoLPTocBQP3pl6jO5yZh1cfalse2022-09-03T05:41:04.231Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:42.675 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:42.676 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:42.676 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:42.677 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:42.677 DEBUG:boto:path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.678 DEBUG:boto:auth_path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.678 DEBUG:boto:Method: GET 2022-09-03T05:41:42.679 DEBUG:boto:Path: /dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.679 DEBUG:boto:Data: 2022-09-03T05:41:42.680 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.680 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:42.681 DEBUG:boto:Port: 8001 2022-09-03T05:41:42.681 DEBUG:boto:Params: {} 2022-09-03T05:41:42.682 DEBUG:boto:Token: None 2022-09-03T05:41:42.682 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/?versions 2022-09-03T05:41:42.683 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Sfkqy/IWcINN7cyDyPdnC4Xz/2Q= 2022-09-03T05:41:42.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Sfkqy/IWcINN7cyDyPdnC4Xz/2Q=', 'Content-Length': '0'} 2022-09-03T05:41:42.686 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ea188b6b3a73d02f-006312e916-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.687 DEBUG:boto:b'dsgvcy-22999falseobjME-STHbynUNWJeX4LBwGonFwf1Mv1Cctrue2022-09-03T05:41:04.261Zrgw-multisite-test-userMultisite Test UserNormalobjZoRNNCWwWzoLPTocBQP3pl6jO5yZh1cfalse2022-09-03T05:41:04.231Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:42.688 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:42.688 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:42.689 DEBUG:boto:path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.689 DEBUG:boto:auth_path=/dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.690 DEBUG:boto:Method: GET 2022-09-03T05:41:42.690 DEBUG:boto:Path: /dsgvcy-22/?versions&max-keys=999 2022-09-03T05:41:42.691 DEBUG:boto:Data: 2022-09-03T05:41:42.691 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.692 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:42.692 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.693 DEBUG:boto:Params: {} 2022-09-03T05:41:42.694 DEBUG:boto:Token: None 2022-09-03T05:41:42.694 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/?versions 2022-09-03T05:41:42.695 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Sfkqy/IWcINN7cyDyPdnC4Xz/2Q= 2022-09-03T05:41:42.695 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Sfkqy/IWcINN7cyDyPdnC4Xz/2Q=', 'Content-Length': '0'} 2022-09-03T05:41:42.698 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000271d30ff3e83522-006312e916-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.699 DEBUG:boto:b'dsgvcy-22999falseobjME-STHbynUNWJeX4LBwGonFwf1Mv1Cctrue2022-09-03T05:41:04.261Zrgw-multisite-test-userMultisite Test UserNormalobjZoRNNCWwWzoLPTocBQP3pl6jO5yZh1cfalse2022-09-03T05:41:04.231Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:42.699 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:42.700 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.701 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.701 DEBUG:boto:Method: HEAD 2022-09-03T05:41:42.702 DEBUG:boto:Path: /dsgvcy-22/obj 2022-09-03T05:41:42.702 DEBUG:boto:Data: 2022-09-03T05:41:42.703 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.703 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:42.704 DEBUG:boto:Port: 8001 2022-09-03T05:41:42.704 DEBUG:boto:Params: {} 2022-09-03T05:41:42.705 DEBUG:boto:Token: None 2022-09-03T05:41:42.706 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj 2022-09-03T05:41:42.706 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY= 2022-09-03T05:41:42.707 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY=', 'Content-Length': '0'} 2022-09-03T05:41:42.709 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx000004a3f93877ee04e6c-006312e916-1103-test-zone3'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.710 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.710 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.711 DEBUG:boto:Method: HEAD 2022-09-03T05:41:42.711 DEBUG:boto:Path: /dsgvcy-22/obj 2022-09-03T05:41:42.712 DEBUG:boto:Data: 2022-09-03T05:41:42.712 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.713 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:42.713 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.714 DEBUG:boto:Params: {} 2022-09-03T05:41:42.714 DEBUG:boto:Token: None 2022-09-03T05:41:42.715 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj 2022-09-03T05:41:42.715 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY= 2022-09-03T05:41:42.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY=', 'Content-Length': '0'} 2022-09-03T05:41:42.718 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx00000296bf74bb8c1cc1b-006312e916-10ea-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.719 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:42.719 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.720 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.720 DEBUG:boto:path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.721 DEBUG:boto:auth_path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.721 DEBUG:boto:Method: GET 2022-09-03T05:41:42.722 DEBUG:boto:Path: /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.723 DEBUG:boto:Data: 2022-09-03T05:41:42.723 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.724 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:42.724 DEBUG:boto:Port: 8001 2022-09-03T05:41:42.725 DEBUG:boto:Params: {} 2022-09-03T05:41:42.725 DEBUG:boto:Token: None 2022-09-03T05:41:42.726 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.726 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M= 2022-09-03T05:41:42.727 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M=', 'Content-Length': '0'} 2022-09-03T05:41:42.729 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000fb38b144a11d60c5-006312e916-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.730 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.730 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.731 DEBUG:boto:path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.731 DEBUG:boto:auth_path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.732 DEBUG:boto:Method: GET 2022-09-03T05:41:42.733 DEBUG:boto:Path: /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.733 DEBUG:boto:Data: 2022-09-03T05:41:42.734 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.734 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:42.735 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.735 DEBUG:boto:Params: {} 2022-09-03T05:41:42.736 DEBUG:boto:Token: None 2022-09-03T05:41:42.737 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.737 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M= 2022-09-03T05:41:42.738 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M=', 'Content-Length': '0'} 2022-09-03T05:41:42.740 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000c50dce598698ccfc-006312e916-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.740 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.741 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.741 DEBUG:boto:path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.742 DEBUG:boto:auth_path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.742 DEBUG:boto:Method: HEAD 2022-09-03T05:41:42.743 DEBUG:boto:Path: /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.743 DEBUG:boto:Data: 2022-09-03T05:41:42.744 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.744 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:42.745 DEBUG:boto:Port: 8001 2022-09-03T05:41:42.745 DEBUG:boto:Params: {} 2022-09-03T05:41:42.746 DEBUG:boto:Token: None 2022-09-03T05:41:42.747 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.747 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dyJMQOF1FGUk+Pc721Tc6D+r9f8= 2022-09-03T05:41:42.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dyJMQOF1FGUk+Pc721Tc6D+r9f8=', 'Content-Length': '0'} 2022-09-03T05:41:42.750 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000bf78d3baad48b6a5-006312e916-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.750 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.751 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.752 DEBUG:boto:path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.752 DEBUG:boto:auth_path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.753 DEBUG:boto:Method: HEAD 2022-09-03T05:41:42.753 DEBUG:boto:Path: /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.754 DEBUG:boto:Data: 2022-09-03T05:41:42.754 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.755 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:42.755 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.756 DEBUG:boto:Params: {} 2022-09-03T05:41:42.756 DEBUG:boto:Token: None 2022-09-03T05:41:42.757 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.757 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dyJMQOF1FGUk+Pc721Tc6D+r9f8= 2022-09-03T05:41:42.758 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dyJMQOF1FGUk+Pc721Tc6D+r9f8=', 'Content-Length': '0'} 2022-09-03T05:41:42.760 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000f368b71571f8b886-006312e916-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.761 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:42.761 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.762 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.762 DEBUG:boto:path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.763 DEBUG:boto:auth_path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.763 DEBUG:boto:Method: GET 2022-09-03T05:41:42.764 DEBUG:boto:Path: /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.764 DEBUG:boto:Data: 2022-09-03T05:41:42.765 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.765 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:42.766 DEBUG:boto:Port: 8001 2022-09-03T05:41:42.767 DEBUG:boto:Params: {} 2022-09-03T05:41:42.767 DEBUG:boto:Token: None 2022-09-03T05:41:42.768 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.768 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M= 2022-09-03T05:41:42.769 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M=', 'Content-Length': '0'} 2022-09-03T05:41:42.771 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000c30ecb65f45f560d-006312e916-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.772 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.772 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.773 DEBUG:boto:path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.773 DEBUG:boto:auth_path=/dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.774 DEBUG:boto:Method: GET 2022-09-03T05:41:42.774 DEBUG:boto:Path: /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.775 DEBUG:boto:Data: 2022-09-03T05:41:42.775 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.776 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:42.776 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.777 DEBUG:boto:Params: {} 2022-09-03T05:41:42.778 DEBUG:boto:Token: None 2022-09-03T05:41:42.778 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj?versionId=ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c 2022-09-03T05:41:42.779 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M= 2022-09-03T05:41:42.779 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uarobvXUAIiMI4vPtL18lIsD5+M=', 'Content-Length': '0'} 2022-09-03T05:41:42.782 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'ZoRNNCWwWzoLPTocBQP3pl6jO5yZh1c'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000007dd5040560a3c367-006312e916-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.782 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:42.783 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:42.784 DEBUG:boto:path=/dsgvcy-22/?prefix=obj 2022-09-03T05:41:42.784 DEBUG:boto:auth_path=/dsgvcy-22/?prefix=obj 2022-09-03T05:41:42.785 DEBUG:boto:Method: GET 2022-09-03T05:41:42.785 DEBUG:boto:Path: /dsgvcy-22/?prefix=obj 2022-09-03T05:41:42.786 DEBUG:boto:Data: 2022-09-03T05:41:42.787 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.788 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:42.789 DEBUG:boto:Port: 8001 2022-09-03T05:41:42.790 DEBUG:boto:Params: {} 2022-09-03T05:41:42.791 DEBUG:boto:Token: None 2022-09-03T05:41:42.792 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/ 2022-09-03T05:41:42.793 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nZdolWpiqQ0tyL4zMpRxabZux9A= 2022-09-03T05:41:42.793 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nZdolWpiqQ0tyL4zMpRxabZux9A=', 'Content-Length': '0'} 2022-09-03T05:41:42.796 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009d1aefb832772186-006312e916-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.838 DEBUG:boto:b'dsgvcy-22obj1000false' 2022-09-03T05:41:42.839 DEBUG:boto:path=/dsgvcy-22/ 2022-09-03T05:41:42.840 DEBUG:boto:auth_path=/dsgvcy-22/ 2022-09-03T05:41:42.841 DEBUG:boto:path=/dsgvcy-22/?prefix=obj 2022-09-03T05:41:42.841 DEBUG:boto:auth_path=/dsgvcy-22/?prefix=obj 2022-09-03T05:41:42.842 DEBUG:boto:Method: GET 2022-09-03T05:41:42.842 DEBUG:boto:Path: /dsgvcy-22/?prefix=obj 2022-09-03T05:41:42.843 DEBUG:boto:Data: 2022-09-03T05:41:42.844 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.845 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:42.846 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.847 DEBUG:boto:Params: {} 2022-09-03T05:41:42.847 DEBUG:boto:Token: None 2022-09-03T05:41:42.848 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/ 2022-09-03T05:41:42.849 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nZdolWpiqQ0tyL4zMpRxabZux9A= 2022-09-03T05:41:42.849 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nZdolWpiqQ0tyL4zMpRxabZux9A=', 'Content-Length': '0'} 2022-09-03T05:41:42.852 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dd5c8d56d917d3b2-006312e916-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.852 DEBUG:boto:b'dsgvcy-22obj1000false' 2022-09-03T05:41:42.853 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.854 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.854 DEBUG:boto:Method: HEAD 2022-09-03T05:41:42.855 DEBUG:boto:Path: /dsgvcy-22/obj 2022-09-03T05:41:42.855 DEBUG:boto:Data: 2022-09-03T05:41:42.856 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.856 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:42.857 DEBUG:boto:Port: 8001 2022-09-03T05:41:42.857 DEBUG:boto:Params: {} 2022-09-03T05:41:42.858 DEBUG:boto:Token: None 2022-09-03T05:41:42.858 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj 2022-09-03T05:41:42.859 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY= 2022-09-03T05:41:42.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY=', 'Content-Length': '0'} 2022-09-03T05:41:42.862 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx000002ba79beee33f1029-006312e916-1103-test-zone3'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.862 DEBUG:boto:path=/dsgvcy-22/obj 2022-09-03T05:41:42.863 DEBUG:boto:auth_path=/dsgvcy-22/obj 2022-09-03T05:41:42.864 DEBUG:boto:Method: HEAD 2022-09-03T05:41:42.864 DEBUG:boto:Path: /dsgvcy-22/obj 2022-09-03T05:41:42.865 DEBUG:boto:Data: 2022-09-03T05:41:42.865 DEBUG:boto:Headers: {} 2022-09-03T05:41:42.866 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:42.866 DEBUG:boto:Port: 8000 2022-09-03T05:41:42.867 DEBUG:boto:Params: {} 2022-09-03T05:41:42.867 DEBUG:boto:Token: None 2022-09-03T05:41:42.868 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:42 GMT /dsgvcy-22/obj 2022-09-03T05:41:42.869 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY= 2022-09-03T05:41:42.869 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJ+Dpdg5fs8ncmcu53uAafpfUmY=', 'Content-Length': '0'} 2022-09-03T05:41:42.872 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx000005be09fb380a86939-006312e916-10ea-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:42.873 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-22 zones={test-zone3, test-zone2} 2022-09-03T05:41:42.873 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-23 2022-09-03T05:41:42.874 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-23', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:42.874 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-23', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:42.875 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-23 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:41:42.933 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:42.934 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:42.966 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:42.965+0000 7f43ac881600 1 bucket sync caught up with source: 2022-09-03T05:41:42.967 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000002.11.5, , ] 2022-09-03T05:41:42.967 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000002.11.5, , ] 2022-09-03T05:41:42.968 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:42.965+0000 7f43ac881600 0 bucket checkpoint complete 2022-09-03T05:41:42.978 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-23 2022-09-03T05:41:42.979 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-23', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:42.980 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-23', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:42.980 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-23 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:41:43.040 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:43.041 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:43.080 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:43.080+0000 7f8e0df90600 1 bucket sync caught up with source: 2022-09-03T05:41:43.081 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000002.11.5, , ] 2022-09-03T05:41:43.081 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000002.11.5, , ] 2022-09-03T05:41:43.082 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:43.080+0000 7f8e0df90600 0 bucket checkpoint complete 2022-09-03T05:41:43.098 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-23 2022-09-03T05:41:43.099 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-23', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:43.100 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-23', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:43.100 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-23 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:41:43.160 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:43.161 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:43.191 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:43.190+0000 7f49827c6600 1 bucket sync caught up with source: 2022-09-03T05:41:43.191 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000002.11.5, , ] 2022-09-03T05:41:43.191 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000002.11.5, , ] 2022-09-03T05:41:43.192 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:43.190+0000 7f49827c6600 0 bucket checkpoint complete 2022-09-03T05:41:43.203 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-23 2022-09-03T05:41:43.204 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-23', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:43.205 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-23', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:43.206 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-23 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:41:43.265 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:43.265 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:43.309 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:43.309+0000 7fdc8a34f600 1 waiting for incremental sync to catch up: 2022-09-03T05:41:43.309 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000003.20.9, , ] 2022-09-03T05:41:43.309 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000004.22.11, , ] 2022-09-03T05:41:48.311 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:48.310+0000 7fdc8a34f600 1 bucket sync caught up with source: 2022-09-03T05:41:48.312 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000004.22.11, , ] 2022-09-03T05:41:48.312 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000004.22.11, , ] 2022-09-03T05:41:48.312 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:48.310+0000 7fdc8a34f600 0 bucket checkpoint complete 2022-09-03T05:41:48.332 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-23 2022-09-03T05:41:48.336 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-23', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:48.336 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-23', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:48.338 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-23 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:41:48.401 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:48.402 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:48.453 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:48.453+0000 7fdb90335600 1 bucket sync caught up with source: 2022-09-03T05:41:48.454 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000004.22.11, , ] 2022-09-03T05:41:48.454 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000004.22.11, , ] 2022-09-03T05:41:48.455 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:48.453+0000 7fdb90335600 0 bucket checkpoint complete 2022-09-03T05:41:48.470 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-23 2022-09-03T05:41:48.471 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-23 2022-09-03T05:41:48.471 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-23', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:48.472 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-23', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:48.473 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-23 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:41:48.536 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:48.536 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:48.568 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:48.568+0000 7efc903ac600 1 bucket sync caught up with source: 2022-09-03T05:41:48.569 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000003.35.11, , ] 2022-09-03T05:41:48.569 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000003.35.11, , ] 2022-09-03T05:41:48.570 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:48.568+0000 7efc903ac600 0 bucket checkpoint complete 2022-09-03T05:41:48.580 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-23 2022-09-03T05:41:48.581 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-23', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:48.582 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-23', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:48.583 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-23 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:41:48.647 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:48.648 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:48.690 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:48.689+0000 7fc514525600 1 bucket sync caught up with source: 2022-09-03T05:41:48.693 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000003.35.11, , ] 2022-09-03T05:41:48.693 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000003.35.11, , ] 2022-09-03T05:41:48.693 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:48.689+0000 7fc514525600 0 bucket checkpoint complete 2022-09-03T05:41:48.703 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-23 2022-09-03T05:41:48.705 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-23 zones={test-zone2, test-zone1} 2022-09-03T05:41:48.705 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:48.708 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:48.709 DEBUG:boto:Method: HEAD 2022-09-03T05:41:48.710 DEBUG:boto:Path: /dsgvcy-23/ 2022-09-03T05:41:48.711 DEBUG:boto:Data: 2022-09-03T05:41:48.712 DEBUG:boto:Headers: {} 2022-09-03T05:41:48.712 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:48.713 DEBUG:boto:Port: 8000 2022-09-03T05:41:48.714 DEBUG:boto:Params: {} 2022-09-03T05:41:48.716 DEBUG:boto:Token: None 2022-09-03T05:41:48.717 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:48 GMT /dsgvcy-23/ 2022-09-03T05:41:48.718 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UzlAd3Sbv/Lqhjy/0bg5RLeqznA= 2022-09-03T05:41:48.719 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UzlAd3Sbv/Lqhjy/0bg5RLeqznA=', 'Content-Length': '0'} 2022-09-03T05:41:48.722 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000960477e6a6a8018c-006312e91c-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:48.723 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:48.723 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:48.724 DEBUG:boto:Method: HEAD 2022-09-03T05:41:48.724 DEBUG:boto:Path: /dsgvcy-23/ 2022-09-03T05:41:48.725 DEBUG:boto:Data: 2022-09-03T05:41:48.725 DEBUG:boto:Headers: {} 2022-09-03T05:41:48.726 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:48.727 DEBUG:boto:Port: 8000 2022-09-03T05:41:48.727 DEBUG:boto:Params: {} 2022-09-03T05:41:48.728 DEBUG:boto:Token: None 2022-09-03T05:41:48.729 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:48 GMT /dsgvcy-23/ 2022-09-03T05:41:48.729 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UzlAd3Sbv/Lqhjy/0bg5RLeqznA= 2022-09-03T05:41:48.730 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UzlAd3Sbv/Lqhjy/0bg5RLeqznA=', 'Content-Length': '0'} 2022-09-03T05:41:48.735 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f702b937cb9501b1-006312e91c-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:48.736 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:41:48.737 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:48.738 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:48.739 DEBUG:boto:path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:48.740 DEBUG:boto:auth_path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:48.741 DEBUG:boto:Method: GET 2022-09-03T05:41:48.742 DEBUG:boto:Path: /dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:48.743 DEBUG:boto:Data: 2022-09-03T05:41:48.744 DEBUG:boto:Headers: {} 2022-09-03T05:41:48.744 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:48.745 DEBUG:boto:Port: 8000 2022-09-03T05:41:48.745 DEBUG:boto:Params: {} 2022-09-03T05:41:48.746 DEBUG:boto:Token: None 2022-09-03T05:41:48.747 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:48 GMT /dsgvcy-23/?versions 2022-09-03T05:41:48.747 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pvJDy+0B5VwCOXmE+SYrP5A97Vc= 2022-09-03T05:41:48.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pvJDy+0B5VwCOXmE+SYrP5A97Vc=', 'Content-Length': '0'} 2022-09-03T05:41:48.750 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000260e3639c5dbc373-006312e91c-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:48.751 DEBUG:boto:b'dsgvcy-23999falseobjV4KwdIyXLOdi9kXyJFsArpACSrMHdqmtrue2022-09-03T05:41:04.314Zrgw-multisite-test-userMultisite Test UserNormalobjOwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZfalse2022-09-03T05:41:04.285Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:48.752 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:48.753 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:48.753 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:41:48.754 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:48.755 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:48.755 DEBUG:boto:path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:48.756 DEBUG:boto:auth_path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:48.756 DEBUG:boto:Method: GET 2022-09-03T05:41:48.757 DEBUG:boto:Path: /dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:48.757 DEBUG:boto:Data: 2022-09-03T05:41:48.758 DEBUG:boto:Headers: {} 2022-09-03T05:41:48.759 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:48.765 DEBUG:boto:Port: 8000 2022-09-03T05:41:48.766 DEBUG:boto:Params: {} 2022-09-03T05:41:48.767 DEBUG:boto:Token: None 2022-09-03T05:41:48.767 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:48 GMT /dsgvcy-23/?versions 2022-09-03T05:41:48.768 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pvJDy+0B5VwCOXmE+SYrP5A97Vc= 2022-09-03T05:41:48.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pvJDy+0B5VwCOXmE+SYrP5A97Vc=', 'Content-Length': '0'} 2022-09-03T05:41:48.771 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000098db2b9cd6ae5ab7-006312e91c-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:48.772 DEBUG:boto:b'dsgvcy-23999falseobjV4KwdIyXLOdi9kXyJFsArpACSrMHdqmtrue2022-09-03T05:41:04.314Zrgw-multisite-test-userMultisite Test UserNormalobjOwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZfalse2022-09-03T05:41:04.285Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:48.773 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:48.774 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:48.774 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:48.775 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:48.775 DEBUG:boto:path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:48.776 DEBUG:boto:auth_path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:48.776 DEBUG:boto:Method: GET 2022-09-03T05:41:48.777 DEBUG:boto:Path: /dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:48.777 DEBUG:boto:Data: 2022-09-03T05:41:48.778 DEBUG:boto:Headers: {} 2022-09-03T05:41:48.779 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:48.779 DEBUG:boto:Port: 8000 2022-09-03T05:41:48.780 DEBUG:boto:Params: {} 2022-09-03T05:41:48.780 DEBUG:boto:Token: None 2022-09-03T05:41:48.781 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:48 GMT /dsgvcy-23/?versions 2022-09-03T05:41:48.782 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pvJDy+0B5VwCOXmE+SYrP5A97Vc= 2022-09-03T05:41:48.782 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pvJDy+0B5VwCOXmE+SYrP5A97Vc=', 'Content-Length': '0'} 2022-09-03T05:41:48.785 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000095b3cbfac75ccca7-006312e91c-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:48.785 DEBUG:boto:b'dsgvcy-23999falseobjV4KwdIyXLOdi9kXyJFsArpACSrMHdqmtrue2022-09-03T05:41:04.314Zrgw-multisite-test-userMultisite Test UserNormalobjOwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZfalse2022-09-03T05:41:04.285Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:48.786 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:48.787 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:48.787 DEBUG:boto:path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:48.788 DEBUG:boto:auth_path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:48.788 DEBUG:boto:Method: GET 2022-09-03T05:41:48.789 DEBUG:boto:Path: /dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:48.789 DEBUG:boto:Data: 2022-09-03T05:41:48.790 DEBUG:boto:Headers: {} 2022-09-03T05:41:48.791 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:48.791 DEBUG:boto:Port: 8000 2022-09-03T05:41:48.792 DEBUG:boto:Params: {} 2022-09-03T05:41:48.793 DEBUG:boto:Token: None 2022-09-03T05:41:48.793 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:48 GMT /dsgvcy-23/?versions 2022-09-03T05:41:48.794 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pvJDy+0B5VwCOXmE+SYrP5A97Vc= 2022-09-03T05:41:48.794 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pvJDy+0B5VwCOXmE+SYrP5A97Vc=', 'Content-Length': '0'} 2022-09-03T05:41:48.797 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007f3743120d4da595-006312e91c-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:48.798 DEBUG:boto:b'dsgvcy-23999falseobjV4KwdIyXLOdi9kXyJFsArpACSrMHdqmtrue2022-09-03T05:41:04.314Zrgw-multisite-test-userMultisite Test UserNormalobjOwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZfalse2022-09-03T05:41:04.285Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:48.799 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:48.800 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:48.800 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:48.801 DEBUG:boto:Method: HEAD 2022-09-03T05:41:48.801 DEBUG:boto:Path: /dsgvcy-23/obj 2022-09-03T05:41:48.802 DEBUG:boto:Data: 2022-09-03T05:41:48.802 DEBUG:boto:Headers: {} 2022-09-03T05:41:48.803 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:48.803 DEBUG:boto:Port: 8000 2022-09-03T05:41:48.804 DEBUG:boto:Params: {} 2022-09-03T05:41:48.804 DEBUG:boto:Token: None 2022-09-03T05:41:48.805 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:48 GMT /dsgvcy-23/obj 2022-09-03T05:41:48.806 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LB+qNAm/Q98EpI/ttwduDWt7Z+M= 2022-09-03T05:41:48.806 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LB+qNAm/Q98EpI/ttwduDWt7Z+M=', 'Content-Length': '0'} 2022-09-03T05:41:48.809 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx000009af1221a2e29d5a2-006312e91c-10ea-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:48.810 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:48.810 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:48.811 DEBUG:boto:Method: HEAD 2022-09-03T05:41:48.811 DEBUG:boto:Path: /dsgvcy-23/obj 2022-09-03T05:41:48.812 DEBUG:boto:Data: 2022-09-03T05:41:48.813 DEBUG:boto:Headers: {} 2022-09-03T05:41:48.813 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:48.814 DEBUG:boto:Port: 8000 2022-09-03T05:41:48.814 DEBUG:boto:Params: {} 2022-09-03T05:41:48.815 DEBUG:boto:Token: None 2022-09-03T05:41:48.816 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:48 GMT /dsgvcy-23/obj 2022-09-03T05:41:48.817 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LB+qNAm/Q98EpI/ttwduDWt7Z+M= 2022-09-03T05:41:48.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LB+qNAm/Q98EpI/ttwduDWt7Z+M=', 'Content-Length': '0'} 2022-09-03T05:41:48.820 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx0000008bd4bfe649b7dae-006312e91c-10ff-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:48.821 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:48.821 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:48.822 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:48.823 DEBUG:boto:path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.823 DEBUG:boto:auth_path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.824 DEBUG:boto:Method: GET 2022-09-03T05:41:48.824 DEBUG:boto:Path: /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.825 DEBUG:boto:Data: 2022-09-03T05:41:48.825 DEBUG:boto:Headers: {} 2022-09-03T05:41:48.826 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:48.826 DEBUG:boto:Port: 8000 2022-09-03T05:41:48.827 DEBUG:boto:Params: {} 2022-09-03T05:41:48.828 DEBUG:boto:Token: None 2022-09-03T05:41:48.829 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:48 GMT /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.830 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FsJjfYZJEOtWHN2p9TM4NjhKHfM= 2022-09-03T05:41:48.831 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FsJjfYZJEOtWHN2p9TM4NjhKHfM=', 'Content-Length': '0'} 2022-09-03T05:41:48.833 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000021ea027b0cea4ea3-006312e91c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:48.834 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:48.834 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:48.835 DEBUG:boto:path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.836 DEBUG:boto:auth_path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.836 DEBUG:boto:Method: GET 2022-09-03T05:41:48.837 DEBUG:boto:Path: /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.837 DEBUG:boto:Data: 2022-09-03T05:41:48.838 DEBUG:boto:Headers: {} 2022-09-03T05:41:48.838 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:48.839 DEBUG:boto:Port: 8000 2022-09-03T05:41:48.839 DEBUG:boto:Params: {} 2022-09-03T05:41:48.840 DEBUG:boto:Token: None 2022-09-03T05:41:48.841 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:48 GMT /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.841 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FsJjfYZJEOtWHN2p9TM4NjhKHfM= 2022-09-03T05:41:48.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FsJjfYZJEOtWHN2p9TM4NjhKHfM=', 'Content-Length': '0'} 2022-09-03T05:41:48.844 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000004f74fe1f55dcebb4-006312e91c-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:48.844 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:48.845 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:48.846 DEBUG:boto:path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.846 DEBUG:boto:auth_path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.847 DEBUG:boto:Method: HEAD 2022-09-03T05:41:48.848 DEBUG:boto:Path: /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.849 DEBUG:boto:Data: 2022-09-03T05:41:48.850 DEBUG:boto:Headers: {} 2022-09-03T05:41:48.850 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:48.851 DEBUG:boto:Port: 8000 2022-09-03T05:41:48.851 DEBUG:boto:Params: {} 2022-09-03T05:41:48.852 DEBUG:boto:Token: None 2022-09-03T05:41:48.853 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:48 GMT /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.853 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:paMx8K044lBEstD/HLL51TJW+Sk= 2022-09-03T05:41:48.854 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:paMx8K044lBEstD/HLL51TJW+Sk=', 'Content-Length': '0'} 2022-09-03T05:41:48.856 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000e45be051065c06c9-006312e91c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:48.857 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:48.857 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:48.858 DEBUG:boto:path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.859 DEBUG:boto:auth_path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.859 DEBUG:boto:Method: HEAD 2022-09-03T05:41:48.860 DEBUG:boto:Path: /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.860 DEBUG:boto:Data: 2022-09-03T05:41:48.861 DEBUG:boto:Headers: {} 2022-09-03T05:41:48.862 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:48.863 DEBUG:boto:Port: 8000 2022-09-03T05:41:48.864 DEBUG:boto:Params: {} 2022-09-03T05:41:48.865 DEBUG:boto:Token: None 2022-09-03T05:41:48.866 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:48 GMT /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.867 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:paMx8K044lBEstD/HLL51TJW+Sk= 2022-09-03T05:41:48.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:paMx8K044lBEstD/HLL51TJW+Sk=', 'Content-Length': '0'} 2022-09-03T05:41:48.870 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000bf1b5dbb3ee2dbea-006312e91c-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:48.871 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:48.871 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:48.872 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:48.873 DEBUG:boto:path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.873 DEBUG:boto:auth_path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.874 DEBUG:boto:Method: GET 2022-09-03T05:41:48.874 DEBUG:boto:Path: /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.875 DEBUG:boto:Data: 2022-09-03T05:41:48.876 DEBUG:boto:Headers: {} 2022-09-03T05:41:48.877 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:48.877 DEBUG:boto:Port: 8000 2022-09-03T05:41:48.878 DEBUG:boto:Params: {} 2022-09-03T05:41:48.878 DEBUG:boto:Token: None 2022-09-03T05:41:48.879 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:48 GMT /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.880 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FsJjfYZJEOtWHN2p9TM4NjhKHfM= 2022-09-03T05:41:48.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FsJjfYZJEOtWHN2p9TM4NjhKHfM=', 'Content-Length': '0'} 2022-09-03T05:41:48.883 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000571712b17b3d9bbc-006312e91c-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:48.884 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:48.884 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:48.885 DEBUG:boto:path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.885 DEBUG:boto:auth_path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.886 DEBUG:boto:Method: GET 2022-09-03T05:41:48.887 DEBUG:boto:Path: /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.888 DEBUG:boto:Data: 2022-09-03T05:41:48.888 DEBUG:boto:Headers: {} 2022-09-03T05:41:48.889 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:48.890 DEBUG:boto:Port: 8000 2022-09-03T05:41:48.890 DEBUG:boto:Params: {} 2022-09-03T05:41:48.891 DEBUG:boto:Token: None 2022-09-03T05:41:48.891 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:48 GMT /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:48.892 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FsJjfYZJEOtWHN2p9TM4NjhKHfM= 2022-09-03T05:41:48.893 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FsJjfYZJEOtWHN2p9TM4NjhKHfM=', 'Content-Length': '0'} 2022-09-03T05:41:48.895 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000001244d8bf02d0b899-006312e91c-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:48.897 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:48.898 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:48.898 DEBUG:boto:path=/dsgvcy-23/?prefix=obj 2022-09-03T05:41:48.899 DEBUG:boto:auth_path=/dsgvcy-23/?prefix=obj 2022-09-03T05:41:48.899 DEBUG:boto:Method: GET 2022-09-03T05:41:48.900 DEBUG:boto:Path: /dsgvcy-23/?prefix=obj 2022-09-03T05:41:48.900 DEBUG:boto:Data: 2022-09-03T05:41:48.901 DEBUG:boto:Headers: {} 2022-09-03T05:41:48.902 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:48.902 DEBUG:boto:Port: 8000 2022-09-03T05:41:48.903 DEBUG:boto:Params: {} 2022-09-03T05:41:48.904 DEBUG:boto:Token: None 2022-09-03T05:41:48.904 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:48 GMT /dsgvcy-23/ 2022-09-03T05:41:48.905 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1/KntT7kJdSBVEJc21isnnBUX1g= 2022-09-03T05:41:48.906 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1/KntT7kJdSBVEJc21isnnBUX1g=', 'Content-Length': '0'} 2022-09-03T05:41:48.908 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007ffa5eeb9cf3ffbc-006312e91c-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:48.909 DEBUG:boto:b'dsgvcy-23obj1000false' 2022-09-03T05:41:48.910 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:48.911 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:48.912 DEBUG:boto:path=/dsgvcy-23/?prefix=obj 2022-09-03T05:41:48.912 DEBUG:boto:auth_path=/dsgvcy-23/?prefix=obj 2022-09-03T05:41:48.913 DEBUG:boto:Method: GET 2022-09-03T05:41:48.913 DEBUG:boto:Path: /dsgvcy-23/?prefix=obj 2022-09-03T05:41:48.914 DEBUG:boto:Data: 2022-09-03T05:41:48.915 DEBUG:boto:Headers: {} 2022-09-03T05:41:48.915 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:48.916 DEBUG:boto:Port: 8000 2022-09-03T05:41:48.916 DEBUG:boto:Params: {} 2022-09-03T05:41:48.917 DEBUG:boto:Token: None 2022-09-03T05:41:48.918 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:48 GMT /dsgvcy-23/ 2022-09-03T05:41:48.919 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1/KntT7kJdSBVEJc21isnnBUX1g= 2022-09-03T05:41:48.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1/KntT7kJdSBVEJc21isnnBUX1g=', 'Content-Length': '0'} 2022-09-03T05:41:48.922 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000083c14adf7f3255cb-006312e91c-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:48.923 DEBUG:boto:b'dsgvcy-23obj1000false' 2022-09-03T05:41:48.925 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:48.926 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:48.926 DEBUG:boto:Method: HEAD 2022-09-03T05:41:48.927 DEBUG:boto:Path: /dsgvcy-23/obj 2022-09-03T05:41:48.927 DEBUG:boto:Data: 2022-09-03T05:41:48.928 DEBUG:boto:Headers: {} 2022-09-03T05:41:48.928 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:48.929 DEBUG:boto:Port: 8000 2022-09-03T05:41:48.930 DEBUG:boto:Params: {} 2022-09-03T05:41:48.930 DEBUG:boto:Token: None 2022-09-03T05:41:48.931 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:48 GMT /dsgvcy-23/obj 2022-09-03T05:41:48.932 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LB+qNAm/Q98EpI/ttwduDWt7Z+M= 2022-09-03T05:41:48.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LB+qNAm/Q98EpI/ttwduDWt7Z+M=', 'Content-Length': '0'} 2022-09-03T05:41:48.935 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx00000c0d083d16fbd63ad-006312e91c-10ea-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:48.935 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:48.936 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:48.937 DEBUG:boto:Method: HEAD 2022-09-03T05:41:48.937 DEBUG:boto:Path: /dsgvcy-23/obj 2022-09-03T05:41:48.938 DEBUG:boto:Data: 2022-09-03T05:41:48.939 DEBUG:boto:Headers: {} 2022-09-03T05:41:48.940 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:48.941 DEBUG:boto:Port: 8000 2022-09-03T05:41:48.942 DEBUG:boto:Params: {} 2022-09-03T05:41:48.943 DEBUG:boto:Token: None 2022-09-03T05:41:48.944 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:48 GMT /dsgvcy-23/obj 2022-09-03T05:41:48.945 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LB+qNAm/Q98EpI/ttwduDWt7Z+M= 2022-09-03T05:41:48.946 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LB+qNAm/Q98EpI/ttwduDWt7Z+M=', 'Content-Length': '0'} 2022-09-03T05:41:48.948 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx000001b456a2de76698eb-006312e91c-10ff-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:48.949 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-23 zones={test-zone2, test-zone1} 2022-09-03T05:41:48.950 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-23 zones={test-zone3, test-zone1} 2022-09-03T05:41:48.950 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:48.951 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:48.952 DEBUG:boto:Method: HEAD 2022-09-03T05:41:48.953 DEBUG:boto:Path: /dsgvcy-23/ 2022-09-03T05:41:48.954 DEBUG:boto:Data: 2022-09-03T05:41:48.955 DEBUG:boto:Headers: {} 2022-09-03T05:41:48.956 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:48.957 DEBUG:boto:Port: 8001 2022-09-03T05:41:48.957 DEBUG:boto:Params: {} 2022-09-03T05:41:48.958 DEBUG:boto:Token: None 2022-09-03T05:41:48.959 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:48 GMT /dsgvcy-23/ 2022-09-03T05:41:48.960 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UzlAd3Sbv/Lqhjy/0bg5RLeqznA= 2022-09-03T05:41:48.961 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UzlAd3Sbv/Lqhjy/0bg5RLeqznA=', 'Content-Length': '0'} 2022-09-03T05:41:48.963 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002b0a192c1bcd7a29-006312e91c-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:48.964 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:48.964 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:48.965 DEBUG:boto:Method: HEAD 2022-09-03T05:41:48.965 DEBUG:boto:Path: /dsgvcy-23/ 2022-09-03T05:41:48.966 DEBUG:boto:Data: 2022-09-03T05:41:48.967 DEBUG:boto:Headers: {} 2022-09-03T05:41:48.968 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:48.969 DEBUG:boto:Port: 8000 2022-09-03T05:41:48.969 DEBUG:boto:Params: {} 2022-09-03T05:41:48.970 DEBUG:boto:Token: None 2022-09-03T05:41:48.970 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:48 GMT /dsgvcy-23/ 2022-09-03T05:41:48.971 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UzlAd3Sbv/Lqhjy/0bg5RLeqznA= 2022-09-03T05:41:48.971 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UzlAd3Sbv/Lqhjy/0bg5RLeqznA=', 'Content-Length': '0'} 2022-09-03T05:41:48.974 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007b9d1a5b1fe837fd-006312e91c-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:48.975 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:41:48.975 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:48.976 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:48.976 DEBUG:boto:path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:48.977 DEBUG:boto:auth_path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:48.977 DEBUG:boto:Method: GET 2022-09-03T05:41:48.978 DEBUG:boto:Path: /dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:48.979 DEBUG:boto:Data: 2022-09-03T05:41:48.980 DEBUG:boto:Headers: {} 2022-09-03T05:41:48.980 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:48.981 DEBUG:boto:Port: 8001 2022-09-03T05:41:48.981 DEBUG:boto:Params: {} 2022-09-03T05:41:48.982 DEBUG:boto:Token: None 2022-09-03T05:41:48.982 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:48 GMT /dsgvcy-23/?versions 2022-09-03T05:41:48.983 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pvJDy+0B5VwCOXmE+SYrP5A97Vc= 2022-09-03T05:41:48.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pvJDy+0B5VwCOXmE+SYrP5A97Vc=', 'Content-Length': '0'} 2022-09-03T05:41:48.986 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000007795410bf170795-006312e91c-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:48.987 DEBUG:boto:b'dsgvcy-23999falseobjV4KwdIyXLOdi9kXyJFsArpACSrMHdqmtrue2022-09-03T05:41:04.314Zrgw-multisite-test-userMultisite Test UserNormalobjOwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZfalse2022-09-03T05:41:04.285Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:48.988 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:48.989 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:48.990 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:41:48.991 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:48.993 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:48.994 DEBUG:boto:path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:48.995 DEBUG:boto:auth_path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:48.996 DEBUG:boto:Method: GET 2022-09-03T05:41:48.997 DEBUG:boto:Path: /dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:48.998 DEBUG:boto:Data: 2022-09-03T05:41:48.999 DEBUG:boto:Headers: {} 2022-09-03T05:41:48.999 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:49.000 DEBUG:boto:Port: 8000 2022-09-03T05:41:49.000 DEBUG:boto:Params: {} 2022-09-03T05:41:49.001 DEBUG:boto:Token: None 2022-09-03T05:41:49.002 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/?versions 2022-09-03T05:41:49.002 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lbzFSdHEbLRx3uttS20HeNgL45o= 2022-09-03T05:41:49.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lbzFSdHEbLRx3uttS20HeNgL45o=', 'Content-Length': '0'} 2022-09-03T05:41:49.005 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009dcdfd8606494824-006312e91d-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.050 DEBUG:boto:b'dsgvcy-23999falseobjV4KwdIyXLOdi9kXyJFsArpACSrMHdqmtrue2022-09-03T05:41:04.314Zrgw-multisite-test-userMultisite Test UserNormalobjOwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZfalse2022-09-03T05:41:04.285Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:49.052 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:49.052 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:49.054 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:49.055 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:49.056 DEBUG:boto:path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:49.057 DEBUG:boto:auth_path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:49.058 DEBUG:boto:Method: GET 2022-09-03T05:41:49.059 DEBUG:boto:Path: /dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:49.060 DEBUG:boto:Data: 2022-09-03T05:41:49.060 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.061 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:49.062 DEBUG:boto:Port: 8001 2022-09-03T05:41:49.063 DEBUG:boto:Params: {} 2022-09-03T05:41:49.064 DEBUG:boto:Token: None 2022-09-03T05:41:49.066 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/?versions 2022-09-03T05:41:49.067 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lbzFSdHEbLRx3uttS20HeNgL45o= 2022-09-03T05:41:49.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lbzFSdHEbLRx3uttS20HeNgL45o=', 'Content-Length': '0'} 2022-09-03T05:41:49.071 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f748a56ca7aa684b-006312e91d-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.072 DEBUG:boto:b'dsgvcy-23999falseobjV4KwdIyXLOdi9kXyJFsArpACSrMHdqmtrue2022-09-03T05:41:04.314Zrgw-multisite-test-userMultisite Test UserNormalobjOwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZfalse2022-09-03T05:41:04.285Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:49.073 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:49.074 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:49.075 DEBUG:boto:path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:49.075 DEBUG:boto:auth_path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:49.076 DEBUG:boto:Method: GET 2022-09-03T05:41:49.076 DEBUG:boto:Path: /dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:49.077 DEBUG:boto:Data: 2022-09-03T05:41:49.077 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.078 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:49.079 DEBUG:boto:Port: 8000 2022-09-03T05:41:49.079 DEBUG:boto:Params: {} 2022-09-03T05:41:49.080 DEBUG:boto:Token: None 2022-09-03T05:41:49.080 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/?versions 2022-09-03T05:41:49.082 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lbzFSdHEbLRx3uttS20HeNgL45o= 2022-09-03T05:41:49.084 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lbzFSdHEbLRx3uttS20HeNgL45o=', 'Content-Length': '0'} 2022-09-03T05:41:49.087 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007297025465d9a7b5-006312e91d-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.089 DEBUG:boto:b'dsgvcy-23999falseobjV4KwdIyXLOdi9kXyJFsArpACSrMHdqmtrue2022-09-03T05:41:04.314Zrgw-multisite-test-userMultisite Test UserNormalobjOwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZfalse2022-09-03T05:41:04.285Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:49.090 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:49.091 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:49.092 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:49.093 DEBUG:boto:Method: HEAD 2022-09-03T05:41:49.094 DEBUG:boto:Path: /dsgvcy-23/obj 2022-09-03T05:41:49.095 DEBUG:boto:Data: 2022-09-03T05:41:49.096 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.097 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:49.098 DEBUG:boto:Port: 8001 2022-09-03T05:41:49.099 DEBUG:boto:Params: {} 2022-09-03T05:41:49.100 DEBUG:boto:Token: None 2022-09-03T05:41:49.101 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/obj 2022-09-03T05:41:49.102 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I2Z74A9eOlvw6RlCeXyLN4cEw6c= 2022-09-03T05:41:49.103 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I2Z74A9eOlvw6RlCeXyLN4cEw6c=', 'Content-Length': '0'} 2022-09-03T05:41:49.106 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx00000ff36a3edbbb204b7-006312e91d-1103-test-zone3'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.107 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:49.108 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:49.109 DEBUG:boto:Method: HEAD 2022-09-03T05:41:49.110 DEBUG:boto:Path: /dsgvcy-23/obj 2022-09-03T05:41:49.111 DEBUG:boto:Data: 2022-09-03T05:41:49.111 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.112 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:49.113 DEBUG:boto:Port: 8000 2022-09-03T05:41:49.114 DEBUG:boto:Params: {} 2022-09-03T05:41:49.115 DEBUG:boto:Token: None 2022-09-03T05:41:49.116 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/obj 2022-09-03T05:41:49.116 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I2Z74A9eOlvw6RlCeXyLN4cEw6c= 2022-09-03T05:41:49.117 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I2Z74A9eOlvw6RlCeXyLN4cEw6c=', 'Content-Length': '0'} 2022-09-03T05:41:49.119 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx0000031e72e1ab96bdb4a-006312e91d-10ff-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.120 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:49.120 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:49.121 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:49.121 DEBUG:boto:path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.122 DEBUG:boto:auth_path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.122 DEBUG:boto:Method: GET 2022-09-03T05:41:49.123 DEBUG:boto:Path: /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.123 DEBUG:boto:Data: 2022-09-03T05:41:49.124 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.124 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:49.125 DEBUG:boto:Port: 8001 2022-09-03T05:41:49.125 DEBUG:boto:Params: {} 2022-09-03T05:41:49.126 DEBUG:boto:Token: None 2022-09-03T05:41:49.126 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.127 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7I728pu0p2xrSE1QrDzpZDvyeVw= 2022-09-03T05:41:49.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7I728pu0p2xrSE1QrDzpZDvyeVw=', 'Content-Length': '0'} 2022-09-03T05:41:49.131 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000005008ef8835292e00-006312e91d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.132 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:49.133 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:49.133 DEBUG:boto:path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.134 DEBUG:boto:auth_path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.134 DEBUG:boto:Method: GET 2022-09-03T05:41:49.135 DEBUG:boto:Path: /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.135 DEBUG:boto:Data: 2022-09-03T05:41:49.136 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.136 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:49.137 DEBUG:boto:Port: 8000 2022-09-03T05:41:49.137 DEBUG:boto:Params: {} 2022-09-03T05:41:49.139 DEBUG:boto:Token: None 2022-09-03T05:41:49.140 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.141 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7I728pu0p2xrSE1QrDzpZDvyeVw= 2022-09-03T05:41:49.141 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7I728pu0p2xrSE1QrDzpZDvyeVw=', 'Content-Length': '0'} 2022-09-03T05:41:49.143 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000a7385417a9530e01-006312e91d-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.145 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:49.146 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:49.147 DEBUG:boto:path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.147 DEBUG:boto:auth_path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.148 DEBUG:boto:Method: HEAD 2022-09-03T05:41:49.149 DEBUG:boto:Path: /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.149 DEBUG:boto:Data: 2022-09-03T05:41:49.150 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.150 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:49.151 DEBUG:boto:Port: 8001 2022-09-03T05:41:49.151 DEBUG:boto:Params: {} 2022-09-03T05:41:49.152 DEBUG:boto:Token: None 2022-09-03T05:41:49.153 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.153 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9sm+b9yQQHULIm3oyssRaoseVXs= 2022-09-03T05:41:49.154 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9sm+b9yQQHULIm3oyssRaoseVXs=', 'Content-Length': '0'} 2022-09-03T05:41:49.157 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000005d1c34e0d937d224-006312e91d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.159 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:49.160 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:49.161 DEBUG:boto:path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.162 DEBUG:boto:auth_path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.163 DEBUG:boto:Method: HEAD 2022-09-03T05:41:49.164 DEBUG:boto:Path: /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.164 DEBUG:boto:Data: 2022-09-03T05:41:49.165 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.167 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:49.167 DEBUG:boto:Port: 8000 2022-09-03T05:41:49.169 DEBUG:boto:Params: {} 2022-09-03T05:41:49.170 DEBUG:boto:Token: None 2022-09-03T05:41:49.171 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.171 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9sm+b9yQQHULIm3oyssRaoseVXs= 2022-09-03T05:41:49.173 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9sm+b9yQQHULIm3oyssRaoseVXs=', 'Content-Length': '0'} 2022-09-03T05:41:49.176 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000000730ef239e8493c9-006312e91d-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.178 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:49.179 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:49.180 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:49.181 DEBUG:boto:path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.183 DEBUG:boto:auth_path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.184 DEBUG:boto:Method: GET 2022-09-03T05:41:49.185 DEBUG:boto:Path: /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.186 DEBUG:boto:Data: 2022-09-03T05:41:49.187 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.188 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:49.189 DEBUG:boto:Port: 8001 2022-09-03T05:41:49.190 DEBUG:boto:Params: {} 2022-09-03T05:41:49.191 DEBUG:boto:Token: None 2022-09-03T05:41:49.192 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.192 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7I728pu0p2xrSE1QrDzpZDvyeVw= 2022-09-03T05:41:49.193 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7I728pu0p2xrSE1QrDzpZDvyeVw=', 'Content-Length': '0'} 2022-09-03T05:41:49.195 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000005beb91fa1dccc6cb-006312e91d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.196 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:49.196 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:49.197 DEBUG:boto:path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.197 DEBUG:boto:auth_path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.198 DEBUG:boto:Method: GET 2022-09-03T05:41:49.199 DEBUG:boto:Path: /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.199 DEBUG:boto:Data: 2022-09-03T05:41:49.200 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.200 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:49.201 DEBUG:boto:Port: 8000 2022-09-03T05:41:49.202 DEBUG:boto:Params: {} 2022-09-03T05:41:49.203 DEBUG:boto:Token: None 2022-09-03T05:41:49.204 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.205 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7I728pu0p2xrSE1QrDzpZDvyeVw= 2022-09-03T05:41:49.207 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7I728pu0p2xrSE1QrDzpZDvyeVw=', 'Content-Length': '0'} 2022-09-03T05:41:49.209 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000002d0676b7d29f2aab-006312e91d-10ff-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.210 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:49.210 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:49.211 DEBUG:boto:path=/dsgvcy-23/?prefix=obj 2022-09-03T05:41:49.211 DEBUG:boto:auth_path=/dsgvcy-23/?prefix=obj 2022-09-03T05:41:49.212 DEBUG:boto:Method: GET 2022-09-03T05:41:49.212 DEBUG:boto:Path: /dsgvcy-23/?prefix=obj 2022-09-03T05:41:49.213 DEBUG:boto:Data: 2022-09-03T05:41:49.213 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.214 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:49.214 DEBUG:boto:Port: 8001 2022-09-03T05:41:49.215 DEBUG:boto:Params: {} 2022-09-03T05:41:49.215 DEBUG:boto:Token: None 2022-09-03T05:41:49.216 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/ 2022-09-03T05:41:49.216 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Xpe+4axYwDR9BRTYXOhmqijVgy4= 2022-09-03T05:41:49.217 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Xpe+4axYwDR9BRTYXOhmqijVgy4=', 'Content-Length': '0'} 2022-09-03T05:41:49.219 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000006b60be298b49f81-006312e91d-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.266 DEBUG:boto:b'dsgvcy-23obj1000false' 2022-09-03T05:41:49.268 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:49.268 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:49.269 DEBUG:boto:path=/dsgvcy-23/?prefix=obj 2022-09-03T05:41:49.270 DEBUG:boto:auth_path=/dsgvcy-23/?prefix=obj 2022-09-03T05:41:49.271 DEBUG:boto:Method: GET 2022-09-03T05:41:49.271 DEBUG:boto:Path: /dsgvcy-23/?prefix=obj 2022-09-03T05:41:49.272 DEBUG:boto:Data: 2022-09-03T05:41:49.272 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.273 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:49.274 DEBUG:boto:Port: 8000 2022-09-03T05:41:49.275 DEBUG:boto:Params: {} 2022-09-03T05:41:49.276 DEBUG:boto:Token: None 2022-09-03T05:41:49.277 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/ 2022-09-03T05:41:49.277 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Xpe+4axYwDR9BRTYXOhmqijVgy4= 2022-09-03T05:41:49.278 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Xpe+4axYwDR9BRTYXOhmqijVgy4=', 'Content-Length': '0'} 2022-09-03T05:41:49.281 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000008074b578da9d7ad-006312e91d-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.281 DEBUG:boto:b'dsgvcy-23obj1000false' 2022-09-03T05:41:49.282 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:49.283 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:49.284 DEBUG:boto:Method: HEAD 2022-09-03T05:41:49.284 DEBUG:boto:Path: /dsgvcy-23/obj 2022-09-03T05:41:49.285 DEBUG:boto:Data: 2022-09-03T05:41:49.285 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.286 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:49.286 DEBUG:boto:Port: 8001 2022-09-03T05:41:49.287 DEBUG:boto:Params: {} 2022-09-03T05:41:49.288 DEBUG:boto:Token: None 2022-09-03T05:41:49.289 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/obj 2022-09-03T05:41:49.289 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I2Z74A9eOlvw6RlCeXyLN4cEw6c= 2022-09-03T05:41:49.290 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I2Z74A9eOlvw6RlCeXyLN4cEw6c=', 'Content-Length': '0'} 2022-09-03T05:41:49.292 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx0000038b79683d717cba1-006312e91d-1103-test-zone3'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.293 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:49.294 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:49.295 DEBUG:boto:Method: HEAD 2022-09-03T05:41:49.296 DEBUG:boto:Path: /dsgvcy-23/obj 2022-09-03T05:41:49.297 DEBUG:boto:Data: 2022-09-03T05:41:49.298 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.299 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:49.300 DEBUG:boto:Port: 8000 2022-09-03T05:41:49.301 DEBUG:boto:Params: {} 2022-09-03T05:41:49.301 DEBUG:boto:Token: None 2022-09-03T05:41:49.302 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/obj 2022-09-03T05:41:49.303 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I2Z74A9eOlvw6RlCeXyLN4cEw6c= 2022-09-03T05:41:49.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I2Z74A9eOlvw6RlCeXyLN4cEw6c=', 'Content-Length': '0'} 2022-09-03T05:41:49.307 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx0000011187f017a12ce1e-006312e91d-10ff-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.308 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-23 zones={test-zone3, test-zone1} 2022-09-03T05:41:49.309 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-23 zones={test-zone3, test-zone2} 2022-09-03T05:41:49.310 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:49.310 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:49.311 DEBUG:boto:Method: HEAD 2022-09-03T05:41:49.311 DEBUG:boto:Path: /dsgvcy-23/ 2022-09-03T05:41:49.312 DEBUG:boto:Data: 2022-09-03T05:41:49.312 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.313 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:49.313 DEBUG:boto:Port: 8001 2022-09-03T05:41:49.314 DEBUG:boto:Params: {} 2022-09-03T05:41:49.315 DEBUG:boto:Token: None 2022-09-03T05:41:49.315 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/ 2022-09-03T05:41:49.316 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:W9syTqNAo4tosFs4b20kONVGQNY= 2022-09-03T05:41:49.317 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:W9syTqNAo4tosFs4b20kONVGQNY=', 'Content-Length': '0'} 2022-09-03T05:41:49.319 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000bd6581e3b0d2075e-006312e91d-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.320 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:49.320 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:49.321 DEBUG:boto:Method: HEAD 2022-09-03T05:41:49.321 DEBUG:boto:Path: /dsgvcy-23/ 2022-09-03T05:41:49.322 DEBUG:boto:Data: 2022-09-03T05:41:49.322 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.323 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:49.323 DEBUG:boto:Port: 8000 2022-09-03T05:41:49.324 DEBUG:boto:Params: {} 2022-09-03T05:41:49.325 DEBUG:boto:Token: None 2022-09-03T05:41:49.326 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/ 2022-09-03T05:41:49.327 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:W9syTqNAo4tosFs4b20kONVGQNY= 2022-09-03T05:41:49.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:W9syTqNAo4tosFs4b20kONVGQNY=', 'Content-Length': '0'} 2022-09-03T05:41:49.331 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e3160c8fecd0cb0b-006312e91d-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.332 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:41:49.332 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:49.333 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:49.333 DEBUG:boto:path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:49.334 DEBUG:boto:auth_path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:49.335 DEBUG:boto:Method: GET 2022-09-03T05:41:49.335 DEBUG:boto:Path: /dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:49.336 DEBUG:boto:Data: 2022-09-03T05:41:49.336 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.337 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:49.337 DEBUG:boto:Port: 8001 2022-09-03T05:41:49.338 DEBUG:boto:Params: {} 2022-09-03T05:41:49.339 DEBUG:boto:Token: None 2022-09-03T05:41:49.339 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/?versions 2022-09-03T05:41:49.340 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lbzFSdHEbLRx3uttS20HeNgL45o= 2022-09-03T05:41:49.341 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lbzFSdHEbLRx3uttS20HeNgL45o=', 'Content-Length': '0'} 2022-09-03T05:41:49.343 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000009fc00b9d7376e02-006312e91d-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.344 DEBUG:boto:b'dsgvcy-23999falseobjV4KwdIyXLOdi9kXyJFsArpACSrMHdqmtrue2022-09-03T05:41:04.314Zrgw-multisite-test-userMultisite Test UserNormalobjOwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZfalse2022-09-03T05:41:04.285Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:49.345 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:49.346 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:49.346 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:41:49.347 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:49.348 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:49.349 DEBUG:boto:path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:49.349 DEBUG:boto:auth_path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:49.350 DEBUG:boto:Method: GET 2022-09-03T05:41:49.350 DEBUG:boto:Path: /dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:49.351 DEBUG:boto:Data: 2022-09-03T05:41:49.352 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.353 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:49.354 DEBUG:boto:Port: 8000 2022-09-03T05:41:49.355 DEBUG:boto:Params: {} 2022-09-03T05:41:49.356 DEBUG:boto:Token: None 2022-09-03T05:41:49.357 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/?versions 2022-09-03T05:41:49.358 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lbzFSdHEbLRx3uttS20HeNgL45o= 2022-09-03T05:41:49.359 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lbzFSdHEbLRx3uttS20HeNgL45o=', 'Content-Length': '0'} 2022-09-03T05:41:49.362 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b5a54b701764e5e2-006312e91d-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.406 DEBUG:boto:b'dsgvcy-23999falseobjV4KwdIyXLOdi9kXyJFsArpACSrMHdqmtrue2022-09-03T05:41:04.314Zrgw-multisite-test-userMultisite Test UserNormalobjOwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZfalse2022-09-03T05:41:04.285Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:49.408 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:49.409 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:41:49.410 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:49.411 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:49.413 DEBUG:boto:path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:49.413 DEBUG:boto:auth_path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:49.414 DEBUG:boto:Method: GET 2022-09-03T05:41:49.414 DEBUG:boto:Path: /dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:49.415 DEBUG:boto:Data: 2022-09-03T05:41:49.415 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.416 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:49.416 DEBUG:boto:Port: 8001 2022-09-03T05:41:49.417 DEBUG:boto:Params: {} 2022-09-03T05:41:49.418 DEBUG:boto:Token: None 2022-09-03T05:41:49.418 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/?versions 2022-09-03T05:41:49.419 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lbzFSdHEbLRx3uttS20HeNgL45o= 2022-09-03T05:41:49.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lbzFSdHEbLRx3uttS20HeNgL45o=', 'Content-Length': '0'} 2022-09-03T05:41:49.422 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002c5e52cd97a3c8b5-006312e91d-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.470 DEBUG:boto:b'dsgvcy-23999falseobjV4KwdIyXLOdi9kXyJFsArpACSrMHdqmtrue2022-09-03T05:41:04.314Zrgw-multisite-test-userMultisite Test UserNormalobjOwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZfalse2022-09-03T05:41:04.285Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:49.472 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:49.472 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:49.473 DEBUG:boto:path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:49.473 DEBUG:boto:auth_path=/dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:49.475 DEBUG:boto:Method: GET 2022-09-03T05:41:49.475 DEBUG:boto:Path: /dsgvcy-23/?versions&max-keys=999 2022-09-03T05:41:49.476 DEBUG:boto:Data: 2022-09-03T05:41:49.476 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.477 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:49.477 DEBUG:boto:Port: 8000 2022-09-03T05:41:49.478 DEBUG:boto:Params: {} 2022-09-03T05:41:49.478 DEBUG:boto:Token: None 2022-09-03T05:41:49.479 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/?versions 2022-09-03T05:41:49.480 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lbzFSdHEbLRx3uttS20HeNgL45o= 2022-09-03T05:41:49.481 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lbzFSdHEbLRx3uttS20HeNgL45o=', 'Content-Length': '0'} 2022-09-03T05:41:49.483 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b4ea3eaba9caa6d5-006312e91d-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.484 DEBUG:boto:b'dsgvcy-23999falseobjV4KwdIyXLOdi9kXyJFsArpACSrMHdqmtrue2022-09-03T05:41:04.314Zrgw-multisite-test-userMultisite Test UserNormalobjOwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZfalse2022-09-03T05:41:04.285Z"d41d8cd98f00b204e9800998ecf8427e"0STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:41:49.485 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:49.486 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:49.487 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:49.488 DEBUG:boto:Method: HEAD 2022-09-03T05:41:49.489 DEBUG:boto:Path: /dsgvcy-23/obj 2022-09-03T05:41:49.490 DEBUG:boto:Data: 2022-09-03T05:41:49.491 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.492 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:49.492 DEBUG:boto:Port: 8001 2022-09-03T05:41:49.493 DEBUG:boto:Params: {} 2022-09-03T05:41:49.493 DEBUG:boto:Token: None 2022-09-03T05:41:49.494 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/obj 2022-09-03T05:41:49.495 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I2Z74A9eOlvw6RlCeXyLN4cEw6c= 2022-09-03T05:41:49.495 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I2Z74A9eOlvw6RlCeXyLN4cEw6c=', 'Content-Length': '0'} 2022-09-03T05:41:49.497 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx00000640c3bea2e33be81-006312e91d-1103-test-zone3'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.498 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:49.498 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:49.499 DEBUG:boto:Method: HEAD 2022-09-03T05:41:49.500 DEBUG:boto:Path: /dsgvcy-23/obj 2022-09-03T05:41:49.500 DEBUG:boto:Data: 2022-09-03T05:41:49.500 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.501 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:49.501 DEBUG:boto:Port: 8000 2022-09-03T05:41:49.502 DEBUG:boto:Params: {} 2022-09-03T05:41:49.503 DEBUG:boto:Token: None 2022-09-03T05:41:49.504 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/obj 2022-09-03T05:41:49.504 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I2Z74A9eOlvw6RlCeXyLN4cEw6c= 2022-09-03T05:41:49.505 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I2Z74A9eOlvw6RlCeXyLN4cEw6c=', 'Content-Length': '0'} 2022-09-03T05:41:49.507 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx00000e49a4e6c0abfabd7-006312e91d-10ea-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.508 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:49.508 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:49.509 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:49.510 DEBUG:boto:path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.510 DEBUG:boto:auth_path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.511 DEBUG:boto:Method: GET 2022-09-03T05:41:49.511 DEBUG:boto:Path: /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.512 DEBUG:boto:Data: 2022-09-03T05:41:49.513 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.514 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:49.515 DEBUG:boto:Port: 8001 2022-09-03T05:41:49.516 DEBUG:boto:Params: {} 2022-09-03T05:41:49.517 DEBUG:boto:Token: None 2022-09-03T05:41:49.518 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.519 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7I728pu0p2xrSE1QrDzpZDvyeVw= 2022-09-03T05:41:49.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7I728pu0p2xrSE1QrDzpZDvyeVw=', 'Content-Length': '0'} 2022-09-03T05:41:49.522 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000c45a259e63521390-006312e91d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.523 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:49.524 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:49.524 DEBUG:boto:path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.525 DEBUG:boto:auth_path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.525 DEBUG:boto:Method: GET 2022-09-03T05:41:49.526 DEBUG:boto:Path: /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.527 DEBUG:boto:Data: 2022-09-03T05:41:49.527 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.528 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:49.528 DEBUG:boto:Port: 8000 2022-09-03T05:41:49.529 DEBUG:boto:Params: {} 2022-09-03T05:41:49.529 DEBUG:boto:Token: None 2022-09-03T05:41:49.530 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.530 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7I728pu0p2xrSE1QrDzpZDvyeVw= 2022-09-03T05:41:49.531 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7I728pu0p2xrSE1QrDzpZDvyeVw=', 'Content-Length': '0'} 2022-09-03T05:41:49.533 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000852bf71ffe2d2364-006312e91d-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.534 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:49.535 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:49.535 DEBUG:boto:path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.536 DEBUG:boto:auth_path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.536 DEBUG:boto:Method: HEAD 2022-09-03T05:41:49.537 DEBUG:boto:Path: /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.537 DEBUG:boto:Data: 2022-09-03T05:41:49.538 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.538 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:49.539 DEBUG:boto:Port: 8001 2022-09-03T05:41:49.539 DEBUG:boto:Params: {} 2022-09-03T05:41:49.540 DEBUG:boto:Token: None 2022-09-03T05:41:49.541 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.541 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9sm+b9yQQHULIm3oyssRaoseVXs= 2022-09-03T05:41:49.542 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9sm+b9yQQHULIm3oyssRaoseVXs=', 'Content-Length': '0'} 2022-09-03T05:41:49.546 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000009364739a8233dce3-006312e91d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.548 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:49.548 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:49.549 DEBUG:boto:path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.549 DEBUG:boto:auth_path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.550 DEBUG:boto:Method: HEAD 2022-09-03T05:41:49.551 DEBUG:boto:Path: /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.551 DEBUG:boto:Data: 2022-09-03T05:41:49.552 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.554 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:49.555 DEBUG:boto:Port: 8000 2022-09-03T05:41:49.555 DEBUG:boto:Params: {} 2022-09-03T05:41:49.556 DEBUG:boto:Token: None 2022-09-03T05:41:49.557 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.557 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9sm+b9yQQHULIm3oyssRaoseVXs= 2022-09-03T05:41:49.558 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9sm+b9yQQHULIm3oyssRaoseVXs=', 'Content-Length': '0'} 2022-09-03T05:41:49.561 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx00000182dcfc7ee91c243-006312e91d-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.562 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:41:49.563 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:49.564 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:49.565 DEBUG:boto:path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.566 DEBUG:boto:auth_path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.566 DEBUG:boto:Method: GET 2022-09-03T05:41:49.567 DEBUG:boto:Path: /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.568 DEBUG:boto:Data: 2022-09-03T05:41:49.568 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.569 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:49.570 DEBUG:boto:Port: 8001 2022-09-03T05:41:49.571 DEBUG:boto:Params: {} 2022-09-03T05:41:49.572 DEBUG:boto:Token: None 2022-09-03T05:41:49.572 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.573 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7I728pu0p2xrSE1QrDzpZDvyeVw= 2022-09-03T05:41:49.574 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7I728pu0p2xrSE1QrDzpZDvyeVw=', 'Content-Length': '0'} 2022-09-03T05:41:49.575 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx000003428a95de26cc7f8-006312e91d-1103-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.577 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:49.578 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:49.578 DEBUG:boto:path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.579 DEBUG:boto:auth_path=/dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.580 DEBUG:boto:Method: GET 2022-09-03T05:41:49.581 DEBUG:boto:Path: /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.581 DEBUG:boto:Data: 2022-09-03T05:41:49.582 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.583 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:49.583 DEBUG:boto:Port: 8000 2022-09-03T05:41:49.584 DEBUG:boto:Params: {} 2022-09-03T05:41:49.584 DEBUG:boto:Token: None 2022-09-03T05:41:49.585 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/obj?versionId=OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ 2022-09-03T05:41:49.586 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7I728pu0p2xrSE1QrDzpZDvyeVw= 2022-09-03T05:41:49.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7I728pu0p2xrSE1QrDzpZDvyeVw=', 'Content-Length': '0'} 2022-09-03T05:41:49.589 DEBUG:boto:Response headers: [('Content-Length', '0'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:41:04 GMT'), ('x-amz-version-id', 'OwpHZuYbtZJg9Lw4v8vgLT4pIkhpUaZ'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('x-amz-request-id', 'tx0000091654d5d5de19635-006312e91d-10ea-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.590 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:49.591 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:49.591 DEBUG:boto:path=/dsgvcy-23/?prefix=obj 2022-09-03T05:41:49.592 DEBUG:boto:auth_path=/dsgvcy-23/?prefix=obj 2022-09-03T05:41:49.593 DEBUG:boto:Method: GET 2022-09-03T05:41:49.594 DEBUG:boto:Path: /dsgvcy-23/?prefix=obj 2022-09-03T05:41:49.594 DEBUG:boto:Data: 2022-09-03T05:41:49.595 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.596 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:49.596 DEBUG:boto:Port: 8001 2022-09-03T05:41:49.597 DEBUG:boto:Params: {} 2022-09-03T05:41:49.597 DEBUG:boto:Token: None 2022-09-03T05:41:49.598 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/ 2022-09-03T05:41:49.599 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Xpe+4axYwDR9BRTYXOhmqijVgy4= 2022-09-03T05:41:49.600 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Xpe+4axYwDR9BRTYXOhmqijVgy4=', 'Content-Length': '0'} 2022-09-03T05:41:49.602 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f6bb99b251e49451-006312e91d-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.603 DEBUG:boto:b'dsgvcy-23obj1000false' 2022-09-03T05:41:49.604 DEBUG:boto:path=/dsgvcy-23/ 2022-09-03T05:41:49.605 DEBUG:boto:auth_path=/dsgvcy-23/ 2022-09-03T05:41:49.605 DEBUG:boto:path=/dsgvcy-23/?prefix=obj 2022-09-03T05:41:49.606 DEBUG:boto:auth_path=/dsgvcy-23/?prefix=obj 2022-09-03T05:41:49.607 DEBUG:boto:Method: GET 2022-09-03T05:41:49.607 DEBUG:boto:Path: /dsgvcy-23/?prefix=obj 2022-09-03T05:41:49.608 DEBUG:boto:Data: 2022-09-03T05:41:49.608 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.609 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:49.609 DEBUG:boto:Port: 8000 2022-09-03T05:41:49.610 DEBUG:boto:Params: {} 2022-09-03T05:41:49.611 DEBUG:boto:Token: None 2022-09-03T05:41:49.612 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/ 2022-09-03T05:41:49.613 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Xpe+4axYwDR9BRTYXOhmqijVgy4= 2022-09-03T05:41:49.613 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Xpe+4axYwDR9BRTYXOhmqijVgy4=', 'Content-Length': '0'} 2022-09-03T05:41:49.616 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e0bfdfd4a7042998-006312e91d-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.617 DEBUG:boto:b'dsgvcy-23obj1000false' 2022-09-03T05:41:49.618 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:49.618 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:49.619 DEBUG:boto:Method: HEAD 2022-09-03T05:41:49.619 DEBUG:boto:Path: /dsgvcy-23/obj 2022-09-03T05:41:49.620 DEBUG:boto:Data: 2022-09-03T05:41:49.620 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.621 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:49.621 DEBUG:boto:Port: 8001 2022-09-03T05:41:49.623 DEBUG:boto:Params: {} 2022-09-03T05:41:49.623 DEBUG:boto:Token: None 2022-09-03T05:41:49.624 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/obj 2022-09-03T05:41:49.624 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I2Z74A9eOlvw6RlCeXyLN4cEw6c= 2022-09-03T05:41:49.625 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I2Z74A9eOlvw6RlCeXyLN4cEw6c=', 'Content-Length': '0'} 2022-09-03T05:41:49.628 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx00000ecd6dfa56a921b69-006312e91d-1103-test-zone3'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.629 DEBUG:boto:path=/dsgvcy-23/obj 2022-09-03T05:41:49.629 DEBUG:boto:auth_path=/dsgvcy-23/obj 2022-09-03T05:41:49.630 DEBUG:boto:Method: HEAD 2022-09-03T05:41:49.630 DEBUG:boto:Path: /dsgvcy-23/obj 2022-09-03T05:41:49.631 DEBUG:boto:Data: 2022-09-03T05:41:49.631 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.632 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:49.633 DEBUG:boto:Port: 8000 2022-09-03T05:41:49.634 DEBUG:boto:Params: {} 2022-09-03T05:41:49.635 DEBUG:boto:Token: None 2022-09-03T05:41:49.635 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-23/obj 2022-09-03T05:41:49.636 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I2Z74A9eOlvw6RlCeXyLN4cEw6c= 2022-09-03T05:41:49.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I2Z74A9eOlvw6RlCeXyLN4cEw6c=', 'Content-Length': '0'} 2022-09-03T05:41:49.639 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx00000fcbf4feb753671f6-006312e91d-10ea-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.640 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-23 zones={test-zone3, test-zone2} 2022-09-03T05:41:49.641 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_delete_marker_full_sync ... ok 2022-09-03T05:41:49.642 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-24 2022-09-03T05:41:49.642 DEBUG:boto:path=/dsgvcy-24/ 2022-09-03T05:41:49.643 DEBUG:boto:auth_path=/dsgvcy-24/ 2022-09-03T05:41:49.644 DEBUG:boto:Method: PUT 2022-09-03T05:41:49.645 DEBUG:boto:Path: /dsgvcy-24/ 2022-09-03T05:41:49.645 DEBUG:boto:Data: 2022-09-03T05:41:49.646 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.646 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:49.647 DEBUG:boto:Port: 8000 2022-09-03T05:41:49.647 DEBUG:boto:Params: {} 2022-09-03T05:41:49.648 DEBUG:boto:Token: None 2022-09-03T05:41:49.649 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-24/ 2022-09-03T05:41:49.650 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pANgm9azqVkB+K+eCES6Z3FpSfs= 2022-09-03T05:41:49.650 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pANgm9azqVkB+K+eCES6Z3FpSfs=', 'Content-Length': '0'} 2022-09-03T05:41:49.667 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000505d76440e8a54af-006312e91d-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.668 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=dsgvcy-25 2022-09-03T05:41:49.669 DEBUG:boto:path=/dsgvcy-25/ 2022-09-03T05:41:49.669 DEBUG:boto:auth_path=/dsgvcy-25/ 2022-09-03T05:41:49.670 DEBUG:boto:Method: PUT 2022-09-03T05:41:49.671 DEBUG:boto:Path: /dsgvcy-25/ 2022-09-03T05:41:49.671 DEBUG:boto:Data: 2022-09-03T05:41:49.672 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.672 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:49.673 DEBUG:boto:Port: 8000 2022-09-03T05:41:49.673 DEBUG:boto:Params: {} 2022-09-03T05:41:49.674 DEBUG:boto:Token: None 2022-09-03T05:41:49.675 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-25/ 2022-09-03T05:41:49.676 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y5Uh2ASqdwDxZ2tOHQcKy2S/nU0= 2022-09-03T05:41:49.676 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y5Uh2ASqdwDxZ2tOHQcKy2S/nU0=', 'Content-Length': '0'} 2022-09-03T05:41:49.704 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000000282f79a3e8fa4bd-006312e91d-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.705 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=dsgvcy-26 2022-09-03T05:41:49.706 DEBUG:boto:path=/dsgvcy-26/ 2022-09-03T05:41:49.707 DEBUG:boto:auth_path=/dsgvcy-26/ 2022-09-03T05:41:49.709 DEBUG:boto:Method: PUT 2022-09-03T05:41:49.710 DEBUG:boto:Path: /dsgvcy-26/ 2022-09-03T05:41:49.712 DEBUG:boto:Data: 2022-09-03T05:41:49.713 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.713 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:49.714 DEBUG:boto:Port: 8001 2022-09-03T05:41:49.715 DEBUG:boto:Params: {} 2022-09-03T05:41:49.716 DEBUG:boto:Token: None 2022-09-03T05:41:49.717 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-26/ 2022-09-03T05:41:49.717 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UknsaEeOPJuwpL+xMuPcr7lRrwM= 2022-09-03T05:41:49.718 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UknsaEeOPJuwpL+xMuPcr7lRrwM=', 'Content-Length': '0'} 2022-09-03T05:41:49.752 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000b3f069f589ba0739-006312e91d-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.753 DEBUG:boto:path=/dsgvcy-24/ 2022-09-03T05:41:49.753 DEBUG:boto:auth_path=/dsgvcy-24/ 2022-09-03T05:41:49.754 DEBUG:boto:path=/dsgvcy-24/?versioning 2022-09-03T05:41:49.754 DEBUG:boto:auth_path=/dsgvcy-24/?versioning 2022-09-03T05:41:49.755 DEBUG:boto:Method: PUT 2022-09-03T05:41:49.756 DEBUG:boto:Path: /dsgvcy-24/?versioning 2022-09-03T05:41:49.756 DEBUG:boto:Data: Enabled Disabled 2022-09-03T05:41:49.757 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.757 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:49.758 DEBUG:boto:Port: 8000 2022-09-03T05:41:49.758 DEBUG:boto:Params: {} 2022-09-03T05:41:49.759 DEBUG:boto:Token: None 2022-09-03T05:41:49.760 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-24/?versioning 2022-09-03T05:41:49.760 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uetXLGsACKiVs0gATp/vnYvuARA= 2022-09-03T05:41:49.761 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uetXLGsACKiVs0gATp/vnYvuARA=', 'Content-Length': '228'} 2022-09-03T05:41:49.769 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000ccfdb67a95bad1aa-006312e91d-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.770 DEBUG:boto:path=/dsgvcy-24/ 2022-09-03T05:41:49.771 DEBUG:boto:auth_path=/dsgvcy-24/ 2022-09-03T05:41:49.771 DEBUG:boto:path=/dsgvcy-24/?versioning 2022-09-03T05:41:49.772 DEBUG:boto:auth_path=/dsgvcy-24/?versioning 2022-09-03T05:41:49.772 DEBUG:boto:Method: PUT 2022-09-03T05:41:49.773 DEBUG:boto:Path: /dsgvcy-24/?versioning 2022-09-03T05:41:49.773 DEBUG:boto:Data: Suspended Disabled 2022-09-03T05:41:49.774 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.774 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:49.775 DEBUG:boto:Port: 8000 2022-09-03T05:41:49.775 DEBUG:boto:Params: {} 2022-09-03T05:41:49.776 DEBUG:boto:Token: None 2022-09-03T05:41:49.777 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-24/?versioning 2022-09-03T05:41:49.777 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uetXLGsACKiVs0gATp/vnYvuARA= 2022-09-03T05:41:49.778 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uetXLGsACKiVs0gATp/vnYvuARA=', 'Content-Length': '230'} 2022-09-03T05:41:49.786 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000718b99c4c20bfea8-006312e91d-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.787 DEBUG:boto:path=/dsgvcy-25/ 2022-09-03T05:41:49.788 DEBUG:boto:auth_path=/dsgvcy-25/ 2022-09-03T05:41:49.788 DEBUG:boto:path=/dsgvcy-25/?versioning 2022-09-03T05:41:49.789 DEBUG:boto:auth_path=/dsgvcy-25/?versioning 2022-09-03T05:41:49.789 DEBUG:boto:Method: PUT 2022-09-03T05:41:49.790 DEBUG:boto:Path: /dsgvcy-25/?versioning 2022-09-03T05:41:49.790 DEBUG:boto:Data: Enabled Disabled 2022-09-03T05:41:49.791 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.791 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:49.792 DEBUG:boto:Port: 8000 2022-09-03T05:41:49.792 DEBUG:boto:Params: {} 2022-09-03T05:41:49.793 DEBUG:boto:Token: None 2022-09-03T05:41:49.793 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-25/?versioning 2022-09-03T05:41:49.794 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:imfoXcSfp+/aFyOs8/O4OTntfDc= 2022-09-03T05:41:49.794 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:imfoXcSfp+/aFyOs8/O4OTntfDc=', 'Content-Length': '228'} 2022-09-03T05:41:49.806 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000039de3765f095f423-006312e91d-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.807 DEBUG:boto:path=/dsgvcy-25/ 2022-09-03T05:41:49.807 DEBUG:boto:auth_path=/dsgvcy-25/ 2022-09-03T05:41:49.808 DEBUG:boto:path=/dsgvcy-25/?versioning 2022-09-03T05:41:49.808 DEBUG:boto:auth_path=/dsgvcy-25/?versioning 2022-09-03T05:41:49.809 DEBUG:boto:Method: PUT 2022-09-03T05:41:49.809 DEBUG:boto:Path: /dsgvcy-25/?versioning 2022-09-03T05:41:49.810 DEBUG:boto:Data: Suspended Disabled 2022-09-03T05:41:49.811 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.812 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:49.812 DEBUG:boto:Port: 8000 2022-09-03T05:41:49.813 DEBUG:boto:Params: {} 2022-09-03T05:41:49.813 DEBUG:boto:Token: None 2022-09-03T05:41:49.814 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-25/?versioning 2022-09-03T05:41:49.815 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:imfoXcSfp+/aFyOs8/O4OTntfDc= 2022-09-03T05:41:49.815 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:imfoXcSfp+/aFyOs8/O4OTntfDc=', 'Content-Length': '230'} 2022-09-03T05:41:49.843 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000ba573b9500b74465-006312e91d-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.844 DEBUG:boto:path=/dsgvcy-26/ 2022-09-03T05:41:49.844 DEBUG:boto:auth_path=/dsgvcy-26/ 2022-09-03T05:41:49.845 DEBUG:boto:path=/dsgvcy-26/?versioning 2022-09-03T05:41:49.845 DEBUG:boto:auth_path=/dsgvcy-26/?versioning 2022-09-03T05:41:49.846 DEBUG:boto:Method: PUT 2022-09-03T05:41:49.846 DEBUG:boto:Path: /dsgvcy-26/?versioning 2022-09-03T05:41:49.847 DEBUG:boto:Data: Enabled Disabled 2022-09-03T05:41:49.847 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.848 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:49.848 DEBUG:boto:Port: 8001 2022-09-03T05:41:49.849 DEBUG:boto:Params: {} 2022-09-03T05:41:49.849 DEBUG:boto:Token: None 2022-09-03T05:41:49.850 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-26/?versioning 2022-09-03T05:41:49.850 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A3HYAkvRmdgY1mPcahq+wcj1U2E= 2022-09-03T05:41:49.851 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A3HYAkvRmdgY1mPcahq+wcj1U2E=', 'Content-Length': '228'} 2022-09-03T05:41:49.859 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000040e050918a1307b3-006312e91d-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.860 DEBUG:boto:path=/dsgvcy-26/ 2022-09-03T05:41:49.860 DEBUG:boto:auth_path=/dsgvcy-26/ 2022-09-03T05:41:49.861 DEBUG:boto:path=/dsgvcy-26/?versioning 2022-09-03T05:41:49.861 DEBUG:boto:auth_path=/dsgvcy-26/?versioning 2022-09-03T05:41:49.862 DEBUG:boto:Method: PUT 2022-09-03T05:41:49.862 DEBUG:boto:Path: /dsgvcy-26/?versioning 2022-09-03T05:41:49.863 DEBUG:boto:Data: Suspended Disabled 2022-09-03T05:41:49.863 DEBUG:boto:Headers: {} 2022-09-03T05:41:49.864 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:49.864 DEBUG:boto:Port: 8001 2022-09-03T05:41:49.865 DEBUG:boto:Params: {} 2022-09-03T05:41:49.865 DEBUG:boto:Token: None 2022-09-03T05:41:49.866 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:41:49 GMT /dsgvcy-26/?versioning 2022-09-03T05:41:49.866 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A3HYAkvRmdgY1mPcahq+wcj1U2E= 2022-09-03T05:41:49.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A3HYAkvRmdgY1mPcahq+wcj1U2E=', 'Content-Length': '230'} 2022-09-03T05:41:49.879 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000008f3ca9fd72883a3e-006312e91d-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:49.880 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:41:49.881 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:41:49.882 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:41:49.930 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:49.930 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:49.960 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=41ecd2a5-afc0-4eae-9349-7dfe39baa1cb 2022-09-03T05:41:49.974 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:41:49.974 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:41:49.974 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:41:49.975 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:18:18.906852Z" 2022-09-03T05:41:49.975 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:41:49.975 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:41:49.976 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:41:49.976 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:41:49.976 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:41:49.976 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:41:49.977 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662183709.876923_230.1", 2022-09-03T05:41:49.977 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:41:49.876923Z" 2022-09-03T05:41:49.977 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:41:49.977 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:41:49.978 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:41:49.978 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:20:30.928498Z" 2022-09-03T05:41:49.978 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:41:49.978 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:41:49.980 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183709.876923_230.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:41:49.980 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:41:49.981 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:41:49.981 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:41:49.982 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:41:50.043 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:50.043 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:50.082 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:50.081+0000 7f5b94a28600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:41:50.082 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:50.082+0000 7f5b94a28600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:41:50.097 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:41:50.097 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:41:50.097 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:41:50.098 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:41:50.098 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:41:50.098 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:41:50.098 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:41:50.099 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:50.099 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:41:50.099 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:41:50.100 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:41:50.100 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:41:50.100 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:41:50.100 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:41:50.101 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:41:50.101 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:41:50.101 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:41:50.102 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:41:50.103 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:41:50.103 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:41:50.104 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:50.104 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:41:50.105 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:41:50.105 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:41:50.106 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:41:50.106 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:41:50.106 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:41:50.107 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:41:50.107 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:41:50.107 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:41:50.107 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:41:50.108 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:41:50.108 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:50.108 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:41:50.108 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:41:50.109 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:41:50.109 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:41:50.109 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183709.657291_207.1", 2022-09-03T05:41:50.110 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:41:50.110 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:41:50.111 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:41:50.111 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.657291Z", 2022-09-03T05:41:50.112 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:41:50.112 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:41:50.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:50.113 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:41:50.113 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:41:50.113 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:41:50.114 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:41:50.114 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:41:50.114 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:41:50.115 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:41:50.115 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:41:50.115 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:41:50.115 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:41:50.116 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:41:50.116 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:41:50.116 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:41:50.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:50.117 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:41:50.117 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:41:50.117 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:41:50.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:50.118 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:41:50Z" 2022-09-03T05:41:50.118 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:41:50.120 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183709.657291_207.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:41:49.657291Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:41:50Z" } 2022-09-03T05:41:50.120 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183709.657291_207.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:41:49.657291Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:41:50.121 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183709.876923_230.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:41:50.121 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183709.657291_207.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:41:50.122 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662183709.876923_230.1 target=1_1662183709.657291_207.1 2022-09-03T05:41:55.123 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:41:55.124 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:41:55.126 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:41:55.184 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:55.184 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:55.223 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:55.222+0000 7f825222a600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:41:55.224 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:55.223+0000 7f825222a600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:41:55.238 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:41:55.239 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:41:55.239 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:41:55.240 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:41:55.240 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:41:55.240 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:41:55.240 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:41:55.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:55.241 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:41:55.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:41:55.242 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:41:55.242 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:41:55.243 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:41:55.243 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:41:55.243 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:41:55.244 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:41:55.244 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:41:55.244 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:41:55.245 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:41:55.245 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:41:55.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:55.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:41:55.247 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:41:55.247 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:41:55.248 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:41:55.248 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:41:55.249 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:41:55.249 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:41:55.250 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:41:55.250 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:41:55.251 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:41:55.251 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:41:55.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:55.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:41:55.253 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:41:55.253 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:41:55.254 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:41:55.254 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183709.876923_230.1", 2022-09-03T05:41:55.255 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:41:55.255 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:41:55.256 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:41:55.256 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.876923Z", 2022-09-03T05:41:55.257 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:41:55.257 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:41:55.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:55.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:41:55.259 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:41:55.259 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:41:55.260 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:41:55.260 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:41:55.261 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:41:55.261 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:41:55.262 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:41:55.262 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:41:55.263 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:41:55.263 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:41:55.264 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:41:55.264 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:41:55.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:55.265 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:41:55.266 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:41:55.266 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:41:55.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:55.267 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:41:55Z" 2022-09-03T05:41:55.268 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:41:55.270 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183709.876923_230.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:41:49.876923Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:41:55Z" } 2022-09-03T05:41:55.271 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183709.876923_230.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:41:49.876923Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:41:55.272 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183709.876923_230.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:41:55.273 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183709.876923_230.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:41:55.274 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:41:55.275 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:41:55.276 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:41:55.277 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:41:55.278 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:41:55.336 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:55.337 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:55.387 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:55.387+0000 7f13949a9600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:41:55.388 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:41:55.388+0000 7f13949a9600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:41:55.411 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:41:55.413 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:41:55.414 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:41:55.414 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:41:55.415 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:41:55.415 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:41:55.416 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:41:55.416 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:41:55.417 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:41:55.417 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:41:55.418 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:41:55.419 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:41:55.419 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:41:55.420 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:41:55.420 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:41:55.421 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:41:55.421 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:41:55.422 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:41:55.422 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:41:55.423 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:41:55.423 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:41:55.424 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:41:55.424 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:41:55.425 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:41:55.425 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:41:55.425 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:41:55.426 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:41:55.426 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:41:55.426 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:41:55.426 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:41:55.427 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:41:55.427 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:41:55.427 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:41:55.428 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:41:55.428 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:41:55.428 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:41:55.429 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:41:55.429 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662183709.876923_230.1", 2022-09-03T05:41:55.430 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:41:55.430 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:41:55.430 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:41:55.431 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.876923Z", 2022-09-03T05:41:55.431 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:41:55.431 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:41:55.432 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:41:55.432 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:41:55.433 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:41:55.433 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:41:55.434 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:41:55.434 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:41:55.435 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:41:55.435 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:41:55.435 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:41:55.435 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:41:55.436 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:41:55.436 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:41:55.436 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:41:55.436 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:41:55.436 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:41:55.437 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:41:55.437 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:41:55.437 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:41:55.437 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:41:55.438 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:41:55Z" 2022-09-03T05:41:55.438 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:41:55.439 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183709.876923_230.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:41:49.876923Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:41:55Z" } 2022-09-03T05:41:55.440 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183709.876923_230.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:41:49.876923Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:41:55.441 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183709.876923_230.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:41:55.441 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183709.876923_230.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:41:55.442 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:41:55.442 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:41:55.443 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:41:55.444 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:41:55.444 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:41:55.496 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:55.497 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:55.530 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:55.527+0000 7f8652505600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:41:55.530 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:55.528+0000 7f8652505600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:41:55.540 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:41:55.541 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:41:55.541 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:41:55.542 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:41:55.542 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:41:55.542 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:41:55.542 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:41:55.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:55.543 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:41:55.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:41:55.544 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:41:55.544 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:41:55.544 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:41:55.545 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:41:55.545 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:41:55.545 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:41:55.545 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:41:55.546 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:41:55.546 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:41:55.546 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:41:55.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:55.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:41:55.547 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:41:55.547 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:41:55.548 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:41:55.548 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:41:55.550 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:41:55.550 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:41:55.551 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:41:55.551 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:41:55.551 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:41:55.551 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:41:55.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:55.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:41:55.552 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:41:55.552 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:41:55.553 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:41:55.553 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183709.876923_230.1", 2022-09-03T05:41:55.553 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:41:55.553 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:41:55.554 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:41:55.554 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.876923Z", 2022-09-03T05:41:55.554 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:41:55.554 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:41:55.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:55.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:41:55.555 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:41:55.556 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:41:55.557 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:41:55.557 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:41:55.558 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:41:55.559 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:41:55.559 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:41:55.560 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:41:55.560 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:41:55.560 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:41:55.561 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:41:55.561 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:41:55.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:55.562 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:41:55.562 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:41:55.562 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:41:55.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:41:55.563 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:41:55Z" 2022-09-03T05:41:55.563 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:41:55.565 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183709.876923_230.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:41:49.876923Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:41:55Z" } 2022-09-03T05:41:55.565 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183709.876923_230.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:41:49.876923Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:41:55.566 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183709.876923_230.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:41:55.566 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183709.876923_230.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:41:55.567 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:41:55.567 DEBUG:boto:path=/dsgvcy-24/ 2022-09-03T05:41:55.568 DEBUG:boto:auth_path=/dsgvcy-24/ 2022-09-03T05:41:55.568 DEBUG:boto:Method: HEAD 2022-09-03T05:41:55.569 DEBUG:boto:Path: /dsgvcy-24/ 2022-09-03T05:41:55.569 DEBUG:boto:Data: 2022-09-03T05:41:55.570 DEBUG:boto:Headers: {} 2022-09-03T05:41:55.570 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:55.571 DEBUG:boto:Port: 8000 2022-09-03T05:41:55.572 DEBUG:boto:Params: {} 2022-09-03T05:41:55.572 DEBUG:boto:Token: None 2022-09-03T05:41:55.573 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:55 GMT /dsgvcy-24/ 2022-09-03T05:41:55.574 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:P/HoHs8e+e+1IqE+54RlP4Nwg1k= 2022-09-03T05:41:55.574 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:P/HoHs8e+e+1IqE+54RlP4Nwg1k=', 'Content-Length': '0'} 2022-09-03T05:41:55.577 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002bdc5afcf042746b-006312e923-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:55.578 DEBUG:boto:path=/dsgvcy-24/obj 2022-09-03T05:41:55.578 DEBUG:boto:auth_path=/dsgvcy-24/obj 2022-09-03T05:41:55.579 DEBUG:boto:Method: PUT 2022-09-03T05:41:55.579 DEBUG:boto:Path: /dsgvcy-24/obj 2022-09-03T05:41:55.580 DEBUG:boto:Data: 2022-09-03T05:41:55.581 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue'} 2022-09-03T05:41:55.582 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:55.583 DEBUG:boto:Port: 8000 2022-09-03T05:41:55.583 DEBUG:boto:Params: {} 2022-09-03T05:41:55.584 DEBUG:boto:Token: None 2022-09-03T05:41:55.585 DEBUG:boto:StringToSign: PUT 1B2M2Y8AsgTpgAmY7PhCfg== application/octet-stream Sat, 03 Sep 2022 05:41:55 GMT /dsgvcy-24/obj 2022-09-03T05:41:55.586 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wdHTRrgtELiak1XtFLgG9wwq/00= 2022-09-03T05:41:55.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wdHTRrgtELiak1XtFLgG9wwq/00='} 2022-09-03T05:41:55.598 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000006eb39f306a638f50-006312e923-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:55.599 DEBUG:boto:path=/dsgvcy-24/ 2022-09-03T05:41:55.600 DEBUG:boto:auth_path=/dsgvcy-24/ 2022-09-03T05:41:55.601 DEBUG:boto:Method: HEAD 2022-09-03T05:41:55.602 DEBUG:boto:Path: /dsgvcy-24/ 2022-09-03T05:41:55.603 DEBUG:boto:Data: 2022-09-03T05:41:55.605 DEBUG:boto:Headers: {} 2022-09-03T05:41:55.606 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:55.607 DEBUG:boto:Port: 8000 2022-09-03T05:41:55.608 DEBUG:boto:Params: {} 2022-09-03T05:41:55.609 DEBUG:boto:Token: None 2022-09-03T05:41:55.610 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:55 GMT /dsgvcy-24/ 2022-09-03T05:41:55.611 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:P/HoHs8e+e+1IqE+54RlP4Nwg1k= 2022-09-03T05:41:55.611 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:P/HoHs8e+e+1IqE+54RlP4Nwg1k=', 'Content-Length': '0'} 2022-09-03T05:41:55.614 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004d349c9b3d59eb97-006312e923-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:55.615 DEBUG:boto:path=/dsgvcy-24/obj 2022-09-03T05:41:55.617 DEBUG:boto:auth_path=/dsgvcy-24/obj 2022-09-03T05:41:55.617 DEBUG:boto:Method: DELETE 2022-09-03T05:41:55.618 DEBUG:boto:Path: /dsgvcy-24/obj 2022-09-03T05:41:55.619 DEBUG:boto:Data: 2022-09-03T05:41:55.620 DEBUG:boto:Headers: {} 2022-09-03T05:41:55.621 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:41:55.622 DEBUG:boto:Port: 8000 2022-09-03T05:41:55.623 DEBUG:boto:Params: {} 2022-09-03T05:41:55.623 DEBUG:boto:Token: None 2022-09-03T05:41:55.624 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:41:55 GMT /dsgvcy-24/obj 2022-09-03T05:41:55.625 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SMPEFQbfmiowA2/Tr01x+9NLy/c= 2022-09-03T05:41:55.625 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SMPEFQbfmiowA2/Tr01x+9NLy/c=', 'Content-Length': '0'} 2022-09-03T05:41:55.639 DEBUG:boto:Response headers: [('x-amz-version-id', 'null'), ('x-amz-delete-marker', 'true'), ('x-amz-request-id', 'tx00000e6ecf18680196362-006312e923-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:55.640 DEBUG:boto:path=/dsgvcy-25/ 2022-09-03T05:41:55.641 DEBUG:boto:auth_path=/dsgvcy-25/ 2022-09-03T05:41:55.642 DEBUG:boto:Method: HEAD 2022-09-03T05:41:55.643 DEBUG:boto:Path: /dsgvcy-25/ 2022-09-03T05:41:55.643 DEBUG:boto:Data: 2022-09-03T05:41:55.644 DEBUG:boto:Headers: {} 2022-09-03T05:41:55.644 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:55.645 DEBUG:boto:Port: 8000 2022-09-03T05:41:55.645 DEBUG:boto:Params: {} 2022-09-03T05:41:55.646 DEBUG:boto:Token: None 2022-09-03T05:41:55.647 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:55 GMT /dsgvcy-25/ 2022-09-03T05:41:55.647 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nN3b53JxHO57vJ3gCIZOtYirkl4= 2022-09-03T05:41:55.648 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nN3b53JxHO57vJ3gCIZOtYirkl4=', 'Content-Length': '0'} 2022-09-03T05:41:55.651 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009090083afddaf1e1-006312e923-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:55.652 DEBUG:boto:path=/dsgvcy-25/obj 2022-09-03T05:41:55.653 DEBUG:boto:auth_path=/dsgvcy-25/obj 2022-09-03T05:41:55.654 DEBUG:boto:Method: PUT 2022-09-03T05:41:55.655 DEBUG:boto:Path: /dsgvcy-25/obj 2022-09-03T05:41:55.657 DEBUG:boto:Data: 2022-09-03T05:41:55.658 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue'} 2022-09-03T05:41:55.659 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:55.660 DEBUG:boto:Port: 8000 2022-09-03T05:41:55.660 DEBUG:boto:Params: {} 2022-09-03T05:41:55.661 DEBUG:boto:Token: None 2022-09-03T05:41:55.663 DEBUG:boto:StringToSign: PUT 1B2M2Y8AsgTpgAmY7PhCfg== application/octet-stream Sat, 03 Sep 2022 05:41:55 GMT /dsgvcy-25/obj 2022-09-03T05:41:55.664 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PlqduLBuvHcxCyhOREv59skYPnU= 2022-09-03T05:41:55.665 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PlqduLBuvHcxCyhOREv59skYPnU='} 2022-09-03T05:41:55.671 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a87599665f80851d-006312e923-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:55.672 DEBUG:boto:path=/dsgvcy-25/ 2022-09-03T05:41:55.673 DEBUG:boto:auth_path=/dsgvcy-25/ 2022-09-03T05:41:55.674 DEBUG:boto:Method: HEAD 2022-09-03T05:41:55.675 DEBUG:boto:Path: /dsgvcy-25/ 2022-09-03T05:41:55.675 DEBUG:boto:Data: 2022-09-03T05:41:55.677 DEBUG:boto:Headers: {} 2022-09-03T05:41:55.677 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:55.679 DEBUG:boto:Port: 8000 2022-09-03T05:41:55.680 DEBUG:boto:Params: {} 2022-09-03T05:41:55.681 DEBUG:boto:Token: None 2022-09-03T05:41:55.682 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:55 GMT /dsgvcy-25/ 2022-09-03T05:41:55.682 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nN3b53JxHO57vJ3gCIZOtYirkl4= 2022-09-03T05:41:55.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nN3b53JxHO57vJ3gCIZOtYirkl4=', 'Content-Length': '0'} 2022-09-03T05:41:55.686 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b61a1604fcd1b46c-006312e923-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:55.687 DEBUG:boto:path=/dsgvcy-25/obj 2022-09-03T05:41:55.688 DEBUG:boto:auth_path=/dsgvcy-25/obj 2022-09-03T05:41:55.689 DEBUG:boto:Method: DELETE 2022-09-03T05:41:55.690 DEBUG:boto:Path: /dsgvcy-25/obj 2022-09-03T05:41:55.691 DEBUG:boto:Data: 2022-09-03T05:41:55.692 DEBUG:boto:Headers: {} 2022-09-03T05:41:55.693 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:41:55.694 DEBUG:boto:Port: 8000 2022-09-03T05:41:55.695 DEBUG:boto:Params: {} 2022-09-03T05:41:55.699 DEBUG:boto:Token: None 2022-09-03T05:41:55.700 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:41:55 GMT /dsgvcy-25/obj 2022-09-03T05:41:55.701 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:laWRbipFyUzvNn0BfTXjeOtf3kM= 2022-09-03T05:41:55.702 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:laWRbipFyUzvNn0BfTXjeOtf3kM=', 'Content-Length': '0'} 2022-09-03T05:41:55.710 DEBUG:boto:Response headers: [('x-amz-version-id', 'null'), ('x-amz-delete-marker', 'true'), ('x-amz-request-id', 'tx000002b77b3ae518caa2d-006312e923-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:55.711 DEBUG:boto:path=/dsgvcy-26/ 2022-09-03T05:41:55.711 DEBUG:boto:auth_path=/dsgvcy-26/ 2022-09-03T05:41:55.712 DEBUG:boto:Method: HEAD 2022-09-03T05:41:55.712 DEBUG:boto:Path: /dsgvcy-26/ 2022-09-03T05:41:55.713 DEBUG:boto:Data: 2022-09-03T05:41:55.714 DEBUG:boto:Headers: {} 2022-09-03T05:41:55.715 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:55.716 DEBUG:boto:Port: 8001 2022-09-03T05:41:55.717 DEBUG:boto:Params: {} 2022-09-03T05:41:55.718 DEBUG:boto:Token: None 2022-09-03T05:41:55.719 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:55 GMT /dsgvcy-26/ 2022-09-03T05:41:55.720 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:P+PgFsiYRpS0jFdS5TKvm9/0zWw= 2022-09-03T05:41:55.722 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:P+PgFsiYRpS0jFdS5TKvm9/0zWw=', 'Content-Length': '0'} 2022-09-03T05:41:55.725 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000064e68b8124090607-006312e923-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:55.727 DEBUG:boto:path=/dsgvcy-26/obj 2022-09-03T05:41:55.728 DEBUG:boto:auth_path=/dsgvcy-26/obj 2022-09-03T05:41:55.729 DEBUG:boto:Method: PUT 2022-09-03T05:41:55.729 DEBUG:boto:Path: /dsgvcy-26/obj 2022-09-03T05:41:55.731 DEBUG:boto:Data: 2022-09-03T05:41:55.732 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue'} 2022-09-03T05:41:55.733 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:55.734 DEBUG:boto:Port: 8001 2022-09-03T05:41:55.735 DEBUG:boto:Params: {} 2022-09-03T05:41:55.736 DEBUG:boto:Token: None 2022-09-03T05:41:55.737 DEBUG:boto:StringToSign: PUT 1B2M2Y8AsgTpgAmY7PhCfg== application/octet-stream Sat, 03 Sep 2022 05:41:55 GMT /dsgvcy-26/obj 2022-09-03T05:41:55.737 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5Nl8A5+d+aQvj07l7rXCRi+uDEE= 2022-09-03T05:41:55.749 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '1B2M2Y8AsgTpgAmY7PhCfg==', 'Content-Length': '0', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5Nl8A5+d+aQvj07l7rXCRi+uDEE='} 2022-09-03T05:41:55.755 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"d41d8cd98f00b204e9800998ecf8427e"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008e29c8ac79c6a4ba-006312e923-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:55.757 DEBUG:boto:path=/dsgvcy-26/ 2022-09-03T05:41:55.757 DEBUG:boto:auth_path=/dsgvcy-26/ 2022-09-03T05:41:55.759 DEBUG:boto:Method: HEAD 2022-09-03T05:41:55.760 DEBUG:boto:Path: /dsgvcy-26/ 2022-09-03T05:41:55.761 DEBUG:boto:Data: 2022-09-03T05:41:55.761 DEBUG:boto:Headers: {} 2022-09-03T05:41:55.762 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:55.762 DEBUG:boto:Port: 8001 2022-09-03T05:41:55.763 DEBUG:boto:Params: {} 2022-09-03T05:41:55.763 DEBUG:boto:Token: None 2022-09-03T05:41:55.764 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:41:55 GMT /dsgvcy-26/ 2022-09-03T05:41:55.764 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:P+PgFsiYRpS0jFdS5TKvm9/0zWw= 2022-09-03T05:41:55.765 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:P+PgFsiYRpS0jFdS5TKvm9/0zWw=', 'Content-Length': '0'} 2022-09-03T05:41:55.767 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008510551d5248c569-006312e923-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:55.768 DEBUG:boto:path=/dsgvcy-26/obj 2022-09-03T05:41:55.769 DEBUG:boto:auth_path=/dsgvcy-26/obj 2022-09-03T05:41:55.769 DEBUG:boto:Method: DELETE 2022-09-03T05:41:55.770 DEBUG:boto:Path: /dsgvcy-26/obj 2022-09-03T05:41:55.770 DEBUG:boto:Data: 2022-09-03T05:41:55.771 DEBUG:boto:Headers: {} 2022-09-03T05:41:55.771 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:41:55.772 DEBUG:boto:Port: 8001 2022-09-03T05:41:55.772 DEBUG:boto:Params: {} 2022-09-03T05:41:55.773 DEBUG:boto:Token: None 2022-09-03T05:41:55.774 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:41:55 GMT /dsgvcy-26/obj 2022-09-03T05:41:55.774 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mgXVwL5Ra8n/dsMTIf5iTZHBans= 2022-09-03T05:41:55.775 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mgXVwL5Ra8n/dsMTIf5iTZHBans=', 'Content-Length': '0'} 2022-09-03T05:41:55.787 DEBUG:boto:Response headers: [('x-amz-version-id', 'null'), ('x-amz-delete-marker', 'true'), ('x-amz-request-id', 'tx0000063980a74174e0fee-006312e923-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:41:55.787 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-24 2022-09-03T05:41:55.788 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-24', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:55.789 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-24', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:41:55.789 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-24 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:41:55.846 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:41:55.847 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:41:55.884 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:41:55.882+0000 7f6989c8f600 1 waiting to reach incremental sync.. 2022-09-03T05:42:00.885 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:42:00.884+0000 7f6989c8f600 1 waiting to reach incremental sync.. 2022-09-03T05:42:05.890 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:42:05.885+0000 7f6989c8f600 1 waiting to reach incremental sync.. 2022-09-03T05:42:10.887 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:42:10.887+0000 7f6989c8f600 1 bucket sync caught up with source: 2022-09-03T05:42:10.888 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000003.15.6, , ] 2022-09-03T05:42:10.888 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000003.15.6, , ] 2022-09-03T05:42:10.888 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:42:10.887+0000 7f6989c8f600 0 bucket checkpoint complete 2022-09-03T05:42:10.904 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-24 2022-09-03T05:42:10.906 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-24', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:42:10.907 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-24', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:42:10.908 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-24 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:42:10.973 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:42:10.973 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:42:11.030 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:42:11.028+0000 7f6fd3d57600 1 waiting to reach incremental sync.. 2022-09-03T05:42:16.031 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:42:16.029+0000 7f6fd3d57600 1 waiting to reach incremental sync.. 2022-09-03T05:42:21.031 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:42:21.030+0000 7f6fd3d57600 1 waiting to reach incremental sync.. 2022-09-03T05:42:26.032 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:42:26.032+0000 7f6fd3d57600 1 waiting to reach incremental sync.. 2022-09-03T05:42:31.034 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:42:31.033+0000 7f6fd3d57600 1 bucket sync caught up with source: 2022-09-03T05:42:31.034 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000003.15.6, , ] 2022-09-03T05:42:31.035 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000003.15.6, , ] 2022-09-03T05:42:31.035 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:42:31.033+0000 7f6fd3d57600 0 bucket checkpoint complete 2022-09-03T05:42:31.054 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-24 2022-09-03T05:42:31.055 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-24', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:42:31.056 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-24', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:42:31.058 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-24 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:42:31.123 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:42:31.124 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:42:31.162 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:42:31.160+0000 7f1158254600 1 bucket sync caught up with source: 2022-09-03T05:42:31.162 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000003.15.6, , ] 2022-09-03T05:42:31.163 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000003.15.6, , ] 2022-09-03T05:42:31.163 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:42:31.160+0000 7f1158254600 0 bucket checkpoint complete 2022-09-03T05:42:31.178 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-24 2022-09-03T05:42:31.179 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-24', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:42:31.180 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-24', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:42:31.181 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-24 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:42:31.253 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:42:31.254 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:42:31.304 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:42:31.304+0000 7f3ff275b600 1 bucket sync caught up with source: 2022-09-03T05:42:31.305 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000001.26.12, , ] 2022-09-03T05:42:31.305 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000001.26.12, , ] 2022-09-03T05:42:31.305 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:42:31.304+0000 7f3ff275b600 0 bucket checkpoint complete 2022-09-03T05:42:31.318 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-24 2022-09-03T05:42:31.319 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-24', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:42:31.319 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-24', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:42:31.320 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-24 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:42:31.377 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:42:31.378 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:42:31.418 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:42:31.417+0000 7f46fb01c600 1 bucket sync caught up with source: 2022-09-03T05:42:31.418 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000001.26.12, , ] 2022-09-03T05:42:31.418 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000001.26.12, , ] 2022-09-03T05:42:31.419 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:42:31.417+0000 7f46fb01c600 0 bucket checkpoint complete 2022-09-03T05:42:31.442 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-24 2022-09-03T05:42:31.443 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-24 2022-09-03T05:42:31.443 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-24', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:42:31.444 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-24', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:42:31.444 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-24 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:42:31.500 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:42:31.500 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:42:31.550 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:42:31.550+0000 7f66e2e02600 1 waiting to reach incremental sync.. 2022-09-03T05:42:36.551 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:42:36.551+0000 7f66e2e02600 1 waiting to reach incremental sync.. 2022-09-03T05:42:41.554 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:42:41.551+0000 7f66e2e02600 1 waiting to reach incremental sync.. 2022-09-03T05:42:46.554 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:42:46.553+0000 7f66e2e02600 1 bucket sync caught up with source: 2022-09-03T05:42:46.555 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000001.44.12, , ] 2022-09-03T05:42:46.555 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000001.44.12, , ] 2022-09-03T05:42:46.555 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:42:46.553+0000 7f66e2e02600 0 bucket checkpoint complete 2022-09-03T05:42:46.576 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-24 2022-09-03T05:42:46.577 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-24', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:42:46.578 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-24', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:42:46.578 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-24 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:42:46.645 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:42:46.646 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:42:46.685 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:42:46.684+0000 7f393ce7b600 1 waiting to reach incremental sync.. 2022-09-03T05:42:51.686 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:42:51.685+0000 7f393ce7b600 1 waiting to reach incremental sync.. 2022-09-03T05:42:56.690 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:42:56.686+0000 7f393ce7b600 1 waiting to reach incremental sync.. 2022-09-03T05:43:01.688 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:43:01.687+0000 7f393ce7b600 1 waiting to reach incremental sync.. 2022-09-03T05:43:06.689 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:43:06.687+0000 7f393ce7b600 1 waiting to reach incremental sync.. 2022-09-03T05:43:11.690 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:43:11.690+0000 7f393ce7b600 1 bucket sync caught up with source: 2022-09-03T05:43:11.691 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.44.12, , ] 2022-09-03T05:43:11.691 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000001.44.12, , ] 2022-09-03T05:43:11.692 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:43:11.690+0000 7f393ce7b600 0 bucket checkpoint complete 2022-09-03T05:43:11.708 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-24 2022-09-03T05:43:11.708 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-24 zones={test-zone2, test-zone1} 2022-09-03T05:43:11.709 DEBUG:boto:path=/dsgvcy-24/ 2022-09-03T05:43:11.710 DEBUG:boto:auth_path=/dsgvcy-24/ 2022-09-03T05:43:11.711 DEBUG:boto:Method: HEAD 2022-09-03T05:43:11.711 DEBUG:boto:Path: /dsgvcy-24/ 2022-09-03T05:43:11.712 DEBUG:boto:Data: 2022-09-03T05:43:11.712 DEBUG:boto:Headers: {} 2022-09-03T05:43:11.713 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:43:11.713 DEBUG:boto:Port: 8000 2022-09-03T05:43:11.715 DEBUG:boto:Params: {} 2022-09-03T05:43:11.716 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:43:11.717 DEBUG:boto:Token: None 2022-09-03T05:43:11.718 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:11 GMT /dsgvcy-24/ 2022-09-03T05:43:11.719 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xxQJbc5oQSYL3jugQy3bxTzjIXU= 2022-09-03T05:43:11.719 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xxQJbc5oQSYL3jugQy3bxTzjIXU=', 'Content-Length': '0'} 2022-09-03T05:43:11.725 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000dcbc13388c36eabc-006312e96f-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:43:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:11.726 DEBUG:boto:path=/dsgvcy-24/ 2022-09-03T05:43:11.727 DEBUG:boto:auth_path=/dsgvcy-24/ 2022-09-03T05:43:11.728 DEBUG:boto:Method: HEAD 2022-09-03T05:43:11.729 DEBUG:boto:Path: /dsgvcy-24/ 2022-09-03T05:43:11.730 DEBUG:boto:Data: 2022-09-03T05:43:11.731 DEBUG:boto:Headers: {} 2022-09-03T05:43:11.732 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:43:11.733 DEBUG:boto:Port: 8000 2022-09-03T05:43:11.734 DEBUG:boto:Params: {} 2022-09-03T05:43:11.735 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:43:11.736 DEBUG:boto:Token: None 2022-09-03T05:43:11.737 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:11 GMT /dsgvcy-24/ 2022-09-03T05:43:11.738 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xxQJbc5oQSYL3jugQy3bxTzjIXU= 2022-09-03T05:43:11.739 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xxQJbc5oQSYL3jugQy3bxTzjIXU=', 'Content-Length': '0'} 2022-09-03T05:43:11.743 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e15661c4d7038992-006312e96f-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:43:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:11.744 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:43:11.745 DEBUG:boto:path=/dsgvcy-24/ 2022-09-03T05:43:11.746 DEBUG:boto:auth_path=/dsgvcy-24/ 2022-09-03T05:43:11.746 DEBUG:boto:path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.747 DEBUG:boto:auth_path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.747 DEBUG:boto:Method: GET 2022-09-03T05:43:11.748 DEBUG:boto:Path: /dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.749 DEBUG:boto:Data: 2022-09-03T05:43:11.749 DEBUG:boto:Headers: {} 2022-09-03T05:43:11.750 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:43:11.751 DEBUG:boto:Port: 8000 2022-09-03T05:43:11.751 DEBUG:boto:Params: {} 2022-09-03T05:43:11.752 DEBUG:boto:Token: None 2022-09-03T05:43:11.753 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:11 GMT /dsgvcy-24/?versions 2022-09-03T05:43:11.753 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v/B7r0a3oI1jMmP2Chh4wv1WoQQ= 2022-09-03T05:43:11.754 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v/B7r0a3oI1jMmP2Chh4wv1WoQQ=', 'Content-Length': '0'} 2022-09-03T05:43:11.757 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d73a8c4f2cbcaabd-006312e96f-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:11.758 DEBUG:boto:b'dsgvcy-24999falseobjnulltrue2022-09-03T05:41:55.628Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:11.759 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:43:11.760 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:43:11.760 DEBUG:boto:path=/dsgvcy-24/ 2022-09-03T05:43:11.761 DEBUG:boto:auth_path=/dsgvcy-24/ 2022-09-03T05:43:11.761 DEBUG:boto:path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.762 DEBUG:boto:auth_path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.763 DEBUG:boto:Method: GET 2022-09-03T05:43:11.764 DEBUG:boto:Path: /dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.765 DEBUG:boto:Data: 2022-09-03T05:43:11.766 DEBUG:boto:Headers: {} 2022-09-03T05:43:11.767 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:43:11.768 DEBUG:boto:Port: 8000 2022-09-03T05:43:11.769 DEBUG:boto:Params: {} 2022-09-03T05:43:11.770 DEBUG:boto:Token: None 2022-09-03T05:43:11.770 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:11 GMT /dsgvcy-24/?versions 2022-09-03T05:43:11.771 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v/B7r0a3oI1jMmP2Chh4wv1WoQQ= 2022-09-03T05:43:11.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v/B7r0a3oI1jMmP2Chh4wv1WoQQ=', 'Content-Length': '0'} 2022-09-03T05:43:11.775 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000006e0bf76fef1861a-006312e96f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:11.776 DEBUG:boto:b'dsgvcy-24999falseobjnulltrue2022-09-03T05:41:55.628Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:11.777 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:43:11.777 DEBUG:boto:path=/dsgvcy-24/ 2022-09-03T05:43:11.778 DEBUG:boto:auth_path=/dsgvcy-24/ 2022-09-03T05:43:11.779 DEBUG:boto:path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.779 DEBUG:boto:auth_path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.780 DEBUG:boto:Method: GET 2022-09-03T05:43:11.780 DEBUG:boto:Path: /dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.781 DEBUG:boto:Data: 2022-09-03T05:43:11.781 DEBUG:boto:Headers: {} 2022-09-03T05:43:11.782 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:43:11.782 DEBUG:boto:Port: 8000 2022-09-03T05:43:11.783 DEBUG:boto:Params: {} 2022-09-03T05:43:11.783 DEBUG:boto:Token: None 2022-09-03T05:43:11.784 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:11 GMT /dsgvcy-24/?versions 2022-09-03T05:43:11.785 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v/B7r0a3oI1jMmP2Chh4wv1WoQQ= 2022-09-03T05:43:11.785 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v/B7r0a3oI1jMmP2Chh4wv1WoQQ=', 'Content-Length': '0'} 2022-09-03T05:43:11.788 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000057835ca38461d21f-006312e96f-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:11.789 DEBUG:boto:b'dsgvcy-24999falseobjnulltrue2022-09-03T05:41:55.628Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:11.790 DEBUG:boto:path=/dsgvcy-24/ 2022-09-03T05:43:11.790 DEBUG:boto:auth_path=/dsgvcy-24/ 2022-09-03T05:43:11.791 DEBUG:boto:path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.791 DEBUG:boto:auth_path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.792 DEBUG:boto:Method: GET 2022-09-03T05:43:11.792 DEBUG:boto:Path: /dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.793 DEBUG:boto:Data: 2022-09-03T05:43:11.793 DEBUG:boto:Headers: {} 2022-09-03T05:43:11.794 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:43:11.795 DEBUG:boto:Port: 8000 2022-09-03T05:43:11.795 DEBUG:boto:Params: {} 2022-09-03T05:43:11.796 DEBUG:boto:Token: None 2022-09-03T05:43:11.796 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:11 GMT /dsgvcy-24/?versions 2022-09-03T05:43:11.797 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v/B7r0a3oI1jMmP2Chh4wv1WoQQ= 2022-09-03T05:43:11.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v/B7r0a3oI1jMmP2Chh4wv1WoQQ=', 'Content-Length': '0'} 2022-09-03T05:43:11.800 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cd1547e82bf5e3e9-006312e96f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:11.801 DEBUG:boto:b'dsgvcy-24999falseobjnulltrue2022-09-03T05:41:55.628Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:11.802 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:43:11.802 DEBUG:boto:path=/dsgvcy-24/obj 2022-09-03T05:43:11.803 DEBUG:boto:auth_path=/dsgvcy-24/obj 2022-09-03T05:43:11.803 DEBUG:boto:Method: HEAD 2022-09-03T05:43:11.804 DEBUG:boto:Path: /dsgvcy-24/obj 2022-09-03T05:43:11.804 DEBUG:boto:Data: 2022-09-03T05:43:11.805 DEBUG:boto:Headers: {} 2022-09-03T05:43:11.805 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:43:11.806 DEBUG:boto:Port: 8000 2022-09-03T05:43:11.806 DEBUG:boto:Params: {} 2022-09-03T05:43:11.807 DEBUG:boto:Token: None 2022-09-03T05:43:11.808 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:11 GMT /dsgvcy-24/obj 2022-09-03T05:43:11.808 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jeT1lLGRHlVbOpYduLNTeRpyqu8= 2022-09-03T05:43:11.809 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jeT1lLGRHlVbOpYduLNTeRpyqu8=', 'Content-Length': '0'} 2022-09-03T05:43:11.811 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx000002ed30c88086beaa3-006312e96f-10ea-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:11.812 DEBUG:boto:path=/dsgvcy-24/obj 2022-09-03T05:43:11.812 DEBUG:boto:auth_path=/dsgvcy-24/obj 2022-09-03T05:43:11.813 DEBUG:boto:Method: HEAD 2022-09-03T05:43:11.813 DEBUG:boto:Path: /dsgvcy-24/obj 2022-09-03T05:43:11.814 DEBUG:boto:Data: 2022-09-03T05:43:11.814 DEBUG:boto:Headers: {} 2022-09-03T05:43:11.815 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:43:11.815 DEBUG:boto:Port: 8000 2022-09-03T05:43:11.816 DEBUG:boto:Params: {} 2022-09-03T05:43:11.816 DEBUG:boto:Token: None 2022-09-03T05:43:11.817 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:11 GMT /dsgvcy-24/obj 2022-09-03T05:43:11.818 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jeT1lLGRHlVbOpYduLNTeRpyqu8= 2022-09-03T05:43:11.818 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jeT1lLGRHlVbOpYduLNTeRpyqu8=', 'Content-Length': '0'} 2022-09-03T05:43:11.820 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx00000057711dd028f8823-006312e96f-10ff-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:11.821 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-24 zones={test-zone2, test-zone1} 2022-09-03T05:43:11.821 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-24 zones={test-zone3, test-zone1} 2022-09-03T05:43:11.822 DEBUG:boto:path=/dsgvcy-24/ 2022-09-03T05:43:11.823 DEBUG:boto:auth_path=/dsgvcy-24/ 2022-09-03T05:43:11.823 DEBUG:boto:Method: HEAD 2022-09-03T05:43:11.824 DEBUG:boto:Path: /dsgvcy-24/ 2022-09-03T05:43:11.824 DEBUG:boto:Data: 2022-09-03T05:43:11.825 DEBUG:boto:Headers: {} 2022-09-03T05:43:11.825 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:43:11.826 DEBUG:boto:Port: 8001 2022-09-03T05:43:11.826 DEBUG:boto:Params: {} 2022-09-03T05:43:11.827 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T05:43:11.827 DEBUG:boto:Token: None 2022-09-03T05:43:11.828 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:11 GMT /dsgvcy-24/ 2022-09-03T05:43:11.829 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xxQJbc5oQSYL3jugQy3bxTzjIXU= 2022-09-03T05:43:11.829 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xxQJbc5oQSYL3jugQy3bxTzjIXU=', 'Content-Length': '0'} 2022-09-03T05:43:11.833 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000026692b5beb97474a-006312e96f-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:43:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:11.834 DEBUG:boto:path=/dsgvcy-24/ 2022-09-03T05:43:11.834 DEBUG:boto:auth_path=/dsgvcy-24/ 2022-09-03T05:43:11.835 DEBUG:boto:Method: HEAD 2022-09-03T05:43:11.835 DEBUG:boto:Path: /dsgvcy-24/ 2022-09-03T05:43:11.836 DEBUG:boto:Data: 2022-09-03T05:43:11.836 DEBUG:boto:Headers: {} 2022-09-03T05:43:11.837 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:43:11.837 DEBUG:boto:Port: 8000 2022-09-03T05:43:11.838 DEBUG:boto:Params: {} 2022-09-03T05:43:11.838 DEBUG:boto:Token: None 2022-09-03T05:43:11.839 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:11 GMT /dsgvcy-24/ 2022-09-03T05:43:11.839 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xxQJbc5oQSYL3jugQy3bxTzjIXU= 2022-09-03T05:43:11.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xxQJbc5oQSYL3jugQy3bxTzjIXU=', 'Content-Length': '0'} 2022-09-03T05:43:11.842 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005dd3c61dfd9a0582-006312e96f-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:43:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:11.843 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:43:11.844 DEBUG:boto:path=/dsgvcy-24/ 2022-09-03T05:43:11.844 DEBUG:boto:auth_path=/dsgvcy-24/ 2022-09-03T05:43:11.845 DEBUG:boto:path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.845 DEBUG:boto:auth_path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.846 DEBUG:boto:Method: GET 2022-09-03T05:43:11.846 DEBUG:boto:Path: /dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.847 DEBUG:boto:Data: 2022-09-03T05:43:11.847 DEBUG:boto:Headers: {} 2022-09-03T05:43:11.848 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:43:11.848 DEBUG:boto:Port: 8001 2022-09-03T05:43:11.849 DEBUG:boto:Params: {} 2022-09-03T05:43:11.849 DEBUG:boto:Token: None 2022-09-03T05:43:11.850 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:11 GMT /dsgvcy-24/?versions 2022-09-03T05:43:11.850 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v/B7r0a3oI1jMmP2Chh4wv1WoQQ= 2022-09-03T05:43:11.851 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v/B7r0a3oI1jMmP2Chh4wv1WoQQ=', 'Content-Length': '0'} 2022-09-03T05:43:11.854 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000042ff8bc02262f0d1-006312e96f-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:11.855 DEBUG:boto:b'dsgvcy-24999falseobjnulltrue2022-09-03T05:41:55.628Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:11.855 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:43:11.856 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:43:11.856 DEBUG:boto:path=/dsgvcy-24/ 2022-09-03T05:43:11.857 DEBUG:boto:auth_path=/dsgvcy-24/ 2022-09-03T05:43:11.857 DEBUG:boto:path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.858 DEBUG:boto:auth_path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.858 DEBUG:boto:Method: GET 2022-09-03T05:43:11.859 DEBUG:boto:Path: /dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.859 DEBUG:boto:Data: 2022-09-03T05:43:11.860 DEBUG:boto:Headers: {} 2022-09-03T05:43:11.860 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:43:11.861 DEBUG:boto:Port: 8000 2022-09-03T05:43:11.861 DEBUG:boto:Params: {} 2022-09-03T05:43:11.862 DEBUG:boto:Token: None 2022-09-03T05:43:11.863 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:11 GMT /dsgvcy-24/?versions 2022-09-03T05:43:11.863 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v/B7r0a3oI1jMmP2Chh4wv1WoQQ= 2022-09-03T05:43:11.864 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v/B7r0a3oI1jMmP2Chh4wv1WoQQ=', 'Content-Length': '0'} 2022-09-03T05:43:11.867 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b6b9105684ddc51e-006312e96f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:11.910 DEBUG:boto:b'dsgvcy-24999falseobjnulltrue2022-09-03T05:41:55.628Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:11.911 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:43:11.912 DEBUG:boto:path=/dsgvcy-24/ 2022-09-03T05:43:11.912 DEBUG:boto:auth_path=/dsgvcy-24/ 2022-09-03T05:43:11.913 DEBUG:boto:path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.914 DEBUG:boto:auth_path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.914 DEBUG:boto:Method: GET 2022-09-03T05:43:11.915 DEBUG:boto:Path: /dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.916 DEBUG:boto:Data: 2022-09-03T05:43:11.916 DEBUG:boto:Headers: {} 2022-09-03T05:43:11.917 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:43:11.918 DEBUG:boto:Port: 8001 2022-09-03T05:43:11.919 DEBUG:boto:Params: {} 2022-09-03T05:43:11.920 DEBUG:boto:Token: None 2022-09-03T05:43:11.921 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:11 GMT /dsgvcy-24/?versions 2022-09-03T05:43:11.922 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v/B7r0a3oI1jMmP2Chh4wv1WoQQ= 2022-09-03T05:43:11.923 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v/B7r0a3oI1jMmP2Chh4wv1WoQQ=', 'Content-Length': '0'} 2022-09-03T05:43:11.926 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f0165cc0d7f15b31-006312e96f-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:11.927 DEBUG:boto:b'dsgvcy-24999falseobjnulltrue2022-09-03T05:41:55.628Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:11.928 DEBUG:boto:path=/dsgvcy-24/ 2022-09-03T05:43:11.929 DEBUG:boto:auth_path=/dsgvcy-24/ 2022-09-03T05:43:11.930 DEBUG:boto:path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.930 DEBUG:boto:auth_path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.931 DEBUG:boto:Method: GET 2022-09-03T05:43:11.932 DEBUG:boto:Path: /dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.932 DEBUG:boto:Data: 2022-09-03T05:43:11.933 DEBUG:boto:Headers: {} 2022-09-03T05:43:11.933 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:43:11.934 DEBUG:boto:Port: 8000 2022-09-03T05:43:11.934 DEBUG:boto:Params: {} 2022-09-03T05:43:11.935 DEBUG:boto:Token: None 2022-09-03T05:43:11.936 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:11 GMT /dsgvcy-24/?versions 2022-09-03T05:43:11.936 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v/B7r0a3oI1jMmP2Chh4wv1WoQQ= 2022-09-03T05:43:11.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v/B7r0a3oI1jMmP2Chh4wv1WoQQ=', 'Content-Length': '0'} 2022-09-03T05:43:11.940 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ac018d3f83bd107a-006312e96f-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:11.941 DEBUG:boto:b'dsgvcy-24999falseobjnulltrue2022-09-03T05:41:55.628Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:11.941 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:43:11.942 DEBUG:boto:path=/dsgvcy-24/obj 2022-09-03T05:43:11.943 DEBUG:boto:auth_path=/dsgvcy-24/obj 2022-09-03T05:43:11.943 DEBUG:boto:Method: HEAD 2022-09-03T05:43:11.944 DEBUG:boto:Path: /dsgvcy-24/obj 2022-09-03T05:43:11.944 DEBUG:boto:Data: 2022-09-03T05:43:11.945 DEBUG:boto:Headers: {} 2022-09-03T05:43:11.945 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:43:11.946 DEBUG:boto:Port: 8001 2022-09-03T05:43:11.946 DEBUG:boto:Params: {} 2022-09-03T05:43:11.947 DEBUG:boto:Token: None 2022-09-03T05:43:11.947 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:11 GMT /dsgvcy-24/obj 2022-09-03T05:43:11.948 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jeT1lLGRHlVbOpYduLNTeRpyqu8= 2022-09-03T05:43:11.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jeT1lLGRHlVbOpYduLNTeRpyqu8=', 'Content-Length': '0'} 2022-09-03T05:43:11.950 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx00000a0afd102eac9643b-006312e96f-1103-test-zone3'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:11.951 DEBUG:boto:path=/dsgvcy-24/obj 2022-09-03T05:43:11.952 DEBUG:boto:auth_path=/dsgvcy-24/obj 2022-09-03T05:43:11.952 DEBUG:boto:Method: HEAD 2022-09-03T05:43:11.953 DEBUG:boto:Path: /dsgvcy-24/obj 2022-09-03T05:43:11.953 DEBUG:boto:Data: 2022-09-03T05:43:11.954 DEBUG:boto:Headers: {} 2022-09-03T05:43:11.954 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:43:11.955 DEBUG:boto:Port: 8000 2022-09-03T05:43:11.955 DEBUG:boto:Params: {} 2022-09-03T05:43:11.956 DEBUG:boto:Token: None 2022-09-03T05:43:11.956 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:11 GMT /dsgvcy-24/obj 2022-09-03T05:43:11.957 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jeT1lLGRHlVbOpYduLNTeRpyqu8= 2022-09-03T05:43:11.957 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jeT1lLGRHlVbOpYduLNTeRpyqu8=', 'Content-Length': '0'} 2022-09-03T05:43:11.959 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx00000e4e01ad05683ed06-006312e96f-10ff-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:11.960 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-24 zones={test-zone3, test-zone1} 2022-09-03T05:43:11.961 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-24 zones={test-zone3, test-zone2} 2022-09-03T05:43:11.961 DEBUG:boto:path=/dsgvcy-24/ 2022-09-03T05:43:11.962 DEBUG:boto:auth_path=/dsgvcy-24/ 2022-09-03T05:43:11.962 DEBUG:boto:Method: HEAD 2022-09-03T05:43:11.963 DEBUG:boto:Path: /dsgvcy-24/ 2022-09-03T05:43:11.963 DEBUG:boto:Data: 2022-09-03T05:43:11.964 DEBUG:boto:Headers: {} 2022-09-03T05:43:11.964 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:43:11.965 DEBUG:boto:Port: 8001 2022-09-03T05:43:11.965 DEBUG:boto:Params: {} 2022-09-03T05:43:11.966 DEBUG:boto:Token: None 2022-09-03T05:43:11.966 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:11 GMT /dsgvcy-24/ 2022-09-03T05:43:11.967 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xxQJbc5oQSYL3jugQy3bxTzjIXU= 2022-09-03T05:43:11.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xxQJbc5oQSYL3jugQy3bxTzjIXU=', 'Content-Length': '0'} 2022-09-03T05:43:11.970 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000034faf34d3b0bb68b-006312e96f-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:43:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:11.971 DEBUG:boto:path=/dsgvcy-24/ 2022-09-03T05:43:11.972 DEBUG:boto:auth_path=/dsgvcy-24/ 2022-09-03T05:43:11.972 DEBUG:boto:Method: HEAD 2022-09-03T05:43:11.973 DEBUG:boto:Path: /dsgvcy-24/ 2022-09-03T05:43:11.973 DEBUG:boto:Data: 2022-09-03T05:43:11.974 DEBUG:boto:Headers: {} 2022-09-03T05:43:11.974 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:43:11.975 DEBUG:boto:Port: 8000 2022-09-03T05:43:11.975 DEBUG:boto:Params: {} 2022-09-03T05:43:11.976 DEBUG:boto:Token: None 2022-09-03T05:43:11.977 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:11 GMT /dsgvcy-24/ 2022-09-03T05:43:11.977 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xxQJbc5oQSYL3jugQy3bxTzjIXU= 2022-09-03T05:43:11.978 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xxQJbc5oQSYL3jugQy3bxTzjIXU=', 'Content-Length': '0'} 2022-09-03T05:43:11.981 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009487290399110967-006312e96f-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:43:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:11.982 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:43:11.982 DEBUG:boto:path=/dsgvcy-24/ 2022-09-03T05:43:11.983 DEBUG:boto:auth_path=/dsgvcy-24/ 2022-09-03T05:43:11.983 DEBUG:boto:path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.984 DEBUG:boto:auth_path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.985 DEBUG:boto:Method: GET 2022-09-03T05:43:11.985 DEBUG:boto:Path: /dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:11.986 DEBUG:boto:Data: 2022-09-03T05:43:11.986 DEBUG:boto:Headers: {} 2022-09-03T05:43:11.987 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:43:11.987 DEBUG:boto:Port: 8001 2022-09-03T05:43:11.988 DEBUG:boto:Params: {} 2022-09-03T05:43:11.989 DEBUG:boto:Token: None 2022-09-03T05:43:11.989 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:11 GMT /dsgvcy-24/?versions 2022-09-03T05:43:11.990 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v/B7r0a3oI1jMmP2Chh4wv1WoQQ= 2022-09-03T05:43:11.991 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v/B7r0a3oI1jMmP2Chh4wv1WoQQ=', 'Content-Length': '0'} 2022-09-03T05:43:11.993 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b681d539b1a71929-006312e96f-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:12.038 DEBUG:boto:b'dsgvcy-24999falseobjnulltrue2022-09-03T05:41:55.628Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:12.039 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:43:12.039 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:43:12.040 DEBUG:boto:path=/dsgvcy-24/ 2022-09-03T05:43:12.041 DEBUG:boto:auth_path=/dsgvcy-24/ 2022-09-03T05:43:12.041 DEBUG:boto:path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:12.042 DEBUG:boto:auth_path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:12.042 DEBUG:boto:Method: GET 2022-09-03T05:43:12.043 DEBUG:boto:Path: /dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:12.043 DEBUG:boto:Data: 2022-09-03T05:43:12.044 DEBUG:boto:Headers: {} 2022-09-03T05:43:12.044 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:43:12.045 DEBUG:boto:Port: 8000 2022-09-03T05:43:12.045 DEBUG:boto:Params: {} 2022-09-03T05:43:12.046 DEBUG:boto:Token: None 2022-09-03T05:43:12.046 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:12 GMT /dsgvcy-24/?versions 2022-09-03T05:43:12.047 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qtkXiacj5iUSxtWKb88sPsusl4k= 2022-09-03T05:43:12.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qtkXiacj5iUSxtWKb88sPsusl4k=', 'Content-Length': '0'} 2022-09-03T05:43:12.050 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000014a9a729c99526fc-006312e970-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:12.051 DEBUG:boto:b'dsgvcy-24999falseobjnulltrue2022-09-03T05:41:55.628Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:12.052 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:43:12.053 DEBUG:boto:path=/dsgvcy-24/ 2022-09-03T05:43:12.053 DEBUG:boto:auth_path=/dsgvcy-24/ 2022-09-03T05:43:12.054 DEBUG:boto:path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:12.054 DEBUG:boto:auth_path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:12.055 DEBUG:boto:Method: GET 2022-09-03T05:43:12.055 DEBUG:boto:Path: /dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:12.056 DEBUG:boto:Data: 2022-09-03T05:43:12.056 DEBUG:boto:Headers: {} 2022-09-03T05:43:12.057 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:43:12.057 DEBUG:boto:Port: 8001 2022-09-03T05:43:12.058 DEBUG:boto:Params: {} 2022-09-03T05:43:12.058 DEBUG:boto:Token: None 2022-09-03T05:43:12.059 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:12 GMT /dsgvcy-24/?versions 2022-09-03T05:43:12.060 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qtkXiacj5iUSxtWKb88sPsusl4k= 2022-09-03T05:43:12.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qtkXiacj5iUSxtWKb88sPsusl4k=', 'Content-Length': '0'} 2022-09-03T05:43:12.063 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bcc11877df59497d-006312e970-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:12.065 DEBUG:boto:b'dsgvcy-24999falseobjnulltrue2022-09-03T05:41:55.628Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:12.066 DEBUG:boto:path=/dsgvcy-24/ 2022-09-03T05:43:12.067 DEBUG:boto:auth_path=/dsgvcy-24/ 2022-09-03T05:43:12.067 DEBUG:boto:path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:12.068 DEBUG:boto:auth_path=/dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:12.068 DEBUG:boto:Method: GET 2022-09-03T05:43:12.069 DEBUG:boto:Path: /dsgvcy-24/?versions&max-keys=999 2022-09-03T05:43:12.069 DEBUG:boto:Data: 2022-09-03T05:43:12.070 DEBUG:boto:Headers: {} 2022-09-03T05:43:12.070 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:43:12.071 DEBUG:boto:Port: 8000 2022-09-03T05:43:12.071 DEBUG:boto:Params: {} 2022-09-03T05:43:12.072 DEBUG:boto:Token: None 2022-09-03T05:43:12.073 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:12 GMT /dsgvcy-24/?versions 2022-09-03T05:43:12.073 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qtkXiacj5iUSxtWKb88sPsusl4k= 2022-09-03T05:43:12.074 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qtkXiacj5iUSxtWKb88sPsusl4k=', 'Content-Length': '0'} 2022-09-03T05:43:12.077 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b883577e8bbfd084-006312e970-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:12.078 DEBUG:boto:b'dsgvcy-24999falseobjnulltrue2022-09-03T05:41:55.628Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:12.079 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:43:12.079 DEBUG:boto:path=/dsgvcy-24/obj 2022-09-03T05:43:12.080 DEBUG:boto:auth_path=/dsgvcy-24/obj 2022-09-03T05:43:12.081 DEBUG:boto:Method: HEAD 2022-09-03T05:43:12.082 DEBUG:boto:Path: /dsgvcy-24/obj 2022-09-03T05:43:12.082 DEBUG:boto:Data: 2022-09-03T05:43:12.083 DEBUG:boto:Headers: {} 2022-09-03T05:43:12.084 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:43:12.084 DEBUG:boto:Port: 8001 2022-09-03T05:43:12.085 DEBUG:boto:Params: {} 2022-09-03T05:43:12.086 DEBUG:boto:Token: None 2022-09-03T05:43:12.086 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:12 GMT /dsgvcy-24/obj 2022-09-03T05:43:12.087 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dO4KHjdd1UfTQW2DmVoez7sIkzA= 2022-09-03T05:43:12.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dO4KHjdd1UfTQW2DmVoez7sIkzA=', 'Content-Length': '0'} 2022-09-03T05:43:12.091 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx0000001107071b84ad80c-006312e970-1103-test-zone3'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:12.092 DEBUG:boto:path=/dsgvcy-24/obj 2022-09-03T05:43:12.093 DEBUG:boto:auth_path=/dsgvcy-24/obj 2022-09-03T05:43:12.095 DEBUG:boto:Method: HEAD 2022-09-03T05:43:12.096 DEBUG:boto:Path: /dsgvcy-24/obj 2022-09-03T05:43:12.097 DEBUG:boto:Data: 2022-09-03T05:43:12.097 DEBUG:boto:Headers: {} 2022-09-03T05:43:12.098 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:43:12.099 DEBUG:boto:Port: 8000 2022-09-03T05:43:12.100 DEBUG:boto:Params: {} 2022-09-03T05:43:12.101 DEBUG:boto:Token: None 2022-09-03T05:43:12.102 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:12 GMT /dsgvcy-24/obj 2022-09-03T05:43:12.103 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dO4KHjdd1UfTQW2DmVoez7sIkzA= 2022-09-03T05:43:12.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dO4KHjdd1UfTQW2DmVoez7sIkzA=', 'Content-Length': '0'} 2022-09-03T05:43:12.107 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx0000050f3f0e2bac8e29a-006312e970-10ea-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:12.108 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-24 zones={test-zone3, test-zone2} 2022-09-03T05:43:12.109 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-25 2022-09-03T05:43:12.110 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-25', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:12.111 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-25', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:12.112 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-25 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:43:12.177 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:43:12.178 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:43:12.222 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:43:12.221+0000 7f734ffaf600 1 bucket sync caught up with source: 2022-09-03T05:43:12.222 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.15.12, , ] 2022-09-03T05:43:12.223 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000001.15.12, , ] 2022-09-03T05:43:12.223 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:43:12.221+0000 7f734ffaf600 0 bucket checkpoint complete 2022-09-03T05:43:12.236 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-25 2022-09-03T05:43:12.237 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-25', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:12.237 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-25', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:12.238 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-25 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:43:12.302 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:43:12.303 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:43:12.359 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:43:12.358+0000 7fe439e20600 1 bucket sync caught up with source: 2022-09-03T05:43:12.360 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000001.15.12, , ] 2022-09-03T05:43:12.361 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000001.15.12, , ] 2022-09-03T05:43:12.361 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:43:12.358+0000 7fe439e20600 0 bucket checkpoint complete 2022-09-03T05:43:12.371 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-25 2022-09-03T05:43:12.372 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-25', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:12.373 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-25', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:12.373 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-25 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:43:12.430 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:43:12.431 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:43:12.472 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:43:12.471+0000 7f96fb31d600 1 bucket sync caught up with source: 2022-09-03T05:43:12.472 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.15.12, , ] 2022-09-03T05:43:12.472 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000001.15.12, , ] 2022-09-03T05:43:12.473 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:43:12.471+0000 7f96fb31d600 0 bucket checkpoint complete 2022-09-03T05:43:12.484 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-25 2022-09-03T05:43:12.485 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-25', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:12.486 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-25', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:12.487 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-25 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:43:12.544 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:43:12.545 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:43:12.580 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:43:12.578+0000 7fbb77224600 1 bucket sync caught up with source: 2022-09-03T05:43:12.580 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000003.148.6, , ] 2022-09-03T05:43:12.581 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000003.148.6, , ] 2022-09-03T05:43:12.581 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:43:12.578+0000 7fbb77224600 0 bucket checkpoint complete 2022-09-03T05:43:12.592 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-25 2022-09-03T05:43:12.593 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-25', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:12.594 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-25', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:12.594 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-25 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:43:12.651 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:43:12.652 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:43:12.699 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:43:12.697+0000 7f0c48c9d600 1 bucket sync caught up with source: 2022-09-03T05:43:12.700 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000003.148.6, , ] 2022-09-03T05:43:12.700 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000003.148.6, , ] 2022-09-03T05:43:12.701 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:43:12.697+0000 7f0c48c9d600 0 bucket checkpoint complete 2022-09-03T05:43:12.722 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-25 2022-09-03T05:43:12.726 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-25 2022-09-03T05:43:12.727 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-25', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:12.728 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-25', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:12.729 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-25 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:43:12.786 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:43:12.787 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:43:12.835 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:43:12.831+0000 7f9c619a5600 1 waiting to reach incremental sync.. 2022-09-03T05:43:17.833 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:43:17.833+0000 7f9c619a5600 1 waiting to reach incremental sync.. 2022-09-03T05:43:22.834 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:43:22.833+0000 7f9c619a5600 1 waiting to reach incremental sync.. 2022-09-03T05:43:27.836 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:43:27.835+0000 7f9c619a5600 1 bucket sync caught up with source: 2022-09-03T05:43:27.837 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000001.22.12, , ] 2022-09-03T05:43:27.837 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000001.22.12, , ] 2022-09-03T05:43:27.838 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:43:27.835+0000 7f9c619a5600 0 bucket checkpoint complete 2022-09-03T05:43:27.858 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-25 2022-09-03T05:43:27.861 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-25', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:27.861 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-25', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:27.862 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-25 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:43:27.925 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:43:27.926 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:43:27.965 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:43:27.965+0000 7f63c4a56600 1 bucket sync caught up with source: 2022-09-03T05:43:27.966 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.22.12, , ] 2022-09-03T05:43:27.966 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000001.22.12, , ] 2022-09-03T05:43:27.966 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:43:27.965+0000 7f63c4a56600 0 bucket checkpoint complete 2022-09-03T05:43:27.982 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-25 2022-09-03T05:43:27.982 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-25 zones={test-zone2, test-zone1} 2022-09-03T05:43:27.983 DEBUG:boto:path=/dsgvcy-25/ 2022-09-03T05:43:27.984 DEBUG:boto:auth_path=/dsgvcy-25/ 2022-09-03T05:43:27.985 DEBUG:boto:Method: HEAD 2022-09-03T05:43:27.986 DEBUG:boto:Path: /dsgvcy-25/ 2022-09-03T05:43:27.987 DEBUG:boto:Data: 2022-09-03T05:43:27.988 DEBUG:boto:Headers: {} 2022-09-03T05:43:27.989 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:43:27.990 DEBUG:boto:Port: 8000 2022-09-03T05:43:27.991 DEBUG:boto:Params: {} 2022-09-03T05:43:27.992 DEBUG:boto:Token: None 2022-09-03T05:43:27.994 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:27 GMT /dsgvcy-25/ 2022-09-03T05:43:27.995 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uk3DEHI7zdG0Si2RUjUdvEjyuys= 2022-09-03T05:43:27.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uk3DEHI7zdG0Si2RUjUdvEjyuys=', 'Content-Length': '0'} 2022-09-03T05:43:27.999 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005b67e47815f318b0-006312e97f-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:43:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.000 DEBUG:boto:path=/dsgvcy-25/ 2022-09-03T05:43:28.001 DEBUG:boto:auth_path=/dsgvcy-25/ 2022-09-03T05:43:28.002 DEBUG:boto:Method: HEAD 2022-09-03T05:43:28.003 DEBUG:boto:Path: /dsgvcy-25/ 2022-09-03T05:43:28.004 DEBUG:boto:Data: 2022-09-03T05:43:28.005 DEBUG:boto:Headers: {} 2022-09-03T05:43:28.005 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:43:28.006 DEBUG:boto:Port: 8000 2022-09-03T05:43:28.007 DEBUG:boto:Params: {} 2022-09-03T05:43:28.007 DEBUG:boto:Token: None 2022-09-03T05:43:28.008 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:28 GMT /dsgvcy-25/ 2022-09-03T05:43:28.009 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IJqfSPOGrHC/69EtJQtD4I4JOY4= 2022-09-03T05:43:28.009 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IJqfSPOGrHC/69EtJQtD4I4JOY4=', 'Content-Length': '0'} 2022-09-03T05:43:28.012 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009d0e823d52d98aeb-006312e980-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:43:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.013 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:43:28.013 DEBUG:boto:path=/dsgvcy-25/ 2022-09-03T05:43:28.014 DEBUG:boto:auth_path=/dsgvcy-25/ 2022-09-03T05:43:28.015 DEBUG:boto:path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.016 DEBUG:boto:auth_path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.017 DEBUG:boto:Method: GET 2022-09-03T05:43:28.018 DEBUG:boto:Path: /dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.020 DEBUG:boto:Data: 2022-09-03T05:43:28.020 DEBUG:boto:Headers: {} 2022-09-03T05:43:28.021 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:43:28.022 DEBUG:boto:Port: 8000 2022-09-03T05:43:28.023 DEBUG:boto:Params: {} 2022-09-03T05:43:28.024 DEBUG:boto:Token: None 2022-09-03T05:43:28.024 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:28 GMT /dsgvcy-25/?versions 2022-09-03T05:43:28.025 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ= 2022-09-03T05:43:28.026 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ=', 'Content-Length': '0'} 2022-09-03T05:43:28.029 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005f0e09f4a928a7eb-006312e980-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.029 DEBUG:boto:b'dsgvcy-25999falseobjnulltrue2022-09-03T05:41:55.704Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:28.030 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:43:28.031 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:43:28.031 DEBUG:boto:path=/dsgvcy-25/ 2022-09-03T05:43:28.032 DEBUG:boto:auth_path=/dsgvcy-25/ 2022-09-03T05:43:28.032 DEBUG:boto:path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.033 DEBUG:boto:auth_path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.034 DEBUG:boto:Method: GET 2022-09-03T05:43:28.034 DEBUG:boto:Path: /dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.035 DEBUG:boto:Data: 2022-09-03T05:43:28.035 DEBUG:boto:Headers: {} 2022-09-03T05:43:28.036 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:43:28.036 DEBUG:boto:Port: 8000 2022-09-03T05:43:28.037 DEBUG:boto:Params: {} 2022-09-03T05:43:28.037 DEBUG:boto:Token: None 2022-09-03T05:43:28.038 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:28 GMT /dsgvcy-25/?versions 2022-09-03T05:43:28.038 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ= 2022-09-03T05:43:28.039 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ=', 'Content-Length': '0'} 2022-09-03T05:43:28.042 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004fa34ad3947f33eb-006312e980-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.043 DEBUG:boto:b'dsgvcy-25999falseobjnulltrue2022-09-03T05:41:55.704Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:28.044 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:43:28.044 DEBUG:boto:path=/dsgvcy-25/ 2022-09-03T05:43:28.045 DEBUG:boto:auth_path=/dsgvcy-25/ 2022-09-03T05:43:28.045 DEBUG:boto:path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.046 DEBUG:boto:auth_path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.047 DEBUG:boto:Method: GET 2022-09-03T05:43:28.047 DEBUG:boto:Path: /dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.048 DEBUG:boto:Data: 2022-09-03T05:43:28.048 DEBUG:boto:Headers: {} 2022-09-03T05:43:28.049 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:43:28.049 DEBUG:boto:Port: 8000 2022-09-03T05:43:28.050 DEBUG:boto:Params: {} 2022-09-03T05:43:28.050 DEBUG:boto:Token: None 2022-09-03T05:43:28.051 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:28 GMT /dsgvcy-25/?versions 2022-09-03T05:43:28.052 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ= 2022-09-03T05:43:28.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ=', 'Content-Length': '0'} 2022-09-03T05:43:28.055 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e68212e83149eda3-006312e980-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.055 DEBUG:boto:b'dsgvcy-25999falseobjnulltrue2022-09-03T05:41:55.704Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:28.056 DEBUG:boto:path=/dsgvcy-25/ 2022-09-03T05:43:28.057 DEBUG:boto:auth_path=/dsgvcy-25/ 2022-09-03T05:43:28.057 DEBUG:boto:path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.058 DEBUG:boto:auth_path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.058 DEBUG:boto:Method: GET 2022-09-03T05:43:28.059 DEBUG:boto:Path: /dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.059 DEBUG:boto:Data: 2022-09-03T05:43:28.060 DEBUG:boto:Headers: {} 2022-09-03T05:43:28.060 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:43:28.061 DEBUG:boto:Port: 8000 2022-09-03T05:43:28.062 DEBUG:boto:Params: {} 2022-09-03T05:43:28.062 DEBUG:boto:Token: None 2022-09-03T05:43:28.063 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:28 GMT /dsgvcy-25/?versions 2022-09-03T05:43:28.064 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ= 2022-09-03T05:43:28.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ=', 'Content-Length': '0'} 2022-09-03T05:43:28.067 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001d0503ab7454317d-006312e980-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.110 DEBUG:boto:b'dsgvcy-25999falseobjnulltrue2022-09-03T05:41:55.704Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:28.111 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:43:28.112 DEBUG:boto:path=/dsgvcy-25/obj 2022-09-03T05:43:28.112 DEBUG:boto:auth_path=/dsgvcy-25/obj 2022-09-03T05:43:28.113 DEBUG:boto:Method: HEAD 2022-09-03T05:43:28.114 DEBUG:boto:Path: /dsgvcy-25/obj 2022-09-03T05:43:28.114 DEBUG:boto:Data: 2022-09-03T05:43:28.115 DEBUG:boto:Headers: {} 2022-09-03T05:43:28.115 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:43:28.116 DEBUG:boto:Port: 8000 2022-09-03T05:43:28.116 DEBUG:boto:Params: {} 2022-09-03T05:43:28.117 DEBUG:boto:Token: None 2022-09-03T05:43:28.118 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:28 GMT /dsgvcy-25/obj 2022-09-03T05:43:28.119 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ljxKib5Vb5fo/Fupl52Tgn0klvU= 2022-09-03T05:43:28.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ljxKib5Vb5fo/Fupl52Tgn0klvU=', 'Content-Length': '0'} 2022-09-03T05:43:28.121 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx000008cf34e573f6d8644-006312e980-10ea-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.122 DEBUG:boto:path=/dsgvcy-25/obj 2022-09-03T05:43:28.123 DEBUG:boto:auth_path=/dsgvcy-25/obj 2022-09-03T05:43:28.124 DEBUG:boto:Method: HEAD 2022-09-03T05:43:28.125 DEBUG:boto:Path: /dsgvcy-25/obj 2022-09-03T05:43:28.126 DEBUG:boto:Data: 2022-09-03T05:43:28.127 DEBUG:boto:Headers: {} 2022-09-03T05:43:28.128 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:43:28.129 DEBUG:boto:Port: 8000 2022-09-03T05:43:28.130 DEBUG:boto:Params: {} 2022-09-03T05:43:28.131 DEBUG:boto:Token: None 2022-09-03T05:43:28.132 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:28 GMT /dsgvcy-25/obj 2022-09-03T05:43:28.133 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ljxKib5Vb5fo/Fupl52Tgn0klvU= 2022-09-03T05:43:28.134 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ljxKib5Vb5fo/Fupl52Tgn0klvU=', 'Content-Length': '0'} 2022-09-03T05:43:28.136 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx000007bd94ef28038b0e0-006312e980-10ff-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.137 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-25 zones={test-zone2, test-zone1} 2022-09-03T05:43:28.138 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-25 zones={test-zone3, test-zone1} 2022-09-03T05:43:28.139 DEBUG:boto:path=/dsgvcy-25/ 2022-09-03T05:43:28.140 DEBUG:boto:auth_path=/dsgvcy-25/ 2022-09-03T05:43:28.141 DEBUG:boto:Method: HEAD 2022-09-03T05:43:28.142 DEBUG:boto:Path: /dsgvcy-25/ 2022-09-03T05:43:28.143 DEBUG:boto:Data: 2022-09-03T05:43:28.143 DEBUG:boto:Headers: {} 2022-09-03T05:43:28.144 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:43:28.145 DEBUG:boto:Port: 8001 2022-09-03T05:43:28.146 DEBUG:boto:Params: {} 2022-09-03T05:43:28.147 DEBUG:boto:Token: None 2022-09-03T05:43:28.148 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:28 GMT /dsgvcy-25/ 2022-09-03T05:43:28.149 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IJqfSPOGrHC/69EtJQtD4I4JOY4= 2022-09-03T05:43:28.150 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IJqfSPOGrHC/69EtJQtD4I4JOY4=', 'Content-Length': '0'} 2022-09-03T05:43:28.153 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000217c6b21d8c9c1d3-006312e980-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:43:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.154 DEBUG:boto:path=/dsgvcy-25/ 2022-09-03T05:43:28.154 DEBUG:boto:auth_path=/dsgvcy-25/ 2022-09-03T05:43:28.155 DEBUG:boto:Method: HEAD 2022-09-03T05:43:28.156 DEBUG:boto:Path: /dsgvcy-25/ 2022-09-03T05:43:28.156 DEBUG:boto:Data: 2022-09-03T05:43:28.157 DEBUG:boto:Headers: {} 2022-09-03T05:43:28.157 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:43:28.158 DEBUG:boto:Port: 8000 2022-09-03T05:43:28.158 DEBUG:boto:Params: {} 2022-09-03T05:43:28.159 DEBUG:boto:Token: None 2022-09-03T05:43:28.159 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:28 GMT /dsgvcy-25/ 2022-09-03T05:43:28.160 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IJqfSPOGrHC/69EtJQtD4I4JOY4= 2022-09-03T05:43:28.161 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IJqfSPOGrHC/69EtJQtD4I4JOY4=', 'Content-Length': '0'} 2022-09-03T05:43:28.164 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000073dd079098c4e05b-006312e980-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:43:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.164 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:43:28.165 DEBUG:boto:path=/dsgvcy-25/ 2022-09-03T05:43:28.166 DEBUG:boto:auth_path=/dsgvcy-25/ 2022-09-03T05:43:28.166 DEBUG:boto:path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.167 DEBUG:boto:auth_path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.167 DEBUG:boto:Method: GET 2022-09-03T05:43:28.168 DEBUG:boto:Path: /dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.168 DEBUG:boto:Data: 2022-09-03T05:43:28.169 DEBUG:boto:Headers: {} 2022-09-03T05:43:28.169 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:43:28.170 DEBUG:boto:Port: 8001 2022-09-03T05:43:28.170 DEBUG:boto:Params: {} 2022-09-03T05:43:28.171 DEBUG:boto:Token: None 2022-09-03T05:43:28.171 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:28 GMT /dsgvcy-25/?versions 2022-09-03T05:43:28.172 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ= 2022-09-03T05:43:28.173 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ=', 'Content-Length': '0'} 2022-09-03T05:43:28.175 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000066451877eb802bd5-006312e980-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.176 DEBUG:boto:b'dsgvcy-25999falseobjnulltrue2022-09-03T05:41:55.704Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:28.177 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:43:28.177 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:43:28.178 DEBUG:boto:path=/dsgvcy-25/ 2022-09-03T05:43:28.178 DEBUG:boto:auth_path=/dsgvcy-25/ 2022-09-03T05:43:28.179 DEBUG:boto:path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.179 DEBUG:boto:auth_path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.180 DEBUG:boto:Method: GET 2022-09-03T05:43:28.181 DEBUG:boto:Path: /dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.181 DEBUG:boto:Data: 2022-09-03T05:43:28.182 DEBUG:boto:Headers: {} 2022-09-03T05:43:28.182 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:43:28.183 DEBUG:boto:Port: 8000 2022-09-03T05:43:28.183 DEBUG:boto:Params: {} 2022-09-03T05:43:28.184 DEBUG:boto:Token: None 2022-09-03T05:43:28.185 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:28 GMT /dsgvcy-25/?versions 2022-09-03T05:43:28.186 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ= 2022-09-03T05:43:28.187 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ=', 'Content-Length': '0'} 2022-09-03T05:43:28.190 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005c865d2380817c43-006312e980-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.191 DEBUG:boto:b'dsgvcy-25999falseobjnulltrue2022-09-03T05:41:55.704Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:28.192 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:43:28.192 DEBUG:boto:path=/dsgvcy-25/ 2022-09-03T05:43:28.193 DEBUG:boto:auth_path=/dsgvcy-25/ 2022-09-03T05:43:28.193 DEBUG:boto:path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.194 DEBUG:boto:auth_path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.194 DEBUG:boto:Method: GET 2022-09-03T05:43:28.195 DEBUG:boto:Path: /dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.196 DEBUG:boto:Data: 2022-09-03T05:43:28.197 DEBUG:boto:Headers: {} 2022-09-03T05:43:28.197 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:43:28.198 DEBUG:boto:Port: 8001 2022-09-03T05:43:28.198 DEBUG:boto:Params: {} 2022-09-03T05:43:28.199 DEBUG:boto:Token: None 2022-09-03T05:43:28.199 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:28 GMT /dsgvcy-25/?versions 2022-09-03T05:43:28.200 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ= 2022-09-03T05:43:28.201 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ=', 'Content-Length': '0'} 2022-09-03T05:43:28.203 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003c025fad8fa63180-006312e980-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.252 DEBUG:boto:b'dsgvcy-25999falseobjnulltrue2022-09-03T05:41:55.704Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:28.255 DEBUG:boto:path=/dsgvcy-25/ 2022-09-03T05:43:28.256 DEBUG:boto:auth_path=/dsgvcy-25/ 2022-09-03T05:43:28.257 DEBUG:boto:path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.258 DEBUG:boto:auth_path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.259 DEBUG:boto:Method: GET 2022-09-03T05:43:28.260 DEBUG:boto:Path: /dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.261 DEBUG:boto:Data: 2022-09-03T05:43:28.262 DEBUG:boto:Headers: {} 2022-09-03T05:43:28.263 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:43:28.264 DEBUG:boto:Port: 8000 2022-09-03T05:43:28.265 DEBUG:boto:Params: {} 2022-09-03T05:43:28.266 DEBUG:boto:Token: None 2022-09-03T05:43:28.267 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:28 GMT /dsgvcy-25/?versions 2022-09-03T05:43:28.268 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ= 2022-09-03T05:43:28.269 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ=', 'Content-Length': '0'} 2022-09-03T05:43:28.273 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f9f8ba68094cec9a-006312e980-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.274 DEBUG:boto:b'dsgvcy-25999falseobjnulltrue2022-09-03T05:41:55.704Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:28.276 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:43:28.277 DEBUG:boto:path=/dsgvcy-25/obj 2022-09-03T05:43:28.278 DEBUG:boto:auth_path=/dsgvcy-25/obj 2022-09-03T05:43:28.279 DEBUG:boto:Method: HEAD 2022-09-03T05:43:28.279 DEBUG:boto:Path: /dsgvcy-25/obj 2022-09-03T05:43:28.280 DEBUG:boto:Data: 2022-09-03T05:43:28.282 DEBUG:boto:Headers: {} 2022-09-03T05:43:28.282 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:43:28.283 DEBUG:boto:Port: 8001 2022-09-03T05:43:28.284 DEBUG:boto:Params: {} 2022-09-03T05:43:28.285 DEBUG:boto:Token: None 2022-09-03T05:43:28.286 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:28 GMT /dsgvcy-25/obj 2022-09-03T05:43:28.286 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ljxKib5Vb5fo/Fupl52Tgn0klvU= 2022-09-03T05:43:28.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ljxKib5Vb5fo/Fupl52Tgn0klvU=', 'Content-Length': '0'} 2022-09-03T05:43:28.290 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx000007955a86667f0e5d4-006312e980-1103-test-zone3'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.291 DEBUG:boto:path=/dsgvcy-25/obj 2022-09-03T05:43:28.292 DEBUG:boto:auth_path=/dsgvcy-25/obj 2022-09-03T05:43:28.293 DEBUG:boto:Method: HEAD 2022-09-03T05:43:28.294 DEBUG:boto:Path: /dsgvcy-25/obj 2022-09-03T05:43:28.295 DEBUG:boto:Data: 2022-09-03T05:43:28.296 DEBUG:boto:Headers: {} 2022-09-03T05:43:28.297 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:43:28.298 DEBUG:boto:Port: 8000 2022-09-03T05:43:28.299 DEBUG:boto:Params: {} 2022-09-03T05:43:28.300 DEBUG:boto:Token: None 2022-09-03T05:43:28.301 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:28 GMT /dsgvcy-25/obj 2022-09-03T05:43:28.302 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ljxKib5Vb5fo/Fupl52Tgn0klvU= 2022-09-03T05:43:28.303 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ljxKib5Vb5fo/Fupl52Tgn0klvU=', 'Content-Length': '0'} 2022-09-03T05:43:28.305 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx000006e1e6dd066ff28f7-006312e980-10ff-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.307 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-25 zones={test-zone3, test-zone1} 2022-09-03T05:43:28.308 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-25 zones={test-zone3, test-zone2} 2022-09-03T05:43:28.309 DEBUG:boto:path=/dsgvcy-25/ 2022-09-03T05:43:28.310 DEBUG:boto:auth_path=/dsgvcy-25/ 2022-09-03T05:43:28.311 DEBUG:boto:Method: HEAD 2022-09-03T05:43:28.312 DEBUG:boto:Path: /dsgvcy-25/ 2022-09-03T05:43:28.313 DEBUG:boto:Data: 2022-09-03T05:43:28.314 DEBUG:boto:Headers: {} 2022-09-03T05:43:28.315 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:43:28.316 DEBUG:boto:Port: 8001 2022-09-03T05:43:28.317 DEBUG:boto:Params: {} 2022-09-03T05:43:28.318 DEBUG:boto:Token: None 2022-09-03T05:43:28.320 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:28 GMT /dsgvcy-25/ 2022-09-03T05:43:28.321 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IJqfSPOGrHC/69EtJQtD4I4JOY4= 2022-09-03T05:43:28.322 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IJqfSPOGrHC/69EtJQtD4I4JOY4=', 'Content-Length': '0'} 2022-09-03T05:43:28.326 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004f2f46a594d27428-006312e980-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:43:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.327 DEBUG:boto:path=/dsgvcy-25/ 2022-09-03T05:43:28.328 DEBUG:boto:auth_path=/dsgvcy-25/ 2022-09-03T05:43:28.329 DEBUG:boto:Method: HEAD 2022-09-03T05:43:28.330 DEBUG:boto:Path: /dsgvcy-25/ 2022-09-03T05:43:28.331 DEBUG:boto:Data: 2022-09-03T05:43:28.332 DEBUG:boto:Headers: {} 2022-09-03T05:43:28.333 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:43:28.334 DEBUG:boto:Port: 8000 2022-09-03T05:43:28.335 DEBUG:boto:Params: {} 2022-09-03T05:43:28.337 DEBUG:boto:Token: None 2022-09-03T05:43:28.338 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:28 GMT /dsgvcy-25/ 2022-09-03T05:43:28.339 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IJqfSPOGrHC/69EtJQtD4I4JOY4= 2022-09-03T05:43:28.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IJqfSPOGrHC/69EtJQtD4I4JOY4=', 'Content-Length': '0'} 2022-09-03T05:43:28.346 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000009a44c335c7caced-006312e980-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:43:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.347 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:43:28.348 DEBUG:boto:path=/dsgvcy-25/ 2022-09-03T05:43:28.351 DEBUG:boto:auth_path=/dsgvcy-25/ 2022-09-03T05:43:28.352 DEBUG:boto:path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.353 DEBUG:boto:auth_path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.354 DEBUG:boto:Method: GET 2022-09-03T05:43:28.355 DEBUG:boto:Path: /dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.356 DEBUG:boto:Data: 2022-09-03T05:43:28.357 DEBUG:boto:Headers: {} 2022-09-03T05:43:28.358 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:43:28.359 DEBUG:boto:Port: 8001 2022-09-03T05:43:28.360 DEBUG:boto:Params: {} 2022-09-03T05:43:28.363 DEBUG:boto:Token: None 2022-09-03T05:43:28.364 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:28 GMT /dsgvcy-25/?versions 2022-09-03T05:43:28.365 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ= 2022-09-03T05:43:28.366 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ=', 'Content-Length': '0'} 2022-09-03T05:43:28.371 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008ea20f9e28d8bb84-006312e980-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.373 DEBUG:boto:b'dsgvcy-25999falseobjnulltrue2022-09-03T05:41:55.704Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:28.375 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:43:28.376 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:43:28.377 DEBUG:boto:path=/dsgvcy-25/ 2022-09-03T05:43:28.378 DEBUG:boto:auth_path=/dsgvcy-25/ 2022-09-03T05:43:28.383 DEBUG:boto:path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.384 DEBUG:boto:auth_path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.384 DEBUG:boto:Method: GET 2022-09-03T05:43:28.385 DEBUG:boto:Path: /dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.386 DEBUG:boto:Data: 2022-09-03T05:43:28.386 DEBUG:boto:Headers: {} 2022-09-03T05:43:28.387 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:43:28.387 DEBUG:boto:Port: 8000 2022-09-03T05:43:28.388 DEBUG:boto:Params: {} 2022-09-03T05:43:28.388 DEBUG:boto:Token: None 2022-09-03T05:43:28.389 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:28 GMT /dsgvcy-25/?versions 2022-09-03T05:43:28.390 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ= 2022-09-03T05:43:28.390 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ=', 'Content-Length': '0'} 2022-09-03T05:43:28.393 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000df62aa168f926546-006312e980-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.394 DEBUG:boto:b'dsgvcy-25999falseobjnulltrue2022-09-03T05:41:55.704Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:28.395 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:43:28.395 DEBUG:boto:path=/dsgvcy-25/ 2022-09-03T05:43:28.396 DEBUG:boto:auth_path=/dsgvcy-25/ 2022-09-03T05:43:28.396 DEBUG:boto:path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.397 DEBUG:boto:auth_path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.397 DEBUG:boto:Method: GET 2022-09-03T05:43:28.398 DEBUG:boto:Path: /dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.398 DEBUG:boto:Data: 2022-09-03T05:43:28.399 DEBUG:boto:Headers: {} 2022-09-03T05:43:28.399 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:43:28.400 DEBUG:boto:Port: 8001 2022-09-03T05:43:28.400 DEBUG:boto:Params: {} 2022-09-03T05:43:28.401 DEBUG:boto:Token: None 2022-09-03T05:43:28.402 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:28 GMT /dsgvcy-25/?versions 2022-09-03T05:43:28.402 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ= 2022-09-03T05:43:28.403 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ=', 'Content-Length': '0'} 2022-09-03T05:43:28.406 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d2b8e5be07060b8e-006312e980-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.406 DEBUG:boto:b'dsgvcy-25999falseobjnulltrue2022-09-03T05:41:55.704Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:28.407 DEBUG:boto:path=/dsgvcy-25/ 2022-09-03T05:43:28.408 DEBUG:boto:auth_path=/dsgvcy-25/ 2022-09-03T05:43:28.408 DEBUG:boto:path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.409 DEBUG:boto:auth_path=/dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.409 DEBUG:boto:Method: GET 2022-09-03T05:43:28.410 DEBUG:boto:Path: /dsgvcy-25/?versions&max-keys=999 2022-09-03T05:43:28.411 DEBUG:boto:Data: 2022-09-03T05:43:28.411 DEBUG:boto:Headers: {} 2022-09-03T05:43:28.412 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:43:28.412 DEBUG:boto:Port: 8000 2022-09-03T05:43:28.413 DEBUG:boto:Params: {} 2022-09-03T05:43:28.413 DEBUG:boto:Token: None 2022-09-03T05:43:28.414 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:43:28 GMT /dsgvcy-25/?versions 2022-09-03T05:43:28.414 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ= 2022-09-03T05:43:28.415 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eOlC7HDN3aJaI6sqVkogP4bOyfQ=', 'Content-Length': '0'} 2022-09-03T05:43:28.417 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d5dd8ee9323037db-006312e980-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.418 DEBUG:boto:b'dsgvcy-25999falseobjnulltrue2022-09-03T05:41:55.704Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:43:28.419 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:43:28.420 DEBUG:boto:path=/dsgvcy-25/obj 2022-09-03T05:43:28.420 DEBUG:boto:auth_path=/dsgvcy-25/obj 2022-09-03T05:43:28.421 DEBUG:boto:Method: HEAD 2022-09-03T05:43:28.421 DEBUG:boto:Path: /dsgvcy-25/obj 2022-09-03T05:43:28.422 DEBUG:boto:Data: 2022-09-03T05:43:28.422 DEBUG:boto:Headers: {} 2022-09-03T05:43:28.423 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:43:28.423 DEBUG:boto:Port: 8001 2022-09-03T05:43:28.424 DEBUG:boto:Params: {} 2022-09-03T05:43:28.424 DEBUG:boto:Token: None 2022-09-03T05:43:28.425 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:28 GMT /dsgvcy-25/obj 2022-09-03T05:43:28.426 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ljxKib5Vb5fo/Fupl52Tgn0klvU= 2022-09-03T05:43:28.426 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ljxKib5Vb5fo/Fupl52Tgn0klvU=', 'Content-Length': '0'} 2022-09-03T05:43:28.428 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx000003abcba5a8deaed34-006312e980-1103-test-zone3'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.429 DEBUG:boto:path=/dsgvcy-25/obj 2022-09-03T05:43:28.430 DEBUG:boto:auth_path=/dsgvcy-25/obj 2022-09-03T05:43:28.430 DEBUG:boto:Method: HEAD 2022-09-03T05:43:28.431 DEBUG:boto:Path: /dsgvcy-25/obj 2022-09-03T05:43:28.431 DEBUG:boto:Data: 2022-09-03T05:43:28.432 DEBUG:boto:Headers: {} 2022-09-03T05:43:28.432 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:43:28.433 DEBUG:boto:Port: 8000 2022-09-03T05:43:28.433 DEBUG:boto:Params: {} 2022-09-03T05:43:28.434 DEBUG:boto:Token: None 2022-09-03T05:43:28.435 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:43:28 GMT /dsgvcy-25/obj 2022-09-03T05:43:28.435 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ljxKib5Vb5fo/Fupl52Tgn0klvU= 2022-09-03T05:43:28.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:43:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ljxKib5Vb5fo/Fupl52Tgn0klvU=', 'Content-Length': '0'} 2022-09-03T05:43:28.437 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx0000089b1d156cb881f48-006312e980-10ea-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:43:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:43:28.438 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-25 zones={test-zone3, test-zone2} 2022-09-03T05:43:28.439 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-26 2022-09-03T05:43:28.440 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-26', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:28.440 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-26', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:28.441 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-26 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:43:28.499 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:43:28.500 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:43:28.542 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:43:28.541+0000 7ff4b30c5600 1 bucket sync caught up with source: 2022-09-03T05:43:28.542 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.10.12, , ] 2022-09-03T05:43:28.544 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000001.10.12, , ] 2022-09-03T05:43:28.545 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:43:28.541+0000 7ff4b30c5600 0 bucket checkpoint complete 2022-09-03T05:43:28.558 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-26 2022-09-03T05:43:28.559 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-26', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:28.560 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-26', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:28.562 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-26 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:43:28.629 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:43:28.629 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:43:28.666 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:43:28.665+0000 7f345704c600 1 bucket sync caught up with source: 2022-09-03T05:43:28.667 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000001.10.12, , ] 2022-09-03T05:43:28.668 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000001.10.12, , ] 2022-09-03T05:43:28.668 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:43:28.665+0000 7f345704c600 0 bucket checkpoint complete 2022-09-03T05:43:28.679 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-26 2022-09-03T05:43:28.680 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-26', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:28.681 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-26', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:28.682 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-26 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:43:28.740 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:43:28.740 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:43:28.774 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:43:28.771+0000 7faae032b600 1 bucket sync caught up with source: 2022-09-03T05:43:28.775 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.10.12, , ] 2022-09-03T05:43:28.776 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000001.10.12, , ] 2022-09-03T05:43:28.777 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:43:28.771+0000 7faae032b600 0 bucket checkpoint complete 2022-09-03T05:43:28.789 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-26 2022-09-03T05:43:28.790 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-26', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:28.791 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-26', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:28.792 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-26 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:43:28.851 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:43:28.852 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:43:28.887 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:43:28.887+0000 7f95477e6600 1 bucket sync caught up with source: 2022-09-03T05:43:28.888 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000001.19.12, , ] 2022-09-03T05:43:28.888 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000001.19.12, , ] 2022-09-03T05:43:28.889 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:43:28.887+0000 7f95477e6600 0 bucket checkpoint complete 2022-09-03T05:43:28.908 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-26 2022-09-03T05:43:28.909 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-26', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:28.910 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-26', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:43:28.911 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-26 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:43:28.968 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:43:28.969 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:43:29.020 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:43:29.020+0000 7f9511554600 1 waiting to reach incremental sync.. 2022-09-03T05:43:34.021 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:43:34.021+0000 7f9511554600 1 waiting to reach incremental sync.. 2022-09-03T05:43:39.022 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:43:39.021+0000 7f9511554600 1 waiting to reach incremental sync.. 2022-09-03T05:43:44.023 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:43:44.021+0000 7f9511554600 1 waiting to reach incremental sync.. 2022-09-03T05:43:49.023 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:43:49.022+0000 7f9511554600 1 waiting to reach incremental sync.. 2022-09-03T05:43:54.024 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:43:54.022+0000 7f9511554600 1 waiting to reach incremental sync.. 2022-09-03T05:43:59.024 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:43:59.023+0000 7f9511554600 1 waiting to reach incremental sync.. 2022-09-03T05:44:04.025 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:44:04.024+0000 7f9511554600 1 waiting to reach incremental sync.. 2022-09-03T05:44:09.027 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:44:09.026+0000 7f9511554600 1 bucket sync caught up with source: 2022-09-03T05:44:09.028 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000001.19.12, , ] 2022-09-03T05:44:09.029 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000001.19.12, , ] 2022-09-03T05:44:09.029 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:44:09.026+0000 7f9511554600 0 bucket checkpoint complete 2022-09-03T05:44:09.043 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-26 2022-09-03T05:44:09.045 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-26 2022-09-03T05:44:09.046 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-26', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:44:09.047 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-26', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:44:09.048 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-26 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:44:09.110 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:44:09.111 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:44:09.169 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:44:09.168+0000 7f853ac29600 1 bucket sync caught up with source: 2022-09-03T05:44:09.169 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000003.50.6, , ] 2022-09-03T05:44:09.170 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000003.50.6, , ] 2022-09-03T05:44:09.170 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:44:09.168+0000 7f853ac29600 0 bucket checkpoint complete 2022-09-03T05:44:09.188 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-26 2022-09-03T05:44:09.189 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-26', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:44:09.189 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-26', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:44:09.190 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-26 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:44:09.253 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:44:09.253 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:44:09.294 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:44:09.294+0000 7fafd51ff600 1 bucket sync caught up with source: 2022-09-03T05:44:09.295 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000003.50.6, , ] 2022-09-03T05:44:09.295 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000003.50.6, , ] 2022-09-03T05:44:09.295 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:44:09.294+0000 7fafd51ff600 0 bucket checkpoint complete 2022-09-03T05:44:09.308 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-26 2022-09-03T05:44:09.309 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-26 zones={test-zone2, test-zone1} 2022-09-03T05:44:09.311 DEBUG:boto:path=/dsgvcy-26/ 2022-09-03T05:44:09.312 DEBUG:boto:auth_path=/dsgvcy-26/ 2022-09-03T05:44:09.314 DEBUG:boto:Method: HEAD 2022-09-03T05:44:09.315 DEBUG:boto:Path: /dsgvcy-26/ 2022-09-03T05:44:09.315 DEBUG:boto:Data: 2022-09-03T05:44:09.316 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.316 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:44:09.317 DEBUG:boto:Port: 8000 2022-09-03T05:44:09.317 DEBUG:boto:Params: {} 2022-09-03T05:44:09.318 DEBUG:boto:Token: None 2022-09-03T05:44:09.319 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/ 2022-09-03T05:44:09.319 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hZiY8VvT+lhfUjvGOOKwqUmHX6c= 2022-09-03T05:44:09.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hZiY8VvT+lhfUjvGOOKwqUmHX6c=', 'Content-Length': '0'} 2022-09-03T05:44:09.323 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000070046fb826752e54-006312e9a9-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.324 DEBUG:boto:path=/dsgvcy-26/ 2022-09-03T05:44:09.325 DEBUG:boto:auth_path=/dsgvcy-26/ 2022-09-03T05:44:09.325 DEBUG:boto:Method: HEAD 2022-09-03T05:44:09.326 DEBUG:boto:Path: /dsgvcy-26/ 2022-09-03T05:44:09.327 DEBUG:boto:Data: 2022-09-03T05:44:09.327 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.328 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:09.330 DEBUG:boto:Port: 8000 2022-09-03T05:44:09.331 DEBUG:boto:Params: {} 2022-09-03T05:44:09.332 DEBUG:boto:Token: None 2022-09-03T05:44:09.333 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/ 2022-09-03T05:44:09.335 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hZiY8VvT+lhfUjvGOOKwqUmHX6c= 2022-09-03T05:44:09.336 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hZiY8VvT+lhfUjvGOOKwqUmHX6c=', 'Content-Length': '0'} 2022-09-03T05:44:09.342 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000177aa06189849a6a-006312e9a9-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.343 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:44:09.345 DEBUG:boto:path=/dsgvcy-26/ 2022-09-03T05:44:09.345 DEBUG:boto:auth_path=/dsgvcy-26/ 2022-09-03T05:44:09.346 DEBUG:boto:path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.347 DEBUG:boto:auth_path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.348 DEBUG:boto:Method: GET 2022-09-03T05:44:09.349 DEBUG:boto:Path: /dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.350 DEBUG:boto:Data: 2022-09-03T05:44:09.351 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.352 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:44:09.352 DEBUG:boto:Port: 8000 2022-09-03T05:44:09.353 DEBUG:boto:Params: {} 2022-09-03T05:44:09.354 DEBUG:boto:Token: None 2022-09-03T05:44:09.355 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/?versions 2022-09-03T05:44:09.356 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4= 2022-09-03T05:44:09.357 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4=', 'Content-Length': '0'} 2022-09-03T05:44:09.361 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000734fcd472b1e6a0b-006312e9a9-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.406 DEBUG:boto:b'dsgvcy-26999falseobjnulltrue2022-09-03T05:41:55.778Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:44:09.408 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:44:09.409 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:44:09.409 DEBUG:boto:path=/dsgvcy-26/ 2022-09-03T05:44:09.410 DEBUG:boto:auth_path=/dsgvcy-26/ 2022-09-03T05:44:09.410 DEBUG:boto:path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.411 DEBUG:boto:auth_path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.413 DEBUG:boto:Method: GET 2022-09-03T05:44:09.413 DEBUG:boto:Path: /dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.414 DEBUG:boto:Data: 2022-09-03T05:44:09.415 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.415 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:09.416 DEBUG:boto:Port: 8000 2022-09-03T05:44:09.416 DEBUG:boto:Params: {} 2022-09-03T05:44:09.417 DEBUG:boto:Token: None 2022-09-03T05:44:09.417 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/?versions 2022-09-03T05:44:09.418 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4= 2022-09-03T05:44:09.419 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4=', 'Content-Length': '0'} 2022-09-03T05:44:09.422 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d18bb85991bc7dcc-006312e9a9-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.423 DEBUG:boto:b'dsgvcy-26999falseobjnulltrue2022-09-03T05:41:55.778Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:44:09.424 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:44:09.425 DEBUG:boto:path=/dsgvcy-26/ 2022-09-03T05:44:09.426 DEBUG:boto:auth_path=/dsgvcy-26/ 2022-09-03T05:44:09.427 DEBUG:boto:path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.428 DEBUG:boto:auth_path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.429 DEBUG:boto:Method: GET 2022-09-03T05:44:09.430 DEBUG:boto:Path: /dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.430 DEBUG:boto:Data: 2022-09-03T05:44:09.431 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.432 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:44:09.433 DEBUG:boto:Port: 8000 2022-09-03T05:44:09.434 DEBUG:boto:Params: {} 2022-09-03T05:44:09.435 DEBUG:boto:Token: None 2022-09-03T05:44:09.436 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/?versions 2022-09-03T05:44:09.437 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4= 2022-09-03T05:44:09.438 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4=', 'Content-Length': '0'} 2022-09-03T05:44:09.441 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fcb62506419cdcbb-006312e9a9-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.443 DEBUG:boto:b'dsgvcy-26999falseobjnulltrue2022-09-03T05:41:55.778Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:44:09.444 DEBUG:boto:path=/dsgvcy-26/ 2022-09-03T05:44:09.445 DEBUG:boto:auth_path=/dsgvcy-26/ 2022-09-03T05:44:09.446 DEBUG:boto:path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.446 DEBUG:boto:auth_path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.447 DEBUG:boto:Method: GET 2022-09-03T05:44:09.448 DEBUG:boto:Path: /dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.448 DEBUG:boto:Data: 2022-09-03T05:44:09.449 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.450 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:09.450 DEBUG:boto:Port: 8000 2022-09-03T05:44:09.451 DEBUG:boto:Params: {} 2022-09-03T05:44:09.451 DEBUG:boto:Token: None 2022-09-03T05:44:09.452 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/?versions 2022-09-03T05:44:09.452 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4= 2022-09-03T05:44:09.453 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4=', 'Content-Length': '0'} 2022-09-03T05:44:09.456 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001fb90de85d9c60e5-006312e9a9-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.457 DEBUG:boto:b'dsgvcy-26999falseobjnulltrue2022-09-03T05:41:55.778Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:44:09.457 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:44:09.458 DEBUG:boto:path=/dsgvcy-26/obj 2022-09-03T05:44:09.459 DEBUG:boto:auth_path=/dsgvcy-26/obj 2022-09-03T05:44:09.459 DEBUG:boto:Method: HEAD 2022-09-03T05:44:09.460 DEBUG:boto:Path: /dsgvcy-26/obj 2022-09-03T05:44:09.460 DEBUG:boto:Data: 2022-09-03T05:44:09.461 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.461 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:44:09.462 DEBUG:boto:Port: 8000 2022-09-03T05:44:09.463 DEBUG:boto:Params: {} 2022-09-03T05:44:09.464 DEBUG:boto:Token: None 2022-09-03T05:44:09.465 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/obj 2022-09-03T05:44:09.466 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fdeFRgZJtqoqoS6vt+ihBlOI7S0= 2022-09-03T05:44:09.467 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fdeFRgZJtqoqoS6vt+ihBlOI7S0=', 'Content-Length': '0'} 2022-09-03T05:44:09.470 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx0000051e777501ef7c7bb-006312e9a9-10ea-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.471 DEBUG:boto:path=/dsgvcy-26/obj 2022-09-03T05:44:09.471 DEBUG:boto:auth_path=/dsgvcy-26/obj 2022-09-03T05:44:09.472 DEBUG:boto:Method: HEAD 2022-09-03T05:44:09.473 DEBUG:boto:Path: /dsgvcy-26/obj 2022-09-03T05:44:09.473 DEBUG:boto:Data: 2022-09-03T05:44:09.474 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.474 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:09.475 DEBUG:boto:Port: 8000 2022-09-03T05:44:09.475 DEBUG:boto:Params: {} 2022-09-03T05:44:09.476 DEBUG:boto:Token: None 2022-09-03T05:44:09.476 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/obj 2022-09-03T05:44:09.477 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fdeFRgZJtqoqoS6vt+ihBlOI7S0= 2022-09-03T05:44:09.478 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fdeFRgZJtqoqoS6vt+ihBlOI7S0=', 'Content-Length': '0'} 2022-09-03T05:44:09.480 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx00000408b70f5831d9044-006312e9a9-10ff-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.481 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-26 zones={test-zone2, test-zone1} 2022-09-03T05:44:09.481 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-26 zones={test-zone3, test-zone1} 2022-09-03T05:44:09.482 DEBUG:boto:path=/dsgvcy-26/ 2022-09-03T05:44:09.482 DEBUG:boto:auth_path=/dsgvcy-26/ 2022-09-03T05:44:09.483 DEBUG:boto:Method: HEAD 2022-09-03T05:44:09.484 DEBUG:boto:Path: /dsgvcy-26/ 2022-09-03T05:44:09.484 DEBUG:boto:Data: 2022-09-03T05:44:09.485 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.485 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:44:09.485 DEBUG:boto:Port: 8001 2022-09-03T05:44:09.486 DEBUG:boto:Params: {} 2022-09-03T05:44:09.487 DEBUG:boto:Token: None 2022-09-03T05:44:09.487 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/ 2022-09-03T05:44:09.488 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hZiY8VvT+lhfUjvGOOKwqUmHX6c= 2022-09-03T05:44:09.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hZiY8VvT+lhfUjvGOOKwqUmHX6c=', 'Content-Length': '0'} 2022-09-03T05:44:09.491 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000031e3f14290597b31-006312e9a9-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.492 DEBUG:boto:path=/dsgvcy-26/ 2022-09-03T05:44:09.492 DEBUG:boto:auth_path=/dsgvcy-26/ 2022-09-03T05:44:09.493 DEBUG:boto:Method: HEAD 2022-09-03T05:44:09.493 DEBUG:boto:Path: /dsgvcy-26/ 2022-09-03T05:44:09.494 DEBUG:boto:Data: 2022-09-03T05:44:09.495 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.495 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:09.495 DEBUG:boto:Port: 8000 2022-09-03T05:44:09.496 DEBUG:boto:Params: {} 2022-09-03T05:44:09.497 DEBUG:boto:Token: None 2022-09-03T05:44:09.497 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/ 2022-09-03T05:44:09.498 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hZiY8VvT+lhfUjvGOOKwqUmHX6c= 2022-09-03T05:44:09.498 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hZiY8VvT+lhfUjvGOOKwqUmHX6c=', 'Content-Length': '0'} 2022-09-03T05:44:09.501 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c67204e235786b5e-006312e9a9-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.502 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:44:09.503 DEBUG:boto:path=/dsgvcy-26/ 2022-09-03T05:44:09.504 DEBUG:boto:auth_path=/dsgvcy-26/ 2022-09-03T05:44:09.505 DEBUG:boto:path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.505 DEBUG:boto:auth_path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.506 DEBUG:boto:Method: GET 2022-09-03T05:44:09.507 DEBUG:boto:Path: /dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.508 DEBUG:boto:Data: 2022-09-03T05:44:09.509 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.510 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:44:09.511 DEBUG:boto:Port: 8001 2022-09-03T05:44:09.512 DEBUG:boto:Params: {} 2022-09-03T05:44:09.513 DEBUG:boto:Token: None 2022-09-03T05:44:09.514 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/?versions 2022-09-03T05:44:09.514 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4= 2022-09-03T05:44:09.515 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4=', 'Content-Length': '0'} 2022-09-03T05:44:09.517 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c870b5891d7cf2b0-006312e9a9-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.562 DEBUG:boto:b'dsgvcy-26999falseobjnulltrue2022-09-03T05:41:55.778Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:44:09.563 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:44:09.563 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:44:09.564 DEBUG:boto:path=/dsgvcy-26/ 2022-09-03T05:44:09.565 DEBUG:boto:auth_path=/dsgvcy-26/ 2022-09-03T05:44:09.565 DEBUG:boto:path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.566 DEBUG:boto:auth_path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.566 DEBUG:boto:Method: GET 2022-09-03T05:44:09.567 DEBUG:boto:Path: /dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.567 DEBUG:boto:Data: 2022-09-03T05:44:09.568 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.568 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:09.569 DEBUG:boto:Port: 8000 2022-09-03T05:44:09.569 DEBUG:boto:Params: {} 2022-09-03T05:44:09.570 DEBUG:boto:Token: None 2022-09-03T05:44:09.570 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/?versions 2022-09-03T05:44:09.571 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4= 2022-09-03T05:44:09.571 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4=', 'Content-Length': '0'} 2022-09-03T05:44:09.574 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000709a015adb37b10c-006312e9a9-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.575 DEBUG:boto:b'dsgvcy-26999falseobjnulltrue2022-09-03T05:41:55.778Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:44:09.576 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:44:09.576 DEBUG:boto:path=/dsgvcy-26/ 2022-09-03T05:44:09.577 DEBUG:boto:auth_path=/dsgvcy-26/ 2022-09-03T05:44:09.577 DEBUG:boto:path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.578 DEBUG:boto:auth_path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.578 DEBUG:boto:Method: GET 2022-09-03T05:44:09.579 DEBUG:boto:Path: /dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.579 DEBUG:boto:Data: 2022-09-03T05:44:09.580 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.580 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:44:09.581 DEBUG:boto:Port: 8001 2022-09-03T05:44:09.581 DEBUG:boto:Params: {} 2022-09-03T05:44:09.582 DEBUG:boto:Token: None 2022-09-03T05:44:09.583 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/?versions 2022-09-03T05:44:09.583 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4= 2022-09-03T05:44:09.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4=', 'Content-Length': '0'} 2022-09-03T05:44:09.586 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006428e1b3df51c75d-006312e9a9-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.587 DEBUG:boto:b'dsgvcy-26999falseobjnulltrue2022-09-03T05:41:55.778Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:44:09.588 DEBUG:boto:path=/dsgvcy-26/ 2022-09-03T05:44:09.588 DEBUG:boto:auth_path=/dsgvcy-26/ 2022-09-03T05:44:09.589 DEBUG:boto:path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.589 DEBUG:boto:auth_path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.590 DEBUG:boto:Method: GET 2022-09-03T05:44:09.590 DEBUG:boto:Path: /dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.591 DEBUG:boto:Data: 2022-09-03T05:44:09.591 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.592 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:09.592 DEBUG:boto:Port: 8000 2022-09-03T05:44:09.593 DEBUG:boto:Params: {} 2022-09-03T05:44:09.593 DEBUG:boto:Token: None 2022-09-03T05:44:09.594 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/?versions 2022-09-03T05:44:09.595 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4= 2022-09-03T05:44:09.595 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4=', 'Content-Length': '0'} 2022-09-03T05:44:09.597 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f55d5096ad3a5e77-006312e9a9-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.638 DEBUG:boto:b'dsgvcy-26999falseobjnulltrue2022-09-03T05:41:55.778Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:44:09.640 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:44:09.641 DEBUG:boto:path=/dsgvcy-26/obj 2022-09-03T05:44:09.642 DEBUG:boto:auth_path=/dsgvcy-26/obj 2022-09-03T05:44:09.643 DEBUG:boto:Method: HEAD 2022-09-03T05:44:09.644 DEBUG:boto:Path: /dsgvcy-26/obj 2022-09-03T05:44:09.645 DEBUG:boto:Data: 2022-09-03T05:44:09.646 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.647 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:44:09.648 DEBUG:boto:Port: 8001 2022-09-03T05:44:09.649 DEBUG:boto:Params: {} 2022-09-03T05:44:09.650 DEBUG:boto:Token: None 2022-09-03T05:44:09.651 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/obj 2022-09-03T05:44:09.652 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fdeFRgZJtqoqoS6vt+ihBlOI7S0= 2022-09-03T05:44:09.653 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fdeFRgZJtqoqoS6vt+ihBlOI7S0=', 'Content-Length': '0'} 2022-09-03T05:44:09.655 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx0000062256051ff197a9e-006312e9a9-1103-test-zone3'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.656 DEBUG:boto:path=/dsgvcy-26/obj 2022-09-03T05:44:09.657 DEBUG:boto:auth_path=/dsgvcy-26/obj 2022-09-03T05:44:09.658 DEBUG:boto:Method: HEAD 2022-09-03T05:44:09.659 DEBUG:boto:Path: /dsgvcy-26/obj 2022-09-03T05:44:09.660 DEBUG:boto:Data: 2022-09-03T05:44:09.661 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.661 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:09.662 DEBUG:boto:Port: 8000 2022-09-03T05:44:09.663 DEBUG:boto:Params: {} 2022-09-03T05:44:09.663 DEBUG:boto:Token: None 2022-09-03T05:44:09.664 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/obj 2022-09-03T05:44:09.664 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fdeFRgZJtqoqoS6vt+ihBlOI7S0= 2022-09-03T05:44:09.665 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fdeFRgZJtqoqoS6vt+ihBlOI7S0=', 'Content-Length': '0'} 2022-09-03T05:44:09.667 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx000001e54d266ac2fe789-006312e9a9-10ff-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.667 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-26 zones={test-zone3, test-zone1} 2022-09-03T05:44:09.668 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-26 zones={test-zone3, test-zone2} 2022-09-03T05:44:09.669 DEBUG:boto:path=/dsgvcy-26/ 2022-09-03T05:44:09.669 DEBUG:boto:auth_path=/dsgvcy-26/ 2022-09-03T05:44:09.670 DEBUG:boto:Method: HEAD 2022-09-03T05:44:09.671 DEBUG:boto:Path: /dsgvcy-26/ 2022-09-03T05:44:09.671 DEBUG:boto:Data: 2022-09-03T05:44:09.672 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.672 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:44:09.673 DEBUG:boto:Port: 8001 2022-09-03T05:44:09.673 DEBUG:boto:Params: {} 2022-09-03T05:44:09.674 DEBUG:boto:Token: None 2022-09-03T05:44:09.675 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/ 2022-09-03T05:44:09.675 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hZiY8VvT+lhfUjvGOOKwqUmHX6c= 2022-09-03T05:44:09.676 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hZiY8VvT+lhfUjvGOOKwqUmHX6c=', 'Content-Length': '0'} 2022-09-03T05:44:09.678 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ea418ed3c30d460e-006312e9a9-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.679 DEBUG:boto:path=/dsgvcy-26/ 2022-09-03T05:44:09.680 DEBUG:boto:auth_path=/dsgvcy-26/ 2022-09-03T05:44:09.681 DEBUG:boto:Method: HEAD 2022-09-03T05:44:09.682 DEBUG:boto:Path: /dsgvcy-26/ 2022-09-03T05:44:09.683 DEBUG:boto:Data: 2022-09-03T05:44:09.684 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.685 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:44:09.686 DEBUG:boto:Port: 8000 2022-09-03T05:44:09.687 DEBUG:boto:Params: {} 2022-09-03T05:44:09.688 DEBUG:boto:Token: None 2022-09-03T05:44:09.689 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/ 2022-09-03T05:44:09.690 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hZiY8VvT+lhfUjvGOOKwqUmHX6c= 2022-09-03T05:44:09.691 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hZiY8VvT+lhfUjvGOOKwqUmHX6c=', 'Content-Length': '0'} 2022-09-03T05:44:09.694 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000048e5ba0b209cb98c-006312e9a9-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.695 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:44:09.696 DEBUG:boto:path=/dsgvcy-26/ 2022-09-03T05:44:09.696 DEBUG:boto:auth_path=/dsgvcy-26/ 2022-09-03T05:44:09.697 DEBUG:boto:path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.697 DEBUG:boto:auth_path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.698 DEBUG:boto:Method: GET 2022-09-03T05:44:09.699 DEBUG:boto:Path: /dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.699 DEBUG:boto:Data: 2022-09-03T05:44:09.700 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.700 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:44:09.701 DEBUG:boto:Port: 8001 2022-09-03T05:44:09.702 DEBUG:boto:Params: {} 2022-09-03T05:44:09.702 DEBUG:boto:Token: None 2022-09-03T05:44:09.703 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/?versions 2022-09-03T05:44:09.704 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4= 2022-09-03T05:44:09.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4=', 'Content-Length': '0'} 2022-09-03T05:44:09.707 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000012fe357e92db8732-006312e9a9-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.750 DEBUG:boto:b'dsgvcy-26999falseobjnulltrue2022-09-03T05:41:55.778Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:44:09.752 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:44:09.752 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:44:09.753 DEBUG:boto:path=/dsgvcy-26/ 2022-09-03T05:44:09.754 DEBUG:boto:auth_path=/dsgvcy-26/ 2022-09-03T05:44:09.754 DEBUG:boto:path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.755 DEBUG:boto:auth_path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.756 DEBUG:boto:Method: GET 2022-09-03T05:44:09.761 DEBUG:boto:Path: /dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.762 DEBUG:boto:Data: 2022-09-03T05:44:09.763 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.763 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:44:09.764 DEBUG:boto:Port: 8000 2022-09-03T05:44:09.765 DEBUG:boto:Params: {} 2022-09-03T05:44:09.766 DEBUG:boto:Token: None 2022-09-03T05:44:09.767 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/?versions 2022-09-03T05:44:09.768 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4= 2022-09-03T05:44:09.769 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4=', 'Content-Length': '0'} 2022-09-03T05:44:09.771 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bada6764c60d5f0c-006312e9a9-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.773 DEBUG:boto:b'dsgvcy-26999falseobjnulltrue2022-09-03T05:41:55.778Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:44:09.775 DEBUG:tasks.rgw_multi.zone_rados:o=obj 2022-09-03T05:44:09.776 DEBUG:boto:path=/dsgvcy-26/ 2022-09-03T05:44:09.777 DEBUG:boto:auth_path=/dsgvcy-26/ 2022-09-03T05:44:09.778 DEBUG:boto:path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.779 DEBUG:boto:auth_path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.780 DEBUG:boto:Method: GET 2022-09-03T05:44:09.781 DEBUG:boto:Path: /dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.782 DEBUG:boto:Data: 2022-09-03T05:44:09.782 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.783 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:44:09.784 DEBUG:boto:Port: 8001 2022-09-03T05:44:09.786 DEBUG:boto:Params: {} 2022-09-03T05:44:09.787 DEBUG:boto:Token: None 2022-09-03T05:44:09.788 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/?versions 2022-09-03T05:44:09.789 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4= 2022-09-03T05:44:09.790 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4=', 'Content-Length': '0'} 2022-09-03T05:44:09.793 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001c4c55c2bdf3f20b-006312e9a9-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.794 DEBUG:boto:b'dsgvcy-26999falseobjnulltrue2022-09-03T05:41:55.778Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:44:09.796 DEBUG:boto:path=/dsgvcy-26/ 2022-09-03T05:44:09.796 DEBUG:boto:auth_path=/dsgvcy-26/ 2022-09-03T05:44:09.797 DEBUG:boto:path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.797 DEBUG:boto:auth_path=/dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.798 DEBUG:boto:Method: GET 2022-09-03T05:44:09.798 DEBUG:boto:Path: /dsgvcy-26/?versions&max-keys=999 2022-09-03T05:44:09.799 DEBUG:boto:Data: 2022-09-03T05:44:09.800 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.800 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:44:09.801 DEBUG:boto:Port: 8000 2022-09-03T05:44:09.801 DEBUG:boto:Params: {} 2022-09-03T05:44:09.802 DEBUG:boto:Token: None 2022-09-03T05:44:09.802 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/?versions 2022-09-03T05:44:09.803 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4= 2022-09-03T05:44:09.803 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LPSMFnuKlKWz5j0SIcH5nrwdnp4=', 'Content-Length': '0'} 2022-09-03T05:44:09.807 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a6b9b155376e79c6-006312e9a9-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.808 DEBUG:boto:b'dsgvcy-26999falseobjnulltrue2022-09-03T05:41:55.778Zrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T05:44:09.809 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj 2022-09-03T05:44:09.810 DEBUG:boto:path=/dsgvcy-26/obj 2022-09-03T05:44:09.811 DEBUG:boto:auth_path=/dsgvcy-26/obj 2022-09-03T05:44:09.812 DEBUG:boto:Method: HEAD 2022-09-03T05:44:09.813 DEBUG:boto:Path: /dsgvcy-26/obj 2022-09-03T05:44:09.814 DEBUG:boto:Data: 2022-09-03T05:44:09.815 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.816 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:44:09.817 DEBUG:boto:Port: 8001 2022-09-03T05:44:09.818 DEBUG:boto:Params: {} 2022-09-03T05:44:09.819 DEBUG:boto:Token: None 2022-09-03T05:44:09.820 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/obj 2022-09-03T05:44:09.821 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fdeFRgZJtqoqoS6vt+ihBlOI7S0= 2022-09-03T05:44:09.822 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fdeFRgZJtqoqoS6vt+ihBlOI7S0=', 'Content-Length': '0'} 2022-09-03T05:44:09.825 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx00000c502e80e6f99da43-006312e9a9-1103-test-zone3'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.826 DEBUG:boto:path=/dsgvcy-26/obj 2022-09-03T05:44:09.827 DEBUG:boto:auth_path=/dsgvcy-26/obj 2022-09-03T05:44:09.828 DEBUG:boto:Method: HEAD 2022-09-03T05:44:09.829 DEBUG:boto:Path: /dsgvcy-26/obj 2022-09-03T05:44:09.830 DEBUG:boto:Data: 2022-09-03T05:44:09.831 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.832 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:44:09.833 DEBUG:boto:Port: 8000 2022-09-03T05:44:09.834 DEBUG:boto:Params: {} 2022-09-03T05:44:09.835 DEBUG:boto:Token: None 2022-09-03T05:44:09.836 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-26/obj 2022-09-03T05:44:09.837 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fdeFRgZJtqoqoS6vt+ihBlOI7S0= 2022-09-03T05:44:09.839 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fdeFRgZJtqoqoS6vt+ihBlOI7S0=', 'Content-Length': '0'} 2022-09-03T05:44:09.842 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx00000919e585d1eff82ca-006312e9a9-10ea-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.843 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-26 zones={test-zone3, test-zone2} 2022-09-03T05:44:09.844 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_suspended_delete_marker_full_sync ... ok 2022-09-03T05:44:09.847 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-27 2022-09-03T05:44:09.848 DEBUG:boto:path=/dsgvcy-27/ 2022-09-03T05:44:09.849 DEBUG:boto:auth_path=/dsgvcy-27/ 2022-09-03T05:44:09.851 DEBUG:boto:Method: PUT 2022-09-03T05:44:09.852 DEBUG:boto:Path: /dsgvcy-27/ 2022-09-03T05:44:09.853 DEBUG:boto:Data: 2022-09-03T05:44:09.854 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.855 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:09.856 DEBUG:boto:Port: 8000 2022-09-03T05:44:09.857 DEBUG:boto:Params: {} 2022-09-03T05:44:09.858 DEBUG:boto:Token: None 2022-09-03T05:44:09.859 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-27/ 2022-09-03T05:44:09.860 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/wFuQSyQmqOIuUvZigSNx5xmFAM= 2022-09-03T05:44:09.861 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/wFuQSyQmqOIuUvZigSNx5xmFAM=', 'Content-Length': '0'} 2022-09-03T05:44:09.891 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000f368baf073f2e6d4-006312e9a9-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.892 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=dsgvcy-28 2022-09-03T05:44:09.892 DEBUG:boto:path=/dsgvcy-28/ 2022-09-03T05:44:09.893 DEBUG:boto:auth_path=/dsgvcy-28/ 2022-09-03T05:44:09.894 DEBUG:boto:Method: PUT 2022-09-03T05:44:09.895 DEBUG:boto:Path: /dsgvcy-28/ 2022-09-03T05:44:09.895 DEBUG:boto:Data: 2022-09-03T05:44:09.896 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.897 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:44:09.898 DEBUG:boto:Port: 8000 2022-09-03T05:44:09.899 DEBUG:boto:Params: {} 2022-09-03T05:44:09.899 DEBUG:boto:Token: None 2022-09-03T05:44:09.900 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-28/ 2022-09-03T05:44:09.901 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:odO9IOFDmZyCG+cr21vBMA6BWfE= 2022-09-03T05:44:09.902 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:odO9IOFDmZyCG+cr21vBMA6BWfE=', 'Content-Length': '0'} 2022-09-03T05:44:09.942 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000451b830e0e055987-006312e9a9-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.943 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=dsgvcy-29 2022-09-03T05:44:09.944 DEBUG:boto:path=/dsgvcy-29/ 2022-09-03T05:44:09.945 DEBUG:boto:auth_path=/dsgvcy-29/ 2022-09-03T05:44:09.946 DEBUG:boto:Method: PUT 2022-09-03T05:44:09.947 DEBUG:boto:Path: /dsgvcy-29/ 2022-09-03T05:44:09.948 DEBUG:boto:Data: 2022-09-03T05:44:09.948 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.949 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:44:09.949 DEBUG:boto:Port: 8001 2022-09-03T05:44:09.950 DEBUG:boto:Params: {} 2022-09-03T05:44:09.951 DEBUG:boto:Token: None 2022-09-03T05:44:09.951 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-29/ 2022-09-03T05:44:09.952 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WgSRXco7i2TCjmS1j9PDum/zttA= 2022-09-03T05:44:09.952 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WgSRXco7i2TCjmS1j9PDum/zttA=', 'Content-Length': '0'} 2022-09-03T05:44:09.977 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000005743b86fe0f6007b-006312e9a9-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:09.979 DEBUG:boto:path=/dsgvcy-27/ 2022-09-03T05:44:09.980 DEBUG:boto:auth_path=/dsgvcy-27/ 2022-09-03T05:44:09.981 DEBUG:boto:path=/dsgvcy-27/?versioning 2022-09-03T05:44:09.982 DEBUG:boto:auth_path=/dsgvcy-27/?versioning 2022-09-03T05:44:09.984 DEBUG:boto:Method: PUT 2022-09-03T05:44:09.985 DEBUG:boto:Path: /dsgvcy-27/?versioning 2022-09-03T05:44:09.990 DEBUG:boto:Data: Enabled Disabled 2022-09-03T05:44:09.991 DEBUG:boto:Headers: {} 2022-09-03T05:44:09.992 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:09.993 DEBUG:boto:Port: 8000 2022-09-03T05:44:09.993 DEBUG:boto:Params: {} 2022-09-03T05:44:09.995 DEBUG:boto:Token: None 2022-09-03T05:44:09.995 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:44:09 GMT /dsgvcy-27/?versioning 2022-09-03T05:44:09.996 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s0Vq4ZN2kjW097dRjDDuyf8D5w0= 2022-09-03T05:44:09.997 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s0Vq4ZN2kjW097dRjDDuyf8D5w0=', 'Content-Length': '228'} 2022-09-03T05:44:10.002 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000006b1ae26db7ad67-006312e9a9-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:10.008 DEBUG:boto:path=/dsgvcy-27/ 2022-09-03T05:44:10.009 DEBUG:boto:auth_path=/dsgvcy-27/ 2022-09-03T05:44:10.010 DEBUG:boto:path=/dsgvcy-27/?versioning 2022-09-03T05:44:10.012 DEBUG:boto:auth_path=/dsgvcy-27/?versioning 2022-09-03T05:44:10.013 DEBUG:boto:Method: GET 2022-09-03T05:44:10.013 DEBUG:boto:Path: /dsgvcy-27/?versioning 2022-09-03T05:44:10.015 DEBUG:boto:Data: 2022-09-03T05:44:10.015 DEBUG:boto:Headers: {} 2022-09-03T05:44:10.017 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:10.018 DEBUG:boto:Port: 8000 2022-09-03T05:44:10.018 DEBUG:boto:Params: {} 2022-09-03T05:44:10.020 DEBUG:boto:Token: None 2022-09-03T05:44:10.021 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:10 GMT /dsgvcy-27/?versioning 2022-09-03T05:44:10.022 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RsCqz8YTFGBhD+5Oj54WDCp4DUw= 2022-09-03T05:44:10.023 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RsCqz8YTFGBhD+5Oj54WDCp4DUw=', 'Content-Length': '0'} 2022-09-03T05:44:10.026 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000e890a4d2be2b1d08-006312e9aa-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '192'), ('Date', 'Sat, 03 Sep 2022 05:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:10.027 DEBUG:boto:EnabledDisabled 2022-09-03T05:44:10.031 DEBUG:boto:path=/dsgvcy-28/ 2022-09-03T05:44:10.032 DEBUG:boto:auth_path=/dsgvcy-28/ 2022-09-03T05:44:10.033 DEBUG:boto:path=/dsgvcy-28/?versioning 2022-09-03T05:44:10.033 DEBUG:boto:auth_path=/dsgvcy-28/?versioning 2022-09-03T05:44:10.034 DEBUG:boto:Method: PUT 2022-09-03T05:44:10.035 DEBUG:boto:Path: /dsgvcy-28/?versioning 2022-09-03T05:44:10.036 DEBUG:boto:Data: Enabled Disabled 2022-09-03T05:44:10.039 DEBUG:boto:Headers: {} 2022-09-03T05:44:10.040 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:44:10.041 DEBUG:boto:Port: 8000 2022-09-03T05:44:10.042 DEBUG:boto:Params: {} 2022-09-03T05:44:10.043 DEBUG:boto:Token: None 2022-09-03T05:44:10.044 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:44:10 GMT /dsgvcy-28/?versioning 2022-09-03T05:44:10.045 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wdLxE4j9ZS7yVcHh3ZrAGCA3xv0= 2022-09-03T05:44:10.046 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wdLxE4j9ZS7yVcHh3ZrAGCA3xv0=', 'Content-Length': '228'} 2022-09-03T05:44:10.054 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000016a95685b439eda1-006312e9aa-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:10.055 DEBUG:boto:path=/dsgvcy-28/ 2022-09-03T05:44:10.056 DEBUG:boto:auth_path=/dsgvcy-28/ 2022-09-03T05:44:10.057 DEBUG:boto:path=/dsgvcy-28/?versioning 2022-09-03T05:44:10.058 DEBUG:boto:auth_path=/dsgvcy-28/?versioning 2022-09-03T05:44:10.059 DEBUG:boto:Method: GET 2022-09-03T05:44:10.060 DEBUG:boto:Path: /dsgvcy-28/?versioning 2022-09-03T05:44:10.061 DEBUG:boto:Data: 2022-09-03T05:44:10.062 DEBUG:boto:Headers: {} 2022-09-03T05:44:10.063 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:44:10.063 DEBUG:boto:Port: 8000 2022-09-03T05:44:10.064 DEBUG:boto:Params: {} 2022-09-03T05:44:10.065 DEBUG:boto:Token: None 2022-09-03T05:44:10.066 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:10 GMT /dsgvcy-28/?versioning 2022-09-03T05:44:10.067 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1MewOrpFLV4qFoZ7AlhGT/ZfkaE= 2022-09-03T05:44:10.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1MewOrpFLV4qFoZ7AlhGT/ZfkaE=', 'Content-Length': '0'} 2022-09-03T05:44:10.073 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000dac7f0da8e0edf8f-006312e9aa-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Content-Length', '192'), ('Date', 'Sat, 03 Sep 2022 05:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:10.074 DEBUG:boto:EnabledDisabled 2022-09-03T05:44:10.075 DEBUG:boto:path=/dsgvcy-29/ 2022-09-03T05:44:10.076 DEBUG:boto:auth_path=/dsgvcy-29/ 2022-09-03T05:44:10.077 DEBUG:boto:path=/dsgvcy-29/?versioning 2022-09-03T05:44:10.078 DEBUG:boto:auth_path=/dsgvcy-29/?versioning 2022-09-03T05:44:10.079 DEBUG:boto:Method: PUT 2022-09-03T05:44:10.080 DEBUG:boto:Path: /dsgvcy-29/?versioning 2022-09-03T05:44:10.080 DEBUG:boto:Data: Enabled Disabled 2022-09-03T05:44:10.081 DEBUG:boto:Headers: {} 2022-09-03T05:44:10.082 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:44:10.083 DEBUG:boto:Port: 8001 2022-09-03T05:44:10.084 DEBUG:boto:Params: {} 2022-09-03T05:44:10.085 DEBUG:boto:Token: None 2022-09-03T05:44:10.086 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:44:10 GMT /dsgvcy-29/?versioning 2022-09-03T05:44:10.087 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X5vBOT03L/bxi9fiGZZxzNQDQyE= 2022-09-03T05:44:10.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X5vBOT03L/bxi9fiGZZxzNQDQyE=', 'Content-Length': '228'} 2022-09-03T05:44:10.104 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000aeb02e3ba6989012-006312e9aa-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:10.105 DEBUG:boto:path=/dsgvcy-29/ 2022-09-03T05:44:10.106 DEBUG:boto:auth_path=/dsgvcy-29/ 2022-09-03T05:44:10.106 DEBUG:boto:path=/dsgvcy-29/?versioning 2022-09-03T05:44:10.107 DEBUG:boto:auth_path=/dsgvcy-29/?versioning 2022-09-03T05:44:10.107 DEBUG:boto:Method: GET 2022-09-03T05:44:10.108 DEBUG:boto:Path: /dsgvcy-29/?versioning 2022-09-03T05:44:10.108 DEBUG:boto:Data: 2022-09-03T05:44:10.108 DEBUG:boto:Headers: {} 2022-09-03T05:44:10.109 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:44:10.109 DEBUG:boto:Port: 8001 2022-09-03T05:44:10.110 DEBUG:boto:Params: {} 2022-09-03T05:44:10.111 DEBUG:boto:Token: None 2022-09-03T05:44:10.111 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:10 GMT /dsgvcy-29/?versioning 2022-09-03T05:44:10.112 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+QcmpIWTOKAwG10hou9YfWgXRPk= 2022-09-03T05:44:10.113 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+QcmpIWTOKAwG10hou9YfWgXRPk=', 'Content-Length': '0'} 2022-09-03T05:44:10.114 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000ce17453d7b2be029-006312e9aa-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Content-Length', '192'), ('Date', 'Sat, 03 Sep 2022 05:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:10.115 DEBUG:boto:EnabledDisabled 2022-09-03T05:44:10.115 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_versioning ... ok 2022-09-03T05:44:10.116 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-30 2022-09-03T05:44:10.117 DEBUG:boto:path=/dsgvcy-30/ 2022-09-03T05:44:10.117 DEBUG:boto:auth_path=/dsgvcy-30/ 2022-09-03T05:44:10.118 DEBUG:boto:Method: PUT 2022-09-03T05:44:10.119 DEBUG:boto:Path: /dsgvcy-30/ 2022-09-03T05:44:10.119 DEBUG:boto:Data: 2022-09-03T05:44:10.120 DEBUG:boto:Headers: {} 2022-09-03T05:44:10.120 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:10.121 DEBUG:boto:Port: 8000 2022-09-03T05:44:10.121 DEBUG:boto:Params: {} 2022-09-03T05:44:10.122 DEBUG:boto:Token: None 2022-09-03T05:44:10.122 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:44:10 GMT /dsgvcy-30/ 2022-09-03T05:44:10.123 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V+KXDvHi1A+87eMPcTVNwnU84eg= 2022-09-03T05:44:10.125 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V+KXDvHi1A+87eMPcTVNwnU84eg=', 'Content-Length': '0'} 2022-09-03T05:44:10.153 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000005931c1183e32bcab-006312e9aa-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:10.154 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=dsgvcy-31 2022-09-03T05:44:10.155 DEBUG:boto:path=/dsgvcy-31/ 2022-09-03T05:44:10.155 DEBUG:boto:auth_path=/dsgvcy-31/ 2022-09-03T05:44:10.156 DEBUG:boto:Method: PUT 2022-09-03T05:44:10.156 DEBUG:boto:Path: /dsgvcy-31/ 2022-09-03T05:44:10.157 DEBUG:boto:Data: 2022-09-03T05:44:10.157 DEBUG:boto:Headers: {} 2022-09-03T05:44:10.158 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:44:10.158 DEBUG:boto:Port: 8000 2022-09-03T05:44:10.159 DEBUG:boto:Params: {} 2022-09-03T05:44:10.160 DEBUG:boto:Token: None 2022-09-03T05:44:10.160 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:44:10 GMT /dsgvcy-31/ 2022-09-03T05:44:10.161 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9Oqo8zaNtltl79n89x7tFTudvCE= 2022-09-03T05:44:10.162 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9Oqo8zaNtltl79n89x7tFTudvCE=', 'Content-Length': '0'} 2022-09-03T05:44:10.195 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000005a30c8068876887c-006312e9aa-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:10.196 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=dsgvcy-32 2022-09-03T05:44:10.197 DEBUG:boto:path=/dsgvcy-32/ 2022-09-03T05:44:10.198 DEBUG:boto:auth_path=/dsgvcy-32/ 2022-09-03T05:44:10.198 DEBUG:boto:Method: PUT 2022-09-03T05:44:10.199 DEBUG:boto:Path: /dsgvcy-32/ 2022-09-03T05:44:10.200 DEBUG:boto:Data: 2022-09-03T05:44:10.201 DEBUG:boto:Headers: {} 2022-09-03T05:44:10.201 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:44:10.203 DEBUG:boto:Port: 8001 2022-09-03T05:44:10.204 DEBUG:boto:Params: {} 2022-09-03T05:44:10.205 DEBUG:boto:Token: None 2022-09-03T05:44:10.206 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:44:10 GMT /dsgvcy-32/ 2022-09-03T05:44:10.207 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hgQYoez3YMqgsb7tdqhz1Lsmh0c= 2022-09-03T05:44:10.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hgQYoez3YMqgsb7tdqhz1Lsmh0c=', 'Content-Length': '0'} 2022-09-03T05:44:10.237 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000676782f9dbf33aed-006312e9aa-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:10.238 DEBUG:boto:path=/dsgvcy-30/ 2022-09-03T05:44:10.239 DEBUG:boto:auth_path=/dsgvcy-30/ 2022-09-03T05:44:10.240 DEBUG:boto:path=/dsgvcy-30/?acl 2022-09-03T05:44:10.240 DEBUG:boto:auth_path=/dsgvcy-30/?acl 2022-09-03T05:44:10.243 DEBUG:boto:Method: GET 2022-09-03T05:44:10.244 DEBUG:boto:Path: /dsgvcy-30/?acl 2022-09-03T05:44:10.245 DEBUG:boto:Data: 2022-09-03T05:44:10.246 DEBUG:boto:Headers: {} 2022-09-03T05:44:10.247 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:10.247 DEBUG:boto:Port: 8000 2022-09-03T05:44:10.248 DEBUG:boto:Params: {} 2022-09-03T05:44:10.248 DEBUG:boto:Token: None 2022-09-03T05:44:10.249 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:10 GMT /dsgvcy-30/?acl 2022-09-03T05:44:10.249 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lVQt0Ter6Csm1YIMGsEw+jH4ySU= 2022-09-03T05:44:10.250 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lVQt0Ter6Csm1YIMGsEw+jH4ySU=', 'Content-Length': '0'} 2022-09-03T05:44:10.252 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000ebb82dcd62b129d3-006312e9aa-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '489'), ('Date', 'Sat, 03 Sep 2022 05:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:10.253 DEBUG:boto:path=/dsgvcy-30/ 2022-09-03T05:44:10.254 DEBUG:boto:auth_path=/dsgvcy-30/ 2022-09-03T05:44:10.254 DEBUG:boto:path=/dsgvcy-30/?acl 2022-09-03T05:44:10.255 DEBUG:boto:auth_path=/dsgvcy-30/?acl 2022-09-03T05:44:10.255 DEBUG:boto:Method: PUT 2022-09-03T05:44:10.256 DEBUG:boto:Path: /dsgvcy-30/?acl 2022-09-03T05:44:10.256 DEBUG:boto:Data: 2022-09-03T05:44:10.257 DEBUG:boto:Headers: {'x-amz-acl': 'public-read'} 2022-09-03T05:44:10.257 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:10.258 DEBUG:boto:Port: 8000 2022-09-03T05:44:10.258 DEBUG:boto:Params: {} 2022-09-03T05:44:10.259 DEBUG:boto:Token: None 2022-09-03T05:44:10.260 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:44:10 GMT x-amz-acl:public-read /dsgvcy-30/?acl 2022-09-03T05:44:10.261 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fUmYxh/xnwd1+hd4P3aOE0FA5ek= 2022-09-03T05:44:10.262 DEBUG:boto:Final headers: {'x-amz-acl': 'public-read', 'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fUmYxh/xnwd1+hd4P3aOE0FA5ek=', 'Content-Length': '0'} 2022-09-03T05:44:10.268 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000000496f64393a8005e-006312e9aa-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:10.270 DEBUG:boto:path=/dsgvcy-30/ 2022-09-03T05:44:10.271 DEBUG:boto:auth_path=/dsgvcy-30/ 2022-09-03T05:44:10.272 DEBUG:boto:path=/dsgvcy-30/?acl 2022-09-03T05:44:10.273 DEBUG:boto:auth_path=/dsgvcy-30/?acl 2022-09-03T05:44:10.274 DEBUG:boto:Method: GET 2022-09-03T05:44:10.275 DEBUG:boto:Path: /dsgvcy-30/?acl 2022-09-03T05:44:10.276 DEBUG:boto:Data: 2022-09-03T05:44:10.277 DEBUG:boto:Headers: {} 2022-09-03T05:44:10.278 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:10.279 DEBUG:boto:Port: 8000 2022-09-03T05:44:10.279 DEBUG:boto:Params: {} 2022-09-03T05:44:10.280 DEBUG:boto:Token: None 2022-09-03T05:44:10.281 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:10 GMT /dsgvcy-30/?acl 2022-09-03T05:44:10.281 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lVQt0Ter6Csm1YIMGsEw+jH4ySU= 2022-09-03T05:44:10.282 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lVQt0Ter6Csm1YIMGsEw+jH4ySU=', 'Content-Length': '0'} 2022-09-03T05:44:10.285 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000008598d13bd140c59-006312e9aa-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '681'), ('Date', 'Sat, 03 Sep 2022 05:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:10.326 DEBUG:boto:path=/dsgvcy-31/ 2022-09-03T05:44:10.328 DEBUG:boto:auth_path=/dsgvcy-31/ 2022-09-03T05:44:10.329 DEBUG:boto:path=/dsgvcy-31/?acl 2022-09-03T05:44:10.330 DEBUG:boto:auth_path=/dsgvcy-31/?acl 2022-09-03T05:44:10.331 DEBUG:boto:Method: GET 2022-09-03T05:44:10.331 DEBUG:boto:Path: /dsgvcy-31/?acl 2022-09-03T05:44:10.332 DEBUG:boto:Data: 2022-09-03T05:44:10.333 DEBUG:boto:Headers: {} 2022-09-03T05:44:10.334 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:44:10.334 DEBUG:boto:Port: 8000 2022-09-03T05:44:10.335 DEBUG:boto:Params: {} 2022-09-03T05:44:10.335 DEBUG:boto:Token: None 2022-09-03T05:44:10.336 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:10 GMT /dsgvcy-31/?acl 2022-09-03T05:44:10.336 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OPpL4Au9mzRs/XJZdOumcagLCbA= 2022-09-03T05:44:10.337 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OPpL4Au9mzRs/XJZdOumcagLCbA=', 'Content-Length': '0'} 2022-09-03T05:44:10.339 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000007f959441ec1c6c30-006312e9aa-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Content-Length', '489'), ('Date', 'Sat, 03 Sep 2022 05:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:10.339 DEBUG:boto:path=/dsgvcy-31/ 2022-09-03T05:44:10.340 DEBUG:boto:auth_path=/dsgvcy-31/ 2022-09-03T05:44:10.341 DEBUG:boto:path=/dsgvcy-31/?acl 2022-09-03T05:44:10.341 DEBUG:boto:auth_path=/dsgvcy-31/?acl 2022-09-03T05:44:10.341 DEBUG:boto:Method: PUT 2022-09-03T05:44:10.342 DEBUG:boto:Path: /dsgvcy-31/?acl 2022-09-03T05:44:10.343 DEBUG:boto:Data: 2022-09-03T05:44:10.343 DEBUG:boto:Headers: {'x-amz-acl': 'public-read'} 2022-09-03T05:44:10.343 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:44:10.344 DEBUG:boto:Port: 8000 2022-09-03T05:44:10.344 DEBUG:boto:Params: {} 2022-09-03T05:44:10.345 DEBUG:boto:Token: None 2022-09-03T05:44:10.346 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:44:10 GMT x-amz-acl:public-read /dsgvcy-31/?acl 2022-09-03T05:44:10.346 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1pnN/yljmAznTj9iNzrLS+fWLWo= 2022-09-03T05:44:10.347 DEBUG:boto:Final headers: {'x-amz-acl': 'public-read', 'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1pnN/yljmAznTj9iNzrLS+fWLWo=', 'Content-Length': '0'} 2022-09-03T05:44:10.359 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000004a2d54c6237d25ea-006312e9aa-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:10.360 DEBUG:boto:path=/dsgvcy-31/ 2022-09-03T05:44:10.361 DEBUG:boto:auth_path=/dsgvcy-31/ 2022-09-03T05:44:10.362 DEBUG:boto:path=/dsgvcy-31/?acl 2022-09-03T05:44:10.363 DEBUG:boto:auth_path=/dsgvcy-31/?acl 2022-09-03T05:44:10.364 DEBUG:boto:Method: GET 2022-09-03T05:44:10.364 DEBUG:boto:Path: /dsgvcy-31/?acl 2022-09-03T05:44:10.365 DEBUG:boto:Data: 2022-09-03T05:44:10.366 DEBUG:boto:Headers: {} 2022-09-03T05:44:10.367 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:44:10.368 DEBUG:boto:Port: 8000 2022-09-03T05:44:10.368 DEBUG:boto:Params: {} 2022-09-03T05:44:10.369 DEBUG:boto:Token: None 2022-09-03T05:44:10.370 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:10 GMT /dsgvcy-31/?acl 2022-09-03T05:44:10.371 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OPpL4Au9mzRs/XJZdOumcagLCbA= 2022-09-03T05:44:10.371 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OPpL4Au9mzRs/XJZdOumcagLCbA=', 'Content-Length': '0'} 2022-09-03T05:44:10.373 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000833157bb2c2a5f2a-006312e9aa-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Content-Length', '681'), ('Date', 'Sat, 03 Sep 2022 05:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:10.375 DEBUG:boto:path=/dsgvcy-32/ 2022-09-03T05:44:10.376 DEBUG:boto:auth_path=/dsgvcy-32/ 2022-09-03T05:44:10.377 DEBUG:boto:path=/dsgvcy-32/?acl 2022-09-03T05:44:10.377 DEBUG:boto:auth_path=/dsgvcy-32/?acl 2022-09-03T05:44:10.378 DEBUG:boto:Method: GET 2022-09-03T05:44:10.379 DEBUG:boto:Path: /dsgvcy-32/?acl 2022-09-03T05:44:10.380 DEBUG:boto:Data: 2022-09-03T05:44:10.381 DEBUG:boto:Headers: {} 2022-09-03T05:44:10.382 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:44:10.383 DEBUG:boto:Port: 8001 2022-09-03T05:44:10.383 DEBUG:boto:Params: {} 2022-09-03T05:44:10.384 DEBUG:boto:Token: None 2022-09-03T05:44:10.385 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:10 GMT /dsgvcy-32/?acl 2022-09-03T05:44:10.386 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M4mbmIAIDGagu5ukUeSDkHhlfmk= 2022-09-03T05:44:10.387 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M4mbmIAIDGagu5ukUeSDkHhlfmk=', 'Content-Length': '0'} 2022-09-03T05:44:10.389 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000008966663e1deda066-006312e9aa-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Content-Length', '489'), ('Date', 'Sat, 03 Sep 2022 05:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:10.389 DEBUG:boto:path=/dsgvcy-32/ 2022-09-03T05:44:10.390 DEBUG:boto:auth_path=/dsgvcy-32/ 2022-09-03T05:44:10.391 DEBUG:boto:path=/dsgvcy-32/?acl 2022-09-03T05:44:10.391 DEBUG:boto:auth_path=/dsgvcy-32/?acl 2022-09-03T05:44:10.392 DEBUG:boto:Method: PUT 2022-09-03T05:44:10.392 DEBUG:boto:Path: /dsgvcy-32/?acl 2022-09-03T05:44:10.393 DEBUG:boto:Data: 2022-09-03T05:44:10.394 DEBUG:boto:Headers: {'x-amz-acl': 'public-read'} 2022-09-03T05:44:10.394 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:44:10.395 DEBUG:boto:Port: 8001 2022-09-03T05:44:10.395 DEBUG:boto:Params: {} 2022-09-03T05:44:10.396 DEBUG:boto:Token: None 2022-09-03T05:44:10.396 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:44:10 GMT x-amz-acl:public-read /dsgvcy-32/?acl 2022-09-03T05:44:10.397 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EQmfVfoQRd71JXwoSZlg+DvKiOY= 2022-09-03T05:44:10.398 DEBUG:boto:Final headers: {'x-amz-acl': 'public-read', 'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EQmfVfoQRd71JXwoSZlg+DvKiOY=', 'Content-Length': '0'} 2022-09-03T05:44:10.407 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000dd063b52142bd08d-006312e9aa-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:10.408 DEBUG:boto:path=/dsgvcy-32/ 2022-09-03T05:44:10.409 DEBUG:boto:auth_path=/dsgvcy-32/ 2022-09-03T05:44:10.409 DEBUG:boto:path=/dsgvcy-32/?acl 2022-09-03T05:44:10.410 DEBUG:boto:auth_path=/dsgvcy-32/?acl 2022-09-03T05:44:10.410 DEBUG:boto:Method: GET 2022-09-03T05:44:10.411 DEBUG:boto:Path: /dsgvcy-32/?acl 2022-09-03T05:44:10.411 DEBUG:boto:Data: 2022-09-03T05:44:10.412 DEBUG:boto:Headers: {} 2022-09-03T05:44:10.412 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:44:10.413 DEBUG:boto:Port: 8001 2022-09-03T05:44:10.413 DEBUG:boto:Params: {} 2022-09-03T05:44:10.414 DEBUG:boto:Token: None 2022-09-03T05:44:10.414 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:10 GMT /dsgvcy-32/?acl 2022-09-03T05:44:10.415 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M4mbmIAIDGagu5ukUeSDkHhlfmk= 2022-09-03T05:44:10.416 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M4mbmIAIDGagu5ukUeSDkHhlfmk=', 'Content-Length': '0'} 2022-09-03T05:44:10.417 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000fc041ed6f4fcc3c9-006312e9aa-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Content-Length', '681'), ('Date', 'Sat, 03 Sep 2022 05:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:10.418 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_acl ... ok 2022-09-03T05:44:10.419 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-33 2022-09-03T05:44:10.420 DEBUG:boto:path=/dsgvcy-33/ 2022-09-03T05:44:10.420 DEBUG:boto:auth_path=/dsgvcy-33/ 2022-09-03T05:44:10.421 DEBUG:boto:Method: PUT 2022-09-03T05:44:10.421 DEBUG:boto:Path: /dsgvcy-33/ 2022-09-03T05:44:10.422 DEBUG:boto:Data: 2022-09-03T05:44:10.422 DEBUG:boto:Headers: {} 2022-09-03T05:44:10.423 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:10.423 DEBUG:boto:Port: 8000 2022-09-03T05:44:10.424 DEBUG:boto:Params: {} 2022-09-03T05:44:10.424 DEBUG:boto:Token: None 2022-09-03T05:44:10.425 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:44:10 GMT /dsgvcy-33/ 2022-09-03T05:44:10.426 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jiqTsgNUJx6D7tV6bjXE/yxuNSw= 2022-09-03T05:44:10.426 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jiqTsgNUJx6D7tV6bjXE/yxuNSw=', 'Content-Length': '0'} 2022-09-03T05:44:10.443 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000003abab3a083e3cd6e-006312e9aa-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:10.443 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=dsgvcy-34 2022-09-03T05:44:10.444 DEBUG:boto:path=/dsgvcy-34/ 2022-09-03T05:44:10.445 DEBUG:boto:auth_path=/dsgvcy-34/ 2022-09-03T05:44:10.445 DEBUG:boto:Method: PUT 2022-09-03T05:44:10.446 DEBUG:boto:Path: /dsgvcy-34/ 2022-09-03T05:44:10.446 DEBUG:boto:Data: 2022-09-03T05:44:10.447 DEBUG:boto:Headers: {} 2022-09-03T05:44:10.448 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:44:10.448 DEBUG:boto:Port: 8000 2022-09-03T05:44:10.449 DEBUG:boto:Params: {} 2022-09-03T05:44:10.449 DEBUG:boto:Token: None 2022-09-03T05:44:10.450 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:44:10 GMT /dsgvcy-34/ 2022-09-03T05:44:10.450 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yfvumZ+u+cU3xZTTVHRZ1FeJgqs= 2022-09-03T05:44:10.451 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yfvumZ+u+cU3xZTTVHRZ1FeJgqs=', 'Content-Length': '0'} 2022-09-03T05:44:10.493 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000b0db31adc9cb28e5-006312e9aa-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:10.494 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=dsgvcy-35 2022-09-03T05:44:10.495 DEBUG:boto:path=/dsgvcy-35/ 2022-09-03T05:44:10.495 DEBUG:boto:auth_path=/dsgvcy-35/ 2022-09-03T05:44:10.496 DEBUG:boto:Method: PUT 2022-09-03T05:44:10.497 DEBUG:boto:Path: /dsgvcy-35/ 2022-09-03T05:44:10.497 DEBUG:boto:Data: 2022-09-03T05:44:10.498 DEBUG:boto:Headers: {} 2022-09-03T05:44:10.498 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:44:10.500 DEBUG:boto:Port: 8001 2022-09-03T05:44:10.501 DEBUG:boto:Params: {} 2022-09-03T05:44:10.502 DEBUG:boto:Token: None 2022-09-03T05:44:10.503 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:44:10 GMT /dsgvcy-35/ 2022-09-03T05:44:10.504 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1wWMJ7jcVvQPX44C2rJTwY5zlxM= 2022-09-03T05:44:10.505 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1wWMJ7jcVvQPX44C2rJTwY5zlxM=', 'Content-Length': '0'} 2022-09-03T05:44:10.561 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000008c45330c75228289-006312e9aa-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:10.563 DEBUG:boto:path=/dsgvcy-33/ 2022-09-03T05:44:10.563 DEBUG:boto:auth_path=/dsgvcy-33/ 2022-09-03T05:44:10.564 DEBUG:boto:path=/dsgvcy-33/?cors 2022-09-03T05:44:10.565 DEBUG:boto:auth_path=/dsgvcy-33/?cors 2022-09-03T05:44:10.565 DEBUG:boto:Method: PUT 2022-09-03T05:44:10.566 DEBUG:boto:Path: /dsgvcy-33/?cors 2022-09-03T05:44:10.567 DEBUG:boto:Data: DELETEhttps://www.example.com*3000 2022-09-03T05:44:10.567 DEBUG:boto:Headers: {'Content-MD5': 'c5gEzAIzM7zcw8WVE+AbrQ==', 'Content-Type': 'text/xml'} 2022-09-03T05:44:10.568 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:10.568 DEBUG:boto:Port: 8000 2022-09-03T05:44:10.569 DEBUG:boto:Params: {} 2022-09-03T05:44:10.570 DEBUG:boto:Token: None 2022-09-03T05:44:10.571 DEBUG:boto:StringToSign: PUT c5gEzAIzM7zcw8WVE+AbrQ== text/xml Sat, 03 Sep 2022 05:44:10 GMT /dsgvcy-33/?cors 2022-09-03T05:44:10.571 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K5DdxGcMPW13kJjhFy7lUNN/QOE= 2022-09-03T05:44:10.572 DEBUG:boto:Final headers: {'Content-MD5': 'c5gEzAIzM7zcw8WVE+AbrQ==', 'Content-Type': 'text/xml', 'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K5DdxGcMPW13kJjhFy7lUNN/QOE=', 'Content-Length': '218'} 2022-09-03T05:44:10.577 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000052400e668de3b67-006312e9aa-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:10.579 DEBUG:boto:path=/dsgvcy-33/ 2022-09-03T05:44:10.579 DEBUG:boto:auth_path=/dsgvcy-33/ 2022-09-03T05:44:10.581 DEBUG:boto:path=/dsgvcy-33/?cors 2022-09-03T05:44:10.581 DEBUG:boto:auth_path=/dsgvcy-33/?cors 2022-09-03T05:44:10.582 DEBUG:boto:Method: GET 2022-09-03T05:44:10.582 DEBUG:boto:Path: /dsgvcy-33/?cors 2022-09-03T05:44:10.583 DEBUG:boto:Data: 2022-09-03T05:44:10.584 DEBUG:boto:Headers: {} 2022-09-03T05:44:10.584 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:10.585 DEBUG:boto:Port: 8000 2022-09-03T05:44:10.586 DEBUG:boto:Params: {} 2022-09-03T05:44:10.586 DEBUG:boto:Token: None 2022-09-03T05:44:10.587 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:10 GMT /dsgvcy-33/?cors 2022-09-03T05:44:10.588 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DwXTHovuORGqqjx5zD8qQJHZGOY= 2022-09-03T05:44:10.588 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DwXTHovuORGqqjx5zD8qQJHZGOY=', 'Content-Length': '0'} 2022-09-03T05:44:10.590 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000a23a6de34b21a035-006312e9aa-10ff-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '266'), ('Date', 'Sat, 03 Sep 2022 05:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:10.590 DEBUG:boto:b'DELETEhttps://www.example.com*3000' 2022-09-03T05:44:10.591 DEBUG:boto:path=/dsgvcy-34/ 2022-09-03T05:44:10.592 DEBUG:boto:auth_path=/dsgvcy-34/ 2022-09-03T05:44:10.592 DEBUG:boto:path=/dsgvcy-34/?cors 2022-09-03T05:44:10.593 DEBUG:boto:auth_path=/dsgvcy-34/?cors 2022-09-03T05:44:10.594 DEBUG:boto:Method: PUT 2022-09-03T05:44:10.594 DEBUG:boto:Path: /dsgvcy-34/?cors 2022-09-03T05:44:10.595 DEBUG:boto:Data: DELETEhttps://www.example.com*3000 2022-09-03T05:44:10.595 DEBUG:boto:Headers: {'Content-MD5': 'c5gEzAIzM7zcw8WVE+AbrQ==', 'Content-Type': 'text/xml'} 2022-09-03T05:44:10.596 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:44:10.596 DEBUG:boto:Port: 8000 2022-09-03T05:44:10.597 DEBUG:boto:Params: {} 2022-09-03T05:44:10.598 DEBUG:boto:Token: None 2022-09-03T05:44:10.599 DEBUG:boto:StringToSign: PUT c5gEzAIzM7zcw8WVE+AbrQ== text/xml Sat, 03 Sep 2022 05:44:10 GMT /dsgvcy-34/?cors 2022-09-03T05:44:10.600 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CeLrRY0j9TTA6ubtzvzL2sBtgVA= 2022-09-03T05:44:10.601 DEBUG:boto:Final headers: {'Content-MD5': 'c5gEzAIzM7zcw8WVE+AbrQ==', 'Content-Type': 'text/xml', 'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CeLrRY0j9TTA6ubtzvzL2sBtgVA=', 'Content-Length': '218'} 2022-09-03T05:44:11.613 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000149dae65a5d2b0a2-006312e9aa-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:11.614 DEBUG:boto:path=/dsgvcy-34/ 2022-09-03T05:44:11.614 DEBUG:boto:auth_path=/dsgvcy-34/ 2022-09-03T05:44:11.615 DEBUG:boto:path=/dsgvcy-34/?cors 2022-09-03T05:44:11.615 DEBUG:boto:auth_path=/dsgvcy-34/?cors 2022-09-03T05:44:11.617 DEBUG:boto:Method: GET 2022-09-03T05:44:11.618 DEBUG:boto:Path: /dsgvcy-34/?cors 2022-09-03T05:44:11.619 DEBUG:boto:Data: 2022-09-03T05:44:11.619 DEBUG:boto:Headers: {} 2022-09-03T05:44:11.620 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:44:11.620 DEBUG:boto:Port: 8000 2022-09-03T05:44:11.621 DEBUG:boto:Params: {} 2022-09-03T05:44:11.622 DEBUG:boto:Token: None 2022-09-03T05:44:11.623 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:11 GMT /dsgvcy-34/?cors 2022-09-03T05:44:11.623 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pPy7vjJY9YvkG/mIBSMYiCcc9AQ= 2022-09-03T05:44:11.625 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pPy7vjJY9YvkG/mIBSMYiCcc9AQ=', 'Content-Length': '0'} 2022-09-03T05:44:11.627 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000b711ebc696d783d9-006312e9ab-10ea-test-zone2'), ('Content-Type', 'application/xml'), ('Content-Length', '266'), ('Date', 'Sat, 03 Sep 2022 05:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:11.670 DEBUG:boto:b'DELETEhttps://www.example.com*3000' 2022-09-03T05:44:11.671 DEBUG:boto:path=/dsgvcy-35/ 2022-09-03T05:44:11.671 DEBUG:boto:auth_path=/dsgvcy-35/ 2022-09-03T05:44:11.672 DEBUG:boto:path=/dsgvcy-35/?cors 2022-09-03T05:44:11.672 DEBUG:boto:auth_path=/dsgvcy-35/?cors 2022-09-03T05:44:11.673 DEBUG:boto:Method: PUT 2022-09-03T05:44:11.674 DEBUG:boto:Path: /dsgvcy-35/?cors 2022-09-03T05:44:11.675 DEBUG:boto:Data: DELETEhttps://www.example.com*3000 2022-09-03T05:44:11.675 DEBUG:boto:Headers: {'Content-MD5': 'c5gEzAIzM7zcw8WVE+AbrQ==', 'Content-Type': 'text/xml'} 2022-09-03T05:44:11.676 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:44:11.676 DEBUG:boto:Port: 8001 2022-09-03T05:44:11.676 DEBUG:boto:Params: {} 2022-09-03T05:44:11.677 DEBUG:boto:Token: None 2022-09-03T05:44:11.678 DEBUG:boto:StringToSign: PUT c5gEzAIzM7zcw8WVE+AbrQ== text/xml Sat, 03 Sep 2022 05:44:11 GMT /dsgvcy-35/?cors 2022-09-03T05:44:11.678 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:34pShmtg+0xp36SNOj+hzz3J3ow= 2022-09-03T05:44:11.679 DEBUG:boto:Final headers: {'Content-MD5': 'c5gEzAIzM7zcw8WVE+AbrQ==', 'Content-Type': 'text/xml', 'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:34pShmtg+0xp36SNOj+hzz3J3ow=', 'Content-Length': '218'} 2022-09-03T05:44:12.686 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000009d5b83a050d19ad8-006312e9ab-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:12.687 DEBUG:boto:path=/dsgvcy-35/ 2022-09-03T05:44:12.688 DEBUG:boto:auth_path=/dsgvcy-35/ 2022-09-03T05:44:12.688 DEBUG:boto:path=/dsgvcy-35/?cors 2022-09-03T05:44:12.689 DEBUG:boto:auth_path=/dsgvcy-35/?cors 2022-09-03T05:44:12.690 DEBUG:boto:Method: GET 2022-09-03T05:44:12.690 DEBUG:boto:Path: /dsgvcy-35/?cors 2022-09-03T05:44:12.691 DEBUG:boto:Data: 2022-09-03T05:44:12.691 DEBUG:boto:Headers: {} 2022-09-03T05:44:12.692 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:44:12.692 DEBUG:boto:Port: 8001 2022-09-03T05:44:12.693 DEBUG:boto:Params: {} 2022-09-03T05:44:12.693 DEBUG:boto:Token: None 2022-09-03T05:44:12.694 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:44:12 GMT /dsgvcy-35/?cors 2022-09-03T05:44:12.694 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a9uKksQcFz1Mse83EEM0ujT30ZU= 2022-09-03T05:44:12.695 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a9uKksQcFz1Mse83EEM0ujT30ZU=', 'Content-Length': '0'} 2022-09-03T05:44:12.699 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000ffba559ec5b0f568-006312e9ac-1103-test-zone3'), ('Content-Type', 'application/xml'), ('Content-Length', '266'), ('Date', 'Sat, 03 Sep 2022 05:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:12.750 DEBUG:boto:b'DELETEhttps://www.example.com*3000' 2022-09-03T05:44:12.751 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_cors ... ok 2022-09-03T05:44:12.752 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-36 2022-09-03T05:44:12.753 DEBUG:boto:path=/dsgvcy-36/ 2022-09-03T05:44:12.753 DEBUG:boto:auth_path=/dsgvcy-36/ 2022-09-03T05:44:12.754 DEBUG:boto:Method: PUT 2022-09-03T05:44:12.754 DEBUG:boto:Path: /dsgvcy-36/ 2022-09-03T05:44:12.755 DEBUG:boto:Data: 2022-09-03T05:44:12.755 DEBUG:boto:Headers: {} 2022-09-03T05:44:12.756 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:12.756 DEBUG:boto:Port: 8000 2022-09-03T05:44:12.757 DEBUG:boto:Params: {} 2022-09-03T05:44:12.758 DEBUG:boto:Token: None 2022-09-03T05:44:12.759 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:44:12 GMT /dsgvcy-36/ 2022-09-03T05:44:12.760 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rVhF7zFerOo8FtN5PNKDR4ba08I= 2022-09-03T05:44:12.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rVhF7zFerOo8FtN5PNKDR4ba08I=', 'Content-Length': '0'} 2022-09-03T05:44:12.786 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000c3bbcf1faf4a56dd-006312e9ac-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:12.787 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=dsgvcy-37 2022-09-03T05:44:12.788 DEBUG:boto:path=/dsgvcy-37/ 2022-09-03T05:44:12.789 DEBUG:boto:auth_path=/dsgvcy-37/ 2022-09-03T05:44:12.790 DEBUG:boto:Method: PUT 2022-09-03T05:44:12.790 DEBUG:boto:Path: /dsgvcy-37/ 2022-09-03T05:44:12.791 DEBUG:boto:Data: 2022-09-03T05:44:12.791 DEBUG:boto:Headers: {} 2022-09-03T05:44:12.792 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:44:12.792 DEBUG:boto:Port: 8000 2022-09-03T05:44:12.793 DEBUG:boto:Params: {} 2022-09-03T05:44:12.794 DEBUG:boto:Token: None 2022-09-03T05:44:12.794 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:44:12 GMT /dsgvcy-37/ 2022-09-03T05:44:12.795 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:g6EZwkoZe3MkBuKp24h49IOcn4U= 2022-09-03T05:44:12.795 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:g6EZwkoZe3MkBuKp24h49IOcn4U=', 'Content-Length': '0'} 2022-09-03T05:44:12.822 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000a66036245386ca40-006312e9ac-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:12.824 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=dsgvcy-38 2022-09-03T05:44:12.825 DEBUG:boto:path=/dsgvcy-38/ 2022-09-03T05:44:12.826 DEBUG:boto:auth_path=/dsgvcy-38/ 2022-09-03T05:44:12.827 DEBUG:boto:Method: PUT 2022-09-03T05:44:12.828 DEBUG:boto:Path: /dsgvcy-38/ 2022-09-03T05:44:12.829 DEBUG:boto:Data: 2022-09-03T05:44:12.830 DEBUG:boto:Headers: {} 2022-09-03T05:44:12.831 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:44:12.831 DEBUG:boto:Port: 8001 2022-09-03T05:44:12.832 DEBUG:boto:Params: {} 2022-09-03T05:44:12.833 DEBUG:boto:Token: None 2022-09-03T05:44:12.834 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:44:12 GMT /dsgvcy-38/ 2022-09-03T05:44:12.835 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4kjxUlauepAxhSkAzaGIU1fYZhw= 2022-09-03T05:44:12.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4kjxUlauepAxhSkAzaGIU1fYZhw=', 'Content-Length': '0'} 2022-09-03T05:44:12.872 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000054f833b29234a6d9-006312e9ac-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:12.873 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:44:12.873 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:44:12.874 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:44:12.925 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:44:12.925 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:44:12.972 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=41ecd2a5-afc0-4eae-9349-7dfe39baa1cb 2022-09-03T05:44:12.990 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:44:12.990 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:44:12.991 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:44:12.991 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:18:18.906852Z" 2022-09-03T05:44:12.991 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:44:12.992 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:44:12.992 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:44:12.992 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:44:12.992 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:44:12.993 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:44:12.993 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662183852.856692_297.1", 2022-09-03T05:44:12.993 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:44:12.856692Z" 2022-09-03T05:44:12.993 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:44:12.994 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:44:12.995 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:44:12.995 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:20:30.928498Z" 2022-09-03T05:44:12.995 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:44:12.995 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:44:12.997 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183852.856692_297.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:44:12.998 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:44:12.999 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:44:13.000 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:44:13.000 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:44:13.051 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:44:13.052 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:44:13.112 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:44:13.111+0000 7fbea3ab4600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:44:13.115 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:44:13.112+0000 7fbea3ab4600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:44:13.146 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:44:13.147 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:44:13.148 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:44:13.148 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:44:13.149 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:44:13.149 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:44:13.150 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:13.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:13.151 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:44:13.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:13.151 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:44:13.152 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:13.152 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:13.153 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:44:13.153 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:13.154 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:44:13.154 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:13.154 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:44:13.155 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:13.155 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:13.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:13.155 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:13.156 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:44:13.156 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:13.156 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:13.157 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:44:13.157 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:13.157 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:44:13.158 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:13.158 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:44:13.158 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:13.158 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:13.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:13.159 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:13.159 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:44:13.159 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:13.160 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:13.160 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183850.172332_254.1", 2022-09-03T05:44:13.160 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:13.160 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:44:13.161 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:13.161 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.172332Z", 2022-09-03T05:44:13.161 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:13.161 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:13.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:13.162 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:13.162 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:44:13.163 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:13.163 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:13.163 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:44:13.164 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:13.164 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:44:13.164 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:13.164 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:44:13.165 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:13.165 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:13.165 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:13.166 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:44:13.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:13.167 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:44:13.167 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:44:13.167 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:44:13.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:13.168 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:44:13Z" 2022-09-03T05:44:13.168 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:44:13.169 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183850.172332_254.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:10.172332Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:44:13Z" } 2022-09-03T05:44:13.170 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183850.172332_254.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:10.172332Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:44:13.170 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183852.856692_297.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:44:13.171 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183850.172332_254.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:44:13.171 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662183852.856692_297.1 target=1_1662183850.172332_254.1 2022-09-03T05:44:18.173 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:44:18.174 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:44:18.175 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:44:18.233 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:44:18.234 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:44:18.268 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:44:18.268+0000 7f6623fbf600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:44:18.270 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:44:18.269+0000 7f6623fbf600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:44:18.283 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:44:18.284 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:44:18.284 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:44:18.284 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:44:18.285 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:44:18.285 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:44:18.286 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:18.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:18.286 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:44:18.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:18.288 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:44:18.288 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:18.289 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:18.289 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:44:18.289 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:18.289 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:44:18.290 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:18.290 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:44:18.290 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:18.291 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:18.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:18.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:18.292 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:44:18.292 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:18.293 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:18.293 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:44:18.294 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:18.294 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:44:18.295 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:18.295 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:44:18.295 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:18.295 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:18.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:18.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:18.296 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:44:18.297 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:18.297 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:18.297 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183850.220311_258.1", 2022-09-03T05:44:18.298 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:18.298 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:44:18.299 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:18.299 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.220311Z", 2022-09-03T05:44:18.299 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:18.299 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:18.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:18.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:18.300 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:44:18.300 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:18.301 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:18.301 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:44:18.301 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:18.301 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:44:18.302 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:18.302 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:44:18.302 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:18.302 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:18.303 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:18.303 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:44:18.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:18.304 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:44:18.304 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:44:18.304 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:44:18.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:18.305 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:44:18Z" 2022-09-03T05:44:18.305 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:44:18.307 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183850.220311_258.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:10.220311Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:44:18Z" } 2022-09-03T05:44:18.308 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183850.220311_258.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:10.220311Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:44:18.309 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183852.856692_297.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:44:18.310 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183850.220311_258.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:44:18.311 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662183852.856692_297.1 target=1_1662183850.220311_258.1 2022-09-03T05:44:23.313 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:44:23.314 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:44:23.315 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:44:23.372 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:44:23.372 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:44:23.404 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:44:23.403+0000 7f54e60cb600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:44:23.405 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:44:23.404+0000 7f54e60cb600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:44:23.420 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:44:23.421 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:44:23.421 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:44:23.422 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:44:23.422 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:44:23.423 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:44:23.423 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:23.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:23.424 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:44:23.425 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:23.425 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:44:23.425 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:23.426 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:23.426 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:44:23.427 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:23.427 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:44:23.428 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:23.428 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:44:23.429 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:23.429 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:23.429 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:23.430 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:23.431 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:44:23.431 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:23.431 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:23.432 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:44:23.433 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:23.433 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:44:23.434 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:23.434 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:44:23.435 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:23.435 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:23.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:23.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:23.437 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:44:23.437 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:23.438 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:23.438 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183852.856692_297.1", 2022-09-03T05:44:23.439 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:23.439 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:44:23.440 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:23.440 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.856692Z", 2022-09-03T05:44:23.441 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:23.441 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:23.441 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:23.442 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:23.442 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:44:23.443 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:23.443 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:23.444 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:44:23.444 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:23.445 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:44:23.445 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:23.446 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:44:23.446 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:23.447 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:23.447 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:23.448 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:44:23.448 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:23.449 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:44:23.449 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:44:23.450 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:44:23.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:23.451 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:44:23Z" 2022-09-03T05:44:23.451 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:44:23.452 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183852.856692_297.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:12.856692Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:44:23Z" } 2022-09-03T05:44:23.453 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183852.856692_297.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:12.856692Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:44:23.455 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183852.856692_297.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:44:23.456 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183852.856692_297.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:44:23.457 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:44:23.457 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:44:23.458 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:44:23.459 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:44:23.461 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:44:23.517 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:44:23.518 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:44:23.558 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:44:23.558+0000 7f4d0089d600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:44:23.559 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:44:23.559+0000 7f4d0089d600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:44:23.572 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:44:23.572 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:44:23.573 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:44:23.573 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:44:23.573 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:44:23.574 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:44:23.574 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:44:23.574 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:44:23.575 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:44:23.575 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:44:23.576 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:44:23.576 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:44:23.576 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:44:23.577 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:44:23.577 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:44:23.578 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:44:23.578 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:44:23.579 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:44:23.579 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:44:23.580 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:44:23.581 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:44:23.581 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:44:23.581 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:44:23.582 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:44:23.582 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:44:23.583 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:44:23.583 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:44:23.584 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:44:23.584 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:44:23.584 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:44:23.585 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:44:23.585 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:44:23.585 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:44:23.589 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:44:23.589 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:44:23.589 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:44:23.591 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:44:23.591 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662183852.856692_297.1", 2022-09-03T05:44:23.591 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:44:23.591 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:44:23.592 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:44:23.592 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.856692Z", 2022-09-03T05:44:23.592 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:44:23.593 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:44:23.593 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:44:23.593 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:44:23.594 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:44:23.594 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:44:23.594 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:44:23.595 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:44:23.595 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:44:23.595 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:44:23.596 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:44:23.597 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:44:23.597 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:44:23.597 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:44:23.598 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:44:23.598 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:44:23.598 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:44:23.598 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:44:23.599 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:44:23.599 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:44:23.600 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:44:23.600 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:44:23Z" 2022-09-03T05:44:23.600 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:44:23.601 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183852.856692_297.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:12.856692Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:44:23Z" } 2022-09-03T05:44:23.602 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183852.856692_297.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:12.856692Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:44:23.603 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183852.856692_297.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:44:23.603 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183852.856692_297.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:44:23.604 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:44:23.604 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:44:23.605 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:44:23.605 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:44:23.606 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:44:23.659 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:44:23.660 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:44:23.685 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:44:23.685+0000 7fafc7393600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:44:23.686 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:44:23.686+0000 7fafc7393600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:44:23.697 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:44:23.698 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:44:23.698 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:44:23.699 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:44:23.699 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:44:23.699 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:44:23.700 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:23.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:23.700 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:44:23.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:23.701 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:44:23.701 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:23.703 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:23.703 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:44:23.704 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:23.704 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:44:23.704 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:23.705 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:44:23.705 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:23.705 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:23.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:23.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:23.706 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:44:23.706 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:23.706 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:23.707 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:44:23.707 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:23.708 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:44:23.708 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:23.708 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:44:23.708 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:23.709 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:23.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:23.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:23.710 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:44:23.710 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:23.711 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:23.711 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183852.856692_297.1", 2022-09-03T05:44:23.711 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:23.711 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:44:23.712 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:23.712 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.856692Z", 2022-09-03T05:44:23.712 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:23.713 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:23.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:23.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:23.713 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:44:23.714 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:23.714 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:23.715 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:44:23.715 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:23.716 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:44:23.716 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:23.716 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:44:23.717 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:23.717 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:23.718 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:23.718 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:44:23.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:23.719 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:44:23.719 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:44:23.720 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:44:23.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:23.720 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:44:23Z" 2022-09-03T05:44:23.721 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:44:23.723 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183852.856692_297.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:12.856692Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:44:23Z" } 2022-09-03T05:44:23.724 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183852.856692_297.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:12.856692Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:44:23.725 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183852.856692_297.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:44:23.726 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183852.856692_297.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:44:23.727 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:44:23.727 DEBUG:boto:path=/dsgvcy-36/ 2022-09-03T05:44:23.729 DEBUG:boto:auth_path=/dsgvcy-36/ 2022-09-03T05:44:23.730 DEBUG:boto:Method: HEAD 2022-09-03T05:44:23.730 DEBUG:boto:Path: /dsgvcy-36/ 2022-09-03T05:44:23.731 DEBUG:boto:Data: 2022-09-03T05:44:23.732 DEBUG:boto:Headers: {} 2022-09-03T05:44:23.733 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:23.733 DEBUG:boto:Port: 8000 2022-09-03T05:44:23.734 DEBUG:boto:Params: {} 2022-09-03T05:44:23.735 DEBUG:boto:Token: None 2022-09-03T05:44:23.735 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:44:23 GMT /dsgvcy-36/ 2022-09-03T05:44:23.736 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9+KHY5SwW7dEBdcEjaiNV8AGJ6w= 2022-09-03T05:44:23.737 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9+KHY5SwW7dEBdcEjaiNV8AGJ6w=', 'Content-Length': '0'} 2022-09-03T05:44:23.740 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000024d06ef1cd064f5d-006312e9b7-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:23.740 DEBUG:boto:path=/dsgvcy-36/foo 2022-09-03T05:44:23.741 DEBUG:boto:auth_path=/dsgvcy-36/foo 2022-09-03T05:44:23.741 DEBUG:boto:Method: PUT 2022-09-03T05:44:23.742 DEBUG:boto:Path: /dsgvcy-36/foo 2022-09-03T05:44:23.743 DEBUG:boto:Data: 2022-09-03T05:44:23.743 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'N7UdGUp1E+RbVvZSTy1R8g==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T05:44:23.744 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:23.745 DEBUG:boto:Port: 8000 2022-09-03T05:44:23.745 DEBUG:boto:Params: {} 2022-09-03T05:44:23.747 DEBUG:boto:Token: None 2022-09-03T05:44:23.747 DEBUG:boto:StringToSign: PUT N7UdGUp1E+RbVvZSTy1R8g== application/octet-stream Sat, 03 Sep 2022 05:44:23 GMT /dsgvcy-36/foo 2022-09-03T05:44:23.748 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MRQoWENDZUtIoOLU1GcIbzFH9L4= 2022-09-03T05:44:23.750 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'N7UdGUp1E+RbVvZSTy1R8g==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:44:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MRQoWENDZUtIoOLU1GcIbzFH9L4='} 2022-09-03T05:44:23.760 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"37b51d194a7513e45b56f6524f2d51f2"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000087ced53398e5523b-006312e9b7-10ff-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:44:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:23.762 DEBUG:boto:path=/dsgvcy-36/ 2022-09-03T05:44:23.762 DEBUG:boto:auth_path=/dsgvcy-36/ 2022-09-03T05:44:23.763 DEBUG:boto:Method: DELETE 2022-09-03T05:44:23.764 DEBUG:boto:Path: /dsgvcy-36/ 2022-09-03T05:44:23.765 DEBUG:boto:Data: 2022-09-03T05:44:23.766 DEBUG:boto:Headers: {} 2022-09-03T05:44:23.767 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:23.768 DEBUG:boto:Port: 8000 2022-09-03T05:44:23.768 DEBUG:boto:Params: {} 2022-09-03T05:44:23.769 DEBUG:boto:Token: None 2022-09-03T05:44:23.771 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:44:23 GMT /dsgvcy-36/ 2022-09-03T05:44:23.771 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3NiqfWP53nbNlTh6DQ0Ge8T18ZY= 2022-09-03T05:44:23.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3NiqfWP53nbNlTh6DQ0Ge8T18ZY=', 'Content-Length': '0'} 2022-09-03T05:44:23.782 DEBUG:boto:Response headers: [('Content-Length', '234'), ('x-amz-request-id', 'tx0000088013f25f3af0a08-006312e9b7-10ff-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:44:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:23.831 DEBUG:boto:path=/dsgvcy-37/ 2022-09-03T05:44:23.832 DEBUG:boto:auth_path=/dsgvcy-37/ 2022-09-03T05:44:23.833 DEBUG:boto:Method: HEAD 2022-09-03T05:44:23.834 DEBUG:boto:Path: /dsgvcy-37/ 2022-09-03T05:44:23.835 DEBUG:boto:Data: 2022-09-03T05:44:23.836 DEBUG:boto:Headers: {} 2022-09-03T05:44:23.837 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:44:23.838 DEBUG:boto:Port: 8000 2022-09-03T05:44:23.839 DEBUG:boto:Params: {} 2022-09-03T05:44:23.840 DEBUG:boto:Token: None 2022-09-03T05:44:23.841 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:44:23 GMT /dsgvcy-37/ 2022-09-03T05:44:23.842 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DSLMc/GAyfDQXJE53rytM2aJBj4= 2022-09-03T05:44:23.843 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DSLMc/GAyfDQXJE53rytM2aJBj4=', 'Content-Length': '0'} 2022-09-03T05:44:23.846 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000089c06b15486d0be8-006312e9b7-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:23.847 DEBUG:boto:path=/dsgvcy-37/foo 2022-09-03T05:44:23.848 DEBUG:boto:auth_path=/dsgvcy-37/foo 2022-09-03T05:44:23.848 DEBUG:boto:Method: PUT 2022-09-03T05:44:23.849 DEBUG:boto:Path: /dsgvcy-37/foo 2022-09-03T05:44:23.849 DEBUG:boto:Data: 2022-09-03T05:44:23.850 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'N7UdGUp1E+RbVvZSTy1R8g==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T05:44:23.851 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:44:23.851 DEBUG:boto:Port: 8000 2022-09-03T05:44:23.852 DEBUG:boto:Params: {} 2022-09-03T05:44:23.852 DEBUG:boto:Token: None 2022-09-03T05:44:23.853 DEBUG:boto:StringToSign: PUT N7UdGUp1E+RbVvZSTy1R8g== application/octet-stream Sat, 03 Sep 2022 05:44:23 GMT /dsgvcy-37/foo 2022-09-03T05:44:23.853 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r+7Z5SiZf3PN2lfTiPIqJ2bj/ok= 2022-09-03T05:44:23.854 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'N7UdGUp1E+RbVvZSTy1R8g==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:44:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r+7Z5SiZf3PN2lfTiPIqJ2bj/ok='} 2022-09-03T05:44:23.864 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"37b51d194a7513e45b56f6524f2d51f2"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003ba14aa445d28183-006312e9b7-10ea-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:44:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:23.865 DEBUG:boto:path=/dsgvcy-37/ 2022-09-03T05:44:23.866 DEBUG:boto:auth_path=/dsgvcy-37/ 2022-09-03T05:44:23.866 DEBUG:boto:Method: DELETE 2022-09-03T05:44:23.867 DEBUG:boto:Path: /dsgvcy-37/ 2022-09-03T05:44:23.867 DEBUG:boto:Data: 2022-09-03T05:44:23.868 DEBUG:boto:Headers: {} 2022-09-03T05:44:23.868 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:44:23.869 DEBUG:boto:Port: 8000 2022-09-03T05:44:23.869 DEBUG:boto:Params: {} 2022-09-03T05:44:23.870 DEBUG:boto:Token: None 2022-09-03T05:44:23.870 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:44:23 GMT /dsgvcy-37/ 2022-09-03T05:44:23.871 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c1ZxyAMIt9ZJasCT8jG4kuErMDk= 2022-09-03T05:44:23.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c1ZxyAMIt9ZJasCT8jG4kuErMDk=', 'Content-Length': '0'} 2022-09-03T05:44:23.879 DEBUG:boto:Response headers: [('Content-Length', '234'), ('x-amz-request-id', 'tx00000a6be5fc11e0a2903-006312e9b7-10ea-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:44:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:23.880 DEBUG:boto:path=/dsgvcy-38/ 2022-09-03T05:44:23.881 DEBUG:boto:auth_path=/dsgvcy-38/ 2022-09-03T05:44:23.882 DEBUG:boto:Method: HEAD 2022-09-03T05:44:23.883 DEBUG:boto:Path: /dsgvcy-38/ 2022-09-03T05:44:23.884 DEBUG:boto:Data: 2022-09-03T05:44:23.885 DEBUG:boto:Headers: {} 2022-09-03T05:44:23.886 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:44:23.887 DEBUG:boto:Port: 8001 2022-09-03T05:44:23.888 DEBUG:boto:Params: {} 2022-09-03T05:44:23.889 DEBUG:boto:Token: None 2022-09-03T05:44:23.890 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:44:23 GMT /dsgvcy-38/ 2022-09-03T05:44:23.891 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cwTEgAljAez3AzZtfIwTrXI8WcY= 2022-09-03T05:44:23.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cwTEgAljAez3AzZtfIwTrXI8WcY=', 'Content-Length': '0'} 2022-09-03T05:44:23.895 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000df72a1903fd4633-006312e9b7-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:23.896 DEBUG:boto:path=/dsgvcy-38/foo 2022-09-03T05:44:23.897 DEBUG:boto:auth_path=/dsgvcy-38/foo 2022-09-03T05:44:23.897 DEBUG:boto:Method: PUT 2022-09-03T05:44:23.898 DEBUG:boto:Path: /dsgvcy-38/foo 2022-09-03T05:44:23.899 DEBUG:boto:Data: 2022-09-03T05:44:23.899 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'N7UdGUp1E+RbVvZSTy1R8g==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T05:44:23.900 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:44:23.900 DEBUG:boto:Port: 8001 2022-09-03T05:44:23.901 DEBUG:boto:Params: {} 2022-09-03T05:44:23.901 DEBUG:boto:Token: None 2022-09-03T05:44:23.902 DEBUG:boto:StringToSign: PUT N7UdGUp1E+RbVvZSTy1R8g== application/octet-stream Sat, 03 Sep 2022 05:44:23 GMT /dsgvcy-38/foo 2022-09-03T05:44:23.904 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TrFsNcxFBdKu9e3EfPpkr+W8wUQ= 2022-09-03T05:44:23.905 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'N7UdGUp1E+RbVvZSTy1R8g==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:44:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TrFsNcxFBdKu9e3EfPpkr+W8wUQ='} 2022-09-03T05:44:23.921 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"37b51d194a7513e45b56f6524f2d51f2"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000077fb74badb6d325e-006312e9b7-1103-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:44:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:23.922 DEBUG:boto:path=/dsgvcy-38/ 2022-09-03T05:44:23.924 DEBUG:boto:auth_path=/dsgvcy-38/ 2022-09-03T05:44:23.925 DEBUG:boto:Method: DELETE 2022-09-03T05:44:23.926 DEBUG:boto:Path: /dsgvcy-38/ 2022-09-03T05:44:23.927 DEBUG:boto:Data: 2022-09-03T05:44:23.928 DEBUG:boto:Headers: {} 2022-09-03T05:44:23.929 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:44:23.930 DEBUG:boto:Port: 8001 2022-09-03T05:44:23.930 DEBUG:boto:Params: {} 2022-09-03T05:44:23.931 DEBUG:boto:Token: None 2022-09-03T05:44:23.932 DEBUG:boto:StringToSign: DELETE Sat, 03 Sep 2022 05:44:23 GMT /dsgvcy-38/ 2022-09-03T05:44:23.932 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wdDJgusJih98L+azGv2bfqiiErA= 2022-09-03T05:44:23.933 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wdDJgusJih98L+azGv2bfqiiErA=', 'Content-Length': '0'} 2022-09-03T05:44:23.940 DEBUG:boto:Response headers: [('Content-Length', '234'), ('x-amz-request-id', 'tx00000eb7fb6746d4c1edf-006312e9b7-1103-test-zone3'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:44:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:23.942 DEBUG:boto:path=/dsgvcy-36/ 2022-09-03T05:44:23.942 DEBUG:boto:auth_path=/dsgvcy-36/ 2022-09-03T05:44:23.943 DEBUG:boto:Method: HEAD 2022-09-03T05:44:23.943 DEBUG:boto:Path: /dsgvcy-36/ 2022-09-03T05:44:23.944 DEBUG:boto:Data: 2022-09-03T05:44:23.945 DEBUG:boto:Headers: {} 2022-09-03T05:44:23.945 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:23.946 DEBUG:boto:Port: 8000 2022-09-03T05:44:23.947 DEBUG:boto:Params: {} 2022-09-03T05:44:23.948 DEBUG:boto:Token: None 2022-09-03T05:44:23.949 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:44:23 GMT /dsgvcy-36/ 2022-09-03T05:44:23.950 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9+KHY5SwW7dEBdcEjaiNV8AGJ6w= 2022-09-03T05:44:23.951 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9+KHY5SwW7dEBdcEjaiNV8AGJ6w=', 'Content-Length': '0'} 2022-09-03T05:44:23.954 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '3'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ff49f35ca2ecc7e1-006312e9b7-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:23.955 DEBUG:boto:path=/dsgvcy-37/ 2022-09-03T05:44:23.956 DEBUG:boto:auth_path=/dsgvcy-37/ 2022-09-03T05:44:23.957 DEBUG:boto:Method: HEAD 2022-09-03T05:44:23.957 DEBUG:boto:Path: /dsgvcy-37/ 2022-09-03T05:44:23.958 DEBUG:boto:Data: 2022-09-03T05:44:23.958 DEBUG:boto:Headers: {} 2022-09-03T05:44:23.959 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:23.959 DEBUG:boto:Port: 8000 2022-09-03T05:44:23.960 DEBUG:boto:Params: {} 2022-09-03T05:44:23.960 DEBUG:boto:Token: None 2022-09-03T05:44:23.961 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:44:23 GMT /dsgvcy-37/ 2022-09-03T05:44:23.962 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DSLMc/GAyfDQXJE53rytM2aJBj4= 2022-09-03T05:44:23.963 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DSLMc/GAyfDQXJE53rytM2aJBj4=', 'Content-Length': '0'} 2022-09-03T05:44:23.966 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002d70b9bcd1a2198a-006312e9b7-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:23.967 DEBUG:boto:path=/dsgvcy-38/ 2022-09-03T05:44:23.968 DEBUG:boto:auth_path=/dsgvcy-38/ 2022-09-03T05:44:23.969 DEBUG:boto:Method: HEAD 2022-09-03T05:44:23.970 DEBUG:boto:Path: /dsgvcy-38/ 2022-09-03T05:44:23.971 DEBUG:boto:Data: 2022-09-03T05:44:23.971 DEBUG:boto:Headers: {} 2022-09-03T05:44:23.972 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:23.972 DEBUG:boto:Port: 8000 2022-09-03T05:44:23.973 DEBUG:boto:Params: {} 2022-09-03T05:44:23.973 DEBUG:boto:Token: None 2022-09-03T05:44:23.974 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:44:23 GMT /dsgvcy-38/ 2022-09-03T05:44:23.975 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cwTEgAljAez3AzZtfIwTrXI8WcY= 2022-09-03T05:44:23.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cwTEgAljAez3AzZtfIwTrXI8WcY=', 'Content-Length': '0'} 2022-09-03T05:44:23.979 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fb51ea109077ee31-006312e9b7-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:23.980 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_delete_notempty ... ok 2022-09-03T05:44:23.982 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-39 2022-09-03T05:44:23.983 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:44:23.984 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:44:23.985 DEBUG:boto:Method: PUT 2022-09-03T05:44:23.986 DEBUG:boto:Path: /dsgvcy-39/ 2022-09-03T05:44:23.987 DEBUG:boto:Data: 2022-09-03T05:44:23.988 DEBUG:boto:Headers: {} 2022-09-03T05:44:23.989 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:44:23.991 DEBUG:boto:Port: 8000 2022-09-03T05:44:23.992 DEBUG:boto:Params: {} 2022-09-03T05:44:23.993 DEBUG:boto:Token: None 2022-09-03T05:44:23.994 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:44:23 GMT /dsgvcy-39/ 2022-09-03T05:44:23.995 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L3o+PvTRPzggh+XMKGN+9C4UdaY= 2022-09-03T05:44:23.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L3o+PvTRPzggh+XMKGN+9C4UdaY=', 'Content-Length': '0'} 2022-09-03T05:44:24.011 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000658993ae40586ab0-006312e9b7-10ff-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:24.013 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=dsgvcy-40 2022-09-03T05:44:24.014 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:44:24.015 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:44:24.016 DEBUG:boto:Method: PUT 2022-09-03T05:44:24.017 DEBUG:boto:Path: /dsgvcy-40/ 2022-09-03T05:44:24.018 DEBUG:boto:Data: 2022-09-03T05:44:24.019 DEBUG:boto:Headers: {} 2022-09-03T05:44:24.020 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:44:24.021 DEBUG:boto:Port: 8000 2022-09-03T05:44:24.022 DEBUG:boto:Params: {} 2022-09-03T05:44:24.023 DEBUG:boto:Token: None 2022-09-03T05:44:24.025 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:44:24 GMT /dsgvcy-40/ 2022-09-03T05:44:24.026 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6ONT9ls2WTnB99C+b7dHPwgyMhg= 2022-09-03T05:44:24.027 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6ONT9ls2WTnB99C+b7dHPwgyMhg=', 'Content-Length': '0'} 2022-09-03T05:44:24.067 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000006f8208c6c6161925-006312e9b8-10ea-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:24.069 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=dsgvcy-41 2022-09-03T05:44:24.070 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:44:24.071 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:44:24.072 DEBUG:boto:Method: PUT 2022-09-03T05:44:24.073 DEBUG:boto:Path: /dsgvcy-41/ 2022-09-03T05:44:24.073 DEBUG:boto:Data: 2022-09-03T05:44:24.074 DEBUG:boto:Headers: {} 2022-09-03T05:44:24.075 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:44:24.077 DEBUG:boto:Port: 8001 2022-09-03T05:44:24.078 DEBUG:boto:Params: {} 2022-09-03T05:44:24.079 DEBUG:boto:Token: None 2022-09-03T05:44:24.080 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:44:24 GMT /dsgvcy-41/ 2022-09-03T05:44:24.081 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0W2G4omtAN7curzftvny2cqCLrE= 2022-09-03T05:44:24.082 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:44:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0W2G4omtAN7curzftvny2cqCLrE=', 'Content-Length': '0'} 2022-09-03T05:44:24.132 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000060d487f0bf010895-006312e9b8-1103-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:44:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:44:24.133 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:44:24.134 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:44:24.136 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:44:24.185 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:44:24.185 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:44:24.212 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=41ecd2a5-afc0-4eae-9349-7dfe39baa1cb 2022-09-03T05:44:24.228 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:44:24.229 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:44:24.229 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:44:24.230 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:18:18.906852Z" 2022-09-03T05:44:24.230 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:44:24.232 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:44:24.232 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:44:24.232 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:44:24.233 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:44:24.233 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:44:24.233 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662183864.119538_309.1", 2022-09-03T05:44:24.233 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:44:24.119538Z" 2022-09-03T05:44:24.234 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:44:24.234 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:44:24.235 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:44:24.235 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:20:30.928498Z" 2022-09-03T05:44:24.235 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:44:24.235 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:44:24.236 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183864.119538_309.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:44:24.237 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:44:24.237 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:44:24.238 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:44:24.238 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:44:24.290 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:44:24.291 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:44:24.327 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:44:24.323+0000 7f57eb05a600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:44:24.328 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:44:24.325+0000 7f57eb05a600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:44:24.346 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:44:24.346 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:44:24.347 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:44:24.347 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:44:24.348 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:44:24.348 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:44:24.349 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:24.349 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:24.350 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:44:24.350 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:24.351 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:44:24.351 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:24.352 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:24.352 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:44:24.352 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:24.353 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:44:24.353 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:24.353 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:44:24.354 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:24.354 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:24.354 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:24.354 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:24.355 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:44:24.355 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:24.355 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:24.355 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:44:24.356 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:24.356 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:44:24.356 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:24.356 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:44:24.357 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:24.357 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:24.357 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:24.357 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:24.358 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:44:24.358 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:24.358 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:24.362 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183864.099806_306.1", 2022-09-03T05:44:24.362 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:24.363 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:44:24.363 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:24.364 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.099806Z", 2022-09-03T05:44:24.364 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:24.365 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:24.365 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:24.366 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:24.366 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:44:24.367 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:24.367 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:24.368 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:44:24.368 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:24.368 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:44:24.369 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:24.369 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:44:24.369 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:24.369 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:24.370 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:24.370 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:44:24.370 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:24.371 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:44:24.371 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:44:24.371 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:44:24.371 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:24.372 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:44:24Z" 2022-09-03T05:44:24.372 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:44:24.373 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183864.099806_306.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:24.099806Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:44:24Z" } 2022-09-03T05:44:24.374 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183864.099806_306.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:24.099806Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:44:24.375 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183864.119538_309.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:44:24.375 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183864.099806_306.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:44:24.376 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662183864.119538_309.1 target=1_1662183864.099806_306.1 2022-09-03T05:44:29.377 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:44:29.379 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:44:29.380 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:44:29.438 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:44:29.439 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:44:29.480 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:44:29.478+0000 7ffa21e01600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:44:29.481 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:44:29.479+0000 7ffa21e01600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:44:29.498 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:44:29.498 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:44:29.499 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:44:29.499 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:44:29.501 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:44:29.501 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:44:29.502 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:29.502 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:29.503 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:44:29.503 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:29.504 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:44:29.504 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:29.505 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:29.505 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:44:29.506 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:29.506 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:44:29.507 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:29.507 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:44:29.508 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:29.508 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:29.509 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:29.509 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:29.510 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:44:29.510 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:29.511 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:29.512 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:44:29.512 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:29.513 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:44:29.513 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:29.514 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:44:29.514 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:29.515 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:29.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:29.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:29.516 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:44:29.516 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:29.517 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:29.517 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183864.119538_309.1", 2022-09-03T05:44:29.517 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:29.518 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:44:29.518 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:29.519 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.119538Z", 2022-09-03T05:44:29.519 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:29.519 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:29.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:29.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:29.520 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:44:29.520 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:29.520 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:29.521 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:44:29.521 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:29.522 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:44:29.522 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:29.522 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:44:29.523 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:29.523 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:29.523 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:29.523 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:44:29.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:29.524 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:44:29.524 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:44:29.525 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:44:29.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:29.526 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:44:29Z" 2022-09-03T05:44:29.526 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:44:29.527 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183864.119538_309.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:24.119538Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:44:29Z" } 2022-09-03T05:44:29.528 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183864.119538_309.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:24.119538Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:44:29.529 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183864.119538_309.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:44:29.529 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183864.119538_309.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:44:29.530 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:44:29.530 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:44:29.531 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:44:29.531 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:44:29.532 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:44:29.590 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:44:29.590 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:44:29.642 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:44:29.640+0000 7f53276ec600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:44:29.643 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:44:29.642+0000 7f53276ec600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:44:29.662 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:44:29.663 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:44:29.663 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:44:29.664 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:44:29.665 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:44:29.665 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:44:29.666 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:44:29.666 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:44:29.667 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:44:29.667 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:44:29.668 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:44:29.668 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:44:29.669 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:44:29.669 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:44:29.670 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:44:29.670 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:44:29.671 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:44:29.671 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:44:29.671 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:44:29.672 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:44:29.672 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:44:29.673 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:44:29.674 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:44:29.674 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:44:29.675 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:44:29.676 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:44:29.676 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:44:29.677 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:44:29.677 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:44:29.678 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:44:29.678 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:44:29.679 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:44:29.680 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:44:29.681 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:44:29.681 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:44:29.682 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:44:29.682 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:44:29.683 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662183864.119538_309.1", 2022-09-03T05:44:29.683 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:44:29.684 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:44:29.684 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:44:29.685 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.119538Z", 2022-09-03T05:44:29.685 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:44:29.686 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:44:29.687 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:44:29.687 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:44:29.687 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:44:29.688 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:44:29.688 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:44:29.688 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:44:29.689 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:44:29.689 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:44:29.690 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:44:29.690 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:44:29.690 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:44:29.691 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:44:29.691 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:44:29.691 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:44:29.691 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:44:29.692 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:44:29.692 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:44:29.692 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:44:29.692 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:44:29.693 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:44:29Z" 2022-09-03T05:44:29.693 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:44:29.695 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183864.119538_309.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:24.119538Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:44:29Z" } 2022-09-03T05:44:29.695 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183864.119538_309.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:24.119538Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:44:29.696 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183864.119538_309.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:44:29.697 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183864.119538_309.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:44:29.697 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:44:29.698 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:44:29.698 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:44:29.699 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:44:29.699 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:44:29.752 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:44:29.752 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:44:29.781 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:44:29.779+0000 7f4e18634600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:44:29.782 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:44:29.780+0000 7f4e18634600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:44:29.797 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:44:29.798 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:44:29.798 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:44:29.798 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:44:29.799 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:44:29.799 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:44:29.799 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:29.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:29.800 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:44:29.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:29.800 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:44:29.800 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:29.801 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:29.801 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:44:29.801 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:29.802 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:44:29.802 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:29.802 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:44:29.802 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:29.803 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:29.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:29.804 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:29.804 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:44:29.804 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:29.804 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:29.805 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:44:29.805 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:29.805 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:44:29.806 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:29.806 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:44:29.806 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:29.806 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:29.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:29.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:29.808 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:44:29.809 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:29.809 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:29.809 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183864.119538_309.1", 2022-09-03T05:44:29.810 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:29.810 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:44:29.810 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:29.810 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.119538Z", 2022-09-03T05:44:29.811 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:29.811 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:29.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:29.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:29.812 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:44:29.812 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:29.812 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:44:29.812 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:44:29.813 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:44:29.813 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:44:29.813 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:44:29.813 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:44:29.814 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:44:29.814 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:29.814 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:29.814 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:44:29.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:29.815 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:44:29.815 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:44:29.815 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:44:29.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:29.816 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:44:29Z" 2022-09-03T05:44:29.816 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:44:29.817 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", "realm_epoch": 2 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183864.119538_309.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:24.119538Z", "realm_epoch": 2 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:44:29Z" } 2022-09-03T05:44:29.817 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183864.119538_309.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:24.119538Z', 'realm_epoch': 2}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:44:29.818 DEBUG:rgw_multi.tests:log_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183864.119538_309.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:44:29.819 DEBUG:rgw_multi.tests:sync_status={0: '1_1662182298.906852_69.1', 1: '', 2: '1_1662183864.119538_309.1', 3: '1_1662182430.928498_66.1'} 2022-09-03T05:44:29.819 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:44:29.820 DEBUG:tasks.rgw.c1.client.1:waiting for process to exit 2022-09-03T05:44:29.820 INFO:teuthology.orchestra.run:waiting for 300 2022-09-03T05:44:29.822 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T05:44:29.819+0000 7f0d1341c700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term radosgw --rgw-frontends beast port=8001 -n client.1 --cluster c1 -k /etc/ceph/c1.client.1.keyring --log-file /var/log/ceph/rgw.c1.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.1.sock --foreground --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm (PID: 113063) UID: 0 2022-09-03T05:44:29.822 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T05:44:29.819+0000 7f0d17687600 -1 shutting down 2022-09-03T05:44:30.807 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c1.rgw.client.1 is failed for ~0s 2022-09-03T05:44:35.924 INFO:tasks.rgw.c1.client.1:Stopped 2022-09-03T05:44:35.925 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--master', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:44:35.926 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--master', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:44:35.928 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --master --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:44:35.985 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:44:35.986 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:44:36.010 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:44:36.009+0000 7f66df4d5600 0 NOTICE: overriding master zone: 91786ef1-cdb4-4061-8a3e-2a28cc497285 2022-09-03T05:44:36.011 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:44:36.009+0000 7f66df4d5600 0 failed reading obj info from .rgw.root:zone_info.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4: (2) No such file or directory 2022-09-03T05:44:36.011 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:44:36.009+0000 7f66df4d5600 0 WARNING: could not read zone params for zone id=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4 name=test-zone3 2022-09-03T05:44:36.011 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:44:36.009+0000 7f66df4d5600 0 failed reading obj info from .rgw.root:zone_info.91786ef1-cdb4-4061-8a3e-2a28cc497285: (2) No such file or directory 2022-09-03T05:44:36.012 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:44:36.009+0000 7f66df4d5600 0 WARNING: could not read zone params for zone id=91786ef1-cdb4-4061-8a3e-2a28cc497285 name=test-zone1 2022-09-03T05:44:36.019 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:44:36.019 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:44:36.020 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone2", 2022-09-03T05:44:36.020 INFO:teuthology.orchestra.run.smithi189.stdout: "domain_root": "test-zone2.rgw.meta:root", 2022-09-03T05:44:36.021 INFO:teuthology.orchestra.run.smithi189.stdout: "control_pool": "test-zone2.rgw.control", 2022-09-03T05:44:36.021 INFO:teuthology.orchestra.run.smithi189.stdout: "gc_pool": "test-zone2.rgw.log:gc", 2022-09-03T05:44:36.021 INFO:teuthology.orchestra.run.smithi189.stdout: "lc_pool": "test-zone2.rgw.log:lc", 2022-09-03T05:44:36.021 INFO:teuthology.orchestra.run.smithi189.stdout: "log_pool": "test-zone2.rgw.log", 2022-09-03T05:44:36.022 INFO:teuthology.orchestra.run.smithi189.stdout: "intent_log_pool": "test-zone2.rgw.log:intent", 2022-09-03T05:44:36.022 INFO:teuthology.orchestra.run.smithi189.stdout: "usage_log_pool": "test-zone2.rgw.log:usage", 2022-09-03T05:44:36.023 INFO:teuthology.orchestra.run.smithi189.stdout: "roles_pool": "test-zone2.rgw.meta:roles", 2022-09-03T05:44:36.023 INFO:teuthology.orchestra.run.smithi189.stdout: "reshard_pool": "test-zone2.rgw.log:reshard", 2022-09-03T05:44:36.024 INFO:teuthology.orchestra.run.smithi189.stdout: "user_keys_pool": "test-zone2.rgw.meta:users.keys", 2022-09-03T05:44:36.024 INFO:teuthology.orchestra.run.smithi189.stdout: "user_email_pool": "test-zone2.rgw.meta:users.email", 2022-09-03T05:44:36.024 INFO:teuthology.orchestra.run.smithi189.stdout: "user_swift_pool": "test-zone2.rgw.meta:users.swift", 2022-09-03T05:44:36.025 INFO:teuthology.orchestra.run.smithi189.stdout: "user_uid_pool": "test-zone2.rgw.meta:users.uid", 2022-09-03T05:44:36.025 INFO:teuthology.orchestra.run.smithi189.stdout: "otp_pool": "test-zone2.rgw.otp", 2022-09-03T05:44:36.025 INFO:teuthology.orchestra.run.smithi189.stdout: "system_key": { 2022-09-03T05:44:36.025 INFO:teuthology.orchestra.run.smithi189.stdout: "access_key": "TW0E8EULCERVYU84", 2022-09-03T05:44:36.026 INFO:teuthology.orchestra.run.smithi189.stdout: "secret_key": "EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP" 2022-09-03T05:44:36.026 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:36.026 INFO:teuthology.orchestra.run.smithi189.stdout: "placement_pools": [ 2022-09-03T05:44:36.026 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:36.027 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "default-placement", 2022-09-03T05:44:36.027 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:44:36.027 INFO:teuthology.orchestra.run.smithi189.stdout: "index_pool": "test-zone2.rgw.buckets.index", 2022-09-03T05:44:36.028 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_classes": { 2022-09-03T05:44:36.028 INFO:teuthology.orchestra.run.smithi189.stdout: "STANDARD": { 2022-09-03T05:44:36.029 INFO:teuthology.orchestra.run.smithi189.stdout: "data_pool": "test-zone2.rgw.buckets.data", 2022-09-03T05:44:36.029 INFO:teuthology.orchestra.run.smithi189.stdout: "compression_type": "random" 2022-09-03T05:44:36.030 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:36.031 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:36.031 INFO:teuthology.orchestra.run.smithi189.stdout: "data_extra_pool": "test-zone2.rgw.buckets.non-ec", 2022-09-03T05:44:36.032 INFO:teuthology.orchestra.run.smithi189.stdout: "index_type": 0 2022-09-03T05:44:36.037 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:36.037 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:36.037 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:44:36.038 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:44:36.038 INFO:teuthology.orchestra.run.smithi189.stdout: "notif_pool": "test-zone2.rgw.log:notif" 2022-09-03T05:44:36.038 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:44:36.040 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:44:36.041 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:44:36.041 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:44:36.132 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:44:36.133 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:44:36.175 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:44:36.174+0000 7f4a1036e600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:44:36.176 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:44:36.175+0000 7f4a1036e600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:44:36.194 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:44:36.197 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "eff2cb82-c479-4631-8568-fb1af4159e02", 2022-09-03T05:44:36.198 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 1, 2022-09-03T05:44:36.198 INFO:teuthology.orchestra.run.smithi189.stdout: "predecessor_uuid": "41ecd2a5-afc0-4eae-9349-7dfe39baa1cb", 2022-09-03T05:44:36.198 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": [ 2022-09-03T05:44:36.199 INFO:teuthology.orchestra.run.smithi189.stdout: "1_1662182298.906852_69.1", 2022-09-03T05:44:36.199 INFO:teuthology.orchestra.run.smithi189.stdout: "", 2022-09-03T05:44:36.199 INFO:teuthology.orchestra.run.smithi189.stdout: "1_1662183864.119538_309.1", 2022-09-03T05:44:36.200 INFO:teuthology.orchestra.run.smithi189.stdout: "1_1662182430.928498_66.1" 2022-09-03T05:44:36.200 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:44:36.200 INFO:teuthology.orchestra.run.smithi189.stdout: "period_map": { 2022-09-03T05:44:36.200 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "eff2cb82-c479-4631-8568-fb1af4159e02", 2022-09-03T05:44:36.201 INFO:teuthology.orchestra.run.smithi189.stdout: "zonegroups": [ 2022-09-03T05:44:36.201 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:36.201 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "test-zonegroup", 2022-09-03T05:44:36.202 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zonegroup", 2022-09-03T05:44:36.202 INFO:teuthology.orchestra.run.smithi189.stdout: "api_name": "", 2022-09-03T05:44:36.202 INFO:teuthology.orchestra.run.smithi189.stdout: "is_master": true, 2022-09-03T05:44:36.202 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:44:36.203 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:44:36.203 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:44:36.204 INFO:teuthology.orchestra.run.smithi189.stdout: "hostnames": [], 2022-09-03T05:44:36.204 INFO:teuthology.orchestra.run.smithi189.stdout: "hostnames_s3website": [], 2022-09-03T05:44:36.205 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zone": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:44:36.205 INFO:teuthology.orchestra.run.smithi189.stdout: "zones": [ 2022-09-03T05:44:36.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:36.206 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:44:36.206 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone3", 2022-09-03T05:44:36.206 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:44:36.206 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi181.front.sepia.ceph.com:8001" 2022-09-03T05:44:36.207 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:44:36.207 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:44:36.207 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:44:36.207 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:44:36.208 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:44:36.208 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:44:36.208 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": true, 2022-09-03T05:44:36.209 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:44:36.209 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:44:36.209 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:44:36.210 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:44:36.210 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:44:36.210 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:36.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:36.211 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:44:36.211 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone4", 2022-09-03T05:44:36.211 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:44:36.212 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi189.front.sepia.ceph.com:8001" 2022-09-03T05:44:36.212 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:44:36.212 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:44:36.212 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:44:36.213 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:44:36.213 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:44:36.213 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "pubsub", 2022-09-03T05:44:36.214 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": false, 2022-09-03T05:44:36.214 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [ 2022-09-03T05:44:36.214 INFO:teuthology.orchestra.run.smithi189.stdout: "test-zone1" 2022-09-03T05:44:36.214 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:44:36.215 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:44:36.215 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:44:36.215 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:44:36.215 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:44:36.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:36.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:36.216 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:44:36.216 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone1", 2022-09-03T05:44:36.217 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:44:36.217 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:44:36.217 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:44:36.217 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:44:36.218 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:44:36.218 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:44:36.218 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:44:36.218 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:44:36.219 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": true, 2022-09-03T05:44:36.219 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:44:36.219 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:44:36.219 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:44:36.220 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:44:36.220 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:44:36.220 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:36.220 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:36.221 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:44:36.221 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone2", 2022-09-03T05:44:36.221 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:44:36.221 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi189.front.sepia.ceph.com:8000" 2022-09-03T05:44:36.222 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:44:36.225 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:44:36.226 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:44:36.227 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:44:36.227 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:44:36.228 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:44:36.228 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": true, 2022-09-03T05:44:36.229 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:44:36.229 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:44:36.230 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:44:36.230 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:44:36.231 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:44:36.231 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:36.232 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:44:36.232 INFO:teuthology.orchestra.run.smithi189.stdout: "placement_targets": [ 2022-09-03T05:44:36.232 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:36.235 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "default-placement", 2022-09-03T05:44:36.235 INFO:teuthology.orchestra.run.smithi189.stdout: "tags": [], 2022-09-03T05:44:36.236 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_classes": [ 2022-09-03T05:44:36.236 INFO:teuthology.orchestra.run.smithi189.stdout: "STANDARD" 2022-09-03T05:44:36.236 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:44:36.237 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:36.237 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:44:36.237 INFO:teuthology.orchestra.run.smithi189.stdout: "default_placement": "default-placement", 2022-09-03T05:44:36.238 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:44:36.238 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_policy": { 2022-09-03T05:44:36.239 INFO:teuthology.orchestra.run.smithi189.stdout: "groups": [] 2022-09-03T05:44:36.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:36.239 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled_features": [ 2022-09-03T05:44:36.239 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:44:36.240 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:44:36.240 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:36.241 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:44:36.241 INFO:teuthology.orchestra.run.smithi189.stdout: "short_zone_ids": [ 2022-09-03T05:44:36.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:36.242 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:44:36.243 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 327838863 2022-09-03T05:44:36.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:36.243 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:36.244 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:44:36.244 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 1690651894 2022-09-03T05:44:36.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:36.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:36.246 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:44:36.246 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 1752562153 2022-09-03T05:44:36.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:36.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:44:36.248 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:44:36.248 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 2011734872 2022-09-03T05:44:36.249 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:36.249 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:44:36.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:36.250 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zonegroup": "test-zonegroup", 2022-09-03T05:44:36.250 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zone": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:44:36.251 INFO:teuthology.orchestra.run.smithi189.stdout: "period_config": { 2022-09-03T05:44:36.251 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_quota": { 2022-09-03T05:44:36.252 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false, 2022-09-03T05:44:36.252 INFO:teuthology.orchestra.run.smithi189.stdout: "check_on_raw": false, 2022-09-03T05:44:36.252 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size": -1, 2022-09-03T05:44:36.253 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size_kb": 0, 2022-09-03T05:44:36.253 INFO:teuthology.orchestra.run.smithi189.stdout: "max_objects": -1 2022-09-03T05:44:36.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:36.254 INFO:teuthology.orchestra.run.smithi189.stdout: "user_quota": { 2022-09-03T05:44:36.255 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false, 2022-09-03T05:44:36.255 INFO:teuthology.orchestra.run.smithi189.stdout: "check_on_raw": false, 2022-09-03T05:44:36.256 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size": -1, 2022-09-03T05:44:36.256 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size_kb": 0, 2022-09-03T05:44:36.257 INFO:teuthology.orchestra.run.smithi189.stdout: "max_objects": -1 2022-09-03T05:44:36.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:36.258 INFO:teuthology.orchestra.run.smithi189.stdout: "user_ratelimit": { 2022-09-03T05:44:36.259 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:44:36.259 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:44:36.259 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:44:36.260 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:44:36.260 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:44:36.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:36.261 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_ratelimit": { 2022-09-03T05:44:36.261 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:44:36.262 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:44:36.262 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:44:36.263 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:44:36.263 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:44:36.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:36.264 INFO:teuthology.orchestra.run.smithi189.stdout: "anonymous_ratelimit": { 2022-09-03T05:44:36.264 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:44:36.264 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:44:36.265 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:44:36.265 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:44:36.266 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:44:36.266 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:44:36.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:44:36.267 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:44:36.267 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_name": "test-realm", 2022-09-03T05:44:36.267 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 3 2022-09-03T05:44:36.268 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:44:36.268 INFO:rgw_multi.tests:Set master zone=test-zone2, waiting 60s for reconfiguration.. 2022-09-03T05:44:36.435 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c1.rgw.client.1 has been restored 2022-09-03T05:45:36.270 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-42 2022-09-03T05:45:36.272 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:45:36.273 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:45:36.273 DEBUG:boto:Method: PUT 2022-09-03T05:45:36.274 DEBUG:boto:Path: /dsgvcy-42/ 2022-09-03T05:45:36.274 DEBUG:boto:Data: 2022-09-03T05:45:36.275 DEBUG:boto:Headers: {} 2022-09-03T05:45:36.275 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:45:36.276 DEBUG:boto:Port: 8000 2022-09-03T05:45:36.276 DEBUG:boto:Params: {} 2022-09-03T05:45:36.277 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:45:36.278 DEBUG:boto:Token: None 2022-09-03T05:45:36.279 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:45:36 GMT /dsgvcy-42/ 2022-09-03T05:45:36.279 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UZ2/SGa1lkfsxkWb/DoxXe+zh8c= 2022-09-03T05:45:36.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UZ2/SGa1lkfsxkWb/DoxXe+zh8c=', 'Content-Length': '0'} 2022-09-03T05:45:36.322 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000019a3c949b9596957-006312ea00-141e-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:45:36.323 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=dsgvcy-43 2022-09-03T05:45:36.324 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:45:36.324 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:45:36.325 DEBUG:boto:Method: PUT 2022-09-03T05:45:36.325 DEBUG:boto:Path: /dsgvcy-43/ 2022-09-03T05:45:36.326 DEBUG:boto:Data: 2022-09-03T05:45:36.326 DEBUG:boto:Headers: {} 2022-09-03T05:45:36.327 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:45:36.327 DEBUG:boto:Port: 8000 2022-09-03T05:45:36.328 DEBUG:boto:Params: {} 2022-09-03T05:45:36.328 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:45:36.329 DEBUG:boto:Token: None 2022-09-03T05:45:36.329 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:45:36 GMT /dsgvcy-43/ 2022-09-03T05:45:36.330 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y95mF85MHnmHeuQE4Pht1/HIeXs= 2022-09-03T05:45:36.331 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y95mF85MHnmHeuQE4Pht1/HIeXs=', 'Content-Length': '0'} 2022-09-03T05:45:36.349 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000002bea4f909872a19c-006312ea00-1434-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:45:36.350 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:45:36.351 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:45:36.352 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:45:36.409 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:45:36.410 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:45:36.440 INFO:teuthology.orchestra.run.smithi189.stderr:No --period given, using current period=eff2cb82-c479-4631-8568-fb1af4159e02 2022-09-03T05:45:36.452 INFO:teuthology.orchestra.run.smithi189.stdout:[ 2022-09-03T05:45:36.453 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:45:36.453 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:45:36.453 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "0.000000" 2022-09-03T05:45:36.453 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:45:36.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:45:36.454 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:45:36.455 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "0.000000" 2022-09-03T05:45:36.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:45:36.455 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:45:36.455 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183936.347465_121.1", 2022-09-03T05:45:36.456 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:45:36.347465Z" 2022-09-03T05:45:36.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:45:36.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:45:36.457 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:45:36.457 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "0.000000" 2022-09-03T05:45:36.457 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:45:36.458 INFO:teuthology.orchestra.run.smithi189.stdout:] 2022-09-03T05:45:36.460 DEBUG:rgw_multi.tests:master meta markers={0: '', 1: '', 2: '1_1662183936.347465_121.1', 3: ''} 2022-09-03T05:45:36.461 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone1 2022-09-03T05:45:36.462 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:45:36.463 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:45:36.464 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:45:36.522 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:45:36.522 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:45:36.552 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:45:36.551+0000 7f9612bc5600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:45:36.553 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:45:36.552+0000 7f9612bc5600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:45:36.566 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:45:36.567 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:45:36.567 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:45:36.568 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:45:36.568 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:45:36.569 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "eff2cb82-c479-4631-8568-fb1af4159e02", 2022-09-03T05:45:36.569 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 3 2022-09-03T05:45:36.569 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:45:36.570 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:45:36.570 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:45:36.570 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:45:36.571 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:45:36.571 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 0, 2022-09-03T05:45:36.571 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "bucket.instance:dsgvcy-40:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.40", 2022-09-03T05:45:36.571 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:45:36.572 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 81, 2022-09-03T05:45:36.572 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 34, 2022-09-03T05:45:36.572 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:45:36.573 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 0 2022-09-03T05:45:36.573 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:45:36.573 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:45:36.573 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:45:36.574 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:45:36.574 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:45:36.574 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:45:36.575 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:45:36.575 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:45:36.575 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:45:36.576 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:45:36.576 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:45:36.577 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 3 2022-09-03T05:45:36.578 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:45:36.578 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:45:36.578 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:45:36.579 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:45:36.579 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:45:36.579 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:45:36.579 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662183936.347465_121.1", 2022-09-03T05:45:36.580 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:45:36.580 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:45:36.581 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:45:36.581 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:45:36.347465Z", 2022-09-03T05:45:36.582 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 3 2022-09-03T05:45:36.582 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:45:36.583 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:45:36.583 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:45:36.584 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:45:36.584 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:45:36.584 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:45:36.585 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:45:36.585 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:45:36.586 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:45:36.587 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:45:36.587 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:45:36.587 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 3 2022-09-03T05:45:36.587 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:45:36.588 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:45:36.588 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:45:36.588 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:45:36.589 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:45:36.589 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 81, 2022-09-03T05:45:36.589 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 34 2022-09-03T05:45:36.590 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:45:36.590 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:45:36Z" 2022-09-03T05:45:36.590 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:45:36.592 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "eff2cb82-c479-4631-8568-fb1af4159e02", "realm_epoch": 3 }, "markers": [ { "key": 0, "val": { "state": 0, "marker": "bucket.instance:dsgvcy-40:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.40", "next_step_marker": "", "total_entries": 81, "pos": 34, "timestamp": "0.000000", "realm_epoch": 0 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 3 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183936.347465_121.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:45:36.347465Z", "realm_epoch": 3 } }, { "key": 3, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 3 } } ] }, "full_sync": { "total": 81, "complete": 34 }, "current_time": "2022-09-03T05:45:36Z" } 2022-09-03T05:45:36.592 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 0, 'marker': 'bucket.instance:dsgvcy-40:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.40', 'next_step_marker': '', 'total_entries': 81, 'pos': 34, 'timestamp': '0.000000', 'realm_epoch': 0}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 3}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183936.347465_121.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:45:36.347465Z', 'realm_epoch': 3}}, {'key': 3, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 3}}] 2022-09-03T05:45:36.593 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662183936.347465_121.1', 3: ''} 2022-09-03T05:45:36.594 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662183936.347465_121.1', 3: ''} 2022-09-03T05:45:36.594 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone1 2022-09-03T05:45:36.595 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--master', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:45:36.595 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'modify', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--master', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:45:36.596 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone modify --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --master --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:45:36.646 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:45:36.646 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:45:36.667 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:45:36.665+0000 7f6d07fcb600 0 NOTICE: overriding master zone: e40d88ba-d542-483a-afa2-5de687affd14 2022-09-03T05:45:36.667 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:45:36.665+0000 7f6d07fcb600 0 failed reading obj info from .rgw.root:zone_info.878fc3d4-c5f9-4296-8cc0-f2a8e029bc48: (2) No such file or directory 2022-09-03T05:45:36.668 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:45:36.665+0000 7f6d07fcb600 0 WARNING: could not read zone params for zone id=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48 name=test-zone4 2022-09-03T05:45:36.669 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:45:36.666+0000 7f6d07fcb600 0 failed reading obj info from .rgw.root:zone_info.e40d88ba-d542-483a-afa2-5de687affd14: (2) No such file or directory 2022-09-03T05:45:36.669 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:45:36.666+0000 7f6d07fcb600 0 WARNING: could not read zone params for zone id=e40d88ba-d542-483a-afa2-5de687affd14 name=test-zone2 2022-09-03T05:45:36.671 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:45:36.672 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:45:36.672 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone1", 2022-09-03T05:45:36.673 INFO:teuthology.orchestra.run.smithi181.stdout: "domain_root": "test-zone1.rgw.meta:root", 2022-09-03T05:45:36.673 INFO:teuthology.orchestra.run.smithi181.stdout: "control_pool": "test-zone1.rgw.control", 2022-09-03T05:45:36.674 INFO:teuthology.orchestra.run.smithi181.stdout: "gc_pool": "test-zone1.rgw.log:gc", 2022-09-03T05:45:36.674 INFO:teuthology.orchestra.run.smithi181.stdout: "lc_pool": "test-zone1.rgw.log:lc", 2022-09-03T05:45:36.675 INFO:teuthology.orchestra.run.smithi181.stdout: "log_pool": "test-zone1.rgw.log", 2022-09-03T05:45:36.675 INFO:teuthology.orchestra.run.smithi181.stdout: "intent_log_pool": "test-zone1.rgw.log:intent", 2022-09-03T05:45:36.675 INFO:teuthology.orchestra.run.smithi181.stdout: "usage_log_pool": "test-zone1.rgw.log:usage", 2022-09-03T05:45:36.676 INFO:teuthology.orchestra.run.smithi181.stdout: "roles_pool": "test-zone1.rgw.meta:roles", 2022-09-03T05:45:36.676 INFO:teuthology.orchestra.run.smithi181.stdout: "reshard_pool": "test-zone1.rgw.log:reshard", 2022-09-03T05:45:36.677 INFO:teuthology.orchestra.run.smithi181.stdout: "user_keys_pool": "test-zone1.rgw.meta:users.keys", 2022-09-03T05:45:36.677 INFO:teuthology.orchestra.run.smithi181.stdout: "user_email_pool": "test-zone1.rgw.meta:users.email", 2022-09-03T05:45:36.677 INFO:teuthology.orchestra.run.smithi181.stdout: "user_swift_pool": "test-zone1.rgw.meta:users.swift", 2022-09-03T05:45:36.678 INFO:teuthology.orchestra.run.smithi181.stdout: "user_uid_pool": "test-zone1.rgw.meta:users.uid", 2022-09-03T05:45:36.678 INFO:teuthology.orchestra.run.smithi181.stdout: "otp_pool": "test-zone1.rgw.otp", 2022-09-03T05:45:36.679 INFO:teuthology.orchestra.run.smithi181.stdout: "system_key": { 2022-09-03T05:45:36.679 INFO:teuthology.orchestra.run.smithi181.stdout: "access_key": "TW0E8EULCERVYU84", 2022-09-03T05:45:36.679 INFO:teuthology.orchestra.run.smithi181.stdout: "secret_key": "EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP" 2022-09-03T05:45:36.680 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:45:36.680 INFO:teuthology.orchestra.run.smithi181.stdout: "placement_pools": [ 2022-09-03T05:45:36.680 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:45:36.680 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "default-placement", 2022-09-03T05:45:36.681 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:45:36.681 INFO:teuthology.orchestra.run.smithi181.stdout: "index_pool": "test-zone1.rgw.buckets.index", 2022-09-03T05:45:36.681 INFO:teuthology.orchestra.run.smithi181.stdout: "storage_classes": { 2022-09-03T05:45:36.681 INFO:teuthology.orchestra.run.smithi181.stdout: "STANDARD": { 2022-09-03T05:45:36.682 INFO:teuthology.orchestra.run.smithi181.stdout: "data_pool": "test-zone1.rgw.buckets.data", 2022-09-03T05:45:36.682 INFO:teuthology.orchestra.run.smithi181.stdout: "compression_type": "random" 2022-09-03T05:45:36.682 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:45:36.683 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:45:36.683 INFO:teuthology.orchestra.run.smithi181.stdout: "data_extra_pool": "test-zone1.rgw.buckets.non-ec", 2022-09-03T05:45:36.684 INFO:teuthology.orchestra.run.smithi181.stdout: "index_type": 0 2022-09-03T05:45:36.684 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:45:36.684 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:45:36.684 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:45:36.685 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:45:36.685 INFO:teuthology.orchestra.run.smithi181.stdout: "notif_pool": "test-zone1.rgw.log:notif" 2022-09-03T05:45:36.686 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:45:36.688 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:45:36.689 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:45:36.690 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:45:36.740 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:45:36.740 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:45:36.784 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:45:36.782+0000 7ff83b237600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:45:36.785 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:45:36.783+0000 7ff83b237600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:45:36.797 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:45:36.798 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:45:36.798 INFO:teuthology.orchestra.run.smithi181.stdout: "epoch": 1, 2022-09-03T05:45:36.798 INFO:teuthology.orchestra.run.smithi181.stdout: "predecessor_uuid": "eff2cb82-c479-4631-8568-fb1af4159e02", 2022-09-03T05:45:36.799 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": [ 2022-09-03T05:45:36.799 INFO:teuthology.orchestra.run.smithi181.stdout: "", 2022-09-03T05:45:36.799 INFO:teuthology.orchestra.run.smithi181.stdout: "", 2022-09-03T05:45:36.800 INFO:teuthology.orchestra.run.smithi181.stdout: "1_1662183936.347465_121.1", 2022-09-03T05:45:36.800 INFO:teuthology.orchestra.run.smithi181.stdout: "" 2022-09-03T05:45:36.800 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:45:36.800 INFO:teuthology.orchestra.run.smithi181.stdout: "period_map": { 2022-09-03T05:45:36.801 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:45:36.801 INFO:teuthology.orchestra.run.smithi181.stdout: "zonegroups": [ 2022-09-03T05:45:36.801 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:45:36.801 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "test-zonegroup", 2022-09-03T05:45:36.802 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zonegroup", 2022-09-03T05:45:36.802 INFO:teuthology.orchestra.run.smithi181.stdout: "api_name": "", 2022-09-03T05:45:36.802 INFO:teuthology.orchestra.run.smithi181.stdout: "is_master": true, 2022-09-03T05:45:36.803 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:45:36.803 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:45:36.804 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:45:36.804 INFO:teuthology.orchestra.run.smithi181.stdout: "hostnames": [], 2022-09-03T05:45:36.804 INFO:teuthology.orchestra.run.smithi181.stdout: "hostnames_s3website": [], 2022-09-03T05:45:36.804 INFO:teuthology.orchestra.run.smithi181.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:45:36.805 INFO:teuthology.orchestra.run.smithi181.stdout: "zones": [ 2022-09-03T05:45:36.805 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:45:36.805 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:45:36.805 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone3", 2022-09-03T05:45:36.806 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:45:36.806 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi181.front.sepia.ceph.com:8001" 2022-09-03T05:45:36.806 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:45:36.806 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:45:36.807 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": true, 2022-09-03T05:45:36.807 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:45:36.807 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:45:36.808 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "", 2022-09-03T05:45:36.808 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": true, 2022-09-03T05:45:36.808 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [], 2022-09-03T05:45:36.808 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:45:36.809 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:45:36.809 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:45:36.809 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:45:36.809 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:45:36.810 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:45:36.810 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:45:36.810 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone4", 2022-09-03T05:45:36.810 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:45:36.811 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi189.front.sepia.ceph.com:8001" 2022-09-03T05:45:36.811 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:45:36.811 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:45:36.811 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": true, 2022-09-03T05:45:36.812 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:45:36.812 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:45:36.812 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "pubsub", 2022-09-03T05:45:36.812 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": false, 2022-09-03T05:45:36.813 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [ 2022-09-03T05:45:36.813 INFO:teuthology.orchestra.run.smithi181.stdout: "test-zone1" 2022-09-03T05:45:36.813 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:45:36.814 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:45:36.814 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:45:36.814 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:45:36.814 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:45:36.815 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:45:36.815 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:45:36.815 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:45:36.815 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone1", 2022-09-03T05:45:36.816 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:45:36.816 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:45:36.816 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:45:36.816 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:45:36.817 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": true, 2022-09-03T05:45:36.817 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:45:36.817 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:45:36.818 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "", 2022-09-03T05:45:36.818 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": true, 2022-09-03T05:45:36.818 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [], 2022-09-03T05:45:36.818 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:45:36.819 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:45:36.819 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:45:36.819 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:45:36.819 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:45:36.820 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:45:36.820 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:45:36.820 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone2", 2022-09-03T05:45:36.820 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:45:36.821 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi189.front.sepia.ceph.com:8000" 2022-09-03T05:45:36.821 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:45:36.821 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:45:36.822 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": true, 2022-09-03T05:45:36.822 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:45:36.822 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:45:36.822 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "", 2022-09-03T05:45:36.823 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": true, 2022-09-03T05:45:36.823 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [], 2022-09-03T05:45:36.823 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:45:36.823 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:45:36.824 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:45:36.824 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:45:36.824 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:45:36.824 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:45:36.825 INFO:teuthology.orchestra.run.smithi181.stdout: "placement_targets": [ 2022-09-03T05:45:36.825 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:45:36.825 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "default-placement", 2022-09-03T05:45:36.827 INFO:teuthology.orchestra.run.smithi181.stdout: "tags": [], 2022-09-03T05:45:36.827 INFO:teuthology.orchestra.run.smithi181.stdout: "storage_classes": [ 2022-09-03T05:45:36.827 INFO:teuthology.orchestra.run.smithi181.stdout: "STANDARD" 2022-09-03T05:45:36.827 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:45:36.828 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:45:36.828 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:45:36.828 INFO:teuthology.orchestra.run.smithi181.stdout: "default_placement": "default-placement", 2022-09-03T05:45:36.828 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:45:36.829 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_policy": { 2022-09-03T05:45:36.829 INFO:teuthology.orchestra.run.smithi181.stdout: "groups": [] 2022-09-03T05:45:36.829 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:45:36.830 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled_features": [ 2022-09-03T05:45:36.830 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:45:36.830 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:45:36.831 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:45:36.831 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:45:36.831 INFO:teuthology.orchestra.run.smithi181.stdout: "short_zone_ids": [ 2022-09-03T05:45:36.831 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:45:36.832 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:45:36.832 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 327838863 2022-09-03T05:45:36.832 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:45:36.833 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:45:36.833 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:45:36.833 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 1690651894 2022-09-03T05:45:36.833 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:45:36.834 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:45:36.834 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:45:36.834 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 1752562153 2022-09-03T05:45:36.835 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:45:36.835 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:45:36.835 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:45:36.835 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 2011734872 2022-09-03T05:45:36.836 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:45:36.836 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:45:36.836 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:45:36.837 INFO:teuthology.orchestra.run.smithi181.stdout: "master_zonegroup": "test-zonegroup", 2022-09-03T05:45:36.837 INFO:teuthology.orchestra.run.smithi181.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:45:36.837 INFO:teuthology.orchestra.run.smithi181.stdout: "period_config": { 2022-09-03T05:45:36.838 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_quota": { 2022-09-03T05:45:36.838 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false, 2022-09-03T05:45:36.838 INFO:teuthology.orchestra.run.smithi181.stdout: "check_on_raw": false, 2022-09-03T05:45:36.839 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size": -1, 2022-09-03T05:45:36.839 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size_kb": 0, 2022-09-03T05:45:36.839 INFO:teuthology.orchestra.run.smithi181.stdout: "max_objects": -1 2022-09-03T05:45:36.839 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:45:36.840 INFO:teuthology.orchestra.run.smithi181.stdout: "user_quota": { 2022-09-03T05:45:36.840 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false, 2022-09-03T05:45:36.840 INFO:teuthology.orchestra.run.smithi181.stdout: "check_on_raw": false, 2022-09-03T05:45:36.841 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size": -1, 2022-09-03T05:45:36.841 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size_kb": 0, 2022-09-03T05:45:36.841 INFO:teuthology.orchestra.run.smithi181.stdout: "max_objects": -1 2022-09-03T05:45:36.841 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:45:36.842 INFO:teuthology.orchestra.run.smithi181.stdout: "user_ratelimit": { 2022-09-03T05:45:36.842 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_ops": 0, 2022-09-03T05:45:36.842 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_ops": 0, 2022-09-03T05:45:36.843 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_bytes": 0, 2022-09-03T05:45:36.843 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_bytes": 0, 2022-09-03T05:45:36.843 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false 2022-09-03T05:45:36.843 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:45:36.844 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_ratelimit": { 2022-09-03T05:45:36.844 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_ops": 0, 2022-09-03T05:45:36.844 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_ops": 0, 2022-09-03T05:45:36.844 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_bytes": 0, 2022-09-03T05:45:36.845 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_bytes": 0, 2022-09-03T05:45:36.845 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false 2022-09-03T05:45:36.845 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:45:36.845 INFO:teuthology.orchestra.run.smithi181.stdout: "anonymous_ratelimit": { 2022-09-03T05:45:36.846 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_ops": 0, 2022-09-03T05:45:36.846 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_ops": 0, 2022-09-03T05:45:36.846 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_bytes": 0, 2022-09-03T05:45:36.847 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_bytes": 0, 2022-09-03T05:45:36.847 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false 2022-09-03T05:45:36.847 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:45:36.847 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:45:36.848 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:45:36.848 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_name": "test-realm", 2022-09-03T05:45:36.848 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:45:36.848 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:45:36.849 INFO:rgw_multi.tests:Set master zone=test-zone1, waiting 60s for reconfiguration.. 2022-09-03T05:46:36.850 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-44 2022-09-03T05:46:36.852 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:36.852 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:36.853 DEBUG:boto:Method: PUT 2022-09-03T05:46:36.853 DEBUG:boto:Path: /dsgvcy-44/ 2022-09-03T05:46:36.854 DEBUG:boto:Data: 2022-09-03T05:46:36.854 DEBUG:boto:Headers: {} 2022-09-03T05:46:36.855 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:36.855 DEBUG:boto:Port: 8000 2022-09-03T05:46:36.856 DEBUG:boto:Params: {} 2022-09-03T05:46:36.857 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:46:36.857 DEBUG:boto:Token: None 2022-09-03T05:46:36.858 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:46:36 GMT /dsgvcy-44/ 2022-09-03T05:46:36.858 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yWA0NN+fxns0SEXYmrocnOFDaZA= 2022-09-03T05:46:36.859 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yWA0NN+fxns0SEXYmrocnOFDaZA=', 'Content-Length': '0'} 2022-09-03T05:46:36.890 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000001e5d94ddaa3b02f8-006312ea3c-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:36.891 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=dsgvcy-45 2022-09-03T05:46:36.892 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:36.893 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:36.893 DEBUG:boto:Method: PUT 2022-09-03T05:46:36.895 DEBUG:boto:Path: /dsgvcy-45/ 2022-09-03T05:46:36.895 DEBUG:boto:Data: 2022-09-03T05:46:36.896 DEBUG:boto:Headers: {} 2022-09-03T05:46:36.896 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:36.897 DEBUG:boto:Port: 8000 2022-09-03T05:46:36.897 DEBUG:boto:Params: {} 2022-09-03T05:46:36.898 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:46:36.899 DEBUG:boto:Token: None 2022-09-03T05:46:36.900 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:46:36 GMT /dsgvcy-45/ 2022-09-03T05:46:36.901 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7tOogWa5HTDlYr1lRHzOsHRqp50= 2022-09-03T05:46:36.902 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7tOogWa5HTDlYr1lRHzOsHRqp50=', 'Content-Length': '0'} 2022-09-03T05:46:36.944 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000b122578546b95ff9-006312ea3c-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:36.945 INFO:tasks.rgw.c1.client.1:Restarting daemon 2022-09-03T05:46:36.945 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8001' -n client.1 --cluster c1 -k /etc/ceph/c1.client.1.keyring --log-file /var/log/ceph/rgw.c1.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.1.sock --foreground --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c1.client.1.stdout 2>&1 2022-09-03T05:46:36.950 INFO:tasks.rgw.c1.client.1:Started 2022-09-03T05:46:36.951 DEBUG:teuthology.orchestra.run.smithi181:> curl http://smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:36.976 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T05:46:36.978 INFO:teuthology.orchestra.run.smithi181.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T05:46:36.978 INFO:teuthology.orchestra.run.smithi181.stderr: Dload Upload Total Spent Left Speed 2022-09-03T05:46:36.979 INFO:teuthology.orchestra.run.smithi181.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi181.front.sepia.ceph.com port 8001: Connection refused 2022-09-03T05:46:37.980 DEBUG:teuthology.orchestra.run.smithi181:> curl http://smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:38.009 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T05:46:38.010 INFO:teuthology.orchestra.run.smithi181.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T05:46:38.011 INFO:teuthology.orchestra.run.smithi181.stderr: Dload Upload Total Spent Left Speed 2022-09-03T05:46:38.011 INFO:teuthology.orchestra.run.smithi181.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi181.front.sepia.ceph.com port 8001: Connection refused 2022-09-03T05:46:40.013 DEBUG:teuthology.orchestra.run.smithi181:> curl http://smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:40.040 INFO:teuthology.orchestra.run.smithi181.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T05:46:40.041 INFO:teuthology.orchestra.run.smithi181.stderr: Dload Upload Total Spent Left Speed 2022-09-03T05:46:40.042 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T05:46:40.043 INFO:teuthology.orchestra.run.smithi181.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi181.front.sepia.ceph.com port 8001: Connection refused 2022-09-03T05:46:44.045 DEBUG:teuthology.orchestra.run.smithi181:> curl http://smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:44.071 INFO:teuthology.orchestra.run.smithi181.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T05:46:44.071 INFO:teuthology.orchestra.run.smithi181.stderr: Dload Upload Total Spent Left Speed 2022-09-03T05:46:44.072 INFO:teuthology.orchestra.run.smithi181.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 208k 0 --:--:-- --:--:-- --:--:-- 208k 2022-09-03T05:46:44.073 INFO:teuthology.orchestra.run.smithi181.stdout:anonymous 2022-09-03T05:46:44.075 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:46:44.076 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:46:44.077 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:46:44.163 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:46:44.163 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:46:44.202 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T05:46:44.214 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:46:44.214 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:44.214 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:46:44.215 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:46:44.215 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:44.215 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:44.215 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:46:44.216 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:46:44.216 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:44.216 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:44.216 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662183996.928852_163.1", 2022-09-03T05:46:44.217 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:46:36.928852Z" 2022-09-03T05:46:44.217 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:44.217 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:44.217 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:46:44.217 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:46:44.218 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:44.218 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:46:44.219 DEBUG:rgw_multi.tests:master meta markers={0: '', 1: '', 2: '1_1662183996.928852_163.1', 3: ''} 2022-09-03T05:46:44.220 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:46:44.220 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:46:44.221 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:46:44.222 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:46:44.281 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:46:44.281 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:46:44.321 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:46:44.320+0000 7f0c5f5b5600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:46:44.324 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:46:44.321+0000 7f0c5f5b5600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:46:44.335 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:46:44.335 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:46:44.336 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:46:44.336 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:46:44.337 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:46:44.337 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:46:44.338 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:46:44.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:44.339 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:46:44.339 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:44.339 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:46:44.339 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:46:44.340 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:46:44.340 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:46:44.340 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:46:44.341 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:46:44.341 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:46:44.341 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:46:44.341 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:46:44.342 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:44.342 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:44.343 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:44.343 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:46:44.344 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:46:44.344 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:46:44.344 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:46:44.344 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:46:44.345 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:46:44.345 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:46:44.345 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:46:44.345 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:46:44.346 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:44.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:44.346 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:44.347 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:46:44.347 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:46:44.347 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:46:44.347 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183996.928852_163.1", 2022-09-03T05:46:44.348 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:46:44.348 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:46:44.348 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:46:44.348 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:46:36.928852Z", 2022-09-03T05:46:44.349 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:46:44.349 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:44.349 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:44.349 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:44.350 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:46:44.350 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:46:44.350 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:46:44.351 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:46:44.351 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:46:44.351 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:46:44.351 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:46:44.352 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:46:44.352 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:46:44.352 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:44.352 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:44.353 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:46:44.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:44.353 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:46:44.353 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:46:44.354 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:46:44.354 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:44.354 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:46:44Z" 2022-09-03T05:46:44.355 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:46:44.356 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183996.928852_163.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:46:36.928852Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:46:44Z" } 2022-09-03T05:46:44.357 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183996.928852_163.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:46:36.928852Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:46:44.357 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662183996.928852_163.1', 3: ''} 2022-09-03T05:46:44.358 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662183996.928852_163.1', 3: ''} 2022-09-03T05:46:44.358 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:46:44.359 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:46:44.359 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:46:44.360 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:46:44.360 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:46:44.415 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:46:44.415 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:46:44.457 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:46:44.456+0000 7f22f84b2600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:46:44.458 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:46:44.457+0000 7f22f84b2600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:46:44.471 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:46:44.471 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:46:44.471 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:46:44.471 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:46:44.472 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:46:44.472 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:46:44.472 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:46:44.473 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:44.473 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:46:44.473 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:44.473 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:46:44.474 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:46:44.474 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:46:44.475 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:46:44.475 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:46:44.475 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:46:44.475 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:46:44.476 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:46:44.476 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:46:44.476 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:44.476 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:44.477 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:44.477 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:46:44.477 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:46:44.477 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:46:44.478 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:46:44.478 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:46:44.478 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:46:44.479 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:46:44.479 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:46:44.479 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:46:44.479 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:44.480 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:44.480 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:44.480 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:46:44.480 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:46:44.481 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:46:44.481 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662183996.928852_163.1", 2022-09-03T05:46:44.481 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:46:44.481 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:46:44.482 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:46:44.482 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:46:36.928852Z", 2022-09-03T05:46:44.482 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:46:44.483 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:44.483 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:44.483 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:44.483 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:46:44.484 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:46:44.484 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:46:44.484 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:46:44.484 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:46:44.485 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:46:44.485 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:46:44.485 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:46:44.485 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:46:44.486 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:44.486 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:44.487 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:46:44.488 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:44.488 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:46:44.488 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:46:44.488 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:46:44.489 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:44.489 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:46:44Z" 2022-09-03T05:46:44.489 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:46:44.490 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183996.928852_163.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:46:36.928852Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:46:44Z" } 2022-09-03T05:46:44.491 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183996.928852_163.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:46:36.928852Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:46:44.492 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662183996.928852_163.1', 3: ''} 2022-09-03T05:46:44.492 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662183996.928852_163.1', 3: ''} 2022-09-03T05:46:44.493 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:46:44.493 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:46:44.494 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:46:44.495 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:46:44.495 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:46:44.549 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:46:44.550 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:46:44.585 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:46:44.584+0000 7f10b26dd600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:46:44.586 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:46:44.585+0000 7f10b26dd600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:46:44.600 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:46:44.601 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:46:44.601 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:46:44.601 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:46:44.602 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:46:44.602 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:46:44.602 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:46:44.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:44.603 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:46:44.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:44.603 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:46:44.604 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:46:44.604 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:46:44.604 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:46:44.604 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:46:44.605 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:46:44.605 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:46:44.605 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:46:44.605 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:46:44.606 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:44.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:44.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:44.607 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:46:44.608 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:46:44.608 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:46:44.609 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:46:44.609 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:46:44.610 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:46:44.610 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:46:44.611 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:46:44.611 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:46:44.611 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:44.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:44.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:44.613 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:46:44.613 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:46:44.614 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:46:44.614 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183936.347465_121.1", 2022-09-03T05:46:44.615 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:46:44.615 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:46:44.616 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:46:44.616 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:45:36.347465Z", 2022-09-03T05:46:44.617 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 3 2022-09-03T05:46:44.617 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:44.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:44.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:44.618 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:46:44.618 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:46:44.618 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:46:44.619 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:46:44.619 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:46:44.619 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:46:44.619 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:46:44.620 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:46:44.620 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:46:44.620 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:44.620 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:44.621 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:46:44.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:44.621 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:46:44.621 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:46:44.622 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:46:44.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:44.622 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:46:44Z" 2022-09-03T05:46:44.623 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:46:44.624 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183936.347465_121.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:45:36.347465Z", "realm_epoch": 3 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:46:44Z" } 2022-09-03T05:46:44.625 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183936.347465_121.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:45:36.347465Z', 'realm_epoch': 3}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:46:44.625 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662183996.928852_163.1', 3: ''} 2022-09-03T05:46:44.626 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '', 3: ''} 2022-09-03T05:46:44.626 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662183996.928852_163.1 target= 2022-09-03T05:46:49.627 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:46:49.629 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:46:49.630 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:46:49.686 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:46:49.687 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:46:49.715 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:46:49.714+0000 7f3def61b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:46:49.716 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:46:49.715+0000 7f3def61b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:46:49.734 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:46:49.735 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:46:49.735 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:46:49.735 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:46:49.736 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:46:49.736 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:46:49.736 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:46:49.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:49.737 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:46:49.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:49.737 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:46:49.738 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:46:49.738 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:46:49.738 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:46:49.738 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:46:49.739 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:46:49.739 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:46:49.739 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:46:49.740 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:46:49.740 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:49.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:49.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:49.741 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:46:49.741 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:46:49.741 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:46:49.742 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:46:49.742 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:46:49.742 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:46:49.742 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:46:49.743 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:46:49.743 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:46:49.743 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:49.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:49.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:49.744 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:46:49.744 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:46:49.744 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:46:49.745 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662183996.928852_163.1", 2022-09-03T05:46:49.745 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:46:49.745 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:46:49.745 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:46:49.746 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:46:36.928852Z", 2022-09-03T05:46:49.746 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:46:49.746 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:49.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:49.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:49.747 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:46:49.747 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:46:49.747 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:46:49.748 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:46:49.748 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:46:49.748 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:46:49.748 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:46:49.749 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:46:49.749 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:46:49.749 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:49.749 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:49.750 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:46:49.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:49.750 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:46:49.751 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:46:49.751 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:46:49.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:49.751 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:46:49Z" 2022-09-03T05:46:49.752 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:46:49.753 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662183996.928852_163.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:46:36.928852Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:46:49Z" } 2022-09-03T05:46:49.754 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662183996.928852_163.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:46:36.928852Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:46:49.754 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662183996.928852_163.1', 3: ''} 2022-09-03T05:46:49.755 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662183996.928852_163.1', 3: ''} 2022-09-03T05:46:49.755 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:46:49.756 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-39 zones={test-zone2, test-zone1} 2022-09-03T05:46:49.757 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:49.757 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:49.758 DEBUG:boto:Method: HEAD 2022-09-03T05:46:49.759 DEBUG:boto:Path: /dsgvcy-39/ 2022-09-03T05:46:49.759 DEBUG:boto:Data: 2022-09-03T05:46:49.760 DEBUG:boto:Headers: {} 2022-09-03T05:46:49.760 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:49.761 DEBUG:boto:Port: 8000 2022-09-03T05:46:49.761 DEBUG:boto:Params: {} 2022-09-03T05:46:49.762 DEBUG:boto:Token: None 2022-09-03T05:46:49.763 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:49 GMT /dsgvcy-39/ 2022-09-03T05:46:49.765 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9OH4gcIneCM5A/d0u9uUDIlZl4Y= 2022-09-03T05:46:49.766 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9OH4gcIneCM5A/d0u9uUDIlZl4Y=', 'Content-Length': '0'} 2022-09-03T05:46:49.769 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003634ca052ea3f0fd-006312ea49-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:49.770 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:49.771 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:49.772 DEBUG:boto:Method: HEAD 2022-09-03T05:46:49.773 DEBUG:boto:Path: /dsgvcy-39/ 2022-09-03T05:46:49.774 DEBUG:boto:Data: 2022-09-03T05:46:49.774 DEBUG:boto:Headers: {} 2022-09-03T05:46:49.775 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:49.776 DEBUG:boto:Port: 8000 2022-09-03T05:46:49.777 DEBUG:boto:Params: {} 2022-09-03T05:46:49.777 DEBUG:boto:Token: None 2022-09-03T05:46:49.778 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:49 GMT /dsgvcy-39/ 2022-09-03T05:46:49.779 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9OH4gcIneCM5A/d0u9uUDIlZl4Y= 2022-09-03T05:46:49.779 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9OH4gcIneCM5A/d0u9uUDIlZl4Y=', 'Content-Length': '0'} 2022-09-03T05:46:49.783 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006ed67af3caec2077-006312ea49-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:49.784 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:49.785 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:49.786 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:49.787 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.788 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.788 DEBUG:boto:Method: GET 2022-09-03T05:46:49.789 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.789 DEBUG:boto:Data: 2022-09-03T05:46:49.790 DEBUG:boto:Headers: {} 2022-09-03T05:46:49.791 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:49.791 DEBUG:boto:Port: 8000 2022-09-03T05:46:49.792 DEBUG:boto:Params: {} 2022-09-03T05:46:49.793 DEBUG:boto:Token: None 2022-09-03T05:46:49.794 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:49 GMT /dsgvcy-39/?versions 2022-09-03T05:46:49.795 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FOwxSqXrMawHNRRJho1QfFJXm/c= 2022-09-03T05:46:49.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FOwxSqXrMawHNRRJho1QfFJXm/c=', 'Content-Length': '0'} 2022-09-03T05:46:49.800 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000073b65daf44a9a22a-006312ea49-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:49.801 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:49.803 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:49.804 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:49.804 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:49.805 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.806 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.807 DEBUG:boto:Method: GET 2022-09-03T05:46:49.808 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.808 DEBUG:boto:Data: 2022-09-03T05:46:49.809 DEBUG:boto:Headers: {} 2022-09-03T05:46:49.809 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:49.810 DEBUG:boto:Port: 8000 2022-09-03T05:46:49.810 DEBUG:boto:Params: {} 2022-09-03T05:46:49.811 DEBUG:boto:Token: None 2022-09-03T05:46:49.812 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:49 GMT /dsgvcy-39/?versions 2022-09-03T05:46:49.812 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FOwxSqXrMawHNRRJho1QfFJXm/c= 2022-09-03T05:46:49.813 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FOwxSqXrMawHNRRJho1QfFJXm/c=', 'Content-Length': '0'} 2022-09-03T05:46:49.816 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000019ba3246b78d3990-006312ea49-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:49.817 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:49.818 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:49.818 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:49.819 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.819 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.820 DEBUG:boto:Method: GET 2022-09-03T05:46:49.821 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.821 DEBUG:boto:Data: 2022-09-03T05:46:49.822 DEBUG:boto:Headers: {} 2022-09-03T05:46:49.822 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:49.823 DEBUG:boto:Port: 8000 2022-09-03T05:46:49.823 DEBUG:boto:Params: {} 2022-09-03T05:46:49.824 DEBUG:boto:Token: None 2022-09-03T05:46:49.824 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:49 GMT /dsgvcy-39/?versions 2022-09-03T05:46:49.825 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FOwxSqXrMawHNRRJho1QfFJXm/c= 2022-09-03T05:46:49.826 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FOwxSqXrMawHNRRJho1QfFJXm/c=', 'Content-Length': '0'} 2022-09-03T05:46:49.829 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a52da80d1c33f160-006312ea49-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:49.830 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:49.830 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:49.831 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:49.832 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.832 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.833 DEBUG:boto:Method: GET 2022-09-03T05:46:49.833 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.834 DEBUG:boto:Data: 2022-09-03T05:46:49.835 DEBUG:boto:Headers: {} 2022-09-03T05:46:49.835 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:49.836 DEBUG:boto:Port: 8000 2022-09-03T05:46:49.837 DEBUG:boto:Params: {} 2022-09-03T05:46:49.839 DEBUG:boto:Token: None 2022-09-03T05:46:49.840 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:49 GMT /dsgvcy-39/?versions 2022-09-03T05:46:49.841 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FOwxSqXrMawHNRRJho1QfFJXm/c= 2022-09-03T05:46:49.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FOwxSqXrMawHNRRJho1QfFJXm/c=', 'Content-Length': '0'} 2022-09-03T05:46:49.847 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003fd573822dcfefff-006312ea49-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:49.848 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:49.849 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-39 zones={test-zone2, test-zone1} 2022-09-03T05:46:49.850 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-39 zones={test-zone3, test-zone1} 2022-09-03T05:46:49.851 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:49.851 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:49.852 DEBUG:boto:Method: HEAD 2022-09-03T05:46:49.852 DEBUG:boto:Path: /dsgvcy-39/ 2022-09-03T05:46:49.853 DEBUG:boto:Data: 2022-09-03T05:46:49.853 DEBUG:boto:Headers: {} 2022-09-03T05:46:49.854 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:49.855 DEBUG:boto:Port: 8001 2022-09-03T05:46:49.856 DEBUG:boto:Params: {} 2022-09-03T05:46:49.857 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T05:46:49.857 DEBUG:boto:Token: None 2022-09-03T05:46:49.858 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:49 GMT /dsgvcy-39/ 2022-09-03T05:46:49.858 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9OH4gcIneCM5A/d0u9uUDIlZl4Y= 2022-09-03T05:46:49.859 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9OH4gcIneCM5A/d0u9uUDIlZl4Y=', 'Content-Length': '0'} 2022-09-03T05:46:49.865 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000150fe50d38ce75a8-006312ea49-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:49.866 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:49.867 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:49.867 DEBUG:boto:Method: HEAD 2022-09-03T05:46:49.868 DEBUG:boto:Path: /dsgvcy-39/ 2022-09-03T05:46:49.868 DEBUG:boto:Data: 2022-09-03T05:46:49.869 DEBUG:boto:Headers: {} 2022-09-03T05:46:49.869 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:49.870 DEBUG:boto:Port: 8000 2022-09-03T05:46:49.870 DEBUG:boto:Params: {} 2022-09-03T05:46:49.871 DEBUG:boto:Token: None 2022-09-03T05:46:49.872 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:49 GMT /dsgvcy-39/ 2022-09-03T05:46:49.873 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9OH4gcIneCM5A/d0u9uUDIlZl4Y= 2022-09-03T05:46:49.873 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9OH4gcIneCM5A/d0u9uUDIlZl4Y=', 'Content-Length': '0'} 2022-09-03T05:46:49.876 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c450dad09f5b4ba0-006312ea49-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:49.877 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:49.878 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:49.878 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:49.879 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.879 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.880 DEBUG:boto:Method: GET 2022-09-03T05:46:49.881 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.882 DEBUG:boto:Data: 2022-09-03T05:46:49.882 DEBUG:boto:Headers: {} 2022-09-03T05:46:49.883 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:49.883 DEBUG:boto:Port: 8001 2022-09-03T05:46:49.884 DEBUG:boto:Params: {} 2022-09-03T05:46:49.885 DEBUG:boto:Token: None 2022-09-03T05:46:49.885 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:49 GMT /dsgvcy-39/?versions 2022-09-03T05:46:49.886 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FOwxSqXrMawHNRRJho1QfFJXm/c= 2022-09-03T05:46:49.887 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FOwxSqXrMawHNRRJho1QfFJXm/c=', 'Content-Length': '0'} 2022-09-03T05:46:49.890 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b465978f21eec290-006312ea49-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:49.891 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:49.892 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:49.893 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:49.893 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:49.894 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.895 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.895 DEBUG:boto:Method: GET 2022-09-03T05:46:49.896 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.896 DEBUG:boto:Data: 2022-09-03T05:46:49.897 DEBUG:boto:Headers: {} 2022-09-03T05:46:49.898 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:49.898 DEBUG:boto:Port: 8000 2022-09-03T05:46:49.899 DEBUG:boto:Params: {} 2022-09-03T05:46:49.900 DEBUG:boto:Token: None 2022-09-03T05:46:49.900 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:49 GMT /dsgvcy-39/?versions 2022-09-03T05:46:49.901 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FOwxSqXrMawHNRRJho1QfFJXm/c= 2022-09-03T05:46:49.902 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FOwxSqXrMawHNRRJho1QfFJXm/c=', 'Content-Length': '0'} 2022-09-03T05:46:49.905 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ab34f34215cbd3e1-006312ea49-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:49.905 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:49.906 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:49.907 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:49.907 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.909 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.909 DEBUG:boto:Method: GET 2022-09-03T05:46:49.910 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.910 DEBUG:boto:Data: 2022-09-03T05:46:49.911 DEBUG:boto:Headers: {} 2022-09-03T05:46:49.911 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:49.912 DEBUG:boto:Port: 8001 2022-09-03T05:46:49.912 DEBUG:boto:Params: {} 2022-09-03T05:46:49.913 DEBUG:boto:Token: None 2022-09-03T05:46:49.914 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:49 GMT /dsgvcy-39/?versions 2022-09-03T05:46:49.915 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FOwxSqXrMawHNRRJho1QfFJXm/c= 2022-09-03T05:46:49.915 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FOwxSqXrMawHNRRJho1QfFJXm/c=', 'Content-Length': '0'} 2022-09-03T05:46:49.919 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000076704b05b2db8377-006312ea49-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:49.920 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:49.921 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:49.921 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:49.922 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.923 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.924 DEBUG:boto:Method: GET 2022-09-03T05:46:49.924 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.925 DEBUG:boto:Data: 2022-09-03T05:46:49.925 DEBUG:boto:Headers: {} 2022-09-03T05:46:49.926 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:49.926 DEBUG:boto:Port: 8000 2022-09-03T05:46:49.927 DEBUG:boto:Params: {} 2022-09-03T05:46:49.928 DEBUG:boto:Token: None 2022-09-03T05:46:49.928 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:49 GMT /dsgvcy-39/?versions 2022-09-03T05:46:49.929 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FOwxSqXrMawHNRRJho1QfFJXm/c= 2022-09-03T05:46:49.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FOwxSqXrMawHNRRJho1QfFJXm/c=', 'Content-Length': '0'} 2022-09-03T05:46:49.932 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007e23db6bcfa77ad9-006312ea49-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:49.933 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:49.934 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-39 zones={test-zone3, test-zone1} 2022-09-03T05:46:49.934 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-39 zones={test-zone1, test-zone2} 2022-09-03T05:46:49.935 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:49.936 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:49.936 DEBUG:boto:Method: HEAD 2022-09-03T05:46:49.937 DEBUG:boto:Path: /dsgvcy-39/ 2022-09-03T05:46:49.937 DEBUG:boto:Data: 2022-09-03T05:46:49.938 DEBUG:boto:Headers: {} 2022-09-03T05:46:49.938 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:49.939 DEBUG:boto:Port: 8000 2022-09-03T05:46:49.939 DEBUG:boto:Params: {} 2022-09-03T05:46:49.940 DEBUG:boto:Token: None 2022-09-03T05:46:49.940 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:49 GMT /dsgvcy-39/ 2022-09-03T05:46:49.941 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9OH4gcIneCM5A/d0u9uUDIlZl4Y= 2022-09-03T05:46:49.942 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9OH4gcIneCM5A/d0u9uUDIlZl4Y=', 'Content-Length': '0'} 2022-09-03T05:46:49.944 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d2664e30ab98d3d2-006312ea49-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:49.945 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:49.945 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:49.946 DEBUG:boto:Method: HEAD 2022-09-03T05:46:49.946 DEBUG:boto:Path: /dsgvcy-39/ 2022-09-03T05:46:49.947 DEBUG:boto:Data: 2022-09-03T05:46:49.947 DEBUG:boto:Headers: {} 2022-09-03T05:46:49.948 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:49.948 DEBUG:boto:Port: 8000 2022-09-03T05:46:49.949 DEBUG:boto:Params: {} 2022-09-03T05:46:49.950 DEBUG:boto:Token: None 2022-09-03T05:46:49.950 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:49 GMT /dsgvcy-39/ 2022-09-03T05:46:49.951 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9OH4gcIneCM5A/d0u9uUDIlZl4Y= 2022-09-03T05:46:49.951 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9OH4gcIneCM5A/d0u9uUDIlZl4Y=', 'Content-Length': '0'} 2022-09-03T05:46:49.954 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000833e370f462ae97f-006312ea49-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:49.955 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:49.955 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:49.956 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:49.956 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.957 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.958 DEBUG:boto:Method: GET 2022-09-03T05:46:49.958 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.959 DEBUG:boto:Data: 2022-09-03T05:46:49.960 DEBUG:boto:Headers: {} 2022-09-03T05:46:49.960 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:49.961 DEBUG:boto:Port: 8000 2022-09-03T05:46:49.961 DEBUG:boto:Params: {} 2022-09-03T05:46:49.962 DEBUG:boto:Token: None 2022-09-03T05:46:49.963 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:49 GMT /dsgvcy-39/?versions 2022-09-03T05:46:49.963 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FOwxSqXrMawHNRRJho1QfFJXm/c= 2022-09-03T05:46:49.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FOwxSqXrMawHNRRJho1QfFJXm/c=', 'Content-Length': '0'} 2022-09-03T05:46:49.966 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eee5e859b45ebd8d-006312ea49-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:49.967 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:49.968 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:49.969 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:49.969 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:49.970 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.971 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.971 DEBUG:boto:Method: GET 2022-09-03T05:46:49.972 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.973 DEBUG:boto:Data: 2022-09-03T05:46:49.973 DEBUG:boto:Headers: {} 2022-09-03T05:46:49.974 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:49.974 DEBUG:boto:Port: 8000 2022-09-03T05:46:49.975 DEBUG:boto:Params: {} 2022-09-03T05:46:49.975 DEBUG:boto:Token: None 2022-09-03T05:46:49.976 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:49 GMT /dsgvcy-39/?versions 2022-09-03T05:46:49.977 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FOwxSqXrMawHNRRJho1QfFJXm/c= 2022-09-03T05:46:49.977 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FOwxSqXrMawHNRRJho1QfFJXm/c=', 'Content-Length': '0'} 2022-09-03T05:46:49.982 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000da4d24cf942642e3-006312ea49-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:49.983 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:49.984 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:49.985 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:49.986 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.987 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.988 DEBUG:boto:Method: GET 2022-09-03T05:46:49.989 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:49.989 DEBUG:boto:Data: 2022-09-03T05:46:49.991 DEBUG:boto:Headers: {} 2022-09-03T05:46:49.992 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:49.993 DEBUG:boto:Port: 8000 2022-09-03T05:46:49.994 DEBUG:boto:Params: {} 2022-09-03T05:46:49.996 DEBUG:boto:Token: None 2022-09-03T05:46:49.997 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:49 GMT /dsgvcy-39/?versions 2022-09-03T05:46:49.998 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FOwxSqXrMawHNRRJho1QfFJXm/c= 2022-09-03T05:46:49.999 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FOwxSqXrMawHNRRJho1QfFJXm/c=', 'Content-Length': '0'} 2022-09-03T05:46:50.002 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000307de59870e21c9e-006312ea4a-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.003 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:50.003 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:50.004 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:50.005 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.005 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.006 DEBUG:boto:Method: GET 2022-09-03T05:46:50.007 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.007 DEBUG:boto:Data: 2022-09-03T05:46:50.008 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.008 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:50.009 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.010 DEBUG:boto:Params: {} 2022-09-03T05:46:50.010 DEBUG:boto:Token: None 2022-09-03T05:46:50.012 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-39/?versions 2022-09-03T05:46:50.013 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4= 2022-09-03T05:46:50.014 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4=', 'Content-Length': '0'} 2022-09-03T05:46:50.017 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dffd3de5d5e40cba-006312ea4a-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.018 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:50.018 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-39 zones={test-zone1, test-zone2} 2022-09-03T05:46:50.019 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-39 zones={test-zone3, test-zone2} 2022-09-03T05:46:50.020 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:50.020 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:50.021 DEBUG:boto:Method: HEAD 2022-09-03T05:46:50.021 DEBUG:boto:Path: /dsgvcy-39/ 2022-09-03T05:46:50.021 DEBUG:boto:Data: 2022-09-03T05:46:50.022 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.022 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:50.023 DEBUG:boto:Port: 8001 2022-09-03T05:46:50.023 DEBUG:boto:Params: {} 2022-09-03T05:46:50.024 DEBUG:boto:Token: None 2022-09-03T05:46:50.025 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-39/ 2022-09-03T05:46:50.025 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kbC/UubxOcps71TnwYXrpO06VY8= 2022-09-03T05:46:50.026 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kbC/UubxOcps71TnwYXrpO06VY8=', 'Content-Length': '0'} 2022-09-03T05:46:50.029 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003ef1fc1e85f781d3-006312ea4a-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.030 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:50.032 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:50.033 DEBUG:boto:Method: HEAD 2022-09-03T05:46:50.034 DEBUG:boto:Path: /dsgvcy-39/ 2022-09-03T05:46:50.034 DEBUG:boto:Data: 2022-09-03T05:46:50.035 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.036 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:50.037 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.038 DEBUG:boto:Params: {} 2022-09-03T05:46:50.038 DEBUG:boto:Token: None 2022-09-03T05:46:50.039 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-39/ 2022-09-03T05:46:50.040 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kbC/UubxOcps71TnwYXrpO06VY8= 2022-09-03T05:46:50.041 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kbC/UubxOcps71TnwYXrpO06VY8=', 'Content-Length': '0'} 2022-09-03T05:46:50.043 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e0b31a5c1a466b36-006312ea4a-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.044 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:50.044 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:50.045 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:50.045 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.046 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.047 DEBUG:boto:Method: GET 2022-09-03T05:46:50.047 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.048 DEBUG:boto:Data: 2022-09-03T05:46:50.048 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.049 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:50.050 DEBUG:boto:Port: 8001 2022-09-03T05:46:50.051 DEBUG:boto:Params: {} 2022-09-03T05:46:50.051 DEBUG:boto:Token: None 2022-09-03T05:46:50.052 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-39/?versions 2022-09-03T05:46:50.053 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4= 2022-09-03T05:46:50.053 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4=', 'Content-Length': '0'} 2022-09-03T05:46:50.056 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000034ce90f677e4c437-006312ea4a-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.057 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:50.058 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:50.059 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:50.060 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:50.061 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.061 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.062 DEBUG:boto:Method: GET 2022-09-03T05:46:50.063 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.064 DEBUG:boto:Data: 2022-09-03T05:46:50.064 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.065 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:50.066 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.067 DEBUG:boto:Params: {} 2022-09-03T05:46:50.067 DEBUG:boto:Token: None 2022-09-03T05:46:50.068 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-39/?versions 2022-09-03T05:46:50.069 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4= 2022-09-03T05:46:50.069 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4=', 'Content-Length': '0'} 2022-09-03T05:46:50.072 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b3cc8f00a6515d42-006312ea4a-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.073 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:50.074 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:50.074 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:50.075 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.075 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.076 DEBUG:boto:Method: GET 2022-09-03T05:46:50.076 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.077 DEBUG:boto:Data: 2022-09-03T05:46:50.077 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.078 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:50.078 DEBUG:boto:Port: 8001 2022-09-03T05:46:50.079 DEBUG:boto:Params: {} 2022-09-03T05:46:50.079 DEBUG:boto:Token: None 2022-09-03T05:46:50.080 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-39/?versions 2022-09-03T05:46:50.080 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4= 2022-09-03T05:46:50.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4=', 'Content-Length': '0'} 2022-09-03T05:46:50.084 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000021203ad7916dee11-006312ea4a-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.084 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:50.085 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:50.086 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:50.086 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.087 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.087 DEBUG:boto:Method: GET 2022-09-03T05:46:50.088 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.088 DEBUG:boto:Data: 2022-09-03T05:46:50.089 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.089 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:50.090 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.090 DEBUG:boto:Params: {} 2022-09-03T05:46:50.091 DEBUG:boto:Token: None 2022-09-03T05:46:50.091 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-39/?versions 2022-09-03T05:46:50.092 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4= 2022-09-03T05:46:50.093 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4=', 'Content-Length': '0'} 2022-09-03T05:46:50.095 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003016c7af0ec4fa35-006312ea4a-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.096 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:50.097 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-39 zones={test-zone3, test-zone2} 2022-09-03T05:46:50.097 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-39 zones={test-zone1, test-zone3} 2022-09-03T05:46:50.098 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:50.098 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:50.099 DEBUG:boto:Method: HEAD 2022-09-03T05:46:50.099 DEBUG:boto:Path: /dsgvcy-39/ 2022-09-03T05:46:50.100 DEBUG:boto:Data: 2022-09-03T05:46:50.100 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.101 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:50.101 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.102 DEBUG:boto:Params: {} 2022-09-03T05:46:50.102 DEBUG:boto:Token: None 2022-09-03T05:46:50.103 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-39/ 2022-09-03T05:46:50.104 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kbC/UubxOcps71TnwYXrpO06VY8= 2022-09-03T05:46:50.106 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kbC/UubxOcps71TnwYXrpO06VY8=', 'Content-Length': '0'} 2022-09-03T05:46:50.109 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000020bf3e815444b92c-006312ea4a-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.110 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:50.111 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:50.112 DEBUG:boto:Method: HEAD 2022-09-03T05:46:50.113 DEBUG:boto:Path: /dsgvcy-39/ 2022-09-03T05:46:50.114 DEBUG:boto:Data: 2022-09-03T05:46:50.115 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.115 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:50.116 DEBUG:boto:Port: 8001 2022-09-03T05:46:50.117 DEBUG:boto:Params: {} 2022-09-03T05:46:50.118 DEBUG:boto:Token: None 2022-09-03T05:46:50.119 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-39/ 2022-09-03T05:46:50.120 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kbC/UubxOcps71TnwYXrpO06VY8= 2022-09-03T05:46:50.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kbC/UubxOcps71TnwYXrpO06VY8=', 'Content-Length': '0'} 2022-09-03T05:46:50.124 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001e6a04b70b0fa464-006312ea4a-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.125 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:50.126 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:50.127 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:50.128 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.129 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.130 DEBUG:boto:Method: GET 2022-09-03T05:46:50.131 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.132 DEBUG:boto:Data: 2022-09-03T05:46:50.133 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.134 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:50.135 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.135 DEBUG:boto:Params: {} 2022-09-03T05:46:50.136 DEBUG:boto:Token: None 2022-09-03T05:46:50.136 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-39/?versions 2022-09-03T05:46:50.137 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4= 2022-09-03T05:46:50.137 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4=', 'Content-Length': '0'} 2022-09-03T05:46:50.140 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cc2f29cf280a64c3-006312ea4a-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.141 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:50.142 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:50.142 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:50.143 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:50.143 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.144 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.145 DEBUG:boto:Method: GET 2022-09-03T05:46:50.145 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.146 DEBUG:boto:Data: 2022-09-03T05:46:50.146 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.147 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:50.147 DEBUG:boto:Port: 8001 2022-09-03T05:46:50.148 DEBUG:boto:Params: {} 2022-09-03T05:46:50.149 DEBUG:boto:Token: None 2022-09-03T05:46:50.149 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-39/?versions 2022-09-03T05:46:50.150 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4= 2022-09-03T05:46:50.151 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4=', 'Content-Length': '0'} 2022-09-03T05:46:50.153 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fcffda858ecb9262-006312ea4a-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.154 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:50.155 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:50.155 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:50.156 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.156 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.157 DEBUG:boto:Method: GET 2022-09-03T05:46:50.157 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.158 DEBUG:boto:Data: 2022-09-03T05:46:50.159 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.159 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:50.160 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.161 DEBUG:boto:Params: {} 2022-09-03T05:46:50.161 DEBUG:boto:Token: None 2022-09-03T05:46:50.162 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-39/?versions 2022-09-03T05:46:50.163 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4= 2022-09-03T05:46:50.163 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4=', 'Content-Length': '0'} 2022-09-03T05:46:50.167 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000a0b165d13a659d5-006312ea4a-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.167 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:50.168 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:50.169 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:50.169 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.170 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.171 DEBUG:boto:Method: GET 2022-09-03T05:46:50.171 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.172 DEBUG:boto:Data: 2022-09-03T05:46:50.173 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.173 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:50.174 DEBUG:boto:Port: 8001 2022-09-03T05:46:50.174 DEBUG:boto:Params: {} 2022-09-03T05:46:50.175 DEBUG:boto:Token: None 2022-09-03T05:46:50.175 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-39/?versions 2022-09-03T05:46:50.176 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4= 2022-09-03T05:46:50.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4=', 'Content-Length': '0'} 2022-09-03T05:46:50.180 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c3bb0760c8d9dca6-006312ea4a-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.180 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:50.181 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-39 zones={test-zone1, test-zone3} 2022-09-03T05:46:50.182 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-39 zones={test-zone2, test-zone3} 2022-09-03T05:46:50.182 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:50.183 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:50.183 DEBUG:boto:Method: HEAD 2022-09-03T05:46:50.184 DEBUG:boto:Path: /dsgvcy-39/ 2022-09-03T05:46:50.184 DEBUG:boto:Data: 2022-09-03T05:46:50.185 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.185 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:50.186 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.186 DEBUG:boto:Params: {} 2022-09-03T05:46:50.187 DEBUG:boto:Token: None 2022-09-03T05:46:50.188 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-39/ 2022-09-03T05:46:50.188 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kbC/UubxOcps71TnwYXrpO06VY8= 2022-09-03T05:46:50.189 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kbC/UubxOcps71TnwYXrpO06VY8=', 'Content-Length': '0'} 2022-09-03T05:46:50.191 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001e37df7753811fb0-006312ea4a-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.192 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:50.192 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:50.193 DEBUG:boto:Method: HEAD 2022-09-03T05:46:50.193 DEBUG:boto:Path: /dsgvcy-39/ 2022-09-03T05:46:50.194 DEBUG:boto:Data: 2022-09-03T05:46:50.194 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.195 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:50.195 DEBUG:boto:Port: 8001 2022-09-03T05:46:50.196 DEBUG:boto:Params: {} 2022-09-03T05:46:50.197 DEBUG:boto:Token: None 2022-09-03T05:46:50.197 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-39/ 2022-09-03T05:46:50.198 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kbC/UubxOcps71TnwYXrpO06VY8= 2022-09-03T05:46:50.198 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kbC/UubxOcps71TnwYXrpO06VY8=', 'Content-Length': '0'} 2022-09-03T05:46:50.201 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000113fdbdba9f7334a-006312ea4a-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.202 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:50.202 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:50.203 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:50.204 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.204 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.205 DEBUG:boto:Method: GET 2022-09-03T05:46:50.205 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.206 DEBUG:boto:Data: 2022-09-03T05:46:50.206 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.207 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:50.207 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.208 DEBUG:boto:Params: {} 2022-09-03T05:46:50.208 DEBUG:boto:Token: None 2022-09-03T05:46:50.209 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-39/?versions 2022-09-03T05:46:50.210 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4= 2022-09-03T05:46:50.210 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4=', 'Content-Length': '0'} 2022-09-03T05:46:50.213 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000df0b9d236ac80c09-006312ea4a-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.213 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:50.214 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:50.215 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:50.216 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:50.217 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.217 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.218 DEBUG:boto:Method: GET 2022-09-03T05:46:50.219 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.219 DEBUG:boto:Data: 2022-09-03T05:46:50.220 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.220 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:50.221 DEBUG:boto:Port: 8001 2022-09-03T05:46:50.221 DEBUG:boto:Params: {} 2022-09-03T05:46:50.222 DEBUG:boto:Token: None 2022-09-03T05:46:50.223 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-39/?versions 2022-09-03T05:46:50.223 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4= 2022-09-03T05:46:50.224 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4=', 'Content-Length': '0'} 2022-09-03T05:46:50.227 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ea7cdc17acbb67f3-006312ea4a-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.270 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:50.272 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:50.272 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:50.273 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.273 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.274 DEBUG:boto:Method: GET 2022-09-03T05:46:50.274 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.275 DEBUG:boto:Data: 2022-09-03T05:46:50.275 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.276 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:50.277 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.277 DEBUG:boto:Params: {} 2022-09-03T05:46:50.278 DEBUG:boto:Token: None 2022-09-03T05:46:50.278 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-39/?versions 2022-09-03T05:46:50.279 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4= 2022-09-03T05:46:50.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4=', 'Content-Length': '0'} 2022-09-03T05:46:50.282 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f727c5fdcbd40513-006312ea4a-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.283 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:50.284 DEBUG:boto:path=/dsgvcy-39/ 2022-09-03T05:46:50.284 DEBUG:boto:auth_path=/dsgvcy-39/ 2022-09-03T05:46:50.285 DEBUG:boto:path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.285 DEBUG:boto:auth_path=/dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.286 DEBUG:boto:Method: GET 2022-09-03T05:46:50.286 DEBUG:boto:Path: /dsgvcy-39/?versions&max-keys=999 2022-09-03T05:46:50.287 DEBUG:boto:Data: 2022-09-03T05:46:50.287 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.288 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:50.288 DEBUG:boto:Port: 8001 2022-09-03T05:46:50.289 DEBUG:boto:Params: {} 2022-09-03T05:46:50.290 DEBUG:boto:Token: None 2022-09-03T05:46:50.290 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-39/?versions 2022-09-03T05:46:50.291 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4= 2022-09-03T05:46:50.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sQrmM7CkcoQIrTjTzsWnyqC3Li4=', 'Content-Length': '0'} 2022-09-03T05:46:50.294 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007020b55e2645426d-006312ea4a-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.295 DEBUG:boto:b'dsgvcy-39999false' 2022-09-03T05:46:50.296 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-39 zones={test-zone2, test-zone3} 2022-09-03T05:46:50.296 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-40 zones={test-zone2, test-zone1} 2022-09-03T05:46:50.297 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.297 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.298 DEBUG:boto:Method: HEAD 2022-09-03T05:46:50.298 DEBUG:boto:Path: /dsgvcy-40/ 2022-09-03T05:46:50.299 DEBUG:boto:Data: 2022-09-03T05:46:50.299 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.300 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:50.300 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.301 DEBUG:boto:Params: {} 2022-09-03T05:46:50.302 DEBUG:boto:Token: None 2022-09-03T05:46:50.302 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/ 2022-09-03T05:46:50.303 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM= 2022-09-03T05:46:50.303 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM=', 'Content-Length': '0'} 2022-09-03T05:46:50.306 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000370a1ce215b4bb14-006312ea4a-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.307 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.307 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.308 DEBUG:boto:Method: HEAD 2022-09-03T05:46:50.308 DEBUG:boto:Path: /dsgvcy-40/ 2022-09-03T05:46:50.309 DEBUG:boto:Data: 2022-09-03T05:46:50.309 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.310 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:50.310 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.311 DEBUG:boto:Params: {} 2022-09-03T05:46:50.311 DEBUG:boto:Token: None 2022-09-03T05:46:50.312 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/ 2022-09-03T05:46:50.313 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM= 2022-09-03T05:46:50.313 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM=', 'Content-Length': '0'} 2022-09-03T05:46:50.316 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fafd6696b8c2c03c-006312ea4a-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.317 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:50.318 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.318 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.319 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.320 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.320 DEBUG:boto:Method: GET 2022-09-03T05:46:50.321 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.321 DEBUG:boto:Data: 2022-09-03T05:46:50.322 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.322 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:50.323 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.323 DEBUG:boto:Params: {} 2022-09-03T05:46:50.324 DEBUG:boto:Token: None 2022-09-03T05:46:50.324 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/?versions 2022-09-03T05:46:50.325 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM= 2022-09-03T05:46:50.326 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM=', 'Content-Length': '0'} 2022-09-03T05:46:50.328 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ca59b2d964765f6b-006312ea4a-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.370 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:50.371 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:50.372 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.372 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.373 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.373 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.374 DEBUG:boto:Method: GET 2022-09-03T05:46:50.374 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.375 DEBUG:boto:Data: 2022-09-03T05:46:50.375 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.376 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:50.376 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.377 DEBUG:boto:Params: {} 2022-09-03T05:46:50.378 DEBUG:boto:Token: None 2022-09-03T05:46:50.379 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/?versions 2022-09-03T05:46:50.379 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM= 2022-09-03T05:46:50.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM=', 'Content-Length': '0'} 2022-09-03T05:46:50.384 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005de6aa972dfa3d35-006312ea4a-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.385 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:50.386 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.387 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.388 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.388 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.389 DEBUG:boto:Method: GET 2022-09-03T05:46:50.390 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.391 DEBUG:boto:Data: 2022-09-03T05:46:50.392 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.393 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:50.394 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.395 DEBUG:boto:Params: {} 2022-09-03T05:46:50.395 DEBUG:boto:Token: None 2022-09-03T05:46:50.396 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/?versions 2022-09-03T05:46:50.397 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM= 2022-09-03T05:46:50.397 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM=', 'Content-Length': '0'} 2022-09-03T05:46:50.400 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000077d550694bd0fe28-006312ea4a-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.400 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:50.401 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.402 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.402 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.403 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.404 DEBUG:boto:Method: GET 2022-09-03T05:46:50.404 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.405 DEBUG:boto:Data: 2022-09-03T05:46:50.405 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.406 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:50.406 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.407 DEBUG:boto:Params: {} 2022-09-03T05:46:50.408 DEBUG:boto:Token: None 2022-09-03T05:46:50.408 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/?versions 2022-09-03T05:46:50.409 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM= 2022-09-03T05:46:50.409 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM=', 'Content-Length': '0'} 2022-09-03T05:46:50.412 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003b297f22560e2774-006312ea4a-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.454 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:50.455 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-40 zones={test-zone2, test-zone1} 2022-09-03T05:46:50.456 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-40 zones={test-zone3, test-zone1} 2022-09-03T05:46:50.456 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.457 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.457 DEBUG:boto:Method: HEAD 2022-09-03T05:46:50.458 DEBUG:boto:Path: /dsgvcy-40/ 2022-09-03T05:46:50.458 DEBUG:boto:Data: 2022-09-03T05:46:50.459 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.460 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:50.461 DEBUG:boto:Port: 8001 2022-09-03T05:46:50.461 DEBUG:boto:Params: {} 2022-09-03T05:46:50.462 DEBUG:boto:Token: None 2022-09-03T05:46:50.463 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/ 2022-09-03T05:46:50.463 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM= 2022-09-03T05:46:50.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM=', 'Content-Length': '0'} 2022-09-03T05:46:50.467 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000213add9aba6ac38a-006312ea4a-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.468 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.469 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.469 DEBUG:boto:Method: HEAD 2022-09-03T05:46:50.470 DEBUG:boto:Path: /dsgvcy-40/ 2022-09-03T05:46:50.471 DEBUG:boto:Data: 2022-09-03T05:46:50.471 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.472 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:50.472 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.473 DEBUG:boto:Params: {} 2022-09-03T05:46:50.473 DEBUG:boto:Token: None 2022-09-03T05:46:50.474 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/ 2022-09-03T05:46:50.475 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM= 2022-09-03T05:46:50.475 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM=', 'Content-Length': '0'} 2022-09-03T05:46:50.479 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000077f66b08c6024ee5-006312ea4a-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.479 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:50.480 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.481 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.481 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.482 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.483 DEBUG:boto:Method: GET 2022-09-03T05:46:50.483 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.484 DEBUG:boto:Data: 2022-09-03T05:46:50.484 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.485 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:50.485 DEBUG:boto:Port: 8001 2022-09-03T05:46:50.486 DEBUG:boto:Params: {} 2022-09-03T05:46:50.486 DEBUG:boto:Token: None 2022-09-03T05:46:50.487 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/?versions 2022-09-03T05:46:50.488 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM= 2022-09-03T05:46:50.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM=', 'Content-Length': '0'} 2022-09-03T05:46:50.497 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eaef5315e0b6097a-006312ea4a-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.497 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:50.499 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:50.500 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.500 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.501 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.501 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.502 DEBUG:boto:Method: GET 2022-09-03T05:46:50.503 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.503 DEBUG:boto:Data: 2022-09-03T05:46:50.504 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.504 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:50.505 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.505 DEBUG:boto:Params: {} 2022-09-03T05:46:50.506 DEBUG:boto:Token: None 2022-09-03T05:46:50.506 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/?versions 2022-09-03T05:46:50.507 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM= 2022-09-03T05:46:50.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM=', 'Content-Length': '0'} 2022-09-03T05:46:50.512 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d2aa8c6d2b2bc715-006312ea4a-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.514 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:50.515 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.516 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.517 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.518 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.519 DEBUG:boto:Method: GET 2022-09-03T05:46:50.520 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.521 DEBUG:boto:Data: 2022-09-03T05:46:50.522 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.523 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:50.524 DEBUG:boto:Port: 8001 2022-09-03T05:46:50.525 DEBUG:boto:Params: {} 2022-09-03T05:46:50.526 DEBUG:boto:Token: None 2022-09-03T05:46:50.527 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/?versions 2022-09-03T05:46:50.528 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM= 2022-09-03T05:46:50.529 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM=', 'Content-Length': '0'} 2022-09-03T05:46:50.533 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eda8e1561e23eac5-006312ea4a-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.574 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:50.575 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.576 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.576 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.577 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.577 DEBUG:boto:Method: GET 2022-09-03T05:46:50.578 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.578 DEBUG:boto:Data: 2022-09-03T05:46:50.579 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.579 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:50.580 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.580 DEBUG:boto:Params: {} 2022-09-03T05:46:50.581 DEBUG:boto:Token: None 2022-09-03T05:46:50.582 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/?versions 2022-09-03T05:46:50.582 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM= 2022-09-03T05:46:50.583 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM=', 'Content-Length': '0'} 2022-09-03T05:46:50.586 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000083712ab9679556d7-006312ea4a-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.587 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:50.588 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-40 zones={test-zone3, test-zone1} 2022-09-03T05:46:50.588 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-40 zones={test-zone1, test-zone2} 2022-09-03T05:46:50.589 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.589 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.590 DEBUG:boto:Method: HEAD 2022-09-03T05:46:50.591 DEBUG:boto:Path: /dsgvcy-40/ 2022-09-03T05:46:50.592 DEBUG:boto:Data: 2022-09-03T05:46:50.592 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.593 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:50.593 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.594 DEBUG:boto:Params: {} 2022-09-03T05:46:50.594 DEBUG:boto:Token: None 2022-09-03T05:46:50.595 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/ 2022-09-03T05:46:50.595 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM= 2022-09-03T05:46:50.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM=', 'Content-Length': '0'} 2022-09-03T05:46:50.599 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007b59392ad00d4e32-006312ea4a-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.601 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.602 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.603 DEBUG:boto:Method: HEAD 2022-09-03T05:46:50.604 DEBUG:boto:Path: /dsgvcy-40/ 2022-09-03T05:46:50.605 DEBUG:boto:Data: 2022-09-03T05:46:50.606 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.607 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:50.608 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.609 DEBUG:boto:Params: {} 2022-09-03T05:46:50.610 DEBUG:boto:Token: None 2022-09-03T05:46:50.611 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/ 2022-09-03T05:46:50.612 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM= 2022-09-03T05:46:50.613 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM=', 'Content-Length': '0'} 2022-09-03T05:46:50.616 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a5ff1ddd08097057-006312ea4a-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.617 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:50.618 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.619 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.620 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.620 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.621 DEBUG:boto:Method: GET 2022-09-03T05:46:50.622 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.623 DEBUG:boto:Data: 2022-09-03T05:46:50.625 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.625 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:50.627 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.628 DEBUG:boto:Params: {} 2022-09-03T05:46:50.629 DEBUG:boto:Token: None 2022-09-03T05:46:50.630 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/?versions 2022-09-03T05:46:50.631 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM= 2022-09-03T05:46:50.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM=', 'Content-Length': '0'} 2022-09-03T05:46:50.635 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bdbaf522037e11a1-006312ea4a-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.679 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:50.682 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:50.683 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.684 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.685 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.686 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.687 DEBUG:boto:Method: GET 2022-09-03T05:46:50.688 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.690 DEBUG:boto:Data: 2022-09-03T05:46:50.691 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.692 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:50.693 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.694 DEBUG:boto:Params: {} 2022-09-03T05:46:50.695 DEBUG:boto:Token: None 2022-09-03T05:46:50.696 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/?versions 2022-09-03T05:46:50.699 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM= 2022-09-03T05:46:50.700 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM=', 'Content-Length': '0'} 2022-09-03T05:46:50.704 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000060a8b80546f16bd5-006312ea4a-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.705 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:50.707 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.708 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.709 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.710 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.711 DEBUG:boto:Method: GET 2022-09-03T05:46:50.711 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.713 DEBUG:boto:Data: 2022-09-03T05:46:50.714 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.714 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:50.715 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.715 DEBUG:boto:Params: {} 2022-09-03T05:46:50.716 DEBUG:boto:Token: None 2022-09-03T05:46:50.717 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/?versions 2022-09-03T05:46:50.717 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM= 2022-09-03T05:46:50.718 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM=', 'Content-Length': '0'} 2022-09-03T05:46:50.721 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fbc92ee4bb2a9199-006312ea4a-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.722 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:50.723 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.723 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.724 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.724 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.725 DEBUG:boto:Method: GET 2022-09-03T05:46:50.726 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.726 DEBUG:boto:Data: 2022-09-03T05:46:50.727 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.727 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:50.728 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.729 DEBUG:boto:Params: {} 2022-09-03T05:46:50.729 DEBUG:boto:Token: None 2022-09-03T05:46:50.730 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/?versions 2022-09-03T05:46:50.731 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM= 2022-09-03T05:46:50.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM=', 'Content-Length': '0'} 2022-09-03T05:46:50.735 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e5a92c85f4b32e71-006312ea4a-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.736 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:50.737 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-40 zones={test-zone1, test-zone2} 2022-09-03T05:46:50.738 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-40 zones={test-zone3, test-zone2} 2022-09-03T05:46:50.739 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.740 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.741 DEBUG:boto:Method: HEAD 2022-09-03T05:46:50.742 DEBUG:boto:Path: /dsgvcy-40/ 2022-09-03T05:46:50.743 DEBUG:boto:Data: 2022-09-03T05:46:50.744 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.745 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:50.746 DEBUG:boto:Port: 8001 2022-09-03T05:46:50.747 DEBUG:boto:Params: {} 2022-09-03T05:46:50.748 DEBUG:boto:Token: None 2022-09-03T05:46:50.749 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/ 2022-09-03T05:46:50.750 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM= 2022-09-03T05:46:50.751 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM=', 'Content-Length': '0'} 2022-09-03T05:46:50.754 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009bd0ae77d39c420e-006312ea4a-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.755 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.755 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.756 DEBUG:boto:Method: HEAD 2022-09-03T05:46:50.756 DEBUG:boto:Path: /dsgvcy-40/ 2022-09-03T05:46:50.757 DEBUG:boto:Data: 2022-09-03T05:46:50.758 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.758 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:50.759 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.760 DEBUG:boto:Params: {} 2022-09-03T05:46:50.761 DEBUG:boto:Token: None 2022-09-03T05:46:50.761 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/ 2022-09-03T05:46:50.764 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM= 2022-09-03T05:46:50.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM=', 'Content-Length': '0'} 2022-09-03T05:46:50.770 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000331ff66d2257ba3e-006312ea4a-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.771 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:50.772 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.773 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.774 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.775 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.777 DEBUG:boto:Method: GET 2022-09-03T05:46:50.778 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.779 DEBUG:boto:Data: 2022-09-03T05:46:50.780 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.781 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:50.782 DEBUG:boto:Port: 8001 2022-09-03T05:46:50.783 DEBUG:boto:Params: {} 2022-09-03T05:46:50.784 DEBUG:boto:Token: None 2022-09-03T05:46:50.785 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/?versions 2022-09-03T05:46:50.786 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM= 2022-09-03T05:46:50.787 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM=', 'Content-Length': '0'} 2022-09-03T05:46:50.791 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000028792e0635950c8b-006312ea4a-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.793 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:50.794 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:50.794 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.795 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.795 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.796 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.796 DEBUG:boto:Method: GET 2022-09-03T05:46:50.797 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.798 DEBUG:boto:Data: 2022-09-03T05:46:50.799 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.799 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:50.800 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.801 DEBUG:boto:Params: {} 2022-09-03T05:46:50.802 DEBUG:boto:Token: None 2022-09-03T05:46:50.803 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/?versions 2022-09-03T05:46:50.804 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM= 2022-09-03T05:46:50.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM=', 'Content-Length': '0'} 2022-09-03T05:46:50.807 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000057df4aa229be1b34-006312ea4a-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.854 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:50.855 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.856 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.856 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.857 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.858 DEBUG:boto:Method: GET 2022-09-03T05:46:50.859 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.860 DEBUG:boto:Data: 2022-09-03T05:46:50.860 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.861 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:50.862 DEBUG:boto:Port: 8001 2022-09-03T05:46:50.863 DEBUG:boto:Params: {} 2022-09-03T05:46:50.864 DEBUG:boto:Token: None 2022-09-03T05:46:50.865 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/?versions 2022-09-03T05:46:50.866 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM= 2022-09-03T05:46:50.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM=', 'Content-Length': '0'} 2022-09-03T05:46:50.870 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a6013dc09ef8129c-006312ea4a-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.871 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:50.872 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.873 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.873 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.874 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.874 DEBUG:boto:Method: GET 2022-09-03T05:46:50.875 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.875 DEBUG:boto:Data: 2022-09-03T05:46:50.876 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.876 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:50.877 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.877 DEBUG:boto:Params: {} 2022-09-03T05:46:50.878 DEBUG:boto:Token: None 2022-09-03T05:46:50.879 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/?versions 2022-09-03T05:46:50.879 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM= 2022-09-03T05:46:50.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM=', 'Content-Length': '0'} 2022-09-03T05:46:50.883 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000538c04f24d02e1e4-006312ea4a-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.883 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:50.884 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-40 zones={test-zone3, test-zone2} 2022-09-03T05:46:50.885 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-40 zones={test-zone1, test-zone3} 2022-09-03T05:46:50.885 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.886 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.887 DEBUG:boto:Method: HEAD 2022-09-03T05:46:50.888 DEBUG:boto:Path: /dsgvcy-40/ 2022-09-03T05:46:50.889 DEBUG:boto:Data: 2022-09-03T05:46:50.889 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.890 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:50.891 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.892 DEBUG:boto:Params: {} 2022-09-03T05:46:50.893 DEBUG:boto:Token: None 2022-09-03T05:46:50.894 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/ 2022-09-03T05:46:50.894 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM= 2022-09-03T05:46:50.895 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM=', 'Content-Length': '0'} 2022-09-03T05:46:50.898 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007cebbe81bea35ff5-006312ea4a-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.898 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.899 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.899 DEBUG:boto:Method: HEAD 2022-09-03T05:46:50.900 DEBUG:boto:Path: /dsgvcy-40/ 2022-09-03T05:46:50.900 DEBUG:boto:Data: 2022-09-03T05:46:50.901 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.901 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:50.902 DEBUG:boto:Port: 8001 2022-09-03T05:46:50.902 DEBUG:boto:Params: {} 2022-09-03T05:46:50.903 DEBUG:boto:Token: None 2022-09-03T05:46:50.904 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/ 2022-09-03T05:46:50.904 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM= 2022-09-03T05:46:50.905 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM=', 'Content-Length': '0'} 2022-09-03T05:46:50.909 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000457780003b0c4dd9-006312ea4a-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.911 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:50.912 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.913 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.914 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.914 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.915 DEBUG:boto:Method: GET 2022-09-03T05:46:50.916 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.917 DEBUG:boto:Data: 2022-09-03T05:46:50.919 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.919 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:50.920 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.921 DEBUG:boto:Params: {} 2022-09-03T05:46:50.921 DEBUG:boto:Token: None 2022-09-03T05:46:50.922 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/?versions 2022-09-03T05:46:50.922 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM= 2022-09-03T05:46:50.923 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM=', 'Content-Length': '0'} 2022-09-03T05:46:50.926 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001039cb44c3b9633d-006312ea4a-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.927 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:50.928 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:50.928 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.929 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.930 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.931 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.931 DEBUG:boto:Method: GET 2022-09-03T05:46:50.932 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.933 DEBUG:boto:Data: 2022-09-03T05:46:50.934 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.935 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:50.936 DEBUG:boto:Port: 8001 2022-09-03T05:46:50.937 DEBUG:boto:Params: {} 2022-09-03T05:46:50.938 DEBUG:boto:Token: None 2022-09-03T05:46:50.939 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/?versions 2022-09-03T05:46:50.939 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM= 2022-09-03T05:46:50.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM=', 'Content-Length': '0'} 2022-09-03T05:46:50.943 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000012cb291de4c49199-006312ea4a-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.944 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:50.945 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.945 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.946 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.947 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.947 DEBUG:boto:Method: GET 2022-09-03T05:46:50.948 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.948 DEBUG:boto:Data: 2022-09-03T05:46:50.949 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.949 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:50.950 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.950 DEBUG:boto:Params: {} 2022-09-03T05:46:50.951 DEBUG:boto:Token: None 2022-09-03T05:46:50.952 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/?versions 2022-09-03T05:46:50.952 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM= 2022-09-03T05:46:50.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM=', 'Content-Length': '0'} 2022-09-03T05:46:50.956 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c0f8bd5d10d0aa4f-006312ea4a-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.957 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:50.958 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.958 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.959 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.959 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.960 DEBUG:boto:Method: GET 2022-09-03T05:46:50.960 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:50.961 DEBUG:boto:Data: 2022-09-03T05:46:50.961 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.962 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:50.962 DEBUG:boto:Port: 8001 2022-09-03T05:46:50.963 DEBUG:boto:Params: {} 2022-09-03T05:46:50.964 DEBUG:boto:Token: None 2022-09-03T05:46:50.965 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/?versions 2022-09-03T05:46:50.965 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM= 2022-09-03T05:46:50.966 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HkXoqM1M34hPVfKzc67+SlD0NJM=', 'Content-Length': '0'} 2022-09-03T05:46:50.977 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f40c24b5424f496e-006312ea4a-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.978 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:50.979 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-40 zones={test-zone1, test-zone3} 2022-09-03T05:46:50.979 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-40 zones={test-zone2, test-zone3} 2022-09-03T05:46:50.980 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.980 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.981 DEBUG:boto:Method: HEAD 2022-09-03T05:46:50.981 DEBUG:boto:Path: /dsgvcy-40/ 2022-09-03T05:46:50.982 DEBUG:boto:Data: 2022-09-03T05:46:50.982 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.983 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:50.983 DEBUG:boto:Port: 8000 2022-09-03T05:46:50.984 DEBUG:boto:Params: {} 2022-09-03T05:46:50.984 DEBUG:boto:Token: None 2022-09-03T05:46:50.986 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/ 2022-09-03T05:46:50.986 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM= 2022-09-03T05:46:50.987 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM=', 'Content-Length': '0'} 2022-09-03T05:46:50.989 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fe80206ca1a4fdb2-006312ea4a-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:50.990 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:50.991 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:50.992 DEBUG:boto:Method: HEAD 2022-09-03T05:46:50.993 DEBUG:boto:Path: /dsgvcy-40/ 2022-09-03T05:46:50.993 DEBUG:boto:Data: 2022-09-03T05:46:50.994 DEBUG:boto:Headers: {} 2022-09-03T05:46:50.995 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:50.996 DEBUG:boto:Port: 8001 2022-09-03T05:46:50.997 DEBUG:boto:Params: {} 2022-09-03T05:46:50.998 DEBUG:boto:Token: None 2022-09-03T05:46:50.999 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:50 GMT /dsgvcy-40/ 2022-09-03T05:46:51.000 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM= 2022-09-03T05:46:51.001 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+/TNYxADLBm8gRefFSYbCpprFBM=', 'Content-Length': '0'} 2022-09-03T05:46:51.004 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000534daa6dd2aae9d7-006312ea4b-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.006 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:51.007 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:51.008 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:51.009 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:51.010 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:51.011 DEBUG:boto:Method: GET 2022-09-03T05:46:51.012 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:51.012 DEBUG:boto:Data: 2022-09-03T05:46:51.014 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.015 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:51.016 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.016 DEBUG:boto:Params: {} 2022-09-03T05:46:51.017 DEBUG:boto:Token: None 2022-09-03T05:46:51.018 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-40/?versions 2022-09-03T05:46:51.019 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KH63HZV0pBOeyI8eoUgR+fR9CJ4= 2022-09-03T05:46:51.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KH63HZV0pBOeyI8eoUgR+fR9CJ4=', 'Content-Length': '0'} 2022-09-03T05:46:51.024 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008925255da6ddfdbe-006312ea4b-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.025 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:51.026 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:51.027 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:51.028 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:51.029 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:51.030 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:51.031 DEBUG:boto:Method: GET 2022-09-03T05:46:51.032 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:51.033 DEBUG:boto:Data: 2022-09-03T05:46:51.036 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.037 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:51.038 DEBUG:boto:Port: 8001 2022-09-03T05:46:51.039 DEBUG:boto:Params: {} 2022-09-03T05:46:51.040 DEBUG:boto:Token: None 2022-09-03T05:46:51.041 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-40/?versions 2022-09-03T05:46:51.042 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KH63HZV0pBOeyI8eoUgR+fR9CJ4= 2022-09-03T05:46:51.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KH63HZV0pBOeyI8eoUgR+fR9CJ4=', 'Content-Length': '0'} 2022-09-03T05:46:51.046 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000774e213c7799539b-006312ea4b-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.090 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:51.092 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:51.093 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:51.093 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:51.094 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:51.095 DEBUG:boto:Method: GET 2022-09-03T05:46:51.096 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:51.097 DEBUG:boto:Data: 2022-09-03T05:46:51.097 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.098 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:51.098 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.099 DEBUG:boto:Params: {} 2022-09-03T05:46:51.100 DEBUG:boto:Token: None 2022-09-03T05:46:51.100 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-40/?versions 2022-09-03T05:46:51.101 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KH63HZV0pBOeyI8eoUgR+fR9CJ4= 2022-09-03T05:46:51.101 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KH63HZV0pBOeyI8eoUgR+fR9CJ4=', 'Content-Length': '0'} 2022-09-03T05:46:51.104 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005b6086cbcee2bf4b-006312ea4b-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.105 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:51.106 DEBUG:boto:path=/dsgvcy-40/ 2022-09-03T05:46:51.107 DEBUG:boto:auth_path=/dsgvcy-40/ 2022-09-03T05:46:51.108 DEBUG:boto:path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:51.109 DEBUG:boto:auth_path=/dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:51.109 DEBUG:boto:Method: GET 2022-09-03T05:46:51.110 DEBUG:boto:Path: /dsgvcy-40/?versions&max-keys=999 2022-09-03T05:46:51.111 DEBUG:boto:Data: 2022-09-03T05:46:51.112 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.113 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:51.114 DEBUG:boto:Port: 8001 2022-09-03T05:46:51.114 DEBUG:boto:Params: {} 2022-09-03T05:46:51.115 DEBUG:boto:Token: None 2022-09-03T05:46:51.116 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-40/?versions 2022-09-03T05:46:51.117 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KH63HZV0pBOeyI8eoUgR+fR9CJ4= 2022-09-03T05:46:51.118 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KH63HZV0pBOeyI8eoUgR+fR9CJ4=', 'Content-Length': '0'} 2022-09-03T05:46:51.122 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000054ac2b7614034f8e-006312ea4b-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.123 DEBUG:boto:b'dsgvcy-40999false' 2022-09-03T05:46:51.124 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-40 zones={test-zone2, test-zone3} 2022-09-03T05:46:51.125 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-41 zones={test-zone2, test-zone1} 2022-09-03T05:46:51.126 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.127 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.128 DEBUG:boto:Method: HEAD 2022-09-03T05:46:51.129 DEBUG:boto:Path: /dsgvcy-41/ 2022-09-03T05:46:51.129 DEBUG:boto:Data: 2022-09-03T05:46:51.130 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.130 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:51.131 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.131 DEBUG:boto:Params: {} 2022-09-03T05:46:51.132 DEBUG:boto:Token: None 2022-09-03T05:46:51.132 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/ 2022-09-03T05:46:51.133 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk= 2022-09-03T05:46:51.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk=', 'Content-Length': '0'} 2022-09-03T05:46:51.136 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b104f13c32e2321b-006312ea4b-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.137 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.138 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.138 DEBUG:boto:Method: HEAD 2022-09-03T05:46:51.139 DEBUG:boto:Path: /dsgvcy-41/ 2022-09-03T05:46:51.139 DEBUG:boto:Data: 2022-09-03T05:46:51.140 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.140 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:51.141 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.141 DEBUG:boto:Params: {} 2022-09-03T05:46:51.142 DEBUG:boto:Token: None 2022-09-03T05:46:51.143 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/ 2022-09-03T05:46:51.143 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk= 2022-09-03T05:46:51.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk=', 'Content-Length': '0'} 2022-09-03T05:46:51.147 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003ced04fc66116847-006312ea4b-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.148 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:51.149 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.149 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.150 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.150 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.151 DEBUG:boto:Method: GET 2022-09-03T05:46:51.151 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.152 DEBUG:boto:Data: 2022-09-03T05:46:51.152 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.153 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:51.153 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.154 DEBUG:boto:Params: {} 2022-09-03T05:46:51.154 DEBUG:boto:Token: None 2022-09-03T05:46:51.155 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.156 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.159 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002e13949d56b64096-006312ea4b-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.160 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.161 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:51.161 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.162 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.162 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.163 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.163 DEBUG:boto:Method: GET 2022-09-03T05:46:51.164 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.164 DEBUG:boto:Data: 2022-09-03T05:46:51.165 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.165 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:51.166 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.166 DEBUG:boto:Params: {} 2022-09-03T05:46:51.167 DEBUG:boto:Token: None 2022-09-03T05:46:51.168 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.168 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.169 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.172 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c598ad6f3ee54a41-006312ea4b-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.172 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.173 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.174 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.174 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.175 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.175 DEBUG:boto:Method: GET 2022-09-03T05:46:51.176 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.176 DEBUG:boto:Data: 2022-09-03T05:46:51.177 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.177 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:51.178 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.178 DEBUG:boto:Params: {} 2022-09-03T05:46:51.179 DEBUG:boto:Token: None 2022-09-03T05:46:51.179 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.180 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.181 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.183 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000524db7880f630054-006312ea4b-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.184 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.185 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.185 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.186 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.186 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.187 DEBUG:boto:Method: GET 2022-09-03T05:46:51.187 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.188 DEBUG:boto:Data: 2022-09-03T05:46:51.188 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.189 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:51.189 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.190 DEBUG:boto:Params: {} 2022-09-03T05:46:51.191 DEBUG:boto:Token: None 2022-09-03T05:46:51.191 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.192 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.195 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006082726926427896-006312ea4b-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.238 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.240 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-41 zones={test-zone2, test-zone1} 2022-09-03T05:46:51.240 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-41 zones={test-zone3, test-zone1} 2022-09-03T05:46:51.241 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.241 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.242 DEBUG:boto:Method: HEAD 2022-09-03T05:46:51.242 DEBUG:boto:Path: /dsgvcy-41/ 2022-09-03T05:46:51.243 DEBUG:boto:Data: 2022-09-03T05:46:51.244 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.244 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:51.245 DEBUG:boto:Port: 8001 2022-09-03T05:46:51.245 DEBUG:boto:Params: {} 2022-09-03T05:46:51.246 DEBUG:boto:Token: None 2022-09-03T05:46:51.246 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/ 2022-09-03T05:46:51.247 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk= 2022-09-03T05:46:51.247 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk=', 'Content-Length': '0'} 2022-09-03T05:46:51.251 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006a5458f5e5e94e6b-006312ea4b-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.252 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.252 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.253 DEBUG:boto:Method: HEAD 2022-09-03T05:46:51.253 DEBUG:boto:Path: /dsgvcy-41/ 2022-09-03T05:46:51.254 DEBUG:boto:Data: 2022-09-03T05:46:51.254 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.255 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:51.255 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.256 DEBUG:boto:Params: {} 2022-09-03T05:46:51.256 DEBUG:boto:Token: None 2022-09-03T05:46:51.257 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/ 2022-09-03T05:46:51.258 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk= 2022-09-03T05:46:51.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk=', 'Content-Length': '0'} 2022-09-03T05:46:51.261 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d6ced24e6d6a6304-006312ea4b-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.261 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:51.262 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.268 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.269 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.270 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.271 DEBUG:boto:Method: GET 2022-09-03T05:46:51.272 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.273 DEBUG:boto:Data: 2022-09-03T05:46:51.274 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.275 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:51.276 DEBUG:boto:Port: 8001 2022-09-03T05:46:51.277 DEBUG:boto:Params: {} 2022-09-03T05:46:51.277 DEBUG:boto:Token: None 2022-09-03T05:46:51.278 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.279 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.283 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008d64b3e0e72a7e9d-006312ea4b-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.285 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.286 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:51.287 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.288 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.288 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.289 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.289 DEBUG:boto:Method: GET 2022-09-03T05:46:51.290 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.291 DEBUG:boto:Data: 2022-09-03T05:46:51.291 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.292 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:51.292 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.293 DEBUG:boto:Params: {} 2022-09-03T05:46:51.293 DEBUG:boto:Token: None 2022-09-03T05:46:51.294 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.295 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.295 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.298 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dcf94d20f28ba1b0-006312ea4b-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.299 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.299 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.300 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.301 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.301 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.302 DEBUG:boto:Method: GET 2022-09-03T05:46:51.302 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.303 DEBUG:boto:Data: 2022-09-03T05:46:51.303 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.304 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:51.304 DEBUG:boto:Port: 8001 2022-09-03T05:46:51.305 DEBUG:boto:Params: {} 2022-09-03T05:46:51.305 DEBUG:boto:Token: None 2022-09-03T05:46:51.306 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.307 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.307 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.310 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c5e9045b72997071-006312ea4b-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.358 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.360 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.360 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.361 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.361 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.363 DEBUG:boto:Method: GET 2022-09-03T05:46:51.363 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.364 DEBUG:boto:Data: 2022-09-03T05:46:51.365 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.365 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:51.367 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.368 DEBUG:boto:Params: {} 2022-09-03T05:46:51.369 DEBUG:boto:Token: None 2022-09-03T05:46:51.370 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.371 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.375 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000895d355d67429df8-006312ea4b-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.376 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.377 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-41 zones={test-zone3, test-zone1} 2022-09-03T05:46:51.377 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-41 zones={test-zone1, test-zone2} 2022-09-03T05:46:51.378 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.378 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.379 DEBUG:boto:Method: HEAD 2022-09-03T05:46:51.379 DEBUG:boto:Path: /dsgvcy-41/ 2022-09-03T05:46:51.380 DEBUG:boto:Data: 2022-09-03T05:46:51.381 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.381 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:51.382 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.383 DEBUG:boto:Params: {} 2022-09-03T05:46:51.384 DEBUG:boto:Token: None 2022-09-03T05:46:51.384 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/ 2022-09-03T05:46:51.385 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk= 2022-09-03T05:46:51.386 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk=', 'Content-Length': '0'} 2022-09-03T05:46:51.390 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f1a2a669942fe0fc-006312ea4b-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.391 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.391 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.392 DEBUG:boto:Method: HEAD 2022-09-03T05:46:51.392 DEBUG:boto:Path: /dsgvcy-41/ 2022-09-03T05:46:51.393 DEBUG:boto:Data: 2022-09-03T05:46:51.393 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.394 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:51.394 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.395 DEBUG:boto:Params: {} 2022-09-03T05:46:51.395 DEBUG:boto:Token: None 2022-09-03T05:46:51.396 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/ 2022-09-03T05:46:51.397 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk= 2022-09-03T05:46:51.397 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk=', 'Content-Length': '0'} 2022-09-03T05:46:51.400 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ba56880a24a37edc-006312ea4b-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.401 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:51.402 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.402 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.403 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.403 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.404 DEBUG:boto:Method: GET 2022-09-03T05:46:51.404 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.405 DEBUG:boto:Data: 2022-09-03T05:46:51.405 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.406 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:51.406 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.407 DEBUG:boto:Params: {} 2022-09-03T05:46:51.408 DEBUG:boto:Token: None 2022-09-03T05:46:51.408 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.409 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.409 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.412 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002e2b99d62853ac83-006312ea4b-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.412 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.413 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:51.414 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.414 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.415 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.415 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.416 DEBUG:boto:Method: GET 2022-09-03T05:46:51.416 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.417 DEBUG:boto:Data: 2022-09-03T05:46:51.417 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.418 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:51.419 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.419 DEBUG:boto:Params: {} 2022-09-03T05:46:51.420 DEBUG:boto:Token: None 2022-09-03T05:46:51.420 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.421 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.421 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.424 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000009a9ab671dc18c55-006312ea4b-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.425 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.426 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.427 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.428 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.428 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.429 DEBUG:boto:Method: GET 2022-09-03T05:46:51.429 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.430 DEBUG:boto:Data: 2022-09-03T05:46:51.431 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.432 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:51.433 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.434 DEBUG:boto:Params: {} 2022-09-03T05:46:51.435 DEBUG:boto:Token: None 2022-09-03T05:46:51.436 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.437 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.437 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.440 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000075bc28bcd13e3ea4-006312ea4b-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.442 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.443 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.444 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.445 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.445 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.446 DEBUG:boto:Method: GET 2022-09-03T05:46:51.447 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.447 DEBUG:boto:Data: 2022-09-03T05:46:51.448 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.448 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:51.449 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.449 DEBUG:boto:Params: {} 2022-09-03T05:46:51.450 DEBUG:boto:Token: None 2022-09-03T05:46:51.451 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.451 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.455 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000059692655a704792b-006312ea4b-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.456 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.456 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-41 zones={test-zone1, test-zone2} 2022-09-03T05:46:51.458 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-41 zones={test-zone3, test-zone2} 2022-09-03T05:46:51.459 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.459 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.460 DEBUG:boto:Method: HEAD 2022-09-03T05:46:51.460 DEBUG:boto:Path: /dsgvcy-41/ 2022-09-03T05:46:51.461 DEBUG:boto:Data: 2022-09-03T05:46:51.461 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.462 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:51.462 DEBUG:boto:Port: 8001 2022-09-03T05:46:51.463 DEBUG:boto:Params: {} 2022-09-03T05:46:51.463 DEBUG:boto:Token: None 2022-09-03T05:46:51.464 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/ 2022-09-03T05:46:51.464 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk= 2022-09-03T05:46:51.465 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk=', 'Content-Length': '0'} 2022-09-03T05:46:51.468 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009e1e6143c0905763-006312ea4b-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.469 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.469 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.470 DEBUG:boto:Method: HEAD 2022-09-03T05:46:51.471 DEBUG:boto:Path: /dsgvcy-41/ 2022-09-03T05:46:51.472 DEBUG:boto:Data: 2022-09-03T05:46:51.472 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.473 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:51.474 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.475 DEBUG:boto:Params: {} 2022-09-03T05:46:51.476 DEBUG:boto:Token: None 2022-09-03T05:46:51.477 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/ 2022-09-03T05:46:51.478 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk= 2022-09-03T05:46:51.478 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk=', 'Content-Length': '0'} 2022-09-03T05:46:51.482 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000dde5e0c1c679b22d-006312ea4b-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.483 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:51.483 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.484 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.484 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.485 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.485 DEBUG:boto:Method: GET 2022-09-03T05:46:51.486 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.486 DEBUG:boto:Data: 2022-09-03T05:46:51.487 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.487 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:51.488 DEBUG:boto:Port: 8001 2022-09-03T05:46:51.488 DEBUG:boto:Params: {} 2022-09-03T05:46:51.489 DEBUG:boto:Token: None 2022-09-03T05:46:51.490 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.490 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.491 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.493 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000041ecf7e0d158d29c-006312ea4b-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.494 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.495 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:51.496 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.496 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.497 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.497 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.498 DEBUG:boto:Method: GET 2022-09-03T05:46:51.498 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.499 DEBUG:boto:Data: 2022-09-03T05:46:51.499 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.500 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:51.500 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.501 DEBUG:boto:Params: {} 2022-09-03T05:46:51.501 DEBUG:boto:Token: None 2022-09-03T05:46:51.502 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.503 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.503 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.506 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000009d75df28d548b57-006312ea4b-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.554 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.556 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.557 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.558 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.558 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.559 DEBUG:boto:Method: GET 2022-09-03T05:46:51.563 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.564 DEBUG:boto:Data: 2022-09-03T05:46:51.565 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.566 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:51.567 DEBUG:boto:Port: 8001 2022-09-03T05:46:51.567 DEBUG:boto:Params: {} 2022-09-03T05:46:51.568 DEBUG:boto:Token: None 2022-09-03T05:46:51.569 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.571 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.571 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.574 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002dc8d8bc02f78c9a-006312ea4b-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.575 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.577 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.578 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.579 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.580 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.580 DEBUG:boto:Method: GET 2022-09-03T05:46:51.581 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.582 DEBUG:boto:Data: 2022-09-03T05:46:51.583 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.583 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:51.584 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.584 DEBUG:boto:Params: {} 2022-09-03T05:46:51.585 DEBUG:boto:Token: None 2022-09-03T05:46:51.586 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.586 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.592 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b82d41c0e5bd8ee5-006312ea4b-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.593 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.594 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-41 zones={test-zone3, test-zone2} 2022-09-03T05:46:51.594 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-41 zones={test-zone1, test-zone3} 2022-09-03T05:46:51.595 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.596 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.597 DEBUG:boto:Method: HEAD 2022-09-03T05:46:51.598 DEBUG:boto:Path: /dsgvcy-41/ 2022-09-03T05:46:51.599 DEBUG:boto:Data: 2022-09-03T05:46:51.600 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.600 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:51.601 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.601 DEBUG:boto:Params: {} 2022-09-03T05:46:51.602 DEBUG:boto:Token: None 2022-09-03T05:46:51.603 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/ 2022-09-03T05:46:51.603 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk= 2022-09-03T05:46:51.604 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk=', 'Content-Length': '0'} 2022-09-03T05:46:51.607 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000dda6b99ce330723a-006312ea4b-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.608 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.608 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.609 DEBUG:boto:Method: HEAD 2022-09-03T05:46:51.610 DEBUG:boto:Path: /dsgvcy-41/ 2022-09-03T05:46:51.611 DEBUG:boto:Data: 2022-09-03T05:46:51.612 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.612 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:51.613 DEBUG:boto:Port: 8001 2022-09-03T05:46:51.614 DEBUG:boto:Params: {} 2022-09-03T05:46:51.615 DEBUG:boto:Token: None 2022-09-03T05:46:51.616 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/ 2022-09-03T05:46:51.617 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk= 2022-09-03T05:46:51.618 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk=', 'Content-Length': '0'} 2022-09-03T05:46:51.622 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f6a8cebaae994233-006312ea4b-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.622 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:51.623 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.624 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.624 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.625 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.625 DEBUG:boto:Method: GET 2022-09-03T05:46:51.626 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.626 DEBUG:boto:Data: 2022-09-03T05:46:51.627 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.628 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:51.628 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.629 DEBUG:boto:Params: {} 2022-09-03T05:46:51.630 DEBUG:boto:Token: None 2022-09-03T05:46:51.631 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.631 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.635 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bf9423ba2f1d951c-006312ea4b-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.682 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.683 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:51.684 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.684 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.685 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.685 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.686 DEBUG:boto:Method: GET 2022-09-03T05:46:51.686 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.687 DEBUG:boto:Data: 2022-09-03T05:46:51.687 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.688 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:51.688 DEBUG:boto:Port: 8001 2022-09-03T05:46:51.689 DEBUG:boto:Params: {} 2022-09-03T05:46:51.689 DEBUG:boto:Token: None 2022-09-03T05:46:51.690 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.691 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.695 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002948deaeb9045795-006312ea4b-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.696 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.697 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.699 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.699 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.700 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.701 DEBUG:boto:Method: GET 2022-09-03T05:46:51.702 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.703 DEBUG:boto:Data: 2022-09-03T05:46:51.704 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.705 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:51.706 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.707 DEBUG:boto:Params: {} 2022-09-03T05:46:51.715 DEBUG:boto:Token: None 2022-09-03T05:46:51.716 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.717 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.718 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.721 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009fb7ccb6b8e54145-006312ea4b-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.722 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.723 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.723 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.724 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.725 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.725 DEBUG:boto:Method: GET 2022-09-03T05:46:51.726 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.727 DEBUG:boto:Data: 2022-09-03T05:46:51.727 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.728 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:51.728 DEBUG:boto:Port: 8001 2022-09-03T05:46:51.729 DEBUG:boto:Params: {} 2022-09-03T05:46:51.730 DEBUG:boto:Token: None 2022-09-03T05:46:51.730 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.731 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.734 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000abbe9a872e2e7d60-006312ea4b-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.735 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.736 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-41 zones={test-zone1, test-zone3} 2022-09-03T05:46:51.736 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-41 zones={test-zone2, test-zone3} 2022-09-03T05:46:51.737 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.737 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.738 DEBUG:boto:Method: HEAD 2022-09-03T05:46:51.738 DEBUG:boto:Path: /dsgvcy-41/ 2022-09-03T05:46:51.739 DEBUG:boto:Data: 2022-09-03T05:46:51.739 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.740 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:51.740 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.741 DEBUG:boto:Params: {} 2022-09-03T05:46:51.741 DEBUG:boto:Token: None 2022-09-03T05:46:51.742 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/ 2022-09-03T05:46:51.743 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk= 2022-09-03T05:46:51.743 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk=', 'Content-Length': '0'} 2022-09-03T05:46:51.746 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001bc161097a8e475d-006312ea4b-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.747 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.747 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.748 DEBUG:boto:Method: HEAD 2022-09-03T05:46:51.749 DEBUG:boto:Path: /dsgvcy-41/ 2022-09-03T05:46:51.749 DEBUG:boto:Data: 2022-09-03T05:46:51.750 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.750 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:51.751 DEBUG:boto:Port: 8001 2022-09-03T05:46:51.751 DEBUG:boto:Params: {} 2022-09-03T05:46:51.752 DEBUG:boto:Token: None 2022-09-03T05:46:51.752 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/ 2022-09-03T05:46:51.753 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk= 2022-09-03T05:46:51.754 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uQa2SeWQUfYegpkvdCDzQByZBDk=', 'Content-Length': '0'} 2022-09-03T05:46:51.757 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fe6443fbd7c971eb-006312ea4b-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.758 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:51.758 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.759 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.759 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.760 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.760 DEBUG:boto:Method: GET 2022-09-03T05:46:51.761 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.761 DEBUG:boto:Data: 2022-09-03T05:46:51.762 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.762 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:51.763 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.763 DEBUG:boto:Params: {} 2022-09-03T05:46:51.764 DEBUG:boto:Token: None 2022-09-03T05:46:51.765 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.765 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.766 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.768 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eff59536085b8be1-006312ea4b-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.769 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.770 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:51.770 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.771 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.772 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.772 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.773 DEBUG:boto:Method: GET 2022-09-03T05:46:51.773 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.774 DEBUG:boto:Data: 2022-09-03T05:46:51.774 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.775 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:51.775 DEBUG:boto:Port: 8001 2022-09-03T05:46:51.776 DEBUG:boto:Params: {} 2022-09-03T05:46:51.776 DEBUG:boto:Token: None 2022-09-03T05:46:51.777 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.777 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.778 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.781 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000209e8fbe11646bad-006312ea4b-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.782 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.782 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.783 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.784 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.784 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.785 DEBUG:boto:Method: GET 2022-09-03T05:46:51.785 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.786 DEBUG:boto:Data: 2022-09-03T05:46:51.786 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.787 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:51.787 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.788 DEBUG:boto:Params: {} 2022-09-03T05:46:51.788 DEBUG:boto:Token: None 2022-09-03T05:46:51.789 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.789 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.790 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.793 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000de0d63d509f89079-006312ea4b-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.794 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.795 DEBUG:boto:path=/dsgvcy-41/ 2022-09-03T05:46:51.795 DEBUG:boto:auth_path=/dsgvcy-41/ 2022-09-03T05:46:51.796 DEBUG:boto:path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.796 DEBUG:boto:auth_path=/dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.797 DEBUG:boto:Method: GET 2022-09-03T05:46:51.797 DEBUG:boto:Path: /dsgvcy-41/?versions&max-keys=999 2022-09-03T05:46:51.798 DEBUG:boto:Data: 2022-09-03T05:46:51.798 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.799 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:51.799 DEBUG:boto:Port: 8001 2022-09-03T05:46:51.800 DEBUG:boto:Params: {} 2022-09-03T05:46:51.801 DEBUG:boto:Token: None 2022-09-03T05:46:51.801 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-41/?versions 2022-09-03T05:46:51.802 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8= 2022-09-03T05:46:51.802 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SafzH2LEqxofbkdLsq09MQBMSt8=', 'Content-Length': '0'} 2022-09-03T05:46:51.805 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000647e6abb3d2a3a60-006312ea4b-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.806 DEBUG:boto:b'dsgvcy-41999false' 2022-09-03T05:46:51.807 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-41 zones={test-zone2, test-zone3} 2022-09-03T05:46:51.807 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-42 zones={test-zone2, test-zone1} 2022-09-03T05:46:51.808 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:51.808 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:51.809 DEBUG:boto:Method: HEAD 2022-09-03T05:46:51.809 DEBUG:boto:Path: /dsgvcy-42/ 2022-09-03T05:46:51.810 DEBUG:boto:Data: 2022-09-03T05:46:51.810 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.811 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:51.811 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.812 DEBUG:boto:Params: {} 2022-09-03T05:46:51.812 DEBUG:boto:Token: None 2022-09-03T05:46:51.813 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-42/ 2022-09-03T05:46:51.813 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tmrKHuHW5aQm3JKj6fLJXLTyBFA= 2022-09-03T05:46:51.814 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tmrKHuHW5aQm3JKj6fLJXLTyBFA=', 'Content-Length': '0'} 2022-09-03T05:46:51.816 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000060a9b527c8c33f4f-006312ea4b-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.817 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:51.818 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:51.818 DEBUG:boto:Method: HEAD 2022-09-03T05:46:51.819 DEBUG:boto:Path: /dsgvcy-42/ 2022-09-03T05:46:51.819 DEBUG:boto:Data: 2022-09-03T05:46:51.820 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.820 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:51.821 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.821 DEBUG:boto:Params: {} 2022-09-03T05:46:51.822 DEBUG:boto:Token: None 2022-09-03T05:46:51.822 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-42/ 2022-09-03T05:46:51.823 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tmrKHuHW5aQm3JKj6fLJXLTyBFA= 2022-09-03T05:46:51.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tmrKHuHW5aQm3JKj6fLJXLTyBFA=', 'Content-Length': '0'} 2022-09-03T05:46:51.826 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000079078bc8d8b8077d-006312ea4b-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.827 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:51.828 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:51.828 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:51.829 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:51.829 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:51.830 DEBUG:boto:Method: GET 2022-09-03T05:46:51.831 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:51.831 DEBUG:boto:Data: 2022-09-03T05:46:51.832 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.832 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:51.833 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.833 DEBUG:boto:Params: {} 2022-09-03T05:46:51.834 DEBUG:boto:Token: None 2022-09-03T05:46:51.834 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-42/?versions 2022-09-03T05:46:51.835 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2HEbVJ0QOQv28BHWy32hB48umWQ= 2022-09-03T05:46:51.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2HEbVJ0QOQv28BHWy32hB48umWQ=', 'Content-Length': '0'} 2022-09-03T05:46:51.838 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dd7432127939647d-006312ea4b-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.840 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:51.841 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:51.842 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:51.843 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:51.844 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:51.845 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:51.846 DEBUG:boto:Method: GET 2022-09-03T05:46:51.847 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:51.847 DEBUG:boto:Data: 2022-09-03T05:46:51.848 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.849 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:51.850 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.851 DEBUG:boto:Params: {} 2022-09-03T05:46:51.852 DEBUG:boto:Token: None 2022-09-03T05:46:51.853 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-42/?versions 2022-09-03T05:46:51.854 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2HEbVJ0QOQv28BHWy32hB48umWQ= 2022-09-03T05:46:51.855 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2HEbVJ0QOQv28BHWy32hB48umWQ=', 'Content-Length': '0'} 2022-09-03T05:46:51.859 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000b7b3400dd39d8b7-006312ea4b-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.860 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:51.862 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:51.863 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:51.864 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:51.864 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:51.865 DEBUG:boto:Method: GET 2022-09-03T05:46:51.866 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:51.867 DEBUG:boto:Data: 2022-09-03T05:46:51.868 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.869 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:51.870 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.871 DEBUG:boto:Params: {} 2022-09-03T05:46:51.872 DEBUG:boto:Token: None 2022-09-03T05:46:51.873 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-42/?versions 2022-09-03T05:46:51.874 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2HEbVJ0QOQv28BHWy32hB48umWQ= 2022-09-03T05:46:51.875 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2HEbVJ0QOQv28BHWy32hB48umWQ=', 'Content-Length': '0'} 2022-09-03T05:46:51.879 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fba77597ef989a34-006312ea4b-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.880 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:51.881 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:51.882 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:51.883 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:51.884 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:51.884 DEBUG:boto:Method: GET 2022-09-03T05:46:51.885 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:51.885 DEBUG:boto:Data: 2022-09-03T05:46:51.886 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.886 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:51.887 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.887 DEBUG:boto:Params: {} 2022-09-03T05:46:51.888 DEBUG:boto:Token: None 2022-09-03T05:46:51.888 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-42/?versions 2022-09-03T05:46:51.889 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2HEbVJ0QOQv28BHWy32hB48umWQ= 2022-09-03T05:46:51.889 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2HEbVJ0QOQv28BHWy32hB48umWQ=', 'Content-Length': '0'} 2022-09-03T05:46:51.892 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000047801efb3ae63f6-006312ea4b-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.935 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:51.936 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-42 zones={test-zone2, test-zone1} 2022-09-03T05:46:51.936 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-42 zones={test-zone3, test-zone1} 2022-09-03T05:46:51.937 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:51.938 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:51.939 DEBUG:boto:Method: HEAD 2022-09-03T05:46:51.940 DEBUG:boto:Path: /dsgvcy-42/ 2022-09-03T05:46:51.940 DEBUG:boto:Data: 2022-09-03T05:46:51.941 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.942 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:51.943 DEBUG:boto:Port: 8001 2022-09-03T05:46:51.944 DEBUG:boto:Params: {} 2022-09-03T05:46:51.945 DEBUG:boto:Token: None 2022-09-03T05:46:51.946 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-42/ 2022-09-03T05:46:51.948 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tmrKHuHW5aQm3JKj6fLJXLTyBFA= 2022-09-03T05:46:51.949 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tmrKHuHW5aQm3JKj6fLJXLTyBFA=', 'Content-Length': '0'} 2022-09-03T05:46:51.955 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000781231540482c6a-006312ea4b-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.956 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:51.957 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:51.958 DEBUG:boto:Method: HEAD 2022-09-03T05:46:51.958 DEBUG:boto:Path: /dsgvcy-42/ 2022-09-03T05:46:51.959 DEBUG:boto:Data: 2022-09-03T05:46:51.959 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.960 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:51.960 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.961 DEBUG:boto:Params: {} 2022-09-03T05:46:51.962 DEBUG:boto:Token: None 2022-09-03T05:46:51.962 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-42/ 2022-09-03T05:46:51.963 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tmrKHuHW5aQm3JKj6fLJXLTyBFA= 2022-09-03T05:46:51.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tmrKHuHW5aQm3JKj6fLJXLTyBFA=', 'Content-Length': '0'} 2022-09-03T05:46:51.966 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005d1b7bef7a5ec7d4-006312ea4b-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.967 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:51.968 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:51.968 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:51.969 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:51.969 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:51.970 DEBUG:boto:Method: GET 2022-09-03T05:46:51.971 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:51.974 DEBUG:boto:Data: 2022-09-03T05:46:51.975 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.975 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:51.976 DEBUG:boto:Port: 8001 2022-09-03T05:46:51.977 DEBUG:boto:Params: {} 2022-09-03T05:46:51.978 DEBUG:boto:Token: None 2022-09-03T05:46:51.980 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-42/?versions 2022-09-03T05:46:51.981 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2HEbVJ0QOQv28BHWy32hB48umWQ= 2022-09-03T05:46:51.982 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2HEbVJ0QOQv28BHWy32hB48umWQ=', 'Content-Length': '0'} 2022-09-03T05:46:51.985 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ddb4e130a5889f11-006312ea4b-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:51.986 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:51.987 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:51.988 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:51.989 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:51.990 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:51.991 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:51.992 DEBUG:boto:Method: GET 2022-09-03T05:46:51.993 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:51.993 DEBUG:boto:Data: 2022-09-03T05:46:51.994 DEBUG:boto:Headers: {} 2022-09-03T05:46:51.995 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:51.996 DEBUG:boto:Port: 8000 2022-09-03T05:46:51.996 DEBUG:boto:Params: {} 2022-09-03T05:46:51.997 DEBUG:boto:Token: None 2022-09-03T05:46:51.998 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:51 GMT /dsgvcy-42/?versions 2022-09-03T05:46:51.998 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2HEbVJ0QOQv28BHWy32hB48umWQ= 2022-09-03T05:46:51.999 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2HEbVJ0QOQv28BHWy32hB48umWQ=', 'Content-Length': '0'} 2022-09-03T05:46:52.001 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cb8d27f112413ff1-006312ea4c-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.002 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:52.003 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.003 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.004 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.006 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.007 DEBUG:boto:Method: GET 2022-09-03T05:46:52.008 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.008 DEBUG:boto:Data: 2022-09-03T05:46:52.009 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.009 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:52.010 DEBUG:boto:Port: 8001 2022-09-03T05:46:52.010 DEBUG:boto:Params: {} 2022-09-03T05:46:52.011 DEBUG:boto:Token: None 2022-09-03T05:46:52.012 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/?versions 2022-09-03T05:46:52.012 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0= 2022-09-03T05:46:52.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0=', 'Content-Length': '0'} 2022-09-03T05:46:52.016 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000722fa5cf63a0d9d9-006312ea4c-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.017 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:52.018 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.019 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.019 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.020 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.020 DEBUG:boto:Method: GET 2022-09-03T05:46:52.021 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.021 DEBUG:boto:Data: 2022-09-03T05:46:52.022 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.022 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:52.023 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.024 DEBUG:boto:Params: {} 2022-09-03T05:46:52.024 DEBUG:boto:Token: None 2022-09-03T05:46:52.025 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/?versions 2022-09-03T05:46:52.025 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0= 2022-09-03T05:46:52.026 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0=', 'Content-Length': '0'} 2022-09-03T05:46:52.029 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000440a2ec6e0e26543-006312ea4c-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.030 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:52.031 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-42 zones={test-zone3, test-zone1} 2022-09-03T05:46:52.031 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-42 zones={test-zone1, test-zone2} 2022-09-03T05:46:52.032 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.032 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.033 DEBUG:boto:Method: HEAD 2022-09-03T05:46:52.033 DEBUG:boto:Path: /dsgvcy-42/ 2022-09-03T05:46:52.034 DEBUG:boto:Data: 2022-09-03T05:46:52.034 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.035 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:52.035 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.036 DEBUG:boto:Params: {} 2022-09-03T05:46:52.036 DEBUG:boto:Token: None 2022-09-03T05:46:52.037 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/ 2022-09-03T05:46:52.038 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3e8fJRgykDJk3lSlb1M3xWN337Y= 2022-09-03T05:46:52.038 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3e8fJRgykDJk3lSlb1M3xWN337Y=', 'Content-Length': '0'} 2022-09-03T05:46:52.041 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000dfbd950eb0173953-006312ea4c-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.041 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.042 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.042 DEBUG:boto:Method: HEAD 2022-09-03T05:46:52.043 DEBUG:boto:Path: /dsgvcy-42/ 2022-09-03T05:46:52.043 DEBUG:boto:Data: 2022-09-03T05:46:52.044 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.044 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:52.045 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.045 DEBUG:boto:Params: {} 2022-09-03T05:46:52.046 DEBUG:boto:Token: None 2022-09-03T05:46:52.047 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/ 2022-09-03T05:46:52.047 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3e8fJRgykDJk3lSlb1M3xWN337Y= 2022-09-03T05:46:52.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3e8fJRgykDJk3lSlb1M3xWN337Y=', 'Content-Length': '0'} 2022-09-03T05:46:52.050 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ca7b6045a9f0c4c5-006312ea4c-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.051 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:52.052 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.052 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.053 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.053 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.054 DEBUG:boto:Method: GET 2022-09-03T05:46:52.054 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.055 DEBUG:boto:Data: 2022-09-03T05:46:52.055 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.056 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:52.056 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.057 DEBUG:boto:Params: {} 2022-09-03T05:46:52.057 DEBUG:boto:Token: None 2022-09-03T05:46:52.058 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/?versions 2022-09-03T05:46:52.058 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0= 2022-09-03T05:46:52.059 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0=', 'Content-Length': '0'} 2022-09-03T05:46:52.062 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000087fea0181b88e5f9-006312ea4c-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.063 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:52.063 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:52.064 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.064 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.065 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.065 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.066 DEBUG:boto:Method: GET 2022-09-03T05:46:52.067 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.067 DEBUG:boto:Data: 2022-09-03T05:46:52.068 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.068 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:52.069 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.069 DEBUG:boto:Params: {} 2022-09-03T05:46:52.070 DEBUG:boto:Token: None 2022-09-03T05:46:52.070 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/?versions 2022-09-03T05:46:52.071 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0= 2022-09-03T05:46:52.072 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0=', 'Content-Length': '0'} 2022-09-03T05:46:52.074 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000004c5d3559c541dc2-006312ea4c-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.118 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:52.119 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.120 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.120 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.121 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.121 DEBUG:boto:Method: GET 2022-09-03T05:46:52.122 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.122 DEBUG:boto:Data: 2022-09-03T05:46:52.123 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.123 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:52.124 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.124 DEBUG:boto:Params: {} 2022-09-03T05:46:52.125 DEBUG:boto:Token: None 2022-09-03T05:46:52.126 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/?versions 2022-09-03T05:46:52.126 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0= 2022-09-03T05:46:52.127 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0=', 'Content-Length': '0'} 2022-09-03T05:46:52.130 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000391465d7f003b648-006312ea4c-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.131 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:52.132 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.132 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.133 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.133 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.134 DEBUG:boto:Method: GET 2022-09-03T05:46:52.134 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.135 DEBUG:boto:Data: 2022-09-03T05:46:52.135 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.136 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:52.136 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.137 DEBUG:boto:Params: {} 2022-09-03T05:46:52.137 DEBUG:boto:Token: None 2022-09-03T05:46:52.138 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/?versions 2022-09-03T05:46:52.139 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0= 2022-09-03T05:46:52.139 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0=', 'Content-Length': '0'} 2022-09-03T05:46:52.142 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000028548d836cdbdf2d-006312ea4c-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.142 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:52.143 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-42 zones={test-zone1, test-zone2} 2022-09-03T05:46:52.144 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-42 zones={test-zone3, test-zone2} 2022-09-03T05:46:52.144 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.145 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.145 DEBUG:boto:Method: HEAD 2022-09-03T05:46:52.146 DEBUG:boto:Path: /dsgvcy-42/ 2022-09-03T05:46:52.146 DEBUG:boto:Data: 2022-09-03T05:46:52.147 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.147 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:52.148 DEBUG:boto:Port: 8001 2022-09-03T05:46:52.148 DEBUG:boto:Params: {} 2022-09-03T05:46:52.149 DEBUG:boto:Token: None 2022-09-03T05:46:52.150 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/ 2022-09-03T05:46:52.150 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3e8fJRgykDJk3lSlb1M3xWN337Y= 2022-09-03T05:46:52.151 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3e8fJRgykDJk3lSlb1M3xWN337Y=', 'Content-Length': '0'} 2022-09-03T05:46:52.154 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003ee2cdb8bcd5e647-006312ea4c-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.155 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.155 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.156 DEBUG:boto:Method: HEAD 2022-09-03T05:46:52.156 DEBUG:boto:Path: /dsgvcy-42/ 2022-09-03T05:46:52.157 DEBUG:boto:Data: 2022-09-03T05:46:52.157 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.158 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:52.158 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.159 DEBUG:boto:Params: {} 2022-09-03T05:46:52.160 DEBUG:boto:Token: None 2022-09-03T05:46:52.160 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/ 2022-09-03T05:46:52.161 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3e8fJRgykDJk3lSlb1M3xWN337Y= 2022-09-03T05:46:52.161 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3e8fJRgykDJk3lSlb1M3xWN337Y=', 'Content-Length': '0'} 2022-09-03T05:46:52.164 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007a03bdbe2f2f19f7-006312ea4c-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.165 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:52.166 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.167 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.167 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.168 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.168 DEBUG:boto:Method: GET 2022-09-03T05:46:52.169 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.169 DEBUG:boto:Data: 2022-09-03T05:46:52.170 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.171 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:52.171 DEBUG:boto:Port: 8001 2022-09-03T05:46:52.172 DEBUG:boto:Params: {} 2022-09-03T05:46:52.172 DEBUG:boto:Token: None 2022-09-03T05:46:52.173 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/?versions 2022-09-03T05:46:52.173 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0= 2022-09-03T05:46:52.174 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0=', 'Content-Length': '0'} 2022-09-03T05:46:52.179 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004dad9641c81249b4-006312ea4c-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.180 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:52.180 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:52.181 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.182 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.182 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.183 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.183 DEBUG:boto:Method: GET 2022-09-03T05:46:52.184 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.184 DEBUG:boto:Data: 2022-09-03T05:46:52.185 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.185 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:52.185 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.186 DEBUG:boto:Params: {} 2022-09-03T05:46:52.187 DEBUG:boto:Token: None 2022-09-03T05:46:52.187 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/?versions 2022-09-03T05:46:52.188 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0= 2022-09-03T05:46:52.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0=', 'Content-Length': '0'} 2022-09-03T05:46:52.191 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003ca4236c8593845c-006312ea4c-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.192 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:52.193 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.193 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.194 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.195 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.196 DEBUG:boto:Method: GET 2022-09-03T05:46:52.196 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.197 DEBUG:boto:Data: 2022-09-03T05:46:52.199 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.199 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:52.200 DEBUG:boto:Port: 8001 2022-09-03T05:46:52.200 DEBUG:boto:Params: {} 2022-09-03T05:46:52.201 DEBUG:boto:Token: None 2022-09-03T05:46:52.202 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/?versions 2022-09-03T05:46:52.203 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0= 2022-09-03T05:46:52.203 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0=', 'Content-Length': '0'} 2022-09-03T05:46:52.206 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002513e58066318053-006312ea4c-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.246 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:52.247 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.248 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.248 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.249 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.250 DEBUG:boto:Method: GET 2022-09-03T05:46:52.250 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.251 DEBUG:boto:Data: 2022-09-03T05:46:52.251 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.252 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:52.253 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.254 DEBUG:boto:Params: {} 2022-09-03T05:46:52.255 DEBUG:boto:Token: None 2022-09-03T05:46:52.256 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/?versions 2022-09-03T05:46:52.257 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0= 2022-09-03T05:46:52.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0=', 'Content-Length': '0'} 2022-09-03T05:46:52.262 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000735d852ab42f3e02-006312ea4c-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.264 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:52.265 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-42 zones={test-zone3, test-zone2} 2022-09-03T05:46:52.266 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-42 zones={test-zone1, test-zone3} 2022-09-03T05:46:52.267 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.267 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.268 DEBUG:boto:Method: HEAD 2022-09-03T05:46:52.269 DEBUG:boto:Path: /dsgvcy-42/ 2022-09-03T05:46:52.270 DEBUG:boto:Data: 2022-09-03T05:46:52.271 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.271 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:52.272 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.272 DEBUG:boto:Params: {} 2022-09-03T05:46:52.273 DEBUG:boto:Token: None 2022-09-03T05:46:52.274 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/ 2022-09-03T05:46:52.274 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3e8fJRgykDJk3lSlb1M3xWN337Y= 2022-09-03T05:46:52.275 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3e8fJRgykDJk3lSlb1M3xWN337Y=', 'Content-Length': '0'} 2022-09-03T05:46:52.278 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000adaca5558c854a8c-006312ea4c-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.279 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.281 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.281 DEBUG:boto:Method: HEAD 2022-09-03T05:46:52.282 DEBUG:boto:Path: /dsgvcy-42/ 2022-09-03T05:46:52.283 DEBUG:boto:Data: 2022-09-03T05:46:52.284 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.285 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:52.286 DEBUG:boto:Port: 8001 2022-09-03T05:46:52.287 DEBUG:boto:Params: {} 2022-09-03T05:46:52.288 DEBUG:boto:Token: None 2022-09-03T05:46:52.289 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/ 2022-09-03T05:46:52.291 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3e8fJRgykDJk3lSlb1M3xWN337Y= 2022-09-03T05:46:52.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3e8fJRgykDJk3lSlb1M3xWN337Y=', 'Content-Length': '0'} 2022-09-03T05:46:52.295 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002acf518ef6aa6f8b-006312ea4c-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.296 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:52.297 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.298 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.299 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.300 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.301 DEBUG:boto:Method: GET 2022-09-03T05:46:52.302 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.303 DEBUG:boto:Data: 2022-09-03T05:46:52.304 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.305 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:52.306 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.307 DEBUG:boto:Params: {} 2022-09-03T05:46:52.307 DEBUG:boto:Token: None 2022-09-03T05:46:52.308 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/?versions 2022-09-03T05:46:52.308 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0= 2022-09-03T05:46:52.309 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0=', 'Content-Length': '0'} 2022-09-03T05:46:52.312 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003545470ed963f146-006312ea4c-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.313 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:52.314 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:52.314 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.315 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.315 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.316 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.316 DEBUG:boto:Method: GET 2022-09-03T05:46:52.317 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.317 DEBUG:boto:Data: 2022-09-03T05:46:52.318 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.318 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:52.319 DEBUG:boto:Port: 8001 2022-09-03T05:46:52.319 DEBUG:boto:Params: {} 2022-09-03T05:46:52.320 DEBUG:boto:Token: None 2022-09-03T05:46:52.320 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/?versions 2022-09-03T05:46:52.321 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0= 2022-09-03T05:46:52.322 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0=', 'Content-Length': '0'} 2022-09-03T05:46:52.325 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d14711b75c9f6cb2-006312ea4c-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.326 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:52.327 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.327 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.328 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.328 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.329 DEBUG:boto:Method: GET 2022-09-03T05:46:52.329 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.330 DEBUG:boto:Data: 2022-09-03T05:46:52.330 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.331 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:52.331 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.332 DEBUG:boto:Params: {} 2022-09-03T05:46:52.333 DEBUG:boto:Token: None 2022-09-03T05:46:52.333 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/?versions 2022-09-03T05:46:52.334 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0= 2022-09-03T05:46:52.334 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0=', 'Content-Length': '0'} 2022-09-03T05:46:52.337 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001b9f6a908f9d1d40-006312ea4c-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.338 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:52.339 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.339 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.340 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.340 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.341 DEBUG:boto:Method: GET 2022-09-03T05:46:52.341 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.342 DEBUG:boto:Data: 2022-09-03T05:46:52.342 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.343 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:52.343 DEBUG:boto:Port: 8001 2022-09-03T05:46:52.344 DEBUG:boto:Params: {} 2022-09-03T05:46:52.345 DEBUG:boto:Token: None 2022-09-03T05:46:52.345 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/?versions 2022-09-03T05:46:52.346 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0= 2022-09-03T05:46:52.347 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0=', 'Content-Length': '0'} 2022-09-03T05:46:52.350 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000006acafa7cbcdc569-006312ea4c-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.352 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:52.353 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-42 zones={test-zone1, test-zone3} 2022-09-03T05:46:52.354 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-42 zones={test-zone2, test-zone3} 2022-09-03T05:46:52.355 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.356 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.357 DEBUG:boto:Method: HEAD 2022-09-03T05:46:52.358 DEBUG:boto:Path: /dsgvcy-42/ 2022-09-03T05:46:52.359 DEBUG:boto:Data: 2022-09-03T05:46:52.360 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.361 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:52.362 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.363 DEBUG:boto:Params: {} 2022-09-03T05:46:52.364 DEBUG:boto:Token: None 2022-09-03T05:46:52.365 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/ 2022-09-03T05:46:52.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3e8fJRgykDJk3lSlb1M3xWN337Y= 2022-09-03T05:46:52.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3e8fJRgykDJk3lSlb1M3xWN337Y=', 'Content-Length': '0'} 2022-09-03T05:46:52.370 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000afea7504d3aa2414-006312ea4c-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.371 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.372 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.373 DEBUG:boto:Method: HEAD 2022-09-03T05:46:52.373 DEBUG:boto:Path: /dsgvcy-42/ 2022-09-03T05:46:52.374 DEBUG:boto:Data: 2022-09-03T05:46:52.375 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.375 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:52.376 DEBUG:boto:Port: 8001 2022-09-03T05:46:52.376 DEBUG:boto:Params: {} 2022-09-03T05:46:52.377 DEBUG:boto:Token: None 2022-09-03T05:46:52.377 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/ 2022-09-03T05:46:52.378 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3e8fJRgykDJk3lSlb1M3xWN337Y= 2022-09-03T05:46:52.379 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3e8fJRgykDJk3lSlb1M3xWN337Y=', 'Content-Length': '0'} 2022-09-03T05:46:52.382 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d480a14d3fbcb35d-006312ea4c-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.382 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:52.383 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.383 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.384 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.384 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.385 DEBUG:boto:Method: GET 2022-09-03T05:46:52.386 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.387 DEBUG:boto:Data: 2022-09-03T05:46:52.387 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.388 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:52.388 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.389 DEBUG:boto:Params: {} 2022-09-03T05:46:52.389 DEBUG:boto:Token: None 2022-09-03T05:46:52.390 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/?versions 2022-09-03T05:46:52.391 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0= 2022-09-03T05:46:52.391 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0=', 'Content-Length': '0'} 2022-09-03T05:46:52.394 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000176b4bd22cd0fc05-006312ea4c-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.395 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:52.395 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:52.396 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.396 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.397 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.397 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.398 DEBUG:boto:Method: GET 2022-09-03T05:46:52.399 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.399 DEBUG:boto:Data: 2022-09-03T05:46:52.400 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.400 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:52.401 DEBUG:boto:Port: 8001 2022-09-03T05:46:52.401 DEBUG:boto:Params: {} 2022-09-03T05:46:52.402 DEBUG:boto:Token: None 2022-09-03T05:46:52.402 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/?versions 2022-09-03T05:46:52.403 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0= 2022-09-03T05:46:52.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0=', 'Content-Length': '0'} 2022-09-03T05:46:52.406 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000bee4473dfe7736b-006312ea4c-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.407 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:52.408 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.408 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.409 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.409 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.410 DEBUG:boto:Method: GET 2022-09-03T05:46:52.410 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.411 DEBUG:boto:Data: 2022-09-03T05:46:52.411 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.412 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:52.412 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.413 DEBUG:boto:Params: {} 2022-09-03T05:46:52.413 DEBUG:boto:Token: None 2022-09-03T05:46:52.414 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/?versions 2022-09-03T05:46:52.415 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0= 2022-09-03T05:46:52.415 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0=', 'Content-Length': '0'} 2022-09-03T05:46:52.418 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004f138f115ddea744-006312ea4c-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.463 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:52.465 DEBUG:boto:path=/dsgvcy-42/ 2022-09-03T05:46:52.466 DEBUG:boto:auth_path=/dsgvcy-42/ 2022-09-03T05:46:52.467 DEBUG:boto:path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.468 DEBUG:boto:auth_path=/dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.469 DEBUG:boto:Method: GET 2022-09-03T05:46:52.470 DEBUG:boto:Path: /dsgvcy-42/?versions&max-keys=999 2022-09-03T05:46:52.471 DEBUG:boto:Data: 2022-09-03T05:46:52.472 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.473 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:52.474 DEBUG:boto:Port: 8001 2022-09-03T05:46:52.475 DEBUG:boto:Params: {} 2022-09-03T05:46:52.476 DEBUG:boto:Token: None 2022-09-03T05:46:52.477 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-42/?versions 2022-09-03T05:46:52.478 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0= 2022-09-03T05:46:52.478 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:po/sAKMqiGaVDnLzItKnLLVLOj0=', 'Content-Length': '0'} 2022-09-03T05:46:52.481 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000b2515f2b2f3bcad-006312ea4c-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.482 DEBUG:boto:b'dsgvcy-42999false' 2022-09-03T05:46:52.483 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-42 zones={test-zone2, test-zone3} 2022-09-03T05:46:52.483 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-43 zones={test-zone2, test-zone1} 2022-09-03T05:46:52.484 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.484 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.485 DEBUG:boto:Method: HEAD 2022-09-03T05:46:52.485 DEBUG:boto:Path: /dsgvcy-43/ 2022-09-03T05:46:52.486 DEBUG:boto:Data: 2022-09-03T05:46:52.488 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.489 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:52.490 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.490 DEBUG:boto:Params: {} 2022-09-03T05:46:52.491 DEBUG:boto:Token: None 2022-09-03T05:46:52.492 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/ 2022-09-03T05:46:52.492 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mv9Wv4SmLrGBc1tKA1iLmEZ2U84= 2022-09-03T05:46:52.493 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mv9Wv4SmLrGBc1tKA1iLmEZ2U84=', 'Content-Length': '0'} 2022-09-03T05:46:52.496 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000042584f775e32280b-006312ea4c-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.497 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.498 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.499 DEBUG:boto:Method: HEAD 2022-09-03T05:46:52.500 DEBUG:boto:Path: /dsgvcy-43/ 2022-09-03T05:46:52.501 DEBUG:boto:Data: 2022-09-03T05:46:52.505 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.506 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:52.507 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.507 DEBUG:boto:Params: {} 2022-09-03T05:46:52.508 DEBUG:boto:Token: None 2022-09-03T05:46:52.509 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/ 2022-09-03T05:46:52.510 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mv9Wv4SmLrGBc1tKA1iLmEZ2U84= 2022-09-03T05:46:52.517 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mv9Wv4SmLrGBc1tKA1iLmEZ2U84=', 'Content-Length': '0'} 2022-09-03T05:46:52.521 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c97a20eabc28e26f-006312ea4c-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.521 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:52.522 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.523 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.523 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.524 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.525 DEBUG:boto:Method: GET 2022-09-03T05:46:52.526 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.528 DEBUG:boto:Data: 2022-09-03T05:46:52.529 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.530 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:52.531 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.532 DEBUG:boto:Params: {} 2022-09-03T05:46:52.533 DEBUG:boto:Token: None 2022-09-03T05:46:52.534 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/?versions 2022-09-03T05:46:52.535 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w= 2022-09-03T05:46:52.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w=', 'Content-Length': '0'} 2022-09-03T05:46:52.540 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f858ec4dcaa4096c-006312ea4c-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.541 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:52.542 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:52.543 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.545 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.546 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.547 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.548 DEBUG:boto:Method: GET 2022-09-03T05:46:52.549 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.550 DEBUG:boto:Data: 2022-09-03T05:46:52.551 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.552 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:52.553 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.554 DEBUG:boto:Params: {} 2022-09-03T05:46:52.555 DEBUG:boto:Token: None 2022-09-03T05:46:52.556 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/?versions 2022-09-03T05:46:52.557 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w= 2022-09-03T05:46:52.558 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w=', 'Content-Length': '0'} 2022-09-03T05:46:52.561 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000007ee5fc145e7ba92-006312ea4c-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.562 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:52.563 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.563 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.564 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.564 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.565 DEBUG:boto:Method: GET 2022-09-03T05:46:52.566 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.566 DEBUG:boto:Data: 2022-09-03T05:46:52.567 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.567 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:52.568 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.568 DEBUG:boto:Params: {} 2022-09-03T05:46:52.569 DEBUG:boto:Token: None 2022-09-03T05:46:52.570 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/?versions 2022-09-03T05:46:52.570 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w= 2022-09-03T05:46:52.571 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w=', 'Content-Length': '0'} 2022-09-03T05:46:52.574 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000059205d2d530adb5-006312ea4c-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.614 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:52.616 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.617 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.618 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.619 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.620 DEBUG:boto:Method: GET 2022-09-03T05:46:52.621 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.622 DEBUG:boto:Data: 2022-09-03T05:46:52.623 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.624 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:52.625 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.626 DEBUG:boto:Params: {} 2022-09-03T05:46:52.627 DEBUG:boto:Token: None 2022-09-03T05:46:52.628 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/?versions 2022-09-03T05:46:52.629 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w= 2022-09-03T05:46:52.630 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w=', 'Content-Length': '0'} 2022-09-03T05:46:52.633 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eedec9c28da8a0e0-006312ea4c-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.634 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:52.635 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-43 zones={test-zone2, test-zone1} 2022-09-03T05:46:52.636 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-43 zones={test-zone3, test-zone1} 2022-09-03T05:46:52.636 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.637 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.637 DEBUG:boto:Method: HEAD 2022-09-03T05:46:52.638 DEBUG:boto:Path: /dsgvcy-43/ 2022-09-03T05:46:52.639 DEBUG:boto:Data: 2022-09-03T05:46:52.639 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.640 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:52.640 DEBUG:boto:Port: 8001 2022-09-03T05:46:52.641 DEBUG:boto:Params: {} 2022-09-03T05:46:52.642 DEBUG:boto:Token: None 2022-09-03T05:46:52.642 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/ 2022-09-03T05:46:52.643 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mv9Wv4SmLrGBc1tKA1iLmEZ2U84= 2022-09-03T05:46:52.644 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mv9Wv4SmLrGBc1tKA1iLmEZ2U84=', 'Content-Length': '0'} 2022-09-03T05:46:52.647 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000704ba02ca8d4cdca-006312ea4c-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.647 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.648 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.648 DEBUG:boto:Method: HEAD 2022-09-03T05:46:52.649 DEBUG:boto:Path: /dsgvcy-43/ 2022-09-03T05:46:52.649 DEBUG:boto:Data: 2022-09-03T05:46:52.650 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.651 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:52.651 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.652 DEBUG:boto:Params: {} 2022-09-03T05:46:52.652 DEBUG:boto:Token: None 2022-09-03T05:46:52.653 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/ 2022-09-03T05:46:52.653 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mv9Wv4SmLrGBc1tKA1iLmEZ2U84= 2022-09-03T05:46:52.654 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mv9Wv4SmLrGBc1tKA1iLmEZ2U84=', 'Content-Length': '0'} 2022-09-03T05:46:52.657 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c1291bbcdf16d09b-006312ea4c-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.658 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:52.658 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.659 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.660 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.661 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.661 DEBUG:boto:Method: GET 2022-09-03T05:46:52.662 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.662 DEBUG:boto:Data: 2022-09-03T05:46:52.663 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.664 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:52.665 DEBUG:boto:Port: 8001 2022-09-03T05:46:52.665 DEBUG:boto:Params: {} 2022-09-03T05:46:52.666 DEBUG:boto:Token: None 2022-09-03T05:46:52.667 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/?versions 2022-09-03T05:46:52.667 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w= 2022-09-03T05:46:52.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w=', 'Content-Length': '0'} 2022-09-03T05:46:52.672 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000099f646560730c053-006312ea4c-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.719 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:52.720 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:52.721 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.722 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.722 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.723 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.723 DEBUG:boto:Method: GET 2022-09-03T05:46:52.724 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.724 DEBUG:boto:Data: 2022-09-03T05:46:52.725 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.725 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:52.726 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.726 DEBUG:boto:Params: {} 2022-09-03T05:46:52.727 DEBUG:boto:Token: None 2022-09-03T05:46:52.728 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/?versions 2022-09-03T05:46:52.728 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w= 2022-09-03T05:46:52.729 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w=', 'Content-Length': '0'} 2022-09-03T05:46:52.731 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005ed44a610b6ecc8d-006312ea4c-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.735 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:52.737 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.738 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.739 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.739 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.740 DEBUG:boto:Method: GET 2022-09-03T05:46:52.740 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.741 DEBUG:boto:Data: 2022-09-03T05:46:52.741 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.742 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:52.743 DEBUG:boto:Port: 8001 2022-09-03T05:46:52.743 DEBUG:boto:Params: {} 2022-09-03T05:46:52.744 DEBUG:boto:Token: None 2022-09-03T05:46:52.745 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/?versions 2022-09-03T05:46:52.746 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w= 2022-09-03T05:46:52.747 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w=', 'Content-Length': '0'} 2022-09-03T05:46:52.750 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003c005ac3868d07da-006312ea4c-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.751 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:52.752 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.752 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.753 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.754 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.755 DEBUG:boto:Method: GET 2022-09-03T05:46:52.755 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.756 DEBUG:boto:Data: 2022-09-03T05:46:52.757 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.757 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:52.758 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.758 DEBUG:boto:Params: {} 2022-09-03T05:46:52.760 DEBUG:boto:Token: None 2022-09-03T05:46:52.760 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/?versions 2022-09-03T05:46:52.762 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w= 2022-09-03T05:46:52.763 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w=', 'Content-Length': '0'} 2022-09-03T05:46:52.766 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000080c6f625097a6554-006312ea4c-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.806 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:52.807 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-43 zones={test-zone3, test-zone1} 2022-09-03T05:46:52.808 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-43 zones={test-zone1, test-zone2} 2022-09-03T05:46:52.808 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.809 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.809 DEBUG:boto:Method: HEAD 2022-09-03T05:46:52.810 DEBUG:boto:Path: /dsgvcy-43/ 2022-09-03T05:46:52.810 DEBUG:boto:Data: 2022-09-03T05:46:52.811 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.811 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:52.812 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.812 DEBUG:boto:Params: {} 2022-09-03T05:46:52.813 DEBUG:boto:Token: None 2022-09-03T05:46:52.813 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/ 2022-09-03T05:46:52.814 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mv9Wv4SmLrGBc1tKA1iLmEZ2U84= 2022-09-03T05:46:52.814 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mv9Wv4SmLrGBc1tKA1iLmEZ2U84=', 'Content-Length': '0'} 2022-09-03T05:46:52.821 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001505bf50048f9456-006312ea4c-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.821 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.822 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.823 DEBUG:boto:Method: HEAD 2022-09-03T05:46:52.823 DEBUG:boto:Path: /dsgvcy-43/ 2022-09-03T05:46:52.824 DEBUG:boto:Data: 2022-09-03T05:46:52.824 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.825 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:52.826 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.827 DEBUG:boto:Params: {} 2022-09-03T05:46:52.827 DEBUG:boto:Token: None 2022-09-03T05:46:52.828 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/ 2022-09-03T05:46:52.828 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mv9Wv4SmLrGBc1tKA1iLmEZ2U84= 2022-09-03T05:46:52.830 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mv9Wv4SmLrGBc1tKA1iLmEZ2U84=', 'Content-Length': '0'} 2022-09-03T05:46:52.833 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000033ba5dd051e18820-006312ea4c-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.834 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:52.835 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.836 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.836 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.837 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.837 DEBUG:boto:Method: GET 2022-09-03T05:46:52.838 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.839 DEBUG:boto:Data: 2022-09-03T05:46:52.839 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.840 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:52.841 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.841 DEBUG:boto:Params: {} 2022-09-03T05:46:52.842 DEBUG:boto:Token: None 2022-09-03T05:46:52.843 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/?versions 2022-09-03T05:46:52.844 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w= 2022-09-03T05:46:52.845 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w=', 'Content-Length': '0'} 2022-09-03T05:46:52.849 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000063471ecc1752f90b-006312ea4c-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.850 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:52.851 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:52.852 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.853 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.854 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.855 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.856 DEBUG:boto:Method: GET 2022-09-03T05:46:52.857 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.858 DEBUG:boto:Data: 2022-09-03T05:46:52.859 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.860 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:52.861 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.861 DEBUG:boto:Params: {} 2022-09-03T05:46:52.862 DEBUG:boto:Token: None 2022-09-03T05:46:52.863 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/?versions 2022-09-03T05:46:52.864 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w= 2022-09-03T05:46:52.864 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w=', 'Content-Length': '0'} 2022-09-03T05:46:52.867 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003ecabb2964611a33-006312ea4c-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.867 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:52.868 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.869 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.869 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.870 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.871 DEBUG:boto:Method: GET 2022-09-03T05:46:52.871 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.872 DEBUG:boto:Data: 2022-09-03T05:46:52.872 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.873 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:52.873 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.874 DEBUG:boto:Params: {} 2022-09-03T05:46:52.874 DEBUG:boto:Token: None 2022-09-03T05:46:52.875 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/?versions 2022-09-03T05:46:52.876 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w= 2022-09-03T05:46:52.876 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w=', 'Content-Length': '0'} 2022-09-03T05:46:52.879 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f72225ca69395baa-006312ea4c-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.880 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:52.882 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.883 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.884 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.885 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.886 DEBUG:boto:Method: GET 2022-09-03T05:46:52.887 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.887 DEBUG:boto:Data: 2022-09-03T05:46:52.888 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.889 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:52.890 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.891 DEBUG:boto:Params: {} 2022-09-03T05:46:52.892 DEBUG:boto:Token: None 2022-09-03T05:46:52.893 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/?versions 2022-09-03T05:46:52.894 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w= 2022-09-03T05:46:52.895 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w=', 'Content-Length': '0'} 2022-09-03T05:46:52.898 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007d3788a21cfa6f28-006312ea4c-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.900 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:52.901 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-43 zones={test-zone1, test-zone2} 2022-09-03T05:46:52.902 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-43 zones={test-zone3, test-zone2} 2022-09-03T05:46:52.903 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.904 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.904 DEBUG:boto:Method: HEAD 2022-09-03T05:46:52.905 DEBUG:boto:Path: /dsgvcy-43/ 2022-09-03T05:46:52.905 DEBUG:boto:Data: 2022-09-03T05:46:52.906 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.906 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:52.907 DEBUG:boto:Port: 8001 2022-09-03T05:46:52.908 DEBUG:boto:Params: {} 2022-09-03T05:46:52.908 DEBUG:boto:Token: None 2022-09-03T05:46:52.909 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/ 2022-09-03T05:46:52.910 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mv9Wv4SmLrGBc1tKA1iLmEZ2U84= 2022-09-03T05:46:52.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mv9Wv4SmLrGBc1tKA1iLmEZ2U84=', 'Content-Length': '0'} 2022-09-03T05:46:52.913 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b61b53781f9f50ab-006312ea4c-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.914 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.915 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.916 DEBUG:boto:Method: HEAD 2022-09-03T05:46:52.916 DEBUG:boto:Path: /dsgvcy-43/ 2022-09-03T05:46:52.917 DEBUG:boto:Data: 2022-09-03T05:46:52.917 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.918 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:52.919 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.920 DEBUG:boto:Params: {} 2022-09-03T05:46:52.920 DEBUG:boto:Token: None 2022-09-03T05:46:52.921 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/ 2022-09-03T05:46:52.921 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mv9Wv4SmLrGBc1tKA1iLmEZ2U84= 2022-09-03T05:46:52.922 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mv9Wv4SmLrGBc1tKA1iLmEZ2U84=', 'Content-Length': '0'} 2022-09-03T05:46:52.924 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f19426aca4a13d4b-006312ea4c-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.925 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:52.926 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.926 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.927 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.927 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.928 DEBUG:boto:Method: GET 2022-09-03T05:46:52.929 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.930 DEBUG:boto:Data: 2022-09-03T05:46:52.930 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.931 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:52.932 DEBUG:boto:Port: 8001 2022-09-03T05:46:52.933 DEBUG:boto:Params: {} 2022-09-03T05:46:52.935 DEBUG:boto:Token: None 2022-09-03T05:46:52.936 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/?versions 2022-09-03T05:46:52.937 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w= 2022-09-03T05:46:52.938 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w=', 'Content-Length': '0'} 2022-09-03T05:46:52.942 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a72e75af19a94a42-006312ea4c-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.943 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:52.944 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:52.945 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.946 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.947 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.948 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.948 DEBUG:boto:Method: GET 2022-09-03T05:46:52.949 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.950 DEBUG:boto:Data: 2022-09-03T05:46:52.951 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.952 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:52.953 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.954 DEBUG:boto:Params: {} 2022-09-03T05:46:52.955 DEBUG:boto:Token: None 2022-09-03T05:46:52.955 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/?versions 2022-09-03T05:46:52.956 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w= 2022-09-03T05:46:52.957 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w=', 'Content-Length': '0'} 2022-09-03T05:46:52.960 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000ae341ee706e6742-006312ea4c-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.961 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:52.962 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.962 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.963 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.963 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.964 DEBUG:boto:Method: GET 2022-09-03T05:46:52.965 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.966 DEBUG:boto:Data: 2022-09-03T05:46:52.967 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.967 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:52.968 DEBUG:boto:Port: 8001 2022-09-03T05:46:52.969 DEBUG:boto:Params: {} 2022-09-03T05:46:52.970 DEBUG:boto:Token: None 2022-09-03T05:46:52.971 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/?versions 2022-09-03T05:46:52.972 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w= 2022-09-03T05:46:52.973 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w=', 'Content-Length': '0'} 2022-09-03T05:46:52.977 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004445aadae2927f48-006312ea4c-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.978 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:52.979 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.980 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.981 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.982 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.982 DEBUG:boto:Method: GET 2022-09-03T05:46:52.983 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:52.984 DEBUG:boto:Data: 2022-09-03T05:46:52.984 DEBUG:boto:Headers: {} 2022-09-03T05:46:52.985 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:52.986 DEBUG:boto:Port: 8000 2022-09-03T05:46:52.986 DEBUG:boto:Params: {} 2022-09-03T05:46:52.987 DEBUG:boto:Token: None 2022-09-03T05:46:52.988 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:52 GMT /dsgvcy-43/?versions 2022-09-03T05:46:52.988 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w= 2022-09-03T05:46:52.989 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rsSXdOzN3SCBWQZNDXXqDZAE76w=', 'Content-Length': '0'} 2022-09-03T05:46:52.992 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000d6c7e1565e1c222-006312ea4c-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:52.993 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:52.995 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-43 zones={test-zone3, test-zone2} 2022-09-03T05:46:52.995 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-43 zones={test-zone1, test-zone3} 2022-09-03T05:46:52.996 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:52.997 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:52.998 DEBUG:boto:Method: HEAD 2022-09-03T05:46:52.999 DEBUG:boto:Path: /dsgvcy-43/ 2022-09-03T05:46:53.000 DEBUG:boto:Data: 2022-09-03T05:46:53.000 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.001 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:53.001 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.002 DEBUG:boto:Params: {} 2022-09-03T05:46:53.003 DEBUG:boto:Token: None 2022-09-03T05:46:53.003 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-43/ 2022-09-03T05:46:53.004 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iByGwvXSdzWbJOlG/mYORW5DNNo= 2022-09-03T05:46:53.005 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iByGwvXSdzWbJOlG/mYORW5DNNo=', 'Content-Length': '0'} 2022-09-03T05:46:53.008 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006a93edbb016b1e9e-006312ea4d-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.009 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:53.009 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:53.010 DEBUG:boto:Method: HEAD 2022-09-03T05:46:53.011 DEBUG:boto:Path: /dsgvcy-43/ 2022-09-03T05:46:53.011 DEBUG:boto:Data: 2022-09-03T05:46:53.012 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.012 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:53.013 DEBUG:boto:Port: 8001 2022-09-03T05:46:53.013 DEBUG:boto:Params: {} 2022-09-03T05:46:53.014 DEBUG:boto:Token: None 2022-09-03T05:46:53.014 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-43/ 2022-09-03T05:46:53.015 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iByGwvXSdzWbJOlG/mYORW5DNNo= 2022-09-03T05:46:53.015 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iByGwvXSdzWbJOlG/mYORW5DNNo=', 'Content-Length': '0'} 2022-09-03T05:46:53.026 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005d25844d1f8d4d0b-006312ea4d-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.027 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:53.027 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:53.028 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:53.028 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.029 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.029 DEBUG:boto:Method: GET 2022-09-03T05:46:53.030 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.030 DEBUG:boto:Data: 2022-09-03T05:46:53.031 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.031 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:53.032 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.032 DEBUG:boto:Params: {} 2022-09-03T05:46:53.033 DEBUG:boto:Token: None 2022-09-03T05:46:53.033 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-43/?versions 2022-09-03T05:46:53.034 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XQvOPdDhpnPdA/9QF2bh8+/jChE= 2022-09-03T05:46:53.035 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XQvOPdDhpnPdA/9QF2bh8+/jChE=', 'Content-Length': '0'} 2022-09-03T05:46:53.038 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c91fe59d75bb56d1-006312ea4d-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.038 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:53.039 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:53.040 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:53.040 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:53.041 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.041 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.042 DEBUG:boto:Method: GET 2022-09-03T05:46:53.042 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.043 DEBUG:boto:Data: 2022-09-03T05:46:53.043 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.044 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:53.044 DEBUG:boto:Port: 8001 2022-09-03T05:46:53.045 DEBUG:boto:Params: {} 2022-09-03T05:46:53.045 DEBUG:boto:Token: None 2022-09-03T05:46:53.046 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-43/?versions 2022-09-03T05:46:53.046 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XQvOPdDhpnPdA/9QF2bh8+/jChE= 2022-09-03T05:46:53.047 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XQvOPdDhpnPdA/9QF2bh8+/jChE=', 'Content-Length': '0'} 2022-09-03T05:46:53.054 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000093b9e7f3557109f5-006312ea4d-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.098 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:53.100 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:53.101 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:53.101 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.102 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.103 DEBUG:boto:Method: GET 2022-09-03T05:46:53.104 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.104 DEBUG:boto:Data: 2022-09-03T05:46:53.105 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.105 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:53.106 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.107 DEBUG:boto:Params: {} 2022-09-03T05:46:53.107 DEBUG:boto:Token: None 2022-09-03T05:46:53.108 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-43/?versions 2022-09-03T05:46:53.109 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XQvOPdDhpnPdA/9QF2bh8+/jChE= 2022-09-03T05:46:53.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XQvOPdDhpnPdA/9QF2bh8+/jChE=', 'Content-Length': '0'} 2022-09-03T05:46:53.113 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fd4a891d5bb990f9-006312ea4d-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.114 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:53.115 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:53.116 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:53.117 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.117 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.118 DEBUG:boto:Method: GET 2022-09-03T05:46:53.119 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.120 DEBUG:boto:Data: 2022-09-03T05:46:53.120 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.121 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:53.121 DEBUG:boto:Port: 8001 2022-09-03T05:46:53.122 DEBUG:boto:Params: {} 2022-09-03T05:46:53.122 DEBUG:boto:Token: None 2022-09-03T05:46:53.123 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-43/?versions 2022-09-03T05:46:53.124 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XQvOPdDhpnPdA/9QF2bh8+/jChE= 2022-09-03T05:46:53.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XQvOPdDhpnPdA/9QF2bh8+/jChE=', 'Content-Length': '0'} 2022-09-03T05:46:53.127 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004c0bfc709da5de9e-006312ea4d-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.128 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:53.130 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-43 zones={test-zone1, test-zone3} 2022-09-03T05:46:53.130 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-43 zones={test-zone2, test-zone3} 2022-09-03T05:46:53.131 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:53.132 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:53.133 DEBUG:boto:Method: HEAD 2022-09-03T05:46:53.133 DEBUG:boto:Path: /dsgvcy-43/ 2022-09-03T05:46:53.133 DEBUG:boto:Data: 2022-09-03T05:46:53.134 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.134 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:53.135 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.135 DEBUG:boto:Params: {} 2022-09-03T05:46:53.136 DEBUG:boto:Token: None 2022-09-03T05:46:53.137 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-43/ 2022-09-03T05:46:53.137 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iByGwvXSdzWbJOlG/mYORW5DNNo= 2022-09-03T05:46:53.138 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iByGwvXSdzWbJOlG/mYORW5DNNo=', 'Content-Length': '0'} 2022-09-03T05:46:53.140 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f27bdb83f925cb04-006312ea4d-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.141 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:53.142 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:53.142 DEBUG:boto:Method: HEAD 2022-09-03T05:46:53.143 DEBUG:boto:Path: /dsgvcy-43/ 2022-09-03T05:46:53.143 DEBUG:boto:Data: 2022-09-03T05:46:53.144 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.144 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:53.144 DEBUG:boto:Port: 8001 2022-09-03T05:46:53.145 DEBUG:boto:Params: {} 2022-09-03T05:46:53.146 DEBUG:boto:Token: None 2022-09-03T05:46:53.146 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-43/ 2022-09-03T05:46:53.147 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iByGwvXSdzWbJOlG/mYORW5DNNo= 2022-09-03T05:46:53.147 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iByGwvXSdzWbJOlG/mYORW5DNNo=', 'Content-Length': '0'} 2022-09-03T05:46:53.150 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005100f46c675b868c-006312ea4d-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.151 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:53.152 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:53.153 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:53.154 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.155 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.155 DEBUG:boto:Method: GET 2022-09-03T05:46:53.157 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.157 DEBUG:boto:Data: 2022-09-03T05:46:53.158 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.159 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:53.160 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.161 DEBUG:boto:Params: {} 2022-09-03T05:46:53.162 DEBUG:boto:Token: None 2022-09-03T05:46:53.164 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-43/?versions 2022-09-03T05:46:53.165 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XQvOPdDhpnPdA/9QF2bh8+/jChE= 2022-09-03T05:46:53.166 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XQvOPdDhpnPdA/9QF2bh8+/jChE=', 'Content-Length': '0'} 2022-09-03T05:46:53.169 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f570a3f11677848e-006312ea4d-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.171 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:53.172 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:53.173 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:53.174 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:53.175 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.176 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.177 DEBUG:boto:Method: GET 2022-09-03T05:46:53.178 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.178 DEBUG:boto:Data: 2022-09-03T05:46:53.179 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.180 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:53.181 DEBUG:boto:Port: 8001 2022-09-03T05:46:53.182 DEBUG:boto:Params: {} 2022-09-03T05:46:53.183 DEBUG:boto:Token: None 2022-09-03T05:46:53.184 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-43/?versions 2022-09-03T05:46:53.186 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XQvOPdDhpnPdA/9QF2bh8+/jChE= 2022-09-03T05:46:53.187 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XQvOPdDhpnPdA/9QF2bh8+/jChE=', 'Content-Length': '0'} 2022-09-03T05:46:53.190 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d55ec106aa2d8aba-006312ea4d-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.191 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:53.192 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:53.192 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:53.193 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.193 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.194 DEBUG:boto:Method: GET 2022-09-03T05:46:53.195 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.195 DEBUG:boto:Data: 2022-09-03T05:46:53.196 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.197 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:53.198 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.199 DEBUG:boto:Params: {} 2022-09-03T05:46:53.199 DEBUG:boto:Token: None 2022-09-03T05:46:53.200 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-43/?versions 2022-09-03T05:46:53.201 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XQvOPdDhpnPdA/9QF2bh8+/jChE= 2022-09-03T05:46:53.201 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XQvOPdDhpnPdA/9QF2bh8+/jChE=', 'Content-Length': '0'} 2022-09-03T05:46:53.204 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001114b49f0869e621-006312ea4d-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.205 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:53.205 DEBUG:boto:path=/dsgvcy-43/ 2022-09-03T05:46:53.206 DEBUG:boto:auth_path=/dsgvcy-43/ 2022-09-03T05:46:53.206 DEBUG:boto:path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.207 DEBUG:boto:auth_path=/dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.208 DEBUG:boto:Method: GET 2022-09-03T05:46:53.208 DEBUG:boto:Path: /dsgvcy-43/?versions&max-keys=999 2022-09-03T05:46:53.209 DEBUG:boto:Data: 2022-09-03T05:46:53.209 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.210 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:53.210 DEBUG:boto:Port: 8001 2022-09-03T05:46:53.211 DEBUG:boto:Params: {} 2022-09-03T05:46:53.211 DEBUG:boto:Token: None 2022-09-03T05:46:53.212 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-43/?versions 2022-09-03T05:46:53.212 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XQvOPdDhpnPdA/9QF2bh8+/jChE= 2022-09-03T05:46:53.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XQvOPdDhpnPdA/9QF2bh8+/jChE=', 'Content-Length': '0'} 2022-09-03T05:46:53.216 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ecb690a858cfcf26-006312ea4d-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.258 DEBUG:boto:b'dsgvcy-43999false' 2022-09-03T05:46:53.259 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-43 zones={test-zone2, test-zone3} 2022-09-03T05:46:53.259 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-44 zones={test-zone2, test-zone1} 2022-09-03T05:46:53.260 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.260 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.261 DEBUG:boto:Method: HEAD 2022-09-03T05:46:53.261 DEBUG:boto:Path: /dsgvcy-44/ 2022-09-03T05:46:53.262 DEBUG:boto:Data: 2022-09-03T05:46:53.262 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.263 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:53.263 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.264 DEBUG:boto:Params: {} 2022-09-03T05:46:53.264 DEBUG:boto:Token: None 2022-09-03T05:46:53.265 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/ 2022-09-03T05:46:53.266 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg= 2022-09-03T05:46:53.266 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg=', 'Content-Length': '0'} 2022-09-03T05:46:53.269 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f7c39323b2d7c4af-006312ea4d-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.270 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.270 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.271 DEBUG:boto:Method: HEAD 2022-09-03T05:46:53.271 DEBUG:boto:Path: /dsgvcy-44/ 2022-09-03T05:46:53.272 DEBUG:boto:Data: 2022-09-03T05:46:53.272 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.273 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:53.273 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.274 DEBUG:boto:Params: {} 2022-09-03T05:46:53.274 DEBUG:boto:Token: None 2022-09-03T05:46:53.275 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/ 2022-09-03T05:46:53.275 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg= 2022-09-03T05:46:53.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg=', 'Content-Length': '0'} 2022-09-03T05:46:53.279 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ad155805743900d1-006312ea4d-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.279 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:53.280 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.281 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.281 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.282 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.282 DEBUG:boto:Method: GET 2022-09-03T05:46:53.283 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.283 DEBUG:boto:Data: 2022-09-03T05:46:53.284 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.284 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:53.285 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.285 DEBUG:boto:Params: {} 2022-09-03T05:46:53.286 DEBUG:boto:Token: None 2022-09-03T05:46:53.286 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.287 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.290 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000099c8446deeea7b14-006312ea4d-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.291 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.292 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:53.293 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.293 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.294 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.294 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.295 DEBUG:boto:Method: GET 2022-09-03T05:46:53.295 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.296 DEBUG:boto:Data: 2022-09-03T05:46:53.296 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.297 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:53.297 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.298 DEBUG:boto:Params: {} 2022-09-03T05:46:53.299 DEBUG:boto:Token: None 2022-09-03T05:46:53.299 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.300 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.303 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001a9bb055dc3d57cd-006312ea4d-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.304 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.304 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.305 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.305 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.306 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.307 DEBUG:boto:Method: GET 2022-09-03T05:46:53.307 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.308 DEBUG:boto:Data: 2022-09-03T05:46:53.308 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.309 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:53.310 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.310 DEBUG:boto:Params: {} 2022-09-03T05:46:53.311 DEBUG:boto:Token: None 2022-09-03T05:46:53.312 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.312 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.313 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.315 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000854a08abe9d89855-006312ea4d-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.358 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.360 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.360 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.361 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.362 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.363 DEBUG:boto:Method: GET 2022-09-03T05:46:53.364 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.365 DEBUG:boto:Data: 2022-09-03T05:46:53.366 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.367 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:53.368 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.369 DEBUG:boto:Params: {} 2022-09-03T05:46:53.370 DEBUG:boto:Token: None 2022-09-03T05:46:53.371 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.372 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.373 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.376 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000057a147a8dc4b9255-006312ea4d-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.377 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.378 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-44 zones={test-zone2, test-zone1} 2022-09-03T05:46:53.379 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-44 zones={test-zone3, test-zone1} 2022-09-03T05:46:53.380 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.381 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.382 DEBUG:boto:Method: HEAD 2022-09-03T05:46:53.382 DEBUG:boto:Path: /dsgvcy-44/ 2022-09-03T05:46:53.383 DEBUG:boto:Data: 2022-09-03T05:46:53.383 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.384 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:53.384 DEBUG:boto:Port: 8001 2022-09-03T05:46:53.385 DEBUG:boto:Params: {} 2022-09-03T05:46:53.386 DEBUG:boto:Token: None 2022-09-03T05:46:53.387 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/ 2022-09-03T05:46:53.388 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg= 2022-09-03T05:46:53.389 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg=', 'Content-Length': '0'} 2022-09-03T05:46:53.392 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a68633be4f47e6eb-006312ea4d-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.394 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.395 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.396 DEBUG:boto:Method: HEAD 2022-09-03T05:46:53.397 DEBUG:boto:Path: /dsgvcy-44/ 2022-09-03T05:46:53.398 DEBUG:boto:Data: 2022-09-03T05:46:53.399 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.400 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:53.400 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.401 DEBUG:boto:Params: {} 2022-09-03T05:46:53.402 DEBUG:boto:Token: None 2022-09-03T05:46:53.402 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/ 2022-09-03T05:46:53.403 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg= 2022-09-03T05:46:53.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg=', 'Content-Length': '0'} 2022-09-03T05:46:53.406 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000dc35852a8aa0b026-006312ea4d-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.408 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:53.409 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.410 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.411 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.412 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.412 DEBUG:boto:Method: GET 2022-09-03T05:46:53.413 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.414 DEBUG:boto:Data: 2022-09-03T05:46:53.415 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.416 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:53.416 DEBUG:boto:Port: 8001 2022-09-03T05:46:53.417 DEBUG:boto:Params: {} 2022-09-03T05:46:53.418 DEBUG:boto:Token: None 2022-09-03T05:46:53.419 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.420 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.421 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.425 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fc81ab68d8361c7e-006312ea4d-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.426 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.427 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:53.428 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.429 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.430 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.430 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.431 DEBUG:boto:Method: GET 2022-09-03T05:46:53.432 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.432 DEBUG:boto:Data: 2022-09-03T05:46:53.433 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.434 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:53.434 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.435 DEBUG:boto:Params: {} 2022-09-03T05:46:53.436 DEBUG:boto:Token: None 2022-09-03T05:46:53.436 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.437 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.438 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.441 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000e7badf1f9c36e75-006312ea4d-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.482 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.484 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.484 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.485 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.486 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.486 DEBUG:boto:Method: GET 2022-09-03T05:46:53.487 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.487 DEBUG:boto:Data: 2022-09-03T05:46:53.488 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.488 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:53.489 DEBUG:boto:Port: 8001 2022-09-03T05:46:53.489 DEBUG:boto:Params: {} 2022-09-03T05:46:53.490 DEBUG:boto:Token: None 2022-09-03T05:46:53.491 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.491 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.492 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.496 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000be3951acd15ff351-006312ea4d-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.497 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.499 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.500 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.501 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.502 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.503 DEBUG:boto:Method: GET 2022-09-03T05:46:53.503 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.504 DEBUG:boto:Data: 2022-09-03T05:46:53.504 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.505 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:53.505 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.506 DEBUG:boto:Params: {} 2022-09-03T05:46:53.507 DEBUG:boto:Token: None 2022-09-03T05:46:53.507 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.508 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.514 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000beef3a381bf48edf-006312ea4d-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.515 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.516 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-44 zones={test-zone3, test-zone1} 2022-09-03T05:46:53.516 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-44 zones={test-zone1, test-zone2} 2022-09-03T05:46:53.517 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.518 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.518 DEBUG:boto:Method: HEAD 2022-09-03T05:46:53.519 DEBUG:boto:Path: /dsgvcy-44/ 2022-09-03T05:46:53.520 DEBUG:boto:Data: 2022-09-03T05:46:53.520 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.521 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:53.522 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.522 DEBUG:boto:Params: {} 2022-09-03T05:46:53.524 DEBUG:boto:Token: None 2022-09-03T05:46:53.525 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/ 2022-09-03T05:46:53.526 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg= 2022-09-03T05:46:53.527 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg=', 'Content-Length': '0'} 2022-09-03T05:46:53.531 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000023a5f312395b2110-006312ea4d-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.531 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.532 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.533 DEBUG:boto:Method: HEAD 2022-09-03T05:46:53.533 DEBUG:boto:Path: /dsgvcy-44/ 2022-09-03T05:46:53.534 DEBUG:boto:Data: 2022-09-03T05:46:53.534 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.535 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:53.536 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.537 DEBUG:boto:Params: {} 2022-09-03T05:46:53.537 DEBUG:boto:Token: None 2022-09-03T05:46:53.538 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/ 2022-09-03T05:46:53.539 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg= 2022-09-03T05:46:53.539 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg=', 'Content-Length': '0'} 2022-09-03T05:46:53.542 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000049c9e49fa45e5c5-006312ea4d-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.543 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:53.543 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.544 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.545 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.545 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.546 DEBUG:boto:Method: GET 2022-09-03T05:46:53.546 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.547 DEBUG:boto:Data: 2022-09-03T05:46:53.547 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.548 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:53.549 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.549 DEBUG:boto:Params: {} 2022-09-03T05:46:53.550 DEBUG:boto:Token: None 2022-09-03T05:46:53.550 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.551 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.555 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001b6bf643ac3a596f-006312ea4d-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.598 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.599 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:53.600 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.601 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.602 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.603 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.604 DEBUG:boto:Method: GET 2022-09-03T05:46:53.605 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.606 DEBUG:boto:Data: 2022-09-03T05:46:53.606 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.607 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:53.609 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.609 DEBUG:boto:Params: {} 2022-09-03T05:46:53.612 DEBUG:boto:Token: None 2022-09-03T05:46:53.613 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.614 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.615 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.618 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000522041a4beebc388-006312ea4d-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.619 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.620 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.620 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.621 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.621 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.622 DEBUG:boto:Method: GET 2022-09-03T05:46:53.623 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.624 DEBUG:boto:Data: 2022-09-03T05:46:53.625 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.625 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:53.626 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.626 DEBUG:boto:Params: {} 2022-09-03T05:46:53.627 DEBUG:boto:Token: None 2022-09-03T05:46:53.628 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.629 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.629 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.632 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000050bd292d22c1476b-006312ea4d-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.633 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.633 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.634 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.635 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.636 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.637 DEBUG:boto:Method: GET 2022-09-03T05:46:53.638 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.639 DEBUG:boto:Data: 2022-09-03T05:46:53.640 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.641 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:53.642 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.642 DEBUG:boto:Params: {} 2022-09-03T05:46:53.643 DEBUG:boto:Token: None 2022-09-03T05:46:53.643 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.644 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.645 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.647 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000328f3a3e683588b9-006312ea4d-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.648 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.649 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-44 zones={test-zone1, test-zone2} 2022-09-03T05:46:53.649 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-44 zones={test-zone3, test-zone2} 2022-09-03T05:46:53.650 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.650 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.651 DEBUG:boto:Method: HEAD 2022-09-03T05:46:53.651 DEBUG:boto:Path: /dsgvcy-44/ 2022-09-03T05:46:53.652 DEBUG:boto:Data: 2022-09-03T05:46:53.652 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.653 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:53.654 DEBUG:boto:Port: 8001 2022-09-03T05:46:53.654 DEBUG:boto:Params: {} 2022-09-03T05:46:53.655 DEBUG:boto:Token: None 2022-09-03T05:46:53.655 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/ 2022-09-03T05:46:53.656 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg= 2022-09-03T05:46:53.657 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg=', 'Content-Length': '0'} 2022-09-03T05:46:53.659 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000387fb8f93093e1df-006312ea4d-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.660 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.661 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.661 DEBUG:boto:Method: HEAD 2022-09-03T05:46:53.662 DEBUG:boto:Path: /dsgvcy-44/ 2022-09-03T05:46:53.662 DEBUG:boto:Data: 2022-09-03T05:46:53.663 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.664 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:53.664 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.665 DEBUG:boto:Params: {} 2022-09-03T05:46:53.665 DEBUG:boto:Token: None 2022-09-03T05:46:53.666 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/ 2022-09-03T05:46:53.667 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg= 2022-09-03T05:46:53.667 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg=', 'Content-Length': '0'} 2022-09-03T05:46:53.670 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c200d7bf02a871b2-006312ea4d-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.671 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:53.671 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.672 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.672 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.673 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.674 DEBUG:boto:Method: GET 2022-09-03T05:46:53.675 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.675 DEBUG:boto:Data: 2022-09-03T05:46:53.676 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.676 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:53.677 DEBUG:boto:Port: 8001 2022-09-03T05:46:53.677 DEBUG:boto:Params: {} 2022-09-03T05:46:53.678 DEBUG:boto:Token: None 2022-09-03T05:46:53.679 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.679 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.682 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002e09ae3442f1a98c-006312ea4d-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.683 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.684 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:53.684 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.685 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.685 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.686 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.686 DEBUG:boto:Method: GET 2022-09-03T05:46:53.687 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.687 DEBUG:boto:Data: 2022-09-03T05:46:53.688 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.688 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:53.689 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.689 DEBUG:boto:Params: {} 2022-09-03T05:46:53.690 DEBUG:boto:Token: None 2022-09-03T05:46:53.691 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.691 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.694 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000614df467049e5c4-006312ea4d-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.695 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.696 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.697 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.697 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.698 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.698 DEBUG:boto:Method: GET 2022-09-03T05:46:53.699 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.699 DEBUG:boto:Data: 2022-09-03T05:46:53.700 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.700 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:53.701 DEBUG:boto:Port: 8001 2022-09-03T05:46:53.701 DEBUG:boto:Params: {} 2022-09-03T05:46:53.702 DEBUG:boto:Token: None 2022-09-03T05:46:53.703 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.703 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.707 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e2163ff5bd2cd781-006312ea4d-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.707 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.708 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.709 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.709 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.710 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.710 DEBUG:boto:Method: GET 2022-09-03T05:46:53.711 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.711 DEBUG:boto:Data: 2022-09-03T05:46:53.712 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.712 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:53.713 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.713 DEBUG:boto:Params: {} 2022-09-03T05:46:53.714 DEBUG:boto:Token: None 2022-09-03T05:46:53.714 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.715 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.718 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000a20e32741beea4f-006312ea4d-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.762 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.763 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-44 zones={test-zone3, test-zone2} 2022-09-03T05:46:53.763 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-44 zones={test-zone1, test-zone3} 2022-09-03T05:46:53.764 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.764 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.765 DEBUG:boto:Method: HEAD 2022-09-03T05:46:53.765 DEBUG:boto:Path: /dsgvcy-44/ 2022-09-03T05:46:53.766 DEBUG:boto:Data: 2022-09-03T05:46:53.766 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.767 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:53.767 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.768 DEBUG:boto:Params: {} 2022-09-03T05:46:53.770 DEBUG:boto:Token: None 2022-09-03T05:46:53.770 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/ 2022-09-03T05:46:53.771 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg= 2022-09-03T05:46:53.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg=', 'Content-Length': '0'} 2022-09-03T05:46:53.774 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000027d0e19ee5d64119-006312ea4d-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.775 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.776 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.776 DEBUG:boto:Method: HEAD 2022-09-03T05:46:53.777 DEBUG:boto:Path: /dsgvcy-44/ 2022-09-03T05:46:53.777 DEBUG:boto:Data: 2022-09-03T05:46:53.778 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.778 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:53.779 DEBUG:boto:Port: 8001 2022-09-03T05:46:53.779 DEBUG:boto:Params: {} 2022-09-03T05:46:53.780 DEBUG:boto:Token: None 2022-09-03T05:46:53.781 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/ 2022-09-03T05:46:53.781 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg= 2022-09-03T05:46:53.782 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg=', 'Content-Length': '0'} 2022-09-03T05:46:53.785 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007d24d0e00388c95b-006312ea4d-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.787 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:53.788 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.789 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.790 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.790 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.791 DEBUG:boto:Method: GET 2022-09-03T05:46:53.792 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.793 DEBUG:boto:Data: 2022-09-03T05:46:53.794 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.794 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:53.795 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.796 DEBUG:boto:Params: {} 2022-09-03T05:46:53.796 DEBUG:boto:Token: None 2022-09-03T05:46:53.797 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.798 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.798 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.801 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002f416f8724fd05ca-006312ea4d-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.802 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.802 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:53.803 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.804 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.805 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.805 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.806 DEBUG:boto:Method: GET 2022-09-03T05:46:53.807 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.808 DEBUG:boto:Data: 2022-09-03T05:46:53.808 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.809 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:53.809 DEBUG:boto:Port: 8001 2022-09-03T05:46:53.810 DEBUG:boto:Params: {} 2022-09-03T05:46:53.811 DEBUG:boto:Token: None 2022-09-03T05:46:53.812 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.812 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.813 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.815 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aebea5ad11e62b70-006312ea4d-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.816 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.817 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.818 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.818 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.819 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.819 DEBUG:boto:Method: GET 2022-09-03T05:46:53.820 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.820 DEBUG:boto:Data: 2022-09-03T05:46:53.821 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.821 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:53.822 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.822 DEBUG:boto:Params: {} 2022-09-03T05:46:53.823 DEBUG:boto:Token: None 2022-09-03T05:46:53.824 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.824 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.828 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000922084946140b1db-006312ea4d-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.828 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.829 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.830 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.830 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.831 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.831 DEBUG:boto:Method: GET 2022-09-03T05:46:53.832 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.833 DEBUG:boto:Data: 2022-09-03T05:46:53.833 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.834 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:53.835 DEBUG:boto:Port: 8001 2022-09-03T05:46:53.836 DEBUG:boto:Params: {} 2022-09-03T05:46:53.837 DEBUG:boto:Token: None 2022-09-03T05:46:53.838 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.839 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.843 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b58035251244130e-006312ea4d-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.886 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.888 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-44 zones={test-zone1, test-zone3} 2022-09-03T05:46:53.889 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-44 zones={test-zone2, test-zone3} 2022-09-03T05:46:53.889 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.890 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.891 DEBUG:boto:Method: HEAD 2022-09-03T05:46:53.891 DEBUG:boto:Path: /dsgvcy-44/ 2022-09-03T05:46:53.892 DEBUG:boto:Data: 2022-09-03T05:46:53.892 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.893 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:53.893 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.894 DEBUG:boto:Params: {} 2022-09-03T05:46:53.895 DEBUG:boto:Token: None 2022-09-03T05:46:53.895 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/ 2022-09-03T05:46:53.896 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg= 2022-09-03T05:46:53.897 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg=', 'Content-Length': '0'} 2022-09-03T05:46:53.900 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a8f8c7dfa633df9d-006312ea4d-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.901 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.902 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.903 DEBUG:boto:Method: HEAD 2022-09-03T05:46:53.904 DEBUG:boto:Path: /dsgvcy-44/ 2022-09-03T05:46:53.905 DEBUG:boto:Data: 2022-09-03T05:46:53.906 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.907 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:53.908 DEBUG:boto:Port: 8001 2022-09-03T05:46:53.909 DEBUG:boto:Params: {} 2022-09-03T05:46:53.910 DEBUG:boto:Token: None 2022-09-03T05:46:53.911 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/ 2022-09-03T05:46:53.912 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg= 2022-09-03T05:46:53.913 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jb3yGFEJ8K+lPvSVL2o3Jf/Orjg=', 'Content-Length': '0'} 2022-09-03T05:46:53.916 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c6511f6ad689f031-006312ea4d-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.917 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:53.917 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.918 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.918 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.919 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.919 DEBUG:boto:Method: GET 2022-09-03T05:46:53.920 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.920 DEBUG:boto:Data: 2022-09-03T05:46:53.921 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.921 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:53.922 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.923 DEBUG:boto:Params: {} 2022-09-03T05:46:53.923 DEBUG:boto:Token: None 2022-09-03T05:46:53.924 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.924 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.925 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.927 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f9633e145f78415e-006312ea4d-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.928 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.929 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:53.930 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.930 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.931 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.931 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.932 DEBUG:boto:Method: GET 2022-09-03T05:46:53.932 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.933 DEBUG:boto:Data: 2022-09-03T05:46:53.933 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.934 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:53.935 DEBUG:boto:Port: 8001 2022-09-03T05:46:53.936 DEBUG:boto:Params: {} 2022-09-03T05:46:53.936 DEBUG:boto:Token: None 2022-09-03T05:46:53.937 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.937 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.938 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.941 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e75c09abf9918f2b-006312ea4d-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.942 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.942 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.943 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.944 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.944 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.945 DEBUG:boto:Method: GET 2022-09-03T05:46:53.945 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.946 DEBUG:boto:Data: 2022-09-03T05:46:53.947 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.947 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:53.948 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.949 DEBUG:boto:Params: {} 2022-09-03T05:46:53.951 DEBUG:boto:Token: None 2022-09-03T05:46:53.952 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.953 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.957 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000015ecdbac8ae11d80-006312ea4d-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.958 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.959 DEBUG:boto:path=/dsgvcy-44/ 2022-09-03T05:46:53.960 DEBUG:boto:auth_path=/dsgvcy-44/ 2022-09-03T05:46:53.961 DEBUG:boto:path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.962 DEBUG:boto:auth_path=/dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.963 DEBUG:boto:Method: GET 2022-09-03T05:46:53.964 DEBUG:boto:Path: /dsgvcy-44/?versions&max-keys=999 2022-09-03T05:46:53.965 DEBUG:boto:Data: 2022-09-03T05:46:53.966 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.967 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:53.968 DEBUG:boto:Port: 8001 2022-09-03T05:46:53.969 DEBUG:boto:Params: {} 2022-09-03T05:46:53.970 DEBUG:boto:Token: None 2022-09-03T05:46:53.971 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-44/?versions 2022-09-03T05:46:53.972 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE= 2022-09-03T05:46:53.973 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/prSwdW+bNHGqrCjRcqQZHqmwE=', 'Content-Length': '0'} 2022-09-03T05:46:53.975 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007efc014c2984501c-006312ea4d-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.976 DEBUG:boto:b'dsgvcy-44999false' 2022-09-03T05:46:53.977 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-44 zones={test-zone2, test-zone3} 2022-09-03T05:46:53.977 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-45 zones={test-zone2, test-zone1} 2022-09-03T05:46:53.978 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:53.979 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:53.979 DEBUG:boto:Method: HEAD 2022-09-03T05:46:53.980 DEBUG:boto:Path: /dsgvcy-45/ 2022-09-03T05:46:53.980 DEBUG:boto:Data: 2022-09-03T05:46:53.981 DEBUG:boto:Headers: {} 2022-09-03T05:46:53.981 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:53.982 DEBUG:boto:Port: 8000 2022-09-03T05:46:53.982 DEBUG:boto:Params: {} 2022-09-03T05:46:53.983 DEBUG:boto:Token: None 2022-09-03T05:46:53.983 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:53 GMT /dsgvcy-45/ 2022-09-03T05:46:53.984 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mzx60dBCz+3IGPchZCyLHjda7ZU= 2022-09-03T05:46:53.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mzx60dBCz+3IGPchZCyLHjda7ZU=', 'Content-Length': '0'} 2022-09-03T05:46:53.995 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ff6de3a191438af8-006312ea4d-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:53.996 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:53.997 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:53.998 DEBUG:boto:Method: HEAD 2022-09-03T05:46:53.999 DEBUG:boto:Path: /dsgvcy-45/ 2022-09-03T05:46:54.000 DEBUG:boto:Data: 2022-09-03T05:46:54.001 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.002 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:54.003 DEBUG:boto:Port: 8000 2022-09-03T05:46:54.003 DEBUG:boto:Params: {} 2022-09-03T05:46:54.005 DEBUG:boto:Token: None 2022-09-03T05:46:54.006 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/ 2022-09-03T05:46:54.007 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oAqoeFecUNOILa5MunTR+zm7MzY= 2022-09-03T05:46:54.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oAqoeFecUNOILa5MunTR+zm7MzY=', 'Content-Length': '0'} 2022-09-03T05:46:54.011 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000eeb4cfe086a7bf61-006312ea4e-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.013 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:54.014 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.015 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.016 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.016 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.017 DEBUG:boto:Method: GET 2022-09-03T05:46:54.018 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.019 DEBUG:boto:Data: 2022-09-03T05:46:54.020 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.021 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:54.022 DEBUG:boto:Port: 8000 2022-09-03T05:46:54.023 DEBUG:boto:Params: {} 2022-09-03T05:46:54.024 DEBUG:boto:Token: None 2022-09-03T05:46:54.026 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.027 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.027 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.031 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bddcacd9ce4054f4-006312ea4e-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.032 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.033 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:54.034 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.035 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.036 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.037 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.038 DEBUG:boto:Method: GET 2022-09-03T05:46:54.039 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.039 DEBUG:boto:Data: 2022-09-03T05:46:54.040 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.041 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:54.042 DEBUG:boto:Port: 8000 2022-09-03T05:46:54.043 DEBUG:boto:Params: {} 2022-09-03T05:46:54.044 DEBUG:boto:Token: None 2022-09-03T05:46:54.044 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.045 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.046 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.050 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002a9f61c061187229-006312ea4e-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.090 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.091 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.091 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.092 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.092 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.093 DEBUG:boto:Method: GET 2022-09-03T05:46:54.093 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.094 DEBUG:boto:Data: 2022-09-03T05:46:54.094 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.095 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:54.096 DEBUG:boto:Port: 8000 2022-09-03T05:46:54.096 DEBUG:boto:Params: {} 2022-09-03T05:46:54.097 DEBUG:boto:Token: None 2022-09-03T05:46:54.097 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.098 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.099 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.102 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d957dc7ec9509fd-006312ea4e-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.102 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.103 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.104 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.104 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.105 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.105 DEBUG:boto:Method: GET 2022-09-03T05:46:54.106 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.106 DEBUG:boto:Data: 2022-09-03T05:46:54.107 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.107 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:54.108 DEBUG:boto:Port: 8000 2022-09-03T05:46:54.108 DEBUG:boto:Params: {} 2022-09-03T05:46:54.109 DEBUG:boto:Token: None 2022-09-03T05:46:54.109 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.110 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.111 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.113 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c08946a0e6d5e1df-006312ea4e-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.114 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.115 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-45 zones={test-zone2, test-zone1} 2022-09-03T05:46:54.116 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-45 zones={test-zone3, test-zone1} 2022-09-03T05:46:54.116 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.117 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.118 DEBUG:boto:Method: HEAD 2022-09-03T05:46:54.118 DEBUG:boto:Path: /dsgvcy-45/ 2022-09-03T05:46:54.119 DEBUG:boto:Data: 2022-09-03T05:46:54.119 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.120 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:54.120 DEBUG:boto:Port: 8001 2022-09-03T05:46:54.121 DEBUG:boto:Params: {} 2022-09-03T05:46:54.122 DEBUG:boto:Token: None 2022-09-03T05:46:54.122 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/ 2022-09-03T05:46:54.123 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oAqoeFecUNOILa5MunTR+zm7MzY= 2022-09-03T05:46:54.123 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oAqoeFecUNOILa5MunTR+zm7MzY=', 'Content-Length': '0'} 2022-09-03T05:46:54.126 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000edfc2c64d5a04a71-006312ea4e-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.128 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.128 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.129 DEBUG:boto:Method: HEAD 2022-09-03T05:46:54.129 DEBUG:boto:Path: /dsgvcy-45/ 2022-09-03T05:46:54.130 DEBUG:boto:Data: 2022-09-03T05:46:54.131 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.131 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:54.132 DEBUG:boto:Port: 8000 2022-09-03T05:46:54.132 DEBUG:boto:Params: {} 2022-09-03T05:46:54.133 DEBUG:boto:Token: None 2022-09-03T05:46:54.134 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/ 2022-09-03T05:46:54.134 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oAqoeFecUNOILa5MunTR+zm7MzY= 2022-09-03T05:46:54.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oAqoeFecUNOILa5MunTR+zm7MzY=', 'Content-Length': '0'} 2022-09-03T05:46:54.138 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000cf8162cdb0f91d3c-006312ea4e-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.139 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:54.141 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.142 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.143 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.143 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.144 DEBUG:boto:Method: GET 2022-09-03T05:46:54.144 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.145 DEBUG:boto:Data: 2022-09-03T05:46:54.145 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.146 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:54.146 DEBUG:boto:Port: 8001 2022-09-03T05:46:54.147 DEBUG:boto:Params: {} 2022-09-03T05:46:54.147 DEBUG:boto:Token: None 2022-09-03T05:46:54.151 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.151 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.155 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000070c6c1364e54792d-006312ea4e-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.155 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.156 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:54.157 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.157 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.158 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.159 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.159 DEBUG:boto:Method: GET 2022-09-03T05:46:54.160 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.160 DEBUG:boto:Data: 2022-09-03T05:46:54.161 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.161 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:54.162 DEBUG:boto:Port: 8000 2022-09-03T05:46:54.162 DEBUG:boto:Params: {} 2022-09-03T05:46:54.163 DEBUG:boto:Token: None 2022-09-03T05:46:54.164 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.164 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.168 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a8c52852aca299a6-006312ea4e-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.169 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.169 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.170 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.171 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.171 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.172 DEBUG:boto:Method: GET 2022-09-03T05:46:54.172 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.173 DEBUG:boto:Data: 2022-09-03T05:46:54.173 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.174 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:54.174 DEBUG:boto:Port: 8001 2022-09-03T05:46:54.175 DEBUG:boto:Params: {} 2022-09-03T05:46:54.176 DEBUG:boto:Token: None 2022-09-03T05:46:54.176 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.177 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.180 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005c68bfa150586e91-006312ea4e-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.222 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.224 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.225 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.227 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.228 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.229 DEBUG:boto:Method: GET 2022-09-03T05:46:54.230 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.231 DEBUG:boto:Data: 2022-09-03T05:46:54.232 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.233 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:54.233 DEBUG:boto:Port: 8000 2022-09-03T05:46:54.234 DEBUG:boto:Params: {} 2022-09-03T05:46:54.236 DEBUG:boto:Token: None 2022-09-03T05:46:54.237 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.239 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.243 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000081554d35892d022b-006312ea4e-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.244 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.244 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-45 zones={test-zone3, test-zone1} 2022-09-03T05:46:54.245 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-45 zones={test-zone1, test-zone2} 2022-09-03T05:46:54.245 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.247 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.247 DEBUG:boto:Method: HEAD 2022-09-03T05:46:54.248 DEBUG:boto:Path: /dsgvcy-45/ 2022-09-03T05:46:54.250 DEBUG:boto:Data: 2022-09-03T05:46:54.251 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.252 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:54.253 DEBUG:boto:Port: 8000 2022-09-03T05:46:54.254 DEBUG:boto:Params: {} 2022-09-03T05:46:54.255 DEBUG:boto:Token: None 2022-09-03T05:46:54.256 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/ 2022-09-03T05:46:54.257 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oAqoeFecUNOILa5MunTR+zm7MzY= 2022-09-03T05:46:54.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oAqoeFecUNOILa5MunTR+zm7MzY=', 'Content-Length': '0'} 2022-09-03T05:46:54.261 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a25a6212ae528491-006312ea4e-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.262 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.263 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.263 DEBUG:boto:Method: HEAD 2022-09-03T05:46:54.264 DEBUG:boto:Path: /dsgvcy-45/ 2022-09-03T05:46:54.264 DEBUG:boto:Data: 2022-09-03T05:46:54.265 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.265 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:54.266 DEBUG:boto:Port: 8000 2022-09-03T05:46:54.267 DEBUG:boto:Params: {} 2022-09-03T05:46:54.267 DEBUG:boto:Token: None 2022-09-03T05:46:54.268 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/ 2022-09-03T05:46:54.269 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oAqoeFecUNOILa5MunTR+zm7MzY= 2022-09-03T05:46:54.269 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oAqoeFecUNOILa5MunTR+zm7MzY=', 'Content-Length': '0'} 2022-09-03T05:46:54.274 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d49b6aa07c805e76-006312ea4e-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.275 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:54.276 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.277 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.277 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.278 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.278 DEBUG:boto:Method: GET 2022-09-03T05:46:54.279 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.279 DEBUG:boto:Data: 2022-09-03T05:46:54.280 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.280 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:54.281 DEBUG:boto:Port: 8000 2022-09-03T05:46:54.282 DEBUG:boto:Params: {} 2022-09-03T05:46:54.283 DEBUG:boto:Token: None 2022-09-03T05:46:54.283 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.284 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.287 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000524000a386d534b9-006312ea4e-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.288 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.289 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:54.289 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.290 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.290 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.291 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.291 DEBUG:boto:Method: GET 2022-09-03T05:46:54.292 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.292 DEBUG:boto:Data: 2022-09-03T05:46:54.293 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.293 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:54.294 DEBUG:boto:Port: 8000 2022-09-03T05:46:54.294 DEBUG:boto:Params: {} 2022-09-03T05:46:54.295 DEBUG:boto:Token: None 2022-09-03T05:46:54.296 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.297 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.297 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.300 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000080f0ddb0071374ad-006312ea4e-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.342 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.344 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.345 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.346 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.347 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.347 DEBUG:boto:Method: GET 2022-09-03T05:46:54.349 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.350 DEBUG:boto:Data: 2022-09-03T05:46:54.351 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.352 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:54.353 DEBUG:boto:Port: 8000 2022-09-03T05:46:54.354 DEBUG:boto:Params: {} 2022-09-03T05:46:54.354 DEBUG:boto:Token: None 2022-09-03T05:46:54.355 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.356 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.357 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.360 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000033e0e91b164ef88-006312ea4e-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.361 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.361 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.362 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.363 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.363 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.364 DEBUG:boto:Method: GET 2022-09-03T05:46:54.365 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.366 DEBUG:boto:Data: 2022-09-03T05:46:54.367 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.368 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:54.368 DEBUG:boto:Port: 8000 2022-09-03T05:46:54.369 DEBUG:boto:Params: {} 2022-09-03T05:46:54.370 DEBUG:boto:Token: None 2022-09-03T05:46:54.372 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.373 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.374 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.377 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000988515a8308fe432-006312ea4e-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.379 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.380 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-45 zones={test-zone1, test-zone2} 2022-09-03T05:46:54.381 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-45 zones={test-zone3, test-zone2} 2022-09-03T05:46:54.382 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.382 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.384 DEBUG:boto:Method: HEAD 2022-09-03T05:46:54.384 DEBUG:boto:Path: /dsgvcy-45/ 2022-09-03T05:46:54.385 DEBUG:boto:Data: 2022-09-03T05:46:54.385 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.386 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:54.387 DEBUG:boto:Port: 8001 2022-09-03T05:46:54.387 DEBUG:boto:Params: {} 2022-09-03T05:46:54.388 DEBUG:boto:Token: None 2022-09-03T05:46:54.389 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/ 2022-09-03T05:46:54.389 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oAqoeFecUNOILa5MunTR+zm7MzY= 2022-09-03T05:46:54.390 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oAqoeFecUNOILa5MunTR+zm7MzY=', 'Content-Length': '0'} 2022-09-03T05:46:54.393 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e9d621a7ab2232e8-006312ea4e-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.393 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.394 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.395 DEBUG:boto:Method: HEAD 2022-09-03T05:46:54.395 DEBUG:boto:Path: /dsgvcy-45/ 2022-09-03T05:46:54.396 DEBUG:boto:Data: 2022-09-03T05:46:54.396 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.397 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:54.398 DEBUG:boto:Port: 8000 2022-09-03T05:46:54.398 DEBUG:boto:Params: {} 2022-09-03T05:46:54.399 DEBUG:boto:Token: None 2022-09-03T05:46:54.400 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/ 2022-09-03T05:46:54.401 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oAqoeFecUNOILa5MunTR+zm7MzY= 2022-09-03T05:46:54.402 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oAqoeFecUNOILa5MunTR+zm7MzY=', 'Content-Length': '0'} 2022-09-03T05:46:54.405 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004837a379d8bdbe4a-006312ea4e-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.406 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:54.407 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.407 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.408 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.409 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.410 DEBUG:boto:Method: GET 2022-09-03T05:46:54.410 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.411 DEBUG:boto:Data: 2022-09-03T05:46:54.412 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.412 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:54.413 DEBUG:boto:Port: 8001 2022-09-03T05:46:54.414 DEBUG:boto:Params: {} 2022-09-03T05:46:54.415 DEBUG:boto:Token: None 2022-09-03T05:46:54.416 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.417 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.418 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.421 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b9ddeca0ca3f9561-006312ea4e-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.423 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.424 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:54.424 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.425 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.426 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.426 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.427 DEBUG:boto:Method: GET 2022-09-03T05:46:54.427 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.428 DEBUG:boto:Data: 2022-09-03T05:46:54.428 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.429 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:54.429 DEBUG:boto:Port: 8000 2022-09-03T05:46:54.430 DEBUG:boto:Params: {} 2022-09-03T05:46:54.430 DEBUG:boto:Token: None 2022-09-03T05:46:54.431 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.431 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.435 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000559770ba3c7763cd-006312ea4e-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.436 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.437 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.438 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.439 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.440 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.441 DEBUG:boto:Method: GET 2022-09-03T05:46:54.442 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.442 DEBUG:boto:Data: 2022-09-03T05:46:54.443 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.443 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:54.444 DEBUG:boto:Port: 8001 2022-09-03T05:46:54.445 DEBUG:boto:Params: {} 2022-09-03T05:46:54.446 DEBUG:boto:Token: None 2022-09-03T05:46:54.447 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.448 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.449 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.452 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000532a97ced6002abd-006312ea4e-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.454 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.455 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.456 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.457 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.458 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.459 DEBUG:boto:Method: GET 2022-09-03T05:46:54.460 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.461 DEBUG:boto:Data: 2022-09-03T05:46:54.462 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.463 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:54.464 DEBUG:boto:Port: 8000 2022-09-03T05:46:54.464 DEBUG:boto:Params: {} 2022-09-03T05:46:54.465 DEBUG:boto:Token: None 2022-09-03T05:46:54.466 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.468 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.469 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.472 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000040c8f00d0ced53ce-006312ea4e-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.473 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.474 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-45 zones={test-zone3, test-zone2} 2022-09-03T05:46:54.475 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-45 zones={test-zone1, test-zone3} 2022-09-03T05:46:54.477 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.478 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.479 DEBUG:boto:Method: HEAD 2022-09-03T05:46:54.480 DEBUG:boto:Path: /dsgvcy-45/ 2022-09-03T05:46:54.481 DEBUG:boto:Data: 2022-09-03T05:46:54.482 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.482 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:54.483 DEBUG:boto:Port: 8000 2022-09-03T05:46:54.484 DEBUG:boto:Params: {} 2022-09-03T05:46:54.485 DEBUG:boto:Token: None 2022-09-03T05:46:54.486 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/ 2022-09-03T05:46:54.487 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oAqoeFecUNOILa5MunTR+zm7MzY= 2022-09-03T05:46:54.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oAqoeFecUNOILa5MunTR+zm7MzY=', 'Content-Length': '0'} 2022-09-03T05:46:54.492 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001a433c697f59cf95-006312ea4e-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.493 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.494 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.495 DEBUG:boto:Method: HEAD 2022-09-03T05:46:54.496 DEBUG:boto:Path: /dsgvcy-45/ 2022-09-03T05:46:54.497 DEBUG:boto:Data: 2022-09-03T05:46:54.499 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.500 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:54.500 DEBUG:boto:Port: 8001 2022-09-03T05:46:54.501 DEBUG:boto:Params: {} 2022-09-03T05:46:54.501 DEBUG:boto:Token: None 2022-09-03T05:46:54.502 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/ 2022-09-03T05:46:54.503 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oAqoeFecUNOILa5MunTR+zm7MzY= 2022-09-03T05:46:54.504 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oAqoeFecUNOILa5MunTR+zm7MzY=', 'Content-Length': '0'} 2022-09-03T05:46:54.507 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000066daed11c29a2d4d-006312ea4e-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.508 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:54.509 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.509 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.510 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.511 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.511 DEBUG:boto:Method: GET 2022-09-03T05:46:54.512 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.512 DEBUG:boto:Data: 2022-09-03T05:46:54.513 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.513 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:54.514 DEBUG:boto:Port: 8000 2022-09-03T05:46:54.515 DEBUG:boto:Params: {} 2022-09-03T05:46:54.515 DEBUG:boto:Token: None 2022-09-03T05:46:54.516 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.516 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.517 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.520 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cc163d7a5fe6d6bd-006312ea4e-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.522 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.531 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:54.533 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.534 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.535 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.536 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.536 DEBUG:boto:Method: GET 2022-09-03T05:46:54.538 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.539 DEBUG:boto:Data: 2022-09-03T05:46:54.540 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.541 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:54.542 DEBUG:boto:Port: 8001 2022-09-03T05:46:54.543 DEBUG:boto:Params: {} 2022-09-03T05:46:54.544 DEBUG:boto:Token: None 2022-09-03T05:46:54.544 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.546 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.549 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000089ed9067ef934f15-006312ea4e-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.550 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.551 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.551 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.552 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.552 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.553 DEBUG:boto:Method: GET 2022-09-03T05:46:54.554 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.554 DEBUG:boto:Data: 2022-09-03T05:46:54.555 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.555 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:46:54.556 DEBUG:boto:Port: 8000 2022-09-03T05:46:54.556 DEBUG:boto:Params: {} 2022-09-03T05:46:54.557 DEBUG:boto:Token: None 2022-09-03T05:46:54.558 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.559 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.559 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.562 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b5db1a0a2fabae43-006312ea4e-1434-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.563 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.563 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.564 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.565 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.565 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.566 DEBUG:boto:Method: GET 2022-09-03T05:46:54.566 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.567 DEBUG:boto:Data: 2022-09-03T05:46:54.567 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.568 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:54.568 DEBUG:boto:Port: 8001 2022-09-03T05:46:54.569 DEBUG:boto:Params: {} 2022-09-03T05:46:54.570 DEBUG:boto:Token: None 2022-09-03T05:46:54.570 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.571 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.578 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e8e16741a9021e71-006312ea4e-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.579 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.580 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-45 zones={test-zone1, test-zone3} 2022-09-03T05:46:54.581 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-45 zones={test-zone2, test-zone3} 2022-09-03T05:46:54.581 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.582 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.582 DEBUG:boto:Method: HEAD 2022-09-03T05:46:54.583 DEBUG:boto:Path: /dsgvcy-45/ 2022-09-03T05:46:54.583 DEBUG:boto:Data: 2022-09-03T05:46:54.584 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.584 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:54.585 DEBUG:boto:Port: 8000 2022-09-03T05:46:54.585 DEBUG:boto:Params: {} 2022-09-03T05:46:54.586 DEBUG:boto:Token: None 2022-09-03T05:46:54.586 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/ 2022-09-03T05:46:54.587 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oAqoeFecUNOILa5MunTR+zm7MzY= 2022-09-03T05:46:54.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oAqoeFecUNOILa5MunTR+zm7MzY=', 'Content-Length': '0'} 2022-09-03T05:46:54.592 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008b70c9e085ea96f7-006312ea4e-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.593 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.593 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.594 DEBUG:boto:Method: HEAD 2022-09-03T05:46:54.595 DEBUG:boto:Path: /dsgvcy-45/ 2022-09-03T05:46:54.597 DEBUG:boto:Data: 2022-09-03T05:46:54.598 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.599 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:54.599 DEBUG:boto:Port: 8001 2022-09-03T05:46:54.600 DEBUG:boto:Params: {} 2022-09-03T05:46:54.600 DEBUG:boto:Token: None 2022-09-03T05:46:54.601 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/ 2022-09-03T05:46:54.601 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oAqoeFecUNOILa5MunTR+zm7MzY= 2022-09-03T05:46:54.602 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oAqoeFecUNOILa5MunTR+zm7MzY=', 'Content-Length': '0'} 2022-09-03T05:46:54.604 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000dac2fba6032620dd-006312ea4e-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.605 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T05:46:54.606 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.606 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.607 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.608 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.608 DEBUG:boto:Method: GET 2022-09-03T05:46:54.609 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.609 DEBUG:boto:Data: 2022-09-03T05:46:54.610 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.610 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:54.611 DEBUG:boto:Port: 8000 2022-09-03T05:46:54.612 DEBUG:boto:Params: {} 2022-09-03T05:46:54.612 DEBUG:boto:Token: None 2022-09-03T05:46:54.613 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.613 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.614 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.623 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004f43a58787438791-006312ea4e-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.658 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.661 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T05:46:54.662 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.663 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.664 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.664 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.666 DEBUG:boto:Method: GET 2022-09-03T05:46:54.666 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.667 DEBUG:boto:Data: 2022-09-03T05:46:54.667 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.668 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:54.668 DEBUG:boto:Port: 8001 2022-09-03T05:46:54.669 DEBUG:boto:Params: {} 2022-09-03T05:46:54.669 DEBUG:boto:Token: None 2022-09-03T05:46:54.671 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.671 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.675 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e7820a7b76f7f9cd-006312ea4e-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.676 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.677 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.677 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.678 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.679 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.679 DEBUG:boto:Method: GET 2022-09-03T05:46:54.680 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.681 DEBUG:boto:Data: 2022-09-03T05:46:54.682 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.682 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:46:54.683 DEBUG:boto:Port: 8000 2022-09-03T05:46:54.684 DEBUG:boto:Params: {} 2022-09-03T05:46:54.685 DEBUG:boto:Token: None 2022-09-03T05:46:54.686 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.686 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.687 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.690 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001f8da24a18fd6b0d-006312ea4e-1444-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.690 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.691 DEBUG:boto:path=/dsgvcy-45/ 2022-09-03T05:46:54.692 DEBUG:boto:auth_path=/dsgvcy-45/ 2022-09-03T05:46:54.693 DEBUG:boto:path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.693 DEBUG:boto:auth_path=/dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.694 DEBUG:boto:Method: GET 2022-09-03T05:46:54.696 DEBUG:boto:Path: /dsgvcy-45/?versions&max-keys=999 2022-09-03T05:46:54.696 DEBUG:boto:Data: 2022-09-03T05:46:54.697 DEBUG:boto:Headers: {} 2022-09-03T05:46:54.697 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:46:54.698 DEBUG:boto:Port: 8001 2022-09-03T05:46:54.698 DEBUG:boto:Params: {} 2022-09-03T05:46:54.699 DEBUG:boto:Token: None 2022-09-03T05:46:54.701 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:46:54 GMT /dsgvcy-45/?versions 2022-09-03T05:46:54.702 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc= 2022-09-03T05:46:54.702 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:46:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fozKIRR5lKvML6GgawoKX/Ndmc=', 'Content-Length': '0'} 2022-09-03T05:46:54.705 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001717647d252f9e1d-006312ea4e-143a-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:46:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:46:54.707 DEBUG:boto:b'dsgvcy-45999false' 2022-09-03T05:46:54.708 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-45 zones={test-zone2, test-zone3} 2022-09-03T05:46:54.708 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '41ecd2a5-afc0-4eae-9349-7dfe39baa1cb', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:46:54.709 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '41ecd2a5-afc0-4eae-9349-7dfe39baa1cb', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:46:54.710 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 41ecd2a5-afc0-4eae-9349-7dfe39baa1cb --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:46:54.765 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:46:54.766 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:46:54.803 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:46:54.804 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.804 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182298.903952_68.1", 2022-09-03T05:46:54.804 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "user", 2022-09-03T05:46:54.804 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "rgw-multisite-test-user", 2022-09-03T05:46:54.805 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.903952433Z", 2022-09-03T05:46:54.805 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.805 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.805 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.806 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.806 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.806 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.807 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_4zvH7kUaBnzKjfeQSSk_55d", 2022-09-03T05:46:54.807 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.807 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.808 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.808 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:54.808 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.808 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.809 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.809 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.809 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182298.906852_69.1", 2022-09-03T05:46:54.809 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "user", 2022-09-03T05:46:54.810 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "rgw-multisite-test-user", 2022-09-03T05:46:54.810 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852070Z", 2022-09-03T05:46:54.810 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.810 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.811 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.811 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.811 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.812 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.812 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_4zvH7kUaBnzKjfeQSSk_55d", 2022-09-03T05:46:54.812 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.812 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.813 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.813 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:54.813 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.813 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.814 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.814 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.814 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.694951_88.1", 2022-09-03T05:46:54.814 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:54.815 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10", 2022-09-03T05:46:54.815 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.694951214Z", 2022-09-03T05:46:54.815 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.815 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.816 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.816 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.816 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.816 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.817 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_9ORw1pyvfMIqMonbAbpO7T7", 2022-09-03T05:46:54.817 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.817 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.817 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.818 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:54.818 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.818 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.818 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.819 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.819 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.700326_89.1", 2022-09-03T05:46:54.819 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:54.819 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10", 2022-09-03T05:46:54.820 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.700326196Z", 2022-09-03T05:46:54.820 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.820 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.821 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.821 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.821 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.821 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.822 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_9ORw1pyvfMIqMonbAbpO7T7", 2022-09-03T05:46:54.822 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.822 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.822 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.823 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:54.823 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.823 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.824 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.824 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.824 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.702420_90.1", 2022-09-03T05:46:54.825 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:54.825 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-10", 2022-09-03T05:46:54.825 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.702420359Z", 2022-09-03T05:46:54.825 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.826 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.826 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.826 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.826 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.827 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.827 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "__K0-JaxveSMyJYue_ni7kIA", 2022-09-03T05:46:54.827 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.827 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.828 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.828 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:54.828 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.828 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.829 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.829 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.829 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.705433_91.1", 2022-09-03T05:46:54.829 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:54.830 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-10", 2022-09-03T05:46:54.830 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.705433036Z", 2022-09-03T05:46:54.830 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.830 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.831 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.831 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.831 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.831 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.832 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "__K0-JaxveSMyJYue_ni7kIA", 2022-09-03T05:46:54.832 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.832 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.833 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.833 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:54.833 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.833 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.834 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.834 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.834 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.724120_92.1", 2022-09-03T05:46:54.834 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:54.835 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11", 2022-09-03T05:46:54.835 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.724120949Z", 2022-09-03T05:46:54.835 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.835 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.836 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.836 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.836 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.836 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.837 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_mm_a_MgGT-mjAscVo5eDJN3", 2022-09-03T05:46:54.837 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.837 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.837 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.838 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:54.838 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.838 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.838 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.839 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.839 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.726647_93.1", 2022-09-03T05:46:54.839 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:54.839 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11", 2022-09-03T05:46:54.840 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.726647140Z", 2022-09-03T05:46:54.840 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.840 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.840 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.841 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.841 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.841 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.841 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_mm_a_MgGT-mjAscVo5eDJN3", 2022-09-03T05:46:54.842 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.842 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.842 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.843 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:54.843 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.843 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.843 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.844 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.844 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.727701_94.1", 2022-09-03T05:46:54.844 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:54.844 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-11", 2022-09-03T05:46:54.844 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.727701875Z", 2022-09-03T05:46:54.845 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.845 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.845 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.845 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.846 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.846 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.846 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_co69vPv41E29NQft_mgvTJ6", 2022-09-03T05:46:54.846 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.847 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.847 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.847 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:54.847 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.848 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.848 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.848 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.849 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.730089_95.1", 2022-09-03T05:46:54.849 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:54.849 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-11", 2022-09-03T05:46:54.849 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.730089458Z", 2022-09-03T05:46:54.850 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.850 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.850 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.850 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.851 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.851 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.851 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_co69vPv41E29NQft_mgvTJ6", 2022-09-03T05:46:54.851 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.852 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.852 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.852 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:54.852 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.853 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.853 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.853 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.853 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.760919_96.1", 2022-09-03T05:46:54.854 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:54.854 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12", 2022-09-03T05:46:54.854 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.760919958Z", 2022-09-03T05:46:54.854 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.855 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.855 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.855 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.856 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.856 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.856 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_87U_6FvKdHGzg3asBWz0TB6", 2022-09-03T05:46:54.856 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.857 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.857 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.857 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:54.857 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.857 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.858 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.858 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.858 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.763582_97.1", 2022-09-03T05:46:54.859 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:54.859 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12", 2022-09-03T05:46:54.859 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.763582008Z", 2022-09-03T05:46:54.859 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.859 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.860 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.860 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.860 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.860 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.861 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_87U_6FvKdHGzg3asBWz0TB6", 2022-09-03T05:46:54.861 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.861 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.862 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.862 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:54.862 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.862 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.863 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.863 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.863 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.764601_98.1", 2022-09-03T05:46:54.863 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:54.864 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-12", 2022-09-03T05:46:54.864 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.764601024Z", 2022-09-03T05:46:54.864 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.864 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.865 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.865 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.865 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.865 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.866 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_F5RlS3fhN6ILhiyoQ3qKUpC", 2022-09-03T05:46:54.866 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.866 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.866 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.867 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:54.867 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.867 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.867 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.868 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.868 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.766898_99.1", 2022-09-03T05:46:54.868 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:54.868 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-12", 2022-09-03T05:46:54.869 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.766898391Z", 2022-09-03T05:46:54.869 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.869 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.869 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.870 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.870 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.870 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.870 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_F5RlS3fhN6ILhiyoQ3qKUpC", 2022-09-03T05:46:54.871 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.871 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.871 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.871 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:54.872 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.872 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.872 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.872 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.873 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182745.944715_100.1", 2022-09-03T05:46:54.873 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:54.873 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13", 2022-09-03T05:46:54.874 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:45.944715403Z", 2022-09-03T05:46:54.874 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.874 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.874 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.875 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.875 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.875 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.875 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_ZwtCDnJJncqHbK6_XX4E-Uv", 2022-09-03T05:46:54.876 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.876 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.876 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.876 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:54.877 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.877 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.877 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.877 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.878 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182745.948069_101.1", 2022-09-03T05:46:54.878 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:54.878 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13", 2022-09-03T05:46:54.878 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:45.948069121Z", 2022-09-03T05:46:54.879 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.879 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.879 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.879 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.880 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.880 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.880 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_ZwtCDnJJncqHbK6_XX4E-Uv", 2022-09-03T05:46:54.880 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.881 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.881 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.881 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:54.881 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.882 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.882 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.882 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.883 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182745.950114_102.1", 2022-09-03T05:46:54.883 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:54.883 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-13", 2022-09-03T05:46:54.883 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:45.950114676Z", 2022-09-03T05:46:54.884 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.884 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.884 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.884 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.885 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.885 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.885 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_kkkmqlwBfjjtwLr8U95H7wF", 2022-09-03T05:46:54.885 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.886 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.886 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.886 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:54.886 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.887 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.887 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.887 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.887 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182745.953048_103.1", 2022-09-03T05:46:54.888 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:54.888 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-13", 2022-09-03T05:46:54.888 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:45.953048780Z", 2022-09-03T05:46:54.888 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.889 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.889 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.889 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.889 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.890 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.890 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_kkkmqlwBfjjtwLr8U95H7wF", 2022-09-03T05:46:54.890 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.890 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.891 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.891 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:54.891 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.891 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.892 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.892 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.892 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182745.988740_104.1", 2022-09-03T05:46:54.892 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:54.893 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14", 2022-09-03T05:46:54.893 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:45.988740832Z", 2022-09-03T05:46:54.893 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.893 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.894 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.894 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.894 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.894 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.895 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Yidm7pQHzXjzyowF0vXCv_q", 2022-09-03T05:46:54.895 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.895 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.895 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.896 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:54.896 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.896 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.896 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.897 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.897 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182745.991668_105.1", 2022-09-03T05:46:54.897 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:54.897 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14", 2022-09-03T05:46:54.898 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:45.991668492Z", 2022-09-03T05:46:54.898 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.898 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.898 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.899 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.899 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.899 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.899 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Yidm7pQHzXjzyowF0vXCv_q", 2022-09-03T05:46:54.900 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.900 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.900 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.900 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:54.901 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.901 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.901 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.901 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.902 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182745.992722_106.1", 2022-09-03T05:46:54.902 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:54.902 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-14", 2022-09-03T05:46:54.902 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:45.992722013Z", 2022-09-03T05:46:54.903 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.903 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.903 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.903 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.904 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.904 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.904 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_0ZpMZLjvQ0yH4NIki0Mppzv", 2022-09-03T05:46:54.904 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.905 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.905 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.905 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:54.905 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.906 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.906 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.906 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.906 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182745.995451_107.1", 2022-09-03T05:46:54.907 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:54.907 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-14", 2022-09-03T05:46:54.907 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:45.995451716Z", 2022-09-03T05:46:54.907 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.908 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.908 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.908 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.908 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.909 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.909 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_0ZpMZLjvQ0yH4NIki0Mppzv", 2022-09-03T05:46:54.909 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.909 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.910 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.910 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:54.910 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.911 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.911 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.911 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.911 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182746.030460_108.1", 2022-09-03T05:46:54.911 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:54.912 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15", 2022-09-03T05:46:54.912 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:46.030460591Z", 2022-09-03T05:46:54.912 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.912 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.913 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.913 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.913 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.913 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.914 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_9EbzjhdvUtwpRj9gUb6Zy_c", 2022-09-03T05:46:54.914 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.914 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.914 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.915 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:54.915 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.915 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.915 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.916 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.916 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182746.033416_109.1", 2022-09-03T05:46:54.916 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:54.916 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15", 2022-09-03T05:46:54.917 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:46.033416114Z", 2022-09-03T05:46:54.917 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.917 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.917 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.918 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.918 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.918 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.919 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_9EbzjhdvUtwpRj9gUb6Zy_c", 2022-09-03T05:46:54.919 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.919 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.919 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.920 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:54.920 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.920 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.920 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.921 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.921 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182746.035561_110.1", 2022-09-03T05:46:54.921 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:54.921 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-15", 2022-09-03T05:46:54.922 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:46.035561132Z", 2022-09-03T05:46:54.922 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.922 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.922 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.923 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.923 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.923 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.923 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_p1RJcPdLE0s_MQy_T3ZeW1Z", 2022-09-03T05:46:54.924 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.924 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.924 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.924 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:54.925 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.925 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.925 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.925 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.926 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182746.041492_111.1", 2022-09-03T05:46:54.926 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:54.926 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-15", 2022-09-03T05:46:54.926 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:46.041492579Z", 2022-09-03T05:46:54.927 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.927 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.927 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.927 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.928 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.928 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.928 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_p1RJcPdLE0s_MQy_T3ZeW1Z", 2022-09-03T05:46:54.928 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.929 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.929 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.929 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:54.929 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.930 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.930 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.930 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.930 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182851.903860_122.1", 2022-09-03T05:46:54.931 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:54.931 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16", 2022-09-03T05:46:54.931 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:31.903860682Z", 2022-09-03T05:46:54.931 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.932 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.932 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.932 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.932 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.933 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.933 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:54.933 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.933 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.934 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.934 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:54.934 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.935 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.935 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.935 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.935 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182851.907991_123.1", 2022-09-03T05:46:54.936 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:54.936 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16", 2022-09-03T05:46:54.936 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:31.907991084Z", 2022-09-03T05:46:54.936 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.936 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.937 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.937 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.937 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.937 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.938 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:54.938 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.938 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.938 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.939 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:54.939 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.939 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.939 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.940 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.940 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182851.909130_124.1", 2022-09-03T05:46:54.940 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:54.940 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-16", 2022-09-03T05:46:54.941 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:31.909130687Z", 2022-09-03T05:46:54.941 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.941 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.941 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.942 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.942 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.942 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.942 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_P0q3AfXqSQL8XW6gVnA4CRA", 2022-09-03T05:46:54.943 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.943 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.943 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.943 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:54.944 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.944 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.944 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.944 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.945 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182851.915199_125.1", 2022-09-03T05:46:54.945 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:54.945 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-16", 2022-09-03T05:46:54.945 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:31.915199302Z", 2022-09-03T05:46:54.946 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.946 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.946 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.946 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.947 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.947 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.947 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_P0q3AfXqSQL8XW6gVnA4CRA", 2022-09-03T05:46:54.947 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.948 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.948 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.948 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:54.948 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.949 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.949 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.949 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.950 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182851.945316_126.1", 2022-09-03T05:46:54.950 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:54.950 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17", 2022-09-03T05:46:54.950 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:31.945316218Z", 2022-09-03T05:46:54.951 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.951 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.951 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.951 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.952 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.952 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.952 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:54.952 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.953 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.953 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.953 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:54.953 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.954 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.954 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.954 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.954 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182851.950120_127.1", 2022-09-03T05:46:54.955 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:54.955 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17", 2022-09-03T05:46:54.955 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:31.950120909Z", 2022-09-03T05:46:54.955 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.956 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.956 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.956 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.956 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.957 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.957 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:54.957 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.957 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.958 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.958 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:54.958 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.958 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.959 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.959 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.959 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182851.952454_128.1", 2022-09-03T05:46:54.959 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:54.960 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-17", 2022-09-03T05:46:54.960 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:31.952454005Z", 2022-09-03T05:46:54.960 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.960 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.961 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.961 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.961 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.961 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.962 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_j4__iZ2_YM9iD4VCcQhy-lG", 2022-09-03T05:46:54.962 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.962 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.962 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.963 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:54.963 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.963 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.963 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.964 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.964 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182851.958649_129.1", 2022-09-03T05:46:54.964 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:54.964 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-17", 2022-09-03T05:46:54.965 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:31.958649291Z", 2022-09-03T05:46:54.965 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.965 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.965 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.966 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.966 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.966 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.966 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_j4__iZ2_YM9iD4VCcQhy-lG", 2022-09-03T05:46:54.967 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.967 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.967 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.967 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:54.968 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.968 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.968 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.968 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.969 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182851.998554_130.1", 2022-09-03T05:46:54.969 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:54.969 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18", 2022-09-03T05:46:54.969 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:31.998554746Z", 2022-09-03T05:46:54.970 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.970 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.970 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.970 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.971 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.971 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.971 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:54.971 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.972 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.972 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.972 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:54.972 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.973 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.973 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.973 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.973 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182852.004610_131.1", 2022-09-03T05:46:54.974 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:54.974 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18", 2022-09-03T05:46:54.974 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:32.004610066Z", 2022-09-03T05:46:54.974 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.975 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.975 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.975 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.975 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.976 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.976 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:54.976 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.976 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.977 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.977 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:54.977 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.977 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.978 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.978 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.978 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182852.007026_132.1", 2022-09-03T05:46:54.978 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:54.979 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-18", 2022-09-03T05:46:54.979 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:32.007026261Z", 2022-09-03T05:46:54.979 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.979 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.980 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.980 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.980 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.980 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.981 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_O7UdALq44EBf7qF8RLPxZMv", 2022-09-03T05:46:54.981 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.981 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.981 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.982 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:54.982 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.982 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.982 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.983 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.983 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182852.013139_133.1", 2022-09-03T05:46:54.983 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:54.983 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-18", 2022-09-03T05:46:54.984 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:32.013139549Z", 2022-09-03T05:46:54.984 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.984 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.984 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:54.985 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:54.985 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.985 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.986 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_O7UdALq44EBf7qF8RLPxZMv", 2022-09-03T05:46:54.986 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.986 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.986 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.987 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:54.987 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.987 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.987 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.988 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.988 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182852.069466_134.1", 2022-09-03T05:46:54.988 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:54.988 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16", 2022-09-03T05:46:54.989 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:32.069466103Z", 2022-09-03T05:46:54.989 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.989 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.989 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:54.990 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.990 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.990 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.990 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:54.991 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:54.991 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.991 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.991 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:54.992 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.992 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.992 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.992 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.993 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182852.078407_135.1", 2022-09-03T05:46:54.993 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:54.993 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16", 2022-09-03T05:46:54.993 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:32.078407455Z", 2022-09-03T05:46:54.994 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.994 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.994 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:54.994 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:54.995 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.995 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:54.995 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:54.996 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:54.996 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.996 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:54.996 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:54.996 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.997 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:54.997 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:54.997 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:54.997 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182852.095941_136.1", 2022-09-03T05:46:54.998 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:54.998 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17", 2022-09-03T05:46:54.998 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:32.095941153Z", 2022-09-03T05:46:54.998 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:54.999 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:54.999 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:54.999 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.000 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.000 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.000 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:55.000 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.001 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.001 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.001 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.001 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.002 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.002 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.002 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.002 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182852.099371_137.1", 2022-09-03T05:46:55.003 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.003 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17", 2022-09-03T05:46:55.003 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:32.099371547Z", 2022-09-03T05:46:55.003 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.004 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.004 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:55.004 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.004 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.005 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.005 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:55.005 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.005 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.006 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.006 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.006 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.006 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.007 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.007 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.007 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182852.121946_138.1", 2022-09-03T05:46:55.007 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.008 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18", 2022-09-03T05:46:55.008 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:32.121946745Z", 2022-09-03T05:46:55.008 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.008 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.009 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:55.009 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.009 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.009 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.010 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:55.010 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.010 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.010 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.011 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.011 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.011 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.011 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.012 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.012 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182852.124949_139.1", 2022-09-03T05:46:55.012 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.012 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18", 2022-09-03T05:46:55.013 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:32.124949763Z", 2022-09-03T05:46:55.013 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.013 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.013 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:55.014 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.014 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.014 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.015 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:55.015 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.015 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.015 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.016 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.016 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.016 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.016 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.016 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.017 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183242.240665_157.1", 2022-09-03T05:46:55.017 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.017 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19", 2022-09-03T05:46:55.018 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:34:02.240665028Z", 2022-09-03T05:46:55.018 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.018 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.018 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.019 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.019 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.019 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.019 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:55.020 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.020 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.020 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.020 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.021 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.021 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.021 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.021 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.022 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183242.243600_158.1", 2022-09-03T05:46:55.022 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.022 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19", 2022-09-03T05:46:55.023 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:34:02.243600484Z", 2022-09-03T05:46:55.023 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.023 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.023 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.024 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.024 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.024 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.024 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:55.025 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.025 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.025 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.025 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.026 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.026 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.026 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.026 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.027 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183242.244744_159.1", 2022-09-03T05:46:55.027 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.027 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-19", 2022-09-03T05:46:55.027 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:34:02.244744943Z", 2022-09-03T05:46:55.028 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.028 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.028 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.028 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.029 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.029 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.029 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_B_TWex9_lpCif6prNO7BbtO", 2022-09-03T05:46:55.029 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.030 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.030 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.030 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.030 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.031 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.031 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.031 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.031 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183242.247625_160.1", 2022-09-03T05:46:55.032 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.032 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-19", 2022-09-03T05:46:55.032 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:34:02.247625174Z", 2022-09-03T05:46:55.032 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.033 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.033 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.033 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.033 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.034 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.034 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_B_TWex9_lpCif6prNO7BbtO", 2022-09-03T05:46:55.034 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.034 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.035 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.035 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.035 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.036 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.036 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.036 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.036 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183242.265141_161.1", 2022-09-03T05:46:55.037 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.037 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19", 2022-09-03T05:46:55.037 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:34:02.265141968Z", 2022-09-03T05:46:55.037 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.038 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.038 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:55.038 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.038 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.039 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.039 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:55.039 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.039 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.040 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.040 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.040 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.040 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.041 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.041 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.041 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183242.267929_162.1", 2022-09-03T05:46:55.041 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.042 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19", 2022-09-03T05:46:55.042 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:34:02.267929270Z", 2022-09-03T05:46:55.042 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.043 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.043 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:55.043 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.043 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.044 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.044 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:55.044 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.044 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.045 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.045 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.045 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.045 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.046 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.046 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.046 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183458.677214_163.1", 2022-09-03T05:46:55.046 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.047 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:55.047 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:37:38.677214984Z", 2022-09-03T05:46:55.047 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.047 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.048 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.048 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.048 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.048 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.049 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:55.049 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.049 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.049 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.050 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.050 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.050 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.050 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.051 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.051 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183458.684800_164.1", 2022-09-03T05:46:55.051 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.051 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:55.052 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:37:38.684800313Z", 2022-09-03T05:46:55.052 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.052 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.052 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.053 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.053 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.053 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.053 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:55.054 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.054 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.054 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.054 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.055 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.055 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.055 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.056 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.056 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183458.685753_165.1", 2022-09-03T05:46:55.056 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.056 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-20", 2022-09-03T05:46:55.057 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:37:38.685753497Z", 2022-09-03T05:46:55.057 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.057 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.057 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.058 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.058 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.058 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.058 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_yPpDuhKk6Xsxh9Gd4Ds8exJ", 2022-09-03T05:46:55.059 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.059 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.059 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.059 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.060 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.060 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.060 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.060 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.061 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183458.694130_166.1", 2022-09-03T05:46:55.061 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.061 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-20", 2022-09-03T05:46:55.061 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:37:38.694130749Z", 2022-09-03T05:46:55.062 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.062 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.062 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.063 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.063 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.063 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.063 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_yPpDuhKk6Xsxh9Gd4Ds8exJ", 2022-09-03T05:46:55.064 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.064 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.064 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.064 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.065 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.065 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.065 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.065 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.066 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183531.352224_183.1", 2022-09-03T05:46:55.066 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.066 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:55.066 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:38:51.352224643Z", 2022-09-03T05:46:55.067 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.067 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.067 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:55.067 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.068 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.068 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.068 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:55.068 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.069 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.069 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.069 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.069 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.070 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.070 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.070 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.071 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183531.355467_184.1", 2022-09-03T05:46:55.071 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.071 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:55.071 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:38:51.355467679Z", 2022-09-03T05:46:55.071 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.072 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.072 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:55.072 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.072 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.073 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.073 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:55.073 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.073 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.074 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.074 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.074 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.074 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.075 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.075 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.075 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183570.365072_185.1", 2022-09-03T05:46:55.075 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.076 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:55.076 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:39:30.365072646Z", 2022-09-03T05:46:55.076 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.076 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.077 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:55.077 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.077 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.077 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.078 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:55.078 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 3 2022-09-03T05:46:55.078 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.078 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.079 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.079 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.079 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.080 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.080 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.080 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183570.368454_186.1", 2022-09-03T05:46:55.080 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.081 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:55.081 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:39:30.368454229Z", 2022-09-03T05:46:55.081 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.081 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.082 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:55.082 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.082 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.082 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.083 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:55.083 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 3 2022-09-03T05:46:55.083 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.083 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.084 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.084 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.084 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.084 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.085 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.085 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183652.125862_187.1", 2022-09-03T05:46:55.085 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.085 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21", 2022-09-03T05:46:55.086 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:52.125862114Z", 2022-09-03T05:46:55.086 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.086 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.086 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.087 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.087 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.087 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.087 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:55.088 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.088 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.088 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.088 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.089 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.089 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.089 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.089 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.090 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183652.132077_188.1", 2022-09-03T05:46:55.090 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.090 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21", 2022-09-03T05:46:55.090 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:52.132077397Z", 2022-09-03T05:46:55.091 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.091 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.091 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.091 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.092 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.092 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.092 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:55.092 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.093 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.093 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.093 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.093 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.094 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.094 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.094 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.094 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183652.134378_189.1", 2022-09-03T05:46:55.095 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.095 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-21", 2022-09-03T05:46:55.095 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:52.134378414Z", 2022-09-03T05:46:55.095 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.096 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.096 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.096 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.096 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.097 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.097 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_EARZCit60fDg0eMLHHZC990", 2022-09-03T05:46:55.097 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.097 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.098 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.098 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.098 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.099 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.099 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.099 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.099 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183652.140411_190.1", 2022-09-03T05:46:55.100 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.100 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-21", 2022-09-03T05:46:55.100 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:52.140411296Z", 2022-09-03T05:46:55.100 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.101 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.101 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.101 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.101 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.102 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.102 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_EARZCit60fDg0eMLHHZC990", 2022-09-03T05:46:55.102 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.102 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.103 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.103 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.103 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.103 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.104 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.104 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.104 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.162940_191.1", 2022-09-03T05:46:55.104 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.105 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22", 2022-09-03T05:46:55.105 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.162940466Z", 2022-09-03T05:46:55.105 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.105 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.106 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.106 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.106 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.106 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.107 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:55.107 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.107 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.108 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.108 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.108 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.108 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.109 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.109 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.109 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.167017_192.1", 2022-09-03T05:46:55.109 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.110 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22", 2022-09-03T05:46:55.110 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.167017131Z", 2022-09-03T05:46:55.110 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.110 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.111 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.111 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.111 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.111 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.112 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:55.112 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.112 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.112 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.113 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.113 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.113 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.113 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.114 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.114 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.167997_193.1", 2022-09-03T05:46:55.114 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.114 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-22", 2022-09-03T05:46:55.115 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.167997739Z", 2022-09-03T05:46:55.115 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.115 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.115 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.116 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.116 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.116 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.116 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tI1b7-ygEU1TnWToSDcKGBc", 2022-09-03T05:46:55.117 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.117 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.117 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.117 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.118 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.118 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.118 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.118 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.119 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.170492_194.1", 2022-09-03T05:46:55.119 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.119 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-22", 2022-09-03T05:46:55.119 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.170492762Z", 2022-09-03T05:46:55.120 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.120 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.120 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.120 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.121 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.121 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.121 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tI1b7-ygEU1TnWToSDcKGBc", 2022-09-03T05:46:55.121 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.122 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.122 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.122 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.122 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.123 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.123 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.123 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.123 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.220411_195.1", 2022-09-03T05:46:55.124 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.124 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23", 2022-09-03T05:46:55.124 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.220411899Z", 2022-09-03T05:46:55.124 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.125 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.125 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.125 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.125 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.126 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.126 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:55.126 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.126 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.127 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.127 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.127 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.127 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.128 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.128 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.128 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.223172_196.1", 2022-09-03T05:46:55.128 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.129 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23", 2022-09-03T05:46:55.129 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.223172774Z", 2022-09-03T05:46:55.129 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.129 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.130 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.130 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.130 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.130 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.131 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:55.131 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.131 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.131 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.132 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.132 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.132 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.133 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.133 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.133 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.224266_197.1", 2022-09-03T05:46:55.133 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.134 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-23", 2022-09-03T05:46:55.134 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.224266788Z", 2022-09-03T05:46:55.134 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.134 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.135 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.135 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.135 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.135 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.136 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_mrTr4jlOIDprI-5OTphGZ6h", 2022-09-03T05:46:55.136 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.136 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.136 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.137 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.137 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.137 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.137 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.138 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.138 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.226840_198.1", 2022-09-03T05:46:55.138 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.138 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-23", 2022-09-03T05:46:55.139 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.226840552Z", 2022-09-03T05:46:55.139 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.139 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.139 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.140 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.140 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.140 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.140 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_mrTr4jlOIDprI-5OTphGZ6h", 2022-09-03T05:46:55.141 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.141 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.141 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.141 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.142 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.142 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.142 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.143 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.143 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.267809_199.1", 2022-09-03T05:46:55.143 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.143 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21", 2022-09-03T05:46:55.143 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.267809597Z", 2022-09-03T05:46:55.144 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.144 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.144 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:55.144 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.145 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.145 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.145 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:55.145 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.146 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.146 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.146 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.147 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.147 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.147 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.147 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.148 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.270381_200.1", 2022-09-03T05:46:55.148 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.148 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21", 2022-09-03T05:46:55.148 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.270381683Z", 2022-09-03T05:46:55.149 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.149 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.149 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:55.149 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.150 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.150 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.150 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:55.150 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.151 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.151 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.151 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.151 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.152 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.152 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.152 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.152 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.283633_201.1", 2022-09-03T05:46:55.153 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.153 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22", 2022-09-03T05:46:55.153 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.283633845Z", 2022-09-03T05:46:55.153 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.154 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.154 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:55.154 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.154 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.155 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.155 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:55.155 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.155 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.156 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.156 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.156 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.156 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.156 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.157 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.157 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.288001_202.1", 2022-09-03T05:46:55.157 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.157 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22", 2022-09-03T05:46:55.158 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.288001351Z", 2022-09-03T05:46:55.158 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.158 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.158 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:55.159 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.159 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.159 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.159 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:55.160 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.160 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.160 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.160 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.161 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.161 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.161 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.161 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.162 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.304152_203.1", 2022-09-03T05:46:55.162 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.162 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23", 2022-09-03T05:46:55.162 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.304152195Z", 2022-09-03T05:46:55.163 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.163 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.163 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:55.163 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.164 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.164 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.164 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:55.164 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.165 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.165 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.165 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.165 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.166 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.166 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.166 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.166 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.308405_204.1", 2022-09-03T05:46:55.167 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.167 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23", 2022-09-03T05:46:55.167 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.308405313Z", 2022-09-03T05:46:55.167 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.168 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.168 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:55.168 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.168 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.169 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.169 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:55.169 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.169 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.170 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.170 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.170 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.170 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.171 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.171 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.171 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.657291_207.1", 2022-09-03T05:46:55.171 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.172 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:55.172 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.657291334Z", 2022-09-03T05:46:55.172 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.172 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.173 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.173 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.173 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.173 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.174 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:55.174 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.174 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.174 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.175 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.175 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.175 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.176 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.176 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.176 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.661364_208.1", 2022-09-03T05:46:55.176 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.177 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:55.177 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.661364231Z", 2022-09-03T05:46:55.177 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.177 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.178 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.178 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.178 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.178 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.179 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:55.179 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.179 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.179 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.180 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.180 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.180 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.180 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.181 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.181 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.662317_209.1", 2022-09-03T05:46:55.181 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.181 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-24", 2022-09-03T05:46:55.182 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.662317133Z", 2022-09-03T05:46:55.182 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.182 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.182 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.183 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.183 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.183 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.183 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_RWbG2F3yjlSXFQy-Qq_ON1N", 2022-09-03T05:46:55.184 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.184 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.184 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.184 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.185 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.185 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.185 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.185 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.186 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.666554_210.1", 2022-09-03T05:46:55.186 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.186 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-24", 2022-09-03T05:46:55.186 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.666554708Z", 2022-09-03T05:46:55.187 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.187 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.187 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.188 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.188 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.188 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.188 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_RWbG2F3yjlSXFQy-Qq_ON1N", 2022-09-03T05:46:55.189 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.189 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.189 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.189 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.190 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.190 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.190 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.190 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.191 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.685377_211.1", 2022-09-03T05:46:55.191 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.191 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:55.191 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.685377688Z", 2022-09-03T05:46:55.192 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.192 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.192 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.192 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.193 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.193 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.193 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:55.194 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.194 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.194 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.194 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.195 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.195 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.195 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.195 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.196 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.689809_212.1", 2022-09-03T05:46:55.196 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.196 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:55.196 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.689809673Z", 2022-09-03T05:46:55.197 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.197 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.197 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.197 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.198 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.198 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.198 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:55.198 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.199 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.199 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.199 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.199 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.200 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.200 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.200 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.200 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.690762_213.1", 2022-09-03T05:46:55.201 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.201 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-25", 2022-09-03T05:46:55.201 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.690762414Z", 2022-09-03T05:46:55.202 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.202 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.202 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.203 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.203 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.203 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.203 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_IaGw1AfiCAqC-aV2Oup3OCi", 2022-09-03T05:46:55.204 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.204 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.204 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.205 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.205 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.205 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.205 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.206 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.206 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.693239_214.1", 2022-09-03T05:46:55.206 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.206 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-25", 2022-09-03T05:46:55.207 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.693239599Z", 2022-09-03T05:46:55.207 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.207 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.207 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.208 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.208 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.208 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.208 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_IaGw1AfiCAqC-aV2Oup3OCi", 2022-09-03T05:46:55.209 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.209 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.209 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.209 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.210 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.210 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.210 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.210 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.211 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.727704_215.1", 2022-09-03T05:46:55.211 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.211 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:55.211 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.727704277Z", 2022-09-03T05:46:55.212 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.212 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.212 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.212 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.213 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.213 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.213 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:55.213 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.214 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.214 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.214 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.214 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.215 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.215 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.215 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.215 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.731553_216.1", 2022-09-03T05:46:55.216 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.216 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:55.216 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.731553123Z", 2022-09-03T05:46:55.216 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.217 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.217 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.217 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.217 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.217 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.218 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:55.218 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.218 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.219 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.219 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.219 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.219 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.220 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.220 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.220 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.732533_217.1", 2022-09-03T05:46:55.220 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.221 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-26", 2022-09-03T05:46:55.221 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.732533246Z", 2022-09-03T05:46:55.221 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.221 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.222 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.222 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.222 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.222 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.223 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_dUSEiciichvry8cYc4lxW96", 2022-09-03T05:46:55.223 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.223 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.223 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.224 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.224 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.224 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.224 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.225 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.225 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.734903_218.1", 2022-09-03T05:46:55.225 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.225 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-26", 2022-09-03T05:46:55.226 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.734903969Z", 2022-09-03T05:46:55.226 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.226 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.226 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.227 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.227 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.227 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.227 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_dUSEiciichvry8cYc4lxW96", 2022-09-03T05:46:55.228 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.228 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.228 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.228 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.229 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.229 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.229 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.229 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.230 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.763839_219.1", 2022-09-03T05:46:55.230 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.230 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:55.230 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.763839115Z", 2022-09-03T05:46:55.231 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.231 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.231 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:55.231 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.232 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.232 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.232 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:55.232 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.233 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.233 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.233 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.233 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.234 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.234 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.234 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.234 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.766491_220.1", 2022-09-03T05:46:55.235 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.235 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:55.235 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.766491425Z", 2022-09-03T05:46:55.235 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.236 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.236 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:55.236 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.236 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.237 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.237 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:55.237 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.237 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.238 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.238 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.238 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.238 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.239 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.239 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.239 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.780440_221.1", 2022-09-03T05:46:55.239 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.240 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:55.240 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.780440878Z", 2022-09-03T05:46:55.240 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.240 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.241 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:55.241 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.241 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.241 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.242 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:55.242 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 3 2022-09-03T05:46:55.242 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.242 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.243 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.243 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.243 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.243 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.244 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.244 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.784428_222.1", 2022-09-03T05:46:55.244 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.244 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:55.245 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.784428128Z", 2022-09-03T05:46:55.245 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.245 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.245 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:55.246 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.246 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.246 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.246 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:55.247 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 3 2022-09-03T05:46:55.247 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.247 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.247 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.248 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.248 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.248 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.248 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.249 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.798318_223.1", 2022-09-03T05:46:55.249 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.249 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:55.249 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.798318959Z", 2022-09-03T05:46:55.250 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.250 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.250 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:55.250 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.251 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.251 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.251 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:55.251 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.252 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.252 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.252 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.252 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.253 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.253 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.253 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.253 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.802397_224.1", 2022-09-03T05:46:55.254 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.254 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:55.254 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.802397402Z", 2022-09-03T05:46:55.254 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.255 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.255 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:55.255 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.255 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.256 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.256 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:55.256 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.256 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.257 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.257 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.257 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.257 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.258 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.258 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.258 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.818536_225.1", 2022-09-03T05:46:55.258 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.259 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:55.259 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.818536670Z", 2022-09-03T05:46:55.259 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.259 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.260 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:55.260 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.260 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.260 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.261 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:55.261 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 3 2022-09-03T05:46:55.261 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.261 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.262 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.262 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.262 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.262 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.263 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.263 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.826419_226.1", 2022-09-03T05:46:55.263 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.263 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:55.264 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.826419304Z", 2022-09-03T05:46:55.264 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.264 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.264 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:55.265 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.265 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.265 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.265 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:55.266 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 3 2022-09-03T05:46:55.266 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.266 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.266 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.267 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.267 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.267 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.267 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.268 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.854560_227.1", 2022-09-03T05:46:55.268 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.268 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:55.268 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.854560270Z", 2022-09-03T05:46:55.269 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.269 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.269 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:55.269 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.270 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.270 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.271 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:55.271 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.271 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.271 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.272 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.272 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.272 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.272 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.273 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.273 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.857527_228.1", 2022-09-03T05:46:55.273 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.273 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:55.274 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.857527676Z", 2022-09-03T05:46:55.274 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.274 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.274 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:55.275 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.275 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.275 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.275 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:55.276 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.276 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.276 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.276 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.277 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.277 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.277 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.277 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.278 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.870879_229.1", 2022-09-03T05:46:55.278 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.278 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:55.278 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.870879960Z", 2022-09-03T05:46:55.279 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.279 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.279 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:55.279 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.280 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.280 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.280 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:55.280 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 3 2022-09-03T05:46:55.281 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.281 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.281 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.281 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.282 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.282 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.282 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.282 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.876923_230.1", 2022-09-03T05:46:55.283 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.283 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:55.283 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.876923670Z", 2022-09-03T05:46:55.283 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.284 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.284 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:55.284 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.284 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.285 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.285 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:55.285 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 3 2022-09-03T05:46:55.285 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.286 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.286 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.286 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.287 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.287 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.287 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.287 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.870437_232.1", 2022-09-03T05:46:55.288 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.288 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-27:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.27", 2022-09-03T05:46:55.288 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.870437527Z", 2022-09-03T05:46:55.288 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.289 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.289 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.289 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.289 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.290 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.290 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:55.290 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.290 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.291 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.291 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.291 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.292 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.292 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.292 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.292 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.874799_233.1", 2022-09-03T05:46:55.293 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.293 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-27:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.27", 2022-09-03T05:46:55.293 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.874799084Z", 2022-09-03T05:46:55.293 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.294 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.294 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.294 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.294 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.295 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.295 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:55.295 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.295 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.296 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.296 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.296 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.296 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.297 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.297 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.297 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.876797_234.1", 2022-09-03T05:46:55.297 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.298 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-27", 2022-09-03T05:46:55.298 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.876797147Z", 2022-09-03T05:46:55.298 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.299 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.299 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.299 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.299 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.299 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.300 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_kvVMhGyMiz3kusJkbwnyhxJ", 2022-09-03T05:46:55.300 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.300 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.300 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.301 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.301 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.301 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.301 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.302 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.302 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.885232_235.1", 2022-09-03T05:46:55.302 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.303 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-27", 2022-09-03T05:46:55.303 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.885232353Z", 2022-09-03T05:46:55.303 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.303 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.304 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.304 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.304 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.304 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.306 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_kvVMhGyMiz3kusJkbwnyhxJ", 2022-09-03T05:46:55.306 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.307 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.307 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.308 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.308 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.309 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.309 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.310 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.310 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.916915_236.1", 2022-09-03T05:46:55.311 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.311 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28", 2022-09-03T05:46:55.312 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.916915113Z", 2022-09-03T05:46:55.312 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.313 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.313 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.314 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.314 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.314 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.315 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:55.315 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.316 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.316 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.317 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.317 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.318 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.318 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.319 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.319 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.920958_237.1", 2022-09-03T05:46:55.320 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.320 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28", 2022-09-03T05:46:55.320 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.920958536Z", 2022-09-03T05:46:55.321 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.321 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.322 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.322 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.323 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.323 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.324 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:55.324 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.325 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.325 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.326 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.326 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.327 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.327 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.328 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.328 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.922045_238.1", 2022-09-03T05:46:55.328 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.329 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-28", 2022-09-03T05:46:55.329 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.922045913Z", 2022-09-03T05:46:55.330 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.330 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.331 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.331 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.332 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.332 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.333 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_OOWZg42AYED_QBCONdEGECB", 2022-09-03T05:46:55.333 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.334 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.334 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.334 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.335 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.335 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.336 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.336 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.336 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.926094_239.1", 2022-09-03T05:46:55.337 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.337 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-28", 2022-09-03T05:46:55.337 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.926094548Z", 2022-09-03T05:46:55.338 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.339 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.339 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.339 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.340 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.340 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.340 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_OOWZg42AYED_QBCONdEGECB", 2022-09-03T05:46:55.340 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.341 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.341 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.342 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.342 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.343 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.343 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.343 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.344 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.962517_240.1", 2022-09-03T05:46:55.344 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.345 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-29:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.29", 2022-09-03T05:46:55.345 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.962517073Z", 2022-09-03T05:46:55.346 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.346 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.347 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.347 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.347 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.347 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.348 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:55.348 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.348 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.348 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.349 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.349 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.349 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.349 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.350 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.350 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.965127_241.1", 2022-09-03T05:46:55.350 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.350 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-29:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.29", 2022-09-03T05:46:55.351 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.965127874Z", 2022-09-03T05:46:55.351 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.351 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.351 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.352 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.352 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.352 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.352 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:55.353 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.353 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.353 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.353 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.354 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.354 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.354 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.354 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.355 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.966081_242.1", 2022-09-03T05:46:55.355 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.355 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-29", 2022-09-03T05:46:55.355 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.966081501Z", 2022-09-03T05:46:55.356 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.356 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.356 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.356 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.357 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.357 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.357 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_yDkLaMdkLBW9LxbH_x72oyR", 2022-09-03T05:46:55.357 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.358 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.358 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.358 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.358 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.359 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.359 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.359 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.359 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.968432_243.1", 2022-09-03T05:46:55.360 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.360 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-29", 2022-09-03T05:46:55.360 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.968432661Z", 2022-09-03T05:46:55.360 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.361 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.361 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.361 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.361 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.362 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.362 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_yDkLaMdkLBW9LxbH_x72oyR", 2022-09-03T05:46:55.362 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.362 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.363 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.363 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.363 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.363 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.364 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.364 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.364 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.998970_244.1", 2022-09-03T05:46:55.364 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.365 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-27:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.27", 2022-09-03T05:46:55.365 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.998970343Z", 2022-09-03T05:46:55.365 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.365 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.366 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:55.366 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.366 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.366 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.367 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:55.367 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.367 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.367 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.368 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.368 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.368 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.368 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.369 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.369 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.001548_245.1", 2022-09-03T05:46:55.369 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.369 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-27:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.27", 2022-09-03T05:46:55.370 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.001548177Z", 2022-09-03T05:46:55.370 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.370 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.370 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:55.371 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.371 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.371 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.371 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:55.372 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.372 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.372 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.372 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.373 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.373 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.373 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.373 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.374 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.049458_246.1", 2022-09-03T05:46:55.374 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.374 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28", 2022-09-03T05:46:55.374 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.049458314Z", 2022-09-03T05:46:55.375 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.375 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.375 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:55.375 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.376 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.376 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.376 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:55.376 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.377 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.377 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.377 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.377 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.378 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.378 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.378 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.378 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.052066_247.1", 2022-09-03T05:46:55.379 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.379 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28", 2022-09-03T05:46:55.379 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.052066976Z", 2022-09-03T05:46:55.379 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.380 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.380 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:55.380 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.380 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.381 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.381 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:55.381 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.381 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.382 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.382 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.382 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.382 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.383 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.383 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.383 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.091593_248.1", 2022-09-03T05:46:55.383 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.383 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-29:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.29", 2022-09-03T05:46:55.384 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.091593357Z", 2022-09-03T05:46:55.384 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.384 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.384 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:55.385 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.385 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.385 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.385 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:55.386 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.386 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.386 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.386 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.387 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.387 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.387 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.387 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.388 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.099123_249.1", 2022-09-03T05:46:55.388 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.388 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-29:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.29", 2022-09-03T05:46:55.388 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.099123834Z", 2022-09-03T05:46:55.389 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.389 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.389 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:55.389 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.390 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.390 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.390 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:55.390 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.391 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.391 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.391 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.391 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.391 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.392 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.392 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.392 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.135078_250.1", 2022-09-03T05:46:55.392 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.393 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30", 2022-09-03T05:46:55.393 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.135078742Z", 2022-09-03T05:46:55.393 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.393 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.394 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.394 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.394 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.394 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.395 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:55.395 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.395 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.395 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.396 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.396 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.396 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.396 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.397 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.397 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.140570_251.1", 2022-09-03T05:46:55.397 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.397 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30", 2022-09-03T05:46:55.398 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.140570450Z", 2022-09-03T05:46:55.398 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.398 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.398 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.399 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.399 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.399 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.399 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:55.399 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.400 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.400 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.400 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.400 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.401 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.401 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.401 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.401 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.142697_252.1", 2022-09-03T05:46:55.402 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.402 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-30", 2022-09-03T05:46:55.402 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.142697729Z", 2022-09-03T05:46:55.402 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.403 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.403 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.403 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.403 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.404 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.404 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_4yhNaX-8bC2cTXp5LKWw-Qo", 2022-09-03T05:46:55.404 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.404 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.405 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.405 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.405 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.405 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.406 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.406 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.406 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.148860_253.1", 2022-09-03T05:46:55.406 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.407 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-30", 2022-09-03T05:46:55.407 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.148860761Z", 2022-09-03T05:46:55.407 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.407 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.408 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.408 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.408 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.408 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.409 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_4yhNaX-8bC2cTXp5LKWw-Qo", 2022-09-03T05:46:55.409 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.409 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.409 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.410 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.410 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.410 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.411 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.411 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.412 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.172332_254.1", 2022-09-03T05:46:55.412 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.412 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31", 2022-09-03T05:46:55.413 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.172332646Z", 2022-09-03T05:46:55.413 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.413 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.414 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.414 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.414 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.415 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.415 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:55.415 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.415 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.416 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.416 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.416 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.416 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.417 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.417 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.417 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.176046_255.1", 2022-09-03T05:46:55.417 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.418 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31", 2022-09-03T05:46:55.418 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.176046838Z", 2022-09-03T05:46:55.418 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.418 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.419 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.419 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.419 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.419 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.420 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:55.420 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.420 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.420 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.421 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.421 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.421 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.421 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.421 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.422 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.176958_256.1", 2022-09-03T05:46:55.422 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.422 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-31", 2022-09-03T05:46:55.422 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.176958510Z", 2022-09-03T05:46:55.423 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.423 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.423 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.423 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.424 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.424 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.424 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_oPtbK62ZjFZvpDTUcbzdHLT", 2022-09-03T05:46:55.424 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.425 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.425 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.425 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.425 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.426 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.426 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.426 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.426 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.179165_257.1", 2022-09-03T05:46:55.427 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.427 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-31", 2022-09-03T05:46:55.427 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.179165643Z", 2022-09-03T05:46:55.427 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.428 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.428 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.428 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.428 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.429 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.429 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_oPtbK62ZjFZvpDTUcbzdHLT", 2022-09-03T05:46:55.429 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.429 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.430 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.430 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.430 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.430 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.431 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.431 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.431 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.220311_258.1", 2022-09-03T05:46:55.431 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.432 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-32:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.32", 2022-09-03T05:46:55.432 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.220311791Z", 2022-09-03T05:46:55.432 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.432 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.433 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.433 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.433 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.433 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.433 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:55.434 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.434 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.434 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.435 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.435 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.435 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.435 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.436 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.222654_259.1", 2022-09-03T05:46:55.436 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.436 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-32:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.32", 2022-09-03T05:46:55.436 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.222654278Z", 2022-09-03T05:46:55.437 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.437 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.437 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.437 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.438 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.438 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.438 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:55.438 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.439 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.439 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.440 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.440 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.440 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.440 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.441 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.441 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.223513_260.1", 2022-09-03T05:46:55.441 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.441 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-32", 2022-09-03T05:46:55.442 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.223513282Z", 2022-09-03T05:46:55.442 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.442 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.442 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.443 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.443 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.443 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.443 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_y0oxyMaYTjSUF-S-HkK29E5", 2022-09-03T05:46:55.443 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.444 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.444 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.444 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.444 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.445 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.445 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.445 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.445 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.228241_261.1", 2022-09-03T05:46:55.446 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.446 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-32", 2022-09-03T05:46:55.446 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.228241236Z", 2022-09-03T05:46:55.447 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.447 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.447 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.447 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.448 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.448 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.448 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_y0oxyMaYTjSUF-S-HkK29E5", 2022-09-03T05:46:55.448 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.449 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.449 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.449 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.449 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.450 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.450 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.450 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.450 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.265477_262.1", 2022-09-03T05:46:55.451 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.451 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30", 2022-09-03T05:46:55.451 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.265477055Z", 2022-09-03T05:46:55.451 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.452 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.452 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:55.452 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.452 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.453 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.453 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:55.453 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.453 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.454 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.454 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.454 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.454 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.455 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.455 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.455 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.268146_263.1", 2022-09-03T05:46:55.455 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.456 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30", 2022-09-03T05:46:55.456 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.268146553Z", 2022-09-03T05:46:55.456 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.456 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.456 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:55.457 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.457 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.457 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.457 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:55.458 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.458 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.458 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.458 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.459 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.459 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.459 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.459 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.460 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.350018_264.1", 2022-09-03T05:46:55.460 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.460 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31", 2022-09-03T05:46:55.460 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.350018238Z", 2022-09-03T05:46:55.461 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.461 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.461 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:55.461 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.462 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.462 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.462 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:55.462 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.463 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.463 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.463 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.463 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.463 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.464 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.464 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.464 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.352665_265.1", 2022-09-03T05:46:55.464 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.465 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31", 2022-09-03T05:46:55.465 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.352665529Z", 2022-09-03T05:46:55.465 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.465 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.466 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:55.466 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.466 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.466 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.467 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:55.467 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.467 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.467 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.468 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.468 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.468 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.468 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.469 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.469 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.401743_266.1", 2022-09-03T05:46:55.469 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.469 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-32:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.32", 2022-09-03T05:46:55.470 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.401743084Z", 2022-09-03T05:46:55.470 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.470 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.470 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:55.471 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.471 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.471 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.471 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:55.472 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.472 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.472 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.472 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.473 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.473 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.473 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.473 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.474 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.404375_267.1", 2022-09-03T05:46:55.474 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.474 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-32:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.32", 2022-09-03T05:46:55.474 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.404375474Z", 2022-09-03T05:46:55.475 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.475 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.475 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:55.475 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.475 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.476 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.476 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:55.476 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.476 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.477 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.477 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.477 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.477 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.478 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.478 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.478 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.434979_268.1", 2022-09-03T05:46:55.478 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.479 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-33:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.33", 2022-09-03T05:46:55.479 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.434979477Z", 2022-09-03T05:46:55.479 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.479 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.480 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.480 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.480 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.480 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.481 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:55.481 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.481 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.481 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.482 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.482 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.482 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.482 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.483 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.483 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.438289_269.1", 2022-09-03T05:46:55.483 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.483 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-33:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.33", 2022-09-03T05:46:55.483 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.438289839Z", 2022-09-03T05:46:55.484 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.484 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.484 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.484 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.485 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.485 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.485 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:55.485 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.486 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.486 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.486 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.486 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.487 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.487 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.487 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.487 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.439390_270.1", 2022-09-03T05:46:55.488 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.488 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-33", 2022-09-03T05:46:55.488 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.439390666Z", 2022-09-03T05:46:55.488 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.489 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.489 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.489 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.489 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.490 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.490 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_XHlZf1wWm_Tr8FwCOGzkHwG", 2022-09-03T05:46:55.490 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.490 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.491 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.491 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.491 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.491 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.492 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.492 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.492 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.441665_271.1", 2022-09-03T05:46:55.492 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.492 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-33", 2022-09-03T05:46:55.493 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.441665178Z", 2022-09-03T05:46:55.493 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.493 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.493 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.494 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.494 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.494 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.494 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_XHlZf1wWm_Tr8FwCOGzkHwG", 2022-09-03T05:46:55.495 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.495 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.495 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.495 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.496 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.496 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.496 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.496 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.497 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.461635_272.1", 2022-09-03T05:46:55.497 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.497 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34", 2022-09-03T05:46:55.497 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.461635262Z", 2022-09-03T05:46:55.497 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.498 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.498 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.498 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.499 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.499 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.499 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:55.499 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.499 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.500 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.500 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.500 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.500 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.501 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.501 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.501 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.466010_273.1", 2022-09-03T05:46:55.501 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.502 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34", 2022-09-03T05:46:55.502 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.466010065Z", 2022-09-03T05:46:55.502 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.502 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.503 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.503 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.503 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.503 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.504 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:55.504 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.504 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.504 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.505 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.505 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.505 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.505 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.505 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.506 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.468430_274.1", 2022-09-03T05:46:55.506 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.506 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-34", 2022-09-03T05:46:55.506 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.468430282Z", 2022-09-03T05:46:55.507 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.507 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.507 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.507 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.508 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.508 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.508 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_8JLNrQKjOCrzmEAJH1Gqebc", 2022-09-03T05:46:55.508 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.509 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.509 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.509 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.509 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.510 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.510 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.510 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.510 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.473656_275.1", 2022-09-03T05:46:55.511 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.511 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-34", 2022-09-03T05:46:55.511 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.473656625Z", 2022-09-03T05:46:55.511 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.512 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.512 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.512 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.512 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.513 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.513 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_8JLNrQKjOCrzmEAJH1Gqebc", 2022-09-03T05:46:55.513 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.513 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.513 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.514 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.514 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.514 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.514 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.515 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.515 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.519749_276.1", 2022-09-03T05:46:55.515 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.515 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35", 2022-09-03T05:46:55.516 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.519749696Z", 2022-09-03T05:46:55.516 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.516 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.516 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.517 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.517 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.517 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.517 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:55.518 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.518 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.518 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.518 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.519 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.519 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.519 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.519 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.520 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.527797_277.1", 2022-09-03T05:46:55.520 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.520 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35", 2022-09-03T05:46:55.520 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.527797485Z", 2022-09-03T05:46:55.520 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.521 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.521 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.521 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.521 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.522 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.522 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:55.522 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.522 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.523 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.523 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.523 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.523 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.524 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.524 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.524 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.530086_278.1", 2022-09-03T05:46:55.524 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.525 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-35", 2022-09-03T05:46:55.525 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.530086284Z", 2022-09-03T05:46:55.525 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.525 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.526 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.526 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.526 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.526 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.527 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_9eV0dD_Np6i5r-IXItyYqq8", 2022-09-03T05:46:55.527 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.527 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.527 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.527 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.528 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.528 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.528 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.528 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.529 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.542414_279.1", 2022-09-03T05:46:55.529 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.529 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-35", 2022-09-03T05:46:55.529 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.542414804Z", 2022-09-03T05:46:55.530 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.530 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.530 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.530 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.531 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.531 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.531 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_9eV0dD_Np6i5r-IXItyYqq8", 2022-09-03T05:46:55.531 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.532 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.532 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.532 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.532 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.533 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.533 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.533 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.533 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.574433_280.1", 2022-09-03T05:46:55.534 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.534 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-33:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.33", 2022-09-03T05:46:55.534 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.574433191Z", 2022-09-03T05:46:55.534 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.535 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.535 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:55.535 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.535 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.535 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.536 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:55.536 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.536 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.536 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.537 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.537 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.537 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.537 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.538 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.538 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.577143_281.1", 2022-09-03T05:46:55.538 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.538 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-33:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.33", 2022-09-03T05:46:55.539 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.577143931Z", 2022-09-03T05:46:55.539 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.540 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.540 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:55.540 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.540 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.541 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.541 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:55.541 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.541 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.542 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.542 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.542 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.542 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.543 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.543 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.543 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183851.605820_282.1", 2022-09-03T05:46:55.543 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.544 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34", 2022-09-03T05:46:55.544 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:11.605820553Z", 2022-09-03T05:46:55.544 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.544 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.544 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:55.545 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.545 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.545 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.545 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:55.546 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.546 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.546 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.547 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.547 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.547 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.547 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.548 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.548 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183851.610169_283.1", 2022-09-03T05:46:55.548 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.548 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34", 2022-09-03T05:46:55.549 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:11.610169538Z", 2022-09-03T05:46:55.549 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.549 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.549 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:55.550 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.550 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.550 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.550 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:55.551 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.551 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.551 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.551 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.552 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.552 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.552 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.552 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.553 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.681884_284.1", 2022-09-03T05:46:55.553 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.553 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35", 2022-09-03T05:46:55.553 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.681884846Z", 2022-09-03T05:46:55.554 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.554 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.554 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:55.554 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.555 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.555 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.555 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:55.555 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.556 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.556 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.556 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.556 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.556 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.557 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.557 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.557 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.684548_285.1", 2022-09-03T05:46:55.557 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.558 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35", 2022-09-03T05:46:55.558 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.684548086Z", 2022-09-03T05:46:55.558 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.558 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.559 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:55.559 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.559 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.559 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.560 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:55.560 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.560 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.560 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.561 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.561 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.561 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.561 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.562 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.562 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.769697_286.1", 2022-09-03T05:46:55.562 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.562 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36", 2022-09-03T05:46:55.563 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.769697830Z", 2022-09-03T05:46:55.563 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.563 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.563 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.564 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.564 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.564 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.564 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lzXJaycHyP7aPeCDF57oxvc", 2022-09-03T05:46:55.565 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.565 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.565 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.565 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.565 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.566 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.566 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.566 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.566 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.780292_287.1", 2022-09-03T05:46:55.567 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.567 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36", 2022-09-03T05:46:55.567 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.780292446Z", 2022-09-03T05:46:55.567 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.568 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.568 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.568 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.568 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.569 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.569 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lzXJaycHyP7aPeCDF57oxvc", 2022-09-03T05:46:55.569 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.569 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.570 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.570 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.570 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.570 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.571 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.571 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.571 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.781473_288.1", 2022-09-03T05:46:55.571 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.572 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-36", 2022-09-03T05:46:55.572 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.781473404Z", 2022-09-03T05:46:55.572 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.572 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.573 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.573 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.573 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.573 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.573 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Aunq5-l6wG1IGbEHKmUcfjj", 2022-09-03T05:46:55.574 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.574 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.574 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.574 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.575 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.575 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.575 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.575 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.576 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.785261_289.1", 2022-09-03T05:46:55.576 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.576 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-36", 2022-09-03T05:46:55.576 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.785261123Z", 2022-09-03T05:46:55.577 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.577 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.577 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.577 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.578 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.578 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.578 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Aunq5-l6wG1IGbEHKmUcfjj", 2022-09-03T05:46:55.578 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.579 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.579 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.579 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.579 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.580 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.580 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.580 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.580 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.806309_290.1", 2022-09-03T05:46:55.580 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.581 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37", 2022-09-03T05:46:55.581 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.806309544Z", 2022-09-03T05:46:55.581 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.581 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.582 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.582 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.582 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.583 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.583 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_kuQ0xr50nHBW7YTCPQavDGz", 2022-09-03T05:46:55.583 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.583 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.584 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.584 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.584 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.584 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.585 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.585 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.585 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.808817_291.1", 2022-09-03T05:46:55.585 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.586 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37", 2022-09-03T05:46:55.586 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.808817564Z", 2022-09-03T05:46:55.586 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.586 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.587 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.587 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.587 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.587 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.588 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_kuQ0xr50nHBW7YTCPQavDGz", 2022-09-03T05:46:55.588 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.588 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.588 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.589 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.589 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.589 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.589 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.590 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.590 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.809764_292.1", 2022-09-03T05:46:55.590 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.590 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-37", 2022-09-03T05:46:55.591 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.809764987Z", 2022-09-03T05:46:55.591 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.591 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.591 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.592 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.592 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.592 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.592 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_2HiJKFit31XaN5gBsoarIgZ", 2022-09-03T05:46:55.593 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.593 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.593 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.593 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.594 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.594 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.594 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.594 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.595 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.814054_293.1", 2022-09-03T05:46:55.595 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.595 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-37", 2022-09-03T05:46:55.595 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.814054902Z", 2022-09-03T05:46:55.596 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.596 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.596 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.596 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.596 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.597 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.597 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_2HiJKFit31XaN5gBsoarIgZ", 2022-09-03T05:46:55.597 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.597 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.598 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.598 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.598 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.598 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.599 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.599 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.599 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.846705_294.1", 2022-09-03T05:46:55.599 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.600 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38", 2022-09-03T05:46:55.600 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.846705503Z", 2022-09-03T05:46:55.600 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.600 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.601 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.601 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.601 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.601 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.602 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_h95HWYhtSYo3Z6W42rIP0YJ", 2022-09-03T05:46:55.602 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.602 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.602 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.603 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.603 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.603 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.603 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.604 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.604 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.850680_295.1", 2022-09-03T05:46:55.604 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.604 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38", 2022-09-03T05:46:55.605 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.850680201Z", 2022-09-03T05:46:55.605 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.605 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.605 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.605 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.606 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.606 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.606 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_h95HWYhtSYo3Z6W42rIP0YJ", 2022-09-03T05:46:55.606 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.607 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.607 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.607 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.607 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.608 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.608 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.608 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.608 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.851766_296.1", 2022-09-03T05:46:55.609 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.609 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-38", 2022-09-03T05:46:55.609 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.851766916Z", 2022-09-03T05:46:55.609 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.610 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.610 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.610 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.610 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.611 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.611 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_K5Yz9Djeac7y4a5O_0TviMm", 2022-09-03T05:46:55.611 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.611 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.612 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.612 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.612 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.612 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.612 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.613 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.613 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.856692_297.1", 2022-09-03T05:46:55.613 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.613 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-38", 2022-09-03T05:46:55.614 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.856692289Z", 2022-09-03T05:46:55.614 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.614 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.614 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.615 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.615 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.615 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.615 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_K5Yz9Djeac7y4a5O_0TviMm", 2022-09-03T05:46:55.616 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.616 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.616 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.616 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.617 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.617 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.617 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.617 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.618 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.003533_298.1", 2022-09-03T05:46:55.618 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.618 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-39:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.39", 2022-09-03T05:46:55.618 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.003533677Z", 2022-09-03T05:46:55.619 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.619 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.619 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.619 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.620 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.620 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.620 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_LjHtI20g0C-nIIB1xqrV4kd", 2022-09-03T05:46:55.620 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.621 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.621 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.621 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.621 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.622 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.622 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.622 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.622 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.006401_299.1", 2022-09-03T05:46:55.623 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.623 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-39:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.39", 2022-09-03T05:46:55.623 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.006401221Z", 2022-09-03T05:46:55.623 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.623 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.624 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.624 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.624 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.624 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.625 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_LjHtI20g0C-nIIB1xqrV4kd", 2022-09-03T05:46:55.625 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.625 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.625 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.626 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.626 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.626 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.626 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.627 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.627 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.007446_300.1", 2022-09-03T05:46:55.627 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.627 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-39", 2022-09-03T05:46:55.628 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.007446652Z", 2022-09-03T05:46:55.628 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.628 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.628 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.629 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.629 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.629 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.629 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_LR1OrAXSVx6okJ40JV0Uh8F", 2022-09-03T05:46:55.630 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.630 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.630 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.630 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.631 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.631 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.631 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.631 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.631 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.010018_301.1", 2022-09-03T05:46:55.632 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.632 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-39", 2022-09-03T05:46:55.632 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.010018357Z", 2022-09-03T05:46:55.632 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.633 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.633 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.633 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.633 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.634 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.634 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_LR1OrAXSVx6okJ40JV0Uh8F", 2022-09-03T05:46:55.634 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.634 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.635 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.635 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.635 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.635 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.636 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.636 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.636 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.039601_302.1", 2022-09-03T05:46:55.636 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.637 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-40:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.40", 2022-09-03T05:46:55.637 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.039601239Z", 2022-09-03T05:46:55.637 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.637 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.638 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.638 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.638 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.638 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.639 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_SEgVmZRRIdgycWQGmgKhmpv", 2022-09-03T05:46:55.639 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.639 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.639 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.640 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.640 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.640 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.641 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.641 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.641 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.048071_303.1", 2022-09-03T05:46:55.641 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.642 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-40:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.40", 2022-09-03T05:46:55.642 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.048071799Z", 2022-09-03T05:46:55.642 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.642 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.643 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.643 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.643 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.643 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.644 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_SEgVmZRRIdgycWQGmgKhmpv", 2022-09-03T05:46:55.644 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.644 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.644 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.645 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.645 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.645 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.645 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.646 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.646 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.049107_304.1", 2022-09-03T05:46:55.646 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.646 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-40", 2022-09-03T05:46:55.647 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.049107658Z", 2022-09-03T05:46:55.647 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.647 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.647 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.648 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.648 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.648 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.648 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_e0zQngowI1boXIeM6KNEWZw", 2022-09-03T05:46:55.648 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.649 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.649 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.649 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.649 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.650 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.650 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.650 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.650 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.054911_305.1", 2022-09-03T05:46:55.651 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.651 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-40", 2022-09-03T05:46:55.651 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.054911391Z", 2022-09-03T05:46:55.651 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.652 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.652 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.652 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.652 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.653 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.653 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_e0zQngowI1boXIeM6KNEWZw", 2022-09-03T05:46:55.653 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.653 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.654 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.654 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.654 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.654 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.655 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.655 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.655 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.099806_306.1", 2022-09-03T05:46:55.655 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.656 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-41:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.41", 2022-09-03T05:46:55.656 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.099806646Z", 2022-09-03T05:46:55.656 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.656 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.657 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.657 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.657 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.657 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.657 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_M2clmHIwHAIIB9oEOnpIKkh", 2022-09-03T05:46:55.658 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.658 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.658 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.658 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.659 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.659 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.659 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.659 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.660 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.111286_307.1", 2022-09-03T05:46:55.660 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.660 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-41:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.41", 2022-09-03T05:46:55.660 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.111286094Z", 2022-09-03T05:46:55.661 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.661 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.661 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.662 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.662 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.662 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.662 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_M2clmHIwHAIIB9oEOnpIKkh", 2022-09-03T05:46:55.663 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.663 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.663 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.663 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.664 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.664 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.664 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.664 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.665 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.113514_308.1", 2022-09-03T05:46:55.665 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.665 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-41", 2022-09-03T05:46:55.665 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.113514552Z", 2022-09-03T05:46:55.666 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.666 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.666 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.666 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.667 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.667 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.667 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_kKVUMELn0ub6gAm6SEjIkvZ", 2022-09-03T05:46:55.667 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.668 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.668 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.668 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.668 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.669 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.669 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.669 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.670 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.119538_309.1", 2022-09-03T05:46:55.670 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.670 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-41", 2022-09-03T05:46:55.670 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.119538060Z", 2022-09-03T05:46:55.671 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.671 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.671 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.671 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.672 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.672 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.672 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_kKVUMELn0ub6gAm6SEjIkvZ", 2022-09-03T05:46:55.672 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.673 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.673 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.673 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.673 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.674 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.674 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.674 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.675 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182414.943600_19.1", 2022-09-03T05:46:55.675 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.675 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-1:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.1", 2022-09-03T05:46:55.675 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:14.943600569Z", 2022-09-03T05:46:55.676 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.676 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.676 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.676 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.676 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.677 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.677 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_rq1mS4spn__TV-QX0MQQJUU", 2022-09-03T05:46:55.677 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.677 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.678 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.678 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.678 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.678 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.679 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.679 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.679 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182414.959460_20.1", 2022-09-03T05:46:55.679 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.680 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-1:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.1", 2022-09-03T05:46:55.680 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:14.959460360Z", 2022-09-03T05:46:55.680 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.680 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.681 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.681 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.681 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.681 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.682 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_rq1mS4spn__TV-QX0MQQJUU", 2022-09-03T05:46:55.682 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.682 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.682 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.683 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.683 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.683 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.683 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.684 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.684 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182414.961669_21.1", 2022-09-03T05:46:55.684 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.684 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-1", 2022-09-03T05:46:55.685 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:14.961669762Z", 2022-09-03T05:46:55.685 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.685 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.685 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.686 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.686 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.686 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.686 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_0bHoPHCFbjz75fM89k7kJ9J", 2022-09-03T05:46:55.687 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.687 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.687 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.687 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.687 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.688 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.688 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.688 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.688 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182414.972666_22.1", 2022-09-03T05:46:55.689 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.689 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-1", 2022-09-03T05:46:55.689 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:14.972666586Z", 2022-09-03T05:46:55.689 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.690 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.690 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.690 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.690 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.691 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.691 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_0bHoPHCFbjz75fM89k7kJ9J", 2022-09-03T05:46:55.691 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.691 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.692 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.692 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.692 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.692 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.693 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.693 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.693 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182415.003071_23.1", 2022-09-03T05:46:55.693 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.694 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-2:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.2", 2022-09-03T05:46:55.694 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:15.003071656Z", 2022-09-03T05:46:55.694 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.694 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.695 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.695 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.695 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.695 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.696 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tfkUve-GLYp4UeLNm-4NFHF", 2022-09-03T05:46:55.696 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.696 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.696 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.696 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.697 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.697 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.697 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.697 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.698 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182415.009391_24.1", 2022-09-03T05:46:55.698 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.698 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-2:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.2", 2022-09-03T05:46:55.698 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:15.009391132Z", 2022-09-03T05:46:55.699 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.699 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.699 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.699 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.700 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.700 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.700 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tfkUve-GLYp4UeLNm-4NFHF", 2022-09-03T05:46:55.700 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.701 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.701 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.701 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.701 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.702 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.702 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.702 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.702 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182415.013579_25.1", 2022-09-03T05:46:55.703 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.703 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-2", 2022-09-03T05:46:55.703 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:15.013579974Z", 2022-09-03T05:46:55.703 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.704 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.704 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.704 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.704 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.705 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.705 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_ach1wph5n7_8QKTUZF1Mary", 2022-09-03T05:46:55.705 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.705 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.706 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.706 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.706 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.706 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.707 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.707 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.707 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182415.025150_26.1", 2022-09-03T05:46:55.707 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.708 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-2", 2022-09-03T05:46:55.708 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:15.025150228Z", 2022-09-03T05:46:55.708 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.708 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.709 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.709 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.709 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.709 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.710 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_ach1wph5n7_8QKTUZF1Mary", 2022-09-03T05:46:55.710 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.710 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.710 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.711 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.711 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.711 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.711 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.712 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.712 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182417.307770_27.1", 2022-09-03T05:46:55.712 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.712 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-3:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.3", 2022-09-03T05:46:55.712 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:17.307770555Z", 2022-09-03T05:46:55.713 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.713 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.713 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.714 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.714 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.714 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.714 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_RKS6tzrI_dmvQpvUbloi7Q_", 2022-09-03T05:46:55.715 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.715 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.715 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.715 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.716 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.716 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.716 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.716 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.717 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182417.312081_28.1", 2022-09-03T05:46:55.717 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.717 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-3:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.3", 2022-09-03T05:46:55.717 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:17.312081255Z", 2022-09-03T05:46:55.718 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.718 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.718 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.718 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.719 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.719 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.719 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_RKS6tzrI_dmvQpvUbloi7Q_", 2022-09-03T05:46:55.719 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.720 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.720 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.720 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.720 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.721 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.721 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.721 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.721 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182417.313242_29.1", 2022-09-03T05:46:55.722 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.722 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-3", 2022-09-03T05:46:55.722 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:17.313242294Z", 2022-09-03T05:46:55.722 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.723 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.723 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.723 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.723 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.724 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.724 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lt8IAlx8xFKZXCBoGrP76MP", 2022-09-03T05:46:55.724 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.724 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.725 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.725 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.725 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.725 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.726 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.726 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.726 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182417.319609_30.1", 2022-09-03T05:46:55.726 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.727 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-3", 2022-09-03T05:46:55.727 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:17.319609012Z", 2022-09-03T05:46:55.727 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.727 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.728 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.728 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.728 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.728 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.729 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lt8IAlx8xFKZXCBoGrP76MP", 2022-09-03T05:46:55.729 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.729 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.729 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.730 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.730 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.730 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.730 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.731 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.731 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.152440_31.1", 2022-09-03T05:46:55.731 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.731 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-4:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.4", 2022-09-03T05:46:55.732 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.152440893Z", 2022-09-03T05:46:55.732 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.732 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.732 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.732 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.733 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.733 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.733 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_QMyoKwlK2G2NuxL_sp2_re5", 2022-09-03T05:46:55.733 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.734 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.734 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.734 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.734 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.735 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.735 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.735 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.735 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.158442_32.1", 2022-09-03T05:46:55.736 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.736 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-4:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.4", 2022-09-03T05:46:55.736 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.158442960Z", 2022-09-03T05:46:55.736 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.737 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.737 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.737 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.737 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.738 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.738 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_QMyoKwlK2G2NuxL_sp2_re5", 2022-09-03T05:46:55.738 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.738 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.739 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.739 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.739 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.739 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.740 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.740 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.741 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.159309_33.1", 2022-09-03T05:46:55.741 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.741 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-4", 2022-09-03T05:46:55.741 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.159309677Z", 2022-09-03T05:46:55.742 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.742 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.742 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.742 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.743 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.743 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.743 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_FxyCJRWlBEhmncaMF3-AMFD", 2022-09-03T05:46:55.743 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.744 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.744 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.744 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.744 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.745 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.745 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.745 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.745 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.163606_34.1", 2022-09-03T05:46:55.746 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.746 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-4", 2022-09-03T05:46:55.746 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.163606904Z", 2022-09-03T05:46:55.746 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.747 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.747 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.747 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.747 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.748 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.748 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_FxyCJRWlBEhmncaMF3-AMFD", 2022-09-03T05:46:55.748 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.748 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.749 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.749 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.749 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.749 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.750 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.750 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.750 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.181237_35.1", 2022-09-03T05:46:55.750 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.751 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-5:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.5", 2022-09-03T05:46:55.751 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.181237915Z", 2022-09-03T05:46:55.751 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.751 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.752 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.752 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.752 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.752 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.753 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_mye4oDj8XK7uSi7YUS75P0l", 2022-09-03T05:46:55.753 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.753 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.753 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.754 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.754 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.754 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.754 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.755 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.755 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.183507_36.1", 2022-09-03T05:46:55.755 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.755 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-5:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.5", 2022-09-03T05:46:55.756 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.183507081Z", 2022-09-03T05:46:55.756 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.756 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.756 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.757 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.757 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.757 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.757 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_mye4oDj8XK7uSi7YUS75P0l", 2022-09-03T05:46:55.758 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.758 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.758 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.758 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.759 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.759 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.759 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.759 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.760 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.184425_37.1", 2022-09-03T05:46:55.760 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.760 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-5", 2022-09-03T05:46:55.760 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.184425827Z", 2022-09-03T05:46:55.761 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.761 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.761 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.761 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.762 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.762 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.762 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_6QXcvNroiwRkEdVqi3Nf85-", 2022-09-03T05:46:55.762 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.763 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.763 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.763 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.763 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.764 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.764 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.764 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.764 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.186822_38.1", 2022-09-03T05:46:55.765 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.765 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-5", 2022-09-03T05:46:55.765 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.186822428Z", 2022-09-03T05:46:55.765 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.765 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.766 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.766 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.766 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.766 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.767 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_6QXcvNroiwRkEdVqi3Nf85-", 2022-09-03T05:46:55.767 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.767 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.767 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.768 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.768 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.768 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.768 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.769 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.769 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.221675_39.1", 2022-09-03T05:46:55.769 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.769 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-6:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.6", 2022-09-03T05:46:55.770 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.221675326Z", 2022-09-03T05:46:55.770 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.770 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.770 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.771 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.771 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.771 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.771 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bqgF7AreaM1kO2cBL_SkGsM", 2022-09-03T05:46:55.772 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.772 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.772 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.772 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.773 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.773 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.773 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.773 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.773 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.230423_40.1", 2022-09-03T05:46:55.774 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.774 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-6:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.6", 2022-09-03T05:46:55.774 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.230423417Z", 2022-09-03T05:46:55.774 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.775 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.775 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.775 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.775 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.776 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.776 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bqgF7AreaM1kO2cBL_SkGsM", 2022-09-03T05:46:55.776 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.776 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.777 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.777 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.777 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.777 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.778 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.778 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.778 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.235081_41.1", 2022-09-03T05:46:55.778 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.779 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-6", 2022-09-03T05:46:55.779 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.235081816Z", 2022-09-03T05:46:55.779 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.779 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.780 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.780 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.780 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.780 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.781 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_wbkk3MMRONgcegk23g-QDT1", 2022-09-03T05:46:55.781 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.781 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.781 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.781 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.782 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.782 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.782 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.782 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.783 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.238098_42.1", 2022-09-03T05:46:55.783 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.783 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-6", 2022-09-03T05:46:55.783 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.238098524Z", 2022-09-03T05:46:55.784 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.784 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.784 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.785 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.785 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.785 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.785 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_wbkk3MMRONgcegk23g-QDT1", 2022-09-03T05:46:55.786 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.786 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.786 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.786 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.787 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.787 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.787 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.787 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.788 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.064697_43.1", 2022-09-03T05:46:55.788 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.788 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7", 2022-09-03T05:46:55.788 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.064697048Z", 2022-09-03T05:46:55.789 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.789 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.789 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.789 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.790 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.790 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.790 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:55.790 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.791 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.791 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.791 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.791 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.792 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.792 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.792 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.792 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.070346_44.1", 2022-09-03T05:46:55.793 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.793 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7", 2022-09-03T05:46:55.793 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.070346892Z", 2022-09-03T05:46:55.793 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.794 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.794 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.794 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.794 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.794 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.795 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:55.795 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.795 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.795 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.796 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.796 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.796 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.796 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.797 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.797 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.071355_45.1", 2022-09-03T05:46:55.797 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.797 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-7", 2022-09-03T05:46:55.798 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.071355932Z", 2022-09-03T05:46:55.798 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.798 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.798 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.799 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.799 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.799 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.799 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:46:55.800 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.800 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.800 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.800 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.801 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.801 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.801 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.801 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.802 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.075532_46.1", 2022-09-03T05:46:55.802 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.802 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-7", 2022-09-03T05:46:55.802 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.075532031Z", 2022-09-03T05:46:55.803 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.803 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.803 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.803 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.804 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.804 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.804 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:46:55.804 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.805 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.805 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.805 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.805 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.806 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.806 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.806 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.806 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.102267_47.1", 2022-09-03T05:46:55.807 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.807 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-8:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.8", 2022-09-03T05:46:55.807 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.102267896Z", 2022-09-03T05:46:55.807 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.808 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.808 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.808 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.808 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.809 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.809 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:46:55.809 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.809 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.810 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.810 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.810 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.810 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.811 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.811 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.811 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.104611_48.1", 2022-09-03T05:46:55.811 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.812 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-8:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.8", 2022-09-03T05:46:55.812 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.104611563Z", 2022-09-03T05:46:55.812 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.812 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.813 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.813 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.813 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.813 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.814 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:46:55.814 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.814 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.814 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.815 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.815 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.815 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.815 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.815 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.816 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.105641_49.1", 2022-09-03T05:46:55.816 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.816 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-8", 2022-09-03T05:46:55.816 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.105641481Z", 2022-09-03T05:46:55.817 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.817 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.817 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.817 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.818 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.818 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.818 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:46:55.818 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.819 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.819 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.819 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.819 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.820 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.820 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.820 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.820 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.108076_50.1", 2022-09-03T05:46:55.821 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.821 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-8", 2022-09-03T05:46:55.821 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.108076436Z", 2022-09-03T05:46:55.821 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.821 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.822 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.822 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.822 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.822 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.823 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:46:55.823 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.823 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.823 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.824 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.824 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.825 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.825 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.825 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.825 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.135979_51.1", 2022-09-03T05:46:55.826 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.826 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9", 2022-09-03T05:46:55.826 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.135979206Z", 2022-09-03T05:46:55.827 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.827 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.827 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.827 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.827 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.828 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.828 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:46:55.828 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.828 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.829 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.829 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.829 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.829 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.830 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.830 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.830 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.138975_52.1", 2022-09-03T05:46:55.830 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.831 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9", 2022-09-03T05:46:55.831 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.138975472Z", 2022-09-03T05:46:55.831 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.831 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.832 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.832 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.832 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.832 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.833 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:46:55.833 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.833 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.833 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.834 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.834 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.834 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.834 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.835 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.835 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.139759_53.1", 2022-09-03T05:46:55.835 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.835 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-9", 2022-09-03T05:46:55.836 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.139759635Z", 2022-09-03T05:46:55.836 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.836 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.836 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.837 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.837 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.837 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.837 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:46:55.838 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.838 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.838 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.838 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.839 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.839 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.839 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.839 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.840 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.141916_54.1", 2022-09-03T05:46:55.840 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.840 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-9", 2022-09-03T05:46:55.841 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.141916666Z", 2022-09-03T05:46:55.841 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.841 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.842 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:55.842 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:55.842 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.842 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.843 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:46:55.843 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.843 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.843 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.844 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.844 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.844 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.844 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.845 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.845 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.782802_55.1", 2022-09-03T05:46:55.845 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.845 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7", 2022-09-03T05:46:55.846 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.782802362Z", 2022-09-03T05:46:55.846 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.846 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.846 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:55.847 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.847 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.847 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.847 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:55.848 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.848 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.848 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.848 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.849 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.849 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.849 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.849 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.850 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.785254_56.1", 2022-09-03T05:46:55.850 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.850 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7", 2022-09-03T05:46:55.850 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.785254883Z", 2022-09-03T05:46:55.851 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.851 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.851 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:55.851 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.852 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.852 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.852 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:55.852 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.853 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.853 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.853 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.853 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.854 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.854 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.854 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.854 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.796744_57.1", 2022-09-03T05:46:55.855 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.855 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-7", 2022-09-03T05:46:55.855 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.796744879Z", 2022-09-03T05:46:55.855 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.856 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.856 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:46:55.856 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.856 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.857 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.857 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:46:55.857 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.857 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.857 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.858 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "remove" 2022-09-03T05:46:55.858 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.858 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.858 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.859 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.859 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.798860_58.1", 2022-09-03T05:46:55.859 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.859 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-7", 2022-09-03T05:46:55.860 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.798860427Z", 2022-09-03T05:46:55.860 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.860 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.860 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:46:55.861 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.861 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.861 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.861 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:46:55.862 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.862 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.862 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.862 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.863 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.863 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.863 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.863 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.864 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.833037_59.1", 2022-09-03T05:46:55.864 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.864 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-8:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.8", 2022-09-03T05:46:55.864 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.833037165Z", 2022-09-03T05:46:55.865 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.865 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.865 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:46:55.865 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.866 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.866 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.866 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:46:55.866 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.867 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.867 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.867 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.867 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.868 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.868 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.868 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.868 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.835414_60.1", 2022-09-03T05:46:55.869 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.869 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-8:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.8", 2022-09-03T05:46:55.869 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.835414206Z", 2022-09-03T05:46:55.869 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.870 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.870 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:46:55.870 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.870 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.871 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.871 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:46:55.871 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.871 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.872 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.872 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.872 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.872 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.873 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.873 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.873 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.843078_61.1", 2022-09-03T05:46:55.873 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.874 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-8", 2022-09-03T05:46:55.874 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.843078156Z", 2022-09-03T05:46:55.874 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.874 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.875 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:46:55.875 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.875 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.875 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.876 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:46:55.876 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.876 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.876 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.876 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "remove" 2022-09-03T05:46:55.877 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.877 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.877 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.877 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.878 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.845207_62.1", 2022-09-03T05:46:55.878 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.878 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-8", 2022-09-03T05:46:55.878 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.845207527Z", 2022-09-03T05:46:55.879 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.879 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.879 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:46:55.879 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.880 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.880 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.880 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:46:55.880 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.881 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.881 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.881 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.881 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.882 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.882 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.882 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.882 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.909335_63.1", 2022-09-03T05:46:55.883 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.883 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9", 2022-09-03T05:46:55.883 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.909335713Z", 2022-09-03T05:46:55.883 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.884 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.884 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:46:55.884 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.884 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.885 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.885 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:46:55.885 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.885 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.885 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.886 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:55.886 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.887 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.887 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.887 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.888 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.911665_64.1", 2022-09-03T05:46:55.888 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:55.889 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9", 2022-09-03T05:46:55.889 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.911665732Z", 2022-09-03T05:46:55.889 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.890 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.890 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:46:55.891 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.891 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.892 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.892 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:46:55.892 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.892 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.893 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.893 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.894 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.894 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.895 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.895 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.896 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.924885_65.1", 2022-09-03T05:46:55.896 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.897 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-9", 2022-09-03T05:46:55.897 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.924885371Z", 2022-09-03T05:46:55.898 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.898 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.899 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:46:55.899 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.900 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.900 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.901 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:46:55.901 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.902 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.902 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.903 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "remove" 2022-09-03T05:46:55.903 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.905 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.906 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.906 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:55.907 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.928498_66.1", 2022-09-03T05:46:55.907 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:55.907 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-9", 2022-09-03T05:46:55.908 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498747Z", 2022-09-03T05:46:55.908 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:55.909 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:55.909 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:46:55.909 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:55.910 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.910 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:55.911 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:46:55.911 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:55.911 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:55.912 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:55.912 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:55.912 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.912 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.913 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:55.913 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:46:55.918 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '41ecd2a5-afc0-4eae-9349-7dfe39baa1cb', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:46:55.919 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '41ecd2a5-afc0-4eae-9349-7dfe39baa1cb', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:46:55.920 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 41ecd2a5-afc0-4eae-9349-7dfe39baa1cb --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:46:55.976 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:46:55.976 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:46:56.020 INFO:teuthology.orchestra.run.smithi189.stdout:[ 2022-09-03T05:46:56.020 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.020 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182298.903952_68.1", 2022-09-03T05:46:56.021 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "user", 2022-09-03T05:46:56.021 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "rgw-multisite-test-user", 2022-09-03T05:46:56.021 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.903952433Z", 2022-09-03T05:46:56.021 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.022 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.023 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.023 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.023 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.024 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.024 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_4zvH7kUaBnzKjfeQSSk_55d", 2022-09-03T05:46:56.024 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.024 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.025 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.025 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.025 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.025 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.026 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.026 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.026 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182298.906852_69.1", 2022-09-03T05:46:56.027 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "user", 2022-09-03T05:46:56.027 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "rgw-multisite-test-user", 2022-09-03T05:46:56.027 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852070Z", 2022-09-03T05:46:56.028 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.028 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.028 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.028 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.029 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.029 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.029 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_4zvH7kUaBnzKjfeQSSk_55d", 2022-09-03T05:46:56.029 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.030 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.030 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.030 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.030 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.031 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.031 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.031 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.032 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.694951_88.1", 2022-09-03T05:46:56.032 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.032 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10", 2022-09-03T05:46:56.032 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.694951214Z", 2022-09-03T05:46:56.033 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.033 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.034 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.034 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.034 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.034 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.035 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_9ORw1pyvfMIqMonbAbpO7T7", 2022-09-03T05:46:56.035 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.036 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.036 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.036 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.036 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.037 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.037 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.037 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.038 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.700326_89.1", 2022-09-03T05:46:56.038 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.038 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10", 2022-09-03T05:46:56.038 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.700326196Z", 2022-09-03T05:46:56.039 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.039 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.039 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.039 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.040 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.040 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.040 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_9ORw1pyvfMIqMonbAbpO7T7", 2022-09-03T05:46:56.040 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.041 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.041 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.041 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.041 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.042 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.042 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.042 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.043 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.702420_90.1", 2022-09-03T05:46:56.043 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.043 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-10", 2022-09-03T05:46:56.043 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.702420359Z", 2022-09-03T05:46:56.044 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.044 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.044 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.045 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.045 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.045 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.045 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "__K0-JaxveSMyJYue_ni7kIA", 2022-09-03T05:46:56.046 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.046 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.046 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.046 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.047 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.047 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.047 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.047 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.048 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.705433_91.1", 2022-09-03T05:46:56.048 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.049 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-10", 2022-09-03T05:46:56.049 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.705433036Z", 2022-09-03T05:46:56.049 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.049 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.050 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.050 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.050 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.051 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.051 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "__K0-JaxveSMyJYue_ni7kIA", 2022-09-03T05:46:56.051 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.051 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.052 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.052 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.052 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.052 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.053 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.053 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.053 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.724120_92.1", 2022-09-03T05:46:56.053 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.053 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11", 2022-09-03T05:46:56.054 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.724120949Z", 2022-09-03T05:46:56.054 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.054 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.055 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.055 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.055 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.056 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.056 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_mm_a_MgGT-mjAscVo5eDJN3", 2022-09-03T05:46:56.057 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.057 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.057 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.058 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.058 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.059 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.060 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.060 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.060 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.726647_93.1", 2022-09-03T05:46:56.061 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.061 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11", 2022-09-03T05:46:56.061 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.726647140Z", 2022-09-03T05:46:56.061 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.062 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.062 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.062 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.062 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.063 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.063 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_mm_a_MgGT-mjAscVo5eDJN3", 2022-09-03T05:46:56.063 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.063 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.064 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.064 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.064 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.065 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.065 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.065 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.065 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.727701_94.1", 2022-09-03T05:46:56.066 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.066 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-11", 2022-09-03T05:46:56.066 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.727701875Z", 2022-09-03T05:46:56.066 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.067 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.067 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.067 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.067 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.068 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.068 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_co69vPv41E29NQft_mgvTJ6", 2022-09-03T05:46:56.068 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.068 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.069 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.069 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.069 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.069 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.069 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.070 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.070 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.730089_95.1", 2022-09-03T05:46:56.071 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.071 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-11", 2022-09-03T05:46:56.071 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.730089458Z", 2022-09-03T05:46:56.071 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.072 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.072 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.072 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.072 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.073 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.073 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_co69vPv41E29NQft_mgvTJ6", 2022-09-03T05:46:56.073 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.073 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.074 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.074 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.075 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.075 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.075 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.075 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.076 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.760919_96.1", 2022-09-03T05:46:56.076 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.076 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12", 2022-09-03T05:46:56.076 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.760919958Z", 2022-09-03T05:46:56.076 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.077 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.077 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.077 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.078 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.078 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.078 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_87U_6FvKdHGzg3asBWz0TB6", 2022-09-03T05:46:56.078 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.079 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.079 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.079 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.079 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.080 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.080 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.080 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.080 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.763582_97.1", 2022-09-03T05:46:56.081 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.081 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12", 2022-09-03T05:46:56.081 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.763582008Z", 2022-09-03T05:46:56.082 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.082 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.082 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.083 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.083 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.083 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.083 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_87U_6FvKdHGzg3asBWz0TB6", 2022-09-03T05:46:56.084 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.084 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.085 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.085 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.086 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.086 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.086 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.087 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.764601_98.1", 2022-09-03T05:46:56.088 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.088 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-12", 2022-09-03T05:46:56.089 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.764601024Z", 2022-09-03T05:46:56.089 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.090 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.090 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.091 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.091 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.092 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.092 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_F5RlS3fhN6ILhiyoQ3qKUpC", 2022-09-03T05:46:56.093 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.093 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.094 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.094 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.095 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.095 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.095 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.096 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.096 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.766898_99.1", 2022-09-03T05:46:56.097 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.097 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-12", 2022-09-03T05:46:56.098 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.766898391Z", 2022-09-03T05:46:56.098 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.099 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.099 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.100 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.100 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.101 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.101 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_F5RlS3fhN6ILhiyoQ3qKUpC", 2022-09-03T05:46:56.101 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.101 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.102 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.102 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.102 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.103 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.103 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.104 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.104 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182745.944715_100.1", 2022-09-03T05:46:56.104 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.104 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13", 2022-09-03T05:46:56.105 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:45.944715403Z", 2022-09-03T05:46:56.105 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.106 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.106 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.107 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.108 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.108 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_ZwtCDnJJncqHbK6_XX4E-Uv", 2022-09-03T05:46:56.108 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.109 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.109 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.110 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.110 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.110 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.111 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.111 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182745.948069_101.1", 2022-09-03T05:46:56.112 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.112 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13", 2022-09-03T05:46:56.113 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:45.948069121Z", 2022-09-03T05:46:56.113 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.114 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.114 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.114 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.114 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.115 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.115 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_ZwtCDnJJncqHbK6_XX4E-Uv", 2022-09-03T05:46:56.115 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.116 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.116 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.116 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.116 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.117 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.117 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182745.950114_102.1", 2022-09-03T05:46:56.117 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.118 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-13", 2022-09-03T05:46:56.118 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:45.950114676Z", 2022-09-03T05:46:56.119 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.119 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.119 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.119 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.120 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.120 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_kkkmqlwBfjjtwLr8U95H7wF", 2022-09-03T05:46:56.120 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.121 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.121 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.121 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.122 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.122 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.123 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182745.953048_103.1", 2022-09-03T05:46:56.123 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.123 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-13", 2022-09-03T05:46:56.124 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:45.953048780Z", 2022-09-03T05:46:56.124 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.124 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.125 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.125 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.125 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.125 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.126 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_kkkmqlwBfjjtwLr8U95H7wF", 2022-09-03T05:46:56.126 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.127 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.127 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.128 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.128 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.129 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.129 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182745.988740_104.1", 2022-09-03T05:46:56.130 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.131 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14", 2022-09-03T05:46:56.131 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:45.988740832Z", 2022-09-03T05:46:56.132 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.132 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.133 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.133 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.134 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.135 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Yidm7pQHzXjzyowF0vXCv_q", 2022-09-03T05:46:56.135 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.136 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.137 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.138 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.138 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.139 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.139 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.140 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182745.991668_105.1", 2022-09-03T05:46:56.141 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.141 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14", 2022-09-03T05:46:56.141 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:45.991668492Z", 2022-09-03T05:46:56.142 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.142 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.142 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.143 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.144 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.144 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Yidm7pQHzXjzyowF0vXCv_q", 2022-09-03T05:46:56.145 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.146 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.146 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.147 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.147 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.148 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182745.992722_106.1", 2022-09-03T05:46:56.148 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.148 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-14", 2022-09-03T05:46:56.149 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:45.992722013Z", 2022-09-03T05:46:56.149 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.149 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.150 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.150 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.150 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.151 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_0ZpMZLjvQ0yH4NIki0Mppzv", 2022-09-03T05:46:56.151 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.152 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.152 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.152 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.152 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.153 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.153 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182745.995451_107.1", 2022-09-03T05:46:56.154 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.154 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-14", 2022-09-03T05:46:56.155 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:45.995451716Z", 2022-09-03T05:46:56.155 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.155 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.156 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.156 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.156 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.157 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_0ZpMZLjvQ0yH4NIki0Mppzv", 2022-09-03T05:46:56.157 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.157 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.157 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.158 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.158 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.158 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.159 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.159 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182746.030460_108.1", 2022-09-03T05:46:56.159 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.160 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15", 2022-09-03T05:46:56.160 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:46.030460591Z", 2022-09-03T05:46:56.160 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.161 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.161 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.161 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.162 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.162 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_9EbzjhdvUtwpRj9gUb6Zy_c", 2022-09-03T05:46:56.163 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.163 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.164 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.164 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.164 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.165 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182746.033416_109.1", 2022-09-03T05:46:56.166 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.166 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15", 2022-09-03T05:46:56.166 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:46.033416114Z", 2022-09-03T05:46:56.166 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.167 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.167 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.167 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.168 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.168 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_9EbzjhdvUtwpRj9gUb6Zy_c", 2022-09-03T05:46:56.169 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.169 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.170 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.170 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.170 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.170 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.171 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182746.035561_110.1", 2022-09-03T05:46:56.171 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.172 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-15", 2022-09-03T05:46:56.172 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:46.035561132Z", 2022-09-03T05:46:56.172 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.172 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.173 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.173 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.173 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.173 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.174 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_p1RJcPdLE0s_MQy_T3ZeW1Z", 2022-09-03T05:46:56.174 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.174 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.174 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.175 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.175 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.175 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.176 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182746.041492_111.1", 2022-09-03T05:46:56.176 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.177 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-15", 2022-09-03T05:46:56.177 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:46.041492579Z", 2022-09-03T05:46:56.177 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.177 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.178 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.178 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.179 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.179 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_p1RJcPdLE0s_MQy_T3ZeW1Z", 2022-09-03T05:46:56.179 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.180 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.180 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.180 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.180 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.181 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.181 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182851.903860_122.1", 2022-09-03T05:46:56.181 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.182 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16", 2022-09-03T05:46:56.182 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:31.903860682Z", 2022-09-03T05:46:56.182 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.183 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.183 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.183 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.183 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.184 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.184 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:56.184 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.185 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.185 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.185 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.185 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.186 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.186 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182851.907991_123.1", 2022-09-03T05:46:56.186 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.187 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16", 2022-09-03T05:46:56.187 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:31.907991084Z", 2022-09-03T05:46:56.187 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.187 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.188 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.188 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.188 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.189 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:56.189 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.189 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.190 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.190 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.190 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.190 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.191 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182851.909130_124.1", 2022-09-03T05:46:56.191 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.192 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-16", 2022-09-03T05:46:56.192 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:31.909130687Z", 2022-09-03T05:46:56.192 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.193 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.193 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.193 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.194 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.194 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_P0q3AfXqSQL8XW6gVnA4CRA", 2022-09-03T05:46:56.194 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.194 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.195 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.195 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.195 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.196 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.196 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.196 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.197 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182851.915199_125.1", 2022-09-03T05:46:56.197 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.197 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-16", 2022-09-03T05:46:56.197 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:31.915199302Z", 2022-09-03T05:46:56.198 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.198 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.198 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.199 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.199 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.199 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_P0q3AfXqSQL8XW6gVnA4CRA", 2022-09-03T05:46:56.200 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.200 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.200 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.201 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.201 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.201 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.202 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182851.945316_126.1", 2022-09-03T05:46:56.203 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.203 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17", 2022-09-03T05:46:56.203 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:31.945316218Z", 2022-09-03T05:46:56.203 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.204 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.204 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.204 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.205 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.205 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:56.206 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.206 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.207 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.207 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.207 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.208 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182851.950120_127.1", 2022-09-03T05:46:56.209 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.209 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17", 2022-09-03T05:46:56.209 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:31.950120909Z", 2022-09-03T05:46:56.210 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.210 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.210 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.210 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.211 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.211 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.211 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:56.212 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.212 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.212 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.213 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.213 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.214 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.214 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182851.952454_128.1", 2022-09-03T05:46:56.214 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.214 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-17", 2022-09-03T05:46:56.215 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:31.952454005Z", 2022-09-03T05:46:56.215 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.215 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.215 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.216 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.216 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.217 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_j4__iZ2_YM9iD4VCcQhy-lG", 2022-09-03T05:46:56.217 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.217 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.218 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.218 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.218 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.219 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.219 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182851.958649_129.1", 2022-09-03T05:46:56.219 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.220 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-17", 2022-09-03T05:46:56.220 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:31.958649291Z", 2022-09-03T05:46:56.220 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.220 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.221 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.221 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.221 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.222 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_j4__iZ2_YM9iD4VCcQhy-lG", 2022-09-03T05:46:56.222 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.223 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.223 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.223 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.224 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.224 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.225 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182851.998554_130.1", 2022-09-03T05:46:56.225 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.225 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18", 2022-09-03T05:46:56.226 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:31.998554746Z", 2022-09-03T05:46:56.226 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.226 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.227 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.227 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.227 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.228 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:56.228 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.229 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.229 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.229 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.229 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.230 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.230 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182852.004610_131.1", 2022-09-03T05:46:56.231 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.231 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18", 2022-09-03T05:46:56.231 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:32.004610066Z", 2022-09-03T05:46:56.231 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.231 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.232 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.232 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.232 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.233 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:56.233 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.233 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.234 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.234 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.234 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.235 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182852.007026_132.1", 2022-09-03T05:46:56.235 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.236 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-18", 2022-09-03T05:46:56.236 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:32.007026261Z", 2022-09-03T05:46:56.236 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.236 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.237 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.237 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.237 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.238 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_O7UdALq44EBf7qF8RLPxZMv", 2022-09-03T05:46:56.238 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.238 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.239 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.239 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.239 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.240 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182852.013139_133.1", 2022-09-03T05:46:56.240 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.240 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-18", 2022-09-03T05:46:56.241 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:32.013139549Z", 2022-09-03T05:46:56.241 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.241 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.241 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.242 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.242 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.242 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_O7UdALq44EBf7qF8RLPxZMv", 2022-09-03T05:46:56.243 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.243 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.244 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.244 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.244 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.245 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182852.069466_134.1", 2022-09-03T05:46:56.245 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.245 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16", 2022-09-03T05:46:56.246 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:32.069466103Z", 2022-09-03T05:46:56.246 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.246 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.246 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:56.247 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.247 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.247 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:56.248 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.248 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.249 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.249 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.250 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.250 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182852.078407_135.1", 2022-09-03T05:46:56.251 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.251 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16", 2022-09-03T05:46:56.251 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:32.078407455Z", 2022-09-03T05:46:56.251 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.252 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.252 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:56.252 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.253 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.253 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:56.253 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.254 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.254 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.254 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.255 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.255 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182852.095941_136.1", 2022-09-03T05:46:56.256 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.256 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17", 2022-09-03T05:46:56.256 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:32.095941153Z", 2022-09-03T05:46:56.257 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.257 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.257 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:56.258 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.259 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.259 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:56.259 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.260 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.260 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.261 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.261 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.262 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182852.099371_137.1", 2022-09-03T05:46:56.262 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.262 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17", 2022-09-03T05:46:56.263 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:32.099371547Z", 2022-09-03T05:46:56.263 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.263 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.264 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:56.264 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.265 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.265 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:56.265 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.266 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.266 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.267 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.267 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.268 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182852.121946_138.1", 2022-09-03T05:46:56.268 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.269 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18", 2022-09-03T05:46:56.269 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:32.121946745Z", 2022-09-03T05:46:56.269 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.269 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.270 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:56.270 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.271 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.271 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:56.271 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.272 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.272 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.272 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.272 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.273 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182852.124949_139.1", 2022-09-03T05:46:56.273 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.274 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18", 2022-09-03T05:46:56.274 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:32.124949763Z", 2022-09-03T05:46:56.274 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.274 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.275 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:56.275 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.275 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.276 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:56.276 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.276 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.277 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.277 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.277 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.278 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183242.240665_157.1", 2022-09-03T05:46:56.278 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.278 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19", 2022-09-03T05:46:56.279 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:34:02.240665028Z", 2022-09-03T05:46:56.279 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.279 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.279 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.280 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.280 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.280 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:56.281 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.281 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.282 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.282 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.282 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.283 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183242.243600_158.1", 2022-09-03T05:46:56.283 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.284 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19", 2022-09-03T05:46:56.284 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:34:02.243600484Z", 2022-09-03T05:46:56.284 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.284 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.285 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.285 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.285 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.286 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:56.286 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.287 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.287 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.287 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.287 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.288 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183242.244744_159.1", 2022-09-03T05:46:56.289 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.289 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-19", 2022-09-03T05:46:56.289 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:34:02.244744943Z", 2022-09-03T05:46:56.290 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.290 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.290 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.290 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.291 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.291 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_B_TWex9_lpCif6prNO7BbtO", 2022-09-03T05:46:56.292 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.292 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.292 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.293 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.293 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.294 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183242.247625_160.1", 2022-09-03T05:46:56.294 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.294 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-19", 2022-09-03T05:46:56.295 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:34:02.247625174Z", 2022-09-03T05:46:56.295 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.295 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.295 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.296 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.296 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.296 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_B_TWex9_lpCif6prNO7BbtO", 2022-09-03T05:46:56.297 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.297 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.297 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.298 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.298 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.299 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183242.265141_161.1", 2022-09-03T05:46:56.299 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.299 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19", 2022-09-03T05:46:56.300 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:34:02.265141968Z", 2022-09-03T05:46:56.300 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.300 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.300 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:56.301 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.301 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.301 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:56.302 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.302 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.302 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.303 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.303 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.303 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.304 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183242.267929_162.1", 2022-09-03T05:46:56.304 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.304 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19", 2022-09-03T05:46:56.304 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:34:02.267929270Z", 2022-09-03T05:46:56.305 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.305 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.305 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:56.305 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.306 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.306 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:56.306 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.307 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.307 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.307 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.308 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.308 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183458.677214_163.1", 2022-09-03T05:46:56.309 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.309 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:56.309 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:37:38.677214984Z", 2022-09-03T05:46:56.310 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.310 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.310 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.310 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.311 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.311 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:56.311 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.312 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.312 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.312 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.313 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.313 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.314 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183458.684800_164.1", 2022-09-03T05:46:56.314 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.314 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:56.314 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:37:38.684800313Z", 2022-09-03T05:46:56.315 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.315 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.315 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.315 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.316 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.316 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:56.316 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.317 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.317 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.317 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.317 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.317 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.318 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183458.685753_165.1", 2022-09-03T05:46:56.319 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.319 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-20", 2022-09-03T05:46:56.319 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:37:38.685753497Z", 2022-09-03T05:46:56.319 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.320 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.320 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.320 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.321 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.321 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.321 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_yPpDuhKk6Xsxh9Gd4Ds8exJ", 2022-09-03T05:46:56.321 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.321 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.322 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.322 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.322 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.322 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.323 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.323 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183458.694130_166.1", 2022-09-03T05:46:56.323 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.324 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-20", 2022-09-03T05:46:56.324 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:37:38.694130749Z", 2022-09-03T05:46:56.324 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.324 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.325 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.325 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.325 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.325 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.326 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_yPpDuhKk6Xsxh9Gd4Ds8exJ", 2022-09-03T05:46:56.326 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.326 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.327 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.327 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.327 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.327 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.328 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.328 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183531.352224_183.1", 2022-09-03T05:46:56.328 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.328 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:56.329 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:38:51.352224643Z", 2022-09-03T05:46:56.329 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.329 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.329 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:56.330 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.330 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.330 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.330 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:56.331 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.331 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.331 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.331 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.332 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.332 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.332 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.332 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.333 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183531.355467_184.1", 2022-09-03T05:46:56.333 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.333 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:56.334 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:38:51.355467679Z", 2022-09-03T05:46:56.334 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.334 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.334 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:56.335 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.335 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.335 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.336 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:56.336 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.336 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.336 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.337 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.337 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.337 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.337 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.338 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.338 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183570.365072_185.1", 2022-09-03T05:46:56.338 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.339 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:56.339 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:39:30.365072646Z", 2022-09-03T05:46:56.339 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.339 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.340 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:56.340 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.340 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.341 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.341 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:56.341 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 3 2022-09-03T05:46:56.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.342 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.342 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.342 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.343 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.343 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.343 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.343 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183570.368454_186.1", 2022-09-03T05:46:56.344 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.344 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:56.344 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:39:30.368454229Z", 2022-09-03T05:46:56.344 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.345 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.345 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:56.345 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.345 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.346 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:56.346 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 3 2022-09-03T05:46:56.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.347 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.347 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.347 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.347 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.348 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.348 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.348 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183652.125862_187.1", 2022-09-03T05:46:56.348 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.349 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21", 2022-09-03T05:46:56.349 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:52.125862114Z", 2022-09-03T05:46:56.349 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.350 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.350 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.350 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.351 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.351 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.351 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:56.351 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.352 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.352 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.352 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.352 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.353 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.353 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.353 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183652.132077_188.1", 2022-09-03T05:46:56.354 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.355 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21", 2022-09-03T05:46:56.355 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:52.132077397Z", 2022-09-03T05:46:56.356 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.356 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.356 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.356 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.357 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.357 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.357 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:56.358 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.358 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.358 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.358 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.359 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.359 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.359 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.359 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.360 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183652.134378_189.1", 2022-09-03T05:46:56.360 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.360 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-21", 2022-09-03T05:46:56.360 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:52.134378414Z", 2022-09-03T05:46:56.361 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.361 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.362 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.362 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.362 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.363 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_EARZCit60fDg0eMLHHZC990", 2022-09-03T05:46:56.363 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.363 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.363 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.364 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.364 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.364 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.364 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.365 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.365 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183652.140411_190.1", 2022-09-03T05:46:56.365 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.366 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-21", 2022-09-03T05:46:56.366 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:52.140411296Z", 2022-09-03T05:46:56.366 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.367 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.367 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.367 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.367 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.368 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.368 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_EARZCit60fDg0eMLHHZC990", 2022-09-03T05:46:56.368 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.368 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.369 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.369 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.370 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.370 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.370 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.371 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.371 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.162940_191.1", 2022-09-03T05:46:56.371 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.371 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22", 2022-09-03T05:46:56.372 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.162940466Z", 2022-09-03T05:46:56.372 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.373 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.373 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.373 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.374 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.374 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:56.374 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.374 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.375 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.375 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.375 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.375 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.376 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.376 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.376 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.167017_192.1", 2022-09-03T05:46:56.376 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.377 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22", 2022-09-03T05:46:56.377 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.167017131Z", 2022-09-03T05:46:56.377 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.377 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.378 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.378 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.379 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.379 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:56.379 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.380 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.380 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.380 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.380 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.381 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.381 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.167997_193.1", 2022-09-03T05:46:56.381 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.382 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-22", 2022-09-03T05:46:56.382 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.167997739Z", 2022-09-03T05:46:56.382 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.382 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.383 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.383 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.383 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.383 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.384 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tI1b7-ygEU1TnWToSDcKGBc", 2022-09-03T05:46:56.384 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.384 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.385 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.385 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.385 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.385 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.386 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.170492_194.1", 2022-09-03T05:46:56.386 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.386 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-22", 2022-09-03T05:46:56.387 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.170492762Z", 2022-09-03T05:46:56.387 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.387 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.387 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.388 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.388 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.388 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tI1b7-ygEU1TnWToSDcKGBc", 2022-09-03T05:46:56.389 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.389 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.389 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.389 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.390 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.390 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.390 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.390 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.391 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.220411_195.1", 2022-09-03T05:46:56.391 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.391 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23", 2022-09-03T05:46:56.391 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.220411899Z", 2022-09-03T05:46:56.392 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.392 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.392 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.392 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.393 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.393 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.393 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:56.394 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.394 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.394 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.394 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.395 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.395 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.395 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.395 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.396 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.223172_196.1", 2022-09-03T05:46:56.396 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.396 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23", 2022-09-03T05:46:56.396 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.223172774Z", 2022-09-03T05:46:56.397 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.397 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.397 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.397 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.398 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.398 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:56.398 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.399 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.399 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.399 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.399 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.400 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.400 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.400 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.400 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.224266_197.1", 2022-09-03T05:46:56.401 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.401 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-23", 2022-09-03T05:46:56.401 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.224266788Z", 2022-09-03T05:46:56.401 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.401 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.402 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.402 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.403 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.403 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_mrTr4jlOIDprI-5OTphGZ6h", 2022-09-03T05:46:56.403 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.403 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.404 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.404 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.404 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.405 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.405 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.405 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.406 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.226840_198.1", 2022-09-03T05:46:56.406 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.406 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-23", 2022-09-03T05:46:56.406 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.226840552Z", 2022-09-03T05:46:56.407 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.407 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.407 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.407 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.408 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.408 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.408 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_mrTr4jlOIDprI-5OTphGZ6h", 2022-09-03T05:46:56.408 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.409 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.409 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.409 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.409 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.410 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.410 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.410 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.410 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.267809_199.1", 2022-09-03T05:46:56.411 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.411 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21", 2022-09-03T05:46:56.411 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.267809597Z", 2022-09-03T05:46:56.411 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.412 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.412 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:56.412 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.412 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.413 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.413 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:56.413 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.413 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.414 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.414 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.414 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.414 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.415 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.415 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.415 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.270381_200.1", 2022-09-03T05:46:56.415 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.416 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21", 2022-09-03T05:46:56.416 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.270381683Z", 2022-09-03T05:46:56.416 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.416 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.417 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:56.417 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.417 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.418 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:56.418 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.418 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.418 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.419 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.419 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.419 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.420 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.420 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.283633_201.1", 2022-09-03T05:46:56.420 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.420 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22", 2022-09-03T05:46:56.421 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.283633845Z", 2022-09-03T05:46:56.421 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.421 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.421 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:56.422 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.422 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.422 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:56.423 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.423 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.423 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.424 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.424 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.424 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.425 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.288001_202.1", 2022-09-03T05:46:56.425 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.425 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22", 2022-09-03T05:46:56.426 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.288001351Z", 2022-09-03T05:46:56.426 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.426 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.426 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:56.427 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.427 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.427 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.427 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:56.428 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.428 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.428 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.429 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.429 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.429 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.429 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.430 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.304152_203.1", 2022-09-03T05:46:56.430 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.430 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23", 2022-09-03T05:46:56.430 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.304152195Z", 2022-09-03T05:46:56.431 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.431 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.431 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:56.432 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.432 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.432 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:56.433 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.433 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.433 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.434 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.434 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.435 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.308405_204.1", 2022-09-03T05:46:56.435 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.435 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23", 2022-09-03T05:46:56.435 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.308405313Z", 2022-09-03T05:46:56.436 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.436 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.436 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:56.437 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.437 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.437 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:56.438 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.438 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.438 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.439 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.439 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.439 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.439 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.440 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.657291_207.1", 2022-09-03T05:46:56.440 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.440 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:56.440 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.657291334Z", 2022-09-03T05:46:56.441 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.441 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.441 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.441 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.442 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.442 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.442 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:56.442 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.443 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.443 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.443 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.443 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.444 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.444 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.444 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.661364_208.1", 2022-09-03T05:46:56.445 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.445 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:56.445 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.661364231Z", 2022-09-03T05:46:56.445 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.446 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.446 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.446 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.446 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.447 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.447 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:56.447 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.447 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.448 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.448 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.448 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.448 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.449 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.449 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.662317_209.1", 2022-09-03T05:46:56.449 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.450 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-24", 2022-09-03T05:46:56.450 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.662317133Z", 2022-09-03T05:46:56.450 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.450 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.451 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.451 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.451 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.452 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_RWbG2F3yjlSXFQy-Qq_ON1N", 2022-09-03T05:46:56.452 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.452 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.453 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.453 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.453 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.453 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.454 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.666554_210.1", 2022-09-03T05:46:56.454 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.455 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-24", 2022-09-03T05:46:56.455 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.666554708Z", 2022-09-03T05:46:56.455 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.455 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.455 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.456 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.456 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.456 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_RWbG2F3yjlSXFQy-Qq_ON1N", 2022-09-03T05:46:56.457 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.457 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.457 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.457 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.458 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.458 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.458 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.459 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.459 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.685377_211.1", 2022-09-03T05:46:56.459 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.459 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:56.460 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.685377688Z", 2022-09-03T05:46:56.460 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.460 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.460 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.460 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.461 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.461 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:56.461 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.462 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.462 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.463 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.463 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.463 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.463 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.464 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.689809_212.1", 2022-09-03T05:46:56.464 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.464 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:56.464 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.689809673Z", 2022-09-03T05:46:56.465 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.465 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.465 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.465 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.466 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.466 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.466 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:56.466 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.467 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.467 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.467 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.467 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.468 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.468 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.468 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.468 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.690762_213.1", 2022-09-03T05:46:56.469 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.469 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-25", 2022-09-03T05:46:56.469 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.690762414Z", 2022-09-03T05:46:56.469 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.470 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.470 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.470 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.471 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.471 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.471 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_IaGw1AfiCAqC-aV2Oup3OCi", 2022-09-03T05:46:56.471 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.472 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.472 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.472 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.472 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.473 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.473 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.473 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.473 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.693239_214.1", 2022-09-03T05:46:56.474 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.474 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-25", 2022-09-03T05:46:56.474 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.693239599Z", 2022-09-03T05:46:56.474 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.475 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.475 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.475 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.475 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.476 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.476 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_IaGw1AfiCAqC-aV2Oup3OCi", 2022-09-03T05:46:56.476 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.476 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.477 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.477 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.477 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.477 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.478 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.478 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.478 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.727704_215.1", 2022-09-03T05:46:56.478 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.479 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:56.479 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.727704277Z", 2022-09-03T05:46:56.479 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.479 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.480 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.480 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.480 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.480 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.481 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:56.481 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.481 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.481 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.482 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.482 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.482 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.482 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.483 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.483 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.731553_216.1", 2022-09-03T05:46:56.483 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.483 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:56.484 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.731553123Z", 2022-09-03T05:46:56.484 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.484 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.484 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.485 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.485 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.485 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.485 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:56.486 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.486 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.486 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.486 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.487 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.487 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.487 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.487 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.488 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.732533_217.1", 2022-09-03T05:46:56.488 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.488 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-26", 2022-09-03T05:46:56.488 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.732533246Z", 2022-09-03T05:46:56.489 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.489 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.489 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.489 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.490 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.490 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.490 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_dUSEiciichvry8cYc4lxW96", 2022-09-03T05:46:56.490 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.491 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.491 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.491 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.491 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.492 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.492 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.492 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.492 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.734903_218.1", 2022-09-03T05:46:56.493 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.493 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-26", 2022-09-03T05:46:56.493 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.734903969Z", 2022-09-03T05:46:56.493 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.494 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.494 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.494 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.494 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.495 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.495 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_dUSEiciichvry8cYc4lxW96", 2022-09-03T05:46:56.495 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.495 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.496 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.496 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.496 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.496 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.497 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.497 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.497 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.763839_219.1", 2022-09-03T05:46:56.497 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.498 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:56.498 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.763839115Z", 2022-09-03T05:46:56.498 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.498 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.499 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:56.499 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.499 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.499 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.500 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:56.500 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.500 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.500 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.501 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.501 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.501 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.501 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.502 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.502 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.766491_220.1", 2022-09-03T05:46:56.502 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.502 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:56.503 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.766491425Z", 2022-09-03T05:46:56.503 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.503 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.503 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:56.504 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.504 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.504 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.504 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:56.505 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.505 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.506 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.506 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.506 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.506 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.507 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.507 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.507 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.780440_221.1", 2022-09-03T05:46:56.507 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.508 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:56.508 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.780440878Z", 2022-09-03T05:46:56.508 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.508 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.509 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:56.509 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.509 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.509 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.510 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:56.510 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 3 2022-09-03T05:46:56.510 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.510 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.511 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.511 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.511 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.511 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.512 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.512 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.784428_222.1", 2022-09-03T05:46:56.512 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.512 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:56.513 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.784428128Z", 2022-09-03T05:46:56.513 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.513 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.513 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:56.514 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.514 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.514 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:56.515 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 3 2022-09-03T05:46:56.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.515 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.515 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.516 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.516 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.516 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.516 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.517 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.798318_223.1", 2022-09-03T05:46:56.517 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.517 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:56.517 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.798318959Z", 2022-09-03T05:46:56.518 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.518 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.518 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:56.518 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.519 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.519 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:56.519 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.520 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.520 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.520 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.521 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.521 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.521 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.522 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.802397_224.1", 2022-09-03T05:46:56.522 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.522 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:56.522 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.802397402Z", 2022-09-03T05:46:56.523 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.523 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.523 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:56.523 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.524 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.524 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:56.524 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.525 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.525 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.525 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.526 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.526 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.526 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.526 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.818536_225.1", 2022-09-03T05:46:56.527 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.527 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:56.527 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.818536670Z", 2022-09-03T05:46:56.527 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.528 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.528 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:56.528 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.529 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.529 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:56.529 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 3 2022-09-03T05:46:56.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.530 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.530 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.530 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.530 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.531 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.826419_226.1", 2022-09-03T05:46:56.531 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.532 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:56.532 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.826419304Z", 2022-09-03T05:46:56.532 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.532 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.533 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:56.533 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.533 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.534 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:56.534 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 3 2022-09-03T05:46:56.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.534 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.535 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.535 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.535 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.536 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.854560_227.1", 2022-09-03T05:46:56.536 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.536 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:56.537 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.854560270Z", 2022-09-03T05:46:56.537 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.537 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.537 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:56.538 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.538 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.539 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:56.539 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.539 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.544 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.544 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.544 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.545 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.857527_228.1", 2022-09-03T05:46:56.545 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.546 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:56.546 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.857527676Z", 2022-09-03T05:46:56.546 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.546 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.547 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:56.547 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.547 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.548 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:56.548 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.548 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.549 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.549 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.549 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.550 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.870879_229.1", 2022-09-03T05:46:56.550 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.550 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:56.551 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.870879960Z", 2022-09-03T05:46:56.551 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.551 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.551 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:56.552 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.552 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.552 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:56.553 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 3 2022-09-03T05:46:56.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.553 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.553 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.554 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.554 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.555 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.876923_230.1", 2022-09-03T05:46:56.555 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.555 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:56.556 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.876923670Z", 2022-09-03T05:46:56.556 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.556 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.556 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:56.557 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.557 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.557 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:56.558 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 3 2022-09-03T05:46:56.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.558 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.558 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.559 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.559 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.560 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.870437_232.1", 2022-09-03T05:46:56.560 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.560 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-27:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.27", 2022-09-03T05:46:56.560 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.870437527Z", 2022-09-03T05:46:56.561 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.561 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.561 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.561 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.562 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.562 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:56.562 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.563 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.563 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.563 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.564 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.564 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.874799_233.1", 2022-09-03T05:46:56.565 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.565 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-27:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.27", 2022-09-03T05:46:56.565 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.874799084Z", 2022-09-03T05:46:56.565 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.566 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.566 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.566 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.567 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.567 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:56.567 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.568 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.568 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.568 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.568 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.569 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.876797_234.1", 2022-09-03T05:46:56.569 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.570 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-27", 2022-09-03T05:46:56.570 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.876797147Z", 2022-09-03T05:46:56.570 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.570 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.571 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.571 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.572 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.572 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_kvVMhGyMiz3kusJkbwnyhxJ", 2022-09-03T05:46:56.572 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.573 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.573 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.573 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.574 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.574 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.885232_235.1", 2022-09-03T05:46:56.575 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.575 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-27", 2022-09-03T05:46:56.575 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.885232353Z", 2022-09-03T05:46:56.575 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.576 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.576 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.576 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.577 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.578 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_kvVMhGyMiz3kusJkbwnyhxJ", 2022-09-03T05:46:56.578 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.579 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.579 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.580 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.580 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.581 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.916915_236.1", 2022-09-03T05:46:56.581 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.582 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28", 2022-09-03T05:46:56.582 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.916915113Z", 2022-09-03T05:46:56.582 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.583 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.583 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.583 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.584 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.584 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:56.585 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.586 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.586 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.587 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.587 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.588 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.920958_237.1", 2022-09-03T05:46:56.588 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.588 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28", 2022-09-03T05:46:56.589 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.920958536Z", 2022-09-03T05:46:56.589 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.590 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.590 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.590 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.591 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.591 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:56.591 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.592 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.592 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.592 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.592 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.593 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.922045_238.1", 2022-09-03T05:46:56.593 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.594 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-28", 2022-09-03T05:46:56.594 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.922045913Z", 2022-09-03T05:46:56.594 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.594 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.595 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.595 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.595 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.596 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_OOWZg42AYED_QBCONdEGECB", 2022-09-03T05:46:56.596 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.596 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.597 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.597 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.597 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.598 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.926094_239.1", 2022-09-03T05:46:56.598 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.599 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-28", 2022-09-03T05:46:56.599 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.926094548Z", 2022-09-03T05:46:56.599 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.599 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.600 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.600 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.601 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.601 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_OOWZg42AYED_QBCONdEGECB", 2022-09-03T05:46:56.601 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.602 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.602 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.602 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.602 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.603 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.603 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.962517_240.1", 2022-09-03T05:46:56.604 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.604 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-29:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.29", 2022-09-03T05:46:56.604 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.962517073Z", 2022-09-03T05:46:56.605 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.605 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.605 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.606 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.606 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.606 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:56.607 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.607 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.607 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.607 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.608 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.608 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.609 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.965127_241.1", 2022-09-03T05:46:56.609 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.609 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-29:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.29", 2022-09-03T05:46:56.610 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.965127874Z", 2022-09-03T05:46:56.610 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.610 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.611 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.611 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.611 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.611 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.612 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:56.612 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.612 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.613 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.613 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.613 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.614 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.966081_242.1", 2022-09-03T05:46:56.614 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.614 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-29", 2022-09-03T05:46:56.615 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.966081501Z", 2022-09-03T05:46:56.615 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.615 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.615 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.616 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.616 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.616 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_yDkLaMdkLBW9LxbH_x72oyR", 2022-09-03T05:46:56.617 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.617 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.618 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.618 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.619 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.620 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.968432_243.1", 2022-09-03T05:46:56.620 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.620 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-29", 2022-09-03T05:46:56.621 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.968432661Z", 2022-09-03T05:46:56.621 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.621 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.622 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.622 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.622 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.623 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_yDkLaMdkLBW9LxbH_x72oyR", 2022-09-03T05:46:56.623 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.623 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.624 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.624 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.624 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.625 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.998970_244.1", 2022-09-03T05:46:56.625 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.626 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-27:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.27", 2022-09-03T05:46:56.626 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.998970343Z", 2022-09-03T05:46:56.626 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.626 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.627 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:56.627 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.627 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.628 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:56.628 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.628 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.629 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.629 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.629 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.630 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.001548_245.1", 2022-09-03T05:46:56.630 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.630 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-27:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.27", 2022-09-03T05:46:56.631 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.001548177Z", 2022-09-03T05:46:56.631 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.631 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.631 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:56.632 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.632 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.632 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:56.633 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.633 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.633 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.634 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.634 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.635 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.049458_246.1", 2022-09-03T05:46:56.635 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.635 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28", 2022-09-03T05:46:56.636 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.049458314Z", 2022-09-03T05:46:56.636 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.636 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.636 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:56.637 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.637 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.637 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:56.638 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.638 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.638 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.639 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.639 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.640 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.052066_247.1", 2022-09-03T05:46:56.640 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.640 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28", 2022-09-03T05:46:56.640 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.052066976Z", 2022-09-03T05:46:56.641 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.641 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.641 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:56.642 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.642 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.642 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:56.643 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.643 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.643 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.644 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.644 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.645 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.645 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.091593_248.1", 2022-09-03T05:46:56.645 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.645 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-29:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.29", 2022-09-03T05:46:56.646 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.091593357Z", 2022-09-03T05:46:56.646 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.646 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.646 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:56.647 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.647 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.647 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:56.648 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.648 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.648 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.649 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.649 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.649 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.650 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.099123_249.1", 2022-09-03T05:46:56.650 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.650 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-29:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.29", 2022-09-03T05:46:56.650 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.099123834Z", 2022-09-03T05:46:56.651 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.651 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.651 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:56.651 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.652 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.652 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:56.652 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.653 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.653 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.653 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.654 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.654 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.135078_250.1", 2022-09-03T05:46:56.655 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.655 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30", 2022-09-03T05:46:56.655 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.135078742Z", 2022-09-03T05:46:56.655 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.656 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.656 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.656 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.657 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.657 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:56.657 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.658 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.658 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.658 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.658 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.659 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.140570_251.1", 2022-09-03T05:46:56.659 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.660 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30", 2022-09-03T05:46:56.660 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.140570450Z", 2022-09-03T05:46:56.660 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.660 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.661 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.661 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.661 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.662 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:56.662 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.662 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.663 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.663 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.663 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.664 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.142697_252.1", 2022-09-03T05:46:56.664 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.665 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-30", 2022-09-03T05:46:56.665 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.142697729Z", 2022-09-03T05:46:56.665 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.665 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.666 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.666 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.666 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.667 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_4yhNaX-8bC2cTXp5LKWw-Qo", 2022-09-03T05:46:56.667 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.667 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.668 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.668 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.668 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.669 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.148860_253.1", 2022-09-03T05:46:56.669 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.669 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-30", 2022-09-03T05:46:56.670 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.148860761Z", 2022-09-03T05:46:56.670 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.670 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.670 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.671 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.671 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.671 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_4yhNaX-8bC2cTXp5LKWw-Qo", 2022-09-03T05:46:56.672 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.672 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.672 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.673 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.673 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.674 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.172332_254.1", 2022-09-03T05:46:56.674 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.674 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31", 2022-09-03T05:46:56.675 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.172332646Z", 2022-09-03T05:46:56.675 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.675 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.675 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.676 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.676 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.676 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:56.677 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.677 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.677 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.678 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.678 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.679 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.176046_255.1", 2022-09-03T05:46:56.679 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.679 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31", 2022-09-03T05:46:56.680 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.176046838Z", 2022-09-03T05:46:56.680 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.680 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.680 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.681 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.681 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.681 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:56.682 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.682 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.682 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.683 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.683 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.684 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.176958_256.1", 2022-09-03T05:46:56.684 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.684 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-31", 2022-09-03T05:46:56.684 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.176958510Z", 2022-09-03T05:46:56.685 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.685 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.685 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.685 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.686 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.686 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_oPtbK62ZjFZvpDTUcbzdHLT", 2022-09-03T05:46:56.686 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.687 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.687 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.687 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.688 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.688 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.179165_257.1", 2022-09-03T05:46:56.689 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.689 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-31", 2022-09-03T05:46:56.689 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.179165643Z", 2022-09-03T05:46:56.690 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.690 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.690 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.690 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.691 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.691 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.691 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_oPtbK62ZjFZvpDTUcbzdHLT", 2022-09-03T05:46:56.691 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.692 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.692 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.692 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.692 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.693 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.693 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.220311_258.1", 2022-09-03T05:46:56.693 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.694 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-32:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.32", 2022-09-03T05:46:56.694 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.220311791Z", 2022-09-03T05:46:56.694 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.695 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.695 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.695 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.696 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.696 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:56.696 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.697 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.697 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.697 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.698 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.698 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.222654_259.1", 2022-09-03T05:46:56.699 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.699 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-32:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.32", 2022-09-03T05:46:56.699 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.222654278Z", 2022-09-03T05:46:56.699 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.700 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.700 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.700 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.701 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.701 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:56.702 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.702 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.703 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.703 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.703 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.704 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.223513_260.1", 2022-09-03T05:46:56.705 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.705 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-32", 2022-09-03T05:46:56.705 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.223513282Z", 2022-09-03T05:46:56.706 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.706 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.707 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.707 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.708 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.708 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_y0oxyMaYTjSUF-S-HkK29E5", 2022-09-03T05:46:56.708 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.709 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.709 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.709 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.710 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.711 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.228241_261.1", 2022-09-03T05:46:56.711 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.711 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-32", 2022-09-03T05:46:56.711 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.228241236Z", 2022-09-03T05:46:56.712 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.712 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.712 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.713 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.713 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.714 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_y0oxyMaYTjSUF-S-HkK29E5", 2022-09-03T05:46:56.714 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.714 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.715 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.715 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.715 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.716 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.265477_262.1", 2022-09-03T05:46:56.716 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.717 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30", 2022-09-03T05:46:56.717 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.265477055Z", 2022-09-03T05:46:56.717 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.717 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.718 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:56.718 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.718 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.719 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:56.719 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.720 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.720 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.720 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.720 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.721 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.268146_263.1", 2022-09-03T05:46:56.721 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.722 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30", 2022-09-03T05:46:56.722 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.268146553Z", 2022-09-03T05:46:56.722 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.722 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.723 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:56.723 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.723 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.724 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:56.724 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.724 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.725 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.725 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.725 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.725 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.726 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.726 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.350018_264.1", 2022-09-03T05:46:56.726 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.726 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31", 2022-09-03T05:46:56.727 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.350018238Z", 2022-09-03T05:46:56.727 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.727 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.727 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:56.728 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.728 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.728 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:56.729 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.729 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.729 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.730 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.730 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.731 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.352665_265.1", 2022-09-03T05:46:56.731 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.731 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31", 2022-09-03T05:46:56.731 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.352665529Z", 2022-09-03T05:46:56.732 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.732 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.732 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:56.732 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.733 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.733 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:56.733 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.734 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.734 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.735 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.735 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.736 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.401743_266.1", 2022-09-03T05:46:56.736 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.736 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-32:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.32", 2022-09-03T05:46:56.737 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.401743084Z", 2022-09-03T05:46:56.737 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.737 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.737 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:56.738 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.738 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.738 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:56.739 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.739 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.739 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.740 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.740 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.741 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.404375_267.1", 2022-09-03T05:46:56.741 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.741 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-32:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.32", 2022-09-03T05:46:56.741 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.404375474Z", 2022-09-03T05:46:56.742 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.742 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.742 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:56.743 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.743 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.743 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:56.743 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.744 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.744 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.744 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.745 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.745 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.434979_268.1", 2022-09-03T05:46:56.746 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.746 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-33:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.33", 2022-09-03T05:46:56.746 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.434979477Z", 2022-09-03T05:46:56.747 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.747 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.747 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.747 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.748 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.748 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:56.748 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.749 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.749 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.749 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.750 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.750 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.438289_269.1", 2022-09-03T05:46:56.751 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.751 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-33:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.33", 2022-09-03T05:46:56.751 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.438289839Z", 2022-09-03T05:46:56.751 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.752 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.752 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.752 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.753 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.753 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:56.753 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.754 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.754 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.754 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.755 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.755 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.439390_270.1", 2022-09-03T05:46:56.756 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.756 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-33", 2022-09-03T05:46:56.756 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.439390666Z", 2022-09-03T05:46:56.756 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.756 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.757 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.757 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.758 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.758 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_XHlZf1wWm_Tr8FwCOGzkHwG", 2022-09-03T05:46:56.758 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.759 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.759 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.759 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.759 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.760 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.441665_271.1", 2022-09-03T05:46:56.760 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.761 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-33", 2022-09-03T05:46:56.761 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.441665178Z", 2022-09-03T05:46:56.761 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.761 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.762 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.762 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.762 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.763 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_XHlZf1wWm_Tr8FwCOGzkHwG", 2022-09-03T05:46:56.763 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.763 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.764 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.764 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.764 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.765 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.461635_272.1", 2022-09-03T05:46:56.765 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.765 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34", 2022-09-03T05:46:56.766 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.461635262Z", 2022-09-03T05:46:56.766 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.766 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.766 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.767 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.767 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.768 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:56.768 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.768 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.769 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.769 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.769 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.770 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.466010_273.1", 2022-09-03T05:46:56.770 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.770 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34", 2022-09-03T05:46:56.771 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.466010065Z", 2022-09-03T05:46:56.771 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.771 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.771 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.772 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.772 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.772 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:56.773 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.773 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.773 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.774 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.774 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.775 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.468430_274.1", 2022-09-03T05:46:56.775 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.775 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-34", 2022-09-03T05:46:56.776 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.468430282Z", 2022-09-03T05:46:56.776 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.776 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.777 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.777 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.777 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.778 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_8JLNrQKjOCrzmEAJH1Gqebc", 2022-09-03T05:46:56.778 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.779 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.779 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.779 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.779 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.780 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.473656_275.1", 2022-09-03T05:46:56.780 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.781 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-34", 2022-09-03T05:46:56.781 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.473656625Z", 2022-09-03T05:46:56.781 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.781 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.782 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.782 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.783 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.783 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_8JLNrQKjOCrzmEAJH1Gqebc", 2022-09-03T05:46:56.783 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.784 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.784 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.784 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.784 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.785 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.519749_276.1", 2022-09-03T05:46:56.786 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.786 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35", 2022-09-03T05:46:56.786 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.519749696Z", 2022-09-03T05:46:56.787 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.787 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.787 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.787 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.788 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.788 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:56.789 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.789 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.789 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.790 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.790 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.791 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.527797_277.1", 2022-09-03T05:46:56.791 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.791 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35", 2022-09-03T05:46:56.792 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.527797485Z", 2022-09-03T05:46:56.792 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.792 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.792 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.793 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.793 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.793 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:56.794 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.794 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.794 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.795 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.795 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.796 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.530086_278.1", 2022-09-03T05:46:56.796 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.796 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-35", 2022-09-03T05:46:56.796 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.530086284Z", 2022-09-03T05:46:56.797 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.797 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.797 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.797 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.798 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.798 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_9eV0dD_Np6i5r-IXItyYqq8", 2022-09-03T05:46:56.798 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.799 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.799 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.799 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.800 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.800 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.542414_279.1", 2022-09-03T05:46:56.801 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.801 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-35", 2022-09-03T05:46:56.801 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.542414804Z", 2022-09-03T05:46:56.801 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.802 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.802 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.802 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.803 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.803 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_9eV0dD_Np6i5r-IXItyYqq8", 2022-09-03T05:46:56.803 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.804 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.804 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.804 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.804 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.805 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.805 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.574433_280.1", 2022-09-03T05:46:56.806 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.806 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-33:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.33", 2022-09-03T05:46:56.806 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.574433191Z", 2022-09-03T05:46:56.807 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.807 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.807 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:56.807 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.808 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.808 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:56.809 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.809 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.810 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.810 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.811 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.812 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.813 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.577143_281.1", 2022-09-03T05:46:56.814 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.814 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-33:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.33", 2022-09-03T05:46:56.815 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.577143931Z", 2022-09-03T05:46:56.816 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.816 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.817 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:56.817 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.818 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.818 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:56.819 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.819 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.819 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.820 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.820 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.821 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183851.605820_282.1", 2022-09-03T05:46:56.821 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.821 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34", 2022-09-03T05:46:56.821 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:11.605820553Z", 2022-09-03T05:46:56.822 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.822 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.822 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:56.823 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.823 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.823 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:56.824 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.824 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.825 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.825 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.826 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.827 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183851.610169_283.1", 2022-09-03T05:46:56.828 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.828 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34", 2022-09-03T05:46:56.829 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:11.610169538Z", 2022-09-03T05:46:56.829 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.830 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.830 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:56.830 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.831 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.832 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:56.832 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.833 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.834 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.834 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.835 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.836 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.681884_284.1", 2022-09-03T05:46:56.837 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.837 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35", 2022-09-03T05:46:56.838 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.681884846Z", 2022-09-03T05:46:56.838 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.839 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.839 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:56.840 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.841 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.841 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:56.842 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.842 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.842 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.843 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.843 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.844 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.684548_285.1", 2022-09-03T05:46:56.844 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.844 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35", 2022-09-03T05:46:56.845 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.684548086Z", 2022-09-03T05:46:56.845 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.845 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.845 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:56.846 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.847 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.847 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:56.848 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:56.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.848 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.849 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.849 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.849 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.850 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.769697_286.1", 2022-09-03T05:46:56.850 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.851 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36", 2022-09-03T05:46:56.851 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.769697830Z", 2022-09-03T05:46:56.851 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.851 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.852 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.852 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.852 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.853 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lzXJaycHyP7aPeCDF57oxvc", 2022-09-03T05:46:56.853 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.853 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.854 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.854 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.854 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.855 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.780292_287.1", 2022-09-03T05:46:56.856 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.856 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36", 2022-09-03T05:46:56.857 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.780292446Z", 2022-09-03T05:46:56.857 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.857 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.858 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.858 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.859 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.859 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lzXJaycHyP7aPeCDF57oxvc", 2022-09-03T05:46:56.859 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.860 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.860 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.860 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.861 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.862 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.781473_288.1", 2022-09-03T05:46:56.862 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.862 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-36", 2022-09-03T05:46:56.863 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.781473404Z", 2022-09-03T05:46:56.863 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.863 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.864 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.864 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.864 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.865 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Aunq5-l6wG1IGbEHKmUcfjj", 2022-09-03T05:46:56.865 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.865 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.866 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.866 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.866 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.867 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.785261_289.1", 2022-09-03T05:46:56.867 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.868 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-36", 2022-09-03T05:46:56.868 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.785261123Z", 2022-09-03T05:46:56.868 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.869 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.869 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.870 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.871 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.871 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Aunq5-l6wG1IGbEHKmUcfjj", 2022-09-03T05:46:56.872 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.872 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.873 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.873 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.874 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.874 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.875 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.875 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.806309_290.1", 2022-09-03T05:46:56.875 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.876 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37", 2022-09-03T05:46:56.876 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.806309544Z", 2022-09-03T05:46:56.876 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.877 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.877 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.878 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.879 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.880 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_kuQ0xr50nHBW7YTCPQavDGz", 2022-09-03T05:46:56.880 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.880 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.881 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.881 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.882 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.882 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.883 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.883 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.808817_291.1", 2022-09-03T05:46:56.884 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.884 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37", 2022-09-03T05:46:56.884 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.808817564Z", 2022-09-03T05:46:56.885 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.886 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.886 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.887 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.888 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.888 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_kuQ0xr50nHBW7YTCPQavDGz", 2022-09-03T05:46:56.889 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.889 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.889 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.890 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.890 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.891 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.892 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.892 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.809764_292.1", 2022-09-03T05:46:56.893 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.893 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-37", 2022-09-03T05:46:56.893 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.809764987Z", 2022-09-03T05:46:56.893 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.894 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.894 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.895 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.896 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.896 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_2HiJKFit31XaN5gBsoarIgZ", 2022-09-03T05:46:56.897 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.898 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.898 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.899 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.899 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.901 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.814054_293.1", 2022-09-03T05:46:56.901 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.902 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-37", 2022-09-03T05:46:56.902 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.814054902Z", 2022-09-03T05:46:56.903 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.903 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.904 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.905 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.906 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.907 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_2HiJKFit31XaN5gBsoarIgZ", 2022-09-03T05:46:56.907 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.907 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.908 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.908 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.908 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.909 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.910 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.846705_294.1", 2022-09-03T05:46:56.910 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.910 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38", 2022-09-03T05:46:56.911 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.846705503Z", 2022-09-03T05:46:56.911 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.911 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.912 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.912 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.913 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.913 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_h95HWYhtSYo3Z6W42rIP0YJ", 2022-09-03T05:46:56.913 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.914 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.914 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.914 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.915 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.915 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.916 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.850680_295.1", 2022-09-03T05:46:56.916 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.916 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38", 2022-09-03T05:46:56.916 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.850680201Z", 2022-09-03T05:46:56.917 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.917 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.917 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.917 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.918 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.918 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_h95HWYhtSYo3Z6W42rIP0YJ", 2022-09-03T05:46:56.918 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.919 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.920 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.920 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.920 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.923 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.851766_296.1", 2022-09-03T05:46:56.924 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.924 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-38", 2022-09-03T05:46:56.924 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.851766916Z", 2022-09-03T05:46:56.924 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.925 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.925 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.925 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.926 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.926 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_K5Yz9Djeac7y4a5O_0TviMm", 2022-09-03T05:46:56.926 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.927 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.927 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.927 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.927 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.928 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.928 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.928 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.856692_297.1", 2022-09-03T05:46:56.928 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.928 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-38", 2022-09-03T05:46:56.929 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.856692289Z", 2022-09-03T05:46:56.929 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.930 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.930 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.930 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.931 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.931 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_K5Yz9Djeac7y4a5O_0TviMm", 2022-09-03T05:46:56.931 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.932 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.932 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.932 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.933 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.933 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.934 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.934 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.934 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.003533_298.1", 2022-09-03T05:46:56.934 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.935 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-39:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.39", 2022-09-03T05:46:56.935 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.003533677Z", 2022-09-03T05:46:56.935 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.935 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.936 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.936 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.937 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.937 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_LjHtI20g0C-nIIB1xqrV4kd", 2022-09-03T05:46:56.937 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.938 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.938 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.938 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.938 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.939 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.939 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.006401_299.1", 2022-09-03T05:46:56.939 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.940 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-39:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.39", 2022-09-03T05:46:56.940 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.006401221Z", 2022-09-03T05:46:56.940 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.940 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.940 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.941 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.941 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.941 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_LjHtI20g0C-nIIB1xqrV4kd", 2022-09-03T05:46:56.942 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.942 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.942 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.943 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.943 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.943 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.944 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.007446_300.1", 2022-09-03T05:46:56.944 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.944 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-39", 2022-09-03T05:46:56.944 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.007446652Z", 2022-09-03T05:46:56.945 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.945 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.945 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.945 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.946 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.946 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.946 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_LR1OrAXSVx6okJ40JV0Uh8F", 2022-09-03T05:46:56.946 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.947 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.947 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.948 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.949 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.950 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.950 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.010018_301.1", 2022-09-03T05:46:56.951 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.951 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-39", 2022-09-03T05:46:56.952 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.010018357Z", 2022-09-03T05:46:56.952 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.953 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.953 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.953 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.954 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.954 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_LR1OrAXSVx6okJ40JV0Uh8F", 2022-09-03T05:46:56.955 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.956 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.956 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.957 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.958 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.959 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.959 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.039601_302.1", 2022-09-03T05:46:56.960 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.960 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-40:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.40", 2022-09-03T05:46:56.961 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.039601239Z", 2022-09-03T05:46:56.961 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.962 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.962 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.963 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.964 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.964 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_SEgVmZRRIdgycWQGmgKhmpv", 2022-09-03T05:46:56.965 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.965 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.966 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.966 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.967 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.967 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.968 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.968 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.048071_303.1", 2022-09-03T05:46:56.969 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.969 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-40:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.40", 2022-09-03T05:46:56.969 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.048071799Z", 2022-09-03T05:46:56.970 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.970 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.971 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.971 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.971 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.972 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.972 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_SEgVmZRRIdgycWQGmgKhmpv", 2022-09-03T05:46:56.973 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.973 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.973 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.973 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.974 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.974 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.974 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.975 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.975 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.049107_304.1", 2022-09-03T05:46:56.975 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.975 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-40", 2022-09-03T05:46:56.976 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.049107658Z", 2022-09-03T05:46:56.976 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.976 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.977 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.977 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.977 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.977 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.978 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_e0zQngowI1boXIeM6KNEWZw", 2022-09-03T05:46:56.978 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.979 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.979 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.979 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.980 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.980 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.980 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.980 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.981 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.054911_305.1", 2022-09-03T05:46:56.981 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:56.982 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-40", 2022-09-03T05:46:56.982 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.054911391Z", 2022-09-03T05:46:56.982 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.982 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.983 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.983 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.983 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.984 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.984 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_e0zQngowI1boXIeM6KNEWZw", 2022-09-03T05:46:56.984 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.984 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.985 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.985 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:56.985 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.986 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.986 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.986 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.987 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.099806_306.1", 2022-09-03T05:46:56.987 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.988 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-41:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.41", 2022-09-03T05:46:56.988 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.099806646Z", 2022-09-03T05:46:56.989 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.989 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.990 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.990 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.991 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.991 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:56.991 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_M2clmHIwHAIIB9oEOnpIKkh", 2022-09-03T05:46:56.992 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:56.992 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.992 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:56.993 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:56.993 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.994 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:56.994 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.995 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:56.995 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.111286_307.1", 2022-09-03T05:46:56.996 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:56.996 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-41:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.41", 2022-09-03T05:46:56.997 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.111286094Z", 2022-09-03T05:46:56.997 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:56.997 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:56.998 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:56.998 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:56.999 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:56.999 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.000 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_M2clmHIwHAIIB9oEOnpIKkh", 2022-09-03T05:46:57.000 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.000 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.001 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.001 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.002 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.002 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.003 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.003 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.004 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.113514_308.1", 2022-09-03T05:46:57.004 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.005 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-41", 2022-09-03T05:46:57.005 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.113514552Z", 2022-09-03T05:46:57.006 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.006 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.007 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.007 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.008 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.008 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.008 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_kKVUMELn0ub6gAm6SEjIkvZ", 2022-09-03T05:46:57.009 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.009 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.009 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.010 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:57.010 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.011 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.012 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.012 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.119538_309.1", 2022-09-03T05:46:57.013 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.013 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-41", 2022-09-03T05:46:57.014 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.119538060Z", 2022-09-03T05:46:57.014 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.015 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.015 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.015 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.016 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.016 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.017 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_kKVUMELn0ub6gAm6SEjIkvZ", 2022-09-03T05:46:57.017 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.018 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.018 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.018 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.019 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.019 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.019 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.020 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.020 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182414.943600_19.1", 2022-09-03T05:46:57.020 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.021 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-1:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.1", 2022-09-03T05:46:57.021 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:14.943600569Z", 2022-09-03T05:46:57.021 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.022 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.022 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.023 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.023 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.023 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.024 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_rq1mS4spn__TV-QX0MQQJUU", 2022-09-03T05:46:57.024 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.025 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.025 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.025 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:57.026 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.026 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.026 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.027 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.027 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182414.959460_20.1", 2022-09-03T05:46:57.027 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.028 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-1:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.1", 2022-09-03T05:46:57.028 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:14.959460360Z", 2022-09-03T05:46:57.028 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.029 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.029 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.029 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.030 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.030 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.031 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_rq1mS4spn__TV-QX0MQQJUU", 2022-09-03T05:46:57.031 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.032 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.032 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.033 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.033 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.034 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.034 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.035 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.036 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182414.961669_21.1", 2022-09-03T05:46:57.037 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.037 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-1", 2022-09-03T05:46:57.038 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:14.961669762Z", 2022-09-03T05:46:57.038 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.039 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.040 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.040 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.041 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.041 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.042 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_0bHoPHCFbjz75fM89k7kJ9J", 2022-09-03T05:46:57.042 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.043 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.044 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.044 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:57.045 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.045 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.046 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.046 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.047 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182414.972666_22.1", 2022-09-03T05:46:57.047 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.047 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-1", 2022-09-03T05:46:57.048 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:14.972666586Z", 2022-09-03T05:46:57.048 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.049 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.049 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.049 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.050 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.050 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.051 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_0bHoPHCFbjz75fM89k7kJ9J", 2022-09-03T05:46:57.051 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.051 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.052 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.052 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.053 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.053 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.053 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.053 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.054 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182415.003071_23.1", 2022-09-03T05:46:57.054 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.054 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-2:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.2", 2022-09-03T05:46:57.055 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:15.003071656Z", 2022-09-03T05:46:57.055 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.055 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.055 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.056 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.056 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.056 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.057 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tfkUve-GLYp4UeLNm-4NFHF", 2022-09-03T05:46:57.057 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.057 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.057 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.058 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:57.058 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.058 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.058 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.059 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.059 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182415.009391_24.1", 2022-09-03T05:46:57.059 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.059 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-2:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.2", 2022-09-03T05:46:57.060 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:15.009391132Z", 2022-09-03T05:46:57.060 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.060 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.061 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.061 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.061 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.061 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.062 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tfkUve-GLYp4UeLNm-4NFHF", 2022-09-03T05:46:57.062 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.062 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.062 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.063 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.063 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.063 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.064 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.064 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.064 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182415.013579_25.1", 2022-09-03T05:46:57.064 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.065 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-2", 2022-09-03T05:46:57.065 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:15.013579974Z", 2022-09-03T05:46:57.065 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.065 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.066 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.066 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.066 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.066 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.067 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_ach1wph5n7_8QKTUZF1Mary", 2022-09-03T05:46:57.067 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.067 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.068 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.068 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:57.068 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.068 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.069 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.069 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.069 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182415.025150_26.1", 2022-09-03T05:46:57.070 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.070 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-2", 2022-09-03T05:46:57.070 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:15.025150228Z", 2022-09-03T05:46:57.070 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.071 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.071 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.071 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.071 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.072 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.072 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_ach1wph5n7_8QKTUZF1Mary", 2022-09-03T05:46:57.072 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.072 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.073 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.073 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.073 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.074 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.074 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.074 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.074 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182417.307770_27.1", 2022-09-03T05:46:57.075 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.075 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-3:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.3", 2022-09-03T05:46:57.075 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:17.307770555Z", 2022-09-03T05:46:57.075 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.076 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.076 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.076 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.076 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.077 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.077 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_RKS6tzrI_dmvQpvUbloi7Q_", 2022-09-03T05:46:57.077 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.078 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.078 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.078 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:57.078 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.079 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.079 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.079 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.079 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182417.312081_28.1", 2022-09-03T05:46:57.080 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.080 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-3:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.3", 2022-09-03T05:46:57.080 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:17.312081255Z", 2022-09-03T05:46:57.080 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.081 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.081 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.081 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.082 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.082 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.082 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_RKS6tzrI_dmvQpvUbloi7Q_", 2022-09-03T05:46:57.082 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.083 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.083 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.083 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.083 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.084 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.084 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.084 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.084 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182417.313242_29.1", 2022-09-03T05:46:57.085 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.085 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-3", 2022-09-03T05:46:57.085 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:17.313242294Z", 2022-09-03T05:46:57.086 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.086 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.086 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.086 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.087 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.087 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lt8IAlx8xFKZXCBoGrP76MP", 2022-09-03T05:46:57.087 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.088 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.088 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:57.088 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.089 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.089 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.089 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.089 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182417.319609_30.1", 2022-09-03T05:46:57.090 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.090 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-3", 2022-09-03T05:46:57.090 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:17.319609012Z", 2022-09-03T05:46:57.091 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.091 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.091 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.091 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.092 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.092 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.092 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lt8IAlx8xFKZXCBoGrP76MP", 2022-09-03T05:46:57.092 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.093 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.093 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.093 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.093 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.094 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.094 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.152440_31.1", 2022-09-03T05:46:57.095 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.095 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-4:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.4", 2022-09-03T05:46:57.095 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.152440893Z", 2022-09-03T05:46:57.096 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.096 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.096 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.096 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.097 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.097 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.097 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_QMyoKwlK2G2NuxL_sp2_re5", 2022-09-03T05:46:57.097 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.098 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.098 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:57.098 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.099 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.099 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.099 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.100 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.158442_32.1", 2022-09-03T05:46:57.100 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.100 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-4:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.4", 2022-09-03T05:46:57.100 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.158442960Z", 2022-09-03T05:46:57.101 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.101 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.101 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.101 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.102 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.102 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.102 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_QMyoKwlK2G2NuxL_sp2_re5", 2022-09-03T05:46:57.103 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.103 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.103 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.103 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.104 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.104 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.104 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.104 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.105 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.159309_33.1", 2022-09-03T05:46:57.105 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.105 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-4", 2022-09-03T05:46:57.105 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.159309677Z", 2022-09-03T05:46:57.106 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.106 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.106 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.106 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.107 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.107 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_FxyCJRWlBEhmncaMF3-AMFD", 2022-09-03T05:46:57.108 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.108 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.108 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.108 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:57.109 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.109 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.109 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.109 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.110 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.163606_34.1", 2022-09-03T05:46:57.110 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.110 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-4", 2022-09-03T05:46:57.110 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.163606904Z", 2022-09-03T05:46:57.111 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.111 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.111 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.112 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.112 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.112 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_FxyCJRWlBEhmncaMF3-AMFD", 2022-09-03T05:46:57.113 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.113 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.113 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.114 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.114 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.114 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.114 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.115 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.181237_35.1", 2022-09-03T05:46:57.115 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.115 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-5:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.5", 2022-09-03T05:46:57.115 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.181237915Z", 2022-09-03T05:46:57.116 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.116 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.116 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.117 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.117 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.117 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_mye4oDj8XK7uSi7YUS75P0l", 2022-09-03T05:46:57.118 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.118 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.118 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:57.119 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.119 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.119 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.120 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.183507_36.1", 2022-09-03T05:46:57.120 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.120 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-5:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.5", 2022-09-03T05:46:57.120 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.183507081Z", 2022-09-03T05:46:57.121 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.121 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.121 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.122 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.122 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.122 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_mye4oDj8XK7uSi7YUS75P0l", 2022-09-03T05:46:57.123 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.123 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.123 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.124 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.124 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.125 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.184425_37.1", 2022-09-03T05:46:57.125 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.125 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-5", 2022-09-03T05:46:57.125 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.184425827Z", 2022-09-03T05:46:57.126 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.126 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.126 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.127 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.127 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.127 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_6QXcvNroiwRkEdVqi3Nf85-", 2022-09-03T05:46:57.128 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.128 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.128 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:57.129 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.129 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.129 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.130 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.186822_38.1", 2022-09-03T05:46:57.130 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.130 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-5", 2022-09-03T05:46:57.130 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.186822428Z", 2022-09-03T05:46:57.131 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.131 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.131 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.132 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.132 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.132 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_6QXcvNroiwRkEdVqi3Nf85-", 2022-09-03T05:46:57.133 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.133 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.133 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.134 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.134 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.135 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.221675_39.1", 2022-09-03T05:46:57.135 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.135 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-6:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.6", 2022-09-03T05:46:57.136 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.221675326Z", 2022-09-03T05:46:57.136 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.136 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.136 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.137 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.137 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.137 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bqgF7AreaM1kO2cBL_SkGsM", 2022-09-03T05:46:57.138 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.138 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.138 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:57.139 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.139 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.139 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.140 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.140 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.230423_40.1", 2022-09-03T05:46:57.140 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.140 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-6:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.6", 2022-09-03T05:46:57.141 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.230423417Z", 2022-09-03T05:46:57.141 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.141 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.141 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.142 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.142 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.142 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bqgF7AreaM1kO2cBL_SkGsM", 2022-09-03T05:46:57.143 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.143 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.144 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.144 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.144 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.144 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.145 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.145 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.235081_41.1", 2022-09-03T05:46:57.145 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.145 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-6", 2022-09-03T05:46:57.146 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.235081816Z", 2022-09-03T05:46:57.146 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.146 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.146 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.147 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.147 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.147 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_wbkk3MMRONgcegk23g-QDT1", 2022-09-03T05:46:57.148 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.148 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.149 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:57.149 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.149 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.150 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.150 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.238098_42.1", 2022-09-03T05:46:57.150 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.150 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-6", 2022-09-03T05:46:57.151 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.238098524Z", 2022-09-03T05:46:57.151 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.151 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.152 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.152 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.152 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.153 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_wbkk3MMRONgcegk23g-QDT1", 2022-09-03T05:46:57.153 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.153 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.154 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.154 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.154 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.155 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.155 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.064697_43.1", 2022-09-03T05:46:57.155 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.156 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7", 2022-09-03T05:46:57.156 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.064697048Z", 2022-09-03T05:46:57.156 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.156 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.157 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.157 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.157 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.157 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.158 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:57.158 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.159 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.159 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:57.159 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.159 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.160 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.070346_44.1", 2022-09-03T05:46:57.160 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.161 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7", 2022-09-03T05:46:57.161 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.070346892Z", 2022-09-03T05:46:57.161 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.162 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.162 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.162 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.163 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.163 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:57.163 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.164 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.164 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.164 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.164 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.165 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.071355_45.1", 2022-09-03T05:46:57.165 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.166 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-7", 2022-09-03T05:46:57.166 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.071355932Z", 2022-09-03T05:46:57.166 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.167 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.167 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.167 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.168 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.168 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:46:57.168 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.169 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.169 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:57.169 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.169 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.170 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.170 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.075532_46.1", 2022-09-03T05:46:57.171 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.171 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-7", 2022-09-03T05:46:57.171 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.075532031Z", 2022-09-03T05:46:57.171 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.172 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.172 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.172 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.172 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.173 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.173 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:46:57.173 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.173 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.174 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.174 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.174 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.175 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.175 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.102267_47.1", 2022-09-03T05:46:57.176 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.176 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-8:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.8", 2022-09-03T05:46:57.176 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.102267896Z", 2022-09-03T05:46:57.176 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.177 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.177 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.177 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.177 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.178 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.178 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:46:57.178 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.179 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.179 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:57.180 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.180 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.180 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.180 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.181 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.104611_48.1", 2022-09-03T05:46:57.181 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.181 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-8:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.8", 2022-09-03T05:46:57.181 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.104611563Z", 2022-09-03T05:46:57.182 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.182 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.182 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.183 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.183 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.183 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.183 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:46:57.184 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.184 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.184 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.185 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.185 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.185 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.186 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.105641_49.1", 2022-09-03T05:46:57.186 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.186 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-8", 2022-09-03T05:46:57.186 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.105641481Z", 2022-09-03T05:46:57.187 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.187 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.187 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.188 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.188 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.188 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:46:57.189 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.189 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.189 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.189 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:57.190 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.190 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.191 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.108076_50.1", 2022-09-03T05:46:57.191 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.191 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-8", 2022-09-03T05:46:57.192 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.108076436Z", 2022-09-03T05:46:57.192 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.192 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.192 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.193 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.193 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.193 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:46:57.194 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.194 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.194 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.194 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.195 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.195 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.195 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.196 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.135979_51.1", 2022-09-03T05:46:57.196 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.196 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9", 2022-09-03T05:46:57.197 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.135979206Z", 2022-09-03T05:46:57.197 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.197 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.197 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.198 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.198 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.198 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:46:57.199 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.199 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.199 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:57.200 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.200 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.201 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.201 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.138975_52.1", 2022-09-03T05:46:57.201 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.201 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9", 2022-09-03T05:46:57.202 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.138975472Z", 2022-09-03T05:46:57.202 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.202 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.202 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.203 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.203 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.204 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:46:57.204 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.204 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.205 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.205 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.205 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.206 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.139759_53.1", 2022-09-03T05:46:57.206 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.206 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-9", 2022-09-03T05:46:57.207 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.139759635Z", 2022-09-03T05:46:57.207 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.207 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.208 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.208 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.208 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.209 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:46:57.209 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.209 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.209 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.210 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:57.210 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.210 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.210 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.211 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.141916_54.1", 2022-09-03T05:46:57.211 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.212 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-9", 2022-09-03T05:46:57.212 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.141916666Z", 2022-09-03T05:46:57.212 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.212 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.213 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:57.213 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:57.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.213 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.214 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:46:57.214 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.214 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.214 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.215 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.215 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.215 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.216 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.782802_55.1", 2022-09-03T05:46:57.216 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.217 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7", 2022-09-03T05:46:57.217 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.782802362Z", 2022-09-03T05:46:57.217 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.217 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.218 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:57.218 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.218 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.219 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:57.219 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:57.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.220 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.220 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:57.220 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.220 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.221 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.785254_56.1", 2022-09-03T05:46:57.221 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.222 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7", 2022-09-03T05:46:57.222 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.785254883Z", 2022-09-03T05:46:57.222 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.222 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.223 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:57.223 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.224 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.224 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:57.224 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:57.224 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.225 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.225 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.225 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.225 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.226 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.226 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.796744_57.1", 2022-09-03T05:46:57.226 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.227 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-7", 2022-09-03T05:46:57.227 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.796744879Z", 2022-09-03T05:46:57.227 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.227 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.228 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:46:57.228 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.229 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.229 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:46:57.229 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:57.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.230 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.230 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "remove" 2022-09-03T05:46:57.230 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.230 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.231 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.231 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.798860_58.1", 2022-09-03T05:46:57.231 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.232 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-7", 2022-09-03T05:46:57.232 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.798860427Z", 2022-09-03T05:46:57.232 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.232 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.233 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:46:57.233 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.234 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.234 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:46:57.234 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:57.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.235 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.235 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.235 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.235 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.236 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.833037_59.1", 2022-09-03T05:46:57.237 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.237 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-8:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.8", 2022-09-03T05:46:57.237 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.833037165Z", 2022-09-03T05:46:57.237 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.238 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.238 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:46:57.238 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.239 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.239 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:46:57.239 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:57.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.240 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.240 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:57.240 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.240 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.241 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.835414_60.1", 2022-09-03T05:46:57.241 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.242 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-8:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.8", 2022-09-03T05:46:57.242 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.835414206Z", 2022-09-03T05:46:57.242 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.243 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.243 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:46:57.243 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.244 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.244 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:46:57.244 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:57.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.245 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.245 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.245 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.245 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.246 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.843078_61.1", 2022-09-03T05:46:57.247 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.247 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-8", 2022-09-03T05:46:57.247 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.843078156Z", 2022-09-03T05:46:57.247 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.248 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.248 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:46:57.248 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.249 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.249 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:46:57.249 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:57.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.250 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.250 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "remove" 2022-09-03T05:46:57.250 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.251 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.251 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.845207_62.1", 2022-09-03T05:46:57.252 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.252 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-8", 2022-09-03T05:46:57.252 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.845207527Z", 2022-09-03T05:46:57.252 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.253 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.253 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:46:57.253 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.254 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.254 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:46:57.254 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:57.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.255 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.255 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.255 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.256 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.256 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.256 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.256 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.909335_63.1", 2022-09-03T05:46:57.257 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.257 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9", 2022-09-03T05:46:57.257 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.909335713Z", 2022-09-03T05:46:57.257 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.258 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.258 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:46:57.258 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.259 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.259 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:46:57.259 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:57.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.260 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.260 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:57.260 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.261 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.261 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.911665_64.1", 2022-09-03T05:46:57.262 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.262 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9", 2022-09-03T05:46:57.262 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.911665732Z", 2022-09-03T05:46:57.262 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.263 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.263 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:46:57.263 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.264 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.264 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:46:57.264 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:57.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.265 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.265 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.265 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.266 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.267 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.924885_65.1", 2022-09-03T05:46:57.267 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.267 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-9", 2022-09-03T05:46:57.267 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.924885371Z", 2022-09-03T05:46:57.268 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.268 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.268 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:46:57.268 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.269 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.269 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:46:57.269 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:57.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.270 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.270 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "remove" 2022-09-03T05:46:57.270 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.272 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:57.273 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.928498_66.1", 2022-09-03T05:46:57.273 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:57.274 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-9", 2022-09-03T05:46:57.274 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498747Z", 2022-09-03T05:46:57.274 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:57.274 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:57.275 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:46:57.275 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:57.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.276 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:57.276 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:46:57.276 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:57.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:57.277 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:57.277 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:57.277 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.277 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.278 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:57.278 INFO:teuthology.orchestra.run.smithi189.stdout:] 2022-09-03T05:46:57.280 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '41ecd2a5-afc0-4eae-9349-7dfe39baa1cb', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:46:57.280 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '41ecd2a5-afc0-4eae-9349-7dfe39baa1cb', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:46:57.281 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 41ecd2a5-afc0-4eae-9349-7dfe39baa1cb --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:46:57.334 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:46:57.335 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:46:57.384 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:46:57.385 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.385 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182298.903952_68.1", 2022-09-03T05:46:57.386 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "user", 2022-09-03T05:46:57.386 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "rgw-multisite-test-user", 2022-09-03T05:46:57.387 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.903952433Z", 2022-09-03T05:46:57.387 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.388 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.388 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.389 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.389 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.390 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.390 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_4zvH7kUaBnzKjfeQSSk_55d", 2022-09-03T05:46:57.390 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.391 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.391 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.392 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.393 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.393 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.394 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.394 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.395 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182298.906852_69.1", 2022-09-03T05:46:57.395 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "user", 2022-09-03T05:46:57.395 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "rgw-multisite-test-user", 2022-09-03T05:46:57.395 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852070Z", 2022-09-03T05:46:57.396 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.396 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.396 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.397 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.397 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.397 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.397 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_4zvH7kUaBnzKjfeQSSk_55d", 2022-09-03T05:46:57.398 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.398 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.398 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.399 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.399 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.399 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.399 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.400 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.400 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.694951_88.1", 2022-09-03T05:46:57.400 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.401 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10", 2022-09-03T05:46:57.401 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.694951214Z", 2022-09-03T05:46:57.401 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.401 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.402 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.402 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.402 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.403 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.403 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_9ORw1pyvfMIqMonbAbpO7T7", 2022-09-03T05:46:57.404 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.404 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.405 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.405 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.406 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.406 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.406 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.407 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.407 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.700326_89.1", 2022-09-03T05:46:57.408 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.408 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10", 2022-09-03T05:46:57.409 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.700326196Z", 2022-09-03T05:46:57.409 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.410 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.410 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.411 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.411 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.412 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.412 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_9ORw1pyvfMIqMonbAbpO7T7", 2022-09-03T05:46:57.413 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.413 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.414 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.414 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.415 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.415 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.416 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.416 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.417 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.702420_90.1", 2022-09-03T05:46:57.417 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.417 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-10", 2022-09-03T05:46:57.418 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.702420359Z", 2022-09-03T05:46:57.418 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.419 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.419 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.419 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.419 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.420 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.420 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "__K0-JaxveSMyJYue_ni7kIA", 2022-09-03T05:46:57.420 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.420 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.421 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.421 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.421 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.421 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.422 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.422 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.422 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.705433_91.1", 2022-09-03T05:46:57.422 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.423 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-10", 2022-09-03T05:46:57.423 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.705433036Z", 2022-09-03T05:46:57.423 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.423 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.424 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.424 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.424 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.424 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.425 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "__K0-JaxveSMyJYue_ni7kIA", 2022-09-03T05:46:57.425 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.425 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.425 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.426 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.426 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.426 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.427 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.427 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.427 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.724120_92.1", 2022-09-03T05:46:57.427 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.428 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11", 2022-09-03T05:46:57.428 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.724120949Z", 2022-09-03T05:46:57.428 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.428 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.429 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.429 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.429 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.429 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.430 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_mm_a_MgGT-mjAscVo5eDJN3", 2022-09-03T05:46:57.430 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.430 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.431 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.431 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.431 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.431 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.432 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.432 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.432 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.726647_93.1", 2022-09-03T05:46:57.432 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.433 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11", 2022-09-03T05:46:57.433 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.726647140Z", 2022-09-03T05:46:57.433 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.433 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.434 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.434 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.435 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.435 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_mm_a_MgGT-mjAscVo5eDJN3", 2022-09-03T05:46:57.435 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.435 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.436 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.436 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.437 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.437 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.438 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.439 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.439 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.727701_94.1", 2022-09-03T05:46:57.440 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.440 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-11", 2022-09-03T05:46:57.441 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.727701875Z", 2022-09-03T05:46:57.441 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.442 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.442 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.442 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.443 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.443 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.444 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_co69vPv41E29NQft_mgvTJ6", 2022-09-03T05:46:57.444 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.444 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.445 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.445 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.446 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.446 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.447 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.447 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.448 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.730089_95.1", 2022-09-03T05:46:57.448 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.449 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-11", 2022-09-03T05:46:57.449 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.730089458Z", 2022-09-03T05:46:57.449 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.450 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.450 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.451 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.451 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.452 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.452 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_co69vPv41E29NQft_mgvTJ6", 2022-09-03T05:46:57.452 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.452 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.453 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.453 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.453 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.453 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.454 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.454 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.454 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.760919_96.1", 2022-09-03T05:46:57.455 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.455 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12", 2022-09-03T05:46:57.455 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.760919958Z", 2022-09-03T05:46:57.455 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.456 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.456 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.456 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.457 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.457 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.457 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_87U_6FvKdHGzg3asBWz0TB6", 2022-09-03T05:46:57.458 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.458 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.459 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.459 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.460 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.460 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.460 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.460 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.461 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.763582_97.1", 2022-09-03T05:46:57.461 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.462 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12", 2022-09-03T05:46:57.462 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.763582008Z", 2022-09-03T05:46:57.463 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.463 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.464 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.464 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.465 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.465 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.466 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_87U_6FvKdHGzg3asBWz0TB6", 2022-09-03T05:46:57.466 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.467 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.467 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.468 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.468 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.469 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.469 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.469 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.470 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.764601_98.1", 2022-09-03T05:46:57.470 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.471 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-12", 2022-09-03T05:46:57.471 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.764601024Z", 2022-09-03T05:46:57.472 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.472 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.473 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.473 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.474 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.474 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.475 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_F5RlS3fhN6ILhiyoQ3qKUpC", 2022-09-03T05:46:57.475 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.476 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.476 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.476 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.477 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.477 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.478 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.478 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.478 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182436.766898_99.1", 2022-09-03T05:46:57.478 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.479 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-12", 2022-09-03T05:46:57.479 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:36.766898391Z", 2022-09-03T05:46:57.480 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.480 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.480 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.481 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.481 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.482 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.482 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_F5RlS3fhN6ILhiyoQ3qKUpC", 2022-09-03T05:46:57.483 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.483 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.484 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.484 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.485 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.485 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.486 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.486 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.487 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182745.944715_100.1", 2022-09-03T05:46:57.487 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.488 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13", 2022-09-03T05:46:57.488 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:45.944715403Z", 2022-09-03T05:46:57.489 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.489 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.489 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.490 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.490 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.491 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.491 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_ZwtCDnJJncqHbK6_XX4E-Uv", 2022-09-03T05:46:57.491 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.492 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.492 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.493 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.493 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.493 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.493 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.494 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.494 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182745.948069_101.1", 2022-09-03T05:46:57.494 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.495 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13", 2022-09-03T05:46:57.495 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:45.948069121Z", 2022-09-03T05:46:57.495 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.495 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.496 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.496 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.497 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.497 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.497 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_ZwtCDnJJncqHbK6_XX4E-Uv", 2022-09-03T05:46:57.498 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.498 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.498 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.498 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.499 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.499 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.499 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.500 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.500 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182745.950114_102.1", 2022-09-03T05:46:57.500 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.500 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-13", 2022-09-03T05:46:57.501 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:45.950114676Z", 2022-09-03T05:46:57.501 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.501 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.502 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.502 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.503 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.503 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.503 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_kkkmqlwBfjjtwLr8U95H7wF", 2022-09-03T05:46:57.504 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.505 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.505 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.505 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.506 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.506 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.507 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.507 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.508 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182745.953048_103.1", 2022-09-03T05:46:57.508 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.509 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-13", 2022-09-03T05:46:57.509 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:45.953048780Z", 2022-09-03T05:46:57.510 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.510 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.511 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.511 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.511 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.512 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.512 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_kkkmqlwBfjjtwLr8U95H7wF", 2022-09-03T05:46:57.512 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.513 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.513 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.514 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.514 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.515 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.515 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.516 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.516 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182745.988740_104.1", 2022-09-03T05:46:57.516 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.517 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14", 2022-09-03T05:46:57.517 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:45.988740832Z", 2022-09-03T05:46:57.518 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.518 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.519 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.519 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.520 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.520 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.521 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Yidm7pQHzXjzyowF0vXCv_q", 2022-09-03T05:46:57.521 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.522 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.522 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.523 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.523 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.524 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.524 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.525 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.525 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182745.991668_105.1", 2022-09-03T05:46:57.525 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.526 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14", 2022-09-03T05:46:57.526 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:45.991668492Z", 2022-09-03T05:46:57.526 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.526 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.527 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.527 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.527 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.528 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.528 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Yidm7pQHzXjzyowF0vXCv_q", 2022-09-03T05:46:57.528 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.528 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.529 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.529 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.529 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.529 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.530 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.530 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.530 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182745.992722_106.1", 2022-09-03T05:46:57.531 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.531 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-14", 2022-09-03T05:46:57.531 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:45.992722013Z", 2022-09-03T05:46:57.531 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.532 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.532 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.532 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.532 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.533 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.533 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_0ZpMZLjvQ0yH4NIki0Mppzv", 2022-09-03T05:46:57.533 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.533 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.534 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.534 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.534 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.535 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.535 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.535 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.535 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182745.995451_107.1", 2022-09-03T05:46:57.536 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.536 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-14", 2022-09-03T05:46:57.536 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:45.995451716Z", 2022-09-03T05:46:57.536 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.537 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.537 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.537 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.537 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.538 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.538 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_0ZpMZLjvQ0yH4NIki0Mppzv", 2022-09-03T05:46:57.538 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.539 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.539 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.539 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.540 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.540 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.540 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.541 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.541 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182746.030460_108.1", 2022-09-03T05:46:57.541 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.542 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15", 2022-09-03T05:46:57.542 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:46.030460591Z", 2022-09-03T05:46:57.542 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.542 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.543 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.543 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.543 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.543 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.544 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_9EbzjhdvUtwpRj9gUb6Zy_c", 2022-09-03T05:46:57.544 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.544 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.544 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.545 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.545 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.545 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.546 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.546 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.546 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182746.033416_109.1", 2022-09-03T05:46:57.546 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.547 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15", 2022-09-03T05:46:57.547 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:46.033416114Z", 2022-09-03T05:46:57.547 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.548 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.548 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.549 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.549 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.549 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.550 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_9EbzjhdvUtwpRj9gUb6Zy_c", 2022-09-03T05:46:57.550 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.551 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.552 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.552 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.553 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.553 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.554 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.554 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.555 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182746.035561_110.1", 2022-09-03T05:46:57.555 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.555 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-15", 2022-09-03T05:46:57.556 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:46.035561132Z", 2022-09-03T05:46:57.556 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.557 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.557 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.558 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.558 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.559 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.559 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_p1RJcPdLE0s_MQy_T3ZeW1Z", 2022-09-03T05:46:57.560 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.560 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.561 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.562 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.562 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.563 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.563 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.564 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.564 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182746.041492_111.1", 2022-09-03T05:46:57.564 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.565 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-15", 2022-09-03T05:46:57.565 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:25:46.041492579Z", 2022-09-03T05:46:57.566 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.566 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.567 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.567 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.568 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.568 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.569 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_p1RJcPdLE0s_MQy_T3ZeW1Z", 2022-09-03T05:46:57.569 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.570 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.570 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.571 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.571 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.572 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.572 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.573 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.573 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182851.903860_122.1", 2022-09-03T05:46:57.574 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.574 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16", 2022-09-03T05:46:57.575 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:31.903860682Z", 2022-09-03T05:46:57.575 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.576 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.576 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.577 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.577 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.578 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.578 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:57.579 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.579 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.579 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.579 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.580 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.580 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.581 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.581 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.581 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182851.907991_123.1", 2022-09-03T05:46:57.581 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.582 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16", 2022-09-03T05:46:57.582 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:31.907991084Z", 2022-09-03T05:46:57.582 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.583 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.583 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.583 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.583 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.584 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.584 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:57.584 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.584 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.585 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.585 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.585 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.586 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.586 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.586 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.586 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182851.909130_124.1", 2022-09-03T05:46:57.587 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.587 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-16", 2022-09-03T05:46:57.587 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:31.909130687Z", 2022-09-03T05:46:57.588 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.588 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.588 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.588 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.589 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.589 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.589 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_P0q3AfXqSQL8XW6gVnA4CRA", 2022-09-03T05:46:57.590 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.590 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.590 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.590 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.591 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.591 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.591 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.592 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.592 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182851.915199_125.1", 2022-09-03T05:46:57.592 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.592 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-16", 2022-09-03T05:46:57.593 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:31.915199302Z", 2022-09-03T05:46:57.593 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.594 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.594 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.594 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.594 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.595 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.595 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_P0q3AfXqSQL8XW6gVnA4CRA", 2022-09-03T05:46:57.595 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.596 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.596 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.596 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.596 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.597 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.597 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.597 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.597 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182851.945316_126.1", 2022-09-03T05:46:57.598 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.598 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17", 2022-09-03T05:46:57.599 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:31.945316218Z", 2022-09-03T05:46:57.599 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.599 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.600 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.600 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.601 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.601 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.602 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:57.602 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.602 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.603 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.603 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.603 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.603 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.604 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.604 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.604 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182851.950120_127.1", 2022-09-03T05:46:57.604 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.605 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17", 2022-09-03T05:46:57.605 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:31.950120909Z", 2022-09-03T05:46:57.606 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.606 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.607 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.607 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.608 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.608 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.609 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:57.609 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.610 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.610 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.611 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.611 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.612 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.612 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.613 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.613 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182851.952454_128.1", 2022-09-03T05:46:57.614 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.614 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-17", 2022-09-03T05:46:57.614 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:31.952454005Z", 2022-09-03T05:46:57.615 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.615 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.616 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.616 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.617 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.617 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.618 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_j4__iZ2_YM9iD4VCcQhy-lG", 2022-09-03T05:46:57.618 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.619 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.619 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.620 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.620 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.620 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.621 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.621 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.621 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182851.958649_129.1", 2022-09-03T05:46:57.622 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.622 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-17", 2022-09-03T05:46:57.622 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:31.958649291Z", 2022-09-03T05:46:57.623 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.623 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.623 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.624 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.624 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.625 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.625 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_j4__iZ2_YM9iD4VCcQhy-lG", 2022-09-03T05:46:57.626 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.626 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.627 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.627 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.628 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.628 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.629 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.629 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.630 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182851.998554_130.1", 2022-09-03T05:46:57.630 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.630 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18", 2022-09-03T05:46:57.631 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:31.998554746Z", 2022-09-03T05:46:57.631 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.631 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.631 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.632 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.632 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.632 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.633 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:57.633 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.633 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.633 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.634 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.634 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.634 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.634 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.635 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.635 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182852.004610_131.1", 2022-09-03T05:46:57.635 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.635 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18", 2022-09-03T05:46:57.636 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:32.004610066Z", 2022-09-03T05:46:57.636 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.636 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.636 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.637 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.637 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.637 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.638 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:57.638 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.638 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.638 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.639 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.639 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.639 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.639 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.640 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.640 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182852.007026_132.1", 2022-09-03T05:46:57.640 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.640 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-18", 2022-09-03T05:46:57.641 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:32.007026261Z", 2022-09-03T05:46:57.641 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.641 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.641 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.642 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.642 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.642 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.643 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_O7UdALq44EBf7qF8RLPxZMv", 2022-09-03T05:46:57.643 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.643 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.643 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.644 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.644 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.644 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.644 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.645 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.645 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182852.013139_133.1", 2022-09-03T05:46:57.645 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.646 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-18", 2022-09-03T05:46:57.646 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:32.013139549Z", 2022-09-03T05:46:57.646 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.647 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.647 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.648 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.648 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.649 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.649 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_O7UdALq44EBf7qF8RLPxZMv", 2022-09-03T05:46:57.649 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.650 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.650 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.650 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.650 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.651 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.651 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.651 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.651 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182852.069466_134.1", 2022-09-03T05:46:57.652 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.652 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16", 2022-09-03T05:46:57.652 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:32.069466103Z", 2022-09-03T05:46:57.652 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.653 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.653 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:57.653 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.653 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.654 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.654 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:57.654 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.655 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.655 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.655 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.655 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.656 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.656 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.656 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.656 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182852.078407_135.1", 2022-09-03T05:46:57.657 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.657 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16", 2022-09-03T05:46:57.657 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:32.078407455Z", 2022-09-03T05:46:57.657 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.658 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.658 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:57.658 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.659 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.659 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.659 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:57.659 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.660 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.660 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.660 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.660 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.661 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.661 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.661 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.662 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182852.095941_136.1", 2022-09-03T05:46:57.662 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.662 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17", 2022-09-03T05:46:57.662 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:32.095941153Z", 2022-09-03T05:46:57.663 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.663 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.663 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:57.664 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.664 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.664 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.665 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:57.665 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.665 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.665 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.666 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.666 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.666 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.666 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.667 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.667 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182852.099371_137.1", 2022-09-03T05:46:57.667 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.668 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17", 2022-09-03T05:46:57.668 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:32.099371547Z", 2022-09-03T05:46:57.668 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.668 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.669 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:57.669 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.669 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.669 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.670 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:57.670 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.670 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.670 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.671 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.671 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.671 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.671 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.672 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.672 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182852.121946_138.1", 2022-09-03T05:46:57.672 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.672 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18", 2022-09-03T05:46:57.673 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:32.121946745Z", 2022-09-03T05:46:57.673 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.673 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.673 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:57.674 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.674 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.674 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.674 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:57.675 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.675 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.675 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.676 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.676 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.676 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.676 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.677 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.677 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182852.124949_139.1", 2022-09-03T05:46:57.677 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.678 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18", 2022-09-03T05:46:57.678 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:27:32.124949763Z", 2022-09-03T05:46:57.678 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.678 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.679 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:57.679 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.679 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.679 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.680 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:57.680 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.680 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.680 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.681 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.681 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.681 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.681 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.682 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.682 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183242.240665_157.1", 2022-09-03T05:46:57.682 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.683 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19", 2022-09-03T05:46:57.683 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:34:02.240665028Z", 2022-09-03T05:46:57.683 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.683 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.684 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.684 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.684 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.684 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.685 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:57.685 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.685 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.685 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.686 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.686 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.686 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.686 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.687 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.687 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183242.243600_158.1", 2022-09-03T05:46:57.687 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.687 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19", 2022-09-03T05:46:57.688 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:34:02.243600484Z", 2022-09-03T05:46:57.688 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.688 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.689 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.689 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.689 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.689 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.690 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:57.690 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.690 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.690 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.691 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.691 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.691 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.691 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.692 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.692 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183242.244744_159.1", 2022-09-03T05:46:57.692 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.692 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-19", 2022-09-03T05:46:57.693 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:34:02.244744943Z", 2022-09-03T05:46:57.693 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.693 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.693 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.694 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.694 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.694 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.695 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_B_TWex9_lpCif6prNO7BbtO", 2022-09-03T05:46:57.695 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.695 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.695 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.696 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.696 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.696 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.696 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.697 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.697 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183242.247625_160.1", 2022-09-03T05:46:57.697 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.697 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-19", 2022-09-03T05:46:57.698 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:34:02.247625174Z", 2022-09-03T05:46:57.698 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.698 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.698 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.699 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.699 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.699 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.700 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_B_TWex9_lpCif6prNO7BbtO", 2022-09-03T05:46:57.700 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.700 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.700 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.701 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.701 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.701 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.701 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.702 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.702 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183242.265141_161.1", 2022-09-03T05:46:57.702 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.702 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19", 2022-09-03T05:46:57.703 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:34:02.265141968Z", 2022-09-03T05:46:57.703 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.703 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.703 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:57.704 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.704 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.704 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.705 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:57.705 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.705 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.705 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.706 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.706 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.706 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.706 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.707 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.707 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183242.267929_162.1", 2022-09-03T05:46:57.707 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.708 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19", 2022-09-03T05:46:57.708 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:34:02.267929270Z", 2022-09-03T05:46:57.708 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.708 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.709 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:57.709 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.709 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.709 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.710 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:57.710 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.710 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.710 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.711 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.711 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.711 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.711 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.712 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.712 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183458.677214_163.1", 2022-09-03T05:46:57.712 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.712 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:57.713 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:37:38.677214984Z", 2022-09-03T05:46:57.713 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.713 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.714 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.714 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.714 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.715 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.715 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:57.715 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.715 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.716 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.716 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.716 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.717 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.717 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.717 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.717 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183458.684800_164.1", 2022-09-03T05:46:57.718 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.718 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:57.718 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:37:38.684800313Z", 2022-09-03T05:46:57.719 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.719 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.719 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.719 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.720 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.720 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.720 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:57.720 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.721 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.721 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.721 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.721 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.722 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.722 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.722 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.723 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183458.685753_165.1", 2022-09-03T05:46:57.723 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.723 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-20", 2022-09-03T05:46:57.723 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:37:38.685753497Z", 2022-09-03T05:46:57.724 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.724 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.724 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.724 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.725 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.725 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.725 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_yPpDuhKk6Xsxh9Gd4Ds8exJ", 2022-09-03T05:46:57.725 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.726 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.726 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.726 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.726 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.727 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.727 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.727 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.728 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183458.694130_166.1", 2022-09-03T05:46:57.728 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.728 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-20", 2022-09-03T05:46:57.728 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:37:38.694130749Z", 2022-09-03T05:46:57.729 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.729 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.729 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.729 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.730 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.730 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.730 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_yPpDuhKk6Xsxh9Gd4Ds8exJ", 2022-09-03T05:46:57.730 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.731 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.731 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.731 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.731 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.732 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.732 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.732 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.732 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183531.352224_183.1", 2022-09-03T05:46:57.733 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.733 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:57.733 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:38:51.352224643Z", 2022-09-03T05:46:57.733 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.734 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.734 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:57.734 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.735 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.735 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.735 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:57.735 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.736 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.736 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.736 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.736 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.737 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.737 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.737 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.737 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183531.355467_184.1", 2022-09-03T05:46:57.738 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.738 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:57.738 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:38:51.355467679Z", 2022-09-03T05:46:57.738 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.739 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.739 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:57.739 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.739 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.740 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.740 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:57.740 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.740 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.741 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.741 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.741 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.741 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.742 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.742 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.742 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183570.365072_185.1", 2022-09-03T05:46:57.743 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.743 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:57.743 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:39:30.365072646Z", 2022-09-03T05:46:57.743 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.744 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.744 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:57.744 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.745 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.745 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.745 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:57.745 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 3 2022-09-03T05:46:57.746 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.746 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.746 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.747 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.747 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.747 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.747 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.748 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183570.368454_186.1", 2022-09-03T05:46:57.748 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.748 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:57.748 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:39:30.368454229Z", 2022-09-03T05:46:57.749 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.749 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.749 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:57.749 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.750 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.750 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.750 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:57.751 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 3 2022-09-03T05:46:57.751 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.751 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.751 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.752 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.752 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.752 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.753 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.753 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183652.125862_187.1", 2022-09-03T05:46:57.753 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.753 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21", 2022-09-03T05:46:57.754 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:52.125862114Z", 2022-09-03T05:46:57.754 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.754 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.755 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.755 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.755 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.755 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.756 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:57.756 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.756 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.757 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.757 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.757 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.758 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.758 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.758 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.759 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183652.132077_188.1", 2022-09-03T05:46:57.759 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.759 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21", 2022-09-03T05:46:57.759 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:52.132077397Z", 2022-09-03T05:46:57.760 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.760 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.760 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.761 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.761 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.761 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.761 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:57.762 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.762 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.763 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.763 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.763 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.763 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.764 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.764 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.764 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183652.134378_189.1", 2022-09-03T05:46:57.765 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.765 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-21", 2022-09-03T05:46:57.765 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:52.134378414Z", 2022-09-03T05:46:57.766 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.766 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.766 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.766 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.767 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.767 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.767 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_EARZCit60fDg0eMLHHZC990", 2022-09-03T05:46:57.768 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.768 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.768 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.768 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.769 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.769 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.769 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.769 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.770 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183652.140411_190.1", 2022-09-03T05:46:57.770 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.770 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-21", 2022-09-03T05:46:57.770 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:52.140411296Z", 2022-09-03T05:46:57.771 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.771 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.771 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.771 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.772 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.772 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.772 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_EARZCit60fDg0eMLHHZC990", 2022-09-03T05:46:57.772 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.773 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.773 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.773 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.773 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.774 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.774 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.774 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.774 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.162940_191.1", 2022-09-03T05:46:57.775 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.775 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22", 2022-09-03T05:46:57.775 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.162940466Z", 2022-09-03T05:46:57.776 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.776 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.776 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.776 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.777 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.777 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.777 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:57.777 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.778 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.778 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.778 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.778 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.779 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.779 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.779 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.779 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.167017_192.1", 2022-09-03T05:46:57.780 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.780 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22", 2022-09-03T05:46:57.780 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.167017131Z", 2022-09-03T05:46:57.781 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.781 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.781 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.781 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.782 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.782 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.782 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:57.782 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.783 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.783 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.783 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.783 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.784 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.784 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.784 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.784 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.167997_193.1", 2022-09-03T05:46:57.785 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.785 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-22", 2022-09-03T05:46:57.785 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.167997739Z", 2022-09-03T05:46:57.785 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.786 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.786 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.786 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.786 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.787 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.787 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tI1b7-ygEU1TnWToSDcKGBc", 2022-09-03T05:46:57.787 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.787 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.788 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.788 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.788 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.788 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.789 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.789 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.789 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.170492_194.1", 2022-09-03T05:46:57.789 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.790 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-22", 2022-09-03T05:46:57.790 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.170492762Z", 2022-09-03T05:46:57.790 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.791 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.791 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.791 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.791 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.792 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.792 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tI1b7-ygEU1TnWToSDcKGBc", 2022-09-03T05:46:57.792 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.792 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.793 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.793 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.793 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.794 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.794 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.794 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.794 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.220411_195.1", 2022-09-03T05:46:57.795 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.795 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23", 2022-09-03T05:46:57.795 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.220411899Z", 2022-09-03T05:46:57.795 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.796 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.796 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.796 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.796 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.797 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.797 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:57.797 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.797 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.798 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.798 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.798 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.799 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.799 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.799 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.799 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.223172_196.1", 2022-09-03T05:46:57.800 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.800 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23", 2022-09-03T05:46:57.800 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.223172774Z", 2022-09-03T05:46:57.800 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.801 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.801 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.801 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.801 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.802 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.802 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:57.802 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.802 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.803 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.803 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.803 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.803 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.804 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.804 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.804 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.224266_197.1", 2022-09-03T05:46:57.804 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.805 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-23", 2022-09-03T05:46:57.805 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.224266788Z", 2022-09-03T05:46:57.805 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.806 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.806 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.806 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.806 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.807 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.807 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_mrTr4jlOIDprI-5OTphGZ6h", 2022-09-03T05:46:57.807 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.807 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.808 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.808 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.808 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.808 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.809 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.809 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.809 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.226840_198.1", 2022-09-03T05:46:57.810 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.810 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-23", 2022-09-03T05:46:57.810 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.226840552Z", 2022-09-03T05:46:57.811 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.811 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.811 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.811 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.812 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.812 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.812 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_mrTr4jlOIDprI-5OTphGZ6h", 2022-09-03T05:46:57.812 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.813 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.813 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.813 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.813 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.814 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.814 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.814 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.815 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.267809_199.1", 2022-09-03T05:46:57.815 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.815 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21", 2022-09-03T05:46:57.816 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.267809597Z", 2022-09-03T05:46:57.816 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.816 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.817 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:57.817 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.818 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.818 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.818 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:57.819 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.819 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.819 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.819 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.820 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.820 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.820 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.820 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.821 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.270381_200.1", 2022-09-03T05:46:57.821 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.821 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21", 2022-09-03T05:46:57.821 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.270381683Z", 2022-09-03T05:46:57.822 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.822 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.822 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:57.823 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.823 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.823 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.823 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:57.824 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.824 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.824 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.824 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.825 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.825 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.826 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.826 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.827 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.283633_201.1", 2022-09-03T05:46:57.827 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.827 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22", 2022-09-03T05:46:57.828 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.283633845Z", 2022-09-03T05:46:57.828 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.829 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.829 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:57.830 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.830 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.831 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.831 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:57.832 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.833 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.833 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.834 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.834 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.835 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.835 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.836 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.836 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.288001_202.1", 2022-09-03T05:46:57.837 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.837 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22", 2022-09-03T05:46:57.837 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.288001351Z", 2022-09-03T05:46:57.838 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.838 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.839 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:57.839 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.840 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.840 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.841 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:57.841 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.842 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.842 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.843 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.843 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.844 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.844 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.845 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.845 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.304152_203.1", 2022-09-03T05:46:57.845 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.846 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23", 2022-09-03T05:46:57.846 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.304152195Z", 2022-09-03T05:46:57.847 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.847 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.847 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:57.848 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.848 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.849 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.849 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:57.850 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.850 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.851 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.851 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.852 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.853 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.853 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.853 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.854 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183653.308405_204.1", 2022-09-03T05:46:57.854 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.855 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23", 2022-09-03T05:46:57.855 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:40:53.308405313Z", 2022-09-03T05:46:57.856 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.856 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.857 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:57.857 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.858 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.858 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.859 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:57.859 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.860 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.860 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.860 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.861 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.862 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.862 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.863 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.863 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.657291_207.1", 2022-09-03T05:46:57.864 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.864 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:57.865 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.657291334Z", 2022-09-03T05:46:57.865 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.866 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.866 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.866 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.867 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.867 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.868 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:57.868 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.869 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.869 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.869 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.870 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.870 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.870 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.870 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.871 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.661364_208.1", 2022-09-03T05:46:57.871 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.871 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:57.872 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.661364231Z", 2022-09-03T05:46:57.872 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.872 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.872 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.873 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.873 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.873 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.874 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:57.874 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.874 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.874 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.875 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.875 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.875 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.875 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.876 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.876 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.662317_209.1", 2022-09-03T05:46:57.876 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.876 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-24", 2022-09-03T05:46:57.877 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.662317133Z", 2022-09-03T05:46:57.877 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.877 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.878 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.878 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.878 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.878 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.879 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_RWbG2F3yjlSXFQy-Qq_ON1N", 2022-09-03T05:46:57.879 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.879 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.879 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.880 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.880 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.880 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.881 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.881 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.881 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.666554_210.1", 2022-09-03T05:46:57.881 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.882 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-24", 2022-09-03T05:46:57.882 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.666554708Z", 2022-09-03T05:46:57.882 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.882 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.883 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.883 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.883 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.884 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.884 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_RWbG2F3yjlSXFQy-Qq_ON1N", 2022-09-03T05:46:57.884 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.884 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.885 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.885 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.885 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.885 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.886 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.886 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.886 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.685377_211.1", 2022-09-03T05:46:57.887 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.887 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:57.887 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.685377688Z", 2022-09-03T05:46:57.887 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.888 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.888 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.888 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.888 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.889 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.889 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:57.889 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.889 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.890 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.890 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.890 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.891 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.891 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.891 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.891 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.689809_212.1", 2022-09-03T05:46:57.892 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.892 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:57.892 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.689809673Z", 2022-09-03T05:46:57.892 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.893 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.893 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.893 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.893 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.894 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.894 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:57.894 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.895 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.895 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.895 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.895 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.896 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.896 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.896 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.896 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.690762_213.1", 2022-09-03T05:46:57.897 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.897 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-25", 2022-09-03T05:46:57.897 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.690762414Z", 2022-09-03T05:46:57.898 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.898 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.898 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.898 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.899 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.899 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.899 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_IaGw1AfiCAqC-aV2Oup3OCi", 2022-09-03T05:46:57.899 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.900 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.900 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.900 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.901 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.901 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.901 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.901 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.902 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.693239_214.1", 2022-09-03T05:46:57.902 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.902 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-25", 2022-09-03T05:46:57.902 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.693239599Z", 2022-09-03T05:46:57.903 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.903 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.903 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.904 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.904 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.904 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.904 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_IaGw1AfiCAqC-aV2Oup3OCi", 2022-09-03T05:46:57.905 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.905 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.905 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.906 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.906 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.906 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.907 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.907 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.907 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.727704_215.1", 2022-09-03T05:46:57.907 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.908 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:57.908 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.727704277Z", 2022-09-03T05:46:57.908 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.908 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.909 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.909 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.909 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.910 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.910 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:57.910 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.910 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.911 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.911 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.911 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.911 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.912 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.912 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.912 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.731553_216.1", 2022-09-03T05:46:57.913 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.913 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:57.913 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.731553123Z", 2022-09-03T05:46:57.913 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.914 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.914 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.914 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.914 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.915 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.915 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:57.915 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.916 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.916 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.916 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.916 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.917 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.917 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.917 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.917 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.732533_217.1", 2022-09-03T05:46:57.918 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.918 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-26", 2022-09-03T05:46:57.918 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.732533246Z", 2022-09-03T05:46:57.919 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.919 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.919 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.920 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.920 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.920 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.920 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_dUSEiciichvry8cYc4lxW96", 2022-09-03T05:46:57.921 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.921 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.921 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.921 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.922 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.922 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.922 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.923 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.923 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.734903_218.1", 2022-09-03T05:46:57.923 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:57.923 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-26", 2022-09-03T05:46:57.924 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.734903969Z", 2022-09-03T05:46:57.924 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.924 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.924 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.925 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.925 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.925 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.926 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_dUSEiciichvry8cYc4lxW96", 2022-09-03T05:46:57.926 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.926 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.926 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.927 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.927 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.927 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.927 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.928 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.928 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.763839_219.1", 2022-09-03T05:46:57.928 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.929 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:57.929 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.763839115Z", 2022-09-03T05:46:57.929 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.929 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.930 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:57.930 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.930 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.930 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.931 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:57.931 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.931 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.932 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.932 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.932 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.932 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.933 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.933 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.933 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.766491_220.1", 2022-09-03T05:46:57.933 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.934 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:57.934 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.766491425Z", 2022-09-03T05:46:57.934 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.934 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.935 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:57.935 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.935 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.936 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.936 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:57.936 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.936 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.937 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.937 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.937 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.938 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.938 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.938 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.938 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.780440_221.1", 2022-09-03T05:46:57.939 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.939 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:57.939 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.780440878Z", 2022-09-03T05:46:57.939 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.940 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.940 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:57.940 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.941 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.941 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.941 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:57.941 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 3 2022-09-03T05:46:57.942 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.942 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.943 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.943 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.943 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.944 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.944 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.944 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.784428_222.1", 2022-09-03T05:46:57.944 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.945 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:57.945 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.784428128Z", 2022-09-03T05:46:57.945 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.945 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.946 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:57.946 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.946 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.947 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.947 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:57.947 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 3 2022-09-03T05:46:57.947 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.948 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.948 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.948 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.948 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.949 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.949 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.949 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.798318_223.1", 2022-09-03T05:46:57.949 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.950 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:57.950 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.798318959Z", 2022-09-03T05:46:57.950 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.951 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.951 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:57.951 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.951 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.952 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.952 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:57.952 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.952 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.953 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.953 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.953 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.954 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.954 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.954 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.954 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.802397_224.1", 2022-09-03T05:46:57.955 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.955 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:57.955 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.802397402Z", 2022-09-03T05:46:57.956 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.956 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.956 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:57.956 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.957 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.957 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.957 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:57.957 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.958 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.958 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.958 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.958 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.959 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.959 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.959 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.960 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.818536_225.1", 2022-09-03T05:46:57.960 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.960 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:57.960 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.818536670Z", 2022-09-03T05:46:57.961 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.961 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.961 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:57.961 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.962 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.962 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.962 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:57.963 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 3 2022-09-03T05:46:57.963 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.963 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.963 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.964 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.964 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.964 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.964 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.965 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.826419_226.1", 2022-09-03T05:46:57.965 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.965 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:57.966 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.826419304Z", 2022-09-03T05:46:57.966 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.966 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.966 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:57.967 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.967 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.967 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.967 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:57.968 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 3 2022-09-03T05:46:57.968 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.968 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.969 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.969 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.969 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.969 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.970 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.970 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.854560_227.1", 2022-09-03T05:46:57.970 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.970 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:57.971 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.854560270Z", 2022-09-03T05:46:57.971 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.971 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.971 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:57.972 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.972 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.972 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.972 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:57.973 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.973 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.973 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.974 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.974 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.974 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.974 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.975 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.975 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.857527_228.1", 2022-09-03T05:46:57.975 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.975 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:57.976 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.857527676Z", 2022-09-03T05:46:57.976 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.976 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.977 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:57.977 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.977 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.977 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.978 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:57.978 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.978 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.978 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.979 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.979 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.979 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.979 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.980 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.980 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.870879_229.1", 2022-09-03T05:46:57.980 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.980 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:57.981 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.870879960Z", 2022-09-03T05:46:57.981 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.981 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.982 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:57.982 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.982 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.982 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.983 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:57.983 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 3 2022-09-03T05:46:57.983 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.983 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.984 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.984 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.984 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.985 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.985 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.985 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183709.876923_230.1", 2022-09-03T05:46:57.985 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.986 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:57.986 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:41:49.876923670Z", 2022-09-03T05:46:57.986 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.986 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.987 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:57.987 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:57.987 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.987 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.988 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:57.988 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 3 2022-09-03T05:46:57.988 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.989 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.989 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:57.989 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.989 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.990 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.990 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.990 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.870437_232.1", 2022-09-03T05:46:57.990 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.991 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-27:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.27", 2022-09-03T05:46:57.991 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.870437527Z", 2022-09-03T05:46:57.991 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.991 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.992 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.992 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.992 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.992 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.993 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:57.993 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.993 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.994 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:57.994 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:57.994 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.994 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:57.995 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.995 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:57.995 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.874799_233.1", 2022-09-03T05:46:57.995 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:57.996 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-27:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.27", 2022-09-03T05:46:57.996 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.874799084Z", 2022-09-03T05:46:57.996 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:57.996 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:57.997 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:57.997 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:57.997 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.997 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:57.998 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:57.999 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:57.999 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:57.999 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.000 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.000 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.000 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.001 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.001 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.001 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.876797_234.1", 2022-09-03T05:46:58.001 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.002 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-27", 2022-09-03T05:46:58.002 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.876797147Z", 2022-09-03T05:46:58.002 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.002 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.003 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.003 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.003 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.004 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.004 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_kvVMhGyMiz3kusJkbwnyhxJ", 2022-09-03T05:46:58.004 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.004 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.005 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.005 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.005 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.006 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.006 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.006 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.006 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.885232_235.1", 2022-09-03T05:46:58.007 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.007 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-27", 2022-09-03T05:46:58.007 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.885232353Z", 2022-09-03T05:46:58.008 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.008 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.008 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.008 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.009 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.009 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.009 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_kvVMhGyMiz3kusJkbwnyhxJ", 2022-09-03T05:46:58.009 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.010 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.010 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.010 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.011 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.011 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.011 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.011 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.012 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.916915_236.1", 2022-09-03T05:46:58.012 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.012 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28", 2022-09-03T05:46:58.013 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.916915113Z", 2022-09-03T05:46:58.013 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.013 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.013 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.014 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.014 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.014 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.014 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:58.015 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.015 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.015 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.016 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.016 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.016 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.016 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.017 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.017 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.920958_237.1", 2022-09-03T05:46:58.017 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.017 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28", 2022-09-03T05:46:58.018 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.920958536Z", 2022-09-03T05:46:58.018 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.018 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.019 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.019 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.019 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.019 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.020 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:58.020 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.020 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.020 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.021 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.021 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.021 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.021 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.022 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.022 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.922045_238.1", 2022-09-03T05:46:58.022 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.023 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-28", 2022-09-03T05:46:58.023 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.922045913Z", 2022-09-03T05:46:58.023 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.023 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.024 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.024 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.024 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.024 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.025 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_OOWZg42AYED_QBCONdEGECB", 2022-09-03T05:46:58.025 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.025 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.025 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.026 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.026 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.026 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.035 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.035 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.035 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.926094_239.1", 2022-09-03T05:46:58.036 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.036 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-28", 2022-09-03T05:46:58.036 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.926094548Z", 2022-09-03T05:46:58.037 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.037 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.037 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.037 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.038 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.038 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.038 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_OOWZg42AYED_QBCONdEGECB", 2022-09-03T05:46:58.039 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.039 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.039 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.039 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.040 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.040 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.040 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.040 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.041 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.962517_240.1", 2022-09-03T05:46:58.041 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.041 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-29:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.29", 2022-09-03T05:46:58.042 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.962517073Z", 2022-09-03T05:46:58.042 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.042 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.042 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.043 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.043 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.043 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.044 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:58.044 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.044 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.044 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.045 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.045 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.045 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.045 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.046 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.046 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.965127_241.1", 2022-09-03T05:46:58.046 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.047 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-29:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.29", 2022-09-03T05:46:58.047 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.965127874Z", 2022-09-03T05:46:58.047 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.047 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.048 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.048 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.048 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.048 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.049 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:58.049 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.049 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.050 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.050 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.050 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.050 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.051 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.051 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.051 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.966081_242.1", 2022-09-03T05:46:58.051 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.052 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-29", 2022-09-03T05:46:58.052 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.966081501Z", 2022-09-03T05:46:58.052 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.052 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.053 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.053 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.053 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.054 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.054 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_yDkLaMdkLBW9LxbH_x72oyR", 2022-09-03T05:46:58.054 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.054 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.055 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.055 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.055 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.055 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.056 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.056 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.056 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.968432_243.1", 2022-09-03T05:46:58.057 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.057 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-29", 2022-09-03T05:46:58.057 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.968432661Z", 2022-09-03T05:46:58.058 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.058 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.058 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.058 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.059 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.059 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.059 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_yDkLaMdkLBW9LxbH_x72oyR", 2022-09-03T05:46:58.059 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.060 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.060 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.060 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.060 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.061 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.061 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.061 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.062 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183849.998970_244.1", 2022-09-03T05:46:58.062 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.062 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-27:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.27", 2022-09-03T05:46:58.062 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:09.998970343Z", 2022-09-03T05:46:58.063 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.063 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.063 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:58.063 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.064 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.064 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.064 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:58.065 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.065 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.065 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.065 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.066 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.066 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.066 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.066 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.067 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.001548_245.1", 2022-09-03T05:46:58.067 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.067 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-27:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.27", 2022-09-03T05:46:58.067 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.001548177Z", 2022-09-03T05:46:58.068 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.068 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.068 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:58.068 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.069 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.069 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.069 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:58.070 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.070 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.070 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.070 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.071 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.071 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.071 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.071 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.072 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.049458_246.1", 2022-09-03T05:46:58.072 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.072 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28", 2022-09-03T05:46:58.072 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.049458314Z", 2022-09-03T05:46:58.073 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.073 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.073 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:58.073 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.074 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.074 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.074 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:58.075 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.075 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.075 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.075 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.076 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.076 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.076 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.077 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.077 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.052066_247.1", 2022-09-03T05:46:58.077 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.077 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28", 2022-09-03T05:46:58.078 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.052066976Z", 2022-09-03T05:46:58.078 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.078 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.078 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:58.079 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.079 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.079 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.079 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:58.080 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.080 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.080 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.081 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.081 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.081 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.081 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.082 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.082 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.091593_248.1", 2022-09-03T05:46:58.082 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.082 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-29:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.29", 2022-09-03T05:46:58.083 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.091593357Z", 2022-09-03T05:46:58.083 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.083 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.084 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:58.084 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.084 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.084 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.085 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:58.085 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.085 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.085 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.086 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.086 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.086 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.087 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.087 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.087 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.099123_249.1", 2022-09-03T05:46:58.087 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.088 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-29:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.29", 2022-09-03T05:46:58.088 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.099123834Z", 2022-09-03T05:46:58.088 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.089 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.089 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:58.089 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.089 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.090 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.090 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:58.090 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.090 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.091 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.091 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.091 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.091 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.092 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.092 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.092 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.135078_250.1", 2022-09-03T05:46:58.093 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.093 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30", 2022-09-03T05:46:58.093 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.135078742Z", 2022-09-03T05:46:58.093 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.094 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.094 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.095 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.095 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.095 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.095 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:58.096 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.096 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.096 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.097 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.097 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.097 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.097 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.098 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.098 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.140570_251.1", 2022-09-03T05:46:58.099 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.099 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30", 2022-09-03T05:46:58.099 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.140570450Z", 2022-09-03T05:46:58.099 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.100 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.100 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.100 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.101 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.101 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.101 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:58.101 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.102 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.102 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.102 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.102 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.103 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.103 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.103 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.103 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.142697_252.1", 2022-09-03T05:46:58.104 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.104 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-30", 2022-09-03T05:46:58.104 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.142697729Z", 2022-09-03T05:46:58.105 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.105 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.105 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.106 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.106 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.106 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.106 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_4yhNaX-8bC2cTXp5LKWw-Qo", 2022-09-03T05:46:58.107 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.107 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.107 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.107 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.108 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.108 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.109 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.109 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.109 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.148860_253.1", 2022-09-03T05:46:58.109 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.110 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-30", 2022-09-03T05:46:58.110 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.148860761Z", 2022-09-03T05:46:58.110 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.111 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.111 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.111 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.111 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.112 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.112 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_4yhNaX-8bC2cTXp5LKWw-Qo", 2022-09-03T05:46:58.112 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.112 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.113 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.113 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.113 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.113 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.114 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.114 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.114 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.172332_254.1", 2022-09-03T05:46:58.114 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.115 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31", 2022-09-03T05:46:58.115 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.172332646Z", 2022-09-03T05:46:58.115 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.116 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.116 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.116 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.116 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.117 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.117 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:58.117 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.118 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.118 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.118 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.119 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.119 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.119 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.119 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.120 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.176046_255.1", 2022-09-03T05:46:58.120 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.120 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31", 2022-09-03T05:46:58.120 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.176046838Z", 2022-09-03T05:46:58.121 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.121 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.121 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.122 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.122 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.122 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.122 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:58.123 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.123 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.123 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.123 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.124 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.124 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.124 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.125 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.125 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.176958_256.1", 2022-09-03T05:46:58.125 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.125 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-31", 2022-09-03T05:46:58.126 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.176958510Z", 2022-09-03T05:46:58.126 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.126 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.126 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.127 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.127 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.127 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.128 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_oPtbK62ZjFZvpDTUcbzdHLT", 2022-09-03T05:46:58.128 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.128 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.128 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.129 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.129 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.129 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.129 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.130 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.130 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.179165_257.1", 2022-09-03T05:46:58.130 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.131 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-31", 2022-09-03T05:46:58.131 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.179165643Z", 2022-09-03T05:46:58.131 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.131 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.132 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.132 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.132 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.132 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.133 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_oPtbK62ZjFZvpDTUcbzdHLT", 2022-09-03T05:46:58.133 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.133 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.133 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.134 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.134 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.134 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.135 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.135 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.135 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.220311_258.1", 2022-09-03T05:46:58.135 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.136 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-32:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.32", 2022-09-03T05:46:58.136 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.220311791Z", 2022-09-03T05:46:58.136 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.136 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.137 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.137 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.137 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.137 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.138 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:58.138 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.138 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.139 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.139 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.139 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.139 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.140 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.140 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.140 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.222654_259.1", 2022-09-03T05:46:58.140 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.141 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-32:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.32", 2022-09-03T05:46:58.141 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.222654278Z", 2022-09-03T05:46:58.141 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.141 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.142 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.142 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.142 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.143 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.143 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:58.143 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.143 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.144 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.144 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.144 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.144 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.145 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.145 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.145 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.223513_260.1", 2022-09-03T05:46:58.145 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.146 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-32", 2022-09-03T05:46:58.146 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.223513282Z", 2022-09-03T05:46:58.146 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.147 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.147 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.147 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.147 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.148 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.148 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_y0oxyMaYTjSUF-S-HkK29E5", 2022-09-03T05:46:58.148 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.148 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.149 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.149 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.149 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.149 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.150 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.150 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.150 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.228241_261.1", 2022-09-03T05:46:58.150 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.151 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-32", 2022-09-03T05:46:58.151 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.228241236Z", 2022-09-03T05:46:58.151 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.152 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.152 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.152 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.152 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.153 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.153 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_y0oxyMaYTjSUF-S-HkK29E5", 2022-09-03T05:46:58.153 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.153 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.154 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.154 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.154 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.154 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.155 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.155 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.155 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.265477_262.1", 2022-09-03T05:46:58.155 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.156 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30", 2022-09-03T05:46:58.156 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.265477055Z", 2022-09-03T05:46:58.156 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.157 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.157 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:58.157 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.157 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.158 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.158 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:58.158 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.158 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.159 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.159 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.159 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.159 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.160 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.160 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.160 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.268146_263.1", 2022-09-03T05:46:58.161 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.161 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30", 2022-09-03T05:46:58.161 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.268146553Z", 2022-09-03T05:46:58.161 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.162 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.162 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:58.162 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.162 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.163 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.163 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:58.163 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.163 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.164 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.164 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.164 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.164 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.165 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.165 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.165 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.350018_264.1", 2022-09-03T05:46:58.166 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.166 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31", 2022-09-03T05:46:58.166 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.350018238Z", 2022-09-03T05:46:58.166 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.167 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.167 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:58.167 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.167 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.168 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.168 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:58.168 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.168 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.169 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.169 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.169 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.170 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.170 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.170 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.170 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.352665_265.1", 2022-09-03T05:46:58.171 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.171 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31", 2022-09-03T05:46:58.171 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.352665529Z", 2022-09-03T05:46:58.172 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.172 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.172 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:58.172 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.173 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.173 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.173 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:58.173 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.174 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.174 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.174 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.175 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.175 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.175 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.175 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.176 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.401743_266.1", 2022-09-03T05:46:58.176 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.176 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-32:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.32", 2022-09-03T05:46:58.176 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.401743084Z", 2022-09-03T05:46:58.177 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.177 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.177 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:58.177 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.178 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.178 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.178 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:58.178 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.179 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.179 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.179 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.180 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.180 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.180 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.180 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.181 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.404375_267.1", 2022-09-03T05:46:58.181 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.181 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-32:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.32", 2022-09-03T05:46:58.182 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.404375474Z", 2022-09-03T05:46:58.182 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.182 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.182 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:58.183 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.183 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.183 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.183 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:58.184 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.184 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.184 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.185 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.185 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.185 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.186 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.186 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.186 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.434979_268.1", 2022-09-03T05:46:58.186 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.187 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-33:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.33", 2022-09-03T05:46:58.187 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.434979477Z", 2022-09-03T05:46:58.187 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.187 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.188 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.188 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.188 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.189 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.189 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:58.189 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.189 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.190 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.190 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.190 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.190 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.191 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.191 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.191 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.438289_269.1", 2022-09-03T05:46:58.192 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.192 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-33:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.33", 2022-09-03T05:46:58.192 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.438289839Z", 2022-09-03T05:46:58.192 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.193 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.193 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.193 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.193 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.194 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.194 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:58.194 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.195 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.195 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.195 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.195 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.196 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.196 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.196 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.197 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.439390_270.1", 2022-09-03T05:46:58.197 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.197 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-33", 2022-09-03T05:46:58.197 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.439390666Z", 2022-09-03T05:46:58.198 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.198 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.198 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.199 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.199 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.199 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.199 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_XHlZf1wWm_Tr8FwCOGzkHwG", 2022-09-03T05:46:58.200 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.200 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.200 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.200 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.201 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.201 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.201 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.201 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.202 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.441665_271.1", 2022-09-03T05:46:58.202 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.202 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-33", 2022-09-03T05:46:58.203 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.441665178Z", 2022-09-03T05:46:58.203 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.203 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.203 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.204 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.204 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.204 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.205 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_XHlZf1wWm_Tr8FwCOGzkHwG", 2022-09-03T05:46:58.205 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.205 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.206 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.206 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.206 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.206 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.207 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.207 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.207 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.461635_272.1", 2022-09-03T05:46:58.207 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.208 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34", 2022-09-03T05:46:58.208 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.461635262Z", 2022-09-03T05:46:58.208 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.209 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.209 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.209 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.209 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.210 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.210 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:58.210 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.210 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.211 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.211 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.211 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.211 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.212 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.212 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.212 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.466010_273.1", 2022-09-03T05:46:58.213 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.213 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34", 2022-09-03T05:46:58.213 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.466010065Z", 2022-09-03T05:46:58.213 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.214 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.214 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.214 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.214 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.215 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.215 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:58.215 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.215 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.216 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.216 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.216 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.216 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.217 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.217 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.217 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.468430_274.1", 2022-09-03T05:46:58.218 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.218 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-34", 2022-09-03T05:46:58.218 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.468430282Z", 2022-09-03T05:46:58.218 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.219 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.219 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.219 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.219 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.220 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.220 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_8JLNrQKjOCrzmEAJH1Gqebc", 2022-09-03T05:46:58.220 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.221 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.221 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.221 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.221 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.222 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.222 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.222 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.222 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.473656_275.1", 2022-09-03T05:46:58.223 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.223 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-34", 2022-09-03T05:46:58.223 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.473656625Z", 2022-09-03T05:46:58.223 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.224 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.224 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.224 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.225 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.225 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.225 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_8JLNrQKjOCrzmEAJH1Gqebc", 2022-09-03T05:46:58.225 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.226 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.226 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.226 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.226 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.227 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.227 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.227 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.227 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.519749_276.1", 2022-09-03T05:46:58.228 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.228 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35", 2022-09-03T05:46:58.228 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.519749696Z", 2022-09-03T05:46:58.229 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.229 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.229 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.229 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.230 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.230 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.230 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:58.230 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.231 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.231 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.231 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.231 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.232 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.232 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.232 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.233 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.527797_277.1", 2022-09-03T05:46:58.233 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.233 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35", 2022-09-03T05:46:58.233 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.527797485Z", 2022-09-03T05:46:58.234 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.234 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.234 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.234 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.235 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.235 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.235 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:58.236 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.236 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.236 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.236 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.237 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.237 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.237 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.237 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.238 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.530086_278.1", 2022-09-03T05:46:58.238 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.238 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-35", 2022-09-03T05:46:58.239 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.530086284Z", 2022-09-03T05:46:58.239 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.239 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.239 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.240 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.240 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.240 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.241 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_9eV0dD_Np6i5r-IXItyYqq8", 2022-09-03T05:46:58.241 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.241 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.241 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.242 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.242 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.242 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.243 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.243 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.243 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.542414_279.1", 2022-09-03T05:46:58.243 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.244 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-35", 2022-09-03T05:46:58.244 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.542414804Z", 2022-09-03T05:46:58.244 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.244 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.245 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.245 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.245 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.246 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.246 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_9eV0dD_Np6i5r-IXItyYqq8", 2022-09-03T05:46:58.246 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.247 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.247 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.247 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.247 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.248 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.248 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.248 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.248 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.574433_280.1", 2022-09-03T05:46:58.249 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.249 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-33:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.33", 2022-09-03T05:46:58.249 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.574433191Z", 2022-09-03T05:46:58.250 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.250 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.250 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:58.250 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.251 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.251 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.251 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:58.252 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.252 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.252 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.252 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.253 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.253 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.253 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.253 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.254 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183850.577143_281.1", 2022-09-03T05:46:58.254 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.254 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-33:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.33", 2022-09-03T05:46:58.255 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:10.577143931Z", 2022-09-03T05:46:58.255 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.255 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.255 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:58.256 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.256 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.256 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.257 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:58.257 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.257 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.257 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.258 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.258 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.258 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.258 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.259 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.259 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183851.605820_282.1", 2022-09-03T05:46:58.259 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.260 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34", 2022-09-03T05:46:58.260 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:11.605820553Z", 2022-09-03T05:46:58.260 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.260 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.261 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:58.261 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.261 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.261 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.262 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:58.262 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.262 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.263 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.263 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.263 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.263 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.264 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.264 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.264 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183851.610169_283.1", 2022-09-03T05:46:58.265 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.265 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34", 2022-09-03T05:46:58.265 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:11.610169538Z", 2022-09-03T05:46:58.265 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.266 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.266 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:58.266 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.266 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.267 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.267 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:58.267 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.268 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.268 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.268 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.268 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.269 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.269 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.269 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.269 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.681884_284.1", 2022-09-03T05:46:58.270 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.270 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35", 2022-09-03T05:46:58.270 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.681884846Z", 2022-09-03T05:46:58.271 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.271 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.271 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:58.271 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.272 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.272 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.272 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:58.272 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.273 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.273 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.273 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.273 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.274 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.274 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.274 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.275 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.684548_285.1", 2022-09-03T05:46:58.275 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.275 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35", 2022-09-03T05:46:58.275 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.684548086Z", 2022-09-03T05:46:58.276 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.276 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.276 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:58.276 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.277 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.277 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.277 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:58.277 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.278 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.278 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.278 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.279 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.279 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.279 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.280 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.280 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.769697_286.1", 2022-09-03T05:46:58.280 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.280 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36", 2022-09-03T05:46:58.281 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.769697830Z", 2022-09-03T05:46:58.281 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.281 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.281 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.282 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.282 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.282 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.282 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lzXJaycHyP7aPeCDF57oxvc", 2022-09-03T05:46:58.283 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.283 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.283 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.284 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.284 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.284 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.284 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.285 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.285 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.780292_287.1", 2022-09-03T05:46:58.285 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.285 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36", 2022-09-03T05:46:58.286 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.780292446Z", 2022-09-03T05:46:58.286 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.286 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.286 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.287 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.287 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.287 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.288 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lzXJaycHyP7aPeCDF57oxvc", 2022-09-03T05:46:58.288 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.288 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.288 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.289 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.289 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.289 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.289 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.290 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.290 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.781473_288.1", 2022-09-03T05:46:58.290 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.291 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-36", 2022-09-03T05:46:58.291 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.781473404Z", 2022-09-03T05:46:58.291 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.291 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.292 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.292 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.292 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.292 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.293 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Aunq5-l6wG1IGbEHKmUcfjj", 2022-09-03T05:46:58.293 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.293 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.293 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.294 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.294 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.295 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.295 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.295 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.295 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.785261_289.1", 2022-09-03T05:46:58.296 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.296 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-36", 2022-09-03T05:46:58.296 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.785261123Z", 2022-09-03T05:46:58.297 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.297 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.297 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.297 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.298 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.298 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.298 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Aunq5-l6wG1IGbEHKmUcfjj", 2022-09-03T05:46:58.299 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.299 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.299 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.299 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.300 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.300 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.300 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.301 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.301 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.806309_290.1", 2022-09-03T05:46:58.301 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.302 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37", 2022-09-03T05:46:58.302 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.806309544Z", 2022-09-03T05:46:58.303 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.303 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.303 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.303 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.304 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.304 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.304 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_kuQ0xr50nHBW7YTCPQavDGz", 2022-09-03T05:46:58.306 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.306 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.307 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.307 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.307 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.308 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.308 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.308 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.308 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.808817_291.1", 2022-09-03T05:46:58.309 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.309 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37", 2022-09-03T05:46:58.309 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.808817564Z", 2022-09-03T05:46:58.309 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.310 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.310 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.311 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.311 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.311 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.312 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_kuQ0xr50nHBW7YTCPQavDGz", 2022-09-03T05:46:58.312 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.313 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.314 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.315 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.315 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.316 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.316 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.317 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.317 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.809764_292.1", 2022-09-03T05:46:58.317 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.318 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-37", 2022-09-03T05:46:58.318 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.809764987Z", 2022-09-03T05:46:58.319 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.319 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.320 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.320 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.320 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.321 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.321 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_2HiJKFit31XaN5gBsoarIgZ", 2022-09-03T05:46:58.321 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.322 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.322 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.322 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.322 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.323 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.323 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.323 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.324 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.814054_293.1", 2022-09-03T05:46:58.324 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.324 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-37", 2022-09-03T05:46:58.324 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.814054902Z", 2022-09-03T05:46:58.325 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.325 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.326 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.326 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.327 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.327 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.328 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_2HiJKFit31XaN5gBsoarIgZ", 2022-09-03T05:46:58.328 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.329 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.329 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.330 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.330 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.331 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.332 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.332 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.333 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.846705_294.1", 2022-09-03T05:46:58.333 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.334 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38", 2022-09-03T05:46:58.334 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.846705503Z", 2022-09-03T05:46:58.334 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.335 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.335 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.335 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.336 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.336 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.337 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_h95HWYhtSYo3Z6W42rIP0YJ", 2022-09-03T05:46:58.337 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.338 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.339 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.339 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.340 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.340 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.341 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.341 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.342 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.850680_295.1", 2022-09-03T05:46:58.342 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.343 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38", 2022-09-03T05:46:58.343 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.850680201Z", 2022-09-03T05:46:58.343 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.343 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.344 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.344 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.344 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.345 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.345 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_h95HWYhtSYo3Z6W42rIP0YJ", 2022-09-03T05:46:58.345 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.346 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.346 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.347 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.347 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.348 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.348 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.348 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.349 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.851766_296.1", 2022-09-03T05:46:58.349 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.349 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-38", 2022-09-03T05:46:58.350 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.851766916Z", 2022-09-03T05:46:58.350 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.350 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.350 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.351 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.351 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.351 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.352 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_K5Yz9Djeac7y4a5O_0TviMm", 2022-09-03T05:46:58.352 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.352 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.353 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.353 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.353 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.353 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.354 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.354 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.354 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183852.856692_297.1", 2022-09-03T05:46:58.354 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.355 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-38", 2022-09-03T05:46:58.355 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:12.856692289Z", 2022-09-03T05:46:58.355 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.356 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.356 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.357 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.357 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.358 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.358 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_K5Yz9Djeac7y4a5O_0TviMm", 2022-09-03T05:46:58.359 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.359 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.360 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.360 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.360 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.360 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.361 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.361 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.361 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.003533_298.1", 2022-09-03T05:46:58.362 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.362 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-39:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.39", 2022-09-03T05:46:58.362 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.003533677Z", 2022-09-03T05:46:58.362 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.363 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.363 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.363 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.363 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.364 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.364 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_LjHtI20g0C-nIIB1xqrV4kd", 2022-09-03T05:46:58.364 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.364 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.365 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.365 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.365 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.365 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.366 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.366 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.367 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.006401_299.1", 2022-09-03T05:46:58.367 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.368 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-39:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.39", 2022-09-03T05:46:58.368 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.006401221Z", 2022-09-03T05:46:58.369 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.369 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.369 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.370 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.370 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.371 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.371 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_LjHtI20g0C-nIIB1xqrV4kd", 2022-09-03T05:46:58.372 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.372 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.373 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.373 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.373 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.374 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.374 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.374 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.375 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.007446_300.1", 2022-09-03T05:46:58.375 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.375 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-39", 2022-09-03T05:46:58.376 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.007446652Z", 2022-09-03T05:46:58.376 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.376 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.377 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.377 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.378 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.378 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.378 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_LR1OrAXSVx6okJ40JV0Uh8F", 2022-09-03T05:46:58.379 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.380 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.380 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.381 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.381 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.382 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.382 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.382 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.382 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.010018_301.1", 2022-09-03T05:46:58.383 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.383 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-39", 2022-09-03T05:46:58.383 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.010018357Z", 2022-09-03T05:46:58.383 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.384 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.384 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.384 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.384 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.385 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.385 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_LR1OrAXSVx6okJ40JV0Uh8F", 2022-09-03T05:46:58.385 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.385 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.386 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.386 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.386 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.387 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.387 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.387 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.387 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.039601_302.1", 2022-09-03T05:46:58.388 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.388 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-40:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.40", 2022-09-03T05:46:58.388 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.039601239Z", 2022-09-03T05:46:58.388 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.389 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.389 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.389 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.389 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.390 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.390 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_SEgVmZRRIdgycWQGmgKhmpv", 2022-09-03T05:46:58.390 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.391 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.391 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.391 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.391 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.392 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.392 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.392 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.392 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.048071_303.1", 2022-09-03T05:46:58.393 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.393 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-40:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.40", 2022-09-03T05:46:58.393 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.048071799Z", 2022-09-03T05:46:58.393 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.394 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.394 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.394 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.395 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.395 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.395 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_SEgVmZRRIdgycWQGmgKhmpv", 2022-09-03T05:46:58.396 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.396 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.396 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.396 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.397 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.397 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.397 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.398 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.398 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.049107_304.1", 2022-09-03T05:46:58.398 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.398 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-40", 2022-09-03T05:46:58.399 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.049107658Z", 2022-09-03T05:46:58.399 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.400 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.400 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.401 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.401 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.401 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.402 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_e0zQngowI1boXIeM6KNEWZw", 2022-09-03T05:46:58.402 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.402 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.402 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.403 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.403 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.403 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.404 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.404 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.404 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.054911_305.1", 2022-09-03T05:46:58.404 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.405 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-40", 2022-09-03T05:46:58.405 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.054911391Z", 2022-09-03T05:46:58.405 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.406 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.406 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.406 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.407 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.407 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.407 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_e0zQngowI1boXIeM6KNEWZw", 2022-09-03T05:46:58.407 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.408 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.408 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.408 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.408 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.409 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.409 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.409 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.409 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.099806_306.1", 2022-09-03T05:46:58.410 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.410 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-41:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.41", 2022-09-03T05:46:58.410 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.099806646Z", 2022-09-03T05:46:58.411 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.411 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.411 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.411 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.412 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.412 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.412 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_M2clmHIwHAIIB9oEOnpIKkh", 2022-09-03T05:46:58.412 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.413 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.413 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.413 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.413 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.414 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.414 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.414 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.414 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.111286_307.1", 2022-09-03T05:46:58.415 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.415 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-41:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.41", 2022-09-03T05:46:58.415 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.111286094Z", 2022-09-03T05:46:58.415 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.416 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.416 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.416 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.417 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.417 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.417 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_M2clmHIwHAIIB9oEOnpIKkh", 2022-09-03T05:46:58.417 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.418 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.418 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.418 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.419 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.419 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.419 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.419 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.420 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.113514_308.1", 2022-09-03T05:46:58.420 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.420 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-41", 2022-09-03T05:46:58.420 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.113514552Z", 2022-09-03T05:46:58.421 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.421 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.421 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.421 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.422 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.422 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.422 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_kKVUMELn0ub6gAm6SEjIkvZ", 2022-09-03T05:46:58.422 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.423 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.423 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.423 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.423 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.424 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.424 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.424 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.425 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183864.119538_309.1", 2022-09-03T05:46:58.425 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.425 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-41", 2022-09-03T05:46:58.426 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:24.119538060Z", 2022-09-03T05:46:58.426 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.426 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.426 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.427 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.427 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.427 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.427 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_kKVUMELn0ub6gAm6SEjIkvZ", 2022-09-03T05:46:58.428 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.428 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.428 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.429 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.429 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.429 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.429 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.430 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.430 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182414.943600_19.1", 2022-09-03T05:46:58.430 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.430 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-1:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.1", 2022-09-03T05:46:58.431 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:14.943600569Z", 2022-09-03T05:46:58.431 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.431 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.432 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.432 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.432 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.432 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.433 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_rq1mS4spn__TV-QX0MQQJUU", 2022-09-03T05:46:58.433 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.433 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.434 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.434 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.434 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.434 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.435 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.435 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.435 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182414.959460_20.1", 2022-09-03T05:46:58.435 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.436 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-1:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.1", 2022-09-03T05:46:58.436 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:14.959460360Z", 2022-09-03T05:46:58.436 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.437 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.437 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.437 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.437 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.438 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.438 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_rq1mS4spn__TV-QX0MQQJUU", 2022-09-03T05:46:58.438 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.438 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.439 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.439 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.439 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.440 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.440 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.440 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.440 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182414.961669_21.1", 2022-09-03T05:46:58.441 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.441 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-1", 2022-09-03T05:46:58.441 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:14.961669762Z", 2022-09-03T05:46:58.442 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.442 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.442 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.442 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.443 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.443 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.443 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_0bHoPHCFbjz75fM89k7kJ9J", 2022-09-03T05:46:58.443 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.444 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.444 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.444 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.445 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.445 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.445 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.445 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.446 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182414.972666_22.1", 2022-09-03T05:46:58.446 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.447 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-1", 2022-09-03T05:46:58.447 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:14.972666586Z", 2022-09-03T05:46:58.447 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.447 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.448 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.448 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.448 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.448 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.449 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_0bHoPHCFbjz75fM89k7kJ9J", 2022-09-03T05:46:58.449 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.449 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.450 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.450 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.450 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.450 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.451 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.451 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.451 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182415.003071_23.1", 2022-09-03T05:46:58.452 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.452 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-2:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.2", 2022-09-03T05:46:58.452 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:15.003071656Z", 2022-09-03T05:46:58.452 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.453 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.453 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.453 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.453 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.454 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.454 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tfkUve-GLYp4UeLNm-4NFHF", 2022-09-03T05:46:58.454 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.455 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.455 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.455 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.455 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.456 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.456 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.456 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.457 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182415.009391_24.1", 2022-09-03T05:46:58.457 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.457 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-2:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.2", 2022-09-03T05:46:58.458 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:15.009391132Z", 2022-09-03T05:46:58.458 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.458 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.458 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.459 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.459 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.459 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.460 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tfkUve-GLYp4UeLNm-4NFHF", 2022-09-03T05:46:58.460 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.460 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.460 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.461 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.461 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.461 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.461 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.462 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.462 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182415.013579_25.1", 2022-09-03T05:46:58.462 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.462 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-2", 2022-09-03T05:46:58.463 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:15.013579974Z", 2022-09-03T05:46:58.463 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.463 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.463 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.464 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.464 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.464 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.464 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_ach1wph5n7_8QKTUZF1Mary", 2022-09-03T05:46:58.465 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.465 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.465 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.466 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.466 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.466 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.466 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.467 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.467 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182415.025150_26.1", 2022-09-03T05:46:58.467 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.467 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-2", 2022-09-03T05:46:58.468 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:15.025150228Z", 2022-09-03T05:46:58.468 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.468 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.468 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.469 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.469 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.469 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.469 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_ach1wph5n7_8QKTUZF1Mary", 2022-09-03T05:46:58.470 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.470 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.470 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.470 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.471 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.471 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.471 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.471 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.472 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182417.307770_27.1", 2022-09-03T05:46:58.472 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.472 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-3:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.3", 2022-09-03T05:46:58.472 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:17.307770555Z", 2022-09-03T05:46:58.473 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.473 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.473 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.474 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.474 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.474 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.474 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_RKS6tzrI_dmvQpvUbloi7Q_", 2022-09-03T05:46:58.475 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.475 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.475 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.476 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.476 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.476 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.477 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.477 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.477 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182417.312081_28.1", 2022-09-03T05:46:58.477 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.478 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-3:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.3", 2022-09-03T05:46:58.478 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:17.312081255Z", 2022-09-03T05:46:58.478 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.479 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.479 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.479 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.479 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.480 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.480 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_RKS6tzrI_dmvQpvUbloi7Q_", 2022-09-03T05:46:58.480 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.480 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.481 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.481 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.481 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.482 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.482 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.482 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.482 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182417.313242_29.1", 2022-09-03T05:46:58.483 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.483 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-3", 2022-09-03T05:46:58.483 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:17.313242294Z", 2022-09-03T05:46:58.483 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.484 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.484 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.484 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.485 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.485 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.485 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lt8IAlx8xFKZXCBoGrP76MP", 2022-09-03T05:46:58.485 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.486 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.486 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.486 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.487 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.487 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.487 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.487 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.488 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182417.319609_30.1", 2022-09-03T05:46:58.488 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.488 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-3", 2022-09-03T05:46:58.489 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:17.319609012Z", 2022-09-03T05:46:58.489 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.489 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.489 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.490 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.490 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.490 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.491 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_lt8IAlx8xFKZXCBoGrP76MP", 2022-09-03T05:46:58.491 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.491 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.492 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.492 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.492 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.492 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.493 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.493 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.493 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.152440_31.1", 2022-09-03T05:46:58.493 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.494 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-4:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.4", 2022-09-03T05:46:58.494 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.152440893Z", 2022-09-03T05:46:58.494 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.495 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.495 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.495 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.495 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.496 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.496 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_QMyoKwlK2G2NuxL_sp2_re5", 2022-09-03T05:46:58.496 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.496 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.497 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.497 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.497 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.497 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.498 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.498 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.498 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.158442_32.1", 2022-09-03T05:46:58.498 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.499 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-4:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.4", 2022-09-03T05:46:58.499 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.158442960Z", 2022-09-03T05:46:58.499 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.499 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.500 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.500 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.500 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.500 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.501 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_QMyoKwlK2G2NuxL_sp2_re5", 2022-09-03T05:46:58.501 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.501 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.501 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.502 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.502 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.502 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.503 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.503 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.503 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.159309_33.1", 2022-09-03T05:46:58.503 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.504 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-4", 2022-09-03T05:46:58.504 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.159309677Z", 2022-09-03T05:46:58.504 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.504 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.505 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.505 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.506 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.506 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.506 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_FxyCJRWlBEhmncaMF3-AMFD", 2022-09-03T05:46:58.506 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.507 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.507 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.507 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.507 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.508 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.508 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.508 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.509 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.163606_34.1", 2022-09-03T05:46:58.509 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.509 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-4", 2022-09-03T05:46:58.509 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.163606904Z", 2022-09-03T05:46:58.510 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.510 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.510 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.510 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.511 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.511 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.511 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_FxyCJRWlBEhmncaMF3-AMFD", 2022-09-03T05:46:58.511 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.512 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.512 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.512 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.512 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.513 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.513 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.513 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.514 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.181237_35.1", 2022-09-03T05:46:58.514 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.514 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-5:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.5", 2022-09-03T05:46:58.514 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.181237915Z", 2022-09-03T05:46:58.515 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.515 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.515 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.516 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.516 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.516 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.516 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_mye4oDj8XK7uSi7YUS75P0l", 2022-09-03T05:46:58.517 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.517 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.517 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.518 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.518 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.518 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.518 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.519 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.519 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.183507_36.1", 2022-09-03T05:46:58.519 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.519 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-5:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.5", 2022-09-03T05:46:58.520 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.183507081Z", 2022-09-03T05:46:58.520 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.520 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.521 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.521 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.521 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.521 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.522 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_mye4oDj8XK7uSi7YUS75P0l", 2022-09-03T05:46:58.522 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.522 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.522 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.523 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.523 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.523 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.523 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.524 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.524 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.184425_37.1", 2022-09-03T05:46:58.524 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.525 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-5", 2022-09-03T05:46:58.525 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.184425827Z", 2022-09-03T05:46:58.525 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.525 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.526 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.526 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.526 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.527 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.527 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_6QXcvNroiwRkEdVqi3Nf85-", 2022-09-03T05:46:58.527 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.527 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.528 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.528 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.528 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.528 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.529 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.529 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.529 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.186822_38.1", 2022-09-03T05:46:58.530 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.530 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-5", 2022-09-03T05:46:58.530 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.186822428Z", 2022-09-03T05:46:58.530 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.531 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.531 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.531 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.532 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.532 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.532 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_6QXcvNroiwRkEdVqi3Nf85-", 2022-09-03T05:46:58.532 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.533 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.533 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.533 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.533 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.534 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.534 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.534 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.535 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.221675_39.1", 2022-09-03T05:46:58.535 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.535 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-6:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.6", 2022-09-03T05:46:58.535 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.221675326Z", 2022-09-03T05:46:58.536 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.536 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.536 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.536 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.537 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.537 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.537 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bqgF7AreaM1kO2cBL_SkGsM", 2022-09-03T05:46:58.538 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.538 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.538 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.539 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.539 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.539 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.540 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.540 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.540 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.230423_40.1", 2022-09-03T05:46:58.540 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.541 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-6:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.6", 2022-09-03T05:46:58.541 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.230423417Z", 2022-09-03T05:46:58.541 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.542 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.542 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.542 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.542 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.543 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.543 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bqgF7AreaM1kO2cBL_SkGsM", 2022-09-03T05:46:58.543 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.543 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.544 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.544 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.544 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.545 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.545 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.545 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.545 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.235081_41.1", 2022-09-03T05:46:58.546 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.546 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-6", 2022-09-03T05:46:58.546 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.235081816Z", 2022-09-03T05:46:58.546 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.547 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.547 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.547 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.548 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.548 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.548 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_wbkk3MMRONgcegk23g-QDT1", 2022-09-03T05:46:58.549 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.549 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.549 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.550 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.550 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.550 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.551 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.551 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.551 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182423.238098_42.1", 2022-09-03T05:46:58.551 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.552 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-6", 2022-09-03T05:46:58.552 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:23.238098524Z", 2022-09-03T05:46:58.552 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.553 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.553 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.553 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.553 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.554 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.554 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_wbkk3MMRONgcegk23g-QDT1", 2022-09-03T05:46:58.554 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.555 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.555 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.555 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.555 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.556 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.556 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.556 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.556 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.064697_43.1", 2022-09-03T05:46:58.557 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.557 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7", 2022-09-03T05:46:58.557 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.064697048Z", 2022-09-03T05:46:58.558 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.558 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.558 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.559 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.559 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.559 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.559 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:58.560 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.560 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.560 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.569 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.570 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.570 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.570 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.571 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.571 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.070346_44.1", 2022-09-03T05:46:58.571 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.571 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7", 2022-09-03T05:46:58.572 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.070346892Z", 2022-09-03T05:46:58.572 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.572 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.572 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.573 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.573 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.573 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.574 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:58.574 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.574 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.574 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.575 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.575 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.575 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.576 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.576 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.576 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.071355_45.1", 2022-09-03T05:46:58.576 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.577 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-7", 2022-09-03T05:46:58.577 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.071355932Z", 2022-09-03T05:46:58.577 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.577 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.578 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.578 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.578 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.579 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.579 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:46:58.579 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.579 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.580 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.580 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.580 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.580 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.581 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.581 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.581 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.075532_46.1", 2022-09-03T05:46:58.581 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.582 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-7", 2022-09-03T05:46:58.582 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.075532031Z", 2022-09-03T05:46:58.582 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.582 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.583 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.583 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.583 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.584 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.584 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:46:58.584 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.584 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.585 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.585 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.585 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.585 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.586 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.586 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.586 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.102267_47.1", 2022-09-03T05:46:58.586 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.587 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-8:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.8", 2022-09-03T05:46:58.587 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.102267896Z", 2022-09-03T05:46:58.587 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.587 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.588 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.588 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.588 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.588 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.589 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:46:58.589 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.589 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.589 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.590 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.590 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.590 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.591 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.591 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.591 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.104611_48.1", 2022-09-03T05:46:58.592 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.592 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-8:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.8", 2022-09-03T05:46:58.592 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.104611563Z", 2022-09-03T05:46:58.592 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.593 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.593 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.593 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.593 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.594 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.594 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:46:58.594 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.594 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.595 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.595 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.595 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.595 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.596 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.596 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.596 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.105641_49.1", 2022-09-03T05:46:58.596 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.597 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-8", 2022-09-03T05:46:58.597 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.105641481Z", 2022-09-03T05:46:58.597 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.598 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.598 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.598 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.598 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.599 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.599 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:46:58.599 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.599 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.600 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.600 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.600 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.600 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.601 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.601 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.601 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.108076_50.1", 2022-09-03T05:46:58.601 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.602 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-8", 2022-09-03T05:46:58.602 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.108076436Z", 2022-09-03T05:46:58.602 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.602 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.603 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.603 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.603 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.603 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.604 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:46:58.604 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.604 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.604 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.605 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.605 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.605 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.605 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.606 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.606 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.135979_51.1", 2022-09-03T05:46:58.606 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.606 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9", 2022-09-03T05:46:58.607 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.135979206Z", 2022-09-03T05:46:58.607 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.607 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.607 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.608 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.608 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.608 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.608 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:46:58.609 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.609 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.609 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.609 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.610 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.610 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.610 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.611 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.611 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.138975_52.1", 2022-09-03T05:46:58.611 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.611 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9", 2022-09-03T05:46:58.612 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.138975472Z", 2022-09-03T05:46:58.612 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.612 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.612 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.613 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.613 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.613 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.613 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:46:58.614 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.614 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.614 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.615 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.615 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.615 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.615 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.616 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.616 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.139759_53.1", 2022-09-03T05:46:58.616 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.616 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-9", 2022-09-03T05:46:58.617 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.139759635Z", 2022-09-03T05:46:58.617 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.617 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.617 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.618 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.618 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.618 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.619 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:46:58.619 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.619 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.619 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.620 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.620 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.620 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.620 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.621 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.621 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.141916_54.1", 2022-09-03T05:46:58.621 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.621 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-9", 2022-09-03T05:46:58.622 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.141916666Z", 2022-09-03T05:46:58.622 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.622 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.623 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:46:58.623 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:46:58.623 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.623 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.624 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:46:58.624 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.624 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.624 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.625 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.625 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.625 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.625 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.626 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.626 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.782802_55.1", 2022-09-03T05:46:58.626 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.627 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7", 2022-09-03T05:46:58.627 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.782802362Z", 2022-09-03T05:46:58.627 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.627 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.628 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:58.628 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.628 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.628 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.629 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:58.629 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.629 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.629 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.630 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.630 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.630 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.630 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.631 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.631 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.785254_56.1", 2022-09-03T05:46:58.631 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.631 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7", 2022-09-03T05:46:58.632 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.785254883Z", 2022-09-03T05:46:58.632 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.632 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.632 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:58.633 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.633 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.633 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.634 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:58.634 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.634 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.634 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.635 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.635 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.635 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.636 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.636 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.636 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.796744_57.1", 2022-09-03T05:46:58.636 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.637 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-7", 2022-09-03T05:46:58.637 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.796744879Z", 2022-09-03T05:46:58.637 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.637 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.638 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:46:58.638 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.638 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.638 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.639 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:46:58.639 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.639 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.639 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.640 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "remove" 2022-09-03T05:46:58.640 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.640 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.640 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.641 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.641 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.798860_58.1", 2022-09-03T05:46:58.641 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.641 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-7", 2022-09-03T05:46:58.642 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.798860427Z", 2022-09-03T05:46:58.642 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.642 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.643 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:46:58.643 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.643 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.643 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.644 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:46:58.644 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.644 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.644 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.645 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.645 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.645 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.645 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.646 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.646 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.833037_59.1", 2022-09-03T05:46:58.646 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.646 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-8:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.8", 2022-09-03T05:46:58.647 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.833037165Z", 2022-09-03T05:46:58.647 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.647 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.647 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:46:58.648 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.648 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.648 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.649 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:46:58.649 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.649 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.649 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.650 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.650 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.650 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.650 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.651 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.651 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.835414_60.1", 2022-09-03T05:46:58.651 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.651 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-8:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.8", 2022-09-03T05:46:58.652 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.835414206Z", 2022-09-03T05:46:58.652 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.652 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.653 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:46:58.653 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.653 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.653 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.654 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:46:58.654 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.654 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.654 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.655 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.655 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.655 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.655 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.656 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.656 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.843078_61.1", 2022-09-03T05:46:58.656 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.656 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-8", 2022-09-03T05:46:58.657 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.843078156Z", 2022-09-03T05:46:58.657 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.657 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.657 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:46:58.658 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.658 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.658 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.659 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:46:58.659 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.659 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.659 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.660 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "remove" 2022-09-03T05:46:58.660 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.660 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.661 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.661 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.661 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.845207_62.1", 2022-09-03T05:46:58.661 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.662 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-8", 2022-09-03T05:46:58.662 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.845207527Z", 2022-09-03T05:46:58.662 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.663 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.663 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:46:58.663 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.664 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.664 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.664 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:46:58.665 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.665 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.666 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.666 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.666 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.667 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.667 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.667 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.668 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.909335_63.1", 2022-09-03T05:46:58.668 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.668 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9", 2022-09-03T05:46:58.669 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.909335713Z", 2022-09-03T05:46:58.669 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.670 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.670 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:46:58.671 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.671 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.672 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.672 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:46:58.672 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.673 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.673 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.674 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:46:58.674 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.675 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.675 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.676 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.676 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.911665_64.1", 2022-09-03T05:46:58.677 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.677 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9", 2022-09-03T05:46:58.678 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.911665732Z", 2022-09-03T05:46:58.678 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.679 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.679 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:46:58.680 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.680 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.681 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.681 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:46:58.681 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.684 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.685 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.685 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.686 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.686 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.687 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.687 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.688 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.924885_65.1", 2022-09-03T05:46:58.689 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.689 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-9", 2022-09-03T05:46:58.690 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.924885371Z", 2022-09-03T05:46:58.690 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.691 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.691 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:46:58.692 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.692 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.693 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.693 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:46:58.693 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.694 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.694 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.694 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "remove" 2022-09-03T05:46:58.695 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.697 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.697 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.698 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:46:58.698 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662182430.928498_66.1", 2022-09-03T05:46:58.699 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:46:58.699 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-9", 2022-09-03T05:46:58.700 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498747Z", 2022-09-03T05:46:58.700 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:46:58.700 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:46:58.701 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:46:58.701 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:46:58.702 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.702 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:46:58.703 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:46:58.703 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 2 2022-09-03T05:46:58.704 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:46:58.704 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:46:58.704 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:46:58.705 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.705 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.705 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:46:58.705 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:46:58.707 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '41ecd2a5-afc0-4eae-9349-7dfe39baa1cb', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:46:58.708 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '41ecd2a5-afc0-4eae-9349-7dfe39baa1cb', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:46:58.709 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 41ecd2a5-afc0-4eae-9349-7dfe39baa1cb --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:46:58.764 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:46:58.765 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:46:58.800 INFO:teuthology.orchestra.run.smithi189.stdout:[ 2022-09-03T05:46:58.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.801 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182298.903952_68.1", 2022-09-03T05:46:58.801 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "user", 2022-09-03T05:46:58.801 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "rgw-multisite-test-user", 2022-09-03T05:46:58.801 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.903952433Z", 2022-09-03T05:46:58.802 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.802 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.802 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.802 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.803 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.803 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_4zvH7kUaBnzKjfeQSSk_55d", 2022-09-03T05:46:58.803 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.804 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.804 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.804 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:58.804 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.805 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.805 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182298.906852_69.1", 2022-09-03T05:46:58.806 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "user", 2022-09-03T05:46:58.806 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "rgw-multisite-test-user", 2022-09-03T05:46:58.806 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852070Z", 2022-09-03T05:46:58.806 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.807 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.807 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.807 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.808 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.808 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_4zvH7kUaBnzKjfeQSSk_55d", 2022-09-03T05:46:58.808 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.809 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.809 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:58.809 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.810 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.810 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.810 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.810 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.694951_88.1", 2022-09-03T05:46:58.811 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.811 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10", 2022-09-03T05:46:58.811 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.694951214Z", 2022-09-03T05:46:58.811 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.812 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.812 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.812 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.813 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.813 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_9ORw1pyvfMIqMonbAbpO7T7", 2022-09-03T05:46:58.813 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.814 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.814 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:58.814 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.814 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.815 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.815 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.700326_89.1", 2022-09-03T05:46:58.815 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.816 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10", 2022-09-03T05:46:58.816 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.700326196Z", 2022-09-03T05:46:58.816 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.816 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.817 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.817 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.817 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.818 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_9ORw1pyvfMIqMonbAbpO7T7", 2022-09-03T05:46:58.818 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.818 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.819 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:58.819 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.819 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.820 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.702420_90.1", 2022-09-03T05:46:58.820 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:58.821 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-10", 2022-09-03T05:46:58.821 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.702420359Z", 2022-09-03T05:46:58.821 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.823 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.823 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.823 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.824 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.824 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "__K0-JaxveSMyJYue_ni7kIA", 2022-09-03T05:46:58.824 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.825 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.825 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:58.825 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.825 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.826 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.705433_91.1", 2022-09-03T05:46:58.826 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:58.826 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-10", 2022-09-03T05:46:58.827 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.705433036Z", 2022-09-03T05:46:58.827 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.827 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.827 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.828 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.828 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.828 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.828 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "__K0-JaxveSMyJYue_ni7kIA", 2022-09-03T05:46:58.829 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.829 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.829 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.829 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:58.830 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.830 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.831 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.724120_92.1", 2022-09-03T05:46:58.831 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.831 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11", 2022-09-03T05:46:58.831 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.724120949Z", 2022-09-03T05:46:58.832 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.832 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.832 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.832 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.833 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.834 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_mm_a_MgGT-mjAscVo5eDJN3", 2022-09-03T05:46:58.834 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.835 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.835 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:58.835 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.836 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.836 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.726647_93.1", 2022-09-03T05:46:58.837 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.837 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11", 2022-09-03T05:46:58.837 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.726647140Z", 2022-09-03T05:46:58.837 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.838 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.838 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.838 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.839 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.839 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_mm_a_MgGT-mjAscVo5eDJN3", 2022-09-03T05:46:58.839 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.840 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.840 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:58.840 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.840 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.841 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.727701_94.1", 2022-09-03T05:46:58.841 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:58.842 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-11", 2022-09-03T05:46:58.842 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.727701875Z", 2022-09-03T05:46:58.842 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.843 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.844 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.844 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.844 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.845 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_co69vPv41E29NQft_mgvTJ6", 2022-09-03T05:46:58.845 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.846 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.846 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:58.846 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.847 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.847 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.730089_95.1", 2022-09-03T05:46:58.848 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:58.848 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-11", 2022-09-03T05:46:58.848 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.730089458Z", 2022-09-03T05:46:58.848 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.849 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.849 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.849 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.850 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.850 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_co69vPv41E29NQft_mgvTJ6", 2022-09-03T05:46:58.851 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.851 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.853 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:58.854 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.854 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.856 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.760919_96.1", 2022-09-03T05:46:58.856 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.857 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12", 2022-09-03T05:46:58.857 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.760919958Z", 2022-09-03T05:46:58.858 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.858 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.859 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.859 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.860 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.861 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_87U_6FvKdHGzg3asBWz0TB6", 2022-09-03T05:46:58.861 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.862 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.863 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:58.863 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.864 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.865 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.763582_97.1", 2022-09-03T05:46:58.866 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.866 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12", 2022-09-03T05:46:58.867 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.763582008Z", 2022-09-03T05:46:58.867 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.867 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.868 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.868 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.869 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.870 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_87U_6FvKdHGzg3asBWz0TB6", 2022-09-03T05:46:58.870 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.871 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.872 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:58.872 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.873 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.874 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.874 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.874 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.764601_98.1", 2022-09-03T05:46:58.875 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:58.875 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-12", 2022-09-03T05:46:58.876 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.764601024Z", 2022-09-03T05:46:58.876 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.877 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.878 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.878 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.879 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.880 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_F5RlS3fhN6ILhiyoQ3qKUpC", 2022-09-03T05:46:58.880 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.881 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.881 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.882 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:58.882 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.883 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.883 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.884 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.884 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182436.766898_99.1", 2022-09-03T05:46:58.885 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:58.885 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-12", 2022-09-03T05:46:58.885 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:36.766898391Z", 2022-09-03T05:46:58.886 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.886 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.887 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.887 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.888 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.888 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_F5RlS3fhN6ILhiyoQ3qKUpC", 2022-09-03T05:46:58.889 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.889 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.889 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.889 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:58.890 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.890 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.891 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.891 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182745.944715_100.1", 2022-09-03T05:46:58.891 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.892 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13", 2022-09-03T05:46:58.892 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:45.944715403Z", 2022-09-03T05:46:58.892 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.892 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.893 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.893 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.893 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.894 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.894 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_ZwtCDnJJncqHbK6_XX4E-Uv", 2022-09-03T05:46:58.894 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.895 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.896 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:58.896 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.896 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.897 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182745.948069_101.1", 2022-09-03T05:46:58.898 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.898 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13", 2022-09-03T05:46:58.899 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:45.948069121Z", 2022-09-03T05:46:58.899 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.899 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.900 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.900 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.901 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.901 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_ZwtCDnJJncqHbK6_XX4E-Uv", 2022-09-03T05:46:58.902 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.902 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.903 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.904 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:58.904 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.904 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.906 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182745.950114_102.1", 2022-09-03T05:46:58.906 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:58.907 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-13", 2022-09-03T05:46:58.907 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:45.950114676Z", 2022-09-03T05:46:58.907 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.907 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.908 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.908 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.909 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.909 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_kkkmqlwBfjjtwLr8U95H7wF", 2022-09-03T05:46:58.909 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.910 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.910 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:58.911 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.911 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.912 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182745.953048_103.1", 2022-09-03T05:46:58.913 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:58.913 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-13", 2022-09-03T05:46:58.913 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:45.953048780Z", 2022-09-03T05:46:58.914 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.914 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.914 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.915 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.915 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.915 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_kkkmqlwBfjjtwLr8U95H7wF", 2022-09-03T05:46:58.916 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.916 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.917 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:58.917 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.917 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.918 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182745.988740_104.1", 2022-09-03T05:46:58.918 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.919 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14", 2022-09-03T05:46:58.919 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:45.988740832Z", 2022-09-03T05:46:58.920 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.920 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.920 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.921 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.921 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.922 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Yidm7pQHzXjzyowF0vXCv_q", 2022-09-03T05:46:58.922 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.922 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.923 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:58.923 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.923 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.924 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182745.991668_105.1", 2022-09-03T05:46:58.925 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.925 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14", 2022-09-03T05:46:58.925 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:45.991668492Z", 2022-09-03T05:46:58.925 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.926 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.926 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.926 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.927 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.927 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Yidm7pQHzXjzyowF0vXCv_q", 2022-09-03T05:46:58.927 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.928 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.928 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.928 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:58.929 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.929 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.929 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.930 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182745.992722_106.1", 2022-09-03T05:46:58.930 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:58.930 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-14", 2022-09-03T05:46:58.931 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:45.992722013Z", 2022-09-03T05:46:58.931 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.931 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.931 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.932 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.932 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.932 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.933 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_0ZpMZLjvQ0yH4NIki0Mppzv", 2022-09-03T05:46:58.933 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.934 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.934 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:58.934 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.934 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.935 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.935 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.935 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182745.995451_107.1", 2022-09-03T05:46:58.936 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:58.936 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-14", 2022-09-03T05:46:58.936 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:45.995451716Z", 2022-09-03T05:46:58.937 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.937 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.937 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.937 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.938 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.938 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.938 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_0ZpMZLjvQ0yH4NIki0Mppzv", 2022-09-03T05:46:58.939 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.939 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.939 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:58.940 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.940 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.941 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.941 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182746.030460_108.1", 2022-09-03T05:46:58.941 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.941 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15", 2022-09-03T05:46:58.942 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:46.030460591Z", 2022-09-03T05:46:58.942 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.942 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.943 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.943 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.944 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.944 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_9EbzjhdvUtwpRj9gUb6Zy_c", 2022-09-03T05:46:58.944 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.945 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.945 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:58.945 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.946 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.946 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.946 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.947 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182746.033416_109.1", 2022-09-03T05:46:58.947 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.947 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15", 2022-09-03T05:46:58.948 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:46.033416114Z", 2022-09-03T05:46:58.948 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.948 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.948 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.949 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.949 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.950 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_9EbzjhdvUtwpRj9gUb6Zy_c", 2022-09-03T05:46:58.950 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.950 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.951 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.951 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:58.951 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.951 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.952 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.952 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182746.035561_110.1", 2022-09-03T05:46:58.953 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:58.953 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-15", 2022-09-03T05:46:58.953 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:46.035561132Z", 2022-09-03T05:46:58.954 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.954 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.954 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.955 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.955 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.955 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_p1RJcPdLE0s_MQy_T3ZeW1Z", 2022-09-03T05:46:58.956 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.956 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.956 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.956 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:58.957 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.957 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.957 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.957 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.958 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182746.041492_111.1", 2022-09-03T05:46:58.958 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:58.958 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-15", 2022-09-03T05:46:58.958 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:25:46.041492579Z", 2022-09-03T05:46:58.959 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.959 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.959 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.959 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.960 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.960 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_p1RJcPdLE0s_MQy_T3ZeW1Z", 2022-09-03T05:46:58.960 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.961 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.961 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:58.961 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.962 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.962 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.962 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.962 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182851.903860_122.1", 2022-09-03T05:46:58.963 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.963 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16", 2022-09-03T05:46:58.963 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:31.903860682Z", 2022-09-03T05:46:58.963 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.964 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.964 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.964 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.965 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.965 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:58.965 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.966 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.966 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.966 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:58.966 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.967 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.967 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.967 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.967 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182851.907991_123.1", 2022-09-03T05:46:58.968 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.968 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16", 2022-09-03T05:46:58.968 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:31.907991084Z", 2022-09-03T05:46:58.969 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.969 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.969 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.969 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.970 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.970 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.970 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:58.971 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.971 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.971 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.971 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:58.972 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.972 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.972 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.973 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.973 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182851.909130_124.1", 2022-09-03T05:46:58.973 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:58.974 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-16", 2022-09-03T05:46:58.974 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:31.909130687Z", 2022-09-03T05:46:58.974 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.975 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.975 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.975 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.975 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.976 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.976 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_P0q3AfXqSQL8XW6gVnA4CRA", 2022-09-03T05:46:58.976 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.977 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.977 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.977 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:58.978 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.978 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.978 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.978 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.979 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182851.915199_125.1", 2022-09-03T05:46:58.979 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:58.979 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-16", 2022-09-03T05:46:58.980 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:31.915199302Z", 2022-09-03T05:46:58.980 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.980 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.981 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.981 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.981 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.982 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.982 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_P0q3AfXqSQL8XW6gVnA4CRA", 2022-09-03T05:46:58.982 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.982 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.983 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.983 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:58.983 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.983 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.984 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.984 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.984 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182851.945316_126.1", 2022-09-03T05:46:58.984 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.985 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17", 2022-09-03T05:46:58.985 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:31.945316218Z", 2022-09-03T05:46:58.986 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.986 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.986 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.987 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.987 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.987 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.987 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:58.988 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.988 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.988 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.989 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:58.989 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.989 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.989 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.990 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.990 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182851.950120_127.1", 2022-09-03T05:46:58.990 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:58.990 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17", 2022-09-03T05:46:58.991 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:31.950120909Z", 2022-09-03T05:46:58.991 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.991 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.991 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.992 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.992 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.992 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.993 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:58.993 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.993 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.993 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.994 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:58.994 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.994 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.994 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.995 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:58.995 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182851.952454_128.1", 2022-09-03T05:46:58.995 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:58.996 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-17", 2022-09-03T05:46:58.996 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:31.952454005Z", 2022-09-03T05:46:58.996 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:58.996 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:58.997 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:58.997 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:58.997 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.997 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:58.998 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_j4__iZ2_YM9iD4VCcQhy-lG", 2022-09-03T05:46:58.998 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:58.998 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:58.999 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:58.999 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:58.999 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:58.999 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.000 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.000 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.000 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182851.958649_129.1", 2022-09-03T05:46:59.001 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.001 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-17", 2022-09-03T05:46:59.001 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:31.958649291Z", 2022-09-03T05:46:59.001 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.002 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.002 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.002 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.003 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.003 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.003 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_j4__iZ2_YM9iD4VCcQhy-lG", 2022-09-03T05:46:59.003 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.004 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.004 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.004 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.004 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.005 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.005 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.005 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.006 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182851.998554_130.1", 2022-09-03T05:46:59.006 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.006 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18", 2022-09-03T05:46:59.006 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:31.998554746Z", 2022-09-03T05:46:59.007 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.007 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.007 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.008 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.008 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.008 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.008 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:59.009 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.009 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.009 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.009 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.010 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.010 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.010 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.011 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182852.004610_131.1", 2022-09-03T05:46:59.011 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.011 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18", 2022-09-03T05:46:59.012 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:32.004610066Z", 2022-09-03T05:46:59.012 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.012 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.012 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.013 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.013 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.013 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.013 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:59.014 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.014 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.014 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.015 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.015 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.015 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.016 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.016 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182852.007026_132.1", 2022-09-03T05:46:59.016 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.016 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-18", 2022-09-03T05:46:59.017 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:32.007026261Z", 2022-09-03T05:46:59.017 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.017 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.018 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.018 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.018 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.018 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.019 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_O7UdALq44EBf7qF8RLPxZMv", 2022-09-03T05:46:59.019 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.019 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.019 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.020 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.020 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.020 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.020 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.021 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.021 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182852.013139_133.1", 2022-09-03T05:46:59.021 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.022 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-18", 2022-09-03T05:46:59.022 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:32.013139549Z", 2022-09-03T05:46:59.022 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.022 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.023 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.023 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.023 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.023 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.024 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_O7UdALq44EBf7qF8RLPxZMv", 2022-09-03T05:46:59.024 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.024 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.025 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.025 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.025 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.025 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.026 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.026 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.026 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182852.069466_134.1", 2022-09-03T05:46:59.026 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.027 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16", 2022-09-03T05:46:59.027 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:32.069466103Z", 2022-09-03T05:46:59.027 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.028 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.028 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:59.028 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.028 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.029 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.029 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:59.029 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.029 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.030 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.030 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.030 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.030 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.031 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.031 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.031 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182852.078407_135.1", 2022-09-03T05:46:59.031 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.032 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16", 2022-09-03T05:46:59.032 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:32.078407455Z", 2022-09-03T05:46:59.032 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.033 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.033 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:59.033 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.033 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.034 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.034 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_jcUVsJmQBcKkvs9Ai2AtpkG", 2022-09-03T05:46:59.034 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.034 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.035 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.035 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.035 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.036 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.036 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.036 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.036 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182852.095941_136.1", 2022-09-03T05:46:59.037 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.037 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17", 2022-09-03T05:46:59.037 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:32.095941153Z", 2022-09-03T05:46:59.037 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.038 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.038 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:59.038 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.039 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.039 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.039 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:59.039 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.040 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.040 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.040 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.040 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.041 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.041 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.041 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.041 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182852.099371_137.1", 2022-09-03T05:46:59.042 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.042 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17", 2022-09-03T05:46:59.042 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:32.099371547Z", 2022-09-03T05:46:59.042 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.043 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.043 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:59.043 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.044 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.044 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.044 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bfgLcf6SfsFYwXnXMkHq5Zv", 2022-09-03T05:46:59.044 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.045 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.045 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.045 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.045 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.046 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.046 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.046 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.047 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182852.121946_138.1", 2022-09-03T05:46:59.047 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.047 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18", 2022-09-03T05:46:59.047 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:32.121946745Z", 2022-09-03T05:46:59.048 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.048 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.048 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:59.048 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.049 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.049 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.049 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:59.049 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.050 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.050 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.050 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.051 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.051 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.051 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.051 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.052 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182852.124949_139.1", 2022-09-03T05:46:59.052 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.052 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18", 2022-09-03T05:46:59.052 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:27:32.124949763Z", 2022-09-03T05:46:59.053 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.053 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.053 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:59.053 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.054 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.054 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.054 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Yf8dWRVd8u5SJytE6g02RFO", 2022-09-03T05:46:59.055 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.055 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.055 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.055 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.056 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.056 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.056 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.056 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.057 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183242.240665_157.1", 2022-09-03T05:46:59.057 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.057 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19", 2022-09-03T05:46:59.057 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:34:02.240665028Z", 2022-09-03T05:46:59.058 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.058 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.058 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.059 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.059 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.059 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.059 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:59.060 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.060 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.060 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.060 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.061 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.061 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.061 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.061 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.062 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183242.243600_158.1", 2022-09-03T05:46:59.062 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.062 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19", 2022-09-03T05:46:59.063 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:34:02.243600484Z", 2022-09-03T05:46:59.063 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.063 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.063 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.064 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.064 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.064 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.065 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:59.065 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.065 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.065 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.066 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.066 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.066 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.066 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.067 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.067 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183242.244744_159.1", 2022-09-03T05:46:59.067 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.067 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-19", 2022-09-03T05:46:59.068 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:34:02.244744943Z", 2022-09-03T05:46:59.068 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.068 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.069 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.069 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.069 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.069 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.070 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_B_TWex9_lpCif6prNO7BbtO", 2022-09-03T05:46:59.070 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.070 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.070 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.071 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.071 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.071 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.071 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.072 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.072 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183242.247625_160.1", 2022-09-03T05:46:59.072 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.072 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-19", 2022-09-03T05:46:59.073 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:34:02.247625174Z", 2022-09-03T05:46:59.073 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.073 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.074 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.074 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.074 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.075 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.075 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_B_TWex9_lpCif6prNO7BbtO", 2022-09-03T05:46:59.075 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.075 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.076 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.076 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.076 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.076 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.077 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.077 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.077 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183242.265141_161.1", 2022-09-03T05:46:59.078 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.078 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19", 2022-09-03T05:46:59.078 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:34:02.265141968Z", 2022-09-03T05:46:59.078 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.079 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.079 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:59.079 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.079 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.080 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.080 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:59.080 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.080 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.081 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.081 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.081 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.081 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.082 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.082 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.082 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183242.267929_162.1", 2022-09-03T05:46:59.082 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.083 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19", 2022-09-03T05:46:59.083 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:34:02.267929270Z", 2022-09-03T05:46:59.083 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.084 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.084 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:59.084 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.084 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.085 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.085 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_DIC2rtrwzDhBSurcUTOG42m", 2022-09-03T05:46:59.085 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.085 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.086 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.086 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.086 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.086 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.087 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183458.677214_163.1", 2022-09-03T05:46:59.087 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.088 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:59.088 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:37:38.677214984Z", 2022-09-03T05:46:59.088 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.088 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.089 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.089 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.089 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.089 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.090 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:59.090 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.091 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.091 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.091 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.091 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.092 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.092 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.092 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183458.684800_164.1", 2022-09-03T05:46:59.092 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.093 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:59.093 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:37:38.684800313Z", 2022-09-03T05:46:59.093 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.093 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.094 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.094 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.094 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.095 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:59.095 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.095 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.095 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.096 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.096 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.096 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.097 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.097 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.097 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183458.685753_165.1", 2022-09-03T05:46:59.097 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.098 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-20", 2022-09-03T05:46:59.098 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:37:38.685753497Z", 2022-09-03T05:46:59.098 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.098 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.099 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.099 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.099 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.099 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.100 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_yPpDuhKk6Xsxh9Gd4Ds8exJ", 2022-09-03T05:46:59.100 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.100 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.101 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.101 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.101 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.101 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.102 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.102 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.102 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183458.694130_166.1", 2022-09-03T05:46:59.103 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.103 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-20", 2022-09-03T05:46:59.103 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:37:38.694130749Z", 2022-09-03T05:46:59.103 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.104 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.104 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.104 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.104 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.105 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.105 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_yPpDuhKk6Xsxh9Gd4Ds8exJ", 2022-09-03T05:46:59.105 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.106 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.106 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.106 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.107 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.107 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.107 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.108 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183531.352224_183.1", 2022-09-03T05:46:59.108 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.108 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:59.109 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:38:51.352224643Z", 2022-09-03T05:46:59.109 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.109 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.109 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:59.110 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.110 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.110 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.111 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:59.111 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.111 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.112 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.112 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.112 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.113 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.113 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183531.355467_184.1", 2022-09-03T05:46:59.113 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.113 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:59.114 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:38:51.355467679Z", 2022-09-03T05:46:59.114 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.114 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.114 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:59.115 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.115 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.115 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:59.116 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.116 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.117 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.117 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.117 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.118 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183570.365072_185.1", 2022-09-03T05:46:59.118 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.119 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:59.119 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:39:30.365072646Z", 2022-09-03T05:46:59.119 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.119 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.120 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:59.120 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.120 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.121 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:59.121 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 3 2022-09-03T05:46:59.121 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.121 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.122 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.122 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.122 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.123 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183570.368454_186.1", 2022-09-03T05:46:59.123 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.124 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:46:59.124 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:39:30.368454229Z", 2022-09-03T05:46:59.124 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.124 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.125 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:59.125 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.125 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.125 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.126 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YIyQWxhkNn1dGnRDc4UpSJ5", 2022-09-03T05:46:59.126 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 3 2022-09-03T05:46:59.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.126 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.127 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.127 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.127 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.128 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.128 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183652.125862_187.1", 2022-09-03T05:46:59.128 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.129 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21", 2022-09-03T05:46:59.129 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:52.125862114Z", 2022-09-03T05:46:59.129 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.129 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.129 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.130 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.130 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.131 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:59.131 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.131 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.132 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.132 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.132 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.133 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183652.132077_188.1", 2022-09-03T05:46:59.133 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.133 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21", 2022-09-03T05:46:59.134 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:52.132077397Z", 2022-09-03T05:46:59.134 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.134 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.135 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.135 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.135 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.136 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:59.136 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.137 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.137 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.137 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.137 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.138 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.138 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183652.134378_189.1", 2022-09-03T05:46:59.138 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.139 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-21", 2022-09-03T05:46:59.139 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:52.134378414Z", 2022-09-03T05:46:59.139 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.140 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.140 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.140 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.140 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.141 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.141 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_EARZCit60fDg0eMLHHZC990", 2022-09-03T05:46:59.141 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.141 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.142 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.142 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.142 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.142 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.143 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.143 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183652.140411_190.1", 2022-09-03T05:46:59.144 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.144 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-21", 2022-09-03T05:46:59.144 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:52.140411296Z", 2022-09-03T05:46:59.144 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.145 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.145 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.145 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.146 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.146 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_EARZCit60fDg0eMLHHZC990", 2022-09-03T05:46:59.146 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.147 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.147 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.147 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.148 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.148 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.148 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.162940_191.1", 2022-09-03T05:46:59.149 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.149 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22", 2022-09-03T05:46:59.149 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.162940466Z", 2022-09-03T05:46:59.149 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.150 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.150 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.150 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.151 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.151 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:59.151 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.152 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.152 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.152 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.153 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.153 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.153 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.167017_192.1", 2022-09-03T05:46:59.154 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.154 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22", 2022-09-03T05:46:59.154 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.167017131Z", 2022-09-03T05:46:59.154 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.155 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.155 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.155 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.156 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.156 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:59.156 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.157 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.157 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.157 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.157 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.158 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.158 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.159 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.167997_193.1", 2022-09-03T05:46:59.159 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.159 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-22", 2022-09-03T05:46:59.160 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.167997739Z", 2022-09-03T05:46:59.160 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.160 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.160 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.161 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.161 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.161 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tI1b7-ygEU1TnWToSDcKGBc", 2022-09-03T05:46:59.162 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.162 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.162 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.163 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.163 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.163 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.164 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.170492_194.1", 2022-09-03T05:46:59.164 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.164 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-22", 2022-09-03T05:46:59.164 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.170492762Z", 2022-09-03T05:46:59.165 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.165 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.165 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.165 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.166 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.166 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tI1b7-ygEU1TnWToSDcKGBc", 2022-09-03T05:46:59.167 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.167 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.167 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.168 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.168 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.168 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.169 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.220411_195.1", 2022-09-03T05:46:59.169 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.169 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23", 2022-09-03T05:46:59.169 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.220411899Z", 2022-09-03T05:46:59.170 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.170 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.170 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.170 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.171 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.171 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:59.172 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.172 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.172 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.173 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.173 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.173 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.173 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.174 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.174 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.223172_196.1", 2022-09-03T05:46:59.175 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.175 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23", 2022-09-03T05:46:59.175 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.223172774Z", 2022-09-03T05:46:59.175 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.176 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.176 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.176 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.177 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.177 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.177 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:59.177 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.178 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.178 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.179 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.179 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.179 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.180 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.224266_197.1", 2022-09-03T05:46:59.180 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.180 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-23", 2022-09-03T05:46:59.180 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.224266788Z", 2022-09-03T05:46:59.181 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.181 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.181 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.182 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.182 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.183 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_mrTr4jlOIDprI-5OTphGZ6h", 2022-09-03T05:46:59.183 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.183 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.183 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.184 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.184 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.184 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.185 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.185 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.226840_198.1", 2022-09-03T05:46:59.185 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.185 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-23", 2022-09-03T05:46:59.186 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.226840552Z", 2022-09-03T05:46:59.186 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.186 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.187 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.187 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.187 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.188 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_mrTr4jlOIDprI-5OTphGZ6h", 2022-09-03T05:46:59.188 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.189 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.189 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.189 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.189 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.190 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.267809_199.1", 2022-09-03T05:46:59.191 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.191 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21", 2022-09-03T05:46:59.191 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.267809597Z", 2022-09-03T05:46:59.191 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.192 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.192 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:59.192 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.192 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.193 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.193 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:59.193 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.194 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.194 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.194 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.194 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.195 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.195 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.270381_200.1", 2022-09-03T05:46:59.195 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.196 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21", 2022-09-03T05:46:59.196 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.270381683Z", 2022-09-03T05:46:59.196 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.196 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.197 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:59.197 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.197 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.197 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.198 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_qKRBYdVvNZz_t-UcT7qSDwC", 2022-09-03T05:46:59.198 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.198 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.199 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.199 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.199 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.200 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.283633_201.1", 2022-09-03T05:46:59.200 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.201 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22", 2022-09-03T05:46:59.201 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.283633845Z", 2022-09-03T05:46:59.201 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.202 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.202 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:59.202 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.203 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.203 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:59.203 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.204 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.204 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.204 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.204 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.205 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.288001_202.1", 2022-09-03T05:46:59.205 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.206 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22", 2022-09-03T05:46:59.206 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.288001351Z", 2022-09-03T05:46:59.206 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.206 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.207 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:59.207 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.207 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.208 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_N6M2beq2-Ejtnps2_jGEYzx", 2022-09-03T05:46:59.208 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.208 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.209 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.209 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.209 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.210 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.210 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.210 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.304152_203.1", 2022-09-03T05:46:59.210 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.211 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23", 2022-09-03T05:46:59.211 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.304152195Z", 2022-09-03T05:46:59.211 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.211 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.212 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:59.212 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.212 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.213 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:59.213 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.213 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.214 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.214 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.214 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.215 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.215 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.215 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183653.308405_204.1", 2022-09-03T05:46:59.215 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.216 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23", 2022-09-03T05:46:59.216 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:40:53.308405313Z", 2022-09-03T05:46:59.216 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.216 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.217 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:59.217 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.218 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.218 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_PV0rMgRRlm0AjitvG5rEESO", 2022-09-03T05:46:59.218 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.219 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.219 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.219 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.219 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.220 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.220 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.220 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.657291_207.1", 2022-09-03T05:46:59.220 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.221 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:59.221 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.657291334Z", 2022-09-03T05:46:59.221 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.221 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.222 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.222 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.222 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.223 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:59.223 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.224 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.224 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.224 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.224 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.225 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.661364_208.1", 2022-09-03T05:46:59.225 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.226 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:59.226 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.661364231Z", 2022-09-03T05:46:59.226 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.226 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.227 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.227 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.228 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.228 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:59.228 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.229 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.229 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.229 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.229 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.230 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.230 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.662317_209.1", 2022-09-03T05:46:59.230 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.231 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-24", 2022-09-03T05:46:59.231 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.662317133Z", 2022-09-03T05:46:59.231 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.231 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.232 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.232 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.233 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.233 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_RWbG2F3yjlSXFQy-Qq_ON1N", 2022-09-03T05:46:59.233 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.234 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.234 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.234 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.235 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.235 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.666554_210.1", 2022-09-03T05:46:59.236 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.236 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-24", 2022-09-03T05:46:59.236 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.666554708Z", 2022-09-03T05:46:59.236 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.237 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.237 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.237 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.238 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.238 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_RWbG2F3yjlSXFQy-Qq_ON1N", 2022-09-03T05:46:59.238 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.239 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.239 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.239 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.240 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.240 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.240 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.685377_211.1", 2022-09-03T05:46:59.241 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.241 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:59.241 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.685377688Z", 2022-09-03T05:46:59.241 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.242 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.242 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.242 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.243 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.243 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:59.243 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.244 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.244 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.244 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.245 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.245 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.689809_212.1", 2022-09-03T05:46:59.246 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.246 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:59.246 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.689809673Z", 2022-09-03T05:46:59.246 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.247 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.247 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.247 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.248 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.248 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:59.248 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.249 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.249 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.249 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.250 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.250 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.690762_213.1", 2022-09-03T05:46:59.251 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.251 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-25", 2022-09-03T05:46:59.251 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.690762414Z", 2022-09-03T05:46:59.252 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.252 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.252 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.252 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.253 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.253 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_IaGw1AfiCAqC-aV2Oup3OCi", 2022-09-03T05:46:59.253 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.254 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.254 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.254 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.255 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.256 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.693239_214.1", 2022-09-03T05:46:59.256 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.256 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-25", 2022-09-03T05:46:59.256 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.693239599Z", 2022-09-03T05:46:59.257 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.257 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.257 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.257 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.258 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.258 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_IaGw1AfiCAqC-aV2Oup3OCi", 2022-09-03T05:46:59.258 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.259 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.259 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.260 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.260 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.261 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.727704_215.1", 2022-09-03T05:46:59.261 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.261 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:59.261 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.727704277Z", 2022-09-03T05:46:59.262 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.262 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.262 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.263 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.263 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.263 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:59.264 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.264 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.264 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.265 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.265 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.266 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.731553_216.1", 2022-09-03T05:46:59.266 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.266 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:59.267 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.731553123Z", 2022-09-03T05:46:59.267 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.267 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.267 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.268 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.268 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.268 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:59.269 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.269 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.269 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.270 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.270 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.271 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.732533_217.1", 2022-09-03T05:46:59.271 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.271 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-26", 2022-09-03T05:46:59.271 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.732533246Z", 2022-09-03T05:46:59.272 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.272 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.272 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.272 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.273 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.273 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_dUSEiciichvry8cYc4lxW96", 2022-09-03T05:46:59.273 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.274 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.274 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.275 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.275 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.276 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.734903_218.1", 2022-09-03T05:46:59.276 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.276 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-26", 2022-09-03T05:46:59.277 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.734903969Z", 2022-09-03T05:46:59.277 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.277 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.278 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.278 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.278 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.279 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_dUSEiciichvry8cYc4lxW96", 2022-09-03T05:46:59.279 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.280 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.280 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.280 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.280 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.281 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.281 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.763839_219.1", 2022-09-03T05:46:59.281 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.282 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:59.282 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.763839115Z", 2022-09-03T05:46:59.282 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.283 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.283 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:59.283 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.284 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.284 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:59.284 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.285 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.285 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.285 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.286 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.286 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.766491_220.1", 2022-09-03T05:46:59.287 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.287 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:59.287 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.766491425Z", 2022-09-03T05:46:59.288 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.288 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.288 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:59.288 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.289 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.289 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:59.289 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.290 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.290 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.290 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.291 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.292 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.780440_221.1", 2022-09-03T05:46:59.292 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.292 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:59.292 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.780440878Z", 2022-09-03T05:46:59.293 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.293 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.293 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:59.293 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.294 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.294 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:59.294 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 3 2022-09-03T05:46:59.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.295 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.295 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.296 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.296 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.297 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.784428_222.1", 2022-09-03T05:46:59.297 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.297 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:46:59.297 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.784428128Z", 2022-09-03T05:46:59.298 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.298 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.298 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:59.298 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.299 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.299 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_fqFS8FjaEbP1OwUn8l3nO0Z", 2022-09-03T05:46:59.299 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 3 2022-09-03T05:46:59.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.300 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.300 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.300 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.301 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.302 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.798318_223.1", 2022-09-03T05:46:59.302 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.302 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:59.302 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.798318959Z", 2022-09-03T05:46:59.303 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.303 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.303 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:59.303 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.304 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.304 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:59.305 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.305 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.305 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.306 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.306 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.307 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.307 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.802397_224.1", 2022-09-03T05:46:59.307 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.307 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:59.308 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.802397402Z", 2022-09-03T05:46:59.308 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.308 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.308 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:59.309 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.309 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.310 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:59.310 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.311 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.311 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.311 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.311 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.312 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.312 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.818536_225.1", 2022-09-03T05:46:59.313 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.313 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:59.313 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.818536670Z", 2022-09-03T05:46:59.313 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.314 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.314 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:59.314 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.315 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.315 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:59.315 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 3 2022-09-03T05:46:59.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.316 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.316 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.317 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.317 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.317 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.317 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.318 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.826419_226.1", 2022-09-03T05:46:59.318 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.318 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:46:59.319 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.826419304Z", 2022-09-03T05:46:59.319 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.320 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.320 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:59.321 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.321 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.322 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.322 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c2Rf9-W31-YRLJJEGAKMSHa", 2022-09-03T05:46:59.323 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 3 2022-09-03T05:46:59.323 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.324 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.324 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.325 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.325 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.326 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.327 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.854560_227.1", 2022-09-03T05:46:59.327 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.328 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:59.328 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.854560270Z", 2022-09-03T05:46:59.329 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.329 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.329 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:59.330 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.330 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.331 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.331 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:59.332 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.332 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.333 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.333 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.334 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.334 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.335 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.335 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.336 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.857527_228.1", 2022-09-03T05:46:59.336 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.337 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:59.337 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.857527676Z", 2022-09-03T05:46:59.338 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.338 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.339 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:59.339 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.340 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.340 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.341 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:59.341 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.342 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.342 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.343 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.343 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.343 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.344 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.344 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.870879_229.1", 2022-09-03T05:46:59.344 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.345 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:59.345 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.870879960Z", 2022-09-03T05:46:59.346 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.346 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.347 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:59.347 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.348 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.348 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.349 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:59.349 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 3 2022-09-03T05:46:59.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.350 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.351 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.351 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.352 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.352 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.353 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.353 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183709.876923_230.1", 2022-09-03T05:46:59.354 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.354 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:46:59.355 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:41:49.876923670Z", 2022-09-03T05:46:59.355 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.356 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.356 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:59.357 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.357 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.358 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.358 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lY7IpJ6dT0E596jdZh10lv3", 2022-09-03T05:46:59.359 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 3 2022-09-03T05:46:59.359 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.360 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.361 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.361 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.362 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.363 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.363 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.870437_232.1", 2022-09-03T05:46:59.364 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.364 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-27:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.27", 2022-09-03T05:46:59.365 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.870437527Z", 2022-09-03T05:46:59.365 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.366 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.366 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.367 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.367 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.368 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.368 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:59.369 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.369 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.369 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.370 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.370 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.371 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.371 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.372 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.372 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.874799_233.1", 2022-09-03T05:46:59.373 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.373 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-27:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.27", 2022-09-03T05:46:59.373 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.874799084Z", 2022-09-03T05:46:59.374 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.374 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.374 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.375 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.375 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.375 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.376 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:59.376 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.376 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.376 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.377 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.377 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.377 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.379 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.876797_234.1", 2022-09-03T05:46:59.379 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.380 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-27", 2022-09-03T05:46:59.380 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.876797147Z", 2022-09-03T05:46:59.381 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.381 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.382 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.382 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.383 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.383 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.384 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_kvVMhGyMiz3kusJkbwnyhxJ", 2022-09-03T05:46:59.384 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.385 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.385 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.386 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.386 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.387 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.387 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.388 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.885232_235.1", 2022-09-03T05:46:59.388 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.389 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-27", 2022-09-03T05:46:59.389 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.885232353Z", 2022-09-03T05:46:59.390 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.390 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.391 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.391 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.392 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.392 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_kvVMhGyMiz3kusJkbwnyhxJ", 2022-09-03T05:46:59.393 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.393 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.394 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.394 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.395 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.395 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.395 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.396 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.396 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.916915_236.1", 2022-09-03T05:46:59.397 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.397 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28", 2022-09-03T05:46:59.397 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.916915113Z", 2022-09-03T05:46:59.398 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.398 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.398 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.399 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.399 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.399 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.399 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:59.400 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.400 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.400 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.401 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.401 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.401 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.402 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.920958_237.1", 2022-09-03T05:46:59.403 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.403 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28", 2022-09-03T05:46:59.403 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.920958536Z", 2022-09-03T05:46:59.404 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.404 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.405 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.405 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.405 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.406 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.406 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:59.406 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.407 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.407 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.407 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.408 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.408 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.408 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.922045_238.1", 2022-09-03T05:46:59.409 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.409 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-28", 2022-09-03T05:46:59.410 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.922045913Z", 2022-09-03T05:46:59.410 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.411 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.411 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.412 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.412 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.413 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.413 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_OOWZg42AYED_QBCONdEGECB", 2022-09-03T05:46:59.414 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.415 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.415 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.416 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.416 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.417 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.418 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.926094_239.1", 2022-09-03T05:46:59.418 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.419 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-28", 2022-09-03T05:46:59.419 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.926094548Z", 2022-09-03T05:46:59.420 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.420 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.421 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.421 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.421 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.422 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.422 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_OOWZg42AYED_QBCONdEGECB", 2022-09-03T05:46:59.422 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.423 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.423 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.424 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.424 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.425 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.425 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.425 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.962517_240.1", 2022-09-03T05:46:59.426 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.426 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-29:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.29", 2022-09-03T05:46:59.426 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.962517073Z", 2022-09-03T05:46:59.426 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.427 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.427 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.427 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.428 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.428 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:59.428 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.429 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.429 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.429 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.429 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.430 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.430 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.430 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.431 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.965127_241.1", 2022-09-03T05:46:59.431 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.431 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-29:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.29", 2022-09-03T05:46:59.431 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.965127874Z", 2022-09-03T05:46:59.432 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.432 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.432 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.432 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.433 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.433 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:59.434 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.434 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.434 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.435 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.435 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.435 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.436 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.966081_242.1", 2022-09-03T05:46:59.436 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.436 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-29", 2022-09-03T05:46:59.437 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.966081501Z", 2022-09-03T05:46:59.437 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.437 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.437 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.438 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.438 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.438 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_yDkLaMdkLBW9LxbH_x72oyR", 2022-09-03T05:46:59.439 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.439 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.439 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.439 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.440 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.440 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.441 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.441 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.968432_243.1", 2022-09-03T05:46:59.441 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.441 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-29", 2022-09-03T05:46:59.442 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.968432661Z", 2022-09-03T05:46:59.442 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.442 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.443 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.443 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.444 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.444 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_yDkLaMdkLBW9LxbH_x72oyR", 2022-09-03T05:46:59.444 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.445 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.445 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.446 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.446 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.446 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.446 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.447 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183849.998970_244.1", 2022-09-03T05:46:59.447 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.447 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-27:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.27", 2022-09-03T05:46:59.448 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:09.998970343Z", 2022-09-03T05:46:59.448 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.448 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.448 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:59.449 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.449 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.450 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:59.450 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.450 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.451 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.451 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.451 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.452 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.452 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.452 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.001548_245.1", 2022-09-03T05:46:59.453 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.453 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-27:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.27", 2022-09-03T05:46:59.454 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.001548177Z", 2022-09-03T05:46:59.454 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.454 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.454 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:59.455 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.455 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.455 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_pT-IuVMZqFs9Q4a-c1AepIg", 2022-09-03T05:46:59.456 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.456 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.457 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.457 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.457 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.457 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.458 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.458 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.049458_246.1", 2022-09-03T05:46:59.458 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.458 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28", 2022-09-03T05:46:59.459 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.049458314Z", 2022-09-03T05:46:59.459 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.459 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.459 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:59.460 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.460 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.461 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:59.461 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.461 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.462 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.462 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.462 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.463 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.463 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.463 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.052066_247.1", 2022-09-03T05:46:59.464 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.464 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28", 2022-09-03T05:46:59.464 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.052066976Z", 2022-09-03T05:46:59.464 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.465 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.465 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:59.465 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.466 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.466 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_xgo_zKtIc3tF8yauH4ls-tW", 2022-09-03T05:46:59.466 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.466 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.467 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.467 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.467 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.468 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.468 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.468 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.468 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.091593_248.1", 2022-09-03T05:46:59.469 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.469 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-29:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.29", 2022-09-03T05:46:59.469 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.091593357Z", 2022-09-03T05:46:59.469 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.470 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.470 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:59.470 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.470 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.471 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.471 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:59.471 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.472 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.472 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.472 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.472 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.473 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.473 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.473 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.473 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.099123_249.1", 2022-09-03T05:46:59.474 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.474 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-29:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.29", 2022-09-03T05:46:59.474 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.099123834Z", 2022-09-03T05:46:59.475 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.475 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.475 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:59.475 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.476 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.476 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.476 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_2LYefgwCcyHtWC7TpDNPFZ7", 2022-09-03T05:46:59.476 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.477 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.477 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.477 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.477 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.478 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.478 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.478 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.479 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.135078_250.1", 2022-09-03T05:46:59.479 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.479 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30", 2022-09-03T05:46:59.479 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.135078742Z", 2022-09-03T05:46:59.480 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.480 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.480 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.480 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.481 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.481 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.481 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:59.481 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.482 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.482 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.482 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.483 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.483 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.483 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.483 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.484 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.140570_251.1", 2022-09-03T05:46:59.484 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.484 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30", 2022-09-03T05:46:59.485 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.140570450Z", 2022-09-03T05:46:59.485 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.485 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.485 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.486 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.486 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.486 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.486 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:59.487 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.487 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.487 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.488 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.488 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.488 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.488 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.489 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.489 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.142697_252.1", 2022-09-03T05:46:59.489 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.490 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-30", 2022-09-03T05:46:59.490 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.142697729Z", 2022-09-03T05:46:59.490 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.490 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.491 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.491 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.491 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.491 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.492 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_4yhNaX-8bC2cTXp5LKWw-Qo", 2022-09-03T05:46:59.492 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.492 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.493 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.493 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.493 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.493 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.494 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.494 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.494 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.148860_253.1", 2022-09-03T05:46:59.494 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.495 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-30", 2022-09-03T05:46:59.495 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.148860761Z", 2022-09-03T05:46:59.495 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.496 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.496 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.496 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.496 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.497 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.497 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_4yhNaX-8bC2cTXp5LKWw-Qo", 2022-09-03T05:46:59.497 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.497 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.498 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.498 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.498 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.499 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.499 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.499 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.499 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.172332_254.1", 2022-09-03T05:46:59.500 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.500 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31", 2022-09-03T05:46:59.500 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.172332646Z", 2022-09-03T05:46:59.500 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.501 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.501 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.501 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.501 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.502 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.502 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:59.502 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.503 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.503 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.503 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.503 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.504 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.504 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.504 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.504 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.176046_255.1", 2022-09-03T05:46:59.505 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.505 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31", 2022-09-03T05:46:59.505 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.176046838Z", 2022-09-03T05:46:59.506 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.506 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.506 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.506 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.507 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.507 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.507 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:59.508 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.508 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.508 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.509 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.509 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.509 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.509 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.510 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.176958_256.1", 2022-09-03T05:46:59.510 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.510 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-31", 2022-09-03T05:46:59.511 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.176958510Z", 2022-09-03T05:46:59.511 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.511 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.511 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.512 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.512 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.512 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.513 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_oPtbK62ZjFZvpDTUcbzdHLT", 2022-09-03T05:46:59.513 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.513 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.514 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.514 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.514 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.515 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.515 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.179165_257.1", 2022-09-03T05:46:59.516 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.516 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-31", 2022-09-03T05:46:59.516 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.179165643Z", 2022-09-03T05:46:59.517 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.517 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.517 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.517 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.518 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.518 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_oPtbK62ZjFZvpDTUcbzdHLT", 2022-09-03T05:46:59.518 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.519 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.519 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.520 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.520 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.521 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.220311_258.1", 2022-09-03T05:46:59.521 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.521 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-32:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.32", 2022-09-03T05:46:59.521 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.220311791Z", 2022-09-03T05:46:59.522 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.522 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.522 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.523 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.523 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.523 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:59.524 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.524 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.524 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.525 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.525 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.526 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.526 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.222654_259.1", 2022-09-03T05:46:59.526 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.527 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-32:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.32", 2022-09-03T05:46:59.527 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.222654278Z", 2022-09-03T05:46:59.527 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.528 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.528 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.528 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.529 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.529 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:59.529 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.530 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.530 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.530 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.530 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.531 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.223513_260.1", 2022-09-03T05:46:59.532 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.532 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-32", 2022-09-03T05:46:59.532 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.223513282Z", 2022-09-03T05:46:59.532 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.533 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.533 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.533 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.534 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.534 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_y0oxyMaYTjSUF-S-HkK29E5", 2022-09-03T05:46:59.534 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.535 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.535 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.535 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.536 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.536 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.228241_261.1", 2022-09-03T05:46:59.537 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.537 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-32", 2022-09-03T05:46:59.537 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.228241236Z", 2022-09-03T05:46:59.537 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.538 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.538 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.538 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.539 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.539 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_y0oxyMaYTjSUF-S-HkK29E5", 2022-09-03T05:46:59.539 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.540 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.540 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.540 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.541 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.541 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.265477_262.1", 2022-09-03T05:46:59.542 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.542 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30", 2022-09-03T05:46:59.542 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.265477055Z", 2022-09-03T05:46:59.542 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.543 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.543 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:59.543 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.544 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.544 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:59.544 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.545 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.545 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.545 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.546 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.546 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.268146_263.1", 2022-09-03T05:46:59.547 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.547 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30", 2022-09-03T05:46:59.547 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.268146553Z", 2022-09-03T05:46:59.548 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.548 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.548 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:59.548 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.549 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.549 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_gDdLHBrONQHZYyA5kwAiWf3", 2022-09-03T05:46:59.549 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.550 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.550 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.551 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.551 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.552 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.350018_264.1", 2022-09-03T05:46:59.552 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.552 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31", 2022-09-03T05:46:59.552 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.350018238Z", 2022-09-03T05:46:59.553 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.553 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.553 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:59.554 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.554 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.554 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:59.555 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.555 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.555 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.556 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.556 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.557 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.352665_265.1", 2022-09-03T05:46:59.557 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.557 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31", 2022-09-03T05:46:59.558 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.352665529Z", 2022-09-03T05:46:59.558 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.558 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.558 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:59.559 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.559 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.559 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_rKZQQwmpTBFyGVU5lN5YWjy", 2022-09-03T05:46:59.560 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.560 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.561 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.561 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.561 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.562 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.401743_266.1", 2022-09-03T05:46:59.562 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.562 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-32:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.32", 2022-09-03T05:46:59.563 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.401743084Z", 2022-09-03T05:46:59.563 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.563 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.563 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:59.564 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.564 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.565 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:59.565 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.565 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.566 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.566 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.566 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.567 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.404375_267.1", 2022-09-03T05:46:59.567 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.568 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-32:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.32", 2022-09-03T05:46:59.568 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.404375474Z", 2022-09-03T05:46:59.568 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.568 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.569 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:59.569 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.569 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.570 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_3XVXRCzWwUtqdfiZZCtZ9bq", 2022-09-03T05:46:59.570 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.570 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.571 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.571 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.571 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.572 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.434979_268.1", 2022-09-03T05:46:59.572 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.573 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-33:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.33", 2022-09-03T05:46:59.573 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.434979477Z", 2022-09-03T05:46:59.573 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.573 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.574 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.574 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.574 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.575 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:59.575 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.576 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.576 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.576 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.576 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.577 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.438289_269.1", 2022-09-03T05:46:59.578 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.578 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-33:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.33", 2022-09-03T05:46:59.578 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.438289839Z", 2022-09-03T05:46:59.579 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.579 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.579 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.579 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.580 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.580 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:59.581 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.581 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.581 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.582 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.582 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.583 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.439390_270.1", 2022-09-03T05:46:59.583 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.583 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-33", 2022-09-03T05:46:59.584 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.439390666Z", 2022-09-03T05:46:59.584 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.584 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.584 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.585 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.585 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.586 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_XHlZf1wWm_Tr8FwCOGzkHwG", 2022-09-03T05:46:59.586 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.586 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.587 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.587 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.587 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.588 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.441665_271.1", 2022-09-03T05:46:59.588 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.588 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-33", 2022-09-03T05:46:59.589 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.441665178Z", 2022-09-03T05:46:59.589 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.589 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.589 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.590 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.590 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.591 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_XHlZf1wWm_Tr8FwCOGzkHwG", 2022-09-03T05:46:59.591 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.591 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.592 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.592 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.592 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.593 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.461635_272.1", 2022-09-03T05:46:59.593 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.593 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34", 2022-09-03T05:46:59.594 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.461635262Z", 2022-09-03T05:46:59.594 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.594 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.595 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.595 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.595 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.596 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:59.596 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.596 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.597 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.597 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.597 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.598 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.466010_273.1", 2022-09-03T05:46:59.599 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.599 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34", 2022-09-03T05:46:59.599 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.466010065Z", 2022-09-03T05:46:59.600 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.600 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.600 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.600 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.601 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.601 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:59.601 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.602 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.602 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.602 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.603 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.603 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.604 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.468430_274.1", 2022-09-03T05:46:59.604 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.604 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-34", 2022-09-03T05:46:59.604 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.468430282Z", 2022-09-03T05:46:59.605 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.605 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.605 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.605 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.606 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.606 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_8JLNrQKjOCrzmEAJH1Gqebc", 2022-09-03T05:46:59.607 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.607 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.607 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.607 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.608 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.608 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.609 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.473656_275.1", 2022-09-03T05:46:59.609 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.609 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-34", 2022-09-03T05:46:59.609 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.473656625Z", 2022-09-03T05:46:59.610 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.610 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.610 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.611 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.611 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.611 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.611 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_8JLNrQKjOCrzmEAJH1Gqebc", 2022-09-03T05:46:59.612 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.612 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.612 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.613 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.613 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.614 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.519749_276.1", 2022-09-03T05:46:59.614 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.615 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35", 2022-09-03T05:46:59.615 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.519749696Z", 2022-09-03T05:46:59.615 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.616 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.616 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.616 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.617 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.617 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:59.617 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.618 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.618 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.618 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.619 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.619 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.527797_277.1", 2022-09-03T05:46:59.620 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.620 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35", 2022-09-03T05:46:59.620 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.527797485Z", 2022-09-03T05:46:59.620 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.621 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.621 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.621 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.622 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.622 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:59.622 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.623 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.623 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.623 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.624 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.624 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.530086_278.1", 2022-09-03T05:46:59.625 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.625 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-35", 2022-09-03T05:46:59.625 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.530086284Z", 2022-09-03T05:46:59.626 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.626 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.626 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.627 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.627 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.628 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_9eV0dD_Np6i5r-IXItyYqq8", 2022-09-03T05:46:59.628 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.629 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.629 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.630 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.630 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.632 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.542414_279.1", 2022-09-03T05:46:59.633 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.633 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-35", 2022-09-03T05:46:59.634 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.542414804Z", 2022-09-03T05:46:59.634 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.634 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.635 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.635 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.636 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.636 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_9eV0dD_Np6i5r-IXItyYqq8", 2022-09-03T05:46:59.637 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.637 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.637 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.638 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.638 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.639 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.574433_280.1", 2022-09-03T05:46:59.639 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.639 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-33:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.33", 2022-09-03T05:46:59.640 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.574433191Z", 2022-09-03T05:46:59.640 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.640 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.641 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:59.641 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.641 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.642 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:59.642 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.642 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.643 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.643 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.643 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.644 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183850.577143_281.1", 2022-09-03T05:46:59.644 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.645 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-33:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.33", 2022-09-03T05:46:59.645 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:10.577143931Z", 2022-09-03T05:46:59.645 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.645 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.646 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:59.646 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.647 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.647 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "__XYcHuAu4mClgFNIskDEId-", 2022-09-03T05:46:59.647 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.648 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.648 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.648 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.648 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.649 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.649 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183851.605820_282.1", 2022-09-03T05:46:59.649 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.650 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34", 2022-09-03T05:46:59.650 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:11.605820553Z", 2022-09-03T05:46:59.650 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.651 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.651 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:59.651 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.652 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.652 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:59.652 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.653 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.653 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.653 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.654 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.654 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183851.610169_283.1", 2022-09-03T05:46:59.655 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.655 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34", 2022-09-03T05:46:59.655 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:11.610169538Z", 2022-09-03T05:46:59.655 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.656 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.656 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:59.656 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.657 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.657 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_L1SUoHh-6AZ6TuJcaUNl0sJ", 2022-09-03T05:46:59.657 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.658 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.658 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.658 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.659 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.660 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.681884_284.1", 2022-09-03T05:46:59.660 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.660 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35", 2022-09-03T05:46:59.660 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.681884846Z", 2022-09-03T05:46:59.661 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.661 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.661 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:59.661 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.662 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.663 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:59.663 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.664 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.664 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.664 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.665 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.666 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.684548_285.1", 2022-09-03T05:46:59.666 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.667 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35", 2022-09-03T05:46:59.667 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.684548086Z", 2022-09-03T05:46:59.667 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.667 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.668 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:59.668 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.669 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.669 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_MUdVaa5RwX7lib6RKPfr0nQ", 2022-09-03T05:46:59.669 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.670 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.670 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.670 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.671 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.671 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.769697_286.1", 2022-09-03T05:46:59.672 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.672 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36", 2022-09-03T05:46:59.672 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.769697830Z", 2022-09-03T05:46:59.673 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.673 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.673 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.673 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.674 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.674 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lzXJaycHyP7aPeCDF57oxvc", 2022-09-03T05:46:59.675 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.675 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.675 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.676 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.676 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.677 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.780292_287.1", 2022-09-03T05:46:59.677 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.677 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36", 2022-09-03T05:46:59.678 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.780292446Z", 2022-09-03T05:46:59.678 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.678 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.679 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.679 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.679 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.680 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lzXJaycHyP7aPeCDF57oxvc", 2022-09-03T05:46:59.680 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.680 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.681 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.681 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.681 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.682 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.781473_288.1", 2022-09-03T05:46:59.682 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.683 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-36", 2022-09-03T05:46:59.683 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.781473404Z", 2022-09-03T05:46:59.683 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.683 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.684 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.684 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.684 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.685 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Aunq5-l6wG1IGbEHKmUcfjj", 2022-09-03T05:46:59.685 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.685 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.686 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.686 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.686 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.687 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.785261_289.1", 2022-09-03T05:46:59.687 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.688 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-36", 2022-09-03T05:46:59.688 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.785261123Z", 2022-09-03T05:46:59.688 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.688 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.689 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.689 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.689 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.690 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Aunq5-l6wG1IGbEHKmUcfjj", 2022-09-03T05:46:59.690 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.691 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.691 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.691 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.691 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.692 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.692 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.692 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.806309_290.1", 2022-09-03T05:46:59.692 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.693 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37", 2022-09-03T05:46:59.693 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.806309544Z", 2022-09-03T05:46:59.693 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.693 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.694 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.694 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.695 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.695 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_kuQ0xr50nHBW7YTCPQavDGz", 2022-09-03T05:46:59.695 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.696 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.696 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.696 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.696 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.697 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.808817_291.1", 2022-09-03T05:46:59.697 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.698 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37", 2022-09-03T05:46:59.698 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.808817564Z", 2022-09-03T05:46:59.698 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.698 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.699 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.699 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.699 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.700 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_kuQ0xr50nHBW7YTCPQavDGz", 2022-09-03T05:46:59.700 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.700 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.701 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.701 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.701 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.702 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.809764_292.1", 2022-09-03T05:46:59.703 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.703 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-37", 2022-09-03T05:46:59.703 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.809764987Z", 2022-09-03T05:46:59.703 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.704 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.704 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.704 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.705 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.705 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_2HiJKFit31XaN5gBsoarIgZ", 2022-09-03T05:46:59.705 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.706 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.706 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.706 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.707 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.707 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.814054_293.1", 2022-09-03T05:46:59.708 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.708 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-37", 2022-09-03T05:46:59.708 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.814054902Z", 2022-09-03T05:46:59.708 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.709 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.709 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.709 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.710 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.710 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_2HiJKFit31XaN5gBsoarIgZ", 2022-09-03T05:46:59.711 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.712 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.712 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.712 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.713 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.714 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.846705_294.1", 2022-09-03T05:46:59.714 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.715 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38", 2022-09-03T05:46:59.715 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.846705503Z", 2022-09-03T05:46:59.715 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.716 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.716 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.716 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.717 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.717 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_h95HWYhtSYo3Z6W42rIP0YJ", 2022-09-03T05:46:59.717 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.718 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.718 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.719 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.719 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.720 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.850680_295.1", 2022-09-03T05:46:59.720 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.720 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38", 2022-09-03T05:46:59.721 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.850680201Z", 2022-09-03T05:46:59.721 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.721 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.721 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.722 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.722 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.723 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_h95HWYhtSYo3Z6W42rIP0YJ", 2022-09-03T05:46:59.723 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.723 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.724 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.724 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.724 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.725 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.725 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.851766_296.1", 2022-09-03T05:46:59.725 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.726 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-38", 2022-09-03T05:46:59.726 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.851766916Z", 2022-09-03T05:46:59.726 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.726 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.727 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.727 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.728 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.728 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_K5Yz9Djeac7y4a5O_0TviMm", 2022-09-03T05:46:59.728 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.729 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.729 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.729 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.729 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.730 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183852.856692_297.1", 2022-09-03T05:46:59.731 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.731 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-38", 2022-09-03T05:46:59.731 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:12.856692289Z", 2022-09-03T05:46:59.731 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.732 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.732 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.732 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.733 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.733 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_K5Yz9Djeac7y4a5O_0TviMm", 2022-09-03T05:46:59.733 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.734 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.734 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.734 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.734 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.735 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.003533_298.1", 2022-09-03T05:46:59.735 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.736 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-39:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.39", 2022-09-03T05:46:59.736 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.003533677Z", 2022-09-03T05:46:59.736 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.737 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.737 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.737 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.738 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.738 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_LjHtI20g0C-nIIB1xqrV4kd", 2022-09-03T05:46:59.738 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.739 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.739 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.740 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.740 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.741 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.006401_299.1", 2022-09-03T05:46:59.741 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.741 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-39:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.39", 2022-09-03T05:46:59.742 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.006401221Z", 2022-09-03T05:46:59.742 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.742 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.742 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.743 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.743 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.744 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_LjHtI20g0C-nIIB1xqrV4kd", 2022-09-03T05:46:59.744 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.744 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.745 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.745 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.745 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.746 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.007446_300.1", 2022-09-03T05:46:59.746 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.746 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-39", 2022-09-03T05:46:59.747 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.007446652Z", 2022-09-03T05:46:59.747 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.747 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.747 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.748 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.748 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.748 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_LR1OrAXSVx6okJ40JV0Uh8F", 2022-09-03T05:46:59.749 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.749 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.749 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.750 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.750 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.751 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.010018_301.1", 2022-09-03T05:46:59.751 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.751 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-39", 2022-09-03T05:46:59.751 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.010018357Z", 2022-09-03T05:46:59.752 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.752 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.753 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.753 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.753 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.754 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_LR1OrAXSVx6okJ40JV0Uh8F", 2022-09-03T05:46:59.754 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.755 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.755 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.755 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.756 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.757 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.039601_302.1", 2022-09-03T05:46:59.757 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.758 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-40:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.40", 2022-09-03T05:46:59.758 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.039601239Z", 2022-09-03T05:46:59.759 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.759 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.759 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.759 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.760 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.760 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_SEgVmZRRIdgycWQGmgKhmpv", 2022-09-03T05:46:59.761 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.761 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.762 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.762 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.762 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.763 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.048071_303.1", 2022-09-03T05:46:59.763 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.763 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-40:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.40", 2022-09-03T05:46:59.764 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.048071799Z", 2022-09-03T05:46:59.764 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.764 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.764 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.765 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.765 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.766 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_SEgVmZRRIdgycWQGmgKhmpv", 2022-09-03T05:46:59.766 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.766 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.767 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.767 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.767 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.768 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.049107_304.1", 2022-09-03T05:46:59.768 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.769 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-40", 2022-09-03T05:46:59.769 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.049107658Z", 2022-09-03T05:46:59.769 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.769 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.770 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.770 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.770 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.771 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_e0zQngowI1boXIeM6KNEWZw", 2022-09-03T05:46:59.771 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.771 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.772 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.772 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.772 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.773 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.054911_305.1", 2022-09-03T05:46:59.773 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.773 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-40", 2022-09-03T05:46:59.774 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.054911391Z", 2022-09-03T05:46:59.774 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.774 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.774 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.775 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.775 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.775 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_e0zQngowI1boXIeM6KNEWZw", 2022-09-03T05:46:59.776 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.776 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.776 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.777 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.777 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.778 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.099806_306.1", 2022-09-03T05:46:59.778 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.778 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-41:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.41", 2022-09-03T05:46:59.779 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.099806646Z", 2022-09-03T05:46:59.779 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.779 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.779 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.780 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.780 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.781 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_M2clmHIwHAIIB9oEOnpIKkh", 2022-09-03T05:46:59.781 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.782 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.782 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.782 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.783 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.783 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.111286_307.1", 2022-09-03T05:46:59.784 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.784 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-41:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.41", 2022-09-03T05:46:59.784 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.111286094Z", 2022-09-03T05:46:59.784 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.785 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.785 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.785 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.787 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.787 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_M2clmHIwHAIIB9oEOnpIKkh", 2022-09-03T05:46:59.788 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.788 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.789 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.789 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.790 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.791 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.113514_308.1", 2022-09-03T05:46:59.792 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.792 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-41", 2022-09-03T05:46:59.792 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.113514552Z", 2022-09-03T05:46:59.793 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.793 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.793 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.793 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.794 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.795 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_kKVUMELn0ub6gAm6SEjIkvZ", 2022-09-03T05:46:59.795 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.795 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.796 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.796 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.796 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.797 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183864.119538_309.1", 2022-09-03T05:46:59.797 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.798 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-41", 2022-09-03T05:46:59.799 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:24.119538060Z", 2022-09-03T05:46:59.799 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.799 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.800 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.801 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.801 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.801 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.802 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_kKVUMELn0ub6gAm6SEjIkvZ", 2022-09-03T05:46:59.802 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.803 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.804 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.804 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.804 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.806 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182414.943600_19.1", 2022-09-03T05:46:59.806 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.806 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-1:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.1", 2022-09-03T05:46:59.807 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:14.943600569Z", 2022-09-03T05:46:59.807 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.807 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.808 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.808 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.808 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.809 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_rq1mS4spn__TV-QX0MQQJUU", 2022-09-03T05:46:59.809 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.810 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.810 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.810 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.811 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.811 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.812 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182414.959460_20.1", 2022-09-03T05:46:59.812 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.812 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-1:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.1", 2022-09-03T05:46:59.812 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:14.959460360Z", 2022-09-03T05:46:59.812 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.813 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.813 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.813 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.814 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.814 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_rq1mS4spn__TV-QX0MQQJUU", 2022-09-03T05:46:59.814 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.815 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.815 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.815 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.816 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.817 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182414.961669_21.1", 2022-09-03T05:46:59.817 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.817 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-1", 2022-09-03T05:46:59.817 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:14.961669762Z", 2022-09-03T05:46:59.818 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.818 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.818 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.819 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.820 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.820 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_0bHoPHCFbjz75fM89k7kJ9J", 2022-09-03T05:46:59.820 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.821 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.821 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.821 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.821 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.822 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182414.972666_22.1", 2022-09-03T05:46:59.823 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.823 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-1", 2022-09-03T05:46:59.823 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:14.972666586Z", 2022-09-03T05:46:59.823 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.824 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.824 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.824 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.824 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.825 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_0bHoPHCFbjz75fM89k7kJ9J", 2022-09-03T05:46:59.825 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.825 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.826 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.826 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.826 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.827 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182415.003071_23.1", 2022-09-03T05:46:59.827 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.827 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-2:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.2", 2022-09-03T05:46:59.828 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:15.003071656Z", 2022-09-03T05:46:59.828 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.828 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.828 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.829 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.829 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.829 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.829 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tfkUve-GLYp4UeLNm-4NFHF", 2022-09-03T05:46:59.830 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.830 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.830 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.831 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.831 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.832 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182415.009391_24.1", 2022-09-03T05:46:59.832 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.832 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-2:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.2", 2022-09-03T05:46:59.832 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:15.009391132Z", 2022-09-03T05:46:59.833 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.833 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.833 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.833 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.834 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.834 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tfkUve-GLYp4UeLNm-4NFHF", 2022-09-03T05:46:59.834 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.835 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.835 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.835 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.836 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.836 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182415.013579_25.1", 2022-09-03T05:46:59.837 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.837 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-2", 2022-09-03T05:46:59.837 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:15.013579974Z", 2022-09-03T05:46:59.837 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.838 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.838 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.838 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.839 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.839 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_ach1wph5n7_8QKTUZF1Mary", 2022-09-03T05:46:59.839 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.840 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.840 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.840 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.840 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.841 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182415.025150_26.1", 2022-09-03T05:46:59.841 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.842 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-2", 2022-09-03T05:46:59.842 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:15.025150228Z", 2022-09-03T05:46:59.842 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.842 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.843 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.843 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.843 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.844 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_ach1wph5n7_8QKTUZF1Mary", 2022-09-03T05:46:59.844 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.844 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.845 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.845 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.845 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.846 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182417.307770_27.1", 2022-09-03T05:46:59.846 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.846 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-3:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.3", 2022-09-03T05:46:59.847 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:17.307770555Z", 2022-09-03T05:46:59.847 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.847 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.847 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.848 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.848 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.848 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_RKS6tzrI_dmvQpvUbloi7Q_", 2022-09-03T05:46:59.849 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.849 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.849 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.850 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.850 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.851 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182417.312081_28.1", 2022-09-03T05:46:59.851 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.851 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-3:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.3", 2022-09-03T05:46:59.851 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:17.312081255Z", 2022-09-03T05:46:59.852 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.852 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.852 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.852 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.853 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.853 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_RKS6tzrI_dmvQpvUbloi7Q_", 2022-09-03T05:46:59.854 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.854 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.854 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.855 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.855 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.856 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182417.313242_29.1", 2022-09-03T05:46:59.856 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.856 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-3", 2022-09-03T05:46:59.856 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:17.313242294Z", 2022-09-03T05:46:59.857 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.857 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.857 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.857 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.858 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.858 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lt8IAlx8xFKZXCBoGrP76MP", 2022-09-03T05:46:59.858 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.859 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.859 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.859 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.860 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.860 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182417.319609_30.1", 2022-09-03T05:46:59.861 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.861 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-3", 2022-09-03T05:46:59.861 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:17.319609012Z", 2022-09-03T05:46:59.861 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.861 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.862 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.862 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.862 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.863 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lt8IAlx8xFKZXCBoGrP76MP", 2022-09-03T05:46:59.863 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.863 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.864 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.864 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.864 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.865 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.152440_31.1", 2022-09-03T05:46:59.865 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.865 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-4:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.4", 2022-09-03T05:46:59.866 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.152440893Z", 2022-09-03T05:46:59.866 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.866 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.866 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.867 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.867 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.867 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_QMyoKwlK2G2NuxL_sp2_re5", 2022-09-03T05:46:59.868 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.868 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.868 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.869 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.869 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.870 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.158442_32.1", 2022-09-03T05:46:59.870 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.870 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-4:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.4", 2022-09-03T05:46:59.871 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.158442960Z", 2022-09-03T05:46:59.871 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.871 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.871 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.872 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.872 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.872 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_QMyoKwlK2G2NuxL_sp2_re5", 2022-09-03T05:46:59.873 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.873 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.873 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.874 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.874 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.874 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.874 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.875 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.159309_33.1", 2022-09-03T05:46:59.875 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.875 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-4", 2022-09-03T05:46:59.875 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.159309677Z", 2022-09-03T05:46:59.876 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.876 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.876 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.876 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.877 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.877 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_FxyCJRWlBEhmncaMF3-AMFD", 2022-09-03T05:46:59.877 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.878 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.878 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.878 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.879 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.879 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.879 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.163606_34.1", 2022-09-03T05:46:59.880 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.880 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-4", 2022-09-03T05:46:59.880 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.163606904Z", 2022-09-03T05:46:59.880 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.881 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.881 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.881 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.881 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.882 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.882 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_FxyCJRWlBEhmncaMF3-AMFD", 2022-09-03T05:46:59.882 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.883 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.883 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.883 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.883 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.884 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.884 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.884 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.181237_35.1", 2022-09-03T05:46:59.884 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.885 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-5:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.5", 2022-09-03T05:46:59.885 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.181237915Z", 2022-09-03T05:46:59.885 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.885 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.886 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.886 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.886 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.887 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.887 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_mye4oDj8XK7uSi7YUS75P0l", 2022-09-03T05:46:59.887 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.887 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.888 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.888 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.888 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.889 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.183507_36.1", 2022-09-03T05:46:59.889 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.889 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-5:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.5", 2022-09-03T05:46:59.890 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.183507081Z", 2022-09-03T05:46:59.890 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.890 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.890 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.891 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.891 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.891 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_mye4oDj8XK7uSi7YUS75P0l", 2022-09-03T05:46:59.892 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.892 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.892 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.893 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.893 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.893 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.893 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.894 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.894 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.184425_37.1", 2022-09-03T05:46:59.894 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.894 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-5", 2022-09-03T05:46:59.895 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.184425827Z", 2022-09-03T05:46:59.895 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.895 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.895 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.896 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.896 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.896 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.896 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_6QXcvNroiwRkEdVqi3Nf85-", 2022-09-03T05:46:59.897 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.897 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.897 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.898 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.898 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.898 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.898 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.899 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.186822_38.1", 2022-09-03T05:46:59.899 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.899 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-5", 2022-09-03T05:46:59.899 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.186822428Z", 2022-09-03T05:46:59.900 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.900 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.900 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.900 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.901 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.901 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_6QXcvNroiwRkEdVqi3Nf85-", 2022-09-03T05:46:59.901 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.902 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.902 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.902 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.902 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.903 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.903 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.903 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.221675_39.1", 2022-09-03T05:46:59.904 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.904 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-6:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.6", 2022-09-03T05:46:59.904 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.221675326Z", 2022-09-03T05:46:59.904 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.904 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.905 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.905 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.906 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.906 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bqgF7AreaM1kO2cBL_SkGsM", 2022-09-03T05:46:59.906 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.906 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.907 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.907 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.907 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.907 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.908 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.230423_40.1", 2022-09-03T05:46:59.908 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.909 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-6:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.6", 2022-09-03T05:46:59.909 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.230423417Z", 2022-09-03T05:46:59.909 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.909 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.910 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.910 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.910 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.911 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bqgF7AreaM1kO2cBL_SkGsM", 2022-09-03T05:46:59.911 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.911 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.911 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.912 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.912 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.912 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.913 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.235081_41.1", 2022-09-03T05:46:59.913 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.913 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-6", 2022-09-03T05:46:59.913 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.235081816Z", 2022-09-03T05:46:59.914 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.914 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.914 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.915 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.915 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.915 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_wbkk3MMRONgcegk23g-QDT1", 2022-09-03T05:46:59.916 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.916 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.916 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.917 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.917 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.917 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.917 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182423.238098_42.1", 2022-09-03T05:46:59.918 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.918 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-6", 2022-09-03T05:46:59.918 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:23.238098524Z", 2022-09-03T05:46:59.919 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.919 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.919 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.919 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.920 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.920 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_wbkk3MMRONgcegk23g-QDT1", 2022-09-03T05:46:59.920 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.921 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.921 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.921 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.922 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.922 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.064697_43.1", 2022-09-03T05:46:59.923 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.923 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7", 2022-09-03T05:46:59.923 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.064697048Z", 2022-09-03T05:46:59.923 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.923 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.924 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.924 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.924 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.925 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:59.925 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.926 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.926 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.926 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.926 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.927 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.927 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.070346_44.1", 2022-09-03T05:46:59.927 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.927 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7", 2022-09-03T05:46:59.928 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.070346892Z", 2022-09-03T05:46:59.928 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.928 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.928 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.929 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.929 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.930 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:59.930 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.930 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.931 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.931 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.932 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.932 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.933 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.934 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.071355_45.1", 2022-09-03T05:46:59.934 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.934 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-7", 2022-09-03T05:46:59.935 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.071355932Z", 2022-09-03T05:46:59.936 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.936 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.936 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.936 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.937 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.937 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:46:59.937 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.938 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.938 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.938 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.938 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.939 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.939 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.939 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.075532_46.1", 2022-09-03T05:46:59.940 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.940 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-7", 2022-09-03T05:46:59.940 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.075532031Z", 2022-09-03T05:46:59.940 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.941 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.941 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.941 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.942 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.942 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:46:59.942 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.943 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.943 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.943 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.944 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.945 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.945 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.102267_47.1", 2022-09-03T05:46:59.945 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.946 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-8:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.8", 2022-09-03T05:46:59.946 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.102267896Z", 2022-09-03T05:46:59.946 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.947 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.947 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.947 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.948 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.948 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:46:59.948 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.949 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.949 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.949 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.950 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.950 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.950 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.950 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.104611_48.1", 2022-09-03T05:46:59.951 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.951 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-8:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.8", 2022-09-03T05:46:59.951 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.104611563Z", 2022-09-03T05:46:59.952 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.952 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.952 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.952 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.953 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.953 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:46:59.954 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.954 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.954 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.955 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.955 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.955 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.956 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.105641_49.1", 2022-09-03T05:46:59.956 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.956 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-8", 2022-09-03T05:46:59.956 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.105641481Z", 2022-09-03T05:46:59.957 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.957 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.957 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.957 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.958 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.958 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:46:59.959 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.959 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.959 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.959 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.960 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.960 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.961 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.108076_50.1", 2022-09-03T05:46:59.961 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.961 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-8", 2022-09-03T05:46:59.962 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.108076436Z", 2022-09-03T05:46:59.962 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.962 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.962 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.963 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.963 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.963 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:46:59.963 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.964 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.964 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.964 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.965 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.965 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.966 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.135979_51.1", 2022-09-03T05:46:59.966 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.966 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9", 2022-09-03T05:46:59.966 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.135979206Z", 2022-09-03T05:46:59.967 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.967 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.967 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.967 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.968 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.968 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.968 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:46:59.968 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.969 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.969 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.969 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.969 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.970 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.970 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.970 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.970 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.138975_52.1", 2022-09-03T05:46:59.971 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.971 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9", 2022-09-03T05:46:59.971 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.138975472Z", 2022-09-03T05:46:59.972 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.972 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.972 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.972 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.973 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.973 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.973 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:46:59.973 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.974 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.974 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.974 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.975 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.975 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.975 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.975 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.976 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.139759_53.1", 2022-09-03T05:46:59.976 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.976 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-9", 2022-09-03T05:46:59.976 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.139759635Z", 2022-09-03T05:46:59.977 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.977 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.977 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.978 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.978 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.978 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.978 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:46:59.979 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.979 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.979 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.979 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.980 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.980 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.980 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.980 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.980 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.141916_54.1", 2022-09-03T05:46:59.981 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.981 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-9", 2022-09-03T05:46:59.981 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.141916666Z", 2022-09-03T05:46:59.981 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.982 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.982 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:46:59.982 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:46:59.982 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.983 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.983 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:46:59.983 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.983 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.984 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.984 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.984 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.984 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.985 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.985 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.985 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.782802_55.1", 2022-09-03T05:46:59.985 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.986 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7", 2022-09-03T05:46:59.986 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.782802362Z", 2022-09-03T05:46:59.986 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.986 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.987 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:59.987 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.987 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.987 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.988 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:59.988 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.988 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.988 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.989 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:46:59.989 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.989 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.989 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.990 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.990 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.785254_56.1", 2022-09-03T05:46:59.990 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:46:59.990 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7", 2022-09-03T05:46:59.991 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.785254883Z", 2022-09-03T05:46:59.991 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.991 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.991 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:59.992 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.992 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.992 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.992 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tI7eALWGgDKIg7vIK3UWR0Q", 2022-09-03T05:46:59.992 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.993 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.993 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.993 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:46:59.993 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.994 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.994 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.994 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.994 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.796744_57.1", 2022-09-03T05:46:59.995 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:46:59.995 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-7", 2022-09-03T05:46:59.995 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.796744879Z", 2022-09-03T05:46:59.995 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:46:59.996 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:46:59.996 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:46:59.996 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:46:59.996 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.997 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:46:59.997 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:46:59.997 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:46:59.997 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.998 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:46:59.998 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "remove" 2022-09-03T05:46:59.998 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.998 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:46:59.999 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:46:59.999 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:46:59.999 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.798860_58.1", 2022-09-03T05:46:59.999 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:47:00.000 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-7", 2022-09-03T05:47:00.000 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.798860427Z", 2022-09-03T05:47:00.000 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.000 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.001 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:47:00.001 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.001 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.001 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.002 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bAKdvTIbWi3ePmiOiObCQo5", 2022-09-03T05:47:00.002 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:47:00.002 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.002 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.003 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:47:00.003 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.003 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.003 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.004 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.004 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.833037_59.1", 2022-09-03T05:47:00.004 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.004 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-8:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.8", 2022-09-03T05:47:00.005 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.833037165Z", 2022-09-03T05:47:00.005 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.005 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.005 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:47:00.006 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.006 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.006 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.006 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:47:00.007 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:47:00.007 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.007 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.007 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:47:00.008 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.008 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.008 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.008 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.009 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.835414_60.1", 2022-09-03T05:47:00.009 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.009 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-8:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.8", 2022-09-03T05:47:00.009 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.835414206Z", 2022-09-03T05:47:00.010 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.010 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.010 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:47:00.010 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.011 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.011 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_JurNxobJiI5jDo98Rg06Lsb", 2022-09-03T05:47:00.011 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:47:00.012 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.012 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.012 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:47:00.012 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.013 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.013 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.013 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.013 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.843078_61.1", 2022-09-03T05:47:00.013 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:47:00.014 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-8", 2022-09-03T05:47:00.014 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.843078156Z", 2022-09-03T05:47:00.014 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.015 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.015 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:47:00.015 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.015 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.016 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:47:00.016 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:47:00.016 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.016 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.017 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "remove" 2022-09-03T05:47:00.017 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.017 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.017 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.018 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.018 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.845207_62.1", 2022-09-03T05:47:00.018 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:47:00.018 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-8", 2022-09-03T05:47:00.019 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.845207527Z", 2022-09-03T05:47:00.019 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.019 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.019 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:47:00.020 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.020 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.020 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.020 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_dXmwlERX5cbA3C34O9GQAyg", 2022-09-03T05:47:00.021 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:47:00.021 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.021 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.021 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:47:00.021 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.022 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.022 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.022 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.023 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.909335_63.1", 2022-09-03T05:47:00.023 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.023 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9", 2022-09-03T05:47:00.023 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.909335713Z", 2022-09-03T05:47:00.024 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.024 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.024 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:47:00.024 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.024 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.025 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.025 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:47:00.025 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:47:00.025 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.026 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.026 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:47:00.026 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.026 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.027 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.027 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.027 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.911665_64.1", 2022-09-03T05:47:00.027 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.028 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9", 2022-09-03T05:47:00.028 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.911665732Z", 2022-09-03T05:47:00.028 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.028 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.029 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:47:00.029 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.029 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.029 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.030 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_GFvK96qpBedSgijlKpSEZoP", 2022-09-03T05:47:00.030 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:47:00.030 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.030 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.031 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:47:00.031 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.031 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.031 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.032 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.032 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.924885_65.1", 2022-09-03T05:47:00.032 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:47:00.032 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-9", 2022-09-03T05:47:00.033 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.924885371Z", 2022-09-03T05:47:00.033 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.033 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.033 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:47:00.034 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.034 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.034 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.034 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:47:00.035 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:47:00.035 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.035 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.035 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "remove" 2022-09-03T05:47:00.035 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.037 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.037 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.038 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.038 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662182430.928498_66.1", 2022-09-03T05:47:00.038 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:47:00.038 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-9", 2022-09-03T05:47:00.039 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498747Z", 2022-09-03T05:47:00.039 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.039 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.039 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:47:00.040 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.040 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.040 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.040 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_uADAaXa-gF0iuYQjTffnbkD", 2022-09-03T05:47:00.041 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 2 2022-09-03T05:47:00.041 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.042 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.042 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:47:00.042 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.042 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.043 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.043 INFO:teuthology.orchestra.run.smithi189.stdout:] 2022-09-03T05:47:00.045 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', 'eff2cb82-c479-4631-8568-fb1af4159e02', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:00.046 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', 'eff2cb82-c479-4631-8568-fb1af4159e02', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:00.046 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period eff2cb82-c479-4631-8568-fb1af4159e02 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:00.102 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:00.103 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:00.134 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:47:00.135 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.135 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183936.297409_114.1", 2022-09-03T05:47:00.135 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.135 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-42:e40d88ba-d542-483a-afa2-5de687affd14.5176.1", 2022-09-03T05:47:00.136 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:45:36.297409375Z", 2022-09-03T05:47:00.136 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.136 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.137 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.137 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.137 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.137 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.138 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YpkDw8rXRiU8c33B9zgKfyS", 2022-09-03T05:47:00.138 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.138 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.138 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.139 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:47:00.139 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.139 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.139 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.140 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.140 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183936.303342_115.1", 2022-09-03T05:47:00.140 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.141 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-42:e40d88ba-d542-483a-afa2-5de687affd14.5176.1", 2022-09-03T05:47:00.141 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:45:36.303342447Z", 2022-09-03T05:47:00.141 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.141 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.142 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.142 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.142 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.142 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.143 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YpkDw8rXRiU8c33B9zgKfyS", 2022-09-03T05:47:00.143 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.143 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.143 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.144 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:47:00.144 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.144 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.145 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.145 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.145 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183936.304603_116.1", 2022-09-03T05:47:00.145 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:47:00.146 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-42", 2022-09-03T05:47:00.146 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:45:36.304603439Z", 2022-09-03T05:47:00.146 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.146 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.147 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.147 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.147 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.148 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.148 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c67Vzh50iaz8PYakT72oJpl", 2022-09-03T05:47:00.148 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.148 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.149 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.149 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:47:00.149 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.149 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.150 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.150 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.150 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183936.307472_117.1", 2022-09-03T05:47:00.151 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:47:00.151 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-42", 2022-09-03T05:47:00.151 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:45:36.307472912Z", 2022-09-03T05:47:00.151 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.152 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.152 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.152 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.152 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.153 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.153 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c67Vzh50iaz8PYakT72oJpl", 2022-09-03T05:47:00.153 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.153 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.154 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.154 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:47:00.154 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.155 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.155 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.155 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.155 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183936.340432_118.1", 2022-09-03T05:47:00.156 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.156 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-43:e40d88ba-d542-483a-afa2-5de687affd14.5176.2", 2022-09-03T05:47:00.156 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:45:36.340432490Z", 2022-09-03T05:47:00.156 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.157 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.157 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.157 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.157 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.158 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.158 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_TmtQ_Dev6v-22p-DZXqByqp", 2022-09-03T05:47:00.158 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.158 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.159 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.159 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:47:00.159 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.160 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.160 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.160 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.160 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183936.343562_119.1", 2022-09-03T05:47:00.161 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.161 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-43:e40d88ba-d542-483a-afa2-5de687affd14.5176.2", 2022-09-03T05:47:00.161 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:45:36.343562777Z", 2022-09-03T05:47:00.162 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.162 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.162 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.162 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.163 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.163 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.163 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_TmtQ_Dev6v-22p-DZXqByqp", 2022-09-03T05:47:00.164 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.164 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.164 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.164 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:47:00.165 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.165 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.165 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.165 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.166 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183936.344701_120.1", 2022-09-03T05:47:00.166 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:47:00.166 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-43", 2022-09-03T05:47:00.166 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:45:36.344701522Z", 2022-09-03T05:47:00.167 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.167 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.167 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.167 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.168 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.168 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.168 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_JDy3Y2wOnTXr7UPMqBLvQwE", 2022-09-03T05:47:00.168 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.169 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.169 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.169 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:47:00.169 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.170 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.170 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.170 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.171 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183936.347465_121.1", 2022-09-03T05:47:00.171 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:47:00.171 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-43", 2022-09-03T05:47:00.171 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:45:36.347465295Z", 2022-09-03T05:47:00.172 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.172 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.172 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.172 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.173 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.173 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.173 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_JDy3Y2wOnTXr7UPMqBLvQwE", 2022-09-03T05:47:00.173 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.174 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.174 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.175 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:47:00.176 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.176 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.176 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.176 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:47:00.177 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', 'eff2cb82-c479-4631-8568-fb1af4159e02', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:00.178 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', 'eff2cb82-c479-4631-8568-fb1af4159e02', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:00.178 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period eff2cb82-c479-4631-8568-fb1af4159e02 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:00.235 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:00.236 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:00.275 INFO:teuthology.orchestra.run.smithi189.stdout:[ 2022-09-03T05:47:00.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.276 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183936.297409_114.1", 2022-09-03T05:47:00.276 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.277 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-42:e40d88ba-d542-483a-afa2-5de687affd14.5176.1", 2022-09-03T05:47:00.277 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:45:36.297409375Z", 2022-09-03T05:47:00.277 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.278 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.279 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.279 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.280 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.281 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YpkDw8rXRiU8c33B9zgKfyS", 2022-09-03T05:47:00.281 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.282 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.282 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:47:00.283 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.283 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.285 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183936.303342_115.1", 2022-09-03T05:47:00.285 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.285 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-42:e40d88ba-d542-483a-afa2-5de687affd14.5176.1", 2022-09-03T05:47:00.286 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:45:36.303342447Z", 2022-09-03T05:47:00.286 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.287 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.287 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.288 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.289 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.289 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YpkDw8rXRiU8c33B9zgKfyS", 2022-09-03T05:47:00.290 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.290 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.290 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:47:00.291 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.291 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.292 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183936.304603_116.1", 2022-09-03T05:47:00.292 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:47:00.292 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-42", 2022-09-03T05:47:00.293 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:45:36.304603439Z", 2022-09-03T05:47:00.293 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.293 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.293 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.294 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.294 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.294 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c67Vzh50iaz8PYakT72oJpl", 2022-09-03T05:47:00.295 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.295 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.295 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:47:00.296 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.296 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.297 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183936.307472_117.1", 2022-09-03T05:47:00.297 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:47:00.297 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-42", 2022-09-03T05:47:00.297 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:45:36.307472912Z", 2022-09-03T05:47:00.298 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.298 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.298 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.298 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.299 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.299 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c67Vzh50iaz8PYakT72oJpl", 2022-09-03T05:47:00.299 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.300 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.300 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:47:00.300 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.301 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.301 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183936.340432_118.1", 2022-09-03T05:47:00.302 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.302 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-43:e40d88ba-d542-483a-afa2-5de687affd14.5176.2", 2022-09-03T05:47:00.302 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:45:36.340432490Z", 2022-09-03T05:47:00.302 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.303 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.303 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.303 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.304 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.304 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_TmtQ_Dev6v-22p-DZXqByqp", 2022-09-03T05:47:00.304 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.305 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.305 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:47:00.305 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.305 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.306 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183936.343562_119.1", 2022-09-03T05:47:00.306 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.307 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-43:e40d88ba-d542-483a-afa2-5de687affd14.5176.2", 2022-09-03T05:47:00.307 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:45:36.343562777Z", 2022-09-03T05:47:00.307 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.307 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.308 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.308 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.308 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.309 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_TmtQ_Dev6v-22p-DZXqByqp", 2022-09-03T05:47:00.309 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.309 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.310 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:47:00.310 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.310 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.311 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.311 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183936.344701_120.1", 2022-09-03T05:47:00.311 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:47:00.311 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-43", 2022-09-03T05:47:00.312 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:45:36.344701522Z", 2022-09-03T05:47:00.312 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.312 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.313 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.313 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.313 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.314 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_JDy3Y2wOnTXr7UPMqBLvQwE", 2022-09-03T05:47:00.314 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.314 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.315 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:47:00.315 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.315 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.316 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.316 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183936.347465_121.1", 2022-09-03T05:47:00.316 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:47:00.316 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-43", 2022-09-03T05:47:00.317 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:45:36.347465295Z", 2022-09-03T05:47:00.317 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.317 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.317 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.318 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.318 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.318 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_JDy3Y2wOnTXr7UPMqBLvQwE", 2022-09-03T05:47:00.319 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.319 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.321 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:47:00.321 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.321 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.321 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.322 INFO:teuthology.orchestra.run.smithi189.stdout:] 2022-09-03T05:47:00.322 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', 'eff2cb82-c479-4631-8568-fb1af4159e02', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:00.323 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', 'eff2cb82-c479-4631-8568-fb1af4159e02', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:00.323 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period eff2cb82-c479-4631-8568-fb1af4159e02 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:00.372 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:00.373 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:00.423 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:47:00.424 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.424 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183936.297409_114.1", 2022-09-03T05:47:00.424 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.425 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-42:e40d88ba-d542-483a-afa2-5de687affd14.5176.1", 2022-09-03T05:47:00.425 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:45:36.297409375Z", 2022-09-03T05:47:00.426 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.426 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.426 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.427 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.427 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.427 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.428 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YpkDw8rXRiU8c33B9zgKfyS", 2022-09-03T05:47:00.428 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.428 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.429 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.429 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:47:00.429 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.429 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.430 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.430 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.430 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183936.303342_115.1", 2022-09-03T05:47:00.431 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.431 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-42:e40d88ba-d542-483a-afa2-5de687affd14.5176.1", 2022-09-03T05:47:00.431 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:45:36.303342447Z", 2022-09-03T05:47:00.432 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.432 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.432 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.433 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.433 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.433 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.433 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_YpkDw8rXRiU8c33B9zgKfyS", 2022-09-03T05:47:00.434 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.434 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.435 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:47:00.435 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.435 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.435 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.436 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.436 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183936.304603_116.1", 2022-09-03T05:47:00.436 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:47:00.436 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-42", 2022-09-03T05:47:00.437 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:45:36.304603439Z", 2022-09-03T05:47:00.437 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.437 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.438 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.438 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.438 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.439 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.439 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c67Vzh50iaz8PYakT72oJpl", 2022-09-03T05:47:00.439 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.440 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.440 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.440 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:47:00.441 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.441 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.441 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.442 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.442 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183936.307472_117.1", 2022-09-03T05:47:00.443 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:47:00.443 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-42", 2022-09-03T05:47:00.443 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:45:36.307472912Z", 2022-09-03T05:47:00.443 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.444 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.444 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.444 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.445 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.445 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.445 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_c67Vzh50iaz8PYakT72oJpl", 2022-09-03T05:47:00.445 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.446 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.446 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.446 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:47:00.447 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.447 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.447 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.447 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.448 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183936.340432_118.1", 2022-09-03T05:47:00.448 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.448 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-43:e40d88ba-d542-483a-afa2-5de687affd14.5176.2", 2022-09-03T05:47:00.448 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:45:36.340432490Z", 2022-09-03T05:47:00.449 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.449 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.449 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.449 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.450 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.450 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.450 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_TmtQ_Dev6v-22p-DZXqByqp", 2022-09-03T05:47:00.451 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.451 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.451 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.451 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:47:00.452 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.452 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.452 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.452 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.453 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183936.343562_119.1", 2022-09-03T05:47:00.453 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.453 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-43:e40d88ba-d542-483a-afa2-5de687affd14.5176.2", 2022-09-03T05:47:00.453 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:45:36.343562777Z", 2022-09-03T05:47:00.454 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.454 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.454 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.455 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.455 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.455 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.455 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_TmtQ_Dev6v-22p-DZXqByqp", 2022-09-03T05:47:00.456 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.456 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.456 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.456 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:47:00.457 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.457 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.457 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.457 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.458 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183936.344701_120.1", 2022-09-03T05:47:00.458 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:47:00.458 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-43", 2022-09-03T05:47:00.459 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:45:36.344701522Z", 2022-09-03T05:47:00.459 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.459 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.459 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.460 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.460 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.461 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.461 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_JDy3Y2wOnTXr7UPMqBLvQwE", 2022-09-03T05:47:00.461 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.461 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.462 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.462 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:47:00.462 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.463 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.463 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.463 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.464 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183936.347465_121.1", 2022-09-03T05:47:00.464 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:47:00.464 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-43", 2022-09-03T05:47:00.464 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:45:36.347465295Z", 2022-09-03T05:47:00.465 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.465 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.465 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.465 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.466 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.466 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.466 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_JDy3Y2wOnTXr7UPMqBLvQwE", 2022-09-03T05:47:00.467 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.468 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.468 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.470 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:47:00.470 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.470 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.471 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.471 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:47:00.472 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', 'eff2cb82-c479-4631-8568-fb1af4159e02', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:00.472 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', 'eff2cb82-c479-4631-8568-fb1af4159e02', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:00.473 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period eff2cb82-c479-4631-8568-fb1af4159e02 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:00.524 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:00.524 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:00.565 INFO:teuthology.orchestra.run.smithi189.stdout:[ 2022-09-03T05:47:00.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.566 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183936.297409_114.1", 2022-09-03T05:47:00.566 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.567 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-42:e40d88ba-d542-483a-afa2-5de687affd14.5176.1", 2022-09-03T05:47:00.567 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:45:36.297409375Z", 2022-09-03T05:47:00.567 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.568 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.568 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.568 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.569 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.569 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YpkDw8rXRiU8c33B9zgKfyS", 2022-09-03T05:47:00.569 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.570 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.570 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:47:00.570 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.571 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.571 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183936.303342_115.1", 2022-09-03T05:47:00.572 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.572 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-42:e40d88ba-d542-483a-afa2-5de687affd14.5176.1", 2022-09-03T05:47:00.572 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:45:36.303342447Z", 2022-09-03T05:47:00.572 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.573 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.573 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.573 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.574 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.574 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_YpkDw8rXRiU8c33B9zgKfyS", 2022-09-03T05:47:00.575 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.575 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.575 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:47:00.576 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.576 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.577 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183936.304603_116.1", 2022-09-03T05:47:00.577 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:47:00.577 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-42", 2022-09-03T05:47:00.577 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:45:36.304603439Z", 2022-09-03T05:47:00.578 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.578 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.578 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.578 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.579 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.579 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c67Vzh50iaz8PYakT72oJpl", 2022-09-03T05:47:00.580 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.580 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.580 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:47:00.581 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.581 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.582 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183936.307472_117.1", 2022-09-03T05:47:00.582 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:47:00.582 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-42", 2022-09-03T05:47:00.582 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:45:36.307472912Z", 2022-09-03T05:47:00.585 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.585 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.586 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.586 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.587 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.588 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_c67Vzh50iaz8PYakT72oJpl", 2022-09-03T05:47:00.588 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.589 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.589 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:47:00.590 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.590 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.592 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183936.340432_118.1", 2022-09-03T05:47:00.592 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.593 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-43:e40d88ba-d542-483a-afa2-5de687affd14.5176.2", 2022-09-03T05:47:00.594 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:45:36.340432490Z", 2022-09-03T05:47:00.594 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.594 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.594 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.595 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.596 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.596 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_TmtQ_Dev6v-22p-DZXqByqp", 2022-09-03T05:47:00.596 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.597 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.597 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:47:00.598 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.599 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.599 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.600 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183936.343562_119.1", 2022-09-03T05:47:00.600 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.601 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-43:e40d88ba-d542-483a-afa2-5de687affd14.5176.2", 2022-09-03T05:47:00.601 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:45:36.343562777Z", 2022-09-03T05:47:00.601 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.602 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.602 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.602 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.603 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.604 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_TmtQ_Dev6v-22p-DZXqByqp", 2022-09-03T05:47:00.604 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.611 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.612 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.612 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:47:00.613 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.613 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.615 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183936.344701_120.1", 2022-09-03T05:47:00.615 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:47:00.616 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-43", 2022-09-03T05:47:00.616 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:45:36.344701522Z", 2022-09-03T05:47:00.617 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.617 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.618 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.618 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.619 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.620 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_JDy3Y2wOnTXr7UPMqBLvQwE", 2022-09-03T05:47:00.621 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.622 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.622 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:47:00.623 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.623 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.625 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183936.347465_121.1", 2022-09-03T05:47:00.625 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:47:00.625 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-43", 2022-09-03T05:47:00.626 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:45:36.347465295Z", 2022-09-03T05:47:00.626 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.627 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.627 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.628 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.628 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.629 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_JDy3Y2wOnTXr7UPMqBLvQwE", 2022-09-03T05:47:00.629 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.629 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.631 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:47:00.631 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.632 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.632 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.632 INFO:teuthology.orchestra.run.smithi189.stdout:] 2022-09-03T05:47:00.633 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '229c309b-f405-4ead-a3a8-baa670c6da51', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:00.633 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '229c309b-f405-4ead-a3a8-baa670c6da51', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:00.634 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 229c309b-f405-4ead-a3a8-baa670c6da51 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:00.683 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:00.684 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:00.725 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:47:00.726 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.726 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183996.876451_156.1", 2022-09-03T05:47:00.726 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.726 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-44:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.1", 2022-09-03T05:47:00.727 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:46:36.876451430Z", 2022-09-03T05:47:00.727 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.727 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.728 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.728 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.728 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.729 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.730 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Ooi5Pr6jAe_eKSnUbc1yhF-", 2022-09-03T05:47:00.730 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.730 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.730 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.731 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:47:00.731 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.731 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.732 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.732 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.732 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183996.882242_157.1", 2022-09-03T05:47:00.732 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.733 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-44:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.1", 2022-09-03T05:47:00.733 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:46:36.882242917Z", 2022-09-03T05:47:00.733 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.734 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.734 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.734 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.734 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.735 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.735 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Ooi5Pr6jAe_eKSnUbc1yhF-", 2022-09-03T05:47:00.735 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.735 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.736 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.736 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:47:00.736 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.736 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.737 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.737 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.737 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183996.883960_158.1", 2022-09-03T05:47:00.737 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:47:00.738 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-44", 2022-09-03T05:47:00.738 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:46:36.883960556Z", 2022-09-03T05:47:00.738 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.739 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.739 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.739 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.739 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.742 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.742 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_g7CXLiNivbLFArxtsjLR0Ea", 2022-09-03T05:47:00.742 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.743 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.743 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.743 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:47:00.743 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.744 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.744 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.744 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.744 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183996.887504_159.1", 2022-09-03T05:47:00.745 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:47:00.745 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-44", 2022-09-03T05:47:00.745 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:46:36.887504896Z", 2022-09-03T05:47:00.746 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.746 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.746 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.746 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.747 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.747 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.747 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_g7CXLiNivbLFArxtsjLR0Ea", 2022-09-03T05:47:00.747 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.748 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.748 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.748 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:47:00.749 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.749 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.749 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.749 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.750 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183996.919568_160.1", 2022-09-03T05:47:00.750 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.750 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-45:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.2", 2022-09-03T05:47:00.751 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:46:36.919568824Z", 2022-09-03T05:47:00.751 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.751 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.751 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.752 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.752 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.752 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.752 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tcKgEcOqLm4xxNq4RqPZPJv", 2022-09-03T05:47:00.753 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.753 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.753 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.753 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:47:00.754 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.754 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.754 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.754 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.755 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183996.924478_161.1", 2022-09-03T05:47:00.755 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.755 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-45:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.2", 2022-09-03T05:47:00.755 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:46:36.924478795Z", 2022-09-03T05:47:00.756 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.756 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.756 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.756 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.757 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.757 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.757 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tcKgEcOqLm4xxNq4RqPZPJv", 2022-09-03T05:47:00.758 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.758 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.758 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.758 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:47:00.759 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.759 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.759 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.759 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.760 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183996.925634_162.1", 2022-09-03T05:47:00.760 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:47:00.760 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-45", 2022-09-03T05:47:00.760 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:46:36.925634413Z", 2022-09-03T05:47:00.761 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.761 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.761 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.761 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.762 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.762 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.762 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_riKRPK3Rsd-5Q4VB2YVG1gO", 2022-09-03T05:47:00.762 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.763 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.763 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.763 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:47:00.764 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.764 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.764 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.765 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.765 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183996.928852_163.1", 2022-09-03T05:47:00.765 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:47:00.765 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-45", 2022-09-03T05:47:00.766 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:46:36.928852209Z", 2022-09-03T05:47:00.766 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.766 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:00.766 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:00.767 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:00.767 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.767 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:00.767 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_riKRPK3Rsd-5Q4VB2YVG1gO", 2022-09-03T05:47:00.768 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:00.768 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:00.768 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:00.770 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:47:00.770 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.770 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.771 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:00.771 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:47:00.771 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '229c309b-f405-4ead-a3a8-baa670c6da51', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:00.772 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '229c309b-f405-4ead-a3a8-baa670c6da51', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:00.773 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 229c309b-f405-4ead-a3a8-baa670c6da51 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:00.821 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:00.821 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:00.849 INFO:teuthology.orchestra.run.smithi189.stdout:[ 2022-09-03T05:47:00.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.850 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183996.876451_156.1", 2022-09-03T05:47:00.850 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.850 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-44:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.1", 2022-09-03T05:47:00.851 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:46:36.876451430Z", 2022-09-03T05:47:00.851 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.852 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.852 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.852 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.853 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.853 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Ooi5Pr6jAe_eKSnUbc1yhF-", 2022-09-03T05:47:00.854 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.854 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.855 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:47:00.855 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.855 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.856 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183996.882242_157.1", 2022-09-03T05:47:00.857 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.857 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-44:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.1", 2022-09-03T05:47:00.857 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:46:36.882242917Z", 2022-09-03T05:47:00.858 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.858 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.859 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.859 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.860 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.861 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Ooi5Pr6jAe_eKSnUbc1yhF-", 2022-09-03T05:47:00.861 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.862 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.863 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:47:00.863 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.863 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.864 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183996.883960_158.1", 2022-09-03T05:47:00.865 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:47:00.865 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-44", 2022-09-03T05:47:00.866 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:46:36.883960556Z", 2022-09-03T05:47:00.866 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.866 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.866 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.867 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.868 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.868 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_g7CXLiNivbLFArxtsjLR0Ea", 2022-09-03T05:47:00.869 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.870 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.870 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:47:00.870 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.871 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.872 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183996.887504_159.1", 2022-09-03T05:47:00.872 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:47:00.873 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-44", 2022-09-03T05:47:00.873 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:46:36.887504896Z", 2022-09-03T05:47:00.873 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.874 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.874 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.874 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.874 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.875 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.875 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_g7CXLiNivbLFArxtsjLR0Ea", 2022-09-03T05:47:00.875 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.876 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.876 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.876 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:47:00.877 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.877 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.877 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.878 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183996.919568_160.1", 2022-09-03T05:47:00.878 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.878 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-45:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.2", 2022-09-03T05:47:00.879 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:46:36.919568824Z", 2022-09-03T05:47:00.879 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.880 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.880 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.880 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.881 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.881 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.882 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tcKgEcOqLm4xxNq4RqPZPJv", 2022-09-03T05:47:00.882 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.883 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.883 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.883 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:47:00.884 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.884 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.886 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183996.924478_161.1", 2022-09-03T05:47:00.886 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.887 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-45:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.2", 2022-09-03T05:47:00.887 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:46:36.924478795Z", 2022-09-03T05:47:00.888 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.888 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.888 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.888 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.889 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.889 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.889 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tcKgEcOqLm4xxNq4RqPZPJv", 2022-09-03T05:47:00.889 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.890 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.890 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:47:00.890 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.891 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.892 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.892 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183996.925634_162.1", 2022-09-03T05:47:00.892 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:47:00.892 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-45", 2022-09-03T05:47:00.893 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:46:36.925634413Z", 2022-09-03T05:47:00.893 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.893 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.894 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.894 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.894 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.895 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_riKRPK3Rsd-5Q4VB2YVG1gO", 2022-09-03T05:47:00.895 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.895 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.896 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:47:00.896 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.896 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.896 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:00.897 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183996.928852_163.1", 2022-09-03T05:47:00.897 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:47:00.897 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-45", 2022-09-03T05:47:00.898 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:46:36.928852209Z", 2022-09-03T05:47:00.898 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:00.898 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:00.898 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:00.899 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:00.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.899 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:00.899 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_riKRPK3Rsd-5Q4VB2YVG1gO", 2022-09-03T05:47:00.900 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:00.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:00.900 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:00.902 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:47:00.902 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.902 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.902 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:00.903 INFO:teuthology.orchestra.run.smithi189.stdout:] 2022-09-03T05:47:00.903 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '229c309b-f405-4ead-a3a8-baa670c6da51', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:00.904 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '229c309b-f405-4ead-a3a8-baa670c6da51', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:00.904 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 229c309b-f405-4ead-a3a8-baa670c6da51 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:00.954 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:00.955 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:00.995 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:47:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183996.876451_156.1", 2022-09-03T05:47:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:47:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-44:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.1", 2022-09-03T05:47:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:46:36.876451430Z", 2022-09-03T05:47:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Ooi5Pr6jAe_eKSnUbc1yhF-", 2022-09-03T05:47:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:47:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183996.882242_157.1", 2022-09-03T05:47:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:47:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-44:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.1", 2022-09-03T05:47:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:46:36.882242917Z", 2022-09-03T05:47:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_Ooi5Pr6jAe_eKSnUbc1yhF-", 2022-09-03T05:47:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:47:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183996.883960_158.1", 2022-09-03T05:47:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:47:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-44", 2022-09-03T05:47:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:46:36.883960556Z", 2022-09-03T05:47:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_g7CXLiNivbLFArxtsjLR0Ea", 2022-09-03T05:47:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:47:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183996.887504_159.1", 2022-09-03T05:47:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:47:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-44", 2022-09-03T05:47:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:46:36.887504896Z", 2022-09-03T05:47:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_g7CXLiNivbLFArxtsjLR0Ea", 2022-09-03T05:47:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:47:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183996.919568_160.1", 2022-09-03T05:47:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:47:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-45:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.2", 2022-09-03T05:47:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:46:36.919568824Z", 2022-09-03T05:47:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tcKgEcOqLm4xxNq4RqPZPJv", 2022-09-03T05:47:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:47:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183996.924478_161.1", 2022-09-03T05:47:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket.instance", 2022-09-03T05:47:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-45:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.2", 2022-09-03T05:47:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:46:36.924478795Z", 2022-09-03T05:47:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_tcKgEcOqLm4xxNq4RqPZPJv", 2022-09-03T05:47:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:47:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183996.925634_162.1", 2022-09-03T05:47:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:47:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-45", 2022-09-03T05:47:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:46:36.925634413Z", 2022-09-03T05:47:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_riKRPK3Rsd-5Q4VB2YVG1gO", 2022-09-03T05:47:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "write" 2022-09-03T05:47:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "1_1662183996.928852_163.1", 2022-09-03T05:47:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: "section": "bucket", 2022-09-03T05:47:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "dsgvcy-45", 2022-09-03T05:47:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:46:36.928852209Z", 2022-09-03T05:47:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: "data": { 2022-09-03T05:47:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: "read_version": { 2022-09-03T05:47:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "", 2022-09-03T05:47:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 0 2022-09-03T05:47:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: "write_version": { 2022-09-03T05:47:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: "tag": "_riKRPK3Rsd-5Q4VB2YVG1gO", 2022-09-03T05:47:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": 1 2022-09-03T05:47:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: "status": { 2022-09-03T05:47:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "complete" 2022-09-03T05:47:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:01.045 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:47:01.045 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '229c309b-f405-4ead-a3a8-baa670c6da51', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:01.046 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '229c309b-f405-4ead-a3a8-baa670c6da51', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:01.046 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 229c309b-f405-4ead-a3a8-baa670c6da51 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:01.095 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:01.096 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:01.133 INFO:teuthology.orchestra.run.smithi189.stdout:[ 2022-09-03T05:47:01.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:01.134 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183996.876451_156.1", 2022-09-03T05:47:01.135 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:47:01.135 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-44:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.1", 2022-09-03T05:47:01.135 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:46:36.876451430Z", 2022-09-03T05:47:01.136 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:01.136 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:01.137 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:01.137 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:01.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:01.138 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:01.139 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Ooi5Pr6jAe_eKSnUbc1yhF-", 2022-09-03T05:47:01.139 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:01.140 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:01.141 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:01.141 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:47:01.142 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:01.142 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:01.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:01.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:01.143 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183996.882242_157.1", 2022-09-03T05:47:01.143 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:47:01.143 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-44:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.1", 2022-09-03T05:47:01.144 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:46:36.882242917Z", 2022-09-03T05:47:01.144 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:01.144 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:01.144 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:01.145 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:01.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:01.145 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:01.145 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Ooi5Pr6jAe_eKSnUbc1yhF-", 2022-09-03T05:47:01.146 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:01.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:01.147 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:01.147 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:47:01.148 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:01.148 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:01.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:01.148 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:01.149 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183996.883960_158.1", 2022-09-03T05:47:01.149 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:47:01.149 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-44", 2022-09-03T05:47:01.149 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:46:36.883960556Z", 2022-09-03T05:47:01.150 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:01.150 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:01.150 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:01.151 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:01.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:01.151 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:01.151 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_g7CXLiNivbLFArxtsjLR0Ea", 2022-09-03T05:47:01.152 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:01.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:01.152 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:01.153 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:47:01.153 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:01.153 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:01.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:01.154 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:01.154 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183996.887504_159.1", 2022-09-03T05:47:01.154 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:47:01.154 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-44", 2022-09-03T05:47:01.155 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:46:36.887504896Z", 2022-09-03T05:47:01.155 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:01.155 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:01.155 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:01.156 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:01.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:01.156 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:01.157 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_g7CXLiNivbLFArxtsjLR0Ea", 2022-09-03T05:47:01.157 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:01.157 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:01.157 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:01.157 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:47:01.158 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:01.158 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:01.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:01.159 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:01.159 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183996.919568_160.1", 2022-09-03T05:47:01.159 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:47:01.159 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-45:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.2", 2022-09-03T05:47:01.160 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:46:36.919568824Z", 2022-09-03T05:47:01.160 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:01.160 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:01.160 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:01.161 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:01.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:01.161 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:01.161 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tcKgEcOqLm4xxNq4RqPZPJv", 2022-09-03T05:47:01.162 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:01.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:01.162 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:01.162 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:47:01.163 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:01.163 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:01.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:01.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:01.164 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183996.924478_161.1", 2022-09-03T05:47:01.164 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket.instance", 2022-09-03T05:47:01.164 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-45:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.2", 2022-09-03T05:47:01.164 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:46:36.924478795Z", 2022-09-03T05:47:01.165 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:01.165 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:01.165 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:01.165 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:01.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:01.166 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:01.166 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_tcKgEcOqLm4xxNq4RqPZPJv", 2022-09-03T05:47:01.167 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:01.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:01.167 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:01.167 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:47:01.168 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:01.168 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:01.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:01.168 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:01.169 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183996.925634_162.1", 2022-09-03T05:47:01.169 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:47:01.169 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-45", 2022-09-03T05:47:01.169 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:46:36.925634413Z", 2022-09-03T05:47:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_riKRPK3Rsd-5Q4VB2YVG1gO", 2022-09-03T05:47:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:01.172 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:01.172 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:01.172 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "write" 2022-09-03T05:47:01.172 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:01.173 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:01.173 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:01.173 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:01.173 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "1_1662183996.928852_163.1", 2022-09-03T05:47:01.174 INFO:teuthology.orchestra.run.smithi189.stdout: "section": "bucket", 2022-09-03T05:47:01.174 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "dsgvcy-45", 2022-09-03T05:47:01.174 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:46:36.928852209Z", 2022-09-03T05:47:01.174 INFO:teuthology.orchestra.run.smithi189.stdout: "data": { 2022-09-03T05:47:01.175 INFO:teuthology.orchestra.run.smithi189.stdout: "read_version": { 2022-09-03T05:47:01.175 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "", 2022-09-03T05:47:01.175 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 0 2022-09-03T05:47:01.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:01.176 INFO:teuthology.orchestra.run.smithi189.stdout: "write_version": { 2022-09-03T05:47:01.176 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_riKRPK3Rsd-5Q4VB2YVG1gO", 2022-09-03T05:47:01.176 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": 1 2022-09-03T05:47:01.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:01.177 INFO:teuthology.orchestra.run.smithi189.stdout: "status": { 2022-09-03T05:47:01.178 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "complete" 2022-09-03T05:47:01.178 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:01.179 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:01.179 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:01.179 INFO:teuthology.orchestra.run.smithi189.stdout:] 2022-09-03T05:47:01.180 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:47:01.180 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:47:01.181 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog autotrim --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:47:01.227 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:01.227 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:01.307 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'autotrim', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:47:01.308 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'autotrim', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:47:01.309 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog autotrim --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:47:01.354 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:01.355 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:01.516 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:47:01.517 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:47:01.519 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog autotrim --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:47:01.586 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:01.587 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:01.650 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'autotrim', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:47:01.651 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'autotrim', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:47:01.652 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog autotrim --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:47:01.699 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:01.699 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:01.828 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '41ecd2a5-afc0-4eae-9349-7dfe39baa1cb', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:01.829 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '41ecd2a5-afc0-4eae-9349-7dfe39baa1cb', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:01.830 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 41ecd2a5-afc0-4eae-9349-7dfe39baa1cb --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:01.879 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:01.880 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:01.921 INFO:teuthology.orchestra.run.smithi181.stdout:[] 2022-09-03T05:47:01.922 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '41ecd2a5-afc0-4eae-9349-7dfe39baa1cb', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:01.923 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '41ecd2a5-afc0-4eae-9349-7dfe39baa1cb', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:01.924 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 41ecd2a5-afc0-4eae-9349-7dfe39baa1cb --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:01.972 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:01.973 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:02.016 INFO:teuthology.orchestra.run.smithi189.stdout:[] 2022-09-03T05:47:02.018 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '41ecd2a5-afc0-4eae-9349-7dfe39baa1cb', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:02.019 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '41ecd2a5-afc0-4eae-9349-7dfe39baa1cb', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:02.020 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 41ecd2a5-afc0-4eae-9349-7dfe39baa1cb --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:02.070 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:02.070 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:02.150 INFO:teuthology.orchestra.run.smithi181.stdout:[] 2022-09-03T05:47:02.152 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '41ecd2a5-afc0-4eae-9349-7dfe39baa1cb', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:02.153 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '41ecd2a5-afc0-4eae-9349-7dfe39baa1cb', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:02.154 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 41ecd2a5-afc0-4eae-9349-7dfe39baa1cb --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:02.203 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:02.204 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:02.258 INFO:teuthology.orchestra.run.smithi189.stdout:[] 2022-09-03T05:47:02.260 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', 'eff2cb82-c479-4631-8568-fb1af4159e02', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:02.261 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', 'eff2cb82-c479-4631-8568-fb1af4159e02', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:02.261 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period eff2cb82-c479-4631-8568-fb1af4159e02 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:02.317 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:02.317 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:02.363 INFO:teuthology.orchestra.run.smithi181.stdout:[] 2022-09-03T05:47:02.365 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', 'eff2cb82-c479-4631-8568-fb1af4159e02', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:02.365 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', 'eff2cb82-c479-4631-8568-fb1af4159e02', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:02.366 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period eff2cb82-c479-4631-8568-fb1af4159e02 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:02.419 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:02.420 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:02.455 INFO:teuthology.orchestra.run.smithi189.stdout:[] 2022-09-03T05:47:02.457 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', 'eff2cb82-c479-4631-8568-fb1af4159e02', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:02.457 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', 'eff2cb82-c479-4631-8568-fb1af4159e02', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:02.458 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period eff2cb82-c479-4631-8568-fb1af4159e02 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:02.506 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:02.507 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:02.555 INFO:teuthology.orchestra.run.smithi181.stdout:[] 2022-09-03T05:47:02.559 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', 'eff2cb82-c479-4631-8568-fb1af4159e02', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:02.560 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', 'eff2cb82-c479-4631-8568-fb1af4159e02', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:02.561 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period eff2cb82-c479-4631-8568-fb1af4159e02 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:02.611 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:02.612 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:02.649 INFO:teuthology.orchestra.run.smithi189.stdout:[] 2022-09-03T05:47:02.652 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '229c309b-f405-4ead-a3a8-baa670c6da51', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:02.652 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '229c309b-f405-4ead-a3a8-baa670c6da51', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:02.653 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 229c309b-f405-4ead-a3a8-baa670c6da51 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:02.704 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:02.705 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:02.773 INFO:teuthology.orchestra.run.smithi181.stdout:[] 2022-09-03T05:47:02.775 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '229c309b-f405-4ead-a3a8-baa670c6da51', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:02.776 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '229c309b-f405-4ead-a3a8-baa670c6da51', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:02.777 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 229c309b-f405-4ead-a3a8-baa670c6da51 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:02.826 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:02.827 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:02.884 INFO:teuthology.orchestra.run.smithi189.stdout:[] 2022-09-03T05:47:02.886 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'list', '--period', '229c309b-f405-4ead-a3a8-baa670c6da51', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:02.887 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'list', '--period', '229c309b-f405-4ead-a3a8-baa670c6da51', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:02.888 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog list --period 229c309b-f405-4ead-a3a8-baa670c6da51 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:02.939 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:02.941 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:03.005 INFO:teuthology.orchestra.run.smithi181.stdout:[] 2022-09-03T05:47:03.007 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['mdlog', 'list', '--period', '229c309b-f405-4ead-a3a8-baa670c6da51', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:03.008 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'mdlog', 'list', '--period', '229c309b-f405-4ead-a3a8-baa670c6da51', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:03.008 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 mdlog list --period 229c309b-f405-4ead-a3a8-baa670c6da51 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:03.056 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:03.057 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:03.103 INFO:teuthology.orchestra.run.smithi189.stdout:[] 2022-09-03T05:47:03.105 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_multi_period_incremental_sync ... ok 2022-09-03T05:47:03.108 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-46 2022-09-03T05:47:03.109 DEBUG:boto:path=/dsgvcy-46/ 2022-09-03T05:47:03.109 DEBUG:boto:auth_path=/dsgvcy-46/ 2022-09-03T05:47:03.110 DEBUG:boto:Method: PUT 2022-09-03T05:47:03.111 DEBUG:boto:Path: /dsgvcy-46/ 2022-09-03T05:47:03.111 DEBUG:boto:Data: 2022-09-03T05:47:03.112 DEBUG:boto:Headers: {} 2022-09-03T05:47:03.112 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:47:03.113 DEBUG:boto:Port: 8000 2022-09-03T05:47:03.113 DEBUG:boto:Params: {} 2022-09-03T05:47:03.114 DEBUG:boto:Token: None 2022-09-03T05:47:03.115 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:47:03 GMT /dsgvcy-46/ 2022-09-03T05:47:03.116 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ISxylXUurIfLoNqeIP6dycxJbkI= 2022-09-03T05:47:03.117 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:47:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ISxylXUurIfLoNqeIP6dycxJbkI=', 'Content-Length': '0'} 2022-09-03T05:47:03.132 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000000c36e8faa117d616-006312ea57-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:47:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:47:03.133 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=dsgvcy-47 2022-09-03T05:47:03.134 DEBUG:boto:path=/dsgvcy-47/ 2022-09-03T05:47:03.134 DEBUG:boto:auth_path=/dsgvcy-47/ 2022-09-03T05:47:03.135 DEBUG:boto:Method: PUT 2022-09-03T05:47:03.135 DEBUG:boto:Path: /dsgvcy-47/ 2022-09-03T05:47:03.136 DEBUG:boto:Data: 2022-09-03T05:47:03.136 DEBUG:boto:Headers: {} 2022-09-03T05:47:03.137 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:47:03.137 DEBUG:boto:Port: 8000 2022-09-03T05:47:03.138 DEBUG:boto:Params: {} 2022-09-03T05:47:03.138 DEBUG:boto:Token: None 2022-09-03T05:47:03.139 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:47:03 GMT /dsgvcy-47/ 2022-09-03T05:47:03.140 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xO+JdceyXcpYG0++hNlnitAbMAQ= 2022-09-03T05:47:03.141 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:47:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xO+JdceyXcpYG0++hNlnitAbMAQ=', 'Content-Length': '0'} 2022-09-03T05:47:03.166 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000001e37bea542ac2cfc-006312ea57-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:47:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:47:03.167 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=dsgvcy-48 2022-09-03T05:47:03.167 DEBUG:boto:path=/dsgvcy-48/ 2022-09-03T05:47:03.168 DEBUG:boto:auth_path=/dsgvcy-48/ 2022-09-03T05:47:03.168 DEBUG:boto:Method: PUT 2022-09-03T05:47:03.169 DEBUG:boto:Path: /dsgvcy-48/ 2022-09-03T05:47:03.170 DEBUG:boto:Data: 2022-09-03T05:47:03.170 DEBUG:boto:Headers: {} 2022-09-03T05:47:03.171 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:47:03.172 DEBUG:boto:Port: 8001 2022-09-03T05:47:03.173 DEBUG:boto:Params: {} 2022-09-03T05:47:03.174 DEBUG:boto:Token: None 2022-09-03T05:47:03.174 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:47:03 GMT /dsgvcy-48/ 2022-09-03T05:47:03.175 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TOoAgLW3fDAQDrk1A2E/G/c+4lQ= 2022-09-03T05:47:03.176 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:47:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TOoAgLW3fDAQDrk1A2E/G/c+4lQ=', 'Content-Length': '0'} 2022-09-03T05:47:03.218 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000006ba4df7afc064136-006312ea57-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:47:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:47:03.219 DEBUG:boto:path=/dsgvcy-46/ 2022-09-03T05:47:03.219 DEBUG:boto:auth_path=/dsgvcy-46/ 2022-09-03T05:47:03.220 DEBUG:boto:Method: HEAD 2022-09-03T05:47:03.220 DEBUG:boto:Path: /dsgvcy-46/ 2022-09-03T05:47:03.221 DEBUG:boto:Data: 2022-09-03T05:47:03.221 DEBUG:boto:Headers: {} 2022-09-03T05:47:03.222 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:47:03.222 DEBUG:boto:Port: 8000 2022-09-03T05:47:03.223 DEBUG:boto:Params: {} 2022-09-03T05:47:03.224 DEBUG:boto:Token: None 2022-09-03T05:47:03.226 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:47:03 GMT /dsgvcy-46/ 2022-09-03T05:47:03.227 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C377Zx6ILAMr8coHOQrifCsMtj0= 2022-09-03T05:47:03.230 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:47:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C377Zx6ILAMr8coHOQrifCsMtj0=', 'Content-Length': '0'} 2022-09-03T05:47:03.233 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000698f80d47269af72-006312ea57-1434-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:47:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:47:03.234 DEBUG:boto:path=/dsgvcy-46/key 2022-09-03T05:47:03.235 DEBUG:boto:auth_path=/dsgvcy-46/key 2022-09-03T05:47:03.236 DEBUG:boto:Method: PUT 2022-09-03T05:47:03.236 DEBUG:boto:Path: /dsgvcy-46/key 2022-09-03T05:47:03.237 DEBUG:boto:Data: 2022-09-03T05:47:03.237 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'hBotaJrYa9FhFEdFPCLG/A==', 'Content-Length': '4', 'Expect': '100-Continue'} 2022-09-03T05:47:03.238 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:47:03.238 DEBUG:boto:Port: 8000 2022-09-03T05:47:03.239 DEBUG:boto:Params: {} 2022-09-03T05:47:03.239 DEBUG:boto:Token: None 2022-09-03T05:47:03.240 DEBUG:boto:StringToSign: PUT hBotaJrYa9FhFEdFPCLG/A== application/octet-stream Sat, 03 Sep 2022 05:47:03 GMT /dsgvcy-46/key 2022-09-03T05:47:03.241 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b5kLZMKcAK22iVyZiI475D9i7A4= 2022-09-03T05:47:03.241 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'hBotaJrYa9FhFEdFPCLG/A==', 'Content-Length': '4', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:47:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b5kLZMKcAK22iVyZiI475D9i7A4='} 2022-09-03T05:47:03.254 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"841a2d689ad86bd1611447453c22c6fc"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d517df6fd0405171-006312ea57-1434-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:47:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:47:03.255 DEBUG:boto:path=/dsgvcy-47/ 2022-09-03T05:47:03.256 DEBUG:boto:auth_path=/dsgvcy-47/ 2022-09-03T05:47:03.257 DEBUG:boto:Method: HEAD 2022-09-03T05:47:03.258 DEBUG:boto:Path: /dsgvcy-47/ 2022-09-03T05:47:03.260 DEBUG:boto:Data: 2022-09-03T05:47:03.261 DEBUG:boto:Headers: {} 2022-09-03T05:47:03.261 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:47:03.262 DEBUG:boto:Port: 8000 2022-09-03T05:47:03.263 DEBUG:boto:Params: {} 2022-09-03T05:47:03.263 DEBUG:boto:Token: None 2022-09-03T05:47:03.264 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:47:03 GMT /dsgvcy-47/ 2022-09-03T05:47:03.265 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yp6Ww4aqSuTPJ3rYEnkP8wazbnk= 2022-09-03T05:47:03.265 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:47:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yp6Ww4aqSuTPJ3rYEnkP8wazbnk=', 'Content-Length': '0'} 2022-09-03T05:47:03.270 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003146ed728a02f070-006312ea57-1444-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:47:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:47:03.271 DEBUG:boto:path=/dsgvcy-47/key 2022-09-03T05:47:03.271 DEBUG:boto:auth_path=/dsgvcy-47/key 2022-09-03T05:47:03.272 DEBUG:boto:Method: PUT 2022-09-03T05:47:03.272 DEBUG:boto:Path: /dsgvcy-47/key 2022-09-03T05:47:03.273 DEBUG:boto:Data: 2022-09-03T05:47:03.273 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'hBotaJrYa9FhFEdFPCLG/A==', 'Content-Length': '4', 'Expect': '100-Continue'} 2022-09-03T05:47:03.274 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:47:03.274 DEBUG:boto:Port: 8000 2022-09-03T05:47:03.275 DEBUG:boto:Params: {} 2022-09-03T05:47:03.276 DEBUG:boto:Token: None 2022-09-03T05:47:03.276 DEBUG:boto:StringToSign: PUT hBotaJrYa9FhFEdFPCLG/A== application/octet-stream Sat, 03 Sep 2022 05:47:03 GMT /dsgvcy-47/key 2022-09-03T05:47:03.277 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2hPg8Yo2jPeBHcoRfarOMFTYPQQ= 2022-09-03T05:47:03.277 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'hBotaJrYa9FhFEdFPCLG/A==', 'Content-Length': '4', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:47:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2hPg8Yo2jPeBHcoRfarOMFTYPQQ='} 2022-09-03T05:47:03.284 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"841a2d689ad86bd1611447453c22c6fc"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000058c8adb0dae529d4-006312ea57-1444-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:47:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:47:03.285 DEBUG:boto:path=/dsgvcy-48/ 2022-09-03T05:47:03.286 DEBUG:boto:auth_path=/dsgvcy-48/ 2022-09-03T05:47:03.287 DEBUG:boto:Method: HEAD 2022-09-03T05:47:03.288 DEBUG:boto:Path: /dsgvcy-48/ 2022-09-03T05:47:03.288 DEBUG:boto:Data: 2022-09-03T05:47:03.289 DEBUG:boto:Headers: {} 2022-09-03T05:47:03.289 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:47:03.291 DEBUG:boto:Port: 8001 2022-09-03T05:47:03.291 DEBUG:boto:Params: {} 2022-09-03T05:47:03.292 DEBUG:boto:Token: None 2022-09-03T05:47:03.293 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:47:03 GMT /dsgvcy-48/ 2022-09-03T05:47:03.294 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tI73B7VswFsyKtG+UaC2vBITqGM= 2022-09-03T05:47:03.294 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:47:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tI73B7VswFsyKtG+UaC2vBITqGM=', 'Content-Length': '0'} 2022-09-03T05:47:03.297 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002bd52f7942eb543b-006312ea57-143a-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:47:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:47:03.298 DEBUG:boto:path=/dsgvcy-48/key 2022-09-03T05:47:03.300 DEBUG:boto:auth_path=/dsgvcy-48/key 2022-09-03T05:47:03.302 DEBUG:boto:Method: PUT 2022-09-03T05:47:03.303 DEBUG:boto:Path: /dsgvcy-48/key 2022-09-03T05:47:03.304 DEBUG:boto:Data: 2022-09-03T05:47:03.304 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'hBotaJrYa9FhFEdFPCLG/A==', 'Content-Length': '4', 'Expect': '100-Continue'} 2022-09-03T05:47:03.305 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:47:03.306 DEBUG:boto:Port: 8001 2022-09-03T05:47:03.307 DEBUG:boto:Params: {} 2022-09-03T05:47:03.308 DEBUG:boto:Token: None 2022-09-03T05:47:03.309 DEBUG:boto:StringToSign: PUT hBotaJrYa9FhFEdFPCLG/A== application/octet-stream Sat, 03 Sep 2022 05:47:03 GMT /dsgvcy-48/key 2022-09-03T05:47:03.310 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:x/uYKfwCcoLmG0OxpzMWCRDtgKM= 2022-09-03T05:47:03.311 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'hBotaJrYa9FhFEdFPCLG/A==', 'Content-Length': '4', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:47:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:x/uYKfwCcoLmG0OxpzMWCRDtgKM='} 2022-09-03T05:47:03.328 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"841a2d689ad86bd1611447453c22c6fc"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000007176196a3b2694e9-006312ea57-143a-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:47:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:47:03.329 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:03.330 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:03.331 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:03.382 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:03.383 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:03.437 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T05:47:03.452 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:47:03.452 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:03.453 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:47:03.453 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:47:03.454 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:03.454 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:03.457 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:47:03.458 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:47:03.458 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:03.458 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:03.459 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184023.196251_177.1", 2022-09-03T05:47:03.459 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:47:03.196251Z" 2022-09-03T05:47:03.460 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:03.460 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:03.461 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:47:03.461 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:47:03.461 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:03.462 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:47:03.464 DEBUG:rgw_multi.tests:master meta markers={0: '', 1: '', 2: '1_1662184023.196251_177.1', 3: ''} 2022-09-03T05:47:03.466 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:47:03.467 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:03.468 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:03.469 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:03.523 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:03.524 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:03.559 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:47:03.556+0000 7f36a1736600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:47:03.560 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:47:03.557+0000 7f36a1736600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:47:03.572 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:47:03.573 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:47:03.573 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:47:03.573 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:47:03.574 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:47:03.574 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:47:03.574 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:47:03.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:03.575 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:47:03.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:03.576 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:47:03.576 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:03.577 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:47:03.577 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:47:03.577 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:03.577 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:47:03.578 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:03.578 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:47:03.579 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:47:03.579 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:03.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:03.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:03.580 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:47:03.581 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:03.581 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:47:03.582 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:47:03.582 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:03.582 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:03.583 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:03.583 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:47:03.583 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:47:03.583 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:03.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:03.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:03.584 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:47:03.584 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:03.584 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:47:03.585 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184023.125261_165.1", 2022-09-03T05:47:03.585 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:03.585 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:03.586 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:03.586 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:47:03.125261Z", 2022-09-03T05:47:03.587 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:47:03.587 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:03.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:03.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:03.589 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:47:03.589 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:03.589 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:47:03.590 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:47:03.590 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:03.591 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:03.591 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:03.592 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:47:03.592 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:47:03.593 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:03.594 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:03.594 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:47:03.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:03.595 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:47:03.596 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:47:03.596 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:47:03.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:03.597 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:47:03Z" 2022-09-03T05:47:03.598 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:47:03.600 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184023.125261_165.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:03.125261Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:47:03Z" } 2022-09-03T05:47:03.601 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184023.125261_165.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:03.125261Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:47:03.602 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184023.196251_177.1', 3: ''} 2022-09-03T05:47:03.602 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184023.125261_165.1', 3: ''} 2022-09-03T05:47:03.603 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662184023.196251_177.1 target=1_1662184023.125261_165.1 2022-09-03T05:47:08.605 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:08.606 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:08.607 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:08.664 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:08.665 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:08.695 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:47:08.693+0000 7fd5342e8600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:47:08.696 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:47:08.694+0000 7fd5342e8600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:47:08.711 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:47:08.712 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:47:08.712 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:47:08.712 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:47:08.712 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:47:08.713 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:47:08.713 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:47:08.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:08.713 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:47:08.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:08.714 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:47:08.715 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:08.715 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:47:08.715 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:47:08.716 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:08.716 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:47:08.716 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:08.716 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:47:08.717 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:47:08.717 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:08.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:08.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:08.718 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:47:08.718 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:08.718 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:47:08.718 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:47:08.719 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:08.719 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:08.720 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:08.720 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:47:08.721 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:47:08.722 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:08.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:08.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:08.723 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:47:08.724 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:08.724 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:47:08.725 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184023.196251_177.1", 2022-09-03T05:47:08.725 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:08.726 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:08.726 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:08.727 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:47:03.196251Z", 2022-09-03T05:47:08.727 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:47:08.728 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:08.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:08.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:08.730 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:47:08.730 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:08.731 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:47:08.731 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:47:08.732 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:08.733 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:08.733 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:08.734 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:47:08.734 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:47:08.735 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:08.735 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:08.736 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:47:08.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:08.737 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:47:08.738 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:47:08.738 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:47:08.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:08.739 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:47:08Z" 2022-09-03T05:47:08.740 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:47:08.741 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184023.196251_177.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:03.196251Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:47:08Z" } 2022-09-03T05:47:08.742 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184023.196251_177.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:03.196251Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:47:08.743 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184023.196251_177.1', 3: ''} 2022-09-03T05:47:08.743 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184023.196251_177.1', 3: ''} 2022-09-03T05:47:08.744 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:47:08.744 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:47:08.745 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:08.745 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:08.746 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:08.805 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:08.806 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:08.862 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:47:08.861+0000 7f32edb4b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:47:08.864 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:47:08.862+0000 7f32edb4b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:47:08.886 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:47:08.887 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:47:08.887 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:47:08.888 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:47:08.888 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:47:08.888 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:47:08.889 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:47:08.889 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:08.889 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:47:08.890 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:08.890 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:47:08.890 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:08.891 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:47:08.891 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:47:08.891 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:08.891 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:47:08.892 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:08.893 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:47:08.893 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:47:08.894 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:08.894 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:08.895 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:08.895 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:47:08.895 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:08.896 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:47:08.896 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:47:08.896 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:08.897 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:08.897 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:08.897 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:47:08.898 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:47:08.898 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:08.898 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:08.898 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:08.899 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:47:08.899 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:08.900 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:47:08.900 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184023.196251_177.1", 2022-09-03T05:47:08.900 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:08.901 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:08.901 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:08.902 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:47:03.196251Z", 2022-09-03T05:47:08.902 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:47:08.903 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:08.903 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:08.904 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:08.904 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:47:08.905 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:08.905 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:47:08.906 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:47:08.906 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:08.906 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:08.907 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:08.907 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:47:08.908 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:47:08.911 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:08.911 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:08.912 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:47:08.912 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:08.913 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:47:08.913 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:47:08.914 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:47:08.915 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:08.915 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:47:08Z" 2022-09-03T05:47:08.916 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:47:08.919 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184023.196251_177.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:03.196251Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:47:08Z" } 2022-09-03T05:47:08.921 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184023.196251_177.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:03.196251Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:47:08.922 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184023.196251_177.1', 3: ''} 2022-09-03T05:47:08.923 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184023.196251_177.1', 3: ''} 2022-09-03T05:47:08.926 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:47:08.927 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:47:08.928 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:08.929 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:08.930 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:08.983 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:08.983 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:09.009 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:47:09.008+0000 7f69da23a600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:47:09.010 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:47:09.009+0000 7f69da23a600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:47:09.021 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:47:09.021 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:47:09.022 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:47:09.022 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:47:09.023 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:47:09.023 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:47:09.024 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:47:09.024 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:09.025 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:47:09.026 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:09.026 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:47:09.027 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:09.027 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:47:09.027 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:47:09.027 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:09.028 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:47:09.028 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:09.028 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:47:09.029 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:47:09.029 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:09.029 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:09.029 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:09.030 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:47:09.030 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:09.030 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:47:09.030 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:47:09.031 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:09.031 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:09.031 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:09.032 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:47:09.032 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:47:09.032 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:09.033 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:09.033 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:09.034 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:47:09.034 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:09.034 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:47:09.034 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184023.196251_177.1", 2022-09-03T05:47:09.035 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:09.035 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:09.035 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:09.035 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:47:03.196251Z", 2022-09-03T05:47:09.036 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:47:09.036 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:09.036 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:09.037 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:09.038 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:47:09.038 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:09.038 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:47:09.038 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:47:09.039 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:09.039 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:09.039 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:09.040 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:47:09.040 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:47:09.040 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:09.040 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:09.041 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:47:09.041 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:09.042 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:47:09.042 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:47:09.043 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:47:09.043 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:09.043 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:47:09Z" 2022-09-03T05:47:09.044 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:47:09.046 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184023.196251_177.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:03.196251Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:47:09Z" } 2022-09-03T05:47:09.046 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184023.196251_177.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:03.196251Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:47:09.048 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184023.196251_177.1', 3: ''} 2022-09-03T05:47:09.048 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184023.196251_177.1', 3: ''} 2022-09-03T05:47:09.048 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:47:09.049 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone1 target=test-zone2 2022-09-03T05:47:09.050 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:09.050 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:09.051 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:09.100 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:09.101 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:09.170 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:47:09.171 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:09.171 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002768", 2022-09-03T05:47:09.171 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:31:26.029653Z" 2022-09-03T05:47:09.172 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:09.172 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:09.172 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000004741", 2022-09-03T05:47:09.172 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:47:08.805959Z" 2022-09-03T05:47:09.173 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:09.173 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:09.173 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:47:09.174 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:37:06.825525Z" 2022-09-03T05:47:09.174 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:09.174 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:09.174 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:47:09.175 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:44:26.899674Z" 2022-09-03T05:47:09.175 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:09.175 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:47:09.176 DEBUG:rgw_multi.tests:data markers for zone=test-zone1 markers={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000004741', 2: '00000000000000000000:00000000000000006434', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:09.177 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2022-09-03T05:47:09.178 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:09.179 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:09.181 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:09.234 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:09.235 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:09.288 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:47:09.289 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:47:09.290 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:47:09.290 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:47:09.291 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:47:09.291 INFO:teuthology.orchestra.run.smithi189.stdout: "instance_id": 4093329069989467412 2022-09-03T05:47:09.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:09.292 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:47:09.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:09.293 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:47:09.293 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:09.296 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:09.296 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000002768", 2022-09-03T05:47:09.297 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:09.297 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:09.298 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:09.298 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:31:26.029653Z" 2022-09-03T05:47:09.299 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:09.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:09.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:09.300 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:47:09.300 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:09.301 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:09.301 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000004601", 2022-09-03T05:47:09.302 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:09.302 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:09.302 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:09.303 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:47:03.248970Z" 2022-09-03T05:47:09.303 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:09.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:09.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:09.307 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:47:09.307 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:09.308 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:09.308 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:47:09.309 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:09.309 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:09.309 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:09.310 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:37:06.825525Z" 2022-09-03T05:47:09.311 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:09.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:09.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:09.312 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:47:09.313 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:09.313 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:09.313 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:47:09.314 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:09.314 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:09.314 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:09.315 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:26.899674Z" 2022-09-03T05:47:09.315 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:09.316 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:09.316 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:47:09.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:09.317 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:47:09.317 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 0, 2022-09-03T05:47:09.317 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 0 2022-09-03T05:47:09.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:09.318 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:47:09Z" 2022-09-03T05:47:09.318 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:47:09.320 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 4093329069989467412 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002768", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:26.029653Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000004601", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:03.248970Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000006434", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:37:06.825525Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000000796", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:26.899674Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:47:09Z" } 2022-09-03T05:47:09.322 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002768', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:26.029653Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004601', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:03.248970Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006434', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:37:06.825525Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000000796', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:26.899674Z'}}] 2022-09-03T05:47:09.322 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000004741', 2: '00000000000000000000:00000000000000006434', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:09.323 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000004601', 2: '00000000000000000000:00000000000000006434', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:09.324 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=1 master=00000000000000000000:00000000000000004741 target=00000000000000000000:00000000000000004601 2022-09-03T05:47:14.328 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:14.329 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:14.329 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:14.390 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:14.390 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:14.454 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:47:14.455 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:47:14.455 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:47:14.456 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:47:14.456 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:47:14.456 INFO:teuthology.orchestra.run.smithi189.stdout: "instance_id": 4093329069989467412 2022-09-03T05:47:14.457 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:14.457 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:47:14.457 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:14.458 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:47:14.458 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:14.459 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:14.459 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000002768", 2022-09-03T05:47:14.460 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:14.460 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:14.461 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:14.461 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:31:26.029653Z" 2022-09-03T05:47:14.462 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:14.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:14.463 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:14.463 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:47:14.464 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:14.464 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:14.465 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000004601", 2022-09-03T05:47:14.465 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:14.466 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:14.466 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:14.467 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:47:03.248970Z" 2022-09-03T05:47:14.467 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:14.468 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:14.468 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:14.469 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:47:14.469 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:14.470 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:14.470 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:47:14.471 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:14.471 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:14.471 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:14.472 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:37:06.825525Z" 2022-09-03T05:47:14.472 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:14.472 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:14.473 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:14.473 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:47:14.473 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:14.473 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:14.474 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:47:14.474 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:14.474 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:14.475 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:14.476 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:26.899674Z" 2022-09-03T05:47:14.476 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:14.477 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:14.477 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:47:14.478 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:14.479 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:47:14.479 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 0, 2022-09-03T05:47:14.480 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 0 2022-09-03T05:47:14.480 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:14.481 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:47:14Z" 2022-09-03T05:47:14.481 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:47:14.484 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 4093329069989467412 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002768", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:26.029653Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000004601", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:03.248970Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000006434", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:37:06.825525Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000000796", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:26.899674Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:47:14Z" } 2022-09-03T05:47:14.485 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002768', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:26.029653Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004601', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:03.248970Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006434', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:37:06.825525Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000000796', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:26.899674Z'}}] 2022-09-03T05:47:14.486 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000004741', 2: '00000000000000000000:00000000000000006434', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:14.487 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000004601', 2: '00000000000000000000:00000000000000006434', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:14.488 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=1 master=00000000000000000000:00000000000000004741 target=00000000000000000000:00000000000000004601 2022-09-03T05:47:19.490 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:19.491 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:19.492 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:19.552 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:19.552 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:19.598 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:47:19.599 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:47:19.599 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:47:19.600 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:47:19.600 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:47:19.601 INFO:teuthology.orchestra.run.smithi189.stdout: "instance_id": 4093329069989467412 2022-09-03T05:47:19.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:19.602 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:47:19.605 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:19.606 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:47:19.606 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:19.607 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:19.607 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000002768", 2022-09-03T05:47:19.607 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:19.608 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:19.608 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:19.609 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:31:26.029653Z" 2022-09-03T05:47:19.609 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:19.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:19.610 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:19.610 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:47:19.611 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:19.611 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:19.611 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000004601", 2022-09-03T05:47:19.611 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:19.612 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:19.612 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:19.613 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:47:03.248970Z" 2022-09-03T05:47:19.613 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:19.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:19.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:19.615 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:47:19.616 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:19.616 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:19.616 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:47:19.617 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:19.617 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:19.618 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:19.619 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:37:06.825525Z" 2022-09-03T05:47:19.620 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:19.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:19.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:19.621 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:47:19.621 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:19.622 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:19.622 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:47:19.623 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:19.623 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:19.624 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:19.624 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:26.899674Z" 2022-09-03T05:47:19.625 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:19.625 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:19.626 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:47:19.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:19.627 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:47:19.628 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 0, 2022-09-03T05:47:19.628 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 0 2022-09-03T05:47:19.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:19.630 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:47:19Z" 2022-09-03T05:47:19.631 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:47:19.633 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 4093329069989467412 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002768", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:26.029653Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000004601", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:03.248970Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000006434", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:37:06.825525Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000000796", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:26.899674Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:47:19Z" } 2022-09-03T05:47:19.634 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002768', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:26.029653Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004601', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:03.248970Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006434', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:37:06.825525Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000000796', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:26.899674Z'}}] 2022-09-03T05:47:19.635 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000004741', 2: '00000000000000000000:00000000000000006434', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:19.635 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000004601', 2: '00000000000000000000:00000000000000006434', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:19.636 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=1 master=00000000000000000000:00000000000000004741 target=00000000000000000000:00000000000000004601 2022-09-03T05:47:24.638 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:24.639 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:24.641 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:24.704 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:24.705 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:24.765 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:47:24.765 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:47:24.765 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:47:24.766 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:47:24.766 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:47:24.766 INFO:teuthology.orchestra.run.smithi189.stdout: "instance_id": 4093329069989467412 2022-09-03T05:47:24.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:24.767 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:47:24.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:24.767 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:47:24.768 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:24.768 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:24.768 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000002768", 2022-09-03T05:47:24.768 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:24.769 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:24.769 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:24.769 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:31:26.029653Z" 2022-09-03T05:47:24.769 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:24.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:24.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:24.770 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:47:24.771 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:24.771 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:24.771 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000004601", 2022-09-03T05:47:24.771 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:24.772 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:24.772 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:24.772 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:47:03.248970Z" 2022-09-03T05:47:24.772 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:24.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:24.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:24.773 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:47:24.773 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:24.774 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:24.774 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:47:24.774 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:24.775 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:24.775 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:24.775 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:37:06.825525Z" 2022-09-03T05:47:24.776 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:24.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:24.776 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:24.776 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:47:24.777 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:24.777 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:24.777 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:47:24.777 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:24.778 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:24.778 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:24.778 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:26.899674Z" 2022-09-03T05:47:24.779 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:24.779 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:24.779 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:47:24.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:24.780 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:47:24.780 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 0, 2022-09-03T05:47:24.780 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 0 2022-09-03T05:47:24.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:24.781 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:47:24Z" 2022-09-03T05:47:24.781 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:47:24.782 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 4093329069989467412 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002768", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:26.029653Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000004601", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:03.248970Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000006434", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:37:06.825525Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000000796", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:26.899674Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:47:24Z" } 2022-09-03T05:47:24.783 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002768', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:26.029653Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004601', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:03.248970Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006434', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:37:06.825525Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000000796', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:26.899674Z'}}] 2022-09-03T05:47:24.784 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000004741', 2: '00000000000000000000:00000000000000006434', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:24.784 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000004601', 2: '00000000000000000000:00000000000000006434', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:24.785 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=1 master=00000000000000000000:00000000000000004741 target=00000000000000000000:00000000000000004601 2022-09-03T05:47:29.786 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:29.787 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:29.788 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:29.847 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:29.847 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:29.903 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:47:29.904 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:47:29.905 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:47:29.905 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:47:29.905 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:47:29.906 INFO:teuthology.orchestra.run.smithi189.stdout: "instance_id": 4093329069989467412 2022-09-03T05:47:29.906 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:29.906 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:47:29.906 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:29.907 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:47:29.907 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:29.909 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:29.909 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000002768", 2022-09-03T05:47:29.910 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:29.910 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:29.911 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:29.911 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:31:26.029653Z" 2022-09-03T05:47:29.912 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:29.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:29.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:29.913 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:47:29.914 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:29.915 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:29.915 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000004881", 2022-09-03T05:47:29.916 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:29.917 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:29.917 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:29.918 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:47:17.095630Z" 2022-09-03T05:47:29.918 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:29.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:29.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:29.920 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:47:29.920 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:29.921 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:29.921 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:47:29.922 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:29.922 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:29.923 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:29.923 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:37:06.825525Z" 2022-09-03T05:47:29.924 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:29.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:29.925 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:29.925 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:47:29.926 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:29.927 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:29.927 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:47:29.928 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:29.928 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:29.929 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:29.929 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:26.899674Z" 2022-09-03T05:47:29.930 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:29.930 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:29.931 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:47:29.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:29.932 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:47:29.932 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 0, 2022-09-03T05:47:29.933 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 0 2022-09-03T05:47:29.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:29.934 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:47:29Z" 2022-09-03T05:47:29.934 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:47:29.943 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 4093329069989467412 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002768", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:26.029653Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000004881", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:17.095630Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000006434", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:37:06.825525Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000000796", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:26.899674Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:47:29Z" } 2022-09-03T05:47:29.944 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002768', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:26.029653Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004881', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:17.095630Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006434', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:37:06.825525Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000000796', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:26.899674Z'}}] 2022-09-03T05:47:29.945 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000004741', 2: '00000000000000000000:00000000000000006434', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:29.946 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000004881', 2: '00000000000000000000:00000000000000006434', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:29.947 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2022-09-03T05:47:29.947 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone1 target=test-zone3 2022-09-03T05:47:29.948 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:29.949 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:29.950 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:30.007 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:30.008 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:30.078 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:47:30.078 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:30.079 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002768", 2022-09-03T05:47:30.079 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:31:26.029653Z" 2022-09-03T05:47:30.080 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:30.080 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:30.080 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000004881", 2022-09-03T05:47:30.080 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:47:17.095630Z" 2022-09-03T05:47:30.081 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:30.081 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:30.082 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:47:30.082 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:37:06.825525Z" 2022-09-03T05:47:30.082 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:30.083 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:30.083 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:47:30.084 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:44:26.899674Z" 2022-09-03T05:47:30.084 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:30.085 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:47:30.087 DEBUG:rgw_multi.tests:data markers for zone=test-zone1 markers={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000004881', 2: '00000000000000000000:00000000000000006434', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:30.088 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2022-09-03T05:47:30.089 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:30.089 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:30.091 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:30.144 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:30.144 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:30.193 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:47:30.194 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:47:30.194 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:47:30.196 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:47:30.196 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:47:30.196 INFO:teuthology.orchestra.run.smithi181.stdout: "instance_id": 16186606431531372785 2022-09-03T05:47:30.197 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:30.198 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:47:30.198 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:30.199 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:47:30.199 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:30.200 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:30.200 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002768", 2022-09-03T05:47:30.200 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:30.201 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:30.202 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:30.202 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:31:26.029653Z" 2022-09-03T05:47:30.202 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:30.203 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:30.203 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:30.203 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:47:30.204 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:30.204 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:30.204 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000004881", 2022-09-03T05:47:30.205 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:30.206 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:30.207 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:30.207 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:47:17.095630Z" 2022-09-03T05:47:30.209 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:30.209 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:30.210 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:30.210 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:47:30.211 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:30.211 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:30.211 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:47:30.212 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:30.212 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:30.213 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:30.214 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:37:06.825525Z" 2022-09-03T05:47:30.214 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:30.215 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:30.215 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:30.216 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:47:30.217 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:30.217 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:30.217 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:47:30.218 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:30.218 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:30.220 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:30.220 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:26.899674Z" 2022-09-03T05:47:30.221 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:30.221 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:30.221 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:47:30.222 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:30.222 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:47:30.222 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 0, 2022-09-03T05:47:30.222 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 0 2022-09-03T05:47:30.223 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:30.223 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:47:30Z" 2022-09-03T05:47:30.223 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:47:30.225 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 16186606431531372785 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002768", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:26.029653Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000004881", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:17.095630Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000006434", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:37:06.825525Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000000796", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:26.899674Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:47:30Z" } 2022-09-03T05:47:30.225 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002768', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:26.029653Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004881', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:17.095630Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006434', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:37:06.825525Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000000796', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:26.899674Z'}}] 2022-09-03T05:47:30.226 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000004881', 2: '00000000000000000000:00000000000000006434', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:30.227 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000004881', 2: '00000000000000000000:00000000000000006434', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:30.227 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2022-09-03T05:47:30.228 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone1 target=test-zone4 2022-09-03T05:47:30.228 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:30.229 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:30.229 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:30.278 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:30.279 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:30.325 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:47:30.326 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:30.326 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002768", 2022-09-03T05:47:30.326 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:31:26.029653Z" 2022-09-03T05:47:30.327 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:30.327 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:30.328 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000004881", 2022-09-03T05:47:30.328 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:47:17.095630Z" 2022-09-03T05:47:30.332 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:30.333 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:30.334 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:47:30.334 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:37:06.825525Z" 2022-09-03T05:47:30.334 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:30.335 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:30.335 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:47:30.336 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:44:26.899674Z" 2022-09-03T05:47:30.336 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:30.337 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:47:30.339 DEBUG:rgw_multi.tests:data markers for zone=test-zone1 markers={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000004881', 2: '00000000000000000000:00000000000000006434', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:30.340 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone4 source_zone=test-zone1 2022-09-03T05:47:30.341 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:30.342 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:30.343 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:30.404 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:30.405 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:30.469 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:47:30.469 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:47:30.470 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:47:30.470 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:47:30.470 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:47:30.471 INFO:teuthology.orchestra.run.smithi189.stdout: "instance_id": 13887610924008747321 2022-09-03T05:47:30.471 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:30.471 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:47:30.472 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:30.472 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:47:30.472 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:30.473 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:30.473 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000002768", 2022-09-03T05:47:30.473 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:30.474 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:30.474 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:30.475 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:31:26.029653Z" 2022-09-03T05:47:30.475 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:30.476 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:30.476 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:30.477 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:47:30.477 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:30.478 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:30.478 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000004881", 2022-09-03T05:47:30.479 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:30.479 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:30.479 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:30.480 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:47:17.095630Z" 2022-09-03T05:47:30.480 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:30.481 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:30.481 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:30.481 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:47:30.482 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:30.482 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:30.483 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:47:30.483 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:30.483 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:30.484 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:30.484 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:37:06.825525Z" 2022-09-03T05:47:30.484 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:30.484 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:30.485 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:30.485 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:47:30.485 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:30.485 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:30.486 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:47:30.486 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:30.486 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:30.487 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:30.487 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:26.899674Z" 2022-09-03T05:47:30.487 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:30.487 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:30.488 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:47:30.488 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:30.488 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:47:30.488 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 0, 2022-09-03T05:47:30.489 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 0 2022-09-03T05:47:30.489 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:30.489 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:47:30Z" 2022-09-03T05:47:30.489 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:47:30.491 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 13887610924008747321 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002768", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:26.029653Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000004881", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:17.095630Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000006434", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:37:06.825525Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000000796", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:26.899674Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:47:30Z" } 2022-09-03T05:47:30.492 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002768', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:26.029653Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004881', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:17.095630Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006434', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:37:06.825525Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000000796', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:26.899674Z'}}] 2022-09-03T05:47:30.493 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000004881', 2: '00000000000000000000:00000000000000006434', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:30.493 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000004881', 2: '00000000000000000000:00000000000000006434', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:30.494 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone4 source_zone=test-zone1 2022-09-03T05:47:30.495 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone2 target=test-zone1 2022-09-03T05:47:30.495 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:30.496 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:30.496 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:30.546 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:30.547 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:30.610 INFO:teuthology.orchestra.run.smithi189.stdout:[ 2022-09-03T05:47:30.611 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:30.611 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:47:30.611 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:31:03.799590Z" 2022-09-03T05:47:30.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:30.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:30.612 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000004881", 2022-09-03T05:47:30.612 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:47:17.795310Z" 2022-09-03T05:47:30.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:30.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:30.613 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:47:30.613 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:34:57.577753Z" 2022-09-03T05:47:30.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:30.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:30.614 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:47:30.615 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:44:27.002354Z" 2022-09-03T05:47:30.615 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:30.615 INFO:teuthology.orchestra.run.smithi189.stdout:] 2022-09-03T05:47:30.616 DEBUG:rgw_multi.tests:data markers for zone=test-zone2 markers={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004881', 2: '00000000000000000000:00000000000000006434', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:30.617 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2022-09-03T05:47:30.618 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:30.618 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:30.619 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:30.674 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:30.674 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:30.741 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:47:30.742 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:47:30.742 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:47:30.743 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:47:30.743 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:47:30.743 INFO:teuthology.orchestra.run.smithi181.stdout: "instance_id": 5551903584477280978 2022-09-03T05:47:30.744 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:30.744 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:47:30.745 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:30.745 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:47:30.745 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:30.745 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:30.746 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:47:30.746 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:30.746 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:30.747 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:30.747 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:31:03.799590Z" 2022-09-03T05:47:30.748 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:30.749 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:30.749 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:30.749 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:47:30.750 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:30.750 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:30.751 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000004881", 2022-09-03T05:47:30.751 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:30.752 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:30.752 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:30.753 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:47:17.795310Z" 2022-09-03T05:47:30.753 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:30.754 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:30.754 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:30.755 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:47:30.755 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:30.756 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:30.756 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:47:30.757 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:30.757 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:30.759 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:30.759 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:34:57.577753Z" 2022-09-03T05:47:30.759 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:30.760 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:30.760 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:30.760 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:47:30.761 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:30.761 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:30.761 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:47:30.761 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:30.762 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:30.762 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:30.763 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:27.002354Z" 2022-09-03T05:47:30.763 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:30.764 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:30.765 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:47:30.765 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:30.765 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:47:30.765 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 0, 2022-09-03T05:47:30.766 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 0 2022-09-03T05:47:30.766 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:30.766 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:47:30Z" 2022-09-03T05:47:30.766 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:47:30.768 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 5551903584477280978 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002627", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:03.799590Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000004881", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:17.795310Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000006434", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:34:57.577753Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000000796", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:27.002354Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:47:30Z" } 2022-09-03T05:47:30.768 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002627', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:03.799590Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004881', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:17.795310Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006434', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:34:57.577753Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000000796', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:27.002354Z'}}] 2022-09-03T05:47:30.769 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004881', 2: '00000000000000000000:00000000000000006434', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:30.769 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004881', 2: '00000000000000000000:00000000000000006434', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:30.770 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2022-09-03T05:47:30.771 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone2 target=test-zone3 2022-09-03T05:47:30.772 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:30.773 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:30.774 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:30.825 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:30.825 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:30.871 INFO:teuthology.orchestra.run.smithi189.stdout:[ 2022-09-03T05:47:30.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:30.872 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:47:30.872 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:31:03.799590Z" 2022-09-03T05:47:30.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:30.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:30.873 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000004881", 2022-09-03T05:47:30.873 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:47:17.795310Z" 2022-09-03T05:47:30.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:30.874 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:30.874 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:47:30.874 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:34:57.577753Z" 2022-09-03T05:47:30.875 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:30.875 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:30.875 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:47:30.876 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:44:27.002354Z" 2022-09-03T05:47:30.876 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:30.876 INFO:teuthology.orchestra.run.smithi189.stdout:] 2022-09-03T05:47:30.876 DEBUG:rgw_multi.tests:data markers for zone=test-zone2 markers={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004881', 2: '00000000000000000000:00000000000000006434', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:30.877 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone3 source_zone=test-zone2 2022-09-03T05:47:30.878 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:30.879 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:30.880 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:30.933 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:30.934 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:31.002 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:47:31.003 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:47:31.004 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:47:31.004 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:47:31.005 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:47:31.005 INFO:teuthology.orchestra.run.smithi181.stdout: "instance_id": 14474342186618979890 2022-09-03T05:47:31.005 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:31.006 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:47:31.007 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:31.007 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:47:31.007 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:31.008 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:31.008 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:47:31.009 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:31.009 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:31.010 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:31.011 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:31:03.799590Z" 2022-09-03T05:47:31.011 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:31.012 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:31.012 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:31.013 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:47:31.013 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:31.014 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:31.014 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000004881", 2022-09-03T05:47:31.014 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:31.015 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:31.015 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:31.015 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:47:17.795310Z" 2022-09-03T05:47:31.016 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:31.016 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:31.016 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:31.017 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:47:31.017 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:31.017 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:31.018 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:47:31.018 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:31.018 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:31.019 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:31.019 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:34:57.577753Z" 2022-09-03T05:47:31.019 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:31.019 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:31.020 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:31.020 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:47:31.020 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:31.021 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:31.021 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:47:31.021 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:31.021 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:31.022 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:31.022 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:27.002354Z" 2022-09-03T05:47:31.023 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:31.023 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:31.023 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:47:31.024 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:31.024 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:47:31.024 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 0, 2022-09-03T05:47:31.025 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 0 2022-09-03T05:47:31.025 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:31.025 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:47:30Z" 2022-09-03T05:47:31.026 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:47:31.028 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 14474342186618979890 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002627", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:03.799590Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000004881", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:17.795310Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000006434", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:34:57.577753Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000000796", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:27.002354Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:47:30Z" } 2022-09-03T05:47:31.028 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002627', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:03.799590Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004881', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:17.795310Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006434', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:34:57.577753Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000000796', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:27.002354Z'}}] 2022-09-03T05:47:31.029 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004881', 2: '00000000000000000000:00000000000000006434', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:31.030 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004881', 2: '00000000000000000000:00000000000000006434', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:31.030 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone3 source_zone=test-zone2 2022-09-03T05:47:31.031 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone2 target=test-zone4 2022-09-03T05:47:31.032 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone3 target=test-zone1 2022-09-03T05:47:31.033 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:31.034 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:31.035 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:31.083 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:31.083 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:31.145 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:47:31.146 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:31.146 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:47:31.146 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:31:06.854659Z" 2022-09-03T05:47:31.147 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:31.147 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:31.147 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000004599", 2022-09-03T05:47:31.147 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:47:22.359142Z" 2022-09-03T05:47:31.148 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:31.148 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:31.148 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000005588", 2022-09-03T05:47:31.148 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:34:35.658751Z" 2022-09-03T05:47:31.149 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:31.149 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:31.149 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:47:31.149 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:44:27.009917Z" 2022-09-03T05:47:31.150 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:31.150 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:47:31.151 DEBUG:rgw_multi.tests:data markers for zone=test-zone3 markers={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004599', 2: '00000000000000000000:00000000000000005588', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:31.152 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone1 source_zone=test-zone3 2022-09-03T05:47:31.152 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:31.153 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:31.154 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone3 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:31.205 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:31.205 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:31.255 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:47:31.256 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:47:31.256 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:47:31.256 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:47:31.257 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:47:31.257 INFO:teuthology.orchestra.run.smithi181.stdout: "instance_id": 4175403477120006881 2022-09-03T05:47:31.258 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:31.259 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:47:31.259 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:31.260 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:47:31.260 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:31.261 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:31.261 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:47:31.262 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:31.262 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:31.263 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:31.263 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:31:06.854659Z" 2022-09-03T05:47:31.263 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:31.264 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:31.264 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:31.265 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:47:31.265 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:31.266 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:31.267 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000004319", 2022-09-03T05:47:31.268 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:31.268 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:31.268 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:31.269 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:47:03.319436Z" 2022-09-03T05:47:31.269 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:31.269 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:31.269 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:31.270 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:47:31.270 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:31.271 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:31.271 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000005588", 2022-09-03T05:47:31.271 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:31.272 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:31.272 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:31.273 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:34:35.658751Z" 2022-09-03T05:47:31.273 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:31.273 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:31.274 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:31.274 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:47:31.274 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:31.274 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:31.275 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:47:31.275 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:31.275 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:31.275 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:31.276 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:27.009917Z" 2022-09-03T05:47:31.276 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:31.276 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:31.277 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:47:31.277 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:31.277 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:47:31.277 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 0, 2022-09-03T05:47:31.278 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 0 2022-09-03T05:47:31.278 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:31.278 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:47:31Z" 2022-09-03T05:47:31.278 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:47:31.280 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 4175403477120006881 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002627", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:06.854659Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000004319", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:03.319436Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000005588", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:34:35.658751Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000000796", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:27.009917Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:47:31Z" } 2022-09-03T05:47:31.281 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002627', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:06.854659Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004319', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:03.319436Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000005588', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:34:35.658751Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000000796', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:27.009917Z'}}] 2022-09-03T05:47:31.282 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004599', 2: '00000000000000000000:00000000000000005588', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:31.283 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004319', 2: '00000000000000000000:00000000000000005588', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:31.283 WARNING:rgw_multi.tests:data of zone test-zone1 behind zone test-zone3: shard=1 master=00000000000000000000:00000000000000004599 target=00000000000000000000:00000000000000004319 2022-09-03T05:47:36.284 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:36.285 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:36.286 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone3 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:36.344 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:36.345 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:36.414 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:47:36.414 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:47:36.415 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:47:36.415 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:47:36.416 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:47:36.416 INFO:teuthology.orchestra.run.smithi181.stdout: "instance_id": 4175403477120006881 2022-09-03T05:47:36.417 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:36.417 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:47:36.418 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:36.419 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:47:36.419 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:36.420 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:36.420 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:47:36.420 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:36.421 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:36.421 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:36.421 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:31:06.854659Z" 2022-09-03T05:47:36.422 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:36.422 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:36.422 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:36.423 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:47:36.423 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:36.424 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:36.424 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000004319", 2022-09-03T05:47:36.425 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:36.425 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:36.425 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:36.426 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:47:03.319436Z" 2022-09-03T05:47:36.426 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:36.427 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:36.427 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:36.428 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:47:36.428 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:36.429 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:36.429 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000005588", 2022-09-03T05:47:36.430 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:36.430 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:36.431 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:36.432 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:34:35.658751Z" 2022-09-03T05:47:36.432 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:36.433 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:36.433 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:36.433 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:47:36.435 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:36.435 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:36.436 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:47:36.436 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:36.437 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:36.437 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:36.437 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:27.009917Z" 2022-09-03T05:47:36.438 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:36.438 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:36.439 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:47:36.439 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:36.440 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:47:36.441 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 0, 2022-09-03T05:47:36.441 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 0 2022-09-03T05:47:36.441 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:36.442 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:47:36Z" 2022-09-03T05:47:36.442 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:47:36.445 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 4175403477120006881 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002627", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:06.854659Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000004319", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:03.319436Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000005588", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:34:35.658751Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000000796", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:27.009917Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:47:36Z" } 2022-09-03T05:47:36.446 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002627', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:06.854659Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004319', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:03.319436Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000005588', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:34:35.658751Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000000796', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:27.009917Z'}}] 2022-09-03T05:47:36.447 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004599', 2: '00000000000000000000:00000000000000005588', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:36.448 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004319', 2: '00000000000000000000:00000000000000005588', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:36.448 WARNING:rgw_multi.tests:data of zone test-zone1 behind zone test-zone3: shard=1 master=00000000000000000000:00000000000000004599 target=00000000000000000000:00000000000000004319 2022-09-03T05:47:41.449 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:41.451 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:41.452 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone3 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:41.511 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:41.512 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:41.559 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:47:41.559 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:47:41.559 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:47:41.560 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:47:41.560 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:47:41.560 INFO:teuthology.orchestra.run.smithi181.stdout: "instance_id": 4175403477120006881 2022-09-03T05:47:41.561 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:41.561 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:47:41.561 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:41.561 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:47:41.562 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:41.562 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:41.562 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:47:41.563 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:41.563 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:41.563 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:41.563 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:31:06.854659Z" 2022-09-03T05:47:41.564 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:41.564 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:41.564 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:41.565 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:47:41.565 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:41.565 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:41.566 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000004599", 2022-09-03T05:47:41.566 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:41.566 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:41.567 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:41.567 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:47:22.359142Z" 2022-09-03T05:47:41.567 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:41.568 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:41.568 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:41.568 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:47:41.568 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:41.569 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:41.569 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000005588", 2022-09-03T05:47:41.569 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:41.569 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:41.570 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:41.570 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:34:35.658751Z" 2022-09-03T05:47:41.570 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:41.570 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:41.571 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:41.571 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:47:41.571 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:47:41.571 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:47:41.572 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:47:41.572 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:47:41.573 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:47:41.573 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:47:41.573 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:27.009917Z" 2022-09-03T05:47:41.573 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:41.574 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:41.574 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:47:41.574 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:41.574 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:47:41.575 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 0, 2022-09-03T05:47:41.575 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 0 2022-09-03T05:47:41.575 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:41.576 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:47:41Z" 2022-09-03T05:47:41.576 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:47:41.577 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 4175403477120006881 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002627", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:06.854659Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000004599", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:22.359142Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000005588", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:34:35.658751Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000000796", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:27.009917Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:47:41Z" } 2022-09-03T05:47:41.578 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002627', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:06.854659Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004599', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:22.359142Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000005588', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:34:35.658751Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000000796', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:27.009917Z'}}] 2022-09-03T05:47:41.579 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004599', 2: '00000000000000000000:00000000000000005588', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:41.579 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004599', 2: '00000000000000000000:00000000000000005588', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:41.580 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone1 source_zone=test-zone3 2022-09-03T05:47:41.580 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone3 target=test-zone2 2022-09-03T05:47:41.581 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:41.581 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:41.582 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:41.633 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:41.634 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:41.714 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:47:41.714 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:41.715 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:47:41.715 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:31:06.854659Z" 2022-09-03T05:47:41.715 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:41.716 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:41.716 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000004599", 2022-09-03T05:47:41.717 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:47:22.359142Z" 2022-09-03T05:47:41.717 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:41.718 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:41.718 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000005588", 2022-09-03T05:47:41.719 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:34:35.658751Z" 2022-09-03T05:47:41.719 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:41.720 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:41.720 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:47:41.720 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:44:27.009917Z" 2022-09-03T05:47:41.721 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:41.721 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:47:41.722 DEBUG:rgw_multi.tests:data markers for zone=test-zone3 markers={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004599', 2: '00000000000000000000:00000000000000005588', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:41.723 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone2 source_zone=test-zone3 2022-09-03T05:47:41.724 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:41.724 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:41.725 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone3 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:41.788 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:41.789 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:41.848 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:47:41.849 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:47:41.849 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:47:41.849 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:47:41.849 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:47:41.850 INFO:teuthology.orchestra.run.smithi189.stdout: "instance_id": 1043408315071002039 2022-09-03T05:47:41.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:41.850 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:47:41.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:41.851 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:47:41.851 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:41.851 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:41.851 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:47:41.852 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:41.852 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:41.852 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:41.852 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:31:06.854659Z" 2022-09-03T05:47:41.853 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:41.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:41.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:41.853 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:47:41.854 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:41.854 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:41.854 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000004599", 2022-09-03T05:47:41.854 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:41.855 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:41.855 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:41.855 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:47:22.359142Z" 2022-09-03T05:47:41.855 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:41.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:41.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:41.856 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:47:41.856 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:41.857 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:41.857 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000005588", 2022-09-03T05:47:41.857 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:41.857 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:41.858 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:41.858 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:34:35.658751Z" 2022-09-03T05:47:41.858 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:41.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:41.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:41.859 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:47:41.859 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:41.859 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:47:41.860 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:47:41.860 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:47:41.860 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:47:41.860 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:47:41.861 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:27.009917Z" 2022-09-03T05:47:41.861 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:41.861 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:41.861 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:47:41.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:41.862 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:47:41.862 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 0, 2022-09-03T05:47:41.862 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 0 2022-09-03T05:47:41.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:41.863 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:47:41Z" 2022-09-03T05:47:41.863 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:47:41.864 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 1043408315071002039 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002627", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:06.854659Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000004599", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:22.359142Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000005588", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:34:35.658751Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000000796", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:27.009917Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:47:41Z" } 2022-09-03T05:47:41.865 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002627', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:06.854659Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004599', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:22.359142Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000005588', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:34:35.658751Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000000796', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:27.009917Z'}}] 2022-09-03T05:47:41.865 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004599', 2: '00000000000000000000:00000000000000005588', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:41.866 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004599', 2: '00000000000000000000:00000000000000005588', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:47:41.866 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone2 source_zone=test-zone3 2022-09-03T05:47:41.867 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone3 target=test-zone4 2022-09-03T05:47:41.867 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:41.868 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:41.868 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:41.915 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:41.916 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:41.972 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:47:41.972 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:41.973 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002768", 2022-09-03T05:47:41.973 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:31:26.029653Z" 2022-09-03T05:47:41.976 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:41.977 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:41.977 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000004881", 2022-09-03T05:47:41.978 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:47:17.095630Z" 2022-09-03T05:47:41.979 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:41.979 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:41.979 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:47:41.981 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:37:06.825525Z" 2022-09-03T05:47:41.981 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:41.981 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:41.982 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:47:41.982 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:44:26.899674Z" 2022-09-03T05:47:41.983 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:41.983 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:47:41.984 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:47:41.985 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:47:41.986 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog autotrim --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:47:42.037 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:42.037 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:42.095 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'list', '--shard-id', '0', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:42.096 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'list', '--shard-id', '0', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:42.096 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog list --shard-id 0 --max-entries 1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:42.146 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:42.147 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:42.196 INFO:teuthology.orchestra.run.smithi181.stdout:[] 2022-09-03T05:47:42.197 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'list', '--shard-id', '1', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:42.198 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'list', '--shard-id', '1', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:42.199 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog list --shard-id 1 --max-entries 1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:42.247 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:42.248 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:42.304 INFO:teuthology.orchestra.run.smithi181.stdout:[] 2022-09-03T05:47:42.305 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'list', '--shard-id', '2', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:42.306 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'list', '--shard-id', '2', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:42.307 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog list --shard-id 2 --max-entries 1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:42.356 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:42.356 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:42.404 INFO:teuthology.orchestra.run.smithi181.stdout:[] 2022-09-03T05:47:42.406 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'list', '--shard-id', '3', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:42.407 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'list', '--shard-id', '3', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:42.408 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog list --shard-id 3 --max-entries 1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:42.457 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:42.458 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:42.525 INFO:teuthology.orchestra.run.smithi181.stdout:[] 2022-09-03T05:47:42.528 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:42.529 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:42.531 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:42.582 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:42.583 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:42.637 INFO:teuthology.orchestra.run.smithi189.stdout:[ 2022-09-03T05:47:42.637 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:42.637 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:47:42.638 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:31:03.799590Z" 2022-09-03T05:47:42.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:42.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:42.639 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000004881", 2022-09-03T05:47:42.639 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:47:17.795310Z" 2022-09-03T05:47:42.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:42.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:42.640 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:47:42.641 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:34:57.577753Z" 2022-09-03T05:47:42.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:42.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:42.642 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:47:42.643 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:44:27.002354Z" 2022-09-03T05:47:42.643 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:42.644 INFO:teuthology.orchestra.run.smithi189.stdout:] 2022-09-03T05:47:42.646 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'autotrim', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:47:42.647 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'autotrim', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:47:42.647 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog autotrim --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:47:42.693 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:42.694 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:43.746 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'list', '--shard-id', '0', '--max-entries', '1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:43.747 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'list', '--shard-id', '0', '--max-entries', '1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:43.748 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog list --shard-id 0 --max-entries 1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:43.799 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:43.799 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:43.853 INFO:teuthology.orchestra.run.smithi189.stdout:[] 2022-09-03T05:47:43.855 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'list', '--shard-id', '1', '--max-entries', '1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:43.856 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'list', '--shard-id', '1', '--max-entries', '1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:43.857 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog list --shard-id 1 --max-entries 1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:43.907 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:43.908 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:43.963 INFO:teuthology.orchestra.run.smithi189.stdout:[] 2022-09-03T05:47:43.965 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'list', '--shard-id', '2', '--max-entries', '1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:43.966 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'list', '--shard-id', '2', '--max-entries', '1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:43.967 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog list --shard-id 2 --max-entries 1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:44.018 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:44.019 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:44.058 INFO:teuthology.orchestra.run.smithi189.stdout:[] 2022-09-03T05:47:44.060 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'list', '--shard-id', '3', '--max-entries', '1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:44.061 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'list', '--shard-id', '3', '--max-entries', '1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:44.063 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog list --shard-id 3 --max-entries 1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:44.149 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:44.150 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:44.192 INFO:teuthology.orchestra.run.smithi189.stdout:[] 2022-09-03T05:47:44.194 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:44.195 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:44.196 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:44.250 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:44.251 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:44.322 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:47:44.323 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:44.323 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002768", 2022-09-03T05:47:44.323 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:31:26.029653Z" 2022-09-03T05:47:44.324 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:44.324 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:44.324 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000004881", 2022-09-03T05:47:44.324 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:47:17.095630Z" 2022-09-03T05:47:44.325 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:44.325 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:44.325 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000006434", 2022-09-03T05:47:44.325 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:37:06.825525Z" 2022-09-03T05:47:44.326 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:47:44.326 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:47:44.326 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:47:44.327 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:44:26.899674Z" 2022-09-03T05:47:44.327 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:47:44.327 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:47:44.328 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:47:44.329 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:47:44.330 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog autotrim --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:47:44.377 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:44.378 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:44.436 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'list', '--shard-id', '0', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:44.437 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'list', '--shard-id', '0', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:44.439 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog list --shard-id 0 --max-entries 1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:44.488 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:44.489 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:44.543 INFO:teuthology.orchestra.run.smithi181.stdout:[] 2022-09-03T05:47:44.546 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'list', '--shard-id', '1', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:44.547 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'list', '--shard-id', '1', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:44.548 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog list --shard-id 1 --max-entries 1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:44.598 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:44.598 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:44.668 INFO:teuthology.orchestra.run.smithi181.stdout:[] 2022-09-03T05:47:44.670 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'list', '--shard-id', '2', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:44.671 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'list', '--shard-id', '2', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:44.672 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog list --shard-id 2 --max-entries 1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:44.722 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:44.722 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:44.762 INFO:teuthology.orchestra.run.smithi181.stdout:[] 2022-09-03T05:47:44.764 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'list', '--shard-id', '3', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:44.764 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'list', '--shard-id', '3', '--max-entries', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:47:44.765 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog list --shard-id 3 --max-entries 1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:47:44.815 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:44.816 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:44.864 INFO:teuthology.orchestra.run.smithi181.stdout:[] 2022-09-03T05:47:44.865 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_datalog_autotrim ... ok 2022-09-03T05:47:44.867 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--sync-from-all=false', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:47:44.868 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--sync-from-all=false', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:47:44.868 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --sync-from-all=false --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:47:44.919 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:44.919 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:44.940 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:47:44.938+0000 7f7fc06d1600 0 failed reading obj info from .rgw.root:zone_info.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4: (2) No such file or directory 2022-09-03T05:47:44.940 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:47:44.938+0000 7f7fc06d1600 0 WARNING: could not read zone params for zone id=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4 name=test-zone3 2022-09-03T05:47:44.941 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:47:44.939+0000 7f7fc06d1600 0 failed reading obj info from .rgw.root:zone_info.91786ef1-cdb4-4061-8a3e-2a28cc497285: (2) No such file or directory 2022-09-03T05:47:44.941 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:47:44.939+0000 7f7fc06d1600 0 WARNING: could not read zone params for zone id=91786ef1-cdb4-4061-8a3e-2a28cc497285 name=test-zone1 2022-09-03T05:47:44.945 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:47:44.946 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:47:44.947 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone2", 2022-09-03T05:47:44.947 INFO:teuthology.orchestra.run.smithi189.stdout: "domain_root": "test-zone2.rgw.meta:root", 2022-09-03T05:47:44.947 INFO:teuthology.orchestra.run.smithi189.stdout: "control_pool": "test-zone2.rgw.control", 2022-09-03T05:47:44.947 INFO:teuthology.orchestra.run.smithi189.stdout: "gc_pool": "test-zone2.rgw.log:gc", 2022-09-03T05:47:44.948 INFO:teuthology.orchestra.run.smithi189.stdout: "lc_pool": "test-zone2.rgw.log:lc", 2022-09-03T05:47:44.948 INFO:teuthology.orchestra.run.smithi189.stdout: "log_pool": "test-zone2.rgw.log", 2022-09-03T05:47:44.948 INFO:teuthology.orchestra.run.smithi189.stdout: "intent_log_pool": "test-zone2.rgw.log:intent", 2022-09-03T05:47:44.949 INFO:teuthology.orchestra.run.smithi189.stdout: "usage_log_pool": "test-zone2.rgw.log:usage", 2022-09-03T05:47:44.949 INFO:teuthology.orchestra.run.smithi189.stdout: "roles_pool": "test-zone2.rgw.meta:roles", 2022-09-03T05:47:44.949 INFO:teuthology.orchestra.run.smithi189.stdout: "reshard_pool": "test-zone2.rgw.log:reshard", 2022-09-03T05:47:44.949 INFO:teuthology.orchestra.run.smithi189.stdout: "user_keys_pool": "test-zone2.rgw.meta:users.keys", 2022-09-03T05:47:44.950 INFO:teuthology.orchestra.run.smithi189.stdout: "user_email_pool": "test-zone2.rgw.meta:users.email", 2022-09-03T05:47:44.950 INFO:teuthology.orchestra.run.smithi189.stdout: "user_swift_pool": "test-zone2.rgw.meta:users.swift", 2022-09-03T05:47:44.950 INFO:teuthology.orchestra.run.smithi189.stdout: "user_uid_pool": "test-zone2.rgw.meta:users.uid", 2022-09-03T05:47:44.951 INFO:teuthology.orchestra.run.smithi189.stdout: "otp_pool": "test-zone2.rgw.otp", 2022-09-03T05:47:44.951 INFO:teuthology.orchestra.run.smithi189.stdout: "system_key": { 2022-09-03T05:47:44.951 INFO:teuthology.orchestra.run.smithi189.stdout: "access_key": "TW0E8EULCERVYU84", 2022-09-03T05:47:44.952 INFO:teuthology.orchestra.run.smithi189.stdout: "secret_key": "EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP" 2022-09-03T05:47:44.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:44.952 INFO:teuthology.orchestra.run.smithi189.stdout: "placement_pools": [ 2022-09-03T05:47:44.953 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:44.953 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "default-placement", 2022-09-03T05:47:44.953 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:47:44.954 INFO:teuthology.orchestra.run.smithi189.stdout: "index_pool": "test-zone2.rgw.buckets.index", 2022-09-03T05:47:44.954 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_classes": { 2022-09-03T05:47:44.954 INFO:teuthology.orchestra.run.smithi189.stdout: "STANDARD": { 2022-09-03T05:47:44.955 INFO:teuthology.orchestra.run.smithi189.stdout: "data_pool": "test-zone2.rgw.buckets.data", 2022-09-03T05:47:44.955 INFO:teuthology.orchestra.run.smithi189.stdout: "compression_type": "random" 2022-09-03T05:47:44.955 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:44.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:44.956 INFO:teuthology.orchestra.run.smithi189.stdout: "data_extra_pool": "test-zone2.rgw.buckets.non-ec", 2022-09-03T05:47:44.956 INFO:teuthology.orchestra.run.smithi189.stdout: "index_type": 0 2022-09-03T05:47:44.956 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:44.957 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:44.957 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:47:44.957 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:47:44.958 INFO:teuthology.orchestra.run.smithi189.stdout: "notif_pool": "test-zone2.rgw.log:notif" 2022-09-03T05:47:44.958 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:47:44.960 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:47:44.960 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:47:44.961 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:47:45.010 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:47:45.011 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:47:45.061 INFO:teuthology.orchestra.run.smithi189.stderr:Sending period to new master zone 91786ef1-cdb4-4061-8a3e-2a28cc497285 2022-09-03T05:47:45.090 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:47:45.091 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:47:45.091 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 2, 2022-09-03T05:47:45.091 INFO:teuthology.orchestra.run.smithi189.stdout: "predecessor_uuid": "eff2cb82-c479-4631-8568-fb1af4159e02", 2022-09-03T05:47:45.092 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": [ 2022-09-03T05:47:45.092 INFO:teuthology.orchestra.run.smithi189.stdout: "", 2022-09-03T05:47:45.093 INFO:teuthology.orchestra.run.smithi189.stdout: "", 2022-09-03T05:47:45.093 INFO:teuthology.orchestra.run.smithi189.stdout: "1_1662183936.347465_121.1", 2022-09-03T05:47:45.093 INFO:teuthology.orchestra.run.smithi189.stdout: "" 2022-09-03T05:47:45.094 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:47:45.094 INFO:teuthology.orchestra.run.smithi189.stdout: "period_map": { 2022-09-03T05:47:45.095 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:47:45.095 INFO:teuthology.orchestra.run.smithi189.stdout: "zonegroups": [ 2022-09-03T05:47:45.096 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:45.096 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "test-zonegroup", 2022-09-03T05:47:45.097 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zonegroup", 2022-09-03T05:47:45.098 INFO:teuthology.orchestra.run.smithi189.stdout: "api_name": "", 2022-09-03T05:47:45.098 INFO:teuthology.orchestra.run.smithi189.stdout: "is_master": true, 2022-09-03T05:47:45.099 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:47:45.099 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:47:45.099 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:47:45.100 INFO:teuthology.orchestra.run.smithi189.stdout: "hostnames": [], 2022-09-03T05:47:45.100 INFO:teuthology.orchestra.run.smithi189.stdout: "hostnames_s3website": [], 2022-09-03T05:47:45.101 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:47:45.101 INFO:teuthology.orchestra.run.smithi189.stdout: "zones": [ 2022-09-03T05:47:45.101 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:45.102 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:47:45.102 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone3", 2022-09-03T05:47:45.103 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:47:45.103 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi181.front.sepia.ceph.com:8001" 2022-09-03T05:47:45.104 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:47:45.104 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:47:45.105 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:47:45.105 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:47:45.106 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:47:45.106 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:47:45.106 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": true, 2022-09-03T05:47:45.106 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:47:45.107 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:47:45.107 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:47:45.107 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:47:45.107 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:47:45.108 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:45.108 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:45.108 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:47:45.109 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone4", 2022-09-03T05:47:45.109 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:47:45.110 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi189.front.sepia.ceph.com:8001" 2022-09-03T05:47:45.110 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:47:45.111 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:47:45.111 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:47:45.112 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:47:45.112 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:47:45.113 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "pubsub", 2022-09-03T05:47:45.113 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": false, 2022-09-03T05:47:45.114 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [ 2022-09-03T05:47:45.115 INFO:teuthology.orchestra.run.smithi189.stdout: "test-zone1" 2022-09-03T05:47:45.115 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:47:45.115 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:47:45.115 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:47:45.116 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:47:45.116 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:47:45.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:45.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:45.117 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:47:45.117 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone1", 2022-09-03T05:47:45.117 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:47:45.117 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:47:45.118 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:47:45.118 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:47:45.118 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:47:45.118 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:47:45.119 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:47:45.119 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:47:45.120 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": true, 2022-09-03T05:47:45.120 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:47:45.121 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:47:45.121 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:47:45.122 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:47:45.122 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:47:45.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:45.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:45.124 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:47:45.124 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone2", 2022-09-03T05:47:45.125 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:47:45.125 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi189.front.sepia.ceph.com:8000" 2022-09-03T05:47:45.126 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:47:45.126 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:47:45.127 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:47:45.127 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:47:45.128 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:47:45.128 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:47:45.129 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": false, 2022-09-03T05:47:45.129 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:47:45.130 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:47:45.130 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:47:45.131 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:47:45.131 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:47:45.132 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:45.132 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:47:45.133 INFO:teuthology.orchestra.run.smithi189.stdout: "placement_targets": [ 2022-09-03T05:47:45.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:45.134 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "default-placement", 2022-09-03T05:47:45.136 INFO:teuthology.orchestra.run.smithi189.stdout: "tags": [], 2022-09-03T05:47:45.136 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_classes": [ 2022-09-03T05:47:45.137 INFO:teuthology.orchestra.run.smithi189.stdout: "STANDARD" 2022-09-03T05:47:45.137 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:47:45.138 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:45.138 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:47:45.139 INFO:teuthology.orchestra.run.smithi189.stdout: "default_placement": "default-placement", 2022-09-03T05:47:45.139 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:47:45.139 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_policy": { 2022-09-03T05:47:45.140 INFO:teuthology.orchestra.run.smithi189.stdout: "groups": [] 2022-09-03T05:47:45.140 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:45.140 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled_features": [ 2022-09-03T05:47:45.140 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:47:45.141 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:47:45.141 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:45.141 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:47:45.141 INFO:teuthology.orchestra.run.smithi189.stdout: "short_zone_ids": [ 2022-09-03T05:47:45.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:45.142 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:47:45.143 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 327838863 2022-09-03T05:47:45.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:45.144 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:45.144 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:47:45.145 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 1690651894 2022-09-03T05:47:45.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:45.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:45.146 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:47:45.146 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 1752562153 2022-09-03T05:47:45.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:45.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:47:45.148 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:47:45.148 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 2011734872 2022-09-03T05:47:45.149 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:45.149 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:47:45.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:45.150 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zonegroup": "test-zonegroup", 2022-09-03T05:47:45.150 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:47:45.150 INFO:teuthology.orchestra.run.smithi189.stdout: "period_config": { 2022-09-03T05:47:45.150 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_quota": { 2022-09-03T05:47:45.151 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false, 2022-09-03T05:47:45.151 INFO:teuthology.orchestra.run.smithi189.stdout: "check_on_raw": false, 2022-09-03T05:47:45.151 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size": -1, 2022-09-03T05:47:45.152 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size_kb": 0, 2022-09-03T05:47:45.152 INFO:teuthology.orchestra.run.smithi189.stdout: "max_objects": -1 2022-09-03T05:47:45.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:45.152 INFO:teuthology.orchestra.run.smithi189.stdout: "user_quota": { 2022-09-03T05:47:45.153 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false, 2022-09-03T05:47:45.153 INFO:teuthology.orchestra.run.smithi189.stdout: "check_on_raw": false, 2022-09-03T05:47:45.153 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size": -1, 2022-09-03T05:47:45.154 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size_kb": 0, 2022-09-03T05:47:45.154 INFO:teuthology.orchestra.run.smithi189.stdout: "max_objects": -1 2022-09-03T05:47:45.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:45.154 INFO:teuthology.orchestra.run.smithi189.stdout: "user_ratelimit": { 2022-09-03T05:47:45.155 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:47:45.155 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:47:45.155 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:47:45.155 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:47:45.156 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:47:45.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:45.156 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_ratelimit": { 2022-09-03T05:47:45.157 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:47:45.157 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:47:45.158 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:47:45.158 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:47:45.159 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:47:45.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:45.160 INFO:teuthology.orchestra.run.smithi189.stdout: "anonymous_ratelimit": { 2022-09-03T05:47:45.160 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:47:45.161 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:47:45.161 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:47:45.162 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:47:45.162 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:47:45.162 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:47:45.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:47:45.163 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:47:45.164 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_name": "test-realm", 2022-09-03T05:47:45.164 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:47:45.164 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:47:45.165 INFO:rgw_multi.tests:Set sync_from_all flag on zone test-zone2 to false 2022-09-03T05:48:45.167 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-49 2022-09-03T05:48:45.169 DEBUG:boto:path=/dsgvcy-49/ 2022-09-03T05:48:45.170 DEBUG:boto:auth_path=/dsgvcy-49/ 2022-09-03T05:48:45.171 DEBUG:boto:Method: PUT 2022-09-03T05:48:45.172 DEBUG:boto:Path: /dsgvcy-49/ 2022-09-03T05:48:45.173 DEBUG:boto:Data: 2022-09-03T05:48:45.174 DEBUG:boto:Headers: {} 2022-09-03T05:48:45.176 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:48:45.177 DEBUG:boto:Port: 8000 2022-09-03T05:48:45.178 DEBUG:boto:Params: {} 2022-09-03T05:48:45.180 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:48:45.180 DEBUG:boto:Token: None 2022-09-03T05:48:45.181 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:48:45 GMT /dsgvcy-49/ 2022-09-03T05:48:45.182 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1d7fJ1F+SauEdX4llyli4WJ5PgQ= 2022-09-03T05:48:45.183 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:48:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1d7fJ1F+SauEdX4llyli4WJ5PgQ=', 'Content-Length': '0'} 2022-09-03T05:48:45.215 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000000d3d8de853694df0-006312eabd-1537-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:48:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:48:45.217 DEBUG:boto:path=/dsgvcy-49/testredirect 2022-09-03T05:48:45.219 DEBUG:boto:auth_path=/dsgvcy-49/testredirect 2022-09-03T05:48:45.220 DEBUG:boto:Method: PUT 2022-09-03T05:48:45.221 DEBUG:boto:Path: /dsgvcy-49/testredirect 2022-09-03T05:48:45.221 DEBUG:boto:Data: 2022-09-03T05:48:45.222 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3FCGuEcom6i4veFJuDiBdQ==', 'Content-Length': '512', 'Expect': '100-Continue'} 2022-09-03T05:48:45.224 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:48:45.225 DEBUG:boto:Port: 8000 2022-09-03T05:48:45.226 DEBUG:boto:Params: {} 2022-09-03T05:48:45.226 DEBUG:boto:Token: None 2022-09-03T05:48:45.229 DEBUG:boto:StringToSign: PUT 3FCGuEcom6i4veFJuDiBdQ== application/octet-stream Sat, 03 Sep 2022 05:48:45 GMT /dsgvcy-49/testredirect 2022-09-03T05:48:45.230 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jXN+TnnaI/0Zn7ij+ux0bytFJdU= 2022-09-03T05:48:45.231 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3FCGuEcom6i4veFJuDiBdQ==', 'Content-Length': '512', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:48:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jXN+TnnaI/0Zn7ij+ux0bytFJdU='} 2022-09-03T05:48:45.246 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"dc5086b847289ba8b8bde149b8388175"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000296c23becf8d7e59-006312eabd-1537-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:48:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:48:45.247 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:48:45.247 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:48:45.248 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:48:45.302 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:48:45.303 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:48:45.346 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T05:48:45.361 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:48:45.361 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:48:45.361 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:48:45.362 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:48:45.362 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:48:45.362 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:48:45.363 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:48:45.363 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:48:45.363 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:48:45.363 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:48:45.364 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:48:45.364 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:48:45.211231Z" 2022-09-03T05:48:45.364 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:48:45.364 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:48:45.365 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:48:45.365 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:48:45.365 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:48:45.365 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:48:45.367 DEBUG:rgw_multi.tests:master meta markers={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:48:45.368 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:48:45.368 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:48:45.369 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:48:45.369 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:48:45.428 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:48:45.428 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:48:45.458 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:48:45.456+0000 7f73853c4600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:48:45.459 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:48:45.457+0000 7f73853c4600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:48:45.473 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:48:45.474 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:48:45.474 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:48:45.475 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:48:45.476 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:48:45.476 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:48:45.476 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:48:45.477 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:48:45.477 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:48:45.477 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:48:45.477 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:48:45.478 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:48:45.478 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:48:45.479 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:48:45.479 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:48:45.479 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:48:45.479 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:48:45.480 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:48:45.480 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:48:45.480 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:48:45.481 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:48:45.481 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:48:45.481 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:48:45.481 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:48:45.482 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:48:45.482 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:48:45.482 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:48:45.483 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:48:45.483 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:48:45.483 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:48:45.484 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:48:45.484 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:48:45.485 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:48:45.485 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:48:45.486 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:48:45.486 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:48:45.486 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:48:45.486 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184023.196251_177.1", 2022-09-03T05:48:45.487 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:48:45.487 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:48:45.487 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:48:45.488 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:47:03.196251Z", 2022-09-03T05:48:45.488 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:48:45.488 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:48:45.488 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:48:45.489 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:48:45.489 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:48:45.490 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:48:45.490 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:48:45.491 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:48:45.491 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:48:45.491 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:48:45.492 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:48:45.492 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:48:45.492 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:48:45.492 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:48:45.493 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:48:45.493 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:48:45.493 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:48:45.493 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:48:45.494 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:48:45.494 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:48:45.494 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:48:45.495 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:48:45Z" 2022-09-03T05:48:45.495 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:48:45.497 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184023.196251_177.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:03.196251Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:48:45Z" } 2022-09-03T05:48:45.497 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184023.196251_177.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:03.196251Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:48:45.498 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:48:45.499 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184023.196251_177.1', 3: ''} 2022-09-03T05:48:45.499 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662184125.211231_206.1 target=1_1662184023.196251_177.1 2022-09-03T05:48:50.500 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:48:50.501 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:48:50.502 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:48:50.559 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:48:50.559 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:48:50.604 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:48:50.602+0000 7f9b59c73600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:48:50.605 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:48:50.603+0000 7f9b59c73600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:48:50.620 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:48:50.621 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:48:50.621 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:48:50.621 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:48:50.622 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:48:50.622 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:48:50.623 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:48:50.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:48:50.623 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:48:50.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:48:50.624 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:48:50.624 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:48:50.625 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:48:50.625 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:48:50.625 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:48:50.626 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:48:50.626 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:48:50.626 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:48:50.627 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:48:50.627 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:48:50.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:48:50.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:48:50.628 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:48:50.628 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:48:50.628 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:48:50.628 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:48:50.629 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:48:50.629 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:48:50.629 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:48:50.629 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:48:50.630 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:48:50.630 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:48:50.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:48:50.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:48:50.631 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:48:50.631 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:48:50.631 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:48:50.631 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184023.196251_177.1", 2022-09-03T05:48:50.632 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:48:50.632 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:48:50.632 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:48:50.632 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:47:03.196251Z", 2022-09-03T05:48:50.633 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:48:50.633 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:48:50.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:48:50.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:48:50.634 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:48:50.634 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:48:50.634 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:48:50.634 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:48:50.635 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:48:50.635 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:48:50.635 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:48:50.635 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:48:50.636 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:48:50.636 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:48:50.636 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:48:50.636 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:48:50.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:48:50.637 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:48:50.637 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:48:50.637 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:48:50.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:48:50.638 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:48:50Z" 2022-09-03T05:48:50.638 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:48:50.640 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184023.196251_177.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:03.196251Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:48:50Z" } 2022-09-03T05:48:50.640 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184023.196251_177.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:03.196251Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:48:50.641 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:48:50.641 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184023.196251_177.1', 3: ''} 2022-09-03T05:48:50.642 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662184125.211231_206.1 target=1_1662184023.196251_177.1 2022-09-03T05:48:55.651 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:48:55.660 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:48:55.662 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:48:55.721 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:48:55.721 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:48:55.755 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:48:55.754+0000 7fa7b8dac600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:48:55.756 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:48:55.755+0000 7fa7b8dac600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:48:55.777 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:48:55.778 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:48:55.779 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:48:55.779 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:48:55.780 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:48:55.780 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:48:55.781 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:48:55.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:48:55.782 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:48:55.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:48:55.783 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:48:55.783 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:48:55.784 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:48:55.784 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:48:55.785 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:48:55.785 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:48:55.786 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:48:55.786 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:48:55.787 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:48:55.787 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:48:55.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:48:55.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:48:55.789 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:48:55.789 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:48:55.790 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:48:55.790 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:48:55.791 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:48:55.791 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:48:55.792 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:48:55.792 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:48:55.793 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:48:55.793 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:48:55.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:48:55.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:48:55.794 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:48:55.794 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:48:55.794 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:48:55.795 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184023.196251_177.1", 2022-09-03T05:48:55.795 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:48:55.795 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:48:55.795 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:48:55.796 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:47:03.196251Z", 2022-09-03T05:48:55.796 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:48:55.796 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:48:55.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:48:55.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:48:55.798 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:48:55.799 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:48:55.800 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:48:55.801 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:48:55.801 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:48:55.802 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:48:55.802 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:48:55.803 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:48:55.803 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:48:55.803 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:48:55.804 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:48:55.804 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:48:55.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:48:55.805 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:48:55.806 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:48:55.806 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:48:55.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:48:55.807 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:48:55Z" 2022-09-03T05:48:55.808 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:48:55.808 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184023.196251_177.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:03.196251Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:48:55Z" } 2022-09-03T05:48:55.813 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184023.196251_177.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:03.196251Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:48:55.814 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:48:55.815 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184023.196251_177.1', 3: ''} 2022-09-03T05:48:55.815 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662184125.211231_206.1 target=1_1662184023.196251_177.1 2022-09-03T05:49:00.817 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:00.819 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:00.820 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:49:00.877 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:49:00.877 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:49:00.908 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:49:00.906+0000 7fd35f6b7600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:00.908 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:49:00.908+0000 7fd35f6b7600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:00.921 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:49:00.922 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:49:00.922 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:49:00.922 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:49:00.923 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:49:00.923 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:49:00.923 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:49:00.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:00.924 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:49:00.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:00.925 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:49:00.926 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:00.926 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:00.926 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:49:00.927 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:00.927 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:49:00.927 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:00.927 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:49:00.928 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:49:00.928 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:00.928 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:00.928 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:00.929 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:49:00.929 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:00.929 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:00.931 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:49:00.931 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:00.931 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:49:00.932 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:00.932 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:49:00.932 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:49:00.932 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:00.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:00.933 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:00.933 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:49:00.935 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:00.935 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:00.936 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184023.196251_177.1", 2022-09-03T05:49:00.936 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:00.936 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:49:00.937 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:00.937 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:47:03.196251Z", 2022-09-03T05:49:00.938 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:49:00.938 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:00.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:00.939 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:00.940 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:49:00.940 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:00.941 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:00.941 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:49:00.941 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:00.942 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:49:00.942 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:00.943 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:49:00.943 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:49:00.943 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:00.944 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:00.944 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:49:00.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:00.945 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:49:00.946 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:49:00.946 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:49:00.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:00.947 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:49:00Z" 2022-09-03T05:49:00.948 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:49:00.949 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184023.196251_177.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:03.196251Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:49:00Z" } 2022-09-03T05:49:00.951 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184023.196251_177.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:03.196251Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:49:00.951 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:49:00.952 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184023.196251_177.1', 3: ''} 2022-09-03T05:49:00.953 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662184125.211231_206.1 target=1_1662184023.196251_177.1 2022-09-03T05:49:05.954 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:05.955 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:05.956 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:49:06.013 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:49:06.013 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:49:06.057 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:49:06.056+0000 7fc24b4d7600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:06.058 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:49:06.057+0000 7fc24b4d7600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:06.075 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:49:06.075 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:49:06.076 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:49:06.076 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:49:06.077 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:49:06.077 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:49:06.077 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:49:06.078 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:06.078 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:49:06.079 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:06.079 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:49:06.080 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:06.080 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:06.081 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:49:06.081 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:06.082 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:49:06.082 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:06.083 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:49:06.083 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:49:06.084 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:06.084 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:06.085 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:06.085 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:49:06.086 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:06.087 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:06.087 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:49:06.088 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:06.088 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:49:06.089 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:06.089 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:49:06.090 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:49:06.090 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:06.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:06.091 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:06.091 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:49:06.091 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:06.092 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:06.092 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184023.196251_177.1", 2022-09-03T05:49:06.092 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:06.092 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:49:06.093 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:06.093 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:47:03.196251Z", 2022-09-03T05:49:06.093 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:49:06.093 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:06.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:06.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:06.094 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:49:06.094 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:06.095 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:06.095 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:49:06.095 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:06.095 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:49:06.096 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:06.096 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:49:06.096 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:49:06.096 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:06.097 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:06.097 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:49:06.097 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:06.097 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:49:06.098 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:49:06.098 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:49:06.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:06.099 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:49:06Z" 2022-09-03T05:49:06.099 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:49:06.101 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184023.196251_177.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:03.196251Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:49:06Z" } 2022-09-03T05:49:06.102 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184023.196251_177.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:03.196251Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:49:06.103 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:49:06.104 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184023.196251_177.1', 3: ''} 2022-09-03T05:49:06.105 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662184125.211231_206.1 target=1_1662184023.196251_177.1 2022-09-03T05:49:11.107 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:11.108 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:11.109 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:49:11.167 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:49:11.167 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:49:11.210 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:49:11.208+0000 7fa0d7994600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:11.211 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:49:11.209+0000 7fa0d7994600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:11.224 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:49:11.224 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:49:11.224 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:49:11.225 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:49:11.225 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:49:11.225 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:49:11.226 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:49:11.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:11.226 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:49:11.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:11.227 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:49:11.228 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:11.228 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:11.229 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:49:11.229 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:11.230 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:49:11.230 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:11.231 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:49:11.231 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:49:11.232 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:11.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:11.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:11.233 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:49:11.234 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:11.234 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:11.235 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:49:11.235 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:11.236 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:49:11.236 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:11.237 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:49:11.237 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:49:11.238 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:11.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:11.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:11.239 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:49:11.240 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:11.240 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:11.241 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184023.196251_177.1", 2022-09-03T05:49:11.241 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:11.242 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:49:11.242 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:11.243 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:47:03.196251Z", 2022-09-03T05:49:11.243 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:49:11.244 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:11.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:11.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:11.245 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:49:11.246 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:11.246 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:11.247 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:49:11.247 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:11.247 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:49:11.248 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:11.248 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:49:11.249 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:49:11.249 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:11.250 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:11.250 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:49:11.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:11.251 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:49:11.252 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:49:11.252 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:49:11.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:11.253 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:49:11Z" 2022-09-03T05:49:11.254 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:49:11.256 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184023.196251_177.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:03.196251Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:49:11Z" } 2022-09-03T05:49:11.257 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184023.196251_177.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:03.196251Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:49:11.258 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:49:11.259 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184023.196251_177.1', 3: ''} 2022-09-03T05:49:11.260 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662184125.211231_206.1 target=1_1662184023.196251_177.1 2022-09-03T05:49:16.262 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:16.263 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:16.264 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:49:16.324 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:49:16.324 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:49:16.362 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:49:16.361+0000 7f0a1f78b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:16.363 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:49:16.362+0000 7f0a1f78b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:16.381 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:49:16.382 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:49:16.382 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:49:16.383 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:49:16.383 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:49:16.383 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:49:16.384 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:49:16.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:16.384 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:49:16.384 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:16.385 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:49:16.385 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:16.385 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:16.386 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:49:16.386 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:16.386 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:49:16.387 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:16.387 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:49:16.387 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:49:16.387 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:16.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:16.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:16.388 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:49:16.388 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:16.389 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:16.389 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:49:16.389 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:16.389 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:49:16.390 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:16.390 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:49:16.390 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:49:16.390 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:16.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:16.391 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:16.391 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:49:16.391 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:16.392 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:16.392 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184125.198887_203.1", 2022-09-03T05:49:16.392 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:16.392 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:49:16.393 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:16.393 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:48:45.198887Z", 2022-09-03T05:49:16.393 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:49:16.393 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:16.394 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:16.394 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:16.394 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:49:16.394 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:16.395 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:16.395 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:49:16.395 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:16.396 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:49:16.396 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:16.396 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:49:16.396 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:49:16.397 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:16.397 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:16.397 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:49:16.397 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:16.397 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:49:16.398 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:49:16.398 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:49:16.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:16.399 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:49:16Z" 2022-09-03T05:49:16.399 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:49:16.400 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184125.198887_203.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:48:45.198887Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:49:16Z" } 2022-09-03T05:49:16.401 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184125.198887_203.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:48:45.198887Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:49:16.402 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:49:16.403 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184125.198887_203.1', 3: ''} 2022-09-03T05:49:16.403 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662184125.211231_206.1 target=1_1662184125.198887_203.1 2022-09-03T05:49:21.405 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:21.407 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:21.408 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:49:21.465 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:49:21.466 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:49:21.494 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:49:21.493+0000 7fefd3685600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:21.495 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:49:21.494+0000 7fefd3685600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:21.508 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:49:21.509 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:49:21.510 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:49:21.510 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:49:21.511 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:49:21.511 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:49:21.512 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:49:21.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:21.514 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:49:21.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:21.515 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:49:21.515 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:21.515 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:21.515 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:49:21.516 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:21.516 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:49:21.516 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:21.517 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:49:21.517 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:49:21.517 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:21.517 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:21.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:21.518 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:49:21.518 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:21.518 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:21.519 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:49:21.519 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:21.519 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:49:21.519 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:21.520 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:49:21.520 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:49:21.520 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:21.521 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:21.521 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:21.521 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:49:21.521 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:21.522 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:21.522 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:49:21.522 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:21.523 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:49:21.523 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:21.523 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:48:45.211231Z", 2022-09-03T05:49:21.523 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:49:21.524 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:21.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:21.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:21.525 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:49:21.525 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:21.526 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:21.526 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:49:21.526 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:21.526 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:49:21.527 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:21.527 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:49:21.527 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:49:21.528 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:21.528 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:21.528 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:49:21.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:21.529 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:49:21.529 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:49:21.529 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:49:21.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:21.530 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:49:21Z" 2022-09-03T05:49:21.530 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:49:21.532 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184125.211231_206.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:48:45.211231Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:49:21Z" } 2022-09-03T05:49:21.532 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184125.211231_206.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:48:45.211231Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:49:21.533 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:49:21.533 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:49:21.534 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:49:21.535 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:49:21.536 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:21.538 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:21.539 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:49:21.596 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:49:21.597 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:49:21.640 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:49:21.639+0000 7f0ba5ae9600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:21.641 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:49:21.640+0000 7f0ba5ae9600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:21.658 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:49:21.659 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:49:21.659 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:49:21.659 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:49:21.660 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:49:21.660 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:49:21.660 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:49:21.661 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:21.661 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:49:21.661 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:21.661 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:49:21.663 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:21.664 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:21.664 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:49:21.665 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:21.665 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:49:21.668 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:21.669 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:49:21.669 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:49:21.670 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:21.670 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:21.671 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:21.671 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:49:21.672 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:21.673 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:21.673 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:49:21.673 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:21.674 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:49:21.674 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:21.675 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:49:21.675 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:49:21.676 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:21.676 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:21.677 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:21.677 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:49:21.677 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:21.678 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:21.678 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184023.196251_177.1", 2022-09-03T05:49:21.679 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:21.679 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:49:21.679 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:21.680 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:47:03.196251Z", 2022-09-03T05:49:21.680 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:49:21.681 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:21.681 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:21.683 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:21.684 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:49:21.685 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:21.685 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:21.686 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:49:21.686 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:21.687 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:49:21.688 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:21.689 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:49:21.689 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:49:21.690 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:21.690 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:21.691 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:49:21.691 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:21.693 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:49:21.696 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:49:21.697 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:49:21.697 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:21.698 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:49:21Z" 2022-09-03T05:49:21.699 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:49:21.701 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184023.196251_177.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:03.196251Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:49:21Z" } 2022-09-03T05:49:21.702 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184023.196251_177.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:03.196251Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:49:21.711 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:49:21.714 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184023.196251_177.1', 3: ''} 2022-09-03T05:49:21.717 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662184125.211231_206.1 target=1_1662184023.196251_177.1 2022-09-03T05:49:26.719 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:26.720 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:26.720 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:49:26.779 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:49:26.779 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:49:26.821 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:49:26.819+0000 7f26e2e10600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:26.822 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:49:26.820+0000 7f26e2e10600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:26.839 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:49:26.839 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:49:26.839 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:49:26.840 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:49:26.840 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:49:26.840 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:49:26.840 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:49:26.841 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:26.841 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:49:26.841 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:26.841 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:49:26.842 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:26.842 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:26.842 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:49:26.842 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:26.843 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:49:26.843 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:26.843 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:49:26.843 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:49:26.844 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:26.844 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:26.844 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:26.844 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:49:26.845 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:26.845 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:26.845 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:49:26.845 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:26.846 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:49:26.846 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:26.846 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:49:26.847 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:49:26.847 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:26.847 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:26.847 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:26.848 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:49:26.848 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:26.848 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:26.848 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184023.196251_177.1", 2022-09-03T05:49:26.849 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:26.849 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:49:26.849 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:26.849 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:47:03.196251Z", 2022-09-03T05:49:26.850 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:49:26.850 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:26.850 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:26.850 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:26.850 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:49:26.851 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:26.851 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:26.851 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:49:26.851 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:26.852 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:49:26.852 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:26.852 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:49:26.852 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:49:26.853 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:26.853 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:26.853 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:49:26.853 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:26.854 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:49:26.854 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:49:26.854 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:49:26.854 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:26.855 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:49:26Z" 2022-09-03T05:49:26.855 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:49:26.856 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184023.196251_177.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:03.196251Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:49:26Z" } 2022-09-03T05:49:26.857 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184023.196251_177.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:03.196251Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:49:26.857 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:49:26.858 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184023.196251_177.1', 3: ''} 2022-09-03T05:49:26.858 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662184125.211231_206.1 target=1_1662184023.196251_177.1 2022-09-03T05:49:31.860 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:31.860 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:31.861 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:49:31.918 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:49:31.918 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:49:31.958 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:49:31.956+0000 7f46c2d33600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:31.959 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:49:31.958+0000 7f46c2d33600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:31.981 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:49:31.983 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:49:31.983 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:49:31.983 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:49:31.984 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:49:31.984 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:49:31.984 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:49:31.984 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:31.985 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:49:31.985 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:31.985 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:49:31.985 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:31.986 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:31.986 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:49:31.986 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:31.987 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:49:31.988 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:31.988 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:49:31.988 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:49:31.989 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:31.989 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:31.989 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:31.990 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:49:31.990 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:31.990 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:31.990 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:49:31.991 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:31.991 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:49:31.991 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:31.991 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:49:31.992 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:49:31.992 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:31.992 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:31.993 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:31.993 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:49:31.993 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:31.993 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:31.994 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184023.196251_177.1", 2022-09-03T05:49:31.994 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:31.995 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:49:31.995 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:31.995 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:47:03.196251Z", 2022-09-03T05:49:31.996 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:49:31.996 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:31.996 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:31.996 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:31.997 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:49:31.997 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:31.997 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:31.997 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:49:31.998 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:31.998 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:49:31.999 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:31.999 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:49:31.999 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:49:32.000 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:32.000 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:32.000 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:49:32.000 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:32.001 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:49:32.001 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:49:32.001 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:49:32.001 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:32.002 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:49:31Z" 2022-09-03T05:49:32.002 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:49:32.002 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184023.196251_177.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:03.196251Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:49:31Z" } 2022-09-03T05:49:32.003 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184023.196251_177.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:03.196251Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:49:32.004 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:49:32.005 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184023.196251_177.1', 3: ''} 2022-09-03T05:49:32.005 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662184125.211231_206.1 target=1_1662184023.196251_177.1 2022-09-03T05:49:37.007 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:37.008 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:37.009 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:49:37.066 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:49:37.067 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:49:37.116 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:49:37.114+0000 7fde08f66600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:37.118 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:49:37.115+0000 7fde08f66600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:37.136 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:49:37.137 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:49:37.137 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:49:37.138 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:49:37.139 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:49:37.139 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:49:37.140 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:49:37.140 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:37.141 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:49:37.141 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:37.142 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:49:37.142 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:37.143 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:37.143 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:49:37.143 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:37.143 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:49:37.144 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:37.144 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:49:37.144 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:49:37.144 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:37.145 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:37.145 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:37.145 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:49:37.145 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:37.146 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:37.146 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:49:37.146 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:37.146 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:49:37.147 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:37.147 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:49:37.147 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:49:37.147 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:37.148 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:37.148 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:37.148 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:49:37.148 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:37.149 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:37.149 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184023.196251_177.1", 2022-09-03T05:49:37.149 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:37.149 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:49:37.150 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:37.150 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:47:03.196251Z", 2022-09-03T05:49:37.150 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:49:37.151 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:37.151 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:37.151 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:37.151 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:49:37.151 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:37.152 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:37.152 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:49:37.152 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:37.153 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:49:37.153 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:37.154 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:49:37.154 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:49:37.155 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:37.155 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:37.156 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:49:37.156 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:37.157 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:49:37.157 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:49:37.158 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:49:37.158 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:37.159 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:49:37Z" 2022-09-03T05:49:37.159 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:49:37.162 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184023.196251_177.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:03.196251Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:49:37Z" } 2022-09-03T05:49:37.163 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184023.196251_177.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:03.196251Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:49:37.164 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:49:37.165 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184023.196251_177.1', 3: ''} 2022-09-03T05:49:37.166 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662184125.211231_206.1 target=1_1662184023.196251_177.1 2022-09-03T05:49:42.167 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:42.168 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:42.168 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:49:42.226 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:49:42.226 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:49:42.268 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:49:42.266+0000 7fc668b1f600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:42.269 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:49:42.267+0000 7fc668b1f600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:42.281 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:49:42.282 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:49:42.283 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:49:42.283 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:49:42.283 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:49:42.284 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:49:42.284 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:49:42.285 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:42.285 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:49:42.286 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:42.286 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:49:42.286 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:42.286 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:42.287 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:49:42.287 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:42.287 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:49:42.288 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:42.288 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:49:42.288 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:49:42.289 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:42.289 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:42.289 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:42.290 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:49:42.290 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:42.290 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:42.290 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:49:42.291 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:42.291 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:49:42.291 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:42.291 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:49:42.292 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:49:42.292 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:42.292 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:42.293 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:42.293 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:49:42.293 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:42.293 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:42.294 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184023.196251_177.1", 2022-09-03T05:49:42.294 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:42.294 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:49:42.294 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:42.295 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:47:03.196251Z", 2022-09-03T05:49:42.295 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:49:42.295 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:42.295 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:42.296 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:42.296 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:49:42.296 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:42.296 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:42.297 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:49:42.297 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:42.297 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:49:42.297 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:42.298 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:49:42.298 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:49:42.299 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:42.299 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:42.299 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:49:42.300 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:42.300 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:49:42.300 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:49:42.301 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:49:42.301 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:42.301 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:49:42Z" 2022-09-03T05:49:42.302 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:49:42.303 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184023.196251_177.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:03.196251Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:49:42Z" } 2022-09-03T05:49:42.304 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184023.196251_177.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:03.196251Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:49:42.304 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:49:42.305 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184023.196251_177.1', 3: ''} 2022-09-03T05:49:42.306 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662184125.211231_206.1 target=1_1662184023.196251_177.1 2022-09-03T05:49:47.307 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:47.309 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:47.310 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:49:47.368 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:49:47.370 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:49:47.405 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:49:47.403+0000 7f41318dc600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:47.405 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:49:47.404+0000 7f41318dc600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:47.419 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:49:47.419 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:49:47.420 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:49:47.421 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:49:47.421 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:49:47.421 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:49:47.422 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:49:47.423 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:47.423 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:49:47.423 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:47.424 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:49:47.424 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:47.425 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:47.425 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:49:47.425 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:47.426 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:49:47.426 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:47.427 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:49:47.427 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:49:47.428 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:47.428 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:47.428 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:47.429 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:49:47.429 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:47.429 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:47.430 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:49:47.430 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:47.430 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:49:47.431 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:47.431 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:49:47.432 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:49:47.432 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:47.433 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:47.433 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:47.434 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:49:47.434 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:47.435 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:47.435 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:49:47.435 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:47.436 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:49:47.436 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:47.437 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:48:45.211231Z", 2022-09-03T05:49:47.437 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:49:47.438 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:47.438 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:47.439 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:49:47.439 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:49:47.440 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:49:47.440 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:49:47.440 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:49:47.441 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:49:47.441 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:49:47.442 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:49:47.442 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:49:47.443 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:49:47.443 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:47.444 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:49:47.444 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:49:47.445 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:47.445 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:49:47.446 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:49:47.446 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:49:47.447 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:49:47.447 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:49:47Z" 2022-09-03T05:49:47.447 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:49:47.450 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184125.211231_206.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:48:45.211231Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:49:47Z" } 2022-09-03T05:49:47.451 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184125.211231_206.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:48:45.211231Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:49:47.452 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:49:47.453 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:49:47.453 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:49:47.454 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:49:47.455 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:47.456 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:49:47.457 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:49:47.514 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:49:47.515 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:49:47.543 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:49:47.542+0000 7ff657379600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:47.544 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:49:47.543+0000 7ff657379600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:49:47.556 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:49:47.557 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:49:47.558 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:49:47.558 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:49:47.559 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:49:47.559 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:49:47.559 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:49:47.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:47.560 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:49:47.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:47.561 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:49:47.561 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:47.561 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:47.562 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:49:47.562 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:47.562 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:49:47.563 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:47.566 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:49:47.566 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:49:47.569 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:47.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:47.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:47.570 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:49:47.571 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:47.571 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:47.572 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:49:47.572 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:47.572 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:49:47.573 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:47.573 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:49:47.573 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:49:47.574 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:47.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:47.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:47.575 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:49:47.575 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:47.576 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:47.576 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:49:47.577 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:47.577 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:49:47.578 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:47.578 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:48:45.211231Z", 2022-09-03T05:49:47.578 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:49:47.578 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:47.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:47.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:47.579 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:49:47.579 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:47.580 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:49:47.580 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:49:47.581 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:49:47.582 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:49:47.582 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:49:47.582 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:49:47.583 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:49:47.583 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:47.584 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:47.584 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:49:47.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:47.585 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:49:47.586 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:49:47.586 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:49:47.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:47.587 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:49:47Z" 2022-09-03T05:49:47.587 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:49:47.589 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184125.211231_206.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:48:45.211231Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:49:47Z" } 2022-09-03T05:49:47.591 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184125.211231_206.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:48:45.211231Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:49:47.592 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:49:47.592 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:49:47.594 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:49:47.595 DEBUG:boto:path=/dsgvcy-49/ 2022-09-03T05:49:47.596 DEBUG:boto:auth_path=/dsgvcy-49/ 2022-09-03T05:49:47.597 DEBUG:boto:Method: HEAD 2022-09-03T05:49:47.597 DEBUG:boto:Path: /dsgvcy-49/ 2022-09-03T05:49:47.598 DEBUG:boto:Data: 2022-09-03T05:49:47.599 DEBUG:boto:Headers: {} 2022-09-03T05:49:47.600 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:49:47.601 DEBUG:boto:Port: 8000 2022-09-03T05:49:47.602 DEBUG:boto:Params: {} 2022-09-03T05:49:47.604 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:49:47.605 DEBUG:boto:Token: None 2022-09-03T05:49:47.606 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:49:47 GMT /dsgvcy-49/ 2022-09-03T05:49:47.607 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n1yWRzFtPJcfPhaz4io8kgnV3zw= 2022-09-03T05:49:47.609 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:49:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n1yWRzFtPJcfPhaz4io8kgnV3zw=', 'Content-Length': '0'} 2022-09-03T05:49:47.616 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002e0e58b0a788a70e-006312eafb-1528-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:49:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:49:47.617 DEBUG:boto:path=/dsgvcy-49/testredirect 2022-09-03T05:49:47.619 DEBUG:boto:auth_path=/dsgvcy-49/testredirect 2022-09-03T05:49:47.619 DEBUG:boto:Method: HEAD 2022-09-03T05:49:47.620 DEBUG:boto:Path: /dsgvcy-49/testredirect 2022-09-03T05:49:47.620 DEBUG:boto:Data: 2022-09-03T05:49:47.621 DEBUG:boto:Headers: {} 2022-09-03T05:49:47.621 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:49:47.622 DEBUG:boto:Port: 8000 2022-09-03T05:49:47.622 DEBUG:boto:Params: {} 2022-09-03T05:49:47.623 DEBUG:boto:Token: None 2022-09-03T05:49:47.624 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:49:47 GMT /dsgvcy-49/testredirect 2022-09-03T05:49:47.625 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:U7X4DxrJyIwrxhfNwYHWNl2sWEw= 2022-09-03T05:49:47.625 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:49:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:U7X4DxrJyIwrxhfNwYHWNl2sWEw=', 'Content-Length': '0'} 2022-09-03T05:49:47.627 DEBUG:boto:Response headers: [('Content-Length', '229'), ('x-amz-request-id', 'tx0000080fb85837e09325b-006312eafb-1528-test-zone2'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:49:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:49:47.628 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--redirect-zone=91786ef1-cdb4-4061-8a3e-2a28cc497285', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:49:47.629 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--redirect-zone=91786ef1-cdb4-4061-8a3e-2a28cc497285', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:49:47.630 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --redirect-zone=91786ef1-cdb4-4061-8a3e-2a28cc497285 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:49:47.682 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:49:47.682 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:49:47.701 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:49:47.700+0000 7fd2b3a42600 0 failed reading obj info from .rgw.root:zone_info.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4: (2) No such file or directory 2022-09-03T05:49:47.701 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:49:47.700+0000 7fd2b3a42600 0 WARNING: could not read zone params for zone id=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4 name=test-zone3 2022-09-03T05:49:47.702 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:49:47.700+0000 7fd2b3a42600 0 failed reading obj info from .rgw.root:zone_info.91786ef1-cdb4-4061-8a3e-2a28cc497285: (2) No such file or directory 2022-09-03T05:49:47.702 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:49:47.700+0000 7fd2b3a42600 0 WARNING: could not read zone params for zone id=91786ef1-cdb4-4061-8a3e-2a28cc497285 name=test-zone1 2022-09-03T05:49:47.705 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:49:47.706 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:49:47.706 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone2", 2022-09-03T05:49:47.706 INFO:teuthology.orchestra.run.smithi189.stdout: "domain_root": "test-zone2.rgw.meta:root", 2022-09-03T05:49:47.707 INFO:teuthology.orchestra.run.smithi189.stdout: "control_pool": "test-zone2.rgw.control", 2022-09-03T05:49:47.707 INFO:teuthology.orchestra.run.smithi189.stdout: "gc_pool": "test-zone2.rgw.log:gc", 2022-09-03T05:49:47.707 INFO:teuthology.orchestra.run.smithi189.stdout: "lc_pool": "test-zone2.rgw.log:lc", 2022-09-03T05:49:47.707 INFO:teuthology.orchestra.run.smithi189.stdout: "log_pool": "test-zone2.rgw.log", 2022-09-03T05:49:47.708 INFO:teuthology.orchestra.run.smithi189.stdout: "intent_log_pool": "test-zone2.rgw.log:intent", 2022-09-03T05:49:47.708 INFO:teuthology.orchestra.run.smithi189.stdout: "usage_log_pool": "test-zone2.rgw.log:usage", 2022-09-03T05:49:47.708 INFO:teuthology.orchestra.run.smithi189.stdout: "roles_pool": "test-zone2.rgw.meta:roles", 2022-09-03T05:49:47.711 INFO:teuthology.orchestra.run.smithi189.stdout: "reshard_pool": "test-zone2.rgw.log:reshard", 2022-09-03T05:49:47.711 INFO:teuthology.orchestra.run.smithi189.stdout: "user_keys_pool": "test-zone2.rgw.meta:users.keys", 2022-09-03T05:49:47.711 INFO:teuthology.orchestra.run.smithi189.stdout: "user_email_pool": "test-zone2.rgw.meta:users.email", 2022-09-03T05:49:47.712 INFO:teuthology.orchestra.run.smithi189.stdout: "user_swift_pool": "test-zone2.rgw.meta:users.swift", 2022-09-03T05:49:47.712 INFO:teuthology.orchestra.run.smithi189.stdout: "user_uid_pool": "test-zone2.rgw.meta:users.uid", 2022-09-03T05:49:47.713 INFO:teuthology.orchestra.run.smithi189.stdout: "otp_pool": "test-zone2.rgw.otp", 2022-09-03T05:49:47.713 INFO:teuthology.orchestra.run.smithi189.stdout: "system_key": { 2022-09-03T05:49:47.713 INFO:teuthology.orchestra.run.smithi189.stdout: "access_key": "TW0E8EULCERVYU84", 2022-09-03T05:49:47.714 INFO:teuthology.orchestra.run.smithi189.stdout: "secret_key": "EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP" 2022-09-03T05:49:47.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:47.715 INFO:teuthology.orchestra.run.smithi189.stdout: "placement_pools": [ 2022-09-03T05:49:47.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:47.715 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "default-placement", 2022-09-03T05:49:47.716 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:49:47.716 INFO:teuthology.orchestra.run.smithi189.stdout: "index_pool": "test-zone2.rgw.buckets.index", 2022-09-03T05:49:47.717 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_classes": { 2022-09-03T05:49:47.717 INFO:teuthology.orchestra.run.smithi189.stdout: "STANDARD": { 2022-09-03T05:49:47.718 INFO:teuthology.orchestra.run.smithi189.stdout: "data_pool": "test-zone2.rgw.buckets.data", 2022-09-03T05:49:47.718 INFO:teuthology.orchestra.run.smithi189.stdout: "compression_type": "random" 2022-09-03T05:49:47.719 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:47.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:47.719 INFO:teuthology.orchestra.run.smithi189.stdout: "data_extra_pool": "test-zone2.rgw.buckets.non-ec", 2022-09-03T05:49:47.720 INFO:teuthology.orchestra.run.smithi189.stdout: "index_type": 0 2022-09-03T05:49:47.721 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:47.721 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:47.722 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:49:47.722 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:49:47.723 INFO:teuthology.orchestra.run.smithi189.stdout: "notif_pool": "test-zone2.rgw.log:notif" 2022-09-03T05:49:47.723 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:49:47.725 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:49:47.727 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:49:47.728 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:49:47.777 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:49:47.778 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:49:47.819 INFO:teuthology.orchestra.run.smithi189.stderr:Sending period to new master zone 91786ef1-cdb4-4061-8a3e-2a28cc497285 2022-09-03T05:49:47.854 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:49:47.855 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:49:47.855 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 3, 2022-09-03T05:49:47.855 INFO:teuthology.orchestra.run.smithi189.stdout: "predecessor_uuid": "eff2cb82-c479-4631-8568-fb1af4159e02", 2022-09-03T05:49:47.856 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": [ 2022-09-03T05:49:47.856 INFO:teuthology.orchestra.run.smithi189.stdout: "", 2022-09-03T05:49:47.856 INFO:teuthology.orchestra.run.smithi189.stdout: "", 2022-09-03T05:49:47.857 INFO:teuthology.orchestra.run.smithi189.stdout: "1_1662183936.347465_121.1", 2022-09-03T05:49:47.857 INFO:teuthology.orchestra.run.smithi189.stdout: "" 2022-09-03T05:49:47.857 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:49:47.857 INFO:teuthology.orchestra.run.smithi189.stdout: "period_map": { 2022-09-03T05:49:47.858 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:49:47.858 INFO:teuthology.orchestra.run.smithi189.stdout: "zonegroups": [ 2022-09-03T05:49:47.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:47.858 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "test-zonegroup", 2022-09-03T05:49:47.859 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zonegroup", 2022-09-03T05:49:47.859 INFO:teuthology.orchestra.run.smithi189.stdout: "api_name": "", 2022-09-03T05:49:47.859 INFO:teuthology.orchestra.run.smithi189.stdout: "is_master": true, 2022-09-03T05:49:47.860 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:49:47.860 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:49:47.860 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:49:47.860 INFO:teuthology.orchestra.run.smithi189.stdout: "hostnames": [], 2022-09-03T05:49:47.861 INFO:teuthology.orchestra.run.smithi189.stdout: "hostnames_s3website": [], 2022-09-03T05:49:47.861 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:49:47.861 INFO:teuthology.orchestra.run.smithi189.stdout: "zones": [ 2022-09-03T05:49:47.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:47.862 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:49:47.862 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone3", 2022-09-03T05:49:47.862 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:49:47.866 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi181.front.sepia.ceph.com:8001" 2022-09-03T05:49:47.867 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:49:47.867 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:49:47.867 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:49:47.867 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:49:47.868 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:49:47.868 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:49:47.868 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": true, 2022-09-03T05:49:47.869 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:49:47.869 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:49:47.869 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:49:47.869 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:49:47.870 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:49:47.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:47.870 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:47.870 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:49:47.871 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone4", 2022-09-03T05:49:47.871 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:49:47.871 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi189.front.sepia.ceph.com:8001" 2022-09-03T05:49:47.871 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:49:47.872 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:49:47.872 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:49:47.872 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:49:47.872 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:49:47.873 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "pubsub", 2022-09-03T05:49:47.873 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": false, 2022-09-03T05:49:47.873 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [ 2022-09-03T05:49:47.873 INFO:teuthology.orchestra.run.smithi189.stdout: "test-zone1" 2022-09-03T05:49:47.874 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:49:47.874 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:49:47.874 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:49:47.875 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:49:47.875 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:49:47.875 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:47.875 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:47.876 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:49:47.876 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone1", 2022-09-03T05:49:47.877 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:49:47.877 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:49:47.877 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:49:47.877 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:49:47.878 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:49:47.878 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:49:47.878 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:49:47.879 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:49:47.879 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": true, 2022-09-03T05:49:47.879 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:49:47.879 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:49:47.880 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:49:47.880 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:49:47.880 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:49:47.880 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:47.881 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:47.881 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:49:47.881 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone2", 2022-09-03T05:49:47.881 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:49:47.882 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi189.front.sepia.ceph.com:8000" 2022-09-03T05:49:47.882 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:49:47.882 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:49:47.883 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:49:47.883 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:49:47.883 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:49:47.884 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:49:47.884 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": false, 2022-09-03T05:49:47.884 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:49:47.884 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:49:47.885 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:49:47.885 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:49:47.886 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:49:47.886 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:47.886 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:49:47.886 INFO:teuthology.orchestra.run.smithi189.stdout: "placement_targets": [ 2022-09-03T05:49:47.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:47.889 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "default-placement", 2022-09-03T05:49:47.889 INFO:teuthology.orchestra.run.smithi189.stdout: "tags": [], 2022-09-03T05:49:47.889 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_classes": [ 2022-09-03T05:49:47.890 INFO:teuthology.orchestra.run.smithi189.stdout: "STANDARD" 2022-09-03T05:49:47.890 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:49:47.890 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:47.890 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:49:47.891 INFO:teuthology.orchestra.run.smithi189.stdout: "default_placement": "default-placement", 2022-09-03T05:49:47.891 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:49:47.891 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_policy": { 2022-09-03T05:49:47.891 INFO:teuthology.orchestra.run.smithi189.stdout: "groups": [] 2022-09-03T05:49:47.892 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:47.892 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled_features": [ 2022-09-03T05:49:47.892 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:49:47.892 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:49:47.893 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:47.893 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:49:47.893 INFO:teuthology.orchestra.run.smithi189.stdout: "short_zone_ids": [ 2022-09-03T05:49:47.893 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:47.894 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:49:47.894 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 327838863 2022-09-03T05:49:47.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:47.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:47.895 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:49:47.896 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 1690651894 2022-09-03T05:49:47.896 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:47.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:47.896 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:49:47.897 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 1752562153 2022-09-03T05:49:47.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:47.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:49:47.898 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:49:47.898 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 2011734872 2022-09-03T05:49:47.898 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:47.898 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:49:47.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:47.899 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zonegroup": "test-zonegroup", 2022-09-03T05:49:47.899 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:49:47.899 INFO:teuthology.orchestra.run.smithi189.stdout: "period_config": { 2022-09-03T05:49:47.900 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_quota": { 2022-09-03T05:49:47.900 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false, 2022-09-03T05:49:47.900 INFO:teuthology.orchestra.run.smithi189.stdout: "check_on_raw": false, 2022-09-03T05:49:47.900 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size": -1, 2022-09-03T05:49:47.901 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size_kb": 0, 2022-09-03T05:49:47.901 INFO:teuthology.orchestra.run.smithi189.stdout: "max_objects": -1 2022-09-03T05:49:47.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:47.901 INFO:teuthology.orchestra.run.smithi189.stdout: "user_quota": { 2022-09-03T05:49:47.902 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false, 2022-09-03T05:49:47.902 INFO:teuthology.orchestra.run.smithi189.stdout: "check_on_raw": false, 2022-09-03T05:49:47.902 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size": -1, 2022-09-03T05:49:47.902 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size_kb": 0, 2022-09-03T05:49:47.903 INFO:teuthology.orchestra.run.smithi189.stdout: "max_objects": -1 2022-09-03T05:49:47.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:47.903 INFO:teuthology.orchestra.run.smithi189.stdout: "user_ratelimit": { 2022-09-03T05:49:47.903 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:49:47.904 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:49:47.904 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:49:47.904 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:49:47.905 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:49:47.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:47.905 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_ratelimit": { 2022-09-03T05:49:47.905 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:49:47.906 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:49:47.906 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:49:47.906 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:49:47.907 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:49:47.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:47.907 INFO:teuthology.orchestra.run.smithi189.stdout: "anonymous_ratelimit": { 2022-09-03T05:49:47.908 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:49:47.908 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:49:47.908 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:49:47.908 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:49:47.909 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:49:47.909 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:49:47.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:49:47.909 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:49:47.910 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_name": "test-realm", 2022-09-03T05:49:47.910 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:49:47.910 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:49:47.911 INFO:rgw_multi.tests:Set redirect_zone zone test-zone2 to "91786ef1-cdb4-4061-8a3e-2a28cc497285" 2022-09-03T05:50:47.912 DEBUG:boto:path=/dsgvcy-49/testredirect 2022-09-03T05:50:47.914 DEBUG:boto:auth_path=/dsgvcy-49/testredirect 2022-09-03T05:50:47.915 DEBUG:boto:Method: HEAD 2022-09-03T05:50:47.916 DEBUG:boto:Path: /dsgvcy-49/testredirect 2022-09-03T05:50:47.917 DEBUG:boto:Data: 2022-09-03T05:50:47.917 DEBUG:boto:Headers: {} 2022-09-03T05:50:47.918 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:50:47.918 DEBUG:boto:Port: 8000 2022-09-03T05:50:47.919 DEBUG:boto:Params: {} 2022-09-03T05:50:47.920 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:50:47.921 DEBUG:boto:Token: None 2022-09-03T05:50:47.922 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:50:47 GMT /dsgvcy-49/testredirect 2022-09-03T05:50:47.923 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZdEnzf0ZCxQqQdtsmBrFQJkVhyw= 2022-09-03T05:50:47.924 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:50:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZdEnzf0ZCxQqQdtsmBrFQJkVhyw=', 'Content-Length': '0'} 2022-09-03T05:50:47.931 DEBUG:boto:Response headers: [('Location', 'http://smithi181.front.sepia.ceph.com:8000/dsgvcy-49/testredirect?'), ('x-amz-request-id', 'tx0000073c17afb77ddfe2d-006312eb37-1572-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:50:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:50:47.932 DEBUG:boto:Redirecting: http://smithi181.front.sepia.ceph.com/dsgvcy-49/testredirect 2022-09-03T05:50:47.932 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': '8000'} 2022-09-03T05:50:47.933 DEBUG:boto:Token: None 2022-09-03T05:50:47.934 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:50:47 GMT /dsgvcy-49/testredirect 2022-09-03T05:50:47.934 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZdEnzf0ZCxQqQdtsmBrFQJkVhyw= 2022-09-03T05:50:47.935 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:50:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZdEnzf0ZCxQqQdtsmBrFQJkVhyw=', 'Content-Length': '0'} 2022-09-03T05:50:47.940 DEBUG:boto:Response headers: [('Content-Length', '512'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:48:45 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"dc5086b847289ba8b8bde149b8388175"'), ('x-amz-request-id', 'tx000005d22b829b0482045-006312eb37-156b-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:50:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:50:47.942 DEBUG:boto:path=/dsgvcy-49/testredirect 2022-09-03T05:50:47.942 DEBUG:boto:auth_path=/dsgvcy-49/testredirect 2022-09-03T05:50:47.943 DEBUG:boto:Method: GET 2022-09-03T05:50:47.944 DEBUG:boto:Path: /dsgvcy-49/testredirect 2022-09-03T05:50:47.944 DEBUG:boto:Data: 2022-09-03T05:50:47.945 DEBUG:boto:Headers: {} 2022-09-03T05:50:47.945 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:50:47.945 DEBUG:boto:Port: 8000 2022-09-03T05:50:47.946 DEBUG:boto:Params: {} 2022-09-03T05:50:47.947 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:50:47.947 DEBUG:boto:Token: None 2022-09-03T05:50:47.948 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:50:47 GMT /dsgvcy-49/testredirect 2022-09-03T05:50:47.948 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJi56vdKqaXnWnVdpUM4iuEis34= 2022-09-03T05:50:47.949 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:50:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJi56vdKqaXnWnVdpUM4iuEis34=', 'Content-Length': '0'} 2022-09-03T05:50:47.952 DEBUG:boto:Response headers: [('Location', 'http://smithi181.front.sepia.ceph.com:8000/dsgvcy-49/testredirect?'), ('x-amz-request-id', 'tx00000f48d5bde86e336cc-006312eb37-1572-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:50:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:50:47.953 DEBUG:boto:Redirecting: http://smithi181.front.sepia.ceph.com/dsgvcy-49/testredirect 2022-09-03T05:50:47.953 DEBUG:boto:Token: None 2022-09-03T05:50:47.954 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:50:47 GMT /dsgvcy-49/testredirect 2022-09-03T05:50:47.954 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJi56vdKqaXnWnVdpUM4iuEis34= 2022-09-03T05:50:47.955 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:50:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJi56vdKqaXnWnVdpUM4iuEis34=', 'Content-Length': '0'} 2022-09-03T05:50:47.957 DEBUG:boto:Response headers: [('Content-Length', '512'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:48:45 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"dc5086b847289ba8b8bde149b8388175"'), ('x-amz-request-id', 'tx00000a7b63ceebbde61a8-006312eb37-156b-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:50:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:50:47.958 DEBUG:boto:path=/dsgvcy-49/testredirect 2022-09-03T05:50:47.959 DEBUG:boto:auth_path=/dsgvcy-49/testredirect 2022-09-03T05:50:47.960 DEBUG:boto:Method: PUT 2022-09-03T05:50:47.961 DEBUG:boto:Path: /dsgvcy-49/testredirect 2022-09-03T05:50:47.961 DEBUG:boto:Data: 2022-09-03T05:50:47.962 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'VpBzljOcorCZvRIkX5Nt3A==', 'Content-Length': '512', 'Expect': '100-Continue'} 2022-09-03T05:50:47.962 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:50:47.963 DEBUG:boto:Port: 8000 2022-09-03T05:50:47.964 DEBUG:boto:Params: {} 2022-09-03T05:50:47.964 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:50:47.965 DEBUG:boto:Token: None 2022-09-03T05:50:47.965 DEBUG:boto:StringToSign: PUT VpBzljOcorCZvRIkX5Nt3A== application/octet-stream Sat, 03 Sep 2022 05:50:47 GMT /dsgvcy-49/testredirect 2022-09-03T05:50:47.966 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ankNVK5sibUEHYsO6Qmv6CPn8nM= 2022-09-03T05:50:47.967 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'VpBzljOcorCZvRIkX5Nt3A==', 'Content-Length': '512', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:50:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ankNVK5sibUEHYsO6Qmv6CPn8nM='} 2022-09-03T05:50:47.980 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"56907396339ca2b099bd12245f936ddc"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000de35a3c22e515224-006312eb37-156b-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:50:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:50:47.981 DEBUG:boto:path=/dsgvcy-49/testredirect 2022-09-03T05:50:47.982 DEBUG:boto:auth_path=/dsgvcy-49/testredirect 2022-09-03T05:50:47.983 DEBUG:boto:Method: GET 2022-09-03T05:50:47.984 DEBUG:boto:Path: /dsgvcy-49/testredirect 2022-09-03T05:50:47.985 DEBUG:boto:Data: 2022-09-03T05:50:47.985 DEBUG:boto:Headers: {} 2022-09-03T05:50:47.986 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:50:47.986 DEBUG:boto:Port: 8000 2022-09-03T05:50:47.987 DEBUG:boto:Params: {} 2022-09-03T05:50:47.988 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:50:47.988 DEBUG:boto:Token: None 2022-09-03T05:50:47.989 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:50:47 GMT /dsgvcy-49/testredirect 2022-09-03T05:50:47.989 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJi56vdKqaXnWnVdpUM4iuEis34= 2022-09-03T05:50:47.990 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:50:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJi56vdKqaXnWnVdpUM4iuEis34=', 'Content-Length': '0'} 2022-09-03T05:50:47.993 DEBUG:boto:Response headers: [('Location', 'http://smithi181.front.sepia.ceph.com:8000/dsgvcy-49/testredirect?'), ('x-amz-request-id', 'tx00000a896b47059e8327d-006312eb37-1572-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:50:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:50:47.994 DEBUG:boto:Redirecting: http://smithi181.front.sepia.ceph.com/dsgvcy-49/testredirect 2022-09-03T05:50:47.995 DEBUG:boto:Token: None 2022-09-03T05:50:47.995 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:50:47 GMT /dsgvcy-49/testredirect 2022-09-03T05:50:47.996 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJi56vdKqaXnWnVdpUM4iuEis34= 2022-09-03T05:50:47.997 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:50:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJi56vdKqaXnWnVdpUM4iuEis34=', 'Content-Length': '0'} 2022-09-03T05:50:47.999 DEBUG:boto:Response headers: [('Content-Length', '512'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:50:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"56907396339ca2b099bd12245f936ddc"'), ('x-amz-request-id', 'tx00000c2577d0572470a46-006312eb37-156b-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:50:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:50:48.000 DEBUG:boto:path=/dsgvcy-49/testredirect 2022-09-03T05:50:48.001 DEBUG:boto:auth_path=/dsgvcy-49/testredirect 2022-09-03T05:50:48.001 DEBUG:boto:Method: PUT 2022-09-03T05:50:48.002 DEBUG:boto:Path: /dsgvcy-49/testredirect 2022-09-03T05:50:48.003 DEBUG:boto:Data: 2022-09-03T05:50:48.003 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'uk9S5NXZfBvPq4jGr+LM5g==', 'Content-Length': '512', 'Expect': '100-Continue'} 2022-09-03T05:50:48.004 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:50:48.005 DEBUG:boto:Port: 8000 2022-09-03T05:50:48.006 DEBUG:boto:Params: {} 2022-09-03T05:50:48.007 DEBUG:boto:Token: None 2022-09-03T05:50:48.007 DEBUG:boto:StringToSign: PUT uk9S5NXZfBvPq4jGr+LM5g== application/octet-stream Sat, 03 Sep 2022 05:50:48 GMT /dsgvcy-49/testredirect 2022-09-03T05:50:48.008 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TwP80hsyl2OonYo2R7e5UZB4iRY= 2022-09-03T05:50:48.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'uk9S5NXZfBvPq4jGr+LM5g==', 'Content-Length': '512', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:50:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TwP80hsyl2OonYo2R7e5UZB4iRY='} 2022-09-03T05:50:48.014 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"ba4f52e4d5d97c1bcfab88c6afe2cce6"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000007026534307b115a5-006312eb38-156b-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:50:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:50:48.015 DEBUG:boto:path=/dsgvcy-49/testredirect 2022-09-03T05:50:48.016 DEBUG:boto:auth_path=/dsgvcy-49/testredirect 2022-09-03T05:50:48.017 DEBUG:boto:Method: GET 2022-09-03T05:50:48.018 DEBUG:boto:Path: /dsgvcy-49/testredirect 2022-09-03T05:50:48.019 DEBUG:boto:Data: 2022-09-03T05:50:48.020 DEBUG:boto:Headers: {} 2022-09-03T05:50:48.021 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:50:48.022 DEBUG:boto:Port: 8000 2022-09-03T05:50:48.023 DEBUG:boto:Params: {} 2022-09-03T05:50:48.024 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:50:48.025 DEBUG:boto:Token: None 2022-09-03T05:50:48.026 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:50:48 GMT /dsgvcy-49/testredirect 2022-09-03T05:50:48.027 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FXjk+GBA85NtWhYxXNR2ZjC+I+A= 2022-09-03T05:50:48.028 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:50:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FXjk+GBA85NtWhYxXNR2ZjC+I+A=', 'Content-Length': '0'} 2022-09-03T05:50:48.031 DEBUG:boto:Response headers: [('Location', 'http://smithi181.front.sepia.ceph.com:8000/dsgvcy-49/testredirect?'), ('x-amz-request-id', 'tx00000716aa01758c009df-006312eb38-1572-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:50:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:50:48.032 DEBUG:boto:Redirecting: http://smithi181.front.sepia.ceph.com/dsgvcy-49/testredirect 2022-09-03T05:50:48.033 DEBUG:boto:Token: None 2022-09-03T05:50:48.033 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:50:48 GMT /dsgvcy-49/testredirect 2022-09-03T05:50:48.034 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FXjk+GBA85NtWhYxXNR2ZjC+I+A= 2022-09-03T05:50:48.035 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:50:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FXjk+GBA85NtWhYxXNR2ZjC+I+A=', 'Content-Length': '0'} 2022-09-03T05:50:48.037 DEBUG:boto:Response headers: [('Content-Length', '512'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:50:48 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"ba4f52e4d5d97c1bcfab88c6afe2cce6"'), ('x-amz-request-id', 'tx00000161116151f3d795a-006312eb38-156b-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:50:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:50:48.039 DEBUG:boto:path=/dsgvcy-49/testredirect 2022-09-03T05:50:48.039 DEBUG:boto:auth_path=/dsgvcy-49/testredirect 2022-09-03T05:50:48.040 DEBUG:boto:Method: PUT 2022-09-03T05:50:48.041 DEBUG:boto:Path: /dsgvcy-49/testredirect 2022-09-03T05:50:48.042 DEBUG:boto:Data: 2022-09-03T05:50:48.043 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '6aKmvaJMgaK7Z2rfDZvVeg==', 'Content-Length': '512', 'Expect': '100-Continue'} 2022-09-03T05:50:48.044 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:50:48.045 DEBUG:boto:Port: 8000 2022-09-03T05:50:48.046 DEBUG:boto:Params: {} 2022-09-03T05:50:48.047 DEBUG:boto:Token: None 2022-09-03T05:50:48.048 DEBUG:boto:StringToSign: PUT 6aKmvaJMgaK7Z2rfDZvVeg== application/octet-stream Sat, 03 Sep 2022 05:50:48 GMT /dsgvcy-49/testredirect 2022-09-03T05:50:48.049 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ug0HxnJJJWovazRbhP02oT6Q/no= 2022-09-03T05:50:48.050 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '6aKmvaJMgaK7Z2rfDZvVeg==', 'Content-Length': '512', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:50:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ug0HxnJJJWovazRbhP02oT6Q/no='} 2022-09-03T05:50:48.056 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"e9a2a6bda24c81a2bb676adf0d9bd57a"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c759cba9932ff877-006312eb38-156b-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:50:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:50:48.057 DEBUG:boto:path=/dsgvcy-49/testredirect 2022-09-03T05:50:48.057 DEBUG:boto:auth_path=/dsgvcy-49/testredirect 2022-09-03T05:50:48.058 DEBUG:boto:Method: GET 2022-09-03T05:50:48.059 DEBUG:boto:Path: /dsgvcy-49/testredirect 2022-09-03T05:50:48.060 DEBUG:boto:Data: 2022-09-03T05:50:48.060 DEBUG:boto:Headers: {} 2022-09-03T05:50:48.061 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:50:48.061 DEBUG:boto:Port: 8000 2022-09-03T05:50:48.062 DEBUG:boto:Params: {} 2022-09-03T05:50:48.063 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:50:48.063 DEBUG:boto:Token: None 2022-09-03T05:50:48.064 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:50:48 GMT /dsgvcy-49/testredirect 2022-09-03T05:50:48.065 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FXjk+GBA85NtWhYxXNR2ZjC+I+A= 2022-09-03T05:50:48.065 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:50:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FXjk+GBA85NtWhYxXNR2ZjC+I+A=', 'Content-Length': '0'} 2022-09-03T05:50:48.069 DEBUG:boto:Response headers: [('Location', 'http://smithi181.front.sepia.ceph.com:8000/dsgvcy-49/testredirect?'), ('x-amz-request-id', 'tx00000793cb3791b64d239-006312eb38-1572-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:50:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:50:48.070 DEBUG:boto:Redirecting: http://smithi181.front.sepia.ceph.com/dsgvcy-49/testredirect 2022-09-03T05:50:48.070 DEBUG:boto:Token: None 2022-09-03T05:50:48.071 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:50:48 GMT /dsgvcy-49/testredirect 2022-09-03T05:50:48.071 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FXjk+GBA85NtWhYxXNR2ZjC+I+A= 2022-09-03T05:50:48.072 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:50:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FXjk+GBA85NtWhYxXNR2ZjC+I+A=', 'Content-Length': '0'} 2022-09-03T05:50:48.075 DEBUG:boto:Response headers: [('Content-Length', '512'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:50:48 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"e9a2a6bda24c81a2bb676adf0d9bd57a"'), ('x-amz-request-id', 'tx00000ae7429e4df070b54-006312eb38-156b-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:50:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:50:48.076 DEBUG:boto:path=/dsgvcy-49/testredirect 2022-09-03T05:50:48.076 DEBUG:boto:auth_path=/dsgvcy-49/testredirect 2022-09-03T05:50:48.077 DEBUG:boto:Method: PUT 2022-09-03T05:50:48.077 DEBUG:boto:Path: /dsgvcy-49/testredirect 2022-09-03T05:50:48.078 DEBUG:boto:Data: 2022-09-03T05:50:48.078 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'GbNgNQ5MAukCtgY+rsT5Xw==', 'Content-Length': '512', 'Expect': '100-Continue'} 2022-09-03T05:50:48.079 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:50:48.079 DEBUG:boto:Port: 8000 2022-09-03T05:50:48.080 DEBUG:boto:Params: {} 2022-09-03T05:50:48.080 DEBUG:boto:Token: None 2022-09-03T05:50:48.081 DEBUG:boto:StringToSign: PUT GbNgNQ5MAukCtgY+rsT5Xw== application/octet-stream Sat, 03 Sep 2022 05:50:48 GMT /dsgvcy-49/testredirect 2022-09-03T05:50:48.082 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zIESwkjpLeigM0tCfc6KEp0V2RA= 2022-09-03T05:50:48.082 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'GbNgNQ5MAukCtgY+rsT5Xw==', 'Content-Length': '512', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:50:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zIESwkjpLeigM0tCfc6KEp0V2RA='} 2022-09-03T05:50:48.131 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"19b360350e4c02e902b6063eaec4f95f"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ea782979c3f8ec42-006312eb38-156b-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:50:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:50:48.133 DEBUG:boto:path=/dsgvcy-49/testredirect 2022-09-03T05:50:48.133 DEBUG:boto:auth_path=/dsgvcy-49/testredirect 2022-09-03T05:50:48.135 DEBUG:boto:Method: GET 2022-09-03T05:50:48.136 DEBUG:boto:Path: /dsgvcy-49/testredirect 2022-09-03T05:50:48.136 DEBUG:boto:Data: 2022-09-03T05:50:48.137 DEBUG:boto:Headers: {} 2022-09-03T05:50:48.137 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:50:48.138 DEBUG:boto:Port: 8000 2022-09-03T05:50:48.139 DEBUG:boto:Params: {} 2022-09-03T05:50:48.139 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:50:48.140 DEBUG:boto:Token: None 2022-09-03T05:50:48.140 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:50:48 GMT /dsgvcy-49/testredirect 2022-09-03T05:50:48.141 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FXjk+GBA85NtWhYxXNR2ZjC+I+A= 2022-09-03T05:50:48.141 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:50:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FXjk+GBA85NtWhYxXNR2ZjC+I+A=', 'Content-Length': '0'} 2022-09-03T05:50:48.144 DEBUG:boto:Response headers: [('Location', 'http://smithi181.front.sepia.ceph.com:8000/dsgvcy-49/testredirect?'), ('x-amz-request-id', 'tx000005189f24a28dd419f-006312eb38-1572-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:50:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:50:48.145 DEBUG:boto:Redirecting: http://smithi181.front.sepia.ceph.com/dsgvcy-49/testredirect 2022-09-03T05:50:48.145 DEBUG:boto:Token: None 2022-09-03T05:50:48.146 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:50:48 GMT /dsgvcy-49/testredirect 2022-09-03T05:50:48.146 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FXjk+GBA85NtWhYxXNR2ZjC+I+A= 2022-09-03T05:50:48.147 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:50:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FXjk+GBA85NtWhYxXNR2ZjC+I+A=', 'Content-Length': '0'} 2022-09-03T05:50:48.149 DEBUG:boto:Response headers: [('Content-Length', '512'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 05:50:48 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"19b360350e4c02e902b6063eaec4f95f"'), ('x-amz-request-id', 'tx00000fd29b93b87dcd383-006312eb38-156b-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 05:50:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:50:48.150 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--sync-from-all=true', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:50:48.151 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--sync-from-all=true', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:50:48.152 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --sync-from-all=true --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:50:48.213 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:50:48.214 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:50:48.240 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:50:48.237+0000 7fd80a95c600 0 failed reading obj info from .rgw.root:zone_info.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4: (2) No such file or directory 2022-09-03T05:50:48.240 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:50:48.237+0000 7fd80a95c600 0 WARNING: could not read zone params for zone id=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4 name=test-zone3 2022-09-03T05:50:48.241 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:50:48.238+0000 7fd80a95c600 0 failed reading obj info from .rgw.root:zone_info.91786ef1-cdb4-4061-8a3e-2a28cc497285: (2) No such file or directory 2022-09-03T05:50:48.242 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:50:48.238+0000 7fd80a95c600 0 WARNING: could not read zone params for zone id=91786ef1-cdb4-4061-8a3e-2a28cc497285 name=test-zone1 2022-09-03T05:50:48.248 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:50:48.250 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:50:48.250 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone2", 2022-09-03T05:50:48.251 INFO:teuthology.orchestra.run.smithi189.stdout: "domain_root": "test-zone2.rgw.meta:root", 2022-09-03T05:50:48.251 INFO:teuthology.orchestra.run.smithi189.stdout: "control_pool": "test-zone2.rgw.control", 2022-09-03T05:50:48.252 INFO:teuthology.orchestra.run.smithi189.stdout: "gc_pool": "test-zone2.rgw.log:gc", 2022-09-03T05:50:48.252 INFO:teuthology.orchestra.run.smithi189.stdout: "lc_pool": "test-zone2.rgw.log:lc", 2022-09-03T05:50:48.253 INFO:teuthology.orchestra.run.smithi189.stdout: "log_pool": "test-zone2.rgw.log", 2022-09-03T05:50:48.253 INFO:teuthology.orchestra.run.smithi189.stdout: "intent_log_pool": "test-zone2.rgw.log:intent", 2022-09-03T05:50:48.254 INFO:teuthology.orchestra.run.smithi189.stdout: "usage_log_pool": "test-zone2.rgw.log:usage", 2022-09-03T05:50:48.254 INFO:teuthology.orchestra.run.smithi189.stdout: "roles_pool": "test-zone2.rgw.meta:roles", 2022-09-03T05:50:48.255 INFO:teuthology.orchestra.run.smithi189.stdout: "reshard_pool": "test-zone2.rgw.log:reshard", 2022-09-03T05:50:48.255 INFO:teuthology.orchestra.run.smithi189.stdout: "user_keys_pool": "test-zone2.rgw.meta:users.keys", 2022-09-03T05:50:48.256 INFO:teuthology.orchestra.run.smithi189.stdout: "user_email_pool": "test-zone2.rgw.meta:users.email", 2022-09-03T05:50:48.256 INFO:teuthology.orchestra.run.smithi189.stdout: "user_swift_pool": "test-zone2.rgw.meta:users.swift", 2022-09-03T05:50:48.257 INFO:teuthology.orchestra.run.smithi189.stdout: "user_uid_pool": "test-zone2.rgw.meta:users.uid", 2022-09-03T05:50:48.258 INFO:teuthology.orchestra.run.smithi189.stdout: "otp_pool": "test-zone2.rgw.otp", 2022-09-03T05:50:48.259 INFO:teuthology.orchestra.run.smithi189.stdout: "system_key": { 2022-09-03T05:50:48.259 INFO:teuthology.orchestra.run.smithi189.stdout: "access_key": "TW0E8EULCERVYU84", 2022-09-03T05:50:48.260 INFO:teuthology.orchestra.run.smithi189.stdout: "secret_key": "EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP" 2022-09-03T05:50:48.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:50:48.261 INFO:teuthology.orchestra.run.smithi189.stdout: "placement_pools": [ 2022-09-03T05:50:48.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:50:48.262 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "default-placement", 2022-09-03T05:50:48.263 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:50:48.263 INFO:teuthology.orchestra.run.smithi189.stdout: "index_pool": "test-zone2.rgw.buckets.index", 2022-09-03T05:50:48.264 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_classes": { 2022-09-03T05:50:48.264 INFO:teuthology.orchestra.run.smithi189.stdout: "STANDARD": { 2022-09-03T05:50:48.265 INFO:teuthology.orchestra.run.smithi189.stdout: "data_pool": "test-zone2.rgw.buckets.data", 2022-09-03T05:50:48.265 INFO:teuthology.orchestra.run.smithi189.stdout: "compression_type": "random" 2022-09-03T05:50:48.266 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:50:48.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:50:48.267 INFO:teuthology.orchestra.run.smithi189.stdout: "data_extra_pool": "test-zone2.rgw.buckets.non-ec", 2022-09-03T05:50:48.267 INFO:teuthology.orchestra.run.smithi189.stdout: "index_type": 0 2022-09-03T05:50:48.269 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:50:48.269 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:50:48.270 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:50:48.270 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:50:48.270 INFO:teuthology.orchestra.run.smithi189.stdout: "notif_pool": "test-zone2.rgw.log:notif" 2022-09-03T05:50:48.271 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:50:48.271 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:50:48.272 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:50:48.273 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:50:48.326 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:50:48.326 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:50:48.365 INFO:teuthology.orchestra.run.smithi189.stderr:Sending period to new master zone 91786ef1-cdb4-4061-8a3e-2a28cc497285 2022-09-03T05:50:48.401 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:50:48.402 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:50:48.403 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 4, 2022-09-03T05:50:48.403 INFO:teuthology.orchestra.run.smithi189.stdout: "predecessor_uuid": "eff2cb82-c479-4631-8568-fb1af4159e02", 2022-09-03T05:50:48.403 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": [ 2022-09-03T05:50:48.404 INFO:teuthology.orchestra.run.smithi189.stdout: "", 2022-09-03T05:50:48.404 INFO:teuthology.orchestra.run.smithi189.stdout: "", 2022-09-03T05:50:48.405 INFO:teuthology.orchestra.run.smithi189.stdout: "1_1662183936.347465_121.1", 2022-09-03T05:50:48.405 INFO:teuthology.orchestra.run.smithi189.stdout: "" 2022-09-03T05:50:48.406 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:50:48.406 INFO:teuthology.orchestra.run.smithi189.stdout: "period_map": { 2022-09-03T05:50:48.407 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:50:48.407 INFO:teuthology.orchestra.run.smithi189.stdout: "zonegroups": [ 2022-09-03T05:50:48.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:50:48.408 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "test-zonegroup", 2022-09-03T05:50:48.409 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zonegroup", 2022-09-03T05:50:48.409 INFO:teuthology.orchestra.run.smithi189.stdout: "api_name": "", 2022-09-03T05:50:48.409 INFO:teuthology.orchestra.run.smithi189.stdout: "is_master": true, 2022-09-03T05:50:48.411 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:50:48.412 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:50:48.412 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:50:48.413 INFO:teuthology.orchestra.run.smithi189.stdout: "hostnames": [], 2022-09-03T05:50:48.413 INFO:teuthology.orchestra.run.smithi189.stdout: "hostnames_s3website": [], 2022-09-03T05:50:48.414 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:50:48.414 INFO:teuthology.orchestra.run.smithi189.stdout: "zones": [ 2022-09-03T05:50:48.415 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:50:48.415 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:50:48.416 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone3", 2022-09-03T05:50:48.416 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:50:48.417 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi181.front.sepia.ceph.com:8001" 2022-09-03T05:50:48.417 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:50:48.417 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:50:48.418 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:50:48.418 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:50:48.419 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:50:48.419 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:50:48.420 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": true, 2022-09-03T05:50:48.420 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:50:48.420 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:50:48.421 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:50:48.421 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:50:48.422 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:50:48.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:50:48.423 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:50:48.423 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:50:48.424 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone4", 2022-09-03T05:50:48.424 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:50:48.424 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi189.front.sepia.ceph.com:8001" 2022-09-03T05:50:48.425 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:50:48.425 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:50:48.425 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:50:48.426 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:50:48.426 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:50:48.426 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "pubsub", 2022-09-03T05:50:48.426 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": false, 2022-09-03T05:50:48.427 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [ 2022-09-03T05:50:48.427 INFO:teuthology.orchestra.run.smithi189.stdout: "test-zone1" 2022-09-03T05:50:48.427 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:50:48.427 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:50:48.428 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:50:48.428 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:50:48.428 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:50:48.429 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:50:48.429 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:50:48.429 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:50:48.430 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone1", 2022-09-03T05:50:48.430 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:50:48.430 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:50:48.430 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:50:48.431 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:50:48.431 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:50:48.431 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:50:48.433 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:50:48.434 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:50:48.434 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": true, 2022-09-03T05:50:48.435 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:50:48.435 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:50:48.435 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:50:48.436 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:50:48.436 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:50:48.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:50:48.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:50:48.437 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:50:48.438 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone2", 2022-09-03T05:50:48.438 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:50:48.438 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi189.front.sepia.ceph.com:8000" 2022-09-03T05:50:48.439 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:50:48.439 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:50:48.440 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:50:48.440 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:50:48.441 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:50:48.441 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:50:48.442 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": true, 2022-09-03T05:50:48.442 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:50:48.443 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:50:48.443 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:50:48.444 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:50:48.445 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:50:48.445 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:50:48.446 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:50:48.446 INFO:teuthology.orchestra.run.smithi189.stdout: "placement_targets": [ 2022-09-03T05:50:48.447 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:50:48.449 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "default-placement", 2022-09-03T05:50:48.450 INFO:teuthology.orchestra.run.smithi189.stdout: "tags": [], 2022-09-03T05:50:48.450 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_classes": [ 2022-09-03T05:50:48.451 INFO:teuthology.orchestra.run.smithi189.stdout: "STANDARD" 2022-09-03T05:50:48.451 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:50:48.452 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:50:48.452 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:50:48.453 INFO:teuthology.orchestra.run.smithi189.stdout: "default_placement": "default-placement", 2022-09-03T05:50:48.453 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:50:48.454 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_policy": { 2022-09-03T05:50:48.455 INFO:teuthology.orchestra.run.smithi189.stdout: "groups": [] 2022-09-03T05:50:48.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:50:48.456 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled_features": [ 2022-09-03T05:50:48.456 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:50:48.457 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:50:48.457 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:50:48.458 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:50:48.458 INFO:teuthology.orchestra.run.smithi189.stdout: "short_zone_ids": [ 2022-09-03T05:50:48.458 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:50:48.459 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:50:48.459 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 327838863 2022-09-03T05:50:48.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:50:48.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:50:48.460 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:50:48.461 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 1690651894 2022-09-03T05:50:48.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:50:48.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:50:48.462 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:50:48.463 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 1752562153 2022-09-03T05:50:48.463 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:50:48.463 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:50:48.464 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:50:48.464 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 2011734872 2022-09-03T05:50:48.465 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:50:48.465 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:50:48.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:50:48.466 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zonegroup": "test-zonegroup", 2022-09-03T05:50:48.466 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:50:48.467 INFO:teuthology.orchestra.run.smithi189.stdout: "period_config": { 2022-09-03T05:50:48.467 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_quota": { 2022-09-03T05:50:48.468 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false, 2022-09-03T05:50:48.468 INFO:teuthology.orchestra.run.smithi189.stdout: "check_on_raw": false, 2022-09-03T05:50:48.468 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size": -1, 2022-09-03T05:50:48.469 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size_kb": 0, 2022-09-03T05:50:48.469 INFO:teuthology.orchestra.run.smithi189.stdout: "max_objects": -1 2022-09-03T05:50:48.470 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:50:48.470 INFO:teuthology.orchestra.run.smithi189.stdout: "user_quota": { 2022-09-03T05:50:48.470 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false, 2022-09-03T05:50:48.471 INFO:teuthology.orchestra.run.smithi189.stdout: "check_on_raw": false, 2022-09-03T05:50:48.471 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size": -1, 2022-09-03T05:50:48.472 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size_kb": 0, 2022-09-03T05:50:48.472 INFO:teuthology.orchestra.run.smithi189.stdout: "max_objects": -1 2022-09-03T05:50:48.472 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:50:48.473 INFO:teuthology.orchestra.run.smithi189.stdout: "user_ratelimit": { 2022-09-03T05:50:48.473 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:50:48.473 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:50:48.474 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:50:48.474 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:50:48.475 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:50:48.475 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:50:48.476 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_ratelimit": { 2022-09-03T05:50:48.476 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:50:48.477 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:50:48.478 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:50:48.479 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:50:48.479 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:50:48.479 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:50:48.480 INFO:teuthology.orchestra.run.smithi189.stdout: "anonymous_ratelimit": { 2022-09-03T05:50:48.480 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:50:48.481 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:50:48.481 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:50:48.483 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:50:48.483 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:50:48.483 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:50:48.484 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:50:48.484 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:50:48.484 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_name": "test-realm", 2022-09-03T05:50:48.485 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:50:48.485 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:50:48.486 INFO:rgw_multi.tests:Set sync_from_all flag on zone test-zone2 to true 2022-09-03T05:51:48.488 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--redirect-zone=', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:51:48.490 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'modify', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--redirect-zone=', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:51:48.491 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone modify --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --redirect-zone= --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:51:48.548 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:51:48.548 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:51:48.572 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:51:48.570+0000 7f2af0938600 0 failed reading obj info from .rgw.root:zone_info.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4: (2) No such file or directory 2022-09-03T05:51:48.573 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:51:48.570+0000 7f2af0938600 0 WARNING: could not read zone params for zone id=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4 name=test-zone3 2022-09-03T05:51:48.573 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:51:48.571+0000 7f2af0938600 0 failed reading obj info from .rgw.root:zone_info.91786ef1-cdb4-4061-8a3e-2a28cc497285: (2) No such file or directory 2022-09-03T05:51:48.573 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:51:48.571+0000 7f2af0938600 0 WARNING: could not read zone params for zone id=91786ef1-cdb4-4061-8a3e-2a28cc497285 name=test-zone1 2022-09-03T05:51:48.578 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:51:48.578 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:51:48.579 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone2", 2022-09-03T05:51:48.579 INFO:teuthology.orchestra.run.smithi189.stdout: "domain_root": "test-zone2.rgw.meta:root", 2022-09-03T05:51:48.579 INFO:teuthology.orchestra.run.smithi189.stdout: "control_pool": "test-zone2.rgw.control", 2022-09-03T05:51:48.579 INFO:teuthology.orchestra.run.smithi189.stdout: "gc_pool": "test-zone2.rgw.log:gc", 2022-09-03T05:51:48.580 INFO:teuthology.orchestra.run.smithi189.stdout: "lc_pool": "test-zone2.rgw.log:lc", 2022-09-03T05:51:48.580 INFO:teuthology.orchestra.run.smithi189.stdout: "log_pool": "test-zone2.rgw.log", 2022-09-03T05:51:48.580 INFO:teuthology.orchestra.run.smithi189.stdout: "intent_log_pool": "test-zone2.rgw.log:intent", 2022-09-03T05:51:48.580 INFO:teuthology.orchestra.run.smithi189.stdout: "usage_log_pool": "test-zone2.rgw.log:usage", 2022-09-03T05:51:48.581 INFO:teuthology.orchestra.run.smithi189.stdout: "roles_pool": "test-zone2.rgw.meta:roles", 2022-09-03T05:51:48.581 INFO:teuthology.orchestra.run.smithi189.stdout: "reshard_pool": "test-zone2.rgw.log:reshard", 2022-09-03T05:51:48.581 INFO:teuthology.orchestra.run.smithi189.stdout: "user_keys_pool": "test-zone2.rgw.meta:users.keys", 2022-09-03T05:51:48.581 INFO:teuthology.orchestra.run.smithi189.stdout: "user_email_pool": "test-zone2.rgw.meta:users.email", 2022-09-03T05:51:48.582 INFO:teuthology.orchestra.run.smithi189.stdout: "user_swift_pool": "test-zone2.rgw.meta:users.swift", 2022-09-03T05:51:48.582 INFO:teuthology.orchestra.run.smithi189.stdout: "user_uid_pool": "test-zone2.rgw.meta:users.uid", 2022-09-03T05:51:48.582 INFO:teuthology.orchestra.run.smithi189.stdout: "otp_pool": "test-zone2.rgw.otp", 2022-09-03T05:51:48.583 INFO:teuthology.orchestra.run.smithi189.stdout: "system_key": { 2022-09-03T05:51:48.583 INFO:teuthology.orchestra.run.smithi189.stdout: "access_key": "TW0E8EULCERVYU84", 2022-09-03T05:51:48.583 INFO:teuthology.orchestra.run.smithi189.stdout: "secret_key": "EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP" 2022-09-03T05:51:48.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:51:48.584 INFO:teuthology.orchestra.run.smithi189.stdout: "placement_pools": [ 2022-09-03T05:51:48.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:51:48.584 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "default-placement", 2022-09-03T05:51:48.584 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:51:48.585 INFO:teuthology.orchestra.run.smithi189.stdout: "index_pool": "test-zone2.rgw.buckets.index", 2022-09-03T05:51:48.585 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_classes": { 2022-09-03T05:51:48.585 INFO:teuthology.orchestra.run.smithi189.stdout: "STANDARD": { 2022-09-03T05:51:48.585 INFO:teuthology.orchestra.run.smithi189.stdout: "data_pool": "test-zone2.rgw.buckets.data", 2022-09-03T05:51:48.586 INFO:teuthology.orchestra.run.smithi189.stdout: "compression_type": "random" 2022-09-03T05:51:48.586 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:51:48.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:51:48.586 INFO:teuthology.orchestra.run.smithi189.stdout: "data_extra_pool": "test-zone2.rgw.buckets.non-ec", 2022-09-03T05:51:48.587 INFO:teuthology.orchestra.run.smithi189.stdout: "index_type": 0 2022-09-03T05:51:48.587 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:51:48.588 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:51:48.588 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:51:48.589 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:51:48.589 INFO:teuthology.orchestra.run.smithi189.stdout: "notif_pool": "test-zone2.rgw.log:notif" 2022-09-03T05:51:48.590 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:51:48.592 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:51:48.593 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:51:48.594 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:51:48.644 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:51:48.645 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:51:48.683 INFO:teuthology.orchestra.run.smithi189.stderr:Sending period to new master zone 91786ef1-cdb4-4061-8a3e-2a28cc497285 2022-09-03T05:51:48.713 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:51:48.714 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:51:48.714 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 5, 2022-09-03T05:51:48.715 INFO:teuthology.orchestra.run.smithi189.stdout: "predecessor_uuid": "eff2cb82-c479-4631-8568-fb1af4159e02", 2022-09-03T05:51:48.715 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": [ 2022-09-03T05:51:48.716 INFO:teuthology.orchestra.run.smithi189.stdout: "", 2022-09-03T05:51:48.716 INFO:teuthology.orchestra.run.smithi189.stdout: "", 2022-09-03T05:51:48.717 INFO:teuthology.orchestra.run.smithi189.stdout: "1_1662183936.347465_121.1", 2022-09-03T05:51:48.717 INFO:teuthology.orchestra.run.smithi189.stdout: "" 2022-09-03T05:51:48.718 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:51:48.718 INFO:teuthology.orchestra.run.smithi189.stdout: "period_map": { 2022-09-03T05:51:48.718 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:51:48.719 INFO:teuthology.orchestra.run.smithi189.stdout: "zonegroups": [ 2022-09-03T05:51:48.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:51:48.720 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "test-zonegroup", 2022-09-03T05:51:48.720 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zonegroup", 2022-09-03T05:51:48.721 INFO:teuthology.orchestra.run.smithi189.stdout: "api_name": "", 2022-09-03T05:51:48.721 INFO:teuthology.orchestra.run.smithi189.stdout: "is_master": true, 2022-09-03T05:51:48.721 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:51:48.722 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:51:48.722 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:51:48.723 INFO:teuthology.orchestra.run.smithi189.stdout: "hostnames": [], 2022-09-03T05:51:48.723 INFO:teuthology.orchestra.run.smithi189.stdout: "hostnames_s3website": [], 2022-09-03T05:51:48.723 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:51:48.723 INFO:teuthology.orchestra.run.smithi189.stdout: "zones": [ 2022-09-03T05:51:48.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:51:48.724 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:51:48.724 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone3", 2022-09-03T05:51:48.724 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:51:48.725 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi181.front.sepia.ceph.com:8001" 2022-09-03T05:51:48.725 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:51:48.725 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:51:48.725 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:51:48.726 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:51:48.727 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:51:48.727 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:51:48.727 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": true, 2022-09-03T05:51:48.727 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:51:48.728 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:51:48.728 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:51:48.728 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:51:48.728 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:51:48.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:51:48.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:51:48.729 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:51:48.730 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone4", 2022-09-03T05:51:48.730 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:51:48.730 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi189.front.sepia.ceph.com:8001" 2022-09-03T05:51:48.731 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:51:48.731 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:51:48.731 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:51:48.731 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:51:48.731 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:51:48.732 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "pubsub", 2022-09-03T05:51:48.732 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": false, 2022-09-03T05:51:48.732 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [ 2022-09-03T05:51:48.732 INFO:teuthology.orchestra.run.smithi189.stdout: "test-zone1" 2022-09-03T05:51:48.733 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:51:48.733 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:51:48.733 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:51:48.734 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:51:48.734 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:51:48.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:51:48.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:51:48.735 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:51:48.735 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone1", 2022-09-03T05:51:48.735 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:51:48.735 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:51:48.736 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:51:48.736 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:51:48.736 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:51:48.737 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:51:48.737 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:51:48.737 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:51:48.737 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": true, 2022-09-03T05:51:48.738 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:51:48.738 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:51:48.738 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:51:48.738 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:51:48.739 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:51:48.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:51:48.739 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:51:48.739 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:51:48.740 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone2", 2022-09-03T05:51:48.740 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:51:48.740 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi189.front.sepia.ceph.com:8000" 2022-09-03T05:51:48.740 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:51:48.741 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:51:48.741 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:51:48.741 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:51:48.741 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:51:48.742 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:51:48.742 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": true, 2022-09-03T05:51:48.742 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:51:48.743 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:51:48.743 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:51:48.743 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:51:48.743 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:51:48.744 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:51:48.744 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:51:48.744 INFO:teuthology.orchestra.run.smithi189.stdout: "placement_targets": [ 2022-09-03T05:51:48.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:51:48.745 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "default-placement", 2022-09-03T05:51:48.746 INFO:teuthology.orchestra.run.smithi189.stdout: "tags": [], 2022-09-03T05:51:48.746 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_classes": [ 2022-09-03T05:51:48.747 INFO:teuthology.orchestra.run.smithi189.stdout: "STANDARD" 2022-09-03T05:51:48.747 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:51:48.747 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:51:48.748 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:51:48.748 INFO:teuthology.orchestra.run.smithi189.stdout: "default_placement": "default-placement", 2022-09-03T05:51:48.748 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:51:48.748 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_policy": { 2022-09-03T05:51:48.749 INFO:teuthology.orchestra.run.smithi189.stdout: "groups": [] 2022-09-03T05:51:48.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:51:48.749 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled_features": [ 2022-09-03T05:51:48.749 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:51:48.750 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:51:48.750 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:51:48.750 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:51:48.750 INFO:teuthology.orchestra.run.smithi189.stdout: "short_zone_ids": [ 2022-09-03T05:51:48.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:51:48.751 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:51:48.751 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 327838863 2022-09-03T05:51:48.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:51:48.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:51:48.752 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:51:48.752 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 1690651894 2022-09-03T05:51:48.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:51:48.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:51:48.753 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:51:48.753 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 1752562153 2022-09-03T05:51:48.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:51:48.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:51:48.754 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:51:48.754 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 2011734872 2022-09-03T05:51:48.755 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:51:48.755 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:51:48.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:51:48.756 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zonegroup": "test-zonegroup", 2022-09-03T05:51:48.756 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:51:48.756 INFO:teuthology.orchestra.run.smithi189.stdout: "period_config": { 2022-09-03T05:51:48.756 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_quota": { 2022-09-03T05:51:48.757 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false, 2022-09-03T05:51:48.757 INFO:teuthology.orchestra.run.smithi189.stdout: "check_on_raw": false, 2022-09-03T05:51:48.757 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size": -1, 2022-09-03T05:51:48.757 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size_kb": 0, 2022-09-03T05:51:48.758 INFO:teuthology.orchestra.run.smithi189.stdout: "max_objects": -1 2022-09-03T05:51:48.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:51:48.758 INFO:teuthology.orchestra.run.smithi189.stdout: "user_quota": { 2022-09-03T05:51:48.758 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false, 2022-09-03T05:51:48.759 INFO:teuthology.orchestra.run.smithi189.stdout: "check_on_raw": false, 2022-09-03T05:51:48.759 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size": -1, 2022-09-03T05:51:48.759 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size_kb": 0, 2022-09-03T05:51:48.760 INFO:teuthology.orchestra.run.smithi189.stdout: "max_objects": -1 2022-09-03T05:51:48.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:51:48.760 INFO:teuthology.orchestra.run.smithi189.stdout: "user_ratelimit": { 2022-09-03T05:51:48.760 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:51:48.761 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:51:48.761 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:51:48.761 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:51:48.761 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:51:48.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:51:48.762 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_ratelimit": { 2022-09-03T05:51:48.762 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:51:48.762 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:51:48.763 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:51:48.763 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:51:48.763 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:51:48.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:51:48.764 INFO:teuthology.orchestra.run.smithi189.stdout: "anonymous_ratelimit": { 2022-09-03T05:51:48.764 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:51:48.764 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:51:48.765 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:51:48.765 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:51:48.765 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:51:48.765 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:51:48.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:51:48.766 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:51:48.766 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_name": "test-realm", 2022-09-03T05:51:48.766 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:51:48.767 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:51:48.767 INFO:rgw_multi.tests:Set redirect_zone zone test-zone2 to "" 2022-09-03T05:52:48.770 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_multi_zone_redirect ... ok 2022-09-03T05:52:48.774 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:52:48.775 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:52:48.776 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:52:48.833 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:52:48.834 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:52:48.882 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T05:52:48.905 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:52:48.906 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:48.906 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:52:48.907 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:52:48.907 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:48.907 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:48.907 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:52:48.908 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:52:48.908 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:48.908 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:48.908 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:52:48.909 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:48:45.211231Z" 2022-09-03T05:52:48.909 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:48.909 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:48.910 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:52:48.910 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:52:48.911 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:48.911 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:52:48.914 DEBUG:rgw_multi.tests:master meta markers={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:52:48.915 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:52:48.916 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:52:48.917 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:52:48.918 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:52:48.976 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:52:48.977 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:52:49.017 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:52:49.015+0000 7f88ff32d600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:52:49.018 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:52:49.017+0000 7f88ff32d600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:52:49.033 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:52:49.034 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:52:49.034 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:52:49.035 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:52:49.035 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:52:49.036 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:52:49.036 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:52:49.037 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:49.037 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:52:49.038 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:52:49.038 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:52:49.039 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:52:49.039 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:52:49.039 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:52:49.040 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:52:49.041 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:52:49.041 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:52:49.042 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:52:49.042 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:52:49.043 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:52:49.043 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:49.043 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:52:49.044 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:52:49.044 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:52:49.045 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:52:49.045 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:52:49.046 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:52:49.046 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:52:49.046 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:52:49.047 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:52:49.048 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:52:49.049 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:52:49.049 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:49.050 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:52:49.050 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:52:49.051 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:52:49.051 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:52:49.052 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:52:49.052 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:52:49.053 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:52:49.053 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:52:49.053 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:48:45.211231Z", 2022-09-03T05:52:49.054 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:52:49.054 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:52:49.055 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:49.055 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:52:49.056 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:52:49.056 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:52:49.056 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:52:49.057 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:52:49.057 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:52:49.057 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:52:49.058 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:52:49.058 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:52:49.058 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:52:49.059 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:52:49.059 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:52:49.059 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:52:49.059 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:49.059 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:52:49.060 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:52:49.060 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:52:49.060 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:49.060 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:52:49Z" 2022-09-03T05:52:49.061 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:52:49.062 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184125.211231_206.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:48:45.211231Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:52:49Z" } 2022-09-03T05:52:49.063 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184125.211231_206.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:48:45.211231Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:52:49.063 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:52:49.064 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:52:49.064 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:52:49.065 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:52:49.065 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:52:49.066 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:52:49.068 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:52:49.121 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:52:49.122 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:52:49.164 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:52:49.162+0000 7f51dc1d9600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:52:49.164 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:52:49.163+0000 7f51dc1d9600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:52:49.176 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:52:49.176 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:52:49.177 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:52:49.177 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:52:49.177 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:52:49.177 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:52:49.178 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:52:49.178 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:49.178 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:52:49.179 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:49.179 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:52:49.179 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:52:49.179 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:52:49.180 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:52:49.180 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:52:49.180 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:52:49.180 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:52:49.181 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:52:49.181 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:52:49.181 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:49.181 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:49.182 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:49.182 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:52:49.182 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:52:49.182 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:52:49.183 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:52:49.183 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:52:49.183 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:52:49.184 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:52:49.184 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:52:49.184 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:52:49.184 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:49.185 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:49.185 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:49.185 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:52:49.185 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:52:49.186 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:52:49.186 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:52:49.186 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:52:49.186 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:52:49.187 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:52:49.187 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:48:45.211231Z", 2022-09-03T05:52:49.187 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:52:49.187 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:49.188 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:49.188 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:49.188 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:52:49.188 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:52:49.189 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:52:49.189 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:52:49.189 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:52:49.189 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:52:49.190 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:52:49.190 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:52:49.190 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:52:49.191 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:49.191 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:49.191 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:49.191 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:49.192 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:52:49.192 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:52:49.192 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:52:49.192 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:49.193 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:52:49Z" 2022-09-03T05:52:49.193 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:52:49.194 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184125.211231_206.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:48:45.211231Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:52:49Z" } 2022-09-03T05:52:49.195 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184125.211231_206.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:48:45.211231Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:52:49.195 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:52:49.196 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:52:49.196 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:52:49.197 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:52:49.197 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:52:49.198 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:52:49.199 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:52:49.250 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:52:49.250 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:52:49.274 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:52:49.273+0000 7f8f6423d600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:52:49.275 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:52:49.274+0000 7f8f6423d600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:52:49.286 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:52:49.287 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:52:49.287 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:52:49.287 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:52:49.288 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:52:49.288 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:52:49.288 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:52:49.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:49.289 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:52:49.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:52:49.290 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:52:49.290 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:52:49.290 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:52:49.291 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:52:49.291 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:52:49.291 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:52:49.291 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:52:49.292 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:52:49.292 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:52:49.292 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:52:49.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:49.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:52:49.293 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:52:49.293 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:52:49.293 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:52:49.294 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:52:49.294 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:52:49.294 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:52:49.294 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:52:49.295 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:52:49.295 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:52:49.295 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:52:49.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:49.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:52:49.296 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:52:49.296 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:52:49.297 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:52:49.297 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:52:49.297 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:52:49.298 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:52:49.298 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:52:49.298 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:48:45.211231Z", 2022-09-03T05:52:49.299 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:52:49.299 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:52:49.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:49.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:52:49.300 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:52:49.300 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:52:49.300 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:52:49.301 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:52:49.301 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:52:49.302 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:52:49.302 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:52:49.303 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:52:49.303 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:52:49.304 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:52:49.304 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:52:49.305 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:52:49.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:49.306 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:52:49.306 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:52:49.307 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:52:49.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:49.308 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:52:49Z" 2022-09-03T05:52:49.308 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:52:49.311 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184125.211231_206.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:48:45.211231Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:52:49Z" } 2022-09-03T05:52:49.312 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184125.211231_206.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:48:45.211231Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:52:49.313 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:52:49.314 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:52:49.315 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:52:49.316 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'create', '--rgw-zone', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', 'TW0E8EULCERVYU84', '--secret', 'EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:49.317 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'create', '--rgw-zone', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--access-key', 'TW0E8EULCERVYU84', '--secret', 'EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:49.318 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone create --rgw-zone remove --rgw-zonegroup test-zonegroup --rgw-realm test-realm --access-key TW0E8EULCERVYU84 --secret EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:52:49.372 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:52:49.373 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:52:49.395 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:52:49.394+0000 7fc193a42600 0 failed reading obj info from .rgw.root:zone_info.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4: (2) No such file or directory 2022-09-03T05:52:49.396 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:52:49.394+0000 7fc193a42600 0 WARNING: could not read zone params for zone id=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4 name=test-zone3 2022-09-03T05:52:49.396 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:52:49.394+0000 7fc193a42600 0 failed reading obj info from .rgw.root:zone_info.91786ef1-cdb4-4061-8a3e-2a28cc497285: (2) No such file or directory 2022-09-03T05:52:49.396 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:52:49.394+0000 7fc193a42600 0 WARNING: could not read zone params for zone id=91786ef1-cdb4-4061-8a3e-2a28cc497285 name=test-zone1 2022-09-03T05:52:49.399 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:52:49.399 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "e93f72de-f01b-431e-bf7a-c483294fc9d9", 2022-09-03T05:52:49.400 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "remove", 2022-09-03T05:52:49.400 INFO:teuthology.orchestra.run.smithi189.stdout: "domain_root": "remove.rgw.meta:root", 2022-09-03T05:52:49.400 INFO:teuthology.orchestra.run.smithi189.stdout: "control_pool": "remove.rgw.control", 2022-09-03T05:52:49.401 INFO:teuthology.orchestra.run.smithi189.stdout: "gc_pool": "remove.rgw.log:gc", 2022-09-03T05:52:49.401 INFO:teuthology.orchestra.run.smithi189.stdout: "lc_pool": "remove.rgw.log:lc", 2022-09-03T05:52:49.401 INFO:teuthology.orchestra.run.smithi189.stdout: "log_pool": "remove.rgw.log", 2022-09-03T05:52:49.402 INFO:teuthology.orchestra.run.smithi189.stdout: "intent_log_pool": "remove.rgw.log:intent", 2022-09-03T05:52:49.403 INFO:teuthology.orchestra.run.smithi189.stdout: "usage_log_pool": "remove.rgw.log:usage", 2022-09-03T05:52:49.403 INFO:teuthology.orchestra.run.smithi189.stdout: "roles_pool": "remove.rgw.meta:roles", 2022-09-03T05:52:49.403 INFO:teuthology.orchestra.run.smithi189.stdout: "reshard_pool": "remove.rgw.log:reshard", 2022-09-03T05:52:49.404 INFO:teuthology.orchestra.run.smithi189.stdout: "user_keys_pool": "remove.rgw.meta:users.keys", 2022-09-03T05:52:49.404 INFO:teuthology.orchestra.run.smithi189.stdout: "user_email_pool": "remove.rgw.meta:users.email", 2022-09-03T05:52:49.405 INFO:teuthology.orchestra.run.smithi189.stdout: "user_swift_pool": "remove.rgw.meta:users.swift", 2022-09-03T05:52:49.405 INFO:teuthology.orchestra.run.smithi189.stdout: "user_uid_pool": "remove.rgw.meta:users.uid", 2022-09-03T05:52:49.406 INFO:teuthology.orchestra.run.smithi189.stdout: "otp_pool": "remove.rgw.otp", 2022-09-03T05:52:49.407 INFO:teuthology.orchestra.run.smithi189.stdout: "system_key": { 2022-09-03T05:52:49.407 INFO:teuthology.orchestra.run.smithi189.stdout: "access_key": "TW0E8EULCERVYU84", 2022-09-03T05:52:49.408 INFO:teuthology.orchestra.run.smithi189.stdout: "secret_key": "EouhQ8KeFv55ap1vMj6tIAk3DOXMlOuP" 2022-09-03T05:52:49.408 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:49.408 INFO:teuthology.orchestra.run.smithi189.stdout: "placement_pools": [ 2022-09-03T05:52:49.409 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:52:49.409 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "default-placement", 2022-09-03T05:52:49.410 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:52:49.410 INFO:teuthology.orchestra.run.smithi189.stdout: "index_pool": "remove.rgw.buckets.index", 2022-09-03T05:52:49.411 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_classes": { 2022-09-03T05:52:49.411 INFO:teuthology.orchestra.run.smithi189.stdout: "STANDARD": { 2022-09-03T05:52:49.412 INFO:teuthology.orchestra.run.smithi189.stdout: "data_pool": "remove.rgw.buckets.data" 2022-09-03T05:52:49.412 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:52:49.413 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:49.413 INFO:teuthology.orchestra.run.smithi189.stdout: "data_extra_pool": "remove.rgw.buckets.non-ec", 2022-09-03T05:52:49.413 INFO:teuthology.orchestra.run.smithi189.stdout: "index_type": 0 2022-09-03T05:52:49.414 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:52:49.414 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:52:49.414 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:52:49.415 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:52:49.415 INFO:teuthology.orchestra.run.smithi189.stdout: "notif_pool": "remove.rgw.log:notif" 2022-09-03T05:52:49.415 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:52:49.417 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['period', 'update', '--rgw-zone', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:49.417 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'period', 'update', '--rgw-zone', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:49.418 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 period update --rgw-zone remove --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:52:49.467 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:52:49.467 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:52:49.476 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:52:49.475+0000 7f3fb2998600 0 period (229c309b-f405-4ead-a3a8-baa670c6da51 does not have zone e93f72de-f01b-431e-bf7a-c483294fc9d9 configured 2022-09-03T05:52:55.655 INFO:teuthology.orchestra.run.smithi189.stderr:Sending period to new master zone 91786ef1-cdb4-4061-8a3e-2a28cc497285 2022-09-03T05:52:55.689 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:52:55.689 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:52:55.690 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 6, 2022-09-03T05:52:55.690 INFO:teuthology.orchestra.run.smithi189.stdout: "predecessor_uuid": "eff2cb82-c479-4631-8568-fb1af4159e02", 2022-09-03T05:52:55.690 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": [ 2022-09-03T05:52:55.690 INFO:teuthology.orchestra.run.smithi189.stdout: "", 2022-09-03T05:52:55.691 INFO:teuthology.orchestra.run.smithi189.stdout: "", 2022-09-03T05:52:55.691 INFO:teuthology.orchestra.run.smithi189.stdout: "1_1662183936.347465_121.1", 2022-09-03T05:52:55.691 INFO:teuthology.orchestra.run.smithi189.stdout: "" 2022-09-03T05:52:55.692 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:52:55.692 INFO:teuthology.orchestra.run.smithi189.stdout: "period_map": { 2022-09-03T05:52:55.692 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:52:55.692 INFO:teuthology.orchestra.run.smithi189.stdout: "zonegroups": [ 2022-09-03T05:52:55.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:52:55.693 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "test-zonegroup", 2022-09-03T05:52:55.693 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zonegroup", 2022-09-03T05:52:55.693 INFO:teuthology.orchestra.run.smithi189.stdout: "api_name": "", 2022-09-03T05:52:55.694 INFO:teuthology.orchestra.run.smithi189.stdout: "is_master": true, 2022-09-03T05:52:55.694 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:52:55.694 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:52:55.695 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:52:55.695 INFO:teuthology.orchestra.run.smithi189.stdout: "hostnames": [], 2022-09-03T05:52:55.695 INFO:teuthology.orchestra.run.smithi189.stdout: "hostnames_s3website": [], 2022-09-03T05:52:55.695 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:52:55.696 INFO:teuthology.orchestra.run.smithi189.stdout: "zones": [ 2022-09-03T05:52:55.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:52:55.696 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:52:55.696 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone3", 2022-09-03T05:52:55.697 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:52:55.697 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi181.front.sepia.ceph.com:8001" 2022-09-03T05:52:55.697 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:52:55.697 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:52:55.698 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:52:55.698 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:52:55.698 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:52:55.699 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:52:55.699 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": true, 2022-09-03T05:52:55.699 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:52:55.699 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:52:55.700 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:52:55.700 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:52:55.700 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:52:55.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:55.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:52:55.701 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:52:55.701 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone4", 2022-09-03T05:52:55.702 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:52:55.702 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi189.front.sepia.ceph.com:8001" 2022-09-03T05:52:55.702 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:52:55.702 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:52:55.703 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:52:55.703 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:52:55.703 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:52:55.704 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "pubsub", 2022-09-03T05:52:55.704 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": false, 2022-09-03T05:52:55.704 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [ 2022-09-03T05:52:55.704 INFO:teuthology.orchestra.run.smithi189.stdout: "test-zone1" 2022-09-03T05:52:55.705 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:52:55.705 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:52:55.705 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:52:55.705 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:52:55.706 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:52:55.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:55.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:52:55.706 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:52:55.707 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone1", 2022-09-03T05:52:55.707 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:52:55.707 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:52:55.707 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:52:55.708 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:52:55.708 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:52:55.708 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:52:55.708 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:52:55.709 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:52:55.709 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": true, 2022-09-03T05:52:55.709 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:52:55.709 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:52:55.710 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:52:55.710 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:52:55.710 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:52:55.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:55.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:52:55.711 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:52:55.711 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "test-zone2", 2022-09-03T05:52:55.711 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [ 2022-09-03T05:52:55.712 INFO:teuthology.orchestra.run.smithi189.stdout: "http://smithi189.front.sepia.ceph.com:8000" 2022-09-03T05:52:55.712 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:52:55.712 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:52:55.713 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:52:55.713 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:52:55.713 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:52:55.714 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:52:55.714 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": true, 2022-09-03T05:52:55.714 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:52:55.714 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:52:55.715 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:52:55.715 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:52:55.715 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:52:55.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:55.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:52:55.716 INFO:teuthology.orchestra.run.smithi189.stdout: "id": "e93f72de-f01b-431e-bf7a-c483294fc9d9", 2022-09-03T05:52:55.716 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "remove", 2022-09-03T05:52:55.718 INFO:teuthology.orchestra.run.smithi189.stdout: "endpoints": [], 2022-09-03T05:52:55.718 INFO:teuthology.orchestra.run.smithi189.stdout: "log_meta": false, 2022-09-03T05:52:55.718 INFO:teuthology.orchestra.run.smithi189.stdout: "log_data": true, 2022-09-03T05:52:55.718 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:52:55.719 INFO:teuthology.orchestra.run.smithi189.stdout: "read_only": false, 2022-09-03T05:52:55.719 INFO:teuthology.orchestra.run.smithi189.stdout: "tier_type": "", 2022-09-03T05:52:55.719 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from_all": true, 2022-09-03T05:52:55.720 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_from": [], 2022-09-03T05:52:55.720 INFO:teuthology.orchestra.run.smithi189.stdout: "redirect_zone": "", 2022-09-03T05:52:55.720 INFO:teuthology.orchestra.run.smithi189.stdout: "supported_features": [ 2022-09-03T05:52:55.720 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:52:55.721 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:52:55.721 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:52:55.721 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:52:55.721 INFO:teuthology.orchestra.run.smithi189.stdout: "placement_targets": [ 2022-09-03T05:52:55.722 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:52:55.722 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "default-placement", 2022-09-03T05:52:55.722 INFO:teuthology.orchestra.run.smithi189.stdout: "tags": [], 2022-09-03T05:52:55.723 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_classes": [ 2022-09-03T05:52:55.723 INFO:teuthology.orchestra.run.smithi189.stdout: "STANDARD" 2022-09-03T05:52:55.724 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:52:55.724 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:52:55.725 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:52:55.725 INFO:teuthology.orchestra.run.smithi189.stdout: "default_placement": "default-placement", 2022-09-03T05:52:55.726 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:52:55.727 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_policy": { 2022-09-03T05:52:55.727 INFO:teuthology.orchestra.run.smithi189.stdout: "groups": [] 2022-09-03T05:52:55.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:55.728 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled_features": [ 2022-09-03T05:52:55.729 INFO:teuthology.orchestra.run.smithi189.stdout: "resharding" 2022-09-03T05:52:55.729 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:52:55.730 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:52:55.730 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2022-09-03T05:52:55.731 INFO:teuthology.orchestra.run.smithi189.stdout: "short_zone_ids": [ 2022-09-03T05:52:55.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:52:55.732 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:52:55.733 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 327838863 2022-09-03T05:52:55.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:55.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:52:55.734 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:52:55.735 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 1690651894 2022-09-03T05:52:55.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:55.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:52:55.736 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:52:55.737 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 1752562153 2022-09-03T05:52:55.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:55.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:52:55.739 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:52:55.739 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 2011734872 2022-09-03T05:52:55.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:55.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:52:55.741 INFO:teuthology.orchestra.run.smithi189.stdout: "key": "e93f72de-f01b-431e-bf7a-c483294fc9d9", 2022-09-03T05:52:55.741 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 935370909 2022-09-03T05:52:55.742 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:52:55.742 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:52:55.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:55.743 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zonegroup": "test-zonegroup", 2022-09-03T05:52:55.744 INFO:teuthology.orchestra.run.smithi189.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:52:55.744 INFO:teuthology.orchestra.run.smithi189.stdout: "period_config": { 2022-09-03T05:52:55.745 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_quota": { 2022-09-03T05:52:55.745 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false, 2022-09-03T05:52:55.746 INFO:teuthology.orchestra.run.smithi189.stdout: "check_on_raw": false, 2022-09-03T05:52:55.747 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size": -1, 2022-09-03T05:52:55.747 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size_kb": 0, 2022-09-03T05:52:55.748 INFO:teuthology.orchestra.run.smithi189.stdout: "max_objects": -1 2022-09-03T05:52:55.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:55.749 INFO:teuthology.orchestra.run.smithi189.stdout: "user_quota": { 2022-09-03T05:52:55.749 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false, 2022-09-03T05:52:55.750 INFO:teuthology.orchestra.run.smithi189.stdout: "check_on_raw": false, 2022-09-03T05:52:55.750 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size": -1, 2022-09-03T05:52:55.751 INFO:teuthology.orchestra.run.smithi189.stdout: "max_size_kb": 0, 2022-09-03T05:52:55.751 INFO:teuthology.orchestra.run.smithi189.stdout: "max_objects": -1 2022-09-03T05:52:55.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:55.752 INFO:teuthology.orchestra.run.smithi189.stdout: "user_ratelimit": { 2022-09-03T05:52:55.753 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:52:55.753 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:52:55.754 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:52:55.754 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:52:55.755 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:52:55.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:55.756 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket_ratelimit": { 2022-09-03T05:52:55.756 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:52:55.757 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:52:55.757 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:52:55.758 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:52:55.758 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:52:55.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:55.759 INFO:teuthology.orchestra.run.smithi189.stdout: "anonymous_ratelimit": { 2022-09-03T05:52:55.760 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_ops": 0, 2022-09-03T05:52:55.760 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_ops": 0, 2022-09-03T05:52:55.761 INFO:teuthology.orchestra.run.smithi189.stdout: "max_read_bytes": 0, 2022-09-03T05:52:55.761 INFO:teuthology.orchestra.run.smithi189.stdout: "max_write_bytes": 0, 2022-09-03T05:52:55.762 INFO:teuthology.orchestra.run.smithi189.stdout: "enabled": false 2022-09-03T05:52:55.762 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:52:55.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:52:55.763 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:52:55.764 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_name": "test-realm", 2022-09-03T05:52:55.764 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:52:55.765 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:52:55.766 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--rgw-zone', 'remove', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:55.767 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--rgw-zone', 'remove', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:55.768 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup remove --rgw-zonegroup test-zonegroup --rgw-realm test-realm --rgw-zone remove --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:52:55.826 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:52:55.827 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:52:55.845 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:52:55.843+0000 7f9f4503c600 0 failed reading obj info from .rgw.root:zone_info.878fc3d4-c5f9-4296-8cc0-f2a8e029bc48: (2) No such file or directory 2022-09-03T05:52:55.846 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:52:55.843+0000 7f9f4503c600 0 WARNING: could not read zone params for zone id=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48 name=test-zone4 2022-09-03T05:52:55.846 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:52:55.844+0000 7f9f4503c600 0 failed reading obj info from .rgw.root:zone_info.e40d88ba-d542-483a-afa2-5de687affd14: (2) No such file or directory 2022-09-03T05:52:55.847 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:52:55.844+0000 7f9f4503c600 0 WARNING: could not read zone params for zone id=e40d88ba-d542-483a-afa2-5de687affd14 name=test-zone2 2022-09-03T05:52:55.858 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:52:55.859 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "test-zonegroup", 2022-09-03T05:52:55.859 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zonegroup", 2022-09-03T05:52:55.859 INFO:teuthology.orchestra.run.smithi181.stdout: "api_name": "", 2022-09-03T05:52:55.860 INFO:teuthology.orchestra.run.smithi181.stdout: "is_master": true, 2022-09-03T05:52:55.860 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:52:55.860 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:52:55.860 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:55.861 INFO:teuthology.orchestra.run.smithi181.stdout: "hostnames": [], 2022-09-03T05:52:55.861 INFO:teuthology.orchestra.run.smithi181.stdout: "hostnames_s3website": [], 2022-09-03T05:52:55.861 INFO:teuthology.orchestra.run.smithi181.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:52:55.861 INFO:teuthology.orchestra.run.smithi181.stdout: "zones": [ 2022-09-03T05:52:55.862 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:55.862 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:52:55.862 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone3", 2022-09-03T05:52:55.862 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:52:55.863 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi181.front.sepia.ceph.com:8001" 2022-09-03T05:52:55.863 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:55.863 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:52:55.863 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": true, 2022-09-03T05:52:55.864 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:52:55.864 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:52:55.864 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "", 2022-09-03T05:52:55.865 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": true, 2022-09-03T05:52:55.865 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [], 2022-09-03T05:52:55.865 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:52:55.865 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:52:55.866 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:52:55.866 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:55.866 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:55.866 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:55.867 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:52:55.867 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone4", 2022-09-03T05:52:55.867 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:52:55.867 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi189.front.sepia.ceph.com:8001" 2022-09-03T05:52:55.868 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:55.868 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:52:55.868 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": true, 2022-09-03T05:52:55.868 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:52:55.869 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:52:55.869 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "pubsub", 2022-09-03T05:52:55.869 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": false, 2022-09-03T05:52:55.869 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [ 2022-09-03T05:52:55.870 INFO:teuthology.orchestra.run.smithi181.stdout: "test-zone1" 2022-09-03T05:52:55.870 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:55.870 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:52:55.871 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:52:55.871 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:52:55.871 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:55.871 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:55.872 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:55.872 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:52:55.872 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone1", 2022-09-03T05:52:55.872 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:52:55.873 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:52:55.873 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:55.873 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:52:55.873 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": true, 2022-09-03T05:52:55.874 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:52:55.874 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:52:55.874 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "", 2022-09-03T05:52:55.874 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": true, 2022-09-03T05:52:55.875 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [], 2022-09-03T05:52:55.875 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:52:55.875 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:52:55.875 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:52:55.876 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:55.876 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:55.876 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:55.876 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:52:55.877 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone2", 2022-09-03T05:52:55.877 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:52:55.877 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi189.front.sepia.ceph.com:8000" 2022-09-03T05:52:55.877 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:55.878 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:52:55.878 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": true, 2022-09-03T05:52:55.878 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:52:55.878 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:52:55.879 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "", 2022-09-03T05:52:55.879 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": true, 2022-09-03T05:52:55.879 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [], 2022-09-03T05:52:55.879 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:52:55.880 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:52:55.880 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:52:55.880 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:55.880 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:55.881 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:55.881 INFO:teuthology.orchestra.run.smithi181.stdout: "placement_targets": [ 2022-09-03T05:52:55.881 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:55.881 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "default-placement", 2022-09-03T05:52:55.882 INFO:teuthology.orchestra.run.smithi181.stdout: "tags": [], 2022-09-03T05:52:55.882 INFO:teuthology.orchestra.run.smithi181.stdout: "storage_classes": [ 2022-09-03T05:52:55.882 INFO:teuthology.orchestra.run.smithi181.stdout: "STANDARD" 2022-09-03T05:52:55.882 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:55.883 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:55.883 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:55.883 INFO:teuthology.orchestra.run.smithi181.stdout: "default_placement": "default-placement", 2022-09-03T05:52:55.883 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:52:55.884 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_policy": { 2022-09-03T05:52:55.884 INFO:teuthology.orchestra.run.smithi181.stdout: "groups": [] 2022-09-03T05:52:55.884 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:55.884 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled_features": [ 2022-09-03T05:52:55.885 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:52:55.885 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:55.885 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:52:55.887 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--rgw-zone', 'remove', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:55.887 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'remove', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--rgw-zone', 'remove', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:55.888 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup remove --rgw-zonegroup test-zonegroup --rgw-realm test-realm --rgw-zone remove --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:52:55.952 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:52:55.953 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:52:55.973 INFO:teuthology.orchestra.run.smithi181.stderr:zone name remove not found in zonegroup test-zonegroup 2022-09-03T05:52:55.977 DEBUG:teuthology.orchestra.run:got remote process result: 2 2022-09-03T05:52:55.978 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['zone', 'delete', '--rgw-zone', 'remove', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:55.978 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'zone', 'delete', '--rgw-zone', 'remove', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:55.979 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 zone delete --rgw-zone remove --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T05:52:56.042 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:52:56.043 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:52:56.105 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:52:56.104+0000 7f2c0b2fe600 0 failed reading obj info from .rgw.root:zone_info.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4: (2) No such file or directory 2022-09-03T05:52:56.106 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:52:56.104+0000 7f2c0b2fe600 0 WARNING: could not read zone params for zone id=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4 name=test-zone3 2022-09-03T05:52:56.107 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:52:56.105+0000 7f2c0b2fe600 0 failed reading obj info from .rgw.root:zone_info.91786ef1-cdb4-4061-8a3e-2a28cc497285: (2) No such file or directory 2022-09-03T05:52:56.107 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:52:56.105+0000 7f2c0b2fe600 0 WARNING: could not read zone params for zone id=91786ef1-cdb4-4061-8a3e-2a28cc497285 name=test-zone1 2022-09-03T05:52:56.112 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:52:56.111+0000 7f2c0b2fe600 0 zone id e93f72de-f01b-431e-bf7a-c483294fc9d9 is not a part of zonegroup default 2022-09-03T05:52:56.127 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:56.128 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:56.129 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:52:56.202 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:52:56.202 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:52:56.216 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:52:56.214+0000 7fa5f4db4600 0 WARNING: can't generate connection for zone e93f72de-f01b-431e-bf7a-c483294fc9d9 id remove: no endpoints defined 2022-09-03T05:52:56.330 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:52:56.330 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:52:56.330 INFO:teuthology.orchestra.run.smithi181.stdout: "epoch": 7, 2022-09-03T05:52:56.331 INFO:teuthology.orchestra.run.smithi181.stdout: "predecessor_uuid": "eff2cb82-c479-4631-8568-fb1af4159e02", 2022-09-03T05:52:56.331 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": [ 2022-09-03T05:52:56.331 INFO:teuthology.orchestra.run.smithi181.stdout: "", 2022-09-03T05:52:56.332 INFO:teuthology.orchestra.run.smithi181.stdout: "", 2022-09-03T05:52:56.332 INFO:teuthology.orchestra.run.smithi181.stdout: "1_1662183936.347465_121.1", 2022-09-03T05:52:56.332 INFO:teuthology.orchestra.run.smithi181.stdout: "" 2022-09-03T05:52:56.332 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:56.333 INFO:teuthology.orchestra.run.smithi181.stdout: "period_map": { 2022-09-03T05:52:56.333 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:52:56.333 INFO:teuthology.orchestra.run.smithi181.stdout: "zonegroups": [ 2022-09-03T05:52:56.333 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.334 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "test-zonegroup", 2022-09-03T05:52:56.334 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zonegroup", 2022-09-03T05:52:56.334 INFO:teuthology.orchestra.run.smithi181.stdout: "api_name": "", 2022-09-03T05:52:56.335 INFO:teuthology.orchestra.run.smithi181.stdout: "is_master": true, 2022-09-03T05:52:56.335 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:52:56.335 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:52:56.335 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:56.336 INFO:teuthology.orchestra.run.smithi181.stdout: "hostnames": [], 2022-09-03T05:52:56.336 INFO:teuthology.orchestra.run.smithi181.stdout: "hostnames_s3website": [], 2022-09-03T05:52:56.336 INFO:teuthology.orchestra.run.smithi181.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:52:56.336 INFO:teuthology.orchestra.run.smithi181.stdout: "zones": [ 2022-09-03T05:52:56.337 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.337 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:52:56.337 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone3", 2022-09-03T05:52:56.337 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:52:56.338 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi181.front.sepia.ceph.com:8001" 2022-09-03T05:52:56.338 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:56.338 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:52:56.339 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": true, 2022-09-03T05:52:56.339 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:52:56.339 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:52:56.339 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "", 2022-09-03T05:52:56.340 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": true, 2022-09-03T05:52:56.340 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [], 2022-09-03T05:52:56.340 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:52:56.340 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:52:56.341 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:52:56.341 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:56.341 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.342 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.342 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:52:56.342 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone4", 2022-09-03T05:52:56.342 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:52:56.343 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi189.front.sepia.ceph.com:8001" 2022-09-03T05:52:56.343 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:56.343 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:52:56.343 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": true, 2022-09-03T05:52:56.344 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:52:56.344 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:52:56.344 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "pubsub", 2022-09-03T05:52:56.344 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": false, 2022-09-03T05:52:56.345 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [ 2022-09-03T05:52:56.345 INFO:teuthology.orchestra.run.smithi181.stdout: "test-zone1" 2022-09-03T05:52:56.345 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:56.346 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:52:56.346 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:52:56.346 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:52:56.346 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:56.347 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.347 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.347 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:52:56.347 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone1", 2022-09-03T05:52:56.348 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:52:56.348 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:52:56.348 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:56.348 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:52:56.349 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": true, 2022-09-03T05:52:56.349 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:52:56.349 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:52:56.349 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "", 2022-09-03T05:52:56.350 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": true, 2022-09-03T05:52:56.350 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [], 2022-09-03T05:52:56.350 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:52:56.351 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:52:56.351 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:52:56.351 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:56.351 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.352 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.352 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:52:56.352 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone2", 2022-09-03T05:52:56.352 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:52:56.353 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi189.front.sepia.ceph.com:8000" 2022-09-03T05:52:56.353 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:56.353 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:52:56.354 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": true, 2022-09-03T05:52:56.354 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:52:56.355 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:52:56.355 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "", 2022-09-03T05:52:56.355 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": true, 2022-09-03T05:52:56.355 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [], 2022-09-03T05:52:56.356 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:52:56.356 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:52:56.356 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:52:56.356 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:56.357 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:56.357 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:56.357 INFO:teuthology.orchestra.run.smithi181.stdout: "placement_targets": [ 2022-09-03T05:52:56.357 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.358 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "default-placement", 2022-09-03T05:52:56.366 INFO:teuthology.orchestra.run.smithi181.stdout: "tags": [], 2022-09-03T05:52:56.366 INFO:teuthology.orchestra.run.smithi181.stdout: "storage_classes": [ 2022-09-03T05:52:56.366 INFO:teuthology.orchestra.run.smithi181.stdout: "STANDARD" 2022-09-03T05:52:56.367 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:56.367 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:56.367 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:56.367 INFO:teuthology.orchestra.run.smithi181.stdout: "default_placement": "default-placement", 2022-09-03T05:52:56.368 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:52:56.368 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_policy": { 2022-09-03T05:52:56.368 INFO:teuthology.orchestra.run.smithi181.stdout: "groups": [] 2022-09-03T05:52:56.368 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.369 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled_features": [ 2022-09-03T05:52:56.369 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:52:56.369 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:56.370 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:56.370 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:56.370 INFO:teuthology.orchestra.run.smithi181.stdout: "short_zone_ids": [ 2022-09-03T05:52:56.370 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.371 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:52:56.371 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 327838863 2022-09-03T05:52:56.371 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.371 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.372 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:52:56.372 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 1690651894 2022-09-03T05:52:56.372 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.372 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.373 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:52:56.373 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 1752562153 2022-09-03T05:52:56.373 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.373 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.374 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:52:56.374 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 2011734872 2022-09-03T05:52:56.374 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:56.375 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:56.375 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.375 INFO:teuthology.orchestra.run.smithi181.stdout: "master_zonegroup": "test-zonegroup", 2022-09-03T05:52:56.376 INFO:teuthology.orchestra.run.smithi181.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:52:56.376 INFO:teuthology.orchestra.run.smithi181.stdout: "period_config": { 2022-09-03T05:52:56.376 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_quota": { 2022-09-03T05:52:56.376 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false, 2022-09-03T05:52:56.377 INFO:teuthology.orchestra.run.smithi181.stdout: "check_on_raw": false, 2022-09-03T05:52:56.377 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size": -1, 2022-09-03T05:52:56.377 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size_kb": 0, 2022-09-03T05:52:56.377 INFO:teuthology.orchestra.run.smithi181.stdout: "max_objects": -1 2022-09-03T05:52:56.378 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.378 INFO:teuthology.orchestra.run.smithi181.stdout: "user_quota": { 2022-09-03T05:52:56.378 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false, 2022-09-03T05:52:56.378 INFO:teuthology.orchestra.run.smithi181.stdout: "check_on_raw": false, 2022-09-03T05:52:56.379 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size": -1, 2022-09-03T05:52:56.379 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size_kb": 0, 2022-09-03T05:52:56.379 INFO:teuthology.orchestra.run.smithi181.stdout: "max_objects": -1 2022-09-03T05:52:56.379 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.380 INFO:teuthology.orchestra.run.smithi181.stdout: "user_ratelimit": { 2022-09-03T05:52:56.380 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_ops": 0, 2022-09-03T05:52:56.380 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_ops": 0, 2022-09-03T05:52:56.380 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_bytes": 0, 2022-09-03T05:52:56.381 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_bytes": 0, 2022-09-03T05:52:56.381 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false 2022-09-03T05:52:56.381 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.381 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_ratelimit": { 2022-09-03T05:52:56.382 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_ops": 0, 2022-09-03T05:52:56.382 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_ops": 0, 2022-09-03T05:52:56.382 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_bytes": 0, 2022-09-03T05:52:56.382 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_bytes": 0, 2022-09-03T05:52:56.383 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false 2022-09-03T05:52:56.383 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.383 INFO:teuthology.orchestra.run.smithi181.stdout: "anonymous_ratelimit": { 2022-09-03T05:52:56.383 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_ops": 0, 2022-09-03T05:52:56.384 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_ops": 0, 2022-09-03T05:52:56.384 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_bytes": 0, 2022-09-03T05:52:56.384 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_bytes": 0, 2022-09-03T05:52:56.384 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false 2022-09-03T05:52:56.385 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:56.385 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.385 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:52:56.385 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_name": "test-realm", 2022-09-03T05:52:56.386 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:52:56.386 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:52:56.388 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_zonegroup_remove ... ok 2022-09-03T05:52:56.388 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'create', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:56.389 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'create', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:56.390 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup create --rgw-zonegroup remove_zg --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:52:56.440 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:52:56.441 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:52:56.472 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:52:56.472 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "ce5ede43-3738-4a48-af9b-cb338501f362", 2022-09-03T05:52:56.472 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "remove_zg", 2022-09-03T05:52:56.473 INFO:teuthology.orchestra.run.smithi181.stdout: "api_name": "remove_zg", 2022-09-03T05:52:56.473 INFO:teuthology.orchestra.run.smithi181.stdout: "is_master": false, 2022-09-03T05:52:56.473 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [], 2022-09-03T05:52:56.473 INFO:teuthology.orchestra.run.smithi181.stdout: "hostnames": [], 2022-09-03T05:52:56.474 INFO:teuthology.orchestra.run.smithi181.stdout: "hostnames_s3website": [], 2022-09-03T05:52:56.474 INFO:teuthology.orchestra.run.smithi181.stdout: "master_zone": "", 2022-09-03T05:52:56.474 INFO:teuthology.orchestra.run.smithi181.stdout: "zones": [], 2022-09-03T05:52:56.475 INFO:teuthology.orchestra.run.smithi181.stdout: "placement_targets": [], 2022-09-03T05:52:56.475 INFO:teuthology.orchestra.run.smithi181.stdout: "default_placement": "", 2022-09-03T05:52:56.475 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:52:56.475 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_policy": { 2022-09-03T05:52:56.476 INFO:teuthology.orchestra.run.smithi181.stdout: "groups": [] 2022-09-03T05:52:56.476 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.476 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled_features": [ 2022-09-03T05:52:56.477 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:52:56.477 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:56.477 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:52:56.479 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'create', '--rgw-zone', 'remove', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:56.479 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'create', '--rgw-zone', 'remove', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:56.480 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone create --rgw-zone remove --rgw-zonegroup remove_zg --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:52:56.529 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:52:56.529 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:52:56.572 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:52:56.572 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "adbefda1-f271-40aa-a82c-8d6c9a2473ab", 2022-09-03T05:52:56.572 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "remove", 2022-09-03T05:52:56.572 INFO:teuthology.orchestra.run.smithi181.stdout: "domain_root": "remove.rgw.meta:root", 2022-09-03T05:52:56.573 INFO:teuthology.orchestra.run.smithi181.stdout: "control_pool": "remove.rgw.control", 2022-09-03T05:52:56.573 INFO:teuthology.orchestra.run.smithi181.stdout: "gc_pool": "remove.rgw.log:gc", 2022-09-03T05:52:56.573 INFO:teuthology.orchestra.run.smithi181.stdout: "lc_pool": "remove.rgw.log:lc", 2022-09-03T05:52:56.574 INFO:teuthology.orchestra.run.smithi181.stdout: "log_pool": "remove.rgw.log", 2022-09-03T05:52:56.574 INFO:teuthology.orchestra.run.smithi181.stdout: "intent_log_pool": "remove.rgw.log:intent", 2022-09-03T05:52:56.574 INFO:teuthology.orchestra.run.smithi181.stdout: "usage_log_pool": "remove.rgw.log:usage", 2022-09-03T05:52:56.574 INFO:teuthology.orchestra.run.smithi181.stdout: "roles_pool": "remove.rgw.meta:roles", 2022-09-03T05:52:56.575 INFO:teuthology.orchestra.run.smithi181.stdout: "reshard_pool": "remove.rgw.log:reshard", 2022-09-03T05:52:56.575 INFO:teuthology.orchestra.run.smithi181.stdout: "user_keys_pool": "remove.rgw.meta:users.keys", 2022-09-03T05:52:56.575 INFO:teuthology.orchestra.run.smithi181.stdout: "user_email_pool": "remove.rgw.meta:users.email", 2022-09-03T05:52:56.575 INFO:teuthology.orchestra.run.smithi181.stdout: "user_swift_pool": "remove.rgw.meta:users.swift", 2022-09-03T05:52:56.576 INFO:teuthology.orchestra.run.smithi181.stdout: "user_uid_pool": "remove.rgw.meta:users.uid", 2022-09-03T05:52:56.576 INFO:teuthology.orchestra.run.smithi181.stdout: "otp_pool": "remove.rgw.otp", 2022-09-03T05:52:56.576 INFO:teuthology.orchestra.run.smithi181.stdout: "system_key": { 2022-09-03T05:52:56.576 INFO:teuthology.orchestra.run.smithi181.stdout: "access_key": "", 2022-09-03T05:52:56.577 INFO:teuthology.orchestra.run.smithi181.stdout: "secret_key": "" 2022-09-03T05:52:56.577 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.577 INFO:teuthology.orchestra.run.smithi181.stdout: "placement_pools": [ 2022-09-03T05:52:56.578 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.578 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "default-placement", 2022-09-03T05:52:56.578 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:52:56.578 INFO:teuthology.orchestra.run.smithi181.stdout: "index_pool": "remove.rgw.buckets.index", 2022-09-03T05:52:56.579 INFO:teuthology.orchestra.run.smithi181.stdout: "storage_classes": { 2022-09-03T05:52:56.579 INFO:teuthology.orchestra.run.smithi181.stdout: "STANDARD": { 2022-09-03T05:52:56.579 INFO:teuthology.orchestra.run.smithi181.stdout: "data_pool": "remove.rgw.buckets.data" 2022-09-03T05:52:56.579 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:56.580 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.580 INFO:teuthology.orchestra.run.smithi181.stdout: "data_extra_pool": "remove.rgw.buckets.non-ec", 2022-09-03T05:52:56.580 INFO:teuthology.orchestra.run.smithi181.stdout: "index_type": 0 2022-09-03T05:52:56.580 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:56.581 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:56.581 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:56.581 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:52:56.581 INFO:teuthology.orchestra.run.smithi181.stdout: "notif_pool": "remove.rgw.log:notif" 2022-09-03T05:52:56.582 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:52:56.583 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:56.584 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:56.584 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:52:56.637 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:52:56.638 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:52:56.771 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:52:56.772 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:52:56.772 INFO:teuthology.orchestra.run.smithi181.stdout: "epoch": 8, 2022-09-03T05:52:56.773 INFO:teuthology.orchestra.run.smithi181.stdout: "predecessor_uuid": "eff2cb82-c479-4631-8568-fb1af4159e02", 2022-09-03T05:52:56.773 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": [ 2022-09-03T05:52:56.774 INFO:teuthology.orchestra.run.smithi181.stdout: "", 2022-09-03T05:52:56.774 INFO:teuthology.orchestra.run.smithi181.stdout: "", 2022-09-03T05:52:56.774 INFO:teuthology.orchestra.run.smithi181.stdout: "1_1662183936.347465_121.1", 2022-09-03T05:52:56.774 INFO:teuthology.orchestra.run.smithi181.stdout: "" 2022-09-03T05:52:56.775 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:56.775 INFO:teuthology.orchestra.run.smithi181.stdout: "period_map": { 2022-09-03T05:52:56.775 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:52:56.775 INFO:teuthology.orchestra.run.smithi181.stdout: "zonegroups": [ 2022-09-03T05:52:56.776 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.776 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "ce5ede43-3738-4a48-af9b-cb338501f362", 2022-09-03T05:52:56.776 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "remove_zg", 2022-09-03T05:52:56.776 INFO:teuthology.orchestra.run.smithi181.stdout: "api_name": "remove_zg", 2022-09-03T05:52:56.777 INFO:teuthology.orchestra.run.smithi181.stdout: "is_master": false, 2022-09-03T05:52:56.777 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [], 2022-09-03T05:52:56.777 INFO:teuthology.orchestra.run.smithi181.stdout: "hostnames": [], 2022-09-03T05:52:56.778 INFO:teuthology.orchestra.run.smithi181.stdout: "hostnames_s3website": [], 2022-09-03T05:52:56.778 INFO:teuthology.orchestra.run.smithi181.stdout: "master_zone": "adbefda1-f271-40aa-a82c-8d6c9a2473ab", 2022-09-03T05:52:56.778 INFO:teuthology.orchestra.run.smithi181.stdout: "zones": [ 2022-09-03T05:52:56.778 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.779 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "adbefda1-f271-40aa-a82c-8d6c9a2473ab", 2022-09-03T05:52:56.779 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "remove", 2022-09-03T05:52:56.779 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [], 2022-09-03T05:52:56.779 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:52:56.780 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": false, 2022-09-03T05:52:56.780 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:52:56.780 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:52:56.780 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "", 2022-09-03T05:52:56.781 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": true, 2022-09-03T05:52:56.781 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [], 2022-09-03T05:52:56.781 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:52:56.781 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:52:56.782 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:52:56.782 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:56.783 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:56.783 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:56.783 INFO:teuthology.orchestra.run.smithi181.stdout: "placement_targets": [ 2022-09-03T05:52:56.783 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.784 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "default-placement", 2022-09-03T05:52:56.784 INFO:teuthology.orchestra.run.smithi181.stdout: "tags": [], 2022-09-03T05:52:56.784 INFO:teuthology.orchestra.run.smithi181.stdout: "storage_classes": [ 2022-09-03T05:52:56.784 INFO:teuthology.orchestra.run.smithi181.stdout: "STANDARD" 2022-09-03T05:52:56.785 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:56.785 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:56.785 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:56.785 INFO:teuthology.orchestra.run.smithi181.stdout: "default_placement": "default-placement", 2022-09-03T05:52:56.786 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:52:56.786 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_policy": { 2022-09-03T05:52:56.786 INFO:teuthology.orchestra.run.smithi181.stdout: "groups": [] 2022-09-03T05:52:56.786 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.787 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled_features": [ 2022-09-03T05:52:56.787 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:52:56.787 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:56.787 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.788 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.788 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "test-zonegroup", 2022-09-03T05:52:56.788 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zonegroup", 2022-09-03T05:52:56.788 INFO:teuthology.orchestra.run.smithi181.stdout: "api_name": "", 2022-09-03T05:52:56.789 INFO:teuthology.orchestra.run.smithi181.stdout: "is_master": true, 2022-09-03T05:52:56.789 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:52:56.789 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:52:56.789 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:56.790 INFO:teuthology.orchestra.run.smithi181.stdout: "hostnames": [], 2022-09-03T05:52:56.790 INFO:teuthology.orchestra.run.smithi181.stdout: "hostnames_s3website": [], 2022-09-03T05:52:56.790 INFO:teuthology.orchestra.run.smithi181.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:52:56.791 INFO:teuthology.orchestra.run.smithi181.stdout: "zones": [ 2022-09-03T05:52:56.791 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.791 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:52:56.791 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone3", 2022-09-03T05:52:56.792 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:52:56.792 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi181.front.sepia.ceph.com:8001" 2022-09-03T05:52:56.792 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:56.792 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:52:56.793 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": true, 2022-09-03T05:52:56.793 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:52:56.793 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:52:56.793 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "", 2022-09-03T05:52:56.794 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": true, 2022-09-03T05:52:56.794 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [], 2022-09-03T05:52:56.794 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:52:56.794 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:52:56.795 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:52:56.795 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:56.795 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.795 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.796 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:52:56.796 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone4", 2022-09-03T05:52:56.796 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:52:56.796 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi189.front.sepia.ceph.com:8001" 2022-09-03T05:52:56.797 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:56.797 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:52:56.797 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": true, 2022-09-03T05:52:56.797 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:52:56.798 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:52:56.798 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "pubsub", 2022-09-03T05:52:56.798 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": false, 2022-09-03T05:52:56.799 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [ 2022-09-03T05:52:56.799 INFO:teuthology.orchestra.run.smithi181.stdout: "test-zone1" 2022-09-03T05:52:56.799 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:56.799 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:52:56.800 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:52:56.800 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:52:56.800 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:56.807 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.807 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.807 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:52:56.808 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone1", 2022-09-03T05:52:56.808 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:52:56.808 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:52:56.809 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:56.809 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:52:56.809 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": true, 2022-09-03T05:52:56.810 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:52:56.810 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:52:56.810 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "", 2022-09-03T05:52:56.810 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": true, 2022-09-03T05:52:56.811 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [], 2022-09-03T05:52:56.811 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:52:56.811 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:52:56.811 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:52:56.812 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:56.812 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.812 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.812 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:52:56.813 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone2", 2022-09-03T05:52:56.813 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:52:56.813 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi189.front.sepia.ceph.com:8000" 2022-09-03T05:52:56.813 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:56.814 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:52:56.814 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": true, 2022-09-03T05:52:56.814 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:52:56.814 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:52:56.815 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "", 2022-09-03T05:52:56.815 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": true, 2022-09-03T05:52:56.815 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [], 2022-09-03T05:52:56.815 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:52:56.816 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:52:56.816 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:52:56.816 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:56.816 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:56.817 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:56.817 INFO:teuthology.orchestra.run.smithi181.stdout: "placement_targets": [ 2022-09-03T05:52:56.817 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.817 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "default-placement", 2022-09-03T05:52:56.818 INFO:teuthology.orchestra.run.smithi181.stdout: "tags": [], 2022-09-03T05:52:56.818 INFO:teuthology.orchestra.run.smithi181.stdout: "storage_classes": [ 2022-09-03T05:52:56.818 INFO:teuthology.orchestra.run.smithi181.stdout: "STANDARD" 2022-09-03T05:52:56.819 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:56.819 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:56.819 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:56.819 INFO:teuthology.orchestra.run.smithi181.stdout: "default_placement": "default-placement", 2022-09-03T05:52:56.820 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:52:56.820 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_policy": { 2022-09-03T05:52:56.820 INFO:teuthology.orchestra.run.smithi181.stdout: "groups": [] 2022-09-03T05:52:56.820 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.821 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled_features": [ 2022-09-03T05:52:56.821 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:52:56.821 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:56.821 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:56.822 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:56.822 INFO:teuthology.orchestra.run.smithi181.stdout: "short_zone_ids": [ 2022-09-03T05:52:56.822 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.822 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:52:56.823 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 327838863 2022-09-03T05:52:56.823 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.823 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.823 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:52:56.824 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 1690651894 2022-09-03T05:52:56.824 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.824 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.824 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:52:56.825 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 1752562153 2022-09-03T05:52:56.825 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.825 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.825 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "adbefda1-f271-40aa-a82c-8d6c9a2473ab", 2022-09-03T05:52:56.826 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 2974178851 2022-09-03T05:52:56.826 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.826 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:56.826 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:52:56.827 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 2011734872 2022-09-03T05:52:56.827 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:56.827 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:56.827 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.828 INFO:teuthology.orchestra.run.smithi181.stdout: "master_zonegroup": "test-zonegroup", 2022-09-03T05:52:56.828 INFO:teuthology.orchestra.run.smithi181.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:52:56.828 INFO:teuthology.orchestra.run.smithi181.stdout: "period_config": { 2022-09-03T05:52:56.828 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_quota": { 2022-09-03T05:52:56.829 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false, 2022-09-03T05:52:56.829 INFO:teuthology.orchestra.run.smithi181.stdout: "check_on_raw": false, 2022-09-03T05:52:56.829 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size": -1, 2022-09-03T05:52:56.829 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size_kb": 0, 2022-09-03T05:52:56.830 INFO:teuthology.orchestra.run.smithi181.stdout: "max_objects": -1 2022-09-03T05:52:56.830 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.830 INFO:teuthology.orchestra.run.smithi181.stdout: "user_quota": { 2022-09-03T05:52:56.830 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false, 2022-09-03T05:52:56.831 INFO:teuthology.orchestra.run.smithi181.stdout: "check_on_raw": false, 2022-09-03T05:52:56.831 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size": -1, 2022-09-03T05:52:56.831 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size_kb": 0, 2022-09-03T05:52:56.831 INFO:teuthology.orchestra.run.smithi181.stdout: "max_objects": -1 2022-09-03T05:52:56.832 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.832 INFO:teuthology.orchestra.run.smithi181.stdout: "user_ratelimit": { 2022-09-03T05:52:56.832 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_ops": 0, 2022-09-03T05:52:56.832 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_ops": 0, 2022-09-03T05:52:56.833 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_bytes": 0, 2022-09-03T05:52:56.833 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_bytes": 0, 2022-09-03T05:52:56.833 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false 2022-09-03T05:52:56.833 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.834 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_ratelimit": { 2022-09-03T05:52:56.834 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_ops": 0, 2022-09-03T05:52:56.834 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_ops": 0, 2022-09-03T05:52:56.835 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_bytes": 0, 2022-09-03T05:52:56.835 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_bytes": 0, 2022-09-03T05:52:56.835 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false 2022-09-03T05:52:56.836 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.836 INFO:teuthology.orchestra.run.smithi181.stdout: "anonymous_ratelimit": { 2022-09-03T05:52:56.836 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_ops": 0, 2022-09-03T05:52:56.836 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_ops": 0, 2022-09-03T05:52:56.837 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_bytes": 0, 2022-09-03T05:52:56.837 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_bytes": 0, 2022-09-03T05:52:56.837 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false 2022-09-03T05:52:56.837 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:56.838 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:56.838 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:52:56.838 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_name": "test-realm", 2022-09-03T05:52:56.839 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:52:56.839 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:52:56.840 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zone', 'delete', '--rgw-zone', 'remove', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:56.841 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zone', 'delete', '--rgw-zone', 'remove', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:56.842 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zone delete --rgw-zone remove --rgw-zonegroup remove_zg --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:52:56.895 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:52:56.895 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:52:56.929 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:52:56.926+0000 7f0aeebdb600 0 zone id adbefda1-f271-40aa-a82c-8d6c9a2473ab is not a part of zonegroup test-zonegroup 2022-09-03T05:52:56.929 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:52:56.927+0000 7f0aeebdb600 0 zone id adbefda1-f271-40aa-a82c-8d6c9a2473ab is not a part of zonegroup default 2022-09-03T05:52:56.939 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:56.940 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:56.941 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:52:56.995 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:52:56.995 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:52:57.041 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:52:57.039+0000 7f2ce5c77600 0 ERROR: zonegroup remove_zg has a non existent master zone 2022-09-03T05:52:57.046 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-09-03T05:52:57.047 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['zonegroup', 'delete', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:57.048 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'zonegroup', 'delete', '--rgw-zonegroup', 'remove_zg', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:57.048 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 zonegroup delete --rgw-zonegroup remove_zg --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:52:57.113 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:52:57.113 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:52:57.161 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:57.162 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'period', 'update', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--commit', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:52:57.163 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 period update --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --commit --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:52:57.222 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:52:57.222 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:52:57.354 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:52:57.354 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:52:57.355 INFO:teuthology.orchestra.run.smithi181.stdout: "epoch": 9, 2022-09-03T05:52:57.355 INFO:teuthology.orchestra.run.smithi181.stdout: "predecessor_uuid": "eff2cb82-c479-4631-8568-fb1af4159e02", 2022-09-03T05:52:57.355 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": [ 2022-09-03T05:52:57.356 INFO:teuthology.orchestra.run.smithi181.stdout: "", 2022-09-03T05:52:57.356 INFO:teuthology.orchestra.run.smithi181.stdout: "", 2022-09-03T05:52:57.356 INFO:teuthology.orchestra.run.smithi181.stdout: "1_1662183936.347465_121.1", 2022-09-03T05:52:57.356 INFO:teuthology.orchestra.run.smithi181.stdout: "" 2022-09-03T05:52:57.357 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:57.357 INFO:teuthology.orchestra.run.smithi181.stdout: "period_map": { 2022-09-03T05:52:57.357 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:52:57.357 INFO:teuthology.orchestra.run.smithi181.stdout: "zonegroups": [ 2022-09-03T05:52:57.358 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:57.358 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "test-zonegroup", 2022-09-03T05:52:57.358 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zonegroup", 2022-09-03T05:52:57.359 INFO:teuthology.orchestra.run.smithi181.stdout: "api_name": "", 2022-09-03T05:52:57.359 INFO:teuthology.orchestra.run.smithi181.stdout: "is_master": true, 2022-09-03T05:52:57.359 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:52:57.359 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:52:57.360 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:57.360 INFO:teuthology.orchestra.run.smithi181.stdout: "hostnames": [], 2022-09-03T05:52:57.360 INFO:teuthology.orchestra.run.smithi181.stdout: "hostnames_s3website": [], 2022-09-03T05:52:57.361 INFO:teuthology.orchestra.run.smithi181.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:52:57.361 INFO:teuthology.orchestra.run.smithi181.stdout: "zones": [ 2022-09-03T05:52:57.361 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:57.362 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:52:57.362 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone3", 2022-09-03T05:52:57.362 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:52:57.362 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi181.front.sepia.ceph.com:8001" 2022-09-03T05:52:57.363 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:57.363 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:52:57.363 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": true, 2022-09-03T05:52:57.363 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:52:57.364 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:52:57.364 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "", 2022-09-03T05:52:57.364 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": true, 2022-09-03T05:52:57.365 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [], 2022-09-03T05:52:57.365 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:52:57.366 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:52:57.366 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:52:57.367 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:57.367 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:57.367 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:57.368 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:52:57.368 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone4", 2022-09-03T05:52:57.368 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:52:57.369 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi189.front.sepia.ceph.com:8001" 2022-09-03T05:52:57.369 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:57.369 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:52:57.369 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": true, 2022-09-03T05:52:57.370 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:52:57.370 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:52:57.370 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "pubsub", 2022-09-03T05:52:57.371 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": false, 2022-09-03T05:52:57.372 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [ 2022-09-03T05:52:57.372 INFO:teuthology.orchestra.run.smithi181.stdout: "test-zone1" 2022-09-03T05:52:57.372 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:57.373 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:52:57.373 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:52:57.373 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:52:57.374 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:57.374 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:57.374 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:57.374 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:52:57.375 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone1", 2022-09-03T05:52:57.375 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:52:57.375 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi181.front.sepia.ceph.com:8000" 2022-09-03T05:52:57.376 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:57.376 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:52:57.376 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": true, 2022-09-03T05:52:57.376 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:52:57.377 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:52:57.377 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "", 2022-09-03T05:52:57.377 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": true, 2022-09-03T05:52:57.377 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [], 2022-09-03T05:52:57.378 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:52:57.378 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:52:57.378 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:52:57.379 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:57.379 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:57.379 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:57.379 INFO:teuthology.orchestra.run.smithi181.stdout: "id": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:52:57.380 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "test-zone2", 2022-09-03T05:52:57.380 INFO:teuthology.orchestra.run.smithi181.stdout: "endpoints": [ 2022-09-03T05:52:57.380 INFO:teuthology.orchestra.run.smithi181.stdout: "http://smithi189.front.sepia.ceph.com:8000" 2022-09-03T05:52:57.380 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:57.381 INFO:teuthology.orchestra.run.smithi181.stdout: "log_meta": false, 2022-09-03T05:52:57.381 INFO:teuthology.orchestra.run.smithi181.stdout: "log_data": true, 2022-09-03T05:52:57.381 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_index_max_shards": 11, 2022-09-03T05:52:57.381 INFO:teuthology.orchestra.run.smithi181.stdout: "read_only": false, 2022-09-03T05:52:57.382 INFO:teuthology.orchestra.run.smithi181.stdout: "tier_type": "", 2022-09-03T05:52:57.382 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from_all": true, 2022-09-03T05:52:57.382 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_from": [], 2022-09-03T05:52:57.382 INFO:teuthology.orchestra.run.smithi181.stdout: "redirect_zone": "", 2022-09-03T05:52:57.383 INFO:teuthology.orchestra.run.smithi181.stdout: "supported_features": [ 2022-09-03T05:52:57.383 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:52:57.383 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:57.383 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:57.384 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:57.384 INFO:teuthology.orchestra.run.smithi181.stdout: "placement_targets": [ 2022-09-03T05:52:57.384 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:57.385 INFO:teuthology.orchestra.run.smithi181.stdout: "name": "default-placement", 2022-09-03T05:52:57.392 INFO:teuthology.orchestra.run.smithi181.stdout: "tags": [], 2022-09-03T05:52:57.393 INFO:teuthology.orchestra.run.smithi181.stdout: "storage_classes": [ 2022-09-03T05:52:57.393 INFO:teuthology.orchestra.run.smithi181.stdout: "STANDARD" 2022-09-03T05:52:57.393 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:57.394 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:57.394 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:57.395 INFO:teuthology.orchestra.run.smithi181.stdout: "default_placement": "default-placement", 2022-09-03T05:52:57.395 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:52:57.396 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_policy": { 2022-09-03T05:52:57.396 INFO:teuthology.orchestra.run.smithi181.stdout: "groups": [] 2022-09-03T05:52:57.396 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:57.397 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled_features": [ 2022-09-03T05:52:57.397 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding" 2022-09-03T05:52:57.398 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:57.398 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:57.399 INFO:teuthology.orchestra.run.smithi181.stdout: ], 2022-09-03T05:52:57.399 INFO:teuthology.orchestra.run.smithi181.stdout: "short_zone_ids": [ 2022-09-03T05:52:57.400 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:57.400 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:52:57.401 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 327838863 2022-09-03T05:52:57.401 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:57.402 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:57.402 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "878fc3d4-c5f9-4296-8cc0-f2a8e029bc48", 2022-09-03T05:52:57.403 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 1690651894 2022-09-03T05:52:57.403 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:57.403 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:57.403 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:52:57.404 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 1752562153 2022-09-03T05:52:57.404 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:57.404 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:52:57.404 INFO:teuthology.orchestra.run.smithi181.stdout: "key": "e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:52:57.405 INFO:teuthology.orchestra.run.smithi181.stdout: "val": 2011734872 2022-09-03T05:52:57.405 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:57.405 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:52:57.406 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:57.406 INFO:teuthology.orchestra.run.smithi181.stdout: "master_zonegroup": "test-zonegroup", 2022-09-03T05:52:57.406 INFO:teuthology.orchestra.run.smithi181.stdout: "master_zone": "91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:52:57.406 INFO:teuthology.orchestra.run.smithi181.stdout: "period_config": { 2022-09-03T05:52:57.407 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_quota": { 2022-09-03T05:52:57.407 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false, 2022-09-03T05:52:57.408 INFO:teuthology.orchestra.run.smithi181.stdout: "check_on_raw": false, 2022-09-03T05:52:57.408 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size": -1, 2022-09-03T05:52:57.408 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size_kb": 0, 2022-09-03T05:52:57.409 INFO:teuthology.orchestra.run.smithi181.stdout: "max_objects": -1 2022-09-03T05:52:57.409 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:57.409 INFO:teuthology.orchestra.run.smithi181.stdout: "user_quota": { 2022-09-03T05:52:57.410 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false, 2022-09-03T05:52:57.410 INFO:teuthology.orchestra.run.smithi181.stdout: "check_on_raw": false, 2022-09-03T05:52:57.410 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size": -1, 2022-09-03T05:52:57.411 INFO:teuthology.orchestra.run.smithi181.stdout: "max_size_kb": 0, 2022-09-03T05:52:57.411 INFO:teuthology.orchestra.run.smithi181.stdout: "max_objects": -1 2022-09-03T05:52:57.411 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:57.411 INFO:teuthology.orchestra.run.smithi181.stdout: "user_ratelimit": { 2022-09-03T05:52:57.412 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_ops": 0, 2022-09-03T05:52:57.412 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_ops": 0, 2022-09-03T05:52:57.412 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_bytes": 0, 2022-09-03T05:52:57.412 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_bytes": 0, 2022-09-03T05:52:57.413 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false 2022-09-03T05:52:57.413 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:57.413 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket_ratelimit": { 2022-09-03T05:52:57.414 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_ops": 0, 2022-09-03T05:52:57.414 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_ops": 0, 2022-09-03T05:52:57.414 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_bytes": 0, 2022-09-03T05:52:57.415 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_bytes": 0, 2022-09-03T05:52:57.415 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false 2022-09-03T05:52:57.415 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:57.415 INFO:teuthology.orchestra.run.smithi181.stdout: "anonymous_ratelimit": { 2022-09-03T05:52:57.416 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_ops": 0, 2022-09-03T05:52:57.416 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_ops": 0, 2022-09-03T05:52:57.416 INFO:teuthology.orchestra.run.smithi181.stdout: "max_read_bytes": 0, 2022-09-03T05:52:57.416 INFO:teuthology.orchestra.run.smithi181.stdout: "max_write_bytes": 0, 2022-09-03T05:52:57.417 INFO:teuthology.orchestra.run.smithi181.stdout: "enabled": false 2022-09-03T05:52:57.417 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:52:57.418 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:52:57.418 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_id": "06c68c5c-7fff-4622-880e-066dbd09a96b", 2022-09-03T05:52:57.418 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_name": "test-realm", 2022-09-03T05:52:57.418 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:52:57.419 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:52:57.420 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_zg_master_zone_delete ... ok 2022-09-03T05:52:57.421 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-50 2022-09-03T05:52:57.422 DEBUG:boto:path=/dsgvcy-50/ 2022-09-03T05:52:57.423 DEBUG:boto:auth_path=/dsgvcy-50/ 2022-09-03T05:52:57.424 DEBUG:boto:Method: PUT 2022-09-03T05:52:57.424 DEBUG:boto:Path: /dsgvcy-50/ 2022-09-03T05:52:57.425 DEBUG:boto:Data: 2022-09-03T05:52:57.425 DEBUG:boto:Headers: {} 2022-09-03T05:52:57.426 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:52:57.426 DEBUG:boto:Port: 8000 2022-09-03T05:52:57.427 DEBUG:boto:Params: {} 2022-09-03T05:52:57.428 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:52:57.429 DEBUG:boto:Token: None 2022-09-03T05:52:57.430 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:52:57 GMT /dsgvcy-50/ 2022-09-03T05:52:57.430 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zlCxepssqJH4w6vY6VXvUWg01IE= 2022-09-03T05:52:57.431 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:52:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zlCxepssqJH4w6vY6VXvUWg01IE=', 'Content-Length': '0'} 2022-09-03T05:52:58.142 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000083cf7604c35ebd8b-006312ebba-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:52:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:52:58.143 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=dsgvcy-51 2022-09-03T05:52:58.144 DEBUG:boto:path=/dsgvcy-51/ 2022-09-03T05:52:58.144 DEBUG:boto:auth_path=/dsgvcy-51/ 2022-09-03T05:52:58.145 DEBUG:boto:Method: PUT 2022-09-03T05:52:58.146 DEBUG:boto:Path: /dsgvcy-51/ 2022-09-03T05:52:58.146 DEBUG:boto:Data: 2022-09-03T05:52:58.147 DEBUG:boto:Headers: {} 2022-09-03T05:52:58.148 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:52:58.148 DEBUG:boto:Port: 8000 2022-09-03T05:52:58.149 DEBUG:boto:Params: {} 2022-09-03T05:52:58.149 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:52:58.150 DEBUG:boto:Token: None 2022-09-03T05:52:58.151 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:52:58 GMT /dsgvcy-51/ 2022-09-03T05:52:58.152 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z/sKJq+YhjhXKJ4XwC35G1uhkQs= 2022-09-03T05:52:58.153 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:52:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z/sKJq+YhjhXKJ4XwC35G1uhkQs=', 'Content-Length': '0'} 2022-09-03T05:52:58.237 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000007a1b576bc9a54eb6-006312ebba-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:52:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:52:58.238 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=dsgvcy-52 2022-09-03T05:52:58.239 DEBUG:boto:path=/dsgvcy-52/ 2022-09-03T05:52:58.239 DEBUG:boto:auth_path=/dsgvcy-52/ 2022-09-03T05:52:58.240 DEBUG:boto:Method: PUT 2022-09-03T05:52:58.240 DEBUG:boto:Path: /dsgvcy-52/ 2022-09-03T05:52:58.241 DEBUG:boto:Data: 2022-09-03T05:52:58.242 DEBUG:boto:Headers: {} 2022-09-03T05:52:58.242 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:52:58.243 DEBUG:boto:Port: 8001 2022-09-03T05:52:58.244 DEBUG:boto:Params: {} 2022-09-03T05:52:58.246 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T05:52:58.246 DEBUG:boto:Token: None 2022-09-03T05:52:58.247 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:52:58 GMT /dsgvcy-52/ 2022-09-03T05:52:58.248 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Um7zS6NHUT36+pmzvCsTIZUgU/w= 2022-09-03T05:52:58.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:52:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Um7zS6NHUT36+pmzvCsTIZUgU/w=', 'Content-Length': '0'} 2022-09-03T05:52:58.474 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000a2f3e00e6628a261-006312ebba-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:52:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:52:58.475 DEBUG:boto:path=/dsgvcy-50/ 2022-09-03T05:52:58.475 DEBUG:boto:auth_path=/dsgvcy-50/ 2022-09-03T05:52:58.476 DEBUG:boto:path=/dsgvcy-50/?website 2022-09-03T05:52:58.476 DEBUG:boto:auth_path=/dsgvcy-50/?website 2022-09-03T05:52:58.477 DEBUG:boto:Method: PUT 2022-09-03T05:52:58.477 DEBUG:boto:Path: /dsgvcy-50/?website 2022-09-03T05:52:58.478 DEBUG:boto:Data: index.htmlerror.html 2022-09-03T05:52:58.478 DEBUG:boto:Headers: {} 2022-09-03T05:52:58.479 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:52:58.479 DEBUG:boto:Port: 8000 2022-09-03T05:52:58.480 DEBUG:boto:Params: {} 2022-09-03T05:52:58.481 DEBUG:boto:Token: None 2022-09-03T05:52:58.481 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:52:58 GMT /dsgvcy-50/?website 2022-09-03T05:52:58.482 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TFevxRkxc3OYLxMowgrfN8wwE9g= 2022-09-03T05:52:58.482 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:52:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TFevxRkxc3OYLxMowgrfN8wwE9g=', 'Content-Length': '241'} 2022-09-03T05:52:58.484 DEBUG:boto:Response headers: [('Content-Length', '202'), ('x-amz-request-id', 'tx000001e868546a9163c7c-006312ebba-15d7-test-zone1'), ('Accept-Ranges', 'bytes'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 05:52:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:52:58.486 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_set_bucket_website ... SKIP: test_set_bucket_website skipped. Requires rgw_enable_static_website = 1. 2022-09-03T05:52:58.487 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-53 2022-09-03T05:52:58.487 DEBUG:boto:path=/dsgvcy-53/ 2022-09-03T05:52:58.488 DEBUG:boto:auth_path=/dsgvcy-53/ 2022-09-03T05:52:58.488 DEBUG:boto:Method: PUT 2022-09-03T05:52:58.489 DEBUG:boto:Path: /dsgvcy-53/ 2022-09-03T05:52:58.489 DEBUG:boto:Data: 2022-09-03T05:52:58.490 DEBUG:boto:Headers: {} 2022-09-03T05:52:58.490 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:52:58.491 DEBUG:boto:Port: 8000 2022-09-03T05:52:58.491 DEBUG:boto:Params: {} 2022-09-03T05:52:58.492 DEBUG:boto:Token: None 2022-09-03T05:52:58.493 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:52:58 GMT /dsgvcy-53/ 2022-09-03T05:52:58.493 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0Gk7f7cJArYB6HR2ZOmTH9jxM+s= 2022-09-03T05:52:58.494 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:52:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0Gk7f7cJArYB6HR2ZOmTH9jxM+s=', 'Content-Length': '0'} 2022-09-03T05:52:58.532 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000eda80823aec04aab-006312ebba-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:52:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:52:58.533 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=dsgvcy-54 2022-09-03T05:52:58.534 DEBUG:boto:path=/dsgvcy-54/ 2022-09-03T05:52:58.534 DEBUG:boto:auth_path=/dsgvcy-54/ 2022-09-03T05:52:58.535 DEBUG:boto:Method: PUT 2022-09-03T05:52:58.535 DEBUG:boto:Path: /dsgvcy-54/ 2022-09-03T05:52:58.536 DEBUG:boto:Data: 2022-09-03T05:52:58.536 DEBUG:boto:Headers: {} 2022-09-03T05:52:58.537 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:52:58.553 DEBUG:boto:Port: 8000 2022-09-03T05:52:58.553 DEBUG:boto:Params: {} 2022-09-03T05:52:58.554 DEBUG:boto:Token: None 2022-09-03T05:52:58.555 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:52:58 GMT /dsgvcy-54/ 2022-09-03T05:52:58.555 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sxuNTDoCk64eM1bEni8EpifYoQw= 2022-09-03T05:52:58.556 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:52:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sxuNTDoCk64eM1bEni8EpifYoQw=', 'Content-Length': '0'} 2022-09-03T05:52:58.629 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000a30621e843223978-006312ebba-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:52:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:52:58.630 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=dsgvcy-55 2022-09-03T05:52:58.630 DEBUG:boto:path=/dsgvcy-55/ 2022-09-03T05:52:58.631 DEBUG:boto:auth_path=/dsgvcy-55/ 2022-09-03T05:52:58.632 DEBUG:boto:Method: PUT 2022-09-03T05:52:58.632 DEBUG:boto:Path: /dsgvcy-55/ 2022-09-03T05:52:58.633 DEBUG:boto:Data: 2022-09-03T05:52:58.633 DEBUG:boto:Headers: {} 2022-09-03T05:52:58.634 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:52:58.634 DEBUG:boto:Port: 8001 2022-09-03T05:52:58.635 DEBUG:boto:Params: {} 2022-09-03T05:52:58.636 DEBUG:boto:Token: None 2022-09-03T05:52:58.636 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:52:58 GMT /dsgvcy-55/ 2022-09-03T05:52:58.637 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:apBy+3rH1VEkfjcW46VbwiFoe0E= 2022-09-03T05:52:58.638 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:52:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:apBy+3rH1VEkfjcW46VbwiFoe0E=', 'Content-Length': '0'} 2022-09-03T05:52:58.694 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000045b8f3644c03d504-006312ebba-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:52:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:52:58.695 DEBUG:boto:path=/dsgvcy-53/ 2022-09-03T05:52:58.695 DEBUG:boto:auth_path=/dsgvcy-53/ 2022-09-03T05:52:58.696 DEBUG:boto:path=/dsgvcy-53/?policy 2022-09-03T05:52:58.696 DEBUG:boto:auth_path=/dsgvcy-53/?policy 2022-09-03T05:52:58.697 DEBUG:boto:Method: PUT 2022-09-03T05:52:58.697 DEBUG:boto:Path: /dsgvcy-53/?policy 2022-09-03T05:52:58.698 DEBUG:boto:Data: { "Version": "2012-10-17", "Statement": [{ "Effect": "Allow", "Principal": "*" }] } 2022-09-03T05:52:58.698 DEBUG:boto:Headers: {} 2022-09-03T05:52:58.699 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:52:58.699 DEBUG:boto:Port: 8000 2022-09-03T05:52:58.700 DEBUG:boto:Params: {} 2022-09-03T05:52:58.702 DEBUG:boto:Token: None 2022-09-03T05:52:58.703 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:52:58 GMT /dsgvcy-53/?policy 2022-09-03T05:52:58.704 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HBP0bZhdhuaqyJ1GZyUnF7tUH/8= 2022-09-03T05:52:58.705 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:52:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HBP0bZhdhuaqyJ1GZyUnF7tUH/8=', 'Content-Length': '97'} 2022-09-03T05:52:58.724 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000055940518fb66a29a-006312ebba-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:52:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:52:58.725 DEBUG:boto:path=/dsgvcy-53/ 2022-09-03T05:52:58.726 DEBUG:boto:auth_path=/dsgvcy-53/ 2022-09-03T05:52:58.726 DEBUG:boto:path=/dsgvcy-53/?policy 2022-09-03T05:52:58.727 DEBUG:boto:auth_path=/dsgvcy-53/?policy 2022-09-03T05:52:58.728 DEBUG:boto:Method: GET 2022-09-03T05:52:58.728 DEBUG:boto:Path: /dsgvcy-53/?policy 2022-09-03T05:52:58.729 DEBUG:boto:Data: 2022-09-03T05:52:58.729 DEBUG:boto:Headers: {} 2022-09-03T05:52:58.730 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:52:58.730 DEBUG:boto:Port: 8000 2022-09-03T05:52:58.731 DEBUG:boto:Params: {} 2022-09-03T05:52:58.732 DEBUG:boto:Token: None 2022-09-03T05:52:58.733 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:52:58 GMT /dsgvcy-53/?policy 2022-09-03T05:52:58.733 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G9Jux3dMrgLR0sIqCGwvItMpaEU= 2022-09-03T05:52:58.734 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:52:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G9Jux3dMrgLR0sIqCGwvItMpaEU=', 'Content-Length': '0'} 2022-09-03T05:52:58.736 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000de8f001d826895cd-006312ebba-15d7-test-zone1'), ('Content-Type', 'application/json'), ('Content-Length', '97'), ('Date', 'Sat, 03 Sep 2022 05:52:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:52:58.736 DEBUG:boto:path=/dsgvcy-54/ 2022-09-03T05:52:58.737 DEBUG:boto:auth_path=/dsgvcy-54/ 2022-09-03T05:52:58.737 DEBUG:boto:path=/dsgvcy-54/?policy 2022-09-03T05:52:58.738 DEBUG:boto:auth_path=/dsgvcy-54/?policy 2022-09-03T05:52:58.738 DEBUG:boto:Method: PUT 2022-09-03T05:52:58.739 DEBUG:boto:Path: /dsgvcy-54/?policy 2022-09-03T05:52:58.739 DEBUG:boto:Data: { "Version": "2012-10-17", "Statement": [{ "Effect": "Allow", "Principal": "*" }] } 2022-09-03T05:52:58.740 DEBUG:boto:Headers: {} 2022-09-03T05:52:58.740 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:52:58.741 DEBUG:boto:Port: 8000 2022-09-03T05:52:58.742 DEBUG:boto:Params: {} 2022-09-03T05:52:58.742 DEBUG:boto:Token: None 2022-09-03T05:52:58.743 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:52:58 GMT /dsgvcy-54/?policy 2022-09-03T05:52:58.743 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XSFDxQGGJWlWWDrZtxX1q4el4T8= 2022-09-03T05:52:58.744 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:52:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XSFDxQGGJWlWWDrZtxX1q4el4T8=', 'Content-Length': '97'} 2022-09-03T05:52:59.753 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000b49f9b7062b0467b-006312ebba-15d2-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:52:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:52:59.754 DEBUG:boto:path=/dsgvcy-54/ 2022-09-03T05:52:59.755 DEBUG:boto:auth_path=/dsgvcy-54/ 2022-09-03T05:52:59.756 DEBUG:boto:path=/dsgvcy-54/?policy 2022-09-03T05:52:59.757 DEBUG:boto:auth_path=/dsgvcy-54/?policy 2022-09-03T05:52:59.758 DEBUG:boto:Method: GET 2022-09-03T05:52:59.759 DEBUG:boto:Path: /dsgvcy-54/?policy 2022-09-03T05:52:59.760 DEBUG:boto:Data: 2022-09-03T05:52:59.761 DEBUG:boto:Headers: {} 2022-09-03T05:52:59.762 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:52:59.763 DEBUG:boto:Port: 8000 2022-09-03T05:52:59.763 DEBUG:boto:Params: {} 2022-09-03T05:52:59.765 DEBUG:boto:Token: None 2022-09-03T05:52:59.765 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:52:59 GMT /dsgvcy-54/?policy 2022-09-03T05:52:59.766 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:u0sb2WdusD6F0A3VDtGBdcvEIcc= 2022-09-03T05:52:59.767 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:52:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:u0sb2WdusD6F0A3VDtGBdcvEIcc=', 'Content-Length': '0'} 2022-09-03T05:52:59.769 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000000fdfb3788f80dc43-006312ebbb-15d2-test-zone2'), ('Content-Type', 'application/json'), ('Content-Length', '97'), ('Date', 'Sat, 03 Sep 2022 05:52:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:52:59.770 DEBUG:boto:path=/dsgvcy-55/ 2022-09-03T05:52:59.771 DEBUG:boto:auth_path=/dsgvcy-55/ 2022-09-03T05:52:59.771 DEBUG:boto:path=/dsgvcy-55/?policy 2022-09-03T05:52:59.772 DEBUG:boto:auth_path=/dsgvcy-55/?policy 2022-09-03T05:52:59.772 DEBUG:boto:Method: PUT 2022-09-03T05:52:59.773 DEBUG:boto:Path: /dsgvcy-55/?policy 2022-09-03T05:52:59.773 DEBUG:boto:Data: { "Version": "2012-10-17", "Statement": [{ "Effect": "Allow", "Principal": "*" }] } 2022-09-03T05:52:59.774 DEBUG:boto:Headers: {} 2022-09-03T05:52:59.775 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:52:59.776 DEBUG:boto:Port: 8001 2022-09-03T05:52:59.777 DEBUG:boto:Params: {} 2022-09-03T05:52:59.778 DEBUG:boto:Token: None 2022-09-03T05:52:59.779 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:52:59 GMT /dsgvcy-55/?policy 2022-09-03T05:52:59.780 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WI/mw1dAp+UatNpLQpmD9sw1nM0= 2022-09-03T05:52:59.780 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:52:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WI/mw1dAp+UatNpLQpmD9sw1nM0=', 'Content-Length': '97'} 2022-09-03T05:53:00.791 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000087db0580d922af89-006312ebbb-15db-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:53:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:53:00.792 DEBUG:boto:path=/dsgvcy-55/ 2022-09-03T05:53:00.793 DEBUG:boto:auth_path=/dsgvcy-55/ 2022-09-03T05:53:00.793 DEBUG:boto:path=/dsgvcy-55/?policy 2022-09-03T05:53:00.794 DEBUG:boto:auth_path=/dsgvcy-55/?policy 2022-09-03T05:53:00.794 DEBUG:boto:Method: GET 2022-09-03T05:53:00.795 DEBUG:boto:Path: /dsgvcy-55/?policy 2022-09-03T05:53:00.795 DEBUG:boto:Data: 2022-09-03T05:53:00.796 DEBUG:boto:Headers: {} 2022-09-03T05:53:00.797 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:53:00.797 DEBUG:boto:Port: 8001 2022-09-03T05:53:00.798 DEBUG:boto:Params: {} 2022-09-03T05:53:00.798 DEBUG:boto:Token: None 2022-09-03T05:53:00.799 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 05:53:00 GMT /dsgvcy-55/?policy 2022-09-03T05:53:00.799 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:21OJjgnrwbJEh0ml9Qy5SZstFac= 2022-09-03T05:53:00.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:53:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:21OJjgnrwbJEh0ml9Qy5SZstFac=', 'Content-Length': '0'} 2022-09-03T05:53:00.802 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000f5a2db56d754ba3e-006312ebbc-15db-test-zone3'), ('Content-Type', 'application/json'), ('Content-Length', '97'), ('Date', 'Sat, 03 Sep 2022 05:53:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:53:00.803 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_set_bucket_policy ... ok 2022-09-03T05:53:00.804 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-56 2022-09-03T05:53:00.804 DEBUG:boto:path=/dsgvcy-56/ 2022-09-03T05:53:00.805 DEBUG:boto:auth_path=/dsgvcy-56/ 2022-09-03T05:53:00.805 DEBUG:boto:Method: PUT 2022-09-03T05:53:00.806 DEBUG:boto:Path: /dsgvcy-56/ 2022-09-03T05:53:00.806 DEBUG:boto:Data: 2022-09-03T05:53:00.807 DEBUG:boto:Headers: {} 2022-09-03T05:53:00.808 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:53:00.808 DEBUG:boto:Port: 8000 2022-09-03T05:53:00.809 DEBUG:boto:Params: {} 2022-09-03T05:53:00.810 DEBUG:boto:Token: None 2022-09-03T05:53:00.810 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:53:00 GMT /dsgvcy-56/ 2022-09-03T05:53:00.811 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TtLHhHmNBITR//8tHDcluJ+mkc0= 2022-09-03T05:53:00.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:53:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TtLHhHmNBITR//8tHDcluJ+mkc0=', 'Content-Length': '0'} 2022-09-03T05:53:00.831 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000009aac7dacfa2e4534-006312ebbc-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:53:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:53:00.832 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=dsgvcy-57 2022-09-03T05:53:00.832 DEBUG:boto:path=/dsgvcy-57/ 2022-09-03T05:53:00.833 DEBUG:boto:auth_path=/dsgvcy-57/ 2022-09-03T05:53:00.833 DEBUG:boto:Method: PUT 2022-09-03T05:53:00.834 DEBUG:boto:Path: /dsgvcy-57/ 2022-09-03T05:53:00.834 DEBUG:boto:Data: 2022-09-03T05:53:00.835 DEBUG:boto:Headers: {} 2022-09-03T05:53:00.835 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:53:00.836 DEBUG:boto:Port: 8000 2022-09-03T05:53:00.836 DEBUG:boto:Params: {} 2022-09-03T05:53:00.837 DEBUG:boto:Token: None 2022-09-03T05:53:00.838 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:53:00 GMT /dsgvcy-57/ 2022-09-03T05:53:00.838 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GUi8btQAxnjNX0xAqX/eVbg+YCE= 2022-09-03T05:53:00.839 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:53:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GUi8btQAxnjNX0xAqX/eVbg+YCE=', 'Content-Length': '0'} 2022-09-03T05:53:00.869 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000607204d2b96ec784-006312ebbc-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:53:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:53:00.870 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=dsgvcy-58 2022-09-03T05:53:00.871 DEBUG:boto:path=/dsgvcy-58/ 2022-09-03T05:53:00.871 DEBUG:boto:auth_path=/dsgvcy-58/ 2022-09-03T05:53:00.872 DEBUG:boto:Method: PUT 2022-09-03T05:53:00.873 DEBUG:boto:Path: /dsgvcy-58/ 2022-09-03T05:53:00.873 DEBUG:boto:Data: 2022-09-03T05:53:00.874 DEBUG:boto:Headers: {} 2022-09-03T05:53:00.874 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:53:00.875 DEBUG:boto:Port: 8001 2022-09-03T05:53:00.875 DEBUG:boto:Params: {} 2022-09-03T05:53:00.876 DEBUG:boto:Token: None 2022-09-03T05:53:00.876 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:53:00 GMT /dsgvcy-58/ 2022-09-03T05:53:00.877 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KtljpHjEiEl7ZdGIM1GjuCJuASo= 2022-09-03T05:53:00.878 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:53:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KtljpHjEiEl7ZdGIM1GjuCJuASo=', 'Content-Length': '0'} 2022-09-03T05:53:00.915 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000e741b784f9ebdaca-006312ebbc-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:53:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:53:00.916 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:00.917 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:00.918 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:53:00.970 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:53:00.970 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:53:01.026 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T05:53:01.049 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:53:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:53:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:53:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:53:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:53:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:53:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:53:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:53:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:53:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:53:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184380.897369_341.1", 2022-09-03T05:53:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:53:00.897369Z" 2022-09-03T05:53:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:53:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:53:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:53:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:53:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:53:01.054 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:53:01.055 DEBUG:rgw_multi.tests:master meta markers={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:53:01.056 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:53:01.056 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:01.057 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:01.058 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:53:01.117 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:53:01.117 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:53:01.150 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:01.148+0000 7f53da44c600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:01.151 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:01.150+0000 7f53da44c600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:01.164 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:53:01.165 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:53:01.165 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:53:01.166 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:53:01.166 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:53:01.166 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:53:01.166 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:01.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:01.167 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:53:01.167 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:01.167 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:53:01.168 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:01.168 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:01.168 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:53:01.169 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:01.169 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:53:01.169 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:01.169 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:53:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:01.170 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:53:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:01.171 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:01.172 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:53:01.173 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:01.173 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:01.174 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:01.174 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:53:01.174 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:01.175 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:01.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:01.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:01.175 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:53:01.176 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:01.176 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:01.176 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:53:01.177 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:01.177 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:01.177 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:01.177 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:48:45.211231Z", 2022-09-03T05:53:01.178 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:01.178 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:01.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:01.178 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:01.179 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:53:01.179 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:01.179 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:01.179 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:53:01.180 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:01.180 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:01.181 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:01.181 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:53:01.182 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:01.182 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:01.183 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:01.183 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:53:01.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:01.184 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:53:01.185 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:53:01.185 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:53:01.186 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:01.186 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:53:01Z" 2022-09-03T05:53:01.187 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:53:01.188 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184125.211231_206.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:48:45.211231Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:53:01Z" } 2022-09-03T05:53:01.188 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184125.211231_206.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:48:45.211231Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:53:01.189 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:53:01.189 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:53:01.190 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184125.211231_206.1 2022-09-03T05:53:06.191 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:06.192 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:06.193 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:53:06.251 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:53:06.252 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:53:06.282 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:06.279+0000 7ff520952600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:06.283 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:06.281+0000 7ff520952600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:06.298 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:53:06.299 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:53:06.299 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:53:06.300 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:53:06.300 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:53:06.300 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:53:06.301 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:06.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:06.301 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:53:06.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:06.302 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:53:06.302 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:06.302 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:06.303 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:53:06.303 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:06.303 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:53:06.304 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:06.304 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:53:06.304 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:06.305 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:06.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:06.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:06.306 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:53:06.306 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:06.306 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:06.306 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:53:06.307 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:06.307 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:06.307 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:06.307 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:53:06.308 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:06.308 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:06.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:06.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:06.310 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:53:06.310 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:06.311 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:06.311 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:53:06.312 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:06.312 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:06.313 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:06.313 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:48:45.211231Z", 2022-09-03T05:53:06.314 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:06.314 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:06.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:06.315 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:06.316 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:53:06.316 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:06.317 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:06.317 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:53:06.317 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:06.318 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:06.319 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:06.319 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:53:06.319 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:06.320 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:06.320 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:06.320 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:53:06.321 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:06.321 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:53:06.321 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:53:06.322 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:53:06.322 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:06.322 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:53:06Z" 2022-09-03T05:53:06.322 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:53:06.324 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184125.211231_206.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:48:45.211231Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:53:06Z" } 2022-09-03T05:53:06.324 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184125.211231_206.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:48:45.211231Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:53:06.325 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:53:06.325 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:53:06.326 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184125.211231_206.1 2022-09-03T05:53:11.327 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:11.328 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:11.329 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:53:11.386 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:53:11.386 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:53:11.417 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:11.416+0000 7f544fadf600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:11.419 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:11.417+0000 7f544fadf600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:11.431 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:53:11.432 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:53:11.432 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:53:11.433 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:53:11.433 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:53:11.433 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:53:11.433 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:11.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:11.434 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:53:11.434 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:11.435 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:53:11.435 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:11.435 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:11.435 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:53:11.436 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:11.436 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:53:11.436 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:11.436 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:53:11.437 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:11.437 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:11.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:11.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:11.438 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:53:11.438 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:11.438 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:11.438 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:53:11.439 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:11.439 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:11.439 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:11.439 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:53:11.440 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:11.440 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:11.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:11.440 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:11.441 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:53:11.441 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:11.441 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:11.441 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:53:11.442 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:11.442 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:11.442 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:11.442 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:48:45.211231Z", 2022-09-03T05:53:11.443 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:11.443 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:11.443 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:11.443 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:11.444 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:53:11.444 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:11.444 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:11.444 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:53:11.445 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:11.445 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:11.445 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:11.445 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:53:11.446 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:11.446 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:11.446 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:11.446 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:53:11.447 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:11.447 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:53:11.447 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:53:11.447 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:53:11.448 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:11.448 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:53:11Z" 2022-09-03T05:53:11.448 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:53:11.449 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184125.211231_206.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:48:45.211231Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:53:11Z" } 2022-09-03T05:53:11.450 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184125.211231_206.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:48:45.211231Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:53:11.451 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:53:11.451 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:53:11.452 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184125.211231_206.1 2022-09-03T05:53:16.453 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:16.454 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:16.454 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:53:16.511 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:53:16.511 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:53:16.551 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:16.549+0000 7f086ba6d600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:16.552 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:16.550+0000 7f086ba6d600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:16.569 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:53:16.570 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:53:16.570 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:53:16.571 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:53:16.571 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:53:16.572 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:53:16.572 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:16.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:16.573 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:53:16.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:16.574 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:53:16.575 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:16.575 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:16.575 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:53:16.576 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:16.576 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:53:16.577 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:16.577 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:53:16.578 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:16.578 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:16.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:16.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:16.580 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:53:16.580 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:16.581 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:16.581 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:53:16.582 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:16.582 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:16.583 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:16.583 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:53:16.584 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:16.584 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:16.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:16.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:16.585 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:53:16.586 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:16.586 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:16.587 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:53:16.587 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:16.588 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:16.588 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:16.589 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:48:45.211231Z", 2022-09-03T05:53:16.589 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:16.590 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:16.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:16.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:16.591 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:53:16.592 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:16.592 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:16.593 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:53:16.593 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:16.594 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:16.594 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:16.595 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:53:16.595 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:16.596 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:16.596 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:16.597 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:53:16.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:16.598 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:53:16.598 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:53:16.598 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:53:16.599 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:16.599 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:53:16Z" 2022-09-03T05:53:16.599 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:53:16.601 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184125.211231_206.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:48:45.211231Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:53:16Z" } 2022-09-03T05:53:16.602 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184125.211231_206.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:48:45.211231Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:53:16.603 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:53:16.603 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:53:16.604 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184125.211231_206.1 2022-09-03T05:53:21.605 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:21.606 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:21.606 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:53:21.663 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:53:21.664 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:53:21.695 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:21.693+0000 7f5801586600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:21.696 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:21.694+0000 7f5801586600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:21.709 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:53:21.710 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:53:21.710 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:53:21.710 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:53:21.711 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:53:21.711 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:53:21.711 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:21.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:21.712 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:53:21.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:21.712 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:53:21.713 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:21.713 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:21.713 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:53:21.714 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:21.714 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:53:21.715 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:21.716 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:53:21.716 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:21.717 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:21.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:21.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:21.718 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:53:21.719 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:21.719 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:21.720 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:53:21.720 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:21.721 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:21.721 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:21.722 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:53:21.722 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:21.723 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:21.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:21.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:21.724 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:53:21.725 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:21.725 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:21.726 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:53:21.726 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:21.726 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:21.727 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:21.727 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:48:45.211231Z", 2022-09-03T05:53:21.728 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:21.728 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:21.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:21.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:21.730 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:53:21.730 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:21.731 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:21.731 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:53:21.731 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:21.732 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:21.732 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:21.733 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:53:21.733 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:21.733 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:21.734 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:21.734 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:53:21.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:21.735 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:53:21.735 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:53:21.735 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:53:21.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:21.736 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:53:21Z" 2022-09-03T05:53:21.736 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:53:21.737 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184125.211231_206.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:48:45.211231Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:53:21Z" } 2022-09-03T05:53:21.738 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184125.211231_206.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:48:45.211231Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:53:21.739 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:53:21.739 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:53:21.740 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184125.211231_206.1 2022-09-03T05:53:26.741 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:26.742 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:26.743 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:53:26.800 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:53:26.801 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:53:26.845 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:26.844+0000 7f7cec81b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:26.846 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:26.845+0000 7f7cec81b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:26.864 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:53:26.864 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:53:26.864 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:53:26.865 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:53:26.865 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:53:26.865 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:53:26.865 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:26.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:26.866 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:53:26.866 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:26.866 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:53:26.867 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:26.867 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:26.867 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:53:26.868 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:26.868 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:53:26.868 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:26.868 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:53:26.869 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:26.869 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:26.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:26.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:26.870 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:53:26.870 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:26.870 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:26.870 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:53:26.871 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:26.871 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:26.871 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:26.871 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:53:26.872 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:26.872 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:26.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:26.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:26.873 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:53:26.873 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:26.873 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:26.873 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:53:26.874 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:26.874 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:26.874 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:26.874 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:48:45.211231Z", 2022-09-03T05:53:26.875 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:26.875 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:26.875 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:26.876 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:26.876 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:53:26.876 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:26.876 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:26.877 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:53:26.877 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:26.877 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:26.877 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:26.878 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:53:26.878 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:26.878 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:26.878 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:26.879 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:53:26.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:26.879 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:53:26.879 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:53:26.880 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:53:26.880 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:26.880 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:53:26Z" 2022-09-03T05:53:26.881 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:53:26.882 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184125.211231_206.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:48:45.211231Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:53:26Z" } 2022-09-03T05:53:26.883 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184125.211231_206.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:48:45.211231Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:53:26.883 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:53:26.884 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:53:26.884 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184125.211231_206.1 2022-09-03T05:53:31.886 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:31.887 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:31.888 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:53:31.940 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:53:31.940 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:53:32.002 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:32.000+0000 7f5d4881b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:32.006 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:32.005+0000 7f5d4881b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:32.029 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:53:32.030 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:53:32.030 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:53:32.031 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:53:32.031 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:53:32.032 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:53:32.032 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:32.032 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:32.032 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:53:32.033 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:32.033 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:53:32.034 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:32.034 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:32.034 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:53:32.035 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:32.035 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:53:32.035 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:32.036 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:53:32.036 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:32.036 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:32.037 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:32.037 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:32.037 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:53:32.037 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:32.038 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:32.038 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:53:32.038 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:32.039 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:32.039 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:32.039 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:53:32.039 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:32.040 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:32.040 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:32.040 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:32.041 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:53:32.041 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:32.041 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:32.041 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.101939_288.1", 2022-09-03T05:53:32.042 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:32.042 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:32.042 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:32.043 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.101939Z", 2022-09-03T05:53:32.043 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:32.043 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:32.044 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:32.044 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:32.045 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:53:32.045 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:32.046 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:32.046 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:53:32.047 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:32.047 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:32.047 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:32.048 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:53:32.048 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:32.049 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:32.049 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:32.049 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:53:32.050 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:32.050 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:53:32.051 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:53:32.051 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:53:32.051 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:32.051 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:53:32Z" 2022-09-03T05:53:32.052 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:53:32.053 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.101939_288.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.101939Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:53:32Z" } 2022-09-03T05:53:32.054 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.101939_288.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.101939Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:53:32.054 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:53:32.055 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.101939_288.1', 3: ''} 2022-09-03T05:53:32.056 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.101939_288.1 2022-09-03T05:53:37.057 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:37.058 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:37.059 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:53:37.119 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:53:37.119 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:53:37.188 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:37.186+0000 7f82768f7600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:37.189 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:37.187+0000 7f82768f7600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:37.203 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:53:37.203 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:53:37.204 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:53:37.204 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:53:37.204 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:53:37.204 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:53:37.205 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:37.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:37.205 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:53:37.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:37.205 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:53:37.206 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:37.206 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:37.206 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:53:37.206 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:37.207 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:53:37.207 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:37.207 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:53:37.208 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:37.208 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:37.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:37.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:37.209 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:53:37.209 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:37.209 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:37.209 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:53:37.210 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:37.210 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:37.210 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:37.210 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:53:37.211 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:37.211 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:37.211 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:37.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:37.211 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:53:37.212 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:37.212 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:37.212 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.506122_312.1", 2022-09-03T05:53:37.212 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:37.213 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:37.213 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:37.213 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.506122Z", 2022-09-03T05:53:37.213 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:37.214 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:37.214 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:37.214 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:37.214 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:53:37.215 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:37.215 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:37.216 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:53:37.216 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:37.216 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:37.217 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:37.217 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:53:37.218 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:37.218 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:37.219 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:37.219 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:53:37.220 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:37.220 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:53:37.221 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:53:37.221 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:53:37.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:37.222 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:53:37Z" 2022-09-03T05:53:37.222 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:53:37.225 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.506122_312.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.506122Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:53:37Z" } 2022-09-03T05:53:37.226 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.506122_312.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.506122Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:53:37.227 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:53:37.227 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.506122_312.1', 3: ''} 2022-09-03T05:53:37.228 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.506122_312.1 2022-09-03T05:53:42.229 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:42.230 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:42.231 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:53:42.299 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:53:42.300 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:53:42.332 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:42.330+0000 7f5638b2f600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:42.333 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:42.331+0000 7f5638b2f600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:42.345 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:53:42.345 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:53:42.345 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:53:42.346 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:53:42.346 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:53:42.346 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:53:42.347 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:42.347 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:42.347 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:53:42.348 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:42.348 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:53:42.348 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:42.349 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:42.349 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:53:42.349 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:42.349 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:53:42.350 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:42.350 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:53:42.350 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:42.351 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:42.351 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:42.352 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:42.352 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:53:42.352 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:42.352 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:42.353 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:53:42.353 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:42.353 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:42.354 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:42.354 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:53:42.354 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:42.354 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:42.355 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:42.355 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:42.356 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:53:42.356 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:42.357 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:42.358 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.515650_313.1", 2022-09-03T05:53:42.359 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:42.359 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:42.359 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:42.360 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.515650Z", 2022-09-03T05:53:42.360 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:42.361 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:42.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:42.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:42.362 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:53:42.363 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:42.363 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:42.363 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:53:42.364 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:42.364 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:42.365 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:42.365 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:53:42.366 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:42.366 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:42.367 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:42.367 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:53:42.368 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:42.368 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:53:42.369 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:53:42.369 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:53:42.370 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:42.370 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:53:42Z" 2022-09-03T05:53:42.371 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:53:42.373 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.515650_313.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.515650Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:53:42Z" } 2022-09-03T05:53:42.374 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.515650_313.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.515650Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:53:42.375 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:53:42.376 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.515650_313.1', 3: ''} 2022-09-03T05:53:42.377 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.515650_313.1 2022-09-03T05:53:47.378 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:47.379 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:47.381 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:53:47.435 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:53:47.436 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:53:47.506 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:47.505+0000 7fcc1683f600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:47.507 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:47.506+0000 7fcc1683f600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:47.526 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:53:47.527 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:53:47.527 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:53:47.527 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:53:47.527 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:53:47.528 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:53:47.528 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:47.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:47.528 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:53:47.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:47.529 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:53:47.529 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:47.529 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:47.530 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:53:47.530 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:47.530 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:53:47.531 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:47.531 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:53:47.531 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:47.531 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:47.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:47.532 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:47.532 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:53:47.532 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:47.533 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:47.533 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:53:47.533 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:47.533 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:47.534 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:47.534 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:53:47.534 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:47.534 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:47.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:47.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:47.535 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:53:47.535 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:47.536 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:47.536 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.116801_289.1", 2022-09-03T05:53:47.536 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:47.536 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:47.536 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:47.537 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.116801Z", 2022-09-03T05:53:47.537 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:47.537 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:47.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:47.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:47.538 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:53:47.538 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:47.538 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:47.539 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:53:47.539 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:47.539 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:47.539 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:47.540 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:53:47.540 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:47.540 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:47.540 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:47.541 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:53:47.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:47.541 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:53:47.541 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:53:47.542 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:53:47.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:47.542 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:53:47Z" 2022-09-03T05:53:47.542 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:53:47.544 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.116801_289.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.116801Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:53:47Z" } 2022-09-03T05:53:47.544 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.116801_289.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.116801Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:53:47.545 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:53:47.545 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.116801_289.1', 3: ''} 2022-09-03T05:53:47.546 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.116801_289.1 2022-09-03T05:53:52.547 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:52.548 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:52.549 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:53:52.607 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:53:52.608 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:53:52.643 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:52.641+0000 7f19185f8600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:52.644 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:52.642+0000 7f19185f8600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:52.659 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:53:52.659 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:53:52.660 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:53:52.660 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:53:52.661 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:53:52.662 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:53:52.662 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:52.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:52.663 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:53:52.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:52.664 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:53:52.664 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:52.665 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:52.665 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:53:52.665 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:52.665 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:53:52.666 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:52.666 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:53:52.666 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:52.667 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:52.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:52.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:52.667 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:53:52.668 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:52.668 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:52.668 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:53:52.669 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:52.669 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:52.669 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:52.670 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:53:52.670 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:52.670 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:52.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:52.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:52.671 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:53:52.671 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:52.671 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:52.672 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.506122_312.1", 2022-09-03T05:53:52.672 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:52.672 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:52.673 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:52.673 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.506122Z", 2022-09-03T05:53:52.673 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:52.673 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:52.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:52.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:52.674 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:53:52.675 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:52.675 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:52.675 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:53:52.675 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:52.676 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:52.676 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:52.676 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:53:52.676 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:52.677 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:52.677 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:52.677 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:53:52.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:52.678 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:53:52.678 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:53:52.678 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:53:52.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:52.679 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:53:52Z" 2022-09-03T05:53:52.679 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:53:52.680 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.506122_312.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.506122Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:53:52Z" } 2022-09-03T05:53:52.681 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.506122_312.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.506122Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:53:52.682 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:53:52.683 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.506122_312.1', 3: ''} 2022-09-03T05:53:52.684 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.506122_312.1 2022-09-03T05:53:57.685 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:57.686 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:53:57.687 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:53:57.737 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:53:57.738 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:53:57.764 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:57.762+0000 7f3c16586600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:57.765 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:53:57.763+0000 7f3c16586600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:53:57.777 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:53:57.778 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:53:57.778 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:53:57.779 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:53:57.779 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:53:57.779 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:53:57.779 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:57.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:57.780 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:53:57.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:57.781 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:53:57.781 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:57.781 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:57.782 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:53:57.782 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:57.783 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:53:57.783 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:57.783 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:53:57.783 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:57.784 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:57.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:57.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:57.785 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:53:57.785 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:57.785 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:57.785 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:53:57.786 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:57.786 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:57.786 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:57.787 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:53:57.787 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:57.787 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:57.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:57.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:57.788 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:53:57.788 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:57.788 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:57.789 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.188127_295.1", 2022-09-03T05:53:57.789 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:57.789 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:57.789 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:57.790 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.188127Z", 2022-09-03T05:53:57.790 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:53:57.790 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:57.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:57.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:53:57.791 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:53:57.791 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:53:57.792 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:53:57.792 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:53:57.792 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:53:57.792 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:53:57.793 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:53:57.793 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:53:57.793 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:53:57.793 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:57.794 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:53:57.794 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:53:57.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:57.794 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:53:57.795 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:53:57.795 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:53:57.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:53:57.795 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:53:57Z" 2022-09-03T05:53:57.796 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:53:57.797 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.188127_295.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.188127Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:53:57Z" } 2022-09-03T05:53:57.798 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.188127_295.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.188127Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:53:57.799 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:53:57.799 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.188127_295.1', 3: ''} 2022-09-03T05:53:57.800 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.188127_295.1 2022-09-03T05:54:02.802 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:02.803 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:02.804 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:54:02.855 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:54:02.856 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:54:02.891 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:54:02.889+0000 7f1653d78600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:02.892 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:54:02.891+0000 7f1653d78600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:02.907 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:54:02.907 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:54:02.908 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:54:02.908 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:54:02.909 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:54:02.909 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:54:02.909 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:54:02.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:54:02.910 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:54:02.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:54:02.911 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:54:02.912 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:54:02.912 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:54:02.912 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:54:02.912 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:54:02.913 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:54:02.913 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:54:02.913 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:54:02.913 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:54:02.914 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:54:02.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:54:02.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:54:02.915 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:54:02.915 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:54:02.916 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:54:02.916 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:54:02.916 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:54:02.917 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:54:02.917 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:54:02.917 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:54:02.917 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:54:02.917 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:54:02.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:54:02.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:54:02.919 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:54:02.919 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:54:02.919 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:54:02.920 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.122931_290.1", 2022-09-03T05:54:02.920 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:54:02.920 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:54:02.921 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:54:02.921 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.122931Z", 2022-09-03T05:54:02.921 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:54:02.922 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:54:02.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:54:02.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:54:02.923 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:54:02.923 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:54:02.923 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:54:02.924 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:54:02.924 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:54:02.924 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:54:02.925 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:54:02.925 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:54:02.926 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:54:02.926 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:54:02.926 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:54:02.926 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:54:02.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:54:02.927 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:54:02.928 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:54:02.928 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:54:02.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:54:02.929 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:54:02Z" 2022-09-03T05:54:02.930 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:54:02.933 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.122931_290.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.122931Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:54:02Z" } 2022-09-03T05:54:02.934 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.122931_290.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.122931Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:54:02.935 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:54:02.936 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.122931_290.1', 3: ''} 2022-09-03T05:54:02.936 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.122931_290.1 2022-09-03T05:54:07.937 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:07.938 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:07.939 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:54:07.990 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:54:07.990 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:54:08.023 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:54:08.022+0000 7f149a14b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:08.024 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:54:08.023+0000 7f149a14b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:08.036 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:54:08.037 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:54:08.037 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:54:08.037 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:54:08.038 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:54:08.038 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:54:08.038 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:54:08.038 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:54:08.039 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:54:08.039 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:54:08.039 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:54:08.039 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:54:08.040 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:54:08.040 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:54:08.040 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:54:08.040 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:54:08.041 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:54:08.041 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:54:08.041 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:54:08.042 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:54:08.042 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:54:08.043 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:54:08.043 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:54:08.043 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:54:08.043 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:54:08.044 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:54:08.044 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:54:08.044 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:54:08.045 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:54:08.045 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:54:08.045 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:54:08.046 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:54:08.046 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:54:08.046 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:54:08.046 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:54:08.047 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:54:08.047 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:54:08.047 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.410645_308.1", 2022-09-03T05:54:08.048 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:54:08.048 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:54:08.048 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:54:08.049 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.410645Z", 2022-09-03T05:54:08.049 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:54:08.049 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:54:08.050 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:54:08.050 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:54:08.051 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:54:08.051 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:54:08.052 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:54:08.052 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:54:08.053 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:54:08.053 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:54:08.054 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:54:08.054 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:54:08.055 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:54:08.055 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:54:08.056 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:54:08.056 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:54:08.057 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:54:08.057 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:54:08.057 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:54:08.058 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:54:08.058 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:54:08.059 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:54:08Z" 2022-09-03T05:54:08.059 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:54:08.062 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.410645_308.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.410645Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:54:08Z" } 2022-09-03T05:54:08.063 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.410645_308.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.410645Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:54:08.064 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:54:08.065 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.410645_308.1', 3: ''} 2022-09-03T05:54:08.066 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.410645_308.1 2022-09-03T05:54:13.068 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:13.069 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:13.070 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:54:13.127 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:54:13.128 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:54:13.171 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:54:13.169+0000 7ff5e8dec600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:13.171 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:54:13.170+0000 7ff5e8dec600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:13.188 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:54:13.188 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:54:13.189 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:54:13.189 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:54:13.189 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:54:13.189 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:54:13.190 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:54:13.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:54:13.190 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:54:13.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:54:13.191 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:54:13.191 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:54:13.191 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:54:13.192 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:54:13.192 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:54:13.193 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:54:13.193 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:54:13.194 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:54:13.194 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:54:13.195 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:54:13.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:54:13.196 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:54:13.196 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:54:13.197 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:54:13.197 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:54:13.198 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:54:13.198 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:54:13.198 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:54:13.199 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:54:13.199 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:54:13.199 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:54:13.199 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:54:13.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:54:13.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:54:13.200 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:54:13.200 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:54:13.201 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:54:13.201 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184380.897369_341.1", 2022-09-03T05:54:13.201 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:54:13.201 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:54:13.202 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:54:13.202 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:53:00.897369Z", 2022-09-03T05:54:13.202 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:54:13.203 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:54:13.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:54:13.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:54:13.204 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:54:13.204 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:54:13.204 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:54:13.204 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:54:13.205 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:54:13.205 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:54:13.205 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:54:13.205 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:54:13.206 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:54:13.206 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:54:13.206 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:54:13.207 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:54:13.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:54:13.207 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:54:13.207 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:54:13.208 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:54:13.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:54:13.208 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:54:13Z" 2022-09-03T05:54:13.209 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:54:13.210 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184380.897369_341.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:53:00.897369Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:54:13Z" } 2022-09-03T05:54:13.211 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184380.897369_341.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:53:00.897369Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:54:13.211 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:54:13.212 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:54:13.212 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:54:13.213 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:54:13.213 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:13.214 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:13.214 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:54:13.271 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:54:13.272 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:54:13.324 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:54:13.323+0000 7f54c0296600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:13.325 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:54:13.324+0000 7f54c0296600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:13.345 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:54:13.346 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:54:13.346 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:54:13.346 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:54:13.346 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:54:13.347 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:54:13.347 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:54:13.347 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:13.347 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:54:13.348 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:13.348 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:54:13.348 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:13.348 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:13.349 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:54:13.349 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:13.349 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:54:13.349 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:13.350 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:54:13.350 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:13.350 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:13.351 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:13.351 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:13.351 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:54:13.351 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:13.351 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:13.352 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:54:13.352 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:13.353 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:13.353 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:13.353 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:54:13.354 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:13.354 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:13.354 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:13.355 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:13.356 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:54:13.356 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:13.356 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:13.357 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:54:13.357 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:13.357 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:13.358 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:13.358 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:48:45.211231Z", 2022-09-03T05:54:13.358 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:54:13.358 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:13.359 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:13.359 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:13.359 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:54:13.360 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:13.360 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:13.360 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:54:13.360 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:13.361 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:13.361 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:13.361 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:54:13.361 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:13.362 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:13.362 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:13.362 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:54:13.362 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:13.363 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:54:13.363 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:54:13.363 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:54:13.363 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:13.364 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:54:13Z" 2022-09-03T05:54:13.364 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:54:13.365 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184125.211231_206.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:48:45.211231Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:54:13Z" } 2022-09-03T05:54:13.366 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184125.211231_206.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:48:45.211231Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:54:13.367 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:54:13.367 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:54:13.368 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184125.211231_206.1 2022-09-03T05:54:18.369 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:18.370 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:18.371 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:54:18.428 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:54:18.428 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:54:18.459 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:54:18.457+0000 7f4deacce600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:18.460 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:54:18.458+0000 7f4deacce600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:18.472 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:54:18.473 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:54:18.473 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:54:18.474 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:54:18.474 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:54:18.474 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:54:18.474 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:54:18.475 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:18.475 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:54:18.475 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:18.475 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:54:18.476 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:18.476 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:18.476 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:54:18.476 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:18.477 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:54:18.477 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:18.478 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:54:18.478 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:18.478 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:18.478 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:18.479 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:18.479 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:54:18.479 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:18.480 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:18.480 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:54:18.480 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:18.481 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:18.481 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:18.481 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:54:18.481 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:18.481 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:18.482 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:18.482 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:18.482 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:54:18.483 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:18.483 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:18.483 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:54:18.483 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:18.484 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:18.484 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:18.484 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:48:45.211231Z", 2022-09-03T05:54:18.484 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:54:18.485 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:18.485 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:18.485 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:18.486 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:54:18.486 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:18.486 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:18.487 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:54:18.487 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:18.487 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:18.487 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:18.488 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:54:18.488 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:18.488 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:18.489 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:18.489 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:54:18.489 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:18.490 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:54:18.490 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:54:18.490 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:54:18.491 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:18.491 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:54:18Z" 2022-09-03T05:54:18.491 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:54:18.492 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184125.211231_206.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:48:45.211231Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:54:18Z" } 2022-09-03T05:54:18.493 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184125.211231_206.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:48:45.211231Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:54:18.494 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:54:18.494 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:54:18.495 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184125.211231_206.1 2022-09-03T05:54:23.496 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:23.497 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:23.498 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:54:23.556 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:54:23.557 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:54:23.597 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:54:23.595+0000 7fce89d97600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:23.598 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:54:23.596+0000 7fce89d97600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:23.613 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:54:23.614 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:54:23.614 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:54:23.615 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:54:23.615 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:54:23.615 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:54:23.616 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:54:23.617 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:23.617 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:54:23.618 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:23.618 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:54:23.618 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:23.619 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:23.619 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:54:23.619 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:23.619 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:54:23.620 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:23.620 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:54:23.620 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:23.621 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:23.621 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:23.621 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:23.621 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:54:23.622 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:23.622 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:23.622 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:54:23.622 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:23.623 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:23.623 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:23.623 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:54:23.623 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:23.624 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:23.624 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:23.624 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:23.625 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:54:23.625 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:23.625 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:23.625 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:54:23.626 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:23.626 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:23.626 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:23.626 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:48:45.211231Z", 2022-09-03T05:54:23.627 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:54:23.627 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:23.627 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:23.628 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:23.628 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:54:23.628 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:23.629 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:23.629 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:54:23.629 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:23.629 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:23.630 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:23.630 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:54:23.630 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:23.630 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:23.631 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:23.631 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:54:23.631 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:23.632 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:54:23.632 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:54:23.632 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:54:23.632 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:23.633 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:54:23Z" 2022-09-03T05:54:23.633 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:54:23.635 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184125.211231_206.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:48:45.211231Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:54:23Z" } 2022-09-03T05:54:23.636 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184125.211231_206.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:48:45.211231Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:54:23.636 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:54:23.637 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:54:23.637 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184125.211231_206.1 2022-09-03T05:54:28.638 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:28.639 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:28.640 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:54:28.698 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:54:28.699 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:54:28.734 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:54:28.732+0000 7fc8579ba600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:28.735 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:54:28.733+0000 7fc8579ba600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:28.749 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:54:28.750 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:54:28.750 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:54:28.750 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:54:28.751 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:54:28.751 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:54:28.751 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:54:28.751 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:28.752 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:54:28.752 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:28.752 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:54:28.752 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:28.753 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:28.753 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:54:28.753 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:28.753 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:54:28.754 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:28.754 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:54:28.754 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:28.754 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:28.755 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:28.755 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:28.755 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:54:28.755 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:28.756 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:28.756 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:54:28.756 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:28.756 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:28.757 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:28.757 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:54:28.757 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:28.757 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:28.758 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:28.758 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:28.759 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:54:28.759 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:28.759 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:28.760 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:54:28.760 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:28.760 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:28.760 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:28.761 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:48:45.211231Z", 2022-09-03T05:54:28.761 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:54:28.761 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:28.761 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:28.762 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:28.762 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:54:28.762 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:28.762 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:28.763 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:54:28.763 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:28.764 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:28.764 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:28.764 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:54:28.764 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:28.765 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:28.765 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:28.765 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:54:28.765 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:28.766 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:54:28.766 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:54:28.766 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:54:28.766 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:28.767 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:54:28Z" 2022-09-03T05:54:28.767 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:54:28.768 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184125.211231_206.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:48:45.211231Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:54:28Z" } 2022-09-03T05:54:28.769 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184125.211231_206.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:48:45.211231Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:54:28.770 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:54:28.770 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:54:28.771 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184125.211231_206.1 2022-09-03T05:54:33.773 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:33.774 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:33.775 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:54:33.832 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:54:33.832 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:54:33.877 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:54:33.875+0000 7f7960f8f600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:33.878 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:54:33.876+0000 7f7960f8f600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:33.902 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:54:33.903 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:54:33.903 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:54:33.904 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:54:33.904 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:54:33.904 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:54:33.905 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:54:33.905 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:33.905 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:54:33.906 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:33.906 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:54:33.907 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:33.908 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:33.908 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:54:33.908 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:33.908 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:54:33.909 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:33.909 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:54:33.909 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:33.909 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:33.910 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:33.910 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:33.910 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:54:33.910 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:33.912 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:33.912 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:54:33.913 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:33.913 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:33.913 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:33.913 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:54:33.914 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:33.914 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:33.914 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:33.914 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:33.915 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:54:33.915 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:33.915 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:33.915 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:54:33.916 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:33.916 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:33.916 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:33.917 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:48:45.211231Z", 2022-09-03T05:54:33.917 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:54:33.917 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:33.917 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:33.918 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:33.918 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:54:33.918 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:33.918 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:33.919 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:54:33.919 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:33.919 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:33.919 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:33.920 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:54:33.920 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:33.920 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:33.920 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:33.921 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:54:33.921 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:33.921 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:54:33.921 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:54:33.922 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:54:33.922 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:33.922 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:54:33Z" 2022-09-03T05:54:33.922 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:54:33.924 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184125.211231_206.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:48:45.211231Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:54:33Z" } 2022-09-03T05:54:33.924 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184125.211231_206.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:48:45.211231Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:54:33.925 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:54:33.925 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:54:33.926 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184125.211231_206.1 2022-09-03T05:54:38.928 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:38.929 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:38.930 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:54:38.994 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:54:38.995 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:54:39.044 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:54:39.042+0000 7f9766094600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:39.045 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:54:39.043+0000 7f9766094600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:39.070 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:54:39.077 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:54:39.078 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:54:39.078 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:54:39.078 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:54:39.078 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:54:39.079 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:54:39.079 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:39.079 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:54:39.079 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:39.080 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:54:39.080 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:39.080 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:39.081 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:54:39.081 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:39.081 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:54:39.081 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:39.082 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:54:39.082 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:39.082 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:39.082 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:39.083 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:39.083 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:54:39.083 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:39.083 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:39.084 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:54:39.084 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:39.084 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:39.084 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:39.085 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:54:39.085 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:39.085 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:39.086 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:39.086 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:39.086 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:54:39.086 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:39.087 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:39.087 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:54:39.087 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:39.087 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:39.088 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:39.088 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:48:45.211231Z", 2022-09-03T05:54:39.088 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:54:39.088 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:39.089 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:39.089 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:39.089 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:54:39.089 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:39.090 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:39.090 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:54:39.090 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:39.091 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:39.091 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:39.091 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:54:39.091 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:39.092 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:39.092 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:39.092 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:54:39.092 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:39.093 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:54:39.093 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:54:39.093 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:54:39.093 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:39.094 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:54:39Z" 2022-09-03T05:54:39.094 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:54:39.095 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184125.211231_206.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:48:45.211231Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:54:39Z" } 2022-09-03T05:54:39.096 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184125.211231_206.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:48:45.211231Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:54:39.096 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:54:39.097 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:54:39.097 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184125.211231_206.1 2022-09-03T05:54:44.099 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:44.100 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:44.101 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:54:44.158 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:54:44.158 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:54:44.193 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:54:44.191+0000 7f74c64ed600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:44.194 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:54:44.193+0000 7f74c64ed600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:44.210 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:54:44.210 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:54:44.211 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:54:44.211 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:54:44.211 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:54:44.212 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:54:44.212 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:54:44.212 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:44.212 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:54:44.213 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:44.213 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:54:44.213 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:44.213 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:44.214 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:54:44.214 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:44.214 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:54:44.215 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:44.215 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:54:44.215 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:44.215 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:44.216 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:44.216 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:44.216 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:54:44.217 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:44.217 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:44.217 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:54:44.217 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:44.218 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:44.218 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:44.218 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:54:44.218 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:44.219 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:44.219 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:44.219 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:44.220 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:54:44.220 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:44.220 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:44.221 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:54:44.221 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:44.221 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:44.221 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:44.222 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:48:45.211231Z", 2022-09-03T05:54:44.222 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:54:44.222 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:44.222 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:44.223 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:44.223 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:54:44.223 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:44.223 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:44.224 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:54:44.224 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:44.224 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:44.224 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:44.225 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:54:44.225 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:44.225 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:44.225 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:44.226 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:54:44.226 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:44.226 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:54:44.226 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:54:44.227 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:54:44.227 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:44.227 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:54:44Z" 2022-09-03T05:54:44.227 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:54:44.229 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184125.211231_206.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:48:45.211231Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:54:44Z" } 2022-09-03T05:54:44.229 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184125.211231_206.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:48:45.211231Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:54:44.230 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:54:44.231 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:54:44.231 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184125.211231_206.1 2022-09-03T05:54:49.233 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:49.234 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:49.234 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:54:49.291 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:54:49.292 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:54:49.343 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:54:49.342+0000 7ffbba080600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:49.344 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:54:49.343+0000 7ffbba080600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:49.368 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:54:49.369 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:54:49.369 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:54:49.369 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:54:49.370 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:54:49.370 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:54:49.370 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:54:49.370 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:49.371 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:54:49.371 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:49.372 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:54:49.372 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:49.372 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:49.373 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:54:49.373 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:49.373 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:54:49.373 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:49.374 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:54:49.374 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:49.374 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:49.375 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:49.375 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:49.375 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:54:49.376 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:49.376 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:49.376 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:54:49.376 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:49.377 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:49.377 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:49.377 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:54:49.377 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:49.378 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:49.378 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:49.378 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:49.378 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:54:49.379 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:49.379 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:49.379 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:54:49.379 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:49.380 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:49.380 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:49.380 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:48:45.211231Z", 2022-09-03T05:54:49.380 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:54:49.381 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:49.381 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:49.381 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:49.382 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:54:49.382 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:49.383 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:49.383 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:54:49.384 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:49.384 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:49.384 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:49.385 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:54:49.385 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:49.386 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:49.386 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:49.387 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:54:49.387 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:49.388 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:54:49.388 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:54:49.389 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:54:49.389 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:49.389 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:54:49Z" 2022-09-03T05:54:49.390 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:54:49.391 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184125.211231_206.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:48:45.211231Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:54:49Z" } 2022-09-03T05:54:49.392 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184125.211231_206.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:48:45.211231Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:54:49.392 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:54:49.393 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:54:49.394 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184125.211231_206.1 2022-09-03T05:54:54.395 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:54.397 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:54.398 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:54:54.455 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:54:54.456 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:54:54.499 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:54:54.498+0000 7f823bd95600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:54.500 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:54:54.499+0000 7f823bd95600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:54.519 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:54:54.520 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:54:54.520 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:54:54.520 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:54:54.521 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:54:54.521 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:54:54.521 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:54:54.521 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:54.522 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:54:54.522 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:54.522 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:54:54.523 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:54.531 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:54.532 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:54:54.532 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:54.533 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:54:54.533 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:54.534 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:54:54.534 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:54.534 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:54.534 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:54.535 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:54.535 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:54:54.535 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:54.535 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:54.536 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:54:54.536 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:54.536 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:54.536 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:54.537 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:54:54.537 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:54.537 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:54.538 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:54.538 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:54.538 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:54:54.538 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:54.539 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:54.539 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:54:54.539 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:54.540 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:54.540 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:54.540 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:48:45.211231Z", 2022-09-03T05:54:54.541 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:54:54.541 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:54.542 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:54.542 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:54.543 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:54:54.543 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:54.543 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:54.544 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:54:54.544 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:54.544 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:54.545 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:54.545 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:54:54.545 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:54.546 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:54.546 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:54.546 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:54:54.546 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:54.547 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:54:54.547 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:54:54.547 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:54:54.548 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:54.548 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:54:54Z" 2022-09-03T05:54:54.548 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:54:54.549 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184125.211231_206.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:48:45.211231Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:54:54Z" } 2022-09-03T05:54:54.550 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184125.211231_206.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:48:45.211231Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:54:54.551 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:54:54.551 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:54:54.552 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184125.211231_206.1 2022-09-03T05:54:59.553 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:59.554 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:54:59.555 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:54:59.611 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:54:59.612 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:54:59.653 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:54:59.652+0000 7ff4392d1600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:59.655 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:54:59.653+0000 7ff4392d1600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:54:59.668 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:54:59.669 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:54:59.669 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:54:59.669 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:54:59.670 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:54:59.670 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:54:59.670 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:54:59.671 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:59.671 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:54:59.671 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:59.671 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:54:59.672 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:59.672 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:59.672 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:54:59.672 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:59.673 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:54:59.673 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:59.673 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:54:59.674 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:59.674 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:59.674 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:59.675 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:59.675 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:54:59.675 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:59.676 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:59.676 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:54:59.676 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:59.677 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:59.677 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:59.677 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:54:59.677 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:59.678 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:59.678 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:59.678 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:59.679 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:54:59.679 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:59.679 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:59.679 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184125.211231_206.1", 2022-09-03T05:54:59.680 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:59.680 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:59.680 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:59.680 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:48:45.211231Z", 2022-09-03T05:54:59.681 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:54:59.681 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:59.681 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:59.681 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:54:59.682 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:54:59.682 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:54:59.682 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:54:59.683 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:54:59.683 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:54:59.684 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:54:59.684 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:54:59.685 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:54:59.685 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:54:59.686 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:59.686 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:54:59.687 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:54:59.687 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:59.687 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:54:59.688 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:54:59.688 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:54:59.689 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:54:59.689 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:54:59Z" 2022-09-03T05:54:59.690 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:54:59.691 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184125.211231_206.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:48:45.211231Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:54:59Z" } 2022-09-03T05:54:59.692 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184125.211231_206.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:48:45.211231Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:54:59.693 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:54:59.693 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184125.211231_206.1', 3: ''} 2022-09-03T05:54:59.694 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184125.211231_206.1 2022-09-03T05:55:04.696 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:04.697 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:04.698 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:55:04.757 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:55:04.758 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:55:04.797 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:55:04.795+0000 7fa19a3e1600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:04.797 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:55:04.796+0000 7fa19a3e1600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:04.819 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:55:04.820 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:55:04.820 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:55:04.821 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:55:04.821 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:55:04.821 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:55:04.822 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:55:04.822 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:55:04.822 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:55:04.823 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:55:04.823 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:55:04.824 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:55:04.824 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:55:04.824 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:55:04.824 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:55:04.825 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:55:04.825 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:55:04.826 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:55:04.826 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:55:04.827 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:55:04.827 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:55:04.827 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:55:04.828 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:55:04.828 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:55:04.829 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:55:04.829 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:55:04.829 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:55:04.830 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:55:04.831 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:55:04.831 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:55:04.831 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:55:04.832 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:55:04.832 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:55:04.832 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:55:04.833 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:55:04.833 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:55:04.833 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:55:04.833 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184378.410645_308.1", 2022-09-03T05:55:04.834 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:55:04.834 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:55:04.834 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:55:04.834 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:52:58.410645Z", 2022-09-03T05:55:04.835 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:55:04.835 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:55:04.835 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:55:04.836 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:55:04.836 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:55:04.836 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:55:04.836 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:55:04.837 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:55:04.837 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:55:04.837 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:55:04.837 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:55:04.838 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:55:04.838 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:55:04.838 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:55:04.839 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:55:04.839 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:55:04.839 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:55:04.840 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:55:04.840 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:55:04.840 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:55:04.840 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:55:04.841 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:55:04Z" 2022-09-03T05:55:04.841 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:55:04.842 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.410645_308.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.410645Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:55:04Z" } 2022-09-03T05:55:04.843 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.410645_308.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.410645Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:55:04.844 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:55:04.844 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.410645_308.1', 3: ''} 2022-09-03T05:55:04.845 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.410645_308.1 2022-09-03T05:55:09.846 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:09.847 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:09.848 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:55:09.904 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:55:09.905 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:55:09.954 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:55:09.952+0000 7f1f6d876600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:09.955 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:55:09.953+0000 7f1f6d876600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:09.976 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:55:09.977 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:55:09.977 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:55:09.978 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:55:09.978 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:55:09.979 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:55:09.979 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:55:09.979 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:55:09.980 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:55:09.980 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:55:09.981 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:55:09.982 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:55:09.982 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:55:09.983 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:55:09.983 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:55:09.984 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:55:09.984 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:55:09.984 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:55:09.985 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:55:09.985 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:55:09.985 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:55:09.985 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:55:09.986 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:55:09.986 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:55:09.987 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:55:09.987 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:55:09.987 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:55:09.987 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:55:09.988 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:55:09.988 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:55:09.988 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:55:09.989 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:55:09.989 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:55:09.989 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:55:09.990 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:55:09.990 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:55:09.990 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:55:09.990 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184380.897369_341.1", 2022-09-03T05:55:09.991 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:55:09.991 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:55:09.992 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:55:09.992 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:53:00.897369Z", 2022-09-03T05:55:09.993 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:55:09.993 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:55:09.993 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:55:09.994 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:55:09.995 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:55:09.995 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:55:09.996 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:55:09.996 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:55:09.997 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:55:09.997 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:55:09.997 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:55:09.998 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:55:09.998 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:55:09.999 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:55:09.999 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:55:09.999 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:55:10.000 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:55:10.000 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:55:10.001 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:55:10.001 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:55:10.001 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:55:10.002 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:55:09Z" 2022-09-03T05:55:10.002 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:55:10.004 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184380.897369_341.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:53:00.897369Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:55:09Z" } 2022-09-03T05:55:10.005 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184380.897369_341.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:53:00.897369Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:55:10.005 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:55:10.006 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:55:10.007 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:55:10.007 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:55:10.008 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:10.008 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:10.009 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:55:10.072 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:55:10.072 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:55:10.116 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:55:10.115+0000 7f3bc9917600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:10.117 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:55:10.116+0000 7f3bc9917600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:10.139 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:55:10.139 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:55:10.140 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:55:10.140 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:55:10.140 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:55:10.141 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:55:10.141 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:55:10.141 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:10.141 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:55:10.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:10.142 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:55:10.142 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:10.143 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:10.143 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:55:10.143 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:10.143 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:55:10.144 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:10.144 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:55:10.144 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:10.145 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:10.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:10.145 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:10.145 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:55:10.146 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:10.146 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:10.146 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:55:10.147 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:10.147 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:10.147 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:10.148 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:55:10.149 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:10.149 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:10.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:10.150 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:10.150 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:55:10.150 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:10.150 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:10.151 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.116801_289.1", 2022-09-03T05:55:10.151 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:10.151 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:10.152 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:10.152 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.116801Z", 2022-09-03T05:55:10.152 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:55:10.153 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:10.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:10.153 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:10.154 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:55:10.154 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:10.155 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:10.155 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:55:10.155 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:10.155 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:10.156 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:10.156 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:55:10.156 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:10.156 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:10.157 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:10.157 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:55:10.157 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:10.158 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:55:10.158 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:55:10.158 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:55:10.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:10.159 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:55:10Z" 2022-09-03T05:55:10.159 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:55:10.160 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.116801_289.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.116801Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:55:10Z" } 2022-09-03T05:55:10.161 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.116801_289.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.116801Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:55:10.161 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:55:10.162 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.116801_289.1', 3: ''} 2022-09-03T05:55:10.162 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.116801_289.1 2022-09-03T05:55:15.164 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:15.165 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:15.165 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:55:15.216 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:55:15.217 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:55:15.242 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:55:15.240+0000 7f694764c600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:15.243 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:55:15.242+0000 7f694764c600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:15.254 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:55:15.254 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:55:15.255 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:55:15.255 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:55:15.255 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:55:15.256 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:55:15.256 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:55:15.256 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:15.256 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:55:15.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:15.257 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:55:15.257 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:15.258 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:15.258 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:55:15.258 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:15.258 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:55:15.259 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:15.259 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:55:15.259 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:15.259 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:15.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:15.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:15.260 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:55:15.261 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:15.261 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:15.261 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:55:15.261 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:15.262 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:15.263 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:15.263 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:55:15.263 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:15.263 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:15.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:15.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:15.264 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:55:15.265 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:15.265 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:15.265 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.192585_296.1", 2022-09-03T05:55:15.265 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:15.266 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:15.266 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:15.266 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.192585Z", 2022-09-03T05:55:15.266 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:55:15.267 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:15.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:15.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:15.268 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:55:15.268 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:15.268 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:15.268 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:55:15.269 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:15.269 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:15.270 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:15.270 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:55:15.270 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:15.270 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:15.271 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:15.271 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:55:15.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:15.271 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:55:15.272 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:55:15.272 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:55:15.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:15.272 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:55:15Z" 2022-09-03T05:55:15.273 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:55:15.274 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.192585_296.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.192585Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:55:15Z" } 2022-09-03T05:55:15.275 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.192585_296.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.192585Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:55:15.275 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:55:15.276 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.192585_296.1', 3: ''} 2022-09-03T05:55:15.276 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.192585_296.1 2022-09-03T05:55:20.277 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:20.279 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:20.280 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:55:20.333 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:55:20.333 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:55:20.361 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:55:20.359+0000 7fce8d914600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:20.362 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:55:20.360+0000 7fce8d914600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:20.376 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:55:20.376 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:55:20.377 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:55:20.377 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:55:20.378 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:55:20.378 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:55:20.379 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:55:20.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:20.380 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:55:20.380 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:20.381 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:55:20.381 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:20.382 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:20.382 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:55:20.383 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:20.383 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:55:20.384 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:20.384 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:55:20.384 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:20.384 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:20.385 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:20.385 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:20.385 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:55:20.386 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:20.386 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:20.386 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:55:20.386 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:20.387 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:20.387 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:20.387 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:55:20.387 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:20.388 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:20.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:20.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:20.388 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:55:20.389 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:20.389 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:20.389 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.420421_310.1", 2022-09-03T05:55:20.389 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:20.390 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:20.390 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:20.390 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.420421Z", 2022-09-03T05:55:20.390 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:55:20.391 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:20.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:20.391 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:20.391 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:55:20.392 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:20.392 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:20.392 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:55:20.392 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:20.393 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:20.393 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:20.393 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:55:20.393 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:20.393 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:20.394 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:20.394 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:55:20.394 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:20.395 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:55:20.395 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:55:20.395 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:55:20.395 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:20.396 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:55:20Z" 2022-09-03T05:55:20.396 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:55:20.396 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.420421_310.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.420421Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:55:20Z" } 2022-09-03T05:55:20.397 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.420421_310.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.420421Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:55:20.398 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:55:20.398 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.420421_310.1', 3: ''} 2022-09-03T05:55:20.399 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.420421_310.1 2022-09-03T05:55:25.401 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:25.403 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:25.404 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:55:25.468 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:55:25.469 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:55:25.502 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:55:25.501+0000 7f65141ce600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:25.508 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:55:25.502+0000 7f65141ce600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:25.524 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:55:25.525 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:55:25.525 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:55:25.525 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:55:25.526 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:55:25.526 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:55:25.527 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:55:25.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:25.527 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:55:25.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:25.528 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:55:25.528 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:25.528 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:25.529 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:55:25.529 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:25.529 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:55:25.530 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:25.530 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:55:25.531 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:25.531 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:25.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:25.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:25.533 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:55:25.533 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:25.533 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:25.534 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:55:25.534 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:25.534 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:25.535 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:25.535 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:55:25.535 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:25.535 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:25.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:25.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:25.536 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:55:25.536 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:25.537 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:25.537 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.410645_308.1", 2022-09-03T05:55:25.537 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:25.537 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:25.537 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:25.538 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.410645Z", 2022-09-03T05:55:25.538 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:55:25.538 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:25.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:25.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:25.540 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:55:25.540 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:25.540 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:25.540 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:55:25.541 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:25.541 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:25.541 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:25.541 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:55:25.542 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:25.542 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:25.542 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:25.543 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:55:25.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:25.543 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:55:25.543 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:55:25.544 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:55:25.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:25.544 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:55:25Z" 2022-09-03T05:55:25.544 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:55:25.546 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.410645_308.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.410645Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:55:25Z" } 2022-09-03T05:55:25.547 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.410645_308.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.410645Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:55:25.547 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:55:25.548 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.410645_308.1', 3: ''} 2022-09-03T05:55:25.548 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.410645_308.1 2022-09-03T05:55:30.549 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:30.550 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:30.551 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:55:30.605 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:55:30.605 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:55:30.633 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:55:30.632+0000 7f7ace451600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:30.634 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:55:30.633+0000 7f7ace451600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:30.646 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:55:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:55:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:55:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:55:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:55:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:55:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:55:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:55:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:55:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:55:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:55:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:55:30.653 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:30.653 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:30.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:30.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:30.654 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:55:30.654 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:30.655 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:30.655 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:55:30.656 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:30.656 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:30.656 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:30.657 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:55:30.657 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:30.658 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:30.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:30.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:30.659 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:55:30.659 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:30.660 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:30.660 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.506122_312.1", 2022-09-03T05:55:30.661 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:30.661 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:30.662 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:30.662 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.506122Z", 2022-09-03T05:55:30.662 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:55:30.663 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:30.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:30.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:30.664 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:55:30.664 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:30.664 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:30.665 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:55:30.665 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:30.665 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:30.666 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:30.666 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:55:30.666 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:30.667 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:30.667 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:30.667 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:55:30.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:30.668 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:55:30.669 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:55:30.669 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:55:30.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:30.670 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:55:30Z" 2022-09-03T05:55:30.670 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:55:30.672 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.506122_312.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.506122Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:55:30Z" } 2022-09-03T05:55:30.673 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.506122_312.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.506122Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:55:30.673 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:55:30.674 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.506122_312.1', 3: ''} 2022-09-03T05:55:30.674 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.506122_312.1 2022-09-03T05:55:35.676 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:35.677 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:35.678 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:55:35.727 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:55:35.727 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:55:35.753 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:55:35.750+0000 7f454cc37600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:35.754 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:55:35.751+0000 7f454cc37600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:35.765 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:55:35.766 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:55:35.766 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:55:35.767 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:55:35.767 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:55:35.767 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:55:35.767 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:55:35.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:35.768 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:55:35.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:35.768 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:55:35.769 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:35.769 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:35.769 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:55:35.772 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:35.772 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:55:35.773 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:35.773 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:55:35.774 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:35.774 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:35.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:35.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:35.775 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:55:35.775 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:35.775 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:35.776 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:55:35.776 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:35.776 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:35.776 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:35.777 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:55:35.777 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:35.777 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:35.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:35.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:35.778 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:55:35.779 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:35.779 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:35.779 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.192585_296.1", 2022-09-03T05:55:35.780 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:35.780 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:35.781 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:35.781 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.192585Z", 2022-09-03T05:55:35.781 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:55:35.782 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:35.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:35.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:35.783 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:55:35.783 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:35.784 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:35.784 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:55:35.785 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:35.785 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:35.786 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:35.786 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:55:35.786 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:35.787 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:35.787 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:35.788 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:55:35.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:35.788 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:55:35.789 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:55:35.789 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:55:35.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:35.790 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:55:35Z" 2022-09-03T05:55:35.790 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:55:35.792 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.192585_296.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.192585Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:55:35Z" } 2022-09-03T05:55:35.792 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.192585_296.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.192585Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:55:35.793 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:55:35.793 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.192585_296.1', 3: ''} 2022-09-03T05:55:35.794 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.192585_296.1 2022-09-03T05:55:40.795 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:40.796 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:40.797 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:55:40.847 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:55:40.848 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:55:40.877 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:55:40.874+0000 7efd6901a600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:40.877 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:55:40.875+0000 7efd6901a600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:40.889 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:55:40.890 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:55:40.891 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:55:40.893 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:55:40.894 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:55:40.894 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:55:40.895 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:55:40.896 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:40.896 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:55:40.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:40.897 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:55:40.897 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:40.897 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:40.897 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:55:40.897 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:40.898 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:55:40.898 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:40.898 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:55:40.899 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:40.899 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:40.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:40.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:40.900 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:55:40.900 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:40.900 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:40.900 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:55:40.900 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:40.901 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:40.901 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:40.901 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:55:40.901 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:40.902 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:40.902 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:40.902 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:40.902 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:55:40.903 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:40.903 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:40.903 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184380.787843_329.1", 2022-09-03T05:55:40.903 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:40.904 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:40.904 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:40.904 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:53:00.787843Z", 2022-09-03T05:55:40.904 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:55:40.905 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:40.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:40.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:40.905 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:55:40.906 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:40.906 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:40.906 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:55:40.906 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:40.907 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:40.907 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:40.907 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:55:40.907 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:40.908 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:40.908 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:40.908 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:55:40.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:40.909 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:55:40.909 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:55:40.909 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:55:40.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:40.909 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:55:40Z" 2022-09-03T05:55:40.910 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:55:40.911 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184380.787843_329.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:53:00.787843Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:55:40Z" } 2022-09-03T05:55:40.912 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184380.787843_329.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:53:00.787843Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:55:40.912 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:55:40.913 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184380.787843_329.1', 3: ''} 2022-09-03T05:55:40.913 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184380.787843_329.1 2022-09-03T05:55:45.915 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:45.916 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:45.916 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:55:45.966 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:55:45.966 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:55:45.993 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:55:45.991+0000 7fa2c6cad600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:45.994 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:55:45.993+0000 7fa2c6cad600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:46.005 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:55:46.006 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:55:46.006 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:55:46.006 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:55:46.007 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:55:46.007 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:55:46.007 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:55:46.007 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:46.008 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:55:46.008 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:46.008 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:55:46.008 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:46.009 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:46.009 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:55:46.009 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:46.009 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:55:46.010 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:46.010 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:55:46.010 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:46.011 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:46.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:46.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:46.011 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:55:46.012 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:46.012 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:46.012 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:55:46.012 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:46.013 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:46.013 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:46.013 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:55:46.013 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:46.014 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:46.014 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:46.014 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:46.014 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:55:46.015 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:46.015 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:46.015 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.506122_312.1", 2022-09-03T05:55:46.016 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:46.016 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:46.016 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:46.016 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.506122Z", 2022-09-03T05:55:46.017 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:55:46.017 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:46.017 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:46.017 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:46.018 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:55:46.018 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:46.018 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:46.018 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:55:46.019 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:46.019 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:46.019 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:46.019 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:55:46.020 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:46.020 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:46.020 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:46.021 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:55:46.021 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:46.021 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:55:46.021 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:55:46.022 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:55:46.022 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:46.022 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:55:45Z" 2022-09-03T05:55:46.023 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:55:46.024 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.506122_312.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.506122Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:55:45Z" } 2022-09-03T05:55:46.025 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.506122_312.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.506122Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:55:46.025 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:55:46.026 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.506122_312.1', 3: ''} 2022-09-03T05:55:46.026 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.506122_312.1 2022-09-03T05:55:51.027 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:51.028 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:51.029 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:55:51.088 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:55:51.089 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:55:51.141 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:55:51.140+0000 7f06e074c600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:51.142 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:55:51.141+0000 7f06e074c600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:51.163 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:55:51.164 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:55:51.165 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:55:51.165 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:55:51.166 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:55:51.166 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:55:51.167 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:55:51.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:51.168 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:55:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:55:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:55:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:55:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:55:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:55:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:55:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:55:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:55:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:51.180 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:51.180 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.506122_312.1", 2022-09-03T05:55:51.180 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:51.181 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:51.181 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:51.181 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.506122Z", 2022-09-03T05:55:51.181 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:55:51.182 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:51.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:51.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:55:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:55:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:55:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:55:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:55:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:55:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:55:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:55:51Z" 2022-09-03T05:55:51.189 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:55:51.191 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.506122_312.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.506122Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:55:51Z" } 2022-09-03T05:55:51.192 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.506122_312.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.506122Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:55:51.193 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:55:51.193 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.506122_312.1', 3: ''} 2022-09-03T05:55:51.194 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.506122_312.1 2022-09-03T05:55:56.196 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:56.197 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:55:56.197 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:55:56.252 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:55:56.252 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:55:56.279 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:55:56.277+0000 7f7e6f962600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:56.280 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:55:56.278+0000 7f7e6f962600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:55:56.291 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:55:56.292 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:55:56.292 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:55:56.292 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:55:56.293 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:55:56.293 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:55:56.293 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:55:56.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:56.295 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:55:56.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:56.296 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:55:56.297 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:56.297 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:56.298 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:55:56.298 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:56.298 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:55:56.299 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:56.299 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:55:56.300 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:56.300 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:56.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:56.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:56.300 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:55:56.301 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:56.301 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:56.301 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:55:56.301 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:56.302 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:56.302 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:56.302 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:55:56.303 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:56.303 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:56.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:56.303 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:56.303 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:55:56.304 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:56.304 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:56.304 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.116801_289.1", 2022-09-03T05:55:56.304 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:56.305 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:56.305 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:56.305 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.116801Z", 2022-09-03T05:55:56.306 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:55:56.306 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:56.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:56.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:55:56.307 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:55:56.307 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:55:56.307 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:55:56.307 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:55:56.308 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:55:56.308 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:55:56.308 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:55:56.308 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:55:56.309 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:55:56.309 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:56.309 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:55:56.309 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:55:56.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:56.310 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:55:56.310 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:55:56.311 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:55:56.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:55:56.311 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:55:56Z" 2022-09-03T05:55:56.312 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:55:56.313 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.116801_289.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.116801Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:55:56Z" } 2022-09-03T05:55:56.314 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.116801_289.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.116801Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:55:56.315 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:55:56.316 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.116801_289.1', 3: ''} 2022-09-03T05:55:56.317 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.116801_289.1 2022-09-03T05:56:01.319 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:01.320 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:01.321 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:56:01.383 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:56:01.383 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:56:01.414 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:01.413+0000 7f3806071600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:01.415 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:01.414+0000 7f3806071600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:01.428 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:56:01.428 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:56:01.429 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:56:01.430 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:56:01.430 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:56:01.431 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:56:01.431 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:01.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:01.432 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:56:01.433 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:01.433 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:56:01.434 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:01.434 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:01.435 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:56:01.436 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:01.436 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:56:01.436 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:01.437 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:56:01.437 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:01.438 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:01.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:01.439 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:01.439 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:56:01.440 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:01.440 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:01.440 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:56:01.441 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:01.441 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:01.441 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:01.441 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:56:01.442 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:01.442 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:01.442 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:01.443 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:01.443 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:56:01.443 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:01.444 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:01.444 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.116801_289.1", 2022-09-03T05:56:01.445 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:01.445 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:01.446 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:01.446 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.116801Z", 2022-09-03T05:56:01.447 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:01.447 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:01.447 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:01.448 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:01.448 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:56:01.448 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:01.449 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:01.449 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:56:01.449 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:01.449 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:01.450 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:01.450 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:56:01.450 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:01.451 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:01.451 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:01.451 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:56:01.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:01.452 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:56:01.452 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:56:01.452 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:56:01.452 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:01.453 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:56:01Z" 2022-09-03T05:56:01.453 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:56:01.454 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.116801_289.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.116801Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:56:01Z" } 2022-09-03T05:56:01.455 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.116801_289.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.116801Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:56:01.456 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:56:01.457 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.116801_289.1', 3: ''} 2022-09-03T05:56:01.458 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.116801_289.1 2022-09-03T05:56:06.460 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:06.461 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:06.462 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:56:06.515 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:56:06.515 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:56:06.558 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:06.557+0000 7f0e9ccd0600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:06.559 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:06.558+0000 7f0e9ccd0600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:06.576 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:56:06.577 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:56:06.578 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:56:06.578 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:56:06.578 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:56:06.579 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:56:06.579 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:06.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:06.580 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:56:06.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:06.581 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:56:06.581 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:06.581 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:06.581 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:56:06.582 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:06.582 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:56:06.582 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:06.583 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:56:06.583 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:06.583 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:06.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:06.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:06.584 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:56:06.584 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:06.585 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:06.585 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:56:06.585 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:06.585 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:06.586 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:06.586 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:56:06.586 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:06.586 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:06.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:06.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:06.587 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:56:06.588 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:06.588 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:06.588 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.506122_312.1", 2022-09-03T05:56:06.588 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:06.589 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:06.589 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:06.589 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.506122Z", 2022-09-03T05:56:06.589 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:06.590 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:06.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:06.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:06.590 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:56:06.591 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:06.591 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:06.591 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:56:06.591 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:06.592 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:06.592 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:06.592 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:56:06.592 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:06.593 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:06.593 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:06.593 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:56:06.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:06.594 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:56:06.594 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:56:06.594 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:56:06.594 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:06.595 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:56:06Z" 2022-09-03T05:56:06.595 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:56:06.596 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.506122_312.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.506122Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:56:06Z" } 2022-09-03T05:56:06.597 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.506122_312.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.506122Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:56:06.597 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:56:06.598 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.506122_312.1', 3: ''} 2022-09-03T05:56:06.598 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.506122_312.1 2022-09-03T05:56:11.600 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:11.601 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:11.602 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:56:11.659 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:56:11.660 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:56:11.705 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:11.703+0000 7f55c9bf2600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:11.706 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:11.704+0000 7f55c9bf2600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:11.721 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:56:11.721 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:56:11.721 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:56:11.722 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:56:11.722 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:56:11.722 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:56:11.722 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:11.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:11.723 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:56:11.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:11.724 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:56:11.724 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:11.725 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:11.725 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:56:11.725 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:11.726 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:56:11.726 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:11.726 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:56:11.726 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:11.727 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:11.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:11.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:11.727 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:56:11.728 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:11.728 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:11.728 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:56:11.729 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:11.729 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:11.729 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:11.729 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:56:11.730 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:11.730 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:11.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:11.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:11.731 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:56:11.731 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:11.731 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:11.731 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.410645_308.1", 2022-09-03T05:56:11.732 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:11.732 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:11.732 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:11.732 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.410645Z", 2022-09-03T05:56:11.733 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:11.733 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:11.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:11.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:11.734 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:56:11.734 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:11.735 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:11.735 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:56:11.735 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:11.736 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:11.736 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:11.736 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:56:11.736 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:11.737 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:11.737 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:11.737 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:56:11.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:11.738 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:56:11.738 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:56:11.738 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:56:11.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:11.739 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:56:11Z" 2022-09-03T05:56:11.739 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:56:11.741 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.410645_308.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.410645Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:56:11Z" } 2022-09-03T05:56:11.741 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.410645_308.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.410645Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:56:11.742 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:56:11.743 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.410645_308.1', 3: ''} 2022-09-03T05:56:11.743 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.410645_308.1 2022-09-03T05:56:16.744 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:16.745 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:16.746 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:56:16.803 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:56:16.803 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:56:16.833 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:16.831+0000 7f5c9786f600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:16.835 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:16.832+0000 7f5c9786f600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:16.852 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:56:16.853 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:56:16.853 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:56:16.855 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:56:16.855 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:56:16.856 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:56:16.856 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:16.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:16.857 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:56:16.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:16.858 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:56:16.859 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:16.859 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:16.860 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:56:16.861 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:16.861 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:56:16.862 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:16.863 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:56:16.863 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:16.864 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:16.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:16.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:16.866 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:56:16.866 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:16.867 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:16.867 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:56:16.867 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:16.868 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:16.869 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:16.869 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:56:16.870 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:16.870 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:16.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:16.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:16.872 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:56:16.873 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:16.873 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:16.874 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.173690_294.1", 2022-09-03T05:56:16.874 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:16.875 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:16.875 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:16.876 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.173690Z", 2022-09-03T05:56:16.876 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:16.876 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:16.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:16.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:16.878 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:56:16.879 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:16.879 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:16.880 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:56:16.880 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:16.881 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:16.881 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:16.882 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:56:16.882 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:16.882 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:16.883 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:16.883 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:56:16.884 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:16.884 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:56:16.885 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:56:16.885 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:56:16.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:16.885 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:56:16Z" 2022-09-03T05:56:16.886 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:56:16.888 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.173690_294.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.173690Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:56:16Z" } 2022-09-03T05:56:16.889 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.173690_294.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.173690Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:56:16.889 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:56:16.890 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.173690_294.1', 3: ''} 2022-09-03T05:56:16.891 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.173690_294.1 2022-09-03T05:56:21.893 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:21.894 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:21.894 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:56:21.952 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:56:21.952 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:56:21.982 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:21.980+0000 7f57a4ccb600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:21.983 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:21.981+0000 7f57a4ccb600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:21.996 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:56:21.996 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:56:21.996 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:56:21.997 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:56:21.997 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:56:21.997 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:56:21.998 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:21.998 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:21.998 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:56:21.999 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:22.000 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:56:22.000 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:22.000 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:22.000 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:56:22.001 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:22.001 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:56:22.001 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:22.002 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:56:22.002 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:22.003 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:22.003 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:22.004 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:22.004 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:56:22.004 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:22.005 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:22.005 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:56:22.006 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:22.006 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:22.007 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:22.007 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:56:22.007 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:22.008 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:22.008 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:22.009 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:22.009 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:56:22.009 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:22.010 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:22.010 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.435468_311.1", 2022-09-03T05:56:22.010 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:22.010 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:22.011 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:22.011 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.435468Z", 2022-09-03T05:56:22.011 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:22.012 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:22.012 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:22.012 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:22.012 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:56:22.013 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:22.013 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:22.013 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:56:22.013 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:22.014 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:22.014 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:22.014 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:56:22.014 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:22.015 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:22.015 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:22.015 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:56:22.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:22.016 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:56:22.016 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:56:22.016 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:56:22.017 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:22.017 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:56:21Z" 2022-09-03T05:56:22.017 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:56:22.019 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.435468_311.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.435468Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:56:21Z" } 2022-09-03T05:56:22.020 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.435468_311.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.435468Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:56:22.021 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:56:22.022 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.435468_311.1', 3: ''} 2022-09-03T05:56:22.023 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.435468_311.1 2022-09-03T05:56:27.025 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:27.026 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:27.027 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:56:27.084 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:56:27.085 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:56:27.115 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:27.112+0000 7f40fe033600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:27.116 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:27.114+0000 7f40fe033600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:27.129 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:56:27.129 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:56:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:56:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:56:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:56:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:56:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:56:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:56:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:56:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:56:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:56:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:27.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:56:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:27.138 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:27.138 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:56:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:56:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:56:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.207016_297.1", 2022-09-03T05:56:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.207016Z", 2022-09-03T05:56:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:56:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:56:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:27.148 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:27.148 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:27.149 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:56:27.149 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:56:27.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:27.151 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:56:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:56:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:56:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:27.153 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:56:27Z" 2022-09-03T05:56:27.153 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:56:27.155 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.207016_297.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.207016Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:56:27Z" } 2022-09-03T05:56:27.156 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.207016_297.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.207016Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:56:27.156 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:56:27.157 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.207016_297.1', 3: ''} 2022-09-03T05:56:27.157 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.207016_297.1 2022-09-03T05:56:32.158 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:32.159 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:32.160 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:56:32.214 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:56:32.215 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:56:32.260 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:32.257+0000 7fcdd3635600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:32.261 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:32.258+0000 7fcdd3635600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:32.278 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:56:32.278 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:56:32.279 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:56:32.279 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:56:32.280 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:56:32.280 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:56:32.281 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:32.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:32.282 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:56:32.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:32.283 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:56:32.283 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:32.284 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:32.284 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:56:32.285 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:32.285 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:56:32.286 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:32.286 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:56:32.287 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:32.287 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:32.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:32.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:32.288 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:56:32.288 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:32.289 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:32.289 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:56:32.290 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:32.290 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:32.290 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:32.291 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:56:32.291 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:32.291 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:32.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:32.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:32.292 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:56:32.292 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:32.293 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:32.293 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.207016_297.1", 2022-09-03T05:56:32.293 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:32.293 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:32.294 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:32.294 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.207016Z", 2022-09-03T05:56:32.294 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:32.294 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:32.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:32.295 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:32.295 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:56:32.295 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:32.296 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:32.296 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:56:32.296 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:32.296 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:32.297 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:32.297 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:56:32.297 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:32.297 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:32.298 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:32.298 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:56:32.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:32.298 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:56:32.299 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:56:32.299 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:56:32.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:32.299 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:56:32Z" 2022-09-03T05:56:32.300 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:56:32.301 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.207016_297.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.207016Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:56:32Z" } 2022-09-03T05:56:32.302 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.207016_297.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.207016Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:56:32.302 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:56:32.303 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.207016_297.1', 3: ''} 2022-09-03T05:56:32.303 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.207016_297.1 2022-09-03T05:56:37.304 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:37.305 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:37.306 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:56:37.360 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:56:37.361 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:56:37.393 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:37.388+0000 7f5e4129b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:37.393 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:37.389+0000 7f5e4129b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:37.403 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:56:37.403 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:56:37.403 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:56:37.404 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:56:37.404 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:56:37.405 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:56:37.405 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:37.405 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:37.406 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:56:37.406 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:37.406 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:56:37.407 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:37.407 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:37.407 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:56:37.407 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:37.408 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:56:37.408 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:37.408 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:56:37.408 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:37.409 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:37.409 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:37.409 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:37.409 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:56:37.410 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:37.410 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:37.410 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:56:37.411 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:37.411 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:37.412 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:37.412 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:56:37.413 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:37.413 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:37.413 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:37.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:37.414 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:56:37.414 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:37.414 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:37.415 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.122931_290.1", 2022-09-03T05:56:37.415 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:37.415 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:37.415 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:37.416 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.122931Z", 2022-09-03T05:56:37.416 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:37.417 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:37.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:37.417 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:37.418 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:56:37.418 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:37.418 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:37.418 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:56:37.419 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:37.419 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:37.420 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:37.420 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:56:37.420 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:37.421 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:37.421 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:37.421 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:56:37.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:37.422 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:56:37.422 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:56:37.422 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:56:37.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:37.423 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:56:37Z" 2022-09-03T05:56:37.424 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:56:37.426 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.122931_290.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.122931Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:56:37Z" } 2022-09-03T05:56:37.427 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.122931_290.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.122931Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:56:37.427 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:56:37.428 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.122931_290.1', 3: ''} 2022-09-03T05:56:37.428 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.122931_290.1 2022-09-03T05:56:42.430 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:42.431 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:42.432 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:56:42.484 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:56:42.485 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:56:42.515 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:42.513+0000 7f6fbda7a600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:42.516 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:42.514+0000 7f6fbda7a600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:42.530 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:56:42.531 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:56:42.531 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:56:42.532 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:56:42.532 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:56:42.532 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:56:42.532 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:42.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:42.533 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:56:42.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:42.533 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:56:42.534 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:42.534 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:42.534 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:56:42.535 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:42.535 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:56:42.535 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:42.536 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:56:42.536 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:42.536 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:42.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:42.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:42.537 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:56:42.538 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:42.538 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:42.539 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:56:42.539 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:42.540 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:42.540 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:42.541 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:56:42.541 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:42.542 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:42.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:42.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:42.543 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:56:42.545 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:42.546 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:42.546 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.506122_312.1", 2022-09-03T05:56:42.547 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:42.547 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:42.548 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:42.549 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.506122Z", 2022-09-03T05:56:42.549 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:42.550 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:42.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:42.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:42.552 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:56:42.552 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:42.553 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:42.553 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:56:42.554 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:42.554 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:42.555 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:42.555 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:56:42.555 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:42.556 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:42.556 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:42.556 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:56:42.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:42.557 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:56:42.557 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:56:42.557 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:56:42.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:42.558 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:56:42Z" 2022-09-03T05:56:42.558 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:56:42.561 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.506122_312.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.506122Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:56:42Z" } 2022-09-03T05:56:42.562 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.506122_312.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.506122Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:56:42.563 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:56:42.564 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.506122_312.1', 3: ''} 2022-09-03T05:56:42.564 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.506122_312.1 2022-09-03T05:56:47.566 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:47.567 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:47.568 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:56:47.632 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:56:47.633 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:56:47.659 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:47.657+0000 7f572b671600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:47.660 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:47.658+0000 7f572b671600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:47.672 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:56:47.673 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:56:47.673 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:56:47.674 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:56:47.675 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:56:47.675 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:56:47.676 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:47.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:47.677 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:56:47.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:47.678 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:56:47.679 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:47.679 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:47.679 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:56:47.681 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:47.682 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:56:47.682 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:47.683 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:56:47.684 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:47.685 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:47.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:47.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:47.686 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:56:47.686 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:47.687 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:47.687 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:56:47.687 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:47.687 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:47.688 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:47.688 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:56:47.688 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:47.688 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:47.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:47.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:47.689 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:56:47.690 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:47.690 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:47.690 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.207016_297.1", 2022-09-03T05:56:47.691 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:47.691 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:47.691 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:47.692 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.207016Z", 2022-09-03T05:56:47.692 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:47.692 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:47.692 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:47.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:47.693 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:56:47.693 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:47.693 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:47.694 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:56:47.694 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:47.694 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:47.694 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:47.695 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:56:47.695 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:47.695 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:47.696 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:47.696 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:56:47.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:47.696 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:56:47.697 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:56:47.697 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:56:47.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:47.697 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:56:47Z" 2022-09-03T05:56:47.698 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:56:47.699 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.207016_297.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.207016Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:56:47Z" } 2022-09-03T05:56:47.700 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.207016_297.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.207016Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:56:47.700 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:56:47.701 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.207016_297.1', 3: ''} 2022-09-03T05:56:47.701 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.207016_297.1 2022-09-03T05:56:52.703 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:52.704 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:52.705 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:56:52.757 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:56:52.757 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:56:52.784 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:52.782+0000 7f1987fde600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:52.785 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:52.783+0000 7f1987fde600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:52.796 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:56:52.797 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:56:52.797 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:56:52.797 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:56:52.798 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:56:52.798 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:56:52.798 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:52.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:52.799 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:56:52.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:52.799 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:56:52.799 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:52.800 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:52.800 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:56:52.800 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:52.801 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:56:52.801 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:52.801 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:56:52.801 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:52.802 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:52.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:52.802 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:52.803 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:56:52.803 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:52.803 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:52.803 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:56:52.804 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:52.804 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:52.804 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:52.805 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:56:52.805 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:52.805 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:52.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:52.806 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:52.806 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:56:52.806 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:52.807 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:52.807 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.420421_310.1", 2022-09-03T05:56:52.807 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:52.807 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:52.808 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:52.808 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.420421Z", 2022-09-03T05:56:52.808 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:52.808 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:52.809 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:52.809 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:52.809 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:56:52.810 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:52.810 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:52.810 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:56:52.810 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:52.811 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:52.811 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:52.811 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:56:52.812 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:52.812 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:52.812 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:52.813 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:56:52.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:52.813 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:56:52.814 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:56:52.814 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:56:52.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:52.814 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:56:52Z" 2022-09-03T05:56:52.815 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:56:52.816 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.420421_310.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.420421Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:56:52Z" } 2022-09-03T05:56:52.817 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.420421_310.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.420421Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:56:52.817 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:56:52.818 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.420421_310.1', 3: ''} 2022-09-03T05:56:52.818 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.420421_310.1 2022-09-03T05:56:57.819 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:57.820 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:56:57.821 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:56:57.877 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:56:57.878 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:56:57.925 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:57.923+0000 7f0e9b7fa600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:57.926 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:56:57.924+0000 7f0e9b7fa600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:56:57.943 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:56:57.944 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:56:57.944 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:56:57.945 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:56:57.945 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:56:57.946 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:56:57.946 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:57.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:57.948 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:56:57.948 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:57.949 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:56:57.949 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:57.950 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:57.950 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:56:57.951 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:57.951 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:56:57.952 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:57.952 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:56:57.953 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:57.953 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:57.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:57.954 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:57.955 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:56:57.955 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:57.956 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:57.956 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:56:57.957 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:57.957 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:57.958 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:57.958 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:56:57.958 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:57.959 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:57.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:57.960 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:57.960 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:56:57.961 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:57.961 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:57.962 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.192585_296.1", 2022-09-03T05:56:57.962 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:57.963 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:57.963 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:57.963 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.192585Z", 2022-09-03T05:56:57.964 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:56:57.964 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:57.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:57.965 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:56:57.966 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:56:57.966 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:56:57.967 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:56:57.967 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:56:57.967 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:56:57.968 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:56:57.968 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:56:57.969 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:56:57.969 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:56:57.969 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:57.969 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:56:57.970 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:56:57.970 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:57.970 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:56:57.971 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:56:57.971 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:56:57.971 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:56:57.971 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:56:57Z" 2022-09-03T05:56:57.972 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:56:57.973 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.192585_296.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.192585Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:56:57Z" } 2022-09-03T05:56:57.974 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.192585_296.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.192585Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:56:57.974 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:56:57.975 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.192585_296.1', 3: ''} 2022-09-03T05:56:57.976 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.192585_296.1 2022-09-03T05:57:02.977 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:02.978 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:02.979 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:57:03.035 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:57:03.035 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:57:03.067 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:03.066+0000 7fbb0fabc600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:03.068 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:03.067+0000 7fbb0fabc600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:03.080 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:57:03.081 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:57:03.081 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:57:03.082 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:57:03.082 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:57:03.083 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:57:03.083 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:03.084 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:03.084 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:57:03.085 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:03.085 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:57:03.086 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:03.086 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:03.087 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:57:03.087 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:03.088 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:57:03.089 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:03.090 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:57:03.090 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:03.090 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:03.091 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:03.091 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:03.091 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:57:03.091 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:03.092 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:03.092 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:57:03.092 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:03.093 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:03.093 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:03.093 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:57:03.093 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:03.094 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:03.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:03.095 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:03.095 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:57:03.095 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:03.095 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:03.096 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.506122_312.1", 2022-09-03T05:57:03.096 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:03.096 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:03.097 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:03.097 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.506122Z", 2022-09-03T05:57:03.097 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:03.097 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:03.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:03.098 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:03.098 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:57:03.098 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:03.099 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:03.099 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:57:03.099 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:03.100 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:03.100 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:03.100 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:57:03.100 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:03.101 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:03.101 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:03.101 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:57:03.101 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:03.102 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:57:03.102 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:57:03.102 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:57:03.102 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:03.103 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:57:03Z" 2022-09-03T05:57:03.103 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:57:03.104 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.506122_312.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.506122Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:57:03Z" } 2022-09-03T05:57:03.105 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.506122_312.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.506122Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:57:03.106 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:57:03.106 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.506122_312.1', 3: ''} 2022-09-03T05:57:03.107 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.506122_312.1 2022-09-03T05:57:08.108 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:08.109 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:08.110 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:57:08.168 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:57:08.169 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:57:08.210 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:08.208+0000 7f7576515600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:08.211 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:08.210+0000 7f7576515600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:08.224 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:57:08.224 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:57:08.225 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:57:08.225 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:57:08.226 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:57:08.226 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:57:08.227 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:08.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:08.228 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:57:08.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:08.229 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:57:08.229 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:08.230 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:08.230 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:57:08.231 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:08.231 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:57:08.232 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:08.232 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:57:08.233 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:08.233 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:08.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:08.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:08.235 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:57:08.235 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:08.235 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:08.236 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:57:08.236 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:08.237 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:08.237 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:08.237 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:57:08.238 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:08.238 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:08.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:08.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:08.239 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:57:08.240 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:08.240 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:08.241 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.207016_297.1", 2022-09-03T05:57:08.241 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:08.241 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:08.241 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:08.242 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.207016Z", 2022-09-03T05:57:08.242 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:08.243 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:08.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:08.243 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:08.243 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:57:08.244 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:08.244 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:08.244 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:57:08.245 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:08.245 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:08.245 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:08.245 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:57:08.246 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:08.246 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:08.247 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:08.247 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:57:08.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:08.248 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:57:08.248 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:57:08.248 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:57:08.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:08.249 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:57:08Z" 2022-09-03T05:57:08.250 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:57:08.251 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.207016_297.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.207016Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:57:08Z" } 2022-09-03T05:57:08.252 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.207016_297.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.207016Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:57:08.253 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:57:08.254 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.207016_297.1', 3: ''} 2022-09-03T05:57:08.255 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.207016_297.1 2022-09-03T05:57:13.258 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:13.259 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:13.260 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:57:13.328 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:57:13.328 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:57:13.355 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:13.353+0000 7fb23c109600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:13.357 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:13.355+0000 7fb23c109600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:13.368 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:57:13.368 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:57:13.368 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:57:13.369 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:57:13.369 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:57:13.369 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:57:13.370 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:13.370 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:13.370 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:57:13.371 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:13.371 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:57:13.371 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:13.371 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:13.372 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:57:13.372 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:13.372 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:57:13.372 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:13.373 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:57:13.373 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:13.373 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:13.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:13.374 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:13.374 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:57:13.374 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:13.375 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:13.375 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:57:13.375 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:13.375 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:13.376 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:13.376 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:57:13.376 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:13.377 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:13.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:13.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:13.378 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:57:13.379 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:13.379 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:13.379 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.410645_308.1", 2022-09-03T05:57:13.379 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:13.380 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:13.380 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:13.380 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.410645Z", 2022-09-03T05:57:13.380 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:13.381 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:13.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:13.381 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:13.381 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:57:13.382 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:13.382 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:13.382 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:57:13.382 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:13.383 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:13.383 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:13.384 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:57:13.384 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:13.385 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:13.385 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:13.386 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:57:13.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:13.386 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:57:13.387 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:57:13.387 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:57:13.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:13.389 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:57:13Z" 2022-09-03T05:57:13.389 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:57:13.391 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.410645_308.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.410645Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:57:13Z" } 2022-09-03T05:57:13.392 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.410645_308.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.410645Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:57:13.393 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:57:13.393 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.410645_308.1', 3: ''} 2022-09-03T05:57:13.394 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.410645_308.1 2022-09-03T05:57:18.395 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:18.396 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:18.397 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:57:18.454 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:57:18.455 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:57:18.488 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:18.485+0000 7fd5ccc1a600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:18.497 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:18.486+0000 7fd5ccc1a600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:18.507 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:57:18.507 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:57:18.508 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:57:18.508 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:57:18.508 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:57:18.508 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:57:18.509 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:18.509 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:18.509 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:57:18.510 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:18.510 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:57:18.510 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:18.510 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:18.511 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:57:18.511 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:18.512 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:57:18.512 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:18.513 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:57:18.513 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:18.514 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:18.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:18.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:18.515 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:57:18.517 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:18.518 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:18.518 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:57:18.518 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:18.519 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:18.519 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:18.520 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:57:18.520 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:18.521 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:18.521 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:18.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:18.523 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:57:18.523 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:18.523 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:18.524 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.192585_296.1", 2022-09-03T05:57:18.524 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:18.525 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:18.525 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:18.526 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.192585Z", 2022-09-03T05:57:18.526 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:18.527 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:18.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:18.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:18.528 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:57:18.529 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:18.529 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:18.530 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:57:18.530 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:18.531 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:18.531 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:18.531 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:57:18.532 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:18.532 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:18.532 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:18.532 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:57:18.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:18.533 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:57:18.533 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:57:18.533 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:57:18.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:18.534 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:57:18Z" 2022-09-03T05:57:18.534 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:57:18.535 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.192585_296.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.192585Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:57:18Z" } 2022-09-03T05:57:18.535 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.192585_296.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.192585Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:57:18.536 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:57:18.536 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.192585_296.1', 3: ''} 2022-09-03T05:57:18.537 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.192585_296.1 2022-09-03T05:57:23.539 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:23.539 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:23.540 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:57:23.598 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:57:23.599 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:57:23.641 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:23.638+0000 7f143f661600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:23.643 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:23.639+0000 7f143f661600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:23.655 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:57:23.656 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:57:23.656 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:57:23.656 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:57:23.656 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:57:23.657 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:57:23.657 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:23.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:23.658 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:57:23.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:23.658 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:57:23.658 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:23.659 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:23.659 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:57:23.659 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:23.659 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:57:23.660 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:23.660 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:57:23.661 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:23.661 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:23.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:23.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:23.662 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:57:23.662 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:23.662 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:23.663 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:57:23.663 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:23.663 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:23.664 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:23.664 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:57:23.664 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:23.664 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:23.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:23.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:23.665 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:57:23.665 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:23.666 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:23.666 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.192585_296.1", 2022-09-03T05:57:23.666 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:23.667 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:23.667 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:23.667 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.192585Z", 2022-09-03T05:57:23.667 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:23.668 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:23.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:23.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:23.669 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:57:23.669 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:23.669 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:23.669 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:57:23.670 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:23.670 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:23.670 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:23.671 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:57:23.671 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:23.671 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:23.672 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:23.672 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:57:23.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:23.672 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:57:23.673 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:57:23.673 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:57:23.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:23.674 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:57:23Z" 2022-09-03T05:57:23.674 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:57:23.677 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.192585_296.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.192585Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:57:23Z" } 2022-09-03T05:57:23.678 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.192585_296.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.192585Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:57:23.679 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:57:23.680 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.192585_296.1', 3: ''} 2022-09-03T05:57:23.680 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.192585_296.1 2022-09-03T05:57:28.681 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:28.682 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:28.683 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:57:28.734 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:57:28.735 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:57:28.761 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:28.759+0000 7fe5fad39600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:28.762 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:28.761+0000 7fe5fad39600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:28.773 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:57:28.774 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:57:28.774 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:57:28.774 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:57:28.775 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:57:28.775 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:57:28.775 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:28.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:28.776 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:57:28.776 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:28.776 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:57:28.776 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:28.777 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:28.777 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:57:28.777 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:28.777 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:57:28.778 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:28.778 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:57:28.778 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:28.779 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:28.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:28.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:28.780 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:57:28.780 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:28.780 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:28.781 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:57:28.781 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:28.781 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:28.782 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:28.782 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:57:28.782 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:28.782 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:28.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:28.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:28.783 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:57:28.783 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:28.784 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:28.784 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.506122_312.1", 2022-09-03T05:57:28.784 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:28.784 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:28.785 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:28.785 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.506122Z", 2022-09-03T05:57:28.785 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:28.785 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:28.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:28.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:28.786 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:57:28.786 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:28.787 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:28.787 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:57:28.787 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:28.787 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:28.788 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:28.788 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:57:28.790 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:28.790 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:28.790 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:28.791 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:57:28.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:28.791 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:57:28.791 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:57:28.792 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:57:28.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:28.792 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:57:28Z" 2022-09-03T05:57:28.793 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:57:28.795 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.506122_312.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.506122Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:57:28Z" } 2022-09-03T05:57:28.796 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.506122_312.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.506122Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:57:28.797 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:57:28.798 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.506122_312.1', 3: ''} 2022-09-03T05:57:28.799 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.506122_312.1 2022-09-03T05:57:33.801 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:33.802 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:33.803 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:57:33.861 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:57:33.862 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:57:33.899 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:33.898+0000 7fbf79e77600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:33.900 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:33.899+0000 7fbf79e77600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:33.918 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:57:33.919 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:57:33.919 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:57:33.919 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:57:33.919 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:57:33.920 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:57:33.920 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:33.920 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:33.920 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:57:33.921 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:33.921 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:57:33.921 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:33.921 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:33.922 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:57:33.922 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:33.922 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:57:33.922 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:33.923 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:57:33.923 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:33.923 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:33.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:33.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:33.924 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:57:33.924 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:33.925 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:33.925 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:57:33.926 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:33.926 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:33.926 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:33.927 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:57:33.927 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:33.927 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:33.928 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:33.928 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:33.928 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:57:33.929 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:33.929 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:33.929 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.571061_316.1", 2022-09-03T05:57:33.929 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:33.930 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:33.930 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:33.930 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.571061Z", 2022-09-03T05:57:33.931 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:33.931 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:33.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:33.931 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:33.932 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:57:33.932 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:33.932 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:33.932 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:57:33.933 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:33.933 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:33.933 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:33.934 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:57:33.934 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:33.934 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:33.934 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:33.935 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:57:33.935 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:33.936 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:57:33.937 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:57:33.937 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:57:33.938 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:33.938 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:57:33Z" 2022-09-03T05:57:33.939 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:57:33.941 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.571061_316.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.571061Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:57:33Z" } 2022-09-03T05:57:33.942 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.571061_316.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.571061Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:57:33.943 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:57:33.944 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.571061_316.1', 3: ''} 2022-09-03T05:57:33.946 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.571061_316.1 2022-09-03T05:57:38.947 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:38.949 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:38.950 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:57:39.018 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:57:39.019 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:57:39.087 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:39.086+0000 7fd0af111600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:39.089 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:39.087+0000 7fd0af111600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:39.100 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:57:39.100 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:57:39.101 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:57:39.101 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:57:39.102 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:57:39.102 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:57:39.103 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:39.103 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:39.104 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:57:39.104 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:39.105 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:57:39.105 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:39.106 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:39.106 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:57:39.107 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:39.107 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:57:39.108 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:39.108 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:57:39.109 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:39.109 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:39.110 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:39.110 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:39.110 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:57:39.111 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:39.111 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:39.112 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:57:39.112 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:39.113 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:39.113 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:39.113 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:57:39.114 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:39.114 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:39.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:39.115 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:39.116 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:57:39.116 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:39.116 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:39.116 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.116801_289.1", 2022-09-03T05:57:39.117 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:39.117 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:39.118 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:39.118 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.116801Z", 2022-09-03T05:57:39.118 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:39.119 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:39.119 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:39.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:39.119 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:57:39.120 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:39.120 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:39.120 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:57:39.120 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:39.121 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:39.121 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:39.121 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:57:39.121 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:39.122 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:39.122 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:39.122 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:57:39.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:39.122 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:57:39.123 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:57:39.123 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:57:39.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:39.123 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:57:39Z" 2022-09-03T05:57:39.124 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:57:39.125 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.116801_289.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.116801Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:57:39Z" } 2022-09-03T05:57:39.126 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.116801_289.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.116801Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:57:39.126 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:57:39.127 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.116801_289.1', 3: ''} 2022-09-03T05:57:39.127 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.116801_289.1 2022-09-03T05:57:44.128 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:44.129 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:44.130 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:57:44.185 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:57:44.186 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:57:44.214 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:44.212+0000 7f9714f6b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:44.214 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:44.213+0000 7f9714f6b600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:44.227 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:57:44.227 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:57:44.228 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:57:44.228 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:57:44.228 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:57:44.229 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:57:44.229 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:44.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:44.229 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:57:44.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:44.230 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:57:44.230 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:44.230 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:57:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:57:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:44.232 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:57:44.232 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:44.232 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:44.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:57:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:57:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:57:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:57:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.173690_294.1", 2022-09-03T05:57:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.173690Z", 2022-09-03T05:57:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:44.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:44.239 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:57:44.240 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:44.240 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:44.240 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:57:44.241 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:44.241 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:44.241 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:44.242 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:57:44.242 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:44.242 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:44.243 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:44.243 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:57:44.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:44.243 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:57:44.244 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:57:44.244 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:57:44.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:44.245 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:57:44Z" 2022-09-03T05:57:44.245 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:57:44.248 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.173690_294.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.173690Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:57:44Z" } 2022-09-03T05:57:44.249 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.173690_294.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.173690Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:57:44.250 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:57:44.250 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.173690_294.1', 3: ''} 2022-09-03T05:57:44.251 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.173690_294.1 2022-09-03T05:57:49.252 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:49.253 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:49.253 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:57:49.314 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:57:49.314 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:57:49.339 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:49.338+0000 7fec496a5600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:49.340 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:49.339+0000 7fec496a5600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:49.354 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:57:49.354 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:57:49.355 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:57:49.355 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:57:49.355 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:57:49.356 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:57:49.356 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:49.356 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:49.356 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:57:49.357 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:49.357 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:57:49.357 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:49.357 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:49.358 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:57:49.358 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:49.358 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:57:49.358 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:49.359 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:57:49.359 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:49.359 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:49.360 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:49.360 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:49.360 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:57:49.361 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:49.361 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:49.362 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:57:49.362 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:49.363 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:49.363 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:49.363 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:57:49.364 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:49.364 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:49.365 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:49.365 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:49.365 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:57:49.365 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:49.366 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:49.366 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.116801_289.1", 2022-09-03T05:57:49.366 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:49.367 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:49.367 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:49.367 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.116801Z", 2022-09-03T05:57:49.367 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:49.368 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:49.368 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:49.368 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:49.368 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:57:49.369 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:49.369 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:49.370 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:57:49.370 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:49.370 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:49.370 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:49.371 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:57:49.371 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:49.371 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:49.371 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:49.372 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:57:49.372 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:49.372 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:57:49.373 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:57:49.373 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:57:49.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:49.373 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:57:49Z" 2022-09-03T05:57:49.374 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:57:49.376 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.116801_289.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.116801Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:57:49Z" } 2022-09-03T05:57:49.376 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.116801_289.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.116801Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:57:49.377 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:57:49.377 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.116801_289.1', 3: ''} 2022-09-03T05:57:49.378 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.116801_289.1 2022-09-03T05:57:54.379 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:54.381 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:54.382 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:57:54.441 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:57:54.441 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:57:54.479 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:54.478+0000 7ffaf82bc600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:54.480 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:54.479+0000 7ffaf82bc600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:54.493 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:57:54.494 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:57:54.494 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:57:54.494 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:57:54.494 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:57:54.495 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:57:54.495 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:54.495 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:54.495 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:57:54.496 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:54.496 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:57:54.496 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:54.496 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:54.497 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:57:54.497 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:54.497 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:57:54.497 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:54.498 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:57:54.498 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:54.498 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:54.498 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:54.499 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:54.499 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:57:54.499 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:54.500 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:54.500 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:57:54.500 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:54.500 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:54.501 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:54.501 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:57:54.501 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:54.501 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:54.502 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:54.502 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:54.502 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:57:54.502 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:54.503 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:54.503 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.122931_290.1", 2022-09-03T05:57:54.503 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:54.503 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:54.504 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:54.504 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.122931Z", 2022-09-03T05:57:54.504 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:54.504 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:54.505 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:54.505 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:54.505 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:57:54.505 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:54.506 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:54.506 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:57:54.506 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:54.506 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:54.506 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:54.507 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:57:54.507 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:54.507 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:54.508 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:54.508 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:57:54.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:54.509 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:57:54.509 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:57:54.509 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:57:54.509 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:54.510 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:57:54Z" 2022-09-03T05:57:54.510 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:57:54.511 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.122931_290.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.122931Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:57:54Z" } 2022-09-03T05:57:54.512 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.122931_290.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.122931Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:57:54.512 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:57:54.513 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.122931_290.1', 3: ''} 2022-09-03T05:57:54.513 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.122931_290.1 2022-09-03T05:57:59.515 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:59.516 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:57:59.517 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:57:59.575 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:57:59.575 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:57:59.618 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:59.616+0000 7f0599cd1600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:59.619 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:57:59.618+0000 7f0599cd1600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:57:59.634 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:57:59.634 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:57:59.635 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:57:59.635 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:57:59.636 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:57:59.637 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:57:59.638 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:59.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:59.638 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:57:59.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:59.639 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:57:59.640 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:59.640 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:59.641 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:57:59.641 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:59.642 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:57:59.642 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:59.643 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:57:59.643 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:59.644 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:59.645 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:59.645 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:59.646 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:57:59.646 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:59.647 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:59.647 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:57:59.648 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:59.648 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:59.649 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:59.649 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:57:59.650 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:59.650 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:59.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:59.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:59.652 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:57:59.652 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:59.652 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:59.652 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.506122_312.1", 2022-09-03T05:57:59.653 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:59.653 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:59.653 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:59.653 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.506122Z", 2022-09-03T05:57:59.654 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:57:59.654 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:59.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:59.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:57:59.655 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:57:59.655 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:57:59.655 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:57:59.656 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:57:59.656 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:57:59.657 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:57:59.657 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:57:59.658 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:57:59.658 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:57:59.659 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:59.659 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:57:59.660 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:57:59.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:59.661 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:57:59.661 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:57:59.662 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:57:59.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:57:59.663 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:57:59Z" 2022-09-03T05:57:59.663 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:57:59.666 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.506122_312.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.506122Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:57:59Z" } 2022-09-03T05:57:59.667 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.506122_312.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.506122Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:57:59.668 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:57:59.668 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.506122_312.1', 3: ''} 2022-09-03T05:57:59.669 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.506122_312.1 2022-09-03T05:58:04.670 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:04.671 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:04.672 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:04.723 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:04.724 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:04.751 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:58:04.750+0000 7f458af36600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:58:04.752 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:58:04.751+0000 7f458af36600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:58:04.764 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:58:04.765 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:58:04.765 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:58:04.767 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:58:04.767 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:58:04.767 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:58:04.767 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:58:04.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:04.768 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:58:04.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:04.769 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:58:04.770 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:04.770 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:04.770 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:58:04.771 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:04.771 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:58:04.771 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:04.772 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:58:04.772 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:58:04.772 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:04.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:04.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:04.773 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:58:04.773 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:04.773 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:04.774 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:58:04.774 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:04.774 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:04.774 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:04.775 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:58:04.775 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:58:04.775 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:04.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:04.776 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:04.776 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:58:04.776 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:04.776 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:04.777 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.410645_308.1", 2022-09-03T05:58:04.777 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:04.777 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:04.778 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:04.778 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.410645Z", 2022-09-03T05:58:04.778 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:58:04.778 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:04.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:04.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:04.779 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:58:04.779 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:04.780 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:04.780 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:58:04.780 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:04.780 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:04.781 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:04.781 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:58:04.781 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:58:04.781 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:04.782 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:04.782 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:58:04.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:04.782 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:58:04.783 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:58:04.783 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:58:04.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:04.783 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:58:04Z" 2022-09-03T05:58:04.784 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:58:04.785 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.410645_308.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.410645Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:58:04Z" } 2022-09-03T05:58:04.786 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.410645_308.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.410645Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:58:04.787 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:58:04.788 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.410645_308.1', 3: ''} 2022-09-03T05:58:04.788 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.410645_308.1 2022-09-03T05:58:09.789 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:09.790 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:09.791 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:09.849 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:09.849 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:09.894 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:58:09.891+0000 7f508d3ac600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:58:09.895 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:58:09.892+0000 7f508d3ac600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:58:09.912 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:58:09.913 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:58:09.914 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:58:09.914 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:58:09.915 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:58:09.915 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:58:09.915 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:58:09.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:09.916 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:58:09.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:09.916 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:58:09.917 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:09.917 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:09.917 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:58:09.917 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:09.918 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:58:09.918 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:09.918 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:58:09.919 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:58:09.919 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:09.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:09.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:09.920 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:58:09.920 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:09.920 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:09.921 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:58:09.921 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:09.921 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:09.921 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:09.922 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:58:09.922 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:58:09.922 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:09.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:09.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:09.923 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:58:09.924 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:09.924 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:09.924 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184378.506122_312.1", 2022-09-03T05:58:09.924 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:09.925 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:09.925 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:09.925 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:52:58.506122Z", 2022-09-03T05:58:09.926 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:58:09.926 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:09.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:09.926 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:09.927 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:58:09.927 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:09.927 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:09.928 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:58:09.928 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:09.928 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:09.929 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:09.929 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:58:09.929 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:58:09.930 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:09.930 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:09.930 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:58:09.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:09.931 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:58:09.931 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:58:09.931 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:58:09.932 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:09.932 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:58:09Z" 2022-09-03T05:58:09.932 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:58:09.934 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184378.506122_312.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:52:58.506122Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:58:09Z" } 2022-09-03T05:58:09.935 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184378.506122_312.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:52:58.506122Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:58:09.935 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:58:09.936 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184378.506122_312.1', 3: ''} 2022-09-03T05:58:09.936 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662184380.897369_341.1 target=1_1662184378.506122_312.1 2022-09-03T05:58:14.938 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:14.939 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:14.940 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:14.998 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:14.999 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:15.030 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:58:15.027+0000 7f4608788600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:58:15.031 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:58:15.028+0000 7f4608788600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:58:15.047 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:58:15.047 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:58:15.048 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:58:15.048 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:58:15.049 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:58:15.049 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:58:15.049 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:58:15.050 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:15.050 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:58:15.051 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:15.051 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:58:15.052 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:15.053 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:15.053 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:58:15.054 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:15.054 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:58:15.054 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:15.055 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:58:15.055 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:58:15.056 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:15.056 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:15.056 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:15.057 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:58:15.057 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:15.058 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:15.058 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:58:15.059 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:15.059 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:15.059 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:15.060 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:58:15.060 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:58:15.061 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:15.061 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:15.061 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:15.062 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:58:15.062 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:15.062 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:15.063 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184380.897369_341.1", 2022-09-03T05:58:15.063 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:15.063 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:15.063 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:15.064 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:53:00.897369Z", 2022-09-03T05:58:15.064 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:58:15.064 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:15.065 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:15.065 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:15.065 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:58:15.065 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:15.066 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:15.066 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:58:15.066 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:15.067 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:15.067 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:15.067 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:58:15.067 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:58:15.068 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:15.068 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:15.068 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:58:15.069 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:15.069 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:58:15.069 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:58:15.069 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:58:15.070 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:15.070 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:58:15Z" 2022-09-03T05:58:15.070 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:58:15.071 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184380.897369_341.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:53:00.897369Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:58:15Z" } 2022-09-03T05:58:15.072 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184380.897369_341.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:53:00.897369Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:58:15.073 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:58:15.074 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184380.897369_341.1', 3: ''} 2022-09-03T05:58:15.075 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:58:15.076 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'disable', '--bucket', 'dsgvcy-56', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:58:15.077 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'disable', '--bucket', 'dsgvcy-56', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:58:15.078 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync disable --bucket dsgvcy-56 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:58:15.138 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:15.138 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:15.264 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'disable', '--bucket', 'dsgvcy-57', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:58:15.265 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'disable', '--bucket', 'dsgvcy-57', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:58:15.267 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync disable --bucket dsgvcy-57 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:58:15.317 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:15.318 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:15.407 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'disable', '--bucket', 'dsgvcy-58', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:58:15.408 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'disable', '--bucket', 'dsgvcy-58', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T05:58:15.408 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync disable --bucket dsgvcy-58 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T05:58:15.464 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:15.464 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:15.575 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['log', 'list', '--rgw-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:15.576 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'log', 'list', '--rgw-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:15.577 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 log list --rgw-zone test-zone1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:15.623 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:15.623 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:15.686 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:58:15.686 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:0", 2022-09-03T05:58:15.687 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000078", 2022-09-03T05:58:15.687 INFO:teuthology.orchestra.run.smithi181.stdout: "meta.history", 2022-09-03T05:58:15.687 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:9", 2022-09-03T05:58:15.687 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:4", 2022-09-03T05:58:15.688 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:1", 2022-09-03T05:58:15.688 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:3", 2022-09-03T05:58:15.688 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.23", 2022-09-03T05:58:15.689 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000070", 2022-09-03T05:58:15.689 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000104", 2022-09-03T05:58:15.689 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:2", 2022-09-03T05:58:15.689 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:4", 2022-09-03T05:58:15.690 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:2", 2022-09-03T05:58:15.690 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:3", 2022-09-03T05:58:15.690 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000026", 2022-09-03T05:58:15.691 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:5", 2022-09-03T05:58:15.691 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:0", 2022-09-03T05:58:15.691 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:0", 2022-09-03T05:58:15.691 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:9", 2022-09-03T05:58:15.692 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:5", 2022-09-03T05:58:15.692 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:5", 2022-09-03T05:58:15.692 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:58:15.692 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.8", 2022-09-03T05:58:15.693 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:4", 2022-09-03T05:58:15.693 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000028", 2022-09-03T05:58:15.693 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:3", 2022-09-03T05:58:15.694 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:9", 2022-09-03T05:58:15.694 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:5", 2022-09-03T05:58:15.694 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:9", 2022-09-03T05:58:15.694 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:5", 2022-09-03T05:58:15.695 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:4", 2022-09-03T05:58:15.695 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:1", 2022-09-03T05:58:15.695 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:6", 2022-09-03T05:58:15.695 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:0", 2022-09-03T05:58:15.696 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:10", 2022-09-03T05:58:15.696 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:3", 2022-09-03T05:58:15.696 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5", 2022-09-03T05:58:15.697 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:2", 2022-09-03T05:58:15.697 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:1", 2022-09-03T05:58:15.697 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000040", 2022-09-03T05:58:15.697 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:4", 2022-09-03T05:58:15.698 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18", 2022-09-03T05:58:15.698 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:6", 2022-09-03T05:58:15.698 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:4", 2022-09-03T05:58:15.698 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.6", 2022-09-03T05:58:15.699 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:9", 2022-09-03T05:58:15.699 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21", 2022-09-03T05:58:15.699 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:9", 2022-09-03T05:58:15.700 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:4", 2022-09-03T05:58:15.700 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000015", 2022-09-03T05:58:15.700 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:3", 2022-09-03T05:58:15.700 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:5", 2022-09-03T05:58:15.701 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:10", 2022-09-03T05:58:15.701 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000069", 2022-09-03T05:58:15.701 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:0", 2022-09-03T05:58:15.702 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000095", 2022-09-03T05:58:15.702 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:8", 2022-09-03T05:58:15.703 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000003", 2022-09-03T05:58:15.703 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000047", 2022-09-03T05:58:15.703 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:0", 2022-09-03T05:58:15.704 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:3", 2022-09-03T05:58:15.704 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:4", 2022-09-03T05:58:15.704 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:3", 2022-09-03T05:58:15.704 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:4", 2022-09-03T05:58:15.705 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:2", 2022-09-03T05:58:15.705 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:8", 2022-09-03T05:58:15.705 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000035", 2022-09-03T05:58:15.705 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:2", 2022-09-03T05:58:15.706 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000037", 2022-09-03T05:58:15.706 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:4", 2022-09-03T05:58:15.706 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:4", 2022-09-03T05:58:15.707 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:3", 2022-09-03T05:58:15.707 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:8", 2022-09-03T05:58:15.707 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:2", 2022-09-03T05:58:15.708 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:6", 2022-09-03T05:58:15.708 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4.0", 2022-09-03T05:58:15.708 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:4", 2022-09-03T05:58:15.708 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:1", 2022-09-03T05:58:15.709 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:58:15.709 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:58:15.709 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:9", 2022-09-03T05:58:15.710 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000024", 2022-09-03T05:58:15.710 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:7", 2022-09-03T05:58:15.710 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:8", 2022-09-03T05:58:15.710 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000000", 2022-09-03T05:58:15.711 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000031", 2022-09-03T05:58:15.711 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:7", 2022-09-03T05:58:15.711 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:3", 2022-09-03T05:58:15.712 INFO:teuthology.orchestra.run.smithi181.stdout: "meta.log.229c309b-f405-4ead-a3a8-baa670c6da51.2", 2022-09-03T05:58:15.712 INFO:teuthology.orchestra.run.smithi181.stdout: "data_log.3", 2022-09-03T05:58:15.712 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:10", 2022-09-03T05:58:15.712 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:10", 2022-09-03T05:58:15.713 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:58:15.713 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:6", 2022-09-03T05:58:15.713 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:1", 2022-09-03T05:58:15.713 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:10", 2022-09-03T05:58:15.714 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:2", 2022-09-03T05:58:15.714 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:10", 2022-09-03T05:58:15.714 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:8", 2022-09-03T05:58:15.714 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:8", 2022-09-03T05:58:15.715 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:7", 2022-09-03T05:58:15.715 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5", 2022-09-03T05:58:15.715 INFO:teuthology.orchestra.run.smithi181.stdout: "data_loggenerations_metadata", 2022-09-03T05:58:15.716 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:7", 2022-09-03T05:58:15.716 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:8", 2022-09-03T05:58:15.716 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:4", 2022-09-03T05:58:15.716 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:5", 2022-09-03T05:58:15.717 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000076", 2022-09-03T05:58:15.717 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:0", 2022-09-03T05:58:15.717 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000113", 2022-09-03T05:58:15.717 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000005", 2022-09-03T05:58:15.718 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:5", 2022-09-03T05:58:15.718 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:4", 2022-09-03T05:58:15.718 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000011", 2022-09-03T05:58:15.719 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000058", 2022-09-03T05:58:15.719 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000052", 2022-09-03T05:58:15.719 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:0", 2022-09-03T05:58:15.719 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:7", 2022-09-03T05:58:15.720 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:6", 2022-09-03T05:58:15.720 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:0", 2022-09-03T05:58:15.721 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:1", 2022-09-03T05:58:15.721 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.21", 2022-09-03T05:58:15.721 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:4", 2022-09-03T05:58:15.722 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:0", 2022-09-03T05:58:15.722 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:5", 2022-09-03T05:58:15.722 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:0", 2022-09-03T05:58:15.723 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:7", 2022-09-03T05:58:15.723 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:58:15.723 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:0", 2022-09-03T05:58:15.723 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:2", 2022-09-03T05:58:15.724 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14", 2022-09-03T05:58:15.724 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37", 2022-09-03T05:58:15.724 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:10", 2022-09-03T05:58:15.725 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:9", 2022-09-03T05:58:15.725 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:9", 2022-09-03T05:58:15.725 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:10", 2022-09-03T05:58:15.725 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000088", 2022-09-03T05:58:15.726 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:10", 2022-09-03T05:58:15.726 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36", 2022-09-03T05:58:15.726 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000080", 2022-09-03T05:58:15.726 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000090", 2022-09-03T05:58:15.727 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:2", 2022-09-03T05:58:15.727 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:5", 2022-09-03T05:58:15.727 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:2", 2022-09-03T05:58:15.728 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000110", 2022-09-03T05:58:15.728 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:0", 2022-09-03T05:58:15.728 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:1", 2022-09-03T05:58:15.728 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:1", 2022-09-03T05:58:15.729 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000096", 2022-09-03T05:58:15.729 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:1", 2022-09-03T05:58:15.729 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:7", 2022-09-03T05:58:15.730 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:3", 2022-09-03T05:58:15.730 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000087", 2022-09-03T05:58:15.730 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:7", 2022-09-03T05:58:15.730 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:10", 2022-09-03T05:58:15.731 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000008", 2022-09-03T05:58:15.731 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:7", 2022-09-03T05:58:15.731 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:2", 2022-09-03T05:58:15.732 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:2", 2022-09-03T05:58:15.732 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:7", 2022-09-03T05:58:15.732 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:10", 2022-09-03T05:58:15.732 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:8", 2022-09-03T05:58:15.733 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:9", 2022-09-03T05:58:15.733 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000006", 2022-09-03T05:58:15.733 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:6", 2022-09-03T05:58:15.733 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000029", 2022-09-03T05:58:15.734 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:4", 2022-09-03T05:58:15.734 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:8", 2022-09-03T05:58:15.734 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:0", 2022-09-03T05:58:15.735 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:7", 2022-09-03T05:58:15.735 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:9", 2022-09-03T05:58:15.735 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:1", 2022-09-03T05:58:15.735 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:6", 2022-09-03T05:58:15.736 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:9", 2022-09-03T05:58:15.736 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000089", 2022-09-03T05:58:15.736 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000022", 2022-09-03T05:58:15.736 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4", 2022-09-03T05:58:15.737 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:3", 2022-09-03T05:58:15.737 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:4", 2022-09-03T05:58:15.737 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:10", 2022-09-03T05:58:15.738 INFO:teuthology.orchestra.run.smithi181.stdout: "bilog.trim", 2022-09-03T05:58:15.738 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:6", 2022-09-03T05:58:15.738 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000016", 2022-09-03T05:58:15.738 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:5", 2022-09-03T05:58:15.739 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:10", 2022-09-03T05:58:15.739 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:8", 2022-09-03T05:58:15.739 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000041", 2022-09-03T05:58:15.739 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:9", 2022-09-03T05:58:15.740 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:10", 2022-09-03T05:58:15.740 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:5", 2022-09-03T05:58:15.740 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:9", 2022-09-03T05:58:15.741 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:6", 2022-09-03T05:58:15.741 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:9", 2022-09-03T05:58:15.741 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000018", 2022-09-03T05:58:15.741 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:3", 2022-09-03T05:58:15.742 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:3", 2022-09-03T05:58:15.742 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000092", 2022-09-03T05:58:15.742 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:1", 2022-09-03T05:58:15.743 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37", 2022-09-03T05:58:15.743 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:8", 2022-09-03T05:58:15.743 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:10", 2022-09-03T05:58:15.743 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.4", 2022-09-03T05:58:15.744 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:9", 2022-09-03T05:58:15.744 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:5", 2022-09-03T05:58:15.744 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:4", 2022-09-03T05:58:15.744 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:7", 2022-09-03T05:58:15.745 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:10", 2022-09-03T05:58:15.745 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:8", 2022-09-03T05:58:15.745 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:2", 2022-09-03T05:58:15.745 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000014", 2022-09-03T05:58:15.746 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:6", 2022-09-03T05:58:15.746 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:8", 2022-09-03T05:58:15.746 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:0", 2022-09-03T05:58:15.747 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:7", 2022-09-03T05:58:15.747 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3", 2022-09-03T05:58:15.747 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:7", 2022-09-03T05:58:15.747 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:4", 2022-09-03T05:58:15.748 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:0", 2022-09-03T05:58:15.748 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000112", 2022-09-03T05:58:15.748 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000007", 2022-09-03T05:58:15.749 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000021", 2022-09-03T05:58:15.749 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.16", 2022-09-03T05:58:15.749 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000064", 2022-09-03T05:58:15.749 INFO:teuthology.orchestra.run.smithi181.stdout: "data_log.1.i3qc9uMKdTf.0", 2022-09-03T05:58:15.750 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:2", 2022-09-03T05:58:15.750 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:5", 2022-09-03T05:58:15.750 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000071", 2022-09-03T05:58:15.751 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4.1", 2022-09-03T05:58:15.751 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.e40d88ba-d542-483a-afa2-5de687affd14.2.retry", 2022-09-03T05:58:15.751 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:3", 2022-09-03T05:58:15.753 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000074", 2022-09-03T05:58:15.754 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:9", 2022-09-03T05:58:15.755 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:5", 2022-09-03T05:58:15.755 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12", 2022-09-03T05:58:15.756 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10", 2022-09-03T05:58:15.756 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:0", 2022-09-03T05:58:15.756 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:10", 2022-09-03T05:58:15.757 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:4", 2022-09-03T05:58:15.757 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:5", 2022-09-03T05:58:15.758 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:3", 2022-09-03T05:58:15.758 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:7", 2022-09-03T05:58:15.759 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:6", 2022-09-03T05:58:15.759 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:10", 2022-09-03T05:58:15.760 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:5", 2022-09-03T05:58:15.760 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000081", 2022-09-03T05:58:15.761 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000009", 2022-09-03T05:58:15.761 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:2", 2022-09-03T05:58:15.761 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000121", 2022-09-03T05:58:15.762 INFO:teuthology.orchestra.run.smithi181.stdout: "data_log.1", 2022-09-03T05:58:15.762 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:6", 2022-09-03T05:58:15.762 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:7", 2022-09-03T05:58:15.763 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:5", 2022-09-03T05:58:15.763 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:9", 2022-09-03T05:58:15.763 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:9", 2022-09-03T05:58:15.763 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.17", 2022-09-03T05:58:15.764 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000125", 2022-09-03T05:58:15.766 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:4", 2022-09-03T05:58:15.767 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000082", 2022-09-03T05:58:15.767 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:9", 2022-09-03T05:58:15.768 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:6", 2022-09-03T05:58:15.768 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:2", 2022-09-03T05:58:15.769 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:8", 2022-09-03T05:58:15.769 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:10", 2022-09-03T05:58:15.770 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:9", 2022-09-03T05:58:15.770 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:4", 2022-09-03T05:58:15.771 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:10", 2022-09-03T05:58:15.771 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:1", 2022-09-03T05:58:15.772 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000105", 2022-09-03T05:58:15.773 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000059", 2022-09-03T05:58:15.773 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.e40d88ba-d542-483a-afa2-5de687affd14.2", 2022-09-03T05:58:15.774 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:6", 2022-09-03T05:58:15.774 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:3", 2022-09-03T05:58:15.775 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:0", 2022-09-03T05:58:15.775 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:7", 2022-09-03T05:58:15.776 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:8", 2022-09-03T05:58:15.776 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:9", 2022-09-03T05:58:15.777 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000077", 2022-09-03T05:58:15.777 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:9", 2022-09-03T05:58:15.779 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000032", 2022-09-03T05:58:15.779 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:0", 2022-09-03T05:58:15.780 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:7", 2022-09-03T05:58:15.780 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:6", 2022-09-03T05:58:15.781 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:7", 2022-09-03T05:58:15.781 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:9", 2022-09-03T05:58:15.782 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:3", 2022-09-03T05:58:15.782 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.24", 2022-09-03T05:58:15.782 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:1", 2022-09-03T05:58:15.783 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:0", 2022-09-03T05:58:15.783 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:10", 2022-09-03T05:58:15.784 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:4", 2022-09-03T05:58:15.784 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:5", 2022-09-03T05:58:15.785 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:7", 2022-09-03T05:58:15.785 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000053", 2022-09-03T05:58:15.786 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:6", 2022-09-03T05:58:15.786 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000091", 2022-09-03T05:58:15.787 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:1", 2022-09-03T05:58:15.787 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:4", 2022-09-03T05:58:15.788 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:8", 2022-09-03T05:58:15.788 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:3", 2022-09-03T05:58:15.789 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000065", 2022-09-03T05:58:15.789 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:0", 2022-09-03T05:58:15.790 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:8", 2022-09-03T05:58:15.790 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:6", 2022-09-03T05:58:15.791 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:3", 2022-09-03T05:58:15.791 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000083", 2022-09-03T05:58:15.792 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000010", 2022-09-03T05:58:15.792 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:0", 2022-09-03T05:58:15.793 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:1", 2022-09-03T05:58:15.793 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:5", 2022-09-03T05:58:15.794 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:3", 2022-09-03T05:58:15.794 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:10", 2022-09-03T05:58:15.795 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:8", 2022-09-03T05:58:15.795 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.7", 2022-09-03T05:58:15.796 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23", 2022-09-03T05:58:15.796 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.3", 2022-09-03T05:58:15.797 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:7", 2022-09-03T05:58:15.798 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:1", 2022-09-03T05:58:15.798 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:7", 2022-09-03T05:58:15.799 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:5", 2022-09-03T05:58:15.799 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:3", 2022-09-03T05:58:15.800 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:3", 2022-09-03T05:58:15.800 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:9", 2022-09-03T05:58:15.801 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:6", 2022-09-03T05:58:15.801 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:2", 2022-09-03T05:58:15.802 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:4", 2022-09-03T05:58:15.802 INFO:teuthology.orchestra.run.smithi181.stdout: "mdlog.sync-status.shard.2", 2022-09-03T05:58:15.803 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:2", 2022-09-03T05:58:15.803 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:7", 2022-09-03T05:58:15.804 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:1", 2022-09-03T05:58:15.804 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000045", 2022-09-03T05:58:15.805 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:5", 2022-09-03T05:58:15.805 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000002", 2022-09-03T05:58:15.806 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.12", 2022-09-03T05:58:15.806 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:10", 2022-09-03T05:58:15.807 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:8", 2022-09-03T05:58:15.807 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:9", 2022-09-03T05:58:15.808 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14", 2022-09-03T05:58:15.808 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000116", 2022-09-03T05:58:15.809 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:5", 2022-09-03T05:58:15.810 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:5", 2022-09-03T05:58:15.811 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:0", 2022-09-03T05:58:15.811 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:2", 2022-09-03T05:58:15.812 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000034", 2022-09-03T05:58:15.812 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:5", 2022-09-03T05:58:15.813 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:3", 2022-09-03T05:58:15.813 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:8", 2022-09-03T05:58:15.814 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000101", 2022-09-03T05:58:15.814 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.13", 2022-09-03T05:58:15.814 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36", 2022-09-03T05:58:15.815 INFO:teuthology.orchestra.run.smithi181.stdout: "mdlog.sync-status.shard.1", 2022-09-03T05:58:15.815 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:1", 2022-09-03T05:58:15.815 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:8", 2022-09-03T05:58:15.815 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:4", 2022-09-03T05:58:15.816 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000079", 2022-09-03T05:58:15.816 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000049", 2022-09-03T05:58:15.817 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:4", 2022-09-03T05:58:15.817 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000117", 2022-09-03T05:58:15.817 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:10", 2022-09-03T05:58:15.818 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:1", 2022-09-03T05:58:15.818 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000044", 2022-09-03T05:58:15.818 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:6", 2022-09-03T05:58:15.818 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:2", 2022-09-03T05:58:15.819 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:8", 2022-09-03T05:58:15.819 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:9", 2022-09-03T05:58:15.819 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:6", 2022-09-03T05:58:15.820 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:6", 2022-09-03T05:58:15.820 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:6", 2022-09-03T05:58:15.820 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:9", 2022-09-03T05:58:15.820 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.14", 2022-09-03T05:58:15.821 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:3", 2022-09-03T05:58:15.821 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:9", 2022-09-03T05:58:15.821 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000066", 2022-09-03T05:58:15.822 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:10", 2022-09-03T05:58:15.823 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:1", 2022-09-03T05:58:15.823 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.e40d88ba-d542-483a-afa2-5de687affd14.1", 2022-09-03T05:58:15.823 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:9", 2022-09-03T05:58:15.824 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:10", 2022-09-03T05:58:15.824 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000068", 2022-09-03T05:58:15.824 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:2", 2022-09-03T05:58:15.824 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000085", 2022-09-03T05:58:15.825 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:1", 2022-09-03T05:58:15.825 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:3", 2022-09-03T05:58:15.825 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:10", 2022-09-03T05:58:15.826 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:7", 2022-09-03T05:58:15.826 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:1", 2022-09-03T05:58:15.826 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:10", 2022-09-03T05:58:15.826 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:7", 2022-09-03T05:58:15.827 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:5", 2022-09-03T05:58:15.827 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000073", 2022-09-03T05:58:15.828 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:6", 2022-09-03T05:58:15.828 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:1", 2022-09-03T05:58:15.829 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:5", 2022-09-03T05:58:15.829 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000038", 2022-09-03T05:58:15.830 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:58:15.830 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:7", 2022-09-03T05:58:15.831 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:7", 2022-09-03T05:58:15.831 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.e40d88ba-d542-483a-afa2-5de687affd14.0", 2022-09-03T05:58:15.832 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:7", 2022-09-03T05:58:15.832 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:5", 2022-09-03T05:58:15.833 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.10", 2022-09-03T05:58:15.833 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:5", 2022-09-03T05:58:15.834 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000118", 2022-09-03T05:58:15.834 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:8", 2022-09-03T05:58:15.835 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1", 2022-09-03T05:58:15.835 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:8", 2022-09-03T05:58:15.836 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.e40d88ba-d542-483a-afa2-5de687affd14.3", 2022-09-03T05:58:15.836 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000036", 2022-09-03T05:58:15.836 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:3", 2022-09-03T05:58:15.837 INFO:teuthology.orchestra.run.smithi181.stdout: "data_log.3.mQeqf0PWo3E.0", 2022-09-03T05:58:15.837 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:2", 2022-09-03T05:58:15.838 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:8", 2022-09-03T05:58:15.838 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:5", 2022-09-03T05:58:15.839 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:8", 2022-09-03T05:58:15.839 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000103", 2022-09-03T05:58:15.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:10", 2022-09-03T05:58:15.840 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:2", 2022-09-03T05:58:15.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:7", 2022-09-03T05:58:15.841 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:10", 2022-09-03T05:58:15.842 INFO:teuthology.orchestra.run.smithi181.stdout: "mdlog.sync-status.shard.3", 2022-09-03T05:58:15.842 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1", 2022-09-03T05:58:15.843 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000119", 2022-09-03T05:58:15.843 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:7", 2022-09-03T05:58:15.844 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:3", 2022-09-03T05:58:15.844 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000098", 2022-09-03T05:58:15.845 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:4", 2022-09-03T05:58:15.845 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4.2", 2022-09-03T05:58:15.846 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:6", 2022-09-03T05:58:15.847 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:9", 2022-09-03T05:58:15.847 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:0", 2022-09-03T05:58:15.848 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:4", 2022-09-03T05:58:15.849 INFO:teuthology.orchestra.run.smithi181.stdout: "mdlog.sync-status.shard.0", 2022-09-03T05:58:15.849 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:3", 2022-09-03T05:58:15.850 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:8", 2022-09-03T05:58:15.851 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:5", 2022-09-03T05:58:15.851 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000027", 2022-09-03T05:58:15.852 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4.2.retry", 2022-09-03T05:58:15.852 INFO:teuthology.orchestra.run.smithi181.stdout: "mdlog.sync-status", 2022-09-03T05:58:15.853 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000019", 2022-09-03T05:58:15.853 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:0", 2022-09-03T05:58:15.853 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:2", 2022-09-03T05:58:15.854 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000039", 2022-09-03T05:58:15.854 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:4", 2022-09-03T05:58:15.855 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:7", 2022-09-03T05:58:15.855 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:0", 2022-09-03T05:58:15.856 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15", 2022-09-03T05:58:15.857 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11", 2022-09-03T05:58:15.857 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:9", 2022-09-03T05:58:15.858 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:6", 2022-09-03T05:58:15.858 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:8", 2022-09-03T05:58:15.859 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:5", 2022-09-03T05:58:15.859 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000100", 2022-09-03T05:58:15.859 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:0", 2022-09-03T05:58:15.860 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:2", 2022-09-03T05:58:15.860 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12", 2022-09-03T05:58:15.860 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11", 2022-09-03T05:58:15.861 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:9", 2022-09-03T05:58:15.861 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:7", 2022-09-03T05:58:15.861 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16", 2022-09-03T05:58:15.862 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:2", 2022-09-03T05:58:15.862 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:7", 2022-09-03T05:58:15.862 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:10", 2022-09-03T05:58:15.863 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:1", 2022-09-03T05:58:15.863 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000093", 2022-09-03T05:58:15.863 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:4", 2022-09-03T05:58:15.863 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:6", 2022-09-03T05:58:15.864 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:5", 2022-09-03T05:58:15.864 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:0", 2022-09-03T05:58:15.865 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:1", 2022-09-03T05:58:15.866 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:10", 2022-09-03T05:58:15.866 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:2", 2022-09-03T05:58:15.867 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000004", 2022-09-03T05:58:15.867 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000063", 2022-09-03T05:58:15.868 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:1", 2022-09-03T05:58:15.868 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000122", 2022-09-03T05:58:15.869 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:10", 2022-09-03T05:58:15.869 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:1", 2022-09-03T05:58:15.870 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:10", 2022-09-03T05:58:15.870 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000057", 2022-09-03T05:58:15.871 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000054", 2022-09-03T05:58:15.871 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4", 2022-09-03T05:58:15.872 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000114", 2022-09-03T05:58:15.872 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:4", 2022-09-03T05:58:15.873 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:2", 2022-09-03T05:58:15.874 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.2", 2022-09-03T05:58:15.874 INFO:teuthology.orchestra.run.smithi181.stdout: "data_log.0", 2022-09-03T05:58:15.875 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:5", 2022-09-03T05:58:15.875 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:5", 2022-09-03T05:58:15.876 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:10", 2022-09-03T05:58:15.876 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:8", 2022-09-03T05:58:15.877 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000012", 2022-09-03T05:58:15.877 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:3", 2022-09-03T05:58:15.878 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:8", 2022-09-03T05:58:15.879 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:8", 2022-09-03T05:58:15.879 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:3", 2022-09-03T05:58:15.880 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:10", 2022-09-03T05:58:15.880 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22", 2022-09-03T05:58:15.881 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:1", 2022-09-03T05:58:15.881 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:6", 2022-09-03T05:58:15.882 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:58:15.882 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000084", 2022-09-03T05:58:15.883 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000043", 2022-09-03T05:58:15.883 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:6", 2022-09-03T05:58:15.884 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:5", 2022-09-03T05:58:15.884 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:9", 2022-09-03T05:58:15.884 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.11", 2022-09-03T05:58:15.885 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4.1.retry", 2022-09-03T05:58:15.885 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:4", 2022-09-03T05:58:15.886 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:4", 2022-09-03T05:58:15.886 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:2", 2022-09-03T05:58:15.887 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:3", 2022-09-03T05:58:15.887 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:1", 2022-09-03T05:58:15.888 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:6", 2022-09-03T05:58:15.888 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:6", 2022-09-03T05:58:15.889 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000111", 2022-09-03T05:58:15.889 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:9", 2022-09-03T05:58:15.890 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:9", 2022-09-03T05:58:15.890 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000048", 2022-09-03T05:58:15.891 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:4", 2022-09-03T05:58:15.891 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:10", 2022-09-03T05:58:15.892 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000020", 2022-09-03T05:58:15.892 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:10", 2022-09-03T05:58:15.893 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:2", 2022-09-03T05:58:15.893 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:2", 2022-09-03T05:58:15.894 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:2", 2022-09-03T05:58:15.894 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:9", 2022-09-03T05:58:15.895 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:6", 2022-09-03T05:58:15.895 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:0", 2022-09-03T05:58:15.896 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000099", 2022-09-03T05:58:15.896 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000056", 2022-09-03T05:58:15.897 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:0", 2022-09-03T05:58:15.897 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:7", 2022-09-03T05:58:15.898 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.19", 2022-09-03T05:58:15.898 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:4", 2022-09-03T05:58:15.899 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:0", 2022-09-03T05:58:15.899 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:0", 2022-09-03T05:58:15.899 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:3", 2022-09-03T05:58:15.900 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:1", 2022-09-03T05:58:15.901 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17", 2022-09-03T05:58:15.901 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:0", 2022-09-03T05:58:15.902 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000072", 2022-09-03T05:58:15.902 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:6", 2022-09-03T05:58:15.903 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:0", 2022-09-03T05:58:15.903 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:7", 2022-09-03T05:58:15.904 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:6", 2022-09-03T05:58:15.905 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000062", 2022-09-03T05:58:15.905 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3", 2022-09-03T05:58:15.906 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000115", 2022-09-03T05:58:15.906 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:4", 2022-09-03T05:58:15.906 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.1", 2022-09-03T05:58:15.907 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22", 2022-09-03T05:58:15.907 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000033", 2022-09-03T05:58:15.907 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:6", 2022-09-03T05:58:15.907 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:10", 2022-09-03T05:58:15.908 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:58:15.908 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:10", 2022-09-03T05:58:15.908 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:6", 2022-09-03T05:58:15.908 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:1", 2022-09-03T05:58:15.909 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15", 2022-09-03T05:58:15.909 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:1", 2022-09-03T05:58:15.910 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:1", 2022-09-03T05:58:15.910 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:3", 2022-09-03T05:58:15.910 INFO:teuthology.orchestra.run.smithi181.stdout: "data_log.0.gbahLsFwZkV.0", 2022-09-03T05:58:15.910 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:2", 2022-09-03T05:58:15.911 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:8", 2022-09-03T05:58:15.911 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:5", 2022-09-03T05:58:15.911 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:1", 2022-09-03T05:58:15.911 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000025", 2022-09-03T05:58:15.912 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:8", 2022-09-03T05:58:15.912 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:5", 2022-09-03T05:58:15.912 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:3", 2022-09-03T05:58:15.913 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000123", 2022-09-03T05:58:15.913 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:1", 2022-09-03T05:58:15.914 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:6", 2022-09-03T05:58:15.915 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:3", 2022-09-03T05:58:15.915 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:7", 2022-09-03T05:58:15.915 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:7", 2022-09-03T05:58:15.916 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:1", 2022-09-03T05:58:15.916 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16", 2022-09-03T05:58:15.917 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:8", 2022-09-03T05:58:15.917 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.18", 2022-09-03T05:58:15.918 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:2", 2022-09-03T05:58:15.918 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:9", 2022-09-03T05:58:15.919 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000108", 2022-09-03T05:58:15.919 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:1", 2022-09-03T05:58:15.920 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:3", 2022-09-03T05:58:15.920 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000094", 2022-09-03T05:58:15.920 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:2", 2022-09-03T05:58:15.921 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19", 2022-09-03T05:58:15.922 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:3", 2022-09-03T05:58:15.922 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000060", 2022-09-03T05:58:15.923 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:5", 2022-09-03T05:58:15.923 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000109", 2022-09-03T05:58:15.924 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.15", 2022-09-03T05:58:15.924 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:6", 2022-09-03T05:58:15.925 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:5", 2022-09-03T05:58:15.925 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:9", 2022-09-03T05:58:15.926 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:7", 2022-09-03T05:58:15.926 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000013", 2022-09-03T05:58:15.927 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:1", 2022-09-03T05:58:15.927 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:0", 2022-09-03T05:58:15.928 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:1", 2022-09-03T05:58:15.928 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:8", 2022-09-03T05:58:15.928 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:7", 2022-09-03T05:58:15.929 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:9", 2022-09-03T05:58:15.929 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:2", 2022-09-03T05:58:15.929 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:4", 2022-09-03T05:58:15.929 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:8", 2022-09-03T05:58:15.930 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:4", 2022-09-03T05:58:15.930 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38", 2022-09-03T05:58:15.931 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:7", 2022-09-03T05:58:15.931 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:0", 2022-09-03T05:58:15.932 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:10", 2022-09-03T05:58:15.932 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:0", 2022-09-03T05:58:15.933 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:4", 2022-09-03T05:58:15.933 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000051", 2022-09-03T05:58:15.934 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:2", 2022-09-03T05:58:15.934 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21", 2022-09-03T05:58:15.935 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:8", 2022-09-03T05:58:15.935 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000106", 2022-09-03T05:58:15.936 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:1", 2022-09-03T05:58:15.936 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:2", 2022-09-03T05:58:15.937 INFO:teuthology.orchestra.run.smithi181.stdout: "data_log.2.gziGYWODx6H.0", 2022-09-03T05:58:15.938 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:9", 2022-09-03T05:58:15.938 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10", 2022-09-03T05:58:15.939 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:3", 2022-09-03T05:58:15.939 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:10", 2022-09-03T05:58:15.940 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:0", 2022-09-03T05:58:15.940 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:1", 2022-09-03T05:58:15.941 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:6", 2022-09-03T05:58:15.941 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000017", 2022-09-03T05:58:15.942 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:8", 2022-09-03T05:58:15.942 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:1", 2022-09-03T05:58:15.943 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:0", 2022-09-03T05:58:15.943 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:4", 2022-09-03T05:58:15.944 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000023", 2022-09-03T05:58:15.944 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:9", 2022-09-03T05:58:15.945 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18", 2022-09-03T05:58:15.945 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:3", 2022-09-03T05:58:15.946 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:0", 2022-09-03T05:58:15.947 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4.0.retry", 2022-09-03T05:58:15.947 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:6", 2022-09-03T05:58:15.948 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:3", 2022-09-03T05:58:15.948 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:8", 2022-09-03T05:58:15.949 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:4", 2022-09-03T05:58:15.949 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000107", 2022-09-03T05:58:15.950 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:10", 2022-09-03T05:58:15.950 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:5", 2022-09-03T05:58:15.951 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000046", 2022-09-03T05:58:15.952 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:6", 2022-09-03T05:58:15.953 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:5", 2022-09-03T05:58:15.953 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:2", 2022-09-03T05:58:15.954 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:6", 2022-09-03T05:58:15.954 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:10", 2022-09-03T05:58:15.955 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.22", 2022-09-03T05:58:15.955 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:8", 2022-09-03T05:58:15.956 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:7", 2022-09-03T05:58:15.957 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:2", 2022-09-03T05:58:15.958 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:6", 2022-09-03T05:58:15.958 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:7", 2022-09-03T05:58:15.959 INFO:teuthology.orchestra.run.smithi181.stdout: "data_log.2", 2022-09-03T05:58:15.959 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:6", 2022-09-03T05:58:15.959 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:10", 2022-09-03T05:58:15.959 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000067", 2022-09-03T05:58:15.960 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:8", 2022-09-03T05:58:15.960 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:2", 2022-09-03T05:58:15.960 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:58:15.961 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38", 2022-09-03T05:58:15.961 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:4", 2022-09-03T05:58:15.961 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:8", 2022-09-03T05:58:15.962 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:9", 2022-09-03T05:58:15.962 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:0", 2022-09-03T05:58:15.962 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:5", 2022-09-03T05:58:15.962 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:10", 2022-09-03T05:58:15.963 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.9", 2022-09-03T05:58:15.963 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000086", 2022-09-03T05:58:15.964 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.5", 2022-09-03T05:58:15.964 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:2", 2022-09-03T05:58:15.964 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:3", 2022-09-03T05:58:15.964 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:7", 2022-09-03T05:58:15.965 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:0", 2022-09-03T05:58:15.965 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:3", 2022-09-03T05:58:15.965 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:6", 2022-09-03T05:58:15.966 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:5", 2022-09-03T05:58:15.966 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:4", 2022-09-03T05:58:15.966 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:2", 2022-09-03T05:58:15.966 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4.3", 2022-09-03T05:58:15.967 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:4", 2022-09-03T05:58:15.967 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.20", 2022-09-03T05:58:15.968 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:2", 2022-09-03T05:58:15.968 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:5", 2022-09-03T05:58:15.968 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:4", 2022-09-03T05:58:15.969 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:5", 2022-09-03T05:58:15.969 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:8", 2022-09-03T05:58:15.969 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.e40d88ba-d542-483a-afa2-5de687affd14.1.retry", 2022-09-03T05:58:15.969 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:2", 2022-09-03T05:58:15.970 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:9", 2022-09-03T05:58:15.970 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:9", 2022-09-03T05:58:15.970 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:1", 2022-09-03T05:58:15.971 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17", 2022-09-03T05:58:15.971 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:8", 2022-09-03T05:58:15.971 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:7", 2022-09-03T05:58:15.972 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:5", 2022-09-03T05:58:15.972 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000050", 2022-09-03T05:58:15.972 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000102", 2022-09-03T05:58:15.972 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23", 2022-09-03T05:58:15.973 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:0", 2022-09-03T05:58:15.973 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:4", 2022-09-03T05:58:15.973 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:8", 2022-09-03T05:58:15.974 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:1", 2022-09-03T05:58:15.974 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:3", 2022-09-03T05:58:15.974 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:6", 2022-09-03T05:58:15.974 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:2", 2022-09-03T05:58:15.975 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000124", 2022-09-03T05:58:15.975 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:9", 2022-09-03T05:58:15.975 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:0", 2022-09-03T05:58:15.975 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000001", 2022-09-03T05:58:15.976 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13", 2022-09-03T05:58:15.976 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19", 2022-09-03T05:58:15.976 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:0", 2022-09-03T05:58:15.976 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000126", 2022-09-03T05:58:15.977 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:5", 2022-09-03T05:58:15.977 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:8", 2022-09-03T05:58:15.977 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:1", 2022-09-03T05:58:15.977 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000055", 2022-09-03T05:58:15.978 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000061", 2022-09-03T05:58:15.978 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:6", 2022-09-03T05:58:15.979 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:10", 2022-09-03T05:58:15.979 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13", 2022-09-03T05:58:15.979 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:0", 2022-09-03T05:58:15.979 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:6", 2022-09-03T05:58:15.980 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:3", 2022-09-03T05:58:15.980 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000097", 2022-09-03T05:58:15.980 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:7", 2022-09-03T05:58:15.980 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:9", 2022-09-03T05:58:15.981 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:58:15.981 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:6", 2022-09-03T05:58:15.981 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:10", 2022-09-03T05:58:15.981 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.e40d88ba-d542-483a-afa2-5de687affd14.0.retry", 2022-09-03T05:58:15.982 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:4", 2022-09-03T05:58:15.982 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000042", 2022-09-03T05:58:15.982 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:2", 2022-09-03T05:58:15.982 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:3", 2022-09-03T05:58:15.983 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:8", 2022-09-03T05:58:15.983 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:1", 2022-09-03T05:58:15.983 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:2", 2022-09-03T05:58:15.983 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:1", 2022-09-03T05:58:15.984 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000075", 2022-09-03T05:58:15.984 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:3", 2022-09-03T05:58:15.984 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000120", 2022-09-03T05:58:15.985 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000030", 2022-09-03T05:58:15.985 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:7", 2022-09-03T05:58:15.985 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:9", 2022-09-03T05:58:15.985 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:7", 2022-09-03T05:58:15.986 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:0", 2022-09-03T05:58:15.986 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:0", 2022-09-03T05:58:15.986 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:1", 2022-09-03T05:58:15.986 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:8", 2022-09-03T05:58:15.987 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:3", 2022-09-03T05:58:15.987 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:7", 2022-09-03T05:58:15.987 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:3" 2022-09-03T05:58:15.987 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:58:15.988 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:58:15.990 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['log', 'list', '--rgw-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:15.990 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'log', 'list', '--rgw-zone', 'test-zone2', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:15.991 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 log list --rgw-zone test-zone2 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:16.045 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:16.046 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:16.102 INFO:teuthology.orchestra.run.smithi189.stdout:[ 2022-09-03T05:58:16.103 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:0", 2022-09-03T05:58:16.103 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000078", 2022-09-03T05:58:16.104 INFO:teuthology.orchestra.run.smithi189.stdout: "meta.history", 2022-09-03T05:58:16.104 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:3", 2022-09-03T05:58:16.104 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:4", 2022-09-03T05:58:16.106 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:5", 2022-09-03T05:58:16.107 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000070", 2022-09-03T05:58:16.107 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000104", 2022-09-03T05:58:16.108 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:9", 2022-09-03T05:58:16.108 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:2", 2022-09-03T05:58:16.110 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:6", 2022-09-03T05:58:16.111 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:4", 2022-09-03T05:58:16.111 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:3", 2022-09-03T05:58:16.112 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:8", 2022-09-03T05:58:16.112 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000026", 2022-09-03T05:58:16.113 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:5", 2022-09-03T05:58:16.113 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:5", 2022-09-03T05:58:16.113 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:8", 2022-09-03T05:58:16.114 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:58:16.114 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.8", 2022-09-03T05:58:16.115 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:4", 2022-09-03T05:58:16.116 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:0", 2022-09-03T05:58:16.116 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:4", 2022-09-03T05:58:16.117 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000028", 2022-09-03T05:58:16.118 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:3", 2022-09-03T05:58:16.118 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:9", 2022-09-03T05:58:16.119 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:9", 2022-09-03T05:58:16.121 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:10", 2022-09-03T05:58:16.122 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:1", 2022-09-03T05:58:16.123 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:6", 2022-09-03T05:58:16.123 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:7", 2022-09-03T05:58:16.124 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:10", 2022-09-03T05:58:16.124 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:3", 2022-09-03T05:58:16.125 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:2", 2022-09-03T05:58:16.125 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:9", 2022-09-03T05:58:16.126 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000040", 2022-09-03T05:58:16.126 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:4", 2022-09-03T05:58:16.127 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:6", 2022-09-03T05:58:16.127 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:10", 2022-09-03T05:58:16.127 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:6", 2022-09-03T05:58:16.128 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:4", 2022-09-03T05:58:16.128 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.6", 2022-09-03T05:58:16.128 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:9", 2022-09-03T05:58:16.128 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:9", 2022-09-03T05:58:16.129 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000015", 2022-09-03T05:58:16.129 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37", 2022-09-03T05:58:16.129 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:1", 2022-09-03T05:58:16.130 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000069", 2022-09-03T05:58:16.130 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000095", 2022-09-03T05:58:16.131 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000003", 2022-09-03T05:58:16.131 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000047", 2022-09-03T05:58:16.132 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:3", 2022-09-03T05:58:16.132 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:7", 2022-09-03T05:58:16.132 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:4", 2022-09-03T05:58:16.134 INFO:teuthology.orchestra.run.smithi189.stdout: "data_log.2.cuRMcss+gv2.0", 2022-09-03T05:58:16.134 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:2", 2022-09-03T05:58:16.134 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:6", 2022-09-03T05:58:16.135 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000035", 2022-09-03T05:58:16.135 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:2", 2022-09-03T05:58:16.136 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:1", 2022-09-03T05:58:16.136 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000037", 2022-09-03T05:58:16.137 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:1", 2022-09-03T05:58:16.137 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:10", 2022-09-03T05:58:16.138 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:8", 2022-09-03T05:58:16.138 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:2", 2022-09-03T05:58:16.140 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:3", 2022-09-03T05:58:16.140 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:6", 2022-09-03T05:58:16.141 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.shard.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4.0", 2022-09-03T05:58:16.141 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:1", 2022-09-03T05:58:16.142 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:6", 2022-09-03T05:58:16.142 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:3", 2022-09-03T05:58:16.143 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:9", 2022-09-03T05:58:16.143 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5", 2022-09-03T05:58:16.145 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:9", 2022-09-03T05:58:16.145 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000024", 2022-09-03T05:58:16.146 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:7", 2022-09-03T05:58:16.146 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:5", 2022-09-03T05:58:16.146 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000000", 2022-09-03T05:58:16.147 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:5", 2022-09-03T05:58:16.147 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000031", 2022-09-03T05:58:16.148 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:5", 2022-09-03T05:58:16.148 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:3", 2022-09-03T05:58:16.149 INFO:teuthology.orchestra.run.smithi189.stdout: "meta.log.229c309b-f405-4ead-a3a8-baa670c6da51.2", 2022-09-03T05:58:16.149 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:10", 2022-09-03T05:58:16.150 INFO:teuthology.orchestra.run.smithi189.stdout: "data_log.3", 2022-09-03T05:58:16.150 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:3", 2022-09-03T05:58:16.150 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:10", 2022-09-03T05:58:16.151 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:10", 2022-09-03T05:58:16.151 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:10", 2022-09-03T05:58:16.152 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:58:16.152 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:58:16.153 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:10", 2022-09-03T05:58:16.153 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:2", 2022-09-03T05:58:16.154 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:7", 2022-09-03T05:58:16.154 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.shard.91786ef1-cdb4-4061-8a3e-2a28cc497285.1", 2022-09-03T05:58:16.156 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:6", 2022-09-03T05:58:16.156 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:5", 2022-09-03T05:58:16.157 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:7", 2022-09-03T05:58:16.157 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5", 2022-09-03T05:58:16.158 INFO:teuthology.orchestra.run.smithi189.stdout: "data_loggenerations_metadata", 2022-09-03T05:58:16.158 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:8", 2022-09-03T05:58:16.159 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:5", 2022-09-03T05:58:16.159 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:10", 2022-09-03T05:58:16.159 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000076", 2022-09-03T05:58:16.160 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:10", 2022-09-03T05:58:16.161 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000113", 2022-09-03T05:58:16.162 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000005", 2022-09-03T05:58:16.162 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:4", 2022-09-03T05:58:16.163 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000011", 2022-09-03T05:58:16.164 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:0", 2022-09-03T05:58:16.164 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000058", 2022-09-03T05:58:16.165 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000052", 2022-09-03T05:58:16.165 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:0", 2022-09-03T05:58:16.166 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:0", 2022-09-03T05:58:16.166 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:1", 2022-09-03T05:58:16.167 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:4", 2022-09-03T05:58:16.167 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:0", 2022-09-03T05:58:16.168 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:6", 2022-09-03T05:58:16.169 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:5", 2022-09-03T05:58:16.169 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:9", 2022-09-03T05:58:16.170 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:7", 2022-09-03T05:58:16.170 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:3", 2022-09-03T05:58:16.171 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:58:16.171 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:9", 2022-09-03T05:58:16.172 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:3", 2022-09-03T05:58:16.172 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:4", 2022-09-03T05:58:16.173 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:0", 2022-09-03T05:58:16.173 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:10", 2022-09-03T05:58:16.173 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:10", 2022-09-03T05:58:16.174 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000088", 2022-09-03T05:58:16.174 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:10", 2022-09-03T05:58:16.175 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000080", 2022-09-03T05:58:16.175 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000090", 2022-09-03T05:58:16.175 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:2", 2022-09-03T05:58:16.176 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:5", 2022-09-03T05:58:16.176 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:8", 2022-09-03T05:58:16.177 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000110", 2022-09-03T05:58:16.177 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21", 2022-09-03T05:58:16.178 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:0", 2022-09-03T05:58:16.178 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:1", 2022-09-03T05:58:16.179 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:4", 2022-09-03T05:58:16.179 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:1", 2022-09-03T05:58:16.180 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000096", 2022-09-03T05:58:16.180 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:1", 2022-09-03T05:58:16.181 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:7", 2022-09-03T05:58:16.181 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:8", 2022-09-03T05:58:16.182 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:8", 2022-09-03T05:58:16.182 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:3", 2022-09-03T05:58:16.182 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11", 2022-09-03T05:58:16.183 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000087", 2022-09-03T05:58:16.183 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000008", 2022-09-03T05:58:16.184 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:5", 2022-09-03T05:58:16.184 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:2", 2022-09-03T05:58:16.185 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22", 2022-09-03T05:58:16.185 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:2", 2022-09-03T05:58:16.186 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:2", 2022-09-03T05:58:16.186 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:7", 2022-09-03T05:58:16.196 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:8", 2022-09-03T05:58:16.197 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:9", 2022-09-03T05:58:16.197 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:1", 2022-09-03T05:58:16.198 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:1", 2022-09-03T05:58:16.198 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000006", 2022-09-03T05:58:16.199 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:6", 2022-09-03T05:58:16.199 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000029", 2022-09-03T05:58:16.200 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:9", 2022-09-03T05:58:16.200 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:4", 2022-09-03T05:58:16.201 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:0", 2022-09-03T05:58:16.201 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:1", 2022-09-03T05:58:16.201 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:6", 2022-09-03T05:58:16.201 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:4", 2022-09-03T05:58:16.202 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:58:16.202 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000089", 2022-09-03T05:58:16.202 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000022", 2022-09-03T05:58:16.202 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:0", 2022-09-03T05:58:16.203 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:1", 2022-09-03T05:58:16.203 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:3", 2022-09-03T05:58:16.203 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:5", 2022-09-03T05:58:16.203 INFO:teuthology.orchestra.run.smithi189.stdout: "bilog.trim", 2022-09-03T05:58:16.204 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:6", 2022-09-03T05:58:16.204 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:3", 2022-09-03T05:58:16.204 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000016", 2022-09-03T05:58:16.204 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:5", 2022-09-03T05:58:16.205 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:10", 2022-09-03T05:58:16.205 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:10", 2022-09-03T05:58:16.205 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23", 2022-09-03T05:58:16.205 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:1", 2022-09-03T05:58:16.206 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:2", 2022-09-03T05:58:16.206 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:8", 2022-09-03T05:58:16.206 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000041", 2022-09-03T05:58:16.207 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:10", 2022-09-03T05:58:16.207 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:5", 2022-09-03T05:58:16.207 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:9", 2022-09-03T05:58:16.207 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:6", 2022-09-03T05:58:16.208 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:6", 2022-09-03T05:58:16.208 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:6", 2022-09-03T05:58:16.208 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:5", 2022-09-03T05:58:16.208 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:9", 2022-09-03T05:58:16.208 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000018", 2022-09-03T05:58:16.209 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:3", 2022-09-03T05:58:16.209 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:3", 2022-09-03T05:58:16.209 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000092", 2022-09-03T05:58:16.209 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:1", 2022-09-03T05:58:16.210 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37", 2022-09-03T05:58:16.210 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:8", 2022-09-03T05:58:16.210 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:3", 2022-09-03T05:58:16.211 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:0", 2022-09-03T05:58:16.211 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.4", 2022-09-03T05:58:16.211 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:4", 2022-09-03T05:58:16.211 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:2", 2022-09-03T05:58:16.212 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:8", 2022-09-03T05:58:16.212 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:7", 2022-09-03T05:58:16.212 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:10", 2022-09-03T05:58:16.212 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:7", 2022-09-03T05:58:16.213 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:2", 2022-09-03T05:58:16.213 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:2", 2022-09-03T05:58:16.213 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000014", 2022-09-03T05:58:16.213 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:8", 2022-09-03T05:58:16.214 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:0", 2022-09-03T05:58:16.214 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13", 2022-09-03T05:58:16.214 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:1", 2022-09-03T05:58:16.214 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3", 2022-09-03T05:58:16.215 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:7", 2022-09-03T05:58:16.215 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:4", 2022-09-03T05:58:16.215 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:10", 2022-09-03T05:58:16.215 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:8", 2022-09-03T05:58:16.216 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000112", 2022-09-03T05:58:16.216 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:3", 2022-09-03T05:58:16.216 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000007", 2022-09-03T05:58:16.216 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000021", 2022-09-03T05:58:16.217 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:4", 2022-09-03T05:58:16.217 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000064", 2022-09-03T05:58:16.217 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:2", 2022-09-03T05:58:16.217 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000071", 2022-09-03T05:58:16.218 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.shard.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4.1", 2022-09-03T05:58:16.218 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:1", 2022-09-03T05:58:16.218 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:7", 2022-09-03T05:58:16.218 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000074", 2022-09-03T05:58:16.219 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:1", 2022-09-03T05:58:16.219 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:5", 2022-09-03T05:58:16.219 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:6", 2022-09-03T05:58:16.220 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:6", 2022-09-03T05:58:16.220 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:4", 2022-09-03T05:58:16.220 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:5", 2022-09-03T05:58:16.220 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:9", 2022-09-03T05:58:16.221 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:7", 2022-09-03T05:58:16.221 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:10", 2022-09-03T05:58:16.221 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:5", 2022-09-03T05:58:16.221 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000081", 2022-09-03T05:58:16.222 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000009", 2022-09-03T05:58:16.222 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:2", 2022-09-03T05:58:16.222 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:2", 2022-09-03T05:58:16.223 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000121", 2022-09-03T05:58:16.223 INFO:teuthology.orchestra.run.smithi189.stdout: "data_log.1", 2022-09-03T05:58:16.223 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:6", 2022-09-03T05:58:16.224 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:10", 2022-09-03T05:58:16.224 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:3", 2022-09-03T05:58:16.224 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:7", 2022-09-03T05:58:16.224 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:8", 2022-09-03T05:58:16.225 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:9", 2022-09-03T05:58:16.225 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000125", 2022-09-03T05:58:16.225 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:0", 2022-09-03T05:58:16.226 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:0", 2022-09-03T05:58:16.226 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000082", 2022-09-03T05:58:16.226 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:9", 2022-09-03T05:58:16.227 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15", 2022-09-03T05:58:16.227 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:2", 2022-09-03T05:58:16.227 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:7", 2022-09-03T05:58:16.228 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:8", 2022-09-03T05:58:16.228 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:10", 2022-09-03T05:58:16.228 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:6", 2022-09-03T05:58:16.228 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:9", 2022-09-03T05:58:16.229 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:4", 2022-09-03T05:58:16.229 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:10", 2022-09-03T05:58:16.229 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:1", 2022-09-03T05:58:16.229 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000105", 2022-09-03T05:58:16.230 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000059", 2022-09-03T05:58:16.230 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:8", 2022-09-03T05:58:16.230 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:6", 2022-09-03T05:58:16.231 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:0", 2022-09-03T05:58:16.231 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:6", 2022-09-03T05:58:16.231 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:1", 2022-09-03T05:58:16.232 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:0", 2022-09-03T05:58:16.232 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:7", 2022-09-03T05:58:16.232 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:8", 2022-09-03T05:58:16.233 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:9", 2022-09-03T05:58:16.233 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:8", 2022-09-03T05:58:16.233 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000077", 2022-09-03T05:58:16.233 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:5", 2022-09-03T05:58:16.234 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:7", 2022-09-03T05:58:16.234 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000032", 2022-09-03T05:58:16.234 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:9", 2022-09-03T05:58:16.234 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:0", 2022-09-03T05:58:16.235 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:2", 2022-09-03T05:58:16.235 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:2", 2022-09-03T05:58:16.235 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:7", 2022-09-03T05:58:16.236 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:7", 2022-09-03T05:58:16.236 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:1", 2022-09-03T05:58:16.236 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:4", 2022-09-03T05:58:16.236 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12", 2022-09-03T05:58:16.237 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:8", 2022-09-03T05:58:16.237 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:9", 2022-09-03T05:58:16.237 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:3", 2022-09-03T05:58:16.238 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:1", 2022-09-03T05:58:16.238 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:10", 2022-09-03T05:58:16.238 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:6", 2022-09-03T05:58:16.239 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:1", 2022-09-03T05:58:16.239 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:4", 2022-09-03T05:58:16.239 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:5", 2022-09-03T05:58:16.239 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:4", 2022-09-03T05:58:16.240 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:4", 2022-09-03T05:58:16.240 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000053", 2022-09-03T05:58:16.240 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:6", 2022-09-03T05:58:16.240 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.shard.91786ef1-cdb4-4061-8a3e-2a28cc497285.2", 2022-09-03T05:58:16.241 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000091", 2022-09-03T05:58:16.241 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:1", 2022-09-03T05:58:16.241 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:4", 2022-09-03T05:58:16.241 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:9", 2022-09-03T05:58:16.242 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:3", 2022-09-03T05:58:16.242 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000065", 2022-09-03T05:58:16.242 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:0", 2022-09-03T05:58:16.242 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:7", 2022-09-03T05:58:16.243 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:8", 2022-09-03T05:58:16.243 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000083", 2022-09-03T05:58:16.243 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000010", 2022-09-03T05:58:16.243 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:2", 2022-09-03T05:58:16.244 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:2", 2022-09-03T05:58:16.244 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:1", 2022-09-03T05:58:16.244 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:5", 2022-09-03T05:58:16.244 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:10", 2022-09-03T05:58:16.245 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:7", 2022-09-03T05:58:16.245 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:3", 2022-09-03T05:58:16.245 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.7", 2022-09-03T05:58:16.245 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.3", 2022-09-03T05:58:16.246 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:7", 2022-09-03T05:58:16.246 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:1", 2022-09-03T05:58:16.246 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:3", 2022-09-03T05:58:16.246 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:9", 2022-09-03T05:58:16.247 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:7", 2022-09-03T05:58:16.247 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:3", 2022-09-03T05:58:16.247 INFO:teuthology.orchestra.run.smithi189.stdout: "mdlog.sync-status.shard.2", 2022-09-03T05:58:16.247 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:4", 2022-09-03T05:58:16.248 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:4", 2022-09-03T05:58:16.248 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:5", 2022-09-03T05:58:16.248 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:5", 2022-09-03T05:58:16.248 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:2", 2022-09-03T05:58:16.249 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000045", 2022-09-03T05:58:16.249 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:5", 2022-09-03T05:58:16.249 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:3", 2022-09-03T05:58:16.249 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000002", 2022-09-03T05:58:16.250 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.12", 2022-09-03T05:58:16.250 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:2", 2022-09-03T05:58:16.250 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:8", 2022-09-03T05:58:16.250 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.shard.91786ef1-cdb4-4061-8a3e-2a28cc497285.1.retry", 2022-09-03T05:58:16.251 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:10", 2022-09-03T05:58:16.251 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:1", 2022-09-03T05:58:16.251 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:9", 2022-09-03T05:58:16.251 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14", 2022-09-03T05:58:16.252 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:10", 2022-09-03T05:58:16.252 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000116", 2022-09-03T05:58:16.252 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:3", 2022-09-03T05:58:16.252 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:2", 2022-09-03T05:58:16.253 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000034", 2022-09-03T05:58:16.253 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:3", 2022-09-03T05:58:16.253 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:8", 2022-09-03T05:58:16.253 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000101", 2022-09-03T05:58:16.254 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:8", 2022-09-03T05:58:16.254 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.13", 2022-09-03T05:58:16.254 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36", 2022-09-03T05:58:16.254 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:5", 2022-09-03T05:58:16.255 INFO:teuthology.orchestra.run.smithi189.stdout: "mdlog.sync-status.shard.1", 2022-09-03T05:58:16.255 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:8", 2022-09-03T05:58:16.255 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:1", 2022-09-03T05:58:16.255 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:4", 2022-09-03T05:58:16.256 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000079", 2022-09-03T05:58:16.256 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000049", 2022-09-03T05:58:16.256 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:4", 2022-09-03T05:58:16.256 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000117", 2022-09-03T05:58:16.257 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:10", 2022-09-03T05:58:16.257 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000044", 2022-09-03T05:58:16.257 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:10", 2022-09-03T05:58:16.257 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:9", 2022-09-03T05:58:16.258 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:0", 2022-09-03T05:58:16.258 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:4", 2022-09-03T05:58:16.258 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:6", 2022-09-03T05:58:16.259 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.shard.91786ef1-cdb4-4061-8a3e-2a28cc497285.0.retry", 2022-09-03T05:58:16.259 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:8", 2022-09-03T05:58:16.259 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:6", 2022-09-03T05:58:16.259 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:6", 2022-09-03T05:58:16.260 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:0", 2022-09-03T05:58:16.260 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:9", 2022-09-03T05:58:16.260 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.14", 2022-09-03T05:58:16.261 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:3", 2022-09-03T05:58:16.261 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000066", 2022-09-03T05:58:16.261 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:10", 2022-09-03T05:58:16.261 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:9", 2022-09-03T05:58:16.262 INFO:teuthology.orchestra.run.smithi189.stdout: "data_log.3.7kcZG0GILEc.0", 2022-09-03T05:58:16.262 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:0", 2022-09-03T05:58:16.262 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000068", 2022-09-03T05:58:16.262 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:4", 2022-09-03T05:58:16.263 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:9", 2022-09-03T05:58:16.263 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000085", 2022-09-03T05:58:16.263 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:1", 2022-09-03T05:58:16.263 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:5", 2022-09-03T05:58:16.264 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:3", 2022-09-03T05:58:16.264 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:1", 2022-09-03T05:58:16.264 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:10", 2022-09-03T05:58:16.265 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:4", 2022-09-03T05:58:16.265 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:7", 2022-09-03T05:58:16.265 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:9", 2022-09-03T05:58:16.265 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:5", 2022-09-03T05:58:16.266 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000073", 2022-09-03T05:58:16.266 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:6", 2022-09-03T05:58:16.266 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:4", 2022-09-03T05:58:16.267 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:1", 2022-09-03T05:58:16.267 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:5", 2022-09-03T05:58:16.267 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000038", 2022-09-03T05:58:16.267 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4", 2022-09-03T05:58:16.268 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:9", 2022-09-03T05:58:16.268 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:7", 2022-09-03T05:58:16.268 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:5", 2022-09-03T05:58:16.268 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:8", 2022-09-03T05:58:16.269 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.10", 2022-09-03T05:58:16.269 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000118", 2022-09-03T05:58:16.269 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.shard.91786ef1-cdb4-4061-8a3e-2a28cc497285.0", 2022-09-03T05:58:16.269 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:8", 2022-09-03T05:58:16.270 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:8", 2022-09-03T05:58:16.270 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1", 2022-09-03T05:58:16.270 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:1", 2022-09-03T05:58:16.270 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000036", 2022-09-03T05:58:16.271 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:2", 2022-09-03T05:58:16.271 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:5", 2022-09-03T05:58:16.271 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:8", 2022-09-03T05:58:16.271 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000103", 2022-09-03T05:58:16.272 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:2", 2022-09-03T05:58:16.272 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:10", 2022-09-03T05:58:16.272 INFO:teuthology.orchestra.run.smithi189.stdout: "mdlog.sync-status.shard.3", 2022-09-03T05:58:16.272 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:3", 2022-09-03T05:58:16.273 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:9", 2022-09-03T05:58:16.273 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:6", 2022-09-03T05:58:16.273 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:10", 2022-09-03T05:58:16.274 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000119", 2022-09-03T05:58:16.274 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:7", 2022-09-03T05:58:16.274 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:3", 2022-09-03T05:58:16.274 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:6", 2022-09-03T05:58:16.275 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000098", 2022-09-03T05:58:16.275 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:9", 2022-09-03T05:58:16.275 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:4", 2022-09-03T05:58:16.275 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.shard.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4.2", 2022-09-03T05:58:16.276 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:3", 2022-09-03T05:58:16.276 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:3", 2022-09-03T05:58:16.276 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:0", 2022-09-03T05:58:16.276 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:4", 2022-09-03T05:58:16.277 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:7", 2022-09-03T05:58:16.277 INFO:teuthology.orchestra.run.smithi189.stdout: "mdlog.sync-status.shard.0", 2022-09-03T05:58:16.277 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:3", 2022-09-03T05:58:16.277 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:10", 2022-09-03T05:58:16.278 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:8", 2022-09-03T05:58:16.278 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000027", 2022-09-03T05:58:16.278 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.shard.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4.2.retry", 2022-09-03T05:58:16.278 INFO:teuthology.orchestra.run.smithi189.stdout: "mdlog.sync-status", 2022-09-03T05:58:16.279 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000019", 2022-09-03T05:58:16.279 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:0", 2022-09-03T05:58:16.279 INFO:teuthology.orchestra.run.smithi189.stdout: "data_log.0.DYpQr9xzTqi.0", 2022-09-03T05:58:16.279 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:8", 2022-09-03T05:58:16.280 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000039", 2022-09-03T05:58:16.280 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:7", 2022-09-03T05:58:16.280 INFO:teuthology.orchestra.run.smithi189.stdout: "data_log.1.l0IOUOACUXX.0", 2022-09-03T05:58:16.280 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:5", 2022-09-03T05:58:16.281 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:5", 2022-09-03T05:58:16.281 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:3", 2022-09-03T05:58:16.281 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:0", 2022-09-03T05:58:16.281 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:9", 2022-09-03T05:58:16.282 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:5", 2022-09-03T05:58:16.282 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000100", 2022-09-03T05:58:16.282 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:7", 2022-09-03T05:58:16.283 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:0", 2022-09-03T05:58:16.283 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:2", 2022-09-03T05:58:16.283 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:5", 2022-09-03T05:58:16.284 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12", 2022-09-03T05:58:16.284 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11", 2022-09-03T05:58:16.284 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10", 2022-09-03T05:58:16.284 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:0", 2022-09-03T05:58:16.285 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16", 2022-09-03T05:58:16.285 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:2", 2022-09-03T05:58:16.285 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:10", 2022-09-03T05:58:16.286 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000093", 2022-09-03T05:58:16.286 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:7", 2022-09-03T05:58:16.286 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:5", 2022-09-03T05:58:16.287 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:8", 2022-09-03T05:58:16.287 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:0", 2022-09-03T05:58:16.287 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:1", 2022-09-03T05:58:16.287 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:6", 2022-09-03T05:58:16.288 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000004", 2022-09-03T05:58:16.288 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000063", 2022-09-03T05:58:16.288 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:2", 2022-09-03T05:58:16.289 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000122", 2022-09-03T05:58:16.289 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:1", 2022-09-03T05:58:16.289 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.shard.91786ef1-cdb4-4061-8a3e-2a28cc497285.3", 2022-09-03T05:58:16.290 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000057", 2022-09-03T05:58:16.290 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:3", 2022-09-03T05:58:16.290 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000054", 2022-09-03T05:58:16.291 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4", 2022-09-03T05:58:16.291 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000114", 2022-09-03T05:58:16.291 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:2", 2022-09-03T05:58:16.291 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.2", 2022-09-03T05:58:16.292 INFO:teuthology.orchestra.run.smithi189.stdout: "data_log.0", 2022-09-03T05:58:16.292 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:5", 2022-09-03T05:58:16.292 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:3", 2022-09-03T05:58:16.292 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:10", 2022-09-03T05:58:16.293 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000012", 2022-09-03T05:58:16.293 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:0", 2022-09-03T05:58:16.293 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:3", 2022-09-03T05:58:16.293 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:8", 2022-09-03T05:58:16.294 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:3", 2022-09-03T05:58:16.294 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:10", 2022-09-03T05:58:16.294 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22", 2022-09-03T05:58:16.294 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:1", 2022-09-03T05:58:16.295 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:6", 2022-09-03T05:58:16.295 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:2", 2022-09-03T05:58:16.295 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:10", 2022-09-03T05:58:16.295 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:8", 2022-09-03T05:58:16.296 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000084", 2022-09-03T05:58:16.296 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000043", 2022-09-03T05:58:16.296 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:8", 2022-09-03T05:58:16.297 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:10", 2022-09-03T05:58:16.297 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.11", 2022-09-03T05:58:16.297 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.shard.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4.1.retry", 2022-09-03T05:58:16.297 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:7", 2022-09-03T05:58:16.298 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:4", 2022-09-03T05:58:16.298 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:7", 2022-09-03T05:58:16.298 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:4", 2022-09-03T05:58:16.298 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:2", 2022-09-03T05:58:16.299 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:3", 2022-09-03T05:58:16.299 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:6", 2022-09-03T05:58:16.299 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000111", 2022-09-03T05:58:16.300 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:9", 2022-09-03T05:58:16.300 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000048", 2022-09-03T05:58:16.300 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:4", 2022-09-03T05:58:16.300 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:10", 2022-09-03T05:58:16.301 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000020", 2022-09-03T05:58:16.301 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:4", 2022-09-03T05:58:16.302 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:1", 2022-09-03T05:58:16.302 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:2", 2022-09-03T05:58:16.302 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:10", 2022-09-03T05:58:16.303 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:8", 2022-09-03T05:58:16.303 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:9", 2022-09-03T05:58:16.303 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:6", 2022-09-03T05:58:16.303 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:0", 2022-09-03T05:58:16.304 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000099", 2022-09-03T05:58:16.304 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000056", 2022-09-03T05:58:16.305 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:6", 2022-09-03T05:58:16.305 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:0", 2022-09-03T05:58:16.305 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:7", 2022-09-03T05:58:16.306 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:7", 2022-09-03T05:58:16.306 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:0", 2022-09-03T05:58:16.306 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:3", 2022-09-03T05:58:16.306 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:1", 2022-09-03T05:58:16.307 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17", 2022-09-03T05:58:16.307 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000072", 2022-09-03T05:58:16.307 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:10", 2022-09-03T05:58:16.307 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:6", 2022-09-03T05:58:16.308 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:5", 2022-09-03T05:58:16.308 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17", 2022-09-03T05:58:16.308 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:0", 2022-09-03T05:58:16.308 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:7", 2022-09-03T05:58:16.309 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000062", 2022-09-03T05:58:16.309 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:58:16.309 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000115", 2022-09-03T05:58:16.309 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.1", 2022-09-03T05:58:16.310 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000033", 2022-09-03T05:58:16.310 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:6", 2022-09-03T05:58:16.310 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:0", 2022-09-03T05:58:16.311 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:1", 2022-09-03T05:58:16.311 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:10", 2022-09-03T05:58:16.311 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:3", 2022-09-03T05:58:16.312 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:6", 2022-09-03T05:58:16.312 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15", 2022-09-03T05:58:16.312 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:1", 2022-09-03T05:58:16.312 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:2", 2022-09-03T05:58:16.313 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3", 2022-09-03T05:58:16.313 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:8", 2022-09-03T05:58:16.313 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:5", 2022-09-03T05:58:16.313 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:5", 2022-09-03T05:58:16.314 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000025", 2022-09-03T05:58:16.314 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:5", 2022-09-03T05:58:16.314 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000123", 2022-09-03T05:58:16.315 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:6", 2022-09-03T05:58:16.315 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:8", 2022-09-03T05:58:16.315 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:2", 2022-09-03T05:58:16.316 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:1", 2022-09-03T05:58:16.316 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:7", 2022-09-03T05:58:16.316 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:8", 2022-09-03T05:58:16.316 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4", 2022-09-03T05:58:16.317 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:3", 2022-09-03T05:58:16.317 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:4", 2022-09-03T05:58:16.317 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000108", 2022-09-03T05:58:16.317 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:9", 2022-09-03T05:58:16.318 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:7", 2022-09-03T05:58:16.318 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:9", 2022-09-03T05:58:16.319 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:3", 2022-09-03T05:58:16.319 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000094", 2022-09-03T05:58:16.319 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:5", 2022-09-03T05:58:16.319 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16", 2022-09-03T05:58:16.320 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:2", 2022-09-03T05:58:16.320 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19", 2022-09-03T05:58:16.320 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:2", 2022-09-03T05:58:16.320 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18", 2022-09-03T05:58:16.321 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:3", 2022-09-03T05:58:16.321 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:7", 2022-09-03T05:58:16.321 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38", 2022-09-03T05:58:16.322 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000060", 2022-09-03T05:58:16.322 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:5", 2022-09-03T05:58:16.322 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.shard.91786ef1-cdb4-4061-8a3e-2a28cc497285.2.retry", 2022-09-03T05:58:16.322 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000109", 2022-09-03T05:58:16.323 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:6", 2022-09-03T05:58:16.323 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:5", 2022-09-03T05:58:16.323 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:0", 2022-09-03T05:58:16.323 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:5", 2022-09-03T05:58:16.324 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:9", 2022-09-03T05:58:16.324 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:7", 2022-09-03T05:58:16.324 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000013", 2022-09-03T05:58:16.324 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:7", 2022-09-03T05:58:16.325 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:2", 2022-09-03T05:58:16.325 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19", 2022-09-03T05:58:16.325 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:6", 2022-09-03T05:58:16.325 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:9", 2022-09-03T05:58:16.326 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:6", 2022-09-03T05:58:16.326 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:9", 2022-09-03T05:58:16.326 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:58:16.326 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38", 2022-09-03T05:58:16.327 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:10", 2022-09-03T05:58:16.327 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:0", 2022-09-03T05:58:16.327 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000051", 2022-09-03T05:58:16.328 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:2", 2022-09-03T05:58:16.328 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21", 2022-09-03T05:58:16.328 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000106", 2022-09-03T05:58:16.328 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10", 2022-09-03T05:58:16.329 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:7", 2022-09-03T05:58:16.329 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000017", 2022-09-03T05:58:16.329 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:4", 2022-09-03T05:58:16.329 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:4", 2022-09-03T05:58:16.330 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:8", 2022-09-03T05:58:16.330 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:1", 2022-09-03T05:58:16.330 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000023", 2022-09-03T05:58:16.330 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18", 2022-09-03T05:58:16.331 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:0", 2022-09-03T05:58:16.331 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:3", 2022-09-03T05:58:16.332 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:0", 2022-09-03T05:58:16.332 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.shard.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4.0.retry", 2022-09-03T05:58:16.332 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000107", 2022-09-03T05:58:16.332 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:0", 2022-09-03T05:58:16.333 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000046", 2022-09-03T05:58:16.333 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:10", 2022-09-03T05:58:16.333 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:1", 2022-09-03T05:58:16.333 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:2", 2022-09-03T05:58:16.334 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:2", 2022-09-03T05:58:16.334 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:8", 2022-09-03T05:58:16.334 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:7", 2022-09-03T05:58:16.335 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:4", 2022-09-03T05:58:16.335 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:0", 2022-09-03T05:58:16.335 INFO:teuthology.orchestra.run.smithi189.stdout: "data_log.2", 2022-09-03T05:58:16.335 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000067", 2022-09-03T05:58:16.336 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:8", 2022-09-03T05:58:16.336 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:10", 2022-09-03T05:58:16.336 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:4", 2022-09-03T05:58:16.337 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:9", 2022-09-03T05:58:16.337 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:6", 2022-09-03T05:58:16.337 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:0", 2022-09-03T05:58:16.337 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:5", 2022-09-03T05:58:16.338 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:6", 2022-09-03T05:58:16.338 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:1", 2022-09-03T05:58:16.338 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.9", 2022-09-03T05:58:16.338 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000086", 2022-09-03T05:58:16.339 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:2", 2022-09-03T05:58:16.339 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.5", 2022-09-03T05:58:16.339 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:1", 2022-09-03T05:58:16.339 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:9", 2022-09-03T05:58:16.340 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:7", 2022-09-03T05:58:16.340 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:0", 2022-09-03T05:58:16.340 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:58:16.340 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:6", 2022-09-03T05:58:16.341 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:5", 2022-09-03T05:58:16.341 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:4", 2022-09-03T05:58:16.341 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.shard.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4.3", 2022-09-03T05:58:16.341 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:4", 2022-09-03T05:58:16.342 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:0", 2022-09-03T05:58:16.342 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:5", 2022-09-03T05:58:16.342 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:8", 2022-09-03T05:58:16.342 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:9", 2022-09-03T05:58:16.343 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:9", 2022-09-03T05:58:16.343 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:6", 2022-09-03T05:58:16.343 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:9", 2022-09-03T05:58:16.343 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:8", 2022-09-03T05:58:16.344 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:7", 2022-09-03T05:58:16.344 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:7", 2022-09-03T05:58:16.344 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:5", 2022-09-03T05:58:16.344 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000050", 2022-09-03T05:58:16.345 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000102", 2022-09-03T05:58:16.345 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:2", 2022-09-03T05:58:16.345 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23", 2022-09-03T05:58:16.345 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:4", 2022-09-03T05:58:16.346 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36", 2022-09-03T05:58:16.346 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:4", 2022-09-03T05:58:16.346 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:7", 2022-09-03T05:58:16.347 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:1", 2022-09-03T05:58:16.347 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:2", 2022-09-03T05:58:16.347 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000124", 2022-09-03T05:58:16.347 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:9", 2022-09-03T05:58:16.348 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:0", 2022-09-03T05:58:16.348 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:9", 2022-09-03T05:58:16.348 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000001", 2022-09-03T05:58:16.348 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:4", 2022-09-03T05:58:16.349 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:6", 2022-09-03T05:58:16.349 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13", 2022-09-03T05:58:16.349 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000126", 2022-09-03T05:58:16.349 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:8", 2022-09-03T05:58:16.350 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:1", 2022-09-03T05:58:16.350 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000055", 2022-09-03T05:58:16.350 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:2", 2022-09-03T05:58:16.350 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:0", 2022-09-03T05:58:16.351 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000061", 2022-09-03T05:58:16.351 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:10", 2022-09-03T05:58:16.351 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:6", 2022-09-03T05:58:16.352 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:2", 2022-09-03T05:58:16.352 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:0", 2022-09-03T05:58:16.352 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:6", 2022-09-03T05:58:16.352 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:2", 2022-09-03T05:58:16.353 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:3", 2022-09-03T05:58:16.353 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000097", 2022-09-03T05:58:16.353 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:58:16.353 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:10", 2022-09-03T05:58:16.354 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:4", 2022-09-03T05:58:16.354 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:8", 2022-09-03T05:58:16.354 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000042", 2022-09-03T05:58:16.354 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14", 2022-09-03T05:58:16.355 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:8", 2022-09-03T05:58:16.355 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:3", 2022-09-03T05:58:16.355 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:1", 2022-09-03T05:58:16.356 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1", 2022-09-03T05:58:16.356 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:1", 2022-09-03T05:58:16.356 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000075", 2022-09-03T05:58:16.357 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000120", 2022-09-03T05:58:16.357 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000030", 2022-09-03T05:58:16.357 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:0", 2022-09-03T05:58:16.358 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:9", 2022-09-03T05:58:16.358 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:7", 2022-09-03T05:58:16.358 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:7", 2022-09-03T05:58:16.358 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:0", 2022-09-03T05:58:16.359 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:0", 2022-09-03T05:58:16.359 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.6ebb780b-73b6-47d3-ad4f-1dc62076c1f4:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:8" 2022-09-03T05:58:16.359 INFO:teuthology.orchestra.run.smithi189.stdout:] 2022-09-03T05:58:16.360 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:58:16.361 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['log', 'list', '--rgw-zone', 'test-zone3', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:16.362 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'log', 'list', '--rgw-zone', 'test-zone3', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:16.362 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 log list --rgw-zone test-zone3 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:16.410 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:16.410 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:16.491 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:58:16.491 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000078", 2022-09-03T05:58:16.492 INFO:teuthology.orchestra.run.smithi181.stdout: "meta.history", 2022-09-03T05:58:16.492 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:9", 2022-09-03T05:58:16.493 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:3", 2022-09-03T05:58:16.493 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:1", 2022-09-03T05:58:16.494 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:3", 2022-09-03T05:58:16.495 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:5", 2022-09-03T05:58:16.495 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000070", 2022-09-03T05:58:16.496 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000104", 2022-09-03T05:58:16.496 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:9", 2022-09-03T05:58:16.496 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:6", 2022-09-03T05:58:16.497 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:2", 2022-09-03T05:58:16.498 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:8", 2022-09-03T05:58:16.498 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000026", 2022-09-03T05:58:16.499 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:5", 2022-09-03T05:58:16.499 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:0", 2022-09-03T05:58:16.500 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:0", 2022-09-03T05:58:16.501 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:5", 2022-09-03T05:58:16.502 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:9", 2022-09-03T05:58:16.502 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:5", 2022-09-03T05:58:16.503 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:8", 2022-09-03T05:58:16.504 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.8", 2022-09-03T05:58:16.504 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:4", 2022-09-03T05:58:16.505 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:0", 2022-09-03T05:58:16.505 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000028", 2022-09-03T05:58:16.506 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:5", 2022-09-03T05:58:16.506 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:5", 2022-09-03T05:58:16.507 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:4", 2022-09-03T05:58:16.507 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:10", 2022-09-03T05:58:16.508 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:0", 2022-09-03T05:58:16.509 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:7", 2022-09-03T05:58:16.509 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5", 2022-09-03T05:58:16.509 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:1", 2022-09-03T05:58:16.510 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:9", 2022-09-03T05:58:16.510 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000040", 2022-09-03T05:58:16.510 INFO:teuthology.orchestra.run.smithi181.stdout: "meta.log.41ecd2a5-afc0-4eae-9349-7dfe39baa1cb.3", 2022-09-03T05:58:16.510 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:6", 2022-09-03T05:58:16.511 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:10", 2022-09-03T05:58:16.511 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18", 2022-09-03T05:58:16.511 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.6", 2022-09-03T05:58:16.512 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21", 2022-09-03T05:58:16.512 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:4", 2022-09-03T05:58:16.512 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000015", 2022-09-03T05:58:16.512 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:3", 2022-09-03T05:58:16.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37", 2022-09-03T05:58:16.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:5", 2022-09-03T05:58:16.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:1", 2022-09-03T05:58:16.513 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:10", 2022-09-03T05:58:16.516 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000069", 2022-09-03T05:58:16.517 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:0", 2022-09-03T05:58:16.517 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000095", 2022-09-03T05:58:16.518 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:8", 2022-09-03T05:58:16.518 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000003", 2022-09-03T05:58:16.518 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000047", 2022-09-03T05:58:16.519 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:0", 2022-09-03T05:58:16.519 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:7", 2022-09-03T05:58:16.519 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:4", 2022-09-03T05:58:16.519 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:3", 2022-09-03T05:58:16.520 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:8", 2022-09-03T05:58:16.520 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:6", 2022-09-03T05:58:16.520 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000035", 2022-09-03T05:58:16.520 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:1", 2022-09-03T05:58:16.521 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000037", 2022-09-03T05:58:16.521 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:4", 2022-09-03T05:58:16.521 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:1", 2022-09-03T05:58:16.521 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:10", 2022-09-03T05:58:16.522 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:4", 2022-09-03T05:58:16.522 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:3", 2022-09-03T05:58:16.522 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:3", 2022-09-03T05:58:16.523 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:4", 2022-09-03T05:58:16.523 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:1", 2022-09-03T05:58:16.523 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:6", 2022-09-03T05:58:16.523 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:1", 2022-09-03T05:58:16.524 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:3", 2022-09-03T05:58:16.524 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:9", 2022-09-03T05:58:16.524 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5", 2022-09-03T05:58:16.524 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:58:16.525 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:58:16.525 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000024", 2022-09-03T05:58:16.525 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:5", 2022-09-03T05:58:16.525 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:8", 2022-09-03T05:58:16.530 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000000", 2022-09-03T05:58:16.530 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:5", 2022-09-03T05:58:16.531 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000031", 2022-09-03T05:58:16.531 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:5", 2022-09-03T05:58:16.531 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:7", 2022-09-03T05:58:16.531 INFO:teuthology.orchestra.run.smithi181.stdout: "meta.log.229c309b-f405-4ead-a3a8-baa670c6da51.2", 2022-09-03T05:58:16.532 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:10", 2022-09-03T05:58:16.532 INFO:teuthology.orchestra.run.smithi181.stdout: "data_log.3", 2022-09-03T05:58:16.532 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:3", 2022-09-03T05:58:16.533 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:10", 2022-09-03T05:58:16.533 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:6", 2022-09-03T05:58:16.533 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:58:16.533 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:1", 2022-09-03T05:58:16.534 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:7", 2022-09-03T05:58:16.534 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.91786ef1-cdb4-4061-8a3e-2a28cc497285.1", 2022-09-03T05:58:16.535 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:6", 2022-09-03T05:58:16.535 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:5", 2022-09-03T05:58:16.536 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:10", 2022-09-03T05:58:16.536 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:8", 2022-09-03T05:58:16.536 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:8", 2022-09-03T05:58:16.536 INFO:teuthology.orchestra.run.smithi181.stdout: "data_loggenerations_metadata", 2022-09-03T05:58:16.537 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:7", 2022-09-03T05:58:16.537 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:5", 2022-09-03T05:58:16.537 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:4", 2022-09-03T05:58:16.538 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:10", 2022-09-03T05:58:16.540 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:5", 2022-09-03T05:58:16.540 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000076", 2022-09-03T05:58:16.541 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:10", 2022-09-03T05:58:16.541 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:0", 2022-09-03T05:58:16.542 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000113", 2022-09-03T05:58:16.543 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000005", 2022-09-03T05:58:16.543 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:5", 2022-09-03T05:58:16.544 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000011", 2022-09-03T05:58:16.544 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:0", 2022-09-03T05:58:16.545 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000058", 2022-09-03T05:58:16.545 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000052", 2022-09-03T05:58:16.546 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:7", 2022-09-03T05:58:16.547 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:6", 2022-09-03T05:58:16.547 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:4", 2022-09-03T05:58:16.548 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:0", 2022-09-03T05:58:16.549 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:6", 2022-09-03T05:58:16.549 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:5", 2022-09-03T05:58:16.550 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:9", 2022-09-03T05:58:16.550 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:4", 2022-09-03T05:58:16.551 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:0", 2022-09-03T05:58:16.551 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:5", 2022-09-03T05:58:16.552 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:0", 2022-09-03T05:58:16.552 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:3", 2022-09-03T05:58:16.553 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:0", 2022-09-03T05:58:16.553 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:2", 2022-09-03T05:58:16.554 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14", 2022-09-03T05:58:16.554 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37", 2022-09-03T05:58:16.555 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:9", 2022-09-03T05:58:16.555 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:3", 2022-09-03T05:58:16.555 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:4", 2022-09-03T05:58:16.556 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:0", 2022-09-03T05:58:16.556 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:10", 2022-09-03T05:58:16.557 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:10", 2022-09-03T05:58:16.557 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:9", 2022-09-03T05:58:16.557 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:9", 2022-09-03T05:58:16.558 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000088", 2022-09-03T05:58:16.558 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36", 2022-09-03T05:58:16.558 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000080", 2022-09-03T05:58:16.559 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000090", 2022-09-03T05:58:16.559 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:8", 2022-09-03T05:58:16.559 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:2", 2022-09-03T05:58:16.559 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000110", 2022-09-03T05:58:16.560 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21", 2022-09-03T05:58:16.560 INFO:teuthology.orchestra.run.smithi181.stdout: "meta.log.41ecd2a5-afc0-4eae-9349-7dfe39baa1cb.2", 2022-09-03T05:58:16.560 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:4", 2022-09-03T05:58:16.560 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000096", 2022-09-03T05:58:16.561 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:8", 2022-09-03T05:58:16.561 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:8", 2022-09-03T05:58:16.561 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11", 2022-09-03T05:58:16.561 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000087", 2022-09-03T05:58:16.562 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:7", 2022-09-03T05:58:16.562 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:10", 2022-09-03T05:58:16.562 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000008", 2022-09-03T05:58:16.563 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:7", 2022-09-03T05:58:16.563 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:5", 2022-09-03T05:58:16.563 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:2", 2022-09-03T05:58:16.563 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22", 2022-09-03T05:58:16.564 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:10", 2022-09-03T05:58:16.564 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:1", 2022-09-03T05:58:16.564 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:1", 2022-09-03T05:58:16.564 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000006", 2022-09-03T05:58:16.565 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000029", 2022-09-03T05:58:16.565 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:9", 2022-09-03T05:58:16.565 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:8", 2022-09-03T05:58:16.566 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:7", 2022-09-03T05:58:16.566 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:9", 2022-09-03T05:58:16.566 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:4", 2022-09-03T05:58:16.567 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:58:16.567 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:9", 2022-09-03T05:58:16.567 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000089", 2022-09-03T05:58:16.567 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000022", 2022-09-03T05:58:16.568 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4", 2022-09-03T05:58:16.568 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:0", 2022-09-03T05:58:16.568 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:1", 2022-09-03T05:58:16.568 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:4", 2022-09-03T05:58:16.569 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:5", 2022-09-03T05:58:16.569 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:10", 2022-09-03T05:58:16.569 INFO:teuthology.orchestra.run.smithi181.stdout: "bilog.trim", 2022-09-03T05:58:16.570 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:3", 2022-09-03T05:58:16.570 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000016", 2022-09-03T05:58:16.570 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:10", 2022-09-03T05:58:16.570 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23", 2022-09-03T05:58:16.571 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:1", 2022-09-03T05:58:16.571 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:2", 2022-09-03T05:58:16.571 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000041", 2022-09-03T05:58:16.571 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:9", 2022-09-03T05:58:16.572 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:6", 2022-09-03T05:58:16.572 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:6", 2022-09-03T05:58:16.572 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:5", 2022-09-03T05:58:16.572 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.e40d88ba-d542-483a-afa2-5de687affd14.3.retry", 2022-09-03T05:58:16.573 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000018", 2022-09-03T05:58:16.573 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000092", 2022-09-03T05:58:16.573 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:3", 2022-09-03T05:58:16.574 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:0", 2022-09-03T05:58:16.574 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:10", 2022-09-03T05:58:16.575 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.4", 2022-09-03T05:58:16.575 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:9", 2022-09-03T05:58:16.576 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:5", 2022-09-03T05:58:16.576 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:2", 2022-09-03T05:58:16.576 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:8", 2022-09-03T05:58:16.577 INFO:teuthology.orchestra.run.smithi181.stdout: "data_log.0.2wqVC/uXbjy.0", 2022-09-03T05:58:16.577 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:7", 2022-09-03T05:58:16.577 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:7", 2022-09-03T05:58:16.577 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:8", 2022-09-03T05:58:16.578 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:7", 2022-09-03T05:58:16.578 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:2", 2022-09-03T05:58:16.578 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000014", 2022-09-03T05:58:16.578 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:6", 2022-09-03T05:58:16.579 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13", 2022-09-03T05:58:16.579 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:7", 2022-09-03T05:58:16.579 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:1", 2022-09-03T05:58:16.579 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:10", 2022-09-03T05:58:16.580 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:0", 2022-09-03T05:58:16.580 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:8", 2022-09-03T05:58:16.580 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000112", 2022-09-03T05:58:16.581 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:3", 2022-09-03T05:58:16.581 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000007", 2022-09-03T05:58:16.581 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000021", 2022-09-03T05:58:16.581 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.16", 2022-09-03T05:58:16.582 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:4", 2022-09-03T05:58:16.582 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000064", 2022-09-03T05:58:16.582 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:2", 2022-09-03T05:58:16.582 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:2", 2022-09-03T05:58:16.583 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:5", 2022-09-03T05:58:16.583 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000071", 2022-09-03T05:58:16.583 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.e40d88ba-d542-483a-afa2-5de687affd14.2.retry", 2022-09-03T05:58:16.583 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:1", 2022-09-03T05:58:16.584 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:7", 2022-09-03T05:58:16.584 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:3", 2022-09-03T05:58:16.584 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000074", 2022-09-03T05:58:16.584 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:9", 2022-09-03T05:58:16.585 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:1", 2022-09-03T05:58:16.585 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12", 2022-09-03T05:58:16.585 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10", 2022-09-03T05:58:16.585 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:6", 2022-09-03T05:58:16.586 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:0", 2022-09-03T05:58:16.586 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:6", 2022-09-03T05:58:16.586 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:10", 2022-09-03T05:58:16.587 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:3", 2022-09-03T05:58:16.587 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:9", 2022-09-03T05:58:16.587 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:6", 2022-09-03T05:58:16.587 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000081", 2022-09-03T05:58:16.588 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000009", 2022-09-03T05:58:16.588 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:2", 2022-09-03T05:58:16.589 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000121", 2022-09-03T05:58:16.589 INFO:teuthology.orchestra.run.smithi181.stdout: "data_log.1", 2022-09-03T05:58:16.589 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:10", 2022-09-03T05:58:16.589 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:3", 2022-09-03T05:58:16.590 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:5", 2022-09-03T05:58:16.590 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:8", 2022-09-03T05:58:16.590 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:9", 2022-09-03T05:58:16.590 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.17", 2022-09-03T05:58:16.591 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000125", 2022-09-03T05:58:16.591 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:0", 2022-09-03T05:58:16.591 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:0", 2022-09-03T05:58:16.591 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:4", 2022-09-03T05:58:16.592 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000082", 2022-09-03T05:58:16.592 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15", 2022-09-03T05:58:16.592 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:2", 2022-09-03T05:58:16.592 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:7", 2022-09-03T05:58:16.593 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:6", 2022-09-03T05:58:16.593 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:2", 2022-09-03T05:58:16.593 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:6", 2022-09-03T05:58:16.593 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000105", 2022-09-03T05:58:16.594 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000059", 2022-09-03T05:58:16.594 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.e40d88ba-d542-483a-afa2-5de687affd14.2", 2022-09-03T05:58:16.594 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:8", 2022-09-03T05:58:16.595 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:3", 2022-09-03T05:58:16.595 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:6", 2022-09-03T05:58:16.595 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:1", 2022-09-03T05:58:16.595 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:0", 2022-09-03T05:58:16.596 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:8", 2022-09-03T05:58:16.596 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000077", 2022-09-03T05:58:16.596 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:9", 2022-09-03T05:58:16.596 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:5", 2022-09-03T05:58:16.597 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:7", 2022-09-03T05:58:16.597 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000032", 2022-09-03T05:58:16.597 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:9", 2022-09-03T05:58:16.598 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:2", 2022-09-03T05:58:16.598 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:2", 2022-09-03T05:58:16.599 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:7", 2022-09-03T05:58:16.599 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:1", 2022-09-03T05:58:16.599 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:6", 2022-09-03T05:58:16.599 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:7", 2022-09-03T05:58:16.600 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:4", 2022-09-03T05:58:16.600 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12", 2022-09-03T05:58:16.600 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:8", 2022-09-03T05:58:16.601 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:0", 2022-09-03T05:58:16.601 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:10", 2022-09-03T05:58:16.601 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:10", 2022-09-03T05:58:16.601 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:6", 2022-09-03T05:58:16.602 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:1", 2022-09-03T05:58:16.602 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:4", 2022-09-03T05:58:16.602 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:4", 2022-09-03T05:58:16.603 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:7", 2022-09-03T05:58:16.603 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000053", 2022-09-03T05:58:16.603 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.91786ef1-cdb4-4061-8a3e-2a28cc497285.2", 2022-09-03T05:58:16.603 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000091", 2022-09-03T05:58:16.604 INFO:teuthology.orchestra.run.smithi181.stdout: "data_log.3.2svPGiyzVYF.0", 2022-09-03T05:58:16.604 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:9", 2022-09-03T05:58:16.604 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:8", 2022-09-03T05:58:16.604 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000065", 2022-09-03T05:58:16.605 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:7", 2022-09-03T05:58:16.605 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:6", 2022-09-03T05:58:16.605 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:3", 2022-09-03T05:58:16.605 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000083", 2022-09-03T05:58:16.606 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000010", 2022-09-03T05:58:16.606 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:2", 2022-09-03T05:58:16.606 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:2", 2022-09-03T05:58:16.606 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:0", 2022-09-03T05:58:16.607 INFO:teuthology.orchestra.run.smithi181.stdout: "data_log.2.huOG7shF39D.0", 2022-09-03T05:58:16.607 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:3", 2022-09-03T05:58:16.607 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:7", 2022-09-03T05:58:16.607 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:3", 2022-09-03T05:58:16.608 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:8", 2022-09-03T05:58:16.608 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.7", 2022-09-03T05:58:16.608 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23", 2022-09-03T05:58:16.608 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.3", 2022-09-03T05:58:16.609 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:7", 2022-09-03T05:58:16.609 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:5", 2022-09-03T05:58:16.609 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:3", 2022-09-03T05:58:16.610 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:9", 2022-09-03T05:58:16.610 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:9", 2022-09-03T05:58:16.610 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:6", 2022-09-03T05:58:16.611 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:7", 2022-09-03T05:58:16.611 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:3", 2022-09-03T05:58:16.611 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:2", 2022-09-03T05:58:16.611 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:4", 2022-09-03T05:58:16.612 INFO:teuthology.orchestra.run.smithi181.stdout: "mdlog.sync-status.shard.2", 2022-09-03T05:58:16.612 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:2", 2022-09-03T05:58:16.612 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:4", 2022-09-03T05:58:16.612 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:4", 2022-09-03T05:58:16.613 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:5", 2022-09-03T05:58:16.613 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:5", 2022-09-03T05:58:16.613 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:7", 2022-09-03T05:58:16.614 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:2", 2022-09-03T05:58:16.614 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:1", 2022-09-03T05:58:16.614 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000045", 2022-09-03T05:58:16.614 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:3", 2022-09-03T05:58:16.615 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000002", 2022-09-03T05:58:16.615 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.12", 2022-09-03T05:58:16.615 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:10", 2022-09-03T05:58:16.616 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:2", 2022-09-03T05:58:16.616 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.91786ef1-cdb4-4061-8a3e-2a28cc497285.1.retry", 2022-09-03T05:58:16.616 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:10", 2022-09-03T05:58:16.616 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:1", 2022-09-03T05:58:16.617 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:10", 2022-09-03T05:58:16.617 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000116", 2022-09-03T05:58:16.617 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:5", 2022-09-03T05:58:16.617 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:3", 2022-09-03T05:58:16.618 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:5", 2022-09-03T05:58:16.618 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:0", 2022-09-03T05:58:16.618 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000034", 2022-09-03T05:58:16.619 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:5", 2022-09-03T05:58:16.619 INFO:teuthology.orchestra.run.smithi181.stdout: "meta.log.41ecd2a5-afc0-4eae-9349-7dfe39baa1cb.0", 2022-09-03T05:58:16.619 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000101", 2022-09-03T05:58:16.619 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:8", 2022-09-03T05:58:16.620 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.13", 2022-09-03T05:58:16.620 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:5", 2022-09-03T05:58:16.620 INFO:teuthology.orchestra.run.smithi181.stdout: "mdlog.sync-status.shard.1", 2022-09-03T05:58:16.621 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:1", 2022-09-03T05:58:16.621 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:1", 2022-09-03T05:58:16.622 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000079", 2022-09-03T05:58:16.622 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000049", 2022-09-03T05:58:16.622 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000117", 2022-09-03T05:58:16.622 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:1", 2022-09-03T05:58:16.623 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000044", 2022-09-03T05:58:16.623 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:6", 2022-09-03T05:58:16.623 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:10", 2022-09-03T05:58:16.623 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:2", 2022-09-03T05:58:16.624 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:8", 2022-09-03T05:58:16.624 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:0", 2022-09-03T05:58:16.624 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:4", 2022-09-03T05:58:16.625 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.91786ef1-cdb4-4061-8a3e-2a28cc497285.0.retry", 2022-09-03T05:58:16.625 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:8", 2022-09-03T05:58:16.625 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:0", 2022-09-03T05:58:16.625 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.14", 2022-09-03T05:58:16.626 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:9", 2022-09-03T05:58:16.626 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000066", 2022-09-03T05:58:16.626 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:1", 2022-09-03T05:58:16.626 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.e40d88ba-d542-483a-afa2-5de687affd14.1", 2022-09-03T05:58:16.627 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:10", 2022-09-03T05:58:16.627 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:0", 2022-09-03T05:58:16.627 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000068", 2022-09-03T05:58:16.627 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:4", 2022-09-03T05:58:16.628 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:9", 2022-09-03T05:58:16.628 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:2", 2022-09-03T05:58:16.628 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000085", 2022-09-03T05:58:16.628 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:5", 2022-09-03T05:58:16.629 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:10", 2022-09-03T05:58:16.629 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:7", 2022-09-03T05:58:16.629 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:4", 2022-09-03T05:58:16.630 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:9", 2022-09-03T05:58:16.630 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000073", 2022-09-03T05:58:16.630 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:4", 2022-09-03T05:58:16.630 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000038", 2022-09-03T05:58:16.631 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:9", 2022-09-03T05:58:16.631 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:7", 2022-09-03T05:58:16.631 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.e40d88ba-d542-483a-afa2-5de687affd14.0", 2022-09-03T05:58:16.632 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:7", 2022-09-03T05:58:16.632 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:8", 2022-09-03T05:58:16.632 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.10", 2022-09-03T05:58:16.632 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:5", 2022-09-03T05:58:16.633 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000118", 2022-09-03T05:58:16.633 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.91786ef1-cdb4-4061-8a3e-2a28cc497285.0", 2022-09-03T05:58:16.633 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:8", 2022-09-03T05:58:16.633 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:1", 2022-09-03T05:58:16.634 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:8", 2022-09-03T05:58:16.634 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.e40d88ba-d542-483a-afa2-5de687affd14.3", 2022-09-03T05:58:16.634 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000036", 2022-09-03T05:58:16.635 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:3", 2022-09-03T05:58:16.635 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:8", 2022-09-03T05:58:16.635 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000103", 2022-09-03T05:58:16.635 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:10", 2022-09-03T05:58:16.636 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:7", 2022-09-03T05:58:16.636 INFO:teuthology.orchestra.run.smithi181.stdout: "mdlog.sync-status.shard.3", 2022-09-03T05:58:16.636 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:3", 2022-09-03T05:58:16.636 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1", 2022-09-03T05:58:16.637 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:9", 2022-09-03T05:58:16.637 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:6", 2022-09-03T05:58:16.637 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:10", 2022-09-03T05:58:16.637 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000119", 2022-09-03T05:58:16.638 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:6", 2022-09-03T05:58:16.638 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000098", 2022-09-03T05:58:16.638 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:9", 2022-09-03T05:58:16.639 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:6", 2022-09-03T05:58:16.639 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:3", 2022-09-03T05:58:16.639 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:9", 2022-09-03T05:58:16.639 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:3", 2022-09-03T05:58:16.640 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:7", 2022-09-03T05:58:16.640 INFO:teuthology.orchestra.run.smithi181.stdout: "mdlog.sync-status.shard.0", 2022-09-03T05:58:16.640 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:10", 2022-09-03T05:58:16.640 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:5", 2022-09-03T05:58:16.641 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000027", 2022-09-03T05:58:16.641 INFO:teuthology.orchestra.run.smithi181.stdout: "mdlog.sync-status", 2022-09-03T05:58:16.642 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000019", 2022-09-03T05:58:16.642 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:2", 2022-09-03T05:58:16.642 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:8", 2022-09-03T05:58:16.642 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000039", 2022-09-03T05:58:16.643 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:4", 2022-09-03T05:58:16.643 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:0", 2022-09-03T05:58:16.643 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15", 2022-09-03T05:58:16.643 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:5", 2022-09-03T05:58:16.644 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:5", 2022-09-03T05:58:16.644 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11", 2022-09-03T05:58:16.644 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:3", 2022-09-03T05:58:16.644 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:0", 2022-09-03T05:58:16.645 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:6", 2022-09-03T05:58:16.645 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:8", 2022-09-03T05:58:16.645 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000100", 2022-09-03T05:58:16.645 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:7", 2022-09-03T05:58:16.646 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:5", 2022-09-03T05:58:16.646 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:9", 2022-09-03T05:58:16.646 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10", 2022-09-03T05:58:16.647 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:0", 2022-09-03T05:58:16.647 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:7", 2022-09-03T05:58:16.647 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:7", 2022-09-03T05:58:16.648 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:1", 2022-09-03T05:58:16.648 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000093", 2022-09-03T05:58:16.648 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:4", 2022-09-03T05:58:16.648 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:6", 2022-09-03T05:58:16.649 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:7", 2022-09-03T05:58:16.649 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:8", 2022-09-03T05:58:16.649 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:10", 2022-09-03T05:58:16.649 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:2", 2022-09-03T05:58:16.650 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:6", 2022-09-03T05:58:16.650 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000004", 2022-09-03T05:58:16.650 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000063", 2022-09-03T05:58:16.650 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:2", 2022-09-03T05:58:16.651 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:1", 2022-09-03T05:58:16.651 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000122", 2022-09-03T05:58:16.651 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:10", 2022-09-03T05:58:16.651 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:10", 2022-09-03T05:58:16.652 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.91786ef1-cdb4-4061-8a3e-2a28cc497285.3", 2022-09-03T05:58:16.652 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000057", 2022-09-03T05:58:16.652 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:3", 2022-09-03T05:58:16.652 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000054", 2022-09-03T05:58:16.653 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000114", 2022-09-03T05:58:16.653 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:2", 2022-09-03T05:58:16.653 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:4", 2022-09-03T05:58:16.653 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:2", 2022-09-03T05:58:16.654 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.2", 2022-09-03T05:58:16.654 INFO:teuthology.orchestra.run.smithi181.stdout: "data_log.0", 2022-09-03T05:58:16.654 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:5", 2022-09-03T05:58:16.655 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:3", 2022-09-03T05:58:16.655 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:5", 2022-09-03T05:58:16.655 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:5", 2022-09-03T05:58:16.655 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:8", 2022-09-03T05:58:16.656 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000012", 2022-09-03T05:58:16.656 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:3", 2022-09-03T05:58:16.656 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:0", 2022-09-03T05:58:16.656 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:3", 2022-09-03T05:58:16.657 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:8", 2022-09-03T05:58:16.657 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:2", 2022-09-03T05:58:16.657 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:10", 2022-09-03T05:58:16.658 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:8", 2022-09-03T05:58:16.658 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.e40d88ba-d542-483a-afa2-5de687affd14", 2022-09-03T05:58:16.658 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000084", 2022-09-03T05:58:16.658 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000043", 2022-09-03T05:58:16.659 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:6", 2022-09-03T05:58:16.659 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:5", 2022-09-03T05:58:16.659 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:8", 2022-09-03T05:58:16.659 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:9", 2022-09-03T05:58:16.660 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:10", 2022-09-03T05:58:16.660 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.11", 2022-09-03T05:58:16.660 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:4", 2022-09-03T05:58:16.661 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:7", 2022-09-03T05:58:16.661 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:4", 2022-09-03T05:58:16.661 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:7", 2022-09-03T05:58:16.661 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:1", 2022-09-03T05:58:16.662 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:6", 2022-09-03T05:58:16.662 INFO:teuthology.orchestra.run.smithi181.stdout: "data_log.1.OvlycksqJBl.0", 2022-09-03T05:58:16.662 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000111", 2022-09-03T05:58:16.662 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:9", 2022-09-03T05:58:16.663 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000048", 2022-09-03T05:58:16.663 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:10", 2022-09-03T05:58:16.663 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:10", 2022-09-03T05:58:16.663 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000020", 2022-09-03T05:58:16.664 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:10", 2022-09-03T05:58:16.664 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:4", 2022-09-03T05:58:16.664 INFO:teuthology.orchestra.run.smithi181.stdout: "meta.log.eff2cb82-c479-4631-8568-fb1af4159e02.2", 2022-09-03T05:58:16.664 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:1", 2022-09-03T05:58:16.665 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:2", 2022-09-03T05:58:16.665 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:2", 2022-09-03T05:58:16.665 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:10", 2022-09-03T05:58:16.666 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:8", 2022-09-03T05:58:16.666 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000099", 2022-09-03T05:58:16.666 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000056", 2022-09-03T05:58:16.666 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:6", 2022-09-03T05:58:16.667 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:7", 2022-09-03T05:58:16.667 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:4", 2022-09-03T05:58:16.667 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:0", 2022-09-03T05:58:16.668 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:0", 2022-09-03T05:58:16.668 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000072", 2022-09-03T05:58:16.668 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:10", 2022-09-03T05:58:16.668 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:5", 2022-09-03T05:58:16.669 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17", 2022-09-03T05:58:16.669 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:6", 2022-09-03T05:58:16.669 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000062", 2022-09-03T05:58:16.669 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:58:16.670 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3", 2022-09-03T05:58:16.670 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000115", 2022-09-03T05:58:16.670 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:4", 2022-09-03T05:58:16.670 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.1", 2022-09-03T05:58:16.671 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22", 2022-09-03T05:58:16.671 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000033", 2022-09-03T05:58:16.671 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:0", 2022-09-03T05:58:16.671 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:10", 2022-09-03T05:58:16.672 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:1", 2022-09-03T05:58:16.672 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:58:16.672 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:3", 2022-09-03T05:58:16.673 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:1", 2022-09-03T05:58:16.673 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:1", 2022-09-03T05:58:16.673 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:3", 2022-09-03T05:58:16.673 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3", 2022-09-03T05:58:16.674 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:5", 2022-09-03T05:58:16.674 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:1", 2022-09-03T05:58:16.674 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000025", 2022-09-03T05:58:16.675 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:8", 2022-09-03T05:58:16.675 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:3", 2022-09-03T05:58:16.676 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000123", 2022-09-03T05:58:16.676 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:1", 2022-09-03T05:58:16.676 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:3", 2022-09-03T05:58:16.676 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:7", 2022-09-03T05:58:16.677 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:8", 2022-09-03T05:58:16.677 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:2", 2022-09-03T05:58:16.677 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:1", 2022-09-03T05:58:16.677 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:1", 2022-09-03T05:58:16.678 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16", 2022-09-03T05:58:16.678 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4", 2022-09-03T05:58:16.678 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:2", 2022-09-03T05:58:16.678 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:3", 2022-09-03T05:58:16.679 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:9", 2022-09-03T05:58:16.679 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:4", 2022-09-03T05:58:16.679 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000108", 2022-09-03T05:58:16.680 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:9", 2022-09-03T05:58:16.680 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:7", 2022-09-03T05:58:16.680 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:1", 2022-09-03T05:58:16.680 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:9", 2022-09-03T05:58:16.681 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000094", 2022-09-03T05:58:16.681 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:5", 2022-09-03T05:58:16.681 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16", 2022-09-03T05:58:16.681 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:2", 2022-09-03T05:58:16.682 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18", 2022-09-03T05:58:16.682 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:7", 2022-09-03T05:58:16.682 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38", 2022-09-03T05:58:16.683 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000060", 2022-09-03T05:58:16.683 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.91786ef1-cdb4-4061-8a3e-2a28cc497285.2.retry", 2022-09-03T05:58:16.683 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000109", 2022-09-03T05:58:16.683 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.15", 2022-09-03T05:58:16.684 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:0", 2022-09-03T05:58:16.684 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:5", 2022-09-03T05:58:16.684 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000013", 2022-09-03T05:58:16.684 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:1", 2022-09-03T05:58:16.685 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:0", 2022-09-03T05:58:16.685 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:1", 2022-09-03T05:58:16.685 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:8", 2022-09-03T05:58:16.685 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:9", 2022-09-03T05:58:16.686 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19", 2022-09-03T05:58:16.686 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:6", 2022-09-03T05:58:16.686 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:9", 2022-09-03T05:58:16.686 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:4", 2022-09-03T05:58:16.687 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:8", 2022-09-03T05:58:16.687 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:6", 2022-09-03T05:58:16.687 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:9", 2022-09-03T05:58:16.687 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:4", 2022-09-03T05:58:16.688 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:58:16.688 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:7", 2022-09-03T05:58:16.688 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:0", 2022-09-03T05:58:16.688 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:4", 2022-09-03T05:58:16.689 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000051", 2022-09-03T05:58:16.689 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:8", 2022-09-03T05:58:16.689 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000106", 2022-09-03T05:58:16.689 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:1", 2022-09-03T05:58:16.690 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:2", 2022-09-03T05:58:16.690 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:9", 2022-09-03T05:58:16.690 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:3", 2022-09-03T05:58:16.690 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:10", 2022-09-03T05:58:16.691 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:0", 2022-09-03T05:58:16.691 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:7", 2022-09-03T05:58:16.691 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:1", 2022-09-03T05:58:16.691 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:6", 2022-09-03T05:58:16.692 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000017", 2022-09-03T05:58:16.692 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:8", 2022-09-03T05:58:16.692 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:1", 2022-09-03T05:58:16.692 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:4", 2022-09-03T05:58:16.693 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:0", 2022-09-03T05:58:16.693 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:4", 2022-09-03T05:58:16.693 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:8", 2022-09-03T05:58:16.694 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:4", 2022-09-03T05:58:16.694 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:1", 2022-09-03T05:58:16.694 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000023", 2022-09-03T05:58:16.694 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:9", 2022-09-03T05:58:16.695 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:0", 2022-09-03T05:58:16.695 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:6", 2022-09-03T05:58:16.695 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:3", 2022-09-03T05:58:16.695 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:8", 2022-09-03T05:58:16.696 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:4", 2022-09-03T05:58:16.696 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000107", 2022-09-03T05:58:16.696 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:0", 2022-09-03T05:58:16.696 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:10", 2022-09-03T05:58:16.697 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:5", 2022-09-03T05:58:16.697 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000046", 2022-09-03T05:58:16.697 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:10", 2022-09-03T05:58:16.697 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:6", 2022-09-03T05:58:16.698 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:5", 2022-09-03T05:58:16.698 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:1", 2022-09-03T05:58:16.698 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:6", 2022-09-03T05:58:16.699 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:2", 2022-09-03T05:58:16.699 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:10", 2022-09-03T05:58:16.699 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:7", 2022-09-03T05:58:16.699 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:7", 2022-09-03T05:58:16.700 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:2", 2022-09-03T05:58:16.700 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:6", 2022-09-03T05:58:16.700 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:4", 2022-09-03T05:58:16.700 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:7", 2022-09-03T05:58:16.701 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:0", 2022-09-03T05:58:16.701 INFO:teuthology.orchestra.run.smithi181.stdout: "data_log.2", 2022-09-03T05:58:16.701 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:6", 2022-09-03T05:58:16.701 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:10", 2022-09-03T05:58:16.702 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000067", 2022-09-03T05:58:16.702 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:8", 2022-09-03T05:58:16.702 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:8", 2022-09-03T05:58:16.703 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:2", 2022-09-03T05:58:16.703 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:58:16.703 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38", 2022-09-03T05:58:16.703 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:10", 2022-09-03T05:58:16.704 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:8", 2022-09-03T05:58:16.704 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:6", 2022-09-03T05:58:16.704 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:0", 2022-09-03T05:58:16.704 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:0", 2022-09-03T05:58:16.705 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:6", 2022-09-03T05:58:16.705 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:10", 2022-09-03T05:58:16.705 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:1", 2022-09-03T05:58:16.705 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.9", 2022-09-03T05:58:16.706 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000086", 2022-09-03T05:58:16.706 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:2", 2022-09-03T05:58:16.706 INFO:teuthology.orchestra.run.smithi181.stdout: "sync.error-log.5", 2022-09-03T05:58:16.706 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:2", 2022-09-03T05:58:16.707 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:3", 2022-09-03T05:58:16.707 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:1", 2022-09-03T05:58:16.707 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:9", 2022-09-03T05:58:16.707 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:58:16.708 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:3", 2022-09-03T05:58:16.708 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:2", 2022-09-03T05:58:16.708 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:0", 2022-09-03T05:58:16.708 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:2", 2022-09-03T05:58:16.709 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:5", 2022-09-03T05:58:16.709 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:4", 2022-09-03T05:58:16.709 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.e40d88ba-d542-483a-afa2-5de687affd14.1.retry", 2022-09-03T05:58:16.709 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:2", 2022-09-03T05:58:16.710 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:9", 2022-09-03T05:58:16.710 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:1", 2022-09-03T05:58:16.710 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17", 2022-09-03T05:58:16.711 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:9", 2022-09-03T05:58:16.711 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:6", 2022-09-03T05:58:16.711 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:9", 2022-09-03T05:58:16.711 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:7", 2022-09-03T05:58:16.712 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000050", 2022-09-03T05:58:16.712 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000102", 2022-09-03T05:58:16.712 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:2", 2022-09-03T05:58:16.712 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:4", 2022-09-03T05:58:16.713 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:0", 2022-09-03T05:58:16.713 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36", 2022-09-03T05:58:16.713 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:7", 2022-09-03T05:58:16.714 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:8", 2022-09-03T05:58:16.714 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:3", 2022-09-03T05:58:16.714 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:6", 2022-09-03T05:58:16.714 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000124", 2022-09-03T05:58:16.715 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:9", 2022-09-03T05:58:16.715 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000001", 2022-09-03T05:58:16.715 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:4", 2022-09-03T05:58:16.716 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:6", 2022-09-03T05:58:16.716 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19", 2022-09-03T05:58:16.716 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:0", 2022-09-03T05:58:16.716 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000126", 2022-09-03T05:58:16.717 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:5", 2022-09-03T05:58:16.717 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000055", 2022-09-03T05:58:16.717 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:2", 2022-09-03T05:58:16.717 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:0", 2022-09-03T05:58:16.718 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000061", 2022-09-03T05:58:16.718 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:6", 2022-09-03T05:58:16.718 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:6", 2022-09-03T05:58:16.719 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:2", 2022-09-03T05:58:16.719 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13", 2022-09-03T05:58:16.719 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:2", 2022-09-03T05:58:16.719 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000097", 2022-09-03T05:58:16.720 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:7", 2022-09-03T05:58:16.720 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:9", 2022-09-03T05:58:16.720 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:6", 2022-09-03T05:58:16.720 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:4", 2022-09-03T05:58:16.721 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:8", 2022-09-03T05:58:16.721 INFO:teuthology.orchestra.run.smithi181.stdout: "datalog.sync-status.shard.e40d88ba-d542-483a-afa2-5de687affd14.0.retry", 2022-09-03T05:58:16.721 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:4", 2022-09-03T05:58:16.722 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000042", 2022-09-03T05:58:16.722 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:2", 2022-09-03T05:58:16.722 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14", 2022-09-03T05:58:16.723 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:3", 2022-09-03T05:58:16.723 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:3", 2022-09-03T05:58:16.723 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:2", 2022-09-03T05:58:16.724 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1", 2022-09-03T05:58:16.724 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000075", 2022-09-03T05:58:16.724 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:3", 2022-09-03T05:58:16.724 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000120", 2022-09-03T05:58:16.725 INFO:teuthology.orchestra.run.smithi181.stdout: "obj_delete_at_hint.0000000030", 2022-09-03T05:58:16.725 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:0", 2022-09-03T05:58:16.725 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:9", 2022-09-03T05:58:16.725 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:9", 2022-09-03T05:58:16.726 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:1", 2022-09-03T05:58:16.726 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:3", 2022-09-03T05:58:16.726 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:7", 2022-09-03T05:58:16.727 INFO:teuthology.orchestra.run.smithi181.stdout: "bucket.sync-status.e40d88ba-d542-483a-afa2-5de687affd14:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:3" 2022-09-03T05:58:16.727 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:58:16.728 INFO:teuthology.orchestra.run.smithi181.stdout: 2022-09-03T05:58:16.730 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['log', 'list', '--rgw-zone', 'test-zone4', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:16.731 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'log', 'list', '--rgw-zone', 'test-zone4', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:16.732 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 log list --rgw-zone test-zone4 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:16.785 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:16.786 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:16.843 INFO:teuthology.orchestra.run.smithi189.stdout:[ 2022-09-03T05:58:16.843 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000078", 2022-09-03T05:58:16.844 INFO:teuthology.orchestra.run.smithi189.stdout: "meta.history", 2022-09-03T05:58:16.844 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:3", 2022-09-03T05:58:16.844 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:5", 2022-09-03T05:58:16.844 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000070", 2022-09-03T05:58:16.845 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000104", 2022-09-03T05:58:16.845 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:9", 2022-09-03T05:58:16.845 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:6", 2022-09-03T05:58:16.845 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:8", 2022-09-03T05:58:16.846 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000026", 2022-09-03T05:58:16.846 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:5", 2022-09-03T05:58:16.846 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:8", 2022-09-03T05:58:16.846 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.8", 2022-09-03T05:58:16.847 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:4", 2022-09-03T05:58:16.847 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:0", 2022-09-03T05:58:16.848 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000028", 2022-09-03T05:58:16.848 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:10", 2022-09-03T05:58:16.849 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:7", 2022-09-03T05:58:16.849 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:9", 2022-09-03T05:58:16.850 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000040", 2022-09-03T05:58:16.850 INFO:teuthology.orchestra.run.smithi189.stdout: "meta.log.41ecd2a5-afc0-4eae-9349-7dfe39baa1cb.3", 2022-09-03T05:58:16.851 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:6", 2022-09-03T05:58:16.851 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:10", 2022-09-03T05:58:16.851 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.6", 2022-09-03T05:58:16.852 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000015", 2022-09-03T05:58:16.852 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37", 2022-09-03T05:58:16.853 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:1", 2022-09-03T05:58:16.853 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000069", 2022-09-03T05:58:16.853 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000095", 2022-09-03T05:58:16.854 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000003", 2022-09-03T05:58:16.854 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000047", 2022-09-03T05:58:16.854 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:7", 2022-09-03T05:58:16.855 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:6", 2022-09-03T05:58:16.855 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000035", 2022-09-03T05:58:16.855 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:1", 2022-09-03T05:58:16.856 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000037", 2022-09-03T05:58:16.856 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:1", 2022-09-03T05:58:16.856 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:10", 2022-09-03T05:58:16.856 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:3", 2022-09-03T05:58:16.857 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:1", 2022-09-03T05:58:16.857 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:6", 2022-09-03T05:58:16.857 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:3", 2022-09-03T05:58:16.857 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:9", 2022-09-03T05:58:16.858 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5", 2022-09-03T05:58:16.858 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000024", 2022-09-03T05:58:16.858 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:5", 2022-09-03T05:58:16.858 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000000", 2022-09-03T05:58:16.859 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:5", 2022-09-03T05:58:16.859 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000031", 2022-09-03T05:58:16.859 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:5", 2022-09-03T05:58:16.860 INFO:teuthology.orchestra.run.smithi189.stdout: "meta.log.229c309b-f405-4ead-a3a8-baa670c6da51.2", 2022-09-03T05:58:16.860 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:10", 2022-09-03T05:58:16.860 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:3", 2022-09-03T05:58:16.860 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:10", 2022-09-03T05:58:16.861 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26", 2022-09-03T05:58:16.861 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:7", 2022-09-03T05:58:16.861 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.shard.91786ef1-cdb4-4061-8a3e-2a28cc497285.1", 2022-09-03T05:58:16.861 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:6", 2022-09-03T05:58:16.862 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:5", 2022-09-03T05:58:16.862 INFO:teuthology.orchestra.run.smithi189.stdout: "data_loggenerations_metadata", 2022-09-03T05:58:16.862 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:5", 2022-09-03T05:58:16.862 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:10", 2022-09-03T05:58:16.863 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000076", 2022-09-03T05:58:16.863 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:10", 2022-09-03T05:58:16.863 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000113", 2022-09-03T05:58:16.863 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000005", 2022-09-03T05:58:16.863 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000011", 2022-09-03T05:58:16.864 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:0", 2022-09-03T05:58:16.864 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000058", 2022-09-03T05:58:16.864 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000052", 2022-09-03T05:58:16.864 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:4", 2022-09-03T05:58:16.865 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:0", 2022-09-03T05:58:16.865 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:6", 2022-09-03T05:58:16.865 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:5", 2022-09-03T05:58:16.865 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:9", 2022-09-03T05:58:16.866 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:3", 2022-09-03T05:58:16.866 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:9", 2022-09-03T05:58:16.866 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:3", 2022-09-03T05:58:16.866 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:4", 2022-09-03T05:58:16.867 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:0", 2022-09-03T05:58:16.867 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:10", 2022-09-03T05:58:16.867 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000088", 2022-09-03T05:58:16.867 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000080", 2022-09-03T05:58:16.868 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000090", 2022-09-03T05:58:16.868 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:8", 2022-09-03T05:58:16.868 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000110", 2022-09-03T05:58:16.868 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21", 2022-09-03T05:58:16.869 INFO:teuthology.orchestra.run.smithi189.stdout: "meta.log.41ecd2a5-afc0-4eae-9349-7dfe39baa1cb.2", 2022-09-03T05:58:16.869 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:4", 2022-09-03T05:58:16.869 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000096", 2022-09-03T05:58:16.869 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:8", 2022-09-03T05:58:16.870 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:8", 2022-09-03T05:58:16.870 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11", 2022-09-03T05:58:16.870 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000087", 2022-09-03T05:58:16.870 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000008", 2022-09-03T05:58:16.871 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:5", 2022-09-03T05:58:16.871 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:2", 2022-09-03T05:58:16.871 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22", 2022-09-03T05:58:16.871 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:1", 2022-09-03T05:58:16.872 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:1", 2022-09-03T05:58:16.872 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000006", 2022-09-03T05:58:16.872 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000029", 2022-09-03T05:58:16.872 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:9", 2022-09-03T05:58:16.873 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:4", 2022-09-03T05:58:16.873 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25", 2022-09-03T05:58:16.873 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000089", 2022-09-03T05:58:16.873 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000022", 2022-09-03T05:58:16.874 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:0", 2022-09-03T05:58:16.874 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:1", 2022-09-03T05:58:16.874 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:5", 2022-09-03T05:58:16.874 INFO:teuthology.orchestra.run.smithi189.stdout: "bilog.trim", 2022-09-03T05:58:16.875 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:3", 2022-09-03T05:58:16.875 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000016", 2022-09-03T05:58:16.875 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:10", 2022-09-03T05:58:16.875 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23", 2022-09-03T05:58:16.876 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:1", 2022-09-03T05:58:16.876 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:2", 2022-09-03T05:58:16.876 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000041", 2022-09-03T05:58:16.876 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:6", 2022-09-03T05:58:16.877 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:6", 2022-09-03T05:58:16.877 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:5", 2022-09-03T05:58:16.877 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000018", 2022-09-03T05:58:16.877 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000092", 2022-09-03T05:58:16.878 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:3", 2022-09-03T05:58:16.878 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:0", 2022-09-03T05:58:16.878 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.4", 2022-09-03T05:58:16.878 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:2", 2022-09-03T05:58:16.879 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:8", 2022-09-03T05:58:16.879 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:7", 2022-09-03T05:58:16.879 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:7", 2022-09-03T05:58:16.879 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:2", 2022-09-03T05:58:16.880 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000014", 2022-09-03T05:58:16.880 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13", 2022-09-03T05:58:16.880 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:1", 2022-09-03T05:58:16.880 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:10", 2022-09-03T05:58:16.881 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:8", 2022-09-03T05:58:16.881 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000112", 2022-09-03T05:58:16.881 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:3", 2022-09-03T05:58:16.881 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000007", 2022-09-03T05:58:16.882 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000021", 2022-09-03T05:58:16.882 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:4", 2022-09-03T05:58:16.882 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000064", 2022-09-03T05:58:16.882 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:2", 2022-09-03T05:58:16.883 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000071", 2022-09-03T05:58:16.883 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:1", 2022-09-03T05:58:16.883 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:7", 2022-09-03T05:58:16.884 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000074", 2022-09-03T05:58:16.884 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:1", 2022-09-03T05:58:16.884 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:6", 2022-09-03T05:58:16.884 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:6", 2022-09-03T05:58:16.885 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:9", 2022-09-03T05:58:16.885 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000081", 2022-09-03T05:58:16.885 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000009", 2022-09-03T05:58:16.886 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:2", 2022-09-03T05:58:16.886 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000121", 2022-09-03T05:58:16.886 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:10", 2022-09-03T05:58:16.886 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:3", 2022-09-03T05:58:16.887 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:8", 2022-09-03T05:58:16.887 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000125", 2022-09-03T05:58:16.887 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:0", 2022-09-03T05:58:16.887 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:0", 2022-09-03T05:58:16.888 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000082", 2022-09-03T05:58:16.888 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15", 2022-09-03T05:58:16.888 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:2", 2022-09-03T05:58:16.888 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:7", 2022-09-03T05:58:16.889 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:6", 2022-09-03T05:58:16.889 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000105", 2022-09-03T05:58:16.889 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000059", 2022-09-03T05:58:16.889 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:8", 2022-09-03T05:58:16.890 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:6", 2022-09-03T05:58:16.890 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:1", 2022-09-03T05:58:16.890 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:0", 2022-09-03T05:58:16.890 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:8", 2022-09-03T05:58:16.891 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000077", 2022-09-03T05:58:16.891 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:5", 2022-09-03T05:58:16.891 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:7", 2022-09-03T05:58:16.891 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000032", 2022-09-03T05:58:16.891 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:9", 2022-09-03T05:58:16.892 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:2", 2022-09-03T05:58:16.892 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:2", 2022-09-03T05:58:16.892 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:7", 2022-09-03T05:58:16.893 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:1", 2022-09-03T05:58:16.893 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:4", 2022-09-03T05:58:16.893 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12", 2022-09-03T05:58:16.893 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:8", 2022-09-03T05:58:16.894 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:10", 2022-09-03T05:58:16.894 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:6", 2022-09-03T05:58:16.894 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:1", 2022-09-03T05:58:16.894 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:4", 2022-09-03T05:58:16.895 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:4", 2022-09-03T05:58:16.895 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000053", 2022-09-03T05:58:16.895 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.shard.91786ef1-cdb4-4061-8a3e-2a28cc497285.2", 2022-09-03T05:58:16.895 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000091", 2022-09-03T05:58:16.896 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:9", 2022-09-03T05:58:16.896 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000065", 2022-09-03T05:58:16.896 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:7", 2022-09-03T05:58:16.896 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000083", 2022-09-03T05:58:16.897 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000010", 2022-09-03T05:58:16.897 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:2", 2022-09-03T05:58:16.897 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:2", 2022-09-03T05:58:16.898 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:7", 2022-09-03T05:58:16.898 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:3", 2022-09-03T05:58:16.898 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.7", 2022-09-03T05:58:16.899 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.3", 2022-09-03T05:58:16.899 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:9", 2022-09-03T05:58:16.899 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:7", 2022-09-03T05:58:16.899 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:3", 2022-09-03T05:58:16.900 INFO:teuthology.orchestra.run.smithi189.stdout: "mdlog.sync-status.shard.2", 2022-09-03T05:58:16.900 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:4", 2022-09-03T05:58:16.900 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:4", 2022-09-03T05:58:16.900 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:5", 2022-09-03T05:58:16.901 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:5", 2022-09-03T05:58:16.901 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:2", 2022-09-03T05:58:16.901 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000045", 2022-09-03T05:58:16.901 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:3", 2022-09-03T05:58:16.901 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000002", 2022-09-03T05:58:16.902 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:2", 2022-09-03T05:58:16.902 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.shard.91786ef1-cdb4-4061-8a3e-2a28cc497285.1.retry", 2022-09-03T05:58:16.902 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:10", 2022-09-03T05:58:16.902 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:1", 2022-09-03T05:58:16.903 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:10", 2022-09-03T05:58:16.903 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000116", 2022-09-03T05:58:16.903 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:3", 2022-09-03T05:58:16.903 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000034", 2022-09-03T05:58:16.904 INFO:teuthology.orchestra.run.smithi189.stdout: "meta.log.41ecd2a5-afc0-4eae-9349-7dfe39baa1cb.0", 2022-09-03T05:58:16.904 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000101", 2022-09-03T05:58:16.904 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:8", 2022-09-03T05:58:16.904 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:5", 2022-09-03T05:58:16.905 INFO:teuthology.orchestra.run.smithi189.stdout: "mdlog.sync-status.shard.1", 2022-09-03T05:58:16.905 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:1", 2022-09-03T05:58:16.905 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000079", 2022-09-03T05:58:16.905 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000049", 2022-09-03T05:58:16.906 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000117", 2022-09-03T05:58:16.906 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000044", 2022-09-03T05:58:16.906 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:10", 2022-09-03T05:58:16.906 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:0", 2022-09-03T05:58:16.907 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:4", 2022-09-03T05:58:16.907 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.shard.91786ef1-cdb4-4061-8a3e-2a28cc497285.0.retry", 2022-09-03T05:58:16.907 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:8", 2022-09-03T05:58:16.907 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:0", 2022-09-03T05:58:16.908 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000066", 2022-09-03T05:58:16.908 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:0", 2022-09-03T05:58:16.908 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000068", 2022-09-03T05:58:16.909 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:4", 2022-09-03T05:58:16.909 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:9", 2022-09-03T05:58:16.909 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000085", 2022-09-03T05:58:16.909 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:5", 2022-09-03T05:58:16.910 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:4", 2022-09-03T05:58:16.910 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:9", 2022-09-03T05:58:16.910 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000073", 2022-09-03T05:58:16.910 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:4", 2022-09-03T05:58:16.911 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000038", 2022-09-03T05:58:16.911 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:9", 2022-09-03T05:58:16.911 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:8", 2022-09-03T05:58:16.911 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.10", 2022-09-03T05:58:16.912 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000118", 2022-09-03T05:58:16.912 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.shard.91786ef1-cdb4-4061-8a3e-2a28cc497285.0", 2022-09-03T05:58:16.912 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:8", 2022-09-03T05:58:16.912 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:1", 2022-09-03T05:58:16.913 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000036", 2022-09-03T05:58:16.913 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000103", 2022-09-03T05:58:16.913 INFO:teuthology.orchestra.run.smithi189.stdout: "mdlog.sync-status.shard.3", 2022-09-03T05:58:16.913 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:3", 2022-09-03T05:58:16.914 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:9", 2022-09-03T05:58:16.914 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:6", 2022-09-03T05:58:16.914 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:10", 2022-09-03T05:58:16.914 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000119", 2022-09-03T05:58:16.915 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:6", 2022-09-03T05:58:16.915 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000098", 2022-09-03T05:58:16.915 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:9", 2022-09-03T05:58:16.915 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:3", 2022-09-03T05:58:16.916 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:3", 2022-09-03T05:58:16.916 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:7", 2022-09-03T05:58:16.916 INFO:teuthology.orchestra.run.smithi189.stdout: "mdlog.sync-status.shard.0", 2022-09-03T05:58:16.916 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:10", 2022-09-03T05:58:16.917 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000027", 2022-09-03T05:58:16.917 INFO:teuthology.orchestra.run.smithi189.stdout: "mdlog.sync-status", 2022-09-03T05:58:16.917 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000019", 2022-09-03T05:58:16.917 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:8", 2022-09-03T05:58:16.918 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000039", 2022-09-03T05:58:16.918 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:5", 2022-09-03T05:58:16.918 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:5", 2022-09-03T05:58:16.918 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:3", 2022-09-03T05:58:16.919 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:0", 2022-09-03T05:58:16.919 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000100", 2022-09-03T05:58:16.919 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:7", 2022-09-03T05:58:16.919 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:5", 2022-09-03T05:58:16.920 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10", 2022-09-03T05:58:16.920 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:0", 2022-09-03T05:58:16.920 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000093", 2022-09-03T05:58:16.921 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:7", 2022-09-03T05:58:16.921 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:8", 2022-09-03T05:58:16.921 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:6", 2022-09-03T05:58:16.921 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000004", 2022-09-03T05:58:16.922 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000063", 2022-09-03T05:58:16.922 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:2", 2022-09-03T05:58:16.922 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000122", 2022-09-03T05:58:16.922 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.shard.91786ef1-cdb4-4061-8a3e-2a28cc497285.3", 2022-09-03T05:58:16.923 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000057", 2022-09-03T05:58:16.923 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:3", 2022-09-03T05:58:16.923 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000054", 2022-09-03T05:58:16.923 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000114", 2022-09-03T05:58:16.924 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:2", 2022-09-03T05:58:16.924 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.2", 2022-09-03T05:58:16.924 INFO:teuthology.orchestra.run.smithi189.stdout: "data_log.0", 2022-09-03T05:58:16.924 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:5", 2022-09-03T05:58:16.925 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:3", 2022-09-03T05:58:16.925 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000012", 2022-09-03T05:58:16.925 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:0", 2022-09-03T05:58:16.925 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:3", 2022-09-03T05:58:16.926 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:2", 2022-09-03T05:58:16.926 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:10", 2022-09-03T05:58:16.926 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:8", 2022-09-03T05:58:16.926 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000084", 2022-09-03T05:58:16.927 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000043", 2022-09-03T05:58:16.927 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:8", 2022-09-03T05:58:16.927 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:10", 2022-09-03T05:58:16.927 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.11", 2022-09-03T05:58:16.928 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:7", 2022-09-03T05:58:16.928 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:4", 2022-09-03T05:58:16.928 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:7", 2022-09-03T05:58:16.928 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000111", 2022-09-03T05:58:16.929 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000048", 2022-09-03T05:58:16.929 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:10", 2022-09-03T05:58:16.929 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000020", 2022-09-03T05:58:16.929 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:4", 2022-09-03T05:58:16.930 INFO:teuthology.orchestra.run.smithi189.stdout: "meta.log.eff2cb82-c479-4631-8568-fb1af4159e02.2", 2022-09-03T05:58:16.930 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:1", 2022-09-03T05:58:16.930 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:10", 2022-09-03T05:58:16.930 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:8", 2022-09-03T05:58:16.931 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000099", 2022-09-03T05:58:16.931 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000056", 2022-09-03T05:58:16.931 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:6", 2022-09-03T05:58:16.931 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4:7", 2022-09-03T05:58:16.932 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000072", 2022-09-03T05:58:16.932 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:10", 2022-09-03T05:58:16.932 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:5", 2022-09-03T05:58:16.932 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17", 2022-09-03T05:58:16.933 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000062", 2022-09-03T05:58:16.933 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20", 2022-09-03T05:58:16.933 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000115", 2022-09-03T05:58:16.933 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.1", 2022-09-03T05:58:16.934 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000033", 2022-09-03T05:58:16.934 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:0", 2022-09-03T05:58:16.934 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:1", 2022-09-03T05:58:16.934 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:3", 2022-09-03T05:58:16.935 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3", 2022-09-03T05:58:16.935 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:5", 2022-09-03T05:58:16.935 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000025", 2022-09-03T05:58:16.935 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000123", 2022-09-03T05:58:16.936 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:8", 2022-09-03T05:58:16.936 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:2", 2022-09-03T05:58:16.936 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:1", 2022-09-03T05:58:16.937 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-47:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.4", 2022-09-03T05:58:16.937 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:3", 2022-09-03T05:58:16.937 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:4", 2022-09-03T05:58:16.938 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000108", 2022-09-03T05:58:16.938 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:9", 2022-09-03T05:58:16.938 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:7", 2022-09-03T05:58:16.938 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:9", 2022-09-03T05:58:16.939 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000094", 2022-09-03T05:58:16.939 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:5", 2022-09-03T05:58:16.939 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16", 2022-09-03T05:58:16.939 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:2", 2022-09-03T05:58:16.940 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18", 2022-09-03T05:58:16.940 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:7", 2022-09-03T05:58:16.940 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38", 2022-09-03T05:58:16.941 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000060", 2022-09-03T05:58:16.941 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.shard.91786ef1-cdb4-4061-8a3e-2a28cc497285.2.retry", 2022-09-03T05:58:16.941 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000109", 2022-09-03T05:58:16.941 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:0", 2022-09-03T05:58:16.942 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:5", 2022-09-03T05:58:16.942 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000013", 2022-09-03T05:58:16.942 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19", 2022-09-03T05:58:16.942 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:6", 2022-09-03T05:58:16.943 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:9", 2022-09-03T05:58:16.943 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:6", 2022-09-03T05:58:16.943 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:9", 2022-09-03T05:58:16.943 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285", 2022-09-03T05:58:16.944 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000051", 2022-09-03T05:58:16.944 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000106", 2022-09-03T05:58:16.944 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-17:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.17:7", 2022-09-03T05:58:16.944 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000017", 2022-09-03T05:58:16.945 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-21:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.21:4", 2022-09-03T05:58:16.945 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-26:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.26:4", 2022-09-03T05:58:16.945 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:8", 2022-09-03T05:58:16.945 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-16:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.16:1", 2022-09-03T05:58:16.946 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000023", 2022-09-03T05:58:16.946 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:0", 2022-09-03T05:58:16.946 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000107", 2022-09-03T05:58:16.946 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-10:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.10:0", 2022-09-03T05:58:16.947 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000046", 2022-09-03T05:58:16.947 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:10", 2022-09-03T05:58:16.947 INFO:teuthology.orchestra.run.smithi189.stdout: "datalog.sync-status.shard.91786ef1-cdb4-4061-8a3e-2a28cc497285.3.retry", 2022-09-03T05:58:16.947 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:1", 2022-09-03T05:58:16.948 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:2", 2022-09-03T05:58:16.948 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:7", 2022-09-03T05:58:16.948 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:4", 2022-09-03T05:58:16.948 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:0", 2022-09-03T05:58:16.949 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000067", 2022-09-03T05:58:16.949 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-19:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.19:8", 2022-09-03T05:58:16.949 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:10", 2022-09-03T05:58:16.949 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:6", 2022-09-03T05:58:16.950 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:0", 2022-09-03T05:58:16.950 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:6", 2022-09-03T05:58:16.950 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:1", 2022-09-03T05:58:16.950 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.9", 2022-09-03T05:58:16.951 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000086", 2022-09-03T05:58:16.951 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24:2", 2022-09-03T05:58:16.951 INFO:teuthology.orchestra.run.smithi189.stdout: "sync.error-log.5", 2022-09-03T05:58:16.951 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:1", 2022-09-03T05:58:16.952 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36:9", 2022-09-03T05:58:16.952 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-24:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.24", 2022-09-03T05:58:16.952 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-48:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.5:0", 2022-09-03T05:58:16.952 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:9", 2022-09-03T05:58:16.953 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-12:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.12:6", 2022-09-03T05:58:16.953 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:9", 2022-09-03T05:58:16.953 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:7", 2022-09-03T05:58:16.953 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000050", 2022-09-03T05:58:16.954 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000102", 2022-09-03T05:58:16.954 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:2", 2022-09-03T05:58:16.954 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:4", 2022-09-03T05:58:16.954 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-36:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.36", 2022-09-03T05:58:16.955 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14:7", 2022-09-03T05:58:16.955 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000124", 2022-09-03T05:58:16.955 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-18:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.18:9", 2022-09-03T05:58:16.955 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000001", 2022-09-03T05:58:16.956 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-23:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.23:4", 2022-09-03T05:58:16.956 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-20:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.20:6", 2022-09-03T05:58:16.956 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000126", 2022-09-03T05:58:16.956 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000055", 2022-09-03T05:58:16.956 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-13:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.13:2", 2022-09-03T05:58:16.957 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-25:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.25:0", 2022-09-03T05:58:16.957 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000061", 2022-09-03T05:58:16.957 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:6", 2022-09-03T05:58:16.957 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-37:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.37:2", 2022-09-03T05:58:16.958 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-46:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.3:2", 2022-09-03T05:58:16.958 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000097", 2022-09-03T05:58:16.958 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1:4", 2022-09-03T05:58:16.959 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-15:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.15:8", 2022-09-03T05:58:16.959 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000042", 2022-09-03T05:58:16.959 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-14:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.14", 2022-09-03T05:58:16.959 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-22:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.22:3", 2022-09-03T05:58:16.960 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.full-sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-49:91786ef1-cdb4-4061-8a3e-2a28cc497285.5435.1", 2022-09-03T05:58:16.960 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000075", 2022-09-03T05:58:16.960 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000120", 2022-09-03T05:58:16.960 INFO:teuthology.orchestra.run.smithi189.stdout: "obj_delete_at_hint.0000000030", 2022-09-03T05:58:16.961 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-11:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.11:0", 2022-09-03T05:58:16.961 INFO:teuthology.orchestra.run.smithi189.stdout: "bucket.sync-status.91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-38:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.38:9" 2022-09-03T05:58:16.961 INFO:teuthology.orchestra.run.smithi189.stdout:] 2022-09-03T05:58:16.961 INFO:teuthology.orchestra.run.smithi189.stdout: 2022-09-03T05:58:16.963 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone1 target=test-zone2 2022-09-03T05:58:16.963 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:16.964 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:16.965 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:17.015 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:17.016 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:17.087 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:58:17.087 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:17.088 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000003609", 2022-09-03T05:58:17.088 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:58:15.553095Z" 2022-09-03T05:58:17.088 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:17.089 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:17.089 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000006141", 2022-09-03T05:58:17.090 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:58:15.554651Z" 2022-09-03T05:58:17.090 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:17.090 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:17.090 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000008115", 2022-09-03T05:58:17.091 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:58:15.555684Z" 2022-09-03T05:58:17.091 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:17.091 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:17.091 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002060", 2022-09-03T05:58:17.092 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:58:15.556779Z" 2022-09-03T05:58:17.092 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:17.092 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:58:17.094 DEBUG:rgw_multi.tests:data markers for zone=test-zone1 markers={0: '00000000000000000000:00000000000000003609', 1: '00000000000000000000:00000000000000006141', 2: '00000000000000000000:00000000000000008115', 3: '00000000000000000000:00000000000000002060'} 2022-09-03T05:58:17.094 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2022-09-03T05:58:17.095 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:17.095 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:17.096 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:17.150 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:17.150 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:17.203 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:58:17.203 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:58:17.204 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:58:17.204 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:58:17.205 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:58:17.205 INFO:teuthology.orchestra.run.smithi189.stdout: "instance_id": 4093329069989467412 2022-09-03T05:58:17.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:17.206 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:58:17.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:17.207 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:58:17.208 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:17.208 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:17.209 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000002768", 2022-09-03T05:58:17.209 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:17.209 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:17.209 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:17.210 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:31:26.029653Z" 2022-09-03T05:58:17.210 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:17.210 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:17.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:17.211 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:58:17.211 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:17.211 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:17.212 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000004881", 2022-09-03T05:58:17.212 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:17.212 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:17.212 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:17.213 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:47:17.095630Z" 2022-09-03T05:58:17.213 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:17.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:17.214 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:17.214 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:58:17.215 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:17.215 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:17.215 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000006855", 2022-09-03T05:58:17.215 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:17.216 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:17.216 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:17.216 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:50:48.498362Z" 2022-09-03T05:58:17.216 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:17.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:17.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:17.217 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:58:17.218 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:17.218 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:17.218 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:58:17.218 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:17.219 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:17.219 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:17.219 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:44:26.899674Z" 2022-09-03T05:58:17.219 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:17.220 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:17.220 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:58:17.220 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:17.220 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:58:17.221 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 0, 2022-09-03T05:58:17.221 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 0 2022-09-03T05:58:17.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:17.221 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:58:17Z" 2022-09-03T05:58:17.222 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:58:17.223 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 4093329069989467412 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002768", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:26.029653Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000004881", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:17.095630Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000006855", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:50:48.498362Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000000796", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:26.899674Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:58:17Z" } 2022-09-03T05:58:17.224 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002768', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:26.029653Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004881', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:17.095630Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006855', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:50:48.498362Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000000796', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:26.899674Z'}}] 2022-09-03T05:58:17.224 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000003609', 1: '00000000000000000000:00000000000000006141', 2: '00000000000000000000:00000000000000008115', 3: '00000000000000000000:00000000000000002060'} 2022-09-03T05:58:17.225 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002768', 1: '00000000000000000000:00000000000000004881', 2: '00000000000000000000:00000000000000006855', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:58:17.226 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone1: shard=0 master=00000000000000000000:00000000000000003609 target=00000000000000000000:00000000000000002768, shard=1 master=00000000000000000000:00000000000000006141 target=00000000000000000000:00000000000000004881, shard=2 master=00000000000000000000:00000000000000008115 target=00000000000000000000:00000000000000006855, shard=3 master=00000000000000000000:00000000000000002060 target=00000000000000000000:00000000000000000796 2022-09-03T05:58:22.227 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:22.228 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:22.229 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:22.289 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:22.290 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:22.336 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:58:22.336 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:58:22.337 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:58:22.337 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:58:22.337 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:58:22.337 INFO:teuthology.orchestra.run.smithi189.stdout: "instance_id": 4093329069989467412 2022-09-03T05:58:22.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:22.338 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:58:22.338 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:22.339 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:58:22.339 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:22.339 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:22.339 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000003609", 2022-09-03T05:58:22.340 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:22.340 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:22.340 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:22.341 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:58:15.553095Z" 2022-09-03T05:58:22.341 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:22.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:22.341 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:22.342 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:58:22.342 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:22.344 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:22.344 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000006141", 2022-09-03T05:58:22.344 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:22.344 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:22.345 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:22.345 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:58:15.554651Z" 2022-09-03T05:58:22.345 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:22.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:22.346 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:22.346 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:58:22.346 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:22.347 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:22.347 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000008115", 2022-09-03T05:58:22.347 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:22.348 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:22.348 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:22.348 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:58:15.555684Z" 2022-09-03T05:58:22.348 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:22.349 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:22.349 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:22.349 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:58:22.349 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:22.350 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:22.350 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000002060", 2022-09-03T05:58:22.350 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:22.350 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:22.351 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:22.351 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:58:15.556779Z" 2022-09-03T05:58:22.351 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:22.351 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:22.352 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:58:22.352 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:22.352 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:58:22.353 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 0, 2022-09-03T05:58:22.353 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 0 2022-09-03T05:58:22.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:22.354 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:58:22Z" 2022-09-03T05:58:22.354 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:58:22.356 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 4093329069989467412 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000003609", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:15.553095Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000006141", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:15.554651Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000008115", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:15.555684Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002060", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:15.556779Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:58:22Z" } 2022-09-03T05:58:22.357 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000003609', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:15.553095Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006141', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:15.554651Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000008115', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:15.555684Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002060', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:15.556779Z'}}] 2022-09-03T05:58:22.357 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000003609', 1: '00000000000000000000:00000000000000006141', 2: '00000000000000000000:00000000000000008115', 3: '00000000000000000000:00000000000000002060'} 2022-09-03T05:58:22.358 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000003609', 1: '00000000000000000000:00000000000000006141', 2: '00000000000000000000:00000000000000008115', 3: '00000000000000000000:00000000000000002060'} 2022-09-03T05:58:22.358 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone2 source_zone=test-zone1 2022-09-03T05:58:22.359 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone1 target=test-zone3 2022-09-03T05:58:22.359 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:22.360 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:22.361 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:22.417 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:22.417 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:22.474 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:58:22.474 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:22.475 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000003609", 2022-09-03T05:58:22.475 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:58:15.553095Z" 2022-09-03T05:58:22.475 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:22.476 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:22.476 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000006141", 2022-09-03T05:58:22.476 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:58:15.554651Z" 2022-09-03T05:58:22.476 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:22.477 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:22.477 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000008115", 2022-09-03T05:58:22.477 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:58:15.555684Z" 2022-09-03T05:58:22.477 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:22.478 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:22.478 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002060", 2022-09-03T05:58:22.478 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:58:15.556779Z" 2022-09-03T05:58:22.478 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:22.479 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:58:22.480 DEBUG:rgw_multi.tests:data markers for zone=test-zone1 markers={0: '00000000000000000000:00000000000000003609', 1: '00000000000000000000:00000000000000006141', 2: '00000000000000000000:00000000000000008115', 3: '00000000000000000000:00000000000000002060'} 2022-09-03T05:58:22.481 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2022-09-03T05:58:22.481 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:22.482 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:22.483 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:22.535 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:22.536 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:22.587 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:58:22.587 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:58:22.587 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:58:22.588 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:58:22.588 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:58:22.588 INFO:teuthology.orchestra.run.smithi181.stdout: "instance_id": 16186606431531372785 2022-09-03T05:58:22.589 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:22.589 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:58:22.589 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:22.589 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:58:22.590 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:22.590 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:22.590 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000003609", 2022-09-03T05:58:22.591 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:22.591 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:22.591 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:22.591 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:58:15.553095Z" 2022-09-03T05:58:22.592 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:22.592 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:22.592 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:22.593 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:58:22.593 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:22.593 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:22.593 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000006141", 2022-09-03T05:58:22.594 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:22.594 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:22.594 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:22.595 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:58:15.554651Z" 2022-09-03T05:58:22.595 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:22.595 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:22.595 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:22.596 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:58:22.596 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:22.596 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:22.596 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000008115", 2022-09-03T05:58:22.597 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:22.597 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:22.597 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:22.597 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:58:15.555684Z" 2022-09-03T05:58:22.598 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:22.598 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:22.598 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:22.598 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:58:22.599 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:22.599 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:22.599 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002060", 2022-09-03T05:58:22.599 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:22.600 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:22.600 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:22.600 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:58:15.556779Z" 2022-09-03T05:58:22.600 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:22.601 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:22.601 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:58:22.601 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:22.601 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:58:22.602 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 0, 2022-09-03T05:58:22.603 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 0 2022-09-03T05:58:22.604 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:22.604 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:58:22Z" 2022-09-03T05:58:22.604 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:58:22.606 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 16186606431531372785 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000003609", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:15.553095Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000006141", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:15.554651Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000008115", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:15.555684Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002060", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:15.556779Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:58:22Z" } 2022-09-03T05:58:22.608 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000003609', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:15.553095Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006141', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:15.554651Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000008115', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:15.555684Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002060', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:15.556779Z'}}] 2022-09-03T05:58:22.609 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000003609', 1: '00000000000000000000:00000000000000006141', 2: '00000000000000000000:00000000000000008115', 3: '00000000000000000000:00000000000000002060'} 2022-09-03T05:58:22.610 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000003609', 1: '00000000000000000000:00000000000000006141', 2: '00000000000000000000:00000000000000008115', 3: '00000000000000000000:00000000000000002060'} 2022-09-03T05:58:22.610 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone3 source_zone=test-zone1 2022-09-03T05:58:22.611 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone1 target=test-zone4 2022-09-03T05:58:22.613 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:22.614 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:22.615 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:22.665 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:22.665 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:22.744 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:58:22.745 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:22.745 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000003609", 2022-09-03T05:58:22.746 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:58:15.553095Z" 2022-09-03T05:58:22.746 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:22.747 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:22.747 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000006141", 2022-09-03T05:58:22.748 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:58:15.554651Z" 2022-09-03T05:58:22.748 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:22.749 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:22.749 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000008115", 2022-09-03T05:58:22.749 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:58:15.555684Z" 2022-09-03T05:58:22.749 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:22.750 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:22.750 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002060", 2022-09-03T05:58:22.750 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:58:15.556779Z" 2022-09-03T05:58:22.750 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:22.751 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:58:22.752 DEBUG:rgw_multi.tests:data markers for zone=test-zone1 markers={0: '00000000000000000000:00000000000000003609', 1: '00000000000000000000:00000000000000006141', 2: '00000000000000000000:00000000000000008115', 3: '00000000000000000000:00000000000000002060'} 2022-09-03T05:58:22.753 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone4 source_zone=test-zone1 2022-09-03T05:58:22.753 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:22.754 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:22.755 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:22.813 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:22.813 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:22.864 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:58:22.864 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:58:22.865 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:58:22.865 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:58:22.865 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:58:22.866 INFO:teuthology.orchestra.run.smithi189.stdout: "instance_id": 13887610924008747321 2022-09-03T05:58:22.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:22.866 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:58:22.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:22.867 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:58:22.867 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:22.867 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:22.868 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000003609", 2022-09-03T05:58:22.868 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:22.869 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:22.869 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:22.869 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:58:15.553095Z" 2022-09-03T05:58:22.869 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:22.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:22.870 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:22.870 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:58:22.870 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:22.871 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:22.871 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000006141", 2022-09-03T05:58:22.871 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:22.872 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:22.872 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:22.872 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:58:15.554651Z" 2022-09-03T05:58:22.872 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:22.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:22.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:22.873 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:58:22.874 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:22.874 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:22.875 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000008115", 2022-09-03T05:58:22.875 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:22.876 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:22.876 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:22.876 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:58:15.555684Z" 2022-09-03T05:58:22.876 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:22.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:22.877 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:22.877 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:58:22.878 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:22.878 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:22.878 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000002060", 2022-09-03T05:58:22.878 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:22.879 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:22.879 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:22.879 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:58:15.556779Z" 2022-09-03T05:58:22.879 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:22.880 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:22.880 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:58:22.880 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:22.880 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:58:22.881 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 0, 2022-09-03T05:58:22.881 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 0 2022-09-03T05:58:22.881 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:22.881 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:58:22Z" 2022-09-03T05:58:22.882 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:58:22.883 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 13887610924008747321 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000003609", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:15.553095Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000006141", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:15.554651Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000008115", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:15.555684Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002060", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:15.556779Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:58:22Z" } 2022-09-03T05:58:22.884 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000003609', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:15.553095Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006141', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:15.554651Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000008115', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:15.555684Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002060', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:15.556779Z'}}] 2022-09-03T05:58:22.884 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000003609', 1: '00000000000000000000:00000000000000006141', 2: '00000000000000000000:00000000000000008115', 3: '00000000000000000000:00000000000000002060'} 2022-09-03T05:58:22.885 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000003609', 1: '00000000000000000000:00000000000000006141', 2: '00000000000000000000:00000000000000008115', 3: '00000000000000000000:00000000000000002060'} 2022-09-03T05:58:22.885 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone4 source_zone=test-zone1 2022-09-03T05:58:22.886 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone2 target=test-zone1 2022-09-03T05:58:22.886 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:22.887 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:22.888 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:22.936 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:22.937 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:23.005 INFO:teuthology.orchestra.run.smithi189.stdout:[ 2022-09-03T05:58:23.005 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:23.006 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:58:23.006 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:31:03.799590Z" 2022-09-03T05:58:23.006 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:23.006 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:23.007 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000004881", 2022-09-03T05:58:23.007 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:47:17.795310Z" 2022-09-03T05:58:23.007 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:23.007 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:23.008 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000006575", 2022-09-03T05:58:23.008 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:50:48.874076Z" 2022-09-03T05:58:23.008 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:23.009 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:23.009 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:58:23.009 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:44:27.002354Z" 2022-09-03T05:58:23.009 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:23.010 INFO:teuthology.orchestra.run.smithi189.stdout:] 2022-09-03T05:58:23.011 DEBUG:rgw_multi.tests:data markers for zone=test-zone2 markers={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004881', 2: '00000000000000000000:00000000000000006575', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:58:23.012 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2022-09-03T05:58:23.013 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:23.014 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:23.015 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:23.068 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:23.069 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:23.130 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:58:23.131 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:58:23.131 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:58:23.131 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:58:23.132 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:58:23.132 INFO:teuthology.orchestra.run.smithi181.stdout: "instance_id": 5551903584477280978 2022-09-03T05:58:23.133 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:23.133 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:58:23.134 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:23.134 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:58:23.135 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:23.135 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:23.136 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:58:23.136 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:23.137 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:23.137 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:23.138 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:31:03.799590Z" 2022-09-03T05:58:23.138 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:23.139 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:23.139 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:23.140 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:58:23.140 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:23.141 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:23.141 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000004881", 2022-09-03T05:58:23.142 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:23.142 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:23.143 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:23.143 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:47:17.795310Z" 2022-09-03T05:58:23.144 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:23.144 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:23.145 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:23.145 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:58:23.146 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:23.146 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:23.147 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000006575", 2022-09-03T05:58:23.147 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:23.148 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:23.148 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:23.148 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:50:48.874076Z" 2022-09-03T05:58:23.149 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:23.149 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:23.150 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:23.150 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:58:23.151 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:23.151 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:23.152 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:58:23.152 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:23.153 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:23.153 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:23.154 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:27.002354Z" 2022-09-03T05:58:23.154 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:23.155 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:23.155 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:58:23.156 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:23.156 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:58:23.157 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 0, 2022-09-03T05:58:23.157 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 0 2022-09-03T05:58:23.158 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:23.158 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:58:23Z" 2022-09-03T05:58:23.159 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:58:23.161 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 5551903584477280978 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002627", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:03.799590Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000004881", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:17.795310Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000006575", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:50:48.874076Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000000796", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:27.002354Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:58:23Z" } 2022-09-03T05:58:23.162 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002627', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:03.799590Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004881', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:17.795310Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006575', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:50:48.874076Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000000796', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:27.002354Z'}}] 2022-09-03T05:58:23.163 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004881', 2: '00000000000000000000:00000000000000006575', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:58:23.164 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004881', 2: '00000000000000000000:00000000000000006575', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:58:23.164 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone1 source_zone=test-zone2 2022-09-03T05:58:23.165 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone2 target=test-zone3 2022-09-03T05:58:23.166 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:23.167 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'datalog', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:23.168 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 datalog status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:23.217 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:23.218 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:23.279 INFO:teuthology.orchestra.run.smithi189.stdout:[ 2022-09-03T05:58:23.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:23.280 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:58:23.281 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:31:03.799590Z" 2022-09-03T05:58:23.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:23.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:23.283 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000004881", 2022-09-03T05:58:23.283 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:47:17.795310Z" 2022-09-03T05:58:23.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:23.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:23.285 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000006575", 2022-09-03T05:58:23.285 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:50:48.874076Z" 2022-09-03T05:58:23.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:23.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:23.287 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:58:23.288 INFO:teuthology.orchestra.run.smithi189.stdout: "last_update": "2022-09-03T05:44:27.002354Z" 2022-09-03T05:58:23.288 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:23.289 INFO:teuthology.orchestra.run.smithi189.stdout:] 2022-09-03T05:58:23.291 DEBUG:rgw_multi.tests:data markers for zone=test-zone2 markers={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004881', 2: '00000000000000000000:00000000000000006575', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:58:23.292 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone3 source_zone=test-zone2 2022-09-03T05:58:23.293 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:23.294 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone2', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:23.295 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone2 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:23.347 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:23.348 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:23.391 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:58:23.392 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:58:23.392 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:58:23.393 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:58:23.393 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:58:23.393 INFO:teuthology.orchestra.run.smithi181.stdout: "instance_id": 14474342186618979890 2022-09-03T05:58:23.393 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:23.394 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:58:23.394 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:23.394 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:58:23.394 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:23.395 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:23.395 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:58:23.395 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:23.396 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:23.396 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:23.396 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:31:03.799590Z" 2022-09-03T05:58:23.396 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:23.397 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:23.397 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:23.397 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:58:23.397 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:23.398 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:23.398 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000004881", 2022-09-03T05:58:23.399 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:23.399 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:23.399 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:23.399 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:47:17.795310Z" 2022-09-03T05:58:23.400 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:23.400 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:23.400 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:23.400 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:58:23.401 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:23.401 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:23.402 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000006575", 2022-09-03T05:58:23.402 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:23.403 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:23.403 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:23.404 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:50:48.874076Z" 2022-09-03T05:58:23.404 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:23.404 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:23.405 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:23.405 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:58:23.406 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:23.406 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:23.407 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:58:23.407 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:23.408 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:23.408 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:23.409 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:27.002354Z" 2022-09-03T05:58:23.409 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:23.410 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:23.410 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:58:23.410 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:23.411 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:58:23.411 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 0, 2022-09-03T05:58:23.412 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 0 2022-09-03T05:58:23.412 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:23.413 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:58:23Z" 2022-09-03T05:58:23.413 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:58:23.414 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 14474342186618979890 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002627", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:03.799590Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000004881", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:17.795310Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000006575", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:50:48.874076Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000000796", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:27.002354Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:58:23Z" } 2022-09-03T05:58:23.415 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002627', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:03.799590Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004881', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:17.795310Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006575', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:50:48.874076Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000000796', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:27.002354Z'}}] 2022-09-03T05:58:23.416 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004881', 2: '00000000000000000000:00000000000000006575', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:58:23.417 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004881', 2: '00000000000000000000:00000000000000006575', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:58:23.417 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone3 source_zone=test-zone2 2022-09-03T05:58:23.418 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone2 target=test-zone4 2022-09-03T05:58:23.419 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone3 target=test-zone1 2022-09-03T05:58:23.420 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:23.421 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:23.422 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:23.470 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:23.471 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:23.543 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:58:23.544 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:23.544 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000003188", 2022-09-03T05:58:23.545 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:58:15.515737Z" 2022-09-03T05:58:23.545 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:23.545 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:23.546 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000005439", 2022-09-03T05:58:23.546 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:58:15.521072Z" 2022-09-03T05:58:23.547 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:23.547 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:23.548 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000006709", 2022-09-03T05:58:23.548 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:58:15.523734Z" 2022-09-03T05:58:23.549 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:23.549 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:23.550 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000001639", 2022-09-03T05:58:23.550 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:58:15.529227Z" 2022-09-03T05:58:23.551 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:23.551 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:58:23.553 DEBUG:rgw_multi.tests:data markers for zone=test-zone3 markers={0: '00000000000000000000:00000000000000003188', 1: '00000000000000000000:00000000000000005439', 2: '00000000000000000000:00000000000000006709', 3: '00000000000000000000:00000000000000001639'} 2022-09-03T05:58:23.554 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone1 source_zone=test-zone3 2022-09-03T05:58:23.555 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:23.556 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:23.557 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone3 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:23.609 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:23.610 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:23.675 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:58:23.675 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:58:23.676 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:58:23.676 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:58:23.676 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:58:23.677 INFO:teuthology.orchestra.run.smithi181.stdout: "instance_id": 4175403477120006881 2022-09-03T05:58:23.677 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:23.677 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:58:23.677 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:23.677 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:58:23.678 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:23.678 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:23.678 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:58:23.679 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:23.679 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:23.679 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:23.679 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:31:06.854659Z" 2022-09-03T05:58:23.679 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:23.680 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:23.680 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:23.680 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:58:23.680 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:23.681 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:23.681 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000004599", 2022-09-03T05:58:23.681 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:23.681 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:23.682 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:23.682 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:47:22.359142Z" 2022-09-03T05:58:23.682 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:23.683 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:23.683 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:23.683 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:58:23.683 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:23.684 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:23.684 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000005869", 2022-09-03T05:58:23.684 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:23.684 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:23.684 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:23.685 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:51:49.179551Z" 2022-09-03T05:58:23.685 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:23.685 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:23.685 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:23.686 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:58:23.686 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:23.686 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:23.686 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:58:23.687 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:23.687 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:23.687 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:23.687 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:27.009917Z" 2022-09-03T05:58:23.688 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:23.688 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:23.688 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:58:23.688 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:23.689 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:58:23.689 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 0, 2022-09-03T05:58:23.689 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 0 2022-09-03T05:58:23.689 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:23.690 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:58:23Z" 2022-09-03T05:58:23.690 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:58:23.691 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 4175403477120006881 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002627", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:06.854659Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000004599", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:22.359142Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000005869", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:51:49.179551Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000000796", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:27.009917Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:58:23Z" } 2022-09-03T05:58:23.692 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002627', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:06.854659Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004599', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:22.359142Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000005869', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:51:49.179551Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000000796', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:27.009917Z'}}] 2022-09-03T05:58:23.692 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000003188', 1: '00000000000000000000:00000000000000005439', 2: '00000000000000000000:00000000000000006709', 3: '00000000000000000000:00000000000000001639'} 2022-09-03T05:58:23.693 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004599', 2: '00000000000000000000:00000000000000005869', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:58:23.693 WARNING:rgw_multi.tests:data of zone test-zone1 behind zone test-zone3: shard=0 master=00000000000000000000:00000000000000003188 target=00000000000000000000:00000000000000002627, shard=1 master=00000000000000000000:00000000000000005439 target=00000000000000000000:00000000000000004599, shard=2 master=00000000000000000000:00000000000000006709 target=00000000000000000000:00000000000000005869, shard=3 master=00000000000000000000:00000000000000001639 target=00000000000000000000:00000000000000000796 2022-09-03T05:58:28.695 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:28.696 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:28.697 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone3 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:28.757 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:28.757 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:28.815 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:58:28.815 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:58:28.816 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:58:28.816 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:58:28.816 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:58:28.817 INFO:teuthology.orchestra.run.smithi181.stdout: "instance_id": 4175403477120006881 2022-09-03T05:58:28.817 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:28.817 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:58:28.817 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:28.818 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:58:28.818 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:28.819 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:28.819 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002627", 2022-09-03T05:58:28.819 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:28.820 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:28.820 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:28.820 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:31:06.854659Z" 2022-09-03T05:58:28.820 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:28.821 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:28.821 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:28.821 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:58:28.822 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:28.822 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:28.822 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000004599", 2022-09-03T05:58:28.822 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:28.824 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:28.824 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:28.825 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:47:22.359142Z" 2022-09-03T05:58:28.825 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:28.825 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:28.825 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:28.826 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:58:28.826 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:28.826 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:28.827 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000005869", 2022-09-03T05:58:28.827 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:28.827 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:28.827 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:28.828 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:51:49.179551Z" 2022-09-03T05:58:28.828 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:28.828 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:28.828 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:28.829 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:58:28.829 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:28.829 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:28.830 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000000796", 2022-09-03T05:58:28.830 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:28.831 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:28.831 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:28.832 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:44:27.009917Z" 2022-09-03T05:58:28.832 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:28.833 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:28.833 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:58:28.835 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:28.835 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:58:28.836 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 0, 2022-09-03T05:58:28.836 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 0 2022-09-03T05:58:28.837 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:28.837 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:58:28Z" 2022-09-03T05:58:28.837 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:58:28.839 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 4175403477120006881 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002627", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:31:06.854659Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000004599", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:47:22.359142Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000005869", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:51:49.179551Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000000796", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:44:27.009917Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:58:28Z" } 2022-09-03T05:58:28.840 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002627', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:31:06.854659Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000004599', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:47:22.359142Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000005869', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:51:49.179551Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000000796', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:44:27.009917Z'}}] 2022-09-03T05:58:28.841 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000003188', 1: '00000000000000000000:00000000000000005439', 2: '00000000000000000000:00000000000000006709', 3: '00000000000000000000:00000000000000001639'} 2022-09-03T05:58:28.841 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000002627', 1: '00000000000000000000:00000000000000004599', 2: '00000000000000000000:00000000000000005869', 3: '00000000000000000000:00000000000000000796'} 2022-09-03T05:58:28.842 WARNING:rgw_multi.tests:data of zone test-zone1 behind zone test-zone3: shard=0 master=00000000000000000000:00000000000000003188 target=00000000000000000000:00000000000000002627, shard=1 master=00000000000000000000:00000000000000005439 target=00000000000000000000:00000000000000004599, shard=2 master=00000000000000000000:00000000000000006709 target=00000000000000000000:00000000000000005869, shard=3 master=00000000000000000000:00000000000000001639 target=00000000000000000000:00000000000000000796 2022-09-03T05:58:33.843 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:33.844 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'data', 'sync', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:33.845 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 data sync status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone3 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:33.904 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:33.904 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:33.964 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:58:33.964 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:58:33.965 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:58:33.965 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:58:33.965 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:58:33.966 INFO:teuthology.orchestra.run.smithi181.stdout: "instance_id": 4175403477120006881 2022-09-03T05:58:33.966 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:33.967 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:58:33.967 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:33.968 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:58:33.968 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:33.968 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:33.969 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000003468", 2022-09-03T05:58:33.969 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:33.969 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:33.970 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:33.970 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:58:24.776889Z" 2022-09-03T05:58:33.971 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:33.971 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:33.971 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:33.972 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:58:33.972 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:33.973 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:33.973 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000005859", 2022-09-03T05:58:33.973 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:33.973 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:33.974 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:33.974 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:58:24.779365Z" 2022-09-03T05:58:33.975 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:33.975 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:33.975 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:33.975 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:58:33.976 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:33.976 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:33.977 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000007129", 2022-09-03T05:58:33.977 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:33.977 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:33.977 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:33.978 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:58:24.781989Z" 2022-09-03T05:58:33.979 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:33.979 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:33.979 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:33.979 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:58:33.980 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:33.980 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "incremental-sync", 2022-09-03T05:58:33.980 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002060", 2022-09-03T05:58:33.980 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:33.981 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:33.981 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:33.981 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:58:24.784533Z" 2022-09-03T05:58:33.981 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:33.982 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:33.983 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:58:33.983 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:33.984 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:58:33.985 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 0, 2022-09-03T05:58:33.985 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 0 2022-09-03T05:58:33.986 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:33.986 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:58:33Z" 2022-09-03T05:58:33.986 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:58:33.990 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 4175403477120006881 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000003468", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:24.776889Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000005859", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:24.779365Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000007129", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:24.781989Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002060", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:24.784533Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:58:33Z" } 2022-09-03T05:58:33.992 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000003468', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:24.776889Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000005859', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:24.779365Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000007129', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:24.781989Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002060', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:24.784533Z'}}] 2022-09-03T05:58:33.993 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000003188', 1: '00000000000000000000:00000000000000005439', 2: '00000000000000000000:00000000000000006709', 3: '00000000000000000000:00000000000000001639'} 2022-09-03T05:58:33.994 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000003468', 1: '00000000000000000000:00000000000000005859', 2: '00000000000000000000:00000000000000007129', 3: '00000000000000000000:00000000000000002060'} 2022-09-03T05:58:33.994 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone1 source_zone=test-zone3 2022-09-03T05:58:33.995 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone3 target=test-zone2 2022-09-03T05:58:33.996 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['datalog', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:33.996 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'datalog', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:33.998 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 datalog status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:34.051 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:34.052 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:34.106 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:58:34.107 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:34.107 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000003468", 2022-09-03T05:58:34.107 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:58:24.776889Z" 2022-09-03T05:58:34.107 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:34.108 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:34.108 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000005859", 2022-09-03T05:58:34.109 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:58:24.779365Z" 2022-09-03T05:58:34.109 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:34.110 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:34.110 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000007129", 2022-09-03T05:58:34.110 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:58:24.781989Z" 2022-09-03T05:58:34.111 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:34.111 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:34.112 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "00000000000000000000:00000000000000002060", 2022-09-03T05:58:34.112 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:58:24.784533Z" 2022-09-03T05:58:34.112 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:34.113 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:58:34.115 DEBUG:rgw_multi.tests:data markers for zone=test-zone3 markers={0: '00000000000000000000:00000000000000003468', 1: '00000000000000000000:00000000000000005859', 2: '00000000000000000000:00000000000000007129', 3: '00000000000000000000:00000000000000002060'} 2022-09-03T05:58:34.116 INFO:rgw_multi.tests:starting data checkpoint for target_zone=test-zone2 source_zone=test-zone3 2022-09-03T05:58:34.117 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:34.118 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:34.119 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone3 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:34.180 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:34.180 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:34.244 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:58:34.245 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:58:34.245 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:58:34.246 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:58:34.247 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:58:34.247 INFO:teuthology.orchestra.run.smithi189.stdout: "instance_id": 1043408315071002039 2022-09-03T05:58:34.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:34.248 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:58:34.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:34.249 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:58:34.250 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:34.250 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:34.250 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000003188", 2022-09-03T05:58:34.251 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:34.251 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:34.251 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:34.251 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:58:15.515737Z" 2022-09-03T05:58:34.252 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:34.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:34.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:34.253 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:58:34.253 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:34.253 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:34.253 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000005439", 2022-09-03T05:58:34.254 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:34.254 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:34.254 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:34.255 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:58:15.521072Z" 2022-09-03T05:58:34.255 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:34.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:34.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:34.263 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:58:34.263 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:34.263 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:34.263 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000006709", 2022-09-03T05:58:34.264 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:34.264 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:34.264 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:34.265 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:58:15.523734Z" 2022-09-03T05:58:34.265 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:34.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:34.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:34.266 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:58:34.266 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:34.266 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:34.266 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000001639", 2022-09-03T05:58:34.267 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:34.267 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:34.267 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:34.267 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:58:15.529227Z" 2022-09-03T05:58:34.268 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:34.268 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:34.268 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:58:34.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:34.269 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:58:34.269 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 0, 2022-09-03T05:58:34.269 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 0 2022-09-03T05:58:34.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:34.270 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:58:34Z" 2022-09-03T05:58:34.270 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:58:34.271 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 1043408315071002039 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000003188", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:15.515737Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000005439", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:15.521072Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000006709", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:15.523734Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000001639", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:15.529227Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:58:34Z" } 2022-09-03T05:58:34.272 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000003188', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:15.515737Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000005439', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:15.521072Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006709', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:15.523734Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000001639', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:15.529227Z'}}] 2022-09-03T05:58:34.273 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000003468', 1: '00000000000000000000:00000000000000005859', 2: '00000000000000000000:00000000000000007129', 3: '00000000000000000000:00000000000000002060'} 2022-09-03T05:58:34.274 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000003188', 1: '00000000000000000000:00000000000000005439', 2: '00000000000000000000:00000000000000006709', 3: '00000000000000000000:00000000000000001639'} 2022-09-03T05:58:34.274 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone3: shard=0 master=00000000000000000000:00000000000000003468 target=00000000000000000000:00000000000000003188, shard=1 master=00000000000000000000:00000000000000005859 target=00000000000000000000:00000000000000005439, shard=2 master=00000000000000000000:00000000000000007129 target=00000000000000000000:00000000000000006709, shard=3 master=00000000000000000000:00000000000000002060 target=00000000000000000000:00000000000000001639 2022-09-03T05:58:39.278 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:39.279 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:39.279 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone3 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:39.338 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:39.339 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:39.384 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:58:39.385 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:58:39.385 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:58:39.385 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:58:39.386 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:58:39.386 INFO:teuthology.orchestra.run.smithi189.stdout: "instance_id": 1043408315071002039 2022-09-03T05:58:39.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:39.387 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:58:39.387 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:39.387 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:58:39.388 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:39.389 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:39.389 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000003188", 2022-09-03T05:58:39.390 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:39.390 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:39.391 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:39.391 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:58:15.515737Z" 2022-09-03T05:58:39.392 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:39.392 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:39.393 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:39.393 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:58:39.394 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:39.394 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:39.395 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000005439", 2022-09-03T05:58:39.395 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:39.396 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:39.396 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:39.397 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:58:15.521072Z" 2022-09-03T05:58:39.397 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:39.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:39.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:39.399 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:58:39.399 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:39.399 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:39.400 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000006709", 2022-09-03T05:58:39.400 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:39.401 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:39.401 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:39.401 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:58:15.523734Z" 2022-09-03T05:58:39.402 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:39.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:39.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:39.403 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:58:39.403 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:39.403 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:39.404 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000001639", 2022-09-03T05:58:39.404 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:39.404 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:39.405 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:39.405 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:58:15.529227Z" 2022-09-03T05:58:39.405 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:39.406 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:39.406 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:58:39.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:39.407 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:58:39.407 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 0, 2022-09-03T05:58:39.407 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 0 2022-09-03T05:58:39.408 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:39.408 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:58:39Z" 2022-09-03T05:58:39.408 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:58:39.410 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 1043408315071002039 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000003188", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:15.515737Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000005439", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:15.521072Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000006709", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:15.523734Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000001639", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:15.529227Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:58:39Z" } 2022-09-03T05:58:39.411 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000003188', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:15.515737Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000005439', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:15.521072Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000006709', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:15.523734Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000001639', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:15.529227Z'}}] 2022-09-03T05:58:39.411 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000003468', 1: '00000000000000000000:00000000000000005859', 2: '00000000000000000000:00000000000000007129', 3: '00000000000000000000:00000000000000002060'} 2022-09-03T05:58:39.412 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000003188', 1: '00000000000000000000:00000000000000005439', 2: '00000000000000000000:00000000000000006709', 3: '00000000000000000000:00000000000000001639'} 2022-09-03T05:58:39.413 WARNING:rgw_multi.tests:data of zone test-zone2 behind zone test-zone3: shard=0 master=00000000000000000000:00000000000000003468 target=00000000000000000000:00000000000000003188, shard=1 master=00000000000000000000:00000000000000005859 target=00000000000000000000:00000000000000005439, shard=2 master=00000000000000000000:00000000000000007129 target=00000000000000000000:00000000000000006709, shard=3 master=00000000000000000000:00000000000000002060 target=00000000000000000000:00000000000000001639 2022-09-03T05:58:44.419 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:44.420 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone3', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:44.421 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone3 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:44.487 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:44.487 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:44.538 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:58:44.539 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:58:44.539 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:58:44.539 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:58:44.540 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:58:44.540 INFO:teuthology.orchestra.run.smithi189.stdout: "instance_id": 1043408315071002039 2022-09-03T05:58:44.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:44.541 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:58:44.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:44.542 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:58:44.542 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:44.542 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:44.543 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000003468", 2022-09-03T05:58:44.543 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:44.543 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:44.545 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:44.545 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:58:24.776889Z" 2022-09-03T05:58:44.546 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:44.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:44.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:44.547 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:58:44.547 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:44.547 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:44.548 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000005859", 2022-09-03T05:58:44.548 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:44.548 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:44.549 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:44.549 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:58:24.779365Z" 2022-09-03T05:58:44.549 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:44.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:44.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:44.550 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:58:44.551 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:44.551 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:44.551 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000007129", 2022-09-03T05:58:44.551 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:44.552 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:44.552 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:44.552 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:58:24.781989Z" 2022-09-03T05:58:44.552 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:44.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:44.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:44.553 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:58:44.554 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:44.554 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "incremental-sync", 2022-09-03T05:58:44.555 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "00000000000000000000:00000000000000002060", 2022-09-03T05:58:44.555 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:44.556 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:44.556 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:44.557 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:58:24.784533Z" 2022-09-03T05:58:44.557 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:44.558 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:44.558 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:58:44.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:44.559 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:58:44.560 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 0, 2022-09-03T05:58:44.560 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 0 2022-09-03T05:58:44.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:44.561 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:58:44Z" 2022-09-03T05:58:44.561 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:58:44.564 DEBUG:rgw_multi.tests:current data sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "instance_id": 1043408315071002039 }, "markers": [ { "key": 0, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000003468", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:24.776889Z" } }, { "key": 1, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000005859", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:24.779365Z" } }, { "key": 2, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000007129", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:24.781989Z" } }, { "key": 3, "val": { "status": "incremental-sync", "marker": "00000000000000000000:00000000000000002060", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:24.784533Z" } } ] }, "full_sync": { "total": 0, "complete": 0 }, "current_time": "2022-09-03T05:58:44Z" } 2022-09-03T05:58:44.565 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000003468', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:24.776889Z'}}, {'key': 1, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000005859', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:24.779365Z'}}, {'key': 2, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000007129', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:24.781989Z'}}, {'key': 3, 'val': {'status': 'incremental-sync', 'marker': '00000000000000000000:00000000000000002060', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:24.784533Z'}}] 2022-09-03T05:58:44.566 DEBUG:rgw_multi.tests:log_status={0: '00000000000000000000:00000000000000003468', 1: '00000000000000000000:00000000000000005859', 2: '00000000000000000000:00000000000000007129', 3: '00000000000000000000:00000000000000002060'} 2022-09-03T05:58:44.567 DEBUG:rgw_multi.tests:sync_status={0: '00000000000000000000:00000000000000003468', 1: '00000000000000000000:00000000000000005859', 2: '00000000000000000000:00000000000000007129', 3: '00000000000000000000:00000000000000002060'} 2022-09-03T05:58:44.568 INFO:rgw_multi.tests:finished data checkpoint for target_zone=test-zone2 source_zone=test-zone3 2022-09-03T05:58:44.568 DEBUG:rgw_multi.tests:data checkpoint: source=test-zone3 target=test-zone4 2022-09-03T05:58:44.570 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_sync_disable ... ok 2022-09-03T05:58:44.573 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-59 2022-09-03T05:58:44.574 DEBUG:boto:path=/dsgvcy-59/ 2022-09-03T05:58:44.575 DEBUG:boto:auth_path=/dsgvcy-59/ 2022-09-03T05:58:44.577 DEBUG:boto:Method: PUT 2022-09-03T05:58:44.578 DEBUG:boto:Path: /dsgvcy-59/ 2022-09-03T05:58:44.578 DEBUG:boto:Data: 2022-09-03T05:58:44.579 DEBUG:boto:Headers: {} 2022-09-03T05:58:44.579 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:58:44.580 DEBUG:boto:Port: 8000 2022-09-03T05:58:44.580 DEBUG:boto:Params: {} 2022-09-03T05:58:44.582 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:58:44.583 DEBUG:boto:Token: None 2022-09-03T05:58:44.583 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:58:44 GMT /dsgvcy-59/ 2022-09-03T05:58:44.584 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NxiVxvKwEkkHD2gQHEcB0Vp6Btw= 2022-09-03T05:58:44.585 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:58:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NxiVxvKwEkkHD2gQHEcB0Vp6Btw=', 'Content-Length': '0'} 2022-09-03T05:58:44.605 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000fde3512cb6435491-006312ed14-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:58:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:44.606 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=dsgvcy-60 2022-09-03T05:58:44.607 DEBUG:boto:path=/dsgvcy-60/ 2022-09-03T05:58:44.608 DEBUG:boto:auth_path=/dsgvcy-60/ 2022-09-03T05:58:44.608 DEBUG:boto:Method: PUT 2022-09-03T05:58:44.609 DEBUG:boto:Path: /dsgvcy-60/ 2022-09-03T05:58:44.610 DEBUG:boto:Data: 2022-09-03T05:58:44.610 DEBUG:boto:Headers: {} 2022-09-03T05:58:44.611 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:58:44.611 DEBUG:boto:Port: 8000 2022-09-03T05:58:44.612 DEBUG:boto:Params: {} 2022-09-03T05:58:44.612 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T05:58:44.613 DEBUG:boto:Token: None 2022-09-03T05:58:44.613 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:58:44 GMT /dsgvcy-60/ 2022-09-03T05:58:44.614 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:chIo2P4gp/WPyvA9q796EXj7KoM= 2022-09-03T05:58:44.615 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:58:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:chIo2P4gp/WPyvA9q796EXj7KoM=', 'Content-Length': '0'} 2022-09-03T05:58:44.651 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000049dfd96466f496ec-006312ed14-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:58:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:44.652 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=dsgvcy-61 2022-09-03T05:58:44.653 DEBUG:boto:path=/dsgvcy-61/ 2022-09-03T05:58:44.653 DEBUG:boto:auth_path=/dsgvcy-61/ 2022-09-03T05:58:44.654 DEBUG:boto:Method: PUT 2022-09-03T05:58:44.655 DEBUG:boto:Path: /dsgvcy-61/ 2022-09-03T05:58:44.655 DEBUG:boto:Data: 2022-09-03T05:58:44.656 DEBUG:boto:Headers: {} 2022-09-03T05:58:44.656 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:58:44.657 DEBUG:boto:Port: 8001 2022-09-03T05:58:44.657 DEBUG:boto:Params: {} 2022-09-03T05:58:44.658 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T05:58:44.658 DEBUG:boto:Token: None 2022-09-03T05:58:44.659 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 05:58:44 GMT /dsgvcy-61/ 2022-09-03T05:58:44.660 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nkqSO+5C+dqjgSfKLFiKRmmykqA= 2022-09-03T05:58:44.660 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:58:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nkqSO+5C+dqjgSfKLFiKRmmykqA=', 'Content-Length': '0'} 2022-09-03T05:58:44.699 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000083d4066e9a96ba34-006312ed14-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:58:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:44.701 DEBUG:boto:path=/dsgvcy-59/ 2022-09-03T05:58:44.702 DEBUG:boto:auth_path=/dsgvcy-59/ 2022-09-03T05:58:44.703 DEBUG:boto:Method: HEAD 2022-09-03T05:58:44.704 DEBUG:boto:Path: /dsgvcy-59/ 2022-09-03T05:58:44.704 DEBUG:boto:Data: 2022-09-03T05:58:44.705 DEBUG:boto:Headers: {} 2022-09-03T05:58:44.705 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:58:44.706 DEBUG:boto:Port: 8000 2022-09-03T05:58:44.707 DEBUG:boto:Params: {} 2022-09-03T05:58:44.708 DEBUG:boto:Token: None 2022-09-03T05:58:44.709 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:58:44 GMT /dsgvcy-59/ 2022-09-03T05:58:44.710 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8zJPiUplh/B0iNpksmKDNU7KAPM= 2022-09-03T05:58:44.711 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:58:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8zJPiUplh/B0iNpksmKDNU7KAPM=', 'Content-Length': '0'} 2022-09-03T05:58:44.715 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000483c874c5abc50a8-006312ed14-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:58:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:44.716 DEBUG:boto:path=/dsgvcy-59/obj1 2022-09-03T05:58:44.717 DEBUG:boto:auth_path=/dsgvcy-59/obj1 2022-09-03T05:58:44.717 DEBUG:boto:Method: PUT 2022-09-03T05:58:44.718 DEBUG:boto:Path: /dsgvcy-59/obj1 2022-09-03T05:58:44.718 DEBUG:boto:Data: 2022-09-03T05:58:44.719 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:58:44.719 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:58:44.720 DEBUG:boto:Port: 8000 2022-09-03T05:58:44.720 DEBUG:boto:Params: {} 2022-09-03T05:58:44.721 DEBUG:boto:Token: None 2022-09-03T05:58:44.722 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:58:44 GMT /dsgvcy-59/obj1 2022-09-03T05:58:44.722 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:udXXRBNXSaqi9Xckk3evDr3PwRI= 2022-09-03T05:58:44.723 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:58:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:udXXRBNXSaqi9Xckk3evDr3PwRI='} 2022-09-03T05:58:44.741 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000027bb512956ae1ce6-006312ed14-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:58:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:44.743 DEBUG:boto:path=/dsgvcy-59/ 2022-09-03T05:58:44.744 DEBUG:boto:auth_path=/dsgvcy-59/ 2022-09-03T05:58:44.745 DEBUG:boto:Method: HEAD 2022-09-03T05:58:44.746 DEBUG:boto:Path: /dsgvcy-59/ 2022-09-03T05:58:44.747 DEBUG:boto:Data: 2022-09-03T05:58:44.747 DEBUG:boto:Headers: {} 2022-09-03T05:58:44.748 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:58:44.749 DEBUG:boto:Port: 8000 2022-09-03T05:58:44.750 DEBUG:boto:Params: {} 2022-09-03T05:58:44.751 DEBUG:boto:Token: None 2022-09-03T05:58:44.752 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:58:44 GMT /dsgvcy-59/ 2022-09-03T05:58:44.753 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8zJPiUplh/B0iNpksmKDNU7KAPM= 2022-09-03T05:58:44.754 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:58:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8zJPiUplh/B0iNpksmKDNU7KAPM=', 'Content-Length': '0'} 2022-09-03T05:58:44.761 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000446747b89f8f7197-006312ed14-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:58:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:44.763 DEBUG:boto:path=/dsgvcy-59/obj2 2022-09-03T05:58:44.764 DEBUG:boto:auth_path=/dsgvcy-59/obj2 2022-09-03T05:58:44.765 DEBUG:boto:Method: PUT 2022-09-03T05:58:44.766 DEBUG:boto:Path: /dsgvcy-59/obj2 2022-09-03T05:58:44.767 DEBUG:boto:Data: 2022-09-03T05:58:44.767 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:58:44.768 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:58:44.769 DEBUG:boto:Port: 8000 2022-09-03T05:58:44.769 DEBUG:boto:Params: {} 2022-09-03T05:58:44.770 DEBUG:boto:Token: None 2022-09-03T05:58:44.771 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:58:44 GMT /dsgvcy-59/obj2 2022-09-03T05:58:44.771 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tCl1ATgSyzSFZ8hOFzsJiaYL5w0= 2022-09-03T05:58:44.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:58:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tCl1ATgSyzSFZ8hOFzsJiaYL5w0='} 2022-09-03T05:58:44.819 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f414938c9577589c-006312ed14-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:58:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:44.820 DEBUG:boto:path=/dsgvcy-59/ 2022-09-03T05:58:44.820 DEBUG:boto:auth_path=/dsgvcy-59/ 2022-09-03T05:58:44.821 DEBUG:boto:Method: HEAD 2022-09-03T05:58:44.821 DEBUG:boto:Path: /dsgvcy-59/ 2022-09-03T05:58:44.822 DEBUG:boto:Data: 2022-09-03T05:58:44.822 DEBUG:boto:Headers: {} 2022-09-03T05:58:44.823 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:58:44.823 DEBUG:boto:Port: 8000 2022-09-03T05:58:44.824 DEBUG:boto:Params: {} 2022-09-03T05:58:44.824 DEBUG:boto:Token: None 2022-09-03T05:58:44.825 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:58:44 GMT /dsgvcy-59/ 2022-09-03T05:58:44.825 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8zJPiUplh/B0iNpksmKDNU7KAPM= 2022-09-03T05:58:44.826 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:58:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8zJPiUplh/B0iNpksmKDNU7KAPM=', 'Content-Length': '0'} 2022-09-03T05:58:44.829 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000087bc6907263747cb-006312ed14-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:58:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:44.829 DEBUG:boto:path=/dsgvcy-59/obj3 2022-09-03T05:58:44.830 DEBUG:boto:auth_path=/dsgvcy-59/obj3 2022-09-03T05:58:44.830 DEBUG:boto:Method: PUT 2022-09-03T05:58:44.831 DEBUG:boto:Path: /dsgvcy-59/obj3 2022-09-03T05:58:44.831 DEBUG:boto:Data: 2022-09-03T05:58:44.832 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:58:44.832 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:58:44.833 DEBUG:boto:Port: 8000 2022-09-03T05:58:44.834 DEBUG:boto:Params: {} 2022-09-03T05:58:44.834 DEBUG:boto:Token: None 2022-09-03T05:58:44.835 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:58:44 GMT /dsgvcy-59/obj3 2022-09-03T05:58:44.835 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bVlZSHfas0lJGXL2kNA12SdQpJs= 2022-09-03T05:58:44.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:58:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bVlZSHfas0lJGXL2kNA12SdQpJs='} 2022-09-03T05:58:44.852 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b649eacf8488d6a6-006312ed14-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:58:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:44.853 DEBUG:boto:path=/dsgvcy-59/ 2022-09-03T05:58:44.854 DEBUG:boto:auth_path=/dsgvcy-59/ 2022-09-03T05:58:44.855 DEBUG:boto:Method: HEAD 2022-09-03T05:58:44.856 DEBUG:boto:Path: /dsgvcy-59/ 2022-09-03T05:58:44.857 DEBUG:boto:Data: 2022-09-03T05:58:44.859 DEBUG:boto:Headers: {} 2022-09-03T05:58:44.860 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:58:44.860 DEBUG:boto:Port: 8000 2022-09-03T05:58:44.860 DEBUG:boto:Params: {} 2022-09-03T05:58:44.861 DEBUG:boto:Token: None 2022-09-03T05:58:44.862 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:58:44 GMT /dsgvcy-59/ 2022-09-03T05:58:44.862 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8zJPiUplh/B0iNpksmKDNU7KAPM= 2022-09-03T05:58:44.863 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:58:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8zJPiUplh/B0iNpksmKDNU7KAPM=', 'Content-Length': '0'} 2022-09-03T05:58:44.866 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c0c6c0c26e1a544d-006312ed14-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:58:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:44.867 DEBUG:boto:path=/dsgvcy-59/obj4 2022-09-03T05:58:44.867 DEBUG:boto:auth_path=/dsgvcy-59/obj4 2022-09-03T05:58:44.868 DEBUG:boto:Method: PUT 2022-09-03T05:58:44.868 DEBUG:boto:Path: /dsgvcy-59/obj4 2022-09-03T05:58:44.869 DEBUG:boto:Data: 2022-09-03T05:58:44.870 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:58:44.870 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T05:58:44.871 DEBUG:boto:Port: 8000 2022-09-03T05:58:44.871 DEBUG:boto:Params: {} 2022-09-03T05:58:44.872 DEBUG:boto:Token: None 2022-09-03T05:58:44.872 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:58:44 GMT /dsgvcy-59/obj4 2022-09-03T05:58:44.873 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xkPijcHuAkbZofy9e1ulzAdATZ8= 2022-09-03T05:58:44.873 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:58:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xkPijcHuAkbZofy9e1ulzAdATZ8='} 2022-09-03T05:58:44.880 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d5f0bc5b87fcb0e2-006312ed14-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 05:58:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:44.880 DEBUG:boto:path=/dsgvcy-60/ 2022-09-03T05:58:44.881 DEBUG:boto:auth_path=/dsgvcy-60/ 2022-09-03T05:58:44.881 DEBUG:boto:Method: HEAD 2022-09-03T05:58:44.882 DEBUG:boto:Path: /dsgvcy-60/ 2022-09-03T05:58:44.883 DEBUG:boto:Data: 2022-09-03T05:58:44.883 DEBUG:boto:Headers: {} 2022-09-03T05:58:44.883 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:58:44.884 DEBUG:boto:Port: 8000 2022-09-03T05:58:44.885 DEBUG:boto:Params: {} 2022-09-03T05:58:44.886 DEBUG:boto:Token: None 2022-09-03T05:58:44.887 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:58:44 GMT /dsgvcy-60/ 2022-09-03T05:58:44.888 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TbFSJ1Lom5djn5Lg5TcK+HIocZ8= 2022-09-03T05:58:44.889 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:58:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TbFSJ1Lom5djn5Lg5TcK+HIocZ8=', 'Content-Length': '0'} 2022-09-03T05:58:44.892 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a2bb367e5702ee87-006312ed14-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:58:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:44.893 DEBUG:boto:path=/dsgvcy-60/obj1 2022-09-03T05:58:44.894 DEBUG:boto:auth_path=/dsgvcy-60/obj1 2022-09-03T05:58:44.895 DEBUG:boto:Method: PUT 2022-09-03T05:58:44.896 DEBUG:boto:Path: /dsgvcy-60/obj1 2022-09-03T05:58:44.897 DEBUG:boto:Data: 2022-09-03T05:58:44.898 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:58:44.899 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:58:44.899 DEBUG:boto:Port: 8000 2022-09-03T05:58:44.900 DEBUG:boto:Params: {} 2022-09-03T05:58:44.900 DEBUG:boto:Token: None 2022-09-03T05:58:44.901 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:58:44 GMT /dsgvcy-60/obj1 2022-09-03T05:58:44.901 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YN51s4DpfdbvA2pqAAvdexLQNTk= 2022-09-03T05:58:44.902 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:58:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YN51s4DpfdbvA2pqAAvdexLQNTk='} 2022-09-03T05:58:44.908 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000698e968ffa44ea24-006312ed14-15d2-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:58:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:44.909 DEBUG:boto:path=/dsgvcy-60/ 2022-09-03T05:58:44.910 DEBUG:boto:auth_path=/dsgvcy-60/ 2022-09-03T05:58:44.910 DEBUG:boto:Method: HEAD 2022-09-03T05:58:44.911 DEBUG:boto:Path: /dsgvcy-60/ 2022-09-03T05:58:44.911 DEBUG:boto:Data: 2022-09-03T05:58:44.912 DEBUG:boto:Headers: {} 2022-09-03T05:58:44.912 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:58:44.913 DEBUG:boto:Port: 8000 2022-09-03T05:58:44.913 DEBUG:boto:Params: {} 2022-09-03T05:58:44.914 DEBUG:boto:Token: None 2022-09-03T05:58:44.915 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:58:44 GMT /dsgvcy-60/ 2022-09-03T05:58:44.916 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TbFSJ1Lom5djn5Lg5TcK+HIocZ8= 2022-09-03T05:58:44.917 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:58:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TbFSJ1Lom5djn5Lg5TcK+HIocZ8=', 'Content-Length': '0'} 2022-09-03T05:58:44.919 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c78c60f6843587ed-006312ed14-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:58:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:44.921 DEBUG:boto:path=/dsgvcy-60/obj2 2022-09-03T05:58:44.922 DEBUG:boto:auth_path=/dsgvcy-60/obj2 2022-09-03T05:58:44.923 DEBUG:boto:Method: PUT 2022-09-03T05:58:44.923 DEBUG:boto:Path: /dsgvcy-60/obj2 2022-09-03T05:58:44.924 DEBUG:boto:Data: 2022-09-03T05:58:44.924 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:58:44.925 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:58:44.925 DEBUG:boto:Port: 8000 2022-09-03T05:58:44.926 DEBUG:boto:Params: {} 2022-09-03T05:58:44.926 DEBUG:boto:Token: None 2022-09-03T05:58:44.927 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:58:44 GMT /dsgvcy-60/obj2 2022-09-03T05:58:44.927 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:96X5uWeY6yWqUwzMi/XPwI5G5Uw= 2022-09-03T05:58:44.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:58:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:96X5uWeY6yWqUwzMi/XPwI5G5Uw='} 2022-09-03T05:58:44.935 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000050784ba1d17b1bf7-006312ed14-15d2-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:58:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:44.935 DEBUG:boto:path=/dsgvcy-60/ 2022-09-03T05:58:44.936 DEBUG:boto:auth_path=/dsgvcy-60/ 2022-09-03T05:58:44.936 DEBUG:boto:Method: HEAD 2022-09-03T05:58:44.937 DEBUG:boto:Path: /dsgvcy-60/ 2022-09-03T05:58:44.937 DEBUG:boto:Data: 2022-09-03T05:58:44.938 DEBUG:boto:Headers: {} 2022-09-03T05:58:44.938 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:58:44.939 DEBUG:boto:Port: 8000 2022-09-03T05:58:44.939 DEBUG:boto:Params: {} 2022-09-03T05:58:44.940 DEBUG:boto:Token: None 2022-09-03T05:58:44.940 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:58:44 GMT /dsgvcy-60/ 2022-09-03T05:58:44.941 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TbFSJ1Lom5djn5Lg5TcK+HIocZ8= 2022-09-03T05:58:44.942 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:58:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TbFSJ1Lom5djn5Lg5TcK+HIocZ8=', 'Content-Length': '0'} 2022-09-03T05:58:44.944 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007249beadc5e5e5c5-006312ed14-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:58:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:44.945 DEBUG:boto:path=/dsgvcy-60/obj3 2022-09-03T05:58:44.945 DEBUG:boto:auth_path=/dsgvcy-60/obj3 2022-09-03T05:58:44.946 DEBUG:boto:Method: PUT 2022-09-03T05:58:44.946 DEBUG:boto:Path: /dsgvcy-60/obj3 2022-09-03T05:58:44.947 DEBUG:boto:Data: 2022-09-03T05:58:44.947 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:58:44.948 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:58:44.948 DEBUG:boto:Port: 8000 2022-09-03T05:58:44.949 DEBUG:boto:Params: {} 2022-09-03T05:58:44.949 DEBUG:boto:Token: None 2022-09-03T05:58:44.950 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:58:44 GMT /dsgvcy-60/obj3 2022-09-03T05:58:44.950 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5fqjkiViU8AoitLv7DthugCTmro= 2022-09-03T05:58:44.951 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:58:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5fqjkiViU8AoitLv7DthugCTmro='} 2022-09-03T05:58:44.957 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009c5fa4bf24f609cc-006312ed14-15d2-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:58:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:44.958 DEBUG:boto:path=/dsgvcy-60/ 2022-09-03T05:58:44.958 DEBUG:boto:auth_path=/dsgvcy-60/ 2022-09-03T05:58:44.959 DEBUG:boto:Method: HEAD 2022-09-03T05:58:44.959 DEBUG:boto:Path: /dsgvcy-60/ 2022-09-03T05:58:44.960 DEBUG:boto:Data: 2022-09-03T05:58:44.960 DEBUG:boto:Headers: {} 2022-09-03T05:58:44.961 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:58:44.962 DEBUG:boto:Port: 8000 2022-09-03T05:58:44.962 DEBUG:boto:Params: {} 2022-09-03T05:58:44.963 DEBUG:boto:Token: None 2022-09-03T05:58:44.964 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:58:44 GMT /dsgvcy-60/ 2022-09-03T05:58:44.965 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TbFSJ1Lom5djn5Lg5TcK+HIocZ8= 2022-09-03T05:58:44.966 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:58:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TbFSJ1Lom5djn5Lg5TcK+HIocZ8=', 'Content-Length': '0'} 2022-09-03T05:58:44.969 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000032a2c2186c5f5bd1-006312ed14-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:58:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:44.970 DEBUG:boto:path=/dsgvcy-60/obj4 2022-09-03T05:58:44.971 DEBUG:boto:auth_path=/dsgvcy-60/obj4 2022-09-03T05:58:44.972 DEBUG:boto:Method: PUT 2022-09-03T05:58:44.973 DEBUG:boto:Path: /dsgvcy-60/obj4 2022-09-03T05:58:44.974 DEBUG:boto:Data: 2022-09-03T05:58:44.975 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:58:44.976 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T05:58:44.977 DEBUG:boto:Port: 8000 2022-09-03T05:58:44.977 DEBUG:boto:Params: {} 2022-09-03T05:58:44.979 DEBUG:boto:Token: None 2022-09-03T05:58:44.980 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:58:44 GMT /dsgvcy-60/obj4 2022-09-03T05:58:44.981 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2udrQg6+aJYCrk43BTdFMjRyqY0= 2022-09-03T05:58:44.982 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:58:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2udrQg6+aJYCrk43BTdFMjRyqY0='} 2022-09-03T05:58:45.031 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000cbcba3b49da61134-006312ed14-15d2-test-zone2'), ('Date', 'Sat, 03 Sep 2022 05:58:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:45.032 DEBUG:boto:path=/dsgvcy-61/ 2022-09-03T05:58:45.033 DEBUG:boto:auth_path=/dsgvcy-61/ 2022-09-03T05:58:45.034 DEBUG:boto:Method: HEAD 2022-09-03T05:58:45.035 DEBUG:boto:Path: /dsgvcy-61/ 2022-09-03T05:58:45.036 DEBUG:boto:Data: 2022-09-03T05:58:45.037 DEBUG:boto:Headers: {} 2022-09-03T05:58:45.037 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:58:45.038 DEBUG:boto:Port: 8001 2022-09-03T05:58:45.039 DEBUG:boto:Params: {} 2022-09-03T05:58:45.040 DEBUG:boto:Token: None 2022-09-03T05:58:45.040 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:58:45 GMT /dsgvcy-61/ 2022-09-03T05:58:45.041 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nuSUvYdxkCD+FDfarHmJ0JPK7Cs= 2022-09-03T05:58:45.042 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:58:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nuSUvYdxkCD+FDfarHmJ0JPK7Cs=', 'Content-Length': '0'} 2022-09-03T05:58:45.045 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000766bdfe8eec4b610-006312ed15-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:58:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:45.046 DEBUG:boto:path=/dsgvcy-61/obj1 2022-09-03T05:58:45.047 DEBUG:boto:auth_path=/dsgvcy-61/obj1 2022-09-03T05:58:45.048 DEBUG:boto:Method: PUT 2022-09-03T05:58:45.049 DEBUG:boto:Path: /dsgvcy-61/obj1 2022-09-03T05:58:45.050 DEBUG:boto:Data: 2022-09-03T05:58:45.051 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:58:45.052 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:58:45.053 DEBUG:boto:Port: 8001 2022-09-03T05:58:45.054 DEBUG:boto:Params: {} 2022-09-03T05:58:45.055 DEBUG:boto:Token: None 2022-09-03T05:58:45.055 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:58:45 GMT /dsgvcy-61/obj1 2022-09-03T05:58:45.056 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fYCeqdDjsJ8mkOSM/lfeNTeeNZQ= 2022-09-03T05:58:45.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:58:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fYCeqdDjsJ8mkOSM/lfeNTeeNZQ='} 2022-09-03T05:58:45.063 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000df8a9d658ca02745-006312ed15-15db-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:58:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:45.064 DEBUG:boto:path=/dsgvcy-61/ 2022-09-03T05:58:45.064 DEBUG:boto:auth_path=/dsgvcy-61/ 2022-09-03T05:58:45.065 DEBUG:boto:Method: HEAD 2022-09-03T05:58:45.066 DEBUG:boto:Path: /dsgvcy-61/ 2022-09-03T05:58:45.066 DEBUG:boto:Data: 2022-09-03T05:58:45.067 DEBUG:boto:Headers: {} 2022-09-03T05:58:45.067 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:58:45.068 DEBUG:boto:Port: 8001 2022-09-03T05:58:45.069 DEBUG:boto:Params: {} 2022-09-03T05:58:45.069 DEBUG:boto:Token: None 2022-09-03T05:58:45.070 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:58:45 GMT /dsgvcy-61/ 2022-09-03T05:58:45.070 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nuSUvYdxkCD+FDfarHmJ0JPK7Cs= 2022-09-03T05:58:45.071 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:58:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nuSUvYdxkCD+FDfarHmJ0JPK7Cs=', 'Content-Length': '0'} 2022-09-03T05:58:45.073 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000009cd60786dbbd809-006312ed15-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:58:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:45.074 DEBUG:boto:path=/dsgvcy-61/obj2 2022-09-03T05:58:45.074 DEBUG:boto:auth_path=/dsgvcy-61/obj2 2022-09-03T05:58:45.075 DEBUG:boto:Method: PUT 2022-09-03T05:58:45.075 DEBUG:boto:Path: /dsgvcy-61/obj2 2022-09-03T05:58:45.076 DEBUG:boto:Data: 2022-09-03T05:58:45.076 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:58:45.077 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:58:45.077 DEBUG:boto:Port: 8001 2022-09-03T05:58:45.078 DEBUG:boto:Params: {} 2022-09-03T05:58:45.078 DEBUG:boto:Token: None 2022-09-03T05:58:45.079 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:58:45 GMT /dsgvcy-61/obj2 2022-09-03T05:58:45.080 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:T1S2phZj2HBy4aNXAVHrnsTi1+E= 2022-09-03T05:58:45.080 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:58:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:T1S2phZj2HBy4aNXAVHrnsTi1+E='} 2022-09-03T05:58:45.087 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009f7d6630c58d0c4d-006312ed15-15db-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:58:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:45.089 DEBUG:boto:path=/dsgvcy-61/ 2022-09-03T05:58:45.090 DEBUG:boto:auth_path=/dsgvcy-61/ 2022-09-03T05:58:45.091 DEBUG:boto:Method: HEAD 2022-09-03T05:58:45.092 DEBUG:boto:Path: /dsgvcy-61/ 2022-09-03T05:58:45.093 DEBUG:boto:Data: 2022-09-03T05:58:45.093 DEBUG:boto:Headers: {} 2022-09-03T05:58:45.095 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:58:45.096 DEBUG:boto:Port: 8001 2022-09-03T05:58:45.097 DEBUG:boto:Params: {} 2022-09-03T05:58:45.098 DEBUG:boto:Token: None 2022-09-03T05:58:45.099 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:58:45 GMT /dsgvcy-61/ 2022-09-03T05:58:45.100 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nuSUvYdxkCD+FDfarHmJ0JPK7Cs= 2022-09-03T05:58:45.101 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:58:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nuSUvYdxkCD+FDfarHmJ0JPK7Cs=', 'Content-Length': '0'} 2022-09-03T05:58:45.104 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d156e8397bf7f456-006312ed15-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:58:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:45.105 DEBUG:boto:path=/dsgvcy-61/obj3 2022-09-03T05:58:45.106 DEBUG:boto:auth_path=/dsgvcy-61/obj3 2022-09-03T05:58:45.107 DEBUG:boto:Method: PUT 2022-09-03T05:58:45.107 DEBUG:boto:Path: /dsgvcy-61/obj3 2022-09-03T05:58:45.108 DEBUG:boto:Data: 2022-09-03T05:58:45.108 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:58:45.109 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:58:45.110 DEBUG:boto:Port: 8001 2022-09-03T05:58:45.110 DEBUG:boto:Params: {} 2022-09-03T05:58:45.111 DEBUG:boto:Token: None 2022-09-03T05:58:45.113 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:58:45 GMT /dsgvcy-61/obj3 2022-09-03T05:58:45.114 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+HA/gnBVjcTEO4BAQrGX0aLwMso= 2022-09-03T05:58:45.115 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:58:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+HA/gnBVjcTEO4BAQrGX0aLwMso='} 2022-09-03T05:58:45.122 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c35d5411bb029a70-006312ed15-15db-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:58:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:45.123 DEBUG:boto:path=/dsgvcy-61/ 2022-09-03T05:58:45.124 DEBUG:boto:auth_path=/dsgvcy-61/ 2022-09-03T05:58:45.125 DEBUG:boto:Method: HEAD 2022-09-03T05:58:45.125 DEBUG:boto:Path: /dsgvcy-61/ 2022-09-03T05:58:45.126 DEBUG:boto:Data: 2022-09-03T05:58:45.126 DEBUG:boto:Headers: {} 2022-09-03T05:58:45.127 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:58:45.127 DEBUG:boto:Port: 8001 2022-09-03T05:58:45.128 DEBUG:boto:Params: {} 2022-09-03T05:58:45.128 DEBUG:boto:Token: None 2022-09-03T05:58:45.129 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 05:58:45 GMT /dsgvcy-61/ 2022-09-03T05:58:45.129 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nuSUvYdxkCD+FDfarHmJ0JPK7Cs= 2022-09-03T05:58:45.130 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 05:58:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nuSUvYdxkCD+FDfarHmJ0JPK7Cs=', 'Content-Length': '0'} 2022-09-03T05:58:45.133 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002fe33b731f1bbca8-006312ed15-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 05:58:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:45.133 DEBUG:boto:path=/dsgvcy-61/obj4 2022-09-03T05:58:45.134 DEBUG:boto:auth_path=/dsgvcy-61/obj4 2022-09-03T05:58:45.135 DEBUG:boto:Method: PUT 2022-09-03T05:58:45.135 DEBUG:boto:Path: /dsgvcy-61/obj4 2022-09-03T05:58:45.136 DEBUG:boto:Data: 2022-09-03T05:58:45.136 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T05:58:45.137 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T05:58:45.137 DEBUG:boto:Port: 8001 2022-09-03T05:58:45.138 DEBUG:boto:Params: {} 2022-09-03T05:58:45.138 DEBUG:boto:Token: None 2022-09-03T05:58:45.139 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 05:58:45 GMT /dsgvcy-61/obj4 2022-09-03T05:58:45.140 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LToUtyaqia+XSZGvFxcv4C5uhD8= 2022-09-03T05:58:45.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 05:58:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LToUtyaqia+XSZGvFxcv4C5uhD8='} 2022-09-03T05:58:45.182 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000097fc600b141bcb9e-006312ed15-15db-test-zone3'), ('Date', 'Sat, 03 Sep 2022 05:58:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T05:58:45.183 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:45.184 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:45.184 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:45.233 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:45.234 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:45.287 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T05:58:45.301 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T05:58:45.302 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:45.302 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:58:45.302 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:58:45.302 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:45.303 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:45.303 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:58:45.303 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:58:45.303 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:45.304 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:45.304 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184724.685611_366.1", 2022-09-03T05:58:45.305 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T05:58:44.685611Z" 2022-09-03T05:58:45.306 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:45.306 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:45.307 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:58:45.307 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T05:58:45.308 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:45.308 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T05:58:45.310 DEBUG:rgw_multi.tests:master meta markers={0: '', 1: '', 2: '1_1662184724.685611_366.1', 3: ''} 2022-09-03T05:58:45.310 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T05:58:45.311 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:45.311 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:45.312 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:45.363 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:45.364 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:45.391 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:58:45.389+0000 7f625ade6600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:58:45.392 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:58:45.390+0000 7f625ade6600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:58:45.404 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:58:45.405 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:58:45.405 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:58:45.406 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:58:45.406 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:58:45.406 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:58:45.407 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:58:45.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:45.407 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:58:45.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:45.408 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:58:45.408 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:45.409 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:45.409 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:58:45.410 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:45.410 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:58:45.411 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:45.411 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:58:45.412 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:58:45.412 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:45.413 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:45.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:45.415 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:58:45.416 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:45.416 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:45.417 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:58:45.418 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:45.418 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:45.419 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:45.419 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:58:45.420 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:58:45.421 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:45.421 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:45.422 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:45.422 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:58:45.422 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:45.423 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:45.423 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184724.595420_352.1", 2022-09-03T05:58:45.423 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:45.423 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:45.424 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:45.424 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:58:44.595420Z", 2022-09-03T05:58:45.424 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:58:45.424 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:45.425 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:45.425 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:45.426 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:58:45.427 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:45.427 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:45.427 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:58:45.427 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:45.428 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:45.428 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:45.428 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:58:45.428 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:58:45.429 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:45.429 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:45.429 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:58:45.430 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:45.430 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:58:45.430 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:58:45.431 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:58:45.431 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:45.431 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:58:45Z" 2022-09-03T05:58:45.432 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:58:45.434 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184724.595420_352.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:44.595420Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:58:45Z" } 2022-09-03T05:58:45.435 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184724.595420_352.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:44.595420Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:58:45.435 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184724.685611_366.1', 3: ''} 2022-09-03T05:58:45.436 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184724.595420_352.1', 3: ''} 2022-09-03T05:58:45.437 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662184724.685611_366.1 target=1_1662184724.595420_352.1 2022-09-03T05:58:50.439 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:50.439 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:50.440 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:50.500 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:50.501 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:50.543 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:58:50.541+0000 7f598b3b3600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:58:50.544 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:58:50.542+0000 7f598b3b3600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:58:50.559 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:58:50.560 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:58:50.560 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:58:50.560 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:58:50.561 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:58:50.561 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:58:50.561 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:58:50.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:50.562 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:58:50.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:50.562 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:58:50.562 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:50.563 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:50.563 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:58:50.563 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:50.563 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:58:50.564 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:50.564 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:58:50.564 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:58:50.565 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:50.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:50.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:50.566 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:58:50.567 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:50.567 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:50.567 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:58:50.568 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:50.568 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:50.568 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:50.569 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:58:50.569 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:58:50.569 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:50.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:50.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:50.570 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:58:50.570 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:50.570 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:50.571 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184724.685611_366.1", 2022-09-03T05:58:50.571 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:50.571 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:50.571 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:50.572 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:58:44.685611Z", 2022-09-03T05:58:50.572 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:58:50.572 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:50.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:50.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:50.573 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:58:50.573 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:50.574 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:50.574 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:58:50.574 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:50.574 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:50.575 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:50.575 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:58:50.575 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:58:50.575 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:50.576 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:50.576 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:58:50.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:50.577 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:58:50.577 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:58:50.577 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:58:50.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:50.578 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:58:50Z" 2022-09-03T05:58:50.578 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:58:50.579 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184724.685611_366.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:44.685611Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:58:50Z" } 2022-09-03T05:58:50.581 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184724.685611_366.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:44.685611Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:58:50.581 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184724.685611_366.1', 3: ''} 2022-09-03T05:58:50.582 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184724.685611_366.1', 3: ''} 2022-09-03T05:58:50.582 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T05:58:50.583 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T05:58:50.583 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:50.584 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:50.585 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:50.638 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:50.639 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:50.683 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:58:50.682+0000 7fc732e58600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:58:50.684 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:58:50.683+0000 7fc732e58600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:58:50.699 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T05:58:50.700 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T05:58:50.707 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T05:58:50.707 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T05:58:50.707 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T05:58:50.707 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:58:50.708 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:58:50.708 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:50.708 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T05:58:50.708 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:50.709 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T05:58:50.709 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:50.709 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:58:50.709 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:58:50.710 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:50.711 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T05:58:50.711 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:50.711 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:58:50.711 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:58:50.712 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:50.712 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:50.712 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:50.713 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T05:58:50.713 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:50.713 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:58:50.713 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T05:58:50.714 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:50.714 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:50.714 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:50.714 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T05:58:50.715 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:58:50.715 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:50.715 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:50.715 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:50.716 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T05:58:50.716 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:50.716 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:58:50.716 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662184724.685611_366.1", 2022-09-03T05:58:50.717 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:50.717 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:50.717 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:50.717 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:58:44.685611Z", 2022-09-03T05:58:50.718 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T05:58:50.718 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:50.718 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:50.718 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T05:58:50.719 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T05:58:50.719 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T05:58:50.719 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T05:58:50.719 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:58:50.720 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T05:58:50.720 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T05:58:50.720 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T05:58:50.720 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:58:50.721 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T05:58:50.721 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:50.721 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T05:58:50.721 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T05:58:50.721 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:50.722 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T05:58:50.722 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T05:58:50.722 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T05:58:50.722 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T05:58:50.723 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T05:58:50Z" 2022-09-03T05:58:50.723 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T05:58:50.724 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184724.685611_366.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:44.685611Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:58:50Z" } 2022-09-03T05:58:50.725 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184724.685611_366.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:44.685611Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:58:50.725 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184724.685611_366.1', 3: ''} 2022-09-03T05:58:50.726 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184724.685611_366.1', 3: ''} 2022-09-03T05:58:50.726 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T05:58:50.727 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T05:58:50.727 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:50.728 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T05:58:50.728 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T05:58:50.780 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:50.781 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:50.808 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:58:50.807+0000 7f0912c7f600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:58:50.809 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:58:50.808+0000 7f0912c7f600 20 RGW-SYNC:meta: read sync status 2022-09-03T05:58:50.821 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T05:58:50.822 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T05:58:50.822 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T05:58:50.823 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T05:58:50.823 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T05:58:50.824 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T05:58:50.824 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:58:50.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:50.825 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T05:58:50.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:50.826 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T05:58:50.826 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:50.827 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:50.828 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T05:58:50.829 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:50.829 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T05:58:50.829 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:50.830 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T05:58:50.830 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:58:50.830 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:50.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:50.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:50.831 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T05:58:50.831 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:50.832 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:50.832 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T05:58:50.832 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:50.832 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:50.833 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:50.833 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T05:58:50.834 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:58:50.834 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:50.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:50.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:50.835 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T05:58:50.835 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:50.836 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:50.836 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662184724.685611_366.1", 2022-09-03T05:58:50.836 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:50.836 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:50.837 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:50.837 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:58:44.685611Z", 2022-09-03T05:58:50.837 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T05:58:50.837 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:50.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:50.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T05:58:50.839 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T05:58:50.839 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T05:58:50.840 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T05:58:50.840 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T05:58:50.841 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T05:58:50.841 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T05:58:50.841 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T05:58:50.842 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T05:58:50.842 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T05:58:50.843 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:50.843 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T05:58:50.844 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T05:58:50.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:50.844 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T05:58:50.845 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T05:58:50.845 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T05:58:50.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T05:58:50.846 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T05:58:50Z" 2022-09-03T05:58:50.847 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T05:58:50.849 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662184724.685611_366.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:58:44.685611Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T05:58:50Z" } 2022-09-03T05:58:50.850 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662184724.685611_366.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:58:44.685611Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T05:58:50.850 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662184724.685611_366.1', 3: ''} 2022-09-03T05:58:50.851 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662184724.685611_366.1', 3: ''} 2022-09-03T05:58:50.851 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T05:58:50.852 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-59 2022-09-03T05:58:50.852 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-59', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:58:50.853 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-59', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:58:50.853 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-59 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:58:50.909 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:58:50.910 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:58:50.943 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:58:50.941+0000 7fc4af1bf600 1 waiting to reach incremental sync.. 2022-09-03T05:58:55.943 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:58:55.941+0000 7fc4af1bf600 1 waiting to reach incremental sync.. 2022-09-03T05:59:00.945 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:59:00.942+0000 7fc4af1bf600 1 bucket sync caught up with source: 2022-09-03T05:59:00.946 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T05:59:00.947 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T05:59:00.947 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:59:00.942+0000 7fc4af1bf600 0 bucket checkpoint complete 2022-09-03T05:59:00.964 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-59 2022-09-03T05:59:00.965 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-59', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:59:00.965 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-59', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:59:00.966 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-59 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T05:59:01.030 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:59:01.031 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:59:01.080 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:59:01.079+0000 7f7f2f42f600 1 waiting to reach incremental sync.. 2022-09-03T05:59:06.081 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:59:06.080+0000 7f7f2f42f600 1 waiting to reach incremental sync.. 2022-09-03T05:59:11.082 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:59:11.080+0000 7f7f2f42f600 1 waiting to reach incremental sync.. 2022-09-03T05:59:16.082 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:59:16.080+0000 7f7f2f42f600 1 waiting to reach incremental sync.. 2022-09-03T05:59:21.083 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:59:21.081+0000 7f7f2f42f600 1 waiting to reach incremental sync.. 2022-09-03T05:59:26.083 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:59:26.081+0000 7f7f2f42f600 1 waiting to reach incremental sync.. 2022-09-03T05:59:31.084 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:59:31.082+0000 7f7f2f42f600 1 waiting to reach incremental sync.. 2022-09-03T05:59:36.085 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:59:36.083+0000 7f7f2f42f600 1 waiting to reach incremental sync.. 2022-09-03T05:59:41.087 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:59:41.085+0000 7f7f2f42f600 1 bucket sync caught up with source: 2022-09-03T05:59:41.087 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T05:59:41.088 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T05:59:41.088 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T05:59:41.085+0000 7f7f2f42f600 0 bucket checkpoint complete 2022-09-03T05:59:41.105 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-59 2022-09-03T05:59:41.106 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-59', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:59:41.107 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-59', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T05:59:41.108 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-59 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T05:59:41.172 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T05:59:41.173 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T05:59:41.223 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:59:41.220+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T05:59:41.224 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T05:59:41.224 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T05:59:46.223 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:59:46.221+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T05:59:46.223 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T05:59:46.224 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T05:59:51.224 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:59:51.222+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T05:59:51.225 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T05:59:51.225 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T05:59:56.226 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T05:59:56.223+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T05:59:56.227 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T05:59:56.227 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:00:01.228 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:00:01.226+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:00:01.228 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:00:01.229 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:00:06.229 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:00:06.227+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:00:06.229 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:00:06.229 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:00:11.235 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:00:11.229+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:00:11.235 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:00:11.235 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:00:16.232 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:00:16.230+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:00:16.233 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:00:16.233 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:00:21.234 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:00:21.232+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:00:21.234 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:00:21.235 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:00:26.237 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:00:26.233+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:00:26.238 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:00:26.239 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:00:31.237 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:00:31.235+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:00:31.238 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:00:31.238 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:00:36.238 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:00:36.236+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:00:36.239 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:00:36.239 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:00:41.240 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:00:41.238+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:00:41.240 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:00:41.241 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:00:46.242 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:00:46.238+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:00:46.242 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:00:46.243 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:00:51.243 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:00:51.241+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:00:51.244 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:00:51.244 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:00:56.244 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:00:56.242+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:00:56.245 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:00:56.245 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:01:01.246 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:01:01.244+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:01:01.246 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:01:01.247 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:01:06.248 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:01:06.246+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:01:06.248 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:01:06.248 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:01:11.250 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:01:11.247+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:01:11.250 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:01:11.250 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:01:16.251 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:01:16.248+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:01:16.251 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:01:16.251 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:01:21.252 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:01:21.250+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:01:21.253 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:01:21.253 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:01:26.253 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:01:26.251+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:01:26.254 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:01:26.254 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:01:31.254 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:01:31.253+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:01:31.255 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:01:31.255 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:01:36.257 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:01:36.253+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:01:36.257 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:01:36.257 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:01:41.258 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:01:41.256+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:01:41.258 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:01:41.259 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:01:46.259 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:01:46.257+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:01:46.260 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:01:46.260 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:01:51.261 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:01:51.259+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:01:51.261 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:01:51.261 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:01:56.262 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:01:56.260+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:01:56.262 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:01:56.263 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:02:01.263 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:02:01.261+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:02:01.264 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:02:01.264 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:02:06.265 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:02:06.263+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:02:06.266 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:02:06.267 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:02:11.266 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:02:11.264+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:02:11.267 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:02:11.267 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:02:16.268 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:02:16.266+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:02:16.268 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:02:16.269 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:02:21.269 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:02:21.267+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:02:21.270 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:02:21.270 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:02:26.270 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:02:26.267+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:02:26.271 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:02:26.272 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:02:31.272 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:02:31.269+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:02:31.273 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:02:31.273 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:02:36.273 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:02:36.271+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:02:36.274 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:02:36.275 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:02:41.275 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:02:41.273+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:02:41.275 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:02:41.275 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:02:46.276 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:02:46.274+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:02:46.277 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:02:46.278 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:02:51.278 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:02:51.275+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:02:51.279 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:02:51.280 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:02:56.279 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:02:56.277+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:02:56.280 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:02:56.280 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:03:01.280 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:03:01.278+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:03:01.281 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:03:01.281 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:03:06.282 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:03:06.280+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:03:06.283 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:03:06.283 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:03:11.285 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:03:11.281+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:03:11.285 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:03:11.286 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:03:16.285 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:03:16.283+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:03:16.286 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:03:16.287 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:03:21.286 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:03:21.283+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:03:21.287 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:03:21.288 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:03:26.288 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:03:26.285+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:03:26.289 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:03:26.289 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:03:31.289 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:03:31.287+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:03:31.290 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:03:31.291 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:03:36.291 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:03:36.288+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:03:36.291 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:03:36.291 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:03:41.292 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:03:41.290+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:03:41.292 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:03:41.293 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:03:46.293 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:03:46.291+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:03:46.294 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:03:46.295 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:03:51.295 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:03:51.293+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:03:51.296 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:03:51.296 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:03:56.296 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:03:56.294+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:03:56.297 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:03:56.297 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:04:01.298 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:04:01.296+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:04:01.299 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:04:01.299 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:04:06.299 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:04:06.297+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:04:06.300 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:04:06.300 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:04:11.301 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:04:11.297+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:04:11.302 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:04:11.302 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:04:16.302 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:04:16.300+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:04:16.302 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:04:16.303 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:04:21.303 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:04:21.301+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:04:21.304 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:04:21.305 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:04:26.305 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:04:26.303+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:04:26.305 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:04:26.305 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:04:31.306 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:04:31.304+0000 7fe6fc6d3600 1 waiting for incremental sync to catch up: 2022-09-03T06:04:31.307 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.61.5, , 00000000001.48.5, , , , , , ] 2022-09-03T06:04:31.308 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.40.5, , 00000000001.61.5, , 00000000001.48.5, , , 00000000001.71.5, , , ] 2022-09-03T06:04:36.308 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:04:36.306+0000 7fe6fc6d3600 0 bucket checkpoint timed out waiting for incremental sync to catch up 2022-09-03T06:04:36.309 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:04:36.306+0000 7fe6fc6d3600 0 bucket sync checkpoint failed: (110) Connection timed out 2022-09-03T06:04:36.309 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:04:36.306+0000 7fe6fc6d3600 -1 bucket sync checkpoint failed: (110) Connection timed out 2022-09-03T06:04:36.325 DEBUG:teuthology.orchestra.run:got remote process result: 110 2022-09-03T06:04:36.330 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_sync_enable_right_after_disable ... FAIL 2022-09-03T06:04:36.332 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-62 2022-09-03T06:04:36.333 DEBUG:boto:path=/dsgvcy-62/ 2022-09-03T06:04:36.334 DEBUG:boto:auth_path=/dsgvcy-62/ 2022-09-03T06:04:36.336 DEBUG:boto:Method: PUT 2022-09-03T06:04:36.337 DEBUG:boto:Path: /dsgvcy-62/ 2022-09-03T06:04:36.337 DEBUG:boto:Data: 2022-09-03T06:04:36.338 DEBUG:boto:Headers: {} 2022-09-03T06:04:36.338 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:04:36.339 DEBUG:boto:Port: 8000 2022-09-03T06:04:36.339 DEBUG:boto:Params: {} 2022-09-03T06:04:36.341 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:04:36.341 DEBUG:boto:Token: None 2022-09-03T06:04:36.342 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-62/ 2022-09-03T06:04:36.343 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F4lrztXZ6wAJvlroRdhHP7Y7Kms= 2022-09-03T06:04:36.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F4lrztXZ6wAJvlroRdhHP7Y7Kms=', 'Content-Length': '0'} 2022-09-03T06:04:36.369 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000040e8e884f2504cf9-006312ee74-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.371 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=dsgvcy-63 2022-09-03T06:04:36.372 DEBUG:boto:path=/dsgvcy-63/ 2022-09-03T06:04:36.373 DEBUG:boto:auth_path=/dsgvcy-63/ 2022-09-03T06:04:36.374 DEBUG:boto:Method: PUT 2022-09-03T06:04:36.375 DEBUG:boto:Path: /dsgvcy-63/ 2022-09-03T06:04:36.375 DEBUG:boto:Data: 2022-09-03T06:04:36.376 DEBUG:boto:Headers: {} 2022-09-03T06:04:36.377 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:04:36.378 DEBUG:boto:Port: 8000 2022-09-03T06:04:36.378 DEBUG:boto:Params: {} 2022-09-03T06:04:36.379 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:04:36.380 DEBUG:boto:Token: None 2022-09-03T06:04:36.380 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-63/ 2022-09-03T06:04:36.381 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BGtOGKHnHWPut0uB7buPB04WO44= 2022-09-03T06:04:36.382 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BGtOGKHnHWPut0uB7buPB04WO44=', 'Content-Length': '0'} 2022-09-03T06:04:36.415 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000005034435bed94e575-006312ee74-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.416 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=dsgvcy-64 2022-09-03T06:04:36.417 DEBUG:boto:path=/dsgvcy-64/ 2022-09-03T06:04:36.417 DEBUG:boto:auth_path=/dsgvcy-64/ 2022-09-03T06:04:36.418 DEBUG:boto:Method: PUT 2022-09-03T06:04:36.419 DEBUG:boto:Path: /dsgvcy-64/ 2022-09-03T06:04:36.419 DEBUG:boto:Data: 2022-09-03T06:04:36.420 DEBUG:boto:Headers: {} 2022-09-03T06:04:36.420 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:04:36.421 DEBUG:boto:Port: 8001 2022-09-03T06:04:36.421 DEBUG:boto:Params: {} 2022-09-03T06:04:36.422 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T06:04:36.422 DEBUG:boto:Token: None 2022-09-03T06:04:36.423 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-64/ 2022-09-03T06:04:36.424 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rm7SysFTvw9Bx8aYgFJSEf6nXMY= 2022-09-03T06:04:36.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rm7SysFTvw9Bx8aYgFJSEf6nXMY=', 'Content-Length': '0'} 2022-09-03T06:04:36.466 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000091a024380ad5811a-006312ee74-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.468 DEBUG:boto:path=/dsgvcy-62/ 2022-09-03T06:04:36.469 DEBUG:boto:auth_path=/dsgvcy-62/ 2022-09-03T06:04:36.471 DEBUG:boto:Method: HEAD 2022-09-03T06:04:36.472 DEBUG:boto:Path: /dsgvcy-62/ 2022-09-03T06:04:36.473 DEBUG:boto:Data: 2022-09-03T06:04:36.474 DEBUG:boto:Headers: {} 2022-09-03T06:04:36.475 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:04:36.476 DEBUG:boto:Port: 8000 2022-09-03T06:04:36.477 DEBUG:boto:Params: {} 2022-09-03T06:04:36.478 DEBUG:boto:Token: None 2022-09-03T06:04:36.479 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-62/ 2022-09-03T06:04:36.480 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CuUapKhpbr05DMswQLWyb/IWM7E= 2022-09-03T06:04:36.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CuUapKhpbr05DMswQLWyb/IWM7E=', 'Content-Length': '0'} 2022-09-03T06:04:36.483 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000032ede288f3b04e23-006312ee74-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.484 DEBUG:boto:path=/dsgvcy-62/obj1 2022-09-03T06:04:36.485 DEBUG:boto:auth_path=/dsgvcy-62/obj1 2022-09-03T06:04:36.485 DEBUG:boto:Method: PUT 2022-09-03T06:04:36.486 DEBUG:boto:Path: /dsgvcy-62/obj1 2022-09-03T06:04:36.487 DEBUG:boto:Data: 2022-09-03T06:04:36.487 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T06:04:36.488 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:04:36.488 DEBUG:boto:Port: 8000 2022-09-03T06:04:36.489 DEBUG:boto:Params: {} 2022-09-03T06:04:36.489 DEBUG:boto:Token: None 2022-09-03T06:04:36.490 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-62/obj1 2022-09-03T06:04:36.491 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3AfGJK+c3hUoSuJA3lF3HZ76Eqs= 2022-09-03T06:04:36.491 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3AfGJK+c3hUoSuJA3lF3HZ76Eqs='} 2022-09-03T06:04:36.501 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f4cf92cb7c225e92-006312ee74-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.502 DEBUG:boto:path=/dsgvcy-62/ 2022-09-03T06:04:36.502 DEBUG:boto:auth_path=/dsgvcy-62/ 2022-09-03T06:04:36.503 DEBUG:boto:Method: HEAD 2022-09-03T06:04:36.504 DEBUG:boto:Path: /dsgvcy-62/ 2022-09-03T06:04:36.504 DEBUG:boto:Data: 2022-09-03T06:04:36.505 DEBUG:boto:Headers: {} 2022-09-03T06:04:36.505 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:04:36.506 DEBUG:boto:Port: 8000 2022-09-03T06:04:36.506 DEBUG:boto:Params: {} 2022-09-03T06:04:36.507 DEBUG:boto:Token: None 2022-09-03T06:04:36.508 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-62/ 2022-09-03T06:04:36.508 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CuUapKhpbr05DMswQLWyb/IWM7E= 2022-09-03T06:04:36.509 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CuUapKhpbr05DMswQLWyb/IWM7E=', 'Content-Length': '0'} 2022-09-03T06:04:36.511 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a42560d8935f7da4-006312ee74-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.512 DEBUG:boto:path=/dsgvcy-62/obj2 2022-09-03T06:04:36.513 DEBUG:boto:auth_path=/dsgvcy-62/obj2 2022-09-03T06:04:36.513 DEBUG:boto:Method: PUT 2022-09-03T06:04:36.514 DEBUG:boto:Path: /dsgvcy-62/obj2 2022-09-03T06:04:36.514 DEBUG:boto:Data: 2022-09-03T06:04:36.515 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T06:04:36.515 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:04:36.516 DEBUG:boto:Port: 8000 2022-09-03T06:04:36.516 DEBUG:boto:Params: {} 2022-09-03T06:04:36.517 DEBUG:boto:Token: None 2022-09-03T06:04:36.518 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-62/obj2 2022-09-03T06:04:36.518 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bHDh6bd+g7nIn0JWl+Rm0qmXJpU= 2022-09-03T06:04:36.519 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bHDh6bd+g7nIn0JWl+Rm0qmXJpU='} 2022-09-03T06:04:36.562 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c53a2999461cac04-006312ee74-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.563 DEBUG:boto:path=/dsgvcy-62/ 2022-09-03T06:04:36.564 DEBUG:boto:auth_path=/dsgvcy-62/ 2022-09-03T06:04:36.565 DEBUG:boto:Method: HEAD 2022-09-03T06:04:36.566 DEBUG:boto:Path: /dsgvcy-62/ 2022-09-03T06:04:36.567 DEBUG:boto:Data: 2022-09-03T06:04:36.567 DEBUG:boto:Headers: {} 2022-09-03T06:04:36.568 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:04:36.569 DEBUG:boto:Port: 8000 2022-09-03T06:04:36.570 DEBUG:boto:Params: {} 2022-09-03T06:04:36.571 DEBUG:boto:Token: None 2022-09-03T06:04:36.572 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-62/ 2022-09-03T06:04:36.573 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CuUapKhpbr05DMswQLWyb/IWM7E= 2022-09-03T06:04:36.574 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CuUapKhpbr05DMswQLWyb/IWM7E=', 'Content-Length': '0'} 2022-09-03T06:04:36.577 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009325d146e3083600-006312ee74-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.578 DEBUG:boto:path=/dsgvcy-62/obj3 2022-09-03T06:04:36.579 DEBUG:boto:auth_path=/dsgvcy-62/obj3 2022-09-03T06:04:36.579 DEBUG:boto:Method: PUT 2022-09-03T06:04:36.580 DEBUG:boto:Path: /dsgvcy-62/obj3 2022-09-03T06:04:36.580 DEBUG:boto:Data: 2022-09-03T06:04:36.581 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T06:04:36.581 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:04:36.582 DEBUG:boto:Port: 8000 2022-09-03T06:04:36.583 DEBUG:boto:Params: {} 2022-09-03T06:04:36.583 DEBUG:boto:Token: None 2022-09-03T06:04:36.584 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-62/obj3 2022-09-03T06:04:36.585 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QWmdvC2cqOQBbl44hB60wSDAO0U= 2022-09-03T06:04:36.585 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QWmdvC2cqOQBbl44hB60wSDAO0U='} 2022-09-03T06:04:36.606 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000058a67d69524e9e86-006312ee74-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.608 DEBUG:boto:path=/dsgvcy-62/ 2022-09-03T06:04:36.608 DEBUG:boto:auth_path=/dsgvcy-62/ 2022-09-03T06:04:36.609 DEBUG:boto:Method: HEAD 2022-09-03T06:04:36.609 DEBUG:boto:Path: /dsgvcy-62/ 2022-09-03T06:04:36.610 DEBUG:boto:Data: 2022-09-03T06:04:36.610 DEBUG:boto:Headers: {} 2022-09-03T06:04:36.611 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:04:36.611 DEBUG:boto:Port: 8000 2022-09-03T06:04:36.612 DEBUG:boto:Params: {} 2022-09-03T06:04:36.613 DEBUG:boto:Token: None 2022-09-03T06:04:36.613 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-62/ 2022-09-03T06:04:36.614 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CuUapKhpbr05DMswQLWyb/IWM7E= 2022-09-03T06:04:36.615 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CuUapKhpbr05DMswQLWyb/IWM7E=', 'Content-Length': '0'} 2022-09-03T06:04:36.617 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006fb0ab4830b9159e-006312ee74-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.618 DEBUG:boto:path=/dsgvcy-62/obj4 2022-09-03T06:04:36.619 DEBUG:boto:auth_path=/dsgvcy-62/obj4 2022-09-03T06:04:36.619 DEBUG:boto:Method: PUT 2022-09-03T06:04:36.620 DEBUG:boto:Path: /dsgvcy-62/obj4 2022-09-03T06:04:36.620 DEBUG:boto:Data: 2022-09-03T06:04:36.621 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T06:04:36.621 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:04:36.622 DEBUG:boto:Port: 8000 2022-09-03T06:04:36.623 DEBUG:boto:Params: {} 2022-09-03T06:04:36.623 DEBUG:boto:Token: None 2022-09-03T06:04:36.624 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-62/obj4 2022-09-03T06:04:36.624 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qjFAztL0aXHT2qg53J9nwfyXXLc= 2022-09-03T06:04:36.625 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qjFAztL0aXHT2qg53J9nwfyXXLc='} 2022-09-03T06:04:36.633 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f277875540d1aa0f-006312ee74-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.634 DEBUG:boto:path=/dsgvcy-63/ 2022-09-03T06:04:36.635 DEBUG:boto:auth_path=/dsgvcy-63/ 2022-09-03T06:04:36.635 DEBUG:boto:Method: HEAD 2022-09-03T06:04:36.636 DEBUG:boto:Path: /dsgvcy-63/ 2022-09-03T06:04:36.636 DEBUG:boto:Data: 2022-09-03T06:04:36.637 DEBUG:boto:Headers: {} 2022-09-03T06:04:36.638 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:04:36.638 DEBUG:boto:Port: 8000 2022-09-03T06:04:36.639 DEBUG:boto:Params: {} 2022-09-03T06:04:36.640 DEBUG:boto:Token: None 2022-09-03T06:04:36.640 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-63/ 2022-09-03T06:04:36.641 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uelrAtFdWGCgATh+u9HX4CM7zUc= 2022-09-03T06:04:36.642 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uelrAtFdWGCgATh+u9HX4CM7zUc=', 'Content-Length': '0'} 2022-09-03T06:04:36.644 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a373a5c9ed46bd97-006312ee74-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.645 DEBUG:boto:path=/dsgvcy-63/obj1 2022-09-03T06:04:36.646 DEBUG:boto:auth_path=/dsgvcy-63/obj1 2022-09-03T06:04:36.646 DEBUG:boto:Method: PUT 2022-09-03T06:04:36.647 DEBUG:boto:Path: /dsgvcy-63/obj1 2022-09-03T06:04:36.647 DEBUG:boto:Data: 2022-09-03T06:04:36.648 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T06:04:36.649 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:04:36.649 DEBUG:boto:Port: 8000 2022-09-03T06:04:36.650 DEBUG:boto:Params: {} 2022-09-03T06:04:36.651 DEBUG:boto:Token: None 2022-09-03T06:04:36.651 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-63/obj1 2022-09-03T06:04:36.652 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tB8/R0oNjAYs31o9tWqInmwWCck= 2022-09-03T06:04:36.653 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tB8/R0oNjAYs31o9tWqInmwWCck='} 2022-09-03T06:04:36.661 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001c25b60aef0e9eb3-006312ee74-15d2-test-zone2'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.661 DEBUG:boto:path=/dsgvcy-63/ 2022-09-03T06:04:36.662 DEBUG:boto:auth_path=/dsgvcy-63/ 2022-09-03T06:04:36.662 DEBUG:boto:Method: HEAD 2022-09-03T06:04:36.663 DEBUG:boto:Path: /dsgvcy-63/ 2022-09-03T06:04:36.663 DEBUG:boto:Data: 2022-09-03T06:04:36.664 DEBUG:boto:Headers: {} 2022-09-03T06:04:36.664 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:04:36.665 DEBUG:boto:Port: 8000 2022-09-03T06:04:36.665 DEBUG:boto:Params: {} 2022-09-03T06:04:36.666 DEBUG:boto:Token: None 2022-09-03T06:04:36.667 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-63/ 2022-09-03T06:04:36.667 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uelrAtFdWGCgATh+u9HX4CM7zUc= 2022-09-03T06:04:36.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uelrAtFdWGCgATh+u9HX4CM7zUc=', 'Content-Length': '0'} 2022-09-03T06:04:36.670 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d0bc758c46a446b6-006312ee74-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.671 DEBUG:boto:path=/dsgvcy-63/obj2 2022-09-03T06:04:36.672 DEBUG:boto:auth_path=/dsgvcy-63/obj2 2022-09-03T06:04:36.672 DEBUG:boto:Method: PUT 2022-09-03T06:04:36.673 DEBUG:boto:Path: /dsgvcy-63/obj2 2022-09-03T06:04:36.674 DEBUG:boto:Data: 2022-09-03T06:04:36.674 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T06:04:36.675 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:04:36.676 DEBUG:boto:Port: 8000 2022-09-03T06:04:36.676 DEBUG:boto:Params: {} 2022-09-03T06:04:36.677 DEBUG:boto:Token: None 2022-09-03T06:04:36.677 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-63/obj2 2022-09-03T06:04:36.678 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FfGWcN8HD/KrOfdeLCL5cKIob8Q= 2022-09-03T06:04:36.679 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FfGWcN8HD/KrOfdeLCL5cKIob8Q='} 2022-09-03T06:04:36.687 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000210eadd191311843-006312ee74-15d2-test-zone2'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.688 DEBUG:boto:path=/dsgvcy-63/ 2022-09-03T06:04:36.689 DEBUG:boto:auth_path=/dsgvcy-63/ 2022-09-03T06:04:36.689 DEBUG:boto:Method: HEAD 2022-09-03T06:04:36.690 DEBUG:boto:Path: /dsgvcy-63/ 2022-09-03T06:04:36.690 DEBUG:boto:Data: 2022-09-03T06:04:36.691 DEBUG:boto:Headers: {} 2022-09-03T06:04:36.691 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:04:36.692 DEBUG:boto:Port: 8000 2022-09-03T06:04:36.693 DEBUG:boto:Params: {} 2022-09-03T06:04:36.694 DEBUG:boto:Token: None 2022-09-03T06:04:36.694 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-63/ 2022-09-03T06:04:36.695 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uelrAtFdWGCgATh+u9HX4CM7zUc= 2022-09-03T06:04:36.695 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uelrAtFdWGCgATh+u9HX4CM7zUc=', 'Content-Length': '0'} 2022-09-03T06:04:36.698 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005771baf7798e1255-006312ee74-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.699 DEBUG:boto:path=/dsgvcy-63/obj3 2022-09-03T06:04:36.699 DEBUG:boto:auth_path=/dsgvcy-63/obj3 2022-09-03T06:04:36.700 DEBUG:boto:Method: PUT 2022-09-03T06:04:36.700 DEBUG:boto:Path: /dsgvcy-63/obj3 2022-09-03T06:04:36.701 DEBUG:boto:Data: 2022-09-03T06:04:36.701 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T06:04:36.702 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:04:36.702 DEBUG:boto:Port: 8000 2022-09-03T06:04:36.703 DEBUG:boto:Params: {} 2022-09-03T06:04:36.703 DEBUG:boto:Token: None 2022-09-03T06:04:36.704 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-63/obj3 2022-09-03T06:04:36.705 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PFpPYYBHVSK0sJLaE9EOaeSxJt8= 2022-09-03T06:04:36.706 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PFpPYYBHVSK0sJLaE9EOaeSxJt8='} 2022-09-03T06:04:36.750 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000498891e024211cc5-006312ee74-15d2-test-zone2'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.752 DEBUG:boto:path=/dsgvcy-63/ 2022-09-03T06:04:36.752 DEBUG:boto:auth_path=/dsgvcy-63/ 2022-09-03T06:04:36.753 DEBUG:boto:Method: HEAD 2022-09-03T06:04:36.754 DEBUG:boto:Path: /dsgvcy-63/ 2022-09-03T06:04:36.754 DEBUG:boto:Data: 2022-09-03T06:04:36.755 DEBUG:boto:Headers: {} 2022-09-03T06:04:36.755 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:04:36.756 DEBUG:boto:Port: 8000 2022-09-03T06:04:36.756 DEBUG:boto:Params: {} 2022-09-03T06:04:36.757 DEBUG:boto:Token: None 2022-09-03T06:04:36.757 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-63/ 2022-09-03T06:04:36.758 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uelrAtFdWGCgATh+u9HX4CM7zUc= 2022-09-03T06:04:36.759 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uelrAtFdWGCgATh+u9HX4CM7zUc=', 'Content-Length': '0'} 2022-09-03T06:04:36.762 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000044b83586a0a49976-006312ee74-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.763 DEBUG:boto:path=/dsgvcy-63/obj4 2022-09-03T06:04:36.763 DEBUG:boto:auth_path=/dsgvcy-63/obj4 2022-09-03T06:04:36.764 DEBUG:boto:Method: PUT 2022-09-03T06:04:36.764 DEBUG:boto:Path: /dsgvcy-63/obj4 2022-09-03T06:04:36.765 DEBUG:boto:Data: 2022-09-03T06:04:36.765 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T06:04:36.766 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:04:36.766 DEBUG:boto:Port: 8000 2022-09-03T06:04:36.767 DEBUG:boto:Params: {} 2022-09-03T06:04:36.767 DEBUG:boto:Token: None 2022-09-03T06:04:36.768 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-63/obj4 2022-09-03T06:04:36.769 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:begz+sjsxk9iitLnTD/zwOKfZa8= 2022-09-03T06:04:36.769 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:begz+sjsxk9iitLnTD/zwOKfZa8='} 2022-09-03T06:04:36.776 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000cb9ee08f28d977af-006312ee74-15d2-test-zone2'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.777 DEBUG:boto:path=/dsgvcy-64/ 2022-09-03T06:04:36.777 DEBUG:boto:auth_path=/dsgvcy-64/ 2022-09-03T06:04:36.778 DEBUG:boto:Method: HEAD 2022-09-03T06:04:36.778 DEBUG:boto:Path: /dsgvcy-64/ 2022-09-03T06:04:36.779 DEBUG:boto:Data: 2022-09-03T06:04:36.779 DEBUG:boto:Headers: {} 2022-09-03T06:04:36.780 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:04:36.781 DEBUG:boto:Port: 8001 2022-09-03T06:04:36.781 DEBUG:boto:Params: {} 2022-09-03T06:04:36.782 DEBUG:boto:Token: None 2022-09-03T06:04:36.782 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-64/ 2022-09-03T06:04:36.783 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4K6Kh3Jz8mheYfxL4lOlAWAkjtI= 2022-09-03T06:04:36.783 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4K6Kh3Jz8mheYfxL4lOlAWAkjtI=', 'Content-Length': '0'} 2022-09-03T06:04:36.786 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b194fe3513295da9-006312ee74-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.787 DEBUG:boto:path=/dsgvcy-64/obj1 2022-09-03T06:04:36.787 DEBUG:boto:auth_path=/dsgvcy-64/obj1 2022-09-03T06:04:36.788 DEBUG:boto:Method: PUT 2022-09-03T06:04:36.788 DEBUG:boto:Path: /dsgvcy-64/obj1 2022-09-03T06:04:36.789 DEBUG:boto:Data: 2022-09-03T06:04:36.789 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T06:04:36.790 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:04:36.790 DEBUG:boto:Port: 8001 2022-09-03T06:04:36.791 DEBUG:boto:Params: {} 2022-09-03T06:04:36.791 DEBUG:boto:Token: None 2022-09-03T06:04:36.792 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-64/obj1 2022-09-03T06:04:36.793 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wwx0RGPO3WsYh7XDiWHXr3WwAeY= 2022-09-03T06:04:36.793 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wwx0RGPO3WsYh7XDiWHXr3WwAeY='} 2022-09-03T06:04:36.801 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f1dd84c544fd3bd1-006312ee74-15db-test-zone3'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.802 DEBUG:boto:path=/dsgvcy-64/ 2022-09-03T06:04:36.802 DEBUG:boto:auth_path=/dsgvcy-64/ 2022-09-03T06:04:36.803 DEBUG:boto:Method: HEAD 2022-09-03T06:04:36.803 DEBUG:boto:Path: /dsgvcy-64/ 2022-09-03T06:04:36.804 DEBUG:boto:Data: 2022-09-03T06:04:36.804 DEBUG:boto:Headers: {} 2022-09-03T06:04:36.805 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:04:36.805 DEBUG:boto:Port: 8001 2022-09-03T06:04:36.806 DEBUG:boto:Params: {} 2022-09-03T06:04:36.807 DEBUG:boto:Token: None 2022-09-03T06:04:36.807 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-64/ 2022-09-03T06:04:36.808 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4K6Kh3Jz8mheYfxL4lOlAWAkjtI= 2022-09-03T06:04:36.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4K6Kh3Jz8mheYfxL4lOlAWAkjtI=', 'Content-Length': '0'} 2022-09-03T06:04:36.811 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001a95e9f3660d97ce-006312ee74-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.811 DEBUG:boto:path=/dsgvcy-64/obj2 2022-09-03T06:04:36.812 DEBUG:boto:auth_path=/dsgvcy-64/obj2 2022-09-03T06:04:36.812 DEBUG:boto:Method: PUT 2022-09-03T06:04:36.813 DEBUG:boto:Path: /dsgvcy-64/obj2 2022-09-03T06:04:36.813 DEBUG:boto:Data: 2022-09-03T06:04:36.814 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T06:04:36.814 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:04:36.815 DEBUG:boto:Port: 8001 2022-09-03T06:04:36.815 DEBUG:boto:Params: {} 2022-09-03T06:04:36.816 DEBUG:boto:Token: None 2022-09-03T06:04:36.816 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-64/obj2 2022-09-03T06:04:36.817 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PB470yKcxAXF5mhuMp9GvIsZjhY= 2022-09-03T06:04:36.818 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PB470yKcxAXF5mhuMp9GvIsZjhY='} 2022-09-03T06:04:36.862 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000996c8a3c641f53e7-006312ee74-15db-test-zone3'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.863 DEBUG:boto:path=/dsgvcy-64/ 2022-09-03T06:04:36.864 DEBUG:boto:auth_path=/dsgvcy-64/ 2022-09-03T06:04:36.864 DEBUG:boto:Method: HEAD 2022-09-03T06:04:36.865 DEBUG:boto:Path: /dsgvcy-64/ 2022-09-03T06:04:36.866 DEBUG:boto:Data: 2022-09-03T06:04:36.867 DEBUG:boto:Headers: {} 2022-09-03T06:04:36.868 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:04:36.869 DEBUG:boto:Port: 8001 2022-09-03T06:04:36.869 DEBUG:boto:Params: {} 2022-09-03T06:04:36.870 DEBUG:boto:Token: None 2022-09-03T06:04:36.871 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-64/ 2022-09-03T06:04:36.871 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4K6Kh3Jz8mheYfxL4lOlAWAkjtI= 2022-09-03T06:04:36.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4K6Kh3Jz8mheYfxL4lOlAWAkjtI=', 'Content-Length': '0'} 2022-09-03T06:04:36.874 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002d4a1072fdd44bfe-006312ee74-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.875 DEBUG:boto:path=/dsgvcy-64/obj3 2022-09-03T06:04:36.876 DEBUG:boto:auth_path=/dsgvcy-64/obj3 2022-09-03T06:04:36.876 DEBUG:boto:Method: PUT 2022-09-03T06:04:36.877 DEBUG:boto:Path: /dsgvcy-64/obj3 2022-09-03T06:04:36.877 DEBUG:boto:Data: 2022-09-03T06:04:36.877 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T06:04:36.878 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:04:36.878 DEBUG:boto:Port: 8001 2022-09-03T06:04:36.879 DEBUG:boto:Params: {} 2022-09-03T06:04:36.879 DEBUG:boto:Token: None 2022-09-03T06:04:36.880 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-64/obj3 2022-09-03T06:04:36.881 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ePcIr8uOGNpnEtDaYR7+xPiVXoY= 2022-09-03T06:04:36.881 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ePcIr8uOGNpnEtDaYR7+xPiVXoY='} 2022-09-03T06:04:36.887 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000006fcbac9c0a357402-006312ee74-15db-test-zone3'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.888 DEBUG:boto:path=/dsgvcy-64/ 2022-09-03T06:04:36.888 DEBUG:boto:auth_path=/dsgvcy-64/ 2022-09-03T06:04:36.889 DEBUG:boto:Method: HEAD 2022-09-03T06:04:36.890 DEBUG:boto:Path: /dsgvcy-64/ 2022-09-03T06:04:36.890 DEBUG:boto:Data: 2022-09-03T06:04:36.891 DEBUG:boto:Headers: {} 2022-09-03T06:04:36.891 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:04:36.892 DEBUG:boto:Port: 8001 2022-09-03T06:04:36.892 DEBUG:boto:Params: {} 2022-09-03T06:04:36.893 DEBUG:boto:Token: None 2022-09-03T06:04:36.893 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-64/ 2022-09-03T06:04:36.894 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4K6Kh3Jz8mheYfxL4lOlAWAkjtI= 2022-09-03T06:04:36.895 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4K6Kh3Jz8mheYfxL4lOlAWAkjtI=', 'Content-Length': '0'} 2022-09-03T06:04:36.897 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fd12d47d4a317a80-006312ee74-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.897 DEBUG:boto:path=/dsgvcy-64/obj4 2022-09-03T06:04:36.898 DEBUG:boto:auth_path=/dsgvcy-64/obj4 2022-09-03T06:04:36.899 DEBUG:boto:Method: PUT 2022-09-03T06:04:36.899 DEBUG:boto:Path: /dsgvcy-64/obj4 2022-09-03T06:04:36.900 DEBUG:boto:Data: 2022-09-03T06:04:36.900 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue'} 2022-09-03T06:04:36.901 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:04:36.901 DEBUG:boto:Port: 8001 2022-09-03T06:04:36.902 DEBUG:boto:Params: {} 2022-09-03T06:04:36.903 DEBUG:boto:Token: None 2022-09-03T06:04:36.904 DEBUG:boto:StringToSign: PUT qPXxZ/RPSWTmyZje6CcRDA== application/octet-stream Sat, 03 Sep 2022 06:04:36 GMT /dsgvcy-64/obj4 2022-09-03T06:04:36.905 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:syIwveuRf+K4WOL+V3jYpln/bI4= 2022-09-03T06:04:36.906 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'qPXxZ/RPSWTmyZje6CcRDA==', 'Content-Length': '6', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:04:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:syIwveuRf+K4WOL+V3jYpln/bI4='} 2022-09-03T06:04:36.950 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"a8f5f167f44f4964e6c998dee827110c"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000e6499c31bc383928-006312ee74-15db-test-zone3'), ('Date', 'Sat, 03 Sep 2022 06:04:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:04:36.951 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:04:36.952 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:04:36.954 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:04:37.005 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:04:37.006 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:04:37.044 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T06:04:37.058 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T06:04:37.059 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:04:37.059 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:04:37.060 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:04:37.060 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:04:37.060 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:04:37.060 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:04:37.061 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:04:37.061 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:04:37.061 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:04:37.061 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662185076.448640_394.1", 2022-09-03T06:04:37.062 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:04:36.448640Z" 2022-09-03T06:04:37.062 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:04:37.062 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:04:37.063 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:04:37.063 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:04:37.063 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:04:37.064 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T06:04:37.065 DEBUG:rgw_multi.tests:master meta markers={0: '', 1: '', 2: '1_1662185076.448640_394.1', 3: ''} 2022-09-03T06:04:37.066 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T06:04:37.066 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:04:37.067 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:04:37.068 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:04:37.121 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:04:37.122 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:04:37.148 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:04:37.146+0000 7f6402b6f600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:04:37.149 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:04:37.147+0000 7f6402b6f600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:04:37.162 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:04:37.162 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:04:37.162 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:04:37.163 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:04:37.163 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:04:37.163 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:04:37.164 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:04:37.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:04:37.165 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:04:37.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:04:37.167 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:04:37.168 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:04:37.168 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:04:37.168 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:04:37.169 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:04:37.169 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:04:37.170 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:04:37.170 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:04:37.171 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:04:37.171 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:04:37.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:04:37.172 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:04:37.172 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:04:37.173 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:04:37.173 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:04:37.173 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:04:37.174 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:04:37.174 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:04:37.175 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:04:37.175 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:04:37.176 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:04:37.176 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:04:37.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:04:37.177 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:04:37.177 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:04:37.178 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:04:37.178 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:04:37.178 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662185076.448640_394.1", 2022-09-03T06:04:37.179 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:04:37.179 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:04:37.179 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:04:37.180 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:04:36.448640Z", 2022-09-03T06:04:37.181 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:04:37.181 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:04:37.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:04:37.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:04:37.183 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:04:37.183 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:04:37.184 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:04:37.184 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:04:37.185 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:04:37.185 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:04:37.186 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:04:37.186 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:04:37.187 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:04:37.187 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:04:37.188 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:04:37.188 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:04:37.189 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:04:37.189 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:04:37.190 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:04:37.190 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:04:37.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:04:37.191 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:04:37Z" 2022-09-03T06:04:37.192 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:04:37.194 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185076.448640_394.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:04:36.448640Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:04:37Z" } 2022-09-03T06:04:37.195 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185076.448640_394.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:04:36.448640Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:04:37.195 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185076.448640_394.1', 3: ''} 2022-09-03T06:04:37.196 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185076.448640_394.1', 3: ''} 2022-09-03T06:04:37.196 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T06:04:37.197 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T06:04:37.197 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:04:37.198 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:04:37.198 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:04:37.248 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:04:37.248 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:04:37.298 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:04:37.296+0000 7f47b11b7600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:04:37.299 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:04:37.297+0000 7f47b11b7600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:04:37.320 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T06:04:37.320 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T06:04:37.321 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T06:04:37.321 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T06:04:37.321 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T06:04:37.322 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:04:37.322 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:04:37.322 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:04:37.322 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T06:04:37.323 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:04:37.323 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T06:04:37.323 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:04:37.324 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:04:37.324 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:04:37.324 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:04:37.324 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T06:04:37.325 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:04:37.325 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:04:37.325 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:04:37.326 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:04:37.326 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:04:37.326 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:04:37.326 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T06:04:37.327 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:04:37.327 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:04:37.327 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:04:37.327 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:04:37.328 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:04:37.328 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:04:37.328 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T06:04:37.328 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:04:37.329 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:04:37.329 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:04:37.329 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:04:37.329 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T06:04:37.330 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:04:37.330 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:04:37.330 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662185076.357192_383.1", 2022-09-03T06:04:37.330 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:04:37.331 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:04:37.331 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:04:37.331 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:04:36.357192Z", 2022-09-03T06:04:37.331 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:04:37.332 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:04:37.332 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:04:37.332 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:04:37.332 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T06:04:37.333 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:04:37.333 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:04:37.333 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:04:37.333 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:04:37.334 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:04:37.334 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:04:37.334 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:04:37.335 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:04:37.335 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:04:37.335 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:04:37.335 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:04:37.336 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:04:37.336 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T06:04:37.336 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T06:04:37.336 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T06:04:37.337 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:04:37.337 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T06:04:37Z" 2022-09-03T06:04:37.337 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T06:04:37.338 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185076.357192_383.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:04:36.357192Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:04:37Z" } 2022-09-03T06:04:37.339 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185076.357192_383.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:04:36.357192Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:04:37.340 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185076.448640_394.1', 3: ''} 2022-09-03T06:04:37.340 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185076.357192_383.1', 3: ''} 2022-09-03T06:04:37.341 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662185076.448640_394.1 target=1_1662185076.357192_383.1 2022-09-03T06:04:42.342 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:04:42.343 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:04:42.344 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:04:42.401 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:04:42.402 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:04:42.431 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:04:42.430+0000 7f193d439600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:04:42.432 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:04:42.431+0000 7f193d439600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:04:42.446 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T06:04:42.446 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T06:04:42.447 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T06:04:42.447 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T06:04:42.447 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T06:04:42.447 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:04:42.448 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:04:42.448 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:04:42.448 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T06:04:42.449 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:04:42.449 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T06:04:42.449 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:04:42.449 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:04:42.450 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:04:42.450 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:04:42.450 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T06:04:42.451 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:04:42.451 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:04:42.451 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:04:42.451 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:04:42.452 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:04:42.452 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:04:42.452 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T06:04:42.452 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:04:42.453 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:04:42.453 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:04:42.453 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:04:42.453 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:04:42.454 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:04:42.454 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T06:04:42.454 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:04:42.455 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:04:42.455 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:04:42.455 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:04:42.455 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T06:04:42.456 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:04:42.456 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:04:42.456 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662185076.448640_394.1", 2022-09-03T06:04:42.456 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:04:42.457 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:04:42.457 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:04:42.457 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:04:36.448640Z", 2022-09-03T06:04:42.458 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:04:42.458 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:04:42.458 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:04:42.458 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:04:42.459 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T06:04:42.459 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:04:42.459 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:04:42.459 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:04:42.460 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:04:42.460 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:04:42.460 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:04:42.460 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:04:42.461 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:04:42.461 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:04:42.461 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:04:42.462 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:04:42.462 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:04:42.462 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T06:04:42.462 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T06:04:42.463 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T06:04:42.463 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:04:42.463 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T06:04:42Z" 2022-09-03T06:04:42.463 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T06:04:42.465 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185076.448640_394.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:04:36.448640Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:04:42Z" } 2022-09-03T06:04:42.465 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185076.448640_394.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:04:36.448640Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:04:42.466 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185076.448640_394.1', 3: ''} 2022-09-03T06:04:42.466 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185076.448640_394.1', 3: ''} 2022-09-03T06:04:42.467 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T06:04:42.468 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T06:04:42.468 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:04:42.469 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:04:42.469 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:04:42.525 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:04:42.526 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:04:42.566 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:04:42.564+0000 7ffbe7080600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:04:42.567 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:04:42.565+0000 7ffbe7080600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:04:42.579 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:04:42.580 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:04:42.580 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:04:42.581 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:04:42.581 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:04:42.582 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:04:42.582 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:04:42.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:04:42.582 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:04:42.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:04:42.583 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:04:42.583 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:04:42.584 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:04:42.584 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:04:42.584 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:04:42.585 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:04:42.585 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:04:42.585 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:04:42.585 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:04:42.586 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:04:42.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:04:42.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:04:42.587 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:04:42.587 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:04:42.587 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:04:42.587 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:04:42.588 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:04:42.588 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:04:42.588 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:04:42.588 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:04:42.589 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:04:42.589 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:04:42.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:04:42.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:04:42.590 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:04:42.590 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:04:42.590 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:04:42.591 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662185076.448640_394.1", 2022-09-03T06:04:42.591 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:04:42.591 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:04:42.592 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:04:42.592 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:04:36.448640Z", 2022-09-03T06:04:42.592 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:04:42.592 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:04:42.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:04:42.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:04:42.593 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:04:42.593 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:04:42.594 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:04:42.594 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:04:42.595 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:04:42.595 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:04:42.596 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:04:42.596 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:04:42.596 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:04:42.597 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:04:42.597 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:04:42.597 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:04:42.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:04:42.598 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:04:42.598 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:04:42.599 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:04:42.599 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:04:42.599 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:04:42Z" 2022-09-03T06:04:42.600 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:04:42.601 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185076.448640_394.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:04:36.448640Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:04:42Z" } 2022-09-03T06:04:42.602 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185076.448640_394.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:04:36.448640Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:04:42.602 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185076.448640_394.1', 3: ''} 2022-09-03T06:04:42.603 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185076.448640_394.1', 3: ''} 2022-09-03T06:04:42.604 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T06:04:42.605 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-62 2022-09-03T06:04:42.606 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-62', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:04:42.607 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-62', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:04:42.608 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-62 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:04:42.665 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:04:42.666 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:04:42.700 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:04:42.698+0000 7f964d030600 1 bucket sync caught up with source: 2022-09-03T06:04:42.700 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:04:42.701 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:04:42.701 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:04:42.698+0000 7f964d030600 0 bucket checkpoint complete 2022-09-03T06:04:42.713 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-62 2022-09-03T06:04:42.714 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-62', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:04:42.715 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-62', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:04:42.715 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-62 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:04:42.775 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:04:42.775 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:04:42.812 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:04:42.811+0000 7f581dd9e600 1 bucket sync caught up with source: 2022-09-03T06:04:42.813 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:04:42.813 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:04:42.813 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:04:42.811+0000 7f581dd9e600 0 bucket checkpoint complete 2022-09-03T06:04:42.838 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-62 2022-09-03T06:04:42.839 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-62', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:04:42.839 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-62', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:04:42.840 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-62 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:04:42.896 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:04:42.897 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:04:42.937 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:04:42.935+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:04:42.938 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , 00000000001.41.5, , , , , , ] 2022-09-03T06:04:42.938 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:04:47.939 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:04:47.937+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:04:47.939 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , 00000000001.41.5, , , , , , ] 2022-09-03T06:04:47.939 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:04:52.940 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:04:52.938+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:04:52.941 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , 00000000001.41.5, , , , , , ] 2022-09-03T06:04:52.941 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:04:57.941 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:04:57.939+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:04:57.942 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , 00000000001.41.5, , , , , , ] 2022-09-03T06:04:57.942 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:05:02.943 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:05:02.940+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:05:02.943 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:05:02.944 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:05:07.944 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:05:07.942+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:05:07.945 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:05:07.945 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:05:12.946 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:05:12.944+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:05:12.946 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:05:12.946 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:05:17.948 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:05:17.945+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:05:17.949 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:05:17.949 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:05:22.949 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:05:22.947+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:05:22.950 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:05:22.950 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:05:27.950 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:05:27.949+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:05:27.951 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:05:27.951 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:05:32.952 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:05:32.950+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:05:32.953 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:05:32.953 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:05:37.954 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:05:37.952+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:05:37.954 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:05:37.955 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:05:42.955 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:05:42.953+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:05:42.956 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:05:42.956 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:05:47.957 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:05:47.955+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:05:47.957 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:05:47.958 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:05:52.959 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:05:52.955+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:05:52.959 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:05:52.960 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:05:57.960 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:05:57.958+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:05:57.961 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:05:57.961 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:06:02.961 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:06:02.959+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:06:02.962 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:06:02.962 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:06:07.962 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:06:07.960+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:06:07.963 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:06:07.963 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:06:12.964 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:06:12.962+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:06:12.964 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:06:12.965 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:06:17.965 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:06:17.963+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:06:17.966 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:06:17.966 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:06:22.967 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:06:22.965+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:06:22.967 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:06:22.967 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:06:27.968 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:06:27.966+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:06:27.969 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:06:27.969 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:06:32.970 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:06:32.968+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:06:32.970 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:06:32.970 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:06:37.971 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:06:37.970+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:06:37.972 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:06:37.972 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:06:42.973 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:06:42.970+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:06:42.974 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:06:42.974 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:06:47.974 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:06:47.971+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:06:47.975 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:06:47.975 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:06:52.976 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:06:52.974+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:06:52.976 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:06:52.977 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:06:57.977 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:06:57.975+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:06:57.977 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:06:57.978 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:07:02.978 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:07:02.977+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:07:02.979 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:07:02.979 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:07:07.980 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:07:07.978+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:07:07.980 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:07:07.981 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:07:12.984 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:07:12.980+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:07:12.985 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:07:12.985 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:07:17.983 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:07:17.981+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:07:17.983 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:07:17.984 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:07:22.984 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:07:22.983+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:07:22.985 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:07:22.985 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:07:27.986 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:07:27.984+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:07:27.986 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:07:27.987 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:07:32.987 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:07:32.985+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:07:32.988 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:07:32.988 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:07:37.989 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:07:37.986+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:07:37.989 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:07:37.989 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:07:42.990 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:07:42.988+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:07:42.991 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:07:42.991 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:07:47.991 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:07:47.990+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:07:47.992 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:07:47.992 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:07:52.997 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:07:52.991+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:07:52.998 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:07:52.998 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:07:57.994 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:07:57.992+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:07:57.995 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:07:57.995 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:08:02.995 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:08:02.994+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:08:02.996 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:08:02.996 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:08:07.997 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:08:07.995+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:08:07.998 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:08:07.998 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:08:12.999 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:08:12.997+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:08:12.999 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:08:12.999 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:08:18.001 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:08:17.999+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:08:18.002 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:08:18.002 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:08:23.003 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:08:23.000+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:08:23.004 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:08:23.004 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:08:28.003 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:08:28.000+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:08:28.004 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:08:28.004 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:08:33.005 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:08:33.003+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:08:33.005 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:08:33.005 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:08:38.006 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:08:38.004+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:08:38.007 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:08:38.007 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:08:43.008 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:08:43.006+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:08:43.008 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:08:43.008 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:08:48.009 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:08:48.007+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:08:48.010 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:08:48.010 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:08:53.010 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:08:53.008+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:08:53.011 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:08:53.011 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:08:58.011 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:08:58.009+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:08:58.011 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:08:58.011 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:09:03.012 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:09:03.010+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:09:03.013 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:09:03.013 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:09:08.014 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:09:08.012+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:09:08.014 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:09:08.015 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:09:13.016 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:09:13.014+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:09:13.016 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:09:13.016 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:09:18.017 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:09:18.014+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:09:18.018 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:09:18.018 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:09:23.019 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:09:23.017+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:09:23.019 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:09:23.020 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:09:28.020 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:09:28.018+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:09:28.020 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:09:28.021 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:09:33.021 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:09:33.020+0000 7f17f33c2600 1 waiting for incremental sync to catch up: 2022-09-03T06:09:33.022 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000001.51.5, , 00000000001.41.5, , , , , , ] 2022-09-03T06:09:33.022 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.39.5, , 00000000001.51.5, , 00000000001.41.5, , , 00000000001.46.5, , , ] 2022-09-03T06:09:38.023 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:09:38.021+0000 7f17f33c2600 0 bucket checkpoint timed out waiting for incremental sync to catch up 2022-09-03T06:09:38.024 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:09:38.021+0000 7f17f33c2600 0 bucket sync checkpoint failed: (110) Connection timed out 2022-09-03T06:09:38.024 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:09:38.021+0000 7f17f33c2600 -1 bucket sync checkpoint failed: (110) Connection timed out 2022-09-03T06:09:38.040 DEBUG:teuthology.orchestra.run:got remote process result: 110 2022-09-03T06:09:38.047 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_sync_disable_enable ... FAIL 2022-09-03T06:09:38.051 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-65 2022-09-03T06:09:38.053 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:09:38.053 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:09:38.055 DEBUG:boto:Method: PUT 2022-09-03T06:09:38.056 DEBUG:boto:Path: /dsgvcy-65/ 2022-09-03T06:09:38.057 DEBUG:boto:Data: 2022-09-03T06:09:38.057 DEBUG:boto:Headers: {} 2022-09-03T06:09:38.058 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:09:38.058 DEBUG:boto:Port: 8000 2022-09-03T06:09:38.059 DEBUG:boto:Params: {} 2022-09-03T06:09:38.060 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:09:38.061 DEBUG:boto:Token: None 2022-09-03T06:09:38.062 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 06:09:38 GMT /dsgvcy-65/ 2022-09-03T06:09:38.063 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GSAwfYa8I53jtqshY87BJhYX2LI= 2022-09-03T06:09:38.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:09:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GSAwfYa8I53jtqshY87BJhYX2LI=', 'Content-Length': '0'} 2022-09-03T06:09:38.088 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000006bc051d70f7a63ea-006312efa2-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:09:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:09:38.089 INFO:rgw_multi.tests:create bucket zone=test-zone2 name=dsgvcy-66 2022-09-03T06:09:38.089 DEBUG:boto:path=/dsgvcy-66/ 2022-09-03T06:09:38.090 DEBUG:boto:auth_path=/dsgvcy-66/ 2022-09-03T06:09:38.091 DEBUG:boto:Method: PUT 2022-09-03T06:09:38.091 DEBUG:boto:Path: /dsgvcy-66/ 2022-09-03T06:09:38.092 DEBUG:boto:Data: 2022-09-03T06:09:38.092 DEBUG:boto:Headers: {} 2022-09-03T06:09:38.093 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:09:38.094 DEBUG:boto:Port: 8000 2022-09-03T06:09:38.094 DEBUG:boto:Params: {} 2022-09-03T06:09:38.095 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:09:38.095 DEBUG:boto:Token: None 2022-09-03T06:09:38.096 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 06:09:38 GMT /dsgvcy-66/ 2022-09-03T06:09:38.097 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lCur7JwWjYrZ0xwV79rwYAIXw0Y= 2022-09-03T06:09:38.098 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:09:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lCur7JwWjYrZ0xwV79rwYAIXw0Y=', 'Content-Length': '0'} 2022-09-03T06:09:38.154 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000008e91ed27628c21b3-006312efa2-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:09:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:09:38.155 INFO:rgw_multi.tests:create bucket zone=test-zone3 name=dsgvcy-67 2022-09-03T06:09:38.156 DEBUG:boto:path=/dsgvcy-67/ 2022-09-03T06:09:38.157 DEBUG:boto:auth_path=/dsgvcy-67/ 2022-09-03T06:09:38.158 DEBUG:boto:Method: PUT 2022-09-03T06:09:38.159 DEBUG:boto:Path: /dsgvcy-67/ 2022-09-03T06:09:38.160 DEBUG:boto:Data: 2022-09-03T06:09:38.161 DEBUG:boto:Headers: {} 2022-09-03T06:09:38.161 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:09:38.162 DEBUG:boto:Port: 8001 2022-09-03T06:09:38.164 DEBUG:boto:Params: {} 2022-09-03T06:09:38.165 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T06:09:38.166 DEBUG:boto:Token: None 2022-09-03T06:09:38.167 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 06:09:38 GMT /dsgvcy-67/ 2022-09-03T06:09:38.168 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:riYhihUSHhXQ/zssNs0zdsPbfb0= 2022-09-03T06:09:38.170 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:09:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:riYhihUSHhXQ/zssNs0zdsPbfb0=', 'Content-Length': '0'} 2022-09-03T06:09:38.209 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000009bd89e41659baed5-006312efa2-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:09:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:09:38.210 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:09:38.211 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:09:38.212 DEBUG:boto:path=/dsgvcy-65/MULTIPART?uploads 2022-09-03T06:09:38.213 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?uploads 2022-09-03T06:09:38.213 DEBUG:boto:Method: POST 2022-09-03T06:09:38.214 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?uploads 2022-09-03T06:09:38.214 DEBUG:boto:Data: 2022-09-03T06:09:38.215 DEBUG:boto:Headers: {} 2022-09-03T06:09:38.215 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:09:38.216 DEBUG:boto:Port: 8000 2022-09-03T06:09:38.216 DEBUG:boto:Params: {} 2022-09-03T06:09:38.217 DEBUG:boto:Token: None 2022-09-03T06:09:38.218 DEBUG:boto:StringToSign: POST Sat, 03 Sep 2022 06:09:38 GMT /dsgvcy-65/MULTIPART?uploads 2022-09-03T06:09:38.218 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S3fAizTjA36ztQdTGjzYPD2Ifkg= 2022-09-03T06:09:38.219 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:09:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S3fAizTjA36ztQdTGjzYPD2Ifkg=', 'Content-Length': '0'} 2022-09-03T06:09:39.391 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000551775e41a1d1dc8-006312efa2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '249'), ('Date', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:09:39.392 DEBUG:boto:b'dsgvcy-65MULTIPART2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw' 2022-09-03T06:09:39.431 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:09:39.435 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:09:39.435 DEBUG:boto:path=/dsgvcy-65/MULTIPART?uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw&partNumber=1 2022-09-03T06:09:39.436 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw&partNumber=1 2022-09-03T06:09:39.436 DEBUG:boto:Method: PUT 2022-09-03T06:09:39.437 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw&partNumber=1 2022-09-03T06:09:39.437 DEBUG:boto:Data: 2022-09-03T06:09:39.438 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ebKBBg0ze5srhMzzkK3PdA==', 'Content-Length': '5242880', 'Expect': '100-Continue'} 2022-09-03T06:09:39.438 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:09:39.439 DEBUG:boto:Port: 8000 2022-09-03T06:09:39.439 DEBUG:boto:Params: {} 2022-09-03T06:09:39.440 DEBUG:boto:Token: None 2022-09-03T06:09:39.440 DEBUG:boto:StringToSign: PUT ebKBBg0ze5srhMzzkK3PdA== application/octet-stream Sat, 03 Sep 2022 06:09:39 GMT /dsgvcy-65/MULTIPART?partNumber=1&uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw 2022-09-03T06:09:39.441 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NwxkiwzSceH0N6nBo12QNljyhXM= 2022-09-03T06:09:39.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'ebKBBg0ze5srhMzzkK3PdA==', 'Content-Length': '5242880', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:09:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NwxkiwzSceH0N6nBo12QNljyhXM='} 2022-09-03T06:09:39.480 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"79b281060d337b9b2b84ccf390adcf74"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000007e69fbde468773ba-006312efa3-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:09:39.519 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:09:39.520 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:09:39.521 DEBUG:boto:path=/dsgvcy-65/MULTIPART?uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw&partNumber=2 2022-09-03T06:09:39.522 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw&partNumber=2 2022-09-03T06:09:39.523 DEBUG:boto:Method: PUT 2022-09-03T06:09:39.523 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw&partNumber=2 2022-09-03T06:09:39.524 DEBUG:boto:Data: 2022-09-03T06:09:39.525 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'dIQ6OrGTo4m87YmUAtmdXw==', 'Content-Length': '5242880', 'Expect': '100-Continue'} 2022-09-03T06:09:39.526 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:09:39.527 DEBUG:boto:Port: 8000 2022-09-03T06:09:39.527 DEBUG:boto:Params: {} 2022-09-03T06:09:39.528 DEBUG:boto:Token: None 2022-09-03T06:09:39.529 DEBUG:boto:StringToSign: PUT dIQ6OrGTo4m87YmUAtmdXw== application/octet-stream Sat, 03 Sep 2022 06:09:39 GMT /dsgvcy-65/MULTIPART?partNumber=2&uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw 2022-09-03T06:09:39.529 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hyzgDwqmfYQ7pdSt9lbT9JqP9eM= 2022-09-03T06:09:39.530 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'dIQ6OrGTo4m87YmUAtmdXw==', 'Content-Length': '5242880', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:09:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hyzgDwqmfYQ7pdSt9lbT9JqP9eM='} 2022-09-03T06:09:39.575 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"74843a3ab193a389bced899402d99d5f"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005a09cd528ae7f44c-006312efa3-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:09:39.597 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:09:39.598 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:09:39.599 DEBUG:boto:path=/dsgvcy-65/MULTIPART?uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw&partNumber=3 2022-09-03T06:09:39.599 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw&partNumber=3 2022-09-03T06:09:39.600 DEBUG:boto:Method: PUT 2022-09-03T06:09:39.600 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw&partNumber=3 2022-09-03T06:09:39.601 DEBUG:boto:Data: 2022-09-03T06:09:39.601 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'e4RW4edMN49Fhh9TYZ51tg==', 'Content-Length': '5242880', 'Expect': '100-Continue'} 2022-09-03T06:09:39.602 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:09:39.602 DEBUG:boto:Port: 8000 2022-09-03T06:09:39.603 DEBUG:boto:Params: {} 2022-09-03T06:09:39.604 DEBUG:boto:Token: None 2022-09-03T06:09:39.604 DEBUG:boto:StringToSign: PUT e4RW4edMN49Fhh9TYZ51tg== application/octet-stream Sat, 03 Sep 2022 06:09:39 GMT /dsgvcy-65/MULTIPART?partNumber=3&uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw 2022-09-03T06:09:39.605 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tBbitBGq9C8pvnQH8LliL1C5sLg= 2022-09-03T06:09:39.606 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'e4RW4edMN49Fhh9TYZ51tg==', 'Content-Length': '5242880', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:09:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tBbitBGq9C8pvnQH8LliL1C5sLg='} 2022-09-03T06:09:39.642 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"7b8456e1e74c378f45861f53619e75b6"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b908148b9edc5f74-006312efa3-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:09:39.663 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:09:39.664 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:09:39.665 DEBUG:boto:path=/dsgvcy-65/MULTIPART?uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw&partNumber=4 2022-09-03T06:09:39.665 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw&partNumber=4 2022-09-03T06:09:39.666 DEBUG:boto:Method: PUT 2022-09-03T06:09:39.666 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw&partNumber=4 2022-09-03T06:09:39.667 DEBUG:boto:Data: 2022-09-03T06:09:39.667 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'l+jDi4yPcIlVvEiQwDQZTQ==', 'Content-Length': '5242880', 'Expect': '100-Continue'} 2022-09-03T06:09:39.668 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:09:39.669 DEBUG:boto:Port: 8000 2022-09-03T06:09:39.669 DEBUG:boto:Params: {} 2022-09-03T06:09:39.670 DEBUG:boto:Token: None 2022-09-03T06:09:39.670 DEBUG:boto:StringToSign: PUT l+jDi4yPcIlVvEiQwDQZTQ== application/octet-stream Sat, 03 Sep 2022 06:09:39 GMT /dsgvcy-65/MULTIPART?partNumber=4&uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw 2022-09-03T06:09:39.671 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/JeoGaA3yxJsZj58ad2wHo8Y3zA= 2022-09-03T06:09:39.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'l+jDi4yPcIlVvEiQwDQZTQ==', 'Content-Length': '5242880', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:09:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/JeoGaA3yxJsZj58ad2wHo8Y3zA='} 2022-09-03T06:09:39.709 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"97e8c38b8c8f708955bc4890c034194d"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009eb45be22b29e54e-006312efa3-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:09:39.710 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:09:39.711 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:09:39.711 DEBUG:boto:path=/dsgvcy-65/MULTIPART?uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw 2022-09-03T06:09:39.712 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw 2022-09-03T06:09:39.712 DEBUG:boto:Method: GET 2022-09-03T06:09:39.713 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw 2022-09-03T06:09:39.714 DEBUG:boto:Data: 2022-09-03T06:09:39.714 DEBUG:boto:Headers: {} 2022-09-03T06:09:39.715 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:09:39.715 DEBUG:boto:Port: 8000 2022-09-03T06:09:39.716 DEBUG:boto:Params: {} 2022-09-03T06:09:39.717 DEBUG:boto:Token: None 2022-09-03T06:09:39.717 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:09:39 GMT /dsgvcy-65/MULTIPART?uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw 2022-09-03T06:09:39.718 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qx10xYfMlIFb0nSOYAi5uOCn2x0= 2022-09-03T06:09:39.719 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:09:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qx10xYfMlIFb0nSOYAi5uOCn2x0=', 'Content-Length': '0'} 2022-09-03T06:09:39.722 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e2385de33c5a06cb-006312efa3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:09:39.766 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:09:39.767 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:09:39.768 DEBUG:boto:path=/dsgvcy-65/MULTIPART?uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw 2022-09-03T06:09:39.768 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw 2022-09-03T06:09:39.769 DEBUG:boto:Method: POST 2022-09-03T06:09:39.770 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw 2022-09-03T06:09:39.770 DEBUG:boto:Data: 1 "79b281060d337b9b2b84ccf390adcf74" 2 "74843a3ab193a389bced899402d99d5f" 3 "7b8456e1e74c378f45861f53619e75b6" 4 "97e8c38b8c8f708955bc4890c034194d" 2022-09-03T06:09:39.771 DEBUG:boto:Headers: {'Content-Type': 'text/xml'} 2022-09-03T06:09:39.771 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:09:39.772 DEBUG:boto:Port: 8000 2022-09-03T06:09:39.772 DEBUG:boto:Params: {} 2022-09-03T06:09:39.773 DEBUG:boto:Token: None 2022-09-03T06:09:39.774 DEBUG:boto:StringToSign: POST text/xml Sat, 03 Sep 2022 06:09:39 GMT /dsgvcy-65/MULTIPART?uploadId=2~k-Ybbse3G-BlbodbgL5Oig1figgAsVw 2022-09-03T06:09:39.775 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+ZK5F/Espt/bnxCjAucTVl4lt9g= 2022-09-03T06:09:39.776 DEBUG:boto:Final headers: {'Content-Type': 'text/xml', 'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:09:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+ZK5F/Espt/bnxCjAucTVl4lt9g=', 'Content-Length': '460'} 2022-09-03T06:09:39.789 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000071bdb4a1f446d9bd-006312efa3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '291'), ('Date', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:09:39.790 DEBUG:boto:http://smithi181.front.sepia.ceph.com:8000/dsgvcy-65/MULTIPARTdsgvcy-65MULTIPART 2022-09-03T06:09:39.791 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:09:39.792 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:09:39.793 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:09:39.847 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:09:39.848 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:09:39.884 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T06:09:39.898 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T06:09:39.898 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:09:39.899 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:09:39.899 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:09:39.899 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:09:39.900 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:09:39.900 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:09:39.900 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:09:39.901 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:09:39.901 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:09:39.901 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662185378.195349_408.1", 2022-09-03T06:09:39.901 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:09:38.195349Z" 2022-09-03T06:09:39.902 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:09:39.902 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:09:39.902 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:09:39.902 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:09:39.903 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:09:39.903 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T06:09:39.905 DEBUG:rgw_multi.tests:master meta markers={0: '', 1: '', 2: '1_1662185378.195349_408.1', 3: ''} 2022-09-03T06:09:39.906 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T06:09:39.907 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:09:39.908 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:09:39.909 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:09:39.968 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:09:39.969 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:09:40.007 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:09:40.005+0000 7fe419155600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:09:40.009 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:09:40.006+0000 7fe419155600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:09:40.025 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:09:40.026 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:09:40.026 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:09:40.027 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:09:40.027 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:09:40.027 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:09:40.027 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:09:40.028 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:09:40.028 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:09:40.028 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:09:40.028 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:09:40.029 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:09:40.029 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:09:40.029 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:09:40.030 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:09:40.031 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:09:40.031 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:09:40.031 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:09:40.032 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:09:40.032 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:09:40.032 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:09:40.032 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:09:40.033 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:09:40.033 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:09:40.033 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:09:40.033 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:09:40.034 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:09:40.034 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:09:40.034 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:09:40.035 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:09:40.035 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:09:40.035 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:09:40.035 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:09:40.036 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:09:40.036 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:09:40.036 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:09:40.037 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:09:40.037 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662185378.195349_408.1", 2022-09-03T06:09:40.037 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:09:40.037 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:09:40.038 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:09:40.038 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:09:38.195349Z", 2022-09-03T06:09:40.038 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:09:40.038 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:09:40.039 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:09:40.039 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:09:40.039 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:09:40.040 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:09:40.040 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:09:40.040 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:09:40.040 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:09:40.041 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:09:40.041 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:09:40.041 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:09:40.041 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:09:40.042 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:09:40.042 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:09:40.042 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:09:40.043 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:09:40.043 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:09:40.043 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:09:40.044 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:09:40.044 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:09:40.044 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:09:40Z" 2022-09-03T06:09:40.044 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:09:40.046 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185378.195349_408.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:09:38.195349Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:09:40Z" } 2022-09-03T06:09:40.046 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185378.195349_408.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:09:38.195349Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:09:40.047 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185378.195349_408.1', 3: ''} 2022-09-03T06:09:40.048 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185378.195349_408.1', 3: ''} 2022-09-03T06:09:40.048 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T06:09:40.049 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T06:09:40.049 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:09:40.050 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:09:40.050 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:09:40.101 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:09:40.101 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:09:40.138 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:09:40.136+0000 7fbfa3247600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:09:40.139 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:09:40.137+0000 7fbfa3247600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:09:40.154 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T06:09:40.155 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T06:09:40.155 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T06:09:40.156 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T06:09:40.157 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T06:09:40.157 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:09:40.158 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:09:40.158 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:09:40.158 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T06:09:40.158 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:09:40.159 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T06:09:40.159 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:09:40.159 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:09:40.159 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:09:40.160 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:09:40.160 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T06:09:40.160 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:09:40.160 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:09:40.161 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:09:40.161 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:09:40.161 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:09:40.161 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:09:40.162 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T06:09:40.162 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:09:40.163 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:09:40.163 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:09:40.163 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:09:40.163 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:09:40.164 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:09:40.164 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T06:09:40.165 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:09:40.165 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:09:40.165 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:09:40.166 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:09:40.166 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T06:09:40.166 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:09:40.166 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:09:40.167 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662185378.125008_403.1", 2022-09-03T06:09:40.167 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:09:40.167 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:09:40.167 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:09:40.168 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:09:38.125008Z", 2022-09-03T06:09:40.168 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:09:40.168 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:09:40.168 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:09:40.169 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:09:40.169 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T06:09:40.169 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:09:40.169 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:09:40.170 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:09:40.170 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:09:40.170 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:09:40.170 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:09:40.171 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:09:40.171 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:09:40.171 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:09:40.171 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:09:40.172 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:09:40.172 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:09:40.172 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T06:09:40.173 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T06:09:40.173 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T06:09:40.173 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:09:40.173 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T06:09:40Z" 2022-09-03T06:09:40.174 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T06:09:40.175 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185378.125008_403.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:09:38.125008Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:09:40Z" } 2022-09-03T06:09:40.176 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185378.125008_403.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:09:38.125008Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:09:40.176 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185378.195349_408.1', 3: ''} 2022-09-03T06:09:40.177 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185378.125008_403.1', 3: ''} 2022-09-03T06:09:40.177 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662185378.195349_408.1 target=1_1662185378.125008_403.1 2022-09-03T06:09:45.179 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:09:45.180 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:09:45.181 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:09:45.239 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:09:45.240 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:09:45.287 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:09:45.285+0000 7f655fa91600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:09:45.288 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:09:45.286+0000 7f655fa91600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:09:45.304 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T06:09:45.304 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T06:09:45.305 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T06:09:45.305 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T06:09:45.305 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T06:09:45.306 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:09:45.306 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:09:45.306 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:09:45.307 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T06:09:45.307 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:09:45.307 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T06:09:45.307 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:09:45.308 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:09:45.308 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:09:45.308 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:09:45.309 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T06:09:45.309 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:09:45.309 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:09:45.309 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:09:45.310 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:09:45.310 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:09:45.310 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:09:45.311 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T06:09:45.311 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:09:45.312 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:09:45.312 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:09:45.312 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:09:45.312 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:09:45.313 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:09:45.313 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T06:09:45.314 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:09:45.314 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:09:45.314 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:09:45.314 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:09:45.315 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T06:09:45.315 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:09:45.315 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:09:45.315 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662185378.195349_408.1", 2022-09-03T06:09:45.316 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:09:45.316 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:09:45.316 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:09:45.316 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:09:38.195349Z", 2022-09-03T06:09:45.317 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:09:45.317 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:09:45.317 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:09:45.317 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:09:45.318 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T06:09:45.318 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:09:45.318 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:09:45.318 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:09:45.319 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:09:45.319 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:09:45.319 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:09:45.319 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:09:45.320 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:09:45.320 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:09:45.320 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:09:45.320 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:09:45.321 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:09:45.321 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T06:09:45.321 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T06:09:45.321 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T06:09:45.322 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:09:45.322 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T06:09:45Z" 2022-09-03T06:09:45.322 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T06:09:45.323 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185378.195349_408.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:09:38.195349Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:09:45Z" } 2022-09-03T06:09:45.324 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185378.195349_408.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:09:38.195349Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:09:45.325 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185378.195349_408.1', 3: ''} 2022-09-03T06:09:45.325 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185378.195349_408.1', 3: ''} 2022-09-03T06:09:45.326 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T06:09:45.326 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T06:09:45.327 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:09:45.327 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:09:45.328 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:09:45.386 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:09:45.387 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:09:45.418 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:09:45.416+0000 7f648152c600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:09:45.419 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:09:45.417+0000 7f648152c600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:09:45.432 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:09:45.432 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:09:45.432 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:09:45.433 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:09:45.433 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:09:45.433 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:09:45.434 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:09:45.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:09:45.435 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:09:45.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:09:45.435 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:09:45.435 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:09:45.436 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:09:45.436 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:09:45.436 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:09:45.436 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:09:45.437 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:09:45.437 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:09:45.437 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:09:45.437 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:09:45.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:09:45.438 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:09:45.438 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:09:45.438 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:09:45.439 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:09:45.439 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:09:45.439 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:09:45.440 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:09:45.440 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:09:45.440 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:09:45.440 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:09:45.441 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:09:45.441 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:09:45.441 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:09:45.441 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:09:45.442 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:09:45.442 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:09:45.442 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662185378.195349_408.1", 2022-09-03T06:09:45.442 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:09:45.443 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:09:45.443 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:09:45.443 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:09:38.195349Z", 2022-09-03T06:09:45.443 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:09:45.444 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:09:45.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:09:45.444 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:09:45.444 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:09:45.445 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:09:45.445 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:09:45.445 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:09:45.445 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:09:45.446 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:09:45.446 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:09:45.446 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:09:45.446 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:09:45.447 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:09:45.447 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:09:45.447 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:09:45.448 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:09:45.448 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:09:45.449 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:09:45.449 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:09:45.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:09:45.450 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:09:45Z" 2022-09-03T06:09:45.450 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:09:45.452 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185378.195349_408.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:09:38.195349Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:09:45Z" } 2022-09-03T06:09:45.453 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185378.195349_408.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:09:38.195349Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:09:45.454 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185378.195349_408.1', 3: ''} 2022-09-03T06:09:45.455 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185378.195349_408.1', 3: ''} 2022-09-03T06:09:45.456 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T06:09:45.457 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-65 2022-09-03T06:09:45.458 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-65', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:09:45.459 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-65', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:09:45.460 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-65 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:09:45.518 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:09:45.519 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:09:45.554 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:09:45.552+0000 7f47a87f6600 1 waiting to reach incremental sync.. 2022-09-03T06:09:50.554 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:09:50.553+0000 7f47a87f6600 1 waiting to reach incremental sync.. 2022-09-03T06:09:55.555 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:09:55.553+0000 7f47a87f6600 1 waiting to reach incremental sync.. 2022-09-03T06:10:00.558 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:10:00.555+0000 7f47a87f6600 1 bucket sync caught up with source: 2022-09-03T06:10:00.559 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000011.58.5, , , , , ] 2022-09-03T06:10:00.559 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000011.58.5, , , , , ] 2022-09-03T06:10:00.559 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:10:00.555+0000 7f47a87f6600 0 bucket checkpoint complete 2022-09-03T06:10:00.575 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-65 2022-09-03T06:10:00.575 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-65', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:10:00.576 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-65', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:10:00.577 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-65 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:10:00.634 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:10:00.635 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:10:00.683 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:10:00.679+0000 7f9515cd0600 1 bucket sync caught up with source: 2022-09-03T06:10:00.683 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000011.58.5, , , , , ] 2022-09-03T06:10:00.684 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000011.58.5, , , , , ] 2022-09-03T06:10:00.684 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:10:00.679+0000 7f9515cd0600 0 bucket checkpoint complete 2022-09-03T06:10:00.704 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-65 2022-09-03T06:10:00.705 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-65', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:10:00.705 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-65', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:10:00.706 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-65 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:10:00.770 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:10:00.771 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:10:00.809 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:10:00.807+0000 7f7e483f3600 1 bucket sync caught up with source: 2022-09-03T06:10:00.809 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000011.58.5, , , , , ] 2022-09-03T06:10:00.809 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000011.58.5, , , , , ] 2022-09-03T06:10:00.810 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:10:00.807+0000 7f7e483f3600 0 bucket checkpoint complete 2022-09-03T06:10:00.828 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-65 2022-09-03T06:10:00.829 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-65', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:10:00.829 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-65', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:10:00.830 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-65 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:10:00.886 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:10:00.887 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:10:00.923 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:10:00.920+0000 7f78e7170600 1 waiting to reach incremental sync.. 2022-09-03T06:10:05.923 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:10:05.921+0000 7f78e7170600 1 waiting to reach incremental sync.. 2022-09-03T06:10:10.924 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:10:10.923+0000 7f78e7170600 1 bucket sync caught up with source: 2022-09-03T06:10:10.925 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000001.512.5, , , , , ] 2022-09-03T06:10:10.925 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000001.512.5, , , , , ] 2022-09-03T06:10:10.926 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:10:10.923+0000 7f78e7170600 0 bucket checkpoint complete 2022-09-03T06:10:10.945 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-65 2022-09-03T06:10:10.946 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-65', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:10:10.946 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-65', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:10:10.947 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-65 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:10:11.011 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:10:11.011 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:10:11.061 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:10:11.060+0000 7f3b048a7600 1 waiting to reach incremental sync.. 2022-09-03T06:10:16.062 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:10:16.060+0000 7f3b048a7600 1 waiting to reach incremental sync.. 2022-09-03T06:10:21.063 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:10:21.061+0000 7f3b048a7600 1 waiting to reach incremental sync.. 2022-09-03T06:10:26.063 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:10:26.061+0000 7f3b048a7600 1 waiting to reach incremental sync.. 2022-09-03T06:10:31.064 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:10:31.062+0000 7f3b048a7600 1 waiting to reach incremental sync.. 2022-09-03T06:10:36.064 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:10:36.062+0000 7f3b048a7600 1 waiting to reach incremental sync.. 2022-09-03T06:10:41.065 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:10:41.062+0000 7f3b048a7600 1 waiting to reach incremental sync.. 2022-09-03T06:10:46.066 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:10:46.064+0000 7f3b048a7600 1 waiting to reach incremental sync.. 2022-09-03T06:10:51.067 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:10:51.065+0000 7f3b048a7600 1 waiting to reach incremental sync.. 2022-09-03T06:10:56.066 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:10:56.065+0000 7f3b048a7600 1 waiting to reach incremental sync.. 2022-09-03T06:11:01.068 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:11:01.067+0000 7f3b048a7600 1 bucket sync caught up with source: 2022-09-03T06:11:01.069 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000001.512.5, , , , , ] 2022-09-03T06:11:01.069 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000001.512.5, , , , , ] 2022-09-03T06:11:01.070 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:11:01.067+0000 7f3b048a7600 0 bucket checkpoint complete 2022-09-03T06:11:01.085 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-65 2022-09-03T06:11:01.086 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-65 2022-09-03T06:11:01.087 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-65', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:11:01.087 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-65', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:11:01.088 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-65 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:11:01.152 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:11:01.152 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:11:01.200 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:11:01.198+0000 7f58cfab3600 1 bucket sync caught up with source: 2022-09-03T06:11:01.200 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000001.130.5, , , , , ] 2022-09-03T06:11:01.201 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000001.130.5, , , , , ] 2022-09-03T06:11:01.201 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:11:01.198+0000 7f58cfab3600 0 bucket checkpoint complete 2022-09-03T06:11:01.217 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-65 2022-09-03T06:11:01.218 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-65', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:11:01.218 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-65', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:11:01.219 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-65 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:11:01.288 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:11:01.288 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:11:01.333 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:01.331+0000 7f8733dfb600 1 bucket sync caught up with source: 2022-09-03T06:11:01.334 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000001.130.5, , , , , ] 2022-09-03T06:11:01.335 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000001.130.5, , , , , ] 2022-09-03T06:11:01.336 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:01.331+0000 7f8733dfb600 0 bucket checkpoint complete 2022-09-03T06:11:01.350 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-65 2022-09-03T06:11:01.350 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-65 zones={test-zone2, test-zone1} 2022-09-03T06:11:01.351 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:01.352 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:01.353 DEBUG:boto:Method: HEAD 2022-09-03T06:11:01.354 DEBUG:boto:Path: /dsgvcy-65/ 2022-09-03T06:11:01.354 DEBUG:boto:Data: 2022-09-03T06:11:01.357 DEBUG:boto:Headers: {} 2022-09-03T06:11:01.358 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:01.358 DEBUG:boto:Port: 8000 2022-09-03T06:11:01.359 DEBUG:boto:Params: {} 2022-09-03T06:11:01.360 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:11:01.361 DEBUG:boto:Token: None 2022-09-03T06:11:01.361 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:01 GMT /dsgvcy-65/ 2022-09-03T06:11:01.362 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nsfbOTjPQ+eCOkLOfqwEboh4dAg= 2022-09-03T06:11:01.363 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nsfbOTjPQ+eCOkLOfqwEboh4dAg=', 'Content-Length': '0'} 2022-09-03T06:11:01.367 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '20971520'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007d021953b59316c0-006312eff5-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:11:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:01.368 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:01.369 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:01.369 DEBUG:boto:Method: HEAD 2022-09-03T06:11:01.370 DEBUG:boto:Path: /dsgvcy-65/ 2022-09-03T06:11:01.370 DEBUG:boto:Data: 2022-09-03T06:11:01.371 DEBUG:boto:Headers: {} 2022-09-03T06:11:01.372 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:01.372 DEBUG:boto:Port: 8000 2022-09-03T06:11:01.373 DEBUG:boto:Params: {} 2022-09-03T06:11:01.373 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:11:01.374 DEBUG:boto:Token: None 2022-09-03T06:11:01.375 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:01 GMT /dsgvcy-65/ 2022-09-03T06:11:01.376 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nsfbOTjPQ+eCOkLOfqwEboh4dAg= 2022-09-03T06:11:01.376 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nsfbOTjPQ+eCOkLOfqwEboh4dAg=', 'Content-Length': '0'} 2022-09-03T06:11:01.380 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '20971520'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000020fe8cd64ba6dcff-006312eff5-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:11:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:01.381 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:11:01.382 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:01.382 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:01.383 DEBUG:boto:path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:01.384 DEBUG:boto:auth_path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:01.384 DEBUG:boto:Method: GET 2022-09-03T06:11:01.385 DEBUG:boto:Path: /dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:01.386 DEBUG:boto:Data: 2022-09-03T06:11:01.386 DEBUG:boto:Headers: {} 2022-09-03T06:11:01.387 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:01.388 DEBUG:boto:Port: 8000 2022-09-03T06:11:01.388 DEBUG:boto:Params: {} 2022-09-03T06:11:01.389 DEBUG:boto:Token: None 2022-09-03T06:11:01.390 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:01 GMT /dsgvcy-65/?versions 2022-09-03T06:11:01.391 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RtyvrJtWKKDJ7md+EE276j7fySQ= 2022-09-03T06:11:01.392 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RtyvrJtWKKDJ7md+EE276j7fySQ=', 'Content-Length': '0'} 2022-09-03T06:11:01.394 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a361415ffe93ba86-006312eff5-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:01.395 DEBUG:boto:b'dsgvcy-65999falseMULTIPARTnulltrue2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:01.396 DEBUG:tasks.rgw_multi.zone_rados:o=MULTIPART 2022-09-03T06:11:01.397 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:11:01.397 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:01.398 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:01.399 DEBUG:boto:path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:01.399 DEBUG:boto:auth_path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:01.400 DEBUG:boto:Method: GET 2022-09-03T06:11:01.400 DEBUG:boto:Path: /dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:01.401 DEBUG:boto:Data: 2022-09-03T06:11:01.402 DEBUG:boto:Headers: {} 2022-09-03T06:11:01.402 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:01.403 DEBUG:boto:Port: 8000 2022-09-03T06:11:01.403 DEBUG:boto:Params: {} 2022-09-03T06:11:01.404 DEBUG:boto:Token: None 2022-09-03T06:11:01.404 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:01 GMT /dsgvcy-65/?versions 2022-09-03T06:11:01.405 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RtyvrJtWKKDJ7md+EE276j7fySQ= 2022-09-03T06:11:01.405 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RtyvrJtWKKDJ7md+EE276j7fySQ=', 'Content-Length': '0'} 2022-09-03T06:11:01.408 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e596aa4ff6b7e8c2-006312eff5-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:01.409 DEBUG:boto:b'dsgvcy-65999falseMULTIPARTnulltrue2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:01.410 DEBUG:tasks.rgw_multi.zone_rados:o=MULTIPART 2022-09-03T06:11:01.411 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:01.411 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:01.412 DEBUG:boto:path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:01.412 DEBUG:boto:auth_path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:01.413 DEBUG:boto:Method: GET 2022-09-03T06:11:01.413 DEBUG:boto:Path: /dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:01.414 DEBUG:boto:Data: 2022-09-03T06:11:01.415 DEBUG:boto:Headers: {} 2022-09-03T06:11:01.416 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:01.417 DEBUG:boto:Port: 8000 2022-09-03T06:11:01.418 DEBUG:boto:Params: {} 2022-09-03T06:11:01.419 DEBUG:boto:Token: None 2022-09-03T06:11:01.420 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:01 GMT /dsgvcy-65/?versions 2022-09-03T06:11:01.421 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RtyvrJtWKKDJ7md+EE276j7fySQ= 2022-09-03T06:11:01.421 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RtyvrJtWKKDJ7md+EE276j7fySQ=', 'Content-Length': '0'} 2022-09-03T06:11:01.424 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e8fc9f329e809d37-006312eff5-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:01.426 DEBUG:boto:b'dsgvcy-65999falseMULTIPARTnulltrue2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:01.427 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:01.428 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:01.430 DEBUG:boto:path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:01.430 DEBUG:boto:auth_path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:01.431 DEBUG:boto:Method: GET 2022-09-03T06:11:01.431 DEBUG:boto:Path: /dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:01.432 DEBUG:boto:Data: 2022-09-03T06:11:01.432 DEBUG:boto:Headers: {} 2022-09-03T06:11:01.433 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:01.433 DEBUG:boto:Port: 8000 2022-09-03T06:11:01.434 DEBUG:boto:Params: {} 2022-09-03T06:11:01.434 DEBUG:boto:Token: None 2022-09-03T06:11:01.435 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:01 GMT /dsgvcy-65/?versions 2022-09-03T06:11:01.435 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RtyvrJtWKKDJ7md+EE276j7fySQ= 2022-09-03T06:11:01.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RtyvrJtWKKDJ7md+EE276j7fySQ=', 'Content-Length': '0'} 2022-09-03T06:11:01.439 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ea3f8f22bca1c9e2-006312eff5-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:01.439 DEBUG:boto:b'dsgvcy-65999falseMULTIPARTnulltrue2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:01.440 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=MULTIPART 2022-09-03T06:11:01.441 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:01.441 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:01.442 DEBUG:boto:path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.443 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.443 DEBUG:boto:Method: GET 2022-09-03T06:11:01.444 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.444 DEBUG:boto:Data: 2022-09-03T06:11:01.445 DEBUG:boto:Headers: {} 2022-09-03T06:11:01.445 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:01.446 DEBUG:boto:Port: 8000 2022-09-03T06:11:01.446 DEBUG:boto:Params: {} 2022-09-03T06:11:01.447 DEBUG:boto:Token: None 2022-09-03T06:11:01.448 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:01 GMT /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.448 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IIR84Vc2KjQdRCpWOlUTsfSydBg= 2022-09-03T06:11:01.449 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IIR84Vc2KjQdRCpWOlUTsfSydBg=', 'Content-Length': '0'} 2022-09-03T06:11:01.452 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx0000026d7f5c1bd2fb992-006312eff5-15d2-test-zone2'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:01.557 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:01.557 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:01.558 DEBUG:boto:path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.558 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.559 DEBUG:boto:Method: GET 2022-09-03T06:11:01.559 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.560 DEBUG:boto:Data: 2022-09-03T06:11:01.560 DEBUG:boto:Headers: {} 2022-09-03T06:11:01.561 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:01.561 DEBUG:boto:Port: 8000 2022-09-03T06:11:01.562 DEBUG:boto:Params: {} 2022-09-03T06:11:01.563 DEBUG:boto:Token: None 2022-09-03T06:11:01.563 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:01 GMT /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.564 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IIR84Vc2KjQdRCpWOlUTsfSydBg= 2022-09-03T06:11:01.564 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IIR84Vc2KjQdRCpWOlUTsfSydBg=', 'Content-Length': '0'} 2022-09-03T06:11:01.567 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx0000016fe6cb4d72c648e-006312eff5-15d7-test-zone1'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:01.666 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:01.667 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:01.667 DEBUG:boto:path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.668 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.668 DEBUG:boto:Method: HEAD 2022-09-03T06:11:01.669 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.669 DEBUG:boto:Data: 2022-09-03T06:11:01.670 DEBUG:boto:Headers: {} 2022-09-03T06:11:01.670 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:01.671 DEBUG:boto:Port: 8000 2022-09-03T06:11:01.671 DEBUG:boto:Params: {} 2022-09-03T06:11:01.672 DEBUG:boto:Token: None 2022-09-03T06:11:01.673 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:01 GMT /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.674 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Nie2r0z0y4vFIu6lHB6uix65B44= 2022-09-03T06:11:01.674 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Nie2r0z0y4vFIu6lHB6uix65B44=', 'Content-Length': '0'} 2022-09-03T06:11:01.677 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx00000695ec3db739fa446-006312eff5-15d2-test-zone2'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:01.677 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:01.678 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:01.678 DEBUG:boto:path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.679 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.679 DEBUG:boto:Method: HEAD 2022-09-03T06:11:01.680 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.681 DEBUG:boto:Data: 2022-09-03T06:11:01.681 DEBUG:boto:Headers: {} 2022-09-03T06:11:01.682 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:01.682 DEBUG:boto:Port: 8000 2022-09-03T06:11:01.683 DEBUG:boto:Params: {} 2022-09-03T06:11:01.683 DEBUG:boto:Token: None 2022-09-03T06:11:01.684 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:01 GMT /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.685 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Nie2r0z0y4vFIu6lHB6uix65B44= 2022-09-03T06:11:01.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Nie2r0z0y4vFIu6lHB6uix65B44=', 'Content-Length': '0'} 2022-09-03T06:11:01.687 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx0000098eb940340d8fbe9-006312eff5-15d7-test-zone1'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:01.688 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=MULTIPART 2022-09-03T06:11:01.689 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:01.690 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:01.690 DEBUG:boto:path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.691 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.691 DEBUG:boto:Method: GET 2022-09-03T06:11:01.692 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.693 DEBUG:boto:Data: 2022-09-03T06:11:01.694 DEBUG:boto:Headers: {} 2022-09-03T06:11:01.694 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:01.695 DEBUG:boto:Port: 8000 2022-09-03T06:11:01.696 DEBUG:boto:Params: {} 2022-09-03T06:11:01.696 DEBUG:boto:Token: None 2022-09-03T06:11:01.697 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:01 GMT /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.698 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IIR84Vc2KjQdRCpWOlUTsfSydBg= 2022-09-03T06:11:01.698 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IIR84Vc2KjQdRCpWOlUTsfSydBg=', 'Content-Length': '0'} 2022-09-03T06:11:01.701 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx0000060c41bf953022c79-006312eff5-15d2-test-zone2'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:01.821 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:01.822 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:01.823 DEBUG:boto:path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.823 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.824 DEBUG:boto:Method: GET 2022-09-03T06:11:01.824 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.825 DEBUG:boto:Data: 2022-09-03T06:11:01.826 DEBUG:boto:Headers: {} 2022-09-03T06:11:01.826 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:01.827 DEBUG:boto:Port: 8000 2022-09-03T06:11:01.827 DEBUG:boto:Params: {} 2022-09-03T06:11:01.828 DEBUG:boto:Token: None 2022-09-03T06:11:01.828 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:01 GMT /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:01.829 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IIR84Vc2KjQdRCpWOlUTsfSydBg= 2022-09-03T06:11:01.830 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IIR84Vc2KjQdRCpWOlUTsfSydBg=', 'Content-Length': '0'} 2022-09-03T06:11:01.833 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx00000ed60916a88574de8-006312eff5-15d7-test-zone1'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:01.933 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:01.934 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:01.934 DEBUG:boto:path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:01.935 DEBUG:boto:auth_path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:01.935 DEBUG:boto:Method: GET 2022-09-03T06:11:01.936 DEBUG:boto:Path: /dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:01.936 DEBUG:boto:Data: 2022-09-03T06:11:01.937 DEBUG:boto:Headers: {} 2022-09-03T06:11:01.938 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:01.938 DEBUG:boto:Port: 8000 2022-09-03T06:11:01.939 DEBUG:boto:Params: {} 2022-09-03T06:11:01.939 DEBUG:boto:Token: None 2022-09-03T06:11:01.940 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:01 GMT /dsgvcy-65/ 2022-09-03T06:11:01.941 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KUC/KAdeqHBnR3F9YMid1TQTnBE= 2022-09-03T06:11:01.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KUC/KAdeqHBnR3F9YMid1TQTnBE=', 'Content-Length': '0'} 2022-09-03T06:11:01.945 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008421ea318ca759e2-006312eff5-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:01.945 DEBUG:boto:b'dsgvcy-65MULTIPART1000falseMULTIPART2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:01.946 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:01.947 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:01.948 DEBUG:boto:path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:01.948 DEBUG:boto:auth_path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:01.949 DEBUG:boto:Method: GET 2022-09-03T06:11:01.949 DEBUG:boto:Path: /dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:01.950 DEBUG:boto:Data: 2022-09-03T06:11:01.950 DEBUG:boto:Headers: {} 2022-09-03T06:11:01.951 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:01.952 DEBUG:boto:Port: 8000 2022-09-03T06:11:01.952 DEBUG:boto:Params: {} 2022-09-03T06:11:01.953 DEBUG:boto:Token: None 2022-09-03T06:11:01.953 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:01 GMT /dsgvcy-65/ 2022-09-03T06:11:01.954 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KUC/KAdeqHBnR3F9YMid1TQTnBE= 2022-09-03T06:11:01.955 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KUC/KAdeqHBnR3F9YMid1TQTnBE=', 'Content-Length': '0'} 2022-09-03T06:11:01.957 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000812f9ed6b60dfd8e-006312eff5-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:01.958 DEBUG:boto:b'dsgvcy-65MULTIPART1000falseMULTIPART2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:01.959 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:01.960 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:01.960 DEBUG:boto:Method: HEAD 2022-09-03T06:11:01.961 DEBUG:boto:Path: /dsgvcy-65/MULTIPART 2022-09-03T06:11:01.961 DEBUG:boto:Data: 2022-09-03T06:11:01.962 DEBUG:boto:Headers: {} 2022-09-03T06:11:01.962 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:01.963 DEBUG:boto:Port: 8000 2022-09-03T06:11:01.963 DEBUG:boto:Params: {} 2022-09-03T06:11:01.964 DEBUG:boto:Token: None 2022-09-03T06:11:01.964 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:01 GMT /dsgvcy-65/MULTIPART 2022-09-03T06:11:01.965 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ua7lAWkPYLJpWyDudOXBJ3UVOFk= 2022-09-03T06:11:01.966 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ua7lAWkPYLJpWyDudOXBJ3UVOFk=', 'Content-Length': '0'} 2022-09-03T06:11:01.968 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx000002ba9ddec8baadbf2-006312eff5-15d2-test-zone2'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:01.969 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:01.969 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:01.970 DEBUG:boto:Method: HEAD 2022-09-03T06:11:01.970 DEBUG:boto:Path: /dsgvcy-65/MULTIPART 2022-09-03T06:11:01.971 DEBUG:boto:Data: 2022-09-03T06:11:01.971 DEBUG:boto:Headers: {} 2022-09-03T06:11:01.972 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:01.972 DEBUG:boto:Port: 8000 2022-09-03T06:11:01.973 DEBUG:boto:Params: {} 2022-09-03T06:11:01.973 DEBUG:boto:Token: None 2022-09-03T06:11:01.974 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:01 GMT /dsgvcy-65/MULTIPART 2022-09-03T06:11:01.974 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ua7lAWkPYLJpWyDudOXBJ3UVOFk= 2022-09-03T06:11:01.975 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ua7lAWkPYLJpWyDudOXBJ3UVOFk=', 'Content-Length': '0'} 2022-09-03T06:11:01.977 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx00000fcbd240a58219fea-006312eff5-15d7-test-zone1'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:01.978 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=MULTIPART 2022-09-03T06:11:01.979 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:01.979 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:01.980 DEBUG:boto:Method: GET 2022-09-03T06:11:01.981 DEBUG:boto:Path: /dsgvcy-65/MULTIPART 2022-09-03T06:11:01.982 DEBUG:boto:Data: 2022-09-03T06:11:01.983 DEBUG:boto:Headers: {} 2022-09-03T06:11:01.984 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:01.985 DEBUG:boto:Port: 8000 2022-09-03T06:11:01.986 DEBUG:boto:Params: {} 2022-09-03T06:11:01.986 DEBUG:boto:Token: None 2022-09-03T06:11:01.987 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:01 GMT /dsgvcy-65/MULTIPART 2022-09-03T06:11:01.988 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xnyGknmQ16dyOB8c1ziZPumaCYI= 2022-09-03T06:11:01.989 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xnyGknmQ16dyOB8c1ziZPumaCYI=', 'Content-Length': '0'} 2022-09-03T06:11:01.992 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx0000067f0c670c126a0e0-006312eff5-15d2-test-zone2'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:02.087 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:02.088 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:02.089 DEBUG:boto:Method: GET 2022-09-03T06:11:02.089 DEBUG:boto:Path: /dsgvcy-65/MULTIPART 2022-09-03T06:11:02.090 DEBUG:boto:Data: 2022-09-03T06:11:02.091 DEBUG:boto:Headers: {} 2022-09-03T06:11:02.091 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:02.092 DEBUG:boto:Port: 8000 2022-09-03T06:11:02.092 DEBUG:boto:Params: {} 2022-09-03T06:11:02.093 DEBUG:boto:Token: None 2022-09-03T06:11:02.093 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:02 GMT /dsgvcy-65/MULTIPART 2022-09-03T06:11:02.094 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eL/7WQoYZPJ5t/78yJOpDAFwgjs= 2022-09-03T06:11:02.095 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eL/7WQoYZPJ5t/78yJOpDAFwgjs=', 'Content-Length': '0'} 2022-09-03T06:11:02.098 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx000006dc06d6e31a0a49d-006312eff6-15d7-test-zone1'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:02.223 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:02.224 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:02.225 DEBUG:boto:path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:02.225 DEBUG:boto:auth_path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:02.226 DEBUG:boto:Method: GET 2022-09-03T06:11:02.226 DEBUG:boto:Path: /dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:02.227 DEBUG:boto:Data: 2022-09-03T06:11:02.228 DEBUG:boto:Headers: {} 2022-09-03T06:11:02.228 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:02.229 DEBUG:boto:Port: 8000 2022-09-03T06:11:02.229 DEBUG:boto:Params: {} 2022-09-03T06:11:02.230 DEBUG:boto:Token: None 2022-09-03T06:11:02.230 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:02 GMT /dsgvcy-65/ 2022-09-03T06:11:02.231 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qDRfsdyXVkdj1AReGffnqSAmP6k= 2022-09-03T06:11:02.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qDRfsdyXVkdj1AReGffnqSAmP6k=', 'Content-Length': '0'} 2022-09-03T06:11:02.234 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e3e6e1868628c048-006312eff6-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:02.235 DEBUG:boto:b'dsgvcy-65MULTIPART1000falseMULTIPART2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:02.236 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:02.237 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:02.238 DEBUG:boto:path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:02.239 DEBUG:boto:auth_path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:02.240 DEBUG:boto:Method: GET 2022-09-03T06:11:02.241 DEBUG:boto:Path: /dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:02.241 DEBUG:boto:Data: 2022-09-03T06:11:02.242 DEBUG:boto:Headers: {} 2022-09-03T06:11:02.243 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:02.243 DEBUG:boto:Port: 8000 2022-09-03T06:11:02.244 DEBUG:boto:Params: {} 2022-09-03T06:11:02.245 DEBUG:boto:Token: None 2022-09-03T06:11:02.246 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:02 GMT /dsgvcy-65/ 2022-09-03T06:11:02.246 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qDRfsdyXVkdj1AReGffnqSAmP6k= 2022-09-03T06:11:02.247 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qDRfsdyXVkdj1AReGffnqSAmP6k=', 'Content-Length': '0'} 2022-09-03T06:11:02.251 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003c95b4289e77d04b-006312eff6-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:02.252 DEBUG:boto:b'dsgvcy-65MULTIPART1000falseMULTIPART2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:02.253 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-65 zones={test-zone2, test-zone1} 2022-09-03T06:11:02.253 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-65 zones={test-zone3, test-zone1} 2022-09-03T06:11:02.254 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:02.254 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:02.255 DEBUG:boto:Method: HEAD 2022-09-03T06:11:02.255 DEBUG:boto:Path: /dsgvcy-65/ 2022-09-03T06:11:02.256 DEBUG:boto:Data: 2022-09-03T06:11:02.257 DEBUG:boto:Headers: {} 2022-09-03T06:11:02.258 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:11:02.258 DEBUG:boto:Port: 8001 2022-09-03T06:11:02.259 DEBUG:boto:Params: {} 2022-09-03T06:11:02.260 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T06:11:02.260 DEBUG:boto:Token: None 2022-09-03T06:11:02.261 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:02 GMT /dsgvcy-65/ 2022-09-03T06:11:02.261 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FdK7ndfVwA4nXzNtEoCwwO49EHQ= 2022-09-03T06:11:02.262 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FdK7ndfVwA4nXzNtEoCwwO49EHQ=', 'Content-Length': '0'} 2022-09-03T06:11:02.266 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '20971520'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007f36cb2dcfd11007-006312eff6-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:11:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:02.267 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:02.267 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:02.268 DEBUG:boto:Method: HEAD 2022-09-03T06:11:02.268 DEBUG:boto:Path: /dsgvcy-65/ 2022-09-03T06:11:02.269 DEBUG:boto:Data: 2022-09-03T06:11:02.270 DEBUG:boto:Headers: {} 2022-09-03T06:11:02.270 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:02.271 DEBUG:boto:Port: 8000 2022-09-03T06:11:02.271 DEBUG:boto:Params: {} 2022-09-03T06:11:02.272 DEBUG:boto:Token: None 2022-09-03T06:11:02.272 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:02 GMT /dsgvcy-65/ 2022-09-03T06:11:02.273 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FdK7ndfVwA4nXzNtEoCwwO49EHQ= 2022-09-03T06:11:02.274 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FdK7ndfVwA4nXzNtEoCwwO49EHQ=', 'Content-Length': '0'} 2022-09-03T06:11:02.280 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '20971520'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008c3c32bf5761053e-006312eff6-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:11:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:02.281 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:11:02.281 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:02.282 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:02.283 DEBUG:boto:path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:02.283 DEBUG:boto:auth_path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:02.284 DEBUG:boto:Method: GET 2022-09-03T06:11:02.284 DEBUG:boto:Path: /dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:02.285 DEBUG:boto:Data: 2022-09-03T06:11:02.286 DEBUG:boto:Headers: {} 2022-09-03T06:11:02.286 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:11:02.287 DEBUG:boto:Port: 8001 2022-09-03T06:11:02.287 DEBUG:boto:Params: {} 2022-09-03T06:11:02.288 DEBUG:boto:Token: None 2022-09-03T06:11:02.288 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:02 GMT /dsgvcy-65/?versions 2022-09-03T06:11:02.289 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YWz8W7vyOOSzewjasDYRP9xl6hc= 2022-09-03T06:11:02.290 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YWz8W7vyOOSzewjasDYRP9xl6hc=', 'Content-Length': '0'} 2022-09-03T06:11:02.293 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fdfbac9534143a5f-006312eff6-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:02.294 DEBUG:boto:b'dsgvcy-65999falseMULTIPARTnulltrue2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:02.295 DEBUG:tasks.rgw_multi.zone_rados:o=MULTIPART 2022-09-03T06:11:02.296 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:11:02.297 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:02.298 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:02.299 DEBUG:boto:path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:02.300 DEBUG:boto:auth_path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:02.301 DEBUG:boto:Method: GET 2022-09-03T06:11:02.302 DEBUG:boto:Path: /dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:02.303 DEBUG:boto:Data: 2022-09-03T06:11:02.304 DEBUG:boto:Headers: {} 2022-09-03T06:11:02.305 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:02.306 DEBUG:boto:Port: 8000 2022-09-03T06:11:02.307 DEBUG:boto:Params: {} 2022-09-03T06:11:02.308 DEBUG:boto:Token: None 2022-09-03T06:11:02.309 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:02 GMT /dsgvcy-65/?versions 2022-09-03T06:11:02.310 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YWz8W7vyOOSzewjasDYRP9xl6hc= 2022-09-03T06:11:02.311 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YWz8W7vyOOSzewjasDYRP9xl6hc=', 'Content-Length': '0'} 2022-09-03T06:11:02.314 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000007b2534efa2dcb61-006312eff6-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:02.315 DEBUG:boto:b'dsgvcy-65999falseMULTIPARTnulltrue2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:02.317 DEBUG:tasks.rgw_multi.zone_rados:o=MULTIPART 2022-09-03T06:11:02.318 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:02.319 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:02.319 DEBUG:boto:path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:02.320 DEBUG:boto:auth_path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:02.321 DEBUG:boto:Method: GET 2022-09-03T06:11:02.322 DEBUG:boto:Path: /dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:02.323 DEBUG:boto:Data: 2022-09-03T06:11:02.324 DEBUG:boto:Headers: {} 2022-09-03T06:11:02.325 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:11:02.326 DEBUG:boto:Port: 8001 2022-09-03T06:11:02.327 DEBUG:boto:Params: {} 2022-09-03T06:11:02.328 DEBUG:boto:Token: None 2022-09-03T06:11:02.329 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:02 GMT /dsgvcy-65/?versions 2022-09-03T06:11:02.330 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YWz8W7vyOOSzewjasDYRP9xl6hc= 2022-09-03T06:11:02.331 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YWz8W7vyOOSzewjasDYRP9xl6hc=', 'Content-Length': '0'} 2022-09-03T06:11:02.335 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007661cfb94593b9cb-006312eff6-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:02.336 DEBUG:boto:b'dsgvcy-65999falseMULTIPARTnulltrue2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:02.337 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:02.338 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:02.338 DEBUG:boto:path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:02.339 DEBUG:boto:auth_path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:02.339 DEBUG:boto:Method: GET 2022-09-03T06:11:02.340 DEBUG:boto:Path: /dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:02.340 DEBUG:boto:Data: 2022-09-03T06:11:02.341 DEBUG:boto:Headers: {} 2022-09-03T06:11:02.341 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:02.342 DEBUG:boto:Port: 8000 2022-09-03T06:11:02.342 DEBUG:boto:Params: {} 2022-09-03T06:11:02.343 DEBUG:boto:Token: None 2022-09-03T06:11:02.344 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:02 GMT /dsgvcy-65/?versions 2022-09-03T06:11:02.344 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YWz8W7vyOOSzewjasDYRP9xl6hc= 2022-09-03T06:11:02.345 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YWz8W7vyOOSzewjasDYRP9xl6hc=', 'Content-Length': '0'} 2022-09-03T06:11:02.348 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005f0b7212d4b01a94-006312eff6-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:02.348 DEBUG:boto:b'dsgvcy-65999falseMULTIPARTnulltrue2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:02.349 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=MULTIPART 2022-09-03T06:11:02.350 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:02.350 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:02.351 DEBUG:boto:path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.352 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.353 DEBUG:boto:Method: GET 2022-09-03T06:11:02.353 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.354 DEBUG:boto:Data: 2022-09-03T06:11:02.355 DEBUG:boto:Headers: {} 2022-09-03T06:11:02.356 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:11:02.356 DEBUG:boto:Port: 8001 2022-09-03T06:11:02.357 DEBUG:boto:Params: {} 2022-09-03T06:11:02.358 DEBUG:boto:Token: None 2022-09-03T06:11:02.359 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:02 GMT /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.359 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dUobA5fKryYxiU5Oz0PmLRh77jY= 2022-09-03T06:11:02.361 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dUobA5fKryYxiU5Oz0PmLRh77jY=', 'Content-Length': '0'} 2022-09-03T06:11:02.370 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx000004a54cdfa61ec0cc0-006312eff6-15db-test-zone3'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:02.468 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:02.468 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:02.469 DEBUG:boto:path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.470 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.470 DEBUG:boto:Method: GET 2022-09-03T06:11:02.471 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.471 DEBUG:boto:Data: 2022-09-03T06:11:02.472 DEBUG:boto:Headers: {} 2022-09-03T06:11:02.472 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:02.473 DEBUG:boto:Port: 8000 2022-09-03T06:11:02.473 DEBUG:boto:Params: {} 2022-09-03T06:11:02.474 DEBUG:boto:Token: None 2022-09-03T06:11:02.474 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:02 GMT /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.475 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dUobA5fKryYxiU5Oz0PmLRh77jY= 2022-09-03T06:11:02.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dUobA5fKryYxiU5Oz0PmLRh77jY=', 'Content-Length': '0'} 2022-09-03T06:11:02.479 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx00000ee04b597b64cca92-006312eff6-15d7-test-zone1'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:02.587 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:02.588 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:02.588 DEBUG:boto:path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.589 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.589 DEBUG:boto:Method: HEAD 2022-09-03T06:11:02.590 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.590 DEBUG:boto:Data: 2022-09-03T06:11:02.591 DEBUG:boto:Headers: {} 2022-09-03T06:11:02.591 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:11:02.592 DEBUG:boto:Port: 8001 2022-09-03T06:11:02.592 DEBUG:boto:Params: {} 2022-09-03T06:11:02.593 DEBUG:boto:Token: None 2022-09-03T06:11:02.594 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:02 GMT /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.594 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nUhJJe8AfGhhlHz9CX+s/VmEAvg= 2022-09-03T06:11:02.595 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nUhJJe8AfGhhlHz9CX+s/VmEAvg=', 'Content-Length': '0'} 2022-09-03T06:11:02.597 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx0000022eddb95d01fa0d6-006312eff6-15db-test-zone3'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:02.598 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:02.598 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:02.599 DEBUG:boto:path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.599 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.600 DEBUG:boto:Method: HEAD 2022-09-03T06:11:02.600 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.601 DEBUG:boto:Data: 2022-09-03T06:11:02.601 DEBUG:boto:Headers: {} 2022-09-03T06:11:02.602 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:02.602 DEBUG:boto:Port: 8000 2022-09-03T06:11:02.603 DEBUG:boto:Params: {} 2022-09-03T06:11:02.603 DEBUG:boto:Token: None 2022-09-03T06:11:02.604 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:02 GMT /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.604 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nUhJJe8AfGhhlHz9CX+s/VmEAvg= 2022-09-03T06:11:02.605 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nUhJJe8AfGhhlHz9CX+s/VmEAvg=', 'Content-Length': '0'} 2022-09-03T06:11:02.607 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx0000067ece36342d71675-006312eff6-15d7-test-zone1'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:02.608 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=MULTIPART 2022-09-03T06:11:02.609 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:02.609 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:02.610 DEBUG:boto:path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.610 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.611 DEBUG:boto:Method: GET 2022-09-03T06:11:02.611 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.612 DEBUG:boto:Data: 2022-09-03T06:11:02.612 DEBUG:boto:Headers: {} 2022-09-03T06:11:02.613 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:11:02.613 DEBUG:boto:Port: 8001 2022-09-03T06:11:02.614 DEBUG:boto:Params: {} 2022-09-03T06:11:02.615 DEBUG:boto:Token: None 2022-09-03T06:11:02.615 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:02 GMT /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.616 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dUobA5fKryYxiU5Oz0PmLRh77jY= 2022-09-03T06:11:02.616 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dUobA5fKryYxiU5Oz0PmLRh77jY=', 'Content-Length': '0'} 2022-09-03T06:11:02.619 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx00000520ed2e34e34bb07-006312eff6-15db-test-zone3'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:02.714 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:02.715 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:02.715 DEBUG:boto:path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.716 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.716 DEBUG:boto:Method: GET 2022-09-03T06:11:02.717 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.717 DEBUG:boto:Data: 2022-09-03T06:11:02.718 DEBUG:boto:Headers: {} 2022-09-03T06:11:02.718 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:02.719 DEBUG:boto:Port: 8000 2022-09-03T06:11:02.719 DEBUG:boto:Params: {} 2022-09-03T06:11:02.720 DEBUG:boto:Token: None 2022-09-03T06:11:02.721 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:02 GMT /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:02.721 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dUobA5fKryYxiU5Oz0PmLRh77jY= 2022-09-03T06:11:02.722 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dUobA5fKryYxiU5Oz0PmLRh77jY=', 'Content-Length': '0'} 2022-09-03T06:11:02.725 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx000006fbb091501b819eb-006312eff6-15d7-test-zone1'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:02.831 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:02.832 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:02.833 DEBUG:boto:path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:02.833 DEBUG:boto:auth_path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:02.834 DEBUG:boto:Method: GET 2022-09-03T06:11:02.835 DEBUG:boto:Path: /dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:02.835 DEBUG:boto:Data: 2022-09-03T06:11:02.836 DEBUG:boto:Headers: {} 2022-09-03T06:11:02.837 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:11:02.837 DEBUG:boto:Port: 8001 2022-09-03T06:11:02.838 DEBUG:boto:Params: {} 2022-09-03T06:11:02.838 DEBUG:boto:Token: None 2022-09-03T06:11:02.839 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:02 GMT /dsgvcy-65/ 2022-09-03T06:11:02.840 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qDRfsdyXVkdj1AReGffnqSAmP6k= 2022-09-03T06:11:02.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qDRfsdyXVkdj1AReGffnqSAmP6k=', 'Content-Length': '0'} 2022-09-03T06:11:02.844 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000371bd90c2e1edef0-006312eff6-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:02.844 DEBUG:boto:b'dsgvcy-65MULTIPART1000falseMULTIPART2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:02.846 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:02.846 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:02.847 DEBUG:boto:path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:02.848 DEBUG:boto:auth_path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:02.848 DEBUG:boto:Method: GET 2022-09-03T06:11:02.849 DEBUG:boto:Path: /dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:02.849 DEBUG:boto:Data: 2022-09-03T06:11:02.850 DEBUG:boto:Headers: {} 2022-09-03T06:11:02.850 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:02.851 DEBUG:boto:Port: 8000 2022-09-03T06:11:02.852 DEBUG:boto:Params: {} 2022-09-03T06:11:02.852 DEBUG:boto:Token: None 2022-09-03T06:11:02.853 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:02 GMT /dsgvcy-65/ 2022-09-03T06:11:02.854 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qDRfsdyXVkdj1AReGffnqSAmP6k= 2022-09-03T06:11:02.854 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qDRfsdyXVkdj1AReGffnqSAmP6k=', 'Content-Length': '0'} 2022-09-03T06:11:02.857 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ff24a55b813976cb-006312eff6-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:02.858 DEBUG:boto:b'dsgvcy-65MULTIPART1000falseMULTIPART2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:02.858 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:02.859 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:02.860 DEBUG:boto:Method: HEAD 2022-09-03T06:11:02.860 DEBUG:boto:Path: /dsgvcy-65/MULTIPART 2022-09-03T06:11:02.861 DEBUG:boto:Data: 2022-09-03T06:11:02.861 DEBUG:boto:Headers: {} 2022-09-03T06:11:02.862 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:11:02.862 DEBUG:boto:Port: 8001 2022-09-03T06:11:02.863 DEBUG:boto:Params: {} 2022-09-03T06:11:02.863 DEBUG:boto:Token: None 2022-09-03T06:11:02.864 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:02 GMT /dsgvcy-65/MULTIPART 2022-09-03T06:11:02.864 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jOENN4mDFCnZrGMCifHg19acCN4= 2022-09-03T06:11:02.865 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jOENN4mDFCnZrGMCifHg19acCN4=', 'Content-Length': '0'} 2022-09-03T06:11:02.867 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx000002bd93fc4586eac93-006312eff6-15db-test-zone3'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:02.868 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:02.869 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:02.869 DEBUG:boto:Method: HEAD 2022-09-03T06:11:02.870 DEBUG:boto:Path: /dsgvcy-65/MULTIPART 2022-09-03T06:11:02.870 DEBUG:boto:Data: 2022-09-03T06:11:02.871 DEBUG:boto:Headers: {} 2022-09-03T06:11:02.871 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:02.872 DEBUG:boto:Port: 8000 2022-09-03T06:11:02.872 DEBUG:boto:Params: {} 2022-09-03T06:11:02.873 DEBUG:boto:Token: None 2022-09-03T06:11:02.874 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:02 GMT /dsgvcy-65/MULTIPART 2022-09-03T06:11:02.874 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jOENN4mDFCnZrGMCifHg19acCN4= 2022-09-03T06:11:02.875 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jOENN4mDFCnZrGMCifHg19acCN4=', 'Content-Length': '0'} 2022-09-03T06:11:02.877 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx000002e004eebf47a46b3-006312eff6-15d7-test-zone1'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:02.879 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=MULTIPART 2022-09-03T06:11:02.879 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:02.880 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:02.880 DEBUG:boto:Method: GET 2022-09-03T06:11:02.881 DEBUG:boto:Path: /dsgvcy-65/MULTIPART 2022-09-03T06:11:02.882 DEBUG:boto:Data: 2022-09-03T06:11:02.882 DEBUG:boto:Headers: {} 2022-09-03T06:11:02.883 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:11:02.883 DEBUG:boto:Port: 8001 2022-09-03T06:11:02.884 DEBUG:boto:Params: {} 2022-09-03T06:11:02.884 DEBUG:boto:Token: None 2022-09-03T06:11:02.885 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:02 GMT /dsgvcy-65/MULTIPART 2022-09-03T06:11:02.886 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eL/7WQoYZPJ5t/78yJOpDAFwgjs= 2022-09-03T06:11:02.886 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eL/7WQoYZPJ5t/78yJOpDAFwgjs=', 'Content-Length': '0'} 2022-09-03T06:11:02.889 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx00000bce1c433a0ac21f7-006312eff6-15db-test-zone3'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:02.980 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:02.981 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:02.982 DEBUG:boto:Method: GET 2022-09-03T06:11:02.982 DEBUG:boto:Path: /dsgvcy-65/MULTIPART 2022-09-03T06:11:02.983 DEBUG:boto:Data: 2022-09-03T06:11:02.984 DEBUG:boto:Headers: {} 2022-09-03T06:11:02.984 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:02.985 DEBUG:boto:Port: 8000 2022-09-03T06:11:02.985 DEBUG:boto:Params: {} 2022-09-03T06:11:02.986 DEBUG:boto:Token: None 2022-09-03T06:11:02.987 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:02 GMT /dsgvcy-65/MULTIPART 2022-09-03T06:11:02.987 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eL/7WQoYZPJ5t/78yJOpDAFwgjs= 2022-09-03T06:11:02.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eL/7WQoYZPJ5t/78yJOpDAFwgjs=', 'Content-Length': '0'} 2022-09-03T06:11:02.991 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx0000097c0c7a583e5cd8a-006312eff6-15d7-test-zone1'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:03.096 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:03.097 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:03.097 DEBUG:boto:path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:03.098 DEBUG:boto:auth_path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:03.099 DEBUG:boto:Method: GET 2022-09-03T06:11:03.099 DEBUG:boto:Path: /dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:03.100 DEBUG:boto:Data: 2022-09-03T06:11:03.100 DEBUG:boto:Headers: {} 2022-09-03T06:11:03.101 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:11:03.101 DEBUG:boto:Port: 8001 2022-09-03T06:11:03.102 DEBUG:boto:Params: {} 2022-09-03T06:11:03.103 DEBUG:boto:Token: None 2022-09-03T06:11:03.103 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:03 GMT /dsgvcy-65/ 2022-09-03T06:11:03.104 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PGyrbEXvn9gbx/KSKGZubysYlo0= 2022-09-03T06:11:03.105 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PGyrbEXvn9gbx/KSKGZubysYlo0=', 'Content-Length': '0'} 2022-09-03T06:11:03.108 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fb3ae089bfef59b9-006312eff7-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:03.108 DEBUG:boto:b'dsgvcy-65MULTIPART1000falseMULTIPART2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:03.109 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:03.110 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:03.110 DEBUG:boto:path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:03.111 DEBUG:boto:auth_path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:03.112 DEBUG:boto:Method: GET 2022-09-03T06:11:03.112 DEBUG:boto:Path: /dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:03.113 DEBUG:boto:Data: 2022-09-03T06:11:03.113 DEBUG:boto:Headers: {} 2022-09-03T06:11:03.114 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:03.114 DEBUG:boto:Port: 8000 2022-09-03T06:11:03.115 DEBUG:boto:Params: {} 2022-09-03T06:11:03.116 DEBUG:boto:Token: None 2022-09-03T06:11:03.116 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:03 GMT /dsgvcy-65/ 2022-09-03T06:11:03.117 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PGyrbEXvn9gbx/KSKGZubysYlo0= 2022-09-03T06:11:03.118 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PGyrbEXvn9gbx/KSKGZubysYlo0=', 'Content-Length': '0'} 2022-09-03T06:11:03.120 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f727bd3fc7584427-006312eff7-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:03.121 DEBUG:boto:b'dsgvcy-65MULTIPART1000falseMULTIPART2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:03.122 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-65 zones={test-zone3, test-zone1} 2022-09-03T06:11:03.122 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-65 zones={test-zone3, test-zone2} 2022-09-03T06:11:03.123 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:03.123 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:03.125 DEBUG:boto:Method: HEAD 2022-09-03T06:11:03.125 DEBUG:boto:Path: /dsgvcy-65/ 2022-09-03T06:11:03.126 DEBUG:boto:Data: 2022-09-03T06:11:03.126 DEBUG:boto:Headers: {} 2022-09-03T06:11:03.127 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:11:03.127 DEBUG:boto:Port: 8001 2022-09-03T06:11:03.128 DEBUG:boto:Params: {} 2022-09-03T06:11:03.129 DEBUG:boto:Token: None 2022-09-03T06:11:03.129 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:03 GMT /dsgvcy-65/ 2022-09-03T06:11:03.130 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EqKZGbo4cYd21b+F3AZhZVvAMo8= 2022-09-03T06:11:03.130 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EqKZGbo4cYd21b+F3AZhZVvAMo8=', 'Content-Length': '0'} 2022-09-03T06:11:03.133 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '20971520'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000008c5f836573dac41-006312eff7-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:11:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:03.134 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:03.134 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:03.135 DEBUG:boto:Method: HEAD 2022-09-03T06:11:03.135 DEBUG:boto:Path: /dsgvcy-65/ 2022-09-03T06:11:03.136 DEBUG:boto:Data: 2022-09-03T06:11:03.136 DEBUG:boto:Headers: {} 2022-09-03T06:11:03.137 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:03.137 DEBUG:boto:Port: 8000 2022-09-03T06:11:03.138 DEBUG:boto:Params: {} 2022-09-03T06:11:03.139 DEBUG:boto:Token: None 2022-09-03T06:11:03.139 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:03 GMT /dsgvcy-65/ 2022-09-03T06:11:03.140 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EqKZGbo4cYd21b+F3AZhZVvAMo8= 2022-09-03T06:11:03.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EqKZGbo4cYd21b+F3AZhZVvAMo8=', 'Content-Length': '0'} 2022-09-03T06:11:03.143 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '20971520'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000834aa7a1275e29c0-006312eff7-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:11:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:03.144 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:11:03.145 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:03.145 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:03.146 DEBUG:boto:path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:03.146 DEBUG:boto:auth_path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:03.147 DEBUG:boto:Method: GET 2022-09-03T06:11:03.147 DEBUG:boto:Path: /dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:03.148 DEBUG:boto:Data: 2022-09-03T06:11:03.148 DEBUG:boto:Headers: {} 2022-09-03T06:11:03.149 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:11:03.150 DEBUG:boto:Port: 8001 2022-09-03T06:11:03.150 DEBUG:boto:Params: {} 2022-09-03T06:11:03.151 DEBUG:boto:Token: None 2022-09-03T06:11:03.151 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:03 GMT /dsgvcy-65/?versions 2022-09-03T06:11:03.152 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iXM7RHm7hDsuudElJtnjLeQQh1A= 2022-09-03T06:11:03.153 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iXM7RHm7hDsuudElJtnjLeQQh1A=', 'Content-Length': '0'} 2022-09-03T06:11:03.157 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ad853c965687462e-006312eff7-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:03.157 DEBUG:boto:b'dsgvcy-65999falseMULTIPARTnulltrue2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:03.158 DEBUG:tasks.rgw_multi.zone_rados:o=MULTIPART 2022-09-03T06:11:03.159 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:11:03.160 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:03.160 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:03.161 DEBUG:boto:path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:03.161 DEBUG:boto:auth_path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:03.162 DEBUG:boto:Method: GET 2022-09-03T06:11:03.163 DEBUG:boto:Path: /dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:03.163 DEBUG:boto:Data: 2022-09-03T06:11:03.164 DEBUG:boto:Headers: {} 2022-09-03T06:11:03.164 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:03.165 DEBUG:boto:Port: 8000 2022-09-03T06:11:03.165 DEBUG:boto:Params: {} 2022-09-03T06:11:03.167 DEBUG:boto:Token: None 2022-09-03T06:11:03.168 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:03 GMT /dsgvcy-65/?versions 2022-09-03T06:11:03.169 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iXM7RHm7hDsuudElJtnjLeQQh1A= 2022-09-03T06:11:03.170 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iXM7RHm7hDsuudElJtnjLeQQh1A=', 'Content-Length': '0'} 2022-09-03T06:11:03.173 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fa4367ba55943e40-006312eff7-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:03.174 DEBUG:boto:b'dsgvcy-65999falseMULTIPARTnulltrue2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:03.175 DEBUG:tasks.rgw_multi.zone_rados:o=MULTIPART 2022-09-03T06:11:03.176 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:03.177 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:03.178 DEBUG:boto:path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:03.179 DEBUG:boto:auth_path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:03.180 DEBUG:boto:Method: GET 2022-09-03T06:11:03.181 DEBUG:boto:Path: /dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:03.181 DEBUG:boto:Data: 2022-09-03T06:11:03.182 DEBUG:boto:Headers: {} 2022-09-03T06:11:03.183 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:11:03.184 DEBUG:boto:Port: 8001 2022-09-03T06:11:03.185 DEBUG:boto:Params: {} 2022-09-03T06:11:03.186 DEBUG:boto:Token: None 2022-09-03T06:11:03.187 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:03 GMT /dsgvcy-65/?versions 2022-09-03T06:11:03.188 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iXM7RHm7hDsuudElJtnjLeQQh1A= 2022-09-03T06:11:03.189 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iXM7RHm7hDsuudElJtnjLeQQh1A=', 'Content-Length': '0'} 2022-09-03T06:11:03.192 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d8b857eb7a4a5b5c-006312eff7-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:03.236 DEBUG:boto:b'dsgvcy-65999falseMULTIPARTnulltrue2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:03.237 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:03.238 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:03.238 DEBUG:boto:path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:03.239 DEBUG:boto:auth_path=/dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:03.240 DEBUG:boto:Method: GET 2022-09-03T06:11:03.241 DEBUG:boto:Path: /dsgvcy-65/?versions&max-keys=999 2022-09-03T06:11:03.242 DEBUG:boto:Data: 2022-09-03T06:11:03.243 DEBUG:boto:Headers: {} 2022-09-03T06:11:03.244 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:03.244 DEBUG:boto:Port: 8000 2022-09-03T06:11:03.245 DEBUG:boto:Params: {} 2022-09-03T06:11:03.245 DEBUG:boto:Token: None 2022-09-03T06:11:03.247 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:03 GMT /dsgvcy-65/?versions 2022-09-03T06:11:03.248 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iXM7RHm7hDsuudElJtnjLeQQh1A= 2022-09-03T06:11:03.249 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iXM7RHm7hDsuudElJtnjLeQQh1A=', 'Content-Length': '0'} 2022-09-03T06:11:03.252 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006e2dce7fc9eaea42-006312eff7-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:03.252 DEBUG:boto:b'dsgvcy-65999falseMULTIPARTnulltrue2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:03.253 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=MULTIPART 2022-09-03T06:11:03.254 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:03.254 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:03.255 DEBUG:boto:path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.256 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.256 DEBUG:boto:Method: GET 2022-09-03T06:11:03.257 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.257 DEBUG:boto:Data: 2022-09-03T06:11:03.258 DEBUG:boto:Headers: {} 2022-09-03T06:11:03.259 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:11:03.260 DEBUG:boto:Port: 8001 2022-09-03T06:11:03.261 DEBUG:boto:Params: {} 2022-09-03T06:11:03.262 DEBUG:boto:Token: None 2022-09-03T06:11:03.263 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:03 GMT /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.264 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tuVMvGVpmBOcHzjBO3hgYlROnIc= 2022-09-03T06:11:03.265 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tuVMvGVpmBOcHzjBO3hgYlROnIc=', 'Content-Length': '0'} 2022-09-03T06:11:03.269 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx00000dd1a67766c96ff09-006312eff7-15db-test-zone3'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:03.370 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:03.443 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:03.444 DEBUG:boto:path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.445 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.445 DEBUG:boto:Method: GET 2022-09-03T06:11:03.446 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.446 DEBUG:boto:Data: 2022-09-03T06:11:03.447 DEBUG:boto:Headers: {} 2022-09-03T06:11:03.448 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:03.448 DEBUG:boto:Port: 8000 2022-09-03T06:11:03.449 DEBUG:boto:Params: {} 2022-09-03T06:11:03.449 DEBUG:boto:Token: None 2022-09-03T06:11:03.450 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:03 GMT /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.451 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tuVMvGVpmBOcHzjBO3hgYlROnIc= 2022-09-03T06:11:03.451 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tuVMvGVpmBOcHzjBO3hgYlROnIc=', 'Content-Length': '0'} 2022-09-03T06:11:03.455 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx0000084824009746db2b5-006312eff7-15d2-test-zone2'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:03.555 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:03.556 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:03.556 DEBUG:boto:path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.557 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.557 DEBUG:boto:Method: HEAD 2022-09-03T06:11:03.558 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.559 DEBUG:boto:Data: 2022-09-03T06:11:03.559 DEBUG:boto:Headers: {} 2022-09-03T06:11:03.559 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:11:03.560 DEBUG:boto:Port: 8001 2022-09-03T06:11:03.561 DEBUG:boto:Params: {} 2022-09-03T06:11:03.561 DEBUG:boto:Token: None 2022-09-03T06:11:03.562 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:03 GMT /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.562 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ddUMq9f5lOrNGK5efIaqzdGPVAQ= 2022-09-03T06:11:03.563 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ddUMq9f5lOrNGK5efIaqzdGPVAQ=', 'Content-Length': '0'} 2022-09-03T06:11:03.565 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx0000044487b51540f650c-006312eff7-15db-test-zone3'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:03.566 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:03.567 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:03.567 DEBUG:boto:path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.568 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.568 DEBUG:boto:Method: HEAD 2022-09-03T06:11:03.569 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.569 DEBUG:boto:Data: 2022-09-03T06:11:03.570 DEBUG:boto:Headers: {} 2022-09-03T06:11:03.570 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:03.571 DEBUG:boto:Port: 8000 2022-09-03T06:11:03.571 DEBUG:boto:Params: {} 2022-09-03T06:11:03.572 DEBUG:boto:Token: None 2022-09-03T06:11:03.572 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:03 GMT /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.573 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ddUMq9f5lOrNGK5efIaqzdGPVAQ= 2022-09-03T06:11:03.574 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ddUMq9f5lOrNGK5efIaqzdGPVAQ=', 'Content-Length': '0'} 2022-09-03T06:11:03.576 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx00000fcf7140c8eb2485f-006312eff7-15d2-test-zone2'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:03.577 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=MULTIPART 2022-09-03T06:11:03.577 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:03.578 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:03.578 DEBUG:boto:path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.579 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.580 DEBUG:boto:Method: GET 2022-09-03T06:11:03.580 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.581 DEBUG:boto:Data: 2022-09-03T06:11:03.581 DEBUG:boto:Headers: {} 2022-09-03T06:11:03.582 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:11:03.582 DEBUG:boto:Port: 8001 2022-09-03T06:11:03.583 DEBUG:boto:Params: {} 2022-09-03T06:11:03.584 DEBUG:boto:Token: None 2022-09-03T06:11:03.584 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:03 GMT /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.585 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tuVMvGVpmBOcHzjBO3hgYlROnIc= 2022-09-03T06:11:03.585 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tuVMvGVpmBOcHzjBO3hgYlROnIc=', 'Content-Length': '0'} 2022-09-03T06:11:03.588 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx00000ca99bd15bc360904-006312eff7-15db-test-zone3'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:03.685 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:03.686 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:03.686 DEBUG:boto:path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.687 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.687 DEBUG:boto:Method: GET 2022-09-03T06:11:03.688 DEBUG:boto:Path: /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.689 DEBUG:boto:Data: 2022-09-03T06:11:03.689 DEBUG:boto:Headers: {} 2022-09-03T06:11:03.690 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:03.690 DEBUG:boto:Port: 8000 2022-09-03T06:11:03.691 DEBUG:boto:Params: {} 2022-09-03T06:11:03.691 DEBUG:boto:Token: None 2022-09-03T06:11:03.692 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:03 GMT /dsgvcy-65/MULTIPART?versionId=null 2022-09-03T06:11:03.693 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tuVMvGVpmBOcHzjBO3hgYlROnIc= 2022-09-03T06:11:03.693 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tuVMvGVpmBOcHzjBO3hgYlROnIc=', 'Content-Length': '0'} 2022-09-03T06:11:03.697 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx0000077383e48be231ea2-006312eff7-15d2-test-zone2'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:03.803 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:03.803 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:03.804 DEBUG:boto:path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:03.805 DEBUG:boto:auth_path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:03.805 DEBUG:boto:Method: GET 2022-09-03T06:11:03.806 DEBUG:boto:Path: /dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:03.806 DEBUG:boto:Data: 2022-09-03T06:11:03.807 DEBUG:boto:Headers: {} 2022-09-03T06:11:03.807 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:11:03.808 DEBUG:boto:Port: 8001 2022-09-03T06:11:03.808 DEBUG:boto:Params: {} 2022-09-03T06:11:03.809 DEBUG:boto:Token: None 2022-09-03T06:11:03.809 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:03 GMT /dsgvcy-65/ 2022-09-03T06:11:03.810 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PGyrbEXvn9gbx/KSKGZubysYlo0= 2022-09-03T06:11:03.811 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PGyrbEXvn9gbx/KSKGZubysYlo0=', 'Content-Length': '0'} 2022-09-03T06:11:03.814 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000048d0e0ed6f7bde70-006312eff7-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:03.814 DEBUG:boto:b'dsgvcy-65MULTIPART1000falseMULTIPART2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:03.815 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:03.816 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:03.817 DEBUG:boto:path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:03.817 DEBUG:boto:auth_path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:03.818 DEBUG:boto:Method: GET 2022-09-03T06:11:03.818 DEBUG:boto:Path: /dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:03.819 DEBUG:boto:Data: 2022-09-03T06:11:03.819 DEBUG:boto:Headers: {} 2022-09-03T06:11:03.820 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:03.821 DEBUG:boto:Port: 8000 2022-09-03T06:11:03.821 DEBUG:boto:Params: {} 2022-09-03T06:11:03.822 DEBUG:boto:Token: None 2022-09-03T06:11:03.822 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:03 GMT /dsgvcy-65/ 2022-09-03T06:11:03.823 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PGyrbEXvn9gbx/KSKGZubysYlo0= 2022-09-03T06:11:03.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PGyrbEXvn9gbx/KSKGZubysYlo0=', 'Content-Length': '0'} 2022-09-03T06:11:03.827 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000076114b2cfa79c50-006312eff7-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:03.828 DEBUG:boto:b'dsgvcy-65MULTIPART1000falseMULTIPART2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:03.829 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:03.829 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:03.830 DEBUG:boto:Method: HEAD 2022-09-03T06:11:03.830 DEBUG:boto:Path: /dsgvcy-65/MULTIPART 2022-09-03T06:11:03.831 DEBUG:boto:Data: 2022-09-03T06:11:03.831 DEBUG:boto:Headers: {} 2022-09-03T06:11:03.832 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:11:03.832 DEBUG:boto:Port: 8001 2022-09-03T06:11:03.833 DEBUG:boto:Params: {} 2022-09-03T06:11:03.833 DEBUG:boto:Token: None 2022-09-03T06:11:03.834 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:03 GMT /dsgvcy-65/MULTIPART 2022-09-03T06:11:03.835 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tvPhLbArzPtDEq71n4LwpqwRfiM= 2022-09-03T06:11:03.835 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tvPhLbArzPtDEq71n4LwpqwRfiM=', 'Content-Length': '0'} 2022-09-03T06:11:03.837 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx0000090996ea04ff87bc2-006312eff7-15db-test-zone3'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:03.838 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:03.839 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:03.839 DEBUG:boto:Method: HEAD 2022-09-03T06:11:03.840 DEBUG:boto:Path: /dsgvcy-65/MULTIPART 2022-09-03T06:11:03.840 DEBUG:boto:Data: 2022-09-03T06:11:03.841 DEBUG:boto:Headers: {} 2022-09-03T06:11:03.841 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:03.842 DEBUG:boto:Port: 8000 2022-09-03T06:11:03.842 DEBUG:boto:Params: {} 2022-09-03T06:11:03.843 DEBUG:boto:Token: None 2022-09-03T06:11:03.843 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:03 GMT /dsgvcy-65/MULTIPART 2022-09-03T06:11:03.844 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tvPhLbArzPtDEq71n4LwpqwRfiM= 2022-09-03T06:11:03.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tvPhLbArzPtDEq71n4LwpqwRfiM=', 'Content-Length': '0'} 2022-09-03T06:11:03.847 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx00000cb875d951a07ad53-006312eff7-15d2-test-zone2'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:03.848 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=MULTIPART 2022-09-03T06:11:03.848 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:03.849 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:03.850 DEBUG:boto:Method: GET 2022-09-03T06:11:03.850 DEBUG:boto:Path: /dsgvcy-65/MULTIPART 2022-09-03T06:11:03.851 DEBUG:boto:Data: 2022-09-03T06:11:03.851 DEBUG:boto:Headers: {} 2022-09-03T06:11:03.852 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:11:03.852 DEBUG:boto:Port: 8001 2022-09-03T06:11:03.853 DEBUG:boto:Params: {} 2022-09-03T06:11:03.854 DEBUG:boto:Token: None 2022-09-03T06:11:03.854 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:03 GMT /dsgvcy-65/MULTIPART 2022-09-03T06:11:03.855 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AcOWkwJJqnzDNYYrDVglDjG1Tp8= 2022-09-03T06:11:03.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AcOWkwJJqnzDNYYrDVglDjG1Tp8=', 'Content-Length': '0'} 2022-09-03T06:11:03.859 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx00000400febcc2a44d546-006312eff7-15db-test-zone3'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:03.955 DEBUG:boto:path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:03.956 DEBUG:boto:auth_path=/dsgvcy-65/MULTIPART 2022-09-03T06:11:03.957 DEBUG:boto:Method: GET 2022-09-03T06:11:03.957 DEBUG:boto:Path: /dsgvcy-65/MULTIPART 2022-09-03T06:11:03.958 DEBUG:boto:Data: 2022-09-03T06:11:03.958 DEBUG:boto:Headers: {} 2022-09-03T06:11:03.959 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:03.959 DEBUG:boto:Port: 8000 2022-09-03T06:11:03.960 DEBUG:boto:Params: {} 2022-09-03T06:11:03.960 DEBUG:boto:Token: None 2022-09-03T06:11:03.961 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:03 GMT /dsgvcy-65/MULTIPART 2022-09-03T06:11:03.962 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AcOWkwJJqnzDNYYrDVglDjG1Tp8= 2022-09-03T06:11:03.962 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AcOWkwJJqnzDNYYrDVglDjG1Tp8=', 'Content-Length': '0'} 2022-09-03T06:11:03.965 DEBUG:boto:Response headers: [('Content-Length', '20971520'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:09:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"d07d411772772538cd4b657da660bfb5-4"'), ('x-amz-request-id', 'tx00000b5ae06e211d66adf-006312eff7-15d2-test-zone2'), ('Content-Type', 'binary/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:04.081 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:04.082 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:04.083 DEBUG:boto:path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:04.083 DEBUG:boto:auth_path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:04.084 DEBUG:boto:Method: GET 2022-09-03T06:11:04.084 DEBUG:boto:Path: /dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:04.085 DEBUG:boto:Data: 2022-09-03T06:11:04.085 DEBUG:boto:Headers: {} 2022-09-03T06:11:04.086 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:11:04.087 DEBUG:boto:Port: 8001 2022-09-03T06:11:04.087 DEBUG:boto:Params: {} 2022-09-03T06:11:04.088 DEBUG:boto:Token: None 2022-09-03T06:11:04.088 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:04 GMT /dsgvcy-65/ 2022-09-03T06:11:04.089 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/FbupE5mkYsVbSGpXutQOi5KgzQ= 2022-09-03T06:11:04.090 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/FbupE5mkYsVbSGpXutQOi5KgzQ=', 'Content-Length': '0'} 2022-09-03T06:11:04.093 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008a106cbb50657836-006312eff8-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:04.094 DEBUG:boto:b'dsgvcy-65MULTIPART1000falseMULTIPART2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:04.095 DEBUG:boto:path=/dsgvcy-65/ 2022-09-03T06:11:04.096 DEBUG:boto:auth_path=/dsgvcy-65/ 2022-09-03T06:11:04.097 DEBUG:boto:path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:04.097 DEBUG:boto:auth_path=/dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:04.098 DEBUG:boto:Method: GET 2022-09-03T06:11:04.098 DEBUG:boto:Path: /dsgvcy-65/?prefix=MULTIPART 2022-09-03T06:11:04.099 DEBUG:boto:Data: 2022-09-03T06:11:04.100 DEBUG:boto:Headers: {} 2022-09-03T06:11:04.101 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:04.101 DEBUG:boto:Port: 8000 2022-09-03T06:11:04.102 DEBUG:boto:Params: {} 2022-09-03T06:11:04.102 DEBUG:boto:Token: None 2022-09-03T06:11:04.103 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:04 GMT /dsgvcy-65/ 2022-09-03T06:11:04.104 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/FbupE5mkYsVbSGpXutQOi5KgzQ= 2022-09-03T06:11:04.105 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/FbupE5mkYsVbSGpXutQOi5KgzQ=', 'Content-Length': '0'} 2022-09-03T06:11:04.108 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f5060ae46b55e104-006312eff8-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:11:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:04.154 DEBUG:boto:b'dsgvcy-65MULTIPART1000falseMULTIPART2022-09-03T06:09:39.779Z"d07d411772772538cd4b657da660bfb5-4"20971520STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:11:04.155 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-65 zones={test-zone3, test-zone2} 2022-09-03T06:11:04.156 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_multipart_object_sync ... ok 2022-09-03T06:11:04.157 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-68 2022-09-03T06:11:04.158 DEBUG:boto:path=/dsgvcy-68/ 2022-09-03T06:11:04.158 DEBUG:boto:auth_path=/dsgvcy-68/ 2022-09-03T06:11:04.159 DEBUG:boto:Method: PUT 2022-09-03T06:11:04.159 DEBUG:boto:Path: /dsgvcy-68/ 2022-09-03T06:11:04.160 DEBUG:boto:Data: 2022-09-03T06:11:04.160 DEBUG:boto:Headers: {} 2022-09-03T06:11:04.161 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:04.161 DEBUG:boto:Port: 8000 2022-09-03T06:11:04.162 DEBUG:boto:Params: {} 2022-09-03T06:11:04.162 DEBUG:boto:Token: None 2022-09-03T06:11:04.163 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 06:11:04 GMT /dsgvcy-68/ 2022-09-03T06:11:04.163 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0kNNYxHtbCds+SEE0fhY2D8NYzY= 2022-09-03T06:11:04.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0kNNYxHtbCds+SEE0fhY2D8NYzY=', 'Content-Length': '0'} 2022-09-03T06:11:04.191 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000c65fbd7855a488a7-006312eff8-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:11:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:04.192 DEBUG:boto:path=/dsgvcy-68/testobj-sse-c 2022-09-03T06:11:04.193 DEBUG:boto:auth_path=/dsgvcy-68/testobj-sse-c 2022-09-03T06:11:04.194 DEBUG:boto:Method: PUT 2022-09-03T06:11:04.194 DEBUG:boto:Path: /dsgvcy-68/testobj-sse-c 2022-09-03T06:11:04.195 DEBUG:boto:Data: 2022-09-03T06:11:04.196 DEBUG:boto:Headers: {'x-amz-server-side-encryption-customer-algorithm': 'AES256', 'x-amz-server-side-encryption-customer-key': 'pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs=', 'x-amz-server-side-encryption-customer-key-md5': 'DWygnHRtgiJ77HCm+1rvHw==', 'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3FCGuEcom6i4veFJuDiBdQ==', 'Content-Length': '512', 'Expect': '100-Continue'} 2022-09-03T06:11:04.197 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:04.198 DEBUG:boto:Port: 8000 2022-09-03T06:11:04.199 DEBUG:boto:Params: {} 2022-09-03T06:11:04.199 DEBUG:boto:Token: None 2022-09-03T06:11:04.200 DEBUG:boto:StringToSign: PUT 3FCGuEcom6i4veFJuDiBdQ== application/octet-stream Sat, 03 Sep 2022 06:11:04 GMT x-amz-server-side-encryption-customer-algorithm:AES256 x-amz-server-side-encryption-customer-key:pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs= x-amz-server-side-encryption-customer-key-md5:DWygnHRtgiJ77HCm+1rvHw== /dsgvcy-68/testobj-sse-c 2022-09-03T06:11:04.201 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RCGyv+OtEKnKKBl0ylNdqLkPmkg= 2022-09-03T06:11:04.202 DEBUG:boto:Final headers: {'x-amz-server-side-encryption-customer-algorithm': 'AES256', 'x-amz-server-side-encryption-customer-key': 'pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs=', 'x-amz-server-side-encryption-customer-key-md5': 'DWygnHRtgiJ77HCm+1rvHw==', 'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3FCGuEcom6i4veFJuDiBdQ==', 'Content-Length': '512', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:11:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RCGyv+OtEKnKKBl0ylNdqLkPmkg='} 2022-09-03T06:11:04.217 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"dc5086b847289ba8b8bde149b8388175"'), ('Accept-Ranges', 'bytes'), ('x-amz-server-side-encryption-customer-algorithm', 'AES256'), ('x-amz-server-side-encryption-customer-key-MD5', 'DWygnHRtgiJ77HCm+1rvHw=='), ('x-amz-request-id', 'tx000009a0e18c33ec19c93-006312eff8-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:11:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:04.218 DEBUG:boto:path=/dsgvcy-68/testobj-sse-kms 2022-09-03T06:11:04.219 DEBUG:boto:auth_path=/dsgvcy-68/testobj-sse-kms 2022-09-03T06:11:04.220 DEBUG:boto:Method: PUT 2022-09-03T06:11:04.222 DEBUG:boto:Path: /dsgvcy-68/testobj-sse-kms 2022-09-03T06:11:04.223 DEBUG:boto:Data: 2022-09-03T06:11:04.224 DEBUG:boto:Headers: {'x-amz-server-side-encryption': 'aws:kms', 'x-amz-server-side-encryption-aws-kms-key-id': 'testkey-1', 'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3FCGuEcom6i4veFJuDiBdQ==', 'Content-Length': '512', 'Expect': '100-Continue'} 2022-09-03T06:11:04.225 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:04.226 DEBUG:boto:Port: 8000 2022-09-03T06:11:04.226 DEBUG:boto:Params: {} 2022-09-03T06:11:04.227 DEBUG:boto:Token: None 2022-09-03T06:11:04.228 DEBUG:boto:StringToSign: PUT 3FCGuEcom6i4veFJuDiBdQ== application/octet-stream Sat, 03 Sep 2022 06:11:04 GMT x-amz-server-side-encryption:aws:kms x-amz-server-side-encryption-aws-kms-key-id:testkey-1 /dsgvcy-68/testobj-sse-kms 2022-09-03T06:11:04.230 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YH4um1aiQwYZ/fsS08lIHNN1rpM= 2022-09-03T06:11:04.230 DEBUG:boto:Final headers: {'x-amz-server-side-encryption': 'aws:kms', 'x-amz-server-side-encryption-aws-kms-key-id': 'testkey-1', 'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': '3FCGuEcom6i4veFJuDiBdQ==', 'Content-Length': '512', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:11:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YH4um1aiQwYZ/fsS08lIHNN1rpM='} 2022-09-03T06:11:04.243 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"dc5086b847289ba8b8bde149b8388175"'), ('Accept-Ranges', 'bytes'), ('x-amz-server-side-encryption', 'aws:kms'), ('x-amz-server-side-encryption-aws-kms-key-id', 'testkey-1'), ('x-amz-server-side-encryption-context', 'eyJhd3M6czM6YXJuIjoiYXJuOmF3czpzMzo6cmd3LW11bHRpc2l0ZS10ZXN0LXVzZXI6ZHNndmN5LTY4L3Rlc3RvYmotc3NlLWttcyJ9'), ('x-amz-request-id', 'tx000004468c944e5498810-006312eff8-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:11:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:04.244 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:11:04.245 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:11:04.245 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:11:04.299 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:11:04.300 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:11:04.336 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T06:11:04.348 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T06:11:04.349 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:11:04.349 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:11:04.349 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:11:04.350 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:11:04.350 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:11:04.350 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:11:04.351 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:11:04.351 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:11:04.351 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:11:04.352 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662185464.187238_413.1", 2022-09-03T06:11:04.352 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:11:04.187238Z" 2022-09-03T06:11:04.352 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:11:04.352 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:11:04.353 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:11:04.353 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:11:04.353 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:11:04.353 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T06:11:04.355 DEBUG:rgw_multi.tests:master meta markers={0: '', 1: '', 2: '1_1662185464.187238_413.1', 3: ''} 2022-09-03T06:11:04.355 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T06:11:04.356 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:11:04.357 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:11:04.357 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:11:04.420 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:11:04.421 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:11:04.469 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:04.467+0000 7f936c1ba600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:11:04.470 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:04.468+0000 7f936c1ba600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:11:04.488 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:11:04.488 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:11:04.489 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:11:04.489 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:11:04.489 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:11:04.489 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:11:04.490 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:11:04.490 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:04.490 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:11:04.490 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:11:04.491 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:11:04.491 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:11:04.491 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:11:04.491 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:11:04.492 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:11:04.492 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:11:04.492 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:11:04.493 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:11:04.493 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:11:04.493 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:04.493 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:04.494 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:11:04.494 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:11:04.494 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:11:04.494 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:11:04.495 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:11:04.495 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:11:04.496 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:11:04.496 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:11:04.496 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:11:04.497 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:11:04.497 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:04.497 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:04.497 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:11:04.498 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:11:04.498 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:11:04.498 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:11:04.498 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662185464.187238_413.1", 2022-09-03T06:11:04.499 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:11:04.499 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:11:04.500 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:11:04.500 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:11:04.187238Z", 2022-09-03T06:11:04.500 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:11:04.501 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:04.501 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:04.501 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:11:04.502 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:11:04.502 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:11:04.502 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:11:04.503 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:11:04.503 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:11:04.503 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:11:04.504 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:11:04.504 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:11:04.504 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:11:04.505 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:04.505 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:04.506 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:11:04.507 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:04.507 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:11:04.507 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:11:04.507 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:11:04.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:04.508 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:11:04Z" 2022-09-03T06:11:04.508 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:11:04.510 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185464.187238_413.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:11:04.187238Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:11:04Z" } 2022-09-03T06:11:04.510 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185464.187238_413.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:11:04.187238Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:11:04.511 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185464.187238_413.1', 3: ''} 2022-09-03T06:11:04.511 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185464.187238_413.1', 3: ''} 2022-09-03T06:11:04.512 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T06:11:04.512 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T06:11:04.513 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:11:04.513 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:11:04.514 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:11:04.564 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:11:04.565 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:11:04.605 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:11:04.604+0000 7fc2ac57e600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:11:04.606 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:11:04.605+0000 7fc2ac57e600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:11:04.626 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T06:11:04.627 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T06:11:04.627 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T06:11:04.627 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T06:11:04.628 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T06:11:04.628 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:11:04.628 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:11:04.629 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:11:04.629 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T06:11:04.629 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:11:04.629 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T06:11:04.630 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:11:04.630 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:11:04.630 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:11:04.631 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:11:04.631 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T06:11:04.631 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:11:04.631 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:11:04.632 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:11:04.632 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:11:04.632 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:11:04.633 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:11:04.633 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T06:11:04.633 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:11:04.633 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:11:04.634 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:11:04.634 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:11:04.634 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:11:04.635 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:11:04.635 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T06:11:04.635 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:11:04.636 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:11:04.636 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:11:04.637 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:11:04.637 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T06:11:04.638 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:11:04.638 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:11:04.638 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662185464.187238_413.1", 2022-09-03T06:11:04.638 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:11:04.639 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:11:04.639 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:11:04.639 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:11:04.187238Z", 2022-09-03T06:11:04.640 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:11:04.640 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:11:04.640 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:11:04.640 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:11:04.641 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T06:11:04.641 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:11:04.641 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:11:04.642 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:11:04.642 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:11:04.642 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:11:04.643 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:11:04.643 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:11:04.643 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:11:04.643 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:11:04.644 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:11:04.644 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:11:04.644 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:11:04.644 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T06:11:04.645 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T06:11:04.645 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T06:11:04.645 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:11:04.645 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T06:11:04Z" 2022-09-03T06:11:04.646 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T06:11:04.647 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185464.187238_413.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:11:04.187238Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:11:04Z" } 2022-09-03T06:11:04.648 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185464.187238_413.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:11:04.187238Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:11:04.648 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185464.187238_413.1', 3: ''} 2022-09-03T06:11:04.649 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185464.187238_413.1', 3: ''} 2022-09-03T06:11:04.649 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T06:11:04.650 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T06:11:04.650 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:11:04.651 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:11:04.652 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:11:04.702 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:11:04.702 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:11:04.738 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:04.736+0000 7f6030858600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:11:04.739 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:04.737+0000 7f6030858600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:11:04.751 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:11:04.751 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:11:04.751 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:11:04.752 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:11:04.752 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:11:04.752 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:11:04.752 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:11:04.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:04.753 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:11:04.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:11:04.753 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:11:04.754 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:11:04.754 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:11:04.754 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:11:04.754 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:11:04.755 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:11:04.755 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:11:04.755 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:11:04.755 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:11:04.756 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:04.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:04.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:11:04.756 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:11:04.756 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:11:04.757 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:11:04.757 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:11:04.758 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:11:04.758 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:11:04.758 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:11:04.758 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:11:04.759 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:11:04.759 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:04.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:04.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:11:04.760 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:11:04.760 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:11:04.760 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:11:04.760 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662185464.187238_413.1", 2022-09-03T06:11:04.761 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:11:04.761 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:11:04.761 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:11:04.761 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:11:04.187238Z", 2022-09-03T06:11:04.762 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:11:04.762 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:04.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:04.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:11:04.763 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:11:04.763 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:11:04.763 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:11:04.763 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:11:04.764 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:11:04.764 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:11:04.764 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:11:04.764 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:11:04.764 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:11:04.765 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:04.765 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:04.765 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:11:04.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:04.766 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:11:04.766 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:11:04.766 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:11:04.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:04.767 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:11:04Z" 2022-09-03T06:11:04.767 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:11:04.769 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185464.187238_413.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:11:04.187238Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:11:04Z" } 2022-09-03T06:11:04.769 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185464.187238_413.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:11:04.187238Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:11:04.770 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185464.187238_413.1', 3: ''} 2022-09-03T06:11:04.770 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185464.187238_413.1', 3: ''} 2022-09-03T06:11:04.771 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T06:11:04.771 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-68', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:11:04.772 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-68', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:11:04.772 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-68 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:11:04.828 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:11:04.829 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:11:04.863 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:04.862+0000 7fed1b15c600 1 waiting to reach incremental sync.. 2022-09-03T06:11:09.865 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:09.862+0000 7fed1b15c600 1 waiting to reach incremental sync.. 2022-09-03T06:11:14.865 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:14.863+0000 7fed1b15c600 1 waiting to reach incremental sync.. 2022-09-03T06:11:19.865 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:19.863+0000 7fed1b15c600 1 waiting to reach incremental sync.. 2022-09-03T06:11:24.868 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:24.865+0000 7fed1b15c600 1 bucket sync caught up with source: 2022-09-03T06:11:24.869 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.47.5, , , , 00000000001.43.5, , , , , ] 2022-09-03T06:11:24.870 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.47.5, , , , 00000000001.43.5, , , , , ] 2022-09-03T06:11:24.870 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:24.865+0000 7fed1b15c600 0 bucket checkpoint complete 2022-09-03T06:11:24.888 DEBUG:boto:path=/dsgvcy-68/ 2022-09-03T06:11:24.889 DEBUG:boto:auth_path=/dsgvcy-68/ 2022-09-03T06:11:24.890 DEBUG:boto:Method: HEAD 2022-09-03T06:11:24.891 DEBUG:boto:Path: /dsgvcy-68/ 2022-09-03T06:11:24.892 DEBUG:boto:Data: 2022-09-03T06:11:24.893 DEBUG:boto:Headers: {} 2022-09-03T06:11:24.894 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:24.895 DEBUG:boto:Port: 8000 2022-09-03T06:11:24.895 DEBUG:boto:Params: {} 2022-09-03T06:11:24.896 DEBUG:boto:Token: None 2022-09-03T06:11:24.897 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:24 GMT /dsgvcy-68/ 2022-09-03T06:11:24.897 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VeS49IkdqUwJ0nViBQYWME2M3fw= 2022-09-03T06:11:24.898 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VeS49IkdqUwJ0nViBQYWME2M3fw=', 'Content-Length': '0'} 2022-09-03T06:11:24.901 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '1024'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000648fc821a4faad93-006312f00c-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:11:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:24.902 DEBUG:boto:path=/dsgvcy-68/testobj-sse-c 2022-09-03T06:11:24.902 DEBUG:boto:auth_path=/dsgvcy-68/testobj-sse-c 2022-09-03T06:11:24.903 DEBUG:boto:Method: HEAD 2022-09-03T06:11:24.904 DEBUG:boto:Path: /dsgvcy-68/testobj-sse-c 2022-09-03T06:11:24.904 DEBUG:boto:Data: 2022-09-03T06:11:24.905 DEBUG:boto:Headers: {'x-amz-server-side-encryption-customer-algorithm': 'AES256', 'x-amz-server-side-encryption-customer-key': 'pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs=', 'x-amz-server-side-encryption-customer-key-md5': 'DWygnHRtgiJ77HCm+1rvHw=='} 2022-09-03T06:11:24.905 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:24.906 DEBUG:boto:Port: 8000 2022-09-03T06:11:24.906 DEBUG:boto:Params: {} 2022-09-03T06:11:24.907 DEBUG:boto:Token: None 2022-09-03T06:11:24.907 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:24 GMT x-amz-server-side-encryption-customer-algorithm:AES256 x-amz-server-side-encryption-customer-key:pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs= x-amz-server-side-encryption-customer-key-md5:DWygnHRtgiJ77HCm+1rvHw== /dsgvcy-68/testobj-sse-c 2022-09-03T06:11:24.908 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7UVoNhV53CubSFbHpoeDmCjx6zI= 2022-09-03T06:11:24.909 DEBUG:boto:Final headers: {'x-amz-server-side-encryption-customer-algorithm': 'AES256', 'x-amz-server-side-encryption-customer-key': 'pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs=', 'x-amz-server-side-encryption-customer-key-md5': 'DWygnHRtgiJ77HCm+1rvHw==', 'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7UVoNhV53CubSFbHpoeDmCjx6zI=', 'Content-Length': '0'} 2022-09-03T06:11:24.911 DEBUG:boto:Response headers: [('Content-Length', '512'), ('x-amz-server-side-encryption-customer-algorithm', 'AES256'), ('x-amz-server-side-encryption-customer-key-MD5', 'DWygnHRtgiJ77HCm+1rvHw=='), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:04 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"dc5086b847289ba8b8bde149b8388175"'), ('x-amz-request-id', 'tx000008158e20bdf5f8a0a-006312f00c-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:24.913 DEBUG:boto:path=/dsgvcy-68/testobj-sse-c 2022-09-03T06:11:24.913 DEBUG:boto:auth_path=/dsgvcy-68/testobj-sse-c 2022-09-03T06:11:24.914 DEBUG:boto:Method: GET 2022-09-03T06:11:24.914 DEBUG:boto:Path: /dsgvcy-68/testobj-sse-c 2022-09-03T06:11:24.915 DEBUG:boto:Data: 2022-09-03T06:11:24.916 DEBUG:boto:Headers: {'x-amz-server-side-encryption-customer-algorithm': 'AES256', 'x-amz-server-side-encryption-customer-key': 'pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs=', 'x-amz-server-side-encryption-customer-key-md5': 'DWygnHRtgiJ77HCm+1rvHw=='} 2022-09-03T06:11:24.917 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:24.917 DEBUG:boto:Port: 8000 2022-09-03T06:11:24.918 DEBUG:boto:Params: {} 2022-09-03T06:11:24.919 DEBUG:boto:Token: None 2022-09-03T06:11:24.920 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:24 GMT x-amz-server-side-encryption-customer-algorithm:AES256 x-amz-server-side-encryption-customer-key:pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs= x-amz-server-side-encryption-customer-key-md5:DWygnHRtgiJ77HCm+1rvHw== /dsgvcy-68/testobj-sse-c 2022-09-03T06:11:24.920 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RypOJ/K5hE0um/uePoS4Br8ybuY= 2022-09-03T06:11:24.921 DEBUG:boto:Final headers: {'x-amz-server-side-encryption-customer-algorithm': 'AES256', 'x-amz-server-side-encryption-customer-key': 'pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs=', 'x-amz-server-side-encryption-customer-key-md5': 'DWygnHRtgiJ77HCm+1rvHw==', 'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RypOJ/K5hE0um/uePoS4Br8ybuY=', 'Content-Length': '0'} 2022-09-03T06:11:24.925 DEBUG:boto:Response headers: [('Content-Length', '512'), ('x-amz-server-side-encryption-customer-algorithm', 'AES256'), ('x-amz-server-side-encryption-customer-key-MD5', 'DWygnHRtgiJ77HCm+1rvHw=='), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:04 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"dc5086b847289ba8b8bde149b8388175"'), ('x-amz-request-id', 'tx000002deaf732026abcd9-006312f00c-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:24.926 DEBUG:boto:path=/dsgvcy-68/testobj-sse-kms 2022-09-03T06:11:24.927 DEBUG:boto:auth_path=/dsgvcy-68/testobj-sse-kms 2022-09-03T06:11:24.928 DEBUG:boto:Method: HEAD 2022-09-03T06:11:24.929 DEBUG:boto:Path: /dsgvcy-68/testobj-sse-kms 2022-09-03T06:11:24.929 DEBUG:boto:Data: 2022-09-03T06:11:24.930 DEBUG:boto:Headers: {} 2022-09-03T06:11:24.931 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:24.932 DEBUG:boto:Port: 8000 2022-09-03T06:11:24.933 DEBUG:boto:Params: {} 2022-09-03T06:11:24.935 DEBUG:boto:Token: None 2022-09-03T06:11:24.936 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:24 GMT /dsgvcy-68/testobj-sse-kms 2022-09-03T06:11:24.937 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wQMRBLMD29XEB9bdtCSEKXGownw= 2022-09-03T06:11:24.938 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wQMRBLMD29XEB9bdtCSEKXGownw=', 'Content-Length': '0'} 2022-09-03T06:11:24.941 DEBUG:boto:Response headers: [('Content-Length', '512'), ('x-amz-server-side-encryption', 'aws:kms'), ('x-amz-server-side-encryption-aws-kms-key-id', 'testkey-1'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:04 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"dc5086b847289ba8b8bde149b8388175"'), ('x-amz-request-id', 'tx000006440b01f14d47509-006312f00c-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:24.942 DEBUG:boto:path=/dsgvcy-68/testobj-sse-kms 2022-09-03T06:11:24.943 DEBUG:boto:auth_path=/dsgvcy-68/testobj-sse-kms 2022-09-03T06:11:24.944 DEBUG:boto:Method: GET 2022-09-03T06:11:24.945 DEBUG:boto:Path: /dsgvcy-68/testobj-sse-kms 2022-09-03T06:11:24.946 DEBUG:boto:Data: 2022-09-03T06:11:24.947 DEBUG:boto:Headers: {} 2022-09-03T06:11:24.948 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:11:24.949 DEBUG:boto:Port: 8000 2022-09-03T06:11:24.950 DEBUG:boto:Params: {} 2022-09-03T06:11:24.951 DEBUG:boto:Token: None 2022-09-03T06:11:24.952 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:11:24 GMT /dsgvcy-68/testobj-sse-kms 2022-09-03T06:11:24.952 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oOU9Ti9OB1hx7G6lodZYxfm3WjM= 2022-09-03T06:11:24.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oOU9Ti9OB1hx7G6lodZYxfm3WjM=', 'Content-Length': '0'} 2022-09-03T06:11:24.956 DEBUG:boto:Response headers: [('Content-Length', '512'), ('x-amz-server-side-encryption', 'aws:kms'), ('x-amz-server-side-encryption-aws-kms-key-id', 'testkey-1'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:04 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"dc5086b847289ba8b8bde149b8388175"'), ('x-amz-request-id', 'tx000001a414199fb6f2118-006312f00c-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:11:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:24.957 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_encrypted_object_sync ... ok 2022-09-03T06:11:24.958 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-69 2022-09-03T06:11:24.959 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:11:24.960 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:11:24.960 DEBUG:boto:Method: PUT 2022-09-03T06:11:24.961 DEBUG:boto:Path: /dsgvcy-69/ 2022-09-03T06:11:24.961 DEBUG:boto:Data: 2022-09-03T06:11:24.962 DEBUG:boto:Headers: {} 2022-09-03T06:11:24.962 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:24.963 DEBUG:boto:Port: 8000 2022-09-03T06:11:24.963 DEBUG:boto:Params: {} 2022-09-03T06:11:24.964 DEBUG:boto:Token: None 2022-09-03T06:11:24.965 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 06:11:24 GMT /dsgvcy-69/ 2022-09-03T06:11:24.965 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3bmxcPTzm8HvSeWoBZfvOGpdLhM= 2022-09-03T06:11:24.966 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3bmxcPTzm8HvSeWoBZfvOGpdLhM=', 'Content-Length': '0'} 2022-09-03T06:11:24.992 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000004ad9e15f215015d5-006312f00c-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:11:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:24.994 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:11:24.995 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:11:24.995 DEBUG:boto:Method: HEAD 2022-09-03T06:11:24.996 DEBUG:boto:Path: /dsgvcy-69/ 2022-09-03T06:11:24.996 DEBUG:boto:Data: 2022-09-03T06:11:24.997 DEBUG:boto:Headers: {} 2022-09-03T06:11:24.997 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:24.998 DEBUG:boto:Port: 8000 2022-09-03T06:11:24.999 DEBUG:boto:Params: {} 2022-09-03T06:11:25.000 DEBUG:boto:Token: None 2022-09-03T06:11:25.000 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:25 GMT /dsgvcy-69/ 2022-09-03T06:11:25.001 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fRr/NcJ8YcQ8nRwKplphCagNmJw= 2022-09-03T06:11:25.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fRr/NcJ8YcQ8nRwKplphCagNmJw=', 'Content-Length': '0'} 2022-09-03T06:11:25.006 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000752e553a950352d9-006312f00d-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:25.008 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:11:25.009 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:11:25.009 DEBUG:boto:Method: PUT 2022-09-03T06:11:25.010 DEBUG:boto:Path: /dsgvcy-69/a 2022-09-03T06:11:25.011 DEBUG:boto:Data: 2022-09-03T06:11:25.012 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:11:25.013 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:25.013 DEBUG:boto:Port: 8000 2022-09-03T06:11:25.014 DEBUG:boto:Params: {} 2022-09-03T06:11:25.016 DEBUG:boto:Token: None 2022-09-03T06:11:25.017 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:11:25 GMT /dsgvcy-69/a 2022-09-03T06:11:25.018 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X4IfdfhZrZmw5pZmsh2A+3T9Zfg= 2022-09-03T06:11:25.019 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:11:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X4IfdfhZrZmw5pZmsh2A+3T9Zfg='} 2022-09-03T06:11:25.062 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000029141000a3fd8f13-006312f00d-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:25.064 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:11:25.065 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:11:25.066 DEBUG:boto:Method: HEAD 2022-09-03T06:11:25.067 DEBUG:boto:Path: /dsgvcy-69/ 2022-09-03T06:11:25.068 DEBUG:boto:Data: 2022-09-03T06:11:25.069 DEBUG:boto:Headers: {} 2022-09-03T06:11:25.070 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:25.071 DEBUG:boto:Port: 8000 2022-09-03T06:11:25.072 DEBUG:boto:Params: {} 2022-09-03T06:11:25.072 DEBUG:boto:Token: None 2022-09-03T06:11:25.073 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:25 GMT /dsgvcy-69/ 2022-09-03T06:11:25.073 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fRr/NcJ8YcQ8nRwKplphCagNmJw= 2022-09-03T06:11:25.074 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fRr/NcJ8YcQ8nRwKplphCagNmJw=', 'Content-Length': '0'} 2022-09-03T06:11:25.077 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '3'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ddbc6d6b0478d5dc-006312f00d-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:25.078 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:11:25.078 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:11:25.079 DEBUG:boto:Method: PUT 2022-09-03T06:11:25.079 DEBUG:boto:Path: /dsgvcy-69/b 2022-09-03T06:11:25.080 DEBUG:boto:Data: 2022-09-03T06:11:25.080 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:11:25.081 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:25.082 DEBUG:boto:Port: 8000 2022-09-03T06:11:25.082 DEBUG:boto:Params: {} 2022-09-03T06:11:25.083 DEBUG:boto:Token: None 2022-09-03T06:11:25.084 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:11:25 GMT /dsgvcy-69/b 2022-09-03T06:11:25.084 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ff/fxNW+7piEExhWSGUGAZj55Sw= 2022-09-03T06:11:25.085 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:11:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ff/fxNW+7piEExhWSGUGAZj55Sw='} 2022-09-03T06:11:25.094 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f16a12e61058f23f-006312f00d-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:25.095 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:11:25.096 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:11:25.097 DEBUG:boto:Method: HEAD 2022-09-03T06:11:25.098 DEBUG:boto:Path: /dsgvcy-69/ 2022-09-03T06:11:25.099 DEBUG:boto:Data: 2022-09-03T06:11:25.100 DEBUG:boto:Headers: {} 2022-09-03T06:11:25.101 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:25.101 DEBUG:boto:Port: 8000 2022-09-03T06:11:25.102 DEBUG:boto:Params: {} 2022-09-03T06:11:25.103 DEBUG:boto:Token: None 2022-09-03T06:11:25.104 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:25 GMT /dsgvcy-69/ 2022-09-03T06:11:25.105 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fRr/NcJ8YcQ8nRwKplphCagNmJw= 2022-09-03T06:11:25.106 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fRr/NcJ8YcQ8nRwKplphCagNmJw=', 'Content-Length': '0'} 2022-09-03T06:11:25.109 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004d649a575a10b9c6-006312f00d-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:25.110 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:11:25.110 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:11:25.111 DEBUG:boto:Method: PUT 2022-09-03T06:11:25.111 DEBUG:boto:Path: /dsgvcy-69/c 2022-09-03T06:11:25.112 DEBUG:boto:Data: 2022-09-03T06:11:25.119 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:11:25.120 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:25.120 DEBUG:boto:Port: 8000 2022-09-03T06:11:25.121 DEBUG:boto:Params: {} 2022-09-03T06:11:25.122 DEBUG:boto:Token: None 2022-09-03T06:11:25.123 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:11:25 GMT /dsgvcy-69/c 2022-09-03T06:11:25.124 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Qrm6rY/HqxIFYtQytmsn9Ckf2Ic= 2022-09-03T06:11:25.125 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:11:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Qrm6rY/HqxIFYtQytmsn9Ckf2Ic='} 2022-09-03T06:11:25.137 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001204e5bd36fa73c3-006312f00d-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:25.138 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:11:25.139 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:11:25.140 DEBUG:boto:Method: HEAD 2022-09-03T06:11:25.141 DEBUG:boto:Path: /dsgvcy-69/ 2022-09-03T06:11:25.142 DEBUG:boto:Data: 2022-09-03T06:11:25.143 DEBUG:boto:Headers: {} 2022-09-03T06:11:25.144 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:25.144 DEBUG:boto:Port: 8000 2022-09-03T06:11:25.145 DEBUG:boto:Params: {} 2022-09-03T06:11:25.146 DEBUG:boto:Token: None 2022-09-03T06:11:25.147 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:11:25 GMT /dsgvcy-69/ 2022-09-03T06:11:25.147 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fRr/NcJ8YcQ8nRwKplphCagNmJw= 2022-09-03T06:11:25.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:11:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fRr/NcJ8YcQ8nRwKplphCagNmJw=', 'Content-Length': '0'} 2022-09-03T06:11:25.151 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '9'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007ca3fe3a72813480-006312f00d-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:25.152 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:11:25.153 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:11:25.153 DEBUG:boto:Method: PUT 2022-09-03T06:11:25.154 DEBUG:boto:Path: /dsgvcy-69/d 2022-09-03T06:11:25.155 DEBUG:boto:Data: 2022-09-03T06:11:25.155 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:11:25.156 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:11:25.157 DEBUG:boto:Port: 8000 2022-09-03T06:11:25.158 DEBUG:boto:Params: {} 2022-09-03T06:11:25.159 DEBUG:boto:Token: None 2022-09-03T06:11:25.161 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:11:25 GMT /dsgvcy-69/d 2022-09-03T06:11:25.162 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YhIcV2xyEpg11RyZIcgLKqxy+dI= 2022-09-03T06:11:25.163 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:11:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YhIcV2xyEpg11RyZIcgLKqxy+dI='} 2022-09-03T06:11:25.211 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000033325f689362c584-006312f00d-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:11:25.213 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:11:25.214 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:11:25.215 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:11:25.272 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:11:25.273 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:11:25.312 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T06:11:25.326 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T06:11:25.326 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:11:25.326 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:11:25.327 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:11:25.327 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:11:25.327 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:11:25.327 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:11:25.328 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:11:25.328 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:11:25.328 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:11:25.328 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662185484.988813_417.1", 2022-09-03T06:11:25.329 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:11:24.988813Z" 2022-09-03T06:11:25.329 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:11:25.329 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:11:25.329 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:11:25.330 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:11:25.330 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:11:25.330 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T06:11:25.331 DEBUG:rgw_multi.tests:master meta markers={0: '', 1: '', 2: '1_1662185484.988813_417.1', 3: ''} 2022-09-03T06:11:25.332 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T06:11:25.333 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:11:25.333 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:11:25.334 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:11:25.394 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:11:25.394 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:11:25.425 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:25.424+0000 7f0c56edd600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:11:25.429 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:25.425+0000 7f0c56edd600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:11:25.444 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:11:25.445 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:11:25.445 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:11:25.445 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:11:25.445 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:11:25.446 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:11:25.446 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:11:25.446 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:25.447 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:11:25.447 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:11:25.447 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:11:25.447 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:11:25.448 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:11:25.448 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:11:25.448 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:11:25.448 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:11:25.449 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:11:25.449 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:11:25.449 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:11:25.449 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:25.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:25.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:11:25.450 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:11:25.450 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:11:25.451 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:11:25.451 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:11:25.451 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:11:25.451 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:11:25.452 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:11:25.452 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:11:25.452 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:11:25.453 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:25.453 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:25.453 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:11:25.453 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:11:25.454 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:11:25.454 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:11:25.455 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662185484.988813_417.1", 2022-09-03T06:11:25.455 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:11:25.456 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:11:25.456 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:11:25.457 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:11:24.988813Z", 2022-09-03T06:11:25.457 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:11:25.457 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:25.458 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:25.458 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:11:25.459 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:11:25.459 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:11:25.460 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:11:25.460 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:11:25.461 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:11:25.461 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:11:25.462 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:11:25.462 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:11:25.463 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:11:25.464 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:25.464 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:25.464 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:11:25.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:25.465 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:11:25.466 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:11:25.466 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:11:25.467 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:25.467 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:11:25Z" 2022-09-03T06:11:25.467 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:11:25.469 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185484.988813_417.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:11:24.988813Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:11:25Z" } 2022-09-03T06:11:25.470 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185484.988813_417.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:11:24.988813Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:11:25.471 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185484.988813_417.1', 3: ''} 2022-09-03T06:11:25.471 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185484.988813_417.1', 3: ''} 2022-09-03T06:11:25.472 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T06:11:25.472 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T06:11:25.473 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:11:25.473 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:11:25.475 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:11:25.526 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:11:25.526 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:11:25.558 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:11:25.556+0000 7f9c74a21600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:11:25.559 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:11:25.557+0000 7f9c74a21600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:11:25.571 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T06:11:25.572 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T06:11:25.572 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T06:11:25.572 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T06:11:25.573 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T06:11:25.573 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:11:25.573 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:11:25.574 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:11:25.574 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T06:11:25.574 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:11:25.575 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T06:11:25.575 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:11:25.575 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:11:25.575 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:11:25.576 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:11:25.576 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T06:11:25.576 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:11:25.577 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:11:25.577 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:11:25.577 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:11:25.578 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:11:25.579 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:11:25.579 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T06:11:25.579 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:11:25.580 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:11:25.580 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:11:25.581 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:11:25.582 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:11:25.582 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:11:25.583 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T06:11:25.584 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:11:25.585 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:11:25.585 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:11:25.586 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:11:25.586 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T06:11:25.587 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:11:25.587 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:11:25.588 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662185484.988813_417.1", 2022-09-03T06:11:25.588 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:11:25.588 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:11:25.589 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:11:25.589 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:11:24.988813Z", 2022-09-03T06:11:25.590 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:11:25.590 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:11:25.591 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:11:25.592 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:11:25.593 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T06:11:25.593 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:11:25.593 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:11:25.593 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:11:25.594 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:11:25.594 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:11:25.594 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:11:25.595 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:11:25.595 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:11:25.595 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:11:25.595 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:11:25.596 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:11:25.596 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:11:25.596 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T06:11:25.596 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T06:11:25.597 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T06:11:25.597 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:11:25.597 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T06:11:25Z" 2022-09-03T06:11:25.597 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T06:11:25.599 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185484.988813_417.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:11:24.988813Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:11:25Z" } 2022-09-03T06:11:25.600 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185484.988813_417.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:11:24.988813Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:11:25.601 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185484.988813_417.1', 3: ''} 2022-09-03T06:11:25.602 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185484.988813_417.1', 3: ''} 2022-09-03T06:11:25.603 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T06:11:25.604 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T06:11:25.605 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:11:25.606 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:11:25.608 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:11:25.661 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:11:25.662 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:11:25.704 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:25.703+0000 7fbbd3b9c600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:11:25.706 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:25.704+0000 7fbbd3b9c600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:11:25.725 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:11:25.726 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:11:25.726 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:11:25.726 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:11:25.726 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:11:25.727 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:11:25.727 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:11:25.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:25.727 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:11:25.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:11:25.728 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:11:25.728 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:11:25.729 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:11:25.729 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:11:25.729 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:11:25.730 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:11:25.730 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:11:25.730 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:11:25.730 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:11:25.731 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:25.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:25.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:11:25.731 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:11:25.732 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:11:25.732 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:11:25.732 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:11:25.732 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:11:25.733 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:11:25.733 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:11:25.734 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:11:25.734 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:11:25.734 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:25.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:25.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:11:25.735 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:11:25.736 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:11:25.736 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:11:25.736 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662185484.977785_414.1", 2022-09-03T06:11:25.736 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:11:25.737 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:11:25.737 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:11:25.737 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:11:24.977785Z", 2022-09-03T06:11:25.738 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:11:25.738 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:25.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:25.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:11:25.739 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:11:25.739 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:11:25.739 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:11:25.739 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:11:25.740 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:11:25.740 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:11:25.740 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:11:25.740 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:11:25.741 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:11:25.741 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:25.741 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:25.741 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:11:25.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:25.742 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:11:25.742 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:11:25.742 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:11:25.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:25.743 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:11:25Z" 2022-09-03T06:11:25.743 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:11:25.745 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185484.977785_414.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:11:24.977785Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:11:25Z" } 2022-09-03T06:11:25.746 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185484.977785_414.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:11:24.977785Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:11:25.747 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185484.988813_417.1', 3: ''} 2022-09-03T06:11:25.748 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185484.977785_414.1', 3: ''} 2022-09-03T06:11:25.748 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662185484.988813_417.1 target=1_1662185484.977785_414.1 2022-09-03T06:11:30.750 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:11:30.751 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:11:30.752 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:11:30.809 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:11:30.809 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:11:30.840 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:30.839+0000 7f5f28eec600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:11:30.841 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:30.840+0000 7f5f28eec600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:11:30.855 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:11:30.855 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:11:30.856 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:11:30.857 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:11:30.858 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:11:30.859 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:11:30.859 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:11:30.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:30.860 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:11:30.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:11:30.861 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:11:30.862 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:11:30.862 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:11:30.863 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:11:30.863 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:11:30.864 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:11:30.864 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:11:30.865 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:11:30.865 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:11:30.866 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:30.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:30.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:11:30.868 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:11:30.868 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:11:30.869 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:11:30.870 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:11:30.870 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:11:30.871 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:11:30.871 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:11:30.872 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:11:30.872 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:11:30.873 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:30.874 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:30.875 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:11:30.875 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:11:30.876 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:11:30.876 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:11:30.877 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662185484.988813_417.1", 2022-09-03T06:11:30.877 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:11:30.877 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:11:30.878 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:11:30.878 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:11:24.988813Z", 2022-09-03T06:11:30.878 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:11:30.878 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:30.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:30.879 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:11:30.879 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:11:30.879 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:11:30.880 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:11:30.880 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:11:30.880 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:11:30.880 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:11:30.881 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:11:30.881 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:11:30.882 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:11:30.882 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:30.882 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:11:30.882 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:11:30.883 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:30.883 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:11:30.883 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:11:30.883 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:11:30.884 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:11:30.884 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:11:30Z" 2022-09-03T06:11:30.884 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:11:30.886 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185484.988813_417.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:11:24.988813Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:11:30Z" } 2022-09-03T06:11:30.886 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185484.988813_417.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:11:24.988813Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:11:30.887 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185484.988813_417.1', 3: ''} 2022-09-03T06:11:30.887 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185484.988813_417.1', 3: ''} 2022-09-03T06:11:30.888 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T06:11:30.888 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-69 2022-09-03T06:11:30.889 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-69', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:11:30.889 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-69', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:11:30.890 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-69 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:11:30.949 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:11:30.949 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:11:30.981 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:30.979+0000 7fc1063fc600 1 waiting to reach incremental sync.. 2022-09-03T06:11:35.982 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:35.980+0000 7fc1063fc600 1 waiting to reach incremental sync.. 2022-09-03T06:11:40.983 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:40.981+0000 7fc1063fc600 1 waiting to reach incremental sync.. 2022-09-03T06:11:45.983 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:45.981+0000 7fc1063fc600 1 waiting to reach incremental sync.. 2022-09-03T06:11:50.983 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:50.981+0000 7fc1063fc600 1 waiting to reach incremental sync.. 2022-09-03T06:11:55.984 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:11:55.982+0000 7fc1063fc600 1 waiting to reach incremental sync.. 2022-09-03T06:12:00.986 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:00.984+0000 7fc1063fc600 1 bucket sync caught up with source: 2022-09-03T06:12:00.987 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.79.5, , , , 00000000001.51.5, , , 00000000001.81.5, , 00000000001.49.5] 2022-09-03T06:12:00.987 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.79.5, , , , 00000000001.51.5, , , 00000000001.81.5, , 00000000001.49.5] 2022-09-03T06:12:00.988 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:00.984+0000 7fc1063fc600 0 bucket checkpoint complete 2022-09-03T06:12:01.003 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-69 2022-09-03T06:12:01.004 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-69', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:01.005 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-69', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:01.005 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-69 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:12:01.072 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:01.073 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:01.113 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:01.111+0000 7fd1ed876600 1 bucket sync caught up with source: 2022-09-03T06:12:01.113 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.79.5, , , , 00000000001.51.5, , , 00000000001.81.5, , 00000000001.49.5] 2022-09-03T06:12:01.114 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.79.5, , , , 00000000001.51.5, , , 00000000001.81.5, , 00000000001.49.5] 2022-09-03T06:12:01.115 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:01.111+0000 7fd1ed876600 0 bucket checkpoint complete 2022-09-03T06:12:01.136 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-69 2022-09-03T06:12:01.137 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-69', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:01.137 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-69', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:01.138 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-69 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:12:01.201 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:01.202 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:01.240 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:01.238+0000 7fc80c41e600 1 bucket sync caught up with source: 2022-09-03T06:12:01.240 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.79.5, , , , 00000000001.51.5, , , 00000000001.81.5, , 00000000001.49.5] 2022-09-03T06:12:01.241 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.79.5, , , , 00000000001.51.5, , , 00000000001.81.5, , 00000000001.49.5] 2022-09-03T06:12:01.241 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:01.238+0000 7fc80c41e600 0 bucket checkpoint complete 2022-09-03T06:12:01.255 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-69 2022-09-03T06:12:01.255 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-69', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:01.256 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-69', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:01.256 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-69 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:12:01.313 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:01.313 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:01.362 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:01.359+0000 7fb135eef600 1 bucket sync caught up with source: 2022-09-03T06:12:01.362 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.327.5, , , , 00000000001.834.5, , , 00000000001.328.5, , 00000000001.833.5] 2022-09-03T06:12:01.363 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.327.5, , , , 00000000001.834.5, , , 00000000001.328.5, , 00000000001.833.5] 2022-09-03T06:12:01.363 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:01.359+0000 7fb135eef600 0 bucket checkpoint complete 2022-09-03T06:12:01.379 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-69 2022-09-03T06:12:01.380 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-69', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:01.380 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-69', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:01.382 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-69 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:12:01.441 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:01.441 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:01.494 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:01.492+0000 7fd3d35e8600 1 bucket sync caught up with source: 2022-09-03T06:12:01.495 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.327.5, , , , 00000000001.834.5, , , 00000000001.328.5, , 00000000001.833.5] 2022-09-03T06:12:01.495 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.327.5, , , , 00000000001.834.5, , , 00000000001.328.5, , 00000000001.833.5] 2022-09-03T06:12:01.496 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:01.492+0000 7fd3d35e8600 0 bucket checkpoint complete 2022-09-03T06:12:01.510 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-69 2022-09-03T06:12:01.511 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-69 2022-09-03T06:12:01.512 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-69', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:01.513 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-69', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:01.514 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-69 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:12:01.570 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:01.571 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:01.605 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:01.602+0000 7f81e2705600 1 bucket sync caught up with source: 2022-09-03T06:12:01.605 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.235.5, , , , 00000000001.298.5, , , 00000000001.236.5, , 00000000001.297.5] 2022-09-03T06:12:01.605 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.235.5, , , , 00000000001.298.5, , , 00000000001.236.5, , 00000000001.297.5] 2022-09-03T06:12:01.606 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:01.602+0000 7f81e2705600 0 bucket checkpoint complete 2022-09-03T06:12:01.618 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-69 2022-09-03T06:12:01.618 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-69', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:01.619 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-69', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:01.619 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-69 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:12:01.680 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:01.680 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:01.730 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:01.728+0000 7f6956a0d600 1 bucket sync caught up with source: 2022-09-03T06:12:01.731 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.235.5, , , , 00000000001.298.5, , , 00000000001.236.5, , 00000000001.297.5] 2022-09-03T06:12:01.732 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.235.5, , , , 00000000001.298.5, , , 00000000001.236.5, , 00000000001.297.5] 2022-09-03T06:12:01.732 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:01.728+0000 7f6956a0d600 0 bucket checkpoint complete 2022-09-03T06:12:01.747 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-69 2022-09-03T06:12:01.748 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-69 zones={test-zone2, test-zone1} 2022-09-03T06:12:01.749 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:01.749 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:01.750 DEBUG:boto:Method: HEAD 2022-09-03T06:12:01.750 DEBUG:boto:Path: /dsgvcy-69/ 2022-09-03T06:12:01.751 DEBUG:boto:Data: 2022-09-03T06:12:01.751 DEBUG:boto:Headers: {} 2022-09-03T06:12:01.752 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:01.752 DEBUG:boto:Port: 8000 2022-09-03T06:12:01.753 DEBUG:boto:Params: {} 2022-09-03T06:12:01.757 DEBUG:boto:Token: None 2022-09-03T06:12:01.757 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:01 GMT /dsgvcy-69/ 2022-09-03T06:12:01.758 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2IbeMw1zY4OPR1EzXWBXjsFRVk0= 2022-09-03T06:12:01.759 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2IbeMw1zY4OPR1EzXWBXjsFRVk0=', 'Content-Length': '0'} 2022-09-03T06:12:01.764 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a712d9f9f100a8d4-006312f031-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:12:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:01.765 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:01.766 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:01.766 DEBUG:boto:Method: HEAD 2022-09-03T06:12:01.767 DEBUG:boto:Path: /dsgvcy-69/ 2022-09-03T06:12:01.767 DEBUG:boto:Data: 2022-09-03T06:12:01.768 DEBUG:boto:Headers: {} 2022-09-03T06:12:01.769 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:01.770 DEBUG:boto:Port: 8000 2022-09-03T06:12:01.771 DEBUG:boto:Params: {} 2022-09-03T06:12:01.773 DEBUG:boto:Token: None 2022-09-03T06:12:01.775 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:01 GMT /dsgvcy-69/ 2022-09-03T06:12:01.776 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2IbeMw1zY4OPR1EzXWBXjsFRVk0= 2022-09-03T06:12:01.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2IbeMw1zY4OPR1EzXWBXjsFRVk0=', 'Content-Length': '0'} 2022-09-03T06:12:01.780 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000300e16a019a71ed6-006312f031-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:12:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:01.780 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:12:01.781 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:01.782 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:01.782 DEBUG:boto:path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:01.783 DEBUG:boto:auth_path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:01.784 DEBUG:boto:Method: GET 2022-09-03T06:12:01.784 DEBUG:boto:Path: /dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:01.785 DEBUG:boto:Data: 2022-09-03T06:12:01.785 DEBUG:boto:Headers: {} 2022-09-03T06:12:01.786 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:01.786 DEBUG:boto:Port: 8000 2022-09-03T06:12:01.787 DEBUG:boto:Params: {} 2022-09-03T06:12:01.788 DEBUG:boto:Token: None 2022-09-03T06:12:01.788 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:01 GMT /dsgvcy-69/?versions 2022-09-03T06:12:01.789 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:glDth3hYSHMk9fMCKY96KZh/Aoc= 2022-09-03T06:12:01.790 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:glDth3hYSHMk9fMCKY96KZh/Aoc=', 'Content-Length': '0'} 2022-09-03T06:12:01.792 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000286d0dd17d98dfd2-006312f031-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:01.794 DEBUG:boto:b'dsgvcy-69999falseanulltrue2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:01.796 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:12:01.797 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:12:01.798 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:12:01.798 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:12:01.799 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:12:01.800 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:01.801 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:01.801 DEBUG:boto:path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:01.802 DEBUG:boto:auth_path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:01.802 DEBUG:boto:Method: GET 2022-09-03T06:12:01.803 DEBUG:boto:Path: /dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:01.804 DEBUG:boto:Data: 2022-09-03T06:12:01.805 DEBUG:boto:Headers: {} 2022-09-03T06:12:01.805 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:01.806 DEBUG:boto:Port: 8000 2022-09-03T06:12:01.807 DEBUG:boto:Params: {} 2022-09-03T06:12:01.809 DEBUG:boto:Token: None 2022-09-03T06:12:01.809 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:01 GMT /dsgvcy-69/?versions 2022-09-03T06:12:01.810 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:glDth3hYSHMk9fMCKY96KZh/Aoc= 2022-09-03T06:12:01.811 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:glDth3hYSHMk9fMCKY96KZh/Aoc=', 'Content-Length': '0'} 2022-09-03T06:12:01.813 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b960a7f40be533de-006312f031-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:01.814 DEBUG:boto:b'dsgvcy-69999falseanulltrue2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:01.816 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:12:01.816 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:12:01.817 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:12:01.817 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:12:01.818 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:01.818 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:01.819 DEBUG:boto:path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:01.819 DEBUG:boto:auth_path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:01.820 DEBUG:boto:Method: GET 2022-09-03T06:12:01.820 DEBUG:boto:Path: /dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:01.821 DEBUG:boto:Data: 2022-09-03T06:12:01.822 DEBUG:boto:Headers: {} 2022-09-03T06:12:01.822 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:01.823 DEBUG:boto:Port: 8000 2022-09-03T06:12:01.823 DEBUG:boto:Params: {} 2022-09-03T06:12:01.824 DEBUG:boto:Token: None 2022-09-03T06:12:01.824 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:01 GMT /dsgvcy-69/?versions 2022-09-03T06:12:01.825 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:glDth3hYSHMk9fMCKY96KZh/Aoc= 2022-09-03T06:12:01.826 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:glDth3hYSHMk9fMCKY96KZh/Aoc=', 'Content-Length': '0'} 2022-09-03T06:12:01.828 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000024ec6657a4b504f6-006312f031-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:01.829 DEBUG:boto:b'dsgvcy-69999falseanulltrue2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:01.830 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:01.831 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:01.831 DEBUG:boto:path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:01.832 DEBUG:boto:auth_path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:01.832 DEBUG:boto:Method: GET 2022-09-03T06:12:01.833 DEBUG:boto:Path: /dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:01.833 DEBUG:boto:Data: 2022-09-03T06:12:01.834 DEBUG:boto:Headers: {} 2022-09-03T06:12:01.834 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:01.835 DEBUG:boto:Port: 8000 2022-09-03T06:12:01.835 DEBUG:boto:Params: {} 2022-09-03T06:12:01.836 DEBUG:boto:Token: None 2022-09-03T06:12:01.837 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:01 GMT /dsgvcy-69/?versions 2022-09-03T06:12:01.837 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:glDth3hYSHMk9fMCKY96KZh/Aoc= 2022-09-03T06:12:01.838 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:glDth3hYSHMk9fMCKY96KZh/Aoc=', 'Content-Length': '0'} 2022-09-03T06:12:01.840 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dcb12317702c9e27-006312f031-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:01.841 DEBUG:boto:b'dsgvcy-69999falseanulltrue2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:01.843 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:12:01.844 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:01.845 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:01.846 DEBUG:boto:path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.847 DEBUG:boto:auth_path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.848 DEBUG:boto:Method: GET 2022-09-03T06:12:01.849 DEBUG:boto:Path: /dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.849 DEBUG:boto:Data: 2022-09-03T06:12:01.850 DEBUG:boto:Headers: {} 2022-09-03T06:12:01.851 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:01.852 DEBUG:boto:Port: 8000 2022-09-03T06:12:01.853 DEBUG:boto:Params: {} 2022-09-03T06:12:01.854 DEBUG:boto:Token: None 2022-09-03T06:12:01.855 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:01 GMT /dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.856 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p/RwSz/+VurAaxOrMY6OtjG+1jY= 2022-09-03T06:12:01.857 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p/RwSz/+VurAaxOrMY6OtjG+1jY=', 'Content-Length': '0'} 2022-09-03T06:12:01.859 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f8a630a56af29186-006312f031-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:01.861 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:01.862 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:01.863 DEBUG:boto:path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.863 DEBUG:boto:auth_path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.864 DEBUG:boto:Method: GET 2022-09-03T06:12:01.865 DEBUG:boto:Path: /dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.866 DEBUG:boto:Data: 2022-09-03T06:12:01.867 DEBUG:boto:Headers: {} 2022-09-03T06:12:01.868 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:01.868 DEBUG:boto:Port: 8000 2022-09-03T06:12:01.869 DEBUG:boto:Params: {} 2022-09-03T06:12:01.870 DEBUG:boto:Token: None 2022-09-03T06:12:01.871 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:01 GMT /dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.872 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p/RwSz/+VurAaxOrMY6OtjG+1jY= 2022-09-03T06:12:01.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p/RwSz/+VurAaxOrMY6OtjG+1jY=', 'Content-Length': '0'} 2022-09-03T06:12:01.874 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000092b57fee48f06878-006312f031-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:01.875 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:01.876 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:01.877 DEBUG:boto:path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.877 DEBUG:boto:auth_path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.878 DEBUG:boto:Method: HEAD 2022-09-03T06:12:01.878 DEBUG:boto:Path: /dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.879 DEBUG:boto:Data: 2022-09-03T06:12:01.879 DEBUG:boto:Headers: {} 2022-09-03T06:12:01.880 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:01.881 DEBUG:boto:Port: 8000 2022-09-03T06:12:01.881 DEBUG:boto:Params: {} 2022-09-03T06:12:01.882 DEBUG:boto:Token: None 2022-09-03T06:12:01.882 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:01 GMT /dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.883 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rVP4TRFR97Ccb4iRFBfc0EWKfnc= 2022-09-03T06:12:01.884 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rVP4TRFR97Ccb4iRFBfc0EWKfnc=', 'Content-Length': '0'} 2022-09-03T06:12:01.885 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000010149f8ab39a4fad-006312f031-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:01.886 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:01.887 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:01.887 DEBUG:boto:path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.888 DEBUG:boto:auth_path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.888 DEBUG:boto:Method: HEAD 2022-09-03T06:12:01.889 DEBUG:boto:Path: /dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.889 DEBUG:boto:Data: 2022-09-03T06:12:01.890 DEBUG:boto:Headers: {} 2022-09-03T06:12:01.890 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:01.891 DEBUG:boto:Port: 8000 2022-09-03T06:12:01.891 DEBUG:boto:Params: {} 2022-09-03T06:12:01.892 DEBUG:boto:Token: None 2022-09-03T06:12:01.892 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:01 GMT /dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.893 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rVP4TRFR97Ccb4iRFBfc0EWKfnc= 2022-09-03T06:12:01.894 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rVP4TRFR97Ccb4iRFBfc0EWKfnc=', 'Content-Length': '0'} 2022-09-03T06:12:01.896 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001079e18c21a62904-006312f031-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:01.897 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:12:01.897 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:01.898 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:01.899 DEBUG:boto:path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.899 DEBUG:boto:auth_path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.900 DEBUG:boto:Method: GET 2022-09-03T06:12:01.901 DEBUG:boto:Path: /dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.902 DEBUG:boto:Data: 2022-09-03T06:12:01.902 DEBUG:boto:Headers: {} 2022-09-03T06:12:01.903 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:01.904 DEBUG:boto:Port: 8000 2022-09-03T06:12:01.905 DEBUG:boto:Params: {} 2022-09-03T06:12:01.906 DEBUG:boto:Token: None 2022-09-03T06:12:01.907 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:01 GMT /dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.908 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p/RwSz/+VurAaxOrMY6OtjG+1jY= 2022-09-03T06:12:01.909 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p/RwSz/+VurAaxOrMY6OtjG+1jY=', 'Content-Length': '0'} 2022-09-03T06:12:01.911 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c13eb4c272a2dcc8-006312f031-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:01.912 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:01.913 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:01.913 DEBUG:boto:path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.914 DEBUG:boto:auth_path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.915 DEBUG:boto:Method: GET 2022-09-03T06:12:01.915 DEBUG:boto:Path: /dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.916 DEBUG:boto:Data: 2022-09-03T06:12:01.916 DEBUG:boto:Headers: {} 2022-09-03T06:12:01.917 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:01.917 DEBUG:boto:Port: 8000 2022-09-03T06:12:01.918 DEBUG:boto:Params: {} 2022-09-03T06:12:01.919 DEBUG:boto:Token: None 2022-09-03T06:12:01.919 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:01 GMT /dsgvcy-69/a?versionId=null 2022-09-03T06:12:01.920 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p/RwSz/+VurAaxOrMY6OtjG+1jY= 2022-09-03T06:12:01.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p/RwSz/+VurAaxOrMY6OtjG+1jY=', 'Content-Length': '0'} 2022-09-03T06:12:01.923 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001554aca5fed33183-006312f031-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:01.924 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:01.925 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:01.925 DEBUG:boto:path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:01.926 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:01.926 DEBUG:boto:Method: GET 2022-09-03T06:12:01.927 DEBUG:boto:Path: /dsgvcy-69/?prefix=a 2022-09-03T06:12:01.928 DEBUG:boto:Data: 2022-09-03T06:12:01.928 DEBUG:boto:Headers: {} 2022-09-03T06:12:01.929 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:01.929 DEBUG:boto:Port: 8000 2022-09-03T06:12:01.930 DEBUG:boto:Params: {} 2022-09-03T06:12:01.930 DEBUG:boto:Token: None 2022-09-03T06:12:01.931 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:01 GMT /dsgvcy-69/ 2022-09-03T06:12:01.932 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HnxDi91LdAcVWGqRwKSTafB1ic4= 2022-09-03T06:12:01.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HnxDi91LdAcVWGqRwKSTafB1ic4=', 'Content-Length': '0'} 2022-09-03T06:12:01.935 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eb95916822a803d8-006312f031-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:01.935 DEBUG:boto:b'dsgvcy-69a1000falsea2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:01.936 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:01.937 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:01.937 DEBUG:boto:path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:01.938 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:01.938 DEBUG:boto:Method: GET 2022-09-03T06:12:01.939 DEBUG:boto:Path: /dsgvcy-69/?prefix=a 2022-09-03T06:12:01.940 DEBUG:boto:Data: 2022-09-03T06:12:01.940 DEBUG:boto:Headers: {} 2022-09-03T06:12:01.941 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:01.941 DEBUG:boto:Port: 8000 2022-09-03T06:12:01.942 DEBUG:boto:Params: {} 2022-09-03T06:12:01.942 DEBUG:boto:Token: None 2022-09-03T06:12:01.943 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:01 GMT /dsgvcy-69/ 2022-09-03T06:12:01.944 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HnxDi91LdAcVWGqRwKSTafB1ic4= 2022-09-03T06:12:01.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HnxDi91LdAcVWGqRwKSTafB1ic4=', 'Content-Length': '0'} 2022-09-03T06:12:01.947 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000843280da0e8b2028-006312f031-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:01.948 DEBUG:boto:b'dsgvcy-69a1000falsea2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:01.948 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:01.949 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:01.949 DEBUG:boto:Method: HEAD 2022-09-03T06:12:01.950 DEBUG:boto:Path: /dsgvcy-69/a 2022-09-03T06:12:01.951 DEBUG:boto:Data: 2022-09-03T06:12:01.951 DEBUG:boto:Headers: {} 2022-09-03T06:12:01.952 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:01.952 DEBUG:boto:Port: 8000 2022-09-03T06:12:01.953 DEBUG:boto:Params: {} 2022-09-03T06:12:01.953 DEBUG:boto:Token: None 2022-09-03T06:12:01.954 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:01 GMT /dsgvcy-69/a 2022-09-03T06:12:01.955 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qhUE5fHP3/M9UWSz4ipMZadu3D8= 2022-09-03T06:12:01.955 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qhUE5fHP3/M9UWSz4ipMZadu3D8=', 'Content-Length': '0'} 2022-09-03T06:12:01.957 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e40d4c9aba0ef037-006312f031-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:01.958 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:01.958 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:01.959 DEBUG:boto:Method: HEAD 2022-09-03T06:12:01.959 DEBUG:boto:Path: /dsgvcy-69/a 2022-09-03T06:12:01.960 DEBUG:boto:Data: 2022-09-03T06:12:01.960 DEBUG:boto:Headers: {} 2022-09-03T06:12:01.961 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:01.961 DEBUG:boto:Port: 8000 2022-09-03T06:12:01.962 DEBUG:boto:Params: {} 2022-09-03T06:12:01.962 DEBUG:boto:Token: None 2022-09-03T06:12:01.963 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:01 GMT /dsgvcy-69/a 2022-09-03T06:12:01.964 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qhUE5fHP3/M9UWSz4ipMZadu3D8= 2022-09-03T06:12:01.965 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qhUE5fHP3/M9UWSz4ipMZadu3D8=', 'Content-Length': '0'} 2022-09-03T06:12:01.967 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000922b73307c837efe-006312f031-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:01.968 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:12:01.969 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:01.970 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:01.971 DEBUG:boto:Method: GET 2022-09-03T06:12:01.972 DEBUG:boto:Path: /dsgvcy-69/a 2022-09-03T06:12:01.973 DEBUG:boto:Data: 2022-09-03T06:12:01.974 DEBUG:boto:Headers: {} 2022-09-03T06:12:01.975 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:01.976 DEBUG:boto:Port: 8000 2022-09-03T06:12:01.977 DEBUG:boto:Params: {} 2022-09-03T06:12:01.978 DEBUG:boto:Token: None 2022-09-03T06:12:01.979 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:01 GMT /dsgvcy-69/a 2022-09-03T06:12:01.980 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xOadwnXgMt2IOvNaeCXEAucbTaY= 2022-09-03T06:12:01.981 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xOadwnXgMt2IOvNaeCXEAucbTaY=', 'Content-Length': '0'} 2022-09-03T06:12:01.984 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021ffb124ab8f8838-006312f031-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:01.985 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:01.986 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:01.987 DEBUG:boto:Method: GET 2022-09-03T06:12:01.988 DEBUG:boto:Path: /dsgvcy-69/a 2022-09-03T06:12:01.988 DEBUG:boto:Data: 2022-09-03T06:12:01.989 DEBUG:boto:Headers: {} 2022-09-03T06:12:01.989 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:01.990 DEBUG:boto:Port: 8000 2022-09-03T06:12:01.990 DEBUG:boto:Params: {} 2022-09-03T06:12:01.991 DEBUG:boto:Token: None 2022-09-03T06:12:01.992 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:01 GMT /dsgvcy-69/a 2022-09-03T06:12:01.992 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xOadwnXgMt2IOvNaeCXEAucbTaY= 2022-09-03T06:12:01.993 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xOadwnXgMt2IOvNaeCXEAucbTaY=', 'Content-Length': '0'} 2022-09-03T06:12:01.995 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e9439d9a87355f90-006312f031-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:01.996 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:01.996 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:01.997 DEBUG:boto:path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:01.997 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:01.998 DEBUG:boto:Method: GET 2022-09-03T06:12:01.998 DEBUG:boto:Path: /dsgvcy-69/?prefix=a 2022-09-03T06:12:01.999 DEBUG:boto:Data: 2022-09-03T06:12:02.000 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.000 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:02.001 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.001 DEBUG:boto:Params: {} 2022-09-03T06:12:02.002 DEBUG:boto:Token: None 2022-09-03T06:12:02.002 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/ 2022-09-03T06:12:02.003 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ= 2022-09-03T06:12:02.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ=', 'Content-Length': '0'} 2022-09-03T06:12:02.006 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008071f6bb72355258-006312f032-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.007 DEBUG:boto:b'dsgvcy-69a1000falsea2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:02.008 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:02.009 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:02.009 DEBUG:boto:path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:02.010 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:02.010 DEBUG:boto:Method: GET 2022-09-03T06:12:02.011 DEBUG:boto:Path: /dsgvcy-69/?prefix=a 2022-09-03T06:12:02.012 DEBUG:boto:Data: 2022-09-03T06:12:02.012 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.013 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.013 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.014 DEBUG:boto:Params: {} 2022-09-03T06:12:02.014 DEBUG:boto:Token: None 2022-09-03T06:12:02.015 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/ 2022-09-03T06:12:02.016 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ= 2022-09-03T06:12:02.022 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ=', 'Content-Length': '0'} 2022-09-03T06:12:02.025 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009fdcc6bbaf8615dd-006312f032-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.026 DEBUG:boto:b'dsgvcy-69a1000falsea2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:02.026 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:12:02.027 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:02.028 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:02.028 DEBUG:boto:path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.029 DEBUG:boto:auth_path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.029 DEBUG:boto:Method: GET 2022-09-03T06:12:02.030 DEBUG:boto:Path: /dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.030 DEBUG:boto:Data: 2022-09-03T06:12:02.031 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.031 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:02.032 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.032 DEBUG:boto:Params: {} 2022-09-03T06:12:02.033 DEBUG:boto:Token: None 2022-09-03T06:12:02.034 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.034 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ts1D+3xlKFzgX4OmfrbpPRRLpd0= 2022-09-03T06:12:02.035 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ts1D+3xlKFzgX4OmfrbpPRRLpd0=', 'Content-Length': '0'} 2022-09-03T06:12:02.037 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000079aab1b70449c8f0-006312f032-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.078 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:02.079 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:02.079 DEBUG:boto:path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.080 DEBUG:boto:auth_path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.081 DEBUG:boto:Method: GET 2022-09-03T06:12:02.081 DEBUG:boto:Path: /dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.082 DEBUG:boto:Data: 2022-09-03T06:12:02.082 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.083 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.083 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.084 DEBUG:boto:Params: {} 2022-09-03T06:12:02.085 DEBUG:boto:Token: None 2022-09-03T06:12:02.085 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.086 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ts1D+3xlKFzgX4OmfrbpPRRLpd0= 2022-09-03T06:12:02.087 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ts1D+3xlKFzgX4OmfrbpPRRLpd0=', 'Content-Length': '0'} 2022-09-03T06:12:02.089 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003084965ddf46463c-006312f032-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.090 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:02.090 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:02.091 DEBUG:boto:path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.091 DEBUG:boto:auth_path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.092 DEBUG:boto:Method: HEAD 2022-09-03T06:12:02.092 DEBUG:boto:Path: /dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.093 DEBUG:boto:Data: 2022-09-03T06:12:02.093 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.094 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:02.094 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.095 DEBUG:boto:Params: {} 2022-09-03T06:12:02.095 DEBUG:boto:Token: None 2022-09-03T06:12:02.096 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.096 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9jKEnGfFg8Le3XiF4ZhAdWeq1m4= 2022-09-03T06:12:02.097 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9jKEnGfFg8Le3XiF4ZhAdWeq1m4=', 'Content-Length': '0'} 2022-09-03T06:12:02.099 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006e25949a11e97bbd-006312f032-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.100 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:02.100 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:02.101 DEBUG:boto:path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.101 DEBUG:boto:auth_path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.102 DEBUG:boto:Method: HEAD 2022-09-03T06:12:02.102 DEBUG:boto:Path: /dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.103 DEBUG:boto:Data: 2022-09-03T06:12:02.103 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.104 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.104 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.105 DEBUG:boto:Params: {} 2022-09-03T06:12:02.105 DEBUG:boto:Token: None 2022-09-03T06:12:02.106 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.107 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9jKEnGfFg8Le3XiF4ZhAdWeq1m4= 2022-09-03T06:12:02.107 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9jKEnGfFg8Le3XiF4ZhAdWeq1m4=', 'Content-Length': '0'} 2022-09-03T06:12:02.109 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2d008df24b078a8-006312f032-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.110 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:12:02.111 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:02.111 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:02.112 DEBUG:boto:path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.112 DEBUG:boto:auth_path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.113 DEBUG:boto:Method: GET 2022-09-03T06:12:02.113 DEBUG:boto:Path: /dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.114 DEBUG:boto:Data: 2022-09-03T06:12:02.114 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.115 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:02.116 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.116 DEBUG:boto:Params: {} 2022-09-03T06:12:02.117 DEBUG:boto:Token: None 2022-09-03T06:12:02.117 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.118 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ts1D+3xlKFzgX4OmfrbpPRRLpd0= 2022-09-03T06:12:02.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ts1D+3xlKFzgX4OmfrbpPRRLpd0=', 'Content-Length': '0'} 2022-09-03T06:12:02.121 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c778c9cc783157b5-006312f032-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.123 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:02.124 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:02.125 DEBUG:boto:path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.126 DEBUG:boto:auth_path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.127 DEBUG:boto:Method: GET 2022-09-03T06:12:02.128 DEBUG:boto:Path: /dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.128 DEBUG:boto:Data: 2022-09-03T06:12:02.129 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.129 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.130 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.130 DEBUG:boto:Params: {} 2022-09-03T06:12:02.131 DEBUG:boto:Token: None 2022-09-03T06:12:02.132 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/b?versionId=null 2022-09-03T06:12:02.132 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ts1D+3xlKFzgX4OmfrbpPRRLpd0= 2022-09-03T06:12:02.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ts1D+3xlKFzgX4OmfrbpPRRLpd0=', 'Content-Length': '0'} 2022-09-03T06:12:02.135 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000007e546639b4b8d02-006312f032-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.137 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:02.137 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:02.138 DEBUG:boto:path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:02.138 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:02.139 DEBUG:boto:Method: GET 2022-09-03T06:12:02.139 DEBUG:boto:Path: /dsgvcy-69/?prefix=b 2022-09-03T06:12:02.140 DEBUG:boto:Data: 2022-09-03T06:12:02.141 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.141 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:02.142 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.142 DEBUG:boto:Params: {} 2022-09-03T06:12:02.143 DEBUG:boto:Token: None 2022-09-03T06:12:02.144 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/ 2022-09-03T06:12:02.144 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ= 2022-09-03T06:12:02.145 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ=', 'Content-Length': '0'} 2022-09-03T06:12:02.148 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006ff1a09f810bc7ac-006312f032-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.148 DEBUG:boto:b'dsgvcy-69b1000falseb2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:02.149 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:02.150 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:02.151 DEBUG:boto:path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:02.152 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:02.152 DEBUG:boto:Method: GET 2022-09-03T06:12:02.153 DEBUG:boto:Path: /dsgvcy-69/?prefix=b 2022-09-03T06:12:02.153 DEBUG:boto:Data: 2022-09-03T06:12:02.154 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.155 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.156 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.157 DEBUG:boto:Params: {} 2022-09-03T06:12:02.158 DEBUG:boto:Token: None 2022-09-03T06:12:02.159 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/ 2022-09-03T06:12:02.160 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ= 2022-09-03T06:12:02.161 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ=', 'Content-Length': '0'} 2022-09-03T06:12:02.165 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006a0cfcc5cd1bd69c-006312f032-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.166 DEBUG:boto:b'dsgvcy-69b1000falseb2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:02.167 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:02.167 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:02.168 DEBUG:boto:Method: HEAD 2022-09-03T06:12:02.168 DEBUG:boto:Path: /dsgvcy-69/b 2022-09-03T06:12:02.169 DEBUG:boto:Data: 2022-09-03T06:12:02.170 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.171 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:02.171 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.172 DEBUG:boto:Params: {} 2022-09-03T06:12:02.172 DEBUG:boto:Token: None 2022-09-03T06:12:02.173 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/b 2022-09-03T06:12:02.174 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MfzbOE8/x7dMFbUkt8r7LVhM+H4= 2022-09-03T06:12:02.174 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MfzbOE8/x7dMFbUkt8r7LVhM+H4=', 'Content-Length': '0'} 2022-09-03T06:12:02.177 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000058cea71e6b424c01-006312f032-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.177 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:02.178 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:02.179 DEBUG:boto:Method: HEAD 2022-09-03T06:12:02.179 DEBUG:boto:Path: /dsgvcy-69/b 2022-09-03T06:12:02.180 DEBUG:boto:Data: 2022-09-03T06:12:02.180 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.181 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.181 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.182 DEBUG:boto:Params: {} 2022-09-03T06:12:02.182 DEBUG:boto:Token: None 2022-09-03T06:12:02.183 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/b 2022-09-03T06:12:02.184 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MfzbOE8/x7dMFbUkt8r7LVhM+H4= 2022-09-03T06:12:02.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MfzbOE8/x7dMFbUkt8r7LVhM+H4=', 'Content-Length': '0'} 2022-09-03T06:12:02.186 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000724ef000efa3a0c8-006312f032-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.187 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:12:02.188 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:02.188 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:02.189 DEBUG:boto:Method: GET 2022-09-03T06:12:02.189 DEBUG:boto:Path: /dsgvcy-69/b 2022-09-03T06:12:02.190 DEBUG:boto:Data: 2022-09-03T06:12:02.191 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.191 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:02.192 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.192 DEBUG:boto:Params: {} 2022-09-03T06:12:02.193 DEBUG:boto:Token: None 2022-09-03T06:12:02.193 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/b 2022-09-03T06:12:02.194 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VtPBV9eiFPSyWtiRpSSlp31qrSI= 2022-09-03T06:12:02.195 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VtPBV9eiFPSyWtiRpSSlp31qrSI=', 'Content-Length': '0'} 2022-09-03T06:12:02.197 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000064796c44ad82089a-006312f032-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.198 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:02.199 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:02.199 DEBUG:boto:Method: GET 2022-09-03T06:12:02.200 DEBUG:boto:Path: /dsgvcy-69/b 2022-09-03T06:12:02.200 DEBUG:boto:Data: 2022-09-03T06:12:02.201 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.201 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.202 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.203 DEBUG:boto:Params: {} 2022-09-03T06:12:02.204 DEBUG:boto:Token: None 2022-09-03T06:12:02.204 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/b 2022-09-03T06:12:02.205 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VtPBV9eiFPSyWtiRpSSlp31qrSI= 2022-09-03T06:12:02.206 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VtPBV9eiFPSyWtiRpSSlp31qrSI=', 'Content-Length': '0'} 2022-09-03T06:12:02.208 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000007ae6776775bd548-006312f032-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.250 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:02.251 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:02.251 DEBUG:boto:path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:02.252 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:02.252 DEBUG:boto:Method: GET 2022-09-03T06:12:02.253 DEBUG:boto:Path: /dsgvcy-69/?prefix=b 2022-09-03T06:12:02.254 DEBUG:boto:Data: 2022-09-03T06:12:02.254 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.255 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:02.255 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.256 DEBUG:boto:Params: {} 2022-09-03T06:12:02.256 DEBUG:boto:Token: None 2022-09-03T06:12:02.257 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/ 2022-09-03T06:12:02.258 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ= 2022-09-03T06:12:02.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ=', 'Content-Length': '0'} 2022-09-03T06:12:02.261 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000829b4c1ab8d7e0ab-006312f032-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.262 DEBUG:boto:b'dsgvcy-69b1000falseb2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:02.263 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:02.263 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:02.264 DEBUG:boto:path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:02.264 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:02.265 DEBUG:boto:Method: GET 2022-09-03T06:12:02.266 DEBUG:boto:Path: /dsgvcy-69/?prefix=b 2022-09-03T06:12:02.266 DEBUG:boto:Data: 2022-09-03T06:12:02.267 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.267 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.268 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.269 DEBUG:boto:Params: {} 2022-09-03T06:12:02.269 DEBUG:boto:Token: None 2022-09-03T06:12:02.270 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/ 2022-09-03T06:12:02.271 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ= 2022-09-03T06:12:02.272 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ=', 'Content-Length': '0'} 2022-09-03T06:12:02.274 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000960ea242dd4595d4-006312f032-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.275 DEBUG:boto:b'dsgvcy-69b1000falseb2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:02.276 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:12:02.277 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:02.277 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:02.278 DEBUG:boto:path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.279 DEBUG:boto:auth_path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.280 DEBUG:boto:Method: GET 2022-09-03T06:12:02.281 DEBUG:boto:Path: /dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.282 DEBUG:boto:Data: 2022-09-03T06:12:02.283 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.284 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:02.285 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.285 DEBUG:boto:Params: {} 2022-09-03T06:12:02.286 DEBUG:boto:Token: None 2022-09-03T06:12:02.287 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.287 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F0AbU/ChEx+eMgF9AxphZK8KobI= 2022-09-03T06:12:02.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F0AbU/ChEx+eMgF9AxphZK8KobI=', 'Content-Length': '0'} 2022-09-03T06:12:02.290 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000336353b8ae8e54d6-006312f032-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.291 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:02.291 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:02.292 DEBUG:boto:path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.293 DEBUG:boto:auth_path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.293 DEBUG:boto:Method: GET 2022-09-03T06:12:02.294 DEBUG:boto:Path: /dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.294 DEBUG:boto:Data: 2022-09-03T06:12:02.295 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.295 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.296 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.296 DEBUG:boto:Params: {} 2022-09-03T06:12:02.297 DEBUG:boto:Token: None 2022-09-03T06:12:02.298 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.298 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F0AbU/ChEx+eMgF9AxphZK8KobI= 2022-09-03T06:12:02.299 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F0AbU/ChEx+eMgF9AxphZK8KobI=', 'Content-Length': '0'} 2022-09-03T06:12:02.301 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c829f24d532b10f-006312f032-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.302 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:02.302 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:02.303 DEBUG:boto:path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.303 DEBUG:boto:auth_path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.304 DEBUG:boto:Method: HEAD 2022-09-03T06:12:02.304 DEBUG:boto:Path: /dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.305 DEBUG:boto:Data: 2022-09-03T06:12:02.305 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.306 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:02.306 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.307 DEBUG:boto:Params: {} 2022-09-03T06:12:02.307 DEBUG:boto:Token: None 2022-09-03T06:12:02.308 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.309 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kGReC9IZG2n6JE9hxQQs1VyXckg= 2022-09-03T06:12:02.309 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kGReC9IZG2n6JE9hxQQs1VyXckg=', 'Content-Length': '0'} 2022-09-03T06:12:02.311 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000093ded28f442aaf33-006312f032-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.312 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:02.313 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:02.313 DEBUG:boto:path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.314 DEBUG:boto:auth_path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.314 DEBUG:boto:Method: HEAD 2022-09-03T06:12:02.315 DEBUG:boto:Path: /dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.315 DEBUG:boto:Data: 2022-09-03T06:12:02.316 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.316 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.317 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.317 DEBUG:boto:Params: {} 2022-09-03T06:12:02.318 DEBUG:boto:Token: None 2022-09-03T06:12:02.319 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.319 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kGReC9IZG2n6JE9hxQQs1VyXckg= 2022-09-03T06:12:02.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kGReC9IZG2n6JE9hxQQs1VyXckg=', 'Content-Length': '0'} 2022-09-03T06:12:02.322 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fa80ffa3ec1ea47e-006312f032-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.323 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:12:02.323 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:02.324 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:02.325 DEBUG:boto:path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.325 DEBUG:boto:auth_path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.326 DEBUG:boto:Method: GET 2022-09-03T06:12:02.326 DEBUG:boto:Path: /dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.327 DEBUG:boto:Data: 2022-09-03T06:12:02.327 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.328 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:02.328 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.329 DEBUG:boto:Params: {} 2022-09-03T06:12:02.330 DEBUG:boto:Token: None 2022-09-03T06:12:02.330 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.331 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F0AbU/ChEx+eMgF9AxphZK8KobI= 2022-09-03T06:12:02.331 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F0AbU/ChEx+eMgF9AxphZK8KobI=', 'Content-Length': '0'} 2022-09-03T06:12:02.333 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000092a4c5da34e220f9-006312f032-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.334 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:02.335 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:02.335 DEBUG:boto:path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.336 DEBUG:boto:auth_path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.336 DEBUG:boto:Method: GET 2022-09-03T06:12:02.337 DEBUG:boto:Path: /dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.337 DEBUG:boto:Data: 2022-09-03T06:12:02.338 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.339 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.339 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.340 DEBUG:boto:Params: {} 2022-09-03T06:12:02.340 DEBUG:boto:Token: None 2022-09-03T06:12:02.341 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/c?versionId=null 2022-09-03T06:12:02.342 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F0AbU/ChEx+eMgF9AxphZK8KobI= 2022-09-03T06:12:02.342 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F0AbU/ChEx+eMgF9AxphZK8KobI=', 'Content-Length': '0'} 2022-09-03T06:12:02.344 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005cabbf80aca5c3f1-006312f032-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.345 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:02.346 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:02.346 DEBUG:boto:path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:02.347 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:02.347 DEBUG:boto:Method: GET 2022-09-03T06:12:02.348 DEBUG:boto:Path: /dsgvcy-69/?prefix=c 2022-09-03T06:12:02.349 DEBUG:boto:Data: 2022-09-03T06:12:02.349 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.350 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:02.350 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.351 DEBUG:boto:Params: {} 2022-09-03T06:12:02.351 DEBUG:boto:Token: None 2022-09-03T06:12:02.352 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/ 2022-09-03T06:12:02.353 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ= 2022-09-03T06:12:02.353 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ=', 'Content-Length': '0'} 2022-09-03T06:12:02.356 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000de405e6b2eefc209-006312f032-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.357 DEBUG:boto:b'dsgvcy-69c1000falsec2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:02.358 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:02.358 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:02.359 DEBUG:boto:path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:02.359 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:02.360 DEBUG:boto:Method: GET 2022-09-03T06:12:02.360 DEBUG:boto:Path: /dsgvcy-69/?prefix=c 2022-09-03T06:12:02.361 DEBUG:boto:Data: 2022-09-03T06:12:02.362 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.362 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.363 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.363 DEBUG:boto:Params: {} 2022-09-03T06:12:02.364 DEBUG:boto:Token: None 2022-09-03T06:12:02.364 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/ 2022-09-03T06:12:02.365 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ= 2022-09-03T06:12:02.366 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ=', 'Content-Length': '0'} 2022-09-03T06:12:02.368 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cc830827ab0ff42e-006312f032-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.369 DEBUG:boto:b'dsgvcy-69c1000falsec2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:02.370 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:02.370 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:02.371 DEBUG:boto:Method: HEAD 2022-09-03T06:12:02.372 DEBUG:boto:Path: /dsgvcy-69/c 2022-09-03T06:12:02.372 DEBUG:boto:Data: 2022-09-03T06:12:02.373 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.373 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:02.374 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.374 DEBUG:boto:Params: {} 2022-09-03T06:12:02.375 DEBUG:boto:Token: None 2022-09-03T06:12:02.375 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/c 2022-09-03T06:12:02.376 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mig0l85dVXh6XIqx/B8PTb0D2iY= 2022-09-03T06:12:02.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mig0l85dVXh6XIqx/B8PTb0D2iY=', 'Content-Length': '0'} 2022-09-03T06:12:02.378 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bfbb6ab7d3cbd30b-006312f032-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.379 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:02.380 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:02.380 DEBUG:boto:Method: HEAD 2022-09-03T06:12:02.381 DEBUG:boto:Path: /dsgvcy-69/c 2022-09-03T06:12:02.381 DEBUG:boto:Data: 2022-09-03T06:12:02.382 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.382 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.383 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.383 DEBUG:boto:Params: {} 2022-09-03T06:12:02.384 DEBUG:boto:Token: None 2022-09-03T06:12:02.385 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/c 2022-09-03T06:12:02.385 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mig0l85dVXh6XIqx/B8PTb0D2iY= 2022-09-03T06:12:02.386 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mig0l85dVXh6XIqx/B8PTb0D2iY=', 'Content-Length': '0'} 2022-09-03T06:12:02.388 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004453d1c92495cbc8-006312f032-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.389 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:12:02.390 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:02.390 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:02.391 DEBUG:boto:Method: GET 2022-09-03T06:12:02.391 DEBUG:boto:Path: /dsgvcy-69/c 2022-09-03T06:12:02.392 DEBUG:boto:Data: 2022-09-03T06:12:02.392 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.393 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:02.393 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.394 DEBUG:boto:Params: {} 2022-09-03T06:12:02.395 DEBUG:boto:Token: None 2022-09-03T06:12:02.395 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/c 2022-09-03T06:12:02.396 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3MhgYRoEPSgTKycbM8xcTzaFoNk= 2022-09-03T06:12:02.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3MhgYRoEPSgTKycbM8xcTzaFoNk=', 'Content-Length': '0'} 2022-09-03T06:12:02.398 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd55bfe49bfe7698-006312f032-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.443 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:02.445 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:02.446 DEBUG:boto:Method: GET 2022-09-03T06:12:02.447 DEBUG:boto:Path: /dsgvcy-69/c 2022-09-03T06:12:02.447 DEBUG:boto:Data: 2022-09-03T06:12:02.448 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.449 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.450 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.451 DEBUG:boto:Params: {} 2022-09-03T06:12:02.453 DEBUG:boto:Token: None 2022-09-03T06:12:02.454 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/c 2022-09-03T06:12:02.455 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3MhgYRoEPSgTKycbM8xcTzaFoNk= 2022-09-03T06:12:02.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3MhgYRoEPSgTKycbM8xcTzaFoNk=', 'Content-Length': '0'} 2022-09-03T06:12:02.459 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a7110a82835089c7-006312f032-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.460 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:02.462 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:02.465 DEBUG:boto:path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:02.465 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:02.466 DEBUG:boto:Method: GET 2022-09-03T06:12:02.466 DEBUG:boto:Path: /dsgvcy-69/?prefix=c 2022-09-03T06:12:02.467 DEBUG:boto:Data: 2022-09-03T06:12:02.467 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.468 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:02.468 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.469 DEBUG:boto:Params: {} 2022-09-03T06:12:02.469 DEBUG:boto:Token: None 2022-09-03T06:12:02.470 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/ 2022-09-03T06:12:02.471 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ= 2022-09-03T06:12:02.471 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ=', 'Content-Length': '0'} 2022-09-03T06:12:02.474 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000912d9b77cdbfb925-006312f032-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.474 DEBUG:boto:b'dsgvcy-69c1000falsec2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:02.475 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:02.476 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:02.476 DEBUG:boto:path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:02.477 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:02.477 DEBUG:boto:Method: GET 2022-09-03T06:12:02.478 DEBUG:boto:Path: /dsgvcy-69/?prefix=c 2022-09-03T06:12:02.478 DEBUG:boto:Data: 2022-09-03T06:12:02.479 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.480 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.480 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.481 DEBUG:boto:Params: {} 2022-09-03T06:12:02.481 DEBUG:boto:Token: None 2022-09-03T06:12:02.482 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/ 2022-09-03T06:12:02.482 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ= 2022-09-03T06:12:02.483 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ=', 'Content-Length': '0'} 2022-09-03T06:12:02.485 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dc968038c721ae3b-006312f032-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.526 DEBUG:boto:b'dsgvcy-69c1000falsec2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:02.527 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:12:02.528 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:02.528 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:02.529 DEBUG:boto:path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.529 DEBUG:boto:auth_path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.530 DEBUG:boto:Method: GET 2022-09-03T06:12:02.530 DEBUG:boto:Path: /dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.531 DEBUG:boto:Data: 2022-09-03T06:12:02.532 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.542 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:02.542 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.543 DEBUG:boto:Params: {} 2022-09-03T06:12:02.544 DEBUG:boto:Token: None 2022-09-03T06:12:02.545 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.545 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fwC8jyinfNLoavqchEYoiFLUY2M= 2022-09-03T06:12:02.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fwC8jyinfNLoavqchEYoiFLUY2M=', 'Content-Length': '0'} 2022-09-03T06:12:02.548 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a7b8e63f9947c615-006312f032-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.549 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:02.549 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:02.550 DEBUG:boto:path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.551 DEBUG:boto:auth_path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.551 DEBUG:boto:Method: GET 2022-09-03T06:12:02.552 DEBUG:boto:Path: /dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.552 DEBUG:boto:Data: 2022-09-03T06:12:02.553 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.553 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.554 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.555 DEBUG:boto:Params: {} 2022-09-03T06:12:02.555 DEBUG:boto:Token: None 2022-09-03T06:12:02.556 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.557 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fwC8jyinfNLoavqchEYoiFLUY2M= 2022-09-03T06:12:02.557 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fwC8jyinfNLoavqchEYoiFLUY2M=', 'Content-Length': '0'} 2022-09-03T06:12:02.559 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007649e323009852b1-006312f032-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.560 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:02.560 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:02.561 DEBUG:boto:path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.561 DEBUG:boto:auth_path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.562 DEBUG:boto:Method: HEAD 2022-09-03T06:12:02.563 DEBUG:boto:Path: /dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.563 DEBUG:boto:Data: 2022-09-03T06:12:02.564 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.564 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:02.565 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.565 DEBUG:boto:Params: {} 2022-09-03T06:12:02.566 DEBUG:boto:Token: None 2022-09-03T06:12:02.566 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.567 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7ovKOlQKLmdrDnpYnLe3ls0iCSM= 2022-09-03T06:12:02.567 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7ovKOlQKLmdrDnpYnLe3ls0iCSM=', 'Content-Length': '0'} 2022-09-03T06:12:02.569 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e5053f8ee30e07bd-006312f032-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.570 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:02.571 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:02.571 DEBUG:boto:path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.572 DEBUG:boto:auth_path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.572 DEBUG:boto:Method: HEAD 2022-09-03T06:12:02.573 DEBUG:boto:Path: /dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.573 DEBUG:boto:Data: 2022-09-03T06:12:02.574 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.574 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.575 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.575 DEBUG:boto:Params: {} 2022-09-03T06:12:02.576 DEBUG:boto:Token: None 2022-09-03T06:12:02.576 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.577 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7ovKOlQKLmdrDnpYnLe3ls0iCSM= 2022-09-03T06:12:02.578 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7ovKOlQKLmdrDnpYnLe3ls0iCSM=', 'Content-Length': '0'} 2022-09-03T06:12:02.579 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ef995b8ebc322a97-006312f032-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.580 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:12:02.581 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:02.581 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:02.582 DEBUG:boto:path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.583 DEBUG:boto:auth_path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.583 DEBUG:boto:Method: GET 2022-09-03T06:12:02.584 DEBUG:boto:Path: /dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.584 DEBUG:boto:Data: 2022-09-03T06:12:02.585 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.585 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:02.586 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.587 DEBUG:boto:Params: {} 2022-09-03T06:12:02.587 DEBUG:boto:Token: None 2022-09-03T06:12:02.588 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.588 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fwC8jyinfNLoavqchEYoiFLUY2M= 2022-09-03T06:12:02.589 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fwC8jyinfNLoavqchEYoiFLUY2M=', 'Content-Length': '0'} 2022-09-03T06:12:02.591 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b4e0926fb3dc6b19-006312f032-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.592 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:02.593 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:02.593 DEBUG:boto:path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.594 DEBUG:boto:auth_path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.595 DEBUG:boto:Method: GET 2022-09-03T06:12:02.595 DEBUG:boto:Path: /dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.596 DEBUG:boto:Data: 2022-09-03T06:12:02.596 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.597 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.597 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.598 DEBUG:boto:Params: {} 2022-09-03T06:12:02.599 DEBUG:boto:Token: None 2022-09-03T06:12:02.599 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/d?versionId=null 2022-09-03T06:12:02.600 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fwC8jyinfNLoavqchEYoiFLUY2M= 2022-09-03T06:12:02.601 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fwC8jyinfNLoavqchEYoiFLUY2M=', 'Content-Length': '0'} 2022-09-03T06:12:02.603 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b49186baaa26bfc-006312f032-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.604 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:02.605 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:02.605 DEBUG:boto:path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:02.606 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:02.607 DEBUG:boto:Method: GET 2022-09-03T06:12:02.607 DEBUG:boto:Path: /dsgvcy-69/?prefix=d 2022-09-03T06:12:02.608 DEBUG:boto:Data: 2022-09-03T06:12:02.608 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.609 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:02.609 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.610 DEBUG:boto:Params: {} 2022-09-03T06:12:02.611 DEBUG:boto:Token: None 2022-09-03T06:12:02.611 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/ 2022-09-03T06:12:02.612 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ= 2022-09-03T06:12:02.612 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ=', 'Content-Length': '0'} 2022-09-03T06:12:02.615 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009a2f49bac4686ac0-006312f032-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.616 DEBUG:boto:b'dsgvcy-69d1000falsed2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:02.617 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:02.618 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:02.619 DEBUG:boto:path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:02.620 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:02.621 DEBUG:boto:Method: GET 2022-09-03T06:12:02.621 DEBUG:boto:Path: /dsgvcy-69/?prefix=d 2022-09-03T06:12:02.622 DEBUG:boto:Data: 2022-09-03T06:12:02.622 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.623 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.624 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.624 DEBUG:boto:Params: {} 2022-09-03T06:12:02.625 DEBUG:boto:Token: None 2022-09-03T06:12:02.625 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/ 2022-09-03T06:12:02.626 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ= 2022-09-03T06:12:02.627 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ=', 'Content-Length': '0'} 2022-09-03T06:12:02.629 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006df4e94e9c305538-006312f032-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.630 DEBUG:boto:b'dsgvcy-69d1000falsed2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:02.632 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:02.633 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:02.634 DEBUG:boto:Method: HEAD 2022-09-03T06:12:02.635 DEBUG:boto:Path: /dsgvcy-69/d 2022-09-03T06:12:02.635 DEBUG:boto:Data: 2022-09-03T06:12:02.636 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.637 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:02.638 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.639 DEBUG:boto:Params: {} 2022-09-03T06:12:02.640 DEBUG:boto:Token: None 2022-09-03T06:12:02.641 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/d 2022-09-03T06:12:02.642 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VoRDnBc++ubYD8g7dFaVkI/5pKA= 2022-09-03T06:12:02.643 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VoRDnBc++ubYD8g7dFaVkI/5pKA=', 'Content-Length': '0'} 2022-09-03T06:12:02.648 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c464539a1bd0f6b-006312f032-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.649 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:02.650 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:02.651 DEBUG:boto:Method: HEAD 2022-09-03T06:12:02.652 DEBUG:boto:Path: /dsgvcy-69/d 2022-09-03T06:12:02.652 DEBUG:boto:Data: 2022-09-03T06:12:02.653 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.654 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.654 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.656 DEBUG:boto:Params: {} 2022-09-03T06:12:02.657 DEBUG:boto:Token: None 2022-09-03T06:12:02.658 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/d 2022-09-03T06:12:02.659 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VoRDnBc++ubYD8g7dFaVkI/5pKA= 2022-09-03T06:12:02.660 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VoRDnBc++ubYD8g7dFaVkI/5pKA=', 'Content-Length': '0'} 2022-09-03T06:12:02.662 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002b6f3445f5e59e15-006312f032-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.663 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:12:02.664 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:02.665 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:02.665 DEBUG:boto:Method: GET 2022-09-03T06:12:02.666 DEBUG:boto:Path: /dsgvcy-69/d 2022-09-03T06:12:02.667 DEBUG:boto:Data: 2022-09-03T06:12:02.667 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.668 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:02.669 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.669 DEBUG:boto:Params: {} 2022-09-03T06:12:02.670 DEBUG:boto:Token: None 2022-09-03T06:12:02.671 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/d 2022-09-03T06:12:02.671 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:T+6V/zrBLmDbg0Aqa6gy7oIf9g8= 2022-09-03T06:12:02.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:T+6V/zrBLmDbg0Aqa6gy7oIf9g8=', 'Content-Length': '0'} 2022-09-03T06:12:02.674 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000524bca4e126fbf8e-006312f032-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.718 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:02.720 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:02.721 DEBUG:boto:Method: GET 2022-09-03T06:12:02.723 DEBUG:boto:Path: /dsgvcy-69/d 2022-09-03T06:12:02.723 DEBUG:boto:Data: 2022-09-03T06:12:02.724 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.725 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.725 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.726 DEBUG:boto:Params: {} 2022-09-03T06:12:02.726 DEBUG:boto:Token: None 2022-09-03T06:12:02.727 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/d 2022-09-03T06:12:02.727 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:T+6V/zrBLmDbg0Aqa6gy7oIf9g8= 2022-09-03T06:12:02.728 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:T+6V/zrBLmDbg0Aqa6gy7oIf9g8=', 'Content-Length': '0'} 2022-09-03T06:12:02.730 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008538d6dc9863456e-006312f032-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.731 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:02.731 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:02.732 DEBUG:boto:path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:02.733 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:02.733 DEBUG:boto:Method: GET 2022-09-03T06:12:02.734 DEBUG:boto:Path: /dsgvcy-69/?prefix=d 2022-09-03T06:12:02.734 DEBUG:boto:Data: 2022-09-03T06:12:02.735 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.735 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:02.736 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.736 DEBUG:boto:Params: {} 2022-09-03T06:12:02.737 DEBUG:boto:Token: None 2022-09-03T06:12:02.738 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/ 2022-09-03T06:12:02.738 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ= 2022-09-03T06:12:02.739 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ=', 'Content-Length': '0'} 2022-09-03T06:12:02.742 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000f048a2faa069148-006312f032-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.743 DEBUG:boto:b'dsgvcy-69d1000falsed2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:02.744 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:02.744 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:02.745 DEBUG:boto:path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:02.746 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:02.746 DEBUG:boto:Method: GET 2022-09-03T06:12:02.747 DEBUG:boto:Path: /dsgvcy-69/?prefix=d 2022-09-03T06:12:02.747 DEBUG:boto:Data: 2022-09-03T06:12:02.748 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.748 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.749 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.750 DEBUG:boto:Params: {} 2022-09-03T06:12:02.750 DEBUG:boto:Token: None 2022-09-03T06:12:02.751 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/ 2022-09-03T06:12:02.751 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ= 2022-09-03T06:12:02.752 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ=', 'Content-Length': '0'} 2022-09-03T06:12:02.755 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000073b57f9852c3aa83-006312f032-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.756 DEBUG:boto:b'dsgvcy-69d1000falsed2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:02.757 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-69 zones={test-zone2, test-zone1} 2022-09-03T06:12:02.757 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-69 zones={test-zone3, test-zone1} 2022-09-03T06:12:02.758 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:02.758 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:02.759 DEBUG:boto:Method: HEAD 2022-09-03T06:12:02.759 DEBUG:boto:Path: /dsgvcy-69/ 2022-09-03T06:12:02.760 DEBUG:boto:Data: 2022-09-03T06:12:02.760 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.761 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:02.761 DEBUG:boto:Port: 8001 2022-09-03T06:12:02.762 DEBUG:boto:Params: {} 2022-09-03T06:12:02.763 DEBUG:boto:Token: None 2022-09-03T06:12:02.763 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/ 2022-09-03T06:12:02.764 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j86NFKiDjihnnvxALTZZyX/PiCE= 2022-09-03T06:12:02.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j86NFKiDjihnnvxALTZZyX/PiCE=', 'Content-Length': '0'} 2022-09-03T06:12:02.767 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000003d12e2525148b42-006312f032-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.768 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:02.768 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:02.769 DEBUG:boto:Method: HEAD 2022-09-03T06:12:02.770 DEBUG:boto:Path: /dsgvcy-69/ 2022-09-03T06:12:02.770 DEBUG:boto:Data: 2022-09-03T06:12:02.771 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.771 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.772 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.772 DEBUG:boto:Params: {} 2022-09-03T06:12:02.773 DEBUG:boto:Token: None 2022-09-03T06:12:02.774 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/ 2022-09-03T06:12:02.774 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j86NFKiDjihnnvxALTZZyX/PiCE= 2022-09-03T06:12:02.775 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j86NFKiDjihnnvxALTZZyX/PiCE=', 'Content-Length': '0'} 2022-09-03T06:12:02.777 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009503cd92854dc83d-006312f032-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.778 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:12:02.778 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:02.779 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:02.779 DEBUG:boto:path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:02.780 DEBUG:boto:auth_path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:02.781 DEBUG:boto:Method: GET 2022-09-03T06:12:02.781 DEBUG:boto:Path: /dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:02.782 DEBUG:boto:Data: 2022-09-03T06:12:02.782 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.783 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:02.783 DEBUG:boto:Port: 8001 2022-09-03T06:12:02.784 DEBUG:boto:Params: {} 2022-09-03T06:12:02.785 DEBUG:boto:Token: None 2022-09-03T06:12:02.786 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/?versions 2022-09-03T06:12:02.786 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qSMRmNfN7NpV3cp1jf/hNbJHnZI= 2022-09-03T06:12:02.787 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qSMRmNfN7NpV3cp1jf/hNbJHnZI=', 'Content-Length': '0'} 2022-09-03T06:12:02.789 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000041c27d9d88813213-006312f032-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.790 DEBUG:boto:b'dsgvcy-69999falseanulltrue2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:02.791 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:12:02.791 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:12:02.792 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:12:02.792 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:12:02.793 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:12:02.793 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:02.794 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:02.794 DEBUG:boto:path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:02.795 DEBUG:boto:auth_path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:02.795 DEBUG:boto:Method: GET 2022-09-03T06:12:02.796 DEBUG:boto:Path: /dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:02.796 DEBUG:boto:Data: 2022-09-03T06:12:02.797 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.797 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.798 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.798 DEBUG:boto:Params: {} 2022-09-03T06:12:02.799 DEBUG:boto:Token: None 2022-09-03T06:12:02.800 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/?versions 2022-09-03T06:12:02.800 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qSMRmNfN7NpV3cp1jf/hNbJHnZI= 2022-09-03T06:12:02.801 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qSMRmNfN7NpV3cp1jf/hNbJHnZI=', 'Content-Length': '0'} 2022-09-03T06:12:02.803 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c986406273c4632c-006312f032-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.804 DEBUG:boto:b'dsgvcy-69999falseanulltrue2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:02.805 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:12:02.805 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:12:02.806 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:12:02.806 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:12:02.807 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:02.807 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:02.808 DEBUG:boto:path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:02.808 DEBUG:boto:auth_path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:02.809 DEBUG:boto:Method: GET 2022-09-03T06:12:02.810 DEBUG:boto:Path: /dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:02.810 DEBUG:boto:Data: 2022-09-03T06:12:02.811 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.811 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:02.812 DEBUG:boto:Port: 8001 2022-09-03T06:12:02.812 DEBUG:boto:Params: {} 2022-09-03T06:12:02.813 DEBUG:boto:Token: None 2022-09-03T06:12:02.813 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/?versions 2022-09-03T06:12:02.814 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qSMRmNfN7NpV3cp1jf/hNbJHnZI= 2022-09-03T06:12:02.814 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qSMRmNfN7NpV3cp1jf/hNbJHnZI=', 'Content-Length': '0'} 2022-09-03T06:12:02.817 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fde184f4ca3da45f-006312f032-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.817 DEBUG:boto:b'dsgvcy-69999falseanulltrue2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:02.818 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:02.819 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:02.819 DEBUG:boto:path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:02.820 DEBUG:boto:auth_path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:02.820 DEBUG:boto:Method: GET 2022-09-03T06:12:02.821 DEBUG:boto:Path: /dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:02.822 DEBUG:boto:Data: 2022-09-03T06:12:02.822 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.823 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.823 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.824 DEBUG:boto:Params: {} 2022-09-03T06:12:02.824 DEBUG:boto:Token: None 2022-09-03T06:12:02.825 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/?versions 2022-09-03T06:12:02.825 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qSMRmNfN7NpV3cp1jf/hNbJHnZI= 2022-09-03T06:12:02.826 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qSMRmNfN7NpV3cp1jf/hNbJHnZI=', 'Content-Length': '0'} 2022-09-03T06:12:02.828 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002c2186c369a7af93-006312f032-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.829 DEBUG:boto:b'dsgvcy-69999falseanulltrue2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:02.830 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:12:02.831 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:02.831 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:02.832 DEBUG:boto:path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.833 DEBUG:boto:auth_path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.833 DEBUG:boto:Method: GET 2022-09-03T06:12:02.834 DEBUG:boto:Path: /dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.834 DEBUG:boto:Data: 2022-09-03T06:12:02.835 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.835 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:02.836 DEBUG:boto:Port: 8001 2022-09-03T06:12:02.836 DEBUG:boto:Params: {} 2022-09-03T06:12:02.837 DEBUG:boto:Token: None 2022-09-03T06:12:02.838 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.838 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VE7QML/6vFrww4TJahY47V5EyOo= 2022-09-03T06:12:02.839 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VE7QML/6vFrww4TJahY47V5EyOo=', 'Content-Length': '0'} 2022-09-03T06:12:02.841 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008ebfe355f4cf16d3-006312f032-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.842 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:02.842 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:02.843 DEBUG:boto:path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.843 DEBUG:boto:auth_path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.844 DEBUG:boto:Method: GET 2022-09-03T06:12:02.844 DEBUG:boto:Path: /dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.845 DEBUG:boto:Data: 2022-09-03T06:12:02.845 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.846 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.847 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.847 DEBUG:boto:Params: {} 2022-09-03T06:12:02.848 DEBUG:boto:Token: None 2022-09-03T06:12:02.849 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.849 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VE7QML/6vFrww4TJahY47V5EyOo= 2022-09-03T06:12:02.850 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VE7QML/6vFrww4TJahY47V5EyOo=', 'Content-Length': '0'} 2022-09-03T06:12:02.852 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006d8c93c19a40730b-006312f032-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.894 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:02.895 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:02.896 DEBUG:boto:path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.897 DEBUG:boto:auth_path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.898 DEBUG:boto:Method: HEAD 2022-09-03T06:12:02.899 DEBUG:boto:Path: /dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.900 DEBUG:boto:Data: 2022-09-03T06:12:02.901 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.901 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:02.902 DEBUG:boto:Port: 8001 2022-09-03T06:12:02.902 DEBUG:boto:Params: {} 2022-09-03T06:12:02.903 DEBUG:boto:Token: None 2022-09-03T06:12:02.904 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.904 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:isXpq+c1Juil7uuKpEYzIs1hJLQ= 2022-09-03T06:12:02.905 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:isXpq+c1Juil7uuKpEYzIs1hJLQ=', 'Content-Length': '0'} 2022-09-03T06:12:02.907 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b45a777e6d4f0c40-006312f032-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.908 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:02.908 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:02.909 DEBUG:boto:path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.910 DEBUG:boto:auth_path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.910 DEBUG:boto:Method: HEAD 2022-09-03T06:12:02.911 DEBUG:boto:Path: /dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.912 DEBUG:boto:Data: 2022-09-03T06:12:02.912 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.913 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.913 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.914 DEBUG:boto:Params: {} 2022-09-03T06:12:02.914 DEBUG:boto:Token: None 2022-09-03T06:12:02.915 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.916 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:isXpq+c1Juil7uuKpEYzIs1hJLQ= 2022-09-03T06:12:02.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:isXpq+c1Juil7uuKpEYzIs1hJLQ=', 'Content-Length': '0'} 2022-09-03T06:12:02.919 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009318ccceb5fb8343-006312f032-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.920 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:12:02.921 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:02.922 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:02.923 DEBUG:boto:path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.924 DEBUG:boto:auth_path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.924 DEBUG:boto:Method: GET 2022-09-03T06:12:02.925 DEBUG:boto:Path: /dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.926 DEBUG:boto:Data: 2022-09-03T06:12:02.927 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.928 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:02.928 DEBUG:boto:Port: 8001 2022-09-03T06:12:02.929 DEBUG:boto:Params: {} 2022-09-03T06:12:02.929 DEBUG:boto:Token: None 2022-09-03T06:12:02.930 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.931 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VE7QML/6vFrww4TJahY47V5EyOo= 2022-09-03T06:12:02.933 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VE7QML/6vFrww4TJahY47V5EyOo=', 'Content-Length': '0'} 2022-09-03T06:12:02.935 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008c2b8c8d4d852313-006312f032-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.937 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:02.937 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:02.938 DEBUG:boto:path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.939 DEBUG:boto:auth_path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.940 DEBUG:boto:Method: GET 2022-09-03T06:12:02.940 DEBUG:boto:Path: /dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.941 DEBUG:boto:Data: 2022-09-03T06:12:02.942 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.942 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.943 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.944 DEBUG:boto:Params: {} 2022-09-03T06:12:02.946 DEBUG:boto:Token: None 2022-09-03T06:12:02.947 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/a?versionId=null 2022-09-03T06:12:02.948 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VE7QML/6vFrww4TJahY47V5EyOo= 2022-09-03T06:12:02.949 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VE7QML/6vFrww4TJahY47V5EyOo=', 'Content-Length': '0'} 2022-09-03T06:12:02.952 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c75d35ead38bf244-006312f032-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.953 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:02.954 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:02.955 DEBUG:boto:path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:02.956 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:02.957 DEBUG:boto:Method: GET 2022-09-03T06:12:02.958 DEBUG:boto:Path: /dsgvcy-69/?prefix=a 2022-09-03T06:12:02.959 DEBUG:boto:Data: 2022-09-03T06:12:02.960 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.961 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:02.962 DEBUG:boto:Port: 8001 2022-09-03T06:12:02.963 DEBUG:boto:Params: {} 2022-09-03T06:12:02.964 DEBUG:boto:Token: None 2022-09-03T06:12:02.965 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/ 2022-09-03T06:12:02.966 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ= 2022-09-03T06:12:02.967 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ=', 'Content-Length': '0'} 2022-09-03T06:12:02.970 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005df5507631174cba-006312f032-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.971 DEBUG:boto:b'dsgvcy-69a1000falsea2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:02.973 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:02.974 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:02.975 DEBUG:boto:path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:02.976 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:02.977 DEBUG:boto:Method: GET 2022-09-03T06:12:02.978 DEBUG:boto:Path: /dsgvcy-69/?prefix=a 2022-09-03T06:12:02.979 DEBUG:boto:Data: 2022-09-03T06:12:02.980 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.981 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:02.981 DEBUG:boto:Port: 8000 2022-09-03T06:12:02.982 DEBUG:boto:Params: {} 2022-09-03T06:12:02.982 DEBUG:boto:Token: None 2022-09-03T06:12:02.983 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/ 2022-09-03T06:12:02.984 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ= 2022-09-03T06:12:02.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E49CzSZ+GzYxSBoKM6FVCK3RunQ=', 'Content-Length': '0'} 2022-09-03T06:12:02.987 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b00c55053314e436-006312f032-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:02.988 DEBUG:boto:b'dsgvcy-69a1000falsea2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:02.989 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:02.989 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:02.990 DEBUG:boto:Method: HEAD 2022-09-03T06:12:02.991 DEBUG:boto:Path: /dsgvcy-69/a 2022-09-03T06:12:02.991 DEBUG:boto:Data: 2022-09-03T06:12:02.992 DEBUG:boto:Headers: {} 2022-09-03T06:12:02.993 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:02.994 DEBUG:boto:Port: 8001 2022-09-03T06:12:02.996 DEBUG:boto:Params: {} 2022-09-03T06:12:02.997 DEBUG:boto:Token: None 2022-09-03T06:12:02.997 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:02 GMT /dsgvcy-69/a 2022-09-03T06:12:02.998 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PWv+C5lKmHojSQWb9B+3kNdQwpY= 2022-09-03T06:12:02.999 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PWv+C5lKmHojSQWb9B+3kNdQwpY=', 'Content-Length': '0'} 2022-09-03T06:12:03.001 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000df5277c64124a870-006312f032-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.002 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:03.002 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:03.003 DEBUG:boto:Method: HEAD 2022-09-03T06:12:03.003 DEBUG:boto:Path: /dsgvcy-69/a 2022-09-03T06:12:03.004 DEBUG:boto:Data: 2022-09-03T06:12:03.004 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.005 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.006 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.007 DEBUG:boto:Params: {} 2022-09-03T06:12:03.008 DEBUG:boto:Token: None 2022-09-03T06:12:03.009 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/a 2022-09-03T06:12:03.010 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r1UcfXbshSD+GzpEI/tXceZJA8s= 2022-09-03T06:12:03.010 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r1UcfXbshSD+GzpEI/tXceZJA8s=', 'Content-Length': '0'} 2022-09-03T06:12:03.012 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007ec7a0654cca1820-006312f033-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.013 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:12:03.013 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:03.014 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:03.014 DEBUG:boto:Method: GET 2022-09-03T06:12:03.015 DEBUG:boto:Path: /dsgvcy-69/a 2022-09-03T06:12:03.016 DEBUG:boto:Data: 2022-09-03T06:12:03.016 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.017 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.017 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.018 DEBUG:boto:Params: {} 2022-09-03T06:12:03.018 DEBUG:boto:Token: None 2022-09-03T06:12:03.019 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/a 2022-09-03T06:12:03.020 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SajwMCNETvlGjviEK/eZrUSSlxM= 2022-09-03T06:12:03.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SajwMCNETvlGjviEK/eZrUSSlxM=', 'Content-Length': '0'} 2022-09-03T06:12:03.022 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000066fa39a34fd4c3fc-006312f033-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.066 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:03.067 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:03.068 DEBUG:boto:Method: GET 2022-09-03T06:12:03.070 DEBUG:boto:Path: /dsgvcy-69/a 2022-09-03T06:12:03.071 DEBUG:boto:Data: 2022-09-03T06:12:03.071 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.072 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.073 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.074 DEBUG:boto:Params: {} 2022-09-03T06:12:03.075 DEBUG:boto:Token: None 2022-09-03T06:12:03.076 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/a 2022-09-03T06:12:03.076 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SajwMCNETvlGjviEK/eZrUSSlxM= 2022-09-03T06:12:03.077 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SajwMCNETvlGjviEK/eZrUSSlxM=', 'Content-Length': '0'} 2022-09-03T06:12:03.079 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000995577cc8f04af6f-006312f033-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.080 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:03.081 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:03.081 DEBUG:boto:path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:03.082 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:03.083 DEBUG:boto:Method: GET 2022-09-03T06:12:03.083 DEBUG:boto:Path: /dsgvcy-69/?prefix=a 2022-09-03T06:12:03.084 DEBUG:boto:Data: 2022-09-03T06:12:03.084 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.085 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.085 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.086 DEBUG:boto:Params: {} 2022-09-03T06:12:03.087 DEBUG:boto:Token: None 2022-09-03T06:12:03.087 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/ 2022-09-03T06:12:03.088 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk= 2022-09-03T06:12:03.089 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk=', 'Content-Length': '0'} 2022-09-03T06:12:03.094 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e8ee832c6215716d-006312f033-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.096 DEBUG:boto:b'dsgvcy-69a1000falsea2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:03.097 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:03.098 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:03.099 DEBUG:boto:path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:03.100 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:03.101 DEBUG:boto:Method: GET 2022-09-03T06:12:03.102 DEBUG:boto:Path: /dsgvcy-69/?prefix=a 2022-09-03T06:12:03.103 DEBUG:boto:Data: 2022-09-03T06:12:03.104 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.106 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.107 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.107 DEBUG:boto:Params: {} 2022-09-03T06:12:03.108 DEBUG:boto:Token: None 2022-09-03T06:12:03.109 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/ 2022-09-03T06:12:03.110 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk= 2022-09-03T06:12:03.111 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk=', 'Content-Length': '0'} 2022-09-03T06:12:03.113 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006830f2ae2a56e369-006312f033-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.114 DEBUG:boto:b'dsgvcy-69a1000falsea2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:03.115 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:12:03.115 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:03.116 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:03.116 DEBUG:boto:path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.117 DEBUG:boto:auth_path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.117 DEBUG:boto:Method: GET 2022-09-03T06:12:03.118 DEBUG:boto:Path: /dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.119 DEBUG:boto:Data: 2022-09-03T06:12:03.119 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.120 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.120 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.121 DEBUG:boto:Params: {} 2022-09-03T06:12:03.121 DEBUG:boto:Token: None 2022-09-03T06:12:03.122 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.123 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Urr+IwuuCy9PPGu8i0TJpTxAGFY= 2022-09-03T06:12:03.123 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Urr+IwuuCy9PPGu8i0TJpTxAGFY=', 'Content-Length': '0'} 2022-09-03T06:12:03.126 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001526f4768c5ba519-006312f033-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.126 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:03.130 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:03.131 DEBUG:boto:path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.131 DEBUG:boto:auth_path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.132 DEBUG:boto:Method: GET 2022-09-03T06:12:03.133 DEBUG:boto:Path: /dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.133 DEBUG:boto:Data: 2022-09-03T06:12:03.134 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.134 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.135 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.136 DEBUG:boto:Params: {} 2022-09-03T06:12:03.136 DEBUG:boto:Token: None 2022-09-03T06:12:03.137 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.138 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Urr+IwuuCy9PPGu8i0TJpTxAGFY= 2022-09-03T06:12:03.138 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Urr+IwuuCy9PPGu8i0TJpTxAGFY=', 'Content-Length': '0'} 2022-09-03T06:12:03.140 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000708a57f253003f47-006312f033-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.141 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:03.142 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:03.142 DEBUG:boto:path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.143 DEBUG:boto:auth_path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.143 DEBUG:boto:Method: HEAD 2022-09-03T06:12:03.144 DEBUG:boto:Path: /dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.145 DEBUG:boto:Data: 2022-09-03T06:12:03.145 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.146 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.146 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.147 DEBUG:boto:Params: {} 2022-09-03T06:12:03.147 DEBUG:boto:Token: None 2022-09-03T06:12:03.148 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.148 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:spU0jVe+8OO4WYCFgPvpCEdZZtY= 2022-09-03T06:12:03.149 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:spU0jVe+8OO4WYCFgPvpCEdZZtY=', 'Content-Length': '0'} 2022-09-03T06:12:03.151 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a5d41fe4fba61e7b-006312f033-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.152 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:03.152 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:03.153 DEBUG:boto:path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.153 DEBUG:boto:auth_path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.154 DEBUG:boto:Method: HEAD 2022-09-03T06:12:03.154 DEBUG:boto:Path: /dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.155 DEBUG:boto:Data: 2022-09-03T06:12:03.155 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.156 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.156 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.157 DEBUG:boto:Params: {} 2022-09-03T06:12:03.157 DEBUG:boto:Token: None 2022-09-03T06:12:03.158 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.159 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:spU0jVe+8OO4WYCFgPvpCEdZZtY= 2022-09-03T06:12:03.159 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:spU0jVe+8OO4WYCFgPvpCEdZZtY=', 'Content-Length': '0'} 2022-09-03T06:12:03.161 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c6b5a3b6634ad0eb-006312f033-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.162 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:12:03.163 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:03.163 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:03.164 DEBUG:boto:path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.165 DEBUG:boto:auth_path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.165 DEBUG:boto:Method: GET 2022-09-03T06:12:03.172 DEBUG:boto:Path: /dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.172 DEBUG:boto:Data: 2022-09-03T06:12:03.173 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.173 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.174 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.175 DEBUG:boto:Params: {} 2022-09-03T06:12:03.175 DEBUG:boto:Token: None 2022-09-03T06:12:03.176 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.176 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Urr+IwuuCy9PPGu8i0TJpTxAGFY= 2022-09-03T06:12:03.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Urr+IwuuCy9PPGu8i0TJpTxAGFY=', 'Content-Length': '0'} 2022-09-03T06:12:03.179 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fbdce261d5b56acf-006312f033-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.180 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:03.181 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:03.181 DEBUG:boto:path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.182 DEBUG:boto:auth_path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.182 DEBUG:boto:Method: GET 2022-09-03T06:12:03.183 DEBUG:boto:Path: /dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.184 DEBUG:boto:Data: 2022-09-03T06:12:03.184 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.185 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.185 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.186 DEBUG:boto:Params: {} 2022-09-03T06:12:03.186 DEBUG:boto:Token: None 2022-09-03T06:12:03.187 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/b?versionId=null 2022-09-03T06:12:03.188 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Urr+IwuuCy9PPGu8i0TJpTxAGFY= 2022-09-03T06:12:03.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Urr+IwuuCy9PPGu8i0TJpTxAGFY=', 'Content-Length': '0'} 2022-09-03T06:12:03.191 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006fc27eb2550fb1fa-006312f033-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.192 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:03.192 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:03.193 DEBUG:boto:path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:03.193 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:03.194 DEBUG:boto:Method: GET 2022-09-03T06:12:03.195 DEBUG:boto:Path: /dsgvcy-69/?prefix=b 2022-09-03T06:12:03.195 DEBUG:boto:Data: 2022-09-03T06:12:03.196 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.196 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.197 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.197 DEBUG:boto:Params: {} 2022-09-03T06:12:03.198 DEBUG:boto:Token: None 2022-09-03T06:12:03.198 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/ 2022-09-03T06:12:03.199 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk= 2022-09-03T06:12:03.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk=', 'Content-Length': '0'} 2022-09-03T06:12:03.202 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e036993240309692-006312f033-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.203 DEBUG:boto:b'dsgvcy-69b1000falseb2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:03.204 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:03.205 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:03.205 DEBUG:boto:path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:03.206 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:03.206 DEBUG:boto:Method: GET 2022-09-03T06:12:03.207 DEBUG:boto:Path: /dsgvcy-69/?prefix=b 2022-09-03T06:12:03.207 DEBUG:boto:Data: 2022-09-03T06:12:03.208 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.208 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.209 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.209 DEBUG:boto:Params: {} 2022-09-03T06:12:03.210 DEBUG:boto:Token: None 2022-09-03T06:12:03.211 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/ 2022-09-03T06:12:03.211 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk= 2022-09-03T06:12:03.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk=', 'Content-Length': '0'} 2022-09-03T06:12:03.214 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005cadc4033589d92c-006312f033-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.215 DEBUG:boto:b'dsgvcy-69b1000falseb2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:03.216 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:03.217 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:03.218 DEBUG:boto:Method: HEAD 2022-09-03T06:12:03.219 DEBUG:boto:Path: /dsgvcy-69/b 2022-09-03T06:12:03.220 DEBUG:boto:Data: 2022-09-03T06:12:03.221 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.221 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.222 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.223 DEBUG:boto:Params: {} 2022-09-03T06:12:03.224 DEBUG:boto:Token: None 2022-09-03T06:12:03.225 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/b 2022-09-03T06:12:03.226 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+9TQTwGPFg2/JmX7XlloCidzSao= 2022-09-03T06:12:03.227 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+9TQTwGPFg2/JmX7XlloCidzSao=', 'Content-Length': '0'} 2022-09-03T06:12:03.229 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ceb5902ca945d6e1-006312f033-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.231 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:03.231 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:03.232 DEBUG:boto:Method: HEAD 2022-09-03T06:12:03.232 DEBUG:boto:Path: /dsgvcy-69/b 2022-09-03T06:12:03.233 DEBUG:boto:Data: 2022-09-03T06:12:03.233 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.234 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.235 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.235 DEBUG:boto:Params: {} 2022-09-03T06:12:03.236 DEBUG:boto:Token: None 2022-09-03T06:12:03.236 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/b 2022-09-03T06:12:03.237 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+9TQTwGPFg2/JmX7XlloCidzSao= 2022-09-03T06:12:03.238 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+9TQTwGPFg2/JmX7XlloCidzSao=', 'Content-Length': '0'} 2022-09-03T06:12:03.241 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dfe8eba9e9ef934d-006312f033-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.242 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:12:03.243 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:03.244 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:03.246 DEBUG:boto:Method: GET 2022-09-03T06:12:03.248 DEBUG:boto:Path: /dsgvcy-69/b 2022-09-03T06:12:03.248 DEBUG:boto:Data: 2022-09-03T06:12:03.249 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.249 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.250 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.251 DEBUG:boto:Params: {} 2022-09-03T06:12:03.251 DEBUG:boto:Token: None 2022-09-03T06:12:03.252 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/b 2022-09-03T06:12:03.253 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iDEclIJz7V5WlPEefQzeU5CZCfc= 2022-09-03T06:12:03.253 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iDEclIJz7V5WlPEefQzeU5CZCfc=', 'Content-Length': '0'} 2022-09-03T06:12:03.256 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000041df4d5e4f24c155-006312f033-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.299 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:03.300 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:03.301 DEBUG:boto:Method: GET 2022-09-03T06:12:03.302 DEBUG:boto:Path: /dsgvcy-69/b 2022-09-03T06:12:03.303 DEBUG:boto:Data: 2022-09-03T06:12:03.304 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.305 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.306 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.307 DEBUG:boto:Params: {} 2022-09-03T06:12:03.309 DEBUG:boto:Token: None 2022-09-03T06:12:03.310 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/b 2022-09-03T06:12:03.311 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iDEclIJz7V5WlPEefQzeU5CZCfc= 2022-09-03T06:12:03.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iDEclIJz7V5WlPEefQzeU5CZCfc=', 'Content-Length': '0'} 2022-09-03T06:12:03.315 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007daeedb35835598c-006312f033-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.316 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:03.317 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:03.318 DEBUG:boto:path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:03.319 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:03.320 DEBUG:boto:Method: GET 2022-09-03T06:12:03.321 DEBUG:boto:Path: /dsgvcy-69/?prefix=b 2022-09-03T06:12:03.322 DEBUG:boto:Data: 2022-09-03T06:12:03.323 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.324 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.325 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.326 DEBUG:boto:Params: {} 2022-09-03T06:12:03.328 DEBUG:boto:Token: None 2022-09-03T06:12:03.329 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/ 2022-09-03T06:12:03.330 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk= 2022-09-03T06:12:03.331 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk=', 'Content-Length': '0'} 2022-09-03T06:12:03.334 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009c2545f4b5950387-006312f033-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.335 DEBUG:boto:b'dsgvcy-69b1000falseb2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:03.337 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:03.338 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:03.339 DEBUG:boto:path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:03.340 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:03.341 DEBUG:boto:Method: GET 2022-09-03T06:12:03.342 DEBUG:boto:Path: /dsgvcy-69/?prefix=b 2022-09-03T06:12:03.343 DEBUG:boto:Data: 2022-09-03T06:12:03.343 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.344 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.344 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.345 DEBUG:boto:Params: {} 2022-09-03T06:12:03.346 DEBUG:boto:Token: None 2022-09-03T06:12:03.346 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/ 2022-09-03T06:12:03.347 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk= 2022-09-03T06:12:03.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk=', 'Content-Length': '0'} 2022-09-03T06:12:03.352 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000054d9b81bf027f05-006312f033-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.398 DEBUG:boto:b'dsgvcy-69b1000falseb2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:03.399 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:12:03.400 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:03.400 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:03.401 DEBUG:boto:path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.402 DEBUG:boto:auth_path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.402 DEBUG:boto:Method: GET 2022-09-03T06:12:03.403 DEBUG:boto:Path: /dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.403 DEBUG:boto:Data: 2022-09-03T06:12:03.404 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.404 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.405 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.406 DEBUG:boto:Params: {} 2022-09-03T06:12:03.407 DEBUG:boto:Token: None 2022-09-03T06:12:03.407 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.409 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zBtLYZLEJqhWJs2YtHh4CkRP4fI= 2022-09-03T06:12:03.410 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zBtLYZLEJqhWJs2YtHh4CkRP4fI=', 'Content-Length': '0'} 2022-09-03T06:12:03.415 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000079d13d45c8d84d6e-006312f033-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.417 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:03.418 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:03.419 DEBUG:boto:path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.420 DEBUG:boto:auth_path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.421 DEBUG:boto:Method: GET 2022-09-03T06:12:03.422 DEBUG:boto:Path: /dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.423 DEBUG:boto:Data: 2022-09-03T06:12:03.424 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.426 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.427 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.427 DEBUG:boto:Params: {} 2022-09-03T06:12:03.428 DEBUG:boto:Token: None 2022-09-03T06:12:03.428 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.429 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zBtLYZLEJqhWJs2YtHh4CkRP4fI= 2022-09-03T06:12:03.430 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zBtLYZLEJqhWJs2YtHh4CkRP4fI=', 'Content-Length': '0'} 2022-09-03T06:12:03.432 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eea620d7fed7cd5d-006312f033-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.433 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:03.433 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:03.434 DEBUG:boto:path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.434 DEBUG:boto:auth_path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.435 DEBUG:boto:Method: HEAD 2022-09-03T06:12:03.436 DEBUG:boto:Path: /dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.437 DEBUG:boto:Data: 2022-09-03T06:12:03.437 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.438 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.439 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.440 DEBUG:boto:Params: {} 2022-09-03T06:12:03.441 DEBUG:boto:Token: None 2022-09-03T06:12:03.442 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.443 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JT28ItcrrRHgAcvYH5cpAY4VrlE= 2022-09-03T06:12:03.444 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JT28ItcrrRHgAcvYH5cpAY4VrlE=', 'Content-Length': '0'} 2022-09-03T06:12:03.446 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c2235e4a37821b4-006312f033-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.448 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:03.449 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:03.450 DEBUG:boto:path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.450 DEBUG:boto:auth_path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.451 DEBUG:boto:Method: HEAD 2022-09-03T06:12:03.452 DEBUG:boto:Path: /dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.453 DEBUG:boto:Data: 2022-09-03T06:12:03.454 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.455 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.455 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.456 DEBUG:boto:Params: {} 2022-09-03T06:12:03.457 DEBUG:boto:Token: None 2022-09-03T06:12:03.457 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.458 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JT28ItcrrRHgAcvYH5cpAY4VrlE= 2022-09-03T06:12:03.458 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JT28ItcrrRHgAcvYH5cpAY4VrlE=', 'Content-Length': '0'} 2022-09-03T06:12:03.460 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000062b6b3b086c4f004-006312f033-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.461 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:12:03.462 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:03.462 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:03.463 DEBUG:boto:path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.463 DEBUG:boto:auth_path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.464 DEBUG:boto:Method: GET 2022-09-03T06:12:03.464 DEBUG:boto:Path: /dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.465 DEBUG:boto:Data: 2022-09-03T06:12:03.465 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.466 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.466 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.467 DEBUG:boto:Params: {} 2022-09-03T06:12:03.468 DEBUG:boto:Token: None 2022-09-03T06:12:03.468 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.469 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zBtLYZLEJqhWJs2YtHh4CkRP4fI= 2022-09-03T06:12:03.470 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zBtLYZLEJqhWJs2YtHh4CkRP4fI=', 'Content-Length': '0'} 2022-09-03T06:12:03.472 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001039a5aeb072823b-006312f033-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.518 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:03.519 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:03.519 DEBUG:boto:path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.520 DEBUG:boto:auth_path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.521 DEBUG:boto:Method: GET 2022-09-03T06:12:03.521 DEBUG:boto:Path: /dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.522 DEBUG:boto:Data: 2022-09-03T06:12:03.522 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.523 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.523 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.524 DEBUG:boto:Params: {} 2022-09-03T06:12:03.525 DEBUG:boto:Token: None 2022-09-03T06:12:03.525 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/c?versionId=null 2022-09-03T06:12:03.526 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zBtLYZLEJqhWJs2YtHh4CkRP4fI= 2022-09-03T06:12:03.527 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zBtLYZLEJqhWJs2YtHh4CkRP4fI=', 'Content-Length': '0'} 2022-09-03T06:12:03.529 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000038d6d78e5e6939f-006312f033-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.530 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:03.531 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:03.532 DEBUG:boto:path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:03.532 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:03.533 DEBUG:boto:Method: GET 2022-09-03T06:12:03.533 DEBUG:boto:Path: /dsgvcy-69/?prefix=c 2022-09-03T06:12:03.534 DEBUG:boto:Data: 2022-09-03T06:12:03.535 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.535 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.536 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.537 DEBUG:boto:Params: {} 2022-09-03T06:12:03.537 DEBUG:boto:Token: None 2022-09-03T06:12:03.538 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/ 2022-09-03T06:12:03.539 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk= 2022-09-03T06:12:03.539 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk=', 'Content-Length': '0'} 2022-09-03T06:12:03.542 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000062a0af45bc24e7ff-006312f033-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.543 DEBUG:boto:b'dsgvcy-69c1000falsec2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:03.543 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:03.544 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:03.544 DEBUG:boto:path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:03.545 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:03.546 DEBUG:boto:Method: GET 2022-09-03T06:12:03.547 DEBUG:boto:Path: /dsgvcy-69/?prefix=c 2022-09-03T06:12:03.548 DEBUG:boto:Data: 2022-09-03T06:12:03.548 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.549 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.550 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.550 DEBUG:boto:Params: {} 2022-09-03T06:12:03.551 DEBUG:boto:Token: None 2022-09-03T06:12:03.552 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/ 2022-09-03T06:12:03.553 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk= 2022-09-03T06:12:03.553 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk=', 'Content-Length': '0'} 2022-09-03T06:12:03.556 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009e4f0377319f94f8-006312f033-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.557 DEBUG:boto:b'dsgvcy-69c1000falsec2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:03.558 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:03.559 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:03.561 DEBUG:boto:Method: HEAD 2022-09-03T06:12:03.562 DEBUG:boto:Path: /dsgvcy-69/c 2022-09-03T06:12:03.563 DEBUG:boto:Data: 2022-09-03T06:12:03.564 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.564 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.565 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.566 DEBUG:boto:Params: {} 2022-09-03T06:12:03.567 DEBUG:boto:Token: None 2022-09-03T06:12:03.568 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/c 2022-09-03T06:12:03.568 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y5UrS5H7OJ8R8tMuDeslHhU5KN8= 2022-09-03T06:12:03.569 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y5UrS5H7OJ8R8tMuDeslHhU5KN8=', 'Content-Length': '0'} 2022-09-03T06:12:03.571 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009a97118b7da7deef-006312f033-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.572 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:03.572 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:03.573 DEBUG:boto:Method: HEAD 2022-09-03T06:12:03.573 DEBUG:boto:Path: /dsgvcy-69/c 2022-09-03T06:12:03.574 DEBUG:boto:Data: 2022-09-03T06:12:03.575 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.575 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.575 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.576 DEBUG:boto:Params: {} 2022-09-03T06:12:03.581 DEBUG:boto:Token: None 2022-09-03T06:12:03.581 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/c 2022-09-03T06:12:03.582 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y5UrS5H7OJ8R8tMuDeslHhU5KN8= 2022-09-03T06:12:03.582 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y5UrS5H7OJ8R8tMuDeslHhU5KN8=', 'Content-Length': '0'} 2022-09-03T06:12:03.584 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b5ac65cd3bf05d71-006312f033-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.585 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:12:03.586 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:03.587 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:03.588 DEBUG:boto:Method: GET 2022-09-03T06:12:03.589 DEBUG:boto:Path: /dsgvcy-69/c 2022-09-03T06:12:03.590 DEBUG:boto:Data: 2022-09-03T06:12:03.591 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.591 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.592 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.593 DEBUG:boto:Params: {} 2022-09-03T06:12:03.594 DEBUG:boto:Token: None 2022-09-03T06:12:03.595 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/c 2022-09-03T06:12:03.596 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vs7K1BT1DIFtoWkBKFv5yGsKzkI= 2022-09-03T06:12:03.597 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vs7K1BT1DIFtoWkBKFv5yGsKzkI=', 'Content-Length': '0'} 2022-09-03T06:12:03.600 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a070c8f81775bea-006312f033-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.601 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:03.601 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:03.602 DEBUG:boto:Method: GET 2022-09-03T06:12:03.602 DEBUG:boto:Path: /dsgvcy-69/c 2022-09-03T06:12:03.603 DEBUG:boto:Data: 2022-09-03T06:12:03.603 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.604 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.605 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.605 DEBUG:boto:Params: {} 2022-09-03T06:12:03.606 DEBUG:boto:Token: None 2022-09-03T06:12:03.607 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/c 2022-09-03T06:12:03.607 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vs7K1BT1DIFtoWkBKFv5yGsKzkI= 2022-09-03T06:12:03.608 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vs7K1BT1DIFtoWkBKFv5yGsKzkI=', 'Content-Length': '0'} 2022-09-03T06:12:03.611 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fabfab3c900b4853-006312f033-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.612 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:03.613 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:03.614 DEBUG:boto:path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:03.615 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:03.616 DEBUG:boto:Method: GET 2022-09-03T06:12:03.617 DEBUG:boto:Path: /dsgvcy-69/?prefix=c 2022-09-03T06:12:03.618 DEBUG:boto:Data: 2022-09-03T06:12:03.618 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.619 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.619 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.620 DEBUG:boto:Params: {} 2022-09-03T06:12:03.621 DEBUG:boto:Token: None 2022-09-03T06:12:03.621 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/ 2022-09-03T06:12:03.622 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk= 2022-09-03T06:12:03.622 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk=', 'Content-Length': '0'} 2022-09-03T06:12:03.625 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e2693aa64b71a05b-006312f033-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.625 DEBUG:boto:b'dsgvcy-69c1000falsec2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:03.626 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:03.627 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:03.627 DEBUG:boto:path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:03.628 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:03.628 DEBUG:boto:Method: GET 2022-09-03T06:12:03.629 DEBUG:boto:Path: /dsgvcy-69/?prefix=c 2022-09-03T06:12:03.629 DEBUG:boto:Data: 2022-09-03T06:12:03.630 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.630 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.631 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.632 DEBUG:boto:Params: {} 2022-09-03T06:12:03.632 DEBUG:boto:Token: None 2022-09-03T06:12:03.633 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/ 2022-09-03T06:12:03.634 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk= 2022-09-03T06:12:03.634 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk=', 'Content-Length': '0'} 2022-09-03T06:12:03.638 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ad96ff9e7276be90-006312f033-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.638 DEBUG:boto:b'dsgvcy-69c1000falsec2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:03.639 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:12:03.640 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:03.640 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:03.641 DEBUG:boto:path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.641 DEBUG:boto:auth_path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.642 DEBUG:boto:Method: GET 2022-09-03T06:12:03.642 DEBUG:boto:Path: /dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.643 DEBUG:boto:Data: 2022-09-03T06:12:03.644 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.645 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.645 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.646 DEBUG:boto:Params: {} 2022-09-03T06:12:03.647 DEBUG:boto:Token: None 2022-09-03T06:12:03.647 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.648 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IEhgbjA9GmsphyRb27I3/uqljss= 2022-09-03T06:12:03.648 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IEhgbjA9GmsphyRb27I3/uqljss=', 'Content-Length': '0'} 2022-09-03T06:12:03.651 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003256866e53ce1e5f-006312f033-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.652 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:03.652 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:03.653 DEBUG:boto:path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.653 DEBUG:boto:auth_path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.654 DEBUG:boto:Method: GET 2022-09-03T06:12:03.654 DEBUG:boto:Path: /dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.656 DEBUG:boto:Data: 2022-09-03T06:12:03.657 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.658 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.659 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.659 DEBUG:boto:Params: {} 2022-09-03T06:12:03.660 DEBUG:boto:Token: None 2022-09-03T06:12:03.660 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.661 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IEhgbjA9GmsphyRb27I3/uqljss= 2022-09-03T06:12:03.662 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IEhgbjA9GmsphyRb27I3/uqljss=', 'Content-Length': '0'} 2022-09-03T06:12:03.664 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a87d6a46e1a5c78a-006312f033-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.665 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:03.666 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:03.667 DEBUG:boto:path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.667 DEBUG:boto:auth_path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.668 DEBUG:boto:Method: HEAD 2022-09-03T06:12:03.668 DEBUG:boto:Path: /dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.669 DEBUG:boto:Data: 2022-09-03T06:12:03.669 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.670 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.670 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.671 DEBUG:boto:Params: {} 2022-09-03T06:12:03.672 DEBUG:boto:Token: None 2022-09-03T06:12:03.672 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.673 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9kHNk79/7id88yKg0SvWAlc6dio= 2022-09-03T06:12:03.673 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9kHNk79/7id88yKg0SvWAlc6dio=', 'Content-Length': '0'} 2022-09-03T06:12:03.675 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009224ab520246dd26-006312f033-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.676 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:03.677 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:03.677 DEBUG:boto:path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.678 DEBUG:boto:auth_path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.678 DEBUG:boto:Method: HEAD 2022-09-03T06:12:03.679 DEBUG:boto:Path: /dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.679 DEBUG:boto:Data: 2022-09-03T06:12:03.680 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.680 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.681 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.681 DEBUG:boto:Params: {} 2022-09-03T06:12:03.682 DEBUG:boto:Token: None 2022-09-03T06:12:03.683 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.683 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9kHNk79/7id88yKg0SvWAlc6dio= 2022-09-03T06:12:03.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9kHNk79/7id88yKg0SvWAlc6dio=', 'Content-Length': '0'} 2022-09-03T06:12:03.686 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aa8a94fd364f4440-006312f033-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.686 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:12:03.687 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:03.688 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:03.688 DEBUG:boto:path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.689 DEBUG:boto:auth_path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.690 DEBUG:boto:Method: GET 2022-09-03T06:12:03.691 DEBUG:boto:Path: /dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.691 DEBUG:boto:Data: 2022-09-03T06:12:03.692 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.692 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.693 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.693 DEBUG:boto:Params: {} 2022-09-03T06:12:03.694 DEBUG:boto:Token: None 2022-09-03T06:12:03.695 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.695 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IEhgbjA9GmsphyRb27I3/uqljss= 2022-09-03T06:12:03.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IEhgbjA9GmsphyRb27I3/uqljss=', 'Content-Length': '0'} 2022-09-03T06:12:03.698 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021d00f693e07d554-006312f033-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.742 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:03.743 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:03.744 DEBUG:boto:path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.745 DEBUG:boto:auth_path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.745 DEBUG:boto:Method: GET 2022-09-03T06:12:03.746 DEBUG:boto:Path: /dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.747 DEBUG:boto:Data: 2022-09-03T06:12:03.747 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.748 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.748 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.749 DEBUG:boto:Params: {} 2022-09-03T06:12:03.750 DEBUG:boto:Token: None 2022-09-03T06:12:03.750 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/d?versionId=null 2022-09-03T06:12:03.751 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IEhgbjA9GmsphyRb27I3/uqljss= 2022-09-03T06:12:03.752 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IEhgbjA9GmsphyRb27I3/uqljss=', 'Content-Length': '0'} 2022-09-03T06:12:03.754 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000041983f7bde811e5-006312f033-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.755 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:03.755 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:03.756 DEBUG:boto:path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:03.756 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:03.757 DEBUG:boto:Method: GET 2022-09-03T06:12:03.758 DEBUG:boto:Path: /dsgvcy-69/?prefix=d 2022-09-03T06:12:03.758 DEBUG:boto:Data: 2022-09-03T06:12:03.759 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.760 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.760 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.761 DEBUG:boto:Params: {} 2022-09-03T06:12:03.762 DEBUG:boto:Token: None 2022-09-03T06:12:03.762 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/ 2022-09-03T06:12:03.763 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk= 2022-09-03T06:12:03.763 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk=', 'Content-Length': '0'} 2022-09-03T06:12:03.766 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008ffd5ed3c6c64485-006312f033-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.767 DEBUG:boto:b'dsgvcy-69d1000falsed2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:03.768 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:03.769 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:03.770 DEBUG:boto:path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:03.770 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:03.771 DEBUG:boto:Method: GET 2022-09-03T06:12:03.771 DEBUG:boto:Path: /dsgvcy-69/?prefix=d 2022-09-03T06:12:03.772 DEBUG:boto:Data: 2022-09-03T06:12:03.772 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.773 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.773 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.774 DEBUG:boto:Params: {} 2022-09-03T06:12:03.775 DEBUG:boto:Token: None 2022-09-03T06:12:03.775 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/ 2022-09-03T06:12:03.776 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk= 2022-09-03T06:12:03.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk=', 'Content-Length': '0'} 2022-09-03T06:12:03.779 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ffebd060a5cfb7af-006312f033-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.780 DEBUG:boto:b'dsgvcy-69d1000falsed2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:03.781 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:03.782 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:03.783 DEBUG:boto:Method: HEAD 2022-09-03T06:12:03.784 DEBUG:boto:Path: /dsgvcy-69/d 2022-09-03T06:12:03.784 DEBUG:boto:Data: 2022-09-03T06:12:03.785 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.786 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.787 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.788 DEBUG:boto:Params: {} 2022-09-03T06:12:03.788 DEBUG:boto:Token: None 2022-09-03T06:12:03.789 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/d 2022-09-03T06:12:03.790 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e8YvXt7yQN8mMKFw6QAWAZfSuEE= 2022-09-03T06:12:03.791 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e8YvXt7yQN8mMKFw6QAWAZfSuEE=', 'Content-Length': '0'} 2022-09-03T06:12:03.794 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dd5d5c4043351fb2-006312f033-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.795 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:03.796 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:03.796 DEBUG:boto:Method: HEAD 2022-09-03T06:12:03.797 DEBUG:boto:Path: /dsgvcy-69/d 2022-09-03T06:12:03.797 DEBUG:boto:Data: 2022-09-03T06:12:03.798 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.798 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.799 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.799 DEBUG:boto:Params: {} 2022-09-03T06:12:03.800 DEBUG:boto:Token: None 2022-09-03T06:12:03.800 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/d 2022-09-03T06:12:03.801 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e8YvXt7yQN8mMKFw6QAWAZfSuEE= 2022-09-03T06:12:03.801 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e8YvXt7yQN8mMKFw6QAWAZfSuEE=', 'Content-Length': '0'} 2022-09-03T06:12:03.803 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cdd4187887d81ff2-006312f033-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.804 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:12:03.805 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:03.805 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:03.806 DEBUG:boto:Method: GET 2022-09-03T06:12:03.806 DEBUG:boto:Path: /dsgvcy-69/d 2022-09-03T06:12:03.807 DEBUG:boto:Data: 2022-09-03T06:12:03.807 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.808 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.808 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.809 DEBUG:boto:Params: {} 2022-09-03T06:12:03.810 DEBUG:boto:Token: None 2022-09-03T06:12:03.810 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/d 2022-09-03T06:12:03.811 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:d1J+ukcA/x8D4KnvekL+RMI/UpI= 2022-09-03T06:12:03.811 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:d1J+ukcA/x8D4KnvekL+RMI/UpI=', 'Content-Length': '0'} 2022-09-03T06:12:03.813 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fe7f5db0bdc67f59-006312f033-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.814 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:03.815 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:03.815 DEBUG:boto:Method: GET 2022-09-03T06:12:03.816 DEBUG:boto:Path: /dsgvcy-69/d 2022-09-03T06:12:03.816 DEBUG:boto:Data: 2022-09-03T06:12:03.817 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.817 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.818 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.818 DEBUG:boto:Params: {} 2022-09-03T06:12:03.819 DEBUG:boto:Token: None 2022-09-03T06:12:03.820 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/d 2022-09-03T06:12:03.820 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:d1J+ukcA/x8D4KnvekL+RMI/UpI= 2022-09-03T06:12:03.821 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:d1J+ukcA/x8D4KnvekL+RMI/UpI=', 'Content-Length': '0'} 2022-09-03T06:12:03.823 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009db54971dafefe43-006312f033-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.870 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:03.871 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:03.872 DEBUG:boto:path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:03.872 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:03.873 DEBUG:boto:Method: GET 2022-09-03T06:12:03.874 DEBUG:boto:Path: /dsgvcy-69/?prefix=d 2022-09-03T06:12:03.875 DEBUG:boto:Data: 2022-09-03T06:12:03.876 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.877 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.878 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.879 DEBUG:boto:Params: {} 2022-09-03T06:12:03.880 DEBUG:boto:Token: None 2022-09-03T06:12:03.881 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/ 2022-09-03T06:12:03.882 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk= 2022-09-03T06:12:03.883 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk=', 'Content-Length': '0'} 2022-09-03T06:12:03.885 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000510d5c614423f26b-006312f033-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.886 DEBUG:boto:b'dsgvcy-69d1000falsed2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:03.887 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:03.887 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:03.888 DEBUG:boto:path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:03.888 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:03.889 DEBUG:boto:Method: GET 2022-09-03T06:12:03.889 DEBUG:boto:Path: /dsgvcy-69/?prefix=d 2022-09-03T06:12:03.890 DEBUG:boto:Data: 2022-09-03T06:12:03.891 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.891 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:03.892 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.892 DEBUG:boto:Params: {} 2022-09-03T06:12:03.893 DEBUG:boto:Token: None 2022-09-03T06:12:03.893 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/ 2022-09-03T06:12:03.894 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk= 2022-09-03T06:12:03.895 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l9kGK+KbkThpD3KSssVAwbU3fpk=', 'Content-Length': '0'} 2022-09-03T06:12:03.897 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008188bdcb6aa80e16-006312f033-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.898 DEBUG:boto:b'dsgvcy-69d1000falsed2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:03.899 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-69 zones={test-zone3, test-zone1} 2022-09-03T06:12:03.899 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-69 zones={test-zone3, test-zone2} 2022-09-03T06:12:03.900 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:03.900 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:03.901 DEBUG:boto:Method: HEAD 2022-09-03T06:12:03.901 DEBUG:boto:Path: /dsgvcy-69/ 2022-09-03T06:12:03.902 DEBUG:boto:Data: 2022-09-03T06:12:03.902 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.903 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.903 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.904 DEBUG:boto:Params: {} 2022-09-03T06:12:03.905 DEBUG:boto:Token: None 2022-09-03T06:12:03.905 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/ 2022-09-03T06:12:03.906 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V+FHB6DZUgQiwjEAUD5+h+iJrNg= 2022-09-03T06:12:03.906 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V+FHB6DZUgQiwjEAUD5+h+iJrNg=', 'Content-Length': '0'} 2022-09-03T06:12:03.909 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d77a75f7f3c76f33-006312f033-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.910 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:03.910 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:03.911 DEBUG:boto:Method: HEAD 2022-09-03T06:12:03.911 DEBUG:boto:Path: /dsgvcy-69/ 2022-09-03T06:12:03.912 DEBUG:boto:Data: 2022-09-03T06:12:03.912 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.913 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:03.913 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.914 DEBUG:boto:Params: {} 2022-09-03T06:12:03.914 DEBUG:boto:Token: None 2022-09-03T06:12:03.915 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/ 2022-09-03T06:12:03.916 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V+FHB6DZUgQiwjEAUD5+h+iJrNg= 2022-09-03T06:12:03.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V+FHB6DZUgQiwjEAUD5+h+iJrNg=', 'Content-Length': '0'} 2022-09-03T06:12:03.919 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009834a15dedcc26d6-006312f033-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.920 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:12:03.920 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:03.921 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:03.921 DEBUG:boto:path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:03.922 DEBUG:boto:auth_path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:03.923 DEBUG:boto:Method: GET 2022-09-03T06:12:03.923 DEBUG:boto:Path: /dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:03.924 DEBUG:boto:Data: 2022-09-03T06:12:03.924 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.925 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.926 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.926 DEBUG:boto:Params: {} 2022-09-03T06:12:03.927 DEBUG:boto:Token: None 2022-09-03T06:12:03.927 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/?versions 2022-09-03T06:12:03.928 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QOtLFYn2hF2Ypdq+NGb0pfB4hlo= 2022-09-03T06:12:03.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QOtLFYn2hF2Ypdq+NGb0pfB4hlo=', 'Content-Length': '0'} 2022-09-03T06:12:03.931 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000391c365ff1fb95a0-006312f033-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.932 DEBUG:boto:b'dsgvcy-69999falseanulltrue2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:03.933 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:12:03.933 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:12:03.934 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:12:03.934 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:12:03.935 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:12:03.935 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:03.936 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:03.937 DEBUG:boto:path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:03.937 DEBUG:boto:auth_path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:03.938 DEBUG:boto:Method: GET 2022-09-03T06:12:03.939 DEBUG:boto:Path: /dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:03.939 DEBUG:boto:Data: 2022-09-03T06:12:03.940 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.940 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:03.941 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.942 DEBUG:boto:Params: {} 2022-09-03T06:12:03.942 DEBUG:boto:Token: None 2022-09-03T06:12:03.943 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/?versions 2022-09-03T06:12:03.944 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QOtLFYn2hF2Ypdq+NGb0pfB4hlo= 2022-09-03T06:12:03.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QOtLFYn2hF2Ypdq+NGb0pfB4hlo=', 'Content-Length': '0'} 2022-09-03T06:12:03.947 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ea41159004dcae0b-006312f033-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.949 DEBUG:boto:b'dsgvcy-69999falseanulltrue2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:03.950 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:12:03.951 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:12:03.952 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:12:03.953 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:12:03.954 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:03.955 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:03.956 DEBUG:boto:path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:03.957 DEBUG:boto:auth_path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:03.957 DEBUG:boto:Method: GET 2022-09-03T06:12:03.958 DEBUG:boto:Path: /dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:03.959 DEBUG:boto:Data: 2022-09-03T06:12:03.960 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.960 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.961 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.962 DEBUG:boto:Params: {} 2022-09-03T06:12:03.962 DEBUG:boto:Token: None 2022-09-03T06:12:03.963 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/?versions 2022-09-03T06:12:03.963 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QOtLFYn2hF2Ypdq+NGb0pfB4hlo= 2022-09-03T06:12:03.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QOtLFYn2hF2Ypdq+NGb0pfB4hlo=', 'Content-Length': '0'} 2022-09-03T06:12:03.966 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002b8f71ef7ec52db5-006312f033-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.967 DEBUG:boto:b'dsgvcy-69999falseanulltrue2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:03.968 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:03.969 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:03.969 DEBUG:boto:path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:03.970 DEBUG:boto:auth_path=/dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:03.970 DEBUG:boto:Method: GET 2022-09-03T06:12:03.971 DEBUG:boto:Path: /dsgvcy-69/?versions&max-keys=999 2022-09-03T06:12:03.971 DEBUG:boto:Data: 2022-09-03T06:12:03.972 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.972 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:03.973 DEBUG:boto:Port: 8000 2022-09-03T06:12:03.973 DEBUG:boto:Params: {} 2022-09-03T06:12:03.974 DEBUG:boto:Token: None 2022-09-03T06:12:03.974 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/?versions 2022-09-03T06:12:03.975 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QOtLFYn2hF2Ypdq+NGb0pfB4hlo= 2022-09-03T06:12:03.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QOtLFYn2hF2Ypdq+NGb0pfB4hlo=', 'Content-Length': '0'} 2022-09-03T06:12:03.978 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000430e84214fd05fea-006312f033-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:03.979 DEBUG:boto:b'dsgvcy-69999falseanulltrue2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:03.980 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:12:03.980 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:03.981 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:03.982 DEBUG:boto:path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:03.982 DEBUG:boto:auth_path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:03.983 DEBUG:boto:Method: GET 2022-09-03T06:12:03.983 DEBUG:boto:Path: /dsgvcy-69/a?versionId=null 2022-09-03T06:12:03.984 DEBUG:boto:Data: 2022-09-03T06:12:03.985 DEBUG:boto:Headers: {} 2022-09-03T06:12:03.985 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:03.986 DEBUG:boto:Port: 8001 2022-09-03T06:12:03.986 DEBUG:boto:Params: {} 2022-09-03T06:12:03.987 DEBUG:boto:Token: None 2022-09-03T06:12:03.987 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:03 GMT /dsgvcy-69/a?versionId=null 2022-09-03T06:12:03.988 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:osMq4xHXGiuYGu8KUW1I8UDLXTI= 2022-09-03T06:12:03.989 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:osMq4xHXGiuYGu8KUW1I8UDLXTI=', 'Content-Length': '0'} 2022-09-03T06:12:03.991 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000005ee7f254200f76d-006312f033-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.038 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:04.039 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:04.039 DEBUG:boto:path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:04.040 DEBUG:boto:auth_path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:04.041 DEBUG:boto:Method: GET 2022-09-03T06:12:04.041 DEBUG:boto:Path: /dsgvcy-69/a?versionId=null 2022-09-03T06:12:04.042 DEBUG:boto:Data: 2022-09-03T06:12:04.042 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.043 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.044 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.045 DEBUG:boto:Params: {} 2022-09-03T06:12:04.046 DEBUG:boto:Token: None 2022-09-03T06:12:04.047 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/a?versionId=null 2022-09-03T06:12:04.048 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pGo3KYKGW/t/jvLSOtV6WLPBbO4= 2022-09-03T06:12:04.049 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pGo3KYKGW/t/jvLSOtV6WLPBbO4=', 'Content-Length': '0'} 2022-09-03T06:12:04.052 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000182f8dfbd367594d-006312f034-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.053 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:04.054 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:04.054 DEBUG:boto:path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:04.055 DEBUG:boto:auth_path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:04.056 DEBUG:boto:Method: HEAD 2022-09-03T06:12:04.056 DEBUG:boto:Path: /dsgvcy-69/a?versionId=null 2022-09-03T06:12:04.057 DEBUG:boto:Data: 2022-09-03T06:12:04.058 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.059 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.060 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.061 DEBUG:boto:Params: {} 2022-09-03T06:12:04.062 DEBUG:boto:Token: None 2022-09-03T06:12:04.063 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/a?versionId=null 2022-09-03T06:12:04.064 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PMH4oBtBk86im8fbvqkhdH8LOGs= 2022-09-03T06:12:04.065 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PMH4oBtBk86im8fbvqkhdH8LOGs=', 'Content-Length': '0'} 2022-09-03T06:12:04.067 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ed143d66e3493a17-006312f034-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.068 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:04.068 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:04.069 DEBUG:boto:path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:04.069 DEBUG:boto:auth_path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:04.070 DEBUG:boto:Method: HEAD 2022-09-03T06:12:04.070 DEBUG:boto:Path: /dsgvcy-69/a?versionId=null 2022-09-03T06:12:04.071 DEBUG:boto:Data: 2022-09-03T06:12:04.071 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.072 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.072 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.073 DEBUG:boto:Params: {} 2022-09-03T06:12:04.073 DEBUG:boto:Token: None 2022-09-03T06:12:04.074 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/a?versionId=null 2022-09-03T06:12:04.074 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PMH4oBtBk86im8fbvqkhdH8LOGs= 2022-09-03T06:12:04.075 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PMH4oBtBk86im8fbvqkhdH8LOGs=', 'Content-Length': '0'} 2022-09-03T06:12:04.077 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c1a02c9b6f238ce6-006312f034-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.078 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:12:04.079 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:04.079 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:04.080 DEBUG:boto:path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:04.080 DEBUG:boto:auth_path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:04.081 DEBUG:boto:Method: GET 2022-09-03T06:12:04.081 DEBUG:boto:Path: /dsgvcy-69/a?versionId=null 2022-09-03T06:12:04.082 DEBUG:boto:Data: 2022-09-03T06:12:04.082 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.083 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.083 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.084 DEBUG:boto:Params: {} 2022-09-03T06:12:04.085 DEBUG:boto:Token: None 2022-09-03T06:12:04.085 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/a?versionId=null 2022-09-03T06:12:04.086 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pGo3KYKGW/t/jvLSOtV6WLPBbO4= 2022-09-03T06:12:04.086 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pGo3KYKGW/t/jvLSOtV6WLPBbO4=', 'Content-Length': '0'} 2022-09-03T06:12:04.088 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009b71251f54b64497-006312f034-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.089 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:04.090 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:04.090 DEBUG:boto:path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:04.091 DEBUG:boto:auth_path=/dsgvcy-69/a?versionId=null 2022-09-03T06:12:04.091 DEBUG:boto:Method: GET 2022-09-03T06:12:04.092 DEBUG:boto:Path: /dsgvcy-69/a?versionId=null 2022-09-03T06:12:04.092 DEBUG:boto:Data: 2022-09-03T06:12:04.093 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.093 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.094 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.095 DEBUG:boto:Params: {} 2022-09-03T06:12:04.095 DEBUG:boto:Token: None 2022-09-03T06:12:04.096 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/a?versionId=null 2022-09-03T06:12:04.096 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pGo3KYKGW/t/jvLSOtV6WLPBbO4= 2022-09-03T06:12:04.097 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pGo3KYKGW/t/jvLSOtV6WLPBbO4=', 'Content-Length': '0'} 2022-09-03T06:12:04.100 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b34cd59fa58923b5-006312f034-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.101 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:04.102 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:04.103 DEBUG:boto:path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:04.104 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:04.105 DEBUG:boto:Method: GET 2022-09-03T06:12:04.106 DEBUG:boto:Path: /dsgvcy-69/?prefix=a 2022-09-03T06:12:04.106 DEBUG:boto:Data: 2022-09-03T06:12:04.107 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.108 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.108 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.109 DEBUG:boto:Params: {} 2022-09-03T06:12:04.110 DEBUG:boto:Token: None 2022-09-03T06:12:04.111 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/ 2022-09-03T06:12:04.111 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8= 2022-09-03T06:12:04.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8=', 'Content-Length': '0'} 2022-09-03T06:12:04.115 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c174d55c34f71784-006312f034-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.115 DEBUG:boto:b'dsgvcy-69a1000falsea2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:04.116 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:04.117 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:04.117 DEBUG:boto:path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:04.118 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:04.118 DEBUG:boto:Method: GET 2022-09-03T06:12:04.119 DEBUG:boto:Path: /dsgvcy-69/?prefix=a 2022-09-03T06:12:04.120 DEBUG:boto:Data: 2022-09-03T06:12:04.120 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.121 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.121 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.122 DEBUG:boto:Params: {} 2022-09-03T06:12:04.123 DEBUG:boto:Token: None 2022-09-03T06:12:04.123 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/ 2022-09-03T06:12:04.125 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8= 2022-09-03T06:12:04.126 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8=', 'Content-Length': '0'} 2022-09-03T06:12:04.128 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eafdbb81c44f3b1b-006312f034-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.130 DEBUG:boto:b'dsgvcy-69a1000falsea2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:04.130 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:04.131 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:04.132 DEBUG:boto:Method: HEAD 2022-09-03T06:12:04.132 DEBUG:boto:Path: /dsgvcy-69/a 2022-09-03T06:12:04.133 DEBUG:boto:Data: 2022-09-03T06:12:04.133 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.134 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.135 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.135 DEBUG:boto:Params: {} 2022-09-03T06:12:04.136 DEBUG:boto:Token: None 2022-09-03T06:12:04.137 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/a 2022-09-03T06:12:04.138 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:maXXI1Btzz6NsTDTw0XKJzjIExE= 2022-09-03T06:12:04.139 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:maXXI1Btzz6NsTDTw0XKJzjIExE=', 'Content-Length': '0'} 2022-09-03T06:12:04.140 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008670f635aaa6e168-006312f034-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.141 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:04.142 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:04.143 DEBUG:boto:Method: HEAD 2022-09-03T06:12:04.143 DEBUG:boto:Path: /dsgvcy-69/a 2022-09-03T06:12:04.144 DEBUG:boto:Data: 2022-09-03T06:12:04.144 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.145 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.145 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.146 DEBUG:boto:Params: {} 2022-09-03T06:12:04.147 DEBUG:boto:Token: None 2022-09-03T06:12:04.147 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/a 2022-09-03T06:12:04.148 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:maXXI1Btzz6NsTDTw0XKJzjIExE= 2022-09-03T06:12:04.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:maXXI1Btzz6NsTDTw0XKJzjIExE=', 'Content-Length': '0'} 2022-09-03T06:12:04.150 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a8e205f0fa62c515-006312f034-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.151 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:12:04.152 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:04.152 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:04.153 DEBUG:boto:Method: GET 2022-09-03T06:12:04.153 DEBUG:boto:Path: /dsgvcy-69/a 2022-09-03T06:12:04.154 DEBUG:boto:Data: 2022-09-03T06:12:04.154 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.155 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.156 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.156 DEBUG:boto:Params: {} 2022-09-03T06:12:04.157 DEBUG:boto:Token: None 2022-09-03T06:12:04.157 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/a 2022-09-03T06:12:04.158 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OLXqZY035+6RGOpGtStAooxLYeY= 2022-09-03T06:12:04.159 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OLXqZY035+6RGOpGtStAooxLYeY=', 'Content-Length': '0'} 2022-09-03T06:12:04.161 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d105dd866c19b9a-006312f034-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.203 DEBUG:boto:path=/dsgvcy-69/a 2022-09-03T06:12:04.204 DEBUG:boto:auth_path=/dsgvcy-69/a 2022-09-03T06:12:04.205 DEBUG:boto:Method: GET 2022-09-03T06:12:04.206 DEBUG:boto:Path: /dsgvcy-69/a 2022-09-03T06:12:04.207 DEBUG:boto:Data: 2022-09-03T06:12:04.208 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.209 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.210 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.211 DEBUG:boto:Params: {} 2022-09-03T06:12:04.214 DEBUG:boto:Token: None 2022-09-03T06:12:04.215 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/a 2022-09-03T06:12:04.216 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OLXqZY035+6RGOpGtStAooxLYeY= 2022-09-03T06:12:04.217 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OLXqZY035+6RGOpGtStAooxLYeY=', 'Content-Length': '0'} 2022-09-03T06:12:04.221 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006cd325f28175c777-006312f034-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.222 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:04.223 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:04.224 DEBUG:boto:path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:04.225 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:04.226 DEBUG:boto:Method: GET 2022-09-03T06:12:04.227 DEBUG:boto:Path: /dsgvcy-69/?prefix=a 2022-09-03T06:12:04.228 DEBUG:boto:Data: 2022-09-03T06:12:04.229 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.230 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.231 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.232 DEBUG:boto:Params: {} 2022-09-03T06:12:04.233 DEBUG:boto:Token: None 2022-09-03T06:12:04.235 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/ 2022-09-03T06:12:04.236 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8= 2022-09-03T06:12:04.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8=', 'Content-Length': '0'} 2022-09-03T06:12:04.242 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000095d4563f7785d215-006312f034-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.243 DEBUG:boto:b'dsgvcy-69a1000falsea2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:04.245 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:04.246 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:04.247 DEBUG:boto:path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:04.248 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=a 2022-09-03T06:12:04.249 DEBUG:boto:Method: GET 2022-09-03T06:12:04.250 DEBUG:boto:Path: /dsgvcy-69/?prefix=a 2022-09-03T06:12:04.251 DEBUG:boto:Data: 2022-09-03T06:12:04.252 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.253 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.253 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.254 DEBUG:boto:Params: {} 2022-09-03T06:12:04.255 DEBUG:boto:Token: None 2022-09-03T06:12:04.256 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/ 2022-09-03T06:12:04.257 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8= 2022-09-03T06:12:04.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8=', 'Content-Length': '0'} 2022-09-03T06:12:04.261 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001eed20cbc8544057-006312f034-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.262 DEBUG:boto:b'dsgvcy-69a1000falsea2022-09-03T06:11:25.021Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:04.263 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:12:04.264 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:04.265 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:04.266 DEBUG:boto:path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.267 DEBUG:boto:auth_path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.268 DEBUG:boto:Method: GET 2022-09-03T06:12:04.269 DEBUG:boto:Path: /dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.270 DEBUG:boto:Data: 2022-09-03T06:12:04.271 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.272 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.273 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.274 DEBUG:boto:Params: {} 2022-09-03T06:12:04.275 DEBUG:boto:Token: None 2022-09-03T06:12:04.276 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.277 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2p1IId5X1mom3sAiZfgR7IwnTFo= 2022-09-03T06:12:04.278 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2p1IId5X1mom3sAiZfgR7IwnTFo=', 'Content-Length': '0'} 2022-09-03T06:12:04.280 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e2b07bebd6710f3b-006312f034-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.281 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:04.282 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:04.283 DEBUG:boto:path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.284 DEBUG:boto:auth_path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.285 DEBUG:boto:Method: GET 2022-09-03T06:12:04.286 DEBUG:boto:Path: /dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.287 DEBUG:boto:Data: 2022-09-03T06:12:04.288 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.288 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.289 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.290 DEBUG:boto:Params: {} 2022-09-03T06:12:04.290 DEBUG:boto:Token: None 2022-09-03T06:12:04.291 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.292 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2p1IId5X1mom3sAiZfgR7IwnTFo= 2022-09-03T06:12:04.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2p1IId5X1mom3sAiZfgR7IwnTFo=', 'Content-Length': '0'} 2022-09-03T06:12:04.296 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000878259c65fa09bc5-006312f034-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.297 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:04.298 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:04.299 DEBUG:boto:path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.300 DEBUG:boto:auth_path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.301 DEBUG:boto:Method: HEAD 2022-09-03T06:12:04.302 DEBUG:boto:Path: /dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.303 DEBUG:boto:Data: 2022-09-03T06:12:04.304 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.305 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.307 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.308 DEBUG:boto:Params: {} 2022-09-03T06:12:04.308 DEBUG:boto:Token: None 2022-09-03T06:12:04.309 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.310 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:joJST5d4C5LG8ZgO2VrpAb1jU/0= 2022-09-03T06:12:04.311 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:joJST5d4C5LG8ZgO2VrpAb1jU/0=', 'Content-Length': '0'} 2022-09-03T06:12:04.313 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d65903ea82bf8da3-006312f034-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.315 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:04.317 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:04.318 DEBUG:boto:path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.318 DEBUG:boto:auth_path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.319 DEBUG:boto:Method: HEAD 2022-09-03T06:12:04.319 DEBUG:boto:Path: /dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.320 DEBUG:boto:Data: 2022-09-03T06:12:04.320 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.321 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.321 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.322 DEBUG:boto:Params: {} 2022-09-03T06:12:04.322 DEBUG:boto:Token: None 2022-09-03T06:12:04.323 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.323 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:joJST5d4C5LG8ZgO2VrpAb1jU/0= 2022-09-03T06:12:04.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:joJST5d4C5LG8ZgO2VrpAb1jU/0=', 'Content-Length': '0'} 2022-09-03T06:12:04.326 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000005f1218c1c19db-006312f034-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.327 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:12:04.328 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:04.329 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:04.329 DEBUG:boto:path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.330 DEBUG:boto:auth_path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.331 DEBUG:boto:Method: GET 2022-09-03T06:12:04.331 DEBUG:boto:Path: /dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.332 DEBUG:boto:Data: 2022-09-03T06:12:04.332 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.333 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.333 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.334 DEBUG:boto:Params: {} 2022-09-03T06:12:04.334 DEBUG:boto:Token: None 2022-09-03T06:12:04.335 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.336 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2p1IId5X1mom3sAiZfgR7IwnTFo= 2022-09-03T06:12:04.336 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2p1IId5X1mom3sAiZfgR7IwnTFo=', 'Content-Length': '0'} 2022-09-03T06:12:04.338 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2eff120a1c2ad56-006312f034-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.339 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:04.340 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:04.340 DEBUG:boto:path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.341 DEBUG:boto:auth_path=/dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.341 DEBUG:boto:Method: GET 2022-09-03T06:12:04.342 DEBUG:boto:Path: /dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.343 DEBUG:boto:Data: 2022-09-03T06:12:04.343 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.344 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.344 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.345 DEBUG:boto:Params: {} 2022-09-03T06:12:04.345 DEBUG:boto:Token: None 2022-09-03T06:12:04.346 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/b?versionId=null 2022-09-03T06:12:04.347 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2p1IId5X1mom3sAiZfgR7IwnTFo= 2022-09-03T06:12:04.347 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2p1IId5X1mom3sAiZfgR7IwnTFo=', 'Content-Length': '0'} 2022-09-03T06:12:04.349 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000056096edab2757b27-006312f034-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.350 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:04.351 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:04.351 DEBUG:boto:path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:04.352 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:04.352 DEBUG:boto:Method: GET 2022-09-03T06:12:04.353 DEBUG:boto:Path: /dsgvcy-69/?prefix=b 2022-09-03T06:12:04.353 DEBUG:boto:Data: 2022-09-03T06:12:04.354 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.354 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.355 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.356 DEBUG:boto:Params: {} 2022-09-03T06:12:04.356 DEBUG:boto:Token: None 2022-09-03T06:12:04.357 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/ 2022-09-03T06:12:04.357 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8= 2022-09-03T06:12:04.358 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8=', 'Content-Length': '0'} 2022-09-03T06:12:04.360 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fe5ea3ab751fe920-006312f034-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.402 DEBUG:boto:b'dsgvcy-69b1000falseb2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:04.403 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:04.404 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:04.405 DEBUG:boto:path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:04.405 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:04.406 DEBUG:boto:Method: GET 2022-09-03T06:12:04.407 DEBUG:boto:Path: /dsgvcy-69/?prefix=b 2022-09-03T06:12:04.408 DEBUG:boto:Data: 2022-09-03T06:12:04.408 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.409 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.409 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.410 DEBUG:boto:Params: {} 2022-09-03T06:12:04.411 DEBUG:boto:Token: None 2022-09-03T06:12:04.411 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/ 2022-09-03T06:12:04.412 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8= 2022-09-03T06:12:04.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8=', 'Content-Length': '0'} 2022-09-03T06:12:04.415 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000056fb350b30c397a4-006312f034-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.417 DEBUG:boto:b'dsgvcy-69b1000falseb2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:04.418 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:04.419 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:04.420 DEBUG:boto:Method: HEAD 2022-09-03T06:12:04.420 DEBUG:boto:Path: /dsgvcy-69/b 2022-09-03T06:12:04.421 DEBUG:boto:Data: 2022-09-03T06:12:04.422 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.422 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.423 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.423 DEBUG:boto:Params: {} 2022-09-03T06:12:04.424 DEBUG:boto:Token: None 2022-09-03T06:12:04.425 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/b 2022-09-03T06:12:04.425 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0aNTs+zWWU6rOW16ycNeeLlRqig= 2022-09-03T06:12:04.426 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0aNTs+zWWU6rOW16ycNeeLlRqig=', 'Content-Length': '0'} 2022-09-03T06:12:04.428 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000391048c953f4e493-006312f034-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.429 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:04.430 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:04.431 DEBUG:boto:Method: HEAD 2022-09-03T06:12:04.432 DEBUG:boto:Path: /dsgvcy-69/b 2022-09-03T06:12:04.432 DEBUG:boto:Data: 2022-09-03T06:12:04.433 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.433 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.434 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.434 DEBUG:boto:Params: {} 2022-09-03T06:12:04.435 DEBUG:boto:Token: None 2022-09-03T06:12:04.436 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/b 2022-09-03T06:12:04.436 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0aNTs+zWWU6rOW16ycNeeLlRqig= 2022-09-03T06:12:04.437 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0aNTs+zWWU6rOW16ycNeeLlRqig=', 'Content-Length': '0'} 2022-09-03T06:12:04.439 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c9baf115796b1020-006312f034-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.440 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:12:04.440 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:04.441 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:04.442 DEBUG:boto:Method: GET 2022-09-03T06:12:04.443 DEBUG:boto:Path: /dsgvcy-69/b 2022-09-03T06:12:04.444 DEBUG:boto:Data: 2022-09-03T06:12:04.445 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.446 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.447 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.448 DEBUG:boto:Params: {} 2022-09-03T06:12:04.449 DEBUG:boto:Token: None 2022-09-03T06:12:04.450 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/b 2022-09-03T06:12:04.451 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Crr8StyY4zdYVMIJHANKyvwxoJ4= 2022-09-03T06:12:04.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Crr8StyY4zdYVMIJHANKyvwxoJ4=', 'Content-Length': '0'} 2022-09-03T06:12:04.454 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000040ddd9c1c6903f90-006312f034-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.455 DEBUG:boto:path=/dsgvcy-69/b 2022-09-03T06:12:04.456 DEBUG:boto:auth_path=/dsgvcy-69/b 2022-09-03T06:12:04.456 DEBUG:boto:Method: GET 2022-09-03T06:12:04.457 DEBUG:boto:Path: /dsgvcy-69/b 2022-09-03T06:12:04.458 DEBUG:boto:Data: 2022-09-03T06:12:04.458 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.459 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.459 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.460 DEBUG:boto:Params: {} 2022-09-03T06:12:04.460 DEBUG:boto:Token: None 2022-09-03T06:12:04.461 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/b 2022-09-03T06:12:04.462 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Crr8StyY4zdYVMIJHANKyvwxoJ4= 2022-09-03T06:12:04.462 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Crr8StyY4zdYVMIJHANKyvwxoJ4=', 'Content-Length': '0'} 2022-09-03T06:12:04.464 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a5bb1a95a6b2e6e6-006312f034-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.506 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:04.507 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:04.508 DEBUG:boto:path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:04.509 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:04.511 DEBUG:boto:Method: GET 2022-09-03T06:12:04.512 DEBUG:boto:Path: /dsgvcy-69/?prefix=b 2022-09-03T06:12:04.512 DEBUG:boto:Data: 2022-09-03T06:12:04.514 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.515 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.516 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.517 DEBUG:boto:Params: {} 2022-09-03T06:12:04.518 DEBUG:boto:Token: None 2022-09-03T06:12:04.519 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/ 2022-09-03T06:12:04.519 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8= 2022-09-03T06:12:04.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8=', 'Content-Length': '0'} 2022-09-03T06:12:04.523 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003cb56f56e46c75b7-006312f034-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.523 DEBUG:boto:b'dsgvcy-69b1000falseb2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:04.524 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:04.525 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:04.525 DEBUG:boto:path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:04.526 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=b 2022-09-03T06:12:04.527 DEBUG:boto:Method: GET 2022-09-03T06:12:04.527 DEBUG:boto:Path: /dsgvcy-69/?prefix=b 2022-09-03T06:12:04.528 DEBUG:boto:Data: 2022-09-03T06:12:04.528 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.529 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.530 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.530 DEBUG:boto:Params: {} 2022-09-03T06:12:04.531 DEBUG:boto:Token: None 2022-09-03T06:12:04.531 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/ 2022-09-03T06:12:04.532 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8= 2022-09-03T06:12:04.533 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8=', 'Content-Length': '0'} 2022-09-03T06:12:04.536 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c73818b24add9649-006312f034-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.537 DEBUG:boto:b'dsgvcy-69b1000falseb2022-09-03T06:11:25.087Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:04.538 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:12:04.538 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:04.539 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:04.540 DEBUG:boto:path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.541 DEBUG:boto:auth_path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.542 DEBUG:boto:Method: GET 2022-09-03T06:12:04.543 DEBUG:boto:Path: /dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.544 DEBUG:boto:Data: 2022-09-03T06:12:04.545 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.547 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.548 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.549 DEBUG:boto:Params: {} 2022-09-03T06:12:04.550 DEBUG:boto:Token: None 2022-09-03T06:12:04.551 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.552 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sBBCQej8Lch0yd6MbE/sUmb8VM0= 2022-09-03T06:12:04.553 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sBBCQej8Lch0yd6MbE/sUmb8VM0=', 'Content-Length': '0'} 2022-09-03T06:12:04.555 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f606f216abfac9c-006312f034-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.556 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:04.557 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:04.557 DEBUG:boto:path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.558 DEBUG:boto:auth_path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.558 DEBUG:boto:Method: GET 2022-09-03T06:12:04.559 DEBUG:boto:Path: /dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.560 DEBUG:boto:Data: 2022-09-03T06:12:04.561 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.562 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.563 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.564 DEBUG:boto:Params: {} 2022-09-03T06:12:04.565 DEBUG:boto:Token: None 2022-09-03T06:12:04.566 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.567 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sBBCQej8Lch0yd6MbE/sUmb8VM0= 2022-09-03T06:12:04.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sBBCQej8Lch0yd6MbE/sUmb8VM0=', 'Content-Length': '0'} 2022-09-03T06:12:04.571 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c0de29236f4d446-006312f034-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.573 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:04.574 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:04.575 DEBUG:boto:path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.576 DEBUG:boto:auth_path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.576 DEBUG:boto:Method: HEAD 2022-09-03T06:12:04.577 DEBUG:boto:Path: /dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.579 DEBUG:boto:Data: 2022-09-03T06:12:04.580 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.581 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.582 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.582 DEBUG:boto:Params: {} 2022-09-03T06:12:04.583 DEBUG:boto:Token: None 2022-09-03T06:12:04.584 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.585 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gtGtxqvMYyJoeLhutsKnXyf2Oto= 2022-09-03T06:12:04.586 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gtGtxqvMYyJoeLhutsKnXyf2Oto=', 'Content-Length': '0'} 2022-09-03T06:12:04.589 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d90990b6a1d4e903-006312f034-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.590 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:04.591 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:04.592 DEBUG:boto:path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.593 DEBUG:boto:auth_path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.594 DEBUG:boto:Method: HEAD 2022-09-03T06:12:04.595 DEBUG:boto:Path: /dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.596 DEBUG:boto:Data: 2022-09-03T06:12:04.597 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.597 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.598 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.599 DEBUG:boto:Params: {} 2022-09-03T06:12:04.599 DEBUG:boto:Token: None 2022-09-03T06:12:04.600 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.600 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gtGtxqvMYyJoeLhutsKnXyf2Oto= 2022-09-03T06:12:04.601 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gtGtxqvMYyJoeLhutsKnXyf2Oto=', 'Content-Length': '0'} 2022-09-03T06:12:04.603 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d2e85ce85eb93d88-006312f034-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.604 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:12:04.605 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:04.606 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:04.607 DEBUG:boto:path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.608 DEBUG:boto:auth_path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.609 DEBUG:boto:Method: GET 2022-09-03T06:12:04.610 DEBUG:boto:Path: /dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.611 DEBUG:boto:Data: 2022-09-03T06:12:04.612 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.612 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.613 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.614 DEBUG:boto:Params: {} 2022-09-03T06:12:04.615 DEBUG:boto:Token: None 2022-09-03T06:12:04.616 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.617 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sBBCQej8Lch0yd6MbE/sUmb8VM0= 2022-09-03T06:12:04.618 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sBBCQej8Lch0yd6MbE/sUmb8VM0=', 'Content-Length': '0'} 2022-09-03T06:12:04.621 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000862f6b2d0d8f5730-006312f034-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.623 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:04.624 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:04.625 DEBUG:boto:path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.626 DEBUG:boto:auth_path=/dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.627 DEBUG:boto:Method: GET 2022-09-03T06:12:04.628 DEBUG:boto:Path: /dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.629 DEBUG:boto:Data: 2022-09-03T06:12:04.629 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.630 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.631 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.632 DEBUG:boto:Params: {} 2022-09-03T06:12:04.633 DEBUG:boto:Token: None 2022-09-03T06:12:04.634 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/c?versionId=null 2022-09-03T06:12:04.635 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sBBCQej8Lch0yd6MbE/sUmb8VM0= 2022-09-03T06:12:04.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sBBCQej8Lch0yd6MbE/sUmb8VM0=', 'Content-Length': '0'} 2022-09-03T06:12:04.638 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007caea6a7e9c5ecb2-006312f034-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.682 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:04.683 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:04.684 DEBUG:boto:path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:04.685 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:04.686 DEBUG:boto:Method: GET 2022-09-03T06:12:04.687 DEBUG:boto:Path: /dsgvcy-69/?prefix=c 2022-09-03T06:12:04.688 DEBUG:boto:Data: 2022-09-03T06:12:04.689 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.690 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.691 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.692 DEBUG:boto:Params: {} 2022-09-03T06:12:04.692 DEBUG:boto:Token: None 2022-09-03T06:12:04.693 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/ 2022-09-03T06:12:04.693 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8= 2022-09-03T06:12:04.694 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8=', 'Content-Length': '0'} 2022-09-03T06:12:04.697 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008b31700688a00fb2-006312f034-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.698 DEBUG:boto:b'dsgvcy-69c1000falsec2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:04.699 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:04.699 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:04.700 DEBUG:boto:path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:04.701 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:04.701 DEBUG:boto:Method: GET 2022-09-03T06:12:04.702 DEBUG:boto:Path: /dsgvcy-69/?prefix=c 2022-09-03T06:12:04.702 DEBUG:boto:Data: 2022-09-03T06:12:04.703 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.704 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.704 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.705 DEBUG:boto:Params: {} 2022-09-03T06:12:04.706 DEBUG:boto:Token: None 2022-09-03T06:12:04.707 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/ 2022-09-03T06:12:04.708 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8= 2022-09-03T06:12:04.709 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8=', 'Content-Length': '0'} 2022-09-03T06:12:04.712 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001a18634585d99f01-006312f034-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.714 DEBUG:boto:b'dsgvcy-69c1000falsec2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:04.715 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:04.715 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:04.716 DEBUG:boto:Method: HEAD 2022-09-03T06:12:04.716 DEBUG:boto:Path: /dsgvcy-69/c 2022-09-03T06:12:04.717 DEBUG:boto:Data: 2022-09-03T06:12:04.718 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.719 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.720 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.721 DEBUG:boto:Params: {} 2022-09-03T06:12:04.722 DEBUG:boto:Token: None 2022-09-03T06:12:04.723 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/c 2022-09-03T06:12:04.723 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gJKvYnA9vWHG23nILxf6oxoEE/8= 2022-09-03T06:12:04.724 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gJKvYnA9vWHG23nILxf6oxoEE/8=', 'Content-Length': '0'} 2022-09-03T06:12:04.727 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fb51e0718b88c93b-006312f034-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.728 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:04.729 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:04.730 DEBUG:boto:Method: HEAD 2022-09-03T06:12:04.730 DEBUG:boto:Path: /dsgvcy-69/c 2022-09-03T06:12:04.731 DEBUG:boto:Data: 2022-09-03T06:12:04.731 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.732 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.732 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.733 DEBUG:boto:Params: {} 2022-09-03T06:12:04.733 DEBUG:boto:Token: None 2022-09-03T06:12:04.734 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/c 2022-09-03T06:12:04.735 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gJKvYnA9vWHG23nILxf6oxoEE/8= 2022-09-03T06:12:04.736 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gJKvYnA9vWHG23nILxf6oxoEE/8=', 'Content-Length': '0'} 2022-09-03T06:12:04.739 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009adb3a5c723e9e9f-006312f034-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.740 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:12:04.741 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:04.742 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:04.743 DEBUG:boto:Method: GET 2022-09-03T06:12:04.744 DEBUG:boto:Path: /dsgvcy-69/c 2022-09-03T06:12:04.745 DEBUG:boto:Data: 2022-09-03T06:12:04.746 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.747 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.748 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.749 DEBUG:boto:Params: {} 2022-09-03T06:12:04.750 DEBUG:boto:Token: None 2022-09-03T06:12:04.751 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/c 2022-09-03T06:12:04.752 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ogAFG3vB+uZ4jcQ0g4EKNcI0Wzw= 2022-09-03T06:12:04.753 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ogAFG3vB+uZ4jcQ0g4EKNcI0Wzw=', 'Content-Length': '0'} 2022-09-03T06:12:04.756 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000183d43045b010750-006312f034-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.802 DEBUG:boto:path=/dsgvcy-69/c 2022-09-03T06:12:04.803 DEBUG:boto:auth_path=/dsgvcy-69/c 2022-09-03T06:12:04.803 DEBUG:boto:Method: GET 2022-09-03T06:12:04.804 DEBUG:boto:Path: /dsgvcy-69/c 2022-09-03T06:12:04.805 DEBUG:boto:Data: 2022-09-03T06:12:04.805 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.806 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.807 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.807 DEBUG:boto:Params: {} 2022-09-03T06:12:04.808 DEBUG:boto:Token: None 2022-09-03T06:12:04.809 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/c 2022-09-03T06:12:04.809 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ogAFG3vB+uZ4jcQ0g4EKNcI0Wzw= 2022-09-03T06:12:04.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ogAFG3vB+uZ4jcQ0g4EKNcI0Wzw=', 'Content-Length': '0'} 2022-09-03T06:12:04.812 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007a580f23ed359224-006312f034-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.813 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:04.814 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:04.814 DEBUG:boto:path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:04.815 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:04.815 DEBUG:boto:Method: GET 2022-09-03T06:12:04.816 DEBUG:boto:Path: /dsgvcy-69/?prefix=c 2022-09-03T06:12:04.817 DEBUG:boto:Data: 2022-09-03T06:12:04.817 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.818 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.819 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.819 DEBUG:boto:Params: {} 2022-09-03T06:12:04.820 DEBUG:boto:Token: None 2022-09-03T06:12:04.820 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/ 2022-09-03T06:12:04.821 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8= 2022-09-03T06:12:04.822 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8=', 'Content-Length': '0'} 2022-09-03T06:12:04.824 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000040ca3890356752ad-006312f034-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.825 DEBUG:boto:b'dsgvcy-69c1000falsec2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:04.826 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:04.826 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:04.827 DEBUG:boto:path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:04.828 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=c 2022-09-03T06:12:04.828 DEBUG:boto:Method: GET 2022-09-03T06:12:04.829 DEBUG:boto:Path: /dsgvcy-69/?prefix=c 2022-09-03T06:12:04.829 DEBUG:boto:Data: 2022-09-03T06:12:04.830 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.831 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.831 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.832 DEBUG:boto:Params: {} 2022-09-03T06:12:04.832 DEBUG:boto:Token: None 2022-09-03T06:12:04.833 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/ 2022-09-03T06:12:04.833 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8= 2022-09-03T06:12:04.834 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8=', 'Content-Length': '0'} 2022-09-03T06:12:04.837 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004b33a409a9df9afa-006312f034-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.838 DEBUG:boto:b'dsgvcy-69c1000falsec2022-09-03T06:11:25.127Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:04.839 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:12:04.839 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:04.840 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:04.840 DEBUG:boto:path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.841 DEBUG:boto:auth_path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.841 DEBUG:boto:Method: GET 2022-09-03T06:12:04.842 DEBUG:boto:Path: /dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.843 DEBUG:boto:Data: 2022-09-03T06:12:04.843 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.844 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.844 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.845 DEBUG:boto:Params: {} 2022-09-03T06:12:04.845 DEBUG:boto:Token: None 2022-09-03T06:12:04.846 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.847 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nYRbk6mFtyGyprNulIDKewtoGjY= 2022-09-03T06:12:04.847 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nYRbk6mFtyGyprNulIDKewtoGjY=', 'Content-Length': '0'} 2022-09-03T06:12:04.850 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f6e9bc74fb0d21e-006312f034-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.851 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:04.852 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:04.852 DEBUG:boto:path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.853 DEBUG:boto:auth_path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.853 DEBUG:boto:Method: GET 2022-09-03T06:12:04.854 DEBUG:boto:Path: /dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.854 DEBUG:boto:Data: 2022-09-03T06:12:04.855 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.855 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.856 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.857 DEBUG:boto:Params: {} 2022-09-03T06:12:04.857 DEBUG:boto:Token: None 2022-09-03T06:12:04.858 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.859 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nYRbk6mFtyGyprNulIDKewtoGjY= 2022-09-03T06:12:04.859 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nYRbk6mFtyGyprNulIDKewtoGjY=', 'Content-Length': '0'} 2022-09-03T06:12:04.862 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d81837325b423a3e-006312f034-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.862 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:04.863 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:04.863 DEBUG:boto:path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.864 DEBUG:boto:auth_path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.864 DEBUG:boto:Method: HEAD 2022-09-03T06:12:04.865 DEBUG:boto:Path: /dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.866 DEBUG:boto:Data: 2022-09-03T06:12:04.866 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.867 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.867 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.868 DEBUG:boto:Params: {} 2022-09-03T06:12:04.868 DEBUG:boto:Token: None 2022-09-03T06:12:04.869 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.869 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lhTDSqmWwa7u6HXflu32lS2/XEI= 2022-09-03T06:12:04.870 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lhTDSqmWwa7u6HXflu32lS2/XEI=', 'Content-Length': '0'} 2022-09-03T06:12:04.872 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000008aaca3966da2ba9-006312f034-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.873 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:04.874 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:04.874 DEBUG:boto:path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.875 DEBUG:boto:auth_path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.875 DEBUG:boto:Method: HEAD 2022-09-03T06:12:04.876 DEBUG:boto:Path: /dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.876 DEBUG:boto:Data: 2022-09-03T06:12:04.877 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.877 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.878 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.878 DEBUG:boto:Params: {} 2022-09-03T06:12:04.879 DEBUG:boto:Token: None 2022-09-03T06:12:04.880 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.880 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lhTDSqmWwa7u6HXflu32lS2/XEI= 2022-09-03T06:12:04.881 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lhTDSqmWwa7u6HXflu32lS2/XEI=', 'Content-Length': '0'} 2022-09-03T06:12:04.883 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002eb2152aa04f90fa-006312f034-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.884 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:12:04.884 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:04.885 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:04.885 DEBUG:boto:path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.886 DEBUG:boto:auth_path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.886 DEBUG:boto:Method: GET 2022-09-03T06:12:04.887 DEBUG:boto:Path: /dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.887 DEBUG:boto:Data: 2022-09-03T06:12:04.888 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.888 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.889 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.890 DEBUG:boto:Params: {} 2022-09-03T06:12:04.890 DEBUG:boto:Token: None 2022-09-03T06:12:04.891 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.891 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nYRbk6mFtyGyprNulIDKewtoGjY= 2022-09-03T06:12:04.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nYRbk6mFtyGyprNulIDKewtoGjY=', 'Content-Length': '0'} 2022-09-03T06:12:04.894 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cbfd8ecab76047c3-006312f034-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.895 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:04.895 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:04.896 DEBUG:boto:path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.896 DEBUG:boto:auth_path=/dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.897 DEBUG:boto:Method: GET 2022-09-03T06:12:04.897 DEBUG:boto:Path: /dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.898 DEBUG:boto:Data: 2022-09-03T06:12:04.899 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.899 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.900 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.900 DEBUG:boto:Params: {} 2022-09-03T06:12:04.901 DEBUG:boto:Token: None 2022-09-03T06:12:04.901 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/d?versionId=null 2022-09-03T06:12:04.902 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nYRbk6mFtyGyprNulIDKewtoGjY= 2022-09-03T06:12:04.903 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nYRbk6mFtyGyprNulIDKewtoGjY=', 'Content-Length': '0'} 2022-09-03T06:12:04.905 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000faa7e2d4dd9e63d0-006312f034-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.906 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:04.906 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:04.907 DEBUG:boto:path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:04.907 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:04.908 DEBUG:boto:Method: GET 2022-09-03T06:12:04.909 DEBUG:boto:Path: /dsgvcy-69/?prefix=d 2022-09-03T06:12:04.909 DEBUG:boto:Data: 2022-09-03T06:12:04.910 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.911 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.911 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.912 DEBUG:boto:Params: {} 2022-09-03T06:12:04.913 DEBUG:boto:Token: None 2022-09-03T06:12:04.913 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/ 2022-09-03T06:12:04.914 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8= 2022-09-03T06:12:04.915 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8=', 'Content-Length': '0'} 2022-09-03T06:12:04.918 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003bbba447494ed3bc-006312f034-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.958 DEBUG:boto:b'dsgvcy-69d1000falsed2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:04.960 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:04.960 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:04.961 DEBUG:boto:path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:04.961 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:04.962 DEBUG:boto:Method: GET 2022-09-03T06:12:04.963 DEBUG:boto:Path: /dsgvcy-69/?prefix=d 2022-09-03T06:12:04.964 DEBUG:boto:Data: 2022-09-03T06:12:04.965 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.965 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.966 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.967 DEBUG:boto:Params: {} 2022-09-03T06:12:04.968 DEBUG:boto:Token: None 2022-09-03T06:12:04.968 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/ 2022-09-03T06:12:04.969 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8= 2022-09-03T06:12:04.970 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YMaixuuaEDVBNQ+JatnF4SPx2h8=', 'Content-Length': '0'} 2022-09-03T06:12:04.973 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000087fd8f818fbd16f7-006312f034-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.973 DEBUG:boto:b'dsgvcy-69d1000falsed2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:04.975 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:04.976 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:04.976 DEBUG:boto:Method: HEAD 2022-09-03T06:12:04.977 DEBUG:boto:Path: /dsgvcy-69/d 2022-09-03T06:12:04.978 DEBUG:boto:Data: 2022-09-03T06:12:04.978 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.979 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:04.979 DEBUG:boto:Port: 8001 2022-09-03T06:12:04.980 DEBUG:boto:Params: {} 2022-09-03T06:12:04.981 DEBUG:boto:Token: None 2022-09-03T06:12:04.982 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/d 2022-09-03T06:12:04.982 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R3uBhXfrBco4DpDZ/KuDWEwW+9c= 2022-09-03T06:12:04.983 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R3uBhXfrBco4DpDZ/KuDWEwW+9c=', 'Content-Length': '0'} 2022-09-03T06:12:04.986 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000dc2b4cf1cce8099-006312f034-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.987 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:04.988 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:04.988 DEBUG:boto:Method: HEAD 2022-09-03T06:12:04.989 DEBUG:boto:Path: /dsgvcy-69/d 2022-09-03T06:12:04.990 DEBUG:boto:Data: 2022-09-03T06:12:04.991 DEBUG:boto:Headers: {} 2022-09-03T06:12:04.991 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:04.992 DEBUG:boto:Port: 8000 2022-09-03T06:12:04.992 DEBUG:boto:Params: {} 2022-09-03T06:12:04.993 DEBUG:boto:Token: None 2022-09-03T06:12:04.994 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:04 GMT /dsgvcy-69/d 2022-09-03T06:12:04.994 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R3uBhXfrBco4DpDZ/KuDWEwW+9c= 2022-09-03T06:12:04.995 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R3uBhXfrBco4DpDZ/KuDWEwW+9c=', 'Content-Length': '0'} 2022-09-03T06:12:04.997 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000087871af64fdc03aa-006312f034-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:04.998 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:12:04.999 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:04.999 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:05.000 DEBUG:boto:Method: GET 2022-09-03T06:12:05.001 DEBUG:boto:Path: /dsgvcy-69/d 2022-09-03T06:12:05.002 DEBUG:boto:Data: 2022-09-03T06:12:05.002 DEBUG:boto:Headers: {} 2022-09-03T06:12:05.003 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:05.003 DEBUG:boto:Port: 8001 2022-09-03T06:12:05.004 DEBUG:boto:Params: {} 2022-09-03T06:12:05.005 DEBUG:boto:Token: None 2022-09-03T06:12:05.005 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:05 GMT /dsgvcy-69/d 2022-09-03T06:12:05.007 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EDTuJdICeWpaFRNi3sWmP5nnc2s= 2022-09-03T06:12:05.007 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EDTuJdICeWpaFRNi3sWmP5nnc2s=', 'Content-Length': '0'} 2022-09-03T06:12:05.010 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004dd0a2e441ab1851-006312f035-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:05.010 DEBUG:boto:path=/dsgvcy-69/d 2022-09-03T06:12:05.011 DEBUG:boto:auth_path=/dsgvcy-69/d 2022-09-03T06:12:05.012 DEBUG:boto:Method: GET 2022-09-03T06:12:05.012 DEBUG:boto:Path: /dsgvcy-69/d 2022-09-03T06:12:05.013 DEBUG:boto:Data: 2022-09-03T06:12:05.013 DEBUG:boto:Headers: {} 2022-09-03T06:12:05.014 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:05.016 DEBUG:boto:Port: 8000 2022-09-03T06:12:05.017 DEBUG:boto:Params: {} 2022-09-03T06:12:05.018 DEBUG:boto:Token: None 2022-09-03T06:12:05.019 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:05 GMT /dsgvcy-69/d 2022-09-03T06:12:05.020 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EDTuJdICeWpaFRNi3sWmP5nnc2s= 2022-09-03T06:12:05.021 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EDTuJdICeWpaFRNi3sWmP5nnc2s=', 'Content-Length': '0'} 2022-09-03T06:12:05.024 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:11:25 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a724fff3bfae4f85-006312f035-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:05.025 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:05.026 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:05.027 DEBUG:boto:path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:05.028 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:05.029 DEBUG:boto:Method: GET 2022-09-03T06:12:05.029 DEBUG:boto:Path: /dsgvcy-69/?prefix=d 2022-09-03T06:12:05.030 DEBUG:boto:Data: 2022-09-03T06:12:05.031 DEBUG:boto:Headers: {} 2022-09-03T06:12:05.032 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:05.033 DEBUG:boto:Port: 8001 2022-09-03T06:12:05.033 DEBUG:boto:Params: {} 2022-09-03T06:12:05.034 DEBUG:boto:Token: None 2022-09-03T06:12:05.035 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:05 GMT /dsgvcy-69/ 2022-09-03T06:12:05.035 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kfETp74vAyl/+bcdaraQ0bkel4A= 2022-09-03T06:12:05.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kfETp74vAyl/+bcdaraQ0bkel4A=', 'Content-Length': '0'} 2022-09-03T06:12:05.039 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d549e9ba7a0c15d-006312f035-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:05.040 DEBUG:boto:b'dsgvcy-69d1000falsed2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:05.041 DEBUG:boto:path=/dsgvcy-69/ 2022-09-03T06:12:05.042 DEBUG:boto:auth_path=/dsgvcy-69/ 2022-09-03T06:12:05.042 DEBUG:boto:path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:05.043 DEBUG:boto:auth_path=/dsgvcy-69/?prefix=d 2022-09-03T06:12:05.044 DEBUG:boto:Method: GET 2022-09-03T06:12:05.044 DEBUG:boto:Path: /dsgvcy-69/?prefix=d 2022-09-03T06:12:05.045 DEBUG:boto:Data: 2022-09-03T06:12:05.045 DEBUG:boto:Headers: {} 2022-09-03T06:12:05.046 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:05.047 DEBUG:boto:Port: 8000 2022-09-03T06:12:05.047 DEBUG:boto:Params: {} 2022-09-03T06:12:05.048 DEBUG:boto:Token: None 2022-09-03T06:12:05.049 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:05 GMT /dsgvcy-69/ 2022-09-03T06:12:05.049 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kfETp74vAyl/+bcdaraQ0bkel4A= 2022-09-03T06:12:05.050 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kfETp74vAyl/+bcdaraQ0bkel4A=', 'Content-Length': '0'} 2022-09-03T06:12:05.053 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003249b259bf9252e5-006312f035-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:05.053 DEBUG:boto:b'dsgvcy-69d1000falsed2022-09-03T06:11:25.166Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:05.054 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-69 zones={test-zone3, test-zone2} 2022-09-03T06:12:05.055 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'autotrim', '--rgw-sync-log-trim-max-buckets', '0', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:12:05.055 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'autotrim', '--rgw-sync-log-trim-max-buckets', '0', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:12:05.056 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog autotrim --rgw-sync-log-trim-max-buckets 0 --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T06:12:05.106 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:05.106 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:05.176 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-70 2022-09-03T06:12:05.177 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:05.177 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:05.178 DEBUG:boto:Method: PUT 2022-09-03T06:12:05.178 DEBUG:boto:Path: /dsgvcy-70/ 2022-09-03T06:12:05.179 DEBUG:boto:Data: 2022-09-03T06:12:05.180 DEBUG:boto:Headers: {} 2022-09-03T06:12:05.180 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:05.181 DEBUG:boto:Port: 8000 2022-09-03T06:12:05.181 DEBUG:boto:Params: {} 2022-09-03T06:12:05.182 DEBUG:boto:Token: None 2022-09-03T06:12:05.183 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 06:12:05 GMT /dsgvcy-70/ 2022-09-03T06:12:05.184 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XIdpIvDUHkHaAYCTf7TclNQw/MY= 2022-09-03T06:12:05.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XIdpIvDUHkHaAYCTf7TclNQw/MY=', 'Content-Length': '0'} 2022-09-03T06:12:05.211 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000001e9d18e7bd284c1d-006312f035-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:05.212 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:05.213 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:05.214 DEBUG:boto:Method: HEAD 2022-09-03T06:12:05.215 DEBUG:boto:Path: /dsgvcy-70/ 2022-09-03T06:12:05.217 DEBUG:boto:Data: 2022-09-03T06:12:05.217 DEBUG:boto:Headers: {} 2022-09-03T06:12:05.218 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:05.220 DEBUG:boto:Port: 8000 2022-09-03T06:12:05.221 DEBUG:boto:Params: {} 2022-09-03T06:12:05.222 DEBUG:boto:Token: None 2022-09-03T06:12:05.223 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:05 GMT /dsgvcy-70/ 2022-09-03T06:12:05.224 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AFkFMAX3HIlaV9Q67RPB155LB2Q= 2022-09-03T06:12:05.225 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AFkFMAX3HIlaV9Q67RPB155LB2Q=', 'Content-Length': '0'} 2022-09-03T06:12:05.228 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000007e69c13ca8761e3-006312f035-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:05.229 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:05.230 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:05.230 DEBUG:boto:Method: PUT 2022-09-03T06:12:05.231 DEBUG:boto:Path: /dsgvcy-70/a 2022-09-03T06:12:05.231 DEBUG:boto:Data: 2022-09-03T06:12:05.232 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:12:05.232 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:05.233 DEBUG:boto:Port: 8000 2022-09-03T06:12:05.233 DEBUG:boto:Params: {} 2022-09-03T06:12:05.234 DEBUG:boto:Token: None 2022-09-03T06:12:05.235 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:12:05 GMT /dsgvcy-70/a 2022-09-03T06:12:05.235 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1lGGDvNOTcsq5sBDWkm7ZOfkw/Q= 2022-09-03T06:12:05.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:12:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1lGGDvNOTcsq5sBDWkm7ZOfkw/Q='} 2022-09-03T06:12:05.245 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000006a9f4b2ed5867e32-006312f035-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:05.246 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:05.246 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:05.247 DEBUG:boto:Method: HEAD 2022-09-03T06:12:05.247 DEBUG:boto:Path: /dsgvcy-70/ 2022-09-03T06:12:05.248 DEBUG:boto:Data: 2022-09-03T06:12:05.249 DEBUG:boto:Headers: {} 2022-09-03T06:12:05.249 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:05.250 DEBUG:boto:Port: 8000 2022-09-03T06:12:05.251 DEBUG:boto:Params: {} 2022-09-03T06:12:05.251 DEBUG:boto:Token: None 2022-09-03T06:12:05.252 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:05 GMT /dsgvcy-70/ 2022-09-03T06:12:05.253 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AFkFMAX3HIlaV9Q67RPB155LB2Q= 2022-09-03T06:12:05.253 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AFkFMAX3HIlaV9Q67RPB155LB2Q=', 'Content-Length': '0'} 2022-09-03T06:12:05.256 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '3'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f96ed2e3cff773e5-006312f035-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:05.257 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:05.258 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:05.259 DEBUG:boto:Method: PUT 2022-09-03T06:12:05.260 DEBUG:boto:Path: /dsgvcy-70/b 2022-09-03T06:12:05.261 DEBUG:boto:Data: 2022-09-03T06:12:05.262 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:12:05.262 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:05.263 DEBUG:boto:Port: 8000 2022-09-03T06:12:05.264 DEBUG:boto:Params: {} 2022-09-03T06:12:05.264 DEBUG:boto:Token: None 2022-09-03T06:12:05.265 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:12:05 GMT /dsgvcy-70/b 2022-09-03T06:12:05.266 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f4VjnG/bnzgV6iDW5HPeyeW1a2w= 2022-09-03T06:12:05.267 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:12:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f4VjnG/bnzgV6iDW5HPeyeW1a2w='} 2022-09-03T06:12:05.310 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000889bef9248ad4dde-006312f035-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:05.311 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:05.312 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:05.313 DEBUG:boto:Method: HEAD 2022-09-03T06:12:05.313 DEBUG:boto:Path: /dsgvcy-70/ 2022-09-03T06:12:05.314 DEBUG:boto:Data: 2022-09-03T06:12:05.315 DEBUG:boto:Headers: {} 2022-09-03T06:12:05.315 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:05.316 DEBUG:boto:Port: 8000 2022-09-03T06:12:05.317 DEBUG:boto:Params: {} 2022-09-03T06:12:05.317 DEBUG:boto:Token: None 2022-09-03T06:12:05.318 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:05 GMT /dsgvcy-70/ 2022-09-03T06:12:05.318 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AFkFMAX3HIlaV9Q67RPB155LB2Q= 2022-09-03T06:12:05.319 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AFkFMAX3HIlaV9Q67RPB155LB2Q=', 'Content-Length': '0'} 2022-09-03T06:12:05.321 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000faaa26eb871d95d3-006312f035-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:05.322 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:05.323 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:05.323 DEBUG:boto:Method: PUT 2022-09-03T06:12:05.324 DEBUG:boto:Path: /dsgvcy-70/c 2022-09-03T06:12:05.325 DEBUG:boto:Data: 2022-09-03T06:12:05.325 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:12:05.326 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:05.326 DEBUG:boto:Port: 8000 2022-09-03T06:12:05.327 DEBUG:boto:Params: {} 2022-09-03T06:12:05.327 DEBUG:boto:Token: None 2022-09-03T06:12:05.328 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:12:05 GMT /dsgvcy-70/c 2022-09-03T06:12:05.329 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l0Jq73HWNRbuRyftkjUgWxc8S48= 2022-09-03T06:12:05.329 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:12:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l0Jq73HWNRbuRyftkjUgWxc8S48='} 2022-09-03T06:12:05.339 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000549cf86cc880f3e5-006312f035-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:05.340 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:05.341 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:05.342 DEBUG:boto:Method: HEAD 2022-09-03T06:12:05.343 DEBUG:boto:Path: /dsgvcy-70/ 2022-09-03T06:12:05.343 DEBUG:boto:Data: 2022-09-03T06:12:05.346 DEBUG:boto:Headers: {} 2022-09-03T06:12:05.347 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:05.348 DEBUG:boto:Port: 8000 2022-09-03T06:12:05.348 DEBUG:boto:Params: {} 2022-09-03T06:12:05.349 DEBUG:boto:Token: None 2022-09-03T06:12:05.351 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:05 GMT /dsgvcy-70/ 2022-09-03T06:12:05.352 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AFkFMAX3HIlaV9Q67RPB155LB2Q= 2022-09-03T06:12:05.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AFkFMAX3HIlaV9Q67RPB155LB2Q=', 'Content-Length': '0'} 2022-09-03T06:12:05.355 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '9'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ba9a12c0c824fb62-006312f035-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:05.357 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:05.357 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:05.358 DEBUG:boto:Method: PUT 2022-09-03T06:12:05.359 DEBUG:boto:Path: /dsgvcy-70/d 2022-09-03T06:12:05.361 DEBUG:boto:Data: 2022-09-03T06:12:05.362 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:12:05.363 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:05.364 DEBUG:boto:Port: 8000 2022-09-03T06:12:05.365 DEBUG:boto:Params: {} 2022-09-03T06:12:05.367 DEBUG:boto:Token: None 2022-09-03T06:12:05.368 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:12:05 GMT /dsgvcy-70/d 2022-09-03T06:12:05.369 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aWXrmGPpSriJu1P/rF1vz1cFprk= 2022-09-03T06:12:05.370 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:12:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aWXrmGPpSriJu1P/rF1vz1cFprk='} 2022-09-03T06:12:05.418 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000323c907c83723050-006312f035-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:05.419 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:05.420 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:05.420 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:12:05.470 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:05.471 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:05.502 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T06:12:05.515 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T06:12:05.516 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:05.516 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:12:05.516 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:12:05.517 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:05.517 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:05.517 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:12:05.518 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:12:05.518 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:05.518 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:05.519 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662185525.207441_422.1", 2022-09-03T06:12:05.519 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:12:05.207441Z" 2022-09-03T06:12:05.520 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:05.520 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:05.521 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:12:05.521 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:12:05.522 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:12:05.522 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T06:12:05.524 DEBUG:rgw_multi.tests:master meta markers={0: '', 1: '', 2: '1_1662185525.207441_422.1', 3: ''} 2022-09-03T06:12:05.525 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T06:12:05.526 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:05.527 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:05.528 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:12:05.587 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:05.587 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:05.617 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:05.616+0000 7f4fc8674600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:12:05.618 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:05.617+0000 7f4fc8674600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:12:05.632 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:12:05.633 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:12:05.633 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:12:05.633 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:12:05.634 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:12:05.634 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:12:05.634 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:12:05.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:05.635 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:12:05.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:05.635 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:12:05.635 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:05.636 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:05.636 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:12:05.636 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:05.637 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:12:05.637 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:05.637 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:12:05.637 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:12:05.638 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:05.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:05.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:05.638 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:12:05.639 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:05.639 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:05.639 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:12:05.639 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:05.640 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:12:05.640 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:05.640 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:12:05.640 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:12:05.641 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:05.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:05.641 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:05.641 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:12:05.642 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:05.642 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:05.642 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662185525.207441_422.1", 2022-09-03T06:12:05.642 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:05.643 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:12:05.643 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:05.643 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:12:05.207441Z", 2022-09-03T06:12:05.644 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:12:05.644 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:05.645 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:05.645 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:05.645 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:12:05.646 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:05.646 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:05.646 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:12:05.647 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:05.647 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:12:05.647 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:05.647 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:12:05.648 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:12:05.648 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:05.648 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:05.648 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:12:05.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:05.649 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:12:05.649 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:12:05.650 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:12:05.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:05.650 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:12:05Z" 2022-09-03T06:12:05.650 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:12:05.652 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185525.207441_422.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:12:05.207441Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:12:05Z" } 2022-09-03T06:12:05.652 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185525.207441_422.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:12:05.207441Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:12:05.653 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185525.207441_422.1', 3: ''} 2022-09-03T06:12:05.653 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185525.207441_422.1', 3: ''} 2022-09-03T06:12:05.654 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T06:12:05.654 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T06:12:05.655 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:05.655 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:05.656 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:12:05.706 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:05.707 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:05.732 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:05.730+0000 7fc5aa481600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:12:05.733 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:05.731+0000 7fc5aa481600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:12:05.751 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T06:12:05.751 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T06:12:05.752 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T06:12:05.752 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T06:12:05.753 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T06:12:05.753 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:12:05.754 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:12:05.754 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:05.755 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T06:12:05.755 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:05.756 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T06:12:05.756 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:12:05.757 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:12:05.757 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:12:05.758 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:12:05.758 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T06:12:05.759 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:12:05.759 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:12:05.760 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:12:05.760 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:12:05.760 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:05.761 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:05.761 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T06:12:05.762 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:12:05.762 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:12:05.762 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:12:05.763 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:12:05.763 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:12:05.763 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:12:05.763 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T06:12:05.764 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:12:05.764 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:12:05.764 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:05.764 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:05.765 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T06:12:05.765 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:12:05.765 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:12:05.765 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662185525.207441_422.1", 2022-09-03T06:12:05.766 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:12:05.766 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:12:05.766 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:12:05.767 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:12:05.207441Z", 2022-09-03T06:12:05.767 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:12:05.767 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:12:05.767 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:05.768 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:05.768 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T06:12:05.768 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:12:05.769 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:12:05.769 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:12:05.769 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:12:05.770 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:12:05.770 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:12:05.770 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:12:05.771 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:12:05.771 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:12:05.771 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:12:05.772 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:12:05.772 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:05.773 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T06:12:05.773 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T06:12:05.774 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T06:12:05.774 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:05.775 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T06:12:05Z" 2022-09-03T06:12:05.775 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T06:12:05.777 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185525.207441_422.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:12:05.207441Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:12:05Z" } 2022-09-03T06:12:05.778 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185525.207441_422.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:12:05.207441Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:12:05.779 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185525.207441_422.1', 3: ''} 2022-09-03T06:12:05.780 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185525.207441_422.1', 3: ''} 2022-09-03T06:12:05.781 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T06:12:05.782 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T06:12:05.783 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:05.784 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:05.785 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:12:05.837 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:05.838 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:05.873 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:05.871+0000 7f2cbfb86600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:12:05.874 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:05.872+0000 7f2cbfb86600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:12:05.895 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:12:05.895 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:12:05.895 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:12:05.896 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:12:05.896 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:12:05.896 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:12:05.896 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:12:05.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:05.897 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:12:05.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:05.897 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:12:05.898 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:05.898 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:05.898 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:12:05.899 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:05.899 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:12:05.899 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:05.899 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:12:05.900 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:12:05.900 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:05.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:05.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:05.901 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:12:05.901 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:05.901 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:05.901 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:12:05.902 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:05.902 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:12:05.902 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:05.902 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:12:05.903 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:12:05.903 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:05.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:05.903 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:05.904 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:12:05.904 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:05.904 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:05.904 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662185525.196815_419.1", 2022-09-03T06:12:05.905 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:05.905 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:12:05.905 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:05.905 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:12:05.196815Z", 2022-09-03T06:12:05.905 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:12:05.906 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:05.906 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:05.906 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:05.907 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:12:05.907 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:05.907 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:05.907 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:12:05.908 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:05.908 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:12:05.908 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:05.908 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:12:05.909 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:12:05.909 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:05.909 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:05.909 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:12:05.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:05.910 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:12:05.911 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:12:05.911 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:12:05.911 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:05.911 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:12:05Z" 2022-09-03T06:12:05.912 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:12:05.913 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185525.196815_419.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:12:05.196815Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:12:05Z" } 2022-09-03T06:12:05.913 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185525.196815_419.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:12:05.196815Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:12:05.914 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185525.207441_422.1', 3: ''} 2022-09-03T06:12:05.915 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185525.196815_419.1', 3: ''} 2022-09-03T06:12:05.915 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662185525.207441_422.1 target=1_1662185525.196815_419.1 2022-09-03T06:12:10.917 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:10.918 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:10.919 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:12:10.977 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:10.978 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:11.007 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:11.006+0000 7ff7a46d5600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:12:11.008 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:11.007+0000 7ff7a46d5600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:12:11.022 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:12:11.022 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:12:11.023 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:12:11.023 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:12:11.024 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:12:11.024 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:12:11.024 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:12:11.025 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:11.025 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:12:11.026 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:11.026 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:12:11.026 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:11.027 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:11.027 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:12:11.028 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:11.028 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:12:11.028 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:11.029 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:12:11.029 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:12:11.029 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:11.030 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:11.030 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:11.030 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:12:11.030 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:11.031 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:11.031 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:12:11.031 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:11.032 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:12:11.032 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:11.032 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:12:11.032 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:12:11.033 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:11.033 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:11.033 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:11.033 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:12:11.034 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:11.034 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:11.035 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662185525.207441_422.1", 2022-09-03T06:12:11.035 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:11.035 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:12:11.036 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:11.036 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:12:05.207441Z", 2022-09-03T06:12:11.037 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:12:11.037 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:11.037 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:11.037 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:11.038 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:12:11.038 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:11.039 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:11.039 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:12:11.040 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:11.040 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:12:11.041 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:11.041 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:12:11.042 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:12:11.042 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:11.043 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:11.043 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:12:11.044 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:11.044 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:12:11.045 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:12:11.045 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:12:11.046 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:11.046 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:12:11Z" 2022-09-03T06:12:11.047 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:12:11.048 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185525.207441_422.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:12:05.207441Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:12:11Z" } 2022-09-03T06:12:11.049 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185525.207441_422.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:12:05.207441Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:12:11.050 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185525.207441_422.1', 3: ''} 2022-09-03T06:12:11.051 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185525.207441_422.1', 3: ''} 2022-09-03T06:12:11.051 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T06:12:11.052 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-70 2022-09-03T06:12:11.052 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-70', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:11.053 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-70', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:11.053 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-70 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:12:11.110 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:11.111 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:11.143 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:11.142+0000 7f26b7e5e600 1 waiting to reach incremental sync.. 2022-09-03T06:12:16.144 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:16.143+0000 7f26b7e5e600 1 waiting to reach incremental sync.. 2022-09-03T06:12:21.146 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:21.145+0000 7f26b7e5e600 1 bucket sync caught up with source: 2022-09-03T06:12:21.147 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.47.5, , , , 00000000001.54.5, , , 00000000001.52.5, , 00000000001.52.5] 2022-09-03T06:12:21.147 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.47.5, , , , 00000000001.54.5, , , 00000000001.52.5, , 00000000001.52.5] 2022-09-03T06:12:21.147 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:21.145+0000 7f26b7e5e600 0 bucket checkpoint complete 2022-09-03T06:12:21.164 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-70 2022-09-03T06:12:21.165 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-70', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:21.166 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-70', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:21.166 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-70 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:12:21.230 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:21.231 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:21.289 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:21.286+0000 7f5113b01600 1 bucket sync caught up with source: 2022-09-03T06:12:21.290 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.47.5, , , , 00000000001.54.5, , , 00000000001.52.5, , 00000000001.52.5] 2022-09-03T06:12:21.290 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.47.5, , , , 00000000001.54.5, , , 00000000001.52.5, , 00000000001.52.5] 2022-09-03T06:12:21.291 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:21.286+0000 7f5113b01600 0 bucket checkpoint complete 2022-09-03T06:12:21.304 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-70 2022-09-03T06:12:21.305 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-70', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:21.306 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-70', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:21.307 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-70 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:12:21.370 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:21.371 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:21.418 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:21.416+0000 7fcf2bac5600 1 bucket sync caught up with source: 2022-09-03T06:12:21.418 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.47.5, , , , 00000000001.54.5, , , 00000000001.52.5, , 00000000001.52.5] 2022-09-03T06:12:21.419 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.47.5, , , , 00000000001.54.5, , , 00000000001.52.5, , 00000000001.52.5] 2022-09-03T06:12:21.419 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:21.416+0000 7fcf2bac5600 0 bucket checkpoint complete 2022-09-03T06:12:21.438 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-70 2022-09-03T06:12:21.438 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-70', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:21.439 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-70', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:21.440 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-70 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:12:21.497 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:21.497 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:21.539 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:21.538+0000 7f1369cc5600 1 waiting to reach incremental sync.. 2022-09-03T06:12:26.540 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:26.539+0000 7f1369cc5600 1 waiting to reach incremental sync.. 2022-09-03T06:12:31.542 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:31.541+0000 7f1369cc5600 1 bucket sync caught up with source: 2022-09-03T06:12:31.543 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.413.5, , , , 00000000001.696.5, , , 00000000001.478.5, , 00000000001.697.5] 2022-09-03T06:12:31.543 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.413.5, , , , 00000000001.696.5, , , 00000000001.478.5, , 00000000001.697.5] 2022-09-03T06:12:31.543 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:31.541+0000 7f1369cc5600 0 bucket checkpoint complete 2022-09-03T06:12:31.561 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-70 2022-09-03T06:12:31.562 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-70', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:31.562 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-70', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:31.563 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-70 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:12:31.625 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:31.625 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:31.664 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:31.663+0000 7f0205d89600 1 waiting to reach incremental sync.. 2022-09-03T06:12:36.665 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:36.664+0000 7f0205d89600 1 waiting to reach incremental sync.. 2022-09-03T06:12:41.667 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:41.665+0000 7f0205d89600 1 bucket sync caught up with source: 2022-09-03T06:12:41.668 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.413.5, , , , 00000000001.696.5, , , 00000000001.478.5, , 00000000001.697.5] 2022-09-03T06:12:41.669 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.413.5, , , , 00000000001.696.5, , , 00000000001.478.5, , 00000000001.697.5] 2022-09-03T06:12:41.669 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:41.665+0000 7f0205d89600 0 bucket checkpoint complete 2022-09-03T06:12:41.693 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-70 2022-09-03T06:12:41.695 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-70 2022-09-03T06:12:41.696 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-70', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:41.697 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-70', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:41.698 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-70 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:12:41.759 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:41.759 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:41.807 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:41.805+0000 7f82eec56600 1 bucket sync caught up with source: 2022-09-03T06:12:41.807 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.281.5, , , , 00000000002.199.5, , , 00000000001.307.5, , 00000000001.197.5] 2022-09-03T06:12:41.808 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.281.5, , , , 00000000002.199.5, , , 00000000001.307.5, , 00000000001.197.5] 2022-09-03T06:12:41.808 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:41.805+0000 7f82eec56600 0 bucket checkpoint complete 2022-09-03T06:12:41.824 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-70 2022-09-03T06:12:41.826 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-70', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:41.827 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-70', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:41.828 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-70 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:12:41.894 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:41.896 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:41.940 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:41.938+0000 7fb2448e7600 1 bucket sync caught up with source: 2022-09-03T06:12:41.941 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.281.5, , , , 00000000002.199.5, , , 00000000001.307.5, , 00000000001.197.5] 2022-09-03T06:12:41.941 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.281.5, , , , 00000000002.199.5, , , 00000000001.307.5, , 00000000001.197.5] 2022-09-03T06:12:41.942 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:41.938+0000 7fb2448e7600 0 bucket checkpoint complete 2022-09-03T06:12:41.955 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-70 2022-09-03T06:12:41.956 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-70 zones={test-zone2, test-zone1} 2022-09-03T06:12:41.956 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:41.957 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:41.958 DEBUG:boto:Method: HEAD 2022-09-03T06:12:41.960 DEBUG:boto:Path: /dsgvcy-70/ 2022-09-03T06:12:41.961 DEBUG:boto:Data: 2022-09-03T06:12:41.962 DEBUG:boto:Headers: {} 2022-09-03T06:12:41.963 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:41.964 DEBUG:boto:Port: 8000 2022-09-03T06:12:41.965 DEBUG:boto:Params: {} 2022-09-03T06:12:41.966 DEBUG:boto:Token: None 2022-09-03T06:12:41.967 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:41 GMT /dsgvcy-70/ 2022-09-03T06:12:41.969 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GsUkqvOHimzAcSJp3cd8aU2EJYo= 2022-09-03T06:12:41.970 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GsUkqvOHimzAcSJp3cd8aU2EJYo=', 'Content-Length': '0'} 2022-09-03T06:12:41.974 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c3adfa949787c5a1-006312f059-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:12:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:41.975 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:41.975 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:41.976 DEBUG:boto:Method: HEAD 2022-09-03T06:12:41.977 DEBUG:boto:Path: /dsgvcy-70/ 2022-09-03T06:12:41.978 DEBUG:boto:Data: 2022-09-03T06:12:41.979 DEBUG:boto:Headers: {} 2022-09-03T06:12:41.980 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:41.981 DEBUG:boto:Port: 8000 2022-09-03T06:12:41.982 DEBUG:boto:Params: {} 2022-09-03T06:12:41.983 DEBUG:boto:Token: None 2022-09-03T06:12:41.984 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:41 GMT /dsgvcy-70/ 2022-09-03T06:12:41.985 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GsUkqvOHimzAcSJp3cd8aU2EJYo= 2022-09-03T06:12:41.986 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GsUkqvOHimzAcSJp3cd8aU2EJYo=', 'Content-Length': '0'} 2022-09-03T06:12:41.990 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000073a71f4f73567a5a-006312f059-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:12:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:41.992 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:12:41.993 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:41.994 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:41.994 DEBUG:boto:path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:41.995 DEBUG:boto:auth_path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:41.995 DEBUG:boto:Method: GET 2022-09-03T06:12:41.996 DEBUG:boto:Path: /dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:41.996 DEBUG:boto:Data: 2022-09-03T06:12:41.997 DEBUG:boto:Headers: {} 2022-09-03T06:12:41.998 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:41.998 DEBUG:boto:Port: 8000 2022-09-03T06:12:41.999 DEBUG:boto:Params: {} 2022-09-03T06:12:41.999 DEBUG:boto:Token: None 2022-09-03T06:12:42.000 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/?versions 2022-09-03T06:12:42.001 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:isKt8tJ8jLRhLYX6MQ7cLee/8ss= 2022-09-03T06:12:42.001 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:isKt8tJ8jLRhLYX6MQ7cLee/8ss=', 'Content-Length': '0'} 2022-09-03T06:12:42.005 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b1769d9ffef53378-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.005 DEBUG:boto:b'dsgvcy-70999falseanulltrue2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:42.007 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:12:42.007 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:12:42.008 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:12:42.008 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:12:42.009 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:12:42.009 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:42.010 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:42.010 DEBUG:boto:path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:42.011 DEBUG:boto:auth_path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:42.012 DEBUG:boto:Method: GET 2022-09-03T06:12:42.012 DEBUG:boto:Path: /dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:42.013 DEBUG:boto:Data: 2022-09-03T06:12:42.014 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.015 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.015 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.016 DEBUG:boto:Params: {} 2022-09-03T06:12:42.016 DEBUG:boto:Token: None 2022-09-03T06:12:42.017 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/?versions 2022-09-03T06:12:42.018 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:isKt8tJ8jLRhLYX6MQ7cLee/8ss= 2022-09-03T06:12:42.018 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:isKt8tJ8jLRhLYX6MQ7cLee/8ss=', 'Content-Length': '0'} 2022-09-03T06:12:42.021 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000002089421fa487b90-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.022 DEBUG:boto:b'dsgvcy-70999falseanulltrue2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:42.023 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:12:42.024 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:12:42.024 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:12:42.025 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:12:42.025 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:42.026 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:42.027 DEBUG:boto:path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:42.027 DEBUG:boto:auth_path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:42.028 DEBUG:boto:Method: GET 2022-09-03T06:12:42.028 DEBUG:boto:Path: /dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:42.029 DEBUG:boto:Data: 2022-09-03T06:12:42.029 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.030 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.030 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.031 DEBUG:boto:Params: {} 2022-09-03T06:12:42.032 DEBUG:boto:Token: None 2022-09-03T06:12:42.032 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/?versions 2022-09-03T06:12:42.033 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:isKt8tJ8jLRhLYX6MQ7cLee/8ss= 2022-09-03T06:12:42.034 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:isKt8tJ8jLRhLYX6MQ7cLee/8ss=', 'Content-Length': '0'} 2022-09-03T06:12:42.036 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eb705008076efdf2-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.037 DEBUG:boto:b'dsgvcy-70999falseanulltrue2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:42.038 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:42.039 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:42.039 DEBUG:boto:path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:42.040 DEBUG:boto:auth_path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:42.040 DEBUG:boto:Method: GET 2022-09-03T06:12:42.041 DEBUG:boto:Path: /dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:42.042 DEBUG:boto:Data: 2022-09-03T06:12:42.042 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.043 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.043 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.044 DEBUG:boto:Params: {} 2022-09-03T06:12:42.045 DEBUG:boto:Token: None 2022-09-03T06:12:42.046 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/?versions 2022-09-03T06:12:42.046 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:isKt8tJ8jLRhLYX6MQ7cLee/8ss= 2022-09-03T06:12:42.047 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:isKt8tJ8jLRhLYX6MQ7cLee/8ss=', 'Content-Length': '0'} 2022-09-03T06:12:42.050 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000474cadf0d9307ae-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.050 DEBUG:boto:b'dsgvcy-70999falseanulltrue2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:42.051 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:12:42.052 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:42.053 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:42.053 DEBUG:boto:path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.054 DEBUG:boto:auth_path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.055 DEBUG:boto:Method: GET 2022-09-03T06:12:42.056 DEBUG:boto:Path: /dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.056 DEBUG:boto:Data: 2022-09-03T06:12:42.057 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.057 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.058 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.059 DEBUG:boto:Params: {} 2022-09-03T06:12:42.059 DEBUG:boto:Token: None 2022-09-03T06:12:42.060 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.060 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RBKohgRCEG6+fGOmSTwh9a4KAM8= 2022-09-03T06:12:42.061 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RBKohgRCEG6+fGOmSTwh9a4KAM8=', 'Content-Length': '0'} 2022-09-03T06:12:42.064 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009ff7e36e970bfab7-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.065 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:42.066 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:42.066 DEBUG:boto:path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.067 DEBUG:boto:auth_path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.067 DEBUG:boto:Method: GET 2022-09-03T06:12:42.068 DEBUG:boto:Path: /dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.068 DEBUG:boto:Data: 2022-09-03T06:12:42.069 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.069 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.070 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.070 DEBUG:boto:Params: {} 2022-09-03T06:12:42.071 DEBUG:boto:Token: None 2022-09-03T06:12:42.072 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.072 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RBKohgRCEG6+fGOmSTwh9a4KAM8= 2022-09-03T06:12:42.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RBKohgRCEG6+fGOmSTwh9a4KAM8=', 'Content-Length': '0'} 2022-09-03T06:12:42.075 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d71040a0fbc87ea0-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.076 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:42.076 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:42.077 DEBUG:boto:path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.077 DEBUG:boto:auth_path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.078 DEBUG:boto:Method: HEAD 2022-09-03T06:12:42.078 DEBUG:boto:Path: /dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.079 DEBUG:boto:Data: 2022-09-03T06:12:42.079 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.080 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.080 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.081 DEBUG:boto:Params: {} 2022-09-03T06:12:42.082 DEBUG:boto:Token: None 2022-09-03T06:12:42.082 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.083 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BmH0FGpUHYx8hwmT6ZWCDi0i4Yw= 2022-09-03T06:12:42.083 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BmH0FGpUHYx8hwmT6ZWCDi0i4Yw=', 'Content-Length': '0'} 2022-09-03T06:12:42.085 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e4a55b80b5af406-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.086 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:42.087 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:42.087 DEBUG:boto:path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.088 DEBUG:boto:auth_path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.088 DEBUG:boto:Method: HEAD 2022-09-03T06:12:42.089 DEBUG:boto:Path: /dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.089 DEBUG:boto:Data: 2022-09-03T06:12:42.091 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.091 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.092 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.092 DEBUG:boto:Params: {} 2022-09-03T06:12:42.093 DEBUG:boto:Token: None 2022-09-03T06:12:42.093 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.094 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BmH0FGpUHYx8hwmT6ZWCDi0i4Yw= 2022-09-03T06:12:42.095 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BmH0FGpUHYx8hwmT6ZWCDi0i4Yw=', 'Content-Length': '0'} 2022-09-03T06:12:42.096 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca6e6de8062cdd31-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.097 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:12:42.098 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:42.098 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:42.099 DEBUG:boto:path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.099 DEBUG:boto:auth_path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.100 DEBUG:boto:Method: GET 2022-09-03T06:12:42.100 DEBUG:boto:Path: /dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.101 DEBUG:boto:Data: 2022-09-03T06:12:42.101 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.102 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.102 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.103 DEBUG:boto:Params: {} 2022-09-03T06:12:42.103 DEBUG:boto:Token: None 2022-09-03T06:12:42.104 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.105 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RBKohgRCEG6+fGOmSTwh9a4KAM8= 2022-09-03T06:12:42.105 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RBKohgRCEG6+fGOmSTwh9a4KAM8=', 'Content-Length': '0'} 2022-09-03T06:12:42.107 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004268eb8e62df0bc2-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.108 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:42.109 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:42.109 DEBUG:boto:path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.110 DEBUG:boto:auth_path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.110 DEBUG:boto:Method: GET 2022-09-03T06:12:42.111 DEBUG:boto:Path: /dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.111 DEBUG:boto:Data: 2022-09-03T06:12:42.112 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.112 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.113 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.113 DEBUG:boto:Params: {} 2022-09-03T06:12:42.114 DEBUG:boto:Token: None 2022-09-03T06:12:42.115 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/a?versionId=null 2022-09-03T06:12:42.115 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RBKohgRCEG6+fGOmSTwh9a4KAM8= 2022-09-03T06:12:42.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RBKohgRCEG6+fGOmSTwh9a4KAM8=', 'Content-Length': '0'} 2022-09-03T06:12:42.118 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a89039e50753a549-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.119 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:42.119 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:42.120 DEBUG:boto:path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:42.120 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:42.121 DEBUG:boto:Method: GET 2022-09-03T06:12:42.121 DEBUG:boto:Path: /dsgvcy-70/?prefix=a 2022-09-03T06:12:42.122 DEBUG:boto:Data: 2022-09-03T06:12:42.122 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.123 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.123 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.124 DEBUG:boto:Params: {} 2022-09-03T06:12:42.124 DEBUG:boto:Token: None 2022-09-03T06:12:42.125 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/ 2022-09-03T06:12:42.126 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU= 2022-09-03T06:12:42.126 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU=', 'Content-Length': '0'} 2022-09-03T06:12:42.129 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001063763aa2406ede-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.129 DEBUG:boto:b'dsgvcy-70a1000falsea2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:42.130 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:42.131 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:42.131 DEBUG:boto:path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:42.132 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:42.132 DEBUG:boto:Method: GET 2022-09-03T06:12:42.133 DEBUG:boto:Path: /dsgvcy-70/?prefix=a 2022-09-03T06:12:42.133 DEBUG:boto:Data: 2022-09-03T06:12:42.134 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.134 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.135 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.136 DEBUG:boto:Params: {} 2022-09-03T06:12:42.136 DEBUG:boto:Token: None 2022-09-03T06:12:42.137 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/ 2022-09-03T06:12:42.138 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU= 2022-09-03T06:12:42.138 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU=', 'Content-Length': '0'} 2022-09-03T06:12:42.141 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003eb43108d52e8b37-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.142 DEBUG:boto:b'dsgvcy-70a1000falsea2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:42.143 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:42.143 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:42.144 DEBUG:boto:Method: HEAD 2022-09-03T06:12:42.144 DEBUG:boto:Path: /dsgvcy-70/a 2022-09-03T06:12:42.145 DEBUG:boto:Data: 2022-09-03T06:12:42.145 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.146 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.146 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.147 DEBUG:boto:Params: {} 2022-09-03T06:12:42.147 DEBUG:boto:Token: None 2022-09-03T06:12:42.148 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/a 2022-09-03T06:12:42.148 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QmDLNOmcLAItoMiPv1ocnJmx94M= 2022-09-03T06:12:42.149 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QmDLNOmcLAItoMiPv1ocnJmx94M=', 'Content-Length': '0'} 2022-09-03T06:12:42.151 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000569e22ec75967d4e-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.152 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:42.153 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:42.153 DEBUG:boto:Method: HEAD 2022-09-03T06:12:42.154 DEBUG:boto:Path: /dsgvcy-70/a 2022-09-03T06:12:42.154 DEBUG:boto:Data: 2022-09-03T06:12:42.155 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.155 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.156 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.156 DEBUG:boto:Params: {} 2022-09-03T06:12:42.157 DEBUG:boto:Token: None 2022-09-03T06:12:42.157 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/a 2022-09-03T06:12:42.158 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QmDLNOmcLAItoMiPv1ocnJmx94M= 2022-09-03T06:12:42.159 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QmDLNOmcLAItoMiPv1ocnJmx94M=', 'Content-Length': '0'} 2022-09-03T06:12:42.161 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000097a214e5458de1eb-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.162 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:12:42.163 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:42.164 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:42.165 DEBUG:boto:Method: GET 2022-09-03T06:12:42.165 DEBUG:boto:Path: /dsgvcy-70/a 2022-09-03T06:12:42.166 DEBUG:boto:Data: 2022-09-03T06:12:42.167 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.167 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.168 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.169 DEBUG:boto:Params: {} 2022-09-03T06:12:42.170 DEBUG:boto:Token: None 2022-09-03T06:12:42.171 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/a 2022-09-03T06:12:42.172 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1kWmn0vUxym+jIwcGIf8pgoB4DM= 2022-09-03T06:12:42.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1kWmn0vUxym+jIwcGIf8pgoB4DM=', 'Content-Length': '0'} 2022-09-03T06:12:42.175 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000943cc476f21c12fe-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.175 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:42.176 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:42.177 DEBUG:boto:Method: GET 2022-09-03T06:12:42.177 DEBUG:boto:Path: /dsgvcy-70/a 2022-09-03T06:12:42.178 DEBUG:boto:Data: 2022-09-03T06:12:42.178 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.179 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.179 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.180 DEBUG:boto:Params: {} 2022-09-03T06:12:42.180 DEBUG:boto:Token: None 2022-09-03T06:12:42.181 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/a 2022-09-03T06:12:42.182 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1kWmn0vUxym+jIwcGIf8pgoB4DM= 2022-09-03T06:12:42.182 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1kWmn0vUxym+jIwcGIf8pgoB4DM=', 'Content-Length': '0'} 2022-09-03T06:12:42.184 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000adb8498504e70ed3-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.185 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:42.186 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:42.186 DEBUG:boto:path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:42.187 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:42.188 DEBUG:boto:Method: GET 2022-09-03T06:12:42.188 DEBUG:boto:Path: /dsgvcy-70/?prefix=a 2022-09-03T06:12:42.189 DEBUG:boto:Data: 2022-09-03T06:12:42.189 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.190 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.190 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.191 DEBUG:boto:Params: {} 2022-09-03T06:12:42.192 DEBUG:boto:Token: None 2022-09-03T06:12:42.192 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/ 2022-09-03T06:12:42.193 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU= 2022-09-03T06:12:42.194 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU=', 'Content-Length': '0'} 2022-09-03T06:12:42.196 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b12a994702c26670-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.238 DEBUG:boto:b'dsgvcy-70a1000falsea2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:42.240 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:42.240 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:42.241 DEBUG:boto:path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:42.241 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:42.242 DEBUG:boto:Method: GET 2022-09-03T06:12:42.243 DEBUG:boto:Path: /dsgvcy-70/?prefix=a 2022-09-03T06:12:42.243 DEBUG:boto:Data: 2022-09-03T06:12:42.244 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.244 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.245 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.245 DEBUG:boto:Params: {} 2022-09-03T06:12:42.246 DEBUG:boto:Token: None 2022-09-03T06:12:42.247 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/ 2022-09-03T06:12:42.247 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU= 2022-09-03T06:12:42.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU=', 'Content-Length': '0'} 2022-09-03T06:12:42.251 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009e15669aa5b36780-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.251 DEBUG:boto:b'dsgvcy-70a1000falsea2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:42.252 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:12:42.253 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:42.253 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:42.254 DEBUG:boto:path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.254 DEBUG:boto:auth_path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.255 DEBUG:boto:Method: GET 2022-09-03T06:12:42.256 DEBUG:boto:Path: /dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.256 DEBUG:boto:Data: 2022-09-03T06:12:42.257 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.257 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.258 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.258 DEBUG:boto:Params: {} 2022-09-03T06:12:42.259 DEBUG:boto:Token: None 2022-09-03T06:12:42.260 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.260 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0KfouqIbo9U8xZMETs2VBdolEoI= 2022-09-03T06:12:42.261 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0KfouqIbo9U8xZMETs2VBdolEoI=', 'Content-Length': '0'} 2022-09-03T06:12:42.263 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000904fb0417e3cd700-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.264 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:42.264 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:42.265 DEBUG:boto:path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.265 DEBUG:boto:auth_path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.266 DEBUG:boto:Method: GET 2022-09-03T06:12:42.266 DEBUG:boto:Path: /dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.267 DEBUG:boto:Data: 2022-09-03T06:12:42.267 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.268 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.268 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.269 DEBUG:boto:Params: {} 2022-09-03T06:12:42.270 DEBUG:boto:Token: None 2022-09-03T06:12:42.270 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.271 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0KfouqIbo9U8xZMETs2VBdolEoI= 2022-09-03T06:12:42.271 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0KfouqIbo9U8xZMETs2VBdolEoI=', 'Content-Length': '0'} 2022-09-03T06:12:42.273 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006e1c7daf688edc6b-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.274 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:42.275 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:42.275 DEBUG:boto:path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.276 DEBUG:boto:auth_path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.276 DEBUG:boto:Method: HEAD 2022-09-03T06:12:42.277 DEBUG:boto:Path: /dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.277 DEBUG:boto:Data: 2022-09-03T06:12:42.278 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.278 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.279 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.279 DEBUG:boto:Params: {} 2022-09-03T06:12:42.280 DEBUG:boto:Token: None 2022-09-03T06:12:42.281 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.281 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oeH0yWCR5U7lJwFSjrS281s/xEQ= 2022-09-03T06:12:42.282 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oeH0yWCR5U7lJwFSjrS281s/xEQ=', 'Content-Length': '0'} 2022-09-03T06:12:42.284 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee5a0f767bf5e260-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.285 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:42.286 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:42.286 DEBUG:boto:path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.287 DEBUG:boto:auth_path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.287 DEBUG:boto:Method: HEAD 2022-09-03T06:12:42.288 DEBUG:boto:Path: /dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.288 DEBUG:boto:Data: 2022-09-03T06:12:42.289 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.289 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.290 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.291 DEBUG:boto:Params: {} 2022-09-03T06:12:42.291 DEBUG:boto:Token: None 2022-09-03T06:12:42.292 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.292 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oeH0yWCR5U7lJwFSjrS281s/xEQ= 2022-09-03T06:12:42.293 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oeH0yWCR5U7lJwFSjrS281s/xEQ=', 'Content-Length': '0'} 2022-09-03T06:12:42.295 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000552ce7a819874c86-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.296 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:12:42.297 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:42.298 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:42.300 DEBUG:boto:path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.301 DEBUG:boto:auth_path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.302 DEBUG:boto:Method: GET 2022-09-03T06:12:42.302 DEBUG:boto:Path: /dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.303 DEBUG:boto:Data: 2022-09-03T06:12:42.303 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.305 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.306 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.307 DEBUG:boto:Params: {} 2022-09-03T06:12:42.307 DEBUG:boto:Token: None 2022-09-03T06:12:42.308 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.309 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0KfouqIbo9U8xZMETs2VBdolEoI= 2022-09-03T06:12:42.309 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0KfouqIbo9U8xZMETs2VBdolEoI=', 'Content-Length': '0'} 2022-09-03T06:12:42.312 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000060282dd75b13f005-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.313 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:42.313 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:42.314 DEBUG:boto:path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.314 DEBUG:boto:auth_path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.315 DEBUG:boto:Method: GET 2022-09-03T06:12:42.315 DEBUG:boto:Path: /dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.316 DEBUG:boto:Data: 2022-09-03T06:12:42.317 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.317 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.318 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.319 DEBUG:boto:Params: {} 2022-09-03T06:12:42.319 DEBUG:boto:Token: None 2022-09-03T06:12:42.320 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/b?versionId=null 2022-09-03T06:12:42.321 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0KfouqIbo9U8xZMETs2VBdolEoI= 2022-09-03T06:12:42.322 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0KfouqIbo9U8xZMETs2VBdolEoI=', 'Content-Length': '0'} 2022-09-03T06:12:42.324 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008aa15fc9a6ac267c-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.325 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:42.325 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:42.326 DEBUG:boto:path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:42.326 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:42.327 DEBUG:boto:Method: GET 2022-09-03T06:12:42.327 DEBUG:boto:Path: /dsgvcy-70/?prefix=b 2022-09-03T06:12:42.328 DEBUG:boto:Data: 2022-09-03T06:12:42.329 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.329 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.330 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.330 DEBUG:boto:Params: {} 2022-09-03T06:12:42.331 DEBUG:boto:Token: None 2022-09-03T06:12:42.332 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/ 2022-09-03T06:12:42.332 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU= 2022-09-03T06:12:42.333 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU=', 'Content-Length': '0'} 2022-09-03T06:12:42.335 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cbfc587398380d9c-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.336 DEBUG:boto:b'dsgvcy-70b1000falseb2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:42.337 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:42.337 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:42.338 DEBUG:boto:path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:42.338 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:42.339 DEBUG:boto:Method: GET 2022-09-03T06:12:42.339 DEBUG:boto:Path: /dsgvcy-70/?prefix=b 2022-09-03T06:12:42.340 DEBUG:boto:Data: 2022-09-03T06:12:42.340 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.341 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.342 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.342 DEBUG:boto:Params: {} 2022-09-03T06:12:42.343 DEBUG:boto:Token: None 2022-09-03T06:12:42.343 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/ 2022-09-03T06:12:42.344 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU= 2022-09-03T06:12:42.345 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU=', 'Content-Length': '0'} 2022-09-03T06:12:42.347 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000bb05a899f4ed5e2-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.348 DEBUG:boto:b'dsgvcy-70b1000falseb2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:42.349 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:42.349 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:42.350 DEBUG:boto:Method: HEAD 2022-09-03T06:12:42.351 DEBUG:boto:Path: /dsgvcy-70/b 2022-09-03T06:12:42.351 DEBUG:boto:Data: 2022-09-03T06:12:42.352 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.352 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.353 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.353 DEBUG:boto:Params: {} 2022-09-03T06:12:42.354 DEBUG:boto:Token: None 2022-09-03T06:12:42.355 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/b 2022-09-03T06:12:42.355 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bdslcrcPNWG12vfKpG4QlaJtgZc= 2022-09-03T06:12:42.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bdslcrcPNWG12vfKpG4QlaJtgZc=', 'Content-Length': '0'} 2022-09-03T06:12:42.358 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f26603cfc57454e6-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.359 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:42.360 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:42.360 DEBUG:boto:Method: HEAD 2022-09-03T06:12:42.361 DEBUG:boto:Path: /dsgvcy-70/b 2022-09-03T06:12:42.361 DEBUG:boto:Data: 2022-09-03T06:12:42.362 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.363 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.364 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.365 DEBUG:boto:Params: {} 2022-09-03T06:12:42.365 DEBUG:boto:Token: None 2022-09-03T06:12:42.366 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/b 2022-09-03T06:12:42.367 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bdslcrcPNWG12vfKpG4QlaJtgZc= 2022-09-03T06:12:42.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bdslcrcPNWG12vfKpG4QlaJtgZc=', 'Content-Length': '0'} 2022-09-03T06:12:42.369 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b26e647f91590f91-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.370 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:12:42.370 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:42.371 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:42.372 DEBUG:boto:Method: GET 2022-09-03T06:12:42.372 DEBUG:boto:Path: /dsgvcy-70/b 2022-09-03T06:12:42.373 DEBUG:boto:Data: 2022-09-03T06:12:42.373 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.374 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.374 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.375 DEBUG:boto:Params: {} 2022-09-03T06:12:42.375 DEBUG:boto:Token: None 2022-09-03T06:12:42.376 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/b 2022-09-03T06:12:42.377 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AjWoI618q1OFJ1gvQXWzSayGep0= 2022-09-03T06:12:42.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AjWoI618q1OFJ1gvQXWzSayGep0=', 'Content-Length': '0'} 2022-09-03T06:12:42.380 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000908b0573ebfc4f56-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.422 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:42.423 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:42.424 DEBUG:boto:Method: GET 2022-09-03T06:12:42.424 DEBUG:boto:Path: /dsgvcy-70/b 2022-09-03T06:12:42.425 DEBUG:boto:Data: 2022-09-03T06:12:42.426 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.426 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.427 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.427 DEBUG:boto:Params: {} 2022-09-03T06:12:42.428 DEBUG:boto:Token: None 2022-09-03T06:12:42.429 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/b 2022-09-03T06:12:42.429 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AjWoI618q1OFJ1gvQXWzSayGep0= 2022-09-03T06:12:42.430 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AjWoI618q1OFJ1gvQXWzSayGep0=', 'Content-Length': '0'} 2022-09-03T06:12:42.432 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006fe8dcbe0246c3f4-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.433 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:42.434 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:42.434 DEBUG:boto:path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:42.435 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:42.436 DEBUG:boto:Method: GET 2022-09-03T06:12:42.436 DEBUG:boto:Path: /dsgvcy-70/?prefix=b 2022-09-03T06:12:42.437 DEBUG:boto:Data: 2022-09-03T06:12:42.437 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.438 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.439 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.439 DEBUG:boto:Params: {} 2022-09-03T06:12:42.440 DEBUG:boto:Token: None 2022-09-03T06:12:42.441 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/ 2022-09-03T06:12:42.441 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU= 2022-09-03T06:12:42.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU=', 'Content-Length': '0'} 2022-09-03T06:12:42.444 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b6c592d56f427ae1-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.445 DEBUG:boto:b'dsgvcy-70b1000falseb2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:42.446 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:42.447 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:42.447 DEBUG:boto:path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:42.448 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:42.448 DEBUG:boto:Method: GET 2022-09-03T06:12:42.449 DEBUG:boto:Path: /dsgvcy-70/?prefix=b 2022-09-03T06:12:42.450 DEBUG:boto:Data: 2022-09-03T06:12:42.450 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.451 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.451 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.452 DEBUG:boto:Params: {} 2022-09-03T06:12:42.452 DEBUG:boto:Token: None 2022-09-03T06:12:42.453 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/ 2022-09-03T06:12:42.454 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU= 2022-09-03T06:12:42.454 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU=', 'Content-Length': '0'} 2022-09-03T06:12:42.457 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e7ca654e125bf270-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.457 DEBUG:boto:b'dsgvcy-70b1000falseb2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:42.458 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:12:42.459 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:42.459 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:42.460 DEBUG:boto:path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.461 DEBUG:boto:auth_path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.461 DEBUG:boto:Method: GET 2022-09-03T06:12:42.463 DEBUG:boto:Path: /dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.464 DEBUG:boto:Data: 2022-09-03T06:12:42.465 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.466 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.467 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.468 DEBUG:boto:Params: {} 2022-09-03T06:12:42.469 DEBUG:boto:Token: None 2022-09-03T06:12:42.470 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.471 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UQnVWyyxJ0FelwpnoxSmDIZ5icU= 2022-09-03T06:12:42.472 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UQnVWyyxJ0FelwpnoxSmDIZ5icU=', 'Content-Length': '0'} 2022-09-03T06:12:42.476 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004ffb4e27b7a4268d-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.477 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:42.477 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:42.478 DEBUG:boto:path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.479 DEBUG:boto:auth_path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.479 DEBUG:boto:Method: GET 2022-09-03T06:12:42.480 DEBUG:boto:Path: /dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.481 DEBUG:boto:Data: 2022-09-03T06:12:42.482 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.483 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.484 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.485 DEBUG:boto:Params: {} 2022-09-03T06:12:42.486 DEBUG:boto:Token: None 2022-09-03T06:12:42.487 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.488 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UQnVWyyxJ0FelwpnoxSmDIZ5icU= 2022-09-03T06:12:42.489 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UQnVWyyxJ0FelwpnoxSmDIZ5icU=', 'Content-Length': '0'} 2022-09-03T06:12:42.492 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ed7b8ff4e0699a4-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.493 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:42.494 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:42.495 DEBUG:boto:path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.496 DEBUG:boto:auth_path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.497 DEBUG:boto:Method: HEAD 2022-09-03T06:12:42.498 DEBUG:boto:Path: /dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.499 DEBUG:boto:Data: 2022-09-03T06:12:42.500 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.501 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.502 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.503 DEBUG:boto:Params: {} 2022-09-03T06:12:42.503 DEBUG:boto:Token: None 2022-09-03T06:12:42.504 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.505 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/sMXNmwkrkXFNOEQPh/AEfsB2h0= 2022-09-03T06:12:42.505 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/sMXNmwkrkXFNOEQPh/AEfsB2h0=', 'Content-Length': '0'} 2022-09-03T06:12:42.508 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006544aaab6930868b-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.508 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:42.509 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:42.510 DEBUG:boto:path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.510 DEBUG:boto:auth_path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.511 DEBUG:boto:Method: HEAD 2022-09-03T06:12:42.511 DEBUG:boto:Path: /dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.512 DEBUG:boto:Data: 2022-09-03T06:12:42.513 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.514 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.514 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.515 DEBUG:boto:Params: {} 2022-09-03T06:12:42.516 DEBUG:boto:Token: None 2022-09-03T06:12:42.516 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.517 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/sMXNmwkrkXFNOEQPh/AEfsB2h0= 2022-09-03T06:12:42.518 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/sMXNmwkrkXFNOEQPh/AEfsB2h0=', 'Content-Length': '0'} 2022-09-03T06:12:42.520 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a66e35a4815166f4-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.521 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:12:42.521 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:42.522 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:42.522 DEBUG:boto:path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.523 DEBUG:boto:auth_path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.524 DEBUG:boto:Method: GET 2022-09-03T06:12:42.524 DEBUG:boto:Path: /dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.525 DEBUG:boto:Data: 2022-09-03T06:12:42.525 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.526 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.527 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.527 DEBUG:boto:Params: {} 2022-09-03T06:12:42.528 DEBUG:boto:Token: None 2022-09-03T06:12:42.529 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.529 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UQnVWyyxJ0FelwpnoxSmDIZ5icU= 2022-09-03T06:12:42.530 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UQnVWyyxJ0FelwpnoxSmDIZ5icU=', 'Content-Length': '0'} 2022-09-03T06:12:42.532 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c572d88bd80555d7-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.534 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:42.534 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:42.535 DEBUG:boto:path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.536 DEBUG:boto:auth_path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.538 DEBUG:boto:Method: GET 2022-09-03T06:12:42.538 DEBUG:boto:Path: /dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.539 DEBUG:boto:Data: 2022-09-03T06:12:42.540 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.540 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.541 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.541 DEBUG:boto:Params: {} 2022-09-03T06:12:42.542 DEBUG:boto:Token: None 2022-09-03T06:12:42.543 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/c?versionId=null 2022-09-03T06:12:42.544 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UQnVWyyxJ0FelwpnoxSmDIZ5icU= 2022-09-03T06:12:42.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UQnVWyyxJ0FelwpnoxSmDIZ5icU=', 'Content-Length': '0'} 2022-09-03T06:12:42.547 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d318c549b2210cc3-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.548 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:42.548 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:42.549 DEBUG:boto:path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:42.550 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:42.550 DEBUG:boto:Method: GET 2022-09-03T06:12:42.551 DEBUG:boto:Path: /dsgvcy-70/?prefix=c 2022-09-03T06:12:42.551 DEBUG:boto:Data: 2022-09-03T06:12:42.552 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.552 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.553 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.554 DEBUG:boto:Params: {} 2022-09-03T06:12:42.554 DEBUG:boto:Token: None 2022-09-03T06:12:42.555 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/ 2022-09-03T06:12:42.556 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU= 2022-09-03T06:12:42.556 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU=', 'Content-Length': '0'} 2022-09-03T06:12:42.558 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bee0fc1f4e0fd096-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.559 DEBUG:boto:b'dsgvcy-70c1000falsec2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:42.560 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:42.561 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:42.561 DEBUG:boto:path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:42.562 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:42.562 DEBUG:boto:Method: GET 2022-09-03T06:12:42.563 DEBUG:boto:Path: /dsgvcy-70/?prefix=c 2022-09-03T06:12:42.563 DEBUG:boto:Data: 2022-09-03T06:12:42.564 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.565 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.565 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.566 DEBUG:boto:Params: {} 2022-09-03T06:12:42.566 DEBUG:boto:Token: None 2022-09-03T06:12:42.567 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/ 2022-09-03T06:12:42.568 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU= 2022-09-03T06:12:42.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU=', 'Content-Length': '0'} 2022-09-03T06:12:42.571 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007664269f78b88a49-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.571 DEBUG:boto:b'dsgvcy-70c1000falsec2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:42.574 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:42.575 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:42.575 DEBUG:boto:Method: HEAD 2022-09-03T06:12:42.576 DEBUG:boto:Path: /dsgvcy-70/c 2022-09-03T06:12:42.576 DEBUG:boto:Data: 2022-09-03T06:12:42.577 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.577 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.578 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.578 DEBUG:boto:Params: {} 2022-09-03T06:12:42.579 DEBUG:boto:Token: None 2022-09-03T06:12:42.580 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/c 2022-09-03T06:12:42.580 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oFdasY6C76duT5N03xEQ6P9vi8I= 2022-09-03T06:12:42.581 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oFdasY6C76duT5N03xEQ6P9vi8I=', 'Content-Length': '0'} 2022-09-03T06:12:42.586 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000318fb8d1df1f98d6-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.587 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:42.588 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:42.588 DEBUG:boto:Method: HEAD 2022-09-03T06:12:42.589 DEBUG:boto:Path: /dsgvcy-70/c 2022-09-03T06:12:42.589 DEBUG:boto:Data: 2022-09-03T06:12:42.590 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.590 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.591 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.591 DEBUG:boto:Params: {} 2022-09-03T06:12:42.592 DEBUG:boto:Token: None 2022-09-03T06:12:42.593 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/c 2022-09-03T06:12:42.593 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oFdasY6C76duT5N03xEQ6P9vi8I= 2022-09-03T06:12:42.594 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oFdasY6C76duT5N03xEQ6P9vi8I=', 'Content-Length': '0'} 2022-09-03T06:12:42.596 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000063c23be73ace3457-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.597 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:12:42.598 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:42.598 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:42.599 DEBUG:boto:Method: GET 2022-09-03T06:12:42.599 DEBUG:boto:Path: /dsgvcy-70/c 2022-09-03T06:12:42.600 DEBUG:boto:Data: 2022-09-03T06:12:42.600 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.601 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.601 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.602 DEBUG:boto:Params: {} 2022-09-03T06:12:42.603 DEBUG:boto:Token: None 2022-09-03T06:12:42.603 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/c 2022-09-03T06:12:42.604 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0GfjZ+nG7JEdzcidLvYLD5G0xZo= 2022-09-03T06:12:42.605 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0GfjZ+nG7JEdzcidLvYLD5G0xZo=', 'Content-Length': '0'} 2022-09-03T06:12:42.606 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f796f66051ba697e-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.650 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:42.651 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:42.652 DEBUG:boto:Method: GET 2022-09-03T06:12:42.653 DEBUG:boto:Path: /dsgvcy-70/c 2022-09-03T06:12:42.653 DEBUG:boto:Data: 2022-09-03T06:12:42.655 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.656 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.657 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.658 DEBUG:boto:Params: {} 2022-09-03T06:12:42.659 DEBUG:boto:Token: None 2022-09-03T06:12:42.660 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/c 2022-09-03T06:12:42.662 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0GfjZ+nG7JEdzcidLvYLD5G0xZo= 2022-09-03T06:12:42.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0GfjZ+nG7JEdzcidLvYLD5G0xZo=', 'Content-Length': '0'} 2022-09-03T06:12:42.665 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000070e8df33cd89dd17-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.667 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:42.668 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:42.668 DEBUG:boto:path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:42.669 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:42.670 DEBUG:boto:Method: GET 2022-09-03T06:12:42.671 DEBUG:boto:Path: /dsgvcy-70/?prefix=c 2022-09-03T06:12:42.672 DEBUG:boto:Data: 2022-09-03T06:12:42.673 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.674 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.675 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.676 DEBUG:boto:Params: {} 2022-09-03T06:12:42.677 DEBUG:boto:Token: None 2022-09-03T06:12:42.678 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/ 2022-09-03T06:12:42.680 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU= 2022-09-03T06:12:42.681 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU=', 'Content-Length': '0'} 2022-09-03T06:12:42.684 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c4234a284129413c-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.686 DEBUG:boto:b'dsgvcy-70c1000falsec2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:42.687 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:42.688 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:42.688 DEBUG:boto:path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:42.689 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:42.690 DEBUG:boto:Method: GET 2022-09-03T06:12:42.690 DEBUG:boto:Path: /dsgvcy-70/?prefix=c 2022-09-03T06:12:42.691 DEBUG:boto:Data: 2022-09-03T06:12:42.691 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.692 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.693 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.693 DEBUG:boto:Params: {} 2022-09-03T06:12:42.694 DEBUG:boto:Token: None 2022-09-03T06:12:42.695 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/ 2022-09-03T06:12:42.695 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU= 2022-09-03T06:12:42.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU=', 'Content-Length': '0'} 2022-09-03T06:12:42.699 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a97cc6d5d7d37540-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.700 DEBUG:boto:b'dsgvcy-70c1000falsec2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:42.700 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:12:42.701 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:42.702 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:42.702 DEBUG:boto:path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.703 DEBUG:boto:auth_path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.703 DEBUG:boto:Method: GET 2022-09-03T06:12:42.704 DEBUG:boto:Path: /dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.704 DEBUG:boto:Data: 2022-09-03T06:12:42.705 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.706 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.706 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.707 DEBUG:boto:Params: {} 2022-09-03T06:12:42.707 DEBUG:boto:Token: None 2022-09-03T06:12:42.708 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.709 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gBEpU43D2LfB5geI1ZYE5BUS6gM= 2022-09-03T06:12:42.709 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gBEpU43D2LfB5geI1ZYE5BUS6gM=', 'Content-Length': '0'} 2022-09-03T06:12:42.711 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000625ebb2f692cd5ba-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.713 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:42.714 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:42.715 DEBUG:boto:path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.716 DEBUG:boto:auth_path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.717 DEBUG:boto:Method: GET 2022-09-03T06:12:42.718 DEBUG:boto:Path: /dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.719 DEBUG:boto:Data: 2022-09-03T06:12:42.720 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.721 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.722 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.723 DEBUG:boto:Params: {} 2022-09-03T06:12:42.724 DEBUG:boto:Token: None 2022-09-03T06:12:42.725 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.726 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gBEpU43D2LfB5geI1ZYE5BUS6gM= 2022-09-03T06:12:42.727 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gBEpU43D2LfB5geI1ZYE5BUS6gM=', 'Content-Length': '0'} 2022-09-03T06:12:42.730 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b630c1001cdd9211-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.778 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:42.780 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:42.781 DEBUG:boto:path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.781 DEBUG:boto:auth_path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.782 DEBUG:boto:Method: HEAD 2022-09-03T06:12:42.782 DEBUG:boto:Path: /dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.783 DEBUG:boto:Data: 2022-09-03T06:12:42.783 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.784 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.784 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.785 DEBUG:boto:Params: {} 2022-09-03T06:12:42.786 DEBUG:boto:Token: None 2022-09-03T06:12:42.787 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.788 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aFiWtfDfl1LBp54kQnQ5+G0h72g= 2022-09-03T06:12:42.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aFiWtfDfl1LBp54kQnQ5+G0h72g=', 'Content-Length': '0'} 2022-09-03T06:12:42.791 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008ad362944c8e94b8-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.793 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:42.794 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:42.797 DEBUG:boto:path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.798 DEBUG:boto:auth_path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.798 DEBUG:boto:Method: HEAD 2022-09-03T06:12:42.799 DEBUG:boto:Path: /dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.800 DEBUG:boto:Data: 2022-09-03T06:12:42.801 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.802 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.803 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.804 DEBUG:boto:Params: {} 2022-09-03T06:12:42.805 DEBUG:boto:Token: None 2022-09-03T06:12:42.807 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.808 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aFiWtfDfl1LBp54kQnQ5+G0h72g= 2022-09-03T06:12:42.809 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aFiWtfDfl1LBp54kQnQ5+G0h72g=', 'Content-Length': '0'} 2022-09-03T06:12:42.811 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a90441e508f5f5c0-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.812 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:12:42.813 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:42.813 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:42.814 DEBUG:boto:path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.814 DEBUG:boto:auth_path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.815 DEBUG:boto:Method: GET 2022-09-03T06:12:42.816 DEBUG:boto:Path: /dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.816 DEBUG:boto:Data: 2022-09-03T06:12:42.817 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.818 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.818 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.819 DEBUG:boto:Params: {} 2022-09-03T06:12:42.820 DEBUG:boto:Token: None 2022-09-03T06:12:42.820 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.821 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gBEpU43D2LfB5geI1ZYE5BUS6gM= 2022-09-03T06:12:42.822 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gBEpU43D2LfB5geI1ZYE5BUS6gM=', 'Content-Length': '0'} 2022-09-03T06:12:42.825 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d9b356eab354d0b9-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.826 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:42.826 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:42.827 DEBUG:boto:path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.828 DEBUG:boto:auth_path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.828 DEBUG:boto:Method: GET 2022-09-03T06:12:42.829 DEBUG:boto:Path: /dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.830 DEBUG:boto:Data: 2022-09-03T06:12:42.830 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.831 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.831 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.832 DEBUG:boto:Params: {} 2022-09-03T06:12:42.833 DEBUG:boto:Token: None 2022-09-03T06:12:42.834 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/d?versionId=null 2022-09-03T06:12:42.834 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gBEpU43D2LfB5geI1ZYE5BUS6gM= 2022-09-03T06:12:42.835 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gBEpU43D2LfB5geI1ZYE5BUS6gM=', 'Content-Length': '0'} 2022-09-03T06:12:42.837 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096c73b2608ddf12f-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.839 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:42.839 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:42.840 DEBUG:boto:path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:42.841 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:42.842 DEBUG:boto:Method: GET 2022-09-03T06:12:42.844 DEBUG:boto:Path: /dsgvcy-70/?prefix=d 2022-09-03T06:12:42.845 DEBUG:boto:Data: 2022-09-03T06:12:42.846 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.847 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.848 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.849 DEBUG:boto:Params: {} 2022-09-03T06:12:42.850 DEBUG:boto:Token: None 2022-09-03T06:12:42.851 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/ 2022-09-03T06:12:42.852 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU= 2022-09-03T06:12:42.854 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU=', 'Content-Length': '0'} 2022-09-03T06:12:42.860 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000058675cb972551b65-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.861 DEBUG:boto:b'dsgvcy-70d1000falsed2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:42.862 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:42.862 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:42.863 DEBUG:boto:path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:42.864 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:42.864 DEBUG:boto:Method: GET 2022-09-03T06:12:42.865 DEBUG:boto:Path: /dsgvcy-70/?prefix=d 2022-09-03T06:12:42.865 DEBUG:boto:Data: 2022-09-03T06:12:42.866 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.866 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.867 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.868 DEBUG:boto:Params: {} 2022-09-03T06:12:42.868 DEBUG:boto:Token: None 2022-09-03T06:12:42.869 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/ 2022-09-03T06:12:42.869 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU= 2022-09-03T06:12:42.870 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU=', 'Content-Length': '0'} 2022-09-03T06:12:42.873 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eec405e353b0e54f-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.873 DEBUG:boto:b'dsgvcy-70d1000falsed2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:42.875 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:42.875 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:42.876 DEBUG:boto:Method: HEAD 2022-09-03T06:12:42.876 DEBUG:boto:Path: /dsgvcy-70/d 2022-09-03T06:12:42.877 DEBUG:boto:Data: 2022-09-03T06:12:42.877 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.879 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.879 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.880 DEBUG:boto:Params: {} 2022-09-03T06:12:42.881 DEBUG:boto:Token: None 2022-09-03T06:12:42.881 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/d 2022-09-03T06:12:42.882 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o7IjsALd6+rgaEkOCLx32NHHiDQ= 2022-09-03T06:12:42.883 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o7IjsALd6+rgaEkOCLx32NHHiDQ=', 'Content-Length': '0'} 2022-09-03T06:12:42.886 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ca06065d7b06e7f-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.887 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:42.887 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:42.888 DEBUG:boto:Method: HEAD 2022-09-03T06:12:42.889 DEBUG:boto:Path: /dsgvcy-70/d 2022-09-03T06:12:42.889 DEBUG:boto:Data: 2022-09-03T06:12:42.890 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.890 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.891 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.892 DEBUG:boto:Params: {} 2022-09-03T06:12:42.894 DEBUG:boto:Token: None 2022-09-03T06:12:42.895 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/d 2022-09-03T06:12:42.896 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o7IjsALd6+rgaEkOCLx32NHHiDQ= 2022-09-03T06:12:42.897 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o7IjsALd6+rgaEkOCLx32NHHiDQ=', 'Content-Length': '0'} 2022-09-03T06:12:42.900 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009b79cfebc69e493d-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.901 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:12:42.903 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:42.904 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:42.905 DEBUG:boto:Method: GET 2022-09-03T06:12:42.906 DEBUG:boto:Path: /dsgvcy-70/d 2022-09-03T06:12:42.907 DEBUG:boto:Data: 2022-09-03T06:12:42.908 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.909 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.909 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.910 DEBUG:boto:Params: {} 2022-09-03T06:12:42.911 DEBUG:boto:Token: None 2022-09-03T06:12:42.911 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/d 2022-09-03T06:12:42.912 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7uMiWAA9T2bCPohl/CFIaGCQ8eg= 2022-09-03T06:12:42.913 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7uMiWAA9T2bCPohl/CFIaGCQ8eg=', 'Content-Length': '0'} 2022-09-03T06:12:42.916 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b1780bf03ad2483f-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.958 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:42.959 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:42.960 DEBUG:boto:Method: GET 2022-09-03T06:12:42.961 DEBUG:boto:Path: /dsgvcy-70/d 2022-09-03T06:12:42.961 DEBUG:boto:Data: 2022-09-03T06:12:42.962 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.963 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:42.964 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.964 DEBUG:boto:Params: {} 2022-09-03T06:12:42.965 DEBUG:boto:Token: None 2022-09-03T06:12:42.967 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/d 2022-09-03T06:12:42.967 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7uMiWAA9T2bCPohl/CFIaGCQ8eg= 2022-09-03T06:12:42.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7uMiWAA9T2bCPohl/CFIaGCQ8eg=', 'Content-Length': '0'} 2022-09-03T06:12:42.973 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000034814e3ad8698735-006312f05a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.974 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:42.975 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:42.976 DEBUG:boto:path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:42.978 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:42.978 DEBUG:boto:Method: GET 2022-09-03T06:12:42.979 DEBUG:boto:Path: /dsgvcy-70/?prefix=d 2022-09-03T06:12:42.980 DEBUG:boto:Data: 2022-09-03T06:12:42.980 DEBUG:boto:Headers: {} 2022-09-03T06:12:42.981 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:42.982 DEBUG:boto:Port: 8000 2022-09-03T06:12:42.983 DEBUG:boto:Params: {} 2022-09-03T06:12:42.984 DEBUG:boto:Token: None 2022-09-03T06:12:42.985 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:42 GMT /dsgvcy-70/ 2022-09-03T06:12:42.986 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU= 2022-09-03T06:12:42.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yJ/qLLWOihEGsHlS2Gb1t3YsCuU=', 'Content-Length': '0'} 2022-09-03T06:12:42.991 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000020cf70c8a2add868-006312f05a-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:42.993 DEBUG:boto:b'dsgvcy-70d1000falsed2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:42.994 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:42.995 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:42.996 DEBUG:boto:path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:42.997 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:42.999 DEBUG:boto:Method: GET 2022-09-03T06:12:43.000 DEBUG:boto:Path: /dsgvcy-70/?prefix=d 2022-09-03T06:12:43.001 DEBUG:boto:Data: 2022-09-03T06:12:43.002 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.003 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:43.003 DEBUG:boto:Port: 8000 2022-09-03T06:12:43.004 DEBUG:boto:Params: {} 2022-09-03T06:12:43.005 DEBUG:boto:Token: None 2022-09-03T06:12:43.007 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/ 2022-09-03T06:12:43.008 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jZclo+PtPXLiUC7j2stP4qVlhco= 2022-09-03T06:12:43.009 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jZclo+PtPXLiUC7j2stP4qVlhco=', 'Content-Length': '0'} 2022-09-03T06:12:43.012 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c161e2907d3ec61e-006312f05b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.056 DEBUG:boto:b'dsgvcy-70d1000falsed2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:43.057 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-70 zones={test-zone2, test-zone1} 2022-09-03T06:12:43.059 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-70 zones={test-zone3, test-zone1} 2022-09-03T06:12:43.059 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:43.060 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:43.061 DEBUG:boto:Method: HEAD 2022-09-03T06:12:43.062 DEBUG:boto:Path: /dsgvcy-70/ 2022-09-03T06:12:43.063 DEBUG:boto:Data: 2022-09-03T06:12:43.064 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.065 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:43.066 DEBUG:boto:Port: 8001 2022-09-03T06:12:43.067 DEBUG:boto:Params: {} 2022-09-03T06:12:43.068 DEBUG:boto:Token: None 2022-09-03T06:12:43.070 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/ 2022-09-03T06:12:43.072 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+jdA43nrUjSYORz6nkjyMbqPX5c= 2022-09-03T06:12:43.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+jdA43nrUjSYORz6nkjyMbqPX5c=', 'Content-Length': '0'} 2022-09-03T06:12:43.076 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005e5dec6ed7248c7a-006312f05b-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.078 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:43.079 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:43.080 DEBUG:boto:Method: HEAD 2022-09-03T06:12:43.081 DEBUG:boto:Path: /dsgvcy-70/ 2022-09-03T06:12:43.082 DEBUG:boto:Data: 2022-09-03T06:12:43.082 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.083 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:43.084 DEBUG:boto:Port: 8000 2022-09-03T06:12:43.085 DEBUG:boto:Params: {} 2022-09-03T06:12:43.087 DEBUG:boto:Token: None 2022-09-03T06:12:43.088 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/ 2022-09-03T06:12:43.089 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+jdA43nrUjSYORz6nkjyMbqPX5c= 2022-09-03T06:12:43.090 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+jdA43nrUjSYORz6nkjyMbqPX5c=', 'Content-Length': '0'} 2022-09-03T06:12:43.101 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002cfcb478516685c6-006312f05b-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.103 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:12:43.105 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:43.107 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:43.107 DEBUG:boto:path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:43.108 DEBUG:boto:auth_path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:43.109 DEBUG:boto:Method: GET 2022-09-03T06:12:43.110 DEBUG:boto:Path: /dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:43.111 DEBUG:boto:Data: 2022-09-03T06:12:43.112 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.113 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:43.114 DEBUG:boto:Port: 8001 2022-09-03T06:12:43.115 DEBUG:boto:Params: {} 2022-09-03T06:12:43.116 DEBUG:boto:Token: None 2022-09-03T06:12:43.117 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/?versions 2022-09-03T06:12:43.118 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qkUggXRPJ5B+Icqh20HWegRjCI8= 2022-09-03T06:12:43.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qkUggXRPJ5B+Icqh20HWegRjCI8=', 'Content-Length': '0'} 2022-09-03T06:12:43.124 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000022b228fa32786d26-006312f05b-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.125 DEBUG:boto:b'dsgvcy-70999falseanulltrue2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:43.126 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:12:43.127 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:12:43.128 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:12:43.129 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:12:43.129 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:12:43.130 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:43.131 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:43.132 DEBUG:boto:path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:43.133 DEBUG:boto:auth_path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:43.133 DEBUG:boto:Method: GET 2022-09-03T06:12:43.135 DEBUG:boto:Path: /dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:43.136 DEBUG:boto:Data: 2022-09-03T06:12:43.136 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.137 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:43.138 DEBUG:boto:Port: 8000 2022-09-03T06:12:43.138 DEBUG:boto:Params: {} 2022-09-03T06:12:43.139 DEBUG:boto:Token: None 2022-09-03T06:12:43.139 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/?versions 2022-09-03T06:12:43.140 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qkUggXRPJ5B+Icqh20HWegRjCI8= 2022-09-03T06:12:43.141 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qkUggXRPJ5B+Icqh20HWegRjCI8=', 'Content-Length': '0'} 2022-09-03T06:12:43.143 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c196c5931cb2ac8d-006312f05b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.145 DEBUG:boto:b'dsgvcy-70999falseanulltrue2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:43.146 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:12:43.147 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:12:43.147 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:12:43.148 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:12:43.149 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:43.150 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:43.151 DEBUG:boto:path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:43.152 DEBUG:boto:auth_path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:43.153 DEBUG:boto:Method: GET 2022-09-03T06:12:43.154 DEBUG:boto:Path: /dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:43.156 DEBUG:boto:Data: 2022-09-03T06:12:43.157 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.158 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:43.159 DEBUG:boto:Port: 8001 2022-09-03T06:12:43.160 DEBUG:boto:Params: {} 2022-09-03T06:12:43.161 DEBUG:boto:Token: None 2022-09-03T06:12:43.162 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/?versions 2022-09-03T06:12:43.163 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qkUggXRPJ5B+Icqh20HWegRjCI8= 2022-09-03T06:12:43.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qkUggXRPJ5B+Icqh20HWegRjCI8=', 'Content-Length': '0'} 2022-09-03T06:12:43.168 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e9928c50022d40d8-006312f05b-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.169 DEBUG:boto:b'dsgvcy-70999falseanulltrue2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:43.172 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:43.174 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:43.175 DEBUG:boto:path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:43.176 DEBUG:boto:auth_path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:43.177 DEBUG:boto:Method: GET 2022-09-03T06:12:43.178 DEBUG:boto:Path: /dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:43.179 DEBUG:boto:Data: 2022-09-03T06:12:43.180 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.180 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:43.181 DEBUG:boto:Port: 8000 2022-09-03T06:12:43.182 DEBUG:boto:Params: {} 2022-09-03T06:12:43.183 DEBUG:boto:Token: None 2022-09-03T06:12:43.184 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/?versions 2022-09-03T06:12:43.185 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qkUggXRPJ5B+Icqh20HWegRjCI8= 2022-09-03T06:12:43.186 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qkUggXRPJ5B+Icqh20HWegRjCI8=', 'Content-Length': '0'} 2022-09-03T06:12:43.189 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002766d62683504657-006312f05b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.191 DEBUG:boto:b'dsgvcy-70999falseanulltrue2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:43.192 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:12:43.193 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:43.194 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:43.195 DEBUG:boto:path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.196 DEBUG:boto:auth_path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.197 DEBUG:boto:Method: GET 2022-09-03T06:12:43.198 DEBUG:boto:Path: /dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.198 DEBUG:boto:Data: 2022-09-03T06:12:43.199 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.200 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:43.201 DEBUG:boto:Port: 8001 2022-09-03T06:12:43.202 DEBUG:boto:Params: {} 2022-09-03T06:12:43.204 DEBUG:boto:Token: None 2022-09-03T06:12:43.205 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.206 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:diGu71PC0rUl86GyeoaEq/N0Rjc= 2022-09-03T06:12:43.207 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:diGu71PC0rUl86GyeoaEq/N0Rjc=', 'Content-Length': '0'} 2022-09-03T06:12:43.211 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc99189d1bdd28cb-006312f05b-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.212 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:43.213 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:43.214 DEBUG:boto:path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.215 DEBUG:boto:auth_path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.216 DEBUG:boto:Method: GET 2022-09-03T06:12:43.217 DEBUG:boto:Path: /dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.218 DEBUG:boto:Data: 2022-09-03T06:12:43.219 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.220 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:43.221 DEBUG:boto:Port: 8000 2022-09-03T06:12:43.222 DEBUG:boto:Params: {} 2022-09-03T06:12:43.223 DEBUG:boto:Token: None 2022-09-03T06:12:43.224 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.224 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:diGu71PC0rUl86GyeoaEq/N0Rjc= 2022-09-03T06:12:43.225 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:diGu71PC0rUl86GyeoaEq/N0Rjc=', 'Content-Length': '0'} 2022-09-03T06:12:43.229 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008108feb666f31281-006312f05b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.230 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:43.232 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:43.233 DEBUG:boto:path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.234 DEBUG:boto:auth_path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.235 DEBUG:boto:Method: HEAD 2022-09-03T06:12:43.236 DEBUG:boto:Path: /dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.237 DEBUG:boto:Data: 2022-09-03T06:12:43.237 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.239 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:43.239 DEBUG:boto:Port: 8001 2022-09-03T06:12:43.240 DEBUG:boto:Params: {} 2022-09-03T06:12:43.242 DEBUG:boto:Token: None 2022-09-03T06:12:43.243 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.244 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IwijHrz9sMyzG7b0mF4pwBVPu7k= 2022-09-03T06:12:43.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IwijHrz9sMyzG7b0mF4pwBVPu7k=', 'Content-Length': '0'} 2022-09-03T06:12:43.249 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db11bda76730fa1d-006312f05b-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.250 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:43.251 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:43.252 DEBUG:boto:path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.253 DEBUG:boto:auth_path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.254 DEBUG:boto:Method: HEAD 2022-09-03T06:12:43.255 DEBUG:boto:Path: /dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.256 DEBUG:boto:Data: 2022-09-03T06:12:43.257 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.258 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:43.259 DEBUG:boto:Port: 8000 2022-09-03T06:12:43.260 DEBUG:boto:Params: {} 2022-09-03T06:12:43.261 DEBUG:boto:Token: None 2022-09-03T06:12:43.262 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.263 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IwijHrz9sMyzG7b0mF4pwBVPu7k= 2022-09-03T06:12:43.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IwijHrz9sMyzG7b0mF4pwBVPu7k=', 'Content-Length': '0'} 2022-09-03T06:12:43.267 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aa8f5bd5a971885a-006312f05b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.269 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:12:43.270 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:43.271 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:43.272 DEBUG:boto:path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.273 DEBUG:boto:auth_path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.274 DEBUG:boto:Method: GET 2022-09-03T06:12:43.275 DEBUG:boto:Path: /dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.276 DEBUG:boto:Data: 2022-09-03T06:12:43.277 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.278 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:43.279 DEBUG:boto:Port: 8001 2022-09-03T06:12:43.280 DEBUG:boto:Params: {} 2022-09-03T06:12:43.282 DEBUG:boto:Token: None 2022-09-03T06:12:43.283 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.284 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:diGu71PC0rUl86GyeoaEq/N0Rjc= 2022-09-03T06:12:43.285 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:diGu71PC0rUl86GyeoaEq/N0Rjc=', 'Content-Length': '0'} 2022-09-03T06:12:43.288 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003079dd36df6dcbeb-006312f05b-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.289 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:43.291 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:43.292 DEBUG:boto:path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.293 DEBUG:boto:auth_path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.294 DEBUG:boto:Method: GET 2022-09-03T06:12:43.295 DEBUG:boto:Path: /dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.296 DEBUG:boto:Data: 2022-09-03T06:12:43.297 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.298 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:43.299 DEBUG:boto:Port: 8000 2022-09-03T06:12:43.300 DEBUG:boto:Params: {} 2022-09-03T06:12:43.302 DEBUG:boto:Token: None 2022-09-03T06:12:43.303 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/a?versionId=null 2022-09-03T06:12:43.304 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:diGu71PC0rUl86GyeoaEq/N0Rjc= 2022-09-03T06:12:43.306 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:diGu71PC0rUl86GyeoaEq/N0Rjc=', 'Content-Length': '0'} 2022-09-03T06:12:43.308 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c00510fd992bf73a-006312f05b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.310 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:43.311 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:43.311 DEBUG:boto:path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:43.312 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:43.313 DEBUG:boto:Method: GET 2022-09-03T06:12:43.314 DEBUG:boto:Path: /dsgvcy-70/?prefix=a 2022-09-03T06:12:43.315 DEBUG:boto:Data: 2022-09-03T06:12:43.316 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.316 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:43.317 DEBUG:boto:Port: 8001 2022-09-03T06:12:43.317 DEBUG:boto:Params: {} 2022-09-03T06:12:43.318 DEBUG:boto:Token: None 2022-09-03T06:12:43.319 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/ 2022-09-03T06:12:43.320 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jZclo+PtPXLiUC7j2stP4qVlhco= 2022-09-03T06:12:43.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jZclo+PtPXLiUC7j2stP4qVlhco=', 'Content-Length': '0'} 2022-09-03T06:12:43.323 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000068dd5fc197ff663d-006312f05b-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.323 DEBUG:boto:b'dsgvcy-70a1000falsea2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:43.325 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:43.326 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:43.327 DEBUG:boto:path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:43.328 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:43.329 DEBUG:boto:Method: GET 2022-09-03T06:12:43.330 DEBUG:boto:Path: /dsgvcy-70/?prefix=a 2022-09-03T06:12:43.331 DEBUG:boto:Data: 2022-09-03T06:12:43.331 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.332 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:43.333 DEBUG:boto:Port: 8000 2022-09-03T06:12:43.334 DEBUG:boto:Params: {} 2022-09-03T06:12:43.335 DEBUG:boto:Token: None 2022-09-03T06:12:43.336 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/ 2022-09-03T06:12:43.337 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jZclo+PtPXLiUC7j2stP4qVlhco= 2022-09-03T06:12:43.338 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jZclo+PtPXLiUC7j2stP4qVlhco=', 'Content-Length': '0'} 2022-09-03T06:12:43.342 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000043164cf4cdd4a795-006312f05b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.343 DEBUG:boto:b'dsgvcy-70a1000falsea2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:43.344 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:43.345 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:43.346 DEBUG:boto:Method: HEAD 2022-09-03T06:12:43.347 DEBUG:boto:Path: /dsgvcy-70/a 2022-09-03T06:12:43.348 DEBUG:boto:Data: 2022-09-03T06:12:43.349 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.350 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:43.351 DEBUG:boto:Port: 8001 2022-09-03T06:12:43.352 DEBUG:boto:Params: {} 2022-09-03T06:12:43.353 DEBUG:boto:Token: None 2022-09-03T06:12:43.354 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/a 2022-09-03T06:12:43.355 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7MHxexPvpBib5T/ma6lmsX7PG9I= 2022-09-03T06:12:43.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7MHxexPvpBib5T/ma6lmsX7PG9I=', 'Content-Length': '0'} 2022-09-03T06:12:43.359 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af9e8ab1e4cfeec8-006312f05b-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.359 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:43.360 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:43.360 DEBUG:boto:Method: HEAD 2022-09-03T06:12:43.361 DEBUG:boto:Path: /dsgvcy-70/a 2022-09-03T06:12:43.361 DEBUG:boto:Data: 2022-09-03T06:12:43.362 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.363 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:43.364 DEBUG:boto:Port: 8000 2022-09-03T06:12:43.364 DEBUG:boto:Params: {} 2022-09-03T06:12:43.365 DEBUG:boto:Token: None 2022-09-03T06:12:43.366 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/a 2022-09-03T06:12:43.367 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7MHxexPvpBib5T/ma6lmsX7PG9I= 2022-09-03T06:12:43.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7MHxexPvpBib5T/ma6lmsX7PG9I=', 'Content-Length': '0'} 2022-09-03T06:12:43.372 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000083dc5fe762c94091-006312f05b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.373 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:12:43.374 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:43.375 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:43.376 DEBUG:boto:Method: GET 2022-09-03T06:12:43.376 DEBUG:boto:Path: /dsgvcy-70/a 2022-09-03T06:12:43.377 DEBUG:boto:Data: 2022-09-03T06:12:43.377 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.378 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:43.379 DEBUG:boto:Port: 8001 2022-09-03T06:12:43.380 DEBUG:boto:Params: {} 2022-09-03T06:12:43.381 DEBUG:boto:Token: None 2022-09-03T06:12:43.387 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/a 2022-09-03T06:12:43.388 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9SjSVsk7XJn05aTXUi64TrfJ624= 2022-09-03T06:12:43.389 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9SjSVsk7XJn05aTXUi64TrfJ624=', 'Content-Length': '0'} 2022-09-03T06:12:43.393 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002b47814619d2dfec-006312f05b-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.434 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:43.435 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:43.436 DEBUG:boto:Method: GET 2022-09-03T06:12:43.436 DEBUG:boto:Path: /dsgvcy-70/a 2022-09-03T06:12:43.437 DEBUG:boto:Data: 2022-09-03T06:12:43.437 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.438 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:43.438 DEBUG:boto:Port: 8000 2022-09-03T06:12:43.439 DEBUG:boto:Params: {} 2022-09-03T06:12:43.439 DEBUG:boto:Token: None 2022-09-03T06:12:43.440 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/a 2022-09-03T06:12:43.441 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9SjSVsk7XJn05aTXUi64TrfJ624= 2022-09-03T06:12:43.441 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9SjSVsk7XJn05aTXUi64TrfJ624=', 'Content-Length': '0'} 2022-09-03T06:12:43.443 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000085def2bf48ac3d11-006312f05b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.444 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:43.445 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:43.445 DEBUG:boto:path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:43.446 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:43.447 DEBUG:boto:Method: GET 2022-09-03T06:12:43.447 DEBUG:boto:Path: /dsgvcy-70/?prefix=a 2022-09-03T06:12:43.448 DEBUG:boto:Data: 2022-09-03T06:12:43.448 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.449 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:43.449 DEBUG:boto:Port: 8001 2022-09-03T06:12:43.450 DEBUG:boto:Params: {} 2022-09-03T06:12:43.450 DEBUG:boto:Token: None 2022-09-03T06:12:43.451 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/ 2022-09-03T06:12:43.451 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jZclo+PtPXLiUC7j2stP4qVlhco= 2022-09-03T06:12:43.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jZclo+PtPXLiUC7j2stP4qVlhco=', 'Content-Length': '0'} 2022-09-03T06:12:43.455 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001060bcbfbc8ce04d-006312f05b-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.456 DEBUG:boto:b'dsgvcy-70a1000falsea2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:43.457 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:43.457 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:43.458 DEBUG:boto:path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:43.458 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:43.459 DEBUG:boto:Method: GET 2022-09-03T06:12:43.459 DEBUG:boto:Path: /dsgvcy-70/?prefix=a 2022-09-03T06:12:43.460 DEBUG:boto:Data: 2022-09-03T06:12:43.460 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.461 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:43.461 DEBUG:boto:Port: 8000 2022-09-03T06:12:43.462 DEBUG:boto:Params: {} 2022-09-03T06:12:43.462 DEBUG:boto:Token: None 2022-09-03T06:12:43.463 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/ 2022-09-03T06:12:43.464 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jZclo+PtPXLiUC7j2stP4qVlhco= 2022-09-03T06:12:43.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jZclo+PtPXLiUC7j2stP4qVlhco=', 'Content-Length': '0'} 2022-09-03T06:12:43.467 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000010d8a111390a94d7-006312f05b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.468 DEBUG:boto:b'dsgvcy-70a1000falsea2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:43.470 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:12:43.471 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:43.472 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:43.473 DEBUG:boto:path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.474 DEBUG:boto:auth_path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.475 DEBUG:boto:Method: GET 2022-09-03T06:12:43.476 DEBUG:boto:Path: /dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.476 DEBUG:boto:Data: 2022-09-03T06:12:43.477 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.478 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:43.479 DEBUG:boto:Port: 8001 2022-09-03T06:12:43.480 DEBUG:boto:Params: {} 2022-09-03T06:12:43.481 DEBUG:boto:Token: None 2022-09-03T06:12:43.481 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.482 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ulgv5hemKNVAxnBpe6EN3fCY+7s= 2022-09-03T06:12:43.483 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ulgv5hemKNVAxnBpe6EN3fCY+7s=', 'Content-Length': '0'} 2022-09-03T06:12:43.497 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000530cba6985133ac7-006312f05b-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.499 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:43.500 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:43.501 DEBUG:boto:path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.501 DEBUG:boto:auth_path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.502 DEBUG:boto:Method: GET 2022-09-03T06:12:43.502 DEBUG:boto:Path: /dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.503 DEBUG:boto:Data: 2022-09-03T06:12:43.503 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.504 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:43.504 DEBUG:boto:Port: 8000 2022-09-03T06:12:43.505 DEBUG:boto:Params: {} 2022-09-03T06:12:43.506 DEBUG:boto:Token: None 2022-09-03T06:12:43.507 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.508 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ulgv5hemKNVAxnBpe6EN3fCY+7s= 2022-09-03T06:12:43.509 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ulgv5hemKNVAxnBpe6EN3fCY+7s=', 'Content-Length': '0'} 2022-09-03T06:12:43.519 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008d7be08e49b9efb6-006312f05b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.520 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:43.521 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:43.521 DEBUG:boto:path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.522 DEBUG:boto:auth_path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.522 DEBUG:boto:Method: HEAD 2022-09-03T06:12:43.523 DEBUG:boto:Path: /dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.524 DEBUG:boto:Data: 2022-09-03T06:12:43.525 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.526 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:43.527 DEBUG:boto:Port: 8001 2022-09-03T06:12:43.527 DEBUG:boto:Params: {} 2022-09-03T06:12:43.529 DEBUG:boto:Token: None 2022-09-03T06:12:43.530 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.531 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DqZQi/Gd4tqeIlDF+5LL9UDcclI= 2022-09-03T06:12:43.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DqZQi/Gd4tqeIlDF+5LL9UDcclI=', 'Content-Length': '0'} 2022-09-03T06:12:43.535 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005a702c2c8b08edb9-006312f05b-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.536 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:43.537 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:43.538 DEBUG:boto:path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.539 DEBUG:boto:auth_path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.540 DEBUG:boto:Method: HEAD 2022-09-03T06:12:43.541 DEBUG:boto:Path: /dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.542 DEBUG:boto:Data: 2022-09-03T06:12:43.543 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.544 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:43.545 DEBUG:boto:Port: 8000 2022-09-03T06:12:43.545 DEBUG:boto:Params: {} 2022-09-03T06:12:43.546 DEBUG:boto:Token: None 2022-09-03T06:12:43.547 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.548 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DqZQi/Gd4tqeIlDF+5LL9UDcclI= 2022-09-03T06:12:43.548 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DqZQi/Gd4tqeIlDF+5LL9UDcclI=', 'Content-Length': '0'} 2022-09-03T06:12:43.551 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000023e9d47b095304b0-006312f05b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.551 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:12:43.552 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:43.553 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:43.553 DEBUG:boto:path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.554 DEBUG:boto:auth_path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.554 DEBUG:boto:Method: GET 2022-09-03T06:12:43.555 DEBUG:boto:Path: /dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.556 DEBUG:boto:Data: 2022-09-03T06:12:43.557 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.558 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:43.559 DEBUG:boto:Port: 8001 2022-09-03T06:12:43.559 DEBUG:boto:Params: {} 2022-09-03T06:12:43.560 DEBUG:boto:Token: None 2022-09-03T06:12:43.561 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.562 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ulgv5hemKNVAxnBpe6EN3fCY+7s= 2022-09-03T06:12:43.563 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ulgv5hemKNVAxnBpe6EN3fCY+7s=', 'Content-Length': '0'} 2022-09-03T06:12:43.565 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f268e4093f069208-006312f05b-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.616 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:43.617 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:43.618 DEBUG:boto:path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.619 DEBUG:boto:auth_path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.620 DEBUG:boto:Method: GET 2022-09-03T06:12:43.621 DEBUG:boto:Path: /dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.622 DEBUG:boto:Data: 2022-09-03T06:12:43.623 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.624 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:43.625 DEBUG:boto:Port: 8000 2022-09-03T06:12:43.627 DEBUG:boto:Params: {} 2022-09-03T06:12:43.628 DEBUG:boto:Token: None 2022-09-03T06:12:43.629 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/b?versionId=null 2022-09-03T06:12:43.637 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ulgv5hemKNVAxnBpe6EN3fCY+7s= 2022-09-03T06:12:43.638 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ulgv5hemKNVAxnBpe6EN3fCY+7s=', 'Content-Length': '0'} 2022-09-03T06:12:43.641 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b584c119e732e6e5-006312f05b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.642 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:43.643 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:43.644 DEBUG:boto:path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:43.645 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:43.647 DEBUG:boto:Method: GET 2022-09-03T06:12:43.648 DEBUG:boto:Path: /dsgvcy-70/?prefix=b 2022-09-03T06:12:43.649 DEBUG:boto:Data: 2022-09-03T06:12:43.650 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.651 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:43.651 DEBUG:boto:Port: 8001 2022-09-03T06:12:43.652 DEBUG:boto:Params: {} 2022-09-03T06:12:43.653 DEBUG:boto:Token: None 2022-09-03T06:12:43.657 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/ 2022-09-03T06:12:43.658 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jZclo+PtPXLiUC7j2stP4qVlhco= 2022-09-03T06:12:43.659 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jZclo+PtPXLiUC7j2stP4qVlhco=', 'Content-Length': '0'} 2022-09-03T06:12:43.663 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000be66a0a80de37ab3-006312f05b-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.664 DEBUG:boto:b'dsgvcy-70b1000falseb2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:43.665 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:43.665 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:43.674 DEBUG:boto:path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:43.676 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:43.677 DEBUG:boto:Method: GET 2022-09-03T06:12:43.678 DEBUG:boto:Path: /dsgvcy-70/?prefix=b 2022-09-03T06:12:43.680 DEBUG:boto:Data: 2022-09-03T06:12:43.682 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.683 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:43.683 DEBUG:boto:Port: 8000 2022-09-03T06:12:43.684 DEBUG:boto:Params: {} 2022-09-03T06:12:43.684 DEBUG:boto:Token: None 2022-09-03T06:12:43.685 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/ 2022-09-03T06:12:43.685 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jZclo+PtPXLiUC7j2stP4qVlhco= 2022-09-03T06:12:43.687 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jZclo+PtPXLiUC7j2stP4qVlhco=', 'Content-Length': '0'} 2022-09-03T06:12:43.691 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000080071f842731f705-006312f05b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.692 DEBUG:boto:b'dsgvcy-70b1000falseb2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:43.692 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:43.693 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:43.694 DEBUG:boto:Method: HEAD 2022-09-03T06:12:43.694 DEBUG:boto:Path: /dsgvcy-70/b 2022-09-03T06:12:43.695 DEBUG:boto:Data: 2022-09-03T06:12:43.695 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.697 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:43.698 DEBUG:boto:Port: 8001 2022-09-03T06:12:43.698 DEBUG:boto:Params: {} 2022-09-03T06:12:43.699 DEBUG:boto:Token: None 2022-09-03T06:12:43.700 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/b 2022-09-03T06:12:43.701 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3hNM0C4i+6oamzmSzwlpauamtZY= 2022-09-03T06:12:43.702 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3hNM0C4i+6oamzmSzwlpauamtZY=', 'Content-Length': '0'} 2022-09-03T06:12:43.707 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b656d3152334927-006312f05b-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.708 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:43.709 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:43.709 DEBUG:boto:Method: HEAD 2022-09-03T06:12:43.711 DEBUG:boto:Path: /dsgvcy-70/b 2022-09-03T06:12:43.712 DEBUG:boto:Data: 2022-09-03T06:12:43.712 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.713 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:43.713 DEBUG:boto:Port: 8000 2022-09-03T06:12:43.714 DEBUG:boto:Params: {} 2022-09-03T06:12:43.714 DEBUG:boto:Token: None 2022-09-03T06:12:43.717 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/b 2022-09-03T06:12:43.717 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3hNM0C4i+6oamzmSzwlpauamtZY= 2022-09-03T06:12:43.723 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3hNM0C4i+6oamzmSzwlpauamtZY=', 'Content-Length': '0'} 2022-09-03T06:12:43.726 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000093cdb4727ccf23bf-006312f05b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.727 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:12:43.728 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:43.728 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:43.729 DEBUG:boto:Method: GET 2022-09-03T06:12:43.731 DEBUG:boto:Path: /dsgvcy-70/b 2022-09-03T06:12:43.731 DEBUG:boto:Data: 2022-09-03T06:12:43.732 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.734 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:43.735 DEBUG:boto:Port: 8001 2022-09-03T06:12:43.736 DEBUG:boto:Params: {} 2022-09-03T06:12:43.737 DEBUG:boto:Token: None 2022-09-03T06:12:43.738 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/b 2022-09-03T06:12:43.739 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OBQgVT0rEGMCMDhW04xQfWu6xz8= 2022-09-03T06:12:43.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OBQgVT0rEGMCMDhW04xQfWu6xz8=', 'Content-Length': '0'} 2022-09-03T06:12:43.742 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c8b1a0bee89d928f-006312f05b-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.744 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:43.745 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:43.746 DEBUG:boto:Method: GET 2022-09-03T06:12:43.747 DEBUG:boto:Path: /dsgvcy-70/b 2022-09-03T06:12:43.747 DEBUG:boto:Data: 2022-09-03T06:12:43.748 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.749 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:43.749 DEBUG:boto:Port: 8000 2022-09-03T06:12:43.750 DEBUG:boto:Params: {} 2022-09-03T06:12:43.751 DEBUG:boto:Token: None 2022-09-03T06:12:43.752 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/b 2022-09-03T06:12:43.754 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OBQgVT0rEGMCMDhW04xQfWu6xz8= 2022-09-03T06:12:43.755 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OBQgVT0rEGMCMDhW04xQfWu6xz8=', 'Content-Length': '0'} 2022-09-03T06:12:43.758 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e02b307ce659b37-006312f05b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.760 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:43.761 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:43.762 DEBUG:boto:path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:43.763 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:43.764 DEBUG:boto:Method: GET 2022-09-03T06:12:43.765 DEBUG:boto:Path: /dsgvcy-70/?prefix=b 2022-09-03T06:12:43.766 DEBUG:boto:Data: 2022-09-03T06:12:43.766 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.767 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:43.768 DEBUG:boto:Port: 8001 2022-09-03T06:12:43.769 DEBUG:boto:Params: {} 2022-09-03T06:12:43.770 DEBUG:boto:Token: None 2022-09-03T06:12:43.771 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/ 2022-09-03T06:12:43.772 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jZclo+PtPXLiUC7j2stP4qVlhco= 2022-09-03T06:12:43.773 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jZclo+PtPXLiUC7j2stP4qVlhco=', 'Content-Length': '0'} 2022-09-03T06:12:43.776 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f3c84029f0bbd655-006312f05b-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.778 DEBUG:boto:b'dsgvcy-70b1000falseb2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:43.779 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:43.781 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:43.781 DEBUG:boto:path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:43.782 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:43.783 DEBUG:boto:Method: GET 2022-09-03T06:12:43.784 DEBUG:boto:Path: /dsgvcy-70/?prefix=b 2022-09-03T06:12:43.785 DEBUG:boto:Data: 2022-09-03T06:12:43.786 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.787 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:43.788 DEBUG:boto:Port: 8000 2022-09-03T06:12:43.789 DEBUG:boto:Params: {} 2022-09-03T06:12:43.791 DEBUG:boto:Token: None 2022-09-03T06:12:43.792 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/ 2022-09-03T06:12:43.793 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jZclo+PtPXLiUC7j2stP4qVlhco= 2022-09-03T06:12:43.793 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jZclo+PtPXLiUC7j2stP4qVlhco=', 'Content-Length': '0'} 2022-09-03T06:12:43.796 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d27becde420bbc35-006312f05b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.797 DEBUG:boto:b'dsgvcy-70b1000falseb2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:43.798 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:12:43.799 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:43.800 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:43.801 DEBUG:boto:path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.802 DEBUG:boto:auth_path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.803 DEBUG:boto:Method: GET 2022-09-03T06:12:43.804 DEBUG:boto:Path: /dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.805 DEBUG:boto:Data: 2022-09-03T06:12:43.806 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.807 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:43.808 DEBUG:boto:Port: 8001 2022-09-03T06:12:43.809 DEBUG:boto:Params: {} 2022-09-03T06:12:43.810 DEBUG:boto:Token: None 2022-09-03T06:12:43.811 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.812 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A0K/8JJCRVkWTgQ/tZPFUc05Kgw= 2022-09-03T06:12:43.814 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A0K/8JJCRVkWTgQ/tZPFUc05Kgw=', 'Content-Length': '0'} 2022-09-03T06:12:43.817 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fd5ead5430142df4-006312f05b-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.818 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:43.819 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:43.820 DEBUG:boto:path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.821 DEBUG:boto:auth_path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.821 DEBUG:boto:Method: GET 2022-09-03T06:12:43.822 DEBUG:boto:Path: /dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.823 DEBUG:boto:Data: 2022-09-03T06:12:43.824 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.825 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:43.826 DEBUG:boto:Port: 8000 2022-09-03T06:12:43.827 DEBUG:boto:Params: {} 2022-09-03T06:12:43.828 DEBUG:boto:Token: None 2022-09-03T06:12:43.829 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.830 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A0K/8JJCRVkWTgQ/tZPFUc05Kgw= 2022-09-03T06:12:43.831 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A0K/8JJCRVkWTgQ/tZPFUc05Kgw=', 'Content-Length': '0'} 2022-09-03T06:12:43.837 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c8b30e91a43a08e9-006312f05b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.838 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:43.839 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:43.841 DEBUG:boto:path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.842 DEBUG:boto:auth_path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.843 DEBUG:boto:Method: HEAD 2022-09-03T06:12:43.844 DEBUG:boto:Path: /dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.845 DEBUG:boto:Data: 2022-09-03T06:12:43.846 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.847 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:43.848 DEBUG:boto:Port: 8001 2022-09-03T06:12:43.850 DEBUG:boto:Params: {} 2022-09-03T06:12:43.851 DEBUG:boto:Token: None 2022-09-03T06:12:43.852 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.853 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9Hsw4BZMJhYOimR1/WXuPTVXI10= 2022-09-03T06:12:43.855 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9Hsw4BZMJhYOimR1/WXuPTVXI10=', 'Content-Length': '0'} 2022-09-03T06:12:43.858 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000045c6a36b510c4c18-006312f05b-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.859 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:43.860 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:43.861 DEBUG:boto:path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.862 DEBUG:boto:auth_path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.863 DEBUG:boto:Method: HEAD 2022-09-03T06:12:43.864 DEBUG:boto:Path: /dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.865 DEBUG:boto:Data: 2022-09-03T06:12:43.866 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.867 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:43.868 DEBUG:boto:Port: 8000 2022-09-03T06:12:43.869 DEBUG:boto:Params: {} 2022-09-03T06:12:43.870 DEBUG:boto:Token: None 2022-09-03T06:12:43.872 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.873 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9Hsw4BZMJhYOimR1/WXuPTVXI10= 2022-09-03T06:12:43.874 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9Hsw4BZMJhYOimR1/WXuPTVXI10=', 'Content-Length': '0'} 2022-09-03T06:12:43.877 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000169522b1d7dd7644-006312f05b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.879 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:12:43.880 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:43.881 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:43.882 DEBUG:boto:path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.883 DEBUG:boto:auth_path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.884 DEBUG:boto:Method: GET 2022-09-03T06:12:43.885 DEBUG:boto:Path: /dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.886 DEBUG:boto:Data: 2022-09-03T06:12:43.888 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.889 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:43.890 DEBUG:boto:Port: 8001 2022-09-03T06:12:43.891 DEBUG:boto:Params: {} 2022-09-03T06:12:43.892 DEBUG:boto:Token: None 2022-09-03T06:12:43.893 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.894 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A0K/8JJCRVkWTgQ/tZPFUc05Kgw= 2022-09-03T06:12:43.895 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A0K/8JJCRVkWTgQ/tZPFUc05Kgw=', 'Content-Length': '0'} 2022-09-03T06:12:43.902 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a6e83fce97cbd595-006312f05b-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.904 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:43.904 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:43.905 DEBUG:boto:path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.906 DEBUG:boto:auth_path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.906 DEBUG:boto:Method: GET 2022-09-03T06:12:43.907 DEBUG:boto:Path: /dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.907 DEBUG:boto:Data: 2022-09-03T06:12:43.908 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.908 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:43.909 DEBUG:boto:Port: 8000 2022-09-03T06:12:43.911 DEBUG:boto:Params: {} 2022-09-03T06:12:43.911 DEBUG:boto:Token: None 2022-09-03T06:12:43.912 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/c?versionId=null 2022-09-03T06:12:43.913 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A0K/8JJCRVkWTgQ/tZPFUc05Kgw= 2022-09-03T06:12:43.914 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A0K/8JJCRVkWTgQ/tZPFUc05Kgw=', 'Content-Length': '0'} 2022-09-03T06:12:43.917 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000498e1de8fc643265-006312f05b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.958 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:43.960 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:43.960 DEBUG:boto:path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:43.963 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:43.964 DEBUG:boto:Method: GET 2022-09-03T06:12:43.965 DEBUG:boto:Path: /dsgvcy-70/?prefix=c 2022-09-03T06:12:43.966 DEBUG:boto:Data: 2022-09-03T06:12:43.967 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.968 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:43.970 DEBUG:boto:Port: 8001 2022-09-03T06:12:43.971 DEBUG:boto:Params: {} 2022-09-03T06:12:43.972 DEBUG:boto:Token: None 2022-09-03T06:12:43.973 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/ 2022-09-03T06:12:43.975 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jZclo+PtPXLiUC7j2stP4qVlhco= 2022-09-03T06:12:43.977 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jZclo+PtPXLiUC7j2stP4qVlhco=', 'Content-Length': '0'} 2022-09-03T06:12:43.980 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000029b8aa130e699095-006312f05b-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:43.981 DEBUG:boto:b'dsgvcy-70c1000falsec2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:43.982 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:43.983 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:43.983 DEBUG:boto:path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:43.984 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:43.984 DEBUG:boto:Method: GET 2022-09-03T06:12:43.985 DEBUG:boto:Path: /dsgvcy-70/?prefix=c 2022-09-03T06:12:43.985 DEBUG:boto:Data: 2022-09-03T06:12:43.986 DEBUG:boto:Headers: {} 2022-09-03T06:12:43.987 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:43.988 DEBUG:boto:Port: 8000 2022-09-03T06:12:43.988 DEBUG:boto:Params: {} 2022-09-03T06:12:43.992 DEBUG:boto:Token: None 2022-09-03T06:12:43.993 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:43 GMT /dsgvcy-70/ 2022-09-03T06:12:43.994 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jZclo+PtPXLiUC7j2stP4qVlhco= 2022-09-03T06:12:43.994 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jZclo+PtPXLiUC7j2stP4qVlhco=', 'Content-Length': '0'} 2022-09-03T06:12:44.003 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000056e445836d8fc23c-006312f05b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.004 DEBUG:boto:b'dsgvcy-70c1000falsec2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:44.005 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:44.006 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:44.007 DEBUG:boto:Method: HEAD 2022-09-03T06:12:44.008 DEBUG:boto:Path: /dsgvcy-70/c 2022-09-03T06:12:44.008 DEBUG:boto:Data: 2022-09-03T06:12:44.009 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.009 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:44.010 DEBUG:boto:Port: 8001 2022-09-03T06:12:44.010 DEBUG:boto:Params: {} 2022-09-03T06:12:44.011 DEBUG:boto:Token: None 2022-09-03T06:12:44.011 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/c 2022-09-03T06:12:44.013 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nTsywuG/68ilefj6aGNEUEjYe+8= 2022-09-03T06:12:44.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nTsywuG/68ilefj6aGNEUEjYe+8=', 'Content-Length': '0'} 2022-09-03T06:12:44.016 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e29f23e9716456f0-006312f05c-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.017 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:44.018 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:44.019 DEBUG:boto:Method: HEAD 2022-09-03T06:12:44.020 DEBUG:boto:Path: /dsgvcy-70/c 2022-09-03T06:12:44.021 DEBUG:boto:Data: 2022-09-03T06:12:44.022 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.022 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:44.023 DEBUG:boto:Port: 8000 2022-09-03T06:12:44.024 DEBUG:boto:Params: {} 2022-09-03T06:12:44.025 DEBUG:boto:Token: None 2022-09-03T06:12:44.026 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/c 2022-09-03T06:12:44.031 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nTsywuG/68ilefj6aGNEUEjYe+8= 2022-09-03T06:12:44.031 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nTsywuG/68ilefj6aGNEUEjYe+8=', 'Content-Length': '0'} 2022-09-03T06:12:44.034 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003f69dd0680b4fbb8-006312f05c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.035 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:12:44.035 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:44.036 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:44.036 DEBUG:boto:Method: GET 2022-09-03T06:12:44.037 DEBUG:boto:Path: /dsgvcy-70/c 2022-09-03T06:12:44.037 DEBUG:boto:Data: 2022-09-03T06:12:44.038 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.039 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:44.039 DEBUG:boto:Port: 8001 2022-09-03T06:12:44.040 DEBUG:boto:Params: {} 2022-09-03T06:12:44.041 DEBUG:boto:Token: None 2022-09-03T06:12:44.042 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/c 2022-09-03T06:12:44.043 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UKuyyHbSVm34sOUmLBwl3Qcz4U4= 2022-09-03T06:12:44.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UKuyyHbSVm34sOUmLBwl3Qcz4U4=', 'Content-Length': '0'} 2022-09-03T06:12:44.045 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a54520fca5df7533-006312f05c-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.046 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:44.047 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:44.048 DEBUG:boto:Method: GET 2022-09-03T06:12:44.048 DEBUG:boto:Path: /dsgvcy-70/c 2022-09-03T06:12:44.049 DEBUG:boto:Data: 2022-09-03T06:12:44.049 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.050 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:44.050 DEBUG:boto:Port: 8000 2022-09-03T06:12:44.051 DEBUG:boto:Params: {} 2022-09-03T06:12:44.052 DEBUG:boto:Token: None 2022-09-03T06:12:44.052 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/c 2022-09-03T06:12:44.053 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UKuyyHbSVm34sOUmLBwl3Qcz4U4= 2022-09-03T06:12:44.054 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UKuyyHbSVm34sOUmLBwl3Qcz4U4=', 'Content-Length': '0'} 2022-09-03T06:12:44.056 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003ba714693dbea37b-006312f05c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.059 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:44.060 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:44.061 DEBUG:boto:path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:44.061 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:44.063 DEBUG:boto:Method: GET 2022-09-03T06:12:44.064 DEBUG:boto:Path: /dsgvcy-70/?prefix=c 2022-09-03T06:12:44.064 DEBUG:boto:Data: 2022-09-03T06:12:44.065 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.067 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:44.067 DEBUG:boto:Port: 8001 2022-09-03T06:12:44.068 DEBUG:boto:Params: {} 2022-09-03T06:12:44.070 DEBUG:boto:Token: None 2022-09-03T06:12:44.071 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/ 2022-09-03T06:12:44.074 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PF6ENIbXAAVQrrJxckKSn8xbCgE= 2022-09-03T06:12:44.075 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PF6ENIbXAAVQrrJxckKSn8xbCgE=', 'Content-Length': '0'} 2022-09-03T06:12:44.079 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e9a6a8e4529d3fb4-006312f05c-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.080 DEBUG:boto:b'dsgvcy-70c1000falsec2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:44.082 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:44.089 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:44.090 DEBUG:boto:path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:44.091 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:44.092 DEBUG:boto:Method: GET 2022-09-03T06:12:44.093 DEBUG:boto:Path: /dsgvcy-70/?prefix=c 2022-09-03T06:12:44.095 DEBUG:boto:Data: 2022-09-03T06:12:44.095 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.096 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:44.097 DEBUG:boto:Port: 8000 2022-09-03T06:12:44.097 DEBUG:boto:Params: {} 2022-09-03T06:12:44.098 DEBUG:boto:Token: None 2022-09-03T06:12:44.099 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/ 2022-09-03T06:12:44.100 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PF6ENIbXAAVQrrJxckKSn8xbCgE= 2022-09-03T06:12:44.100 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PF6ENIbXAAVQrrJxckKSn8xbCgE=', 'Content-Length': '0'} 2022-09-03T06:12:44.104 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005725f67e1e1d1a68-006312f05c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.109 DEBUG:boto:b'dsgvcy-70c1000falsec2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:44.117 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:12:44.118 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:44.120 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:44.121 DEBUG:boto:path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.122 DEBUG:boto:auth_path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.123 DEBUG:boto:Method: GET 2022-09-03T06:12:44.124 DEBUG:boto:Path: /dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.125 DEBUG:boto:Data: 2022-09-03T06:12:44.126 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.127 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:44.128 DEBUG:boto:Port: 8001 2022-09-03T06:12:44.129 DEBUG:boto:Params: {} 2022-09-03T06:12:44.131 DEBUG:boto:Token: None 2022-09-03T06:12:44.132 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.134 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TMLqSu0cMv4Ey/e9eM9glBgDKbM= 2022-09-03T06:12:44.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TMLqSu0cMv4Ey/e9eM9glBgDKbM=', 'Content-Length': '0'} 2022-09-03T06:12:44.138 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009de9a5883541207a-006312f05c-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.139 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:44.139 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:44.140 DEBUG:boto:path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.142 DEBUG:boto:auth_path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.143 DEBUG:boto:Method: GET 2022-09-03T06:12:44.143 DEBUG:boto:Path: /dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.144 DEBUG:boto:Data: 2022-09-03T06:12:44.145 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.146 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:44.147 DEBUG:boto:Port: 8000 2022-09-03T06:12:44.147 DEBUG:boto:Params: {} 2022-09-03T06:12:44.148 DEBUG:boto:Token: None 2022-09-03T06:12:44.148 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.149 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TMLqSu0cMv4Ey/e9eM9glBgDKbM= 2022-09-03T06:12:44.150 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TMLqSu0cMv4Ey/e9eM9glBgDKbM=', 'Content-Length': '0'} 2022-09-03T06:12:44.152 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cc91e4944a27311a-006312f05c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.153 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:44.154 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:44.154 DEBUG:boto:path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.155 DEBUG:boto:auth_path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.156 DEBUG:boto:Method: HEAD 2022-09-03T06:12:44.157 DEBUG:boto:Path: /dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.158 DEBUG:boto:Data: 2022-09-03T06:12:44.158 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.159 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:44.160 DEBUG:boto:Port: 8001 2022-09-03T06:12:44.161 DEBUG:boto:Params: {} 2022-09-03T06:12:44.161 DEBUG:boto:Token: None 2022-09-03T06:12:44.162 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.162 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/r0zddTMgg/Pty1+3TwwiH6UqLM= 2022-09-03T06:12:44.163 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/r0zddTMgg/Pty1+3TwwiH6UqLM=', 'Content-Length': '0'} 2022-09-03T06:12:44.165 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004eea4922b3272303-006312f05c-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.173 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:44.174 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:44.175 DEBUG:boto:path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.176 DEBUG:boto:auth_path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.176 DEBUG:boto:Method: HEAD 2022-09-03T06:12:44.178 DEBUG:boto:Path: /dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.178 DEBUG:boto:Data: 2022-09-03T06:12:44.179 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.179 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:44.180 DEBUG:boto:Port: 8000 2022-09-03T06:12:44.181 DEBUG:boto:Params: {} 2022-09-03T06:12:44.181 DEBUG:boto:Token: None 2022-09-03T06:12:44.182 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.183 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/r0zddTMgg/Pty1+3TwwiH6UqLM= 2022-09-03T06:12:44.183 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/r0zddTMgg/Pty1+3TwwiH6UqLM=', 'Content-Length': '0'} 2022-09-03T06:12:44.187 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c1f88225bc7226ca-006312f05c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.188 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:12:44.189 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:44.189 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:44.190 DEBUG:boto:path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.190 DEBUG:boto:auth_path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.191 DEBUG:boto:Method: GET 2022-09-03T06:12:44.191 DEBUG:boto:Path: /dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.192 DEBUG:boto:Data: 2022-09-03T06:12:44.195 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.195 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:44.196 DEBUG:boto:Port: 8001 2022-09-03T06:12:44.196 DEBUG:boto:Params: {} 2022-09-03T06:12:44.197 DEBUG:boto:Token: None 2022-09-03T06:12:44.198 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.198 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TMLqSu0cMv4Ey/e9eM9glBgDKbM= 2022-09-03T06:12:44.201 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TMLqSu0cMv4Ey/e9eM9glBgDKbM=', 'Content-Length': '0'} 2022-09-03T06:12:44.203 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b1f2b8ca24bb8c1b-006312f05c-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.204 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:44.205 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:44.206 DEBUG:boto:path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.206 DEBUG:boto:auth_path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.207 DEBUG:boto:Method: GET 2022-09-03T06:12:44.207 DEBUG:boto:Path: /dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.208 DEBUG:boto:Data: 2022-09-03T06:12:44.208 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.209 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:44.209 DEBUG:boto:Port: 8000 2022-09-03T06:12:44.210 DEBUG:boto:Params: {} 2022-09-03T06:12:44.211 DEBUG:boto:Token: None 2022-09-03T06:12:44.211 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/d?versionId=null 2022-09-03T06:12:44.221 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TMLqSu0cMv4Ey/e9eM9glBgDKbM= 2022-09-03T06:12:44.222 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TMLqSu0cMv4Ey/e9eM9glBgDKbM=', 'Content-Length': '0'} 2022-09-03T06:12:44.230 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000049b77251ae881244-006312f05c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.232 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:44.233 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:44.235 DEBUG:boto:path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:44.236 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:44.237 DEBUG:boto:Method: GET 2022-09-03T06:12:44.242 DEBUG:boto:Path: /dsgvcy-70/?prefix=d 2022-09-03T06:12:44.243 DEBUG:boto:Data: 2022-09-03T06:12:44.244 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.244 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:44.245 DEBUG:boto:Port: 8001 2022-09-03T06:12:44.245 DEBUG:boto:Params: {} 2022-09-03T06:12:44.247 DEBUG:boto:Token: None 2022-09-03T06:12:44.248 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/ 2022-09-03T06:12:44.249 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PF6ENIbXAAVQrrJxckKSn8xbCgE= 2022-09-03T06:12:44.250 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PF6ENIbXAAVQrrJxckKSn8xbCgE=', 'Content-Length': '0'} 2022-09-03T06:12:44.258 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004716323ce8210c25-006312f05c-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.259 DEBUG:boto:b'dsgvcy-70d1000falsed2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:44.260 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:44.260 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:44.261 DEBUG:boto:path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:44.261 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:44.262 DEBUG:boto:Method: GET 2022-09-03T06:12:44.262 DEBUG:boto:Path: /dsgvcy-70/?prefix=d 2022-09-03T06:12:44.263 DEBUG:boto:Data: 2022-09-03T06:12:44.264 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.265 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:44.265 DEBUG:boto:Port: 8000 2022-09-03T06:12:44.266 DEBUG:boto:Params: {} 2022-09-03T06:12:44.267 DEBUG:boto:Token: None 2022-09-03T06:12:44.268 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/ 2022-09-03T06:12:44.268 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PF6ENIbXAAVQrrJxckKSn8xbCgE= 2022-09-03T06:12:44.269 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PF6ENIbXAAVQrrJxckKSn8xbCgE=', 'Content-Length': '0'} 2022-09-03T06:12:44.272 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000006df3d0638c1fe6e-006312f05c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.315 DEBUG:boto:b'dsgvcy-70d1000falsed2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:44.317 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:44.317 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:44.318 DEBUG:boto:Method: HEAD 2022-09-03T06:12:44.318 DEBUG:boto:Path: /dsgvcy-70/d 2022-09-03T06:12:44.319 DEBUG:boto:Data: 2022-09-03T06:12:44.319 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.320 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:44.321 DEBUG:boto:Port: 8001 2022-09-03T06:12:44.322 DEBUG:boto:Params: {} 2022-09-03T06:12:44.322 DEBUG:boto:Token: None 2022-09-03T06:12:44.326 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/d 2022-09-03T06:12:44.327 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:esR0+VbQVXilgnZeQQiPtEJNots= 2022-09-03T06:12:44.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:esR0+VbQVXilgnZeQQiPtEJNots=', 'Content-Length': '0'} 2022-09-03T06:12:44.330 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ac70834aebb812bf-006312f05c-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.333 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:44.334 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:44.335 DEBUG:boto:Method: HEAD 2022-09-03T06:12:44.336 DEBUG:boto:Path: /dsgvcy-70/d 2022-09-03T06:12:44.337 DEBUG:boto:Data: 2022-09-03T06:12:44.338 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.339 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:44.340 DEBUG:boto:Port: 8000 2022-09-03T06:12:44.341 DEBUG:boto:Params: {} 2022-09-03T06:12:44.342 DEBUG:boto:Token: None 2022-09-03T06:12:44.344 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/d 2022-09-03T06:12:44.345 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:esR0+VbQVXilgnZeQQiPtEJNots= 2022-09-03T06:12:44.346 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:esR0+VbQVXilgnZeQQiPtEJNots=', 'Content-Length': '0'} 2022-09-03T06:12:44.352 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008feb58d0087c7b4a-006312f05c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.353 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:12:44.355 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:44.356 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:44.357 DEBUG:boto:Method: GET 2022-09-03T06:12:44.358 DEBUG:boto:Path: /dsgvcy-70/d 2022-09-03T06:12:44.358 DEBUG:boto:Data: 2022-09-03T06:12:44.359 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.360 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:44.361 DEBUG:boto:Port: 8001 2022-09-03T06:12:44.362 DEBUG:boto:Params: {} 2022-09-03T06:12:44.363 DEBUG:boto:Token: None 2022-09-03T06:12:44.364 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/d 2022-09-03T06:12:44.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Pc0euTpvweVaPeqwpdI0vUeFJ3Q= 2022-09-03T06:12:44.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Pc0euTpvweVaPeqwpdI0vUeFJ3Q=', 'Content-Length': '0'} 2022-09-03T06:12:44.381 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074746588399bf2a7-006312f05c-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.391 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:44.392 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:44.393 DEBUG:boto:Method: GET 2022-09-03T06:12:44.394 DEBUG:boto:Path: /dsgvcy-70/d 2022-09-03T06:12:44.395 DEBUG:boto:Data: 2022-09-03T06:12:44.396 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.397 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:44.398 DEBUG:boto:Port: 8000 2022-09-03T06:12:44.399 DEBUG:boto:Params: {} 2022-09-03T06:12:44.400 DEBUG:boto:Token: None 2022-09-03T06:12:44.401 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/d 2022-09-03T06:12:44.401 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Pc0euTpvweVaPeqwpdI0vUeFJ3Q= 2022-09-03T06:12:44.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Pc0euTpvweVaPeqwpdI0vUeFJ3Q=', 'Content-Length': '0'} 2022-09-03T06:12:44.407 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026368b98f5344fb8-006312f05c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.408 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:44.409 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:44.410 DEBUG:boto:path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:44.411 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:44.412 DEBUG:boto:Method: GET 2022-09-03T06:12:44.413 DEBUG:boto:Path: /dsgvcy-70/?prefix=d 2022-09-03T06:12:44.414 DEBUG:boto:Data: 2022-09-03T06:12:44.415 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.416 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:44.417 DEBUG:boto:Port: 8001 2022-09-03T06:12:44.418 DEBUG:boto:Params: {} 2022-09-03T06:12:44.419 DEBUG:boto:Token: None 2022-09-03T06:12:44.420 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/ 2022-09-03T06:12:44.421 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PF6ENIbXAAVQrrJxckKSn8xbCgE= 2022-09-03T06:12:44.422 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PF6ENIbXAAVQrrJxckKSn8xbCgE=', 'Content-Length': '0'} 2022-09-03T06:12:44.426 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e884ad679b19d70e-006312f05c-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.427 DEBUG:boto:b'dsgvcy-70d1000falsed2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:44.429 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:44.429 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:44.430 DEBUG:boto:path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:44.430 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:44.431 DEBUG:boto:Method: GET 2022-09-03T06:12:44.432 DEBUG:boto:Path: /dsgvcy-70/?prefix=d 2022-09-03T06:12:44.433 DEBUG:boto:Data: 2022-09-03T06:12:44.434 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.435 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:44.436 DEBUG:boto:Port: 8000 2022-09-03T06:12:44.437 DEBUG:boto:Params: {} 2022-09-03T06:12:44.438 DEBUG:boto:Token: None 2022-09-03T06:12:44.439 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/ 2022-09-03T06:12:44.440 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PF6ENIbXAAVQrrJxckKSn8xbCgE= 2022-09-03T06:12:44.441 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PF6ENIbXAAVQrrJxckKSn8xbCgE=', 'Content-Length': '0'} 2022-09-03T06:12:44.444 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000adf7fafd2f20c3c4-006312f05c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.486 DEBUG:boto:b'dsgvcy-70d1000falsed2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:44.488 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-70 zones={test-zone3, test-zone1} 2022-09-03T06:12:44.489 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-70 zones={test-zone3, test-zone2} 2022-09-03T06:12:44.490 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:44.491 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:44.492 DEBUG:boto:Method: HEAD 2022-09-03T06:12:44.493 DEBUG:boto:Path: /dsgvcy-70/ 2022-09-03T06:12:44.494 DEBUG:boto:Data: 2022-09-03T06:12:44.495 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.496 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:44.497 DEBUG:boto:Port: 8001 2022-09-03T06:12:44.498 DEBUG:boto:Params: {} 2022-09-03T06:12:44.499 DEBUG:boto:Token: None 2022-09-03T06:12:44.501 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/ 2022-09-03T06:12:44.502 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bHY2z/ag4ohvmnGPicI2/j4skCU= 2022-09-03T06:12:44.503 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bHY2z/ag4ohvmnGPicI2/j4skCU=', 'Content-Length': '0'} 2022-09-03T06:12:44.507 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000526d4f0dc30e5f3-006312f05c-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.509 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:44.510 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:44.511 DEBUG:boto:Method: HEAD 2022-09-03T06:12:44.512 DEBUG:boto:Path: /dsgvcy-70/ 2022-09-03T06:12:44.513 DEBUG:boto:Data: 2022-09-03T06:12:44.513 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.514 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:44.515 DEBUG:boto:Port: 8000 2022-09-03T06:12:44.516 DEBUG:boto:Params: {} 2022-09-03T06:12:44.517 DEBUG:boto:Token: None 2022-09-03T06:12:44.518 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/ 2022-09-03T06:12:44.520 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bHY2z/ag4ohvmnGPicI2/j4skCU= 2022-09-03T06:12:44.521 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bHY2z/ag4ohvmnGPicI2/j4skCU=', 'Content-Length': '0'} 2022-09-03T06:12:44.524 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000015a68b07fb49a02b-006312f05c-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.526 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:12:44.527 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:44.528 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:44.529 DEBUG:boto:path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:44.530 DEBUG:boto:auth_path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:44.531 DEBUG:boto:Method: GET 2022-09-03T06:12:44.532 DEBUG:boto:Path: /dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:44.533 DEBUG:boto:Data: 2022-09-03T06:12:44.534 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.535 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:44.536 DEBUG:boto:Port: 8001 2022-09-03T06:12:44.537 DEBUG:boto:Params: {} 2022-09-03T06:12:44.538 DEBUG:boto:Token: None 2022-09-03T06:12:44.539 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/?versions 2022-09-03T06:12:44.540 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:whImY7PD6S3wHChuJWXpiO37FZs= 2022-09-03T06:12:44.541 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:whImY7PD6S3wHChuJWXpiO37FZs=', 'Content-Length': '0'} 2022-09-03T06:12:44.544 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000011d855bc5c6891f1-006312f05c-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.546 DEBUG:boto:b'dsgvcy-70999falseanulltrue2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:44.548 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:12:44.549 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:12:44.550 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:12:44.550 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:12:44.552 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:12:44.553 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:44.554 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:44.555 DEBUG:boto:path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:44.555 DEBUG:boto:auth_path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:44.556 DEBUG:boto:Method: GET 2022-09-03T06:12:44.558 DEBUG:boto:Path: /dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:44.559 DEBUG:boto:Data: 2022-09-03T06:12:44.560 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.561 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:44.562 DEBUG:boto:Port: 8000 2022-09-03T06:12:44.563 DEBUG:boto:Params: {} 2022-09-03T06:12:44.564 DEBUG:boto:Token: None 2022-09-03T06:12:44.566 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/?versions 2022-09-03T06:12:44.567 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:whImY7PD6S3wHChuJWXpiO37FZs= 2022-09-03T06:12:44.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:whImY7PD6S3wHChuJWXpiO37FZs=', 'Content-Length': '0'} 2022-09-03T06:12:44.572 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000021b4ac6bb2757f71-006312f05c-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.573 DEBUG:boto:b'dsgvcy-70999falseanulltrue2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:44.575 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:12:44.576 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:12:44.577 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:12:44.578 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:12:44.579 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:44.580 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:44.581 DEBUG:boto:path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:44.582 DEBUG:boto:auth_path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:44.583 DEBUG:boto:Method: GET 2022-09-03T06:12:44.584 DEBUG:boto:Path: /dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:44.585 DEBUG:boto:Data: 2022-09-03T06:12:44.586 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.587 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:44.589 DEBUG:boto:Port: 8001 2022-09-03T06:12:44.590 DEBUG:boto:Params: {} 2022-09-03T06:12:44.591 DEBUG:boto:Token: None 2022-09-03T06:12:44.592 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/?versions 2022-09-03T06:12:44.593 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:whImY7PD6S3wHChuJWXpiO37FZs= 2022-09-03T06:12:44.595 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:whImY7PD6S3wHChuJWXpiO37FZs=', 'Content-Length': '0'} 2022-09-03T06:12:44.598 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008406c7b32228a5b5-006312f05c-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.599 DEBUG:boto:b'dsgvcy-70999falseanulltrue2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:44.600 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:44.601 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:44.601 DEBUG:boto:path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:44.602 DEBUG:boto:auth_path=/dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:44.604 DEBUG:boto:Method: GET 2022-09-03T06:12:44.605 DEBUG:boto:Path: /dsgvcy-70/?versions&max-keys=999 2022-09-03T06:12:44.607 DEBUG:boto:Data: 2022-09-03T06:12:44.608 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.609 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:44.611 DEBUG:boto:Port: 8000 2022-09-03T06:12:44.612 DEBUG:boto:Params: {} 2022-09-03T06:12:44.613 DEBUG:boto:Token: None 2022-09-03T06:12:44.614 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/?versions 2022-09-03T06:12:44.615 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:whImY7PD6S3wHChuJWXpiO37FZs= 2022-09-03T06:12:44.617 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:whImY7PD6S3wHChuJWXpiO37FZs=', 'Content-Length': '0'} 2022-09-03T06:12:44.622 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000042647e809dc99d57-006312f05c-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.624 DEBUG:boto:b'dsgvcy-70999falseanulltrue2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:44.626 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:12:44.627 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:44.628 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:44.629 DEBUG:boto:path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.631 DEBUG:boto:auth_path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.632 DEBUG:boto:Method: GET 2022-09-03T06:12:44.633 DEBUG:boto:Path: /dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.634 DEBUG:boto:Data: 2022-09-03T06:12:44.635 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.636 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:44.637 DEBUG:boto:Port: 8001 2022-09-03T06:12:44.638 DEBUG:boto:Params: {} 2022-09-03T06:12:44.639 DEBUG:boto:Token: None 2022-09-03T06:12:44.640 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.641 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pbkdd4Evss2wIiMmhzEOGbUBHCQ= 2022-09-03T06:12:44.642 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pbkdd4Evss2wIiMmhzEOGbUBHCQ=', 'Content-Length': '0'} 2022-09-03T06:12:44.646 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f8f5aa0ead0ca6d7-006312f05c-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.647 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:44.648 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:44.649 DEBUG:boto:path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.650 DEBUG:boto:auth_path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.651 DEBUG:boto:Method: GET 2022-09-03T06:12:44.652 DEBUG:boto:Path: /dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.653 DEBUG:boto:Data: 2022-09-03T06:12:44.654 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.655 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:44.656 DEBUG:boto:Port: 8000 2022-09-03T06:12:44.657 DEBUG:boto:Params: {} 2022-09-03T06:12:44.658 DEBUG:boto:Token: None 2022-09-03T06:12:44.659 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.660 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pbkdd4Evss2wIiMmhzEOGbUBHCQ= 2022-09-03T06:12:44.662 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pbkdd4Evss2wIiMmhzEOGbUBHCQ=', 'Content-Length': '0'} 2022-09-03T06:12:44.665 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a7a919b77f0d0eb5-006312f05c-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.666 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:44.667 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:44.680 DEBUG:boto:path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.680 DEBUG:boto:auth_path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.681 DEBUG:boto:Method: HEAD 2022-09-03T06:12:44.682 DEBUG:boto:Path: /dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.682 DEBUG:boto:Data: 2022-09-03T06:12:44.683 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.683 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:44.684 DEBUG:boto:Port: 8001 2022-09-03T06:12:44.684 DEBUG:boto:Params: {} 2022-09-03T06:12:44.685 DEBUG:boto:Token: None 2022-09-03T06:12:44.685 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.686 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZDvrW7kdrF/KZ2+oIljmP3OcxZc= 2022-09-03T06:12:44.687 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZDvrW7kdrF/KZ2+oIljmP3OcxZc=', 'Content-Length': '0'} 2022-09-03T06:12:44.690 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a99dbd74bf8ddab-006312f05c-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.692 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:44.699 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:44.700 DEBUG:boto:path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.702 DEBUG:boto:auth_path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.703 DEBUG:boto:Method: HEAD 2022-09-03T06:12:44.703 DEBUG:boto:Path: /dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.704 DEBUG:boto:Data: 2022-09-03T06:12:44.705 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.705 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:44.706 DEBUG:boto:Port: 8000 2022-09-03T06:12:44.706 DEBUG:boto:Params: {} 2022-09-03T06:12:44.707 DEBUG:boto:Token: None 2022-09-03T06:12:44.708 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.709 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZDvrW7kdrF/KZ2+oIljmP3OcxZc= 2022-09-03T06:12:44.711 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZDvrW7kdrF/KZ2+oIljmP3OcxZc=', 'Content-Length': '0'} 2022-09-03T06:12:44.715 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009be35f786996adc6-006312f05c-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.716 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:12:44.717 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:44.721 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:44.721 DEBUG:boto:path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.722 DEBUG:boto:auth_path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.723 DEBUG:boto:Method: GET 2022-09-03T06:12:44.723 DEBUG:boto:Path: /dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.724 DEBUG:boto:Data: 2022-09-03T06:12:44.725 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.725 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:44.726 DEBUG:boto:Port: 8001 2022-09-03T06:12:44.727 DEBUG:boto:Params: {} 2022-09-03T06:12:44.728 DEBUG:boto:Token: None 2022-09-03T06:12:44.729 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.729 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pbkdd4Evss2wIiMmhzEOGbUBHCQ= 2022-09-03T06:12:44.730 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pbkdd4Evss2wIiMmhzEOGbUBHCQ=', 'Content-Length': '0'} 2022-09-03T06:12:44.734 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f604e7879ff0c39-006312f05c-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.735 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:44.736 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:44.737 DEBUG:boto:path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.738 DEBUG:boto:auth_path=/dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.739 DEBUG:boto:Method: GET 2022-09-03T06:12:44.740 DEBUG:boto:Path: /dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.741 DEBUG:boto:Data: 2022-09-03T06:12:44.742 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.743 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:44.745 DEBUG:boto:Port: 8000 2022-09-03T06:12:44.746 DEBUG:boto:Params: {} 2022-09-03T06:12:44.747 DEBUG:boto:Token: None 2022-09-03T06:12:44.748 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/a?versionId=null 2022-09-03T06:12:44.749 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pbkdd4Evss2wIiMmhzEOGbUBHCQ= 2022-09-03T06:12:44.751 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pbkdd4Evss2wIiMmhzEOGbUBHCQ=', 'Content-Length': '0'} 2022-09-03T06:12:44.753 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dd8f9584050737e8-006312f05c-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.755 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:44.756 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:44.757 DEBUG:boto:path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:44.758 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:44.759 DEBUG:boto:Method: GET 2022-09-03T06:12:44.760 DEBUG:boto:Path: /dsgvcy-70/?prefix=a 2022-09-03T06:12:44.761 DEBUG:boto:Data: 2022-09-03T06:12:44.762 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.763 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:44.764 DEBUG:boto:Port: 8001 2022-09-03T06:12:44.765 DEBUG:boto:Params: {} 2022-09-03T06:12:44.766 DEBUG:boto:Token: None 2022-09-03T06:12:44.767 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/ 2022-09-03T06:12:44.768 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PF6ENIbXAAVQrrJxckKSn8xbCgE= 2022-09-03T06:12:44.770 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PF6ENIbXAAVQrrJxckKSn8xbCgE=', 'Content-Length': '0'} 2022-09-03T06:12:44.776 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000090c91c1f43889fa9-006312f05c-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.778 DEBUG:boto:b'dsgvcy-70a1000falsea2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:44.779 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:44.780 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:44.781 DEBUG:boto:path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:44.782 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:44.783 DEBUG:boto:Method: GET 2022-09-03T06:12:44.784 DEBUG:boto:Path: /dsgvcy-70/?prefix=a 2022-09-03T06:12:44.785 DEBUG:boto:Data: 2022-09-03T06:12:44.787 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.787 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:44.788 DEBUG:boto:Port: 8000 2022-09-03T06:12:44.789 DEBUG:boto:Params: {} 2022-09-03T06:12:44.790 DEBUG:boto:Token: None 2022-09-03T06:12:44.790 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/ 2022-09-03T06:12:44.791 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PF6ENIbXAAVQrrJxckKSn8xbCgE= 2022-09-03T06:12:44.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PF6ENIbXAAVQrrJxckKSn8xbCgE=', 'Content-Length': '0'} 2022-09-03T06:12:44.796 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000015c99508f3890799-006312f05c-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.798 DEBUG:boto:b'dsgvcy-70a1000falsea2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:44.799 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:44.800 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:44.801 DEBUG:boto:Method: HEAD 2022-09-03T06:12:44.802 DEBUG:boto:Path: /dsgvcy-70/a 2022-09-03T06:12:44.803 DEBUG:boto:Data: 2022-09-03T06:12:44.804 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.804 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:44.805 DEBUG:boto:Port: 8001 2022-09-03T06:12:44.806 DEBUG:boto:Params: {} 2022-09-03T06:12:44.807 DEBUG:boto:Token: None 2022-09-03T06:12:44.808 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/a 2022-09-03T06:12:44.809 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rfYOGizoiuGlNKX+ggWGqWH9tfI= 2022-09-03T06:12:44.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rfYOGizoiuGlNKX+ggWGqWH9tfI=', 'Content-Length': '0'} 2022-09-03T06:12:44.813 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cedc94e6abf862cd-006312f05c-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.814 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:44.815 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:44.815 DEBUG:boto:Method: HEAD 2022-09-03T06:12:44.816 DEBUG:boto:Path: /dsgvcy-70/a 2022-09-03T06:12:44.817 DEBUG:boto:Data: 2022-09-03T06:12:44.817 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.818 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:44.819 DEBUG:boto:Port: 8000 2022-09-03T06:12:44.820 DEBUG:boto:Params: {} 2022-09-03T06:12:44.821 DEBUG:boto:Token: None 2022-09-03T06:12:44.823 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/a 2022-09-03T06:12:44.824 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rfYOGizoiuGlNKX+ggWGqWH9tfI= 2022-09-03T06:12:44.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rfYOGizoiuGlNKX+ggWGqWH9tfI=', 'Content-Length': '0'} 2022-09-03T06:12:44.829 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001aa56beeda0cdd11-006312f05c-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.831 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:12:44.832 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:44.833 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:44.834 DEBUG:boto:Method: GET 2022-09-03T06:12:44.835 DEBUG:boto:Path: /dsgvcy-70/a 2022-09-03T06:12:44.836 DEBUG:boto:Data: 2022-09-03T06:12:44.837 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.837 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:44.838 DEBUG:boto:Port: 8001 2022-09-03T06:12:44.839 DEBUG:boto:Params: {} 2022-09-03T06:12:44.840 DEBUG:boto:Token: None 2022-09-03T06:12:44.841 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/a 2022-09-03T06:12:44.843 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ay60HD4O5SxDPqgdfc8ah7RAoeg= 2022-09-03T06:12:44.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ay60HD4O5SxDPqgdfc8ah7RAoeg=', 'Content-Length': '0'} 2022-09-03T06:12:44.846 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bf6b456fb98d33e9-006312f05c-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.890 DEBUG:boto:path=/dsgvcy-70/a 2022-09-03T06:12:44.891 DEBUG:boto:auth_path=/dsgvcy-70/a 2022-09-03T06:12:44.892 DEBUG:boto:Method: GET 2022-09-03T06:12:44.893 DEBUG:boto:Path: /dsgvcy-70/a 2022-09-03T06:12:44.893 DEBUG:boto:Data: 2022-09-03T06:12:44.894 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.895 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:44.895 DEBUG:boto:Port: 8000 2022-09-03T06:12:44.896 DEBUG:boto:Params: {} 2022-09-03T06:12:44.896 DEBUG:boto:Token: None 2022-09-03T06:12:44.897 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/a 2022-09-03T06:12:44.898 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ay60HD4O5SxDPqgdfc8ah7RAoeg= 2022-09-03T06:12:44.898 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ay60HD4O5SxDPqgdfc8ah7RAoeg=', 'Content-Length': '0'} 2022-09-03T06:12:44.901 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007801cce21c5ea2aa-006312f05c-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.903 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:44.904 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:44.905 DEBUG:boto:path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:44.906 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:44.907 DEBUG:boto:Method: GET 2022-09-03T06:12:44.910 DEBUG:boto:Path: /dsgvcy-70/?prefix=a 2022-09-03T06:12:44.911 DEBUG:boto:Data: 2022-09-03T06:12:44.912 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.917 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:44.918 DEBUG:boto:Port: 8001 2022-09-03T06:12:44.919 DEBUG:boto:Params: {} 2022-09-03T06:12:44.919 DEBUG:boto:Token: None 2022-09-03T06:12:44.920 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/ 2022-09-03T06:12:44.921 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PF6ENIbXAAVQrrJxckKSn8xbCgE= 2022-09-03T06:12:44.922 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PF6ENIbXAAVQrrJxckKSn8xbCgE=', 'Content-Length': '0'} 2022-09-03T06:12:44.925 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008a77b4da48282e96-006312f05c-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.926 DEBUG:boto:b'dsgvcy-70a1000falsea2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:44.927 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:44.928 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:44.928 DEBUG:boto:path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:44.929 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=a 2022-09-03T06:12:44.929 DEBUG:boto:Method: GET 2022-09-03T06:12:44.930 DEBUG:boto:Path: /dsgvcy-70/?prefix=a 2022-09-03T06:12:44.931 DEBUG:boto:Data: 2022-09-03T06:12:44.932 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.933 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:44.933 DEBUG:boto:Port: 8000 2022-09-03T06:12:44.934 DEBUG:boto:Params: {} 2022-09-03T06:12:44.935 DEBUG:boto:Token: None 2022-09-03T06:12:44.935 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/ 2022-09-03T06:12:44.936 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PF6ENIbXAAVQrrJxckKSn8xbCgE= 2022-09-03T06:12:44.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PF6ENIbXAAVQrrJxckKSn8xbCgE=', 'Content-Length': '0'} 2022-09-03T06:12:44.939 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cf00770c1310f249-006312f05c-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.940 DEBUG:boto:b'dsgvcy-70a1000falsea2022-09-03T06:12:05.237Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:44.942 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:12:44.944 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:44.945 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:44.946 DEBUG:boto:path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:44.948 DEBUG:boto:auth_path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:44.949 DEBUG:boto:Method: GET 2022-09-03T06:12:44.950 DEBUG:boto:Path: /dsgvcy-70/b?versionId=null 2022-09-03T06:12:44.951 DEBUG:boto:Data: 2022-09-03T06:12:44.952 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.953 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:44.954 DEBUG:boto:Port: 8001 2022-09-03T06:12:44.955 DEBUG:boto:Params: {} 2022-09-03T06:12:44.956 DEBUG:boto:Token: None 2022-09-03T06:12:44.957 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/b?versionId=null 2022-09-03T06:12:44.958 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YL1tHBYiFTOFHafk4r2P3M2I1CY= 2022-09-03T06:12:44.960 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YL1tHBYiFTOFHafk4r2P3M2I1CY=', 'Content-Length': '0'} 2022-09-03T06:12:44.964 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003ff51a48e94db900-006312f05c-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.967 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:44.968 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:44.970 DEBUG:boto:path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:44.971 DEBUG:boto:auth_path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:44.972 DEBUG:boto:Method: GET 2022-09-03T06:12:44.973 DEBUG:boto:Path: /dsgvcy-70/b?versionId=null 2022-09-03T06:12:44.974 DEBUG:boto:Data: 2022-09-03T06:12:44.975 DEBUG:boto:Headers: {} 2022-09-03T06:12:44.976 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:44.977 DEBUG:boto:Port: 8000 2022-09-03T06:12:44.978 DEBUG:boto:Params: {} 2022-09-03T06:12:44.980 DEBUG:boto:Token: None 2022-09-03T06:12:44.981 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:44 GMT /dsgvcy-70/b?versionId=null 2022-09-03T06:12:44.986 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YL1tHBYiFTOFHafk4r2P3M2I1CY= 2022-09-03T06:12:44.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YL1tHBYiFTOFHafk4r2P3M2I1CY=', 'Content-Length': '0'} 2022-09-03T06:12:44.992 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d01eb91ae19db8c5-006312f05c-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:44.994 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:44.995 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:44.996 DEBUG:boto:path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:44.997 DEBUG:boto:auth_path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:44.998 DEBUG:boto:Method: HEAD 2022-09-03T06:12:44.999 DEBUG:boto:Path: /dsgvcy-70/b?versionId=null 2022-09-03T06:12:45.000 DEBUG:boto:Data: 2022-09-03T06:12:45.000 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.001 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:45.003 DEBUG:boto:Port: 8001 2022-09-03T06:12:45.003 DEBUG:boto:Params: {} 2022-09-03T06:12:45.004 DEBUG:boto:Token: None 2022-09-03T06:12:45.005 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/b?versionId=null 2022-09-03T06:12:45.006 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:onhJrWHMdaVdPeBK593cCQudF6w= 2022-09-03T06:12:45.007 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:onhJrWHMdaVdPeBK593cCQudF6w=', 'Content-Length': '0'} 2022-09-03T06:12:45.009 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c1d4987d517f6270-006312f05d-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.010 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:45.011 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:45.011 DEBUG:boto:path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:45.012 DEBUG:boto:auth_path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:45.013 DEBUG:boto:Method: HEAD 2022-09-03T06:12:45.014 DEBUG:boto:Path: /dsgvcy-70/b?versionId=null 2022-09-03T06:12:45.015 DEBUG:boto:Data: 2022-09-03T06:12:45.016 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.017 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:45.017 DEBUG:boto:Port: 8000 2022-09-03T06:12:45.018 DEBUG:boto:Params: {} 2022-09-03T06:12:45.020 DEBUG:boto:Token: None 2022-09-03T06:12:45.021 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/b?versionId=null 2022-09-03T06:12:45.022 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:onhJrWHMdaVdPeBK593cCQudF6w= 2022-09-03T06:12:45.025 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:onhJrWHMdaVdPeBK593cCQudF6w=', 'Content-Length': '0'} 2022-09-03T06:12:45.028 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e0f1e1314c2ae408-006312f05d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.029 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:12:45.030 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:45.031 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:45.032 DEBUG:boto:path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:45.033 DEBUG:boto:auth_path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:45.033 DEBUG:boto:Method: GET 2022-09-03T06:12:45.039 DEBUG:boto:Path: /dsgvcy-70/b?versionId=null 2022-09-03T06:12:45.039 DEBUG:boto:Data: 2022-09-03T06:12:45.040 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.041 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:45.041 DEBUG:boto:Port: 8001 2022-09-03T06:12:45.042 DEBUG:boto:Params: {} 2022-09-03T06:12:45.044 DEBUG:boto:Token: None 2022-09-03T06:12:45.045 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/b?versionId=null 2022-09-03T06:12:45.045 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f3Fyoo+q0MvHW/6k1dGMlg89T94= 2022-09-03T06:12:45.046 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f3Fyoo+q0MvHW/6k1dGMlg89T94=', 'Content-Length': '0'} 2022-09-03T06:12:45.048 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000071533bd7d9b090ec-006312f05d-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.050 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:45.053 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:45.055 DEBUG:boto:path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:45.056 DEBUG:boto:auth_path=/dsgvcy-70/b?versionId=null 2022-09-03T06:12:45.057 DEBUG:boto:Method: GET 2022-09-03T06:12:45.060 DEBUG:boto:Path: /dsgvcy-70/b?versionId=null 2022-09-03T06:12:45.061 DEBUG:boto:Data: 2022-09-03T06:12:45.062 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.065 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:45.065 DEBUG:boto:Port: 8000 2022-09-03T06:12:45.066 DEBUG:boto:Params: {} 2022-09-03T06:12:45.067 DEBUG:boto:Token: None 2022-09-03T06:12:45.068 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/b?versionId=null 2022-09-03T06:12:45.070 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f3Fyoo+q0MvHW/6k1dGMlg89T94= 2022-09-03T06:12:45.071 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f3Fyoo+q0MvHW/6k1dGMlg89T94=', 'Content-Length': '0'} 2022-09-03T06:12:45.076 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000c63e5333329a819-006312f05d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.078 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:45.079 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:45.081 DEBUG:boto:path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:45.082 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:45.083 DEBUG:boto:Method: GET 2022-09-03T06:12:45.085 DEBUG:boto:Path: /dsgvcy-70/?prefix=b 2022-09-03T06:12:45.086 DEBUG:boto:Data: 2022-09-03T06:12:45.087 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.088 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:45.089 DEBUG:boto:Port: 8001 2022-09-03T06:12:45.091 DEBUG:boto:Params: {} 2022-09-03T06:12:45.092 DEBUG:boto:Token: None 2022-09-03T06:12:45.093 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/ 2022-09-03T06:12:45.094 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0= 2022-09-03T06:12:45.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0=', 'Content-Length': '0'} 2022-09-03T06:12:45.099 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006068056f98db4bdc-006312f05d-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.100 DEBUG:boto:b'dsgvcy-70b1000falseb2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:45.101 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:45.101 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:45.102 DEBUG:boto:path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:45.103 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:45.103 DEBUG:boto:Method: GET 2022-09-03T06:12:45.104 DEBUG:boto:Path: /dsgvcy-70/?prefix=b 2022-09-03T06:12:45.105 DEBUG:boto:Data: 2022-09-03T06:12:45.107 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.107 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:45.108 DEBUG:boto:Port: 8000 2022-09-03T06:12:45.108 DEBUG:boto:Params: {} 2022-09-03T06:12:45.109 DEBUG:boto:Token: None 2022-09-03T06:12:45.111 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/ 2022-09-03T06:12:45.112 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0= 2022-09-03T06:12:45.113 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0=', 'Content-Length': '0'} 2022-09-03T06:12:45.117 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006599545b547ea3c2-006312f05d-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.118 DEBUG:boto:b'dsgvcy-70b1000falseb2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:45.119 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:45.119 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:45.120 DEBUG:boto:Method: HEAD 2022-09-03T06:12:45.120 DEBUG:boto:Path: /dsgvcy-70/b 2022-09-03T06:12:45.121 DEBUG:boto:Data: 2022-09-03T06:12:45.122 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.122 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:45.123 DEBUG:boto:Port: 8001 2022-09-03T06:12:45.123 DEBUG:boto:Params: {} 2022-09-03T06:12:45.124 DEBUG:boto:Token: None 2022-09-03T06:12:45.125 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/b 2022-09-03T06:12:45.127 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hYhjQuRRfhjLd4lX2OK13uuLIfI= 2022-09-03T06:12:45.127 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hYhjQuRRfhjLd4lX2OK13uuLIfI=', 'Content-Length': '0'} 2022-09-03T06:12:45.130 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e343c3ffdee6219a-006312f05d-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.132 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:45.133 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:45.137 DEBUG:boto:Method: HEAD 2022-09-03T06:12:45.138 DEBUG:boto:Path: /dsgvcy-70/b 2022-09-03T06:12:45.139 DEBUG:boto:Data: 2022-09-03T06:12:45.139 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.140 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:45.140 DEBUG:boto:Port: 8000 2022-09-03T06:12:45.141 DEBUG:boto:Params: {} 2022-09-03T06:12:45.142 DEBUG:boto:Token: None 2022-09-03T06:12:45.142 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/b 2022-09-03T06:12:45.143 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hYhjQuRRfhjLd4lX2OK13uuLIfI= 2022-09-03T06:12:45.143 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hYhjQuRRfhjLd4lX2OK13uuLIfI=', 'Content-Length': '0'} 2022-09-03T06:12:45.146 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000965cd6cbf8c2e0be-006312f05d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.147 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:12:45.148 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:45.148 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:45.149 DEBUG:boto:Method: GET 2022-09-03T06:12:45.150 DEBUG:boto:Path: /dsgvcy-70/b 2022-09-03T06:12:45.150 DEBUG:boto:Data: 2022-09-03T06:12:45.151 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.151 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:45.152 DEBUG:boto:Port: 8001 2022-09-03T06:12:45.152 DEBUG:boto:Params: {} 2022-09-03T06:12:45.153 DEBUG:boto:Token: None 2022-09-03T06:12:45.154 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/b 2022-09-03T06:12:45.154 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Cp/lQgmpgeLxOiWQ0Wi8plQh22M= 2022-09-03T06:12:45.155 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Cp/lQgmpgeLxOiWQ0Wi8plQh22M=', 'Content-Length': '0'} 2022-09-03T06:12:45.157 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cf8b221d30b018c1-006312f05d-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.198 DEBUG:boto:path=/dsgvcy-70/b 2022-09-03T06:12:45.199 DEBUG:boto:auth_path=/dsgvcy-70/b 2022-09-03T06:12:45.199 DEBUG:boto:Method: GET 2022-09-03T06:12:45.200 DEBUG:boto:Path: /dsgvcy-70/b 2022-09-03T06:12:45.201 DEBUG:boto:Data: 2022-09-03T06:12:45.201 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.202 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:45.202 DEBUG:boto:Port: 8000 2022-09-03T06:12:45.203 DEBUG:boto:Params: {} 2022-09-03T06:12:45.204 DEBUG:boto:Token: None 2022-09-03T06:12:45.204 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/b 2022-09-03T06:12:45.205 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Cp/lQgmpgeLxOiWQ0Wi8plQh22M= 2022-09-03T06:12:45.205 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Cp/lQgmpgeLxOiWQ0Wi8plQh22M=', 'Content-Length': '0'} 2022-09-03T06:12:45.209 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f22d966f553406b-006312f05d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.210 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:45.210 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:45.211 DEBUG:boto:path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:45.212 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:45.212 DEBUG:boto:Method: GET 2022-09-03T06:12:45.213 DEBUG:boto:Path: /dsgvcy-70/?prefix=b 2022-09-03T06:12:45.213 DEBUG:boto:Data: 2022-09-03T06:12:45.214 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.214 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:45.215 DEBUG:boto:Port: 8001 2022-09-03T06:12:45.215 DEBUG:boto:Params: {} 2022-09-03T06:12:45.216 DEBUG:boto:Token: None 2022-09-03T06:12:45.217 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/ 2022-09-03T06:12:45.217 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0= 2022-09-03T06:12:45.218 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0=', 'Content-Length': '0'} 2022-09-03T06:12:45.224 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001b191b7bae52c34a-006312f05d-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.226 DEBUG:boto:b'dsgvcy-70b1000falseb2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:45.227 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:45.227 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:45.228 DEBUG:boto:path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:45.229 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=b 2022-09-03T06:12:45.230 DEBUG:boto:Method: GET 2022-09-03T06:12:45.230 DEBUG:boto:Path: /dsgvcy-70/?prefix=b 2022-09-03T06:12:45.231 DEBUG:boto:Data: 2022-09-03T06:12:45.231 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.232 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:45.233 DEBUG:boto:Port: 8000 2022-09-03T06:12:45.233 DEBUG:boto:Params: {} 2022-09-03T06:12:45.234 DEBUG:boto:Token: None 2022-09-03T06:12:45.235 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/ 2022-09-03T06:12:45.236 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0= 2022-09-03T06:12:45.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0=', 'Content-Length': '0'} 2022-09-03T06:12:45.239 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004a6e5508f969828b-006312f05d-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.240 DEBUG:boto:b'dsgvcy-70b1000falseb2022-09-03T06:12:05.268Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:45.241 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:12:45.242 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:45.251 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:45.252 DEBUG:boto:path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.253 DEBUG:boto:auth_path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.253 DEBUG:boto:Method: GET 2022-09-03T06:12:45.254 DEBUG:boto:Path: /dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.254 DEBUG:boto:Data: 2022-09-03T06:12:45.255 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.256 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:45.256 DEBUG:boto:Port: 8001 2022-09-03T06:12:45.257 DEBUG:boto:Params: {} 2022-09-03T06:12:45.257 DEBUG:boto:Token: None 2022-09-03T06:12:45.258 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.259 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ISBgG221xkCOSYy+Wq59+iqZFek= 2022-09-03T06:12:45.259 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ISBgG221xkCOSYy+Wq59+iqZFek=', 'Content-Length': '0'} 2022-09-03T06:12:45.262 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005cc5131a29bf7aea-006312f05d-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.263 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:45.263 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:45.264 DEBUG:boto:path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.264 DEBUG:boto:auth_path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.265 DEBUG:boto:Method: GET 2022-09-03T06:12:45.265 DEBUG:boto:Path: /dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.266 DEBUG:boto:Data: 2022-09-03T06:12:45.267 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.267 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:45.268 DEBUG:boto:Port: 8000 2022-09-03T06:12:45.268 DEBUG:boto:Params: {} 2022-09-03T06:12:45.269 DEBUG:boto:Token: None 2022-09-03T06:12:45.269 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.270 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ISBgG221xkCOSYy+Wq59+iqZFek= 2022-09-03T06:12:45.271 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ISBgG221xkCOSYy+Wq59+iqZFek=', 'Content-Length': '0'} 2022-09-03T06:12:45.273 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000055c96d76df3ee7fd-006312f05d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.275 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:45.275 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:45.276 DEBUG:boto:path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.277 DEBUG:boto:auth_path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.277 DEBUG:boto:Method: HEAD 2022-09-03T06:12:45.278 DEBUG:boto:Path: /dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.279 DEBUG:boto:Data: 2022-09-03T06:12:45.280 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.281 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:45.282 DEBUG:boto:Port: 8001 2022-09-03T06:12:45.283 DEBUG:boto:Params: {} 2022-09-03T06:12:45.284 DEBUG:boto:Token: None 2022-09-03T06:12:45.285 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.286 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0p3GD46I7PrsooX0+vsH+ArKPiA= 2022-09-03T06:12:45.287 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0p3GD46I7PrsooX0+vsH+ArKPiA=', 'Content-Length': '0'} 2022-09-03T06:12:45.290 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e47f5f86bab72ce4-006312f05d-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.292 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:45.293 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:45.293 DEBUG:boto:path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.294 DEBUG:boto:auth_path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.295 DEBUG:boto:Method: HEAD 2022-09-03T06:12:45.296 DEBUG:boto:Path: /dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.297 DEBUG:boto:Data: 2022-09-03T06:12:45.298 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.299 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:45.300 DEBUG:boto:Port: 8000 2022-09-03T06:12:45.301 DEBUG:boto:Params: {} 2022-09-03T06:12:45.302 DEBUG:boto:Token: None 2022-09-03T06:12:45.303 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.304 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0p3GD46I7PrsooX0+vsH+ArKPiA= 2022-09-03T06:12:45.306 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0p3GD46I7PrsooX0+vsH+ArKPiA=', 'Content-Length': '0'} 2022-09-03T06:12:45.309 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000063140c0865f5402e-006312f05d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.311 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:12:45.312 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:45.313 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:45.315 DEBUG:boto:path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.316 DEBUG:boto:auth_path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.317 DEBUG:boto:Method: GET 2022-09-03T06:12:45.318 DEBUG:boto:Path: /dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.319 DEBUG:boto:Data: 2022-09-03T06:12:45.320 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.321 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:45.321 DEBUG:boto:Port: 8001 2022-09-03T06:12:45.322 DEBUG:boto:Params: {} 2022-09-03T06:12:45.323 DEBUG:boto:Token: None 2022-09-03T06:12:45.324 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.325 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ISBgG221xkCOSYy+Wq59+iqZFek= 2022-09-03T06:12:45.326 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ISBgG221xkCOSYy+Wq59+iqZFek=', 'Content-Length': '0'} 2022-09-03T06:12:45.329 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e2e43a5e5fee72f-006312f05d-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.331 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:45.331 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:45.332 DEBUG:boto:path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.333 DEBUG:boto:auth_path=/dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.334 DEBUG:boto:Method: GET 2022-09-03T06:12:45.335 DEBUG:boto:Path: /dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.335 DEBUG:boto:Data: 2022-09-03T06:12:45.336 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.339 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:45.340 DEBUG:boto:Port: 8000 2022-09-03T06:12:45.341 DEBUG:boto:Params: {} 2022-09-03T06:12:45.342 DEBUG:boto:Token: None 2022-09-03T06:12:45.343 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/c?versionId=null 2022-09-03T06:12:45.344 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ISBgG221xkCOSYy+Wq59+iqZFek= 2022-09-03T06:12:45.346 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ISBgG221xkCOSYy+Wq59+iqZFek=', 'Content-Length': '0'} 2022-09-03T06:12:45.349 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000028029070e0a84000-006312f05d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.351 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:45.352 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:45.353 DEBUG:boto:path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:45.354 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:45.355 DEBUG:boto:Method: GET 2022-09-03T06:12:45.356 DEBUG:boto:Path: /dsgvcy-70/?prefix=c 2022-09-03T06:12:45.357 DEBUG:boto:Data: 2022-09-03T06:12:45.358 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.359 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:45.360 DEBUG:boto:Port: 8001 2022-09-03T06:12:45.360 DEBUG:boto:Params: {} 2022-09-03T06:12:45.361 DEBUG:boto:Token: None 2022-09-03T06:12:45.363 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/ 2022-09-03T06:12:45.364 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0= 2022-09-03T06:12:45.365 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0=', 'Content-Length': '0'} 2022-09-03T06:12:45.368 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a62a9224d119a3ae-006312f05d-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.370 DEBUG:boto:b'dsgvcy-70c1000falsec2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:45.371 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:45.372 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:45.373 DEBUG:boto:path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:45.374 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:45.375 DEBUG:boto:Method: GET 2022-09-03T06:12:45.376 DEBUG:boto:Path: /dsgvcy-70/?prefix=c 2022-09-03T06:12:45.377 DEBUG:boto:Data: 2022-09-03T06:12:45.378 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.380 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:45.381 DEBUG:boto:Port: 8000 2022-09-03T06:12:45.381 DEBUG:boto:Params: {} 2022-09-03T06:12:45.382 DEBUG:boto:Token: None 2022-09-03T06:12:45.383 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/ 2022-09-03T06:12:45.384 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0= 2022-09-03T06:12:45.384 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0=', 'Content-Length': '0'} 2022-09-03T06:12:45.388 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000393f38b3e64b3e0c-006312f05d-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.389 DEBUG:boto:b'dsgvcy-70c1000falsec2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:45.391 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:45.392 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:45.393 DEBUG:boto:Method: HEAD 2022-09-03T06:12:45.394 DEBUG:boto:Path: /dsgvcy-70/c 2022-09-03T06:12:45.394 DEBUG:boto:Data: 2022-09-03T06:12:45.395 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.396 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:45.397 DEBUG:boto:Port: 8001 2022-09-03T06:12:45.398 DEBUG:boto:Params: {} 2022-09-03T06:12:45.399 DEBUG:boto:Token: None 2022-09-03T06:12:45.399 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/c 2022-09-03T06:12:45.400 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:es1QOlrjlZV35b0HmxjekxxuuhA= 2022-09-03T06:12:45.402 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:es1QOlrjlZV35b0HmxjekxxuuhA=', 'Content-Length': '0'} 2022-09-03T06:12:45.405 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008253a4b57978ce30-006312f05d-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.407 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:45.408 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:45.409 DEBUG:boto:Method: HEAD 2022-09-03T06:12:45.410 DEBUG:boto:Path: /dsgvcy-70/c 2022-09-03T06:12:45.411 DEBUG:boto:Data: 2022-09-03T06:12:45.412 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.413 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:45.414 DEBUG:boto:Port: 8000 2022-09-03T06:12:45.414 DEBUG:boto:Params: {} 2022-09-03T06:12:45.415 DEBUG:boto:Token: None 2022-09-03T06:12:45.415 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/c 2022-09-03T06:12:45.416 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:es1QOlrjlZV35b0HmxjekxxuuhA= 2022-09-03T06:12:45.417 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:es1QOlrjlZV35b0HmxjekxxuuhA=', 'Content-Length': '0'} 2022-09-03T06:12:45.419 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d02daceb8cb3e499-006312f05d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.420 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:12:45.420 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:45.421 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:45.421 DEBUG:boto:Method: GET 2022-09-03T06:12:45.422 DEBUG:boto:Path: /dsgvcy-70/c 2022-09-03T06:12:45.423 DEBUG:boto:Data: 2022-09-03T06:12:45.423 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.424 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:45.424 DEBUG:boto:Port: 8001 2022-09-03T06:12:45.425 DEBUG:boto:Params: {} 2022-09-03T06:12:45.426 DEBUG:boto:Token: None 2022-09-03T06:12:45.426 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/c 2022-09-03T06:12:45.427 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NxPwHgplrl2FvvjXWBCbhnORQrk= 2022-09-03T06:12:45.427 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NxPwHgplrl2FvvjXWBCbhnORQrk=', 'Content-Length': '0'} 2022-09-03T06:12:45.430 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000093612056cda240dd-006312f05d-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.433 DEBUG:boto:path=/dsgvcy-70/c 2022-09-03T06:12:45.434 DEBUG:boto:auth_path=/dsgvcy-70/c 2022-09-03T06:12:45.435 DEBUG:boto:Method: GET 2022-09-03T06:12:45.435 DEBUG:boto:Path: /dsgvcy-70/c 2022-09-03T06:12:45.436 DEBUG:boto:Data: 2022-09-03T06:12:45.436 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.437 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:45.437 DEBUG:boto:Port: 8000 2022-09-03T06:12:45.438 DEBUG:boto:Params: {} 2022-09-03T06:12:45.439 DEBUG:boto:Token: None 2022-09-03T06:12:45.439 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/c 2022-09-03T06:12:45.440 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NxPwHgplrl2FvvjXWBCbhnORQrk= 2022-09-03T06:12:45.441 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NxPwHgplrl2FvvjXWBCbhnORQrk=', 'Content-Length': '0'} 2022-09-03T06:12:45.443 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007ed93f5d563af3e2-006312f05d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.487 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:45.488 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:45.489 DEBUG:boto:path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:45.489 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:45.490 DEBUG:boto:Method: GET 2022-09-03T06:12:45.491 DEBUG:boto:Path: /dsgvcy-70/?prefix=c 2022-09-03T06:12:45.493 DEBUG:boto:Data: 2022-09-03T06:12:45.493 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.494 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:45.494 DEBUG:boto:Port: 8001 2022-09-03T06:12:45.495 DEBUG:boto:Params: {} 2022-09-03T06:12:45.495 DEBUG:boto:Token: None 2022-09-03T06:12:45.496 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/ 2022-09-03T06:12:45.497 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0= 2022-09-03T06:12:45.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0=', 'Content-Length': '0'} 2022-09-03T06:12:45.509 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c004df00ff890d17-006312f05d-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.509 DEBUG:boto:b'dsgvcy-70c1000falsec2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:45.510 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:45.511 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:45.512 DEBUG:boto:path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:45.512 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=c 2022-09-03T06:12:45.513 DEBUG:boto:Method: GET 2022-09-03T06:12:45.513 DEBUG:boto:Path: /dsgvcy-70/?prefix=c 2022-09-03T06:12:45.514 DEBUG:boto:Data: 2022-09-03T06:12:45.514 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.515 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:45.515 DEBUG:boto:Port: 8000 2022-09-03T06:12:45.516 DEBUG:boto:Params: {} 2022-09-03T06:12:45.517 DEBUG:boto:Token: None 2022-09-03T06:12:45.517 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/ 2022-09-03T06:12:45.518 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0= 2022-09-03T06:12:45.519 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0=', 'Content-Length': '0'} 2022-09-03T06:12:45.525 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000d1c878ee39dab6f-006312f05d-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.526 DEBUG:boto:b'dsgvcy-70c1000falsec2022-09-03T06:12:05.331Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:45.527 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:12:45.528 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:45.528 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:45.529 DEBUG:boto:path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.529 DEBUG:boto:auth_path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.530 DEBUG:boto:Method: GET 2022-09-03T06:12:45.530 DEBUG:boto:Path: /dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.531 DEBUG:boto:Data: 2022-09-03T06:12:45.532 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.532 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:45.533 DEBUG:boto:Port: 8001 2022-09-03T06:12:45.533 DEBUG:boto:Params: {} 2022-09-03T06:12:45.534 DEBUG:boto:Token: None 2022-09-03T06:12:45.534 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.535 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KqkJyv5ODtJsGB0480+9x0ND4CU= 2022-09-03T06:12:45.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KqkJyv5ODtJsGB0480+9x0ND4CU=', 'Content-Length': '0'} 2022-09-03T06:12:45.538 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d640f46babdd577a-006312f05d-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.543 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:45.544 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:45.546 DEBUG:boto:path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.547 DEBUG:boto:auth_path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.548 DEBUG:boto:Method: GET 2022-09-03T06:12:45.549 DEBUG:boto:Path: /dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.550 DEBUG:boto:Data: 2022-09-03T06:12:45.551 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.552 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:45.553 DEBUG:boto:Port: 8000 2022-09-03T06:12:45.555 DEBUG:boto:Params: {} 2022-09-03T06:12:45.556 DEBUG:boto:Token: None 2022-09-03T06:12:45.557 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.558 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KqkJyv5ODtJsGB0480+9x0ND4CU= 2022-09-03T06:12:45.560 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KqkJyv5ODtJsGB0480+9x0ND4CU=', 'Content-Length': '0'} 2022-09-03T06:12:45.565 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000560b6239b28f719f-006312f05d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.567 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:45.567 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:45.569 DEBUG:boto:path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.569 DEBUG:boto:auth_path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.570 DEBUG:boto:Method: HEAD 2022-09-03T06:12:45.570 DEBUG:boto:Path: /dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.571 DEBUG:boto:Data: 2022-09-03T06:12:45.571 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.572 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:45.572 DEBUG:boto:Port: 8001 2022-09-03T06:12:45.573 DEBUG:boto:Params: {} 2022-09-03T06:12:45.574 DEBUG:boto:Token: None 2022-09-03T06:12:45.575 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.576 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yHQVBt5cu27r/bttuapbt2JCJgo= 2022-09-03T06:12:45.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yHQVBt5cu27r/bttuapbt2JCJgo=', 'Content-Length': '0'} 2022-09-03T06:12:45.578 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009082c07bae4d35f8-006312f05d-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.580 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:45.581 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:45.582 DEBUG:boto:path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.583 DEBUG:boto:auth_path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.584 DEBUG:boto:Method: HEAD 2022-09-03T06:12:45.587 DEBUG:boto:Path: /dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.587 DEBUG:boto:Data: 2022-09-03T06:12:45.588 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.589 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:45.590 DEBUG:boto:Port: 8000 2022-09-03T06:12:45.591 DEBUG:boto:Params: {} 2022-09-03T06:12:45.592 DEBUG:boto:Token: None 2022-09-03T06:12:45.593 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.595 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yHQVBt5cu27r/bttuapbt2JCJgo= 2022-09-03T06:12:45.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yHQVBt5cu27r/bttuapbt2JCJgo=', 'Content-Length': '0'} 2022-09-03T06:12:45.599 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eb7c90f650338cca-006312f05d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.600 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:12:45.601 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:45.601 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:45.602 DEBUG:boto:path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.602 DEBUG:boto:auth_path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.603 DEBUG:boto:Method: GET 2022-09-03T06:12:45.604 DEBUG:boto:Path: /dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.605 DEBUG:boto:Data: 2022-09-03T06:12:45.605 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.607 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:45.608 DEBUG:boto:Port: 8001 2022-09-03T06:12:45.608 DEBUG:boto:Params: {} 2022-09-03T06:12:45.609 DEBUG:boto:Token: None 2022-09-03T06:12:45.610 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.610 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KqkJyv5ODtJsGB0480+9x0ND4CU= 2022-09-03T06:12:45.611 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KqkJyv5ODtJsGB0480+9x0ND4CU=', 'Content-Length': '0'} 2022-09-03T06:12:45.614 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b3b294f053fb831-006312f05d-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.654 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:45.655 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:45.656 DEBUG:boto:path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.656 DEBUG:boto:auth_path=/dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.657 DEBUG:boto:Method: GET 2022-09-03T06:12:45.657 DEBUG:boto:Path: /dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.658 DEBUG:boto:Data: 2022-09-03T06:12:45.659 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.659 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:45.660 DEBUG:boto:Port: 8000 2022-09-03T06:12:45.660 DEBUG:boto:Params: {} 2022-09-03T06:12:45.661 DEBUG:boto:Token: None 2022-09-03T06:12:45.662 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/d?versionId=null 2022-09-03T06:12:45.663 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KqkJyv5ODtJsGB0480+9x0ND4CU= 2022-09-03T06:12:45.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KqkJyv5ODtJsGB0480+9x0ND4CU=', 'Content-Length': '0'} 2022-09-03T06:12:45.666 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000def43d316ded79fd-006312f05d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.667 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:45.668 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:45.669 DEBUG:boto:path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:45.669 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:45.670 DEBUG:boto:Method: GET 2022-09-03T06:12:45.670 DEBUG:boto:Path: /dsgvcy-70/?prefix=d 2022-09-03T06:12:45.671 DEBUG:boto:Data: 2022-09-03T06:12:45.672 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.672 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:45.673 DEBUG:boto:Port: 8001 2022-09-03T06:12:45.674 DEBUG:boto:Params: {} 2022-09-03T06:12:45.675 DEBUG:boto:Token: None 2022-09-03T06:12:45.676 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/ 2022-09-03T06:12:45.676 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0= 2022-09-03T06:12:45.677 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0=', 'Content-Length': '0'} 2022-09-03T06:12:45.680 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006f8bcbb5c7a8bf45-006312f05d-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.681 DEBUG:boto:b'dsgvcy-70d1000falsed2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:45.682 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:45.683 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:45.683 DEBUG:boto:path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:45.684 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:45.684 DEBUG:boto:Method: GET 2022-09-03T06:12:45.685 DEBUG:boto:Path: /dsgvcy-70/?prefix=d 2022-09-03T06:12:45.686 DEBUG:boto:Data: 2022-09-03T06:12:45.687 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.687 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:45.688 DEBUG:boto:Port: 8000 2022-09-03T06:12:45.688 DEBUG:boto:Params: {} 2022-09-03T06:12:45.689 DEBUG:boto:Token: None 2022-09-03T06:12:45.690 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/ 2022-09-03T06:12:45.691 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0= 2022-09-03T06:12:45.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0=', 'Content-Length': '0'} 2022-09-03T06:12:45.694 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000699ab1fa2fcc14d4-006312f05d-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.695 DEBUG:boto:b'dsgvcy-70d1000falsed2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:45.696 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:45.697 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:45.697 DEBUG:boto:Method: HEAD 2022-09-03T06:12:45.698 DEBUG:boto:Path: /dsgvcy-70/d 2022-09-03T06:12:45.698 DEBUG:boto:Data: 2022-09-03T06:12:45.699 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.699 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:45.700 DEBUG:boto:Port: 8001 2022-09-03T06:12:45.700 DEBUG:boto:Params: {} 2022-09-03T06:12:45.701 DEBUG:boto:Token: None 2022-09-03T06:12:45.702 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/d 2022-09-03T06:12:45.702 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CNNKh6XT9tU4UDlK/YoI4xdYVR8= 2022-09-03T06:12:45.703 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CNNKh6XT9tU4UDlK/YoI4xdYVR8=', 'Content-Length': '0'} 2022-09-03T06:12:45.706 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e003e780c09b648f-006312f05d-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.708 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:45.709 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:45.710 DEBUG:boto:Method: HEAD 2022-09-03T06:12:45.711 DEBUG:boto:Path: /dsgvcy-70/d 2022-09-03T06:12:45.711 DEBUG:boto:Data: 2022-09-03T06:12:45.712 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.713 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:45.714 DEBUG:boto:Port: 8000 2022-09-03T06:12:45.715 DEBUG:boto:Params: {} 2022-09-03T06:12:45.717 DEBUG:boto:Token: None 2022-09-03T06:12:45.718 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/d 2022-09-03T06:12:45.719 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CNNKh6XT9tU4UDlK/YoI4xdYVR8= 2022-09-03T06:12:45.722 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CNNKh6XT9tU4UDlK/YoI4xdYVR8=', 'Content-Length': '0'} 2022-09-03T06:12:45.724 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008ac71c1080ece829-006312f05d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.725 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:12:45.726 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:45.727 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:45.727 DEBUG:boto:Method: GET 2022-09-03T06:12:45.728 DEBUG:boto:Path: /dsgvcy-70/d 2022-09-03T06:12:45.729 DEBUG:boto:Data: 2022-09-03T06:12:45.730 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.731 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:45.732 DEBUG:boto:Port: 8001 2022-09-03T06:12:45.732 DEBUG:boto:Params: {} 2022-09-03T06:12:45.733 DEBUG:boto:Token: None 2022-09-03T06:12:45.733 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/d 2022-09-03T06:12:45.734 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vhf0sUNkylm/jlAUVeWB3dYV/CE= 2022-09-03T06:12:45.735 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vhf0sUNkylm/jlAUVeWB3dYV/CE=', 'Content-Length': '0'} 2022-09-03T06:12:45.737 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b6b0e1d352c9165-006312f05d-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.739 DEBUG:boto:path=/dsgvcy-70/d 2022-09-03T06:12:45.740 DEBUG:boto:auth_path=/dsgvcy-70/d 2022-09-03T06:12:45.741 DEBUG:boto:Method: GET 2022-09-03T06:12:45.742 DEBUG:boto:Path: /dsgvcy-70/d 2022-09-03T06:12:45.744 DEBUG:boto:Data: 2022-09-03T06:12:45.745 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.745 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:45.747 DEBUG:boto:Port: 8000 2022-09-03T06:12:45.748 DEBUG:boto:Params: {} 2022-09-03T06:12:45.749 DEBUG:boto:Token: None 2022-09-03T06:12:45.750 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/d 2022-09-03T06:12:45.751 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vhf0sUNkylm/jlAUVeWB3dYV/CE= 2022-09-03T06:12:45.751 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vhf0sUNkylm/jlAUVeWB3dYV/CE=', 'Content-Length': '0'} 2022-09-03T06:12:45.754 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001329f0f7006e61b4-006312f05d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.755 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:45.755 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:45.756 DEBUG:boto:path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:45.756 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:45.757 DEBUG:boto:Method: GET 2022-09-03T06:12:45.757 DEBUG:boto:Path: /dsgvcy-70/?prefix=d 2022-09-03T06:12:45.758 DEBUG:boto:Data: 2022-09-03T06:12:45.759 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.759 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:12:45.760 DEBUG:boto:Port: 8001 2022-09-03T06:12:45.760 DEBUG:boto:Params: {} 2022-09-03T06:12:45.761 DEBUG:boto:Token: None 2022-09-03T06:12:45.762 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/ 2022-09-03T06:12:45.762 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0= 2022-09-03T06:12:45.763 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0=', 'Content-Length': '0'} 2022-09-03T06:12:45.767 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000acf7310a7d8f4b24-006312f05d-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.768 DEBUG:boto:b'dsgvcy-70d1000falsed2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:45.769 DEBUG:boto:path=/dsgvcy-70/ 2022-09-03T06:12:45.772 DEBUG:boto:auth_path=/dsgvcy-70/ 2022-09-03T06:12:45.773 DEBUG:boto:path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:45.773 DEBUG:boto:auth_path=/dsgvcy-70/?prefix=d 2022-09-03T06:12:45.774 DEBUG:boto:Method: GET 2022-09-03T06:12:45.774 DEBUG:boto:Path: /dsgvcy-70/?prefix=d 2022-09-03T06:12:45.775 DEBUG:boto:Data: 2022-09-03T06:12:45.776 DEBUG:boto:Headers: {} 2022-09-03T06:12:45.776 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:12:45.777 DEBUG:boto:Port: 8000 2022-09-03T06:12:45.777 DEBUG:boto:Params: {} 2022-09-03T06:12:45.778 DEBUG:boto:Token: None 2022-09-03T06:12:45.779 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:12:45 GMT /dsgvcy-70/ 2022-09-03T06:12:45.779 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0= 2022-09-03T06:12:45.780 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ara8mwBHf8TJFxFm9F4Y8dn1fM0=', 'Content-Length': '0'} 2022-09-03T06:12:45.783 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000045e286211e1ef6d1-006312f05d-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:12:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:45.783 DEBUG:boto:b'dsgvcy-70d1000falsed2022-09-03T06:12:05.372Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:12:45.784 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-70 zones={test-zone3, test-zone2} 2022-09-03T06:12:45.785 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'autotrim', '--rgw-sync-log-trim-max-buckets', '1', '--rgw-sync-log-trim-min-cold-buckets', '0', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:12:45.785 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'autotrim', '--rgw-sync-log-trim-max-buckets', '1', '--rgw-sync-log-trim-min-cold-buckets', '0', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:12:45.786 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog autotrim --rgw-sync-log-trim-max-buckets 1 --rgw-sync-log-trim-min-cold-buckets 0 --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T06:12:45.841 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:45.841 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:45.915 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'list', '--bucket', 'dsgvcy-70', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:45.916 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'list', '--bucket', 'dsgvcy-70', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:45.917 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog list --bucket dsgvcy-70 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:12:45.967 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:45.968 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:46.011 INFO:teuthology.orchestra.run.smithi181.stdout:[] 2022-09-03T06:12:46.011 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'list', '--bucket', 'dsgvcy-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:46.012 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'list', '--bucket', 'dsgvcy-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:46.013 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog list --bucket dsgvcy-69 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:12:46.070 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:46.071 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:46.137 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T06:12:46.138 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:46.138 INFO:teuthology.orchestra.run.smithi181.stdout: "op_id": "1#00000000001.79.5", 2022-09-03T06:12:46.139 INFO:teuthology.orchestra.run.smithi181.stdout: "op_tag": "91786ef1-cdb4-4061-8a3e-2a28cc497285.5591.2960008450026606355", 2022-09-03T06:12:46.139 INFO:teuthology.orchestra.run.smithi181.stdout: "op": "write", 2022-09-03T06:12:46.140 INFO:teuthology.orchestra.run.smithi181.stdout: "object": "a", 2022-09-03T06:12:46.140 INFO:teuthology.orchestra.run.smithi181.stdout: "instance": "", 2022-09-03T06:12:46.141 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "complete", 2022-09-03T06:12:46.141 INFO:teuthology.orchestra.run.smithi181.stdout: "index_ver": 1, 2022-09-03T06:12:46.144 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:11:25.021814494Z", 2022-09-03T06:12:46.144 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": { 2022-09-03T06:12:46.144 INFO:teuthology.orchestra.run.smithi181.stdout: "pool": 9, 2022-09-03T06:12:46.145 INFO:teuthology.orchestra.run.smithi181.stdout: "epoch": 1 2022-09-03T06:12:46.145 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:46.145 INFO:teuthology.orchestra.run.smithi181.stdout: "bilog_flags": 0, 2022-09-03T06:12:46.145 INFO:teuthology.orchestra.run.smithi181.stdout: "versioned": false, 2022-09-03T06:12:46.146 INFO:teuthology.orchestra.run.smithi181.stdout: "owner": "", 2022-09-03T06:12:46.146 INFO:teuthology.orchestra.run.smithi181.stdout: "owner_display_name": "", 2022-09-03T06:12:46.147 INFO:teuthology.orchestra.run.smithi181.stdout: "zones_trace": [ 2022-09-03T06:12:46.147 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:46.148 INFO:teuthology.orchestra.run.smithi181.stdout: "entry": "91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-69:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.20" 2022-09-03T06:12:46.148 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:12:46.149 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:12:46.150 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:46.150 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:46.151 INFO:teuthology.orchestra.run.smithi181.stdout: "op_id": "5#00000000001.51.5", 2022-09-03T06:12:46.151 INFO:teuthology.orchestra.run.smithi181.stdout: "op_tag": "91786ef1-cdb4-4061-8a3e-2a28cc497285.5591.3689115947523818884", 2022-09-03T06:12:46.151 INFO:teuthology.orchestra.run.smithi181.stdout: "op": "write", 2022-09-03T06:12:46.152 INFO:teuthology.orchestra.run.smithi181.stdout: "object": "d", 2022-09-03T06:12:46.152 INFO:teuthology.orchestra.run.smithi181.stdout: "instance": "", 2022-09-03T06:12:46.153 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "complete", 2022-09-03T06:12:46.154 INFO:teuthology.orchestra.run.smithi181.stdout: "index_ver": 1, 2022-09-03T06:12:46.154 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:11:25.166064187Z", 2022-09-03T06:12:46.155 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": { 2022-09-03T06:12:46.155 INFO:teuthology.orchestra.run.smithi181.stdout: "pool": 9, 2022-09-03T06:12:46.155 INFO:teuthology.orchestra.run.smithi181.stdout: "epoch": 19 2022-09-03T06:12:46.155 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:46.156 INFO:teuthology.orchestra.run.smithi181.stdout: "bilog_flags": 0, 2022-09-03T06:12:46.156 INFO:teuthology.orchestra.run.smithi181.stdout: "versioned": false, 2022-09-03T06:12:46.156 INFO:teuthology.orchestra.run.smithi181.stdout: "owner": "", 2022-09-03T06:12:46.156 INFO:teuthology.orchestra.run.smithi181.stdout: "owner_display_name": "", 2022-09-03T06:12:46.157 INFO:teuthology.orchestra.run.smithi181.stdout: "zones_trace": [ 2022-09-03T06:12:46.157 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:46.159 INFO:teuthology.orchestra.run.smithi181.stdout: "entry": "91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-69:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.20" 2022-09-03T06:12:46.159 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:12:46.159 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:12:46.160 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:46.160 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:46.161 INFO:teuthology.orchestra.run.smithi181.stdout: "op_id": "8#00000000001.81.5", 2022-09-03T06:12:46.161 INFO:teuthology.orchestra.run.smithi181.stdout: "op_tag": "91786ef1-cdb4-4061-8a3e-2a28cc497285.5591.1298415193423508419", 2022-09-03T06:12:46.162 INFO:teuthology.orchestra.run.smithi181.stdout: "op": "write", 2022-09-03T06:12:46.163 INFO:teuthology.orchestra.run.smithi181.stdout: "object": "c", 2022-09-03T06:12:46.163 INFO:teuthology.orchestra.run.smithi181.stdout: "instance": "", 2022-09-03T06:12:46.163 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "complete", 2022-09-03T06:12:46.164 INFO:teuthology.orchestra.run.smithi181.stdout: "index_ver": 1, 2022-09-03T06:12:46.164 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:11:25.127463615Z", 2022-09-03T06:12:46.165 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": { 2022-09-03T06:12:46.165 INFO:teuthology.orchestra.run.smithi181.stdout: "pool": 9, 2022-09-03T06:12:46.165 INFO:teuthology.orchestra.run.smithi181.stdout: "epoch": 8 2022-09-03T06:12:46.166 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:46.166 INFO:teuthology.orchestra.run.smithi181.stdout: "bilog_flags": 0, 2022-09-03T06:12:46.166 INFO:teuthology.orchestra.run.smithi181.stdout: "versioned": false, 2022-09-03T06:12:46.166 INFO:teuthology.orchestra.run.smithi181.stdout: "owner": "", 2022-09-03T06:12:46.167 INFO:teuthology.orchestra.run.smithi181.stdout: "owner_display_name": "", 2022-09-03T06:12:46.167 INFO:teuthology.orchestra.run.smithi181.stdout: "zones_trace": [ 2022-09-03T06:12:46.167 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:46.167 INFO:teuthology.orchestra.run.smithi181.stdout: "entry": "91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-69:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.20" 2022-09-03T06:12:46.168 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:12:46.169 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:12:46.169 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:46.169 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:46.169 INFO:teuthology.orchestra.run.smithi181.stdout: "op_id": "10#00000000001.49.5", 2022-09-03T06:12:46.170 INFO:teuthology.orchestra.run.smithi181.stdout: "op_tag": "91786ef1-cdb4-4061-8a3e-2a28cc497285.5591.17395737289998004799", 2022-09-03T06:12:46.171 INFO:teuthology.orchestra.run.smithi181.stdout: "op": "write", 2022-09-03T06:12:46.171 INFO:teuthology.orchestra.run.smithi181.stdout: "object": "b", 2022-09-03T06:12:46.171 INFO:teuthology.orchestra.run.smithi181.stdout: "instance": "", 2022-09-03T06:12:46.172 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "complete", 2022-09-03T06:12:46.172 INFO:teuthology.orchestra.run.smithi181.stdout: "index_ver": 1, 2022-09-03T06:12:46.173 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:11:25.087436869Z", 2022-09-03T06:12:46.173 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": { 2022-09-03T06:12:46.174 INFO:teuthology.orchestra.run.smithi181.stdout: "pool": 9, 2022-09-03T06:12:46.174 INFO:teuthology.orchestra.run.smithi181.stdout: "epoch": 6 2022-09-03T06:12:46.174 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:46.175 INFO:teuthology.orchestra.run.smithi181.stdout: "bilog_flags": 0, 2022-09-03T06:12:46.175 INFO:teuthology.orchestra.run.smithi181.stdout: "versioned": false, 2022-09-03T06:12:46.176 INFO:teuthology.orchestra.run.smithi181.stdout: "owner": "", 2022-09-03T06:12:46.176 INFO:teuthology.orchestra.run.smithi181.stdout: "owner_display_name": "", 2022-09-03T06:12:46.177 INFO:teuthology.orchestra.run.smithi181.stdout: "zones_trace": [ 2022-09-03T06:12:46.177 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:46.178 INFO:teuthology.orchestra.run.smithi181.stdout: "entry": "91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-69:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.20" 2022-09-03T06:12:46.178 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:12:46.179 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:12:46.179 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:12:46.180 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T06:12:46.183 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'autotrim', '--rgw-sync-log-trim-max-buckets', '999', '--rgw-sync-log-trim-min-cold-buckets', '999', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:12:46.184 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'autotrim', '--rgw-sync-log-trim-max-buckets', '999', '--rgw-sync-log-trim-min-cold-buckets', '999', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:12:46.185 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog autotrim --rgw-sync-log-trim-max-buckets 999 --rgw-sync-log-trim-min-cold-buckets 999 --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T06:12:46.232 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:46.233 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:46.292 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.290+0000 7fab4e80a700 -1 req 17749788585774296015 0.001999958s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-43:e40d88ba-d542-483a-afa2-5de687affd14.5176.2,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-43:e40d88ba-d542-483a-afa2-5de687affd14.5176.2,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.293 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.290+0000 7fab277bc700 -1 req 15490591089229232571 0.001999958s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.293 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.290+0000 7fab2dfc9700 -1 req 12251740973298361404 0.001999958s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.294 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.291+0000 7f6c03acb700 -1 req 4439829535127510660 0.002999935s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.295 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.291+0000 7f6c03acb700 -1 req 1258404693718640214 0.002999935s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.296 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.291+0000 7fc7b964b700 -1 req 15447000725596482159 0.002999935s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.297 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.292+0000 7f6bf1aa7700 -1 req 2625344358841084959 0.003999914s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-43:e40d88ba-d542-483a-afa2-5de687affd14.5176.2,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-43:e40d88ba-d542-483a-afa2-5de687affd14.5176.2,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.297 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.292+0000 7fc7b4e42700 -1 req 9353562170898079149 0.003999914s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-43:e40d88ba-d542-483a-afa2-5de687affd14.5176.2,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-43:e40d88ba-d542-483a-afa2-5de687affd14.5176.2,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.298 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.292+0000 7fc7b8e4a700 -1 req 10984574900788549445 0.003999914s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.299 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.297+0000 7f6bc7a53700 -1 req 16474068722604601556 0.002999935s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-40:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.40,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-40:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.40,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.300 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.295+0000 7fab0d788700 -1 req 9444004579834310276 0.001999958s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-2:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.2,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-2:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.2,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.301 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.295+0000 7fab0cf87700 -1 req 2784182556954107746 0.000999979s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-40:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.40,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-40:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.40,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.302 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.297+0000 7fc7a1e1c700 -1 req 10232389961839402501 0.003999913s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-2:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.2,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-2:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.2,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.303 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.298+0000 7f6c012c6700 -1 req 17564985288250039773 0.004999892s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-2:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.2,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-2:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.2,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.303 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.298+0000 7fc79ee16700 -1 req 6366052462846079924 0.004999892s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-40:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.40,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-40:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.40,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.304 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.301+0000 7faafaf63700 -1 req 1110575864986138108 0.001999958s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-53:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.4,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-53:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.4,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.305 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.303+0000 7f6bf12a6700 -1 req 8223299228652881594 0.003999913s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-53:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.4,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-53:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.4,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.305 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.303+0000 7fc8927fd700 -1 req 11262538361414995853 0.003999913s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-53:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.4,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-53:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.4,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.347 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.342+0000 7f6bc0a45700 -1 req 17272728122590063684 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.347 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.342+0000 7f6b891d6700 -1 req 11064192582433208831 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.348 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.342+0000 7fc888fea700 -1 req 10870797756223592714 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.348 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.342+0000 7fc8897eb700 -1 req 17408098455078761232 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.349 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.342+0000 7fab0ef8b700 -1 req 1722508230098910787 0.001999958s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.350 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.342+0000 7fab0877e700 -1 req 8623315261200431661 0.001999958s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.388 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.387+0000 7fabe913f700 -1 req 11232047866511376591 0.001999958s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.390 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.388+0000 7f6bc424c700 -1 req 555036408649995076 0.002999935s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.390 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.388+0000 7fc86cfb2700 -1 req 15321807047712599594 0.002999935s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.398 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.397+0000 7fabe4936700 -1 req 11782443582517164743 0.001999958s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-55:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.6,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-55:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.6,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.399 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.397+0000 7fc864fa2700 -1 req 6823559942994572647 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-55:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.6,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-55:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.6,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.400 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.398+0000 7f6b6f1a2700 -1 req 4215919528496568046 0.002999935s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-55:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.6,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-55:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.6,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.443 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.441+0000 7fabc9900700 -1 req 6375083269191619114 0.000999979s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-1:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.1,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-1:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.1,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.444 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.441+0000 7fc854781700 -1 req 18107705519806268313 0.000999978s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-1:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.1,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-1:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.1,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.444 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.441+0000 7f6b829c9700 -1 req 9119302068935701847 0.000999978s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-1:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.1,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-1:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.1,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.453 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.448+0000 7f6b6418c700 -1 req 3545812832113769629 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-5:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.5,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-5:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.5,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.454 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.447+0000 7fabca101700 -1 req 4774967534565550062 0.000999978s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-5:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.5,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-5:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.5,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.455 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.448+0000 7fc84c771700 -1 req 7591460479239771397 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-5:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.5,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-5:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.5,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.483 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.481+0000 7fc849f6c700 -1 req 10093942682111443069 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-54:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.5,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-54:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.5,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.483 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.481+0000 7f6b7f9c3700 -1 req 2284507631940620076 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-54:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.5,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-54:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.5,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.484 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.482+0000 7fabdc926700 -1 req 3291915839977953104 0.002999938s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-54:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.5,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-54:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.5,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.486 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.484+0000 7fabe6139700 -1 req 15559561319744732172 0.000999979s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.486 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.484+0000 7f6b9b9fb700 -1 req 9886216238895300009 0.000999979s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.487 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.484+0000 7fc884fe2700 -1 req 1920926538760997431 0.000999979s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.527 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.526+0000 7fabbd8e8700 -1 req 502741654445594180 0.001999959s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-42:e40d88ba-d542-483a-afa2-5de687affd14.5176.1,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-42:e40d88ba-d542-483a-afa2-5de687affd14.5176.1,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.529 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.526+0000 7fc83373f700 -1 req 1787087943182992958 0.001999956s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-42:e40d88ba-d542-483a-afa2-5de687affd14.5176.1,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-42:e40d88ba-d542-483a-afa2-5de687affd14.5176.1,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.529 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.526+0000 7f6b881d4700 -1 req 1992643619926110574 0.001999956s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-42:e40d88ba-d542-483a-afa2-5de687affd14.5176.1,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-42:e40d88ba-d542-483a-afa2-5de687affd14.5176.1,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.531 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.529+0000 7fabc28f2700 -1 req 10281530538245767080 0.000999979s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-67:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.18,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-67:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.18,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.532 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.530+0000 7fc82c731700 -1 req 11716070128877088073 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-67:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.18,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-67:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.18,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.532 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.530+0000 7f6b739ab700 -1 req 15353539562388298154 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-67:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.18,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-67:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.18,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.533 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.532+0000 7fabbf0eb700 -1 req 1061307407075281026 0.001999958s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-3:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.3,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-3:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.3,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.534 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.532+0000 7f6b741ac700 -1 req 17034728994703746887 0.001999956s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-3:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.3,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-3:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.3,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.535 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.532+0000 7fc826f26700 -1 req 11470885488097101303 0.001999956s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-3:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.3,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-3:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.3,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.575 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.574+0000 7faba18b0700 -1 req 16933237262141083820 0.001999958s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-50:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.1,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-50:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.1,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.577 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.574+0000 7fc80def4700 -1 req 2133254602998105533 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-50:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.1,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-50:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.1,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.578 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.575+0000 7fabb18d0700 -1 req 15581478026464250598 0.000999979s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.579 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.574+0000 7f6b46150700 -1 req 1719220791307747202 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-50:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.1,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-50:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.1,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.579 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.576+0000 7f6b30925700 -1 req 3780500808508904009 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.580 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.576+0000 7fc81df14700 -1 req 17175539469830356893 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.670 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.669+0000 7fab84876700 -1 req 1237965428655861664 0.001999958s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-27:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.27,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-27:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.27,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.672 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.669+0000 7fc7ea6ad700 -1 req 9128059979513658803 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-27:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.27,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-27:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.27,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.673 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.670+0000 7fab8a882700 -1 req 9845352953875475562 0.001999958s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-6:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.6,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-6:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.6,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.674 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.670+0000 7f6c02ac9700 -1 req 8089242442733834609 0.002999935s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-27:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.27,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-27:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.27,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.675 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.670+0000 7fc7e76a7700 -1 req 6679544647714243678 0.001999956s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-6:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.6,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-6:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.6,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.676 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.670+0000 7f6b4414c700 -1 req 14409391316736588017 0.001999956s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-6:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.6,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-6:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.6,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.677 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.672+0000 7fc7e3ea0700 -1 req 14185547755917161676 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-32:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.32,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-32:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.32,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.678 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.672+0000 7f6beca9d700 -1 req 10473752288037847011 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-32:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.32,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-32:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.32,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.678 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.671+0000 7fab5d027700 -1 req 11128690134297459675 0.000999979s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-32:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.32,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-32:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.32,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.716 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.715+0000 7fab66039700 -1 req 9333262555642275542 0.001999958s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-33:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.33,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-33:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.33,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.717 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.715+0000 7fc7cf677700 -1 req 5774482808699920445 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-33:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.33,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-33:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.33,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.718 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.715+0000 7f6c12ae9700 -1 req 15215440276518762366 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-33:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.33,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-33:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.33,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.731 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.730+0000 7fab397e0700 -1 req 4786048523086178293 0.001999958s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-52:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.3,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-52:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.3,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.732 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.730+0000 7fc7c8669700 -1 req 18329802361739619043 0.001999956s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-52:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.3,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-52:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.3,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.733 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.730+0000 7f6be9a97700 -1 req 11051387711546338997 0.001999956s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-52:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.3,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-52:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.3,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.759 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.757+0000 7f6bd8274700 -1 req 8829992826540254143 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-45:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.2,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-45:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.2,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.761 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.757+0000 7fab2efcb700 -1 req 10040000483671924652 0.001999958s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-45:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.2,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-45:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.2,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.762 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.757+0000 7fc7bde54700 -1 req 15463607731003137412 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-45:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.2,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-45:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.2,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.762 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.758+0000 7fc7bbe50700 -1 req 10349478638003105024 0.000999978s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-66:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.17,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-66:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.17,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.763 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.759+0000 7f6bf42ac700 -1 req 6653962595811311380 0.001999956s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-66:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.17,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-66:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.17,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.765 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.758+0000 7fab6d047700 -1 req 13734643923414564963 0.000999979s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-66:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.17,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-66:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.17,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.766 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.764+0000 7fab4e80a700 -1 req 2507119399385141111 0.000999979s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-8:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.8,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-8:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.8,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.767 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.765+0000 7f6c0d2de700 -1 req 9160260997183201555 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-8:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.8,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-8:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.8,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.768 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.765+0000 7fc7b0639700 -1 req 4102105376056304766 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-8:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.8,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-8:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.8,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.825 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.823+0000 7fab0df89700 -1 req 15062306853229474321 0.001999958s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-39:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.39,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-39:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.39,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.827 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.823+0000 7f6b2f122700 -1 req 4288836280724810521 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-39:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.39,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-39:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.39,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.828 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.823+0000 7fc897807700 -1 req 5862185185300493111 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-39:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.39,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-39:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.39,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.856 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.853+0000 7fc8807d9700 -1 req 11273891367375521041 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-44:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.1,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-44:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.1,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.857 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.852+0000 7fab12f93700 -1 req 7965879515428753545 0.000999979s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-44:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.1,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-44:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.1,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.857 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.853+0000 7f6bc6250700 -1 req 17819124754662021334 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-44:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.1,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-44:91786ef1-cdb4-4061-8a3e-2a28cc497285.5174.1,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.907 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.906+0000 7fab05778700 -1 req 6391398933949301316 0.001999958s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-29:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.29,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-29:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.29,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.908 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.906+0000 7fc861f9c700 -1 req 13869424363298812042 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-29:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.29,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-29:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.29,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.909 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.906+0000 7f6bb4a2d700 -1 req 17265014562957472382 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-29:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.29,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-29:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.29,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.954 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.950+0000 7fc85277d700 -1 req 4617011540376172358 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-41:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.41,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-41:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.41,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.954 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.950+0000 7fabcf10b700 -1 req 10900945675412431251 0.001999958s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-41:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.41,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-41:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.41,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.955 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.950+0000 7f6b829c9700 -1 req 4192958644779220659 0.001999957s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-41:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.41,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-41:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.41,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:46.964 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:46.962+0000 7fabdc125700 -1 req 8091642965985359229 0.000999979s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-4:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.4,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-4:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.4,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:46.965 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:46.962+0000 7fc84ef76700 -1 req 13714017954662385787 0.001999956s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-4:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.4,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-4:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.4,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:46.965 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:46.962+0000 7f6b5c97d700 -1 req 15782019081341749959 0.001999956s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-4:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.4,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-4:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.4,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:47.046 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:12:47.044+0000 7fabb80dd700 -1 req 17121378617189681310 0.001999959s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-51:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.2,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-51:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.2,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:12:47.047 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:12:47.045+0000 7fc82d733700 -1 req 4169824277385278120 0.002999935s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-51:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.2,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-51:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.2,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:12:47.048 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:12:47.045+0000 7f6b791b6700 -1 req 2973084363388184290 0.002999935s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-51:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.2,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-51:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.2,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:12:47.107 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'list', '--bucket', 'dsgvcy-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:47.108 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'list', '--bucket', 'dsgvcy-69', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:47.109 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog list --bucket dsgvcy-69 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:12:47.157 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:47.157 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:47.210 INFO:teuthology.orchestra.run.smithi181.stdout:[] 2022-09-03T06:12:47.212 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_index_log_trim ... ok 2022-09-03T06:12:47.214 INFO:rgw_multi.tests:create bucket zone=test-zone1 name=dsgvcy-71 2022-09-03T06:12:47.215 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:12:47.217 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:12:47.218 DEBUG:boto:Method: PUT 2022-09-03T06:12:47.219 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:12:47.219 DEBUG:boto:Data: 2022-09-03T06:12:47.220 DEBUG:boto:Headers: {} 2022-09-03T06:12:47.221 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:47.222 DEBUG:boto:Port: 8000 2022-09-03T06:12:47.223 DEBUG:boto:Params: {} 2022-09-03T06:12:47.224 DEBUG:boto:Token: None 2022-09-03T06:12:47.225 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 06:12:47 GMT /dsgvcy-71/ 2022-09-03T06:12:47.225 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:soO8bxb9swieA/6DYK8K06+JXa4= 2022-09-03T06:12:47.226 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:soO8bxb9swieA/6DYK8K06+JXa4=', 'Content-Length': '0'} 2022-09-03T06:12:47.245 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000d3f447c9455c46a5-006312f05f-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:47.246 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:12:47.248 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:12:47.249 DEBUG:boto:Method: HEAD 2022-09-03T06:12:47.250 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:12:47.256 DEBUG:boto:Data: 2022-09-03T06:12:47.257 DEBUG:boto:Headers: {} 2022-09-03T06:12:47.257 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:47.258 DEBUG:boto:Port: 8000 2022-09-03T06:12:47.259 DEBUG:boto:Params: {} 2022-09-03T06:12:47.259 DEBUG:boto:Token: None 2022-09-03T06:12:47.260 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:47 GMT /dsgvcy-71/ 2022-09-03T06:12:47.261 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rBJR3yhtv/yK5tuKOLbf1KLKavM= 2022-09-03T06:12:47.261 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rBJR3yhtv/yK5tuKOLbf1KLKavM=', 'Content-Length': '0'} 2022-09-03T06:12:47.264 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005d7e030f81ff3779-006312f05f-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:47.265 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:12:47.265 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:12:47.266 DEBUG:boto:Method: PUT 2022-09-03T06:12:47.266 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:12:47.267 DEBUG:boto:Data: 2022-09-03T06:12:47.268 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:12:47.268 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:47.269 DEBUG:boto:Port: 8000 2022-09-03T06:12:47.269 DEBUG:boto:Params: {} 2022-09-03T06:12:47.271 DEBUG:boto:Token: None 2022-09-03T06:12:47.272 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:12:47 GMT /dsgvcy-71/a 2022-09-03T06:12:47.273 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RjWsOi8vtCcM31a24qTkeAd6wCk= 2022-09-03T06:12:47.275 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:12:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RjWsOi8vtCcM31a24qTkeAd6wCk='} 2022-09-03T06:12:47.284 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a7a4daddd1a955ac-006312f05f-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:47.285 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:12:47.286 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:12:47.287 DEBUG:boto:Method: HEAD 2022-09-03T06:12:47.289 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:12:47.290 DEBUG:boto:Data: 2022-09-03T06:12:47.290 DEBUG:boto:Headers: {} 2022-09-03T06:12:47.291 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:47.292 DEBUG:boto:Port: 8000 2022-09-03T06:12:47.293 DEBUG:boto:Params: {} 2022-09-03T06:12:47.295 DEBUG:boto:Token: None 2022-09-03T06:12:47.296 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:47 GMT /dsgvcy-71/ 2022-09-03T06:12:47.297 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rBJR3yhtv/yK5tuKOLbf1KLKavM= 2022-09-03T06:12:47.298 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rBJR3yhtv/yK5tuKOLbf1KLKavM=', 'Content-Length': '0'} 2022-09-03T06:12:47.300 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '3'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000008b4908aea2933c2-006312f05f-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:47.302 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:12:47.303 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:12:47.303 DEBUG:boto:Method: PUT 2022-09-03T06:12:47.304 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:12:47.305 DEBUG:boto:Data: 2022-09-03T06:12:47.306 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:12:47.306 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:47.307 DEBUG:boto:Port: 8000 2022-09-03T06:12:47.307 DEBUG:boto:Params: {} 2022-09-03T06:12:47.308 DEBUG:boto:Token: None 2022-09-03T06:12:47.309 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:12:47 GMT /dsgvcy-71/b 2022-09-03T06:12:47.310 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TbQQNdBcMsYHZ7KT/WNMTOvFSpU= 2022-09-03T06:12:47.311 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:12:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TbQQNdBcMsYHZ7KT/WNMTOvFSpU='} 2022-09-03T06:12:47.323 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000065bc9d6ced15f91b-006312f05f-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:47.325 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:12:47.327 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:12:47.328 DEBUG:boto:Method: HEAD 2022-09-03T06:12:47.329 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:12:47.330 DEBUG:boto:Data: 2022-09-03T06:12:47.330 DEBUG:boto:Headers: {} 2022-09-03T06:12:47.331 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:47.332 DEBUG:boto:Port: 8000 2022-09-03T06:12:47.333 DEBUG:boto:Params: {} 2022-09-03T06:12:47.334 DEBUG:boto:Token: None 2022-09-03T06:12:47.335 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:47 GMT /dsgvcy-71/ 2022-09-03T06:12:47.336 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rBJR3yhtv/yK5tuKOLbf1KLKavM= 2022-09-03T06:12:47.338 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rBJR3yhtv/yK5tuKOLbf1KLKavM=', 'Content-Length': '0'} 2022-09-03T06:12:47.341 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000068a867dc6dd6122-006312f05f-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:47.342 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:12:47.343 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:12:47.344 DEBUG:boto:Method: PUT 2022-09-03T06:12:47.345 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:12:47.346 DEBUG:boto:Data: 2022-09-03T06:12:47.347 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:12:47.347 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:47.348 DEBUG:boto:Port: 8000 2022-09-03T06:12:47.349 DEBUG:boto:Params: {} 2022-09-03T06:12:47.350 DEBUG:boto:Token: None 2022-09-03T06:12:47.351 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:12:47 GMT /dsgvcy-71/c 2022-09-03T06:12:47.352 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VjJ2DG/fPIGwfZocaHlDJQbT0iU= 2022-09-03T06:12:47.353 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:12:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VjJ2DG/fPIGwfZocaHlDJQbT0iU='} 2022-09-03T06:12:47.398 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a4e309b8579cad81-006312f05f-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:47.399 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:12:47.400 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:12:47.400 DEBUG:boto:Method: HEAD 2022-09-03T06:12:47.401 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:12:47.401 DEBUG:boto:Data: 2022-09-03T06:12:47.402 DEBUG:boto:Headers: {} 2022-09-03T06:12:47.403 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:47.403 DEBUG:boto:Port: 8000 2022-09-03T06:12:47.404 DEBUG:boto:Params: {} 2022-09-03T06:12:47.404 DEBUG:boto:Token: None 2022-09-03T06:12:47.405 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:12:47 GMT /dsgvcy-71/ 2022-09-03T06:12:47.405 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rBJR3yhtv/yK5tuKOLbf1KLKavM= 2022-09-03T06:12:47.406 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:12:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rBJR3yhtv/yK5tuKOLbf1KLKavM=', 'Content-Length': '0'} 2022-09-03T06:12:47.409 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '9'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000bc6d621770d32fc2-006312f05f-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:47.409 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:12:47.410 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:12:47.411 DEBUG:boto:Method: PUT 2022-09-03T06:12:47.411 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:12:47.412 DEBUG:boto:Data: 2022-09-03T06:12:47.412 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:12:47.413 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:12:47.413 DEBUG:boto:Port: 8000 2022-09-03T06:12:47.414 DEBUG:boto:Params: {} 2022-09-03T06:12:47.415 DEBUG:boto:Token: None 2022-09-03T06:12:47.415 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:12:47 GMT /dsgvcy-71/d 2022-09-03T06:12:47.416 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ud7d8ODh3UOJpNv82Z/YcNGAKvg= 2022-09-03T06:12:47.416 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:12:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ud7d8ODh3UOJpNv82Z/YcNGAKvg='} 2022-09-03T06:12:47.425 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004bc8d8012e98479e-006312f05f-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:12:47.426 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:47.427 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:47.427 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:12:47.479 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:47.479 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:47.525 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T06:12:47.542 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T06:12:47.543 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:47.543 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:12:47.543 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:12:47.544 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:47.544 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:47.545 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:12:47.545 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:12:47.546 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:47.546 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:47.546 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662185567.242556_428.1", 2022-09-03T06:12:47.547 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:12:47.242556Z" 2022-09-03T06:12:47.547 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:47.547 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:47.548 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:12:47.548 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:12:47.549 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:12:47.549 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T06:12:47.550 DEBUG:rgw_multi.tests:master meta markers={0: '', 1: '', 2: '1_1662185567.242556_428.1', 3: ''} 2022-09-03T06:12:47.551 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T06:12:47.552 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:47.553 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:47.554 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:12:47.604 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:47.605 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:47.632 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:47.630+0000 7fd54f6da600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:12:47.633 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:47.631+0000 7fd54f6da600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:12:47.646 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:12:47.647 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:12:47.647 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:12:47.648 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:12:47.648 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:12:47.649 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:12:47.649 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:12:47.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:47.652 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:12:47.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:47.652 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:12:47.653 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:47.653 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:47.653 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:12:47.653 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:47.654 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:12:47.654 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:47.654 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:12:47.656 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:12:47.657 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:47.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:47.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:47.658 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:12:47.659 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:47.659 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:47.660 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:12:47.660 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:47.661 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:12:47.661 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:47.662 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:12:47.662 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:12:47.664 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:47.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:47.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:47.667 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:12:47.668 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:47.668 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:47.669 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662185525.207441_422.1", 2022-09-03T06:12:47.669 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:47.670 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:12:47.670 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:47.671 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:12:05.207441Z", 2022-09-03T06:12:47.671 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:12:47.672 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:47.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:47.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:47.673 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:12:47.673 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:47.674 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:47.675 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:12:47.675 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:47.675 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:12:47.676 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:47.677 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:12:47.677 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:12:47.677 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:47.678 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:47.678 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:12:47.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:47.679 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:12:47.679 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:12:47.679 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:12:47.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:47.680 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:12:47Z" 2022-09-03T06:12:47.680 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:12:47.681 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185525.207441_422.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:12:05.207441Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:12:47Z" } 2022-09-03T06:12:47.682 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185525.207441_422.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:12:05.207441Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:12:47.683 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185567.242556_428.1', 3: ''} 2022-09-03T06:12:47.683 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185525.207441_422.1', 3: ''} 2022-09-03T06:12:47.684 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662185567.242556_428.1 target=1_1662185525.207441_422.1 2022-09-03T06:12:52.685 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:52.686 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:52.687 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:12:52.745 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:52.746 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:52.777 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:52.773+0000 7f0e7bfd6600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:12:52.777 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:52.775+0000 7f0e7bfd6600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:12:52.791 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:12:52.791 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:12:52.792 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:12:52.792 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:12:52.793 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:12:52.793 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:12:52.794 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:12:52.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:52.795 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:12:52.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:52.796 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:12:52.796 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:52.797 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:52.798 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:12:52.798 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:52.798 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:12:52.799 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:52.799 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:12:52.799 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:12:52.799 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:52.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:52.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:52.800 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:12:52.800 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:52.801 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:52.801 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:12:52.801 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:52.802 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:12:52.802 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:52.802 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:12:52.802 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:12:52.803 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:52.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:52.803 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:52.804 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:12:52.804 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:52.804 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:52.804 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662185567.242556_428.1", 2022-09-03T06:12:52.805 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:52.805 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:12:52.805 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:52.805 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:12:47.242556Z", 2022-09-03T06:12:52.806 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:12:52.806 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:52.806 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:52.807 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:52.807 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:12:52.807 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:52.807 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:52.808 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:12:52.808 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:52.808 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:12:52.808 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:52.808 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:12:52.809 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:12:52.809 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:52.809 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:52.810 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:12:52.810 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:52.810 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:12:52.811 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:12:52.811 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:12:52.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:52.811 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:12:52Z" 2022-09-03T06:12:52.812 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:12:52.813 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185567.242556_428.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:12:47.242556Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:12:52Z" } 2022-09-03T06:12:52.814 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185567.242556_428.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:12:47.242556Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:12:52.815 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185567.242556_428.1', 3: ''} 2022-09-03T06:12:52.815 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185567.242556_428.1', 3: ''} 2022-09-03T06:12:52.816 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T06:12:52.816 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T06:12:52.817 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:52.817 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:52.818 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:12:52.875 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:52.876 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:52.907 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:52.904+0000 7f53651a1600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:12:52.908 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:12:52.905+0000 7f53651a1600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:12:52.921 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T06:12:52.922 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T06:12:52.922 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T06:12:52.923 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T06:12:52.923 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T06:12:52.923 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:12:52.924 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:12:52.924 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:52.925 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T06:12:52.925 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:52.927 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T06:12:52.927 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:12:52.927 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:12:52.927 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:12:52.928 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:12:52.928 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T06:12:52.928 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:12:52.929 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:12:52.929 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:12:52.929 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:12:52.930 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:52.931 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:52.932 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T06:12:52.932 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:12:52.933 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:12:52.933 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:12:52.934 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:12:52.934 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:12:52.934 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:12:52.935 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T06:12:52.935 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:12:52.936 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:12:52.936 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:52.937 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:52.937 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T06:12:52.937 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:12:52.938 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:12:52.938 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662185567.242556_428.1", 2022-09-03T06:12:52.938 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:12:52.939 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:12:52.939 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:12:52.939 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:12:47.242556Z", 2022-09-03T06:12:52.940 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:12:52.940 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:12:52.940 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:52.941 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:12:52.941 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T06:12:52.941 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:12:52.941 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:12:52.942 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:12:52.942 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:12:52.942 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:12:52.943 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:12:52.943 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:12:52.944 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:12:52.944 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:12:52.945 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:12:52.945 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:12:52.946 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:52.946 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T06:12:52.946 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T06:12:52.947 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T06:12:52.947 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:12:52.948 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T06:12:52Z" 2022-09-03T06:12:52.949 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T06:12:52.951 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185567.242556_428.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:12:47.242556Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:12:52Z" } 2022-09-03T06:12:52.952 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185567.242556_428.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:12:47.242556Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:12:52.952 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185567.242556_428.1', 3: ''} 2022-09-03T06:12:52.953 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185567.242556_428.1', 3: ''} 2022-09-03T06:12:52.954 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T06:12:52.955 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T06:12:52.955 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:52.956 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:12:52.956 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:12:53.008 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:53.009 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:53.058 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:53.056+0000 7f91bba4d600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:12:53.059 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:53.057+0000 7f91bba4d600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:12:53.074 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:12:53.074 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:12:53.074 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:12:53.075 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:12:53.075 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:12:53.075 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:12:53.076 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:12:53.076 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:53.077 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:12:53.077 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:53.077 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:12:53.077 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:53.078 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:53.078 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:12:53.078 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:53.079 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:12:53.079 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:53.079 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:12:53.079 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:12:53.080 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:53.080 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:53.080 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:53.081 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:12:53.081 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:53.081 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:53.082 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:12:53.082 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:53.082 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:12:53.083 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:53.083 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:12:53.083 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:12:53.083 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:53.084 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:53.084 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:53.084 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:12:53.085 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:53.085 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:53.085 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662185567.242556_428.1", 2022-09-03T06:12:53.085 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:53.086 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:12:53.086 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:53.086 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:12:47.242556Z", 2022-09-03T06:12:53.086 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:12:53.087 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:53.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:53.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:12:53.088 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:12:53.088 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:12:53.088 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:12:53.088 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:12:53.089 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:12:53.089 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:12:53.089 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:12:53.089 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:12:53.090 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:12:53.090 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:53.090 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:12:53.091 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:12:53.091 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:53.091 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:12:53.091 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:12:53.092 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:12:53.092 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:12:53.092 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:12:53Z" 2022-09-03T06:12:53.092 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:12:53.094 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185567.242556_428.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:12:47.242556Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:12:53Z" } 2022-09-03T06:12:53.095 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185567.242556_428.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:12:47.242556Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:12:53.095 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185567.242556_428.1', 3: ''} 2022-09-03T06:12:53.096 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185567.242556_428.1', 3: ''} 2022-09-03T06:12:53.096 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T06:12:53.097 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-71 2022-09-03T06:12:53.097 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:53.098 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:12:53.098 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-71 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:12:53.154 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:12:53.155 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:12:53.191 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:53.187+0000 7f14ac7f5600 1 waiting to reach incremental sync.. 2022-09-03T06:12:58.190 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:12:58.188+0000 7f14ac7f5600 1 waiting to reach incremental sync.. 2022-09-03T06:13:03.191 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:13:03.189+0000 7f14ac7f5600 1 waiting to reach incremental sync.. 2022-09-03T06:13:08.191 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:13:08.189+0000 7f14ac7f5600 1 waiting to reach incremental sync.. 2022-09-03T06:13:13.195 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:13:13.190+0000 7f14ac7f5600 1 waiting to reach incremental sync.. 2022-09-03T06:13:18.193 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:13:18.191+0000 7f14ac7f5600 1 waiting to reach incremental sync.. 2022-09-03T06:13:23.194 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:13:23.191+0000 7f14ac7f5600 1 bucket sync caught up with source: 2022-09-03T06:13:23.195 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.42.5, , , , 00000000001.49.5, , , 00000000001.88.5, , 00000000001.45.5] 2022-09-03T06:13:23.195 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.42.5, , , , 00000000001.49.5, , , 00000000001.88.5, , 00000000001.45.5] 2022-09-03T06:13:23.195 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:13:23.191+0000 7f14ac7f5600 0 bucket checkpoint complete 2022-09-03T06:13:23.214 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-71 2022-09-03T06:13:23.215 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:13:23.218 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:13:23.220 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-71 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:13:23.286 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:13:23.286 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:13:23.325 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:13:23.323+0000 7f63c5139600 1 bucket sync caught up with source: 2022-09-03T06:13:23.325 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.42.5, , , , 00000000001.49.5, , , 00000000001.88.5, , 00000000001.45.5] 2022-09-03T06:13:23.326 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.42.5, , , , 00000000001.49.5, , , 00000000001.88.5, , 00000000001.45.5] 2022-09-03T06:13:23.326 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:13:23.323+0000 7f63c5139600 0 bucket checkpoint complete 2022-09-03T06:13:23.339 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-71 2022-09-03T06:13:23.340 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:13:23.341 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:13:23.342 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-71 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:13:23.408 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:13:23.408 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:13:23.446 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:13:23.443+0000 7f9b5ff0f600 1 bucket sync caught up with source: 2022-09-03T06:13:23.448 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.42.5, , , , 00000000001.49.5, , , 00000000001.88.5, , 00000000001.45.5] 2022-09-03T06:13:23.449 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.42.5, , , , 00000000001.49.5, , , 00000000001.88.5, , 00000000001.45.5] 2022-09-03T06:13:23.449 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:13:23.443+0000 7f9b5ff0f600 0 bucket checkpoint complete 2022-09-03T06:13:23.460 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-71 2022-09-03T06:13:23.461 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:13:23.461 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:13:23.462 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-71 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:13:23.520 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:13:23.521 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:13:23.561 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:13:23.560+0000 7f34ed0ca600 1 bucket sync caught up with source: 2022-09-03T06:13:23.562 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.731.5, , , , 00000000001.685.5, , , 00000000001.334.5, , 00000000001.417.5] 2022-09-03T06:13:23.562 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.731.5, , , , 00000000001.685.5, , , 00000000001.334.5, , 00000000001.417.5] 2022-09-03T06:13:23.563 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:13:23.560+0000 7f34ed0ca600 0 bucket checkpoint complete 2022-09-03T06:13:23.586 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-71 2022-09-03T06:13:23.586 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:13:23.587 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:13:23.588 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-71 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:13:23.644 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:13:23.645 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:13:23.696 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:13:23.695+0000 7f4d7c7ee600 1 bucket sync caught up with source: 2022-09-03T06:13:23.696 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.731.5, , , , 00000000001.685.5, , , 00000000001.334.5, , 00000000001.417.5] 2022-09-03T06:13:23.697 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.731.5, , , , 00000000001.685.5, , , 00000000001.334.5, , 00000000001.417.5] 2022-09-03T06:13:23.697 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:13:23.695+0000 7f4d7c7ee600 0 bucket checkpoint complete 2022-09-03T06:13:23.715 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-71 2022-09-03T06:13:23.716 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-71 2022-09-03T06:13:23.716 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:13:23.717 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:13:23.718 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-71 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:13:23.774 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:13:23.774 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:13:23.827 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:13:23.825+0000 7fcdabd5d600 1 waiting to reach incremental sync.. 2022-09-03T06:13:28.827 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:13:28.826+0000 7fcdabd5d600 1 waiting to reach incremental sync.. 2022-09-03T06:13:33.830 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:13:33.828+0000 7fcdabd5d600 1 bucket sync caught up with source: 2022-09-03T06:13:33.831 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.336.5, , , , 00000000001.244.5, , , 00000000001.239.5, , 00000000001.178.5] 2022-09-03T06:13:33.831 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.336.5, , , , 00000000001.244.5, , , 00000000001.239.5, , 00000000001.178.5] 2022-09-03T06:13:33.832 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:13:33.828+0000 7fcdabd5d600 0 bucket checkpoint complete 2022-09-03T06:13:33.853 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-71 2022-09-03T06:13:33.854 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:13:33.855 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:13:33.856 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-71 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:13:33.928 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:13:33.928 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:13:33.975 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:13:33.973+0000 7fda7d576600 1 bucket sync caught up with source: 2022-09-03T06:13:33.976 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.336.5, , , , 00000000001.244.5, , , 00000000001.239.5, , 00000000001.178.5] 2022-09-03T06:13:33.977 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.336.5, , , , 00000000001.244.5, , , 00000000001.239.5, , 00000000001.178.5] 2022-09-03T06:13:33.979 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:13:33.973+0000 7fda7d576600 0 bucket checkpoint complete 2022-09-03T06:13:33.993 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-71 2022-09-03T06:13:33.995 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-71 zones={test-zone2, test-zone1} 2022-09-03T06:13:33.996 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:33.997 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:33.999 DEBUG:boto:Method: HEAD 2022-09-03T06:13:34.000 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:13:34.002 DEBUG:boto:Data: 2022-09-03T06:13:34.005 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.005 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.007 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.008 DEBUG:boto:Params: {} 2022-09-03T06:13:34.012 DEBUG:boto:Token: None 2022-09-03T06:13:34.013 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/ 2022-09-03T06:13:34.014 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7qinZtC/ueZHsEzuErBYFCDI6cM= 2022-09-03T06:13:34.014 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7qinZtC/ueZHsEzuErBYFCDI6cM=', 'Content-Length': '0'} 2022-09-03T06:13:34.017 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d477801d111d49ce-006312f08e-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.018 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:34.018 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:34.019 DEBUG:boto:Method: HEAD 2022-09-03T06:13:34.019 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:13:34.020 DEBUG:boto:Data: 2022-09-03T06:13:34.021 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.021 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.022 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.023 DEBUG:boto:Params: {} 2022-09-03T06:13:34.023 DEBUG:boto:Token: None 2022-09-03T06:13:34.024 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/ 2022-09-03T06:13:34.025 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7qinZtC/ueZHsEzuErBYFCDI6cM= 2022-09-03T06:13:34.025 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7qinZtC/ueZHsEzuErBYFCDI6cM=', 'Content-Length': '0'} 2022-09-03T06:13:34.032 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005267056c098f3056-006312f08e-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.033 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:13:34.034 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:34.035 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:34.035 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:34.036 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:34.037 DEBUG:boto:Method: GET 2022-09-03T06:13:34.037 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:34.038 DEBUG:boto:Data: 2022-09-03T06:13:34.038 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.044 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.045 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.046 DEBUG:boto:Params: {} 2022-09-03T06:13:34.053 DEBUG:boto:Token: None 2022-09-03T06:13:34.055 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/?versions 2022-09-03T06:13:34.056 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VbmVp+AW3ns9j6N7T+Q+dD4Rz4Q= 2022-09-03T06:13:34.057 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VbmVp+AW3ns9j6N7T+Q+dD4Rz4Q=', 'Content-Length': '0'} 2022-09-03T06:13:34.063 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ba8f9da6eb32d3d3-006312f08e-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.064 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:34.067 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:13:34.068 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:13:34.069 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:13:34.069 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:13:34.070 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:13:34.072 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:34.074 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:34.075 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:34.076 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:34.077 DEBUG:boto:Method: GET 2022-09-03T06:13:34.078 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:34.079 DEBUG:boto:Data: 2022-09-03T06:13:34.080 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.081 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.082 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.084 DEBUG:boto:Params: {} 2022-09-03T06:13:34.085 DEBUG:boto:Token: None 2022-09-03T06:13:34.087 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/?versions 2022-09-03T06:13:34.088 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VbmVp+AW3ns9j6N7T+Q+dD4Rz4Q= 2022-09-03T06:13:34.089 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VbmVp+AW3ns9j6N7T+Q+dD4Rz4Q=', 'Content-Length': '0'} 2022-09-03T06:13:34.093 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000526ff9f5a1483c9e-006312f08e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.098 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:34.100 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:13:34.101 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:13:34.102 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:13:34.103 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:13:34.111 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:34.111 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:34.112 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:34.112 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:34.113 DEBUG:boto:Method: GET 2022-09-03T06:13:34.113 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:34.114 DEBUG:boto:Data: 2022-09-03T06:13:34.115 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.115 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.116 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.116 DEBUG:boto:Params: {} 2022-09-03T06:13:34.117 DEBUG:boto:Token: None 2022-09-03T06:13:34.117 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/?versions 2022-09-03T06:13:34.118 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VbmVp+AW3ns9j6N7T+Q+dD4Rz4Q= 2022-09-03T06:13:34.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VbmVp+AW3ns9j6N7T+Q+dD4Rz4Q=', 'Content-Length': '0'} 2022-09-03T06:13:34.129 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c14ff440cb1cf2eb-006312f08e-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.130 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:34.131 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:34.131 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:34.132 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:34.132 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:34.133 DEBUG:boto:Method: GET 2022-09-03T06:13:34.133 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:34.134 DEBUG:boto:Data: 2022-09-03T06:13:34.134 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.135 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.135 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.136 DEBUG:boto:Params: {} 2022-09-03T06:13:34.137 DEBUG:boto:Token: None 2022-09-03T06:13:34.137 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/?versions 2022-09-03T06:13:34.146 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VbmVp+AW3ns9j6N7T+Q+dD4Rz4Q= 2022-09-03T06:13:34.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VbmVp+AW3ns9j6N7T+Q+dD4Rz4Q=', 'Content-Length': '0'} 2022-09-03T06:13:34.154 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000223d392439f096d2-006312f08e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.154 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:34.155 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:13:34.156 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:34.157 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:34.157 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.158 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.158 DEBUG:boto:Method: GET 2022-09-03T06:13:34.159 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.159 DEBUG:boto:Data: 2022-09-03T06:13:34.160 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.160 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.161 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.161 DEBUG:boto:Params: {} 2022-09-03T06:13:34.162 DEBUG:boto:Token: None 2022-09-03T06:13:34.163 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.164 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t0d+MNHs5s6WF2ov4K/AjADS67I= 2022-09-03T06:13:34.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t0d+MNHs5s6WF2ov4K/AjADS67I=', 'Content-Length': '0'} 2022-09-03T06:13:34.180 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007be4291290fa5fd1-006312f08e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.181 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:34.183 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:34.183 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.184 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.185 DEBUG:boto:Method: GET 2022-09-03T06:13:34.186 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.187 DEBUG:boto:Data: 2022-09-03T06:13:34.188 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.189 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.190 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.191 DEBUG:boto:Params: {} 2022-09-03T06:13:34.193 DEBUG:boto:Token: None 2022-09-03T06:13:34.194 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.195 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t0d+MNHs5s6WF2ov4K/AjADS67I= 2022-09-03T06:13:34.196 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t0d+MNHs5s6WF2ov4K/AjADS67I=', 'Content-Length': '0'} 2022-09-03T06:13:34.199 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000006c29cdb0a85b0e3-006312f08e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.201 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:34.202 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:34.203 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.204 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.205 DEBUG:boto:Method: HEAD 2022-09-03T06:13:34.206 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.207 DEBUG:boto:Data: 2022-09-03T06:13:34.208 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.209 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.210 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.211 DEBUG:boto:Params: {} 2022-09-03T06:13:34.212 DEBUG:boto:Token: None 2022-09-03T06:13:34.213 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.214 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gnoNpYCU48Bzla1yeqC9WXqfxWw= 2022-09-03T06:13:34.215 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gnoNpYCU48Bzla1yeqC9WXqfxWw=', 'Content-Length': '0'} 2022-09-03T06:13:34.218 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a5a9b10bb8a8c6f0-006312f08e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.220 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:34.222 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:34.223 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.224 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.224 DEBUG:boto:Method: HEAD 2022-09-03T06:13:34.225 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.226 DEBUG:boto:Data: 2022-09-03T06:13:34.227 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.228 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.229 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.230 DEBUG:boto:Params: {} 2022-09-03T06:13:34.231 DEBUG:boto:Token: None 2022-09-03T06:13:34.232 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.233 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gnoNpYCU48Bzla1yeqC9WXqfxWw= 2022-09-03T06:13:34.234 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gnoNpYCU48Bzla1yeqC9WXqfxWw=', 'Content-Length': '0'} 2022-09-03T06:13:34.236 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f0e1030f459f9a32-006312f08e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.237 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:13:34.238 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:34.239 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:34.239 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.240 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.240 DEBUG:boto:Method: GET 2022-09-03T06:13:34.241 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.241 DEBUG:boto:Data: 2022-09-03T06:13:34.242 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.243 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.244 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.245 DEBUG:boto:Params: {} 2022-09-03T06:13:34.246 DEBUG:boto:Token: None 2022-09-03T06:13:34.247 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.249 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t0d+MNHs5s6WF2ov4K/AjADS67I= 2022-09-03T06:13:34.250 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t0d+MNHs5s6WF2ov4K/AjADS67I=', 'Content-Length': '0'} 2022-09-03T06:13:34.253 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004f6c62be6bdd5ea1-006312f08e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.254 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:34.255 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:34.256 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.257 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.258 DEBUG:boto:Method: GET 2022-09-03T06:13:34.259 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.260 DEBUG:boto:Data: 2022-09-03T06:13:34.261 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.263 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.264 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.265 DEBUG:boto:Params: {} 2022-09-03T06:13:34.266 DEBUG:boto:Token: None 2022-09-03T06:13:34.266 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:13:34.267 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t0d+MNHs5s6WF2ov4K/AjADS67I= 2022-09-03T06:13:34.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t0d+MNHs5s6WF2ov4K/AjADS67I=', 'Content-Length': '0'} 2022-09-03T06:13:34.271 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f3d65a202c18397f-006312f08e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.272 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:34.273 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:34.274 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:34.274 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:34.275 DEBUG:boto:Method: GET 2022-09-03T06:13:34.275 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:13:34.276 DEBUG:boto:Data: 2022-09-03T06:13:34.276 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.277 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.277 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.278 DEBUG:boto:Params: {} 2022-09-03T06:13:34.278 DEBUG:boto:Token: None 2022-09-03T06:13:34.279 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/ 2022-09-03T06:13:34.280 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY= 2022-09-03T06:13:34.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY=', 'Content-Length': '0'} 2022-09-03T06:13:34.288 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000004f0ff54405210ff-006312f08e-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.289 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:34.291 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:34.292 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:34.293 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:34.294 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:34.295 DEBUG:boto:Method: GET 2022-09-03T06:13:34.296 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:13:34.297 DEBUG:boto:Data: 2022-09-03T06:13:34.298 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.299 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.300 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.301 DEBUG:boto:Params: {} 2022-09-03T06:13:34.302 DEBUG:boto:Token: None 2022-09-03T06:13:34.304 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/ 2022-09-03T06:13:34.305 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY= 2022-09-03T06:13:34.307 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY=', 'Content-Length': '0'} 2022-09-03T06:13:34.310 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001fdeb4a9acd3997b-006312f08e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.350 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:34.351 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:34.352 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:34.353 DEBUG:boto:Method: HEAD 2022-09-03T06:13:34.353 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:13:34.354 DEBUG:boto:Data: 2022-09-03T06:13:34.354 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.355 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.355 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.356 DEBUG:boto:Params: {} 2022-09-03T06:13:34.357 DEBUG:boto:Token: None 2022-09-03T06:13:34.357 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/a 2022-09-03T06:13:34.358 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:88Q3ngXOEJeAmOJ6w+J1RopNOc0= 2022-09-03T06:13:34.359 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:88Q3ngXOEJeAmOJ6w+J1RopNOc0=', 'Content-Length': '0'} 2022-09-03T06:13:34.365 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b549df8534d89ab-006312f08e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.366 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:34.367 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:34.368 DEBUG:boto:Method: HEAD 2022-09-03T06:13:34.368 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:13:34.369 DEBUG:boto:Data: 2022-09-03T06:13:34.369 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.370 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.370 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.371 DEBUG:boto:Params: {} 2022-09-03T06:13:34.373 DEBUG:boto:Token: None 2022-09-03T06:13:34.373 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/a 2022-09-03T06:13:34.374 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:88Q3ngXOEJeAmOJ6w+J1RopNOc0= 2022-09-03T06:13:34.375 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:88Q3ngXOEJeAmOJ6w+J1RopNOc0=', 'Content-Length': '0'} 2022-09-03T06:13:34.380 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b2103aa3c6cb2db-006312f08e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.384 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:13:34.386 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:34.387 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:34.387 DEBUG:boto:Method: GET 2022-09-03T06:13:34.388 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:13:34.388 DEBUG:boto:Data: 2022-09-03T06:13:34.389 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.389 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.390 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.390 DEBUG:boto:Params: {} 2022-09-03T06:13:34.391 DEBUG:boto:Token: None 2022-09-03T06:13:34.395 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/a 2022-09-03T06:13:34.396 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rAqHqyHbDrc+1cHNjfqyf5WSCmc= 2022-09-03T06:13:34.397 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rAqHqyHbDrc+1cHNjfqyf5WSCmc=', 'Content-Length': '0'} 2022-09-03T06:13:34.399 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008a377ff85a8980a6-006312f08e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.446 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:34.447 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:34.447 DEBUG:boto:Method: GET 2022-09-03T06:13:34.448 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:13:34.449 DEBUG:boto:Data: 2022-09-03T06:13:34.449 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.450 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.450 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.451 DEBUG:boto:Params: {} 2022-09-03T06:13:34.451 DEBUG:boto:Token: None 2022-09-03T06:13:34.452 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/a 2022-09-03T06:13:34.453 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rAqHqyHbDrc+1cHNjfqyf5WSCmc= 2022-09-03T06:13:34.453 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rAqHqyHbDrc+1cHNjfqyf5WSCmc=', 'Content-Length': '0'} 2022-09-03T06:13:34.456 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e3ddf1ecf0f7f160-006312f08e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.456 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:34.457 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:34.458 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:34.458 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:34.459 DEBUG:boto:Method: GET 2022-09-03T06:13:34.459 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:13:34.460 DEBUG:boto:Data: 2022-09-03T06:13:34.460 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.461 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.461 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.462 DEBUG:boto:Params: {} 2022-09-03T06:13:34.463 DEBUG:boto:Token: None 2022-09-03T06:13:34.464 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/ 2022-09-03T06:13:34.464 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY= 2022-09-03T06:13:34.465 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY=', 'Content-Length': '0'} 2022-09-03T06:13:34.468 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b2d7601868711537-006312f08e-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.469 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:34.471 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:34.472 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:34.473 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:34.476 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:34.477 DEBUG:boto:Method: GET 2022-09-03T06:13:34.478 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:13:34.479 DEBUG:boto:Data: 2022-09-03T06:13:34.480 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.481 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.482 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.483 DEBUG:boto:Params: {} 2022-09-03T06:13:34.484 DEBUG:boto:Token: None 2022-09-03T06:13:34.485 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/ 2022-09-03T06:13:34.487 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY= 2022-09-03T06:13:34.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY=', 'Content-Length': '0'} 2022-09-03T06:13:34.492 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008abba6cca09b3675-006312f08e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.493 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:34.495 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:13:34.496 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:34.498 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:34.499 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.499 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.500 DEBUG:boto:Method: GET 2022-09-03T06:13:34.501 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.502 DEBUG:boto:Data: 2022-09-03T06:13:34.503 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.504 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.505 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.506 DEBUG:boto:Params: {} 2022-09-03T06:13:34.508 DEBUG:boto:Token: None 2022-09-03T06:13:34.509 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.510 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:424GVxME3Dj1Kok/0Zkojr1Roe4= 2022-09-03T06:13:34.511 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:424GVxME3Dj1Kok/0Zkojr1Roe4=', 'Content-Length': '0'} 2022-09-03T06:13:34.514 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000081edf4c4530b5d78-006312f08e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.516 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:34.517 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:34.519 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.520 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.521 DEBUG:boto:Method: GET 2022-09-03T06:13:34.522 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.523 DEBUG:boto:Data: 2022-09-03T06:13:34.524 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.524 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.525 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.526 DEBUG:boto:Params: {} 2022-09-03T06:13:34.527 DEBUG:boto:Token: None 2022-09-03T06:13:34.528 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.530 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:424GVxME3Dj1Kok/0Zkojr1Roe4= 2022-09-03T06:13:34.530 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:424GVxME3Dj1Kok/0Zkojr1Roe4=', 'Content-Length': '0'} 2022-09-03T06:13:34.533 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000046717c116f4c2069-006312f08e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.535 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:34.536 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:34.537 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.538 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.539 DEBUG:boto:Method: HEAD 2022-09-03T06:13:34.540 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.541 DEBUG:boto:Data: 2022-09-03T06:13:34.542 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.543 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.543 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.544 DEBUG:boto:Params: {} 2022-09-03T06:13:34.545 DEBUG:boto:Token: None 2022-09-03T06:13:34.545 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.546 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UUV00NtAXZaLJ5d27xhoPjhbo+A= 2022-09-03T06:13:34.547 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UUV00NtAXZaLJ5d27xhoPjhbo+A=', 'Content-Length': '0'} 2022-09-03T06:13:34.549 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a8b334aa77cf407-006312f08e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.550 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:34.551 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:34.552 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.553 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.554 DEBUG:boto:Method: HEAD 2022-09-03T06:13:34.555 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.556 DEBUG:boto:Data: 2022-09-03T06:13:34.557 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.558 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.560 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.561 DEBUG:boto:Params: {} 2022-09-03T06:13:34.562 DEBUG:boto:Token: None 2022-09-03T06:13:34.562 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.563 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UUV00NtAXZaLJ5d27xhoPjhbo+A= 2022-09-03T06:13:34.563 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UUV00NtAXZaLJ5d27xhoPjhbo+A=', 'Content-Length': '0'} 2022-09-03T06:13:34.566 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003537688abc841310-006312f08e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.566 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:13:34.567 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:34.568 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:34.569 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.570 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.571 DEBUG:boto:Method: GET 2022-09-03T06:13:34.572 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.572 DEBUG:boto:Data: 2022-09-03T06:13:34.573 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.574 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.575 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.576 DEBUG:boto:Params: {} 2022-09-03T06:13:34.577 DEBUG:boto:Token: None 2022-09-03T06:13:34.578 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.579 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:424GVxME3Dj1Kok/0Zkojr1Roe4= 2022-09-03T06:13:34.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:424GVxME3Dj1Kok/0Zkojr1Roe4=', 'Content-Length': '0'} 2022-09-03T06:13:34.584 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a7b38900d90394d9-006312f08e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.585 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:34.586 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:34.587 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.588 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.589 DEBUG:boto:Method: GET 2022-09-03T06:13:34.590 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.591 DEBUG:boto:Data: 2022-09-03T06:13:34.592 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.593 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.594 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.595 DEBUG:boto:Params: {} 2022-09-03T06:13:34.596 DEBUG:boto:Token: None 2022-09-03T06:13:34.597 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:13:34.598 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:424GVxME3Dj1Kok/0Zkojr1Roe4= 2022-09-03T06:13:34.599 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:424GVxME3Dj1Kok/0Zkojr1Roe4=', 'Content-Length': '0'} 2022-09-03T06:13:34.604 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ec51cfc52cfba27b-006312f08e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.606 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:34.607 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:34.608 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:34.609 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:34.609 DEBUG:boto:Method: GET 2022-09-03T06:13:34.610 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:13:34.611 DEBUG:boto:Data: 2022-09-03T06:13:34.611 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.612 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.612 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.613 DEBUG:boto:Params: {} 2022-09-03T06:13:34.613 DEBUG:boto:Token: None 2022-09-03T06:13:34.614 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/ 2022-09-03T06:13:34.615 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY= 2022-09-03T06:13:34.615 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY=', 'Content-Length': '0'} 2022-09-03T06:13:34.618 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b3fe75ea3de7db1a-006312f08e-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.619 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:34.620 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:34.621 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:34.622 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:34.623 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:34.624 DEBUG:boto:Method: GET 2022-09-03T06:13:34.624 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:13:34.625 DEBUG:boto:Data: 2022-09-03T06:13:34.625 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.626 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.626 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.627 DEBUG:boto:Params: {} 2022-09-03T06:13:34.628 DEBUG:boto:Token: None 2022-09-03T06:13:34.629 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/ 2022-09-03T06:13:34.630 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY= 2022-09-03T06:13:34.631 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY=', 'Content-Length': '0'} 2022-09-03T06:13:34.635 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000016ab16d01a2539c-006312f08e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.682 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:34.683 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:34.684 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:34.685 DEBUG:boto:Method: HEAD 2022-09-03T06:13:34.686 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:13:34.687 DEBUG:boto:Data: 2022-09-03T06:13:34.688 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.689 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.690 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.691 DEBUG:boto:Params: {} 2022-09-03T06:13:34.692 DEBUG:boto:Token: None 2022-09-03T06:13:34.693 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/b 2022-09-03T06:13:34.694 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/sDQBp807blm3PKnL93YlkJBCSk= 2022-09-03T06:13:34.695 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/sDQBp807blm3PKnL93YlkJBCSk=', 'Content-Length': '0'} 2022-09-03T06:13:34.697 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000080201357f32dacc1-006312f08e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.698 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:34.699 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:34.700 DEBUG:boto:Method: HEAD 2022-09-03T06:13:34.700 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:13:34.701 DEBUG:boto:Data: 2022-09-03T06:13:34.701 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.702 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.703 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.704 DEBUG:boto:Params: {} 2022-09-03T06:13:34.705 DEBUG:boto:Token: None 2022-09-03T06:13:34.706 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/b 2022-09-03T06:13:34.707 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/sDQBp807blm3PKnL93YlkJBCSk= 2022-09-03T06:13:34.707 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/sDQBp807blm3PKnL93YlkJBCSk=', 'Content-Length': '0'} 2022-09-03T06:13:34.710 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a9021dd2efee1e5-006312f08e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.711 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:13:34.711 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:34.712 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:34.713 DEBUG:boto:Method: GET 2022-09-03T06:13:34.713 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:13:34.714 DEBUG:boto:Data: 2022-09-03T06:13:34.714 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.715 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.716 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.716 DEBUG:boto:Params: {} 2022-09-03T06:13:34.717 DEBUG:boto:Token: None 2022-09-03T06:13:34.718 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/b 2022-09-03T06:13:34.720 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JrgxcZACU7LHkFc3vM2w5TWkspo= 2022-09-03T06:13:34.721 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JrgxcZACU7LHkFc3vM2w5TWkspo=', 'Content-Length': '0'} 2022-09-03T06:13:34.724 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c50555d82e7bb304-006312f08e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.726 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:34.727 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:34.728 DEBUG:boto:Method: GET 2022-09-03T06:13:34.729 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:13:34.729 DEBUG:boto:Data: 2022-09-03T06:13:34.731 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.732 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.732 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.733 DEBUG:boto:Params: {} 2022-09-03T06:13:34.734 DEBUG:boto:Token: None 2022-09-03T06:13:34.735 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/b 2022-09-03T06:13:34.736 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JrgxcZACU7LHkFc3vM2w5TWkspo= 2022-09-03T06:13:34.736 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JrgxcZACU7LHkFc3vM2w5TWkspo=', 'Content-Length': '0'} 2022-09-03T06:13:34.739 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000079881643c4bd008f-006312f08e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.740 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:34.741 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:34.742 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:34.742 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:34.743 DEBUG:boto:Method: GET 2022-09-03T06:13:34.743 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:13:34.744 DEBUG:boto:Data: 2022-09-03T06:13:34.744 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.745 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.746 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.746 DEBUG:boto:Params: {} 2022-09-03T06:13:34.747 DEBUG:boto:Token: None 2022-09-03T06:13:34.747 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/ 2022-09-03T06:13:34.748 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY= 2022-09-03T06:13:34.749 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY=', 'Content-Length': '0'} 2022-09-03T06:13:34.751 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bc04054d67e2fef2-006312f08e-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.752 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:34.753 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:34.754 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:34.755 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:34.755 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:34.756 DEBUG:boto:Method: GET 2022-09-03T06:13:34.756 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:13:34.757 DEBUG:boto:Data: 2022-09-03T06:13:34.757 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.758 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.759 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.759 DEBUG:boto:Params: {} 2022-09-03T06:13:34.760 DEBUG:boto:Token: None 2022-09-03T06:13:34.761 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/ 2022-09-03T06:13:34.761 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY= 2022-09-03T06:13:34.762 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY=', 'Content-Length': '0'} 2022-09-03T06:13:34.765 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000478e90b0234132de-006312f08e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.766 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:34.767 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:13:34.767 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:34.768 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:34.769 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.769 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.770 DEBUG:boto:Method: GET 2022-09-03T06:13:34.771 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.771 DEBUG:boto:Data: 2022-09-03T06:13:34.772 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.773 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.773 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.774 DEBUG:boto:Params: {} 2022-09-03T06:13:34.774 DEBUG:boto:Token: None 2022-09-03T06:13:34.775 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.776 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/t3z+43BtgFvYSpfeHyPh+Ssdng= 2022-09-03T06:13:34.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/t3z+43BtgFvYSpfeHyPh+Ssdng=', 'Content-Length': '0'} 2022-09-03T06:13:34.778 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000240adda87a91f479-006312f08e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.822 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:34.823 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:34.824 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.825 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.826 DEBUG:boto:Method: GET 2022-09-03T06:13:34.826 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.827 DEBUG:boto:Data: 2022-09-03T06:13:34.827 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.828 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.828 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.829 DEBUG:boto:Params: {} 2022-09-03T06:13:34.830 DEBUG:boto:Token: None 2022-09-03T06:13:34.831 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.832 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/t3z+43BtgFvYSpfeHyPh+Ssdng= 2022-09-03T06:13:34.833 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/t3z+43BtgFvYSpfeHyPh+Ssdng=', 'Content-Length': '0'} 2022-09-03T06:13:34.836 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026cc80cadd058520-006312f08e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.837 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:34.838 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:34.838 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.839 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.840 DEBUG:boto:Method: HEAD 2022-09-03T06:13:34.840 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.841 DEBUG:boto:Data: 2022-09-03T06:13:34.841 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.842 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.842 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.843 DEBUG:boto:Params: {} 2022-09-03T06:13:34.844 DEBUG:boto:Token: None 2022-09-03T06:13:34.845 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.845 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Dhtmg4NhCYXfMvu2Z1TgSBci6ww= 2022-09-03T06:13:34.846 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Dhtmg4NhCYXfMvu2Z1TgSBci6ww=', 'Content-Length': '0'} 2022-09-03T06:13:34.848 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e7168240512c25ab-006312f08e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.849 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:34.850 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:34.851 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.852 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.853 DEBUG:boto:Method: HEAD 2022-09-03T06:13:34.854 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.855 DEBUG:boto:Data: 2022-09-03T06:13:34.856 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.857 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.857 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.858 DEBUG:boto:Params: {} 2022-09-03T06:13:34.859 DEBUG:boto:Token: None 2022-09-03T06:13:34.860 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.861 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Dhtmg4NhCYXfMvu2Z1TgSBci6ww= 2022-09-03T06:13:34.861 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Dhtmg4NhCYXfMvu2Z1TgSBci6ww=', 'Content-Length': '0'} 2022-09-03T06:13:34.864 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a2026eb7135e5944-006312f08e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.864 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:13:34.865 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:34.865 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:34.866 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.867 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.867 DEBUG:boto:Method: GET 2022-09-03T06:13:34.868 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.868 DEBUG:boto:Data: 2022-09-03T06:13:34.869 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.869 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.870 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.871 DEBUG:boto:Params: {} 2022-09-03T06:13:34.872 DEBUG:boto:Token: None 2022-09-03T06:13:34.873 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.874 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/t3z+43BtgFvYSpfeHyPh+Ssdng= 2022-09-03T06:13:34.875 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/t3z+43BtgFvYSpfeHyPh+Ssdng=', 'Content-Length': '0'} 2022-09-03T06:13:34.878 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c9e93f9fb8fe6233-006312f08e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.879 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:34.880 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:34.880 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.881 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.881 DEBUG:boto:Method: GET 2022-09-03T06:13:34.882 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.883 DEBUG:boto:Data: 2022-09-03T06:13:34.883 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.884 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.884 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.885 DEBUG:boto:Params: {} 2022-09-03T06:13:34.885 DEBUG:boto:Token: None 2022-09-03T06:13:34.886 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:13:34.887 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/t3z+43BtgFvYSpfeHyPh+Ssdng= 2022-09-03T06:13:34.887 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/t3z+43BtgFvYSpfeHyPh+Ssdng=', 'Content-Length': '0'} 2022-09-03T06:13:34.890 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c9dcf5186b4dc221-006312f08e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.891 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:34.891 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:34.892 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:34.892 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:34.893 DEBUG:boto:Method: GET 2022-09-03T06:13:34.893 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:13:34.894 DEBUG:boto:Data: 2022-09-03T06:13:34.894 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.895 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.895 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.896 DEBUG:boto:Params: {} 2022-09-03T06:13:34.897 DEBUG:boto:Token: None 2022-09-03T06:13:34.897 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/ 2022-09-03T06:13:34.898 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY= 2022-09-03T06:13:34.898 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY=', 'Content-Length': '0'} 2022-09-03T06:13:34.901 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000045f420d2a55e82b2-006312f08e-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.902 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:34.903 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:34.903 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:34.904 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:34.904 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:34.905 DEBUG:boto:Method: GET 2022-09-03T06:13:34.905 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:13:34.906 DEBUG:boto:Data: 2022-09-03T06:13:34.906 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.907 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.907 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.908 DEBUG:boto:Params: {} 2022-09-03T06:13:34.909 DEBUG:boto:Token: None 2022-09-03T06:13:34.909 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/ 2022-09-03T06:13:34.910 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY= 2022-09-03T06:13:34.910 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY=', 'Content-Length': '0'} 2022-09-03T06:13:34.913 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000643fc2099abaa251-006312f08e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.914 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:34.915 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:34.916 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:34.916 DEBUG:boto:Method: HEAD 2022-09-03T06:13:34.917 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:13:34.917 DEBUG:boto:Data: 2022-09-03T06:13:34.918 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.918 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.919 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.919 DEBUG:boto:Params: {} 2022-09-03T06:13:34.920 DEBUG:boto:Token: None 2022-09-03T06:13:34.920 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/c 2022-09-03T06:13:34.921 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jkmAhCr3pssKWLCcyZu4ewt9/G0= 2022-09-03T06:13:34.922 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jkmAhCr3pssKWLCcyZu4ewt9/G0=', 'Content-Length': '0'} 2022-09-03T06:13:34.924 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004297ecb77e0d9e72-006312f08e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.926 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:34.927 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:34.928 DEBUG:boto:Method: HEAD 2022-09-03T06:13:34.929 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:13:34.931 DEBUG:boto:Data: 2022-09-03T06:13:34.932 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.932 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.933 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.934 DEBUG:boto:Params: {} 2022-09-03T06:13:34.935 DEBUG:boto:Token: None 2022-09-03T06:13:34.936 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/c 2022-09-03T06:13:34.937 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jkmAhCr3pssKWLCcyZu4ewt9/G0= 2022-09-03T06:13:34.938 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jkmAhCr3pssKWLCcyZu4ewt9/G0=', 'Content-Length': '0'} 2022-09-03T06:13:34.941 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000664d5a3ca78abc2a-006312f08e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.942 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:13:34.942 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:34.943 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:34.944 DEBUG:boto:Method: GET 2022-09-03T06:13:34.944 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:13:34.945 DEBUG:boto:Data: 2022-09-03T06:13:34.946 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.947 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.948 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.949 DEBUG:boto:Params: {} 2022-09-03T06:13:34.950 DEBUG:boto:Token: None 2022-09-03T06:13:34.951 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/c 2022-09-03T06:13:34.952 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WRbpsBNqlIdAtSnbphjnC73TW2s= 2022-09-03T06:13:34.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WRbpsBNqlIdAtSnbphjnC73TW2s=', 'Content-Length': '0'} 2022-09-03T06:13:34.955 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008d91834f6244003c-006312f08e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.957 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:34.957 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:34.958 DEBUG:boto:Method: GET 2022-09-03T06:13:34.958 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:13:34.959 DEBUG:boto:Data: 2022-09-03T06:13:34.960 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.961 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.962 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.963 DEBUG:boto:Params: {} 2022-09-03T06:13:34.964 DEBUG:boto:Token: None 2022-09-03T06:13:34.965 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/c 2022-09-03T06:13:34.967 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WRbpsBNqlIdAtSnbphjnC73TW2s= 2022-09-03T06:13:34.969 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WRbpsBNqlIdAtSnbphjnC73TW2s=', 'Content-Length': '0'} 2022-09-03T06:13:34.972 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006693c29e434ecd21-006312f08e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.973 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:34.974 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:34.974 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:34.975 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:34.975 DEBUG:boto:Method: GET 2022-09-03T06:13:34.976 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:13:34.976 DEBUG:boto:Data: 2022-09-03T06:13:34.977 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.978 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:34.978 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.979 DEBUG:boto:Params: {} 2022-09-03T06:13:34.979 DEBUG:boto:Token: None 2022-09-03T06:13:34.980 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/ 2022-09-03T06:13:34.981 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY= 2022-09-03T06:13:34.981 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY=', 'Content-Length': '0'} 2022-09-03T06:13:34.984 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008e1b5af9937855ed-006312f08e-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.985 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:34.986 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:34.987 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:34.987 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:34.988 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:34.989 DEBUG:boto:Method: GET 2022-09-03T06:13:34.989 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:13:34.990 DEBUG:boto:Data: 2022-09-03T06:13:34.991 DEBUG:boto:Headers: {} 2022-09-03T06:13:34.991 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:34.992 DEBUG:boto:Port: 8000 2022-09-03T06:13:34.992 DEBUG:boto:Params: {} 2022-09-03T06:13:34.993 DEBUG:boto:Token: None 2022-09-03T06:13:34.994 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:34 GMT /dsgvcy-71/ 2022-09-03T06:13:34.994 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY= 2022-09-03T06:13:34.995 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4eqsvsgnmFpmnRrVZvK+2ewLrdY=', 'Content-Length': '0'} 2022-09-03T06:13:34.998 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000036a17df26e223aee-006312f08e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:34.999 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:34.999 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:13:35.000 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:35.001 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:35.002 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.002 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.003 DEBUG:boto:Method: GET 2022-09-03T06:13:35.003 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.004 DEBUG:boto:Data: 2022-09-03T06:13:35.005 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.005 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:35.006 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.006 DEBUG:boto:Params: {} 2022-09-03T06:13:35.007 DEBUG:boto:Token: None 2022-09-03T06:13:35.008 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.008 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bKCwX03YOoHZS+Eo+W799wb+LZE= 2022-09-03T06:13:35.009 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bKCwX03YOoHZS+Eo+W799wb+LZE=', 'Content-Length': '0'} 2022-09-03T06:13:35.012 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000084aad6fec568b4b0-006312f08f-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.013 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:35.013 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:35.014 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.015 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.015 DEBUG:boto:Method: GET 2022-09-03T06:13:35.016 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.016 DEBUG:boto:Data: 2022-09-03T06:13:35.017 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.017 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:35.018 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.019 DEBUG:boto:Params: {} 2022-09-03T06:13:35.019 DEBUG:boto:Token: None 2022-09-03T06:13:35.020 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.021 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bKCwX03YOoHZS+Eo+W799wb+LZE= 2022-09-03T06:13:35.022 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bKCwX03YOoHZS+Eo+W799wb+LZE=', 'Content-Length': '0'} 2022-09-03T06:13:35.028 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f576417e77beacb-006312f08f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.029 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:35.030 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:35.031 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.032 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.033 DEBUG:boto:Method: HEAD 2022-09-03T06:13:35.034 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.035 DEBUG:boto:Data: 2022-09-03T06:13:35.036 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.037 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:35.038 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.039 DEBUG:boto:Params: {} 2022-09-03T06:13:35.040 DEBUG:boto:Token: None 2022-09-03T06:13:35.041 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.042 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gStb86uBZFnoFZaPJx2rjaLRldg= 2022-09-03T06:13:35.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gStb86uBZFnoFZaPJx2rjaLRldg=', 'Content-Length': '0'} 2022-09-03T06:13:35.046 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007e258ce2adf4ac41-006312f08f-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.047 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:35.049 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:35.050 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.051 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.052 DEBUG:boto:Method: HEAD 2022-09-03T06:13:35.054 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.056 DEBUG:boto:Data: 2022-09-03T06:13:35.057 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.059 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:35.060 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.061 DEBUG:boto:Params: {} 2022-09-03T06:13:35.062 DEBUG:boto:Token: None 2022-09-03T06:13:35.063 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.064 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gStb86uBZFnoFZaPJx2rjaLRldg= 2022-09-03T06:13:35.066 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gStb86uBZFnoFZaPJx2rjaLRldg=', 'Content-Length': '0'} 2022-09-03T06:13:35.070 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e542d63778f40fd7-006312f08f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.071 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:13:35.072 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:35.073 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:35.074 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.075 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.076 DEBUG:boto:Method: GET 2022-09-03T06:13:35.078 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.080 DEBUG:boto:Data: 2022-09-03T06:13:35.081 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.082 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:35.083 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.084 DEBUG:boto:Params: {} 2022-09-03T06:13:35.085 DEBUG:boto:Token: None 2022-09-03T06:13:35.086 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.087 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bKCwX03YOoHZS+Eo+W799wb+LZE= 2022-09-03T06:13:35.089 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bKCwX03YOoHZS+Eo+W799wb+LZE=', 'Content-Length': '0'} 2022-09-03T06:13:35.092 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000055ceb03e91f04c2d-006312f08f-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.093 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:35.094 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:35.095 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.095 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.096 DEBUG:boto:Method: GET 2022-09-03T06:13:35.096 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.097 DEBUG:boto:Data: 2022-09-03T06:13:35.097 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.098 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:35.099 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.100 DEBUG:boto:Params: {} 2022-09-03T06:13:35.100 DEBUG:boto:Token: None 2022-09-03T06:13:35.101 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:13:35.102 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bKCwX03YOoHZS+Eo+W799wb+LZE= 2022-09-03T06:13:35.103 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bKCwX03YOoHZS+Eo+W799wb+LZE=', 'Content-Length': '0'} 2022-09-03T06:13:35.106 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005476f7a64b1f26dd-006312f08f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.153 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:35.154 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:35.155 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:35.156 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:35.157 DEBUG:boto:Method: GET 2022-09-03T06:13:35.159 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:13:35.160 DEBUG:boto:Data: 2022-09-03T06:13:35.161 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.171 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:35.172 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.173 DEBUG:boto:Params: {} 2022-09-03T06:13:35.173 DEBUG:boto:Token: None 2022-09-03T06:13:35.174 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/ 2022-09-03T06:13:35.175 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lq/knDomUHb7SERSNecKwsbu9Tg= 2022-09-03T06:13:35.175 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lq/knDomUHb7SERSNecKwsbu9Tg=', 'Content-Length': '0'} 2022-09-03T06:13:35.179 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003e1c40ef696616bc-006312f08f-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.181 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:35.182 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:35.183 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:35.184 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:35.185 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:35.186 DEBUG:boto:Method: GET 2022-09-03T06:13:35.187 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:13:35.188 DEBUG:boto:Data: 2022-09-03T06:13:35.189 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.190 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:35.191 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.191 DEBUG:boto:Params: {} 2022-09-03T06:13:35.192 DEBUG:boto:Token: None 2022-09-03T06:13:35.194 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/ 2022-09-03T06:13:35.195 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lq/knDomUHb7SERSNecKwsbu9Tg= 2022-09-03T06:13:35.196 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lq/knDomUHb7SERSNecKwsbu9Tg=', 'Content-Length': '0'} 2022-09-03T06:13:35.203 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003c11ce3af9ae8ef7-006312f08f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.204 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:35.207 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:35.208 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:35.209 DEBUG:boto:Method: HEAD 2022-09-03T06:13:35.209 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:13:35.210 DEBUG:boto:Data: 2022-09-03T06:13:35.210 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.211 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:35.211 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.212 DEBUG:boto:Params: {} 2022-09-03T06:13:35.213 DEBUG:boto:Token: None 2022-09-03T06:13:35.213 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/d 2022-09-03T06:13:35.214 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oujsHrUrMXeeYT8Xm6K1U8fTtas= 2022-09-03T06:13:35.214 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oujsHrUrMXeeYT8Xm6K1U8fTtas=', 'Content-Length': '0'} 2022-09-03T06:13:35.217 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e6b3336a63460f0-006312f08f-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.219 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:35.220 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:35.221 DEBUG:boto:Method: HEAD 2022-09-03T06:13:35.222 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:13:35.223 DEBUG:boto:Data: 2022-09-03T06:13:35.225 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.226 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:35.227 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.227 DEBUG:boto:Params: {} 2022-09-03T06:13:35.229 DEBUG:boto:Token: None 2022-09-03T06:13:35.229 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/d 2022-09-03T06:13:35.230 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oujsHrUrMXeeYT8Xm6K1U8fTtas= 2022-09-03T06:13:35.231 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oujsHrUrMXeeYT8Xm6K1U8fTtas=', 'Content-Length': '0'} 2022-09-03T06:13:35.233 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cbe73b1344152e21-006312f08f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.234 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:13:35.235 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:35.235 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:35.236 DEBUG:boto:Method: GET 2022-09-03T06:13:35.236 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:13:35.237 DEBUG:boto:Data: 2022-09-03T06:13:35.238 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.240 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:35.241 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.242 DEBUG:boto:Params: {} 2022-09-03T06:13:35.243 DEBUG:boto:Token: None 2022-09-03T06:13:35.244 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/d 2022-09-03T06:13:35.245 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tELem4o3Q5bvylLqBabYRq9kwdk= 2022-09-03T06:13:35.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tELem4o3Q5bvylLqBabYRq9kwdk=', 'Content-Length': '0'} 2022-09-03T06:13:35.248 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008779e6248a038a8e-006312f08f-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.294 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:35.295 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:35.296 DEBUG:boto:Method: GET 2022-09-03T06:13:35.297 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:13:35.303 DEBUG:boto:Data: 2022-09-03T06:13:35.304 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.305 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:35.309 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.316 DEBUG:boto:Params: {} 2022-09-03T06:13:35.317 DEBUG:boto:Token: None 2022-09-03T06:13:35.318 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/d 2022-09-03T06:13:35.319 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tELem4o3Q5bvylLqBabYRq9kwdk= 2022-09-03T06:13:35.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tELem4o3Q5bvylLqBabYRq9kwdk=', 'Content-Length': '0'} 2022-09-03T06:13:35.328 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000034884ce0eebe1538-006312f08f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.330 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:35.332 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:35.333 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:35.334 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:35.335 DEBUG:boto:Method: GET 2022-09-03T06:13:35.335 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:13:35.336 DEBUG:boto:Data: 2022-09-03T06:13:35.336 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.337 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:35.338 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.338 DEBUG:boto:Params: {} 2022-09-03T06:13:35.339 DEBUG:boto:Token: None 2022-09-03T06:13:35.340 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/ 2022-09-03T06:13:35.341 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lq/knDomUHb7SERSNecKwsbu9Tg= 2022-09-03T06:13:35.342 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lq/knDomUHb7SERSNecKwsbu9Tg=', 'Content-Length': '0'} 2022-09-03T06:13:35.346 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a62bca51f1332957-006312f08f-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.349 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:35.352 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:35.353 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:35.354 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:35.355 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:35.356 DEBUG:boto:Method: GET 2022-09-03T06:13:35.357 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:13:35.358 DEBUG:boto:Data: 2022-09-03T06:13:35.359 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.360 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:35.361 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.363 DEBUG:boto:Params: {} 2022-09-03T06:13:35.364 DEBUG:boto:Token: None 2022-09-03T06:13:35.365 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/ 2022-09-03T06:13:35.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lq/knDomUHb7SERSNecKwsbu9Tg= 2022-09-03T06:13:35.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lq/knDomUHb7SERSNecKwsbu9Tg=', 'Content-Length': '0'} 2022-09-03T06:13:35.372 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e345547eac4ae352-006312f08f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.374 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:35.376 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-71 zones={test-zone2, test-zone1} 2022-09-03T06:13:35.377 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-71 zones={test-zone3, test-zone1} 2022-09-03T06:13:35.378 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:35.379 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:35.380 DEBUG:boto:Method: HEAD 2022-09-03T06:13:35.381 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:13:35.382 DEBUG:boto:Data: 2022-09-03T06:13:35.383 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.383 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:35.384 DEBUG:boto:Port: 8001 2022-09-03T06:13:35.385 DEBUG:boto:Params: {} 2022-09-03T06:13:35.387 DEBUG:boto:Token: None 2022-09-03T06:13:35.388 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/ 2022-09-03T06:13:35.389 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aOJIKhJHaNh4HTsOt4GJzTb2Jm4= 2022-09-03T06:13:35.390 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aOJIKhJHaNh4HTsOt4GJzTb2Jm4=', 'Content-Length': '0'} 2022-09-03T06:13:35.394 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b55e24b019ff7219-006312f08f-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.396 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:35.397 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:35.398 DEBUG:boto:Method: HEAD 2022-09-03T06:13:35.399 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:13:35.400 DEBUG:boto:Data: 2022-09-03T06:13:35.401 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.402 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:35.403 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.404 DEBUG:boto:Params: {} 2022-09-03T06:13:35.405 DEBUG:boto:Token: None 2022-09-03T06:13:35.406 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/ 2022-09-03T06:13:35.408 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aOJIKhJHaNh4HTsOt4GJzTb2Jm4= 2022-09-03T06:13:35.409 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aOJIKhJHaNh4HTsOt4GJzTb2Jm4=', 'Content-Length': '0'} 2022-09-03T06:13:35.412 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000508473228d3548a8-006312f08f-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.413 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:13:35.414 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:35.415 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:35.416 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:35.417 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:35.418 DEBUG:boto:Method: GET 2022-09-03T06:13:35.419 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:35.420 DEBUG:boto:Data: 2022-09-03T06:13:35.421 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.422 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:35.424 DEBUG:boto:Port: 8001 2022-09-03T06:13:35.425 DEBUG:boto:Params: {} 2022-09-03T06:13:35.426 DEBUG:boto:Token: None 2022-09-03T06:13:35.427 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/?versions 2022-09-03T06:13:35.428 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M9nAWDKQnSw7gXdKqUHJxyRf/Gk= 2022-09-03T06:13:35.429 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M9nAWDKQnSw7gXdKqUHJxyRf/Gk=', 'Content-Length': '0'} 2022-09-03T06:13:35.438 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008b39052c6c239eb6-006312f08f-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.440 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:35.441 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:13:35.443 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:13:35.444 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:13:35.445 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:13:35.446 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:13:35.448 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:35.449 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:35.450 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:35.451 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:35.452 DEBUG:boto:Method: GET 2022-09-03T06:13:35.453 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:35.454 DEBUG:boto:Data: 2022-09-03T06:13:35.455 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.456 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:35.457 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.458 DEBUG:boto:Params: {} 2022-09-03T06:13:35.459 DEBUG:boto:Token: None 2022-09-03T06:13:35.460 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/?versions 2022-09-03T06:13:35.461 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M9nAWDKQnSw7gXdKqUHJxyRf/Gk= 2022-09-03T06:13:35.463 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M9nAWDKQnSw7gXdKqUHJxyRf/Gk=', 'Content-Length': '0'} 2022-09-03T06:13:35.467 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cd490e45396ed200-006312f08f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.468 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:35.469 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:13:35.469 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:13:35.470 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:13:35.471 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:13:35.472 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:35.472 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:35.473 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:35.473 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:35.474 DEBUG:boto:Method: GET 2022-09-03T06:13:35.474 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:35.475 DEBUG:boto:Data: 2022-09-03T06:13:35.475 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.476 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:35.477 DEBUG:boto:Port: 8001 2022-09-03T06:13:35.478 DEBUG:boto:Params: {} 2022-09-03T06:13:35.479 DEBUG:boto:Token: None 2022-09-03T06:13:35.479 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/?versions 2022-09-03T06:13:35.480 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M9nAWDKQnSw7gXdKqUHJxyRf/Gk= 2022-09-03T06:13:35.481 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M9nAWDKQnSw7gXdKqUHJxyRf/Gk=', 'Content-Length': '0'} 2022-09-03T06:13:35.484 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000edbc314d85a4aee4-006312f08f-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.485 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:35.487 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:35.488 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:35.489 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:35.489 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:35.490 DEBUG:boto:Method: GET 2022-09-03T06:13:35.491 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:35.491 DEBUG:boto:Data: 2022-09-03T06:13:35.492 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.492 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:35.493 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.493 DEBUG:boto:Params: {} 2022-09-03T06:13:35.494 DEBUG:boto:Token: None 2022-09-03T06:13:35.495 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/?versions 2022-09-03T06:13:35.495 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M9nAWDKQnSw7gXdKqUHJxyRf/Gk= 2022-09-03T06:13:35.496 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M9nAWDKQnSw7gXdKqUHJxyRf/Gk=', 'Content-Length': '0'} 2022-09-03T06:13:35.502 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000928b4fc6a61a04bc-006312f08f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.503 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:35.505 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:13:35.506 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:35.507 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:35.508 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.508 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.509 DEBUG:boto:Method: GET 2022-09-03T06:13:35.511 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.512 DEBUG:boto:Data: 2022-09-03T06:13:35.513 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.514 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:35.515 DEBUG:boto:Port: 8001 2022-09-03T06:13:35.516 DEBUG:boto:Params: {} 2022-09-03T06:13:35.517 DEBUG:boto:Token: None 2022-09-03T06:13:35.518 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.519 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wAlyzfjpVZkEZo2mmyTps0dCQ0E= 2022-09-03T06:13:35.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wAlyzfjpVZkEZo2mmyTps0dCQ0E=', 'Content-Length': '0'} 2022-09-03T06:13:35.523 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000413f19a1265c150f-006312f08f-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.524 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:35.524 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:35.525 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.526 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.527 DEBUG:boto:Method: GET 2022-09-03T06:13:35.529 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.530 DEBUG:boto:Data: 2022-09-03T06:13:35.531 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.532 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:35.533 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.534 DEBUG:boto:Params: {} 2022-09-03T06:13:35.534 DEBUG:boto:Token: None 2022-09-03T06:13:35.535 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.536 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wAlyzfjpVZkEZo2mmyTps0dCQ0E= 2022-09-03T06:13:35.538 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wAlyzfjpVZkEZo2mmyTps0dCQ0E=', 'Content-Length': '0'} 2022-09-03T06:13:35.541 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a7b0ad9e61ed0483-006312f08f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.587 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:35.590 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:35.591 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.592 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.593 DEBUG:boto:Method: HEAD 2022-09-03T06:13:35.594 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.595 DEBUG:boto:Data: 2022-09-03T06:13:35.596 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.597 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:35.598 DEBUG:boto:Port: 8001 2022-09-03T06:13:35.599 DEBUG:boto:Params: {} 2022-09-03T06:13:35.600 DEBUG:boto:Token: None 2022-09-03T06:13:35.601 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.602 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OYgzGTA+6CRG0X7LPgX3eQELL6A= 2022-09-03T06:13:35.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OYgzGTA+6CRG0X7LPgX3eQELL6A=', 'Content-Length': '0'} 2022-09-03T06:13:35.607 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008164e9950ab51456-006312f08f-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.608 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:35.609 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:35.610 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.611 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.612 DEBUG:boto:Method: HEAD 2022-09-03T06:13:35.613 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.613 DEBUG:boto:Data: 2022-09-03T06:13:35.614 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.615 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:35.616 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.616 DEBUG:boto:Params: {} 2022-09-03T06:13:35.617 DEBUG:boto:Token: None 2022-09-03T06:13:35.618 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.618 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OYgzGTA+6CRG0X7LPgX3eQELL6A= 2022-09-03T06:13:35.619 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OYgzGTA+6CRG0X7LPgX3eQELL6A=', 'Content-Length': '0'} 2022-09-03T06:13:35.624 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af72064990302ee5-006312f08f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.625 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:13:35.626 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:35.627 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:35.628 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.629 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.630 DEBUG:boto:Method: GET 2022-09-03T06:13:35.631 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.632 DEBUG:boto:Data: 2022-09-03T06:13:35.633 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.633 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:35.634 DEBUG:boto:Port: 8001 2022-09-03T06:13:35.635 DEBUG:boto:Params: {} 2022-09-03T06:13:35.636 DEBUG:boto:Token: None 2022-09-03T06:13:35.636 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.637 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wAlyzfjpVZkEZo2mmyTps0dCQ0E= 2022-09-03T06:13:35.638 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wAlyzfjpVZkEZo2mmyTps0dCQ0E=', 'Content-Length': '0'} 2022-09-03T06:13:35.640 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000439afe7fdb0a1b4b-006312f08f-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.683 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:35.684 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:35.685 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.686 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.688 DEBUG:boto:Method: GET 2022-09-03T06:13:35.689 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.690 DEBUG:boto:Data: 2022-09-03T06:13:35.691 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.692 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:35.693 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.695 DEBUG:boto:Params: {} 2022-09-03T06:13:35.696 DEBUG:boto:Token: None 2022-09-03T06:13:35.697 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:13:35.698 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wAlyzfjpVZkEZo2mmyTps0dCQ0E= 2022-09-03T06:13:35.699 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wAlyzfjpVZkEZo2mmyTps0dCQ0E=', 'Content-Length': '0'} 2022-09-03T06:13:35.702 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f46ce8d162a24de9-006312f08f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.703 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:35.704 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:35.704 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:35.705 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:35.706 DEBUG:boto:Method: GET 2022-09-03T06:13:35.707 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:13:35.707 DEBUG:boto:Data: 2022-09-03T06:13:35.708 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.708 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:35.709 DEBUG:boto:Port: 8001 2022-09-03T06:13:35.709 DEBUG:boto:Params: {} 2022-09-03T06:13:35.710 DEBUG:boto:Token: None 2022-09-03T06:13:35.711 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/ 2022-09-03T06:13:35.711 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lq/knDomUHb7SERSNecKwsbu9Tg= 2022-09-03T06:13:35.713 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lq/knDomUHb7SERSNecKwsbu9Tg=', 'Content-Length': '0'} 2022-09-03T06:13:35.715 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000011de6026757bd29b-006312f08f-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.716 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:35.717 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:35.717 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:35.718 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:35.719 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:35.719 DEBUG:boto:Method: GET 2022-09-03T06:13:35.720 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:13:35.720 DEBUG:boto:Data: 2022-09-03T06:13:35.721 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.721 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:35.722 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.723 DEBUG:boto:Params: {} 2022-09-03T06:13:35.724 DEBUG:boto:Token: None 2022-09-03T06:13:35.725 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/ 2022-09-03T06:13:35.726 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lq/knDomUHb7SERSNecKwsbu9Tg= 2022-09-03T06:13:35.728 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lq/knDomUHb7SERSNecKwsbu9Tg=', 'Content-Length': '0'} 2022-09-03T06:13:35.734 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000411cd758148990d2-006312f08f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.737 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:35.738 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:35.740 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:35.741 DEBUG:boto:Method: HEAD 2022-09-03T06:13:35.741 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:13:35.742 DEBUG:boto:Data: 2022-09-03T06:13:35.742 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.743 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:35.744 DEBUG:boto:Port: 8001 2022-09-03T06:13:35.744 DEBUG:boto:Params: {} 2022-09-03T06:13:35.745 DEBUG:boto:Token: None 2022-09-03T06:13:35.745 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/a 2022-09-03T06:13:35.746 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TPSbLylzfR7UB4QDabU7LnusLv8= 2022-09-03T06:13:35.747 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TPSbLylzfR7UB4QDabU7LnusLv8=', 'Content-Length': '0'} 2022-09-03T06:13:35.749 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b4666b768eafbc3-006312f08f-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.750 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:35.751 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:35.751 DEBUG:boto:Method: HEAD 2022-09-03T06:13:35.752 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:13:35.752 DEBUG:boto:Data: 2022-09-03T06:13:35.753 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.753 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:35.754 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.754 DEBUG:boto:Params: {} 2022-09-03T06:13:35.755 DEBUG:boto:Token: None 2022-09-03T06:13:35.756 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/a 2022-09-03T06:13:35.756 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TPSbLylzfR7UB4QDabU7LnusLv8= 2022-09-03T06:13:35.757 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TPSbLylzfR7UB4QDabU7LnusLv8=', 'Content-Length': '0'} 2022-09-03T06:13:35.759 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007a428cdbe553113e-006312f08f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.760 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:13:35.761 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:35.762 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:35.762 DEBUG:boto:Method: GET 2022-09-03T06:13:35.763 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:13:35.764 DEBUG:boto:Data: 2022-09-03T06:13:35.764 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.765 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:35.765 DEBUG:boto:Port: 8001 2022-09-03T06:13:35.766 DEBUG:boto:Params: {} 2022-09-03T06:13:35.767 DEBUG:boto:Token: None 2022-09-03T06:13:35.767 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/a 2022-09-03T06:13:35.768 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:unGIwtALt1Q2iog7ZgVYc44UzMU= 2022-09-03T06:13:35.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:unGIwtALt1Q2iog7ZgVYc44UzMU=', 'Content-Length': '0'} 2022-09-03T06:13:35.770 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000565533718df55275-006312f08f-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.772 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:35.772 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:35.773 DEBUG:boto:Method: GET 2022-09-03T06:13:35.773 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:13:35.774 DEBUG:boto:Data: 2022-09-03T06:13:35.774 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.775 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:35.776 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.776 DEBUG:boto:Params: {} 2022-09-03T06:13:35.777 DEBUG:boto:Token: None 2022-09-03T06:13:35.777 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/a 2022-09-03T06:13:35.778 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:unGIwtALt1Q2iog7ZgVYc44UzMU= 2022-09-03T06:13:35.779 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:unGIwtALt1Q2iog7ZgVYc44UzMU=', 'Content-Length': '0'} 2022-09-03T06:13:35.782 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a277ad99b4706eb5-006312f08f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.822 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:35.824 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:35.825 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:35.826 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:35.827 DEBUG:boto:Method: GET 2022-09-03T06:13:35.827 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:13:35.828 DEBUG:boto:Data: 2022-09-03T06:13:35.829 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.830 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:35.831 DEBUG:boto:Port: 8001 2022-09-03T06:13:35.833 DEBUG:boto:Params: {} 2022-09-03T06:13:35.835 DEBUG:boto:Token: None 2022-09-03T06:13:35.835 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/ 2022-09-03T06:13:35.836 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lq/knDomUHb7SERSNecKwsbu9Tg= 2022-09-03T06:13:35.837 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lq/knDomUHb7SERSNecKwsbu9Tg=', 'Content-Length': '0'} 2022-09-03T06:13:35.840 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000799d8591c2e41731-006312f08f-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.841 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:35.842 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:35.843 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:35.843 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:35.844 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:35.845 DEBUG:boto:Method: GET 2022-09-03T06:13:35.845 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:13:35.846 DEBUG:boto:Data: 2022-09-03T06:13:35.847 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.850 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:35.851 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.851 DEBUG:boto:Params: {} 2022-09-03T06:13:35.852 DEBUG:boto:Token: None 2022-09-03T06:13:35.853 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/ 2022-09-03T06:13:35.854 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lq/knDomUHb7SERSNecKwsbu9Tg= 2022-09-03T06:13:35.855 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lq/knDomUHb7SERSNecKwsbu9Tg=', 'Content-Length': '0'} 2022-09-03T06:13:35.859 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d133f9ea91eca45c-006312f08f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.860 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:35.861 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:13:35.862 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:35.863 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:35.863 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.864 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.865 DEBUG:boto:Method: GET 2022-09-03T06:13:35.866 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.867 DEBUG:boto:Data: 2022-09-03T06:13:35.868 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.869 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:35.870 DEBUG:boto:Port: 8001 2022-09-03T06:13:35.870 DEBUG:boto:Params: {} 2022-09-03T06:13:35.871 DEBUG:boto:Token: None 2022-09-03T06:13:35.872 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.872 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D6QfESzUMsmJ66Gfy3Pv6pnSbWI= 2022-09-03T06:13:35.873 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D6QfESzUMsmJ66Gfy3Pv6pnSbWI=', 'Content-Length': '0'} 2022-09-03T06:13:35.876 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005a0a5c8cd45c4ab6-006312f08f-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.877 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:35.877 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:35.878 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.879 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.880 DEBUG:boto:Method: GET 2022-09-03T06:13:35.880 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.881 DEBUG:boto:Data: 2022-09-03T06:13:35.882 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.883 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:35.883 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.884 DEBUG:boto:Params: {} 2022-09-03T06:13:35.884 DEBUG:boto:Token: None 2022-09-03T06:13:35.885 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.886 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D6QfESzUMsmJ66Gfy3Pv6pnSbWI= 2022-09-03T06:13:35.886 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D6QfESzUMsmJ66Gfy3Pv6pnSbWI=', 'Content-Length': '0'} 2022-09-03T06:13:35.888 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000217795035dd2417d-006312f08f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.890 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:35.891 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:35.892 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.893 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.894 DEBUG:boto:Method: HEAD 2022-09-03T06:13:35.896 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.897 DEBUG:boto:Data: 2022-09-03T06:13:35.898 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.899 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:35.900 DEBUG:boto:Port: 8001 2022-09-03T06:13:35.901 DEBUG:boto:Params: {} 2022-09-03T06:13:35.902 DEBUG:boto:Token: None 2022-09-03T06:13:35.903 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.904 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rgjj9UXCxOFX1UyLT5y8UUzNNWY= 2022-09-03T06:13:35.906 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rgjj9UXCxOFX1UyLT5y8UUzNNWY=', 'Content-Length': '0'} 2022-09-03T06:13:35.909 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b29209b6d9323841-006312f08f-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.910 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:35.910 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:35.911 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.912 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.912 DEBUG:boto:Method: HEAD 2022-09-03T06:13:35.913 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.914 DEBUG:boto:Data: 2022-09-03T06:13:35.915 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.915 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:35.916 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.916 DEBUG:boto:Params: {} 2022-09-03T06:13:35.917 DEBUG:boto:Token: None 2022-09-03T06:13:35.918 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.918 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rgjj9UXCxOFX1UyLT5y8UUzNNWY= 2022-09-03T06:13:35.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rgjj9UXCxOFX1UyLT5y8UUzNNWY=', 'Content-Length': '0'} 2022-09-03T06:13:35.922 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000078ce5938834f77fd-006312f08f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.923 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:13:35.924 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:35.925 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:35.926 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.927 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.928 DEBUG:boto:Method: GET 2022-09-03T06:13:35.940 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.941 DEBUG:boto:Data: 2022-09-03T06:13:35.942 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.943 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:35.944 DEBUG:boto:Port: 8001 2022-09-03T06:13:35.945 DEBUG:boto:Params: {} 2022-09-03T06:13:35.947 DEBUG:boto:Token: None 2022-09-03T06:13:35.948 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.949 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D6QfESzUMsmJ66Gfy3Pv6pnSbWI= 2022-09-03T06:13:35.952 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D6QfESzUMsmJ66Gfy3Pv6pnSbWI=', 'Content-Length': '0'} 2022-09-03T06:13:35.955 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000057e84b8647cd0925-006312f08f-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:35.956 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:35.957 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:35.958 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.959 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.960 DEBUG:boto:Method: GET 2022-09-03T06:13:35.963 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.965 DEBUG:boto:Data: 2022-09-03T06:13:35.966 DEBUG:boto:Headers: {} 2022-09-03T06:13:35.967 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:35.967 DEBUG:boto:Port: 8000 2022-09-03T06:13:35.968 DEBUG:boto:Params: {} 2022-09-03T06:13:35.969 DEBUG:boto:Token: None 2022-09-03T06:13:35.969 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:35 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:13:35.970 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D6QfESzUMsmJ66Gfy3Pv6pnSbWI= 2022-09-03T06:13:35.971 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D6QfESzUMsmJ66Gfy3Pv6pnSbWI=', 'Content-Length': '0'} 2022-09-03T06:13:35.973 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000080782c52a16d56aa-006312f08f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.018 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:36.019 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:36.020 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:36.021 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:36.023 DEBUG:boto:Method: GET 2022-09-03T06:13:36.023 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:13:36.024 DEBUG:boto:Data: 2022-09-03T06:13:36.025 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.026 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:36.026 DEBUG:boto:Port: 8001 2022-09-03T06:13:36.027 DEBUG:boto:Params: {} 2022-09-03T06:13:36.028 DEBUG:boto:Token: None 2022-09-03T06:13:36.029 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/ 2022-09-03T06:13:36.030 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g= 2022-09-03T06:13:36.030 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g=', 'Content-Length': '0'} 2022-09-03T06:13:36.033 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000028ff7067105be01f-006312f090-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.034 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:36.035 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:36.036 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:36.037 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:36.037 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:36.040 DEBUG:boto:Method: GET 2022-09-03T06:13:36.041 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:13:36.041 DEBUG:boto:Data: 2022-09-03T06:13:36.042 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.043 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:36.043 DEBUG:boto:Port: 8000 2022-09-03T06:13:36.044 DEBUG:boto:Params: {} 2022-09-03T06:13:36.045 DEBUG:boto:Token: None 2022-09-03T06:13:36.046 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/ 2022-09-03T06:13:36.047 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g= 2022-09-03T06:13:36.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g=', 'Content-Length': '0'} 2022-09-03T06:13:36.051 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c39e014766570ca9-006312f090-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.052 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:36.053 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:36.054 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:36.054 DEBUG:boto:Method: HEAD 2022-09-03T06:13:36.055 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:13:36.055 DEBUG:boto:Data: 2022-09-03T06:13:36.057 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.057 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:36.058 DEBUG:boto:Port: 8001 2022-09-03T06:13:36.058 DEBUG:boto:Params: {} 2022-09-03T06:13:36.059 DEBUG:boto:Token: None 2022-09-03T06:13:36.060 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/b 2022-09-03T06:13:36.060 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AObRxP7cAru+dosQthgaanwLqN0= 2022-09-03T06:13:36.061 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AObRxP7cAru+dosQthgaanwLqN0=', 'Content-Length': '0'} 2022-09-03T06:13:36.064 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000194bbd1391a5fdbb-006312f090-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.064 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:36.065 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:36.065 DEBUG:boto:Method: HEAD 2022-09-03T06:13:36.066 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:13:36.067 DEBUG:boto:Data: 2022-09-03T06:13:36.067 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.068 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:36.068 DEBUG:boto:Port: 8000 2022-09-03T06:13:36.069 DEBUG:boto:Params: {} 2022-09-03T06:13:36.070 DEBUG:boto:Token: None 2022-09-03T06:13:36.071 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/b 2022-09-03T06:13:36.072 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AObRxP7cAru+dosQthgaanwLqN0= 2022-09-03T06:13:36.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AObRxP7cAru+dosQthgaanwLqN0=', 'Content-Length': '0'} 2022-09-03T06:13:36.076 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e6e3b319429ce72-006312f090-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.077 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:13:36.078 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:36.079 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:36.080 DEBUG:boto:Method: GET 2022-09-03T06:13:36.082 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:13:36.083 DEBUG:boto:Data: 2022-09-03T06:13:36.084 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.085 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:36.086 DEBUG:boto:Port: 8001 2022-09-03T06:13:36.086 DEBUG:boto:Params: {} 2022-09-03T06:13:36.087 DEBUG:boto:Token: None 2022-09-03T06:13:36.088 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/b 2022-09-03T06:13:36.089 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BMGkaQs3GIyivYLE27Ey8oWr7Jk= 2022-09-03T06:13:36.090 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BMGkaQs3GIyivYLE27Ey8oWr7Jk=', 'Content-Length': '0'} 2022-09-03T06:13:36.092 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000067eae754be48cec2-006312f090-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.093 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:36.094 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:36.095 DEBUG:boto:Method: GET 2022-09-03T06:13:36.096 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:13:36.097 DEBUG:boto:Data: 2022-09-03T06:13:36.098 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.099 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:36.100 DEBUG:boto:Port: 8000 2022-09-03T06:13:36.101 DEBUG:boto:Params: {} 2022-09-03T06:13:36.102 DEBUG:boto:Token: None 2022-09-03T06:13:36.103 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/b 2022-09-03T06:13:36.104 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BMGkaQs3GIyivYLE27Ey8oWr7Jk= 2022-09-03T06:13:36.106 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BMGkaQs3GIyivYLE27Ey8oWr7Jk=', 'Content-Length': '0'} 2022-09-03T06:13:36.110 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fafcb6135e3c6fb8-006312f090-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.111 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:36.112 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:36.112 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:36.113 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:36.114 DEBUG:boto:Method: GET 2022-09-03T06:13:36.115 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:13:36.116 DEBUG:boto:Data: 2022-09-03T06:13:36.116 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.117 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:36.118 DEBUG:boto:Port: 8001 2022-09-03T06:13:36.119 DEBUG:boto:Params: {} 2022-09-03T06:13:36.120 DEBUG:boto:Token: None 2022-09-03T06:13:36.121 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/ 2022-09-03T06:13:36.122 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g= 2022-09-03T06:13:36.123 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g=', 'Content-Length': '0'} 2022-09-03T06:13:36.126 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000742fe9d89a852180-006312f090-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.127 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:36.128 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:36.129 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:36.130 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:36.131 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:36.132 DEBUG:boto:Method: GET 2022-09-03T06:13:36.133 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:13:36.133 DEBUG:boto:Data: 2022-09-03T06:13:36.134 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.135 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:36.136 DEBUG:boto:Port: 8000 2022-09-03T06:13:36.137 DEBUG:boto:Params: {} 2022-09-03T06:13:36.138 DEBUG:boto:Token: None 2022-09-03T06:13:36.139 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/ 2022-09-03T06:13:36.139 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g= 2022-09-03T06:13:36.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g=', 'Content-Length': '0'} 2022-09-03T06:13:36.144 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e7c06fc576d61f71-006312f090-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.186 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:36.188 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:13:36.189 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:36.190 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:36.191 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.192 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.193 DEBUG:boto:Method: GET 2022-09-03T06:13:36.194 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.195 DEBUG:boto:Data: 2022-09-03T06:13:36.196 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.197 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:36.198 DEBUG:boto:Port: 8001 2022-09-03T06:13:36.198 DEBUG:boto:Params: {} 2022-09-03T06:13:36.200 DEBUG:boto:Token: None 2022-09-03T06:13:36.201 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.203 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v7AJllRnhaCaBLgEshDt06cdE1k= 2022-09-03T06:13:36.203 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v7AJllRnhaCaBLgEshDt06cdE1k=', 'Content-Length': '0'} 2022-09-03T06:13:36.206 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f9c65ee035e820bc-006312f090-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.207 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:36.208 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:36.209 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.210 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.211 DEBUG:boto:Method: GET 2022-09-03T06:13:36.212 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.213 DEBUG:boto:Data: 2022-09-03T06:13:36.214 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.215 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:36.217 DEBUG:boto:Port: 8000 2022-09-03T06:13:36.217 DEBUG:boto:Params: {} 2022-09-03T06:13:36.218 DEBUG:boto:Token: None 2022-09-03T06:13:36.219 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.219 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v7AJllRnhaCaBLgEshDt06cdE1k= 2022-09-03T06:13:36.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v7AJllRnhaCaBLgEshDt06cdE1k=', 'Content-Length': '0'} 2022-09-03T06:13:36.223 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af46caade4306146-006312f090-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.225 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:36.227 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:36.229 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.230 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.231 DEBUG:boto:Method: HEAD 2022-09-03T06:13:36.232 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.233 DEBUG:boto:Data: 2022-09-03T06:13:36.234 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.235 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:36.236 DEBUG:boto:Port: 8001 2022-09-03T06:13:36.237 DEBUG:boto:Params: {} 2022-09-03T06:13:36.238 DEBUG:boto:Token: None 2022-09-03T06:13:36.240 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.242 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QXINf6sLHHEnj8P4Vf+R2CiWY3U= 2022-09-03T06:13:36.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QXINf6sLHHEnj8P4Vf+R2CiWY3U=', 'Content-Length': '0'} 2022-09-03T06:13:36.246 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fff336b9f152d870-006312f090-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.249 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:36.250 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:36.252 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.253 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.254 DEBUG:boto:Method: HEAD 2022-09-03T06:13:36.254 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.255 DEBUG:boto:Data: 2022-09-03T06:13:36.255 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.256 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:36.256 DEBUG:boto:Port: 8000 2022-09-03T06:13:36.257 DEBUG:boto:Params: {} 2022-09-03T06:13:36.258 DEBUG:boto:Token: None 2022-09-03T06:13:36.258 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.259 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QXINf6sLHHEnj8P4Vf+R2CiWY3U= 2022-09-03T06:13:36.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QXINf6sLHHEnj8P4Vf+R2CiWY3U=', 'Content-Length': '0'} 2022-09-03T06:13:36.263 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007d434aa824a996f0-006312f090-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.264 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:13:36.265 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:36.266 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:36.267 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.267 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.268 DEBUG:boto:Method: GET 2022-09-03T06:13:36.268 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.269 DEBUG:boto:Data: 2022-09-03T06:13:36.270 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.270 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:36.271 DEBUG:boto:Port: 8001 2022-09-03T06:13:36.272 DEBUG:boto:Params: {} 2022-09-03T06:13:36.272 DEBUG:boto:Token: None 2022-09-03T06:13:36.273 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.274 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v7AJllRnhaCaBLgEshDt06cdE1k= 2022-09-03T06:13:36.274 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v7AJllRnhaCaBLgEshDt06cdE1k=', 'Content-Length': '0'} 2022-09-03T06:13:36.277 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000023001f88ae2c9e27-006312f090-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.322 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:36.323 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:36.325 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.327 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.328 DEBUG:boto:Method: GET 2022-09-03T06:13:36.329 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.331 DEBUG:boto:Data: 2022-09-03T06:13:36.332 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.333 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:36.334 DEBUG:boto:Port: 8000 2022-09-03T06:13:36.335 DEBUG:boto:Params: {} 2022-09-03T06:13:36.336 DEBUG:boto:Token: None 2022-09-03T06:13:36.337 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:13:36.339 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v7AJllRnhaCaBLgEshDt06cdE1k= 2022-09-03T06:13:36.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v7AJllRnhaCaBLgEshDt06cdE1k=', 'Content-Length': '0'} 2022-09-03T06:13:36.347 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f0a654192d20e334-006312f090-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.349 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:36.349 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:36.350 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:36.351 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:36.351 DEBUG:boto:Method: GET 2022-09-03T06:13:36.352 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:13:36.353 DEBUG:boto:Data: 2022-09-03T06:13:36.354 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.357 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:36.358 DEBUG:boto:Port: 8001 2022-09-03T06:13:36.359 DEBUG:boto:Params: {} 2022-09-03T06:13:36.360 DEBUG:boto:Token: None 2022-09-03T06:13:36.361 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/ 2022-09-03T06:13:36.364 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g= 2022-09-03T06:13:36.365 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g=', 'Content-Length': '0'} 2022-09-03T06:13:36.368 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c4da55b8a0fd8b3d-006312f090-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.369 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:36.370 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:36.371 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:36.371 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:36.372 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:36.372 DEBUG:boto:Method: GET 2022-09-03T06:13:36.374 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:13:36.374 DEBUG:boto:Data: 2022-09-03T06:13:36.375 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.375 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:36.376 DEBUG:boto:Port: 8000 2022-09-03T06:13:36.376 DEBUG:boto:Params: {} 2022-09-03T06:13:36.377 DEBUG:boto:Token: None 2022-09-03T06:13:36.378 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/ 2022-09-03T06:13:36.379 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g= 2022-09-03T06:13:36.379 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g=', 'Content-Length': '0'} 2022-09-03T06:13:36.382 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003f608e663ac8616f-006312f090-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.427 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:36.428 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:36.429 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:36.430 DEBUG:boto:Method: HEAD 2022-09-03T06:13:36.430 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:13:36.431 DEBUG:boto:Data: 2022-09-03T06:13:36.432 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.433 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:36.434 DEBUG:boto:Port: 8001 2022-09-03T06:13:36.435 DEBUG:boto:Params: {} 2022-09-03T06:13:36.436 DEBUG:boto:Token: None 2022-09-03T06:13:36.437 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/c 2022-09-03T06:13:36.439 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:71RO1ddLW+zUOEl39+kDpdbB6ZY= 2022-09-03T06:13:36.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:71RO1ddLW+zUOEl39+kDpdbB6ZY=', 'Content-Length': '0'} 2022-09-03T06:13:36.443 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000053cc18b529a51fde-006312f090-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.445 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:36.446 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:36.447 DEBUG:boto:Method: HEAD 2022-09-03T06:13:36.448 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:13:36.449 DEBUG:boto:Data: 2022-09-03T06:13:36.450 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.452 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:36.453 DEBUG:boto:Port: 8000 2022-09-03T06:13:36.454 DEBUG:boto:Params: {} 2022-09-03T06:13:36.455 DEBUG:boto:Token: None 2022-09-03T06:13:36.456 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/c 2022-09-03T06:13:36.457 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:71RO1ddLW+zUOEl39+kDpdbB6ZY= 2022-09-03T06:13:36.458 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:71RO1ddLW+zUOEl39+kDpdbB6ZY=', 'Content-Length': '0'} 2022-09-03T06:13:36.462 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018cb68c441693162-006312f090-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.463 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:13:36.464 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:36.465 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:36.466 DEBUG:boto:Method: GET 2022-09-03T06:13:36.467 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:13:36.468 DEBUG:boto:Data: 2022-09-03T06:13:36.468 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.469 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:36.470 DEBUG:boto:Port: 8001 2022-09-03T06:13:36.470 DEBUG:boto:Params: {} 2022-09-03T06:13:36.471 DEBUG:boto:Token: None 2022-09-03T06:13:36.472 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/c 2022-09-03T06:13:36.473 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DcuGtJRJ3iQk5NxO+6pW4jBEHt4= 2022-09-03T06:13:36.474 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DcuGtJRJ3iQk5NxO+6pW4jBEHt4=', 'Content-Length': '0'} 2022-09-03T06:13:36.477 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000076e93ba5b8f2f233-006312f090-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.480 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:36.481 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:36.483 DEBUG:boto:Method: GET 2022-09-03T06:13:36.484 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:13:36.485 DEBUG:boto:Data: 2022-09-03T06:13:36.487 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.488 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:36.489 DEBUG:boto:Port: 8000 2022-09-03T06:13:36.490 DEBUG:boto:Params: {} 2022-09-03T06:13:36.492 DEBUG:boto:Token: None 2022-09-03T06:13:36.493 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/c 2022-09-03T06:13:36.494 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DcuGtJRJ3iQk5NxO+6pW4jBEHt4= 2022-09-03T06:13:36.495 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DcuGtJRJ3iQk5NxO+6pW4jBEHt4=', 'Content-Length': '0'} 2022-09-03T06:13:36.499 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000312a1b34c218b20c-006312f090-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.500 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:36.501 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:36.502 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:36.503 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:36.503 DEBUG:boto:Method: GET 2022-09-03T06:13:36.504 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:13:36.504 DEBUG:boto:Data: 2022-09-03T06:13:36.505 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.506 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:36.506 DEBUG:boto:Port: 8001 2022-09-03T06:13:36.507 DEBUG:boto:Params: {} 2022-09-03T06:13:36.507 DEBUG:boto:Token: None 2022-09-03T06:13:36.508 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/ 2022-09-03T06:13:36.509 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g= 2022-09-03T06:13:36.510 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g=', 'Content-Length': '0'} 2022-09-03T06:13:36.519 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009076961c711a5976-006312f090-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.521 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:36.523 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:36.524 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:36.525 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:36.526 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:36.527 DEBUG:boto:Method: GET 2022-09-03T06:13:36.528 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:13:36.529 DEBUG:boto:Data: 2022-09-03T06:13:36.529 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.530 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:36.531 DEBUG:boto:Port: 8000 2022-09-03T06:13:36.532 DEBUG:boto:Params: {} 2022-09-03T06:13:36.532 DEBUG:boto:Token: None 2022-09-03T06:13:36.533 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/ 2022-09-03T06:13:36.534 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g= 2022-09-03T06:13:36.534 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g=', 'Content-Length': '0'} 2022-09-03T06:13:36.537 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aa4fdd8a197663c6-006312f090-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.538 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:36.539 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:13:36.540 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:36.540 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:36.541 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.542 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.542 DEBUG:boto:Method: GET 2022-09-03T06:13:36.543 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.544 DEBUG:boto:Data: 2022-09-03T06:13:36.545 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.546 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:36.546 DEBUG:boto:Port: 8001 2022-09-03T06:13:36.547 DEBUG:boto:Params: {} 2022-09-03T06:13:36.548 DEBUG:boto:Token: None 2022-09-03T06:13:36.550 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.551 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5zmE43hokVn+MUd6xxhYUHWrmG8= 2022-09-03T06:13:36.551 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5zmE43hokVn+MUd6xxhYUHWrmG8=', 'Content-Length': '0'} 2022-09-03T06:13:36.554 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004274fa4f11e81299-006312f090-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.555 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:36.556 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:36.557 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.558 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.559 DEBUG:boto:Method: GET 2022-09-03T06:13:36.560 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.562 DEBUG:boto:Data: 2022-09-03T06:13:36.563 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.564 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:36.565 DEBUG:boto:Port: 8000 2022-09-03T06:13:36.568 DEBUG:boto:Params: {} 2022-09-03T06:13:36.570 DEBUG:boto:Token: None 2022-09-03T06:13:36.571 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.572 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5zmE43hokVn+MUd6xxhYUHWrmG8= 2022-09-03T06:13:36.573 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5zmE43hokVn+MUd6xxhYUHWrmG8=', 'Content-Length': '0'} 2022-09-03T06:13:36.576 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c3449efb0f33be56-006312f090-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.578 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:36.578 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:36.579 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.579 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.580 DEBUG:boto:Method: HEAD 2022-09-03T06:13:36.581 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.582 DEBUG:boto:Data: 2022-09-03T06:13:36.583 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.584 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:36.585 DEBUG:boto:Port: 8001 2022-09-03T06:13:36.586 DEBUG:boto:Params: {} 2022-09-03T06:13:36.587 DEBUG:boto:Token: None 2022-09-03T06:13:36.588 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.588 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YNnxzanVyXukdE7BL8L/+rQeENI= 2022-09-03T06:13:36.589 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YNnxzanVyXukdE7BL8L/+rQeENI=', 'Content-Length': '0'} 2022-09-03T06:13:36.591 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000627d0a26f4584fe4-006312f090-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.592 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:36.593 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:36.594 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.594 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.595 DEBUG:boto:Method: HEAD 2022-09-03T06:13:36.596 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.596 DEBUG:boto:Data: 2022-09-03T06:13:36.597 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.597 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:36.598 DEBUG:boto:Port: 8000 2022-09-03T06:13:36.598 DEBUG:boto:Params: {} 2022-09-03T06:13:36.599 DEBUG:boto:Token: None 2022-09-03T06:13:36.599 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.600 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YNnxzanVyXukdE7BL8L/+rQeENI= 2022-09-03T06:13:36.601 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YNnxzanVyXukdE7BL8L/+rQeENI=', 'Content-Length': '0'} 2022-09-03T06:13:36.603 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fe4df629e7847d20-006312f090-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.604 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:13:36.604 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:36.605 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:36.605 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.606 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.607 DEBUG:boto:Method: GET 2022-09-03T06:13:36.607 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.608 DEBUG:boto:Data: 2022-09-03T06:13:36.608 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.609 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:36.610 DEBUG:boto:Port: 8001 2022-09-03T06:13:36.610 DEBUG:boto:Params: {} 2022-09-03T06:13:36.611 DEBUG:boto:Token: None 2022-09-03T06:13:36.612 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.612 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5zmE43hokVn+MUd6xxhYUHWrmG8= 2022-09-03T06:13:36.613 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5zmE43hokVn+MUd6xxhYUHWrmG8=', 'Content-Length': '0'} 2022-09-03T06:13:36.616 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005897505189d84d86-006312f090-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.662 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:36.663 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:36.664 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.665 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.667 DEBUG:boto:Method: GET 2022-09-03T06:13:36.668 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.669 DEBUG:boto:Data: 2022-09-03T06:13:36.672 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.673 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:36.675 DEBUG:boto:Port: 8000 2022-09-03T06:13:36.676 DEBUG:boto:Params: {} 2022-09-03T06:13:36.677 DEBUG:boto:Token: None 2022-09-03T06:13:36.678 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:13:36.679 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5zmE43hokVn+MUd6xxhYUHWrmG8= 2022-09-03T06:13:36.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5zmE43hokVn+MUd6xxhYUHWrmG8=', 'Content-Length': '0'} 2022-09-03T06:13:36.690 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000fb888bc79504dc3-006312f090-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.691 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:36.692 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:36.692 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:36.693 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:36.693 DEBUG:boto:Method: GET 2022-09-03T06:13:36.694 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:13:36.694 DEBUG:boto:Data: 2022-09-03T06:13:36.695 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.696 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:36.696 DEBUG:boto:Port: 8001 2022-09-03T06:13:36.697 DEBUG:boto:Params: {} 2022-09-03T06:13:36.698 DEBUG:boto:Token: None 2022-09-03T06:13:36.699 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/ 2022-09-03T06:13:36.700 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g= 2022-09-03T06:13:36.701 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g=', 'Content-Length': '0'} 2022-09-03T06:13:36.705 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b6bdce5f737b4ebd-006312f090-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.706 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:36.707 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:36.707 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:36.708 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:36.709 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:36.709 DEBUG:boto:Method: GET 2022-09-03T06:13:36.710 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:13:36.711 DEBUG:boto:Data: 2022-09-03T06:13:36.712 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.712 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:36.713 DEBUG:boto:Port: 8000 2022-09-03T06:13:36.715 DEBUG:boto:Params: {} 2022-09-03T06:13:36.716 DEBUG:boto:Token: None 2022-09-03T06:13:36.717 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/ 2022-09-03T06:13:36.717 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g= 2022-09-03T06:13:36.718 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g=', 'Content-Length': '0'} 2022-09-03T06:13:36.721 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000060848190349bcede-006312f090-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.762 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:36.764 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:36.764 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:36.766 DEBUG:boto:Method: HEAD 2022-09-03T06:13:36.767 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:13:36.768 DEBUG:boto:Data: 2022-09-03T06:13:36.769 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.770 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:36.771 DEBUG:boto:Port: 8001 2022-09-03T06:13:36.772 DEBUG:boto:Params: {} 2022-09-03T06:13:36.773 DEBUG:boto:Token: None 2022-09-03T06:13:36.774 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/d 2022-09-03T06:13:36.775 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QmNUDfcxmIxtpbgYByyP0xOYfo0= 2022-09-03T06:13:36.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QmNUDfcxmIxtpbgYByyP0xOYfo0=', 'Content-Length': '0'} 2022-09-03T06:13:36.779 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000080fb3658d56f4eec-006312f090-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.780 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:36.782 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:36.783 DEBUG:boto:Method: HEAD 2022-09-03T06:13:36.784 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:13:36.785 DEBUG:boto:Data: 2022-09-03T06:13:36.786 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.787 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:36.787 DEBUG:boto:Port: 8000 2022-09-03T06:13:36.788 DEBUG:boto:Params: {} 2022-09-03T06:13:36.789 DEBUG:boto:Token: None 2022-09-03T06:13:36.790 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/d 2022-09-03T06:13:36.791 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QmNUDfcxmIxtpbgYByyP0xOYfo0= 2022-09-03T06:13:36.791 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QmNUDfcxmIxtpbgYByyP0xOYfo0=', 'Content-Length': '0'} 2022-09-03T06:13:36.794 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d3bd62671e1e9da-006312f090-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.794 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:13:36.795 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:36.796 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:36.796 DEBUG:boto:Method: GET 2022-09-03T06:13:36.797 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:13:36.798 DEBUG:boto:Data: 2022-09-03T06:13:36.799 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.800 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:36.801 DEBUG:boto:Port: 8001 2022-09-03T06:13:36.802 DEBUG:boto:Params: {} 2022-09-03T06:13:36.803 DEBUG:boto:Token: None 2022-09-03T06:13:36.804 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/d 2022-09-03T06:13:36.805 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ijcpe+FoTlogtkWIOKzp7W8O6bY= 2022-09-03T06:13:36.807 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ijcpe+FoTlogtkWIOKzp7W8O6bY=', 'Content-Length': '0'} 2022-09-03T06:13:36.809 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e821c7f8e31bf938-006312f090-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.810 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:36.811 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:36.812 DEBUG:boto:Method: GET 2022-09-03T06:13:36.812 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:13:36.813 DEBUG:boto:Data: 2022-09-03T06:13:36.813 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.814 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:36.814 DEBUG:boto:Port: 8000 2022-09-03T06:13:36.815 DEBUG:boto:Params: {} 2022-09-03T06:13:36.816 DEBUG:boto:Token: None 2022-09-03T06:13:36.817 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/d 2022-09-03T06:13:36.817 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ijcpe+FoTlogtkWIOKzp7W8O6bY= 2022-09-03T06:13:36.818 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ijcpe+FoTlogtkWIOKzp7W8O6bY=', 'Content-Length': '0'} 2022-09-03T06:13:36.820 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000003d2321faa68e12c-006312f090-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.821 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:36.822 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:36.822 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:36.823 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:36.824 DEBUG:boto:Method: GET 2022-09-03T06:13:36.824 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:13:36.825 DEBUG:boto:Data: 2022-09-03T06:13:36.825 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.826 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:36.827 DEBUG:boto:Port: 8001 2022-09-03T06:13:36.827 DEBUG:boto:Params: {} 2022-09-03T06:13:36.829 DEBUG:boto:Token: None 2022-09-03T06:13:36.830 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/ 2022-09-03T06:13:36.831 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g= 2022-09-03T06:13:36.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g=', 'Content-Length': '0'} 2022-09-03T06:13:36.836 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000068d00abf41a48bf6-006312f090-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.837 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:36.839 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:36.840 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:36.841 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:36.842 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:36.843 DEBUG:boto:Method: GET 2022-09-03T06:13:36.844 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:13:36.844 DEBUG:boto:Data: 2022-09-03T06:13:36.845 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.846 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:36.847 DEBUG:boto:Port: 8000 2022-09-03T06:13:36.848 DEBUG:boto:Params: {} 2022-09-03T06:13:36.849 DEBUG:boto:Token: None 2022-09-03T06:13:36.850 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/ 2022-09-03T06:13:36.851 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g= 2022-09-03T06:13:36.851 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9i39SzuecbUrX/6bF6zFXWgjw8g=', 'Content-Length': '0'} 2022-09-03T06:13:36.864 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000626111929c0eb900-006312f090-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.865 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:36.866 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-71 zones={test-zone3, test-zone1} 2022-09-03T06:13:36.867 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-71 zones={test-zone3, test-zone2} 2022-09-03T06:13:36.867 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:36.868 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:36.868 DEBUG:boto:Method: HEAD 2022-09-03T06:13:36.869 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:13:36.871 DEBUG:boto:Data: 2022-09-03T06:13:36.871 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.872 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:36.873 DEBUG:boto:Port: 8001 2022-09-03T06:13:36.874 DEBUG:boto:Params: {} 2022-09-03T06:13:36.875 DEBUG:boto:Token: None 2022-09-03T06:13:36.876 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/ 2022-09-03T06:13:36.877 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/TKoMrj/9U1IfCZ61yFcMXWZBqs= 2022-09-03T06:13:36.878 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/TKoMrj/9U1IfCZ61yFcMXWZBqs=', 'Content-Length': '0'} 2022-09-03T06:13:36.881 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005f8721d8c3633c4f-006312f090-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.882 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:36.883 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:36.884 DEBUG:boto:Method: HEAD 2022-09-03T06:13:36.885 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:13:36.885 DEBUG:boto:Data: 2022-09-03T06:13:36.887 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.887 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:36.888 DEBUG:boto:Port: 8000 2022-09-03T06:13:36.889 DEBUG:boto:Params: {} 2022-09-03T06:13:36.891 DEBUG:boto:Token: None 2022-09-03T06:13:36.891 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/ 2022-09-03T06:13:36.892 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/TKoMrj/9U1IfCZ61yFcMXWZBqs= 2022-09-03T06:13:36.893 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/TKoMrj/9U1IfCZ61yFcMXWZBqs=', 'Content-Length': '0'} 2022-09-03T06:13:36.896 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003ccecdaee708c341-006312f090-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.897 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:13:36.898 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:36.899 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:36.900 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:36.900 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:36.901 DEBUG:boto:Method: GET 2022-09-03T06:13:36.902 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:36.903 DEBUG:boto:Data: 2022-09-03T06:13:36.903 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.904 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:36.904 DEBUG:boto:Port: 8001 2022-09-03T06:13:36.905 DEBUG:boto:Params: {} 2022-09-03T06:13:36.906 DEBUG:boto:Token: None 2022-09-03T06:13:36.906 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/?versions 2022-09-03T06:13:36.907 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a3mbet+sL8khWRrqaXCmjgewqew= 2022-09-03T06:13:36.908 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a3mbet+sL8khWRrqaXCmjgewqew=', 'Content-Length': '0'} 2022-09-03T06:13:36.911 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000086b50aba94d4df20-006312f090-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.912 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:36.914 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:13:36.915 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:13:36.916 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:13:36.916 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:13:36.917 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:13:36.918 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:36.919 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:36.921 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:36.921 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:36.922 DEBUG:boto:Method: GET 2022-09-03T06:13:36.923 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:36.923 DEBUG:boto:Data: 2022-09-03T06:13:36.924 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.925 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:36.926 DEBUG:boto:Port: 8000 2022-09-03T06:13:36.926 DEBUG:boto:Params: {} 2022-09-03T06:13:36.927 DEBUG:boto:Token: None 2022-09-03T06:13:36.927 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/?versions 2022-09-03T06:13:36.928 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a3mbet+sL8khWRrqaXCmjgewqew= 2022-09-03T06:13:36.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a3mbet+sL8khWRrqaXCmjgewqew=', 'Content-Length': '0'} 2022-09-03T06:13:36.932 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fc6d2e34d6384f61-006312f090-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.933 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:36.934 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:13:36.935 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:13:36.936 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:13:36.936 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:13:36.938 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:36.938 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:36.939 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:36.940 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:36.940 DEBUG:boto:Method: GET 2022-09-03T06:13:36.942 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:36.943 DEBUG:boto:Data: 2022-09-03T06:13:36.944 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.945 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:36.946 DEBUG:boto:Port: 8001 2022-09-03T06:13:36.947 DEBUG:boto:Params: {} 2022-09-03T06:13:36.948 DEBUG:boto:Token: None 2022-09-03T06:13:36.949 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/?versions 2022-09-03T06:13:36.950 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a3mbet+sL8khWRrqaXCmjgewqew= 2022-09-03T06:13:36.951 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a3mbet+sL8khWRrqaXCmjgewqew=', 'Content-Length': '0'} 2022-09-03T06:13:36.954 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000045316e9c4be6214-006312f090-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.955 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:36.956 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:36.957 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:36.957 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:36.958 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:36.958 DEBUG:boto:Method: GET 2022-09-03T06:13:36.959 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:13:36.959 DEBUG:boto:Data: 2022-09-03T06:13:36.960 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.960 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:36.961 DEBUG:boto:Port: 8000 2022-09-03T06:13:36.961 DEBUG:boto:Params: {} 2022-09-03T06:13:36.962 DEBUG:boto:Token: None 2022-09-03T06:13:36.963 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/?versions 2022-09-03T06:13:36.964 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a3mbet+sL8khWRrqaXCmjgewqew= 2022-09-03T06:13:36.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a3mbet+sL8khWRrqaXCmjgewqew=', 'Content-Length': '0'} 2022-09-03T06:13:36.967 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c0b924e3736828c9-006312f090-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.968 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:36.969 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:13:36.970 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:36.970 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:36.971 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:36.971 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:36.972 DEBUG:boto:Method: GET 2022-09-03T06:13:36.973 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:13:36.973 DEBUG:boto:Data: 2022-09-03T06:13:36.974 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.974 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:36.975 DEBUG:boto:Port: 8001 2022-09-03T06:13:36.975 DEBUG:boto:Params: {} 2022-09-03T06:13:36.976 DEBUG:boto:Token: None 2022-09-03T06:13:36.977 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:13:36.977 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jOC01dz7WirnPCwErDjSQDU70OM= 2022-09-03T06:13:36.978 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jOC01dz7WirnPCwErDjSQDU70OM=', 'Content-Length': '0'} 2022-09-03T06:13:36.980 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b2f3ed388cfc7c3-006312f090-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.981 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:36.981 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:36.982 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:36.983 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:36.983 DEBUG:boto:Method: GET 2022-09-03T06:13:36.984 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:13:36.984 DEBUG:boto:Data: 2022-09-03T06:13:36.985 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.985 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:36.986 DEBUG:boto:Port: 8000 2022-09-03T06:13:36.986 DEBUG:boto:Params: {} 2022-09-03T06:13:36.987 DEBUG:boto:Token: None 2022-09-03T06:13:36.988 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:13:36.988 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jOC01dz7WirnPCwErDjSQDU70OM= 2022-09-03T06:13:36.989 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jOC01dz7WirnPCwErDjSQDU70OM=', 'Content-Length': '0'} 2022-09-03T06:13:36.991 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000badd5538469958b2-006312f090-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:36.992 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:36.993 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:36.993 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:36.994 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:36.994 DEBUG:boto:Method: HEAD 2022-09-03T06:13:36.995 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:13:36.995 DEBUG:boto:Data: 2022-09-03T06:13:36.996 DEBUG:boto:Headers: {} 2022-09-03T06:13:36.996 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:36.997 DEBUG:boto:Port: 8001 2022-09-03T06:13:36.997 DEBUG:boto:Params: {} 2022-09-03T06:13:36.998 DEBUG:boto:Token: None 2022-09-03T06:13:36.999 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:36 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:13:36.999 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:adzXdvYTzS1ZKg1N4YvGowXjD3k= 2022-09-03T06:13:37.000 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:adzXdvYTzS1ZKg1N4YvGowXjD3k=', 'Content-Length': '0'} 2022-09-03T06:13:37.002 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000081b692c96be599b1-006312f090-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.003 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:37.004 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:37.004 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:37.005 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:37.005 DEBUG:boto:Method: HEAD 2022-09-03T06:13:37.006 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:13:37.007 DEBUG:boto:Data: 2022-09-03T06:13:37.007 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.007 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:37.008 DEBUG:boto:Port: 8000 2022-09-03T06:13:37.008 DEBUG:boto:Params: {} 2022-09-03T06:13:37.009 DEBUG:boto:Token: None 2022-09-03T06:13:37.010 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:13:37.011 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zjG1XpPGi7S1VpoRFUNzReLJQ5w= 2022-09-03T06:13:37.011 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zjG1XpPGi7S1VpoRFUNzReLJQ5w=', 'Content-Length': '0'} 2022-09-03T06:13:37.014 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f7c516ecf9b7b7ca-006312f091-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.015 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:13:37.015 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:37.016 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:37.016 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:37.017 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:37.017 DEBUG:boto:Method: GET 2022-09-03T06:13:37.018 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:13:37.018 DEBUG:boto:Data: 2022-09-03T06:13:37.019 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.019 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:37.020 DEBUG:boto:Port: 8001 2022-09-03T06:13:37.020 DEBUG:boto:Params: {} 2022-09-03T06:13:37.021 DEBUG:boto:Token: None 2022-09-03T06:13:37.022 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:13:37.023 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jWI5jOeIvIgFjXqsUIfiB8Hg5rg= 2022-09-03T06:13:37.023 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jWI5jOeIvIgFjXqsUIfiB8Hg5rg=', 'Content-Length': '0'} 2022-09-03T06:13:37.026 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001267682fbc350267-006312f091-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.070 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:37.071 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:37.072 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:37.072 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:13:37.073 DEBUG:boto:Method: GET 2022-09-03T06:13:37.073 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:13:37.074 DEBUG:boto:Data: 2022-09-03T06:13:37.074 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.075 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:37.076 DEBUG:boto:Port: 8000 2022-09-03T06:13:37.076 DEBUG:boto:Params: {} 2022-09-03T06:13:37.077 DEBUG:boto:Token: None 2022-09-03T06:13:37.077 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:13:37.078 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jWI5jOeIvIgFjXqsUIfiB8Hg5rg= 2022-09-03T06:13:37.079 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jWI5jOeIvIgFjXqsUIfiB8Hg5rg=', 'Content-Length': '0'} 2022-09-03T06:13:37.081 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004982186ed21ced6f-006312f091-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.081 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:37.082 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:37.083 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:37.083 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:37.084 DEBUG:boto:Method: GET 2022-09-03T06:13:37.084 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:13:37.085 DEBUG:boto:Data: 2022-09-03T06:13:37.085 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.086 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:37.087 DEBUG:boto:Port: 8001 2022-09-03T06:13:37.088 DEBUG:boto:Params: {} 2022-09-03T06:13:37.089 DEBUG:boto:Token: None 2022-09-03T06:13:37.089 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/ 2022-09-03T06:13:37.090 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc= 2022-09-03T06:13:37.090 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc=', 'Content-Length': '0'} 2022-09-03T06:13:37.093 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f3955a8dd686a309-006312f091-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.094 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:37.095 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:37.095 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:37.096 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:37.097 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:37.097 DEBUG:boto:Method: GET 2022-09-03T06:13:37.098 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:13:37.098 DEBUG:boto:Data: 2022-09-03T06:13:37.099 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.100 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:37.101 DEBUG:boto:Port: 8000 2022-09-03T06:13:37.102 DEBUG:boto:Params: {} 2022-09-03T06:13:37.103 DEBUG:boto:Token: None 2022-09-03T06:13:37.103 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/ 2022-09-03T06:13:37.104 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc= 2022-09-03T06:13:37.105 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc=', 'Content-Length': '0'} 2022-09-03T06:13:37.108 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000120e386a7efdb2f-006312f091-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.109 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:37.111 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:37.112 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:37.113 DEBUG:boto:Method: HEAD 2022-09-03T06:13:37.114 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:13:37.115 DEBUG:boto:Data: 2022-09-03T06:13:37.116 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.117 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:37.118 DEBUG:boto:Port: 8001 2022-09-03T06:13:37.119 DEBUG:boto:Params: {} 2022-09-03T06:13:37.120 DEBUG:boto:Token: None 2022-09-03T06:13:37.121 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/a 2022-09-03T06:13:37.122 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rANaE0GgX9cFzey2GZwNrEXIEQw= 2022-09-03T06:13:37.123 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rANaE0GgX9cFzey2GZwNrEXIEQw=', 'Content-Length': '0'} 2022-09-03T06:13:37.126 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e23837c228d7a86-006312f091-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.128 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:37.129 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:37.130 DEBUG:boto:Method: HEAD 2022-09-03T06:13:37.131 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:13:37.132 DEBUG:boto:Data: 2022-09-03T06:13:37.133 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.134 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:37.135 DEBUG:boto:Port: 8000 2022-09-03T06:13:37.136 DEBUG:boto:Params: {} 2022-09-03T06:13:37.137 DEBUG:boto:Token: None 2022-09-03T06:13:37.138 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/a 2022-09-03T06:13:37.139 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rANaE0GgX9cFzey2GZwNrEXIEQw= 2022-09-03T06:13:37.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rANaE0GgX9cFzey2GZwNrEXIEQw=', 'Content-Length': '0'} 2022-09-03T06:13:37.142 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e78192a8b9845dce-006312f091-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.143 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:13:37.143 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:37.144 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:37.145 DEBUG:boto:Method: GET 2022-09-03T06:13:37.147 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:13:37.148 DEBUG:boto:Data: 2022-09-03T06:13:37.149 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.150 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:37.150 DEBUG:boto:Port: 8001 2022-09-03T06:13:37.151 DEBUG:boto:Params: {} 2022-09-03T06:13:37.152 DEBUG:boto:Token: None 2022-09-03T06:13:37.152 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/a 2022-09-03T06:13:37.153 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vIB8VFRb3WCsu62rDZRVm0EBlOU= 2022-09-03T06:13:37.154 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vIB8VFRb3WCsu62rDZRVm0EBlOU=', 'Content-Length': '0'} 2022-09-03T06:13:37.156 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001df2ec0777007109-006312f091-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.157 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:13:37.157 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:13:37.158 DEBUG:boto:Method: GET 2022-09-03T06:13:37.159 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:13:37.159 DEBUG:boto:Data: 2022-09-03T06:13:37.160 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.160 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:37.161 DEBUG:boto:Port: 8000 2022-09-03T06:13:37.161 DEBUG:boto:Params: {} 2022-09-03T06:13:37.162 DEBUG:boto:Token: None 2022-09-03T06:13:37.163 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/a 2022-09-03T06:13:37.163 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vIB8VFRb3WCsu62rDZRVm0EBlOU= 2022-09-03T06:13:37.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vIB8VFRb3WCsu62rDZRVm0EBlOU=', 'Content-Length': '0'} 2022-09-03T06:13:37.166 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000077afaabe7d289993-006312f091-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.214 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:37.215 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:37.216 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:37.216 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:37.217 DEBUG:boto:Method: GET 2022-09-03T06:13:37.217 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:13:37.218 DEBUG:boto:Data: 2022-09-03T06:13:37.219 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.219 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:37.220 DEBUG:boto:Port: 8001 2022-09-03T06:13:37.220 DEBUG:boto:Params: {} 2022-09-03T06:13:37.221 DEBUG:boto:Token: None 2022-09-03T06:13:37.221 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/ 2022-09-03T06:13:37.222 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc= 2022-09-03T06:13:37.223 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc=', 'Content-Length': '0'} 2022-09-03T06:13:37.225 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b74778ee8d8dc9a0-006312f091-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.226 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:37.227 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:37.228 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:37.228 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:37.229 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:13:37.229 DEBUG:boto:Method: GET 2022-09-03T06:13:37.230 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:13:37.230 DEBUG:boto:Data: 2022-09-03T06:13:37.231 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.231 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:37.232 DEBUG:boto:Port: 8000 2022-09-03T06:13:37.233 DEBUG:boto:Params: {} 2022-09-03T06:13:37.233 DEBUG:boto:Token: None 2022-09-03T06:13:37.234 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/ 2022-09-03T06:13:37.235 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc= 2022-09-03T06:13:37.235 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc=', 'Content-Length': '0'} 2022-09-03T06:13:37.238 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eeddd9ab92877539-006312f091-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.239 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:37.240 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:13:37.240 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:37.241 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:37.241 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.242 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.242 DEBUG:boto:Method: GET 2022-09-03T06:13:37.243 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.243 DEBUG:boto:Data: 2022-09-03T06:13:37.244 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.244 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:37.245 DEBUG:boto:Port: 8001 2022-09-03T06:13:37.246 DEBUG:boto:Params: {} 2022-09-03T06:13:37.246 DEBUG:boto:Token: None 2022-09-03T06:13:37.247 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.247 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IrJp3WgVbpIwrGIWM8JcP6sefZw= 2022-09-03T06:13:37.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IrJp3WgVbpIwrGIWM8JcP6sefZw=', 'Content-Length': '0'} 2022-09-03T06:13:37.250 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bb10343f5d6e65ec-006312f091-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.251 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:37.252 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:37.252 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.253 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.254 DEBUG:boto:Method: GET 2022-09-03T06:13:37.254 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.255 DEBUG:boto:Data: 2022-09-03T06:13:37.255 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.256 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:37.256 DEBUG:boto:Port: 8000 2022-09-03T06:13:37.257 DEBUG:boto:Params: {} 2022-09-03T06:13:37.258 DEBUG:boto:Token: None 2022-09-03T06:13:37.259 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.260 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IrJp3WgVbpIwrGIWM8JcP6sefZw= 2022-09-03T06:13:37.261 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IrJp3WgVbpIwrGIWM8JcP6sefZw=', 'Content-Length': '0'} 2022-09-03T06:13:37.263 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000066d4bf4b9764a00b-006312f091-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.264 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:37.264 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:37.265 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.265 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.266 DEBUG:boto:Method: HEAD 2022-09-03T06:13:37.266 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.267 DEBUG:boto:Data: 2022-09-03T06:13:37.267 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.268 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:37.268 DEBUG:boto:Port: 8001 2022-09-03T06:13:37.269 DEBUG:boto:Params: {} 2022-09-03T06:13:37.269 DEBUG:boto:Token: None 2022-09-03T06:13:37.270 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.271 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:veXzkreb8Tb1dbp2swjiKlzVThs= 2022-09-03T06:13:37.271 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:veXzkreb8Tb1dbp2swjiKlzVThs=', 'Content-Length': '0'} 2022-09-03T06:13:37.274 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000f460f7f442f0aac-006312f091-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.275 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:37.275 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:37.276 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.276 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.277 DEBUG:boto:Method: HEAD 2022-09-03T06:13:37.277 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.278 DEBUG:boto:Data: 2022-09-03T06:13:37.279 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.279 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:37.280 DEBUG:boto:Port: 8000 2022-09-03T06:13:37.280 DEBUG:boto:Params: {} 2022-09-03T06:13:37.281 DEBUG:boto:Token: None 2022-09-03T06:13:37.282 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.283 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:veXzkreb8Tb1dbp2swjiKlzVThs= 2022-09-03T06:13:37.283 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:veXzkreb8Tb1dbp2swjiKlzVThs=', 'Content-Length': '0'} 2022-09-03T06:13:37.286 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000326d972e280af451-006312f091-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.286 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:13:37.287 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:37.288 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:37.288 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.289 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.289 DEBUG:boto:Method: GET 2022-09-03T06:13:37.290 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.291 DEBUG:boto:Data: 2022-09-03T06:13:37.291 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.292 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:37.292 DEBUG:boto:Port: 8001 2022-09-03T06:13:37.293 DEBUG:boto:Params: {} 2022-09-03T06:13:37.294 DEBUG:boto:Token: None 2022-09-03T06:13:37.294 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.295 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IrJp3WgVbpIwrGIWM8JcP6sefZw= 2022-09-03T06:13:37.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IrJp3WgVbpIwrGIWM8JcP6sefZw=', 'Content-Length': '0'} 2022-09-03T06:13:37.298 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fca98842c9bf81ab-006312f091-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.299 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:37.299 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:37.300 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.300 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.301 DEBUG:boto:Method: GET 2022-09-03T06:13:37.301 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.302 DEBUG:boto:Data: 2022-09-03T06:13:37.303 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.303 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:37.304 DEBUG:boto:Port: 8000 2022-09-03T06:13:37.304 DEBUG:boto:Params: {} 2022-09-03T06:13:37.305 DEBUG:boto:Token: None 2022-09-03T06:13:37.306 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:13:37.306 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IrJp3WgVbpIwrGIWM8JcP6sefZw= 2022-09-03T06:13:37.307 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IrJp3WgVbpIwrGIWM8JcP6sefZw=', 'Content-Length': '0'} 2022-09-03T06:13:37.309 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000043f10ab97c2b5268-006312f091-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.354 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:37.356 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:37.357 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:37.357 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:37.358 DEBUG:boto:Method: GET 2022-09-03T06:13:37.359 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:13:37.360 DEBUG:boto:Data: 2022-09-03T06:13:37.361 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.362 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:37.363 DEBUG:boto:Port: 8001 2022-09-03T06:13:37.364 DEBUG:boto:Params: {} 2022-09-03T06:13:37.366 DEBUG:boto:Token: None 2022-09-03T06:13:37.367 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/ 2022-09-03T06:13:37.368 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc= 2022-09-03T06:13:37.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc=', 'Content-Length': '0'} 2022-09-03T06:13:37.377 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000492b909c71993463-006312f091-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.378 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:37.379 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:37.379 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:37.380 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:37.381 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:37.381 DEBUG:boto:Method: GET 2022-09-03T06:13:37.382 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:13:37.382 DEBUG:boto:Data: 2022-09-03T06:13:37.383 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.383 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:37.384 DEBUG:boto:Port: 8000 2022-09-03T06:13:37.385 DEBUG:boto:Params: {} 2022-09-03T06:13:37.385 DEBUG:boto:Token: None 2022-09-03T06:13:37.386 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/ 2022-09-03T06:13:37.387 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc= 2022-09-03T06:13:37.388 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc=', 'Content-Length': '0'} 2022-09-03T06:13:37.390 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f4dc4455fea79028-006312f091-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.392 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:37.394 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:37.395 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:37.395 DEBUG:boto:Method: HEAD 2022-09-03T06:13:37.396 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:13:37.396 DEBUG:boto:Data: 2022-09-03T06:13:37.397 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.398 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:37.398 DEBUG:boto:Port: 8001 2022-09-03T06:13:37.399 DEBUG:boto:Params: {} 2022-09-03T06:13:37.400 DEBUG:boto:Token: None 2022-09-03T06:13:37.400 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/b 2022-09-03T06:13:37.401 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yAMOfR6cDl6IvG9mN9HGB9vAg7c= 2022-09-03T06:13:37.402 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yAMOfR6cDl6IvG9mN9HGB9vAg7c=', 'Content-Length': '0'} 2022-09-03T06:13:37.406 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002c73238d447a1ea5-006312f091-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.408 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:37.409 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:37.410 DEBUG:boto:Method: HEAD 2022-09-03T06:13:37.411 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:13:37.412 DEBUG:boto:Data: 2022-09-03T06:13:37.413 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.413 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:37.414 DEBUG:boto:Port: 8000 2022-09-03T06:13:37.415 DEBUG:boto:Params: {} 2022-09-03T06:13:37.416 DEBUG:boto:Token: None 2022-09-03T06:13:37.417 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/b 2022-09-03T06:13:37.419 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yAMOfR6cDl6IvG9mN9HGB9vAg7c= 2022-09-03T06:13:37.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yAMOfR6cDl6IvG9mN9HGB9vAg7c=', 'Content-Length': '0'} 2022-09-03T06:13:37.422 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006df988db7b2b9aff-006312f091-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.424 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:13:37.425 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:37.426 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:37.427 DEBUG:boto:Method: GET 2022-09-03T06:13:37.428 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:13:37.429 DEBUG:boto:Data: 2022-09-03T06:13:37.430 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.431 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:37.432 DEBUG:boto:Port: 8001 2022-09-03T06:13:37.433 DEBUG:boto:Params: {} 2022-09-03T06:13:37.434 DEBUG:boto:Token: None 2022-09-03T06:13:37.435 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/b 2022-09-03T06:13:37.436 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6HJeJPidIZDguAApDXqN1SSG+lY= 2022-09-03T06:13:37.437 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6HJeJPidIZDguAApDXqN1SSG+lY=', 'Content-Length': '0'} 2022-09-03T06:13:37.440 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2f7e27db4910940-006312f091-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.482 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:13:37.484 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:13:37.485 DEBUG:boto:Method: GET 2022-09-03T06:13:37.486 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:13:37.487 DEBUG:boto:Data: 2022-09-03T06:13:37.487 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.488 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:37.489 DEBUG:boto:Port: 8000 2022-09-03T06:13:37.490 DEBUG:boto:Params: {} 2022-09-03T06:13:37.491 DEBUG:boto:Token: None 2022-09-03T06:13:37.491 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/b 2022-09-03T06:13:37.492 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6HJeJPidIZDguAApDXqN1SSG+lY= 2022-09-03T06:13:37.493 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6HJeJPidIZDguAApDXqN1SSG+lY=', 'Content-Length': '0'} 2022-09-03T06:13:37.499 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005120b1922e526403-006312f091-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.500 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:37.500 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:37.501 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:37.501 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:37.502 DEBUG:boto:Method: GET 2022-09-03T06:13:37.503 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:13:37.503 DEBUG:boto:Data: 2022-09-03T06:13:37.504 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.504 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:37.505 DEBUG:boto:Port: 8001 2022-09-03T06:13:37.506 DEBUG:boto:Params: {} 2022-09-03T06:13:37.507 DEBUG:boto:Token: None 2022-09-03T06:13:37.509 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/ 2022-09-03T06:13:37.511 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc= 2022-09-03T06:13:37.512 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc=', 'Content-Length': '0'} 2022-09-03T06:13:37.518 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005064a4813d46e958-006312f091-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.520 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:37.521 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:37.522 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:37.523 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:37.524 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:13:37.525 DEBUG:boto:Method: GET 2022-09-03T06:13:37.526 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:13:37.527 DEBUG:boto:Data: 2022-09-03T06:13:37.527 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.528 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:37.529 DEBUG:boto:Port: 8000 2022-09-03T06:13:37.529 DEBUG:boto:Params: {} 2022-09-03T06:13:37.531 DEBUG:boto:Token: None 2022-09-03T06:13:37.532 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/ 2022-09-03T06:13:37.532 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc= 2022-09-03T06:13:37.533 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc=', 'Content-Length': '0'} 2022-09-03T06:13:37.535 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006190b12abe7da21a-006312f091-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.583 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:37.584 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:13:37.586 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:37.586 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:37.587 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.588 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.589 DEBUG:boto:Method: GET 2022-09-03T06:13:37.590 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.591 DEBUG:boto:Data: 2022-09-03T06:13:37.592 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.596 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:37.597 DEBUG:boto:Port: 8001 2022-09-03T06:13:37.597 DEBUG:boto:Params: {} 2022-09-03T06:13:37.599 DEBUG:boto:Token: None 2022-09-03T06:13:37.600 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.601 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fxVljNuE6XIdrB6RR2JpsRS4H0c= 2022-09-03T06:13:37.602 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fxVljNuE6XIdrB6RR2JpsRS4H0c=', 'Content-Length': '0'} 2022-09-03T06:13:37.606 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008984042fce4fdcd3-006312f091-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.608 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:37.609 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:37.610 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.611 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.612 DEBUG:boto:Method: GET 2022-09-03T06:13:37.613 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.614 DEBUG:boto:Data: 2022-09-03T06:13:37.615 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.616 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:37.616 DEBUG:boto:Port: 8000 2022-09-03T06:13:37.617 DEBUG:boto:Params: {} 2022-09-03T06:13:37.618 DEBUG:boto:Token: None 2022-09-03T06:13:37.619 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.620 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fxVljNuE6XIdrB6RR2JpsRS4H0c= 2022-09-03T06:13:37.621 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fxVljNuE6XIdrB6RR2JpsRS4H0c=', 'Content-Length': '0'} 2022-09-03T06:13:37.624 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ff3413ed6b44230f-006312f091-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.626 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:37.627 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:37.628 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.629 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.630 DEBUG:boto:Method: HEAD 2022-09-03T06:13:37.631 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.631 DEBUG:boto:Data: 2022-09-03T06:13:37.632 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.632 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:37.633 DEBUG:boto:Port: 8001 2022-09-03T06:13:37.633 DEBUG:boto:Params: {} 2022-09-03T06:13:37.634 DEBUG:boto:Token: None 2022-09-03T06:13:37.635 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.635 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6J/m/w9JIQBlfnb4qyA6AxAPBE0= 2022-09-03T06:13:37.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6J/m/w9JIQBlfnb4qyA6AxAPBE0=', 'Content-Length': '0'} 2022-09-03T06:13:37.638 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009ccaf92eb03180f5-006312f091-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.639 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:37.640 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:37.640 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.641 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.642 DEBUG:boto:Method: HEAD 2022-09-03T06:13:37.643 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.644 DEBUG:boto:Data: 2022-09-03T06:13:37.645 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.645 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:37.646 DEBUG:boto:Port: 8000 2022-09-03T06:13:37.647 DEBUG:boto:Params: {} 2022-09-03T06:13:37.648 DEBUG:boto:Token: None 2022-09-03T06:13:37.650 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.651 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6J/m/w9JIQBlfnb4qyA6AxAPBE0= 2022-09-03T06:13:37.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6J/m/w9JIQBlfnb4qyA6AxAPBE0=', 'Content-Length': '0'} 2022-09-03T06:13:37.655 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074b03c8269ea0528-006312f091-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.656 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:13:37.657 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:37.658 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:37.658 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.659 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.660 DEBUG:boto:Method: GET 2022-09-03T06:13:37.660 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.661 DEBUG:boto:Data: 2022-09-03T06:13:37.661 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.662 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:37.663 DEBUG:boto:Port: 8001 2022-09-03T06:13:37.664 DEBUG:boto:Params: {} 2022-09-03T06:13:37.664 DEBUG:boto:Token: None 2022-09-03T06:13:37.665 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.666 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fxVljNuE6XIdrB6RR2JpsRS4H0c= 2022-09-03T06:13:37.666 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fxVljNuE6XIdrB6RR2JpsRS4H0c=', 'Content-Length': '0'} 2022-09-03T06:13:37.669 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000845dfdcfdc0edfcb-006312f091-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.710 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:37.711 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:37.712 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.713 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.714 DEBUG:boto:Method: GET 2022-09-03T06:13:37.715 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.716 DEBUG:boto:Data: 2022-09-03T06:13:37.717 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.718 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:37.719 DEBUG:boto:Port: 8000 2022-09-03T06:13:37.720 DEBUG:boto:Params: {} 2022-09-03T06:13:37.721 DEBUG:boto:Token: None 2022-09-03T06:13:37.723 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:13:37.724 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fxVljNuE6XIdrB6RR2JpsRS4H0c= 2022-09-03T06:13:37.725 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fxVljNuE6XIdrB6RR2JpsRS4H0c=', 'Content-Length': '0'} 2022-09-03T06:13:37.728 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002ce8ab998b11d83b-006312f091-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.730 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:37.731 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:37.732 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:37.733 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:37.734 DEBUG:boto:Method: GET 2022-09-03T06:13:37.734 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:13:37.735 DEBUG:boto:Data: 2022-09-03T06:13:37.736 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.737 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:37.737 DEBUG:boto:Port: 8001 2022-09-03T06:13:37.738 DEBUG:boto:Params: {} 2022-09-03T06:13:37.738 DEBUG:boto:Token: None 2022-09-03T06:13:37.739 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/ 2022-09-03T06:13:37.741 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc= 2022-09-03T06:13:37.741 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc=', 'Content-Length': '0'} 2022-09-03T06:13:37.745 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ff971a0ab405007c-006312f091-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.746 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:37.746 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:37.747 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:37.748 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:37.748 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:37.749 DEBUG:boto:Method: GET 2022-09-03T06:13:37.749 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:13:37.750 DEBUG:boto:Data: 2022-09-03T06:13:37.751 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.752 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:37.754 DEBUG:boto:Port: 8000 2022-09-03T06:13:37.755 DEBUG:boto:Params: {} 2022-09-03T06:13:37.756 DEBUG:boto:Token: None 2022-09-03T06:13:37.759 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/ 2022-09-03T06:13:37.760 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc= 2022-09-03T06:13:37.761 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc=', 'Content-Length': '0'} 2022-09-03T06:13:37.765 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000082f25ea560801c8d-006312f091-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.767 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:37.768 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:37.769 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:37.770 DEBUG:boto:Method: HEAD 2022-09-03T06:13:37.771 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:13:37.772 DEBUG:boto:Data: 2022-09-03T06:13:37.773 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.774 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:37.775 DEBUG:boto:Port: 8001 2022-09-03T06:13:37.775 DEBUG:boto:Params: {} 2022-09-03T06:13:37.777 DEBUG:boto:Token: None 2022-09-03T06:13:37.778 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/c 2022-09-03T06:13:37.779 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wz8+rxw5TbXHci9oddcnQHQ/528= 2022-09-03T06:13:37.780 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wz8+rxw5TbXHci9oddcnQHQ/528=', 'Content-Length': '0'} 2022-09-03T06:13:37.783 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000de3e1dd2873b0c72-006312f091-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.784 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:37.785 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:37.786 DEBUG:boto:Method: HEAD 2022-09-03T06:13:37.787 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:13:37.787 DEBUG:boto:Data: 2022-09-03T06:13:37.788 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.788 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:37.789 DEBUG:boto:Port: 8000 2022-09-03T06:13:37.789 DEBUG:boto:Params: {} 2022-09-03T06:13:37.790 DEBUG:boto:Token: None 2022-09-03T06:13:37.790 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/c 2022-09-03T06:13:37.791 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wz8+rxw5TbXHci9oddcnQHQ/528= 2022-09-03T06:13:37.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wz8+rxw5TbXHci9oddcnQHQ/528=', 'Content-Length': '0'} 2022-09-03T06:13:37.794 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000546eeecd2c45cec-006312f091-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.795 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:13:37.795 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:37.796 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:37.796 DEBUG:boto:Method: GET 2022-09-03T06:13:37.797 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:13:37.798 DEBUG:boto:Data: 2022-09-03T06:13:37.798 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.799 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:37.799 DEBUG:boto:Port: 8001 2022-09-03T06:13:37.800 DEBUG:boto:Params: {} 2022-09-03T06:13:37.800 DEBUG:boto:Token: None 2022-09-03T06:13:37.801 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/c 2022-09-03T06:13:37.802 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wl6Wlv7f+A/oKtHz8RQbNsth5eI= 2022-09-03T06:13:37.802 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wl6Wlv7f+A/oKtHz8RQbNsth5eI=', 'Content-Length': '0'} 2022-09-03T06:13:37.804 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021693a3401341499-006312f091-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.805 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:13:37.806 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:13:37.807 DEBUG:boto:Method: GET 2022-09-03T06:13:37.807 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:13:37.808 DEBUG:boto:Data: 2022-09-03T06:13:37.808 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.809 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:37.809 DEBUG:boto:Port: 8000 2022-09-03T06:13:37.810 DEBUG:boto:Params: {} 2022-09-03T06:13:37.810 DEBUG:boto:Token: None 2022-09-03T06:13:37.811 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/c 2022-09-03T06:13:37.812 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wl6Wlv7f+A/oKtHz8RQbNsth5eI= 2022-09-03T06:13:37.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wl6Wlv7f+A/oKtHz8RQbNsth5eI=', 'Content-Length': '0'} 2022-09-03T06:13:37.815 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000034e35e2822e8691b-006312f091-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.815 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:37.816 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:37.817 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:37.817 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:37.818 DEBUG:boto:Method: GET 2022-09-03T06:13:37.818 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:13:37.819 DEBUG:boto:Data: 2022-09-03T06:13:37.819 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.820 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:37.820 DEBUG:boto:Port: 8001 2022-09-03T06:13:37.821 DEBUG:boto:Params: {} 2022-09-03T06:13:37.822 DEBUG:boto:Token: None 2022-09-03T06:13:37.822 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/ 2022-09-03T06:13:37.823 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc= 2022-09-03T06:13:37.823 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc=', 'Content-Length': '0'} 2022-09-03T06:13:37.826 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000377b21398b98c353-006312f091-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.827 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:37.828 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:37.828 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:37.829 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:37.829 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:13:37.830 DEBUG:boto:Method: GET 2022-09-03T06:13:37.830 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:13:37.831 DEBUG:boto:Data: 2022-09-03T06:13:37.831 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.832 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:37.832 DEBUG:boto:Port: 8000 2022-09-03T06:13:37.833 DEBUG:boto:Params: {} 2022-09-03T06:13:37.834 DEBUG:boto:Token: None 2022-09-03T06:13:37.834 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/ 2022-09-03T06:13:37.835 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc= 2022-09-03T06:13:37.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vEpOlRZmFf1kVM30eP36OHeMdfc=', 'Content-Length': '0'} 2022-09-03T06:13:37.838 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000da058a4d5b631306-006312f091-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.839 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:37.840 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:13:37.841 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:37.841 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:37.842 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.843 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.843 DEBUG:boto:Method: GET 2022-09-03T06:13:37.844 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.844 DEBUG:boto:Data: 2022-09-03T06:13:37.845 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.846 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:37.847 DEBUG:boto:Port: 8001 2022-09-03T06:13:37.847 DEBUG:boto:Params: {} 2022-09-03T06:13:37.848 DEBUG:boto:Token: None 2022-09-03T06:13:37.849 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.849 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:P1XIoIFN2Twl8lO6KsTB04qB7Gk= 2022-09-03T06:13:37.850 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:P1XIoIFN2Twl8lO6KsTB04qB7Gk=', 'Content-Length': '0'} 2022-09-03T06:13:37.852 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009a45577993d0df67-006312f091-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.894 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:37.896 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:37.897 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.898 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.899 DEBUG:boto:Method: GET 2022-09-03T06:13:37.900 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.900 DEBUG:boto:Data: 2022-09-03T06:13:37.901 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.902 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:37.903 DEBUG:boto:Port: 8000 2022-09-03T06:13:37.904 DEBUG:boto:Params: {} 2022-09-03T06:13:37.905 DEBUG:boto:Token: None 2022-09-03T06:13:37.907 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.908 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:P1XIoIFN2Twl8lO6KsTB04qB7Gk= 2022-09-03T06:13:37.908 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:P1XIoIFN2Twl8lO6KsTB04qB7Gk=', 'Content-Length': '0'} 2022-09-03T06:13:37.910 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000869581ac689516f4-006312f091-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.911 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:37.912 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:37.912 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.913 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.913 DEBUG:boto:Method: HEAD 2022-09-03T06:13:37.914 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.914 DEBUG:boto:Data: 2022-09-03T06:13:37.915 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.915 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:37.916 DEBUG:boto:Port: 8001 2022-09-03T06:13:37.916 DEBUG:boto:Params: {} 2022-09-03T06:13:37.917 DEBUG:boto:Token: None 2022-09-03T06:13:37.918 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.918 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AzUO0LVu1Uro3zIygy/+fC385Pw= 2022-09-03T06:13:37.919 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AzUO0LVu1Uro3zIygy/+fC385Pw=', 'Content-Length': '0'} 2022-09-03T06:13:37.921 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000000c4444c27aee467-006312f091-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.922 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:37.923 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:37.924 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.925 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.926 DEBUG:boto:Method: HEAD 2022-09-03T06:13:37.927 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.928 DEBUG:boto:Data: 2022-09-03T06:13:37.929 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.930 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:37.931 DEBUG:boto:Port: 8000 2022-09-03T06:13:37.932 DEBUG:boto:Params: {} 2022-09-03T06:13:37.933 DEBUG:boto:Token: None 2022-09-03T06:13:37.934 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.935 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AzUO0LVu1Uro3zIygy/+fC385Pw= 2022-09-03T06:13:37.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AzUO0LVu1Uro3zIygy/+fC385Pw=', 'Content-Length': '0'} 2022-09-03T06:13:37.939 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008eebb2fbecbb0efc-006312f091-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.940 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:13:37.941 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:37.942 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:37.943 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.944 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.945 DEBUG:boto:Method: GET 2022-09-03T06:13:37.945 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.946 DEBUG:boto:Data: 2022-09-03T06:13:37.947 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.948 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:37.949 DEBUG:boto:Port: 8001 2022-09-03T06:13:37.950 DEBUG:boto:Params: {} 2022-09-03T06:13:37.961 DEBUG:boto:Token: None 2022-09-03T06:13:37.963 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.964 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:P1XIoIFN2Twl8lO6KsTB04qB7Gk= 2022-09-03T06:13:37.965 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:P1XIoIFN2Twl8lO6KsTB04qB7Gk=', 'Content-Length': '0'} 2022-09-03T06:13:37.968 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a238ce7bf61cf86-006312f091-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:37.971 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:37.972 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:37.973 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.974 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.975 DEBUG:boto:Method: GET 2022-09-03T06:13:37.976 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.977 DEBUG:boto:Data: 2022-09-03T06:13:37.978 DEBUG:boto:Headers: {} 2022-09-03T06:13:37.979 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:37.980 DEBUG:boto:Port: 8000 2022-09-03T06:13:37.980 DEBUG:boto:Params: {} 2022-09-03T06:13:37.981 DEBUG:boto:Token: None 2022-09-03T06:13:37.982 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:37 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:13:37.983 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:P1XIoIFN2Twl8lO6KsTB04qB7Gk= 2022-09-03T06:13:37.983 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:P1XIoIFN2Twl8lO6KsTB04qB7Gk=', 'Content-Length': '0'} 2022-09-03T06:13:37.986 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ac589f7a2b3c453c-006312f091-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:38.026 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:38.028 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:38.029 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:38.030 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:38.031 DEBUG:boto:Method: GET 2022-09-03T06:13:38.032 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:13:38.033 DEBUG:boto:Data: 2022-09-03T06:13:38.034 DEBUG:boto:Headers: {} 2022-09-03T06:13:38.035 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:38.036 DEBUG:boto:Port: 8001 2022-09-03T06:13:38.037 DEBUG:boto:Params: {} 2022-09-03T06:13:38.038 DEBUG:boto:Token: None 2022-09-03T06:13:38.039 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:38 GMT /dsgvcy-71/ 2022-09-03T06:13:38.040 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:apBwabUqeVbWFRjOu4t7yVUGTS0= 2022-09-03T06:13:38.041 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:apBwabUqeVbWFRjOu4t7yVUGTS0=', 'Content-Length': '0'} 2022-09-03T06:13:38.045 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000188a89ed7ca0c1f9-006312f092-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:38.046 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:38.047 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:38.048 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:38.049 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:38.050 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:38.051 DEBUG:boto:Method: GET 2022-09-03T06:13:38.052 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:13:38.053 DEBUG:boto:Data: 2022-09-03T06:13:38.054 DEBUG:boto:Headers: {} 2022-09-03T06:13:38.055 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:38.056 DEBUG:boto:Port: 8000 2022-09-03T06:13:38.057 DEBUG:boto:Params: {} 2022-09-03T06:13:38.058 DEBUG:boto:Token: None 2022-09-03T06:13:38.059 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:38 GMT /dsgvcy-71/ 2022-09-03T06:13:38.060 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:apBwabUqeVbWFRjOu4t7yVUGTS0= 2022-09-03T06:13:38.061 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:apBwabUqeVbWFRjOu4t7yVUGTS0=', 'Content-Length': '0'} 2022-09-03T06:13:38.065 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f969c9703aac9dfd-006312f092-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:38.067 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:38.068 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:38.069 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:38.070 DEBUG:boto:Method: HEAD 2022-09-03T06:13:38.070 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:13:38.071 DEBUG:boto:Data: 2022-09-03T06:13:38.072 DEBUG:boto:Headers: {} 2022-09-03T06:13:38.073 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:38.074 DEBUG:boto:Port: 8001 2022-09-03T06:13:38.075 DEBUG:boto:Params: {} 2022-09-03T06:13:38.076 DEBUG:boto:Token: None 2022-09-03T06:13:38.077 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:38 GMT /dsgvcy-71/d 2022-09-03T06:13:38.078 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:W+iWkGuATXPZpKYPASqANFOpx5E= 2022-09-03T06:13:38.079 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:W+iWkGuATXPZpKYPASqANFOpx5E=', 'Content-Length': '0'} 2022-09-03T06:13:38.081 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000063e402d08d5b3079-006312f092-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:38.083 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:38.084 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:38.084 DEBUG:boto:Method: HEAD 2022-09-03T06:13:38.085 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:13:38.086 DEBUG:boto:Data: 2022-09-03T06:13:38.087 DEBUG:boto:Headers: {} 2022-09-03T06:13:38.088 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:38.089 DEBUG:boto:Port: 8000 2022-09-03T06:13:38.090 DEBUG:boto:Params: {} 2022-09-03T06:13:38.091 DEBUG:boto:Token: None 2022-09-03T06:13:38.092 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:38 GMT /dsgvcy-71/d 2022-09-03T06:13:38.093 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:W+iWkGuATXPZpKYPASqANFOpx5E= 2022-09-03T06:13:38.094 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:W+iWkGuATXPZpKYPASqANFOpx5E=', 'Content-Length': '0'} 2022-09-03T06:13:38.097 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000000beaf76cae89e2a-006312f092-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:38.098 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:13:38.099 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:38.100 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:38.101 DEBUG:boto:Method: GET 2022-09-03T06:13:38.102 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:13:38.103 DEBUG:boto:Data: 2022-09-03T06:13:38.104 DEBUG:boto:Headers: {} 2022-09-03T06:13:38.105 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:38.106 DEBUG:boto:Port: 8001 2022-09-03T06:13:38.107 DEBUG:boto:Params: {} 2022-09-03T06:13:38.109 DEBUG:boto:Token: None 2022-09-03T06:13:38.110 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:38 GMT /dsgvcy-71/d 2022-09-03T06:13:38.111 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1QQ42/dsslHMAdW9Jz2ddvQ+ntY= 2022-09-03T06:13:38.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1QQ42/dsslHMAdW9Jz2ddvQ+ntY=', 'Content-Length': '0'} 2022-09-03T06:13:38.115 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000064d10d141e20b36f-006312f092-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:38.117 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:13:38.118 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:13:38.119 DEBUG:boto:Method: GET 2022-09-03T06:13:38.120 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:13:38.121 DEBUG:boto:Data: 2022-09-03T06:13:38.121 DEBUG:boto:Headers: {} 2022-09-03T06:13:38.123 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:38.123 DEBUG:boto:Port: 8000 2022-09-03T06:13:38.124 DEBUG:boto:Params: {} 2022-09-03T06:13:38.124 DEBUG:boto:Token: None 2022-09-03T06:13:38.125 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:38 GMT /dsgvcy-71/d 2022-09-03T06:13:38.126 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1QQ42/dsslHMAdW9Jz2ddvQ+ntY= 2022-09-03T06:13:38.126 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1QQ42/dsslHMAdW9Jz2ddvQ+ntY=', 'Content-Length': '0'} 2022-09-03T06:13:38.129 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c9d6cb994255d3d-006312f092-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:38.131 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:38.132 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:38.133 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:38.134 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:38.135 DEBUG:boto:Method: GET 2022-09-03T06:13:38.135 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:13:38.136 DEBUG:boto:Data: 2022-09-03T06:13:38.137 DEBUG:boto:Headers: {} 2022-09-03T06:13:38.138 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:13:38.138 DEBUG:boto:Port: 8001 2022-09-03T06:13:38.139 DEBUG:boto:Params: {} 2022-09-03T06:13:38.141 DEBUG:boto:Token: None 2022-09-03T06:13:38.141 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:38 GMT /dsgvcy-71/ 2022-09-03T06:13:38.143 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:apBwabUqeVbWFRjOu4t7yVUGTS0= 2022-09-03T06:13:38.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:apBwabUqeVbWFRjOu4t7yVUGTS0=', 'Content-Length': '0'} 2022-09-03T06:13:38.149 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fa562ba7754327d5-006312f092-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:38.151 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:38.152 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:38.153 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:38.154 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:38.155 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:13:38.156 DEBUG:boto:Method: GET 2022-09-03T06:13:38.157 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:13:38.158 DEBUG:boto:Data: 2022-09-03T06:13:38.158 DEBUG:boto:Headers: {} 2022-09-03T06:13:38.159 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:13:38.160 DEBUG:boto:Port: 8000 2022-09-03T06:13:38.161 DEBUG:boto:Params: {} 2022-09-03T06:13:38.161 DEBUG:boto:Token: None 2022-09-03T06:13:38.162 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:13:38 GMT /dsgvcy-71/ 2022-09-03T06:13:38.163 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:apBwabUqeVbWFRjOu4t7yVUGTS0= 2022-09-03T06:13:38.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:apBwabUqeVbWFRjOu4t7yVUGTS0=', 'Content-Length': '0'} 2022-09-03T06:13:38.167 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005e2283b75f6260b2-006312f092-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:38.168 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:13:38.169 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-71 zones={test-zone3, test-zone2} 2022-09-03T06:13:38.170 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'layout', '--bucket', 'dsgvcy-71', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:13:38.170 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'layout', '--bucket', 'dsgvcy-71', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:13:38.171 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket layout --bucket dsgvcy-71 --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T06:13:38.227 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:13:38.228 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:13:38.303 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T06:13:38.304 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:13:38.304 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding": "None", 2022-09-03T06:13:38.304 INFO:teuthology.orchestra.run.smithi181.stdout: "current_index": { 2022-09-03T06:13:38.305 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 0, 2022-09-03T06:13:38.305 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:13:38.305 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "Normal", 2022-09-03T06:13:38.306 INFO:teuthology.orchestra.run.smithi181.stdout: "normal": { 2022-09-03T06:13:38.306 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 11, 2022-09-03T06:13:38.306 INFO:teuthology.orchestra.run.smithi181.stdout: "hash_type": "Mod" 2022-09-03T06:13:38.306 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:13:38.307 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:13:38.307 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:13:38.307 INFO:teuthology.orchestra.run.smithi181.stdout: "logs": [ 2022-09-03T06:13:38.308 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:13:38.308 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 0, 2022-09-03T06:13:38.308 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:13:38.308 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "InIndex", 2022-09-03T06:13:38.309 INFO:teuthology.orchestra.run.smithi181.stdout: "in_index": { 2022-09-03T06:13:38.309 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 0, 2022-09-03T06:13:38.309 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:13:38.309 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 11, 2022-09-03T06:13:38.310 INFO:teuthology.orchestra.run.smithi181.stdout: "hash_type": "Mod" 2022-09-03T06:13:38.310 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:13:38.310 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:13:38.310 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:13:38.311 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:13:38.311 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:13:38.311 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:13:38.312 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T06:13:38.313 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'list', '--bucket', 'dsgvcy-71', '--gen', '0', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:13:38.314 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'list', '--bucket', 'dsgvcy-71', '--gen', '0', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:13:38.314 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog list --bucket dsgvcy-71 --gen 0 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:13:38.366 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:13:38.366 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:13:38.416 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T06:13:38.417 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:13:38.418 INFO:teuthology.orchestra.run.smithi181.stdout: "op_id": "1#00000000001.42.5", 2022-09-03T06:13:38.418 INFO:teuthology.orchestra.run.smithi181.stdout: "op_tag": "91786ef1-cdb4-4061-8a3e-2a28cc497285.5591.12080020746754676140", 2022-09-03T06:13:38.419 INFO:teuthology.orchestra.run.smithi181.stdout: "op": "write", 2022-09-03T06:13:38.419 INFO:teuthology.orchestra.run.smithi181.stdout: "object": "a", 2022-09-03T06:13:38.419 INFO:teuthology.orchestra.run.smithi181.stdout: "instance": "", 2022-09-03T06:13:38.420 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "complete", 2022-09-03T06:13:38.421 INFO:teuthology.orchestra.run.smithi181.stdout: "index_ver": 1, 2022-09-03T06:13:38.421 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:12:47.277765879Z", 2022-09-03T06:13:38.422 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": { 2022-09-03T06:13:38.422 INFO:teuthology.orchestra.run.smithi181.stdout: "pool": 9, 2022-09-03T06:13:38.423 INFO:teuthology.orchestra.run.smithi181.stdout: "epoch": 4 2022-09-03T06:13:38.423 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:13:38.423 INFO:teuthology.orchestra.run.smithi181.stdout: "bilog_flags": 0, 2022-09-03T06:13:38.424 INFO:teuthology.orchestra.run.smithi181.stdout: "versioned": false, 2022-09-03T06:13:38.424 INFO:teuthology.orchestra.run.smithi181.stdout: "owner": "", 2022-09-03T06:13:38.424 INFO:teuthology.orchestra.run.smithi181.stdout: "owner_display_name": "", 2022-09-03T06:13:38.425 INFO:teuthology.orchestra.run.smithi181.stdout: "zones_trace": [ 2022-09-03T06:13:38.425 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:13:38.425 INFO:teuthology.orchestra.run.smithi181.stdout: "entry": "91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-71:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.22" 2022-09-03T06:13:38.425 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:13:38.426 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:13:38.427 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:13:38.428 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:13:38.428 INFO:teuthology.orchestra.run.smithi181.stdout: "op_id": "5#00000000001.49.5", 2022-09-03T06:13:38.429 INFO:teuthology.orchestra.run.smithi181.stdout: "op_tag": "91786ef1-cdb4-4061-8a3e-2a28cc497285.5591.5460852047775025054", 2022-09-03T06:13:38.429 INFO:teuthology.orchestra.run.smithi181.stdout: "op": "write", 2022-09-03T06:13:38.430 INFO:teuthology.orchestra.run.smithi181.stdout: "object": "d", 2022-09-03T06:13:38.430 INFO:teuthology.orchestra.run.smithi181.stdout: "instance": "", 2022-09-03T06:13:38.431 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "complete", 2022-09-03T06:13:38.431 INFO:teuthology.orchestra.run.smithi181.stdout: "index_ver": 1, 2022-09-03T06:13:38.432 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:12:47.418273536Z", 2022-09-03T06:13:38.432 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": { 2022-09-03T06:13:38.433 INFO:teuthology.orchestra.run.smithi181.stdout: "pool": 9, 2022-09-03T06:13:38.433 INFO:teuthology.orchestra.run.smithi181.stdout: "epoch": 6 2022-09-03T06:13:38.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:13:38.434 INFO:teuthology.orchestra.run.smithi181.stdout: "bilog_flags": 0, 2022-09-03T06:13:38.435 INFO:teuthology.orchestra.run.smithi181.stdout: "versioned": false, 2022-09-03T06:13:38.435 INFO:teuthology.orchestra.run.smithi181.stdout: "owner": "", 2022-09-03T06:13:38.436 INFO:teuthology.orchestra.run.smithi181.stdout: "owner_display_name": "", 2022-09-03T06:13:38.440 INFO:teuthology.orchestra.run.smithi181.stdout: "zones_trace": [ 2022-09-03T06:13:38.440 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:13:38.441 INFO:teuthology.orchestra.run.smithi181.stdout: "entry": "91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-71:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.22" 2022-09-03T06:13:38.441 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:13:38.442 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:13:38.442 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:13:38.443 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:13:38.443 INFO:teuthology.orchestra.run.smithi181.stdout: "op_id": "8#00000000001.88.5", 2022-09-03T06:13:38.444 INFO:teuthology.orchestra.run.smithi181.stdout: "op_tag": "91786ef1-cdb4-4061-8a3e-2a28cc497285.5591.11881350929282018689", 2022-09-03T06:13:38.444 INFO:teuthology.orchestra.run.smithi181.stdout: "op": "write", 2022-09-03T06:13:38.445 INFO:teuthology.orchestra.run.smithi181.stdout: "object": "c", 2022-09-03T06:13:38.446 INFO:teuthology.orchestra.run.smithi181.stdout: "instance": "", 2022-09-03T06:13:38.446 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "complete", 2022-09-03T06:13:38.447 INFO:teuthology.orchestra.run.smithi181.stdout: "index_ver": 1, 2022-09-03T06:13:38.447 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:12:47.356015996Z", 2022-09-03T06:13:38.447 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": { 2022-09-03T06:13:38.448 INFO:teuthology.orchestra.run.smithi181.stdout: "pool": 9, 2022-09-03T06:13:38.448 INFO:teuthology.orchestra.run.smithi181.stdout: "epoch": 7 2022-09-03T06:13:38.449 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:13:38.449 INFO:teuthology.orchestra.run.smithi181.stdout: "bilog_flags": 0, 2022-09-03T06:13:38.450 INFO:teuthology.orchestra.run.smithi181.stdout: "versioned": false, 2022-09-03T06:13:38.452 INFO:teuthology.orchestra.run.smithi181.stdout: "owner": "", 2022-09-03T06:13:38.453 INFO:teuthology.orchestra.run.smithi181.stdout: "owner_display_name": "", 2022-09-03T06:13:38.453 INFO:teuthology.orchestra.run.smithi181.stdout: "zones_trace": [ 2022-09-03T06:13:38.454 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:13:38.454 INFO:teuthology.orchestra.run.smithi181.stdout: "entry": "91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-71:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.22" 2022-09-03T06:13:38.455 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:13:38.455 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:13:38.456 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:13:38.456 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:13:38.457 INFO:teuthology.orchestra.run.smithi181.stdout: "op_id": "10#00000000001.45.5", 2022-09-03T06:13:38.457 INFO:teuthology.orchestra.run.smithi181.stdout: "op_tag": "91786ef1-cdb4-4061-8a3e-2a28cc497285.5591.7330907384611993883", 2022-09-03T06:13:38.457 INFO:teuthology.orchestra.run.smithi181.stdout: "op": "write", 2022-09-03T06:13:38.458 INFO:teuthology.orchestra.run.smithi181.stdout: "object": "b", 2022-09-03T06:13:38.458 INFO:teuthology.orchestra.run.smithi181.stdout: "instance": "", 2022-09-03T06:13:38.459 INFO:teuthology.orchestra.run.smithi181.stdout: "state": "complete", 2022-09-03T06:13:38.459 INFO:teuthology.orchestra.run.smithi181.stdout: "index_ver": 1, 2022-09-03T06:13:38.459 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:12:47.312967166Z", 2022-09-03T06:13:38.460 INFO:teuthology.orchestra.run.smithi181.stdout: "ver": { 2022-09-03T06:13:38.460 INFO:teuthology.orchestra.run.smithi181.stdout: "pool": 9, 2022-09-03T06:13:38.461 INFO:teuthology.orchestra.run.smithi181.stdout: "epoch": 6 2022-09-03T06:13:38.461 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:13:38.461 INFO:teuthology.orchestra.run.smithi181.stdout: "bilog_flags": 0, 2022-09-03T06:13:38.462 INFO:teuthology.orchestra.run.smithi181.stdout: "versioned": false, 2022-09-03T06:13:38.462 INFO:teuthology.orchestra.run.smithi181.stdout: "owner": "", 2022-09-03T06:13:38.462 INFO:teuthology.orchestra.run.smithi181.stdout: "owner_display_name": "", 2022-09-03T06:13:38.463 INFO:teuthology.orchestra.run.smithi181.stdout: "zones_trace": [ 2022-09-03T06:13:38.464 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:13:38.465 INFO:teuthology.orchestra.run.smithi181.stdout: "entry": "91786ef1-cdb4-4061-8a3e-2a28cc497285:dsgvcy-71:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.22" 2022-09-03T06:13:38.465 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:13:38.466 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:13:38.466 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:13:38.467 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T06:13:38.469 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--bucket', 'dsgvcy-71', '--num-shards', '3', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:13:38.470 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--bucket', 'dsgvcy-71', '--num-shards', '3', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:13:38.471 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --bucket dsgvcy-71 --num-shards 3 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T06:13:38.528 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:13:38.529 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:13:38.585 INFO:teuthology.orchestra.run.smithi181.stdout:tenant: 2022-09-03T06:13:38.586 INFO:teuthology.orchestra.run.smithi181.stdout:bucket name: dsgvcy-71 2022-09-03T06:13:38.590 INFO:teuthology.orchestra.run.smithi181.stdout:total entries: 4 2022-09-03T06:13:38.647 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'layout', '--bucket', 'dsgvcy-71', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:13:38.649 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'layout', '--bucket', 'dsgvcy-71', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:13:38.651 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket layout --bucket dsgvcy-71 --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T06:13:38.702 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:13:38.703 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:13:38.753 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T06:13:38.753 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:13:38.754 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding": "None", 2022-09-03T06:13:38.755 INFO:teuthology.orchestra.run.smithi181.stdout: "current_index": { 2022-09-03T06:13:38.755 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 1, 2022-09-03T06:13:38.756 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:13:38.756 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "Normal", 2022-09-03T06:13:38.757 INFO:teuthology.orchestra.run.smithi181.stdout: "normal": { 2022-09-03T06:13:38.757 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 3, 2022-09-03T06:13:38.758 INFO:teuthology.orchestra.run.smithi181.stdout: "hash_type": "Mod" 2022-09-03T06:13:38.758 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:13:38.759 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:13:38.759 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:13:38.760 INFO:teuthology.orchestra.run.smithi181.stdout: "logs": [ 2022-09-03T06:13:38.760 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:13:38.761 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 0, 2022-09-03T06:13:38.762 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:13:38.762 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "InIndex", 2022-09-03T06:13:38.762 INFO:teuthology.orchestra.run.smithi181.stdout: "in_index": { 2022-09-03T06:13:38.763 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 0, 2022-09-03T06:13:38.763 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:13:38.763 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 11, 2022-09-03T06:13:38.764 INFO:teuthology.orchestra.run.smithi181.stdout: "hash_type": "Mod" 2022-09-03T06:13:38.764 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:13:38.764 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:13:38.764 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:13:38.765 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:13:38.765 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:13:38.765 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 1, 2022-09-03T06:13:38.765 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:13:38.766 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "InIndex", 2022-09-03T06:13:38.766 INFO:teuthology.orchestra.run.smithi181.stdout: "in_index": { 2022-09-03T06:13:38.766 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 1, 2022-09-03T06:13:38.767 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:13:38.767 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 3, 2022-09-03T06:13:38.767 INFO:teuthology.orchestra.run.smithi181.stdout: "hash_type": "Mod" 2022-09-03T06:13:38.768 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:13:38.768 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:13:38.769 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:13:38.769 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:13:38.770 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:13:38.770 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:13:38.771 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T06:13:38.773 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'list', '--bucket', 'dsgvcy-71', '--gen', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:13:38.774 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'list', '--bucket', 'dsgvcy-71', '--gen', '1', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:13:38.776 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog list --bucket dsgvcy-71 --gen 1 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:13:38.862 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:13:38.863 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:13:38.921 INFO:teuthology.orchestra.run.smithi181.stdout:[] 2022-09-03T06:13:38.923 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:38.924 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:38.925 DEBUG:boto:Method: HEAD 2022-09-03T06:13:38.926 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:13:38.927 DEBUG:boto:Data: 2022-09-03T06:13:38.928 DEBUG:boto:Headers: {} 2022-09-03T06:13:38.929 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:38.930 DEBUG:boto:Port: 8000 2022-09-03T06:13:38.931 DEBUG:boto:Params: {} 2022-09-03T06:13:38.934 DEBUG:boto:Token: None 2022-09-03T06:13:38.935 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:38 GMT /dsgvcy-71/ 2022-09-03T06:13:38.936 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pQn7LRK1svVIngp5wCN6TM9+0T4= 2022-09-03T06:13:38.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pQn7LRK1svVIngp5wCN6TM9+0T4=', 'Content-Length': '0'} 2022-09-03T06:13:38.940 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f0512ea10257a62d-006312f092-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:38.941 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:13:38.942 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:13:38.943 DEBUG:boto:Method: PUT 2022-09-03T06:13:38.944 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:13:38.945 DEBUG:boto:Data: 2022-09-03T06:13:38.946 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:13:38.946 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:38.947 DEBUG:boto:Port: 8000 2022-09-03T06:13:38.948 DEBUG:boto:Params: {} 2022-09-03T06:13:38.949 DEBUG:boto:Token: None 2022-09-03T06:13:38.951 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:13:38 GMT /dsgvcy-71/e 2022-09-03T06:13:38.952 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JBqT/afKt3X5KR1ceLEPTM2yq44= 2022-09-03T06:13:38.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:13:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JBqT/afKt3X5KR1ceLEPTM2yq44='} 2022-09-03T06:13:38.965 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a9f91f1d9cf1d290-006312f092-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:38.966 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:38.966 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:38.967 DEBUG:boto:Method: HEAD 2022-09-03T06:13:38.967 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:13:38.968 DEBUG:boto:Data: 2022-09-03T06:13:38.969 DEBUG:boto:Headers: {} 2022-09-03T06:13:38.969 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:38.971 DEBUG:boto:Port: 8000 2022-09-03T06:13:38.972 DEBUG:boto:Params: {} 2022-09-03T06:13:38.973 DEBUG:boto:Token: None 2022-09-03T06:13:38.974 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:38 GMT /dsgvcy-71/ 2022-09-03T06:13:38.975 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pQn7LRK1svVIngp5wCN6TM9+0T4= 2022-09-03T06:13:38.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pQn7LRK1svVIngp5wCN6TM9+0T4=', 'Content-Length': '0'} 2022-09-03T06:13:38.979 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '15'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000088ab8267455390a-006312f092-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:38.980 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:13:38.981 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:13:38.983 DEBUG:boto:Method: PUT 2022-09-03T06:13:38.984 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:13:38.985 DEBUG:boto:Data: 2022-09-03T06:13:38.986 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:13:38.987 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:38.989 DEBUG:boto:Port: 8000 2022-09-03T06:13:38.990 DEBUG:boto:Params: {} 2022-09-03T06:13:38.991 DEBUG:boto:Token: None 2022-09-03T06:13:38.992 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:13:38 GMT /dsgvcy-71/f 2022-09-03T06:13:38.993 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0MsFElAn1FhHCZDtWW2gC7Dn+9Q= 2022-09-03T06:13:38.994 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:13:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0MsFElAn1FhHCZDtWW2gC7Dn+9Q='} 2022-09-03T06:13:39.039 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004928f5fc1bb8607b-006312f092-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:39.040 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:39.041 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:39.042 DEBUG:boto:Method: HEAD 2022-09-03T06:13:39.043 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:13:39.043 DEBUG:boto:Data: 2022-09-03T06:13:39.044 DEBUG:boto:Headers: {} 2022-09-03T06:13:39.045 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:39.046 DEBUG:boto:Port: 8000 2022-09-03T06:13:39.046 DEBUG:boto:Params: {} 2022-09-03T06:13:39.047 DEBUG:boto:Token: None 2022-09-03T06:13:39.048 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:39 GMT /dsgvcy-71/ 2022-09-03T06:13:39.048 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:huzyTJBD9jusEFSB7Do3ev0Jok0= 2022-09-03T06:13:39.049 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:huzyTJBD9jusEFSB7Do3ev0Jok0=', 'Content-Length': '0'} 2022-09-03T06:13:39.052 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007197ced684bb4a43-006312f093-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:39.053 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:13:39.053 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:13:39.054 DEBUG:boto:Method: PUT 2022-09-03T06:13:39.054 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:13:39.055 DEBUG:boto:Data: 2022-09-03T06:13:39.056 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:13:39.056 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:39.057 DEBUG:boto:Port: 8000 2022-09-03T06:13:39.059 DEBUG:boto:Params: {} 2022-09-03T06:13:39.060 DEBUG:boto:Token: None 2022-09-03T06:13:39.061 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:13:39 GMT /dsgvcy-71/g 2022-09-03T06:13:39.062 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o/v1YDVIbfC3GMSmPq+3Yszeeeo= 2022-09-03T06:13:39.063 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:13:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o/v1YDVIbfC3GMSmPq+3Yszeeeo='} 2022-09-03T06:13:39.073 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ef14775594ff0695-006312f093-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:39.075 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:13:39.075 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:13:39.076 DEBUG:boto:Method: HEAD 2022-09-03T06:13:39.077 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:13:39.078 DEBUG:boto:Data: 2022-09-03T06:13:39.079 DEBUG:boto:Headers: {} 2022-09-03T06:13:39.079 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:39.080 DEBUG:boto:Port: 8000 2022-09-03T06:13:39.080 DEBUG:boto:Params: {} 2022-09-03T06:13:39.081 DEBUG:boto:Token: None 2022-09-03T06:13:39.081 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:13:39 GMT /dsgvcy-71/ 2022-09-03T06:13:39.082 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:huzyTJBD9jusEFSB7Do3ev0Jok0= 2022-09-03T06:13:39.083 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:13:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:huzyTJBD9jusEFSB7Do3ev0Jok0=', 'Content-Length': '0'} 2022-09-03T06:13:39.085 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '21'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000344fd6f6fa15415a-006312f093-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:39.087 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:13:39.088 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:13:39.089 DEBUG:boto:Method: PUT 2022-09-03T06:13:39.089 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:13:39.090 DEBUG:boto:Data: 2022-09-03T06:13:39.091 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:13:39.092 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:13:39.093 DEBUG:boto:Port: 8000 2022-09-03T06:13:39.093 DEBUG:boto:Params: {} 2022-09-03T06:13:39.094 DEBUG:boto:Token: None 2022-09-03T06:13:39.095 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:13:39 GMT /dsgvcy-71/h 2022-09-03T06:13:39.096 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A+mIR0ryAtdmiJpFWecIDrsq2MQ= 2022-09-03T06:13:39.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:13:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A+mIR0ryAtdmiJpFWecIDrsq2MQ='} 2022-09-03T06:13:39.114 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000110cd4957e7e46da-006312f093-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:13:39.115 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-71 2022-09-03T06:13:39.116 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:13:39.118 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:13:39.119 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-71 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:13:39.186 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:13:39.187 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:13:39.230 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:13:39.224+0000 7eff75ca3600 1 waiting to reach latest gen 1, on 0.. 2022-09-03T06:13:44.229 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:13:44.226+0000 7eff75ca3600 1 waiting for incremental sync to catch up: 2022-09-03T06:13:44.230 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , ] 2022-09-03T06:13:44.230 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:13:49.229 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:13:49.227+0000 7eff75ca3600 1 waiting for incremental sync to catch up: 2022-09-03T06:13:49.229 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , ] 2022-09-03T06:13:49.230 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:13:54.230 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:13:54.228+0000 7eff75ca3600 1 waiting for incremental sync to catch up: 2022-09-03T06:13:54.231 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , ] 2022-09-03T06:13:54.231 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:13:59.231 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:13:59.229+0000 7eff75ca3600 1 waiting for incremental sync to catch up: 2022-09-03T06:13:59.231 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , ] 2022-09-03T06:13:59.232 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:14:04.231 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:14:04.229+0000 7eff75ca3600 1 waiting for incremental sync to catch up: 2022-09-03T06:14:04.232 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.55.5, , ] 2022-09-03T06:14:04.232 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:14:09.232 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:14:09.230+0000 7eff75ca3600 1 waiting for incremental sync to catch up: 2022-09-03T06:14:09.233 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.55.5, , ] 2022-09-03T06:14:09.233 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:14:14.233 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:14:14.230+0000 7eff75ca3600 1 waiting for incremental sync to catch up: 2022-09-03T06:14:14.234 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.55.5, , ] 2022-09-03T06:14:14.234 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:14:19.234 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:14:19.232+0000 7eff75ca3600 1 waiting for incremental sync to catch up: 2022-09-03T06:14:19.235 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:14:19.235 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:14:24.235 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:14:24.233+0000 7eff75ca3600 1 waiting for incremental sync to catch up: 2022-09-03T06:14:24.236 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:14:24.236 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:14:29.236 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:14:29.234+0000 7eff75ca3600 1 waiting for incremental sync to catch up: 2022-09-03T06:14:29.237 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:14:29.237 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:14:34.237 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:14:34.236+0000 7eff75ca3600 1 waiting for incremental sync to catch up: 2022-09-03T06:14:34.238 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:14:34.238 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:14:39.238 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:14:39.237+0000 7eff75ca3600 1 waiting for incremental sync to catch up: 2022-09-03T06:14:39.239 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:14:39.239 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:14:44.241 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:14:44.237+0000 7eff75ca3600 1 waiting for incremental sync to catch up: 2022-09-03T06:14:44.242 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:14:44.243 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:14:49.242 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:14:49.237+0000 7eff75ca3600 1 waiting for incremental sync to catch up: 2022-09-03T06:14:49.243 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:14:49.243 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:14:54.240 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:14:54.238+0000 7eff75ca3600 1 waiting for incremental sync to catch up: 2022-09-03T06:14:54.241 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:14:54.241 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:14:59.241 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:14:59.239+0000 7eff75ca3600 1 waiting for incremental sync to catch up: 2022-09-03T06:14:59.242 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:14:59.242 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:15:04.243 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:15:04.239+0000 7eff75ca3600 1 bucket sync caught up with source: 2022-09-03T06:15:04.243 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:15:04.244 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:15:04.244 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:15:04.239+0000 7eff75ca3600 0 bucket checkpoint complete 2022-09-03T06:15:04.260 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-71 2022-09-03T06:15:04.261 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:15:04.262 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:15:04.263 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-71 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:15:04.334 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:15:04.335 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:15:04.378 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:15:04.376+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:15:04.378 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:15:04.378 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:15:09.386 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:15:09.376+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:15:09.387 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:15:09.388 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:15:14.380 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:15:14.375+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:15:14.380 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:15:14.381 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:15:19.381 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:15:19.376+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:15:19.382 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:15:19.382 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:15:24.383 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:15:24.377+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:15:24.384 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:15:24.384 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:15:29.381 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:15:29.378+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:15:29.382 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:15:29.382 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:15:34.382 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:15:34.380+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:15:34.382 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:15:34.382 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:15:39.392 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:15:39.381+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:15:39.393 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:15:39.393 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:15:44.384 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:15:44.382+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:15:44.385 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:15:44.385 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:15:49.385 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:15:49.383+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:15:49.385 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:15:49.385 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:15:54.386 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:15:54.384+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:15:54.386 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:15:54.387 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:15:59.387 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:15:59.385+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:15:59.388 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:15:59.389 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:16:04.388 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:16:04.385+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:16:04.389 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:16:04.390 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:16:09.390 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:16:09.386+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:16:09.390 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:16:09.391 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:16:14.390 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:16:14.387+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:16:14.391 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:16:14.391 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:16:19.393 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:16:19.388+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:16:19.394 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:16:19.397 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:16:24.393 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:16:24.390+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:16:24.394 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:16:24.394 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:16:29.393 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:16:29.391+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:16:29.394 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:16:29.394 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:16:34.400 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:16:34.392+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:16:34.401 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:16:34.401 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:16:39.395 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:16:39.393+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:16:39.395 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:16:39.395 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:16:44.397 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:16:44.394+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:16:44.398 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:16:44.398 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:16:49.397 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:16:49.394+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:16:49.398 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:16:49.398 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:16:54.400 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:16:54.395+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:16:54.400 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:16:54.401 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:16:59.399 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:16:59.396+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:16:59.399 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:16:59.400 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:17:04.400 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:17:04.397+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:17:04.401 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:17:04.401 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:17:09.401 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:17:09.398+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:17:09.401 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:17:09.401 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:17:14.402 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:17:14.400+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:17:14.402 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:17:14.403 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:17:19.403 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:17:19.401+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:17:19.403 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:17:19.403 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:17:24.404 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:17:24.402+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:17:24.405 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:17:24.405 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:17:29.405 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:17:29.403+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:17:29.406 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:17:29.406 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:17:34.406 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:17:34.404+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:17:34.407 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:17:34.407 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:17:39.407 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:17:39.404+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:17:39.407 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:17:39.407 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:17:44.408 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:17:44.405+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:17:44.409 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:17:44.410 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:17:49.410 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:17:49.407+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:17:49.411 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:17:49.412 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:17:54.410 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:17:54.408+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:17:54.411 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:17:54.411 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:17:59.412 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:17:59.409+0000 7f65ab647600 1 waiting for incremental sync to catch up: 2022-09-03T06:17:59.412 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, ] 2022-09-03T06:17:59.412 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:18:04.412 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:18:04.410+0000 7f65ab647600 1 bucket sync caught up with source: 2022-09-03T06:18:04.413 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:18:04.413 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:18:04.413 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:18:04.410+0000 7f65ab647600 0 bucket checkpoint complete 2022-09-03T06:18:04.432 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-71 2022-09-03T06:18:04.435 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:04.436 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:04.438 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-71 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:18:04.505 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:04.505 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:04.544 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:18:04.542+0000 7f1973fc1600 1 bucket sync caught up with source: 2022-09-03T06:18:04.544 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:18:04.544 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.55.5, 00000000002.55.5, 00000000002.31.5] 2022-09-03T06:18:04.545 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:18:04.542+0000 7f1973fc1600 0 bucket checkpoint complete 2022-09-03T06:18:04.560 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-71 2022-09-03T06:18:04.561 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:04.561 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:04.563 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-71 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:18:04.632 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:04.633 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:04.684 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:18:04.682+0000 7f3ac1066600 1 bucket sync caught up with source: 2022-09-03T06:18:04.684 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.736.5, , , 00000000001.284.5, 00000000001.685.5, , , 00000000002.339.5, , 00000000002.422.5] 2022-09-03T06:18:04.685 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.736.5, , , 00000000001.284.5, 00000000001.685.5, , , 00000000002.339.5, , 00000000002.422.5] 2022-09-03T06:18:04.685 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:18:04.682+0000 7f3ac1066600 0 bucket checkpoint complete 2022-09-03T06:18:04.703 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-71 2022-09-03T06:18:04.704 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:04.705 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:04.707 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-71 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:18:04.774 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:04.775 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:04.820 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:18:04.818+0000 7ff0b0780600 1 bucket sync caught up with source: 2022-09-03T06:18:04.821 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.736.5, , , 00000000001.284.5, 00000000001.685.5, , , 00000000002.339.5, , 00000000002.422.5] 2022-09-03T06:18:04.821 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.736.5, , , 00000000001.284.5, 00000000001.685.5, , , 00000000002.339.5, , 00000000002.422.5] 2022-09-03T06:18:04.822 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:18:04.818+0000 7ff0b0780600 0 bucket checkpoint complete 2022-09-03T06:18:04.833 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-71 2022-09-03T06:18:04.834 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-71 2022-09-03T06:18:04.835 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:04.836 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:04.837 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-71 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:18:04.893 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:04.893 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:04.928 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:18:04.926+0000 7f4bede0e600 1 bucket sync caught up with source: 2022-09-03T06:18:04.928 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.343.5, , , 00000000001.219.5, 00000000001.244.5, , , 00000000002.246.5, , 00000000002.185.5] 2022-09-03T06:18:04.929 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.343.5, , , 00000000001.219.5, 00000000001.244.5, , , 00000000002.246.5, , 00000000002.185.5] 2022-09-03T06:18:04.929 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:18:04.926+0000 7f4bede0e600 0 bucket checkpoint complete 2022-09-03T06:18:04.942 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-71 2022-09-03T06:18:04.943 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:04.944 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:04.944 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-71 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:18:05.006 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:05.007 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:05.041 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:18:05.039+0000 7f9291c08600 1 bucket sync caught up with source: 2022-09-03T06:18:05.042 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000002.343.5, , , 00000000001.219.5, 00000000001.244.5, , , 00000000002.246.5, , 00000000002.185.5] 2022-09-03T06:18:05.042 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000002.343.5, , , 00000000001.219.5, 00000000001.244.5, , , 00000000002.246.5, , 00000000002.185.5] 2022-09-03T06:18:05.042 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:18:05.039+0000 7f9291c08600 0 bucket checkpoint complete 2022-09-03T06:18:05.058 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-71 2022-09-03T06:18:05.059 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-71 zones={test-zone2, test-zone1} 2022-09-03T06:18:05.061 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:05.063 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:05.065 DEBUG:boto:Method: HEAD 2022-09-03T06:18:05.066 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:18:05.067 DEBUG:boto:Data: 2022-09-03T06:18:05.068 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.069 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.070 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.070 DEBUG:boto:Params: {} 2022-09-03T06:18:05.074 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:18:05.075 DEBUG:boto:Token: None 2022-09-03T06:18:05.076 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/ 2022-09-03T06:18:05.077 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZrX3F8VgLReq4uXzHcLcyeplc18= 2022-09-03T06:18:05.078 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZrX3F8VgLReq4uXzHcLcyeplc18=', 'Content-Length': '0'} 2022-09-03T06:18:05.083 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002fa5ee0d2ec82aed-006312f19d-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.084 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:05.085 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:05.086 DEBUG:boto:Method: HEAD 2022-09-03T06:18:05.087 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:18:05.087 DEBUG:boto:Data: 2022-09-03T06:18:05.088 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.088 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.089 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.089 DEBUG:boto:Params: {} 2022-09-03T06:18:05.090 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:18:05.091 DEBUG:boto:Token: None 2022-09-03T06:18:05.092 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/ 2022-09-03T06:18:05.093 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZrX3F8VgLReq4uXzHcLcyeplc18= 2022-09-03T06:18:05.093 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZrX3F8VgLReq4uXzHcLcyeplc18=', 'Content-Length': '0'} 2022-09-03T06:18:05.098 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fb2860fc4351123f-006312f19d-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.099 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:18:05.100 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:05.100 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:05.101 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:05.101 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:05.102 DEBUG:boto:Method: GET 2022-09-03T06:18:05.103 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:05.104 DEBUG:boto:Data: 2022-09-03T06:18:05.104 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.105 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.106 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.108 DEBUG:boto:Params: {} 2022-09-03T06:18:05.108 DEBUG:boto:Token: None 2022-09-03T06:18:05.109 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/?versions 2022-09-03T06:18:05.110 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dqgnJtiT45xxIQwAXawZBWjbWdc= 2022-09-03T06:18:05.111 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dqgnJtiT45xxIQwAXawZBWjbWdc=', 'Content-Length': '0'} 2022-09-03T06:18:05.114 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000020d45c02a8788531-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.115 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:05.117 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:18:05.118 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:18:05.118 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:18:05.119 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:18:05.120 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:18:05.120 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:18:05.121 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:18:05.121 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:18:05.122 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:18:05.123 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:05.123 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:05.124 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:05.124 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:05.125 DEBUG:boto:Method: GET 2022-09-03T06:18:05.125 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:05.126 DEBUG:boto:Data: 2022-09-03T06:18:05.126 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.127 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.128 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.128 DEBUG:boto:Params: {} 2022-09-03T06:18:05.129 DEBUG:boto:Token: None 2022-09-03T06:18:05.130 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/?versions 2022-09-03T06:18:05.130 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dqgnJtiT45xxIQwAXawZBWjbWdc= 2022-09-03T06:18:05.131 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dqgnJtiT45xxIQwAXawZBWjbWdc=', 'Content-Length': '0'} 2022-09-03T06:18:05.134 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009417c1d877123587-006312f19d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.136 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:05.138 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:18:05.138 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:18:05.139 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:18:05.140 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:18:05.140 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:18:05.141 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:18:05.141 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:18:05.142 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:18:05.143 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:05.143 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:05.145 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:05.146 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:05.146 DEBUG:boto:Method: GET 2022-09-03T06:18:05.147 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:05.147 DEBUG:boto:Data: 2022-09-03T06:18:05.148 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.149 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.150 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.152 DEBUG:boto:Params: {} 2022-09-03T06:18:05.154 DEBUG:boto:Token: None 2022-09-03T06:18:05.155 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/?versions 2022-09-03T06:18:05.156 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dqgnJtiT45xxIQwAXawZBWjbWdc= 2022-09-03T06:18:05.157 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dqgnJtiT45xxIQwAXawZBWjbWdc=', 'Content-Length': '0'} 2022-09-03T06:18:05.159 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000a26691816e1f4ff-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.160 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:05.161 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:05.162 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:05.163 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:05.163 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:05.164 DEBUG:boto:Method: GET 2022-09-03T06:18:05.165 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:05.166 DEBUG:boto:Data: 2022-09-03T06:18:05.166 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.167 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.168 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.168 DEBUG:boto:Params: {} 2022-09-03T06:18:05.169 DEBUG:boto:Token: None 2022-09-03T06:18:05.170 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/?versions 2022-09-03T06:18:05.171 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dqgnJtiT45xxIQwAXawZBWjbWdc= 2022-09-03T06:18:05.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dqgnJtiT45xxIQwAXawZBWjbWdc=', 'Content-Length': '0'} 2022-09-03T06:18:05.174 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006547b9337a829384-006312f19d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.175 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:05.177 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:18:05.178 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:05.179 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:05.180 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.181 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.182 DEBUG:boto:Method: GET 2022-09-03T06:18:05.183 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.184 DEBUG:boto:Data: 2022-09-03T06:18:05.185 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.186 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.187 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.188 DEBUG:boto:Params: {} 2022-09-03T06:18:05.188 DEBUG:boto:Token: None 2022-09-03T06:18:05.189 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.190 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q4kOKzwUP17rgxXAwS8+7m0KLL4= 2022-09-03T06:18:05.191 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q4kOKzwUP17rgxXAwS8+7m0KLL4=', 'Content-Length': '0'} 2022-09-03T06:18:05.194 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ab49e07351387720-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.195 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:05.196 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:05.197 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.198 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.199 DEBUG:boto:Method: GET 2022-09-03T06:18:05.200 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.201 DEBUG:boto:Data: 2022-09-03T06:18:05.203 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.204 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.205 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.206 DEBUG:boto:Params: {} 2022-09-03T06:18:05.207 DEBUG:boto:Token: None 2022-09-03T06:18:05.208 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.209 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q4kOKzwUP17rgxXAwS8+7m0KLL4= 2022-09-03T06:18:05.209 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q4kOKzwUP17rgxXAwS8+7m0KLL4=', 'Content-Length': '0'} 2022-09-03T06:18:05.211 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a2b8869af63aede7-006312f19d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.254 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:05.255 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:05.255 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.256 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.257 DEBUG:boto:Method: HEAD 2022-09-03T06:18:05.257 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.258 DEBUG:boto:Data: 2022-09-03T06:18:05.259 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.260 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.261 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.262 DEBUG:boto:Params: {} 2022-09-03T06:18:05.264 DEBUG:boto:Token: None 2022-09-03T06:18:05.265 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.266 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Tx4QVhA2ylLuCIJ1qqJNPtm40L0= 2022-09-03T06:18:05.267 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Tx4QVhA2ylLuCIJ1qqJNPtm40L0=', 'Content-Length': '0'} 2022-09-03T06:18:05.270 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e301b811ffb42630-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.271 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:05.272 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:05.274 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.275 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.277 DEBUG:boto:Method: HEAD 2022-09-03T06:18:05.278 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.279 DEBUG:boto:Data: 2022-09-03T06:18:05.280 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.281 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.281 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.282 DEBUG:boto:Params: {} 2022-09-03T06:18:05.283 DEBUG:boto:Token: None 2022-09-03T06:18:05.284 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.285 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Tx4QVhA2ylLuCIJ1qqJNPtm40L0= 2022-09-03T06:18:05.286 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Tx4QVhA2ylLuCIJ1qqJNPtm40L0=', 'Content-Length': '0'} 2022-09-03T06:18:05.288 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009b58ba40cef784ad-006312f19d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.290 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:18:05.290 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:05.291 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:05.292 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.292 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.293 DEBUG:boto:Method: GET 2022-09-03T06:18:05.293 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.294 DEBUG:boto:Data: 2022-09-03T06:18:05.294 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.295 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.295 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.296 DEBUG:boto:Params: {} 2022-09-03T06:18:05.297 DEBUG:boto:Token: None 2022-09-03T06:18:05.297 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.298 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q4kOKzwUP17rgxXAwS8+7m0KLL4= 2022-09-03T06:18:05.298 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q4kOKzwUP17rgxXAwS8+7m0KLL4=', 'Content-Length': '0'} 2022-09-03T06:18:05.300 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e6ac824ef4a3123-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.302 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:05.303 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:05.304 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.305 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.306 DEBUG:boto:Method: GET 2022-09-03T06:18:05.307 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.308 DEBUG:boto:Data: 2022-09-03T06:18:05.309 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.310 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.311 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.312 DEBUG:boto:Params: {} 2022-09-03T06:18:05.313 DEBUG:boto:Token: None 2022-09-03T06:18:05.314 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:05.315 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q4kOKzwUP17rgxXAwS8+7m0KLL4= 2022-09-03T06:18:05.316 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q4kOKzwUP17rgxXAwS8+7m0KLL4=', 'Content-Length': '0'} 2022-09-03T06:18:05.319 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000003e0d84eadaef7d1-006312f19d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.320 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:05.321 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:05.321 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:05.322 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:05.323 DEBUG:boto:Method: GET 2022-09-03T06:18:05.323 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:05.324 DEBUG:boto:Data: 2022-09-03T06:18:05.324 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.325 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.326 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.327 DEBUG:boto:Params: {} 2022-09-03T06:18:05.328 DEBUG:boto:Token: None 2022-09-03T06:18:05.329 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/ 2022-09-03T06:18:05.331 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc= 2022-09-03T06:18:05.332 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc=', 'Content-Length': '0'} 2022-09-03T06:18:05.335 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d1c844379ecf8636-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.337 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:05.338 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:05.339 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:05.341 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:05.342 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:05.343 DEBUG:boto:Method: GET 2022-09-03T06:18:05.344 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:05.345 DEBUG:boto:Data: 2022-09-03T06:18:05.346 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.348 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.350 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.351 DEBUG:boto:Params: {} 2022-09-03T06:18:05.352 DEBUG:boto:Token: None 2022-09-03T06:18:05.353 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/ 2022-09-03T06:18:05.354 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc= 2022-09-03T06:18:05.355 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc=', 'Content-Length': '0'} 2022-09-03T06:18:05.358 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e13988098b41e867-006312f19d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.359 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:05.360 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:05.362 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:05.363 DEBUG:boto:Method: HEAD 2022-09-03T06:18:05.363 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:05.364 DEBUG:boto:Data: 2022-09-03T06:18:05.365 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.366 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.367 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.368 DEBUG:boto:Params: {} 2022-09-03T06:18:05.369 DEBUG:boto:Token: None 2022-09-03T06:18:05.370 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/a 2022-09-03T06:18:05.371 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y8KtdsIP8YGqacSL0BhlICj/yGE= 2022-09-03T06:18:05.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y8KtdsIP8YGqacSL0BhlICj/yGE=', 'Content-Length': '0'} 2022-09-03T06:18:05.375 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002cd9a068d3fecaa7-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.377 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:05.378 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:05.379 DEBUG:boto:Method: HEAD 2022-09-03T06:18:05.380 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:05.381 DEBUG:boto:Data: 2022-09-03T06:18:05.382 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.383 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.384 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.385 DEBUG:boto:Params: {} 2022-09-03T06:18:05.386 DEBUG:boto:Token: None 2022-09-03T06:18:05.387 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/a 2022-09-03T06:18:05.388 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y8KtdsIP8YGqacSL0BhlICj/yGE= 2022-09-03T06:18:05.389 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y8KtdsIP8YGqacSL0BhlICj/yGE=', 'Content-Length': '0'} 2022-09-03T06:18:05.391 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dbc15d8bf8c7205e-006312f19d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.393 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:18:05.394 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:05.395 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:05.396 DEBUG:boto:Method: GET 2022-09-03T06:18:05.396 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:05.397 DEBUG:boto:Data: 2022-09-03T06:18:05.398 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.398 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.399 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.400 DEBUG:boto:Params: {} 2022-09-03T06:18:05.400 DEBUG:boto:Token: None 2022-09-03T06:18:05.401 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/a 2022-09-03T06:18:05.402 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/dDPtAJM2tQeCeV9AQcojClCRlI= 2022-09-03T06:18:05.403 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/dDPtAJM2tQeCeV9AQcojClCRlI=', 'Content-Length': '0'} 2022-09-03T06:18:05.406 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cd46c7c00eb0608c-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.448 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:05.450 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:05.451 DEBUG:boto:Method: GET 2022-09-03T06:18:05.451 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:05.452 DEBUG:boto:Data: 2022-09-03T06:18:05.453 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.454 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.454 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.455 DEBUG:boto:Params: {} 2022-09-03T06:18:05.456 DEBUG:boto:Token: None 2022-09-03T06:18:05.457 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/a 2022-09-03T06:18:05.457 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/dDPtAJM2tQeCeV9AQcojClCRlI= 2022-09-03T06:18:05.459 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/dDPtAJM2tQeCeV9AQcojClCRlI=', 'Content-Length': '0'} 2022-09-03T06:18:05.461 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ed64fa18cb2edbde-006312f19d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.462 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:05.462 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:05.463 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:05.463 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:05.464 DEBUG:boto:Method: GET 2022-09-03T06:18:05.465 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:05.465 DEBUG:boto:Data: 2022-09-03T06:18:05.466 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.466 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.467 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.468 DEBUG:boto:Params: {} 2022-09-03T06:18:05.468 DEBUG:boto:Token: None 2022-09-03T06:18:05.470 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/ 2022-09-03T06:18:05.471 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc= 2022-09-03T06:18:05.472 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc=', 'Content-Length': '0'} 2022-09-03T06:18:05.474 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007381b4f1c317e2ad-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.475 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:05.476 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:05.477 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:05.477 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:05.478 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:05.479 DEBUG:boto:Method: GET 2022-09-03T06:18:05.480 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:05.481 DEBUG:boto:Data: 2022-09-03T06:18:05.482 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.483 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.485 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.486 DEBUG:boto:Params: {} 2022-09-03T06:18:05.487 DEBUG:boto:Token: None 2022-09-03T06:18:05.488 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/ 2022-09-03T06:18:05.489 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc= 2022-09-03T06:18:05.490 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc=', 'Content-Length': '0'} 2022-09-03T06:18:05.493 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003633bde5a5220fbb-006312f19d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.494 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:05.496 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:18:05.498 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:05.499 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:05.499 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.500 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.502 DEBUG:boto:Method: GET 2022-09-03T06:18:05.503 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.504 DEBUG:boto:Data: 2022-09-03T06:18:05.505 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.506 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.507 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.508 DEBUG:boto:Params: {} 2022-09-03T06:18:05.509 DEBUG:boto:Token: None 2022-09-03T06:18:05.510 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.511 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nllPEajQmTYkXGjgBrLftUy4UwQ= 2022-09-03T06:18:05.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nllPEajQmTYkXGjgBrLftUy4UwQ=', 'Content-Length': '0'} 2022-09-03T06:18:05.515 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007ee29e9524aa221c-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.517 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:05.518 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:05.518 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.519 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.520 DEBUG:boto:Method: GET 2022-09-03T06:18:05.520 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.521 DEBUG:boto:Data: 2022-09-03T06:18:05.522 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.523 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.524 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.525 DEBUG:boto:Params: {} 2022-09-03T06:18:05.527 DEBUG:boto:Token: None 2022-09-03T06:18:05.528 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.529 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nllPEajQmTYkXGjgBrLftUy4UwQ= 2022-09-03T06:18:05.530 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nllPEajQmTYkXGjgBrLftUy4UwQ=', 'Content-Length': '0'} 2022-09-03T06:18:05.533 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e25e7ff9cae772b-006312f19d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.535 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:05.536 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:05.537 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.538 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.539 DEBUG:boto:Method: HEAD 2022-09-03T06:18:05.540 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.540 DEBUG:boto:Data: 2022-09-03T06:18:05.541 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.541 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.542 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.542 DEBUG:boto:Params: {} 2022-09-03T06:18:05.543 DEBUG:boto:Token: None 2022-09-03T06:18:05.544 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.545 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y7FqCqrmvaV8/aNBeHsl4KIhdFM= 2022-09-03T06:18:05.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y7FqCqrmvaV8/aNBeHsl4KIhdFM=', 'Content-Length': '0'} 2022-09-03T06:18:05.549 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000055a3adac03309628-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.550 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:05.551 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:05.551 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.552 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.552 DEBUG:boto:Method: HEAD 2022-09-03T06:18:05.553 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.553 DEBUG:boto:Data: 2022-09-03T06:18:05.554 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.554 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.555 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.555 DEBUG:boto:Params: {} 2022-09-03T06:18:05.556 DEBUG:boto:Token: None 2022-09-03T06:18:05.556 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.557 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y7FqCqrmvaV8/aNBeHsl4KIhdFM= 2022-09-03T06:18:05.558 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y7FqCqrmvaV8/aNBeHsl4KIhdFM=', 'Content-Length': '0'} 2022-09-03T06:18:05.560 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004b6104446f3b9d37-006312f19d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.561 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:18:05.561 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:05.562 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:05.562 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.563 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.564 DEBUG:boto:Method: GET 2022-09-03T06:18:05.564 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.565 DEBUG:boto:Data: 2022-09-03T06:18:05.565 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.566 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.567 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.567 DEBUG:boto:Params: {} 2022-09-03T06:18:05.568 DEBUG:boto:Token: None 2022-09-03T06:18:05.568 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.569 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nllPEajQmTYkXGjgBrLftUy4UwQ= 2022-09-03T06:18:05.570 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nllPEajQmTYkXGjgBrLftUy4UwQ=', 'Content-Length': '0'} 2022-09-03T06:18:05.572 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002b0808ab8cc2f382-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.573 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:05.573 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:05.574 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.575 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.575 DEBUG:boto:Method: GET 2022-09-03T06:18:05.576 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.577 DEBUG:boto:Data: 2022-09-03T06:18:05.577 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.578 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.578 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.579 DEBUG:boto:Params: {} 2022-09-03T06:18:05.580 DEBUG:boto:Token: None 2022-09-03T06:18:05.580 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:05.581 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nllPEajQmTYkXGjgBrLftUy4UwQ= 2022-09-03T06:18:05.582 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nllPEajQmTYkXGjgBrLftUy4UwQ=', 'Content-Length': '0'} 2022-09-03T06:18:05.584 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d11e33f3e7544785-006312f19d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.585 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:05.585 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:05.586 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:05.586 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:05.587 DEBUG:boto:Method: GET 2022-09-03T06:18:05.587 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:05.588 DEBUG:boto:Data: 2022-09-03T06:18:05.588 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.589 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.589 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.591 DEBUG:boto:Params: {} 2022-09-03T06:18:05.591 DEBUG:boto:Token: None 2022-09-03T06:18:05.592 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/ 2022-09-03T06:18:05.593 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc= 2022-09-03T06:18:05.593 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc=', 'Content-Length': '0'} 2022-09-03T06:18:05.596 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000063276cbf560cda28-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.597 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:05.598 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:05.599 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:05.600 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:05.601 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:05.602 DEBUG:boto:Method: GET 2022-09-03T06:18:05.603 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:05.604 DEBUG:boto:Data: 2022-09-03T06:18:05.607 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.608 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.609 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.610 DEBUG:boto:Params: {} 2022-09-03T06:18:05.611 DEBUG:boto:Token: None 2022-09-03T06:18:05.611 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/ 2022-09-03T06:18:05.612 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc= 2022-09-03T06:18:05.613 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc=', 'Content-Length': '0'} 2022-09-03T06:18:05.616 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f4ab14ce451d7c4b-006312f19d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.617 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:05.618 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:05.619 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:05.619 DEBUG:boto:Method: HEAD 2022-09-03T06:18:05.620 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:05.620 DEBUG:boto:Data: 2022-09-03T06:18:05.621 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.621 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.622 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.623 DEBUG:boto:Params: {} 2022-09-03T06:18:05.623 DEBUG:boto:Token: None 2022-09-03T06:18:05.624 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/b 2022-09-03T06:18:05.625 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MuDocc6F45pwm/IZM3vsq4OToDs= 2022-09-03T06:18:05.625 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MuDocc6F45pwm/IZM3vsq4OToDs=', 'Content-Length': '0'} 2022-09-03T06:18:05.627 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002fea7d8ed8092cc8-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.629 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:05.630 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:05.631 DEBUG:boto:Method: HEAD 2022-09-03T06:18:05.632 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:05.633 DEBUG:boto:Data: 2022-09-03T06:18:05.634 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.636 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.637 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.638 DEBUG:boto:Params: {} 2022-09-03T06:18:05.640 DEBUG:boto:Token: None 2022-09-03T06:18:05.641 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/b 2022-09-03T06:18:05.642 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MuDocc6F45pwm/IZM3vsq4OToDs= 2022-09-03T06:18:05.643 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MuDocc6F45pwm/IZM3vsq4OToDs=', 'Content-Length': '0'} 2022-09-03T06:18:05.647 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000090d369870c1ef61e-006312f19d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.648 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:18:05.649 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:05.649 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:05.650 DEBUG:boto:Method: GET 2022-09-03T06:18:05.651 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:05.651 DEBUG:boto:Data: 2022-09-03T06:18:05.652 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.652 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.653 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.654 DEBUG:boto:Params: {} 2022-09-03T06:18:05.654 DEBUG:boto:Token: None 2022-09-03T06:18:05.655 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/b 2022-09-03T06:18:05.656 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hy8PmBVWsii+rYUbl/l4NfhYavw= 2022-09-03T06:18:05.657 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hy8PmBVWsii+rYUbl/l4NfhYavw=', 'Content-Length': '0'} 2022-09-03T06:18:05.659 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000203f63d7426a8585-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.660 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:05.661 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:05.661 DEBUG:boto:Method: GET 2022-09-03T06:18:05.662 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:05.662 DEBUG:boto:Data: 2022-09-03T06:18:05.663 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.663 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.664 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.664 DEBUG:boto:Params: {} 2022-09-03T06:18:05.665 DEBUG:boto:Token: None 2022-09-03T06:18:05.667 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/b 2022-09-03T06:18:05.668 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hy8PmBVWsii+rYUbl/l4NfhYavw= 2022-09-03T06:18:05.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hy8PmBVWsii+rYUbl/l4NfhYavw=', 'Content-Length': '0'} 2022-09-03T06:18:05.672 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001af9fd37b8d73782-006312f19d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.674 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:05.674 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:05.675 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:05.676 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:05.677 DEBUG:boto:Method: GET 2022-09-03T06:18:05.678 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:05.679 DEBUG:boto:Data: 2022-09-03T06:18:05.680 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.681 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.682 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.682 DEBUG:boto:Params: {} 2022-09-03T06:18:05.683 DEBUG:boto:Token: None 2022-09-03T06:18:05.684 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/ 2022-09-03T06:18:05.686 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc= 2022-09-03T06:18:05.687 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc=', 'Content-Length': '0'} 2022-09-03T06:18:05.690 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002f9c437db9b56ee8-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.691 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:05.693 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:05.693 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:05.694 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:05.696 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:05.697 DEBUG:boto:Method: GET 2022-09-03T06:18:05.698 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:05.700 DEBUG:boto:Data: 2022-09-03T06:18:05.701 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.701 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.702 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.702 DEBUG:boto:Params: {} 2022-09-03T06:18:05.703 DEBUG:boto:Token: None 2022-09-03T06:18:05.704 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/ 2022-09-03T06:18:05.714 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc= 2022-09-03T06:18:05.714 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc=', 'Content-Length': '0'} 2022-09-03T06:18:05.717 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009b55f62363cfbba3-006312f19d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.718 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:05.719 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:18:05.719 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:05.720 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:05.720 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.721 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.721 DEBUG:boto:Method: GET 2022-09-03T06:18:05.724 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.729 DEBUG:boto:Data: 2022-09-03T06:18:05.730 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.731 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.732 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.733 DEBUG:boto:Params: {} 2022-09-03T06:18:05.735 DEBUG:boto:Token: None 2022-09-03T06:18:05.739 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.741 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JXFs5zb62zMioxYdITwyVF2FRVY= 2022-09-03T06:18:05.742 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JXFs5zb62zMioxYdITwyVF2FRVY=', 'Content-Length': '0'} 2022-09-03T06:18:05.745 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a6e7a569489d43bd-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.746 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:05.748 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:05.751 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.752 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.753 DEBUG:boto:Method: GET 2022-09-03T06:18:05.753 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.754 DEBUG:boto:Data: 2022-09-03T06:18:05.755 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.756 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.757 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.758 DEBUG:boto:Params: {} 2022-09-03T06:18:05.759 DEBUG:boto:Token: None 2022-09-03T06:18:05.762 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.763 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JXFs5zb62zMioxYdITwyVF2FRVY= 2022-09-03T06:18:05.763 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JXFs5zb62zMioxYdITwyVF2FRVY=', 'Content-Length': '0'} 2022-09-03T06:18:05.765 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074bda9c2ccb39b5b-006312f19d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.766 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:05.767 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:05.767 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.768 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.768 DEBUG:boto:Method: HEAD 2022-09-03T06:18:05.769 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.770 DEBUG:boto:Data: 2022-09-03T06:18:05.771 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.772 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.772 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.773 DEBUG:boto:Params: {} 2022-09-03T06:18:05.774 DEBUG:boto:Token: None 2022-09-03T06:18:05.774 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.775 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O+2DAT7wcM6F+uc3PU+OKH77wxs= 2022-09-03T06:18:05.775 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O+2DAT7wcM6F+uc3PU+OKH77wxs=', 'Content-Length': '0'} 2022-09-03T06:18:05.778 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a15d0411bdfad443-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.779 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:05.780 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:05.781 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.782 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.783 DEBUG:boto:Method: HEAD 2022-09-03T06:18:05.783 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.784 DEBUG:boto:Data: 2022-09-03T06:18:05.784 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.785 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.785 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.786 DEBUG:boto:Params: {} 2022-09-03T06:18:05.786 DEBUG:boto:Token: None 2022-09-03T06:18:05.787 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.788 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O+2DAT7wcM6F+uc3PU+OKH77wxs= 2022-09-03T06:18:05.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O+2DAT7wcM6F+uc3PU+OKH77wxs=', 'Content-Length': '0'} 2022-09-03T06:18:05.792 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005255d03ab65d7b75-006312f19d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.793 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:18:05.796 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:05.797 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:05.798 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.798 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.799 DEBUG:boto:Method: GET 2022-09-03T06:18:05.799 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.800 DEBUG:boto:Data: 2022-09-03T06:18:05.801 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.802 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.803 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.804 DEBUG:boto:Params: {} 2022-09-03T06:18:05.805 DEBUG:boto:Token: None 2022-09-03T06:18:05.806 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.807 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JXFs5zb62zMioxYdITwyVF2FRVY= 2022-09-03T06:18:05.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JXFs5zb62zMioxYdITwyVF2FRVY=', 'Content-Length': '0'} 2022-09-03T06:18:05.810 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ec2a44b46563fe0e-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.811 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:05.812 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:05.813 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.814 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.814 DEBUG:boto:Method: GET 2022-09-03T06:18:05.815 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.816 DEBUG:boto:Data: 2022-09-03T06:18:05.816 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.817 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.817 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.819 DEBUG:boto:Params: {} 2022-09-03T06:18:05.820 DEBUG:boto:Token: None 2022-09-03T06:18:05.820 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:05.821 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JXFs5zb62zMioxYdITwyVF2FRVY= 2022-09-03T06:18:05.822 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JXFs5zb62zMioxYdITwyVF2FRVY=', 'Content-Length': '0'} 2022-09-03T06:18:05.824 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b166c0415c17da97-006312f19d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.826 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:05.827 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:05.827 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:05.828 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:05.829 DEBUG:boto:Method: GET 2022-09-03T06:18:05.829 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:05.830 DEBUG:boto:Data: 2022-09-03T06:18:05.831 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.831 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.832 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.833 DEBUG:boto:Params: {} 2022-09-03T06:18:05.833 DEBUG:boto:Token: None 2022-09-03T06:18:05.834 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/ 2022-09-03T06:18:05.835 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc= 2022-09-03T06:18:05.835 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc=', 'Content-Length': '0'} 2022-09-03T06:18:05.839 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cb11aebe3f8b1db4-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.882 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:05.884 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:05.885 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:05.886 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:05.887 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:05.887 DEBUG:boto:Method: GET 2022-09-03T06:18:05.888 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:05.890 DEBUG:boto:Data: 2022-09-03T06:18:05.890 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.891 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.892 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.892 DEBUG:boto:Params: {} 2022-09-03T06:18:05.893 DEBUG:boto:Token: None 2022-09-03T06:18:05.894 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/ 2022-09-03T06:18:05.895 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc= 2022-09-03T06:18:05.895 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc=', 'Content-Length': '0'} 2022-09-03T06:18:05.898 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000df0e1ad3b1d1b291-006312f19d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.900 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:05.903 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:05.904 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:05.904 DEBUG:boto:Method: HEAD 2022-09-03T06:18:05.905 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:05.905 DEBUG:boto:Data: 2022-09-03T06:18:05.906 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.906 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.907 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.907 DEBUG:boto:Params: {} 2022-09-03T06:18:05.908 DEBUG:boto:Token: None 2022-09-03T06:18:05.909 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/c 2022-09-03T06:18:05.909 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gTYaNDnkV05H4zoyCI+Sct348mQ= 2022-09-03T06:18:05.910 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gTYaNDnkV05H4zoyCI+Sct348mQ=', 'Content-Length': '0'} 2022-09-03T06:18:05.913 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000abf60c7d9d86925a-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.914 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:05.915 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:05.916 DEBUG:boto:Method: HEAD 2022-09-03T06:18:05.917 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:05.917 DEBUG:boto:Data: 2022-09-03T06:18:05.918 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.918 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.919 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.919 DEBUG:boto:Params: {} 2022-09-03T06:18:05.920 DEBUG:boto:Token: None 2022-09-03T06:18:05.921 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/c 2022-09-03T06:18:05.921 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gTYaNDnkV05H4zoyCI+Sct348mQ= 2022-09-03T06:18:05.923 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gTYaNDnkV05H4zoyCI+Sct348mQ=', 'Content-Length': '0'} 2022-09-03T06:18:05.925 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc8eb54948087678-006312f19d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.926 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:18:05.926 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:05.927 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:05.927 DEBUG:boto:Method: GET 2022-09-03T06:18:05.928 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:05.928 DEBUG:boto:Data: 2022-09-03T06:18:05.929 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.930 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.930 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.931 DEBUG:boto:Params: {} 2022-09-03T06:18:05.931 DEBUG:boto:Token: None 2022-09-03T06:18:05.933 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/c 2022-09-03T06:18:05.934 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J8OoDt/U8fg8W57jrHNA7ngG/64= 2022-09-03T06:18:05.935 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J8OoDt/U8fg8W57jrHNA7ngG/64=', 'Content-Length': '0'} 2022-09-03T06:18:05.938 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006c841fb47a6c3f3a-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.939 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:05.939 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:05.940 DEBUG:boto:Method: GET 2022-09-03T06:18:05.940 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:05.941 DEBUG:boto:Data: 2022-09-03T06:18:05.942 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.943 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.943 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.944 DEBUG:boto:Params: {} 2022-09-03T06:18:05.945 DEBUG:boto:Token: None 2022-09-03T06:18:05.945 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/c 2022-09-03T06:18:05.947 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J8OoDt/U8fg8W57jrHNA7ngG/64= 2022-09-03T06:18:05.947 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J8OoDt/U8fg8W57jrHNA7ngG/64=', 'Content-Length': '0'} 2022-09-03T06:18:05.950 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003fff4b3ba34c3849-006312f19d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.951 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:05.952 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:05.953 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:05.953 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:05.954 DEBUG:boto:Method: GET 2022-09-03T06:18:05.954 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:05.955 DEBUG:boto:Data: 2022-09-03T06:18:05.955 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.956 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.957 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.957 DEBUG:boto:Params: {} 2022-09-03T06:18:05.958 DEBUG:boto:Token: None 2022-09-03T06:18:05.959 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/ 2022-09-03T06:18:05.960 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc= 2022-09-03T06:18:05.960 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc=', 'Content-Length': '0'} 2022-09-03T06:18:05.963 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f0951d56d921c90f-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.964 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:05.965 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:05.966 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:05.967 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:05.967 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:05.968 DEBUG:boto:Method: GET 2022-09-03T06:18:05.969 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:05.969 DEBUG:boto:Data: 2022-09-03T06:18:05.970 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.970 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:05.971 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.971 DEBUG:boto:Params: {} 2022-09-03T06:18:05.972 DEBUG:boto:Token: None 2022-09-03T06:18:05.973 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/ 2022-09-03T06:18:05.974 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc= 2022-09-03T06:18:05.974 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ezl4AFxuUslDXEtxPle4re1/rfc=', 'Content-Length': '0'} 2022-09-03T06:18:05.977 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002200b0cbaeabe37f-006312f19d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:05.978 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:05.980 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:18:05.981 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:05.982 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:05.982 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:05.983 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:05.984 DEBUG:boto:Method: GET 2022-09-03T06:18:05.985 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:05.986 DEBUG:boto:Data: 2022-09-03T06:18:05.988 DEBUG:boto:Headers: {} 2022-09-03T06:18:05.989 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:05.990 DEBUG:boto:Port: 8000 2022-09-03T06:18:05.991 DEBUG:boto:Params: {} 2022-09-03T06:18:05.992 DEBUG:boto:Token: None 2022-09-03T06:18:05.993 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:05 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:05.995 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yji6cDiu9S82E/w9FRUqK+l70Lw= 2022-09-03T06:18:05.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yji6cDiu9S82E/w9FRUqK+l70Lw=', 'Content-Length': '0'} 2022-09-03T06:18:05.999 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c107d947d077bba7-006312f19d-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.001 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:06.002 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:06.003 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:06.004 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:06.005 DEBUG:boto:Method: GET 2022-09-03T06:18:06.006 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:06.007 DEBUG:boto:Data: 2022-09-03T06:18:06.008 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.009 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:06.010 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.011 DEBUG:boto:Params: {} 2022-09-03T06:18:06.012 DEBUG:boto:Token: None 2022-09-03T06:18:06.013 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:06.014 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0ozvfCS0CiTR2Ca6lmzDwugEiho= 2022-09-03T06:18:06.015 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0ozvfCS0CiTR2Ca6lmzDwugEiho=', 'Content-Length': '0'} 2022-09-03T06:18:06.020 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000226ad2bd9cb889e0-006312f19e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.021 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:06.021 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:06.022 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:06.022 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:06.023 DEBUG:boto:Method: HEAD 2022-09-03T06:18:06.023 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:06.024 DEBUG:boto:Data: 2022-09-03T06:18:06.024 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.025 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:06.025 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.026 DEBUG:boto:Params: {} 2022-09-03T06:18:06.027 DEBUG:boto:Token: None 2022-09-03T06:18:06.028 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:06.028 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WyMZpUMv36SuWxbfPV+cGONIubM= 2022-09-03T06:18:06.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WyMZpUMv36SuWxbfPV+cGONIubM=', 'Content-Length': '0'} 2022-09-03T06:18:06.031 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000015c2a86be6744b30-006312f19e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.032 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:06.033 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:06.033 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:06.034 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:06.034 DEBUG:boto:Method: HEAD 2022-09-03T06:18:06.035 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:06.035 DEBUG:boto:Data: 2022-09-03T06:18:06.036 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.036 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:06.037 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.037 DEBUG:boto:Params: {} 2022-09-03T06:18:06.038 DEBUG:boto:Token: None 2022-09-03T06:18:06.039 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:06.039 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WyMZpUMv36SuWxbfPV+cGONIubM= 2022-09-03T06:18:06.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WyMZpUMv36SuWxbfPV+cGONIubM=', 'Content-Length': '0'} 2022-09-03T06:18:06.042 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004f66a16d78470a31-006312f19e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.043 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:18:06.044 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:06.044 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:06.045 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:06.045 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:06.046 DEBUG:boto:Method: GET 2022-09-03T06:18:06.047 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:06.047 DEBUG:boto:Data: 2022-09-03T06:18:06.048 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.049 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:06.049 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.050 DEBUG:boto:Params: {} 2022-09-03T06:18:06.050 DEBUG:boto:Token: None 2022-09-03T06:18:06.051 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:06.052 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0ozvfCS0CiTR2Ca6lmzDwugEiho= 2022-09-03T06:18:06.053 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0ozvfCS0CiTR2Ca6lmzDwugEiho=', 'Content-Length': '0'} 2022-09-03T06:18:06.055 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000905dd7f9c0a9042-006312f19e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.103 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:06.104 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:06.105 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:06.106 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:06.107 DEBUG:boto:Method: GET 2022-09-03T06:18:06.108 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:06.109 DEBUG:boto:Data: 2022-09-03T06:18:06.110 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.112 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:06.113 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.114 DEBUG:boto:Params: {} 2022-09-03T06:18:06.115 DEBUG:boto:Token: None 2022-09-03T06:18:06.116 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:06.117 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0ozvfCS0CiTR2Ca6lmzDwugEiho= 2022-09-03T06:18:06.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0ozvfCS0CiTR2Ca6lmzDwugEiho=', 'Content-Length': '0'} 2022-09-03T06:18:06.130 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e5c3b9139256fd2-006312f19e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.134 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:06.135 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:06.136 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:06.137 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:06.152 DEBUG:boto:Method: GET 2022-09-03T06:18:06.154 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:06.158 DEBUG:boto:Data: 2022-09-03T06:18:06.159 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.160 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:06.161 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.163 DEBUG:boto:Params: {} 2022-09-03T06:18:06.170 DEBUG:boto:Token: None 2022-09-03T06:18:06.172 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/ 2022-09-03T06:18:06.174 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:synIQcbH4yY55nOkJ/X7LqyU/RQ= 2022-09-03T06:18:06.175 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:synIQcbH4yY55nOkJ/X7LqyU/RQ=', 'Content-Length': '0'} 2022-09-03T06:18:06.181 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fb5dcd83a94bc821-006312f19e-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.183 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:06.184 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:06.185 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:06.188 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:06.189 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:06.190 DEBUG:boto:Method: GET 2022-09-03T06:18:06.193 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:06.193 DEBUG:boto:Data: 2022-09-03T06:18:06.194 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.196 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:06.197 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.197 DEBUG:boto:Params: {} 2022-09-03T06:18:06.198 DEBUG:boto:Token: None 2022-09-03T06:18:06.199 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/ 2022-09-03T06:18:06.199 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:synIQcbH4yY55nOkJ/X7LqyU/RQ= 2022-09-03T06:18:06.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:synIQcbH4yY55nOkJ/X7LqyU/RQ=', 'Content-Length': '0'} 2022-09-03T06:18:06.204 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006e2df92adfccc0b0-006312f19e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.205 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:06.206 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:06.207 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:06.207 DEBUG:boto:Method: HEAD 2022-09-03T06:18:06.208 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:06.209 DEBUG:boto:Data: 2022-09-03T06:18:06.210 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.210 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:06.211 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.212 DEBUG:boto:Params: {} 2022-09-03T06:18:06.212 DEBUG:boto:Token: None 2022-09-03T06:18:06.214 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/d 2022-09-03T06:18:06.214 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:51+wSb7DKOUvlk7G00jPECdauts= 2022-09-03T06:18:06.215 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:51+wSb7DKOUvlk7G00jPECdauts=', 'Content-Length': '0'} 2022-09-03T06:18:06.224 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d857552539f8e77-006312f19e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.230 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:06.232 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:06.233 DEBUG:boto:Method: HEAD 2022-09-03T06:18:06.236 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:06.243 DEBUG:boto:Data: 2022-09-03T06:18:06.244 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.245 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:06.254 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.255 DEBUG:boto:Params: {} 2022-09-03T06:18:06.257 DEBUG:boto:Token: None 2022-09-03T06:18:06.259 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/d 2022-09-03T06:18:06.260 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:51+wSb7DKOUvlk7G00jPECdauts= 2022-09-03T06:18:06.262 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:51+wSb7DKOUvlk7G00jPECdauts=', 'Content-Length': '0'} 2022-09-03T06:18:06.269 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000211bba4ac25aed2-006312f19e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.270 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:18:06.271 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:06.272 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:06.273 DEBUG:boto:Method: GET 2022-09-03T06:18:06.276 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:06.278 DEBUG:boto:Data: 2022-09-03T06:18:06.280 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.280 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:06.281 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.281 DEBUG:boto:Params: {} 2022-09-03T06:18:06.282 DEBUG:boto:Token: None 2022-09-03T06:18:06.283 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/d 2022-09-03T06:18:06.283 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ciBZZMM5csvIKDMfzoXRxrfG6ys= 2022-09-03T06:18:06.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ciBZZMM5csvIKDMfzoXRxrfG6ys=', 'Content-Length': '0'} 2022-09-03T06:18:06.293 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000215a7bd2f18776b-006312f19e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.295 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:06.295 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:06.296 DEBUG:boto:Method: GET 2022-09-03T06:18:06.297 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:06.297 DEBUG:boto:Data: 2022-09-03T06:18:06.298 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.298 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:06.299 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.300 DEBUG:boto:Params: {} 2022-09-03T06:18:06.300 DEBUG:boto:Token: None 2022-09-03T06:18:06.301 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/d 2022-09-03T06:18:06.301 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ciBZZMM5csvIKDMfzoXRxrfG6ys= 2022-09-03T06:18:06.303 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ciBZZMM5csvIKDMfzoXRxrfG6ys=', 'Content-Length': '0'} 2022-09-03T06:18:06.311 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000727e75bbcecefa07-006312f19e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.313 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:06.314 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:06.315 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:06.316 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:06.317 DEBUG:boto:Method: GET 2022-09-03T06:18:06.329 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:06.330 DEBUG:boto:Data: 2022-09-03T06:18:06.331 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.332 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:06.333 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.334 DEBUG:boto:Params: {} 2022-09-03T06:18:06.335 DEBUG:boto:Token: None 2022-09-03T06:18:06.336 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/ 2022-09-03T06:18:06.337 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:synIQcbH4yY55nOkJ/X7LqyU/RQ= 2022-09-03T06:18:06.339 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:synIQcbH4yY55nOkJ/X7LqyU/RQ=', 'Content-Length': '0'} 2022-09-03T06:18:06.350 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000032d6dc5b24defabf-006312f19e-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.352 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:06.356 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:06.357 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:06.359 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:06.360 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:06.361 DEBUG:boto:Method: GET 2022-09-03T06:18:06.362 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:06.363 DEBUG:boto:Data: 2022-09-03T06:18:06.365 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.367 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:06.368 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.369 DEBUG:boto:Params: {} 2022-09-03T06:18:06.370 DEBUG:boto:Token: None 2022-09-03T06:18:06.371 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/ 2022-09-03T06:18:06.372 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:synIQcbH4yY55nOkJ/X7LqyU/RQ= 2022-09-03T06:18:06.374 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:synIQcbH4yY55nOkJ/X7LqyU/RQ=', 'Content-Length': '0'} 2022-09-03T06:18:06.377 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000025a2cd25300aec3-006312f19e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.378 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:06.380 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:18:06.381 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:06.384 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:06.385 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.385 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.386 DEBUG:boto:Method: GET 2022-09-03T06:18:06.386 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.388 DEBUG:boto:Data: 2022-09-03T06:18:06.388 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.389 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:06.389 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.390 DEBUG:boto:Params: {} 2022-09-03T06:18:06.390 DEBUG:boto:Token: None 2022-09-03T06:18:06.391 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.391 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hrk70Bl5fd7n1FDbQsVkYXBhDko= 2022-09-03T06:18:06.393 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hrk70Bl5fd7n1FDbQsVkYXBhDko=', 'Content-Length': '0'} 2022-09-03T06:18:06.395 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ddd47746587701e8-006312f19e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.396 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:06.396 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:06.397 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.398 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.399 DEBUG:boto:Method: GET 2022-09-03T06:18:06.411 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.412 DEBUG:boto:Data: 2022-09-03T06:18:06.413 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.414 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:06.415 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.416 DEBUG:boto:Params: {} 2022-09-03T06:18:06.417 DEBUG:boto:Token: None 2022-09-03T06:18:06.418 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.419 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hrk70Bl5fd7n1FDbQsVkYXBhDko= 2022-09-03T06:18:06.421 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hrk70Bl5fd7n1FDbQsVkYXBhDko=', 'Content-Length': '0'} 2022-09-03T06:18:06.429 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cb46e6f58ab1c476-006312f19e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.431 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:06.431 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:06.434 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.436 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.437 DEBUG:boto:Method: HEAD 2022-09-03T06:18:06.438 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.439 DEBUG:boto:Data: 2022-09-03T06:18:06.440 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.441 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:06.443 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.444 DEBUG:boto:Params: {} 2022-09-03T06:18:06.446 DEBUG:boto:Token: None 2022-09-03T06:18:06.449 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.451 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/SGfwAkO2u+WEXhbNPbotHpRsA8= 2022-09-03T06:18:06.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/SGfwAkO2u+WEXhbNPbotHpRsA8=', 'Content-Length': '0'} 2022-09-03T06:18:06.459 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000054c4e0a63fdf812d-006312f19e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.460 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:06.461 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:06.461 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.467 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.468 DEBUG:boto:Method: HEAD 2022-09-03T06:18:06.468 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.469 DEBUG:boto:Data: 2022-09-03T06:18:06.470 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.476 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:06.476 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.477 DEBUG:boto:Params: {} 2022-09-03T06:18:06.477 DEBUG:boto:Token: None 2022-09-03T06:18:06.478 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.479 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/SGfwAkO2u+WEXhbNPbotHpRsA8= 2022-09-03T06:18:06.479 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/SGfwAkO2u+WEXhbNPbotHpRsA8=', 'Content-Length': '0'} 2022-09-03T06:18:06.482 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002b5d529f4a41eebc-006312f19e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.483 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:18:06.483 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:06.484 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:06.485 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.485 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.486 DEBUG:boto:Method: GET 2022-09-03T06:18:06.486 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.487 DEBUG:boto:Data: 2022-09-03T06:18:06.488 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.488 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:06.489 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.489 DEBUG:boto:Params: {} 2022-09-03T06:18:06.490 DEBUG:boto:Token: None 2022-09-03T06:18:06.491 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.491 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hrk70Bl5fd7n1FDbQsVkYXBhDko= 2022-09-03T06:18:06.493 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hrk70Bl5fd7n1FDbQsVkYXBhDko=', 'Content-Length': '0'} 2022-09-03T06:18:06.501 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000889c5a8d7f8aed12-006312f19e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.503 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:06.506 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:06.511 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.520 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.520 DEBUG:boto:Method: GET 2022-09-03T06:18:06.521 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.522 DEBUG:boto:Data: 2022-09-03T06:18:06.522 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.523 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:06.530 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.531 DEBUG:boto:Params: {} 2022-09-03T06:18:06.532 DEBUG:boto:Token: None 2022-09-03T06:18:06.532 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:06.533 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hrk70Bl5fd7n1FDbQsVkYXBhDko= 2022-09-03T06:18:06.534 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hrk70Bl5fd7n1FDbQsVkYXBhDko=', 'Content-Length': '0'} 2022-09-03T06:18:06.536 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000066a2d9e3809417c8-006312f19e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.537 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:06.538 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:06.538 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:06.539 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:06.539 DEBUG:boto:Method: GET 2022-09-03T06:18:06.540 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:06.541 DEBUG:boto:Data: 2022-09-03T06:18:06.541 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.542 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:06.542 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.543 DEBUG:boto:Params: {} 2022-09-03T06:18:06.544 DEBUG:boto:Token: None 2022-09-03T06:18:06.544 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/ 2022-09-03T06:18:06.545 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:synIQcbH4yY55nOkJ/X7LqyU/RQ= 2022-09-03T06:18:06.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:synIQcbH4yY55nOkJ/X7LqyU/RQ=', 'Content-Length': '0'} 2022-09-03T06:18:06.549 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004b7d4b356fb571fe-006312f19e-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.551 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:06.552 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:06.553 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:06.554 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:06.555 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:06.556 DEBUG:boto:Method: GET 2022-09-03T06:18:06.557 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:06.558 DEBUG:boto:Data: 2022-09-03T06:18:06.559 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.560 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:06.561 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.562 DEBUG:boto:Params: {} 2022-09-03T06:18:06.563 DEBUG:boto:Token: None 2022-09-03T06:18:06.565 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/ 2022-09-03T06:18:06.567 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:synIQcbH4yY55nOkJ/X7LqyU/RQ= 2022-09-03T06:18:06.567 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:synIQcbH4yY55nOkJ/X7LqyU/RQ=', 'Content-Length': '0'} 2022-09-03T06:18:06.573 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a36a8c2c4289bd87-006312f19e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.574 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:06.575 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:06.576 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:06.576 DEBUG:boto:Method: HEAD 2022-09-03T06:18:06.577 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:06.577 DEBUG:boto:Data: 2022-09-03T06:18:06.578 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.578 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:06.579 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.579 DEBUG:boto:Params: {} 2022-09-03T06:18:06.580 DEBUG:boto:Token: None 2022-09-03T06:18:06.581 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/e 2022-09-03T06:18:06.582 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XByq8/HglT0X2QbpAGDcXz+6zks= 2022-09-03T06:18:06.583 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XByq8/HglT0X2QbpAGDcXz+6zks=', 'Content-Length': '0'} 2022-09-03T06:18:06.585 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e933609033475af6-006312f19e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.586 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:06.586 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:06.587 DEBUG:boto:Method: HEAD 2022-09-03T06:18:06.587 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:06.588 DEBUG:boto:Data: 2022-09-03T06:18:06.588 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.589 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:06.589 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.590 DEBUG:boto:Params: {} 2022-09-03T06:18:06.590 DEBUG:boto:Token: None 2022-09-03T06:18:06.591 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/e 2022-09-03T06:18:06.592 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XByq8/HglT0X2QbpAGDcXz+6zks= 2022-09-03T06:18:06.594 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XByq8/HglT0X2QbpAGDcXz+6zks=', 'Content-Length': '0'} 2022-09-03T06:18:06.596 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000488f22b33822f1c3-006312f19e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.598 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:18:06.599 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:06.600 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:06.601 DEBUG:boto:Method: GET 2022-09-03T06:18:06.601 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:06.602 DEBUG:boto:Data: 2022-09-03T06:18:06.605 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.605 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:06.606 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.606 DEBUG:boto:Params: {} 2022-09-03T06:18:06.607 DEBUG:boto:Token: None 2022-09-03T06:18:06.608 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/e 2022-09-03T06:18:06.609 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jy0e+mE4lp9LrBD/ENvFhSo13G8= 2022-09-03T06:18:06.609 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jy0e+mE4lp9LrBD/ENvFhSo13G8=', 'Content-Length': '0'} 2022-09-03T06:18:06.613 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e05da3d345ea0c44-006312f19e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.614 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:06.615 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:06.615 DEBUG:boto:Method: GET 2022-09-03T06:18:06.616 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:06.617 DEBUG:boto:Data: 2022-09-03T06:18:06.617 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.618 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:06.618 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.619 DEBUG:boto:Params: {} 2022-09-03T06:18:06.619 DEBUG:boto:Token: None 2022-09-03T06:18:06.620 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/e 2022-09-03T06:18:06.621 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jy0e+mE4lp9LrBD/ENvFhSo13G8= 2022-09-03T06:18:06.621 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jy0e+mE4lp9LrBD/ENvFhSo13G8=', 'Content-Length': '0'} 2022-09-03T06:18:06.623 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f4b6032ee041c5bb-006312f19e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.624 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:06.625 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:06.625 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:06.626 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:06.626 DEBUG:boto:Method: GET 2022-09-03T06:18:06.627 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:06.627 DEBUG:boto:Data: 2022-09-03T06:18:06.628 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.628 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:06.629 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.630 DEBUG:boto:Params: {} 2022-09-03T06:18:06.631 DEBUG:boto:Token: None 2022-09-03T06:18:06.632 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/ 2022-09-03T06:18:06.632 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:synIQcbH4yY55nOkJ/X7LqyU/RQ= 2022-09-03T06:18:06.633 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:synIQcbH4yY55nOkJ/X7LqyU/RQ=', 'Content-Length': '0'} 2022-09-03T06:18:06.636 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002fe15d0e29c3c474-006312f19e-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.636 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:06.637 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:06.638 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:06.639 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:06.639 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:06.640 DEBUG:boto:Method: GET 2022-09-03T06:18:06.641 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:06.641 DEBUG:boto:Data: 2022-09-03T06:18:06.642 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.642 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:06.643 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.643 DEBUG:boto:Params: {} 2022-09-03T06:18:06.644 DEBUG:boto:Token: None 2022-09-03T06:18:06.645 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/ 2022-09-03T06:18:06.645 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:synIQcbH4yY55nOkJ/X7LqyU/RQ= 2022-09-03T06:18:06.646 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:synIQcbH4yY55nOkJ/X7LqyU/RQ=', 'Content-Length': '0'} 2022-09-03T06:18:06.649 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000faa31cf06af9423e-006312f19e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.651 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:06.652 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:18:06.654 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:06.656 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:06.657 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.658 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.659 DEBUG:boto:Method: GET 2022-09-03T06:18:06.660 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.661 DEBUG:boto:Data: 2022-09-03T06:18:06.662 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.663 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:06.664 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.665 DEBUG:boto:Params: {} 2022-09-03T06:18:06.666 DEBUG:boto:Token: None 2022-09-03T06:18:06.667 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.668 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:byzfNbGlgXKVfu9MY3xsptKy2cw= 2022-09-03T06:18:06.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:byzfNbGlgXKVfu9MY3xsptKy2cw=', 'Content-Length': '0'} 2022-09-03T06:18:06.673 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a20f057c0828bd43-006312f19e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.714 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:06.716 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:06.717 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.718 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.719 DEBUG:boto:Method: GET 2022-09-03T06:18:06.720 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.720 DEBUG:boto:Data: 2022-09-03T06:18:06.721 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.722 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:06.723 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.724 DEBUG:boto:Params: {} 2022-09-03T06:18:06.724 DEBUG:boto:Token: None 2022-09-03T06:18:06.725 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.726 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:byzfNbGlgXKVfu9MY3xsptKy2cw= 2022-09-03T06:18:06.726 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:byzfNbGlgXKVfu9MY3xsptKy2cw=', 'Content-Length': '0'} 2022-09-03T06:18:06.730 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c8f13278828f9eef-006312f19e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.731 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:06.732 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:06.733 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.733 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.734 DEBUG:boto:Method: HEAD 2022-09-03T06:18:06.734 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.739 DEBUG:boto:Data: 2022-09-03T06:18:06.739 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.740 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:06.740 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.741 DEBUG:boto:Params: {} 2022-09-03T06:18:06.741 DEBUG:boto:Token: None 2022-09-03T06:18:06.742 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.743 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lZo8BTb8d7SLOcJ4h8mopDYwlyc= 2022-09-03T06:18:06.743 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lZo8BTb8d7SLOcJ4h8mopDYwlyc=', 'Content-Length': '0'} 2022-09-03T06:18:06.750 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee2d3c5cfefad053-006312f19e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.751 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:06.752 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:06.752 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.753 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.759 DEBUG:boto:Method: HEAD 2022-09-03T06:18:06.760 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.760 DEBUG:boto:Data: 2022-09-03T06:18:06.761 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.761 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:06.762 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.763 DEBUG:boto:Params: {} 2022-09-03T06:18:06.763 DEBUG:boto:Token: None 2022-09-03T06:18:06.764 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.765 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lZo8BTb8d7SLOcJ4h8mopDYwlyc= 2022-09-03T06:18:06.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lZo8BTb8d7SLOcJ4h8mopDYwlyc=', 'Content-Length': '0'} 2022-09-03T06:18:06.786 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001e7cdc3db4cbc4d8-006312f19e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.788 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:18:06.789 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:06.802 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:06.804 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.805 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.806 DEBUG:boto:Method: GET 2022-09-03T06:18:06.807 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.808 DEBUG:boto:Data: 2022-09-03T06:18:06.809 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.810 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:06.811 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.812 DEBUG:boto:Params: {} 2022-09-03T06:18:06.813 DEBUG:boto:Token: None 2022-09-03T06:18:06.815 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.816 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:byzfNbGlgXKVfu9MY3xsptKy2cw= 2022-09-03T06:18:06.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:byzfNbGlgXKVfu9MY3xsptKy2cw=', 'Content-Length': '0'} 2022-09-03T06:18:06.821 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000024d21ba828e39fa0-006312f19e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.823 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:06.824 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:06.825 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.826 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.826 DEBUG:boto:Method: GET 2022-09-03T06:18:06.829 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.831 DEBUG:boto:Data: 2022-09-03T06:18:06.838 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.839 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:06.841 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.842 DEBUG:boto:Params: {} 2022-09-03T06:18:06.843 DEBUG:boto:Token: None 2022-09-03T06:18:06.844 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:06.845 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:byzfNbGlgXKVfu9MY3xsptKy2cw= 2022-09-03T06:18:06.846 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:byzfNbGlgXKVfu9MY3xsptKy2cw=', 'Content-Length': '0'} 2022-09-03T06:18:06.851 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007e4d61f81b824521-006312f19e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.852 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:06.853 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:06.855 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:06.856 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:06.857 DEBUG:boto:Method: GET 2022-09-03T06:18:06.858 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:06.859 DEBUG:boto:Data: 2022-09-03T06:18:06.860 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.862 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:06.863 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.864 DEBUG:boto:Params: {} 2022-09-03T06:18:06.865 DEBUG:boto:Token: None 2022-09-03T06:18:06.866 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/ 2022-09-03T06:18:06.867 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:synIQcbH4yY55nOkJ/X7LqyU/RQ= 2022-09-03T06:18:06.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:synIQcbH4yY55nOkJ/X7LqyU/RQ=', 'Content-Length': '0'} 2022-09-03T06:18:06.873 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000665017210efe43ba-006312f19e-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.875 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:06.877 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:06.878 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:06.879 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:06.880 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:06.881 DEBUG:boto:Method: GET 2022-09-03T06:18:06.884 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:06.885 DEBUG:boto:Data: 2022-09-03T06:18:06.886 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.887 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:06.888 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.889 DEBUG:boto:Params: {} 2022-09-03T06:18:06.892 DEBUG:boto:Token: None 2022-09-03T06:18:06.893 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/ 2022-09-03T06:18:06.895 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:synIQcbH4yY55nOkJ/X7LqyU/RQ= 2022-09-03T06:18:06.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:synIQcbH4yY55nOkJ/X7LqyU/RQ=', 'Content-Length': '0'} 2022-09-03T06:18:06.899 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001e146e5e2662a352-006312f19e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.900 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:06.901 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:06.901 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:06.902 DEBUG:boto:Method: HEAD 2022-09-03T06:18:06.902 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:06.903 DEBUG:boto:Data: 2022-09-03T06:18:06.904 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.904 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:06.905 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.905 DEBUG:boto:Params: {} 2022-09-03T06:18:06.906 DEBUG:boto:Token: None 2022-09-03T06:18:06.906 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/f 2022-09-03T06:18:06.907 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MzPfbX0rAej00adjwS988h9YsDA= 2022-09-03T06:18:06.907 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MzPfbX0rAej00adjwS988h9YsDA=', 'Content-Length': '0'} 2022-09-03T06:18:06.910 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000000a7d094103e491a-006312f19e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.911 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:06.911 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:06.912 DEBUG:boto:Method: HEAD 2022-09-03T06:18:06.912 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:06.913 DEBUG:boto:Data: 2022-09-03T06:18:06.914 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.914 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:06.915 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.915 DEBUG:boto:Params: {} 2022-09-03T06:18:06.916 DEBUG:boto:Token: None 2022-09-03T06:18:06.916 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/f 2022-09-03T06:18:06.917 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MzPfbX0rAej00adjwS988h9YsDA= 2022-09-03T06:18:06.918 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MzPfbX0rAej00adjwS988h9YsDA=', 'Content-Length': '0'} 2022-09-03T06:18:06.920 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b1a21b9d4d00576c-006312f19e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.921 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:18:06.921 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:06.922 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:06.923 DEBUG:boto:Method: GET 2022-09-03T06:18:06.923 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:06.924 DEBUG:boto:Data: 2022-09-03T06:18:06.924 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.925 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:06.925 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.926 DEBUG:boto:Params: {} 2022-09-03T06:18:06.927 DEBUG:boto:Token: None 2022-09-03T06:18:06.928 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/f 2022-09-03T06:18:06.928 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WW7XG7HpWBT3IH0ggm/esm2x6yQ= 2022-09-03T06:18:06.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WW7XG7HpWBT3IH0ggm/esm2x6yQ=', 'Content-Length': '0'} 2022-09-03T06:18:06.934 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a70ea400c82a6230-006312f19e-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.935 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:06.936 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:06.936 DEBUG:boto:Method: GET 2022-09-03T06:18:06.937 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:06.938 DEBUG:boto:Data: 2022-09-03T06:18:06.939 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.939 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:06.940 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.940 DEBUG:boto:Params: {} 2022-09-03T06:18:06.942 DEBUG:boto:Token: None 2022-09-03T06:18:06.943 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/f 2022-09-03T06:18:06.943 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WW7XG7HpWBT3IH0ggm/esm2x6yQ= 2022-09-03T06:18:06.945 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WW7XG7HpWBT3IH0ggm/esm2x6yQ=', 'Content-Length': '0'} 2022-09-03T06:18:06.950 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e61aebd6ca7aadaa-006312f19e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:06.951 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:06.952 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:06.953 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:06.953 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:06.954 DEBUG:boto:Method: GET 2022-09-03T06:18:06.955 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:06.956 DEBUG:boto:Data: 2022-09-03T06:18:06.956 DEBUG:boto:Headers: {} 2022-09-03T06:18:06.957 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:06.958 DEBUG:boto:Port: 8000 2022-09-03T06:18:06.958 DEBUG:boto:Params: {} 2022-09-03T06:18:06.959 DEBUG:boto:Token: None 2022-09-03T06:18:06.959 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:06 GMT /dsgvcy-71/ 2022-09-03T06:18:06.960 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:synIQcbH4yY55nOkJ/X7LqyU/RQ= 2022-09-03T06:18:06.961 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:synIQcbH4yY55nOkJ/X7LqyU/RQ=', 'Content-Length': '0'} 2022-09-03T06:18:06.964 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000022e881cd91a25501-006312f19e-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.010 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:07.011 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:07.012 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:07.012 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:07.013 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:07.013 DEBUG:boto:Method: GET 2022-09-03T06:18:07.014 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:07.014 DEBUG:boto:Data: 2022-09-03T06:18:07.015 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.016 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:07.016 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.017 DEBUG:boto:Params: {} 2022-09-03T06:18:07.017 DEBUG:boto:Token: None 2022-09-03T06:18:07.018 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/ 2022-09-03T06:18:07.018 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QfRK0A/eWZVje7U/rMRn86pzrSI= 2022-09-03T06:18:07.019 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QfRK0A/eWZVje7U/rMRn86pzrSI=', 'Content-Length': '0'} 2022-09-03T06:18:07.021 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000b50929e005188ed-006312f19f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.022 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:07.023 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:18:07.024 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:07.024 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:07.025 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.025 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.026 DEBUG:boto:Method: GET 2022-09-03T06:18:07.026 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.027 DEBUG:boto:Data: 2022-09-03T06:18:07.027 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.028 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:07.028 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.029 DEBUG:boto:Params: {} 2022-09-03T06:18:07.029 DEBUG:boto:Token: None 2022-09-03T06:18:07.030 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.032 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q9r+cjO3kSKmTOaQM5PsCYpl00M= 2022-09-03T06:18:07.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q9r+cjO3kSKmTOaQM5PsCYpl00M=', 'Content-Length': '0'} 2022-09-03T06:18:07.035 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a30509cd337b70a9-006312f19f-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.036 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:07.036 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:07.037 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.037 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.038 DEBUG:boto:Method: GET 2022-09-03T06:18:07.038 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.039 DEBUG:boto:Data: 2022-09-03T06:18:07.039 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.040 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:07.040 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.041 DEBUG:boto:Params: {} 2022-09-03T06:18:07.042 DEBUG:boto:Token: None 2022-09-03T06:18:07.042 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.043 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q9r+cjO3kSKmTOaQM5PsCYpl00M= 2022-09-03T06:18:07.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q9r+cjO3kSKmTOaQM5PsCYpl00M=', 'Content-Length': '0'} 2022-09-03T06:18:07.046 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d1877913e421a331-006312f19f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.047 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:07.047 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:07.048 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.048 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.049 DEBUG:boto:Method: HEAD 2022-09-03T06:18:07.049 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.050 DEBUG:boto:Data: 2022-09-03T06:18:07.050 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.051 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:07.051 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.052 DEBUG:boto:Params: {} 2022-09-03T06:18:07.052 DEBUG:boto:Token: None 2022-09-03T06:18:07.053 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.054 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r5bxiMmctrJWfMGg2PUzBMiV60Q= 2022-09-03T06:18:07.054 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r5bxiMmctrJWfMGg2PUzBMiV60Q=', 'Content-Length': '0'} 2022-09-03T06:18:07.056 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000067b634e137bc1ea9-006312f19f-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.057 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:07.058 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:07.058 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.059 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.059 DEBUG:boto:Method: HEAD 2022-09-03T06:18:07.060 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.060 DEBUG:boto:Data: 2022-09-03T06:18:07.061 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.061 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:07.062 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.062 DEBUG:boto:Params: {} 2022-09-03T06:18:07.063 DEBUG:boto:Token: None 2022-09-03T06:18:07.064 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.064 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r5bxiMmctrJWfMGg2PUzBMiV60Q= 2022-09-03T06:18:07.065 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r5bxiMmctrJWfMGg2PUzBMiV60Q=', 'Content-Length': '0'} 2022-09-03T06:18:07.067 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004e473b0193ca29d2-006312f19f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.068 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:18:07.068 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:07.069 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:07.069 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.070 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.070 DEBUG:boto:Method: GET 2022-09-03T06:18:07.071 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.071 DEBUG:boto:Data: 2022-09-03T06:18:07.072 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.072 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:07.073 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.073 DEBUG:boto:Params: {} 2022-09-03T06:18:07.074 DEBUG:boto:Token: None 2022-09-03T06:18:07.075 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.076 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q9r+cjO3kSKmTOaQM5PsCYpl00M= 2022-09-03T06:18:07.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q9r+cjO3kSKmTOaQM5PsCYpl00M=', 'Content-Length': '0'} 2022-09-03T06:18:07.079 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dbb3a178ed9109d2-006312f19f-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.079 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:07.080 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:07.081 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.081 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.082 DEBUG:boto:Method: GET 2022-09-03T06:18:07.082 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.083 DEBUG:boto:Data: 2022-09-03T06:18:07.083 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.084 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:07.084 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.085 DEBUG:boto:Params: {} 2022-09-03T06:18:07.085 DEBUG:boto:Token: None 2022-09-03T06:18:07.086 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:07.087 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q9r+cjO3kSKmTOaQM5PsCYpl00M= 2022-09-03T06:18:07.087 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q9r+cjO3kSKmTOaQM5PsCYpl00M=', 'Content-Length': '0'} 2022-09-03T06:18:07.089 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000838ad80392ae8dfb-006312f19f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.131 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:07.132 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:07.133 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:07.134 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:07.135 DEBUG:boto:Method: GET 2022-09-03T06:18:07.135 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:07.136 DEBUG:boto:Data: 2022-09-03T06:18:07.137 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.138 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:07.139 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.139 DEBUG:boto:Params: {} 2022-09-03T06:18:07.141 DEBUG:boto:Token: None 2022-09-03T06:18:07.142 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/ 2022-09-03T06:18:07.143 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QfRK0A/eWZVje7U/rMRn86pzrSI= 2022-09-03T06:18:07.145 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QfRK0A/eWZVje7U/rMRn86pzrSI=', 'Content-Length': '0'} 2022-09-03T06:18:07.148 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006f9b03f795d886c3-006312f19f-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.149 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:07.150 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:07.151 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:07.151 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:07.152 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:07.152 DEBUG:boto:Method: GET 2022-09-03T06:18:07.153 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:07.153 DEBUG:boto:Data: 2022-09-03T06:18:07.154 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.154 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:07.155 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.155 DEBUG:boto:Params: {} 2022-09-03T06:18:07.157 DEBUG:boto:Token: None 2022-09-03T06:18:07.158 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/ 2022-09-03T06:18:07.158 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QfRK0A/eWZVje7U/rMRn86pzrSI= 2022-09-03T06:18:07.159 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QfRK0A/eWZVje7U/rMRn86pzrSI=', 'Content-Length': '0'} 2022-09-03T06:18:07.161 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000809d79cd42db6bb9-006312f19f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.162 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:07.163 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:07.164 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:07.164 DEBUG:boto:Method: HEAD 2022-09-03T06:18:07.165 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:07.165 DEBUG:boto:Data: 2022-09-03T06:18:07.166 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.166 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:07.167 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.168 DEBUG:boto:Params: {} 2022-09-03T06:18:07.168 DEBUG:boto:Token: None 2022-09-03T06:18:07.169 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/g 2022-09-03T06:18:07.169 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rc1ANBrXBitnu9KBzz8msxj+8t8= 2022-09-03T06:18:07.170 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rc1ANBrXBitnu9KBzz8msxj+8t8=', 'Content-Length': '0'} 2022-09-03T06:18:07.172 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007893797451c97d19-006312f19f-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.173 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:07.173 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:07.174 DEBUG:boto:Method: HEAD 2022-09-03T06:18:07.174 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:07.175 DEBUG:boto:Data: 2022-09-03T06:18:07.175 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.176 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:07.176 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.177 DEBUG:boto:Params: {} 2022-09-03T06:18:07.178 DEBUG:boto:Token: None 2022-09-03T06:18:07.179 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/g 2022-09-03T06:18:07.179 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rc1ANBrXBitnu9KBzz8msxj+8t8= 2022-09-03T06:18:07.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rc1ANBrXBitnu9KBzz8msxj+8t8=', 'Content-Length': '0'} 2022-09-03T06:18:07.184 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee9ae6d63cea02bd-006312f19f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.185 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:18:07.186 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:07.187 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:07.189 DEBUG:boto:Method: GET 2022-09-03T06:18:07.190 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:07.190 DEBUG:boto:Data: 2022-09-03T06:18:07.191 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.192 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:07.193 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.194 DEBUG:boto:Params: {} 2022-09-03T06:18:07.195 DEBUG:boto:Token: None 2022-09-03T06:18:07.196 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/g 2022-09-03T06:18:07.198 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4sSCjo8XImn9KMpygZIYSTmZrtg= 2022-09-03T06:18:07.199 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4sSCjo8XImn9KMpygZIYSTmZrtg=', 'Content-Length': '0'} 2022-09-03T06:18:07.202 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000605d5d3da1b5175d-006312f19f-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.204 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:07.204 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:07.205 DEBUG:boto:Method: GET 2022-09-03T06:18:07.205 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:07.206 DEBUG:boto:Data: 2022-09-03T06:18:07.207 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.207 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:07.208 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.209 DEBUG:boto:Params: {} 2022-09-03T06:18:07.210 DEBUG:boto:Token: None 2022-09-03T06:18:07.211 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/g 2022-09-03T06:18:07.213 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4sSCjo8XImn9KMpygZIYSTmZrtg= 2022-09-03T06:18:07.214 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4sSCjo8XImn9KMpygZIYSTmZrtg=', 'Content-Length': '0'} 2022-09-03T06:18:07.217 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ce7747a4382705ac-006312f19f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.220 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:07.221 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:07.222 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:07.223 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:07.224 DEBUG:boto:Method: GET 2022-09-03T06:18:07.225 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:07.226 DEBUG:boto:Data: 2022-09-03T06:18:07.227 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.228 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:07.229 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.230 DEBUG:boto:Params: {} 2022-09-03T06:18:07.232 DEBUG:boto:Token: None 2022-09-03T06:18:07.233 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/ 2022-09-03T06:18:07.234 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QfRK0A/eWZVje7U/rMRn86pzrSI= 2022-09-03T06:18:07.235 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QfRK0A/eWZVje7U/rMRn86pzrSI=', 'Content-Length': '0'} 2022-09-03T06:18:07.239 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ab1b913211474b27-006312f19f-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.288 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:07.290 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:07.291 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:07.292 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:07.293 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:07.306 DEBUG:boto:Method: GET 2022-09-03T06:18:07.306 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:07.307 DEBUG:boto:Data: 2022-09-03T06:18:07.307 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.308 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:07.308 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.309 DEBUG:boto:Params: {} 2022-09-03T06:18:07.310 DEBUG:boto:Token: None 2022-09-03T06:18:07.310 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/ 2022-09-03T06:18:07.311 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QfRK0A/eWZVje7U/rMRn86pzrSI= 2022-09-03T06:18:07.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QfRK0A/eWZVje7U/rMRn86pzrSI=', 'Content-Length': '0'} 2022-09-03T06:18:07.314 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000028d1acf5181e0afc-006312f19f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.316 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:07.317 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:18:07.318 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:07.319 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:07.320 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.320 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.321 DEBUG:boto:Method: GET 2022-09-03T06:18:07.322 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.322 DEBUG:boto:Data: 2022-09-03T06:18:07.323 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.325 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:07.325 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.326 DEBUG:boto:Params: {} 2022-09-03T06:18:07.327 DEBUG:boto:Token: None 2022-09-03T06:18:07.327 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.328 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:luVJEUe+yW2wUzSrXadER7xrTo4= 2022-09-03T06:18:07.329 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:luVJEUe+yW2wUzSrXadER7xrTo4=', 'Content-Length': '0'} 2022-09-03T06:18:07.343 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006959c229624f10cd-006312f19f-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.344 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:07.345 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:07.347 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.347 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.348 DEBUG:boto:Method: GET 2022-09-03T06:18:07.349 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.350 DEBUG:boto:Data: 2022-09-03T06:18:07.351 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.352 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:07.353 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.354 DEBUG:boto:Params: {} 2022-09-03T06:18:07.355 DEBUG:boto:Token: None 2022-09-03T06:18:07.356 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.358 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:luVJEUe+yW2wUzSrXadER7xrTo4= 2022-09-03T06:18:07.359 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:luVJEUe+yW2wUzSrXadER7xrTo4=', 'Content-Length': '0'} 2022-09-03T06:18:07.361 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007eeb0945f5b14aa9-006312f19f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.362 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:07.363 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:07.363 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.364 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.365 DEBUG:boto:Method: HEAD 2022-09-03T06:18:07.365 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.366 DEBUG:boto:Data: 2022-09-03T06:18:07.366 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.367 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:07.368 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.368 DEBUG:boto:Params: {} 2022-09-03T06:18:07.369 DEBUG:boto:Token: None 2022-09-03T06:18:07.369 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.370 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eRziBkVuIr+Zij1UHM0SuxGf7fk= 2022-09-03T06:18:07.371 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eRziBkVuIr+Zij1UHM0SuxGf7fk=', 'Content-Length': '0'} 2022-09-03T06:18:07.373 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003d590ddfcdd864f9-006312f19f-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.374 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:07.374 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:07.375 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.376 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.376 DEBUG:boto:Method: HEAD 2022-09-03T06:18:07.377 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.377 DEBUG:boto:Data: 2022-09-03T06:18:07.378 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.379 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:07.379 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.380 DEBUG:boto:Params: {} 2022-09-03T06:18:07.380 DEBUG:boto:Token: None 2022-09-03T06:18:07.381 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.382 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eRziBkVuIr+Zij1UHM0SuxGf7fk= 2022-09-03T06:18:07.382 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eRziBkVuIr+Zij1UHM0SuxGf7fk=', 'Content-Length': '0'} 2022-09-03T06:18:07.385 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e8cd1ad440280c4-006312f19f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.387 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:18:07.390 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:07.391 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:07.392 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.393 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.394 DEBUG:boto:Method: GET 2022-09-03T06:18:07.395 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.401 DEBUG:boto:Data: 2022-09-03T06:18:07.403 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.404 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:07.406 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.407 DEBUG:boto:Params: {} 2022-09-03T06:18:07.408 DEBUG:boto:Token: None 2022-09-03T06:18:07.409 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.411 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:luVJEUe+yW2wUzSrXadER7xrTo4= 2022-09-03T06:18:07.415 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:luVJEUe+yW2wUzSrXadER7xrTo4=', 'Content-Length': '0'} 2022-09-03T06:18:07.417 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d56824cd7af3374a-006312f19f-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.418 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:07.419 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:07.419 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.420 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.420 DEBUG:boto:Method: GET 2022-09-03T06:18:07.421 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.421 DEBUG:boto:Data: 2022-09-03T06:18:07.422 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.423 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:07.423 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.424 DEBUG:boto:Params: {} 2022-09-03T06:18:07.425 DEBUG:boto:Token: None 2022-09-03T06:18:07.426 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:07.429 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:luVJEUe+yW2wUzSrXadER7xrTo4= 2022-09-03T06:18:07.431 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:luVJEUe+yW2wUzSrXadER7xrTo4=', 'Content-Length': '0'} 2022-09-03T06:18:07.441 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c776509cb376bfd2-006312f19f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.444 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:07.445 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:07.446 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:07.447 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:07.448 DEBUG:boto:Method: GET 2022-09-03T06:18:07.449 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:07.450 DEBUG:boto:Data: 2022-09-03T06:18:07.452 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.453 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:07.454 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.458 DEBUG:boto:Params: {} 2022-09-03T06:18:07.459 DEBUG:boto:Token: None 2022-09-03T06:18:07.460 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/ 2022-09-03T06:18:07.463 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QfRK0A/eWZVje7U/rMRn86pzrSI= 2022-09-03T06:18:07.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QfRK0A/eWZVje7U/rMRn86pzrSI=', 'Content-Length': '0'} 2022-09-03T06:18:07.476 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cb84b4a5bcd04995-006312f19f-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.477 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:07.479 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:07.480 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:07.480 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:07.481 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:07.481 DEBUG:boto:Method: GET 2022-09-03T06:18:07.482 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:07.482 DEBUG:boto:Data: 2022-09-03T06:18:07.483 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.484 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:07.484 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.485 DEBUG:boto:Params: {} 2022-09-03T06:18:07.486 DEBUG:boto:Token: None 2022-09-03T06:18:07.486 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/ 2022-09-03T06:18:07.487 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QfRK0A/eWZVje7U/rMRn86pzrSI= 2022-09-03T06:18:07.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QfRK0A/eWZVje7U/rMRn86pzrSI=', 'Content-Length': '0'} 2022-09-03T06:18:07.493 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000da67ff8d59d8b7eb-006312f19f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.494 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:07.496 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:07.497 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:07.498 DEBUG:boto:Method: HEAD 2022-09-03T06:18:07.499 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:07.506 DEBUG:boto:Data: 2022-09-03T06:18:07.507 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.507 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:07.508 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.508 DEBUG:boto:Params: {} 2022-09-03T06:18:07.509 DEBUG:boto:Token: None 2022-09-03T06:18:07.509 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/h 2022-09-03T06:18:07.510 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2/YdXjWwksTWNUii+p2ISjws/SI= 2022-09-03T06:18:07.510 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2/YdXjWwksTWNUii+p2ISjws/SI=', 'Content-Length': '0'} 2022-09-03T06:18:07.516 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f17aaf229519114d-006312f19f-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.517 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:07.518 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:07.519 DEBUG:boto:Method: HEAD 2022-09-03T06:18:07.519 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:07.520 DEBUG:boto:Data: 2022-09-03T06:18:07.520 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.521 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:07.521 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.522 DEBUG:boto:Params: {} 2022-09-03T06:18:07.523 DEBUG:boto:Token: None 2022-09-03T06:18:07.523 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/h 2022-09-03T06:18:07.524 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2/YdXjWwksTWNUii+p2ISjws/SI= 2022-09-03T06:18:07.524 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2/YdXjWwksTWNUii+p2ISjws/SI=', 'Content-Length': '0'} 2022-09-03T06:18:07.529 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000256701b385a6020c-006312f19f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.530 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:18:07.531 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:07.531 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:07.532 DEBUG:boto:Method: GET 2022-09-03T06:18:07.533 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:07.534 DEBUG:boto:Data: 2022-09-03T06:18:07.535 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.536 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:07.537 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.539 DEBUG:boto:Params: {} 2022-09-03T06:18:07.541 DEBUG:boto:Token: None 2022-09-03T06:18:07.542 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/h 2022-09-03T06:18:07.543 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S9T4By4tWFMcT8BEpld5tkOPmT4= 2022-09-03T06:18:07.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S9T4By4tWFMcT8BEpld5tkOPmT4=', 'Content-Length': '0'} 2022-09-03T06:18:07.547 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c12c58d9884c5201-006312f19f-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.595 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:07.596 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:07.597 DEBUG:boto:Method: GET 2022-09-03T06:18:07.598 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:07.599 DEBUG:boto:Data: 2022-09-03T06:18:07.600 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.601 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:07.603 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.604 DEBUG:boto:Params: {} 2022-09-03T06:18:07.605 DEBUG:boto:Token: None 2022-09-03T06:18:07.606 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/h 2022-09-03T06:18:07.607 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S9T4By4tWFMcT8BEpld5tkOPmT4= 2022-09-03T06:18:07.608 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S9T4By4tWFMcT8BEpld5tkOPmT4=', 'Content-Length': '0'} 2022-09-03T06:18:07.612 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a35c126b5613738-006312f19f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.614 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:07.614 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:07.615 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:07.616 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:07.617 DEBUG:boto:Method: GET 2022-09-03T06:18:07.618 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:07.619 DEBUG:boto:Data: 2022-09-03T06:18:07.620 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.621 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:07.622 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.623 DEBUG:boto:Params: {} 2022-09-03T06:18:07.624 DEBUG:boto:Token: None 2022-09-03T06:18:07.626 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/ 2022-09-03T06:18:07.627 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QfRK0A/eWZVje7U/rMRn86pzrSI= 2022-09-03T06:18:07.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QfRK0A/eWZVje7U/rMRn86pzrSI=', 'Content-Length': '0'} 2022-09-03T06:18:07.632 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ef4c1b519dfe61a6-006312f19f-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.637 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:07.638 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:07.639 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:07.640 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:07.640 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:07.641 DEBUG:boto:Method: GET 2022-09-03T06:18:07.642 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:07.642 DEBUG:boto:Data: 2022-09-03T06:18:07.643 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.644 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:07.645 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.646 DEBUG:boto:Params: {} 2022-09-03T06:18:07.647 DEBUG:boto:Token: None 2022-09-03T06:18:07.648 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/ 2022-09-03T06:18:07.649 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QfRK0A/eWZVje7U/rMRn86pzrSI= 2022-09-03T06:18:07.650 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QfRK0A/eWZVje7U/rMRn86pzrSI=', 'Content-Length': '0'} 2022-09-03T06:18:07.653 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000730ab62c3c532c9c-006312f19f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.655 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:07.656 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-71 zones={test-zone2, test-zone1} 2022-09-03T06:18:07.657 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-71 zones={test-zone3, test-zone1} 2022-09-03T06:18:07.658 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:07.659 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:07.660 DEBUG:boto:Method: HEAD 2022-09-03T06:18:07.662 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:18:07.662 DEBUG:boto:Data: 2022-09-03T06:18:07.663 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.664 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:07.664 DEBUG:boto:Port: 8001 2022-09-03T06:18:07.665 DEBUG:boto:Params: {} 2022-09-03T06:18:07.665 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T06:18:07.666 DEBUG:boto:Token: None 2022-09-03T06:18:07.666 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/ 2022-09-03T06:18:07.667 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qTDxhiIhbB0REqf0TizDRYCbgu0= 2022-09-03T06:18:07.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qTDxhiIhbB0REqf0TizDRYCbgu0=', 'Content-Length': '0'} 2022-09-03T06:18:07.671 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b5ea4583774d2a4c-006312f19f-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.672 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:07.673 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:07.674 DEBUG:boto:Method: HEAD 2022-09-03T06:18:07.674 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:18:07.675 DEBUG:boto:Data: 2022-09-03T06:18:07.675 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.676 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:07.676 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.677 DEBUG:boto:Params: {} 2022-09-03T06:18:07.677 DEBUG:boto:Token: None 2022-09-03T06:18:07.678 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/ 2022-09-03T06:18:07.679 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qTDxhiIhbB0REqf0TizDRYCbgu0= 2022-09-03T06:18:07.682 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qTDxhiIhbB0REqf0TizDRYCbgu0=', 'Content-Length': '0'} 2022-09-03T06:18:07.685 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d0075a92657c8bb8-006312f19f-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.688 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:18:07.688 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:07.689 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:07.690 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:07.691 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:07.691 DEBUG:boto:Method: GET 2022-09-03T06:18:07.692 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:07.693 DEBUG:boto:Data: 2022-09-03T06:18:07.693 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.694 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:07.694 DEBUG:boto:Port: 8001 2022-09-03T06:18:07.695 DEBUG:boto:Params: {} 2022-09-03T06:18:07.695 DEBUG:boto:Token: None 2022-09-03T06:18:07.696 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/?versions 2022-09-03T06:18:07.697 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nZ26gty5NOZX8PzEuIx6dic940s= 2022-09-03T06:18:07.698 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nZ26gty5NOZX8PzEuIx6dic940s=', 'Content-Length': '0'} 2022-09-03T06:18:07.700 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004f1b32b6c8d1a5a3-006312f19f-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.701 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:07.704 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:18:07.705 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:18:07.706 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:18:07.706 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:18:07.707 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:18:07.707 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:18:07.708 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:18:07.708 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:18:07.709 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:18:07.710 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:07.711 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:07.712 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:07.713 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:07.714 DEBUG:boto:Method: GET 2022-09-03T06:18:07.715 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:07.715 DEBUG:boto:Data: 2022-09-03T06:18:07.716 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.717 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:07.718 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.719 DEBUG:boto:Params: {} 2022-09-03T06:18:07.720 DEBUG:boto:Token: None 2022-09-03T06:18:07.722 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/?versions 2022-09-03T06:18:07.723 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nZ26gty5NOZX8PzEuIx6dic940s= 2022-09-03T06:18:07.724 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nZ26gty5NOZX8PzEuIx6dic940s=', 'Content-Length': '0'} 2022-09-03T06:18:07.727 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001bcf4ea39f2b0595-006312f19f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.728 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:07.730 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:18:07.731 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:18:07.732 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:18:07.733 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:18:07.733 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:18:07.734 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:18:07.735 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:18:07.736 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:18:07.737 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:07.738 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:07.739 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:07.740 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:07.741 DEBUG:boto:Method: GET 2022-09-03T06:18:07.742 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:07.743 DEBUG:boto:Data: 2022-09-03T06:18:07.745 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.746 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:07.747 DEBUG:boto:Port: 8001 2022-09-03T06:18:07.748 DEBUG:boto:Params: {} 2022-09-03T06:18:07.749 DEBUG:boto:Token: None 2022-09-03T06:18:07.751 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/?versions 2022-09-03T06:18:07.752 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nZ26gty5NOZX8PzEuIx6dic940s= 2022-09-03T06:18:07.753 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nZ26gty5NOZX8PzEuIx6dic940s=', 'Content-Length': '0'} 2022-09-03T06:18:07.757 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b5c1f69bcb45e4b8-006312f19f-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.768 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:07.772 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:07.773 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:07.774 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:07.775 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:07.776 DEBUG:boto:Method: GET 2022-09-03T06:18:07.777 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:07.778 DEBUG:boto:Data: 2022-09-03T06:18:07.779 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.780 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:07.781 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.782 DEBUG:boto:Params: {} 2022-09-03T06:18:07.783 DEBUG:boto:Token: None 2022-09-03T06:18:07.784 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/?versions 2022-09-03T06:18:07.785 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nZ26gty5NOZX8PzEuIx6dic940s= 2022-09-03T06:18:07.786 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nZ26gty5NOZX8PzEuIx6dic940s=', 'Content-Length': '0'} 2022-09-03T06:18:07.789 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003539ad456657607b-006312f19f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.791 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:07.794 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:18:07.796 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:07.797 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:07.797 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.798 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.799 DEBUG:boto:Method: GET 2022-09-03T06:18:07.800 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.801 DEBUG:boto:Data: 2022-09-03T06:18:07.801 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.802 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:07.802 DEBUG:boto:Port: 8001 2022-09-03T06:18:07.803 DEBUG:boto:Params: {} 2022-09-03T06:18:07.803 DEBUG:boto:Token: None 2022-09-03T06:18:07.804 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.805 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H7PofIFNyV/6iCNxYDcXsiEtDH4= 2022-09-03T06:18:07.806 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H7PofIFNyV/6iCNxYDcXsiEtDH4=', 'Content-Length': '0'} 2022-09-03T06:18:07.812 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000260f4bd354398e53-006312f19f-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.815 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:07.816 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:07.817 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.818 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.819 DEBUG:boto:Method: GET 2022-09-03T06:18:07.820 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.821 DEBUG:boto:Data: 2022-09-03T06:18:07.826 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.829 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:07.829 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.830 DEBUG:boto:Params: {} 2022-09-03T06:18:07.831 DEBUG:boto:Token: None 2022-09-03T06:18:07.833 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.833 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H7PofIFNyV/6iCNxYDcXsiEtDH4= 2022-09-03T06:18:07.834 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H7PofIFNyV/6iCNxYDcXsiEtDH4=', 'Content-Length': '0'} 2022-09-03T06:18:07.836 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004b111f2b75e798ab-006312f19f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.837 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:07.838 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:07.838 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.839 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.839 DEBUG:boto:Method: HEAD 2022-09-03T06:18:07.840 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.841 DEBUG:boto:Data: 2022-09-03T06:18:07.842 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.843 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:07.844 DEBUG:boto:Port: 8001 2022-09-03T06:18:07.845 DEBUG:boto:Params: {} 2022-09-03T06:18:07.847 DEBUG:boto:Token: None 2022-09-03T06:18:07.848 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.849 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vqwURvoT+XEZ7sKHdEd77XjQ5Tk= 2022-09-03T06:18:07.850 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vqwURvoT+XEZ7sKHdEd77XjQ5Tk=', 'Content-Length': '0'} 2022-09-03T06:18:07.855 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b2d6ef61aeb9ae0-006312f19f-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.856 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:07.857 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:07.858 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.858 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.859 DEBUG:boto:Method: HEAD 2022-09-03T06:18:07.859 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.863 DEBUG:boto:Data: 2022-09-03T06:18:07.864 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.864 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:07.865 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.865 DEBUG:boto:Params: {} 2022-09-03T06:18:07.866 DEBUG:boto:Token: None 2022-09-03T06:18:07.867 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.868 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vqwURvoT+XEZ7sKHdEd77XjQ5Tk= 2022-09-03T06:18:07.869 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vqwURvoT+XEZ7sKHdEd77XjQ5Tk=', 'Content-Length': '0'} 2022-09-03T06:18:07.874 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f101cd8d16b85b1a-006312f19f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.875 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:18:07.876 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:07.877 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:07.879 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.880 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.881 DEBUG:boto:Method: GET 2022-09-03T06:18:07.882 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.885 DEBUG:boto:Data: 2022-09-03T06:18:07.887 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.887 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:07.888 DEBUG:boto:Port: 8001 2022-09-03T06:18:07.888 DEBUG:boto:Params: {} 2022-09-03T06:18:07.889 DEBUG:boto:Token: None 2022-09-03T06:18:07.890 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.890 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H7PofIFNyV/6iCNxYDcXsiEtDH4= 2022-09-03T06:18:07.891 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H7PofIFNyV/6iCNxYDcXsiEtDH4=', 'Content-Length': '0'} 2022-09-03T06:18:07.893 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e118535283d4fa5-006312f19f-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.895 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:07.895 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:07.897 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.898 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.899 DEBUG:boto:Method: GET 2022-09-03T06:18:07.900 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.901 DEBUG:boto:Data: 2022-09-03T06:18:07.902 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.903 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:07.904 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.905 DEBUG:boto:Params: {} 2022-09-03T06:18:07.906 DEBUG:boto:Token: None 2022-09-03T06:18:07.907 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:07.909 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H7PofIFNyV/6iCNxYDcXsiEtDH4= 2022-09-03T06:18:07.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H7PofIFNyV/6iCNxYDcXsiEtDH4=', 'Content-Length': '0'} 2022-09-03T06:18:07.913 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ccd628c6cccc14e-006312f19f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.922 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:07.924 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:07.925 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:07.926 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:07.927 DEBUG:boto:Method: GET 2022-09-03T06:18:07.928 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:07.931 DEBUG:boto:Data: 2022-09-03T06:18:07.932 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.933 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:07.934 DEBUG:boto:Port: 8001 2022-09-03T06:18:07.934 DEBUG:boto:Params: {} 2022-09-03T06:18:07.935 DEBUG:boto:Token: None 2022-09-03T06:18:07.936 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/ 2022-09-03T06:18:07.936 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QfRK0A/eWZVje7U/rMRn86pzrSI= 2022-09-03T06:18:07.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QfRK0A/eWZVje7U/rMRn86pzrSI=', 'Content-Length': '0'} 2022-09-03T06:18:07.940 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d744a53fda6af1cc-006312f19f-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:07.942 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:07.944 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:07.945 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:07.946 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:07.947 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:07.948 DEBUG:boto:Method: GET 2022-09-03T06:18:07.949 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:07.950 DEBUG:boto:Data: 2022-09-03T06:18:07.951 DEBUG:boto:Headers: {} 2022-09-03T06:18:07.953 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:07.955 DEBUG:boto:Port: 8000 2022-09-03T06:18:07.955 DEBUG:boto:Params: {} 2022-09-03T06:18:07.956 DEBUG:boto:Token: None 2022-09-03T06:18:07.957 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:07 GMT /dsgvcy-71/ 2022-09-03T06:18:07.958 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QfRK0A/eWZVje7U/rMRn86pzrSI= 2022-09-03T06:18:07.960 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QfRK0A/eWZVje7U/rMRn86pzrSI=', 'Content-Length': '0'} 2022-09-03T06:18:07.965 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a7f56d1ba3c955eb-006312f19f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.008 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:08.009 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:08.010 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:08.011 DEBUG:boto:Method: HEAD 2022-09-03T06:18:08.012 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:08.012 DEBUG:boto:Data: 2022-09-03T06:18:08.013 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.014 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.015 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.016 DEBUG:boto:Params: {} 2022-09-03T06:18:08.017 DEBUG:boto:Token: None 2022-09-03T06:18:08.018 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/a 2022-09-03T06:18:08.019 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bdFFMKCZGifTr/Ue5tDbowMAkPg= 2022-09-03T06:18:08.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bdFFMKCZGifTr/Ue5tDbowMAkPg=', 'Content-Length': '0'} 2022-09-03T06:18:08.024 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000209665a0e17b269d-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.025 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:08.026 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:08.027 DEBUG:boto:Method: HEAD 2022-09-03T06:18:08.030 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:08.031 DEBUG:boto:Data: 2022-09-03T06:18:08.031 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.032 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:08.032 DEBUG:boto:Port: 8000 2022-09-03T06:18:08.033 DEBUG:boto:Params: {} 2022-09-03T06:18:08.033 DEBUG:boto:Token: None 2022-09-03T06:18:08.034 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/a 2022-09-03T06:18:08.036 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bdFFMKCZGifTr/Ue5tDbowMAkPg= 2022-09-03T06:18:08.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bdFFMKCZGifTr/Ue5tDbowMAkPg=', 'Content-Length': '0'} 2022-09-03T06:18:08.039 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008c10eb16a532d337-006312f1a0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.040 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:18:08.041 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:08.043 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:08.044 DEBUG:boto:Method: GET 2022-09-03T06:18:08.045 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:08.047 DEBUG:boto:Data: 2022-09-03T06:18:08.048 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.050 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.051 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.052 DEBUG:boto:Params: {} 2022-09-03T06:18:08.053 DEBUG:boto:Token: None 2022-09-03T06:18:08.054 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/a 2022-09-03T06:18:08.055 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gkJhC3v9C9EVGNN6Mr22VFrnjqo= 2022-09-03T06:18:08.057 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gkJhC3v9C9EVGNN6Mr22VFrnjqo=', 'Content-Length': '0'} 2022-09-03T06:18:08.060 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000977accc42049e393-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.061 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:08.062 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:08.062 DEBUG:boto:Method: GET 2022-09-03T06:18:08.063 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:08.063 DEBUG:boto:Data: 2022-09-03T06:18:08.064 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.064 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:08.065 DEBUG:boto:Port: 8000 2022-09-03T06:18:08.065 DEBUG:boto:Params: {} 2022-09-03T06:18:08.066 DEBUG:boto:Token: None 2022-09-03T06:18:08.067 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/a 2022-09-03T06:18:08.068 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gkJhC3v9C9EVGNN6Mr22VFrnjqo= 2022-09-03T06:18:08.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gkJhC3v9C9EVGNN6Mr22VFrnjqo=', 'Content-Length': '0'} 2022-09-03T06:18:08.070 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000041cda7c8c24f638c-006312f1a0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.071 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:08.072 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:08.072 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:08.073 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:08.073 DEBUG:boto:Method: GET 2022-09-03T06:18:08.074 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:08.074 DEBUG:boto:Data: 2022-09-03T06:18:08.075 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.075 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.076 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.077 DEBUG:boto:Params: {} 2022-09-03T06:18:08.078 DEBUG:boto:Token: None 2022-09-03T06:18:08.079 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/ 2022-09-03T06:18:08.080 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914= 2022-09-03T06:18:08.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914=', 'Content-Length': '0'} 2022-09-03T06:18:08.085 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c4ec48c7558e3900-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.087 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:08.088 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:08.089 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:08.091 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:08.092 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:08.092 DEBUG:boto:Method: GET 2022-09-03T06:18:08.093 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:08.094 DEBUG:boto:Data: 2022-09-03T06:18:08.095 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.095 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:08.096 DEBUG:boto:Port: 8000 2022-09-03T06:18:08.097 DEBUG:boto:Params: {} 2022-09-03T06:18:08.097 DEBUG:boto:Token: None 2022-09-03T06:18:08.098 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/ 2022-09-03T06:18:08.099 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914= 2022-09-03T06:18:08.099 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914=', 'Content-Length': '0'} 2022-09-03T06:18:08.102 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002922db5dba54ef04-006312f1a0-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.142 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:08.144 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:18:08.146 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:08.147 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:08.148 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.149 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.153 DEBUG:boto:Method: GET 2022-09-03T06:18:08.154 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.156 DEBUG:boto:Data: 2022-09-03T06:18:08.158 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.159 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.160 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.162 DEBUG:boto:Params: {} 2022-09-03T06:18:08.164 DEBUG:boto:Token: None 2022-09-03T06:18:08.167 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.170 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dCszjz4E54xfGI10PpTo0vvsHd8= 2022-09-03T06:18:08.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dCszjz4E54xfGI10PpTo0vvsHd8=', 'Content-Length': '0'} 2022-09-03T06:18:08.177 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006e102c4397550f26-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.178 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:08.183 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:08.184 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.185 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.186 DEBUG:boto:Method: GET 2022-09-03T06:18:08.187 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.189 DEBUG:boto:Data: 2022-09-03T06:18:08.190 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.191 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:08.192 DEBUG:boto:Port: 8000 2022-09-03T06:18:08.194 DEBUG:boto:Params: {} 2022-09-03T06:18:08.195 DEBUG:boto:Token: None 2022-09-03T06:18:08.198 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.199 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dCszjz4E54xfGI10PpTo0vvsHd8= 2022-09-03T06:18:08.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dCszjz4E54xfGI10PpTo0vvsHd8=', 'Content-Length': '0'} 2022-09-03T06:18:08.203 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000092f4894a1cf2efab-006312f1a0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.206 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:08.207 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:08.208 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.209 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.210 DEBUG:boto:Method: HEAD 2022-09-03T06:18:08.211 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.212 DEBUG:boto:Data: 2022-09-03T06:18:08.215 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.217 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.218 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.219 DEBUG:boto:Params: {} 2022-09-03T06:18:08.221 DEBUG:boto:Token: None 2022-09-03T06:18:08.223 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.225 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V7Ch328Wss61aJZnht5wS7rs/EE= 2022-09-03T06:18:08.227 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V7Ch328Wss61aJZnht5wS7rs/EE=', 'Content-Length': '0'} 2022-09-03T06:18:08.230 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b4db21bd699e7ffa-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.231 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:08.231 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:08.232 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.232 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.233 DEBUG:boto:Method: HEAD 2022-09-03T06:18:08.233 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.234 DEBUG:boto:Data: 2022-09-03T06:18:08.235 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.235 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:08.236 DEBUG:boto:Port: 8000 2022-09-03T06:18:08.236 DEBUG:boto:Params: {} 2022-09-03T06:18:08.237 DEBUG:boto:Token: None 2022-09-03T06:18:08.237 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.238 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V7Ch328Wss61aJZnht5wS7rs/EE= 2022-09-03T06:18:08.239 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V7Ch328Wss61aJZnht5wS7rs/EE=', 'Content-Length': '0'} 2022-09-03T06:18:08.241 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002ec57fd615822921-006312f1a0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.242 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:18:08.242 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:08.243 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:08.243 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.244 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.244 DEBUG:boto:Method: GET 2022-09-03T06:18:08.245 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.245 DEBUG:boto:Data: 2022-09-03T06:18:08.246 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.246 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.247 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.248 DEBUG:boto:Params: {} 2022-09-03T06:18:08.248 DEBUG:boto:Token: None 2022-09-03T06:18:08.249 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.250 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dCszjz4E54xfGI10PpTo0vvsHd8= 2022-09-03T06:18:08.250 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dCszjz4E54xfGI10PpTo0vvsHd8=', 'Content-Length': '0'} 2022-09-03T06:18:08.252 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008d3d59d9b8b8a1c9-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.253 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:08.254 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:08.254 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.255 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.255 DEBUG:boto:Method: GET 2022-09-03T06:18:08.256 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.257 DEBUG:boto:Data: 2022-09-03T06:18:08.257 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.258 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:08.258 DEBUG:boto:Port: 8000 2022-09-03T06:18:08.259 DEBUG:boto:Params: {} 2022-09-03T06:18:08.259 DEBUG:boto:Token: None 2022-09-03T06:18:08.260 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:08.261 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dCszjz4E54xfGI10PpTo0vvsHd8= 2022-09-03T06:18:08.261 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dCszjz4E54xfGI10PpTo0vvsHd8=', 'Content-Length': '0'} 2022-09-03T06:18:08.264 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000075fe702ecef63bdc-006312f1a0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.264 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:08.265 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:08.266 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:08.266 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:08.267 DEBUG:boto:Method: GET 2022-09-03T06:18:08.267 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:08.268 DEBUG:boto:Data: 2022-09-03T06:18:08.268 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.269 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.269 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.270 DEBUG:boto:Params: {} 2022-09-03T06:18:08.270 DEBUG:boto:Token: None 2022-09-03T06:18:08.271 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/ 2022-09-03T06:18:08.272 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914= 2022-09-03T06:18:08.272 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914=', 'Content-Length': '0'} 2022-09-03T06:18:08.275 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007e33f8a21342d02b-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.276 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:08.277 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:08.277 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:08.278 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:08.278 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:08.279 DEBUG:boto:Method: GET 2022-09-03T06:18:08.280 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:08.280 DEBUG:boto:Data: 2022-09-03T06:18:08.281 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.281 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:08.282 DEBUG:boto:Port: 8000 2022-09-03T06:18:08.282 DEBUG:boto:Params: {} 2022-09-03T06:18:08.283 DEBUG:boto:Token: None 2022-09-03T06:18:08.284 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/ 2022-09-03T06:18:08.284 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914= 2022-09-03T06:18:08.285 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914=', 'Content-Length': '0'} 2022-09-03T06:18:08.290 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000020fe232c95769580-006312f1a0-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.291 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:08.293 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:08.294 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:08.295 DEBUG:boto:Method: HEAD 2022-09-03T06:18:08.298 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:08.299 DEBUG:boto:Data: 2022-09-03T06:18:08.300 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.301 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.302 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.303 DEBUG:boto:Params: {} 2022-09-03T06:18:08.305 DEBUG:boto:Token: None 2022-09-03T06:18:08.305 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/b 2022-09-03T06:18:08.306 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HH2A9I3EBffiGgcDntAEJfDCkAM= 2022-09-03T06:18:08.307 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HH2A9I3EBffiGgcDntAEJfDCkAM=', 'Content-Length': '0'} 2022-09-03T06:18:08.309 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000164b85f3176069c2-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.310 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:08.310 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:08.311 DEBUG:boto:Method: HEAD 2022-09-03T06:18:08.311 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:08.312 DEBUG:boto:Data: 2022-09-03T06:18:08.312 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.313 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:08.313 DEBUG:boto:Port: 8000 2022-09-03T06:18:08.315 DEBUG:boto:Params: {} 2022-09-03T06:18:08.317 DEBUG:boto:Token: None 2022-09-03T06:18:08.318 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/b 2022-09-03T06:18:08.321 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HH2A9I3EBffiGgcDntAEJfDCkAM= 2022-09-03T06:18:08.322 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HH2A9I3EBffiGgcDntAEJfDCkAM=', 'Content-Length': '0'} 2022-09-03T06:18:08.325 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d8644c27dff9ea03-006312f1a0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.326 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:18:08.327 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:08.329 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:08.330 DEBUG:boto:Method: GET 2022-09-03T06:18:08.330 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:08.331 DEBUG:boto:Data: 2022-09-03T06:18:08.332 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.332 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.333 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.333 DEBUG:boto:Params: {} 2022-09-03T06:18:08.334 DEBUG:boto:Token: None 2022-09-03T06:18:08.335 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/b 2022-09-03T06:18:08.336 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jzAUnh89WVOhbPHpv9a9M6nG6eQ= 2022-09-03T06:18:08.336 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jzAUnh89WVOhbPHpv9a9M6nG6eQ=', 'Content-Length': '0'} 2022-09-03T06:18:08.338 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d5a0511ec9ddbb93-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.340 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:08.340 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:08.349 DEBUG:boto:Method: GET 2022-09-03T06:18:08.351 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:08.352 DEBUG:boto:Data: 2022-09-03T06:18:08.353 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.353 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:08.356 DEBUG:boto:Port: 8000 2022-09-03T06:18:08.357 DEBUG:boto:Params: {} 2022-09-03T06:18:08.358 DEBUG:boto:Token: None 2022-09-03T06:18:08.359 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/b 2022-09-03T06:18:08.364 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jzAUnh89WVOhbPHpv9a9M6nG6eQ= 2022-09-03T06:18:08.365 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jzAUnh89WVOhbPHpv9a9M6nG6eQ=', 'Content-Length': '0'} 2022-09-03T06:18:08.368 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007d1d245d940a65c0-006312f1a0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.369 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:08.370 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:08.371 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:08.374 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:08.374 DEBUG:boto:Method: GET 2022-09-03T06:18:08.375 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:08.375 DEBUG:boto:Data: 2022-09-03T06:18:08.376 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.376 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.377 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.378 DEBUG:boto:Params: {} 2022-09-03T06:18:08.378 DEBUG:boto:Token: None 2022-09-03T06:18:08.379 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/ 2022-09-03T06:18:08.380 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914= 2022-09-03T06:18:08.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914=', 'Content-Length': '0'} 2022-09-03T06:18:08.383 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a0f91c81819b343d-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.426 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:08.428 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:08.428 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:08.429 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:08.430 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:08.430 DEBUG:boto:Method: GET 2022-09-03T06:18:08.431 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:08.431 DEBUG:boto:Data: 2022-09-03T06:18:08.432 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.432 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:08.433 DEBUG:boto:Port: 8000 2022-09-03T06:18:08.433 DEBUG:boto:Params: {} 2022-09-03T06:18:08.434 DEBUG:boto:Token: None 2022-09-03T06:18:08.435 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/ 2022-09-03T06:18:08.435 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914= 2022-09-03T06:18:08.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914=', 'Content-Length': '0'} 2022-09-03T06:18:08.439 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b1c9e10c3286f505-006312f1a0-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.442 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:08.443 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:18:08.444 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:08.445 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:08.447 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.447 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.448 DEBUG:boto:Method: GET 2022-09-03T06:18:08.448 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.449 DEBUG:boto:Data: 2022-09-03T06:18:08.449 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.450 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.451 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.451 DEBUG:boto:Params: {} 2022-09-03T06:18:08.452 DEBUG:boto:Token: None 2022-09-03T06:18:08.452 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.453 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:svXIu2bnFmNjbsFNNK1DnB9MXEM= 2022-09-03T06:18:08.454 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:svXIu2bnFmNjbsFNNK1DnB9MXEM=', 'Content-Length': '0'} 2022-09-03T06:18:08.456 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005594aea8d25c563b-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.457 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:08.458 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:08.458 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.459 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.460 DEBUG:boto:Method: GET 2022-09-03T06:18:08.460 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.461 DEBUG:boto:Data: 2022-09-03T06:18:08.461 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.462 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:08.463 DEBUG:boto:Port: 8000 2022-09-03T06:18:08.464 DEBUG:boto:Params: {} 2022-09-03T06:18:08.464 DEBUG:boto:Token: None 2022-09-03T06:18:08.465 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.466 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:svXIu2bnFmNjbsFNNK1DnB9MXEM= 2022-09-03T06:18:08.466 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:svXIu2bnFmNjbsFNNK1DnB9MXEM=', 'Content-Length': '0'} 2022-09-03T06:18:08.469 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b1166d5eaa75bba6-006312f1a0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.470 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:08.470 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:08.471 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.471 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.472 DEBUG:boto:Method: HEAD 2022-09-03T06:18:08.473 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.474 DEBUG:boto:Data: 2022-09-03T06:18:08.474 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.475 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.475 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.476 DEBUG:boto:Params: {} 2022-09-03T06:18:08.476 DEBUG:boto:Token: None 2022-09-03T06:18:08.477 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.478 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TT0DVQh9PkrPTDmSxRykGSrpqkI= 2022-09-03T06:18:08.478 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TT0DVQh9PkrPTDmSxRykGSrpqkI=', 'Content-Length': '0'} 2022-09-03T06:18:08.481 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000016fd0f7ea414d0f1-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.482 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:08.482 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:08.483 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.483 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.484 DEBUG:boto:Method: HEAD 2022-09-03T06:18:08.484 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.485 DEBUG:boto:Data: 2022-09-03T06:18:08.485 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.486 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:08.486 DEBUG:boto:Port: 8000 2022-09-03T06:18:08.487 DEBUG:boto:Params: {} 2022-09-03T06:18:08.487 DEBUG:boto:Token: None 2022-09-03T06:18:08.488 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.489 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TT0DVQh9PkrPTDmSxRykGSrpqkI= 2022-09-03T06:18:08.489 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TT0DVQh9PkrPTDmSxRykGSrpqkI=', 'Content-Length': '0'} 2022-09-03T06:18:08.492 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ef76db480a950293-006312f1a0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.493 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:18:08.493 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:08.494 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:08.495 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.495 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.496 DEBUG:boto:Method: GET 2022-09-03T06:18:08.496 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.497 DEBUG:boto:Data: 2022-09-03T06:18:08.498 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.498 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.499 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.500 DEBUG:boto:Params: {} 2022-09-03T06:18:08.500 DEBUG:boto:Token: None 2022-09-03T06:18:08.501 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.502 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:svXIu2bnFmNjbsFNNK1DnB9MXEM= 2022-09-03T06:18:08.502 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:svXIu2bnFmNjbsFNNK1DnB9MXEM=', 'Content-Length': '0'} 2022-09-03T06:18:08.505 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000becee41b068b150-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.506 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:08.506 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:08.507 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.508 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.508 DEBUG:boto:Method: GET 2022-09-03T06:18:08.509 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.509 DEBUG:boto:Data: 2022-09-03T06:18:08.510 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.511 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:08.511 DEBUG:boto:Port: 8000 2022-09-03T06:18:08.512 DEBUG:boto:Params: {} 2022-09-03T06:18:08.512 DEBUG:boto:Token: None 2022-09-03T06:18:08.513 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:08.514 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:svXIu2bnFmNjbsFNNK1DnB9MXEM= 2022-09-03T06:18:08.514 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:svXIu2bnFmNjbsFNNK1DnB9MXEM=', 'Content-Length': '0'} 2022-09-03T06:18:08.516 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ebdf268ddf3f9320-006312f1a0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.517 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:08.518 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:08.519 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:08.519 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:08.520 DEBUG:boto:Method: GET 2022-09-03T06:18:08.520 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:08.521 DEBUG:boto:Data: 2022-09-03T06:18:08.521 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.522 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.522 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.523 DEBUG:boto:Params: {} 2022-09-03T06:18:08.523 DEBUG:boto:Token: None 2022-09-03T06:18:08.524 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/ 2022-09-03T06:18:08.525 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914= 2022-09-03T06:18:08.525 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914=', 'Content-Length': '0'} 2022-09-03T06:18:08.529 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b15431155776f3cc-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.531 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:08.532 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:08.533 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:08.535 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:08.536 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:08.537 DEBUG:boto:Method: GET 2022-09-03T06:18:08.538 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:08.539 DEBUG:boto:Data: 2022-09-03T06:18:08.539 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.540 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:08.541 DEBUG:boto:Port: 8000 2022-09-03T06:18:08.541 DEBUG:boto:Params: {} 2022-09-03T06:18:08.542 DEBUG:boto:Token: None 2022-09-03T06:18:08.542 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/ 2022-09-03T06:18:08.543 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914= 2022-09-03T06:18:08.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914=', 'Content-Length': '0'} 2022-09-03T06:18:08.547 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004a1a834efcdcb471-006312f1a0-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.549 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:08.550 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:08.551 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:08.552 DEBUG:boto:Method: HEAD 2022-09-03T06:18:08.553 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:08.554 DEBUG:boto:Data: 2022-09-03T06:18:08.555 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.556 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.557 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.557 DEBUG:boto:Params: {} 2022-09-03T06:18:08.558 DEBUG:boto:Token: None 2022-09-03T06:18:08.559 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/c 2022-09-03T06:18:08.561 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ea3AlBL73pprGvelaQxFhs8xOAc= 2022-09-03T06:18:08.562 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ea3AlBL73pprGvelaQxFhs8xOAc=', 'Content-Length': '0'} 2022-09-03T06:18:08.564 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eaea1018ade8f3ad-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.566 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:08.567 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:08.568 DEBUG:boto:Method: HEAD 2022-09-03T06:18:08.568 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:08.569 DEBUG:boto:Data: 2022-09-03T06:18:08.569 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.570 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:08.570 DEBUG:boto:Port: 8000 2022-09-03T06:18:08.571 DEBUG:boto:Params: {} 2022-09-03T06:18:08.572 DEBUG:boto:Token: None 2022-09-03T06:18:08.573 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/c 2022-09-03T06:18:08.575 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ea3AlBL73pprGvelaQxFhs8xOAc= 2022-09-03T06:18:08.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ea3AlBL73pprGvelaQxFhs8xOAc=', 'Content-Length': '0'} 2022-09-03T06:18:08.580 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a911d79d007fe1b-006312f1a0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.581 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:18:08.582 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:08.583 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:08.585 DEBUG:boto:Method: GET 2022-09-03T06:18:08.586 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:08.587 DEBUG:boto:Data: 2022-09-03T06:18:08.588 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.589 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.590 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.591 DEBUG:boto:Params: {} 2022-09-03T06:18:08.592 DEBUG:boto:Token: None 2022-09-03T06:18:08.593 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/c 2022-09-03T06:18:08.594 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H9M48scNks3vq6m6z0waSU/rmpc= 2022-09-03T06:18:08.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H9M48scNks3vq6m6z0waSU/rmpc=', 'Content-Length': '0'} 2022-09-03T06:18:08.599 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007fdb24e11939d913-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.601 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:08.602 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:08.603 DEBUG:boto:Method: GET 2022-09-03T06:18:08.604 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:08.605 DEBUG:boto:Data: 2022-09-03T06:18:08.607 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.607 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:08.608 DEBUG:boto:Port: 8000 2022-09-03T06:18:08.609 DEBUG:boto:Params: {} 2022-09-03T06:18:08.609 DEBUG:boto:Token: None 2022-09-03T06:18:08.610 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/c 2022-09-03T06:18:08.610 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H9M48scNks3vq6m6z0waSU/rmpc= 2022-09-03T06:18:08.611 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H9M48scNks3vq6m6z0waSU/rmpc=', 'Content-Length': '0'} 2022-09-03T06:18:08.613 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c8e00ac6b64d64d-006312f1a0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.614 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:08.615 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:08.616 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:08.616 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:08.617 DEBUG:boto:Method: GET 2022-09-03T06:18:08.618 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:08.618 DEBUG:boto:Data: 2022-09-03T06:18:08.619 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.619 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.620 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.620 DEBUG:boto:Params: {} 2022-09-03T06:18:08.621 DEBUG:boto:Token: None 2022-09-03T06:18:08.621 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/ 2022-09-03T06:18:08.622 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914= 2022-09-03T06:18:08.623 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914=', 'Content-Length': '0'} 2022-09-03T06:18:08.631 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009c4213797b133971-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.632 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:08.632 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:08.633 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:08.634 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:08.635 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:08.635 DEBUG:boto:Method: GET 2022-09-03T06:18:08.636 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:08.637 DEBUG:boto:Data: 2022-09-03T06:18:08.637 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.638 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:08.639 DEBUG:boto:Port: 8000 2022-09-03T06:18:08.640 DEBUG:boto:Params: {} 2022-09-03T06:18:08.640 DEBUG:boto:Token: None 2022-09-03T06:18:08.641 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/ 2022-09-03T06:18:08.641 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914= 2022-09-03T06:18:08.642 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914=', 'Content-Length': '0'} 2022-09-03T06:18:08.645 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004b12a47cbd909516-006312f1a0-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.647 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:08.648 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:18:08.649 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:08.651 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:08.651 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.652 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.652 DEBUG:boto:Method: GET 2022-09-03T06:18:08.653 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.653 DEBUG:boto:Data: 2022-09-03T06:18:08.654 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.656 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.656 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.657 DEBUG:boto:Params: {} 2022-09-03T06:18:08.658 DEBUG:boto:Token: None 2022-09-03T06:18:08.659 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.660 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TFlmR4K+oir8Cehw/hxZyAaR25I= 2022-09-03T06:18:08.662 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TFlmR4K+oir8Cehw/hxZyAaR25I=', 'Content-Length': '0'} 2022-09-03T06:18:08.664 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000027ae71d9bcafe15e-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.706 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:08.707 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:08.708 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.709 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.710 DEBUG:boto:Method: GET 2022-09-03T06:18:08.711 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.712 DEBUG:boto:Data: 2022-09-03T06:18:08.713 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.714 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:08.715 DEBUG:boto:Port: 8000 2022-09-03T06:18:08.716 DEBUG:boto:Params: {} 2022-09-03T06:18:08.717 DEBUG:boto:Token: None 2022-09-03T06:18:08.718 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.720 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TFlmR4K+oir8Cehw/hxZyAaR25I= 2022-09-03T06:18:08.721 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TFlmR4K+oir8Cehw/hxZyAaR25I=', 'Content-Length': '0'} 2022-09-03T06:18:08.724 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000014a72db884b8f7aa-006312f1a0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.726 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:08.726 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:08.727 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.727 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.728 DEBUG:boto:Method: HEAD 2022-09-03T06:18:08.729 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.729 DEBUG:boto:Data: 2022-09-03T06:18:08.730 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.732 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.733 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.734 DEBUG:boto:Params: {} 2022-09-03T06:18:08.736 DEBUG:boto:Token: None 2022-09-03T06:18:08.737 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.738 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SJKyXs9Xn0UUOdJHbyFOIXvkFzI= 2022-09-03T06:18:08.739 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SJKyXs9Xn0UUOdJHbyFOIXvkFzI=', 'Content-Length': '0'} 2022-09-03T06:18:08.742 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f454504dc57aed4d-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.743 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:08.744 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:08.744 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.745 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.745 DEBUG:boto:Method: HEAD 2022-09-03T06:18:08.746 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.747 DEBUG:boto:Data: 2022-09-03T06:18:08.747 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.748 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:08.748 DEBUG:boto:Port: 8000 2022-09-03T06:18:08.749 DEBUG:boto:Params: {} 2022-09-03T06:18:08.751 DEBUG:boto:Token: None 2022-09-03T06:18:08.751 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.752 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SJKyXs9Xn0UUOdJHbyFOIXvkFzI= 2022-09-03T06:18:08.753 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SJKyXs9Xn0UUOdJHbyFOIXvkFzI=', 'Content-Length': '0'} 2022-09-03T06:18:08.755 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000044f06e6275fd9d48-006312f1a0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.755 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:18:08.756 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:08.757 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:08.757 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.758 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.758 DEBUG:boto:Method: GET 2022-09-03T06:18:08.759 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.759 DEBUG:boto:Data: 2022-09-03T06:18:08.760 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.761 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.762 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.762 DEBUG:boto:Params: {} 2022-09-03T06:18:08.763 DEBUG:boto:Token: None 2022-09-03T06:18:08.764 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.764 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TFlmR4K+oir8Cehw/hxZyAaR25I= 2022-09-03T06:18:08.765 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TFlmR4K+oir8Cehw/hxZyAaR25I=', 'Content-Length': '0'} 2022-09-03T06:18:08.767 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b8cf79a19740012-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.768 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:08.769 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:08.769 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.770 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.770 DEBUG:boto:Method: GET 2022-09-03T06:18:08.772 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.772 DEBUG:boto:Data: 2022-09-03T06:18:08.773 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.773 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:08.774 DEBUG:boto:Port: 8000 2022-09-03T06:18:08.774 DEBUG:boto:Params: {} 2022-09-03T06:18:08.775 DEBUG:boto:Token: None 2022-09-03T06:18:08.776 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:08.776 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TFlmR4K+oir8Cehw/hxZyAaR25I= 2022-09-03T06:18:08.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TFlmR4K+oir8Cehw/hxZyAaR25I=', 'Content-Length': '0'} 2022-09-03T06:18:08.779 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001148143f6132ac3b-006312f1a0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.826 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:08.827 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:08.828 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:08.829 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:08.829 DEBUG:boto:Method: GET 2022-09-03T06:18:08.830 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:08.836 DEBUG:boto:Data: 2022-09-03T06:18:08.837 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.838 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.839 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.839 DEBUG:boto:Params: {} 2022-09-03T06:18:08.840 DEBUG:boto:Token: None 2022-09-03T06:18:08.841 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/ 2022-09-03T06:18:08.841 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914= 2022-09-03T06:18:08.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914=', 'Content-Length': '0'} 2022-09-03T06:18:08.845 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000939dcda1ebd9b9c9-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.846 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:08.848 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:08.849 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:08.850 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:08.850 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:08.851 DEBUG:boto:Method: GET 2022-09-03T06:18:08.851 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:08.852 DEBUG:boto:Data: 2022-09-03T06:18:08.852 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.853 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:08.854 DEBUG:boto:Port: 8000 2022-09-03T06:18:08.854 DEBUG:boto:Params: {} 2022-09-03T06:18:08.855 DEBUG:boto:Token: None 2022-09-03T06:18:08.855 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/ 2022-09-03T06:18:08.856 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914= 2022-09-03T06:18:08.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914=', 'Content-Length': '0'} 2022-09-03T06:18:08.860 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000081a8f0c66e0d3bc5-006312f1a0-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.861 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:08.861 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:08.862 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:08.863 DEBUG:boto:Method: HEAD 2022-09-03T06:18:08.863 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:08.864 DEBUG:boto:Data: 2022-09-03T06:18:08.864 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.864 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.865 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.865 DEBUG:boto:Params: {} 2022-09-03T06:18:08.866 DEBUG:boto:Token: None 2022-09-03T06:18:08.867 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/d 2022-09-03T06:18:08.867 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CA/oNVZFyguH2fdmyEtW05Gon0o= 2022-09-03T06:18:08.869 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CA/oNVZFyguH2fdmyEtW05Gon0o=', 'Content-Length': '0'} 2022-09-03T06:18:08.872 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d206e90abf43661a-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.873 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:08.873 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:08.874 DEBUG:boto:Method: HEAD 2022-09-03T06:18:08.874 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:08.875 DEBUG:boto:Data: 2022-09-03T06:18:08.876 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.877 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:08.880 DEBUG:boto:Port: 8000 2022-09-03T06:18:08.880 DEBUG:boto:Params: {} 2022-09-03T06:18:08.881 DEBUG:boto:Token: None 2022-09-03T06:18:08.882 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/d 2022-09-03T06:18:08.883 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CA/oNVZFyguH2fdmyEtW05Gon0o= 2022-09-03T06:18:08.883 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CA/oNVZFyguH2fdmyEtW05Gon0o=', 'Content-Length': '0'} 2022-09-03T06:18:08.885 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c28d76cda0b2d7f3-006312f1a0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.886 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:18:08.887 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:08.888 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:08.888 DEBUG:boto:Method: GET 2022-09-03T06:18:08.892 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:08.892 DEBUG:boto:Data: 2022-09-03T06:18:08.893 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.894 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.894 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.895 DEBUG:boto:Params: {} 2022-09-03T06:18:08.895 DEBUG:boto:Token: None 2022-09-03T06:18:08.896 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/d 2022-09-03T06:18:08.897 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i6UBeCG6T9geqbEfKWffg3HEDUo= 2022-09-03T06:18:08.899 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i6UBeCG6T9geqbEfKWffg3HEDUo=', 'Content-Length': '0'} 2022-09-03T06:18:08.910 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b169fcb784f7b6b6-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.911 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:08.912 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:08.912 DEBUG:boto:Method: GET 2022-09-03T06:18:08.913 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:08.913 DEBUG:boto:Data: 2022-09-03T06:18:08.914 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.915 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:08.916 DEBUG:boto:Port: 8000 2022-09-03T06:18:08.925 DEBUG:boto:Params: {} 2022-09-03T06:18:08.926 DEBUG:boto:Token: None 2022-09-03T06:18:08.928 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/d 2022-09-03T06:18:08.930 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i6UBeCG6T9geqbEfKWffg3HEDUo= 2022-09-03T06:18:08.931 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i6UBeCG6T9geqbEfKWffg3HEDUo=', 'Content-Length': '0'} 2022-09-03T06:18:08.934 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007063b7a203ccb65a-006312f1a0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:08.936 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:08.937 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:08.938 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:08.940 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:08.941 DEBUG:boto:Method: GET 2022-09-03T06:18:08.942 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:08.943 DEBUG:boto:Data: 2022-09-03T06:18:08.944 DEBUG:boto:Headers: {} 2022-09-03T06:18:08.945 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:08.947 DEBUG:boto:Port: 8001 2022-09-03T06:18:08.948 DEBUG:boto:Params: {} 2022-09-03T06:18:08.950 DEBUG:boto:Token: None 2022-09-03T06:18:08.951 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:08 GMT /dsgvcy-71/ 2022-09-03T06:18:08.952 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914= 2022-09-03T06:18:08.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8wcz9CsAeaAQQCgMiMeOawdt914=', 'Content-Length': '0'} 2022-09-03T06:18:08.962 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004337e2f9874e942a-006312f1a0-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.003 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:09.004 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:09.005 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:09.005 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:09.006 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:09.007 DEBUG:boto:Method: GET 2022-09-03T06:18:09.008 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:09.009 DEBUG:boto:Data: 2022-09-03T06:18:09.010 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.011 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:09.012 DEBUG:boto:Port: 8000 2022-09-03T06:18:09.013 DEBUG:boto:Params: {} 2022-09-03T06:18:09.014 DEBUG:boto:Token: None 2022-09-03T06:18:09.015 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/ 2022-09-03T06:18:09.016 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N1GgT+QJIegvwTDS+dXorthgagw= 2022-09-03T06:18:09.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N1GgT+QJIegvwTDS+dXorthgagw=', 'Content-Length': '0'} 2022-09-03T06:18:09.034 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007c5d66fa3eb3bbb8-006312f1a1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.036 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:09.037 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:18:09.038 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:09.039 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:09.040 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.041 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.042 DEBUG:boto:Method: GET 2022-09-03T06:18:09.043 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.044 DEBUG:boto:Data: 2022-09-03T06:18:09.045 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.046 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:09.047 DEBUG:boto:Port: 8001 2022-09-03T06:18:09.048 DEBUG:boto:Params: {} 2022-09-03T06:18:09.050 DEBUG:boto:Token: None 2022-09-03T06:18:09.052 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.053 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:klTQjgyA1bCgw/wwF1b4ajssKa4= 2022-09-03T06:18:09.055 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:klTQjgyA1bCgw/wwF1b4ajssKa4=', 'Content-Length': '0'} 2022-09-03T06:18:09.063 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c1422ae23915620c-006312f1a1-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.065 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:09.074 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:09.075 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.076 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.077 DEBUG:boto:Method: GET 2022-09-03T06:18:09.081 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.082 DEBUG:boto:Data: 2022-09-03T06:18:09.083 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.085 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:09.086 DEBUG:boto:Port: 8000 2022-09-03T06:18:09.087 DEBUG:boto:Params: {} 2022-09-03T06:18:09.088 DEBUG:boto:Token: None 2022-09-03T06:18:09.090 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.091 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:klTQjgyA1bCgw/wwF1b4ajssKa4= 2022-09-03T06:18:09.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:klTQjgyA1bCgw/wwF1b4ajssKa4=', 'Content-Length': '0'} 2022-09-03T06:18:09.118 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000042c1e5ec84b68be-006312f1a1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.119 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:09.120 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:09.121 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.122 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.123 DEBUG:boto:Method: HEAD 2022-09-03T06:18:09.124 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.125 DEBUG:boto:Data: 2022-09-03T06:18:09.126 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.136 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:09.137 DEBUG:boto:Port: 8001 2022-09-03T06:18:09.147 DEBUG:boto:Params: {} 2022-09-03T06:18:09.150 DEBUG:boto:Token: None 2022-09-03T06:18:09.151 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.153 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eh1REwiX5ENADtDeEcGYg7gNh2U= 2022-09-03T06:18:09.154 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eh1REwiX5ENADtDeEcGYg7gNh2U=', 'Content-Length': '0'} 2022-09-03T06:18:09.168 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aff6e7f2b612e547-006312f1a1-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.169 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:09.170 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:09.170 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.171 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.175 DEBUG:boto:Method: HEAD 2022-09-03T06:18:09.182 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.183 DEBUG:boto:Data: 2022-09-03T06:18:09.184 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.184 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:09.185 DEBUG:boto:Port: 8000 2022-09-03T06:18:09.185 DEBUG:boto:Params: {} 2022-09-03T06:18:09.186 DEBUG:boto:Token: None 2022-09-03T06:18:09.187 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.187 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eh1REwiX5ENADtDeEcGYg7gNh2U= 2022-09-03T06:18:09.189 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eh1REwiX5ENADtDeEcGYg7gNh2U=', 'Content-Length': '0'} 2022-09-03T06:18:09.197 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a757626616a897a-006312f1a1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.198 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:18:09.199 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:09.199 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:09.200 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.201 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.201 DEBUG:boto:Method: GET 2022-09-03T06:18:09.202 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.202 DEBUG:boto:Data: 2022-09-03T06:18:09.203 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.204 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:09.204 DEBUG:boto:Port: 8001 2022-09-03T06:18:09.205 DEBUG:boto:Params: {} 2022-09-03T06:18:09.205 DEBUG:boto:Token: None 2022-09-03T06:18:09.206 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.207 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:klTQjgyA1bCgw/wwF1b4ajssKa4= 2022-09-03T06:18:09.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:klTQjgyA1bCgw/wwF1b4ajssKa4=', 'Content-Length': '0'} 2022-09-03T06:18:09.221 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009cf5824e00a80116-006312f1a1-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.222 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:09.223 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:09.223 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.224 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.224 DEBUG:boto:Method: GET 2022-09-03T06:18:09.225 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.225 DEBUG:boto:Data: 2022-09-03T06:18:09.226 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.227 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:09.227 DEBUG:boto:Port: 8000 2022-09-03T06:18:09.228 DEBUG:boto:Params: {} 2022-09-03T06:18:09.229 DEBUG:boto:Token: None 2022-09-03T06:18:09.231 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:09.232 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:klTQjgyA1bCgw/wwF1b4ajssKa4= 2022-09-03T06:18:09.233 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:klTQjgyA1bCgw/wwF1b4ajssKa4=', 'Content-Length': '0'} 2022-09-03T06:18:09.240 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ef3f7ffc26dcba62-006312f1a1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.242 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:09.252 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:09.253 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:09.254 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:09.255 DEBUG:boto:Method: GET 2022-09-03T06:18:09.256 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:09.257 DEBUG:boto:Data: 2022-09-03T06:18:09.259 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.261 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:09.267 DEBUG:boto:Port: 8001 2022-09-03T06:18:09.268 DEBUG:boto:Params: {} 2022-09-03T06:18:09.269 DEBUG:boto:Token: None 2022-09-03T06:18:09.269 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/ 2022-09-03T06:18:09.272 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N1GgT+QJIegvwTDS+dXorthgagw= 2022-09-03T06:18:09.275 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N1GgT+QJIegvwTDS+dXorthgagw=', 'Content-Length': '0'} 2022-09-03T06:18:09.293 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005356b8b8f7996c8e-006312f1a1-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.294 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:09.295 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:09.297 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:09.299 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:09.301 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:09.302 DEBUG:boto:Method: GET 2022-09-03T06:18:09.304 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:09.305 DEBUG:boto:Data: 2022-09-03T06:18:09.306 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.307 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:09.309 DEBUG:boto:Port: 8000 2022-09-03T06:18:09.310 DEBUG:boto:Params: {} 2022-09-03T06:18:09.312 DEBUG:boto:Token: None 2022-09-03T06:18:09.312 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/ 2022-09-03T06:18:09.313 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N1GgT+QJIegvwTDS+dXorthgagw= 2022-09-03T06:18:09.318 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N1GgT+QJIegvwTDS+dXorthgagw=', 'Content-Length': '0'} 2022-09-03T06:18:09.323 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aa5c91609d292354-006312f1a1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.324 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:09.326 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:09.327 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:09.328 DEBUG:boto:Method: HEAD 2022-09-03T06:18:09.329 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:09.330 DEBUG:boto:Data: 2022-09-03T06:18:09.331 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.332 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:09.334 DEBUG:boto:Port: 8001 2022-09-03T06:18:09.334 DEBUG:boto:Params: {} 2022-09-03T06:18:09.335 DEBUG:boto:Token: None 2022-09-03T06:18:09.337 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/e 2022-09-03T06:18:09.338 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iNfkVnHsaYnETqygOGrJBmqGC1k= 2022-09-03T06:18:09.339 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iNfkVnHsaYnETqygOGrJBmqGC1k=', 'Content-Length': '0'} 2022-09-03T06:18:09.355 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005dc856211ffd70fa-006312f1a1-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.357 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:09.358 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:09.359 DEBUG:boto:Method: HEAD 2022-09-03T06:18:09.361 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:09.363 DEBUG:boto:Data: 2022-09-03T06:18:09.364 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.365 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:09.366 DEBUG:boto:Port: 8000 2022-09-03T06:18:09.367 DEBUG:boto:Params: {} 2022-09-03T06:18:09.368 DEBUG:boto:Token: None 2022-09-03T06:18:09.370 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/e 2022-09-03T06:18:09.371 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iNfkVnHsaYnETqygOGrJBmqGC1k= 2022-09-03T06:18:09.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iNfkVnHsaYnETqygOGrJBmqGC1k=', 'Content-Length': '0'} 2022-09-03T06:18:09.377 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d0e23d6326d2ce5d-006312f1a1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.378 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:18:09.379 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:09.380 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:09.380 DEBUG:boto:Method: GET 2022-09-03T06:18:09.387 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:09.388 DEBUG:boto:Data: 2022-09-03T06:18:09.389 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.391 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:09.392 DEBUG:boto:Port: 8001 2022-09-03T06:18:09.393 DEBUG:boto:Params: {} 2022-09-03T06:18:09.394 DEBUG:boto:Token: None 2022-09-03T06:18:09.395 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/e 2022-09-03T06:18:09.396 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wZfgEf4APtMcqO5UWsYH4S4zoMI= 2022-09-03T06:18:09.398 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wZfgEf4APtMcqO5UWsYH4S4zoMI=', 'Content-Length': '0'} 2022-09-03T06:18:09.401 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000062d788ae5d4039ec-006312f1a1-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.403 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:09.404 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:09.405 DEBUG:boto:Method: GET 2022-09-03T06:18:09.406 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:09.407 DEBUG:boto:Data: 2022-09-03T06:18:09.408 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.409 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:09.410 DEBUG:boto:Port: 8000 2022-09-03T06:18:09.411 DEBUG:boto:Params: {} 2022-09-03T06:18:09.412 DEBUG:boto:Token: None 2022-09-03T06:18:09.414 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/e 2022-09-03T06:18:09.415 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wZfgEf4APtMcqO5UWsYH4S4zoMI= 2022-09-03T06:18:09.416 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wZfgEf4APtMcqO5UWsYH4S4zoMI=', 'Content-Length': '0'} 2022-09-03T06:18:09.419 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000011552da498fab27a-006312f1a1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.420 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:09.421 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:09.421 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:09.422 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:09.423 DEBUG:boto:Method: GET 2022-09-03T06:18:09.424 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:09.425 DEBUG:boto:Data: 2022-09-03T06:18:09.426 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.426 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:09.427 DEBUG:boto:Port: 8001 2022-09-03T06:18:09.427 DEBUG:boto:Params: {} 2022-09-03T06:18:09.428 DEBUG:boto:Token: None 2022-09-03T06:18:09.429 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/ 2022-09-03T06:18:09.430 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N1GgT+QJIegvwTDS+dXorthgagw= 2022-09-03T06:18:09.431 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N1GgT+QJIegvwTDS+dXorthgagw=', 'Content-Length': '0'} 2022-09-03T06:18:09.434 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b8188ebdc82746f2-006312f1a1-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.435 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:09.436 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:09.438 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:09.438 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:09.441 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:09.441 DEBUG:boto:Method: GET 2022-09-03T06:18:09.442 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:09.443 DEBUG:boto:Data: 2022-09-03T06:18:09.443 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.444 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:09.444 DEBUG:boto:Port: 8000 2022-09-03T06:18:09.445 DEBUG:boto:Params: {} 2022-09-03T06:18:09.446 DEBUG:boto:Token: None 2022-09-03T06:18:09.447 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/ 2022-09-03T06:18:09.447 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N1GgT+QJIegvwTDS+dXorthgagw= 2022-09-03T06:18:09.448 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N1GgT+QJIegvwTDS+dXorthgagw=', 'Content-Length': '0'} 2022-09-03T06:18:09.450 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d68006d40483e1ca-006312f1a1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.498 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:09.500 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:18:09.501 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:09.502 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:09.503 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.504 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.505 DEBUG:boto:Method: GET 2022-09-03T06:18:09.506 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.507 DEBUG:boto:Data: 2022-09-03T06:18:09.508 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.509 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:09.510 DEBUG:boto:Port: 8001 2022-09-03T06:18:09.517 DEBUG:boto:Params: {} 2022-09-03T06:18:09.518 DEBUG:boto:Token: None 2022-09-03T06:18:09.519 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.522 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VGTwS8cXFo2aOyh0aNXv8aj/lTI= 2022-09-03T06:18:09.524 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VGTwS8cXFo2aOyh0aNXv8aj/lTI=', 'Content-Length': '0'} 2022-09-03T06:18:09.527 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000df6afe50d5146c12-006312f1a1-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.528 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:09.529 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:09.530 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.531 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.532 DEBUG:boto:Method: GET 2022-09-03T06:18:09.532 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.533 DEBUG:boto:Data: 2022-09-03T06:18:09.533 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.538 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:09.539 DEBUG:boto:Port: 8000 2022-09-03T06:18:09.540 DEBUG:boto:Params: {} 2022-09-03T06:18:09.542 DEBUG:boto:Token: None 2022-09-03T06:18:09.543 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.545 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VGTwS8cXFo2aOyh0aNXv8aj/lTI= 2022-09-03T06:18:09.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VGTwS8cXFo2aOyh0aNXv8aj/lTI=', 'Content-Length': '0'} 2022-09-03T06:18:09.549 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000811677d74fd3f919-006312f1a1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.550 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:09.552 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:09.553 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.555 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.555 DEBUG:boto:Method: HEAD 2022-09-03T06:18:09.556 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.557 DEBUG:boto:Data: 2022-09-03T06:18:09.557 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.558 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:09.559 DEBUG:boto:Port: 8001 2022-09-03T06:18:09.559 DEBUG:boto:Params: {} 2022-09-03T06:18:09.560 DEBUG:boto:Token: None 2022-09-03T06:18:09.560 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.561 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BxOx3yq1Xqda4ikIjtz7Qcl7Sl8= 2022-09-03T06:18:09.562 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BxOx3yq1Xqda4ikIjtz7Qcl7Sl8=', 'Content-Length': '0'} 2022-09-03T06:18:09.569 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000077f6184e9b94f57b-006312f1a1-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.571 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:09.571 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:09.572 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.574 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.575 DEBUG:boto:Method: HEAD 2022-09-03T06:18:09.576 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.577 DEBUG:boto:Data: 2022-09-03T06:18:09.577 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.578 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:09.579 DEBUG:boto:Port: 8000 2022-09-03T06:18:09.580 DEBUG:boto:Params: {} 2022-09-03T06:18:09.582 DEBUG:boto:Token: None 2022-09-03T06:18:09.583 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.584 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BxOx3yq1Xqda4ikIjtz7Qcl7Sl8= 2022-09-03T06:18:09.585 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BxOx3yq1Xqda4ikIjtz7Qcl7Sl8=', 'Content-Length': '0'} 2022-09-03T06:18:09.588 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c574de8de1c25aa-006312f1a1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.589 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:18:09.590 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:09.591 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:09.593 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.594 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.595 DEBUG:boto:Method: GET 2022-09-03T06:18:09.596 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.597 DEBUG:boto:Data: 2022-09-03T06:18:09.598 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.599 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:09.600 DEBUG:boto:Port: 8001 2022-09-03T06:18:09.601 DEBUG:boto:Params: {} 2022-09-03T06:18:09.603 DEBUG:boto:Token: None 2022-09-03T06:18:09.604 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.605 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VGTwS8cXFo2aOyh0aNXv8aj/lTI= 2022-09-03T06:18:09.607 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VGTwS8cXFo2aOyh0aNXv8aj/lTI=', 'Content-Length': '0'} 2022-09-03T06:18:09.610 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f21c72aa8aea2f43-006312f1a1-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.612 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:09.614 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:09.615 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.615 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.616 DEBUG:boto:Method: GET 2022-09-03T06:18:09.617 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.618 DEBUG:boto:Data: 2022-09-03T06:18:09.619 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.620 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:09.621 DEBUG:boto:Port: 8000 2022-09-03T06:18:09.621 DEBUG:boto:Params: {} 2022-09-03T06:18:09.622 DEBUG:boto:Token: None 2022-09-03T06:18:09.623 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:09.625 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VGTwS8cXFo2aOyh0aNXv8aj/lTI= 2022-09-03T06:18:09.626 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VGTwS8cXFo2aOyh0aNXv8aj/lTI=', 'Content-Length': '0'} 2022-09-03T06:18:09.628 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000968bc57a0a76e93a-006312f1a1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.629 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:09.630 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:09.630 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:09.631 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:09.631 DEBUG:boto:Method: GET 2022-09-03T06:18:09.632 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:09.632 DEBUG:boto:Data: 2022-09-03T06:18:09.633 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.634 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:09.634 DEBUG:boto:Port: 8001 2022-09-03T06:18:09.635 DEBUG:boto:Params: {} 2022-09-03T06:18:09.636 DEBUG:boto:Token: None 2022-09-03T06:18:09.636 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/ 2022-09-03T06:18:09.637 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N1GgT+QJIegvwTDS+dXorthgagw= 2022-09-03T06:18:09.637 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N1GgT+QJIegvwTDS+dXorthgagw=', 'Content-Length': '0'} 2022-09-03T06:18:09.640 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007990a7b729a5267b-006312f1a1-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.682 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:09.684 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:09.685 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:09.686 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:09.686 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:09.687 DEBUG:boto:Method: GET 2022-09-03T06:18:09.688 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:09.689 DEBUG:boto:Data: 2022-09-03T06:18:09.690 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.691 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:09.692 DEBUG:boto:Port: 8000 2022-09-03T06:18:09.693 DEBUG:boto:Params: {} 2022-09-03T06:18:09.693 DEBUG:boto:Token: None 2022-09-03T06:18:09.694 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/ 2022-09-03T06:18:09.695 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N1GgT+QJIegvwTDS+dXorthgagw= 2022-09-03T06:18:09.695 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N1GgT+QJIegvwTDS+dXorthgagw=', 'Content-Length': '0'} 2022-09-03T06:18:09.698 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000013f548e0c9328a29-006312f1a1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.699 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:09.700 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:09.700 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:09.701 DEBUG:boto:Method: HEAD 2022-09-03T06:18:09.701 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:09.702 DEBUG:boto:Data: 2022-09-03T06:18:09.702 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.703 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:09.703 DEBUG:boto:Port: 8001 2022-09-03T06:18:09.704 DEBUG:boto:Params: {} 2022-09-03T06:18:09.704 DEBUG:boto:Token: None 2022-09-03T06:18:09.705 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/f 2022-09-03T06:18:09.706 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y0HX3kHxo89erRvJclHZ3nqlvhQ= 2022-09-03T06:18:09.706 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y0HX3kHxo89erRvJclHZ3nqlvhQ=', 'Content-Length': '0'} 2022-09-03T06:18:09.709 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f2bcdde6d03c608-006312f1a1-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.710 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:09.711 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:09.712 DEBUG:boto:Method: HEAD 2022-09-03T06:18:09.713 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:09.714 DEBUG:boto:Data: 2022-09-03T06:18:09.715 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.715 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:09.716 DEBUG:boto:Port: 8000 2022-09-03T06:18:09.717 DEBUG:boto:Params: {} 2022-09-03T06:18:09.719 DEBUG:boto:Token: None 2022-09-03T06:18:09.720 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/f 2022-09-03T06:18:09.720 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y0HX3kHxo89erRvJclHZ3nqlvhQ= 2022-09-03T06:18:09.721 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y0HX3kHxo89erRvJclHZ3nqlvhQ=', 'Content-Length': '0'} 2022-09-03T06:18:09.724 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ecff73d0ba1ef100-006312f1a1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.726 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:18:09.727 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:09.727 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:09.728 DEBUG:boto:Method: GET 2022-09-03T06:18:09.728 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:09.730 DEBUG:boto:Data: 2022-09-03T06:18:09.731 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.732 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:09.739 DEBUG:boto:Port: 8001 2022-09-03T06:18:09.740 DEBUG:boto:Params: {} 2022-09-03T06:18:09.740 DEBUG:boto:Token: None 2022-09-03T06:18:09.741 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/f 2022-09-03T06:18:09.742 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OchO3hghE+tJFzzfqBbJ6hh17UU= 2022-09-03T06:18:09.742 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OchO3hghE+tJFzzfqBbJ6hh17UU=', 'Content-Length': '0'} 2022-09-03T06:18:09.749 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e3f086747858371e-006312f1a1-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.750 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:09.752 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:09.754 DEBUG:boto:Method: GET 2022-09-03T06:18:09.756 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:09.757 DEBUG:boto:Data: 2022-09-03T06:18:09.758 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.759 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:09.760 DEBUG:boto:Port: 8000 2022-09-03T06:18:09.761 DEBUG:boto:Params: {} 2022-09-03T06:18:09.763 DEBUG:boto:Token: None 2022-09-03T06:18:09.764 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/f 2022-09-03T06:18:09.766 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OchO3hghE+tJFzzfqBbJ6hh17UU= 2022-09-03T06:18:09.766 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OchO3hghE+tJFzzfqBbJ6hh17UU=', 'Content-Length': '0'} 2022-09-03T06:18:09.773 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000054b616d02a50659c-006312f1a1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.775 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:09.776 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:09.777 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:09.778 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:09.779 DEBUG:boto:Method: GET 2022-09-03T06:18:09.780 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:09.784 DEBUG:boto:Data: 2022-09-03T06:18:09.785 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.787 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:09.788 DEBUG:boto:Port: 8001 2022-09-03T06:18:09.789 DEBUG:boto:Params: {} 2022-09-03T06:18:09.791 DEBUG:boto:Token: None 2022-09-03T06:18:09.792 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/ 2022-09-03T06:18:09.793 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N1GgT+QJIegvwTDS+dXorthgagw= 2022-09-03T06:18:09.794 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N1GgT+QJIegvwTDS+dXorthgagw=', 'Content-Length': '0'} 2022-09-03T06:18:09.806 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a36150db868466b9-006312f1a1-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.807 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:09.809 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:09.811 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:09.812 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:09.813 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:09.814 DEBUG:boto:Method: GET 2022-09-03T06:18:09.816 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:09.817 DEBUG:boto:Data: 2022-09-03T06:18:09.817 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.818 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:09.820 DEBUG:boto:Port: 8000 2022-09-03T06:18:09.821 DEBUG:boto:Params: {} 2022-09-03T06:18:09.823 DEBUG:boto:Token: None 2022-09-03T06:18:09.824 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/ 2022-09-03T06:18:09.825 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N1GgT+QJIegvwTDS+dXorthgagw= 2022-09-03T06:18:09.827 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N1GgT+QJIegvwTDS+dXorthgagw=', 'Content-Length': '0'} 2022-09-03T06:18:09.830 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000072f720f5b14ee5a1-006312f1a1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.832 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:09.834 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:18:09.835 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:09.836 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:09.837 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.838 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.839 DEBUG:boto:Method: GET 2022-09-03T06:18:09.840 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.841 DEBUG:boto:Data: 2022-09-03T06:18:09.843 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.844 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:09.845 DEBUG:boto:Port: 8001 2022-09-03T06:18:09.846 DEBUG:boto:Params: {} 2022-09-03T06:18:09.847 DEBUG:boto:Token: None 2022-09-03T06:18:09.848 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.850 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zYo42bJFT4Rn0uRottkBT+VRMvo= 2022-09-03T06:18:09.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zYo42bJFT4Rn0uRottkBT+VRMvo=', 'Content-Length': '0'} 2022-09-03T06:18:09.857 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001e7a4225df30f1e5-006312f1a1-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.858 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:09.859 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:09.860 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.860 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.861 DEBUG:boto:Method: GET 2022-09-03T06:18:09.861 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.862 DEBUG:boto:Data: 2022-09-03T06:18:09.863 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.863 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:09.865 DEBUG:boto:Port: 8000 2022-09-03T06:18:09.865 DEBUG:boto:Params: {} 2022-09-03T06:18:09.866 DEBUG:boto:Token: None 2022-09-03T06:18:09.867 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.867 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zYo42bJFT4Rn0uRottkBT+VRMvo= 2022-09-03T06:18:09.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zYo42bJFT4Rn0uRottkBT+VRMvo=', 'Content-Length': '0'} 2022-09-03T06:18:09.870 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc26df770799efb4-006312f1a1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.871 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:09.872 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:09.872 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.873 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.873 DEBUG:boto:Method: HEAD 2022-09-03T06:18:09.874 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.874 DEBUG:boto:Data: 2022-09-03T06:18:09.875 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.876 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:09.877 DEBUG:boto:Port: 8001 2022-09-03T06:18:09.877 DEBUG:boto:Params: {} 2022-09-03T06:18:09.878 DEBUG:boto:Token: None 2022-09-03T06:18:09.879 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.879 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B7DpAtsTDxaaczwhTD1/I15eHLk= 2022-09-03T06:18:09.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B7DpAtsTDxaaczwhTD1/I15eHLk=', 'Content-Length': '0'} 2022-09-03T06:18:09.882 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000479eb0a068427ff4-006312f1a1-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.883 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:09.884 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:09.884 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.885 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.885 DEBUG:boto:Method: HEAD 2022-09-03T06:18:09.886 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.887 DEBUG:boto:Data: 2022-09-03T06:18:09.887 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.888 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:09.888 DEBUG:boto:Port: 8000 2022-09-03T06:18:09.889 DEBUG:boto:Params: {} 2022-09-03T06:18:09.891 DEBUG:boto:Token: None 2022-09-03T06:18:09.891 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.892 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B7DpAtsTDxaaczwhTD1/I15eHLk= 2022-09-03T06:18:09.893 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B7DpAtsTDxaaczwhTD1/I15eHLk=', 'Content-Length': '0'} 2022-09-03T06:18:09.895 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005d90c98522faf7a9-006312f1a1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.896 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:18:09.897 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:09.897 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:09.898 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.899 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.899 DEBUG:boto:Method: GET 2022-09-03T06:18:09.900 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.900 DEBUG:boto:Data: 2022-09-03T06:18:09.901 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.901 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:09.902 DEBUG:boto:Port: 8001 2022-09-03T06:18:09.903 DEBUG:boto:Params: {} 2022-09-03T06:18:09.903 DEBUG:boto:Token: None 2022-09-03T06:18:09.904 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.905 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zYo42bJFT4Rn0uRottkBT+VRMvo= 2022-09-03T06:18:09.905 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zYo42bJFT4Rn0uRottkBT+VRMvo=', 'Content-Length': '0'} 2022-09-03T06:18:09.907 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000017e6ec4713a8fbb0-006312f1a1-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.954 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:09.956 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:09.956 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.958 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.959 DEBUG:boto:Method: GET 2022-09-03T06:18:09.960 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.961 DEBUG:boto:Data: 2022-09-03T06:18:09.961 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.962 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:09.963 DEBUG:boto:Port: 8000 2022-09-03T06:18:09.964 DEBUG:boto:Params: {} 2022-09-03T06:18:09.965 DEBUG:boto:Token: None 2022-09-03T06:18:09.966 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:09.967 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zYo42bJFT4Rn0uRottkBT+VRMvo= 2022-09-03T06:18:09.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zYo42bJFT4Rn0uRottkBT+VRMvo=', 'Content-Length': '0'} 2022-09-03T06:18:09.971 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e758c95e4eb90e9b-006312f1a1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.972 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:09.973 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:09.973 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:09.974 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:09.974 DEBUG:boto:Method: GET 2022-09-03T06:18:09.975 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:09.975 DEBUG:boto:Data: 2022-09-03T06:18:09.976 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.977 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:09.977 DEBUG:boto:Port: 8001 2022-09-03T06:18:09.978 DEBUG:boto:Params: {} 2022-09-03T06:18:09.978 DEBUG:boto:Token: None 2022-09-03T06:18:09.979 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/ 2022-09-03T06:18:09.980 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N1GgT+QJIegvwTDS+dXorthgagw= 2022-09-03T06:18:09.980 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N1GgT+QJIegvwTDS+dXorthgagw=', 'Content-Length': '0'} 2022-09-03T06:18:09.983 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000205acb0dd0273ee2-006312f1a1-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.984 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:09.985 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:09.986 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:09.986 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:09.987 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:09.987 DEBUG:boto:Method: GET 2022-09-03T06:18:09.988 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:09.988 DEBUG:boto:Data: 2022-09-03T06:18:09.989 DEBUG:boto:Headers: {} 2022-09-03T06:18:09.989 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:09.990 DEBUG:boto:Port: 8000 2022-09-03T06:18:09.990 DEBUG:boto:Params: {} 2022-09-03T06:18:09.991 DEBUG:boto:Token: None 2022-09-03T06:18:09.992 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:09 GMT /dsgvcy-71/ 2022-09-03T06:18:09.993 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N1GgT+QJIegvwTDS+dXorthgagw= 2022-09-03T06:18:09.993 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N1GgT+QJIegvwTDS+dXorthgagw=', 'Content-Length': '0'} 2022-09-03T06:18:09.996 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000074cfc29f94a7a0aa-006312f1a1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:09.996 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:09.997 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:09.998 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:09.998 DEBUG:boto:Method: HEAD 2022-09-03T06:18:09.999 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:09.999 DEBUG:boto:Data: 2022-09-03T06:18:10.000 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.000 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.001 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.001 DEBUG:boto:Params: {} 2022-09-03T06:18:10.002 DEBUG:boto:Token: None 2022-09-03T06:18:10.003 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/g 2022-09-03T06:18:10.003 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3Cm2/N5mUC6J6H8Y00HkWBAiXm8= 2022-09-03T06:18:10.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3Cm2/N5mUC6J6H8Y00HkWBAiXm8=', 'Content-Length': '0'} 2022-09-03T06:18:10.006 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e80c64ef6b3f7be3-006312f1a2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.007 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:10.007 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:10.008 DEBUG:boto:Method: HEAD 2022-09-03T06:18:10.008 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:10.009 DEBUG:boto:Data: 2022-09-03T06:18:10.009 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.010 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:10.010 DEBUG:boto:Port: 8000 2022-09-03T06:18:10.011 DEBUG:boto:Params: {} 2022-09-03T06:18:10.011 DEBUG:boto:Token: None 2022-09-03T06:18:10.012 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/g 2022-09-03T06:18:10.013 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3Cm2/N5mUC6J6H8Y00HkWBAiXm8= 2022-09-03T06:18:10.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3Cm2/N5mUC6J6H8Y00HkWBAiXm8=', 'Content-Length': '0'} 2022-09-03T06:18:10.016 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000011559f0dd324777a-006312f1a2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.017 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:18:10.018 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:10.019 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:10.019 DEBUG:boto:Method: GET 2022-09-03T06:18:10.020 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:10.021 DEBUG:boto:Data: 2022-09-03T06:18:10.022 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.022 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.023 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.023 DEBUG:boto:Params: {} 2022-09-03T06:18:10.024 DEBUG:boto:Token: None 2022-09-03T06:18:10.025 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/g 2022-09-03T06:18:10.025 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3jW7hUpArNNEfVh5tTsKvRDfJhY= 2022-09-03T06:18:10.026 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3jW7hUpArNNEfVh5tTsKvRDfJhY=', 'Content-Length': '0'} 2022-09-03T06:18:10.028 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000072655f52f1180cd-006312f1a2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.029 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:10.030 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:10.030 DEBUG:boto:Method: GET 2022-09-03T06:18:10.031 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:10.031 DEBUG:boto:Data: 2022-09-03T06:18:10.032 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.032 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:10.033 DEBUG:boto:Port: 8000 2022-09-03T06:18:10.033 DEBUG:boto:Params: {} 2022-09-03T06:18:10.034 DEBUG:boto:Token: None 2022-09-03T06:18:10.035 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/g 2022-09-03T06:18:10.035 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3jW7hUpArNNEfVh5tTsKvRDfJhY= 2022-09-03T06:18:10.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3jW7hUpArNNEfVh5tTsKvRDfJhY=', 'Content-Length': '0'} 2022-09-03T06:18:10.038 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000266c495484d41aa2-006312f1a2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.039 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:10.040 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:10.040 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:10.041 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:10.041 DEBUG:boto:Method: GET 2022-09-03T06:18:10.042 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:10.042 DEBUG:boto:Data: 2022-09-03T06:18:10.043 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.043 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.044 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.044 DEBUG:boto:Params: {} 2022-09-03T06:18:10.045 DEBUG:boto:Token: None 2022-09-03T06:18:10.046 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/ 2022-09-03T06:18:10.047 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v+soWKMfHuDnjQFZmTfDnaHGUR4= 2022-09-03T06:18:10.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v+soWKMfHuDnjQFZmTfDnaHGUR4=', 'Content-Length': '0'} 2022-09-03T06:18:10.051 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003b9988644b2d9afb-006312f1a2-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.052 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:10.053 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:10.053 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:10.054 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:10.054 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:10.055 DEBUG:boto:Method: GET 2022-09-03T06:18:10.055 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:10.056 DEBUG:boto:Data: 2022-09-03T06:18:10.056 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.057 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:10.058 DEBUG:boto:Port: 8000 2022-09-03T06:18:10.058 DEBUG:boto:Params: {} 2022-09-03T06:18:10.059 DEBUG:boto:Token: None 2022-09-03T06:18:10.059 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/ 2022-09-03T06:18:10.060 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v+soWKMfHuDnjQFZmTfDnaHGUR4= 2022-09-03T06:18:10.061 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v+soWKMfHuDnjQFZmTfDnaHGUR4=', 'Content-Length': '0'} 2022-09-03T06:18:10.063 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009655ab164b525dc0-006312f1a2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.064 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:10.065 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:18:10.065 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:10.066 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:10.066 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.067 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.067 DEBUG:boto:Method: GET 2022-09-03T06:18:10.068 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.069 DEBUG:boto:Data: 2022-09-03T06:18:10.069 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.070 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.070 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.071 DEBUG:boto:Params: {} 2022-09-03T06:18:10.071 DEBUG:boto:Token: None 2022-09-03T06:18:10.072 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.073 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CbrXs+8p1USjT5FLLQf55hkxE4I= 2022-09-03T06:18:10.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CbrXs+8p1USjT5FLLQf55hkxE4I=', 'Content-Length': '0'} 2022-09-03T06:18:10.082 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000156ac75a54e14747-006312f1a2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.118 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:10.119 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:10.120 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.121 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.123 DEBUG:boto:Method: GET 2022-09-03T06:18:10.124 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.125 DEBUG:boto:Data: 2022-09-03T06:18:10.126 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.126 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:10.127 DEBUG:boto:Port: 8000 2022-09-03T06:18:10.127 DEBUG:boto:Params: {} 2022-09-03T06:18:10.128 DEBUG:boto:Token: None 2022-09-03T06:18:10.129 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.129 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CbrXs+8p1USjT5FLLQf55hkxE4I= 2022-09-03T06:18:10.130 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CbrXs+8p1USjT5FLLQf55hkxE4I=', 'Content-Length': '0'} 2022-09-03T06:18:10.132 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f6a2902e7aed2ed-006312f1a2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.133 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:10.134 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:10.134 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.135 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.135 DEBUG:boto:Method: HEAD 2022-09-03T06:18:10.136 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.136 DEBUG:boto:Data: 2022-09-03T06:18:10.137 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.137 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.138 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.138 DEBUG:boto:Params: {} 2022-09-03T06:18:10.139 DEBUG:boto:Token: None 2022-09-03T06:18:10.140 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.140 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IhiWl/3gxpZOaNnN4nFX4Y7Sg+Y= 2022-09-03T06:18:10.141 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IhiWl/3gxpZOaNnN4nFX4Y7Sg+Y=', 'Content-Length': '0'} 2022-09-03T06:18:10.143 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a7005d91f5ff7ab1-006312f1a2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.144 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:10.144 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:10.145 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.145 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.146 DEBUG:boto:Method: HEAD 2022-09-03T06:18:10.146 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.147 DEBUG:boto:Data: 2022-09-03T06:18:10.147 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.148 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:10.149 DEBUG:boto:Port: 8000 2022-09-03T06:18:10.149 DEBUG:boto:Params: {} 2022-09-03T06:18:10.150 DEBUG:boto:Token: None 2022-09-03T06:18:10.150 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.151 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IhiWl/3gxpZOaNnN4nFX4Y7Sg+Y= 2022-09-03T06:18:10.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IhiWl/3gxpZOaNnN4nFX4Y7Sg+Y=', 'Content-Length': '0'} 2022-09-03T06:18:10.155 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000512d0c2d0441e5de-006312f1a2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.155 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:18:10.156 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:10.156 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:10.157 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.157 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.158 DEBUG:boto:Method: GET 2022-09-03T06:18:10.159 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.159 DEBUG:boto:Data: 2022-09-03T06:18:10.160 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.160 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.161 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.161 DEBUG:boto:Params: {} 2022-09-03T06:18:10.162 DEBUG:boto:Token: None 2022-09-03T06:18:10.163 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.164 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CbrXs+8p1USjT5FLLQf55hkxE4I= 2022-09-03T06:18:10.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CbrXs+8p1USjT5FLLQf55hkxE4I=', 'Content-Length': '0'} 2022-09-03T06:18:10.167 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b23e1983cd04b02d-006312f1a2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.168 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:10.169 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:10.169 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.170 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.170 DEBUG:boto:Method: GET 2022-09-03T06:18:10.171 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.171 DEBUG:boto:Data: 2022-09-03T06:18:10.172 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.173 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:10.173 DEBUG:boto:Port: 8000 2022-09-03T06:18:10.174 DEBUG:boto:Params: {} 2022-09-03T06:18:10.175 DEBUG:boto:Token: None 2022-09-03T06:18:10.176 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:10.176 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CbrXs+8p1USjT5FLLQf55hkxE4I= 2022-09-03T06:18:10.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CbrXs+8p1USjT5FLLQf55hkxE4I=', 'Content-Length': '0'} 2022-09-03T06:18:10.183 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a75bee266b771c3-006312f1a2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.230 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:10.232 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:10.234 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:10.235 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:10.237 DEBUG:boto:Method: GET 2022-09-03T06:18:10.238 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:10.239 DEBUG:boto:Data: 2022-09-03T06:18:10.240 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.241 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.243 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.245 DEBUG:boto:Params: {} 2022-09-03T06:18:10.246 DEBUG:boto:Token: None 2022-09-03T06:18:10.246 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/ 2022-09-03T06:18:10.247 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v+soWKMfHuDnjQFZmTfDnaHGUR4= 2022-09-03T06:18:10.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v+soWKMfHuDnjQFZmTfDnaHGUR4=', 'Content-Length': '0'} 2022-09-03T06:18:10.250 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000a971521df7cce57-006312f1a2-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.252 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:10.254 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:10.257 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:10.258 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:10.259 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:10.260 DEBUG:boto:Method: GET 2022-09-03T06:18:10.260 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:10.261 DEBUG:boto:Data: 2022-09-03T06:18:10.261 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.262 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:10.262 DEBUG:boto:Port: 8000 2022-09-03T06:18:10.263 DEBUG:boto:Params: {} 2022-09-03T06:18:10.264 DEBUG:boto:Token: None 2022-09-03T06:18:10.264 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/ 2022-09-03T06:18:10.265 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v+soWKMfHuDnjQFZmTfDnaHGUR4= 2022-09-03T06:18:10.266 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v+soWKMfHuDnjQFZmTfDnaHGUR4=', 'Content-Length': '0'} 2022-09-03T06:18:10.269 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003345a6c076776d08-006312f1a2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.271 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:10.271 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:10.272 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:10.273 DEBUG:boto:Method: HEAD 2022-09-03T06:18:10.274 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:10.276 DEBUG:boto:Data: 2022-09-03T06:18:10.277 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.277 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.279 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.281 DEBUG:boto:Params: {} 2022-09-03T06:18:10.282 DEBUG:boto:Token: None 2022-09-03T06:18:10.283 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/h 2022-09-03T06:18:10.285 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+c363h8pFeMo9EFxzk9pFu+r5Bk= 2022-09-03T06:18:10.286 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+c363h8pFeMo9EFxzk9pFu+r5Bk=', 'Content-Length': '0'} 2022-09-03T06:18:10.288 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f67a8df5f615ba11-006312f1a2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.289 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:10.290 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:10.291 DEBUG:boto:Method: HEAD 2022-09-03T06:18:10.291 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:10.292 DEBUG:boto:Data: 2022-09-03T06:18:10.292 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.293 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:10.293 DEBUG:boto:Port: 8000 2022-09-03T06:18:10.294 DEBUG:boto:Params: {} 2022-09-03T06:18:10.295 DEBUG:boto:Token: None 2022-09-03T06:18:10.295 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/h 2022-09-03T06:18:10.296 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+c363h8pFeMo9EFxzk9pFu+r5Bk= 2022-09-03T06:18:10.297 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+c363h8pFeMo9EFxzk9pFu+r5Bk=', 'Content-Length': '0'} 2022-09-03T06:18:10.299 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000097564c9de8296672-006312f1a2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.299 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:18:10.300 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:10.301 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:10.302 DEBUG:boto:Method: GET 2022-09-03T06:18:10.303 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:10.303 DEBUG:boto:Data: 2022-09-03T06:18:10.304 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.305 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.305 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.306 DEBUG:boto:Params: {} 2022-09-03T06:18:10.306 DEBUG:boto:Token: None 2022-09-03T06:18:10.307 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/h 2022-09-03T06:18:10.308 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oGveX+o+ccrYE2jNKycxIdYpBNM= 2022-09-03T06:18:10.309 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oGveX+o+ccrYE2jNKycxIdYpBNM=', 'Content-Length': '0'} 2022-09-03T06:18:10.311 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f60b0492428a185c-006312f1a2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.358 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:10.360 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:10.360 DEBUG:boto:Method: GET 2022-09-03T06:18:10.361 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:10.361 DEBUG:boto:Data: 2022-09-03T06:18:10.362 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.363 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:10.364 DEBUG:boto:Port: 8000 2022-09-03T06:18:10.364 DEBUG:boto:Params: {} 2022-09-03T06:18:10.365 DEBUG:boto:Token: None 2022-09-03T06:18:10.365 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/h 2022-09-03T06:18:10.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oGveX+o+ccrYE2jNKycxIdYpBNM= 2022-09-03T06:18:10.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oGveX+o+ccrYE2jNKycxIdYpBNM=', 'Content-Length': '0'} 2022-09-03T06:18:10.369 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a3d2c2ac349d687-006312f1a2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.371 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:10.372 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:10.373 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:10.377 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:10.378 DEBUG:boto:Method: GET 2022-09-03T06:18:10.379 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:10.380 DEBUG:boto:Data: 2022-09-03T06:18:10.381 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.382 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.383 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.384 DEBUG:boto:Params: {} 2022-09-03T06:18:10.385 DEBUG:boto:Token: None 2022-09-03T06:18:10.386 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/ 2022-09-03T06:18:10.388 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v+soWKMfHuDnjQFZmTfDnaHGUR4= 2022-09-03T06:18:10.389 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v+soWKMfHuDnjQFZmTfDnaHGUR4=', 'Content-Length': '0'} 2022-09-03T06:18:10.393 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000626bb30f8fc45e2d-006312f1a2-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.394 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:10.395 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:10.396 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:10.397 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:10.398 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:10.399 DEBUG:boto:Method: GET 2022-09-03T06:18:10.400 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:10.402 DEBUG:boto:Data: 2022-09-03T06:18:10.403 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.404 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:10.404 DEBUG:boto:Port: 8000 2022-09-03T06:18:10.405 DEBUG:boto:Params: {} 2022-09-03T06:18:10.407 DEBUG:boto:Token: None 2022-09-03T06:18:10.408 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/ 2022-09-03T06:18:10.409 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v+soWKMfHuDnjQFZmTfDnaHGUR4= 2022-09-03T06:18:10.410 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v+soWKMfHuDnjQFZmTfDnaHGUR4=', 'Content-Length': '0'} 2022-09-03T06:18:10.418 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007e50fc6cf4968851-006312f1a2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.420 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:10.421 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-71 zones={test-zone3, test-zone1} 2022-09-03T06:18:10.422 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-71 zones={test-zone3, test-zone2} 2022-09-03T06:18:10.424 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:10.424 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:10.425 DEBUG:boto:Method: HEAD 2022-09-03T06:18:10.426 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:18:10.427 DEBUG:boto:Data: 2022-09-03T06:18:10.429 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.430 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.431 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.432 DEBUG:boto:Params: {} 2022-09-03T06:18:10.434 DEBUG:boto:Token: None 2022-09-03T06:18:10.435 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/ 2022-09-03T06:18:10.436 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iq7NIfsvBDZ7gduSKl5UWBCoa5E= 2022-09-03T06:18:10.437 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iq7NIfsvBDZ7gduSKl5UWBCoa5E=', 'Content-Length': '0'} 2022-09-03T06:18:10.440 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000058aa51c64915986c-006312f1a2-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.441 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:10.441 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:10.442 DEBUG:boto:Method: HEAD 2022-09-03T06:18:10.443 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:18:10.444 DEBUG:boto:Data: 2022-09-03T06:18:10.444 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.445 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:10.446 DEBUG:boto:Port: 8000 2022-09-03T06:18:10.446 DEBUG:boto:Params: {} 2022-09-03T06:18:10.447 DEBUG:boto:Token: None 2022-09-03T06:18:10.447 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/ 2022-09-03T06:18:10.449 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iq7NIfsvBDZ7gduSKl5UWBCoa5E= 2022-09-03T06:18:10.449 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iq7NIfsvBDZ7gduSKl5UWBCoa5E=', 'Content-Length': '0'} 2022-09-03T06:18:10.453 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001b576f852557eac9-006312f1a2-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.454 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:18:10.455 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:10.455 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:10.456 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:10.457 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:10.458 DEBUG:boto:Method: GET 2022-09-03T06:18:10.459 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:10.461 DEBUG:boto:Data: 2022-09-03T06:18:10.462 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.463 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.464 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.465 DEBUG:boto:Params: {} 2022-09-03T06:18:10.466 DEBUG:boto:Token: None 2022-09-03T06:18:10.467 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/?versions 2022-09-03T06:18:10.467 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jKuElyP6dDEFv7rUvo/QGy840AY= 2022-09-03T06:18:10.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jKuElyP6dDEFv7rUvo/QGy840AY=', 'Content-Length': '0'} 2022-09-03T06:18:10.471 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bba0b1187715883a-006312f1a2-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.472 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:10.473 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:18:10.474 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:18:10.474 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:18:10.475 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:18:10.475 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:18:10.476 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:18:10.476 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:18:10.477 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:18:10.477 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:18:10.478 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:10.478 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:10.479 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:10.479 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:10.480 DEBUG:boto:Method: GET 2022-09-03T06:18:10.480 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:10.481 DEBUG:boto:Data: 2022-09-03T06:18:10.481 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.482 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:10.482 DEBUG:boto:Port: 8000 2022-09-03T06:18:10.483 DEBUG:boto:Params: {} 2022-09-03T06:18:10.483 DEBUG:boto:Token: None 2022-09-03T06:18:10.484 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/?versions 2022-09-03T06:18:10.485 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jKuElyP6dDEFv7rUvo/QGy840AY= 2022-09-03T06:18:10.485 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jKuElyP6dDEFv7rUvo/QGy840AY=', 'Content-Length': '0'} 2022-09-03T06:18:10.488 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e4939fcc9d759335-006312f1a2-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.490 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:10.492 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:18:10.493 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:18:10.494 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:18:10.495 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:18:10.496 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:18:10.497 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:18:10.498 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:18:10.499 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:18:10.500 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:10.500 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:10.501 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:10.502 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:10.503 DEBUG:boto:Method: GET 2022-09-03T06:18:10.503 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:10.504 DEBUG:boto:Data: 2022-09-03T06:18:10.505 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.506 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.506 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.507 DEBUG:boto:Params: {} 2022-09-03T06:18:10.508 DEBUG:boto:Token: None 2022-09-03T06:18:10.508 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/?versions 2022-09-03T06:18:10.509 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jKuElyP6dDEFv7rUvo/QGy840AY= 2022-09-03T06:18:10.510 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jKuElyP6dDEFv7rUvo/QGy840AY=', 'Content-Length': '0'} 2022-09-03T06:18:10.513 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000b51be799a1988e1-006312f1a2-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.514 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:10.516 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:10.517 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:10.518 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:10.519 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:10.520 DEBUG:boto:Method: GET 2022-09-03T06:18:10.521 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:10.521 DEBUG:boto:Data: 2022-09-03T06:18:10.523 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.524 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:10.524 DEBUG:boto:Port: 8000 2022-09-03T06:18:10.525 DEBUG:boto:Params: {} 2022-09-03T06:18:10.526 DEBUG:boto:Token: None 2022-09-03T06:18:10.527 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/?versions 2022-09-03T06:18:10.528 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jKuElyP6dDEFv7rUvo/QGy840AY= 2022-09-03T06:18:10.529 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jKuElyP6dDEFv7rUvo/QGy840AY=', 'Content-Length': '0'} 2022-09-03T06:18:10.533 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000057ca1479c8e5b9e9-006312f1a2-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.534 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:10.536 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:18:10.537 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:10.538 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:10.539 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.540 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.540 DEBUG:boto:Method: GET 2022-09-03T06:18:10.541 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.541 DEBUG:boto:Data: 2022-09-03T06:18:10.542 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.543 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.543 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.544 DEBUG:boto:Params: {} 2022-09-03T06:18:10.544 DEBUG:boto:Token: None 2022-09-03T06:18:10.545 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.546 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VOMkNCU0EJp41RU5UUEzillemJI= 2022-09-03T06:18:10.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VOMkNCU0EJp41RU5UUEzillemJI=', 'Content-Length': '0'} 2022-09-03T06:18:10.548 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000687ab91b803447dc-006312f1a2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.551 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:10.551 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:10.552 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.553 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.553 DEBUG:boto:Method: GET 2022-09-03T06:18:10.554 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.554 DEBUG:boto:Data: 2022-09-03T06:18:10.555 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.555 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:10.556 DEBUG:boto:Port: 8000 2022-09-03T06:18:10.557 DEBUG:boto:Params: {} 2022-09-03T06:18:10.558 DEBUG:boto:Token: None 2022-09-03T06:18:10.559 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.560 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VOMkNCU0EJp41RU5UUEzillemJI= 2022-09-03T06:18:10.561 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VOMkNCU0EJp41RU5UUEzillemJI=', 'Content-Length': '0'} 2022-09-03T06:18:10.564 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000087f7fce234191656-006312f1a2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.567 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:10.568 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:10.568 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.569 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.570 DEBUG:boto:Method: HEAD 2022-09-03T06:18:10.571 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.571 DEBUG:boto:Data: 2022-09-03T06:18:10.572 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.572 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.573 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.573 DEBUG:boto:Params: {} 2022-09-03T06:18:10.574 DEBUG:boto:Token: None 2022-09-03T06:18:10.575 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.575 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hTLqpNdoz/IlHEd/gS43EWxf4Jw= 2022-09-03T06:18:10.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hTLqpNdoz/IlHEd/gS43EWxf4Jw=', 'Content-Length': '0'} 2022-09-03T06:18:10.578 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000910de29b88fb73c7-006312f1a2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.579 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:10.579 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:10.580 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.580 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.581 DEBUG:boto:Method: HEAD 2022-09-03T06:18:10.581 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.582 DEBUG:boto:Data: 2022-09-03T06:18:10.583 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.583 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:10.584 DEBUG:boto:Port: 8000 2022-09-03T06:18:10.584 DEBUG:boto:Params: {} 2022-09-03T06:18:10.585 DEBUG:boto:Token: None 2022-09-03T06:18:10.585 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.586 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hTLqpNdoz/IlHEd/gS43EWxf4Jw= 2022-09-03T06:18:10.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hTLqpNdoz/IlHEd/gS43EWxf4Jw=', 'Content-Length': '0'} 2022-09-03T06:18:10.590 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000be8d08d6c04d63cb-006312f1a2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.591 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:18:10.592 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:10.593 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:10.595 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.595 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.596 DEBUG:boto:Method: GET 2022-09-03T06:18:10.597 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.597 DEBUG:boto:Data: 2022-09-03T06:18:10.598 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.599 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.599 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.600 DEBUG:boto:Params: {} 2022-09-03T06:18:10.600 DEBUG:boto:Token: None 2022-09-03T06:18:10.601 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.601 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VOMkNCU0EJp41RU5UUEzillemJI= 2022-09-03T06:18:10.602 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VOMkNCU0EJp41RU5UUEzillemJI=', 'Content-Length': '0'} 2022-09-03T06:18:10.604 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000035c5e2ad4e33091b-006312f1a2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.605 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:10.606 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:10.606 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.607 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.607 DEBUG:boto:Method: GET 2022-09-03T06:18:10.608 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.608 DEBUG:boto:Data: 2022-09-03T06:18:10.609 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.609 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:10.610 DEBUG:boto:Port: 8000 2022-09-03T06:18:10.610 DEBUG:boto:Params: {} 2022-09-03T06:18:10.611 DEBUG:boto:Token: None 2022-09-03T06:18:10.612 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:10.612 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VOMkNCU0EJp41RU5UUEzillemJI= 2022-09-03T06:18:10.613 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VOMkNCU0EJp41RU5UUEzillemJI=', 'Content-Length': '0'} 2022-09-03T06:18:10.616 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fe05ee6376b7a9f5-006312f1a2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.618 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:10.619 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:10.620 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:10.621 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:10.622 DEBUG:boto:Method: GET 2022-09-03T06:18:10.623 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:10.624 DEBUG:boto:Data: 2022-09-03T06:18:10.625 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.627 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.628 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.629 DEBUG:boto:Params: {} 2022-09-03T06:18:10.630 DEBUG:boto:Token: None 2022-09-03T06:18:10.632 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/ 2022-09-03T06:18:10.633 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v+soWKMfHuDnjQFZmTfDnaHGUR4= 2022-09-03T06:18:10.634 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v+soWKMfHuDnjQFZmTfDnaHGUR4=', 'Content-Length': '0'} 2022-09-03T06:18:10.638 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f3ea4500e30b419e-006312f1a2-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.640 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:10.641 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:10.642 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:10.643 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:10.644 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:10.645 DEBUG:boto:Method: GET 2022-09-03T06:18:10.646 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:10.647 DEBUG:boto:Data: 2022-09-03T06:18:10.648 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.649 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:10.650 DEBUG:boto:Port: 8000 2022-09-03T06:18:10.651 DEBUG:boto:Params: {} 2022-09-03T06:18:10.652 DEBUG:boto:Token: None 2022-09-03T06:18:10.653 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/ 2022-09-03T06:18:10.655 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v+soWKMfHuDnjQFZmTfDnaHGUR4= 2022-09-03T06:18:10.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v+soWKMfHuDnjQFZmTfDnaHGUR4=', 'Content-Length': '0'} 2022-09-03T06:18:10.659 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006a3dcfdad73cdb86-006312f1a2-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.660 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:10.662 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:10.663 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:10.664 DEBUG:boto:Method: HEAD 2022-09-03T06:18:10.665 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:10.666 DEBUG:boto:Data: 2022-09-03T06:18:10.667 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.668 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.668 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.669 DEBUG:boto:Params: {} 2022-09-03T06:18:10.669 DEBUG:boto:Token: None 2022-09-03T06:18:10.670 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/a 2022-09-03T06:18:10.671 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0a+5MgawDubwWA7sqXGi88SKm9E= 2022-09-03T06:18:10.673 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0a+5MgawDubwWA7sqXGi88SKm9E=', 'Content-Length': '0'} 2022-09-03T06:18:10.676 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dde09779107269b3-006312f1a2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.676 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:10.677 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:10.678 DEBUG:boto:Method: HEAD 2022-09-03T06:18:10.679 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:10.679 DEBUG:boto:Data: 2022-09-03T06:18:10.680 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.681 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:10.682 DEBUG:boto:Port: 8000 2022-09-03T06:18:10.684 DEBUG:boto:Params: {} 2022-09-03T06:18:10.685 DEBUG:boto:Token: None 2022-09-03T06:18:10.685 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/a 2022-09-03T06:18:10.686 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0a+5MgawDubwWA7sqXGi88SKm9E= 2022-09-03T06:18:10.687 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0a+5MgawDubwWA7sqXGi88SKm9E=', 'Content-Length': '0'} 2022-09-03T06:18:10.690 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000625e6eee421187f0-006312f1a2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.691 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:18:10.692 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:10.694 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:10.695 DEBUG:boto:Method: GET 2022-09-03T06:18:10.696 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:10.697 DEBUG:boto:Data: 2022-09-03T06:18:10.698 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.699 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.699 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.700 DEBUG:boto:Params: {} 2022-09-03T06:18:10.700 DEBUG:boto:Token: None 2022-09-03T06:18:10.701 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/a 2022-09-03T06:18:10.701 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L63H8L1yBYAU8Ja8DRg9kQ5Gyec= 2022-09-03T06:18:10.702 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L63H8L1yBYAU8Ja8DRg9kQ5Gyec=', 'Content-Length': '0'} 2022-09-03T06:18:10.704 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000227dbe7850d4b47-006312f1a2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.758 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:10.760 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:10.761 DEBUG:boto:Method: GET 2022-09-03T06:18:10.762 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:10.766 DEBUG:boto:Data: 2022-09-03T06:18:10.767 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.769 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:10.770 DEBUG:boto:Port: 8000 2022-09-03T06:18:10.771 DEBUG:boto:Params: {} 2022-09-03T06:18:10.772 DEBUG:boto:Token: None 2022-09-03T06:18:10.773 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/a 2022-09-03T06:18:10.774 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L63H8L1yBYAU8Ja8DRg9kQ5Gyec= 2022-09-03T06:18:10.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L63H8L1yBYAU8Ja8DRg9kQ5Gyec=', 'Content-Length': '0'} 2022-09-03T06:18:10.779 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cdb04948aa78abed-006312f1a2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.780 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:10.780 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:10.781 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:10.782 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:10.783 DEBUG:boto:Method: GET 2022-09-03T06:18:10.784 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:10.784 DEBUG:boto:Data: 2022-09-03T06:18:10.785 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.786 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.787 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.789 DEBUG:boto:Params: {} 2022-09-03T06:18:10.789 DEBUG:boto:Token: None 2022-09-03T06:18:10.793 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/ 2022-09-03T06:18:10.796 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v+soWKMfHuDnjQFZmTfDnaHGUR4= 2022-09-03T06:18:10.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v+soWKMfHuDnjQFZmTfDnaHGUR4=', 'Content-Length': '0'} 2022-09-03T06:18:10.800 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000c550e0c96c6c369-006312f1a2-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.801 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:10.803 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:10.804 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:10.805 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:10.805 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:10.806 DEBUG:boto:Method: GET 2022-09-03T06:18:10.807 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:10.808 DEBUG:boto:Data: 2022-09-03T06:18:10.808 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.809 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:10.810 DEBUG:boto:Port: 8000 2022-09-03T06:18:10.811 DEBUG:boto:Params: {} 2022-09-03T06:18:10.813 DEBUG:boto:Token: None 2022-09-03T06:18:10.814 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/ 2022-09-03T06:18:10.815 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v+soWKMfHuDnjQFZmTfDnaHGUR4= 2022-09-03T06:18:10.816 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v+soWKMfHuDnjQFZmTfDnaHGUR4=', 'Content-Length': '0'} 2022-09-03T06:18:10.818 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004895f92e39dde258-006312f1a2-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.819 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:10.820 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:18:10.821 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:10.821 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:10.822 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.822 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.823 DEBUG:boto:Method: GET 2022-09-03T06:18:10.824 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.825 DEBUG:boto:Data: 2022-09-03T06:18:10.826 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.827 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.829 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.830 DEBUG:boto:Params: {} 2022-09-03T06:18:10.831 DEBUG:boto:Token: None 2022-09-03T06:18:10.832 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.834 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X3+ed1Zm4eeCqEBgctlzEBLHTn4= 2022-09-03T06:18:10.835 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X3+ed1Zm4eeCqEBgctlzEBLHTn4=', 'Content-Length': '0'} 2022-09-03T06:18:10.846 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cd1cbcf17f019530-006312f1a2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.848 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:10.849 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:10.849 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.850 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.851 DEBUG:boto:Method: GET 2022-09-03T06:18:10.851 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.852 DEBUG:boto:Data: 2022-09-03T06:18:10.852 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.853 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:10.854 DEBUG:boto:Port: 8000 2022-09-03T06:18:10.854 DEBUG:boto:Params: {} 2022-09-03T06:18:10.855 DEBUG:boto:Token: None 2022-09-03T06:18:10.856 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.857 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X3+ed1Zm4eeCqEBgctlzEBLHTn4= 2022-09-03T06:18:10.858 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X3+ed1Zm4eeCqEBgctlzEBLHTn4=', 'Content-Length': '0'} 2022-09-03T06:18:10.861 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d306c88478952cbf-006312f1a2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.862 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:10.862 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:10.863 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.864 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.865 DEBUG:boto:Method: HEAD 2022-09-03T06:18:10.866 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.867 DEBUG:boto:Data: 2022-09-03T06:18:10.867 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.868 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.869 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.870 DEBUG:boto:Params: {} 2022-09-03T06:18:10.870 DEBUG:boto:Token: None 2022-09-03T06:18:10.871 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.872 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:u3wlIAdmNe6gDG0M7mMVpP5ZIq0= 2022-09-03T06:18:10.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:u3wlIAdmNe6gDG0M7mMVpP5ZIq0=', 'Content-Length': '0'} 2022-09-03T06:18:10.874 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000044572170574d4711-006312f1a2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.875 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:10.876 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:10.877 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.877 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.878 DEBUG:boto:Method: HEAD 2022-09-03T06:18:10.879 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.879 DEBUG:boto:Data: 2022-09-03T06:18:10.880 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.880 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:10.881 DEBUG:boto:Port: 8000 2022-09-03T06:18:10.881 DEBUG:boto:Params: {} 2022-09-03T06:18:10.882 DEBUG:boto:Token: None 2022-09-03T06:18:10.883 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.884 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:u3wlIAdmNe6gDG0M7mMVpP5ZIq0= 2022-09-03T06:18:10.885 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:u3wlIAdmNe6gDG0M7mMVpP5ZIq0=', 'Content-Length': '0'} 2022-09-03T06:18:10.888 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006df8daf821fdecbb-006312f1a2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.889 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:18:10.890 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:10.891 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:10.892 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.893 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.894 DEBUG:boto:Method: GET 2022-09-03T06:18:10.894 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.895 DEBUG:boto:Data: 2022-09-03T06:18:10.896 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.897 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.897 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.898 DEBUG:boto:Params: {} 2022-09-03T06:18:10.899 DEBUG:boto:Token: None 2022-09-03T06:18:10.900 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.901 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X3+ed1Zm4eeCqEBgctlzEBLHTn4= 2022-09-03T06:18:10.902 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X3+ed1Zm4eeCqEBgctlzEBLHTn4=', 'Content-Length': '0'} 2022-09-03T06:18:10.911 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000406d6aeb1d58719c-006312f1a2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.912 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:10.913 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:10.913 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.914 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.914 DEBUG:boto:Method: GET 2022-09-03T06:18:10.915 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.916 DEBUG:boto:Data: 2022-09-03T06:18:10.916 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.917 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:10.917 DEBUG:boto:Port: 8000 2022-09-03T06:18:10.918 DEBUG:boto:Params: {} 2022-09-03T06:18:10.918 DEBUG:boto:Token: None 2022-09-03T06:18:10.919 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:10.919 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X3+ed1Zm4eeCqEBgctlzEBLHTn4= 2022-09-03T06:18:10.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X3+ed1Zm4eeCqEBgctlzEBLHTn4=', 'Content-Length': '0'} 2022-09-03T06:18:10.923 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009a88fc2444a1fa1b-006312f1a2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.925 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:10.926 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:10.927 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:10.928 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:10.929 DEBUG:boto:Method: GET 2022-09-03T06:18:10.930 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:10.931 DEBUG:boto:Data: 2022-09-03T06:18:10.932 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.933 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:10.934 DEBUG:boto:Port: 8001 2022-09-03T06:18:10.935 DEBUG:boto:Params: {} 2022-09-03T06:18:10.937 DEBUG:boto:Token: None 2022-09-03T06:18:10.938 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:10 GMT /dsgvcy-71/ 2022-09-03T06:18:10.939 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v+soWKMfHuDnjQFZmTfDnaHGUR4= 2022-09-03T06:18:10.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v+soWKMfHuDnjQFZmTfDnaHGUR4=', 'Content-Length': '0'} 2022-09-03T06:18:10.944 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004e9d41b33214edf9-006312f1a2-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:10.986 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:10.988 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:10.989 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:10.991 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:10.992 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:10.994 DEBUG:boto:Method: GET 2022-09-03T06:18:10.995 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:10.996 DEBUG:boto:Data: 2022-09-03T06:18:10.997 DEBUG:boto:Headers: {} 2022-09-03T06:18:10.998 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:10.999 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.000 DEBUG:boto:Params: {} 2022-09-03T06:18:11.002 DEBUG:boto:Token: None 2022-09-03T06:18:11.002 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/ 2022-09-03T06:18:11.003 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA= 2022-09-03T06:18:11.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA=', 'Content-Length': '0'} 2022-09-03T06:18:11.007 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dd2938e4860ad2e4-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.008 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:11.009 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:11.010 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:11.011 DEBUG:boto:Method: HEAD 2022-09-03T06:18:11.012 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:11.013 DEBUG:boto:Data: 2022-09-03T06:18:11.014 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.015 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.016 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.017 DEBUG:boto:Params: {} 2022-09-03T06:18:11.017 DEBUG:boto:Token: None 2022-09-03T06:18:11.018 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/b 2022-09-03T06:18:11.019 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aEExqjScQnaRFPaTNVV2oeVvrPc= 2022-09-03T06:18:11.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aEExqjScQnaRFPaTNVV2oeVvrPc=', 'Content-Length': '0'} 2022-09-03T06:18:11.023 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2bb6bfe327389a7-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.024 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:11.025 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:11.026 DEBUG:boto:Method: HEAD 2022-09-03T06:18:11.027 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:11.028 DEBUG:boto:Data: 2022-09-03T06:18:11.029 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.031 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.032 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.033 DEBUG:boto:Params: {} 2022-09-03T06:18:11.034 DEBUG:boto:Token: None 2022-09-03T06:18:11.035 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/b 2022-09-03T06:18:11.036 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aEExqjScQnaRFPaTNVV2oeVvrPc= 2022-09-03T06:18:11.037 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aEExqjScQnaRFPaTNVV2oeVvrPc=', 'Content-Length': '0'} 2022-09-03T06:18:11.039 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002fb1a45830733da7-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.040 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:18:11.041 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:11.042 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:11.042 DEBUG:boto:Method: GET 2022-09-03T06:18:11.043 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:11.043 DEBUG:boto:Data: 2022-09-03T06:18:11.044 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.045 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.046 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.047 DEBUG:boto:Params: {} 2022-09-03T06:18:11.048 DEBUG:boto:Token: None 2022-09-03T06:18:11.049 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/b 2022-09-03T06:18:11.051 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:inwyW/vxj9j5oMF4Iq+Mhs6BPUQ= 2022-09-03T06:18:11.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:inwyW/vxj9j5oMF4Iq+Mhs6BPUQ=', 'Content-Length': '0'} 2022-09-03T06:18:11.054 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003eee131884981413-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.055 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:11.055 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:11.056 DEBUG:boto:Method: GET 2022-09-03T06:18:11.057 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:11.057 DEBUG:boto:Data: 2022-09-03T06:18:11.058 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.058 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.059 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.059 DEBUG:boto:Params: {} 2022-09-03T06:18:11.060 DEBUG:boto:Token: None 2022-09-03T06:18:11.060 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/b 2022-09-03T06:18:11.062 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:inwyW/vxj9j5oMF4Iq+Mhs6BPUQ= 2022-09-03T06:18:11.063 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:inwyW/vxj9j5oMF4Iq+Mhs6BPUQ=', 'Content-Length': '0'} 2022-09-03T06:18:11.070 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000edbc9f47be0b6728-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.072 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:11.073 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:11.074 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:11.075 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:11.076 DEBUG:boto:Method: GET 2022-09-03T06:18:11.078 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:11.079 DEBUG:boto:Data: 2022-09-03T06:18:11.080 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.080 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.081 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.082 DEBUG:boto:Params: {} 2022-09-03T06:18:11.082 DEBUG:boto:Token: None 2022-09-03T06:18:11.083 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/ 2022-09-03T06:18:11.084 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA= 2022-09-03T06:18:11.086 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA=', 'Content-Length': '0'} 2022-09-03T06:18:11.090 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000977b648f6e22e6e7-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.091 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:11.093 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:11.094 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:11.095 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:11.096 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:11.097 DEBUG:boto:Method: GET 2022-09-03T06:18:11.098 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:11.098 DEBUG:boto:Data: 2022-09-03T06:18:11.099 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.099 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.100 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.101 DEBUG:boto:Params: {} 2022-09-03T06:18:11.101 DEBUG:boto:Token: None 2022-09-03T06:18:11.102 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/ 2022-09-03T06:18:11.103 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA= 2022-09-03T06:18:11.103 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA=', 'Content-Length': '0'} 2022-09-03T06:18:11.106 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001fe61ada7841f507-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.150 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:11.151 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:18:11.152 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:11.153 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:11.153 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.154 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.154 DEBUG:boto:Method: GET 2022-09-03T06:18:11.155 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.156 DEBUG:boto:Data: 2022-09-03T06:18:11.156 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.157 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.157 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.158 DEBUG:boto:Params: {} 2022-09-03T06:18:11.160 DEBUG:boto:Token: None 2022-09-03T06:18:11.161 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.162 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AzfUUX11xogeKdY4y6cq0+jZt+I= 2022-09-03T06:18:11.162 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AzfUUX11xogeKdY4y6cq0+jZt+I=', 'Content-Length': '0'} 2022-09-03T06:18:11.165 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000992104ae28f30e8e-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.166 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:11.167 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:11.167 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.168 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.169 DEBUG:boto:Method: GET 2022-09-03T06:18:11.170 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.171 DEBUG:boto:Data: 2022-09-03T06:18:11.172 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.173 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.173 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.174 DEBUG:boto:Params: {} 2022-09-03T06:18:11.175 DEBUG:boto:Token: None 2022-09-03T06:18:11.176 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.177 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AzfUUX11xogeKdY4y6cq0+jZt+I= 2022-09-03T06:18:11.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AzfUUX11xogeKdY4y6cq0+jZt+I=', 'Content-Length': '0'} 2022-09-03T06:18:11.184 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b0d94df451b768c4-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.185 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:11.186 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:11.187 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.188 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.189 DEBUG:boto:Method: HEAD 2022-09-03T06:18:11.190 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.191 DEBUG:boto:Data: 2022-09-03T06:18:11.192 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.193 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.194 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.195 DEBUG:boto:Params: {} 2022-09-03T06:18:11.196 DEBUG:boto:Token: None 2022-09-03T06:18:11.197 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.197 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wDRhZIcChgJtPHgWp11XVzHgwao= 2022-09-03T06:18:11.198 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wDRhZIcChgJtPHgWp11XVzHgwao=', 'Content-Length': '0'} 2022-09-03T06:18:11.200 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000094e31541bd874f57-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.202 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:11.203 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:11.204 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.205 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.206 DEBUG:boto:Method: HEAD 2022-09-03T06:18:11.207 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.208 DEBUG:boto:Data: 2022-09-03T06:18:11.209 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.209 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.210 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.210 DEBUG:boto:Params: {} 2022-09-03T06:18:11.211 DEBUG:boto:Token: None 2022-09-03T06:18:11.212 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.212 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wDRhZIcChgJtPHgWp11XVzHgwao= 2022-09-03T06:18:11.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wDRhZIcChgJtPHgWp11XVzHgwao=', 'Content-Length': '0'} 2022-09-03T06:18:11.215 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a10e347552f5bb35-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.216 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:18:11.217 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:11.217 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:11.218 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.219 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.219 DEBUG:boto:Method: GET 2022-09-03T06:18:11.220 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.221 DEBUG:boto:Data: 2022-09-03T06:18:11.222 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.223 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.224 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.225 DEBUG:boto:Params: {} 2022-09-03T06:18:11.226 DEBUG:boto:Token: None 2022-09-03T06:18:11.227 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.228 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AzfUUX11xogeKdY4y6cq0+jZt+I= 2022-09-03T06:18:11.229 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AzfUUX11xogeKdY4y6cq0+jZt+I=', 'Content-Length': '0'} 2022-09-03T06:18:11.231 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000460330a55ed1cd1a-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.232 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:11.234 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:11.234 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.235 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.235 DEBUG:boto:Method: GET 2022-09-03T06:18:11.236 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.237 DEBUG:boto:Data: 2022-09-03T06:18:11.238 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.239 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.240 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.240 DEBUG:boto:Params: {} 2022-09-03T06:18:11.241 DEBUG:boto:Token: None 2022-09-03T06:18:11.242 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:11.243 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AzfUUX11xogeKdY4y6cq0+jZt+I= 2022-09-03T06:18:11.243 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AzfUUX11xogeKdY4y6cq0+jZt+I=', 'Content-Length': '0'} 2022-09-03T06:18:11.245 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004644f3929893ae85-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.246 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:11.247 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:11.248 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:11.249 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:11.249 DEBUG:boto:Method: GET 2022-09-03T06:18:11.250 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:11.251 DEBUG:boto:Data: 2022-09-03T06:18:11.251 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.252 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.253 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.253 DEBUG:boto:Params: {} 2022-09-03T06:18:11.254 DEBUG:boto:Token: None 2022-09-03T06:18:11.254 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/ 2022-09-03T06:18:11.255 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA= 2022-09-03T06:18:11.256 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA=', 'Content-Length': '0'} 2022-09-03T06:18:11.259 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e389e8bb57f8b295-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.259 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:11.260 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:11.261 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:11.261 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:11.262 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:11.263 DEBUG:boto:Method: GET 2022-09-03T06:18:11.264 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:11.264 DEBUG:boto:Data: 2022-09-03T06:18:11.265 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.265 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.266 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.267 DEBUG:boto:Params: {} 2022-09-03T06:18:11.268 DEBUG:boto:Token: None 2022-09-03T06:18:11.269 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/ 2022-09-03T06:18:11.270 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA= 2022-09-03T06:18:11.271 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA=', 'Content-Length': '0'} 2022-09-03T06:18:11.275 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dcfac12e315a1619-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.276 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:11.277 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:11.278 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:11.279 DEBUG:boto:Method: HEAD 2022-09-03T06:18:11.280 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:11.281 DEBUG:boto:Data: 2022-09-03T06:18:11.282 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.283 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.284 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.285 DEBUG:boto:Params: {} 2022-09-03T06:18:11.286 DEBUG:boto:Token: None 2022-09-03T06:18:11.287 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/c 2022-09-03T06:18:11.288 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rX4pWEu4hdVyRWeFEOBbkJ6XmKA= 2022-09-03T06:18:11.289 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rX4pWEu4hdVyRWeFEOBbkJ6XmKA=', 'Content-Length': '0'} 2022-09-03T06:18:11.292 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e1f0783ddd0df42a-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.292 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:11.293 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:11.293 DEBUG:boto:Method: HEAD 2022-09-03T06:18:11.294 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:11.295 DEBUG:boto:Data: 2022-09-03T06:18:11.295 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.296 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.296 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.297 DEBUG:boto:Params: {} 2022-09-03T06:18:11.298 DEBUG:boto:Token: None 2022-09-03T06:18:11.298 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/c 2022-09-03T06:18:11.299 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rX4pWEu4hdVyRWeFEOBbkJ6XmKA= 2022-09-03T06:18:11.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rX4pWEu4hdVyRWeFEOBbkJ6XmKA=', 'Content-Length': '0'} 2022-09-03T06:18:11.302 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ec8d8badcf682f13-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.303 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:18:11.304 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:11.305 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:11.306 DEBUG:boto:Method: GET 2022-09-03T06:18:11.307 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:11.308 DEBUG:boto:Data: 2022-09-03T06:18:11.309 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.309 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.310 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.311 DEBUG:boto:Params: {} 2022-09-03T06:18:11.312 DEBUG:boto:Token: None 2022-09-03T06:18:11.313 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/c 2022-09-03T06:18:11.314 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OKtR6VAboiUHfe3FahH6UH59IeA= 2022-09-03T06:18:11.315 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OKtR6VAboiUHfe3FahH6UH59IeA=', 'Content-Length': '0'} 2022-09-03T06:18:11.318 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000042a4843d733b3ee-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.362 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:11.363 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:11.364 DEBUG:boto:Method: GET 2022-09-03T06:18:11.365 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:11.366 DEBUG:boto:Data: 2022-09-03T06:18:11.367 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.368 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.369 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.370 DEBUG:boto:Params: {} 2022-09-03T06:18:11.371 DEBUG:boto:Token: None 2022-09-03T06:18:11.372 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/c 2022-09-03T06:18:11.373 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OKtR6VAboiUHfe3FahH6UH59IeA= 2022-09-03T06:18:11.374 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OKtR6VAboiUHfe3FahH6UH59IeA=', 'Content-Length': '0'} 2022-09-03T06:18:11.378 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ace753465a7de46c-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.379 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:11.380 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:11.381 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:11.382 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:11.383 DEBUG:boto:Method: GET 2022-09-03T06:18:11.384 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:11.385 DEBUG:boto:Data: 2022-09-03T06:18:11.386 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.387 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.388 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.389 DEBUG:boto:Params: {} 2022-09-03T06:18:11.390 DEBUG:boto:Token: None 2022-09-03T06:18:11.391 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/ 2022-09-03T06:18:11.392 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA= 2022-09-03T06:18:11.393 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA=', 'Content-Length': '0'} 2022-09-03T06:18:11.396 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008117aec86f4b2dc4-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.398 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:11.399 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:11.400 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:11.400 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:11.401 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:11.401 DEBUG:boto:Method: GET 2022-09-03T06:18:11.402 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:11.402 DEBUG:boto:Data: 2022-09-03T06:18:11.403 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.403 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.404 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.404 DEBUG:boto:Params: {} 2022-09-03T06:18:11.405 DEBUG:boto:Token: None 2022-09-03T06:18:11.406 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/ 2022-09-03T06:18:11.406 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA= 2022-09-03T06:18:11.407 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA=', 'Content-Length': '0'} 2022-09-03T06:18:11.410 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c7671850654195cb-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.411 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:11.412 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:18:11.412 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:11.413 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:11.414 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.415 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.416 DEBUG:boto:Method: GET 2022-09-03T06:18:11.417 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.418 DEBUG:boto:Data: 2022-09-03T06:18:11.420 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.421 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.422 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.423 DEBUG:boto:Params: {} 2022-09-03T06:18:11.424 DEBUG:boto:Token: None 2022-09-03T06:18:11.425 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.426 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WLU1qSi1B7fo+GQeQCHC2QDkzFQ= 2022-09-03T06:18:11.427 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WLU1qSi1B7fo+GQeQCHC2QDkzFQ=', 'Content-Length': '0'} 2022-09-03T06:18:11.430 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001321f870e9f3f8f0-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.432 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:11.433 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:11.434 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.435 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.436 DEBUG:boto:Method: GET 2022-09-03T06:18:11.437 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.438 DEBUG:boto:Data: 2022-09-03T06:18:11.439 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.440 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.441 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.442 DEBUG:boto:Params: {} 2022-09-03T06:18:11.443 DEBUG:boto:Token: None 2022-09-03T06:18:11.444 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.446 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WLU1qSi1B7fo+GQeQCHC2QDkzFQ= 2022-09-03T06:18:11.447 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WLU1qSi1B7fo+GQeQCHC2QDkzFQ=', 'Content-Length': '0'} 2022-09-03T06:18:11.449 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bb1c9637b724e14e-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.451 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:11.451 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:11.452 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.453 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.454 DEBUG:boto:Method: HEAD 2022-09-03T06:18:11.454 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.455 DEBUG:boto:Data: 2022-09-03T06:18:11.455 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.456 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.456 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.457 DEBUG:boto:Params: {} 2022-09-03T06:18:11.458 DEBUG:boto:Token: None 2022-09-03T06:18:11.459 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.460 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mfdREFMxhGqJ1oQAWZ7CUiEXQfQ= 2022-09-03T06:18:11.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mfdREFMxhGqJ1oQAWZ7CUiEXQfQ=', 'Content-Length': '0'} 2022-09-03T06:18:11.463 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000616f4f201fabeb6f-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.464 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:11.465 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:11.465 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.466 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.467 DEBUG:boto:Method: HEAD 2022-09-03T06:18:11.468 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.468 DEBUG:boto:Data: 2022-09-03T06:18:11.469 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.469 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.470 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.470 DEBUG:boto:Params: {} 2022-09-03T06:18:11.471 DEBUG:boto:Token: None 2022-09-03T06:18:11.473 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.473 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mfdREFMxhGqJ1oQAWZ7CUiEXQfQ= 2022-09-03T06:18:11.474 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mfdREFMxhGqJ1oQAWZ7CUiEXQfQ=', 'Content-Length': '0'} 2022-09-03T06:18:11.476 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000339d711ffc66cb7d-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.477 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:18:11.478 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:11.480 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:11.481 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.481 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.482 DEBUG:boto:Method: GET 2022-09-03T06:18:11.483 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.484 DEBUG:boto:Data: 2022-09-03T06:18:11.484 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.485 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.485 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.487 DEBUG:boto:Params: {} 2022-09-03T06:18:11.487 DEBUG:boto:Token: None 2022-09-03T06:18:11.488 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.489 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WLU1qSi1B7fo+GQeQCHC2QDkzFQ= 2022-09-03T06:18:11.490 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WLU1qSi1B7fo+GQeQCHC2QDkzFQ=', 'Content-Length': '0'} 2022-09-03T06:18:11.493 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004658f3b25a6bd634-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.494 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:11.495 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:11.496 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.497 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.498 DEBUG:boto:Method: GET 2022-09-03T06:18:11.499 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.500 DEBUG:boto:Data: 2022-09-03T06:18:11.501 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.502 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.502 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.503 DEBUG:boto:Params: {} 2022-09-03T06:18:11.504 DEBUG:boto:Token: None 2022-09-03T06:18:11.504 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:11.505 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WLU1qSi1B7fo+GQeQCHC2QDkzFQ= 2022-09-03T06:18:11.506 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WLU1qSi1B7fo+GQeQCHC2QDkzFQ=', 'Content-Length': '0'} 2022-09-03T06:18:11.508 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007963d9752f25c139-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.509 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:11.510 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:11.511 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:11.512 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:11.513 DEBUG:boto:Method: GET 2022-09-03T06:18:11.514 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:11.515 DEBUG:boto:Data: 2022-09-03T06:18:11.516 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.517 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.517 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.518 DEBUG:boto:Params: {} 2022-09-03T06:18:11.519 DEBUG:boto:Token: None 2022-09-03T06:18:11.519 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/ 2022-09-03T06:18:11.520 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA= 2022-09-03T06:18:11.521 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA=', 'Content-Length': '0'} 2022-09-03T06:18:11.524 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006db4a27a255d1e09-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.524 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:11.525 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:11.526 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:11.526 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:11.527 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:11.527 DEBUG:boto:Method: GET 2022-09-03T06:18:11.528 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:11.529 DEBUG:boto:Data: 2022-09-03T06:18:11.529 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.530 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.530 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.531 DEBUG:boto:Params: {} 2022-09-03T06:18:11.531 DEBUG:boto:Token: None 2022-09-03T06:18:11.532 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/ 2022-09-03T06:18:11.533 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA= 2022-09-03T06:18:11.533 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA=', 'Content-Length': '0'} 2022-09-03T06:18:11.536 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000917cd700f01e0a10-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.537 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:11.538 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:11.538 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:11.539 DEBUG:boto:Method: HEAD 2022-09-03T06:18:11.539 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:11.540 DEBUG:boto:Data: 2022-09-03T06:18:11.540 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.541 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.542 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.542 DEBUG:boto:Params: {} 2022-09-03T06:18:11.543 DEBUG:boto:Token: None 2022-09-03T06:18:11.543 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/d 2022-09-03T06:18:11.544 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IgIn6enDznPyYFmuRAac2E5xP/E= 2022-09-03T06:18:11.545 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IgIn6enDznPyYFmuRAac2E5xP/E=', 'Content-Length': '0'} 2022-09-03T06:18:11.547 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ef74b35a6f86ef6a-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.547 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:11.548 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:11.549 DEBUG:boto:Method: HEAD 2022-09-03T06:18:11.549 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:11.550 DEBUG:boto:Data: 2022-09-03T06:18:11.550 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.551 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.551 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.552 DEBUG:boto:Params: {} 2022-09-03T06:18:11.553 DEBUG:boto:Token: None 2022-09-03T06:18:11.553 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/d 2022-09-03T06:18:11.554 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IgIn6enDznPyYFmuRAac2E5xP/E= 2022-09-03T06:18:11.555 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IgIn6enDznPyYFmuRAac2E5xP/E=', 'Content-Length': '0'} 2022-09-03T06:18:11.557 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000007e19d5a3abcd3bf-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.558 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:18:11.558 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:11.559 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:11.560 DEBUG:boto:Method: GET 2022-09-03T06:18:11.560 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:11.561 DEBUG:boto:Data: 2022-09-03T06:18:11.561 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.562 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.562 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.563 DEBUG:boto:Params: {} 2022-09-03T06:18:11.563 DEBUG:boto:Token: None 2022-09-03T06:18:11.564 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/d 2022-09-03T06:18:11.565 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gELog8RbLlDibPjdO+xJZ4X2ViQ= 2022-09-03T06:18:11.565 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gELog8RbLlDibPjdO+xJZ4X2ViQ=', 'Content-Length': '0'} 2022-09-03T06:18:11.567 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002df96525e8da6170-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.610 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:11.611 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:11.611 DEBUG:boto:Method: GET 2022-09-03T06:18:11.612 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:11.613 DEBUG:boto:Data: 2022-09-03T06:18:11.613 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.614 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.614 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.615 DEBUG:boto:Params: {} 2022-09-03T06:18:11.615 DEBUG:boto:Token: None 2022-09-03T06:18:11.616 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/d 2022-09-03T06:18:11.617 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gELog8RbLlDibPjdO+xJZ4X2ViQ= 2022-09-03T06:18:11.617 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gELog8RbLlDibPjdO+xJZ4X2ViQ=', 'Content-Length': '0'} 2022-09-03T06:18:11.619 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a69f8401cab835c-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.620 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:11.621 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:11.621 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:11.622 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:11.622 DEBUG:boto:Method: GET 2022-09-03T06:18:11.623 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:11.624 DEBUG:boto:Data: 2022-09-03T06:18:11.624 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.625 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.626 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.626 DEBUG:boto:Params: {} 2022-09-03T06:18:11.627 DEBUG:boto:Token: None 2022-09-03T06:18:11.628 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/ 2022-09-03T06:18:11.628 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA= 2022-09-03T06:18:11.629 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA=', 'Content-Length': '0'} 2022-09-03T06:18:11.632 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002412c21586bdc303-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.633 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:11.634 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:11.635 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:11.635 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:11.636 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:11.636 DEBUG:boto:Method: GET 2022-09-03T06:18:11.637 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:11.637 DEBUG:boto:Data: 2022-09-03T06:18:11.638 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.638 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.639 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.639 DEBUG:boto:Params: {} 2022-09-03T06:18:11.640 DEBUG:boto:Token: None 2022-09-03T06:18:11.640 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/ 2022-09-03T06:18:11.641 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA= 2022-09-03T06:18:11.642 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA=', 'Content-Length': '0'} 2022-09-03T06:18:11.646 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dfe76d8925f67c67-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.647 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:11.647 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:18:11.648 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:11.649 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:11.649 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.650 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.651 DEBUG:boto:Method: GET 2022-09-03T06:18:11.652 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.653 DEBUG:boto:Data: 2022-09-03T06:18:11.654 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.655 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.656 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.657 DEBUG:boto:Params: {} 2022-09-03T06:18:11.658 DEBUG:boto:Token: None 2022-09-03T06:18:11.660 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.661 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nGQBy7LG5e2efbpMcu9Tk4pR6TM= 2022-09-03T06:18:11.662 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nGQBy7LG5e2efbpMcu9Tk4pR6TM=', 'Content-Length': '0'} 2022-09-03T06:18:11.664 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005ab06212f92d761c-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.666 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:11.667 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:11.668 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.669 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.670 DEBUG:boto:Method: GET 2022-09-03T06:18:11.670 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.671 DEBUG:boto:Data: 2022-09-03T06:18:11.672 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.672 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.675 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.675 DEBUG:boto:Params: {} 2022-09-03T06:18:11.676 DEBUG:boto:Token: None 2022-09-03T06:18:11.676 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.677 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nGQBy7LG5e2efbpMcu9Tk4pR6TM= 2022-09-03T06:18:11.678 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nGQBy7LG5e2efbpMcu9Tk4pR6TM=', 'Content-Length': '0'} 2022-09-03T06:18:11.688 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008db7484c40fc26df-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.689 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:11.690 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:11.691 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.692 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.693 DEBUG:boto:Method: HEAD 2022-09-03T06:18:11.694 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.697 DEBUG:boto:Data: 2022-09-03T06:18:11.698 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.699 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.700 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.701 DEBUG:boto:Params: {} 2022-09-03T06:18:11.702 DEBUG:boto:Token: None 2022-09-03T06:18:11.703 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.704 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OJvLt290dwFYwOnTrIKyYbRnxWM= 2022-09-03T06:18:11.705 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OJvLt290dwFYwOnTrIKyYbRnxWM=', 'Content-Length': '0'} 2022-09-03T06:18:11.708 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e1287ede4ea3ff42-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.709 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:11.710 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:11.710 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.711 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.712 DEBUG:boto:Method: HEAD 2022-09-03T06:18:11.712 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.713 DEBUG:boto:Data: 2022-09-03T06:18:11.713 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.714 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.714 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.715 DEBUG:boto:Params: {} 2022-09-03T06:18:11.715 DEBUG:boto:Token: None 2022-09-03T06:18:11.716 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.717 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OJvLt290dwFYwOnTrIKyYbRnxWM= 2022-09-03T06:18:11.718 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OJvLt290dwFYwOnTrIKyYbRnxWM=', 'Content-Length': '0'} 2022-09-03T06:18:11.720 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000002d8886c2066f7c9-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.721 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:18:11.722 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:11.723 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:11.724 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.725 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.726 DEBUG:boto:Method: GET 2022-09-03T06:18:11.727 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.728 DEBUG:boto:Data: 2022-09-03T06:18:11.729 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.730 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.731 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.732 DEBUG:boto:Params: {} 2022-09-03T06:18:11.734 DEBUG:boto:Token: None 2022-09-03T06:18:11.735 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.736 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nGQBy7LG5e2efbpMcu9Tk4pR6TM= 2022-09-03T06:18:11.737 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nGQBy7LG5e2efbpMcu9Tk4pR6TM=', 'Content-Length': '0'} 2022-09-03T06:18:11.740 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b0ba5dc073e4574-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.742 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:11.743 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:11.744 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.745 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.746 DEBUG:boto:Method: GET 2022-09-03T06:18:11.747 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.751 DEBUG:boto:Data: 2022-09-03T06:18:11.751 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.752 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.753 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.754 DEBUG:boto:Params: {} 2022-09-03T06:18:11.754 DEBUG:boto:Token: None 2022-09-03T06:18:11.756 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:11.756 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nGQBy7LG5e2efbpMcu9Tk4pR6TM= 2022-09-03T06:18:11.757 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nGQBy7LG5e2efbpMcu9Tk4pR6TM=', 'Content-Length': '0'} 2022-09-03T06:18:11.760 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c27a0ae72adeb3be-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.765 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:11.766 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:11.768 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:11.769 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:11.770 DEBUG:boto:Method: GET 2022-09-03T06:18:11.771 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:11.772 DEBUG:boto:Data: 2022-09-03T06:18:11.773 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.774 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.775 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.777 DEBUG:boto:Params: {} 2022-09-03T06:18:11.779 DEBUG:boto:Token: None 2022-09-03T06:18:11.780 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/ 2022-09-03T06:18:11.781 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA= 2022-09-03T06:18:11.782 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA=', 'Content-Length': '0'} 2022-09-03T06:18:11.785 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000060064436f6b945fd-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.788 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:11.790 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:11.791 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:11.791 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:11.792 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:11.792 DEBUG:boto:Method: GET 2022-09-03T06:18:11.793 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:11.794 DEBUG:boto:Data: 2022-09-03T06:18:11.795 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.796 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.797 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.798 DEBUG:boto:Params: {} 2022-09-03T06:18:11.800 DEBUG:boto:Token: None 2022-09-03T06:18:11.801 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/ 2022-09-03T06:18:11.802 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA= 2022-09-03T06:18:11.802 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA=', 'Content-Length': '0'} 2022-09-03T06:18:11.806 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000815a6b31a8ffa108-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.807 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:11.810 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:11.811 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:11.812 DEBUG:boto:Method: HEAD 2022-09-03T06:18:11.813 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:11.814 DEBUG:boto:Data: 2022-09-03T06:18:11.815 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.816 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.816 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.817 DEBUG:boto:Params: {} 2022-09-03T06:18:11.818 DEBUG:boto:Token: None 2022-09-03T06:18:11.818 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/e 2022-09-03T06:18:11.820 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SKaItnfIzoyWP5e0nUqvuPsHGG8= 2022-09-03T06:18:11.821 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SKaItnfIzoyWP5e0nUqvuPsHGG8=', 'Content-Length': '0'} 2022-09-03T06:18:11.823 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004975fd989ce1a987-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.824 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:11.824 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:11.825 DEBUG:boto:Method: HEAD 2022-09-03T06:18:11.826 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:11.826 DEBUG:boto:Data: 2022-09-03T06:18:11.827 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.827 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.828 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.828 DEBUG:boto:Params: {} 2022-09-03T06:18:11.829 DEBUG:boto:Token: None 2022-09-03T06:18:11.829 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/e 2022-09-03T06:18:11.830 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SKaItnfIzoyWP5e0nUqvuPsHGG8= 2022-09-03T06:18:11.831 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SKaItnfIzoyWP5e0nUqvuPsHGG8=', 'Content-Length': '0'} 2022-09-03T06:18:11.833 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096dd59af60e32185-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.834 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:18:11.836 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:11.837 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:11.837 DEBUG:boto:Method: GET 2022-09-03T06:18:11.838 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:11.839 DEBUG:boto:Data: 2022-09-03T06:18:11.840 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.841 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.842 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.843 DEBUG:boto:Params: {} 2022-09-03T06:18:11.844 DEBUG:boto:Token: None 2022-09-03T06:18:11.844 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/e 2022-09-03T06:18:11.845 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HlJRBUxkt20NC5pFyX0/AfCsuw4= 2022-09-03T06:18:11.846 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HlJRBUxkt20NC5pFyX0/AfCsuw4=', 'Content-Length': '0'} 2022-09-03T06:18:11.858 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ea5ceaec4b166ebf-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.898 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:11.900 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:11.901 DEBUG:boto:Method: GET 2022-09-03T06:18:11.902 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:11.902 DEBUG:boto:Data: 2022-09-03T06:18:11.903 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.903 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.904 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.904 DEBUG:boto:Params: {} 2022-09-03T06:18:11.905 DEBUG:boto:Token: None 2022-09-03T06:18:11.906 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/e 2022-09-03T06:18:11.907 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HlJRBUxkt20NC5pFyX0/AfCsuw4= 2022-09-03T06:18:11.907 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HlJRBUxkt20NC5pFyX0/AfCsuw4=', 'Content-Length': '0'} 2022-09-03T06:18:11.915 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ded294212aaf6b3b-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.926 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:11.927 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:11.928 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:11.928 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:11.929 DEBUG:boto:Method: GET 2022-09-03T06:18:11.929 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:11.935 DEBUG:boto:Data: 2022-09-03T06:18:11.936 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.936 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:11.937 DEBUG:boto:Port: 8001 2022-09-03T06:18:11.938 DEBUG:boto:Params: {} 2022-09-03T06:18:11.939 DEBUG:boto:Token: None 2022-09-03T06:18:11.939 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/ 2022-09-03T06:18:11.940 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA= 2022-09-03T06:18:11.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA=', 'Content-Length': '0'} 2022-09-03T06:18:11.943 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000378a0abfb16817c4-006312f1a3-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:11.944 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:11.945 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:11.946 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:11.946 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:11.947 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:11.948 DEBUG:boto:Method: GET 2022-09-03T06:18:11.948 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:11.949 DEBUG:boto:Data: 2022-09-03T06:18:11.949 DEBUG:boto:Headers: {} 2022-09-03T06:18:11.950 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:11.950 DEBUG:boto:Port: 8000 2022-09-03T06:18:11.951 DEBUG:boto:Params: {} 2022-09-03T06:18:11.951 DEBUG:boto:Token: None 2022-09-03T06:18:11.952 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:11 GMT /dsgvcy-71/ 2022-09-03T06:18:11.953 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA= 2022-09-03T06:18:11.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FlJCtQ6BrFnM0WuItabKD28RdGA=', 'Content-Length': '0'} 2022-09-03T06:18:11.958 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002e2e8cf52b96d07c-006312f1a3-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.002 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:12.004 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:18:12.004 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:12.005 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:12.006 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.006 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.007 DEBUG:boto:Method: GET 2022-09-03T06:18:12.008 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.008 DEBUG:boto:Data: 2022-09-03T06:18:12.009 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.010 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:12.010 DEBUG:boto:Port: 8001 2022-09-03T06:18:12.011 DEBUG:boto:Params: {} 2022-09-03T06:18:12.011 DEBUG:boto:Token: None 2022-09-03T06:18:12.012 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.013 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9QXDJT4NT68E6XVgEJH4cKhayXY= 2022-09-03T06:18:12.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9QXDJT4NT68E6XVgEJH4cKhayXY=', 'Content-Length': '0'} 2022-09-03T06:18:12.016 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008c3b5ded6a3c0a89-006312f1a4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.018 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:12.019 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:12.020 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.021 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.022 DEBUG:boto:Method: GET 2022-09-03T06:18:12.023 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.033 DEBUG:boto:Data: 2022-09-03T06:18:12.033 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.034 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:12.034 DEBUG:boto:Port: 8000 2022-09-03T06:18:12.035 DEBUG:boto:Params: {} 2022-09-03T06:18:12.036 DEBUG:boto:Token: None 2022-09-03T06:18:12.036 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.037 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9QXDJT4NT68E6XVgEJH4cKhayXY= 2022-09-03T06:18:12.038 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9QXDJT4NT68E6XVgEJH4cKhayXY=', 'Content-Length': '0'} 2022-09-03T06:18:12.044 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009001c24706eedcc7-006312f1a4-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.045 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:12.046 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:12.046 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.047 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.048 DEBUG:boto:Method: HEAD 2022-09-03T06:18:12.048 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.049 DEBUG:boto:Data: 2022-09-03T06:18:12.049 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.050 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:12.050 DEBUG:boto:Port: 8001 2022-09-03T06:18:12.051 DEBUG:boto:Params: {} 2022-09-03T06:18:12.051 DEBUG:boto:Token: None 2022-09-03T06:18:12.052 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.053 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:POqcZssW0RSAVcGJIVjAdRPvuWo= 2022-09-03T06:18:12.053 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:POqcZssW0RSAVcGJIVjAdRPvuWo=', 'Content-Length': '0'} 2022-09-03T06:18:12.062 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000067d1472e19ec8e6-006312f1a4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.064 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:12.065 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:12.065 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.066 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.067 DEBUG:boto:Method: HEAD 2022-09-03T06:18:12.068 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.069 DEBUG:boto:Data: 2022-09-03T06:18:12.070 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.071 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:12.071 DEBUG:boto:Port: 8000 2022-09-03T06:18:12.072 DEBUG:boto:Params: {} 2022-09-03T06:18:12.073 DEBUG:boto:Token: None 2022-09-03T06:18:12.075 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.077 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:POqcZssW0RSAVcGJIVjAdRPvuWo= 2022-09-03T06:18:12.078 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:POqcZssW0RSAVcGJIVjAdRPvuWo=', 'Content-Length': '0'} 2022-09-03T06:18:12.080 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a87f25e4452759ed-006312f1a4-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.082 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:18:12.082 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:12.083 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:12.083 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.084 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.084 DEBUG:boto:Method: GET 2022-09-03T06:18:12.085 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.086 DEBUG:boto:Data: 2022-09-03T06:18:12.086 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.087 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:12.087 DEBUG:boto:Port: 8001 2022-09-03T06:18:12.088 DEBUG:boto:Params: {} 2022-09-03T06:18:12.089 DEBUG:boto:Token: None 2022-09-03T06:18:12.089 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.090 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9QXDJT4NT68E6XVgEJH4cKhayXY= 2022-09-03T06:18:12.091 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9QXDJT4NT68E6XVgEJH4cKhayXY=', 'Content-Length': '0'} 2022-09-03T06:18:12.093 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000966ded2cbc26fbda-006312f1a4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.095 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:12.095 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:12.097 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.098 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.099 DEBUG:boto:Method: GET 2022-09-03T06:18:12.100 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.101 DEBUG:boto:Data: 2022-09-03T06:18:12.102 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.103 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:12.103 DEBUG:boto:Port: 8000 2022-09-03T06:18:12.105 DEBUG:boto:Params: {} 2022-09-03T06:18:12.106 DEBUG:boto:Token: None 2022-09-03T06:18:12.107 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:12.108 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9QXDJT4NT68E6XVgEJH4cKhayXY= 2022-09-03T06:18:12.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9QXDJT4NT68E6XVgEJH4cKhayXY=', 'Content-Length': '0'} 2022-09-03T06:18:12.111 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009673d72bc49a1c96-006312f1a4-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.116 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:12.118 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:12.119 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:12.119 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:12.120 DEBUG:boto:Method: GET 2022-09-03T06:18:12.121 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:12.122 DEBUG:boto:Data: 2022-09-03T06:18:12.123 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.124 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:12.125 DEBUG:boto:Port: 8001 2022-09-03T06:18:12.126 DEBUG:boto:Params: {} 2022-09-03T06:18:12.128 DEBUG:boto:Token: None 2022-09-03T06:18:12.129 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/ 2022-09-03T06:18:12.130 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U= 2022-09-03T06:18:12.130 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U=', 'Content-Length': '0'} 2022-09-03T06:18:12.135 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000003200f594f1f3f94-006312f1a4-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.137 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:12.139 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:12.140 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:12.141 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:12.142 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:12.143 DEBUG:boto:Method: GET 2022-09-03T06:18:12.144 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:12.145 DEBUG:boto:Data: 2022-09-03T06:18:12.146 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.147 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:12.147 DEBUG:boto:Port: 8000 2022-09-03T06:18:12.148 DEBUG:boto:Params: {} 2022-09-03T06:18:12.149 DEBUG:boto:Token: None 2022-09-03T06:18:12.150 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/ 2022-09-03T06:18:12.150 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U= 2022-09-03T06:18:12.151 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U=', 'Content-Length': '0'} 2022-09-03T06:18:12.154 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000028f0931f70d452e5-006312f1a4-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.199 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:12.201 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:12.203 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:12.204 DEBUG:boto:Method: HEAD 2022-09-03T06:18:12.204 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:12.205 DEBUG:boto:Data: 2022-09-03T06:18:12.205 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.207 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:12.207 DEBUG:boto:Port: 8001 2022-09-03T06:18:12.208 DEBUG:boto:Params: {} 2022-09-03T06:18:12.208 DEBUG:boto:Token: None 2022-09-03T06:18:12.209 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/f 2022-09-03T06:18:12.210 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5QaQ28LAa/RLp3fcHom0SuVllYY= 2022-09-03T06:18:12.211 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5QaQ28LAa/RLp3fcHom0SuVllYY=', 'Content-Length': '0'} 2022-09-03T06:18:12.213 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000003bad864b0be4006-006312f1a4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.215 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:12.216 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:12.218 DEBUG:boto:Method: HEAD 2022-09-03T06:18:12.219 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:12.220 DEBUG:boto:Data: 2022-09-03T06:18:12.222 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.223 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:12.224 DEBUG:boto:Port: 8000 2022-09-03T06:18:12.225 DEBUG:boto:Params: {} 2022-09-03T06:18:12.226 DEBUG:boto:Token: None 2022-09-03T06:18:12.227 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/f 2022-09-03T06:18:12.227 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5QaQ28LAa/RLp3fcHom0SuVllYY= 2022-09-03T06:18:12.228 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5QaQ28LAa/RLp3fcHom0SuVllYY=', 'Content-Length': '0'} 2022-09-03T06:18:12.231 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000043185c8fdc697b65-006312f1a4-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.232 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:18:12.233 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:12.233 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:12.234 DEBUG:boto:Method: GET 2022-09-03T06:18:12.235 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:12.235 DEBUG:boto:Data: 2022-09-03T06:18:12.237 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.239 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:12.240 DEBUG:boto:Port: 8001 2022-09-03T06:18:12.241 DEBUG:boto:Params: {} 2022-09-03T06:18:12.242 DEBUG:boto:Token: None 2022-09-03T06:18:12.243 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/f 2022-09-03T06:18:12.244 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IQN7gnEPJNLqY7iHVgiWZq7D7hE= 2022-09-03T06:18:12.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IQN7gnEPJNLqY7iHVgiWZq7D7hE=', 'Content-Length': '0'} 2022-09-03T06:18:12.249 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000050ae8df0a518725f-006312f1a4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.250 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:12.251 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:12.252 DEBUG:boto:Method: GET 2022-09-03T06:18:12.253 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:12.254 DEBUG:boto:Data: 2022-09-03T06:18:12.256 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.257 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:12.258 DEBUG:boto:Port: 8000 2022-09-03T06:18:12.259 DEBUG:boto:Params: {} 2022-09-03T06:18:12.260 DEBUG:boto:Token: None 2022-09-03T06:18:12.261 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/f 2022-09-03T06:18:12.262 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IQN7gnEPJNLqY7iHVgiWZq7D7hE= 2022-09-03T06:18:12.263 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IQN7gnEPJNLqY7iHVgiWZq7D7hE=', 'Content-Length': '0'} 2022-09-03T06:18:12.267 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007886c664ce7ccc11-006312f1a4-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.268 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:12.269 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:12.270 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:12.270 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:12.271 DEBUG:boto:Method: GET 2022-09-03T06:18:12.272 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:12.273 DEBUG:boto:Data: 2022-09-03T06:18:12.273 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.275 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:12.276 DEBUG:boto:Port: 8001 2022-09-03T06:18:12.277 DEBUG:boto:Params: {} 2022-09-03T06:18:12.279 DEBUG:boto:Token: None 2022-09-03T06:18:12.280 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/ 2022-09-03T06:18:12.281 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U= 2022-09-03T06:18:12.283 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U=', 'Content-Length': '0'} 2022-09-03T06:18:12.288 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b7eca62bc3735691-006312f1a4-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.289 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:12.290 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:12.292 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:12.292 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:12.293 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:12.306 DEBUG:boto:Method: GET 2022-09-03T06:18:12.307 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:12.308 DEBUG:boto:Data: 2022-09-03T06:18:12.308 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.309 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:12.310 DEBUG:boto:Port: 8000 2022-09-03T06:18:12.311 DEBUG:boto:Params: {} 2022-09-03T06:18:12.312 DEBUG:boto:Token: None 2022-09-03T06:18:12.313 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/ 2022-09-03T06:18:12.316 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U= 2022-09-03T06:18:12.317 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U=', 'Content-Length': '0'} 2022-09-03T06:18:12.321 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000097fa06b35fb631d1-006312f1a4-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.323 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:12.324 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:18:12.327 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:12.328 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:12.329 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.329 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.334 DEBUG:boto:Method: GET 2022-09-03T06:18:12.335 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.336 DEBUG:boto:Data: 2022-09-03T06:18:12.338 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.339 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:12.342 DEBUG:boto:Port: 8001 2022-09-03T06:18:12.345 DEBUG:boto:Params: {} 2022-09-03T06:18:12.346 DEBUG:boto:Token: None 2022-09-03T06:18:12.349 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.351 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GtHr0HT6BjZkBeI3EqHkAkh5NjQ= 2022-09-03T06:18:12.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GtHr0HT6BjZkBeI3EqHkAkh5NjQ=', 'Content-Length': '0'} 2022-09-03T06:18:12.355 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000fee1a64f255a22f-006312f1a4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.357 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:12.358 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:12.361 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.362 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.363 DEBUG:boto:Method: GET 2022-09-03T06:18:12.364 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.365 DEBUG:boto:Data: 2022-09-03T06:18:12.366 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.367 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:12.368 DEBUG:boto:Port: 8000 2022-09-03T06:18:12.371 DEBUG:boto:Params: {} 2022-09-03T06:18:12.372 DEBUG:boto:Token: None 2022-09-03T06:18:12.373 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.374 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GtHr0HT6BjZkBeI3EqHkAkh5NjQ= 2022-09-03T06:18:12.376 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GtHr0HT6BjZkBeI3EqHkAkh5NjQ=', 'Content-Length': '0'} 2022-09-03T06:18:12.380 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e3fa5d6d0445c6b-006312f1a4-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.386 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:12.386 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:12.387 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.388 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.389 DEBUG:boto:Method: HEAD 2022-09-03T06:18:12.389 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.390 DEBUG:boto:Data: 2022-09-03T06:18:12.392 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.392 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:12.393 DEBUG:boto:Port: 8001 2022-09-03T06:18:12.394 DEBUG:boto:Params: {} 2022-09-03T06:18:12.395 DEBUG:boto:Token: None 2022-09-03T06:18:12.396 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.397 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iFilxjC45/yqW7wfMoaI3wNRnaI= 2022-09-03T06:18:12.398 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iFilxjC45/yqW7wfMoaI3wNRnaI=', 'Content-Length': '0'} 2022-09-03T06:18:12.401 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a199a1c6543e0a1-006312f1a4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.402 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:12.403 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:12.404 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.405 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.406 DEBUG:boto:Method: HEAD 2022-09-03T06:18:12.407 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.408 DEBUG:boto:Data: 2022-09-03T06:18:12.409 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.410 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:12.410 DEBUG:boto:Port: 8000 2022-09-03T06:18:12.411 DEBUG:boto:Params: {} 2022-09-03T06:18:12.412 DEBUG:boto:Token: None 2022-09-03T06:18:12.413 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.413 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iFilxjC45/yqW7wfMoaI3wNRnaI= 2022-09-03T06:18:12.414 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iFilxjC45/yqW7wfMoaI3wNRnaI=', 'Content-Length': '0'} 2022-09-03T06:18:12.416 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a1ea6c27c327aa2-006312f1a4-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.417 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:18:12.419 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:12.420 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:12.425 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.427 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.428 DEBUG:boto:Method: GET 2022-09-03T06:18:12.429 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.430 DEBUG:boto:Data: 2022-09-03T06:18:12.432 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.434 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:12.435 DEBUG:boto:Port: 8001 2022-09-03T06:18:12.436 DEBUG:boto:Params: {} 2022-09-03T06:18:12.437 DEBUG:boto:Token: None 2022-09-03T06:18:12.438 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.440 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GtHr0HT6BjZkBeI3EqHkAkh5NjQ= 2022-09-03T06:18:12.441 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GtHr0HT6BjZkBeI3EqHkAkh5NjQ=', 'Content-Length': '0'} 2022-09-03T06:18:12.445 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000045ed47f0a2682b57-006312f1a4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.446 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:12.447 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:12.448 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.449 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.450 DEBUG:boto:Method: GET 2022-09-03T06:18:12.451 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.452 DEBUG:boto:Data: 2022-09-03T06:18:12.453 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.454 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:12.456 DEBUG:boto:Port: 8000 2022-09-03T06:18:12.457 DEBUG:boto:Params: {} 2022-09-03T06:18:12.458 DEBUG:boto:Token: None 2022-09-03T06:18:12.459 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:12.460 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GtHr0HT6BjZkBeI3EqHkAkh5NjQ= 2022-09-03T06:18:12.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GtHr0HT6BjZkBeI3EqHkAkh5NjQ=', 'Content-Length': '0'} 2022-09-03T06:18:12.464 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f97e602049bc3cf2-006312f1a4-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.467 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:12.468 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:12.469 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:12.470 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:12.471 DEBUG:boto:Method: GET 2022-09-03T06:18:12.472 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:12.472 DEBUG:boto:Data: 2022-09-03T06:18:12.473 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.474 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:12.475 DEBUG:boto:Port: 8001 2022-09-03T06:18:12.476 DEBUG:boto:Params: {} 2022-09-03T06:18:12.477 DEBUG:boto:Token: None 2022-09-03T06:18:12.478 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/ 2022-09-03T06:18:12.479 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U= 2022-09-03T06:18:12.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U=', 'Content-Length': '0'} 2022-09-03T06:18:12.486 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000026279525c8c3f7c5-006312f1a4-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.488 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:12.490 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:12.491 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:12.492 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:12.493 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:12.493 DEBUG:boto:Method: GET 2022-09-03T06:18:12.494 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:12.495 DEBUG:boto:Data: 2022-09-03T06:18:12.496 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.497 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:12.498 DEBUG:boto:Port: 8000 2022-09-03T06:18:12.499 DEBUG:boto:Params: {} 2022-09-03T06:18:12.500 DEBUG:boto:Token: None 2022-09-03T06:18:12.501 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/ 2022-09-03T06:18:12.502 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U= 2022-09-03T06:18:12.503 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U=', 'Content-Length': '0'} 2022-09-03T06:18:12.506 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f5019996a4123673-006312f1a4-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.507 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:12.508 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:12.508 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:12.509 DEBUG:boto:Method: HEAD 2022-09-03T06:18:12.509 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:12.510 DEBUG:boto:Data: 2022-09-03T06:18:12.510 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.511 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:12.512 DEBUG:boto:Port: 8001 2022-09-03T06:18:12.512 DEBUG:boto:Params: {} 2022-09-03T06:18:12.513 DEBUG:boto:Token: None 2022-09-03T06:18:12.514 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/g 2022-09-03T06:18:12.515 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XbbLSgTaU2uBMnpPdK+VdyiZeLc= 2022-09-03T06:18:12.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XbbLSgTaU2uBMnpPdK+VdyiZeLc=', 'Content-Length': '0'} 2022-09-03T06:18:12.524 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009ab93f0004cba03f-006312f1a4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.525 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:12.528 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:12.528 DEBUG:boto:Method: HEAD 2022-09-03T06:18:12.529 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:12.530 DEBUG:boto:Data: 2022-09-03T06:18:12.531 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.532 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:12.533 DEBUG:boto:Port: 8000 2022-09-03T06:18:12.534 DEBUG:boto:Params: {} 2022-09-03T06:18:12.535 DEBUG:boto:Token: None 2022-09-03T06:18:12.537 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/g 2022-09-03T06:18:12.538 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XbbLSgTaU2uBMnpPdK+VdyiZeLc= 2022-09-03T06:18:12.539 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XbbLSgTaU2uBMnpPdK+VdyiZeLc=', 'Content-Length': '0'} 2022-09-03T06:18:12.543 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db7a56b30bbe1a5b-006312f1a4-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.544 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:18:12.544 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:12.545 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:12.546 DEBUG:boto:Method: GET 2022-09-03T06:18:12.546 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:12.547 DEBUG:boto:Data: 2022-09-03T06:18:12.548 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.549 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:12.549 DEBUG:boto:Port: 8001 2022-09-03T06:18:12.550 DEBUG:boto:Params: {} 2022-09-03T06:18:12.550 DEBUG:boto:Token: None 2022-09-03T06:18:12.551 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/g 2022-09-03T06:18:12.552 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Xkje3fVUjJhzgTOqcbeBHX6fLfM= 2022-09-03T06:18:12.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Xkje3fVUjJhzgTOqcbeBHX6fLfM=', 'Content-Length': '0'} 2022-09-03T06:18:12.555 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c6b02a371489b530-006312f1a4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.598 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:12.599 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:12.599 DEBUG:boto:Method: GET 2022-09-03T06:18:12.600 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:12.600 DEBUG:boto:Data: 2022-09-03T06:18:12.601 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.602 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:12.602 DEBUG:boto:Port: 8000 2022-09-03T06:18:12.603 DEBUG:boto:Params: {} 2022-09-03T06:18:12.604 DEBUG:boto:Token: None 2022-09-03T06:18:12.604 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/g 2022-09-03T06:18:12.605 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Xkje3fVUjJhzgTOqcbeBHX6fLfM= 2022-09-03T06:18:12.606 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Xkje3fVUjJhzgTOqcbeBHX6fLfM=', 'Content-Length': '0'} 2022-09-03T06:18:12.610 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006f7689f260dcb22c-006312f1a4-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.612 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:12.613 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:12.614 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:12.615 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:12.616 DEBUG:boto:Method: GET 2022-09-03T06:18:12.618 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:12.619 DEBUG:boto:Data: 2022-09-03T06:18:12.620 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.621 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:12.623 DEBUG:boto:Port: 8001 2022-09-03T06:18:12.624 DEBUG:boto:Params: {} 2022-09-03T06:18:12.626 DEBUG:boto:Token: None 2022-09-03T06:18:12.627 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/ 2022-09-03T06:18:12.628 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U= 2022-09-03T06:18:12.629 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U=', 'Content-Length': '0'} 2022-09-03T06:18:12.641 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007ee9de2b5c4c457f-006312f1a4-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.643 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:12.645 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:12.646 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:12.647 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:12.647 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:12.649 DEBUG:boto:Method: GET 2022-09-03T06:18:12.650 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:12.651 DEBUG:boto:Data: 2022-09-03T06:18:12.652 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.653 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:12.654 DEBUG:boto:Port: 8000 2022-09-03T06:18:12.655 DEBUG:boto:Params: {} 2022-09-03T06:18:12.656 DEBUG:boto:Token: None 2022-09-03T06:18:12.657 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/ 2022-09-03T06:18:12.658 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U= 2022-09-03T06:18:12.659 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U=', 'Content-Length': '0'} 2022-09-03T06:18:12.663 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eeca282cf9026b14-006312f1a4-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.664 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:12.665 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:18:12.666 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:12.667 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:12.668 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.669 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.670 DEBUG:boto:Method: GET 2022-09-03T06:18:12.672 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.674 DEBUG:boto:Data: 2022-09-03T06:18:12.674 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.675 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:12.675 DEBUG:boto:Port: 8001 2022-09-03T06:18:12.676 DEBUG:boto:Params: {} 2022-09-03T06:18:12.677 DEBUG:boto:Token: None 2022-09-03T06:18:12.677 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.678 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xHTPH1YsyOF1SBTTh2LwasxhgEE= 2022-09-03T06:18:12.679 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xHTPH1YsyOF1SBTTh2LwasxhgEE=', 'Content-Length': '0'} 2022-09-03T06:18:12.683 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003607fa6b3374204b-006312f1a4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.685 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:12.686 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:12.687 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.688 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.688 DEBUG:boto:Method: GET 2022-09-03T06:18:12.689 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.689 DEBUG:boto:Data: 2022-09-03T06:18:12.690 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.690 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:12.691 DEBUG:boto:Port: 8000 2022-09-03T06:18:12.691 DEBUG:boto:Params: {} 2022-09-03T06:18:12.692 DEBUG:boto:Token: None 2022-09-03T06:18:12.693 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.693 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xHTPH1YsyOF1SBTTh2LwasxhgEE= 2022-09-03T06:18:12.695 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xHTPH1YsyOF1SBTTh2LwasxhgEE=', 'Content-Length': '0'} 2022-09-03T06:18:12.697 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000add4ddfb2d225b0c-006312f1a4-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.698 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:12.698 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:12.699 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.700 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.700 DEBUG:boto:Method: HEAD 2022-09-03T06:18:12.701 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.701 DEBUG:boto:Data: 2022-09-03T06:18:12.702 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.702 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:12.703 DEBUG:boto:Port: 8001 2022-09-03T06:18:12.703 DEBUG:boto:Params: {} 2022-09-03T06:18:12.704 DEBUG:boto:Token: None 2022-09-03T06:18:12.704 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.705 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FnLAAPOsaydkJc7R8KDnYDABmU0= 2022-09-03T06:18:12.705 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FnLAAPOsaydkJc7R8KDnYDABmU0=', 'Content-Length': '0'} 2022-09-03T06:18:12.708 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f1f0eda8394ce4a-006312f1a4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.709 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:12.710 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:12.710 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.711 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.711 DEBUG:boto:Method: HEAD 2022-09-03T06:18:12.712 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.712 DEBUG:boto:Data: 2022-09-03T06:18:12.713 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.713 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:12.714 DEBUG:boto:Port: 8000 2022-09-03T06:18:12.714 DEBUG:boto:Params: {} 2022-09-03T06:18:12.715 DEBUG:boto:Token: None 2022-09-03T06:18:12.715 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.716 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FnLAAPOsaydkJc7R8KDnYDABmU0= 2022-09-03T06:18:12.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FnLAAPOsaydkJc7R8KDnYDABmU0=', 'Content-Length': '0'} 2022-09-03T06:18:12.721 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000007d5309ddb80efb5-006312f1a4-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.722 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:18:12.722 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:12.723 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:12.723 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.724 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.724 DEBUG:boto:Method: GET 2022-09-03T06:18:12.725 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.725 DEBUG:boto:Data: 2022-09-03T06:18:12.726 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.727 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:12.727 DEBUG:boto:Port: 8001 2022-09-03T06:18:12.728 DEBUG:boto:Params: {} 2022-09-03T06:18:12.728 DEBUG:boto:Token: None 2022-09-03T06:18:12.729 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.730 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xHTPH1YsyOF1SBTTh2LwasxhgEE= 2022-09-03T06:18:12.730 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xHTPH1YsyOF1SBTTh2LwasxhgEE=', 'Content-Length': '0'} 2022-09-03T06:18:12.734 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dd2348defa80a0c7-006312f1a4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.735 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:12.735 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:12.736 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.736 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.737 DEBUG:boto:Method: GET 2022-09-03T06:18:12.737 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.739 DEBUG:boto:Data: 2022-09-03T06:18:12.739 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.740 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:12.740 DEBUG:boto:Port: 8000 2022-09-03T06:18:12.741 DEBUG:boto:Params: {} 2022-09-03T06:18:12.741 DEBUG:boto:Token: None 2022-09-03T06:18:12.744 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:12.745 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xHTPH1YsyOF1SBTTh2LwasxhgEE= 2022-09-03T06:18:12.746 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xHTPH1YsyOF1SBTTh2LwasxhgEE=', 'Content-Length': '0'} 2022-09-03T06:18:12.748 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000017e42cf62af77ce9-006312f1a4-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.749 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:12.750 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:12.750 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:12.751 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:12.752 DEBUG:boto:Method: GET 2022-09-03T06:18:12.755 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:12.755 DEBUG:boto:Data: 2022-09-03T06:18:12.756 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.757 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:12.758 DEBUG:boto:Port: 8001 2022-09-03T06:18:12.759 DEBUG:boto:Params: {} 2022-09-03T06:18:12.760 DEBUG:boto:Token: None 2022-09-03T06:18:12.761 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/ 2022-09-03T06:18:12.762 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U= 2022-09-03T06:18:12.765 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U=', 'Content-Length': '0'} 2022-09-03T06:18:12.774 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e36b34be98fe75f9-006312f1a4-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.775 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:12.776 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:12.776 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:12.777 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:12.777 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:12.778 DEBUG:boto:Method: GET 2022-09-03T06:18:12.778 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:12.779 DEBUG:boto:Data: 2022-09-03T06:18:12.779 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.780 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:12.780 DEBUG:boto:Port: 8000 2022-09-03T06:18:12.781 DEBUG:boto:Params: {} 2022-09-03T06:18:12.782 DEBUG:boto:Token: None 2022-09-03T06:18:12.782 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/ 2022-09-03T06:18:12.783 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U= 2022-09-03T06:18:12.784 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U=', 'Content-Length': '0'} 2022-09-03T06:18:12.786 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005f1da554aa2db1bb-006312f1a4-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.787 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:12.788 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:12.789 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:12.789 DEBUG:boto:Method: HEAD 2022-09-03T06:18:12.790 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:12.790 DEBUG:boto:Data: 2022-09-03T06:18:12.791 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.791 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:12.792 DEBUG:boto:Port: 8001 2022-09-03T06:18:12.792 DEBUG:boto:Params: {} 2022-09-03T06:18:12.793 DEBUG:boto:Token: None 2022-09-03T06:18:12.794 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/h 2022-09-03T06:18:12.794 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ym/i/ZlD9EmmugsqwXY8HFDftFA= 2022-09-03T06:18:12.795 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ym/i/ZlD9EmmugsqwXY8HFDftFA=', 'Content-Length': '0'} 2022-09-03T06:18:12.797 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fb4a39051ea61ffc-006312f1a4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.798 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:12.798 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:12.799 DEBUG:boto:Method: HEAD 2022-09-03T06:18:12.799 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:12.800 DEBUG:boto:Data: 2022-09-03T06:18:12.800 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.801 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:12.802 DEBUG:boto:Port: 8000 2022-09-03T06:18:12.802 DEBUG:boto:Params: {} 2022-09-03T06:18:12.803 DEBUG:boto:Token: None 2022-09-03T06:18:12.803 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/h 2022-09-03T06:18:12.804 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ym/i/ZlD9EmmugsqwXY8HFDftFA= 2022-09-03T06:18:12.805 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ym/i/ZlD9EmmugsqwXY8HFDftFA=', 'Content-Length': '0'} 2022-09-03T06:18:12.807 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c78612bbc85ab9d8-006312f1a4-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.808 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:18:12.810 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:12.811 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:12.811 DEBUG:boto:Method: GET 2022-09-03T06:18:12.812 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:12.812 DEBUG:boto:Data: 2022-09-03T06:18:12.813 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.813 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:12.814 DEBUG:boto:Port: 8001 2022-09-03T06:18:12.814 DEBUG:boto:Params: {} 2022-09-03T06:18:12.815 DEBUG:boto:Token: None 2022-09-03T06:18:12.816 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/h 2022-09-03T06:18:12.816 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h8w0IcWafqFFcn6fOC1zJVRJIJo= 2022-09-03T06:18:12.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h8w0IcWafqFFcn6fOC1zJVRJIJo=', 'Content-Length': '0'} 2022-09-03T06:18:12.819 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000090024476c9b30ea8-006312f1a4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.867 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:12.869 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:12.870 DEBUG:boto:Method: GET 2022-09-03T06:18:12.871 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:12.873 DEBUG:boto:Data: 2022-09-03T06:18:12.874 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.874 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:12.875 DEBUG:boto:Port: 8000 2022-09-03T06:18:12.877 DEBUG:boto:Params: {} 2022-09-03T06:18:12.879 DEBUG:boto:Token: None 2022-09-03T06:18:12.880 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/h 2022-09-03T06:18:12.881 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h8w0IcWafqFFcn6fOC1zJVRJIJo= 2022-09-03T06:18:12.884 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h8w0IcWafqFFcn6fOC1zJVRJIJo=', 'Content-Length': '0'} 2022-09-03T06:18:12.900 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000e27c517f15a911b-006312f1a4-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.901 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:12.903 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:12.904 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:12.904 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:12.905 DEBUG:boto:Method: GET 2022-09-03T06:18:12.906 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:12.907 DEBUG:boto:Data: 2022-09-03T06:18:12.907 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.908 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:12.908 DEBUG:boto:Port: 8001 2022-09-03T06:18:12.909 DEBUG:boto:Params: {} 2022-09-03T06:18:12.909 DEBUG:boto:Token: None 2022-09-03T06:18:12.910 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/ 2022-09-03T06:18:12.911 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U= 2022-09-03T06:18:12.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U=', 'Content-Length': '0'} 2022-09-03T06:18:12.914 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d05819064e48f593-006312f1a4-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.915 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:12.921 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:12.921 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:12.922 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:12.922 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:12.923 DEBUG:boto:Method: GET 2022-09-03T06:18:12.924 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:12.924 DEBUG:boto:Data: 2022-09-03T06:18:12.925 DEBUG:boto:Headers: {} 2022-09-03T06:18:12.925 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:12.926 DEBUG:boto:Port: 8000 2022-09-03T06:18:12.926 DEBUG:boto:Params: {} 2022-09-03T06:18:12.927 DEBUG:boto:Token: None 2022-09-03T06:18:12.928 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:12 GMT /dsgvcy-71/ 2022-09-03T06:18:12.928 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U= 2022-09-03T06:18:12.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cfYX4pWU6rN5u7gdHUdWfrLbR9U=', 'Content-Length': '0'} 2022-09-03T06:18:12.937 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e12a61f24662df0c-006312f1a4-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:12.941 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:12.942 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-71 zones={test-zone3, test-zone2} 2022-09-03T06:18:12.943 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--bucket', 'dsgvcy-71', '--num-shards', '3', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:18:12.944 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--bucket', 'dsgvcy-71', '--num-shards', '3', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:18:12.945 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --bucket dsgvcy-71 --num-shards 3 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T06:18:13.010 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:13.011 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:13.067 INFO:teuthology.orchestra.run.smithi181.stdout:tenant: 2022-09-03T06:18:13.067 INFO:teuthology.orchestra.run.smithi181.stdout:bucket name: dsgvcy-71 2022-09-03T06:18:13.082 INFO:teuthology.orchestra.run.smithi181.stdout:total entries: 8 2022-09-03T06:18:13.103 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'layout', '--bucket', 'dsgvcy-71', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:18:13.103 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'layout', '--bucket', 'dsgvcy-71', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:18:13.104 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket layout --bucket dsgvcy-71 --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T06:18:13.151 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:13.152 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:13.208 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T06:18:13.208 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:18:13.208 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding": "None", 2022-09-03T06:18:13.209 INFO:teuthology.orchestra.run.smithi181.stdout: "current_index": { 2022-09-03T06:18:13.209 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 2, 2022-09-03T06:18:13.209 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:18:13.211 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "Normal", 2022-09-03T06:18:13.212 INFO:teuthology.orchestra.run.smithi181.stdout: "normal": { 2022-09-03T06:18:13.212 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 3, 2022-09-03T06:18:13.213 INFO:teuthology.orchestra.run.smithi181.stdout: "hash_type": "Mod" 2022-09-03T06:18:13.214 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:13.215 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:13.215 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:18:13.216 INFO:teuthology.orchestra.run.smithi181.stdout: "logs": [ 2022-09-03T06:18:13.216 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:18:13.217 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 0, 2022-09-03T06:18:13.217 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:18:13.218 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "InIndex", 2022-09-03T06:18:13.219 INFO:teuthology.orchestra.run.smithi181.stdout: "in_index": { 2022-09-03T06:18:13.219 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 0, 2022-09-03T06:18:13.220 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:18:13.220 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 11, 2022-09-03T06:18:13.220 INFO:teuthology.orchestra.run.smithi181.stdout: "hash_type": "Mod" 2022-09-03T06:18:13.221 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:13.222 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:13.223 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:13.223 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:18:13.224 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:18:13.224 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 1, 2022-09-03T06:18:13.225 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:18:13.225 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "InIndex", 2022-09-03T06:18:13.227 INFO:teuthology.orchestra.run.smithi181.stdout: "in_index": { 2022-09-03T06:18:13.228 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 1, 2022-09-03T06:18:13.228 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:18:13.229 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 3, 2022-09-03T06:18:13.229 INFO:teuthology.orchestra.run.smithi181.stdout: "hash_type": "Mod" 2022-09-03T06:18:13.230 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:13.230 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:13.231 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:13.234 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:18:13.235 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:18:13.235 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 2, 2022-09-03T06:18:13.235 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:18:13.236 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "InIndex", 2022-09-03T06:18:13.236 INFO:teuthology.orchestra.run.smithi181.stdout: "in_index": { 2022-09-03T06:18:13.236 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 2, 2022-09-03T06:18:13.236 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:18:13.237 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 3, 2022-09-03T06:18:13.237 INFO:teuthology.orchestra.run.smithi181.stdout: "hash_type": "Mod" 2022-09-03T06:18:13.237 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:13.238 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:13.238 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:13.239 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:13.239 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:18:13.240 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:13.240 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T06:18:13.241 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-71 2022-09-03T06:18:13.243 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:13.244 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:13.247 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-71 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:18:13.316 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:13.317 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:13.357 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:18:13.355+0000 7f9403a04600 1 waiting to reach latest gen 2, on 1.. 2022-09-03T06:18:18.359 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:18:18.356+0000 7f9403a04600 1 waiting to reach latest gen 2, on 1.. 2022-09-03T06:18:23.359 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:18:23.356+0000 7f9403a04600 1 bucket sync caught up with empty source 2022-09-03T06:18:23.359 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:18:23.356+0000 7f9403a04600 0 bucket checkpoint complete 2022-09-03T06:18:23.385 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-71 2022-09-03T06:18:23.386 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:23.387 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:23.388 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-71 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:18:23.458 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:23.459 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:23.498 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:18:23.489+0000 7f52ab1f8600 1 bucket sync caught up with empty source 2022-09-03T06:18:23.499 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:18:23.489+0000 7f52ab1f8600 0 bucket checkpoint complete 2022-09-03T06:18:23.512 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-71 2022-09-03T06:18:23.513 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:23.515 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:23.516 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-71 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:18:23.582 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:23.583 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:23.633 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:18:23.629+0000 7fd37c510600 1 bucket sync caught up with empty source 2022-09-03T06:18:23.634 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:18:23.629+0000 7fd37c510600 0 bucket checkpoint complete 2022-09-03T06:18:23.651 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-71 2022-09-03T06:18:23.652 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:23.653 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:23.658 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-71 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:18:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:23.718 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:23.765 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:18:23.762+0000 7f44337c0600 1 bucket sync caught up with source: 2022-09-03T06:18:23.765 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.736.5, , , 00000000001.284.5, 00000000001.685.5, , , 00000000002.339.5, , 00000000002.422.5] 2022-09-03T06:18:23.765 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.736.5, , , 00000000001.284.5, 00000000001.685.5, , , 00000000002.339.5, , 00000000002.422.5] 2022-09-03T06:18:23.766 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:18:23.762+0000 7f44337c0600 0 bucket checkpoint complete 2022-09-03T06:18:23.780 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-71 2022-09-03T06:18:23.781 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:23.783 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:23.784 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-71 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:18:23.850 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:23.851 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:23.883 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:18:23.881+0000 7ff33cd68600 1 bucket sync caught up with source: 2022-09-03T06:18:23.884 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.736.5, , , 00000000001.284.5, 00000000001.685.5, , , 00000000002.339.5, , 00000000002.422.5] 2022-09-03T06:18:23.884 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.736.5, , , 00000000001.284.5, 00000000001.685.5, , , 00000000002.339.5, , 00000000002.422.5] 2022-09-03T06:18:23.885 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:18:23.881+0000 7ff33cd68600 0 bucket checkpoint complete 2022-09-03T06:18:23.904 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-71 2022-09-03T06:18:23.905 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-71 2022-09-03T06:18:23.906 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:23.907 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:23.908 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-71 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:18:23.973 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:23.973 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:24.018 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:18:24.015+0000 7fada1799600 1 bucket sync caught up with source: 2022-09-03T06:18:24.018 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.343.5, , , 00000000001.219.5, 00000000001.244.5, , , 00000000002.246.5, , 00000000002.185.5] 2022-09-03T06:18:24.019 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.343.5, , , 00000000001.219.5, 00000000001.244.5, , , 00000000002.246.5, , 00000000002.185.5] 2022-09-03T06:18:24.020 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:18:24.015+0000 7fada1799600 0 bucket checkpoint complete 2022-09-03T06:18:24.036 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-71 2022-09-03T06:18:24.037 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:24.038 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-71', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:24.040 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-71 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:18:24.102 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:24.102 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:24.141 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:18:24.137+0000 7faf6d60b600 1 bucket sync caught up with source: 2022-09-03T06:18:24.141 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000002.343.5, , , 00000000001.219.5, 00000000001.244.5, , , 00000000002.246.5, , 00000000002.185.5] 2022-09-03T06:18:24.142 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000002.343.5, , , 00000000001.219.5, 00000000001.244.5, , , 00000000002.246.5, , 00000000002.185.5] 2022-09-03T06:18:24.143 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:18:24.137+0000 7faf6d60b600 0 bucket checkpoint complete 2022-09-03T06:18:24.158 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-71 2022-09-03T06:18:24.159 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-71 zones={test-zone2, test-zone1} 2022-09-03T06:18:24.160 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:24.160 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:24.161 DEBUG:boto:Method: HEAD 2022-09-03T06:18:24.161 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:18:24.162 DEBUG:boto:Data: 2022-09-03T06:18:24.163 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.163 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:24.164 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.165 DEBUG:boto:Params: {} 2022-09-03T06:18:24.165 DEBUG:boto:Token: None 2022-09-03T06:18:24.166 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/ 2022-09-03T06:18:24.167 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ya136RgYrnekihRJGTTUsEjvpQE= 2022-09-03T06:18:24.167 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ya136RgYrnekihRJGTTUsEjvpQE=', 'Content-Length': '0'} 2022-09-03T06:18:24.170 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001ada4d9c78d1e1ee-006312f1b0-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.171 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:24.171 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:24.172 DEBUG:boto:Method: HEAD 2022-09-03T06:18:24.172 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:18:24.173 DEBUG:boto:Data: 2022-09-03T06:18:24.173 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.174 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:24.174 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.178 DEBUG:boto:Params: {} 2022-09-03T06:18:24.179 DEBUG:boto:Token: None 2022-09-03T06:18:24.180 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/ 2022-09-03T06:18:24.182 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ya136RgYrnekihRJGTTUsEjvpQE= 2022-09-03T06:18:24.182 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ya136RgYrnekihRJGTTUsEjvpQE=', 'Content-Length': '0'} 2022-09-03T06:18:24.185 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001880161b2056d13b-006312f1b0-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.186 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:18:24.189 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:24.190 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:24.191 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:24.192 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:24.193 DEBUG:boto:Method: GET 2022-09-03T06:18:24.194 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:24.195 DEBUG:boto:Data: 2022-09-03T06:18:24.196 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.197 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:24.199 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.199 DEBUG:boto:Params: {} 2022-09-03T06:18:24.200 DEBUG:boto:Token: None 2022-09-03T06:18:24.201 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/?versions 2022-09-03T06:18:24.201 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rwHjaeHSWLYoYD8ZFt1HQ0ZTzME= 2022-09-03T06:18:24.202 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rwHjaeHSWLYoYD8ZFt1HQ0ZTzME=', 'Content-Length': '0'} 2022-09-03T06:18:24.204 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000093a43409b41a8be0-006312f1b0-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.205 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:24.207 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:18:24.207 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:18:24.208 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:18:24.209 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:18:24.209 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:18:24.210 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:18:24.211 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:18:24.211 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:18:24.212 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:18:24.213 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:24.213 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:24.214 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:24.215 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:24.215 DEBUG:boto:Method: GET 2022-09-03T06:18:24.216 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:24.216 DEBUG:boto:Data: 2022-09-03T06:18:24.217 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.217 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:24.218 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.218 DEBUG:boto:Params: {} 2022-09-03T06:18:24.219 DEBUG:boto:Token: None 2022-09-03T06:18:24.220 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/?versions 2022-09-03T06:18:24.220 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rwHjaeHSWLYoYD8ZFt1HQ0ZTzME= 2022-09-03T06:18:24.221 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rwHjaeHSWLYoYD8ZFt1HQ0ZTzME=', 'Content-Length': '0'} 2022-09-03T06:18:24.232 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000040d7e5b438f4cb23-006312f1b0-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.233 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:24.236 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:18:24.237 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:18:24.239 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:18:24.240 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:18:24.240 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:18:24.242 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:18:24.243 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:18:24.243 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:18:24.244 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:24.245 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:24.245 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:24.246 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:24.246 DEBUG:boto:Method: GET 2022-09-03T06:18:24.247 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:24.247 DEBUG:boto:Data: 2022-09-03T06:18:24.248 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.249 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:24.249 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.250 DEBUG:boto:Params: {} 2022-09-03T06:18:24.250 DEBUG:boto:Token: None 2022-09-03T06:18:24.251 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/?versions 2022-09-03T06:18:24.252 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rwHjaeHSWLYoYD8ZFt1HQ0ZTzME= 2022-09-03T06:18:24.253 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rwHjaeHSWLYoYD8ZFt1HQ0ZTzME=', 'Content-Length': '0'} 2022-09-03T06:18:24.257 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c09dc0bae73c7b78-006312f1b0-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.257 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:24.259 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:24.260 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:24.261 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:24.262 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:24.263 DEBUG:boto:Method: GET 2022-09-03T06:18:24.263 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:24.264 DEBUG:boto:Data: 2022-09-03T06:18:24.264 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.265 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:24.266 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.268 DEBUG:boto:Params: {} 2022-09-03T06:18:24.269 DEBUG:boto:Token: None 2022-09-03T06:18:24.278 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/?versions 2022-09-03T06:18:24.279 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rwHjaeHSWLYoYD8ZFt1HQ0ZTzME= 2022-09-03T06:18:24.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rwHjaeHSWLYoYD8ZFt1HQ0ZTzME=', 'Content-Length': '0'} 2022-09-03T06:18:24.283 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005f676430fc29719e-006312f1b0-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.285 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:24.288 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:18:24.289 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:24.290 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:24.291 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.292 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.293 DEBUG:boto:Method: GET 2022-09-03T06:18:24.294 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.297 DEBUG:boto:Data: 2022-09-03T06:18:24.298 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.299 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:24.300 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.301 DEBUG:boto:Params: {} 2022-09-03T06:18:24.302 DEBUG:boto:Token: None 2022-09-03T06:18:24.304 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.304 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JOR/+wRNQwlU/JNXz8jjpvXPxvw= 2022-09-03T06:18:24.305 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JOR/+wRNQwlU/JNXz8jjpvXPxvw=', 'Content-Length': '0'} 2022-09-03T06:18:24.308 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008a74b0593bfade49-006312f1b0-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.309 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:24.309 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:24.310 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.311 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.311 DEBUG:boto:Method: GET 2022-09-03T06:18:24.312 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.312 DEBUG:boto:Data: 2022-09-03T06:18:24.313 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.314 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:24.315 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.320 DEBUG:boto:Params: {} 2022-09-03T06:18:24.321 DEBUG:boto:Token: None 2022-09-03T06:18:24.323 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.325 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JOR/+wRNQwlU/JNXz8jjpvXPxvw= 2022-09-03T06:18:24.326 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JOR/+wRNQwlU/JNXz8jjpvXPxvw=', 'Content-Length': '0'} 2022-09-03T06:18:24.333 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ac0e542591364103-006312f1b0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.334 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:24.334 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:24.335 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.336 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.336 DEBUG:boto:Method: HEAD 2022-09-03T06:18:24.337 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.337 DEBUG:boto:Data: 2022-09-03T06:18:24.338 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.338 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:24.339 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.339 DEBUG:boto:Params: {} 2022-09-03T06:18:24.340 DEBUG:boto:Token: None 2022-09-03T06:18:24.340 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.342 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0VbeZz8CmZ0nd47ZOthq9CQGUxI= 2022-09-03T06:18:24.342 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0VbeZz8CmZ0nd47ZOthq9CQGUxI=', 'Content-Length': '0'} 2022-09-03T06:18:24.346 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000070ca1c4b86e88c42-006312f1b0-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.347 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:24.348 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:24.348 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.349 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.349 DEBUG:boto:Method: HEAD 2022-09-03T06:18:24.350 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.351 DEBUG:boto:Data: 2022-09-03T06:18:24.351 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.352 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:24.352 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.353 DEBUG:boto:Params: {} 2022-09-03T06:18:24.353 DEBUG:boto:Token: None 2022-09-03T06:18:24.354 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.355 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0VbeZz8CmZ0nd47ZOthq9CQGUxI= 2022-09-03T06:18:24.355 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0VbeZz8CmZ0nd47ZOthq9CQGUxI=', 'Content-Length': '0'} 2022-09-03T06:18:24.357 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000061feb7baaba7382b-006312f1b0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.358 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:18:24.359 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:24.359 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:24.360 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.360 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.361 DEBUG:boto:Method: GET 2022-09-03T06:18:24.361 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.362 DEBUG:boto:Data: 2022-09-03T06:18:24.363 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.363 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:24.364 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.364 DEBUG:boto:Params: {} 2022-09-03T06:18:24.365 DEBUG:boto:Token: None 2022-09-03T06:18:24.366 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JOR/+wRNQwlU/JNXz8jjpvXPxvw= 2022-09-03T06:18:24.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JOR/+wRNQwlU/JNXz8jjpvXPxvw=', 'Content-Length': '0'} 2022-09-03T06:18:24.369 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c082d88c9aaa301-006312f1b0-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.371 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:24.372 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:24.373 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.373 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.374 DEBUG:boto:Method: GET 2022-09-03T06:18:24.374 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.375 DEBUG:boto:Data: 2022-09-03T06:18:24.375 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.376 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:24.377 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.377 DEBUG:boto:Params: {} 2022-09-03T06:18:24.378 DEBUG:boto:Token: None 2022-09-03T06:18:24.379 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:24.379 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JOR/+wRNQwlU/JNXz8jjpvXPxvw= 2022-09-03T06:18:24.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JOR/+wRNQwlU/JNXz8jjpvXPxvw=', 'Content-Length': '0'} 2022-09-03T06:18:24.382 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000160bd3acecbd0f45-006312f1b0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.383 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:24.383 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:24.384 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:24.384 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:24.385 DEBUG:boto:Method: GET 2022-09-03T06:18:24.385 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:24.386 DEBUG:boto:Data: 2022-09-03T06:18:24.387 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.387 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:24.388 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.388 DEBUG:boto:Params: {} 2022-09-03T06:18:24.389 DEBUG:boto:Token: None 2022-09-03T06:18:24.389 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/ 2022-09-03T06:18:24.390 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hyq/AjV/rgZu3l4EkllGa0vdGRU= 2022-09-03T06:18:24.391 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hyq/AjV/rgZu3l4EkllGa0vdGRU=', 'Content-Length': '0'} 2022-09-03T06:18:24.393 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c0e2224df8eee93c-006312f1b0-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.394 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:24.395 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:24.396 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:24.396 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:24.397 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:24.397 DEBUG:boto:Method: GET 2022-09-03T06:18:24.398 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:24.398 DEBUG:boto:Data: 2022-09-03T06:18:24.399 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.399 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:24.400 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.400 DEBUG:boto:Params: {} 2022-09-03T06:18:24.401 DEBUG:boto:Token: None 2022-09-03T06:18:24.402 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/ 2022-09-03T06:18:24.402 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hyq/AjV/rgZu3l4EkllGa0vdGRU= 2022-09-03T06:18:24.403 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hyq/AjV/rgZu3l4EkllGa0vdGRU=', 'Content-Length': '0'} 2022-09-03T06:18:24.405 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000071bf5441496d9c5a-006312f1b0-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.446 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:24.447 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:24.448 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:24.448 DEBUG:boto:Method: HEAD 2022-09-03T06:18:24.449 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:24.449 DEBUG:boto:Data: 2022-09-03T06:18:24.450 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.451 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:24.451 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.452 DEBUG:boto:Params: {} 2022-09-03T06:18:24.452 DEBUG:boto:Token: None 2022-09-03T06:18:24.453 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/a 2022-09-03T06:18:24.454 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I2ZldxpK+uFVk0jMxtGkAtPawBI= 2022-09-03T06:18:24.454 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I2ZldxpK+uFVk0jMxtGkAtPawBI=', 'Content-Length': '0'} 2022-09-03T06:18:24.456 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000711ed9197c048a25-006312f1b0-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.457 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:24.458 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:24.458 DEBUG:boto:Method: HEAD 2022-09-03T06:18:24.459 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:24.459 DEBUG:boto:Data: 2022-09-03T06:18:24.460 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.460 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:24.461 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.462 DEBUG:boto:Params: {} 2022-09-03T06:18:24.462 DEBUG:boto:Token: None 2022-09-03T06:18:24.463 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/a 2022-09-03T06:18:24.463 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I2ZldxpK+uFVk0jMxtGkAtPawBI= 2022-09-03T06:18:24.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I2ZldxpK+uFVk0jMxtGkAtPawBI=', 'Content-Length': '0'} 2022-09-03T06:18:24.466 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000030f1d207e46183d0-006312f1b0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.467 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:18:24.468 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:24.468 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:24.469 DEBUG:boto:Method: GET 2022-09-03T06:18:24.470 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:24.471 DEBUG:boto:Data: 2022-09-03T06:18:24.472 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.472 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:24.473 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.474 DEBUG:boto:Params: {} 2022-09-03T06:18:24.474 DEBUG:boto:Token: None 2022-09-03T06:18:24.475 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/a 2022-09-03T06:18:24.476 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L8h8nhJ4HjZcgs0LatdmLtZILV4= 2022-09-03T06:18:24.477 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L8h8nhJ4HjZcgs0LatdmLtZILV4=', 'Content-Length': '0'} 2022-09-03T06:18:24.479 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000023101e85e247ca59-006312f1b0-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.479 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:24.480 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:24.481 DEBUG:boto:Method: GET 2022-09-03T06:18:24.481 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:24.482 DEBUG:boto:Data: 2022-09-03T06:18:24.482 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.483 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:24.483 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.484 DEBUG:boto:Params: {} 2022-09-03T06:18:24.485 DEBUG:boto:Token: None 2022-09-03T06:18:24.485 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/a 2022-09-03T06:18:24.486 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L8h8nhJ4HjZcgs0LatdmLtZILV4= 2022-09-03T06:18:24.487 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L8h8nhJ4HjZcgs0LatdmLtZILV4=', 'Content-Length': '0'} 2022-09-03T06:18:24.489 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000084804af0f371501e-006312f1b0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.490 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:24.491 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:24.491 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:24.492 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:24.493 DEBUG:boto:Method: GET 2022-09-03T06:18:24.493 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:24.494 DEBUG:boto:Data: 2022-09-03T06:18:24.495 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.495 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:24.496 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.496 DEBUG:boto:Params: {} 2022-09-03T06:18:24.497 DEBUG:boto:Token: None 2022-09-03T06:18:24.498 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/ 2022-09-03T06:18:24.498 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hyq/AjV/rgZu3l4EkllGa0vdGRU= 2022-09-03T06:18:24.499 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hyq/AjV/rgZu3l4EkllGa0vdGRU=', 'Content-Length': '0'} 2022-09-03T06:18:24.501 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000962ac49ccafb91f5-006312f1b0-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.542 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:24.543 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:24.544 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:24.545 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:24.545 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:24.546 DEBUG:boto:Method: GET 2022-09-03T06:18:24.546 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:24.547 DEBUG:boto:Data: 2022-09-03T06:18:24.547 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.548 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:24.548 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.549 DEBUG:boto:Params: {} 2022-09-03T06:18:24.550 DEBUG:boto:Token: None 2022-09-03T06:18:24.550 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/ 2022-09-03T06:18:24.551 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hyq/AjV/rgZu3l4EkllGa0vdGRU= 2022-09-03T06:18:24.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hyq/AjV/rgZu3l4EkllGa0vdGRU=', 'Content-Length': '0'} 2022-09-03T06:18:24.554 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000059982a94c3c7f091-006312f1b0-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.555 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:24.556 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:18:24.556 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:24.557 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:24.557 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.558 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.559 DEBUG:boto:Method: GET 2022-09-03T06:18:24.559 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.560 DEBUG:boto:Data: 2022-09-03T06:18:24.560 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.561 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:24.561 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.562 DEBUG:boto:Params: {} 2022-09-03T06:18:24.562 DEBUG:boto:Token: None 2022-09-03T06:18:24.563 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.564 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BiNJLk8FtH7Eq7EsiYxrjz2b5TY= 2022-09-03T06:18:24.565 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BiNJLk8FtH7Eq7EsiYxrjz2b5TY=', 'Content-Length': '0'} 2022-09-03T06:18:24.567 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004dc8be0d3548ed23-006312f1b0-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.568 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:24.569 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:24.569 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.570 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.570 DEBUG:boto:Method: GET 2022-09-03T06:18:24.571 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.571 DEBUG:boto:Data: 2022-09-03T06:18:24.572 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.573 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:24.573 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.574 DEBUG:boto:Params: {} 2022-09-03T06:18:24.574 DEBUG:boto:Token: None 2022-09-03T06:18:24.575 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.576 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BiNJLk8FtH7Eq7EsiYxrjz2b5TY= 2022-09-03T06:18:24.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BiNJLk8FtH7Eq7EsiYxrjz2b5TY=', 'Content-Length': '0'} 2022-09-03T06:18:24.579 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c04f7619c06a859e-006312f1b0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.622 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:24.624 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:24.624 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.625 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.625 DEBUG:boto:Method: HEAD 2022-09-03T06:18:24.627 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.627 DEBUG:boto:Data: 2022-09-03T06:18:24.628 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.629 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:24.630 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.630 DEBUG:boto:Params: {} 2022-09-03T06:18:24.631 DEBUG:boto:Token: None 2022-09-03T06:18:24.632 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.637 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iY24Wr4bCxySsShY7RoRHpvhPhY= 2022-09-03T06:18:24.638 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iY24Wr4bCxySsShY7RoRHpvhPhY=', 'Content-Length': '0'} 2022-09-03T06:18:24.641 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000bc932085cc31db2-006312f1b0-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.642 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:24.643 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:24.643 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.644 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.644 DEBUG:boto:Method: HEAD 2022-09-03T06:18:24.645 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.645 DEBUG:boto:Data: 2022-09-03T06:18:24.646 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.647 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:24.648 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.648 DEBUG:boto:Params: {} 2022-09-03T06:18:24.649 DEBUG:boto:Token: None 2022-09-03T06:18:24.650 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.652 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iY24Wr4bCxySsShY7RoRHpvhPhY= 2022-09-03T06:18:24.653 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iY24Wr4bCxySsShY7RoRHpvhPhY=', 'Content-Length': '0'} 2022-09-03T06:18:24.656 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018f8e80e16b2994f-006312f1b0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.660 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:18:24.661 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:24.662 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:24.663 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.664 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.665 DEBUG:boto:Method: GET 2022-09-03T06:18:24.666 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.667 DEBUG:boto:Data: 2022-09-03T06:18:24.668 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.669 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:24.670 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.671 DEBUG:boto:Params: {} 2022-09-03T06:18:24.672 DEBUG:boto:Token: None 2022-09-03T06:18:24.672 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.679 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BiNJLk8FtH7Eq7EsiYxrjz2b5TY= 2022-09-03T06:18:24.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BiNJLk8FtH7Eq7EsiYxrjz2b5TY=', 'Content-Length': '0'} 2022-09-03T06:18:24.682 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000072d3f92ca9c98a63-006312f1b0-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.686 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:24.687 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:24.687 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.688 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.688 DEBUG:boto:Method: GET 2022-09-03T06:18:24.689 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.689 DEBUG:boto:Data: 2022-09-03T06:18:24.690 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.691 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:24.691 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.692 DEBUG:boto:Params: {} 2022-09-03T06:18:24.692 DEBUG:boto:Token: None 2022-09-03T06:18:24.693 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:24.694 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BiNJLk8FtH7Eq7EsiYxrjz2b5TY= 2022-09-03T06:18:24.695 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BiNJLk8FtH7Eq7EsiYxrjz2b5TY=', 'Content-Length': '0'} 2022-09-03T06:18:24.697 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c9d31f1b5ff56ff5-006312f1b0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.698 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:24.698 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:24.699 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:24.700 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:24.700 DEBUG:boto:Method: GET 2022-09-03T06:18:24.701 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:24.702 DEBUG:boto:Data: 2022-09-03T06:18:24.703 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.704 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:24.705 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.707 DEBUG:boto:Params: {} 2022-09-03T06:18:24.708 DEBUG:boto:Token: None 2022-09-03T06:18:24.709 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/ 2022-09-03T06:18:24.711 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hyq/AjV/rgZu3l4EkllGa0vdGRU= 2022-09-03T06:18:24.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hyq/AjV/rgZu3l4EkllGa0vdGRU=', 'Content-Length': '0'} 2022-09-03T06:18:24.715 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008dc2bfd022753f13-006312f1b0-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.716 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:24.724 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:24.725 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:24.726 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:24.727 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:24.728 DEBUG:boto:Method: GET 2022-09-03T06:18:24.728 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:24.729 DEBUG:boto:Data: 2022-09-03T06:18:24.730 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.731 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:24.731 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.732 DEBUG:boto:Params: {} 2022-09-03T06:18:24.732 DEBUG:boto:Token: None 2022-09-03T06:18:24.733 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/ 2022-09-03T06:18:24.734 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hyq/AjV/rgZu3l4EkllGa0vdGRU= 2022-09-03T06:18:24.734 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hyq/AjV/rgZu3l4EkllGa0vdGRU=', 'Content-Length': '0'} 2022-09-03T06:18:24.738 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008cf83c044679c14b-006312f1b0-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.739 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:24.739 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:24.740 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:24.741 DEBUG:boto:Method: HEAD 2022-09-03T06:18:24.741 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:24.742 DEBUG:boto:Data: 2022-09-03T06:18:24.742 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.743 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:24.743 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.744 DEBUG:boto:Params: {} 2022-09-03T06:18:24.744 DEBUG:boto:Token: None 2022-09-03T06:18:24.745 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/b 2022-09-03T06:18:24.746 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:30PkFudoqji/qv0BlKtpKMfUEwo= 2022-09-03T06:18:24.747 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:30PkFudoqji/qv0BlKtpKMfUEwo=', 'Content-Length': '0'} 2022-09-03T06:18:24.750 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006c0217024a62639b-006312f1b0-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.751 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:24.751 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:24.752 DEBUG:boto:Method: HEAD 2022-09-03T06:18:24.753 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:24.753 DEBUG:boto:Data: 2022-09-03T06:18:24.754 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.755 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:24.756 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.756 DEBUG:boto:Params: {} 2022-09-03T06:18:24.757 DEBUG:boto:Token: None 2022-09-03T06:18:24.757 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/b 2022-09-03T06:18:24.758 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:30PkFudoqji/qv0BlKtpKMfUEwo= 2022-09-03T06:18:24.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:30PkFudoqji/qv0BlKtpKMfUEwo=', 'Content-Length': '0'} 2022-09-03T06:18:24.762 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f1fc384b03ba7107-006312f1b0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.764 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:18:24.764 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:24.766 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:24.767 DEBUG:boto:Method: GET 2022-09-03T06:18:24.767 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:24.768 DEBUG:boto:Data: 2022-09-03T06:18:24.770 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.771 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:24.772 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.773 DEBUG:boto:Params: {} 2022-09-03T06:18:24.774 DEBUG:boto:Token: None 2022-09-03T06:18:24.775 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/b 2022-09-03T06:18:24.777 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V0U5u2wElqZMiluLr9Ombsi7FOY= 2022-09-03T06:18:24.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V0U5u2wElqZMiluLr9Ombsi7FOY=', 'Content-Length': '0'} 2022-09-03T06:18:24.782 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006d3636c2cfb355a2-006312f1b0-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.787 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:24.788 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:24.789 DEBUG:boto:Method: GET 2022-09-03T06:18:24.789 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:24.790 DEBUG:boto:Data: 2022-09-03T06:18:24.791 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.792 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:24.792 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.794 DEBUG:boto:Params: {} 2022-09-03T06:18:24.795 DEBUG:boto:Token: None 2022-09-03T06:18:24.796 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/b 2022-09-03T06:18:24.797 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V0U5u2wElqZMiluLr9Ombsi7FOY= 2022-09-03T06:18:24.798 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V0U5u2wElqZMiluLr9Ombsi7FOY=', 'Content-Length': '0'} 2022-09-03T06:18:24.803 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000150cee8d33cfb4a5-006312f1b0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.805 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:24.806 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:24.814 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:24.815 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:24.816 DEBUG:boto:Method: GET 2022-09-03T06:18:24.816 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:24.817 DEBUG:boto:Data: 2022-09-03T06:18:24.817 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.819 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:24.820 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.821 DEBUG:boto:Params: {} 2022-09-03T06:18:24.821 DEBUG:boto:Token: None 2022-09-03T06:18:24.823 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/ 2022-09-03T06:18:24.824 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hyq/AjV/rgZu3l4EkllGa0vdGRU= 2022-09-03T06:18:24.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hyq/AjV/rgZu3l4EkllGa0vdGRU=', 'Content-Length': '0'} 2022-09-03T06:18:24.828 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d7a8daecbed9fd35-006312f1b0-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.871 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:24.872 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:24.873 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:24.874 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:24.875 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:24.875 DEBUG:boto:Method: GET 2022-09-03T06:18:24.876 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:24.877 DEBUG:boto:Data: 2022-09-03T06:18:24.877 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.878 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:24.878 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.879 DEBUG:boto:Params: {} 2022-09-03T06:18:24.879 DEBUG:boto:Token: None 2022-09-03T06:18:24.880 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/ 2022-09-03T06:18:24.881 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hyq/AjV/rgZu3l4EkllGa0vdGRU= 2022-09-03T06:18:24.881 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hyq/AjV/rgZu3l4EkllGa0vdGRU=', 'Content-Length': '0'} 2022-09-03T06:18:24.885 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000009a8b62e9ffec07f-006312f1b0-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.886 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:24.888 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:18:24.889 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:24.890 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:24.891 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.892 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.893 DEBUG:boto:Method: GET 2022-09-03T06:18:24.894 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.895 DEBUG:boto:Data: 2022-09-03T06:18:24.896 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.897 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:24.898 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.899 DEBUG:boto:Params: {} 2022-09-03T06:18:24.900 DEBUG:boto:Token: None 2022-09-03T06:18:24.901 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.903 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A0EC48jJMp1U5Kl2RrLxy5xaauc= 2022-09-03T06:18:24.906 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A0EC48jJMp1U5Kl2RrLxy5xaauc=', 'Content-Length': '0'} 2022-09-03T06:18:24.909 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a4cea47561e59bc-006312f1b0-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.910 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:24.911 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:24.911 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.912 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.912 DEBUG:boto:Method: GET 2022-09-03T06:18:24.913 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.913 DEBUG:boto:Data: 2022-09-03T06:18:24.914 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.915 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:24.915 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.916 DEBUG:boto:Params: {} 2022-09-03T06:18:24.916 DEBUG:boto:Token: None 2022-09-03T06:18:24.917 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.918 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A0EC48jJMp1U5Kl2RrLxy5xaauc= 2022-09-03T06:18:24.919 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A0EC48jJMp1U5Kl2RrLxy5xaauc=', 'Content-Length': '0'} 2022-09-03T06:18:24.921 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ec8796b2c152abd9-006312f1b0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.933 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:24.934 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:24.934 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.935 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.936 DEBUG:boto:Method: HEAD 2022-09-03T06:18:24.936 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.937 DEBUG:boto:Data: 2022-09-03T06:18:24.937 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.938 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:24.941 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.942 DEBUG:boto:Params: {} 2022-09-03T06:18:24.947 DEBUG:boto:Token: None 2022-09-03T06:18:24.948 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.949 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IWStshblpzw9Ftxs2q+ffKMl7fM= 2022-09-03T06:18:24.950 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IWStshblpzw9Ftxs2q+ffKMl7fM=', 'Content-Length': '0'} 2022-09-03T06:18:24.953 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003fb27dcbbff8859a-006312f1b0-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.955 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:24.955 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:24.956 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.957 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.958 DEBUG:boto:Method: HEAD 2022-09-03T06:18:24.959 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.960 DEBUG:boto:Data: 2022-09-03T06:18:24.960 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.961 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:24.961 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.962 DEBUG:boto:Params: {} 2022-09-03T06:18:24.963 DEBUG:boto:Token: None 2022-09-03T06:18:24.963 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.964 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IWStshblpzw9Ftxs2q+ffKMl7fM= 2022-09-03T06:18:24.965 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IWStshblpzw9Ftxs2q+ffKMl7fM=', 'Content-Length': '0'} 2022-09-03T06:18:24.967 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048a794e941a59778-006312f1b0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.968 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:18:24.969 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:24.969 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:24.970 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.970 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.971 DEBUG:boto:Method: GET 2022-09-03T06:18:24.971 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.973 DEBUG:boto:Data: 2022-09-03T06:18:24.973 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.974 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:24.975 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.976 DEBUG:boto:Params: {} 2022-09-03T06:18:24.977 DEBUG:boto:Token: None 2022-09-03T06:18:24.977 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.978 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A0EC48jJMp1U5Kl2RrLxy5xaauc= 2022-09-03T06:18:24.979 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A0EC48jJMp1U5Kl2RrLxy5xaauc=', 'Content-Length': '0'} 2022-09-03T06:18:24.981 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b899a57efc1c9e40-006312f1b0-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.982 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:24.983 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:24.984 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.984 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.985 DEBUG:boto:Method: GET 2022-09-03T06:18:24.986 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.986 DEBUG:boto:Data: 2022-09-03T06:18:24.987 DEBUG:boto:Headers: {} 2022-09-03T06:18:24.988 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:24.989 DEBUG:boto:Port: 8000 2022-09-03T06:18:24.989 DEBUG:boto:Params: {} 2022-09-03T06:18:24.990 DEBUG:boto:Token: None 2022-09-03T06:18:24.991 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:24 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:24.992 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A0EC48jJMp1U5Kl2RrLxy5xaauc= 2022-09-03T06:18:24.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A0EC48jJMp1U5Kl2RrLxy5xaauc=', 'Content-Length': '0'} 2022-09-03T06:18:24.994 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004d076b3490023048-006312f1b0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:24.995 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:24.996 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:24.996 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:24.997 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:24.998 DEBUG:boto:Method: GET 2022-09-03T06:18:24.998 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:24.999 DEBUG:boto:Data: 2022-09-03T06:18:24.999 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.000 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.000 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.001 DEBUG:boto:Params: {} 2022-09-03T06:18:25.002 DEBUG:boto:Token: None 2022-09-03T06:18:25.002 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/ 2022-09-03T06:18:25.003 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc= 2022-09-03T06:18:25.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc=', 'Content-Length': '0'} 2022-09-03T06:18:25.006 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000365d380feaa31faf-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.007 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:25.008 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:25.009 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:25.009 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:25.010 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:25.011 DEBUG:boto:Method: GET 2022-09-03T06:18:25.011 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:25.012 DEBUG:boto:Data: 2022-09-03T06:18:25.012 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.013 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.013 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.014 DEBUG:boto:Params: {} 2022-09-03T06:18:25.015 DEBUG:boto:Token: None 2022-09-03T06:18:25.015 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/ 2022-09-03T06:18:25.016 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc= 2022-09-03T06:18:25.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc=', 'Content-Length': '0'} 2022-09-03T06:18:25.020 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007cadf3200db3b107-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.062 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:25.064 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:25.065 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:25.065 DEBUG:boto:Method: HEAD 2022-09-03T06:18:25.066 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:25.067 DEBUG:boto:Data: 2022-09-03T06:18:25.068 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.069 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.069 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.070 DEBUG:boto:Params: {} 2022-09-03T06:18:25.070 DEBUG:boto:Token: None 2022-09-03T06:18:25.071 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/c 2022-09-03T06:18:25.072 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6Mb/QYBic/CoN3fuF/jO75KMZ48= 2022-09-03T06:18:25.072 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6Mb/QYBic/CoN3fuF/jO75KMZ48=', 'Content-Length': '0'} 2022-09-03T06:18:25.080 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000013c4d2da6ecb1896-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.081 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:25.082 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:25.082 DEBUG:boto:Method: HEAD 2022-09-03T06:18:25.083 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:25.083 DEBUG:boto:Data: 2022-09-03T06:18:25.084 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.084 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.085 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.085 DEBUG:boto:Params: {} 2022-09-03T06:18:25.086 DEBUG:boto:Token: None 2022-09-03T06:18:25.086 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/c 2022-09-03T06:18:25.087 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6Mb/QYBic/CoN3fuF/jO75KMZ48= 2022-09-03T06:18:25.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6Mb/QYBic/CoN3fuF/jO75KMZ48=', 'Content-Length': '0'} 2022-09-03T06:18:25.091 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e59240a3b35e1760-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.091 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:18:25.092 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:25.093 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:25.093 DEBUG:boto:Method: GET 2022-09-03T06:18:25.094 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:25.094 DEBUG:boto:Data: 2022-09-03T06:18:25.095 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.095 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.097 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.097 DEBUG:boto:Params: {} 2022-09-03T06:18:25.098 DEBUG:boto:Token: None 2022-09-03T06:18:25.098 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/c 2022-09-03T06:18:25.099 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4z/n5IQKE3jWAUJz2x9ckDP2kV4= 2022-09-03T06:18:25.100 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4z/n5IQKE3jWAUJz2x9ckDP2kV4=', 'Content-Length': '0'} 2022-09-03T06:18:25.102 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ae6eb6dc93967159-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.103 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:25.104 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:25.105 DEBUG:boto:Method: GET 2022-09-03T06:18:25.108 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:25.109 DEBUG:boto:Data: 2022-09-03T06:18:25.110 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.114 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.115 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.116 DEBUG:boto:Params: {} 2022-09-03T06:18:25.117 DEBUG:boto:Token: None 2022-09-03T06:18:25.117 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/c 2022-09-03T06:18:25.118 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4z/n5IQKE3jWAUJz2x9ckDP2kV4= 2022-09-03T06:18:25.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4z/n5IQKE3jWAUJz2x9ckDP2kV4=', 'Content-Length': '0'} 2022-09-03T06:18:25.122 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000079ea4e4423c97299-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.123 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:25.124 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:25.125 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:25.126 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:25.127 DEBUG:boto:Method: GET 2022-09-03T06:18:25.128 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:25.128 DEBUG:boto:Data: 2022-09-03T06:18:25.129 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.130 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.130 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.131 DEBUG:boto:Params: {} 2022-09-03T06:18:25.132 DEBUG:boto:Token: None 2022-09-03T06:18:25.132 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/ 2022-09-03T06:18:25.133 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc= 2022-09-03T06:18:25.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc=', 'Content-Length': '0'} 2022-09-03T06:18:25.137 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d82658301a397c13-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.138 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:25.140 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:25.141 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:25.142 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:25.143 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:25.144 DEBUG:boto:Method: GET 2022-09-03T06:18:25.144 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:25.145 DEBUG:boto:Data: 2022-09-03T06:18:25.146 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.147 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.148 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.149 DEBUG:boto:Params: {} 2022-09-03T06:18:25.150 DEBUG:boto:Token: None 2022-09-03T06:18:25.151 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/ 2022-09-03T06:18:25.152 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc= 2022-09-03T06:18:25.153 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc=', 'Content-Length': '0'} 2022-09-03T06:18:25.156 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000028d6268e63a2bde0-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.157 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:25.158 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:18:25.159 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:25.160 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:25.161 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.162 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.164 DEBUG:boto:Method: GET 2022-09-03T06:18:25.164 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.165 DEBUG:boto:Data: 2022-09-03T06:18:25.166 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.167 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.168 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.168 DEBUG:boto:Params: {} 2022-09-03T06:18:25.169 DEBUG:boto:Token: None 2022-09-03T06:18:25.170 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.171 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qchLEaPCzz2//e+EcMTJ7CoC+6g= 2022-09-03T06:18:25.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qchLEaPCzz2//e+EcMTJ7CoC+6g=', 'Content-Length': '0'} 2022-09-03T06:18:25.175 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d350c77deb728daa-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.176 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:25.177 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:25.178 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.179 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.180 DEBUG:boto:Method: GET 2022-09-03T06:18:25.181 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.182 DEBUG:boto:Data: 2022-09-03T06:18:25.183 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.185 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.186 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.187 DEBUG:boto:Params: {} 2022-09-03T06:18:25.188 DEBUG:boto:Token: None 2022-09-03T06:18:25.189 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.190 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qchLEaPCzz2//e+EcMTJ7CoC+6g= 2022-09-03T06:18:25.190 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qchLEaPCzz2//e+EcMTJ7CoC+6g=', 'Content-Length': '0'} 2022-09-03T06:18:25.193 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000988d9ac60f247741-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.194 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:25.195 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:25.196 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.196 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.197 DEBUG:boto:Method: HEAD 2022-09-03T06:18:25.197 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.198 DEBUG:boto:Data: 2022-09-03T06:18:25.198 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.199 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.199 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.200 DEBUG:boto:Params: {} 2022-09-03T06:18:25.200 DEBUG:boto:Token: None 2022-09-03T06:18:25.201 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.202 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0Zr/jnm8vji3WTEvrYzPQYRUvEc= 2022-09-03T06:18:25.203 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0Zr/jnm8vji3WTEvrYzPQYRUvEc=', 'Content-Length': '0'} 2022-09-03T06:18:25.207 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a376864d66feee1d-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.208 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:25.209 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:25.209 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.210 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.211 DEBUG:boto:Method: HEAD 2022-09-03T06:18:25.211 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.212 DEBUG:boto:Data: 2022-09-03T06:18:25.212 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.213 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.213 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.214 DEBUG:boto:Params: {} 2022-09-03T06:18:25.215 DEBUG:boto:Token: None 2022-09-03T06:18:25.215 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.216 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0Zr/jnm8vji3WTEvrYzPQYRUvEc= 2022-09-03T06:18:25.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0Zr/jnm8vji3WTEvrYzPQYRUvEc=', 'Content-Length': '0'} 2022-09-03T06:18:25.218 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f633d76a323b92f6-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.220 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:18:25.220 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:25.221 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:25.222 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.223 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.224 DEBUG:boto:Method: GET 2022-09-03T06:18:25.224 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.225 DEBUG:boto:Data: 2022-09-03T06:18:25.225 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.226 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.229 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.230 DEBUG:boto:Params: {} 2022-09-03T06:18:25.231 DEBUG:boto:Token: None 2022-09-03T06:18:25.232 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.233 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qchLEaPCzz2//e+EcMTJ7CoC+6g= 2022-09-03T06:18:25.234 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qchLEaPCzz2//e+EcMTJ7CoC+6g=', 'Content-Length': '0'} 2022-09-03T06:18:25.237 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc3af6a7d546b6f3-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.279 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:25.280 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:25.282 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.282 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.283 DEBUG:boto:Method: GET 2022-09-03T06:18:25.289 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.290 DEBUG:boto:Data: 2022-09-03T06:18:25.291 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.292 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.292 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.293 DEBUG:boto:Params: {} 2022-09-03T06:18:25.294 DEBUG:boto:Token: None 2022-09-03T06:18:25.294 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:25.295 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qchLEaPCzz2//e+EcMTJ7CoC+6g= 2022-09-03T06:18:25.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qchLEaPCzz2//e+EcMTJ7CoC+6g=', 'Content-Length': '0'} 2022-09-03T06:18:25.299 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f46cc1b44ea7ac6-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.301 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:25.302 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:25.302 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:25.303 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:25.304 DEBUG:boto:Method: GET 2022-09-03T06:18:25.305 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:25.305 DEBUG:boto:Data: 2022-09-03T06:18:25.308 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.309 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.310 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.311 DEBUG:boto:Params: {} 2022-09-03T06:18:25.313 DEBUG:boto:Token: None 2022-09-03T06:18:25.314 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/ 2022-09-03T06:18:25.317 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc= 2022-09-03T06:18:25.318 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc=', 'Content-Length': '0'} 2022-09-03T06:18:25.322 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000033f599353265628f-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.336 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:25.339 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:25.340 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:25.340 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:25.341 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:25.341 DEBUG:boto:Method: GET 2022-09-03T06:18:25.342 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:25.343 DEBUG:boto:Data: 2022-09-03T06:18:25.344 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.344 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.345 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.347 DEBUG:boto:Params: {} 2022-09-03T06:18:25.348 DEBUG:boto:Token: None 2022-09-03T06:18:25.349 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/ 2022-09-03T06:18:25.351 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc= 2022-09-03T06:18:25.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc=', 'Content-Length': '0'} 2022-09-03T06:18:25.354 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000098dc1d85d2be8312-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.355 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:25.356 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:25.357 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:25.358 DEBUG:boto:Method: HEAD 2022-09-03T06:18:25.358 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:25.359 DEBUG:boto:Data: 2022-09-03T06:18:25.359 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.363 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.365 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.365 DEBUG:boto:Params: {} 2022-09-03T06:18:25.366 DEBUG:boto:Token: None 2022-09-03T06:18:25.368 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/d 2022-09-03T06:18:25.369 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Qvm2bA5HmRvH3Tt1OcJXgaU52PY= 2022-09-03T06:18:25.370 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Qvm2bA5HmRvH3Tt1OcJXgaU52PY=', 'Content-Length': '0'} 2022-09-03T06:18:25.372 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a34fd5d40f81fffc-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.373 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:25.374 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:25.374 DEBUG:boto:Method: HEAD 2022-09-03T06:18:25.375 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:25.375 DEBUG:boto:Data: 2022-09-03T06:18:25.376 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.376 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.377 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.377 DEBUG:boto:Params: {} 2022-09-03T06:18:25.378 DEBUG:boto:Token: None 2022-09-03T06:18:25.378 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/d 2022-09-03T06:18:25.379 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Qvm2bA5HmRvH3Tt1OcJXgaU52PY= 2022-09-03T06:18:25.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Qvm2bA5HmRvH3Tt1OcJXgaU52PY=', 'Content-Length': '0'} 2022-09-03T06:18:25.382 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009edf5bf338a09d64-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.383 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:18:25.383 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:25.384 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:25.385 DEBUG:boto:Method: GET 2022-09-03T06:18:25.387 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:25.387 DEBUG:boto:Data: 2022-09-03T06:18:25.388 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.389 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.390 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.391 DEBUG:boto:Params: {} 2022-09-03T06:18:25.392 DEBUG:boto:Token: None 2022-09-03T06:18:25.394 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/d 2022-09-03T06:18:25.397 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V6dgg/QnogVgKJjl4vUNSxI7Xv8= 2022-09-03T06:18:25.398 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V6dgg/QnogVgKJjl4vUNSxI7Xv8=', 'Content-Length': '0'} 2022-09-03T06:18:25.401 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000171acd8be4ee67d1-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.402 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:25.404 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:25.406 DEBUG:boto:Method: GET 2022-09-03T06:18:25.408 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:25.408 DEBUG:boto:Data: 2022-09-03T06:18:25.409 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.409 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.410 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.411 DEBUG:boto:Params: {} 2022-09-03T06:18:25.412 DEBUG:boto:Token: None 2022-09-03T06:18:25.412 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/d 2022-09-03T06:18:25.413 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V6dgg/QnogVgKJjl4vUNSxI7Xv8= 2022-09-03T06:18:25.414 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V6dgg/QnogVgKJjl4vUNSxI7Xv8=', 'Content-Length': '0'} 2022-09-03T06:18:25.416 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f99761279c7d5fb7-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.417 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:25.418 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:25.418 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:25.419 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:25.420 DEBUG:boto:Method: GET 2022-09-03T06:18:25.420 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:25.421 DEBUG:boto:Data: 2022-09-03T06:18:25.421 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.422 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.422 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.423 DEBUG:boto:Params: {} 2022-09-03T06:18:25.423 DEBUG:boto:Token: None 2022-09-03T06:18:25.424 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/ 2022-09-03T06:18:25.425 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc= 2022-09-03T06:18:25.425 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc=', 'Content-Length': '0'} 2022-09-03T06:18:25.428 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006c35388983526b7b-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.429 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:25.430 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:25.431 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:25.431 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:25.432 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:25.432 DEBUG:boto:Method: GET 2022-09-03T06:18:25.433 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:25.433 DEBUG:boto:Data: 2022-09-03T06:18:25.434 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.434 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.435 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.435 DEBUG:boto:Params: {} 2022-09-03T06:18:25.436 DEBUG:boto:Token: None 2022-09-03T06:18:25.437 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/ 2022-09-03T06:18:25.437 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc= 2022-09-03T06:18:25.438 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc=', 'Content-Length': '0'} 2022-09-03T06:18:25.441 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000046a83571fffa1806-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.443 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:25.443 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:18:25.444 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:25.444 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:25.445 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.446 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.446 DEBUG:boto:Method: GET 2022-09-03T06:18:25.447 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.448 DEBUG:boto:Data: 2022-09-03T06:18:25.448 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.449 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.449 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.450 DEBUG:boto:Params: {} 2022-09-03T06:18:25.450 DEBUG:boto:Token: None 2022-09-03T06:18:25.451 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.452 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8RCFmCvt1FP7ly86Jh3+yKCkFx8= 2022-09-03T06:18:25.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8RCFmCvt1FP7ly86Jh3+yKCkFx8=', 'Content-Length': '0'} 2022-09-03T06:18:25.455 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f79fda7fe1087922-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.456 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:25.456 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:25.457 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.457 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.458 DEBUG:boto:Method: GET 2022-09-03T06:18:25.458 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.459 DEBUG:boto:Data: 2022-09-03T06:18:25.459 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.460 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.461 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.461 DEBUG:boto:Params: {} 2022-09-03T06:18:25.462 DEBUG:boto:Token: None 2022-09-03T06:18:25.463 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.463 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8RCFmCvt1FP7ly86Jh3+yKCkFx8= 2022-09-03T06:18:25.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8RCFmCvt1FP7ly86Jh3+yKCkFx8=', 'Content-Length': '0'} 2022-09-03T06:18:25.466 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000067b545afbd13b9a-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.467 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:25.468 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:25.468 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.469 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.469 DEBUG:boto:Method: HEAD 2022-09-03T06:18:25.470 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.470 DEBUG:boto:Data: 2022-09-03T06:18:25.471 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.471 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.472 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.472 DEBUG:boto:Params: {} 2022-09-03T06:18:25.473 DEBUG:boto:Token: None 2022-09-03T06:18:25.474 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.474 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/Wca1Vpfh+dj/XO0nUIsQ77pmc= 2022-09-03T06:18:25.475 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/Wca1Vpfh+dj/XO0nUIsQ77pmc=', 'Content-Length': '0'} 2022-09-03T06:18:25.477 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e1be882bc0c558c3-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.479 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:25.479 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:25.480 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.480 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.481 DEBUG:boto:Method: HEAD 2022-09-03T06:18:25.481 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.482 DEBUG:boto:Data: 2022-09-03T06:18:25.482 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.483 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.483 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.484 DEBUG:boto:Params: {} 2022-09-03T06:18:25.484 DEBUG:boto:Token: None 2022-09-03T06:18:25.485 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.486 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/Wca1Vpfh+dj/XO0nUIsQ77pmc= 2022-09-03T06:18:25.486 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/Wca1Vpfh+dj/XO0nUIsQ77pmc=', 'Content-Length': '0'} 2022-09-03T06:18:25.489 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000df7a0dfc0e73b635-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.490 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:18:25.491 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:25.491 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:25.492 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.492 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.493 DEBUG:boto:Method: GET 2022-09-03T06:18:25.493 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.494 DEBUG:boto:Data: 2022-09-03T06:18:25.495 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.495 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.496 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.496 DEBUG:boto:Params: {} 2022-09-03T06:18:25.497 DEBUG:boto:Token: None 2022-09-03T06:18:25.498 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.498 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8RCFmCvt1FP7ly86Jh3+yKCkFx8= 2022-09-03T06:18:25.499 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8RCFmCvt1FP7ly86Jh3+yKCkFx8=', 'Content-Length': '0'} 2022-09-03T06:18:25.501 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000053faba2813bda44c-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.502 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:25.502 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:25.503 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.503 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.504 DEBUG:boto:Method: GET 2022-09-03T06:18:25.504 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.505 DEBUG:boto:Data: 2022-09-03T06:18:25.506 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.506 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.507 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.507 DEBUG:boto:Params: {} 2022-09-03T06:18:25.508 DEBUG:boto:Token: None 2022-09-03T06:18:25.508 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:25.509 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8RCFmCvt1FP7ly86Jh3+yKCkFx8= 2022-09-03T06:18:25.510 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8RCFmCvt1FP7ly86Jh3+yKCkFx8=', 'Content-Length': '0'} 2022-09-03T06:18:25.517 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000045179a908cebecbc-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.554 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:25.556 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:25.557 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:25.559 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:25.559 DEBUG:boto:Method: GET 2022-09-03T06:18:25.560 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:25.561 DEBUG:boto:Data: 2022-09-03T06:18:25.562 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.563 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.564 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.565 DEBUG:boto:Params: {} 2022-09-03T06:18:25.565 DEBUG:boto:Token: None 2022-09-03T06:18:25.566 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/ 2022-09-03T06:18:25.567 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc= 2022-09-03T06:18:25.567 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc=', 'Content-Length': '0'} 2022-09-03T06:18:25.570 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e8e54682a55eea3f-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.571 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:25.572 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:25.573 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:25.573 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:25.574 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:25.574 DEBUG:boto:Method: GET 2022-09-03T06:18:25.575 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:25.575 DEBUG:boto:Data: 2022-09-03T06:18:25.576 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.576 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.577 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.578 DEBUG:boto:Params: {} 2022-09-03T06:18:25.578 DEBUG:boto:Token: None 2022-09-03T06:18:25.579 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/ 2022-09-03T06:18:25.580 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc= 2022-09-03T06:18:25.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc=', 'Content-Length': '0'} 2022-09-03T06:18:25.583 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000da95c64ed3c93089-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.584 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:25.586 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:25.586 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:25.587 DEBUG:boto:Method: HEAD 2022-09-03T06:18:25.587 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:25.588 DEBUG:boto:Data: 2022-09-03T06:18:25.588 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.589 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.590 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.590 DEBUG:boto:Params: {} 2022-09-03T06:18:25.591 DEBUG:boto:Token: None 2022-09-03T06:18:25.592 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/e 2022-09-03T06:18:25.592 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xReklsxvMyjKIcgV/JV/ZK3mnkA= 2022-09-03T06:18:25.593 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xReklsxvMyjKIcgV/JV/ZK3mnkA=', 'Content-Length': '0'} 2022-09-03T06:18:25.595 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b1d4bcb84413ba6e-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.596 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:25.597 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:25.597 DEBUG:boto:Method: HEAD 2022-09-03T06:18:25.598 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:25.599 DEBUG:boto:Data: 2022-09-03T06:18:25.599 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.600 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.600 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.601 DEBUG:boto:Params: {} 2022-09-03T06:18:25.601 DEBUG:boto:Token: None 2022-09-03T06:18:25.602 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/e 2022-09-03T06:18:25.603 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xReklsxvMyjKIcgV/JV/ZK3mnkA= 2022-09-03T06:18:25.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xReklsxvMyjKIcgV/JV/ZK3mnkA=', 'Content-Length': '0'} 2022-09-03T06:18:25.606 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000061c3afbee580f516-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.607 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:18:25.608 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:25.608 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:25.609 DEBUG:boto:Method: GET 2022-09-03T06:18:25.609 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:25.610 DEBUG:boto:Data: 2022-09-03T06:18:25.611 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.611 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.612 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.612 DEBUG:boto:Params: {} 2022-09-03T06:18:25.613 DEBUG:boto:Token: None 2022-09-03T06:18:25.614 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/e 2022-09-03T06:18:25.614 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sBewjslhpJHyt2o43GAdD1+BB6k= 2022-09-03T06:18:25.615 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sBewjslhpJHyt2o43GAdD1+BB6k=', 'Content-Length': '0'} 2022-09-03T06:18:25.624 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f185c228c308e4ba-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.666 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:25.668 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:25.669 DEBUG:boto:Method: GET 2022-09-03T06:18:25.669 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:25.670 DEBUG:boto:Data: 2022-09-03T06:18:25.671 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.672 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.673 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.674 DEBUG:boto:Params: {} 2022-09-03T06:18:25.674 DEBUG:boto:Token: None 2022-09-03T06:18:25.675 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/e 2022-09-03T06:18:25.676 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sBewjslhpJHyt2o43GAdD1+BB6k= 2022-09-03T06:18:25.676 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sBewjslhpJHyt2o43GAdD1+BB6k=', 'Content-Length': '0'} 2022-09-03T06:18:25.679 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000070c655ff97585646-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.681 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:25.683 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:25.684 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:25.685 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:25.686 DEBUG:boto:Method: GET 2022-09-03T06:18:25.687 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:25.689 DEBUG:boto:Data: 2022-09-03T06:18:25.690 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.691 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.692 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.693 DEBUG:boto:Params: {} 2022-09-03T06:18:25.694 DEBUG:boto:Token: None 2022-09-03T06:18:25.695 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/ 2022-09-03T06:18:25.696 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc= 2022-09-03T06:18:25.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc=', 'Content-Length': '0'} 2022-09-03T06:18:25.700 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cdab54781fc33623-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.701 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:25.703 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:25.703 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:25.704 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:25.705 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:25.706 DEBUG:boto:Method: GET 2022-09-03T06:18:25.707 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:25.708 DEBUG:boto:Data: 2022-09-03T06:18:25.709 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.711 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.711 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.712 DEBUG:boto:Params: {} 2022-09-03T06:18:25.713 DEBUG:boto:Token: None 2022-09-03T06:18:25.714 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/ 2022-09-03T06:18:25.715 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc= 2022-09-03T06:18:25.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc=', 'Content-Length': '0'} 2022-09-03T06:18:25.718 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aa0428cc4616ea8c-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.762 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:25.763 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:18:25.764 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:25.765 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:25.766 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.767 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.767 DEBUG:boto:Method: GET 2022-09-03T06:18:25.768 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.768 DEBUG:boto:Data: 2022-09-03T06:18:25.769 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.770 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.770 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.771 DEBUG:boto:Params: {} 2022-09-03T06:18:25.771 DEBUG:boto:Token: None 2022-09-03T06:18:25.772 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.773 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XR0KI+kc1HGDcKF0oryhtTTgWzs= 2022-09-03T06:18:25.774 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XR0KI+kc1HGDcKF0oryhtTTgWzs=', 'Content-Length': '0'} 2022-09-03T06:18:25.776 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000917333bd7ec94e03-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.777 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:25.777 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:25.778 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.779 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.779 DEBUG:boto:Method: GET 2022-09-03T06:18:25.780 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.780 DEBUG:boto:Data: 2022-09-03T06:18:25.781 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.781 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.782 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.783 DEBUG:boto:Params: {} 2022-09-03T06:18:25.783 DEBUG:boto:Token: None 2022-09-03T06:18:25.784 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.785 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XR0KI+kc1HGDcKF0oryhtTTgWzs= 2022-09-03T06:18:25.785 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XR0KI+kc1HGDcKF0oryhtTTgWzs=', 'Content-Length': '0'} 2022-09-03T06:18:25.788 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000384c7476e5b34c71-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.788 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:25.789 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:25.790 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.791 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.791 DEBUG:boto:Method: HEAD 2022-09-03T06:18:25.792 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.792 DEBUG:boto:Data: 2022-09-03T06:18:25.793 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.793 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.794 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.794 DEBUG:boto:Params: {} 2022-09-03T06:18:25.795 DEBUG:boto:Token: None 2022-09-03T06:18:25.796 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.796 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VMt7IXb9sfEExjTnise7fa5FRmg= 2022-09-03T06:18:25.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VMt7IXb9sfEExjTnise7fa5FRmg=', 'Content-Length': '0'} 2022-09-03T06:18:25.799 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e2b51cfc36330f85-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.800 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:25.800 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:25.801 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.801 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.802 DEBUG:boto:Method: HEAD 2022-09-03T06:18:25.802 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.803 DEBUG:boto:Data: 2022-09-03T06:18:25.804 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.804 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.805 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.805 DEBUG:boto:Params: {} 2022-09-03T06:18:25.806 DEBUG:boto:Token: None 2022-09-03T06:18:25.806 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.807 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VMt7IXb9sfEExjTnise7fa5FRmg= 2022-09-03T06:18:25.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VMt7IXb9sfEExjTnise7fa5FRmg=', 'Content-Length': '0'} 2022-09-03T06:18:25.810 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d7f0fc16bcac11a3-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.811 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:18:25.812 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:25.812 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:25.813 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.813 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.814 DEBUG:boto:Method: GET 2022-09-03T06:18:25.814 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.815 DEBUG:boto:Data: 2022-09-03T06:18:25.816 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.816 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.817 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.817 DEBUG:boto:Params: {} 2022-09-03T06:18:25.818 DEBUG:boto:Token: None 2022-09-03T06:18:25.819 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.819 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XR0KI+kc1HGDcKF0oryhtTTgWzs= 2022-09-03T06:18:25.820 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XR0KI+kc1HGDcKF0oryhtTTgWzs=', 'Content-Length': '0'} 2022-09-03T06:18:25.822 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007e6fda64c9189a21-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.823 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:25.824 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:25.824 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.825 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.825 DEBUG:boto:Method: GET 2022-09-03T06:18:25.826 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.826 DEBUG:boto:Data: 2022-09-03T06:18:25.827 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.828 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.828 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.829 DEBUG:boto:Params: {} 2022-09-03T06:18:25.829 DEBUG:boto:Token: None 2022-09-03T06:18:25.830 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:25.831 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XR0KI+kc1HGDcKF0oryhtTTgWzs= 2022-09-03T06:18:25.831 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XR0KI+kc1HGDcKF0oryhtTTgWzs=', 'Content-Length': '0'} 2022-09-03T06:18:25.834 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d499012ebfd8cf25-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.834 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:25.835 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:25.836 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:25.836 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:25.837 DEBUG:boto:Method: GET 2022-09-03T06:18:25.837 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:25.838 DEBUG:boto:Data: 2022-09-03T06:18:25.838 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.839 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.839 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.840 DEBUG:boto:Params: {} 2022-09-03T06:18:25.840 DEBUG:boto:Token: None 2022-09-03T06:18:25.841 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/ 2022-09-03T06:18:25.842 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc= 2022-09-03T06:18:25.843 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc=', 'Content-Length': '0'} 2022-09-03T06:18:25.845 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000058ceb8ea7462d4f9-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.886 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:25.887 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:25.887 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:25.888 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:25.889 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:25.889 DEBUG:boto:Method: GET 2022-09-03T06:18:25.890 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:25.890 DEBUG:boto:Data: 2022-09-03T06:18:25.891 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.892 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.892 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.893 DEBUG:boto:Params: {} 2022-09-03T06:18:25.893 DEBUG:boto:Token: None 2022-09-03T06:18:25.894 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/ 2022-09-03T06:18:25.895 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc= 2022-09-03T06:18:25.895 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc=', 'Content-Length': '0'} 2022-09-03T06:18:25.898 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d84fa44748198189-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.898 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:25.899 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:25.900 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:25.901 DEBUG:boto:Method: HEAD 2022-09-03T06:18:25.901 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:25.902 DEBUG:boto:Data: 2022-09-03T06:18:25.902 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.903 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.903 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.904 DEBUG:boto:Params: {} 2022-09-03T06:18:25.904 DEBUG:boto:Token: None 2022-09-03T06:18:25.905 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/f 2022-09-03T06:18:25.906 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EJctvU68lhzPm4iWq16UdzYYpho= 2022-09-03T06:18:25.906 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EJctvU68lhzPm4iWq16UdzYYpho=', 'Content-Length': '0'} 2022-09-03T06:18:25.909 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002aaf5f3a146ce3d8-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.909 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:25.910 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:25.911 DEBUG:boto:Method: HEAD 2022-09-03T06:18:25.911 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:25.912 DEBUG:boto:Data: 2022-09-03T06:18:25.912 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.913 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.913 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.914 DEBUG:boto:Params: {} 2022-09-03T06:18:25.914 DEBUG:boto:Token: None 2022-09-03T06:18:25.915 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/f 2022-09-03T06:18:25.915 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EJctvU68lhzPm4iWq16UdzYYpho= 2022-09-03T06:18:25.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EJctvU68lhzPm4iWq16UdzYYpho=', 'Content-Length': '0'} 2022-09-03T06:18:25.918 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018f47770cfc9b4e9-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.919 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:18:25.920 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:25.920 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:25.921 DEBUG:boto:Method: GET 2022-09-03T06:18:25.921 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:25.922 DEBUG:boto:Data: 2022-09-03T06:18:25.922 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.923 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.923 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.924 DEBUG:boto:Params: {} 2022-09-03T06:18:25.924 DEBUG:boto:Token: None 2022-09-03T06:18:25.925 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/f 2022-09-03T06:18:25.926 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DiRgswYCp6gLq8RwWrKfejQNSsA= 2022-09-03T06:18:25.926 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DiRgswYCp6gLq8RwWrKfejQNSsA=', 'Content-Length': '0'} 2022-09-03T06:18:25.929 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cb183d2f9c545f0a-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.929 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:25.930 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:25.931 DEBUG:boto:Method: GET 2022-09-03T06:18:25.931 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:25.932 DEBUG:boto:Data: 2022-09-03T06:18:25.932 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.933 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.934 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.934 DEBUG:boto:Params: {} 2022-09-03T06:18:25.935 DEBUG:boto:Token: None 2022-09-03T06:18:25.936 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/f 2022-09-03T06:18:25.936 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DiRgswYCp6gLq8RwWrKfejQNSsA= 2022-09-03T06:18:25.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DiRgswYCp6gLq8RwWrKfejQNSsA=', 'Content-Length': '0'} 2022-09-03T06:18:25.941 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000071258568e81f07a7-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.941 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:25.942 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:25.943 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:25.943 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:25.944 DEBUG:boto:Method: GET 2022-09-03T06:18:25.944 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:25.945 DEBUG:boto:Data: 2022-09-03T06:18:25.945 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.946 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.947 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.947 DEBUG:boto:Params: {} 2022-09-03T06:18:25.948 DEBUG:boto:Token: None 2022-09-03T06:18:25.948 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/ 2022-09-03T06:18:25.949 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc= 2022-09-03T06:18:25.950 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc=', 'Content-Length': '0'} 2022-09-03T06:18:25.953 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000009e8e6640b3810b8-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.953 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:25.954 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:25.955 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:25.955 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:25.956 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:25.956 DEBUG:boto:Method: GET 2022-09-03T06:18:25.957 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:25.957 DEBUG:boto:Data: 2022-09-03T06:18:25.958 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.958 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.959 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.959 DEBUG:boto:Params: {} 2022-09-03T06:18:25.960 DEBUG:boto:Token: None 2022-09-03T06:18:25.961 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/ 2022-09-03T06:18:25.961 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc= 2022-09-03T06:18:25.962 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TYqgZ8Wg41/Ob5xwugYh6perGlc=', 'Content-Length': '0'} 2022-09-03T06:18:25.964 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002089a3d2569e755f-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.964 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:25.965 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:18:25.966 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:25.966 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:25.967 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:25.967 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:25.968 DEBUG:boto:Method: GET 2022-09-03T06:18:25.969 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:25.969 DEBUG:boto:Data: 2022-09-03T06:18:25.970 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.970 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.971 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.971 DEBUG:boto:Params: {} 2022-09-03T06:18:25.972 DEBUG:boto:Token: None 2022-09-03T06:18:25.972 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:25.973 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0LQDO8pj+LeCCEFkvLPCuJwATIM= 2022-09-03T06:18:25.974 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0LQDO8pj+LeCCEFkvLPCuJwATIM=', 'Content-Length': '0'} 2022-09-03T06:18:25.976 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000009eff5aaaa138ea0-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.977 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:25.978 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:25.978 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:25.979 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:25.979 DEBUG:boto:Method: GET 2022-09-03T06:18:25.980 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:25.980 DEBUG:boto:Data: 2022-09-03T06:18:25.981 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.981 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:25.982 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.983 DEBUG:boto:Params: {} 2022-09-03T06:18:25.983 DEBUG:boto:Token: None 2022-09-03T06:18:25.984 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:25.984 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0LQDO8pj+LeCCEFkvLPCuJwATIM= 2022-09-03T06:18:25.985 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0LQDO8pj+LeCCEFkvLPCuJwATIM=', 'Content-Length': '0'} 2022-09-03T06:18:25.987 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000081b4b0f674cb59b1-006312f1b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.988 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:25.989 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:25.989 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:25.989 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:25.990 DEBUG:boto:Method: HEAD 2022-09-03T06:18:25.991 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:25.991 DEBUG:boto:Data: 2022-09-03T06:18:25.991 DEBUG:boto:Headers: {} 2022-09-03T06:18:25.992 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:25.993 DEBUG:boto:Port: 8000 2022-09-03T06:18:25.993 DEBUG:boto:Params: {} 2022-09-03T06:18:25.994 DEBUG:boto:Token: None 2022-09-03T06:18:25.994 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:25 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:25.995 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kJXOR+Y/2l0VC1U4fzTqLvuU2z4= 2022-09-03T06:18:25.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kJXOR+Y/2l0VC1U4fzTqLvuU2z4=', 'Content-Length': '0'} 2022-09-03T06:18:25.998 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008bf9754b1d23a3cc-006312f1b1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:25.998 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:25.999 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:25.999 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:26.000 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:26.000 DEBUG:boto:Method: HEAD 2022-09-03T06:18:26.001 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:26.001 DEBUG:boto:Data: 2022-09-03T06:18:26.002 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.003 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.003 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.004 DEBUG:boto:Params: {} 2022-09-03T06:18:26.004 DEBUG:boto:Token: None 2022-09-03T06:18:26.005 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:26.005 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I5WCZ4W7NfS0IpsPYI12wa5i/wE= 2022-09-03T06:18:26.006 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I5WCZ4W7NfS0IpsPYI12wa5i/wE=', 'Content-Length': '0'} 2022-09-03T06:18:26.008 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e06f3a714b127002-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.009 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:18:26.009 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:26.010 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:26.011 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:26.011 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:26.012 DEBUG:boto:Method: GET 2022-09-03T06:18:26.012 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:26.013 DEBUG:boto:Data: 2022-09-03T06:18:26.013 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.014 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:26.014 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.015 DEBUG:boto:Params: {} 2022-09-03T06:18:26.016 DEBUG:boto:Token: None 2022-09-03T06:18:26.016 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:26.017 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uUJaFA/F99krqD8Rs9UrnbMvArM= 2022-09-03T06:18:26.018 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uUJaFA/F99krqD8Rs9UrnbMvArM=', 'Content-Length': '0'} 2022-09-03T06:18:26.023 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000029d95a66d2174359-006312f1b2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.023 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:26.024 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:26.025 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:26.025 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:26.026 DEBUG:boto:Method: GET 2022-09-03T06:18:26.026 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:26.027 DEBUG:boto:Data: 2022-09-03T06:18:26.027 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.028 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.029 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.029 DEBUG:boto:Params: {} 2022-09-03T06:18:26.030 DEBUG:boto:Token: None 2022-09-03T06:18:26.030 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:26.031 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uUJaFA/F99krqD8Rs9UrnbMvArM= 2022-09-03T06:18:26.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uUJaFA/F99krqD8Rs9UrnbMvArM=', 'Content-Length': '0'} 2022-09-03T06:18:26.034 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000163ab0cbf56fd76b-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.035 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:26.035 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:26.036 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:26.037 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:26.037 DEBUG:boto:Method: GET 2022-09-03T06:18:26.038 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:26.039 DEBUG:boto:Data: 2022-09-03T06:18:26.039 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.040 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:26.040 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.041 DEBUG:boto:Params: {} 2022-09-03T06:18:26.041 DEBUG:boto:Token: None 2022-09-03T06:18:26.042 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/ 2022-09-03T06:18:26.042 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A= 2022-09-03T06:18:26.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A=', 'Content-Length': '0'} 2022-09-03T06:18:26.046 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f74957d9832b8128-006312f1b2-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.048 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:26.049 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:26.050 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:26.050 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:26.051 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:26.052 DEBUG:boto:Method: GET 2022-09-03T06:18:26.053 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:26.053 DEBUG:boto:Data: 2022-09-03T06:18:26.054 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.054 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.055 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.056 DEBUG:boto:Params: {} 2022-09-03T06:18:26.056 DEBUG:boto:Token: None 2022-09-03T06:18:26.057 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/ 2022-09-03T06:18:26.058 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A= 2022-09-03T06:18:26.059 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A=', 'Content-Length': '0'} 2022-09-03T06:18:26.061 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005be837e4e5bbbc10-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.106 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:26.107 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:26.108 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:26.108 DEBUG:boto:Method: HEAD 2022-09-03T06:18:26.109 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:26.109 DEBUG:boto:Data: 2022-09-03T06:18:26.110 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.110 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:26.111 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.112 DEBUG:boto:Params: {} 2022-09-03T06:18:26.113 DEBUG:boto:Token: None 2022-09-03T06:18:26.113 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/g 2022-09-03T06:18:26.114 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ybjNeulfy2P0uHqznBUyvxwhMk8= 2022-09-03T06:18:26.115 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ybjNeulfy2P0uHqznBUyvxwhMk8=', 'Content-Length': '0'} 2022-09-03T06:18:26.117 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b34b4a235ad7728-006312f1b2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.117 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:26.118 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:26.119 DEBUG:boto:Method: HEAD 2022-09-03T06:18:26.119 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:26.120 DEBUG:boto:Data: 2022-09-03T06:18:26.120 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.121 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.121 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.122 DEBUG:boto:Params: {} 2022-09-03T06:18:26.123 DEBUG:boto:Token: None 2022-09-03T06:18:26.124 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/g 2022-09-03T06:18:26.124 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ybjNeulfy2P0uHqznBUyvxwhMk8= 2022-09-03T06:18:26.125 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ybjNeulfy2P0uHqznBUyvxwhMk8=', 'Content-Length': '0'} 2022-09-03T06:18:26.131 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000020a70977ea2187d4-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.133 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:18:26.134 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:26.135 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:26.136 DEBUG:boto:Method: GET 2022-09-03T06:18:26.138 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:26.139 DEBUG:boto:Data: 2022-09-03T06:18:26.140 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.141 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:26.142 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.143 DEBUG:boto:Params: {} 2022-09-03T06:18:26.145 DEBUG:boto:Token: None 2022-09-03T06:18:26.146 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/g 2022-09-03T06:18:26.147 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tU45aR5mtj094pXoGT9EtR7i1Lc= 2022-09-03T06:18:26.149 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tU45aR5mtj094pXoGT9EtR7i1Lc=', 'Content-Length': '0'} 2022-09-03T06:18:26.165 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006daf6a14f7ff21d2-006312f1b2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.166 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:26.167 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:26.167 DEBUG:boto:Method: GET 2022-09-03T06:18:26.168 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:26.169 DEBUG:boto:Data: 2022-09-03T06:18:26.169 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.174 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.175 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.176 DEBUG:boto:Params: {} 2022-09-03T06:18:26.178 DEBUG:boto:Token: None 2022-09-03T06:18:26.179 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/g 2022-09-03T06:18:26.180 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tU45aR5mtj094pXoGT9EtR7i1Lc= 2022-09-03T06:18:26.181 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tU45aR5mtj094pXoGT9EtR7i1Lc=', 'Content-Length': '0'} 2022-09-03T06:18:26.184 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cbe4d58c897001be-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.185 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:26.186 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:26.187 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:26.187 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:26.188 DEBUG:boto:Method: GET 2022-09-03T06:18:26.189 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:26.189 DEBUG:boto:Data: 2022-09-03T06:18:26.190 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.190 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:26.191 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.192 DEBUG:boto:Params: {} 2022-09-03T06:18:26.192 DEBUG:boto:Token: None 2022-09-03T06:18:26.193 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/ 2022-09-03T06:18:26.194 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A= 2022-09-03T06:18:26.195 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A=', 'Content-Length': '0'} 2022-09-03T06:18:26.198 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000013f21959a6f4767a-006312f1b2-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.198 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:26.199 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:26.200 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:26.200 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:26.201 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:26.202 DEBUG:boto:Method: GET 2022-09-03T06:18:26.202 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:26.203 DEBUG:boto:Data: 2022-09-03T06:18:26.203 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.204 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.204 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.205 DEBUG:boto:Params: {} 2022-09-03T06:18:26.205 DEBUG:boto:Token: None 2022-09-03T06:18:26.206 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/ 2022-09-03T06:18:26.207 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A= 2022-09-03T06:18:26.207 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A=', 'Content-Length': '0'} 2022-09-03T06:18:26.211 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d243c9027777ca63-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.212 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:26.214 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:18:26.215 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:26.216 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:26.217 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.217 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.218 DEBUG:boto:Method: GET 2022-09-03T06:18:26.219 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.220 DEBUG:boto:Data: 2022-09-03T06:18:26.221 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.221 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:26.222 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.223 DEBUG:boto:Params: {} 2022-09-03T06:18:26.224 DEBUG:boto:Token: None 2022-09-03T06:18:26.224 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.225 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SYEErnvyQFw3kHMy7F+PBdTP+QI= 2022-09-03T06:18:26.226 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SYEErnvyQFw3kHMy7F+PBdTP+QI=', 'Content-Length': '0'} 2022-09-03T06:18:26.230 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c88dcabd07beafd6-006312f1b2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.231 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:26.232 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:26.234 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.235 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.235 DEBUG:boto:Method: GET 2022-09-03T06:18:26.236 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.236 DEBUG:boto:Data: 2022-09-03T06:18:26.237 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.238 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.238 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.239 DEBUG:boto:Params: {} 2022-09-03T06:18:26.240 DEBUG:boto:Token: None 2022-09-03T06:18:26.241 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.242 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SYEErnvyQFw3kHMy7F+PBdTP+QI= 2022-09-03T06:18:26.243 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SYEErnvyQFw3kHMy7F+PBdTP+QI=', 'Content-Length': '0'} 2022-09-03T06:18:26.247 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c43d3e7390ac81aa-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.248 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:26.249 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:26.250 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.250 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.251 DEBUG:boto:Method: HEAD 2022-09-03T06:18:26.252 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.253 DEBUG:boto:Data: 2022-09-03T06:18:26.254 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.256 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:26.256 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.257 DEBUG:boto:Params: {} 2022-09-03T06:18:26.259 DEBUG:boto:Token: None 2022-09-03T06:18:26.260 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.261 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G9ZAxr4DMYyV02N7Hdi/LmhAnt8= 2022-09-03T06:18:26.262 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G9ZAxr4DMYyV02N7Hdi/LmhAnt8=', 'Content-Length': '0'} 2022-09-03T06:18:26.264 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007bcc6bdae44a0588-006312f1b2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.266 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:26.266 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:26.267 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.268 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.268 DEBUG:boto:Method: HEAD 2022-09-03T06:18:26.269 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.269 DEBUG:boto:Data: 2022-09-03T06:18:26.270 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.270 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.271 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.272 DEBUG:boto:Params: {} 2022-09-03T06:18:26.273 DEBUG:boto:Token: None 2022-09-03T06:18:26.274 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.275 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G9ZAxr4DMYyV02N7Hdi/LmhAnt8= 2022-09-03T06:18:26.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G9ZAxr4DMYyV02N7Hdi/LmhAnt8=', 'Content-Length': '0'} 2022-09-03T06:18:26.279 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d577bb4bf977736b-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.280 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:18:26.281 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:26.282 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:26.283 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.283 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.284 DEBUG:boto:Method: GET 2022-09-03T06:18:26.285 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.286 DEBUG:boto:Data: 2022-09-03T06:18:26.286 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.287 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:26.288 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.288 DEBUG:boto:Params: {} 2022-09-03T06:18:26.290 DEBUG:boto:Token: None 2022-09-03T06:18:26.291 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.292 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SYEErnvyQFw3kHMy7F+PBdTP+QI= 2022-09-03T06:18:26.293 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SYEErnvyQFw3kHMy7F+PBdTP+QI=', 'Content-Length': '0'} 2022-09-03T06:18:26.296 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003976676fb328f49d-006312f1b2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.297 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:26.298 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:26.299 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.301 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.302 DEBUG:boto:Method: GET 2022-09-03T06:18:26.303 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.304 DEBUG:boto:Data: 2022-09-03T06:18:26.304 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.305 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.307 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.308 DEBUG:boto:Params: {} 2022-09-03T06:18:26.309 DEBUG:boto:Token: None 2022-09-03T06:18:26.310 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:26.311 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SYEErnvyQFw3kHMy7F+PBdTP+QI= 2022-09-03T06:18:26.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SYEErnvyQFw3kHMy7F+PBdTP+QI=', 'Content-Length': '0'} 2022-09-03T06:18:26.315 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002e6442e0b734e946-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.358 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:26.359 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:26.362 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:26.363 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:26.364 DEBUG:boto:Method: GET 2022-09-03T06:18:26.364 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:26.365 DEBUG:boto:Data: 2022-09-03T06:18:26.366 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.366 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:26.367 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.367 DEBUG:boto:Params: {} 2022-09-03T06:18:26.368 DEBUG:boto:Token: None 2022-09-03T06:18:26.368 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/ 2022-09-03T06:18:26.369 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A= 2022-09-03T06:18:26.370 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A=', 'Content-Length': '0'} 2022-09-03T06:18:26.373 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000c410a7ed32c53ee-006312f1b2-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.374 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:26.375 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:26.375 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:26.376 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:26.376 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:26.377 DEBUG:boto:Method: GET 2022-09-03T06:18:26.377 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:26.378 DEBUG:boto:Data: 2022-09-03T06:18:26.378 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.379 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.380 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.380 DEBUG:boto:Params: {} 2022-09-03T06:18:26.381 DEBUG:boto:Token: None 2022-09-03T06:18:26.381 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/ 2022-09-03T06:18:26.382 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A= 2022-09-03T06:18:26.383 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A=', 'Content-Length': '0'} 2022-09-03T06:18:26.386 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006cef2775de548217-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.387 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:26.388 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:26.388 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:26.389 DEBUG:boto:Method: HEAD 2022-09-03T06:18:26.390 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:26.390 DEBUG:boto:Data: 2022-09-03T06:18:26.391 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.391 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:26.392 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.392 DEBUG:boto:Params: {} 2022-09-03T06:18:26.393 DEBUG:boto:Token: None 2022-09-03T06:18:26.393 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/h 2022-09-03T06:18:26.394 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wlv+6fmoY4SG7/ZawZROFre1yAM= 2022-09-03T06:18:26.395 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wlv+6fmoY4SG7/ZawZROFre1yAM=', 'Content-Length': '0'} 2022-09-03T06:18:26.397 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000025824e6c05cc98fd-006312f1b2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.399 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:26.400 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:26.401 DEBUG:boto:Method: HEAD 2022-09-03T06:18:26.402 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:26.402 DEBUG:boto:Data: 2022-09-03T06:18:26.403 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.404 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.405 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.406 DEBUG:boto:Params: {} 2022-09-03T06:18:26.407 DEBUG:boto:Token: None 2022-09-03T06:18:26.408 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/h 2022-09-03T06:18:26.409 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wlv+6fmoY4SG7/ZawZROFre1yAM= 2022-09-03T06:18:26.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wlv+6fmoY4SG7/ZawZROFre1yAM=', 'Content-Length': '0'} 2022-09-03T06:18:26.415 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f6f2a0aa8a5d8039-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.416 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:18:26.417 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:26.418 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:26.419 DEBUG:boto:Method: GET 2022-09-03T06:18:26.420 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:26.421 DEBUG:boto:Data: 2022-09-03T06:18:26.422 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.423 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:26.424 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.425 DEBUG:boto:Params: {} 2022-09-03T06:18:26.426 DEBUG:boto:Token: None 2022-09-03T06:18:26.427 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/h 2022-09-03T06:18:26.428 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YRpAxya3jwCqFt/ZMJe0jypMqck= 2022-09-03T06:18:26.430 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YRpAxya3jwCqFt/ZMJe0jypMqck=', 'Content-Length': '0'} 2022-09-03T06:18:26.432 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000456f2f03f733f0f9-006312f1b2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.433 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:26.434 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:26.434 DEBUG:boto:Method: GET 2022-09-03T06:18:26.435 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:26.436 DEBUG:boto:Data: 2022-09-03T06:18:26.436 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.437 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.437 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.438 DEBUG:boto:Params: {} 2022-09-03T06:18:26.439 DEBUG:boto:Token: None 2022-09-03T06:18:26.440 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/h 2022-09-03T06:18:26.441 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YRpAxya3jwCqFt/ZMJe0jypMqck= 2022-09-03T06:18:26.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YRpAxya3jwCqFt/ZMJe0jypMqck=', 'Content-Length': '0'} 2022-09-03T06:18:26.446 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002939ec840ec697bd-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.486 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:26.487 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:26.487 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:26.488 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:26.488 DEBUG:boto:Method: GET 2022-09-03T06:18:26.489 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:26.490 DEBUG:boto:Data: 2022-09-03T06:18:26.490 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.491 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:26.491 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.492 DEBUG:boto:Params: {} 2022-09-03T06:18:26.492 DEBUG:boto:Token: None 2022-09-03T06:18:26.493 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/ 2022-09-03T06:18:26.494 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A= 2022-09-03T06:18:26.494 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A=', 'Content-Length': '0'} 2022-09-03T06:18:26.498 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006b239ab830789753-006312f1b2-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.499 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:26.501 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:26.502 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:26.503 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:26.504 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:26.505 DEBUG:boto:Method: GET 2022-09-03T06:18:26.505 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:26.507 DEBUG:boto:Data: 2022-09-03T06:18:26.508 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.509 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.509 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.510 DEBUG:boto:Params: {} 2022-09-03T06:18:26.511 DEBUG:boto:Token: None 2022-09-03T06:18:26.512 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/ 2022-09-03T06:18:26.513 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A= 2022-09-03T06:18:26.515 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A=', 'Content-Length': '0'} 2022-09-03T06:18:26.518 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c078c4e86faef8f9-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.519 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:26.521 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-71 zones={test-zone2, test-zone1} 2022-09-03T06:18:26.521 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-71 zones={test-zone3, test-zone1} 2022-09-03T06:18:26.522 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:26.523 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:26.524 DEBUG:boto:Method: HEAD 2022-09-03T06:18:26.525 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:18:26.526 DEBUG:boto:Data: 2022-09-03T06:18:26.527 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.528 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:26.529 DEBUG:boto:Port: 8001 2022-09-03T06:18:26.530 DEBUG:boto:Params: {} 2022-09-03T06:18:26.531 DEBUG:boto:Token: None 2022-09-03T06:18:26.531 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/ 2022-09-03T06:18:26.532 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WOcf1Fp1K4fpQrthCUAMiPRfH3o= 2022-09-03T06:18:26.534 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WOcf1Fp1K4fpQrthCUAMiPRfH3o=', 'Content-Length': '0'} 2022-09-03T06:18:26.537 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000013fa4423573632e7-006312f1b2-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.538 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:26.539 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:26.540 DEBUG:boto:Method: HEAD 2022-09-03T06:18:26.540 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:18:26.541 DEBUG:boto:Data: 2022-09-03T06:18:26.541 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.542 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.543 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.543 DEBUG:boto:Params: {} 2022-09-03T06:18:26.544 DEBUG:boto:Token: None 2022-09-03T06:18:26.545 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/ 2022-09-03T06:18:26.546 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WOcf1Fp1K4fpQrthCUAMiPRfH3o= 2022-09-03T06:18:26.547 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WOcf1Fp1K4fpQrthCUAMiPRfH3o=', 'Content-Length': '0'} 2022-09-03T06:18:26.550 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006d4f66b6ba84cc9e-006312f1b2-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.552 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:18:26.553 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:26.554 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:26.555 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:26.556 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:26.557 DEBUG:boto:Method: GET 2022-09-03T06:18:26.558 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:26.558 DEBUG:boto:Data: 2022-09-03T06:18:26.559 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.559 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:26.560 DEBUG:boto:Port: 8001 2022-09-03T06:18:26.560 DEBUG:boto:Params: {} 2022-09-03T06:18:26.561 DEBUG:boto:Token: None 2022-09-03T06:18:26.562 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/?versions 2022-09-03T06:18:26.563 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mEDGRVaBj7PzUUXl5L0uHQxpb/w= 2022-09-03T06:18:26.564 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mEDGRVaBj7PzUUXl5L0uHQxpb/w=', 'Content-Length': '0'} 2022-09-03T06:18:26.568 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d23c42b71b925608-006312f1b2-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.568 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:26.569 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:18:26.570 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:18:26.571 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:18:26.571 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:18:26.571 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:18:26.572 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:18:26.572 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:18:26.573 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:18:26.573 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:18:26.574 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:26.575 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:26.575 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:26.576 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:26.576 DEBUG:boto:Method: GET 2022-09-03T06:18:26.577 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:26.577 DEBUG:boto:Data: 2022-09-03T06:18:26.578 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.579 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.579 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.580 DEBUG:boto:Params: {} 2022-09-03T06:18:26.580 DEBUG:boto:Token: None 2022-09-03T06:18:26.581 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/?versions 2022-09-03T06:18:26.581 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mEDGRVaBj7PzUUXl5L0uHQxpb/w= 2022-09-03T06:18:26.582 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mEDGRVaBj7PzUUXl5L0uHQxpb/w=', 'Content-Length': '0'} 2022-09-03T06:18:26.585 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000014931e92bb2c663f-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.586 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:26.587 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:18:26.588 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:18:26.588 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:18:26.589 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:18:26.589 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:18:26.590 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:18:26.590 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:18:26.591 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:18:26.591 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:26.592 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:26.592 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:26.593 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:26.594 DEBUG:boto:Method: GET 2022-09-03T06:18:26.594 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:26.595 DEBUG:boto:Data: 2022-09-03T06:18:26.595 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.596 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:26.596 DEBUG:boto:Port: 8001 2022-09-03T06:18:26.597 DEBUG:boto:Params: {} 2022-09-03T06:18:26.597 DEBUG:boto:Token: None 2022-09-03T06:18:26.598 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/?versions 2022-09-03T06:18:26.599 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mEDGRVaBj7PzUUXl5L0uHQxpb/w= 2022-09-03T06:18:26.599 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mEDGRVaBj7PzUUXl5L0uHQxpb/w=', 'Content-Length': '0'} 2022-09-03T06:18:26.602 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000019ac4d2697495c9c-006312f1b2-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.603 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:26.605 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:26.605 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:26.607 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:26.608 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:26.608 DEBUG:boto:Method: GET 2022-09-03T06:18:26.609 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:26.609 DEBUG:boto:Data: 2022-09-03T06:18:26.610 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.611 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.612 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.613 DEBUG:boto:Params: {} 2022-09-03T06:18:26.615 DEBUG:boto:Token: None 2022-09-03T06:18:26.616 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/?versions 2022-09-03T06:18:26.617 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mEDGRVaBj7PzUUXl5L0uHQxpb/w= 2022-09-03T06:18:26.618 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mEDGRVaBj7PzUUXl5L0uHQxpb/w=', 'Content-Length': '0'} 2022-09-03T06:18:26.621 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000025b00453f718e63c-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.623 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:26.625 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:18:26.626 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:26.627 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:26.629 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.630 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.631 DEBUG:boto:Method: GET 2022-09-03T06:18:26.632 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.633 DEBUG:boto:Data: 2022-09-03T06:18:26.637 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.639 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:26.640 DEBUG:boto:Port: 8001 2022-09-03T06:18:26.643 DEBUG:boto:Params: {} 2022-09-03T06:18:26.645 DEBUG:boto:Token: None 2022-09-03T06:18:26.646 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.647 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kCv5hcwlIBLg0UPos33TEDARhCA= 2022-09-03T06:18:26.648 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kCv5hcwlIBLg0UPos33TEDARhCA=', 'Content-Length': '0'} 2022-09-03T06:18:26.652 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc26a1e36805a891-006312f1b2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.655 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:26.657 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:26.658 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.659 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.660 DEBUG:boto:Method: GET 2022-09-03T06:18:26.662 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.663 DEBUG:boto:Data: 2022-09-03T06:18:26.665 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.666 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.667 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.668 DEBUG:boto:Params: {} 2022-09-03T06:18:26.669 DEBUG:boto:Token: None 2022-09-03T06:18:26.670 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.671 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kCv5hcwlIBLg0UPos33TEDARhCA= 2022-09-03T06:18:26.673 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kCv5hcwlIBLg0UPos33TEDARhCA=', 'Content-Length': '0'} 2022-09-03T06:18:26.675 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000076ccf36837ce97cf-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.677 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:26.678 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:26.679 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.680 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.681 DEBUG:boto:Method: HEAD 2022-09-03T06:18:26.682 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.683 DEBUG:boto:Data: 2022-09-03T06:18:26.684 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.685 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:26.686 DEBUG:boto:Port: 8001 2022-09-03T06:18:26.687 DEBUG:boto:Params: {} 2022-09-03T06:18:26.688 DEBUG:boto:Token: None 2022-09-03T06:18:26.689 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.690 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OaeDamlgpGMDk9Knh+6il5qVHz0= 2022-09-03T06:18:26.691 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OaeDamlgpGMDk9Knh+6il5qVHz0=', 'Content-Length': '0'} 2022-09-03T06:18:26.694 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fde2b9e2e8acbb2c-006312f1b2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.695 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:26.696 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:26.697 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.697 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.698 DEBUG:boto:Method: HEAD 2022-09-03T06:18:26.698 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.699 DEBUG:boto:Data: 2022-09-03T06:18:26.699 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.700 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.700 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.701 DEBUG:boto:Params: {} 2022-09-03T06:18:26.702 DEBUG:boto:Token: None 2022-09-03T06:18:26.702 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.703 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OaeDamlgpGMDk9Knh+6il5qVHz0= 2022-09-03T06:18:26.703 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OaeDamlgpGMDk9Knh+6il5qVHz0=', 'Content-Length': '0'} 2022-09-03T06:18:26.706 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000063e90ce3a2f7d2fc-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.706 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:18:26.707 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:26.708 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:26.708 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.709 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.709 DEBUG:boto:Method: GET 2022-09-03T06:18:26.710 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.711 DEBUG:boto:Data: 2022-09-03T06:18:26.711 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.712 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:26.712 DEBUG:boto:Port: 8001 2022-09-03T06:18:26.713 DEBUG:boto:Params: {} 2022-09-03T06:18:26.714 DEBUG:boto:Token: None 2022-09-03T06:18:26.714 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.715 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kCv5hcwlIBLg0UPos33TEDARhCA= 2022-09-03T06:18:26.715 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kCv5hcwlIBLg0UPos33TEDARhCA=', 'Content-Length': '0'} 2022-09-03T06:18:26.718 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b8765c5fe387d97-006312f1b2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.719 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:26.719 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:26.720 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.721 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.722 DEBUG:boto:Method: GET 2022-09-03T06:18:26.723 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.724 DEBUG:boto:Data: 2022-09-03T06:18:26.725 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.726 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.727 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.728 DEBUG:boto:Params: {} 2022-09-03T06:18:26.728 DEBUG:boto:Token: None 2022-09-03T06:18:26.729 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:26.730 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kCv5hcwlIBLg0UPos33TEDARhCA= 2022-09-03T06:18:26.731 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kCv5hcwlIBLg0UPos33TEDARhCA=', 'Content-Length': '0'} 2022-09-03T06:18:26.733 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b08fc13e9977729-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.735 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:26.736 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:26.737 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:26.738 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:26.739 DEBUG:boto:Method: GET 2022-09-03T06:18:26.740 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:26.741 DEBUG:boto:Data: 2022-09-03T06:18:26.742 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.742 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:26.743 DEBUG:boto:Port: 8001 2022-09-03T06:18:26.743 DEBUG:boto:Params: {} 2022-09-03T06:18:26.744 DEBUG:boto:Token: None 2022-09-03T06:18:26.745 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/ 2022-09-03T06:18:26.746 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A= 2022-09-03T06:18:26.746 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A=', 'Content-Length': '0'} 2022-09-03T06:18:26.749 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000093e4880b31af9911-006312f1b2-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.750 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:26.751 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:26.752 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:26.752 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:26.753 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:26.753 DEBUG:boto:Method: GET 2022-09-03T06:18:26.754 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:26.754 DEBUG:boto:Data: 2022-09-03T06:18:26.755 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.755 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.756 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.756 DEBUG:boto:Params: {} 2022-09-03T06:18:26.757 DEBUG:boto:Token: None 2022-09-03T06:18:26.758 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/ 2022-09-03T06:18:26.758 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A= 2022-09-03T06:18:26.759 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A=', 'Content-Length': '0'} 2022-09-03T06:18:26.762 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b4632ad7e399b0a8-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.762 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:26.763 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:26.764 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:26.764 DEBUG:boto:Method: HEAD 2022-09-03T06:18:26.765 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:26.765 DEBUG:boto:Data: 2022-09-03T06:18:26.766 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.767 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:26.767 DEBUG:boto:Port: 8001 2022-09-03T06:18:26.768 DEBUG:boto:Params: {} 2022-09-03T06:18:26.768 DEBUG:boto:Token: None 2022-09-03T06:18:26.769 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/a 2022-09-03T06:18:26.769 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RVuZn8kW3xlwYdWb36hTi4JAHDo= 2022-09-03T06:18:26.770 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RVuZn8kW3xlwYdWb36hTi4JAHDo=', 'Content-Length': '0'} 2022-09-03T06:18:26.772 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fcde364a4e07a68c-006312f1b2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.774 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:26.775 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:26.775 DEBUG:boto:Method: HEAD 2022-09-03T06:18:26.776 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:26.777 DEBUG:boto:Data: 2022-09-03T06:18:26.778 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.779 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.779 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.780 DEBUG:boto:Params: {} 2022-09-03T06:18:26.781 DEBUG:boto:Token: None 2022-09-03T06:18:26.781 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/a 2022-09-03T06:18:26.782 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RVuZn8kW3xlwYdWb36hTi4JAHDo= 2022-09-03T06:18:26.783 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RVuZn8kW3xlwYdWb36hTi4JAHDo=', 'Content-Length': '0'} 2022-09-03T06:18:26.785 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002423dd89ccbc0e9c-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.786 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:18:26.787 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:26.787 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:26.788 DEBUG:boto:Method: GET 2022-09-03T06:18:26.789 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:26.789 DEBUG:boto:Data: 2022-09-03T06:18:26.790 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.790 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:26.791 DEBUG:boto:Port: 8001 2022-09-03T06:18:26.791 DEBUG:boto:Params: {} 2022-09-03T06:18:26.792 DEBUG:boto:Token: None 2022-09-03T06:18:26.792 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/a 2022-09-03T06:18:26.793 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HQTmJIJIvPEx8N4XCp7xdtLnwW0= 2022-09-03T06:18:26.794 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HQTmJIJIvPEx8N4XCp7xdtLnwW0=', 'Content-Length': '0'} 2022-09-03T06:18:26.796 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000454888b124eb63d3-006312f1b2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.797 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:26.797 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:26.798 DEBUG:boto:Method: GET 2022-09-03T06:18:26.798 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:26.799 DEBUG:boto:Data: 2022-09-03T06:18:26.799 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.800 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.800 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.801 DEBUG:boto:Params: {} 2022-09-03T06:18:26.802 DEBUG:boto:Token: None 2022-09-03T06:18:26.802 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/a 2022-09-03T06:18:26.803 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HQTmJIJIvPEx8N4XCp7xdtLnwW0= 2022-09-03T06:18:26.803 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HQTmJIJIvPEx8N4XCp7xdtLnwW0=', 'Content-Length': '0'} 2022-09-03T06:18:26.806 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b236e25219b025b1-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.806 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:26.807 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:26.807 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:26.808 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:26.809 DEBUG:boto:Method: GET 2022-09-03T06:18:26.809 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:26.810 DEBUG:boto:Data: 2022-09-03T06:18:26.810 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.811 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:26.811 DEBUG:boto:Port: 8001 2022-09-03T06:18:26.812 DEBUG:boto:Params: {} 2022-09-03T06:18:26.812 DEBUG:boto:Token: None 2022-09-03T06:18:26.813 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/ 2022-09-03T06:18:26.813 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A= 2022-09-03T06:18:26.814 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A=', 'Content-Length': '0'} 2022-09-03T06:18:26.825 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008ff72c82193bc2ca-006312f1b2-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.826 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:26.827 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:26.827 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:26.828 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:26.828 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:26.829 DEBUG:boto:Method: GET 2022-09-03T06:18:26.830 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:26.830 DEBUG:boto:Data: 2022-09-03T06:18:26.831 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.831 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.832 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.832 DEBUG:boto:Params: {} 2022-09-03T06:18:26.833 DEBUG:boto:Token: None 2022-09-03T06:18:26.833 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/ 2022-09-03T06:18:26.834 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A= 2022-09-03T06:18:26.835 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A=', 'Content-Length': '0'} 2022-09-03T06:18:26.837 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c64eb65d1a15a9c8-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.838 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:26.839 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:18:26.839 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:26.840 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:26.840 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.841 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.841 DEBUG:boto:Method: GET 2022-09-03T06:18:26.842 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.842 DEBUG:boto:Data: 2022-09-03T06:18:26.843 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.843 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:26.844 DEBUG:boto:Port: 8001 2022-09-03T06:18:26.844 DEBUG:boto:Params: {} 2022-09-03T06:18:26.845 DEBUG:boto:Token: None 2022-09-03T06:18:26.846 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.846 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OLUR3dDKXk1wuAMaOqS3YGb73aU= 2022-09-03T06:18:26.847 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OLUR3dDKXk1wuAMaOqS3YGb73aU=', 'Content-Length': '0'} 2022-09-03T06:18:26.849 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d42ad879412c29bd-006312f1b2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.851 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:26.852 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:26.853 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.854 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.854 DEBUG:boto:Method: GET 2022-09-03T06:18:26.855 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.856 DEBUG:boto:Data: 2022-09-03T06:18:26.857 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.857 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.858 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.858 DEBUG:boto:Params: {} 2022-09-03T06:18:26.859 DEBUG:boto:Token: None 2022-09-03T06:18:26.860 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.861 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OLUR3dDKXk1wuAMaOqS3YGb73aU= 2022-09-03T06:18:26.861 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OLUR3dDKXk1wuAMaOqS3YGb73aU=', 'Content-Length': '0'} 2022-09-03T06:18:26.864 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000acc216ecc1d86441-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.865 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:26.866 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:26.867 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.867 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.868 DEBUG:boto:Method: HEAD 2022-09-03T06:18:26.869 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.870 DEBUG:boto:Data: 2022-09-03T06:18:26.870 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.871 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:26.871 DEBUG:boto:Port: 8001 2022-09-03T06:18:26.872 DEBUG:boto:Params: {} 2022-09-03T06:18:26.873 DEBUG:boto:Token: None 2022-09-03T06:18:26.873 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.874 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fQtXVe5b6dQDeWI9H6qlsKpOFo= 2022-09-03T06:18:26.874 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fQtXVe5b6dQDeWI9H6qlsKpOFo=', 'Content-Length': '0'} 2022-09-03T06:18:26.876 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006bc000fce60376e6-006312f1b2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.877 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:26.878 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:26.878 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.879 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.879 DEBUG:boto:Method: HEAD 2022-09-03T06:18:26.880 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.881 DEBUG:boto:Data: 2022-09-03T06:18:26.881 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.882 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.882 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.883 DEBUG:boto:Params: {} 2022-09-03T06:18:26.884 DEBUG:boto:Token: None 2022-09-03T06:18:26.885 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.885 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4fQtXVe5b6dQDeWI9H6qlsKpOFo= 2022-09-03T06:18:26.886 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4fQtXVe5b6dQDeWI9H6qlsKpOFo=', 'Content-Length': '0'} 2022-09-03T06:18:26.888 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000e4bdc7573be2ac7-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.890 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:18:26.891 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:26.892 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:26.892 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.893 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.894 DEBUG:boto:Method: GET 2022-09-03T06:18:26.895 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.895 DEBUG:boto:Data: 2022-09-03T06:18:26.896 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.897 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:26.897 DEBUG:boto:Port: 8001 2022-09-03T06:18:26.898 DEBUG:boto:Params: {} 2022-09-03T06:18:26.899 DEBUG:boto:Token: None 2022-09-03T06:18:26.900 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.901 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OLUR3dDKXk1wuAMaOqS3YGb73aU= 2022-09-03T06:18:26.903 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OLUR3dDKXk1wuAMaOqS3YGb73aU=', 'Content-Length': '0'} 2022-09-03T06:18:26.905 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b557a1cc432331d0-006312f1b2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.906 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:26.906 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:26.907 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.907 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.908 DEBUG:boto:Method: GET 2022-09-03T06:18:26.909 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.909 DEBUG:boto:Data: 2022-09-03T06:18:26.910 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.911 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.911 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.912 DEBUG:boto:Params: {} 2022-09-03T06:18:26.912 DEBUG:boto:Token: None 2022-09-03T06:18:26.913 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:26.914 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OLUR3dDKXk1wuAMaOqS3YGb73aU= 2022-09-03T06:18:26.914 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OLUR3dDKXk1wuAMaOqS3YGb73aU=', 'Content-Length': '0'} 2022-09-03T06:18:26.916 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000476aa48637057095-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.918 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:26.919 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:26.920 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:26.921 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:26.922 DEBUG:boto:Method: GET 2022-09-03T06:18:26.923 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:26.924 DEBUG:boto:Data: 2022-09-03T06:18:26.924 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.926 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:26.927 DEBUG:boto:Port: 8001 2022-09-03T06:18:26.928 DEBUG:boto:Params: {} 2022-09-03T06:18:26.929 DEBUG:boto:Token: None 2022-09-03T06:18:26.931 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/ 2022-09-03T06:18:26.932 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A= 2022-09-03T06:18:26.933 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A=', 'Content-Length': '0'} 2022-09-03T06:18:26.936 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000f0fd041f2244a2d-006312f1b2-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.937 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:26.939 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:26.940 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:26.940 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:26.941 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:26.941 DEBUG:boto:Method: GET 2022-09-03T06:18:26.942 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:26.943 DEBUG:boto:Data: 2022-09-03T06:18:26.943 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.944 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.944 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.945 DEBUG:boto:Params: {} 2022-09-03T06:18:26.946 DEBUG:boto:Token: None 2022-09-03T06:18:26.946 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/ 2022-09-03T06:18:26.947 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A= 2022-09-03T06:18:26.947 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0l5QFfeUkEWara356XnKQkRWi5A=', 'Content-Length': '0'} 2022-09-03T06:18:26.950 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000047bfa413ce5e009-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.951 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:26.951 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:26.952 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:26.953 DEBUG:boto:Method: HEAD 2022-09-03T06:18:26.954 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:26.954 DEBUG:boto:Data: 2022-09-03T06:18:26.955 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.955 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:26.956 DEBUG:boto:Port: 8001 2022-09-03T06:18:26.956 DEBUG:boto:Params: {} 2022-09-03T06:18:26.957 DEBUG:boto:Token: None 2022-09-03T06:18:26.958 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/b 2022-09-03T06:18:26.958 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sFHLwClOBoO1WXhzl5N+JJyaQHs= 2022-09-03T06:18:26.959 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sFHLwClOBoO1WXhzl5N+JJyaQHs=', 'Content-Length': '0'} 2022-09-03T06:18:26.961 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b227348c0f3b9865-006312f1b2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.962 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:26.962 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:26.963 DEBUG:boto:Method: HEAD 2022-09-03T06:18:26.964 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:26.964 DEBUG:boto:Data: 2022-09-03T06:18:26.965 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.965 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:26.966 DEBUG:boto:Port: 8000 2022-09-03T06:18:26.967 DEBUG:boto:Params: {} 2022-09-03T06:18:26.967 DEBUG:boto:Token: None 2022-09-03T06:18:26.968 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/b 2022-09-03T06:18:26.968 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sFHLwClOBoO1WXhzl5N+JJyaQHs= 2022-09-03T06:18:26.969 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sFHLwClOBoO1WXhzl5N+JJyaQHs=', 'Content-Length': '0'} 2022-09-03T06:18:26.971 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e739de84739bdf1d-006312f1b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.972 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:18:26.973 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:26.973 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:26.980 DEBUG:boto:Method: GET 2022-09-03T06:18:26.981 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:26.982 DEBUG:boto:Data: 2022-09-03T06:18:26.983 DEBUG:boto:Headers: {} 2022-09-03T06:18:26.984 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:26.985 DEBUG:boto:Port: 8001 2022-09-03T06:18:26.986 DEBUG:boto:Params: {} 2022-09-03T06:18:26.988 DEBUG:boto:Token: None 2022-09-03T06:18:26.989 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:26 GMT /dsgvcy-71/b 2022-09-03T06:18:26.990 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nLBwOkr8vibp6zY/J0+fY6MiN6A= 2022-09-03T06:18:26.991 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nLBwOkr8vibp6zY/J0+fY6MiN6A=', 'Content-Length': '0'} 2022-09-03T06:18:26.994 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d8a7d8a904668a92-006312f1b2-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:26.996 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:26.997 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:26.998 DEBUG:boto:Method: GET 2022-09-03T06:18:26.999 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:27.000 DEBUG:boto:Data: 2022-09-03T06:18:27.001 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.002 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.003 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.004 DEBUG:boto:Params: {} 2022-09-03T06:18:27.006 DEBUG:boto:Token: None 2022-09-03T06:18:27.007 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/b 2022-09-03T06:18:27.008 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ROFpA/KUx/XiF/+go4/ebIZDzsc= 2022-09-03T06:18:27.009 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ROFpA/KUx/XiF/+go4/ebIZDzsc=', 'Content-Length': '0'} 2022-09-03T06:18:27.012 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c29f0a362435edba-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.014 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:27.015 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:27.016 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:27.017 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:27.018 DEBUG:boto:Method: GET 2022-09-03T06:18:27.019 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:27.020 DEBUG:boto:Data: 2022-09-03T06:18:27.021 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.022 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.023 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.024 DEBUG:boto:Params: {} 2022-09-03T06:18:27.025 DEBUG:boto:Token: None 2022-09-03T06:18:27.026 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/ 2022-09-03T06:18:27.027 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM= 2022-09-03T06:18:27.028 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM=', 'Content-Length': '0'} 2022-09-03T06:18:27.031 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a405043bfaa0a2c6-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.033 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:27.034 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:27.035 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:27.036 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:27.038 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:27.039 DEBUG:boto:Method: GET 2022-09-03T06:18:27.040 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:27.041 DEBUG:boto:Data: 2022-09-03T06:18:27.042 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.043 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.044 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.045 DEBUG:boto:Params: {} 2022-09-03T06:18:27.046 DEBUG:boto:Token: None 2022-09-03T06:18:27.047 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/ 2022-09-03T06:18:27.049 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM= 2022-09-03T06:18:27.050 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM=', 'Content-Length': '0'} 2022-09-03T06:18:27.053 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002a05bd02156156fa-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.054 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:27.055 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:18:27.056 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:27.057 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:27.059 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.061 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.062 DEBUG:boto:Method: GET 2022-09-03T06:18:27.063 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.064 DEBUG:boto:Data: 2022-09-03T06:18:27.065 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.066 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.067 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.067 DEBUG:boto:Params: {} 2022-09-03T06:18:27.068 DEBUG:boto:Token: None 2022-09-03T06:18:27.068 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.069 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UXUMpAZPgRgmx88kByoZYjLdAQ4= 2022-09-03T06:18:27.070 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UXUMpAZPgRgmx88kByoZYjLdAQ4=', 'Content-Length': '0'} 2022-09-03T06:18:27.072 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b390230b299b878d-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.076 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:27.077 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:27.078 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.078 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.079 DEBUG:boto:Method: GET 2022-09-03T06:18:27.080 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.081 DEBUG:boto:Data: 2022-09-03T06:18:27.083 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.084 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.084 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.085 DEBUG:boto:Params: {} 2022-09-03T06:18:27.086 DEBUG:boto:Token: None 2022-09-03T06:18:27.086 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.087 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UXUMpAZPgRgmx88kByoZYjLdAQ4= 2022-09-03T06:18:27.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UXUMpAZPgRgmx88kByoZYjLdAQ4=', 'Content-Length': '0'} 2022-09-03T06:18:27.090 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d5bfff34091accc5-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.091 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:27.092 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:27.092 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.093 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.094 DEBUG:boto:Method: HEAD 2022-09-03T06:18:27.095 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.096 DEBUG:boto:Data: 2022-09-03T06:18:27.097 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.098 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.100 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.101 DEBUG:boto:Params: {} 2022-09-03T06:18:27.102 DEBUG:boto:Token: None 2022-09-03T06:18:27.103 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.104 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ML+Rgv8G6gftYGV6/3ozE89RRLk= 2022-09-03T06:18:27.105 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ML+Rgv8G6gftYGV6/3ozE89RRLk=', 'Content-Length': '0'} 2022-09-03T06:18:27.108 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e1ff3470ebb5050-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.109 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:27.109 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:27.110 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.110 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.111 DEBUG:boto:Method: HEAD 2022-09-03T06:18:27.111 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.112 DEBUG:boto:Data: 2022-09-03T06:18:27.113 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.114 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.114 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.115 DEBUG:boto:Params: {} 2022-09-03T06:18:27.116 DEBUG:boto:Token: None 2022-09-03T06:18:27.116 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.117 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ML+Rgv8G6gftYGV6/3ozE89RRLk= 2022-09-03T06:18:27.117 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ML+Rgv8G6gftYGV6/3ozE89RRLk=', 'Content-Length': '0'} 2022-09-03T06:18:27.120 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c564a4318f2025c5-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.120 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:18:27.121 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:27.122 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:27.122 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.123 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.124 DEBUG:boto:Method: GET 2022-09-03T06:18:27.124 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.125 DEBUG:boto:Data: 2022-09-03T06:18:27.125 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.126 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.126 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.127 DEBUG:boto:Params: {} 2022-09-03T06:18:27.128 DEBUG:boto:Token: None 2022-09-03T06:18:27.128 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.129 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UXUMpAZPgRgmx88kByoZYjLdAQ4= 2022-09-03T06:18:27.130 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UXUMpAZPgRgmx88kByoZYjLdAQ4=', 'Content-Length': '0'} 2022-09-03T06:18:27.132 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a02fe3fbcf844ab2-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.174 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:27.175 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:27.176 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.177 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.177 DEBUG:boto:Method: GET 2022-09-03T06:18:27.178 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.178 DEBUG:boto:Data: 2022-09-03T06:18:27.179 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.180 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.180 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.181 DEBUG:boto:Params: {} 2022-09-03T06:18:27.182 DEBUG:boto:Token: None 2022-09-03T06:18:27.183 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:27.184 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UXUMpAZPgRgmx88kByoZYjLdAQ4= 2022-09-03T06:18:27.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UXUMpAZPgRgmx88kByoZYjLdAQ4=', 'Content-Length': '0'} 2022-09-03T06:18:27.189 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007bf07a6762281fca-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.191 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:27.192 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:27.193 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:27.197 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:27.198 DEBUG:boto:Method: GET 2022-09-03T06:18:27.199 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:27.199 DEBUG:boto:Data: 2022-09-03T06:18:27.200 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.200 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.201 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.202 DEBUG:boto:Params: {} 2022-09-03T06:18:27.202 DEBUG:boto:Token: None 2022-09-03T06:18:27.203 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/ 2022-09-03T06:18:27.204 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM= 2022-09-03T06:18:27.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM=', 'Content-Length': '0'} 2022-09-03T06:18:27.208 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000be50eb8b6bef493a-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.209 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:27.210 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:27.210 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:27.211 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:27.211 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:27.212 DEBUG:boto:Method: GET 2022-09-03T06:18:27.213 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:27.213 DEBUG:boto:Data: 2022-09-03T06:18:27.214 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.214 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.215 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.215 DEBUG:boto:Params: {} 2022-09-03T06:18:27.216 DEBUG:boto:Token: None 2022-09-03T06:18:27.217 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/ 2022-09-03T06:18:27.217 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM= 2022-09-03T06:18:27.218 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM=', 'Content-Length': '0'} 2022-09-03T06:18:27.221 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000354012d7ae2ab45b-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.221 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:27.222 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:27.223 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:27.224 DEBUG:boto:Method: HEAD 2022-09-03T06:18:27.224 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:27.225 DEBUG:boto:Data: 2022-09-03T06:18:27.225 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.226 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.226 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.227 DEBUG:boto:Params: {} 2022-09-03T06:18:27.227 DEBUG:boto:Token: None 2022-09-03T06:18:27.228 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/c 2022-09-03T06:18:27.229 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WmB3vWkY2Td3cyq4k6Q/aeicnu0= 2022-09-03T06:18:27.230 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WmB3vWkY2Td3cyq4k6Q/aeicnu0=', 'Content-Length': '0'} 2022-09-03T06:18:27.234 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000145254b60125f51a-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.235 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:27.235 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:27.236 DEBUG:boto:Method: HEAD 2022-09-03T06:18:27.237 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:27.237 DEBUG:boto:Data: 2022-09-03T06:18:27.238 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.238 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.239 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.239 DEBUG:boto:Params: {} 2022-09-03T06:18:27.240 DEBUG:boto:Token: None 2022-09-03T06:18:27.240 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/c 2022-09-03T06:18:27.241 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WmB3vWkY2Td3cyq4k6Q/aeicnu0= 2022-09-03T06:18:27.242 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WmB3vWkY2Td3cyq4k6Q/aeicnu0=', 'Content-Length': '0'} 2022-09-03T06:18:27.244 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007a7f9a25b4c5c6e1-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.245 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:18:27.245 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:27.246 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:27.246 DEBUG:boto:Method: GET 2022-09-03T06:18:27.247 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:27.248 DEBUG:boto:Data: 2022-09-03T06:18:27.248 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.249 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.249 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.250 DEBUG:boto:Params: {} 2022-09-03T06:18:27.251 DEBUG:boto:Token: None 2022-09-03T06:18:27.251 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/c 2022-09-03T06:18:27.252 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nIFO71P/zYbxUrIVR6QpnAQSkMA= 2022-09-03T06:18:27.253 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nIFO71P/zYbxUrIVR6QpnAQSkMA=', 'Content-Length': '0'} 2022-09-03T06:18:27.255 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cde07246dd5ec130-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.256 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:27.256 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:27.257 DEBUG:boto:Method: GET 2022-09-03T06:18:27.257 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:27.258 DEBUG:boto:Data: 2022-09-03T06:18:27.258 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.259 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.260 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.261 DEBUG:boto:Params: {} 2022-09-03T06:18:27.261 DEBUG:boto:Token: None 2022-09-03T06:18:27.262 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/c 2022-09-03T06:18:27.263 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nIFO71P/zYbxUrIVR6QpnAQSkMA= 2022-09-03T06:18:27.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nIFO71P/zYbxUrIVR6QpnAQSkMA=', 'Content-Length': '0'} 2022-09-03T06:18:27.266 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ab4d48b8108fea16-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.267 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:27.268 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:27.269 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:27.269 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:27.270 DEBUG:boto:Method: GET 2022-09-03T06:18:27.271 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:27.271 DEBUG:boto:Data: 2022-09-03T06:18:27.272 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.273 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.273 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.274 DEBUG:boto:Params: {} 2022-09-03T06:18:27.274 DEBUG:boto:Token: None 2022-09-03T06:18:27.275 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/ 2022-09-03T06:18:27.276 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM= 2022-09-03T06:18:27.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM=', 'Content-Length': '0'} 2022-09-03T06:18:27.279 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000abc4cf6edd0c8077-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.322 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:27.324 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:27.325 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:27.325 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:27.326 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:27.326 DEBUG:boto:Method: GET 2022-09-03T06:18:27.327 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:27.328 DEBUG:boto:Data: 2022-09-03T06:18:27.328 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.329 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.329 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.330 DEBUG:boto:Params: {} 2022-09-03T06:18:27.330 DEBUG:boto:Token: None 2022-09-03T06:18:27.331 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/ 2022-09-03T06:18:27.332 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM= 2022-09-03T06:18:27.332 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM=', 'Content-Length': '0'} 2022-09-03T06:18:27.335 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e9921cfcc9155834-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.336 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:27.337 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:18:27.338 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:27.339 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:27.340 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.341 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.342 DEBUG:boto:Method: GET 2022-09-03T06:18:27.343 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.344 DEBUG:boto:Data: 2022-09-03T06:18:27.345 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.346 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.347 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.348 DEBUG:boto:Params: {} 2022-09-03T06:18:27.349 DEBUG:boto:Token: None 2022-09-03T06:18:27.350 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.351 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S1ZiPBf0CDEg9PdBdfpUaTggRVI= 2022-09-03T06:18:27.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S1ZiPBf0CDEg9PdBdfpUaTggRVI=', 'Content-Length': '0'} 2022-09-03T06:18:27.355 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000339c985d8d3c9bc3-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.358 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:27.359 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:27.360 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.360 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.361 DEBUG:boto:Method: GET 2022-09-03T06:18:27.362 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.363 DEBUG:boto:Data: 2022-09-03T06:18:27.364 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.365 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.366 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.367 DEBUG:boto:Params: {} 2022-09-03T06:18:27.369 DEBUG:boto:Token: None 2022-09-03T06:18:27.370 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.371 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S1ZiPBf0CDEg9PdBdfpUaTggRVI= 2022-09-03T06:18:27.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S1ZiPBf0CDEg9PdBdfpUaTggRVI=', 'Content-Length': '0'} 2022-09-03T06:18:27.375 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a10c9dd3826b8c05-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.418 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:27.419 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:27.420 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.421 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.421 DEBUG:boto:Method: HEAD 2022-09-03T06:18:27.422 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.423 DEBUG:boto:Data: 2022-09-03T06:18:27.424 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.424 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.426 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.428 DEBUG:boto:Params: {} 2022-09-03T06:18:27.429 DEBUG:boto:Token: None 2022-09-03T06:18:27.430 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.431 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GkwcGZ5A2RBdPUBR8IAQsdKgtKs= 2022-09-03T06:18:27.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GkwcGZ5A2RBdPUBR8IAQsdKgtKs=', 'Content-Length': '0'} 2022-09-03T06:18:27.434 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000077cfadb167ebccf2-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.437 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:27.438 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:27.438 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.439 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.440 DEBUG:boto:Method: HEAD 2022-09-03T06:18:27.441 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.442 DEBUG:boto:Data: 2022-09-03T06:18:27.442 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.443 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.444 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.444 DEBUG:boto:Params: {} 2022-09-03T06:18:27.445 DEBUG:boto:Token: None 2022-09-03T06:18:27.445 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.446 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GkwcGZ5A2RBdPUBR8IAQsdKgtKs= 2022-09-03T06:18:27.447 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GkwcGZ5A2RBdPUBR8IAQsdKgtKs=', 'Content-Length': '0'} 2022-09-03T06:18:27.449 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c6b51eb21f18f60-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.450 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:18:27.451 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:27.451 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:27.452 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.452 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.453 DEBUG:boto:Method: GET 2022-09-03T06:18:27.454 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.454 DEBUG:boto:Data: 2022-09-03T06:18:27.455 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.455 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.456 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.456 DEBUG:boto:Params: {} 2022-09-03T06:18:27.457 DEBUG:boto:Token: None 2022-09-03T06:18:27.458 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.459 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S1ZiPBf0CDEg9PdBdfpUaTggRVI= 2022-09-03T06:18:27.459 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S1ZiPBf0CDEg9PdBdfpUaTggRVI=', 'Content-Length': '0'} 2022-09-03T06:18:27.461 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dee322472835f448-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.462 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:27.463 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:27.463 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.464 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.464 DEBUG:boto:Method: GET 2022-09-03T06:18:27.465 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.465 DEBUG:boto:Data: 2022-09-03T06:18:27.466 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.466 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.468 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.469 DEBUG:boto:Params: {} 2022-09-03T06:18:27.470 DEBUG:boto:Token: None 2022-09-03T06:18:27.471 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:27.472 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S1ZiPBf0CDEg9PdBdfpUaTggRVI= 2022-09-03T06:18:27.473 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S1ZiPBf0CDEg9PdBdfpUaTggRVI=', 'Content-Length': '0'} 2022-09-03T06:18:27.478 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c20286295f651a57-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.479 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:27.479 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:27.480 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:27.480 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:27.481 DEBUG:boto:Method: GET 2022-09-03T06:18:27.481 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:27.482 DEBUG:boto:Data: 2022-09-03T06:18:27.483 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.483 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.484 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.484 DEBUG:boto:Params: {} 2022-09-03T06:18:27.485 DEBUG:boto:Token: None 2022-09-03T06:18:27.486 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/ 2022-09-03T06:18:27.486 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM= 2022-09-03T06:18:27.487 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM=', 'Content-Length': '0'} 2022-09-03T06:18:27.490 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000078fe584820b462cf-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.491 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:27.492 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:27.493 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:27.493 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:27.494 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:27.494 DEBUG:boto:Method: GET 2022-09-03T06:18:27.495 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:27.495 DEBUG:boto:Data: 2022-09-03T06:18:27.497 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.498 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.499 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.500 DEBUG:boto:Params: {} 2022-09-03T06:18:27.501 DEBUG:boto:Token: None 2022-09-03T06:18:27.503 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/ 2022-09-03T06:18:27.503 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM= 2022-09-03T06:18:27.504 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM=', 'Content-Length': '0'} 2022-09-03T06:18:27.507 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ff50c7ca239a380f-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.507 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:27.508 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:27.509 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:27.510 DEBUG:boto:Method: HEAD 2022-09-03T06:18:27.511 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:27.511 DEBUG:boto:Data: 2022-09-03T06:18:27.512 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.512 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.513 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.513 DEBUG:boto:Params: {} 2022-09-03T06:18:27.514 DEBUG:boto:Token: None 2022-09-03T06:18:27.515 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/d 2022-09-03T06:18:27.515 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X1R8jeQo0u/TRyf9C9ejfUAFLLw= 2022-09-03T06:18:27.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X1R8jeQo0u/TRyf9C9ejfUAFLLw=', 'Content-Length': '0'} 2022-09-03T06:18:27.518 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000052012b47cd7db586-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.519 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:27.520 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:27.521 DEBUG:boto:Method: HEAD 2022-09-03T06:18:27.521 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:27.522 DEBUG:boto:Data: 2022-09-03T06:18:27.522 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.523 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.523 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.524 DEBUG:boto:Params: {} 2022-09-03T06:18:27.524 DEBUG:boto:Token: None 2022-09-03T06:18:27.525 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/d 2022-09-03T06:18:27.526 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X1R8jeQo0u/TRyf9C9ejfUAFLLw= 2022-09-03T06:18:27.526 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X1R8jeQo0u/TRyf9C9ejfUAFLLw=', 'Content-Length': '0'} 2022-09-03T06:18:27.528 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000090885d698f3a5cca-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.529 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:18:27.530 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:27.531 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:27.531 DEBUG:boto:Method: GET 2022-09-03T06:18:27.532 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:27.532 DEBUG:boto:Data: 2022-09-03T06:18:27.533 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.534 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.534 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.535 DEBUG:boto:Params: {} 2022-09-03T06:18:27.535 DEBUG:boto:Token: None 2022-09-03T06:18:27.536 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/d 2022-09-03T06:18:27.536 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q9Yur6JVd6V7ok4E/HXWP/hJjE4= 2022-09-03T06:18:27.537 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q9Yur6JVd6V7ok4E/HXWP/hJjE4=', 'Content-Length': '0'} 2022-09-03T06:18:27.539 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ccddf6934d5a23c4-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.586 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:27.587 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:27.587 DEBUG:boto:Method: GET 2022-09-03T06:18:27.588 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:27.589 DEBUG:boto:Data: 2022-09-03T06:18:27.589 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.590 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.590 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.591 DEBUG:boto:Params: {} 2022-09-03T06:18:27.592 DEBUG:boto:Token: None 2022-09-03T06:18:27.592 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/d 2022-09-03T06:18:27.593 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q9Yur6JVd6V7ok4E/HXWP/hJjE4= 2022-09-03T06:18:27.594 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q9Yur6JVd6V7ok4E/HXWP/hJjE4=', 'Content-Length': '0'} 2022-09-03T06:18:27.596 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005ec7180a46dbba59-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.597 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:27.597 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:27.598 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:27.599 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:27.600 DEBUG:boto:Method: GET 2022-09-03T06:18:27.601 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:27.601 DEBUG:boto:Data: 2022-09-03T06:18:27.602 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.603 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.604 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.605 DEBUG:boto:Params: {} 2022-09-03T06:18:27.605 DEBUG:boto:Token: None 2022-09-03T06:18:27.606 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/ 2022-09-03T06:18:27.607 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM= 2022-09-03T06:18:27.608 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM=', 'Content-Length': '0'} 2022-09-03T06:18:27.611 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000000a481875dd5c854-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.612 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:27.614 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:27.615 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:27.616 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:27.616 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:27.617 DEBUG:boto:Method: GET 2022-09-03T06:18:27.618 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:27.620 DEBUG:boto:Data: 2022-09-03T06:18:27.620 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.621 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.622 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.623 DEBUG:boto:Params: {} 2022-09-03T06:18:27.625 DEBUG:boto:Token: None 2022-09-03T06:18:27.626 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/ 2022-09-03T06:18:27.627 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM= 2022-09-03T06:18:27.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM=', 'Content-Length': '0'} 2022-09-03T06:18:27.635 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000047b8527a0d7d956d-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.636 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:27.638 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:18:27.639 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:27.640 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:27.641 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.642 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.643 DEBUG:boto:Method: GET 2022-09-03T06:18:27.644 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.646 DEBUG:boto:Data: 2022-09-03T06:18:27.647 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.648 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.649 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.650 DEBUG:boto:Params: {} 2022-09-03T06:18:27.651 DEBUG:boto:Token: None 2022-09-03T06:18:27.652 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.653 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oUoRKyus/08rZNNpTr8Zxi4FKlc= 2022-09-03T06:18:27.654 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oUoRKyus/08rZNNpTr8Zxi4FKlc=', 'Content-Length': '0'} 2022-09-03T06:18:27.657 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000011c6f2c8f1bf5cfc-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.658 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:27.659 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:27.660 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.660 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.661 DEBUG:boto:Method: GET 2022-09-03T06:18:27.662 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.663 DEBUG:boto:Data: 2022-09-03T06:18:27.664 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.666 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.667 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.668 DEBUG:boto:Params: {} 2022-09-03T06:18:27.671 DEBUG:boto:Token: None 2022-09-03T06:18:27.671 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.672 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oUoRKyus/08rZNNpTr8Zxi4FKlc= 2022-09-03T06:18:27.673 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oUoRKyus/08rZNNpTr8Zxi4FKlc=', 'Content-Length': '0'} 2022-09-03T06:18:27.676 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003f8501186c029f08-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.677 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:27.677 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:27.678 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.679 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.679 DEBUG:boto:Method: HEAD 2022-09-03T06:18:27.680 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.680 DEBUG:boto:Data: 2022-09-03T06:18:27.681 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.682 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.683 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.684 DEBUG:boto:Params: {} 2022-09-03T06:18:27.685 DEBUG:boto:Token: None 2022-09-03T06:18:27.686 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.687 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+D7W8l3NStUVGqvXJbZJ62sOElI= 2022-09-03T06:18:27.688 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+D7W8l3NStUVGqvXJbZJ62sOElI=', 'Content-Length': '0'} 2022-09-03T06:18:27.691 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e5c0e6971fc5aa73-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.693 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:27.695 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:27.695 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.697 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.698 DEBUG:boto:Method: HEAD 2022-09-03T06:18:27.699 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.700 DEBUG:boto:Data: 2022-09-03T06:18:27.701 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.702 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.703 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.704 DEBUG:boto:Params: {} 2022-09-03T06:18:27.705 DEBUG:boto:Token: None 2022-09-03T06:18:27.706 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.707 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+D7W8l3NStUVGqvXJbZJ62sOElI= 2022-09-03T06:18:27.707 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+D7W8l3NStUVGqvXJbZJ62sOElI=', 'Content-Length': '0'} 2022-09-03T06:18:27.710 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000584153c770c2af4a-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.710 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:18:27.711 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:27.712 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:27.712 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.713 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.713 DEBUG:boto:Method: GET 2022-09-03T06:18:27.714 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.714 DEBUG:boto:Data: 2022-09-03T06:18:27.716 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.716 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.717 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.718 DEBUG:boto:Params: {} 2022-09-03T06:18:27.719 DEBUG:boto:Token: None 2022-09-03T06:18:27.719 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.720 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oUoRKyus/08rZNNpTr8Zxi4FKlc= 2022-09-03T06:18:27.721 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oUoRKyus/08rZNNpTr8Zxi4FKlc=', 'Content-Length': '0'} 2022-09-03T06:18:27.723 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b24b5ab01955eb2-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.724 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:27.724 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:27.725 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.726 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.727 DEBUG:boto:Method: GET 2022-09-03T06:18:27.727 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.728 DEBUG:boto:Data: 2022-09-03T06:18:27.728 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.729 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.729 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.730 DEBUG:boto:Params: {} 2022-09-03T06:18:27.730 DEBUG:boto:Token: None 2022-09-03T06:18:27.731 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:27.731 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oUoRKyus/08rZNNpTr8Zxi4FKlc= 2022-09-03T06:18:27.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oUoRKyus/08rZNNpTr8Zxi4FKlc=', 'Content-Length': '0'} 2022-09-03T06:18:27.734 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ed5243833257526e-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.735 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:27.736 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:27.737 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:27.737 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:27.738 DEBUG:boto:Method: GET 2022-09-03T06:18:27.738 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:27.739 DEBUG:boto:Data: 2022-09-03T06:18:27.739 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.740 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.740 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.741 DEBUG:boto:Params: {} 2022-09-03T06:18:27.742 DEBUG:boto:Token: None 2022-09-03T06:18:27.742 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/ 2022-09-03T06:18:27.743 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM= 2022-09-03T06:18:27.743 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM=', 'Content-Length': '0'} 2022-09-03T06:18:27.747 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000463543fc0d30e974-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.790 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:27.791 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:27.792 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:27.792 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:27.793 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:27.793 DEBUG:boto:Method: GET 2022-09-03T06:18:27.794 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:27.794 DEBUG:boto:Data: 2022-09-03T06:18:27.795 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.796 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.796 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.797 DEBUG:boto:Params: {} 2022-09-03T06:18:27.797 DEBUG:boto:Token: None 2022-09-03T06:18:27.798 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/ 2022-09-03T06:18:27.798 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM= 2022-09-03T06:18:27.799 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM=', 'Content-Length': '0'} 2022-09-03T06:18:27.804 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000814d80a1093be1b6-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.805 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:27.806 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:27.806 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:27.807 DEBUG:boto:Method: HEAD 2022-09-03T06:18:27.807 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:27.808 DEBUG:boto:Data: 2022-09-03T06:18:27.808 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.809 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.809 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.810 DEBUG:boto:Params: {} 2022-09-03T06:18:27.810 DEBUG:boto:Token: None 2022-09-03T06:18:27.811 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/e 2022-09-03T06:18:27.811 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IXhmypVgf7E1M4jwVy3UtxnpkZo= 2022-09-03T06:18:27.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IXhmypVgf7E1M4jwVy3UtxnpkZo=', 'Content-Length': '0'} 2022-09-03T06:18:27.814 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b32a8ff111f8b3a1-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.815 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:27.815 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:27.816 DEBUG:boto:Method: HEAD 2022-09-03T06:18:27.816 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:27.817 DEBUG:boto:Data: 2022-09-03T06:18:27.817 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.818 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.818 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.819 DEBUG:boto:Params: {} 2022-09-03T06:18:27.820 DEBUG:boto:Token: None 2022-09-03T06:18:27.820 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/e 2022-09-03T06:18:27.821 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IXhmypVgf7E1M4jwVy3UtxnpkZo= 2022-09-03T06:18:27.821 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IXhmypVgf7E1M4jwVy3UtxnpkZo=', 'Content-Length': '0'} 2022-09-03T06:18:27.824 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d84533056d1ffe6a-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.824 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:18:27.825 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:27.826 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:27.826 DEBUG:boto:Method: GET 2022-09-03T06:18:27.827 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:27.827 DEBUG:boto:Data: 2022-09-03T06:18:27.828 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.828 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.829 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.829 DEBUG:boto:Params: {} 2022-09-03T06:18:27.830 DEBUG:boto:Token: None 2022-09-03T06:18:27.831 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/e 2022-09-03T06:18:27.831 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K4w/yPCcbDIya2S7rY+xL+kFU2Q= 2022-09-03T06:18:27.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K4w/yPCcbDIya2S7rY+xL+kFU2Q=', 'Content-Length': '0'} 2022-09-03T06:18:27.834 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000947983e819a23ce6-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.836 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:27.837 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:27.838 DEBUG:boto:Method: GET 2022-09-03T06:18:27.839 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:27.840 DEBUG:boto:Data: 2022-09-03T06:18:27.841 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.842 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.842 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.843 DEBUG:boto:Params: {} 2022-09-03T06:18:27.845 DEBUG:boto:Token: None 2022-09-03T06:18:27.846 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/e 2022-09-03T06:18:27.847 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K4w/yPCcbDIya2S7rY+xL+kFU2Q= 2022-09-03T06:18:27.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K4w/yPCcbDIya2S7rY+xL+kFU2Q=', 'Content-Length': '0'} 2022-09-03T06:18:27.851 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c9dbd2a7d65ccc9-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.853 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:27.854 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:27.855 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:27.856 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:27.857 DEBUG:boto:Method: GET 2022-09-03T06:18:27.858 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:27.859 DEBUG:boto:Data: 2022-09-03T06:18:27.860 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.860 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.861 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.862 DEBUG:boto:Params: {} 2022-09-03T06:18:27.864 DEBUG:boto:Token: None 2022-09-03T06:18:27.865 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/ 2022-09-03T06:18:27.865 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM= 2022-09-03T06:18:27.866 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM=', 'Content-Length': '0'} 2022-09-03T06:18:27.868 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000094cd4d98120f9a1c-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.910 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:27.911 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:27.912 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:27.912 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:27.913 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:27.914 DEBUG:boto:Method: GET 2022-09-03T06:18:27.915 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:27.915 DEBUG:boto:Data: 2022-09-03T06:18:27.916 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.917 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.917 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.919 DEBUG:boto:Params: {} 2022-09-03T06:18:27.919 DEBUG:boto:Token: None 2022-09-03T06:18:27.920 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/ 2022-09-03T06:18:27.920 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM= 2022-09-03T06:18:27.926 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vT5W88SJ8/hsNIOjbvVgLejyCSM=', 'Content-Length': '0'} 2022-09-03T06:18:27.928 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b144e06cfcc36987-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.929 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:27.930 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:18:27.931 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:27.931 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:27.932 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:27.933 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:27.935 DEBUG:boto:Method: GET 2022-09-03T06:18:27.937 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:27.938 DEBUG:boto:Data: 2022-09-03T06:18:27.939 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.939 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.940 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.940 DEBUG:boto:Params: {} 2022-09-03T06:18:27.941 DEBUG:boto:Token: None 2022-09-03T06:18:27.942 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:27.942 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YxEIaXJRn1QOm/LYzyzAvvRpKlo= 2022-09-03T06:18:27.943 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YxEIaXJRn1QOm/LYzyzAvvRpKlo=', 'Content-Length': '0'} 2022-09-03T06:18:27.945 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000063c49d8873741189-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.946 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:27.947 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:27.947 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:27.948 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:27.948 DEBUG:boto:Method: GET 2022-09-03T06:18:27.949 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:27.949 DEBUG:boto:Data: 2022-09-03T06:18:27.950 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.951 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.951 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.952 DEBUG:boto:Params: {} 2022-09-03T06:18:27.952 DEBUG:boto:Token: None 2022-09-03T06:18:27.953 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:27.954 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YxEIaXJRn1QOm/LYzyzAvvRpKlo= 2022-09-03T06:18:27.955 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YxEIaXJRn1QOm/LYzyzAvvRpKlo=', 'Content-Length': '0'} 2022-09-03T06:18:27.957 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002b6c1f98616cdb6b-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.959 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:27.960 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:27.961 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:27.962 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:27.963 DEBUG:boto:Method: HEAD 2022-09-03T06:18:27.964 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:27.965 DEBUG:boto:Data: 2022-09-03T06:18:27.966 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.967 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.968 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.968 DEBUG:boto:Params: {} 2022-09-03T06:18:27.969 DEBUG:boto:Token: None 2022-09-03T06:18:27.970 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:27.970 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FlfLIlX2Bl0D8XqqRTun+a+CbTE= 2022-09-03T06:18:27.971 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FlfLIlX2Bl0D8XqqRTun+a+CbTE=', 'Content-Length': '0'} 2022-09-03T06:18:27.973 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000744e3bb3272b9a48-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.974 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:27.975 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:27.975 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:27.976 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:27.976 DEBUG:boto:Method: HEAD 2022-09-03T06:18:27.977 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:27.978 DEBUG:boto:Data: 2022-09-03T06:18:27.978 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.979 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:27.979 DEBUG:boto:Port: 8000 2022-09-03T06:18:27.980 DEBUG:boto:Params: {} 2022-09-03T06:18:27.980 DEBUG:boto:Token: None 2022-09-03T06:18:27.981 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:27.982 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FlfLIlX2Bl0D8XqqRTun+a+CbTE= 2022-09-03T06:18:27.982 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FlfLIlX2Bl0D8XqqRTun+a+CbTE=', 'Content-Length': '0'} 2022-09-03T06:18:27.984 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004c2275e7a44ce01b-006312f1b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.985 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:18:27.986 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:27.986 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:27.987 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:27.987 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:27.988 DEBUG:boto:Method: GET 2022-09-03T06:18:27.989 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:27.989 DEBUG:boto:Data: 2022-09-03T06:18:27.990 DEBUG:boto:Headers: {} 2022-09-03T06:18:27.990 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:27.991 DEBUG:boto:Port: 8001 2022-09-03T06:18:27.991 DEBUG:boto:Params: {} 2022-09-03T06:18:27.992 DEBUG:boto:Token: None 2022-09-03T06:18:27.992 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:27 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:27.993 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YxEIaXJRn1QOm/LYzyzAvvRpKlo= 2022-09-03T06:18:27.994 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YxEIaXJRn1QOm/LYzyzAvvRpKlo=', 'Content-Length': '0'} 2022-09-03T06:18:27.996 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004e7abfe47faebca5-006312f1b3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:27.997 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:27.997 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:27.998 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:27.999 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:27.999 DEBUG:boto:Method: GET 2022-09-03T06:18:28.000 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:28.000 DEBUG:boto:Data: 2022-09-03T06:18:28.001 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.001 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:28.002 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.002 DEBUG:boto:Params: {} 2022-09-03T06:18:28.003 DEBUG:boto:Token: None 2022-09-03T06:18:28.004 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:28.004 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ybgRf8dJOqn84xvoJNxUfwqHBKE= 2022-09-03T06:18:28.005 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ybgRf8dJOqn84xvoJNxUfwqHBKE=', 'Content-Length': '0'} 2022-09-03T06:18:28.007 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c3a89f3ad83c4f97-006312f1b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.009 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:28.009 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:28.010 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:28.011 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:28.012 DEBUG:boto:Method: GET 2022-09-03T06:18:28.013 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:28.014 DEBUG:boto:Data: 2022-09-03T06:18:28.015 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.016 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.017 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.017 DEBUG:boto:Params: {} 2022-09-03T06:18:28.018 DEBUG:boto:Token: None 2022-09-03T06:18:28.019 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/ 2022-09-03T06:18:28.019 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY= 2022-09-03T06:18:28.021 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY=', 'Content-Length': '0'} 2022-09-03T06:18:28.023 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000402987c4cc7c14b5-006312f1b4-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.025 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:28.026 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:28.027 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:28.028 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:28.029 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:28.030 DEBUG:boto:Method: GET 2022-09-03T06:18:28.031 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:28.031 DEBUG:boto:Data: 2022-09-03T06:18:28.032 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.032 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:28.033 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.033 DEBUG:boto:Params: {} 2022-09-03T06:18:28.034 DEBUG:boto:Token: None 2022-09-03T06:18:28.034 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/ 2022-09-03T06:18:28.035 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY= 2022-09-03T06:18:28.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY=', 'Content-Length': '0'} 2022-09-03T06:18:28.038 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ef46f9de7c64dd63-006312f1b4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.039 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:28.040 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:28.040 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:28.041 DEBUG:boto:Method: HEAD 2022-09-03T06:18:28.041 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:28.042 DEBUG:boto:Data: 2022-09-03T06:18:28.043 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.043 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.044 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.044 DEBUG:boto:Params: {} 2022-09-03T06:18:28.045 DEBUG:boto:Token: None 2022-09-03T06:18:28.046 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/f 2022-09-03T06:18:28.047 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zBTgL7dyNWbolRzRmZaMXvjnQYY= 2022-09-03T06:18:28.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zBTgL7dyNWbolRzRmZaMXvjnQYY=', 'Content-Length': '0'} 2022-09-03T06:18:28.051 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001255ed8c4565e492-006312f1b4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.052 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:28.052 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:28.053 DEBUG:boto:Method: HEAD 2022-09-03T06:18:28.053 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:28.054 DEBUG:boto:Data: 2022-09-03T06:18:28.055 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.055 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:28.056 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.057 DEBUG:boto:Params: {} 2022-09-03T06:18:28.058 DEBUG:boto:Token: None 2022-09-03T06:18:28.059 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/f 2022-09-03T06:18:28.060 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zBTgL7dyNWbolRzRmZaMXvjnQYY= 2022-09-03T06:18:28.061 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zBTgL7dyNWbolRzRmZaMXvjnQYY=', 'Content-Length': '0'} 2022-09-03T06:18:28.063 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cd877355392f38e0-006312f1b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.064 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:18:28.065 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:28.065 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:28.066 DEBUG:boto:Method: GET 2022-09-03T06:18:28.066 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:28.067 DEBUG:boto:Data: 2022-09-03T06:18:28.068 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.068 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.069 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.070 DEBUG:boto:Params: {} 2022-09-03T06:18:28.070 DEBUG:boto:Token: None 2022-09-03T06:18:28.071 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/f 2022-09-03T06:18:28.072 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PnXI/5+toTCqdpGHvD0CXNsILWI= 2022-09-03T06:18:28.072 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PnXI/5+toTCqdpGHvD0CXNsILWI=', 'Content-Length': '0'} 2022-09-03T06:18:28.075 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000077c900f27b9820b5-006312f1b4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.118 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:28.119 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:28.119 DEBUG:boto:Method: GET 2022-09-03T06:18:28.120 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:28.121 DEBUG:boto:Data: 2022-09-03T06:18:28.122 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.122 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:28.123 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.124 DEBUG:boto:Params: {} 2022-09-03T06:18:28.125 DEBUG:boto:Token: None 2022-09-03T06:18:28.126 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/f 2022-09-03T06:18:28.127 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PnXI/5+toTCqdpGHvD0CXNsILWI= 2022-09-03T06:18:28.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PnXI/5+toTCqdpGHvD0CXNsILWI=', 'Content-Length': '0'} 2022-09-03T06:18:28.131 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001fa78631e018c292-006312f1b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.133 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:28.134 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:28.135 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:28.136 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:28.137 DEBUG:boto:Method: GET 2022-09-03T06:18:28.138 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:28.139 DEBUG:boto:Data: 2022-09-03T06:18:28.140 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.141 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.142 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.143 DEBUG:boto:Params: {} 2022-09-03T06:18:28.143 DEBUG:boto:Token: None 2022-09-03T06:18:28.144 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/ 2022-09-03T06:18:28.145 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY= 2022-09-03T06:18:28.146 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY=', 'Content-Length': '0'} 2022-09-03T06:18:28.149 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009a06d3e204684edc-006312f1b4-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.149 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:28.150 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:28.151 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:28.151 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:28.152 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:28.152 DEBUG:boto:Method: GET 2022-09-03T06:18:28.153 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:28.153 DEBUG:boto:Data: 2022-09-03T06:18:28.154 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.154 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:28.155 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.155 DEBUG:boto:Params: {} 2022-09-03T06:18:28.156 DEBUG:boto:Token: None 2022-09-03T06:18:28.156 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/ 2022-09-03T06:18:28.157 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY= 2022-09-03T06:18:28.158 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY=', 'Content-Length': '0'} 2022-09-03T06:18:28.160 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b133db97c25f04e4-006312f1b4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.202 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:28.204 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:18:28.204 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:28.205 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:28.206 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.206 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.207 DEBUG:boto:Method: GET 2022-09-03T06:18:28.207 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.209 DEBUG:boto:Data: 2022-09-03T06:18:28.210 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.211 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.212 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.213 DEBUG:boto:Params: {} 2022-09-03T06:18:28.215 DEBUG:boto:Token: None 2022-09-03T06:18:28.216 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.217 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BpkeCV77hIJeWF6wduoiXzgGJPo= 2022-09-03T06:18:28.218 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BpkeCV77hIJeWF6wduoiXzgGJPo=', 'Content-Length': '0'} 2022-09-03T06:18:28.220 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e954cee15864b182-006312f1b4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.221 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:28.221 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:28.222 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.222 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.223 DEBUG:boto:Method: GET 2022-09-03T06:18:28.224 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.224 DEBUG:boto:Data: 2022-09-03T06:18:28.225 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.226 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:28.227 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.228 DEBUG:boto:Params: {} 2022-09-03T06:18:28.228 DEBUG:boto:Token: None 2022-09-03T06:18:28.229 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.230 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BpkeCV77hIJeWF6wduoiXzgGJPo= 2022-09-03T06:18:28.230 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BpkeCV77hIJeWF6wduoiXzgGJPo=', 'Content-Length': '0'} 2022-09-03T06:18:28.233 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000008737c4389f8c054-006312f1b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.234 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:28.235 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:28.237 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.237 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.238 DEBUG:boto:Method: HEAD 2022-09-03T06:18:28.239 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.240 DEBUG:boto:Data: 2022-09-03T06:18:28.241 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.242 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.243 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.243 DEBUG:boto:Params: {} 2022-09-03T06:18:28.244 DEBUG:boto:Token: None 2022-09-03T06:18:28.245 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.246 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pnxVpYK8u6B7R0J/nBvd1351rDs= 2022-09-03T06:18:28.247 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pnxVpYK8u6B7R0J/nBvd1351rDs=', 'Content-Length': '0'} 2022-09-03T06:18:28.250 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f4c691e45e1b221-006312f1b4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.251 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:28.253 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:28.254 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.255 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.255 DEBUG:boto:Method: HEAD 2022-09-03T06:18:28.256 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.257 DEBUG:boto:Data: 2022-09-03T06:18:28.258 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.260 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:28.261 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.262 DEBUG:boto:Params: {} 2022-09-03T06:18:28.263 DEBUG:boto:Token: None 2022-09-03T06:18:28.264 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.265 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pnxVpYK8u6B7R0J/nBvd1351rDs= 2022-09-03T06:18:28.265 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pnxVpYK8u6B7R0J/nBvd1351rDs=', 'Content-Length': '0'} 2022-09-03T06:18:28.267 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e14be532f65da45-006312f1b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.268 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:18:28.269 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:28.270 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:28.271 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.272 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.273 DEBUG:boto:Method: GET 2022-09-03T06:18:28.274 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.275 DEBUG:boto:Data: 2022-09-03T06:18:28.275 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.276 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.276 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.277 DEBUG:boto:Params: {} 2022-09-03T06:18:28.278 DEBUG:boto:Token: None 2022-09-03T06:18:28.278 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.279 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BpkeCV77hIJeWF6wduoiXzgGJPo= 2022-09-03T06:18:28.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BpkeCV77hIJeWF6wduoiXzgGJPo=', 'Content-Length': '0'} 2022-09-03T06:18:28.282 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e528e6ef2c02cb47-006312f1b4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.283 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:28.283 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:28.284 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.284 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.285 DEBUG:boto:Method: GET 2022-09-03T06:18:28.285 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.286 DEBUG:boto:Data: 2022-09-03T06:18:28.287 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.287 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:28.288 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.288 DEBUG:boto:Params: {} 2022-09-03T06:18:28.289 DEBUG:boto:Token: None 2022-09-03T06:18:28.290 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:28.290 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BpkeCV77hIJeWF6wduoiXzgGJPo= 2022-09-03T06:18:28.291 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BpkeCV77hIJeWF6wduoiXzgGJPo=', 'Content-Length': '0'} 2022-09-03T06:18:28.293 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000010b0f9d2e2ff82c3-006312f1b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.294 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:28.294 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:28.295 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:28.296 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:28.296 DEBUG:boto:Method: GET 2022-09-03T06:18:28.297 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:28.297 DEBUG:boto:Data: 2022-09-03T06:18:28.298 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.298 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.299 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.299 DEBUG:boto:Params: {} 2022-09-03T06:18:28.300 DEBUG:boto:Token: None 2022-09-03T06:18:28.301 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/ 2022-09-03T06:18:28.301 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY= 2022-09-03T06:18:28.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY=', 'Content-Length': '0'} 2022-09-03T06:18:28.305 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d76905dc025eb309-006312f1b4-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.306 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:28.307 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:28.307 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:28.308 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:28.309 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:28.309 DEBUG:boto:Method: GET 2022-09-03T06:18:28.310 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:28.310 DEBUG:boto:Data: 2022-09-03T06:18:28.311 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.312 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:28.312 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.313 DEBUG:boto:Params: {} 2022-09-03T06:18:28.313 DEBUG:boto:Token: None 2022-09-03T06:18:28.314 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/ 2022-09-03T06:18:28.315 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY= 2022-09-03T06:18:28.316 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY=', 'Content-Length': '0'} 2022-09-03T06:18:28.318 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005e0efe0c5c573c30-006312f1b4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.319 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:28.320 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:28.320 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:28.321 DEBUG:boto:Method: HEAD 2022-09-03T06:18:28.322 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:28.322 DEBUG:boto:Data: 2022-09-03T06:18:28.323 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.323 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.324 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.324 DEBUG:boto:Params: {} 2022-09-03T06:18:28.325 DEBUG:boto:Token: None 2022-09-03T06:18:28.326 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/g 2022-09-03T06:18:28.327 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yDCvbD4Dc9l5tu2qCNnlNgusGKM= 2022-09-03T06:18:28.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yDCvbD4Dc9l5tu2qCNnlNgusGKM=', 'Content-Length': '0'} 2022-09-03T06:18:28.330 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002eee3906780b82d0-006312f1b4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.331 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:28.332 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:28.332 DEBUG:boto:Method: HEAD 2022-09-03T06:18:28.333 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:28.334 DEBUG:boto:Data: 2022-09-03T06:18:28.334 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.335 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:28.335 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.336 DEBUG:boto:Params: {} 2022-09-03T06:18:28.337 DEBUG:boto:Token: None 2022-09-03T06:18:28.338 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/g 2022-09-03T06:18:28.339 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yDCvbD4Dc9l5tu2qCNnlNgusGKM= 2022-09-03T06:18:28.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yDCvbD4Dc9l5tu2qCNnlNgusGKM=', 'Content-Length': '0'} 2022-09-03T06:18:28.342 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000005c118a4f96dd63f-006312f1b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.343 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:18:28.343 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:28.344 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:28.344 DEBUG:boto:Method: GET 2022-09-03T06:18:28.345 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:28.346 DEBUG:boto:Data: 2022-09-03T06:18:28.347 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.348 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.348 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.349 DEBUG:boto:Params: {} 2022-09-03T06:18:28.350 DEBUG:boto:Token: None 2022-09-03T06:18:28.351 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/g 2022-09-03T06:18:28.352 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Kwp6WKuihAHBl3b0JVl+7DQvXpo= 2022-09-03T06:18:28.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Kwp6WKuihAHBl3b0JVl+7DQvXpo=', 'Content-Length': '0'} 2022-09-03T06:18:28.355 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000184085e4072aba6e-006312f1b4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.356 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:28.357 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:28.358 DEBUG:boto:Method: GET 2022-09-03T06:18:28.358 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:28.359 DEBUG:boto:Data: 2022-09-03T06:18:28.360 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.361 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:28.362 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.363 DEBUG:boto:Params: {} 2022-09-03T06:18:28.363 DEBUG:boto:Token: None 2022-09-03T06:18:28.364 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/g 2022-09-03T06:18:28.365 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Kwp6WKuihAHBl3b0JVl+7DQvXpo= 2022-09-03T06:18:28.366 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Kwp6WKuihAHBl3b0JVl+7DQvXpo=', 'Content-Length': '0'} 2022-09-03T06:18:28.369 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e9b2d53e90d4616a-006312f1b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.370 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:28.371 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:28.372 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:28.373 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:28.373 DEBUG:boto:Method: GET 2022-09-03T06:18:28.374 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:28.375 DEBUG:boto:Data: 2022-09-03T06:18:28.375 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.376 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.376 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.377 DEBUG:boto:Params: {} 2022-09-03T06:18:28.378 DEBUG:boto:Token: None 2022-09-03T06:18:28.378 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/ 2022-09-03T06:18:28.379 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY= 2022-09-03T06:18:28.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY=', 'Content-Length': '0'} 2022-09-03T06:18:28.382 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000383db25ed41ce972-006312f1b4-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.430 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:28.432 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:28.433 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:28.434 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:28.434 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:28.435 DEBUG:boto:Method: GET 2022-09-03T06:18:28.436 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:28.437 DEBUG:boto:Data: 2022-09-03T06:18:28.437 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.438 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:28.438 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.439 DEBUG:boto:Params: {} 2022-09-03T06:18:28.440 DEBUG:boto:Token: None 2022-09-03T06:18:28.441 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/ 2022-09-03T06:18:28.441 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY= 2022-09-03T06:18:28.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY=', 'Content-Length': '0'} 2022-09-03T06:18:28.444 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c6a72ad0773e6e06-006312f1b4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.445 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:28.446 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:18:28.447 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:28.448 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:28.448 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.449 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.450 DEBUG:boto:Method: GET 2022-09-03T06:18:28.451 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.452 DEBUG:boto:Data: 2022-09-03T06:18:28.453 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.454 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.455 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.456 DEBUG:boto:Params: {} 2022-09-03T06:18:28.457 DEBUG:boto:Token: None 2022-09-03T06:18:28.458 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.459 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iykDKcc7IeeN0KS6qwoFEdOiO+M= 2022-09-03T06:18:28.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iykDKcc7IeeN0KS6qwoFEdOiO+M=', 'Content-Length': '0'} 2022-09-03T06:18:28.463 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000030e6ed4d4e2ff65c-006312f1b4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.465 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:28.466 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:28.467 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.468 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.469 DEBUG:boto:Method: GET 2022-09-03T06:18:28.469 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.470 DEBUG:boto:Data: 2022-09-03T06:18:28.470 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.471 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:28.471 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.472 DEBUG:boto:Params: {} 2022-09-03T06:18:28.473 DEBUG:boto:Token: None 2022-09-03T06:18:28.474 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.474 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iykDKcc7IeeN0KS6qwoFEdOiO+M= 2022-09-03T06:18:28.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iykDKcc7IeeN0KS6qwoFEdOiO+M=', 'Content-Length': '0'} 2022-09-03T06:18:28.479 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d0ee1e803e73f971-006312f1b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.480 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:28.481 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:28.482 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.483 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.484 DEBUG:boto:Method: HEAD 2022-09-03T06:18:28.485 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.486 DEBUG:boto:Data: 2022-09-03T06:18:28.487 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.488 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.489 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.489 DEBUG:boto:Params: {} 2022-09-03T06:18:28.490 DEBUG:boto:Token: None 2022-09-03T06:18:28.491 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.492 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v93DGmI3mWZYVBKBPOwDqzINMNQ= 2022-09-03T06:18:28.492 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v93DGmI3mWZYVBKBPOwDqzINMNQ=', 'Content-Length': '0'} 2022-09-03T06:18:28.495 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000059556a53881ad8dd-006312f1b4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.496 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:28.497 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:28.498 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.499 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.500 DEBUG:boto:Method: HEAD 2022-09-03T06:18:28.500 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.501 DEBUG:boto:Data: 2022-09-03T06:18:28.502 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.502 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:28.503 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.503 DEBUG:boto:Params: {} 2022-09-03T06:18:28.504 DEBUG:boto:Token: None 2022-09-03T06:18:28.505 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.505 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v93DGmI3mWZYVBKBPOwDqzINMNQ= 2022-09-03T06:18:28.506 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v93DGmI3mWZYVBKBPOwDqzINMNQ=', 'Content-Length': '0'} 2022-09-03T06:18:28.509 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f063ffd128d3ae20-006312f1b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.511 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:18:28.512 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:28.513 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:28.514 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.516 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.517 DEBUG:boto:Method: GET 2022-09-03T06:18:28.518 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.519 DEBUG:boto:Data: 2022-09-03T06:18:28.520 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.521 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.521 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.522 DEBUG:boto:Params: {} 2022-09-03T06:18:28.523 DEBUG:boto:Token: None 2022-09-03T06:18:28.524 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.525 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iykDKcc7IeeN0KS6qwoFEdOiO+M= 2022-09-03T06:18:28.526 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iykDKcc7IeeN0KS6qwoFEdOiO+M=', 'Content-Length': '0'} 2022-09-03T06:18:28.529 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a4b4258d4b4cb7e3-006312f1b4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.570 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:28.571 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:28.573 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.573 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.574 DEBUG:boto:Method: GET 2022-09-03T06:18:28.575 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.575 DEBUG:boto:Data: 2022-09-03T06:18:28.576 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.577 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:28.577 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.578 DEBUG:boto:Params: {} 2022-09-03T06:18:28.578 DEBUG:boto:Token: None 2022-09-03T06:18:28.579 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:28.580 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iykDKcc7IeeN0KS6qwoFEdOiO+M= 2022-09-03T06:18:28.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iykDKcc7IeeN0KS6qwoFEdOiO+M=', 'Content-Length': '0'} 2022-09-03T06:18:28.583 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004ddd132622b55172-006312f1b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.584 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:28.585 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:28.585 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:28.586 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:28.587 DEBUG:boto:Method: GET 2022-09-03T06:18:28.587 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:28.588 DEBUG:boto:Data: 2022-09-03T06:18:28.588 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.589 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.590 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.590 DEBUG:boto:Params: {} 2022-09-03T06:18:28.591 DEBUG:boto:Token: None 2022-09-03T06:18:28.591 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/ 2022-09-03T06:18:28.592 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY= 2022-09-03T06:18:28.593 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY=', 'Content-Length': '0'} 2022-09-03T06:18:28.601 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003ce0db202921baa3-006312f1b4-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.602 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:28.603 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:28.604 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:28.604 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:28.605 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:28.605 DEBUG:boto:Method: GET 2022-09-03T06:18:28.606 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:28.607 DEBUG:boto:Data: 2022-09-03T06:18:28.607 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.608 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:28.608 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.609 DEBUG:boto:Params: {} 2022-09-03T06:18:28.609 DEBUG:boto:Token: None 2022-09-03T06:18:28.610 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/ 2022-09-03T06:18:28.611 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY= 2022-09-03T06:18:28.611 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY=', 'Content-Length': '0'} 2022-09-03T06:18:28.613 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002cb981cddb0ca0b8-006312f1b4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.614 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:28.615 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:28.616 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:28.616 DEBUG:boto:Method: HEAD 2022-09-03T06:18:28.617 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:28.618 DEBUG:boto:Data: 2022-09-03T06:18:28.618 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.619 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.619 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.620 DEBUG:boto:Params: {} 2022-09-03T06:18:28.620 DEBUG:boto:Token: None 2022-09-03T06:18:28.621 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/h 2022-09-03T06:18:28.621 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:U0V4Osh0kCGQw1dLaf2/1JT28Pw= 2022-09-03T06:18:28.622 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:U0V4Osh0kCGQw1dLaf2/1JT28Pw=', 'Content-Length': '0'} 2022-09-03T06:18:28.630 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c422157722d97311-006312f1b4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.631 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:28.632 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:28.633 DEBUG:boto:Method: HEAD 2022-09-03T06:18:28.633 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:28.634 DEBUG:boto:Data: 2022-09-03T06:18:28.634 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.635 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:28.635 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.636 DEBUG:boto:Params: {} 2022-09-03T06:18:28.636 DEBUG:boto:Token: None 2022-09-03T06:18:28.637 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/h 2022-09-03T06:18:28.638 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:U0V4Osh0kCGQw1dLaf2/1JT28Pw= 2022-09-03T06:18:28.638 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:U0V4Osh0kCGQw1dLaf2/1JT28Pw=', 'Content-Length': '0'} 2022-09-03T06:18:28.640 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b9e7d8a160e8784-006312f1b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.641 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:18:28.642 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:28.643 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:28.643 DEBUG:boto:Method: GET 2022-09-03T06:18:28.644 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:28.644 DEBUG:boto:Data: 2022-09-03T06:18:28.645 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.645 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.646 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.646 DEBUG:boto:Params: {} 2022-09-03T06:18:28.647 DEBUG:boto:Token: None 2022-09-03T06:18:28.648 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/h 2022-09-03T06:18:28.648 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XUlXQJVjavXzlM+kKl5/1EqCrVo= 2022-09-03T06:18:28.649 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XUlXQJVjavXzlM+kKl5/1EqCrVo=', 'Content-Length': '0'} 2022-09-03T06:18:28.653 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002954e82f655c19ec-006312f1b4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.701 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:28.702 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:28.703 DEBUG:boto:Method: GET 2022-09-03T06:18:28.703 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:28.704 DEBUG:boto:Data: 2022-09-03T06:18:28.704 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.705 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:28.706 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.706 DEBUG:boto:Params: {} 2022-09-03T06:18:28.707 DEBUG:boto:Token: None 2022-09-03T06:18:28.708 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/h 2022-09-03T06:18:28.708 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XUlXQJVjavXzlM+kKl5/1EqCrVo= 2022-09-03T06:18:28.709 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XUlXQJVjavXzlM+kKl5/1EqCrVo=', 'Content-Length': '0'} 2022-09-03T06:18:28.711 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ceff9c27a2cebebb-006312f1b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.712 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:28.712 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:28.713 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:28.713 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:28.714 DEBUG:boto:Method: GET 2022-09-03T06:18:28.715 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:28.715 DEBUG:boto:Data: 2022-09-03T06:18:28.716 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.716 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.717 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.717 DEBUG:boto:Params: {} 2022-09-03T06:18:28.718 DEBUG:boto:Token: None 2022-09-03T06:18:28.718 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/ 2022-09-03T06:18:28.719 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY= 2022-09-03T06:18:28.720 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY=', 'Content-Length': '0'} 2022-09-03T06:18:28.722 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000be201368df37278f-006312f1b4-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.723 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:28.724 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:28.725 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:28.725 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:28.726 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:28.726 DEBUG:boto:Method: GET 2022-09-03T06:18:28.727 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:28.727 DEBUG:boto:Data: 2022-09-03T06:18:28.728 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.728 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:28.729 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.729 DEBUG:boto:Params: {} 2022-09-03T06:18:28.730 DEBUG:boto:Token: None 2022-09-03T06:18:28.730 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/ 2022-09-03T06:18:28.731 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY= 2022-09-03T06:18:28.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1zerWVZuewUWCOL994l87mcEvOY=', 'Content-Length': '0'} 2022-09-03T06:18:28.734 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d5872af764127d8e-006312f1b4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.782 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:28.783 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-71 zones={test-zone3, test-zone1} 2022-09-03T06:18:28.784 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-71 zones={test-zone3, test-zone2} 2022-09-03T06:18:28.784 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:28.785 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:28.786 DEBUG:boto:Method: HEAD 2022-09-03T06:18:28.786 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:18:28.787 DEBUG:boto:Data: 2022-09-03T06:18:28.788 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.788 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.789 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.790 DEBUG:boto:Params: {} 2022-09-03T06:18:28.791 DEBUG:boto:Token: None 2022-09-03T06:18:28.791 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/ 2022-09-03T06:18:28.792 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rwH67GPUyTls6lxVOuuNpBeaqxY= 2022-09-03T06:18:28.793 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rwH67GPUyTls6lxVOuuNpBeaqxY=', 'Content-Length': '0'} 2022-09-03T06:18:28.795 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000279d389767688e39-006312f1b4-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.796 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:28.797 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:28.797 DEBUG:boto:Method: HEAD 2022-09-03T06:18:28.798 DEBUG:boto:Path: /dsgvcy-71/ 2022-09-03T06:18:28.798 DEBUG:boto:Data: 2022-09-03T06:18:28.799 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.800 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:28.800 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.801 DEBUG:boto:Params: {} 2022-09-03T06:18:28.802 DEBUG:boto:Token: None 2022-09-03T06:18:28.802 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/ 2022-09-03T06:18:28.803 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rwH67GPUyTls6lxVOuuNpBeaqxY= 2022-09-03T06:18:28.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rwH67GPUyTls6lxVOuuNpBeaqxY=', 'Content-Length': '0'} 2022-09-03T06:18:28.807 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004b180d2083acc084-006312f1b4-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.808 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:18:28.808 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:28.809 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:28.809 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:28.810 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:28.811 DEBUG:boto:Method: GET 2022-09-03T06:18:28.811 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:28.812 DEBUG:boto:Data: 2022-09-03T06:18:28.812 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.813 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.813 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.814 DEBUG:boto:Params: {} 2022-09-03T06:18:28.815 DEBUG:boto:Token: None 2022-09-03T06:18:28.816 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/?versions 2022-09-03T06:18:28.816 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1JcR6HOEhGLFiYvot0P6Ow/SDRs= 2022-09-03T06:18:28.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1JcR6HOEhGLFiYvot0P6Ow/SDRs=', 'Content-Length': '0'} 2022-09-03T06:18:28.822 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000076b1b4dc3eecdf3b-006312f1b4-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.823 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:28.824 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:18:28.824 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:18:28.825 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:18:28.826 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:18:28.826 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:18:28.827 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:18:28.827 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:18:28.828 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:18:28.829 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:18:28.830 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:28.830 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:28.831 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:28.831 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:28.832 DEBUG:boto:Method: GET 2022-09-03T06:18:28.832 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:28.833 DEBUG:boto:Data: 2022-09-03T06:18:28.834 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.834 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:28.835 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.835 DEBUG:boto:Params: {} 2022-09-03T06:18:28.836 DEBUG:boto:Token: None 2022-09-03T06:18:28.837 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/?versions 2022-09-03T06:18:28.837 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1JcR6HOEhGLFiYvot0P6Ow/SDRs= 2022-09-03T06:18:28.838 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1JcR6HOEhGLFiYvot0P6Ow/SDRs=', 'Content-Length': '0'} 2022-09-03T06:18:28.841 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cc44fb9ed8b67afa-006312f1b4-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.842 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:28.843 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:18:28.844 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:18:28.844 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:18:28.845 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:18:28.846 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:18:28.847 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:18:28.848 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:18:28.849 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:18:28.850 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:28.851 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:28.852 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:28.853 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:28.855 DEBUG:boto:Method: GET 2022-09-03T06:18:28.855 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:28.856 DEBUG:boto:Data: 2022-09-03T06:18:28.856 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.857 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.858 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.858 DEBUG:boto:Params: {} 2022-09-03T06:18:28.859 DEBUG:boto:Token: None 2022-09-03T06:18:28.860 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/?versions 2022-09-03T06:18:28.860 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1JcR6HOEhGLFiYvot0P6Ow/SDRs= 2022-09-03T06:18:28.861 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1JcR6HOEhGLFiYvot0P6Ow/SDRs=', 'Content-Length': '0'} 2022-09-03T06:18:28.864 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000709bd74ecc3b6596-006312f1b4-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.865 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:28.867 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:28.867 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:28.868 DEBUG:boto:path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:28.868 DEBUG:boto:auth_path=/dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:28.869 DEBUG:boto:Method: GET 2022-09-03T06:18:28.870 DEBUG:boto:Path: /dsgvcy-71/?versions&max-keys=999 2022-09-03T06:18:28.870 DEBUG:boto:Data: 2022-09-03T06:18:28.871 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.871 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:28.872 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.873 DEBUG:boto:Params: {} 2022-09-03T06:18:28.873 DEBUG:boto:Token: None 2022-09-03T06:18:28.874 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/?versions 2022-09-03T06:18:28.875 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1JcR6HOEhGLFiYvot0P6Ow/SDRs= 2022-09-03T06:18:28.876 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1JcR6HOEhGLFiYvot0P6Ow/SDRs=', 'Content-Length': '0'} 2022-09-03T06:18:28.879 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ed7222400a886c31-006312f1b4-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.880 DEBUG:boto:b'dsgvcy-71999falseanulltrue2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:28.882 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:18:28.883 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:28.884 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:28.885 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:28.886 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:28.887 DEBUG:boto:Method: GET 2022-09-03T06:18:28.888 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:28.889 DEBUG:boto:Data: 2022-09-03T06:18:28.890 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.891 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.892 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.893 DEBUG:boto:Params: {} 2022-09-03T06:18:28.894 DEBUG:boto:Token: None 2022-09-03T06:18:28.895 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:28.896 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qJcqxTTqKuWeL7uMB1WvOI8OrFE= 2022-09-03T06:18:28.897 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qJcqxTTqKuWeL7uMB1WvOI8OrFE=', 'Content-Length': '0'} 2022-09-03T06:18:28.901 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e23d546cc7f1a70f-006312f1b4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.902 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:28.902 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:28.903 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:28.904 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:28.904 DEBUG:boto:Method: GET 2022-09-03T06:18:28.905 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:28.905 DEBUG:boto:Data: 2022-09-03T06:18:28.906 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.906 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:28.907 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.907 DEBUG:boto:Params: {} 2022-09-03T06:18:28.908 DEBUG:boto:Token: None 2022-09-03T06:18:28.909 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:28.910 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qJcqxTTqKuWeL7uMB1WvOI8OrFE= 2022-09-03T06:18:28.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qJcqxTTqKuWeL7uMB1WvOI8OrFE=', 'Content-Length': '0'} 2022-09-03T06:18:28.913 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ed1cc03440adf061-006312f1b4-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.954 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:28.955 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:28.956 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:28.956 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:28.957 DEBUG:boto:Method: HEAD 2022-09-03T06:18:28.957 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:28.958 DEBUG:boto:Data: 2022-09-03T06:18:28.958 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.959 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.959 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.960 DEBUG:boto:Params: {} 2022-09-03T06:18:28.961 DEBUG:boto:Token: None 2022-09-03T06:18:28.961 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:28.962 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vWW7M1WpW2FiP591DjzPBHaYYA4= 2022-09-03T06:18:28.962 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vWW7M1WpW2FiP591DjzPBHaYYA4=', 'Content-Length': '0'} 2022-09-03T06:18:28.965 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002e1c7b476751ebc3-006312f1b4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.966 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:28.967 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:28.967 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:28.968 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:28.968 DEBUG:boto:Method: HEAD 2022-09-03T06:18:28.969 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:28.969 DEBUG:boto:Data: 2022-09-03T06:18:28.970 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.970 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:28.971 DEBUG:boto:Port: 8000 2022-09-03T06:18:28.971 DEBUG:boto:Params: {} 2022-09-03T06:18:28.972 DEBUG:boto:Token: None 2022-09-03T06:18:28.973 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:28.974 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vWW7M1WpW2FiP591DjzPBHaYYA4= 2022-09-03T06:18:28.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vWW7M1WpW2FiP591DjzPBHaYYA4=', 'Content-Length': '0'} 2022-09-03T06:18:28.978 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003d33b2dca5798cf5-006312f1b4-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:28.979 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:18:28.980 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:28.980 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:28.981 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:28.981 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:28.982 DEBUG:boto:Method: GET 2022-09-03T06:18:28.982 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:28.983 DEBUG:boto:Data: 2022-09-03T06:18:28.983 DEBUG:boto:Headers: {} 2022-09-03T06:18:28.984 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:28.985 DEBUG:boto:Port: 8001 2022-09-03T06:18:28.992 DEBUG:boto:Params: {} 2022-09-03T06:18:28.996 DEBUG:boto:Token: None 2022-09-03T06:18:28.997 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:28 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:28.998 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qJcqxTTqKuWeL7uMB1WvOI8OrFE= 2022-09-03T06:18:29.000 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qJcqxTTqKuWeL7uMB1WvOI8OrFE=', 'Content-Length': '0'} 2022-09-03T06:18:29.003 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001f97525510dfef45-006312f1b4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.005 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:29.006 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:29.006 DEBUG:boto:path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:29.007 DEBUG:boto:auth_path=/dsgvcy-71/a?versionId=null 2022-09-03T06:18:29.007 DEBUG:boto:Method: GET 2022-09-03T06:18:29.008 DEBUG:boto:Path: /dsgvcy-71/a?versionId=null 2022-09-03T06:18:29.009 DEBUG:boto:Data: 2022-09-03T06:18:29.009 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.010 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.010 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.011 DEBUG:boto:Params: {} 2022-09-03T06:18:29.011 DEBUG:boto:Token: None 2022-09-03T06:18:29.012 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/a?versionId=null 2022-09-03T06:18:29.013 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/l//QGK+FjOr6egNCc63QZjUarE= 2022-09-03T06:18:29.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/l//QGK+FjOr6egNCc63QZjUarE=', 'Content-Length': '0'} 2022-09-03T06:18:29.016 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cad8cc2b132865fc-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.017 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:29.017 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:29.018 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:29.018 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:29.019 DEBUG:boto:Method: GET 2022-09-03T06:18:29.019 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:29.020 DEBUG:boto:Data: 2022-09-03T06:18:29.021 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.021 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.022 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.023 DEBUG:boto:Params: {} 2022-09-03T06:18:29.023 DEBUG:boto:Token: None 2022-09-03T06:18:29.024 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/ 2022-09-03T06:18:29.025 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M= 2022-09-03T06:18:29.026 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M=', 'Content-Length': '0'} 2022-09-03T06:18:29.029 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000038b9b16e6b7992ec-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.074 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:29.076 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:29.077 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:29.077 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:29.078 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:29.079 DEBUG:boto:Method: GET 2022-09-03T06:18:29.079 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:29.081 DEBUG:boto:Data: 2022-09-03T06:18:29.082 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.082 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.083 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.083 DEBUG:boto:Params: {} 2022-09-03T06:18:29.084 DEBUG:boto:Token: None 2022-09-03T06:18:29.085 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/ 2022-09-03T06:18:29.086 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M= 2022-09-03T06:18:29.087 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M=', 'Content-Length': '0'} 2022-09-03T06:18:29.090 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bed2e5ce94922bc3-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.091 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:29.093 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:29.094 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:29.095 DEBUG:boto:Method: HEAD 2022-09-03T06:18:29.096 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:29.097 DEBUG:boto:Data: 2022-09-03T06:18:29.098 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.099 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.100 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.100 DEBUG:boto:Params: {} 2022-09-03T06:18:29.101 DEBUG:boto:Token: None 2022-09-03T06:18:29.102 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/a 2022-09-03T06:18:29.102 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lAYRfXYVTwqLgImd2erl9w9gUIU= 2022-09-03T06:18:29.103 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lAYRfXYVTwqLgImd2erl9w9gUIU=', 'Content-Length': '0'} 2022-09-03T06:18:29.105 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007ad6b721e1c91c12-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.106 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:29.107 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:29.107 DEBUG:boto:Method: HEAD 2022-09-03T06:18:29.108 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:29.108 DEBUG:boto:Data: 2022-09-03T06:18:29.109 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.109 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.110 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.110 DEBUG:boto:Params: {} 2022-09-03T06:18:29.111 DEBUG:boto:Token: None 2022-09-03T06:18:29.112 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/a 2022-09-03T06:18:29.112 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lAYRfXYVTwqLgImd2erl9w9gUIU= 2022-09-03T06:18:29.113 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lAYRfXYVTwqLgImd2erl9w9gUIU=', 'Content-Length': '0'} 2022-09-03T06:18:29.115 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e622cf8e5649673d-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.116 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:18:29.116 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:29.117 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:29.117 DEBUG:boto:Method: GET 2022-09-03T06:18:29.118 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:29.118 DEBUG:boto:Data: 2022-09-03T06:18:29.119 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.119 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.120 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.120 DEBUG:boto:Params: {} 2022-09-03T06:18:29.121 DEBUG:boto:Token: None 2022-09-03T06:18:29.122 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/a 2022-09-03T06:18:29.122 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:au2Lp4jWLuzat3waFurUEuW1+pg= 2022-09-03T06:18:29.123 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:au2Lp4jWLuzat3waFurUEuW1+pg=', 'Content-Length': '0'} 2022-09-03T06:18:29.125 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f3f1554b826c16ae-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.126 DEBUG:boto:path=/dsgvcy-71/a 2022-09-03T06:18:29.127 DEBUG:boto:auth_path=/dsgvcy-71/a 2022-09-03T06:18:29.127 DEBUG:boto:Method: GET 2022-09-03T06:18:29.128 DEBUG:boto:Path: /dsgvcy-71/a 2022-09-03T06:18:29.128 DEBUG:boto:Data: 2022-09-03T06:18:29.129 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.129 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.130 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.130 DEBUG:boto:Params: {} 2022-09-03T06:18:29.131 DEBUG:boto:Token: None 2022-09-03T06:18:29.132 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/a 2022-09-03T06:18:29.132 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:au2Lp4jWLuzat3waFurUEuW1+pg= 2022-09-03T06:18:29.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:au2Lp4jWLuzat3waFurUEuW1+pg=', 'Content-Length': '0'} 2022-09-03T06:18:29.135 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000009e9c5dcfdd394b3-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.182 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:29.184 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:29.185 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:29.186 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:29.187 DEBUG:boto:Method: GET 2022-09-03T06:18:29.188 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:29.189 DEBUG:boto:Data: 2022-09-03T06:18:29.190 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.191 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.192 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.193 DEBUG:boto:Params: {} 2022-09-03T06:18:29.195 DEBUG:boto:Token: None 2022-09-03T06:18:29.196 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/ 2022-09-03T06:18:29.197 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M= 2022-09-03T06:18:29.198 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M=', 'Content-Length': '0'} 2022-09-03T06:18:29.201 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008bd28f3d2a2bf998-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.203 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:29.204 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:29.204 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:29.205 DEBUG:boto:path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:29.205 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=a 2022-09-03T06:18:29.206 DEBUG:boto:Method: GET 2022-09-03T06:18:29.206 DEBUG:boto:Path: /dsgvcy-71/?prefix=a 2022-09-03T06:18:29.207 DEBUG:boto:Data: 2022-09-03T06:18:29.207 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.208 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.208 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.209 DEBUG:boto:Params: {} 2022-09-03T06:18:29.210 DEBUG:boto:Token: None 2022-09-03T06:18:29.211 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/ 2022-09-03T06:18:29.211 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M= 2022-09-03T06:18:29.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M=', 'Content-Length': '0'} 2022-09-03T06:18:29.215 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004bf8839a212f2d09-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.216 DEBUG:boto:b'dsgvcy-71a1000falsea2022-09-03T06:12:47.277Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:29.216 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:18:29.217 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:29.217 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:29.218 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.219 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.219 DEBUG:boto:Method: GET 2022-09-03T06:18:29.220 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.220 DEBUG:boto:Data: 2022-09-03T06:18:29.221 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.221 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.222 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.223 DEBUG:boto:Params: {} 2022-09-03T06:18:29.223 DEBUG:boto:Token: None 2022-09-03T06:18:29.224 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.224 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NWBIKeFZHq9R4xQyXzcjZYXD5Rs= 2022-09-03T06:18:29.225 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NWBIKeFZHq9R4xQyXzcjZYXD5Rs=', 'Content-Length': '0'} 2022-09-03T06:18:29.227 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002ef10aadea24d3be-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.228 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:29.229 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:29.230 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.230 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.231 DEBUG:boto:Method: GET 2022-09-03T06:18:29.232 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.232 DEBUG:boto:Data: 2022-09-03T06:18:29.233 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.233 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.234 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.235 DEBUG:boto:Params: {} 2022-09-03T06:18:29.235 DEBUG:boto:Token: None 2022-09-03T06:18:29.236 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.236 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NWBIKeFZHq9R4xQyXzcjZYXD5Rs= 2022-09-03T06:18:29.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NWBIKeFZHq9R4xQyXzcjZYXD5Rs=', 'Content-Length': '0'} 2022-09-03T06:18:29.242 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cabbdfbdbdd005b7-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.243 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:29.243 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:29.244 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.244 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.245 DEBUG:boto:Method: HEAD 2022-09-03T06:18:29.245 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.246 DEBUG:boto:Data: 2022-09-03T06:18:29.247 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.247 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.248 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.248 DEBUG:boto:Params: {} 2022-09-03T06:18:29.249 DEBUG:boto:Token: None 2022-09-03T06:18:29.249 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.250 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:d+kq9IiNoxQ2NFPBFpfGtftoz08= 2022-09-03T06:18:29.251 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:d+kq9IiNoxQ2NFPBFpfGtftoz08=', 'Content-Length': '0'} 2022-09-03T06:18:29.253 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000056b15ce651d02da6-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.253 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:29.254 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:29.255 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.255 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.256 DEBUG:boto:Method: HEAD 2022-09-03T06:18:29.256 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.257 DEBUG:boto:Data: 2022-09-03T06:18:29.257 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.258 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.258 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.259 DEBUG:boto:Params: {} 2022-09-03T06:18:29.259 DEBUG:boto:Token: None 2022-09-03T06:18:29.260 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.261 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:d+kq9IiNoxQ2NFPBFpfGtftoz08= 2022-09-03T06:18:29.261 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:d+kq9IiNoxQ2NFPBFpfGtftoz08=', 'Content-Length': '0'} 2022-09-03T06:18:29.263 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000304496da8d061f77-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.264 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:18:29.264 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:29.265 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:29.265 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.266 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.266 DEBUG:boto:Method: GET 2022-09-03T06:18:29.267 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.268 DEBUG:boto:Data: 2022-09-03T06:18:29.268 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.269 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.269 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.270 DEBUG:boto:Params: {} 2022-09-03T06:18:29.270 DEBUG:boto:Token: None 2022-09-03T06:18:29.271 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.271 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NWBIKeFZHq9R4xQyXzcjZYXD5Rs= 2022-09-03T06:18:29.272 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NWBIKeFZHq9R4xQyXzcjZYXD5Rs=', 'Content-Length': '0'} 2022-09-03T06:18:29.275 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000090a5718278eaf722-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.276 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:29.277 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:29.277 DEBUG:boto:path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.278 DEBUG:boto:auth_path=/dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.278 DEBUG:boto:Method: GET 2022-09-03T06:18:29.279 DEBUG:boto:Path: /dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.279 DEBUG:boto:Data: 2022-09-03T06:18:29.280 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.281 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.282 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.283 DEBUG:boto:Params: {} 2022-09-03T06:18:29.285 DEBUG:boto:Token: None 2022-09-03T06:18:29.286 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/b?versionId=null 2022-09-03T06:18:29.287 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NWBIKeFZHq9R4xQyXzcjZYXD5Rs= 2022-09-03T06:18:29.289 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NWBIKeFZHq9R4xQyXzcjZYXD5Rs=', 'Content-Length': '0'} 2022-09-03T06:18:29.292 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000364efb639d795043-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.334 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:29.336 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:29.337 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:29.338 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:29.339 DEBUG:boto:Method: GET 2022-09-03T06:18:29.340 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:29.341 DEBUG:boto:Data: 2022-09-03T06:18:29.342 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.343 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.344 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.345 DEBUG:boto:Params: {} 2022-09-03T06:18:29.347 DEBUG:boto:Token: None 2022-09-03T06:18:29.347 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/ 2022-09-03T06:18:29.348 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M= 2022-09-03T06:18:29.349 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M=', 'Content-Length': '0'} 2022-09-03T06:18:29.352 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009fe16fb4935658dc-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.352 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:29.353 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:29.354 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:29.354 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:29.355 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:29.356 DEBUG:boto:Method: GET 2022-09-03T06:18:29.356 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:29.357 DEBUG:boto:Data: 2022-09-03T06:18:29.357 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.358 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.358 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.359 DEBUG:boto:Params: {} 2022-09-03T06:18:29.360 DEBUG:boto:Token: None 2022-09-03T06:18:29.360 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/ 2022-09-03T06:18:29.361 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M= 2022-09-03T06:18:29.362 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M=', 'Content-Length': '0'} 2022-09-03T06:18:29.364 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000884a2c4e5a1f0fe2-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.365 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:29.367 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:29.367 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:29.368 DEBUG:boto:Method: HEAD 2022-09-03T06:18:29.369 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:29.369 DEBUG:boto:Data: 2022-09-03T06:18:29.370 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.371 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.371 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.372 DEBUG:boto:Params: {} 2022-09-03T06:18:29.372 DEBUG:boto:Token: None 2022-09-03T06:18:29.373 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/b 2022-09-03T06:18:29.373 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0402CFkK9HZLH1guXykSfPLkBaY= 2022-09-03T06:18:29.374 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0402CFkK9HZLH1guXykSfPLkBaY=', 'Content-Length': '0'} 2022-09-03T06:18:29.376 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000fd8e5d1924cf4f6-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.377 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:29.378 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:29.378 DEBUG:boto:Method: HEAD 2022-09-03T06:18:29.379 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:29.379 DEBUG:boto:Data: 2022-09-03T06:18:29.380 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.380 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.381 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.381 DEBUG:boto:Params: {} 2022-09-03T06:18:29.382 DEBUG:boto:Token: None 2022-09-03T06:18:29.383 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/b 2022-09-03T06:18:29.384 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0402CFkK9HZLH1guXykSfPLkBaY= 2022-09-03T06:18:29.385 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0402CFkK9HZLH1guXykSfPLkBaY=', 'Content-Length': '0'} 2022-09-03T06:18:29.387 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000080bedc57e1eec764-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.387 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:18:29.388 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:29.389 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:29.390 DEBUG:boto:Method: GET 2022-09-03T06:18:29.391 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:29.392 DEBUG:boto:Data: 2022-09-03T06:18:29.392 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.393 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.394 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.395 DEBUG:boto:Params: {} 2022-09-03T06:18:29.397 DEBUG:boto:Token: None 2022-09-03T06:18:29.398 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/b 2022-09-03T06:18:29.399 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rNaUXSeopKQe9en3fbP/opc979E= 2022-09-03T06:18:29.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rNaUXSeopKQe9en3fbP/opc979E=', 'Content-Length': '0'} 2022-09-03T06:18:29.402 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004e4d6eb8c20bc1be-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.446 DEBUG:boto:path=/dsgvcy-71/b 2022-09-03T06:18:29.448 DEBUG:boto:auth_path=/dsgvcy-71/b 2022-09-03T06:18:29.449 DEBUG:boto:Method: GET 2022-09-03T06:18:29.450 DEBUG:boto:Path: /dsgvcy-71/b 2022-09-03T06:18:29.450 DEBUG:boto:Data: 2022-09-03T06:18:29.451 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.451 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.452 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.453 DEBUG:boto:Params: {} 2022-09-03T06:18:29.453 DEBUG:boto:Token: None 2022-09-03T06:18:29.454 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/b 2022-09-03T06:18:29.455 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rNaUXSeopKQe9en3fbP/opc979E= 2022-09-03T06:18:29.455 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rNaUXSeopKQe9en3fbP/opc979E=', 'Content-Length': '0'} 2022-09-03T06:18:29.458 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c714f32c44812b69-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.459 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:29.460 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:29.461 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:29.461 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:29.462 DEBUG:boto:Method: GET 2022-09-03T06:18:29.462 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:29.463 DEBUG:boto:Data: 2022-09-03T06:18:29.464 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.464 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.465 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.465 DEBUG:boto:Params: {} 2022-09-03T06:18:29.466 DEBUG:boto:Token: None 2022-09-03T06:18:29.466 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/ 2022-09-03T06:18:29.467 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M= 2022-09-03T06:18:29.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M=', 'Content-Length': '0'} 2022-09-03T06:18:29.471 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007c5895e0464efc2f-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.472 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:29.473 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:29.474 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:29.474 DEBUG:boto:path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:29.475 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=b 2022-09-03T06:18:29.476 DEBUG:boto:Method: GET 2022-09-03T06:18:29.476 DEBUG:boto:Path: /dsgvcy-71/?prefix=b 2022-09-03T06:18:29.477 DEBUG:boto:Data: 2022-09-03T06:18:29.477 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.478 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.478 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.479 DEBUG:boto:Params: {} 2022-09-03T06:18:29.479 DEBUG:boto:Token: None 2022-09-03T06:18:29.480 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/ 2022-09-03T06:18:29.481 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M= 2022-09-03T06:18:29.481 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M=', 'Content-Length': '0'} 2022-09-03T06:18:29.485 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004802e6e64cffc0f1-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.485 DEBUG:boto:b'dsgvcy-71b1000falseb2022-09-03T06:12:47.312Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:29.486 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:18:29.487 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:29.488 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:29.488 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.489 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.490 DEBUG:boto:Method: GET 2022-09-03T06:18:29.490 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.491 DEBUG:boto:Data: 2022-09-03T06:18:29.491 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.492 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.493 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.494 DEBUG:boto:Params: {} 2022-09-03T06:18:29.495 DEBUG:boto:Token: None 2022-09-03T06:18:29.495 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.496 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BLJbM/xb5K+wkeaZ/sSv/iuJCpI= 2022-09-03T06:18:29.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BLJbM/xb5K+wkeaZ/sSv/iuJCpI=', 'Content-Length': '0'} 2022-09-03T06:18:29.499 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bad1a30ac70a8eff-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.500 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:29.501 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:29.501 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.502 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.502 DEBUG:boto:Method: GET 2022-09-03T06:18:29.503 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.503 DEBUG:boto:Data: 2022-09-03T06:18:29.504 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.504 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.505 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.505 DEBUG:boto:Params: {} 2022-09-03T06:18:29.506 DEBUG:boto:Token: None 2022-09-03T06:18:29.507 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.508 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BLJbM/xb5K+wkeaZ/sSv/iuJCpI= 2022-09-03T06:18:29.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BLJbM/xb5K+wkeaZ/sSv/iuJCpI=', 'Content-Length': '0'} 2022-09-03T06:18:29.511 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000034650827d8618c9a-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.511 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:29.512 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:29.513 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.513 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.514 DEBUG:boto:Method: HEAD 2022-09-03T06:18:29.514 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.515 DEBUG:boto:Data: 2022-09-03T06:18:29.515 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.516 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.516 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.517 DEBUG:boto:Params: {} 2022-09-03T06:18:29.517 DEBUG:boto:Token: None 2022-09-03T06:18:29.518 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.519 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4oEBOMdV7psf/c/Ll7Ps66MPr5Q= 2022-09-03T06:18:29.519 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4oEBOMdV7psf/c/Ll7Ps66MPr5Q=', 'Content-Length': '0'} 2022-09-03T06:18:29.521 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000020ffed94008e0ded-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.522 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:29.523 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:29.523 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.524 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.524 DEBUG:boto:Method: HEAD 2022-09-03T06:18:29.525 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.525 DEBUG:boto:Data: 2022-09-03T06:18:29.526 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.526 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.527 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.527 DEBUG:boto:Params: {} 2022-09-03T06:18:29.528 DEBUG:boto:Token: None 2022-09-03T06:18:29.529 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.529 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4oEBOMdV7psf/c/Ll7Ps66MPr5Q= 2022-09-03T06:18:29.530 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4oEBOMdV7psf/c/Ll7Ps66MPr5Q=', 'Content-Length': '0'} 2022-09-03T06:18:29.532 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aacd9f4597f356fa-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.533 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:18:29.533 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:29.534 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:29.534 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.535 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.536 DEBUG:boto:Method: GET 2022-09-03T06:18:29.536 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.537 DEBUG:boto:Data: 2022-09-03T06:18:29.537 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.538 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.538 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.539 DEBUG:boto:Params: {} 2022-09-03T06:18:29.539 DEBUG:boto:Token: None 2022-09-03T06:18:29.540 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.541 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BLJbM/xb5K+wkeaZ/sSv/iuJCpI= 2022-09-03T06:18:29.541 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BLJbM/xb5K+wkeaZ/sSv/iuJCpI=', 'Content-Length': '0'} 2022-09-03T06:18:29.543 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd5059c5572cd2f7-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.586 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:29.589 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:29.590 DEBUG:boto:path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.590 DEBUG:boto:auth_path=/dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.591 DEBUG:boto:Method: GET 2022-09-03T06:18:29.591 DEBUG:boto:Path: /dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.592 DEBUG:boto:Data: 2022-09-03T06:18:29.592 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.593 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.593 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.594 DEBUG:boto:Params: {} 2022-09-03T06:18:29.594 DEBUG:boto:Token: None 2022-09-03T06:18:29.595 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/c?versionId=null 2022-09-03T06:18:29.596 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BLJbM/xb5K+wkeaZ/sSv/iuJCpI= 2022-09-03T06:18:29.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BLJbM/xb5K+wkeaZ/sSv/iuJCpI=', 'Content-Length': '0'} 2022-09-03T06:18:29.599 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000092e0a6f0ea5ff6dd-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.601 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:29.602 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:29.603 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:29.603 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:29.604 DEBUG:boto:Method: GET 2022-09-03T06:18:29.604 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:29.606 DEBUG:boto:Data: 2022-09-03T06:18:29.607 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.607 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.608 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.608 DEBUG:boto:Params: {} 2022-09-03T06:18:29.609 DEBUG:boto:Token: None 2022-09-03T06:18:29.610 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/ 2022-09-03T06:18:29.610 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M= 2022-09-03T06:18:29.611 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M=', 'Content-Length': '0'} 2022-09-03T06:18:29.614 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c84e0476d2d94bb3-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.615 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:29.616 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:29.617 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:29.618 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:29.618 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:29.619 DEBUG:boto:Method: GET 2022-09-03T06:18:29.619 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:29.620 DEBUG:boto:Data: 2022-09-03T06:18:29.620 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.621 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.621 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.622 DEBUG:boto:Params: {} 2022-09-03T06:18:29.623 DEBUG:boto:Token: None 2022-09-03T06:18:29.623 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/ 2022-09-03T06:18:29.624 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M= 2022-09-03T06:18:29.624 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M=', 'Content-Length': '0'} 2022-09-03T06:18:29.627 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000167967231edf11d1-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.674 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:29.675 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:29.676 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:29.677 DEBUG:boto:Method: HEAD 2022-09-03T06:18:29.677 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:29.678 DEBUG:boto:Data: 2022-09-03T06:18:29.678 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.679 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.679 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.680 DEBUG:boto:Params: {} 2022-09-03T06:18:29.681 DEBUG:boto:Token: None 2022-09-03T06:18:29.681 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/c 2022-09-03T06:18:29.682 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TnjB5mGwIs80lEB97YuXLUbz5DM= 2022-09-03T06:18:29.683 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TnjB5mGwIs80lEB97YuXLUbz5DM=', 'Content-Length': '0'} 2022-09-03T06:18:29.685 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004750a76de2feff8b-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.685 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:29.686 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:29.687 DEBUG:boto:Method: HEAD 2022-09-03T06:18:29.687 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:29.688 DEBUG:boto:Data: 2022-09-03T06:18:29.688 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.689 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.689 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.690 DEBUG:boto:Params: {} 2022-09-03T06:18:29.690 DEBUG:boto:Token: None 2022-09-03T06:18:29.691 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/c 2022-09-03T06:18:29.692 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TnjB5mGwIs80lEB97YuXLUbz5DM= 2022-09-03T06:18:29.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TnjB5mGwIs80lEB97YuXLUbz5DM=', 'Content-Length': '0'} 2022-09-03T06:18:29.695 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003292d8b9435a494b-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.695 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:18:29.696 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:29.697 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:29.698 DEBUG:boto:Method: GET 2022-09-03T06:18:29.698 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:29.699 DEBUG:boto:Data: 2022-09-03T06:18:29.700 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.700 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.701 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.701 DEBUG:boto:Params: {} 2022-09-03T06:18:29.702 DEBUG:boto:Token: None 2022-09-03T06:18:29.703 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/c 2022-09-03T06:18:29.704 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DCgKMCGtYzXMJP4CMulF6+SV+Mg= 2022-09-03T06:18:29.705 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DCgKMCGtYzXMJP4CMulF6+SV+Mg=', 'Content-Length': '0'} 2022-09-03T06:18:29.707 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000003ef989c3e5a36c7-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.708 DEBUG:boto:path=/dsgvcy-71/c 2022-09-03T06:18:29.709 DEBUG:boto:auth_path=/dsgvcy-71/c 2022-09-03T06:18:29.710 DEBUG:boto:Method: GET 2022-09-03T06:18:29.711 DEBUG:boto:Path: /dsgvcy-71/c 2022-09-03T06:18:29.711 DEBUG:boto:Data: 2022-09-03T06:18:29.712 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.712 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.713 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.713 DEBUG:boto:Params: {} 2022-09-03T06:18:29.715 DEBUG:boto:Token: None 2022-09-03T06:18:29.716 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/c 2022-09-03T06:18:29.717 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DCgKMCGtYzXMJP4CMulF6+SV+Mg= 2022-09-03T06:18:29.718 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DCgKMCGtYzXMJP4CMulF6+SV+Mg=', 'Content-Length': '0'} 2022-09-03T06:18:29.721 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee3a522fb4edd6b9-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.723 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:29.724 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:29.725 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:29.726 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:29.726 DEBUG:boto:Method: GET 2022-09-03T06:18:29.727 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:29.727 DEBUG:boto:Data: 2022-09-03T06:18:29.728 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.729 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.729 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.730 DEBUG:boto:Params: {} 2022-09-03T06:18:29.730 DEBUG:boto:Token: None 2022-09-03T06:18:29.731 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/ 2022-09-03T06:18:29.732 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M= 2022-09-03T06:18:29.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M=', 'Content-Length': '0'} 2022-09-03T06:18:29.735 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cdbceaaaa4b4784e-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.736 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:29.737 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:29.738 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:29.738 DEBUG:boto:path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:29.739 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=c 2022-09-03T06:18:29.739 DEBUG:boto:Method: GET 2022-09-03T06:18:29.741 DEBUG:boto:Path: /dsgvcy-71/?prefix=c 2022-09-03T06:18:29.741 DEBUG:boto:Data: 2022-09-03T06:18:29.742 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.742 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.743 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.743 DEBUG:boto:Params: {} 2022-09-03T06:18:29.744 DEBUG:boto:Token: None 2022-09-03T06:18:29.745 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/ 2022-09-03T06:18:29.745 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M= 2022-09-03T06:18:29.746 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M=', 'Content-Length': '0'} 2022-09-03T06:18:29.749 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c291d43b5dc4e21e-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.750 DEBUG:boto:b'dsgvcy-71c1000falsec2022-09-03T06:12:47.356Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:29.751 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:18:29.752 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:29.753 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:29.753 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.754 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.755 DEBUG:boto:Method: GET 2022-09-03T06:18:29.755 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.756 DEBUG:boto:Data: 2022-09-03T06:18:29.756 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.757 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.758 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.758 DEBUG:boto:Params: {} 2022-09-03T06:18:29.759 DEBUG:boto:Token: None 2022-09-03T06:18:29.760 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.760 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O2+bA39ZjJ3Ot4i5sEpTGl1pKaM= 2022-09-03T06:18:29.761 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O2+bA39ZjJ3Ot4i5sEpTGl1pKaM=', 'Content-Length': '0'} 2022-09-03T06:18:29.763 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000004b37022ae4057f1-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.764 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:29.765 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:29.765 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.766 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.766 DEBUG:boto:Method: GET 2022-09-03T06:18:29.767 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.767 DEBUG:boto:Data: 2022-09-03T06:18:29.768 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.769 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.769 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.770 DEBUG:boto:Params: {} 2022-09-03T06:18:29.770 DEBUG:boto:Token: None 2022-09-03T06:18:29.771 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.772 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O2+bA39ZjJ3Ot4i5sEpTGl1pKaM= 2022-09-03T06:18:29.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O2+bA39ZjJ3Ot4i5sEpTGl1pKaM=', 'Content-Length': '0'} 2022-09-03T06:18:29.775 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000239d985a7084bfd7-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.776 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:29.776 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:29.777 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.778 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.778 DEBUG:boto:Method: HEAD 2022-09-03T06:18:29.779 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.779 DEBUG:boto:Data: 2022-09-03T06:18:29.780 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.780 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.781 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.781 DEBUG:boto:Params: {} 2022-09-03T06:18:29.782 DEBUG:boto:Token: None 2022-09-03T06:18:29.782 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.783 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HQOEEIAL4+GGDO32GpkRVGIs2V0= 2022-09-03T06:18:29.784 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HQOEEIAL4+GGDO32GpkRVGIs2V0=', 'Content-Length': '0'} 2022-09-03T06:18:29.788 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d80e2e6db5e1b46-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.789 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:29.790 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:29.790 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.791 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.791 DEBUG:boto:Method: HEAD 2022-09-03T06:18:29.792 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.792 DEBUG:boto:Data: 2022-09-03T06:18:29.793 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.793 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.794 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.795 DEBUG:boto:Params: {} 2022-09-03T06:18:29.795 DEBUG:boto:Token: None 2022-09-03T06:18:29.796 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.796 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HQOEEIAL4+GGDO32GpkRVGIs2V0= 2022-09-03T06:18:29.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HQOEEIAL4+GGDO32GpkRVGIs2V0=', 'Content-Length': '0'} 2022-09-03T06:18:29.799 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000533bdcd63150b3d4-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.800 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:18:29.800 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:29.801 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:29.802 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.802 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.803 DEBUG:boto:Method: GET 2022-09-03T06:18:29.803 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.804 DEBUG:boto:Data: 2022-09-03T06:18:29.804 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.805 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.806 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.806 DEBUG:boto:Params: {} 2022-09-03T06:18:29.807 DEBUG:boto:Token: None 2022-09-03T06:18:29.807 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.808 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O2+bA39ZjJ3Ot4i5sEpTGl1pKaM= 2022-09-03T06:18:29.809 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O2+bA39ZjJ3Ot4i5sEpTGl1pKaM=', 'Content-Length': '0'} 2022-09-03T06:18:29.811 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000395afa22224cb39a-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.812 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:29.813 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:29.813 DEBUG:boto:path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.814 DEBUG:boto:auth_path=/dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.815 DEBUG:boto:Method: GET 2022-09-03T06:18:29.816 DEBUG:boto:Path: /dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.817 DEBUG:boto:Data: 2022-09-03T06:18:29.818 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.819 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.820 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.821 DEBUG:boto:Params: {} 2022-09-03T06:18:29.822 DEBUG:boto:Token: None 2022-09-03T06:18:29.823 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/d?versionId=null 2022-09-03T06:18:29.824 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O2+bA39ZjJ3Ot4i5sEpTGl1pKaM= 2022-09-03T06:18:29.826 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O2+bA39ZjJ3Ot4i5sEpTGl1pKaM=', 'Content-Length': '0'} 2022-09-03T06:18:29.829 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008a8f797678a3196c-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.874 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:29.876 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:29.877 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:29.879 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:29.880 DEBUG:boto:Method: GET 2022-09-03T06:18:29.881 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:29.882 DEBUG:boto:Data: 2022-09-03T06:18:29.883 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.883 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.884 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.885 DEBUG:boto:Params: {} 2022-09-03T06:18:29.886 DEBUG:boto:Token: None 2022-09-03T06:18:29.887 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/ 2022-09-03T06:18:29.888 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M= 2022-09-03T06:18:29.889 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M=', 'Content-Length': '0'} 2022-09-03T06:18:29.892 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000011ec12fe7bf60d19-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.893 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:29.894 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:29.895 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:29.896 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:29.897 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:29.898 DEBUG:boto:Method: GET 2022-09-03T06:18:29.899 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:29.900 DEBUG:boto:Data: 2022-09-03T06:18:29.901 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.901 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.902 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.902 DEBUG:boto:Params: {} 2022-09-03T06:18:29.903 DEBUG:boto:Token: None 2022-09-03T06:18:29.904 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/ 2022-09-03T06:18:29.905 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M= 2022-09-03T06:18:29.906 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M=', 'Content-Length': '0'} 2022-09-03T06:18:29.909 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bce51b87d7535c42-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.910 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:29.912 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:29.912 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:29.913 DEBUG:boto:Method: HEAD 2022-09-03T06:18:29.913 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:29.914 DEBUG:boto:Data: 2022-09-03T06:18:29.915 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.915 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.916 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.916 DEBUG:boto:Params: {} 2022-09-03T06:18:29.917 DEBUG:boto:Token: None 2022-09-03T06:18:29.918 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/d 2022-09-03T06:18:29.919 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8OeKxfm7TUnR0ZEcB4ufovnuIxY= 2022-09-03T06:18:29.919 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8OeKxfm7TUnR0ZEcB4ufovnuIxY=', 'Content-Length': '0'} 2022-09-03T06:18:29.922 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cb734011c91cdabb-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.922 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:29.923 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:29.924 DEBUG:boto:Method: HEAD 2022-09-03T06:18:29.924 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:29.925 DEBUG:boto:Data: 2022-09-03T06:18:29.925 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.926 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.926 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.927 DEBUG:boto:Params: {} 2022-09-03T06:18:29.928 DEBUG:boto:Token: None 2022-09-03T06:18:29.928 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/d 2022-09-03T06:18:29.929 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8OeKxfm7TUnR0ZEcB4ufovnuIxY= 2022-09-03T06:18:29.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8OeKxfm7TUnR0ZEcB4ufovnuIxY=', 'Content-Length': '0'} 2022-09-03T06:18:29.932 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e4f7b8b973260f1e-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.933 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:18:29.933 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:29.934 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:29.935 DEBUG:boto:Method: GET 2022-09-03T06:18:29.935 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:29.936 DEBUG:boto:Data: 2022-09-03T06:18:29.937 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.937 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.938 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.938 DEBUG:boto:Params: {} 2022-09-03T06:18:29.939 DEBUG:boto:Token: None 2022-09-03T06:18:29.940 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/d 2022-09-03T06:18:29.941 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qKTq5/BYVkbIwTvO/L5O4Iz1uac= 2022-09-03T06:18:29.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qKTq5/BYVkbIwTvO/L5O4Iz1uac=', 'Content-Length': '0'} 2022-09-03T06:18:29.944 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000041ab77e839edab04-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.945 DEBUG:boto:path=/dsgvcy-71/d 2022-09-03T06:18:29.945 DEBUG:boto:auth_path=/dsgvcy-71/d 2022-09-03T06:18:29.946 DEBUG:boto:Method: GET 2022-09-03T06:18:29.946 DEBUG:boto:Path: /dsgvcy-71/d 2022-09-03T06:18:29.947 DEBUG:boto:Data: 2022-09-03T06:18:29.947 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.948 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.949 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.949 DEBUG:boto:Params: {} 2022-09-03T06:18:29.950 DEBUG:boto:Token: None 2022-09-03T06:18:29.951 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/d 2022-09-03T06:18:29.951 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qKTq5/BYVkbIwTvO/L5O4Iz1uac= 2022-09-03T06:18:29.952 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qKTq5/BYVkbIwTvO/L5O4Iz1uac=', 'Content-Length': '0'} 2022-09-03T06:18:29.954 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:12:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d5618e609a11f4c1-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.955 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:29.955 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:29.956 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:29.957 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:29.957 DEBUG:boto:Method: GET 2022-09-03T06:18:29.958 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:29.958 DEBUG:boto:Data: 2022-09-03T06:18:29.959 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.960 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.960 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.961 DEBUG:boto:Params: {} 2022-09-03T06:18:29.961 DEBUG:boto:Token: None 2022-09-03T06:18:29.962 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/ 2022-09-03T06:18:29.963 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M= 2022-09-03T06:18:29.963 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M=', 'Content-Length': '0'} 2022-09-03T06:18:29.966 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ccff4df9e6b6db07-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.967 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:29.968 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:29.969 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:29.969 DEBUG:boto:path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:29.970 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=d 2022-09-03T06:18:29.970 DEBUG:boto:Method: GET 2022-09-03T06:18:29.971 DEBUG:boto:Path: /dsgvcy-71/?prefix=d 2022-09-03T06:18:29.971 DEBUG:boto:Data: 2022-09-03T06:18:29.972 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.972 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:29.973 DEBUG:boto:Port: 8000 2022-09-03T06:18:29.973 DEBUG:boto:Params: {} 2022-09-03T06:18:29.974 DEBUG:boto:Token: None 2022-09-03T06:18:29.975 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/ 2022-09-03T06:18:29.975 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M= 2022-09-03T06:18:29.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NVWDB41mka8uZ3jRPxul+f03M/M=', 'Content-Length': '0'} 2022-09-03T06:18:29.979 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000da7b13cc22b80a4-006312f1b5-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:29.980 DEBUG:boto:b'dsgvcy-71d1000falsed2022-09-03T06:12:47.418Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:29.981 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:18:29.981 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:29.982 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:29.983 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:29.983 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:29.984 DEBUG:boto:Method: GET 2022-09-03T06:18:29.984 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:29.985 DEBUG:boto:Data: 2022-09-03T06:18:29.986 DEBUG:boto:Headers: {} 2022-09-03T06:18:29.987 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:29.988 DEBUG:boto:Port: 8001 2022-09-03T06:18:29.989 DEBUG:boto:Params: {} 2022-09-03T06:18:29.990 DEBUG:boto:Token: None 2022-09-03T06:18:29.991 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:29 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:29.992 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IJlYSuCXXqhUg24RplsEuMDvqI0= 2022-09-03T06:18:29.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IJlYSuCXXqhUg24RplsEuMDvqI0=', 'Content-Length': '0'} 2022-09-03T06:18:29.995 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000014b3043a96e97f4a-006312f1b5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.038 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:30.040 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:30.041 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:30.042 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:30.043 DEBUG:boto:Method: GET 2022-09-03T06:18:30.043 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:30.044 DEBUG:boto:Data: 2022-09-03T06:18:30.045 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.046 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.047 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.048 DEBUG:boto:Params: {} 2022-09-03T06:18:30.048 DEBUG:boto:Token: None 2022-09-03T06:18:30.049 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:30.049 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NgQ7kWZYLfKRajCnntAnOv+M/0E= 2022-09-03T06:18:30.050 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NgQ7kWZYLfKRajCnntAnOv+M/0E=', 'Content-Length': '0'} 2022-09-03T06:18:30.052 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006df8571a5e5d7634-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.053 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:30.054 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:30.054 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:30.055 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:30.055 DEBUG:boto:Method: HEAD 2022-09-03T06:18:30.056 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:30.056 DEBUG:boto:Data: 2022-09-03T06:18:30.057 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.058 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.058 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.059 DEBUG:boto:Params: {} 2022-09-03T06:18:30.059 DEBUG:boto:Token: None 2022-09-03T06:18:30.060 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:30.061 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ABDAkXzSd6wEXAK5KLdx9/t5TEo= 2022-09-03T06:18:30.061 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ABDAkXzSd6wEXAK5KLdx9/t5TEo=', 'Content-Length': '0'} 2022-09-03T06:18:30.064 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b1550ce7c8333626-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.064 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:30.065 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:30.065 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:30.066 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:30.066 DEBUG:boto:Method: HEAD 2022-09-03T06:18:30.067 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:30.067 DEBUG:boto:Data: 2022-09-03T06:18:30.068 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.068 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.069 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.070 DEBUG:boto:Params: {} 2022-09-03T06:18:30.070 DEBUG:boto:Token: None 2022-09-03T06:18:30.071 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:30.071 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ABDAkXzSd6wEXAK5KLdx9/t5TEo= 2022-09-03T06:18:30.072 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ABDAkXzSd6wEXAK5KLdx9/t5TEo=', 'Content-Length': '0'} 2022-09-03T06:18:30.074 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c4fb37bbfe26aae2-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.075 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:18:30.075 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:30.076 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:30.077 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:30.077 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:30.078 DEBUG:boto:Method: GET 2022-09-03T06:18:30.078 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:30.079 DEBUG:boto:Data: 2022-09-03T06:18:30.079 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.080 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.080 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.081 DEBUG:boto:Params: {} 2022-09-03T06:18:30.082 DEBUG:boto:Token: None 2022-09-03T06:18:30.082 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:30.083 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NgQ7kWZYLfKRajCnntAnOv+M/0E= 2022-09-03T06:18:30.084 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NgQ7kWZYLfKRajCnntAnOv+M/0E=', 'Content-Length': '0'} 2022-09-03T06:18:30.086 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2ee31bbbc50ab22-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.087 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:30.088 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:30.088 DEBUG:boto:path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:30.089 DEBUG:boto:auth_path=/dsgvcy-71/e?versionId=null 2022-09-03T06:18:30.089 DEBUG:boto:Method: GET 2022-09-03T06:18:30.090 DEBUG:boto:Path: /dsgvcy-71/e?versionId=null 2022-09-03T06:18:30.091 DEBUG:boto:Data: 2022-09-03T06:18:30.091 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.092 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.092 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.093 DEBUG:boto:Params: {} 2022-09-03T06:18:30.093 DEBUG:boto:Token: None 2022-09-03T06:18:30.094 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/e?versionId=null 2022-09-03T06:18:30.095 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NgQ7kWZYLfKRajCnntAnOv+M/0E= 2022-09-03T06:18:30.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NgQ7kWZYLfKRajCnntAnOv+M/0E=', 'Content-Length': '0'} 2022-09-03T06:18:30.098 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d4975380e382b86-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.138 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:30.139 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:30.141 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:30.142 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:30.143 DEBUG:boto:Method: GET 2022-09-03T06:18:30.144 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:30.145 DEBUG:boto:Data: 2022-09-03T06:18:30.145 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.146 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.147 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.148 DEBUG:boto:Params: {} 2022-09-03T06:18:30.149 DEBUG:boto:Token: None 2022-09-03T06:18:30.150 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/ 2022-09-03T06:18:30.150 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI= 2022-09-03T06:18:30.151 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI=', 'Content-Length': '0'} 2022-09-03T06:18:30.154 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e5ff270b8cbfe9e0-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.155 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:30.156 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:30.157 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:30.157 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:30.158 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:30.158 DEBUG:boto:Method: GET 2022-09-03T06:18:30.159 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:30.159 DEBUG:boto:Data: 2022-09-03T06:18:30.160 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.161 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.161 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.162 DEBUG:boto:Params: {} 2022-09-03T06:18:30.163 DEBUG:boto:Token: None 2022-09-03T06:18:30.164 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/ 2022-09-03T06:18:30.164 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI= 2022-09-03T06:18:30.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI=', 'Content-Length': '0'} 2022-09-03T06:18:30.168 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cf782ab276991465-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.169 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:30.170 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:30.171 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:30.171 DEBUG:boto:Method: HEAD 2022-09-03T06:18:30.172 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:30.172 DEBUG:boto:Data: 2022-09-03T06:18:30.173 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.174 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.174 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.175 DEBUG:boto:Params: {} 2022-09-03T06:18:30.176 DEBUG:boto:Token: None 2022-09-03T06:18:30.176 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/e 2022-09-03T06:18:30.177 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tZH/tCaCbt+JpCZoxndoBD+RsBo= 2022-09-03T06:18:30.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tZH/tCaCbt+JpCZoxndoBD+RsBo=', 'Content-Length': '0'} 2022-09-03T06:18:30.180 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d6fba4c56a121e98-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.181 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:30.182 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:30.182 DEBUG:boto:Method: HEAD 2022-09-03T06:18:30.183 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:30.183 DEBUG:boto:Data: 2022-09-03T06:18:30.184 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.185 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.185 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.186 DEBUG:boto:Params: {} 2022-09-03T06:18:30.187 DEBUG:boto:Token: None 2022-09-03T06:18:30.188 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/e 2022-09-03T06:18:30.188 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tZH/tCaCbt+JpCZoxndoBD+RsBo= 2022-09-03T06:18:30.189 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tZH/tCaCbt+JpCZoxndoBD+RsBo=', 'Content-Length': '0'} 2022-09-03T06:18:30.192 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e578fe1f3f070f79-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.193 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:18:30.195 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:30.196 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:30.197 DEBUG:boto:Method: GET 2022-09-03T06:18:30.197 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:30.198 DEBUG:boto:Data: 2022-09-03T06:18:30.199 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.200 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.200 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.201 DEBUG:boto:Params: {} 2022-09-03T06:18:30.201 DEBUG:boto:Token: None 2022-09-03T06:18:30.202 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/e 2022-09-03T06:18:30.203 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i1ZeVjPt6q3Av092R6OnNIbX2aQ= 2022-09-03T06:18:30.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i1ZeVjPt6q3Av092R6OnNIbX2aQ=', 'Content-Length': '0'} 2022-09-03T06:18:30.206 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008d0a60ea149a1dda-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.207 DEBUG:boto:path=/dsgvcy-71/e 2022-09-03T06:18:30.207 DEBUG:boto:auth_path=/dsgvcy-71/e 2022-09-03T06:18:30.208 DEBUG:boto:Method: GET 2022-09-03T06:18:30.209 DEBUG:boto:Path: /dsgvcy-71/e 2022-09-03T06:18:30.210 DEBUG:boto:Data: 2022-09-03T06:18:30.211 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.211 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.212 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.212 DEBUG:boto:Params: {} 2022-09-03T06:18:30.213 DEBUG:boto:Token: None 2022-09-03T06:18:30.213 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/e 2022-09-03T06:18:30.214 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i1ZeVjPt6q3Av092R6OnNIbX2aQ= 2022-09-03T06:18:30.215 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i1ZeVjPt6q3Av092R6OnNIbX2aQ=', 'Content-Length': '0'} 2022-09-03T06:18:30.218 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000075d1eda6fb4ce85e-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.219 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:30.220 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:30.220 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:30.221 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:30.222 DEBUG:boto:Method: GET 2022-09-03T06:18:30.222 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:30.223 DEBUG:boto:Data: 2022-09-03T06:18:30.224 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.224 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.225 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.225 DEBUG:boto:Params: {} 2022-09-03T06:18:30.226 DEBUG:boto:Token: None 2022-09-03T06:18:30.226 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/ 2022-09-03T06:18:30.228 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI= 2022-09-03T06:18:30.229 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI=', 'Content-Length': '0'} 2022-09-03T06:18:30.232 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000089e5a5b9e0326ec1-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.274 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:30.276 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:30.277 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:30.278 DEBUG:boto:path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:30.280 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=e 2022-09-03T06:18:30.280 DEBUG:boto:Method: GET 2022-09-03T06:18:30.281 DEBUG:boto:Path: /dsgvcy-71/?prefix=e 2022-09-03T06:18:30.282 DEBUG:boto:Data: 2022-09-03T06:18:30.283 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.283 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.284 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.285 DEBUG:boto:Params: {} 2022-09-03T06:18:30.286 DEBUG:boto:Token: None 2022-09-03T06:18:30.286 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/ 2022-09-03T06:18:30.287 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI= 2022-09-03T06:18:30.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI=', 'Content-Length': '0'} 2022-09-03T06:18:30.291 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000010e826b32ee91bbb-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.292 DEBUG:boto:b'dsgvcy-71e1000falsee2022-09-03T06:13:38.954Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:30.293 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:18:30.294 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:30.294 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:30.295 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.296 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.297 DEBUG:boto:Method: GET 2022-09-03T06:18:30.297 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.298 DEBUG:boto:Data: 2022-09-03T06:18:30.298 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.299 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.300 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.301 DEBUG:boto:Params: {} 2022-09-03T06:18:30.302 DEBUG:boto:Token: None 2022-09-03T06:18:30.303 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.304 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:W10FRGX8Y3UQS4GGZlxHNPOCt7o= 2022-09-03T06:18:30.306 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:W10FRGX8Y3UQS4GGZlxHNPOCt7o=', 'Content-Length': '0'} 2022-09-03T06:18:30.309 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fcf2eb596caaa9f7-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.311 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:30.312 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:30.313 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.314 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.315 DEBUG:boto:Method: GET 2022-09-03T06:18:30.316 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.316 DEBUG:boto:Data: 2022-09-03T06:18:30.317 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.317 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.318 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.319 DEBUG:boto:Params: {} 2022-09-03T06:18:30.320 DEBUG:boto:Token: None 2022-09-03T06:18:30.321 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.322 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:W10FRGX8Y3UQS4GGZlxHNPOCt7o= 2022-09-03T06:18:30.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:W10FRGX8Y3UQS4GGZlxHNPOCt7o=', 'Content-Length': '0'} 2022-09-03T06:18:30.327 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f3ffd738f31a2de4-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.328 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:30.328 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:30.329 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.329 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.330 DEBUG:boto:Method: HEAD 2022-09-03T06:18:30.331 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.331 DEBUG:boto:Data: 2022-09-03T06:18:30.332 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.332 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.333 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.335 DEBUG:boto:Params: {} 2022-09-03T06:18:30.336 DEBUG:boto:Token: None 2022-09-03T06:18:30.337 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.338 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iZ2arNU0OAn+kYd5X+Q5cDaJCt0= 2022-09-03T06:18:30.339 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iZ2arNU0OAn+kYd5X+Q5cDaJCt0=', 'Content-Length': '0'} 2022-09-03T06:18:30.343 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007adf5890cee39331-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.344 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:30.345 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:30.346 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.347 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.347 DEBUG:boto:Method: HEAD 2022-09-03T06:18:30.348 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.348 DEBUG:boto:Data: 2022-09-03T06:18:30.349 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.349 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.350 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.351 DEBUG:boto:Params: {} 2022-09-03T06:18:30.352 DEBUG:boto:Token: None 2022-09-03T06:18:30.353 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.354 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iZ2arNU0OAn+kYd5X+Q5cDaJCt0= 2022-09-03T06:18:30.355 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iZ2arNU0OAn+kYd5X+Q5cDaJCt0=', 'Content-Length': '0'} 2022-09-03T06:18:30.362 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005ce43a3923c5a700-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.364 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:18:30.364 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:30.365 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:30.366 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.366 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.367 DEBUG:boto:Method: GET 2022-09-03T06:18:30.367 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.368 DEBUG:boto:Data: 2022-09-03T06:18:30.369 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.369 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.370 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.370 DEBUG:boto:Params: {} 2022-09-03T06:18:30.371 DEBUG:boto:Token: None 2022-09-03T06:18:30.372 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.372 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:W10FRGX8Y3UQS4GGZlxHNPOCt7o= 2022-09-03T06:18:30.373 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:W10FRGX8Y3UQS4GGZlxHNPOCt7o=', 'Content-Length': '0'} 2022-09-03T06:18:30.375 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000050c6c16b0e6f0361-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.376 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:30.377 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:30.377 DEBUG:boto:path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.378 DEBUG:boto:auth_path=/dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.379 DEBUG:boto:Method: GET 2022-09-03T06:18:30.379 DEBUG:boto:Path: /dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.380 DEBUG:boto:Data: 2022-09-03T06:18:30.380 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.381 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.381 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.382 DEBUG:boto:Params: {} 2022-09-03T06:18:30.382 DEBUG:boto:Token: None 2022-09-03T06:18:30.383 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/f?versionId=null 2022-09-03T06:18:30.384 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:W10FRGX8Y3UQS4GGZlxHNPOCt7o= 2022-09-03T06:18:30.385 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:W10FRGX8Y3UQS4GGZlxHNPOCt7o=', 'Content-Length': '0'} 2022-09-03T06:18:30.387 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000676ec6b915b7ced0-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.434 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:30.436 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:30.437 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:30.438 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:30.439 DEBUG:boto:Method: GET 2022-09-03T06:18:30.440 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:30.441 DEBUG:boto:Data: 2022-09-03T06:18:30.443 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.444 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.445 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.446 DEBUG:boto:Params: {} 2022-09-03T06:18:30.447 DEBUG:boto:Token: None 2022-09-03T06:18:30.448 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/ 2022-09-03T06:18:30.449 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI= 2022-09-03T06:18:30.450 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI=', 'Content-Length': '0'} 2022-09-03T06:18:30.454 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001c672e8682a2a529-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.455 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:30.457 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:30.457 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:30.458 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:30.459 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:30.460 DEBUG:boto:Method: GET 2022-09-03T06:18:30.461 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:30.462 DEBUG:boto:Data: 2022-09-03T06:18:30.463 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.464 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.465 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.466 DEBUG:boto:Params: {} 2022-09-03T06:18:30.467 DEBUG:boto:Token: None 2022-09-03T06:18:30.468 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/ 2022-09-03T06:18:30.469 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI= 2022-09-03T06:18:30.470 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI=', 'Content-Length': '0'} 2022-09-03T06:18:30.474 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000be03d6e06413ce7d-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.475 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:30.476 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:30.477 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:30.478 DEBUG:boto:Method: HEAD 2022-09-03T06:18:30.478 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:30.479 DEBUG:boto:Data: 2022-09-03T06:18:30.479 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.480 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.480 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.481 DEBUG:boto:Params: {} 2022-09-03T06:18:30.481 DEBUG:boto:Token: None 2022-09-03T06:18:30.482 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/f 2022-09-03T06:18:30.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vrAZDWavTvA03+1+7LXtwJE6MQo= 2022-09-03T06:18:30.483 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vrAZDWavTvA03+1+7LXtwJE6MQo=', 'Content-Length': '0'} 2022-09-03T06:18:30.485 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000042ea0dda1cbfda3f-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.486 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:30.487 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:30.487 DEBUG:boto:Method: HEAD 2022-09-03T06:18:30.488 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:30.488 DEBUG:boto:Data: 2022-09-03T06:18:30.489 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.489 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.490 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.490 DEBUG:boto:Params: {} 2022-09-03T06:18:30.491 DEBUG:boto:Token: None 2022-09-03T06:18:30.491 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/f 2022-09-03T06:18:30.492 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vrAZDWavTvA03+1+7LXtwJE6MQo= 2022-09-03T06:18:30.493 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vrAZDWavTvA03+1+7LXtwJE6MQo=', 'Content-Length': '0'} 2022-09-03T06:18:30.495 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b104f2f9b3c6ab62-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.496 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:18:30.496 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:30.497 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:30.497 DEBUG:boto:Method: GET 2022-09-03T06:18:30.498 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:30.498 DEBUG:boto:Data: 2022-09-03T06:18:30.499 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.499 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.500 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.500 DEBUG:boto:Params: {} 2022-09-03T06:18:30.501 DEBUG:boto:Token: None 2022-09-03T06:18:30.502 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/f 2022-09-03T06:18:30.502 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mewZc2MzLUS5oDM+tUttea4o6rs= 2022-09-03T06:18:30.503 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mewZc2MzLUS5oDM+tUttea4o6rs=', 'Content-Length': '0'} 2022-09-03T06:18:30.505 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002e7ba154b34b92e6-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.550 DEBUG:boto:path=/dsgvcy-71/f 2022-09-03T06:18:30.551 DEBUG:boto:auth_path=/dsgvcy-71/f 2022-09-03T06:18:30.552 DEBUG:boto:Method: GET 2022-09-03T06:18:30.552 DEBUG:boto:Path: /dsgvcy-71/f 2022-09-03T06:18:30.553 DEBUG:boto:Data: 2022-09-03T06:18:30.553 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.554 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.554 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.555 DEBUG:boto:Params: {} 2022-09-03T06:18:30.556 DEBUG:boto:Token: None 2022-09-03T06:18:30.556 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/f 2022-09-03T06:18:30.557 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mewZc2MzLUS5oDM+tUttea4o6rs= 2022-09-03T06:18:30.557 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mewZc2MzLUS5oDM+tUttea4o6rs=', 'Content-Length': '0'} 2022-09-03T06:18:30.559 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:38 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f50c109281fb8f56-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.560 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:30.561 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:30.561 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:30.562 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:30.563 DEBUG:boto:Method: GET 2022-09-03T06:18:30.563 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:30.564 DEBUG:boto:Data: 2022-09-03T06:18:30.564 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.565 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.566 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.567 DEBUG:boto:Params: {} 2022-09-03T06:18:30.568 DEBUG:boto:Token: None 2022-09-03T06:18:30.569 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/ 2022-09-03T06:18:30.570 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI= 2022-09-03T06:18:30.571 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI=', 'Content-Length': '0'} 2022-09-03T06:18:30.575 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c301f0375c7eed69-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.575 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:30.576 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:30.577 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:30.577 DEBUG:boto:path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:30.578 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=f 2022-09-03T06:18:30.578 DEBUG:boto:Method: GET 2022-09-03T06:18:30.579 DEBUG:boto:Path: /dsgvcy-71/?prefix=f 2022-09-03T06:18:30.580 DEBUG:boto:Data: 2022-09-03T06:18:30.580 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.581 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.581 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.582 DEBUG:boto:Params: {} 2022-09-03T06:18:30.582 DEBUG:boto:Token: None 2022-09-03T06:18:30.583 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/ 2022-09-03T06:18:30.583 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI= 2022-09-03T06:18:30.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI=', 'Content-Length': '0'} 2022-09-03T06:18:30.587 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004e2919381fca48f4-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.588 DEBUG:boto:b'dsgvcy-71f1000falsef2022-09-03T06:13:38.996Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:30.588 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:18:30.589 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:30.590 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:30.590 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.591 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.591 DEBUG:boto:Method: GET 2022-09-03T06:18:30.592 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.592 DEBUG:boto:Data: 2022-09-03T06:18:30.593 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.593 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.594 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.595 DEBUG:boto:Params: {} 2022-09-03T06:18:30.595 DEBUG:boto:Token: None 2022-09-03T06:18:30.596 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.596 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:77Mz09hSNmLPmv37+kpDPBfvkKs= 2022-09-03T06:18:30.597 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:77Mz09hSNmLPmv37+kpDPBfvkKs=', 'Content-Length': '0'} 2022-09-03T06:18:30.600 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002648d7d957831a4a-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.601 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:30.601 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:30.602 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.602 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.603 DEBUG:boto:Method: GET 2022-09-03T06:18:30.603 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.604 DEBUG:boto:Data: 2022-09-03T06:18:30.605 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.605 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.606 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.606 DEBUG:boto:Params: {} 2022-09-03T06:18:30.607 DEBUG:boto:Token: None 2022-09-03T06:18:30.608 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.608 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:77Mz09hSNmLPmv37+kpDPBfvkKs= 2022-09-03T06:18:30.609 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:77Mz09hSNmLPmv37+kpDPBfvkKs=', 'Content-Length': '0'} 2022-09-03T06:18:30.611 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c9d9be06c5455fd-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.658 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:30.659 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:30.660 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.661 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.661 DEBUG:boto:Method: HEAD 2022-09-03T06:18:30.662 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.662 DEBUG:boto:Data: 2022-09-03T06:18:30.663 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.664 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.664 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.665 DEBUG:boto:Params: {} 2022-09-03T06:18:30.665 DEBUG:boto:Token: None 2022-09-03T06:18:30.666 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.667 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2MkaScdVuc+59Y+5gUErE8mTUXQ= 2022-09-03T06:18:30.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2MkaScdVuc+59Y+5gUErE8mTUXQ=', 'Content-Length': '0'} 2022-09-03T06:18:30.671 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ef180f086ca97957-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.671 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:30.672 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:30.673 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.674 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.674 DEBUG:boto:Method: HEAD 2022-09-03T06:18:30.675 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.675 DEBUG:boto:Data: 2022-09-03T06:18:30.676 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.676 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.677 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.677 DEBUG:boto:Params: {} 2022-09-03T06:18:30.678 DEBUG:boto:Token: None 2022-09-03T06:18:30.679 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.679 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2MkaScdVuc+59Y+5gUErE8mTUXQ= 2022-09-03T06:18:30.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2MkaScdVuc+59Y+5gUErE8mTUXQ=', 'Content-Length': '0'} 2022-09-03T06:18:30.683 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ff3a3e6a94caa3dd-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.683 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:18:30.684 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:30.685 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:30.686 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.687 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.688 DEBUG:boto:Method: GET 2022-09-03T06:18:30.689 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.690 DEBUG:boto:Data: 2022-09-03T06:18:30.690 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.691 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.691 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.692 DEBUG:boto:Params: {} 2022-09-03T06:18:30.693 DEBUG:boto:Token: None 2022-09-03T06:18:30.694 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.695 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:77Mz09hSNmLPmv37+kpDPBfvkKs= 2022-09-03T06:18:30.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:77Mz09hSNmLPmv37+kpDPBfvkKs=', 'Content-Length': '0'} 2022-09-03T06:18:30.698 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f22336b20335192-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.738 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:30.739 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:30.740 DEBUG:boto:path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.741 DEBUG:boto:auth_path=/dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.742 DEBUG:boto:Method: GET 2022-09-03T06:18:30.743 DEBUG:boto:Path: /dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.744 DEBUG:boto:Data: 2022-09-03T06:18:30.745 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.746 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.747 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.754 DEBUG:boto:Params: {} 2022-09-03T06:18:30.756 DEBUG:boto:Token: None 2022-09-03T06:18:30.757 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/g?versionId=null 2022-09-03T06:18:30.758 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:77Mz09hSNmLPmv37+kpDPBfvkKs= 2022-09-03T06:18:30.759 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:77Mz09hSNmLPmv37+kpDPBfvkKs=', 'Content-Length': '0'} 2022-09-03T06:18:30.762 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004dddfa30993279fc-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.763 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:30.764 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:30.764 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:30.765 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:30.767 DEBUG:boto:Method: GET 2022-09-03T06:18:30.768 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:30.768 DEBUG:boto:Data: 2022-09-03T06:18:30.769 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.769 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.770 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.770 DEBUG:boto:Params: {} 2022-09-03T06:18:30.771 DEBUG:boto:Token: None 2022-09-03T06:18:30.771 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/ 2022-09-03T06:18:30.772 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI= 2022-09-03T06:18:30.773 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI=', 'Content-Length': '0'} 2022-09-03T06:18:30.776 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003d8bb812c995b0eb-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.776 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:30.777 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:30.778 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:30.778 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:30.779 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:30.779 DEBUG:boto:Method: GET 2022-09-03T06:18:30.780 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:30.780 DEBUG:boto:Data: 2022-09-03T06:18:30.781 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.782 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.782 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.783 DEBUG:boto:Params: {} 2022-09-03T06:18:30.783 DEBUG:boto:Token: None 2022-09-03T06:18:30.784 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/ 2022-09-03T06:18:30.784 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI= 2022-09-03T06:18:30.785 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI=', 'Content-Length': '0'} 2022-09-03T06:18:30.789 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ee909f26f742bd1d-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.790 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:30.792 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:30.794 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:30.795 DEBUG:boto:Method: HEAD 2022-09-03T06:18:30.796 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:30.797 DEBUG:boto:Data: 2022-09-03T06:18:30.798 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.798 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.799 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.802 DEBUG:boto:Params: {} 2022-09-03T06:18:30.804 DEBUG:boto:Token: None 2022-09-03T06:18:30.805 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/g 2022-09-03T06:18:30.806 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0Hf/fkAfq5WbT2hIysMQHfpTZEw= 2022-09-03T06:18:30.807 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0Hf/fkAfq5WbT2hIysMQHfpTZEw=', 'Content-Length': '0'} 2022-09-03T06:18:30.809 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009175a2e8be60b319-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.810 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:30.811 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:30.811 DEBUG:boto:Method: HEAD 2022-09-03T06:18:30.812 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:30.812 DEBUG:boto:Data: 2022-09-03T06:18:30.813 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.813 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.814 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.815 DEBUG:boto:Params: {} 2022-09-03T06:18:30.815 DEBUG:boto:Token: None 2022-09-03T06:18:30.816 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/g 2022-09-03T06:18:30.817 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0Hf/fkAfq5WbT2hIysMQHfpTZEw= 2022-09-03T06:18:30.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0Hf/fkAfq5WbT2hIysMQHfpTZEw=', 'Content-Length': '0'} 2022-09-03T06:18:30.819 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bac9e22fcd1cbea9-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.820 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:18:30.821 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:30.821 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:30.822 DEBUG:boto:Method: GET 2022-09-03T06:18:30.824 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:30.825 DEBUG:boto:Data: 2022-09-03T06:18:30.825 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.826 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.827 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.827 DEBUG:boto:Params: {} 2022-09-03T06:18:30.828 DEBUG:boto:Token: None 2022-09-03T06:18:30.828 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/g 2022-09-03T06:18:30.829 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cLgRAoaTfNQvaJCNYs00ozVxEj8= 2022-09-03T06:18:30.830 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cLgRAoaTfNQvaJCNYs00ozVxEj8=', 'Content-Length': '0'} 2022-09-03T06:18:30.833 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000422908f61f73c39-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.833 DEBUG:boto:path=/dsgvcy-71/g 2022-09-03T06:18:30.834 DEBUG:boto:auth_path=/dsgvcy-71/g 2022-09-03T06:18:30.835 DEBUG:boto:Method: GET 2022-09-03T06:18:30.835 DEBUG:boto:Path: /dsgvcy-71/g 2022-09-03T06:18:30.836 DEBUG:boto:Data: 2022-09-03T06:18:30.837 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.837 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.838 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.838 DEBUG:boto:Params: {} 2022-09-03T06:18:30.839 DEBUG:boto:Token: None 2022-09-03T06:18:30.840 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/g 2022-09-03T06:18:30.841 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cLgRAoaTfNQvaJCNYs00ozVxEj8= 2022-09-03T06:18:30.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cLgRAoaTfNQvaJCNYs00ozVxEj8=', 'Content-Length': '0'} 2022-09-03T06:18:30.844 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bdfb2008ebfeada9-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.845 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:30.845 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:30.846 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:30.847 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:30.847 DEBUG:boto:Method: GET 2022-09-03T06:18:30.848 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:30.848 DEBUG:boto:Data: 2022-09-03T06:18:30.849 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.849 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.850 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.851 DEBUG:boto:Params: {} 2022-09-03T06:18:30.851 DEBUG:boto:Token: None 2022-09-03T06:18:30.852 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/ 2022-09-03T06:18:30.852 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI= 2022-09-03T06:18:30.853 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI=', 'Content-Length': '0'} 2022-09-03T06:18:30.856 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f54b1e3e5834874f-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.857 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:30.858 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:30.858 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:30.859 DEBUG:boto:path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:30.859 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=g 2022-09-03T06:18:30.860 DEBUG:boto:Method: GET 2022-09-03T06:18:30.860 DEBUG:boto:Path: /dsgvcy-71/?prefix=g 2022-09-03T06:18:30.861 DEBUG:boto:Data: 2022-09-03T06:18:30.861 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.862 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.863 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.863 DEBUG:boto:Params: {} 2022-09-03T06:18:30.864 DEBUG:boto:Token: None 2022-09-03T06:18:30.864 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/ 2022-09-03T06:18:30.865 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI= 2022-09-03T06:18:30.865 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI=', 'Content-Length': '0'} 2022-09-03T06:18:30.868 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000af847985067cf0ab-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.869 DEBUG:boto:b'dsgvcy-71g1000falseg2022-09-03T06:13:39.065Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:30.870 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:18:30.870 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:30.871 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:30.871 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.872 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.872 DEBUG:boto:Method: GET 2022-09-03T06:18:30.873 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.874 DEBUG:boto:Data: 2022-09-03T06:18:30.874 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.875 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.875 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.876 DEBUG:boto:Params: {} 2022-09-03T06:18:30.877 DEBUG:boto:Token: None 2022-09-03T06:18:30.877 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.878 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MJ7DBgo832mxEEm0mFz0Xa4LuyE= 2022-09-03T06:18:30.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MJ7DBgo832mxEEm0mFz0Xa4LuyE=', 'Content-Length': '0'} 2022-09-03T06:18:30.881 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2ad4df6d500b2dd-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.882 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:30.883 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:30.883 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.884 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.884 DEBUG:boto:Method: GET 2022-09-03T06:18:30.885 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.885 DEBUG:boto:Data: 2022-09-03T06:18:30.886 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.886 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.887 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.887 DEBUG:boto:Params: {} 2022-09-03T06:18:30.888 DEBUG:boto:Token: None 2022-09-03T06:18:30.889 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.889 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MJ7DBgo832mxEEm0mFz0Xa4LuyE= 2022-09-03T06:18:30.890 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MJ7DBgo832mxEEm0mFz0Xa4LuyE=', 'Content-Length': '0'} 2022-09-03T06:18:30.893 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000012e9f328af0e5841-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.894 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:30.894 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:30.895 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.895 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.896 DEBUG:boto:Method: HEAD 2022-09-03T06:18:30.896 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.897 DEBUG:boto:Data: 2022-09-03T06:18:30.897 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.898 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.898 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.899 DEBUG:boto:Params: {} 2022-09-03T06:18:30.899 DEBUG:boto:Token: None 2022-09-03T06:18:30.900 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.900 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zdILhUH7ZDz3pSLMH1vN7lqwNiU= 2022-09-03T06:18:30.901 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zdILhUH7ZDz3pSLMH1vN7lqwNiU=', 'Content-Length': '0'} 2022-09-03T06:18:30.905 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000442897887947f377-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.906 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:30.907 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:30.907 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.908 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.908 DEBUG:boto:Method: HEAD 2022-09-03T06:18:30.909 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.909 DEBUG:boto:Data: 2022-09-03T06:18:30.910 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.910 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.911 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.911 DEBUG:boto:Params: {} 2022-09-03T06:18:30.912 DEBUG:boto:Token: None 2022-09-03T06:18:30.913 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.913 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zdILhUH7ZDz3pSLMH1vN7lqwNiU= 2022-09-03T06:18:30.914 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zdILhUH7ZDz3pSLMH1vN7lqwNiU=', 'Content-Length': '0'} 2022-09-03T06:18:30.916 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b8221e36af55aaf6-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.917 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:18:30.918 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:30.918 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:30.919 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.919 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.920 DEBUG:boto:Method: GET 2022-09-03T06:18:30.920 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.921 DEBUG:boto:Data: 2022-09-03T06:18:30.921 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.922 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.923 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.923 DEBUG:boto:Params: {} 2022-09-03T06:18:30.924 DEBUG:boto:Token: None 2022-09-03T06:18:30.924 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.925 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MJ7DBgo832mxEEm0mFz0Xa4LuyE= 2022-09-03T06:18:30.926 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MJ7DBgo832mxEEm0mFz0Xa4LuyE=', 'Content-Length': '0'} 2022-09-03T06:18:30.928 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c6385cfd23e3b3f1-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.929 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:30.929 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:30.930 DEBUG:boto:path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.930 DEBUG:boto:auth_path=/dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.931 DEBUG:boto:Method: GET 2022-09-03T06:18:30.931 DEBUG:boto:Path: /dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.932 DEBUG:boto:Data: 2022-09-03T06:18:30.932 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.934 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.934 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.935 DEBUG:boto:Params: {} 2022-09-03T06:18:30.935 DEBUG:boto:Token: None 2022-09-03T06:18:30.936 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/h?versionId=null 2022-09-03T06:18:30.936 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MJ7DBgo832mxEEm0mFz0Xa4LuyE= 2022-09-03T06:18:30.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MJ7DBgo832mxEEm0mFz0Xa4LuyE=', 'Content-Length': '0'} 2022-09-03T06:18:30.939 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b92c48fbb028d7f9-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.940 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:30.941 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:30.941 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:30.942 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:30.942 DEBUG:boto:Method: GET 2022-09-03T06:18:30.943 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:30.943 DEBUG:boto:Data: 2022-09-03T06:18:30.944 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.944 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.945 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.946 DEBUG:boto:Params: {} 2022-09-03T06:18:30.946 DEBUG:boto:Token: None 2022-09-03T06:18:30.947 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/ 2022-09-03T06:18:30.947 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI= 2022-09-03T06:18:30.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI=', 'Content-Length': '0'} 2022-09-03T06:18:30.951 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000556b56529e34898f-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.951 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:30.952 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:30.953 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:30.953 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:30.954 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:30.955 DEBUG:boto:Method: GET 2022-09-03T06:18:30.955 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:30.956 DEBUG:boto:Data: 2022-09-03T06:18:30.956 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.957 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.957 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.958 DEBUG:boto:Params: {} 2022-09-03T06:18:30.958 DEBUG:boto:Token: None 2022-09-03T06:18:30.959 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/ 2022-09-03T06:18:30.960 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI= 2022-09-03T06:18:30.960 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qYH/Hd2ZcgKE2Z2i0O+SjX9wiwI=', 'Content-Length': '0'} 2022-09-03T06:18:30.963 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000000838f1b7844bfd-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.964 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:30.964 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:30.965 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:30.965 DEBUG:boto:Method: HEAD 2022-09-03T06:18:30.966 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:30.967 DEBUG:boto:Data: 2022-09-03T06:18:30.967 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.968 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.968 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.969 DEBUG:boto:Params: {} 2022-09-03T06:18:30.969 DEBUG:boto:Token: None 2022-09-03T06:18:30.970 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/h 2022-09-03T06:18:30.970 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NpmXDU1mwTwZU7WwoIikZlfZXFk= 2022-09-03T06:18:30.971 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NpmXDU1mwTwZU7WwoIikZlfZXFk=', 'Content-Length': '0'} 2022-09-03T06:18:30.973 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000069b85994e522d8c1-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.974 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:30.974 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:30.975 DEBUG:boto:Method: HEAD 2022-09-03T06:18:30.975 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:30.976 DEBUG:boto:Data: 2022-09-03T06:18:30.976 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.977 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:30.977 DEBUG:boto:Port: 8000 2022-09-03T06:18:30.978 DEBUG:boto:Params: {} 2022-09-03T06:18:30.978 DEBUG:boto:Token: None 2022-09-03T06:18:30.979 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/h 2022-09-03T06:18:30.980 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NpmXDU1mwTwZU7WwoIikZlfZXFk= 2022-09-03T06:18:30.980 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NpmXDU1mwTwZU7WwoIikZlfZXFk=', 'Content-Length': '0'} 2022-09-03T06:18:30.982 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000411926c5b01d8ef5-006312f1b6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:30.983 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:18:30.984 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:30.985 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:30.985 DEBUG:boto:Method: GET 2022-09-03T06:18:30.986 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:30.986 DEBUG:boto:Data: 2022-09-03T06:18:30.987 DEBUG:boto:Headers: {} 2022-09-03T06:18:30.987 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:30.988 DEBUG:boto:Port: 8001 2022-09-03T06:18:30.988 DEBUG:boto:Params: {} 2022-09-03T06:18:30.989 DEBUG:boto:Token: None 2022-09-03T06:18:30.990 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:30 GMT /dsgvcy-71/h 2022-09-03T06:18:30.990 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kN5c/dULrAHLpcKJUdn3k6a7+88= 2022-09-03T06:18:30.991 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kN5c/dULrAHLpcKJUdn3k6a7+88=', 'Content-Length': '0'} 2022-09-03T06:18:30.993 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a6ad284449ab27cf-006312f1b6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:31.034 DEBUG:boto:path=/dsgvcy-71/h 2022-09-03T06:18:31.035 DEBUG:boto:auth_path=/dsgvcy-71/h 2022-09-03T06:18:31.036 DEBUG:boto:Method: GET 2022-09-03T06:18:31.036 DEBUG:boto:Path: /dsgvcy-71/h 2022-09-03T06:18:31.037 DEBUG:boto:Data: 2022-09-03T06:18:31.038 DEBUG:boto:Headers: {} 2022-09-03T06:18:31.039 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:31.039 DEBUG:boto:Port: 8000 2022-09-03T06:18:31.040 DEBUG:boto:Params: {} 2022-09-03T06:18:31.040 DEBUG:boto:Token: None 2022-09-03T06:18:31.041 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:31 GMT /dsgvcy-71/h 2022-09-03T06:18:31.042 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pdTSCt9qnhHsGvW1XADHKnKzu5k= 2022-09-03T06:18:31.042 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pdTSCt9qnhHsGvW1XADHKnKzu5k=', 'Content-Length': '0'} 2022-09-03T06:18:31.045 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:13:39 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cf2e991abbd43629-006312f1b7-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:18:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:31.046 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:31.047 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:31.047 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:31.048 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:31.048 DEBUG:boto:Method: GET 2022-09-03T06:18:31.049 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:31.049 DEBUG:boto:Data: 2022-09-03T06:18:31.050 DEBUG:boto:Headers: {} 2022-09-03T06:18:31.051 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:18:31.051 DEBUG:boto:Port: 8001 2022-09-03T06:18:31.052 DEBUG:boto:Params: {} 2022-09-03T06:18:31.052 DEBUG:boto:Token: None 2022-09-03T06:18:31.053 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:31 GMT /dsgvcy-71/ 2022-09-03T06:18:31.053 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dW6AIK1fabSTna2KesJ1Ruf5Mpc= 2022-09-03T06:18:31.054 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dW6AIK1fabSTna2KesJ1Ruf5Mpc=', 'Content-Length': '0'} 2022-09-03T06:18:31.057 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b52881bd181bfea4-006312f1b7-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:31.058 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:31.059 DEBUG:boto:path=/dsgvcy-71/ 2022-09-03T06:18:31.059 DEBUG:boto:auth_path=/dsgvcy-71/ 2022-09-03T06:18:31.060 DEBUG:boto:path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:31.060 DEBUG:boto:auth_path=/dsgvcy-71/?prefix=h 2022-09-03T06:18:31.061 DEBUG:boto:Method: GET 2022-09-03T06:18:31.061 DEBUG:boto:Path: /dsgvcy-71/?prefix=h 2022-09-03T06:18:31.062 DEBUG:boto:Data: 2022-09-03T06:18:31.063 DEBUG:boto:Headers: {} 2022-09-03T06:18:31.063 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:18:31.064 DEBUG:boto:Port: 8000 2022-09-03T06:18:31.064 DEBUG:boto:Params: {} 2022-09-03T06:18:31.065 DEBUG:boto:Token: None 2022-09-03T06:18:31.065 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:18:31 GMT /dsgvcy-71/ 2022-09-03T06:18:31.066 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dW6AIK1fabSTna2KesJ1Ruf5Mpc= 2022-09-03T06:18:31.067 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dW6AIK1fabSTna2KesJ1Ruf5Mpc=', 'Content-Length': '0'} 2022-09-03T06:18:31.069 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a7f575db2845dbc6-006312f1b7-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:18:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:31.070 DEBUG:boto:b'dsgvcy-71h1000falseh2022-09-03T06:13:39.105Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:18:31.071 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-71 zones={test-zone3, test-zone2} 2022-09-03T06:18:31.072 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:18:31.072 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:18:31.073 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog autotrim --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T06:18:31.125 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:31.126 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:31.183 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:18:31.180+0000 7faba80bd700 -1 req 10677223718953673294 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:18:31.184 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:18:31.180+0000 7fabae0c9700 -1 req 4817067511408813108 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-43:e40d88ba-d542-483a-afa2-5de687affd14.5176.2,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-43:e40d88ba-d542-483a-afa2-5de687affd14.5176.2,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:18:31.185 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:18:31.181+0000 7fc81970b700 -1 req 4300406920823760894 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-43:e40d88ba-d542-483a-afa2-5de687affd14.5176.2,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-43:e40d88ba-d542-483a-afa2-5de687affd14.5176.2,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:18:31.185 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:18:31.181+0000 7fc81ef16700 -1 req 9593394673608734207 0.000999980s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:18:31.186 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:18:31.182+0000 7f6b42148700 -1 req 16471514451205644854 0.000999980s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-43:e40d88ba-d542-483a-afa2-5de687affd14.5176.2,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-43:e40d88ba-d542-483a-afa2-5de687affd14.5176.2,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:18:31.186 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:18:31.182+0000 7f6ba2a09700 -1 req 5598424130873317542 0.001999960s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-28:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.28,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:18:31.188 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:18:31.184+0000 7fc809eec700 -1 req 18355520900181342209 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:18:31.188 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:18:31.184+0000 7fabb60d9700 -1 req 15578493685887040239 0.000999979s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:18:31.189 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:18:31.184+0000 7faba40b5700 -1 req 8892593813379320395 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-2:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.2,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-2:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.2,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:18:31.190 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:18:31.185+0000 7f6c152ee700 -1 req 14463138408317433219 0.000999980s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-34:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.34,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:18:31.191 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:18:31.185+0000 7f6b26110700 -1 req 2068704070150178726 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-2:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.2,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-2:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.2,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:18:31.191 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:18:31.185+0000 7fc80a6ed700 -1 req 5010872181637399722 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-2:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.2,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-2:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.2,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:18:31.197 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:18:31.196+0000 7f6b2d11e700 -1 req 866902499732545035 0.000999980s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-40:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.40,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-40:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.40,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:18:31.198 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:18:31.196+0000 7fc81a70d700 -1 req 7065404529074780073 0.000999980s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-40:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.40,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-40:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.40,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:18:31.199 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:18:31.195+0000 7fabb00cd700 -1 req 16219627983354051819 0.000999979s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-40:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.40,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-40:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.40,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:18:31.229 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:18:31.226+0000 7fabb38d4700 -1 req 15885317643475685366 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-53:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.4,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-53:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.4,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:18:31.230 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:18:31.227+0000 7fc80aeee700 -1 req 741791556451114748 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-53:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.4,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-53:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.4,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:18:31.231 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:18:31.227+0000 7fab8f88c700 -1 req 8438266540881871608 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:18:31.231 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:18:31.227+0000 7f6b2d91f700 -1 req 5299295734715967124 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-53:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.4,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-53:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.4,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:18:31.232 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:18:31.228+0000 7fc7feed6700 -1 req 2988103857002347540 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:18:31.233 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:18:31.228+0000 7f6b991f6700 -1 req 8670550781364820486 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-9:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.9,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:18:31.238 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:18:31.236+0000 7fab7f06b700 -1 req 675314711401728393 0.000999979s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:18:31.239 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:18:31.237+0000 7fc81af0e700 -1 req 15646037776384138487 0.000999980s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:18:31.240 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:18:31.237+0000 7f6b6318a700 -1 req 8459694723460805717 0.000999980s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-35:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.35,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:18:31.275 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:18:31.272+0000 7fc7e0699700 -1 req 10547963897139314396 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:18:31.275 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:18:31.271+0000 7fab6e84a700 -1 req 4396800740715752907 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:18:31.276 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:18:31.272+0000 7f6c032ca700 -1 req 15325912265413818262 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-7:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.7,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:18:31.286 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:18:31.283+0000 7fab72852700 -1 req 9039237816719307952 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-55:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.6,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-55:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.6,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:18:31.287 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:18:31.284+0000 7fc7dae8e700 -1 req 914466166556170878 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-55:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.6,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-55:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.6,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:18:31.288 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:18:31.284+0000 7f6b67993700 -1 req 3110921813117453988 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-55:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.6,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-55:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.6,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:18:31.345 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'layout', '--bucket', 'dsgvcy-71', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:18:31.347 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'layout', '--bucket', 'dsgvcy-71', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:18:31.348 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket layout --bucket dsgvcy-71 --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T06:18:31.396 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:31.397 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:31.449 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T06:18:31.449 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:18:31.449 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding": "None", 2022-09-03T06:18:31.450 INFO:teuthology.orchestra.run.smithi181.stdout: "current_index": { 2022-09-03T06:18:31.450 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 2, 2022-09-03T06:18:31.450 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:18:31.451 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "Normal", 2022-09-03T06:18:31.451 INFO:teuthology.orchestra.run.smithi181.stdout: "normal": { 2022-09-03T06:18:31.451 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 3, 2022-09-03T06:18:31.451 INFO:teuthology.orchestra.run.smithi181.stdout: "hash_type": "Mod" 2022-09-03T06:18:31.452 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:31.452 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:31.453 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:18:31.453 INFO:teuthology.orchestra.run.smithi181.stdout: "logs": [ 2022-09-03T06:18:31.453 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:18:31.453 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 1, 2022-09-03T06:18:31.454 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:18:31.454 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "InIndex", 2022-09-03T06:18:31.454 INFO:teuthology.orchestra.run.smithi181.stdout: "in_index": { 2022-09-03T06:18:31.455 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 1, 2022-09-03T06:18:31.455 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:18:31.455 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 3, 2022-09-03T06:18:31.455 INFO:teuthology.orchestra.run.smithi181.stdout: "hash_type": "Mod" 2022-09-03T06:18:31.456 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:31.456 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:31.456 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:31.456 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:18:31.457 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:18:31.457 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 2, 2022-09-03T06:18:31.457 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:18:31.457 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "InIndex", 2022-09-03T06:18:31.458 INFO:teuthology.orchestra.run.smithi181.stdout: "in_index": { 2022-09-03T06:18:31.458 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 2, 2022-09-03T06:18:31.459 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:18:31.460 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 3, 2022-09-03T06:18:31.460 INFO:teuthology.orchestra.run.smithi181.stdout: "hash_type": "Mod" 2022-09-03T06:18:31.460 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:31.460 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:31.461 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:31.461 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:31.461 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:18:31.461 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:31.462 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T06:18:31.463 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:18:31.464 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'autotrim', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:18:31.465 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog autotrim --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T06:18:31.510 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:31.511 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:31.560 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:18:31.557+0000 7fab32fd3700 -1 req 749147913743339942 0.000999979s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-1:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.1,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-1:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.1,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:18:31.562 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:18:31.559+0000 7fc7bee56700 -1 req 17814071905140856032 0.001999960s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-1:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.1,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-1:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.1,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:18:31.563 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:18:31.558+0000 7f6c132ea700 -1 req 16404159108818378488 0.000999980s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-1:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.1,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-1:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.1,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:18:31.564 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:18:31.561+0000 7fab56019700 -1 req 12168485178923549445 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-5:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.5,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-5:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.5,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:18:31.565 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:18:31.562+0000 7fc7c7e68700 -1 req 6862753488429559721 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-5:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.5,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-5:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.5,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:18:31.566 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:18:31.563+0000 7f6c102e4700 -1 req 564642966309518537 0.000999980s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-5:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.5,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-5:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.5,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:18:31.575 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:18:31.572+0000 7fab1d7a8700 -1 req 605506631436923165 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-54:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.5,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-54:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.5,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:18:31.575 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:18:31.572+0000 7fab6803d700 -1 req 14254194248591854268 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:18:31.576 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:18:31.573+0000 7f6bf82b4700 -1 req 8382509113996825974 0.000999981s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-54:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.5,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-54:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.5,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:18:31.577 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:18:31.573+0000 7fc7b163b700 -1 req 14359271424288576436 0.000999981s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-54:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.5,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-54:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.5,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:18:31.577 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:18:31.573+0000 7fc7b964b700 -1 req 2464773632524996791 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:18:31.578 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:18:31.573+0000 7f6bc9a57700 -1 req 15935433610292852057 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-31:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.31,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:18:31.618 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:18:31.615+0000 7fab1bfa5700 -1 req 13573168959820167126 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-42:e40d88ba-d542-483a-afa2-5de687affd14.5176.1,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-42:e40d88ba-d542-483a-afa2-5de687affd14.5176.1,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:18:31.619 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:18:31.616+0000 7fab25fb9700 -1 req 6146694345420611778 0.000999979s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-67:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.18,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-67:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.18,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:18:31.620 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:18:31.617+0000 7f6bd426c700 -1 req 12014118935754122754 0.000999980s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-42:e40d88ba-d542-483a-afa2-5de687affd14.5176.1,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-42:e40d88ba-d542-483a-afa2-5de687affd14.5176.1,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:18:31.621 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:18:31.617+0000 7fc79ee16700 -1 req 16582434067242641329 0.000999980s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-42:e40d88ba-d542-483a-afa2-5de687affd14.5176.1,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-42:e40d88ba-d542-483a-afa2-5de687affd14.5176.1,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:18:31.621 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:18:31.617+0000 7fc897006700 -1 req 2842303765505427623 0.000999980s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-67:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.18,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-67:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.18,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:18:31.622 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:18:31.617+0000 7f6bd3a6b700 -1 req 15078695670442587515 0.000999980s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-67:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.18,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-67:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.18,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:18:31.624 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:18:31.622+0000 7faafaf63700 -1 req 10526431064715788338 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-3:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.3,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-3:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.3,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:18:31.625 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:18:31.623+0000 7f6bc4a4d700 -1 req 7600454114499969610 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-3:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.3,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-3:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.3,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:18:31.626 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:18:31.623+0000 7fc892ffe700 -1 req 2124578952425000742 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-3:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.3,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-3:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.3,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:18:31.661 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:18:31.658+0000 7faafd768700 -1 req 7138183398888920424 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-50:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.1,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-50:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.1,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:18:31.662 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:18:31.659+0000 7fc8807d9700 -1 req 6443310885460435891 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-50:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.1,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-50:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.1,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:18:31.663 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:18:31.659+0000 7f6c082d4700 -1 req 126032946830987625 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-50:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.1,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-50:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.1,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:18:31.668 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:18:31.665+0000 7faaf1f51700 -1 req 4923120743241494624 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30,z=6ebb780b-73b6-47d3-ad4f-1dc62076c1f4,az=0}} returned ret=-2 2022-09-03T06:18:31.669 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:18:31.666+0000 7fc87dfd4700 -1 req 13010642099654660630 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30,z=878fc3d4-c5f9-4296-8cc0-f2a8e029bc48,az=0}} returned ret=-2 2022-09-03T06:18:31.669 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:18:31.666+0000 7f6bda278700 -1 req 3501106135467969456 0.000000000s :get_bucket_index_log_status ERROR: rgw_read_bucket_full_sync_status() on pipe={s={b=dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30,z=91786ef1-cdb4-4061-8a3e-2a28cc497285,az=0},d={b=dsgvcy-30:91786ef1-cdb4-4061-8a3e-2a28cc497285.4353.30,z=e40d88ba-d542-483a-afa2-5de687affd14,az=0}} returned ret=-2 2022-09-03T06:18:31.728 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'layout', '--bucket', 'dsgvcy-71', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:18:31.729 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'layout', '--bucket', 'dsgvcy-71', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:18:31.730 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket layout --bucket dsgvcy-71 --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T06:18:31.777 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:31.778 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:31.821 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T06:18:31.821 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:18:31.822 INFO:teuthology.orchestra.run.smithi181.stdout: "resharding": "None", 2022-09-03T06:18:31.822 INFO:teuthology.orchestra.run.smithi181.stdout: "current_index": { 2022-09-03T06:18:31.822 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 2, 2022-09-03T06:18:31.823 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:18:31.823 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "Normal", 2022-09-03T06:18:31.823 INFO:teuthology.orchestra.run.smithi181.stdout: "normal": { 2022-09-03T06:18:31.824 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 3, 2022-09-03T06:18:31.824 INFO:teuthology.orchestra.run.smithi181.stdout: "hash_type": "Mod" 2022-09-03T06:18:31.824 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:31.825 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:31.825 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:18:31.825 INFO:teuthology.orchestra.run.smithi181.stdout: "logs": [ 2022-09-03T06:18:31.826 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:18:31.826 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 1, 2022-09-03T06:18:31.826 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:18:31.830 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "InIndex", 2022-09-03T06:18:31.830 INFO:teuthology.orchestra.run.smithi181.stdout: "in_index": { 2022-09-03T06:18:31.831 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 1, 2022-09-03T06:18:31.832 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:18:31.833 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 3, 2022-09-03T06:18:31.833 INFO:teuthology.orchestra.run.smithi181.stdout: "hash_type": "Mod" 2022-09-03T06:18:31.833 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:31.834 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:31.834 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:31.835 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:18:31.836 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:18:31.836 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 2, 2022-09-03T06:18:31.836 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:18:31.837 INFO:teuthology.orchestra.run.smithi181.stdout: "type": "InIndex", 2022-09-03T06:18:31.837 INFO:teuthology.orchestra.run.smithi181.stdout: "in_index": { 2022-09-03T06:18:31.837 INFO:teuthology.orchestra.run.smithi181.stdout: "gen": 2, 2022-09-03T06:18:31.838 INFO:teuthology.orchestra.run.smithi181.stdout: "layout": { 2022-09-03T06:18:31.838 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 3, 2022-09-03T06:18:31.838 INFO:teuthology.orchestra.run.smithi181.stdout: "hash_type": "Mod" 2022-09-03T06:18:31.839 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:31.839 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:31.839 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:31.839 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:31.840 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:18:31.840 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:31.840 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T06:18:31.857 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_reshard_index_log_trim ... FAIL 2022-09-03T06:18:31.859 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:18:31.859 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:18:31.860 DEBUG:boto:Method: PUT 2022-09-03T06:18:31.861 DEBUG:boto:Path: /dsgvcy-72/ 2022-09-03T06:18:31.861 DEBUG:boto:Data: 2022-09-03T06:18:31.862 DEBUG:boto:Headers: {} 2022-09-03T06:18:31.862 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:31.863 DEBUG:boto:Port: 8000 2022-09-03T06:18:31.864 DEBUG:boto:Params: {} 2022-09-03T06:18:31.865 DEBUG:boto:Token: None 2022-09-03T06:18:31.865 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 06:18:31 GMT /dsgvcy-72/ 2022-09-03T06:18:31.867 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fpxNjJllfLCzh45S5brpeBbe9JY= 2022-09-03T06:18:31.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fpxNjJllfLCzh45S5brpeBbe9JY=', 'Content-Length': '0'} 2022-09-03T06:18:31.883 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000d1b86eabfff56953-006312f1b7-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:18:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:31.884 DEBUG:rgw_multi.tests:created bucket=dsgvcy-72 2022-09-03T06:18:31.885 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:18:31.885 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:18:31.886 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:18:31.936 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:31.937 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:31.980 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T06:18:31.997 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T06:18:31.998 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:18:31.998 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:18:31.998 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:18:31.999 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:18:31.999 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:18:31.999 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:18:31.999 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:18:32.000 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:18:32.000 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:18:32.000 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662185911.881271_450.1", 2022-09-03T06:18:32.001 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:18:31.881271Z" 2022-09-03T06:18:32.001 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:18:32.001 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:18:32.002 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:18:32.002 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:18:32.002 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:32.002 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T06:18:32.004 DEBUG:rgw_multi.tests:master meta markers={0: '', 1: '', 2: '1_1662185911.881271_450.1', 3: ''} 2022-09-03T06:18:32.004 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T06:18:32.005 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:18:32.006 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:18:32.006 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:18:32.066 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:32.067 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:32.097 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:18:32.095+0000 7f2f536f3600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:18:32.098 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:18:32.096+0000 7f2f536f3600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:18:32.112 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:18:32.112 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:18:32.112 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:18:32.112 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:18:32.113 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:18:32.113 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:18:32.113 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:18:32.114 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:18:32.114 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:18:32.114 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:18:32.114 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:18:32.115 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:18:32.115 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:18:32.115 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:18:32.115 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:18:32.116 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:18:32.116 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:18:32.116 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:18:32.117 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:18:32.117 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:18:32.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:18:32.117 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:18:32.118 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:18:32.118 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:18:32.118 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:18:32.118 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:18:32.119 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:18:32.119 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:18:32.119 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:18:32.119 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:18:32.120 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:18:32.120 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:18:32.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:18:32.121 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:18:32.121 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:18:32.121 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:18:32.121 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:18:32.122 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662185911.881271_450.1", 2022-09-03T06:18:32.122 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:18:32.122 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:18:32.123 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:18:32.124 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:18:31.881271Z", 2022-09-03T06:18:32.124 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:18:32.125 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:18:32.125 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:18:32.126 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:18:32.126 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:18:32.127 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:18:32.127 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:18:32.128 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:18:32.128 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:18:32.129 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:18:32.129 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:18:32.129 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:18:32.130 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:18:32.130 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:18:32.131 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:18:32.131 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:18:32.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:18:32.132 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:18:32.132 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:18:32.132 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:18:32.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:18:32.133 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:18:32Z" 2022-09-03T06:18:32.133 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:18:32.135 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185911.881271_450.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:18:31.881271Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:18:32Z" } 2022-09-03T06:18:32.135 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185911.881271_450.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:18:31.881271Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:18:32.136 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185911.881271_450.1', 3: ''} 2022-09-03T06:18:32.137 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185911.881271_450.1', 3: ''} 2022-09-03T06:18:32.138 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T06:18:32.140 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T06:18:32.141 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:18:32.142 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:18:32.144 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:18:32.193 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:32.194 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:32.242 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:18:32.240+0000 7f44b6ba6600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:18:32.243 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:18:32.241+0000 7f44b6ba6600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:18:32.255 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T06:18:32.256 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T06:18:32.256 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T06:18:32.257 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T06:18:32.257 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T06:18:32.257 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:18:32.257 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:18:32.258 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:18:32.258 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T06:18:32.259 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:18:32.259 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T06:18:32.259 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:18:32.260 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:18:32.260 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:18:32.261 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:18:32.261 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T06:18:32.261 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:18:32.261 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:18:32.262 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:18:32.262 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:32.262 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:18:32.263 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:18:32.263 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T06:18:32.263 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:18:32.264 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:18:32.264 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:18:32.265 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:18:32.265 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:18:32.265 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:18:32.266 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T06:18:32.266 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:18:32.266 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:32.267 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:18:32.267 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:18:32.267 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T06:18:32.268 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:18:32.268 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:18:32.268 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662185911.190250_444.1", 2022-09-03T06:18:32.268 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:18:32.269 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:18:32.269 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:18:32.269 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:18:31.190250Z", 2022-09-03T06:18:32.270 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:18:32.270 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:32.270 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:18:32.271 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:18:32.271 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T06:18:32.271 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:18:32.271 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:18:32.272 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:18:32.272 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:18:32.272 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:18:32.272 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:18:32.273 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:18:32.273 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:18:32.273 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:32.274 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:32.274 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:18:32.274 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:18:32.275 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T06:18:32.276 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T06:18:32.276 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T06:18:32.276 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:18:32.276 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T06:18:32Z" 2022-09-03T06:18:32.277 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T06:18:32.279 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185911.190250_444.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:18:31.190250Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:18:32Z" } 2022-09-03T06:18:32.280 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185911.190250_444.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:18:31.190250Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:18:32.281 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185911.881271_450.1', 3: ''} 2022-09-03T06:18:32.281 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185911.190250_444.1', 3: ''} 2022-09-03T06:18:32.282 WARNING:rgw_multi.tests:zone test-zone3 behind master: shard=2 master=1_1662185911.881271_450.1 target=1_1662185911.190250_444.1 2022-09-03T06:18:37.283 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:18:37.285 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:18:37.286 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:18:37.345 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:37.346 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:37.388 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:18:37.385+0000 7fd18778b600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:18:37.391 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:18:37.386+0000 7fd18778b600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:18:37.404 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T06:18:37.405 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T06:18:37.406 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T06:18:37.407 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T06:18:37.407 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T06:18:37.407 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:18:37.408 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:18:37.408 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:18:37.408 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T06:18:37.409 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:18:37.409 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T06:18:37.409 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:18:37.409 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:18:37.410 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:18:37.410 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:18:37.410 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T06:18:37.410 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:18:37.411 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:18:37.411 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:18:37.412 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:37.412 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:18:37.413 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:18:37.413 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T06:18:37.414 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:18:37.414 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:18:37.415 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:18:37.416 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:18:37.418 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:18:37.419 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:18:37.419 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T06:18:37.419 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:18:37.420 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:37.420 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:18:37.420 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:18:37.420 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T06:18:37.421 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:18:37.421 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:18:37.421 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662185911.881271_450.1", 2022-09-03T06:18:37.421 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:18:37.422 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:18:37.422 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:18:37.422 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:18:31.881271Z", 2022-09-03T06:18:37.423 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:18:37.423 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:37.423 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:18:37.423 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:18:37.424 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T06:18:37.424 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:18:37.424 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:18:37.425 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:18:37.425 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:18:37.425 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:18:37.426 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:18:37.426 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:18:37.426 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:18:37.426 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:37.427 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:18:37.427 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:18:37.428 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:18:37.428 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T06:18:37.429 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T06:18:37.429 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T06:18:37.429 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:18:37.430 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T06:18:37Z" 2022-09-03T06:18:37.430 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T06:18:37.431 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185911.881271_450.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:18:31.881271Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:18:37Z" } 2022-09-03T06:18:37.433 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185911.881271_450.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:18:31.881271Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:18:37.433 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185911.881271_450.1', 3: ''} 2022-09-03T06:18:37.434 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185911.881271_450.1', 3: ''} 2022-09-03T06:18:37.435 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T06:18:37.435 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T06:18:37.436 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:18:37.436 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:18:37.437 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:18:37.496 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:37.497 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:37.543 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:18:37.541+0000 7f50cac57600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:18:37.543 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:18:37.542+0000 7f50cac57600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:18:37.566 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:18:37.566 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:18:37.566 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:18:37.567 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:18:37.567 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:18:37.567 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:18:37.567 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:18:37.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:18:37.568 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:18:37.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:18:37.569 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:18:37.569 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:18:37.570 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:18:37.570 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:18:37.570 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:18:37.570 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:18:37.571 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:18:37.571 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:18:37.571 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:18:37.572 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:18:37.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:18:37.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:18:37.572 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:18:37.573 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:18:37.573 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:18:37.574 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:18:37.574 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:18:37.574 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:18:37.575 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:18:37.575 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:18:37.575 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:18:37.576 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:18:37.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:18:37.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:18:37.576 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:18:37.577 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:18:37.577 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:18:37.577 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662185911.881271_450.1", 2022-09-03T06:18:37.577 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:18:37.578 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:18:37.578 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:18:37.578 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:18:31.881271Z", 2022-09-03T06:18:37.579 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:18:37.579 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:18:37.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:18:37.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:18:37.580 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:18:37.581 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:18:37.581 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:18:37.581 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:18:37.581 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:18:37.582 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:18:37.582 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:18:37.582 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:18:37.583 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:18:37.583 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:18:37.584 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:18:37.584 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:18:37.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:18:37.584 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:18:37.585 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:18:37.585 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:18:37.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:18:37.585 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:18:37Z" 2022-09-03T06:18:37.586 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:18:37.586 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185911.881271_450.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:18:31.881271Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:18:37Z" } 2022-09-03T06:18:37.587 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185911.881271_450.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:18:31.881271Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:18:37.587 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662185911.881271_450.1', 3: ''} 2022-09-03T06:18:37.588 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185911.881271_450.1', 3: ''} 2022-09-03T06:18:37.588 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T06:18:37.589 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:18:37.590 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:18:37.591 DEBUG:boto:Method: HEAD 2022-09-03T06:18:37.591 DEBUG:boto:Path: /dsgvcy-72/ 2022-09-03T06:18:37.592 DEBUG:boto:Data: 2022-09-03T06:18:37.592 DEBUG:boto:Headers: {} 2022-09-03T06:18:37.593 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:37.594 DEBUG:boto:Port: 8000 2022-09-03T06:18:37.594 DEBUG:boto:Params: {} 2022-09-03T06:18:37.598 DEBUG:boto:Token: None 2022-09-03T06:18:37.599 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:37 GMT /dsgvcy-72/ 2022-09-03T06:18:37.599 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b6XzcSDqweoIcroijpmDQrEz6jg= 2022-09-03T06:18:37.600 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b6XzcSDqweoIcroijpmDQrEz6jg=', 'Content-Length': '0'} 2022-09-03T06:18:37.605 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000029897f00ddbce731-006312f1bd-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:37.607 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:18:37.616 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:18:37.621 DEBUG:boto:Method: PUT 2022-09-03T06:18:37.621 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:18:37.622 DEBUG:boto:Data: 2022-09-03T06:18:37.623 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:18:37.624 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:37.635 DEBUG:boto:Port: 8000 2022-09-03T06:18:37.643 DEBUG:boto:Params: {} 2022-09-03T06:18:37.646 DEBUG:boto:Token: None 2022-09-03T06:18:37.648 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:18:37 GMT /dsgvcy-72/a 2022-09-03T06:18:37.649 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y/FywmBNYw+3PX1YjMCV5xoP52M= 2022-09-03T06:18:37.650 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:18:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y/FywmBNYw+3PX1YjMCV5xoP52M='} 2022-09-03T06:18:37.663 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000007306aeb7ab5825c5-006312f1bd-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:37.664 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:18:37.669 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:18:37.671 DEBUG:boto:Method: HEAD 2022-09-03T06:18:37.672 DEBUG:boto:Path: /dsgvcy-72/ 2022-09-03T06:18:37.673 DEBUG:boto:Data: 2022-09-03T06:18:37.676 DEBUG:boto:Headers: {} 2022-09-03T06:18:37.677 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:37.678 DEBUG:boto:Port: 8000 2022-09-03T06:18:37.679 DEBUG:boto:Params: {} 2022-09-03T06:18:37.682 DEBUG:boto:Token: None 2022-09-03T06:18:37.683 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:37 GMT /dsgvcy-72/ 2022-09-03T06:18:37.684 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b6XzcSDqweoIcroijpmDQrEz6jg= 2022-09-03T06:18:37.686 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b6XzcSDqweoIcroijpmDQrEz6jg=', 'Content-Length': '0'} 2022-09-03T06:18:37.693 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '3'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006f5d519871bc5a0e-006312f1bd-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:37.695 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:18:37.696 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:18:37.696 DEBUG:boto:Method: PUT 2022-09-03T06:18:37.697 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:18:37.698 DEBUG:boto:Data: 2022-09-03T06:18:37.698 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:18:37.703 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:37.704 DEBUG:boto:Port: 8000 2022-09-03T06:18:37.705 DEBUG:boto:Params: {} 2022-09-03T06:18:37.706 DEBUG:boto:Token: None 2022-09-03T06:18:37.707 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:18:37 GMT /dsgvcy-72/b 2022-09-03T06:18:37.708 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EMAGscp+aB3bDcWQk3MgoJlk8rE= 2022-09-03T06:18:37.709 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:18:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EMAGscp+aB3bDcWQk3MgoJlk8rE='} 2022-09-03T06:18:37.718 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f22c774441e651d8-006312f1bd-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:37.720 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:18:37.721 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:18:37.722 DEBUG:boto:Method: HEAD 2022-09-03T06:18:37.723 DEBUG:boto:Path: /dsgvcy-72/ 2022-09-03T06:18:37.725 DEBUG:boto:Data: 2022-09-03T06:18:37.725 DEBUG:boto:Headers: {} 2022-09-03T06:18:37.726 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:37.727 DEBUG:boto:Port: 8000 2022-09-03T06:18:37.728 DEBUG:boto:Params: {} 2022-09-03T06:18:37.729 DEBUG:boto:Token: None 2022-09-03T06:18:37.731 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:37 GMT /dsgvcy-72/ 2022-09-03T06:18:37.732 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b6XzcSDqweoIcroijpmDQrEz6jg= 2022-09-03T06:18:37.733 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b6XzcSDqweoIcroijpmDQrEz6jg=', 'Content-Length': '0'} 2022-09-03T06:18:37.738 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ccdeca6ab9fe64ea-006312f1bd-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:37.740 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:18:37.741 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:18:37.743 DEBUG:boto:Method: PUT 2022-09-03T06:18:37.744 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:18:37.746 DEBUG:boto:Data: 2022-09-03T06:18:37.746 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:18:37.747 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:37.747 DEBUG:boto:Port: 8000 2022-09-03T06:18:37.748 DEBUG:boto:Params: {} 2022-09-03T06:18:37.748 DEBUG:boto:Token: None 2022-09-03T06:18:37.749 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:18:37 GMT /dsgvcy-72/c 2022-09-03T06:18:37.749 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xZnay58VKMz7lP7vzP386Nm5nuc= 2022-09-03T06:18:37.751 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:18:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xZnay58VKMz7lP7vzP386Nm5nuc='} 2022-09-03T06:18:37.761 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000eaa0e29d267b9b2f-006312f1bd-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:37.764 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:18:37.765 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:18:37.766 DEBUG:boto:Method: HEAD 2022-09-03T06:18:37.767 DEBUG:boto:Path: /dsgvcy-72/ 2022-09-03T06:18:37.768 DEBUG:boto:Data: 2022-09-03T06:18:37.769 DEBUG:boto:Headers: {} 2022-09-03T06:18:37.770 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:37.771 DEBUG:boto:Port: 8000 2022-09-03T06:18:37.772 DEBUG:boto:Params: {} 2022-09-03T06:18:37.773 DEBUG:boto:Token: None 2022-09-03T06:18:37.775 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:18:37 GMT /dsgvcy-72/ 2022-09-03T06:18:37.776 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b6XzcSDqweoIcroijpmDQrEz6jg= 2022-09-03T06:18:37.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:18:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b6XzcSDqweoIcroijpmDQrEz6jg=', 'Content-Length': '0'} 2022-09-03T06:18:37.781 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '9'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000781363b9a1a4c52d-006312f1bd-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:37.782 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:18:37.783 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:18:37.784 DEBUG:boto:Method: PUT 2022-09-03T06:18:37.786 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:18:37.787 DEBUG:boto:Data: 2022-09-03T06:18:37.788 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:18:37.789 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:18:37.790 DEBUG:boto:Port: 8000 2022-09-03T06:18:37.791 DEBUG:boto:Params: {} 2022-09-03T06:18:37.792 DEBUG:boto:Token: None 2022-09-03T06:18:37.793 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:18:37 GMT /dsgvcy-72/d 2022-09-03T06:18:37.793 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j/bcu8xh8aEXUdWDM0dcTkXLpZs= 2022-09-03T06:18:37.794 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:18:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j/bcu8xh8aEXUdWDM0dcTkXLpZs='} 2022-09-03T06:18:37.842 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004187ed277fa70dbb-006312f1bd-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:18:37.843 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-72 2022-09-03T06:18:37.844 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:37.844 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:37.845 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-72 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:18:37.903 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:37.903 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:37.950 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:18:37.940+0000 7f900057a600 1 waiting to reach incremental sync.. 2022-09-03T06:18:42.944 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:18:42.942+0000 7f900057a600 1 bucket sync caught up with source: 2022-09-03T06:18:42.945 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.50.5, , , , 00000000001.39.5, , , 00000000001.75.5, , 00000000001.57.5] 2022-09-03T06:18:42.946 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.50.5, , , , 00000000001.39.5, , , 00000000001.75.5, , 00000000001.57.5] 2022-09-03T06:18:42.946 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:18:42.942+0000 7f900057a600 0 bucket checkpoint complete 2022-09-03T06:18:42.964 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-72 2022-09-03T06:18:42.966 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:42.967 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:42.968 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-72 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:18:43.035 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:43.036 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:43.074 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:18:43.071+0000 7fc39caa2600 1 bucket sync caught up with source: 2022-09-03T06:18:43.074 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.50.5, , , , 00000000001.39.5, , , 00000000001.75.5, , 00000000001.57.5] 2022-09-03T06:18:43.075 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.50.5, , , , 00000000001.39.5, , , 00000000001.75.5, , 00000000001.57.5] 2022-09-03T06:18:43.075 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:18:43.071+0000 7fc39caa2600 0 bucket checkpoint complete 2022-09-03T06:18:43.095 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-72 2022-09-03T06:18:43.095 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:43.096 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:18:43.097 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-72 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:18:43.163 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:18:43.164 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:18:43.214 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:18:43.211+0000 7efc89f69600 1 waiting for incremental sync to catch up: 2022-09-03T06:18:43.214 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.75.5, , 00000000001.57.5] 2022-09-03T06:18:43.215 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.50.5, , , , 00000000001.39.5, , , 00000000001.75.5, , 00000000001.57.5] 2022-09-03T06:18:48.215 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:18:48.213+0000 7efc89f69600 1 waiting for incremental sync to catch up: 2022-09-03T06:18:48.215 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.75.5, , 00000000001.57.5] 2022-09-03T06:18:48.216 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.50.5, , , , 00000000001.39.5, , , 00000000001.75.5, , 00000000001.57.5] 2022-09-03T06:18:53.216 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:18:53.214+0000 7efc89f69600 1 waiting for incremental sync to catch up: 2022-09-03T06:18:53.217 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.75.5, , 00000000001.57.5] 2022-09-03T06:18:53.217 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.50.5, , , , 00000000001.39.5, , , 00000000001.75.5, , 00000000001.57.5] 2022-09-03T06:18:58.219 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:18:58.216+0000 7efc89f69600 1 waiting for incremental sync to catch up: 2022-09-03T06:18:58.219 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.75.5, , 00000000001.57.5] 2022-09-03T06:18:58.219 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.50.5, , , , 00000000001.39.5, , , 00000000001.75.5, , 00000000001.57.5] 2022-09-03T06:19:03.222 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:19:03.217+0000 7efc89f69600 1 waiting for incremental sync to catch up: 2022-09-03T06:19:03.223 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.75.5, , 00000000001.57.5] 2022-09-03T06:19:03.224 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.50.5, , , , 00000000001.39.5, , , 00000000001.75.5, , 00000000001.57.5] 2022-09-03T06:19:08.221 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:19:08.218+0000 7efc89f69600 1 waiting for incremental sync to catch up: 2022-09-03T06:19:08.221 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.75.5, , 00000000001.57.5] 2022-09-03T06:19:08.221 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.50.5, , , , 00000000001.39.5, , , 00000000001.75.5, , 00000000001.57.5] 2022-09-03T06:19:13.222 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:19:13.219+0000 7efc89f69600 1 waiting for incremental sync to catch up: 2022-09-03T06:19:13.223 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.75.5, , 00000000001.57.5] 2022-09-03T06:19:13.224 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.50.5, , , , 00000000001.39.5, , , 00000000001.75.5, , 00000000001.57.5] 2022-09-03T06:19:18.226 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:19:18.221+0000 7efc89f69600 1 waiting for incremental sync to catch up: 2022-09-03T06:19:18.227 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.75.5, , 00000000001.57.5] 2022-09-03T06:19:18.227 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.50.5, , , , 00000000001.39.5, , , 00000000001.75.5, , 00000000001.57.5] 2022-09-03T06:19:23.226 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:19:23.223+0000 7efc89f69600 1 bucket sync caught up with source: 2022-09-03T06:19:23.228 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.50.5, , , , 00000000001.39.5, , , 00000000001.75.5, , 00000000001.57.5] 2022-09-03T06:19:23.228 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.50.5, , , , 00000000001.39.5, , , 00000000001.75.5, , 00000000001.57.5] 2022-09-03T06:19:23.228 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:19:23.223+0000 7efc89f69600 0 bucket checkpoint complete 2022-09-03T06:19:23.247 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-72 2022-09-03T06:19:23.248 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:19:23.249 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:19:23.251 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-72 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:19:23.317 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:19:23.317 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:19:23.358 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:19:23.355+0000 7fdc39d7f600 1 bucket sync caught up with source: 2022-09-03T06:19:23.359 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.184.5, , , , 00000000001.569.5, , , 00000000001.385.5, , 00000000001.304.5] 2022-09-03T06:19:23.359 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.184.5, , , , 00000000001.569.5, , , 00000000001.385.5, , 00000000001.304.5] 2022-09-03T06:19:23.360 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:19:23.355+0000 7fdc39d7f600 0 bucket checkpoint complete 2022-09-03T06:19:23.377 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-72 2022-09-03T06:19:23.379 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:19:23.380 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:19:23.384 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-72 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:19:23.444 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:19:23.445 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:19:23.506 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:19:23.500+0000 7f82b05d7600 1 bucket sync caught up with source: 2022-09-03T06:19:23.507 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.184.5, , , , 00000000001.569.5, , , 00000000001.385.5, , 00000000001.304.5] 2022-09-03T06:19:23.507 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.184.5, , , , 00000000001.569.5, , , 00000000001.385.5, , 00000000001.304.5] 2022-09-03T06:19:23.507 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:19:23.500+0000 7f82b05d7600 0 bucket checkpoint complete 2022-09-03T06:19:23.522 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-72 2022-09-03T06:19:23.523 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-72 2022-09-03T06:19:23.524 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:19:23.524 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:19:23.525 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-72 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:19:23.582 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:19:23.583 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:19:23.630 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:19:23.627+0000 7f113e2c2600 1 bucket sync caught up with source: 2022-09-03T06:19:23.631 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.192.5, , , , 00000000001.183.5, , , 00000000001.295.5, , 00000000001.259.5] 2022-09-03T06:19:23.631 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.192.5, , , , 00000000001.183.5, , , 00000000001.295.5, , 00000000001.259.5] 2022-09-03T06:19:23.631 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:19:23.627+0000 7f113e2c2600 0 bucket checkpoint complete 2022-09-03T06:19:23.644 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-72 2022-09-03T06:19:23.645 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:19:23.645 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:19:23.646 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-72 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:19:23.710 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:19:23.711 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:19:23.756 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:19:23.753+0000 7fc169d22600 1 bucket sync caught up with source: 2022-09-03T06:19:23.757 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.192.5, , , , 00000000001.183.5, , , 00000000001.295.5, , 00000000001.259.5] 2022-09-03T06:19:23.757 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.192.5, , , , 00000000001.183.5, , , 00000000001.295.5, , 00000000001.259.5] 2022-09-03T06:19:23.757 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:19:23.753+0000 7fc169d22600 0 bucket checkpoint complete 2022-09-03T06:19:23.773 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-72 2022-09-03T06:19:23.774 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-72 zones={test-zone2, test-zone1} 2022-09-03T06:19:23.775 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:23.776 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:23.781 DEBUG:boto:Method: HEAD 2022-09-03T06:19:23.783 DEBUG:boto:Path: /dsgvcy-72/ 2022-09-03T06:19:23.784 DEBUG:boto:Data: 2022-09-03T06:19:23.784 DEBUG:boto:Headers: {} 2022-09-03T06:19:23.785 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:23.786 DEBUG:boto:Port: 8000 2022-09-03T06:19:23.786 DEBUG:boto:Params: {} 2022-09-03T06:19:23.787 DEBUG:boto:Token: None 2022-09-03T06:19:23.788 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:23 GMT /dsgvcy-72/ 2022-09-03T06:19:23.789 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YccXoA62iH4qT2xsbnCKdHGMVk4= 2022-09-03T06:19:23.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YccXoA62iH4qT2xsbnCKdHGMVk4=', 'Content-Length': '0'} 2022-09-03T06:19:23.793 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000dc5131d5bbfc5c56-006312f1eb-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:19:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:23.793 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:23.794 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:23.794 DEBUG:boto:Method: HEAD 2022-09-03T06:19:23.795 DEBUG:boto:Path: /dsgvcy-72/ 2022-09-03T06:19:23.796 DEBUG:boto:Data: 2022-09-03T06:19:23.796 DEBUG:boto:Headers: {} 2022-09-03T06:19:23.797 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:23.797 DEBUG:boto:Port: 8000 2022-09-03T06:19:23.798 DEBUG:boto:Params: {} 2022-09-03T06:19:23.799 DEBUG:boto:Token: None 2022-09-03T06:19:23.799 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:23 GMT /dsgvcy-72/ 2022-09-03T06:19:23.800 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YccXoA62iH4qT2xsbnCKdHGMVk4= 2022-09-03T06:19:23.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YccXoA62iH4qT2xsbnCKdHGMVk4=', 'Content-Length': '0'} 2022-09-03T06:19:23.803 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007ee5ce3b46f7da1b-006312f1eb-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:19:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:23.805 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:19:23.806 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:23.807 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:23.808 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:23.809 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:23.809 DEBUG:boto:Method: GET 2022-09-03T06:19:23.810 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:23.810 DEBUG:boto:Data: 2022-09-03T06:19:23.811 DEBUG:boto:Headers: {} 2022-09-03T06:19:23.812 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:23.813 DEBUG:boto:Port: 8000 2022-09-03T06:19:23.813 DEBUG:boto:Params: {} 2022-09-03T06:19:23.814 DEBUG:boto:Token: None 2022-09-03T06:19:23.816 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:23 GMT /dsgvcy-72/?versions 2022-09-03T06:19:23.817 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bX8DFWSVugV7JCTUvWJcepteH/Q= 2022-09-03T06:19:23.818 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bX8DFWSVugV7JCTUvWJcepteH/Q=', 'Content-Length': '0'} 2022-09-03T06:19:23.821 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000055367b9b8301ab73-006312f1eb-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:23.821 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:23.823 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:19:23.824 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:19:23.824 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:19:23.824 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:19:23.825 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:19:23.826 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:23.826 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:23.827 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:23.828 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:23.828 DEBUG:boto:Method: GET 2022-09-03T06:19:23.829 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:23.829 DEBUG:boto:Data: 2022-09-03T06:19:23.830 DEBUG:boto:Headers: {} 2022-09-03T06:19:23.830 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:23.831 DEBUG:boto:Port: 8000 2022-09-03T06:19:23.831 DEBUG:boto:Params: {} 2022-09-03T06:19:23.832 DEBUG:boto:Token: None 2022-09-03T06:19:23.833 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:23 GMT /dsgvcy-72/?versions 2022-09-03T06:19:23.833 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bX8DFWSVugV7JCTUvWJcepteH/Q= 2022-09-03T06:19:23.834 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bX8DFWSVugV7JCTUvWJcepteH/Q=', 'Content-Length': '0'} 2022-09-03T06:19:23.836 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000351cde9d20436a28-006312f1eb-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:23.837 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:23.838 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:19:23.839 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:19:23.839 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:19:23.840 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:19:23.840 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:23.841 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:23.841 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:23.842 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:23.843 DEBUG:boto:Method: GET 2022-09-03T06:19:23.843 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:23.844 DEBUG:boto:Data: 2022-09-03T06:19:23.844 DEBUG:boto:Headers: {} 2022-09-03T06:19:23.845 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:23.846 DEBUG:boto:Port: 8000 2022-09-03T06:19:23.846 DEBUG:boto:Params: {} 2022-09-03T06:19:23.847 DEBUG:boto:Token: None 2022-09-03T06:19:23.847 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:23 GMT /dsgvcy-72/?versions 2022-09-03T06:19:23.848 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bX8DFWSVugV7JCTUvWJcepteH/Q= 2022-09-03T06:19:23.849 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bX8DFWSVugV7JCTUvWJcepteH/Q=', 'Content-Length': '0'} 2022-09-03T06:19:23.851 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000037de926e8b459c0b-006312f1eb-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:23.852 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:23.853 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:23.854 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:23.855 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:23.855 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:23.856 DEBUG:boto:Method: GET 2022-09-03T06:19:23.856 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:23.857 DEBUG:boto:Data: 2022-09-03T06:19:23.857 DEBUG:boto:Headers: {} 2022-09-03T06:19:23.858 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:23.858 DEBUG:boto:Port: 8000 2022-09-03T06:19:23.859 DEBUG:boto:Params: {} 2022-09-03T06:19:23.859 DEBUG:boto:Token: None 2022-09-03T06:19:23.860 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:23 GMT /dsgvcy-72/?versions 2022-09-03T06:19:23.861 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bX8DFWSVugV7JCTUvWJcepteH/Q= 2022-09-03T06:19:23.861 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bX8DFWSVugV7JCTUvWJcepteH/Q=', 'Content-Length': '0'} 2022-09-03T06:19:23.865 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d1148324ef0d3ea5-006312f1eb-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:23.867 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:23.869 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:19:23.870 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:23.871 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:23.872 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.872 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.873 DEBUG:boto:Method: GET 2022-09-03T06:19:23.874 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.875 DEBUG:boto:Data: 2022-09-03T06:19:23.876 DEBUG:boto:Headers: {} 2022-09-03T06:19:23.876 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:23.878 DEBUG:boto:Port: 8000 2022-09-03T06:19:23.879 DEBUG:boto:Params: {} 2022-09-03T06:19:23.880 DEBUG:boto:Token: None 2022-09-03T06:19:23.880 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:23 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.882 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zqO0fkvOFcqual449NG0L01U6oA= 2022-09-03T06:19:23.883 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zqO0fkvOFcqual449NG0L01U6oA=', 'Content-Length': '0'} 2022-09-03T06:19:23.886 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ea9dd2cc94815767-006312f1eb-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:23.888 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:23.888 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:23.889 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.890 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.891 DEBUG:boto:Method: GET 2022-09-03T06:19:23.892 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.892 DEBUG:boto:Data: 2022-09-03T06:19:23.893 DEBUG:boto:Headers: {} 2022-09-03T06:19:23.894 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:23.896 DEBUG:boto:Port: 8000 2022-09-03T06:19:23.897 DEBUG:boto:Params: {} 2022-09-03T06:19:23.899 DEBUG:boto:Token: None 2022-09-03T06:19:23.900 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:23 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.901 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zqO0fkvOFcqual449NG0L01U6oA= 2022-09-03T06:19:23.903 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zqO0fkvOFcqual449NG0L01U6oA=', 'Content-Length': '0'} 2022-09-03T06:19:23.906 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006f1a23217d8a0663-006312f1eb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:23.907 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:23.908 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:23.909 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.909 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.910 DEBUG:boto:Method: HEAD 2022-09-03T06:19:23.910 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.911 DEBUG:boto:Data: 2022-09-03T06:19:23.911 DEBUG:boto:Headers: {} 2022-09-03T06:19:23.912 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:23.913 DEBUG:boto:Port: 8000 2022-09-03T06:19:23.914 DEBUG:boto:Params: {} 2022-09-03T06:19:23.916 DEBUG:boto:Token: None 2022-09-03T06:19:23.918 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:23 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.919 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C/Bcu0fW2rEpCHhWv3XJW4nkyNI= 2022-09-03T06:19:23.919 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C/Bcu0fW2rEpCHhWv3XJW4nkyNI=', 'Content-Length': '0'} 2022-09-03T06:19:23.921 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aa8b01aeee0dc5b5-006312f1eb-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:23.922 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:23.923 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:23.923 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.924 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.924 DEBUG:boto:Method: HEAD 2022-09-03T06:19:23.925 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.926 DEBUG:boto:Data: 2022-09-03T06:19:23.926 DEBUG:boto:Headers: {} 2022-09-03T06:19:23.927 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:23.927 DEBUG:boto:Port: 8000 2022-09-03T06:19:23.928 DEBUG:boto:Params: {} 2022-09-03T06:19:23.929 DEBUG:boto:Token: None 2022-09-03T06:19:23.930 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:23 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.930 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C/Bcu0fW2rEpCHhWv3XJW4nkyNI= 2022-09-03T06:19:23.931 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C/Bcu0fW2rEpCHhWv3XJW4nkyNI=', 'Content-Length': '0'} 2022-09-03T06:19:23.933 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000df14c4261c63dbf1-006312f1eb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:23.934 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:19:23.935 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:23.936 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:23.937 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.938 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.939 DEBUG:boto:Method: GET 2022-09-03T06:19:23.940 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.941 DEBUG:boto:Data: 2022-09-03T06:19:23.943 DEBUG:boto:Headers: {} 2022-09-03T06:19:23.944 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:23.945 DEBUG:boto:Port: 8000 2022-09-03T06:19:23.946 DEBUG:boto:Params: {} 2022-09-03T06:19:23.947 DEBUG:boto:Token: None 2022-09-03T06:19:23.948 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:23 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.950 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zqO0fkvOFcqual449NG0L01U6oA= 2022-09-03T06:19:23.951 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zqO0fkvOFcqual449NG0L01U6oA=', 'Content-Length': '0'} 2022-09-03T06:19:23.954 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009740ef86f359b8b6-006312f1eb-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:23.955 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:23.956 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:23.956 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.957 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.957 DEBUG:boto:Method: GET 2022-09-03T06:19:23.958 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.959 DEBUG:boto:Data: 2022-09-03T06:19:23.960 DEBUG:boto:Headers: {} 2022-09-03T06:19:23.961 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:23.962 DEBUG:boto:Port: 8000 2022-09-03T06:19:23.963 DEBUG:boto:Params: {} 2022-09-03T06:19:23.964 DEBUG:boto:Token: None 2022-09-03T06:19:23.965 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:23 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:19:23.966 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zqO0fkvOFcqual449NG0L01U6oA= 2022-09-03T06:19:23.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zqO0fkvOFcqual449NG0L01U6oA=', 'Content-Length': '0'} 2022-09-03T06:19:23.976 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009b08d093534a2fb4-006312f1eb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:23.978 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:23.978 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:23.979 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:23.980 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:23.981 DEBUG:boto:Method: GET 2022-09-03T06:19:23.982 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:19:23.983 DEBUG:boto:Data: 2022-09-03T06:19:23.984 DEBUG:boto:Headers: {} 2022-09-03T06:19:23.985 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:23.986 DEBUG:boto:Port: 8000 2022-09-03T06:19:23.987 DEBUG:boto:Params: {} 2022-09-03T06:19:23.988 DEBUG:boto:Token: None 2022-09-03T06:19:23.990 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:23 GMT /dsgvcy-72/ 2022-09-03T06:19:23.991 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1Dl+heIazO7hvXHXmA0qjV2egvI= 2022-09-03T06:19:23.994 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1Dl+heIazO7hvXHXmA0qjV2egvI=', 'Content-Length': '0'} 2022-09-03T06:19:24.000 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000000ed98c359da33f7-006312f1eb-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.002 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:24.005 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:24.006 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:24.007 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:24.008 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:24.009 DEBUG:boto:Method: GET 2022-09-03T06:19:24.010 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:19:24.011 DEBUG:boto:Data: 2022-09-03T06:19:24.012 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.013 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.014 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.014 DEBUG:boto:Params: {} 2022-09-03T06:19:24.016 DEBUG:boto:Token: None 2022-09-03T06:19:24.017 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/ 2022-09-03T06:19:24.018 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M= 2022-09-03T06:19:24.018 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M=', 'Content-Length': '0'} 2022-09-03T06:19:24.022 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f76f6f0b0c0c8a1e-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.023 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:24.024 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:24.024 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:24.025 DEBUG:boto:Method: HEAD 2022-09-03T06:19:24.025 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:19:24.026 DEBUG:boto:Data: 2022-09-03T06:19:24.026 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.027 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.027 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.028 DEBUG:boto:Params: {} 2022-09-03T06:19:24.028 DEBUG:boto:Token: None 2022-09-03T06:19:24.029 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/a 2022-09-03T06:19:24.029 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w8H069HmUYmb2E3AUFy+HMTxrwE= 2022-09-03T06:19:24.030 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w8H069HmUYmb2E3AUFy+HMTxrwE=', 'Content-Length': '0'} 2022-09-03T06:19:24.032 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000801935d56dd7dd81-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.033 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:24.034 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:24.034 DEBUG:boto:Method: HEAD 2022-09-03T06:19:24.035 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:19:24.036 DEBUG:boto:Data: 2022-09-03T06:19:24.038 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.039 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.039 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.040 DEBUG:boto:Params: {} 2022-09-03T06:19:24.041 DEBUG:boto:Token: None 2022-09-03T06:19:24.042 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/a 2022-09-03T06:19:24.043 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w8H069HmUYmb2E3AUFy+HMTxrwE= 2022-09-03T06:19:24.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w8H069HmUYmb2E3AUFy+HMTxrwE=', 'Content-Length': '0'} 2022-09-03T06:19:24.046 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000701fe87abd319e29-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.048 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:19:24.048 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:24.049 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:24.049 DEBUG:boto:Method: GET 2022-09-03T06:19:24.050 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:19:24.051 DEBUG:boto:Data: 2022-09-03T06:19:24.052 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.052 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.053 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.053 DEBUG:boto:Params: {} 2022-09-03T06:19:24.054 DEBUG:boto:Token: None 2022-09-03T06:19:24.055 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/a 2022-09-03T06:19:24.056 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HLn2xfInw559cHBb6pOydIcTOcs= 2022-09-03T06:19:24.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HLn2xfInw559cHBb6pOydIcTOcs=', 'Content-Length': '0'} 2022-09-03T06:19:24.058 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fc4dd7ea44013e38-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.059 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:24.060 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:24.061 DEBUG:boto:Method: GET 2022-09-03T06:19:24.062 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:19:24.062 DEBUG:boto:Data: 2022-09-03T06:19:24.063 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.063 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.064 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.064 DEBUG:boto:Params: {} 2022-09-03T06:19:24.065 DEBUG:boto:Token: None 2022-09-03T06:19:24.066 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/a 2022-09-03T06:19:24.066 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HLn2xfInw559cHBb6pOydIcTOcs= 2022-09-03T06:19:24.067 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HLn2xfInw559cHBb6pOydIcTOcs=', 'Content-Length': '0'} 2022-09-03T06:19:24.069 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000500e97169b89471-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.110 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:24.111 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:24.112 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:24.112 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:24.113 DEBUG:boto:Method: GET 2022-09-03T06:19:24.113 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:19:24.114 DEBUG:boto:Data: 2022-09-03T06:19:24.114 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.115 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.115 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.116 DEBUG:boto:Params: {} 2022-09-03T06:19:24.117 DEBUG:boto:Token: None 2022-09-03T06:19:24.117 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/ 2022-09-03T06:19:24.118 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M= 2022-09-03T06:19:24.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M=', 'Content-Length': '0'} 2022-09-03T06:19:24.121 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005b75ec82481e89a2-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.123 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:24.124 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:24.125 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:24.125 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:24.126 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:24.127 DEBUG:boto:Method: GET 2022-09-03T06:19:24.128 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:19:24.129 DEBUG:boto:Data: 2022-09-03T06:19:24.129 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.130 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.131 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.131 DEBUG:boto:Params: {} 2022-09-03T06:19:24.132 DEBUG:boto:Token: None 2022-09-03T06:19:24.132 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/ 2022-09-03T06:19:24.133 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M= 2022-09-03T06:19:24.134 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M=', 'Content-Length': '0'} 2022-09-03T06:19:24.137 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000180bb6a671da59f1-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.138 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:24.140 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:19:24.141 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:24.142 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:24.143 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.144 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.145 DEBUG:boto:Method: GET 2022-09-03T06:19:24.146 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.147 DEBUG:boto:Data: 2022-09-03T06:19:24.147 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.148 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.148 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.149 DEBUG:boto:Params: {} 2022-09-03T06:19:24.149 DEBUG:boto:Token: None 2022-09-03T06:19:24.150 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.151 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hx4cWh/yLQos//+o4zLxndTaCmM= 2022-09-03T06:19:24.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hx4cWh/yLQos//+o4zLxndTaCmM=', 'Content-Length': '0'} 2022-09-03T06:19:24.154 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000020f08a179c329660-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.155 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:24.156 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:24.156 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.158 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.158 DEBUG:boto:Method: GET 2022-09-03T06:19:24.159 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.160 DEBUG:boto:Data: 2022-09-03T06:19:24.160 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.161 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.162 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.163 DEBUG:boto:Params: {} 2022-09-03T06:19:24.163 DEBUG:boto:Token: None 2022-09-03T06:19:24.164 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.165 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hx4cWh/yLQos//+o4zLxndTaCmM= 2022-09-03T06:19:24.166 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hx4cWh/yLQos//+o4zLxndTaCmM=', 'Content-Length': '0'} 2022-09-03T06:19:24.169 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000956b8b3cb958db71-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.170 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:24.171 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:24.172 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.172 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.173 DEBUG:boto:Method: HEAD 2022-09-03T06:19:24.175 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.176 DEBUG:boto:Data: 2022-09-03T06:19:24.177 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.177 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.178 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.179 DEBUG:boto:Params: {} 2022-09-03T06:19:24.180 DEBUG:boto:Token: None 2022-09-03T06:19:24.181 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.182 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SbJIbLsZPclG5UaNgVC3ffUWNZ0= 2022-09-03T06:19:24.183 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SbJIbLsZPclG5UaNgVC3ffUWNZ0=', 'Content-Length': '0'} 2022-09-03T06:19:24.185 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000043d5f28416f2f255-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.186 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:24.186 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:24.187 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.188 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.189 DEBUG:boto:Method: HEAD 2022-09-03T06:19:24.189 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.190 DEBUG:boto:Data: 2022-09-03T06:19:24.190 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.191 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.192 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.192 DEBUG:boto:Params: {} 2022-09-03T06:19:24.193 DEBUG:boto:Token: None 2022-09-03T06:19:24.194 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.194 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SbJIbLsZPclG5UaNgVC3ffUWNZ0= 2022-09-03T06:19:24.195 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SbJIbLsZPclG5UaNgVC3ffUWNZ0=', 'Content-Length': '0'} 2022-09-03T06:19:24.197 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003f115f1d9eb29ca9-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.198 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:19:24.199 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:24.200 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:24.201 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.201 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.202 DEBUG:boto:Method: GET 2022-09-03T06:19:24.203 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.204 DEBUG:boto:Data: 2022-09-03T06:19:24.204 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.205 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.205 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.206 DEBUG:boto:Params: {} 2022-09-03T06:19:24.207 DEBUG:boto:Token: None 2022-09-03T06:19:24.207 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.208 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hx4cWh/yLQos//+o4zLxndTaCmM= 2022-09-03T06:19:24.209 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hx4cWh/yLQos//+o4zLxndTaCmM=', 'Content-Length': '0'} 2022-09-03T06:19:24.212 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000003da92d8765381af-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.258 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:24.259 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:24.260 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.260 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.261 DEBUG:boto:Method: GET 2022-09-03T06:19:24.262 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.263 DEBUG:boto:Data: 2022-09-03T06:19:24.264 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.264 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.265 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.265 DEBUG:boto:Params: {} 2022-09-03T06:19:24.266 DEBUG:boto:Token: None 2022-09-03T06:19:24.267 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:19:24.270 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hx4cWh/yLQos//+o4zLxndTaCmM= 2022-09-03T06:19:24.271 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hx4cWh/yLQos//+o4zLxndTaCmM=', 'Content-Length': '0'} 2022-09-03T06:19:24.273 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cd31a2a2b6958d6f-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.279 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:24.280 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:24.281 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:24.282 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:24.284 DEBUG:boto:Method: GET 2022-09-03T06:19:24.285 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:19:24.286 DEBUG:boto:Data: 2022-09-03T06:19:24.287 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.288 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.289 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.290 DEBUG:boto:Params: {} 2022-09-03T06:19:24.291 DEBUG:boto:Token: None 2022-09-03T06:19:24.291 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/ 2022-09-03T06:19:24.292 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M= 2022-09-03T06:19:24.293 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M=', 'Content-Length': '0'} 2022-09-03T06:19:24.295 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bb93f09eece7b88e-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.296 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:24.297 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:24.297 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:24.298 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:24.299 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:24.299 DEBUG:boto:Method: GET 2022-09-03T06:19:24.300 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:19:24.301 DEBUG:boto:Data: 2022-09-03T06:19:24.301 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.302 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.302 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.303 DEBUG:boto:Params: {} 2022-09-03T06:19:24.304 DEBUG:boto:Token: None 2022-09-03T06:19:24.306 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/ 2022-09-03T06:19:24.307 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M= 2022-09-03T06:19:24.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M=', 'Content-Length': '0'} 2022-09-03T06:19:24.311 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005a6d8ce432c50ae7-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.313 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:24.314 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:24.314 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:24.315 DEBUG:boto:Method: HEAD 2022-09-03T06:19:24.315 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:19:24.316 DEBUG:boto:Data: 2022-09-03T06:19:24.317 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.317 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.318 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.319 DEBUG:boto:Params: {} 2022-09-03T06:19:24.320 DEBUG:boto:Token: None 2022-09-03T06:19:24.320 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/b 2022-09-03T06:19:24.321 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GITwFOniCagM8yYb9swtCsEdLuQ= 2022-09-03T06:19:24.323 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GITwFOniCagM8yYb9swtCsEdLuQ=', 'Content-Length': '0'} 2022-09-03T06:19:24.327 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000da7dd0ef0a4c22b0-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.328 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:24.329 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:24.329 DEBUG:boto:Method: HEAD 2022-09-03T06:19:24.330 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:19:24.331 DEBUG:boto:Data: 2022-09-03T06:19:24.331 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.332 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.332 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.333 DEBUG:boto:Params: {} 2022-09-03T06:19:24.333 DEBUG:boto:Token: None 2022-09-03T06:19:24.334 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/b 2022-09-03T06:19:24.334 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GITwFOniCagM8yYb9swtCsEdLuQ= 2022-09-03T06:19:24.335 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GITwFOniCagM8yYb9swtCsEdLuQ=', 'Content-Length': '0'} 2022-09-03T06:19:24.337 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ea8cd4055d37dcd0-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.338 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:19:24.338 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:24.339 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:24.340 DEBUG:boto:Method: GET 2022-09-03T06:19:24.340 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:19:24.341 DEBUG:boto:Data: 2022-09-03T06:19:24.341 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.342 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.342 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.343 DEBUG:boto:Params: {} 2022-09-03T06:19:24.344 DEBUG:boto:Token: None 2022-09-03T06:19:24.344 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/b 2022-09-03T06:19:24.345 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VQX3CRYcHDTdfFc4ntKEjFrJEB4= 2022-09-03T06:19:24.345 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VQX3CRYcHDTdfFc4ntKEjFrJEB4=', 'Content-Length': '0'} 2022-09-03T06:19:24.348 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000974a6445ccd334f6-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.349 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:24.350 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:24.358 DEBUG:boto:Method: GET 2022-09-03T06:19:24.359 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:19:24.359 DEBUG:boto:Data: 2022-09-03T06:19:24.360 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.360 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.361 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.361 DEBUG:boto:Params: {} 2022-09-03T06:19:24.362 DEBUG:boto:Token: None 2022-09-03T06:19:24.363 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/b 2022-09-03T06:19:24.363 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VQX3CRYcHDTdfFc4ntKEjFrJEB4= 2022-09-03T06:19:24.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VQX3CRYcHDTdfFc4ntKEjFrJEB4=', 'Content-Length': '0'} 2022-09-03T06:19:24.366 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af2d6a9b60b12453-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.367 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:24.368 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:24.369 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:24.370 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:24.371 DEBUG:boto:Method: GET 2022-09-03T06:19:24.372 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:19:24.373 DEBUG:boto:Data: 2022-09-03T06:19:24.374 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.375 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.376 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.377 DEBUG:boto:Params: {} 2022-09-03T06:19:24.378 DEBUG:boto:Token: None 2022-09-03T06:19:24.380 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/ 2022-09-03T06:19:24.381 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M= 2022-09-03T06:19:24.382 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M=', 'Content-Length': '0'} 2022-09-03T06:19:24.388 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000072c77fbf440ba9d5-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.389 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:24.390 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:24.391 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:24.391 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:24.392 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:24.392 DEBUG:boto:Method: GET 2022-09-03T06:19:24.393 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:19:24.394 DEBUG:boto:Data: 2022-09-03T06:19:24.394 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.395 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.395 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.396 DEBUG:boto:Params: {} 2022-09-03T06:19:24.396 DEBUG:boto:Token: None 2022-09-03T06:19:24.397 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/ 2022-09-03T06:19:24.398 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M= 2022-09-03T06:19:24.399 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M=', 'Content-Length': '0'} 2022-09-03T06:19:24.401 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008670ab1e0424261d-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.442 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:24.443 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:19:24.444 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:24.445 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:24.446 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.447 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.447 DEBUG:boto:Method: GET 2022-09-03T06:19:24.448 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.448 DEBUG:boto:Data: 2022-09-03T06:19:24.449 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.449 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.450 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.451 DEBUG:boto:Params: {} 2022-09-03T06:19:24.451 DEBUG:boto:Token: None 2022-09-03T06:19:24.452 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.453 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XvJtgdNA1UoDpIxjGTJVQKulT7w= 2022-09-03T06:19:24.455 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XvJtgdNA1UoDpIxjGTJVQKulT7w=', 'Content-Length': '0'} 2022-09-03T06:19:24.458 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e55c524c0ad06b5e-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.460 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:24.460 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:24.461 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.461 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.462 DEBUG:boto:Method: GET 2022-09-03T06:19:24.463 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.463 DEBUG:boto:Data: 2022-09-03T06:19:24.464 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.465 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.465 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.466 DEBUG:boto:Params: {} 2022-09-03T06:19:24.467 DEBUG:boto:Token: None 2022-09-03T06:19:24.467 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.468 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XvJtgdNA1UoDpIxjGTJVQKulT7w= 2022-09-03T06:19:24.469 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XvJtgdNA1UoDpIxjGTJVQKulT7w=', 'Content-Length': '0'} 2022-09-03T06:19:24.471 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000064b6106647065d6-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.472 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:24.473 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:24.473 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.474 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.475 DEBUG:boto:Method: HEAD 2022-09-03T06:19:24.475 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.476 DEBUG:boto:Data: 2022-09-03T06:19:24.476 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.477 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.478 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.478 DEBUG:boto:Params: {} 2022-09-03T06:19:24.479 DEBUG:boto:Token: None 2022-09-03T06:19:24.480 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.480 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sZPf/W7GVzBAEhq86vMQgOU4xLc= 2022-09-03T06:19:24.481 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sZPf/W7GVzBAEhq86vMQgOU4xLc=', 'Content-Length': '0'} 2022-09-03T06:19:24.483 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c88b206f29dcc69-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.484 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:24.485 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:24.485 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.486 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.486 DEBUG:boto:Method: HEAD 2022-09-03T06:19:24.487 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.488 DEBUG:boto:Data: 2022-09-03T06:19:24.488 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.489 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.489 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.490 DEBUG:boto:Params: {} 2022-09-03T06:19:24.490 DEBUG:boto:Token: None 2022-09-03T06:19:24.491 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.492 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sZPf/W7GVzBAEhq86vMQgOU4xLc= 2022-09-03T06:19:24.492 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sZPf/W7GVzBAEhq86vMQgOU4xLc=', 'Content-Length': '0'} 2022-09-03T06:19:24.494 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005ed17e721c448c74-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.495 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:19:24.496 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:24.496 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:24.497 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.498 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.498 DEBUG:boto:Method: GET 2022-09-03T06:19:24.499 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.499 DEBUG:boto:Data: 2022-09-03T06:19:24.500 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.500 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.501 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.502 DEBUG:boto:Params: {} 2022-09-03T06:19:24.502 DEBUG:boto:Token: None 2022-09-03T06:19:24.503 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.504 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XvJtgdNA1UoDpIxjGTJVQKulT7w= 2022-09-03T06:19:24.504 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XvJtgdNA1UoDpIxjGTJVQKulT7w=', 'Content-Length': '0'} 2022-09-03T06:19:24.507 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b33615747a9f9630-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.550 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:24.551 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:24.553 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.554 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.555 DEBUG:boto:Method: GET 2022-09-03T06:19:24.555 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.556 DEBUG:boto:Data: 2022-09-03T06:19:24.556 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.557 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.558 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.559 DEBUG:boto:Params: {} 2022-09-03T06:19:24.560 DEBUG:boto:Token: None 2022-09-03T06:19:24.563 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:19:24.564 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XvJtgdNA1UoDpIxjGTJVQKulT7w= 2022-09-03T06:19:24.566 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XvJtgdNA1UoDpIxjGTJVQKulT7w=', 'Content-Length': '0'} 2022-09-03T06:19:24.569 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bc5804b97bdf1880-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.571 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:24.572 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:24.573 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:24.574 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:24.576 DEBUG:boto:Method: GET 2022-09-03T06:19:24.577 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:19:24.578 DEBUG:boto:Data: 2022-09-03T06:19:24.579 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.580 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.581 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.582 DEBUG:boto:Params: {} 2022-09-03T06:19:24.583 DEBUG:boto:Token: None 2022-09-03T06:19:24.584 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/ 2022-09-03T06:19:24.585 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M= 2022-09-03T06:19:24.586 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M=', 'Content-Length': '0'} 2022-09-03T06:19:24.589 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bd582ffcdc0f258a-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.590 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:24.592 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:24.593 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:24.594 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:24.595 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:24.596 DEBUG:boto:Method: GET 2022-09-03T06:19:24.596 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:19:24.597 DEBUG:boto:Data: 2022-09-03T06:19:24.598 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.598 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.599 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.599 DEBUG:boto:Params: {} 2022-09-03T06:19:24.600 DEBUG:boto:Token: None 2022-09-03T06:19:24.600 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/ 2022-09-03T06:19:24.601 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M= 2022-09-03T06:19:24.602 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M=', 'Content-Length': '0'} 2022-09-03T06:19:24.604 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e41eab8922a86cba-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.605 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:24.606 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:24.607 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:24.607 DEBUG:boto:Method: HEAD 2022-09-03T06:19:24.608 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:19:24.608 DEBUG:boto:Data: 2022-09-03T06:19:24.609 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.609 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.610 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.610 DEBUG:boto:Params: {} 2022-09-03T06:19:24.611 DEBUG:boto:Token: None 2022-09-03T06:19:24.612 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/c 2022-09-03T06:19:24.613 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DFBitTHHJrnZE7/H0WiW5+9MBg8= 2022-09-03T06:19:24.614 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DFBitTHHJrnZE7/H0WiW5+9MBg8=', 'Content-Length': '0'} 2022-09-03T06:19:24.616 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d40dd9f8628fdd9-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.617 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:24.617 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:24.618 DEBUG:boto:Method: HEAD 2022-09-03T06:19:24.618 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:19:24.619 DEBUG:boto:Data: 2022-09-03T06:19:24.621 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.621 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.622 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.623 DEBUG:boto:Params: {} 2022-09-03T06:19:24.623 DEBUG:boto:Token: None 2022-09-03T06:19:24.624 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/c 2022-09-03T06:19:24.624 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DFBitTHHJrnZE7/H0WiW5+9MBg8= 2022-09-03T06:19:24.625 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DFBitTHHJrnZE7/H0WiW5+9MBg8=', 'Content-Length': '0'} 2022-09-03T06:19:24.628 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c1756430fe3d87a-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.629 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:19:24.630 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:24.630 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:24.631 DEBUG:boto:Method: GET 2022-09-03T06:19:24.632 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:19:24.633 DEBUG:boto:Data: 2022-09-03T06:19:24.633 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.634 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.634 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.635 DEBUG:boto:Params: {} 2022-09-03T06:19:24.636 DEBUG:boto:Token: None 2022-09-03T06:19:24.637 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/c 2022-09-03T06:19:24.638 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o5mb5BRDUkRYLcL//UIpabDxTd4= 2022-09-03T06:19:24.639 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o5mb5BRDUkRYLcL//UIpabDxTd4=', 'Content-Length': '0'} 2022-09-03T06:19:24.641 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a94775407fbcd60-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.643 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:24.643 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:24.644 DEBUG:boto:Method: GET 2022-09-03T06:19:24.644 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:19:24.645 DEBUG:boto:Data: 2022-09-03T06:19:24.647 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.648 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.649 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.649 DEBUG:boto:Params: {} 2022-09-03T06:19:24.650 DEBUG:boto:Token: None 2022-09-03T06:19:24.651 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/c 2022-09-03T06:19:24.651 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o5mb5BRDUkRYLcL//UIpabDxTd4= 2022-09-03T06:19:24.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o5mb5BRDUkRYLcL//UIpabDxTd4=', 'Content-Length': '0'} 2022-09-03T06:19:24.655 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000035c39f2524fcaa23-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.656 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:24.657 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:24.658 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:24.658 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:24.659 DEBUG:boto:Method: GET 2022-09-03T06:19:24.659 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:19:24.660 DEBUG:boto:Data: 2022-09-03T06:19:24.660 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.661 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.662 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.662 DEBUG:boto:Params: {} 2022-09-03T06:19:24.663 DEBUG:boto:Token: None 2022-09-03T06:19:24.663 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/ 2022-09-03T06:19:24.664 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M= 2022-09-03T06:19:24.665 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M=', 'Content-Length': '0'} 2022-09-03T06:19:24.668 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006e011ec9df15fd78-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.669 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:24.670 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:24.671 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:24.671 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:24.672 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:24.672 DEBUG:boto:Method: GET 2022-09-03T06:19:24.673 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:19:24.673 DEBUG:boto:Data: 2022-09-03T06:19:24.674 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.674 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.675 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.675 DEBUG:boto:Params: {} 2022-09-03T06:19:24.676 DEBUG:boto:Token: None 2022-09-03T06:19:24.677 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/ 2022-09-03T06:19:24.677 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M= 2022-09-03T06:19:24.678 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M=', 'Content-Length': '0'} 2022-09-03T06:19:24.682 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d9afcce162ac0d76-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.722 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:24.724 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:19:24.725 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:24.726 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:24.727 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.728 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.729 DEBUG:boto:Method: GET 2022-09-03T06:19:24.732 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.734 DEBUG:boto:Data: 2022-09-03T06:19:24.735 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.736 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.737 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.739 DEBUG:boto:Params: {} 2022-09-03T06:19:24.740 DEBUG:boto:Token: None 2022-09-03T06:19:24.741 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.742 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qnU4qE7GIGRIPKS5oovpEfLT/+M= 2022-09-03T06:19:24.744 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qnU4qE7GIGRIPKS5oovpEfLT/+M=', 'Content-Length': '0'} 2022-09-03T06:19:24.747 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000090001d7f2f22e2f5-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.749 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:24.750 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:24.751 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.752 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.753 DEBUG:boto:Method: GET 2022-09-03T06:19:24.753 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.754 DEBUG:boto:Data: 2022-09-03T06:19:24.759 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.759 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.760 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.761 DEBUG:boto:Params: {} 2022-09-03T06:19:24.761 DEBUG:boto:Token: None 2022-09-03T06:19:24.762 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.763 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qnU4qE7GIGRIPKS5oovpEfLT/+M= 2022-09-03T06:19:24.763 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qnU4qE7GIGRIPKS5oovpEfLT/+M=', 'Content-Length': '0'} 2022-09-03T06:19:24.768 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e9c380f41d83e472-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.769 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:24.769 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:24.770 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.770 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.771 DEBUG:boto:Method: HEAD 2022-09-03T06:19:24.771 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.772 DEBUG:boto:Data: 2022-09-03T06:19:24.773 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.773 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.774 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.774 DEBUG:boto:Params: {} 2022-09-03T06:19:24.775 DEBUG:boto:Token: None 2022-09-03T06:19:24.775 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.776 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8nxXlM2gQhyHtFaH6neqrLJ/U14= 2022-09-03T06:19:24.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8nxXlM2gQhyHtFaH6neqrLJ/U14=', 'Content-Length': '0'} 2022-09-03T06:19:24.780 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d2623f1496b121b6-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.781 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:24.782 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:24.782 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.783 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.783 DEBUG:boto:Method: HEAD 2022-09-03T06:19:24.784 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.784 DEBUG:boto:Data: 2022-09-03T06:19:24.785 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.785 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.786 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.787 DEBUG:boto:Params: {} 2022-09-03T06:19:24.787 DEBUG:boto:Token: None 2022-09-03T06:19:24.788 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.788 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8nxXlM2gQhyHtFaH6neqrLJ/U14= 2022-09-03T06:19:24.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8nxXlM2gQhyHtFaH6neqrLJ/U14=', 'Content-Length': '0'} 2022-09-03T06:19:24.792 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af2e71bb67a29394-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.793 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:19:24.793 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:24.794 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:24.794 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.795 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.795 DEBUG:boto:Method: GET 2022-09-03T06:19:24.796 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.797 DEBUG:boto:Data: 2022-09-03T06:19:24.797 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.798 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.798 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.799 DEBUG:boto:Params: {} 2022-09-03T06:19:24.799 DEBUG:boto:Token: None 2022-09-03T06:19:24.800 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.801 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qnU4qE7GIGRIPKS5oovpEfLT/+M= 2022-09-03T06:19:24.802 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qnU4qE7GIGRIPKS5oovpEfLT/+M=', 'Content-Length': '0'} 2022-09-03T06:19:24.804 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003fb88fc75ced6ad3-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.805 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:24.805 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:24.806 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.807 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.808 DEBUG:boto:Method: GET 2022-09-03T06:19:24.809 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.810 DEBUG:boto:Data: 2022-09-03T06:19:24.811 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.812 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.814 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.814 DEBUG:boto:Params: {} 2022-09-03T06:19:24.815 DEBUG:boto:Token: None 2022-09-03T06:19:24.817 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:19:24.817 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qnU4qE7GIGRIPKS5oovpEfLT/+M= 2022-09-03T06:19:24.818 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qnU4qE7GIGRIPKS5oovpEfLT/+M=', 'Content-Length': '0'} 2022-09-03T06:19:24.824 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000004c61752bf323be4-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.826 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:24.827 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:24.828 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:24.829 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:24.830 DEBUG:boto:Method: GET 2022-09-03T06:19:24.831 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:19:24.831 DEBUG:boto:Data: 2022-09-03T06:19:24.833 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.834 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.834 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.835 DEBUG:boto:Params: {} 2022-09-03T06:19:24.836 DEBUG:boto:Token: None 2022-09-03T06:19:24.837 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/ 2022-09-03T06:19:24.838 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M= 2022-09-03T06:19:24.839 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M=', 'Content-Length': '0'} 2022-09-03T06:19:24.842 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000083f909cc9602cd86-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.844 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:24.846 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:24.847 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:24.847 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:24.848 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:24.849 DEBUG:boto:Method: GET 2022-09-03T06:19:24.850 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:19:24.851 DEBUG:boto:Data: 2022-09-03T06:19:24.852 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.854 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.855 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.856 DEBUG:boto:Params: {} 2022-09-03T06:19:24.857 DEBUG:boto:Token: None 2022-09-03T06:19:24.858 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/ 2022-09-03T06:19:24.859 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M= 2022-09-03T06:19:24.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M=', 'Content-Length': '0'} 2022-09-03T06:19:24.865 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ebb219edc4c3cb0c-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.866 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:24.868 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:24.869 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:24.870 DEBUG:boto:Method: HEAD 2022-09-03T06:19:24.871 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:19:24.872 DEBUG:boto:Data: 2022-09-03T06:19:24.873 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.874 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.875 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.876 DEBUG:boto:Params: {} 2022-09-03T06:19:24.877 DEBUG:boto:Token: None 2022-09-03T06:19:24.878 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/d 2022-09-03T06:19:24.879 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h62+pakgdHWOPE9yUdgnUeSr3Qg= 2022-09-03T06:19:24.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h62+pakgdHWOPE9yUdgnUeSr3Qg=', 'Content-Length': '0'} 2022-09-03T06:19:24.884 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000f6db522996754f2-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.885 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:24.887 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:24.887 DEBUG:boto:Method: HEAD 2022-09-03T06:19:24.888 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:19:24.888 DEBUG:boto:Data: 2022-09-03T06:19:24.889 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.889 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.890 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.890 DEBUG:boto:Params: {} 2022-09-03T06:19:24.891 DEBUG:boto:Token: None 2022-09-03T06:19:24.892 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/d 2022-09-03T06:19:24.892 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h62+pakgdHWOPE9yUdgnUeSr3Qg= 2022-09-03T06:19:24.893 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h62+pakgdHWOPE9yUdgnUeSr3Qg=', 'Content-Length': '0'} 2022-09-03T06:19:24.898 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003bfec002b3e5cab4-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.899 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:19:24.900 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:24.902 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:24.903 DEBUG:boto:Method: GET 2022-09-03T06:19:24.904 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:19:24.906 DEBUG:boto:Data: 2022-09-03T06:19:24.909 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.910 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.911 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.912 DEBUG:boto:Params: {} 2022-09-03T06:19:24.913 DEBUG:boto:Token: None 2022-09-03T06:19:24.916 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/d 2022-09-03T06:19:24.917 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ByksF6L5Q7q7seliDze9jhgZESM= 2022-09-03T06:19:24.918 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ByksF6L5Q7q7seliDze9jhgZESM=', 'Content-Length': '0'} 2022-09-03T06:19:24.921 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000798a251dd2eed995-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.967 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:24.968 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:24.968 DEBUG:boto:Method: GET 2022-09-03T06:19:24.969 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:19:24.970 DEBUG:boto:Data: 2022-09-03T06:19:24.970 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.971 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:24.971 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.972 DEBUG:boto:Params: {} 2022-09-03T06:19:24.973 DEBUG:boto:Token: None 2022-09-03T06:19:24.974 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/d 2022-09-03T06:19:24.974 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ByksF6L5Q7q7seliDze9jhgZESM= 2022-09-03T06:19:24.975 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ByksF6L5Q7q7seliDze9jhgZESM=', 'Content-Length': '0'} 2022-09-03T06:19:24.978 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fbeaa88f6619fc84-006312f1ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.979 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:24.980 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:24.980 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:24.981 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:24.981 DEBUG:boto:Method: GET 2022-09-03T06:19:24.982 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:19:24.983 DEBUG:boto:Data: 2022-09-03T06:19:24.984 DEBUG:boto:Headers: {} 2022-09-03T06:19:24.984 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:24.985 DEBUG:boto:Port: 8000 2022-09-03T06:19:24.985 DEBUG:boto:Params: {} 2022-09-03T06:19:24.986 DEBUG:boto:Token: None 2022-09-03T06:19:24.987 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:24 GMT /dsgvcy-72/ 2022-09-03T06:19:24.988 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M= 2022-09-03T06:19:24.989 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hAmWKBglvki4K1dRtAyEZU+gs9M=', 'Content-Length': '0'} 2022-09-03T06:19:24.992 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ae5fad7edd72e3c3-006312f1ec-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:24.993 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:24.993 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:24.994 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:24.995 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:24.996 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:24.997 DEBUG:boto:Method: GET 2022-09-03T06:19:24.998 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:19:24.999 DEBUG:boto:Data: 2022-09-03T06:19:25.001 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.002 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:25.003 DEBUG:boto:Port: 8000 2022-09-03T06:19:25.004 DEBUG:boto:Params: {} 2022-09-03T06:19:25.005 DEBUG:boto:Token: None 2022-09-03T06:19:25.006 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/ 2022-09-03T06:19:25.007 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6jGNixUQDpwQFAxRk3OcENT1uaE= 2022-09-03T06:19:25.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6jGNixUQDpwQFAxRk3OcENT1uaE=', 'Content-Length': '0'} 2022-09-03T06:19:25.011 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000074675ff40bfd582c-006312f1ed-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.054 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:25.056 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-72 zones={test-zone2, test-zone1} 2022-09-03T06:19:25.056 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-72 zones={test-zone3, test-zone1} 2022-09-03T06:19:25.057 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:25.057 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:25.058 DEBUG:boto:Method: HEAD 2022-09-03T06:19:25.059 DEBUG:boto:Path: /dsgvcy-72/ 2022-09-03T06:19:25.059 DEBUG:boto:Data: 2022-09-03T06:19:25.060 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.060 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:25.061 DEBUG:boto:Port: 8001 2022-09-03T06:19:25.062 DEBUG:boto:Params: {} 2022-09-03T06:19:25.063 DEBUG:boto:Token: None 2022-09-03T06:19:25.064 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/ 2022-09-03T06:19:25.065 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WmcqxpgAxz4GB1D8ED751lcGum4= 2022-09-03T06:19:25.065 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WmcqxpgAxz4GB1D8ED751lcGum4=', 'Content-Length': '0'} 2022-09-03T06:19:25.069 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b6e2c64ce909820a-006312f1ed-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.070 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:25.070 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:25.071 DEBUG:boto:Method: HEAD 2022-09-03T06:19:25.071 DEBUG:boto:Path: /dsgvcy-72/ 2022-09-03T06:19:25.072 DEBUG:boto:Data: 2022-09-03T06:19:25.072 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.073 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:25.073 DEBUG:boto:Port: 8000 2022-09-03T06:19:25.074 DEBUG:boto:Params: {} 2022-09-03T06:19:25.074 DEBUG:boto:Token: None 2022-09-03T06:19:25.075 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/ 2022-09-03T06:19:25.075 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WmcqxpgAxz4GB1D8ED751lcGum4= 2022-09-03T06:19:25.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WmcqxpgAxz4GB1D8ED751lcGum4=', 'Content-Length': '0'} 2022-09-03T06:19:25.079 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000001f9c3adf1549c52e-006312f1ed-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.080 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:19:25.080 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:25.081 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:25.081 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:25.082 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:25.083 DEBUG:boto:Method: GET 2022-09-03T06:19:25.083 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:25.084 DEBUG:boto:Data: 2022-09-03T06:19:25.086 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.086 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:25.089 DEBUG:boto:Port: 8001 2022-09-03T06:19:25.089 DEBUG:boto:Params: {} 2022-09-03T06:19:25.090 DEBUG:boto:Token: None 2022-09-03T06:19:25.091 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/?versions 2022-09-03T06:19:25.093 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MfjuA2NLQUbN+GFH1M+03GoJ5jc= 2022-09-03T06:19:25.093 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MfjuA2NLQUbN+GFH1M+03GoJ5jc=', 'Content-Length': '0'} 2022-09-03T06:19:25.096 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f484975607c88dce-006312f1ed-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.098 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:25.100 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:19:25.101 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:19:25.102 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:19:25.103 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:19:25.104 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:19:25.105 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:25.106 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:25.107 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:25.108 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:25.109 DEBUG:boto:Method: GET 2022-09-03T06:19:25.111 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:25.111 DEBUG:boto:Data: 2022-09-03T06:19:25.112 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.113 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:25.114 DEBUG:boto:Port: 8000 2022-09-03T06:19:25.114 DEBUG:boto:Params: {} 2022-09-03T06:19:25.115 DEBUG:boto:Token: None 2022-09-03T06:19:25.116 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/?versions 2022-09-03T06:19:25.116 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MfjuA2NLQUbN+GFH1M+03GoJ5jc= 2022-09-03T06:19:25.117 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MfjuA2NLQUbN+GFH1M+03GoJ5jc=', 'Content-Length': '0'} 2022-09-03T06:19:25.120 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e59669c2d59cd75a-006312f1ed-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.120 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:25.122 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:19:25.123 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:19:25.123 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:19:25.124 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:19:25.125 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:25.125 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:25.126 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:25.126 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:25.127 DEBUG:boto:Method: GET 2022-09-03T06:19:25.127 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:25.128 DEBUG:boto:Data: 2022-09-03T06:19:25.128 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.129 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:25.130 DEBUG:boto:Port: 8001 2022-09-03T06:19:25.131 DEBUG:boto:Params: {} 2022-09-03T06:19:25.132 DEBUG:boto:Token: None 2022-09-03T06:19:25.132 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/?versions 2022-09-03T06:19:25.133 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MfjuA2NLQUbN+GFH1M+03GoJ5jc= 2022-09-03T06:19:25.134 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MfjuA2NLQUbN+GFH1M+03GoJ5jc=', 'Content-Length': '0'} 2022-09-03T06:19:25.137 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001ff8bc27862f3e02-006312f1ed-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.138 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:25.140 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:25.141 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:25.141 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:25.142 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:25.143 DEBUG:boto:Method: GET 2022-09-03T06:19:25.143 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:25.144 DEBUG:boto:Data: 2022-09-03T06:19:25.145 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.146 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:25.147 DEBUG:boto:Port: 8000 2022-09-03T06:19:25.149 DEBUG:boto:Params: {} 2022-09-03T06:19:25.150 DEBUG:boto:Token: None 2022-09-03T06:19:25.151 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/?versions 2022-09-03T06:19:25.152 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MfjuA2NLQUbN+GFH1M+03GoJ5jc= 2022-09-03T06:19:25.153 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MfjuA2NLQUbN+GFH1M+03GoJ5jc=', 'Content-Length': '0'} 2022-09-03T06:19:25.156 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000064f4d21df64bc678-006312f1ed-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.158 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:25.160 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:19:25.161 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:25.162 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:25.163 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.164 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.165 DEBUG:boto:Method: GET 2022-09-03T06:19:25.166 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.167 DEBUG:boto:Data: 2022-09-03T06:19:25.168 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.169 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:25.170 DEBUG:boto:Port: 8001 2022-09-03T06:19:25.171 DEBUG:boto:Params: {} 2022-09-03T06:19:25.172 DEBUG:boto:Token: None 2022-09-03T06:19:25.173 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.173 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NbX6W75md2Q6yHT6X1XKx/NX19Q= 2022-09-03T06:19:25.175 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NbX6W75md2Q6yHT6X1XKx/NX19Q=', 'Content-Length': '0'} 2022-09-03T06:19:25.178 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004659d0bb4bc755cf-006312f1ed-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.179 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:25.180 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:25.181 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.181 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.182 DEBUG:boto:Method: GET 2022-09-03T06:19:25.183 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.184 DEBUG:boto:Data: 2022-09-03T06:19:25.184 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.185 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:25.186 DEBUG:boto:Port: 8000 2022-09-03T06:19:25.187 DEBUG:boto:Params: {} 2022-09-03T06:19:25.187 DEBUG:boto:Token: None 2022-09-03T06:19:25.188 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.189 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NbX6W75md2Q6yHT6X1XKx/NX19Q= 2022-09-03T06:19:25.189 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NbX6W75md2Q6yHT6X1XKx/NX19Q=', 'Content-Length': '0'} 2022-09-03T06:19:25.192 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b3761ab74fdbe1c-006312f1ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.193 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:25.194 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:25.194 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.195 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.195 DEBUG:boto:Method: HEAD 2022-09-03T06:19:25.196 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.196 DEBUG:boto:Data: 2022-09-03T06:19:25.197 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.197 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:25.198 DEBUG:boto:Port: 8001 2022-09-03T06:19:25.199 DEBUG:boto:Params: {} 2022-09-03T06:19:25.199 DEBUG:boto:Token: None 2022-09-03T06:19:25.200 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.201 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HhTExvZYoTvyKLODRZZiQgynzew= 2022-09-03T06:19:25.201 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HhTExvZYoTvyKLODRZZiQgynzew=', 'Content-Length': '0'} 2022-09-03T06:19:25.205 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000056a7543cac7af01d-006312f1ed-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.206 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:25.206 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:25.207 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.208 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.208 DEBUG:boto:Method: HEAD 2022-09-03T06:19:25.209 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.209 DEBUG:boto:Data: 2022-09-03T06:19:25.210 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.210 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:25.211 DEBUG:boto:Port: 8000 2022-09-03T06:19:25.211 DEBUG:boto:Params: {} 2022-09-03T06:19:25.212 DEBUG:boto:Token: None 2022-09-03T06:19:25.213 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.213 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HhTExvZYoTvyKLODRZZiQgynzew= 2022-09-03T06:19:25.214 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HhTExvZYoTvyKLODRZZiQgynzew=', 'Content-Length': '0'} 2022-09-03T06:19:25.216 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000229f0eb7e682ce59-006312f1ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.218 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:19:25.219 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:25.220 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:25.221 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.223 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.223 DEBUG:boto:Method: GET 2022-09-03T06:19:25.224 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.225 DEBUG:boto:Data: 2022-09-03T06:19:25.226 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.228 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:25.228 DEBUG:boto:Port: 8001 2022-09-03T06:19:25.229 DEBUG:boto:Params: {} 2022-09-03T06:19:25.229 DEBUG:boto:Token: None 2022-09-03T06:19:25.231 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.233 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NbX6W75md2Q6yHT6X1XKx/NX19Q= 2022-09-03T06:19:25.234 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NbX6W75md2Q6yHT6X1XKx/NX19Q=', 'Content-Length': '0'} 2022-09-03T06:19:25.237 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e8901804f618e64-006312f1ed-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.239 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:25.239 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:25.240 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.241 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.242 DEBUG:boto:Method: GET 2022-09-03T06:19:25.243 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.244 DEBUG:boto:Data: 2022-09-03T06:19:25.245 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.246 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:25.247 DEBUG:boto:Port: 8000 2022-09-03T06:19:25.248 DEBUG:boto:Params: {} 2022-09-03T06:19:25.249 DEBUG:boto:Token: None 2022-09-03T06:19:25.250 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:19:25.252 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NbX6W75md2Q6yHT6X1XKx/NX19Q= 2022-09-03T06:19:25.253 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NbX6W75md2Q6yHT6X1XKx/NX19Q=', 'Content-Length': '0'} 2022-09-03T06:19:25.256 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000e2132ba128e27e8-006312f1ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.258 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:25.259 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:25.260 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:25.261 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:25.261 DEBUG:boto:Method: GET 2022-09-03T06:19:25.262 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:19:25.263 DEBUG:boto:Data: 2022-09-03T06:19:25.265 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.266 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:25.267 DEBUG:boto:Port: 8001 2022-09-03T06:19:25.268 DEBUG:boto:Params: {} 2022-09-03T06:19:25.269 DEBUG:boto:Token: None 2022-09-03T06:19:25.270 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/ 2022-09-03T06:19:25.271 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6jGNixUQDpwQFAxRk3OcENT1uaE= 2022-09-03T06:19:25.272 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6jGNixUQDpwQFAxRk3OcENT1uaE=', 'Content-Length': '0'} 2022-09-03T06:19:25.277 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008fca0771f237dbfc-006312f1ed-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.278 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:25.279 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:25.279 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:25.280 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:25.281 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:25.283 DEBUG:boto:Method: GET 2022-09-03T06:19:25.283 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:19:25.284 DEBUG:boto:Data: 2022-09-03T06:19:25.285 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.286 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:25.287 DEBUG:boto:Port: 8000 2022-09-03T06:19:25.288 DEBUG:boto:Params: {} 2022-09-03T06:19:25.289 DEBUG:boto:Token: None 2022-09-03T06:19:25.290 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/ 2022-09-03T06:19:25.291 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6jGNixUQDpwQFAxRk3OcENT1uaE= 2022-09-03T06:19:25.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6jGNixUQDpwQFAxRk3OcENT1uaE=', 'Content-Length': '0'} 2022-09-03T06:19:25.296 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004998c27e71b04627-006312f1ed-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.297 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:25.299 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:25.300 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:25.300 DEBUG:boto:Method: HEAD 2022-09-03T06:19:25.301 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:19:25.302 DEBUG:boto:Data: 2022-09-03T06:19:25.303 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.304 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:25.305 DEBUG:boto:Port: 8001 2022-09-03T06:19:25.306 DEBUG:boto:Params: {} 2022-09-03T06:19:25.307 DEBUG:boto:Token: None 2022-09-03T06:19:25.307 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/a 2022-09-03T06:19:25.308 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q0pnReD2hL6NGfvSSIaAfRdqrWE= 2022-09-03T06:19:25.310 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q0pnReD2hL6NGfvSSIaAfRdqrWE=', 'Content-Length': '0'} 2022-09-03T06:19:25.315 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000992552035dc3e824-006312f1ed-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.317 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:25.318 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:25.319 DEBUG:boto:Method: HEAD 2022-09-03T06:19:25.321 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:19:25.322 DEBUG:boto:Data: 2022-09-03T06:19:25.323 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.324 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:25.325 DEBUG:boto:Port: 8000 2022-09-03T06:19:25.326 DEBUG:boto:Params: {} 2022-09-03T06:19:25.326 DEBUG:boto:Token: None 2022-09-03T06:19:25.327 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/a 2022-09-03T06:19:25.328 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q0pnReD2hL6NGfvSSIaAfRdqrWE= 2022-09-03T06:19:25.329 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q0pnReD2hL6NGfvSSIaAfRdqrWE=', 'Content-Length': '0'} 2022-09-03T06:19:25.338 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aa738ba665df4da8-006312f1ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.339 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:19:25.340 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:25.341 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:25.341 DEBUG:boto:Method: GET 2022-09-03T06:19:25.342 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:19:25.343 DEBUG:boto:Data: 2022-09-03T06:19:25.344 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.344 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:25.345 DEBUG:boto:Port: 8001 2022-09-03T06:19:25.347 DEBUG:boto:Params: {} 2022-09-03T06:19:25.348 DEBUG:boto:Token: None 2022-09-03T06:19:25.348 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/a 2022-09-03T06:19:25.349 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wAm/b8ZxZ9cPyv7bUqY70DtNx/g= 2022-09-03T06:19:25.350 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wAm/b8ZxZ9cPyv7bUqY70DtNx/g=', 'Content-Length': '0'} 2022-09-03T06:19:25.352 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a55e63d71a65287-006312f1ed-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.353 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:25.354 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:25.354 DEBUG:boto:Method: GET 2022-09-03T06:19:25.355 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:19:25.356 DEBUG:boto:Data: 2022-09-03T06:19:25.356 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.357 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:25.358 DEBUG:boto:Port: 8000 2022-09-03T06:19:25.359 DEBUG:boto:Params: {} 2022-09-03T06:19:25.360 DEBUG:boto:Token: None 2022-09-03T06:19:25.361 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/a 2022-09-03T06:19:25.362 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wAm/b8ZxZ9cPyv7bUqY70DtNx/g= 2022-09-03T06:19:25.363 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wAm/b8ZxZ9cPyv7bUqY70DtNx/g=', 'Content-Length': '0'} 2022-09-03T06:19:25.366 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e64c703dafb012b-006312f1ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.408 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:25.409 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:25.410 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:25.411 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:25.412 DEBUG:boto:Method: GET 2022-09-03T06:19:25.413 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:19:25.414 DEBUG:boto:Data: 2022-09-03T06:19:25.415 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.416 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:25.417 DEBUG:boto:Port: 8001 2022-09-03T06:19:25.418 DEBUG:boto:Params: {} 2022-09-03T06:19:25.419 DEBUG:boto:Token: None 2022-09-03T06:19:25.421 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/ 2022-09-03T06:19:25.422 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6jGNixUQDpwQFAxRk3OcENT1uaE= 2022-09-03T06:19:25.423 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6jGNixUQDpwQFAxRk3OcENT1uaE=', 'Content-Length': '0'} 2022-09-03T06:19:25.427 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000034579febf82ab734-006312f1ed-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.428 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:25.431 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:25.432 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:25.433 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:25.434 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:25.435 DEBUG:boto:Method: GET 2022-09-03T06:19:25.436 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:19:25.437 DEBUG:boto:Data: 2022-09-03T06:19:25.437 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.438 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:25.439 DEBUG:boto:Port: 8000 2022-09-03T06:19:25.440 DEBUG:boto:Params: {} 2022-09-03T06:19:25.442 DEBUG:boto:Token: None 2022-09-03T06:19:25.443 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/ 2022-09-03T06:19:25.443 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6jGNixUQDpwQFAxRk3OcENT1uaE= 2022-09-03T06:19:25.444 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6jGNixUQDpwQFAxRk3OcENT1uaE=', 'Content-Length': '0'} 2022-09-03T06:19:25.448 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a1cfaadf8a7a7645-006312f1ed-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.450 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:25.452 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:19:25.453 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:25.454 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:25.455 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.457 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.458 DEBUG:boto:Method: GET 2022-09-03T06:19:25.459 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.460 DEBUG:boto:Data: 2022-09-03T06:19:25.461 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.462 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:25.463 DEBUG:boto:Port: 8001 2022-09-03T06:19:25.464 DEBUG:boto:Params: {} 2022-09-03T06:19:25.464 DEBUG:boto:Token: None 2022-09-03T06:19:25.465 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.467 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZOk2+koqwyCaIWuGHSiIQ7RBmOo= 2022-09-03T06:19:25.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZOk2+koqwyCaIWuGHSiIQ7RBmOo=', 'Content-Length': '0'} 2022-09-03T06:19:25.471 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a76355a63797aec-006312f1ed-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.472 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:25.473 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:25.474 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.475 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.477 DEBUG:boto:Method: GET 2022-09-03T06:19:25.477 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.478 DEBUG:boto:Data: 2022-09-03T06:19:25.479 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.480 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:25.481 DEBUG:boto:Port: 8000 2022-09-03T06:19:25.482 DEBUG:boto:Params: {} 2022-09-03T06:19:25.483 DEBUG:boto:Token: None 2022-09-03T06:19:25.484 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.485 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZOk2+koqwyCaIWuGHSiIQ7RBmOo= 2022-09-03T06:19:25.486 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZOk2+koqwyCaIWuGHSiIQ7RBmOo=', 'Content-Length': '0'} 2022-09-03T06:19:25.489 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ac340d7c69f55950-006312f1ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.493 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:25.494 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:25.494 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.495 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.495 DEBUG:boto:Method: HEAD 2022-09-03T06:19:25.496 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.497 DEBUG:boto:Data: 2022-09-03T06:19:25.497 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.498 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:25.499 DEBUG:boto:Port: 8001 2022-09-03T06:19:25.499 DEBUG:boto:Params: {} 2022-09-03T06:19:25.500 DEBUG:boto:Token: None 2022-09-03T06:19:25.501 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.501 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bmqIHfFXsbWbYbC4bnRtt1PEJ+4= 2022-09-03T06:19:25.502 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bmqIHfFXsbWbYbC4bnRtt1PEJ+4=', 'Content-Length': '0'} 2022-09-03T06:19:25.512 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b75540881ead088d-006312f1ed-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.513 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:25.513 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:25.514 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.514 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.515 DEBUG:boto:Method: HEAD 2022-09-03T06:19:25.516 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.516 DEBUG:boto:Data: 2022-09-03T06:19:25.517 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.517 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:25.518 DEBUG:boto:Port: 8000 2022-09-03T06:19:25.518 DEBUG:boto:Params: {} 2022-09-03T06:19:25.519 DEBUG:boto:Token: None 2022-09-03T06:19:25.520 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.520 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bmqIHfFXsbWbYbC4bnRtt1PEJ+4= 2022-09-03T06:19:25.521 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bmqIHfFXsbWbYbC4bnRtt1PEJ+4=', 'Content-Length': '0'} 2022-09-03T06:19:25.524 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a824f7df8589bec4-006312f1ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.525 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:19:25.526 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:25.526 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:25.527 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.528 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.528 DEBUG:boto:Method: GET 2022-09-03T06:19:25.529 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.529 DEBUG:boto:Data: 2022-09-03T06:19:25.530 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.530 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:25.531 DEBUG:boto:Port: 8001 2022-09-03T06:19:25.532 DEBUG:boto:Params: {} 2022-09-03T06:19:25.533 DEBUG:boto:Token: None 2022-09-03T06:19:25.534 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.535 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZOk2+koqwyCaIWuGHSiIQ7RBmOo= 2022-09-03T06:19:25.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZOk2+koqwyCaIWuGHSiIQ7RBmOo=', 'Content-Length': '0'} 2022-09-03T06:19:25.539 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a261b2756e922891-006312f1ed-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.585 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:25.586 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:25.587 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.590 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.591 DEBUG:boto:Method: GET 2022-09-03T06:19:25.592 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.592 DEBUG:boto:Data: 2022-09-03T06:19:25.593 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.593 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:25.594 DEBUG:boto:Port: 8000 2022-09-03T06:19:25.595 DEBUG:boto:Params: {} 2022-09-03T06:19:25.596 DEBUG:boto:Token: None 2022-09-03T06:19:25.596 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:19:25.597 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZOk2+koqwyCaIWuGHSiIQ7RBmOo= 2022-09-03T06:19:25.598 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZOk2+koqwyCaIWuGHSiIQ7RBmOo=', 'Content-Length': '0'} 2022-09-03T06:19:25.601 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c1854a2d257fceff-006312f1ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.602 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:25.603 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:25.604 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:25.605 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:25.605 DEBUG:boto:Method: GET 2022-09-03T06:19:25.606 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:19:25.607 DEBUG:boto:Data: 2022-09-03T06:19:25.607 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.608 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:25.609 DEBUG:boto:Port: 8001 2022-09-03T06:19:25.610 DEBUG:boto:Params: {} 2022-09-03T06:19:25.611 DEBUG:boto:Token: None 2022-09-03T06:19:25.612 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/ 2022-09-03T06:19:25.613 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6jGNixUQDpwQFAxRk3OcENT1uaE= 2022-09-03T06:19:25.614 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6jGNixUQDpwQFAxRk3OcENT1uaE=', 'Content-Length': '0'} 2022-09-03T06:19:25.617 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e410c654025a0c5a-006312f1ed-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.619 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:25.621 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:25.622 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:25.623 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:25.624 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:25.625 DEBUG:boto:Method: GET 2022-09-03T06:19:25.627 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:19:25.628 DEBUG:boto:Data: 2022-09-03T06:19:25.629 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.630 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:25.631 DEBUG:boto:Port: 8000 2022-09-03T06:19:25.631 DEBUG:boto:Params: {} 2022-09-03T06:19:25.632 DEBUG:boto:Token: None 2022-09-03T06:19:25.633 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/ 2022-09-03T06:19:25.634 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6jGNixUQDpwQFAxRk3OcENT1uaE= 2022-09-03T06:19:25.635 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6jGNixUQDpwQFAxRk3OcENT1uaE=', 'Content-Length': '0'} 2022-09-03T06:19:25.639 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b802ee4230c7b0a9-006312f1ed-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.640 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:25.643 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:25.644 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:25.646 DEBUG:boto:Method: HEAD 2022-09-03T06:19:25.647 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:19:25.648 DEBUG:boto:Data: 2022-09-03T06:19:25.649 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.650 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:25.651 DEBUG:boto:Port: 8001 2022-09-03T06:19:25.651 DEBUG:boto:Params: {} 2022-09-03T06:19:25.652 DEBUG:boto:Token: None 2022-09-03T06:19:25.652 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/b 2022-09-03T06:19:25.653 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aar8DWOzvY/J2/YzN5CMpJi1Kis= 2022-09-03T06:19:25.654 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aar8DWOzvY/J2/YzN5CMpJi1Kis=', 'Content-Length': '0'} 2022-09-03T06:19:25.656 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000084863b34ff1b8fcc-006312f1ed-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.657 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:25.657 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:25.658 DEBUG:boto:Method: HEAD 2022-09-03T06:19:25.659 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:19:25.659 DEBUG:boto:Data: 2022-09-03T06:19:25.660 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.661 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:25.662 DEBUG:boto:Port: 8000 2022-09-03T06:19:25.663 DEBUG:boto:Params: {} 2022-09-03T06:19:25.664 DEBUG:boto:Token: None 2022-09-03T06:19:25.665 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/b 2022-09-03T06:19:25.666 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aar8DWOzvY/J2/YzN5CMpJi1Kis= 2022-09-03T06:19:25.667 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aar8DWOzvY/J2/YzN5CMpJi1Kis=', 'Content-Length': '0'} 2022-09-03T06:19:25.670 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006901473f81d5cace-006312f1ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.671 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:19:25.672 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:25.672 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:25.673 DEBUG:boto:Method: GET 2022-09-03T06:19:25.673 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:19:25.674 DEBUG:boto:Data: 2022-09-03T06:19:25.675 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.676 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:25.677 DEBUG:boto:Port: 8001 2022-09-03T06:19:25.678 DEBUG:boto:Params: {} 2022-09-03T06:19:25.679 DEBUG:boto:Token: None 2022-09-03T06:19:25.680 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/b 2022-09-03T06:19:25.681 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BVTu9nKc7RV8+yX3tptWNF3J+iQ= 2022-09-03T06:19:25.682 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BVTu9nKc7RV8+yX3tptWNF3J+iQ=', 'Content-Length': '0'} 2022-09-03T06:19:25.685 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b8a9a7efb1fd25fb-006312f1ed-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.687 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:25.688 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:25.689 DEBUG:boto:Method: GET 2022-09-03T06:19:25.690 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:19:25.691 DEBUG:boto:Data: 2022-09-03T06:19:25.692 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.692 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:25.693 DEBUG:boto:Port: 8000 2022-09-03T06:19:25.694 DEBUG:boto:Params: {} 2022-09-03T06:19:25.695 DEBUG:boto:Token: None 2022-09-03T06:19:25.696 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/b 2022-09-03T06:19:25.698 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BVTu9nKc7RV8+yX3tptWNF3J+iQ= 2022-09-03T06:19:25.699 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BVTu9nKc7RV8+yX3tptWNF3J+iQ=', 'Content-Length': '0'} 2022-09-03T06:19:25.702 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d8c00bc66d8d69b2-006312f1ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.751 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:25.751 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:25.752 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:25.753 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:25.753 DEBUG:boto:Method: GET 2022-09-03T06:19:25.754 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:19:25.754 DEBUG:boto:Data: 2022-09-03T06:19:25.755 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.756 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:25.756 DEBUG:boto:Port: 8001 2022-09-03T06:19:25.757 DEBUG:boto:Params: {} 2022-09-03T06:19:25.757 DEBUG:boto:Token: None 2022-09-03T06:19:25.758 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/ 2022-09-03T06:19:25.759 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6jGNixUQDpwQFAxRk3OcENT1uaE= 2022-09-03T06:19:25.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6jGNixUQDpwQFAxRk3OcENT1uaE=', 'Content-Length': '0'} 2022-09-03T06:19:25.763 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000033b24295299a3664-006312f1ed-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.764 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:25.765 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:25.766 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:25.766 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:25.767 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:25.768 DEBUG:boto:Method: GET 2022-09-03T06:19:25.769 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:19:25.770 DEBUG:boto:Data: 2022-09-03T06:19:25.770 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.771 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:25.771 DEBUG:boto:Port: 8000 2022-09-03T06:19:25.772 DEBUG:boto:Params: {} 2022-09-03T06:19:25.773 DEBUG:boto:Token: None 2022-09-03T06:19:25.773 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/ 2022-09-03T06:19:25.774 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6jGNixUQDpwQFAxRk3OcENT1uaE= 2022-09-03T06:19:25.775 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6jGNixUQDpwQFAxRk3OcENT1uaE=', 'Content-Length': '0'} 2022-09-03T06:19:25.778 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b67cd72f223608f5-006312f1ed-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.779 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:25.780 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:19:25.780 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:25.781 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:25.782 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.782 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.783 DEBUG:boto:Method: GET 2022-09-03T06:19:25.788 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.789 DEBUG:boto:Data: 2022-09-03T06:19:25.789 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.790 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:25.791 DEBUG:boto:Port: 8001 2022-09-03T06:19:25.791 DEBUG:boto:Params: {} 2022-09-03T06:19:25.792 DEBUG:boto:Token: None 2022-09-03T06:19:25.792 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.793 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2S25aJP67Y82xXIwe0w/UjfHFo4= 2022-09-03T06:19:25.794 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2S25aJP67Y82xXIwe0w/UjfHFo4=', 'Content-Length': '0'} 2022-09-03T06:19:25.797 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ab7b5c0631c268d1-006312f1ed-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.798 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:25.800 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:25.801 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.802 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.803 DEBUG:boto:Method: GET 2022-09-03T06:19:25.804 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.805 DEBUG:boto:Data: 2022-09-03T06:19:25.806 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.807 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:25.808 DEBUG:boto:Port: 8000 2022-09-03T06:19:25.809 DEBUG:boto:Params: {} 2022-09-03T06:19:25.811 DEBUG:boto:Token: None 2022-09-03T06:19:25.812 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.813 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2S25aJP67Y82xXIwe0w/UjfHFo4= 2022-09-03T06:19:25.815 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2S25aJP67Y82xXIwe0w/UjfHFo4=', 'Content-Length': '0'} 2022-09-03T06:19:25.818 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e870b4d6dd45b0d-006312f1ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.820 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:25.821 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:25.822 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.823 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.824 DEBUG:boto:Method: HEAD 2022-09-03T06:19:25.825 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.826 DEBUG:boto:Data: 2022-09-03T06:19:25.827 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.827 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:25.828 DEBUG:boto:Port: 8001 2022-09-03T06:19:25.828 DEBUG:boto:Params: {} 2022-09-03T06:19:25.829 DEBUG:boto:Token: None 2022-09-03T06:19:25.830 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.830 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aRUONUJdEACE93Uoppf4cp6qz2k= 2022-09-03T06:19:25.841 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aRUONUJdEACE93Uoppf4cp6qz2k=', 'Content-Length': '0'} 2022-09-03T06:19:25.843 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad8fc008bad49b2c-006312f1ed-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.844 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:25.846 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:25.847 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.848 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.849 DEBUG:boto:Method: HEAD 2022-09-03T06:19:25.850 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.851 DEBUG:boto:Data: 2022-09-03T06:19:25.852 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.853 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:25.854 DEBUG:boto:Port: 8000 2022-09-03T06:19:25.855 DEBUG:boto:Params: {} 2022-09-03T06:19:25.856 DEBUG:boto:Token: None 2022-09-03T06:19:25.857 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.858 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aRUONUJdEACE93Uoppf4cp6qz2k= 2022-09-03T06:19:25.859 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aRUONUJdEACE93Uoppf4cp6qz2k=', 'Content-Length': '0'} 2022-09-03T06:19:25.862 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000073ea6a10f60b1b0e-006312f1ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.863 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:19:25.864 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:25.866 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:25.868 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.869 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.871 DEBUG:boto:Method: GET 2022-09-03T06:19:25.871 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.872 DEBUG:boto:Data: 2022-09-03T06:19:25.872 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.873 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:25.874 DEBUG:boto:Port: 8001 2022-09-03T06:19:25.875 DEBUG:boto:Params: {} 2022-09-03T06:19:25.876 DEBUG:boto:Token: None 2022-09-03T06:19:25.877 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.878 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2S25aJP67Y82xXIwe0w/UjfHFo4= 2022-09-03T06:19:25.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2S25aJP67Y82xXIwe0w/UjfHFo4=', 'Content-Length': '0'} 2022-09-03T06:19:25.882 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000afa284e26efd1251-006312f1ed-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.922 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:25.924 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:25.925 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.926 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.927 DEBUG:boto:Method: GET 2022-09-03T06:19:25.928 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.929 DEBUG:boto:Data: 2022-09-03T06:19:25.930 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.931 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:25.932 DEBUG:boto:Port: 8000 2022-09-03T06:19:25.933 DEBUG:boto:Params: {} 2022-09-03T06:19:25.935 DEBUG:boto:Token: None 2022-09-03T06:19:25.936 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:19:25.937 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2S25aJP67Y82xXIwe0w/UjfHFo4= 2022-09-03T06:19:25.938 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2S25aJP67Y82xXIwe0w/UjfHFo4=', 'Content-Length': '0'} 2022-09-03T06:19:25.941 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000add240fd560b4143-006312f1ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.942 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:25.943 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:25.944 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:25.945 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:25.946 DEBUG:boto:Method: GET 2022-09-03T06:19:25.947 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:19:25.947 DEBUG:boto:Data: 2022-09-03T06:19:25.949 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.949 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:25.950 DEBUG:boto:Port: 8001 2022-09-03T06:19:25.950 DEBUG:boto:Params: {} 2022-09-03T06:19:25.951 DEBUG:boto:Token: None 2022-09-03T06:19:25.952 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/ 2022-09-03T06:19:25.952 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6jGNixUQDpwQFAxRk3OcENT1uaE= 2022-09-03T06:19:25.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6jGNixUQDpwQFAxRk3OcENT1uaE=', 'Content-Length': '0'} 2022-09-03T06:19:25.958 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000389ce0b2a4554a1c-006312f1ed-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.960 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:25.962 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:25.963 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:25.964 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:25.965 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:25.966 DEBUG:boto:Method: GET 2022-09-03T06:19:25.967 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:19:25.967 DEBUG:boto:Data: 2022-09-03T06:19:25.968 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.968 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:25.969 DEBUG:boto:Port: 8000 2022-09-03T06:19:25.970 DEBUG:boto:Params: {} 2022-09-03T06:19:25.971 DEBUG:boto:Token: None 2022-09-03T06:19:25.972 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/ 2022-09-03T06:19:25.973 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6jGNixUQDpwQFAxRk3OcENT1uaE= 2022-09-03T06:19:25.974 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6jGNixUQDpwQFAxRk3OcENT1uaE=', 'Content-Length': '0'} 2022-09-03T06:19:25.978 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d13fddc6aaf61600-006312f1ed-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.979 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:25.981 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:25.981 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:25.982 DEBUG:boto:Method: HEAD 2022-09-03T06:19:25.982 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:19:25.983 DEBUG:boto:Data: 2022-09-03T06:19:25.984 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.985 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:25.985 DEBUG:boto:Port: 8001 2022-09-03T06:19:25.986 DEBUG:boto:Params: {} 2022-09-03T06:19:25.986 DEBUG:boto:Token: None 2022-09-03T06:19:25.987 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/c 2022-09-03T06:19:25.988 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L6og4XYoX3KXNI7fwNaSMpAvgyY= 2022-09-03T06:19:25.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L6og4XYoX3KXNI7fwNaSMpAvgyY=', 'Content-Length': '0'} 2022-09-03T06:19:25.990 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dba4702d22f32770-006312f1ed-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:25.991 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:25.992 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:25.992 DEBUG:boto:Method: HEAD 2022-09-03T06:19:25.993 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:19:25.993 DEBUG:boto:Data: 2022-09-03T06:19:25.994 DEBUG:boto:Headers: {} 2022-09-03T06:19:25.995 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:25.996 DEBUG:boto:Port: 8000 2022-09-03T06:19:25.997 DEBUG:boto:Params: {} 2022-09-03T06:19:25.997 DEBUG:boto:Token: None 2022-09-03T06:19:25.998 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:25 GMT /dsgvcy-72/c 2022-09-03T06:19:25.999 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L6og4XYoX3KXNI7fwNaSMpAvgyY= 2022-09-03T06:19:25.999 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L6og4XYoX3KXNI7fwNaSMpAvgyY=', 'Content-Length': '0'} 2022-09-03T06:19:26.001 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b9ada8a0097f69e5-006312f1ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.002 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:19:26.003 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:26.003 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:26.004 DEBUG:boto:Method: GET 2022-09-03T06:19:26.004 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:19:26.005 DEBUG:boto:Data: 2022-09-03T06:19:26.006 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.007 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:26.007 DEBUG:boto:Port: 8001 2022-09-03T06:19:26.008 DEBUG:boto:Params: {} 2022-09-03T06:19:26.008 DEBUG:boto:Token: None 2022-09-03T06:19:26.009 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/c 2022-09-03T06:19:26.010 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uBmBEdCI7isd3kiTOOtoFITW6Dg= 2022-09-03T06:19:26.011 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uBmBEdCI7isd3kiTOOtoFITW6Dg=', 'Content-Length': '0'} 2022-09-03T06:19:26.013 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b12f89a88b31c724-006312f1ee-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.061 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:26.062 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:26.062 DEBUG:boto:Method: GET 2022-09-03T06:19:26.063 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:19:26.063 DEBUG:boto:Data: 2022-09-03T06:19:26.064 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.064 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:26.065 DEBUG:boto:Port: 8000 2022-09-03T06:19:26.065 DEBUG:boto:Params: {} 2022-09-03T06:19:26.066 DEBUG:boto:Token: None 2022-09-03T06:19:26.067 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/c 2022-09-03T06:19:26.067 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uBmBEdCI7isd3kiTOOtoFITW6Dg= 2022-09-03T06:19:26.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uBmBEdCI7isd3kiTOOtoFITW6Dg=', 'Content-Length': '0'} 2022-09-03T06:19:26.072 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000370f06219a7ba5f4-006312f1ee-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.074 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:26.075 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:26.076 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:26.077 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:26.078 DEBUG:boto:Method: GET 2022-09-03T06:19:26.079 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:19:26.083 DEBUG:boto:Data: 2022-09-03T06:19:26.084 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.085 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:26.087 DEBUG:boto:Port: 8001 2022-09-03T06:19:26.088 DEBUG:boto:Params: {} 2022-09-03T06:19:26.089 DEBUG:boto:Token: None 2022-09-03T06:19:26.091 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/ 2022-09-03T06:19:26.094 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iHHWBmAzEC38Q3vZ5Gf94ZuT8GA= 2022-09-03T06:19:26.095 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iHHWBmAzEC38Q3vZ5Gf94ZuT8GA=', 'Content-Length': '0'} 2022-09-03T06:19:26.106 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d4d48ff54b1fa843-006312f1ee-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.107 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:26.109 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:26.110 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:26.111 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:26.112 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:26.113 DEBUG:boto:Method: GET 2022-09-03T06:19:26.123 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:19:26.124 DEBUG:boto:Data: 2022-09-03T06:19:26.125 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.125 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:26.126 DEBUG:boto:Port: 8000 2022-09-03T06:19:26.126 DEBUG:boto:Params: {} 2022-09-03T06:19:26.127 DEBUG:boto:Token: None 2022-09-03T06:19:26.128 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/ 2022-09-03T06:19:26.128 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iHHWBmAzEC38Q3vZ5Gf94ZuT8GA= 2022-09-03T06:19:26.129 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iHHWBmAzEC38Q3vZ5Gf94ZuT8GA=', 'Content-Length': '0'} 2022-09-03T06:19:26.132 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000098716ff51d1723ba-006312f1ee-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.134 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:26.135 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:19:26.136 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:26.137 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:26.137 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.138 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.139 DEBUG:boto:Method: GET 2022-09-03T06:19:26.140 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.141 DEBUG:boto:Data: 2022-09-03T06:19:26.142 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.143 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:26.144 DEBUG:boto:Port: 8001 2022-09-03T06:19:26.145 DEBUG:boto:Params: {} 2022-09-03T06:19:26.147 DEBUG:boto:Token: None 2022-09-03T06:19:26.148 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.149 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K/+yJr5rVG68M6JY9CrA6Nwk6xA= 2022-09-03T06:19:26.150 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K/+yJr5rVG68M6JY9CrA6Nwk6xA=', 'Content-Length': '0'} 2022-09-03T06:19:26.159 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008c30df73087acb95-006312f1ee-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.161 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:26.162 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:26.164 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.164 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.165 DEBUG:boto:Method: GET 2022-09-03T06:19:26.166 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.167 DEBUG:boto:Data: 2022-09-03T06:19:26.168 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.169 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:26.170 DEBUG:boto:Port: 8000 2022-09-03T06:19:26.171 DEBUG:boto:Params: {} 2022-09-03T06:19:26.172 DEBUG:boto:Token: None 2022-09-03T06:19:26.173 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.174 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K/+yJr5rVG68M6JY9CrA6Nwk6xA= 2022-09-03T06:19:26.176 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K/+yJr5rVG68M6JY9CrA6Nwk6xA=', 'Content-Length': '0'} 2022-09-03T06:19:26.178 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000036343451e1ddd099-006312f1ee-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.179 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:26.180 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:26.181 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.181 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.182 DEBUG:boto:Method: HEAD 2022-09-03T06:19:26.182 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.183 DEBUG:boto:Data: 2022-09-03T06:19:26.183 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.184 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:26.185 DEBUG:boto:Port: 8001 2022-09-03T06:19:26.185 DEBUG:boto:Params: {} 2022-09-03T06:19:26.186 DEBUG:boto:Token: None 2022-09-03T06:19:26.186 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.187 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r2F2fwQFPHBl1Z04I3htTaAriNc= 2022-09-03T06:19:26.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r2F2fwQFPHBl1Z04I3htTaAriNc=', 'Content-Length': '0'} 2022-09-03T06:19:26.190 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004d168569cec9244f-006312f1ee-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.191 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:26.192 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:26.192 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.193 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.194 DEBUG:boto:Method: HEAD 2022-09-03T06:19:26.195 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.196 DEBUG:boto:Data: 2022-09-03T06:19:26.197 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.198 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:26.199 DEBUG:boto:Port: 8000 2022-09-03T06:19:26.199 DEBUG:boto:Params: {} 2022-09-03T06:19:26.200 DEBUG:boto:Token: None 2022-09-03T06:19:26.201 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.202 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r2F2fwQFPHBl1Z04I3htTaAriNc= 2022-09-03T06:19:26.203 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r2F2fwQFPHBl1Z04I3htTaAriNc=', 'Content-Length': '0'} 2022-09-03T06:19:26.207 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000035a2aa6bbe3b4188-006312f1ee-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.208 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:19:26.209 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:26.210 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:26.211 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.212 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.213 DEBUG:boto:Method: GET 2022-09-03T06:19:26.214 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.215 DEBUG:boto:Data: 2022-09-03T06:19:26.216 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.219 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:26.219 DEBUG:boto:Port: 8001 2022-09-03T06:19:26.220 DEBUG:boto:Params: {} 2022-09-03T06:19:26.221 DEBUG:boto:Token: None 2022-09-03T06:19:26.223 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.224 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K/+yJr5rVG68M6JY9CrA6Nwk6xA= 2022-09-03T06:19:26.225 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K/+yJr5rVG68M6JY9CrA6Nwk6xA=', 'Content-Length': '0'} 2022-09-03T06:19:26.228 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc3dbc8a4156d1ef-006312f1ee-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.274 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:26.275 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:26.276 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.277 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.278 DEBUG:boto:Method: GET 2022-09-03T06:19:26.279 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.280 DEBUG:boto:Data: 2022-09-03T06:19:26.281 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.282 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:26.282 DEBUG:boto:Port: 8000 2022-09-03T06:19:26.283 DEBUG:boto:Params: {} 2022-09-03T06:19:26.283 DEBUG:boto:Token: None 2022-09-03T06:19:26.285 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:19:26.285 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K/+yJr5rVG68M6JY9CrA6Nwk6xA= 2022-09-03T06:19:26.286 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K/+yJr5rVG68M6JY9CrA6Nwk6xA=', 'Content-Length': '0'} 2022-09-03T06:19:26.289 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a0d2f4d870480096-006312f1ee-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.291 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:26.292 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:26.293 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:26.294 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:26.295 DEBUG:boto:Method: GET 2022-09-03T06:19:26.296 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:19:26.297 DEBUG:boto:Data: 2022-09-03T06:19:26.298 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.299 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:26.300 DEBUG:boto:Port: 8001 2022-09-03T06:19:26.301 DEBUG:boto:Params: {} 2022-09-03T06:19:26.302 DEBUG:boto:Token: None 2022-09-03T06:19:26.303 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/ 2022-09-03T06:19:26.304 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iHHWBmAzEC38Q3vZ5Gf94ZuT8GA= 2022-09-03T06:19:26.306 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iHHWBmAzEC38Q3vZ5Gf94ZuT8GA=', 'Content-Length': '0'} 2022-09-03T06:19:26.309 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005b7a5af80dec981d-006312f1ee-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.311 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:26.313 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:26.314 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:26.315 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:26.316 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:26.317 DEBUG:boto:Method: GET 2022-09-03T06:19:26.318 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:19:26.319 DEBUG:boto:Data: 2022-09-03T06:19:26.320 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.321 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:26.322 DEBUG:boto:Port: 8000 2022-09-03T06:19:26.323 DEBUG:boto:Params: {} 2022-09-03T06:19:26.325 DEBUG:boto:Token: None 2022-09-03T06:19:26.326 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/ 2022-09-03T06:19:26.327 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iHHWBmAzEC38Q3vZ5Gf94ZuT8GA= 2022-09-03T06:19:26.329 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iHHWBmAzEC38Q3vZ5Gf94ZuT8GA=', 'Content-Length': '0'} 2022-09-03T06:19:26.337 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000399882a8b0f2fe3a-006312f1ee-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.339 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:26.340 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:26.341 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:26.342 DEBUG:boto:Method: HEAD 2022-09-03T06:19:26.344 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:19:26.345 DEBUG:boto:Data: 2022-09-03T06:19:26.347 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.348 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:26.350 DEBUG:boto:Port: 8001 2022-09-03T06:19:26.351 DEBUG:boto:Params: {} 2022-09-03T06:19:26.353 DEBUG:boto:Token: None 2022-09-03T06:19:26.354 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/d 2022-09-03T06:19:26.355 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7v5mqGcj3ODdRg5O028W9DFoqYM= 2022-09-03T06:19:26.357 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7v5mqGcj3ODdRg5O028W9DFoqYM=', 'Content-Length': '0'} 2022-09-03T06:19:26.360 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001513c266b4313e0b-006312f1ee-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.362 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:26.363 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:26.365 DEBUG:boto:Method: HEAD 2022-09-03T06:19:26.366 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:19:26.367 DEBUG:boto:Data: 2022-09-03T06:19:26.368 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.369 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:26.370 DEBUG:boto:Port: 8000 2022-09-03T06:19:26.371 DEBUG:boto:Params: {} 2022-09-03T06:19:26.372 DEBUG:boto:Token: None 2022-09-03T06:19:26.373 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/d 2022-09-03T06:19:26.374 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7v5mqGcj3ODdRg5O028W9DFoqYM= 2022-09-03T06:19:26.374 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7v5mqGcj3ODdRg5O028W9DFoqYM=', 'Content-Length': '0'} 2022-09-03T06:19:26.381 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fee00063ccf7207a-006312f1ee-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.382 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:19:26.382 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:26.383 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:26.384 DEBUG:boto:Method: GET 2022-09-03T06:19:26.384 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:19:26.385 DEBUG:boto:Data: 2022-09-03T06:19:26.385 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.386 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:26.387 DEBUG:boto:Port: 8001 2022-09-03T06:19:26.387 DEBUG:boto:Params: {} 2022-09-03T06:19:26.388 DEBUG:boto:Token: None 2022-09-03T06:19:26.388 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/d 2022-09-03T06:19:26.389 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TBSdprSicL9VnUp/TDI2fNfQnek= 2022-09-03T06:19:26.390 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TBSdprSicL9VnUp/TDI2fNfQnek=', 'Content-Length': '0'} 2022-09-03T06:19:26.392 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a3087f8479b85e38-006312f1ee-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.393 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:26.394 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:26.394 DEBUG:boto:Method: GET 2022-09-03T06:19:26.395 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:19:26.395 DEBUG:boto:Data: 2022-09-03T06:19:26.396 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.399 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:26.400 DEBUG:boto:Port: 8000 2022-09-03T06:19:26.401 DEBUG:boto:Params: {} 2022-09-03T06:19:26.404 DEBUG:boto:Token: None 2022-09-03T06:19:26.406 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/d 2022-09-03T06:19:26.407 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TBSdprSicL9VnUp/TDI2fNfQnek= 2022-09-03T06:19:26.408 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TBSdprSicL9VnUp/TDI2fNfQnek=', 'Content-Length': '0'} 2022-09-03T06:19:26.410 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006dc82816a860d2da-006312f1ee-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.454 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:26.455 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:26.455 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:26.456 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:26.457 DEBUG:boto:Method: GET 2022-09-03T06:19:26.459 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:19:26.460 DEBUG:boto:Data: 2022-09-03T06:19:26.461 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.462 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:26.464 DEBUG:boto:Port: 8001 2022-09-03T06:19:26.464 DEBUG:boto:Params: {} 2022-09-03T06:19:26.465 DEBUG:boto:Token: None 2022-09-03T06:19:26.466 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/ 2022-09-03T06:19:26.466 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iHHWBmAzEC38Q3vZ5Gf94ZuT8GA= 2022-09-03T06:19:26.467 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iHHWBmAzEC38Q3vZ5Gf94ZuT8GA=', 'Content-Length': '0'} 2022-09-03T06:19:26.470 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005d297e25298d0d7f-006312f1ee-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.471 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:26.472 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:26.473 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:26.473 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:26.474 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:26.474 DEBUG:boto:Method: GET 2022-09-03T06:19:26.475 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:19:26.476 DEBUG:boto:Data: 2022-09-03T06:19:26.476 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.477 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:26.478 DEBUG:boto:Port: 8000 2022-09-03T06:19:26.479 DEBUG:boto:Params: {} 2022-09-03T06:19:26.479 DEBUG:boto:Token: None 2022-09-03T06:19:26.480 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/ 2022-09-03T06:19:26.481 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iHHWBmAzEC38Q3vZ5Gf94ZuT8GA= 2022-09-03T06:19:26.481 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iHHWBmAzEC38Q3vZ5Gf94ZuT8GA=', 'Content-Length': '0'} 2022-09-03T06:19:26.484 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005386747772782d50-006312f1ee-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.485 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:26.486 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-72 zones={test-zone3, test-zone1} 2022-09-03T06:19:26.486 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-72 zones={test-zone3, test-zone2} 2022-09-03T06:19:26.487 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:26.487 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:26.488 DEBUG:boto:Method: HEAD 2022-09-03T06:19:26.489 DEBUG:boto:Path: /dsgvcy-72/ 2022-09-03T06:19:26.490 DEBUG:boto:Data: 2022-09-03T06:19:26.491 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.491 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:26.492 DEBUG:boto:Port: 8001 2022-09-03T06:19:26.493 DEBUG:boto:Params: {} 2022-09-03T06:19:26.494 DEBUG:boto:Token: None 2022-09-03T06:19:26.494 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/ 2022-09-03T06:19:26.495 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TyLvF4edX83STqCZwqph9aNqaV0= 2022-09-03T06:19:26.496 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TyLvF4edX83STqCZwqph9aNqaV0=', 'Content-Length': '0'} 2022-09-03T06:19:26.499 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000087a79a27fa575e40-006312f1ee-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.499 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:26.500 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:26.500 DEBUG:boto:Method: HEAD 2022-09-03T06:19:26.501 DEBUG:boto:Path: /dsgvcy-72/ 2022-09-03T06:19:26.501 DEBUG:boto:Data: 2022-09-03T06:19:26.502 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.504 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:26.505 DEBUG:boto:Port: 8000 2022-09-03T06:19:26.505 DEBUG:boto:Params: {} 2022-09-03T06:19:26.506 DEBUG:boto:Token: None 2022-09-03T06:19:26.507 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/ 2022-09-03T06:19:26.507 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TyLvF4edX83STqCZwqph9aNqaV0= 2022-09-03T06:19:26.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TyLvF4edX83STqCZwqph9aNqaV0=', 'Content-Length': '0'} 2022-09-03T06:19:26.512 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009ebae756866ad282-006312f1ee-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.513 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:19:26.514 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:26.515 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:26.515 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:26.516 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:26.516 DEBUG:boto:Method: GET 2022-09-03T06:19:26.517 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:26.517 DEBUG:boto:Data: 2022-09-03T06:19:26.518 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.518 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:26.519 DEBUG:boto:Port: 8001 2022-09-03T06:19:26.520 DEBUG:boto:Params: {} 2022-09-03T06:19:26.521 DEBUG:boto:Token: None 2022-09-03T06:19:26.521 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/?versions 2022-09-03T06:19:26.522 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qts9grR1+CTuyk37KigLr8lwy+w= 2022-09-03T06:19:26.523 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qts9grR1+CTuyk37KigLr8lwy+w=', 'Content-Length': '0'} 2022-09-03T06:19:26.525 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000042472dff151e134a-006312f1ee-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.528 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:26.530 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:19:26.531 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:19:26.532 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:19:26.532 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:19:26.533 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:19:26.533 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:26.534 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:26.534 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:26.535 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:26.536 DEBUG:boto:Method: GET 2022-09-03T06:19:26.536 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:26.537 DEBUG:boto:Data: 2022-09-03T06:19:26.537 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.538 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:26.538 DEBUG:boto:Port: 8000 2022-09-03T06:19:26.539 DEBUG:boto:Params: {} 2022-09-03T06:19:26.540 DEBUG:boto:Token: None 2022-09-03T06:19:26.541 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/?versions 2022-09-03T06:19:26.541 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qts9grR1+CTuyk37KigLr8lwy+w= 2022-09-03T06:19:26.542 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qts9grR1+CTuyk37KigLr8lwy+w=', 'Content-Length': '0'} 2022-09-03T06:19:26.545 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007cd66d1fcfd4b38b-006312f1ee-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.545 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:26.546 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:19:26.547 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:19:26.547 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:19:26.548 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:19:26.549 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:26.549 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:26.550 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:26.551 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:26.554 DEBUG:boto:Method: GET 2022-09-03T06:19:26.555 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:26.556 DEBUG:boto:Data: 2022-09-03T06:19:26.557 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.558 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:26.559 DEBUG:boto:Port: 8001 2022-09-03T06:19:26.560 DEBUG:boto:Params: {} 2022-09-03T06:19:26.561 DEBUG:boto:Token: None 2022-09-03T06:19:26.563 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/?versions 2022-09-03T06:19:26.564 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qts9grR1+CTuyk37KigLr8lwy+w= 2022-09-03T06:19:26.565 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qts9grR1+CTuyk37KigLr8lwy+w=', 'Content-Length': '0'} 2022-09-03T06:19:26.569 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a69ce0e970733816-006312f1ee-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.571 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:26.573 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:26.574 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:26.576 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:26.577 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:26.578 DEBUG:boto:Method: GET 2022-09-03T06:19:26.579 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:19:26.580 DEBUG:boto:Data: 2022-09-03T06:19:26.581 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.582 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:26.584 DEBUG:boto:Port: 8000 2022-09-03T06:19:26.585 DEBUG:boto:Params: {} 2022-09-03T06:19:26.585 DEBUG:boto:Token: None 2022-09-03T06:19:26.586 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/?versions 2022-09-03T06:19:26.587 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qts9grR1+CTuyk37KigLr8lwy+w= 2022-09-03T06:19:26.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qts9grR1+CTuyk37KigLr8lwy+w=', 'Content-Length': '0'} 2022-09-03T06:19:26.592 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000643385c0d457da2a-006312f1ee-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.594 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:26.596 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:19:26.597 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:26.597 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:26.598 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.599 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.599 DEBUG:boto:Method: GET 2022-09-03T06:19:26.600 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.600 DEBUG:boto:Data: 2022-09-03T06:19:26.601 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.601 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:26.602 DEBUG:boto:Port: 8001 2022-09-03T06:19:26.603 DEBUG:boto:Params: {} 2022-09-03T06:19:26.604 DEBUG:boto:Token: None 2022-09-03T06:19:26.605 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.606 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M34viFRKIfOdJkMUc69uPRRmjCo= 2022-09-03T06:19:26.607 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M34viFRKIfOdJkMUc69uPRRmjCo=', 'Content-Length': '0'} 2022-09-03T06:19:26.613 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006311838c19625854-006312f1ee-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.655 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:26.656 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:26.658 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.659 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.661 DEBUG:boto:Method: GET 2022-09-03T06:19:26.662 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.663 DEBUG:boto:Data: 2022-09-03T06:19:26.664 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.664 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:26.665 DEBUG:boto:Port: 8000 2022-09-03T06:19:26.665 DEBUG:boto:Params: {} 2022-09-03T06:19:26.666 DEBUG:boto:Token: None 2022-09-03T06:19:26.667 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.668 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M34viFRKIfOdJkMUc69uPRRmjCo= 2022-09-03T06:19:26.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M34viFRKIfOdJkMUc69uPRRmjCo=', 'Content-Length': '0'} 2022-09-03T06:19:26.674 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ae5eb4985dccfda1-006312f1ee-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.675 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:26.676 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:26.676 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.677 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.677 DEBUG:boto:Method: HEAD 2022-09-03T06:19:26.680 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.681 DEBUG:boto:Data: 2022-09-03T06:19:26.682 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.683 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:26.684 DEBUG:boto:Port: 8001 2022-09-03T06:19:26.685 DEBUG:boto:Params: {} 2022-09-03T06:19:26.686 DEBUG:boto:Token: None 2022-09-03T06:19:26.687 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.688 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HoJo6jZ4dOR7U0azYUjxMAZ1HPo= 2022-09-03T06:19:26.689 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HoJo6jZ4dOR7U0azYUjxMAZ1HPo=', 'Content-Length': '0'} 2022-09-03T06:19:26.697 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a129ca5cf8af8d9-006312f1ee-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.698 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:26.698 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:26.699 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.700 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.700 DEBUG:boto:Method: HEAD 2022-09-03T06:19:26.701 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.701 DEBUG:boto:Data: 2022-09-03T06:19:26.702 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.703 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:26.703 DEBUG:boto:Port: 8000 2022-09-03T06:19:26.704 DEBUG:boto:Params: {} 2022-09-03T06:19:26.704 DEBUG:boto:Token: None 2022-09-03T06:19:26.705 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.706 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HoJo6jZ4dOR7U0azYUjxMAZ1HPo= 2022-09-03T06:19:26.706 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HoJo6jZ4dOR7U0azYUjxMAZ1HPo=', 'Content-Length': '0'} 2022-09-03T06:19:26.710 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c48a1b14fbcaed28-006312f1ee-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.711 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:19:26.711 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:26.712 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:26.712 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.713 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.713 DEBUG:boto:Method: GET 2022-09-03T06:19:26.714 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.715 DEBUG:boto:Data: 2022-09-03T06:19:26.715 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.716 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:26.716 DEBUG:boto:Port: 8001 2022-09-03T06:19:26.717 DEBUG:boto:Params: {} 2022-09-03T06:19:26.717 DEBUG:boto:Token: None 2022-09-03T06:19:26.720 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.721 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M34viFRKIfOdJkMUc69uPRRmjCo= 2022-09-03T06:19:26.722 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M34viFRKIfOdJkMUc69uPRRmjCo=', 'Content-Length': '0'} 2022-09-03T06:19:26.726 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007ba17964ddcd8d40-006312f1ee-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.728 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:26.729 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:26.729 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.731 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.731 DEBUG:boto:Method: GET 2022-09-03T06:19:26.732 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.733 DEBUG:boto:Data: 2022-09-03T06:19:26.733 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.734 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:26.735 DEBUG:boto:Port: 8000 2022-09-03T06:19:26.736 DEBUG:boto:Params: {} 2022-09-03T06:19:26.737 DEBUG:boto:Token: None 2022-09-03T06:19:26.738 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:19:26.739 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M34viFRKIfOdJkMUc69uPRRmjCo= 2022-09-03T06:19:26.741 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M34viFRKIfOdJkMUc69uPRRmjCo=', 'Content-Length': '0'} 2022-09-03T06:19:26.744 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000ed840467366f8c1-006312f1ee-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.786 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:26.787 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:26.788 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:26.789 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:26.790 DEBUG:boto:Method: GET 2022-09-03T06:19:26.791 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:19:26.792 DEBUG:boto:Data: 2022-09-03T06:19:26.794 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.795 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:26.795 DEBUG:boto:Port: 8001 2022-09-03T06:19:26.796 DEBUG:boto:Params: {} 2022-09-03T06:19:26.797 DEBUG:boto:Token: None 2022-09-03T06:19:26.798 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/ 2022-09-03T06:19:26.799 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iHHWBmAzEC38Q3vZ5Gf94ZuT8GA= 2022-09-03T06:19:26.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iHHWBmAzEC38Q3vZ5Gf94ZuT8GA=', 'Content-Length': '0'} 2022-09-03T06:19:26.803 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003cea0005d9a7d583-006312f1ee-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.805 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:26.807 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:26.807 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:26.808 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:26.809 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:26.810 DEBUG:boto:Method: GET 2022-09-03T06:19:26.811 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:19:26.812 DEBUG:boto:Data: 2022-09-03T06:19:26.813 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.814 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:26.815 DEBUG:boto:Port: 8000 2022-09-03T06:19:26.816 DEBUG:boto:Params: {} 2022-09-03T06:19:26.817 DEBUG:boto:Token: None 2022-09-03T06:19:26.817 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/ 2022-09-03T06:19:26.818 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iHHWBmAzEC38Q3vZ5Gf94ZuT8GA= 2022-09-03T06:19:26.819 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iHHWBmAzEC38Q3vZ5Gf94ZuT8GA=', 'Content-Length': '0'} 2022-09-03T06:19:26.823 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004f7fd589e53ab008-006312f1ee-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.824 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:26.825 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:26.826 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:26.827 DEBUG:boto:Method: HEAD 2022-09-03T06:19:26.829 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:19:26.830 DEBUG:boto:Data: 2022-09-03T06:19:26.831 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.831 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:26.832 DEBUG:boto:Port: 8001 2022-09-03T06:19:26.833 DEBUG:boto:Params: {} 2022-09-03T06:19:26.834 DEBUG:boto:Token: None 2022-09-03T06:19:26.835 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/a 2022-09-03T06:19:26.837 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i2s/WKTSxLjaQwFR4YSNHIWwwfo= 2022-09-03T06:19:26.838 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i2s/WKTSxLjaQwFR4YSNHIWwwfo=', 'Content-Length': '0'} 2022-09-03T06:19:26.841 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000461644bfa79f2c57-006312f1ee-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.843 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:26.844 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:26.845 DEBUG:boto:Method: HEAD 2022-09-03T06:19:26.846 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:19:26.846 DEBUG:boto:Data: 2022-09-03T06:19:26.847 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.848 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:26.848 DEBUG:boto:Port: 8000 2022-09-03T06:19:26.850 DEBUG:boto:Params: {} 2022-09-03T06:19:26.851 DEBUG:boto:Token: None 2022-09-03T06:19:26.852 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/a 2022-09-03T06:19:26.853 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i2s/WKTSxLjaQwFR4YSNHIWwwfo= 2022-09-03T06:19:26.854 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i2s/WKTSxLjaQwFR4YSNHIWwwfo=', 'Content-Length': '0'} 2022-09-03T06:19:26.857 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000078f2c2d0afdf07b8-006312f1ee-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.857 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:19:26.858 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:26.859 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:26.859 DEBUG:boto:Method: GET 2022-09-03T06:19:26.860 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:19:26.861 DEBUG:boto:Data: 2022-09-03T06:19:26.862 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.863 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:26.863 DEBUG:boto:Port: 8001 2022-09-03T06:19:26.864 DEBUG:boto:Params: {} 2022-09-03T06:19:26.865 DEBUG:boto:Token: None 2022-09-03T06:19:26.865 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/a 2022-09-03T06:19:26.866 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y5rTtJLI30kkFOlCj79+1Yz3j/U= 2022-09-03T06:19:26.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y5rTtJLI30kkFOlCj79+1Yz3j/U=', 'Content-Length': '0'} 2022-09-03T06:19:26.876 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d40e2b030c1e61de-006312f1ee-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.877 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:19:26.878 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:19:26.879 DEBUG:boto:Method: GET 2022-09-03T06:19:26.880 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:19:26.881 DEBUG:boto:Data: 2022-09-03T06:19:26.882 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.883 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:26.884 DEBUG:boto:Port: 8000 2022-09-03T06:19:26.885 DEBUG:boto:Params: {} 2022-09-03T06:19:26.886 DEBUG:boto:Token: None 2022-09-03T06:19:26.887 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/a 2022-09-03T06:19:26.888 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y5rTtJLI30kkFOlCj79+1Yz3j/U= 2022-09-03T06:19:26.890 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y5rTtJLI30kkFOlCj79+1Yz3j/U=', 'Content-Length': '0'} 2022-09-03T06:19:26.893 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008ca3580abd472e55-006312f1ee-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.894 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:26.895 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:26.895 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:26.896 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:26.896 DEBUG:boto:Method: GET 2022-09-03T06:19:26.897 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:19:26.897 DEBUG:boto:Data: 2022-09-03T06:19:26.898 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.898 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:26.899 DEBUG:boto:Port: 8001 2022-09-03T06:19:26.900 DEBUG:boto:Params: {} 2022-09-03T06:19:26.900 DEBUG:boto:Token: None 2022-09-03T06:19:26.901 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/ 2022-09-03T06:19:26.902 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iHHWBmAzEC38Q3vZ5Gf94ZuT8GA= 2022-09-03T06:19:26.902 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iHHWBmAzEC38Q3vZ5Gf94ZuT8GA=', 'Content-Length': '0'} 2022-09-03T06:19:26.906 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000097906745b499133e-006312f1ee-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.950 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:26.952 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:26.952 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:26.953 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:26.954 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:19:26.954 DEBUG:boto:Method: GET 2022-09-03T06:19:26.955 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:19:26.956 DEBUG:boto:Data: 2022-09-03T06:19:26.956 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.957 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:26.957 DEBUG:boto:Port: 8000 2022-09-03T06:19:26.958 DEBUG:boto:Params: {} 2022-09-03T06:19:26.959 DEBUG:boto:Token: None 2022-09-03T06:19:26.959 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/ 2022-09-03T06:19:26.960 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iHHWBmAzEC38Q3vZ5Gf94ZuT8GA= 2022-09-03T06:19:26.961 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iHHWBmAzEC38Q3vZ5Gf94ZuT8GA=', 'Content-Length': '0'} 2022-09-03T06:19:26.963 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000064ce74514934c20e-006312f1ee-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.964 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:26.965 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:19:26.966 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:26.967 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:26.967 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:26.968 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:26.969 DEBUG:boto:Method: GET 2022-09-03T06:19:26.970 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:19:26.971 DEBUG:boto:Data: 2022-09-03T06:19:26.972 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.977 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:26.977 DEBUG:boto:Port: 8001 2022-09-03T06:19:26.978 DEBUG:boto:Params: {} 2022-09-03T06:19:26.982 DEBUG:boto:Token: None 2022-09-03T06:19:26.982 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:19:26.983 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nNvdqpGh2hkVseCDNyrrnml16OM= 2022-09-03T06:19:26.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nNvdqpGh2hkVseCDNyrrnml16OM=', 'Content-Length': '0'} 2022-09-03T06:19:26.986 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a95a33d69c9c7a67-006312f1ee-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:26.987 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:26.988 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:26.988 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:26.989 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:26.989 DEBUG:boto:Method: GET 2022-09-03T06:19:26.990 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:19:26.991 DEBUG:boto:Data: 2022-09-03T06:19:26.991 DEBUG:boto:Headers: {} 2022-09-03T06:19:26.992 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:26.992 DEBUG:boto:Port: 8000 2022-09-03T06:19:26.993 DEBUG:boto:Params: {} 2022-09-03T06:19:26.994 DEBUG:boto:Token: None 2022-09-03T06:19:26.994 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:26 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:19:26.995 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nNvdqpGh2hkVseCDNyrrnml16OM= 2022-09-03T06:19:27.006 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nNvdqpGh2hkVseCDNyrrnml16OM=', 'Content-Length': '0'} 2022-09-03T06:19:27.010 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021867a7b818d3fe2-006312f1ef-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.011 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:27.012 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:27.013 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:27.013 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:27.015 DEBUG:boto:Method: HEAD 2022-09-03T06:19:27.016 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:19:27.017 DEBUG:boto:Data: 2022-09-03T06:19:27.017 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.018 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:27.019 DEBUG:boto:Port: 8001 2022-09-03T06:19:27.020 DEBUG:boto:Params: {} 2022-09-03T06:19:27.021 DEBUG:boto:Token: None 2022-09-03T06:19:27.022 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:19:27.023 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dikOm3B80Y9pW0Duv3ZKUZ3Epe0= 2022-09-03T06:19:27.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dikOm3B80Y9pW0Duv3ZKUZ3Epe0=', 'Content-Length': '0'} 2022-09-03T06:19:27.031 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b8bc2d17b18e98e-006312f1ef-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.033 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:27.034 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:27.035 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:27.036 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:27.037 DEBUG:boto:Method: HEAD 2022-09-03T06:19:27.038 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:19:27.039 DEBUG:boto:Data: 2022-09-03T06:19:27.040 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.040 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:27.041 DEBUG:boto:Port: 8000 2022-09-03T06:19:27.042 DEBUG:boto:Params: {} 2022-09-03T06:19:27.044 DEBUG:boto:Token: None 2022-09-03T06:19:27.044 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:19:27.045 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dikOm3B80Y9pW0Duv3ZKUZ3Epe0= 2022-09-03T06:19:27.046 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dikOm3B80Y9pW0Duv3ZKUZ3Epe0=', 'Content-Length': '0'} 2022-09-03T06:19:27.055 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000095768300d94dc624-006312f1ef-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.056 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:19:27.057 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:27.057 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:27.059 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:27.060 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:27.061 DEBUG:boto:Method: GET 2022-09-03T06:19:27.062 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:19:27.063 DEBUG:boto:Data: 2022-09-03T06:19:27.065 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.065 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:27.066 DEBUG:boto:Port: 8001 2022-09-03T06:19:27.067 DEBUG:boto:Params: {} 2022-09-03T06:19:27.068 DEBUG:boto:Token: None 2022-09-03T06:19:27.069 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:19:27.070 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R7QK6EHUwsKJvpMVY9z75OqekcE= 2022-09-03T06:19:27.071 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R7QK6EHUwsKJvpMVY9z75OqekcE=', 'Content-Length': '0'} 2022-09-03T06:19:27.074 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000381df3678c370f46-006312f1ef-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.115 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:27.116 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:27.117 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:27.117 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:19:27.118 DEBUG:boto:Method: GET 2022-09-03T06:19:27.119 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:19:27.121 DEBUG:boto:Data: 2022-09-03T06:19:27.123 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.124 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:27.125 DEBUG:boto:Port: 8000 2022-09-03T06:19:27.126 DEBUG:boto:Params: {} 2022-09-03T06:19:27.127 DEBUG:boto:Token: None 2022-09-03T06:19:27.127 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:19:27.128 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R7QK6EHUwsKJvpMVY9z75OqekcE= 2022-09-03T06:19:27.129 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R7QK6EHUwsKJvpMVY9z75OqekcE=', 'Content-Length': '0'} 2022-09-03T06:19:27.131 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002097ea1228b183ee-006312f1ef-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.134 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:27.135 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:27.137 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:27.138 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:27.139 DEBUG:boto:Method: GET 2022-09-03T06:19:27.140 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:19:27.141 DEBUG:boto:Data: 2022-09-03T06:19:27.142 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.143 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:27.145 DEBUG:boto:Port: 8001 2022-09-03T06:19:27.146 DEBUG:boto:Params: {} 2022-09-03T06:19:27.147 DEBUG:boto:Token: None 2022-09-03T06:19:27.148 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/ 2022-09-03T06:19:27.148 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BK45BOvsegmR80LzcDBRZfMatD0= 2022-09-03T06:19:27.149 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BK45BOvsegmR80LzcDBRZfMatD0=', 'Content-Length': '0'} 2022-09-03T06:19:27.152 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000086cd4a28844c7f05-006312f1ef-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.153 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:27.154 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:27.155 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:27.156 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:27.157 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:27.158 DEBUG:boto:Method: GET 2022-09-03T06:19:27.159 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:19:27.160 DEBUG:boto:Data: 2022-09-03T06:19:27.160 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.161 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:27.161 DEBUG:boto:Port: 8000 2022-09-03T06:19:27.162 DEBUG:boto:Params: {} 2022-09-03T06:19:27.163 DEBUG:boto:Token: None 2022-09-03T06:19:27.163 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/ 2022-09-03T06:19:27.164 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BK45BOvsegmR80LzcDBRZfMatD0= 2022-09-03T06:19:27.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BK45BOvsegmR80LzcDBRZfMatD0=', 'Content-Length': '0'} 2022-09-03T06:19:27.169 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b49304b26de97982-006312f1ef-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.170 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:27.172 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:27.173 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:27.175 DEBUG:boto:Method: HEAD 2022-09-03T06:19:27.176 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:19:27.177 DEBUG:boto:Data: 2022-09-03T06:19:27.178 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.178 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:27.179 DEBUG:boto:Port: 8001 2022-09-03T06:19:27.180 DEBUG:boto:Params: {} 2022-09-03T06:19:27.181 DEBUG:boto:Token: None 2022-09-03T06:19:27.182 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/b 2022-09-03T06:19:27.184 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yAZ36kV73KIBt+0aBnOvAdauiiw= 2022-09-03T06:19:27.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yAZ36kV73KIBt+0aBnOvAdauiiw=', 'Content-Length': '0'} 2022-09-03T06:19:27.188 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f69281583e8d7c23-006312f1ef-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.188 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:27.189 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:27.189 DEBUG:boto:Method: HEAD 2022-09-03T06:19:27.190 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:19:27.191 DEBUG:boto:Data: 2022-09-03T06:19:27.191 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.192 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:27.192 DEBUG:boto:Port: 8000 2022-09-03T06:19:27.193 DEBUG:boto:Params: {} 2022-09-03T06:19:27.193 DEBUG:boto:Token: None 2022-09-03T06:19:27.198 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/b 2022-09-03T06:19:27.200 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yAZ36kV73KIBt+0aBnOvAdauiiw= 2022-09-03T06:19:27.201 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yAZ36kV73KIBt+0aBnOvAdauiiw=', 'Content-Length': '0'} 2022-09-03T06:19:27.204 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000028c2554c39d33912-006312f1ef-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.205 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:19:27.207 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:27.207 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:27.208 DEBUG:boto:Method: GET 2022-09-03T06:19:27.208 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:19:27.209 DEBUG:boto:Data: 2022-09-03T06:19:27.209 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.210 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:27.210 DEBUG:boto:Port: 8001 2022-09-03T06:19:27.211 DEBUG:boto:Params: {} 2022-09-03T06:19:27.212 DEBUG:boto:Token: None 2022-09-03T06:19:27.213 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/b 2022-09-03T06:19:27.213 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V5dtvhYEKT0sB9SjaTLS4gAK7KY= 2022-09-03T06:19:27.214 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V5dtvhYEKT0sB9SjaTLS4gAK7KY=', 'Content-Length': '0'} 2022-09-03T06:19:27.226 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000904f70d1ddb47245-006312f1ef-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.238 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:19:27.239 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:19:27.240 DEBUG:boto:Method: GET 2022-09-03T06:19:27.240 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:19:27.241 DEBUG:boto:Data: 2022-09-03T06:19:27.241 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.244 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:27.245 DEBUG:boto:Port: 8000 2022-09-03T06:19:27.246 DEBUG:boto:Params: {} 2022-09-03T06:19:27.248 DEBUG:boto:Token: None 2022-09-03T06:19:27.249 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/b 2022-09-03T06:19:27.250 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V5dtvhYEKT0sB9SjaTLS4gAK7KY= 2022-09-03T06:19:27.251 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V5dtvhYEKT0sB9SjaTLS4gAK7KY=', 'Content-Length': '0'} 2022-09-03T06:19:27.257 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065111559f4dee9ed-006312f1ef-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.258 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:27.259 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:27.260 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:27.261 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:27.262 DEBUG:boto:Method: GET 2022-09-03T06:19:27.263 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:19:27.264 DEBUG:boto:Data: 2022-09-03T06:19:27.264 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.265 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:27.266 DEBUG:boto:Port: 8001 2022-09-03T06:19:27.266 DEBUG:boto:Params: {} 2022-09-03T06:19:27.267 DEBUG:boto:Token: None 2022-09-03T06:19:27.268 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/ 2022-09-03T06:19:27.268 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BK45BOvsegmR80LzcDBRZfMatD0= 2022-09-03T06:19:27.269 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BK45BOvsegmR80LzcDBRZfMatD0=', 'Content-Length': '0'} 2022-09-03T06:19:27.272 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000f66d1e9112339cc-006312f1ef-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.273 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:27.274 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:27.274 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:27.276 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:27.276 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:19:27.277 DEBUG:boto:Method: GET 2022-09-03T06:19:27.277 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:19:27.278 DEBUG:boto:Data: 2022-09-03T06:19:27.279 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.279 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:27.280 DEBUG:boto:Port: 8000 2022-09-03T06:19:27.280 DEBUG:boto:Params: {} 2022-09-03T06:19:27.281 DEBUG:boto:Token: None 2022-09-03T06:19:27.281 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/ 2022-09-03T06:19:27.282 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BK45BOvsegmR80LzcDBRZfMatD0= 2022-09-03T06:19:27.283 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BK45BOvsegmR80LzcDBRZfMatD0=', 'Content-Length': '0'} 2022-09-03T06:19:27.288 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e5079f929adfd4e8-006312f1ef-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.290 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:27.292 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:19:27.293 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:27.294 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:27.295 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.297 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.298 DEBUG:boto:Method: GET 2022-09-03T06:19:27.300 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.301 DEBUG:boto:Data: 2022-09-03T06:19:27.301 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.302 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:27.304 DEBUG:boto:Port: 8001 2022-09-03T06:19:27.305 DEBUG:boto:Params: {} 2022-09-03T06:19:27.306 DEBUG:boto:Token: None 2022-09-03T06:19:27.314 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.317 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OU+GlHdPw2wcdHNzBvPpM/lmWBE= 2022-09-03T06:19:27.318 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OU+GlHdPw2wcdHNzBvPpM/lmWBE=', 'Content-Length': '0'} 2022-09-03T06:19:27.325 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000015dd1e7ef88625c9-006312f1ef-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.328 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:27.329 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:27.330 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.333 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.335 DEBUG:boto:Method: GET 2022-09-03T06:19:27.336 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.337 DEBUG:boto:Data: 2022-09-03T06:19:27.339 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.340 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:27.341 DEBUG:boto:Port: 8000 2022-09-03T06:19:27.351 DEBUG:boto:Params: {} 2022-09-03T06:19:27.352 DEBUG:boto:Token: None 2022-09-03T06:19:27.353 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.354 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OU+GlHdPw2wcdHNzBvPpM/lmWBE= 2022-09-03T06:19:27.355 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OU+GlHdPw2wcdHNzBvPpM/lmWBE=', 'Content-Length': '0'} 2022-09-03T06:19:27.357 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f1a3c4e6a7408ee-006312f1ef-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.358 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:27.358 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:27.359 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.360 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.360 DEBUG:boto:Method: HEAD 2022-09-03T06:19:27.361 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.361 DEBUG:boto:Data: 2022-09-03T06:19:27.362 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.362 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:27.363 DEBUG:boto:Port: 8001 2022-09-03T06:19:27.364 DEBUG:boto:Params: {} 2022-09-03T06:19:27.364 DEBUG:boto:Token: None 2022-09-03T06:19:27.365 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:00Dc24azmRFSuN2yT9FR3dBxAIU= 2022-09-03T06:19:27.366 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:00Dc24azmRFSuN2yT9FR3dBxAIU=', 'Content-Length': '0'} 2022-09-03T06:19:27.371 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000954097d1b25ebd57-006312f1ef-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.376 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:27.378 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:27.379 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.380 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.392 DEBUG:boto:Method: HEAD 2022-09-03T06:19:27.393 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.401 DEBUG:boto:Data: 2022-09-03T06:19:27.403 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.403 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:27.414 DEBUG:boto:Port: 8000 2022-09-03T06:19:27.415 DEBUG:boto:Params: {} 2022-09-03T06:19:27.417 DEBUG:boto:Token: None 2022-09-03T06:19:27.418 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.419 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:00Dc24azmRFSuN2yT9FR3dBxAIU= 2022-09-03T06:19:27.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:00Dc24azmRFSuN2yT9FR3dBxAIU=', 'Content-Length': '0'} 2022-09-03T06:19:27.424 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000004363e2df97fc944-006312f1ef-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.429 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:19:27.433 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:27.434 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:27.435 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.436 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.436 DEBUG:boto:Method: GET 2022-09-03T06:19:27.437 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.437 DEBUG:boto:Data: 2022-09-03T06:19:27.438 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.439 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:27.439 DEBUG:boto:Port: 8001 2022-09-03T06:19:27.440 DEBUG:boto:Params: {} 2022-09-03T06:19:27.440 DEBUG:boto:Token: None 2022-09-03T06:19:27.441 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.442 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OU+GlHdPw2wcdHNzBvPpM/lmWBE= 2022-09-03T06:19:27.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OU+GlHdPw2wcdHNzBvPpM/lmWBE=', 'Content-Length': '0'} 2022-09-03T06:19:27.446 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000032eb6bbc302f0fde-006312f1ef-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.448 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:27.449 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:27.450 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.451 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.452 DEBUG:boto:Method: GET 2022-09-03T06:19:27.452 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.453 DEBUG:boto:Data: 2022-09-03T06:19:27.454 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.455 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:27.457 DEBUG:boto:Port: 8000 2022-09-03T06:19:27.458 DEBUG:boto:Params: {} 2022-09-03T06:19:27.459 DEBUG:boto:Token: None 2022-09-03T06:19:27.460 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:19:27.461 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OU+GlHdPw2wcdHNzBvPpM/lmWBE= 2022-09-03T06:19:27.462 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OU+GlHdPw2wcdHNzBvPpM/lmWBE=', 'Content-Length': '0'} 2022-09-03T06:19:27.465 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d4b19f595fe64707-006312f1ef-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.466 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:27.467 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:27.468 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:27.468 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:27.469 DEBUG:boto:Method: GET 2022-09-03T06:19:27.471 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:19:27.471 DEBUG:boto:Data: 2022-09-03T06:19:27.472 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.472 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:27.473 DEBUG:boto:Port: 8001 2022-09-03T06:19:27.475 DEBUG:boto:Params: {} 2022-09-03T06:19:27.475 DEBUG:boto:Token: None 2022-09-03T06:19:27.476 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/ 2022-09-03T06:19:27.477 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BK45BOvsegmR80LzcDBRZfMatD0= 2022-09-03T06:19:27.478 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BK45BOvsegmR80LzcDBRZfMatD0=', 'Content-Length': '0'} 2022-09-03T06:19:27.480 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000ad36f1352f5213e-006312f1ef-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.482 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:27.484 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:27.485 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:27.486 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:27.487 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:27.488 DEBUG:boto:Method: GET 2022-09-03T06:19:27.489 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:19:27.491 DEBUG:boto:Data: 2022-09-03T06:19:27.491 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.492 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:27.493 DEBUG:boto:Port: 8000 2022-09-03T06:19:27.493 DEBUG:boto:Params: {} 2022-09-03T06:19:27.494 DEBUG:boto:Token: None 2022-09-03T06:19:27.494 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/ 2022-09-03T06:19:27.495 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BK45BOvsegmR80LzcDBRZfMatD0= 2022-09-03T06:19:27.496 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BK45BOvsegmR80LzcDBRZfMatD0=', 'Content-Length': '0'} 2022-09-03T06:19:27.499 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000039dc8bf3155e8be3-006312f1ef-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.501 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:27.503 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:27.504 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:27.505 DEBUG:boto:Method: HEAD 2022-09-03T06:19:27.506 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:19:27.507 DEBUG:boto:Data: 2022-09-03T06:19:27.508 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.508 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:27.509 DEBUG:boto:Port: 8001 2022-09-03T06:19:27.510 DEBUG:boto:Params: {} 2022-09-03T06:19:27.512 DEBUG:boto:Token: None 2022-09-03T06:19:27.512 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/c 2022-09-03T06:19:27.513 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GuVjVx1O5rArUzC7AccOjZVDZcE= 2022-09-03T06:19:27.514 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GuVjVx1O5rArUzC7AccOjZVDZcE=', 'Content-Length': '0'} 2022-09-03T06:19:27.524 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000072aa503a4c4ade45-006312f1ef-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.526 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:27.528 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:27.529 DEBUG:boto:Method: HEAD 2022-09-03T06:19:27.537 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:19:27.538 DEBUG:boto:Data: 2022-09-03T06:19:27.538 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.539 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:27.539 DEBUG:boto:Port: 8000 2022-09-03T06:19:27.540 DEBUG:boto:Params: {} 2022-09-03T06:19:27.540 DEBUG:boto:Token: None 2022-09-03T06:19:27.542 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/c 2022-09-03T06:19:27.542 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GuVjVx1O5rArUzC7AccOjZVDZcE= 2022-09-03T06:19:27.543 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GuVjVx1O5rArUzC7AccOjZVDZcE=', 'Content-Length': '0'} 2022-09-03T06:19:27.547 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cf643f7ebf7b7b48-006312f1ef-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.548 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:19:27.549 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:27.549 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:27.550 DEBUG:boto:Method: GET 2022-09-03T06:19:27.551 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:19:27.551 DEBUG:boto:Data: 2022-09-03T06:19:27.552 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.552 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:27.553 DEBUG:boto:Port: 8001 2022-09-03T06:19:27.554 DEBUG:boto:Params: {} 2022-09-03T06:19:27.554 DEBUG:boto:Token: None 2022-09-03T06:19:27.556 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/c 2022-09-03T06:19:27.557 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZvYEfXvwwHUgXmD/ifGiXYkC0SQ= 2022-09-03T06:19:27.558 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZvYEfXvwwHUgXmD/ifGiXYkC0SQ=', 'Content-Length': '0'} 2022-09-03T06:19:27.561 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007a08f8ceddeb9e41-006312f1ef-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.563 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:19:27.564 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:19:27.565 DEBUG:boto:Method: GET 2022-09-03T06:19:27.566 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:19:27.568 DEBUG:boto:Data: 2022-09-03T06:19:27.569 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.570 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:27.571 DEBUG:boto:Port: 8000 2022-09-03T06:19:27.572 DEBUG:boto:Params: {} 2022-09-03T06:19:27.573 DEBUG:boto:Token: None 2022-09-03T06:19:27.574 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/c 2022-09-03T06:19:27.575 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZvYEfXvwwHUgXmD/ifGiXYkC0SQ= 2022-09-03T06:19:27.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZvYEfXvwwHUgXmD/ifGiXYkC0SQ=', 'Content-Length': '0'} 2022-09-03T06:19:27.580 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018e2337af1bb2dba-006312f1ef-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.625 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:27.627 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:27.628 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:27.629 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:27.630 DEBUG:boto:Method: GET 2022-09-03T06:19:27.631 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:19:27.634 DEBUG:boto:Data: 2022-09-03T06:19:27.634 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.635 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:27.642 DEBUG:boto:Port: 8001 2022-09-03T06:19:27.644 DEBUG:boto:Params: {} 2022-09-03T06:19:27.645 DEBUG:boto:Token: None 2022-09-03T06:19:27.646 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/ 2022-09-03T06:19:27.646 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BK45BOvsegmR80LzcDBRZfMatD0= 2022-09-03T06:19:27.647 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BK45BOvsegmR80LzcDBRZfMatD0=', 'Content-Length': '0'} 2022-09-03T06:19:27.650 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008d2b0400517dc5fd-006312f1ef-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.651 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:27.652 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:27.653 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:27.653 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:27.654 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:19:27.655 DEBUG:boto:Method: GET 2022-09-03T06:19:27.655 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:19:27.656 DEBUG:boto:Data: 2022-09-03T06:19:27.656 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.657 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:27.657 DEBUG:boto:Port: 8000 2022-09-03T06:19:27.658 DEBUG:boto:Params: {} 2022-09-03T06:19:27.658 DEBUG:boto:Token: None 2022-09-03T06:19:27.659 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/ 2022-09-03T06:19:27.660 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BK45BOvsegmR80LzcDBRZfMatD0= 2022-09-03T06:19:27.660 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BK45BOvsegmR80LzcDBRZfMatD0=', 'Content-Length': '0'} 2022-09-03T06:19:27.663 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dc2b88bceb91aff5-006312f1ef-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.664 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:27.665 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:19:27.666 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:27.666 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:27.667 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.668 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.668 DEBUG:boto:Method: GET 2022-09-03T06:19:27.669 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.669 DEBUG:boto:Data: 2022-09-03T06:19:27.670 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.671 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:27.672 DEBUG:boto:Port: 8001 2022-09-03T06:19:27.673 DEBUG:boto:Params: {} 2022-09-03T06:19:27.674 DEBUG:boto:Token: None 2022-09-03T06:19:27.675 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.676 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UF2lGR4enrcqNr0XfU0Sg4r/6CU= 2022-09-03T06:19:27.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UF2lGR4enrcqNr0XfU0Sg4r/6CU=', 'Content-Length': '0'} 2022-09-03T06:19:27.683 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d9e2ac13f99769c3-006312f1ef-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.726 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:27.728 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:27.728 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.729 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.730 DEBUG:boto:Method: GET 2022-09-03T06:19:27.731 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.732 DEBUG:boto:Data: 2022-09-03T06:19:27.733 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.734 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:27.735 DEBUG:boto:Port: 8000 2022-09-03T06:19:27.738 DEBUG:boto:Params: {} 2022-09-03T06:19:27.739 DEBUG:boto:Token: None 2022-09-03T06:19:27.740 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.742 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UF2lGR4enrcqNr0XfU0Sg4r/6CU= 2022-09-03T06:19:27.743 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UF2lGR4enrcqNr0XfU0Sg4r/6CU=', 'Content-Length': '0'} 2022-09-03T06:19:27.747 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc88c48e3c212f64-006312f1ef-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.748 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:27.748 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:27.749 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.749 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.750 DEBUG:boto:Method: HEAD 2022-09-03T06:19:27.751 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.752 DEBUG:boto:Data: 2022-09-03T06:19:27.753 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.753 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:27.754 DEBUG:boto:Port: 8001 2022-09-03T06:19:27.757 DEBUG:boto:Params: {} 2022-09-03T06:19:27.758 DEBUG:boto:Token: None 2022-09-03T06:19:27.759 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.760 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9cAKSkwE8C90kNPKj374BLffjEo= 2022-09-03T06:19:27.761 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9cAKSkwE8C90kNPKj374BLffjEo=', 'Content-Length': '0'} 2022-09-03T06:19:27.763 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e3ccc967a445ea37-006312f1ef-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.764 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:27.764 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:27.765 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.765 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.766 DEBUG:boto:Method: HEAD 2022-09-03T06:19:27.766 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.767 DEBUG:boto:Data: 2022-09-03T06:19:27.767 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.768 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:27.769 DEBUG:boto:Port: 8000 2022-09-03T06:19:27.770 DEBUG:boto:Params: {} 2022-09-03T06:19:27.770 DEBUG:boto:Token: None 2022-09-03T06:19:27.771 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.771 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9cAKSkwE8C90kNPKj374BLffjEo= 2022-09-03T06:19:27.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9cAKSkwE8C90kNPKj374BLffjEo=', 'Content-Length': '0'} 2022-09-03T06:19:27.787 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000089b1b1c9bb59d986-006312f1ef-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.788 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:19:27.788 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:27.789 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:27.789 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.799 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.800 DEBUG:boto:Method: GET 2022-09-03T06:19:27.801 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.802 DEBUG:boto:Data: 2022-09-03T06:19:27.803 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.805 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:27.806 DEBUG:boto:Port: 8001 2022-09-03T06:19:27.807 DEBUG:boto:Params: {} 2022-09-03T06:19:27.808 DEBUG:boto:Token: None 2022-09-03T06:19:27.809 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.810 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UF2lGR4enrcqNr0XfU0Sg4r/6CU= 2022-09-03T06:19:27.811 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UF2lGR4enrcqNr0XfU0Sg4r/6CU=', 'Content-Length': '0'} 2022-09-03T06:19:27.825 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b36e3018407935d2-006312f1ef-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.826 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:27.827 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:27.828 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.828 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.829 DEBUG:boto:Method: GET 2022-09-03T06:19:27.829 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.830 DEBUG:boto:Data: 2022-09-03T06:19:27.831 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.831 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:27.832 DEBUG:boto:Port: 8000 2022-09-03T06:19:27.832 DEBUG:boto:Params: {} 2022-09-03T06:19:27.833 DEBUG:boto:Token: None 2022-09-03T06:19:27.838 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:19:27.840 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UF2lGR4enrcqNr0XfU0Sg4r/6CU= 2022-09-03T06:19:27.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UF2lGR4enrcqNr0XfU0Sg4r/6CU=', 'Content-Length': '0'} 2022-09-03T06:19:27.847 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d9e3172db084ad48-006312f1ef-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.848 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:27.850 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:27.851 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:27.852 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:27.853 DEBUG:boto:Method: GET 2022-09-03T06:19:27.854 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:19:27.855 DEBUG:boto:Data: 2022-09-03T06:19:27.856 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.863 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:27.863 DEBUG:boto:Port: 8001 2022-09-03T06:19:27.864 DEBUG:boto:Params: {} 2022-09-03T06:19:27.866 DEBUG:boto:Token: None 2022-09-03T06:19:27.866 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/ 2022-09-03T06:19:27.867 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BK45BOvsegmR80LzcDBRZfMatD0= 2022-09-03T06:19:27.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BK45BOvsegmR80LzcDBRZfMatD0=', 'Content-Length': '0'} 2022-09-03T06:19:27.872 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dd33e30dfe333f6e-006312f1ef-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.914 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:27.917 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:27.919 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:27.920 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:27.921 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:27.921 DEBUG:boto:Method: GET 2022-09-03T06:19:27.923 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:19:27.924 DEBUG:boto:Data: 2022-09-03T06:19:27.924 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.925 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:27.925 DEBUG:boto:Port: 8000 2022-09-03T06:19:27.926 DEBUG:boto:Params: {} 2022-09-03T06:19:27.927 DEBUG:boto:Token: None 2022-09-03T06:19:27.929 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/ 2022-09-03T06:19:27.929 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BK45BOvsegmR80LzcDBRZfMatD0= 2022-09-03T06:19:27.930 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BK45BOvsegmR80LzcDBRZfMatD0=', 'Content-Length': '0'} 2022-09-03T06:19:27.935 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000206208f906eea77d-006312f1ef-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.936 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:27.937 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:27.938 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:27.938 DEBUG:boto:Method: HEAD 2022-09-03T06:19:27.939 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:19:27.939 DEBUG:boto:Data: 2022-09-03T06:19:27.940 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.940 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:27.941 DEBUG:boto:Port: 8001 2022-09-03T06:19:27.943 DEBUG:boto:Params: {} 2022-09-03T06:19:27.944 DEBUG:boto:Token: None 2022-09-03T06:19:27.945 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/d 2022-09-03T06:19:27.949 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AemFF4z0/3AerOX8/ONLiJ+MrfM= 2022-09-03T06:19:27.950 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AemFF4z0/3AerOX8/ONLiJ+MrfM=', 'Content-Length': '0'} 2022-09-03T06:19:27.953 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cabb1d46da152bf6-006312f1ef-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.954 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:27.955 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:27.956 DEBUG:boto:Method: HEAD 2022-09-03T06:19:27.957 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:19:27.959 DEBUG:boto:Data: 2022-09-03T06:19:27.960 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.961 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:27.962 DEBUG:boto:Port: 8000 2022-09-03T06:19:27.963 DEBUG:boto:Params: {} 2022-09-03T06:19:27.964 DEBUG:boto:Token: None 2022-09-03T06:19:27.965 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/d 2022-09-03T06:19:27.966 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AemFF4z0/3AerOX8/ONLiJ+MrfM= 2022-09-03T06:19:27.967 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AemFF4z0/3AerOX8/ONLiJ+MrfM=', 'Content-Length': '0'} 2022-09-03T06:19:27.971 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b58e4e624d41d764-006312f1ef-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:27.972 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:19:27.973 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:27.974 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:27.975 DEBUG:boto:Method: GET 2022-09-03T06:19:27.977 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:19:27.981 DEBUG:boto:Data: 2022-09-03T06:19:27.983 DEBUG:boto:Headers: {} 2022-09-03T06:19:27.983 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:27.984 DEBUG:boto:Port: 8001 2022-09-03T06:19:27.985 DEBUG:boto:Params: {} 2022-09-03T06:19:27.986 DEBUG:boto:Token: None 2022-09-03T06:19:27.987 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:27 GMT /dsgvcy-72/d 2022-09-03T06:19:27.989 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GBXxM2tlmSj4gvMGnBnUjGAb08s= 2022-09-03T06:19:27.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GBXxM2tlmSj4gvMGnBnUjGAb08s=', 'Content-Length': '0'} 2022-09-03T06:19:28.000 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e9501c05ef82ed30-006312f1ef-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:28.001 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:19:28.003 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:19:28.003 DEBUG:boto:Method: GET 2022-09-03T06:19:28.006 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:19:28.007 DEBUG:boto:Data: 2022-09-03T06:19:28.008 DEBUG:boto:Headers: {} 2022-09-03T06:19:28.009 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:28.010 DEBUG:boto:Port: 8000 2022-09-03T06:19:28.011 DEBUG:boto:Params: {} 2022-09-03T06:19:28.012 DEBUG:boto:Token: None 2022-09-03T06:19:28.013 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:28 GMT /dsgvcy-72/d 2022-09-03T06:19:28.014 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RW5aYkw1GydUhLr/LFa5reWKNXs= 2022-09-03T06:19:28.015 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RW5aYkw1GydUhLr/LFa5reWKNXs=', 'Content-Length': '0'} 2022-09-03T06:19:28.019 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000029c5f5d8e007c9df-006312f1f0-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:28.021 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:28.023 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:28.024 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:28.025 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:28.026 DEBUG:boto:Method: GET 2022-09-03T06:19:28.028 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:19:28.029 DEBUG:boto:Data: 2022-09-03T06:19:28.030 DEBUG:boto:Headers: {} 2022-09-03T06:19:28.031 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:19:28.032 DEBUG:boto:Port: 8001 2022-09-03T06:19:28.033 DEBUG:boto:Params: {} 2022-09-03T06:19:28.034 DEBUG:boto:Token: None 2022-09-03T06:19:28.035 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:28 GMT /dsgvcy-72/ 2022-09-03T06:19:28.036 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yS12RcU0J8/qvQGhCBoUIfois24= 2022-09-03T06:19:28.037 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yS12RcU0J8/qvQGhCBoUIfois24=', 'Content-Length': '0'} 2022-09-03T06:19:28.041 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007c11f3be0cec766d-006312f1f0-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:28.042 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:28.043 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:28.044 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:28.045 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:28.046 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:19:28.047 DEBUG:boto:Method: GET 2022-09-03T06:19:28.047 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:19:28.048 DEBUG:boto:Data: 2022-09-03T06:19:28.049 DEBUG:boto:Headers: {} 2022-09-03T06:19:28.054 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:19:28.055 DEBUG:boto:Port: 8000 2022-09-03T06:19:28.056 DEBUG:boto:Params: {} 2022-09-03T06:19:28.057 DEBUG:boto:Token: None 2022-09-03T06:19:28.059 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:19:28 GMT /dsgvcy-72/ 2022-09-03T06:19:28.060 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yS12RcU0J8/qvQGhCBoUIfois24= 2022-09-03T06:19:28.061 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yS12RcU0J8/qvQGhCBoUIfois24=', 'Content-Length': '0'} 2022-09-03T06:19:28.065 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005bfcab86e2d10ef9-006312f1f0-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:28.067 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:19:28.068 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-72 zones={test-zone3, test-zone2} 2022-09-03T06:19:28.069 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--bucket', 'dsgvcy-72', '--num-shards', '3', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:19:28.070 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--bucket', 'dsgvcy-72', '--num-shards', '3', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:19:28.072 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --bucket dsgvcy-72 --num-shards 3 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T06:19:28.132 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:19:28.132 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:19:28.195 INFO:teuthology.orchestra.run.smithi181.stdout:tenant: 2022-09-03T06:19:28.195 INFO:teuthology.orchestra.run.smithi181.stdout:bucket name: dsgvcy-72 2022-09-03T06:19:28.200 INFO:teuthology.orchestra.run.smithi181.stdout:total entries: 4 2022-09-03T06:19:28.275 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'reshard', '--bucket', 'dsgvcy-72', '--num-shards', '3', '--yes-i-really-mean-it', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:19:28.276 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'reshard', '--bucket', 'dsgvcy-72', '--num-shards', '3', '--yes-i-really-mean-it', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:19:28.276 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket reshard --bucket dsgvcy-72 --num-shards 3 --yes-i-really-mean-it --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:19:28.335 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:19:28.336 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:19:28.381 INFO:teuthology.orchestra.run.smithi189.stdout:tenant: 2022-09-03T06:19:28.381 INFO:teuthology.orchestra.run.smithi189.stdout:bucket name: dsgvcy-72 2022-09-03T06:19:28.391 INFO:teuthology.orchestra.run.smithi189.stdout:total entries: 4 2022-09-03T06:19:28.430 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--bucket', 'dsgvcy-72', '--num-shards', '3', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:19:28.430 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--bucket', 'dsgvcy-72', '--num-shards', '3', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:19:28.431 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --bucket dsgvcy-72 --num-shards 3 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T06:19:28.482 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:19:28.483 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:19:28.542 INFO:teuthology.orchestra.run.smithi181.stdout:tenant: 2022-09-03T06:19:28.542 INFO:teuthology.orchestra.run.smithi181.stdout:bucket name: dsgvcy-72 2022-09-03T06:19:28.544 INFO:teuthology.orchestra.run.smithi181.stdout:total entries: 4 2022-09-03T06:19:28.574 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:28.577 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:28.578 DEBUG:boto:Method: HEAD 2022-09-03T06:19:28.580 DEBUG:boto:Path: /dsgvcy-72/ 2022-09-03T06:19:28.581 DEBUG:boto:Data: 2022-09-03T06:19:28.581 DEBUG:boto:Headers: {} 2022-09-03T06:19:28.582 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:28.583 DEBUG:boto:Port: 8000 2022-09-03T06:19:28.584 DEBUG:boto:Params: {} 2022-09-03T06:19:28.585 DEBUG:boto:Token: None 2022-09-03T06:19:28.586 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:28 GMT /dsgvcy-72/ 2022-09-03T06:19:28.587 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Uow7R+N+nQiYiCD1CrpQ864cbt0= 2022-09-03T06:19:28.588 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Uow7R+N+nQiYiCD1CrpQ864cbt0=', 'Content-Length': '0'} 2022-09-03T06:19:28.593 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e055053fda46bb18-006312f1f0-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:28.595 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:19:28.596 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:19:28.596 DEBUG:boto:Method: PUT 2022-09-03T06:19:28.597 DEBUG:boto:Path: /dsgvcy-72/e 2022-09-03T06:19:28.598 DEBUG:boto:Data: 2022-09-03T06:19:28.599 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:19:28.599 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:28.600 DEBUG:boto:Port: 8000 2022-09-03T06:19:28.600 DEBUG:boto:Params: {} 2022-09-03T06:19:28.601 DEBUG:boto:Token: None 2022-09-03T06:19:28.601 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:19:28 GMT /dsgvcy-72/e 2022-09-03T06:19:28.602 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5M8Ab2Hg74bhhTRctnmGKlDIRcU= 2022-09-03T06:19:28.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:19:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5M8Ab2Hg74bhhTRctnmGKlDIRcU='} 2022-09-03T06:19:28.610 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000d266efa4fe661d00-006312f1f0-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:28.611 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:28.612 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:28.612 DEBUG:boto:Method: HEAD 2022-09-03T06:19:28.613 DEBUG:boto:Path: /dsgvcy-72/ 2022-09-03T06:19:28.613 DEBUG:boto:Data: 2022-09-03T06:19:28.614 DEBUG:boto:Headers: {} 2022-09-03T06:19:28.615 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:28.616 DEBUG:boto:Port: 8000 2022-09-03T06:19:28.617 DEBUG:boto:Params: {} 2022-09-03T06:19:28.617 DEBUG:boto:Token: None 2022-09-03T06:19:28.618 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:28 GMT /dsgvcy-72/ 2022-09-03T06:19:28.618 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Uow7R+N+nQiYiCD1CrpQ864cbt0= 2022-09-03T06:19:28.619 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Uow7R+N+nQiYiCD1CrpQ864cbt0=', 'Content-Length': '0'} 2022-09-03T06:19:28.622 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '15'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000db864bff2db92805-006312f1f0-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:28.623 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:19:28.623 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:19:28.624 DEBUG:boto:Method: PUT 2022-09-03T06:19:28.624 DEBUG:boto:Path: /dsgvcy-72/f 2022-09-03T06:19:28.625 DEBUG:boto:Data: 2022-09-03T06:19:28.625 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:19:28.626 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:28.627 DEBUG:boto:Port: 8000 2022-09-03T06:19:28.627 DEBUG:boto:Params: {} 2022-09-03T06:19:28.628 DEBUG:boto:Token: None 2022-09-03T06:19:28.628 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:19:28 GMT /dsgvcy-72/f 2022-09-03T06:19:28.629 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NWe+XHU2sM/vsOCwi9BkvMtN5NI= 2022-09-03T06:19:28.630 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:19:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NWe+XHU2sM/vsOCwi9BkvMtN5NI='} 2022-09-03T06:19:28.636 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f0ad9fbdeaf22fee-006312f1f0-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:28.638 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:28.638 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:28.639 DEBUG:boto:Method: HEAD 2022-09-03T06:19:28.639 DEBUG:boto:Path: /dsgvcy-72/ 2022-09-03T06:19:28.640 DEBUG:boto:Data: 2022-09-03T06:19:28.641 DEBUG:boto:Headers: {} 2022-09-03T06:19:28.642 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:28.643 DEBUG:boto:Port: 8000 2022-09-03T06:19:28.644 DEBUG:boto:Params: {} 2022-09-03T06:19:28.644 DEBUG:boto:Token: None 2022-09-03T06:19:28.645 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:28 GMT /dsgvcy-72/ 2022-09-03T06:19:28.645 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Uow7R+N+nQiYiCD1CrpQ864cbt0= 2022-09-03T06:19:28.646 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Uow7R+N+nQiYiCD1CrpQ864cbt0=', 'Content-Length': '0'} 2022-09-03T06:19:28.649 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000160a7483b8a5926a-006312f1f0-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:28.650 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:19:28.651 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:19:28.651 DEBUG:boto:Method: PUT 2022-09-03T06:19:28.652 DEBUG:boto:Path: /dsgvcy-72/g 2022-09-03T06:19:28.654 DEBUG:boto:Data: 2022-09-03T06:19:28.655 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:19:28.655 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:28.656 DEBUG:boto:Port: 8000 2022-09-03T06:19:28.656 DEBUG:boto:Params: {} 2022-09-03T06:19:28.657 DEBUG:boto:Token: None 2022-09-03T06:19:28.657 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:19:28 GMT /dsgvcy-72/g 2022-09-03T06:19:28.659 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UXp93dqGVmYiWKsynasDotZ1qvA= 2022-09-03T06:19:28.660 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:19:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UXp93dqGVmYiWKsynasDotZ1qvA='} 2022-09-03T06:19:28.703 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001035fd78d0508241-006312f1f0-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:28.704 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:19:28.705 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:19:28.706 DEBUG:boto:Method: HEAD 2022-09-03T06:19:28.706 DEBUG:boto:Path: /dsgvcy-72/ 2022-09-03T06:19:28.707 DEBUG:boto:Data: 2022-09-03T06:19:28.708 DEBUG:boto:Headers: {} 2022-09-03T06:19:28.709 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:28.710 DEBUG:boto:Port: 8000 2022-09-03T06:19:28.711 DEBUG:boto:Params: {} 2022-09-03T06:19:28.712 DEBUG:boto:Token: None 2022-09-03T06:19:28.713 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:19:28 GMT /dsgvcy-72/ 2022-09-03T06:19:28.714 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Uow7R+N+nQiYiCD1CrpQ864cbt0= 2022-09-03T06:19:28.715 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:19:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Uow7R+N+nQiYiCD1CrpQ864cbt0=', 'Content-Length': '0'} 2022-09-03T06:19:28.741 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '21'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a08871e9b9be1bd9-006312f1f0-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:28.742 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:19:28.743 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:19:28.743 DEBUG:boto:Method: PUT 2022-09-03T06:19:28.744 DEBUG:boto:Path: /dsgvcy-72/h 2022-09-03T06:19:28.744 DEBUG:boto:Data: 2022-09-03T06:19:28.745 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:19:28.746 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:19:28.747 DEBUG:boto:Port: 8000 2022-09-03T06:19:28.748 DEBUG:boto:Params: {} 2022-09-03T06:19:28.749 DEBUG:boto:Token: None 2022-09-03T06:19:28.750 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:19:28 GMT /dsgvcy-72/h 2022-09-03T06:19:28.751 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0vIXvI445TqVT5epwyQ/2SCCFKU= 2022-09-03T06:19:28.752 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:19:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0vIXvI445TqVT5epwyQ/2SCCFKU='} 2022-09-03T06:19:28.757 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004f45375b55572adb-006312f1f0-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:19:28.769 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-72 2022-09-03T06:19:28.770 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:19:28.770 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:19:28.771 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-72 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:19:28.829 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:19:28.829 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:19:28.867 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:19:28.861+0000 7f85eeacc600 1 waiting to reach latest gen 2, on 0.. 2022-09-03T06:19:33.863 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:19:33.861+0000 7f85eeacc600 1 waiting to reach latest gen 2, on 0.. 2022-09-03T06:19:38.864 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:19:38.862+0000 7f85eeacc600 1 waiting to reach latest gen 2, on 0.. 2022-09-03T06:19:43.865 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:19:43.863+0000 7f85eeacc600 1 waiting to reach latest gen 2, on 1.. 2022-09-03T06:19:48.865 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:19:48.863+0000 7f85eeacc600 1 waiting to reach latest gen 2, on 1.. 2022-09-03T06:19:53.866 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:19:53.864+0000 7f85eeacc600 1 waiting to reach latest gen 2, on 1.. 2022-09-03T06:19:58.866 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:19:58.865+0000 7f85eeacc600 1 waiting to reach latest gen 2, on 1.. 2022-09-03T06:20:03.868 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:20:03.864+0000 7f85eeacc600 1 waiting to reach latest gen 2, on 1.. 2022-09-03T06:20:08.868 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:20:08.866+0000 7f85eeacc600 1 waiting to reach latest gen 2, on 1.. 2022-09-03T06:20:13.868 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:20:13.866+0000 7f85eeacc600 1 waiting to reach latest gen 2, on 1.. 2022-09-03T06:20:18.869 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:20:18.867+0000 7f85eeacc600 1 waiting to reach latest gen 2, on 1.. 2022-09-03T06:20:23.870 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:20:23.868+0000 7f85eeacc600 1 waiting for incremental sync to catch up: 2022-09-03T06:20:23.871 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , ] 2022-09-03T06:20:23.872 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:20:28.871 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:20:28.869+0000 7f85eeacc600 1 waiting for incremental sync to catch up: 2022-09-03T06:20:28.872 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , ] 2022-09-03T06:20:28.873 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:20:33.872 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:20:33.870+0000 7f85eeacc600 1 waiting for incremental sync to catch up: 2022-09-03T06:20:33.873 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , ] 2022-09-03T06:20:33.873 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:20:38.873 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:20:38.871+0000 7f85eeacc600 1 waiting for incremental sync to catch up: 2022-09-03T06:20:38.874 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , ] 2022-09-03T06:20:38.875 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:20:43.874 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:20:43.872+0000 7f85eeacc600 1 waiting for incremental sync to catch up: 2022-09-03T06:20:43.875 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , ] 2022-09-03T06:20:43.875 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:20:48.875 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:20:48.873+0000 7f85eeacc600 1 waiting for incremental sync to catch up: 2022-09-03T06:20:48.875 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , ] 2022-09-03T06:20:48.875 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:20:53.875 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:20:53.872+0000 7f85eeacc600 1 waiting for incremental sync to catch up: 2022-09-03T06:20:53.876 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , ] 2022-09-03T06:20:53.876 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:20:58.877 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:20:58.873+0000 7f85eeacc600 1 waiting for incremental sync to catch up: 2022-09-03T06:20:58.878 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , ] 2022-09-03T06:20:58.878 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:21:03.877 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:21:03.875+0000 7f85eeacc600 1 waiting for incremental sync to catch up: 2022-09-03T06:21:03.878 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.58.5, , ] 2022-09-03T06:21:03.878 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:21:08.878 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:21:08.876+0000 7f85eeacc600 1 waiting for incremental sync to catch up: 2022-09-03T06:21:08.878 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.58.5, , ] 2022-09-03T06:21:08.879 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:21:13.879 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:21:13.877+0000 7f85eeacc600 1 waiting for incremental sync to catch up: 2022-09-03T06:21:13.879 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.58.5, , ] 2022-09-03T06:21:13.880 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:21:18.880 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:21:18.878+0000 7f85eeacc600 1 waiting for incremental sync to catch up: 2022-09-03T06:21:18.881 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.58.5, , ] 2022-09-03T06:21:18.881 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:21:23.881 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:21:23.879+0000 7f85eeacc600 1 waiting for incremental sync to catch up: 2022-09-03T06:21:23.882 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.58.5, , ] 2022-09-03T06:21:23.882 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:21:28.882 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:21:28.880+0000 7f85eeacc600 1 waiting for incremental sync to catch up: 2022-09-03T06:21:28.882 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.58.5, , ] 2022-09-03T06:21:28.882 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:21:33.882 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:21:33.880+0000 7f85eeacc600 1 waiting for incremental sync to catch up: 2022-09-03T06:21:33.883 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.58.5, , ] 2022-09-03T06:21:33.883 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:21:38.884 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:21:38.881+0000 7f85eeacc600 1 waiting for incremental sync to catch up: 2022-09-03T06:21:38.885 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.58.5, , ] 2022-09-03T06:21:38.885 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:21:43.884 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:21:43.881+0000 7f85eeacc600 1 bucket sync caught up with source: 2022-09-03T06:21:43.885 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:21:43.885 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:21:43.885 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:21:43.881+0000 7f85eeacc600 0 bucket checkpoint complete 2022-09-03T06:21:43.904 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-72 2022-09-03T06:21:43.906 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:21:43.907 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:21:43.909 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-72 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:21:43.973 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:21:43.973 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:21:44.015 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:21:44.012+0000 7fc447477600 1 waiting for incremental sync to catch up: 2022-09-03T06:21:44.015 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:21:44.016 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:21:49.018 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:21:49.013+0000 7fc447477600 1 waiting for incremental sync to catch up: 2022-09-03T06:21:49.018 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:21:49.019 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:21:54.017 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:21:54.014+0000 7fc447477600 1 waiting for incremental sync to catch up: 2022-09-03T06:21:54.017 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:21:54.018 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:21:59.018 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:21:59.015+0000 7fc447477600 1 waiting for incremental sync to catch up: 2022-09-03T06:21:59.019 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:21:59.019 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:04.019 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:22:04.017+0000 7fc447477600 1 waiting for incremental sync to catch up: 2022-09-03T06:22:04.019 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:04.020 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:09.020 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:22:09.018+0000 7fc447477600 1 waiting for incremental sync to catch up: 2022-09-03T06:22:09.021 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:09.021 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:14.021 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:22:14.019+0000 7fc447477600 1 waiting for incremental sync to catch up: 2022-09-03T06:22:14.021 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:14.021 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:19.022 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:22:19.020+0000 7fc447477600 1 waiting for incremental sync to catch up: 2022-09-03T06:22:19.022 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:19.022 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:24.023 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:22:24.021+0000 7fc447477600 1 waiting for incremental sync to catch up: 2022-09-03T06:22:24.024 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:24.024 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:29.023 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:22:29.021+0000 7fc447477600 1 waiting for incremental sync to catch up: 2022-09-03T06:22:29.024 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:29.024 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:34.024 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:22:34.022+0000 7fc447477600 1 waiting for incremental sync to catch up: 2022-09-03T06:22:34.025 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:34.025 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:39.025 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:22:39.022+0000 7fc447477600 1 waiting for incremental sync to catch up: 2022-09-03T06:22:39.026 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:39.026 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:44.028 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:22:44.023+0000 7fc447477600 1 waiting for incremental sync to catch up: 2022-09-03T06:22:44.029 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:44.030 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:49.027 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:22:49.024+0000 7fc447477600 1 waiting for incremental sync to catch up: 2022-09-03T06:22:49.027 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:49.028 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:54.028 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:22:54.026+0000 7fc447477600 1 waiting for incremental sync to catch up: 2022-09-03T06:22:54.029 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:54.029 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:59.029 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:22:59.027+0000 7fc447477600 1 waiting for incremental sync to catch up: 2022-09-03T06:22:59.030 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:22:59.031 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:23:04.030 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:23:04.028+0000 7fc447477600 1 bucket sync caught up with source: 2022-09-03T06:23:04.294 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:23:04.294 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:23:04.295 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:23:04.028+0000 7fc447477600 0 bucket checkpoint complete 2022-09-03T06:23:04.298 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-72 2022-09-03T06:23:04.299 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:23:04.300 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:23:04.301 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-72 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:23:04.367 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:23:04.367 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:23:04.426 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:23:04.425+0000 7f9798653600 1 bucket sync caught up with source: 2022-09-03T06:23:04.427 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:23:04.427 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.58.5, 00000000002.69.5, 00000000002.54.5] 2022-09-03T06:23:04.428 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:23:04.425+0000 7f9798653600 0 bucket checkpoint complete 2022-09-03T06:23:04.446 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-72 2022-09-03T06:23:04.446 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:23:04.447 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:23:04.448 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-72 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:23:04.519 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:23:04.520 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:23:04.573 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:23:04.571+0000 7f7437367600 1 bucket sync caught up with source: 2022-09-03T06:23:04.576 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.435.5, 00000000002.394.5, 00000000002.747.5] 2022-09-03T06:23:04.577 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.435.5, 00000000002.394.5, 00000000002.747.5] 2022-09-03T06:23:04.578 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:23:04.571+0000 7f7437367600 0 bucket checkpoint complete 2022-09-03T06:23:04.593 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-72 2022-09-03T06:23:04.595 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:23:04.596 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:23:04.598 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-72 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:23:04.665 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:23:04.665 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:23:04.703 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:23:04.702+0000 7f4059bf5600 1 waiting for incremental sync to catch up: 2022-09-03T06:23:04.704 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.394.5, 00000000002.747.5] 2022-09-03T06:23:04.705 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.435.5, 00000000002.394.5, 00000000002.747.5] 2022-09-03T06:23:09.704 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:23:09.702+0000 7f4059bf5600 1 waiting for incremental sync to catch up: 2022-09-03T06:23:09.705 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.394.5, 00000000002.747.5] 2022-09-03T06:23:09.705 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.435.5, 00000000002.394.5, 00000000002.747.5] 2022-09-03T06:23:14.706 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:23:14.703+0000 7f4059bf5600 1 waiting for incremental sync to catch up: 2022-09-03T06:23:14.707 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.394.5, 00000000002.747.5] 2022-09-03T06:23:14.707 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.435.5, 00000000002.394.5, 00000000002.747.5] 2022-09-03T06:23:19.706 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:23:19.703+0000 7f4059bf5600 1 waiting for incremental sync to catch up: 2022-09-03T06:23:19.707 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.394.5, 00000000002.747.5] 2022-09-03T06:23:19.707 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.435.5, 00000000002.394.5, 00000000002.747.5] 2022-09-03T06:23:24.707 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:23:24.704+0000 7f4059bf5600 1 waiting for incremental sync to catch up: 2022-09-03T06:23:24.707 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.394.5, 00000000002.747.5] 2022-09-03T06:23:24.708 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.435.5, 00000000002.394.5, 00000000002.747.5] 2022-09-03T06:23:29.708 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:23:29.705+0000 7f4059bf5600 1 waiting for incremental sync to catch up: 2022-09-03T06:23:29.709 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.394.5, 00000000002.747.5] 2022-09-03T06:23:29.710 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.435.5, 00000000002.394.5, 00000000002.747.5] 2022-09-03T06:23:34.709 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:23:34.706+0000 7f4059bf5600 1 waiting for incremental sync to catch up: 2022-09-03T06:23:34.777 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.394.5, 00000000002.747.5] 2022-09-03T06:23:34.778 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.435.5, 00000000002.394.5, 00000000002.747.5] 2022-09-03T06:23:39.710 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:23:39.708+0000 7f4059bf5600 1 waiting for incremental sync to catch up: 2022-09-03T06:23:39.710 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.394.5, 00000000002.747.5] 2022-09-03T06:23:39.710 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.435.5, 00000000002.394.5, 00000000002.747.5] 2022-09-03T06:23:44.711 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:23:44.709+0000 7f4059bf5600 1 bucket sync caught up with source: 2022-09-03T06:23:44.711 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.435.5, 00000000002.394.5, 00000000002.747.5] 2022-09-03T06:23:44.712 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.435.5, 00000000002.394.5, 00000000002.747.5] 2022-09-03T06:23:44.712 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:23:44.709+0000 7f4059bf5600 0 bucket checkpoint complete 2022-09-03T06:23:44.731 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-72 2022-09-03T06:23:44.731 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-72 2022-09-03T06:23:44.732 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:23:44.733 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:23:44.733 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-72 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:23:44.796 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:23:44.796 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:23:44.846 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:23:44.844+0000 7efd61303600 1 bucket sync caught up with source: 2022-09-03T06:23:44.847 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.198.5, , , 00000000001.193.5, 00000000001.183.5, , , 00000000002.301.5, , 00000000002.265.5] 2022-09-03T06:23:44.847 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.198.5, , , 00000000001.193.5, 00000000001.183.5, , , 00000000002.301.5, , 00000000002.265.5] 2022-09-03T06:23:44.847 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:23:44.844+0000 7efd61303600 0 bucket checkpoint complete 2022-09-03T06:23:44.861 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-72 2022-09-03T06:23:44.861 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:23:44.862 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-72', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:23:44.863 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-72 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:23:44.928 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:23:44.928 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:23:44.970 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:23:44.968+0000 7f5331bb9600 1 bucket sync caught up with source: 2022-09-03T06:23:44.970 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000002.198.5, , , 00000000001.193.5, 00000000001.183.5, , , 00000000002.301.5, , 00000000002.265.5] 2022-09-03T06:23:44.971 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000002.198.5, , , 00000000001.193.5, 00000000001.183.5, , , 00000000002.301.5, , 00000000002.265.5] 2022-09-03T06:23:44.971 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:23:44.968+0000 7f5331bb9600 0 bucket checkpoint complete 2022-09-03T06:23:44.987 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-72 2022-09-03T06:23:44.988 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-72 zones={test-zone2, test-zone1} 2022-09-03T06:23:44.989 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:44.989 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:44.990 DEBUG:boto:Method: HEAD 2022-09-03T06:23:44.991 DEBUG:boto:Path: /dsgvcy-72/ 2022-09-03T06:23:44.991 DEBUG:boto:Data: 2022-09-03T06:23:44.992 DEBUG:boto:Headers: {} 2022-09-03T06:23:44.993 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:44.994 DEBUG:boto:Port: 8000 2022-09-03T06:23:44.994 DEBUG:boto:Params: {} 2022-09-03T06:23:44.996 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:23:44.997 DEBUG:boto:Token: None 2022-09-03T06:23:44.997 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:44 GMT /dsgvcy-72/ 2022-09-03T06:23:44.998 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KcDxjM9n7LRzFvhU6bq0iyzdYys= 2022-09-03T06:23:44.999 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KcDxjM9n7LRzFvhU6bq0iyzdYys=', 'Content-Length': '0'} 2022-09-03T06:23:45.005 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000687727fdabe9fb2c-006312f2f1-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.007 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:45.008 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:45.008 DEBUG:boto:Method: HEAD 2022-09-03T06:23:45.009 DEBUG:boto:Path: /dsgvcy-72/ 2022-09-03T06:23:45.009 DEBUG:boto:Data: 2022-09-03T06:23:45.010 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.010 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.011 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.011 DEBUG:boto:Params: {} 2022-09-03T06:23:45.012 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:23:45.013 DEBUG:boto:Token: None 2022-09-03T06:23:45.014 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/ 2022-09-03T06:23:45.015 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5VV9+riqJzNrCN+LoZQ5ia4AzU4= 2022-09-03T06:23:45.015 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5VV9+riqJzNrCN+LoZQ5ia4AzU4=', 'Content-Length': '0'} 2022-09-03T06:23:45.018 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000077c98ad5424f2554-006312f2f1-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.019 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:23:45.020 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:45.021 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:45.021 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:45.022 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:45.022 DEBUG:boto:Method: GET 2022-09-03T06:23:45.023 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:45.023 DEBUG:boto:Data: 2022-09-03T06:23:45.024 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.024 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.025 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.025 DEBUG:boto:Params: {} 2022-09-03T06:23:45.026 DEBUG:boto:Token: None 2022-09-03T06:23:45.027 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/?versions 2022-09-03T06:23:45.028 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yYpPiMO5Ll6w17M9ykMhvgDcakU= 2022-09-03T06:23:45.028 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yYpPiMO5Ll6w17M9ykMhvgDcakU=', 'Content-Length': '0'} 2022-09-03T06:23:45.031 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000097a80a3c53d25217-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.031 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:45.033 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:23:45.034 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:23:45.035 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:23:45.035 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:23:45.036 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:23:45.036 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:23:45.037 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:23:45.038 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:23:45.039 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:23:45.039 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:45.040 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:45.041 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:45.042 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:45.043 DEBUG:boto:Method: GET 2022-09-03T06:23:45.044 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:45.045 DEBUG:boto:Data: 2022-09-03T06:23:45.046 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.047 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.048 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.048 DEBUG:boto:Params: {} 2022-09-03T06:23:45.049 DEBUG:boto:Token: None 2022-09-03T06:23:45.050 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/?versions 2022-09-03T06:23:45.050 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yYpPiMO5Ll6w17M9ykMhvgDcakU= 2022-09-03T06:23:45.051 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yYpPiMO5Ll6w17M9ykMhvgDcakU=', 'Content-Length': '0'} 2022-09-03T06:23:45.053 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000072317ad25428f5b9-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.054 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:45.055 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:23:45.056 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:23:45.056 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:23:45.057 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:23:45.057 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:23:45.058 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:23:45.058 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:23:45.059 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:23:45.059 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:45.060 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:45.060 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:45.061 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:45.061 DEBUG:boto:Method: GET 2022-09-03T06:23:45.062 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:45.063 DEBUG:boto:Data: 2022-09-03T06:23:45.063 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.064 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.064 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.065 DEBUG:boto:Params: {} 2022-09-03T06:23:45.066 DEBUG:boto:Token: None 2022-09-03T06:23:45.066 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/?versions 2022-09-03T06:23:45.067 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yYpPiMO5Ll6w17M9ykMhvgDcakU= 2022-09-03T06:23:45.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yYpPiMO5Ll6w17M9ykMhvgDcakU=', 'Content-Length': '0'} 2022-09-03T06:23:45.070 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000396b2097eaa5f221-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.070 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:45.072 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:45.072 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:45.073 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:45.073 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:45.074 DEBUG:boto:Method: GET 2022-09-03T06:23:45.074 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:45.075 DEBUG:boto:Data: 2022-09-03T06:23:45.075 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.076 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.077 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.078 DEBUG:boto:Params: {} 2022-09-03T06:23:45.079 DEBUG:boto:Token: None 2022-09-03T06:23:45.080 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/?versions 2022-09-03T06:23:45.081 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yYpPiMO5Ll6w17M9ykMhvgDcakU= 2022-09-03T06:23:45.082 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yYpPiMO5Ll6w17M9ykMhvgDcakU=', 'Content-Length': '0'} 2022-09-03T06:23:45.084 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001a0c7102e56d903c-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.085 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:45.086 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:23:45.087 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:45.087 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:45.088 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.089 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.089 DEBUG:boto:Method: GET 2022-09-03T06:23:45.090 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.090 DEBUG:boto:Data: 2022-09-03T06:23:45.091 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.091 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.092 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.093 DEBUG:boto:Params: {} 2022-09-03T06:23:45.094 DEBUG:boto:Token: None 2022-09-03T06:23:45.095 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.096 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xOzUun9/Imv/ao9frfR8ha66zXI= 2022-09-03T06:23:45.097 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xOzUun9/Imv/ao9frfR8ha66zXI=', 'Content-Length': '0'} 2022-09-03T06:23:45.100 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e87ea44b82b1e0f0-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.101 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:45.102 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:45.103 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.104 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.105 DEBUG:boto:Method: GET 2022-09-03T06:23:45.106 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.107 DEBUG:boto:Data: 2022-09-03T06:23:45.108 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.109 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.110 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.110 DEBUG:boto:Params: {} 2022-09-03T06:23:45.112 DEBUG:boto:Token: None 2022-09-03T06:23:45.113 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.114 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xOzUun9/Imv/ao9frfR8ha66zXI= 2022-09-03T06:23:45.115 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xOzUun9/Imv/ao9frfR8ha66zXI=', 'Content-Length': '0'} 2022-09-03T06:23:45.117 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000215711af31384f14-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.119 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:45.119 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:45.120 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.121 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.122 DEBUG:boto:Method: HEAD 2022-09-03T06:23:45.123 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.124 DEBUG:boto:Data: 2022-09-03T06:23:45.125 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.125 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.126 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.126 DEBUG:boto:Params: {} 2022-09-03T06:23:45.127 DEBUG:boto:Token: None 2022-09-03T06:23:45.128 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.128 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MrsixIa9ktHeQmi7myvbSi6zJWw= 2022-09-03T06:23:45.129 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MrsixIa9ktHeQmi7myvbSi6zJWw=', 'Content-Length': '0'} 2022-09-03T06:23:45.131 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000003246f3b0d739ced-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.132 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:45.132 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:45.133 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.133 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.134 DEBUG:boto:Method: HEAD 2022-09-03T06:23:45.134 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.135 DEBUG:boto:Data: 2022-09-03T06:23:45.136 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.137 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.137 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.138 DEBUG:boto:Params: {} 2022-09-03T06:23:45.138 DEBUG:boto:Token: None 2022-09-03T06:23:45.139 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.140 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MrsixIa9ktHeQmi7myvbSi6zJWw= 2022-09-03T06:23:45.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MrsixIa9ktHeQmi7myvbSi6zJWw=', 'Content-Length': '0'} 2022-09-03T06:23:45.142 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc742c9f2c6df739-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.143 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:23:45.143 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:45.144 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:45.145 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.145 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.146 DEBUG:boto:Method: GET 2022-09-03T06:23:45.147 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.147 DEBUG:boto:Data: 2022-09-03T06:23:45.148 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.149 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.149 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.150 DEBUG:boto:Params: {} 2022-09-03T06:23:45.151 DEBUG:boto:Token: None 2022-09-03T06:23:45.151 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.152 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xOzUun9/Imv/ao9frfR8ha66zXI= 2022-09-03T06:23:45.153 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xOzUun9/Imv/ao9frfR8ha66zXI=', 'Content-Length': '0'} 2022-09-03T06:23:45.155 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001026b9597c0ab296-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.156 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:45.157 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:45.157 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.158 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.159 DEBUG:boto:Method: GET 2022-09-03T06:23:45.159 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.160 DEBUG:boto:Data: 2022-09-03T06:23:45.160 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.161 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.161 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.162 DEBUG:boto:Params: {} 2022-09-03T06:23:45.162 DEBUG:boto:Token: None 2022-09-03T06:23:45.163 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:23:45.164 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xOzUun9/Imv/ao9frfR8ha66zXI= 2022-09-03T06:23:45.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xOzUun9/Imv/ao9frfR8ha66zXI=', 'Content-Length': '0'} 2022-09-03T06:23:45.167 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008893e2dd27d90284-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.169 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:45.169 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:45.170 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:45.171 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:45.171 DEBUG:boto:Method: GET 2022-09-03T06:23:45.172 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:23:45.172 DEBUG:boto:Data: 2022-09-03T06:23:45.173 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.173 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.174 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.175 DEBUG:boto:Params: {} 2022-09-03T06:23:45.175 DEBUG:boto:Token: None 2022-09-03T06:23:45.176 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/ 2022-09-03T06:23:45.177 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc= 2022-09-03T06:23:45.178 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc=', 'Content-Length': '0'} 2022-09-03T06:23:45.180 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b73a233936d76441-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.181 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:45.182 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:45.183 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:45.184 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:45.185 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:45.186 DEBUG:boto:Method: GET 2022-09-03T06:23:45.187 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:23:45.188 DEBUG:boto:Data: 2022-09-03T06:23:45.189 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.190 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.191 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.192 DEBUG:boto:Params: {} 2022-09-03T06:23:45.193 DEBUG:boto:Token: None 2022-09-03T06:23:45.194 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/ 2022-09-03T06:23:45.195 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc= 2022-09-03T06:23:45.196 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc=', 'Content-Length': '0'} 2022-09-03T06:23:45.199 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003d4e1b5d309583b5-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.200 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:45.201 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:45.202 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:45.203 DEBUG:boto:Method: HEAD 2022-09-03T06:23:45.204 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:23:45.205 DEBUG:boto:Data: 2022-09-03T06:23:45.206 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.207 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.208 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.210 DEBUG:boto:Params: {} 2022-09-03T06:23:45.211 DEBUG:boto:Token: None 2022-09-03T06:23:45.211 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/a 2022-09-03T06:23:45.212 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rnclKE0yfdgAcQqToRyMKYUolqg= 2022-09-03T06:23:45.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rnclKE0yfdgAcQqToRyMKYUolqg=', 'Content-Length': '0'} 2022-09-03T06:23:45.215 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d41a53e6c88edfd3-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.215 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:45.216 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:45.216 DEBUG:boto:Method: HEAD 2022-09-03T06:23:45.217 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:23:45.218 DEBUG:boto:Data: 2022-09-03T06:23:45.218 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.219 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.219 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.220 DEBUG:boto:Params: {} 2022-09-03T06:23:45.220 DEBUG:boto:Token: None 2022-09-03T06:23:45.221 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/a 2022-09-03T06:23:45.221 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rnclKE0yfdgAcQqToRyMKYUolqg= 2022-09-03T06:23:45.222 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rnclKE0yfdgAcQqToRyMKYUolqg=', 'Content-Length': '0'} 2022-09-03T06:23:45.224 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bca16e44a219fd9f-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.225 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:23:45.225 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:45.226 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:45.226 DEBUG:boto:Method: GET 2022-09-03T06:23:45.227 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:23:45.227 DEBUG:boto:Data: 2022-09-03T06:23:45.228 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.228 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.229 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.230 DEBUG:boto:Params: {} 2022-09-03T06:23:45.231 DEBUG:boto:Token: None 2022-09-03T06:23:45.231 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/a 2022-09-03T06:23:45.232 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1bendvaLAcfu/OIHrkS5FXfn8YU= 2022-09-03T06:23:45.233 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1bendvaLAcfu/OIHrkS5FXfn8YU=', 'Content-Length': '0'} 2022-09-03T06:23:45.235 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004ae08e6f8dcd8e21-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.278 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:45.279 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:45.279 DEBUG:boto:Method: GET 2022-09-03T06:23:45.280 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:23:45.280 DEBUG:boto:Data: 2022-09-03T06:23:45.281 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.282 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.282 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.283 DEBUG:boto:Params: {} 2022-09-03T06:23:45.284 DEBUG:boto:Token: None 2022-09-03T06:23:45.284 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/a 2022-09-03T06:23:45.285 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1bendvaLAcfu/OIHrkS5FXfn8YU= 2022-09-03T06:23:45.285 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1bendvaLAcfu/OIHrkS5FXfn8YU=', 'Content-Length': '0'} 2022-09-03T06:23:45.288 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007c1c2b514a51019b-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.288 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:45.289 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:45.290 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:45.290 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:45.291 DEBUG:boto:Method: GET 2022-09-03T06:23:45.291 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:23:45.292 DEBUG:boto:Data: 2022-09-03T06:23:45.292 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.293 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.293 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.294 DEBUG:boto:Params: {} 2022-09-03T06:23:45.294 DEBUG:boto:Token: None 2022-09-03T06:23:45.295 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/ 2022-09-03T06:23:45.296 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc= 2022-09-03T06:23:45.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc=', 'Content-Length': '0'} 2022-09-03T06:23:45.298 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000089c15988c2b713cc-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.299 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:45.300 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:45.300 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:45.301 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:45.302 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:45.303 DEBUG:boto:Method: GET 2022-09-03T06:23:45.303 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:23:45.304 DEBUG:boto:Data: 2022-09-03T06:23:45.304 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.305 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.305 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.306 DEBUG:boto:Params: {} 2022-09-03T06:23:45.307 DEBUG:boto:Token: None 2022-09-03T06:23:45.307 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/ 2022-09-03T06:23:45.308 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc= 2022-09-03T06:23:45.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc=', 'Content-Length': '0'} 2022-09-03T06:23:45.310 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000088d1cdb268ea24fb-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.311 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:45.312 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:23:45.313 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:45.313 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:45.314 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.314 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.315 DEBUG:boto:Method: GET 2022-09-03T06:23:45.316 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.316 DEBUG:boto:Data: 2022-09-03T06:23:45.317 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.317 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.318 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.318 DEBUG:boto:Params: {} 2022-09-03T06:23:45.319 DEBUG:boto:Token: None 2022-09-03T06:23:45.320 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.320 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K+70rnlUW5fyX1mP4tRLiwdDNgQ= 2022-09-03T06:23:45.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K+70rnlUW5fyX1mP4tRLiwdDNgQ=', 'Content-Length': '0'} 2022-09-03T06:23:45.323 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000080e6bd65559fbdf4-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.324 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:45.324 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:45.325 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.326 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.326 DEBUG:boto:Method: GET 2022-09-03T06:23:45.327 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.327 DEBUG:boto:Data: 2022-09-03T06:23:45.328 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.328 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.329 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.330 DEBUG:boto:Params: {} 2022-09-03T06:23:45.330 DEBUG:boto:Token: None 2022-09-03T06:23:45.331 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.331 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K+70rnlUW5fyX1mP4tRLiwdDNgQ= 2022-09-03T06:23:45.332 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K+70rnlUW5fyX1mP4tRLiwdDNgQ=', 'Content-Length': '0'} 2022-09-03T06:23:45.334 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001bc7f81edbc5f41b-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.335 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:45.335 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:45.336 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.336 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.337 DEBUG:boto:Method: HEAD 2022-09-03T06:23:45.337 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.338 DEBUG:boto:Data: 2022-09-03T06:23:45.339 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.339 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.340 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.340 DEBUG:boto:Params: {} 2022-09-03T06:23:45.341 DEBUG:boto:Token: None 2022-09-03T06:23:45.341 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.342 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QOVsMl5EdMuh2n8oSrKozKoXceQ= 2022-09-03T06:23:45.343 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QOVsMl5EdMuh2n8oSrKozKoXceQ=', 'Content-Length': '0'} 2022-09-03T06:23:45.345 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b6aef7fb577c29c6-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.345 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:45.346 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:45.347 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.347 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.348 DEBUG:boto:Method: HEAD 2022-09-03T06:23:45.348 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.349 DEBUG:boto:Data: 2022-09-03T06:23:45.349 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.350 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.350 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.351 DEBUG:boto:Params: {} 2022-09-03T06:23:45.351 DEBUG:boto:Token: None 2022-09-03T06:23:45.352 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.352 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QOVsMl5EdMuh2n8oSrKozKoXceQ= 2022-09-03T06:23:45.353 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QOVsMl5EdMuh2n8oSrKozKoXceQ=', 'Content-Length': '0'} 2022-09-03T06:23:45.355 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000842e19354c3bbb71-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.356 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:23:45.356 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:45.357 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:45.358 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.358 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.359 DEBUG:boto:Method: GET 2022-09-03T06:23:45.359 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.360 DEBUG:boto:Data: 2022-09-03T06:23:45.361 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.361 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.362 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.362 DEBUG:boto:Params: {} 2022-09-03T06:23:45.363 DEBUG:boto:Token: None 2022-09-03T06:23:45.364 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.364 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K+70rnlUW5fyX1mP4tRLiwdDNgQ= 2022-09-03T06:23:45.365 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K+70rnlUW5fyX1mP4tRLiwdDNgQ=', 'Content-Length': '0'} 2022-09-03T06:23:45.367 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000040ddf33ad3a0878a-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.414 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:45.415 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:45.415 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.416 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.416 DEBUG:boto:Method: GET 2022-09-03T06:23:45.417 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.417 DEBUG:boto:Data: 2022-09-03T06:23:45.418 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.419 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.419 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.420 DEBUG:boto:Params: {} 2022-09-03T06:23:45.420 DEBUG:boto:Token: None 2022-09-03T06:23:45.421 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:23:45.421 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K+70rnlUW5fyX1mP4tRLiwdDNgQ= 2022-09-03T06:23:45.422 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K+70rnlUW5fyX1mP4tRLiwdDNgQ=', 'Content-Length': '0'} 2022-09-03T06:23:45.424 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003523aeda1ecad2d1-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.425 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:45.425 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:45.426 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:45.426 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:45.427 DEBUG:boto:Method: GET 2022-09-03T06:23:45.427 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:23:45.428 DEBUG:boto:Data: 2022-09-03T06:23:45.428 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.429 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.429 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.430 DEBUG:boto:Params: {} 2022-09-03T06:23:45.431 DEBUG:boto:Token: None 2022-09-03T06:23:45.431 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/ 2022-09-03T06:23:45.432 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc= 2022-09-03T06:23:45.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc=', 'Content-Length': '0'} 2022-09-03T06:23:45.435 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e42ad11add64bd55-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.435 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:45.436 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:45.437 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:45.437 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:45.438 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:45.438 DEBUG:boto:Method: GET 2022-09-03T06:23:45.439 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:23:45.439 DEBUG:boto:Data: 2022-09-03T06:23:45.440 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.440 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.441 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.441 DEBUG:boto:Params: {} 2022-09-03T06:23:45.442 DEBUG:boto:Token: None 2022-09-03T06:23:45.443 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/ 2022-09-03T06:23:45.443 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc= 2022-09-03T06:23:45.444 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc=', 'Content-Length': '0'} 2022-09-03T06:23:45.446 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004e88d5c138f0e3f0-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.447 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:45.448 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:45.448 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:45.449 DEBUG:boto:Method: HEAD 2022-09-03T06:23:45.449 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:23:45.450 DEBUG:boto:Data: 2022-09-03T06:23:45.450 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.451 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.451 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.452 DEBUG:boto:Params: {} 2022-09-03T06:23:45.452 DEBUG:boto:Token: None 2022-09-03T06:23:45.453 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/b 2022-09-03T06:23:45.453 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EDHHqccXemXn5GlSiVcqDCQRmdA= 2022-09-03T06:23:45.454 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EDHHqccXemXn5GlSiVcqDCQRmdA=', 'Content-Length': '0'} 2022-09-03T06:23:45.456 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000994b3663cc954db0-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.457 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:45.457 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:45.458 DEBUG:boto:Method: HEAD 2022-09-03T06:23:45.458 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:23:45.459 DEBUG:boto:Data: 2022-09-03T06:23:45.459 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.460 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.460 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.461 DEBUG:boto:Params: {} 2022-09-03T06:23:45.461 DEBUG:boto:Token: None 2022-09-03T06:23:45.462 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/b 2022-09-03T06:23:45.462 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EDHHqccXemXn5GlSiVcqDCQRmdA= 2022-09-03T06:23:45.463 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EDHHqccXemXn5GlSiVcqDCQRmdA=', 'Content-Length': '0'} 2022-09-03T06:23:45.465 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a0dcdbab84495d7-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.465 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:23:45.466 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:45.466 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:45.467 DEBUG:boto:Method: GET 2022-09-03T06:23:45.468 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:23:45.468 DEBUG:boto:Data: 2022-09-03T06:23:45.469 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.469 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.470 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.470 DEBUG:boto:Params: {} 2022-09-03T06:23:45.471 DEBUG:boto:Token: None 2022-09-03T06:23:45.471 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/b 2022-09-03T06:23:45.472 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8UuwCYHPvafeRPl7ikbsLP0nEX0= 2022-09-03T06:23:45.473 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8UuwCYHPvafeRPl7ikbsLP0nEX0=', 'Content-Length': '0'} 2022-09-03T06:23:45.475 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000941a72f6587bc236-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.475 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:45.476 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:45.477 DEBUG:boto:Method: GET 2022-09-03T06:23:45.477 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:23:45.478 DEBUG:boto:Data: 2022-09-03T06:23:45.478 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.479 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.479 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.480 DEBUG:boto:Params: {} 2022-09-03T06:23:45.480 DEBUG:boto:Token: None 2022-09-03T06:23:45.481 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/b 2022-09-03T06:23:45.482 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8UuwCYHPvafeRPl7ikbsLP0nEX0= 2022-09-03T06:23:45.482 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8UuwCYHPvafeRPl7ikbsLP0nEX0=', 'Content-Length': '0'} 2022-09-03T06:23:45.484 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000081be187547794171-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.526 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:45.527 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:45.528 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:45.529 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:45.529 DEBUG:boto:Method: GET 2022-09-03T06:23:45.531 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:23:45.531 DEBUG:boto:Data: 2022-09-03T06:23:45.532 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.533 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.534 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.535 DEBUG:boto:Params: {} 2022-09-03T06:23:45.537 DEBUG:boto:Token: None 2022-09-03T06:23:45.538 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/ 2022-09-03T06:23:45.539 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc= 2022-09-03T06:23:45.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc=', 'Content-Length': '0'} 2022-09-03T06:23:45.543 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001bfb7cdac79686a8-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.544 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:45.545 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:45.546 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:45.547 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:45.547 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:45.548 DEBUG:boto:Method: GET 2022-09-03T06:23:45.549 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:23:45.549 DEBUG:boto:Data: 2022-09-03T06:23:45.550 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.550 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.551 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.551 DEBUG:boto:Params: {} 2022-09-03T06:23:45.552 DEBUG:boto:Token: None 2022-09-03T06:23:45.553 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/ 2022-09-03T06:23:45.553 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc= 2022-09-03T06:23:45.554 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc=', 'Content-Length': '0'} 2022-09-03T06:23:45.556 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001a2ddb2f086ac516-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.557 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:45.558 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:23:45.558 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:45.559 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:45.560 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.560 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.561 DEBUG:boto:Method: GET 2022-09-03T06:23:45.561 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.562 DEBUG:boto:Data: 2022-09-03T06:23:45.562 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.563 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.563 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.564 DEBUG:boto:Params: {} 2022-09-03T06:23:45.565 DEBUG:boto:Token: None 2022-09-03T06:23:45.565 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.566 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iwKLUHhRwDohlX2e7RVWmsBiXN0= 2022-09-03T06:23:45.567 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iwKLUHhRwDohlX2e7RVWmsBiXN0=', 'Content-Length': '0'} 2022-09-03T06:23:45.569 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000039707b2d298d66d-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.610 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:45.611 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:45.611 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.612 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.612 DEBUG:boto:Method: GET 2022-09-03T06:23:45.613 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.613 DEBUG:boto:Data: 2022-09-03T06:23:45.614 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.615 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.615 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.616 DEBUG:boto:Params: {} 2022-09-03T06:23:45.616 DEBUG:boto:Token: None 2022-09-03T06:23:45.617 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.617 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iwKLUHhRwDohlX2e7RVWmsBiXN0= 2022-09-03T06:23:45.618 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iwKLUHhRwDohlX2e7RVWmsBiXN0=', 'Content-Length': '0'} 2022-09-03T06:23:45.620 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006fdac330471ff87f-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.621 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:45.621 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:45.622 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.623 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.623 DEBUG:boto:Method: HEAD 2022-09-03T06:23:45.624 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.624 DEBUG:boto:Data: 2022-09-03T06:23:45.625 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.625 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.626 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.626 DEBUG:boto:Params: {} 2022-09-03T06:23:45.627 DEBUG:boto:Token: None 2022-09-03T06:23:45.627 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.628 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q3D5B7GaZll6ban4/quva5Tqnu0= 2022-09-03T06:23:45.629 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q3D5B7GaZll6ban4/quva5Tqnu0=', 'Content-Length': '0'} 2022-09-03T06:23:45.631 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e0472834be2834dc-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.631 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:45.632 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:45.632 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.633 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.633 DEBUG:boto:Method: HEAD 2022-09-03T06:23:45.634 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.635 DEBUG:boto:Data: 2022-09-03T06:23:45.635 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.636 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.636 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.637 DEBUG:boto:Params: {} 2022-09-03T06:23:45.637 DEBUG:boto:Token: None 2022-09-03T06:23:45.638 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.638 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q3D5B7GaZll6ban4/quva5Tqnu0= 2022-09-03T06:23:45.639 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q3D5B7GaZll6ban4/quva5Tqnu0=', 'Content-Length': '0'} 2022-09-03T06:23:45.641 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000024cb7c88c38c81fe-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.642 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:23:45.642 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:45.643 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:45.643 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.644 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.644 DEBUG:boto:Method: GET 2022-09-03T06:23:45.645 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.645 DEBUG:boto:Data: 2022-09-03T06:23:45.646 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.646 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.647 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.648 DEBUG:boto:Params: {} 2022-09-03T06:23:45.648 DEBUG:boto:Token: None 2022-09-03T06:23:45.649 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.649 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iwKLUHhRwDohlX2e7RVWmsBiXN0= 2022-09-03T06:23:45.650 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iwKLUHhRwDohlX2e7RVWmsBiXN0=', 'Content-Length': '0'} 2022-09-03T06:23:45.652 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c26d916045137dab-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.653 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:45.654 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:45.654 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.655 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.655 DEBUG:boto:Method: GET 2022-09-03T06:23:45.656 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.656 DEBUG:boto:Data: 2022-09-03T06:23:45.657 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.657 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.658 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.659 DEBUG:boto:Params: {} 2022-09-03T06:23:45.659 DEBUG:boto:Token: None 2022-09-03T06:23:45.660 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:23:45.660 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iwKLUHhRwDohlX2e7RVWmsBiXN0= 2022-09-03T06:23:45.661 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iwKLUHhRwDohlX2e7RVWmsBiXN0=', 'Content-Length': '0'} 2022-09-03T06:23:45.663 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000268d3ac969e8c596-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.664 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:45.665 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:45.665 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:45.666 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:45.666 DEBUG:boto:Method: GET 2022-09-03T06:23:45.667 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:23:45.667 DEBUG:boto:Data: 2022-09-03T06:23:45.668 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.668 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.669 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.669 DEBUG:boto:Params: {} 2022-09-03T06:23:45.670 DEBUG:boto:Token: None 2022-09-03T06:23:45.671 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/ 2022-09-03T06:23:45.671 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc= 2022-09-03T06:23:45.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc=', 'Content-Length': '0'} 2022-09-03T06:23:45.674 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008552cfb76a3d0431-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.675 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:45.676 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:45.676 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:45.677 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:45.677 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:45.678 DEBUG:boto:Method: GET 2022-09-03T06:23:45.679 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:23:45.679 DEBUG:boto:Data: 2022-09-03T06:23:45.680 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.680 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.681 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.681 DEBUG:boto:Params: {} 2022-09-03T06:23:45.682 DEBUG:boto:Token: None 2022-09-03T06:23:45.683 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/ 2022-09-03T06:23:45.683 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc= 2022-09-03T06:23:45.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc=', 'Content-Length': '0'} 2022-09-03T06:23:45.686 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000d17b516ea856cba-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.686 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:45.687 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:45.688 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:45.688 DEBUG:boto:Method: HEAD 2022-09-03T06:23:45.689 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:23:45.690 DEBUG:boto:Data: 2022-09-03T06:23:45.690 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.691 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.691 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.692 DEBUG:boto:Params: {} 2022-09-03T06:23:45.692 DEBUG:boto:Token: None 2022-09-03T06:23:45.693 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/c 2022-09-03T06:23:45.693 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lF8OP1Qlifc4OrbiYFKQ7N5hmJI= 2022-09-03T06:23:45.694 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lF8OP1Qlifc4OrbiYFKQ7N5hmJI=', 'Content-Length': '0'} 2022-09-03T06:23:45.696 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d695ef9cc0f806fb-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.697 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:45.697 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:45.698 DEBUG:boto:Method: HEAD 2022-09-03T06:23:45.698 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:23:45.699 DEBUG:boto:Data: 2022-09-03T06:23:45.700 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.700 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.701 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.701 DEBUG:boto:Params: {} 2022-09-03T06:23:45.702 DEBUG:boto:Token: None 2022-09-03T06:23:45.702 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/c 2022-09-03T06:23:45.703 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lF8OP1Qlifc4OrbiYFKQ7N5hmJI= 2022-09-03T06:23:45.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lF8OP1Qlifc4OrbiYFKQ7N5hmJI=', 'Content-Length': '0'} 2022-09-03T06:23:45.705 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c8ad4b85afc0836b-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.706 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:23:45.707 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:45.707 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:45.708 DEBUG:boto:Method: GET 2022-09-03T06:23:45.708 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:23:45.709 DEBUG:boto:Data: 2022-09-03T06:23:45.710 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.710 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.711 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.711 DEBUG:boto:Params: {} 2022-09-03T06:23:45.712 DEBUG:boto:Token: None 2022-09-03T06:23:45.713 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/c 2022-09-03T06:23:45.713 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:krmgZ0M6djAsugubRmY+Q2Zdhm0= 2022-09-03T06:23:45.714 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:krmgZ0M6djAsugubRmY+Q2Zdhm0=', 'Content-Length': '0'} 2022-09-03T06:23:45.716 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000097582b8b5dc75f9b-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.761 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:45.762 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:45.763 DEBUG:boto:Method: GET 2022-09-03T06:23:45.763 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:23:45.764 DEBUG:boto:Data: 2022-09-03T06:23:45.764 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.765 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.766 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.766 DEBUG:boto:Params: {} 2022-09-03T06:23:45.767 DEBUG:boto:Token: None 2022-09-03T06:23:45.768 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/c 2022-09-03T06:23:45.768 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:krmgZ0M6djAsugubRmY+Q2Zdhm0= 2022-09-03T06:23:45.769 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:krmgZ0M6djAsugubRmY+Q2Zdhm0=', 'Content-Length': '0'} 2022-09-03T06:23:45.772 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000476e53409a7f6202-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.773 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:45.774 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:45.775 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:45.776 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:45.777 DEBUG:boto:Method: GET 2022-09-03T06:23:45.778 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:23:45.779 DEBUG:boto:Data: 2022-09-03T06:23:45.779 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.780 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.780 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.781 DEBUG:boto:Params: {} 2022-09-03T06:23:45.782 DEBUG:boto:Token: None 2022-09-03T06:23:45.782 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/ 2022-09-03T06:23:45.783 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc= 2022-09-03T06:23:45.784 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc=', 'Content-Length': '0'} 2022-09-03T06:23:45.786 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a5ef0c0c4a4656dc-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.786 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:45.787 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:45.788 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:45.788 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:45.789 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:45.789 DEBUG:boto:Method: GET 2022-09-03T06:23:45.790 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:23:45.790 DEBUG:boto:Data: 2022-09-03T06:23:45.791 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.792 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.792 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.793 DEBUG:boto:Params: {} 2022-09-03T06:23:45.793 DEBUG:boto:Token: None 2022-09-03T06:23:45.794 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/ 2022-09-03T06:23:45.794 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc= 2022-09-03T06:23:45.795 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc=', 'Content-Length': '0'} 2022-09-03T06:23:45.797 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000703ceec11fcb8228-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.798 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:45.799 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:23:45.800 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:45.800 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:45.801 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.801 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.802 DEBUG:boto:Method: GET 2022-09-03T06:23:45.803 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.803 DEBUG:boto:Data: 2022-09-03T06:23:45.804 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.804 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.805 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.805 DEBUG:boto:Params: {} 2022-09-03T06:23:45.806 DEBUG:boto:Token: None 2022-09-03T06:23:45.807 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.807 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qtyhJ/s7bd+wPjX2D0Li7a/qFX8= 2022-09-03T06:23:45.809 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qtyhJ/s7bd+wPjX2D0Li7a/qFX8=', 'Content-Length': '0'} 2022-09-03T06:23:45.811 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d75f956d2b1ba78f-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.813 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:45.814 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:45.815 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.816 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.817 DEBUG:boto:Method: GET 2022-09-03T06:23:45.818 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.819 DEBUG:boto:Data: 2022-09-03T06:23:45.819 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.820 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.821 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.822 DEBUG:boto:Params: {} 2022-09-03T06:23:45.823 DEBUG:boto:Token: None 2022-09-03T06:23:45.824 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.825 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qtyhJ/s7bd+wPjX2D0Li7a/qFX8= 2022-09-03T06:23:45.826 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qtyhJ/s7bd+wPjX2D0Li7a/qFX8=', 'Content-Length': '0'} 2022-09-03T06:23:45.828 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000095f937e1e1a532b4-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.829 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:45.830 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:45.830 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.831 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.831 DEBUG:boto:Method: HEAD 2022-09-03T06:23:45.832 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.832 DEBUG:boto:Data: 2022-09-03T06:23:45.833 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.833 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.834 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.834 DEBUG:boto:Params: {} 2022-09-03T06:23:45.835 DEBUG:boto:Token: None 2022-09-03T06:23:45.836 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.836 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vyZLqa21xEA8uIbXz6qBGTd9CVI= 2022-09-03T06:23:45.837 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vyZLqa21xEA8uIbXz6qBGTd9CVI=', 'Content-Length': '0'} 2022-09-03T06:23:45.839 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eaee4d61da7f4771-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.840 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:45.840 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:45.841 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.841 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.842 DEBUG:boto:Method: HEAD 2022-09-03T06:23:45.842 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.843 DEBUG:boto:Data: 2022-09-03T06:23:45.843 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.844 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.844 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.845 DEBUG:boto:Params: {} 2022-09-03T06:23:45.846 DEBUG:boto:Token: None 2022-09-03T06:23:45.846 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.847 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vyZLqa21xEA8uIbXz6qBGTd9CVI= 2022-09-03T06:23:45.847 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vyZLqa21xEA8uIbXz6qBGTd9CVI=', 'Content-Length': '0'} 2022-09-03T06:23:45.849 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000052dc801149475526-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.851 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:23:45.851 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:45.852 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:45.853 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.854 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.854 DEBUG:boto:Method: GET 2022-09-03T06:23:45.855 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.856 DEBUG:boto:Data: 2022-09-03T06:23:45.857 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.858 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.859 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.860 DEBUG:boto:Params: {} 2022-09-03T06:23:45.861 DEBUG:boto:Token: None 2022-09-03T06:23:45.862 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.863 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qtyhJ/s7bd+wPjX2D0Li7a/qFX8= 2022-09-03T06:23:45.864 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qtyhJ/s7bd+wPjX2D0Li7a/qFX8=', 'Content-Length': '0'} 2022-09-03T06:23:45.866 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000000b0885988cd78ba-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.867 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:45.868 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:45.868 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.869 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.869 DEBUG:boto:Method: GET 2022-09-03T06:23:45.870 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.870 DEBUG:boto:Data: 2022-09-03T06:23:45.871 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.871 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.872 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.873 DEBUG:boto:Params: {} 2022-09-03T06:23:45.873 DEBUG:boto:Token: None 2022-09-03T06:23:45.874 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:23:45.874 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qtyhJ/s7bd+wPjX2D0Li7a/qFX8= 2022-09-03T06:23:45.875 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qtyhJ/s7bd+wPjX2D0Li7a/qFX8=', 'Content-Length': '0'} 2022-09-03T06:23:45.877 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000079ca8c48da57ad45-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.878 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:45.879 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:45.879 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:45.880 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:45.880 DEBUG:boto:Method: GET 2022-09-03T06:23:45.881 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:23:45.881 DEBUG:boto:Data: 2022-09-03T06:23:45.882 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.883 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.883 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.884 DEBUG:boto:Params: {} 2022-09-03T06:23:45.884 DEBUG:boto:Token: None 2022-09-03T06:23:45.885 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/ 2022-09-03T06:23:45.886 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc= 2022-09-03T06:23:45.886 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc=', 'Content-Length': '0'} 2022-09-03T06:23:45.888 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000141d137f316237a3-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.930 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:45.931 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:45.932 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:45.932 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:45.933 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:45.934 DEBUG:boto:Method: GET 2022-09-03T06:23:45.935 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:23:45.935 DEBUG:boto:Data: 2022-09-03T06:23:45.936 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.936 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.937 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.937 DEBUG:boto:Params: {} 2022-09-03T06:23:45.938 DEBUG:boto:Token: None 2022-09-03T06:23:45.939 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/ 2022-09-03T06:23:45.939 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc= 2022-09-03T06:23:45.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v9T164QlIoMDwu6K5bvKCFTPKcc=', 'Content-Length': '0'} 2022-09-03T06:23:45.942 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005d167f44eba27f06-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.943 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:45.944 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:45.945 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:45.945 DEBUG:boto:Method: HEAD 2022-09-03T06:23:45.946 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:23:45.947 DEBUG:boto:Data: 2022-09-03T06:23:45.948 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.949 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.950 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.951 DEBUG:boto:Params: {} 2022-09-03T06:23:45.952 DEBUG:boto:Token: None 2022-09-03T06:23:45.953 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/d 2022-09-03T06:23:45.954 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rp313MiOOsZyfY/4Z5zSj9AtCys= 2022-09-03T06:23:45.955 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rp313MiOOsZyfY/4Z5zSj9AtCys=', 'Content-Length': '0'} 2022-09-03T06:23:45.958 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000073f6645d9a969ab7-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.959 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:45.960 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:45.961 DEBUG:boto:Method: HEAD 2022-09-03T06:23:45.962 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:23:45.963 DEBUG:boto:Data: 2022-09-03T06:23:45.964 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.965 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.966 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.967 DEBUG:boto:Params: {} 2022-09-03T06:23:45.968 DEBUG:boto:Token: None 2022-09-03T06:23:45.969 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/d 2022-09-03T06:23:45.970 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rp313MiOOsZyfY/4Z5zSj9AtCys= 2022-09-03T06:23:45.971 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rp313MiOOsZyfY/4Z5zSj9AtCys=', 'Content-Length': '0'} 2022-09-03T06:23:45.973 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a1e3d12e5feac85-006312f2f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.975 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:23:45.975 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:45.976 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:45.977 DEBUG:boto:Method: GET 2022-09-03T06:23:45.978 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:23:45.986 DEBUG:boto:Data: 2022-09-03T06:23:45.987 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.987 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:45.988 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.988 DEBUG:boto:Params: {} 2022-09-03T06:23:45.989 DEBUG:boto:Token: None 2022-09-03T06:23:45.990 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/d 2022-09-03T06:23:45.990 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yCJg6Z59I8yKv1HH3yILtH4+ag8= 2022-09-03T06:23:45.991 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yCJg6Z59I8yKv1HH3yILtH4+ag8=', 'Content-Length': '0'} 2022-09-03T06:23:45.993 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f8b518c6902e74e-006312f2f1-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:45.994 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:45.994 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:45.995 DEBUG:boto:Method: GET 2022-09-03T06:23:45.996 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:23:45.996 DEBUG:boto:Data: 2022-09-03T06:23:45.997 DEBUG:boto:Headers: {} 2022-09-03T06:23:45.997 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:45.998 DEBUG:boto:Port: 8000 2022-09-03T06:23:45.998 DEBUG:boto:Params: {} 2022-09-03T06:23:45.999 DEBUG:boto:Token: None 2022-09-03T06:23:45.999 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:45 GMT /dsgvcy-72/d 2022-09-03T06:23:46.000 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yCJg6Z59I8yKv1HH3yILtH4+ag8= 2022-09-03T06:23:46.001 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yCJg6Z59I8yKv1HH3yILtH4+ag8=', 'Content-Length': '0'} 2022-09-03T06:23:46.003 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000088fcb0cc678618ab-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.046 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:46.047 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:46.048 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:46.048 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:46.049 DEBUG:boto:Method: GET 2022-09-03T06:23:46.049 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:23:46.050 DEBUG:boto:Data: 2022-09-03T06:23:46.051 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.051 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.052 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.053 DEBUG:boto:Params: {} 2022-09-03T06:23:46.053 DEBUG:boto:Token: None 2022-09-03T06:23:46.054 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/ 2022-09-03T06:23:46.055 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA= 2022-09-03T06:23:46.055 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA=', 'Content-Length': '0'} 2022-09-03T06:23:46.058 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007626a12ceba65f71-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.058 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:46.059 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:46.060 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:46.061 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:46.061 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:46.062 DEBUG:boto:Method: GET 2022-09-03T06:23:46.063 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:23:46.063 DEBUG:boto:Data: 2022-09-03T06:23:46.064 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.065 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.066 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.066 DEBUG:boto:Params: {} 2022-09-03T06:23:46.067 DEBUG:boto:Token: None 2022-09-03T06:23:46.068 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/ 2022-09-03T06:23:46.068 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA= 2022-09-03T06:23:46.069 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA=', 'Content-Length': '0'} 2022-09-03T06:23:46.071 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004cb92de0700279b0-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.072 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:46.072 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:23:46.073 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:46.074 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:46.074 DEBUG:boto:path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.075 DEBUG:boto:auth_path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.076 DEBUG:boto:Method: GET 2022-09-03T06:23:46.076 DEBUG:boto:Path: /dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.077 DEBUG:boto:Data: 2022-09-03T06:23:46.077 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.078 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.079 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.079 DEBUG:boto:Params: {} 2022-09-03T06:23:46.080 DEBUG:boto:Token: None 2022-09-03T06:23:46.081 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.081 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z2jcn/jI8ZhuTk6KZHH2xJw/SGQ= 2022-09-03T06:23:46.082 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z2jcn/jI8ZhuTk6KZHH2xJw/SGQ=', 'Content-Length': '0'} 2022-09-03T06:23:46.084 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000057a6d393347f03a4-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.085 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:46.085 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:46.086 DEBUG:boto:path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.087 DEBUG:boto:auth_path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.087 DEBUG:boto:Method: GET 2022-09-03T06:23:46.088 DEBUG:boto:Path: /dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.088 DEBUG:boto:Data: 2022-09-03T06:23:46.089 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.089 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.090 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.091 DEBUG:boto:Params: {} 2022-09-03T06:23:46.091 DEBUG:boto:Token: None 2022-09-03T06:23:46.092 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.092 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z2jcn/jI8ZhuTk6KZHH2xJw/SGQ= 2022-09-03T06:23:46.093 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z2jcn/jI8ZhuTk6KZHH2xJw/SGQ=', 'Content-Length': '0'} 2022-09-03T06:23:46.095 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000033564b9f7fa5328f-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.096 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:46.097 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:46.097 DEBUG:boto:path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.098 DEBUG:boto:auth_path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.098 DEBUG:boto:Method: HEAD 2022-09-03T06:23:46.099 DEBUG:boto:Path: /dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.099 DEBUG:boto:Data: 2022-09-03T06:23:46.100 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.100 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.101 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.101 DEBUG:boto:Params: {} 2022-09-03T06:23:46.102 DEBUG:boto:Token: None 2022-09-03T06:23:46.103 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.103 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k1sxBFydW9UiPU9Y3w8phy4UrjQ= 2022-09-03T06:23:46.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k1sxBFydW9UiPU9Y3w8phy4UrjQ=', 'Content-Length': '0'} 2022-09-03T06:23:46.106 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bde61fb6a6c29fe0-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.107 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:46.107 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:46.108 DEBUG:boto:path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.108 DEBUG:boto:auth_path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.109 DEBUG:boto:Method: HEAD 2022-09-03T06:23:46.109 DEBUG:boto:Path: /dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.110 DEBUG:boto:Data: 2022-09-03T06:23:46.110 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.111 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.111 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.112 DEBUG:boto:Params: {} 2022-09-03T06:23:46.113 DEBUG:boto:Token: None 2022-09-03T06:23:46.113 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.114 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k1sxBFydW9UiPU9Y3w8phy4UrjQ= 2022-09-03T06:23:46.114 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k1sxBFydW9UiPU9Y3w8phy4UrjQ=', 'Content-Length': '0'} 2022-09-03T06:23:46.116 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005ae62ac58e6cd16c-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.117 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:23:46.118 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:46.118 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:46.119 DEBUG:boto:path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.119 DEBUG:boto:auth_path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.120 DEBUG:boto:Method: GET 2022-09-03T06:23:46.120 DEBUG:boto:Path: /dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.121 DEBUG:boto:Data: 2022-09-03T06:23:46.122 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.122 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.123 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.123 DEBUG:boto:Params: {} 2022-09-03T06:23:46.124 DEBUG:boto:Token: None 2022-09-03T06:23:46.124 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.125 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z2jcn/jI8ZhuTk6KZHH2xJw/SGQ= 2022-09-03T06:23:46.126 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z2jcn/jI8ZhuTk6KZHH2xJw/SGQ=', 'Content-Length': '0'} 2022-09-03T06:23:46.128 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b78e3e2d58f20b1f-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.129 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:46.130 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:46.130 DEBUG:boto:path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.131 DEBUG:boto:auth_path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.131 DEBUG:boto:Method: GET 2022-09-03T06:23:46.132 DEBUG:boto:Path: /dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.132 DEBUG:boto:Data: 2022-09-03T06:23:46.133 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.133 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.134 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.134 DEBUG:boto:Params: {} 2022-09-03T06:23:46.135 DEBUG:boto:Token: None 2022-09-03T06:23:46.136 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/e?versionId=null 2022-09-03T06:23:46.137 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z2jcn/jI8ZhuTk6KZHH2xJw/SGQ= 2022-09-03T06:23:46.137 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z2jcn/jI8ZhuTk6KZHH2xJw/SGQ=', 'Content-Length': '0'} 2022-09-03T06:23:46.139 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000007c5c824f4971351-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.140 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:46.141 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:46.141 DEBUG:boto:path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:46.142 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:46.142 DEBUG:boto:Method: GET 2022-09-03T06:23:46.143 DEBUG:boto:Path: /dsgvcy-72/?prefix=e 2022-09-03T06:23:46.144 DEBUG:boto:Data: 2022-09-03T06:23:46.144 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.145 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.145 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.146 DEBUG:boto:Params: {} 2022-09-03T06:23:46.147 DEBUG:boto:Token: None 2022-09-03T06:23:46.147 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/ 2022-09-03T06:23:46.148 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA= 2022-09-03T06:23:46.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA=', 'Content-Length': '0'} 2022-09-03T06:23:46.151 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bf6614128fca5802-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.152 DEBUG:boto:b'dsgvcy-72e1000falsee2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:46.152 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:46.153 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:46.153 DEBUG:boto:path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:46.154 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:46.155 DEBUG:boto:Method: GET 2022-09-03T06:23:46.155 DEBUG:boto:Path: /dsgvcy-72/?prefix=e 2022-09-03T06:23:46.156 DEBUG:boto:Data: 2022-09-03T06:23:46.157 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.158 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.158 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.159 DEBUG:boto:Params: {} 2022-09-03T06:23:46.159 DEBUG:boto:Token: None 2022-09-03T06:23:46.160 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/ 2022-09-03T06:23:46.161 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA= 2022-09-03T06:23:46.161 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA=', 'Content-Length': '0'} 2022-09-03T06:23:46.164 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a335b7c02a911219-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.164 DEBUG:boto:b'dsgvcy-72e1000falsee2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:46.165 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:46.166 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:46.166 DEBUG:boto:Method: HEAD 2022-09-03T06:23:46.167 DEBUG:boto:Path: /dsgvcy-72/e 2022-09-03T06:23:46.167 DEBUG:boto:Data: 2022-09-03T06:23:46.168 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.168 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.169 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.170 DEBUG:boto:Params: {} 2022-09-03T06:23:46.171 DEBUG:boto:Token: None 2022-09-03T06:23:46.172 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/e 2022-09-03T06:23:46.173 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DeXu7Rt6s39odhqMCSIxrJBv5QA= 2022-09-03T06:23:46.174 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DeXu7Rt6s39odhqMCSIxrJBv5QA=', 'Content-Length': '0'} 2022-09-03T06:23:46.176 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000be76ad964925018f-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.177 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:46.178 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:46.178 DEBUG:boto:Method: HEAD 2022-09-03T06:23:46.179 DEBUG:boto:Path: /dsgvcy-72/e 2022-09-03T06:23:46.179 DEBUG:boto:Data: 2022-09-03T06:23:46.180 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.180 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.181 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.181 DEBUG:boto:Params: {} 2022-09-03T06:23:46.182 DEBUG:boto:Token: None 2022-09-03T06:23:46.182 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/e 2022-09-03T06:23:46.183 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DeXu7Rt6s39odhqMCSIxrJBv5QA= 2022-09-03T06:23:46.183 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DeXu7Rt6s39odhqMCSIxrJBv5QA=', 'Content-Length': '0'} 2022-09-03T06:23:46.186 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000060ccece3d8768fcb-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.186 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:23:46.187 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:46.188 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:46.188 DEBUG:boto:Method: GET 2022-09-03T06:23:46.189 DEBUG:boto:Path: /dsgvcy-72/e 2022-09-03T06:23:46.189 DEBUG:boto:Data: 2022-09-03T06:23:46.190 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.190 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.191 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.191 DEBUG:boto:Params: {} 2022-09-03T06:23:46.192 DEBUG:boto:Token: None 2022-09-03T06:23:46.192 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/e 2022-09-03T06:23:46.193 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5MeFGJsKscBLcjFPFSE0LqNIdws= 2022-09-03T06:23:46.194 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5MeFGJsKscBLcjFPFSE0LqNIdws=', 'Content-Length': '0'} 2022-09-03T06:23:46.196 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a9f8f77b88af22e-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.197 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:46.197 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:46.198 DEBUG:boto:Method: GET 2022-09-03T06:23:46.199 DEBUG:boto:Path: /dsgvcy-72/e 2022-09-03T06:23:46.199 DEBUG:boto:Data: 2022-09-03T06:23:46.200 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.200 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.201 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.201 DEBUG:boto:Params: {} 2022-09-03T06:23:46.202 DEBUG:boto:Token: None 2022-09-03T06:23:46.203 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/e 2022-09-03T06:23:46.203 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5MeFGJsKscBLcjFPFSE0LqNIdws= 2022-09-03T06:23:46.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5MeFGJsKscBLcjFPFSE0LqNIdws=', 'Content-Length': '0'} 2022-09-03T06:23:46.206 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004b92eac7869d4848-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.207 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:46.207 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:46.208 DEBUG:boto:path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:46.208 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:46.209 DEBUG:boto:Method: GET 2022-09-03T06:23:46.210 DEBUG:boto:Path: /dsgvcy-72/?prefix=e 2022-09-03T06:23:46.210 DEBUG:boto:Data: 2022-09-03T06:23:46.211 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.211 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.212 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.212 DEBUG:boto:Params: {} 2022-09-03T06:23:46.213 DEBUG:boto:Token: None 2022-09-03T06:23:46.213 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/ 2022-09-03T06:23:46.214 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA= 2022-09-03T06:23:46.215 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA=', 'Content-Length': '0'} 2022-09-03T06:23:46.217 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000007e53edc9770d4e-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.218 DEBUG:boto:b'dsgvcy-72e1000falsee2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:46.219 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:46.220 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:46.220 DEBUG:boto:path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:46.221 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:46.221 DEBUG:boto:Method: GET 2022-09-03T06:23:46.222 DEBUG:boto:Path: /dsgvcy-72/?prefix=e 2022-09-03T06:23:46.223 DEBUG:boto:Data: 2022-09-03T06:23:46.223 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.224 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.224 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.225 DEBUG:boto:Params: {} 2022-09-03T06:23:46.226 DEBUG:boto:Token: None 2022-09-03T06:23:46.226 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/ 2022-09-03T06:23:46.227 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA= 2022-09-03T06:23:46.228 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA=', 'Content-Length': '0'} 2022-09-03T06:23:46.230 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d72a8cb1784eed00-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.231 DEBUG:boto:b'dsgvcy-72e1000falsee2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:46.233 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:23:46.234 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:46.235 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:46.236 DEBUG:boto:path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.236 DEBUG:boto:auth_path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.237 DEBUG:boto:Method: GET 2022-09-03T06:23:46.238 DEBUG:boto:Path: /dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.239 DEBUG:boto:Data: 2022-09-03T06:23:46.241 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.241 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.242 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.243 DEBUG:boto:Params: {} 2022-09-03T06:23:46.243 DEBUG:boto:Token: None 2022-09-03T06:23:46.244 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.245 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EhbV0Z8qNfZxDSkqXNSRicf8DRo= 2022-09-03T06:23:46.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EhbV0Z8qNfZxDSkqXNSRicf8DRo=', 'Content-Length': '0'} 2022-09-03T06:23:46.248 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cc802005ad6f7457-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.249 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:46.250 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:46.250 DEBUG:boto:path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.251 DEBUG:boto:auth_path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.252 DEBUG:boto:Method: GET 2022-09-03T06:23:46.252 DEBUG:boto:Path: /dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.253 DEBUG:boto:Data: 2022-09-03T06:23:46.253 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.254 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.254 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.255 DEBUG:boto:Params: {} 2022-09-03T06:23:46.256 DEBUG:boto:Token: None 2022-09-03T06:23:46.257 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.259 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EhbV0Z8qNfZxDSkqXNSRicf8DRo= 2022-09-03T06:23:46.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EhbV0Z8qNfZxDSkqXNSRicf8DRo=', 'Content-Length': '0'} 2022-09-03T06:23:46.264 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad8ddb75c2514fac-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.310 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:46.311 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:46.312 DEBUG:boto:path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.312 DEBUG:boto:auth_path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.313 DEBUG:boto:Method: HEAD 2022-09-03T06:23:46.313 DEBUG:boto:Path: /dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.314 DEBUG:boto:Data: 2022-09-03T06:23:46.314 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.315 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.315 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.316 DEBUG:boto:Params: {} 2022-09-03T06:23:46.316 DEBUG:boto:Token: None 2022-09-03T06:23:46.317 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.318 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PauNEtGL/rG1/Y2EzvP9BQiwPvo= 2022-09-03T06:23:46.318 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PauNEtGL/rG1/Y2EzvP9BQiwPvo=', 'Content-Length': '0'} 2022-09-03T06:23:46.322 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f4cb3fd7c46fc9ab-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.323 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:46.324 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:46.325 DEBUG:boto:path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.326 DEBUG:boto:auth_path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.327 DEBUG:boto:Method: HEAD 2022-09-03T06:23:46.328 DEBUG:boto:Path: /dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.329 DEBUG:boto:Data: 2022-09-03T06:23:46.329 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.330 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.331 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.332 DEBUG:boto:Params: {} 2022-09-03T06:23:46.333 DEBUG:boto:Token: None 2022-09-03T06:23:46.334 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.336 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PauNEtGL/rG1/Y2EzvP9BQiwPvo= 2022-09-03T06:23:46.337 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PauNEtGL/rG1/Y2EzvP9BQiwPvo=', 'Content-Length': '0'} 2022-09-03T06:23:46.340 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002624c28d562f0e2d-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.342 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:23:46.343 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:46.344 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:46.345 DEBUG:boto:path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.346 DEBUG:boto:auth_path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.347 DEBUG:boto:Method: GET 2022-09-03T06:23:46.348 DEBUG:boto:Path: /dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.349 DEBUG:boto:Data: 2022-09-03T06:23:46.350 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.351 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.352 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.353 DEBUG:boto:Params: {} 2022-09-03T06:23:46.354 DEBUG:boto:Token: None 2022-09-03T06:23:46.355 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.356 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EhbV0Z8qNfZxDSkqXNSRicf8DRo= 2022-09-03T06:23:46.357 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EhbV0Z8qNfZxDSkqXNSRicf8DRo=', 'Content-Length': '0'} 2022-09-03T06:23:46.360 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d0f46fe6fbeef498-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.361 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:46.362 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:46.363 DEBUG:boto:path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.364 DEBUG:boto:auth_path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.365 DEBUG:boto:Method: GET 2022-09-03T06:23:46.365 DEBUG:boto:Path: /dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.366 DEBUG:boto:Data: 2022-09-03T06:23:46.367 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.368 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.369 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.370 DEBUG:boto:Params: {} 2022-09-03T06:23:46.371 DEBUG:boto:Token: None 2022-09-03T06:23:46.372 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/f?versionId=null 2022-09-03T06:23:46.373 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EhbV0Z8qNfZxDSkqXNSRicf8DRo= 2022-09-03T06:23:46.374 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EhbV0Z8qNfZxDSkqXNSRicf8DRo=', 'Content-Length': '0'} 2022-09-03T06:23:46.376 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000f0317ab6c01eb1f-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.377 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:46.378 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:46.378 DEBUG:boto:path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:46.379 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:46.380 DEBUG:boto:Method: GET 2022-09-03T06:23:46.380 DEBUG:boto:Path: /dsgvcy-72/?prefix=f 2022-09-03T06:23:46.381 DEBUG:boto:Data: 2022-09-03T06:23:46.382 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.382 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.383 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.383 DEBUG:boto:Params: {} 2022-09-03T06:23:46.384 DEBUG:boto:Token: None 2022-09-03T06:23:46.385 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/ 2022-09-03T06:23:46.385 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA= 2022-09-03T06:23:46.386 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA=', 'Content-Length': '0'} 2022-09-03T06:23:46.389 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000011aeb8af85038504-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.390 DEBUG:boto:b'dsgvcy-72f1000falsef2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:46.391 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:46.391 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:46.392 DEBUG:boto:path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:46.392 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:46.393 DEBUG:boto:Method: GET 2022-09-03T06:23:46.394 DEBUG:boto:Path: /dsgvcy-72/?prefix=f 2022-09-03T06:23:46.394 DEBUG:boto:Data: 2022-09-03T06:23:46.395 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.395 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.396 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.397 DEBUG:boto:Params: {} 2022-09-03T06:23:46.397 DEBUG:boto:Token: None 2022-09-03T06:23:46.398 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/ 2022-09-03T06:23:46.399 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA= 2022-09-03T06:23:46.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA=', 'Content-Length': '0'} 2022-09-03T06:23:46.402 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000728a54566526623c-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.403 DEBUG:boto:b'dsgvcy-72f1000falsef2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:46.404 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:46.404 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:46.405 DEBUG:boto:Method: HEAD 2022-09-03T06:23:46.406 DEBUG:boto:Path: /dsgvcy-72/f 2022-09-03T06:23:46.406 DEBUG:boto:Data: 2022-09-03T06:23:46.407 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.408 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.409 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.409 DEBUG:boto:Params: {} 2022-09-03T06:23:46.410 DEBUG:boto:Token: None 2022-09-03T06:23:46.410 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/f 2022-09-03T06:23:46.411 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WniSQnzDj7vRx3BjzYujsmd/nPU= 2022-09-03T06:23:46.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WniSQnzDj7vRx3BjzYujsmd/nPU=', 'Content-Length': '0'} 2022-09-03T06:23:46.414 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007fcb91c39c7139fc-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.415 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:46.416 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:46.416 DEBUG:boto:Method: HEAD 2022-09-03T06:23:46.417 DEBUG:boto:Path: /dsgvcy-72/f 2022-09-03T06:23:46.417 DEBUG:boto:Data: 2022-09-03T06:23:46.418 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.418 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.419 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.420 DEBUG:boto:Params: {} 2022-09-03T06:23:46.420 DEBUG:boto:Token: None 2022-09-03T06:23:46.421 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/f 2022-09-03T06:23:46.422 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WniSQnzDj7vRx3BjzYujsmd/nPU= 2022-09-03T06:23:46.422 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WniSQnzDj7vRx3BjzYujsmd/nPU=', 'Content-Length': '0'} 2022-09-03T06:23:46.426 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006a3d32d106b358a1-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.427 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:23:46.427 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:46.428 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:46.428 DEBUG:boto:Method: GET 2022-09-03T06:23:46.429 DEBUG:boto:Path: /dsgvcy-72/f 2022-09-03T06:23:46.429 DEBUG:boto:Data: 2022-09-03T06:23:46.430 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.430 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.431 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.432 DEBUG:boto:Params: {} 2022-09-03T06:23:46.432 DEBUG:boto:Token: None 2022-09-03T06:23:46.433 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/f 2022-09-03T06:23:46.433 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MWOAnAZyEH/jRl/eFQ9aM0TVNp4= 2022-09-03T06:23:46.434 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MWOAnAZyEH/jRl/eFQ9aM0TVNp4=', 'Content-Length': '0'} 2022-09-03T06:23:46.436 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000062d4a7d2635aa651-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.478 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:46.479 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:46.479 DEBUG:boto:Method: GET 2022-09-03T06:23:46.480 DEBUG:boto:Path: /dsgvcy-72/f 2022-09-03T06:23:46.481 DEBUG:boto:Data: 2022-09-03T06:23:46.481 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.482 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.482 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.483 DEBUG:boto:Params: {} 2022-09-03T06:23:46.483 DEBUG:boto:Token: None 2022-09-03T06:23:46.484 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/f 2022-09-03T06:23:46.485 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MWOAnAZyEH/jRl/eFQ9aM0TVNp4= 2022-09-03T06:23:46.485 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MWOAnAZyEH/jRl/eFQ9aM0TVNp4=', 'Content-Length': '0'} 2022-09-03T06:23:46.487 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001fade33aa38b4233-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.488 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:46.489 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:46.489 DEBUG:boto:path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:46.490 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:46.490 DEBUG:boto:Method: GET 2022-09-03T06:23:46.491 DEBUG:boto:Path: /dsgvcy-72/?prefix=f 2022-09-03T06:23:46.491 DEBUG:boto:Data: 2022-09-03T06:23:46.492 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.492 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.493 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.494 DEBUG:boto:Params: {} 2022-09-03T06:23:46.494 DEBUG:boto:Token: None 2022-09-03T06:23:46.495 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/ 2022-09-03T06:23:46.495 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA= 2022-09-03T06:23:46.496 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA=', 'Content-Length': '0'} 2022-09-03T06:23:46.498 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f12339e3bad1efbd-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.499 DEBUG:boto:b'dsgvcy-72f1000falsef2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:46.500 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:46.501 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:46.501 DEBUG:boto:path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:46.502 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:46.502 DEBUG:boto:Method: GET 2022-09-03T06:23:46.503 DEBUG:boto:Path: /dsgvcy-72/?prefix=f 2022-09-03T06:23:46.503 DEBUG:boto:Data: 2022-09-03T06:23:46.504 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.504 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.505 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.505 DEBUG:boto:Params: {} 2022-09-03T06:23:46.506 DEBUG:boto:Token: None 2022-09-03T06:23:46.507 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/ 2022-09-03T06:23:46.507 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA= 2022-09-03T06:23:46.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA=', 'Content-Length': '0'} 2022-09-03T06:23:46.510 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000022a209686da796e1-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.511 DEBUG:boto:b'dsgvcy-72f1000falsef2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:46.512 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:23:46.512 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:46.513 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:46.513 DEBUG:boto:path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.514 DEBUG:boto:auth_path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.514 DEBUG:boto:Method: GET 2022-09-03T06:23:46.515 DEBUG:boto:Path: /dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.515 DEBUG:boto:Data: 2022-09-03T06:23:46.516 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.516 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.517 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.518 DEBUG:boto:Params: {} 2022-09-03T06:23:46.518 DEBUG:boto:Token: None 2022-09-03T06:23:46.519 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.519 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pB8VSZw9+M3taQPxokPJqoOq8ss= 2022-09-03T06:23:46.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pB8VSZw9+M3taQPxokPJqoOq8ss=', 'Content-Length': '0'} 2022-09-03T06:23:46.522 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004130538592b6be8d-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.523 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:46.524 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:46.524 DEBUG:boto:path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.525 DEBUG:boto:auth_path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.526 DEBUG:boto:Method: GET 2022-09-03T06:23:46.526 DEBUG:boto:Path: /dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.527 DEBUG:boto:Data: 2022-09-03T06:23:46.527 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.528 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.529 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.529 DEBUG:boto:Params: {} 2022-09-03T06:23:46.530 DEBUG:boto:Token: None 2022-09-03T06:23:46.530 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.531 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pB8VSZw9+M3taQPxokPJqoOq8ss= 2022-09-03T06:23:46.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pB8VSZw9+M3taQPxokPJqoOq8ss=', 'Content-Length': '0'} 2022-09-03T06:23:46.534 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009058c29ea541bceb-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.578 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:46.579 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:46.579 DEBUG:boto:path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.580 DEBUG:boto:auth_path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.580 DEBUG:boto:Method: HEAD 2022-09-03T06:23:46.581 DEBUG:boto:Path: /dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.581 DEBUG:boto:Data: 2022-09-03T06:23:46.582 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.582 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.583 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.583 DEBUG:boto:Params: {} 2022-09-03T06:23:46.584 DEBUG:boto:Token: None 2022-09-03T06:23:46.584 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.585 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WXWe1sQHewj1iUJKUF9fPJ2RPUk= 2022-09-03T06:23:46.586 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WXWe1sQHewj1iUJKUF9fPJ2RPUk=', 'Content-Length': '0'} 2022-09-03T06:23:46.588 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004b0d0d94dfba7fa6-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.588 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:46.589 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:46.589 DEBUG:boto:path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.590 DEBUG:boto:auth_path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.590 DEBUG:boto:Method: HEAD 2022-09-03T06:23:46.591 DEBUG:boto:Path: /dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.591 DEBUG:boto:Data: 2022-09-03T06:23:46.592 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.592 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.593 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.593 DEBUG:boto:Params: {} 2022-09-03T06:23:46.594 DEBUG:boto:Token: None 2022-09-03T06:23:46.595 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.595 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WXWe1sQHewj1iUJKUF9fPJ2RPUk= 2022-09-03T06:23:46.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WXWe1sQHewj1iUJKUF9fPJ2RPUk=', 'Content-Length': '0'} 2022-09-03T06:23:46.598 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001da95bafbcdff0f3-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.598 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:23:46.599 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:46.600 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:46.600 DEBUG:boto:path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.601 DEBUG:boto:auth_path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.601 DEBUG:boto:Method: GET 2022-09-03T06:23:46.602 DEBUG:boto:Path: /dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.602 DEBUG:boto:Data: 2022-09-03T06:23:46.603 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.603 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.604 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.604 DEBUG:boto:Params: {} 2022-09-03T06:23:46.605 DEBUG:boto:Token: None 2022-09-03T06:23:46.606 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.606 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pB8VSZw9+M3taQPxokPJqoOq8ss= 2022-09-03T06:23:46.607 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pB8VSZw9+M3taQPxokPJqoOq8ss=', 'Content-Length': '0'} 2022-09-03T06:23:46.609 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000910ff33618134ff2-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.610 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:46.610 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:46.611 DEBUG:boto:path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.612 DEBUG:boto:auth_path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.612 DEBUG:boto:Method: GET 2022-09-03T06:23:46.613 DEBUG:boto:Path: /dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.613 DEBUG:boto:Data: 2022-09-03T06:23:46.614 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.614 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.615 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.615 DEBUG:boto:Params: {} 2022-09-03T06:23:46.616 DEBUG:boto:Token: None 2022-09-03T06:23:46.616 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/g?versionId=null 2022-09-03T06:23:46.617 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pB8VSZw9+M3taQPxokPJqoOq8ss= 2022-09-03T06:23:46.618 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pB8VSZw9+M3taQPxokPJqoOq8ss=', 'Content-Length': '0'} 2022-09-03T06:23:46.620 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fedecfd7b0c77401-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.621 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:46.621 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:46.622 DEBUG:boto:path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:46.622 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:46.623 DEBUG:boto:Method: GET 2022-09-03T06:23:46.623 DEBUG:boto:Path: /dsgvcy-72/?prefix=g 2022-09-03T06:23:46.624 DEBUG:boto:Data: 2022-09-03T06:23:46.624 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.625 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.626 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.626 DEBUG:boto:Params: {} 2022-09-03T06:23:46.627 DEBUG:boto:Token: None 2022-09-03T06:23:46.627 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/ 2022-09-03T06:23:46.628 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA= 2022-09-03T06:23:46.629 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA=', 'Content-Length': '0'} 2022-09-03T06:23:46.631 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e667cdde385ddaa4-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.631 DEBUG:boto:b'dsgvcy-72g1000falseg2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:46.632 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:46.633 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:46.633 DEBUG:boto:path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:46.634 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:46.634 DEBUG:boto:Method: GET 2022-09-03T06:23:46.635 DEBUG:boto:Path: /dsgvcy-72/?prefix=g 2022-09-03T06:23:46.636 DEBUG:boto:Data: 2022-09-03T06:23:46.636 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.637 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.637 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.638 DEBUG:boto:Params: {} 2022-09-03T06:23:46.638 DEBUG:boto:Token: None 2022-09-03T06:23:46.639 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/ 2022-09-03T06:23:46.639 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA= 2022-09-03T06:23:46.640 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA=', 'Content-Length': '0'} 2022-09-03T06:23:46.642 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f0c06e1f8ffdbcb0-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.643 DEBUG:boto:b'dsgvcy-72g1000falseg2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:46.643 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:46.644 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:46.645 DEBUG:boto:Method: HEAD 2022-09-03T06:23:46.645 DEBUG:boto:Path: /dsgvcy-72/g 2022-09-03T06:23:46.646 DEBUG:boto:Data: 2022-09-03T06:23:46.646 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.647 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.647 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.648 DEBUG:boto:Params: {} 2022-09-03T06:23:46.648 DEBUG:boto:Token: None 2022-09-03T06:23:46.649 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/g 2022-09-03T06:23:46.649 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VI1OxWlUYXv5lPoy3GJ3lQvpWow= 2022-09-03T06:23:46.650 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VI1OxWlUYXv5lPoy3GJ3lQvpWow=', 'Content-Length': '0'} 2022-09-03T06:23:46.652 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000041c5a995a8ebcfc6-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.653 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:46.653 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:46.654 DEBUG:boto:Method: HEAD 2022-09-03T06:23:46.654 DEBUG:boto:Path: /dsgvcy-72/g 2022-09-03T06:23:46.655 DEBUG:boto:Data: 2022-09-03T06:23:46.655 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.656 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.656 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.657 DEBUG:boto:Params: {} 2022-09-03T06:23:46.657 DEBUG:boto:Token: None 2022-09-03T06:23:46.658 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/g 2022-09-03T06:23:46.658 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VI1OxWlUYXv5lPoy3GJ3lQvpWow= 2022-09-03T06:23:46.659 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VI1OxWlUYXv5lPoy3GJ3lQvpWow=', 'Content-Length': '0'} 2022-09-03T06:23:46.661 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007674a341ed43b8d3-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.662 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:23:46.663 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:46.663 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:46.664 DEBUG:boto:Method: GET 2022-09-03T06:23:46.664 DEBUG:boto:Path: /dsgvcy-72/g 2022-09-03T06:23:46.665 DEBUG:boto:Data: 2022-09-03T06:23:46.665 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.666 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.666 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.667 DEBUG:boto:Params: {} 2022-09-03T06:23:46.667 DEBUG:boto:Token: None 2022-09-03T06:23:46.668 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/g 2022-09-03T06:23:46.669 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lnGAGlKDiGl8s5PxQuhMozRVaSE= 2022-09-03T06:23:46.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lnGAGlKDiGl8s5PxQuhMozRVaSE=', 'Content-Length': '0'} 2022-09-03T06:23:46.672 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000092df94d15dca9b41-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.718 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:46.719 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:46.719 DEBUG:boto:Method: GET 2022-09-03T06:23:46.720 DEBUG:boto:Path: /dsgvcy-72/g 2022-09-03T06:23:46.721 DEBUG:boto:Data: 2022-09-03T06:23:46.721 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.722 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.722 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.723 DEBUG:boto:Params: {} 2022-09-03T06:23:46.724 DEBUG:boto:Token: None 2022-09-03T06:23:46.724 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/g 2022-09-03T06:23:46.725 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lnGAGlKDiGl8s5PxQuhMozRVaSE= 2022-09-03T06:23:46.725 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lnGAGlKDiGl8s5PxQuhMozRVaSE=', 'Content-Length': '0'} 2022-09-03T06:23:46.727 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000332f9fc72a87bc52-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.728 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:46.729 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:46.729 DEBUG:boto:path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:46.730 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:46.731 DEBUG:boto:Method: GET 2022-09-03T06:23:46.731 DEBUG:boto:Path: /dsgvcy-72/?prefix=g 2022-09-03T06:23:46.732 DEBUG:boto:Data: 2022-09-03T06:23:46.732 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.733 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.733 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.734 DEBUG:boto:Params: {} 2022-09-03T06:23:46.735 DEBUG:boto:Token: None 2022-09-03T06:23:46.736 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/ 2022-09-03T06:23:46.738 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA= 2022-09-03T06:23:46.739 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA=', 'Content-Length': '0'} 2022-09-03T06:23:46.742 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000039e6d9575df3587b-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.743 DEBUG:boto:b'dsgvcy-72g1000falseg2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:46.744 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:46.744 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:46.745 DEBUG:boto:path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:46.745 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:46.746 DEBUG:boto:Method: GET 2022-09-03T06:23:46.746 DEBUG:boto:Path: /dsgvcy-72/?prefix=g 2022-09-03T06:23:46.747 DEBUG:boto:Data: 2022-09-03T06:23:46.747 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.748 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.748 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.749 DEBUG:boto:Params: {} 2022-09-03T06:23:46.749 DEBUG:boto:Token: None 2022-09-03T06:23:46.750 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/ 2022-09-03T06:23:46.751 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA= 2022-09-03T06:23:46.751 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA=', 'Content-Length': '0'} 2022-09-03T06:23:46.753 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e0b3db257d250050-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.754 DEBUG:boto:b'dsgvcy-72g1000falseg2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:46.755 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:23:46.755 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:46.756 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:46.756 DEBUG:boto:path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.757 DEBUG:boto:auth_path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.757 DEBUG:boto:Method: GET 2022-09-03T06:23:46.758 DEBUG:boto:Path: /dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.758 DEBUG:boto:Data: 2022-09-03T06:23:46.759 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.759 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.760 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.760 DEBUG:boto:Params: {} 2022-09-03T06:23:46.761 DEBUG:boto:Token: None 2022-09-03T06:23:46.762 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.762 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vbu9y4+832ejNw9VxHeXbD6FSII= 2022-09-03T06:23:46.763 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vbu9y4+832ejNw9VxHeXbD6FSII=', 'Content-Length': '0'} 2022-09-03T06:23:46.765 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b256482fb3f7af4-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.766 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:46.767 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:46.767 DEBUG:boto:path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.768 DEBUG:boto:auth_path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.768 DEBUG:boto:Method: GET 2022-09-03T06:23:46.769 DEBUG:boto:Path: /dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.769 DEBUG:boto:Data: 2022-09-03T06:23:46.770 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.770 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.771 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.771 DEBUG:boto:Params: {} 2022-09-03T06:23:46.772 DEBUG:boto:Token: None 2022-09-03T06:23:46.772 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.773 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vbu9y4+832ejNw9VxHeXbD6FSII= 2022-09-03T06:23:46.774 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vbu9y4+832ejNw9VxHeXbD6FSII=', 'Content-Length': '0'} 2022-09-03T06:23:46.776 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a96f0d74ce0f640-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.777 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:46.777 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:46.778 DEBUG:boto:path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.778 DEBUG:boto:auth_path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.779 DEBUG:boto:Method: HEAD 2022-09-03T06:23:46.779 DEBUG:boto:Path: /dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.780 DEBUG:boto:Data: 2022-09-03T06:23:46.780 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.781 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.781 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.782 DEBUG:boto:Params: {} 2022-09-03T06:23:46.782 DEBUG:boto:Token: None 2022-09-03T06:23:46.783 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.783 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:U/mlSTcDErjuMGFeIfLWQ75LPJE= 2022-09-03T06:23:46.784 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:U/mlSTcDErjuMGFeIfLWQ75LPJE=', 'Content-Length': '0'} 2022-09-03T06:23:46.786 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005324e7246af3d9ee-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.787 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:46.788 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:46.788 DEBUG:boto:path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.789 DEBUG:boto:auth_path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.789 DEBUG:boto:Method: HEAD 2022-09-03T06:23:46.790 DEBUG:boto:Path: /dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.791 DEBUG:boto:Data: 2022-09-03T06:23:46.792 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.792 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.793 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.793 DEBUG:boto:Params: {} 2022-09-03T06:23:46.794 DEBUG:boto:Token: None 2022-09-03T06:23:46.794 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.795 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:U/mlSTcDErjuMGFeIfLWQ75LPJE= 2022-09-03T06:23:46.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:U/mlSTcDErjuMGFeIfLWQ75LPJE=', 'Content-Length': '0'} 2022-09-03T06:23:46.798 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004ac47c05a843fd60-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.799 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:23:46.799 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:46.800 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:46.801 DEBUG:boto:path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.801 DEBUG:boto:auth_path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.802 DEBUG:boto:Method: GET 2022-09-03T06:23:46.802 DEBUG:boto:Path: /dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.803 DEBUG:boto:Data: 2022-09-03T06:23:46.803 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.804 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.804 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.805 DEBUG:boto:Params: {} 2022-09-03T06:23:46.806 DEBUG:boto:Token: None 2022-09-03T06:23:46.806 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.807 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vbu9y4+832ejNw9VxHeXbD6FSII= 2022-09-03T06:23:46.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vbu9y4+832ejNw9VxHeXbD6FSII=', 'Content-Length': '0'} 2022-09-03T06:23:46.810 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000544f928696fc8b74-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.854 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:46.855 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:46.856 DEBUG:boto:path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.857 DEBUG:boto:auth_path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.858 DEBUG:boto:Method: GET 2022-09-03T06:23:46.859 DEBUG:boto:Path: /dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.860 DEBUG:boto:Data: 2022-09-03T06:23:46.860 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.861 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.862 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.863 DEBUG:boto:Params: {} 2022-09-03T06:23:46.864 DEBUG:boto:Token: None 2022-09-03T06:23:46.866 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/h?versionId=null 2022-09-03T06:23:46.867 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vbu9y4+832ejNw9VxHeXbD6FSII= 2022-09-03T06:23:46.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vbu9y4+832ejNw9VxHeXbD6FSII=', 'Content-Length': '0'} 2022-09-03T06:23:46.869 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bbad31c5608b4149-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.871 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:46.872 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:46.872 DEBUG:boto:path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:46.873 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:46.873 DEBUG:boto:Method: GET 2022-09-03T06:23:46.874 DEBUG:boto:Path: /dsgvcy-72/?prefix=h 2022-09-03T06:23:46.874 DEBUG:boto:Data: 2022-09-03T06:23:46.875 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.875 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.876 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.876 DEBUG:boto:Params: {} 2022-09-03T06:23:46.877 DEBUG:boto:Token: None 2022-09-03T06:23:46.877 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/ 2022-09-03T06:23:46.878 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA= 2022-09-03T06:23:46.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA=', 'Content-Length': '0'} 2022-09-03T06:23:46.881 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000076c956385d5397eb-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.882 DEBUG:boto:b'dsgvcy-72h1000falseh2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:46.883 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:46.883 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:46.884 DEBUG:boto:path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:46.884 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:46.885 DEBUG:boto:Method: GET 2022-09-03T06:23:46.885 DEBUG:boto:Path: /dsgvcy-72/?prefix=h 2022-09-03T06:23:46.886 DEBUG:boto:Data: 2022-09-03T06:23:46.886 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.887 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.887 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.888 DEBUG:boto:Params: {} 2022-09-03T06:23:46.888 DEBUG:boto:Token: None 2022-09-03T06:23:46.889 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/ 2022-09-03T06:23:46.890 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA= 2022-09-03T06:23:46.890 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA=', 'Content-Length': '0'} 2022-09-03T06:23:46.892 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f472ec6078318245-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.893 DEBUG:boto:b'dsgvcy-72h1000falseh2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:46.894 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:46.894 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:46.895 DEBUG:boto:Method: HEAD 2022-09-03T06:23:46.895 DEBUG:boto:Path: /dsgvcy-72/h 2022-09-03T06:23:46.896 DEBUG:boto:Data: 2022-09-03T06:23:46.897 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.897 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.898 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.898 DEBUG:boto:Params: {} 2022-09-03T06:23:46.899 DEBUG:boto:Token: None 2022-09-03T06:23:46.899 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/h 2022-09-03T06:23:46.900 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oL79B9Mq7kMOgZIPpzqHTieoDCQ= 2022-09-03T06:23:46.901 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oL79B9Mq7kMOgZIPpzqHTieoDCQ=', 'Content-Length': '0'} 2022-09-03T06:23:46.903 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004cfa4dc75628e7b9-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.904 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:46.904 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:46.905 DEBUG:boto:Method: HEAD 2022-09-03T06:23:46.905 DEBUG:boto:Path: /dsgvcy-72/h 2022-09-03T06:23:46.906 DEBUG:boto:Data: 2022-09-03T06:23:46.907 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.907 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.908 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.908 DEBUG:boto:Params: {} 2022-09-03T06:23:46.909 DEBUG:boto:Token: None 2022-09-03T06:23:46.909 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/h 2022-09-03T06:23:46.910 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oL79B9Mq7kMOgZIPpzqHTieoDCQ= 2022-09-03T06:23:46.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oL79B9Mq7kMOgZIPpzqHTieoDCQ=', 'Content-Length': '0'} 2022-09-03T06:23:46.913 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001f75a0444903c338-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.914 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:23:46.915 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:46.916 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:46.916 DEBUG:boto:Method: GET 2022-09-03T06:23:46.917 DEBUG:boto:Path: /dsgvcy-72/h 2022-09-03T06:23:46.917 DEBUG:boto:Data: 2022-09-03T06:23:46.918 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.919 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.919 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.920 DEBUG:boto:Params: {} 2022-09-03T06:23:46.920 DEBUG:boto:Token: None 2022-09-03T06:23:46.921 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/h 2022-09-03T06:23:46.922 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L+QTIQ38RV8U2njjBpTmXYExa1g= 2022-09-03T06:23:46.922 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L+QTIQ38RV8U2njjBpTmXYExa1g=', 'Content-Length': '0'} 2022-09-03T06:23:46.925 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a89a6b40c187bc46-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.926 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:46.927 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:46.927 DEBUG:boto:Method: GET 2022-09-03T06:23:46.928 DEBUG:boto:Path: /dsgvcy-72/h 2022-09-03T06:23:46.928 DEBUG:boto:Data: 2022-09-03T06:23:46.929 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.929 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.930 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.931 DEBUG:boto:Params: {} 2022-09-03T06:23:46.931 DEBUG:boto:Token: None 2022-09-03T06:23:46.932 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/h 2022-09-03T06:23:46.933 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L+QTIQ38RV8U2njjBpTmXYExa1g= 2022-09-03T06:23:46.933 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L+QTIQ38RV8U2njjBpTmXYExa1g=', 'Content-Length': '0'} 2022-09-03T06:23:46.935 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000081313d368a90e550-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.936 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:46.937 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:46.937 DEBUG:boto:path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:46.938 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:46.938 DEBUG:boto:Method: GET 2022-09-03T06:23:46.939 DEBUG:boto:Path: /dsgvcy-72/?prefix=h 2022-09-03T06:23:46.940 DEBUG:boto:Data: 2022-09-03T06:23:46.940 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.941 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:46.942 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.942 DEBUG:boto:Params: {} 2022-09-03T06:23:46.943 DEBUG:boto:Token: None 2022-09-03T06:23:46.944 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/ 2022-09-03T06:23:46.944 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA= 2022-09-03T06:23:46.945 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA=', 'Content-Length': '0'} 2022-09-03T06:23:46.947 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003f5ec36acf65023a-006312f2f2-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.948 DEBUG:boto:b'dsgvcy-72h1000falseh2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:46.949 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:46.949 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:46.950 DEBUG:boto:path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:46.951 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:46.951 DEBUG:boto:Method: GET 2022-09-03T06:23:46.952 DEBUG:boto:Path: /dsgvcy-72/?prefix=h 2022-09-03T06:23:46.952 DEBUG:boto:Data: 2022-09-03T06:23:46.953 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.953 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.954 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.954 DEBUG:boto:Params: {} 2022-09-03T06:23:46.955 DEBUG:boto:Token: None 2022-09-03T06:23:46.956 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/ 2022-09-03T06:23:46.956 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA= 2022-09-03T06:23:46.957 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RBJCuqMFlRAt9vw61Rwu8AqHByA=', 'Content-Length': '0'} 2022-09-03T06:23:46.960 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000069229e791b1b6e0f-006312f2f2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.960 DEBUG:boto:b'dsgvcy-72h1000falseh2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:46.961 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-72 zones={test-zone2, test-zone1} 2022-09-03T06:23:46.962 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-72 zones={test-zone3, test-zone1} 2022-09-03T06:23:46.962 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:46.963 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:46.963 DEBUG:boto:Method: HEAD 2022-09-03T06:23:46.964 DEBUG:boto:Path: /dsgvcy-72/ 2022-09-03T06:23:46.964 DEBUG:boto:Data: 2022-09-03T06:23:46.965 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.966 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:46.966 DEBUG:boto:Port: 8001 2022-09-03T06:23:46.967 DEBUG:boto:Params: {} 2022-09-03T06:23:46.967 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T06:23:46.968 DEBUG:boto:Token: None 2022-09-03T06:23:46.969 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/ 2022-09-03T06:23:46.969 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dp+FAaPsNUnfJeufSADC7gWvMCs= 2022-09-03T06:23:46.970 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dp+FAaPsNUnfJeufSADC7gWvMCs=', 'Content-Length': '0'} 2022-09-03T06:23:46.973 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000760a289a51b88d7b-006312f2f2-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.974 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:46.975 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:46.975 DEBUG:boto:Method: HEAD 2022-09-03T06:23:46.976 DEBUG:boto:Path: /dsgvcy-72/ 2022-09-03T06:23:46.976 DEBUG:boto:Data: 2022-09-03T06:23:46.977 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.977 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:46.978 DEBUG:boto:Port: 8000 2022-09-03T06:23:46.978 DEBUG:boto:Params: {} 2022-09-03T06:23:46.979 DEBUG:boto:Token: None 2022-09-03T06:23:46.980 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/ 2022-09-03T06:23:46.980 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dp+FAaPsNUnfJeufSADC7gWvMCs= 2022-09-03T06:23:46.981 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dp+FAaPsNUnfJeufSADC7gWvMCs=', 'Content-Length': '0'} 2022-09-03T06:23:46.983 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ab588846d5567678-006312f2f2-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.984 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:23:46.984 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:46.985 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:46.985 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:46.986 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:46.987 DEBUG:boto:Method: GET 2022-09-03T06:23:46.987 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:46.988 DEBUG:boto:Data: 2022-09-03T06:23:46.988 DEBUG:boto:Headers: {} 2022-09-03T06:23:46.989 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:46.989 DEBUG:boto:Port: 8001 2022-09-03T06:23:46.990 DEBUG:boto:Params: {} 2022-09-03T06:23:46.990 DEBUG:boto:Token: None 2022-09-03T06:23:46.991 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:46 GMT /dsgvcy-72/?versions 2022-09-03T06:23:46.992 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2vJoscfY/ki3D4raAfzKXjLwVk8= 2022-09-03T06:23:46.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2vJoscfY/ki3D4raAfzKXjLwVk8=', 'Content-Length': '0'} 2022-09-03T06:23:46.995 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000620a3cba140a6892-006312f2f2-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:46.996 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:46.997 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:23:46.998 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:23:46.998 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:23:46.999 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:23:46.999 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:23:47.000 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:23:47.000 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:23:47.001 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:23:47.001 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:23:47.002 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:47.002 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:47.003 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:47.003 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:47.004 DEBUG:boto:Method: GET 2022-09-03T06:23:47.004 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:47.005 DEBUG:boto:Data: 2022-09-03T06:23:47.006 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.006 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.007 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.007 DEBUG:boto:Params: {} 2022-09-03T06:23:47.008 DEBUG:boto:Token: None 2022-09-03T06:23:47.008 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/?versions 2022-09-03T06:23:47.009 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:36Jj7QPABYrgL9UQuxNS9vx4L6w= 2022-09-03T06:23:47.010 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:36Jj7QPABYrgL9UQuxNS9vx4L6w=', 'Content-Length': '0'} 2022-09-03T06:23:47.012 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a6519904d4e64096-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.013 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:47.014 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:23:47.014 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:23:47.015 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:23:47.015 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:23:47.016 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:23:47.016 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:23:47.017 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:23:47.017 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:23:47.018 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:47.018 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:47.019 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:47.020 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:47.020 DEBUG:boto:Method: GET 2022-09-03T06:23:47.021 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:47.021 DEBUG:boto:Data: 2022-09-03T06:23:47.022 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.022 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.023 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.023 DEBUG:boto:Params: {} 2022-09-03T06:23:47.024 DEBUG:boto:Token: None 2022-09-03T06:23:47.025 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/?versions 2022-09-03T06:23:47.025 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:36Jj7QPABYrgL9UQuxNS9vx4L6w= 2022-09-03T06:23:47.026 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:36Jj7QPABYrgL9UQuxNS9vx4L6w=', 'Content-Length': '0'} 2022-09-03T06:23:47.029 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000022829711cd0e4640-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.030 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:47.031 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:47.031 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:47.032 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:47.033 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:47.033 DEBUG:boto:Method: GET 2022-09-03T06:23:47.034 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:47.034 DEBUG:boto:Data: 2022-09-03T06:23:47.035 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.035 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.036 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.036 DEBUG:boto:Params: {} 2022-09-03T06:23:47.037 DEBUG:boto:Token: None 2022-09-03T06:23:47.038 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/?versions 2022-09-03T06:23:47.038 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:36Jj7QPABYrgL9UQuxNS9vx4L6w= 2022-09-03T06:23:47.039 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:36Jj7QPABYrgL9UQuxNS9vx4L6w=', 'Content-Length': '0'} 2022-09-03T06:23:47.041 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000002bff3798513d217-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.042 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:47.043 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:23:47.044 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:47.044 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:47.045 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.045 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.046 DEBUG:boto:Method: GET 2022-09-03T06:23:47.047 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.047 DEBUG:boto:Data: 2022-09-03T06:23:47.048 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.048 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.049 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.049 DEBUG:boto:Params: {} 2022-09-03T06:23:47.050 DEBUG:boto:Token: None 2022-09-03T06:23:47.051 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.051 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0lllsN+IzWIN95SkUiNFNrqESrE= 2022-09-03T06:23:47.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0lllsN+IzWIN95SkUiNFNrqESrE=', 'Content-Length': '0'} 2022-09-03T06:23:47.054 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b637a24524259cb-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.055 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:47.056 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:47.056 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.057 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.057 DEBUG:boto:Method: GET 2022-09-03T06:23:47.058 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.059 DEBUG:boto:Data: 2022-09-03T06:23:47.059 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.060 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.061 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.061 DEBUG:boto:Params: {} 2022-09-03T06:23:47.062 DEBUG:boto:Token: None 2022-09-03T06:23:47.062 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.063 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0lllsN+IzWIN95SkUiNFNrqESrE= 2022-09-03T06:23:47.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0lllsN+IzWIN95SkUiNFNrqESrE=', 'Content-Length': '0'} 2022-09-03T06:23:47.066 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004b29def4a732e3de-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.110 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:47.111 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:47.112 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.113 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.114 DEBUG:boto:Method: HEAD 2022-09-03T06:23:47.115 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.116 DEBUG:boto:Data: 2022-09-03T06:23:47.117 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.119 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.120 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.120 DEBUG:boto:Params: {} 2022-09-03T06:23:47.121 DEBUG:boto:Token: None 2022-09-03T06:23:47.122 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.123 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BPnCAyZr1jmEWgTAdns20BL9m68= 2022-09-03T06:23:47.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BPnCAyZr1jmEWgTAdns20BL9m68=', 'Content-Length': '0'} 2022-09-03T06:23:47.130 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000006b60dfb99c805bb-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.131 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:47.131 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:47.132 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.132 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.133 DEBUG:boto:Method: HEAD 2022-09-03T06:23:47.133 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.134 DEBUG:boto:Data: 2022-09-03T06:23:47.135 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.135 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.136 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.136 DEBUG:boto:Params: {} 2022-09-03T06:23:47.137 DEBUG:boto:Token: None 2022-09-03T06:23:47.138 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.139 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BPnCAyZr1jmEWgTAdns20BL9m68= 2022-09-03T06:23:47.139 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BPnCAyZr1jmEWgTAdns20BL9m68=', 'Content-Length': '0'} 2022-09-03T06:23:47.141 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ac6e1256ab364e8f-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.142 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:23:47.143 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:47.143 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:47.144 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.144 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.145 DEBUG:boto:Method: GET 2022-09-03T06:23:47.146 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.146 DEBUG:boto:Data: 2022-09-03T06:23:47.147 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.147 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.148 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.148 DEBUG:boto:Params: {} 2022-09-03T06:23:47.149 DEBUG:boto:Token: None 2022-09-03T06:23:47.150 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.151 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0lllsN+IzWIN95SkUiNFNrqESrE= 2022-09-03T06:23:47.153 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0lllsN+IzWIN95SkUiNFNrqESrE=', 'Content-Length': '0'} 2022-09-03T06:23:47.155 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d28779642e4aab57-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.157 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:47.158 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:47.159 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.160 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.160 DEBUG:boto:Method: GET 2022-09-03T06:23:47.161 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.162 DEBUG:boto:Data: 2022-09-03T06:23:47.162 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.163 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.163 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.164 DEBUG:boto:Params: {} 2022-09-03T06:23:47.164 DEBUG:boto:Token: None 2022-09-03T06:23:47.165 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:23:47.166 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0lllsN+IzWIN95SkUiNFNrqESrE= 2022-09-03T06:23:47.166 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0lllsN+IzWIN95SkUiNFNrqESrE=', 'Content-Length': '0'} 2022-09-03T06:23:47.169 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000098f4e70189047d4-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.170 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:47.170 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:47.171 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:47.171 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:47.172 DEBUG:boto:Method: GET 2022-09-03T06:23:47.173 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:23:47.173 DEBUG:boto:Data: 2022-09-03T06:23:47.174 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.174 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.175 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.175 DEBUG:boto:Params: {} 2022-09-03T06:23:47.176 DEBUG:boto:Token: None 2022-09-03T06:23:47.177 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/ 2022-09-03T06:23:47.177 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA= 2022-09-03T06:23:47.178 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA=', 'Content-Length': '0'} 2022-09-03T06:23:47.181 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000657c1af2d7ab91ef-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.182 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:47.183 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:47.184 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:47.184 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:47.185 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:47.185 DEBUG:boto:Method: GET 2022-09-03T06:23:47.186 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:23:47.186 DEBUG:boto:Data: 2022-09-03T06:23:47.187 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.187 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.188 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.188 DEBUG:boto:Params: {} 2022-09-03T06:23:47.189 DEBUG:boto:Token: None 2022-09-03T06:23:47.189 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/ 2022-09-03T06:23:47.190 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA= 2022-09-03T06:23:47.191 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA=', 'Content-Length': '0'} 2022-09-03T06:23:47.193 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000de80541a7c2553e0-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.194 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:47.195 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:47.195 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:47.196 DEBUG:boto:Method: HEAD 2022-09-03T06:23:47.197 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:23:47.197 DEBUG:boto:Data: 2022-09-03T06:23:47.198 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.199 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.200 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.201 DEBUG:boto:Params: {} 2022-09-03T06:23:47.202 DEBUG:boto:Token: None 2022-09-03T06:23:47.203 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/a 2022-09-03T06:23:47.204 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HD1vel9R0q35MQ7dQ5Ij72W4f/k= 2022-09-03T06:23:47.205 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HD1vel9R0q35MQ7dQ5Ij72W4f/k=', 'Content-Length': '0'} 2022-09-03T06:23:47.207 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ed39774c7aac6714-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.207 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:47.208 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:47.209 DEBUG:boto:Method: HEAD 2022-09-03T06:23:47.210 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:23:47.211 DEBUG:boto:Data: 2022-09-03T06:23:47.212 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.213 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.213 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.214 DEBUG:boto:Params: {} 2022-09-03T06:23:47.215 DEBUG:boto:Token: None 2022-09-03T06:23:47.216 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/a 2022-09-03T06:23:47.217 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HD1vel9R0q35MQ7dQ5Ij72W4f/k= 2022-09-03T06:23:47.218 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HD1vel9R0q35MQ7dQ5Ij72W4f/k=', 'Content-Length': '0'} 2022-09-03T06:23:47.220 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002fd9fbc1e0d7c075-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.221 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:23:47.222 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:47.222 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:47.223 DEBUG:boto:Method: GET 2022-09-03T06:23:47.223 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:23:47.224 DEBUG:boto:Data: 2022-09-03T06:23:47.224 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.225 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.225 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.226 DEBUG:boto:Params: {} 2022-09-03T06:23:47.227 DEBUG:boto:Token: None 2022-09-03T06:23:47.227 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/a 2022-09-03T06:23:47.228 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ETiLTsGqF62sSWVt5GJtFc76FcA= 2022-09-03T06:23:47.228 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ETiLTsGqF62sSWVt5GJtFc76FcA=', 'Content-Length': '0'} 2022-09-03T06:23:47.230 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000df21a87a38f4d846-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.278 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:47.279 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:47.280 DEBUG:boto:Method: GET 2022-09-03T06:23:47.281 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:23:47.282 DEBUG:boto:Data: 2022-09-03T06:23:47.283 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.284 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.285 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.286 DEBUG:boto:Params: {} 2022-09-03T06:23:47.287 DEBUG:boto:Token: None 2022-09-03T06:23:47.288 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/a 2022-09-03T06:23:47.289 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ETiLTsGqF62sSWVt5GJtFc76FcA= 2022-09-03T06:23:47.290 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ETiLTsGqF62sSWVt5GJtFc76FcA=', 'Content-Length': '0'} 2022-09-03T06:23:47.293 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001f2d2b59203db600-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.294 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:47.294 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:47.295 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:47.295 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:47.296 DEBUG:boto:Method: GET 2022-09-03T06:23:47.297 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:23:47.297 DEBUG:boto:Data: 2022-09-03T06:23:47.298 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.299 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.299 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.300 DEBUG:boto:Params: {} 2022-09-03T06:23:47.301 DEBUG:boto:Token: None 2022-09-03T06:23:47.301 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/ 2022-09-03T06:23:47.302 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA= 2022-09-03T06:23:47.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA=', 'Content-Length': '0'} 2022-09-03T06:23:47.305 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007462562f7688f7da-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.306 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:47.307 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:47.308 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:47.308 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:47.309 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:47.309 DEBUG:boto:Method: GET 2022-09-03T06:23:47.310 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:23:47.311 DEBUG:boto:Data: 2022-09-03T06:23:47.311 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.312 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.312 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.313 DEBUG:boto:Params: {} 2022-09-03T06:23:47.314 DEBUG:boto:Token: None 2022-09-03T06:23:47.314 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/ 2022-09-03T06:23:47.315 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA= 2022-09-03T06:23:47.315 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA=', 'Content-Length': '0'} 2022-09-03T06:23:47.318 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000605f7078322595c8-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.319 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:47.320 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:23:47.320 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:47.321 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:47.321 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.322 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.323 DEBUG:boto:Method: GET 2022-09-03T06:23:47.323 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.324 DEBUG:boto:Data: 2022-09-03T06:23:47.324 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.325 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.325 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.326 DEBUG:boto:Params: {} 2022-09-03T06:23:47.327 DEBUG:boto:Token: None 2022-09-03T06:23:47.328 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.329 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ccs9pLOHn62fuHAzbpRHKjKrzKg= 2022-09-03T06:23:47.330 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ccs9pLOHn62fuHAzbpRHKjKrzKg=', 'Content-Length': '0'} 2022-09-03T06:23:47.332 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bdc44688cad74e71-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.333 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:47.333 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:47.334 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.335 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.335 DEBUG:boto:Method: GET 2022-09-03T06:23:47.336 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.336 DEBUG:boto:Data: 2022-09-03T06:23:47.337 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.338 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.338 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.339 DEBUG:boto:Params: {} 2022-09-03T06:23:47.339 DEBUG:boto:Token: None 2022-09-03T06:23:47.340 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.341 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ccs9pLOHn62fuHAzbpRHKjKrzKg= 2022-09-03T06:23:47.342 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ccs9pLOHn62fuHAzbpRHKjKrzKg=', 'Content-Length': '0'} 2022-09-03T06:23:47.344 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008fcaf0ff69fec764-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.345 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:47.345 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:47.346 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.346 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.347 DEBUG:boto:Method: HEAD 2022-09-03T06:23:47.347 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.348 DEBUG:boto:Data: 2022-09-03T06:23:47.348 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.349 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.349 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.350 DEBUG:boto:Params: {} 2022-09-03T06:23:47.350 DEBUG:boto:Token: None 2022-09-03T06:23:47.351 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.352 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xg0OZE3q7S5YJVCYI5PfDatewhc= 2022-09-03T06:23:47.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xg0OZE3q7S5YJVCYI5PfDatewhc=', 'Content-Length': '0'} 2022-09-03T06:23:47.354 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001bc812b8e96a729f-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.355 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:47.356 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:47.356 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.357 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.357 DEBUG:boto:Method: HEAD 2022-09-03T06:23:47.358 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.358 DEBUG:boto:Data: 2022-09-03T06:23:47.359 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.359 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.360 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.360 DEBUG:boto:Params: {} 2022-09-03T06:23:47.361 DEBUG:boto:Token: None 2022-09-03T06:23:47.361 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.362 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xg0OZE3q7S5YJVCYI5PfDatewhc= 2022-09-03T06:23:47.363 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xg0OZE3q7S5YJVCYI5PfDatewhc=', 'Content-Length': '0'} 2022-09-03T06:23:47.365 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000abb0aed60a8cc928-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.365 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:23:47.366 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:47.366 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:47.367 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.367 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.368 DEBUG:boto:Method: GET 2022-09-03T06:23:47.369 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.369 DEBUG:boto:Data: 2022-09-03T06:23:47.370 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.370 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.371 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.371 DEBUG:boto:Params: {} 2022-09-03T06:23:47.372 DEBUG:boto:Token: None 2022-09-03T06:23:47.372 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.373 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ccs9pLOHn62fuHAzbpRHKjKrzKg= 2022-09-03T06:23:47.374 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ccs9pLOHn62fuHAzbpRHKjKrzKg=', 'Content-Length': '0'} 2022-09-03T06:23:47.376 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000442e818a01bc5c6c-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.377 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:47.377 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:47.378 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.378 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.379 DEBUG:boto:Method: GET 2022-09-03T06:23:47.379 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.380 DEBUG:boto:Data: 2022-09-03T06:23:47.380 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.381 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.382 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.382 DEBUG:boto:Params: {} 2022-09-03T06:23:47.383 DEBUG:boto:Token: None 2022-09-03T06:23:47.383 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:23:47.384 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ccs9pLOHn62fuHAzbpRHKjKrzKg= 2022-09-03T06:23:47.385 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ccs9pLOHn62fuHAzbpRHKjKrzKg=', 'Content-Length': '0'} 2022-09-03T06:23:47.387 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000082f324904602e25-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.387 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:47.388 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:47.389 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:47.389 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:47.390 DEBUG:boto:Method: GET 2022-09-03T06:23:47.390 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:23:47.391 DEBUG:boto:Data: 2022-09-03T06:23:47.391 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.392 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.392 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.393 DEBUG:boto:Params: {} 2022-09-03T06:23:47.393 DEBUG:boto:Token: None 2022-09-03T06:23:47.394 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/ 2022-09-03T06:23:47.395 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA= 2022-09-03T06:23:47.395 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA=', 'Content-Length': '0'} 2022-09-03T06:23:47.398 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a21407a219b4ce44-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.399 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:47.400 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:47.400 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:47.401 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:47.401 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:47.402 DEBUG:boto:Method: GET 2022-09-03T06:23:47.403 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:23:47.403 DEBUG:boto:Data: 2022-09-03T06:23:47.404 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.404 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.405 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.405 DEBUG:boto:Params: {} 2022-09-03T06:23:47.406 DEBUG:boto:Token: None 2022-09-03T06:23:47.406 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/ 2022-09-03T06:23:47.407 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA= 2022-09-03T06:23:47.408 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA=', 'Content-Length': '0'} 2022-09-03T06:23:47.410 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003cad231222ff5c62-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.454 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:47.456 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:47.457 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:47.458 DEBUG:boto:Method: HEAD 2022-09-03T06:23:47.459 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:23:47.459 DEBUG:boto:Data: 2022-09-03T06:23:47.460 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.461 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.462 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.463 DEBUG:boto:Params: {} 2022-09-03T06:23:47.464 DEBUG:boto:Token: None 2022-09-03T06:23:47.465 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/b 2022-09-03T06:23:47.466 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VeoQz1G2Xchu6Bq4+niiETh7N/U= 2022-09-03T06:23:47.467 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VeoQz1G2Xchu6Bq4+niiETh7N/U=', 'Content-Length': '0'} 2022-09-03T06:23:47.471 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000090add9b2c8aaaba-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.472 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:47.473 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:47.474 DEBUG:boto:Method: HEAD 2022-09-03T06:23:47.475 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:23:47.476 DEBUG:boto:Data: 2022-09-03T06:23:47.477 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.478 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.478 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.479 DEBUG:boto:Params: {} 2022-09-03T06:23:47.480 DEBUG:boto:Token: None 2022-09-03T06:23:47.481 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/b 2022-09-03T06:23:47.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VeoQz1G2Xchu6Bq4+niiETh7N/U= 2022-09-03T06:23:47.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VeoQz1G2Xchu6Bq4+niiETh7N/U=', 'Content-Length': '0'} 2022-09-03T06:23:47.487 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b2ebbc02597dfce-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.488 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:23:47.489 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:47.490 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:47.491 DEBUG:boto:Method: GET 2022-09-03T06:23:47.492 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:23:47.493 DEBUG:boto:Data: 2022-09-03T06:23:47.494 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.495 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.496 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.497 DEBUG:boto:Params: {} 2022-09-03T06:23:47.498 DEBUG:boto:Token: None 2022-09-03T06:23:47.499 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/b 2022-09-03T06:23:47.500 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/eSx7h6GBHSYATn2mysUapRPD2E= 2022-09-03T06:23:47.501 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/eSx7h6GBHSYATn2mysUapRPD2E=', 'Content-Length': '0'} 2022-09-03T06:23:47.504 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000859867201ad583d3-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.550 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:47.551 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:47.552 DEBUG:boto:Method: GET 2022-09-03T06:23:47.552 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:23:47.553 DEBUG:boto:Data: 2022-09-03T06:23:47.554 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.554 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.555 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.555 DEBUG:boto:Params: {} 2022-09-03T06:23:47.556 DEBUG:boto:Token: None 2022-09-03T06:23:47.557 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/b 2022-09-03T06:23:47.557 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/eSx7h6GBHSYATn2mysUapRPD2E= 2022-09-03T06:23:47.558 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/eSx7h6GBHSYATn2mysUapRPD2E=', 'Content-Length': '0'} 2022-09-03T06:23:47.560 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b413bc1d26f9fd48-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.561 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:47.562 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:47.562 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:47.563 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:47.563 DEBUG:boto:Method: GET 2022-09-03T06:23:47.564 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:23:47.564 DEBUG:boto:Data: 2022-09-03T06:23:47.565 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.565 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.566 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.567 DEBUG:boto:Params: {} 2022-09-03T06:23:47.567 DEBUG:boto:Token: None 2022-09-03T06:23:47.568 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/ 2022-09-03T06:23:47.568 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA= 2022-09-03T06:23:47.569 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA=', 'Content-Length': '0'} 2022-09-03T06:23:47.572 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c740bf013a781628-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.573 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:47.573 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:47.574 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:47.575 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:47.575 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:47.576 DEBUG:boto:Method: GET 2022-09-03T06:23:47.576 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:23:47.577 DEBUG:boto:Data: 2022-09-03T06:23:47.577 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.578 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.579 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.579 DEBUG:boto:Params: {} 2022-09-03T06:23:47.580 DEBUG:boto:Token: None 2022-09-03T06:23:47.581 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/ 2022-09-03T06:23:47.581 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA= 2022-09-03T06:23:47.582 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA=', 'Content-Length': '0'} 2022-09-03T06:23:47.584 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e3574d2d63be1155-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.585 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:47.585 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:23:47.586 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:47.586 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:47.587 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.588 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.588 DEBUG:boto:Method: GET 2022-09-03T06:23:47.589 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.589 DEBUG:boto:Data: 2022-09-03T06:23:47.590 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.590 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.591 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.591 DEBUG:boto:Params: {} 2022-09-03T06:23:47.592 DEBUG:boto:Token: None 2022-09-03T06:23:47.592 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.593 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6dH+c7W+JRH1Q32E2ySuiTn3uew= 2022-09-03T06:23:47.594 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6dH+c7W+JRH1Q32E2ySuiTn3uew=', 'Content-Length': '0'} 2022-09-03T06:23:47.596 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000024e2bafd2f3a2d43-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.597 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:47.597 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:47.598 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.598 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.599 DEBUG:boto:Method: GET 2022-09-03T06:23:47.600 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.600 DEBUG:boto:Data: 2022-09-03T06:23:47.601 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.601 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.602 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.602 DEBUG:boto:Params: {} 2022-09-03T06:23:47.603 DEBUG:boto:Token: None 2022-09-03T06:23:47.603 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.604 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6dH+c7W+JRH1Q32E2ySuiTn3uew= 2022-09-03T06:23:47.605 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6dH+c7W+JRH1Q32E2ySuiTn3uew=', 'Content-Length': '0'} 2022-09-03T06:23:47.607 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dea8954f21264c5c-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.607 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:47.608 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:47.608 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.609 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.609 DEBUG:boto:Method: HEAD 2022-09-03T06:23:47.610 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.610 DEBUG:boto:Data: 2022-09-03T06:23:47.611 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.611 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.612 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.612 DEBUG:boto:Params: {} 2022-09-03T06:23:47.613 DEBUG:boto:Token: None 2022-09-03T06:23:47.614 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.614 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1ZKAtg5d4klrIsohFHA15PAZUiM= 2022-09-03T06:23:47.615 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1ZKAtg5d4klrIsohFHA15PAZUiM=', 'Content-Length': '0'} 2022-09-03T06:23:47.617 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006f82069dc4fa7e4b-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.618 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:47.618 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:47.619 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.619 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.620 DEBUG:boto:Method: HEAD 2022-09-03T06:23:47.620 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.621 DEBUG:boto:Data: 2022-09-03T06:23:47.621 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.622 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.622 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.623 DEBUG:boto:Params: {} 2022-09-03T06:23:47.623 DEBUG:boto:Token: None 2022-09-03T06:23:47.624 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.625 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1ZKAtg5d4klrIsohFHA15PAZUiM= 2022-09-03T06:23:47.625 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1ZKAtg5d4klrIsohFHA15PAZUiM=', 'Content-Length': '0'} 2022-09-03T06:23:47.627 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002e6a1fa9b8e19666-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.628 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:23:47.628 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:47.629 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:47.630 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.631 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.632 DEBUG:boto:Method: GET 2022-09-03T06:23:47.632 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.633 DEBUG:boto:Data: 2022-09-03T06:23:47.633 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.634 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.634 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.635 DEBUG:boto:Params: {} 2022-09-03T06:23:47.636 DEBUG:boto:Token: None 2022-09-03T06:23:47.636 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.637 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6dH+c7W+JRH1Q32E2ySuiTn3uew= 2022-09-03T06:23:47.638 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6dH+c7W+JRH1Q32E2ySuiTn3uew=', 'Content-Length': '0'} 2022-09-03T06:23:47.640 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000282f680c5ff325a1-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.641 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:47.641 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:47.642 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.643 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.643 DEBUG:boto:Method: GET 2022-09-03T06:23:47.644 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.644 DEBUG:boto:Data: 2022-09-03T06:23:47.645 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.645 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.646 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.646 DEBUG:boto:Params: {} 2022-09-03T06:23:47.647 DEBUG:boto:Token: None 2022-09-03T06:23:47.648 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:23:47.649 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6dH+c7W+JRH1Q32E2ySuiTn3uew= 2022-09-03T06:23:47.650 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6dH+c7W+JRH1Q32E2ySuiTn3uew=', 'Content-Length': '0'} 2022-09-03T06:23:47.653 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009105a1084d0a4641-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.654 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:47.655 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:47.655 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:47.656 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:47.657 DEBUG:boto:Method: GET 2022-09-03T06:23:47.657 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:23:47.658 DEBUG:boto:Data: 2022-09-03T06:23:47.659 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.659 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.660 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.660 DEBUG:boto:Params: {} 2022-09-03T06:23:47.661 DEBUG:boto:Token: None 2022-09-03T06:23:47.661 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/ 2022-09-03T06:23:47.662 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA= 2022-09-03T06:23:47.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA=', 'Content-Length': '0'} 2022-09-03T06:23:47.665 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000078fbac6a41fc1aa0-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.666 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:47.667 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:47.668 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:47.668 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:47.669 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:47.669 DEBUG:boto:Method: GET 2022-09-03T06:23:47.670 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:23:47.670 DEBUG:boto:Data: 2022-09-03T06:23:47.671 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.671 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.672 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.672 DEBUG:boto:Params: {} 2022-09-03T06:23:47.673 DEBUG:boto:Token: None 2022-09-03T06:23:47.674 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/ 2022-09-03T06:23:47.674 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA= 2022-09-03T06:23:47.675 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA=', 'Content-Length': '0'} 2022-09-03T06:23:47.677 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000833fbc218db3d85a-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.678 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:47.679 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:47.680 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:47.680 DEBUG:boto:Method: HEAD 2022-09-03T06:23:47.681 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:23:47.681 DEBUG:boto:Data: 2022-09-03T06:23:47.682 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.682 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.683 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.683 DEBUG:boto:Params: {} 2022-09-03T06:23:47.684 DEBUG:boto:Token: None 2022-09-03T06:23:47.685 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/c 2022-09-03T06:23:47.686 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ySAa42HM2VKB7JwzS/40jW17G04= 2022-09-03T06:23:47.687 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ySAa42HM2VKB7JwzS/40jW17G04=', 'Content-Length': '0'} 2022-09-03T06:23:47.689 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d802700f300fc10a-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.690 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:47.690 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:47.691 DEBUG:boto:Method: HEAD 2022-09-03T06:23:47.691 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:23:47.692 DEBUG:boto:Data: 2022-09-03T06:23:47.692 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.693 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.693 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.694 DEBUG:boto:Params: {} 2022-09-03T06:23:47.694 DEBUG:boto:Token: None 2022-09-03T06:23:47.695 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/c 2022-09-03T06:23:47.697 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ySAa42HM2VKB7JwzS/40jW17G04= 2022-09-03T06:23:47.698 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ySAa42HM2VKB7JwzS/40jW17G04=', 'Content-Length': '0'} 2022-09-03T06:23:47.700 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000440efc9cd79a6724-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.701 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:23:47.701 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:47.702 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:47.702 DEBUG:boto:Method: GET 2022-09-03T06:23:47.703 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:23:47.704 DEBUG:boto:Data: 2022-09-03T06:23:47.704 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.705 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.705 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.706 DEBUG:boto:Params: {} 2022-09-03T06:23:47.707 DEBUG:boto:Token: None 2022-09-03T06:23:47.707 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/c 2022-09-03T06:23:47.708 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wRR0oGFfC4aE4Ld2VdLwPSFbK/8= 2022-09-03T06:23:47.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wRR0oGFfC4aE4Ld2VdLwPSFbK/8=', 'Content-Length': '0'} 2022-09-03T06:23:47.711 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d077affe81ab8163-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.711 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:47.712 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:47.712 DEBUG:boto:Method: GET 2022-09-03T06:23:47.713 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:23:47.714 DEBUG:boto:Data: 2022-09-03T06:23:47.714 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.715 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.716 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.717 DEBUG:boto:Params: {} 2022-09-03T06:23:47.718 DEBUG:boto:Token: None 2022-09-03T06:23:47.719 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/c 2022-09-03T06:23:47.721 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wRR0oGFfC4aE4Ld2VdLwPSFbK/8= 2022-09-03T06:23:47.721 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wRR0oGFfC4aE4Ld2VdLwPSFbK/8=', 'Content-Length': '0'} 2022-09-03T06:23:47.724 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000057b77073b9933e1a-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.725 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:47.726 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:47.727 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:47.728 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:47.729 DEBUG:boto:Method: GET 2022-09-03T06:23:47.730 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:23:47.731 DEBUG:boto:Data: 2022-09-03T06:23:47.732 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.733 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.733 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.734 DEBUG:boto:Params: {} 2022-09-03T06:23:47.735 DEBUG:boto:Token: None 2022-09-03T06:23:47.736 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/ 2022-09-03T06:23:47.737 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA= 2022-09-03T06:23:47.737 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA=', 'Content-Length': '0'} 2022-09-03T06:23:47.740 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fddfef62fa3eb229-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.741 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:47.742 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:47.742 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:47.743 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:47.744 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:47.744 DEBUG:boto:Method: GET 2022-09-03T06:23:47.745 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:23:47.746 DEBUG:boto:Data: 2022-09-03T06:23:47.746 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.747 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.747 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.748 DEBUG:boto:Params: {} 2022-09-03T06:23:47.748 DEBUG:boto:Token: None 2022-09-03T06:23:47.749 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/ 2022-09-03T06:23:47.750 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA= 2022-09-03T06:23:47.750 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA=', 'Content-Length': '0'} 2022-09-03T06:23:47.753 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000094d61e79977f5fd4-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.754 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:47.756 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:23:47.756 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:47.757 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:47.758 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.759 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.760 DEBUG:boto:Method: GET 2022-09-03T06:23:47.761 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.761 DEBUG:boto:Data: 2022-09-03T06:23:47.762 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.762 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.763 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.763 DEBUG:boto:Params: {} 2022-09-03T06:23:47.764 DEBUG:boto:Token: None 2022-09-03T06:23:47.765 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.765 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FtTkscRCxpF48opyojHCm6t7rVE= 2022-09-03T06:23:47.766 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FtTkscRCxpF48opyojHCm6t7rVE=', 'Content-Length': '0'} 2022-09-03T06:23:47.768 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000756bd75c20621547-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.769 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:47.770 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:47.770 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.771 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.771 DEBUG:boto:Method: GET 2022-09-03T06:23:47.772 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.772 DEBUG:boto:Data: 2022-09-03T06:23:47.773 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.773 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.774 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.775 DEBUG:boto:Params: {} 2022-09-03T06:23:47.775 DEBUG:boto:Token: None 2022-09-03T06:23:47.776 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.777 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FtTkscRCxpF48opyojHCm6t7rVE= 2022-09-03T06:23:47.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FtTkscRCxpF48opyojHCm6t7rVE=', 'Content-Length': '0'} 2022-09-03T06:23:47.779 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000047beba5e5681495b-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.780 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:47.781 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:47.781 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.782 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.782 DEBUG:boto:Method: HEAD 2022-09-03T06:23:47.783 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.783 DEBUG:boto:Data: 2022-09-03T06:23:47.784 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.784 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.785 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.785 DEBUG:boto:Params: {} 2022-09-03T06:23:47.786 DEBUG:boto:Token: None 2022-09-03T06:23:47.787 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.787 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JQ5kGfHbQ6Vuc8hpl141/cstc/I= 2022-09-03T06:23:47.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JQ5kGfHbQ6Vuc8hpl141/cstc/I=', 'Content-Length': '0'} 2022-09-03T06:23:47.790 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000533acdf7a6668c02-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.792 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:47.793 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:47.793 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.794 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.795 DEBUG:boto:Method: HEAD 2022-09-03T06:23:47.796 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.797 DEBUG:boto:Data: 2022-09-03T06:23:47.798 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.799 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.800 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.801 DEBUG:boto:Params: {} 2022-09-03T06:23:47.801 DEBUG:boto:Token: None 2022-09-03T06:23:47.802 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.803 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JQ5kGfHbQ6Vuc8hpl141/cstc/I= 2022-09-03T06:23:47.803 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JQ5kGfHbQ6Vuc8hpl141/cstc/I=', 'Content-Length': '0'} 2022-09-03T06:23:47.805 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2634f23e238172b-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.806 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:23:47.807 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:47.807 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:47.808 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.808 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.809 DEBUG:boto:Method: GET 2022-09-03T06:23:47.809 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.810 DEBUG:boto:Data: 2022-09-03T06:23:47.811 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.811 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.812 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.812 DEBUG:boto:Params: {} 2022-09-03T06:23:47.813 DEBUG:boto:Token: None 2022-09-03T06:23:47.813 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.814 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FtTkscRCxpF48opyojHCm6t7rVE= 2022-09-03T06:23:47.815 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FtTkscRCxpF48opyojHCm6t7rVE=', 'Content-Length': '0'} 2022-09-03T06:23:47.817 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000488802349072d1ab-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.862 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:47.863 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:47.863 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.864 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.864 DEBUG:boto:Method: GET 2022-09-03T06:23:47.865 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.866 DEBUG:boto:Data: 2022-09-03T06:23:47.866 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.867 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.867 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.868 DEBUG:boto:Params: {} 2022-09-03T06:23:47.868 DEBUG:boto:Token: None 2022-09-03T06:23:47.869 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:23:47.870 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FtTkscRCxpF48opyojHCm6t7rVE= 2022-09-03T06:23:47.870 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FtTkscRCxpF48opyojHCm6t7rVE=', 'Content-Length': '0'} 2022-09-03T06:23:47.873 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000198e3e492cfdf99c-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.874 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:47.874 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:47.875 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:47.875 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:47.876 DEBUG:boto:Method: GET 2022-09-03T06:23:47.877 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:23:47.877 DEBUG:boto:Data: 2022-09-03T06:23:47.878 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.878 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.879 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.879 DEBUG:boto:Params: {} 2022-09-03T06:23:47.880 DEBUG:boto:Token: None 2022-09-03T06:23:47.880 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/ 2022-09-03T06:23:47.881 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA= 2022-09-03T06:23:47.882 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA=', 'Content-Length': '0'} 2022-09-03T06:23:47.884 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000022dda7a600dd0f0b-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.885 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:47.886 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:47.886 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:47.887 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:47.888 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:47.888 DEBUG:boto:Method: GET 2022-09-03T06:23:47.889 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:23:47.889 DEBUG:boto:Data: 2022-09-03T06:23:47.890 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.890 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.891 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.891 DEBUG:boto:Params: {} 2022-09-03T06:23:47.892 DEBUG:boto:Token: None 2022-09-03T06:23:47.892 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/ 2022-09-03T06:23:47.893 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA= 2022-09-03T06:23:47.894 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA=', 'Content-Length': '0'} 2022-09-03T06:23:47.896 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000087feb34925608523-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.897 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:47.898 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:47.899 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:47.899 DEBUG:boto:Method: HEAD 2022-09-03T06:23:47.900 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:23:47.900 DEBUG:boto:Data: 2022-09-03T06:23:47.901 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.902 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.903 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.904 DEBUG:boto:Params: {} 2022-09-03T06:23:47.905 DEBUG:boto:Token: None 2022-09-03T06:23:47.906 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/d 2022-09-03T06:23:47.907 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mlxMGOY4D+s4/fgGSMUn4FZgcXM= 2022-09-03T06:23:47.908 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mlxMGOY4D+s4/fgGSMUn4FZgcXM=', 'Content-Length': '0'} 2022-09-03T06:23:47.910 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f18802c26ac4452-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.911 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:47.912 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:47.912 DEBUG:boto:Method: HEAD 2022-09-03T06:23:47.913 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:23:47.913 DEBUG:boto:Data: 2022-09-03T06:23:47.914 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.914 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.915 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.916 DEBUG:boto:Params: {} 2022-09-03T06:23:47.916 DEBUG:boto:Token: None 2022-09-03T06:23:47.917 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/d 2022-09-03T06:23:47.917 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mlxMGOY4D+s4/fgGSMUn4FZgcXM= 2022-09-03T06:23:47.918 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mlxMGOY4D+s4/fgGSMUn4FZgcXM=', 'Content-Length': '0'} 2022-09-03T06:23:47.920 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002de6bf7c58b0c1de-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.921 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:23:47.922 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:47.922 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:47.923 DEBUG:boto:Method: GET 2022-09-03T06:23:47.924 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:23:47.924 DEBUG:boto:Data: 2022-09-03T06:23:47.925 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.925 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.926 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.926 DEBUG:boto:Params: {} 2022-09-03T06:23:47.927 DEBUG:boto:Token: None 2022-09-03T06:23:47.927 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/d 2022-09-03T06:23:47.928 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6Rg/Rf6Ty0HP+fp0lXPbD00U6M8= 2022-09-03T06:23:47.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6Rg/Rf6Ty0HP+fp0lXPbD00U6M8=', 'Content-Length': '0'} 2022-09-03T06:23:47.931 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005eec80fff661b1f6-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.932 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:47.933 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:47.933 DEBUG:boto:Method: GET 2022-09-03T06:23:47.934 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:23:47.935 DEBUG:boto:Data: 2022-09-03T06:23:47.936 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.938 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.939 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.940 DEBUG:boto:Params: {} 2022-09-03T06:23:47.941 DEBUG:boto:Token: None 2022-09-03T06:23:47.942 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/d 2022-09-03T06:23:47.943 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6Rg/Rf6Ty0HP+fp0lXPbD00U6M8= 2022-09-03T06:23:47.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6Rg/Rf6Ty0HP+fp0lXPbD00U6M8=', 'Content-Length': '0'} 2022-09-03T06:23:47.947 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d584c2ee28238e4c-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.948 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:47.949 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:47.950 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:47.951 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:47.951 DEBUG:boto:Method: GET 2022-09-03T06:23:47.952 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:23:47.952 DEBUG:boto:Data: 2022-09-03T06:23:47.953 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.954 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:47.954 DEBUG:boto:Port: 8001 2022-09-03T06:23:47.955 DEBUG:boto:Params: {} 2022-09-03T06:23:47.955 DEBUG:boto:Token: None 2022-09-03T06:23:47.956 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/ 2022-09-03T06:23:47.957 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA= 2022-09-03T06:23:47.957 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA=', 'Content-Length': '0'} 2022-09-03T06:23:47.960 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000076f7f38ac54e4974-006312f2f3-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:47.961 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:47.962 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:47.963 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:47.963 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:47.964 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:47.964 DEBUG:boto:Method: GET 2022-09-03T06:23:47.965 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:23:47.965 DEBUG:boto:Data: 2022-09-03T06:23:47.966 DEBUG:boto:Headers: {} 2022-09-03T06:23:47.966 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:47.967 DEBUG:boto:Port: 8000 2022-09-03T06:23:47.968 DEBUG:boto:Params: {} 2022-09-03T06:23:47.968 DEBUG:boto:Token: None 2022-09-03T06:23:47.969 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:47 GMT /dsgvcy-72/ 2022-09-03T06:23:47.969 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA= 2022-09-03T06:23:47.970 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dly626uAvZwnIQqtNQDwDHx5uRA=', 'Content-Length': '0'} 2022-09-03T06:23:47.972 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000066948905290edbe8-006312f2f3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.014 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:48.016 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:23:48.017 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:48.018 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:48.019 DEBUG:boto:path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.020 DEBUG:boto:auth_path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.021 DEBUG:boto:Method: GET 2022-09-03T06:23:48.023 DEBUG:boto:Path: /dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.023 DEBUG:boto:Data: 2022-09-03T06:23:48.024 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.025 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.026 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.027 DEBUG:boto:Params: {} 2022-09-03T06:23:48.028 DEBUG:boto:Token: None 2022-09-03T06:23:48.030 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.031 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dxWr5sOTr+86tDNNQwtko1kKIQs= 2022-09-03T06:23:48.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dxWr5sOTr+86tDNNQwtko1kKIQs=', 'Content-Length': '0'} 2022-09-03T06:23:48.035 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000972119bc9fbf4c8c-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.036 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:48.036 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:48.037 DEBUG:boto:path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.037 DEBUG:boto:auth_path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.039 DEBUG:boto:Method: GET 2022-09-03T06:23:48.039 DEBUG:boto:Path: /dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.040 DEBUG:boto:Data: 2022-09-03T06:23:48.041 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.041 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.042 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.042 DEBUG:boto:Params: {} 2022-09-03T06:23:48.043 DEBUG:boto:Token: None 2022-09-03T06:23:48.044 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.045 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dxWr5sOTr+86tDNNQwtko1kKIQs= 2022-09-03T06:23:48.045 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dxWr5sOTr+86tDNNQwtko1kKIQs=', 'Content-Length': '0'} 2022-09-03T06:23:48.048 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b0d85af055161d9b-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.049 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:48.050 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:48.051 DEBUG:boto:path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.051 DEBUG:boto:auth_path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.052 DEBUG:boto:Method: HEAD 2022-09-03T06:23:48.053 DEBUG:boto:Path: /dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.054 DEBUG:boto:Data: 2022-09-03T06:23:48.054 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.055 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.055 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.056 DEBUG:boto:Params: {} 2022-09-03T06:23:48.056 DEBUG:boto:Token: None 2022-09-03T06:23:48.057 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.058 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WaOhg0+pdwBRIXF9k2Qdaw8IwQQ= 2022-09-03T06:23:48.058 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WaOhg0+pdwBRIXF9k2Qdaw8IwQQ=', 'Content-Length': '0'} 2022-09-03T06:23:48.061 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006aab3a78f46c87a2-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.061 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:48.062 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:48.062 DEBUG:boto:path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.063 DEBUG:boto:auth_path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.063 DEBUG:boto:Method: HEAD 2022-09-03T06:23:48.064 DEBUG:boto:Path: /dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.064 DEBUG:boto:Data: 2022-09-03T06:23:48.065 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.065 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.066 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.066 DEBUG:boto:Params: {} 2022-09-03T06:23:48.067 DEBUG:boto:Token: None 2022-09-03T06:23:48.067 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.068 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WaOhg0+pdwBRIXF9k2Qdaw8IwQQ= 2022-09-03T06:23:48.069 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WaOhg0+pdwBRIXF9k2Qdaw8IwQQ=', 'Content-Length': '0'} 2022-09-03T06:23:48.071 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f7b04312cb4fd29-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.072 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:23:48.072 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:48.073 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:48.074 DEBUG:boto:path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.074 DEBUG:boto:auth_path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.075 DEBUG:boto:Method: GET 2022-09-03T06:23:48.075 DEBUG:boto:Path: /dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.076 DEBUG:boto:Data: 2022-09-03T06:23:48.076 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.077 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.077 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.078 DEBUG:boto:Params: {} 2022-09-03T06:23:48.079 DEBUG:boto:Token: None 2022-09-03T06:23:48.079 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.080 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dxWr5sOTr+86tDNNQwtko1kKIQs= 2022-09-03T06:23:48.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dxWr5sOTr+86tDNNQwtko1kKIQs=', 'Content-Length': '0'} 2022-09-03T06:23:48.083 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a629f1a0fec7f32b-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.126 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:48.127 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:48.128 DEBUG:boto:path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.129 DEBUG:boto:auth_path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.130 DEBUG:boto:Method: GET 2022-09-03T06:23:48.131 DEBUG:boto:Path: /dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.132 DEBUG:boto:Data: 2022-09-03T06:23:48.133 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.133 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.134 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.135 DEBUG:boto:Params: {} 2022-09-03T06:23:48.135 DEBUG:boto:Token: None 2022-09-03T06:23:48.136 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/e?versionId=null 2022-09-03T06:23:48.136 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dxWr5sOTr+86tDNNQwtko1kKIQs= 2022-09-03T06:23:48.137 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dxWr5sOTr+86tDNNQwtko1kKIQs=', 'Content-Length': '0'} 2022-09-03T06:23:48.139 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000024901aa1cf48fffb-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.140 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:48.141 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:48.141 DEBUG:boto:path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:48.142 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:48.142 DEBUG:boto:Method: GET 2022-09-03T06:23:48.143 DEBUG:boto:Path: /dsgvcy-72/?prefix=e 2022-09-03T06:23:48.143 DEBUG:boto:Data: 2022-09-03T06:23:48.144 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.144 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.145 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.145 DEBUG:boto:Params: {} 2022-09-03T06:23:48.146 DEBUG:boto:Token: None 2022-09-03T06:23:48.147 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/ 2022-09-03T06:23:48.147 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU= 2022-09-03T06:23:48.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU=', 'Content-Length': '0'} 2022-09-03T06:23:48.151 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008c172acdd99c24c9-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.151 DEBUG:boto:b'dsgvcy-72e1000falsee2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:48.152 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:48.153 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:48.153 DEBUG:boto:path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:48.154 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:48.154 DEBUG:boto:Method: GET 2022-09-03T06:23:48.155 DEBUG:boto:Path: /dsgvcy-72/?prefix=e 2022-09-03T06:23:48.155 DEBUG:boto:Data: 2022-09-03T06:23:48.156 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.156 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.157 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.157 DEBUG:boto:Params: {} 2022-09-03T06:23:48.158 DEBUG:boto:Token: None 2022-09-03T06:23:48.159 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/ 2022-09-03T06:23:48.159 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU= 2022-09-03T06:23:48.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU=', 'Content-Length': '0'} 2022-09-03T06:23:48.162 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000059c2a30f0bb53200-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.163 DEBUG:boto:b'dsgvcy-72e1000falsee2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:48.164 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:48.165 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:48.165 DEBUG:boto:Method: HEAD 2022-09-03T06:23:48.166 DEBUG:boto:Path: /dsgvcy-72/e 2022-09-03T06:23:48.166 DEBUG:boto:Data: 2022-09-03T06:23:48.167 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.167 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.168 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.168 DEBUG:boto:Params: {} 2022-09-03T06:23:48.169 DEBUG:boto:Token: None 2022-09-03T06:23:48.170 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/e 2022-09-03T06:23:48.170 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vstWWbJVSPgn2u2RWM6gbbErd4E= 2022-09-03T06:23:48.171 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vstWWbJVSPgn2u2RWM6gbbErd4E=', 'Content-Length': '0'} 2022-09-03T06:23:48.173 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000806825f26989657-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.174 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:48.174 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:48.175 DEBUG:boto:Method: HEAD 2022-09-03T06:23:48.175 DEBUG:boto:Path: /dsgvcy-72/e 2022-09-03T06:23:48.176 DEBUG:boto:Data: 2022-09-03T06:23:48.176 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.177 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.177 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.178 DEBUG:boto:Params: {} 2022-09-03T06:23:48.178 DEBUG:boto:Token: None 2022-09-03T06:23:48.179 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/e 2022-09-03T06:23:48.179 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vstWWbJVSPgn2u2RWM6gbbErd4E= 2022-09-03T06:23:48.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vstWWbJVSPgn2u2RWM6gbbErd4E=', 'Content-Length': '0'} 2022-09-03T06:23:48.182 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009b5ecba3eb9e3451-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.183 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:23:48.183 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:48.184 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:48.184 DEBUG:boto:Method: GET 2022-09-03T06:23:48.185 DEBUG:boto:Path: /dsgvcy-72/e 2022-09-03T06:23:48.185 DEBUG:boto:Data: 2022-09-03T06:23:48.186 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.186 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.187 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.188 DEBUG:boto:Params: {} 2022-09-03T06:23:48.188 DEBUG:boto:Token: None 2022-09-03T06:23:48.189 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/e 2022-09-03T06:23:48.189 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WefLaNlQeaqFbHOMN/mt09SrIms= 2022-09-03T06:23:48.190 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WefLaNlQeaqFbHOMN/mt09SrIms=', 'Content-Length': '0'} 2022-09-03T06:23:48.192 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000741be1c05496827c-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.193 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:48.194 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:48.194 DEBUG:boto:Method: GET 2022-09-03T06:23:48.195 DEBUG:boto:Path: /dsgvcy-72/e 2022-09-03T06:23:48.195 DEBUG:boto:Data: 2022-09-03T06:23:48.196 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.196 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.197 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.197 DEBUG:boto:Params: {} 2022-09-03T06:23:48.198 DEBUG:boto:Token: None 2022-09-03T06:23:48.199 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/e 2022-09-03T06:23:48.199 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WefLaNlQeaqFbHOMN/mt09SrIms= 2022-09-03T06:23:48.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WefLaNlQeaqFbHOMN/mt09SrIms=', 'Content-Length': '0'} 2022-09-03T06:23:48.202 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d2a93425fa9bed8e-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.250 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:48.251 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:48.252 DEBUG:boto:path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:48.253 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:48.254 DEBUG:boto:Method: GET 2022-09-03T06:23:48.255 DEBUG:boto:Path: /dsgvcy-72/?prefix=e 2022-09-03T06:23:48.256 DEBUG:boto:Data: 2022-09-03T06:23:48.257 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.258 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.259 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.259 DEBUG:boto:Params: {} 2022-09-03T06:23:48.260 DEBUG:boto:Token: None 2022-09-03T06:23:48.260 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/ 2022-09-03T06:23:48.261 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU= 2022-09-03T06:23:48.262 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU=', 'Content-Length': '0'} 2022-09-03T06:23:48.264 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d64b7c7312843e8a-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.265 DEBUG:boto:b'dsgvcy-72e1000falsee2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:48.266 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:48.267 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:48.267 DEBUG:boto:path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:48.268 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:48.269 DEBUG:boto:Method: GET 2022-09-03T06:23:48.269 DEBUG:boto:Path: /dsgvcy-72/?prefix=e 2022-09-03T06:23:48.270 DEBUG:boto:Data: 2022-09-03T06:23:48.271 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.271 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.272 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.273 DEBUG:boto:Params: {} 2022-09-03T06:23:48.273 DEBUG:boto:Token: None 2022-09-03T06:23:48.274 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/ 2022-09-03T06:23:48.275 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU= 2022-09-03T06:23:48.275 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU=', 'Content-Length': '0'} 2022-09-03T06:23:48.278 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a01e4c1149b16157-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.278 DEBUG:boto:b'dsgvcy-72e1000falsee2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:48.279 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:23:48.280 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:48.280 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:48.281 DEBUG:boto:path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.282 DEBUG:boto:auth_path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.282 DEBUG:boto:Method: GET 2022-09-03T06:23:48.283 DEBUG:boto:Path: /dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.284 DEBUG:boto:Data: 2022-09-03T06:23:48.284 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.285 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.285 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.286 DEBUG:boto:Params: {} 2022-09-03T06:23:48.287 DEBUG:boto:Token: None 2022-09-03T06:23:48.287 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.288 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O://Yx0KOK7neUj9emxHlY3zx3LBQ= 2022-09-03T06:23:48.289 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O://Yx0KOK7neUj9emxHlY3zx3LBQ=', 'Content-Length': '0'} 2022-09-03T06:23:48.291 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000713c216b007d2652-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.292 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:48.293 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:48.293 DEBUG:boto:path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.294 DEBUG:boto:auth_path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.295 DEBUG:boto:Method: GET 2022-09-03T06:23:48.295 DEBUG:boto:Path: /dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.296 DEBUG:boto:Data: 2022-09-03T06:23:48.296 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.297 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.297 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.298 DEBUG:boto:Params: {} 2022-09-03T06:23:48.299 DEBUG:boto:Token: None 2022-09-03T06:23:48.299 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.300 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O://Yx0KOK7neUj9emxHlY3zx3LBQ= 2022-09-03T06:23:48.301 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O://Yx0KOK7neUj9emxHlY3zx3LBQ=', 'Content-Length': '0'} 2022-09-03T06:23:48.303 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003bfc36c8e6f171ed-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.304 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:48.305 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:48.305 DEBUG:boto:path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.306 DEBUG:boto:auth_path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.306 DEBUG:boto:Method: HEAD 2022-09-03T06:23:48.307 DEBUG:boto:Path: /dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.307 DEBUG:boto:Data: 2022-09-03T06:23:48.308 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.309 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.309 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.310 DEBUG:boto:Params: {} 2022-09-03T06:23:48.310 DEBUG:boto:Token: None 2022-09-03T06:23:48.311 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.312 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tBzSdFXYByU4pxOJqStr55LN98M= 2022-09-03T06:23:48.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tBzSdFXYByU4pxOJqStr55LN98M=', 'Content-Length': '0'} 2022-09-03T06:23:48.314 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b77edc14754ce4d5-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.315 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:48.316 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:48.316 DEBUG:boto:path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.317 DEBUG:boto:auth_path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.317 DEBUG:boto:Method: HEAD 2022-09-03T06:23:48.318 DEBUG:boto:Path: /dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.318 DEBUG:boto:Data: 2022-09-03T06:23:48.319 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.319 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.320 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.320 DEBUG:boto:Params: {} 2022-09-03T06:23:48.321 DEBUG:boto:Token: None 2022-09-03T06:23:48.321 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.322 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tBzSdFXYByU4pxOJqStr55LN98M= 2022-09-03T06:23:48.323 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tBzSdFXYByU4pxOJqStr55LN98M=', 'Content-Length': '0'} 2022-09-03T06:23:48.325 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e9b9e9abb25ed52-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.326 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:23:48.327 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:48.327 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:48.328 DEBUG:boto:path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.329 DEBUG:boto:auth_path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.330 DEBUG:boto:Method: GET 2022-09-03T06:23:48.331 DEBUG:boto:Path: /dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.332 DEBUG:boto:Data: 2022-09-03T06:23:48.333 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.334 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.335 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.336 DEBUG:boto:Params: {} 2022-09-03T06:23:48.337 DEBUG:boto:Token: None 2022-09-03T06:23:48.338 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.339 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O://Yx0KOK7neUj9emxHlY3zx3LBQ= 2022-09-03T06:23:48.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O://Yx0KOK7neUj9emxHlY3zx3LBQ=', 'Content-Length': '0'} 2022-09-03T06:23:48.342 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005a2f39cbbc3b7c12-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.386 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:48.387 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:48.388 DEBUG:boto:path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.388 DEBUG:boto:auth_path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.389 DEBUG:boto:Method: GET 2022-09-03T06:23:48.390 DEBUG:boto:Path: /dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.390 DEBUG:boto:Data: 2022-09-03T06:23:48.391 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.391 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.392 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.393 DEBUG:boto:Params: {} 2022-09-03T06:23:48.395 DEBUG:boto:Token: None 2022-09-03T06:23:48.396 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/f?versionId=null 2022-09-03T06:23:48.397 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O://Yx0KOK7neUj9emxHlY3zx3LBQ= 2022-09-03T06:23:48.397 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O://Yx0KOK7neUj9emxHlY3zx3LBQ=', 'Content-Length': '0'} 2022-09-03T06:23:48.400 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f491632f17dec13-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.401 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:48.401 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:48.402 DEBUG:boto:path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:48.403 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:48.404 DEBUG:boto:Method: GET 2022-09-03T06:23:48.405 DEBUG:boto:Path: /dsgvcy-72/?prefix=f 2022-09-03T06:23:48.406 DEBUG:boto:Data: 2022-09-03T06:23:48.407 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.408 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.409 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.410 DEBUG:boto:Params: {} 2022-09-03T06:23:48.411 DEBUG:boto:Token: None 2022-09-03T06:23:48.413 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/ 2022-09-03T06:23:48.414 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU= 2022-09-03T06:23:48.414 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU=', 'Content-Length': '0'} 2022-09-03T06:23:48.418 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000cbc6f919b900345-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.418 DEBUG:boto:b'dsgvcy-72f1000falsef2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:48.419 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:48.420 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:48.420 DEBUG:boto:path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:48.421 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:48.421 DEBUG:boto:Method: GET 2022-09-03T06:23:48.422 DEBUG:boto:Path: /dsgvcy-72/?prefix=f 2022-09-03T06:23:48.422 DEBUG:boto:Data: 2022-09-03T06:23:48.423 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.424 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.424 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.425 DEBUG:boto:Params: {} 2022-09-03T06:23:48.426 DEBUG:boto:Token: None 2022-09-03T06:23:48.426 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/ 2022-09-03T06:23:48.427 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU= 2022-09-03T06:23:48.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU=', 'Content-Length': '0'} 2022-09-03T06:23:48.430 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ab403ecfe0b954f9-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.431 DEBUG:boto:b'dsgvcy-72f1000falsef2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:48.432 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:48.432 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:48.433 DEBUG:boto:Method: HEAD 2022-09-03T06:23:48.433 DEBUG:boto:Path: /dsgvcy-72/f 2022-09-03T06:23:48.434 DEBUG:boto:Data: 2022-09-03T06:23:48.435 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.436 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.438 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.438 DEBUG:boto:Params: {} 2022-09-03T06:23:48.439 DEBUG:boto:Token: None 2022-09-03T06:23:48.440 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/f 2022-09-03T06:23:48.440 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UTy1iqec+M3uA4S2ugsTHd4nPtI= 2022-09-03T06:23:48.441 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UTy1iqec+M3uA4S2ugsTHd4nPtI=', 'Content-Length': '0'} 2022-09-03T06:23:48.446 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002dde8aab234a48c6-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.447 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:48.448 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:48.449 DEBUG:boto:Method: HEAD 2022-09-03T06:23:48.450 DEBUG:boto:Path: /dsgvcy-72/f 2022-09-03T06:23:48.451 DEBUG:boto:Data: 2022-09-03T06:23:48.452 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.453 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.454 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.455 DEBUG:boto:Params: {} 2022-09-03T06:23:48.456 DEBUG:boto:Token: None 2022-09-03T06:23:48.457 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/f 2022-09-03T06:23:48.458 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UTy1iqec+M3uA4S2ugsTHd4nPtI= 2022-09-03T06:23:48.459 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UTy1iqec+M3uA4S2ugsTHd4nPtI=', 'Content-Length': '0'} 2022-09-03T06:23:48.461 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c43251bd7662f303-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.462 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:23:48.463 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:48.464 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:48.465 DEBUG:boto:Method: GET 2022-09-03T06:23:48.466 DEBUG:boto:Path: /dsgvcy-72/f 2022-09-03T06:23:48.467 DEBUG:boto:Data: 2022-09-03T06:23:48.468 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.469 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.470 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.471 DEBUG:boto:Params: {} 2022-09-03T06:23:48.472 DEBUG:boto:Token: None 2022-09-03T06:23:48.473 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/f 2022-09-03T06:23:48.474 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:toYbJ51M7AbejRuMKF3kdJWt/48= 2022-09-03T06:23:48.475 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:toYbJ51M7AbejRuMKF3kdJWt/48=', 'Content-Length': '0'} 2022-09-03T06:23:48.478 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f0849dffe33374d8-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.478 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:48.479 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:48.480 DEBUG:boto:Method: GET 2022-09-03T06:23:48.480 DEBUG:boto:Path: /dsgvcy-72/f 2022-09-03T06:23:48.481 DEBUG:boto:Data: 2022-09-03T06:23:48.481 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.482 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.482 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.483 DEBUG:boto:Params: {} 2022-09-03T06:23:48.483 DEBUG:boto:Token: None 2022-09-03T06:23:48.484 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/f 2022-09-03T06:23:48.485 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:toYbJ51M7AbejRuMKF3kdJWt/48= 2022-09-03T06:23:48.485 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:toYbJ51M7AbejRuMKF3kdJWt/48=', 'Content-Length': '0'} 2022-09-03T06:23:48.488 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000943fc8a0fe1cc525-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.534 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:48.535 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:48.535 DEBUG:boto:path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:48.536 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:48.536 DEBUG:boto:Method: GET 2022-09-03T06:23:48.537 DEBUG:boto:Path: /dsgvcy-72/?prefix=f 2022-09-03T06:23:48.537 DEBUG:boto:Data: 2022-09-03T06:23:48.538 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.539 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.539 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.540 DEBUG:boto:Params: {} 2022-09-03T06:23:48.540 DEBUG:boto:Token: None 2022-09-03T06:23:48.541 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/ 2022-09-03T06:23:48.541 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU= 2022-09-03T06:23:48.542 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU=', 'Content-Length': '0'} 2022-09-03T06:23:48.545 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007c46e43bb34a7510-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.546 DEBUG:boto:b'dsgvcy-72f1000falsef2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:48.547 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:48.547 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:48.548 DEBUG:boto:path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:48.548 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:48.549 DEBUG:boto:Method: GET 2022-09-03T06:23:48.550 DEBUG:boto:Path: /dsgvcy-72/?prefix=f 2022-09-03T06:23:48.550 DEBUG:boto:Data: 2022-09-03T06:23:48.551 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.551 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.552 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.552 DEBUG:boto:Params: {} 2022-09-03T06:23:48.553 DEBUG:boto:Token: None 2022-09-03T06:23:48.554 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/ 2022-09-03T06:23:48.554 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU= 2022-09-03T06:23:48.555 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU=', 'Content-Length': '0'} 2022-09-03T06:23:48.557 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000311c3cf5255bb37d-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.558 DEBUG:boto:b'dsgvcy-72f1000falsef2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:48.559 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:23:48.559 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:48.560 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:48.560 DEBUG:boto:path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.561 DEBUG:boto:auth_path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.561 DEBUG:boto:Method: GET 2022-09-03T06:23:48.562 DEBUG:boto:Path: /dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.563 DEBUG:boto:Data: 2022-09-03T06:23:48.563 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.564 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.564 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.565 DEBUG:boto:Params: {} 2022-09-03T06:23:48.565 DEBUG:boto:Token: None 2022-09-03T06:23:48.566 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.566 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JZ1Ri5iEpeJ++jl2iVmjeLNz/6k= 2022-09-03T06:23:48.567 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JZ1Ri5iEpeJ++jl2iVmjeLNz/6k=', 'Content-Length': '0'} 2022-09-03T06:23:48.569 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074ca6814d5c6951c-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.570 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:48.570 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:48.571 DEBUG:boto:path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.571 DEBUG:boto:auth_path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.572 DEBUG:boto:Method: GET 2022-09-03T06:23:48.572 DEBUG:boto:Path: /dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.573 DEBUG:boto:Data: 2022-09-03T06:23:48.574 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.574 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.575 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.575 DEBUG:boto:Params: {} 2022-09-03T06:23:48.576 DEBUG:boto:Token: None 2022-09-03T06:23:48.576 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.577 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JZ1Ri5iEpeJ++jl2iVmjeLNz/6k= 2022-09-03T06:23:48.578 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JZ1Ri5iEpeJ++jl2iVmjeLNz/6k=', 'Content-Length': '0'} 2022-09-03T06:23:48.580 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000068974c93a7ee574-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.581 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:48.581 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:48.582 DEBUG:boto:path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.582 DEBUG:boto:auth_path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.583 DEBUG:boto:Method: HEAD 2022-09-03T06:23:48.584 DEBUG:boto:Path: /dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.584 DEBUG:boto:Data: 2022-09-03T06:23:48.585 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.585 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.586 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.586 DEBUG:boto:Params: {} 2022-09-03T06:23:48.587 DEBUG:boto:Token: None 2022-09-03T06:23:48.588 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.588 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:x9xTULR+Kh9Fae/+uwTvFcKqzxU= 2022-09-03T06:23:48.589 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:x9xTULR+Kh9Fae/+uwTvFcKqzxU=', 'Content-Length': '0'} 2022-09-03T06:23:48.591 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000078354b7e850b519c-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.592 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:48.592 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:48.593 DEBUG:boto:path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.593 DEBUG:boto:auth_path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.594 DEBUG:boto:Method: HEAD 2022-09-03T06:23:48.594 DEBUG:boto:Path: /dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.595 DEBUG:boto:Data: 2022-09-03T06:23:48.595 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.596 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.596 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.597 DEBUG:boto:Params: {} 2022-09-03T06:23:48.597 DEBUG:boto:Token: None 2022-09-03T06:23:48.598 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.599 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:x9xTULR+Kh9Fae/+uwTvFcKqzxU= 2022-09-03T06:23:48.599 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:x9xTULR+Kh9Fae/+uwTvFcKqzxU=', 'Content-Length': '0'} 2022-09-03T06:23:48.601 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000027febfb4ebbdb5a4-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.602 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:23:48.603 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:48.603 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:48.604 DEBUG:boto:path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.604 DEBUG:boto:auth_path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.605 DEBUG:boto:Method: GET 2022-09-03T06:23:48.605 DEBUG:boto:Path: /dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.606 DEBUG:boto:Data: 2022-09-03T06:23:48.606 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.607 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.607 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.608 DEBUG:boto:Params: {} 2022-09-03T06:23:48.608 DEBUG:boto:Token: None 2022-09-03T06:23:48.609 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.610 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JZ1Ri5iEpeJ++jl2iVmjeLNz/6k= 2022-09-03T06:23:48.610 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JZ1Ri5iEpeJ++jl2iVmjeLNz/6k=', 'Content-Length': '0'} 2022-09-03T06:23:48.612 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000acf28ca21c17520-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.613 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:48.614 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:48.614 DEBUG:boto:path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.615 DEBUG:boto:auth_path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.615 DEBUG:boto:Method: GET 2022-09-03T06:23:48.616 DEBUG:boto:Path: /dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.616 DEBUG:boto:Data: 2022-09-03T06:23:48.617 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.617 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.618 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.618 DEBUG:boto:Params: {} 2022-09-03T06:23:48.619 DEBUG:boto:Token: None 2022-09-03T06:23:48.620 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/g?versionId=null 2022-09-03T06:23:48.620 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JZ1Ri5iEpeJ++jl2iVmjeLNz/6k= 2022-09-03T06:23:48.621 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JZ1Ri5iEpeJ++jl2iVmjeLNz/6k=', 'Content-Length': '0'} 2022-09-03T06:23:48.623 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000001bd03c1398f1c38-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.624 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:48.624 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:48.625 DEBUG:boto:path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:48.625 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:48.626 DEBUG:boto:Method: GET 2022-09-03T06:23:48.626 DEBUG:boto:Path: /dsgvcy-72/?prefix=g 2022-09-03T06:23:48.627 DEBUG:boto:Data: 2022-09-03T06:23:48.627 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.628 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.628 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.629 DEBUG:boto:Params: {} 2022-09-03T06:23:48.630 DEBUG:boto:Token: None 2022-09-03T06:23:48.631 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/ 2022-09-03T06:23:48.631 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU= 2022-09-03T06:23:48.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU=', 'Content-Length': '0'} 2022-09-03T06:23:48.638 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001a9a7f3bbfc4d848-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.639 DEBUG:boto:b'dsgvcy-72g1000falseg2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:48.640 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:48.640 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:48.641 DEBUG:boto:path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:48.642 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:48.642 DEBUG:boto:Method: GET 2022-09-03T06:23:48.643 DEBUG:boto:Path: /dsgvcy-72/?prefix=g 2022-09-03T06:23:48.643 DEBUG:boto:Data: 2022-09-03T06:23:48.644 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.644 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.645 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.645 DEBUG:boto:Params: {} 2022-09-03T06:23:48.646 DEBUG:boto:Token: None 2022-09-03T06:23:48.646 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/ 2022-09-03T06:23:48.647 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU= 2022-09-03T06:23:48.648 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU=', 'Content-Length': '0'} 2022-09-03T06:23:48.650 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000041c7b1cc33b61309-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.651 DEBUG:boto:b'dsgvcy-72g1000falseg2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:48.652 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:48.652 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:48.653 DEBUG:boto:Method: HEAD 2022-09-03T06:23:48.653 DEBUG:boto:Path: /dsgvcy-72/g 2022-09-03T06:23:48.654 DEBUG:boto:Data: 2022-09-03T06:23:48.654 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.655 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.655 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.656 DEBUG:boto:Params: {} 2022-09-03T06:23:48.657 DEBUG:boto:Token: None 2022-09-03T06:23:48.658 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/g 2022-09-03T06:23:48.658 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O1CVPlHxP2w+VjbYUWZkV3+wO14= 2022-09-03T06:23:48.659 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O1CVPlHxP2w+VjbYUWZkV3+wO14=', 'Content-Length': '0'} 2022-09-03T06:23:48.661 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e0960c2fb3b286b-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.662 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:48.662 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:48.663 DEBUG:boto:Method: HEAD 2022-09-03T06:23:48.663 DEBUG:boto:Path: /dsgvcy-72/g 2022-09-03T06:23:48.664 DEBUG:boto:Data: 2022-09-03T06:23:48.665 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.666 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.667 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.668 DEBUG:boto:Params: {} 2022-09-03T06:23:48.669 DEBUG:boto:Token: None 2022-09-03T06:23:48.670 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/g 2022-09-03T06:23:48.671 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O1CVPlHxP2w+VjbYUWZkV3+wO14= 2022-09-03T06:23:48.671 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O1CVPlHxP2w+VjbYUWZkV3+wO14=', 'Content-Length': '0'} 2022-09-03T06:23:48.674 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000430ec6c4f1fd4dad-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.675 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:23:48.676 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:48.676 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:48.677 DEBUG:boto:Method: GET 2022-09-03T06:23:48.678 DEBUG:boto:Path: /dsgvcy-72/g 2022-09-03T06:23:48.678 DEBUG:boto:Data: 2022-09-03T06:23:48.679 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.679 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.680 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.680 DEBUG:boto:Params: {} 2022-09-03T06:23:48.681 DEBUG:boto:Token: None 2022-09-03T06:23:48.681 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/g 2022-09-03T06:23:48.682 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DL/Z8RQtU0RaR1fvEgFACmVfMuI= 2022-09-03T06:23:48.683 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DL/Z8RQtU0RaR1fvEgFACmVfMuI=', 'Content-Length': '0'} 2022-09-03T06:23:48.685 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee45ce6718d02550-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.726 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:48.728 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:48.729 DEBUG:boto:Method: GET 2022-09-03T06:23:48.730 DEBUG:boto:Path: /dsgvcy-72/g 2022-09-03T06:23:48.731 DEBUG:boto:Data: 2022-09-03T06:23:48.732 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.733 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.733 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.735 DEBUG:boto:Params: {} 2022-09-03T06:23:48.736 DEBUG:boto:Token: None 2022-09-03T06:23:48.737 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/g 2022-09-03T06:23:48.738 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DL/Z8RQtU0RaR1fvEgFACmVfMuI= 2022-09-03T06:23:48.739 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DL/Z8RQtU0RaR1fvEgFACmVfMuI=', 'Content-Length': '0'} 2022-09-03T06:23:48.742 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e7ef95eea4fe227b-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.743 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:48.744 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:48.745 DEBUG:boto:path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:48.746 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:48.747 DEBUG:boto:Method: GET 2022-09-03T06:23:48.747 DEBUG:boto:Path: /dsgvcy-72/?prefix=g 2022-09-03T06:23:48.748 DEBUG:boto:Data: 2022-09-03T06:23:48.749 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.749 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.750 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.750 DEBUG:boto:Params: {} 2022-09-03T06:23:48.751 DEBUG:boto:Token: None 2022-09-03T06:23:48.752 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/ 2022-09-03T06:23:48.752 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU= 2022-09-03T06:23:48.754 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU=', 'Content-Length': '0'} 2022-09-03T06:23:48.757 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a3ce0816c7b48f36-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.757 DEBUG:boto:b'dsgvcy-72g1000falseg2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:48.758 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:48.759 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:48.759 DEBUG:boto:path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:48.760 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:48.761 DEBUG:boto:Method: GET 2022-09-03T06:23:48.761 DEBUG:boto:Path: /dsgvcy-72/?prefix=g 2022-09-03T06:23:48.762 DEBUG:boto:Data: 2022-09-03T06:23:48.763 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.763 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.764 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.765 DEBUG:boto:Params: {} 2022-09-03T06:23:48.765 DEBUG:boto:Token: None 2022-09-03T06:23:48.766 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/ 2022-09-03T06:23:48.767 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU= 2022-09-03T06:23:48.767 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU=', 'Content-Length': '0'} 2022-09-03T06:23:48.770 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b293e6244d87f86a-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.770 DEBUG:boto:b'dsgvcy-72g1000falseg2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:48.771 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:23:48.772 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:48.773 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:48.780 DEBUG:boto:path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.781 DEBUG:boto:auth_path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.782 DEBUG:boto:Method: GET 2022-09-03T06:23:48.783 DEBUG:boto:Path: /dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.784 DEBUG:boto:Data: 2022-09-03T06:23:48.785 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.785 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.786 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.787 DEBUG:boto:Params: {} 2022-09-03T06:23:48.787 DEBUG:boto:Token: None 2022-09-03T06:23:48.788 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.789 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ztSZ90w4pcj6ZH/9wtgsLxfMOfk= 2022-09-03T06:23:48.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ztSZ90w4pcj6ZH/9wtgsLxfMOfk=', 'Content-Length': '0'} 2022-09-03T06:23:48.792 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bcc14c87fa0c861d-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.793 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:48.793 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:48.794 DEBUG:boto:path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.795 DEBUG:boto:auth_path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.795 DEBUG:boto:Method: GET 2022-09-03T06:23:48.796 DEBUG:boto:Path: /dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.797 DEBUG:boto:Data: 2022-09-03T06:23:48.797 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.798 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.798 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.799 DEBUG:boto:Params: {} 2022-09-03T06:23:48.800 DEBUG:boto:Token: None 2022-09-03T06:23:48.801 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.802 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ztSZ90w4pcj6ZH/9wtgsLxfMOfk= 2022-09-03T06:23:48.802 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ztSZ90w4pcj6ZH/9wtgsLxfMOfk=', 'Content-Length': '0'} 2022-09-03T06:23:48.805 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000614619e77d869f70-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.806 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:48.807 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:48.808 DEBUG:boto:path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.809 DEBUG:boto:auth_path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.810 DEBUG:boto:Method: HEAD 2022-09-03T06:23:48.810 DEBUG:boto:Path: /dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.811 DEBUG:boto:Data: 2022-09-03T06:23:48.812 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.812 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.812 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.813 DEBUG:boto:Params: {} 2022-09-03T06:23:48.814 DEBUG:boto:Token: None 2022-09-03T06:23:48.814 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.815 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sng/GzyObL5NAObPkYsX3vohDAY= 2022-09-03T06:23:48.815 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sng/GzyObL5NAObPkYsX3vohDAY=', 'Content-Length': '0'} 2022-09-03T06:23:48.819 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000989c4bf79810041f-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.821 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:48.821 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:48.822 DEBUG:boto:path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.823 DEBUG:boto:auth_path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.823 DEBUG:boto:Method: HEAD 2022-09-03T06:23:48.824 DEBUG:boto:Path: /dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.825 DEBUG:boto:Data: 2022-09-03T06:23:48.825 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.826 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.826 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.827 DEBUG:boto:Params: {} 2022-09-03T06:23:48.827 DEBUG:boto:Token: None 2022-09-03T06:23:48.828 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.829 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sng/GzyObL5NAObPkYsX3vohDAY= 2022-09-03T06:23:48.829 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sng/GzyObL5NAObPkYsX3vohDAY=', 'Content-Length': '0'} 2022-09-03T06:23:48.831 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000016f800008bb09ada-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.832 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:23:48.833 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:48.833 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:48.834 DEBUG:boto:path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.834 DEBUG:boto:auth_path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.835 DEBUG:boto:Method: GET 2022-09-03T06:23:48.835 DEBUG:boto:Path: /dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.836 DEBUG:boto:Data: 2022-09-03T06:23:48.837 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.837 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.838 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.839 DEBUG:boto:Params: {} 2022-09-03T06:23:48.839 DEBUG:boto:Token: None 2022-09-03T06:23:48.840 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.840 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ztSZ90w4pcj6ZH/9wtgsLxfMOfk= 2022-09-03T06:23:48.841 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ztSZ90w4pcj6ZH/9wtgsLxfMOfk=', 'Content-Length': '0'} 2022-09-03T06:23:48.843 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008738d6632c24f7dc-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.844 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:48.845 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:48.845 DEBUG:boto:path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.846 DEBUG:boto:auth_path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.846 DEBUG:boto:Method: GET 2022-09-03T06:23:48.847 DEBUG:boto:Path: /dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.848 DEBUG:boto:Data: 2022-09-03T06:23:48.848 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.849 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.849 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.850 DEBUG:boto:Params: {} 2022-09-03T06:23:48.851 DEBUG:boto:Token: None 2022-09-03T06:23:48.851 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/h?versionId=null 2022-09-03T06:23:48.852 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ztSZ90w4pcj6ZH/9wtgsLxfMOfk= 2022-09-03T06:23:48.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ztSZ90w4pcj6ZH/9wtgsLxfMOfk=', 'Content-Length': '0'} 2022-09-03T06:23:48.855 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004cda32225ebb7ae6-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.856 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:48.856 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:48.857 DEBUG:boto:path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:48.857 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:48.858 DEBUG:boto:Method: GET 2022-09-03T06:23:48.859 DEBUG:boto:Path: /dsgvcy-72/?prefix=h 2022-09-03T06:23:48.859 DEBUG:boto:Data: 2022-09-03T06:23:48.860 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.860 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.861 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.861 DEBUG:boto:Params: {} 2022-09-03T06:23:48.862 DEBUG:boto:Token: None 2022-09-03T06:23:48.862 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/ 2022-09-03T06:23:48.863 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU= 2022-09-03T06:23:48.864 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU=', 'Content-Length': '0'} 2022-09-03T06:23:48.867 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002d4a78aab346d289-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.867 DEBUG:boto:b'dsgvcy-72h1000falseh2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:48.868 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:48.869 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:48.869 DEBUG:boto:path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:48.870 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:48.871 DEBUG:boto:Method: GET 2022-09-03T06:23:48.872 DEBUG:boto:Path: /dsgvcy-72/?prefix=h 2022-09-03T06:23:48.873 DEBUG:boto:Data: 2022-09-03T06:23:48.874 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.875 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.876 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.877 DEBUG:boto:Params: {} 2022-09-03T06:23:48.878 DEBUG:boto:Token: None 2022-09-03T06:23:48.879 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/ 2022-09-03T06:23:48.879 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU= 2022-09-03T06:23:48.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU=', 'Content-Length': '0'} 2022-09-03T06:23:48.882 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b67cd4bc20f47ab7-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.883 DEBUG:boto:b'dsgvcy-72h1000falseh2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:48.884 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:48.884 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:48.885 DEBUG:boto:Method: HEAD 2022-09-03T06:23:48.885 DEBUG:boto:Path: /dsgvcy-72/h 2022-09-03T06:23:48.886 DEBUG:boto:Data: 2022-09-03T06:23:48.887 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.887 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.888 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.888 DEBUG:boto:Params: {} 2022-09-03T06:23:48.889 DEBUG:boto:Token: None 2022-09-03T06:23:48.890 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/h 2022-09-03T06:23:48.891 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XUwmThDkq+gJ/XnS5DWUDFpZCXc= 2022-09-03T06:23:48.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XUwmThDkq+gJ/XnS5DWUDFpZCXc=', 'Content-Length': '0'} 2022-09-03T06:23:48.895 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d6dd39dd50927f69-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.896 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:48.897 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:48.898 DEBUG:boto:Method: HEAD 2022-09-03T06:23:48.900 DEBUG:boto:Path: /dsgvcy-72/h 2022-09-03T06:23:48.900 DEBUG:boto:Data: 2022-09-03T06:23:48.901 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.902 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.903 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.903 DEBUG:boto:Params: {} 2022-09-03T06:23:48.904 DEBUG:boto:Token: None 2022-09-03T06:23:48.905 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/h 2022-09-03T06:23:48.905 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XUwmThDkq+gJ/XnS5DWUDFpZCXc= 2022-09-03T06:23:48.906 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XUwmThDkq+gJ/XnS5DWUDFpZCXc=', 'Content-Length': '0'} 2022-09-03T06:23:48.908 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006984addbe5538a12-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.909 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:23:48.910 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:48.911 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:48.912 DEBUG:boto:Method: GET 2022-09-03T06:23:48.912 DEBUG:boto:Path: /dsgvcy-72/h 2022-09-03T06:23:48.913 DEBUG:boto:Data: 2022-09-03T06:23:48.914 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.914 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.915 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.915 DEBUG:boto:Params: {} 2022-09-03T06:23:48.916 DEBUG:boto:Token: None 2022-09-03T06:23:48.917 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/h 2022-09-03T06:23:48.918 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WMo6RIpapC1ZT7BLs/4Ym9zGw3E= 2022-09-03T06:23:48.919 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WMo6RIpapC1ZT7BLs/4Ym9zGw3E=', 'Content-Length': '0'} 2022-09-03T06:23:48.921 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006bce0e4b9ae27b25-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.962 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:48.963 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:48.965 DEBUG:boto:Method: GET 2022-09-03T06:23:48.966 DEBUG:boto:Path: /dsgvcy-72/h 2022-09-03T06:23:48.967 DEBUG:boto:Data: 2022-09-03T06:23:48.968 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.968 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.969 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.969 DEBUG:boto:Params: {} 2022-09-03T06:23:48.970 DEBUG:boto:Token: None 2022-09-03T06:23:48.971 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/h 2022-09-03T06:23:48.972 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WMo6RIpapC1ZT7BLs/4Ym9zGw3E= 2022-09-03T06:23:48.972 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WMo6RIpapC1ZT7BLs/4Ym9zGw3E=', 'Content-Length': '0'} 2022-09-03T06:23:48.974 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000278dad256e8ee935-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.975 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:48.976 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:48.976 DEBUG:boto:path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:48.977 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:48.977 DEBUG:boto:Method: GET 2022-09-03T06:23:48.978 DEBUG:boto:Path: /dsgvcy-72/?prefix=h 2022-09-03T06:23:48.979 DEBUG:boto:Data: 2022-09-03T06:23:48.979 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.980 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:48.980 DEBUG:boto:Port: 8001 2022-09-03T06:23:48.981 DEBUG:boto:Params: {} 2022-09-03T06:23:48.982 DEBUG:boto:Token: None 2022-09-03T06:23:48.983 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/ 2022-09-03T06:23:48.984 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU= 2022-09-03T06:23:48.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU=', 'Content-Length': '0'} 2022-09-03T06:23:48.987 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ac58b129344deed2-006312f2f4-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:48.988 DEBUG:boto:b'dsgvcy-72h1000falseh2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:48.989 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:48.989 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:48.990 DEBUG:boto:path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:48.991 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:48.991 DEBUG:boto:Method: GET 2022-09-03T06:23:48.992 DEBUG:boto:Path: /dsgvcy-72/?prefix=h 2022-09-03T06:23:48.992 DEBUG:boto:Data: 2022-09-03T06:23:48.993 DEBUG:boto:Headers: {} 2022-09-03T06:23:48.993 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:48.994 DEBUG:boto:Port: 8000 2022-09-03T06:23:48.995 DEBUG:boto:Params: {} 2022-09-03T06:23:48.995 DEBUG:boto:Token: None 2022-09-03T06:23:48.996 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:48 GMT /dsgvcy-72/ 2022-09-03T06:23:48.996 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU= 2022-09-03T06:23:48.997 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K3Gj/oP0gmSOUS3rZz/lnJWkBIU=', 'Content-Length': '0'} 2022-09-03T06:23:49.000 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a7ad0a7d8465f9f5-006312f2f4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.001 DEBUG:boto:b'dsgvcy-72h1000falseh2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:49.002 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-72 zones={test-zone3, test-zone1} 2022-09-03T06:23:49.003 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-72 zones={test-zone3, test-zone2} 2022-09-03T06:23:49.003 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:49.004 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:49.005 DEBUG:boto:Method: HEAD 2022-09-03T06:23:49.005 DEBUG:boto:Path: /dsgvcy-72/ 2022-09-03T06:23:49.006 DEBUG:boto:Data: 2022-09-03T06:23:49.006 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.007 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.007 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.008 DEBUG:boto:Params: {} 2022-09-03T06:23:49.008 DEBUG:boto:Token: None 2022-09-03T06:23:49.009 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/ 2022-09-03T06:23:49.010 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Xdf5TpfgRs5LiGGdIynBmLBQreA= 2022-09-03T06:23:49.011 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Xdf5TpfgRs5LiGGdIynBmLBQreA=', 'Content-Length': '0'} 2022-09-03T06:23:49.014 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c09fe56f80a3b6cd-006312f2f5-15db-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.016 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:49.016 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:49.017 DEBUG:boto:Method: HEAD 2022-09-03T06:23:49.017 DEBUG:boto:Path: /dsgvcy-72/ 2022-09-03T06:23:49.018 DEBUG:boto:Data: 2022-09-03T06:23:49.019 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.019 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.020 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.020 DEBUG:boto:Params: {} 2022-09-03T06:23:49.021 DEBUG:boto:Token: None 2022-09-03T06:23:49.022 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/ 2022-09-03T06:23:49.022 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Xdf5TpfgRs5LiGGdIynBmLBQreA= 2022-09-03T06:23:49.023 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Xdf5TpfgRs5LiGGdIynBmLBQreA=', 'Content-Length': '0'} 2022-09-03T06:23:49.025 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000008ec69ae366ad8e41-006312f2f5-15d2-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.026 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:23:49.027 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:49.027 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:49.028 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:49.029 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:49.029 DEBUG:boto:Method: GET 2022-09-03T06:23:49.030 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:49.030 DEBUG:boto:Data: 2022-09-03T06:23:49.031 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.031 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.032 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.033 DEBUG:boto:Params: {} 2022-09-03T06:23:49.033 DEBUG:boto:Token: None 2022-09-03T06:23:49.034 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/?versions 2022-09-03T06:23:49.035 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xUNngrfPqo3Hv0XjeGUNRO+xMBU= 2022-09-03T06:23:49.035 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xUNngrfPqo3Hv0XjeGUNRO+xMBU=', 'Content-Length': '0'} 2022-09-03T06:23:49.038 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000024feb3be079ad82a-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.039 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:49.040 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:23:49.041 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:23:49.042 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:23:49.042 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:23:49.043 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:23:49.043 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:23:49.044 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:23:49.045 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:23:49.045 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:23:49.046 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:49.047 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:49.047 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:49.048 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:49.048 DEBUG:boto:Method: GET 2022-09-03T06:23:49.049 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:49.049 DEBUG:boto:Data: 2022-09-03T06:23:49.050 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.051 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.051 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.053 DEBUG:boto:Params: {} 2022-09-03T06:23:49.054 DEBUG:boto:Token: None 2022-09-03T06:23:49.054 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/?versions 2022-09-03T06:23:49.055 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xUNngrfPqo3Hv0XjeGUNRO+xMBU= 2022-09-03T06:23:49.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xUNngrfPqo3Hv0XjeGUNRO+xMBU=', 'Content-Length': '0'} 2022-09-03T06:23:49.058 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000088216685f1208be9-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.059 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:49.060 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:23:49.061 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:23:49.061 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:23:49.062 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:23:49.063 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:23:49.064 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:23:49.064 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:23:49.064 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:23:49.065 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:49.066 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:49.066 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:49.067 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:49.067 DEBUG:boto:Method: GET 2022-09-03T06:23:49.068 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:49.068 DEBUG:boto:Data: 2022-09-03T06:23:49.069 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.069 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.070 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.071 DEBUG:boto:Params: {} 2022-09-03T06:23:49.071 DEBUG:boto:Token: None 2022-09-03T06:23:49.072 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/?versions 2022-09-03T06:23:49.073 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xUNngrfPqo3Hv0XjeGUNRO+xMBU= 2022-09-03T06:23:49.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xUNngrfPqo3Hv0XjeGUNRO+xMBU=', 'Content-Length': '0'} 2022-09-03T06:23:49.076 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e9e3741400298bd3-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.077 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:49.078 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:49.079 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:49.079 DEBUG:boto:path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:49.080 DEBUG:boto:auth_path=/dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:49.080 DEBUG:boto:Method: GET 2022-09-03T06:23:49.081 DEBUG:boto:Path: /dsgvcy-72/?versions&max-keys=999 2022-09-03T06:23:49.081 DEBUG:boto:Data: 2022-09-03T06:23:49.082 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.083 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.083 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.084 DEBUG:boto:Params: {} 2022-09-03T06:23:49.084 DEBUG:boto:Token: None 2022-09-03T06:23:49.085 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/?versions 2022-09-03T06:23:49.085 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xUNngrfPqo3Hv0XjeGUNRO+xMBU= 2022-09-03T06:23:49.086 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xUNngrfPqo3Hv0XjeGUNRO+xMBU=', 'Content-Length': '0'} 2022-09-03T06:23:49.088 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000af43c7d658dd4878-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.089 DEBUG:boto:b'dsgvcy-72999falseanulltrue2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:49.091 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:23:49.091 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:49.092 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:49.092 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.093 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.093 DEBUG:boto:Method: GET 2022-09-03T06:23:49.094 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.095 DEBUG:boto:Data: 2022-09-03T06:23:49.095 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.096 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.096 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.097 DEBUG:boto:Params: {} 2022-09-03T06:23:49.097 DEBUG:boto:Token: None 2022-09-03T06:23:49.098 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.099 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XESqdm9aDpB1bgvBDyPRYoflWnk= 2022-09-03T06:23:49.099 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XESqdm9aDpB1bgvBDyPRYoflWnk=', 'Content-Length': '0'} 2022-09-03T06:23:49.101 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001620622477956465-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.142 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:49.143 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:49.144 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.144 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.145 DEBUG:boto:Method: GET 2022-09-03T06:23:49.146 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.146 DEBUG:boto:Data: 2022-09-03T06:23:49.147 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.148 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.148 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.149 DEBUG:boto:Params: {} 2022-09-03T06:23:49.150 DEBUG:boto:Token: None 2022-09-03T06:23:49.150 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.151 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XESqdm9aDpB1bgvBDyPRYoflWnk= 2022-09-03T06:23:49.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XESqdm9aDpB1bgvBDyPRYoflWnk=', 'Content-Length': '0'} 2022-09-03T06:23:49.154 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000067e40030340250ae-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.155 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:49.155 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:49.156 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.157 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.157 DEBUG:boto:Method: HEAD 2022-09-03T06:23:49.158 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.158 DEBUG:boto:Data: 2022-09-03T06:23:49.159 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.159 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.160 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.160 DEBUG:boto:Params: {} 2022-09-03T06:23:49.161 DEBUG:boto:Token: None 2022-09-03T06:23:49.161 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.162 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1kPW0sYplYP8sLNcil7hovbj61E= 2022-09-03T06:23:49.163 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1kPW0sYplYP8sLNcil7hovbj61E=', 'Content-Length': '0'} 2022-09-03T06:23:49.165 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c479b0be10a1aaf1-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.167 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:49.168 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:49.169 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.170 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.171 DEBUG:boto:Method: HEAD 2022-09-03T06:23:49.172 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.173 DEBUG:boto:Data: 2022-09-03T06:23:49.174 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.174 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.175 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.176 DEBUG:boto:Params: {} 2022-09-03T06:23:49.177 DEBUG:boto:Token: None 2022-09-03T06:23:49.179 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.180 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1kPW0sYplYP8sLNcil7hovbj61E= 2022-09-03T06:23:49.181 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1kPW0sYplYP8sLNcil7hovbj61E=', 'Content-Length': '0'} 2022-09-03T06:23:49.183 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002e4ee5eb427669b9-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.183 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:23:49.184 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:49.185 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:49.185 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.186 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.186 DEBUG:boto:Method: GET 2022-09-03T06:23:49.187 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.188 DEBUG:boto:Data: 2022-09-03T06:23:49.188 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.189 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.189 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.190 DEBUG:boto:Params: {} 2022-09-03T06:23:49.191 DEBUG:boto:Token: None 2022-09-03T06:23:49.191 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.192 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XESqdm9aDpB1bgvBDyPRYoflWnk= 2022-09-03T06:23:49.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XESqdm9aDpB1bgvBDyPRYoflWnk=', 'Content-Length': '0'} 2022-09-03T06:23:49.195 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000973fd146e08c3a35-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.196 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:49.196 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:49.197 DEBUG:boto:path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.197 DEBUG:boto:auth_path=/dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.198 DEBUG:boto:Method: GET 2022-09-03T06:23:49.198 DEBUG:boto:Path: /dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.199 DEBUG:boto:Data: 2022-09-03T06:23:49.199 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.200 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.201 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.201 DEBUG:boto:Params: {} 2022-09-03T06:23:49.202 DEBUG:boto:Token: None 2022-09-03T06:23:49.202 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/a?versionId=null 2022-09-03T06:23:49.203 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XESqdm9aDpB1bgvBDyPRYoflWnk= 2022-09-03T06:23:49.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XESqdm9aDpB1bgvBDyPRYoflWnk=', 'Content-Length': '0'} 2022-09-03T06:23:49.206 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f01e0bdfc8c6d72f-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.207 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:49.207 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:49.208 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:49.208 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:49.209 DEBUG:boto:Method: GET 2022-09-03T06:23:49.209 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:23:49.210 DEBUG:boto:Data: 2022-09-03T06:23:49.211 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.211 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.212 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.212 DEBUG:boto:Params: {} 2022-09-03T06:23:49.213 DEBUG:boto:Token: None 2022-09-03T06:23:49.213 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/ 2022-09-03T06:23:49.214 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg= 2022-09-03T06:23:49.215 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg=', 'Content-Length': '0'} 2022-09-03T06:23:49.218 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000260d6363d9bb4f96-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.218 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:49.219 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:49.220 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:49.221 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:49.221 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:49.222 DEBUG:boto:Method: GET 2022-09-03T06:23:49.223 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:23:49.223 DEBUG:boto:Data: 2022-09-03T06:23:49.224 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.224 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.225 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.226 DEBUG:boto:Params: {} 2022-09-03T06:23:49.226 DEBUG:boto:Token: None 2022-09-03T06:23:49.227 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/ 2022-09-03T06:23:49.228 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg= 2022-09-03T06:23:49.228 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg=', 'Content-Length': '0'} 2022-09-03T06:23:49.230 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b68fe1a466112f80-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.274 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:49.275 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:49.276 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:49.277 DEBUG:boto:Method: HEAD 2022-09-03T06:23:49.277 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:23:49.278 DEBUG:boto:Data: 2022-09-03T06:23:49.278 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.279 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.279 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.280 DEBUG:boto:Params: {} 2022-09-03T06:23:49.280 DEBUG:boto:Token: None 2022-09-03T06:23:49.281 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/a 2022-09-03T06:23:49.281 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:egQ1AMg7KoYWS33vWcdUhGuL5VY= 2022-09-03T06:23:49.282 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:egQ1AMg7KoYWS33vWcdUhGuL5VY=', 'Content-Length': '0'} 2022-09-03T06:23:49.284 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006144237ffbde6f03-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.285 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:49.286 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:49.287 DEBUG:boto:Method: HEAD 2022-09-03T06:23:49.287 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:23:49.288 DEBUG:boto:Data: 2022-09-03T06:23:49.288 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.289 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.289 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.290 DEBUG:boto:Params: {} 2022-09-03T06:23:49.290 DEBUG:boto:Token: None 2022-09-03T06:23:49.291 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/a 2022-09-03T06:23:49.291 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:egQ1AMg7KoYWS33vWcdUhGuL5VY= 2022-09-03T06:23:49.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:egQ1AMg7KoYWS33vWcdUhGuL5VY=', 'Content-Length': '0'} 2022-09-03T06:23:49.294 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000114ee17dde30fbd3-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.295 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:23:49.296 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:49.297 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:49.297 DEBUG:boto:Method: GET 2022-09-03T06:23:49.298 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:23:49.298 DEBUG:boto:Data: 2022-09-03T06:23:49.299 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.300 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.300 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.301 DEBUG:boto:Params: {} 2022-09-03T06:23:49.301 DEBUG:boto:Token: None 2022-09-03T06:23:49.302 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/a 2022-09-03T06:23:49.303 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mDjSS3+RuxWYVedPpfnPVEdj9uc= 2022-09-03T06:23:49.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mDjSS3+RuxWYVedPpfnPVEdj9uc=', 'Content-Length': '0'} 2022-09-03T06:23:49.307 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e085f152f138b1c8-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.308 DEBUG:boto:path=/dsgvcy-72/a 2022-09-03T06:23:49.309 DEBUG:boto:auth_path=/dsgvcy-72/a 2022-09-03T06:23:49.310 DEBUG:boto:Method: GET 2022-09-03T06:23:49.311 DEBUG:boto:Path: /dsgvcy-72/a 2022-09-03T06:23:49.312 DEBUG:boto:Data: 2022-09-03T06:23:49.313 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.313 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.314 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.315 DEBUG:boto:Params: {} 2022-09-03T06:23:49.316 DEBUG:boto:Token: None 2022-09-03T06:23:49.316 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/a 2022-09-03T06:23:49.318 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mDjSS3+RuxWYVedPpfnPVEdj9uc= 2022-09-03T06:23:49.319 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mDjSS3+RuxWYVedPpfnPVEdj9uc=', 'Content-Length': '0'} 2022-09-03T06:23:49.322 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000110b582e3b675a6d-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.324 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:49.325 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:49.326 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:49.326 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:49.327 DEBUG:boto:Method: GET 2022-09-03T06:23:49.327 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:23:49.328 DEBUG:boto:Data: 2022-09-03T06:23:49.329 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.329 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.330 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.330 DEBUG:boto:Params: {} 2022-09-03T06:23:49.331 DEBUG:boto:Token: None 2022-09-03T06:23:49.331 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/ 2022-09-03T06:23:49.332 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg= 2022-09-03T06:23:49.333 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg=', 'Content-Length': '0'} 2022-09-03T06:23:49.335 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e283c0e065cc1b3a-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.378 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:49.379 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:49.380 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:49.380 DEBUG:boto:path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:49.381 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=a 2022-09-03T06:23:49.381 DEBUG:boto:Method: GET 2022-09-03T06:23:49.382 DEBUG:boto:Path: /dsgvcy-72/?prefix=a 2022-09-03T06:23:49.382 DEBUG:boto:Data: 2022-09-03T06:23:49.383 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.383 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.384 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.384 DEBUG:boto:Params: {} 2022-09-03T06:23:49.385 DEBUG:boto:Token: None 2022-09-03T06:23:49.386 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/ 2022-09-03T06:23:49.386 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg= 2022-09-03T06:23:49.387 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg=', 'Content-Length': '0'} 2022-09-03T06:23:49.389 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007f20e19512f2805c-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.390 DEBUG:boto:b'dsgvcy-72a1000falsea2022-09-03T06:18:37.652Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:49.390 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:23:49.391 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:49.392 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:49.392 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.393 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.393 DEBUG:boto:Method: GET 2022-09-03T06:23:49.394 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.394 DEBUG:boto:Data: 2022-09-03T06:23:49.395 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.395 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.396 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.396 DEBUG:boto:Params: {} 2022-09-03T06:23:49.397 DEBUG:boto:Token: None 2022-09-03T06:23:49.398 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.398 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oQC9AC5R8W4KuWNBtWK1z2aiOuk= 2022-09-03T06:23:49.399 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oQC9AC5R8W4KuWNBtWK1z2aiOuk=', 'Content-Length': '0'} 2022-09-03T06:23:49.401 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000fe27dd87dde7c06-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.402 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:49.402 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:49.403 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.403 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.404 DEBUG:boto:Method: GET 2022-09-03T06:23:49.405 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.405 DEBUG:boto:Data: 2022-09-03T06:23:49.406 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.406 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.407 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.407 DEBUG:boto:Params: {} 2022-09-03T06:23:49.408 DEBUG:boto:Token: None 2022-09-03T06:23:49.408 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.409 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oQC9AC5R8W4KuWNBtWK1z2aiOuk= 2022-09-03T06:23:49.410 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oQC9AC5R8W4KuWNBtWK1z2aiOuk=', 'Content-Length': '0'} 2022-09-03T06:23:49.412 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b004d99d5cc541c4-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.413 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:49.413 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:49.414 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.414 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.415 DEBUG:boto:Method: HEAD 2022-09-03T06:23:49.415 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.416 DEBUG:boto:Data: 2022-09-03T06:23:49.416 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.417 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.417 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.418 DEBUG:boto:Params: {} 2022-09-03T06:23:49.418 DEBUG:boto:Token: None 2022-09-03T06:23:49.419 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.419 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/sboONB9i9U3+ObeC4vNeeCQi5o= 2022-09-03T06:23:49.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/sboONB9i9U3+ObeC4vNeeCQi5o=', 'Content-Length': '0'} 2022-09-03T06:23:49.422 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a429416490660a20-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.423 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:49.423 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:49.424 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.424 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.425 DEBUG:boto:Method: HEAD 2022-09-03T06:23:49.425 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.426 DEBUG:boto:Data: 2022-09-03T06:23:49.427 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.427 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.428 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.428 DEBUG:boto:Params: {} 2022-09-03T06:23:49.429 DEBUG:boto:Token: None 2022-09-03T06:23:49.429 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.430 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/sboONB9i9U3+ObeC4vNeeCQi5o= 2022-09-03T06:23:49.430 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/sboONB9i9U3+ObeC4vNeeCQi5o=', 'Content-Length': '0'} 2022-09-03T06:23:49.432 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b2695cf97d22238-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.433 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:23:49.434 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:49.434 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:49.435 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.435 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.436 DEBUG:boto:Method: GET 2022-09-03T06:23:49.436 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.437 DEBUG:boto:Data: 2022-09-03T06:23:49.437 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.438 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.439 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.439 DEBUG:boto:Params: {} 2022-09-03T06:23:49.440 DEBUG:boto:Token: None 2022-09-03T06:23:49.440 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.441 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oQC9AC5R8W4KuWNBtWK1z2aiOuk= 2022-09-03T06:23:49.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oQC9AC5R8W4KuWNBtWK1z2aiOuk=', 'Content-Length': '0'} 2022-09-03T06:23:49.444 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e87f0f7239b117b-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.445 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:49.446 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:49.447 DEBUG:boto:path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.448 DEBUG:boto:auth_path=/dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.448 DEBUG:boto:Method: GET 2022-09-03T06:23:49.449 DEBUG:boto:Path: /dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.449 DEBUG:boto:Data: 2022-09-03T06:23:49.450 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.451 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.451 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.452 DEBUG:boto:Params: {} 2022-09-03T06:23:49.452 DEBUG:boto:Token: None 2022-09-03T06:23:49.453 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/b?versionId=null 2022-09-03T06:23:49.454 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oQC9AC5R8W4KuWNBtWK1z2aiOuk= 2022-09-03T06:23:49.454 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oQC9AC5R8W4KuWNBtWK1z2aiOuk=', 'Content-Length': '0'} 2022-09-03T06:23:49.456 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000061bb89c7b7b896a8-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.457 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:49.458 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:49.459 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:49.459 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:49.460 DEBUG:boto:Method: GET 2022-09-03T06:23:49.460 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:23:49.461 DEBUG:boto:Data: 2022-09-03T06:23:49.461 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.462 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.462 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.463 DEBUG:boto:Params: {} 2022-09-03T06:23:49.463 DEBUG:boto:Token: None 2022-09-03T06:23:49.464 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/ 2022-09-03T06:23:49.465 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg= 2022-09-03T06:23:49.465 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg=', 'Content-Length': '0'} 2022-09-03T06:23:49.468 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005fefb50964ef9a3f-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.469 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:49.470 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:49.471 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:49.471 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:49.472 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:49.472 DEBUG:boto:Method: GET 2022-09-03T06:23:49.473 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:23:49.473 DEBUG:boto:Data: 2022-09-03T06:23:49.474 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.475 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.475 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.476 DEBUG:boto:Params: {} 2022-09-03T06:23:49.476 DEBUG:boto:Token: None 2022-09-03T06:23:49.477 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/ 2022-09-03T06:23:49.477 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg= 2022-09-03T06:23:49.478 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg=', 'Content-Length': '0'} 2022-09-03T06:23:49.480 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007fc160d0e272bab3-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.481 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:49.482 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:49.482 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:49.483 DEBUG:boto:Method: HEAD 2022-09-03T06:23:49.483 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:23:49.484 DEBUG:boto:Data: 2022-09-03T06:23:49.484 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.485 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.485 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.486 DEBUG:boto:Params: {} 2022-09-03T06:23:49.487 DEBUG:boto:Token: None 2022-09-03T06:23:49.487 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/b 2022-09-03T06:23:49.488 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qdRuFAhXY8Zea53BoC7BJ4vGMis= 2022-09-03T06:23:49.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qdRuFAhXY8Zea53BoC7BJ4vGMis=', 'Content-Length': '0'} 2022-09-03T06:23:49.490 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007bdc967f2ec886a6-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.491 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:49.492 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:49.492 DEBUG:boto:Method: HEAD 2022-09-03T06:23:49.493 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:23:49.493 DEBUG:boto:Data: 2022-09-03T06:23:49.494 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.494 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.495 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.495 DEBUG:boto:Params: {} 2022-09-03T06:23:49.496 DEBUG:boto:Token: None 2022-09-03T06:23:49.496 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/b 2022-09-03T06:23:49.497 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qdRuFAhXY8Zea53BoC7BJ4vGMis= 2022-09-03T06:23:49.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qdRuFAhXY8Zea53BoC7BJ4vGMis=', 'Content-Length': '0'} 2022-09-03T06:23:49.499 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7234fb2a200bdd9-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.500 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:23:49.500 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:49.501 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:49.502 DEBUG:boto:Method: GET 2022-09-03T06:23:49.502 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:23:49.503 DEBUG:boto:Data: 2022-09-03T06:23:49.503 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.504 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.504 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.505 DEBUG:boto:Params: {} 2022-09-03T06:23:49.505 DEBUG:boto:Token: None 2022-09-03T06:23:49.506 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/b 2022-09-03T06:23:49.507 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N0XdnOA9SReJas3i2WS+vl3Gm4s= 2022-09-03T06:23:49.507 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N0XdnOA9SReJas3i2WS+vl3Gm4s=', 'Content-Length': '0'} 2022-09-03T06:23:49.509 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005da00bc5f64b1fdd-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.550 DEBUG:boto:path=/dsgvcy-72/b 2022-09-03T06:23:49.551 DEBUG:boto:auth_path=/dsgvcy-72/b 2022-09-03T06:23:49.551 DEBUG:boto:Method: GET 2022-09-03T06:23:49.552 DEBUG:boto:Path: /dsgvcy-72/b 2022-09-03T06:23:49.552 DEBUG:boto:Data: 2022-09-03T06:23:49.553 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.554 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.554 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.555 DEBUG:boto:Params: {} 2022-09-03T06:23:49.555 DEBUG:boto:Token: None 2022-09-03T06:23:49.556 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/b 2022-09-03T06:23:49.556 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N0XdnOA9SReJas3i2WS+vl3Gm4s= 2022-09-03T06:23:49.557 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N0XdnOA9SReJas3i2WS+vl3Gm4s=', 'Content-Length': '0'} 2022-09-03T06:23:49.559 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000000f4c2d686745822-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.560 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:49.560 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:49.561 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:49.561 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:49.562 DEBUG:boto:Method: GET 2022-09-03T06:23:49.563 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:23:49.563 DEBUG:boto:Data: 2022-09-03T06:23:49.564 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.564 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.565 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.565 DEBUG:boto:Params: {} 2022-09-03T06:23:49.566 DEBUG:boto:Token: None 2022-09-03T06:23:49.566 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/ 2022-09-03T06:23:49.567 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg= 2022-09-03T06:23:49.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg=', 'Content-Length': '0'} 2022-09-03T06:23:49.570 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000909ed52014fb5b22-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.571 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:49.572 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:49.573 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:49.573 DEBUG:boto:path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:49.574 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=b 2022-09-03T06:23:49.574 DEBUG:boto:Method: GET 2022-09-03T06:23:49.575 DEBUG:boto:Path: /dsgvcy-72/?prefix=b 2022-09-03T06:23:49.575 DEBUG:boto:Data: 2022-09-03T06:23:49.576 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.576 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.577 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.577 DEBUG:boto:Params: {} 2022-09-03T06:23:49.578 DEBUG:boto:Token: None 2022-09-03T06:23:49.579 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/ 2022-09-03T06:23:49.579 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg= 2022-09-03T06:23:49.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg=', 'Content-Length': '0'} 2022-09-03T06:23:49.582 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000017ec0d1e34f2c081-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.583 DEBUG:boto:b'dsgvcy-72b1000falseb2022-09-03T06:18:37.711Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:49.584 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:23:49.584 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:49.585 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:49.586 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.586 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.587 DEBUG:boto:Method: GET 2022-09-03T06:23:49.587 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.588 DEBUG:boto:Data: 2022-09-03T06:23:49.588 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.589 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.589 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.590 DEBUG:boto:Params: {} 2022-09-03T06:23:49.590 DEBUG:boto:Token: None 2022-09-03T06:23:49.591 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.592 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p18ZjY+N03bWQ4xE++WvOxkjfRc= 2022-09-03T06:23:49.592 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p18ZjY+N03bWQ4xE++WvOxkjfRc=', 'Content-Length': '0'} 2022-09-03T06:23:49.594 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007e2f037f08630b08-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.595 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:49.596 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:49.596 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.597 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.597 DEBUG:boto:Method: GET 2022-09-03T06:23:49.598 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.598 DEBUG:boto:Data: 2022-09-03T06:23:49.599 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.599 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.600 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.601 DEBUG:boto:Params: {} 2022-09-03T06:23:49.601 DEBUG:boto:Token: None 2022-09-03T06:23:49.602 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.602 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p18ZjY+N03bWQ4xE++WvOxkjfRc= 2022-09-03T06:23:49.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p18ZjY+N03bWQ4xE++WvOxkjfRc=', 'Content-Length': '0'} 2022-09-03T06:23:49.605 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a7d81cfb9b4fcc8d-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.646 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:49.647 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:49.647 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.648 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.649 DEBUG:boto:Method: HEAD 2022-09-03T06:23:49.649 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.650 DEBUG:boto:Data: 2022-09-03T06:23:49.650 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.651 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.652 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.653 DEBUG:boto:Params: {} 2022-09-03T06:23:49.654 DEBUG:boto:Token: None 2022-09-03T06:23:49.654 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.655 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eddOXeZSN7OfMEgBhBUP5sOsWrY= 2022-09-03T06:23:49.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eddOXeZSN7OfMEgBhBUP5sOsWrY=', 'Content-Length': '0'} 2022-09-03T06:23:49.658 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004fb4a12f78b33e95-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.659 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:49.660 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:49.661 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.662 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.663 DEBUG:boto:Method: HEAD 2022-09-03T06:23:49.664 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.664 DEBUG:boto:Data: 2022-09-03T06:23:49.665 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.665 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.666 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.666 DEBUG:boto:Params: {} 2022-09-03T06:23:49.667 DEBUG:boto:Token: None 2022-09-03T06:23:49.668 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.668 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eddOXeZSN7OfMEgBhBUP5sOsWrY= 2022-09-03T06:23:49.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eddOXeZSN7OfMEgBhBUP5sOsWrY=', 'Content-Length': '0'} 2022-09-03T06:23:49.671 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000088e801a34be93a16-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.673 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:23:49.674 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:49.675 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:49.676 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.677 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.677 DEBUG:boto:Method: GET 2022-09-03T06:23:49.678 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.679 DEBUG:boto:Data: 2022-09-03T06:23:49.679 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.680 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.680 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.681 DEBUG:boto:Params: {} 2022-09-03T06:23:49.682 DEBUG:boto:Token: None 2022-09-03T06:23:49.682 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.683 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p18ZjY+N03bWQ4xE++WvOxkjfRc= 2022-09-03T06:23:49.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p18ZjY+N03bWQ4xE++WvOxkjfRc=', 'Content-Length': '0'} 2022-09-03T06:23:49.686 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000388419926f46191d-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.734 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:49.735 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:49.736 DEBUG:boto:path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.737 DEBUG:boto:auth_path=/dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.738 DEBUG:boto:Method: GET 2022-09-03T06:23:49.739 DEBUG:boto:Path: /dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.740 DEBUG:boto:Data: 2022-09-03T06:23:49.741 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.741 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.742 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.743 DEBUG:boto:Params: {} 2022-09-03T06:23:49.743 DEBUG:boto:Token: None 2022-09-03T06:23:49.744 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/c?versionId=null 2022-09-03T06:23:49.744 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p18ZjY+N03bWQ4xE++WvOxkjfRc= 2022-09-03T06:23:49.745 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p18ZjY+N03bWQ4xE++WvOxkjfRc=', 'Content-Length': '0'} 2022-09-03T06:23:49.748 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009687075f9bd0660c-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.748 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:49.749 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:49.750 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:49.750 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:49.751 DEBUG:boto:Method: GET 2022-09-03T06:23:49.751 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:23:49.752 DEBUG:boto:Data: 2022-09-03T06:23:49.752 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.753 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.753 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.754 DEBUG:boto:Params: {} 2022-09-03T06:23:49.755 DEBUG:boto:Token: None 2022-09-03T06:23:49.755 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/ 2022-09-03T06:23:49.756 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg= 2022-09-03T06:23:49.757 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg=', 'Content-Length': '0'} 2022-09-03T06:23:49.760 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005d71c6febbb96928-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.761 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:49.762 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:49.763 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:49.764 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:49.765 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:49.766 DEBUG:boto:Method: GET 2022-09-03T06:23:49.767 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:23:49.768 DEBUG:boto:Data: 2022-09-03T06:23:49.769 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.770 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.771 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.772 DEBUG:boto:Params: {} 2022-09-03T06:23:49.773 DEBUG:boto:Token: None 2022-09-03T06:23:49.774 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/ 2022-09-03T06:23:49.775 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg= 2022-09-03T06:23:49.775 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg=', 'Content-Length': '0'} 2022-09-03T06:23:49.778 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000003673fe5518aec55-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.779 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:49.780 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:49.781 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:49.782 DEBUG:boto:Method: HEAD 2022-09-03T06:23:49.782 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:23:49.783 DEBUG:boto:Data: 2022-09-03T06:23:49.783 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.784 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.784 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.785 DEBUG:boto:Params: {} 2022-09-03T06:23:49.786 DEBUG:boto:Token: None 2022-09-03T06:23:49.787 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/c 2022-09-03T06:23:49.788 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ntlKwX5wGWd/BfuKqvecL2e/U0g= 2022-09-03T06:23:49.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ntlKwX5wGWd/BfuKqvecL2e/U0g=', 'Content-Length': '0'} 2022-09-03T06:23:49.793 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c385e9a48b481018-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.794 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:49.795 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:49.796 DEBUG:boto:Method: HEAD 2022-09-03T06:23:49.796 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:23:49.797 DEBUG:boto:Data: 2022-09-03T06:23:49.798 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.799 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.800 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.800 DEBUG:boto:Params: {} 2022-09-03T06:23:49.801 DEBUG:boto:Token: None 2022-09-03T06:23:49.802 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/c 2022-09-03T06:23:49.803 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ntlKwX5wGWd/BfuKqvecL2e/U0g= 2022-09-03T06:23:49.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ntlKwX5wGWd/BfuKqvecL2e/U0g=', 'Content-Length': '0'} 2022-09-03T06:23:49.807 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001909e11dcb0dc801-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.808 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:23:49.809 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:49.810 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:49.811 DEBUG:boto:Method: GET 2022-09-03T06:23:49.812 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:23:49.813 DEBUG:boto:Data: 2022-09-03T06:23:49.814 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.815 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.816 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.816 DEBUG:boto:Params: {} 2022-09-03T06:23:49.817 DEBUG:boto:Token: None 2022-09-03T06:23:49.818 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/c 2022-09-03T06:23:49.818 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AL85juwDZ61I0SrrR0+oC8E/cZc= 2022-09-03T06:23:49.819 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AL85juwDZ61I0SrrR0+oC8E/cZc=', 'Content-Length': '0'} 2022-09-03T06:23:49.821 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004171e842b4c87472-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.822 DEBUG:boto:path=/dsgvcy-72/c 2022-09-03T06:23:49.823 DEBUG:boto:auth_path=/dsgvcy-72/c 2022-09-03T06:23:49.824 DEBUG:boto:Method: GET 2022-09-03T06:23:49.824 DEBUG:boto:Path: /dsgvcy-72/c 2022-09-03T06:23:49.825 DEBUG:boto:Data: 2022-09-03T06:23:49.825 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.826 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.826 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.827 DEBUG:boto:Params: {} 2022-09-03T06:23:49.829 DEBUG:boto:Token: None 2022-09-03T06:23:49.830 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/c 2022-09-03T06:23:49.831 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AL85juwDZ61I0SrrR0+oC8E/cZc= 2022-09-03T06:23:49.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AL85juwDZ61I0SrrR0+oC8E/cZc=', 'Content-Length': '0'} 2022-09-03T06:23:49.834 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dade0822625b7276-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.836 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:49.836 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:49.837 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:49.838 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:49.839 DEBUG:boto:Method: GET 2022-09-03T06:23:49.840 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:23:49.841 DEBUG:boto:Data: 2022-09-03T06:23:49.842 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.843 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.844 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.845 DEBUG:boto:Params: {} 2022-09-03T06:23:49.845 DEBUG:boto:Token: None 2022-09-03T06:23:49.846 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/ 2022-09-03T06:23:49.847 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg= 2022-09-03T06:23:49.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg=', 'Content-Length': '0'} 2022-09-03T06:23:49.851 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d185403b40666b1a-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.852 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:49.852 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:49.853 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:49.854 DEBUG:boto:path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:49.854 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=c 2022-09-03T06:23:49.855 DEBUG:boto:Method: GET 2022-09-03T06:23:49.856 DEBUG:boto:Path: /dsgvcy-72/?prefix=c 2022-09-03T06:23:49.856 DEBUG:boto:Data: 2022-09-03T06:23:49.857 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.857 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.858 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.858 DEBUG:boto:Params: {} 2022-09-03T06:23:49.859 DEBUG:boto:Token: None 2022-09-03T06:23:49.860 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/ 2022-09-03T06:23:49.860 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg= 2022-09-03T06:23:49.861 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3P2hnoilH7KFj3+iu2YkdGHsIwg=', 'Content-Length': '0'} 2022-09-03T06:23:49.863 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000087d3cadaa408c986-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.906 DEBUG:boto:b'dsgvcy-72c1000falsec2022-09-03T06:18:37.752Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:49.907 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:23:49.908 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:49.908 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:49.909 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.909 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.910 DEBUG:boto:Method: GET 2022-09-03T06:23:49.911 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.912 DEBUG:boto:Data: 2022-09-03T06:23:49.912 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.913 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.915 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.915 DEBUG:boto:Params: {} 2022-09-03T06:23:49.916 DEBUG:boto:Token: None 2022-09-03T06:23:49.917 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.918 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ur77uFN2skKJF+uV4Uz6XrKVtGI= 2022-09-03T06:23:49.919 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ur77uFN2skKJF+uV4Uz6XrKVtGI=', 'Content-Length': '0'} 2022-09-03T06:23:49.924 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f71ddedbb7b0cf35-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.925 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:49.926 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:49.926 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.927 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.928 DEBUG:boto:Method: GET 2022-09-03T06:23:49.929 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.930 DEBUG:boto:Data: 2022-09-03T06:23:49.931 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.932 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.933 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.934 DEBUG:boto:Params: {} 2022-09-03T06:23:49.935 DEBUG:boto:Token: None 2022-09-03T06:23:49.936 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.936 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ur77uFN2skKJF+uV4Uz6XrKVtGI= 2022-09-03T06:23:49.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ur77uFN2skKJF+uV4Uz6XrKVtGI=', 'Content-Length': '0'} 2022-09-03T06:23:49.940 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bf8029c8f070c0b8-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.941 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:49.942 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:49.943 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.944 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.944 DEBUG:boto:Method: HEAD 2022-09-03T06:23:49.945 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.946 DEBUG:boto:Data: 2022-09-03T06:23:49.947 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.948 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.949 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.949 DEBUG:boto:Params: {} 2022-09-03T06:23:49.951 DEBUG:boto:Token: None 2022-09-03T06:23:49.952 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.953 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sO8tEXy2HVNeFavDiqCbDo3kAII= 2022-09-03T06:23:49.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sO8tEXy2HVNeFavDiqCbDo3kAII=', 'Content-Length': '0'} 2022-09-03T06:23:49.957 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005692fafd3ceda52d-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.958 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:49.959 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:49.959 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.960 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.960 DEBUG:boto:Method: HEAD 2022-09-03T06:23:49.961 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.962 DEBUG:boto:Data: 2022-09-03T06:23:49.963 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.963 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.964 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.965 DEBUG:boto:Params: {} 2022-09-03T06:23:49.966 DEBUG:boto:Token: None 2022-09-03T06:23:49.967 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.968 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sO8tEXy2HVNeFavDiqCbDo3kAII= 2022-09-03T06:23:49.969 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sO8tEXy2HVNeFavDiqCbDo3kAII=', 'Content-Length': '0'} 2022-09-03T06:23:49.972 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000302f160785e2e963-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.974 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:23:49.975 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:49.975 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:49.976 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.976 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.977 DEBUG:boto:Method: GET 2022-09-03T06:23:49.978 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.978 DEBUG:boto:Data: 2022-09-03T06:23:49.979 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.979 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:49.980 DEBUG:boto:Port: 8001 2022-09-03T06:23:49.980 DEBUG:boto:Params: {} 2022-09-03T06:23:49.981 DEBUG:boto:Token: None 2022-09-03T06:23:49.982 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.983 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ur77uFN2skKJF+uV4Uz6XrKVtGI= 2022-09-03T06:23:49.983 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ur77uFN2skKJF+uV4Uz6XrKVtGI=', 'Content-Length': '0'} 2022-09-03T06:23:49.986 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000071ae450689b4043a-006312f2f5-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:49.987 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:49.987 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:49.988 DEBUG:boto:path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.988 DEBUG:boto:auth_path=/dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.989 DEBUG:boto:Method: GET 2022-09-03T06:23:49.989 DEBUG:boto:Path: /dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.990 DEBUG:boto:Data: 2022-09-03T06:23:49.990 DEBUG:boto:Headers: {} 2022-09-03T06:23:49.991 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:49.991 DEBUG:boto:Port: 8000 2022-09-03T06:23:49.992 DEBUG:boto:Params: {} 2022-09-03T06:23:49.993 DEBUG:boto:Token: None 2022-09-03T06:23:49.993 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:49 GMT /dsgvcy-72/d?versionId=null 2022-09-03T06:23:49.995 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ur77uFN2skKJF+uV4Uz6XrKVtGI= 2022-09-03T06:23:49.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ur77uFN2skKJF+uV4Uz6XrKVtGI=', 'Content-Length': '0'} 2022-09-03T06:23:49.999 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003cc6495408d7efea-006312f2f5-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.001 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:50.001 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:50.002 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:50.002 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:50.003 DEBUG:boto:Method: GET 2022-09-03T06:23:50.003 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:23:50.004 DEBUG:boto:Data: 2022-09-03T06:23:50.004 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.005 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.005 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.006 DEBUG:boto:Params: {} 2022-09-03T06:23:50.006 DEBUG:boto:Token: None 2022-09-03T06:23:50.007 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/ 2022-09-03T06:23:50.008 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo= 2022-09-03T06:23:50.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo=', 'Content-Length': '0'} 2022-09-03T06:23:50.021 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dc65d6625227d288-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.022 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:50.023 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:50.023 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:50.024 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:50.024 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:50.025 DEBUG:boto:Method: GET 2022-09-03T06:23:50.025 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:23:50.026 DEBUG:boto:Data: 2022-09-03T06:23:50.026 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.027 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.027 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.028 DEBUG:boto:Params: {} 2022-09-03T06:23:50.029 DEBUG:boto:Token: None 2022-09-03T06:23:50.029 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/ 2022-09-03T06:23:50.030 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo= 2022-09-03T06:23:50.030 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo=', 'Content-Length': '0'} 2022-09-03T06:23:50.033 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000372eb04bd7b695de-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.074 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:50.076 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:50.076 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:50.077 DEBUG:boto:Method: HEAD 2022-09-03T06:23:50.078 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:23:50.079 DEBUG:boto:Data: 2022-09-03T06:23:50.080 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.081 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.082 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.083 DEBUG:boto:Params: {} 2022-09-03T06:23:50.084 DEBUG:boto:Token: None 2022-09-03T06:23:50.085 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/d 2022-09-03T06:23:50.086 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jF1OmAbns3biMpyjoTxjEn/jCqs= 2022-09-03T06:23:50.087 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jF1OmAbns3biMpyjoTxjEn/jCqs=', 'Content-Length': '0'} 2022-09-03T06:23:50.090 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a0267ecffe0787f0-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.091 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:50.092 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:50.092 DEBUG:boto:Method: HEAD 2022-09-03T06:23:50.093 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:23:50.093 DEBUG:boto:Data: 2022-09-03T06:23:50.094 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.094 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.095 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.095 DEBUG:boto:Params: {} 2022-09-03T06:23:50.096 DEBUG:boto:Token: None 2022-09-03T06:23:50.097 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/d 2022-09-03T06:23:50.097 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jF1OmAbns3biMpyjoTxjEn/jCqs= 2022-09-03T06:23:50.098 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jF1OmAbns3biMpyjoTxjEn/jCqs=', 'Content-Length': '0'} 2022-09-03T06:23:50.100 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e4fafbb61f5ea19e-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.101 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:23:50.102 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:50.102 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:50.103 DEBUG:boto:Method: GET 2022-09-03T06:23:50.104 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:23:50.104 DEBUG:boto:Data: 2022-09-03T06:23:50.105 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.105 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.106 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.106 DEBUG:boto:Params: {} 2022-09-03T06:23:50.107 DEBUG:boto:Token: None 2022-09-03T06:23:50.108 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/d 2022-09-03T06:23:50.109 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v+X0SPLEa0KjzSp2OV4SVkFcs1k= 2022-09-03T06:23:50.110 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v+X0SPLEa0KjzSp2OV4SVkFcs1k=', 'Content-Length': '0'} 2022-09-03T06:23:50.113 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000580db5c577fbfc81-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.114 DEBUG:boto:path=/dsgvcy-72/d 2022-09-03T06:23:50.115 DEBUG:boto:auth_path=/dsgvcy-72/d 2022-09-03T06:23:50.115 DEBUG:boto:Method: GET 2022-09-03T06:23:50.116 DEBUG:boto:Path: /dsgvcy-72/d 2022-09-03T06:23:50.116 DEBUG:boto:Data: 2022-09-03T06:23:50.117 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.118 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.118 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.119 DEBUG:boto:Params: {} 2022-09-03T06:23:50.120 DEBUG:boto:Token: None 2022-09-03T06:23:50.121 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/d 2022-09-03T06:23:50.121 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v+X0SPLEa0KjzSp2OV4SVkFcs1k= 2022-09-03T06:23:50.122 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v+X0SPLEa0KjzSp2OV4SVkFcs1k=', 'Content-Length': '0'} 2022-09-03T06:23:50.124 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:18:37 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006c69810f538bf46d-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.125 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:50.126 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:50.126 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:50.127 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:50.127 DEBUG:boto:Method: GET 2022-09-03T06:23:50.128 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:23:50.128 DEBUG:boto:Data: 2022-09-03T06:23:50.129 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.129 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.130 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.130 DEBUG:boto:Params: {} 2022-09-03T06:23:50.131 DEBUG:boto:Token: None 2022-09-03T06:23:50.132 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/ 2022-09-03T06:23:50.132 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo= 2022-09-03T06:23:50.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo=', 'Content-Length': '0'} 2022-09-03T06:23:50.136 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000039ce2d1ab6ae9965-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.136 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:50.137 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:50.138 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:50.138 DEBUG:boto:path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:50.139 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=d 2022-09-03T06:23:50.139 DEBUG:boto:Method: GET 2022-09-03T06:23:50.140 DEBUG:boto:Path: /dsgvcy-72/?prefix=d 2022-09-03T06:23:50.141 DEBUG:boto:Data: 2022-09-03T06:23:50.142 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.143 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.143 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.144 DEBUG:boto:Params: {} 2022-09-03T06:23:50.144 DEBUG:boto:Token: None 2022-09-03T06:23:50.145 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/ 2022-09-03T06:23:50.145 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo= 2022-09-03T06:23:50.146 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo=', 'Content-Length': '0'} 2022-09-03T06:23:50.149 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000062bfb372243ee27c-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.149 DEBUG:boto:b'dsgvcy-72d1000falsed2022-09-03T06:18:37.795Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:50.150 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:23:50.151 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:50.151 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:50.152 DEBUG:boto:path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.152 DEBUG:boto:auth_path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.153 DEBUG:boto:Method: GET 2022-09-03T06:23:50.153 DEBUG:boto:Path: /dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.154 DEBUG:boto:Data: 2022-09-03T06:23:50.155 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.155 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.156 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.156 DEBUG:boto:Params: {} 2022-09-03T06:23:50.157 DEBUG:boto:Token: None 2022-09-03T06:23:50.157 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.158 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:moXo/MBwwn8XraSanvL+QWAsbaw= 2022-09-03T06:23:50.159 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:moXo/MBwwn8XraSanvL+QWAsbaw=', 'Content-Length': '0'} 2022-09-03T06:23:50.161 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007006775330efb224-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.206 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:50.207 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:50.207 DEBUG:boto:path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.208 DEBUG:boto:auth_path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.208 DEBUG:boto:Method: GET 2022-09-03T06:23:50.209 DEBUG:boto:Path: /dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.210 DEBUG:boto:Data: 2022-09-03T06:23:50.210 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.211 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.212 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.212 DEBUG:boto:Params: {} 2022-09-03T06:23:50.213 DEBUG:boto:Token: None 2022-09-03T06:23:50.213 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.215 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:moXo/MBwwn8XraSanvL+QWAsbaw= 2022-09-03T06:23:50.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:moXo/MBwwn8XraSanvL+QWAsbaw=', 'Content-Length': '0'} 2022-09-03T06:23:50.218 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f3f804b7f01f5192-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.220 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:50.221 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:50.222 DEBUG:boto:path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.223 DEBUG:boto:auth_path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.224 DEBUG:boto:Method: HEAD 2022-09-03T06:23:50.225 DEBUG:boto:Path: /dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.226 DEBUG:boto:Data: 2022-09-03T06:23:50.227 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.227 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.228 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.229 DEBUG:boto:Params: {} 2022-09-03T06:23:50.230 DEBUG:boto:Token: None 2022-09-03T06:23:50.231 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.232 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8ADpkS0owK+WeMio9abbz9lpPiE= 2022-09-03T06:23:50.233 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8ADpkS0owK+WeMio9abbz9lpPiE=', 'Content-Length': '0'} 2022-09-03T06:23:50.235 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001bc58f5b1f77a116-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.236 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:50.237 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:50.237 DEBUG:boto:path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.238 DEBUG:boto:auth_path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.238 DEBUG:boto:Method: HEAD 2022-09-03T06:23:50.239 DEBUG:boto:Path: /dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.239 DEBUG:boto:Data: 2022-09-03T06:23:50.240 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.240 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.241 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.242 DEBUG:boto:Params: {} 2022-09-03T06:23:50.242 DEBUG:boto:Token: None 2022-09-03T06:23:50.243 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.244 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8ADpkS0owK+WeMio9abbz9lpPiE= 2022-09-03T06:23:50.245 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8ADpkS0owK+WeMio9abbz9lpPiE=', 'Content-Length': '0'} 2022-09-03T06:23:50.247 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000019f7fe8068ebaeee-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.248 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:23:50.249 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:50.250 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:50.251 DEBUG:boto:path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.252 DEBUG:boto:auth_path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.253 DEBUG:boto:Method: GET 2022-09-03T06:23:50.253 DEBUG:boto:Path: /dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.254 DEBUG:boto:Data: 2022-09-03T06:23:50.255 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.255 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.256 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.256 DEBUG:boto:Params: {} 2022-09-03T06:23:50.257 DEBUG:boto:Token: None 2022-09-03T06:23:50.257 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.258 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:moXo/MBwwn8XraSanvL+QWAsbaw= 2022-09-03T06:23:50.259 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:moXo/MBwwn8XraSanvL+QWAsbaw=', 'Content-Length': '0'} 2022-09-03T06:23:50.261 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b9ee02fe0a57a353-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.263 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:50.264 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:50.265 DEBUG:boto:path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.266 DEBUG:boto:auth_path=/dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.267 DEBUG:boto:Method: GET 2022-09-03T06:23:50.268 DEBUG:boto:Path: /dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.269 DEBUG:boto:Data: 2022-09-03T06:23:50.271 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.272 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.273 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.274 DEBUG:boto:Params: {} 2022-09-03T06:23:50.275 DEBUG:boto:Token: None 2022-09-03T06:23:50.276 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/e?versionId=null 2022-09-03T06:23:50.277 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:moXo/MBwwn8XraSanvL+QWAsbaw= 2022-09-03T06:23:50.278 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:moXo/MBwwn8XraSanvL+QWAsbaw=', 'Content-Length': '0'} 2022-09-03T06:23:50.288 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000001be7b1f7a10f90b-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.290 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:50.290 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:50.291 DEBUG:boto:path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:50.292 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:50.293 DEBUG:boto:Method: GET 2022-09-03T06:23:50.294 DEBUG:boto:Path: /dsgvcy-72/?prefix=e 2022-09-03T06:23:50.296 DEBUG:boto:Data: 2022-09-03T06:23:50.296 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.297 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.298 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.298 DEBUG:boto:Params: {} 2022-09-03T06:23:50.299 DEBUG:boto:Token: None 2022-09-03T06:23:50.299 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/ 2022-09-03T06:23:50.300 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo= 2022-09-03T06:23:50.301 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo=', 'Content-Length': '0'} 2022-09-03T06:23:50.304 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000048ff157921ba7172-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.305 DEBUG:boto:b'dsgvcy-72e1000falsee2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:50.306 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:50.307 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:50.308 DEBUG:boto:path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:50.309 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:50.310 DEBUG:boto:Method: GET 2022-09-03T06:23:50.311 DEBUG:boto:Path: /dsgvcy-72/?prefix=e 2022-09-03T06:23:50.312 DEBUG:boto:Data: 2022-09-03T06:23:50.312 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.313 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.314 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.318 DEBUG:boto:Params: {} 2022-09-03T06:23:50.319 DEBUG:boto:Token: None 2022-09-03T06:23:50.320 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/ 2022-09-03T06:23:50.320 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo= 2022-09-03T06:23:50.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo=', 'Content-Length': '0'} 2022-09-03T06:23:50.323 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000075cf05ad7dc12dbc-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.370 DEBUG:boto:b'dsgvcy-72e1000falsee2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:50.372 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:50.374 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:50.376 DEBUG:boto:Method: HEAD 2022-09-03T06:23:50.377 DEBUG:boto:Path: /dsgvcy-72/e 2022-09-03T06:23:50.378 DEBUG:boto:Data: 2022-09-03T06:23:50.378 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.379 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.379 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.380 DEBUG:boto:Params: {} 2022-09-03T06:23:50.380 DEBUG:boto:Token: None 2022-09-03T06:23:50.381 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/e 2022-09-03T06:23:50.382 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OrpuHi3YVYGIpDvOIEVQzbpOE+o= 2022-09-03T06:23:50.382 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OrpuHi3YVYGIpDvOIEVQzbpOE+o=', 'Content-Length': '0'} 2022-09-03T06:23:50.384 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b5e0466d56e12be6-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.385 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:50.386 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:50.386 DEBUG:boto:Method: HEAD 2022-09-03T06:23:50.387 DEBUG:boto:Path: /dsgvcy-72/e 2022-09-03T06:23:50.387 DEBUG:boto:Data: 2022-09-03T06:23:50.388 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.389 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.389 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.390 DEBUG:boto:Params: {} 2022-09-03T06:23:50.391 DEBUG:boto:Token: None 2022-09-03T06:23:50.392 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/e 2022-09-03T06:23:50.393 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OrpuHi3YVYGIpDvOIEVQzbpOE+o= 2022-09-03T06:23:50.394 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OrpuHi3YVYGIpDvOIEVQzbpOE+o=', 'Content-Length': '0'} 2022-09-03T06:23:50.396 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000274391b8ed1a760c-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.396 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:23:50.397 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:50.398 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:50.398 DEBUG:boto:Method: GET 2022-09-03T06:23:50.399 DEBUG:boto:Path: /dsgvcy-72/e 2022-09-03T06:23:50.400 DEBUG:boto:Data: 2022-09-03T06:23:50.400 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.401 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.402 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.403 DEBUG:boto:Params: {} 2022-09-03T06:23:50.404 DEBUG:boto:Token: None 2022-09-03T06:23:50.404 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/e 2022-09-03T06:23:50.405 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q4joIaipxIfAzAR2bCax1O3FGUo= 2022-09-03T06:23:50.405 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q4joIaipxIfAzAR2bCax1O3FGUo=', 'Content-Length': '0'} 2022-09-03T06:23:50.408 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000210a52b755503315-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.409 DEBUG:boto:path=/dsgvcy-72/e 2022-09-03T06:23:50.409 DEBUG:boto:auth_path=/dsgvcy-72/e 2022-09-03T06:23:50.410 DEBUG:boto:Method: GET 2022-09-03T06:23:50.411 DEBUG:boto:Path: /dsgvcy-72/e 2022-09-03T06:23:50.412 DEBUG:boto:Data: 2022-09-03T06:23:50.413 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.414 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.414 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.415 DEBUG:boto:Params: {} 2022-09-03T06:23:50.416 DEBUG:boto:Token: None 2022-09-03T06:23:50.416 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/e 2022-09-03T06:23:50.417 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q4joIaipxIfAzAR2bCax1O3FGUo= 2022-09-03T06:23:50.417 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q4joIaipxIfAzAR2bCax1O3FGUo=', 'Content-Length': '0'} 2022-09-03T06:23:50.420 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f3107c5f326c5bea-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.420 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:50.421 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:50.421 DEBUG:boto:path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:50.422 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:50.423 DEBUG:boto:Method: GET 2022-09-03T06:23:50.423 DEBUG:boto:Path: /dsgvcy-72/?prefix=e 2022-09-03T06:23:50.424 DEBUG:boto:Data: 2022-09-03T06:23:50.424 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.425 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.425 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.426 DEBUG:boto:Params: {} 2022-09-03T06:23:50.426 DEBUG:boto:Token: None 2022-09-03T06:23:50.427 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/ 2022-09-03T06:23:50.427 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo= 2022-09-03T06:23:50.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo=', 'Content-Length': '0'} 2022-09-03T06:23:50.431 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000de8e3d699e8e736a-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.432 DEBUG:boto:b'dsgvcy-72e1000falsee2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:50.433 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:50.433 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:50.434 DEBUG:boto:path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:50.434 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=e 2022-09-03T06:23:50.435 DEBUG:boto:Method: GET 2022-09-03T06:23:50.435 DEBUG:boto:Path: /dsgvcy-72/?prefix=e 2022-09-03T06:23:50.436 DEBUG:boto:Data: 2022-09-03T06:23:50.436 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.437 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.437 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.438 DEBUG:boto:Params: {} 2022-09-03T06:23:50.439 DEBUG:boto:Token: None 2022-09-03T06:23:50.439 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/ 2022-09-03T06:23:50.440 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo= 2022-09-03T06:23:50.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo=', 'Content-Length': '0'} 2022-09-03T06:23:50.443 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007bf8cb2aa5e382d6-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.444 DEBUG:boto:b'dsgvcy-72e1000falsee2022-09-03T06:19:28.604Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:50.444 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:23:50.445 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:50.446 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:50.446 DEBUG:boto:path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.447 DEBUG:boto:auth_path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.447 DEBUG:boto:Method: GET 2022-09-03T06:23:50.448 DEBUG:boto:Path: /dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.448 DEBUG:boto:Data: 2022-09-03T06:23:50.449 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.449 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.450 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.450 DEBUG:boto:Params: {} 2022-09-03T06:23:50.451 DEBUG:boto:Token: None 2022-09-03T06:23:50.452 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.452 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ETUoMlJ+U8wVJdiH72/frme3LEs= 2022-09-03T06:23:50.453 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ETUoMlJ+U8wVJdiH72/frme3LEs=', 'Content-Length': '0'} 2022-09-03T06:23:50.455 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000849af9f1a2b527de-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.456 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:50.456 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:50.457 DEBUG:boto:path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.458 DEBUG:boto:auth_path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.458 DEBUG:boto:Method: GET 2022-09-03T06:23:50.459 DEBUG:boto:Path: /dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.459 DEBUG:boto:Data: 2022-09-03T06:23:50.460 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.460 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.461 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.461 DEBUG:boto:Params: {} 2022-09-03T06:23:50.462 DEBUG:boto:Token: None 2022-09-03T06:23:50.463 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.463 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ETUoMlJ+U8wVJdiH72/frme3LEs= 2022-09-03T06:23:50.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ETUoMlJ+U8wVJdiH72/frme3LEs=', 'Content-Length': '0'} 2022-09-03T06:23:50.466 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000035f37862aefc3041-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.467 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:50.468 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:50.469 DEBUG:boto:path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.469 DEBUG:boto:auth_path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.470 DEBUG:boto:Method: HEAD 2022-09-03T06:23:50.471 DEBUG:boto:Path: /dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.471 DEBUG:boto:Data: 2022-09-03T06:23:50.472 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.472 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.473 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.473 DEBUG:boto:Params: {} 2022-09-03T06:23:50.474 DEBUG:boto:Token: None 2022-09-03T06:23:50.475 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.476 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gVC1b2iECwpf3xjAt4ZvNi1+0wc= 2022-09-03T06:23:50.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gVC1b2iECwpf3xjAt4ZvNi1+0wc=', 'Content-Length': '0'} 2022-09-03T06:23:50.478 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ea13606593f5272-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.479 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:50.479 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:50.480 DEBUG:boto:path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.480 DEBUG:boto:auth_path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.481 DEBUG:boto:Method: HEAD 2022-09-03T06:23:50.481 DEBUG:boto:Path: /dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.482 DEBUG:boto:Data: 2022-09-03T06:23:50.482 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.483 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.484 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.484 DEBUG:boto:Params: {} 2022-09-03T06:23:50.485 DEBUG:boto:Token: None 2022-09-03T06:23:50.486 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.487 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gVC1b2iECwpf3xjAt4ZvNi1+0wc= 2022-09-03T06:23:50.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gVC1b2iECwpf3xjAt4ZvNi1+0wc=', 'Content-Length': '0'} 2022-09-03T06:23:50.490 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003333c79acc1cbb9f-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.491 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:23:50.491 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:50.492 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:50.492 DEBUG:boto:path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.493 DEBUG:boto:auth_path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.493 DEBUG:boto:Method: GET 2022-09-03T06:23:50.494 DEBUG:boto:Path: /dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.494 DEBUG:boto:Data: 2022-09-03T06:23:50.495 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.495 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.496 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.496 DEBUG:boto:Params: {} 2022-09-03T06:23:50.497 DEBUG:boto:Token: None 2022-09-03T06:23:50.498 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.498 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ETUoMlJ+U8wVJdiH72/frme3LEs= 2022-09-03T06:23:50.499 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ETUoMlJ+U8wVJdiH72/frme3LEs=', 'Content-Length': '0'} 2022-09-03T06:23:50.502 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dcd6ebaed0be2099-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.546 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:50.548 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:50.549 DEBUG:boto:path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.549 DEBUG:boto:auth_path=/dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.550 DEBUG:boto:Method: GET 2022-09-03T06:23:50.553 DEBUG:boto:Path: /dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.554 DEBUG:boto:Data: 2022-09-03T06:23:50.555 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.556 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.557 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.558 DEBUG:boto:Params: {} 2022-09-03T06:23:50.560 DEBUG:boto:Token: None 2022-09-03T06:23:50.560 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/f?versionId=null 2022-09-03T06:23:50.561 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ETUoMlJ+U8wVJdiH72/frme3LEs= 2022-09-03T06:23:50.562 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ETUoMlJ+U8wVJdiH72/frme3LEs=', 'Content-Length': '0'} 2022-09-03T06:23:50.565 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000067ca935629f54b90-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.566 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:50.567 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:50.568 DEBUG:boto:path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:50.569 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:50.570 DEBUG:boto:Method: GET 2022-09-03T06:23:50.570 DEBUG:boto:Path: /dsgvcy-72/?prefix=f 2022-09-03T06:23:50.571 DEBUG:boto:Data: 2022-09-03T06:23:50.571 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.572 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.573 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.574 DEBUG:boto:Params: {} 2022-09-03T06:23:50.575 DEBUG:boto:Token: None 2022-09-03T06:23:50.576 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/ 2022-09-03T06:23:50.577 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo= 2022-09-03T06:23:50.578 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo=', 'Content-Length': '0'} 2022-09-03T06:23:50.585 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a47cbeb661a8f884-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.586 DEBUG:boto:b'dsgvcy-72f1000falsef2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:50.587 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:50.588 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:50.588 DEBUG:boto:path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:50.589 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:50.590 DEBUG:boto:Method: GET 2022-09-03T06:23:50.590 DEBUG:boto:Path: /dsgvcy-72/?prefix=f 2022-09-03T06:23:50.591 DEBUG:boto:Data: 2022-09-03T06:23:50.591 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.592 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.593 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.593 DEBUG:boto:Params: {} 2022-09-03T06:23:50.594 DEBUG:boto:Token: None 2022-09-03T06:23:50.594 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/ 2022-09-03T06:23:50.595 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo= 2022-09-03T06:23:50.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo=', 'Content-Length': '0'} 2022-09-03T06:23:50.598 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000003dff574b957d6cc-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.599 DEBUG:boto:b'dsgvcy-72f1000falsef2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:50.600 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:50.601 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:50.602 DEBUG:boto:Method: HEAD 2022-09-03T06:23:50.603 DEBUG:boto:Path: /dsgvcy-72/f 2022-09-03T06:23:50.603 DEBUG:boto:Data: 2022-09-03T06:23:50.604 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.605 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.606 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.607 DEBUG:boto:Params: {} 2022-09-03T06:23:50.608 DEBUG:boto:Token: None 2022-09-03T06:23:50.609 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/f 2022-09-03T06:23:50.610 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DzoCo7XNff+86wH89a4sesDHq7A= 2022-09-03T06:23:50.611 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DzoCo7XNff+86wH89a4sesDHq7A=', 'Content-Length': '0'} 2022-09-03T06:23:50.630 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f4a99d030463b8f6-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.631 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:50.632 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:50.633 DEBUG:boto:Method: HEAD 2022-09-03T06:23:50.634 DEBUG:boto:Path: /dsgvcy-72/f 2022-09-03T06:23:50.635 DEBUG:boto:Data: 2022-09-03T06:23:50.636 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.637 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.637 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.638 DEBUG:boto:Params: {} 2022-09-03T06:23:50.638 DEBUG:boto:Token: None 2022-09-03T06:23:50.639 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/f 2022-09-03T06:23:50.640 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DzoCo7XNff+86wH89a4sesDHq7A= 2022-09-03T06:23:50.640 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DzoCo7XNff+86wH89a4sesDHq7A=', 'Content-Length': '0'} 2022-09-03T06:23:50.643 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ff09b32d2f81eb64-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.644 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:23:50.645 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:50.646 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:50.647 DEBUG:boto:Method: GET 2022-09-03T06:23:50.647 DEBUG:boto:Path: /dsgvcy-72/f 2022-09-03T06:23:50.648 DEBUG:boto:Data: 2022-09-03T06:23:50.648 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.649 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.650 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.651 DEBUG:boto:Params: {} 2022-09-03T06:23:50.651 DEBUG:boto:Token: None 2022-09-03T06:23:50.652 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/f 2022-09-03T06:23:50.653 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:huEcTNJnH4XDWXS2GCpMSQMUzDs= 2022-09-03T06:23:50.653 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:huEcTNJnH4XDWXS2GCpMSQMUzDs=', 'Content-Length': '0'} 2022-09-03T06:23:50.656 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000b4ab1107c32ed80-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.702 DEBUG:boto:path=/dsgvcy-72/f 2022-09-03T06:23:50.703 DEBUG:boto:auth_path=/dsgvcy-72/f 2022-09-03T06:23:50.703 DEBUG:boto:Method: GET 2022-09-03T06:23:50.704 DEBUG:boto:Path: /dsgvcy-72/f 2022-09-03T06:23:50.705 DEBUG:boto:Data: 2022-09-03T06:23:50.705 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.706 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.706 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.707 DEBUG:boto:Params: {} 2022-09-03T06:23:50.707 DEBUG:boto:Token: None 2022-09-03T06:23:50.708 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/f 2022-09-03T06:23:50.709 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:huEcTNJnH4XDWXS2GCpMSQMUzDs= 2022-09-03T06:23:50.710 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:huEcTNJnH4XDWXS2GCpMSQMUzDs=', 'Content-Length': '0'} 2022-09-03T06:23:50.712 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d369198fe4797cf7-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.713 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:50.714 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:50.714 DEBUG:boto:path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:50.715 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:50.715 DEBUG:boto:Method: GET 2022-09-03T06:23:50.716 DEBUG:boto:Path: /dsgvcy-72/?prefix=f 2022-09-03T06:23:50.717 DEBUG:boto:Data: 2022-09-03T06:23:50.717 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.718 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.719 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.719 DEBUG:boto:Params: {} 2022-09-03T06:23:50.720 DEBUG:boto:Token: None 2022-09-03T06:23:50.720 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/ 2022-09-03T06:23:50.721 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo= 2022-09-03T06:23:50.722 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo=', 'Content-Length': '0'} 2022-09-03T06:23:50.725 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000590e9761b5b04a46-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.726 DEBUG:boto:b'dsgvcy-72f1000falsef2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:50.727 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:50.727 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:50.728 DEBUG:boto:path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:50.728 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=f 2022-09-03T06:23:50.729 DEBUG:boto:Method: GET 2022-09-03T06:23:50.729 DEBUG:boto:Path: /dsgvcy-72/?prefix=f 2022-09-03T06:23:50.730 DEBUG:boto:Data: 2022-09-03T06:23:50.731 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.731 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.732 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.732 DEBUG:boto:Params: {} 2022-09-03T06:23:50.733 DEBUG:boto:Token: None 2022-09-03T06:23:50.733 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/ 2022-09-03T06:23:50.734 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo= 2022-09-03T06:23:50.735 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo=', 'Content-Length': '0'} 2022-09-03T06:23:50.738 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002e158940577660b6-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.740 DEBUG:boto:b'dsgvcy-72f1000falsef2022-09-03T06:19:28.631Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:50.742 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:23:50.743 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:50.744 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:50.745 DEBUG:boto:path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.745 DEBUG:boto:auth_path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.746 DEBUG:boto:Method: GET 2022-09-03T06:23:50.747 DEBUG:boto:Path: /dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.747 DEBUG:boto:Data: 2022-09-03T06:23:50.748 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.749 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.750 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.751 DEBUG:boto:Params: {} 2022-09-03T06:23:50.752 DEBUG:boto:Token: None 2022-09-03T06:23:50.753 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.754 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OxFyI2bOTHnIkswPCJgNYYZ6YqE= 2022-09-03T06:23:50.755 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OxFyI2bOTHnIkswPCJgNYYZ6YqE=', 'Content-Length': '0'} 2022-09-03T06:23:50.761 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001691a1228b8d9445-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.762 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:50.762 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:50.763 DEBUG:boto:path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.764 DEBUG:boto:auth_path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.764 DEBUG:boto:Method: GET 2022-09-03T06:23:50.765 DEBUG:boto:Path: /dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.765 DEBUG:boto:Data: 2022-09-03T06:23:50.766 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.767 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.768 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.769 DEBUG:boto:Params: {} 2022-09-03T06:23:50.770 DEBUG:boto:Token: None 2022-09-03T06:23:50.771 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.772 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OxFyI2bOTHnIkswPCJgNYYZ6YqE= 2022-09-03T06:23:50.773 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OxFyI2bOTHnIkswPCJgNYYZ6YqE=', 'Content-Length': '0'} 2022-09-03T06:23:50.776 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000051dd0093c12f4159-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.778 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:50.779 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:50.780 DEBUG:boto:path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.781 DEBUG:boto:auth_path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.782 DEBUG:boto:Method: HEAD 2022-09-03T06:23:50.783 DEBUG:boto:Path: /dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.784 DEBUG:boto:Data: 2022-09-03T06:23:50.785 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.786 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.786 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.787 DEBUG:boto:Params: {} 2022-09-03T06:23:50.788 DEBUG:boto:Token: None 2022-09-03T06:23:50.789 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.790 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xoM1gRcg988DTrgjyh9g2Vp8Fn4= 2022-09-03T06:23:50.791 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xoM1gRcg988DTrgjyh9g2Vp8Fn4=', 'Content-Length': '0'} 2022-09-03T06:23:50.794 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005452c2fa39b3e900-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.795 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:50.796 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:50.797 DEBUG:boto:path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.797 DEBUG:boto:auth_path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.798 DEBUG:boto:Method: HEAD 2022-09-03T06:23:50.799 DEBUG:boto:Path: /dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.799 DEBUG:boto:Data: 2022-09-03T06:23:50.800 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.800 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.801 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.802 DEBUG:boto:Params: {} 2022-09-03T06:23:50.803 DEBUG:boto:Token: None 2022-09-03T06:23:50.804 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.805 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xoM1gRcg988DTrgjyh9g2Vp8Fn4= 2022-09-03T06:23:50.806 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xoM1gRcg988DTrgjyh9g2Vp8Fn4=', 'Content-Length': '0'} 2022-09-03T06:23:50.808 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000023c216cfa7c76e74-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.810 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:23:50.810 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:50.812 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:50.812 DEBUG:boto:path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.813 DEBUG:boto:auth_path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.813 DEBUG:boto:Method: GET 2022-09-03T06:23:50.814 DEBUG:boto:Path: /dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.814 DEBUG:boto:Data: 2022-09-03T06:23:50.815 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.815 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.816 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.816 DEBUG:boto:Params: {} 2022-09-03T06:23:50.817 DEBUG:boto:Token: None 2022-09-03T06:23:50.818 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.819 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OxFyI2bOTHnIkswPCJgNYYZ6YqE= 2022-09-03T06:23:50.820 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OxFyI2bOTHnIkswPCJgNYYZ6YqE=', 'Content-Length': '0'} 2022-09-03T06:23:50.823 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e6f1ffde97efcc2-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.824 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:50.824 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:50.825 DEBUG:boto:path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.826 DEBUG:boto:auth_path=/dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.826 DEBUG:boto:Method: GET 2022-09-03T06:23:50.827 DEBUG:boto:Path: /dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.827 DEBUG:boto:Data: 2022-09-03T06:23:50.828 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.828 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.829 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.829 DEBUG:boto:Params: {} 2022-09-03T06:23:50.830 DEBUG:boto:Token: None 2022-09-03T06:23:50.831 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/g?versionId=null 2022-09-03T06:23:50.831 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OxFyI2bOTHnIkswPCJgNYYZ6YqE= 2022-09-03T06:23:50.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OxFyI2bOTHnIkswPCJgNYYZ6YqE=', 'Content-Length': '0'} 2022-09-03T06:23:50.834 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000083f15a9f2525e1d7-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.835 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:50.836 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:50.836 DEBUG:boto:path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:50.837 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:50.837 DEBUG:boto:Method: GET 2022-09-03T06:23:50.838 DEBUG:boto:Path: /dsgvcy-72/?prefix=g 2022-09-03T06:23:50.838 DEBUG:boto:Data: 2022-09-03T06:23:50.839 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.839 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.840 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.841 DEBUG:boto:Params: {} 2022-09-03T06:23:50.841 DEBUG:boto:Token: None 2022-09-03T06:23:50.842 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/ 2022-09-03T06:23:50.842 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo= 2022-09-03T06:23:50.843 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo=', 'Content-Length': '0'} 2022-09-03T06:23:50.846 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000009bbc82ce7b72fa4-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.847 DEBUG:boto:b'dsgvcy-72g1000falseg2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:50.847 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:50.848 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:50.849 DEBUG:boto:path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:50.849 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:50.850 DEBUG:boto:Method: GET 2022-09-03T06:23:50.850 DEBUG:boto:Path: /dsgvcy-72/?prefix=g 2022-09-03T06:23:50.851 DEBUG:boto:Data: 2022-09-03T06:23:50.851 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.852 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.852 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.853 DEBUG:boto:Params: {} 2022-09-03T06:23:50.854 DEBUG:boto:Token: None 2022-09-03T06:23:50.855 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/ 2022-09-03T06:23:50.855 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo= 2022-09-03T06:23:50.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo=', 'Content-Length': '0'} 2022-09-03T06:23:50.858 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d61d950387d149a0-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.902 DEBUG:boto:b'dsgvcy-72g1000falseg2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:50.903 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:50.904 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:50.904 DEBUG:boto:Method: HEAD 2022-09-03T06:23:50.905 DEBUG:boto:Path: /dsgvcy-72/g 2022-09-03T06:23:50.906 DEBUG:boto:Data: 2022-09-03T06:23:50.907 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.907 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.908 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.908 DEBUG:boto:Params: {} 2022-09-03T06:23:50.909 DEBUG:boto:Token: None 2022-09-03T06:23:50.909 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/g 2022-09-03T06:23:50.911 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CQNhPyM7Q6A7ABwK7ZLCJVWAEJY= 2022-09-03T06:23:50.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CQNhPyM7Q6A7ABwK7ZLCJVWAEJY=', 'Content-Length': '0'} 2022-09-03T06:23:50.914 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000015ea23c525fdc65-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.915 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:50.916 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:50.917 DEBUG:boto:Method: HEAD 2022-09-03T06:23:50.917 DEBUG:boto:Path: /dsgvcy-72/g 2022-09-03T06:23:50.918 DEBUG:boto:Data: 2022-09-03T06:23:50.918 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.919 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.919 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.920 DEBUG:boto:Params: {} 2022-09-03T06:23:50.921 DEBUG:boto:Token: None 2022-09-03T06:23:50.921 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/g 2022-09-03T06:23:50.922 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CQNhPyM7Q6A7ABwK7ZLCJVWAEJY= 2022-09-03T06:23:50.923 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CQNhPyM7Q6A7ABwK7ZLCJVWAEJY=', 'Content-Length': '0'} 2022-09-03T06:23:50.930 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000bd70b10f6d333b5-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.932 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:23:50.932 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:50.933 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:50.933 DEBUG:boto:Method: GET 2022-09-03T06:23:50.934 DEBUG:boto:Path: /dsgvcy-72/g 2022-09-03T06:23:50.935 DEBUG:boto:Data: 2022-09-03T06:23:50.935 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.936 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.936 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.937 DEBUG:boto:Params: {} 2022-09-03T06:23:50.938 DEBUG:boto:Token: None 2022-09-03T06:23:50.938 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/g 2022-09-03T06:23:50.939 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SPjaxssq4InzZLJ9CvXgr+GiM+w= 2022-09-03T06:23:50.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SPjaxssq4InzZLJ9CvXgr+GiM+w=', 'Content-Length': '0'} 2022-09-03T06:23:50.943 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e0b1875bcb98ab8f-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.944 DEBUG:boto:path=/dsgvcy-72/g 2022-09-03T06:23:50.945 DEBUG:boto:auth_path=/dsgvcy-72/g 2022-09-03T06:23:50.947 DEBUG:boto:Method: GET 2022-09-03T06:23:50.948 DEBUG:boto:Path: /dsgvcy-72/g 2022-09-03T06:23:50.949 DEBUG:boto:Data: 2022-09-03T06:23:50.950 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.951 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:50.952 DEBUG:boto:Port: 8000 2022-09-03T06:23:50.953 DEBUG:boto:Params: {} 2022-09-03T06:23:50.953 DEBUG:boto:Token: None 2022-09-03T06:23:50.954 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/g 2022-09-03T06:23:50.955 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SPjaxssq4InzZLJ9CvXgr+GiM+w= 2022-09-03T06:23:50.957 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SPjaxssq4InzZLJ9CvXgr+GiM+w=', 'Content-Length': '0'} 2022-09-03T06:23:50.963 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ecdb9d3bd8f4ec6f-006312f2f6-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:50.963 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:50.964 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:50.964 DEBUG:boto:path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:50.965 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:50.965 DEBUG:boto:Method: GET 2022-09-03T06:23:50.966 DEBUG:boto:Path: /dsgvcy-72/?prefix=g 2022-09-03T06:23:50.968 DEBUG:boto:Data: 2022-09-03T06:23:50.968 DEBUG:boto:Headers: {} 2022-09-03T06:23:50.969 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:50.970 DEBUG:boto:Port: 8001 2022-09-03T06:23:50.970 DEBUG:boto:Params: {} 2022-09-03T06:23:50.971 DEBUG:boto:Token: None 2022-09-03T06:23:50.972 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:50 GMT /dsgvcy-72/ 2022-09-03T06:23:50.972 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo= 2022-09-03T06:23:50.973 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0cIB9C57wOOgUrQxg9HLI+TGOjo=', 'Content-Length': '0'} 2022-09-03T06:23:50.983 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008b52122c45081f2a-006312f2f6-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:51.022 DEBUG:boto:b'dsgvcy-72g1000falseg2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:51.023 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:51.024 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:51.024 DEBUG:boto:path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:51.025 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=g 2022-09-03T06:23:51.026 DEBUG:boto:Method: GET 2022-09-03T06:23:51.026 DEBUG:boto:Path: /dsgvcy-72/?prefix=g 2022-09-03T06:23:51.027 DEBUG:boto:Data: 2022-09-03T06:23:51.027 DEBUG:boto:Headers: {} 2022-09-03T06:23:51.028 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:51.029 DEBUG:boto:Port: 8000 2022-09-03T06:23:51.029 DEBUG:boto:Params: {} 2022-09-03T06:23:51.030 DEBUG:boto:Token: None 2022-09-03T06:23:51.031 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:51 GMT /dsgvcy-72/ 2022-09-03T06:23:51.031 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dKshYB2D2l/cXQ1WFBnxvrbnP4M= 2022-09-03T06:23:51.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dKshYB2D2l/cXQ1WFBnxvrbnP4M=', 'Content-Length': '0'} 2022-09-03T06:23:51.036 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004ca13794260c21fa-006312f2f7-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:51.042 DEBUG:boto:b'dsgvcy-72g1000falseg2022-09-03T06:19:28.662Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:51.043 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:23:51.043 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:51.044 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:51.044 DEBUG:boto:path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.045 DEBUG:boto:auth_path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.046 DEBUG:boto:Method: GET 2022-09-03T06:23:51.046 DEBUG:boto:Path: /dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.047 DEBUG:boto:Data: 2022-09-03T06:23:51.048 DEBUG:boto:Headers: {} 2022-09-03T06:23:51.049 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:51.050 DEBUG:boto:Port: 8001 2022-09-03T06:23:51.051 DEBUG:boto:Params: {} 2022-09-03T06:23:51.051 DEBUG:boto:Token: None 2022-09-03T06:23:51.052 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:51 GMT /dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.053 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6lnkKe40Z3lrF/vsq3TPVcbgC5E= 2022-09-03T06:23:51.053 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6lnkKe40Z3lrF/vsq3TPVcbgC5E=', 'Content-Length': '0'} 2022-09-03T06:23:51.055 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000945b6fa42c91951f-006312f2f7-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:51.057 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:51.057 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:51.058 DEBUG:boto:path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.058 DEBUG:boto:auth_path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.059 DEBUG:boto:Method: GET 2022-09-03T06:23:51.060 DEBUG:boto:Path: /dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.060 DEBUG:boto:Data: 2022-09-03T06:23:51.061 DEBUG:boto:Headers: {} 2022-09-03T06:23:51.061 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:51.062 DEBUG:boto:Port: 8000 2022-09-03T06:23:51.063 DEBUG:boto:Params: {} 2022-09-03T06:23:51.063 DEBUG:boto:Token: None 2022-09-03T06:23:51.064 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:51 GMT /dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.065 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6lnkKe40Z3lrF/vsq3TPVcbgC5E= 2022-09-03T06:23:51.065 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6lnkKe40Z3lrF/vsq3TPVcbgC5E=', 'Content-Length': '0'} 2022-09-03T06:23:51.068 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000357ae02a563b45f0-006312f2f7-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:51.110 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:51.111 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:51.112 DEBUG:boto:path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.114 DEBUG:boto:auth_path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.115 DEBUG:boto:Method: HEAD 2022-09-03T06:23:51.115 DEBUG:boto:Path: /dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.117 DEBUG:boto:Data: 2022-09-03T06:23:51.117 DEBUG:boto:Headers: {} 2022-09-03T06:23:51.118 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:51.118 DEBUG:boto:Port: 8001 2022-09-03T06:23:51.119 DEBUG:boto:Params: {} 2022-09-03T06:23:51.119 DEBUG:boto:Token: None 2022-09-03T06:23:51.120 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:51 GMT /dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.120 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8CiVwaK9dTIwvq7GJX81w3aZ9Vk= 2022-09-03T06:23:51.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8CiVwaK9dTIwvq7GJX81w3aZ9Vk=', 'Content-Length': '0'} 2022-09-03T06:23:51.123 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f94c201c04989cf-006312f2f7-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:51.124 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:51.124 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:51.125 DEBUG:boto:path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.125 DEBUG:boto:auth_path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.126 DEBUG:boto:Method: HEAD 2022-09-03T06:23:51.126 DEBUG:boto:Path: /dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.127 DEBUG:boto:Data: 2022-09-03T06:23:51.127 DEBUG:boto:Headers: {} 2022-09-03T06:23:51.128 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:51.128 DEBUG:boto:Port: 8000 2022-09-03T06:23:51.129 DEBUG:boto:Params: {} 2022-09-03T06:23:51.129 DEBUG:boto:Token: None 2022-09-03T06:23:51.130 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:51 GMT /dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.130 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8CiVwaK9dTIwvq7GJX81w3aZ9Vk= 2022-09-03T06:23:51.131 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8CiVwaK9dTIwvq7GJX81w3aZ9Vk=', 'Content-Length': '0'} 2022-09-03T06:23:51.134 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bc954d4ae9b8874e-006312f2f7-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:51.134 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:23:51.135 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:51.136 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:51.136 DEBUG:boto:path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.137 DEBUG:boto:auth_path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.138 DEBUG:boto:Method: GET 2022-09-03T06:23:51.138 DEBUG:boto:Path: /dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.139 DEBUG:boto:Data: 2022-09-03T06:23:51.139 DEBUG:boto:Headers: {} 2022-09-03T06:23:51.140 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:51.140 DEBUG:boto:Port: 8001 2022-09-03T06:23:51.141 DEBUG:boto:Params: {} 2022-09-03T06:23:51.141 DEBUG:boto:Token: None 2022-09-03T06:23:51.142 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:51 GMT /dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.143 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6lnkKe40Z3lrF/vsq3TPVcbgC5E= 2022-09-03T06:23:51.143 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6lnkKe40Z3lrF/vsq3TPVcbgC5E=', 'Content-Length': '0'} 2022-09-03T06:23:51.145 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000095464fee213c4974-006312f2f7-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:51.146 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:51.147 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:51.147 DEBUG:boto:path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.148 DEBUG:boto:auth_path=/dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.148 DEBUG:boto:Method: GET 2022-09-03T06:23:51.149 DEBUG:boto:Path: /dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.149 DEBUG:boto:Data: 2022-09-03T06:23:51.150 DEBUG:boto:Headers: {} 2022-09-03T06:23:51.151 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:51.151 DEBUG:boto:Port: 8000 2022-09-03T06:23:51.152 DEBUG:boto:Params: {} 2022-09-03T06:23:51.153 DEBUG:boto:Token: None 2022-09-03T06:23:51.153 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:51 GMT /dsgvcy-72/h?versionId=null 2022-09-03T06:23:51.154 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6lnkKe40Z3lrF/vsq3TPVcbgC5E= 2022-09-03T06:23:51.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6lnkKe40Z3lrF/vsq3TPVcbgC5E=', 'Content-Length': '0'} 2022-09-03T06:23:51.158 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd31f7cf42439fbc-006312f2f7-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:51.159 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:51.159 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:51.160 DEBUG:boto:path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:51.160 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:51.161 DEBUG:boto:Method: GET 2022-09-03T06:23:51.161 DEBUG:boto:Path: /dsgvcy-72/?prefix=h 2022-09-03T06:23:51.162 DEBUG:boto:Data: 2022-09-03T06:23:51.162 DEBUG:boto:Headers: {} 2022-09-03T06:23:51.163 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:51.163 DEBUG:boto:Port: 8001 2022-09-03T06:23:51.164 DEBUG:boto:Params: {} 2022-09-03T06:23:51.164 DEBUG:boto:Token: None 2022-09-03T06:23:51.165 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:51 GMT /dsgvcy-72/ 2022-09-03T06:23:51.166 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dKshYB2D2l/cXQ1WFBnxvrbnP4M= 2022-09-03T06:23:51.166 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dKshYB2D2l/cXQ1WFBnxvrbnP4M=', 'Content-Length': '0'} 2022-09-03T06:23:51.169 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000080f6aaf600cb9cef-006312f2f7-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:51.170 DEBUG:boto:b'dsgvcy-72h1000falseh2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:51.171 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:51.171 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:51.172 DEBUG:boto:path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:51.172 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:51.173 DEBUG:boto:Method: GET 2022-09-03T06:23:51.174 DEBUG:boto:Path: /dsgvcy-72/?prefix=h 2022-09-03T06:23:51.174 DEBUG:boto:Data: 2022-09-03T06:23:51.175 DEBUG:boto:Headers: {} 2022-09-03T06:23:51.175 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:51.176 DEBUG:boto:Port: 8000 2022-09-03T06:23:51.176 DEBUG:boto:Params: {} 2022-09-03T06:23:51.177 DEBUG:boto:Token: None 2022-09-03T06:23:51.178 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:51 GMT /dsgvcy-72/ 2022-09-03T06:23:51.178 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dKshYB2D2l/cXQ1WFBnxvrbnP4M= 2022-09-03T06:23:51.179 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dKshYB2D2l/cXQ1WFBnxvrbnP4M=', 'Content-Length': '0'} 2022-09-03T06:23:51.181 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000011f10122833f4f86-006312f2f7-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:51.182 DEBUG:boto:b'dsgvcy-72h1000falseh2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:51.183 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:51.184 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:51.184 DEBUG:boto:Method: HEAD 2022-09-03T06:23:51.185 DEBUG:boto:Path: /dsgvcy-72/h 2022-09-03T06:23:51.185 DEBUG:boto:Data: 2022-09-03T06:23:51.186 DEBUG:boto:Headers: {} 2022-09-03T06:23:51.187 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:51.187 DEBUG:boto:Port: 8001 2022-09-03T06:23:51.188 DEBUG:boto:Params: {} 2022-09-03T06:23:51.188 DEBUG:boto:Token: None 2022-09-03T06:23:51.189 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:51 GMT /dsgvcy-72/h 2022-09-03T06:23:51.189 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4FVqu+g4X7agZzpsImEqlkUREzg= 2022-09-03T06:23:51.190 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4FVqu+g4X7agZzpsImEqlkUREzg=', 'Content-Length': '0'} 2022-09-03T06:23:51.192 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f04df90cdcff6d62-006312f2f7-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:51.194 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:51.195 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:51.196 DEBUG:boto:Method: HEAD 2022-09-03T06:23:51.197 DEBUG:boto:Path: /dsgvcy-72/h 2022-09-03T06:23:51.197 DEBUG:boto:Data: 2022-09-03T06:23:51.198 DEBUG:boto:Headers: {} 2022-09-03T06:23:51.199 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:51.200 DEBUG:boto:Port: 8000 2022-09-03T06:23:51.200 DEBUG:boto:Params: {} 2022-09-03T06:23:51.201 DEBUG:boto:Token: None 2022-09-03T06:23:51.201 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:23:51 GMT /dsgvcy-72/h 2022-09-03T06:23:51.202 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4FVqu+g4X7agZzpsImEqlkUREzg= 2022-09-03T06:23:51.203 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4FVqu+g4X7agZzpsImEqlkUREzg=', 'Content-Length': '0'} 2022-09-03T06:23:51.205 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000004f36525fb1df5b6-006312f2f7-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:51.206 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:23:51.206 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:51.207 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:51.207 DEBUG:boto:Method: GET 2022-09-03T06:23:51.208 DEBUG:boto:Path: /dsgvcy-72/h 2022-09-03T06:23:51.209 DEBUG:boto:Data: 2022-09-03T06:23:51.209 DEBUG:boto:Headers: {} 2022-09-03T06:23:51.210 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:51.210 DEBUG:boto:Port: 8001 2022-09-03T06:23:51.211 DEBUG:boto:Params: {} 2022-09-03T06:23:51.212 DEBUG:boto:Token: None 2022-09-03T06:23:51.212 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:51 GMT /dsgvcy-72/h 2022-09-03T06:23:51.213 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7Kewa0KMTwaJ+6sgywdD5yCVG2s= 2022-09-03T06:23:51.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7Kewa0KMTwaJ+6sgywdD5yCVG2s=', 'Content-Length': '0'} 2022-09-03T06:23:51.215 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005d9bc251437746ac-006312f2f7-15db-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:51.216 DEBUG:boto:path=/dsgvcy-72/h 2022-09-03T06:23:51.217 DEBUG:boto:auth_path=/dsgvcy-72/h 2022-09-03T06:23:51.217 DEBUG:boto:Method: GET 2022-09-03T06:23:51.218 DEBUG:boto:Path: /dsgvcy-72/h 2022-09-03T06:23:51.218 DEBUG:boto:Data: 2022-09-03T06:23:51.219 DEBUG:boto:Headers: {} 2022-09-03T06:23:51.220 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:51.220 DEBUG:boto:Port: 8000 2022-09-03T06:23:51.221 DEBUG:boto:Params: {} 2022-09-03T06:23:51.221 DEBUG:boto:Token: None 2022-09-03T06:23:51.222 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:51 GMT /dsgvcy-72/h 2022-09-03T06:23:51.223 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7Kewa0KMTwaJ+6sgywdD5yCVG2s= 2022-09-03T06:23:51.223 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7Kewa0KMTwaJ+6sgywdD5yCVG2s=', 'Content-Length': '0'} 2022-09-03T06:23:51.226 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:19:28 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007251a43452a2122e-006312f2f7-15d2-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:23:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:51.227 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:51.228 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:51.229 DEBUG:boto:path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:51.230 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:51.232 DEBUG:boto:Method: GET 2022-09-03T06:23:51.233 DEBUG:boto:Path: /dsgvcy-72/?prefix=h 2022-09-03T06:23:51.234 DEBUG:boto:Data: 2022-09-03T06:23:51.235 DEBUG:boto:Headers: {} 2022-09-03T06:23:51.236 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:23:51.242 DEBUG:boto:Port: 8001 2022-09-03T06:23:51.242 DEBUG:boto:Params: {} 2022-09-03T06:23:51.243 DEBUG:boto:Token: None 2022-09-03T06:23:51.245 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:51 GMT /dsgvcy-72/ 2022-09-03T06:23:51.247 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dKshYB2D2l/cXQ1WFBnxvrbnP4M= 2022-09-03T06:23:51.249 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dKshYB2D2l/cXQ1WFBnxvrbnP4M=', 'Content-Length': '0'} 2022-09-03T06:23:51.253 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000278e367134fb4f5-006312f2f7-15db-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:51.254 DEBUG:boto:b'dsgvcy-72h1000falseh2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:51.255 DEBUG:boto:path=/dsgvcy-72/ 2022-09-03T06:23:51.256 DEBUG:boto:auth_path=/dsgvcy-72/ 2022-09-03T06:23:51.256 DEBUG:boto:path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:51.257 DEBUG:boto:auth_path=/dsgvcy-72/?prefix=h 2022-09-03T06:23:51.258 DEBUG:boto:Method: GET 2022-09-03T06:23:51.258 DEBUG:boto:Path: /dsgvcy-72/?prefix=h 2022-09-03T06:23:51.259 DEBUG:boto:Data: 2022-09-03T06:23:51.259 DEBUG:boto:Headers: {} 2022-09-03T06:23:51.260 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:23:51.260 DEBUG:boto:Port: 8000 2022-09-03T06:23:51.261 DEBUG:boto:Params: {} 2022-09-03T06:23:51.262 DEBUG:boto:Token: None 2022-09-03T06:23:51.262 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:23:51 GMT /dsgvcy-72/ 2022-09-03T06:23:51.263 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dKshYB2D2l/cXQ1WFBnxvrbnP4M= 2022-09-03T06:23:51.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dKshYB2D2l/cXQ1WFBnxvrbnP4M=', 'Content-Length': '0'} 2022-09-03T06:23:51.266 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f8d8fc0777b4820e-006312f2f7-15d2-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:23:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:51.267 DEBUG:boto:b'dsgvcy-72h1000falseh2022-09-03T06:19:28.753Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:23:51.268 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-72 zones={test-zone3, test-zone2} 2022-09-03T06:23:51.269 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_reshard_incremental ... ok 2022-09-03T06:23:51.272 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:23:51.273 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:23:51.273 DEBUG:boto:Method: PUT 2022-09-03T06:23:51.274 DEBUG:boto:Path: /dsgvcy-73/ 2022-09-03T06:23:51.274 DEBUG:boto:Data: 2022-09-03T06:23:51.275 DEBUG:boto:Headers: {} 2022-09-03T06:23:51.275 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:23:51.276 DEBUG:boto:Port: 8000 2022-09-03T06:23:51.277 DEBUG:boto:Params: {} 2022-09-03T06:23:51.278 DEBUG:boto:Token: None 2022-09-03T06:23:51.279 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 06:23:51 GMT /dsgvcy-73/ 2022-09-03T06:23:51.290 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pfqNszI/Aoa4FSoltNrZd/Cjt3k= 2022-09-03T06:23:51.291 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:23:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pfqNszI/Aoa4FSoltNrZd/Cjt3k=', 'Content-Length': '0'} 2022-09-03T06:23:51.312 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000e57bd4713384f930-006312f2f7-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:23:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:23:51.313 DEBUG:rgw_multi.tests:created bucket=dsgvcy-73 2022-09-03T06:23:51.314 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:23:51.315 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:23:51.316 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:23:51.373 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:23:51.373 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:23:51.403 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T06:23:51.437 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T06:23:51.437 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:23:51.438 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:23:51.438 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:23:51.438 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:23:51.439 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:23:51.439 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:23:51.439 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:23:51.439 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:23:51.440 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:23:51.440 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662186231.308222_476.1", 2022-09-03T06:23:51.440 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:23:51.308222Z" 2022-09-03T06:23:51.440 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:23:51.441 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:23:51.441 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:23:51.441 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:23:51.441 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:23:51.442 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T06:23:51.444 DEBUG:rgw_multi.tests:master meta markers={0: '', 1: '', 2: '1_1662186231.308222_476.1', 3: ''} 2022-09-03T06:23:51.444 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T06:23:51.445 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:23:51.445 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:23:51.446 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:23:51.502 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:23:51.502 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:23:51.533 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:23:51.531+0000 7f8528cc2600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:23:51.534 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:23:51.532+0000 7f8528cc2600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:23:51.548 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:23:51.548 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:23:51.548 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:23:51.549 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:23:51.549 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:23:51.549 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:23:51.549 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:23:51.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:23:51.550 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:23:51.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:23:51.550 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:23:51.551 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:23:51.551 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:23:51.551 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:23:51.552 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:23:51.552 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:23:51.552 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:23:51.552 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:23:51.553 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:23:51.553 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:23:51.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:23:51.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:23:51.554 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:23:51.554 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:23:51.554 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:23:51.555 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:23:51.555 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:23:51.555 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:23:51.555 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:23:51.556 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:23:51.556 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:23:51.556 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:23:51.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:23:51.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:23:51.557 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:23:51.557 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:23:51.557 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:23:51.558 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662186231.308222_476.1", 2022-09-03T06:23:51.558 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:23:51.558 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:23:51.559 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:23:51.559 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:23:51.308222Z", 2022-09-03T06:23:51.559 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:23:51.559 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:23:51.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:23:51.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:23:51.560 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:23:51.560 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:23:51.561 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:23:51.561 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:23:51.561 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:23:51.561 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:23:51.562 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:23:51.562 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:23:51.562 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:23:51.563 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:23:51.563 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:23:51.563 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:23:51.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:23:51.564 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:23:51.564 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:23:51.564 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:23:51.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:23:51.565 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:23:51Z" 2022-09-03T06:23:51.565 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:23:51.566 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662186231.308222_476.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:23:51.308222Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:23:51Z" } 2022-09-03T06:23:51.567 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662186231.308222_476.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:23:51.308222Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:23:51.568 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662186231.308222_476.1', 3: ''} 2022-09-03T06:23:51.568 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662186231.308222_476.1', 3: ''} 2022-09-03T06:23:51.569 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T06:23:51.569 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T06:23:51.570 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:23:51.570 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:23:51.571 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:23:51.620 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:23:51.620 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:23:51.664 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:23:51.662+0000 7f23c8fbb600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:23:51.665 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:23:51.663+0000 7f23c8fbb600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:23:51.683 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T06:23:51.683 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T06:23:51.684 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T06:23:51.684 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T06:23:51.684 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T06:23:51.684 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:23:51.685 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:23:51.685 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:23:51.685 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T06:23:51.685 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:23:51.686 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T06:23:51.686 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:23:51.686 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:23:51.686 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:23:51.687 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:23:51.687 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T06:23:51.687 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:23:51.687 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:23:51.688 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:23:51.688 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:23:51.688 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:23:51.688 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:23:51.689 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T06:23:51.689 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:23:51.689 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:23:51.690 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:23:51.690 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:23:51.690 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:23:51.690 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:23:51.691 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T06:23:51.691 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:23:51.691 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:23:51.692 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:23:51.692 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:23:51.692 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T06:23:51.692 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:23:51.693 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:23:51.693 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662186231.308222_476.1", 2022-09-03T06:23:51.693 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:23:51.693 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:23:51.694 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:23:51.694 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:23:51.308222Z", 2022-09-03T06:23:51.694 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:23:51.694 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:23:51.695 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:23:51.695 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:23:51.695 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T06:23:51.695 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:23:51.696 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:23:51.696 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:23:51.696 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:23:51.696 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:23:51.697 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:23:51.697 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:23:51.697 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:23:51.697 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:23:51.697 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:23:51.698 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:23:51.698 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:23:51.698 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T06:23:51.699 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T06:23:51.699 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T06:23:51.699 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:23:51.699 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T06:23:51Z" 2022-09-03T06:23:51.700 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T06:23:51.701 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662186231.308222_476.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:23:51.308222Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:23:51Z" } 2022-09-03T06:23:51.701 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662186231.308222_476.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:23:51.308222Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:23:51.702 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662186231.308222_476.1', 3: ''} 2022-09-03T06:23:51.702 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662186231.308222_476.1', 3: ''} 2022-09-03T06:23:51.703 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T06:23:51.703 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T06:23:51.704 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:23:51.704 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:23:51.705 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:23:51.757 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:23:51.757 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:23:51.785 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:23:51.783+0000 7f532ff05600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:23:51.787 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:23:51.784+0000 7f532ff05600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:23:51.798 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:23:51.798 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:23:51.799 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:23:51.799 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:23:51.799 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:23:51.800 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:23:51.800 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:23:51.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:23:51.800 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:23:51.801 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:23:51.801 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:23:51.802 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:23:51.802 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:23:51.803 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:23:51.803 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:23:51.804 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:23:51.804 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:23:51.805 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:23:51.805 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:23:51.806 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:23:51.806 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:23:51.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:23:51.808 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:23:51.808 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:23:51.808 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:23:51.809 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:23:51.809 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:23:51.809 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:23:51.809 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:23:51.810 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:23:51.810 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:23:51.810 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:23:51.810 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:23:51.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:23:51.811 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:23:51.811 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:23:51.811 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:23:51.812 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662185968.546784_463.1", 2022-09-03T06:23:51.812 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:23:51.812 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:23:51.812 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:23:51.813 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:19:28.546784Z", 2022-09-03T06:23:51.813 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:23:51.813 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:23:51.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:23:51.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:23:51.814 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:23:51.814 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:23:51.814 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:23:51.815 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:23:51.815 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:23:51.815 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:23:51.815 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:23:51.816 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:23:51.816 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:23:51.816 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:23:51.816 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:23:51.817 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:23:51.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:23:51.817 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:23:51.817 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:23:51.818 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:23:51.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:23:51.818 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:23:51Z" 2022-09-03T06:23:51.818 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:23:51.820 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662185968.546784_463.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:19:28.546784Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:23:51Z" } 2022-09-03T06:23:51.820 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662185968.546784_463.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:19:28.546784Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:23:51.821 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662186231.308222_476.1', 3: ''} 2022-09-03T06:23:51.821 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662185968.546784_463.1', 3: ''} 2022-09-03T06:23:51.822 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662186231.308222_476.1 target=1_1662185968.546784_463.1 2022-09-03T06:23:56.823 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:23:56.824 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:23:56.825 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:23:56.882 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:23:56.882 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:23:56.911 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:23:56.909+0000 7f0659c52600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:23:56.912 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:23:56.910+0000 7f0659c52600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:23:56.925 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:23:56.926 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:23:56.926 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:23:56.926 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:23:56.927 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:23:56.927 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:23:56.927 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:23:56.928 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:23:56.928 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:23:56.928 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:23:56.928 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:23:56.929 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:23:56.929 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:23:56.929 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:23:56.929 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:23:56.930 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:23:56.930 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:23:56.930 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:23:56.931 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:23:56.931 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:23:56.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:23:56.932 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:23:56.932 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:23:56.933 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:23:56.933 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:23:56.933 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:23:56.934 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:23:56.935 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:23:56.935 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:23:56.935 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:23:56.936 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:23:56.936 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:23:56.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:23:56.937 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:23:56.938 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:23:56.938 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:23:56.939 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:23:56.939 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662186231.308222_476.1", 2022-09-03T06:23:56.940 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:23:56.940 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:23:56.941 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:23:56.941 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:23:51.308222Z", 2022-09-03T06:23:56.943 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:23:56.943 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:23:56.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:23:56.944 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:23:56.944 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:23:56.944 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:23:56.945 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:23:56.945 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:23:56.945 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:23:56.946 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:23:56.946 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:23:56.946 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:23:56.947 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:23:56.947 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:23:56.947 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:23:56.948 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:23:56.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:23:56.948 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:23:56.949 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:23:56.949 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:23:56.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:23:56.950 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:23:56Z" 2022-09-03T06:23:56.950 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:23:56.953 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662186231.308222_476.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:23:51.308222Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:23:56Z" } 2022-09-03T06:23:56.954 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662186231.308222_476.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:23:51.308222Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:23:56.955 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662186231.308222_476.1', 3: ''} 2022-09-03T06:23:56.956 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662186231.308222_476.1', 3: ''} 2022-09-03T06:23:56.957 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T06:23:56.958 DEBUG:tasks.rgw.c2.client.0:waiting for process to exit 2022-09-03T06:23:56.959 INFO:teuthology.orchestra.run:waiting for 300 2022-09-03T06:23:56.961 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:23:56.957+0000 7f6c45d77700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term radosgw --rgw-frontends beast port=8000 -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm (PID: 112111) UID: 0 2022-09-03T06:23:56.962 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:23:56.958+0000 7f6c49fe2600 -1 shutting down 2022-09-03T06:23:59.535 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:23:59.534+0000 7fabf9180700 -1 ERROR: failed operation: ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-5)ERROR: collect() returned error (ret=-5)ERROR: collect() returned error (ret=-5)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16) 2022-09-03T06:23:59.579 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:23:59.577+0000 7fabf9180700 -1 ERROR: failed operation: ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-5)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-5)ERROR: collect() returned error (ret=-5) 2022-09-03T06:23:59.661 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:23:59.659+0000 7fabf9180700 -1 ERROR: failed operation: ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-5)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-5)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-5)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-5)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16) 2022-09-03T06:24:01.613 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~0s 2022-09-03T06:24:03.063 INFO:tasks.rgw.c2.client.0:Stopped 2022-09-03T06:24:03.065 DEBUG:tasks.rgw.c1.client.1:waiting for process to exit 2022-09-03T06:24:03.065 INFO:teuthology.orchestra.run:waiting for 300 2022-09-03T06:24:03.066 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:24:03.064+0000 7fac17bc4700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term radosgw --rgw-frontends beast port=8001 -n client.1 --cluster c1 -k /etc/ceph/c1.client.1.keyring --log-file /var/log/ceph/rgw.c1.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.1.sock --foreground --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm (PID: 127824) UID: 0 2022-09-03T06:24:03.067 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:24:03.064+0000 7fac1be2f600 -1 shutting down 2022-09-03T06:24:06.664 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c1.rgw.client.1 is failed for ~0s 2022-09-03T06:24:07.221 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 has been restored 2022-09-03T06:24:09.168 INFO:tasks.rgw.c1.client.1:Stopped 2022-09-03T06:24:09.168 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:24:09.169 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:24:09.169 DEBUG:boto:Method: HEAD 2022-09-03T06:24:09.170 DEBUG:boto:Path: /dsgvcy-73/ 2022-09-03T06:24:09.171 DEBUG:boto:Data: 2022-09-03T06:24:09.171 DEBUG:boto:Headers: {} 2022-09-03T06:24:09.172 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:24:09.172 DEBUG:boto:Port: 8000 2022-09-03T06:24:09.172 DEBUG:boto:Params: {} 2022-09-03T06:24:09.173 DEBUG:boto:Token: None 2022-09-03T06:24:09.174 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:24:09 GMT /dsgvcy-73/ 2022-09-03T06:24:09.174 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R+PWnc/jh7Z8xkWiBGiT8kTeMmQ= 2022-09-03T06:24:09.175 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:24:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R+PWnc/jh7Z8xkWiBGiT8kTeMmQ=', 'Content-Length': '0'} 2022-09-03T06:24:09.176 INFO:tasks.rgw.c1.client.0.smithi181.stdout:2022-09-03T06:24:09.167+0000 7f7eab425700 -1 ERROR: failed operation: ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-5)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16) 2022-09-03T06:24:09.178 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005678234c78cbb8e7-006312f309-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:24:09.179 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:24:09.179 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:24:09.180 DEBUG:boto:Method: PUT 2022-09-03T06:24:09.180 DEBUG:boto:Path: /dsgvcy-73/a 2022-09-03T06:24:09.181 DEBUG:boto:Data: 2022-09-03T06:24:09.181 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:24:09.182 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:24:09.182 DEBUG:boto:Port: 8000 2022-09-03T06:24:09.183 DEBUG:boto:Params: {} 2022-09-03T06:24:09.183 DEBUG:boto:Token: None 2022-09-03T06:24:09.184 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:24:09 GMT /dsgvcy-73/a 2022-09-03T06:24:09.184 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c+8khaYAfpNYiJEXXgwObLjwH7M= 2022-09-03T06:24:09.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:24:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c+8khaYAfpNYiJEXXgwObLjwH7M='} 2022-09-03T06:24:09.193 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a74cb5169cd994cc-006312f309-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:24:09.194 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:24:09.195 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:24:09.195 DEBUG:boto:Method: HEAD 2022-09-03T06:24:09.196 DEBUG:boto:Path: /dsgvcy-73/ 2022-09-03T06:24:09.196 DEBUG:boto:Data: 2022-09-03T06:24:09.197 DEBUG:boto:Headers: {} 2022-09-03T06:24:09.197 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:24:09.198 DEBUG:boto:Port: 8000 2022-09-03T06:24:09.198 DEBUG:boto:Params: {} 2022-09-03T06:24:09.199 DEBUG:boto:Token: None 2022-09-03T06:24:09.199 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:24:09 GMT /dsgvcy-73/ 2022-09-03T06:24:09.200 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R+PWnc/jh7Z8xkWiBGiT8kTeMmQ= 2022-09-03T06:24:09.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:24:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R+PWnc/jh7Z8xkWiBGiT8kTeMmQ=', 'Content-Length': '0'} 2022-09-03T06:24:09.203 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '3'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000712fb72a22872c7b-006312f309-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:24:09.204 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:24:09.205 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:24:09.205 DEBUG:boto:Method: PUT 2022-09-03T06:24:09.206 DEBUG:boto:Path: /dsgvcy-73/b 2022-09-03T06:24:09.206 DEBUG:boto:Data: 2022-09-03T06:24:09.207 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:24:09.207 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:24:09.208 DEBUG:boto:Port: 8000 2022-09-03T06:24:09.208 DEBUG:boto:Params: {} 2022-09-03T06:24:09.209 DEBUG:boto:Token: None 2022-09-03T06:24:09.210 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:24:09 GMT /dsgvcy-73/b 2022-09-03T06:24:09.210 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IY0gbGPEjOy5Cl8BPfrinjmkM70= 2022-09-03T06:24:09.211 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:24:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IY0gbGPEjOy5Cl8BPfrinjmkM70='} 2022-09-03T06:24:09.213 INFO:tasks.rgw.c1.client.0.smithi181.stdout:2022-09-03T06:24:09.210+0000 7f7eab425700 -1 ERROR: failed operation: ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16) 2022-09-03T06:24:09.254 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000e9df4935dd058da3-006312f309-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:24:09.255 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:24:09.256 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:24:09.256 DEBUG:boto:Method: HEAD 2022-09-03T06:24:09.257 DEBUG:boto:Path: /dsgvcy-73/ 2022-09-03T06:24:09.257 DEBUG:boto:Data: 2022-09-03T06:24:09.258 DEBUG:boto:Headers: {} 2022-09-03T06:24:09.259 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:24:09.259 DEBUG:boto:Port: 8000 2022-09-03T06:24:09.260 DEBUG:boto:Params: {} 2022-09-03T06:24:09.260 DEBUG:boto:Token: None 2022-09-03T06:24:09.261 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:24:09 GMT /dsgvcy-73/ 2022-09-03T06:24:09.261 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R+PWnc/jh7Z8xkWiBGiT8kTeMmQ= 2022-09-03T06:24:09.262 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:24:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R+PWnc/jh7Z8xkWiBGiT8kTeMmQ=', 'Content-Length': '0'} 2022-09-03T06:24:09.265 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009061b6673c97ce69-006312f309-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:24:09.265 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:24:09.266 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:24:09.267 DEBUG:boto:Method: PUT 2022-09-03T06:24:09.267 DEBUG:boto:Path: /dsgvcy-73/c 2022-09-03T06:24:09.268 DEBUG:boto:Data: 2022-09-03T06:24:09.268 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:24:09.269 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:24:09.269 DEBUG:boto:Port: 8000 2022-09-03T06:24:09.270 DEBUG:boto:Params: {} 2022-09-03T06:24:09.270 DEBUG:boto:Token: None 2022-09-03T06:24:09.271 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:24:09 GMT /dsgvcy-73/c 2022-09-03T06:24:09.272 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KMJ8UFGfht1T38QkP3TC6QcVwMQ= 2022-09-03T06:24:09.272 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:24:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KMJ8UFGfht1T38QkP3TC6QcVwMQ='} 2022-09-03T06:24:09.279 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000fae57ae2f43b821b-006312f309-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:24:09.280 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:24:09.281 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:24:09.281 DEBUG:boto:Method: HEAD 2022-09-03T06:24:09.282 DEBUG:boto:Path: /dsgvcy-73/ 2022-09-03T06:24:09.282 DEBUG:boto:Data: 2022-09-03T06:24:09.283 DEBUG:boto:Headers: {} 2022-09-03T06:24:09.283 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:24:09.284 DEBUG:boto:Port: 8000 2022-09-03T06:24:09.284 DEBUG:boto:Params: {} 2022-09-03T06:24:09.285 DEBUG:boto:Token: None 2022-09-03T06:24:09.285 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:24:09 GMT /dsgvcy-73/ 2022-09-03T06:24:09.286 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R+PWnc/jh7Z8xkWiBGiT8kTeMmQ= 2022-09-03T06:24:09.287 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:24:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R+PWnc/jh7Z8xkWiBGiT8kTeMmQ=', 'Content-Length': '0'} 2022-09-03T06:24:09.289 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '9'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006b3d3bf4d5636a10-006312f309-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:24:09.290 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:24:09.290 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:24:09.291 DEBUG:boto:Method: PUT 2022-09-03T06:24:09.292 DEBUG:boto:Path: /dsgvcy-73/d 2022-09-03T06:24:09.292 DEBUG:boto:Data: 2022-09-03T06:24:09.293 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:24:09.293 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:24:09.293 DEBUG:boto:Port: 8000 2022-09-03T06:24:09.294 DEBUG:boto:Params: {} 2022-09-03T06:24:09.295 DEBUG:boto:Token: None 2022-09-03T06:24:09.295 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:24:09 GMT /dsgvcy-73/d 2022-09-03T06:24:09.296 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:piNZVtQ8fHIQbk5Ldivl6lqrkRg= 2022-09-03T06:24:09.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:24:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:piNZVtQ8fHIQbk5Ldivl6lqrkRg='} 2022-09-03T06:24:09.303 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000021cdd7279a9a0de3-006312f309-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:24:09.304 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--bucket', 'dsgvcy-73', '--num-shards', '3', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:24:09.304 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--bucket', 'dsgvcy-73', '--num-shards', '3', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:24:09.305 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --bucket dsgvcy-73 --num-shards 3 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T06:24:09.319 INFO:tasks.rgw.c1.client.0.smithi181.stdout:2022-09-03T06:24:09.317+0000 7f7eab425700 -1 ERROR: failed operation: ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16) 2022-09-03T06:24:09.360 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:24:09.361 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:24:09.407 INFO:teuthology.orchestra.run.smithi181.stdout:tenant: 2022-09-03T06:24:09.407 INFO:teuthology.orchestra.run.smithi181.stdout:bucket name: dsgvcy-73 2022-09-03T06:24:09.412 INFO:teuthology.orchestra.run.smithi181.stdout:total entries: 4 2022-09-03T06:24:09.455 INFO:tasks.rgw.c1.client.0.smithi181.stdout:2022-09-03T06:24:09.452+0000 7f7eacc28700 -1 ERROR: failed operation: ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16) 2022-09-03T06:24:09.467 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:24:09.468 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:24:09.468 DEBUG:boto:Method: HEAD 2022-09-03T06:24:09.469 DEBUG:boto:Path: /dsgvcy-73/ 2022-09-03T06:24:09.469 DEBUG:boto:Data: 2022-09-03T06:24:09.470 DEBUG:boto:Headers: {} 2022-09-03T06:24:09.470 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:24:09.471 DEBUG:boto:Port: 8000 2022-09-03T06:24:09.471 DEBUG:boto:Params: {} 2022-09-03T06:24:09.472 DEBUG:boto:Token: None 2022-09-03T06:24:09.472 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:24:09 GMT /dsgvcy-73/ 2022-09-03T06:24:09.473 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R+PWnc/jh7Z8xkWiBGiT8kTeMmQ= 2022-09-03T06:24:09.473 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:24:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R+PWnc/jh7Z8xkWiBGiT8kTeMmQ=', 'Content-Length': '0'} 2022-09-03T06:24:09.476 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000427de9c427062974-006312f309-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:24:09.477 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:24:09.478 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:24:09.478 DEBUG:boto:Method: PUT 2022-09-03T06:24:09.479 DEBUG:boto:Path: /dsgvcy-73/e 2022-09-03T06:24:09.479 DEBUG:boto:Data: 2022-09-03T06:24:09.480 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:24:09.480 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:24:09.481 DEBUG:boto:Port: 8000 2022-09-03T06:24:09.481 DEBUG:boto:Params: {} 2022-09-03T06:24:09.482 DEBUG:boto:Token: None 2022-09-03T06:24:09.483 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:24:09 GMT /dsgvcy-73/e 2022-09-03T06:24:09.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dFCkPh5kigTM/eZn2NvpRvlOUBU= 2022-09-03T06:24:09.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:24:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dFCkPh5kigTM/eZn2NvpRvlOUBU='} 2022-09-03T06:24:09.492 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000fb8aadb02cfb5aac-006312f309-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:24:09.493 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:24:09.494 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:24:09.494 DEBUG:boto:Method: HEAD 2022-09-03T06:24:09.495 DEBUG:boto:Path: /dsgvcy-73/ 2022-09-03T06:24:09.495 DEBUG:boto:Data: 2022-09-03T06:24:09.496 DEBUG:boto:Headers: {} 2022-09-03T06:24:09.497 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:24:09.497 DEBUG:boto:Port: 8000 2022-09-03T06:24:09.498 DEBUG:boto:Params: {} 2022-09-03T06:24:09.498 DEBUG:boto:Token: None 2022-09-03T06:24:09.499 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:24:09 GMT /dsgvcy-73/ 2022-09-03T06:24:09.500 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R+PWnc/jh7Z8xkWiBGiT8kTeMmQ= 2022-09-03T06:24:09.501 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:24:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R+PWnc/jh7Z8xkWiBGiT8kTeMmQ=', 'Content-Length': '0'} 2022-09-03T06:24:09.504 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '15'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002269a87e03c3d504-006312f309-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:24:09.505 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:24:09.505 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:24:09.506 DEBUG:boto:Method: PUT 2022-09-03T06:24:09.507 DEBUG:boto:Path: /dsgvcy-73/f 2022-09-03T06:24:09.508 DEBUG:boto:Data: 2022-09-03T06:24:09.508 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:24:09.509 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:24:09.509 DEBUG:boto:Port: 8000 2022-09-03T06:24:09.510 DEBUG:boto:Params: {} 2022-09-03T06:24:09.511 DEBUG:boto:Token: None 2022-09-03T06:24:09.512 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:24:09 GMT /dsgvcy-73/f 2022-09-03T06:24:09.512 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y5op4GerHlS8NKFpWlPOC6wEWd0= 2022-09-03T06:24:09.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:24:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y5op4GerHlS8NKFpWlPOC6wEWd0='} 2022-09-03T06:24:09.537 INFO:tasks.rgw.c1.client.0.smithi181.stdout:2022-09-03T06:24:09.535+0000 7f7eacc28700 -1 ERROR: failed operation: ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-5)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-5)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-5)ERROR: collect() returned error (ret=-16) 2022-09-03T06:24:09.559 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ca6312a73332395b-006312f309-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:24:09.560 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:24:09.562 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:24:09.562 DEBUG:boto:Method: HEAD 2022-09-03T06:24:09.563 DEBUG:boto:Path: /dsgvcy-73/ 2022-09-03T06:24:09.564 DEBUG:boto:Data: 2022-09-03T06:24:09.564 DEBUG:boto:Headers: {} 2022-09-03T06:24:09.565 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:24:09.565 DEBUG:boto:Port: 8000 2022-09-03T06:24:09.566 DEBUG:boto:Params: {} 2022-09-03T06:24:09.567 DEBUG:boto:Token: None 2022-09-03T06:24:09.568 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:24:09 GMT /dsgvcy-73/ 2022-09-03T06:24:09.569 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R+PWnc/jh7Z8xkWiBGiT8kTeMmQ= 2022-09-03T06:24:09.570 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:24:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R+PWnc/jh7Z8xkWiBGiT8kTeMmQ=', 'Content-Length': '0'} 2022-09-03T06:24:09.573 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000644d1830f1abc04a-006312f309-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:24:09.574 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:24:09.576 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:24:09.576 DEBUG:boto:Method: PUT 2022-09-03T06:24:09.578 DEBUG:boto:Path: /dsgvcy-73/g 2022-09-03T06:24:09.579 DEBUG:boto:Data: 2022-09-03T06:24:09.579 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:24:09.580 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:24:09.581 DEBUG:boto:Port: 8000 2022-09-03T06:24:09.582 DEBUG:boto:Params: {} 2022-09-03T06:24:09.583 DEBUG:boto:Token: None 2022-09-03T06:24:09.584 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:24:09 GMT /dsgvcy-73/g 2022-09-03T06:24:09.585 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XfNTjCXzeYa1D06yXnzaJcwBrLk= 2022-09-03T06:24:09.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:24:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XfNTjCXzeYa1D06yXnzaJcwBrLk='} 2022-09-03T06:24:09.588 INFO:tasks.rgw.c1.client.0.smithi181.stdout:2022-09-03T06:24:09.576+0000 7f7eacc28700 -1 ERROR: failed operation: ERROR: collect() returned error (ret=-5)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16) 2022-09-03T06:24:09.589 INFO:tasks.rgw.c1.client.0.smithi181.stdout:2022-09-03T06:24:09.576+0000 7f7eacc28700 -1 ERROR: failed operation: ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16) 2022-09-03T06:24:09.595 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001627a56db11d742c-006312f309-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:24:09.596 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:24:09.597 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:24:09.597 DEBUG:boto:Method: HEAD 2022-09-03T06:24:09.598 DEBUG:boto:Path: /dsgvcy-73/ 2022-09-03T06:24:09.598 DEBUG:boto:Data: 2022-09-03T06:24:09.599 DEBUG:boto:Headers: {} 2022-09-03T06:24:09.599 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:24:09.600 DEBUG:boto:Port: 8000 2022-09-03T06:24:09.601 DEBUG:boto:Params: {} 2022-09-03T06:24:09.602 DEBUG:boto:Token: None 2022-09-03T06:24:09.602 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:24:09 GMT /dsgvcy-73/ 2022-09-03T06:24:09.603 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R+PWnc/jh7Z8xkWiBGiT8kTeMmQ= 2022-09-03T06:24:09.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:24:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R+PWnc/jh7Z8xkWiBGiT8kTeMmQ=', 'Content-Length': '0'} 2022-09-03T06:24:09.607 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '21'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a3b01ae5eeadebed-006312f309-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:24:09.609 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:24:09.610 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:24:09.611 DEBUG:boto:Method: PUT 2022-09-03T06:24:09.612 DEBUG:boto:Path: /dsgvcy-73/h 2022-09-03T06:24:09.612 DEBUG:boto:Data: 2022-09-03T06:24:09.614 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:24:09.615 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:24:09.616 DEBUG:boto:Port: 8000 2022-09-03T06:24:09.617 DEBUG:boto:Params: {} 2022-09-03T06:24:09.618 DEBUG:boto:Token: None 2022-09-03T06:24:09.619 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:24:09 GMT /dsgvcy-73/h 2022-09-03T06:24:09.620 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sT5zcc4mPHw6mTIeo1jFKbPr0Vs= 2022-09-03T06:24:09.621 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:24:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sT5zcc4mPHw6mTIeo1jFKbPr0Vs='} 2022-09-03T06:24:09.666 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003a6339d3a746f278-006312f309-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:24:09.667 INFO:tasks.rgw.c2.client.0:Restarting daemon 2022-09-03T06:24:09.668 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c2.client.0.stdout 2>&1 2022-09-03T06:24:09.672 INFO:tasks.rgw.c2.client.0:Started 2022-09-03T06:24:09.673 DEBUG:teuthology.orchestra.run.smithi189:> curl http://smithi189.front.sepia.ceph.com:8000 2022-09-03T06:24:09.699 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T06:24:09.700 INFO:teuthology.orchestra.run.smithi189.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T06:24:09.701 INFO:teuthology.orchestra.run.smithi189.stderr: Dload Upload Total Spent Left Speed 2022-09-03T06:24:09.701 INFO:teuthology.orchestra.run.smithi189.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi189.front.sepia.ceph.com port 8000: Connection refused 2022-09-03T06:24:10.702 DEBUG:teuthology.orchestra.run.smithi189:> curl http://smithi189.front.sepia.ceph.com:8000 2022-09-03T06:24:10.730 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T06:24:10.731 INFO:teuthology.orchestra.run.smithi189.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T06:24:10.732 INFO:teuthology.orchestra.run.smithi189.stderr: Dload Upload Total Spent Left Speed 2022-09-03T06:24:10.732 INFO:teuthology.orchestra.run.smithi189.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi189.front.sepia.ceph.com port 8000: Connection refused 2022-09-03T06:24:12.272 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c1.rgw.client.1 has been restored 2022-09-03T06:24:12.733 DEBUG:teuthology.orchestra.run.smithi189:> curl http://smithi189.front.sepia.ceph.com:8000 2022-09-03T06:24:12.761 INFO:teuthology.orchestra.run.smithi189.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T06:24:12.761 INFO:teuthology.orchestra.run.smithi189.stderr: Dload Upload Total Spent Left Speed 2022-09-03T06:24:12.762 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T06:24:12.763 INFO:teuthology.orchestra.run.smithi189.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi189.front.sepia.ceph.com port 8000: Connection refused 2022-09-03T06:24:16.765 DEBUG:teuthology.orchestra.run.smithi189:> curl http://smithi189.front.sepia.ceph.com:8000 2022-09-03T06:24:16.795 INFO:teuthology.orchestra.run.smithi189.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T06:24:16.796 INFO:teuthology.orchestra.run.smithi189.stderr: Dload Upload Total Spent Left Speed 2022-09-03T06:24:16.796 INFO:teuthology.orchestra.run.smithi189.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 208k 0 --:--:-- --:--:-- --:--:-- 208k 2022-09-03T06:24:16.797 INFO:teuthology.orchestra.run.smithi189.stdout:anonymous 2022-09-03T06:24:16.798 INFO:tasks.rgw.c1.client.1:Restarting daemon 2022-09-03T06:24:16.799 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8001' -n client.1 --cluster c1 -k /etc/ceph/c1.client.1.keyring --log-file /var/log/ceph/rgw.c1.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c1.client.1.sock --foreground --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c1.client.1.stdout 2>&1 2022-09-03T06:24:16.804 INFO:tasks.rgw.c1.client.1:Started 2022-09-03T06:24:16.805 DEBUG:teuthology.orchestra.run.smithi181:> curl http://smithi181.front.sepia.ceph.com:8001 2022-09-03T06:24:16.829 INFO:teuthology.orchestra.run.smithi181.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T06:24:16.830 INFO:teuthology.orchestra.run.smithi181.stderr: Dload Upload Total Spent Left Speed 2022-09-03T06:24:16.830 INFO:teuthology.orchestra.run.smithi181.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi181.front.sepia.ceph.com port 8001: Connection refused 2022-09-03T06:24:16.831 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T06:24:17.833 DEBUG:teuthology.orchestra.run.smithi181:> curl http://smithi181.front.sepia.ceph.com:8001 2022-09-03T06:24:17.863 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T06:24:17.865 INFO:teuthology.orchestra.run.smithi181.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T06:24:17.865 INFO:teuthology.orchestra.run.smithi181.stderr: Dload Upload Total Spent Left Speed 2022-09-03T06:24:17.866 INFO:teuthology.orchestra.run.smithi181.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi181.front.sepia.ceph.com port 8001: Connection refused 2022-09-03T06:24:19.868 DEBUG:teuthology.orchestra.run.smithi181:> curl http://smithi181.front.sepia.ceph.com:8001 2022-09-03T06:24:19.897 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T06:24:19.898 INFO:teuthology.orchestra.run.smithi181.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T06:24:19.898 INFO:teuthology.orchestra.run.smithi181.stderr: Dload Upload Total Spent Left Speed 2022-09-03T06:24:19.899 INFO:teuthology.orchestra.run.smithi181.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi181.front.sepia.ceph.com port 8001: Connection refused 2022-09-03T06:24:23.901 DEBUG:teuthology.orchestra.run.smithi181:> curl http://smithi181.front.sepia.ceph.com:8001 2022-09-03T06:24:23.928 INFO:teuthology.orchestra.run.smithi181.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T06:24:23.929 INFO:teuthology.orchestra.run.smithi181.stderr: Dload Upload Total Spent Left Speed 2022-09-03T06:24:23.931 INFO:teuthology.orchestra.run.smithi181.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 208k 0 --:--:-- --:--:-- --:--:-- 208k 2022-09-03T06:24:23.932 INFO:teuthology.orchestra.run.smithi181.stdout:anonymous 2022-09-03T06:24:23.932 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-73 2022-09-03T06:24:23.934 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-73', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:24:23.935 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-73', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:24:23.936 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-73 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:24:24.002 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:24:24.002 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:24:24.052 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:24:24.049+0000 7f50b1737600 1 waiting to reach incremental sync.. 2022-09-03T06:24:29.052 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:24:29.050+0000 7f50b1737600 1 waiting to reach incremental sync.. 2022-09-03T06:24:34.053 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:24:34.051+0000 7f50b1737600 1 waiting to reach incremental sync.. 2022-09-03T06:24:39.053 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:24:39.051+0000 7f50b1737600 1 waiting to reach incremental sync.. 2022-09-03T06:24:44.054 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:24:44.051+0000 7f50b1737600 1 waiting to reach incremental sync.. 2022-09-03T06:24:49.054 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:24:49.052+0000 7f50b1737600 1 waiting to reach incremental sync.. 2022-09-03T06:24:54.055 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:24:54.053+0000 7f50b1737600 1 waiting to reach incremental sync.. 2022-09-03T06:24:59.055 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:24:59.053+0000 7f50b1737600 1 waiting to reach incremental sync.. 2022-09-03T06:25:04.056 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:25:04.054+0000 7f50b1737600 1 waiting to reach incremental sync.. 2022-09-03T06:25:09.057 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:25:09.054+0000 7f50b1737600 1 waiting to reach incremental sync.. 2022-09-03T06:25:14.057 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:25:14.055+0000 7f50b1737600 1 waiting to reach incremental sync.. 2022-09-03T06:25:19.060 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:25:19.056+0000 7f50b1737600 1 waiting to reach incremental sync.. 2022-09-03T06:25:24.059 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:25:24.057+0000 7f50b1737600 1 waiting to reach incremental sync.. 2022-09-03T06:25:29.059 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:25:29.057+0000 7f50b1737600 1 waiting to reach incremental sync.. 2022-09-03T06:25:34.060 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:25:34.058+0000 7f50b1737600 1 waiting to reach incremental sync.. 2022-09-03T06:25:39.061 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:25:39.058+0000 7f50b1737600 1 waiting to reach incremental sync.. 2022-09-03T06:25:44.061 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:25:44.059+0000 7f50b1737600 1 waiting to reach incremental sync.. 2022-09-03T06:25:49.062 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:25:49.059+0000 7f50b1737600 1 waiting to reach incremental sync.. 2022-09-03T06:25:54.062 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:25:54.060+0000 7f50b1737600 1 waiting to reach incremental sync.. 2022-09-03T06:25:59.063 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:25:59.060+0000 7f50b1737600 1 waiting to reach incremental sync.. 2022-09-03T06:26:04.063 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:26:04.061+0000 7f50b1737600 1 waiting to reach incremental sync.. 2022-09-03T06:26:09.064 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:26:09.062+0000 7f50b1737600 1 waiting to reach incremental sync.. 2022-09-03T06:26:14.067 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:26:14.062+0000 7f50b1737600 1 waiting to reach incremental sync.. 2022-09-03T06:26:19.066 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:26:19.064+0000 7f50b1737600 1 bucket sync caught up with source: 2022-09-03T06:26:19.067 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.53.5, 00000000002.44.5, 00000000002.45.5] 2022-09-03T06:26:19.067 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.53.5, 00000000002.44.5, 00000000002.45.5] 2022-09-03T06:26:19.068 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:26:19.064+0000 7f50b1737600 0 bucket checkpoint complete 2022-09-03T06:26:19.085 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-73 2022-09-03T06:26:19.086 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-73', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:26:19.088 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-73', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:26:19.089 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-73 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:26:19.160 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:26:19.161 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:26:19.217 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:26:19.214+0000 7f80599a7600 1 waiting to reach incremental sync.. 2022-09-03T06:26:24.218 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:26:24.216+0000 7f80599a7600 1 bucket sync caught up with source: 2022-09-03T06:26:24.219 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000003.53.5, 00000000002.44.5, 00000000002.45.5] 2022-09-03T06:26:24.220 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000003.53.5, 00000000002.44.5, 00000000002.45.5] 2022-09-03T06:26:24.220 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:26:24.216+0000 7f80599a7600 0 bucket checkpoint complete 2022-09-03T06:26:24.239 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-73 2022-09-03T06:26:24.240 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-73', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:26:24.241 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-73', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:26:24.242 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-73 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:26:24.309 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:26:24.310 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:26:24.346 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:26:24.344+0000 7f4439cd9600 1 bucket sync caught up with source: 2022-09-03T06:26:24.346 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000003.53.5, 00000000002.44.5, 00000000002.45.5] 2022-09-03T06:26:24.347 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000003.53.5, 00000000002.44.5, 00000000002.45.5] 2022-09-03T06:26:24.347 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:26:24.344+0000 7f4439cd9600 0 bucket checkpoint complete 2022-09-03T06:26:24.363 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-73 2022-09-03T06:26:24.365 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-73', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:26:24.366 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-73', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:26:24.367 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-73 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:26:24.451 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:26:24.452 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:26:24.476 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:26:24.475+0000 7f424d94d600 1 bucket sync caught up with source: 2022-09-03T06:26:24.477 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.565.5, , , 00000000001.438.5, 00000000001.705.5, , , 00000000002.349.5, , 00000000002.372.5] 2022-09-03T06:26:24.477 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.565.5, , , 00000000001.438.5, 00000000001.705.5, , , 00000000002.349.5, , 00000000002.372.5] 2022-09-03T06:26:24.478 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:26:24.475+0000 7f424d94d600 0 bucket checkpoint complete 2022-09-03T06:26:24.495 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-73 2022-09-03T06:26:24.496 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-73', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:26:24.497 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-73', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:26:24.499 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-73 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:26:24.558 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:26:24.559 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:26:24.621 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:26:24.611+0000 7f3f082e8600 1 waiting to reach incremental sync.. 2022-09-03T06:26:29.613 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:26:29.611+0000 7f3f082e8600 1 waiting to reach incremental sync.. 2022-09-03T06:26:34.614 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:26:34.611+0000 7f3f082e8600 1 waiting to reach incremental sync.. 2022-09-03T06:26:39.615 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:26:39.612+0000 7f3f082e8600 1 waiting to reach incremental sync.. 2022-09-03T06:26:44.616 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:26:44.613+0000 7f3f082e8600 1 waiting to reach incremental sync.. 2022-09-03T06:26:49.616 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:26:49.614+0000 7f3f082e8600 1 waiting to reach incremental sync.. 2022-09-03T06:26:54.617 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:26:54.615+0000 7f3f082e8600 1 waiting to reach incremental sync.. 2022-09-03T06:26:59.618 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:26:59.615+0000 7f3f082e8600 1 waiting to reach incremental sync.. 2022-09-03T06:27:04.618 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:27:04.616+0000 7f3f082e8600 1 waiting to reach incremental sync.. 2022-09-03T06:27:09.618 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:27:09.616+0000 7f3f082e8600 1 waiting to reach incremental sync.. 2022-09-03T06:27:14.619 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:27:14.617+0000 7f3f082e8600 1 waiting to reach incremental sync.. 2022-09-03T06:27:19.621 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:27:19.617+0000 7f3f082e8600 1 waiting to reach incremental sync.. 2022-09-03T06:27:24.627 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:27:24.617+0000 7f3f082e8600 1 waiting to reach incremental sync.. 2022-09-03T06:27:29.621 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:27:29.618+0000 7f3f082e8600 1 waiting to reach incremental sync.. 2022-09-03T06:27:34.629 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:27:34.618+0000 7f3f082e8600 1 waiting to reach incremental sync.. 2022-09-03T06:27:39.622 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:27:39.620+0000 7f3f082e8600 1 waiting to reach incremental sync.. 2022-09-03T06:27:44.624 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:27:44.622+0000 7f3f082e8600 1 bucket sync caught up with source: 2022-09-03T06:27:44.625 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.565.5, , , 00000000001.438.5, 00000000001.705.5, , , 00000000002.349.5, , 00000000002.372.5] 2022-09-03T06:27:44.627 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.565.5, , , 00000000001.438.5, 00000000001.705.5, , , 00000000002.349.5, , 00000000002.372.5] 2022-09-03T06:27:44.627 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:27:44.622+0000 7f3f082e8600 0 bucket checkpoint complete 2022-09-03T06:27:44.647 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-73 2022-09-03T06:27:44.652 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-73 2022-09-03T06:27:44.653 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-73', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:27:44.655 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-73', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:27:44.656 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-73 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:27:44.724 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:27:44.724 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:27:44.774 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:27:44.772+0000 7f77889ee600 1 bucket sync caught up with source: 2022-09-03T06:27:44.774 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.268.5, , , 00000000001.191.5, 00000000001.180.5, , , 00000000002.234.5, , 00000000002.218.5] 2022-09-03T06:27:44.774 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.268.5, , , 00000000001.191.5, 00000000001.180.5, , , 00000000002.234.5, , 00000000002.218.5] 2022-09-03T06:27:44.775 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:27:44.772+0000 7f77889ee600 0 bucket checkpoint complete 2022-09-03T06:27:44.797 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-73 2022-09-03T06:27:44.799 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-73', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:27:44.800 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-73', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:27:44.800 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-73 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:27:44.872 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:27:44.873 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:27:44.919 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:27:44.915+0000 7f61b98d8600 1 bucket sync caught up with source: 2022-09-03T06:27:44.920 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000002.268.5, , , 00000000001.191.5, 00000000001.180.5, , , 00000000002.234.5, , 00000000002.218.5] 2022-09-03T06:27:44.920 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000002.268.5, , , 00000000001.191.5, 00000000001.180.5, , , 00000000002.234.5, , 00000000002.218.5] 2022-09-03T06:27:44.921 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:27:44.915+0000 7f61b98d8600 0 bucket checkpoint complete 2022-09-03T06:27:44.935 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-73 2022-09-03T06:27:44.937 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-73 zones={test-zone2, test-zone1} 2022-09-03T06:27:44.939 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:44.940 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:44.941 DEBUG:boto:Method: HEAD 2022-09-03T06:27:44.942 DEBUG:boto:Path: /dsgvcy-73/ 2022-09-03T06:27:44.944 DEBUG:boto:Data: 2022-09-03T06:27:44.944 DEBUG:boto:Headers: {} 2022-09-03T06:27:44.945 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:44.946 DEBUG:boto:Port: 8000 2022-09-03T06:27:44.947 DEBUG:boto:Params: {} 2022-09-03T06:27:44.948 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:27:44.950 DEBUG:boto:Token: None 2022-09-03T06:27:44.950 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:44 GMT /dsgvcy-73/ 2022-09-03T06:27:44.951 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Lb0CfX+G1Q9xRR+Iv4GGayOhdBg= 2022-09-03T06:27:44.952 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Lb0CfX+G1Q9xRR+Iv4GGayOhdBg=', 'Content-Length': '0'} 2022-09-03T06:27:44.959 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003b7971625620ee46-006312f3e0-188c-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:27:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:44.961 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:44.962 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:44.963 DEBUG:boto:Method: HEAD 2022-09-03T06:27:44.964 DEBUG:boto:Path: /dsgvcy-73/ 2022-09-03T06:27:44.964 DEBUG:boto:Data: 2022-09-03T06:27:44.965 DEBUG:boto:Headers: {} 2022-09-03T06:27:44.966 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:44.967 DEBUG:boto:Port: 8000 2022-09-03T06:27:44.968 DEBUG:boto:Params: {} 2022-09-03T06:27:44.969 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:27:44.970 DEBUG:boto:Token: None 2022-09-03T06:27:44.971 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:44 GMT /dsgvcy-73/ 2022-09-03T06:27:44.973 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Lb0CfX+G1Q9xRR+Iv4GGayOhdBg= 2022-09-03T06:27:44.974 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Lb0CfX+G1Q9xRR+Iv4GGayOhdBg=', 'Content-Length': '0'} 2022-09-03T06:27:44.979 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000989c948ebd4a866b-006312f3e0-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:27:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:44.980 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:27:44.981 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:44.981 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:44.982 DEBUG:boto:path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:44.982 DEBUG:boto:auth_path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:44.983 DEBUG:boto:Method: GET 2022-09-03T06:27:44.983 DEBUG:boto:Path: /dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:44.984 DEBUG:boto:Data: 2022-09-03T06:27:44.985 DEBUG:boto:Headers: {} 2022-09-03T06:27:44.985 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:44.986 DEBUG:boto:Port: 8000 2022-09-03T06:27:44.986 DEBUG:boto:Params: {} 2022-09-03T06:27:44.987 DEBUG:boto:Token: None 2022-09-03T06:27:44.987 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:44 GMT /dsgvcy-73/?versions 2022-09-03T06:27:44.988 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IdSdaIzCgggGCm++2B687vnQDVI= 2022-09-03T06:27:44.989 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IdSdaIzCgggGCm++2B687vnQDVI=', 'Content-Length': '0'} 2022-09-03T06:27:44.991 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006ea5815bad0af0af-006312f3e0-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:44.992 DEBUG:boto:b'dsgvcy-73999falseanulltrue2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:44.994 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:27:44.995 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:27:44.996 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:27:44.996 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:27:44.997 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:27:44.997 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:27:44.997 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:27:44.998 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:27:44.999 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:27:45.000 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:45.000 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:45.001 DEBUG:boto:path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:45.002 DEBUG:boto:auth_path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:45.002 DEBUG:boto:Method: GET 2022-09-03T06:27:45.003 DEBUG:boto:Path: /dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:45.003 DEBUG:boto:Data: 2022-09-03T06:27:45.004 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.005 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.005 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.006 DEBUG:boto:Params: {} 2022-09-03T06:27:45.006 DEBUG:boto:Token: None 2022-09-03T06:27:45.007 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/?versions 2022-09-03T06:27:45.008 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/U4WPh37iS1+VKgQf2man5kTODo= 2022-09-03T06:27:45.009 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/U4WPh37iS1+VKgQf2man5kTODo=', 'Content-Length': '0'} 2022-09-03T06:27:45.011 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000528a291e8efbd993-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.012 DEBUG:boto:b'dsgvcy-73999falseanulltrue2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:45.013 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:27:45.013 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:27:45.014 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:27:45.014 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:27:45.015 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:27:45.015 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:27:45.016 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:27:45.017 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:27:45.018 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:45.018 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:45.019 DEBUG:boto:path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:45.019 DEBUG:boto:auth_path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:45.020 DEBUG:boto:Method: GET 2022-09-03T06:27:45.020 DEBUG:boto:Path: /dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:45.022 DEBUG:boto:Data: 2022-09-03T06:27:45.023 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.024 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.024 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.025 DEBUG:boto:Params: {} 2022-09-03T06:27:45.026 DEBUG:boto:Token: None 2022-09-03T06:27:45.026 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/?versions 2022-09-03T06:27:45.027 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/U4WPh37iS1+VKgQf2man5kTODo= 2022-09-03T06:27:45.028 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/U4WPh37iS1+VKgQf2man5kTODo=', 'Content-Length': '0'} 2022-09-03T06:27:45.030 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000033a23b327130ac5b-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.031 DEBUG:boto:b'dsgvcy-73999falseanulltrue2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:45.032 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:45.033 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:45.033 DEBUG:boto:path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:45.034 DEBUG:boto:auth_path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:45.034 DEBUG:boto:Method: GET 2022-09-03T06:27:45.035 DEBUG:boto:Path: /dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:45.036 DEBUG:boto:Data: 2022-09-03T06:27:45.036 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.037 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.037 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.038 DEBUG:boto:Params: {} 2022-09-03T06:27:45.038 DEBUG:boto:Token: None 2022-09-03T06:27:45.039 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/?versions 2022-09-03T06:27:45.040 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/U4WPh37iS1+VKgQf2man5kTODo= 2022-09-03T06:27:45.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/U4WPh37iS1+VKgQf2man5kTODo=', 'Content-Length': '0'} 2022-09-03T06:27:45.043 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000646d88fb14c3b8cf-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.043 DEBUG:boto:b'dsgvcy-73999falseanulltrue2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:45.045 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:27:45.046 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:45.046 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:45.047 DEBUG:boto:path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.047 DEBUG:boto:auth_path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.048 DEBUG:boto:Method: GET 2022-09-03T06:27:45.050 DEBUG:boto:Path: /dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.050 DEBUG:boto:Data: 2022-09-03T06:27:45.051 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.051 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.052 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.053 DEBUG:boto:Params: {} 2022-09-03T06:27:45.053 DEBUG:boto:Token: None 2022-09-03T06:27:45.054 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.055 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/fZrNO2aP52ODqlptWM/BvoDmgc= 2022-09-03T06:27:45.055 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/fZrNO2aP52ODqlptWM/BvoDmgc=', 'Content-Length': '0'} 2022-09-03T06:27:45.058 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d858904cdb748ba9-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.098 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:45.100 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:45.100 DEBUG:boto:path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.101 DEBUG:boto:auth_path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.102 DEBUG:boto:Method: GET 2022-09-03T06:27:45.102 DEBUG:boto:Path: /dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.103 DEBUG:boto:Data: 2022-09-03T06:27:45.104 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.105 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.106 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.107 DEBUG:boto:Params: {} 2022-09-03T06:27:45.108 DEBUG:boto:Token: None 2022-09-03T06:27:45.109 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.110 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/fZrNO2aP52ODqlptWM/BvoDmgc= 2022-09-03T06:27:45.111 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/fZrNO2aP52ODqlptWM/BvoDmgc=', 'Content-Length': '0'} 2022-09-03T06:27:45.114 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad72a2334c25fc7d-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.116 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:45.117 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:45.118 DEBUG:boto:path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.119 DEBUG:boto:auth_path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.120 DEBUG:boto:Method: HEAD 2022-09-03T06:27:45.122 DEBUG:boto:Path: /dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.123 DEBUG:boto:Data: 2022-09-03T06:27:45.124 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.124 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.125 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.126 DEBUG:boto:Params: {} 2022-09-03T06:27:45.127 DEBUG:boto:Token: None 2022-09-03T06:27:45.128 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.129 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vfWD0lItF5jA18amD+RvHnFbApw= 2022-09-03T06:27:45.130 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vfWD0lItF5jA18amD+RvHnFbApw=', 'Content-Length': '0'} 2022-09-03T06:27:45.134 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2ea73b850fff39b-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.135 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:45.136 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:45.137 DEBUG:boto:path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.138 DEBUG:boto:auth_path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.138 DEBUG:boto:Method: HEAD 2022-09-03T06:27:45.139 DEBUG:boto:Path: /dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.139 DEBUG:boto:Data: 2022-09-03T06:27:45.140 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.141 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.141 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.142 DEBUG:boto:Params: {} 2022-09-03T06:27:45.143 DEBUG:boto:Token: None 2022-09-03T06:27:45.143 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.144 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vfWD0lItF5jA18amD+RvHnFbApw= 2022-09-03T06:27:45.145 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vfWD0lItF5jA18amD+RvHnFbApw=', 'Content-Length': '0'} 2022-09-03T06:27:45.150 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005891fbe64e1b2593-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.151 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:27:45.152 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:45.152 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:45.154 DEBUG:boto:path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.155 DEBUG:boto:auth_path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.156 DEBUG:boto:Method: GET 2022-09-03T06:27:45.157 DEBUG:boto:Path: /dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.158 DEBUG:boto:Data: 2022-09-03T06:27:45.159 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.160 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.161 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.162 DEBUG:boto:Params: {} 2022-09-03T06:27:45.163 DEBUG:boto:Token: None 2022-09-03T06:27:45.164 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.166 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/fZrNO2aP52ODqlptWM/BvoDmgc= 2022-09-03T06:27:45.167 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/fZrNO2aP52ODqlptWM/BvoDmgc=', 'Content-Length': '0'} 2022-09-03T06:27:45.171 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000082f537efd6840b99-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.172 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:45.173 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:45.174 DEBUG:boto:path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.174 DEBUG:boto:auth_path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.175 DEBUG:boto:Method: GET 2022-09-03T06:27:45.175 DEBUG:boto:Path: /dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.176 DEBUG:boto:Data: 2022-09-03T06:27:45.176 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.177 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.178 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.178 DEBUG:boto:Params: {} 2022-09-03T06:27:45.179 DEBUG:boto:Token: None 2022-09-03T06:27:45.179 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/a?versionId=null 2022-09-03T06:27:45.180 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/fZrNO2aP52ODqlptWM/BvoDmgc= 2022-09-03T06:27:45.181 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/fZrNO2aP52ODqlptWM/BvoDmgc=', 'Content-Length': '0'} 2022-09-03T06:27:45.183 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002ca22db9e44816d8-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.226 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:45.228 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:45.229 DEBUG:boto:path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:45.229 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:45.230 DEBUG:boto:Method: GET 2022-09-03T06:27:45.236 DEBUG:boto:Path: /dsgvcy-73/?prefix=a 2022-09-03T06:27:45.237 DEBUG:boto:Data: 2022-09-03T06:27:45.238 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.239 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.240 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.241 DEBUG:boto:Params: {} 2022-09-03T06:27:45.242 DEBUG:boto:Token: None 2022-09-03T06:27:45.243 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/ 2022-09-03T06:27:45.244 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o= 2022-09-03T06:27:45.245 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o=', 'Content-Length': '0'} 2022-09-03T06:27:45.249 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001997ad9b83af1d59-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.250 DEBUG:boto:b'dsgvcy-73a1000falsea2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:45.252 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:45.253 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:45.254 DEBUG:boto:path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:45.255 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:45.256 DEBUG:boto:Method: GET 2022-09-03T06:27:45.257 DEBUG:boto:Path: /dsgvcy-73/?prefix=a 2022-09-03T06:27:45.257 DEBUG:boto:Data: 2022-09-03T06:27:45.258 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.258 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.259 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.260 DEBUG:boto:Params: {} 2022-09-03T06:27:45.261 DEBUG:boto:Token: None 2022-09-03T06:27:45.261 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/ 2022-09-03T06:27:45.262 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o= 2022-09-03T06:27:45.262 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o=', 'Content-Length': '0'} 2022-09-03T06:27:45.268 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000645c9d38673ac95f-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.269 DEBUG:boto:b'dsgvcy-73a1000falsea2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:45.271 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:45.272 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:45.273 DEBUG:boto:Method: HEAD 2022-09-03T06:27:45.273 DEBUG:boto:Path: /dsgvcy-73/a 2022-09-03T06:27:45.274 DEBUG:boto:Data: 2022-09-03T06:27:45.274 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.275 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.275 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.276 DEBUG:boto:Params: {} 2022-09-03T06:27:45.277 DEBUG:boto:Token: None 2022-09-03T06:27:45.277 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/a 2022-09-03T06:27:45.278 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RTRCy1FmMmHIa4cV+ObbHVkOaug= 2022-09-03T06:27:45.279 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RTRCy1FmMmHIa4cV+ObbHVkOaug=', 'Content-Length': '0'} 2022-09-03T06:27:45.282 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000071e9764b65069169-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.283 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:45.284 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:45.284 DEBUG:boto:Method: HEAD 2022-09-03T06:27:45.285 DEBUG:boto:Path: /dsgvcy-73/a 2022-09-03T06:27:45.285 DEBUG:boto:Data: 2022-09-03T06:27:45.286 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.286 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.287 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.288 DEBUG:boto:Params: {} 2022-09-03T06:27:45.288 DEBUG:boto:Token: None 2022-09-03T06:27:45.290 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/a 2022-09-03T06:27:45.291 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RTRCy1FmMmHIa4cV+ObbHVkOaug= 2022-09-03T06:27:45.291 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RTRCy1FmMmHIa4cV+ObbHVkOaug=', 'Content-Length': '0'} 2022-09-03T06:27:45.293 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006db7a460b0349e2d-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.294 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:27:45.294 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:45.295 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:45.296 DEBUG:boto:Method: GET 2022-09-03T06:27:45.296 DEBUG:boto:Path: /dsgvcy-73/a 2022-09-03T06:27:45.297 DEBUG:boto:Data: 2022-09-03T06:27:45.297 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.298 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.299 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.299 DEBUG:boto:Params: {} 2022-09-03T06:27:45.300 DEBUG:boto:Token: None 2022-09-03T06:27:45.301 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/a 2022-09-03T06:27:45.301 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8HDCCz07mCDKKwV2DvWUoci+J8M= 2022-09-03T06:27:45.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8HDCCz07mCDKKwV2DvWUoci+J8M=', 'Content-Length': '0'} 2022-09-03T06:27:45.304 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ba923e78c443570-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.309 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:45.310 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:45.311 DEBUG:boto:Method: GET 2022-09-03T06:27:45.311 DEBUG:boto:Path: /dsgvcy-73/a 2022-09-03T06:27:45.312 DEBUG:boto:Data: 2022-09-03T06:27:45.313 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.313 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.314 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.315 DEBUG:boto:Params: {} 2022-09-03T06:27:45.315 DEBUG:boto:Token: None 2022-09-03T06:27:45.316 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/a 2022-09-03T06:27:45.317 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8HDCCz07mCDKKwV2DvWUoci+J8M= 2022-09-03T06:27:45.318 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8HDCCz07mCDKKwV2DvWUoci+J8M=', 'Content-Length': '0'} 2022-09-03T06:27:45.320 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000081325fc21fb045-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.321 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:45.322 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:45.322 DEBUG:boto:path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:45.323 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:45.324 DEBUG:boto:Method: GET 2022-09-03T06:27:45.324 DEBUG:boto:Path: /dsgvcy-73/?prefix=a 2022-09-03T06:27:45.325 DEBUG:boto:Data: 2022-09-03T06:27:45.325 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.326 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.326 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.327 DEBUG:boto:Params: {} 2022-09-03T06:27:45.328 DEBUG:boto:Token: None 2022-09-03T06:27:45.328 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/ 2022-09-03T06:27:45.329 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o= 2022-09-03T06:27:45.329 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o=', 'Content-Length': '0'} 2022-09-03T06:27:45.332 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006a23151721f092c2-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.334 DEBUG:boto:b'dsgvcy-73a1000falsea2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:45.335 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:45.336 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:45.336 DEBUG:boto:path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:45.337 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:45.337 DEBUG:boto:Method: GET 2022-09-03T06:27:45.338 DEBUG:boto:Path: /dsgvcy-73/?prefix=a 2022-09-03T06:27:45.339 DEBUG:boto:Data: 2022-09-03T06:27:45.339 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.340 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.340 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.341 DEBUG:boto:Params: {} 2022-09-03T06:27:45.342 DEBUG:boto:Token: None 2022-09-03T06:27:45.343 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/ 2022-09-03T06:27:45.344 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o= 2022-09-03T06:27:45.345 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o=', 'Content-Length': '0'} 2022-09-03T06:27:45.347 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000da326b7edf988003-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.348 DEBUG:boto:b'dsgvcy-73a1000falsea2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:45.349 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:27:45.349 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:45.352 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:45.352 DEBUG:boto:path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.353 DEBUG:boto:auth_path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.354 DEBUG:boto:Method: GET 2022-09-03T06:27:45.355 DEBUG:boto:Path: /dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.356 DEBUG:boto:Data: 2022-09-03T06:27:45.358 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.359 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.360 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.361 DEBUG:boto:Params: {} 2022-09-03T06:27:45.362 DEBUG:boto:Token: None 2022-09-03T06:27:45.364 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.365 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ustxK8B55jwVpCO69PO2Yy5dc5Q= 2022-09-03T06:27:45.366 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ustxK8B55jwVpCO69PO2Yy5dc5Q=', 'Content-Length': '0'} 2022-09-03T06:27:45.369 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000fc58e82a7084b0c-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.371 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:45.372 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:45.372 DEBUG:boto:path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.374 DEBUG:boto:auth_path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.375 DEBUG:boto:Method: GET 2022-09-03T06:27:45.376 DEBUG:boto:Path: /dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.377 DEBUG:boto:Data: 2022-09-03T06:27:45.378 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.379 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.380 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.380 DEBUG:boto:Params: {} 2022-09-03T06:27:45.381 DEBUG:boto:Token: None 2022-09-03T06:27:45.382 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.383 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ustxK8B55jwVpCO69PO2Yy5dc5Q= 2022-09-03T06:27:45.383 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ustxK8B55jwVpCO69PO2Yy5dc5Q=', 'Content-Length': '0'} 2022-09-03T06:27:45.385 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000098e7ceb551cb0959-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.386 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:45.387 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:45.387 DEBUG:boto:path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.388 DEBUG:boto:auth_path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.388 DEBUG:boto:Method: HEAD 2022-09-03T06:27:45.390 DEBUG:boto:Path: /dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.390 DEBUG:boto:Data: 2022-09-03T06:27:45.391 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.391 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.392 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.394 DEBUG:boto:Params: {} 2022-09-03T06:27:45.395 DEBUG:boto:Token: None 2022-09-03T06:27:45.395 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.396 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iGNeRDaXl3sdHz8CWixwbm0GZmI= 2022-09-03T06:27:45.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iGNeRDaXl3sdHz8CWixwbm0GZmI=', 'Content-Length': '0'} 2022-09-03T06:27:45.398 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096123768b3a124d4-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.399 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:45.400 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:45.400 DEBUG:boto:path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.401 DEBUG:boto:auth_path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.401 DEBUG:boto:Method: HEAD 2022-09-03T06:27:45.402 DEBUG:boto:Path: /dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.405 DEBUG:boto:Data: 2022-09-03T06:27:45.406 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.407 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.408 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.409 DEBUG:boto:Params: {} 2022-09-03T06:27:45.410 DEBUG:boto:Token: None 2022-09-03T06:27:45.411 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.412 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iGNeRDaXl3sdHz8CWixwbm0GZmI= 2022-09-03T06:27:45.413 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iGNeRDaXl3sdHz8CWixwbm0GZmI=', 'Content-Length': '0'} 2022-09-03T06:27:45.417 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000094171f3623aa0b0d-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.418 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:27:45.419 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:45.420 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:45.420 DEBUG:boto:path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.421 DEBUG:boto:auth_path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.421 DEBUG:boto:Method: GET 2022-09-03T06:27:45.422 DEBUG:boto:Path: /dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.423 DEBUG:boto:Data: 2022-09-03T06:27:45.423 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.424 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.424 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.425 DEBUG:boto:Params: {} 2022-09-03T06:27:45.426 DEBUG:boto:Token: None 2022-09-03T06:27:45.426 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.427 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ustxK8B55jwVpCO69PO2Yy5dc5Q= 2022-09-03T06:27:45.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ustxK8B55jwVpCO69PO2Yy5dc5Q=', 'Content-Length': '0'} 2022-09-03T06:27:45.430 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000675968a5a066a297-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.431 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:45.431 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:45.432 DEBUG:boto:path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.432 DEBUG:boto:auth_path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.433 DEBUG:boto:Method: GET 2022-09-03T06:27:45.434 DEBUG:boto:Path: /dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.434 DEBUG:boto:Data: 2022-09-03T06:27:45.435 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.435 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.436 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.437 DEBUG:boto:Params: {} 2022-09-03T06:27:45.437 DEBUG:boto:Token: None 2022-09-03T06:27:45.438 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/b?versionId=null 2022-09-03T06:27:45.439 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ustxK8B55jwVpCO69PO2Yy5dc5Q= 2022-09-03T06:27:45.439 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ustxK8B55jwVpCO69PO2Yy5dc5Q=', 'Content-Length': '0'} 2022-09-03T06:27:45.441 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008ef8e5b4b18021de-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.442 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:45.443 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:45.443 DEBUG:boto:path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:45.444 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:45.445 DEBUG:boto:Method: GET 2022-09-03T06:27:45.446 DEBUG:boto:Path: /dsgvcy-73/?prefix=b 2022-09-03T06:27:45.446 DEBUG:boto:Data: 2022-09-03T06:27:45.447 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.447 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.448 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.448 DEBUG:boto:Params: {} 2022-09-03T06:27:45.449 DEBUG:boto:Token: None 2022-09-03T06:27:45.450 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/ 2022-09-03T06:27:45.450 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o= 2022-09-03T06:27:45.451 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o=', 'Content-Length': '0'} 2022-09-03T06:27:45.453 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006335df7c3ef0bbd9-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.454 DEBUG:boto:b'dsgvcy-73b1000falseb2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:45.455 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:45.456 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:45.456 DEBUG:boto:path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:45.457 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:45.457 DEBUG:boto:Method: GET 2022-09-03T06:27:45.458 DEBUG:boto:Path: /dsgvcy-73/?prefix=b 2022-09-03T06:27:45.458 DEBUG:boto:Data: 2022-09-03T06:27:45.459 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.460 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.460 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.461 DEBUG:boto:Params: {} 2022-09-03T06:27:45.461 DEBUG:boto:Token: None 2022-09-03T06:27:45.462 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/ 2022-09-03T06:27:45.463 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o= 2022-09-03T06:27:45.463 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o=', 'Content-Length': '0'} 2022-09-03T06:27:45.467 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005e8a12befb828fc7-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.468 DEBUG:boto:b'dsgvcy-73b1000falseb2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:45.469 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:45.475 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:45.477 DEBUG:boto:Method: HEAD 2022-09-03T06:27:45.478 DEBUG:boto:Path: /dsgvcy-73/b 2022-09-03T06:27:45.479 DEBUG:boto:Data: 2022-09-03T06:27:45.480 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.481 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.481 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.482 DEBUG:boto:Params: {} 2022-09-03T06:27:45.483 DEBUG:boto:Token: None 2022-09-03T06:27:45.484 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/b 2022-09-03T06:27:45.485 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OHLGvh7yUdRK9Qx/qI0BEB0PS+w= 2022-09-03T06:27:45.486 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OHLGvh7yUdRK9Qx/qI0BEB0PS+w=', 'Content-Length': '0'} 2022-09-03T06:27:45.489 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001438dac070601389-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.491 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:45.492 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:45.492 DEBUG:boto:Method: HEAD 2022-09-03T06:27:45.493 DEBUG:boto:Path: /dsgvcy-73/b 2022-09-03T06:27:45.493 DEBUG:boto:Data: 2022-09-03T06:27:45.494 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.495 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.496 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.497 DEBUG:boto:Params: {} 2022-09-03T06:27:45.498 DEBUG:boto:Token: None 2022-09-03T06:27:45.500 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/b 2022-09-03T06:27:45.501 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OHLGvh7yUdRK9Qx/qI0BEB0PS+w= 2022-09-03T06:27:45.502 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OHLGvh7yUdRK9Qx/qI0BEB0PS+w=', 'Content-Length': '0'} 2022-09-03T06:27:45.504 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006ca356b37c96a210-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.506 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:27:45.507 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:45.508 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:45.509 DEBUG:boto:Method: GET 2022-09-03T06:27:45.510 DEBUG:boto:Path: /dsgvcy-73/b 2022-09-03T06:27:45.511 DEBUG:boto:Data: 2022-09-03T06:27:45.512 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.513 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.514 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.515 DEBUG:boto:Params: {} 2022-09-03T06:27:45.516 DEBUG:boto:Token: None 2022-09-03T06:27:45.517 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/b 2022-09-03T06:27:45.518 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2MF8lmbWIkugVKq2O4zjzrifyO0= 2022-09-03T06:27:45.519 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2MF8lmbWIkugVKq2O4zjzrifyO0=', 'Content-Length': '0'} 2022-09-03T06:27:45.522 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000038ff8169391522c2-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.523 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:45.524 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:45.525 DEBUG:boto:Method: GET 2022-09-03T06:27:45.526 DEBUG:boto:Path: /dsgvcy-73/b 2022-09-03T06:27:45.527 DEBUG:boto:Data: 2022-09-03T06:27:45.528 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.529 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.529 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.531 DEBUG:boto:Params: {} 2022-09-03T06:27:45.532 DEBUG:boto:Token: None 2022-09-03T06:27:45.533 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/b 2022-09-03T06:27:45.534 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2MF8lmbWIkugVKq2O4zjzrifyO0= 2022-09-03T06:27:45.535 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2MF8lmbWIkugVKq2O4zjzrifyO0=', 'Content-Length': '0'} 2022-09-03T06:27:45.539 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc6761941cd4a4fe-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.542 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:45.543 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:45.544 DEBUG:boto:path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:45.546 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:45.547 DEBUG:boto:Method: GET 2022-09-03T06:27:45.548 DEBUG:boto:Path: /dsgvcy-73/?prefix=b 2022-09-03T06:27:45.548 DEBUG:boto:Data: 2022-09-03T06:27:45.549 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.550 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.550 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.551 DEBUG:boto:Params: {} 2022-09-03T06:27:45.552 DEBUG:boto:Token: None 2022-09-03T06:27:45.552 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/ 2022-09-03T06:27:45.553 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o= 2022-09-03T06:27:45.553 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o=', 'Content-Length': '0'} 2022-09-03T06:27:45.556 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000370ef0aa8f798c8e-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.603 DEBUG:boto:b'dsgvcy-73b1000falseb2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:45.604 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:45.605 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:45.606 DEBUG:boto:path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:45.607 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:45.608 DEBUG:boto:Method: GET 2022-09-03T06:27:45.609 DEBUG:boto:Path: /dsgvcy-73/?prefix=b 2022-09-03T06:27:45.610 DEBUG:boto:Data: 2022-09-03T06:27:45.611 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.611 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.612 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.613 DEBUG:boto:Params: {} 2022-09-03T06:27:45.613 DEBUG:boto:Token: None 2022-09-03T06:27:45.615 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/ 2022-09-03T06:27:45.616 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o= 2022-09-03T06:27:45.617 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o=', 'Content-Length': '0'} 2022-09-03T06:27:45.620 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e79662df6614b2ca-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.621 DEBUG:boto:b'dsgvcy-73b1000falseb2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:45.623 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:27:45.623 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:45.630 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:45.631 DEBUG:boto:path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.632 DEBUG:boto:auth_path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.632 DEBUG:boto:Method: GET 2022-09-03T06:27:45.633 DEBUG:boto:Path: /dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.633 DEBUG:boto:Data: 2022-09-03T06:27:45.634 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.634 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.635 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.635 DEBUG:boto:Params: {} 2022-09-03T06:27:45.636 DEBUG:boto:Token: None 2022-09-03T06:27:45.637 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.637 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oVVkOhREvXghVHDWJhZAuwOf05g= 2022-09-03T06:27:45.638 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oVVkOhREvXghVHDWJhZAuwOf05g=', 'Content-Length': '0'} 2022-09-03T06:27:45.642 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ec39dba16420e5a0-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.643 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:45.643 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:45.644 DEBUG:boto:path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.644 DEBUG:boto:auth_path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.645 DEBUG:boto:Method: GET 2022-09-03T06:27:45.645 DEBUG:boto:Path: /dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.646 DEBUG:boto:Data: 2022-09-03T06:27:45.646 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.647 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.647 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.648 DEBUG:boto:Params: {} 2022-09-03T06:27:45.648 DEBUG:boto:Token: None 2022-09-03T06:27:45.649 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.650 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oVVkOhREvXghVHDWJhZAuwOf05g= 2022-09-03T06:27:45.651 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oVVkOhREvXghVHDWJhZAuwOf05g=', 'Content-Length': '0'} 2022-09-03T06:27:45.654 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c29883564b62518-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.655 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:45.656 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:45.657 DEBUG:boto:path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.658 DEBUG:boto:auth_path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.658 DEBUG:boto:Method: HEAD 2022-09-03T06:27:45.659 DEBUG:boto:Path: /dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.659 DEBUG:boto:Data: 2022-09-03T06:27:45.660 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.661 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.662 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.663 DEBUG:boto:Params: {} 2022-09-03T06:27:45.664 DEBUG:boto:Token: None 2022-09-03T06:27:45.665 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.666 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aSDIHjcmBUslnSXut23K2pt+5YU= 2022-09-03T06:27:45.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aSDIHjcmBUslnSXut23K2pt+5YU=', 'Content-Length': '0'} 2022-09-03T06:27:45.670 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000359b25ede3427b14-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.671 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:45.672 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:45.673 DEBUG:boto:path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.674 DEBUG:boto:auth_path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.675 DEBUG:boto:Method: HEAD 2022-09-03T06:27:45.675 DEBUG:boto:Path: /dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.676 DEBUG:boto:Data: 2022-09-03T06:27:45.678 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.679 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.680 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.681 DEBUG:boto:Params: {} 2022-09-03T06:27:45.682 DEBUG:boto:Token: None 2022-09-03T06:27:45.683 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.684 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aSDIHjcmBUslnSXut23K2pt+5YU= 2022-09-03T06:27:45.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aSDIHjcmBUslnSXut23K2pt+5YU=', 'Content-Length': '0'} 2022-09-03T06:27:45.688 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eb77c15f589587c5-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.692 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:27:45.693 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:45.694 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:45.697 DEBUG:boto:path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.699 DEBUG:boto:auth_path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.700 DEBUG:boto:Method: GET 2022-09-03T06:27:45.701 DEBUG:boto:Path: /dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.704 DEBUG:boto:Data: 2022-09-03T06:27:45.705 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.707 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.708 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.708 DEBUG:boto:Params: {} 2022-09-03T06:27:45.710 DEBUG:boto:Token: None 2022-09-03T06:27:45.711 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.713 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oVVkOhREvXghVHDWJhZAuwOf05g= 2022-09-03T06:27:45.714 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oVVkOhREvXghVHDWJhZAuwOf05g=', 'Content-Length': '0'} 2022-09-03T06:27:45.720 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cce9e9f86e5b2e02-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.721 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:45.722 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:45.723 DEBUG:boto:path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.724 DEBUG:boto:auth_path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.725 DEBUG:boto:Method: GET 2022-09-03T06:27:45.726 DEBUG:boto:Path: /dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.728 DEBUG:boto:Data: 2022-09-03T06:27:45.729 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.730 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.731 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.732 DEBUG:boto:Params: {} 2022-09-03T06:27:45.733 DEBUG:boto:Token: None 2022-09-03T06:27:45.734 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/c?versionId=null 2022-09-03T06:27:45.735 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oVVkOhREvXghVHDWJhZAuwOf05g= 2022-09-03T06:27:45.737 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oVVkOhREvXghVHDWJhZAuwOf05g=', 'Content-Length': '0'} 2022-09-03T06:27:45.741 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009886283abd603a0c-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.742 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:45.743 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:45.743 DEBUG:boto:path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:45.746 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:45.749 DEBUG:boto:Method: GET 2022-09-03T06:27:45.750 DEBUG:boto:Path: /dsgvcy-73/?prefix=c 2022-09-03T06:27:45.751 DEBUG:boto:Data: 2022-09-03T06:27:45.752 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.753 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.754 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.755 DEBUG:boto:Params: {} 2022-09-03T06:27:45.756 DEBUG:boto:Token: None 2022-09-03T06:27:45.757 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/ 2022-09-03T06:27:45.759 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o= 2022-09-03T06:27:45.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o=', 'Content-Length': '0'} 2022-09-03T06:27:45.772 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003a8cab03682cf8dd-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.773 DEBUG:boto:b'dsgvcy-73c1000falsec2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:45.775 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:45.776 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:45.777 DEBUG:boto:path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:45.779 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:45.781 DEBUG:boto:Method: GET 2022-09-03T06:27:45.782 DEBUG:boto:Path: /dsgvcy-73/?prefix=c 2022-09-03T06:27:45.783 DEBUG:boto:Data: 2022-09-03T06:27:45.784 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.785 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.785 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.786 DEBUG:boto:Params: {} 2022-09-03T06:27:45.787 DEBUG:boto:Token: None 2022-09-03T06:27:45.788 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/ 2022-09-03T06:27:45.788 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o= 2022-09-03T06:27:45.790 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o=', 'Content-Length': '0'} 2022-09-03T06:27:45.793 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bebf5ccc8fd08b1b-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.794 DEBUG:boto:b'dsgvcy-73c1000falsec2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:45.795 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:45.797 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:45.798 DEBUG:boto:Method: HEAD 2022-09-03T06:27:45.798 DEBUG:boto:Path: /dsgvcy-73/c 2022-09-03T06:27:45.799 DEBUG:boto:Data: 2022-09-03T06:27:45.800 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.801 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.801 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.802 DEBUG:boto:Params: {} 2022-09-03T06:27:45.802 DEBUG:boto:Token: None 2022-09-03T06:27:45.803 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/c 2022-09-03T06:27:45.804 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XUUwnzKRvIy1Z0REEjtVHRKX7Hg= 2022-09-03T06:27:45.806 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XUUwnzKRvIy1Z0REEjtVHRKX7Hg=', 'Content-Length': '0'} 2022-09-03T06:27:45.808 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e6940578e599fcfa-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.810 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:45.811 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:45.812 DEBUG:boto:Method: HEAD 2022-09-03T06:27:45.813 DEBUG:boto:Path: /dsgvcy-73/c 2022-09-03T06:27:45.816 DEBUG:boto:Data: 2022-09-03T06:27:45.817 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.818 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.819 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.820 DEBUG:boto:Params: {} 2022-09-03T06:27:45.821 DEBUG:boto:Token: None 2022-09-03T06:27:45.823 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/c 2022-09-03T06:27:45.824 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XUUwnzKRvIy1Z0REEjtVHRKX7Hg= 2022-09-03T06:27:45.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XUUwnzKRvIy1Z0REEjtVHRKX7Hg=', 'Content-Length': '0'} 2022-09-03T06:27:45.828 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000029d8e575161de10a-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.829 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:27:45.830 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:45.831 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:45.831 DEBUG:boto:Method: GET 2022-09-03T06:27:45.832 DEBUG:boto:Path: /dsgvcy-73/c 2022-09-03T06:27:45.832 DEBUG:boto:Data: 2022-09-03T06:27:45.833 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.833 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.834 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.835 DEBUG:boto:Params: {} 2022-09-03T06:27:45.835 DEBUG:boto:Token: None 2022-09-03T06:27:45.836 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/c 2022-09-03T06:27:45.837 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pKcpmPJlQ+yhx76DnX5MduibV2Y= 2022-09-03T06:27:45.839 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pKcpmPJlQ+yhx76DnX5MduibV2Y=', 'Content-Length': '0'} 2022-09-03T06:27:45.841 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c43dec69e9e3acc6-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.842 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:45.843 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:45.843 DEBUG:boto:Method: GET 2022-09-03T06:27:45.844 DEBUG:boto:Path: /dsgvcy-73/c 2022-09-03T06:27:45.844 DEBUG:boto:Data: 2022-09-03T06:27:45.845 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.846 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.846 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.847 DEBUG:boto:Params: {} 2022-09-03T06:27:45.848 DEBUG:boto:Token: None 2022-09-03T06:27:45.849 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/c 2022-09-03T06:27:45.850 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pKcpmPJlQ+yhx76DnX5MduibV2Y= 2022-09-03T06:27:45.851 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pKcpmPJlQ+yhx76DnX5MduibV2Y=', 'Content-Length': '0'} 2022-09-03T06:27:45.853 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000002b5b9c8c69991e2-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.854 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:45.854 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:45.855 DEBUG:boto:path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:45.855 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:45.856 DEBUG:boto:Method: GET 2022-09-03T06:27:45.856 DEBUG:boto:Path: /dsgvcy-73/?prefix=c 2022-09-03T06:27:45.857 DEBUG:boto:Data: 2022-09-03T06:27:45.857 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.858 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.858 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.859 DEBUG:boto:Params: {} 2022-09-03T06:27:45.860 DEBUG:boto:Token: None 2022-09-03T06:27:45.861 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/ 2022-09-03T06:27:45.861 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o= 2022-09-03T06:27:45.862 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o=', 'Content-Length': '0'} 2022-09-03T06:27:45.865 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d7a5fe8847508c6-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.866 DEBUG:boto:b'dsgvcy-73c1000falsec2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:45.867 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:45.868 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:45.870 DEBUG:boto:path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:45.871 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:45.871 DEBUG:boto:Method: GET 2022-09-03T06:27:45.872 DEBUG:boto:Path: /dsgvcy-73/?prefix=c 2022-09-03T06:27:45.873 DEBUG:boto:Data: 2022-09-03T06:27:45.874 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.874 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.875 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.875 DEBUG:boto:Params: {} 2022-09-03T06:27:45.876 DEBUG:boto:Token: None 2022-09-03T06:27:45.876 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/ 2022-09-03T06:27:45.877 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o= 2022-09-03T06:27:45.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o=', 'Content-Length': '0'} 2022-09-03T06:27:45.881 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000578f11c285730a21-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.882 DEBUG:boto:b'dsgvcy-73c1000falsec2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:45.884 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:27:45.885 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:45.886 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:45.886 DEBUG:boto:path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.887 DEBUG:boto:auth_path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.887 DEBUG:boto:Method: GET 2022-09-03T06:27:45.888 DEBUG:boto:Path: /dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.889 DEBUG:boto:Data: 2022-09-03T06:27:45.890 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.892 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.893 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.894 DEBUG:boto:Params: {} 2022-09-03T06:27:45.895 DEBUG:boto:Token: None 2022-09-03T06:27:45.896 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.897 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TXSeNgLR0JrrV1spbu1dl0nChNc= 2022-09-03T06:27:45.898 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TXSeNgLR0JrrV1spbu1dl0nChNc=', 'Content-Length': '0'} 2022-09-03T06:27:45.900 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000076c227b12e710fde-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.902 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:45.903 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:45.904 DEBUG:boto:path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.905 DEBUG:boto:auth_path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.906 DEBUG:boto:Method: GET 2022-09-03T06:27:45.907 DEBUG:boto:Path: /dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.908 DEBUG:boto:Data: 2022-09-03T06:27:45.909 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.910 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.911 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.912 DEBUG:boto:Params: {} 2022-09-03T06:27:45.913 DEBUG:boto:Token: None 2022-09-03T06:27:45.914 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.915 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TXSeNgLR0JrrV1spbu1dl0nChNc= 2022-09-03T06:27:45.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TXSeNgLR0JrrV1spbu1dl0nChNc=', 'Content-Length': '0'} 2022-09-03T06:27:45.919 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e52fa22d9fc5e6df-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.920 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:45.920 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:45.921 DEBUG:boto:path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.922 DEBUG:boto:auth_path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.922 DEBUG:boto:Method: HEAD 2022-09-03T06:27:45.923 DEBUG:boto:Path: /dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.923 DEBUG:boto:Data: 2022-09-03T06:27:45.924 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.924 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.925 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.925 DEBUG:boto:Params: {} 2022-09-03T06:27:45.926 DEBUG:boto:Token: None 2022-09-03T06:27:45.927 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.927 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R68I7aWYlbcL1sT/RnDUeBFemJ4= 2022-09-03T06:27:45.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R68I7aWYlbcL1sT/RnDUeBFemJ4=', 'Content-Length': '0'} 2022-09-03T06:27:45.930 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000962c290901c5dd2e-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.931 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:45.932 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:45.933 DEBUG:boto:path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.933 DEBUG:boto:auth_path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.934 DEBUG:boto:Method: HEAD 2022-09-03T06:27:45.934 DEBUG:boto:Path: /dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.935 DEBUG:boto:Data: 2022-09-03T06:27:45.935 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.936 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.936 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.937 DEBUG:boto:Params: {} 2022-09-03T06:27:45.937 DEBUG:boto:Token: None 2022-09-03T06:27:45.938 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.939 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R68I7aWYlbcL1sT/RnDUeBFemJ4= 2022-09-03T06:27:45.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R68I7aWYlbcL1sT/RnDUeBFemJ4=', 'Content-Length': '0'} 2022-09-03T06:27:45.943 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006bed9cc6c9ea9bd3-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.944 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:27:45.945 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:45.946 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:45.947 DEBUG:boto:path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.948 DEBUG:boto:auth_path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.949 DEBUG:boto:Method: GET 2022-09-03T06:27:45.950 DEBUG:boto:Path: /dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.950 DEBUG:boto:Data: 2022-09-03T06:27:45.951 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.951 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.952 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.953 DEBUG:boto:Params: {} 2022-09-03T06:27:45.953 DEBUG:boto:Token: None 2022-09-03T06:27:45.954 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.955 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TXSeNgLR0JrrV1spbu1dl0nChNc= 2022-09-03T06:27:45.955 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TXSeNgLR0JrrV1spbu1dl0nChNc=', 'Content-Length': '0'} 2022-09-03T06:27:45.958 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ed4f4e042ddc5eb5-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.958 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:45.959 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:45.960 DEBUG:boto:path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.960 DEBUG:boto:auth_path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.961 DEBUG:boto:Method: GET 2022-09-03T06:27:45.961 DEBUG:boto:Path: /dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.962 DEBUG:boto:Data: 2022-09-03T06:27:45.962 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.963 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.963 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.964 DEBUG:boto:Params: {} 2022-09-03T06:27:45.964 DEBUG:boto:Token: None 2022-09-03T06:27:45.965 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/d?versionId=null 2022-09-03T06:27:45.966 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TXSeNgLR0JrrV1spbu1dl0nChNc= 2022-09-03T06:27:45.967 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TXSeNgLR0JrrV1spbu1dl0nChNc=', 'Content-Length': '0'} 2022-09-03T06:27:45.969 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d4cabd12415e899b-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.970 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:45.971 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:45.972 DEBUG:boto:path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:45.974 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:45.975 DEBUG:boto:Method: GET 2022-09-03T06:27:45.975 DEBUG:boto:Path: /dsgvcy-73/?prefix=d 2022-09-03T06:27:45.976 DEBUG:boto:Data: 2022-09-03T06:27:45.977 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.978 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:45.978 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.979 DEBUG:boto:Params: {} 2022-09-03T06:27:45.980 DEBUG:boto:Token: None 2022-09-03T06:27:45.980 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/ 2022-09-03T06:27:45.981 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o= 2022-09-03T06:27:45.982 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o=', 'Content-Length': '0'} 2022-09-03T06:27:45.984 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e7cca9193bb46c20-006312f3e1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:45.985 DEBUG:boto:b'dsgvcy-73d1000falsed2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:45.986 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:45.987 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:45.987 DEBUG:boto:path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:45.988 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:45.988 DEBUG:boto:Method: GET 2022-09-03T06:27:45.989 DEBUG:boto:Path: /dsgvcy-73/?prefix=d 2022-09-03T06:27:45.990 DEBUG:boto:Data: 2022-09-03T06:27:45.991 DEBUG:boto:Headers: {} 2022-09-03T06:27:45.992 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:45.993 DEBUG:boto:Port: 8000 2022-09-03T06:27:45.993 DEBUG:boto:Params: {} 2022-09-03T06:27:45.995 DEBUG:boto:Token: None 2022-09-03T06:27:45.996 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:45 GMT /dsgvcy-73/ 2022-09-03T06:27:45.997 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o= 2022-09-03T06:27:45.998 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:83BI1utyhmaUvDY+DgbRl3ffG8o=', 'Content-Length': '0'} 2022-09-03T06:27:46.001 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000518e444f6015c7bb-006312f3e1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.002 DEBUG:boto:b'dsgvcy-73d1000falsed2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:46.003 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:46.004 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:46.005 DEBUG:boto:Method: HEAD 2022-09-03T06:27:46.005 DEBUG:boto:Path: /dsgvcy-73/d 2022-09-03T06:27:46.006 DEBUG:boto:Data: 2022-09-03T06:27:46.007 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.008 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.009 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.009 DEBUG:boto:Params: {} 2022-09-03T06:27:46.011 DEBUG:boto:Token: None 2022-09-03T06:27:46.012 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/d 2022-09-03T06:27:46.012 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e2a1yKamS7CeUdsVywSzVcH+zMw= 2022-09-03T06:27:46.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e2a1yKamS7CeUdsVywSzVcH+zMw=', 'Content-Length': '0'} 2022-09-03T06:27:46.016 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ef17657e95b6d972-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.017 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:46.018 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:46.019 DEBUG:boto:Method: HEAD 2022-09-03T06:27:46.020 DEBUG:boto:Path: /dsgvcy-73/d 2022-09-03T06:27:46.021 DEBUG:boto:Data: 2022-09-03T06:27:46.022 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.023 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:46.024 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.025 DEBUG:boto:Params: {} 2022-09-03T06:27:46.026 DEBUG:boto:Token: None 2022-09-03T06:27:46.027 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/d 2022-09-03T06:27:46.028 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e2a1yKamS7CeUdsVywSzVcH+zMw= 2022-09-03T06:27:46.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e2a1yKamS7CeUdsVywSzVcH+zMw=', 'Content-Length': '0'} 2022-09-03T06:27:46.032 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000318f854e44b46a53-006312f3e2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.035 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:27:46.036 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:46.037 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:46.040 DEBUG:boto:Method: GET 2022-09-03T06:27:46.041 DEBUG:boto:Path: /dsgvcy-73/d 2022-09-03T06:27:46.042 DEBUG:boto:Data: 2022-09-03T06:27:46.043 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.044 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.045 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.046 DEBUG:boto:Params: {} 2022-09-03T06:27:46.048 DEBUG:boto:Token: None 2022-09-03T06:27:46.049 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/d 2022-09-03T06:27:46.050 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9AExaBpGXj8zBd6AVqgJOHe53LE= 2022-09-03T06:27:46.051 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9AExaBpGXj8zBd6AVqgJOHe53LE=', 'Content-Length': '0'} 2022-09-03T06:27:46.054 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fb48ecdbf44cdf7b-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.056 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:46.057 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:46.058 DEBUG:boto:Method: GET 2022-09-03T06:27:46.059 DEBUG:boto:Path: /dsgvcy-73/d 2022-09-03T06:27:46.060 DEBUG:boto:Data: 2022-09-03T06:27:46.061 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.062 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:46.063 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.064 DEBUG:boto:Params: {} 2022-09-03T06:27:46.065 DEBUG:boto:Token: None 2022-09-03T06:27:46.066 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/d 2022-09-03T06:27:46.067 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9AExaBpGXj8zBd6AVqgJOHe53LE= 2022-09-03T06:27:46.069 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9AExaBpGXj8zBd6AVqgJOHe53LE=', 'Content-Length': '0'} 2022-09-03T06:27:46.071 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007d2ae282a0504b80-006312f3e2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.073 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:46.074 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:46.075 DEBUG:boto:path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:46.076 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:46.077 DEBUG:boto:Method: GET 2022-09-03T06:27:46.078 DEBUG:boto:Path: /dsgvcy-73/?prefix=d 2022-09-03T06:27:46.079 DEBUG:boto:Data: 2022-09-03T06:27:46.080 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.081 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.082 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.083 DEBUG:boto:Params: {} 2022-09-03T06:27:46.084 DEBUG:boto:Token: None 2022-09-03T06:27:46.086 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/ 2022-09-03T06:27:46.087 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90= 2022-09-03T06:27:46.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90=', 'Content-Length': '0'} 2022-09-03T06:27:46.091 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003ac4709b8268df06-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.144 DEBUG:boto:b'dsgvcy-73d1000falsed2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:46.146 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:46.147 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:46.148 DEBUG:boto:path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:46.149 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:46.149 DEBUG:boto:Method: GET 2022-09-03T06:27:46.150 DEBUG:boto:Path: /dsgvcy-73/?prefix=d 2022-09-03T06:27:46.150 DEBUG:boto:Data: 2022-09-03T06:27:46.151 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.152 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:46.152 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.153 DEBUG:boto:Params: {} 2022-09-03T06:27:46.154 DEBUG:boto:Token: None 2022-09-03T06:27:46.155 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/ 2022-09-03T06:27:46.156 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90= 2022-09-03T06:27:46.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90=', 'Content-Length': '0'} 2022-09-03T06:27:46.159 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cbbcaa87efd044c8-006312f3e2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.160 DEBUG:boto:b'dsgvcy-73d1000falsed2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:46.161 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:27:46.162 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:46.162 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:46.163 DEBUG:boto:path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.163 DEBUG:boto:auth_path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.164 DEBUG:boto:Method: GET 2022-09-03T06:27:46.164 DEBUG:boto:Path: /dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.165 DEBUG:boto:Data: 2022-09-03T06:27:46.165 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.166 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.166 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.167 DEBUG:boto:Params: {} 2022-09-03T06:27:46.168 DEBUG:boto:Token: None 2022-09-03T06:27:46.168 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.169 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4mOBR4anMdbwWAyPyP1qlKFND80= 2022-09-03T06:27:46.169 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4mOBR4anMdbwWAyPyP1qlKFND80=', 'Content-Length': '0'} 2022-09-03T06:27:46.171 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002089072806462e3b-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.172 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:46.173 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:46.173 DEBUG:boto:path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.174 DEBUG:boto:auth_path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.174 DEBUG:boto:Method: GET 2022-09-03T06:27:46.176 DEBUG:boto:Path: /dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.177 DEBUG:boto:Data: 2022-09-03T06:27:46.178 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.179 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:46.180 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.181 DEBUG:boto:Params: {} 2022-09-03T06:27:46.182 DEBUG:boto:Token: None 2022-09-03T06:27:46.183 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.184 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4mOBR4anMdbwWAyPyP1qlKFND80= 2022-09-03T06:27:46.187 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4mOBR4anMdbwWAyPyP1qlKFND80=', 'Content-Length': '0'} 2022-09-03T06:27:46.190 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000039b2b0341dd28b28-006312f3e2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.230 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:46.231 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:46.232 DEBUG:boto:path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.232 DEBUG:boto:auth_path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.233 DEBUG:boto:Method: HEAD 2022-09-03T06:27:46.233 DEBUG:boto:Path: /dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.234 DEBUG:boto:Data: 2022-09-03T06:27:46.235 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.236 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.237 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.238 DEBUG:boto:Params: {} 2022-09-03T06:27:46.239 DEBUG:boto:Token: None 2022-09-03T06:27:46.240 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.241 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NetUEWm88OAnzsoMHSn9a0GUw7s= 2022-09-03T06:27:46.242 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NetUEWm88OAnzsoMHSn9a0GUw7s=', 'Content-Length': '0'} 2022-09-03T06:27:46.246 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005679644c9d6b82e8-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.247 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:46.248 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:46.249 DEBUG:boto:path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.249 DEBUG:boto:auth_path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.250 DEBUG:boto:Method: HEAD 2022-09-03T06:27:46.251 DEBUG:boto:Path: /dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.252 DEBUG:boto:Data: 2022-09-03T06:27:46.252 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.253 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:46.253 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.254 DEBUG:boto:Params: {} 2022-09-03T06:27:46.255 DEBUG:boto:Token: None 2022-09-03T06:27:46.255 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.257 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NetUEWm88OAnzsoMHSn9a0GUw7s= 2022-09-03T06:27:46.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NetUEWm88OAnzsoMHSn9a0GUw7s=', 'Content-Length': '0'} 2022-09-03T06:27:46.260 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ae9dd8acbf83a7a9-006312f3e2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.262 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:27:46.263 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:46.264 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:46.265 DEBUG:boto:path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.266 DEBUG:boto:auth_path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.267 DEBUG:boto:Method: GET 2022-09-03T06:27:46.268 DEBUG:boto:Path: /dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.269 DEBUG:boto:Data: 2022-09-03T06:27:46.270 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.271 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.272 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.273 DEBUG:boto:Params: {} 2022-09-03T06:27:46.274 DEBUG:boto:Token: None 2022-09-03T06:27:46.275 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.276 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4mOBR4anMdbwWAyPyP1qlKFND80= 2022-09-03T06:27:46.277 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4mOBR4anMdbwWAyPyP1qlKFND80=', 'Content-Length': '0'} 2022-09-03T06:27:46.279 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000839b5ab88a56f55e-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.281 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:46.282 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:46.283 DEBUG:boto:path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.284 DEBUG:boto:auth_path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.285 DEBUG:boto:Method: GET 2022-09-03T06:27:46.291 DEBUG:boto:Path: /dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.291 DEBUG:boto:Data: 2022-09-03T06:27:46.292 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.293 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:46.293 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.294 DEBUG:boto:Params: {} 2022-09-03T06:27:46.295 DEBUG:boto:Token: None 2022-09-03T06:27:46.295 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/e?versionId=null 2022-09-03T06:27:46.296 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4mOBR4anMdbwWAyPyP1qlKFND80= 2022-09-03T06:27:46.297 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4mOBR4anMdbwWAyPyP1qlKFND80=', 'Content-Length': '0'} 2022-09-03T06:27:46.299 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000089de11d5fd5402d1-006312f3e2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.300 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:46.301 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:46.301 DEBUG:boto:path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:46.302 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:46.303 DEBUG:boto:Method: GET 2022-09-03T06:27:46.303 DEBUG:boto:Path: /dsgvcy-73/?prefix=e 2022-09-03T06:27:46.304 DEBUG:boto:Data: 2022-09-03T06:27:46.304 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.305 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.305 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.306 DEBUG:boto:Params: {} 2022-09-03T06:27:46.307 DEBUG:boto:Token: None 2022-09-03T06:27:46.307 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/ 2022-09-03T06:27:46.308 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90= 2022-09-03T06:27:46.309 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90=', 'Content-Length': '0'} 2022-09-03T06:27:46.312 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004101dde43f17192e-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.313 DEBUG:boto:b'dsgvcy-73e1000falsee2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:46.314 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:46.314 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:46.315 DEBUG:boto:path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:46.315 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:46.316 DEBUG:boto:Method: GET 2022-09-03T06:27:46.316 DEBUG:boto:Path: /dsgvcy-73/?prefix=e 2022-09-03T06:27:46.317 DEBUG:boto:Data: 2022-09-03T06:27:46.317 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.318 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:46.318 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.319 DEBUG:boto:Params: {} 2022-09-03T06:27:46.320 DEBUG:boto:Token: None 2022-09-03T06:27:46.320 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/ 2022-09-03T06:27:46.321 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90= 2022-09-03T06:27:46.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90=', 'Content-Length': '0'} 2022-09-03T06:27:46.325 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000018acbd7d86e659bc-006312f3e2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.325 DEBUG:boto:b'dsgvcy-73e1000falsee2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:46.326 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:46.327 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:46.328 DEBUG:boto:Method: HEAD 2022-09-03T06:27:46.328 DEBUG:boto:Path: /dsgvcy-73/e 2022-09-03T06:27:46.329 DEBUG:boto:Data: 2022-09-03T06:27:46.329 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.330 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.330 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.331 DEBUG:boto:Params: {} 2022-09-03T06:27:46.331 DEBUG:boto:Token: None 2022-09-03T06:27:46.332 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/e 2022-09-03T06:27:46.333 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m5EXV5Ldvu6spY/R4IGED68SKZ0= 2022-09-03T06:27:46.333 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m5EXV5Ldvu6spY/R4IGED68SKZ0=', 'Content-Length': '0'} 2022-09-03T06:27:46.335 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e461e0839a31ece8-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.336 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:46.337 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:46.337 DEBUG:boto:Method: HEAD 2022-09-03T06:27:46.338 DEBUG:boto:Path: /dsgvcy-73/e 2022-09-03T06:27:46.338 DEBUG:boto:Data: 2022-09-03T06:27:46.339 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.339 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:46.340 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.341 DEBUG:boto:Params: {} 2022-09-03T06:27:46.341 DEBUG:boto:Token: None 2022-09-03T06:27:46.342 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/e 2022-09-03T06:27:46.342 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m5EXV5Ldvu6spY/R4IGED68SKZ0= 2022-09-03T06:27:46.343 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m5EXV5Ldvu6spY/R4IGED68SKZ0=', 'Content-Length': '0'} 2022-09-03T06:27:46.345 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007750883db300de8d-006312f3e2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.346 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:27:46.347 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:46.347 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:46.348 DEBUG:boto:Method: GET 2022-09-03T06:27:46.349 DEBUG:boto:Path: /dsgvcy-73/e 2022-09-03T06:27:46.350 DEBUG:boto:Data: 2022-09-03T06:27:46.351 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.352 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.353 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.354 DEBUG:boto:Params: {} 2022-09-03T06:27:46.355 DEBUG:boto:Token: None 2022-09-03T06:27:46.356 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/e 2022-09-03T06:27:46.356 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KInq01/8i6k1usYB4bTzCt/iFUg= 2022-09-03T06:27:46.357 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KInq01/8i6k1usYB4bTzCt/iFUg=', 'Content-Length': '0'} 2022-09-03T06:27:46.359 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000422f6533e27de5de-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.360 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:46.361 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:46.361 DEBUG:boto:Method: GET 2022-09-03T06:27:46.362 DEBUG:boto:Path: /dsgvcy-73/e 2022-09-03T06:27:46.362 DEBUG:boto:Data: 2022-09-03T06:27:46.363 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.364 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:46.364 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.365 DEBUG:boto:Params: {} 2022-09-03T06:27:46.366 DEBUG:boto:Token: None 2022-09-03T06:27:46.367 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/e 2022-09-03T06:27:46.368 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KInq01/8i6k1usYB4bTzCt/iFUg= 2022-09-03T06:27:46.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KInq01/8i6k1usYB4bTzCt/iFUg=', 'Content-Length': '0'} 2022-09-03T06:27:46.372 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000112afa39f4785606-006312f3e2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.373 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:46.374 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:46.374 DEBUG:boto:path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:46.375 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:46.376 DEBUG:boto:Method: GET 2022-09-03T06:27:46.377 DEBUG:boto:Path: /dsgvcy-73/?prefix=e 2022-09-03T06:27:46.377 DEBUG:boto:Data: 2022-09-03T06:27:46.378 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.378 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.379 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.379 DEBUG:boto:Params: {} 2022-09-03T06:27:46.380 DEBUG:boto:Token: None 2022-09-03T06:27:46.381 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/ 2022-09-03T06:27:46.381 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90= 2022-09-03T06:27:46.382 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90=', 'Content-Length': '0'} 2022-09-03T06:27:46.392 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003927be695bedc4eb-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.394 DEBUG:boto:b'dsgvcy-73e1000falsee2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:46.394 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:46.395 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:46.396 DEBUG:boto:path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:46.397 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:46.398 DEBUG:boto:Method: GET 2022-09-03T06:27:46.399 DEBUG:boto:Path: /dsgvcy-73/?prefix=e 2022-09-03T06:27:46.400 DEBUG:boto:Data: 2022-09-03T06:27:46.401 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.401 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:46.402 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.403 DEBUG:boto:Params: {} 2022-09-03T06:27:46.403 DEBUG:boto:Token: None 2022-09-03T06:27:46.404 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/ 2022-09-03T06:27:46.404 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90= 2022-09-03T06:27:46.405 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90=', 'Content-Length': '0'} 2022-09-03T06:27:46.409 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000940cdc7b608d519a-006312f3e2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.412 DEBUG:boto:b'dsgvcy-73e1000falsee2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:46.413 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:27:46.414 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:46.415 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:46.416 DEBUG:boto:path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.417 DEBUG:boto:auth_path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.417 DEBUG:boto:Method: GET 2022-09-03T06:27:46.418 DEBUG:boto:Path: /dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.419 DEBUG:boto:Data: 2022-09-03T06:27:46.419 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.420 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.420 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.421 DEBUG:boto:Params: {} 2022-09-03T06:27:46.422 DEBUG:boto:Token: None 2022-09-03T06:27:46.422 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.423 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pDETrhVmj4NAVE6mbnjaTv16Ln8= 2022-09-03T06:27:46.423 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pDETrhVmj4NAVE6mbnjaTv16Ln8=', 'Content-Length': '0'} 2022-09-03T06:27:46.426 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000ef7219f49a4a79b-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.466 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:46.467 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:46.468 DEBUG:boto:path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.469 DEBUG:boto:auth_path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.469 DEBUG:boto:Method: GET 2022-09-03T06:27:46.470 DEBUG:boto:Path: /dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.470 DEBUG:boto:Data: 2022-09-03T06:27:46.471 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.472 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:46.473 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.474 DEBUG:boto:Params: {} 2022-09-03T06:27:46.475 DEBUG:boto:Token: None 2022-09-03T06:27:46.475 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.477 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pDETrhVmj4NAVE6mbnjaTv16Ln8= 2022-09-03T06:27:46.478 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pDETrhVmj4NAVE6mbnjaTv16Ln8=', 'Content-Length': '0'} 2022-09-03T06:27:46.480 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d9b585d0d1fc38c-006312f3e2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.481 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:46.482 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:46.483 DEBUG:boto:path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.483 DEBUG:boto:auth_path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.484 DEBUG:boto:Method: HEAD 2022-09-03T06:27:46.485 DEBUG:boto:Path: /dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.485 DEBUG:boto:Data: 2022-09-03T06:27:46.486 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.487 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.488 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.488 DEBUG:boto:Params: {} 2022-09-03T06:27:46.489 DEBUG:boto:Token: None 2022-09-03T06:27:46.491 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.492 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FuORx+3GELOaDFHyD9BfNTyMfKE= 2022-09-03T06:27:46.493 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FuORx+3GELOaDFHyD9BfNTyMfKE=', 'Content-Length': '0'} 2022-09-03T06:27:46.496 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000481164773b713fea-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.498 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:46.499 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:46.500 DEBUG:boto:path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.500 DEBUG:boto:auth_path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.501 DEBUG:boto:Method: HEAD 2022-09-03T06:27:46.502 DEBUG:boto:Path: /dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.503 DEBUG:boto:Data: 2022-09-03T06:27:46.503 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.504 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:46.504 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.505 DEBUG:boto:Params: {} 2022-09-03T06:27:46.506 DEBUG:boto:Token: None 2022-09-03T06:27:46.507 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.508 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FuORx+3GELOaDFHyD9BfNTyMfKE= 2022-09-03T06:27:46.509 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FuORx+3GELOaDFHyD9BfNTyMfKE=', 'Content-Length': '0'} 2022-09-03T06:27:46.511 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002eaf85f7d04b54df-006312f3e2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.521 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:27:46.522 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:46.523 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:46.526 DEBUG:boto:path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.527 DEBUG:boto:auth_path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.528 DEBUG:boto:Method: GET 2022-09-03T06:27:46.529 DEBUG:boto:Path: /dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.530 DEBUG:boto:Data: 2022-09-03T06:27:46.531 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.532 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.533 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.535 DEBUG:boto:Params: {} 2022-09-03T06:27:46.536 DEBUG:boto:Token: None 2022-09-03T06:27:46.537 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.539 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pDETrhVmj4NAVE6mbnjaTv16Ln8= 2022-09-03T06:27:46.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pDETrhVmj4NAVE6mbnjaTv16Ln8=', 'Content-Length': '0'} 2022-09-03T06:27:46.543 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000054641aea1eb6e2cc-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.544 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:46.546 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:46.546 DEBUG:boto:path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.547 DEBUG:boto:auth_path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.548 DEBUG:boto:Method: GET 2022-09-03T06:27:46.548 DEBUG:boto:Path: /dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.549 DEBUG:boto:Data: 2022-09-03T06:27:46.549 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.550 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:46.551 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.551 DEBUG:boto:Params: {} 2022-09-03T06:27:46.552 DEBUG:boto:Token: None 2022-09-03T06:27:46.553 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/f?versionId=null 2022-09-03T06:27:46.553 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pDETrhVmj4NAVE6mbnjaTv16Ln8= 2022-09-03T06:27:46.554 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pDETrhVmj4NAVE6mbnjaTv16Ln8=', 'Content-Length': '0'} 2022-09-03T06:27:46.556 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e5e19247e1129d19-006312f3e2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.557 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:46.558 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:46.559 DEBUG:boto:path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:46.560 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:46.561 DEBUG:boto:Method: GET 2022-09-03T06:27:46.562 DEBUG:boto:Path: /dsgvcy-73/?prefix=f 2022-09-03T06:27:46.563 DEBUG:boto:Data: 2022-09-03T06:27:46.564 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.565 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.566 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.567 DEBUG:boto:Params: {} 2022-09-03T06:27:46.570 DEBUG:boto:Token: None 2022-09-03T06:27:46.571 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/ 2022-09-03T06:27:46.572 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90= 2022-09-03T06:27:46.573 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90=', 'Content-Length': '0'} 2022-09-03T06:27:46.576 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a06f0e80348862c0-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.577 DEBUG:boto:b'dsgvcy-73f1000falsef2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:46.578 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:46.579 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:46.579 DEBUG:boto:path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:46.580 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:46.580 DEBUG:boto:Method: GET 2022-09-03T06:27:46.582 DEBUG:boto:Path: /dsgvcy-73/?prefix=f 2022-09-03T06:27:46.582 DEBUG:boto:Data: 2022-09-03T06:27:46.583 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.583 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:46.584 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.584 DEBUG:boto:Params: {} 2022-09-03T06:27:46.585 DEBUG:boto:Token: None 2022-09-03T06:27:46.586 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/ 2022-09-03T06:27:46.586 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90= 2022-09-03T06:27:46.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90=', 'Content-Length': '0'} 2022-09-03T06:27:46.590 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e0114c289efc752d-006312f3e2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.591 DEBUG:boto:b'dsgvcy-73f1000falsef2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:46.592 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:46.593 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:46.593 DEBUG:boto:Method: HEAD 2022-09-03T06:27:46.594 DEBUG:boto:Path: /dsgvcy-73/f 2022-09-03T06:27:46.594 DEBUG:boto:Data: 2022-09-03T06:27:46.595 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.595 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.598 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.599 DEBUG:boto:Params: {} 2022-09-03T06:27:46.601 DEBUG:boto:Token: None 2022-09-03T06:27:46.601 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/f 2022-09-03T06:27:46.602 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4BYhcUMpuV8USIRpw4hqC25IrVo= 2022-09-03T06:27:46.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4BYhcUMpuV8USIRpw4hqC25IrVo=', 'Content-Length': '0'} 2022-09-03T06:27:46.605 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cb30a6d4a17257d5-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.605 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:46.607 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:46.607 DEBUG:boto:Method: HEAD 2022-09-03T06:27:46.608 DEBUG:boto:Path: /dsgvcy-73/f 2022-09-03T06:27:46.608 DEBUG:boto:Data: 2022-09-03T06:27:46.609 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.609 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:46.610 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.611 DEBUG:boto:Params: {} 2022-09-03T06:27:46.612 DEBUG:boto:Token: None 2022-09-03T06:27:46.613 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/f 2022-09-03T06:27:46.613 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4BYhcUMpuV8USIRpw4hqC25IrVo= 2022-09-03T06:27:46.614 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4BYhcUMpuV8USIRpw4hqC25IrVo=', 'Content-Length': '0'} 2022-09-03T06:27:46.616 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d2eea59517bd98b-006312f3e2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.616 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:27:46.617 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:46.618 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:46.618 DEBUG:boto:Method: GET 2022-09-03T06:27:46.619 DEBUG:boto:Path: /dsgvcy-73/f 2022-09-03T06:27:46.619 DEBUG:boto:Data: 2022-09-03T06:27:46.620 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.621 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.621 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.623 DEBUG:boto:Params: {} 2022-09-03T06:27:46.624 DEBUG:boto:Token: None 2022-09-03T06:27:46.624 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/f 2022-09-03T06:27:46.625 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w80ZcGlJ2jwPHSv2HwSOW+Xioaw= 2022-09-03T06:27:46.626 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w80ZcGlJ2jwPHSv2HwSOW+Xioaw=', 'Content-Length': '0'} 2022-09-03T06:27:46.628 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003361fc637e7a9743-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.629 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:46.629 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:46.630 DEBUG:boto:Method: GET 2022-09-03T06:27:46.631 DEBUG:boto:Path: /dsgvcy-73/f 2022-09-03T06:27:46.631 DEBUG:boto:Data: 2022-09-03T06:27:46.632 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.632 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:46.633 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.633 DEBUG:boto:Params: {} 2022-09-03T06:27:46.634 DEBUG:boto:Token: None 2022-09-03T06:27:46.635 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/f 2022-09-03T06:27:46.635 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w80ZcGlJ2jwPHSv2HwSOW+Xioaw= 2022-09-03T06:27:46.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w80ZcGlJ2jwPHSv2HwSOW+Xioaw=', 'Content-Length': '0'} 2022-09-03T06:27:46.638 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000058b25777e6511243-006312f3e2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.640 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:46.640 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:46.641 DEBUG:boto:path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:46.642 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:46.642 DEBUG:boto:Method: GET 2022-09-03T06:27:46.643 DEBUG:boto:Path: /dsgvcy-73/?prefix=f 2022-09-03T06:27:46.644 DEBUG:boto:Data: 2022-09-03T06:27:46.644 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.645 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.645 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.646 DEBUG:boto:Params: {} 2022-09-03T06:27:46.647 DEBUG:boto:Token: None 2022-09-03T06:27:46.647 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/ 2022-09-03T06:27:46.648 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90= 2022-09-03T06:27:46.649 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90=', 'Content-Length': '0'} 2022-09-03T06:27:46.651 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e3d0519bf3fe26e8-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.652 DEBUG:boto:b'dsgvcy-73f1000falsef2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:46.653 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:46.653 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:46.654 DEBUG:boto:path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:46.655 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:46.655 DEBUG:boto:Method: GET 2022-09-03T06:27:46.656 DEBUG:boto:Path: /dsgvcy-73/?prefix=f 2022-09-03T06:27:46.656 DEBUG:boto:Data: 2022-09-03T06:27:46.657 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.657 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:46.658 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.659 DEBUG:boto:Params: {} 2022-09-03T06:27:46.659 DEBUG:boto:Token: None 2022-09-03T06:27:46.660 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/ 2022-09-03T06:27:46.661 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90= 2022-09-03T06:27:46.661 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90=', 'Content-Length': '0'} 2022-09-03T06:27:46.664 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ecb404ed0a927870-006312f3e2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.710 DEBUG:boto:b'dsgvcy-73f1000falsef2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:46.712 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:27:46.712 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:46.713 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:46.713 DEBUG:boto:path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.714 DEBUG:boto:auth_path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.715 DEBUG:boto:Method: GET 2022-09-03T06:27:46.718 DEBUG:boto:Path: /dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.719 DEBUG:boto:Data: 2022-09-03T06:27:46.720 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.722 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.723 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.724 DEBUG:boto:Params: {} 2022-09-03T06:27:46.725 DEBUG:boto:Token: None 2022-09-03T06:27:46.726 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.727 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oe9c/91J7ChoHrAPt+dBCRRIIIk= 2022-09-03T06:27:46.743 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oe9c/91J7ChoHrAPt+dBCRRIIIk=', 'Content-Length': '0'} 2022-09-03T06:27:46.749 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000088b381665f1d32e4-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.751 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:46.752 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:46.753 DEBUG:boto:path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.754 DEBUG:boto:auth_path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.756 DEBUG:boto:Method: GET 2022-09-03T06:27:46.757 DEBUG:boto:Path: /dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.758 DEBUG:boto:Data: 2022-09-03T06:27:46.759 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.760 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:46.761 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.762 DEBUG:boto:Params: {} 2022-09-03T06:27:46.763 DEBUG:boto:Token: None 2022-09-03T06:27:46.764 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.766 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oe9c/91J7ChoHrAPt+dBCRRIIIk= 2022-09-03T06:27:46.767 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oe9c/91J7ChoHrAPt+dBCRRIIIk=', 'Content-Length': '0'} 2022-09-03T06:27:46.770 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af7889e6e76bd2f3-006312f3e2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.772 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:46.773 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:46.774 DEBUG:boto:path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.775 DEBUG:boto:auth_path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.776 DEBUG:boto:Method: HEAD 2022-09-03T06:27:46.777 DEBUG:boto:Path: /dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.778 DEBUG:boto:Data: 2022-09-03T06:27:46.779 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.780 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.781 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.782 DEBUG:boto:Params: {} 2022-09-03T06:27:46.783 DEBUG:boto:Token: None 2022-09-03T06:27:46.784 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.785 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZYiq+zxlByuFMFlPDFF4si6jVJ0= 2022-09-03T06:27:46.786 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZYiq+zxlByuFMFlPDFF4si6jVJ0=', 'Content-Length': '0'} 2022-09-03T06:27:46.789 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2c52c5192b04f36-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.790 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:46.790 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:46.791 DEBUG:boto:path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.791 DEBUG:boto:auth_path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.792 DEBUG:boto:Method: HEAD 2022-09-03T06:27:46.792 DEBUG:boto:Path: /dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.793 DEBUG:boto:Data: 2022-09-03T06:27:46.793 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.794 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:46.794 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.795 DEBUG:boto:Params: {} 2022-09-03T06:27:46.796 DEBUG:boto:Token: None 2022-09-03T06:27:46.796 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.797 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZYiq+zxlByuFMFlPDFF4si6jVJ0= 2022-09-03T06:27:46.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZYiq+zxlByuFMFlPDFF4si6jVJ0=', 'Content-Length': '0'} 2022-09-03T06:27:46.800 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000050136af4955960e8-006312f3e2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.801 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:27:46.801 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:46.802 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:46.802 DEBUG:boto:path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.803 DEBUG:boto:auth_path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.804 DEBUG:boto:Method: GET 2022-09-03T06:27:46.804 DEBUG:boto:Path: /dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.805 DEBUG:boto:Data: 2022-09-03T06:27:46.806 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.806 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.807 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.807 DEBUG:boto:Params: {} 2022-09-03T06:27:46.808 DEBUG:boto:Token: None 2022-09-03T06:27:46.809 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.810 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oe9c/91J7ChoHrAPt+dBCRRIIIk= 2022-09-03T06:27:46.811 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oe9c/91J7ChoHrAPt+dBCRRIIIk=', 'Content-Length': '0'} 2022-09-03T06:27:46.813 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000074e30761f66ad40-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.814 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:46.815 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:46.815 DEBUG:boto:path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.816 DEBUG:boto:auth_path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.817 DEBUG:boto:Method: GET 2022-09-03T06:27:46.818 DEBUG:boto:Path: /dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.819 DEBUG:boto:Data: 2022-09-03T06:27:46.820 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.821 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:46.835 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.836 DEBUG:boto:Params: {} 2022-09-03T06:27:46.837 DEBUG:boto:Token: None 2022-09-03T06:27:46.837 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/g?versionId=null 2022-09-03T06:27:46.838 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oe9c/91J7ChoHrAPt+dBCRRIIIk= 2022-09-03T06:27:46.839 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oe9c/91J7ChoHrAPt+dBCRRIIIk=', 'Content-Length': '0'} 2022-09-03T06:27:46.843 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f3cff4d16467c548-006312f3e2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.844 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:46.845 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:46.846 DEBUG:boto:path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:46.847 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:46.849 DEBUG:boto:Method: GET 2022-09-03T06:27:46.849 DEBUG:boto:Path: /dsgvcy-73/?prefix=g 2022-09-03T06:27:46.852 DEBUG:boto:Data: 2022-09-03T06:27:46.854 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.855 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.856 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.857 DEBUG:boto:Params: {} 2022-09-03T06:27:46.859 DEBUG:boto:Token: None 2022-09-03T06:27:46.860 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/ 2022-09-03T06:27:46.864 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90= 2022-09-03T06:27:46.866 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90=', 'Content-Length': '0'} 2022-09-03T06:27:46.879 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a1ce9ab47ee5c8d6-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.880 DEBUG:boto:b'dsgvcy-73g1000falseg2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:46.882 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:46.884 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:46.885 DEBUG:boto:path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:46.886 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:46.887 DEBUG:boto:Method: GET 2022-09-03T06:27:46.889 DEBUG:boto:Path: /dsgvcy-73/?prefix=g 2022-09-03T06:27:46.890 DEBUG:boto:Data: 2022-09-03T06:27:46.891 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.892 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:46.892 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.893 DEBUG:boto:Params: {} 2022-09-03T06:27:46.895 DEBUG:boto:Token: None 2022-09-03T06:27:46.897 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/ 2022-09-03T06:27:46.898 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90= 2022-09-03T06:27:46.899 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90=', 'Content-Length': '0'} 2022-09-03T06:27:46.902 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005896795c9b9acbb3-006312f3e2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.904 DEBUG:boto:b'dsgvcy-73g1000falseg2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:46.906 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:46.907 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:46.908 DEBUG:boto:Method: HEAD 2022-09-03T06:27:46.909 DEBUG:boto:Path: /dsgvcy-73/g 2022-09-03T06:27:46.910 DEBUG:boto:Data: 2022-09-03T06:27:46.911 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.912 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.913 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.914 DEBUG:boto:Params: {} 2022-09-03T06:27:46.916 DEBUG:boto:Token: None 2022-09-03T06:27:46.917 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/g 2022-09-03T06:27:46.918 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mq30LPA4dTBm0lmi8LXlUZtxb7U= 2022-09-03T06:27:46.919 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mq30LPA4dTBm0lmi8LXlUZtxb7U=', 'Content-Length': '0'} 2022-09-03T06:27:46.922 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b31550f3ac98c5c4-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.924 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:46.925 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:46.925 DEBUG:boto:Method: HEAD 2022-09-03T06:27:46.926 DEBUG:boto:Path: /dsgvcy-73/g 2022-09-03T06:27:46.927 DEBUG:boto:Data: 2022-09-03T06:27:46.928 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.929 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:46.930 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.932 DEBUG:boto:Params: {} 2022-09-03T06:27:46.934 DEBUG:boto:Token: None 2022-09-03T06:27:46.935 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/g 2022-09-03T06:27:46.936 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mq30LPA4dTBm0lmi8LXlUZtxb7U= 2022-09-03T06:27:46.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mq30LPA4dTBm0lmi8LXlUZtxb7U=', 'Content-Length': '0'} 2022-09-03T06:27:46.940 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007a25eeeb383b581c-006312f3e2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.942 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:27:46.943 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:46.944 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:46.945 DEBUG:boto:Method: GET 2022-09-03T06:27:46.946 DEBUG:boto:Path: /dsgvcy-73/g 2022-09-03T06:27:46.947 DEBUG:boto:Data: 2022-09-03T06:27:46.948 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.948 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.949 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.949 DEBUG:boto:Params: {} 2022-09-03T06:27:46.951 DEBUG:boto:Token: None 2022-09-03T06:27:46.951 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/g 2022-09-03T06:27:46.952 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/zUbj+/D6p7iLDYR5wGlinvQmN0= 2022-09-03T06:27:46.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/zUbj+/D6p7iLDYR5wGlinvQmN0=', 'Content-Length': '0'} 2022-09-03T06:27:46.955 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e275a342288fb180-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.956 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:46.957 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:46.957 DEBUG:boto:Method: GET 2022-09-03T06:27:46.958 DEBUG:boto:Path: /dsgvcy-73/g 2022-09-03T06:27:46.959 DEBUG:boto:Data: 2022-09-03T06:27:46.959 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.960 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:46.960 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.961 DEBUG:boto:Params: {} 2022-09-03T06:27:46.962 DEBUG:boto:Token: None 2022-09-03T06:27:46.963 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/g 2022-09-03T06:27:46.964 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/zUbj+/D6p7iLDYR5wGlinvQmN0= 2022-09-03T06:27:46.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/zUbj+/D6p7iLDYR5wGlinvQmN0=', 'Content-Length': '0'} 2022-09-03T06:27:46.968 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000003e98d95a4f6b360-006312f3e2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.970 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:46.972 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:46.973 DEBUG:boto:path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:46.977 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:46.979 DEBUG:boto:Method: GET 2022-09-03T06:27:46.980 DEBUG:boto:Path: /dsgvcy-73/?prefix=g 2022-09-03T06:27:46.980 DEBUG:boto:Data: 2022-09-03T06:27:46.981 DEBUG:boto:Headers: {} 2022-09-03T06:27:46.982 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:46.982 DEBUG:boto:Port: 8000 2022-09-03T06:27:46.983 DEBUG:boto:Params: {} 2022-09-03T06:27:46.984 DEBUG:boto:Token: None 2022-09-03T06:27:46.985 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:46 GMT /dsgvcy-73/ 2022-09-03T06:27:46.985 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90= 2022-09-03T06:27:46.986 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mua2njDEVpWKh4DcAbxcUH0lw90=', 'Content-Length': '0'} 2022-09-03T06:27:46.990 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dda43ee61dad95f7-006312f3e2-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:46.991 DEBUG:boto:b'dsgvcy-73g1000falseg2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:46.992 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:46.993 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:46.994 DEBUG:boto:path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:46.995 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:46.995 DEBUG:boto:Method: GET 2022-09-03T06:27:46.996 DEBUG:boto:Path: /dsgvcy-73/?prefix=g 2022-09-03T06:27:46.997 DEBUG:boto:Data: 2022-09-03T06:27:46.998 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.000 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:47.000 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.001 DEBUG:boto:Params: {} 2022-09-03T06:27:47.003 DEBUG:boto:Token: None 2022-09-03T06:27:47.004 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/ 2022-09-03T06:27:47.005 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:31VIOOe4VJCqlNerexAEtt/lq/U= 2022-09-03T06:27:47.006 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:31VIOOe4VJCqlNerexAEtt/lq/U=', 'Content-Length': '0'} 2022-09-03T06:27:47.009 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fa13006302b49843-006312f3e3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.054 DEBUG:boto:b'dsgvcy-73g1000falseg2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:47.056 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:27:47.057 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:47.058 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:47.064 DEBUG:boto:path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.065 DEBUG:boto:auth_path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.066 DEBUG:boto:Method: GET 2022-09-03T06:27:47.067 DEBUG:boto:Path: /dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.068 DEBUG:boto:Data: 2022-09-03T06:27:47.069 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.070 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:47.071 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.072 DEBUG:boto:Params: {} 2022-09-03T06:27:47.073 DEBUG:boto:Token: None 2022-09-03T06:27:47.074 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.075 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OMVdM4IlXHOo7Nh1UYJYTWQ+6/A= 2022-09-03T06:27:47.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OMVdM4IlXHOo7Nh1UYJYTWQ+6/A=', 'Content-Length': '0'} 2022-09-03T06:27:47.078 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000067369f91c97346ab-006312f3e3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.079 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:47.080 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:47.081 DEBUG:boto:path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.081 DEBUG:boto:auth_path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.082 DEBUG:boto:Method: GET 2022-09-03T06:27:47.083 DEBUG:boto:Path: /dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.084 DEBUG:boto:Data: 2022-09-03T06:27:47.084 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.085 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:47.086 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.087 DEBUG:boto:Params: {} 2022-09-03T06:27:47.087 DEBUG:boto:Token: None 2022-09-03T06:27:47.088 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.089 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OMVdM4IlXHOo7Nh1UYJYTWQ+6/A= 2022-09-03T06:27:47.089 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OMVdM4IlXHOo7Nh1UYJYTWQ+6/A=', 'Content-Length': '0'} 2022-09-03T06:27:47.092 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003fd12b6bfbaf95f9-006312f3e3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.092 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:47.093 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:47.094 DEBUG:boto:path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.094 DEBUG:boto:auth_path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.095 DEBUG:boto:Method: HEAD 2022-09-03T06:27:47.095 DEBUG:boto:Path: /dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.096 DEBUG:boto:Data: 2022-09-03T06:27:47.096 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.097 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:47.098 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.098 DEBUG:boto:Params: {} 2022-09-03T06:27:47.099 DEBUG:boto:Token: None 2022-09-03T06:27:47.100 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.100 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gaq+uwineXXtkwM1R30msCKlMj0= 2022-09-03T06:27:47.102 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gaq+uwineXXtkwM1R30msCKlMj0=', 'Content-Length': '0'} 2022-09-03T06:27:47.109 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2204c7f847e1f41-006312f3e3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.110 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:47.111 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:47.112 DEBUG:boto:path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.112 DEBUG:boto:auth_path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.113 DEBUG:boto:Method: HEAD 2022-09-03T06:27:47.114 DEBUG:boto:Path: /dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.115 DEBUG:boto:Data: 2022-09-03T06:27:47.115 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.116 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:47.116 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.117 DEBUG:boto:Params: {} 2022-09-03T06:27:47.118 DEBUG:boto:Token: None 2022-09-03T06:27:47.118 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.119 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gaq+uwineXXtkwM1R30msCKlMj0= 2022-09-03T06:27:47.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gaq+uwineXXtkwM1R30msCKlMj0=', 'Content-Length': '0'} 2022-09-03T06:27:47.121 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006fb521737a7b80b6-006312f3e3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.122 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:27:47.123 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:47.124 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:47.124 DEBUG:boto:path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.125 DEBUG:boto:auth_path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.126 DEBUG:boto:Method: GET 2022-09-03T06:27:47.126 DEBUG:boto:Path: /dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.127 DEBUG:boto:Data: 2022-09-03T06:27:47.128 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.128 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:47.129 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.130 DEBUG:boto:Params: {} 2022-09-03T06:27:47.132 DEBUG:boto:Token: None 2022-09-03T06:27:47.132 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.133 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OMVdM4IlXHOo7Nh1UYJYTWQ+6/A= 2022-09-03T06:27:47.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OMVdM4IlXHOo7Nh1UYJYTWQ+6/A=', 'Content-Length': '0'} 2022-09-03T06:27:47.137 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000079c7188a951dbcb0-006312f3e3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.178 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:47.179 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:47.180 DEBUG:boto:path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.180 DEBUG:boto:auth_path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.181 DEBUG:boto:Method: GET 2022-09-03T06:27:47.182 DEBUG:boto:Path: /dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.182 DEBUG:boto:Data: 2022-09-03T06:27:47.183 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.184 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:47.185 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.185 DEBUG:boto:Params: {} 2022-09-03T06:27:47.186 DEBUG:boto:Token: None 2022-09-03T06:27:47.187 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/h?versionId=null 2022-09-03T06:27:47.188 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OMVdM4IlXHOo7Nh1UYJYTWQ+6/A= 2022-09-03T06:27:47.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OMVdM4IlXHOo7Nh1UYJYTWQ+6/A=', 'Content-Length': '0'} 2022-09-03T06:27:47.191 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000031696a02b9b8a233-006312f3e3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.192 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:47.192 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:47.193 DEBUG:boto:path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:47.193 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:47.195 DEBUG:boto:Method: GET 2022-09-03T06:27:47.196 DEBUG:boto:Path: /dsgvcy-73/?prefix=h 2022-09-03T06:27:47.196 DEBUG:boto:Data: 2022-09-03T06:27:47.197 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.198 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:47.198 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.200 DEBUG:boto:Params: {} 2022-09-03T06:27:47.201 DEBUG:boto:Token: None 2022-09-03T06:27:47.202 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/ 2022-09-03T06:27:47.203 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:31VIOOe4VJCqlNerexAEtt/lq/U= 2022-09-03T06:27:47.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:31VIOOe4VJCqlNerexAEtt/lq/U=', 'Content-Length': '0'} 2022-09-03T06:27:47.207 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000271d5d625546cfbe-006312f3e3-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.208 DEBUG:boto:b'dsgvcy-73h1000falseh2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:47.211 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:47.212 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:47.213 DEBUG:boto:path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:47.214 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:47.216 DEBUG:boto:Method: GET 2022-09-03T06:27:47.217 DEBUG:boto:Path: /dsgvcy-73/?prefix=h 2022-09-03T06:27:47.217 DEBUG:boto:Data: 2022-09-03T06:27:47.218 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.220 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:47.221 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.222 DEBUG:boto:Params: {} 2022-09-03T06:27:47.223 DEBUG:boto:Token: None 2022-09-03T06:27:47.224 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/ 2022-09-03T06:27:47.224 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:31VIOOe4VJCqlNerexAEtt/lq/U= 2022-09-03T06:27:47.225 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:31VIOOe4VJCqlNerexAEtt/lq/U=', 'Content-Length': '0'} 2022-09-03T06:27:47.228 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b1e2a9e5e8789223-006312f3e3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.228 DEBUG:boto:b'dsgvcy-73h1000falseh2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:47.229 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:47.230 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:47.230 DEBUG:boto:Method: HEAD 2022-09-03T06:27:47.231 DEBUG:boto:Path: /dsgvcy-73/h 2022-09-03T06:27:47.231 DEBUG:boto:Data: 2022-09-03T06:27:47.233 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.233 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:47.234 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.234 DEBUG:boto:Params: {} 2022-09-03T06:27:47.235 DEBUG:boto:Token: None 2022-09-03T06:27:47.236 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/h 2022-09-03T06:27:47.236 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zKFfn9fFJ+zuV7LmghWw6/VIRyA= 2022-09-03T06:27:47.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zKFfn9fFJ+zuV7LmghWw6/VIRyA=', 'Content-Length': '0'} 2022-09-03T06:27:47.239 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000016ca9924c2ee8cb9-006312f3e3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.240 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:47.241 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:47.241 DEBUG:boto:Method: HEAD 2022-09-03T06:27:47.242 DEBUG:boto:Path: /dsgvcy-73/h 2022-09-03T06:27:47.242 DEBUG:boto:Data: 2022-09-03T06:27:47.243 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.243 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:47.244 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.244 DEBUG:boto:Params: {} 2022-09-03T06:27:47.245 DEBUG:boto:Token: None 2022-09-03T06:27:47.246 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/h 2022-09-03T06:27:47.246 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zKFfn9fFJ+zuV7LmghWw6/VIRyA= 2022-09-03T06:27:47.247 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zKFfn9fFJ+zuV7LmghWw6/VIRyA=', 'Content-Length': '0'} 2022-09-03T06:27:47.250 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000be5dd28689dcc07b-006312f3e3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.251 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:27:47.251 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:47.252 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:47.253 DEBUG:boto:Method: GET 2022-09-03T06:27:47.253 DEBUG:boto:Path: /dsgvcy-73/h 2022-09-03T06:27:47.254 DEBUG:boto:Data: 2022-09-03T06:27:47.254 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.255 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:47.256 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.256 DEBUG:boto:Params: {} 2022-09-03T06:27:47.257 DEBUG:boto:Token: None 2022-09-03T06:27:47.258 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/h 2022-09-03T06:27:47.258 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S3GoZDhbtNWdjdJ3BujMz1qd9Ho= 2022-09-03T06:27:47.259 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S3GoZDhbtNWdjdJ3BujMz1qd9Ho=', 'Content-Length': '0'} 2022-09-03T06:27:47.262 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ec9419232baff1eb-006312f3e3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.302 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:47.303 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:47.304 DEBUG:boto:Method: GET 2022-09-03T06:27:47.305 DEBUG:boto:Path: /dsgvcy-73/h 2022-09-03T06:27:47.306 DEBUG:boto:Data: 2022-09-03T06:27:47.307 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.308 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:47.309 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.318 DEBUG:boto:Params: {} 2022-09-03T06:27:47.319 DEBUG:boto:Token: None 2022-09-03T06:27:47.320 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/h 2022-09-03T06:27:47.321 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S3GoZDhbtNWdjdJ3BujMz1qd9Ho= 2022-09-03T06:27:47.322 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S3GoZDhbtNWdjdJ3BujMz1qd9Ho=', 'Content-Length': '0'} 2022-09-03T06:27:47.325 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000055c48f26e013b4d8-006312f3e3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.326 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:47.327 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:47.328 DEBUG:boto:path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:47.329 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:47.330 DEBUG:boto:Method: GET 2022-09-03T06:27:47.331 DEBUG:boto:Path: /dsgvcy-73/?prefix=h 2022-09-03T06:27:47.331 DEBUG:boto:Data: 2022-09-03T06:27:47.332 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.332 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:47.333 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.333 DEBUG:boto:Params: {} 2022-09-03T06:27:47.334 DEBUG:boto:Token: None 2022-09-03T06:27:47.335 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/ 2022-09-03T06:27:47.336 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:31VIOOe4VJCqlNerexAEtt/lq/U= 2022-09-03T06:27:47.337 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:31VIOOe4VJCqlNerexAEtt/lq/U=', 'Content-Length': '0'} 2022-09-03T06:27:47.339 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000de300df7510d2470-006312f3e3-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.340 DEBUG:boto:b'dsgvcy-73h1000falseh2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:47.341 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:47.342 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:47.342 DEBUG:boto:path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:47.343 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:47.344 DEBUG:boto:Method: GET 2022-09-03T06:27:47.344 DEBUG:boto:Path: /dsgvcy-73/?prefix=h 2022-09-03T06:27:47.345 DEBUG:boto:Data: 2022-09-03T06:27:47.346 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.346 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:47.351 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.352 DEBUG:boto:Params: {} 2022-09-03T06:27:47.352 DEBUG:boto:Token: None 2022-09-03T06:27:47.353 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/ 2022-09-03T06:27:47.354 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:31VIOOe4VJCqlNerexAEtt/lq/U= 2022-09-03T06:27:47.355 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:31VIOOe4VJCqlNerexAEtt/lq/U=', 'Content-Length': '0'} 2022-09-03T06:27:47.361 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d6dcd4919c9f5370-006312f3e3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.363 DEBUG:boto:b'dsgvcy-73h1000falseh2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:47.363 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-73 zones={test-zone2, test-zone1} 2022-09-03T06:27:47.364 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-73 zones={test-zone3, test-zone1} 2022-09-03T06:27:47.368 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:47.369 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:47.369 DEBUG:boto:Method: HEAD 2022-09-03T06:27:47.370 DEBUG:boto:Path: /dsgvcy-73/ 2022-09-03T06:27:47.371 DEBUG:boto:Data: 2022-09-03T06:27:47.371 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.372 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:47.372 DEBUG:boto:Port: 8001 2022-09-03T06:27:47.373 DEBUG:boto:Params: {} 2022-09-03T06:27:47.374 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T06:27:47.375 DEBUG:boto:Token: None 2022-09-03T06:27:47.375 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/ 2022-09-03T06:27:47.376 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:54TDAGsn7s5GBqc5eqPvET845IY= 2022-09-03T06:27:47.376 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:54TDAGsn7s5GBqc5eqPvET845IY=', 'Content-Length': '0'} 2022-09-03T06:27:47.382 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000815290a9a1e6ddb0-006312f3e3-1851-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.383 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:47.384 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:47.384 DEBUG:boto:Method: HEAD 2022-09-03T06:27:47.385 DEBUG:boto:Path: /dsgvcy-73/ 2022-09-03T06:27:47.385 DEBUG:boto:Data: 2022-09-03T06:27:47.386 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.387 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:47.388 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.389 DEBUG:boto:Params: {} 2022-09-03T06:27:47.391 DEBUG:boto:Token: None 2022-09-03T06:27:47.392 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/ 2022-09-03T06:27:47.393 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:54TDAGsn7s5GBqc5eqPvET845IY= 2022-09-03T06:27:47.394 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:54TDAGsn7s5GBqc5eqPvET845IY=', 'Content-Length': '0'} 2022-09-03T06:27:47.397 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000006a3fca9d5b20df4-006312f3e3-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.397 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:27:47.398 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:47.400 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:47.400 DEBUG:boto:path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:47.401 DEBUG:boto:auth_path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:47.401 DEBUG:boto:Method: GET 2022-09-03T06:27:47.402 DEBUG:boto:Path: /dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:47.402 DEBUG:boto:Data: 2022-09-03T06:27:47.403 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.404 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:47.405 DEBUG:boto:Port: 8001 2022-09-03T06:27:47.405 DEBUG:boto:Params: {} 2022-09-03T06:27:47.406 DEBUG:boto:Token: None 2022-09-03T06:27:47.407 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/?versions 2022-09-03T06:27:47.408 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MI//4lrMMQ16A2FRYnZhOpSF/VI= 2022-09-03T06:27:47.411 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MI//4lrMMQ16A2FRYnZhOpSF/VI=', 'Content-Length': '0'} 2022-09-03T06:27:47.414 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000496314b5ea3e707b-006312f3e3-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.415 DEBUG:boto:b'dsgvcy-73999falseanulltrue2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:47.417 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:27:47.418 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:27:47.419 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:27:47.420 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:27:47.420 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:27:47.421 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:27:47.422 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:27:47.423 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:27:47.423 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:27:47.424 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:47.425 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:47.426 DEBUG:boto:path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:47.427 DEBUG:boto:auth_path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:47.427 DEBUG:boto:Method: GET 2022-09-03T06:27:47.429 DEBUG:boto:Path: /dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:47.429 DEBUG:boto:Data: 2022-09-03T06:27:47.430 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.431 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:47.431 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.432 DEBUG:boto:Params: {} 2022-09-03T06:27:47.432 DEBUG:boto:Token: None 2022-09-03T06:27:47.433 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/?versions 2022-09-03T06:27:47.433 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MI//4lrMMQ16A2FRYnZhOpSF/VI= 2022-09-03T06:27:47.434 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MI//4lrMMQ16A2FRYnZhOpSF/VI=', 'Content-Length': '0'} 2022-09-03T06:27:47.437 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a282866ac4860e2a-006312f3e3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.437 DEBUG:boto:b'dsgvcy-73999falseanulltrue2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:47.439 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:27:47.440 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:27:47.441 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:27:47.442 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:27:47.443 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:27:47.444 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:27:47.445 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:27:47.445 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:27:47.447 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:47.447 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:47.448 DEBUG:boto:path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:47.449 DEBUG:boto:auth_path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:47.450 DEBUG:boto:Method: GET 2022-09-03T06:27:47.451 DEBUG:boto:Path: /dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:47.452 DEBUG:boto:Data: 2022-09-03T06:27:47.452 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.453 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:47.454 DEBUG:boto:Port: 8001 2022-09-03T06:27:47.454 DEBUG:boto:Params: {} 2022-09-03T06:27:47.455 DEBUG:boto:Token: None 2022-09-03T06:27:47.457 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/?versions 2022-09-03T06:27:47.458 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MI//4lrMMQ16A2FRYnZhOpSF/VI= 2022-09-03T06:27:47.458 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MI//4lrMMQ16A2FRYnZhOpSF/VI=', 'Content-Length': '0'} 2022-09-03T06:27:47.464 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d3afb6061af3195b-006312f3e3-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.465 DEBUG:boto:b'dsgvcy-73999falseanulltrue2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:47.467 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:47.468 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:47.469 DEBUG:boto:path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:47.471 DEBUG:boto:auth_path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:47.472 DEBUG:boto:Method: GET 2022-09-03T06:27:47.473 DEBUG:boto:Path: /dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:47.474 DEBUG:boto:Data: 2022-09-03T06:27:47.475 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.475 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:47.476 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.476 DEBUG:boto:Params: {} 2022-09-03T06:27:47.477 DEBUG:boto:Token: None 2022-09-03T06:27:47.478 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/?versions 2022-09-03T06:27:47.479 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MI//4lrMMQ16A2FRYnZhOpSF/VI= 2022-09-03T06:27:47.479 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MI//4lrMMQ16A2FRYnZhOpSF/VI=', 'Content-Length': '0'} 2022-09-03T06:27:47.482 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ae765990fe0dd70e-006312f3e3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.483 DEBUG:boto:b'dsgvcy-73999falseanulltrue2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:47.484 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:27:47.485 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:47.486 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:47.486 DEBUG:boto:path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.487 DEBUG:boto:auth_path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.487 DEBUG:boto:Method: GET 2022-09-03T06:27:47.488 DEBUG:boto:Path: /dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.489 DEBUG:boto:Data: 2022-09-03T06:27:47.490 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.490 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:47.491 DEBUG:boto:Port: 8001 2022-09-03T06:27:47.492 DEBUG:boto:Params: {} 2022-09-03T06:27:47.492 DEBUG:boto:Token: None 2022-09-03T06:27:47.493 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.494 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S9HZNF930etBCajIN28Jwpyeocc= 2022-09-03T06:27:47.495 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S9HZNF930etBCajIN28Jwpyeocc=', 'Content-Length': '0'} 2022-09-03T06:27:47.498 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f6d703aee10d6cea-006312f3e3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.500 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:47.501 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:47.502 DEBUG:boto:path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.503 DEBUG:boto:auth_path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.504 DEBUG:boto:Method: GET 2022-09-03T06:27:47.505 DEBUG:boto:Path: /dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.506 DEBUG:boto:Data: 2022-09-03T06:27:47.507 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.507 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:47.508 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.510 DEBUG:boto:Params: {} 2022-09-03T06:27:47.511 DEBUG:boto:Token: None 2022-09-03T06:27:47.512 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.513 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S9HZNF930etBCajIN28Jwpyeocc= 2022-09-03T06:27:47.514 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S9HZNF930etBCajIN28Jwpyeocc=', 'Content-Length': '0'} 2022-09-03T06:27:47.518 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000547e8492aa6aa71c-006312f3e3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.558 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:47.559 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:47.560 DEBUG:boto:path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.560 DEBUG:boto:auth_path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.561 DEBUG:boto:Method: HEAD 2022-09-03T06:27:47.561 DEBUG:boto:Path: /dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.562 DEBUG:boto:Data: 2022-09-03T06:27:47.563 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.563 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:47.564 DEBUG:boto:Port: 8001 2022-09-03T06:27:47.564 DEBUG:boto:Params: {} 2022-09-03T06:27:47.565 DEBUG:boto:Token: None 2022-09-03T06:27:47.566 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.567 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZM7kebIseahUiXdqcCpZLDeWXck= 2022-09-03T06:27:47.567 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZM7kebIseahUiXdqcCpZLDeWXck=', 'Content-Length': '0'} 2022-09-03T06:27:47.570 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b0641d363809bc8-006312f3e3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.571 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:47.571 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:47.572 DEBUG:boto:path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.573 DEBUG:boto:auth_path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.574 DEBUG:boto:Method: HEAD 2022-09-03T06:27:47.574 DEBUG:boto:Path: /dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.575 DEBUG:boto:Data: 2022-09-03T06:27:47.575 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.576 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:47.576 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.577 DEBUG:boto:Params: {} 2022-09-03T06:27:47.577 DEBUG:boto:Token: None 2022-09-03T06:27:47.578 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.579 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZM7kebIseahUiXdqcCpZLDeWXck= 2022-09-03T06:27:47.579 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZM7kebIseahUiXdqcCpZLDeWXck=', 'Content-Length': '0'} 2022-09-03T06:27:47.582 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009022659bc3f2927b-006312f3e3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.583 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:27:47.583 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:47.584 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:47.584 DEBUG:boto:path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.585 DEBUG:boto:auth_path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.585 DEBUG:boto:Method: GET 2022-09-03T06:27:47.586 DEBUG:boto:Path: /dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.587 DEBUG:boto:Data: 2022-09-03T06:27:47.587 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.588 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:47.588 DEBUG:boto:Port: 8001 2022-09-03T06:27:47.589 DEBUG:boto:Params: {} 2022-09-03T06:27:47.589 DEBUG:boto:Token: None 2022-09-03T06:27:47.590 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.591 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S9HZNF930etBCajIN28Jwpyeocc= 2022-09-03T06:27:47.591 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S9HZNF930etBCajIN28Jwpyeocc=', 'Content-Length': '0'} 2022-09-03T06:27:47.594 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000092aa2197f460dfeb-006312f3e3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.595 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:47.596 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:47.596 DEBUG:boto:path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.597 DEBUG:boto:auth_path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.597 DEBUG:boto:Method: GET 2022-09-03T06:27:47.598 DEBUG:boto:Path: /dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.599 DEBUG:boto:Data: 2022-09-03T06:27:47.599 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.600 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:47.600 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.601 DEBUG:boto:Params: {} 2022-09-03T06:27:47.601 DEBUG:boto:Token: None 2022-09-03T06:27:47.602 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/a?versionId=null 2022-09-03T06:27:47.602 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S9HZNF930etBCajIN28Jwpyeocc= 2022-09-03T06:27:47.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S9HZNF930etBCajIN28Jwpyeocc=', 'Content-Length': '0'} 2022-09-03T06:27:47.605 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000040c50e978f6f0753-006312f3e3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.606 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:47.607 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:47.607 DEBUG:boto:path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:47.608 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:47.608 DEBUG:boto:Method: GET 2022-09-03T06:27:47.609 DEBUG:boto:Path: /dsgvcy-73/?prefix=a 2022-09-03T06:27:47.609 DEBUG:boto:Data: 2022-09-03T06:27:47.610 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.610 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:47.611 DEBUG:boto:Port: 8001 2022-09-03T06:27:47.612 DEBUG:boto:Params: {} 2022-09-03T06:27:47.612 DEBUG:boto:Token: None 2022-09-03T06:27:47.613 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/ 2022-09-03T06:27:47.613 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:31VIOOe4VJCqlNerexAEtt/lq/U= 2022-09-03T06:27:47.614 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:31VIOOe4VJCqlNerexAEtt/lq/U=', 'Content-Length': '0'} 2022-09-03T06:27:47.618 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000021c44ad1b5077afd-006312f3e3-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.619 DEBUG:boto:b'dsgvcy-73a1000falsea2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:47.621 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:47.623 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:47.624 DEBUG:boto:path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:47.624 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:47.625 DEBUG:boto:Method: GET 2022-09-03T06:27:47.626 DEBUG:boto:Path: /dsgvcy-73/?prefix=a 2022-09-03T06:27:47.628 DEBUG:boto:Data: 2022-09-03T06:27:47.629 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.629 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:47.631 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.632 DEBUG:boto:Params: {} 2022-09-03T06:27:47.633 DEBUG:boto:Token: None 2022-09-03T06:27:47.633 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/ 2022-09-03T06:27:47.634 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:31VIOOe4VJCqlNerexAEtt/lq/U= 2022-09-03T06:27:47.635 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:31VIOOe4VJCqlNerexAEtt/lq/U=', 'Content-Length': '0'} 2022-09-03T06:27:47.641 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000db7aa10c31df64bb-006312f3e3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.642 DEBUG:boto:b'dsgvcy-73a1000falsea2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:47.643 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:47.643 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:47.644 DEBUG:boto:Method: HEAD 2022-09-03T06:27:47.645 DEBUG:boto:Path: /dsgvcy-73/a 2022-09-03T06:27:47.645 DEBUG:boto:Data: 2022-09-03T06:27:47.646 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.646 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:47.647 DEBUG:boto:Port: 8001 2022-09-03T06:27:47.647 DEBUG:boto:Params: {} 2022-09-03T06:27:47.648 DEBUG:boto:Token: None 2022-09-03T06:27:47.648 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/a 2022-09-03T06:27:47.649 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H6E2yXnifvvNOy/IHfqD/WKJ9Bo= 2022-09-03T06:27:47.650 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H6E2yXnifvvNOy/IHfqD/WKJ9Bo=', 'Content-Length': '0'} 2022-09-03T06:27:47.652 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a3cffe00c9255c8-006312f3e3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.653 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:47.654 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:47.655 DEBUG:boto:Method: HEAD 2022-09-03T06:27:47.656 DEBUG:boto:Path: /dsgvcy-73/a 2022-09-03T06:27:47.657 DEBUG:boto:Data: 2022-09-03T06:27:47.658 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.659 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:47.660 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.661 DEBUG:boto:Params: {} 2022-09-03T06:27:47.662 DEBUG:boto:Token: None 2022-09-03T06:27:47.663 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/a 2022-09-03T06:27:47.664 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H6E2yXnifvvNOy/IHfqD/WKJ9Bo= 2022-09-03T06:27:47.665 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H6E2yXnifvvNOy/IHfqD/WKJ9Bo=', 'Content-Length': '0'} 2022-09-03T06:27:47.668 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c5ecb1c15564804b-006312f3e3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.669 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:27:47.671 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:47.672 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:47.674 DEBUG:boto:Method: GET 2022-09-03T06:27:47.675 DEBUG:boto:Path: /dsgvcy-73/a 2022-09-03T06:27:47.676 DEBUG:boto:Data: 2022-09-03T06:27:47.677 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.678 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:47.679 DEBUG:boto:Port: 8001 2022-09-03T06:27:47.680 DEBUG:boto:Params: {} 2022-09-03T06:27:47.681 DEBUG:boto:Token: None 2022-09-03T06:27:47.682 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/a 2022-09-03T06:27:47.683 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IqyL72fVp/McqFYbwVvV4Gd3ub8= 2022-09-03T06:27:47.683 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IqyL72fVp/McqFYbwVvV4Gd3ub8=', 'Content-Length': '0'} 2022-09-03T06:27:47.685 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a7c407f5cec923b-006312f3e3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.735 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:47.736 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:47.736 DEBUG:boto:Method: GET 2022-09-03T06:27:47.737 DEBUG:boto:Path: /dsgvcy-73/a 2022-09-03T06:27:47.738 DEBUG:boto:Data: 2022-09-03T06:27:47.738 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.739 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:47.740 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.741 DEBUG:boto:Params: {} 2022-09-03T06:27:47.741 DEBUG:boto:Token: None 2022-09-03T06:27:47.742 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/a 2022-09-03T06:27:47.743 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IqyL72fVp/McqFYbwVvV4Gd3ub8= 2022-09-03T06:27:47.744 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IqyL72fVp/McqFYbwVvV4Gd3ub8=', 'Content-Length': '0'} 2022-09-03T06:27:47.748 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000062ab18c4932363c0-006312f3e3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.749 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:47.750 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:47.750 DEBUG:boto:path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:47.751 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:47.752 DEBUG:boto:Method: GET 2022-09-03T06:27:47.752 DEBUG:boto:Path: /dsgvcy-73/?prefix=a 2022-09-03T06:27:47.753 DEBUG:boto:Data: 2022-09-03T06:27:47.753 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.754 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:47.754 DEBUG:boto:Port: 8001 2022-09-03T06:27:47.755 DEBUG:boto:Params: {} 2022-09-03T06:27:47.756 DEBUG:boto:Token: None 2022-09-03T06:27:47.756 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/ 2022-09-03T06:27:47.757 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:31VIOOe4VJCqlNerexAEtt/lq/U= 2022-09-03T06:27:47.758 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:31VIOOe4VJCqlNerexAEtt/lq/U=', 'Content-Length': '0'} 2022-09-03T06:27:47.761 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000094930c19b1b53175-006312f3e3-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.762 DEBUG:boto:b'dsgvcy-73a1000falsea2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:47.763 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:47.763 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:47.764 DEBUG:boto:path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:47.765 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:47.767 DEBUG:boto:Method: GET 2022-09-03T06:27:47.768 DEBUG:boto:Path: /dsgvcy-73/?prefix=a 2022-09-03T06:27:47.769 DEBUG:boto:Data: 2022-09-03T06:27:47.770 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.771 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:47.772 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.773 DEBUG:boto:Params: {} 2022-09-03T06:27:47.774 DEBUG:boto:Token: None 2022-09-03T06:27:47.775 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/ 2022-09-03T06:27:47.776 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:31VIOOe4VJCqlNerexAEtt/lq/U= 2022-09-03T06:27:47.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:31VIOOe4VJCqlNerexAEtt/lq/U=', 'Content-Length': '0'} 2022-09-03T06:27:47.781 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a2d3f20e0623d641-006312f3e3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.824 DEBUG:boto:b'dsgvcy-73a1000falsea2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:47.826 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:27:47.827 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:47.829 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:47.831 DEBUG:boto:path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.832 DEBUG:boto:auth_path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.833 DEBUG:boto:Method: GET 2022-09-03T06:27:47.835 DEBUG:boto:Path: /dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.836 DEBUG:boto:Data: 2022-09-03T06:27:47.837 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.838 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:47.840 DEBUG:boto:Port: 8001 2022-09-03T06:27:47.841 DEBUG:boto:Params: {} 2022-09-03T06:27:47.842 DEBUG:boto:Token: None 2022-09-03T06:27:47.843 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.844 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JV80nXxFOwZxnCl/iljr9oxSEaM= 2022-09-03T06:27:47.846 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JV80nXxFOwZxnCl/iljr9oxSEaM=', 'Content-Length': '0'} 2022-09-03T06:27:47.849 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af6870da25f2ee20-006312f3e3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.851 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:47.852 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:47.853 DEBUG:boto:path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.853 DEBUG:boto:auth_path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.854 DEBUG:boto:Method: GET 2022-09-03T06:27:47.855 DEBUG:boto:Path: /dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.855 DEBUG:boto:Data: 2022-09-03T06:27:47.856 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.857 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:47.858 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.859 DEBUG:boto:Params: {} 2022-09-03T06:27:47.860 DEBUG:boto:Token: None 2022-09-03T06:27:47.862 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.863 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JV80nXxFOwZxnCl/iljr9oxSEaM= 2022-09-03T06:27:47.865 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JV80nXxFOwZxnCl/iljr9oxSEaM=', 'Content-Length': '0'} 2022-09-03T06:27:47.868 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007cd0b293dfb7230d-006312f3e3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.870 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:47.871 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:47.872 DEBUG:boto:path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.873 DEBUG:boto:auth_path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.874 DEBUG:boto:Method: HEAD 2022-09-03T06:27:47.875 DEBUG:boto:Path: /dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.875 DEBUG:boto:Data: 2022-09-03T06:27:47.876 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.877 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:47.878 DEBUG:boto:Port: 8001 2022-09-03T06:27:47.879 DEBUG:boto:Params: {} 2022-09-03T06:27:47.879 DEBUG:boto:Token: None 2022-09-03T06:27:47.880 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.881 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BMXyqNcm3e9/2jzyzPAYIrPzyIM= 2022-09-03T06:27:47.881 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BMXyqNcm3e9/2jzyzPAYIrPzyIM=', 'Content-Length': '0'} 2022-09-03T06:27:47.884 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000880d9445c6cec3af-006312f3e3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.885 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:47.886 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:47.886 DEBUG:boto:path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.887 DEBUG:boto:auth_path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.887 DEBUG:boto:Method: HEAD 2022-09-03T06:27:47.888 DEBUG:boto:Path: /dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.888 DEBUG:boto:Data: 2022-09-03T06:27:47.889 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.890 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:47.890 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.891 DEBUG:boto:Params: {} 2022-09-03T06:27:47.891 DEBUG:boto:Token: None 2022-09-03T06:27:47.892 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.893 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BMXyqNcm3e9/2jzyzPAYIrPzyIM= 2022-09-03T06:27:47.893 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BMXyqNcm3e9/2jzyzPAYIrPzyIM=', 'Content-Length': '0'} 2022-09-03T06:27:47.895 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007e5c6c42521f1c94-006312f3e3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.896 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:27:47.897 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:47.897 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:47.898 DEBUG:boto:path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.898 DEBUG:boto:auth_path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.899 DEBUG:boto:Method: GET 2022-09-03T06:27:47.900 DEBUG:boto:Path: /dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.900 DEBUG:boto:Data: 2022-09-03T06:27:47.901 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.901 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:47.902 DEBUG:boto:Port: 8001 2022-09-03T06:27:47.903 DEBUG:boto:Params: {} 2022-09-03T06:27:47.903 DEBUG:boto:Token: None 2022-09-03T06:27:47.904 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.905 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JV80nXxFOwZxnCl/iljr9oxSEaM= 2022-09-03T06:27:47.905 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JV80nXxFOwZxnCl/iljr9oxSEaM=', 'Content-Length': '0'} 2022-09-03T06:27:47.907 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000029be65e9ff30e1c1-006312f3e3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.908 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:47.909 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:47.909 DEBUG:boto:path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.910 DEBUG:boto:auth_path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.911 DEBUG:boto:Method: GET 2022-09-03T06:27:47.912 DEBUG:boto:Path: /dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.912 DEBUG:boto:Data: 2022-09-03T06:27:47.913 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.913 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:47.914 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.915 DEBUG:boto:Params: {} 2022-09-03T06:27:47.915 DEBUG:boto:Token: None 2022-09-03T06:27:47.916 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/b?versionId=null 2022-09-03T06:27:47.917 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JV80nXxFOwZxnCl/iljr9oxSEaM= 2022-09-03T06:27:47.918 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JV80nXxFOwZxnCl/iljr9oxSEaM=', 'Content-Length': '0'} 2022-09-03T06:27:47.920 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000051b4a94f90ddf9e2-006312f3e3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.922 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:47.923 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:47.923 DEBUG:boto:path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:47.924 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:47.924 DEBUG:boto:Method: GET 2022-09-03T06:27:47.925 DEBUG:boto:Path: /dsgvcy-73/?prefix=b 2022-09-03T06:27:47.926 DEBUG:boto:Data: 2022-09-03T06:27:47.926 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.927 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:47.928 DEBUG:boto:Port: 8001 2022-09-03T06:27:47.929 DEBUG:boto:Params: {} 2022-09-03T06:27:47.929 DEBUG:boto:Token: None 2022-09-03T06:27:47.930 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/ 2022-09-03T06:27:47.931 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:31VIOOe4VJCqlNerexAEtt/lq/U= 2022-09-03T06:27:47.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:31VIOOe4VJCqlNerexAEtt/lq/U=', 'Content-Length': '0'} 2022-09-03T06:27:47.935 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000027a2236a6cfe683c-006312f3e3-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.935 DEBUG:boto:b'dsgvcy-73b1000falseb2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:47.936 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:47.937 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:47.937 DEBUG:boto:path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:47.938 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:47.939 DEBUG:boto:Method: GET 2022-09-03T06:27:47.940 DEBUG:boto:Path: /dsgvcy-73/?prefix=b 2022-09-03T06:27:47.941 DEBUG:boto:Data: 2022-09-03T06:27:47.942 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.942 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:47.943 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.943 DEBUG:boto:Params: {} 2022-09-03T06:27:47.946 DEBUG:boto:Token: None 2022-09-03T06:27:47.947 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/ 2022-09-03T06:27:47.947 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:31VIOOe4VJCqlNerexAEtt/lq/U= 2022-09-03T06:27:47.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:31VIOOe4VJCqlNerexAEtt/lq/U=', 'Content-Length': '0'} 2022-09-03T06:27:47.955 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f85f3a15064359f8-006312f3e3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.957 DEBUG:boto:b'dsgvcy-73b1000falseb2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:47.959 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:47.960 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:47.962 DEBUG:boto:Method: HEAD 2022-09-03T06:27:47.962 DEBUG:boto:Path: /dsgvcy-73/b 2022-09-03T06:27:47.963 DEBUG:boto:Data: 2022-09-03T06:27:47.963 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.964 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:47.965 DEBUG:boto:Port: 8001 2022-09-03T06:27:47.965 DEBUG:boto:Params: {} 2022-09-03T06:27:47.967 DEBUG:boto:Token: None 2022-09-03T06:27:47.967 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/b 2022-09-03T06:27:47.968 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XbS3BHTmahif4wAd6Vm1F/criwA= 2022-09-03T06:27:47.969 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XbS3BHTmahif4wAd6Vm1F/criwA=', 'Content-Length': '0'} 2022-09-03T06:27:47.971 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c84d277ee186c5a8-006312f3e3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.972 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:47.973 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:47.973 DEBUG:boto:Method: HEAD 2022-09-03T06:27:47.974 DEBUG:boto:Path: /dsgvcy-73/b 2022-09-03T06:27:47.974 DEBUG:boto:Data: 2022-09-03T06:27:47.975 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.976 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:47.976 DEBUG:boto:Port: 8000 2022-09-03T06:27:47.977 DEBUG:boto:Params: {} 2022-09-03T06:27:47.977 DEBUG:boto:Token: None 2022-09-03T06:27:47.978 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/b 2022-09-03T06:27:47.979 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XbS3BHTmahif4wAd6Vm1F/criwA= 2022-09-03T06:27:47.979 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XbS3BHTmahif4wAd6Vm1F/criwA=', 'Content-Length': '0'} 2022-09-03T06:27:47.982 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002139fdb4c797590d-006312f3e3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:47.983 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:27:47.985 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:47.986 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:47.986 DEBUG:boto:Method: GET 2022-09-03T06:27:47.987 DEBUG:boto:Path: /dsgvcy-73/b 2022-09-03T06:27:47.987 DEBUG:boto:Data: 2022-09-03T06:27:47.988 DEBUG:boto:Headers: {} 2022-09-03T06:27:47.989 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:47.989 DEBUG:boto:Port: 8001 2022-09-03T06:27:47.990 DEBUG:boto:Params: {} 2022-09-03T06:27:47.991 DEBUG:boto:Token: None 2022-09-03T06:27:47.992 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:47 GMT /dsgvcy-73/b 2022-09-03T06:27:47.992 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZYV8nt5aKz8BsojPMPn2Z/aTyE4= 2022-09-03T06:27:47.993 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZYV8nt5aKz8BsojPMPn2Z/aTyE4=', 'Content-Length': '0'} 2022-09-03T06:27:47.996 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1aba0e30ef35da3-006312f3e3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.038 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:48.039 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:48.040 DEBUG:boto:Method: GET 2022-09-03T06:27:48.041 DEBUG:boto:Path: /dsgvcy-73/b 2022-09-03T06:27:48.042 DEBUG:boto:Data: 2022-09-03T06:27:48.042 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.043 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:48.043 DEBUG:boto:Port: 8000 2022-09-03T06:27:48.044 DEBUG:boto:Params: {} 2022-09-03T06:27:48.045 DEBUG:boto:Token: None 2022-09-03T06:27:48.046 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/b 2022-09-03T06:27:48.046 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NgWx+No3T+z80/MtdXUn0xuAJMg= 2022-09-03T06:27:48.047 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NgWx+No3T+z80/MtdXUn0xuAJMg=', 'Content-Length': '0'} 2022-09-03T06:27:48.050 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000498cfa01e6b09c9e-006312f3e4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.051 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:48.052 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:48.052 DEBUG:boto:path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:48.053 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:48.053 DEBUG:boto:Method: GET 2022-09-03T06:27:48.054 DEBUG:boto:Path: /dsgvcy-73/?prefix=b 2022-09-03T06:27:48.055 DEBUG:boto:Data: 2022-09-03T06:27:48.056 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.057 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:48.058 DEBUG:boto:Port: 8001 2022-09-03T06:27:48.059 DEBUG:boto:Params: {} 2022-09-03T06:27:48.060 DEBUG:boto:Token: None 2022-09-03T06:27:48.061 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/ 2022-09-03T06:27:48.063 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek= 2022-09-03T06:27:48.066 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek=', 'Content-Length': '0'} 2022-09-03T06:27:48.069 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000155ca48b14fb08bd-006312f3e4-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.071 DEBUG:boto:b'dsgvcy-73b1000falseb2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:48.072 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:48.074 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:48.075 DEBUG:boto:path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:48.077 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:48.078 DEBUG:boto:Method: GET 2022-09-03T06:27:48.079 DEBUG:boto:Path: /dsgvcy-73/?prefix=b 2022-09-03T06:27:48.080 DEBUG:boto:Data: 2022-09-03T06:27:48.081 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.082 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:48.083 DEBUG:boto:Port: 8000 2022-09-03T06:27:48.084 DEBUG:boto:Params: {} 2022-09-03T06:27:48.085 DEBUG:boto:Token: None 2022-09-03T06:27:48.088 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/ 2022-09-03T06:27:48.089 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek= 2022-09-03T06:27:48.091 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek=', 'Content-Length': '0'} 2022-09-03T06:27:48.095 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002e98d6bb2e107625-006312f3e4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.096 DEBUG:boto:b'dsgvcy-73b1000falseb2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:48.098 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:27:48.099 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:48.101 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:48.102 DEBUG:boto:path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.103 DEBUG:boto:auth_path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.104 DEBUG:boto:Method: GET 2022-09-03T06:27:48.105 DEBUG:boto:Path: /dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.107 DEBUG:boto:Data: 2022-09-03T06:27:48.109 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.109 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:48.111 DEBUG:boto:Port: 8001 2022-09-03T06:27:48.112 DEBUG:boto:Params: {} 2022-09-03T06:27:48.112 DEBUG:boto:Token: None 2022-09-03T06:27:48.113 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.113 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eB0Ip+5IAf7JTrGLTuDjGwtpVJU= 2022-09-03T06:27:48.114 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eB0Ip+5IAf7JTrGLTuDjGwtpVJU=', 'Content-Length': '0'} 2022-09-03T06:27:48.116 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000064d16beacec3e0f0-006312f3e4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.117 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:48.118 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:48.118 DEBUG:boto:path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.119 DEBUG:boto:auth_path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.120 DEBUG:boto:Method: GET 2022-09-03T06:27:48.120 DEBUG:boto:Path: /dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.121 DEBUG:boto:Data: 2022-09-03T06:27:48.121 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.122 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:48.122 DEBUG:boto:Port: 8000 2022-09-03T06:27:48.123 DEBUG:boto:Params: {} 2022-09-03T06:27:48.123 DEBUG:boto:Token: None 2022-09-03T06:27:48.124 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.124 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eB0Ip+5IAf7JTrGLTuDjGwtpVJU= 2022-09-03T06:27:48.125 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eB0Ip+5IAf7JTrGLTuDjGwtpVJU=', 'Content-Length': '0'} 2022-09-03T06:27:48.127 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000036657763256ce6b5-006312f3e4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.128 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:48.129 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:48.129 DEBUG:boto:path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.130 DEBUG:boto:auth_path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.130 DEBUG:boto:Method: HEAD 2022-09-03T06:27:48.131 DEBUG:boto:Path: /dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.131 DEBUG:boto:Data: 2022-09-03T06:27:48.132 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.132 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:48.133 DEBUG:boto:Port: 8001 2022-09-03T06:27:48.134 DEBUG:boto:Params: {} 2022-09-03T06:27:48.134 DEBUG:boto:Token: None 2022-09-03T06:27:48.135 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.135 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lFTtw4KhiQV7cfVwaWabVjt11JM= 2022-09-03T06:27:48.136 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lFTtw4KhiQV7cfVwaWabVjt11JM=', 'Content-Length': '0'} 2022-09-03T06:27:48.138 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000890fd50e389c182d-006312f3e4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.139 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:48.140 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:48.140 DEBUG:boto:path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.141 DEBUG:boto:auth_path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.141 DEBUG:boto:Method: HEAD 2022-09-03T06:27:48.143 DEBUG:boto:Path: /dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.144 DEBUG:boto:Data: 2022-09-03T06:27:48.144 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.145 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:48.145 DEBUG:boto:Port: 8000 2022-09-03T06:27:48.146 DEBUG:boto:Params: {} 2022-09-03T06:27:48.147 DEBUG:boto:Token: None 2022-09-03T06:27:48.147 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.148 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lFTtw4KhiQV7cfVwaWabVjt11JM= 2022-09-03T06:27:48.149 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lFTtw4KhiQV7cfVwaWabVjt11JM=', 'Content-Length': '0'} 2022-09-03T06:27:48.151 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003f2c8250098da0de-006312f3e4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.152 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:27:48.153 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:48.154 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:48.155 DEBUG:boto:path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.155 DEBUG:boto:auth_path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.156 DEBUG:boto:Method: GET 2022-09-03T06:27:48.157 DEBUG:boto:Path: /dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.157 DEBUG:boto:Data: 2022-09-03T06:27:48.158 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.158 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:48.159 DEBUG:boto:Port: 8001 2022-09-03T06:27:48.159 DEBUG:boto:Params: {} 2022-09-03T06:27:48.160 DEBUG:boto:Token: None 2022-09-03T06:27:48.161 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.162 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eB0Ip+5IAf7JTrGLTuDjGwtpVJU= 2022-09-03T06:27:48.162 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eB0Ip+5IAf7JTrGLTuDjGwtpVJU=', 'Content-Length': '0'} 2022-09-03T06:27:48.164 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dd1cf94fbdf7746b-006312f3e4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.166 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:48.167 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:48.168 DEBUG:boto:path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.168 DEBUG:boto:auth_path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.169 DEBUG:boto:Method: GET 2022-09-03T06:27:48.169 DEBUG:boto:Path: /dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.170 DEBUG:boto:Data: 2022-09-03T06:27:48.170 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.171 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:48.171 DEBUG:boto:Port: 8000 2022-09-03T06:27:48.172 DEBUG:boto:Params: {} 2022-09-03T06:27:48.173 DEBUG:boto:Token: None 2022-09-03T06:27:48.173 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/c?versionId=null 2022-09-03T06:27:48.174 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eB0Ip+5IAf7JTrGLTuDjGwtpVJU= 2022-09-03T06:27:48.176 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eB0Ip+5IAf7JTrGLTuDjGwtpVJU=', 'Content-Length': '0'} 2022-09-03T06:27:48.188 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ba35237b84874063-006312f3e4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.235 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:48.236 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:48.237 DEBUG:boto:path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:48.239 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:48.240 DEBUG:boto:Method: GET 2022-09-03T06:27:48.240 DEBUG:boto:Path: /dsgvcy-73/?prefix=c 2022-09-03T06:27:48.241 DEBUG:boto:Data: 2022-09-03T06:27:48.242 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.242 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:48.243 DEBUG:boto:Port: 8001 2022-09-03T06:27:48.244 DEBUG:boto:Params: {} 2022-09-03T06:27:48.245 DEBUG:boto:Token: None 2022-09-03T06:27:48.246 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/ 2022-09-03T06:27:48.247 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek= 2022-09-03T06:27:48.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek=', 'Content-Length': '0'} 2022-09-03T06:27:48.252 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001a2c5f45c594b8bb-006312f3e4-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.254 DEBUG:boto:b'dsgvcy-73c1000falsec2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:48.255 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:48.255 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:48.256 DEBUG:boto:path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:48.257 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:48.258 DEBUG:boto:Method: GET 2022-09-03T06:27:48.258 DEBUG:boto:Path: /dsgvcy-73/?prefix=c 2022-09-03T06:27:48.259 DEBUG:boto:Data: 2022-09-03T06:27:48.260 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.261 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:48.261 DEBUG:boto:Port: 8000 2022-09-03T06:27:48.262 DEBUG:boto:Params: {} 2022-09-03T06:27:48.262 DEBUG:boto:Token: None 2022-09-03T06:27:48.263 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/ 2022-09-03T06:27:48.264 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek= 2022-09-03T06:27:48.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek=', 'Content-Length': '0'} 2022-09-03T06:27:48.268 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003e2660749a7483de-006312f3e4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.269 DEBUG:boto:b'dsgvcy-73c1000falsec2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:48.270 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:48.270 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:48.271 DEBUG:boto:Method: HEAD 2022-09-03T06:27:48.272 DEBUG:boto:Path: /dsgvcy-73/c 2022-09-03T06:27:48.272 DEBUG:boto:Data: 2022-09-03T06:27:48.273 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.273 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:48.274 DEBUG:boto:Port: 8001 2022-09-03T06:27:48.274 DEBUG:boto:Params: {} 2022-09-03T06:27:48.275 DEBUG:boto:Token: None 2022-09-03T06:27:48.276 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/c 2022-09-03T06:27:48.276 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E4jNTLjZ4s3oMB4jMaCxhukIv/Q= 2022-09-03T06:27:48.277 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E4jNTLjZ4s3oMB4jMaCxhukIv/Q=', 'Content-Length': '0'} 2022-09-03T06:27:48.295 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f4e20af9f4f8e796-006312f3e4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.296 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:48.298 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:48.298 DEBUG:boto:Method: HEAD 2022-09-03T06:27:48.300 DEBUG:boto:Path: /dsgvcy-73/c 2022-09-03T06:27:48.301 DEBUG:boto:Data: 2022-09-03T06:27:48.301 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.302 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:48.303 DEBUG:boto:Port: 8000 2022-09-03T06:27:48.304 DEBUG:boto:Params: {} 2022-09-03T06:27:48.305 DEBUG:boto:Token: None 2022-09-03T06:27:48.306 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/c 2022-09-03T06:27:48.307 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E4jNTLjZ4s3oMB4jMaCxhukIv/Q= 2022-09-03T06:27:48.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E4jNTLjZ4s3oMB4jMaCxhukIv/Q=', 'Content-Length': '0'} 2022-09-03T06:27:48.312 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b81b9fdadacc1057-006312f3e4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.313 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:27:48.315 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:48.316 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:48.317 DEBUG:boto:Method: GET 2022-09-03T06:27:48.318 DEBUG:boto:Path: /dsgvcy-73/c 2022-09-03T06:27:48.319 DEBUG:boto:Data: 2022-09-03T06:27:48.321 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.322 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:48.323 DEBUG:boto:Port: 8001 2022-09-03T06:27:48.324 DEBUG:boto:Params: {} 2022-09-03T06:27:48.325 DEBUG:boto:Token: None 2022-09-03T06:27:48.326 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/c 2022-09-03T06:27:48.327 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hdBN5nZmY+Tcpv6Gfa9AbTwtJ9c= 2022-09-03T06:27:48.329 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hdBN5nZmY+Tcpv6Gfa9AbTwtJ9c=', 'Content-Length': '0'} 2022-09-03T06:27:48.333 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c614a078d01767ff-006312f3e4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.376 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:48.378 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:48.380 DEBUG:boto:Method: GET 2022-09-03T06:27:48.380 DEBUG:boto:Path: /dsgvcy-73/c 2022-09-03T06:27:48.381 DEBUG:boto:Data: 2022-09-03T06:27:48.382 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.382 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:48.383 DEBUG:boto:Port: 8000 2022-09-03T06:27:48.383 DEBUG:boto:Params: {} 2022-09-03T06:27:48.384 DEBUG:boto:Token: None 2022-09-03T06:27:48.385 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/c 2022-09-03T06:27:48.385 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hdBN5nZmY+Tcpv6Gfa9AbTwtJ9c= 2022-09-03T06:27:48.386 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hdBN5nZmY+Tcpv6Gfa9AbTwtJ9c=', 'Content-Length': '0'} 2022-09-03T06:27:48.389 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000100ea881a4774c0d-006312f3e4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.390 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:48.391 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:48.392 DEBUG:boto:path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:48.393 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:48.394 DEBUG:boto:Method: GET 2022-09-03T06:27:48.395 DEBUG:boto:Path: /dsgvcy-73/?prefix=c 2022-09-03T06:27:48.395 DEBUG:boto:Data: 2022-09-03T06:27:48.396 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.396 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:48.397 DEBUG:boto:Port: 8001 2022-09-03T06:27:48.397 DEBUG:boto:Params: {} 2022-09-03T06:27:48.398 DEBUG:boto:Token: None 2022-09-03T06:27:48.399 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/ 2022-09-03T06:27:48.400 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek= 2022-09-03T06:27:48.401 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek=', 'Content-Length': '0'} 2022-09-03T06:27:48.404 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000000e83b57997f305f-006312f3e4-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.404 DEBUG:boto:b'dsgvcy-73c1000falsec2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:48.405 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:48.406 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:48.407 DEBUG:boto:path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:48.407 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:48.408 DEBUG:boto:Method: GET 2022-09-03T06:27:48.408 DEBUG:boto:Path: /dsgvcy-73/?prefix=c 2022-09-03T06:27:48.409 DEBUG:boto:Data: 2022-09-03T06:27:48.409 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.411 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:48.412 DEBUG:boto:Port: 8000 2022-09-03T06:27:48.412 DEBUG:boto:Params: {} 2022-09-03T06:27:48.413 DEBUG:boto:Token: None 2022-09-03T06:27:48.414 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/ 2022-09-03T06:27:48.414 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek= 2022-09-03T06:27:48.415 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek=', 'Content-Length': '0'} 2022-09-03T06:27:48.417 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000070c645fb8f21a91e-006312f3e4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.423 DEBUG:boto:b'dsgvcy-73c1000falsec2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:48.426 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:27:48.427 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:48.427 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:48.428 DEBUG:boto:path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.428 DEBUG:boto:auth_path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.429 DEBUG:boto:Method: GET 2022-09-03T06:27:48.430 DEBUG:boto:Path: /dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.431 DEBUG:boto:Data: 2022-09-03T06:27:48.431 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.432 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:48.433 DEBUG:boto:Port: 8001 2022-09-03T06:27:48.434 DEBUG:boto:Params: {} 2022-09-03T06:27:48.434 DEBUG:boto:Token: None 2022-09-03T06:27:48.435 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.436 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KP8ucK3N58D3BhZU7IrqDG1jpfY= 2022-09-03T06:27:48.437 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KP8ucK3N58D3BhZU7IrqDG1jpfY=', 'Content-Length': '0'} 2022-09-03T06:27:48.439 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a181b5138f2a04fd-006312f3e4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.440 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:48.441 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:48.441 DEBUG:boto:path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.442 DEBUG:boto:auth_path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.442 DEBUG:boto:Method: GET 2022-09-03T06:27:48.443 DEBUG:boto:Path: /dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.443 DEBUG:boto:Data: 2022-09-03T06:27:48.444 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.444 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:48.445 DEBUG:boto:Port: 8000 2022-09-03T06:27:48.445 DEBUG:boto:Params: {} 2022-09-03T06:27:48.446 DEBUG:boto:Token: None 2022-09-03T06:27:48.447 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.448 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KP8ucK3N58D3BhZU7IrqDG1jpfY= 2022-09-03T06:27:48.449 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KP8ucK3N58D3BhZU7IrqDG1jpfY=', 'Content-Length': '0'} 2022-09-03T06:27:48.451 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000086277103763dcaa9-006312f3e4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.494 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:48.495 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:48.496 DEBUG:boto:path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.497 DEBUG:boto:auth_path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.499 DEBUG:boto:Method: HEAD 2022-09-03T06:27:48.499 DEBUG:boto:Path: /dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.500 DEBUG:boto:Data: 2022-09-03T06:27:48.501 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.501 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:48.502 DEBUG:boto:Port: 8001 2022-09-03T06:27:48.503 DEBUG:boto:Params: {} 2022-09-03T06:27:48.503 DEBUG:boto:Token: None 2022-09-03T06:27:48.504 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.504 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wXmOShVYkYWikEqbX5K1rNA99a0= 2022-09-03T06:27:48.505 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wXmOShVYkYWikEqbX5K1rNA99a0=', 'Content-Length': '0'} 2022-09-03T06:27:48.508 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d077a1d37a6767f5-006312f3e4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.509 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:48.509 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:48.510 DEBUG:boto:path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.510 DEBUG:boto:auth_path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.511 DEBUG:boto:Method: HEAD 2022-09-03T06:27:48.511 DEBUG:boto:Path: /dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.512 DEBUG:boto:Data: 2022-09-03T06:27:48.512 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.513 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:48.513 DEBUG:boto:Port: 8000 2022-09-03T06:27:48.514 DEBUG:boto:Params: {} 2022-09-03T06:27:48.515 DEBUG:boto:Token: None 2022-09-03T06:27:48.515 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.516 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wXmOShVYkYWikEqbX5K1rNA99a0= 2022-09-03T06:27:48.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wXmOShVYkYWikEqbX5K1rNA99a0=', 'Content-Length': '0'} 2022-09-03T06:27:48.519 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004427c2d38c31d824-006312f3e4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.520 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:27:48.520 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:48.521 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:48.521 DEBUG:boto:path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.526 DEBUG:boto:auth_path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.526 DEBUG:boto:Method: GET 2022-09-03T06:27:48.527 DEBUG:boto:Path: /dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.528 DEBUG:boto:Data: 2022-09-03T06:27:48.531 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.533 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:48.533 DEBUG:boto:Port: 8001 2022-09-03T06:27:48.535 DEBUG:boto:Params: {} 2022-09-03T06:27:48.535 DEBUG:boto:Token: None 2022-09-03T06:27:48.536 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.537 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KP8ucK3N58D3BhZU7IrqDG1jpfY= 2022-09-03T06:27:48.538 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KP8ucK3N58D3BhZU7IrqDG1jpfY=', 'Content-Length': '0'} 2022-09-03T06:27:48.540 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003df540e16918e888-006312f3e4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.542 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:48.544 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:48.546 DEBUG:boto:path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.547 DEBUG:boto:auth_path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.547 DEBUG:boto:Method: GET 2022-09-03T06:27:48.548 DEBUG:boto:Path: /dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.548 DEBUG:boto:Data: 2022-09-03T06:27:48.549 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.549 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:48.550 DEBUG:boto:Port: 8000 2022-09-03T06:27:48.551 DEBUG:boto:Params: {} 2022-09-03T06:27:48.552 DEBUG:boto:Token: None 2022-09-03T06:27:48.553 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/d?versionId=null 2022-09-03T06:27:48.553 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KP8ucK3N58D3BhZU7IrqDG1jpfY= 2022-09-03T06:27:48.554 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KP8ucK3N58D3BhZU7IrqDG1jpfY=', 'Content-Length': '0'} 2022-09-03T06:27:48.557 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000ec8afd2f7fc1cce-006312f3e4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.557 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:48.558 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:48.559 DEBUG:boto:path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:48.559 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:48.560 DEBUG:boto:Method: GET 2022-09-03T06:27:48.560 DEBUG:boto:Path: /dsgvcy-73/?prefix=d 2022-09-03T06:27:48.561 DEBUG:boto:Data: 2022-09-03T06:27:48.562 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.563 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:48.564 DEBUG:boto:Port: 8001 2022-09-03T06:27:48.565 DEBUG:boto:Params: {} 2022-09-03T06:27:48.566 DEBUG:boto:Token: None 2022-09-03T06:27:48.567 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/ 2022-09-03T06:27:48.569 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek= 2022-09-03T06:27:48.570 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek=', 'Content-Length': '0'} 2022-09-03T06:27:48.585 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008d03ffc1cf0a9044-006312f3e4-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.590 DEBUG:boto:b'dsgvcy-73d1000falsed2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:48.592 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:48.593 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:48.594 DEBUG:boto:path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:48.595 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:48.596 DEBUG:boto:Method: GET 2022-09-03T06:27:48.597 DEBUG:boto:Path: /dsgvcy-73/?prefix=d 2022-09-03T06:27:48.598 DEBUG:boto:Data: 2022-09-03T06:27:48.598 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.599 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:48.600 DEBUG:boto:Port: 8000 2022-09-03T06:27:48.600 DEBUG:boto:Params: {} 2022-09-03T06:27:48.602 DEBUG:boto:Token: None 2022-09-03T06:27:48.603 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/ 2022-09-03T06:27:48.604 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek= 2022-09-03T06:27:48.605 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek=', 'Content-Length': '0'} 2022-09-03T06:27:48.608 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000064a450a3acc4a93c-006312f3e4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.609 DEBUG:boto:b'dsgvcy-73d1000falsed2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:48.610 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:48.611 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:48.612 DEBUG:boto:Method: HEAD 2022-09-03T06:27:48.612 DEBUG:boto:Path: /dsgvcy-73/d 2022-09-03T06:27:48.613 DEBUG:boto:Data: 2022-09-03T06:27:48.613 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.614 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:48.614 DEBUG:boto:Port: 8001 2022-09-03T06:27:48.615 DEBUG:boto:Params: {} 2022-09-03T06:27:48.616 DEBUG:boto:Token: None 2022-09-03T06:27:48.616 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/d 2022-09-03T06:27:48.617 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TVcI7jndrJ82MjMKMpTJpmLy70E= 2022-09-03T06:27:48.617 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TVcI7jndrJ82MjMKMpTJpmLy70E=', 'Content-Length': '0'} 2022-09-03T06:27:48.620 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000522cde9836e17ad2-006312f3e4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.621 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:48.622 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:48.622 DEBUG:boto:Method: HEAD 2022-09-03T06:27:48.623 DEBUG:boto:Path: /dsgvcy-73/d 2022-09-03T06:27:48.624 DEBUG:boto:Data: 2022-09-03T06:27:48.624 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.625 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:48.626 DEBUG:boto:Port: 8000 2022-09-03T06:27:48.627 DEBUG:boto:Params: {} 2022-09-03T06:27:48.628 DEBUG:boto:Token: None 2022-09-03T06:27:48.628 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/d 2022-09-03T06:27:48.629 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TVcI7jndrJ82MjMKMpTJpmLy70E= 2022-09-03T06:27:48.630 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TVcI7jndrJ82MjMKMpTJpmLy70E=', 'Content-Length': '0'} 2022-09-03T06:27:48.632 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000042cc4d915ac0ea06-006312f3e4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.633 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:27:48.633 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:48.634 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:48.635 DEBUG:boto:Method: GET 2022-09-03T06:27:48.636 DEBUG:boto:Path: /dsgvcy-73/d 2022-09-03T06:27:48.637 DEBUG:boto:Data: 2022-09-03T06:27:48.638 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.639 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:48.639 DEBUG:boto:Port: 8001 2022-09-03T06:27:48.640 DEBUG:boto:Params: {} 2022-09-03T06:27:48.641 DEBUG:boto:Token: None 2022-09-03T06:27:48.642 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/d 2022-09-03T06:27:48.642 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OlEMkpOCDl8oSP2buKuR11s/O50= 2022-09-03T06:27:48.643 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OlEMkpOCDl8oSP2buKuR11s/O50=', 'Content-Length': '0'} 2022-09-03T06:27:48.645 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000037b320954374a825-006312f3e4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.646 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:48.647 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:48.648 DEBUG:boto:Method: GET 2022-09-03T06:27:48.648 DEBUG:boto:Path: /dsgvcy-73/d 2022-09-03T06:27:48.649 DEBUG:boto:Data: 2022-09-03T06:27:48.651 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.652 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:48.653 DEBUG:boto:Port: 8000 2022-09-03T06:27:48.653 DEBUG:boto:Params: {} 2022-09-03T06:27:48.655 DEBUG:boto:Token: None 2022-09-03T06:27:48.656 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/d 2022-09-03T06:27:48.657 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OlEMkpOCDl8oSP2buKuR11s/O50= 2022-09-03T06:27:48.658 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OlEMkpOCDl8oSP2buKuR11s/O50=', 'Content-Length': '0'} 2022-09-03T06:27:48.663 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001862aeef8c0b7a64-006312f3e4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.664 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:48.665 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:48.666 DEBUG:boto:path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:48.666 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:48.667 DEBUG:boto:Method: GET 2022-09-03T06:27:48.668 DEBUG:boto:Path: /dsgvcy-73/?prefix=d 2022-09-03T06:27:48.668 DEBUG:boto:Data: 2022-09-03T06:27:48.669 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.669 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:48.670 DEBUG:boto:Port: 8001 2022-09-03T06:27:48.671 DEBUG:boto:Params: {} 2022-09-03T06:27:48.672 DEBUG:boto:Token: None 2022-09-03T06:27:48.672 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/ 2022-09-03T06:27:48.673 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek= 2022-09-03T06:27:48.673 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek=', 'Content-Length': '0'} 2022-09-03T06:27:48.676 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000b526213ee00e30b-006312f3e4-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.677 DEBUG:boto:b'dsgvcy-73d1000falsed2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:48.678 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:48.679 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:48.679 DEBUG:boto:path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:48.680 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:48.680 DEBUG:boto:Method: GET 2022-09-03T06:27:48.681 DEBUG:boto:Path: /dsgvcy-73/?prefix=d 2022-09-03T06:27:48.681 DEBUG:boto:Data: 2022-09-03T06:27:48.682 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.683 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:48.683 DEBUG:boto:Port: 8000 2022-09-03T06:27:48.684 DEBUG:boto:Params: {} 2022-09-03T06:27:48.685 DEBUG:boto:Token: None 2022-09-03T06:27:48.685 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/ 2022-09-03T06:27:48.686 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek= 2022-09-03T06:27:48.687 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek=', 'Content-Length': '0'} 2022-09-03T06:27:48.689 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000207dba863acb215b-006312f3e4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.734 DEBUG:boto:b'dsgvcy-73d1000falsed2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:48.735 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:27:48.737 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:48.738 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:48.738 DEBUG:boto:path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.739 DEBUG:boto:auth_path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.739 DEBUG:boto:Method: GET 2022-09-03T06:27:48.740 DEBUG:boto:Path: /dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.741 DEBUG:boto:Data: 2022-09-03T06:27:48.741 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.742 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:48.743 DEBUG:boto:Port: 8001 2022-09-03T06:27:48.744 DEBUG:boto:Params: {} 2022-09-03T06:27:48.745 DEBUG:boto:Token: None 2022-09-03T06:27:48.746 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.747 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0vn/KDwTYZdr9TF5S7Fozeb6Mik= 2022-09-03T06:27:48.747 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0vn/KDwTYZdr9TF5S7Fozeb6Mik=', 'Content-Length': '0'} 2022-09-03T06:27:48.750 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f5bb38d6d6384157-006312f3e4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.751 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:48.752 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:48.752 DEBUG:boto:path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.753 DEBUG:boto:auth_path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.753 DEBUG:boto:Method: GET 2022-09-03T06:27:48.754 DEBUG:boto:Path: /dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.754 DEBUG:boto:Data: 2022-09-03T06:27:48.755 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.756 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:48.757 DEBUG:boto:Port: 8000 2022-09-03T06:27:48.757 DEBUG:boto:Params: {} 2022-09-03T06:27:48.758 DEBUG:boto:Token: None 2022-09-03T06:27:48.759 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.759 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0vn/KDwTYZdr9TF5S7Fozeb6Mik= 2022-09-03T06:27:48.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0vn/KDwTYZdr9TF5S7Fozeb6Mik=', 'Content-Length': '0'} 2022-09-03T06:27:48.763 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065a53387eb212dd1-006312f3e4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.765 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:48.766 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:48.767 DEBUG:boto:path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.768 DEBUG:boto:auth_path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.769 DEBUG:boto:Method: HEAD 2022-09-03T06:27:48.769 DEBUG:boto:Path: /dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.770 DEBUG:boto:Data: 2022-09-03T06:27:48.772 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.773 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:48.774 DEBUG:boto:Port: 8001 2022-09-03T06:27:48.775 DEBUG:boto:Params: {} 2022-09-03T06:27:48.776 DEBUG:boto:Token: None 2022-09-03T06:27:48.777 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.778 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aSS9UHnpBkvR4ZgWpR2jiUv/uus= 2022-09-03T06:27:48.779 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aSS9UHnpBkvR4ZgWpR2jiUv/uus=', 'Content-Length': '0'} 2022-09-03T06:27:48.782 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000516d600601c22dda-006312f3e4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.785 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:48.786 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:48.787 DEBUG:boto:path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.789 DEBUG:boto:auth_path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.790 DEBUG:boto:Method: HEAD 2022-09-03T06:27:48.791 DEBUG:boto:Path: /dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.792 DEBUG:boto:Data: 2022-09-03T06:27:48.793 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.794 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:48.795 DEBUG:boto:Port: 8000 2022-09-03T06:27:48.796 DEBUG:boto:Params: {} 2022-09-03T06:27:48.798 DEBUG:boto:Token: None 2022-09-03T06:27:48.799 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.801 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aSS9UHnpBkvR4ZgWpR2jiUv/uus= 2022-09-03T06:27:48.802 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aSS9UHnpBkvR4ZgWpR2jiUv/uus=', 'Content-Length': '0'} 2022-09-03T06:27:48.812 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006d533bda31b19365-006312f3e4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.813 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:27:48.813 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:48.814 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:48.815 DEBUG:boto:path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.816 DEBUG:boto:auth_path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.816 DEBUG:boto:Method: GET 2022-09-03T06:27:48.817 DEBUG:boto:Path: /dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.817 DEBUG:boto:Data: 2022-09-03T06:27:48.818 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.818 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:48.819 DEBUG:boto:Port: 8001 2022-09-03T06:27:48.820 DEBUG:boto:Params: {} 2022-09-03T06:27:48.821 DEBUG:boto:Token: None 2022-09-03T06:27:48.821 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.822 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0vn/KDwTYZdr9TF5S7Fozeb6Mik= 2022-09-03T06:27:48.822 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0vn/KDwTYZdr9TF5S7Fozeb6Mik=', 'Content-Length': '0'} 2022-09-03T06:27:48.824 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b9a618cda8b7612a-006312f3e4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.870 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:48.871 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:48.872 DEBUG:boto:path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.873 DEBUG:boto:auth_path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.873 DEBUG:boto:Method: GET 2022-09-03T06:27:48.874 DEBUG:boto:Path: /dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.875 DEBUG:boto:Data: 2022-09-03T06:27:48.875 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.876 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:48.876 DEBUG:boto:Port: 8000 2022-09-03T06:27:48.877 DEBUG:boto:Params: {} 2022-09-03T06:27:48.883 DEBUG:boto:Token: None 2022-09-03T06:27:48.885 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/e?versionId=null 2022-09-03T06:27:48.886 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0vn/KDwTYZdr9TF5S7Fozeb6Mik= 2022-09-03T06:27:48.887 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0vn/KDwTYZdr9TF5S7Fozeb6Mik=', 'Content-Length': '0'} 2022-09-03T06:27:48.897 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003158c4acb04dd546-006312f3e4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.898 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:48.899 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:48.900 DEBUG:boto:path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:48.900 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:48.901 DEBUG:boto:Method: GET 2022-09-03T06:27:48.901 DEBUG:boto:Path: /dsgvcy-73/?prefix=e 2022-09-03T06:27:48.902 DEBUG:boto:Data: 2022-09-03T06:27:48.903 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.903 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:48.904 DEBUG:boto:Port: 8001 2022-09-03T06:27:48.904 DEBUG:boto:Params: {} 2022-09-03T06:27:48.905 DEBUG:boto:Token: None 2022-09-03T06:27:48.906 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/ 2022-09-03T06:27:48.906 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek= 2022-09-03T06:27:48.907 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek=', 'Content-Length': '0'} 2022-09-03T06:27:48.910 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000ca7bd2b7e4a3fb4-006312f3e4-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.911 DEBUG:boto:b'dsgvcy-73e1000falsee2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:48.912 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:48.912 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:48.913 DEBUG:boto:path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:48.913 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:48.914 DEBUG:boto:Method: GET 2022-09-03T06:27:48.914 DEBUG:boto:Path: /dsgvcy-73/?prefix=e 2022-09-03T06:27:48.915 DEBUG:boto:Data: 2022-09-03T06:27:48.915 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.916 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:48.916 DEBUG:boto:Port: 8000 2022-09-03T06:27:48.917 DEBUG:boto:Params: {} 2022-09-03T06:27:48.918 DEBUG:boto:Token: None 2022-09-03T06:27:48.918 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/ 2022-09-03T06:27:48.919 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek= 2022-09-03T06:27:48.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UEiyHOYoW3LIAf27yiGhg/DUsek=', 'Content-Length': '0'} 2022-09-03T06:27:48.922 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005fd48fe67915f505-006312f3e4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.923 DEBUG:boto:b'dsgvcy-73e1000falsee2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:48.924 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:48.924 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:48.925 DEBUG:boto:Method: HEAD 2022-09-03T06:27:48.925 DEBUG:boto:Path: /dsgvcy-73/e 2022-09-03T06:27:48.926 DEBUG:boto:Data: 2022-09-03T06:27:48.926 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.927 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:48.927 DEBUG:boto:Port: 8001 2022-09-03T06:27:48.928 DEBUG:boto:Params: {} 2022-09-03T06:27:48.928 DEBUG:boto:Token: None 2022-09-03T06:27:48.929 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/e 2022-09-03T06:27:48.930 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:afy4mCTh2ajuRpByi+v1JzbeIXI= 2022-09-03T06:27:48.930 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:afy4mCTh2ajuRpByi+v1JzbeIXI=', 'Content-Length': '0'} 2022-09-03T06:27:48.932 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d6070219741d13e5-006312f3e4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.933 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:48.934 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:48.934 DEBUG:boto:Method: HEAD 2022-09-03T06:27:48.935 DEBUG:boto:Path: /dsgvcy-73/e 2022-09-03T06:27:48.935 DEBUG:boto:Data: 2022-09-03T06:27:48.936 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.936 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:48.937 DEBUG:boto:Port: 8000 2022-09-03T06:27:48.938 DEBUG:boto:Params: {} 2022-09-03T06:27:48.938 DEBUG:boto:Token: None 2022-09-03T06:27:48.939 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/e 2022-09-03T06:27:48.939 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:afy4mCTh2ajuRpByi+v1JzbeIXI= 2022-09-03T06:27:48.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:afy4mCTh2ajuRpByi+v1JzbeIXI=', 'Content-Length': '0'} 2022-09-03T06:27:48.942 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000058c8dc0e72ac695b-006312f3e4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.943 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:27:48.943 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:48.944 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:48.945 DEBUG:boto:Method: GET 2022-09-03T06:27:48.945 DEBUG:boto:Path: /dsgvcy-73/e 2022-09-03T06:27:48.946 DEBUG:boto:Data: 2022-09-03T06:27:48.946 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.947 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:48.947 DEBUG:boto:Port: 8001 2022-09-03T06:27:48.948 DEBUG:boto:Params: {} 2022-09-03T06:27:48.948 DEBUG:boto:Token: None 2022-09-03T06:27:48.949 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/e 2022-09-03T06:27:48.950 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ChNWBCisJlVPhUut/3fxnLlFwqA= 2022-09-03T06:27:48.950 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ChNWBCisJlVPhUut/3fxnLlFwqA=', 'Content-Length': '0'} 2022-09-03T06:27:48.953 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000da46b19d45566069-006312f3e4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:48.954 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:48.954 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:48.955 DEBUG:boto:Method: GET 2022-09-03T06:27:48.956 DEBUG:boto:Path: /dsgvcy-73/e 2022-09-03T06:27:48.956 DEBUG:boto:Data: 2022-09-03T06:27:48.957 DEBUG:boto:Headers: {} 2022-09-03T06:27:48.957 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:48.958 DEBUG:boto:Port: 8000 2022-09-03T06:27:48.958 DEBUG:boto:Params: {} 2022-09-03T06:27:48.959 DEBUG:boto:Token: None 2022-09-03T06:27:48.959 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:48 GMT /dsgvcy-73/e 2022-09-03T06:27:48.960 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ChNWBCisJlVPhUut/3fxnLlFwqA= 2022-09-03T06:27:48.961 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ChNWBCisJlVPhUut/3fxnLlFwqA=', 'Content-Length': '0'} 2022-09-03T06:27:48.963 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000092b98a945b8e6e78-006312f3e4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.006 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:49.007 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:49.008 DEBUG:boto:path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:49.009 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:49.011 DEBUG:boto:Method: GET 2022-09-03T06:27:49.012 DEBUG:boto:Path: /dsgvcy-73/?prefix=e 2022-09-03T06:27:49.013 DEBUG:boto:Data: 2022-09-03T06:27:49.014 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.015 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:49.016 DEBUG:boto:Port: 8001 2022-09-03T06:27:49.018 DEBUG:boto:Params: {} 2022-09-03T06:27:49.019 DEBUG:boto:Token: None 2022-09-03T06:27:49.020 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/ 2022-09-03T06:27:49.021 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM= 2022-09-03T06:27:49.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM=', 'Content-Length': '0'} 2022-09-03T06:27:49.027 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009a7cf7da1af6deab-006312f3e5-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.028 DEBUG:boto:b'dsgvcy-73e1000falsee2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:49.029 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:49.029 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:49.030 DEBUG:boto:path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:49.030 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:49.031 DEBUG:boto:Method: GET 2022-09-03T06:27:49.032 DEBUG:boto:Path: /dsgvcy-73/?prefix=e 2022-09-03T06:27:49.032 DEBUG:boto:Data: 2022-09-03T06:27:49.033 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.034 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:49.035 DEBUG:boto:Port: 8000 2022-09-03T06:27:49.036 DEBUG:boto:Params: {} 2022-09-03T06:27:49.037 DEBUG:boto:Token: None 2022-09-03T06:27:49.038 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/ 2022-09-03T06:27:49.040 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM= 2022-09-03T06:27:49.041 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM=', 'Content-Length': '0'} 2022-09-03T06:27:49.045 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000074579e04f731fc06-006312f3e5-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.046 DEBUG:boto:b'dsgvcy-73e1000falsee2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:49.047 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:27:49.048 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:49.048 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:49.049 DEBUG:boto:path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.049 DEBUG:boto:auth_path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.050 DEBUG:boto:Method: GET 2022-09-03T06:27:49.050 DEBUG:boto:Path: /dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.051 DEBUG:boto:Data: 2022-09-03T06:27:49.051 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.052 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:49.053 DEBUG:boto:Port: 8001 2022-09-03T06:27:49.053 DEBUG:boto:Params: {} 2022-09-03T06:27:49.054 DEBUG:boto:Token: None 2022-09-03T06:27:49.054 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.056 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vgtZY77o5ALSsvilemjk0kQwk4M= 2022-09-03T06:27:49.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vgtZY77o5ALSsvilemjk0kQwk4M=', 'Content-Length': '0'} 2022-09-03T06:27:49.059 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000e572ff842c0160b-006312f3e5-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.106 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:49.107 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:49.108 DEBUG:boto:path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.109 DEBUG:boto:auth_path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.110 DEBUG:boto:Method: GET 2022-09-03T06:27:49.111 DEBUG:boto:Path: /dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.112 DEBUG:boto:Data: 2022-09-03T06:27:49.113 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.114 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:49.115 DEBUG:boto:Port: 8000 2022-09-03T06:27:49.116 DEBUG:boto:Params: {} 2022-09-03T06:27:49.117 DEBUG:boto:Token: None 2022-09-03T06:27:49.118 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.119 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vgtZY77o5ALSsvilemjk0kQwk4M= 2022-09-03T06:27:49.120 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vgtZY77o5ALSsvilemjk0kQwk4M=', 'Content-Length': '0'} 2022-09-03T06:27:49.130 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eeac33109bebfb38-006312f3e5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.131 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:49.133 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:49.133 DEBUG:boto:path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.134 DEBUG:boto:auth_path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.134 DEBUG:boto:Method: HEAD 2022-09-03T06:27:49.135 DEBUG:boto:Path: /dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.135 DEBUG:boto:Data: 2022-09-03T06:27:49.136 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.136 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:49.137 DEBUG:boto:Port: 8001 2022-09-03T06:27:49.137 DEBUG:boto:Params: {} 2022-09-03T06:27:49.138 DEBUG:boto:Token: None 2022-09-03T06:27:49.139 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.139 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V47LSnFvcixTglqumpg88B0AAAI= 2022-09-03T06:27:49.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V47LSnFvcixTglqumpg88B0AAAI=', 'Content-Length': '0'} 2022-09-03T06:27:49.144 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006bcd9ad75facbf9d-006312f3e5-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.146 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:49.147 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:49.148 DEBUG:boto:path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.148 DEBUG:boto:auth_path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.149 DEBUG:boto:Method: HEAD 2022-09-03T06:27:49.149 DEBUG:boto:Path: /dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.150 DEBUG:boto:Data: 2022-09-03T06:27:49.151 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.151 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:49.152 DEBUG:boto:Port: 8000 2022-09-03T06:27:49.153 DEBUG:boto:Params: {} 2022-09-03T06:27:49.154 DEBUG:boto:Token: None 2022-09-03T06:27:49.155 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.156 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V47LSnFvcixTglqumpg88B0AAAI= 2022-09-03T06:27:49.157 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V47LSnFvcixTglqumpg88B0AAAI=', 'Content-Length': '0'} 2022-09-03T06:27:49.162 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a17d0856a96b5f56-006312f3e5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.163 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:27:49.164 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:49.166 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:49.167 DEBUG:boto:path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.168 DEBUG:boto:auth_path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.169 DEBUG:boto:Method: GET 2022-09-03T06:27:49.170 DEBUG:boto:Path: /dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.171 DEBUG:boto:Data: 2022-09-03T06:27:49.172 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.173 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:49.174 DEBUG:boto:Port: 8001 2022-09-03T06:27:49.175 DEBUG:boto:Params: {} 2022-09-03T06:27:49.176 DEBUG:boto:Token: None 2022-09-03T06:27:49.177 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.178 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vgtZY77o5ALSsvilemjk0kQwk4M= 2022-09-03T06:27:49.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vgtZY77o5ALSsvilemjk0kQwk4M=', 'Content-Length': '0'} 2022-09-03T06:27:49.184 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006d919b21ae7ab4d0-006312f3e5-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.186 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:49.187 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:49.188 DEBUG:boto:path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.189 DEBUG:boto:auth_path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.190 DEBUG:boto:Method: GET 2022-09-03T06:27:49.191 DEBUG:boto:Path: /dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.191 DEBUG:boto:Data: 2022-09-03T06:27:49.192 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.193 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:49.194 DEBUG:boto:Port: 8000 2022-09-03T06:27:49.195 DEBUG:boto:Params: {} 2022-09-03T06:27:49.196 DEBUG:boto:Token: None 2022-09-03T06:27:49.196 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/f?versionId=null 2022-09-03T06:27:49.197 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vgtZY77o5ALSsvilemjk0kQwk4M= 2022-09-03T06:27:49.198 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vgtZY77o5ALSsvilemjk0kQwk4M=', 'Content-Length': '0'} 2022-09-03T06:27:49.200 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000024156463608e55ec-006312f3e5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.201 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:49.202 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:49.203 DEBUG:boto:path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:49.204 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:49.205 DEBUG:boto:Method: GET 2022-09-03T06:27:49.206 DEBUG:boto:Path: /dsgvcy-73/?prefix=f 2022-09-03T06:27:49.207 DEBUG:boto:Data: 2022-09-03T06:27:49.208 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.209 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:49.210 DEBUG:boto:Port: 8001 2022-09-03T06:27:49.211 DEBUG:boto:Params: {} 2022-09-03T06:27:49.212 DEBUG:boto:Token: None 2022-09-03T06:27:49.213 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/ 2022-09-03T06:27:49.214 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM= 2022-09-03T06:27:49.215 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM=', 'Content-Length': '0'} 2022-09-03T06:27:49.219 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b53b95d760e95de1-006312f3e5-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.220 DEBUG:boto:b'dsgvcy-73f1000falsef2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:49.221 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:49.222 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:49.223 DEBUG:boto:path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:49.224 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:49.226 DEBUG:boto:Method: GET 2022-09-03T06:27:49.227 DEBUG:boto:Path: /dsgvcy-73/?prefix=f 2022-09-03T06:27:49.228 DEBUG:boto:Data: 2022-09-03T06:27:49.229 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.230 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:49.231 DEBUG:boto:Port: 8000 2022-09-03T06:27:49.232 DEBUG:boto:Params: {} 2022-09-03T06:27:49.233 DEBUG:boto:Token: None 2022-09-03T06:27:49.234 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/ 2022-09-03T06:27:49.236 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM= 2022-09-03T06:27:49.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM=', 'Content-Length': '0'} 2022-09-03T06:27:49.241 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002ee15446db4364e8-006312f3e5-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.243 DEBUG:boto:b'dsgvcy-73f1000falsef2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:49.244 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:49.245 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:49.246 DEBUG:boto:Method: HEAD 2022-09-03T06:27:49.247 DEBUG:boto:Path: /dsgvcy-73/f 2022-09-03T06:27:49.247 DEBUG:boto:Data: 2022-09-03T06:27:49.248 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.248 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:49.249 DEBUG:boto:Port: 8001 2022-09-03T06:27:49.249 DEBUG:boto:Params: {} 2022-09-03T06:27:49.250 DEBUG:boto:Token: None 2022-09-03T06:27:49.251 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/f 2022-09-03T06:27:49.252 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hc4TJnRyTBKOqAX2Pdeu43y4N+A= 2022-09-03T06:27:49.253 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hc4TJnRyTBKOqAX2Pdeu43y4N+A=', 'Content-Length': '0'} 2022-09-03T06:27:49.256 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000bb2ea786fb3bdfb-006312f3e5-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.257 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:49.259 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:49.259 DEBUG:boto:Method: HEAD 2022-09-03T06:27:49.260 DEBUG:boto:Path: /dsgvcy-73/f 2022-09-03T06:27:49.261 DEBUG:boto:Data: 2022-09-03T06:27:49.262 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.263 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:49.264 DEBUG:boto:Port: 8000 2022-09-03T06:27:49.265 DEBUG:boto:Params: {} 2022-09-03T06:27:49.266 DEBUG:boto:Token: None 2022-09-03T06:27:49.267 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/f 2022-09-03T06:27:49.267 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hc4TJnRyTBKOqAX2Pdeu43y4N+A= 2022-09-03T06:27:49.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hc4TJnRyTBKOqAX2Pdeu43y4N+A=', 'Content-Length': '0'} 2022-09-03T06:27:49.271 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000052afb5365086a0aa-006312f3e5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.272 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:27:49.273 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:49.274 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:49.275 DEBUG:boto:Method: GET 2022-09-03T06:27:49.276 DEBUG:boto:Path: /dsgvcy-73/f 2022-09-03T06:27:49.278 DEBUG:boto:Data: 2022-09-03T06:27:49.279 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.280 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:49.281 DEBUG:boto:Port: 8001 2022-09-03T06:27:49.283 DEBUG:boto:Params: {} 2022-09-03T06:27:49.284 DEBUG:boto:Token: None 2022-09-03T06:27:49.285 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/f 2022-09-03T06:27:49.287 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eLG45QEkaPNSjdN1FlrkMiSR4wg= 2022-09-03T06:27:49.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eLG45QEkaPNSjdN1FlrkMiSR4wg=', 'Content-Length': '0'} 2022-09-03T06:27:49.290 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000040f19d6a696607d9-006312f3e5-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.334 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:49.335 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:49.338 DEBUG:boto:Method: GET 2022-09-03T06:27:49.341 DEBUG:boto:Path: /dsgvcy-73/f 2022-09-03T06:27:49.341 DEBUG:boto:Data: 2022-09-03T06:27:49.342 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.343 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:49.343 DEBUG:boto:Port: 8000 2022-09-03T06:27:49.344 DEBUG:boto:Params: {} 2022-09-03T06:27:49.346 DEBUG:boto:Token: None 2022-09-03T06:27:49.347 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/f 2022-09-03T06:27:49.348 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eLG45QEkaPNSjdN1FlrkMiSR4wg= 2022-09-03T06:27:49.349 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eLG45QEkaPNSjdN1FlrkMiSR4wg=', 'Content-Length': '0'} 2022-09-03T06:27:49.354 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000f04339e7da1585e-006312f3e5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.355 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:49.356 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:49.357 DEBUG:boto:path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:49.357 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:49.358 DEBUG:boto:Method: GET 2022-09-03T06:27:49.359 DEBUG:boto:Path: /dsgvcy-73/?prefix=f 2022-09-03T06:27:49.360 DEBUG:boto:Data: 2022-09-03T06:27:49.361 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.363 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:49.364 DEBUG:boto:Port: 8001 2022-09-03T06:27:49.364 DEBUG:boto:Params: {} 2022-09-03T06:27:49.365 DEBUG:boto:Token: None 2022-09-03T06:27:49.366 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/ 2022-09-03T06:27:49.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM= 2022-09-03T06:27:49.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM=', 'Content-Length': '0'} 2022-09-03T06:27:49.370 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000003ae6deb52ab4c5a-006312f3e5-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.371 DEBUG:boto:b'dsgvcy-73f1000falsef2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:49.372 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:49.374 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:49.375 DEBUG:boto:path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:49.375 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:49.376 DEBUG:boto:Method: GET 2022-09-03T06:27:49.377 DEBUG:boto:Path: /dsgvcy-73/?prefix=f 2022-09-03T06:27:49.378 DEBUG:boto:Data: 2022-09-03T06:27:49.378 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.379 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:49.379 DEBUG:boto:Port: 8000 2022-09-03T06:27:49.380 DEBUG:boto:Params: {} 2022-09-03T06:27:49.381 DEBUG:boto:Token: None 2022-09-03T06:27:49.381 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/ 2022-09-03T06:27:49.383 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM= 2022-09-03T06:27:49.384 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM=', 'Content-Length': '0'} 2022-09-03T06:27:49.387 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000038d06aaa92d4d25b-006312f3e5-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.388 DEBUG:boto:b'dsgvcy-73f1000falsef2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:49.389 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:27:49.390 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:49.391 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:49.391 DEBUG:boto:path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.394 DEBUG:boto:auth_path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.396 DEBUG:boto:Method: GET 2022-09-03T06:27:49.397 DEBUG:boto:Path: /dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.397 DEBUG:boto:Data: 2022-09-03T06:27:49.398 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.398 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:49.399 DEBUG:boto:Port: 8001 2022-09-03T06:27:49.399 DEBUG:boto:Params: {} 2022-09-03T06:27:49.400 DEBUG:boto:Token: None 2022-09-03T06:27:49.400 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.401 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5VscA4NMbKvgNVjAaMr57GinVXU= 2022-09-03T06:27:49.402 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5VscA4NMbKvgNVjAaMr57GinVXU=', 'Content-Length': '0'} 2022-09-03T06:27:49.404 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007aae283294eb3c2d-006312f3e5-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.405 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:49.406 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:49.407 DEBUG:boto:path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.407 DEBUG:boto:auth_path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.408 DEBUG:boto:Method: GET 2022-09-03T06:27:49.409 DEBUG:boto:Path: /dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.409 DEBUG:boto:Data: 2022-09-03T06:27:49.411 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.412 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:49.412 DEBUG:boto:Port: 8000 2022-09-03T06:27:49.413 DEBUG:boto:Params: {} 2022-09-03T06:27:49.414 DEBUG:boto:Token: None 2022-09-03T06:27:49.415 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.416 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5VscA4NMbKvgNVjAaMr57GinVXU= 2022-09-03T06:27:49.417 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5VscA4NMbKvgNVjAaMr57GinVXU=', 'Content-Length': '0'} 2022-09-03T06:27:49.419 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007d0b7f34076c2989-006312f3e5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.462 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:49.463 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:49.464 DEBUG:boto:path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.465 DEBUG:boto:auth_path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.466 DEBUG:boto:Method: HEAD 2022-09-03T06:27:49.467 DEBUG:boto:Path: /dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.468 DEBUG:boto:Data: 2022-09-03T06:27:49.468 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.469 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:49.469 DEBUG:boto:Port: 8001 2022-09-03T06:27:49.470 DEBUG:boto:Params: {} 2022-09-03T06:27:49.471 DEBUG:boto:Token: None 2022-09-03T06:27:49.472 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.472 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vquTH2h+5SMb4yHpJBnNL2Bf0k0= 2022-09-03T06:27:49.473 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vquTH2h+5SMb4yHpJBnNL2Bf0k0=', 'Content-Length': '0'} 2022-09-03T06:27:49.475 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002893b2714f021494-006312f3e5-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.477 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:49.479 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:49.479 DEBUG:boto:path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.480 DEBUG:boto:auth_path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.480 DEBUG:boto:Method: HEAD 2022-09-03T06:27:49.481 DEBUG:boto:Path: /dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.482 DEBUG:boto:Data: 2022-09-03T06:27:49.483 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.484 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:49.485 DEBUG:boto:Port: 8000 2022-09-03T06:27:49.486 DEBUG:boto:Params: {} 2022-09-03T06:27:49.487 DEBUG:boto:Token: None 2022-09-03T06:27:49.488 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.489 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vquTH2h+5SMb4yHpJBnNL2Bf0k0= 2022-09-03T06:27:49.490 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vquTH2h+5SMb4yHpJBnNL2Bf0k0=', 'Content-Length': '0'} 2022-09-03T06:27:49.493 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f1de1fd494973f0-006312f3e5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.495 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:27:49.496 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:49.496 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:49.497 DEBUG:boto:path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.497 DEBUG:boto:auth_path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.498 DEBUG:boto:Method: GET 2022-09-03T06:27:49.498 DEBUG:boto:Path: /dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.499 DEBUG:boto:Data: 2022-09-03T06:27:49.500 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.500 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:49.501 DEBUG:boto:Port: 8001 2022-09-03T06:27:49.502 DEBUG:boto:Params: {} 2022-09-03T06:27:49.503 DEBUG:boto:Token: None 2022-09-03T06:27:49.504 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.505 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5VscA4NMbKvgNVjAaMr57GinVXU= 2022-09-03T06:27:49.506 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5VscA4NMbKvgNVjAaMr57GinVXU=', 'Content-Length': '0'} 2022-09-03T06:27:49.509 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000df68a3811fedfd95-006312f3e5-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.510 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:49.511 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:49.511 DEBUG:boto:path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.512 DEBUG:boto:auth_path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.512 DEBUG:boto:Method: GET 2022-09-03T06:27:49.513 DEBUG:boto:Path: /dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.513 DEBUG:boto:Data: 2022-09-03T06:27:49.514 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.515 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:49.516 DEBUG:boto:Port: 8000 2022-09-03T06:27:49.516 DEBUG:boto:Params: {} 2022-09-03T06:27:49.517 DEBUG:boto:Token: None 2022-09-03T06:27:49.518 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/g?versionId=null 2022-09-03T06:27:49.519 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5VscA4NMbKvgNVjAaMr57GinVXU= 2022-09-03T06:27:49.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5VscA4NMbKvgNVjAaMr57GinVXU=', 'Content-Length': '0'} 2022-09-03T06:27:49.522 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000034afb709a5757e95-006312f3e5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.523 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:49.524 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:49.525 DEBUG:boto:path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:49.526 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:49.526 DEBUG:boto:Method: GET 2022-09-03T06:27:49.527 DEBUG:boto:Path: /dsgvcy-73/?prefix=g 2022-09-03T06:27:49.527 DEBUG:boto:Data: 2022-09-03T06:27:49.528 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.528 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:49.529 DEBUG:boto:Port: 8001 2022-09-03T06:27:49.529 DEBUG:boto:Params: {} 2022-09-03T06:27:49.530 DEBUG:boto:Token: None 2022-09-03T06:27:49.531 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/ 2022-09-03T06:27:49.532 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM= 2022-09-03T06:27:49.533 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM=', 'Content-Length': '0'} 2022-09-03T06:27:49.537 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009b476890ed28d7bb-006312f3e5-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.538 DEBUG:boto:b'dsgvcy-73g1000falseg2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:49.540 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:49.541 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:49.542 DEBUG:boto:path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:49.542 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:49.543 DEBUG:boto:Method: GET 2022-09-03T06:27:49.544 DEBUG:boto:Path: /dsgvcy-73/?prefix=g 2022-09-03T06:27:49.545 DEBUG:boto:Data: 2022-09-03T06:27:49.545 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.546 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:49.547 DEBUG:boto:Port: 8000 2022-09-03T06:27:49.548 DEBUG:boto:Params: {} 2022-09-03T06:27:49.549 DEBUG:boto:Token: None 2022-09-03T06:27:49.550 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/ 2022-09-03T06:27:49.551 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM= 2022-09-03T06:27:49.553 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM=', 'Content-Length': '0'} 2022-09-03T06:27:49.557 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e851c71600878ecb-006312f3e5-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.559 DEBUG:boto:b'dsgvcy-73g1000falseg2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:49.561 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:49.561 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:49.562 DEBUG:boto:Method: HEAD 2022-09-03T06:27:49.564 DEBUG:boto:Path: /dsgvcy-73/g 2022-09-03T06:27:49.565 DEBUG:boto:Data: 2022-09-03T06:27:49.565 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.566 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:49.566 DEBUG:boto:Port: 8001 2022-09-03T06:27:49.567 DEBUG:boto:Params: {} 2022-09-03T06:27:49.567 DEBUG:boto:Token: None 2022-09-03T06:27:49.568 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/g 2022-09-03T06:27:49.569 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fnPN69qj1qVgKT1ORHQp+25gpwc= 2022-09-03T06:27:49.569 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fnPN69qj1qVgKT1ORHQp+25gpwc=', 'Content-Length': '0'} 2022-09-03T06:27:49.571 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000041dee23f98352f04-006312f3e5-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.572 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:49.573 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:49.573 DEBUG:boto:Method: HEAD 2022-09-03T06:27:49.574 DEBUG:boto:Path: /dsgvcy-73/g 2022-09-03T06:27:49.574 DEBUG:boto:Data: 2022-09-03T06:27:49.575 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.576 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:49.576 DEBUG:boto:Port: 8000 2022-09-03T06:27:49.577 DEBUG:boto:Params: {} 2022-09-03T06:27:49.577 DEBUG:boto:Token: None 2022-09-03T06:27:49.578 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/g 2022-09-03T06:27:49.578 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fnPN69qj1qVgKT1ORHQp+25gpwc= 2022-09-03T06:27:49.579 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fnPN69qj1qVgKT1ORHQp+25gpwc=', 'Content-Length': '0'} 2022-09-03T06:27:49.583 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000088766c8e0bca6290-006312f3e5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.584 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:27:49.585 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:49.586 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:49.587 DEBUG:boto:Method: GET 2022-09-03T06:27:49.588 DEBUG:boto:Path: /dsgvcy-73/g 2022-09-03T06:27:49.589 DEBUG:boto:Data: 2022-09-03T06:27:49.590 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.591 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:49.592 DEBUG:boto:Port: 8001 2022-09-03T06:27:49.593 DEBUG:boto:Params: {} 2022-09-03T06:27:49.594 DEBUG:boto:Token: None 2022-09-03T06:27:49.595 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/g 2022-09-03T06:27:49.595 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ky+1+zTbkbSAPmbJvh4iekFE2wQ= 2022-09-03T06:27:49.597 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ky+1+zTbkbSAPmbJvh4iekFE2wQ=', 'Content-Length': '0'} 2022-09-03T06:27:49.600 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000568df8a4ab47ae13-006312f3e5-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.602 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:49.603 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:49.604 DEBUG:boto:Method: GET 2022-09-03T06:27:49.605 DEBUG:boto:Path: /dsgvcy-73/g 2022-09-03T06:27:49.606 DEBUG:boto:Data: 2022-09-03T06:27:49.607 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.608 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:49.609 DEBUG:boto:Port: 8000 2022-09-03T06:27:49.610 DEBUG:boto:Params: {} 2022-09-03T06:27:49.611 DEBUG:boto:Token: None 2022-09-03T06:27:49.612 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/g 2022-09-03T06:27:49.613 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ky+1+zTbkbSAPmbJvh4iekFE2wQ= 2022-09-03T06:27:49.614 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ky+1+zTbkbSAPmbJvh4iekFE2wQ=', 'Content-Length': '0'} 2022-09-03T06:27:49.618 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006082d33ed7ebb6cd-006312f3e5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.662 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:49.663 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:49.664 DEBUG:boto:path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:49.664 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:49.665 DEBUG:boto:Method: GET 2022-09-03T06:27:49.665 DEBUG:boto:Path: /dsgvcy-73/?prefix=g 2022-09-03T06:27:49.666 DEBUG:boto:Data: 2022-09-03T06:27:49.666 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.667 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:49.668 DEBUG:boto:Port: 8001 2022-09-03T06:27:49.668 DEBUG:boto:Params: {} 2022-09-03T06:27:49.669 DEBUG:boto:Token: None 2022-09-03T06:27:49.669 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/ 2022-09-03T06:27:49.671 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM= 2022-09-03T06:27:49.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM=', 'Content-Length': '0'} 2022-09-03T06:27:49.675 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002f2b6e045bdace7b-006312f3e5-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.677 DEBUG:boto:b'dsgvcy-73g1000falseg2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:49.680 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:49.680 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:49.682 DEBUG:boto:path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:49.682 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:49.683 DEBUG:boto:Method: GET 2022-09-03T06:27:49.683 DEBUG:boto:Path: /dsgvcy-73/?prefix=g 2022-09-03T06:27:49.684 DEBUG:boto:Data: 2022-09-03T06:27:49.685 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.685 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:49.686 DEBUG:boto:Port: 8000 2022-09-03T06:27:49.686 DEBUG:boto:Params: {} 2022-09-03T06:27:49.687 DEBUG:boto:Token: None 2022-09-03T06:27:49.688 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/ 2022-09-03T06:27:49.688 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM= 2022-09-03T06:27:49.689 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM=', 'Content-Length': '0'} 2022-09-03T06:27:49.692 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d186fc2fa4149ba3-006312f3e5-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.693 DEBUG:boto:b'dsgvcy-73g1000falseg2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:49.694 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:27:49.694 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:49.695 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:49.696 DEBUG:boto:path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.696 DEBUG:boto:auth_path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.697 DEBUG:boto:Method: GET 2022-09-03T06:27:49.698 DEBUG:boto:Path: /dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.698 DEBUG:boto:Data: 2022-09-03T06:27:49.699 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.699 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:49.700 DEBUG:boto:Port: 8001 2022-09-03T06:27:49.700 DEBUG:boto:Params: {} 2022-09-03T06:27:49.701 DEBUG:boto:Token: None 2022-09-03T06:27:49.702 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.702 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iyjdvlx3/kiaSRVvtmJn8ZM+x4E= 2022-09-03T06:27:49.703 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iyjdvlx3/kiaSRVvtmJn8ZM+x4E=', 'Content-Length': '0'} 2022-09-03T06:27:49.706 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000809ed18af324c101-006312f3e5-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.750 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:49.752 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:49.753 DEBUG:boto:path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.753 DEBUG:boto:auth_path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.755 DEBUG:boto:Method: GET 2022-09-03T06:27:49.756 DEBUG:boto:Path: /dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.757 DEBUG:boto:Data: 2022-09-03T06:27:49.758 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.759 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:49.760 DEBUG:boto:Port: 8000 2022-09-03T06:27:49.761 DEBUG:boto:Params: {} 2022-09-03T06:27:49.762 DEBUG:boto:Token: None 2022-09-03T06:27:49.764 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.765 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iyjdvlx3/kiaSRVvtmJn8ZM+x4E= 2022-09-03T06:27:49.767 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iyjdvlx3/kiaSRVvtmJn8ZM+x4E=', 'Content-Length': '0'} 2022-09-03T06:27:49.777 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000052acddd4db7d79f8-006312f3e5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.779 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:49.780 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:49.780 DEBUG:boto:path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.781 DEBUG:boto:auth_path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.782 DEBUG:boto:Method: HEAD 2022-09-03T06:27:49.783 DEBUG:boto:Path: /dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.789 DEBUG:boto:Data: 2022-09-03T06:27:49.791 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.791 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:49.792 DEBUG:boto:Port: 8001 2022-09-03T06:27:49.793 DEBUG:boto:Params: {} 2022-09-03T06:27:49.794 DEBUG:boto:Token: None 2022-09-03T06:27:49.795 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.796 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Cv2Kuc59YhA3KcjVfknwXHXs+Io= 2022-09-03T06:27:49.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Cv2Kuc59YhA3KcjVfknwXHXs+Io=', 'Content-Length': '0'} 2022-09-03T06:27:49.800 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ba3d4a1332a2305c-006312f3e5-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.801 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:49.801 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:49.802 DEBUG:boto:path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.803 DEBUG:boto:auth_path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.803 DEBUG:boto:Method: HEAD 2022-09-03T06:27:49.804 DEBUG:boto:Path: /dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.805 DEBUG:boto:Data: 2022-09-03T06:27:49.805 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.806 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:49.807 DEBUG:boto:Port: 8000 2022-09-03T06:27:49.807 DEBUG:boto:Params: {} 2022-09-03T06:27:49.808 DEBUG:boto:Token: None 2022-09-03T06:27:49.809 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.809 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Cv2Kuc59YhA3KcjVfknwXHXs+Io= 2022-09-03T06:27:49.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Cv2Kuc59YhA3KcjVfknwXHXs+Io=', 'Content-Length': '0'} 2022-09-03T06:27:49.813 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cc1a55eb7997ba6b-006312f3e5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.815 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:27:49.815 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:49.816 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:49.816 DEBUG:boto:path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.817 DEBUG:boto:auth_path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.818 DEBUG:boto:Method: GET 2022-09-03T06:27:49.819 DEBUG:boto:Path: /dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.819 DEBUG:boto:Data: 2022-09-03T06:27:49.820 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.821 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:49.821 DEBUG:boto:Port: 8001 2022-09-03T06:27:49.822 DEBUG:boto:Params: {} 2022-09-03T06:27:49.822 DEBUG:boto:Token: None 2022-09-03T06:27:49.823 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.824 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iyjdvlx3/kiaSRVvtmJn8ZM+x4E= 2022-09-03T06:27:49.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iyjdvlx3/kiaSRVvtmJn8ZM+x4E=', 'Content-Length': '0'} 2022-09-03T06:27:49.827 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001e056f69f595d315-006312f3e5-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.828 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:49.829 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:49.830 DEBUG:boto:path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.830 DEBUG:boto:auth_path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.831 DEBUG:boto:Method: GET 2022-09-03T06:27:49.832 DEBUG:boto:Path: /dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.832 DEBUG:boto:Data: 2022-09-03T06:27:49.833 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.833 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:49.834 DEBUG:boto:Port: 8000 2022-09-03T06:27:49.834 DEBUG:boto:Params: {} 2022-09-03T06:27:49.835 DEBUG:boto:Token: None 2022-09-03T06:27:49.836 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/h?versionId=null 2022-09-03T06:27:49.837 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iyjdvlx3/kiaSRVvtmJn8ZM+x4E= 2022-09-03T06:27:49.838 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iyjdvlx3/kiaSRVvtmJn8ZM+x4E=', 'Content-Length': '0'} 2022-09-03T06:27:49.840 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d45353060399131-006312f3e5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.842 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:49.843 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:49.844 DEBUG:boto:path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:49.844 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:49.845 DEBUG:boto:Method: GET 2022-09-03T06:27:49.847 DEBUG:boto:Path: /dsgvcy-73/?prefix=h 2022-09-03T06:27:49.848 DEBUG:boto:Data: 2022-09-03T06:27:49.848 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.849 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:49.850 DEBUG:boto:Port: 8001 2022-09-03T06:27:49.851 DEBUG:boto:Params: {} 2022-09-03T06:27:49.852 DEBUG:boto:Token: None 2022-09-03T06:27:49.853 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/ 2022-09-03T06:27:49.854 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM= 2022-09-03T06:27:49.855 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM=', 'Content-Length': '0'} 2022-09-03T06:27:49.858 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005823c8d0ca1c606b-006312f3e5-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.860 DEBUG:boto:b'dsgvcy-73h1000falseh2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:49.861 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:49.862 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:49.863 DEBUG:boto:path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:49.864 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:49.865 DEBUG:boto:Method: GET 2022-09-03T06:27:49.866 DEBUG:boto:Path: /dsgvcy-73/?prefix=h 2022-09-03T06:27:49.866 DEBUG:boto:Data: 2022-09-03T06:27:49.867 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.868 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:49.869 DEBUG:boto:Port: 8000 2022-09-03T06:27:49.870 DEBUG:boto:Params: {} 2022-09-03T06:27:49.871 DEBUG:boto:Token: None 2022-09-03T06:27:49.872 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/ 2022-09-03T06:27:49.873 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM= 2022-09-03T06:27:49.874 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM=', 'Content-Length': '0'} 2022-09-03T06:27:49.880 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a56ae7240530f345-006312f3e5-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.882 DEBUG:boto:b'dsgvcy-73h1000falseh2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:49.883 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:49.884 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:49.885 DEBUG:boto:Method: HEAD 2022-09-03T06:27:49.885 DEBUG:boto:Path: /dsgvcy-73/h 2022-09-03T06:27:49.886 DEBUG:boto:Data: 2022-09-03T06:27:49.887 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.888 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:49.888 DEBUG:boto:Port: 8001 2022-09-03T06:27:49.889 DEBUG:boto:Params: {} 2022-09-03T06:27:49.890 DEBUG:boto:Token: None 2022-09-03T06:27:49.890 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/h 2022-09-03T06:27:49.891 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kCfjvwKRpgdP+1cKYdTjvRBHOEQ= 2022-09-03T06:27:49.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kCfjvwKRpgdP+1cKYdTjvRBHOEQ=', 'Content-Length': '0'} 2022-09-03T06:27:49.894 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000014d14e181f7731f0-006312f3e5-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.895 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:49.896 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:49.896 DEBUG:boto:Method: HEAD 2022-09-03T06:27:49.897 DEBUG:boto:Path: /dsgvcy-73/h 2022-09-03T06:27:49.898 DEBUG:boto:Data: 2022-09-03T06:27:49.898 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.900 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:49.901 DEBUG:boto:Port: 8000 2022-09-03T06:27:49.902 DEBUG:boto:Params: {} 2022-09-03T06:27:49.903 DEBUG:boto:Token: None 2022-09-03T06:27:49.904 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/h 2022-09-03T06:27:49.905 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kCfjvwKRpgdP+1cKYdTjvRBHOEQ= 2022-09-03T06:27:49.906 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kCfjvwKRpgdP+1cKYdTjvRBHOEQ=', 'Content-Length': '0'} 2022-09-03T06:27:49.909 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b87c9d1a186c81f0-006312f3e5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.911 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:27:49.912 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:49.913 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:49.914 DEBUG:boto:Method: GET 2022-09-03T06:27:49.915 DEBUG:boto:Path: /dsgvcy-73/h 2022-09-03T06:27:49.916 DEBUG:boto:Data: 2022-09-03T06:27:49.917 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.918 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:49.919 DEBUG:boto:Port: 8001 2022-09-03T06:27:49.920 DEBUG:boto:Params: {} 2022-09-03T06:27:49.921 DEBUG:boto:Token: None 2022-09-03T06:27:49.922 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/h 2022-09-03T06:27:49.924 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qwI8vVoUaf98IrbRNeUH8GFmhyw= 2022-09-03T06:27:49.924 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qwI8vVoUaf98IrbRNeUH8GFmhyw=', 'Content-Length': '0'} 2022-09-03T06:27:49.927 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cfe94d48fc0017da-006312f3e5-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.970 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:49.971 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:49.973 DEBUG:boto:Method: GET 2022-09-03T06:27:49.974 DEBUG:boto:Path: /dsgvcy-73/h 2022-09-03T06:27:49.975 DEBUG:boto:Data: 2022-09-03T06:27:49.976 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.977 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:49.978 DEBUG:boto:Port: 8000 2022-09-03T06:27:49.980 DEBUG:boto:Params: {} 2022-09-03T06:27:49.981 DEBUG:boto:Token: None 2022-09-03T06:27:49.982 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/h 2022-09-03T06:27:49.983 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qwI8vVoUaf98IrbRNeUH8GFmhyw= 2022-09-03T06:27:49.985 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qwI8vVoUaf98IrbRNeUH8GFmhyw=', 'Content-Length': '0'} 2022-09-03T06:27:49.988 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004e28d713fddebe8c-006312f3e5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:49.990 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:49.991 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:49.992 DEBUG:boto:path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:49.993 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:49.994 DEBUG:boto:Method: GET 2022-09-03T06:27:49.995 DEBUG:boto:Path: /dsgvcy-73/?prefix=h 2022-09-03T06:27:49.995 DEBUG:boto:Data: 2022-09-03T06:27:49.996 DEBUG:boto:Headers: {} 2022-09-03T06:27:49.997 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:49.998 DEBUG:boto:Port: 8001 2022-09-03T06:27:49.998 DEBUG:boto:Params: {} 2022-09-03T06:27:49.999 DEBUG:boto:Token: None 2022-09-03T06:27:50.000 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:49 GMT /dsgvcy-73/ 2022-09-03T06:27:50.000 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM= 2022-09-03T06:27:50.001 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GgnXueaBtjdx6yX2W8rYT53wcbM=', 'Content-Length': '0'} 2022-09-03T06:27:50.004 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006dd5c857a55268b2-006312f3e6-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.004 DEBUG:boto:b'dsgvcy-73h1000falseh2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:50.005 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:50.006 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:50.007 DEBUG:boto:path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:50.007 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:50.008 DEBUG:boto:Method: GET 2022-09-03T06:27:50.009 DEBUG:boto:Path: /dsgvcy-73/?prefix=h 2022-09-03T06:27:50.010 DEBUG:boto:Data: 2022-09-03T06:27:50.010 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.011 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:50.012 DEBUG:boto:Port: 8000 2022-09-03T06:27:50.012 DEBUG:boto:Params: {} 2022-09-03T06:27:50.013 DEBUG:boto:Token: None 2022-09-03T06:27:50.014 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/ 2022-09-03T06:27:50.014 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OP+fyIYGoEJDRujOhGnd8IF//bg= 2022-09-03T06:27:50.015 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OP+fyIYGoEJDRujOhGnd8IF//bg=', 'Content-Length': '0'} 2022-09-03T06:27:50.018 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c4a7c22330d96152-006312f3e6-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.019 DEBUG:boto:b'dsgvcy-73h1000falseh2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:50.020 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-73 zones={test-zone3, test-zone1} 2022-09-03T06:27:50.021 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-73 zones={test-zone3, test-zone2} 2022-09-03T06:27:50.022 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:50.022 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:50.023 DEBUG:boto:Method: HEAD 2022-09-03T06:27:50.024 DEBUG:boto:Path: /dsgvcy-73/ 2022-09-03T06:27:50.025 DEBUG:boto:Data: 2022-09-03T06:27:50.026 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.026 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:50.028 DEBUG:boto:Port: 8001 2022-09-03T06:27:50.029 DEBUG:boto:Params: {} 2022-09-03T06:27:50.029 DEBUG:boto:Token: None 2022-09-03T06:27:50.030 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/ 2022-09-03T06:27:50.031 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UhzQRLXCnYJ1jflEEGU3TT9fElM= 2022-09-03T06:27:50.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UhzQRLXCnYJ1jflEEGU3TT9fElM=', 'Content-Length': '0'} 2022-09-03T06:27:50.035 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ca59835fcc1d3826-006312f3e6-1851-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.036 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:50.036 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:50.037 DEBUG:boto:Method: HEAD 2022-09-03T06:27:50.038 DEBUG:boto:Path: /dsgvcy-73/ 2022-09-03T06:27:50.039 DEBUG:boto:Data: 2022-09-03T06:27:50.039 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.040 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:50.041 DEBUG:boto:Port: 8000 2022-09-03T06:27:50.042 DEBUG:boto:Params: {} 2022-09-03T06:27:50.043 DEBUG:boto:Token: None 2022-09-03T06:27:50.044 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/ 2022-09-03T06:27:50.045 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UhzQRLXCnYJ1jflEEGU3TT9fElM= 2022-09-03T06:27:50.046 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UhzQRLXCnYJ1jflEEGU3TT9fElM=', 'Content-Length': '0'} 2022-09-03T06:27:50.049 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ef23d16e7489dcdf-006312f3e6-188c-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.051 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:27:50.051 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:50.052 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:50.052 DEBUG:boto:path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:50.053 DEBUG:boto:auth_path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:50.054 DEBUG:boto:Method: GET 2022-09-03T06:27:50.055 DEBUG:boto:Path: /dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:50.056 DEBUG:boto:Data: 2022-09-03T06:27:50.057 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.058 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:50.059 DEBUG:boto:Port: 8001 2022-09-03T06:27:50.060 DEBUG:boto:Params: {} 2022-09-03T06:27:50.061 DEBUG:boto:Token: None 2022-09-03T06:27:50.062 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/?versions 2022-09-03T06:27:50.063 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zf8gOpgr+US4f9fsVkWaRJNykwY= 2022-09-03T06:27:50.065 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zf8gOpgr+US4f9fsVkWaRJNykwY=', 'Content-Length': '0'} 2022-09-03T06:27:50.068 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000868b373921827706-006312f3e6-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.069 DEBUG:boto:b'dsgvcy-73999falseanulltrue2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:50.071 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:27:50.071 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:27:50.072 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:27:50.073 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:27:50.074 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:27:50.075 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:27:50.075 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:27:50.076 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:27:50.077 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:27:50.078 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:50.079 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:50.080 DEBUG:boto:path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:50.080 DEBUG:boto:auth_path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:50.081 DEBUG:boto:Method: GET 2022-09-03T06:27:50.082 DEBUG:boto:Path: /dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:50.083 DEBUG:boto:Data: 2022-09-03T06:27:50.084 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.085 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:50.086 DEBUG:boto:Port: 8000 2022-09-03T06:27:50.087 DEBUG:boto:Params: {} 2022-09-03T06:27:50.088 DEBUG:boto:Token: None 2022-09-03T06:27:50.089 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/?versions 2022-09-03T06:27:50.090 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zf8gOpgr+US4f9fsVkWaRJNykwY= 2022-09-03T06:27:50.091 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zf8gOpgr+US4f9fsVkWaRJNykwY=', 'Content-Length': '0'} 2022-09-03T06:27:50.094 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000072a263393bb8d8f3-006312f3e6-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.102 DEBUG:boto:b'dsgvcy-73999falseanulltrue2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:50.103 DEBUG:tasks.rgw_multi.zone_rados:o=a 2022-09-03T06:27:50.104 DEBUG:tasks.rgw_multi.zone_rados:o=b 2022-09-03T06:27:50.105 DEBUG:tasks.rgw_multi.zone_rados:o=c 2022-09-03T06:27:50.106 DEBUG:tasks.rgw_multi.zone_rados:o=d 2022-09-03T06:27:50.106 DEBUG:tasks.rgw_multi.zone_rados:o=e 2022-09-03T06:27:50.107 DEBUG:tasks.rgw_multi.zone_rados:o=f 2022-09-03T06:27:50.107 DEBUG:tasks.rgw_multi.zone_rados:o=g 2022-09-03T06:27:50.108 DEBUG:tasks.rgw_multi.zone_rados:o=h 2022-09-03T06:27:50.108 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:50.109 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:50.110 DEBUG:boto:path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:50.111 DEBUG:boto:auth_path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:50.112 DEBUG:boto:Method: GET 2022-09-03T06:27:50.113 DEBUG:boto:Path: /dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:50.114 DEBUG:boto:Data: 2022-09-03T06:27:50.115 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.115 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:50.116 DEBUG:boto:Port: 8001 2022-09-03T06:27:50.117 DEBUG:boto:Params: {} 2022-09-03T06:27:50.118 DEBUG:boto:Token: None 2022-09-03T06:27:50.119 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/?versions 2022-09-03T06:27:50.120 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zf8gOpgr+US4f9fsVkWaRJNykwY= 2022-09-03T06:27:50.122 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zf8gOpgr+US4f9fsVkWaRJNykwY=', 'Content-Length': '0'} 2022-09-03T06:27:50.127 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000044cc8e460ff03f66-006312f3e6-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.128 DEBUG:boto:b'dsgvcy-73999falseanulltrue2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:50.129 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:50.130 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:50.131 DEBUG:boto:path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:50.132 DEBUG:boto:auth_path=/dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:50.133 DEBUG:boto:Method: GET 2022-09-03T06:27:50.133 DEBUG:boto:Path: /dsgvcy-73/?versions&max-keys=999 2022-09-03T06:27:50.134 DEBUG:boto:Data: 2022-09-03T06:27:50.134 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.135 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:50.136 DEBUG:boto:Port: 8000 2022-09-03T06:27:50.136 DEBUG:boto:Params: {} 2022-09-03T06:27:50.137 DEBUG:boto:Token: None 2022-09-03T06:27:50.138 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/?versions 2022-09-03T06:27:50.138 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zf8gOpgr+US4f9fsVkWaRJNykwY= 2022-09-03T06:27:50.139 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zf8gOpgr+US4f9fsVkWaRJNykwY=', 'Content-Length': '0'} 2022-09-03T06:27:50.142 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001789589d960c8fd1-006312f3e6-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.143 DEBUG:boto:b'dsgvcy-73999falseanulltrue2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalbnulltrue2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalcnulltrue2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormaldnulltrue2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalenulltrue2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalfnulltrue2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalgnulltrue2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalhnulltrue2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:50.145 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:27:50.146 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:50.147 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:50.148 DEBUG:boto:path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.149 DEBUG:boto:auth_path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.150 DEBUG:boto:Method: GET 2022-09-03T06:27:50.151 DEBUG:boto:Path: /dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.152 DEBUG:boto:Data: 2022-09-03T06:27:50.153 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.154 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:50.155 DEBUG:boto:Port: 8001 2022-09-03T06:27:50.156 DEBUG:boto:Params: {} 2022-09-03T06:27:50.157 DEBUG:boto:Token: None 2022-09-03T06:27:50.170 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.172 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yYp+vQQlrjsZOTR/g+OW+12nmkg= 2022-09-03T06:27:50.173 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yYp+vQQlrjsZOTR/g+OW+12nmkg=', 'Content-Length': '0'} 2022-09-03T06:27:50.176 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000607f503d0ade13db-006312f3e6-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.178 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:50.179 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:50.180 DEBUG:boto:path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.181 DEBUG:boto:auth_path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.182 DEBUG:boto:Method: GET 2022-09-03T06:27:50.183 DEBUG:boto:Path: /dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.184 DEBUG:boto:Data: 2022-09-03T06:27:50.185 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.186 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:50.187 DEBUG:boto:Port: 8000 2022-09-03T06:27:50.188 DEBUG:boto:Params: {} 2022-09-03T06:27:50.189 DEBUG:boto:Token: None 2022-09-03T06:27:50.190 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.191 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yYp+vQQlrjsZOTR/g+OW+12nmkg= 2022-09-03T06:27:50.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yYp+vQQlrjsZOTR/g+OW+12nmkg=', 'Content-Length': '0'} 2022-09-03T06:27:50.195 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000032cce14a0682b4cb-006312f3e6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.197 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:50.198 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:50.199 DEBUG:boto:path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.200 DEBUG:boto:auth_path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.201 DEBUG:boto:Method: HEAD 2022-09-03T06:27:50.202 DEBUG:boto:Path: /dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.202 DEBUG:boto:Data: 2022-09-03T06:27:50.203 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.204 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:50.205 DEBUG:boto:Port: 8001 2022-09-03T06:27:50.206 DEBUG:boto:Params: {} 2022-09-03T06:27:50.207 DEBUG:boto:Token: None 2022-09-03T06:27:50.208 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.209 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gNPXbeRWHISfN8mYvLvxBDSyXFM= 2022-09-03T06:27:50.211 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gNPXbeRWHISfN8mYvLvxBDSyXFM=', 'Content-Length': '0'} 2022-09-03T06:27:50.216 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fa9a65b072ae89b5-006312f3e6-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.218 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:50.219 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:50.221 DEBUG:boto:path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.222 DEBUG:boto:auth_path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.223 DEBUG:boto:Method: HEAD 2022-09-03T06:27:50.224 DEBUG:boto:Path: /dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.225 DEBUG:boto:Data: 2022-09-03T06:27:50.226 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.227 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:50.228 DEBUG:boto:Port: 8000 2022-09-03T06:27:50.229 DEBUG:boto:Params: {} 2022-09-03T06:27:50.230 DEBUG:boto:Token: None 2022-09-03T06:27:50.231 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.231 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gNPXbeRWHISfN8mYvLvxBDSyXFM= 2022-09-03T06:27:50.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gNPXbeRWHISfN8mYvLvxBDSyXFM=', 'Content-Length': '0'} 2022-09-03T06:27:50.235 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d27be3b93d5b2883-006312f3e6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.235 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:27:50.236 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:50.237 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:50.238 DEBUG:boto:path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.238 DEBUG:boto:auth_path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.239 DEBUG:boto:Method: GET 2022-09-03T06:27:50.240 DEBUG:boto:Path: /dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.240 DEBUG:boto:Data: 2022-09-03T06:27:50.241 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.241 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:50.242 DEBUG:boto:Port: 8001 2022-09-03T06:27:50.243 DEBUG:boto:Params: {} 2022-09-03T06:27:50.243 DEBUG:boto:Token: None 2022-09-03T06:27:50.244 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.245 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yYp+vQQlrjsZOTR/g+OW+12nmkg= 2022-09-03T06:27:50.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yYp+vQQlrjsZOTR/g+OW+12nmkg=', 'Content-Length': '0'} 2022-09-03T06:27:50.248 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a0dc3a743194f3b8-006312f3e6-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.249 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:50.250 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:50.251 DEBUG:boto:path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.251 DEBUG:boto:auth_path=/dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.252 DEBUG:boto:Method: GET 2022-09-03T06:27:50.253 DEBUG:boto:Path: /dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.253 DEBUG:boto:Data: 2022-09-03T06:27:50.254 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.254 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:50.255 DEBUG:boto:Port: 8000 2022-09-03T06:27:50.255 DEBUG:boto:Params: {} 2022-09-03T06:27:50.257 DEBUG:boto:Token: None 2022-09-03T06:27:50.258 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/a?versionId=null 2022-09-03T06:27:50.259 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yYp+vQQlrjsZOTR/g+OW+12nmkg= 2022-09-03T06:27:50.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yYp+vQQlrjsZOTR/g+OW+12nmkg=', 'Content-Length': '0'} 2022-09-03T06:27:50.263 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000978ee103ec747e28-006312f3e6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.310 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:50.312 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:50.313 DEBUG:boto:path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:50.314 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:50.314 DEBUG:boto:Method: GET 2022-09-03T06:27:50.315 DEBUG:boto:Path: /dsgvcy-73/?prefix=a 2022-09-03T06:27:50.316 DEBUG:boto:Data: 2022-09-03T06:27:50.316 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.317 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:50.317 DEBUG:boto:Port: 8001 2022-09-03T06:27:50.318 DEBUG:boto:Params: {} 2022-09-03T06:27:50.319 DEBUG:boto:Token: None 2022-09-03T06:27:50.319 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/ 2022-09-03T06:27:50.320 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OP+fyIYGoEJDRujOhGnd8IF//bg= 2022-09-03T06:27:50.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OP+fyIYGoEJDRujOhGnd8IF//bg=', 'Content-Length': '0'} 2022-09-03T06:27:50.327 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009de209dfce66e83d-006312f3e6-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.329 DEBUG:boto:b'dsgvcy-73a1000falsea2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:50.331 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:50.333 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:50.334 DEBUG:boto:path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:50.334 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:50.335 DEBUG:boto:Method: GET 2022-09-03T06:27:50.336 DEBUG:boto:Path: /dsgvcy-73/?prefix=a 2022-09-03T06:27:50.336 DEBUG:boto:Data: 2022-09-03T06:27:50.337 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.337 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:50.338 DEBUG:boto:Port: 8000 2022-09-03T06:27:50.339 DEBUG:boto:Params: {} 2022-09-03T06:27:50.340 DEBUG:boto:Token: None 2022-09-03T06:27:50.341 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/ 2022-09-03T06:27:50.342 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OP+fyIYGoEJDRujOhGnd8IF//bg= 2022-09-03T06:27:50.343 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OP+fyIYGoEJDRujOhGnd8IF//bg=', 'Content-Length': '0'} 2022-09-03T06:27:50.347 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a3eabfe482101ae0-006312f3e6-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.348 DEBUG:boto:b'dsgvcy-73a1000falsea2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:50.349 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:50.350 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:50.351 DEBUG:boto:Method: HEAD 2022-09-03T06:27:50.352 DEBUG:boto:Path: /dsgvcy-73/a 2022-09-03T06:27:50.353 DEBUG:boto:Data: 2022-09-03T06:27:50.353 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.354 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:50.355 DEBUG:boto:Port: 8001 2022-09-03T06:27:50.356 DEBUG:boto:Params: {} 2022-09-03T06:27:50.357 DEBUG:boto:Token: None 2022-09-03T06:27:50.358 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/a 2022-09-03T06:27:50.359 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C4iHKj4rv360wrzCHCJpZyP608Y= 2022-09-03T06:27:50.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C4iHKj4rv360wrzCHCJpZyP608Y=', 'Content-Length': '0'} 2022-09-03T06:27:50.362 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000088def717a8e6cb3a-006312f3e6-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.363 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:50.364 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:50.364 DEBUG:boto:Method: HEAD 2022-09-03T06:27:50.365 DEBUG:boto:Path: /dsgvcy-73/a 2022-09-03T06:27:50.366 DEBUG:boto:Data: 2022-09-03T06:27:50.367 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.367 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:50.368 DEBUG:boto:Port: 8000 2022-09-03T06:27:50.369 DEBUG:boto:Params: {} 2022-09-03T06:27:50.370 DEBUG:boto:Token: None 2022-09-03T06:27:50.371 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/a 2022-09-03T06:27:50.372 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C4iHKj4rv360wrzCHCJpZyP608Y= 2022-09-03T06:27:50.373 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C4iHKj4rv360wrzCHCJpZyP608Y=', 'Content-Length': '0'} 2022-09-03T06:27:50.377 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008bf8adb4b6c7eeca-006312f3e6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.378 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=a 2022-09-03T06:27:50.379 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:50.380 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:50.381 DEBUG:boto:Method: GET 2022-09-03T06:27:50.381 DEBUG:boto:Path: /dsgvcy-73/a 2022-09-03T06:27:50.382 DEBUG:boto:Data: 2022-09-03T06:27:50.382 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.383 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:50.383 DEBUG:boto:Port: 8001 2022-09-03T06:27:50.384 DEBUG:boto:Params: {} 2022-09-03T06:27:50.384 DEBUG:boto:Token: None 2022-09-03T06:27:50.385 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/a 2022-09-03T06:27:50.386 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:g3l8p+ae/z8XThu9dHqMPxbUmmk= 2022-09-03T06:27:50.386 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:g3l8p+ae/z8XThu9dHqMPxbUmmk=', 'Content-Length': '0'} 2022-09-03T06:27:50.389 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1686b36170e525c-006312f3e6-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.434 DEBUG:boto:path=/dsgvcy-73/a 2022-09-03T06:27:50.435 DEBUG:boto:auth_path=/dsgvcy-73/a 2022-09-03T06:27:50.435 DEBUG:boto:Method: GET 2022-09-03T06:27:50.436 DEBUG:boto:Path: /dsgvcy-73/a 2022-09-03T06:27:50.437 DEBUG:boto:Data: 2022-09-03T06:27:50.437 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.438 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:50.439 DEBUG:boto:Port: 8000 2022-09-03T06:27:50.439 DEBUG:boto:Params: {} 2022-09-03T06:27:50.440 DEBUG:boto:Token: None 2022-09-03T06:27:50.440 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/a 2022-09-03T06:27:50.441 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:g3l8p+ae/z8XThu9dHqMPxbUmmk= 2022-09-03T06:27:50.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:g3l8p+ae/z8XThu9dHqMPxbUmmk=', 'Content-Length': '0'} 2022-09-03T06:27:50.444 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f040d6602074e052-006312f3e6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.445 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:50.447 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:50.447 DEBUG:boto:path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:50.449 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:50.450 DEBUG:boto:Method: GET 2022-09-03T06:27:50.451 DEBUG:boto:Path: /dsgvcy-73/?prefix=a 2022-09-03T06:27:50.452 DEBUG:boto:Data: 2022-09-03T06:27:50.453 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.454 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:50.455 DEBUG:boto:Port: 8001 2022-09-03T06:27:50.456 DEBUG:boto:Params: {} 2022-09-03T06:27:50.457 DEBUG:boto:Token: None 2022-09-03T06:27:50.458 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/ 2022-09-03T06:27:50.459 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OP+fyIYGoEJDRujOhGnd8IF//bg= 2022-09-03T06:27:50.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OP+fyIYGoEJDRujOhGnd8IF//bg=', 'Content-Length': '0'} 2022-09-03T06:27:50.463 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d9c7e7553f2d0f17-006312f3e6-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.464 DEBUG:boto:b'dsgvcy-73a1000falsea2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:50.465 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:50.466 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:50.467 DEBUG:boto:path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:50.468 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=a 2022-09-03T06:27:50.469 DEBUG:boto:Method: GET 2022-09-03T06:27:50.470 DEBUG:boto:Path: /dsgvcy-73/?prefix=a 2022-09-03T06:27:50.470 DEBUG:boto:Data: 2022-09-03T06:27:50.471 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.472 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:50.473 DEBUG:boto:Port: 8000 2022-09-03T06:27:50.473 DEBUG:boto:Params: {} 2022-09-03T06:27:50.474 DEBUG:boto:Token: None 2022-09-03T06:27:50.475 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/ 2022-09-03T06:27:50.475 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OP+fyIYGoEJDRujOhGnd8IF//bg= 2022-09-03T06:27:50.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OP+fyIYGoEJDRujOhGnd8IF//bg=', 'Content-Length': '0'} 2022-09-03T06:27:50.484 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b6c797b7f857cbdc-006312f3e6-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.486 DEBUG:boto:b'dsgvcy-73a1000falsea2022-09-03T06:24:09.187Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:50.488 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:27:50.489 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:50.490 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:50.491 DEBUG:boto:path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.492 DEBUG:boto:auth_path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.494 DEBUG:boto:Method: GET 2022-09-03T06:27:50.495 DEBUG:boto:Path: /dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.496 DEBUG:boto:Data: 2022-09-03T06:27:50.497 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.497 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:50.498 DEBUG:boto:Port: 8001 2022-09-03T06:27:50.499 DEBUG:boto:Params: {} 2022-09-03T06:27:50.499 DEBUG:boto:Token: None 2022-09-03T06:27:50.500 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.501 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QLesdiJezs5Dmi74tnGn+FJjtzc= 2022-09-03T06:27:50.502 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QLesdiJezs5Dmi74tnGn+FJjtzc=', 'Content-Length': '0'} 2022-09-03T06:27:50.505 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006a59abd3a0da9af7-006312f3e6-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.506 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:50.507 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:50.507 DEBUG:boto:path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.508 DEBUG:boto:auth_path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.509 DEBUG:boto:Method: GET 2022-09-03T06:27:50.509 DEBUG:boto:Path: /dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.510 DEBUG:boto:Data: 2022-09-03T06:27:50.511 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.512 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:50.512 DEBUG:boto:Port: 8000 2022-09-03T06:27:50.513 DEBUG:boto:Params: {} 2022-09-03T06:27:50.514 DEBUG:boto:Token: None 2022-09-03T06:27:50.514 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.515 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QLesdiJezs5Dmi74tnGn+FJjtzc= 2022-09-03T06:27:50.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QLesdiJezs5Dmi74tnGn+FJjtzc=', 'Content-Length': '0'} 2022-09-03T06:27:50.519 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cbb23436123e1b09-006312f3e6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.562 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:50.563 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:50.564 DEBUG:boto:path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.564 DEBUG:boto:auth_path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.565 DEBUG:boto:Method: HEAD 2022-09-03T06:27:50.566 DEBUG:boto:Path: /dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.566 DEBUG:boto:Data: 2022-09-03T06:27:50.567 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.568 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:50.569 DEBUG:boto:Port: 8001 2022-09-03T06:27:50.569 DEBUG:boto:Params: {} 2022-09-03T06:27:50.570 DEBUG:boto:Token: None 2022-09-03T06:27:50.571 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.571 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7Nesnqf3OmGiYy/+2oVupR3U0gA= 2022-09-03T06:27:50.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7Nesnqf3OmGiYy/+2oVupR3U0gA=', 'Content-Length': '0'} 2022-09-03T06:27:50.574 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004e9805b482de5ae3-006312f3e6-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.575 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:50.576 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:50.577 DEBUG:boto:path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.577 DEBUG:boto:auth_path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.578 DEBUG:boto:Method: HEAD 2022-09-03T06:27:50.578 DEBUG:boto:Path: /dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.579 DEBUG:boto:Data: 2022-09-03T06:27:50.580 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.581 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:50.582 DEBUG:boto:Port: 8000 2022-09-03T06:27:50.582 DEBUG:boto:Params: {} 2022-09-03T06:27:50.583 DEBUG:boto:Token: None 2022-09-03T06:27:50.584 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.585 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7Nesnqf3OmGiYy/+2oVupR3U0gA= 2022-09-03T06:27:50.586 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7Nesnqf3OmGiYy/+2oVupR3U0gA=', 'Content-Length': '0'} 2022-09-03T06:27:50.599 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074ead8fef8eef6e1-006312f3e6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.601 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:27:50.602 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:50.603 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:50.604 DEBUG:boto:path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.605 DEBUG:boto:auth_path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.606 DEBUG:boto:Method: GET 2022-09-03T06:27:50.607 DEBUG:boto:Path: /dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.608 DEBUG:boto:Data: 2022-09-03T06:27:50.609 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.611 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:50.612 DEBUG:boto:Port: 8001 2022-09-03T06:27:50.613 DEBUG:boto:Params: {} 2022-09-03T06:27:50.614 DEBUG:boto:Token: None 2022-09-03T06:27:50.615 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.616 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QLesdiJezs5Dmi74tnGn+FJjtzc= 2022-09-03T06:27:50.617 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QLesdiJezs5Dmi74tnGn+FJjtzc=', 'Content-Length': '0'} 2022-09-03T06:27:50.621 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f4d774fe072998b3-006312f3e6-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.663 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:50.664 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:50.664 DEBUG:boto:path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.665 DEBUG:boto:auth_path=/dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.665 DEBUG:boto:Method: GET 2022-09-03T06:27:50.666 DEBUG:boto:Path: /dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.666 DEBUG:boto:Data: 2022-09-03T06:27:50.667 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.667 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:50.668 DEBUG:boto:Port: 8000 2022-09-03T06:27:50.668 DEBUG:boto:Params: {} 2022-09-03T06:27:50.669 DEBUG:boto:Token: None 2022-09-03T06:27:50.670 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/b?versionId=null 2022-09-03T06:27:50.670 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QLesdiJezs5Dmi74tnGn+FJjtzc= 2022-09-03T06:27:50.671 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QLesdiJezs5Dmi74tnGn+FJjtzc=', 'Content-Length': '0'} 2022-09-03T06:27:50.674 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a8ded8fe59776dd3-006312f3e6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.679 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:50.681 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:50.682 DEBUG:boto:path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:50.683 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:50.684 DEBUG:boto:Method: GET 2022-09-03T06:27:50.684 DEBUG:boto:Path: /dsgvcy-73/?prefix=b 2022-09-03T06:27:50.685 DEBUG:boto:Data: 2022-09-03T06:27:50.686 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.687 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:50.688 DEBUG:boto:Port: 8001 2022-09-03T06:27:50.689 DEBUG:boto:Params: {} 2022-09-03T06:27:50.690 DEBUG:boto:Token: None 2022-09-03T06:27:50.691 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/ 2022-09-03T06:27:50.691 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OP+fyIYGoEJDRujOhGnd8IF//bg= 2022-09-03T06:27:50.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OP+fyIYGoEJDRujOhGnd8IF//bg=', 'Content-Length': '0'} 2022-09-03T06:27:50.695 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000a27f563621f2554-006312f3e6-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.696 DEBUG:boto:b'dsgvcy-73b1000falseb2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:50.697 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:50.698 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:50.698 DEBUG:boto:path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:50.699 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:50.700 DEBUG:boto:Method: GET 2022-09-03T06:27:50.701 DEBUG:boto:Path: /dsgvcy-73/?prefix=b 2022-09-03T06:27:50.702 DEBUG:boto:Data: 2022-09-03T06:27:50.702 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.703 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:50.704 DEBUG:boto:Port: 8000 2022-09-03T06:27:50.705 DEBUG:boto:Params: {} 2022-09-03T06:27:50.706 DEBUG:boto:Token: None 2022-09-03T06:27:50.706 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/ 2022-09-03T06:27:50.707 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OP+fyIYGoEJDRujOhGnd8IF//bg= 2022-09-03T06:27:50.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OP+fyIYGoEJDRujOhGnd8IF//bg=', 'Content-Length': '0'} 2022-09-03T06:27:50.712 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000014b2acc56a0f5878-006312f3e6-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.713 DEBUG:boto:b'dsgvcy-73b1000falseb2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:50.715 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:50.716 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:50.717 DEBUG:boto:Method: HEAD 2022-09-03T06:27:50.718 DEBUG:boto:Path: /dsgvcy-73/b 2022-09-03T06:27:50.719 DEBUG:boto:Data: 2022-09-03T06:27:50.720 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.721 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:50.721 DEBUG:boto:Port: 8001 2022-09-03T06:27:50.722 DEBUG:boto:Params: {} 2022-09-03T06:27:50.723 DEBUG:boto:Token: None 2022-09-03T06:27:50.724 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/b 2022-09-03T06:27:50.724 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mK09sMWAGZtTZS+WrCvwpyinViw= 2022-09-03T06:27:50.725 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mK09sMWAGZtTZS+WrCvwpyinViw=', 'Content-Length': '0'} 2022-09-03T06:27:50.728 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000688706314f34909f-006312f3e6-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.730 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:50.730 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:50.731 DEBUG:boto:Method: HEAD 2022-09-03T06:27:50.732 DEBUG:boto:Path: /dsgvcy-73/b 2022-09-03T06:27:50.733 DEBUG:boto:Data: 2022-09-03T06:27:50.734 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.735 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:50.736 DEBUG:boto:Port: 8000 2022-09-03T06:27:50.737 DEBUG:boto:Params: {} 2022-09-03T06:27:50.739 DEBUG:boto:Token: None 2022-09-03T06:27:50.739 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/b 2022-09-03T06:27:50.740 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mK09sMWAGZtTZS+WrCvwpyinViw= 2022-09-03T06:27:50.741 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mK09sMWAGZtTZS+WrCvwpyinViw=', 'Content-Length': '0'} 2022-09-03T06:27:50.743 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000702d3f89149933e8-006312f3e6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.744 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=b 2022-09-03T06:27:50.744 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:50.745 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:50.746 DEBUG:boto:Method: GET 2022-09-03T06:27:50.747 DEBUG:boto:Path: /dsgvcy-73/b 2022-09-03T06:27:50.748 DEBUG:boto:Data: 2022-09-03T06:27:50.749 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.750 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:50.751 DEBUG:boto:Port: 8001 2022-09-03T06:27:50.752 DEBUG:boto:Params: {} 2022-09-03T06:27:50.753 DEBUG:boto:Token: None 2022-09-03T06:27:50.754 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/b 2022-09-03T06:27:50.754 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7x0CPHz4l8VTo9EGhLahOirhSEQ= 2022-09-03T06:27:50.755 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7x0CPHz4l8VTo9EGhLahOirhSEQ=', 'Content-Length': '0'} 2022-09-03T06:27:50.757 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db581f13ff63a942-006312f3e6-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.758 DEBUG:boto:path=/dsgvcy-73/b 2022-09-03T06:27:50.759 DEBUG:boto:auth_path=/dsgvcy-73/b 2022-09-03T06:27:50.759 DEBUG:boto:Method: GET 2022-09-03T06:27:50.760 DEBUG:boto:Path: /dsgvcy-73/b 2022-09-03T06:27:50.761 DEBUG:boto:Data: 2022-09-03T06:27:50.762 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.763 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:50.764 DEBUG:boto:Port: 8000 2022-09-03T06:27:50.765 DEBUG:boto:Params: {} 2022-09-03T06:27:50.766 DEBUG:boto:Token: None 2022-09-03T06:27:50.766 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/b 2022-09-03T06:27:50.767 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7x0CPHz4l8VTo9EGhLahOirhSEQ= 2022-09-03T06:27:50.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7x0CPHz4l8VTo9EGhLahOirhSEQ=', 'Content-Length': '0'} 2022-09-03T06:27:50.774 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000002b3c0ab438d76d9-006312f3e6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.818 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:50.819 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:50.820 DEBUG:boto:path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:50.820 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:50.821 DEBUG:boto:Method: GET 2022-09-03T06:27:50.821 DEBUG:boto:Path: /dsgvcy-73/?prefix=b 2022-09-03T06:27:50.822 DEBUG:boto:Data: 2022-09-03T06:27:50.822 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.823 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:50.823 DEBUG:boto:Port: 8001 2022-09-03T06:27:50.824 DEBUG:boto:Params: {} 2022-09-03T06:27:50.825 DEBUG:boto:Token: None 2022-09-03T06:27:50.825 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/ 2022-09-03T06:27:50.826 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OP+fyIYGoEJDRujOhGnd8IF//bg= 2022-09-03T06:27:50.827 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OP+fyIYGoEJDRujOhGnd8IF//bg=', 'Content-Length': '0'} 2022-09-03T06:27:50.830 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007dba57b4b2c0c287-006312f3e6-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.831 DEBUG:boto:b'dsgvcy-73b1000falseb2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:50.832 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:50.832 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:50.833 DEBUG:boto:path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:50.833 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=b 2022-09-03T06:27:50.834 DEBUG:boto:Method: GET 2022-09-03T06:27:50.835 DEBUG:boto:Path: /dsgvcy-73/?prefix=b 2022-09-03T06:27:50.835 DEBUG:boto:Data: 2022-09-03T06:27:50.836 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.836 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:50.837 DEBUG:boto:Port: 8000 2022-09-03T06:27:50.837 DEBUG:boto:Params: {} 2022-09-03T06:27:50.838 DEBUG:boto:Token: None 2022-09-03T06:27:50.838 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/ 2022-09-03T06:27:50.839 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OP+fyIYGoEJDRujOhGnd8IF//bg= 2022-09-03T06:27:50.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OP+fyIYGoEJDRujOhGnd8IF//bg=', 'Content-Length': '0'} 2022-09-03T06:27:50.842 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001b6fc9a4227fd4d8-006312f3e6-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.843 DEBUG:boto:b'dsgvcy-73b1000falseb2022-09-03T06:24:09.212Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:50.844 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:27:50.846 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:50.846 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:50.847 DEBUG:boto:path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.847 DEBUG:boto:auth_path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.849 DEBUG:boto:Method: GET 2022-09-03T06:27:50.849 DEBUG:boto:Path: /dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.850 DEBUG:boto:Data: 2022-09-03T06:27:50.850 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.851 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:50.852 DEBUG:boto:Port: 8001 2022-09-03T06:27:50.852 DEBUG:boto:Params: {} 2022-09-03T06:27:50.853 DEBUG:boto:Token: None 2022-09-03T06:27:50.854 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.855 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tIi8ddJx74GVHSEQNNyzp4LzQ2o= 2022-09-03T06:27:50.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tIi8ddJx74GVHSEQNNyzp4LzQ2o=', 'Content-Length': '0'} 2022-09-03T06:27:50.859 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f8a7688f8afc732-006312f3e6-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.906 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:50.908 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:50.909 DEBUG:boto:path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.910 DEBUG:boto:auth_path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.911 DEBUG:boto:Method: GET 2022-09-03T06:27:50.911 DEBUG:boto:Path: /dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.912 DEBUG:boto:Data: 2022-09-03T06:27:50.913 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.915 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:50.916 DEBUG:boto:Port: 8000 2022-09-03T06:27:50.917 DEBUG:boto:Params: {} 2022-09-03T06:27:50.919 DEBUG:boto:Token: None 2022-09-03T06:27:50.920 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.921 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tIi8ddJx74GVHSEQNNyzp4LzQ2o= 2022-09-03T06:27:50.922 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tIi8ddJx74GVHSEQNNyzp4LzQ2o=', 'Content-Length': '0'} 2022-09-03T06:27:50.925 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e353fc7b307da7de-006312f3e6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.926 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:50.927 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:50.927 DEBUG:boto:path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.928 DEBUG:boto:auth_path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.928 DEBUG:boto:Method: HEAD 2022-09-03T06:27:50.929 DEBUG:boto:Path: /dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.930 DEBUG:boto:Data: 2022-09-03T06:27:50.931 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.932 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:50.933 DEBUG:boto:Port: 8001 2022-09-03T06:27:50.933 DEBUG:boto:Params: {} 2022-09-03T06:27:50.934 DEBUG:boto:Token: None 2022-09-03T06:27:50.935 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.936 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9yIgZ3VFYg+3E4Uuh4M6zDUoJCA= 2022-09-03T06:27:50.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9yIgZ3VFYg+3E4Uuh4M6zDUoJCA=', 'Content-Length': '0'} 2022-09-03T06:27:50.940 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000081a9eb306e79dc6a-006312f3e6-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.941 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:50.942 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:50.942 DEBUG:boto:path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.943 DEBUG:boto:auth_path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.944 DEBUG:boto:Method: HEAD 2022-09-03T06:27:50.944 DEBUG:boto:Path: /dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.945 DEBUG:boto:Data: 2022-09-03T06:27:50.946 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.947 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:50.948 DEBUG:boto:Port: 8000 2022-09-03T06:27:50.948 DEBUG:boto:Params: {} 2022-09-03T06:27:50.949 DEBUG:boto:Token: None 2022-09-03T06:27:50.950 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.951 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9yIgZ3VFYg+3E4Uuh4M6zDUoJCA= 2022-09-03T06:27:50.952 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9yIgZ3VFYg+3E4Uuh4M6zDUoJCA=', 'Content-Length': '0'} 2022-09-03T06:27:50.961 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000630a1463c95a7d17-006312f3e6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.962 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:27:50.963 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:50.964 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:50.964 DEBUG:boto:path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.965 DEBUG:boto:auth_path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.966 DEBUG:boto:Method: GET 2022-09-03T06:27:50.967 DEBUG:boto:Path: /dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.968 DEBUG:boto:Data: 2022-09-03T06:27:50.969 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.970 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:50.971 DEBUG:boto:Port: 8001 2022-09-03T06:27:50.972 DEBUG:boto:Params: {} 2022-09-03T06:27:50.973 DEBUG:boto:Token: None 2022-09-03T06:27:50.974 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.975 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tIi8ddJx74GVHSEQNNyzp4LzQ2o= 2022-09-03T06:27:50.977 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tIi8ddJx74GVHSEQNNyzp4LzQ2o=', 'Content-Length': '0'} 2022-09-03T06:27:50.980 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005427a3511c090554-006312f3e6-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.981 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:50.982 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:50.983 DEBUG:boto:path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.984 DEBUG:boto:auth_path=/dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.984 DEBUG:boto:Method: GET 2022-09-03T06:27:50.985 DEBUG:boto:Path: /dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.985 DEBUG:boto:Data: 2022-09-03T06:27:50.986 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.986 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:50.987 DEBUG:boto:Port: 8000 2022-09-03T06:27:50.988 DEBUG:boto:Params: {} 2022-09-03T06:27:50.988 DEBUG:boto:Token: None 2022-09-03T06:27:50.989 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:50 GMT /dsgvcy-73/c?versionId=null 2022-09-03T06:27:50.989 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tIi8ddJx74GVHSEQNNyzp4LzQ2o= 2022-09-03T06:27:50.990 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tIi8ddJx74GVHSEQNNyzp4LzQ2o=', 'Content-Length': '0'} 2022-09-03T06:27:50.992 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002873c3516e69d6d9-006312f3e6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:50.993 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:50.994 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:50.994 DEBUG:boto:path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:50.995 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:50.995 DEBUG:boto:Method: GET 2022-09-03T06:27:50.996 DEBUG:boto:Path: /dsgvcy-73/?prefix=c 2022-09-03T06:27:50.996 DEBUG:boto:Data: 2022-09-03T06:27:50.997 DEBUG:boto:Headers: {} 2022-09-03T06:27:50.997 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:50.998 DEBUG:boto:Port: 8001 2022-09-03T06:27:50.998 DEBUG:boto:Params: {} 2022-09-03T06:27:50.999 DEBUG:boto:Token: None 2022-09-03T06:27:51.000 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/ 2022-09-03T06:27:51.000 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A= 2022-09-03T06:27:51.001 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A=', 'Content-Length': '0'} 2022-09-03T06:27:51.004 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004abdb182acac6a7b-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.004 DEBUG:boto:b'dsgvcy-73c1000falsec2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:51.005 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:51.006 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:51.007 DEBUG:boto:path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:51.007 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:51.008 DEBUG:boto:Method: GET 2022-09-03T06:27:51.009 DEBUG:boto:Path: /dsgvcy-73/?prefix=c 2022-09-03T06:27:51.009 DEBUG:boto:Data: 2022-09-03T06:27:51.010 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.010 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.011 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.012 DEBUG:boto:Params: {} 2022-09-03T06:27:51.012 DEBUG:boto:Token: None 2022-09-03T06:27:51.013 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/ 2022-09-03T06:27:51.013 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A= 2022-09-03T06:27:51.014 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A=', 'Content-Length': '0'} 2022-09-03T06:27:51.017 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d262bd00d7458d94-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.017 DEBUG:boto:b'dsgvcy-73c1000falsec2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:51.018 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:51.019 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:51.019 DEBUG:boto:Method: HEAD 2022-09-03T06:27:51.020 DEBUG:boto:Path: /dsgvcy-73/c 2022-09-03T06:27:51.020 DEBUG:boto:Data: 2022-09-03T06:27:51.021 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.021 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:51.022 DEBUG:boto:Port: 8001 2022-09-03T06:27:51.022 DEBUG:boto:Params: {} 2022-09-03T06:27:51.023 DEBUG:boto:Token: None 2022-09-03T06:27:51.023 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/c 2022-09-03T06:27:51.024 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3EEi04uhSYLONNnRsBPEiWHbzeU= 2022-09-03T06:27:51.025 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3EEi04uhSYLONNnRsBPEiWHbzeU=', 'Content-Length': '0'} 2022-09-03T06:27:51.027 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026dca05421d1ff70-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.028 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:51.029 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:51.029 DEBUG:boto:Method: HEAD 2022-09-03T06:27:51.030 DEBUG:boto:Path: /dsgvcy-73/c 2022-09-03T06:27:51.030 DEBUG:boto:Data: 2022-09-03T06:27:51.031 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.032 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.033 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.034 DEBUG:boto:Params: {} 2022-09-03T06:27:51.035 DEBUG:boto:Token: None 2022-09-03T06:27:51.036 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/c 2022-09-03T06:27:51.037 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3EEi04uhSYLONNnRsBPEiWHbzeU= 2022-09-03T06:27:51.038 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3EEi04uhSYLONNnRsBPEiWHbzeU=', 'Content-Length': '0'} 2022-09-03T06:27:51.046 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000083ab56cd3cf450db-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.047 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=c 2022-09-03T06:27:51.048 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:51.048 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:51.049 DEBUG:boto:Method: GET 2022-09-03T06:27:51.049 DEBUG:boto:Path: /dsgvcy-73/c 2022-09-03T06:27:51.050 DEBUG:boto:Data: 2022-09-03T06:27:51.050 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.051 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:51.052 DEBUG:boto:Port: 8001 2022-09-03T06:27:51.052 DEBUG:boto:Params: {} 2022-09-03T06:27:51.053 DEBUG:boto:Token: None 2022-09-03T06:27:51.054 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/c 2022-09-03T06:27:51.055 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wgqYToLmEdSyGIJE6UYijc8nRqI= 2022-09-03T06:27:51.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wgqYToLmEdSyGIJE6UYijc8nRqI=', 'Content-Length': '0'} 2022-09-03T06:27:51.060 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000405ae8b09f5add20-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.061 DEBUG:boto:path=/dsgvcy-73/c 2022-09-03T06:27:51.063 DEBUG:boto:auth_path=/dsgvcy-73/c 2022-09-03T06:27:51.064 DEBUG:boto:Method: GET 2022-09-03T06:27:51.065 DEBUG:boto:Path: /dsgvcy-73/c 2022-09-03T06:27:51.066 DEBUG:boto:Data: 2022-09-03T06:27:51.067 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.067 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.068 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.069 DEBUG:boto:Params: {} 2022-09-03T06:27:51.070 DEBUG:boto:Token: None 2022-09-03T06:27:51.071 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/c 2022-09-03T06:27:51.072 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wgqYToLmEdSyGIJE6UYijc8nRqI= 2022-09-03T06:27:51.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wgqYToLmEdSyGIJE6UYijc8nRqI=', 'Content-Length': '0'} 2022-09-03T06:27:51.075 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f87bd262b0774aa2-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.122 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:51.123 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:51.123 DEBUG:boto:path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:51.124 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:51.125 DEBUG:boto:Method: GET 2022-09-03T06:27:51.125 DEBUG:boto:Path: /dsgvcy-73/?prefix=c 2022-09-03T06:27:51.126 DEBUG:boto:Data: 2022-09-03T06:27:51.127 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.127 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:51.128 DEBUG:boto:Port: 8001 2022-09-03T06:27:51.128 DEBUG:boto:Params: {} 2022-09-03T06:27:51.129 DEBUG:boto:Token: None 2022-09-03T06:27:51.130 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/ 2022-09-03T06:27:51.130 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A= 2022-09-03T06:27:51.131 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A=', 'Content-Length': '0'} 2022-09-03T06:27:51.134 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002c3a8599bb48db88-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.135 DEBUG:boto:b'dsgvcy-73c1000falsec2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:51.137 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:51.138 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:51.138 DEBUG:boto:path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:51.139 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=c 2022-09-03T06:27:51.140 DEBUG:boto:Method: GET 2022-09-03T06:27:51.140 DEBUG:boto:Path: /dsgvcy-73/?prefix=c 2022-09-03T06:27:51.141 DEBUG:boto:Data: 2022-09-03T06:27:51.142 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.143 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.143 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.144 DEBUG:boto:Params: {} 2022-09-03T06:27:51.145 DEBUG:boto:Token: None 2022-09-03T06:27:51.145 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/ 2022-09-03T06:27:51.147 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A= 2022-09-03T06:27:51.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A=', 'Content-Length': '0'} 2022-09-03T06:27:51.151 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e74a492a5eed6384-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.153 DEBUG:boto:b'dsgvcy-73c1000falsec2022-09-03T06:24:09.274Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:51.154 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:27:51.155 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:51.156 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:51.157 DEBUG:boto:path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.158 DEBUG:boto:auth_path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.159 DEBUG:boto:Method: GET 2022-09-03T06:27:51.160 DEBUG:boto:Path: /dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.161 DEBUG:boto:Data: 2022-09-03T06:27:51.162 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.163 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:51.164 DEBUG:boto:Port: 8001 2022-09-03T06:27:51.165 DEBUG:boto:Params: {} 2022-09-03T06:27:51.166 DEBUG:boto:Token: None 2022-09-03T06:27:51.168 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.169 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fVjA9D8JngtUlQgh6pUCcRaeQ+o= 2022-09-03T06:27:51.170 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fVjA9D8JngtUlQgh6pUCcRaeQ+o=', 'Content-Length': '0'} 2022-09-03T06:27:51.173 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a8570c1f3463aee-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.218 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:51.220 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:51.221 DEBUG:boto:path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.222 DEBUG:boto:auth_path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.224 DEBUG:boto:Method: GET 2022-09-03T06:27:51.224 DEBUG:boto:Path: /dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.225 DEBUG:boto:Data: 2022-09-03T06:27:51.226 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.226 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.228 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.229 DEBUG:boto:Params: {} 2022-09-03T06:27:51.230 DEBUG:boto:Token: None 2022-09-03T06:27:51.231 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.234 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fVjA9D8JngtUlQgh6pUCcRaeQ+o= 2022-09-03T06:27:51.235 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fVjA9D8JngtUlQgh6pUCcRaeQ+o=', 'Content-Length': '0'} 2022-09-03T06:27:51.237 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d51a580f14b08c69-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.238 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:51.239 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:51.240 DEBUG:boto:path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.240 DEBUG:boto:auth_path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.241 DEBUG:boto:Method: HEAD 2022-09-03T06:27:51.241 DEBUG:boto:Path: /dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.242 DEBUG:boto:Data: 2022-09-03T06:27:51.243 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.243 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:51.244 DEBUG:boto:Port: 8001 2022-09-03T06:27:51.244 DEBUG:boto:Params: {} 2022-09-03T06:27:51.246 DEBUG:boto:Token: None 2022-09-03T06:27:51.247 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.247 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4hxTBsGha8chNG+zXOTyHa4s4Ms= 2022-09-03T06:27:51.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4hxTBsGha8chNG+zXOTyHa4s4Ms=', 'Content-Length': '0'} 2022-09-03T06:27:51.251 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000714ce11117667e5f-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.252 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:51.253 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:51.254 DEBUG:boto:path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.255 DEBUG:boto:auth_path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.255 DEBUG:boto:Method: HEAD 2022-09-03T06:27:51.257 DEBUG:boto:Path: /dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.258 DEBUG:boto:Data: 2022-09-03T06:27:51.258 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.259 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.259 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.260 DEBUG:boto:Params: {} 2022-09-03T06:27:51.261 DEBUG:boto:Token: None 2022-09-03T06:27:51.261 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.262 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4hxTBsGha8chNG+zXOTyHa4s4Ms= 2022-09-03T06:27:51.263 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4hxTBsGha8chNG+zXOTyHa4s4Ms=', 'Content-Length': '0'} 2022-09-03T06:27:51.265 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000004cab3801c72a990-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.266 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:27:51.267 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:51.268 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:51.268 DEBUG:boto:path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.269 DEBUG:boto:auth_path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.269 DEBUG:boto:Method: GET 2022-09-03T06:27:51.270 DEBUG:boto:Path: /dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.270 DEBUG:boto:Data: 2022-09-03T06:27:51.271 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.271 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:51.272 DEBUG:boto:Port: 8001 2022-09-03T06:27:51.272 DEBUG:boto:Params: {} 2022-09-03T06:27:51.273 DEBUG:boto:Token: None 2022-09-03T06:27:51.274 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.274 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fVjA9D8JngtUlQgh6pUCcRaeQ+o= 2022-09-03T06:27:51.275 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fVjA9D8JngtUlQgh6pUCcRaeQ+o=', 'Content-Length': '0'} 2022-09-03T06:27:51.277 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000cdf1e5265034a33-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.278 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:51.279 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:51.279 DEBUG:boto:path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.280 DEBUG:boto:auth_path=/dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.280 DEBUG:boto:Method: GET 2022-09-03T06:27:51.281 DEBUG:boto:Path: /dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.281 DEBUG:boto:Data: 2022-09-03T06:27:51.282 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.283 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.283 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.284 DEBUG:boto:Params: {} 2022-09-03T06:27:51.284 DEBUG:boto:Token: None 2022-09-03T06:27:51.285 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/d?versionId=null 2022-09-03T06:27:51.286 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fVjA9D8JngtUlQgh6pUCcRaeQ+o= 2022-09-03T06:27:51.286 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fVjA9D8JngtUlQgh6pUCcRaeQ+o=', 'Content-Length': '0'} 2022-09-03T06:27:51.289 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1685cb28f9cd5d9-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.290 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:51.290 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:51.291 DEBUG:boto:path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:51.291 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:51.292 DEBUG:boto:Method: GET 2022-09-03T06:27:51.292 DEBUG:boto:Path: /dsgvcy-73/?prefix=d 2022-09-03T06:27:51.293 DEBUG:boto:Data: 2022-09-03T06:27:51.293 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.294 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:51.294 DEBUG:boto:Port: 8001 2022-09-03T06:27:51.295 DEBUG:boto:Params: {} 2022-09-03T06:27:51.296 DEBUG:boto:Token: None 2022-09-03T06:27:51.296 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/ 2022-09-03T06:27:51.297 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A= 2022-09-03T06:27:51.297 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A=', 'Content-Length': '0'} 2022-09-03T06:27:51.301 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007eb82c5ddff2b765-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.302 DEBUG:boto:b'dsgvcy-73d1000falsed2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:51.303 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:51.304 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:51.304 DEBUG:boto:path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:51.305 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:51.305 DEBUG:boto:Method: GET 2022-09-03T06:27:51.306 DEBUG:boto:Path: /dsgvcy-73/?prefix=d 2022-09-03T06:27:51.307 DEBUG:boto:Data: 2022-09-03T06:27:51.308 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.309 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.309 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.310 DEBUG:boto:Params: {} 2022-09-03T06:27:51.310 DEBUG:boto:Token: None 2022-09-03T06:27:51.311 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/ 2022-09-03T06:27:51.312 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A= 2022-09-03T06:27:51.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A=', 'Content-Length': '0'} 2022-09-03T06:27:51.315 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b68e1bed7d616398-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.316 DEBUG:boto:b'dsgvcy-73d1000falsed2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:51.317 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:51.317 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:51.319 DEBUG:boto:Method: HEAD 2022-09-03T06:27:51.319 DEBUG:boto:Path: /dsgvcy-73/d 2022-09-03T06:27:51.320 DEBUG:boto:Data: 2022-09-03T06:27:51.320 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.321 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:51.321 DEBUG:boto:Port: 8001 2022-09-03T06:27:51.322 DEBUG:boto:Params: {} 2022-09-03T06:27:51.323 DEBUG:boto:Token: None 2022-09-03T06:27:51.326 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/d 2022-09-03T06:27:51.327 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4M9fC6KoPrMAb0wEgbquV6RPpA8= 2022-09-03T06:27:51.327 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4M9fC6KoPrMAb0wEgbquV6RPpA8=', 'Content-Length': '0'} 2022-09-03T06:27:51.329 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007a4c6ea276b6e829-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.330 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:51.331 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:51.331 DEBUG:boto:Method: HEAD 2022-09-03T06:27:51.332 DEBUG:boto:Path: /dsgvcy-73/d 2022-09-03T06:27:51.333 DEBUG:boto:Data: 2022-09-03T06:27:51.333 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.334 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.335 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.336 DEBUG:boto:Params: {} 2022-09-03T06:27:51.336 DEBUG:boto:Token: None 2022-09-03T06:27:51.337 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/d 2022-09-03T06:27:51.338 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4M9fC6KoPrMAb0wEgbquV6RPpA8= 2022-09-03T06:27:51.338 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4M9fC6KoPrMAb0wEgbquV6RPpA8=', 'Content-Length': '0'} 2022-09-03T06:27:51.344 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002838ec8d4181344f-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.345 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=d 2022-09-03T06:27:51.347 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:51.348 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:51.349 DEBUG:boto:Method: GET 2022-09-03T06:27:51.350 DEBUG:boto:Path: /dsgvcy-73/d 2022-09-03T06:27:51.351 DEBUG:boto:Data: 2022-09-03T06:27:51.352 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.352 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:51.353 DEBUG:boto:Port: 8001 2022-09-03T06:27:51.353 DEBUG:boto:Params: {} 2022-09-03T06:27:51.354 DEBUG:boto:Token: None 2022-09-03T06:27:51.355 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/d 2022-09-03T06:27:51.355 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:P/94c8xHu0H9sa5x4wfz83xKEKg= 2022-09-03T06:27:51.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:P/94c8xHu0H9sa5x4wfz83xKEKg=', 'Content-Length': '0'} 2022-09-03T06:27:51.358 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a010efa06f86d59f-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.359 DEBUG:boto:path=/dsgvcy-73/d 2022-09-03T06:27:51.360 DEBUG:boto:auth_path=/dsgvcy-73/d 2022-09-03T06:27:51.360 DEBUG:boto:Method: GET 2022-09-03T06:27:51.361 DEBUG:boto:Path: /dsgvcy-73/d 2022-09-03T06:27:51.362 DEBUG:boto:Data: 2022-09-03T06:27:51.362 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.363 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.364 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.364 DEBUG:boto:Params: {} 2022-09-03T06:27:51.365 DEBUG:boto:Token: None 2022-09-03T06:27:51.365 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/d 2022-09-03T06:27:51.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:P/94c8xHu0H9sa5x4wfz83xKEKg= 2022-09-03T06:27:51.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:P/94c8xHu0H9sa5x4wfz83xKEKg=', 'Content-Length': '0'} 2022-09-03T06:27:51.369 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004f6bf0fe3186fb7b-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.370 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:51.371 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:51.372 DEBUG:boto:path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:51.372 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:51.373 DEBUG:boto:Method: GET 2022-09-03T06:27:51.373 DEBUG:boto:Path: /dsgvcy-73/?prefix=d 2022-09-03T06:27:51.374 DEBUG:boto:Data: 2022-09-03T06:27:51.374 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.375 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:51.375 DEBUG:boto:Port: 8001 2022-09-03T06:27:51.376 DEBUG:boto:Params: {} 2022-09-03T06:27:51.376 DEBUG:boto:Token: None 2022-09-03T06:27:51.377 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/ 2022-09-03T06:27:51.378 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A= 2022-09-03T06:27:51.378 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A=', 'Content-Length': '0'} 2022-09-03T06:27:51.385 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007026b6f5bad23ebe-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.426 DEBUG:boto:b'dsgvcy-73d1000falsed2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:51.429 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:51.430 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:51.431 DEBUG:boto:path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:51.432 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=d 2022-09-03T06:27:51.432 DEBUG:boto:Method: GET 2022-09-03T06:27:51.433 DEBUG:boto:Path: /dsgvcy-73/?prefix=d 2022-09-03T06:27:51.434 DEBUG:boto:Data: 2022-09-03T06:27:51.435 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.436 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.437 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.438 DEBUG:boto:Params: {} 2022-09-03T06:27:51.439 DEBUG:boto:Token: None 2022-09-03T06:27:51.440 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/ 2022-09-03T06:27:51.440 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A= 2022-09-03T06:27:51.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A=', 'Content-Length': '0'} 2022-09-03T06:27:51.446 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c9698f4b3cda2bc3-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.448 DEBUG:boto:b'dsgvcy-73d1000falsed2022-09-03T06:24:09.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:51.449 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:27:51.450 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:51.451 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:51.452 DEBUG:boto:path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.453 DEBUG:boto:auth_path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.454 DEBUG:boto:Method: GET 2022-09-03T06:27:51.455 DEBUG:boto:Path: /dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.456 DEBUG:boto:Data: 2022-09-03T06:27:51.457 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.457 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:51.458 DEBUG:boto:Port: 8001 2022-09-03T06:27:51.459 DEBUG:boto:Params: {} 2022-09-03T06:27:51.459 DEBUG:boto:Token: None 2022-09-03T06:27:51.460 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.461 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1SErXtGA8zmljUELhBydloUI9hE= 2022-09-03T06:27:51.462 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1SErXtGA8zmljUELhBydloUI9hE=', 'Content-Length': '0'} 2022-09-03T06:27:51.464 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000024f7685549e9576-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.466 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:51.466 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:51.467 DEBUG:boto:path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.468 DEBUG:boto:auth_path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.469 DEBUG:boto:Method: GET 2022-09-03T06:27:51.470 DEBUG:boto:Path: /dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.471 DEBUG:boto:Data: 2022-09-03T06:27:51.472 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.473 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.473 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.474 DEBUG:boto:Params: {} 2022-09-03T06:27:51.476 DEBUG:boto:Token: None 2022-09-03T06:27:51.476 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.477 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1SErXtGA8zmljUELhBydloUI9hE= 2022-09-03T06:27:51.478 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1SErXtGA8zmljUELhBydloUI9hE=', 'Content-Length': '0'} 2022-09-03T06:27:51.480 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d2609aeebf387e5e-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.523 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:51.524 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:51.524 DEBUG:boto:path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.525 DEBUG:boto:auth_path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.525 DEBUG:boto:Method: HEAD 2022-09-03T06:27:51.526 DEBUG:boto:Path: /dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.526 DEBUG:boto:Data: 2022-09-03T06:27:51.527 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.528 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:51.528 DEBUG:boto:Port: 8001 2022-09-03T06:27:51.529 DEBUG:boto:Params: {} 2022-09-03T06:27:51.530 DEBUG:boto:Token: None 2022-09-03T06:27:51.531 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.531 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xfepHu+CeHFhrOzX7PlAI6eDR6k= 2022-09-03T06:27:51.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xfepHu+CeHFhrOzX7PlAI6eDR6k=', 'Content-Length': '0'} 2022-09-03T06:27:51.535 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006382a9cf20abb126-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.535 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:51.536 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:51.536 DEBUG:boto:path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.537 DEBUG:boto:auth_path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.537 DEBUG:boto:Method: HEAD 2022-09-03T06:27:51.538 DEBUG:boto:Path: /dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.538 DEBUG:boto:Data: 2022-09-03T06:27:51.539 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.540 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.541 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.542 DEBUG:boto:Params: {} 2022-09-03T06:27:51.542 DEBUG:boto:Token: None 2022-09-03T06:27:51.543 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.544 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xfepHu+CeHFhrOzX7PlAI6eDR6k= 2022-09-03T06:27:51.545 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xfepHu+CeHFhrOzX7PlAI6eDR6k=', 'Content-Length': '0'} 2022-09-03T06:27:51.548 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ac9e094cc47393dc-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.549 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:27:51.550 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:51.551 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:51.551 DEBUG:boto:path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.552 DEBUG:boto:auth_path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.553 DEBUG:boto:Method: GET 2022-09-03T06:27:51.553 DEBUG:boto:Path: /dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.554 DEBUG:boto:Data: 2022-09-03T06:27:51.555 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.556 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:51.556 DEBUG:boto:Port: 8001 2022-09-03T06:27:51.557 DEBUG:boto:Params: {} 2022-09-03T06:27:51.558 DEBUG:boto:Token: None 2022-09-03T06:27:51.559 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.560 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1SErXtGA8zmljUELhBydloUI9hE= 2022-09-03T06:27:51.561 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1SErXtGA8zmljUELhBydloUI9hE=', 'Content-Length': '0'} 2022-09-03T06:27:51.563 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000017acb74339b4ddf2-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.564 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:51.565 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:51.566 DEBUG:boto:path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.567 DEBUG:boto:auth_path=/dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.568 DEBUG:boto:Method: GET 2022-09-03T06:27:51.569 DEBUG:boto:Path: /dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.569 DEBUG:boto:Data: 2022-09-03T06:27:51.570 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.570 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.571 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.571 DEBUG:boto:Params: {} 2022-09-03T06:27:51.573 DEBUG:boto:Token: None 2022-09-03T06:27:51.573 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/e?versionId=null 2022-09-03T06:27:51.574 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1SErXtGA8zmljUELhBydloUI9hE= 2022-09-03T06:27:51.575 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1SErXtGA8zmljUELhBydloUI9hE=', 'Content-Length': '0'} 2022-09-03T06:27:51.577 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aca1f6e148293e4d-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.580 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:51.581 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:51.581 DEBUG:boto:path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:51.582 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:51.582 DEBUG:boto:Method: GET 2022-09-03T06:27:51.583 DEBUG:boto:Path: /dsgvcy-73/?prefix=e 2022-09-03T06:27:51.584 DEBUG:boto:Data: 2022-09-03T06:27:51.584 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.585 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:51.585 DEBUG:boto:Port: 8001 2022-09-03T06:27:51.586 DEBUG:boto:Params: {} 2022-09-03T06:27:51.587 DEBUG:boto:Token: None 2022-09-03T06:27:51.588 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/ 2022-09-03T06:27:51.588 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A= 2022-09-03T06:27:51.589 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A=', 'Content-Length': '0'} 2022-09-03T06:27:51.593 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000050b9fa0800b5b7b6-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.594 DEBUG:boto:b'dsgvcy-73e1000falsee2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:51.595 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:51.596 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:51.597 DEBUG:boto:path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:51.598 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:51.598 DEBUG:boto:Method: GET 2022-09-03T06:27:51.599 DEBUG:boto:Path: /dsgvcy-73/?prefix=e 2022-09-03T06:27:51.600 DEBUG:boto:Data: 2022-09-03T06:27:51.601 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.602 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.603 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.604 DEBUG:boto:Params: {} 2022-09-03T06:27:51.605 DEBUG:boto:Token: None 2022-09-03T06:27:51.606 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/ 2022-09-03T06:27:51.607 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A= 2022-09-03T06:27:51.608 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A=', 'Content-Length': '0'} 2022-09-03T06:27:51.611 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004ea20c33a71738a3-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.612 DEBUG:boto:b'dsgvcy-73e1000falsee2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:51.613 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:51.614 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:51.614 DEBUG:boto:Method: HEAD 2022-09-03T06:27:51.615 DEBUG:boto:Path: /dsgvcy-73/e 2022-09-03T06:27:51.616 DEBUG:boto:Data: 2022-09-03T06:27:51.616 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.617 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:51.617 DEBUG:boto:Port: 8001 2022-09-03T06:27:51.618 DEBUG:boto:Params: {} 2022-09-03T06:27:51.619 DEBUG:boto:Token: None 2022-09-03T06:27:51.619 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/e 2022-09-03T06:27:51.620 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dAF0Uk9Wqi9WWoYpI3HjAA+W/i8= 2022-09-03T06:27:51.621 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dAF0Uk9Wqi9WWoYpI3HjAA+W/i8=', 'Content-Length': '0'} 2022-09-03T06:27:51.623 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e29b46df762a95ae-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.624 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:51.625 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:51.626 DEBUG:boto:Method: HEAD 2022-09-03T06:27:51.626 DEBUG:boto:Path: /dsgvcy-73/e 2022-09-03T06:27:51.627 DEBUG:boto:Data: 2022-09-03T06:27:51.628 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.629 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.629 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.630 DEBUG:boto:Params: {} 2022-09-03T06:27:51.631 DEBUG:boto:Token: None 2022-09-03T06:27:51.632 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/e 2022-09-03T06:27:51.633 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dAF0Uk9Wqi9WWoYpI3HjAA+W/i8= 2022-09-03T06:27:51.634 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dAF0Uk9Wqi9WWoYpI3HjAA+W/i8=', 'Content-Length': '0'} 2022-09-03T06:27:51.636 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d925bc55144e729-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.638 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=e 2022-09-03T06:27:51.639 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:51.640 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:51.640 DEBUG:boto:Method: GET 2022-09-03T06:27:51.641 DEBUG:boto:Path: /dsgvcy-73/e 2022-09-03T06:27:51.641 DEBUG:boto:Data: 2022-09-03T06:27:51.642 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.642 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:51.643 DEBUG:boto:Port: 8001 2022-09-03T06:27:51.644 DEBUG:boto:Params: {} 2022-09-03T06:27:51.644 DEBUG:boto:Token: None 2022-09-03T06:27:51.645 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/e 2022-09-03T06:27:51.646 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:u8cr7nQVwPTGxCKDSi/h7Kdql80= 2022-09-03T06:27:51.646 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:u8cr7nQVwPTGxCKDSi/h7Kdql80=', 'Content-Length': '0'} 2022-09-03T06:27:51.648 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fb3959b5777fa2ca-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.694 DEBUG:boto:path=/dsgvcy-73/e 2022-09-03T06:27:51.696 DEBUG:boto:auth_path=/dsgvcy-73/e 2022-09-03T06:27:51.697 DEBUG:boto:Method: GET 2022-09-03T06:27:51.698 DEBUG:boto:Path: /dsgvcy-73/e 2022-09-03T06:27:51.699 DEBUG:boto:Data: 2022-09-03T06:27:51.700 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.701 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.705 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.706 DEBUG:boto:Params: {} 2022-09-03T06:27:51.707 DEBUG:boto:Token: None 2022-09-03T06:27:51.708 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/e 2022-09-03T06:27:51.709 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:u8cr7nQVwPTGxCKDSi/h7Kdql80= 2022-09-03T06:27:51.710 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:u8cr7nQVwPTGxCKDSi/h7Kdql80=', 'Content-Length': '0'} 2022-09-03T06:27:51.712 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e2ca5f651d5dfd3-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.713 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:51.714 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:51.715 DEBUG:boto:path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:51.715 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:51.716 DEBUG:boto:Method: GET 2022-09-03T06:27:51.716 DEBUG:boto:Path: /dsgvcy-73/?prefix=e 2022-09-03T06:27:51.717 DEBUG:boto:Data: 2022-09-03T06:27:51.717 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.718 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:51.718 DEBUG:boto:Port: 8001 2022-09-03T06:27:51.719 DEBUG:boto:Params: {} 2022-09-03T06:27:51.720 DEBUG:boto:Token: None 2022-09-03T06:27:51.721 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/ 2022-09-03T06:27:51.721 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A= 2022-09-03T06:27:51.722 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A=', 'Content-Length': '0'} 2022-09-03T06:27:51.726 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007d75ebfa1bd92ed8-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.727 DEBUG:boto:b'dsgvcy-73e1000falsee2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:51.728 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:51.728 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:51.729 DEBUG:boto:path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:51.729 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=e 2022-09-03T06:27:51.730 DEBUG:boto:Method: GET 2022-09-03T06:27:51.731 DEBUG:boto:Path: /dsgvcy-73/?prefix=e 2022-09-03T06:27:51.733 DEBUG:boto:Data: 2022-09-03T06:27:51.734 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.735 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.736 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.737 DEBUG:boto:Params: {} 2022-09-03T06:27:51.738 DEBUG:boto:Token: None 2022-09-03T06:27:51.751 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/ 2022-09-03T06:27:51.752 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A= 2022-09-03T06:27:51.752 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A=', 'Content-Length': '0'} 2022-09-03T06:27:51.755 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000611504e2d194fccb-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.756 DEBUG:boto:b'dsgvcy-73e1000falsee2022-09-03T06:24:09.485Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:51.757 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:27:51.758 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:51.759 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:51.759 DEBUG:boto:path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.760 DEBUG:boto:auth_path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.761 DEBUG:boto:Method: GET 2022-09-03T06:27:51.762 DEBUG:boto:Path: /dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.763 DEBUG:boto:Data: 2022-09-03T06:27:51.764 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.764 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:51.765 DEBUG:boto:Port: 8001 2022-09-03T06:27:51.765 DEBUG:boto:Params: {} 2022-09-03T06:27:51.766 DEBUG:boto:Token: None 2022-09-03T06:27:51.767 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.767 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4ywml1TcWf1j/+oIsKYsdI5i6VY= 2022-09-03T06:27:51.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4ywml1TcWf1j/+oIsKYsdI5i6VY=', 'Content-Length': '0'} 2022-09-03T06:27:51.771 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c26c09a77c2ed8d5-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.772 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:51.773 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:51.774 DEBUG:boto:path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.775 DEBUG:boto:auth_path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.776 DEBUG:boto:Method: GET 2022-09-03T06:27:51.777 DEBUG:boto:Path: /dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.777 DEBUG:boto:Data: 2022-09-03T06:27:51.778 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.779 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.780 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.781 DEBUG:boto:Params: {} 2022-09-03T06:27:51.782 DEBUG:boto:Token: None 2022-09-03T06:27:51.783 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.784 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4ywml1TcWf1j/+oIsKYsdI5i6VY= 2022-09-03T06:27:51.785 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4ywml1TcWf1j/+oIsKYsdI5i6VY=', 'Content-Length': '0'} 2022-09-03T06:27:51.787 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e5b01cbd83c56f1-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.788 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:51.789 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:51.789 DEBUG:boto:path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.790 DEBUG:boto:auth_path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.791 DEBUG:boto:Method: HEAD 2022-09-03T06:27:51.791 DEBUG:boto:Path: /dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.792 DEBUG:boto:Data: 2022-09-03T06:27:51.793 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.793 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:51.794 DEBUG:boto:Port: 8001 2022-09-03T06:27:51.794 DEBUG:boto:Params: {} 2022-09-03T06:27:51.795 DEBUG:boto:Token: None 2022-09-03T06:27:51.796 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.797 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EflexenZjA0W4VtYoj0buKKyyq0= 2022-09-03T06:27:51.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EflexenZjA0W4VtYoj0buKKyyq0=', 'Content-Length': '0'} 2022-09-03T06:27:51.799 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ce12b758253e6b4d-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.800 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:51.801 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:51.801 DEBUG:boto:path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.802 DEBUG:boto:auth_path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.803 DEBUG:boto:Method: HEAD 2022-09-03T06:27:51.804 DEBUG:boto:Path: /dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.804 DEBUG:boto:Data: 2022-09-03T06:27:51.805 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.806 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.807 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.808 DEBUG:boto:Params: {} 2022-09-03T06:27:51.808 DEBUG:boto:Token: None 2022-09-03T06:27:51.809 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.810 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EflexenZjA0W4VtYoj0buKKyyq0= 2022-09-03T06:27:51.811 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EflexenZjA0W4VtYoj0buKKyyq0=', 'Content-Length': '0'} 2022-09-03T06:27:51.813 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004c468c55a635a60c-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.814 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:27:51.815 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:51.816 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:51.818 DEBUG:boto:path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.819 DEBUG:boto:auth_path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.820 DEBUG:boto:Method: GET 2022-09-03T06:27:51.821 DEBUG:boto:Path: /dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.822 DEBUG:boto:Data: 2022-09-03T06:27:51.823 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.824 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:51.825 DEBUG:boto:Port: 8001 2022-09-03T06:27:51.826 DEBUG:boto:Params: {} 2022-09-03T06:27:51.827 DEBUG:boto:Token: None 2022-09-03T06:27:51.827 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.828 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4ywml1TcWf1j/+oIsKYsdI5i6VY= 2022-09-03T06:27:51.829 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4ywml1TcWf1j/+oIsKYsdI5i6VY=', 'Content-Length': '0'} 2022-09-03T06:27:51.831 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000010b56c3f2ad5d9ff-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.878 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:51.879 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:51.879 DEBUG:boto:path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.880 DEBUG:boto:auth_path=/dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.881 DEBUG:boto:Method: GET 2022-09-03T06:27:51.881 DEBUG:boto:Path: /dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.882 DEBUG:boto:Data: 2022-09-03T06:27:51.883 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.883 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.884 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.884 DEBUG:boto:Params: {} 2022-09-03T06:27:51.885 DEBUG:boto:Token: None 2022-09-03T06:27:51.885 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/f?versionId=null 2022-09-03T06:27:51.886 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4ywml1TcWf1j/+oIsKYsdI5i6VY= 2022-09-03T06:27:51.891 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4ywml1TcWf1j/+oIsKYsdI5i6VY=', 'Content-Length': '0'} 2022-09-03T06:27:51.894 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b1a8a796a885386c-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.896 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:51.897 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:51.898 DEBUG:boto:path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:51.899 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:51.899 DEBUG:boto:Method: GET 2022-09-03T06:27:51.900 DEBUG:boto:Path: /dsgvcy-73/?prefix=f 2022-09-03T06:27:51.901 DEBUG:boto:Data: 2022-09-03T06:27:51.902 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.902 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:51.903 DEBUG:boto:Port: 8001 2022-09-03T06:27:51.904 DEBUG:boto:Params: {} 2022-09-03T06:27:51.904 DEBUG:boto:Token: None 2022-09-03T06:27:51.905 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/ 2022-09-03T06:27:51.906 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A= 2022-09-03T06:27:51.906 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A=', 'Content-Length': '0'} 2022-09-03T06:27:51.909 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000679a14064aca8e3e-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.910 DEBUG:boto:b'dsgvcy-73f1000falsef2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:51.911 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:51.912 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:51.912 DEBUG:boto:path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:51.913 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:51.913 DEBUG:boto:Method: GET 2022-09-03T06:27:51.914 DEBUG:boto:Path: /dsgvcy-73/?prefix=f 2022-09-03T06:27:51.915 DEBUG:boto:Data: 2022-09-03T06:27:51.915 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.916 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.916 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.917 DEBUG:boto:Params: {} 2022-09-03T06:27:51.917 DEBUG:boto:Token: None 2022-09-03T06:27:51.918 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/ 2022-09-03T06:27:51.919 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A= 2022-09-03T06:27:51.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GweO0P7TDyWFbivW8SltikqDp9A=', 'Content-Length': '0'} 2022-09-03T06:27:51.923 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000205b6c982303678-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.925 DEBUG:boto:b'dsgvcy-73f1000falsef2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:51.926 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:51.927 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:51.928 DEBUG:boto:Method: HEAD 2022-09-03T06:27:51.929 DEBUG:boto:Path: /dsgvcy-73/f 2022-09-03T06:27:51.930 DEBUG:boto:Data: 2022-09-03T06:27:51.931 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.932 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:51.933 DEBUG:boto:Port: 8001 2022-09-03T06:27:51.934 DEBUG:boto:Params: {} 2022-09-03T06:27:51.935 DEBUG:boto:Token: None 2022-09-03T06:27:51.936 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/f 2022-09-03T06:27:51.937 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uZuHdJ/KS2y/OZYEfsH8S/RDX7o= 2022-09-03T06:27:51.938 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uZuHdJ/KS2y/OZYEfsH8S/RDX7o=', 'Content-Length': '0'} 2022-09-03T06:27:51.940 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000042630929de522d2f-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.941 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:51.942 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:51.942 DEBUG:boto:Method: HEAD 2022-09-03T06:27:51.943 DEBUG:boto:Path: /dsgvcy-73/f 2022-09-03T06:27:51.944 DEBUG:boto:Data: 2022-09-03T06:27:51.945 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.946 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.947 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.947 DEBUG:boto:Params: {} 2022-09-03T06:27:51.948 DEBUG:boto:Token: None 2022-09-03T06:27:51.948 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/f 2022-09-03T06:27:51.949 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uZuHdJ/KS2y/OZYEfsH8S/RDX7o= 2022-09-03T06:27:51.950 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uZuHdJ/KS2y/OZYEfsH8S/RDX7o=', 'Content-Length': '0'} 2022-09-03T06:27:51.952 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004d1f0cb62e7c244f-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.953 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=f 2022-09-03T06:27:51.954 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:51.956 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:51.957 DEBUG:boto:Method: GET 2022-09-03T06:27:51.958 DEBUG:boto:Path: /dsgvcy-73/f 2022-09-03T06:27:51.959 DEBUG:boto:Data: 2022-09-03T06:27:51.960 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.961 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:51.962 DEBUG:boto:Port: 8001 2022-09-03T06:27:51.963 DEBUG:boto:Params: {} 2022-09-03T06:27:51.964 DEBUG:boto:Token: None 2022-09-03T06:27:51.964 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/f 2022-09-03T06:27:51.965 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8yGrL24NqPDjR/Q76YVTjHdj0pM= 2022-09-03T06:27:51.965 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8yGrL24NqPDjR/Q76YVTjHdj0pM=', 'Content-Length': '0'} 2022-09-03T06:27:51.968 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f61d5d33ab325b9e-006312f3e7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:51.969 DEBUG:boto:path=/dsgvcy-73/f 2022-09-03T06:27:51.970 DEBUG:boto:auth_path=/dsgvcy-73/f 2022-09-03T06:27:51.972 DEBUG:boto:Method: GET 2022-09-03T06:27:51.973 DEBUG:boto:Path: /dsgvcy-73/f 2022-09-03T06:27:51.974 DEBUG:boto:Data: 2022-09-03T06:27:51.974 DEBUG:boto:Headers: {} 2022-09-03T06:27:51.975 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:51.976 DEBUG:boto:Port: 8000 2022-09-03T06:27:51.976 DEBUG:boto:Params: {} 2022-09-03T06:27:51.977 DEBUG:boto:Token: None 2022-09-03T06:27:51.978 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:51 GMT /dsgvcy-73/f 2022-09-03T06:27:51.978 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8yGrL24NqPDjR/Q76YVTjHdj0pM= 2022-09-03T06:27:51.979 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8yGrL24NqPDjR/Q76YVTjHdj0pM=', 'Content-Length': '0'} 2022-09-03T06:27:51.981 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000851c865b1301bb62-006312f3e7-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.022 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:52.023 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:52.024 DEBUG:boto:path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:52.025 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:52.025 DEBUG:boto:Method: GET 2022-09-03T06:27:52.026 DEBUG:boto:Path: /dsgvcy-73/?prefix=f 2022-09-03T06:27:52.027 DEBUG:boto:Data: 2022-09-03T06:27:52.027 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.028 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:52.029 DEBUG:boto:Port: 8001 2022-09-03T06:27:52.029 DEBUG:boto:Params: {} 2022-09-03T06:27:52.031 DEBUG:boto:Token: None 2022-09-03T06:27:52.032 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/ 2022-09-03T06:27:52.034 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CS1PwVDNWU7Mf96DOavhRYvvfC4= 2022-09-03T06:27:52.037 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CS1PwVDNWU7Mf96DOavhRYvvfC4=', 'Content-Length': '0'} 2022-09-03T06:27:52.040 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000975913ccd4e4070b-006312f3e8-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.041 DEBUG:boto:b'dsgvcy-73f1000falsef2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:52.043 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:52.044 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:52.045 DEBUG:boto:path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:52.045 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=f 2022-09-03T06:27:52.046 DEBUG:boto:Method: GET 2022-09-03T06:27:52.047 DEBUG:boto:Path: /dsgvcy-73/?prefix=f 2022-09-03T06:27:52.048 DEBUG:boto:Data: 2022-09-03T06:27:52.048 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.049 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:52.050 DEBUG:boto:Port: 8000 2022-09-03T06:27:52.050 DEBUG:boto:Params: {} 2022-09-03T06:27:52.051 DEBUG:boto:Token: None 2022-09-03T06:27:52.052 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/ 2022-09-03T06:27:52.052 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CS1PwVDNWU7Mf96DOavhRYvvfC4= 2022-09-03T06:27:52.053 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CS1PwVDNWU7Mf96DOavhRYvvfC4=', 'Content-Length': '0'} 2022-09-03T06:27:52.056 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000d45edec1ef6d97b-006312f3e8-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.057 DEBUG:boto:b'dsgvcy-73f1000falsef2022-09-03T06:24:09.514Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:52.058 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:27:52.058 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:52.059 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:52.060 DEBUG:boto:path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.060 DEBUG:boto:auth_path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.061 DEBUG:boto:Method: GET 2022-09-03T06:27:52.062 DEBUG:boto:Path: /dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.063 DEBUG:boto:Data: 2022-09-03T06:27:52.064 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.065 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:52.066 DEBUG:boto:Port: 8001 2022-09-03T06:27:52.067 DEBUG:boto:Params: {} 2022-09-03T06:27:52.068 DEBUG:boto:Token: None 2022-09-03T06:27:52.069 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.070 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:03FVlzrAxNNLlcLctmFkx01Wcrg= 2022-09-03T06:27:52.071 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:03FVlzrAxNNLlcLctmFkx01Wcrg=', 'Content-Length': '0'} 2022-09-03T06:27:52.074 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d4e32057c337cc4b-006312f3e8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.076 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:52.077 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:52.078 DEBUG:boto:path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.079 DEBUG:boto:auth_path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.080 DEBUG:boto:Method: GET 2022-09-03T06:27:52.081 DEBUG:boto:Path: /dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.082 DEBUG:boto:Data: 2022-09-03T06:27:52.083 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.084 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:52.084 DEBUG:boto:Port: 8000 2022-09-03T06:27:52.086 DEBUG:boto:Params: {} 2022-09-03T06:27:52.087 DEBUG:boto:Token: None 2022-09-03T06:27:52.088 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.089 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:03FVlzrAxNNLlcLctmFkx01Wcrg= 2022-09-03T06:27:52.090 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:03FVlzrAxNNLlcLctmFkx01Wcrg=', 'Content-Length': '0'} 2022-09-03T06:27:52.092 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e2a21ecc78a34737-006312f3e8-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.093 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:52.094 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:52.095 DEBUG:boto:path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.095 DEBUG:boto:auth_path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.096 DEBUG:boto:Method: HEAD 2022-09-03T06:27:52.097 DEBUG:boto:Path: /dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.097 DEBUG:boto:Data: 2022-09-03T06:27:52.098 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.099 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:52.100 DEBUG:boto:Port: 8001 2022-09-03T06:27:52.101 DEBUG:boto:Params: {} 2022-09-03T06:27:52.101 DEBUG:boto:Token: None 2022-09-03T06:27:52.102 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.103 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kHgm7LC3nVHMsqI9AfEX3ekZIiw= 2022-09-03T06:27:52.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kHgm7LC3nVHMsqI9AfEX3ekZIiw=', 'Content-Length': '0'} 2022-09-03T06:27:52.107 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000744563d99e28e8e0-006312f3e8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.108 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:52.108 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:52.109 DEBUG:boto:path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.110 DEBUG:boto:auth_path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.111 DEBUG:boto:Method: HEAD 2022-09-03T06:27:52.111 DEBUG:boto:Path: /dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.112 DEBUG:boto:Data: 2022-09-03T06:27:52.113 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.114 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:52.114 DEBUG:boto:Port: 8000 2022-09-03T06:27:52.115 DEBUG:boto:Params: {} 2022-09-03T06:27:52.116 DEBUG:boto:Token: None 2022-09-03T06:27:52.117 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.117 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kHgm7LC3nVHMsqI9AfEX3ekZIiw= 2022-09-03T06:27:52.118 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kHgm7LC3nVHMsqI9AfEX3ekZIiw=', 'Content-Length': '0'} 2022-09-03T06:27:52.121 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ff5721e4e58bd33e-006312f3e8-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.123 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:27:52.124 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:52.125 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:52.125 DEBUG:boto:path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.126 DEBUG:boto:auth_path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.127 DEBUG:boto:Method: GET 2022-09-03T06:27:52.127 DEBUG:boto:Path: /dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.128 DEBUG:boto:Data: 2022-09-03T06:27:52.128 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.129 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:52.130 DEBUG:boto:Port: 8001 2022-09-03T06:27:52.131 DEBUG:boto:Params: {} 2022-09-03T06:27:52.131 DEBUG:boto:Token: None 2022-09-03T06:27:52.132 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.133 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:03FVlzrAxNNLlcLctmFkx01Wcrg= 2022-09-03T06:27:52.134 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:03FVlzrAxNNLlcLctmFkx01Wcrg=', 'Content-Length': '0'} 2022-09-03T06:27:52.136 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009aed58a13cf8ada4-006312f3e8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.182 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:52.183 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:52.184 DEBUG:boto:path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.184 DEBUG:boto:auth_path=/dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.185 DEBUG:boto:Method: GET 2022-09-03T06:27:52.185 DEBUG:boto:Path: /dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.186 DEBUG:boto:Data: 2022-09-03T06:27:52.186 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.187 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:52.187 DEBUG:boto:Port: 8000 2022-09-03T06:27:52.188 DEBUG:boto:Params: {} 2022-09-03T06:27:52.189 DEBUG:boto:Token: None 2022-09-03T06:27:52.189 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/g?versionId=null 2022-09-03T06:27:52.190 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:03FVlzrAxNNLlcLctmFkx01Wcrg= 2022-09-03T06:27:52.191 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:03FVlzrAxNNLlcLctmFkx01Wcrg=', 'Content-Length': '0'} 2022-09-03T06:27:52.193 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f29c2ad38b1266a0-006312f3e8-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.194 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:52.195 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:52.195 DEBUG:boto:path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:52.196 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:52.196 DEBUG:boto:Method: GET 2022-09-03T06:27:52.197 DEBUG:boto:Path: /dsgvcy-73/?prefix=g 2022-09-03T06:27:52.197 DEBUG:boto:Data: 2022-09-03T06:27:52.198 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.198 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:52.199 DEBUG:boto:Port: 8001 2022-09-03T06:27:52.199 DEBUG:boto:Params: {} 2022-09-03T06:27:52.200 DEBUG:boto:Token: None 2022-09-03T06:27:52.201 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/ 2022-09-03T06:27:52.202 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CS1PwVDNWU7Mf96DOavhRYvvfC4= 2022-09-03T06:27:52.202 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CS1PwVDNWU7Mf96DOavhRYvvfC4=', 'Content-Length': '0'} 2022-09-03T06:27:52.205 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000073916d6ad30920b7-006312f3e8-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.206 DEBUG:boto:b'dsgvcy-73g1000falseg2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:52.207 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:52.207 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:52.208 DEBUG:boto:path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:52.208 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:52.209 DEBUG:boto:Method: GET 2022-09-03T06:27:52.209 DEBUG:boto:Path: /dsgvcy-73/?prefix=g 2022-09-03T06:27:52.210 DEBUG:boto:Data: 2022-09-03T06:27:52.210 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.211 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:52.212 DEBUG:boto:Port: 8000 2022-09-03T06:27:52.212 DEBUG:boto:Params: {} 2022-09-03T06:27:52.213 DEBUG:boto:Token: None 2022-09-03T06:27:52.213 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/ 2022-09-03T06:27:52.214 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CS1PwVDNWU7Mf96DOavhRYvvfC4= 2022-09-03T06:27:52.215 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CS1PwVDNWU7Mf96DOavhRYvvfC4=', 'Content-Length': '0'} 2022-09-03T06:27:52.217 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000edecd714f889b141-006312f3e8-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.218 DEBUG:boto:b'dsgvcy-73g1000falseg2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:52.219 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:52.220 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:52.220 DEBUG:boto:Method: HEAD 2022-09-03T06:27:52.221 DEBUG:boto:Path: /dsgvcy-73/g 2022-09-03T06:27:52.221 DEBUG:boto:Data: 2022-09-03T06:27:52.222 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.222 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:52.223 DEBUG:boto:Port: 8001 2022-09-03T06:27:52.223 DEBUG:boto:Params: {} 2022-09-03T06:27:52.224 DEBUG:boto:Token: None 2022-09-03T06:27:52.224 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/g 2022-09-03T06:27:52.225 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KuR29cOIRp5fBDbkVAHncAjVUGE= 2022-09-03T06:27:52.225 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KuR29cOIRp5fBDbkVAHncAjVUGE=', 'Content-Length': '0'} 2022-09-03T06:27:52.227 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004ba1fcaec121aa1d-006312f3e8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.228 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:52.229 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:52.229 DEBUG:boto:Method: HEAD 2022-09-03T06:27:52.230 DEBUG:boto:Path: /dsgvcy-73/g 2022-09-03T06:27:52.231 DEBUG:boto:Data: 2022-09-03T06:27:52.231 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.232 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:52.232 DEBUG:boto:Port: 8000 2022-09-03T06:27:52.233 DEBUG:boto:Params: {} 2022-09-03T06:27:52.233 DEBUG:boto:Token: None 2022-09-03T06:27:52.234 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/g 2022-09-03T06:27:52.234 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KuR29cOIRp5fBDbkVAHncAjVUGE= 2022-09-03T06:27:52.235 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KuR29cOIRp5fBDbkVAHncAjVUGE=', 'Content-Length': '0'} 2022-09-03T06:27:52.237 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000542a13405c04ef12-006312f3e8-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.238 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=g 2022-09-03T06:27:52.239 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:52.239 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:52.240 DEBUG:boto:Method: GET 2022-09-03T06:27:52.240 DEBUG:boto:Path: /dsgvcy-73/g 2022-09-03T06:27:52.241 DEBUG:boto:Data: 2022-09-03T06:27:52.242 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.242 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:52.243 DEBUG:boto:Port: 8001 2022-09-03T06:27:52.243 DEBUG:boto:Params: {} 2022-09-03T06:27:52.244 DEBUG:boto:Token: None 2022-09-03T06:27:52.245 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/g 2022-09-03T06:27:52.245 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/qkUdjSA3juEcC2A+dbyVGFrI5o= 2022-09-03T06:27:52.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/qkUdjSA3juEcC2A+dbyVGFrI5o=', 'Content-Length': '0'} 2022-09-03T06:27:52.248 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000824f1aa9a584c2b3-006312f3e8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.249 DEBUG:boto:path=/dsgvcy-73/g 2022-09-03T06:27:52.250 DEBUG:boto:auth_path=/dsgvcy-73/g 2022-09-03T06:27:52.250 DEBUG:boto:Method: GET 2022-09-03T06:27:52.251 DEBUG:boto:Path: /dsgvcy-73/g 2022-09-03T06:27:52.251 DEBUG:boto:Data: 2022-09-03T06:27:52.252 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.252 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:52.253 DEBUG:boto:Port: 8000 2022-09-03T06:27:52.253 DEBUG:boto:Params: {} 2022-09-03T06:27:52.254 DEBUG:boto:Token: None 2022-09-03T06:27:52.255 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/g 2022-09-03T06:27:52.255 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/qkUdjSA3juEcC2A+dbyVGFrI5o= 2022-09-03T06:27:52.256 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/qkUdjSA3juEcC2A+dbyVGFrI5o=', 'Content-Length': '0'} 2022-09-03T06:27:52.258 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007bbd503024c3c6b0-006312f3e8-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.259 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:52.260 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:52.260 DEBUG:boto:path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:52.261 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:52.262 DEBUG:boto:Method: GET 2022-09-03T06:27:52.262 DEBUG:boto:Path: /dsgvcy-73/?prefix=g 2022-09-03T06:27:52.263 DEBUG:boto:Data: 2022-09-03T06:27:52.263 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.264 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:52.264 DEBUG:boto:Port: 8001 2022-09-03T06:27:52.265 DEBUG:boto:Params: {} 2022-09-03T06:27:52.265 DEBUG:boto:Token: None 2022-09-03T06:27:52.266 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/ 2022-09-03T06:27:52.267 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CS1PwVDNWU7Mf96DOavhRYvvfC4= 2022-09-03T06:27:52.267 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CS1PwVDNWU7Mf96DOavhRYvvfC4=', 'Content-Length': '0'} 2022-09-03T06:27:52.274 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003dfd14066c67e042-006312f3e8-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.275 DEBUG:boto:b'dsgvcy-73g1000falseg2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:52.276 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:52.277 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:52.278 DEBUG:boto:path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:52.278 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=g 2022-09-03T06:27:52.279 DEBUG:boto:Method: GET 2022-09-03T06:27:52.280 DEBUG:boto:Path: /dsgvcy-73/?prefix=g 2022-09-03T06:27:52.280 DEBUG:boto:Data: 2022-09-03T06:27:52.281 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.281 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:52.282 DEBUG:boto:Port: 8000 2022-09-03T06:27:52.283 DEBUG:boto:Params: {} 2022-09-03T06:27:52.283 DEBUG:boto:Token: None 2022-09-03T06:27:52.284 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/ 2022-09-03T06:27:52.285 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CS1PwVDNWU7Mf96DOavhRYvvfC4= 2022-09-03T06:27:52.285 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CS1PwVDNWU7Mf96DOavhRYvvfC4=', 'Content-Length': '0'} 2022-09-03T06:27:52.288 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000401139f7d5a5b05a-006312f3e8-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.289 DEBUG:boto:b'dsgvcy-73g1000falseg2022-09-03T06:24:09.588Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:52.289 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:27:52.290 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:52.291 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:52.291 DEBUG:boto:path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.292 DEBUG:boto:auth_path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.292 DEBUG:boto:Method: GET 2022-09-03T06:27:52.293 DEBUG:boto:Path: /dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.293 DEBUG:boto:Data: 2022-09-03T06:27:52.294 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.294 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:52.295 DEBUG:boto:Port: 8001 2022-09-03T06:27:52.295 DEBUG:boto:Params: {} 2022-09-03T06:27:52.296 DEBUG:boto:Token: None 2022-09-03T06:27:52.297 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.303 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ar+a7/Il+SiZ55Ul2VJ1aLz8tNc= 2022-09-03T06:27:52.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ar+a7/Il+SiZ55Ul2VJ1aLz8tNc=', 'Content-Length': '0'} 2022-09-03T06:27:52.307 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000880739ff84c6d96d-006312f3e8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.308 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:52.308 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:52.309 DEBUG:boto:path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.310 DEBUG:boto:auth_path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.310 DEBUG:boto:Method: GET 2022-09-03T06:27:52.311 DEBUG:boto:Path: /dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.311 DEBUG:boto:Data: 2022-09-03T06:27:52.312 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.312 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:52.313 DEBUG:boto:Port: 8000 2022-09-03T06:27:52.313 DEBUG:boto:Params: {} 2022-09-03T06:27:52.314 DEBUG:boto:Token: None 2022-09-03T06:27:52.315 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.315 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ar+a7/Il+SiZ55Ul2VJ1aLz8tNc= 2022-09-03T06:27:52.316 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ar+a7/Il+SiZ55Ul2VJ1aLz8tNc=', 'Content-Length': '0'} 2022-09-03T06:27:52.319 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000003c04d447d2232b-006312f3e8-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.321 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:52.322 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:52.323 DEBUG:boto:path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.323 DEBUG:boto:auth_path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.324 DEBUG:boto:Method: HEAD 2022-09-03T06:27:52.324 DEBUG:boto:Path: /dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.325 DEBUG:boto:Data: 2022-09-03T06:27:52.327 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.327 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:52.328 DEBUG:boto:Port: 8001 2022-09-03T06:27:52.328 DEBUG:boto:Params: {} 2022-09-03T06:27:52.329 DEBUG:boto:Token: None 2022-09-03T06:27:52.330 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.330 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:am+c4xo7JQ34DhbszaJc9Ey7Lw8= 2022-09-03T06:27:52.331 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:am+c4xo7JQ34DhbszaJc9Ey7Lw8=', 'Content-Length': '0'} 2022-09-03T06:27:52.333 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f210b2be8472d7c-006312f3e8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.335 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:52.336 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:52.338 DEBUG:boto:path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.339 DEBUG:boto:auth_path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.340 DEBUG:boto:Method: HEAD 2022-09-03T06:27:52.341 DEBUG:boto:Path: /dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.341 DEBUG:boto:Data: 2022-09-03T06:27:52.342 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.342 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:52.343 DEBUG:boto:Port: 8000 2022-09-03T06:27:52.343 DEBUG:boto:Params: {} 2022-09-03T06:27:52.344 DEBUG:boto:Token: None 2022-09-03T06:27:52.345 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.346 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:am+c4xo7JQ34DhbszaJc9Ey7Lw8= 2022-09-03T06:27:52.346 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:am+c4xo7JQ34DhbszaJc9Ey7Lw8=', 'Content-Length': '0'} 2022-09-03T06:27:52.348 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001f8e8f619294fa59-006312f3e8-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.349 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:27:52.350 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:52.350 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:52.351 DEBUG:boto:path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.351 DEBUG:boto:auth_path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.352 DEBUG:boto:Method: GET 2022-09-03T06:27:52.352 DEBUG:boto:Path: /dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.353 DEBUG:boto:Data: 2022-09-03T06:27:52.353 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.354 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:52.354 DEBUG:boto:Port: 8001 2022-09-03T06:27:52.355 DEBUG:boto:Params: {} 2022-09-03T06:27:52.356 DEBUG:boto:Token: None 2022-09-03T06:27:52.357 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.359 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ar+a7/Il+SiZ55Ul2VJ1aLz8tNc= 2022-09-03T06:27:52.359 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ar+a7/Il+SiZ55Ul2VJ1aLz8tNc=', 'Content-Length': '0'} 2022-09-03T06:27:52.361 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000da005fadb5152266-006312f3e8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.402 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:52.403 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:52.403 DEBUG:boto:path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.404 DEBUG:boto:auth_path=/dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.405 DEBUG:boto:Method: GET 2022-09-03T06:27:52.406 DEBUG:boto:Path: /dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.408 DEBUG:boto:Data: 2022-09-03T06:27:52.409 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.409 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:52.411 DEBUG:boto:Port: 8000 2022-09-03T06:27:52.411 DEBUG:boto:Params: {} 2022-09-03T06:27:52.412 DEBUG:boto:Token: None 2022-09-03T06:27:52.413 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/h?versionId=null 2022-09-03T06:27:52.413 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ar+a7/Il+SiZ55Ul2VJ1aLz8tNc= 2022-09-03T06:27:52.414 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ar+a7/Il+SiZ55Ul2VJ1aLz8tNc=', 'Content-Length': '0'} 2022-09-03T06:27:52.417 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008bc5184dc26b09cf-006312f3e8-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.418 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:52.419 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:52.419 DEBUG:boto:path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:52.420 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:52.420 DEBUG:boto:Method: GET 2022-09-03T06:27:52.421 DEBUG:boto:Path: /dsgvcy-73/?prefix=h 2022-09-03T06:27:52.421 DEBUG:boto:Data: 2022-09-03T06:27:52.422 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.423 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:52.423 DEBUG:boto:Port: 8001 2022-09-03T06:27:52.424 DEBUG:boto:Params: {} 2022-09-03T06:27:52.424 DEBUG:boto:Token: None 2022-09-03T06:27:52.425 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/ 2022-09-03T06:27:52.426 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CS1PwVDNWU7Mf96DOavhRYvvfC4= 2022-09-03T06:27:52.427 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CS1PwVDNWU7Mf96DOavhRYvvfC4=', 'Content-Length': '0'} 2022-09-03T06:27:52.430 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000037de2779d46b93de-006312f3e8-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.430 DEBUG:boto:b'dsgvcy-73h1000falseh2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:52.431 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:52.432 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:52.432 DEBUG:boto:path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:52.433 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:52.433 DEBUG:boto:Method: GET 2022-09-03T06:27:52.434 DEBUG:boto:Path: /dsgvcy-73/?prefix=h 2022-09-03T06:27:52.435 DEBUG:boto:Data: 2022-09-03T06:27:52.436 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.436 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:52.437 DEBUG:boto:Port: 8000 2022-09-03T06:27:52.437 DEBUG:boto:Params: {} 2022-09-03T06:27:52.438 DEBUG:boto:Token: None 2022-09-03T06:27:52.439 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/ 2022-09-03T06:27:52.439 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CS1PwVDNWU7Mf96DOavhRYvvfC4= 2022-09-03T06:27:52.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CS1PwVDNWU7Mf96DOavhRYvvfC4=', 'Content-Length': '0'} 2022-09-03T06:27:52.443 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004bb8d882fa3af322-006312f3e8-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.443 DEBUG:boto:b'dsgvcy-73h1000falseh2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:52.444 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:52.445 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:52.445 DEBUG:boto:Method: HEAD 2022-09-03T06:27:52.446 DEBUG:boto:Path: /dsgvcy-73/h 2022-09-03T06:27:52.447 DEBUG:boto:Data: 2022-09-03T06:27:52.447 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.448 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:52.448 DEBUG:boto:Port: 8001 2022-09-03T06:27:52.449 DEBUG:boto:Params: {} 2022-09-03T06:27:52.450 DEBUG:boto:Token: None 2022-09-03T06:27:52.450 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/h 2022-09-03T06:27:52.451 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AQn/T3Gw9xNavF9F4hJPM7bB77g= 2022-09-03T06:27:52.451 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AQn/T3Gw9xNavF9F4hJPM7bB77g=', 'Content-Length': '0'} 2022-09-03T06:27:52.453 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008361b8d17bd4f35d-006312f3e8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.454 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:52.456 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:52.458 DEBUG:boto:Method: HEAD 2022-09-03T06:27:52.460 DEBUG:boto:Path: /dsgvcy-73/h 2022-09-03T06:27:52.460 DEBUG:boto:Data: 2022-09-03T06:27:52.461 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.461 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:52.462 DEBUG:boto:Port: 8000 2022-09-03T06:27:52.462 DEBUG:boto:Params: {} 2022-09-03T06:27:52.463 DEBUG:boto:Token: None 2022-09-03T06:27:52.464 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/h 2022-09-03T06:27:52.464 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AQn/T3Gw9xNavF9F4hJPM7bB77g= 2022-09-03T06:27:52.465 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AQn/T3Gw9xNavF9F4hJPM7bB77g=', 'Content-Length': '0'} 2022-09-03T06:27:52.467 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000069fe904d676f13de-006312f3e8-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.468 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=h 2022-09-03T06:27:52.469 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:52.469 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:52.470 DEBUG:boto:Method: GET 2022-09-03T06:27:52.470 DEBUG:boto:Path: /dsgvcy-73/h 2022-09-03T06:27:52.471 DEBUG:boto:Data: 2022-09-03T06:27:52.472 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.472 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:52.473 DEBUG:boto:Port: 8001 2022-09-03T06:27:52.474 DEBUG:boto:Params: {} 2022-09-03T06:27:52.474 DEBUG:boto:Token: None 2022-09-03T06:27:52.475 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/h 2022-09-03T06:27:52.475 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EJ/+Og076rcSSxxVo88dDxGxXBA= 2022-09-03T06:27:52.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EJ/+Og076rcSSxxVo88dDxGxXBA=', 'Content-Length': '0'} 2022-09-03T06:27:52.478 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c164595c40bf3f73-006312f3e8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.479 DEBUG:boto:path=/dsgvcy-73/h 2022-09-03T06:27:52.479 DEBUG:boto:auth_path=/dsgvcy-73/h 2022-09-03T06:27:52.480 DEBUG:boto:Method: GET 2022-09-03T06:27:52.480 DEBUG:boto:Path: /dsgvcy-73/h 2022-09-03T06:27:52.481 DEBUG:boto:Data: 2022-09-03T06:27:52.481 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.482 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:52.483 DEBUG:boto:Port: 8000 2022-09-03T06:27:52.484 DEBUG:boto:Params: {} 2022-09-03T06:27:52.485 DEBUG:boto:Token: None 2022-09-03T06:27:52.486 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/h 2022-09-03T06:27:52.486 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EJ/+Og076rcSSxxVo88dDxGxXBA= 2022-09-03T06:27:52.487 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EJ/+Og076rcSSxxVo88dDxGxXBA=', 'Content-Length': '0'} 2022-09-03T06:27:52.489 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:24:09 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000098721b3574eaa5e0-006312f3e8-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.490 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:52.490 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:52.491 DEBUG:boto:path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:52.492 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:52.493 DEBUG:boto:Method: GET 2022-09-03T06:27:52.493 DEBUG:boto:Path: /dsgvcy-73/?prefix=h 2022-09-03T06:27:52.494 DEBUG:boto:Data: 2022-09-03T06:27:52.495 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.495 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:52.496 DEBUG:boto:Port: 8001 2022-09-03T06:27:52.496 DEBUG:boto:Params: {} 2022-09-03T06:27:52.497 DEBUG:boto:Token: None 2022-09-03T06:27:52.497 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/ 2022-09-03T06:27:52.498 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CS1PwVDNWU7Mf96DOavhRYvvfC4= 2022-09-03T06:27:52.499 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CS1PwVDNWU7Mf96DOavhRYvvfC4=', 'Content-Length': '0'} 2022-09-03T06:27:52.502 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000092be8a32ebce6dce-006312f3e8-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.542 DEBUG:boto:b'dsgvcy-73h1000falseh2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:52.543 DEBUG:boto:path=/dsgvcy-73/ 2022-09-03T06:27:52.544 DEBUG:boto:auth_path=/dsgvcy-73/ 2022-09-03T06:27:52.544 DEBUG:boto:path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:52.545 DEBUG:boto:auth_path=/dsgvcy-73/?prefix=h 2022-09-03T06:27:52.545 DEBUG:boto:Method: GET 2022-09-03T06:27:52.546 DEBUG:boto:Path: /dsgvcy-73/?prefix=h 2022-09-03T06:27:52.546 DEBUG:boto:Data: 2022-09-03T06:27:52.547 DEBUG:boto:Headers: {} 2022-09-03T06:27:52.547 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:52.548 DEBUG:boto:Port: 8000 2022-09-03T06:27:52.548 DEBUG:boto:Params: {} 2022-09-03T06:27:52.549 DEBUG:boto:Token: None 2022-09-03T06:27:52.550 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:52 GMT /dsgvcy-73/ 2022-09-03T06:27:52.550 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CS1PwVDNWU7Mf96DOavhRYvvfC4= 2022-09-03T06:27:52.551 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CS1PwVDNWU7Mf96DOavhRYvvfC4=', 'Content-Length': '0'} 2022-09-03T06:27:52.554 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a295907c082debcd-006312f3e8-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:52.554 DEBUG:boto:b'dsgvcy-73h1000falseh2022-09-03T06:24:09.623Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:27:52.555 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-73 zones={test-zone3, test-zone2} 2022-09-03T06:27:52.557 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_reshard_full ... ok 2022-09-03T06:27:52.559 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:27:52.559 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:27:52.560 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:27:52.617 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:27:52.618 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:27:52.651 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T06:27:52.667 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T06:27:52.667 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:27:52.668 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:27:52.668 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:27:52.669 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:27:52.670 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:27:52.670 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:27:52.671 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:27:52.671 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:27:52.672 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:27:52.672 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662186249.417961_480.1", 2022-09-03T06:27:52.673 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:24:09.417961Z" 2022-09-03T06:27:52.673 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:27:52.674 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:27:52.674 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:27:52.675 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:27:52.675 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:27:52.675 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T06:27:52.678 DEBUG:rgw_multi.tests:master meta markers={0: '', 1: '', 2: '1_1662186249.417961_480.1', 3: ''} 2022-09-03T06:27:52.679 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T06:27:52.679 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:27:52.680 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:27:52.682 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:27:52.741 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:27:52.741 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:27:52.772 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:27:52.770+0000 7f2626c3f600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:27:52.773 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:27:52.771+0000 7f2626c3f600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:27:52.787 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:27:52.787 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:27:52.787 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:27:52.788 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:27:52.788 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:27:52.789 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:27:52.790 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:27:52.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:52.790 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:27:52.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:27:52.791 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:27:52.791 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:27:52.791 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:27:52.792 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:27:52.792 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:27:52.792 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:27:52.793 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:27:52.793 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:27:52.793 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:27:52.793 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:27:52.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:52.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:27:52.794 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:27:52.795 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:27:52.795 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:27:52.795 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:27:52.796 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:27:52.796 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:27:52.798 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:27:52.798 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:27:52.799 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:27:52.799 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:27:52.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:52.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:27:52.800 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:27:52.801 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:27:52.802 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:27:52.803 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662186249.417961_480.1", 2022-09-03T06:27:52.803 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:27:52.803 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:27:52.804 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:27:52.805 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:24:09.417961Z", 2022-09-03T06:27:52.805 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:27:52.806 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:27:52.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:52.807 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:27:52.807 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:27:52.810 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:27:52.810 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:27:52.811 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:27:52.811 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:27:52.812 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:27:52.812 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:27:52.813 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:27:52.813 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:27:52.814 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:27:52.815 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:27:52.815 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:27:52.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:52.816 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:27:52.817 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:27:52.817 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:27:52.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:52.818 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:27:52Z" 2022-09-03T06:27:52.819 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:27:52.820 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662186249.417961_480.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:24:09.417961Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:27:52Z" } 2022-09-03T06:27:52.821 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662186249.417961_480.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:24:09.417961Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:27:52.822 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662186249.417961_480.1', 3: ''} 2022-09-03T06:27:52.823 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662186249.417961_480.1', 3: ''} 2022-09-03T06:27:52.823 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T06:27:52.824 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T06:27:52.824 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:27:52.825 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:27:52.825 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:27:52.877 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:27:52.878 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:27:52.914 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:27:52.913+0000 7f0178ef0600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:27:52.916 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:27:52.914+0000 7f0178ef0600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:27:52.941 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T06:27:52.941 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T06:27:52.942 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T06:27:52.942 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T06:27:52.943 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T06:27:52.943 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:27:52.943 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:27:52.944 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:27:52.944 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T06:27:52.945 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:27:52.945 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T06:27:52.945 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:27:52.947 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:27:52.947 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:27:52.947 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:27:52.948 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T06:27:52.948 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:27:52.948 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:27:52.949 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:27:52.949 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:27:52.949 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:27:52.949 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:27:52.950 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T06:27:52.950 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:27:52.950 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:27:52.950 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:27:52.951 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:27:52.951 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:27:52.951 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:27:52.951 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T06:27:52.952 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:27:52.952 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:27:52.952 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:27:52.952 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:27:52.953 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T06:27:52.953 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:27:52.953 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:27:52.954 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662186249.417961_480.1", 2022-09-03T06:27:52.954 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:27:52.954 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:27:52.954 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:27:52.955 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:24:09.417961Z", 2022-09-03T06:27:52.955 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:27:52.955 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:27:52.956 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:27:52.956 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:27:52.956 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T06:27:52.957 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:27:52.957 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:27:52.957 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:27:52.957 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:27:52.958 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:27:52.958 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:27:52.958 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:27:52.959 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:27:52.959 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:27:52.959 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:27:52.960 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:27:52.960 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:27:52.960 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T06:27:52.960 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T06:27:52.961 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T06:27:52.961 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:27:52.961 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T06:27:52Z" 2022-09-03T06:27:52.962 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T06:27:52.963 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662186249.417961_480.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:24:09.417961Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:27:52Z" } 2022-09-03T06:27:52.964 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662186249.417961_480.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:24:09.417961Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:27:52.964 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662186249.417961_480.1', 3: ''} 2022-09-03T06:27:52.965 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662186249.417961_480.1', 3: ''} 2022-09-03T06:27:52.966 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T06:27:52.966 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T06:27:52.967 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:27:52.967 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:27:52.968 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:27:53.021 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:27:53.021 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:27:53.061 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:27:53.059+0000 7f8764227600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:27:53.061 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:27:53.060+0000 7f8764227600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:27:53.083 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:27:53.083 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:27:53.084 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:27:53.084 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:27:53.084 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:27:53.084 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:27:53.085 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:27:53.085 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:53.085 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:27:53.086 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:27:53.086 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:27:53.086 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:27:53.086 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:27:53.087 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:27:53.087 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:27:53.087 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:27:53.087 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:27:53.088 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:27:53.088 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:27:53.088 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:27:53.089 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:53.089 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:27:53.089 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:27:53.090 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:27:53.090 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:27:53.091 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:27:53.091 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:27:53.092 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:27:53.092 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:27:53.093 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:27:53.093 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:27:53.094 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:27:53.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:53.095 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:27:53.095 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:27:53.096 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:27:53.096 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:27:53.098 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662186249.417961_480.1", 2022-09-03T06:27:53.098 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:27:53.099 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:27:53.099 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:27:53.100 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:24:09.417961Z", 2022-09-03T06:27:53.100 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:27:53.101 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:27:53.102 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:53.102 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:27:53.103 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:27:53.103 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:27:53.110 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:27:53.111 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:27:53.111 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:27:53.112 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:27:53.112 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:27:53.113 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:27:53.113 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:27:53.114 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:27:53.114 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:27:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:27:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:27:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:27:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:27:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:27:53Z" 2022-09-03T06:27:53.117 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:27:53.118 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662186249.417961_480.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:24:09.417961Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:27:53Z" } 2022-09-03T06:27:53.119 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662186249.417961_480.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:24:09.417961Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:27:53.119 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662186249.417961_480.1', 3: ''} 2022-09-03T06:27:53.120 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662186249.417961_480.1', 3: ''} 2022-09-03T06:27:53.120 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T06:27:53.121 DEBUG:boto:path=/ 2022-09-03T06:27:53.121 DEBUG:boto:auth_path=/ 2022-09-03T06:27:53.122 DEBUG:boto:Method: GET 2022-09-03T06:27:53.123 DEBUG:boto:Path: / 2022-09-03T06:27:53.124 DEBUG:boto:Data: 2022-09-03T06:27:53.124 DEBUG:boto:Headers: {} 2022-09-03T06:27:53.125 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:53.126 DEBUG:boto:Port: 8000 2022-09-03T06:27:53.127 DEBUG:boto:Params: {} 2022-09-03T06:27:53.128 DEBUG:boto:Token: None 2022-09-03T06:27:53.129 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:53 GMT / 2022-09-03T06:27:53.130 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SFTYBxpMC4sq7uMtZUfHRHMEdbs= 2022-09-03T06:27:53.131 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SFTYBxpMC4sq7uMtZUfHRHMEdbs=', 'Content-Length': '0'} 2022-09-03T06:27:53.136 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000481ccb50145e05b0-006312f3e9-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:53.139 DEBUG:boto:path=/ 2022-09-03T06:27:53.140 DEBUG:boto:auth_path=/ 2022-09-03T06:27:53.141 DEBUG:boto:Method: GET 2022-09-03T06:27:53.141 DEBUG:boto:Path: / 2022-09-03T06:27:53.142 DEBUG:boto:Data: 2022-09-03T06:27:53.144 DEBUG:boto:Headers: {} 2022-09-03T06:27:53.145 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:27:53.146 DEBUG:boto:Port: 8000 2022-09-03T06:27:53.147 DEBUG:boto:Params: {} 2022-09-03T06:27:53.149 DEBUG:boto:Token: None 2022-09-03T06:27:53.150 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:53 GMT / 2022-09-03T06:27:53.151 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SFTYBxpMC4sq7uMtZUfHRHMEdbs= 2022-09-03T06:27:53.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SFTYBxpMC4sq7uMtZUfHRHMEdbs=', 'Content-Length': '0'} 2022-09-03T06:27:53.157 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000137d9449652ecf63-006312f3e9-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:53.160 DEBUG:boto:path=/ 2022-09-03T06:27:53.161 DEBUG:boto:auth_path=/ 2022-09-03T06:27:53.162 DEBUG:boto:Method: GET 2022-09-03T06:27:53.163 DEBUG:boto:Path: / 2022-09-03T06:27:53.164 DEBUG:boto:Data: 2022-09-03T06:27:53.164 DEBUG:boto:Headers: {} 2022-09-03T06:27:53.165 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:27:53.165 DEBUG:boto:Port: 8001 2022-09-03T06:27:53.166 DEBUG:boto:Params: {} 2022-09-03T06:27:53.167 DEBUG:boto:Token: None 2022-09-03T06:27:53.168 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:27:53 GMT / 2022-09-03T06:27:53.169 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SFTYBxpMC4sq7uMtZUfHRHMEdbs= 2022-09-03T06:27:53.169 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SFTYBxpMC4sq7uMtZUfHRHMEdbs=', 'Content-Length': '0'} 2022-09-03T06:27:53.172 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000202c3532ec00dc64-006312f3e9-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:27:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:53.175 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_creation_time ... ok 2022-09-03T06:27:53.176 DEBUG:boto:path=/dsgvcy-74/ 2022-09-03T06:27:53.178 DEBUG:boto:auth_path=/dsgvcy-74/ 2022-09-03T06:27:53.179 DEBUG:boto:Method: PUT 2022-09-03T06:27:53.179 DEBUG:boto:Path: /dsgvcy-74/ 2022-09-03T06:27:53.180 DEBUG:boto:Data: 2022-09-03T06:27:53.181 DEBUG:boto:Headers: {} 2022-09-03T06:27:53.181 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:53.182 DEBUG:boto:Port: 8000 2022-09-03T06:27:53.183 DEBUG:boto:Params: {} 2022-09-03T06:27:53.183 DEBUG:boto:Token: None 2022-09-03T06:27:53.184 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 06:27:53 GMT /dsgvcy-74/ 2022-09-03T06:27:53.185 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZPMhlSrUQSkaZ+PgnfIt6J12Stw= 2022-09-03T06:27:53.186 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZPMhlSrUQSkaZ+PgnfIt6J12Stw=', 'Content-Length': '0'} 2022-09-03T06:27:53.211 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000048ac322d1f755c7-006312f3e9-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:27:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:53.212 DEBUG:rgw_multi.tests:created bucket=dsgvcy-74 2022-09-03T06:27:53.213 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:27:53.214 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:27:53.214 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:27:53.263 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:27:53.263 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:27:53.300 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T06:27:53.314 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T06:27:53.314 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:27:53.315 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:27:53.315 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:27:53.316 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:27:53.316 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:27:53.316 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:27:53.317 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:27:53.317 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:27:53.318 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:27:53.318 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662186473.206979_485.1", 2022-09-03T06:27:53.319 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:27:53.206979Z" 2022-09-03T06:27:53.319 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:27:53.320 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:27:53.320 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:27:53.320 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:27:53.321 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:27:53.326 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T06:27:53.328 DEBUG:rgw_multi.tests:master meta markers={0: '', 1: '', 2: '1_1662186473.206979_485.1', 3: ''} 2022-09-03T06:27:53.329 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T06:27:53.329 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:27:53.330 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:27:53.330 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:27:53.382 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:27:53.383 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:27:53.409 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:27:53.407+0000 7fe2c1f29600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:27:53.410 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:27:53.408+0000 7fe2c1f29600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:27:53.423 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:27:53.424 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:27:53.424 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:27:53.424 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:27:53.424 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:27:53.425 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:27:53.425 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:27:53.425 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:53.428 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:27:53.428 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:27:53.428 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:27:53.429 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:27:53.429 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:27:53.429 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:27:53.429 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:27:53.430 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:27:53.430 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:27:53.430 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:27:53.431 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:27:53.431 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:27:53.431 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:53.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:27:53.432 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:27:53.432 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:27:53.434 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:27:53.434 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:27:53.434 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:27:53.435 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:27:53.435 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:27:53.435 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:27:53.435 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:27:53.436 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:27:53.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:53.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:27:53.437 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:27:53.437 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:27:53.437 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:27:53.438 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662186473.206979_485.1", 2022-09-03T06:27:53.438 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:27:53.438 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:27:53.439 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:27:53.439 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:27:53.206979Z", 2022-09-03T06:27:53.439 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:27:53.439 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:27:53.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:53.440 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:27:53.440 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:27:53.441 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:27:53.441 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:27:53.441 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:27:53.442 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:27:53.442 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:27:53.442 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:27:53.442 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:27:53.444 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:27:53.444 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:27:53.445 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:27:53.445 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:27:53.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:53.446 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:27:53.446 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:27:53.447 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:27:53.447 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:53.447 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:27:53Z" 2022-09-03T06:27:53.448 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:27:53.449 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662186473.206979_485.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:27:53.206979Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:27:53Z" } 2022-09-03T06:27:53.450 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662186473.206979_485.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:27:53.206979Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:27:53.451 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662186473.206979_485.1', 3: ''} 2022-09-03T06:27:53.451 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662186473.206979_485.1', 3: ''} 2022-09-03T06:27:53.452 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T06:27:53.452 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T06:27:53.453 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:27:53.454 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:27:53.455 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:27:53.506 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:27:53.526 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:27:53.537 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:27:53.535+0000 7f99fa7a7600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:27:53.538 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:27:53.536+0000 7f99fa7a7600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:27:53.563 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T06:27:53.564 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T06:27:53.564 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T06:27:53.564 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T06:27:53.564 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T06:27:53.565 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:27:53.565 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:27:53.566 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:27:53.566 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T06:27:53.567 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:27:53.569 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T06:27:53.569 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:27:53.569 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:27:53.570 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:27:53.570 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:27:53.571 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T06:27:53.571 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:27:53.571 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:27:53.572 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:27:53.572 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:27:53.572 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:27:53.573 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:27:53.573 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T06:27:53.573 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:27:53.573 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:27:53.574 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:27:53.574 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:27:53.574 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:27:53.575 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:27:53.575 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T06:27:53.575 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:27:53.576 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:27:53.576 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:27:53.576 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:27:53.577 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T06:27:53.577 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:27:53.578 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:27:53.578 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662186473.206979_485.1", 2022-09-03T06:27:53.578 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:27:53.578 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:27:53.579 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:27:53.579 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:27:53.206979Z", 2022-09-03T06:27:53.579 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:27:53.580 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:27:53.580 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:27:53.580 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:27:53.581 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T06:27:53.581 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:27:53.581 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:27:53.581 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:27:53.582 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:27:53.583 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:27:53.583 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:27:53.583 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:27:53.583 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:27:53.584 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:27:53.584 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:27:53.585 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:27:53.585 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:27:53.585 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T06:27:53.586 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T06:27:53.586 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T06:27:53.586 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:27:53.586 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T06:27:53Z" 2022-09-03T06:27:53.587 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T06:27:53.587 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662186473.206979_485.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:27:53.206979Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:27:53Z" } 2022-09-03T06:27:53.588 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662186473.206979_485.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:27:53.206979Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:27:53.588 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662186473.206979_485.1', 3: ''} 2022-09-03T06:27:53.589 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662186473.206979_485.1', 3: ''} 2022-09-03T06:27:53.590 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T06:27:53.590 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T06:27:53.591 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:27:53.591 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:27:53.592 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:27:53.643 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:27:53.645 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:27:53.672 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:27:53.669+0000 7f70d9101600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:27:53.673 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:27:53.670+0000 7f70d9101600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:27:53.687 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:27:53.688 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:27:53.688 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:27:53.688 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:27:53.689 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:27:53.689 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:27:53.689 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:27:53.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:53.690 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:27:53.691 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:27:53.691 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:27:53.692 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:27:53.692 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:27:53.693 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:27:53.693 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:27:53.693 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:27:53.694 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:27:53.695 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:27:53.695 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:27:53.695 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:27:53.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:53.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:27:53.697 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:27:53.697 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:27:53.699 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:27:53.699 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:27:53.700 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:27:53.700 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:27:53.701 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:27:53.701 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:27:53.702 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:27:53.702 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:27:53.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:53.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:27:53.704 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:27:53.704 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:27:53.705 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:27:53.705 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662186473.206979_485.1", 2022-09-03T06:27:53.706 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:27:53.706 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:27:53.707 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:27:53.707 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:27:53.206979Z", 2022-09-03T06:27:53.708 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:27:53.708 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:27:53.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:53.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:27:53.711 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:27:53.711 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:27:53.711 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:27:53.712 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:27:53.712 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:27:53.713 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:27:53.713 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:27:53.713 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:27:53.713 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:27:53.714 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:27:53.714 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:27:53.714 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:27:53.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:53.715 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:27:53.715 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:27:53.715 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:27:53.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:27:53.716 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:27:53Z" 2022-09-03T06:27:53.716 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:27:53.718 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662186473.206979_485.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:27:53.206979Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:27:53Z" } 2022-09-03T06:27:53.719 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662186473.206979_485.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:27:53.206979Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:27:53.719 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662186473.206979_485.1', 3: ''} 2022-09-03T06:27:53.720 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662186473.206979_485.1', 3: ''} 2022-09-03T06:27:53.720 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T06:27:53.721 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'shard', 'objects', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--num-shards', '11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:27:53.721 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'shard', 'objects', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--num-shards', '11', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:27:53.722 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket shard objects --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --num-shards 11 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:27:53.775 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:27:53.776 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:27:53.842 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T06:27:53.843 INFO:teuthology.orchestra.run.smithi181.stdout: "objs": [ 2022-09-03T06:27:53.843 INFO:teuthology.orchestra.run.smithi181.stdout: "obj00000000000000000001", 2022-09-03T06:27:53.843 INFO:teuthology.orchestra.run.smithi181.stdout: "obj00000000000000000005", 2022-09-03T06:27:53.844 INFO:teuthology.orchestra.run.smithi181.stdout: "obj00000000000000000012", 2022-09-03T06:27:53.844 INFO:teuthology.orchestra.run.smithi181.stdout: "obj00000000000000000004", 2022-09-03T06:27:53.845 INFO:teuthology.orchestra.run.smithi181.stdout: "obj00000000000000000011", 2022-09-03T06:27:53.845 INFO:teuthology.orchestra.run.smithi181.stdout: "obj00000000000000000000", 2022-09-03T06:27:53.846 INFO:teuthology.orchestra.run.smithi181.stdout: "obj00000000000000000010", 2022-09-03T06:27:53.846 INFO:teuthology.orchestra.run.smithi181.stdout: "obj00000000000000000003", 2022-09-03T06:27:53.846 INFO:teuthology.orchestra.run.smithi181.stdout: "obj00000000000000000013", 2022-09-03T06:27:53.847 INFO:teuthology.orchestra.run.smithi181.stdout: "obj00000000000000000002", 2022-09-03T06:27:53.848 INFO:teuthology.orchestra.run.smithi181.stdout: "obj00000000000000000006" 2022-09-03T06:27:53.848 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:27:53.849 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T06:27:53.855 DEBUG:boto:path=/dsgvcy-74/ 2022-09-03T06:27:53.856 DEBUG:boto:auth_path=/dsgvcy-74/ 2022-09-03T06:27:53.856 DEBUG:boto:Method: HEAD 2022-09-03T06:27:53.857 DEBUG:boto:Path: /dsgvcy-74/ 2022-09-03T06:27:53.857 DEBUG:boto:Data: 2022-09-03T06:27:53.858 DEBUG:boto:Headers: {} 2022-09-03T06:27:53.859 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:53.859 DEBUG:boto:Port: 8000 2022-09-03T06:27:53.863 DEBUG:boto:Params: {} 2022-09-03T06:27:53.864 DEBUG:boto:Token: None 2022-09-03T06:27:53.865 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:53 GMT /dsgvcy-74/ 2022-09-03T06:27:53.865 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R+CnHJ6cJdyi4uC7NwL/rtnE3Rg= 2022-09-03T06:27:53.866 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R+CnHJ6cJdyi4uC7NwL/rtnE3Rg=', 'Content-Length': '0'} 2022-09-03T06:27:53.873 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002ce0848d5cbb6abb-006312f3e9-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:27:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:53.874 DEBUG:boto:path=/dsgvcy-74/obj00000000000000000010 2022-09-03T06:27:53.875 DEBUG:boto:auth_path=/dsgvcy-74/obj00000000000000000010 2022-09-03T06:27:53.875 DEBUG:boto:Method: PUT 2022-09-03T06:27:53.876 DEBUG:boto:Path: /dsgvcy-74/obj00000000000000000010 2022-09-03T06:27:53.876 DEBUG:boto:Data: 2022-09-03T06:27:53.877 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:27:53.877 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:53.878 DEBUG:boto:Port: 8000 2022-09-03T06:27:53.878 DEBUG:boto:Params: {} 2022-09-03T06:27:53.879 DEBUG:boto:Token: None 2022-09-03T06:27:53.880 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:27:53 GMT /dsgvcy-74/obj00000000000000000010 2022-09-03T06:27:53.880 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:krL8mLm6uNhtw+yXHmIptU1DqVU= 2022-09-03T06:27:53.881 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:27:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:krL8mLm6uNhtw+yXHmIptU1DqVU='} 2022-09-03T06:27:53.931 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000dc667f3828be2265-006312f3e9-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:27:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:53.934 DEBUG:boto:path=/dsgvcy-74/ 2022-09-03T06:27:53.935 DEBUG:boto:auth_path=/dsgvcy-74/ 2022-09-03T06:27:53.936 DEBUG:boto:Method: HEAD 2022-09-03T06:27:53.936 DEBUG:boto:Path: /dsgvcy-74/ 2022-09-03T06:27:53.937 DEBUG:boto:Data: 2022-09-03T06:27:53.938 DEBUG:boto:Headers: {} 2022-09-03T06:27:53.939 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:53.940 DEBUG:boto:Port: 8000 2022-09-03T06:27:53.940 DEBUG:boto:Params: {} 2022-09-03T06:27:53.941 DEBUG:boto:Token: None 2022-09-03T06:27:53.942 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:53 GMT /dsgvcy-74/ 2022-09-03T06:27:53.943 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R+CnHJ6cJdyi4uC7NwL/rtnE3Rg= 2022-09-03T06:27:53.945 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R+CnHJ6cJdyi4uC7NwL/rtnE3Rg=', 'Content-Length': '0'} 2022-09-03T06:27:53.953 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '3'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000dac4cf557300201a-006312f3e9-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:27:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:53.956 DEBUG:boto:path=/dsgvcy-74/obj00000000000000000003 2022-09-03T06:27:53.957 DEBUG:boto:auth_path=/dsgvcy-74/obj00000000000000000003 2022-09-03T06:27:53.957 DEBUG:boto:Method: PUT 2022-09-03T06:27:53.958 DEBUG:boto:Path: /dsgvcy-74/obj00000000000000000003 2022-09-03T06:27:53.958 DEBUG:boto:Data: 2022-09-03T06:27:53.959 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:27:53.959 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:53.960 DEBUG:boto:Port: 8000 2022-09-03T06:27:53.960 DEBUG:boto:Params: {} 2022-09-03T06:27:53.961 DEBUG:boto:Token: None 2022-09-03T06:27:53.962 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:27:53 GMT /dsgvcy-74/obj00000000000000000003 2022-09-03T06:27:53.962 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YRE5mMQ4Je53+5ON8AMdRn9bQFo= 2022-09-03T06:27:53.965 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:27:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YRE5mMQ4Je53+5ON8AMdRn9bQFo='} 2022-09-03T06:27:53.988 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ea7f33e5e0577527-006312f3e9-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:27:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:53.989 DEBUG:boto:path=/dsgvcy-74/ 2022-09-03T06:27:53.989 DEBUG:boto:auth_path=/dsgvcy-74/ 2022-09-03T06:27:53.990 DEBUG:boto:Method: HEAD 2022-09-03T06:27:53.991 DEBUG:boto:Path: /dsgvcy-74/ 2022-09-03T06:27:53.992 DEBUG:boto:Data: 2022-09-03T06:27:53.993 DEBUG:boto:Headers: {} 2022-09-03T06:27:53.993 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:53.994 DEBUG:boto:Port: 8000 2022-09-03T06:27:53.995 DEBUG:boto:Params: {} 2022-09-03T06:27:53.996 DEBUG:boto:Token: None 2022-09-03T06:27:53.996 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:53 GMT /dsgvcy-74/ 2022-09-03T06:27:53.997 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R+CnHJ6cJdyi4uC7NwL/rtnE3Rg= 2022-09-03T06:27:53.998 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R+CnHJ6cJdyi4uC7NwL/rtnE3Rg=', 'Content-Length': '0'} 2022-09-03T06:27:54.010 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a9b4595457da898a-006312f3ea-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:27:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:54.011 DEBUG:boto:path=/dsgvcy-74/obj00000000000000000002 2022-09-03T06:27:54.012 DEBUG:boto:auth_path=/dsgvcy-74/obj00000000000000000002 2022-09-03T06:27:54.012 DEBUG:boto:Method: PUT 2022-09-03T06:27:54.013 DEBUG:boto:Path: /dsgvcy-74/obj00000000000000000002 2022-09-03T06:27:54.013 DEBUG:boto:Data: 2022-09-03T06:27:54.014 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:27:54.014 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:54.015 DEBUG:boto:Port: 8000 2022-09-03T06:27:54.015 DEBUG:boto:Params: {} 2022-09-03T06:27:54.016 DEBUG:boto:Token: None 2022-09-03T06:27:54.017 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:27:54 GMT /dsgvcy-74/obj00000000000000000002 2022-09-03T06:27:54.018 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r4rNGkYCiiMcaC099+GhUc3nRas= 2022-09-03T06:27:54.019 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:27:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r4rNGkYCiiMcaC099+GhUc3nRas='} 2022-09-03T06:27:54.063 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000e09362d2562fbf31-006312f3ea-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:27:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:54.065 DEBUG:boto:path=/dsgvcy-74/ 2022-09-03T06:27:54.065 DEBUG:boto:auth_path=/dsgvcy-74/ 2022-09-03T06:27:54.066 DEBUG:boto:Method: HEAD 2022-09-03T06:27:54.067 DEBUG:boto:Path: /dsgvcy-74/ 2022-09-03T06:27:54.068 DEBUG:boto:Data: 2022-09-03T06:27:54.069 DEBUG:boto:Headers: {} 2022-09-03T06:27:54.070 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:54.071 DEBUG:boto:Port: 8000 2022-09-03T06:27:54.071 DEBUG:boto:Params: {} 2022-09-03T06:27:54.072 DEBUG:boto:Token: None 2022-09-03T06:27:54.073 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:54 GMT /dsgvcy-74/ 2022-09-03T06:27:54.073 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IiYwm5ePR0WMxFbhYrPqQd9jHlA= 2022-09-03T06:27:54.074 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IiYwm5ePR0WMxFbhYrPqQd9jHlA=', 'Content-Length': '0'} 2022-09-03T06:27:54.077 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '9'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000068964b8ba41bec58-006312f3ea-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:27:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:54.079 DEBUG:boto:path=/dsgvcy-74/obj00000000000000000000 2022-09-03T06:27:54.080 DEBUG:boto:auth_path=/dsgvcy-74/obj00000000000000000000 2022-09-03T06:27:54.081 DEBUG:boto:Method: PUT 2022-09-03T06:27:54.081 DEBUG:boto:Path: /dsgvcy-74/obj00000000000000000000 2022-09-03T06:27:54.082 DEBUG:boto:Data: 2022-09-03T06:27:54.082 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:27:54.083 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:54.084 DEBUG:boto:Port: 8000 2022-09-03T06:27:54.084 DEBUG:boto:Params: {} 2022-09-03T06:27:54.085 DEBUG:boto:Token: None 2022-09-03T06:27:54.085 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:27:54 GMT /dsgvcy-74/obj00000000000000000000 2022-09-03T06:27:54.086 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/5cro9F4Er+qd/T7xkIppyX6UqI= 2022-09-03T06:27:54.086 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:27:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/5cro9F4Er+qd/T7xkIppyX6UqI='} 2022-09-03T06:27:54.093 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000e99892b619314c5a-006312f3ea-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:27:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:54.095 DEBUG:boto:path=/dsgvcy-74/ 2022-09-03T06:27:54.096 DEBUG:boto:auth_path=/dsgvcy-74/ 2022-09-03T06:27:54.096 DEBUG:boto:Method: HEAD 2022-09-03T06:27:54.097 DEBUG:boto:Path: /dsgvcy-74/ 2022-09-03T06:27:54.097 DEBUG:boto:Data: 2022-09-03T06:27:54.098 DEBUG:boto:Headers: {} 2022-09-03T06:27:54.099 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:54.099 DEBUG:boto:Port: 8000 2022-09-03T06:27:54.100 DEBUG:boto:Params: {} 2022-09-03T06:27:54.101 DEBUG:boto:Token: None 2022-09-03T06:27:54.101 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:54 GMT /dsgvcy-74/ 2022-09-03T06:27:54.102 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IiYwm5ePR0WMxFbhYrPqQd9jHlA= 2022-09-03T06:27:54.103 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IiYwm5ePR0WMxFbhYrPqQd9jHlA=', 'Content-Length': '0'} 2022-09-03T06:27:54.110 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f7c6a93ac809ec58-006312f3ea-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:27:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:54.111 DEBUG:boto:path=/dsgvcy-74/obj00000000000000000004 2022-09-03T06:27:54.112 DEBUG:boto:auth_path=/dsgvcy-74/obj00000000000000000004 2022-09-03T06:27:54.112 DEBUG:boto:Method: PUT 2022-09-03T06:27:54.113 DEBUG:boto:Path: /dsgvcy-74/obj00000000000000000004 2022-09-03T06:27:54.113 DEBUG:boto:Data: 2022-09-03T06:27:54.114 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:27:54.114 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:54.115 DEBUG:boto:Port: 8000 2022-09-03T06:27:54.115 DEBUG:boto:Params: {} 2022-09-03T06:27:54.116 DEBUG:boto:Token: None 2022-09-03T06:27:54.116 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:27:54 GMT /dsgvcy-74/obj00000000000000000004 2022-09-03T06:27:54.117 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C3NUScd/L+2GBj7j+2zNiVb+zMY= 2022-09-03T06:27:54.118 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:27:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C3NUScd/L+2GBj7j+2zNiVb+zMY='} 2022-09-03T06:27:54.168 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001c8bc9004843ba93-006312f3ea-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:27:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:54.168 DEBUG:boto:path=/dsgvcy-74/ 2022-09-03T06:27:54.169 DEBUG:boto:auth_path=/dsgvcy-74/ 2022-09-03T06:27:54.170 DEBUG:boto:Method: HEAD 2022-09-03T06:27:54.170 DEBUG:boto:Path: /dsgvcy-74/ 2022-09-03T06:27:54.171 DEBUG:boto:Data: 2022-09-03T06:27:54.171 DEBUG:boto:Headers: {} 2022-09-03T06:27:54.172 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:54.172 DEBUG:boto:Port: 8000 2022-09-03T06:27:54.173 DEBUG:boto:Params: {} 2022-09-03T06:27:54.175 DEBUG:boto:Token: None 2022-09-03T06:27:54.175 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:54 GMT /dsgvcy-74/ 2022-09-03T06:27:54.176 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IiYwm5ePR0WMxFbhYrPqQd9jHlA= 2022-09-03T06:27:54.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IiYwm5ePR0WMxFbhYrPqQd9jHlA=', 'Content-Length': '0'} 2022-09-03T06:27:54.179 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '15'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b1bafea3105dc79c-006312f3ea-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:27:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:54.180 DEBUG:boto:path=/dsgvcy-74/obj00000000000000000011 2022-09-03T06:27:54.181 DEBUG:boto:auth_path=/dsgvcy-74/obj00000000000000000011 2022-09-03T06:27:54.182 DEBUG:boto:Method: PUT 2022-09-03T06:27:54.183 DEBUG:boto:Path: /dsgvcy-74/obj00000000000000000011 2022-09-03T06:27:54.183 DEBUG:boto:Data: 2022-09-03T06:27:54.184 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:27:54.185 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:54.185 DEBUG:boto:Port: 8000 2022-09-03T06:27:54.186 DEBUG:boto:Params: {} 2022-09-03T06:27:54.186 DEBUG:boto:Token: None 2022-09-03T06:27:54.187 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:27:54 GMT /dsgvcy-74/obj00000000000000000011 2022-09-03T06:27:54.188 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fsRq9MVnagtjJ4uMwMZIJojtawc= 2022-09-03T06:27:54.190 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:27:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fsRq9MVnagtjJ4uMwMZIJojtawc='} 2022-09-03T06:27:54.197 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000640f43b0bc0e9165-006312f3ea-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:27:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:54.199 DEBUG:boto:path=/dsgvcy-74/ 2022-09-03T06:27:54.200 DEBUG:boto:auth_path=/dsgvcy-74/ 2022-09-03T06:27:54.201 DEBUG:boto:Method: HEAD 2022-09-03T06:27:54.201 DEBUG:boto:Path: /dsgvcy-74/ 2022-09-03T06:27:54.202 DEBUG:boto:Data: 2022-09-03T06:27:54.203 DEBUG:boto:Headers: {} 2022-09-03T06:27:54.204 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:54.205 DEBUG:boto:Port: 8000 2022-09-03T06:27:54.205 DEBUG:boto:Params: {} 2022-09-03T06:27:54.206 DEBUG:boto:Token: None 2022-09-03T06:27:54.207 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:54 GMT /dsgvcy-74/ 2022-09-03T06:27:54.208 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IiYwm5ePR0WMxFbhYrPqQd9jHlA= 2022-09-03T06:27:54.209 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IiYwm5ePR0WMxFbhYrPqQd9jHlA=', 'Content-Length': '0'} 2022-09-03T06:27:54.212 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b8ff2c130d534170-006312f3ea-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:27:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:54.214 DEBUG:boto:path=/dsgvcy-74/obj00000000000000000001 2022-09-03T06:27:54.215 DEBUG:boto:auth_path=/dsgvcy-74/obj00000000000000000001 2022-09-03T06:27:54.215 DEBUG:boto:Method: PUT 2022-09-03T06:27:54.216 DEBUG:boto:Path: /dsgvcy-74/obj00000000000000000001 2022-09-03T06:27:54.217 DEBUG:boto:Data: 2022-09-03T06:27:54.218 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:27:54.219 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:54.219 DEBUG:boto:Port: 8000 2022-09-03T06:27:54.220 DEBUG:boto:Params: {} 2022-09-03T06:27:54.221 DEBUG:boto:Token: None 2022-09-03T06:27:54.221 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:27:54 GMT /dsgvcy-74/obj00000000000000000001 2022-09-03T06:27:54.222 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p6NLu60uVtCZsHlVXkMplsK23fo= 2022-09-03T06:27:54.223 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:27:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p6NLu60uVtCZsHlVXkMplsK23fo='} 2022-09-03T06:27:54.274 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b70f4c5cbe8272d8-006312f3ea-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:27:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:54.276 DEBUG:boto:path=/dsgvcy-74/ 2022-09-03T06:27:54.277 DEBUG:boto:auth_path=/dsgvcy-74/ 2022-09-03T06:27:54.278 DEBUG:boto:Method: HEAD 2022-09-03T06:27:54.279 DEBUG:boto:Path: /dsgvcy-74/ 2022-09-03T06:27:54.280 DEBUG:boto:Data: 2022-09-03T06:27:54.283 DEBUG:boto:Headers: {} 2022-09-03T06:27:54.286 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:54.288 DEBUG:boto:Port: 8000 2022-09-03T06:27:54.288 DEBUG:boto:Params: {} 2022-09-03T06:27:54.289 DEBUG:boto:Token: None 2022-09-03T06:27:54.290 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:54 GMT /dsgvcy-74/ 2022-09-03T06:27:54.291 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IiYwm5ePR0WMxFbhYrPqQd9jHlA= 2022-09-03T06:27:54.291 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IiYwm5ePR0WMxFbhYrPqQd9jHlA=', 'Content-Length': '0'} 2022-09-03T06:27:54.295 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '21'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000725cc36586cf2963-006312f3ea-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:27:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:54.298 DEBUG:boto:path=/dsgvcy-74/obj00000000000000000013 2022-09-03T06:27:54.299 DEBUG:boto:auth_path=/dsgvcy-74/obj00000000000000000013 2022-09-03T06:27:54.300 DEBUG:boto:Method: PUT 2022-09-03T06:27:54.300 DEBUG:boto:Path: /dsgvcy-74/obj00000000000000000013 2022-09-03T06:27:54.301 DEBUG:boto:Data: 2022-09-03T06:27:54.302 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:27:54.302 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:54.303 DEBUG:boto:Port: 8000 2022-09-03T06:27:54.305 DEBUG:boto:Params: {} 2022-09-03T06:27:54.305 DEBUG:boto:Token: None 2022-09-03T06:27:54.307 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:27:54 GMT /dsgvcy-74/obj00000000000000000013 2022-09-03T06:27:54.308 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4ScyzjAVJHOCHO7vURLXSTWqfig= 2022-09-03T06:27:54.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:27:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4ScyzjAVJHOCHO7vURLXSTWqfig='} 2022-09-03T06:27:54.318 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001bb7273fbc077694-006312f3ea-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:27:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:54.319 DEBUG:boto:path=/dsgvcy-74/ 2022-09-03T06:27:54.320 DEBUG:boto:auth_path=/dsgvcy-74/ 2022-09-03T06:27:54.322 DEBUG:boto:Method: HEAD 2022-09-03T06:27:54.323 DEBUG:boto:Path: /dsgvcy-74/ 2022-09-03T06:27:54.324 DEBUG:boto:Data: 2022-09-03T06:27:54.324 DEBUG:boto:Headers: {} 2022-09-03T06:27:54.325 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:54.326 DEBUG:boto:Port: 8000 2022-09-03T06:27:54.327 DEBUG:boto:Params: {} 2022-09-03T06:27:54.329 DEBUG:boto:Token: None 2022-09-03T06:27:54.330 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:54 GMT /dsgvcy-74/ 2022-09-03T06:27:54.331 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IiYwm5ePR0WMxFbhYrPqQd9jHlA= 2022-09-03T06:27:54.332 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IiYwm5ePR0WMxFbhYrPqQd9jHlA=', 'Content-Length': '0'} 2022-09-03T06:27:54.334 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000bc9e7f2aa0e69218-006312f3ea-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:27:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:54.335 DEBUG:boto:path=/dsgvcy-74/obj00000000000000000006 2022-09-03T06:27:54.336 DEBUG:boto:auth_path=/dsgvcy-74/obj00000000000000000006 2022-09-03T06:27:54.337 DEBUG:boto:Method: PUT 2022-09-03T06:27:54.338 DEBUG:boto:Path: /dsgvcy-74/obj00000000000000000006 2022-09-03T06:27:54.340 DEBUG:boto:Data: 2022-09-03T06:27:54.341 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:27:54.342 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:54.343 DEBUG:boto:Port: 8000 2022-09-03T06:27:54.344 DEBUG:boto:Params: {} 2022-09-03T06:27:54.345 DEBUG:boto:Token: None 2022-09-03T06:27:54.345 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:27:54 GMT /dsgvcy-74/obj00000000000000000006 2022-09-03T06:27:54.346 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SwCsmvXQUtO75uW/5HG7kEgXFUY= 2022-09-03T06:27:54.347 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:27:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SwCsmvXQUtO75uW/5HG7kEgXFUY='} 2022-09-03T06:27:54.395 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000008e0ce14227004053-006312f3ea-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:27:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:54.396 DEBUG:boto:path=/dsgvcy-74/ 2022-09-03T06:27:54.397 DEBUG:boto:auth_path=/dsgvcy-74/ 2022-09-03T06:27:54.397 DEBUG:boto:Method: HEAD 2022-09-03T06:27:54.398 DEBUG:boto:Path: /dsgvcy-74/ 2022-09-03T06:27:54.398 DEBUG:boto:Data: 2022-09-03T06:27:54.399 DEBUG:boto:Headers: {} 2022-09-03T06:27:54.399 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:54.400 DEBUG:boto:Port: 8000 2022-09-03T06:27:54.400 DEBUG:boto:Params: {} 2022-09-03T06:27:54.401 DEBUG:boto:Token: None 2022-09-03T06:27:54.402 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:27:54 GMT /dsgvcy-74/ 2022-09-03T06:27:54.402 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IiYwm5ePR0WMxFbhYrPqQd9jHlA= 2022-09-03T06:27:54.403 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:27:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IiYwm5ePR0WMxFbhYrPqQd9jHlA=', 'Content-Length': '0'} 2022-09-03T06:27:54.406 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '9'), ('X-RGW-Bytes-Used', '27'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000474fe3cc7eaace5b-006312f3ea-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:27:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:54.407 DEBUG:boto:path=/dsgvcy-74/obj00000000000000000012 2022-09-03T06:27:54.407 DEBUG:boto:auth_path=/dsgvcy-74/obj00000000000000000012 2022-09-03T06:27:54.408 DEBUG:boto:Method: PUT 2022-09-03T06:27:54.408 DEBUG:boto:Path: /dsgvcy-74/obj00000000000000000012 2022-09-03T06:27:54.409 DEBUG:boto:Data: 2022-09-03T06:27:54.410 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:27:54.410 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:27:54.411 DEBUG:boto:Port: 8000 2022-09-03T06:27:54.412 DEBUG:boto:Params: {} 2022-09-03T06:27:54.412 DEBUG:boto:Token: None 2022-09-03T06:27:54.413 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:27:54 GMT /dsgvcy-74/obj00000000000000000012 2022-09-03T06:27:54.416 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zWWEp2IcFiy6wWlWSfaMeEYvv7o= 2022-09-03T06:27:54.417 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:27:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zWWEp2IcFiy6wWlWSfaMeEYvv7o='} 2022-09-03T06:27:54.426 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000060e902033f6511c-006312f3ea-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:27:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:27:54.427 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-74 2022-09-03T06:27:54.428 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-74', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:27:54.428 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-74', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:27:54.429 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-74 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:27:54.490 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:27:54.491 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:27:54.540 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:27:54.537+0000 7f07e4cbc600 1 waiting to reach incremental sync.. 2022-09-03T06:27:59.543 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:27:59.538+0000 7f07e4cbc600 1 waiting to reach incremental sync.. 2022-09-03T06:28:04.541 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:28:04.539+0000 7f07e4cbc600 1 waiting to reach incremental sync.. 2022-09-03T06:28:09.541 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:28:09.539+0000 7f07e4cbc600 1 waiting to reach incremental sync.. 2022-09-03T06:28:14.542 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:28:14.540+0000 7f07e4cbc600 1 waiting to reach incremental sync.. 2022-09-03T06:28:19.543 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:28:19.540+0000 7f07e4cbc600 1 waiting to reach incremental sync.. 2022-09-03T06:28:24.544 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:28:24.541+0000 7f07e4cbc600 1 waiting to reach incremental sync.. 2022-09-03T06:28:29.544 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:28:29.542+0000 7f07e4cbc600 1 waiting to reach incremental sync.. 2022-09-03T06:28:34.547 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:28:34.541+0000 7f07e4cbc600 1 waiting to reach incremental sync.. 2022-09-03T06:28:39.545 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:28:39.543+0000 7f07e4cbc600 1 waiting to reach incremental sync.. 2022-09-03T06:28:44.546 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:28:44.544+0000 7f07e4cbc600 1 waiting to reach incremental sync.. 2022-09-03T06:28:49.548 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:28:49.544+0000 7f07e4cbc600 1 waiting to reach incremental sync.. 2022-09-03T06:28:54.548 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:28:54.545+0000 7f07e4cbc600 1 waiting to reach incremental sync.. 2022-09-03T06:28:59.549 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:28:59.547+0000 7f07e4cbc600 1 bucket sync caught up with source: 2022-09-03T06:28:59.558 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:28:59.559 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:28:59.559 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:28:59.547+0000 7f07e4cbc600 0 bucket checkpoint complete 2022-09-03T06:28:59.574 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-74 2022-09-03T06:28:59.575 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-74', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:28:59.576 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-74', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:28:59.577 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-74 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:28:59.651 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:28:59.651 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:28:59.701 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:28:59.698+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:29:04.706 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:29:04.699+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:29:09.707 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:29:09.700+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:29:14.704 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:29:14.701+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:29:19.704 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:29:19.702+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:29:24.705 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:29:24.703+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:29:29.707 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:29:29.703+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:29:34.711 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:29:34.704+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:29:39.707 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:29:39.705+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:29:44.707 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:29:44.704+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:29:49.713 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:29:49.705+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:29:54.708 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:29:54.705+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:29:59.714 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:29:59.706+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:30:04.733 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:30:04.708+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:30:09.711 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:30:09.709+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:30:14.711 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:30:14.709+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:30:19.712 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:30:19.710+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:30:24.713 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:30:24.711+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:30:29.714 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:30:29.711+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:30:34.714 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:30:34.711+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:30:39.718 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:30:39.712+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:30:44.716 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:30:44.712+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:30:49.716 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:30:49.713+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:30:54.717 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:30:54.715+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:30:59.718 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:30:59.715+0000 7fb925b1c600 1 waiting to reach incremental sync.. 2022-09-03T06:31:04.727 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:31:04.717+0000 7fb925b1c600 1 bucket sync caught up with source: 2022-09-03T06:31:04.729 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:04.729 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:04.730 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:31:04.717+0000 7fb925b1c600 0 bucket checkpoint complete 2022-09-03T06:31:04.739 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-74 2022-09-03T06:31:04.742 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-74', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:31:04.743 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-74', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:31:04.745 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-74 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:31:04.809 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:31:04.810 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:31:04.857 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:31:04.844+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:31:04.858 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:04.858 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:09.849 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:31:09.845+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:31:09.851 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:09.852 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:14.855 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:31:14.848+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:31:14.855 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:14.856 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:19.853 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:31:19.849+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:31:19.853 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:19.853 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:24.856 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:31:24.850+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:31:24.857 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:24.858 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:29.855 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:31:29.852+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:31:29.855 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:29.856 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:34.860 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:31:34.854+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:31:34.860 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:34.861 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:39.858 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:31:39.856+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:31:39.859 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:39.859 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:44.859 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:31:44.857+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:31:44.860 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:44.861 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:49.861 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:31:49.858+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:31:49.862 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:49.863 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:54.862 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:31:54.860+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:31:54.862 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:54.863 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:59.867 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:31:59.860+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:31:59.868 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:31:59.869 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:04.865 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:32:04.863+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:32:04.866 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:04.867 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:09.867 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:32:09.864+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:32:09.868 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:09.869 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:14.877 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:32:14.865+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:32:14.877 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:14.878 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:19.869 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:32:19.867+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:32:19.870 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:19.871 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:24.870 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:32:24.868+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:32:24.871 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:24.871 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:29.872 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:32:29.870+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:32:29.873 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:29.873 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:34.874 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:32:34.871+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:32:34.875 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:34.875 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:39.874 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:32:39.872+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:32:39.875 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:39.875 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:44.876 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:32:44.874+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:32:44.876 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:44.877 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:49.878 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:32:49.874+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:32:49.879 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:49.879 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:54.880 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:32:54.877+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:32:54.881 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:54.882 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:59.882 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:32:59.878+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:32:59.883 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:32:59.883 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:04.885 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:33:04.879+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:33:04.886 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:04.887 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:09.884 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:33:09.881+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:33:09.885 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:09.885 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:14.885 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:33:14.882+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:33:14.885 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:14.891 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:19.887 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:33:19.884+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:33:19.888 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:19.889 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:24.903 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:33:24.885+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:33:24.904 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:24.905 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:29.891 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:33:29.887+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:33:29.892 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:29.892 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:34.900 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:33:34.888+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:33:34.901 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:34.903 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:39.892 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:33:39.889+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:33:39.893 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:39.894 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:44.896 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:33:44.891+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:33:44.897 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:44.898 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:49.898 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:33:49.893+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:33:49.899 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:49.899 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:54.897 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:33:54.894+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:33:54.898 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:54.899 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:59.901 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:33:59.896+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:33:59.902 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:33:59.902 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:04.899 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:34:04.897+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:34:04.899 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:04.900 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:09.904 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:34:09.898+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:34:09.904 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:09.905 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:14.902 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:34:14.900+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:34:14.904 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:14.904 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:19.907 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:34:19.901+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:34:19.908 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:19.909 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:24.910 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:34:24.903+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:34:24.910 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:24.911 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:29.922 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:34:29.903+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:34:29.922 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:29.923 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:34.908 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:34:34.904+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:34:34.908 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:34.909 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:39.909 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:34:39.907+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:34:39.910 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:39.910 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:44.911 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:34:44.908+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:34:44.911 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:44.912 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:49.912 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:34:49.909+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:34:49.913 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:49.913 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:54.921 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:34:54.911+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:34:54.921 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:54.922 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:59.919 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:34:59.912+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:34:59.920 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:34:59.921 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:35:04.916 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:35:04.914+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:35:04.917 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:35:04.922 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:35:09.918 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:35:09.915+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:35:09.918 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:35:09.919 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:35:14.919 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:35:14.917+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:35:14.920 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:35:14.921 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:35:19.921 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:35:19.918+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:35:19.921 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:35:19.922 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:35:24.922 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:35:24.919+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:35:24.923 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:35:24.924 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:35:29.923 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:35:29.921+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:35:29.924 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:35:29.924 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:35:34.925 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:35:34.923+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:35:34.926 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:35:34.926 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:35:39.926 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:35:39.924+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:35:39.927 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:35:39.927 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:35:44.928 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:35:44.925+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:35:44.929 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:35:44.929 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:35:49.929 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:35:49.927+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:35:49.930 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:35:49.930 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:35:54.931 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:35:54.928+0000 7fa0746c8600 1 waiting for incremental sync to catch up: 2022-09-03T06:35:54.932 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, , 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:35:54.933 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.47.5, , 00000000001.147.5, 00000000001.97.5, 00000000001.52.5, 00000000001.72.5, 00000000001.50.5, 00000000001.98.5, 00000000001.66.5, 00000000001.64.5, 00000000001.62.5] 2022-09-03T06:35:59.932 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:35:59.930+0000 7fa0746c8600 0 bucket checkpoint timed out waiting for incremental sync to catch up 2022-09-03T06:35:59.933 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:35:59.930+0000 7fa0746c8600 0 bucket sync checkpoint failed: (110) Connection timed out 2022-09-03T06:35:59.933 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:35:59.930+0000 7fa0746c8600 -1 bucket sync checkpoint failed: (110) Connection timed out 2022-09-03T06:35:59.950 DEBUG:teuthology.orchestra.run:got remote process result: 110 2022-09-03T06:35:59.959 INFO:tasks.rgw_multisite_tests:Create several generations of objects, then run bucket sync ... FAIL 2022-09-03T06:35:59.964 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:35:59.965 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:35:59.966 DEBUG:boto:Method: PUT 2022-09-03T06:35:59.967 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:35:59.968 DEBUG:boto:Data: 2022-09-03T06:35:59.969 DEBUG:boto:Headers: {} 2022-09-03T06:35:59.970 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:35:59.971 DEBUG:boto:Port: 8000 2022-09-03T06:35:59.971 DEBUG:boto:Params: {} 2022-09-03T06:35:59.973 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:35:59.974 DEBUG:boto:Token: None 2022-09-03T06:35:59.975 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 06:35:59 GMT /dsgvcy-75/ 2022-09-03T06:35:59.975 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i20rqDai4IS57hzgto8pbZ5lRn0= 2022-09-03T06:35:59.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:35:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i20rqDai4IS57hzgto8pbZ5lRn0=', 'Content-Length': '0'} 2022-09-03T06:36:00.010 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000009d02eea062f7887d-006312f5cf-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:36:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:36:00.011 DEBUG:rgw_multi.tests:created bucket=dsgvcy-75 2022-09-03T06:36:00.012 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:36:00.013 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:36:00.014 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:36:00.069 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:36:00.070 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:36:00.100 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T06:36:00.113 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T06:36:00.114 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:36:00.115 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:36:00.115 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:36:00.115 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:36:00.116 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:36:00.116 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:36:00.116 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:36:00.116 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:36:00.117 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:36:00.117 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662186960.001691_515.1", 2022-09-03T06:36:00.117 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:36:00.001691Z" 2022-09-03T06:36:00.118 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:36:00.118 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:36:00.118 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:36:00.119 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:36:00.119 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:36:00.119 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T06:36:00.121 DEBUG:rgw_multi.tests:master meta markers={0: '', 1: '', 2: '1_1662186960.001691_515.1', 3: ''} 2022-09-03T06:36:00.122 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T06:36:00.123 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:36:00.124 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:36:00.125 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:36:00.187 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:36:00.188 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:36:00.229 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:36:00.226+0000 7f2932b88600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:36:00.230 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:36:00.228+0000 7f2932b88600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:36:00.246 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:36:00.246 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:36:00.247 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:36:00.247 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:36:00.247 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:36:00.248 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:36:00.248 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:36:00.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:36:00.248 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:36:00.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:36:00.249 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:36:00.249 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:36:00.250 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:36:00.250 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:36:00.250 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:36:00.250 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:36:00.251 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:36:00.251 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:36:00.252 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:36:00.252 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:36:00.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:36:00.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:36:00.253 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:36:00.253 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:36:00.253 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:36:00.254 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:36:00.255 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:36:00.255 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:36:00.255 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:36:00.256 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:36:00.256 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:36:00.256 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:36:00.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:36:00.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:36:00.257 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:36:00.257 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:36:00.258 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:36:00.258 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662186960.001691_515.1", 2022-09-03T06:36:00.258 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:36:00.259 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:36:00.259 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:36:00.259 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:36:00.001691Z", 2022-09-03T06:36:00.260 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:36:00.260 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:36:00.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:36:00.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:36:00.261 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:36:00.261 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:36:00.261 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:36:00.262 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:36:00.262 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:36:00.262 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:36:00.262 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:36:00.263 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:36:00.263 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:36:00.263 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:36:00.263 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:36:00.264 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:36:00.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:36:00.264 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:36:00.265 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:36:00.265 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:36:00.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:36:00.265 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:36:00Z" 2022-09-03T06:36:00.266 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:36:00.267 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662186960.001691_515.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:36:00.001691Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:36:00Z" } 2022-09-03T06:36:00.268 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662186960.001691_515.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:36:00.001691Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:36:00.268 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662186960.001691_515.1', 3: ''} 2022-09-03T06:36:00.269 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662186960.001691_515.1', 3: ''} 2022-09-03T06:36:00.269 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T06:36:00.270 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T06:36:00.270 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:36:00.271 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:36:00.272 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:36:00.322 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:36:00.322 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:36:00.364 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:36:00.361+0000 7f8ce01d1600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:36:00.365 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:36:00.363+0000 7f8ce01d1600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:36:00.384 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T06:36:00.385 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T06:36:00.386 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T06:36:00.386 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T06:36:00.387 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T06:36:00.387 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:36:00.387 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:36:00.387 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:36:00.388 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T06:36:00.388 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:36:00.390 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T06:36:00.390 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:36:00.390 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:36:00.391 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:36:00.391 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:36:00.391 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T06:36:00.391 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:36:00.392 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:36:00.392 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:36:00.393 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:36:00.393 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:36:00.393 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:36:00.393 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T06:36:00.394 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:36:00.394 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:36:00.394 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:36:00.395 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:36:00.395 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:36:00.395 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:36:00.395 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "0.000000", 2022-09-03T06:36:00.396 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:36:00.396 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:36:00.396 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:36:00.396 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:36:00.397 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T06:36:00.397 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:36:00.397 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:36:00.397 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662186960.001691_515.1", 2022-09-03T06:36:00.398 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:36:00.398 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:36:00.398 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:36:00.398 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:36:00.001691Z", 2022-09-03T06:36:00.399 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:36:00.399 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:36:00.399 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:36:00.400 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:36:00.400 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T06:36:00.400 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:36:00.400 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:36:00.401 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:36:00.401 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:36:00.401 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:36:00.401 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:36:00.402 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:36:00.402 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:36:00.402 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:36:00.403 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:36:00.403 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:36:00.403 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:36:00.404 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T06:36:00.404 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T06:36:00.404 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T06:36:00.405 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:36:00.405 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T06:36:00Z" 2022-09-03T06:36:00.405 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T06:36:00.407 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662186960.001691_515.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:36:00.001691Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:36:00Z" } 2022-09-03T06:36:00.408 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662186960.001691_515.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:36:00.001691Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:36:00.408 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662186960.001691_515.1', 3: ''} 2022-09-03T06:36:00.409 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662186960.001691_515.1', 3: ''} 2022-09-03T06:36:00.409 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T06:36:00.410 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T06:36:00.410 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:36:00.411 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:36:00.412 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:36:00.464 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:36:00.465 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:36:00.499 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:36:00.494+0000 7f7e19f3c600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:36:00.499 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:36:00.496+0000 7f7e19f3c600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:36:00.509 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:36:00.510 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:36:00.510 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:36:00.510 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:36:00.511 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:36:00.511 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:36:00.511 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:36:00.511 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:36:00.512 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:36:00.512 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:36:00.513 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:36:00.513 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:36:00.513 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:36:00.513 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:36:00.514 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:36:00.514 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:36:00.514 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:36:00.515 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:36:00.515 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:36:00.515 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:36:00.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:36:00.516 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:36:00.516 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:36:00.516 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:36:00.516 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:36:00.517 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:36:00.517 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:36:00.517 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:36:00.517 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:36:00.518 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:36:00.518 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:36:00.518 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:36:00.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:36:00.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:36:00.519 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:36:00.519 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:36:00.520 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:36:00.520 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662186959.993897_512.1", 2022-09-03T06:36:00.520 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:36:00.520 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:36:00.521 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:36:00.521 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:35:59.993897Z", 2022-09-03T06:36:00.521 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:36:00.521 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:36:00.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:36:00.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:36:00.522 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:36:00.523 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:36:00.523 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:36:00.524 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:36:00.524 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:36:00.524 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:36:00.524 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:36:00.525 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:36:00.525 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:36:00.525 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:36:00.525 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:36:00.526 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:36:00.526 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:36:00.526 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:36:00.527 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:36:00.527 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:36:00.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:36:00.527 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:36:00Z" 2022-09-03T06:36:00.528 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:36:00.529 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662186959.993897_512.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:35:59.993897Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:36:00Z" } 2022-09-03T06:36:00.530 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662186959.993897_512.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:35:59.993897Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:36:00.530 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662186960.001691_515.1', 3: ''} 2022-09-03T06:36:00.531 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662186959.993897_512.1', 3: ''} 2022-09-03T06:36:00.531 WARNING:rgw_multi.tests:zone test-zone4 behind master: shard=2 master=1_1662186960.001691_515.1 target=1_1662186959.993897_512.1 2022-09-03T06:36:05.534 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:36:05.536 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:36:05.539 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:36:05.599 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:36:05.600 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:36:05.629 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:36:05.626+0000 7fe33c06c600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:36:05.629 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:36:05.628+0000 7fe33c06c600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:36:05.643 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:36:05.644 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:36:05.644 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:36:05.644 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:36:05.645 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:36:05.645 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:36:05.645 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:36:05.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:36:05.646 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:36:05.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:36:05.647 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:36:05.647 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:36:05.647 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:36:05.648 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:36:05.648 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:36:05.648 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:36:05.648 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:36:05.649 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:36:05.649 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:36:05.649 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:36:05.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:36:05.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:36:05.650 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:36:05.650 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:36:05.651 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:36:05.651 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:36:05.651 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:36:05.651 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:36:05.652 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:36:05.652 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:36:05.653 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:36:05.653 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:36:05.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:36:05.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:36:05.654 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:36:05.654 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:36:05.654 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:36:05.654 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662186960.001691_515.1", 2022-09-03T06:36:05.655 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:36:05.655 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:36:05.656 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:36:05.656 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:36:00.001691Z", 2022-09-03T06:36:05.657 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:36:05.657 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:36:05.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:36:05.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:36:05.658 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:36:05.658 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:36:05.658 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:36:05.659 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:36:05.659 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:36:05.659 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:36:05.659 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:36:05.660 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:36:05.660 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:36:05.660 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:36:05.661 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:36:05.661 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:36:05.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:36:05.661 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:36:05.662 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:36:05.662 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:36:05.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:36:05.662 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:36:05Z" 2022-09-03T06:36:05.663 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:36:05.664 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662186960.001691_515.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:36:00.001691Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:36:05Z" } 2022-09-03T06:36:05.665 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662186960.001691_515.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:36:00.001691Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:36:05.666 DEBUG:rgw_multi.tests:log_status={0: '', 1: '', 2: '1_1662186960.001691_515.1', 3: ''} 2022-09-03T06:36:05.667 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662186960.001691_515.1', 3: ''} 2022-09-03T06:36:05.668 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T06:36:05.668 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:36:05.669 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:36:05.669 DEBUG:boto:Method: HEAD 2022-09-03T06:36:05.670 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:36:05.671 DEBUG:boto:Data: 2022-09-03T06:36:05.671 DEBUG:boto:Headers: {} 2022-09-03T06:36:05.672 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:36:05.672 DEBUG:boto:Port: 8000 2022-09-03T06:36:05.673 DEBUG:boto:Params: {} 2022-09-03T06:36:05.674 DEBUG:boto:Token: None 2022-09-03T06:36:05.674 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:36:05 GMT /dsgvcy-75/ 2022-09-03T06:36:05.675 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mdUC4njLONDiP7Kgz9oJbMEj1N0= 2022-09-03T06:36:05.676 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:36:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mdUC4njLONDiP7Kgz9oJbMEj1N0=', 'Content-Length': '0'} 2022-09-03T06:36:05.679 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003becee5710746c8f-006312f5d5-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:36:05.680 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:36:05.680 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:36:05.681 DEBUG:boto:Method: PUT 2022-09-03T06:36:05.681 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:36:05.682 DEBUG:boto:Data: 2022-09-03T06:36:05.683 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:36:05.683 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:36:05.684 DEBUG:boto:Port: 8000 2022-09-03T06:36:05.684 DEBUG:boto:Params: {} 2022-09-03T06:36:05.685 DEBUG:boto:Token: None 2022-09-03T06:36:05.685 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:36:05 GMT /dsgvcy-75/obj11 2022-09-03T06:36:05.686 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A8HCdjilJp2zAjgmIZQ0w9P++Qw= 2022-09-03T06:36:05.686 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:36:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A8HCdjilJp2zAjgmIZQ0w9P++Qw='} 2022-09-03T06:36:05.695 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000006bb58fe38b1eb8fc-006312f5d5-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:36:05.696 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:36:05.697 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:36:05.698 DEBUG:boto:Method: HEAD 2022-09-03T06:36:05.699 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:36:05.700 DEBUG:boto:Data: 2022-09-03T06:36:05.701 DEBUG:boto:Headers: {} 2022-09-03T06:36:05.701 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:36:05.702 DEBUG:boto:Port: 8000 2022-09-03T06:36:05.703 DEBUG:boto:Params: {} 2022-09-03T06:36:05.704 DEBUG:boto:Token: None 2022-09-03T06:36:05.705 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:36:05 GMT /dsgvcy-75/ 2022-09-03T06:36:05.706 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mdUC4njLONDiP7Kgz9oJbMEj1N0= 2022-09-03T06:36:05.707 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:36:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mdUC4njLONDiP7Kgz9oJbMEj1N0=', 'Content-Length': '0'} 2022-09-03T06:36:05.710 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '3'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000bd9d74e3e2574597-006312f5d5-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:36:05.712 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:36:05.713 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:36:05.714 DEBUG:boto:Method: PUT 2022-09-03T06:36:05.715 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:36:05.715 DEBUG:boto:Data: 2022-09-03T06:36:05.716 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:36:05.717 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:36:05.718 DEBUG:boto:Port: 8000 2022-09-03T06:36:05.718 DEBUG:boto:Params: {} 2022-09-03T06:36:05.719 DEBUG:boto:Token: None 2022-09-03T06:36:05.720 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:36:05 GMT /dsgvcy-75/obj22 2022-09-03T06:36:05.721 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qs0ZiIvtHiTlfBrmfhLytN85IyY= 2022-09-03T06:36:05.722 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:36:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qs0ZiIvtHiTlfBrmfhLytN85IyY='} 2022-09-03T06:36:05.768 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001008db0a2aa5c0fc-006312f5d5-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:36:05.769 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:36:05.770 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:36:05.771 DEBUG:boto:Method: HEAD 2022-09-03T06:36:05.771 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:36:05.772 DEBUG:boto:Data: 2022-09-03T06:36:05.773 DEBUG:boto:Headers: {} 2022-09-03T06:36:05.774 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:36:05.774 DEBUG:boto:Port: 8000 2022-09-03T06:36:05.775 DEBUG:boto:Params: {} 2022-09-03T06:36:05.776 DEBUG:boto:Token: None 2022-09-03T06:36:05.778 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:36:05 GMT /dsgvcy-75/ 2022-09-03T06:36:05.779 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mdUC4njLONDiP7Kgz9oJbMEj1N0= 2022-09-03T06:36:05.780 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:36:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mdUC4njLONDiP7Kgz9oJbMEj1N0=', 'Content-Length': '0'} 2022-09-03T06:36:05.784 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f5cbea4a65d9a74b-006312f5d5-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:36:05.785 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:36:05.786 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:36:05.787 DEBUG:boto:Method: PUT 2022-09-03T06:36:05.788 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:36:05.789 DEBUG:boto:Data: 2022-09-03T06:36:05.790 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:36:05.791 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:36:05.791 DEBUG:boto:Port: 8000 2022-09-03T06:36:05.792 DEBUG:boto:Params: {} 2022-09-03T06:36:05.792 DEBUG:boto:Token: None 2022-09-03T06:36:05.793 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:36:05 GMT /dsgvcy-75/obj33 2022-09-03T06:36:05.794 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qAE4Kc9NKp75RYPJ98Sg5BUdB6k= 2022-09-03T06:36:05.794 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:36:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qAE4Kc9NKp75RYPJ98Sg5BUdB6k='} 2022-09-03T06:36:05.802 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000007a9820074861f27-006312f5d5-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:36:05.803 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:36:05.804 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:36:05.805 DEBUG:boto:Method: HEAD 2022-09-03T06:36:05.806 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:36:05.807 DEBUG:boto:Data: 2022-09-03T06:36:05.808 DEBUG:boto:Headers: {} 2022-09-03T06:36:05.809 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:36:05.810 DEBUG:boto:Port: 8000 2022-09-03T06:36:05.811 DEBUG:boto:Params: {} 2022-09-03T06:36:05.812 DEBUG:boto:Token: None 2022-09-03T06:36:05.813 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:36:05 GMT /dsgvcy-75/ 2022-09-03T06:36:05.814 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mdUC4njLONDiP7Kgz9oJbMEj1N0= 2022-09-03T06:36:05.815 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:36:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mdUC4njLONDiP7Kgz9oJbMEj1N0=', 'Content-Length': '0'} 2022-09-03T06:36:05.819 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '9'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a241bebb0b9d86ad-006312f5d5-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:36:05.820 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:36:05.821 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:36:05.822 DEBUG:boto:Method: PUT 2022-09-03T06:36:05.823 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:36:05.824 DEBUG:boto:Data: 2022-09-03T06:36:05.825 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:36:05.826 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:36:05.827 DEBUG:boto:Port: 8000 2022-09-03T06:36:05.828 DEBUG:boto:Params: {} 2022-09-03T06:36:05.829 DEBUG:boto:Token: None 2022-09-03T06:36:05.830 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:36:05 GMT /dsgvcy-75/obj44 2022-09-03T06:36:05.831 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0D9HCqzN1O7ALqryarYUGNiPzzo= 2022-09-03T06:36:05.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:36:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0D9HCqzN1O7ALqryarYUGNiPzzo='} 2022-09-03T06:36:05.844 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000585268b333677f94-006312f5d5-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:36:05.847 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:36:05.847 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:36:05.848 DEBUG:boto:Method: HEAD 2022-09-03T06:36:05.848 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:36:05.849 DEBUG:boto:Data: 2022-09-03T06:36:05.850 DEBUG:boto:Headers: {} 2022-09-03T06:36:05.850 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:36:05.851 DEBUG:boto:Port: 8000 2022-09-03T06:36:05.851 DEBUG:boto:Params: {} 2022-09-03T06:36:05.852 DEBUG:boto:Token: None 2022-09-03T06:36:05.852 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:36:05 GMT /dsgvcy-75/ 2022-09-03T06:36:05.853 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mdUC4njLONDiP7Kgz9oJbMEj1N0= 2022-09-03T06:36:05.854 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:36:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mdUC4njLONDiP7Kgz9oJbMEj1N0=', 'Content-Length': '0'} 2022-09-03T06:36:05.857 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d0d9a54fbe851fd7-006312f5d5-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:36:05.858 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:36:05.859 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:36:05.860 DEBUG:boto:Method: PUT 2022-09-03T06:36:05.860 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:36:05.861 DEBUG:boto:Data: 2022-09-03T06:36:05.861 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:36:05.862 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:36:05.862 DEBUG:boto:Port: 8000 2022-09-03T06:36:05.863 DEBUG:boto:Params: {} 2022-09-03T06:36:05.864 DEBUG:boto:Token: None 2022-09-03T06:36:05.864 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:36:05 GMT /dsgvcy-75/obj55 2022-09-03T06:36:05.865 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4rYRrd4B35baCV3dQ2KqwvFbRGs= 2022-09-03T06:36:05.865 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:36:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4rYRrd4B35baCV3dQ2KqwvFbRGs='} 2022-09-03T06:36:05.919 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005e5ec3b6c58c9f4b-006312f5d5-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:36:05.920 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-75 2022-09-03T06:36:05.921 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:36:05.922 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:36:05.923 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:36:05.986 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:36:05.987 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:36:06.021 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:36:06.018+0000 7f910618f600 1 waiting to reach incremental sync.. 2022-09-03T06:36:11.021 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:36:11.019+0000 7f910618f600 1 waiting to reach incremental sync.. 2022-09-03T06:36:16.022 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:36:16.018+0000 7f910618f600 1 waiting to reach incremental sync.. 2022-09-03T06:36:21.025 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:36:21.021+0000 7f910618f600 1 bucket sync caught up with source: 2022-09-03T06:36:21.026 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.62.5, 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:36:21.026 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000001.62.5, 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:36:21.027 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:36:21.021+0000 7f910618f600 0 bucket checkpoint complete 2022-09-03T06:36:21.043 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-75 2022-09-03T06:36:21.044 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:36:21.045 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:36:21.046 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:36:21.125 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:36:21.126 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:36:21.157 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:36:21.153+0000 7f15e40a6600 1 waiting to reach incremental sync.. 2022-09-03T06:36:26.169 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:36:26.155+0000 7f15e40a6600 1 bucket sync caught up with source: 2022-09-03T06:36:26.170 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000001.62.5, 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:36:26.170 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000001.62.5, 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:36:26.172 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:36:26.155+0000 7f15e40a6600 0 bucket checkpoint complete 2022-09-03T06:36:26.185 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-75 2022-09-03T06:36:26.187 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:36:26.188 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:36:26.189 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:36:26.273 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:36:26.273 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:36:26.326 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:36:26.316+0000 7f96240d5600 1 waiting for incremental sync to catch up: 2022-09-03T06:36:26.327 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:36:26.328 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000001.62.5, 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:36:31.321 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:36:31.318+0000 7f96240d5600 1 waiting for incremental sync to catch up: 2022-09-03T06:36:31.322 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:36:31.322 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000001.62.5, 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:36:36.322 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:36:36.319+0000 7f96240d5600 1 waiting for incremental sync to catch up: 2022-09-03T06:36:36.323 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:36:36.323 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000001.62.5, 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:36:41.323 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:36:41.321+0000 7f96240d5600 1 waiting for incremental sync to catch up: 2022-09-03T06:36:41.324 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:36:41.324 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000001.62.5, 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:36:46.328 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:36:46.322+0000 7f96240d5600 1 waiting for incremental sync to catch up: 2022-09-03T06:36:46.329 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:36:46.329 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000001.62.5, 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:36:51.331 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:36:51.324+0000 7f96240d5600 1 waiting for incremental sync to catch up: 2022-09-03T06:36:51.332 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:36:51.332 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000001.62.5, 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:36:56.328 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:36:56.326+0000 7f96240d5600 1 waiting for incremental sync to catch up: 2022-09-03T06:36:56.328 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:36:56.329 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000001.62.5, 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:37:01.330 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:37:01.327+0000 7f96240d5600 1 waiting for incremental sync to catch up: 2022-09-03T06:37:01.331 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:37:01.332 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000001.62.5, 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:37:06.331 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:37:06.327+0000 7f96240d5600 1 waiting for incremental sync to catch up: 2022-09-03T06:37:06.332 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:37:06.332 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000001.62.5, 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:37:11.332 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:37:11.330+0000 7f96240d5600 1 waiting for incremental sync to catch up: 2022-09-03T06:37:11.333 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:37:11.333 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000001.62.5, 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:37:16.336 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:37:16.331+0000 7f96240d5600 1 waiting for incremental sync to catch up: 2022-09-03T06:37:16.336 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:37:16.337 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000001.62.5, 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:37:21.335 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:37:21.332+0000 7f96240d5600 1 bucket sync caught up with source: 2022-09-03T06:37:21.338 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.62.5, 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:37:21.339 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000001.62.5, 00000000002.55.5, 00000000002.67.5] 2022-09-03T06:37:21.339 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:37:21.332+0000 7f96240d5600 0 bucket checkpoint complete 2022-09-03T06:37:21.353 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-75 2022-09-03T06:37:21.355 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:37:21.355 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:37:21.356 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:37:21.420 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:37:21.421 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:37:21.478 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:37:21.475+0000 7fe77f0d5600 1 bucket sync caught up with source: 2022-09-03T06:37:21.479 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000001.841.5, 00000000002.298.5, 00000000002.575.5] 2022-09-03T06:37:21.479 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000001.841.5, 00000000002.298.5, 00000000002.575.5] 2022-09-03T06:37:21.479 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:37:21.475+0000 7fe77f0d5600 0 bucket checkpoint complete 2022-09-03T06:37:21.496 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-75 2022-09-03T06:37:21.497 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:37:21.498 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:37:21.499 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:37:21.555 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:37:21.556 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:37:21.606 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:37:21.603+0000 7fb714761600 1 bucket sync caught up with source: 2022-09-03T06:37:21.607 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000001.841.5, 00000000002.298.5, 00000000002.575.5] 2022-09-03T06:37:21.607 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000001.841.5, 00000000002.298.5, 00000000002.575.5] 2022-09-03T06:37:21.607 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:37:21.603+0000 7fb714761600 0 bucket checkpoint complete 2022-09-03T06:37:21.620 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-75 2022-09-03T06:37:21.620 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-75 2022-09-03T06:37:21.621 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:37:21.621 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:37:21.622 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:37:21.679 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:37:21.679 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:37:21.718 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:37:21.713+0000 7f8280f78600 1 bucket sync caught up with source: 2022-09-03T06:37:21.719 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , 00000000001.307.5, 00000000002.235.5, 00000000002.280.5] 2022-09-03T06:37:21.720 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , , , 00000000001.307.5, 00000000002.235.5, 00000000002.280.5] 2022-09-03T06:37:21.720 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:37:21.713+0000 7f8280f78600 0 bucket checkpoint complete 2022-09-03T06:37:21.734 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-75 2022-09-03T06:37:21.735 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:37:21.736 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:37:21.737 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:37:21.800 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:37:21.801 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:37:21.842 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:37:21.837+0000 7f3f7f1c0600 1 bucket sync caught up with source: 2022-09-03T06:37:21.842 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , 00000000001.307.5, 00000000002.235.5, 00000000002.280.5] 2022-09-03T06:37:21.843 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , , , 00000000001.307.5, 00000000002.235.5, 00000000002.280.5] 2022-09-03T06:37:21.843 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:37:21.837+0000 7f3f7f1c0600 0 bucket checkpoint complete 2022-09-03T06:37:21.854 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-75 2022-09-03T06:37:21.855 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-75 zones={test-zone2, test-zone1} 2022-09-03T06:37:21.856 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:21.859 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:21.861 DEBUG:boto:Method: HEAD 2022-09-03T06:37:21.862 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:37:21.862 DEBUG:boto:Data: 2022-09-03T06:37:21.863 DEBUG:boto:Headers: {} 2022-09-03T06:37:21.864 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:21.864 DEBUG:boto:Port: 8000 2022-09-03T06:37:21.865 DEBUG:boto:Params: {} 2022-09-03T06:37:21.866 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:37:21.868 DEBUG:boto:Token: None 2022-09-03T06:37:21.873 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:21 GMT /dsgvcy-75/ 2022-09-03T06:37:21.874 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IfodS9Jd8KUeSZs428LIjvpjkDQ= 2022-09-03T06:37:21.875 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IfodS9Jd8KUeSZs428LIjvpjkDQ=', 'Content-Length': '0'} 2022-09-03T06:37:21.880 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '15'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009b9197a878b37b52-006312f621-188c-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:21.880 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:21.881 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:21.884 DEBUG:boto:Method: HEAD 2022-09-03T06:37:21.885 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:37:21.886 DEBUG:boto:Data: 2022-09-03T06:37:21.887 DEBUG:boto:Headers: {} 2022-09-03T06:37:21.889 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:21.890 DEBUG:boto:Port: 8000 2022-09-03T06:37:21.891 DEBUG:boto:Params: {} 2022-09-03T06:37:21.894 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:37:21.895 DEBUG:boto:Token: None 2022-09-03T06:37:21.897 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:21 GMT /dsgvcy-75/ 2022-09-03T06:37:21.897 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IfodS9Jd8KUeSZs428LIjvpjkDQ= 2022-09-03T06:37:21.898 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IfodS9Jd8KUeSZs428LIjvpjkDQ=', 'Content-Length': '0'} 2022-09-03T06:37:21.902 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '15'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009a97ad42cd83c87b-006312f621-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:21.903 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:37:21.905 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:21.906 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:21.907 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:21.908 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:21.909 DEBUG:boto:Method: GET 2022-09-03T06:37:21.910 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:21.911 DEBUG:boto:Data: 2022-09-03T06:37:21.912 DEBUG:boto:Headers: {} 2022-09-03T06:37:21.913 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:21.915 DEBUG:boto:Port: 8000 2022-09-03T06:37:21.916 DEBUG:boto:Params: {} 2022-09-03T06:37:21.917 DEBUG:boto:Token: None 2022-09-03T06:37:21.919 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:21 GMT /dsgvcy-75/?versions 2022-09-03T06:37:21.920 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZJpJt22h+W9ndOcMfBlcoDBALd8= 2022-09-03T06:37:21.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZJpJt22h+W9ndOcMfBlcoDBALd8=', 'Content-Length': '0'} 2022-09-03T06:37:21.925 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ab0196263e4d9594-006312f621-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:21.927 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:21.930 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:37:21.932 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:37:21.934 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:37:21.936 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:37:21.936 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:37:21.937 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:37:21.938 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:21.939 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:21.940 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:21.941 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:21.941 DEBUG:boto:Method: GET 2022-09-03T06:37:21.942 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:21.943 DEBUG:boto:Data: 2022-09-03T06:37:21.945 DEBUG:boto:Headers: {} 2022-09-03T06:37:21.946 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:21.947 DEBUG:boto:Port: 8000 2022-09-03T06:37:21.947 DEBUG:boto:Params: {} 2022-09-03T06:37:21.948 DEBUG:boto:Token: None 2022-09-03T06:37:21.949 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:21 GMT /dsgvcy-75/?versions 2022-09-03T06:37:21.950 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZJpJt22h+W9ndOcMfBlcoDBALd8= 2022-09-03T06:37:21.951 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZJpJt22h+W9ndOcMfBlcoDBALd8=', 'Content-Length': '0'} 2022-09-03T06:37:21.956 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dc7026b39c3cbea2-006312f621-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:21.958 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:21.960 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:37:21.961 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:37:21.962 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:37:21.963 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:37:21.963 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:37:21.964 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:21.965 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:21.966 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:21.967 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:21.968 DEBUG:boto:Method: GET 2022-09-03T06:37:21.969 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:21.970 DEBUG:boto:Data: 2022-09-03T06:37:21.971 DEBUG:boto:Headers: {} 2022-09-03T06:37:21.972 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:21.973 DEBUG:boto:Port: 8000 2022-09-03T06:37:21.974 DEBUG:boto:Params: {} 2022-09-03T06:37:21.975 DEBUG:boto:Token: None 2022-09-03T06:37:21.976 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:21 GMT /dsgvcy-75/?versions 2022-09-03T06:37:21.977 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZJpJt22h+W9ndOcMfBlcoDBALd8= 2022-09-03T06:37:21.978 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZJpJt22h+W9ndOcMfBlcoDBALd8=', 'Content-Length': '0'} 2022-09-03T06:37:21.982 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002c514aee68d86532-006312f621-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:21.983 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:21.985 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:21.986 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:21.987 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:21.987 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:21.989 DEBUG:boto:Method: GET 2022-09-03T06:37:21.990 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:21.991 DEBUG:boto:Data: 2022-09-03T06:37:21.992 DEBUG:boto:Headers: {} 2022-09-03T06:37:21.993 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:21.994 DEBUG:boto:Port: 8000 2022-09-03T06:37:21.995 DEBUG:boto:Params: {} 2022-09-03T06:37:21.996 DEBUG:boto:Token: None 2022-09-03T06:37:21.999 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:21 GMT /dsgvcy-75/?versions 2022-09-03T06:37:22.000 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZJpJt22h+W9ndOcMfBlcoDBALd8= 2022-09-03T06:37:22.001 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZJpJt22h+W9ndOcMfBlcoDBALd8=', 'Content-Length': '0'} 2022-09-03T06:37:22.003 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000d2711694b34127d-006312f622-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.004 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:22.006 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:37:22.008 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:22.009 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:22.010 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.012 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.013 DEBUG:boto:Method: GET 2022-09-03T06:37:22.015 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.015 DEBUG:boto:Data: 2022-09-03T06:37:22.016 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.017 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.018 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.019 DEBUG:boto:Params: {} 2022-09-03T06:37:22.019 DEBUG:boto:Token: None 2022-09-03T06:37:22.020 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.021 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zxb3IxFKsDBtny1iyAnaPAbYoJk= 2022-09-03T06:37:22.021 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zxb3IxFKsDBtny1iyAnaPAbYoJk=', 'Content-Length': '0'} 2022-09-03T06:37:22.024 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad143d606344de1e-006312f622-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.025 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:22.025 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:22.026 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.027 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.028 DEBUG:boto:Method: GET 2022-09-03T06:37:22.028 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.029 DEBUG:boto:Data: 2022-09-03T06:37:22.029 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.030 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.031 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.031 DEBUG:boto:Params: {} 2022-09-03T06:37:22.032 DEBUG:boto:Token: None 2022-09-03T06:37:22.032 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.033 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zxb3IxFKsDBtny1iyAnaPAbYoJk= 2022-09-03T06:37:22.034 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zxb3IxFKsDBtny1iyAnaPAbYoJk=', 'Content-Length': '0'} 2022-09-03T06:37:22.036 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002779a124269eac31-006312f622-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.037 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:22.037 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:22.038 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.038 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.039 DEBUG:boto:Method: HEAD 2022-09-03T06:37:22.039 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.040 DEBUG:boto:Data: 2022-09-03T06:37:22.041 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.041 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.042 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.042 DEBUG:boto:Params: {} 2022-09-03T06:37:22.043 DEBUG:boto:Token: None 2022-09-03T06:37:22.043 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.044 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/lS84M+6Y1lCp7PLnsyLnlNabok= 2022-09-03T06:37:22.045 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/lS84M+6Y1lCp7PLnsyLnlNabok=', 'Content-Length': '0'} 2022-09-03T06:37:22.047 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d3bf5fe42366cd97-006312f622-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.047 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:22.048 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:22.048 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.049 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.050 DEBUG:boto:Method: HEAD 2022-09-03T06:37:22.050 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.051 DEBUG:boto:Data: 2022-09-03T06:37:22.051 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.052 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.052 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.053 DEBUG:boto:Params: {} 2022-09-03T06:37:22.053 DEBUG:boto:Token: None 2022-09-03T06:37:22.054 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.055 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/lS84M+6Y1lCp7PLnsyLnlNabok= 2022-09-03T06:37:22.055 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/lS84M+6Y1lCp7PLnsyLnlNabok=', 'Content-Length': '0'} 2022-09-03T06:37:22.057 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007eb667067773e598-006312f622-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.058 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:37:22.059 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:22.059 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:22.060 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.061 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.061 DEBUG:boto:Method: GET 2022-09-03T06:37:22.062 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.062 DEBUG:boto:Data: 2022-09-03T06:37:22.063 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.064 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.064 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.065 DEBUG:boto:Params: {} 2022-09-03T06:37:22.066 DEBUG:boto:Token: None 2022-09-03T06:37:22.066 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.067 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zxb3IxFKsDBtny1iyAnaPAbYoJk= 2022-09-03T06:37:22.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zxb3IxFKsDBtny1iyAnaPAbYoJk=', 'Content-Length': '0'} 2022-09-03T06:37:22.070 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d342e4b7cc63ae60-006312f622-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.071 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:22.071 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:22.072 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.072 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.073 DEBUG:boto:Method: GET 2022-09-03T06:37:22.073 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.074 DEBUG:boto:Data: 2022-09-03T06:37:22.075 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.075 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.076 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.076 DEBUG:boto:Params: {} 2022-09-03T06:37:22.077 DEBUG:boto:Token: None 2022-09-03T06:37:22.077 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:22.078 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zxb3IxFKsDBtny1iyAnaPAbYoJk= 2022-09-03T06:37:22.079 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zxb3IxFKsDBtny1iyAnaPAbYoJk=', 'Content-Length': '0'} 2022-09-03T06:37:22.081 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002efe01b0904a8204-006312f622-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.082 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:22.082 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:22.083 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:22.083 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:22.084 DEBUG:boto:Method: GET 2022-09-03T06:37:22.085 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:37:22.085 DEBUG:boto:Data: 2022-09-03T06:37:22.086 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.086 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.087 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.088 DEBUG:boto:Params: {} 2022-09-03T06:37:22.088 DEBUG:boto:Token: None 2022-09-03T06:37:22.089 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/ 2022-09-03T06:37:22.090 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g= 2022-09-03T06:37:22.090 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g=', 'Content-Length': '0'} 2022-09-03T06:37:22.093 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e3f42b70d6d2dbf9-006312f622-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.093 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:22.094 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:22.095 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:22.095 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:22.096 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:22.097 DEBUG:boto:Method: GET 2022-09-03T06:37:22.097 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:37:22.098 DEBUG:boto:Data: 2022-09-03T06:37:22.098 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.099 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.099 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.100 DEBUG:boto:Params: {} 2022-09-03T06:37:22.100 DEBUG:boto:Token: None 2022-09-03T06:37:22.101 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/ 2022-09-03T06:37:22.102 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g= 2022-09-03T06:37:22.102 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g=', 'Content-Length': '0'} 2022-09-03T06:37:22.105 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d721ae5cebf96b0a-006312f622-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.105 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:22.106 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:22.107 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:22.118 DEBUG:boto:Method: HEAD 2022-09-03T06:37:22.119 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:37:22.122 DEBUG:boto:Data: 2022-09-03T06:37:22.122 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.123 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.124 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.124 DEBUG:boto:Params: {} 2022-09-03T06:37:22.125 DEBUG:boto:Token: None 2022-09-03T06:37:22.130 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj11 2022-09-03T06:37:22.131 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:skxUNCOrEVv7SVBUgnqLzrrG8fc= 2022-09-03T06:37:22.132 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:skxUNCOrEVv7SVBUgnqLzrrG8fc=', 'Content-Length': '0'} 2022-09-03T06:37:22.134 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f7ff40b34de85ee7-006312f622-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.136 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:22.137 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:22.138 DEBUG:boto:Method: HEAD 2022-09-03T06:37:22.139 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:37:22.140 DEBUG:boto:Data: 2022-09-03T06:37:22.141 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.142 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.143 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.144 DEBUG:boto:Params: {} 2022-09-03T06:37:22.145 DEBUG:boto:Token: None 2022-09-03T06:37:22.146 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj11 2022-09-03T06:37:22.147 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:skxUNCOrEVv7SVBUgnqLzrrG8fc= 2022-09-03T06:37:22.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:skxUNCOrEVv7SVBUgnqLzrrG8fc=', 'Content-Length': '0'} 2022-09-03T06:37:22.150 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008cc559e22e0cfd43-006312f622-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.151 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:37:22.151 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:22.152 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:22.153 DEBUG:boto:Method: GET 2022-09-03T06:37:22.153 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:37:22.154 DEBUG:boto:Data: 2022-09-03T06:37:22.155 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.155 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.156 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.157 DEBUG:boto:Params: {} 2022-09-03T06:37:22.158 DEBUG:boto:Token: None 2022-09-03T06:37:22.159 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj11 2022-09-03T06:37:22.160 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M2o5vpYHiRf5IEf6yARlTYCjjws= 2022-09-03T06:37:22.162 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M2o5vpYHiRf5IEf6yARlTYCjjws=', 'Content-Length': '0'} 2022-09-03T06:37:22.164 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000043bda860a1fd5979-006312f622-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.210 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:22.211 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:22.211 DEBUG:boto:Method: GET 2022-09-03T06:37:22.212 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:37:22.213 DEBUG:boto:Data: 2022-09-03T06:37:22.213 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.214 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.215 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.215 DEBUG:boto:Params: {} 2022-09-03T06:37:22.216 DEBUG:boto:Token: None 2022-09-03T06:37:22.217 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj11 2022-09-03T06:37:22.217 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M2o5vpYHiRf5IEf6yARlTYCjjws= 2022-09-03T06:37:22.218 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M2o5vpYHiRf5IEf6yARlTYCjjws=', 'Content-Length': '0'} 2022-09-03T06:37:22.220 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c5a658474148488-006312f622-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.221 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:22.222 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:22.222 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:22.223 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:22.223 DEBUG:boto:Method: GET 2022-09-03T06:37:22.224 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:37:22.225 DEBUG:boto:Data: 2022-09-03T06:37:22.226 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.226 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.227 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.228 DEBUG:boto:Params: {} 2022-09-03T06:37:22.228 DEBUG:boto:Token: None 2022-09-03T06:37:22.229 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/ 2022-09-03T06:37:22.230 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g= 2022-09-03T06:37:22.230 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g=', 'Content-Length': '0'} 2022-09-03T06:37:22.233 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006ef0f2535da880ad-006312f622-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.234 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:22.235 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:22.236 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:22.236 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:22.237 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:22.237 DEBUG:boto:Method: GET 2022-09-03T06:37:22.238 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:37:22.238 DEBUG:boto:Data: 2022-09-03T06:37:22.239 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.240 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.240 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.241 DEBUG:boto:Params: {} 2022-09-03T06:37:22.242 DEBUG:boto:Token: None 2022-09-03T06:37:22.242 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/ 2022-09-03T06:37:22.243 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g= 2022-09-03T06:37:22.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g=', 'Content-Length': '0'} 2022-09-03T06:37:22.247 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c943f3eff37bb24f-006312f622-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.248 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:22.250 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:37:22.251 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:22.252 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:22.253 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.254 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.255 DEBUG:boto:Method: GET 2022-09-03T06:37:22.256 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.257 DEBUG:boto:Data: 2022-09-03T06:37:22.257 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.258 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.258 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.259 DEBUG:boto:Params: {} 2022-09-03T06:37:22.260 DEBUG:boto:Token: None 2022-09-03T06:37:22.260 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.261 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h3ffPeFzCr7e419n+eY36pFREws= 2022-09-03T06:37:22.262 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h3ffPeFzCr7e419n+eY36pFREws=', 'Content-Length': '0'} 2022-09-03T06:37:22.264 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fbc640e8ba89d2be-006312f622-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.265 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:22.266 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:22.267 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.267 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.270 DEBUG:boto:Method: GET 2022-09-03T06:37:22.271 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.272 DEBUG:boto:Data: 2022-09-03T06:37:22.273 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.274 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.275 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.276 DEBUG:boto:Params: {} 2022-09-03T06:37:22.277 DEBUG:boto:Token: None 2022-09-03T06:37:22.278 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.279 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h3ffPeFzCr7e419n+eY36pFREws= 2022-09-03T06:37:22.281 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h3ffPeFzCr7e419n+eY36pFREws=', 'Content-Length': '0'} 2022-09-03T06:37:22.285 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000083518266ab4c775e-006312f622-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.286 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:22.287 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:22.287 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.288 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.289 DEBUG:boto:Method: HEAD 2022-09-03T06:37:22.289 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.290 DEBUG:boto:Data: 2022-09-03T06:37:22.290 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.291 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.291 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.292 DEBUG:boto:Params: {} 2022-09-03T06:37:22.292 DEBUG:boto:Token: None 2022-09-03T06:37:22.293 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.294 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5QJynitZRO0HUYT5Tg7Eb3MmGb8= 2022-09-03T06:37:22.294 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5QJynitZRO0HUYT5Tg7Eb3MmGb8=', 'Content-Length': '0'} 2022-09-03T06:37:22.297 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000f6f6c75fab60d97-006312f622-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.298 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:22.299 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:22.300 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.301 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.301 DEBUG:boto:Method: HEAD 2022-09-03T06:37:22.302 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.302 DEBUG:boto:Data: 2022-09-03T06:37:22.303 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.304 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.305 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.305 DEBUG:boto:Params: {} 2022-09-03T06:37:22.306 DEBUG:boto:Token: None 2022-09-03T06:37:22.306 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.307 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5QJynitZRO0HUYT5Tg7Eb3MmGb8= 2022-09-03T06:37:22.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5QJynitZRO0HUYT5Tg7Eb3MmGb8=', 'Content-Length': '0'} 2022-09-03T06:37:22.309 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a7a955051d76db9c-006312f622-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.310 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:37:22.311 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:22.311 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:22.312 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.312 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.313 DEBUG:boto:Method: GET 2022-09-03T06:37:22.314 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.314 DEBUG:boto:Data: 2022-09-03T06:37:22.315 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.315 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.320 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.320 DEBUG:boto:Params: {} 2022-09-03T06:37:22.323 DEBUG:boto:Token: None 2022-09-03T06:37:22.324 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.327 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h3ffPeFzCr7e419n+eY36pFREws= 2022-09-03T06:37:22.327 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h3ffPeFzCr7e419n+eY36pFREws=', 'Content-Length': '0'} 2022-09-03T06:37:22.330 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000145a043d723108cd-006312f622-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.331 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:22.331 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:22.332 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.332 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.333 DEBUG:boto:Method: GET 2022-09-03T06:37:22.333 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.334 DEBUG:boto:Data: 2022-09-03T06:37:22.335 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.335 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.336 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.336 DEBUG:boto:Params: {} 2022-09-03T06:37:22.337 DEBUG:boto:Token: None 2022-09-03T06:37:22.338 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:22.338 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h3ffPeFzCr7e419n+eY36pFREws= 2022-09-03T06:37:22.339 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h3ffPeFzCr7e419n+eY36pFREws=', 'Content-Length': '0'} 2022-09-03T06:37:22.341 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c6e993f9064cdc9-006312f622-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.342 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:22.343 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:22.343 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:22.344 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:22.350 DEBUG:boto:Method: GET 2022-09-03T06:37:22.351 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:37:22.351 DEBUG:boto:Data: 2022-09-03T06:37:22.352 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.353 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.353 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.354 DEBUG:boto:Params: {} 2022-09-03T06:37:22.354 DEBUG:boto:Token: None 2022-09-03T06:37:22.355 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/ 2022-09-03T06:37:22.356 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g= 2022-09-03T06:37:22.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g=', 'Content-Length': '0'} 2022-09-03T06:37:22.366 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000674c408be113bd05-006312f622-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.367 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:22.368 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:22.370 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:22.370 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:22.371 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:22.371 DEBUG:boto:Method: GET 2022-09-03T06:37:22.372 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:37:22.372 DEBUG:boto:Data: 2022-09-03T06:37:22.373 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.374 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.374 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.375 DEBUG:boto:Params: {} 2022-09-03T06:37:22.375 DEBUG:boto:Token: None 2022-09-03T06:37:22.376 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/ 2022-09-03T06:37:22.377 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g= 2022-09-03T06:37:22.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g=', 'Content-Length': '0'} 2022-09-03T06:37:22.380 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009fe83e8c9126ab53-006312f622-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.381 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:22.383 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:22.383 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:22.384 DEBUG:boto:Method: HEAD 2022-09-03T06:37:22.385 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:37:22.385 DEBUG:boto:Data: 2022-09-03T06:37:22.386 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.386 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.387 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.387 DEBUG:boto:Params: {} 2022-09-03T06:37:22.388 DEBUG:boto:Token: None 2022-09-03T06:37:22.389 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj22 2022-09-03T06:37:22.389 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MagdkHRVuJ3kLgC1kVZZAlND78M= 2022-09-03T06:37:22.391 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MagdkHRVuJ3kLgC1kVZZAlND78M=', 'Content-Length': '0'} 2022-09-03T06:37:22.393 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000072b802d552a5d021-006312f622-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.393 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:22.394 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:22.395 DEBUG:boto:Method: HEAD 2022-09-03T06:37:22.395 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:37:22.396 DEBUG:boto:Data: 2022-09-03T06:37:22.396 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.397 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.397 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.398 DEBUG:boto:Params: {} 2022-09-03T06:37:22.398 DEBUG:boto:Token: None 2022-09-03T06:37:22.399 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj22 2022-09-03T06:37:22.400 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MagdkHRVuJ3kLgC1kVZZAlND78M= 2022-09-03T06:37:22.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MagdkHRVuJ3kLgC1kVZZAlND78M=', 'Content-Length': '0'} 2022-09-03T06:37:22.403 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ea17b2574b1fd466-006312f622-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.404 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:37:22.405 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:22.406 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:22.406 DEBUG:boto:Method: GET 2022-09-03T06:37:22.407 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:37:22.408 DEBUG:boto:Data: 2022-09-03T06:37:22.409 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.410 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.411 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.412 DEBUG:boto:Params: {} 2022-09-03T06:37:22.413 DEBUG:boto:Token: None 2022-09-03T06:37:22.414 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj22 2022-09-03T06:37:22.415 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iSmKw17aa/7LzPSCsAMwhWZTeH4= 2022-09-03T06:37:22.416 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iSmKw17aa/7LzPSCsAMwhWZTeH4=', 'Content-Length': '0'} 2022-09-03T06:37:22.424 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003895c8e28118b64f-006312f622-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.428 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:22.429 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:22.429 DEBUG:boto:Method: GET 2022-09-03T06:37:22.430 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:37:22.431 DEBUG:boto:Data: 2022-09-03T06:37:22.431 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.433 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.434 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.435 DEBUG:boto:Params: {} 2022-09-03T06:37:22.436 DEBUG:boto:Token: None 2022-09-03T06:37:22.437 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj22 2022-09-03T06:37:22.438 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iSmKw17aa/7LzPSCsAMwhWZTeH4= 2022-09-03T06:37:22.439 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iSmKw17aa/7LzPSCsAMwhWZTeH4=', 'Content-Length': '0'} 2022-09-03T06:37:22.442 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000003286f159a88619a-006312f622-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.443 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:22.444 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:22.444 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:22.445 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:22.446 DEBUG:boto:Method: GET 2022-09-03T06:37:22.447 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:37:22.448 DEBUG:boto:Data: 2022-09-03T06:37:22.448 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.449 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.450 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.451 DEBUG:boto:Params: {} 2022-09-03T06:37:22.452 DEBUG:boto:Token: None 2022-09-03T06:37:22.452 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/ 2022-09-03T06:37:22.453 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g= 2022-09-03T06:37:22.453 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g=', 'Content-Length': '0'} 2022-09-03T06:37:22.456 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008bf0ba9723138fbb-006312f622-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.499 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:22.500 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:22.501 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:22.502 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:22.503 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:22.503 DEBUG:boto:Method: GET 2022-09-03T06:37:22.504 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:37:22.505 DEBUG:boto:Data: 2022-09-03T06:37:22.506 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.506 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.507 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.508 DEBUG:boto:Params: {} 2022-09-03T06:37:22.508 DEBUG:boto:Token: None 2022-09-03T06:37:22.509 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/ 2022-09-03T06:37:22.510 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g= 2022-09-03T06:37:22.510 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g=', 'Content-Length': '0'} 2022-09-03T06:37:22.513 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000654606be9bb31eb0-006312f622-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.514 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:22.514 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:37:22.515 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:22.516 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:22.516 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.517 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.517 DEBUG:boto:Method: GET 2022-09-03T06:37:22.518 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.519 DEBUG:boto:Data: 2022-09-03T06:37:22.520 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.520 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.521 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.521 DEBUG:boto:Params: {} 2022-09-03T06:37:22.523 DEBUG:boto:Token: None 2022-09-03T06:37:22.523 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.524 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wAhQ9h+oQsm3S+1er1y3IHLQODY= 2022-09-03T06:37:22.524 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wAhQ9h+oQsm3S+1er1y3IHLQODY=', 'Content-Length': '0'} 2022-09-03T06:37:22.527 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000025299898871cceff-006312f622-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.528 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:22.529 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:22.530 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.531 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.532 DEBUG:boto:Method: GET 2022-09-03T06:37:22.533 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.534 DEBUG:boto:Data: 2022-09-03T06:37:22.534 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.535 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.536 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.536 DEBUG:boto:Params: {} 2022-09-03T06:37:22.537 DEBUG:boto:Token: None 2022-09-03T06:37:22.537 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.538 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wAhQ9h+oQsm3S+1er1y3IHLQODY= 2022-09-03T06:37:22.539 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wAhQ9h+oQsm3S+1er1y3IHLQODY=', 'Content-Length': '0'} 2022-09-03T06:37:22.541 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd1006ca5a1e6527-006312f622-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.542 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:22.542 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:22.543 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.543 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.544 DEBUG:boto:Method: HEAD 2022-09-03T06:37:22.544 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.545 DEBUG:boto:Data: 2022-09-03T06:37:22.545 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.546 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.546 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.547 DEBUG:boto:Params: {} 2022-09-03T06:37:22.547 DEBUG:boto:Token: None 2022-09-03T06:37:22.548 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.548 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5CwLb3BmoX8y542ZuWy1ir8qKPw= 2022-09-03T06:37:22.549 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5CwLb3BmoX8y542ZuWy1ir8qKPw=', 'Content-Length': '0'} 2022-09-03T06:37:22.551 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000062a16a7b9c27a79c-006312f622-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.552 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:22.552 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:22.553 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.553 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.554 DEBUG:boto:Method: HEAD 2022-09-03T06:37:22.554 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.555 DEBUG:boto:Data: 2022-09-03T06:37:22.556 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.556 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.557 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.557 DEBUG:boto:Params: {} 2022-09-03T06:37:22.558 DEBUG:boto:Token: None 2022-09-03T06:37:22.559 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.560 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5CwLb3BmoX8y542ZuWy1ir8qKPw= 2022-09-03T06:37:22.560 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5CwLb3BmoX8y542ZuWy1ir8qKPw=', 'Content-Length': '0'} 2022-09-03T06:37:22.562 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000796d91dcab5205ae-006312f622-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.563 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:37:22.563 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:22.564 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:22.565 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.565 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.566 DEBUG:boto:Method: GET 2022-09-03T06:37:22.566 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.567 DEBUG:boto:Data: 2022-09-03T06:37:22.567 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.568 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.569 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.569 DEBUG:boto:Params: {} 2022-09-03T06:37:22.570 DEBUG:boto:Token: None 2022-09-03T06:37:22.570 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.571 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wAhQ9h+oQsm3S+1er1y3IHLQODY= 2022-09-03T06:37:22.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wAhQ9h+oQsm3S+1er1y3IHLQODY=', 'Content-Length': '0'} 2022-09-03T06:37:22.573 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c27532e52eef3d25-006312f622-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.574 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:22.575 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:22.575 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.576 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.576 DEBUG:boto:Method: GET 2022-09-03T06:37:22.577 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.577 DEBUG:boto:Data: 2022-09-03T06:37:22.578 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.579 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.579 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.580 DEBUG:boto:Params: {} 2022-09-03T06:37:22.580 DEBUG:boto:Token: None 2022-09-03T06:37:22.581 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:22.581 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wAhQ9h+oQsm3S+1er1y3IHLQODY= 2022-09-03T06:37:22.582 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wAhQ9h+oQsm3S+1er1y3IHLQODY=', 'Content-Length': '0'} 2022-09-03T06:37:22.584 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000016c9c7a9d7800e2a-006312f622-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.626 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:22.627 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:22.628 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:22.628 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:22.629 DEBUG:boto:Method: GET 2022-09-03T06:37:22.629 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:37:22.631 DEBUG:boto:Data: 2022-09-03T06:37:22.632 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.632 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.633 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.635 DEBUG:boto:Params: {} 2022-09-03T06:37:22.637 DEBUG:boto:Token: None 2022-09-03T06:37:22.638 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/ 2022-09-03T06:37:22.639 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g= 2022-09-03T06:37:22.640 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g=', 'Content-Length': '0'} 2022-09-03T06:37:22.644 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c2b3de09d2a06372-006312f622-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.645 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:22.647 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:22.648 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:22.650 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:22.651 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:22.652 DEBUG:boto:Method: GET 2022-09-03T06:37:22.652 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:37:22.653 DEBUG:boto:Data: 2022-09-03T06:37:22.653 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.655 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.656 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.657 DEBUG:boto:Params: {} 2022-09-03T06:37:22.659 DEBUG:boto:Token: None 2022-09-03T06:37:22.660 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/ 2022-09-03T06:37:22.661 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g= 2022-09-03T06:37:22.662 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g=', 'Content-Length': '0'} 2022-09-03T06:37:22.665 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000081b9d17f95efae7-006312f622-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.666 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:22.668 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:22.669 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:22.670 DEBUG:boto:Method: HEAD 2022-09-03T06:37:22.671 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:37:22.672 DEBUG:boto:Data: 2022-09-03T06:37:22.673 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.674 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.675 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.676 DEBUG:boto:Params: {} 2022-09-03T06:37:22.677 DEBUG:boto:Token: None 2022-09-03T06:37:22.678 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj33 2022-09-03T06:37:22.679 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Aw1HMPiuX7/1/OHVYE1WS5GxzvY= 2022-09-03T06:37:22.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Aw1HMPiuX7/1/OHVYE1WS5GxzvY=', 'Content-Length': '0'} 2022-09-03T06:37:22.682 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d2f1f5094a581a1c-006312f622-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.684 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:22.685 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:22.686 DEBUG:boto:Method: HEAD 2022-09-03T06:37:22.687 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:37:22.688 DEBUG:boto:Data: 2022-09-03T06:37:22.689 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.690 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.691 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.692 DEBUG:boto:Params: {} 2022-09-03T06:37:22.694 DEBUG:boto:Token: None 2022-09-03T06:37:22.695 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj33 2022-09-03T06:37:22.696 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Aw1HMPiuX7/1/OHVYE1WS5GxzvY= 2022-09-03T06:37:22.697 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Aw1HMPiuX7/1/OHVYE1WS5GxzvY=', 'Content-Length': '0'} 2022-09-03T06:37:22.700 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003501b9c834920720-006312f622-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.701 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:37:22.702 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:22.702 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:22.703 DEBUG:boto:Method: GET 2022-09-03T06:37:22.704 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:37:22.704 DEBUG:boto:Data: 2022-09-03T06:37:22.705 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.705 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.706 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.706 DEBUG:boto:Params: {} 2022-09-03T06:37:22.707 DEBUG:boto:Token: None 2022-09-03T06:37:22.707 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj33 2022-09-03T06:37:22.708 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zlWMBfRrc++aS8e2kkUeXprrAW8= 2022-09-03T06:37:22.709 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zlWMBfRrc++aS8e2kkUeXprrAW8=', 'Content-Length': '0'} 2022-09-03T06:37:22.711 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cbe3c8357f4ea197-006312f622-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.754 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:22.755 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:22.756 DEBUG:boto:Method: GET 2022-09-03T06:37:22.756 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:37:22.757 DEBUG:boto:Data: 2022-09-03T06:37:22.757 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.758 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.759 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.760 DEBUG:boto:Params: {} 2022-09-03T06:37:22.760 DEBUG:boto:Token: None 2022-09-03T06:37:22.761 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj33 2022-09-03T06:37:22.761 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zlWMBfRrc++aS8e2kkUeXprrAW8= 2022-09-03T06:37:22.762 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zlWMBfRrc++aS8e2kkUeXprrAW8=', 'Content-Length': '0'} 2022-09-03T06:37:22.764 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007269ae1926a4dbd3-006312f622-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.765 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:22.766 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:22.767 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:22.767 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:22.768 DEBUG:boto:Method: GET 2022-09-03T06:37:22.769 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:37:22.769 DEBUG:boto:Data: 2022-09-03T06:37:22.770 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.771 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.772 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.773 DEBUG:boto:Params: {} 2022-09-03T06:37:22.776 DEBUG:boto:Token: None 2022-09-03T06:37:22.777 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/ 2022-09-03T06:37:22.778 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g= 2022-09-03T06:37:22.779 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g=', 'Content-Length': '0'} 2022-09-03T06:37:22.782 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000965fdd0a85d4aa52-006312f622-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.783 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:22.784 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:22.785 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:22.786 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:22.786 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:22.787 DEBUG:boto:Method: GET 2022-09-03T06:37:22.787 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:37:22.788 DEBUG:boto:Data: 2022-09-03T06:37:22.789 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.789 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.790 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.790 DEBUG:boto:Params: {} 2022-09-03T06:37:22.791 DEBUG:boto:Token: None 2022-09-03T06:37:22.791 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/ 2022-09-03T06:37:22.792 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g= 2022-09-03T06:37:22.793 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g=', 'Content-Length': '0'} 2022-09-03T06:37:22.795 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000880972d9a882d673-006312f622-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.796 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:22.797 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:37:22.798 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:22.798 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:22.799 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.799 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.800 DEBUG:boto:Method: GET 2022-09-03T06:37:22.800 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.801 DEBUG:boto:Data: 2022-09-03T06:37:22.801 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.802 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.803 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.803 DEBUG:boto:Params: {} 2022-09-03T06:37:22.804 DEBUG:boto:Token: None 2022-09-03T06:37:22.805 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.805 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0sv8fM644IbyhlhhVBHndM/mK6w= 2022-09-03T06:37:22.806 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0sv8fM644IbyhlhhVBHndM/mK6w=', 'Content-Length': '0'} 2022-09-03T06:37:22.808 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c96c3d181c905008-006312f622-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.809 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:22.810 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:22.810 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.811 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.811 DEBUG:boto:Method: GET 2022-09-03T06:37:22.812 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.812 DEBUG:boto:Data: 2022-09-03T06:37:22.813 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.814 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.814 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.815 DEBUG:boto:Params: {} 2022-09-03T06:37:22.815 DEBUG:boto:Token: None 2022-09-03T06:37:22.816 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.817 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0sv8fM644IbyhlhhVBHndM/mK6w= 2022-09-03T06:37:22.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0sv8fM644IbyhlhhVBHndM/mK6w=', 'Content-Length': '0'} 2022-09-03T06:37:22.820 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005d17f0df292f6531-006312f622-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.820 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:22.821 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:22.821 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.822 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.823 DEBUG:boto:Method: HEAD 2022-09-03T06:37:22.823 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.824 DEBUG:boto:Data: 2022-09-03T06:37:22.824 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.825 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.825 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.833 DEBUG:boto:Params: {} 2022-09-03T06:37:22.834 DEBUG:boto:Token: None 2022-09-03T06:37:22.835 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.835 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:W3uhy3G8IUHjaYucma+knvd3zpA= 2022-09-03T06:37:22.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:W3uhy3G8IUHjaYucma+knvd3zpA=', 'Content-Length': '0'} 2022-09-03T06:37:22.839 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e02a47a26ee452e5-006312f622-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.840 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:22.841 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:22.841 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.842 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.842 DEBUG:boto:Method: HEAD 2022-09-03T06:37:22.843 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.843 DEBUG:boto:Data: 2022-09-03T06:37:22.844 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.845 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.845 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.846 DEBUG:boto:Params: {} 2022-09-03T06:37:22.847 DEBUG:boto:Token: None 2022-09-03T06:37:22.847 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.848 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:W3uhy3G8IUHjaYucma+knvd3zpA= 2022-09-03T06:37:22.849 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:W3uhy3G8IUHjaYucma+knvd3zpA=', 'Content-Length': '0'} 2022-09-03T06:37:22.851 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006482accdec34cb3d-006312f622-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.853 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:37:22.854 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:22.855 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:22.856 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.856 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.857 DEBUG:boto:Method: GET 2022-09-03T06:37:22.858 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.860 DEBUG:boto:Data: 2022-09-03T06:37:22.861 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.862 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.863 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.863 DEBUG:boto:Params: {} 2022-09-03T06:37:22.864 DEBUG:boto:Token: None 2022-09-03T06:37:22.865 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.865 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0sv8fM644IbyhlhhVBHndM/mK6w= 2022-09-03T06:37:22.866 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0sv8fM644IbyhlhhVBHndM/mK6w=', 'Content-Length': '0'} 2022-09-03T06:37:22.868 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000890d16d05077603e-006312f622-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.914 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:22.915 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:22.916 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.916 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.917 DEBUG:boto:Method: GET 2022-09-03T06:37:22.918 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.918 DEBUG:boto:Data: 2022-09-03T06:37:22.919 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.919 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.920 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.920 DEBUG:boto:Params: {} 2022-09-03T06:37:22.921 DEBUG:boto:Token: None 2022-09-03T06:37:22.922 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:22.923 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0sv8fM644IbyhlhhVBHndM/mK6w= 2022-09-03T06:37:22.923 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0sv8fM644IbyhlhhVBHndM/mK6w=', 'Content-Length': '0'} 2022-09-03T06:37:22.926 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000006c69c78977b2cee-006312f622-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.927 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:22.927 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:22.928 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:22.928 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:22.929 DEBUG:boto:Method: GET 2022-09-03T06:37:22.930 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:37:22.930 DEBUG:boto:Data: 2022-09-03T06:37:22.931 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.931 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.932 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.933 DEBUG:boto:Params: {} 2022-09-03T06:37:22.934 DEBUG:boto:Token: None 2022-09-03T06:37:22.935 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/ 2022-09-03T06:37:22.935 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g= 2022-09-03T06:37:22.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g=', 'Content-Length': '0'} 2022-09-03T06:37:22.939 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002a65b187cb3e5673-006312f622-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.940 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:22.941 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:22.941 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:22.942 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:22.942 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:22.943 DEBUG:boto:Method: GET 2022-09-03T06:37:22.944 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:37:22.944 DEBUG:boto:Data: 2022-09-03T06:37:22.945 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.945 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.946 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.947 DEBUG:boto:Params: {} 2022-09-03T06:37:22.947 DEBUG:boto:Token: None 2022-09-03T06:37:22.948 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/ 2022-09-03T06:37:22.949 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g= 2022-09-03T06:37:22.949 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VMb1AIWnqYgVdU0+6GF/d2z1v/g=', 'Content-Length': '0'} 2022-09-03T06:37:22.952 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ff514110e1cc59f0-006312f622-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.953 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:22.954 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:22.954 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:22.955 DEBUG:boto:Method: HEAD 2022-09-03T06:37:22.955 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:37:22.956 DEBUG:boto:Data: 2022-09-03T06:37:22.957 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.957 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.958 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.959 DEBUG:boto:Params: {} 2022-09-03T06:37:22.960 DEBUG:boto:Token: None 2022-09-03T06:37:22.960 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj44 2022-09-03T06:37:22.961 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y3C9SzOMjX8N5DFqTjnjOX2r6bM= 2022-09-03T06:37:22.962 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y3C9SzOMjX8N5DFqTjnjOX2r6bM=', 'Content-Length': '0'} 2022-09-03T06:37:22.964 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000060850c8b7be26c6c-006312f622-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.965 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:22.965 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:22.966 DEBUG:boto:Method: HEAD 2022-09-03T06:37:22.967 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:37:22.967 DEBUG:boto:Data: 2022-09-03T06:37:22.968 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.968 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.969 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.969 DEBUG:boto:Params: {} 2022-09-03T06:37:22.970 DEBUG:boto:Token: None 2022-09-03T06:37:22.971 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj44 2022-09-03T06:37:22.971 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y3C9SzOMjX8N5DFqTjnjOX2r6bM= 2022-09-03T06:37:22.972 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y3C9SzOMjX8N5DFqTjnjOX2r6bM=', 'Content-Length': '0'} 2022-09-03T06:37:22.974 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ef2096a4cc899ad-006312f622-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.974 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:37:22.975 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:22.976 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:22.976 DEBUG:boto:Method: GET 2022-09-03T06:37:22.977 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:37:22.977 DEBUG:boto:Data: 2022-09-03T06:37:22.978 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.979 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:22.979 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.980 DEBUG:boto:Params: {} 2022-09-03T06:37:22.980 DEBUG:boto:Token: None 2022-09-03T06:37:22.981 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj44 2022-09-03T06:37:22.981 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BCeinrq6Odl7BqDGGbZYwGC7I20= 2022-09-03T06:37:22.982 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BCeinrq6Odl7BqDGGbZYwGC7I20=', 'Content-Length': '0'} 2022-09-03T06:37:22.984 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096e2ed0cbce89312-006312f622-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:22.985 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:22.985 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:22.986 DEBUG:boto:Method: GET 2022-09-03T06:37:22.987 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:37:22.987 DEBUG:boto:Data: 2022-09-03T06:37:22.988 DEBUG:boto:Headers: {} 2022-09-03T06:37:22.988 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:22.989 DEBUG:boto:Port: 8000 2022-09-03T06:37:22.989 DEBUG:boto:Params: {} 2022-09-03T06:37:22.990 DEBUG:boto:Token: None 2022-09-03T06:37:22.991 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:22 GMT /dsgvcy-75/obj44 2022-09-03T06:37:22.991 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BCeinrq6Odl7BqDGGbZYwGC7I20= 2022-09-03T06:37:22.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BCeinrq6Odl7BqDGGbZYwGC7I20=', 'Content-Length': '0'} 2022-09-03T06:37:22.994 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000058f84d6d4766cc02-006312f622-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.042 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:23.043 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:23.044 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:23.046 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:23.046 DEBUG:boto:Method: GET 2022-09-03T06:37:23.047 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:37:23.047 DEBUG:boto:Data: 2022-09-03T06:37:23.048 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.049 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:23.050 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.050 DEBUG:boto:Params: {} 2022-09-03T06:37:23.051 DEBUG:boto:Token: None 2022-09-03T06:37:23.051 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/ 2022-09-03T06:37:23.052 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE= 2022-09-03T06:37:23.053 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE=', 'Content-Length': '0'} 2022-09-03T06:37:23.056 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009ae4083cf624030d-006312f623-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.057 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:23.058 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:23.059 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:23.060 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:23.061 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:23.062 DEBUG:boto:Method: GET 2022-09-03T06:37:23.063 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:37:23.064 DEBUG:boto:Data: 2022-09-03T06:37:23.065 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.066 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.067 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.068 DEBUG:boto:Params: {} 2022-09-03T06:37:23.069 DEBUG:boto:Token: None 2022-09-03T06:37:23.071 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/ 2022-09-03T06:37:23.072 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE= 2022-09-03T06:37:23.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE=', 'Content-Length': '0'} 2022-09-03T06:37:23.076 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000431d272951789849-006312f623-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.077 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:23.078 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:37:23.079 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:23.079 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:23.080 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.080 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.081 DEBUG:boto:Method: GET 2022-09-03T06:37:23.081 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.082 DEBUG:boto:Data: 2022-09-03T06:37:23.083 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.083 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:23.084 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.084 DEBUG:boto:Params: {} 2022-09-03T06:37:23.085 DEBUG:boto:Token: None 2022-09-03T06:37:23.086 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.086 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QILm1aGZoEI3A8eHJoj5ciVP+lE= 2022-09-03T06:37:23.087 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QILm1aGZoEI3A8eHJoj5ciVP+lE=', 'Content-Length': '0'} 2022-09-03T06:37:23.089 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f41b70b81aa089b6-006312f623-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.130 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:23.131 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:23.131 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.132 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.132 DEBUG:boto:Method: GET 2022-09-03T06:37:23.133 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.134 DEBUG:boto:Data: 2022-09-03T06:37:23.135 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.135 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.136 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.137 DEBUG:boto:Params: {} 2022-09-03T06:37:23.137 DEBUG:boto:Token: None 2022-09-03T06:37:23.138 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.139 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QILm1aGZoEI3A8eHJoj5ciVP+lE= 2022-09-03T06:37:23.139 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QILm1aGZoEI3A8eHJoj5ciVP+lE=', 'Content-Length': '0'} 2022-09-03T06:37:23.143 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000dab0dd37b85894e-006312f623-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.145 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:23.145 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:23.146 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.146 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.147 DEBUG:boto:Method: HEAD 2022-09-03T06:37:23.148 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.148 DEBUG:boto:Data: 2022-09-03T06:37:23.150 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.150 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:23.151 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.152 DEBUG:boto:Params: {} 2022-09-03T06:37:23.153 DEBUG:boto:Token: None 2022-09-03T06:37:23.153 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.154 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9DXUcitZAFmo7FstVfiajmTJnMQ= 2022-09-03T06:37:23.155 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9DXUcitZAFmo7FstVfiajmTJnMQ=', 'Content-Length': '0'} 2022-09-03T06:37:23.157 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d3aa6937e31d7bca-006312f623-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.158 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:23.159 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:23.159 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.160 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.161 DEBUG:boto:Method: HEAD 2022-09-03T06:37:23.162 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.162 DEBUG:boto:Data: 2022-09-03T06:37:23.163 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.163 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.164 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.165 DEBUG:boto:Params: {} 2022-09-03T06:37:23.165 DEBUG:boto:Token: None 2022-09-03T06:37:23.166 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.166 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9DXUcitZAFmo7FstVfiajmTJnMQ= 2022-09-03T06:37:23.167 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9DXUcitZAFmo7FstVfiajmTJnMQ=', 'Content-Length': '0'} 2022-09-03T06:37:23.169 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d5e4fd6a01e26dd-006312f623-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.170 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:37:23.171 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:23.172 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:23.172 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.173 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.173 DEBUG:boto:Method: GET 2022-09-03T06:37:23.174 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.174 DEBUG:boto:Data: 2022-09-03T06:37:23.175 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.175 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:23.176 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.176 DEBUG:boto:Params: {} 2022-09-03T06:37:23.177 DEBUG:boto:Token: None 2022-09-03T06:37:23.178 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.178 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QILm1aGZoEI3A8eHJoj5ciVP+lE= 2022-09-03T06:37:23.179 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QILm1aGZoEI3A8eHJoj5ciVP+lE=', 'Content-Length': '0'} 2022-09-03T06:37:23.181 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dab58c739f5d665c-006312f623-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.182 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:23.183 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:23.183 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.184 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.184 DEBUG:boto:Method: GET 2022-09-03T06:37:23.185 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.185 DEBUG:boto:Data: 2022-09-03T06:37:23.186 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.187 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.187 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.188 DEBUG:boto:Params: {} 2022-09-03T06:37:23.188 DEBUG:boto:Token: None 2022-09-03T06:37:23.189 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:23.189 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QILm1aGZoEI3A8eHJoj5ciVP+lE= 2022-09-03T06:37:23.190 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QILm1aGZoEI3A8eHJoj5ciVP+lE=', 'Content-Length': '0'} 2022-09-03T06:37:23.193 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d7319e12983e9611-006312f623-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.193 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:23.194 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:23.195 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:23.195 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:23.196 DEBUG:boto:Method: GET 2022-09-03T06:37:23.196 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:37:23.197 DEBUG:boto:Data: 2022-09-03T06:37:23.197 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.198 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:23.198 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.199 DEBUG:boto:Params: {} 2022-09-03T06:37:23.199 DEBUG:boto:Token: None 2022-09-03T06:37:23.200 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/ 2022-09-03T06:37:23.201 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE= 2022-09-03T06:37:23.201 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE=', 'Content-Length': '0'} 2022-09-03T06:37:23.204 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000da81907d76676d65-006312f623-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.205 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:23.206 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:23.206 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:23.207 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:23.207 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:23.208 DEBUG:boto:Method: GET 2022-09-03T06:37:23.209 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:37:23.209 DEBUG:boto:Data: 2022-09-03T06:37:23.210 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.210 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.211 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.211 DEBUG:boto:Params: {} 2022-09-03T06:37:23.212 DEBUG:boto:Token: None 2022-09-03T06:37:23.213 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/ 2022-09-03T06:37:23.213 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE= 2022-09-03T06:37:23.214 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE=', 'Content-Length': '0'} 2022-09-03T06:37:23.217 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004a4f9a4636f5bd0d-006312f623-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.218 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:23.219 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:23.219 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:23.220 DEBUG:boto:Method: HEAD 2022-09-03T06:37:23.220 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:37:23.221 DEBUG:boto:Data: 2022-09-03T06:37:23.221 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.222 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:23.222 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.223 DEBUG:boto:Params: {} 2022-09-03T06:37:23.224 DEBUG:boto:Token: None 2022-09-03T06:37:23.225 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj55 2022-09-03T06:37:23.225 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CTSwOUYEVD5koesyv3xYnnhKaC8= 2022-09-03T06:37:23.226 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CTSwOUYEVD5koesyv3xYnnhKaC8=', 'Content-Length': '0'} 2022-09-03T06:37:23.228 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000075a597233e369bae-006312f623-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.229 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:23.230 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:23.231 DEBUG:boto:Method: HEAD 2022-09-03T06:37:23.232 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:37:23.232 DEBUG:boto:Data: 2022-09-03T06:37:23.233 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.233 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.234 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.235 DEBUG:boto:Params: {} 2022-09-03T06:37:23.235 DEBUG:boto:Token: None 2022-09-03T06:37:23.236 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj55 2022-09-03T06:37:23.238 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CTSwOUYEVD5koesyv3xYnnhKaC8= 2022-09-03T06:37:23.239 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CTSwOUYEVD5koesyv3xYnnhKaC8=', 'Content-Length': '0'} 2022-09-03T06:37:23.242 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000420d8f49883f5218-006312f623-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.243 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:37:23.243 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:23.244 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:23.245 DEBUG:boto:Method: GET 2022-09-03T06:37:23.246 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:37:23.246 DEBUG:boto:Data: 2022-09-03T06:37:23.247 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.247 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:23.248 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.249 DEBUG:boto:Params: {} 2022-09-03T06:37:23.249 DEBUG:boto:Token: None 2022-09-03T06:37:23.250 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj55 2022-09-03T06:37:23.251 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:URsFunPBY2V1C3GFZXrpw3J4pTo= 2022-09-03T06:37:23.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:URsFunPBY2V1C3GFZXrpw3J4pTo=', 'Content-Length': '0'} 2022-09-03T06:37:23.254 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000019b83c8036379aad-006312f623-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.255 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:23.256 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:23.256 DEBUG:boto:Method: GET 2022-09-03T06:37:23.257 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:37:23.257 DEBUG:boto:Data: 2022-09-03T06:37:23.258 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.259 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.259 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.260 DEBUG:boto:Params: {} 2022-09-03T06:37:23.261 DEBUG:boto:Token: None 2022-09-03T06:37:23.262 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj55 2022-09-03T06:37:23.263 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:URsFunPBY2V1C3GFZXrpw3J4pTo= 2022-09-03T06:37:23.263 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:URsFunPBY2V1C3GFZXrpw3J4pTo=', 'Content-Length': '0'} 2022-09-03T06:37:23.265 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f41953b1853fe01-006312f623-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.306 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:23.307 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:23.308 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:23.308 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:23.309 DEBUG:boto:Method: GET 2022-09-03T06:37:23.309 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:37:23.310 DEBUG:boto:Data: 2022-09-03T06:37:23.310 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.311 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:23.311 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.312 DEBUG:boto:Params: {} 2022-09-03T06:37:23.313 DEBUG:boto:Token: None 2022-09-03T06:37:23.313 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/ 2022-09-03T06:37:23.314 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE= 2022-09-03T06:37:23.315 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE=', 'Content-Length': '0'} 2022-09-03T06:37:23.317 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006baddf611992ebbb-006312f623-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.318 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:23.319 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:23.320 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:23.320 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:23.321 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:23.321 DEBUG:boto:Method: GET 2022-09-03T06:37:23.322 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:37:23.323 DEBUG:boto:Data: 2022-09-03T06:37:23.323 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.324 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.325 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.325 DEBUG:boto:Params: {} 2022-09-03T06:37:23.326 DEBUG:boto:Token: None 2022-09-03T06:37:23.327 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/ 2022-09-03T06:37:23.327 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE= 2022-09-03T06:37:23.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE=', 'Content-Length': '0'} 2022-09-03T06:37:23.331 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005318ee4cc727a517-006312f623-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.331 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:23.332 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-75 zones={test-zone2, test-zone1} 2022-09-03T06:37:23.333 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-75 zones={test-zone3, test-zone1} 2022-09-03T06:37:23.333 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:23.334 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:23.334 DEBUG:boto:Method: HEAD 2022-09-03T06:37:23.335 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:37:23.336 DEBUG:boto:Data: 2022-09-03T06:37:23.336 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.337 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:23.337 DEBUG:boto:Port: 8001 2022-09-03T06:37:23.338 DEBUG:boto:Params: {} 2022-09-03T06:37:23.338 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T06:37:23.339 DEBUG:boto:Token: None 2022-09-03T06:37:23.340 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/ 2022-09-03T06:37:23.340 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fB8Cfx4rkJKg2euoOR6af0wOsXk= 2022-09-03T06:37:23.341 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fB8Cfx4rkJKg2euoOR6af0wOsXk=', 'Content-Length': '0'} 2022-09-03T06:37:23.344 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '15'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003e3a4b1ed14b5716-006312f623-1851-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.345 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:23.346 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:23.346 DEBUG:boto:Method: HEAD 2022-09-03T06:37:23.347 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:37:23.347 DEBUG:boto:Data: 2022-09-03T06:37:23.348 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.348 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.349 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.350 DEBUG:boto:Params: {} 2022-09-03T06:37:23.350 DEBUG:boto:Token: None 2022-09-03T06:37:23.351 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/ 2022-09-03T06:37:23.351 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fB8Cfx4rkJKg2euoOR6af0wOsXk= 2022-09-03T06:37:23.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fB8Cfx4rkJKg2euoOR6af0wOsXk=', 'Content-Length': '0'} 2022-09-03T06:37:23.355 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '15'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000001a739000c81bd58-006312f623-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.355 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:37:23.356 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:23.356 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:23.357 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:23.357 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:23.358 DEBUG:boto:Method: GET 2022-09-03T06:37:23.359 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:23.359 DEBUG:boto:Data: 2022-09-03T06:37:23.360 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.360 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:23.361 DEBUG:boto:Port: 8001 2022-09-03T06:37:23.361 DEBUG:boto:Params: {} 2022-09-03T06:37:23.362 DEBUG:boto:Token: None 2022-09-03T06:37:23.362 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/?versions 2022-09-03T06:37:23.363 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yejcTbWcPMelY6loPiuxhRgvxoY= 2022-09-03T06:37:23.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yejcTbWcPMelY6loPiuxhRgvxoY=', 'Content-Length': '0'} 2022-09-03T06:37:23.366 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002bc5ba9a55a5b05b-006312f623-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.367 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:23.368 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:37:23.368 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:37:23.369 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:37:23.369 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:37:23.370 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:37:23.370 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:37:23.371 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:23.372 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:23.372 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:23.373 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:23.374 DEBUG:boto:Method: GET 2022-09-03T06:37:23.374 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:23.375 DEBUG:boto:Data: 2022-09-03T06:37:23.375 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.376 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.376 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.377 DEBUG:boto:Params: {} 2022-09-03T06:37:23.377 DEBUG:boto:Token: None 2022-09-03T06:37:23.378 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/?versions 2022-09-03T06:37:23.379 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yejcTbWcPMelY6loPiuxhRgvxoY= 2022-09-03T06:37:23.379 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yejcTbWcPMelY6loPiuxhRgvxoY=', 'Content-Length': '0'} 2022-09-03T06:37:23.382 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002be9dd1bddbfee6f-006312f623-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.383 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:23.384 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:37:23.384 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:37:23.385 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:37:23.386 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:37:23.386 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:37:23.387 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:23.387 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:23.388 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:23.388 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:23.389 DEBUG:boto:Method: GET 2022-09-03T06:37:23.389 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:23.390 DEBUG:boto:Data: 2022-09-03T06:37:23.390 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.391 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:23.392 DEBUG:boto:Port: 8001 2022-09-03T06:37:23.392 DEBUG:boto:Params: {} 2022-09-03T06:37:23.393 DEBUG:boto:Token: None 2022-09-03T06:37:23.393 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/?versions 2022-09-03T06:37:23.394 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yejcTbWcPMelY6loPiuxhRgvxoY= 2022-09-03T06:37:23.395 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yejcTbWcPMelY6loPiuxhRgvxoY=', 'Content-Length': '0'} 2022-09-03T06:37:23.397 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002fcdeadcb0b56d91-006312f623-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.398 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:23.399 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:23.400 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:23.400 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:23.401 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:23.402 DEBUG:boto:Method: GET 2022-09-03T06:37:23.402 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:23.403 DEBUG:boto:Data: 2022-09-03T06:37:23.403 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.404 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.404 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.405 DEBUG:boto:Params: {} 2022-09-03T06:37:23.406 DEBUG:boto:Token: None 2022-09-03T06:37:23.406 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/?versions 2022-09-03T06:37:23.407 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yejcTbWcPMelY6loPiuxhRgvxoY= 2022-09-03T06:37:23.407 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yejcTbWcPMelY6loPiuxhRgvxoY=', 'Content-Length': '0'} 2022-09-03T06:37:23.410 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000062bda10a143a586a-006312f623-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.412 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:23.413 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:37:23.414 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:23.415 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:23.416 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.417 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.417 DEBUG:boto:Method: GET 2022-09-03T06:37:23.418 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.418 DEBUG:boto:Data: 2022-09-03T06:37:23.419 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.420 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:23.420 DEBUG:boto:Port: 8001 2022-09-03T06:37:23.421 DEBUG:boto:Params: {} 2022-09-03T06:37:23.422 DEBUG:boto:Token: None 2022-09-03T06:37:23.422 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.423 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eTzB4Nk5Blw3u0Hb0StydmzkNJM= 2022-09-03T06:37:23.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eTzB4Nk5Blw3u0Hb0StydmzkNJM=', 'Content-Length': '0'} 2022-09-03T06:37:23.426 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c812575b38042f03-006312f623-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.427 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:23.427 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:23.428 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.428 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.429 DEBUG:boto:Method: GET 2022-09-03T06:37:23.430 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.431 DEBUG:boto:Data: 2022-09-03T06:37:23.432 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.433 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.434 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.436 DEBUG:boto:Params: {} 2022-09-03T06:37:23.437 DEBUG:boto:Token: None 2022-09-03T06:37:23.438 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.439 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eTzB4Nk5Blw3u0Hb0StydmzkNJM= 2022-09-03T06:37:23.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eTzB4Nk5Blw3u0Hb0StydmzkNJM=', 'Content-Length': '0'} 2022-09-03T06:37:23.443 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e862bf13b460bbd-006312f623-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.490 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:23.491 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:23.492 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.492 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.493 DEBUG:boto:Method: HEAD 2022-09-03T06:37:23.494 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.494 DEBUG:boto:Data: 2022-09-03T06:37:23.495 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.495 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:23.496 DEBUG:boto:Port: 8001 2022-09-03T06:37:23.496 DEBUG:boto:Params: {} 2022-09-03T06:37:23.497 DEBUG:boto:Token: None 2022-09-03T06:37:23.497 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.498 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gvPKeXmNEUZ0A9CQXWLCD8wxfB8= 2022-09-03T06:37:23.498 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gvPKeXmNEUZ0A9CQXWLCD8wxfB8=', 'Content-Length': '0'} 2022-09-03T06:37:23.501 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc78e9bd1a92fbea-006312f623-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.503 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:23.504 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:23.505 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.508 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.509 DEBUG:boto:Method: HEAD 2022-09-03T06:37:23.510 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.511 DEBUG:boto:Data: 2022-09-03T06:37:23.512 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.513 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.514 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.515 DEBUG:boto:Params: {} 2022-09-03T06:37:23.516 DEBUG:boto:Token: None 2022-09-03T06:37:23.517 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.518 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gvPKeXmNEUZ0A9CQXWLCD8wxfB8= 2022-09-03T06:37:23.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gvPKeXmNEUZ0A9CQXWLCD8wxfB8=', 'Content-Length': '0'} 2022-09-03T06:37:23.524 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000de84c5b3c38fb6c4-006312f623-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.525 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:37:23.525 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:23.526 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:23.526 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.527 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.528 DEBUG:boto:Method: GET 2022-09-03T06:37:23.528 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.529 DEBUG:boto:Data: 2022-09-03T06:37:23.529 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.530 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:23.530 DEBUG:boto:Port: 8001 2022-09-03T06:37:23.531 DEBUG:boto:Params: {} 2022-09-03T06:37:23.531 DEBUG:boto:Token: None 2022-09-03T06:37:23.532 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.533 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eTzB4Nk5Blw3u0Hb0StydmzkNJM= 2022-09-03T06:37:23.533 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eTzB4Nk5Blw3u0Hb0StydmzkNJM=', 'Content-Length': '0'} 2022-09-03T06:37:23.535 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001bf2cc7852d122a6-006312f623-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.537 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:23.538 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:23.539 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.540 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.541 DEBUG:boto:Method: GET 2022-09-03T06:37:23.542 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.543 DEBUG:boto:Data: 2022-09-03T06:37:23.544 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.545 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.546 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.547 DEBUG:boto:Params: {} 2022-09-03T06:37:23.548 DEBUG:boto:Token: None 2022-09-03T06:37:23.549 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:23.551 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eTzB4Nk5Blw3u0Hb0StydmzkNJM= 2022-09-03T06:37:23.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eTzB4Nk5Blw3u0Hb0StydmzkNJM=', 'Content-Length': '0'} 2022-09-03T06:37:23.555 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000060314ecd89959ad0-006312f623-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.556 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:23.557 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:23.558 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:23.559 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:23.560 DEBUG:boto:Method: GET 2022-09-03T06:37:23.561 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:37:23.562 DEBUG:boto:Data: 2022-09-03T06:37:23.563 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.563 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:23.564 DEBUG:boto:Port: 8001 2022-09-03T06:37:23.565 DEBUG:boto:Params: {} 2022-09-03T06:37:23.566 DEBUG:boto:Token: None 2022-09-03T06:37:23.567 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/ 2022-09-03T06:37:23.567 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE= 2022-09-03T06:37:23.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE=', 'Content-Length': '0'} 2022-09-03T06:37:23.571 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a161ebeac209e402-006312f623-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.572 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:23.573 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:23.573 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:23.574 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:23.575 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:23.576 DEBUG:boto:Method: GET 2022-09-03T06:37:23.576 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:37:23.577 DEBUG:boto:Data: 2022-09-03T06:37:23.578 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.578 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.579 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.579 DEBUG:boto:Params: {} 2022-09-03T06:37:23.580 DEBUG:boto:Token: None 2022-09-03T06:37:23.581 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/ 2022-09-03T06:37:23.582 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE= 2022-09-03T06:37:23.583 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE=', 'Content-Length': '0'} 2022-09-03T06:37:23.586 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d1eeb249d44fbc0-006312f623-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.634 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:23.635 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:23.636 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:23.636 DEBUG:boto:Method: HEAD 2022-09-03T06:37:23.637 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:37:23.638 DEBUG:boto:Data: 2022-09-03T06:37:23.638 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.639 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:23.640 DEBUG:boto:Port: 8001 2022-09-03T06:37:23.641 DEBUG:boto:Params: {} 2022-09-03T06:37:23.642 DEBUG:boto:Token: None 2022-09-03T06:37:23.642 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj11 2022-09-03T06:37:23.643 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mH4Tot25w1z52A79Bn8DErK7+UA= 2022-09-03T06:37:23.643 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mH4Tot25w1z52A79Bn8DErK7+UA=', 'Content-Length': '0'} 2022-09-03T06:37:23.646 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021d0bdcd9777abe2-006312f623-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.646 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:23.647 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:23.648 DEBUG:boto:Method: HEAD 2022-09-03T06:37:23.648 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:37:23.649 DEBUG:boto:Data: 2022-09-03T06:37:23.649 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.650 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.650 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.651 DEBUG:boto:Params: {} 2022-09-03T06:37:23.651 DEBUG:boto:Token: None 2022-09-03T06:37:23.652 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj11 2022-09-03T06:37:23.652 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mH4Tot25w1z52A79Bn8DErK7+UA= 2022-09-03T06:37:23.653 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mH4Tot25w1z52A79Bn8DErK7+UA=', 'Content-Length': '0'} 2022-09-03T06:37:23.655 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001da5526aaaf6f3fd-006312f623-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.656 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:37:23.657 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:23.657 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:23.658 DEBUG:boto:Method: GET 2022-09-03T06:37:23.658 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:37:23.659 DEBUG:boto:Data: 2022-09-03T06:37:23.660 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.660 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:23.661 DEBUG:boto:Port: 8001 2022-09-03T06:37:23.661 DEBUG:boto:Params: {} 2022-09-03T06:37:23.662 DEBUG:boto:Token: None 2022-09-03T06:37:23.662 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj11 2022-09-03T06:37:23.663 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uppUddZEGncgJv3TsQggo/FlGa4= 2022-09-03T06:37:23.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uppUddZEGncgJv3TsQggo/FlGa4=', 'Content-Length': '0'} 2022-09-03T06:37:23.666 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000e3c6b28084cc1d0-006312f623-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.667 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:23.667 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:23.668 DEBUG:boto:Method: GET 2022-09-03T06:37:23.669 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:37:23.671 DEBUG:boto:Data: 2022-09-03T06:37:23.672 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.673 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.674 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.675 DEBUG:boto:Params: {} 2022-09-03T06:37:23.675 DEBUG:boto:Token: None 2022-09-03T06:37:23.676 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj11 2022-09-03T06:37:23.676 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uppUddZEGncgJv3TsQggo/FlGa4= 2022-09-03T06:37:23.677 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uppUddZEGncgJv3TsQggo/FlGa4=', 'Content-Length': '0'} 2022-09-03T06:37:23.679 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005a8f5b2d0919c57e-006312f623-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.680 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:23.681 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:23.681 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:23.682 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:23.682 DEBUG:boto:Method: GET 2022-09-03T06:37:23.683 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:37:23.684 DEBUG:boto:Data: 2022-09-03T06:37:23.684 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.685 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:23.685 DEBUG:boto:Port: 8001 2022-09-03T06:37:23.686 DEBUG:boto:Params: {} 2022-09-03T06:37:23.686 DEBUG:boto:Token: None 2022-09-03T06:37:23.687 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/ 2022-09-03T06:37:23.688 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE= 2022-09-03T06:37:23.688 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE=', 'Content-Length': '0'} 2022-09-03T06:37:23.691 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a4bea06b2b33d61a-006312f623-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.691 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:23.692 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:23.693 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:23.693 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:23.694 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:23.695 DEBUG:boto:Method: GET 2022-09-03T06:37:23.695 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:37:23.696 DEBUG:boto:Data: 2022-09-03T06:37:23.696 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.697 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.697 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.698 DEBUG:boto:Params: {} 2022-09-03T06:37:23.698 DEBUG:boto:Token: None 2022-09-03T06:37:23.699 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/ 2022-09-03T06:37:23.700 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE= 2022-09-03T06:37:23.701 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE=', 'Content-Length': '0'} 2022-09-03T06:37:23.703 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006333889d0bcf8261-006312f623-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.704 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:23.705 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:37:23.706 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:23.706 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:23.707 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.707 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.708 DEBUG:boto:Method: GET 2022-09-03T06:37:23.708 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.709 DEBUG:boto:Data: 2022-09-03T06:37:23.709 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.710 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:23.710 DEBUG:boto:Port: 8001 2022-09-03T06:37:23.711 DEBUG:boto:Params: {} 2022-09-03T06:37:23.712 DEBUG:boto:Token: None 2022-09-03T06:37:23.712 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.713 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s7cSSzE7Mf7jSQp/KmMdbxHPjnA= 2022-09-03T06:37:23.713 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s7cSSzE7Mf7jSQp/KmMdbxHPjnA=', 'Content-Length': '0'} 2022-09-03T06:37:23.716 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002bb921a65fbc9dc6-006312f623-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.762 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:23.763 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:23.763 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.764 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.765 DEBUG:boto:Method: GET 2022-09-03T06:37:23.765 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.766 DEBUG:boto:Data: 2022-09-03T06:37:23.767 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.767 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.768 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.769 DEBUG:boto:Params: {} 2022-09-03T06:37:23.770 DEBUG:boto:Token: None 2022-09-03T06:37:23.771 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.773 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s7cSSzE7Mf7jSQp/KmMdbxHPjnA= 2022-09-03T06:37:23.773 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s7cSSzE7Mf7jSQp/KmMdbxHPjnA=', 'Content-Length': '0'} 2022-09-03T06:37:23.777 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000056c90c647776622a-006312f623-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.778 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:23.779 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:23.780 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.781 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.782 DEBUG:boto:Method: HEAD 2022-09-03T06:37:23.784 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.785 DEBUG:boto:Data: 2022-09-03T06:37:23.786 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.787 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:23.789 DEBUG:boto:Port: 8001 2022-09-03T06:37:23.790 DEBUG:boto:Params: {} 2022-09-03T06:37:23.791 DEBUG:boto:Token: None 2022-09-03T06:37:23.792 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.793 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MW6Y5tsguh6UwhhvZNRWej5udl8= 2022-09-03T06:37:23.795 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MW6Y5tsguh6UwhhvZNRWej5udl8=', 'Content-Length': '0'} 2022-09-03T06:37:23.798 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d5e04fa19bc0cdf-006312f623-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.799 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:23.803 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:23.804 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.805 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.806 DEBUG:boto:Method: HEAD 2022-09-03T06:37:23.807 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.808 DEBUG:boto:Data: 2022-09-03T06:37:23.809 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.810 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.812 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.813 DEBUG:boto:Params: {} 2022-09-03T06:37:23.814 DEBUG:boto:Token: None 2022-09-03T06:37:23.815 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.817 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MW6Y5tsguh6UwhhvZNRWej5udl8= 2022-09-03T06:37:23.818 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MW6Y5tsguh6UwhhvZNRWej5udl8=', 'Content-Length': '0'} 2022-09-03T06:37:23.821 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000816f41215880d26-006312f623-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.822 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:37:23.823 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:23.823 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:23.824 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.825 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.825 DEBUG:boto:Method: GET 2022-09-03T06:37:23.826 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.827 DEBUG:boto:Data: 2022-09-03T06:37:23.828 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.828 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:23.829 DEBUG:boto:Port: 8001 2022-09-03T06:37:23.830 DEBUG:boto:Params: {} 2022-09-03T06:37:23.830 DEBUG:boto:Token: None 2022-09-03T06:37:23.831 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.832 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s7cSSzE7Mf7jSQp/KmMdbxHPjnA= 2022-09-03T06:37:23.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s7cSSzE7Mf7jSQp/KmMdbxHPjnA=', 'Content-Length': '0'} 2022-09-03T06:37:23.835 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bf623a3a163395e9-006312f623-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.835 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:23.836 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:23.837 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.838 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.838 DEBUG:boto:Method: GET 2022-09-03T06:37:23.839 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.840 DEBUG:boto:Data: 2022-09-03T06:37:23.840 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.841 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.842 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.842 DEBUG:boto:Params: {} 2022-09-03T06:37:23.843 DEBUG:boto:Token: None 2022-09-03T06:37:23.844 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:23.845 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s7cSSzE7Mf7jSQp/KmMdbxHPjnA= 2022-09-03T06:37:23.846 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s7cSSzE7Mf7jSQp/KmMdbxHPjnA=', 'Content-Length': '0'} 2022-09-03T06:37:23.853 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009216f75bfb39f5fc-006312f623-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.854 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:23.855 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:23.855 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:23.856 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:23.856 DEBUG:boto:Method: GET 2022-09-03T06:37:23.857 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:37:23.857 DEBUG:boto:Data: 2022-09-03T06:37:23.858 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.858 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:23.859 DEBUG:boto:Port: 8001 2022-09-03T06:37:23.860 DEBUG:boto:Params: {} 2022-09-03T06:37:23.860 DEBUG:boto:Token: None 2022-09-03T06:37:23.861 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/ 2022-09-03T06:37:23.862 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE= 2022-09-03T06:37:23.862 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE=', 'Content-Length': '0'} 2022-09-03T06:37:23.865 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bc07af54c8f3fba1-006312f623-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.866 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:23.867 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:23.868 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:23.868 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:23.869 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:23.869 DEBUG:boto:Method: GET 2022-09-03T06:37:23.870 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:37:23.871 DEBUG:boto:Data: 2022-09-03T06:37:23.871 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.872 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.872 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.873 DEBUG:boto:Params: {} 2022-09-03T06:37:23.873 DEBUG:boto:Token: None 2022-09-03T06:37:23.874 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/ 2022-09-03T06:37:23.875 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE= 2022-09-03T06:37:23.876 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE=', 'Content-Length': '0'} 2022-09-03T06:37:23.879 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d45e7b5fbc753413-006312f623-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.880 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:23.881 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:23.882 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:23.883 DEBUG:boto:Method: HEAD 2022-09-03T06:37:23.883 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:37:23.884 DEBUG:boto:Data: 2022-09-03T06:37:23.884 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.885 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:23.886 DEBUG:boto:Port: 8001 2022-09-03T06:37:23.887 DEBUG:boto:Params: {} 2022-09-03T06:37:23.888 DEBUG:boto:Token: None 2022-09-03T06:37:23.889 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj22 2022-09-03T06:37:23.890 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BdWkSPaYf3Jx6No7RuL5Ddtwmf0= 2022-09-03T06:37:23.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BdWkSPaYf3Jx6No7RuL5Ddtwmf0=', 'Content-Length': '0'} 2022-09-03T06:37:23.894 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ae8ffd3824359a6e-006312f623-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.895 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:23.895 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:23.896 DEBUG:boto:Method: HEAD 2022-09-03T06:37:23.896 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:37:23.897 DEBUG:boto:Data: 2022-09-03T06:37:23.897 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.898 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.898 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.899 DEBUG:boto:Params: {} 2022-09-03T06:37:23.899 DEBUG:boto:Token: None 2022-09-03T06:37:23.900 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj22 2022-09-03T06:37:23.901 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BdWkSPaYf3Jx6No7RuL5Ddtwmf0= 2022-09-03T06:37:23.901 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BdWkSPaYf3Jx6No7RuL5Ddtwmf0=', 'Content-Length': '0'} 2022-09-03T06:37:23.903 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d76db15c677b07b3-006312f623-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.904 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:37:23.905 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:23.905 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:23.906 DEBUG:boto:Method: GET 2022-09-03T06:37:23.906 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:37:23.907 DEBUG:boto:Data: 2022-09-03T06:37:23.908 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.908 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:23.909 DEBUG:boto:Port: 8001 2022-09-03T06:37:23.909 DEBUG:boto:Params: {} 2022-09-03T06:37:23.910 DEBUG:boto:Token: None 2022-09-03T06:37:23.910 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj22 2022-09-03T06:37:23.911 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F9O+McSYLq5HsNw38rbKM/hRRE0= 2022-09-03T06:37:23.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F9O+McSYLq5HsNw38rbKM/hRRE0=', 'Content-Length': '0'} 2022-09-03T06:37:23.916 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003974219c08c54952-006312f623-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.958 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:23.959 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:23.960 DEBUG:boto:Method: GET 2022-09-03T06:37:23.961 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:37:23.961 DEBUG:boto:Data: 2022-09-03T06:37:23.962 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.963 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.963 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.964 DEBUG:boto:Params: {} 2022-09-03T06:37:23.964 DEBUG:boto:Token: None 2022-09-03T06:37:23.965 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/obj22 2022-09-03T06:37:23.965 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F9O+McSYLq5HsNw38rbKM/hRRE0= 2022-09-03T06:37:23.966 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F9O+McSYLq5HsNw38rbKM/hRRE0=', 'Content-Length': '0'} 2022-09-03T06:37:23.970 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e3d83ebd04a0957a-006312f623-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.972 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:23.973 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:23.974 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:23.975 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:23.976 DEBUG:boto:Method: GET 2022-09-03T06:37:23.976 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:37:23.977 DEBUG:boto:Data: 2022-09-03T06:37:23.978 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.980 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:23.981 DEBUG:boto:Port: 8001 2022-09-03T06:37:23.982 DEBUG:boto:Params: {} 2022-09-03T06:37:23.982 DEBUG:boto:Token: None 2022-09-03T06:37:23.983 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/ 2022-09-03T06:37:23.984 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE= 2022-09-03T06:37:23.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE=', 'Content-Length': '0'} 2022-09-03T06:37:23.987 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007ab2b21168c833a8-006312f623-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:23.988 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:23.989 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:23.989 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:23.990 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:23.991 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:23.992 DEBUG:boto:Method: GET 2022-09-03T06:37:23.993 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:37:23.993 DEBUG:boto:Data: 2022-09-03T06:37:23.994 DEBUG:boto:Headers: {} 2022-09-03T06:37:23.994 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:23.995 DEBUG:boto:Port: 8000 2022-09-03T06:37:23.996 DEBUG:boto:Params: {} 2022-09-03T06:37:23.996 DEBUG:boto:Token: None 2022-09-03T06:37:23.997 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:23 GMT /dsgvcy-75/ 2022-09-03T06:37:23.997 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE= 2022-09-03T06:37:24.001 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:73E+KiQ3uLTZmBAH/9T02YFjOVE=', 'Content-Length': '0'} 2022-09-03T06:37:24.005 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000058fd964bfd37b7d8-006312f624-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.006 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:24.007 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:37:24.007 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:24.008 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:24.009 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.010 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.010 DEBUG:boto:Method: GET 2022-09-03T06:37:24.012 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.012 DEBUG:boto:Data: 2022-09-03T06:37:24.013 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.013 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.014 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.015 DEBUG:boto:Params: {} 2022-09-03T06:37:24.016 DEBUG:boto:Token: None 2022-09-03T06:37:24.017 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.018 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J9F2vTL0NnI2E0ku0EIVupql3IY= 2022-09-03T06:37:24.019 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J9F2vTL0NnI2E0ku0EIVupql3IY=', 'Content-Length': '0'} 2022-09-03T06:37:24.022 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002561a700c6b03148-006312f624-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.023 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:24.023 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:24.024 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.024 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.025 DEBUG:boto:Method: GET 2022-09-03T06:37:24.026 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.026 DEBUG:boto:Data: 2022-09-03T06:37:24.027 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.027 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:24.028 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.028 DEBUG:boto:Params: {} 2022-09-03T06:37:24.029 DEBUG:boto:Token: None 2022-09-03T06:37:24.030 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.030 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J9F2vTL0NnI2E0ku0EIVupql3IY= 2022-09-03T06:37:24.031 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J9F2vTL0NnI2E0ku0EIVupql3IY=', 'Content-Length': '0'} 2022-09-03T06:37:24.033 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000082ac64f5fd8162b0-006312f624-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.034 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:24.035 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:24.035 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.036 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.037 DEBUG:boto:Method: HEAD 2022-09-03T06:37:24.037 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.038 DEBUG:boto:Data: 2022-09-03T06:37:24.038 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.039 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.039 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.040 DEBUG:boto:Params: {} 2022-09-03T06:37:24.040 DEBUG:boto:Token: None 2022-09-03T06:37:24.041 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.041 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5o6yB2E04pw7haUviyNPCkt2eM0= 2022-09-03T06:37:24.042 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5o6yB2E04pw7haUviyNPCkt2eM0=', 'Content-Length': '0'} 2022-09-03T06:37:24.044 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca776bcda2b2b211-006312f624-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.045 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:24.046 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:24.046 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.047 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.047 DEBUG:boto:Method: HEAD 2022-09-03T06:37:24.048 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.049 DEBUG:boto:Data: 2022-09-03T06:37:24.049 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.050 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:24.050 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.051 DEBUG:boto:Params: {} 2022-09-03T06:37:24.051 DEBUG:boto:Token: None 2022-09-03T06:37:24.052 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.053 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5o6yB2E04pw7haUviyNPCkt2eM0= 2022-09-03T06:37:24.053 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5o6yB2E04pw7haUviyNPCkt2eM0=', 'Content-Length': '0'} 2022-09-03T06:37:24.056 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cd2dde0747e325f8-006312f624-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.056 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:37:24.057 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:24.058 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:24.058 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.059 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.060 DEBUG:boto:Method: GET 2022-09-03T06:37:24.060 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.061 DEBUG:boto:Data: 2022-09-03T06:37:24.061 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.062 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.062 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.063 DEBUG:boto:Params: {} 2022-09-03T06:37:24.064 DEBUG:boto:Token: None 2022-09-03T06:37:24.064 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.065 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J9F2vTL0NnI2E0ku0EIVupql3IY= 2022-09-03T06:37:24.066 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J9F2vTL0NnI2E0ku0EIVupql3IY=', 'Content-Length': '0'} 2022-09-03T06:37:24.068 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c033c5790d07559d-006312f624-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.072 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:24.073 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:24.074 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.075 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.076 DEBUG:boto:Method: GET 2022-09-03T06:37:24.077 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.078 DEBUG:boto:Data: 2022-09-03T06:37:24.079 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.080 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:24.081 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.082 DEBUG:boto:Params: {} 2022-09-03T06:37:24.083 DEBUG:boto:Token: None 2022-09-03T06:37:24.084 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:24.085 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J9F2vTL0NnI2E0ku0EIVupql3IY= 2022-09-03T06:37:24.085 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J9F2vTL0NnI2E0ku0EIVupql3IY=', 'Content-Length': '0'} 2022-09-03T06:37:24.088 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000131ec8b24799b05c-006312f624-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.091 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:24.092 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:24.095 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:24.096 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:24.097 DEBUG:boto:Method: GET 2022-09-03T06:37:24.098 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:37:24.099 DEBUG:boto:Data: 2022-09-03T06:37:24.100 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.101 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.102 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.103 DEBUG:boto:Params: {} 2022-09-03T06:37:24.104 DEBUG:boto:Token: None 2022-09-03T06:37:24.105 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/ 2022-09-03T06:37:24.105 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g= 2022-09-03T06:37:24.106 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g=', 'Content-Length': '0'} 2022-09-03T06:37:24.110 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f61e3d3d1e17b938-006312f624-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.150 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:24.152 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:24.153 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:24.154 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:24.154 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:24.155 DEBUG:boto:Method: GET 2022-09-03T06:37:24.156 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:37:24.157 DEBUG:boto:Data: 2022-09-03T06:37:24.157 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.158 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:24.159 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.160 DEBUG:boto:Params: {} 2022-09-03T06:37:24.160 DEBUG:boto:Token: None 2022-09-03T06:37:24.161 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/ 2022-09-03T06:37:24.162 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g= 2022-09-03T06:37:24.163 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g=', 'Content-Length': '0'} 2022-09-03T06:37:24.166 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000059495e54da7c25c0-006312f624-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.167 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:24.168 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:24.169 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:24.169 DEBUG:boto:Method: HEAD 2022-09-03T06:37:24.170 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:37:24.170 DEBUG:boto:Data: 2022-09-03T06:37:24.171 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.171 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.172 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.172 DEBUG:boto:Params: {} 2022-09-03T06:37:24.173 DEBUG:boto:Token: None 2022-09-03T06:37:24.173 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj33 2022-09-03T06:37:24.174 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HnJ6+DpMpJLitd+Z4Lzb71C5SC0= 2022-09-03T06:37:24.175 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HnJ6+DpMpJLitd+Z4Lzb71C5SC0=', 'Content-Length': '0'} 2022-09-03T06:37:24.177 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006d3305612c81e94a-006312f624-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.178 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:24.178 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:24.179 DEBUG:boto:Method: HEAD 2022-09-03T06:37:24.179 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:37:24.180 DEBUG:boto:Data: 2022-09-03T06:37:24.180 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.181 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:24.181 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.182 DEBUG:boto:Params: {} 2022-09-03T06:37:24.183 DEBUG:boto:Token: None 2022-09-03T06:37:24.184 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj33 2022-09-03T06:37:24.185 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HnJ6+DpMpJLitd+Z4Lzb71C5SC0= 2022-09-03T06:37:24.186 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HnJ6+DpMpJLitd+Z4Lzb71C5SC0=', 'Content-Length': '0'} 2022-09-03T06:37:24.188 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008ca28508d1106b66-006312f624-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.189 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:37:24.190 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:24.191 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:24.191 DEBUG:boto:Method: GET 2022-09-03T06:37:24.192 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:37:24.192 DEBUG:boto:Data: 2022-09-03T06:37:24.193 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.193 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.194 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.194 DEBUG:boto:Params: {} 2022-09-03T06:37:24.195 DEBUG:boto:Token: None 2022-09-03T06:37:24.196 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj33 2022-09-03T06:37:24.196 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1SWzMNkkzvM4JMfDAQoGNQdWhqs= 2022-09-03T06:37:24.197 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1SWzMNkkzvM4JMfDAQoGNQdWhqs=', 'Content-Length': '0'} 2022-09-03T06:37:24.199 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d30ca12cf82406fa-006312f624-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.200 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:24.201 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:24.201 DEBUG:boto:Method: GET 2022-09-03T06:37:24.202 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:37:24.202 DEBUG:boto:Data: 2022-09-03T06:37:24.203 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.203 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:24.204 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.204 DEBUG:boto:Params: {} 2022-09-03T06:37:24.205 DEBUG:boto:Token: None 2022-09-03T06:37:24.206 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj33 2022-09-03T06:37:24.208 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1SWzMNkkzvM4JMfDAQoGNQdWhqs= 2022-09-03T06:37:24.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1SWzMNkkzvM4JMfDAQoGNQdWhqs=', 'Content-Length': '0'} 2022-09-03T06:37:24.211 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000021514b3e3b01a2b-006312f624-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.211 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:24.212 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:24.213 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:24.213 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:24.214 DEBUG:boto:Method: GET 2022-09-03T06:37:24.214 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:37:24.215 DEBUG:boto:Data: 2022-09-03T06:37:24.215 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.216 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.216 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.217 DEBUG:boto:Params: {} 2022-09-03T06:37:24.218 DEBUG:boto:Token: None 2022-09-03T06:37:24.219 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/ 2022-09-03T06:37:24.219 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g= 2022-09-03T06:37:24.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g=', 'Content-Length': '0'} 2022-09-03T06:37:24.223 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000086a1b50baa6610c-006312f624-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.223 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:24.224 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:24.225 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:24.225 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:24.226 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:24.226 DEBUG:boto:Method: GET 2022-09-03T06:37:24.227 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:37:24.228 DEBUG:boto:Data: 2022-09-03T06:37:24.228 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.229 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:24.229 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.230 DEBUG:boto:Params: {} 2022-09-03T06:37:24.230 DEBUG:boto:Token: None 2022-09-03T06:37:24.231 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/ 2022-09-03T06:37:24.232 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g= 2022-09-03T06:37:24.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g=', 'Content-Length': '0'} 2022-09-03T06:37:24.235 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000033549b96a213b4f3-006312f624-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.236 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:24.237 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:37:24.237 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:24.238 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:24.239 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.239 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.240 DEBUG:boto:Method: GET 2022-09-03T06:37:24.241 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.242 DEBUG:boto:Data: 2022-09-03T06:37:24.242 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.243 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.243 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.244 DEBUG:boto:Params: {} 2022-09-03T06:37:24.245 DEBUG:boto:Token: None 2022-09-03T06:37:24.246 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.246 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pWI8D7w5gEN3OuszJCKALlRhfoY= 2022-09-03T06:37:24.247 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pWI8D7w5gEN3OuszJCKALlRhfoY=', 'Content-Length': '0'} 2022-09-03T06:37:24.249 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b21921d95798d93e-006312f624-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.250 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:24.251 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:24.251 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.252 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.252 DEBUG:boto:Method: GET 2022-09-03T06:37:24.253 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.253 DEBUG:boto:Data: 2022-09-03T06:37:24.254 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.254 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:24.255 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.256 DEBUG:boto:Params: {} 2022-09-03T06:37:24.256 DEBUG:boto:Token: None 2022-09-03T06:37:24.257 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.257 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pWI8D7w5gEN3OuszJCKALlRhfoY= 2022-09-03T06:37:24.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pWI8D7w5gEN3OuszJCKALlRhfoY=', 'Content-Length': '0'} 2022-09-03T06:37:24.261 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000987e6428c1cc56d0-006312f624-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.306 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:24.307 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:24.308 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.309 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.310 DEBUG:boto:Method: HEAD 2022-09-03T06:37:24.311 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.312 DEBUG:boto:Data: 2022-09-03T06:37:24.313 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.313 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.314 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.315 DEBUG:boto:Params: {} 2022-09-03T06:37:24.316 DEBUG:boto:Token: None 2022-09-03T06:37:24.316 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.317 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3ySfazRgBwQOY7bj/uiGaxZXARg= 2022-09-03T06:37:24.317 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3ySfazRgBwQOY7bj/uiGaxZXARg=', 'Content-Length': '0'} 2022-09-03T06:37:24.320 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008de3ce3f1666384b-006312f624-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.320 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:24.321 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:24.321 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.322 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.323 DEBUG:boto:Method: HEAD 2022-09-03T06:37:24.323 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.324 DEBUG:boto:Data: 2022-09-03T06:37:24.324 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.325 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:24.325 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.326 DEBUG:boto:Params: {} 2022-09-03T06:37:24.326 DEBUG:boto:Token: None 2022-09-03T06:37:24.327 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.327 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3ySfazRgBwQOY7bj/uiGaxZXARg= 2022-09-03T06:37:24.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3ySfazRgBwQOY7bj/uiGaxZXARg=', 'Content-Length': '0'} 2022-09-03T06:37:24.330 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b5cbbbcf4e024025-006312f624-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.331 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:37:24.332 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:24.332 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:24.333 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.333 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.334 DEBUG:boto:Method: GET 2022-09-03T06:37:24.334 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.335 DEBUG:boto:Data: 2022-09-03T06:37:24.336 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.336 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.337 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.337 DEBUG:boto:Params: {} 2022-09-03T06:37:24.338 DEBUG:boto:Token: None 2022-09-03T06:37:24.339 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.340 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pWI8D7w5gEN3OuszJCKALlRhfoY= 2022-09-03T06:37:24.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pWI8D7w5gEN3OuszJCKALlRhfoY=', 'Content-Length': '0'} 2022-09-03T06:37:24.343 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004cc8c3556dfe8c92-006312f624-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.343 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:24.344 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:24.345 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.345 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.346 DEBUG:boto:Method: GET 2022-09-03T06:37:24.346 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.347 DEBUG:boto:Data: 2022-09-03T06:37:24.347 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.348 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:24.348 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.349 DEBUG:boto:Params: {} 2022-09-03T06:37:24.350 DEBUG:boto:Token: None 2022-09-03T06:37:24.350 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:24.351 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pWI8D7w5gEN3OuszJCKALlRhfoY= 2022-09-03T06:37:24.351 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pWI8D7w5gEN3OuszJCKALlRhfoY=', 'Content-Length': '0'} 2022-09-03T06:37:24.354 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000acea20524a3a0c41-006312f624-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.355 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:24.355 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:24.356 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:24.356 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:24.357 DEBUG:boto:Method: GET 2022-09-03T06:37:24.357 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:37:24.358 DEBUG:boto:Data: 2022-09-03T06:37:24.359 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.359 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.360 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.360 DEBUG:boto:Params: {} 2022-09-03T06:37:24.361 DEBUG:boto:Token: None 2022-09-03T06:37:24.361 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/ 2022-09-03T06:37:24.362 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g= 2022-09-03T06:37:24.363 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g=', 'Content-Length': '0'} 2022-09-03T06:37:24.366 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dd75b8984180d93b-006312f624-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.410 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:24.412 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:24.412 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:24.413 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:24.413 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:24.414 DEBUG:boto:Method: GET 2022-09-03T06:37:24.414 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:37:24.415 DEBUG:boto:Data: 2022-09-03T06:37:24.416 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.416 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:24.417 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.417 DEBUG:boto:Params: {} 2022-09-03T06:37:24.418 DEBUG:boto:Token: None 2022-09-03T06:37:24.418 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/ 2022-09-03T06:37:24.419 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g= 2022-09-03T06:37:24.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g=', 'Content-Length': '0'} 2022-09-03T06:37:24.422 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dd0da6ae6240ed2c-006312f624-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.423 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:24.424 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:24.424 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:24.425 DEBUG:boto:Method: HEAD 2022-09-03T06:37:24.427 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:37:24.427 DEBUG:boto:Data: 2022-09-03T06:37:24.428 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.429 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.430 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.432 DEBUG:boto:Params: {} 2022-09-03T06:37:24.433 DEBUG:boto:Token: None 2022-09-03T06:37:24.434 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj44 2022-09-03T06:37:24.435 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4FhzBIxkKEGtt6UH8TnAm07pY44= 2022-09-03T06:37:24.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4FhzBIxkKEGtt6UH8TnAm07pY44=', 'Content-Length': '0'} 2022-09-03T06:37:24.439 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a20898dc3cb776fc-006312f624-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.440 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:24.441 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:24.442 DEBUG:boto:Method: HEAD 2022-09-03T06:37:24.443 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:37:24.445 DEBUG:boto:Data: 2022-09-03T06:37:24.445 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.446 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:24.447 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.448 DEBUG:boto:Params: {} 2022-09-03T06:37:24.449 DEBUG:boto:Token: None 2022-09-03T06:37:24.450 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj44 2022-09-03T06:37:24.451 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4FhzBIxkKEGtt6UH8TnAm07pY44= 2022-09-03T06:37:24.453 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4FhzBIxkKEGtt6UH8TnAm07pY44=', 'Content-Length': '0'} 2022-09-03T06:37:24.456 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000094d5b62f62035c70-006312f624-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.457 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:37:24.458 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:24.459 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:24.460 DEBUG:boto:Method: GET 2022-09-03T06:37:24.461 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:37:24.462 DEBUG:boto:Data: 2022-09-03T06:37:24.463 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.464 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.465 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.466 DEBUG:boto:Params: {} 2022-09-03T06:37:24.467 DEBUG:boto:Token: None 2022-09-03T06:37:24.468 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj44 2022-09-03T06:37:24.469 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+4i8rPvbld4MIO0abIOCIJDC5rk= 2022-09-03T06:37:24.471 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+4i8rPvbld4MIO0abIOCIJDC5rk=', 'Content-Length': '0'} 2022-09-03T06:37:24.473 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eda9151db8b1f8f0-006312f624-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.475 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:24.476 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:24.477 DEBUG:boto:Method: GET 2022-09-03T06:37:24.478 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:37:24.479 DEBUG:boto:Data: 2022-09-03T06:37:24.480 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.481 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:24.482 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.483 DEBUG:boto:Params: {} 2022-09-03T06:37:24.484 DEBUG:boto:Token: None 2022-09-03T06:37:24.485 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj44 2022-09-03T06:37:24.486 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+4i8rPvbld4MIO0abIOCIJDC5rk= 2022-09-03T06:37:24.487 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+4i8rPvbld4MIO0abIOCIJDC5rk=', 'Content-Length': '0'} 2022-09-03T06:37:24.491 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c75036942920dfb6-006312f624-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.530 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:24.531 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:24.531 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:24.532 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:24.532 DEBUG:boto:Method: GET 2022-09-03T06:37:24.533 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:37:24.533 DEBUG:boto:Data: 2022-09-03T06:37:24.534 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.535 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.535 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.536 DEBUG:boto:Params: {} 2022-09-03T06:37:24.536 DEBUG:boto:Token: None 2022-09-03T06:37:24.537 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/ 2022-09-03T06:37:24.538 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g= 2022-09-03T06:37:24.538 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g=', 'Content-Length': '0'} 2022-09-03T06:37:24.541 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000048e39162852ebf55-006312f624-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.542 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:24.543 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:24.544 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:24.544 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:24.545 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:24.546 DEBUG:boto:Method: GET 2022-09-03T06:37:24.546 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:37:24.547 DEBUG:boto:Data: 2022-09-03T06:37:24.547 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.548 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:24.549 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.550 DEBUG:boto:Params: {} 2022-09-03T06:37:24.550 DEBUG:boto:Token: None 2022-09-03T06:37:24.551 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/ 2022-09-03T06:37:24.552 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g= 2022-09-03T06:37:24.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g=', 'Content-Length': '0'} 2022-09-03T06:37:24.555 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000049063a784bb9a02f-006312f624-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.556 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:24.557 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:37:24.557 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:24.558 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:24.559 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.559 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.560 DEBUG:boto:Method: GET 2022-09-03T06:37:24.561 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.561 DEBUG:boto:Data: 2022-09-03T06:37:24.562 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.562 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.563 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.563 DEBUG:boto:Params: {} 2022-09-03T06:37:24.564 DEBUG:boto:Token: None 2022-09-03T06:37:24.565 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.565 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SAiD9T8O/enQCjBJK5eXKN1glus= 2022-09-03T06:37:24.566 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SAiD9T8O/enQCjBJK5eXKN1glus=', 'Content-Length': '0'} 2022-09-03T06:37:24.580 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000012d4c026cfd3d53d-006312f624-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.581 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:24.582 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:24.583 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.583 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.584 DEBUG:boto:Method: GET 2022-09-03T06:37:24.585 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.585 DEBUG:boto:Data: 2022-09-03T06:37:24.586 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.588 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:24.588 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.589 DEBUG:boto:Params: {} 2022-09-03T06:37:24.590 DEBUG:boto:Token: None 2022-09-03T06:37:24.590 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.591 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SAiD9T8O/enQCjBJK5eXKN1glus= 2022-09-03T06:37:24.592 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SAiD9T8O/enQCjBJK5eXKN1glus=', 'Content-Length': '0'} 2022-09-03T06:37:24.594 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e98586b4d4affcb-006312f624-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.595 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:24.596 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:24.597 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.597 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.598 DEBUG:boto:Method: HEAD 2022-09-03T06:37:24.599 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.599 DEBUG:boto:Data: 2022-09-03T06:37:24.600 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.600 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.601 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.601 DEBUG:boto:Params: {} 2022-09-03T06:37:24.602 DEBUG:boto:Token: None 2022-09-03T06:37:24.603 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.604 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6BpxEcu7AcoTpBAtmreyuQzyV+s= 2022-09-03T06:37:24.604 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6BpxEcu7AcoTpBAtmreyuQzyV+s=', 'Content-Length': '0'} 2022-09-03T06:37:24.606 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b9933590f8f97a74-006312f624-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.608 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:24.610 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:24.610 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.611 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.611 DEBUG:boto:Method: HEAD 2022-09-03T06:37:24.612 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.613 DEBUG:boto:Data: 2022-09-03T06:37:24.613 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.614 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:24.615 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.615 DEBUG:boto:Params: {} 2022-09-03T06:37:24.616 DEBUG:boto:Token: None 2022-09-03T06:37:24.616 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.617 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6BpxEcu7AcoTpBAtmreyuQzyV+s= 2022-09-03T06:37:24.618 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6BpxEcu7AcoTpBAtmreyuQzyV+s=', 'Content-Length': '0'} 2022-09-03T06:37:24.623 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008d35271704606e08-006312f624-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.624 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:37:24.624 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:24.625 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:24.625 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.626 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.627 DEBUG:boto:Method: GET 2022-09-03T06:37:24.627 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.628 DEBUG:boto:Data: 2022-09-03T06:37:24.629 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.629 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.630 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.631 DEBUG:boto:Params: {} 2022-09-03T06:37:24.634 DEBUG:boto:Token: None 2022-09-03T06:37:24.635 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.636 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SAiD9T8O/enQCjBJK5eXKN1glus= 2022-09-03T06:37:24.637 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SAiD9T8O/enQCjBJK5eXKN1glus=', 'Content-Length': '0'} 2022-09-03T06:37:24.639 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000aaeb6527dcd37f4-006312f624-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.684 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:24.685 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:24.686 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.687 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.688 DEBUG:boto:Method: GET 2022-09-03T06:37:24.689 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.691 DEBUG:boto:Data: 2022-09-03T06:37:24.692 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.692 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:24.693 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.694 DEBUG:boto:Params: {} 2022-09-03T06:37:24.695 DEBUG:boto:Token: None 2022-09-03T06:37:24.697 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:24.698 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SAiD9T8O/enQCjBJK5eXKN1glus= 2022-09-03T06:37:24.699 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SAiD9T8O/enQCjBJK5eXKN1glus=', 'Content-Length': '0'} 2022-09-03T06:37:24.701 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c65e473595c8942-006312f624-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.704 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:24.705 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:24.706 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:24.707 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:24.708 DEBUG:boto:Method: GET 2022-09-03T06:37:24.708 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:37:24.709 DEBUG:boto:Data: 2022-09-03T06:37:24.710 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.711 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.712 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.712 DEBUG:boto:Params: {} 2022-09-03T06:37:24.713 DEBUG:boto:Token: None 2022-09-03T06:37:24.714 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/ 2022-09-03T06:37:24.715 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g= 2022-09-03T06:37:24.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g=', 'Content-Length': '0'} 2022-09-03T06:37:24.719 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008e8bc497ba02b80c-006312f624-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.720 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:24.721 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:24.722 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:24.723 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:24.724 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:24.725 DEBUG:boto:Method: GET 2022-09-03T06:37:24.726 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:37:24.727 DEBUG:boto:Data: 2022-09-03T06:37:24.728 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.729 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:24.730 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.731 DEBUG:boto:Params: {} 2022-09-03T06:37:24.731 DEBUG:boto:Token: None 2022-09-03T06:37:24.732 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/ 2022-09-03T06:37:24.733 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g= 2022-09-03T06:37:24.733 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g=', 'Content-Length': '0'} 2022-09-03T06:37:24.736 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008c5a6924d5f2af34-006312f624-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.737 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:24.738 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:24.738 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:24.739 DEBUG:boto:Method: HEAD 2022-09-03T06:37:24.740 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:37:24.740 DEBUG:boto:Data: 2022-09-03T06:37:24.741 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.741 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.742 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.742 DEBUG:boto:Params: {} 2022-09-03T06:37:24.743 DEBUG:boto:Token: None 2022-09-03T06:37:24.743 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj55 2022-09-03T06:37:24.744 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CdG/hScju/24BgMUTFzcZhlB++M= 2022-09-03T06:37:24.745 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CdG/hScju/24BgMUTFzcZhlB++M=', 'Content-Length': '0'} 2022-09-03T06:37:24.748 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d98236bd81aa3adf-006312f624-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.748 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:24.749 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:24.749 DEBUG:boto:Method: HEAD 2022-09-03T06:37:24.750 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:37:24.751 DEBUG:boto:Data: 2022-09-03T06:37:24.751 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.752 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:24.753 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.754 DEBUG:boto:Params: {} 2022-09-03T06:37:24.755 DEBUG:boto:Token: None 2022-09-03T06:37:24.756 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj55 2022-09-03T06:37:24.757 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CdG/hScju/24BgMUTFzcZhlB++M= 2022-09-03T06:37:24.758 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CdG/hScju/24BgMUTFzcZhlB++M=', 'Content-Length': '0'} 2022-09-03T06:37:24.762 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000cb8f086508aa914-006312f624-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.763 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:37:24.764 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:24.765 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:24.765 DEBUG:boto:Method: GET 2022-09-03T06:37:24.766 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:37:24.767 DEBUG:boto:Data: 2022-09-03T06:37:24.767 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.768 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.768 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.770 DEBUG:boto:Params: {} 2022-09-03T06:37:24.771 DEBUG:boto:Token: None 2022-09-03T06:37:24.771 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj55 2022-09-03T06:37:24.772 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q8AUoOyXgah4IClY60WrmGq+lw4= 2022-09-03T06:37:24.773 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q8AUoOyXgah4IClY60WrmGq+lw4=', 'Content-Length': '0'} 2022-09-03T06:37:24.775 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ddb805c142272259-006312f624-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.776 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:24.776 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:24.777 DEBUG:boto:Method: GET 2022-09-03T06:37:24.778 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:37:24.778 DEBUG:boto:Data: 2022-09-03T06:37:24.779 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.780 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:24.780 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.781 DEBUG:boto:Params: {} 2022-09-03T06:37:24.782 DEBUG:boto:Token: None 2022-09-03T06:37:24.782 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj55 2022-09-03T06:37:24.783 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q8AUoOyXgah4IClY60WrmGq+lw4= 2022-09-03T06:37:24.784 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q8AUoOyXgah4IClY60WrmGq+lw4=', 'Content-Length': '0'} 2022-09-03T06:37:24.786 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e1511c1422d41308-006312f624-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.788 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:24.789 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:24.790 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:24.791 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:24.792 DEBUG:boto:Method: GET 2022-09-03T06:37:24.792 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:37:24.793 DEBUG:boto:Data: 2022-09-03T06:37:24.795 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.796 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.797 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.798 DEBUG:boto:Params: {} 2022-09-03T06:37:24.799 DEBUG:boto:Token: None 2022-09-03T06:37:24.800 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/ 2022-09-03T06:37:24.801 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g= 2022-09-03T06:37:24.803 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g=', 'Content-Length': '0'} 2022-09-03T06:37:24.807 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003a86b7345ab19201-006312f624-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.859 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:24.861 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:24.861 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:24.862 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:24.862 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:24.863 DEBUG:boto:Method: GET 2022-09-03T06:37:24.864 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:37:24.864 DEBUG:boto:Data: 2022-09-03T06:37:24.865 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.865 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:24.866 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.867 DEBUG:boto:Params: {} 2022-09-03T06:37:24.867 DEBUG:boto:Token: None 2022-09-03T06:37:24.869 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/ 2022-09-03T06:37:24.870 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g= 2022-09-03T06:37:24.870 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FTr3l1mswkdRo/3diXh1N2ur28g=', 'Content-Length': '0'} 2022-09-03T06:37:24.873 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005bb19989e2e9648a-006312f624-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.874 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:24.875 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-75 zones={test-zone3, test-zone1} 2022-09-03T06:37:24.876 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-75 zones={test-zone3, test-zone2} 2022-09-03T06:37:24.877 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:24.878 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:24.879 DEBUG:boto:Method: HEAD 2022-09-03T06:37:24.880 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:37:24.881 DEBUG:boto:Data: 2022-09-03T06:37:24.882 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.884 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.885 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.887 DEBUG:boto:Params: {} 2022-09-03T06:37:24.887 DEBUG:boto:Token: None 2022-09-03T06:37:24.888 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/ 2022-09-03T06:37:24.889 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GIBgU2WL0I1AK0U5HuGgqZLnZHw= 2022-09-03T06:37:24.889 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GIBgU2WL0I1AK0U5HuGgqZLnZHw=', 'Content-Length': '0'} 2022-09-03T06:37:24.893 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '15'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000901fa7625bd02959-006312f624-1851-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.894 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:24.894 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:24.895 DEBUG:boto:Method: HEAD 2022-09-03T06:37:24.895 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:37:24.896 DEBUG:boto:Data: 2022-09-03T06:37:24.896 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.897 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:24.898 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.899 DEBUG:boto:Params: {} 2022-09-03T06:37:24.899 DEBUG:boto:Token: None 2022-09-03T06:37:24.900 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/ 2022-09-03T06:37:24.901 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GIBgU2WL0I1AK0U5HuGgqZLnZHw= 2022-09-03T06:37:24.901 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GIBgU2WL0I1AK0U5HuGgqZLnZHw=', 'Content-Length': '0'} 2022-09-03T06:37:24.906 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '15'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000023fd60240f70de9e-006312f624-188c-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.907 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:37:24.907 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:24.908 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:24.909 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:24.909 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:24.910 DEBUG:boto:Method: GET 2022-09-03T06:37:24.911 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:24.912 DEBUG:boto:Data: 2022-09-03T06:37:24.913 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.913 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.914 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.915 DEBUG:boto:Params: {} 2022-09-03T06:37:24.915 DEBUG:boto:Token: None 2022-09-03T06:37:24.916 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/?versions 2022-09-03T06:37:24.917 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:12RH+nMZjc1CPivUYc05D2tky7I= 2022-09-03T06:37:24.918 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:12RH+nMZjc1CPivUYc05D2tky7I=', 'Content-Length': '0'} 2022-09-03T06:37:24.921 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009328d8b8f0b7b2f7-006312f624-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.922 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:24.923 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:37:24.924 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:37:24.924 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:37:24.925 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:37:24.925 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:37:24.926 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:37:24.927 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:24.928 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:24.929 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:24.929 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:24.930 DEBUG:boto:Method: GET 2022-09-03T06:37:24.930 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:24.931 DEBUG:boto:Data: 2022-09-03T06:37:24.931 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.932 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:24.932 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.933 DEBUG:boto:Params: {} 2022-09-03T06:37:24.934 DEBUG:boto:Token: None 2022-09-03T06:37:24.934 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/?versions 2022-09-03T06:37:24.935 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:12RH+nMZjc1CPivUYc05D2tky7I= 2022-09-03T06:37:24.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:12RH+nMZjc1CPivUYc05D2tky7I=', 'Content-Length': '0'} 2022-09-03T06:37:24.942 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fcc6a354b7038afd-006312f624-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.942 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:24.944 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:37:24.945 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:37:24.945 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:37:24.946 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:37:24.946 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:37:24.947 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:24.947 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:24.948 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:24.949 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:24.950 DEBUG:boto:Method: GET 2022-09-03T06:37:24.950 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:24.951 DEBUG:boto:Data: 2022-09-03T06:37:24.951 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.952 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.953 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.953 DEBUG:boto:Params: {} 2022-09-03T06:37:24.955 DEBUG:boto:Token: None 2022-09-03T06:37:24.956 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/?versions 2022-09-03T06:37:24.956 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:12RH+nMZjc1CPivUYc05D2tky7I= 2022-09-03T06:37:24.957 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:12RH+nMZjc1CPivUYc05D2tky7I=', 'Content-Length': '0'} 2022-09-03T06:37:24.963 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f66ac909dfc209c9-006312f624-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.963 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:24.965 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:24.965 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:24.966 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:24.966 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:24.967 DEBUG:boto:Method: GET 2022-09-03T06:37:24.968 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:37:24.968 DEBUG:boto:Data: 2022-09-03T06:37:24.970 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.970 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:24.971 DEBUG:boto:Port: 8000 2022-09-03T06:37:24.972 DEBUG:boto:Params: {} 2022-09-03T06:37:24.972 DEBUG:boto:Token: None 2022-09-03T06:37:24.973 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/?versions 2022-09-03T06:37:24.974 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:12RH+nMZjc1CPivUYc05D2tky7I= 2022-09-03T06:37:24.975 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:12RH+nMZjc1CPivUYc05D2tky7I=', 'Content-Length': '0'} 2022-09-03T06:37:24.978 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000065ee122b4a1ef239-006312f624-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.979 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:24.980 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:37:24.980 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:24.981 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:24.981 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:24.982 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:24.983 DEBUG:boto:Method: GET 2022-09-03T06:37:24.983 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:24.984 DEBUG:boto:Data: 2022-09-03T06:37:24.984 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.985 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:24.985 DEBUG:boto:Port: 8001 2022-09-03T06:37:24.986 DEBUG:boto:Params: {} 2022-09-03T06:37:24.987 DEBUG:boto:Token: None 2022-09-03T06:37:24.988 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:24 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:24.989 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dE4+nVOCr7DYfSyB5K3ZqeT9jL8= 2022-09-03T06:37:24.989 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dE4+nVOCr7DYfSyB5K3ZqeT9jL8=', 'Content-Length': '0'} 2022-09-03T06:37:24.991 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000623eec0098f626a9-006312f624-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:24.992 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:24.993 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:24.993 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:24.994 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:24.995 DEBUG:boto:Method: GET 2022-09-03T06:37:24.995 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:24.996 DEBUG:boto:Data: 2022-09-03T06:37:24.996 DEBUG:boto:Headers: {} 2022-09-03T06:37:24.997 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:24.997 DEBUG:boto:Port: 8000 2022-09-03T06:37:25.000 DEBUG:boto:Params: {} 2022-09-03T06:37:25.002 DEBUG:boto:Token: None 2022-09-03T06:37:25.003 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:25.005 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3uw0TKJzb6nTr+KhOs2BKwH+d3w= 2022-09-03T06:37:25.006 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3uw0TKJzb6nTr+KhOs2BKwH+d3w=', 'Content-Length': '0'} 2022-09-03T06:37:25.009 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c830536998aa76d3-006312f625-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.051 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:25.052 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:25.053 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:25.053 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:25.054 DEBUG:boto:Method: HEAD 2022-09-03T06:37:25.055 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:25.056 DEBUG:boto:Data: 2022-09-03T06:37:25.057 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.059 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:25.062 DEBUG:boto:Port: 8001 2022-09-03T06:37:25.063 DEBUG:boto:Params: {} 2022-09-03T06:37:25.064 DEBUG:boto:Token: None 2022-09-03T06:37:25.065 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:25.066 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4u4fhC8Fu1X+/16Ht6M4qb9t/ME= 2022-09-03T06:37:25.067 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4u4fhC8Fu1X+/16Ht6M4qb9t/ME=', 'Content-Length': '0'} 2022-09-03T06:37:25.070 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000055f8a0fe944223d8-006312f625-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.072 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:25.072 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:25.073 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:25.074 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:25.074 DEBUG:boto:Method: HEAD 2022-09-03T06:37:25.075 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:25.076 DEBUG:boto:Data: 2022-09-03T06:37:25.077 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.078 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:25.078 DEBUG:boto:Port: 8000 2022-09-03T06:37:25.079 DEBUG:boto:Params: {} 2022-09-03T06:37:25.080 DEBUG:boto:Token: None 2022-09-03T06:37:25.081 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:25.083 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4u4fhC8Fu1X+/16Ht6M4qb9t/ME= 2022-09-03T06:37:25.085 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4u4fhC8Fu1X+/16Ht6M4qb9t/ME=', 'Content-Length': '0'} 2022-09-03T06:37:25.088 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f569570f5803e6d8-006312f625-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.090 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:37:25.091 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:25.092 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:25.093 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:25.094 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:25.095 DEBUG:boto:Method: GET 2022-09-03T06:37:25.096 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:25.097 DEBUG:boto:Data: 2022-09-03T06:37:25.099 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.100 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:25.100 DEBUG:boto:Port: 8001 2022-09-03T06:37:25.103 DEBUG:boto:Params: {} 2022-09-03T06:37:25.104 DEBUG:boto:Token: None 2022-09-03T06:37:25.105 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:25.106 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3uw0TKJzb6nTr+KhOs2BKwH+d3w= 2022-09-03T06:37:25.110 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3uw0TKJzb6nTr+KhOs2BKwH+d3w=', 'Content-Length': '0'} 2022-09-03T06:37:25.113 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000782d93130f710001-006312f625-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.113 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:25.114 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:25.115 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:25.115 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:25.116 DEBUG:boto:Method: GET 2022-09-03T06:37:25.117 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:25.118 DEBUG:boto:Data: 2022-09-03T06:37:25.253 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.254 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:25.255 DEBUG:boto:Port: 8000 2022-09-03T06:37:25.255 DEBUG:boto:Params: {} 2022-09-03T06:37:25.256 DEBUG:boto:Token: None 2022-09-03T06:37:25.257 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:37:25.258 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3uw0TKJzb6nTr+KhOs2BKwH+d3w= 2022-09-03T06:37:25.259 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3uw0TKJzb6nTr+KhOs2BKwH+d3w=', 'Content-Length': '0'} 2022-09-03T06:37:25.269 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005d7c7ef0baf36b27-006312f625-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.275 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:25.276 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:25.277 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:25.277 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:25.278 DEBUG:boto:Method: GET 2022-09-03T06:37:25.279 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:37:25.279 DEBUG:boto:Data: 2022-09-03T06:37:25.280 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.280 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:25.281 DEBUG:boto:Port: 8001 2022-09-03T06:37:25.282 DEBUG:boto:Params: {} 2022-09-03T06:37:25.284 DEBUG:boto:Token: None 2022-09-03T06:37:25.285 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/ 2022-09-03T06:37:25.286 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N8c9i8+7TB1mLrjDdozM5VFlG38= 2022-09-03T06:37:25.287 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N8c9i8+7TB1mLrjDdozM5VFlG38=', 'Content-Length': '0'} 2022-09-03T06:37:25.297 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000004b2ca20d920ec7-006312f625-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.309 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:25.311 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:25.312 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:25.313 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:25.313 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:25.314 DEBUG:boto:Method: GET 2022-09-03T06:37:25.315 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:37:25.315 DEBUG:boto:Data: 2022-09-03T06:37:25.316 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.317 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:25.318 DEBUG:boto:Port: 8000 2022-09-03T06:37:25.319 DEBUG:boto:Params: {} 2022-09-03T06:37:25.320 DEBUG:boto:Token: None 2022-09-03T06:37:25.321 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/ 2022-09-03T06:37:25.323 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N8c9i8+7TB1mLrjDdozM5VFlG38= 2022-09-03T06:37:25.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N8c9i8+7TB1mLrjDdozM5VFlG38=', 'Content-Length': '0'} 2022-09-03T06:37:25.339 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004aead939d50ccc35-006312f625-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.341 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:25.343 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:25.344 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:25.345 DEBUG:boto:Method: HEAD 2022-09-03T06:37:25.346 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:37:25.347 DEBUG:boto:Data: 2022-09-03T06:37:25.348 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.350 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:25.352 DEBUG:boto:Port: 8001 2022-09-03T06:37:25.353 DEBUG:boto:Params: {} 2022-09-03T06:37:25.354 DEBUG:boto:Token: None 2022-09-03T06:37:25.355 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj11 2022-09-03T06:37:25.356 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:haECMTOjDczYaiPmtLMd5BQ6zvQ= 2022-09-03T06:37:25.358 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:haECMTOjDczYaiPmtLMd5BQ6zvQ=', 'Content-Length': '0'} 2022-09-03T06:37:25.364 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000067ea7a0d063da197-006312f625-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.366 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:25.368 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:25.369 DEBUG:boto:Method: HEAD 2022-09-03T06:37:25.370 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:37:25.371 DEBUG:boto:Data: 2022-09-03T06:37:25.372 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.374 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:25.375 DEBUG:boto:Port: 8000 2022-09-03T06:37:25.376 DEBUG:boto:Params: {} 2022-09-03T06:37:25.377 DEBUG:boto:Token: None 2022-09-03T06:37:25.378 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj11 2022-09-03T06:37:25.379 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:haECMTOjDczYaiPmtLMd5BQ6zvQ= 2022-09-03T06:37:25.381 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:haECMTOjDczYaiPmtLMd5BQ6zvQ=', 'Content-Length': '0'} 2022-09-03T06:37:25.384 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee807b3330327c84-006312f625-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.386 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:37:25.387 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:25.389 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:25.390 DEBUG:boto:Method: GET 2022-09-03T06:37:25.391 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:37:25.392 DEBUG:boto:Data: 2022-09-03T06:37:25.393 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.395 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:25.396 DEBUG:boto:Port: 8001 2022-09-03T06:37:25.397 DEBUG:boto:Params: {} 2022-09-03T06:37:25.398 DEBUG:boto:Token: None 2022-09-03T06:37:25.400 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj11 2022-09-03T06:37:25.401 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:moheVRYFMV40ABfZJkTjqI3PV4Y= 2022-09-03T06:37:25.407 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:moheVRYFMV40ABfZJkTjqI3PV4Y=', 'Content-Length': '0'} 2022-09-03T06:37:25.411 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026e6bca5d13be389-006312f625-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.413 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:37:25.414 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:37:25.415 DEBUG:boto:Method: GET 2022-09-03T06:37:25.416 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:37:25.418 DEBUG:boto:Data: 2022-09-03T06:37:25.419 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.420 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:25.421 DEBUG:boto:Port: 8000 2022-09-03T06:37:25.422 DEBUG:boto:Params: {} 2022-09-03T06:37:25.423 DEBUG:boto:Token: None 2022-09-03T06:37:25.424 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj11 2022-09-03T06:37:25.426 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:moheVRYFMV40ABfZJkTjqI3PV4Y= 2022-09-03T06:37:25.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:moheVRYFMV40ABfZJkTjqI3PV4Y=', 'Content-Length': '0'} 2022-09-03T06:37:25.446 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000030c9cfad5f9d4af-006312f625-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.447 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:25.448 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:25.449 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:25.450 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:25.451 DEBUG:boto:Method: GET 2022-09-03T06:37:25.453 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:37:25.454 DEBUG:boto:Data: 2022-09-03T06:37:25.455 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.455 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:25.456 DEBUG:boto:Port: 8001 2022-09-03T06:37:25.457 DEBUG:boto:Params: {} 2022-09-03T06:37:25.458 DEBUG:boto:Token: None 2022-09-03T06:37:25.459 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/ 2022-09-03T06:37:25.459 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N8c9i8+7TB1mLrjDdozM5VFlG38= 2022-09-03T06:37:25.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N8c9i8+7TB1mLrjDdozM5VFlG38=', 'Content-Length': '0'} 2022-09-03T06:37:25.463 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ceeeb359f6ca4ac3-006312f625-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.464 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:25.466 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:25.466 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:25.468 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:25.469 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:37:25.470 DEBUG:boto:Method: GET 2022-09-03T06:37:25.470 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:37:25.471 DEBUG:boto:Data: 2022-09-03T06:37:25.472 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.472 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:25.473 DEBUG:boto:Port: 8000 2022-09-03T06:37:25.473 DEBUG:boto:Params: {} 2022-09-03T06:37:25.474 DEBUG:boto:Token: None 2022-09-03T06:37:25.475 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/ 2022-09-03T06:37:25.476 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N8c9i8+7TB1mLrjDdozM5VFlG38= 2022-09-03T06:37:25.477 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N8c9i8+7TB1mLrjDdozM5VFlG38=', 'Content-Length': '0'} 2022-09-03T06:37:25.482 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b994d932502e6ba5-006312f625-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.483 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:25.483 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:37:25.484 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:25.485 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:25.485 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.486 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.486 DEBUG:boto:Method: GET 2022-09-03T06:37:25.487 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.487 DEBUG:boto:Data: 2022-09-03T06:37:25.488 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.488 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:25.489 DEBUG:boto:Port: 8001 2022-09-03T06:37:25.490 DEBUG:boto:Params: {} 2022-09-03T06:37:25.491 DEBUG:boto:Token: None 2022-09-03T06:37:25.492 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.492 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ii4kH2bKGi0nibISUfClf8QKOwA= 2022-09-03T06:37:25.493 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ii4kH2bKGi0nibISUfClf8QKOwA=', 'Content-Length': '0'} 2022-09-03T06:37:25.498 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000efe377053dabbea4-006312f625-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.547 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:25.548 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:25.549 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.550 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.551 DEBUG:boto:Method: GET 2022-09-03T06:37:25.552 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.553 DEBUG:boto:Data: 2022-09-03T06:37:25.555 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.555 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:25.556 DEBUG:boto:Port: 8000 2022-09-03T06:37:25.557 DEBUG:boto:Params: {} 2022-09-03T06:37:25.558 DEBUG:boto:Token: None 2022-09-03T06:37:25.559 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.560 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ii4kH2bKGi0nibISUfClf8QKOwA= 2022-09-03T06:37:25.561 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ii4kH2bKGi0nibISUfClf8QKOwA=', 'Content-Length': '0'} 2022-09-03T06:37:25.563 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000043adfa67976ea46f-006312f625-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.565 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:25.567 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:25.567 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.568 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.569 DEBUG:boto:Method: HEAD 2022-09-03T06:37:25.570 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.571 DEBUG:boto:Data: 2022-09-03T06:37:25.571 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.572 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:25.573 DEBUG:boto:Port: 8001 2022-09-03T06:37:25.573 DEBUG:boto:Params: {} 2022-09-03T06:37:25.575 DEBUG:boto:Token: None 2022-09-03T06:37:25.576 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.577 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kTUwakDcXkiyqivw9Qv+lwe7hiA= 2022-09-03T06:37:25.578 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kTUwakDcXkiyqivw9Qv+lwe7hiA=', 'Content-Length': '0'} 2022-09-03T06:37:25.588 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003bee6eff55b99520-006312f625-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.589 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:25.589 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:25.590 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.591 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.592 DEBUG:boto:Method: HEAD 2022-09-03T06:37:25.592 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.593 DEBUG:boto:Data: 2022-09-03T06:37:25.594 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.595 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:25.595 DEBUG:boto:Port: 8000 2022-09-03T06:37:25.596 DEBUG:boto:Params: {} 2022-09-03T06:37:25.597 DEBUG:boto:Token: None 2022-09-03T06:37:25.598 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.599 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kTUwakDcXkiyqivw9Qv+lwe7hiA= 2022-09-03T06:37:25.599 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kTUwakDcXkiyqivw9Qv+lwe7hiA=', 'Content-Length': '0'} 2022-09-03T06:37:25.602 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000966b5dd4ec0f8467-006312f625-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.604 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:37:25.605 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:25.606 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:25.607 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.607 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.608 DEBUG:boto:Method: GET 2022-09-03T06:37:25.608 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.610 DEBUG:boto:Data: 2022-09-03T06:37:25.611 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.611 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:25.612 DEBUG:boto:Port: 8001 2022-09-03T06:37:25.612 DEBUG:boto:Params: {} 2022-09-03T06:37:25.614 DEBUG:boto:Token: None 2022-09-03T06:37:25.615 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.616 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ii4kH2bKGi0nibISUfClf8QKOwA= 2022-09-03T06:37:25.617 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ii4kH2bKGi0nibISUfClf8QKOwA=', 'Content-Length': '0'} 2022-09-03T06:37:25.620 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bed2ba32fe34bcaa-006312f625-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.622 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:25.622 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:25.623 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.624 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.624 DEBUG:boto:Method: GET 2022-09-03T06:37:25.625 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.626 DEBUG:boto:Data: 2022-09-03T06:37:25.627 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.628 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:25.629 DEBUG:boto:Port: 8000 2022-09-03T06:37:25.630 DEBUG:boto:Params: {} 2022-09-03T06:37:25.631 DEBUG:boto:Token: None 2022-09-03T06:37:25.631 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:37:25.632 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ii4kH2bKGi0nibISUfClf8QKOwA= 2022-09-03T06:37:25.633 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ii4kH2bKGi0nibISUfClf8QKOwA=', 'Content-Length': '0'} 2022-09-03T06:37:25.639 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e85d4dd2e9cf3879-006312f625-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.689 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:25.690 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:25.692 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:25.692 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:25.693 DEBUG:boto:Method: GET 2022-09-03T06:37:25.694 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:37:25.695 DEBUG:boto:Data: 2022-09-03T06:37:25.696 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.697 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:25.697 DEBUG:boto:Port: 8001 2022-09-03T06:37:25.698 DEBUG:boto:Params: {} 2022-09-03T06:37:25.700 DEBUG:boto:Token: None 2022-09-03T06:37:25.701 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/ 2022-09-03T06:37:25.706 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N8c9i8+7TB1mLrjDdozM5VFlG38= 2022-09-03T06:37:25.707 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N8c9i8+7TB1mLrjDdozM5VFlG38=', 'Content-Length': '0'} 2022-09-03T06:37:25.712 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b7ce70d5c56ce996-006312f625-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.715 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:25.718 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:25.718 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:25.719 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:25.720 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:25.720 DEBUG:boto:Method: GET 2022-09-03T06:37:25.721 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:37:25.721 DEBUG:boto:Data: 2022-09-03T06:37:25.723 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.723 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:25.724 DEBUG:boto:Port: 8000 2022-09-03T06:37:25.724 DEBUG:boto:Params: {} 2022-09-03T06:37:25.725 DEBUG:boto:Token: None 2022-09-03T06:37:25.725 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/ 2022-09-03T06:37:25.726 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N8c9i8+7TB1mLrjDdozM5VFlG38= 2022-09-03T06:37:25.727 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N8c9i8+7TB1mLrjDdozM5VFlG38=', 'Content-Length': '0'} 2022-09-03T06:37:25.736 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c81f32a8cff43bf7-006312f625-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.743 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:25.745 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:25.746 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:25.747 DEBUG:boto:Method: HEAD 2022-09-03T06:37:25.747 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:37:25.748 DEBUG:boto:Data: 2022-09-03T06:37:25.749 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.750 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:25.751 DEBUG:boto:Port: 8001 2022-09-03T06:37:25.752 DEBUG:boto:Params: {} 2022-09-03T06:37:25.753 DEBUG:boto:Token: None 2022-09-03T06:37:25.754 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj22 2022-09-03T06:37:25.755 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YpFDmrI2Ap5lELbXM71l60Rw208= 2022-09-03T06:37:25.755 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YpFDmrI2Ap5lELbXM71l60Rw208=', 'Content-Length': '0'} 2022-09-03T06:37:25.759 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005dbe51047c7a1f8d-006312f625-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.761 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:25.762 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:25.763 DEBUG:boto:Method: HEAD 2022-09-03T06:37:25.765 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:37:25.765 DEBUG:boto:Data: 2022-09-03T06:37:25.766 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.766 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:25.767 DEBUG:boto:Port: 8000 2022-09-03T06:37:25.768 DEBUG:boto:Params: {} 2022-09-03T06:37:25.768 DEBUG:boto:Token: None 2022-09-03T06:37:25.769 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj22 2022-09-03T06:37:25.770 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YpFDmrI2Ap5lELbXM71l60Rw208= 2022-09-03T06:37:25.771 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YpFDmrI2Ap5lELbXM71l60Rw208=', 'Content-Length': '0'} 2022-09-03T06:37:25.773 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e87ea367823cf308-006312f625-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.775 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:37:25.775 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:25.776 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:25.777 DEBUG:boto:Method: GET 2022-09-03T06:37:25.778 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:37:25.779 DEBUG:boto:Data: 2022-09-03T06:37:25.780 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.781 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:25.782 DEBUG:boto:Port: 8001 2022-09-03T06:37:25.783 DEBUG:boto:Params: {} 2022-09-03T06:37:25.784 DEBUG:boto:Token: None 2022-09-03T06:37:25.785 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj22 2022-09-03T06:37:25.786 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q7pNBIAVj6BuLEsM1LfCUwBIXic= 2022-09-03T06:37:25.787 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q7pNBIAVj6BuLEsM1LfCUwBIXic=', 'Content-Length': '0'} 2022-09-03T06:37:25.800 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b5b462edd4c72a5e-006312f625-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.802 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:37:25.803 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:37:25.805 DEBUG:boto:Method: GET 2022-09-03T06:37:25.805 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:37:25.806 DEBUG:boto:Data: 2022-09-03T06:37:25.807 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.808 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:25.808 DEBUG:boto:Port: 8000 2022-09-03T06:37:25.809 DEBUG:boto:Params: {} 2022-09-03T06:37:25.811 DEBUG:boto:Token: None 2022-09-03T06:37:25.811 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj22 2022-09-03T06:37:25.812 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q7pNBIAVj6BuLEsM1LfCUwBIXic= 2022-09-03T06:37:25.813 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q7pNBIAVj6BuLEsM1LfCUwBIXic=', 'Content-Length': '0'} 2022-09-03T06:37:25.820 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b6f58c0789a51986-006312f625-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.822 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:25.823 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:25.824 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:25.825 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:25.825 DEBUG:boto:Method: GET 2022-09-03T06:37:25.827 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:37:25.828 DEBUG:boto:Data: 2022-09-03T06:37:25.829 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.830 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:25.831 DEBUG:boto:Port: 8001 2022-09-03T06:37:25.832 DEBUG:boto:Params: {} 2022-09-03T06:37:25.833 DEBUG:boto:Token: None 2022-09-03T06:37:25.834 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/ 2022-09-03T06:37:25.835 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N8c9i8+7TB1mLrjDdozM5VFlG38= 2022-09-03T06:37:25.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N8c9i8+7TB1mLrjDdozM5VFlG38=', 'Content-Length': '0'} 2022-09-03T06:37:25.851 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c741a18baea671b2-006312f625-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.856 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:25.858 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:25.859 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:25.862 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:25.863 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:37:25.864 DEBUG:boto:Method: GET 2022-09-03T06:37:25.865 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:37:25.866 DEBUG:boto:Data: 2022-09-03T06:37:25.867 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.868 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:25.869 DEBUG:boto:Port: 8000 2022-09-03T06:37:25.870 DEBUG:boto:Params: {} 2022-09-03T06:37:25.871 DEBUG:boto:Token: None 2022-09-03T06:37:25.872 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/ 2022-09-03T06:37:25.876 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N8c9i8+7TB1mLrjDdozM5VFlG38= 2022-09-03T06:37:25.877 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N8c9i8+7TB1mLrjDdozM5VFlG38=', 'Content-Length': '0'} 2022-09-03T06:37:25.881 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000a08eb0e76dca9a2-006312f625-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.892 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:25.893 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:37:25.894 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:25.895 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:25.895 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:25.896 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:25.896 DEBUG:boto:Method: GET 2022-09-03T06:37:25.897 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:25.898 DEBUG:boto:Data: 2022-09-03T06:37:25.899 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.899 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:25.900 DEBUG:boto:Port: 8001 2022-09-03T06:37:25.900 DEBUG:boto:Params: {} 2022-09-03T06:37:25.901 DEBUG:boto:Token: None 2022-09-03T06:37:25.902 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:25.903 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m/hWMOGD/jVbhmIIEx4GRWxh2C0= 2022-09-03T06:37:25.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m/hWMOGD/jVbhmIIEx4GRWxh2C0=', 'Content-Length': '0'} 2022-09-03T06:37:25.907 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b8bb1541a1f7b5c1-006312f625-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.908 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:25.909 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:25.910 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:25.911 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:25.912 DEBUG:boto:Method: GET 2022-09-03T06:37:25.913 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:25.914 DEBUG:boto:Data: 2022-09-03T06:37:25.915 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.916 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:25.917 DEBUG:boto:Port: 8000 2022-09-03T06:37:25.918 DEBUG:boto:Params: {} 2022-09-03T06:37:25.919 DEBUG:boto:Token: None 2022-09-03T06:37:25.920 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:25.921 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m/hWMOGD/jVbhmIIEx4GRWxh2C0= 2022-09-03T06:37:25.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m/hWMOGD/jVbhmIIEx4GRWxh2C0=', 'Content-Length': '0'} 2022-09-03T06:37:25.924 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003d9d6238d8c251f9-006312f625-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.926 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:25.927 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:25.928 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:25.929 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:25.930 DEBUG:boto:Method: HEAD 2022-09-03T06:37:25.931 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:25.932 DEBUG:boto:Data: 2022-09-03T06:37:25.933 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.934 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:25.935 DEBUG:boto:Port: 8001 2022-09-03T06:37:25.936 DEBUG:boto:Params: {} 2022-09-03T06:37:25.937 DEBUG:boto:Token: None 2022-09-03T06:37:25.938 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:25.940 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tZ6neqt+Ab5d7GdfuDYlHjHGQQg= 2022-09-03T06:37:25.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tZ6neqt+Ab5d7GdfuDYlHjHGQQg=', 'Content-Length': '0'} 2022-09-03T06:37:25.945 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e95a9404ce14bf1-006312f625-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.947 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:25.949 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:25.949 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:25.950 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:25.951 DEBUG:boto:Method: HEAD 2022-09-03T06:37:25.953 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:25.955 DEBUG:boto:Data: 2022-09-03T06:37:25.956 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.956 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:25.957 DEBUG:boto:Port: 8000 2022-09-03T06:37:25.958 DEBUG:boto:Params: {} 2022-09-03T06:37:25.959 DEBUG:boto:Token: None 2022-09-03T06:37:25.960 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:25.961 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tZ6neqt+Ab5d7GdfuDYlHjHGQQg= 2022-09-03T06:37:25.962 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tZ6neqt+Ab5d7GdfuDYlHjHGQQg=', 'Content-Length': '0'} 2022-09-03T06:37:25.977 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f4ef3a2b37eee073-006312f625-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:25.979 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:37:25.982 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:25.983 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:25.983 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:25.985 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:25.986 DEBUG:boto:Method: GET 2022-09-03T06:37:25.988 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:25.989 DEBUG:boto:Data: 2022-09-03T06:37:25.990 DEBUG:boto:Headers: {} 2022-09-03T06:37:25.991 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:25.991 DEBUG:boto:Port: 8001 2022-09-03T06:37:25.992 DEBUG:boto:Params: {} 2022-09-03T06:37:25.994 DEBUG:boto:Token: None 2022-09-03T06:37:25.995 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:25 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:25.996 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m/hWMOGD/jVbhmIIEx4GRWxh2C0= 2022-09-03T06:37:25.998 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m/hWMOGD/jVbhmIIEx4GRWxh2C0=', 'Content-Length': '0'} 2022-09-03T06:37:26.001 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001679c71d5017ffbb-006312f625-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.003 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:26.004 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:26.005 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:26.006 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:26.007 DEBUG:boto:Method: GET 2022-09-03T06:37:26.008 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:26.009 DEBUG:boto:Data: 2022-09-03T06:37:26.009 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.010 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:26.010 DEBUG:boto:Port: 8000 2022-09-03T06:37:26.011 DEBUG:boto:Params: {} 2022-09-03T06:37:26.012 DEBUG:boto:Token: None 2022-09-03T06:37:26.012 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:37:26.014 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vP+a0Vs+aEQnKJt6m1D+jExzewc= 2022-09-03T06:37:26.015 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vP+a0Vs+aEQnKJt6m1D+jExzewc=', 'Content-Length': '0'} 2022-09-03T06:37:26.020 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2203cce7c7309af-006312f626-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.022 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:26.023 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:26.023 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:26.024 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:26.025 DEBUG:boto:Method: GET 2022-09-03T06:37:26.026 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:37:26.027 DEBUG:boto:Data: 2022-09-03T06:37:26.028 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.029 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:26.030 DEBUG:boto:Port: 8001 2022-09-03T06:37:26.031 DEBUG:boto:Params: {} 2022-09-03T06:37:26.031 DEBUG:boto:Token: None 2022-09-03T06:37:26.032 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/ 2022-09-03T06:37:26.033 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0= 2022-09-03T06:37:26.034 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0=', 'Content-Length': '0'} 2022-09-03T06:37:26.037 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000df902fa6e813b114-006312f626-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.038 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:26.040 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:26.041 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:26.042 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:26.043 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:26.047 DEBUG:boto:Method: GET 2022-09-03T06:37:26.047 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:37:26.048 DEBUG:boto:Data: 2022-09-03T06:37:26.048 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.049 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:26.049 DEBUG:boto:Port: 8000 2022-09-03T06:37:26.050 DEBUG:boto:Params: {} 2022-09-03T06:37:26.051 DEBUG:boto:Token: None 2022-09-03T06:37:26.051 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/ 2022-09-03T06:37:26.052 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0= 2022-09-03T06:37:26.053 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0=', 'Content-Length': '0'} 2022-09-03T06:37:26.057 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c474d6991399ad3a-006312f626-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.099 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:26.101 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:26.102 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:26.103 DEBUG:boto:Method: HEAD 2022-09-03T06:37:26.103 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:37:26.104 DEBUG:boto:Data: 2022-09-03T06:37:26.105 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.106 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:26.107 DEBUG:boto:Port: 8001 2022-09-03T06:37:26.107 DEBUG:boto:Params: {} 2022-09-03T06:37:26.108 DEBUG:boto:Token: None 2022-09-03T06:37:26.109 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj33 2022-09-03T06:37:26.110 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:11NFE2R94w6zFeLmVzBwdMgxGME= 2022-09-03T06:37:26.110 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:11NFE2R94w6zFeLmVzBwdMgxGME=', 'Content-Length': '0'} 2022-09-03T06:37:26.114 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e9e9a5ff2d7f3ac7-006312f626-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.116 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:26.117 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:26.118 DEBUG:boto:Method: HEAD 2022-09-03T06:37:26.119 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:37:26.120 DEBUG:boto:Data: 2022-09-03T06:37:26.121 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.122 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:26.123 DEBUG:boto:Port: 8000 2022-09-03T06:37:26.124 DEBUG:boto:Params: {} 2022-09-03T06:37:26.125 DEBUG:boto:Token: None 2022-09-03T06:37:26.126 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj33 2022-09-03T06:37:26.127 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:11NFE2R94w6zFeLmVzBwdMgxGME= 2022-09-03T06:37:26.129 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:11NFE2R94w6zFeLmVzBwdMgxGME=', 'Content-Length': '0'} 2022-09-03T06:37:26.132 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c260a101a25b8cb-006312f626-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.133 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:37:26.134 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:26.134 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:26.135 DEBUG:boto:Method: GET 2022-09-03T06:37:26.136 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:37:26.137 DEBUG:boto:Data: 2022-09-03T06:37:26.138 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.148 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:26.149 DEBUG:boto:Port: 8001 2022-09-03T06:37:26.150 DEBUG:boto:Params: {} 2022-09-03T06:37:26.151 DEBUG:boto:Token: None 2022-09-03T06:37:26.152 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj33 2022-09-03T06:37:26.153 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GJnlFF7m2/DuwbjIqT6vEy95Ln4= 2022-09-03T06:37:26.155 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GJnlFF7m2/DuwbjIqT6vEy95Ln4=', 'Content-Length': '0'} 2022-09-03T06:37:26.160 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bec7cd1fe3bb7c21-006312f626-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.162 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:37:26.164 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:37:26.165 DEBUG:boto:Method: GET 2022-09-03T06:37:26.166 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:37:26.167 DEBUG:boto:Data: 2022-09-03T06:37:26.168 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.169 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:26.170 DEBUG:boto:Port: 8000 2022-09-03T06:37:26.171 DEBUG:boto:Params: {} 2022-09-03T06:37:26.172 DEBUG:boto:Token: None 2022-09-03T06:37:26.172 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj33 2022-09-03T06:37:26.173 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GJnlFF7m2/DuwbjIqT6vEy95Ln4= 2022-09-03T06:37:26.174 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GJnlFF7m2/DuwbjIqT6vEy95Ln4=', 'Content-Length': '0'} 2022-09-03T06:37:26.176 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009cba11a984d543cc-006312f626-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.177 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:26.178 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:26.179 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:26.180 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:26.180 DEBUG:boto:Method: GET 2022-09-03T06:37:26.181 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:37:26.181 DEBUG:boto:Data: 2022-09-03T06:37:26.182 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.183 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:26.184 DEBUG:boto:Port: 8001 2022-09-03T06:37:26.185 DEBUG:boto:Params: {} 2022-09-03T06:37:26.185 DEBUG:boto:Token: None 2022-09-03T06:37:26.187 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/ 2022-09-03T06:37:26.188 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0= 2022-09-03T06:37:26.189 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0=', 'Content-Length': '0'} 2022-09-03T06:37:26.193 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b0088af7c197d571-006312f626-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.194 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:26.196 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:26.197 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:26.198 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:26.198 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:37:26.199 DEBUG:boto:Method: GET 2022-09-03T06:37:26.200 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:37:26.201 DEBUG:boto:Data: 2022-09-03T06:37:26.202 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.203 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:26.204 DEBUG:boto:Port: 8000 2022-09-03T06:37:26.205 DEBUG:boto:Params: {} 2022-09-03T06:37:26.208 DEBUG:boto:Token: None 2022-09-03T06:37:26.209 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/ 2022-09-03T06:37:26.210 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0= 2022-09-03T06:37:26.211 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0=', 'Content-Length': '0'} 2022-09-03T06:37:26.214 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000000990140f7dea067-006312f626-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.215 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:26.217 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:37:26.218 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:26.218 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:26.219 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.220 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.220 DEBUG:boto:Method: GET 2022-09-03T06:37:26.221 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.227 DEBUG:boto:Data: 2022-09-03T06:37:26.232 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.233 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:26.233 DEBUG:boto:Port: 8001 2022-09-03T06:37:26.234 DEBUG:boto:Params: {} 2022-09-03T06:37:26.235 DEBUG:boto:Token: None 2022-09-03T06:37:26.235 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.236 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6ahiDkZjgrINqmDu6peqyIZGO4s= 2022-09-03T06:37:26.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6ahiDkZjgrINqmDu6peqyIZGO4s=', 'Content-Length': '0'} 2022-09-03T06:37:26.239 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b97854adb0204fbb-006312f626-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.286 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:26.287 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:26.288 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.288 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.289 DEBUG:boto:Method: GET 2022-09-03T06:37:26.289 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.290 DEBUG:boto:Data: 2022-09-03T06:37:26.291 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.291 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:26.292 DEBUG:boto:Port: 8000 2022-09-03T06:37:26.293 DEBUG:boto:Params: {} 2022-09-03T06:37:26.294 DEBUG:boto:Token: None 2022-09-03T06:37:26.294 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.295 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6ahiDkZjgrINqmDu6peqyIZGO4s= 2022-09-03T06:37:26.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6ahiDkZjgrINqmDu6peqyIZGO4s=', 'Content-Length': '0'} 2022-09-03T06:37:26.299 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000babca34d021a67fd-006312f626-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.300 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:26.301 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:26.301 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.302 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.302 DEBUG:boto:Method: HEAD 2022-09-03T06:37:26.303 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.303 DEBUG:boto:Data: 2022-09-03T06:37:26.304 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.304 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:26.305 DEBUG:boto:Port: 8001 2022-09-03T06:37:26.305 DEBUG:boto:Params: {} 2022-09-03T06:37:26.306 DEBUG:boto:Token: None 2022-09-03T06:37:26.306 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.307 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tZUBnTkqF+uZWRRRl6iHYit/Z+M= 2022-09-03T06:37:26.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tZUBnTkqF+uZWRRRl6iHYit/Z+M=', 'Content-Length': '0'} 2022-09-03T06:37:26.311 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000186c9b4722dad3df-006312f626-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.311 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:26.312 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:26.313 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.313 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.314 DEBUG:boto:Method: HEAD 2022-09-03T06:37:26.315 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.315 DEBUG:boto:Data: 2022-09-03T06:37:26.316 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.317 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:26.317 DEBUG:boto:Port: 8000 2022-09-03T06:37:26.318 DEBUG:boto:Params: {} 2022-09-03T06:37:26.318 DEBUG:boto:Token: None 2022-09-03T06:37:26.319 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.321 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tZUBnTkqF+uZWRRRl6iHYit/Z+M= 2022-09-03T06:37:26.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tZUBnTkqF+uZWRRRl6iHYit/Z+M=', 'Content-Length': '0'} 2022-09-03T06:37:26.323 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e79c091bcbc77427-006312f626-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.324 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:37:26.324 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:26.326 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:26.326 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.327 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.327 DEBUG:boto:Method: GET 2022-09-03T06:37:26.328 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.328 DEBUG:boto:Data: 2022-09-03T06:37:26.329 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.330 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:26.331 DEBUG:boto:Port: 8001 2022-09-03T06:37:26.331 DEBUG:boto:Params: {} 2022-09-03T06:37:26.332 DEBUG:boto:Token: None 2022-09-03T06:37:26.333 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.334 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6ahiDkZjgrINqmDu6peqyIZGO4s= 2022-09-03T06:37:26.334 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6ahiDkZjgrINqmDu6peqyIZGO4s=', 'Content-Length': '0'} 2022-09-03T06:37:26.337 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000096d80f042a260b1-006312f626-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.337 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:26.338 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:26.339 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.339 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.340 DEBUG:boto:Method: GET 2022-09-03T06:37:26.340 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.341 DEBUG:boto:Data: 2022-09-03T06:37:26.342 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.342 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:26.343 DEBUG:boto:Port: 8000 2022-09-03T06:37:26.343 DEBUG:boto:Params: {} 2022-09-03T06:37:26.344 DEBUG:boto:Token: None 2022-09-03T06:37:26.345 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:37:26.345 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6ahiDkZjgrINqmDu6peqyIZGO4s= 2022-09-03T06:37:26.346 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6ahiDkZjgrINqmDu6peqyIZGO4s=', 'Content-Length': '0'} 2022-09-03T06:37:26.348 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000078be019821bf43ad-006312f626-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.349 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:26.350 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:26.351 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:26.351 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:26.352 DEBUG:boto:Method: GET 2022-09-03T06:37:26.352 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:37:26.353 DEBUG:boto:Data: 2022-09-03T06:37:26.353 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.354 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:26.354 DEBUG:boto:Port: 8001 2022-09-03T06:37:26.355 DEBUG:boto:Params: {} 2022-09-03T06:37:26.356 DEBUG:boto:Token: None 2022-09-03T06:37:26.357 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/ 2022-09-03T06:37:26.357 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0= 2022-09-03T06:37:26.358 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0=', 'Content-Length': '0'} 2022-09-03T06:37:26.361 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000671fc5211110908f-006312f626-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.405 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:26.406 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:26.407 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:26.408 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:26.408 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:26.409 DEBUG:boto:Method: GET 2022-09-03T06:37:26.409 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:37:26.410 DEBUG:boto:Data: 2022-09-03T06:37:26.411 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.411 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:26.412 DEBUG:boto:Port: 8000 2022-09-03T06:37:26.412 DEBUG:boto:Params: {} 2022-09-03T06:37:26.413 DEBUG:boto:Token: None 2022-09-03T06:37:26.414 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/ 2022-09-03T06:37:26.415 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0= 2022-09-03T06:37:26.415 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0=', 'Content-Length': '0'} 2022-09-03T06:37:26.418 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004ac684dd3360d22d-006312f626-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.419 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:26.420 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:26.421 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:26.421 DEBUG:boto:Method: HEAD 2022-09-03T06:37:26.422 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:37:26.422 DEBUG:boto:Data: 2022-09-03T06:37:26.423 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.423 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:26.424 DEBUG:boto:Port: 8001 2022-09-03T06:37:26.425 DEBUG:boto:Params: {} 2022-09-03T06:37:26.425 DEBUG:boto:Token: None 2022-09-03T06:37:26.426 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj44 2022-09-03T06:37:26.427 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SudsUyTd5N0wyUmP1PQ6JQEYReI= 2022-09-03T06:37:26.427 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SudsUyTd5N0wyUmP1PQ6JQEYReI=', 'Content-Length': '0'} 2022-09-03T06:37:26.429 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000453fb5b21e65e914-006312f626-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.430 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:26.431 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:26.431 DEBUG:boto:Method: HEAD 2022-09-03T06:37:26.432 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:37:26.433 DEBUG:boto:Data: 2022-09-03T06:37:26.433 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.434 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:26.434 DEBUG:boto:Port: 8000 2022-09-03T06:37:26.435 DEBUG:boto:Params: {} 2022-09-03T06:37:26.435 DEBUG:boto:Token: None 2022-09-03T06:37:26.436 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj44 2022-09-03T06:37:26.437 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SudsUyTd5N0wyUmP1PQ6JQEYReI= 2022-09-03T06:37:26.437 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SudsUyTd5N0wyUmP1PQ6JQEYReI=', 'Content-Length': '0'} 2022-09-03T06:37:26.440 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000307d90c4c5615cfb-006312f626-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.441 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:37:26.442 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:26.443 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:26.443 DEBUG:boto:Method: GET 2022-09-03T06:37:26.444 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:37:26.445 DEBUG:boto:Data: 2022-09-03T06:37:26.446 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.447 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:26.447 DEBUG:boto:Port: 8001 2022-09-03T06:37:26.448 DEBUG:boto:Params: {} 2022-09-03T06:37:26.449 DEBUG:boto:Token: None 2022-09-03T06:37:26.450 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj44 2022-09-03T06:37:26.451 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:leOR5Xg3utOvt5t+hDFK18a4O9w= 2022-09-03T06:37:26.451 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:leOR5Xg3utOvt5t+hDFK18a4O9w=', 'Content-Length': '0'} 2022-09-03T06:37:26.454 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a44fd8e85b1e558-006312f626-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.455 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:37:26.456 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:37:26.456 DEBUG:boto:Method: GET 2022-09-03T06:37:26.457 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:37:26.458 DEBUG:boto:Data: 2022-09-03T06:37:26.459 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.460 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:26.461 DEBUG:boto:Port: 8000 2022-09-03T06:37:26.462 DEBUG:boto:Params: {} 2022-09-03T06:37:26.464 DEBUG:boto:Token: None 2022-09-03T06:37:26.465 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj44 2022-09-03T06:37:26.466 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:leOR5Xg3utOvt5t+hDFK18a4O9w= 2022-09-03T06:37:26.467 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:leOR5Xg3utOvt5t+hDFK18a4O9w=', 'Content-Length': '0'} 2022-09-03T06:37:26.470 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cbc250a9d575b0a2-006312f626-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.510 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:26.511 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:26.512 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:26.513 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:26.513 DEBUG:boto:Method: GET 2022-09-03T06:37:26.514 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:37:26.515 DEBUG:boto:Data: 2022-09-03T06:37:26.517 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.517 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:26.518 DEBUG:boto:Port: 8001 2022-09-03T06:37:26.519 DEBUG:boto:Params: {} 2022-09-03T06:37:26.520 DEBUG:boto:Token: None 2022-09-03T06:37:26.521 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/ 2022-09-03T06:37:26.523 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0= 2022-09-03T06:37:26.524 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0=', 'Content-Length': '0'} 2022-09-03T06:37:26.529 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000093b613a0deb13e2-006312f626-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.530 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:26.531 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:26.531 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:26.532 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:26.532 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:37:26.533 DEBUG:boto:Method: GET 2022-09-03T06:37:26.533 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:37:26.534 DEBUG:boto:Data: 2022-09-03T06:37:26.535 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.535 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:26.536 DEBUG:boto:Port: 8000 2022-09-03T06:37:26.536 DEBUG:boto:Params: {} 2022-09-03T06:37:26.537 DEBUG:boto:Token: None 2022-09-03T06:37:26.538 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/ 2022-09-03T06:37:26.538 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0= 2022-09-03T06:37:26.539 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0=', 'Content-Length': '0'} 2022-09-03T06:37:26.542 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a9778109a395e443-006312f626-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.543 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:26.544 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:37:26.544 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:26.545 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:26.546 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.546 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.547 DEBUG:boto:Method: GET 2022-09-03T06:37:26.548 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.548 DEBUG:boto:Data: 2022-09-03T06:37:26.549 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.549 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:26.550 DEBUG:boto:Port: 8001 2022-09-03T06:37:26.551 DEBUG:boto:Params: {} 2022-09-03T06:37:26.552 DEBUG:boto:Token: None 2022-09-03T06:37:26.553 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.554 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IrtynCa0C2Sn9PWtTzVGhRuwZEo= 2022-09-03T06:37:26.555 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IrtynCa0C2Sn9PWtTzVGhRuwZEo=', 'Content-Length': '0'} 2022-09-03T06:37:26.559 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f14a06e8a109536-006312f626-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.560 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:26.561 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:26.562 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.563 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.564 DEBUG:boto:Method: GET 2022-09-03T06:37:26.565 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.566 DEBUG:boto:Data: 2022-09-03T06:37:26.567 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.568 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:26.569 DEBUG:boto:Port: 8000 2022-09-03T06:37:26.570 DEBUG:boto:Params: {} 2022-09-03T06:37:26.571 DEBUG:boto:Token: None 2022-09-03T06:37:26.572 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.573 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IrtynCa0C2Sn9PWtTzVGhRuwZEo= 2022-09-03T06:37:26.574 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IrtynCa0C2Sn9PWtTzVGhRuwZEo=', 'Content-Length': '0'} 2022-09-03T06:37:26.578 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e324c2e2785df819-006312f626-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.579 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:26.580 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:26.581 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.581 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.582 DEBUG:boto:Method: HEAD 2022-09-03T06:37:26.583 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.583 DEBUG:boto:Data: 2022-09-03T06:37:26.584 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.584 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:26.585 DEBUG:boto:Port: 8001 2022-09-03T06:37:26.585 DEBUG:boto:Params: {} 2022-09-03T06:37:26.586 DEBUG:boto:Token: None 2022-09-03T06:37:26.587 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.588 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FGVGcGs5rC1+N0cl+FdQfH0bANY= 2022-09-03T06:37:26.589 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FGVGcGs5rC1+N0cl+FdQfH0bANY=', 'Content-Length': '0'} 2022-09-03T06:37:26.591 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cd7dc956c466d9c5-006312f626-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.592 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:26.592 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:26.593 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.593 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.594 DEBUG:boto:Method: HEAD 2022-09-03T06:37:26.595 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.596 DEBUG:boto:Data: 2022-09-03T06:37:26.597 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.598 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:26.599 DEBUG:boto:Port: 8000 2022-09-03T06:37:26.600 DEBUG:boto:Params: {} 2022-09-03T06:37:26.601 DEBUG:boto:Token: None 2022-09-03T06:37:26.602 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.603 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FGVGcGs5rC1+N0cl+FdQfH0bANY= 2022-09-03T06:37:26.604 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FGVGcGs5rC1+N0cl+FdQfH0bANY=', 'Content-Length': '0'} 2022-09-03T06:37:26.607 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b69e9724f1326834-006312f626-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.609 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:37:26.610 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:26.611 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:26.612 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.613 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.614 DEBUG:boto:Method: GET 2022-09-03T06:37:26.615 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.616 DEBUG:boto:Data: 2022-09-03T06:37:26.617 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.618 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:26.619 DEBUG:boto:Port: 8001 2022-09-03T06:37:26.620 DEBUG:boto:Params: {} 2022-09-03T06:37:26.621 DEBUG:boto:Token: None 2022-09-03T06:37:26.623 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.624 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IrtynCa0C2Sn9PWtTzVGhRuwZEo= 2022-09-03T06:37:26.625 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IrtynCa0C2Sn9PWtTzVGhRuwZEo=', 'Content-Length': '0'} 2022-09-03T06:37:26.628 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d7c7ab968c890ad3-006312f626-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.674 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:26.675 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:26.675 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.676 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.677 DEBUG:boto:Method: GET 2022-09-03T06:37:26.677 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.678 DEBUG:boto:Data: 2022-09-03T06:37:26.679 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.679 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:26.680 DEBUG:boto:Port: 8000 2022-09-03T06:37:26.680 DEBUG:boto:Params: {} 2022-09-03T06:37:26.681 DEBUG:boto:Token: None 2022-09-03T06:37:26.682 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:37:26.682 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IrtynCa0C2Sn9PWtTzVGhRuwZEo= 2022-09-03T06:37:26.683 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IrtynCa0C2Sn9PWtTzVGhRuwZEo=', 'Content-Length': '0'} 2022-09-03T06:37:26.686 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c05cb8520d229381-006312f626-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.687 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:26.688 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:26.688 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:26.689 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:26.690 DEBUG:boto:Method: GET 2022-09-03T06:37:26.690 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:37:26.691 DEBUG:boto:Data: 2022-09-03T06:37:26.692 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.692 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:26.693 DEBUG:boto:Port: 8001 2022-09-03T06:37:26.693 DEBUG:boto:Params: {} 2022-09-03T06:37:26.694 DEBUG:boto:Token: None 2022-09-03T06:37:26.695 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/ 2022-09-03T06:37:26.696 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0= 2022-09-03T06:37:26.697 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0=', 'Content-Length': '0'} 2022-09-03T06:37:26.704 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000276c6a69dfcb6aba-006312f626-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.706 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:26.707 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:26.707 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:26.708 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:26.708 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:26.709 DEBUG:boto:Method: GET 2022-09-03T06:37:26.710 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:37:26.711 DEBUG:boto:Data: 2022-09-03T06:37:26.711 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.712 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:26.713 DEBUG:boto:Port: 8000 2022-09-03T06:37:26.713 DEBUG:boto:Params: {} 2022-09-03T06:37:26.714 DEBUG:boto:Token: None 2022-09-03T06:37:26.715 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/ 2022-09-03T06:37:26.715 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0= 2022-09-03T06:37:26.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0=', 'Content-Length': '0'} 2022-09-03T06:37:26.719 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000257c755c8608ebfc-006312f626-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.720 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:26.721 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:26.722 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:26.723 DEBUG:boto:Method: HEAD 2022-09-03T06:37:26.724 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:37:26.725 DEBUG:boto:Data: 2022-09-03T06:37:26.726 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.727 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:26.728 DEBUG:boto:Port: 8001 2022-09-03T06:37:26.729 DEBUG:boto:Params: {} 2022-09-03T06:37:26.730 DEBUG:boto:Token: None 2022-09-03T06:37:26.730 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj55 2022-09-03T06:37:26.731 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BNW8EwnjbX6QZgJULCf5YCqRVuo= 2022-09-03T06:37:26.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BNW8EwnjbX6QZgJULCf5YCqRVuo=', 'Content-Length': '0'} 2022-09-03T06:37:26.734 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e1d5f566446fdb44-006312f626-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.736 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:26.736 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:26.737 DEBUG:boto:Method: HEAD 2022-09-03T06:37:26.738 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:37:26.739 DEBUG:boto:Data: 2022-09-03T06:37:26.739 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.740 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:26.740 DEBUG:boto:Port: 8000 2022-09-03T06:37:26.741 DEBUG:boto:Params: {} 2022-09-03T06:37:26.741 DEBUG:boto:Token: None 2022-09-03T06:37:26.742 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj55 2022-09-03T06:37:26.742 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BNW8EwnjbX6QZgJULCf5YCqRVuo= 2022-09-03T06:37:26.743 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BNW8EwnjbX6QZgJULCf5YCqRVuo=', 'Content-Length': '0'} 2022-09-03T06:37:26.745 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000300b3962219c70e7-006312f626-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.747 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:37:26.748 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:26.749 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:26.750 DEBUG:boto:Method: GET 2022-09-03T06:37:26.751 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:37:26.752 DEBUG:boto:Data: 2022-09-03T06:37:26.753 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.753 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:26.754 DEBUG:boto:Port: 8001 2022-09-03T06:37:26.756 DEBUG:boto:Params: {} 2022-09-03T06:37:26.757 DEBUG:boto:Token: None 2022-09-03T06:37:26.758 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj55 2022-09-03T06:37:26.759 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gKiIJh1xfy2gDtugPM1XiDymzIw= 2022-09-03T06:37:26.759 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gKiIJh1xfy2gDtugPM1XiDymzIw=', 'Content-Length': '0'} 2022-09-03T06:37:26.761 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000070b68819fc6a615f-006312f626-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.806 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:37:26.807 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:37:26.808 DEBUG:boto:Method: GET 2022-09-03T06:37:26.809 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:37:26.809 DEBUG:boto:Data: 2022-09-03T06:37:26.810 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.811 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:26.811 DEBUG:boto:Port: 8000 2022-09-03T06:37:26.812 DEBUG:boto:Params: {} 2022-09-03T06:37:26.813 DEBUG:boto:Token: None 2022-09-03T06:37:26.813 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/obj55 2022-09-03T06:37:26.814 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gKiIJh1xfy2gDtugPM1XiDymzIw= 2022-09-03T06:37:26.815 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gKiIJh1xfy2gDtugPM1XiDymzIw=', 'Content-Length': '0'} 2022-09-03T06:37:26.817 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000688fe6ba5cab9342-006312f626-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.818 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:26.819 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:26.819 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:26.820 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:26.821 DEBUG:boto:Method: GET 2022-09-03T06:37:26.821 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:37:26.822 DEBUG:boto:Data: 2022-09-03T06:37:26.822 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.823 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:37:26.823 DEBUG:boto:Port: 8001 2022-09-03T06:37:26.824 DEBUG:boto:Params: {} 2022-09-03T06:37:26.825 DEBUG:boto:Token: None 2022-09-03T06:37:26.825 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/ 2022-09-03T06:37:26.826 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0= 2022-09-03T06:37:26.827 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0=', 'Content-Length': '0'} 2022-09-03T06:37:26.829 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a98566b44420871a-006312f626-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.830 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:26.831 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:26.832 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:26.832 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:26.833 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:37:26.833 DEBUG:boto:Method: GET 2022-09-03T06:37:26.834 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:37:26.834 DEBUG:boto:Data: 2022-09-03T06:37:26.835 DEBUG:boto:Headers: {} 2022-09-03T06:37:26.835 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:37:26.836 DEBUG:boto:Port: 8000 2022-09-03T06:37:26.837 DEBUG:boto:Params: {} 2022-09-03T06:37:26.837 DEBUG:boto:Token: None 2022-09-03T06:37:26.838 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:37:26 GMT /dsgvcy-75/ 2022-09-03T06:37:26.839 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0= 2022-09-03T06:37:26.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:shL81qcTESOQ94m1++akp8KVHx0=', 'Content-Length': '0'} 2022-09-03T06:37:26.843 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dc1c321f0dcfcc39-006312f626-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:37:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:26.844 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:37:26.845 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-75 zones={test-zone3, test-zone2} 2022-09-03T06:37:26.846 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--num-shards', '17', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:37:26.846 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--num-shards', '17', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:37:26.848 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --num-shards 17 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T06:37:26.910 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:37:26.911 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:37:26.967 INFO:teuthology.orchestra.run.smithi181.stdout:tenant: 2022-09-03T06:37:26.968 INFO:teuthology.orchestra.run.smithi181.stdout:bucket name: dsgvcy-75 2022-09-03T06:37:26.977 INFO:teuthology.orchestra.run.smithi181.stdout:total entries: 5 2022-09-03T06:37:27.011 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:27.012 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:27.013 DEBUG:boto:Method: HEAD 2022-09-03T06:37:27.013 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:37:27.014 DEBUG:boto:Data: 2022-09-03T06:37:27.014 DEBUG:boto:Headers: {} 2022-09-03T06:37:27.015 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:27.015 DEBUG:boto:Port: 8000 2022-09-03T06:37:27.016 DEBUG:boto:Params: {} 2022-09-03T06:37:27.016 DEBUG:boto:Token: None 2022-09-03T06:37:27.017 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:27 GMT /dsgvcy-75/ 2022-09-03T06:37:27.018 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:erxf1ksW/zxxbYRdTQM425nj1HI= 2022-09-03T06:37:27.018 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:erxf1ksW/zxxbYRdTQM425nj1HI=', 'Content-Length': '0'} 2022-09-03T06:37:27.024 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '5'), ('X-RGW-Bytes-Used', '15'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000237aa15001ff38d0-006312f627-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:27.025 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:37:27.026 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:37:27.026 DEBUG:boto:Method: PUT 2022-09-03T06:37:27.028 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:37:27.029 DEBUG:boto:Data: 2022-09-03T06:37:27.030 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:37:27.031 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:27.032 DEBUG:boto:Port: 8000 2022-09-03T06:37:27.034 DEBUG:boto:Params: {} 2022-09-03T06:37:27.036 DEBUG:boto:Token: None 2022-09-03T06:37:27.037 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:37:27 GMT /dsgvcy-75/obj17 2022-09-03T06:37:27.038 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xaZOOMnZR3boCjpOiSGaKmameZY= 2022-09-03T06:37:27.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xaZOOMnZR3boCjpOiSGaKmameZY='} 2022-09-03T06:37:27.053 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000193d9e5153d3d399-006312f627-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:27.054 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:27.055 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:27.056 DEBUG:boto:Method: HEAD 2022-09-03T06:37:27.056 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:37:27.057 DEBUG:boto:Data: 2022-09-03T06:37:27.057 DEBUG:boto:Headers: {} 2022-09-03T06:37:27.058 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:27.058 DEBUG:boto:Port: 8000 2022-09-03T06:37:27.059 DEBUG:boto:Params: {} 2022-09-03T06:37:27.060 DEBUG:boto:Token: None 2022-09-03T06:37:27.060 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:27 GMT /dsgvcy-75/ 2022-09-03T06:37:27.061 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:erxf1ksW/zxxbYRdTQM425nj1HI= 2022-09-03T06:37:27.062 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:erxf1ksW/zxxbYRdTQM425nj1HI=', 'Content-Length': '0'} 2022-09-03T06:37:27.069 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '6'), ('X-RGW-Bytes-Used', '18'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a4dfc92baca41f61-006312f627-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:27.070 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:37:27.070 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:37:27.071 DEBUG:boto:Method: PUT 2022-09-03T06:37:27.073 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:37:27.074 DEBUG:boto:Data: 2022-09-03T06:37:27.074 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:37:27.075 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:27.075 DEBUG:boto:Port: 8000 2022-09-03T06:37:27.076 DEBUG:boto:Params: {} 2022-09-03T06:37:27.076 DEBUG:boto:Token: None 2022-09-03T06:37:27.077 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:37:27 GMT /dsgvcy-75/obj34 2022-09-03T06:37:27.078 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yems9u+vUHIA4uZIyq0m2T1KMdc= 2022-09-03T06:37:27.078 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yems9u+vUHIA4uZIyq0m2T1KMdc='} 2022-09-03T06:37:27.122 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000001aec7efecd59d1a-006312f627-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:27.124 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:27.125 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:27.125 DEBUG:boto:Method: HEAD 2022-09-03T06:37:27.127 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:37:27.130 DEBUG:boto:Data: 2022-09-03T06:37:27.131 DEBUG:boto:Headers: {} 2022-09-03T06:37:27.132 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:27.132 DEBUG:boto:Port: 8000 2022-09-03T06:37:27.133 DEBUG:boto:Params: {} 2022-09-03T06:37:27.134 DEBUG:boto:Token: None 2022-09-03T06:37:27.135 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:27 GMT /dsgvcy-75/ 2022-09-03T06:37:27.136 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:erxf1ksW/zxxbYRdTQM425nj1HI= 2022-09-03T06:37:27.137 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:erxf1ksW/zxxbYRdTQM425nj1HI=', 'Content-Length': '0'} 2022-09-03T06:37:27.143 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '7'), ('X-RGW-Bytes-Used', '21'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d18d38c456e25dbc-006312f627-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:27.145 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:37:27.146 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:37:27.147 DEBUG:boto:Method: PUT 2022-09-03T06:37:27.148 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:37:27.150 DEBUG:boto:Data: 2022-09-03T06:37:27.151 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:37:27.152 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:27.153 DEBUG:boto:Port: 8000 2022-09-03T06:37:27.154 DEBUG:boto:Params: {} 2022-09-03T06:37:27.155 DEBUG:boto:Token: None 2022-09-03T06:37:27.157 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:37:27 GMT /dsgvcy-75/obj51 2022-09-03T06:37:27.158 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fQ3mjfdOXuZ18DZCc/E4uahzQyc= 2022-09-03T06:37:27.159 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fQ3mjfdOXuZ18DZCc/E4uahzQyc='} 2022-09-03T06:37:27.165 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000c7d094a5fe5f476e-006312f627-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:27.167 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:27.168 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:27.169 DEBUG:boto:Method: HEAD 2022-09-03T06:37:27.169 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:37:27.170 DEBUG:boto:Data: 2022-09-03T06:37:27.171 DEBUG:boto:Headers: {} 2022-09-03T06:37:27.172 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:27.174 DEBUG:boto:Port: 8000 2022-09-03T06:37:27.175 DEBUG:boto:Params: {} 2022-09-03T06:37:27.175 DEBUG:boto:Token: None 2022-09-03T06:37:27.176 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:27 GMT /dsgvcy-75/ 2022-09-03T06:37:27.177 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:erxf1ksW/zxxbYRdTQM425nj1HI= 2022-09-03T06:37:27.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:erxf1ksW/zxxbYRdTQM425nj1HI=', 'Content-Length': '0'} 2022-09-03T06:37:27.180 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '8'), ('X-RGW-Bytes-Used', '24'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e47fdfbefc7926ba-006312f627-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:27.181 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:37:27.182 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:37:27.182 DEBUG:boto:Method: PUT 2022-09-03T06:37:27.183 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:37:27.184 DEBUG:boto:Data: 2022-09-03T06:37:27.184 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:37:27.185 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:27.185 DEBUG:boto:Port: 8000 2022-09-03T06:37:27.186 DEBUG:boto:Params: {} 2022-09-03T06:37:27.187 DEBUG:boto:Token: None 2022-09-03T06:37:27.187 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:37:27 GMT /dsgvcy-75/obj68 2022-09-03T06:37:27.188 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:99hV2Qv1bhc+H/gv7MyKq9oSHHQ= 2022-09-03T06:37:27.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:99hV2Qv1bhc+H/gv7MyKq9oSHHQ='} 2022-09-03T06:37:27.196 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000db89f1b68c93b3ee-006312f627-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:27.196 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:37:27.197 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:37:27.197 DEBUG:boto:Method: HEAD 2022-09-03T06:37:27.198 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:37:27.199 DEBUG:boto:Data: 2022-09-03T06:37:27.199 DEBUG:boto:Headers: {} 2022-09-03T06:37:27.199 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:27.200 DEBUG:boto:Port: 8000 2022-09-03T06:37:27.200 DEBUG:boto:Params: {} 2022-09-03T06:37:27.201 DEBUG:boto:Token: None 2022-09-03T06:37:27.202 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:37:27 GMT /dsgvcy-75/ 2022-09-03T06:37:27.202 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:erxf1ksW/zxxbYRdTQM425nj1HI= 2022-09-03T06:37:27.203 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:erxf1ksW/zxxbYRdTQM425nj1HI=', 'Content-Length': '0'} 2022-09-03T06:37:27.206 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '9'), ('X-RGW-Bytes-Used', '27'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d3c597ecd3df9b38-006312f627-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:27.207 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:37:27.209 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:37:27.209 DEBUG:boto:Method: PUT 2022-09-03T06:37:27.210 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:37:27.210 DEBUG:boto:Data: 2022-09-03T06:37:27.211 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:37:27.212 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:37:27.212 DEBUG:boto:Port: 8000 2022-09-03T06:37:27.213 DEBUG:boto:Params: {} 2022-09-03T06:37:27.214 DEBUG:boto:Token: None 2022-09-03T06:37:27.214 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:37:27 GMT /dsgvcy-75/obj85 2022-09-03T06:37:27.215 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ih/2kvpqjOLcVWjsFoYDPenfXts= 2022-09-03T06:37:27.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:37:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ih/2kvpqjOLcVWjsFoYDPenfXts='} 2022-09-03T06:37:27.259 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000097c44fa85384a76c-006312f627-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:37:27.260 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-75 2022-09-03T06:37:27.260 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:37:27.261 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:37:27.261 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:37:27.325 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:37:27.325 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:37:27.363 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:37:27.361+0000 7fb03157e600 1 waiting to reach latest gen 1, on 0.. 2022-09-03T06:37:32.364 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:37:32.361+0000 7fb03157e600 1 waiting to reach latest gen 1, on 0.. 2022-09-03T06:37:37.368 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:37:37.364+0000 7fb03157e600 1 waiting for incremental sync to catch up: 2022-09-03T06:37:37.369 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , ] 2022-09-03T06:37:37.370 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:37:42.368 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:37:42.366+0000 7fb03157e600 1 waiting for incremental sync to catch up: 2022-09-03T06:37:42.369 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , ] 2022-09-03T06:37:42.369 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:37:47.370 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:37:47.368+0000 7fb03157e600 1 waiting for incremental sync to catch up: 2022-09-03T06:37:47.371 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , ] 2022-09-03T06:37:47.371 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:37:52.372 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:37:52.369+0000 7fb03157e600 1 waiting for incremental sync to catch up: 2022-09-03T06:37:52.373 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , ] 2022-09-03T06:37:52.374 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:37:57.374 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:37:57.370+0000 7fb03157e600 1 waiting for incremental sync to catch up: 2022-09-03T06:37:57.375 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , , , , , , 00000000001.55.5, , , ] 2022-09-03T06:37:57.375 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:38:02.375 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:38:02.372+0000 7fb03157e600 1 waiting for incremental sync to catch up: 2022-09-03T06:38:02.376 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , , , , , , 00000000001.55.5, , , ] 2022-09-03T06:38:02.376 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:38:07.376 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:38:07.374+0000 7fb03157e600 1 waiting for incremental sync to catch up: 2022-09-03T06:38:07.377 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , , , , , , 00000000001.55.5, , , ] 2022-09-03T06:38:07.377 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:38:12.378 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:38:12.376+0000 7fb03157e600 1 waiting for incremental sync to catch up: 2022-09-03T06:38:12.378 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , , , , , , 00000000001.55.5, , , ] 2022-09-03T06:38:12.379 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:38:17.380 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:38:17.377+0000 7fb03157e600 1 bucket sync caught up with source: 2022-09-03T06:38:17.380 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:38:17.381 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:38:17.381 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:38:17.377+0000 7fb03157e600 0 bucket checkpoint complete 2022-09-03T06:38:17.400 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-75 2022-09-03T06:38:17.402 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:38:17.403 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:38:17.404 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:38:17.468 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:38:17.469 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:38:17.510 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:38:17.508+0000 7ff36266a600 1 bucket sync caught up with source: 2022-09-03T06:38:17.511 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:38:17.511 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:38:17.511 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:38:17.508+0000 7ff36266a600 0 bucket checkpoint complete 2022-09-03T06:38:17.536 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-75 2022-09-03T06:38:17.537 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:38:17.537 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:38:17.538 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:38:17.600 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:38:17.601 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:38:17.657 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:38:17.654+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:38:17.658 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , , , , , , , , , ] 2022-09-03T06:38:17.659 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:38:22.658 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:38:22.656+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:38:22.659 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:38:22.659 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:38:27.661 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:38:27.658+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:38:27.662 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:38:27.663 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:38:32.662 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:38:32.660+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:38:32.663 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:38:32.663 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:38:37.664 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:38:37.662+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:38:37.664 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:38:37.665 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:38:42.665 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:38:42.663+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:38:42.666 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:38:42.666 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:38:47.667 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:38:47.664+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:38:47.668 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:38:47.668 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:38:52.669 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:38:52.666+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:38:52.670 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:38:52.671 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:38:57.671 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:38:57.668+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:38:57.671 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:38:57.671 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:39:02.672 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:39:02.670+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:39:02.672 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:39:02.673 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:39:07.674 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:39:07.671+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:39:07.674 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:39:07.674 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:39:12.675 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:39:12.673+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:39:12.676 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:39:12.676 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:39:17.677 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:39:17.675+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:39:17.678 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:39:17.679 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:39:22.679 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:39:22.676+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:39:22.679 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:39:22.680 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:39:27.680 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:39:27.678+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:39:27.681 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:39:27.681 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:39:32.684 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:39:32.680+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:39:32.685 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:39:32.686 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:39:37.684 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:39:37.682+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:39:37.685 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:39:37.685 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:39:42.686 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:39:42.683+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:39:42.687 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:39:42.687 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:39:47.698 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:39:47.685+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:39:47.699 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:39:47.699 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:39:52.689 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:39:52.687+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:39:52.690 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:39:52.690 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:39:57.691 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:39:57.689+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:39:57.691 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:39:57.692 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:40:02.692 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:40:02.690+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:40:02.693 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:40:02.693 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:40:07.694 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:40:07.692+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:40:07.695 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:40:07.695 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:40:12.696 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:40:12.694+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:40:12.697 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:40:12.697 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:40:17.704 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:40:17.696+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:40:17.707 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:40:17.707 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:40:22.700 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:40:22.697+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:40:22.700 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:40:22.701 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:40:27.701 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:40:27.699+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:40:27.702 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:40:27.703 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:40:32.703 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:40:32.699+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:40:32.704 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:40:32.704 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:40:37.716 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:40:37.701+0000 7f564af8b600 1 waiting for incremental sync to catch up: 2022-09-03T06:40:37.717 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , , 00000000001.55.5, , , ] 2022-09-03T06:40:37.717 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:40:42.706 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:40:42.703+0000 7f564af8b600 1 bucket sync caught up with source: 2022-09-03T06:40:42.707 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:40:42.707 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000001.70.5, , , , , , 00000000001.65.5, , 00000000001.57.5, , , , 00000000001.35.5, 00000000001.55.5, , , ] 2022-09-03T06:40:42.707 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:40:42.703+0000 7f564af8b600 0 bucket checkpoint complete 2022-09-03T06:40:42.729 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-75 2022-09-03T06:40:42.732 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:40:42.733 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:40:42.738 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:40:42.804 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:40:42.804 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:40:42.843 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:40:42.841+0000 7fd25e586600 1 bucket sync caught up with source: 2022-09-03T06:40:42.843 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.576.5, 00000000001.731.5, , , 00000000001.356.5, , , 00000000001.841.5, 00000000003.302.5, 00000000003.579.5] 2022-09-03T06:40:42.844 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.576.5, 00000000001.731.5, , , 00000000001.356.5, , , 00000000001.841.5, 00000000003.302.5, 00000000003.579.5] 2022-09-03T06:40:42.844 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:40:42.841+0000 7fd25e586600 0 bucket checkpoint complete 2022-09-03T06:40:42.857 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-75 2022-09-03T06:40:42.858 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:40:42.859 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:40:42.859 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:40:42.918 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:40:42.918 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:40:42.961 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:40:42.960+0000 7f7b871c7600 1 bucket sync caught up with source: 2022-09-03T06:40:42.963 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.576.5, 00000000001.731.5, , , 00000000001.356.5, , , 00000000001.841.5, 00000000003.302.5, 00000000003.579.5] 2022-09-03T06:40:42.964 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.576.5, 00000000001.731.5, , , 00000000001.356.5, , , 00000000001.841.5, 00000000003.302.5, 00000000003.579.5] 2022-09-03T06:40:42.964 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:40:42.960+0000 7f7b871c7600 0 bucket checkpoint complete 2022-09-03T06:40:42.986 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-75 2022-09-03T06:40:42.987 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-75 2022-09-03T06:40:42.987 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:40:42.988 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:40:42.989 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:40:43.047 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:40:43.047 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:40:43.084 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:40:43.080+0000 7fd0a6cd9600 1 bucket sync caught up with source: 2022-09-03T06:40:43.085 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000001.198.5, 00000000001.151.5, , , 00000000001.265.5, , , 00000000001.307.5, 00000000003.239.5, 00000000003.284.5] 2022-09-03T06:40:43.085 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000001.198.5, 00000000001.151.5, , , 00000000001.265.5, , , 00000000001.307.5, 00000000003.239.5, 00000000003.284.5] 2022-09-03T06:40:43.085 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:40:43.080+0000 7fd0a6cd9600 0 bucket checkpoint complete 2022-09-03T06:40:43.098 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-75 2022-09-03T06:40:43.099 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:40:43.100 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:40:43.102 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:40:43.171 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:40:43.171 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:40:43.212 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:40:43.205+0000 7f1bf793e600 1 bucket sync caught up with source: 2022-09-03T06:40:43.213 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000001.198.5, 00000000001.151.5, , , 00000000001.265.5, , , 00000000001.307.5, 00000000003.239.5, 00000000003.284.5] 2022-09-03T06:40:43.213 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000001.198.5, 00000000001.151.5, , , 00000000001.265.5, , , 00000000001.307.5, 00000000003.239.5, 00000000003.284.5] 2022-09-03T06:40:43.214 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:40:43.205+0000 7f1bf793e600 0 bucket checkpoint complete 2022-09-03T06:40:43.233 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-75 2022-09-03T06:40:43.234 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-75 zones={test-zone2, test-zone1} 2022-09-03T06:40:43.235 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:43.236 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:43.237 DEBUG:boto:Method: HEAD 2022-09-03T06:40:43.238 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:40:43.238 DEBUG:boto:Data: 2022-09-03T06:40:43.239 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.239 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:43.240 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.240 DEBUG:boto:Params: {} 2022-09-03T06:40:43.242 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:40:43.243 DEBUG:boto:Token: None 2022-09-03T06:40:43.244 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/ 2022-09-03T06:40:43.245 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OIKX2hWFqy/x1jvP5FJrY47eWF0= 2022-09-03T06:40:43.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OIKX2hWFqy/x1jvP5FJrY47eWF0=', 'Content-Length': '0'} 2022-09-03T06:40:43.254 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '10'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002fac3d28eb165d0c-006312f6eb-188c-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.255 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:43.256 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:43.256 DEBUG:boto:Method: HEAD 2022-09-03T06:40:43.257 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:40:43.257 DEBUG:boto:Data: 2022-09-03T06:40:43.258 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.258 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:43.259 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.260 DEBUG:boto:Params: {} 2022-09-03T06:40:43.261 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:40:43.262 DEBUG:boto:Token: None 2022-09-03T06:40:43.262 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/ 2022-09-03T06:40:43.263 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OIKX2hWFqy/x1jvP5FJrY47eWF0= 2022-09-03T06:40:43.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OIKX2hWFqy/x1jvP5FJrY47eWF0=', 'Content-Length': '0'} 2022-09-03T06:40:43.267 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '10'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000dd6adc292f90541a-006312f6eb-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.268 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:40:43.269 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:43.270 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:43.270 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:43.271 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:43.271 DEBUG:boto:Method: GET 2022-09-03T06:40:43.272 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:43.272 DEBUG:boto:Data: 2022-09-03T06:40:43.273 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.273 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:43.274 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.274 DEBUG:boto:Params: {} 2022-09-03T06:40:43.275 DEBUG:boto:Token: None 2022-09-03T06:40:43.276 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/?versions 2022-09-03T06:40:43.276 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jOpFUEv5T3RbWeKR840vGm/pjzI= 2022-09-03T06:40:43.277 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jOpFUEv5T3RbWeKR840vGm/pjzI=', 'Content-Length': '0'} 2022-09-03T06:40:43.281 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000874b7fe63e4800cd-006312f6eb-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.282 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:43.285 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:40:43.286 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:40:43.286 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:40:43.291 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:40:43.292 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:40:43.293 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:40:43.293 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:40:43.294 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:40:43.294 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:40:43.295 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:40:43.295 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:40:43.296 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:43.296 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:43.297 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:43.297 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:43.298 DEBUG:boto:Method: GET 2022-09-03T06:40:43.298 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:43.299 DEBUG:boto:Data: 2022-09-03T06:40:43.299 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.300 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:43.300 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.301 DEBUG:boto:Params: {} 2022-09-03T06:40:43.302 DEBUG:boto:Token: None 2022-09-03T06:40:43.302 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/?versions 2022-09-03T06:40:43.303 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jOpFUEv5T3RbWeKR840vGm/pjzI= 2022-09-03T06:40:43.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jOpFUEv5T3RbWeKR840vGm/pjzI=', 'Content-Length': '0'} 2022-09-03T06:40:43.307 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f21e4ad5fd10ee58-006312f6eb-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.308 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:43.310 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:40:43.311 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:40:43.311 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:40:43.312 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:40:43.313 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:40:43.315 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:40:43.316 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:40:43.317 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:40:43.317 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:40:43.318 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:40:43.319 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:43.320 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:43.321 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:43.322 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:43.322 DEBUG:boto:Method: GET 2022-09-03T06:40:43.323 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:43.324 DEBUG:boto:Data: 2022-09-03T06:40:43.324 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.325 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:43.326 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.326 DEBUG:boto:Params: {} 2022-09-03T06:40:43.327 DEBUG:boto:Token: None 2022-09-03T06:40:43.328 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/?versions 2022-09-03T06:40:43.329 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jOpFUEv5T3RbWeKR840vGm/pjzI= 2022-09-03T06:40:43.329 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jOpFUEv5T3RbWeKR840vGm/pjzI=', 'Content-Length': '0'} 2022-09-03T06:40:43.332 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000041e304b137073857-006312f6eb-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.333 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:43.334 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:43.335 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:43.335 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:43.336 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:43.336 DEBUG:boto:Method: GET 2022-09-03T06:40:43.337 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:43.337 DEBUG:boto:Data: 2022-09-03T06:40:43.338 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.338 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:43.339 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.339 DEBUG:boto:Params: {} 2022-09-03T06:40:43.340 DEBUG:boto:Token: None 2022-09-03T06:40:43.342 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/?versions 2022-09-03T06:40:43.342 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jOpFUEv5T3RbWeKR840vGm/pjzI= 2022-09-03T06:40:43.343 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jOpFUEv5T3RbWeKR840vGm/pjzI=', 'Content-Length': '0'} 2022-09-03T06:40:43.347 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a29a679e83aac34b-006312f6eb-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.348 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:43.349 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:40:43.350 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:43.350 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:43.351 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.352 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.352 DEBUG:boto:Method: GET 2022-09-03T06:40:43.354 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.355 DEBUG:boto:Data: 2022-09-03T06:40:43.356 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.357 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:43.358 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.359 DEBUG:boto:Params: {} 2022-09-03T06:40:43.359 DEBUG:boto:Token: None 2022-09-03T06:40:43.360 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.361 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wgWWFpGj5uETTFlonYkXpRJ3gsY= 2022-09-03T06:40:43.361 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wgWWFpGj5uETTFlonYkXpRJ3gsY=', 'Content-Length': '0'} 2022-09-03T06:40:43.364 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000850fe02a164e9e2b-006312f6eb-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.365 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:43.366 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:43.368 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.369 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.370 DEBUG:boto:Method: GET 2022-09-03T06:40:43.371 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.372 DEBUG:boto:Data: 2022-09-03T06:40:43.373 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.374 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:43.375 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.376 DEBUG:boto:Params: {} 2022-09-03T06:40:43.377 DEBUG:boto:Token: None 2022-09-03T06:40:43.378 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.379 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wgWWFpGj5uETTFlonYkXpRJ3gsY= 2022-09-03T06:40:43.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wgWWFpGj5uETTFlonYkXpRJ3gsY=', 'Content-Length': '0'} 2022-09-03T06:40:43.385 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a55c72ad9b6aa575-006312f6eb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.428 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:43.428 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:43.429 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.429 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.430 DEBUG:boto:Method: HEAD 2022-09-03T06:40:43.431 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.432 DEBUG:boto:Data: 2022-09-03T06:40:43.433 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.434 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:43.436 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.437 DEBUG:boto:Params: {} 2022-09-03T06:40:43.439 DEBUG:boto:Token: None 2022-09-03T06:40:43.441 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.443 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1CB/J53gyhVyFrdFToVxweJ+iw4= 2022-09-03T06:40:43.444 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1CB/J53gyhVyFrdFToVxweJ+iw4=', 'Content-Length': '0'} 2022-09-03T06:40:43.447 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f28cd8302f62489-006312f6eb-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.448 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:43.449 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:43.450 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.451 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.453 DEBUG:boto:Method: HEAD 2022-09-03T06:40:43.454 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.456 DEBUG:boto:Data: 2022-09-03T06:40:43.457 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.458 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:43.459 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.460 DEBUG:boto:Params: {} 2022-09-03T06:40:43.461 DEBUG:boto:Token: None 2022-09-03T06:40:43.462 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.463 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1CB/J53gyhVyFrdFToVxweJ+iw4= 2022-09-03T06:40:43.465 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1CB/J53gyhVyFrdFToVxweJ+iw4=', 'Content-Length': '0'} 2022-09-03T06:40:43.468 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018b1bf739859ccdd-006312f6eb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.470 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:40:43.471 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:43.472 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:43.474 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.475 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.477 DEBUG:boto:Method: GET 2022-09-03T06:40:43.478 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.480 DEBUG:boto:Data: 2022-09-03T06:40:43.481 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.482 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:43.484 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.485 DEBUG:boto:Params: {} 2022-09-03T06:40:43.486 DEBUG:boto:Token: None 2022-09-03T06:40:43.488 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.489 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wgWWFpGj5uETTFlonYkXpRJ3gsY= 2022-09-03T06:40:43.490 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wgWWFpGj5uETTFlonYkXpRJ3gsY=', 'Content-Length': '0'} 2022-09-03T06:40:43.494 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006ac01e79ab22b15c-006312f6eb-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.496 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:43.497 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:43.498 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.500 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.501 DEBUG:boto:Method: GET 2022-09-03T06:40:43.502 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.504 DEBUG:boto:Data: 2022-09-03T06:40:43.505 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.506 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:43.507 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.508 DEBUG:boto:Params: {} 2022-09-03T06:40:43.510 DEBUG:boto:Token: None 2022-09-03T06:40:43.511 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:43.512 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wgWWFpGj5uETTFlonYkXpRJ3gsY= 2022-09-03T06:40:43.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wgWWFpGj5uETTFlonYkXpRJ3gsY=', 'Content-Length': '0'} 2022-09-03T06:40:43.518 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ecf9957a37abcb1-006312f6eb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.520 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:43.521 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:43.523 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:43.524 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:43.525 DEBUG:boto:Method: GET 2022-09-03T06:40:43.527 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:40:43.528 DEBUG:boto:Data: 2022-09-03T06:40:43.529 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.531 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:43.532 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.533 DEBUG:boto:Params: {} 2022-09-03T06:40:43.533 DEBUG:boto:Token: None 2022-09-03T06:40:43.535 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/ 2022-09-03T06:40:43.536 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZyxKsAY4RVaszhRYVHJ+W5Rx/MU= 2022-09-03T06:40:43.537 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZyxKsAY4RVaszhRYVHJ+W5Rx/MU=', 'Content-Length': '0'} 2022-09-03T06:40:43.541 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000693cc6748520af5d-006312f6eb-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.543 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:43.546 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:43.547 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:43.548 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:43.551 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:43.552 DEBUG:boto:Method: GET 2022-09-03T06:40:43.554 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:40:43.562 DEBUG:boto:Data: 2022-09-03T06:40:43.565 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.566 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:43.567 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.568 DEBUG:boto:Params: {} 2022-09-03T06:40:43.569 DEBUG:boto:Token: None 2022-09-03T06:40:43.570 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/ 2022-09-03T06:40:43.571 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZyxKsAY4RVaszhRYVHJ+W5Rx/MU= 2022-09-03T06:40:43.573 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZyxKsAY4RVaszhRYVHJ+W5Rx/MU=', 'Content-Length': '0'} 2022-09-03T06:40:43.580 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006f36286cdc5930d3-006312f6eb-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.581 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:43.583 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:43.585 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:43.585 DEBUG:boto:Method: HEAD 2022-09-03T06:40:43.586 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:40:43.587 DEBUG:boto:Data: 2022-09-03T06:40:43.588 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.589 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:43.590 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.591 DEBUG:boto:Params: {} 2022-09-03T06:40:43.592 DEBUG:boto:Token: None 2022-09-03T06:40:43.593 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/obj11 2022-09-03T06:40:43.594 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VZWkEygpmE9SmrB8HsCSs5U8OGM= 2022-09-03T06:40:43.595 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VZWkEygpmE9SmrB8HsCSs5U8OGM=', 'Content-Length': '0'} 2022-09-03T06:40:43.602 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e065ecdd05d2ec5-006312f6eb-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.603 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:43.604 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:43.606 DEBUG:boto:Method: HEAD 2022-09-03T06:40:43.607 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:40:43.612 DEBUG:boto:Data: 2022-09-03T06:40:43.616 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.617 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:43.618 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.620 DEBUG:boto:Params: {} 2022-09-03T06:40:43.621 DEBUG:boto:Token: None 2022-09-03T06:40:43.621 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/obj11 2022-09-03T06:40:43.622 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VZWkEygpmE9SmrB8HsCSs5U8OGM= 2022-09-03T06:40:43.623 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VZWkEygpmE9SmrB8HsCSs5U8OGM=', 'Content-Length': '0'} 2022-09-03T06:40:43.625 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd4b0ef7a05e0b0b-006312f6eb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.627 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:40:43.628 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:43.629 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:43.630 DEBUG:boto:Method: GET 2022-09-03T06:40:43.630 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:40:43.631 DEBUG:boto:Data: 2022-09-03T06:40:43.631 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.632 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:43.632 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.633 DEBUG:boto:Params: {} 2022-09-03T06:40:43.634 DEBUG:boto:Token: None 2022-09-03T06:40:43.635 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/obj11 2022-09-03T06:40:43.636 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f20YF5rmMT4Qr5nGwiV1etSR3xc= 2022-09-03T06:40:43.637 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f20YF5rmMT4Qr5nGwiV1etSR3xc=', 'Content-Length': '0'} 2022-09-03T06:40:43.641 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b6a634afd91cb1a-006312f6eb-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.642 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:43.643 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:43.644 DEBUG:boto:Method: GET 2022-09-03T06:40:43.645 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:40:43.645 DEBUG:boto:Data: 2022-09-03T06:40:43.646 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.647 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:43.648 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.649 DEBUG:boto:Params: {} 2022-09-03T06:40:43.650 DEBUG:boto:Token: None 2022-09-03T06:40:43.651 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/obj11 2022-09-03T06:40:43.652 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f20YF5rmMT4Qr5nGwiV1etSR3xc= 2022-09-03T06:40:43.654 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f20YF5rmMT4Qr5nGwiV1etSR3xc=', 'Content-Length': '0'} 2022-09-03T06:40:43.657 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000347a2ef43159694d-006312f6eb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.664 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:43.666 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:43.666 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:43.668 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:43.669 DEBUG:boto:Method: GET 2022-09-03T06:40:43.670 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:40:43.671 DEBUG:boto:Data: 2022-09-03T06:40:43.672 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.672 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:43.673 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.673 DEBUG:boto:Params: {} 2022-09-03T06:40:43.674 DEBUG:boto:Token: None 2022-09-03T06:40:43.675 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/ 2022-09-03T06:40:43.675 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZyxKsAY4RVaszhRYVHJ+W5Rx/MU= 2022-09-03T06:40:43.677 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZyxKsAY4RVaszhRYVHJ+W5Rx/MU=', 'Content-Length': '0'} 2022-09-03T06:40:43.679 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000467bf70ccf421e01-006312f6eb-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.681 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:43.682 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:43.683 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:43.684 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:43.685 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:43.686 DEBUG:boto:Method: GET 2022-09-03T06:40:43.687 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:40:43.688 DEBUG:boto:Data: 2022-09-03T06:40:43.689 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.690 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:43.691 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.692 DEBUG:boto:Params: {} 2022-09-03T06:40:43.693 DEBUG:boto:Token: None 2022-09-03T06:40:43.694 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/ 2022-09-03T06:40:43.695 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZyxKsAY4RVaszhRYVHJ+W5Rx/MU= 2022-09-03T06:40:43.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZyxKsAY4RVaszhRYVHJ+W5Rx/MU=', 'Content-Length': '0'} 2022-09-03T06:40:43.707 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000009760f4ca5f4dcf7-006312f6eb-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.712 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:43.713 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:40:43.714 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:43.714 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:43.715 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.715 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.716 DEBUG:boto:Method: GET 2022-09-03T06:40:43.717 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.717 DEBUG:boto:Data: 2022-09-03T06:40:43.718 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.718 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:43.719 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.720 DEBUG:boto:Params: {} 2022-09-03T06:40:43.720 DEBUG:boto:Token: None 2022-09-03T06:40:43.721 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.721 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R+HQufC0kF+aAWSTv/ACWP/w3xg= 2022-09-03T06:40:43.722 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R+HQufC0kF+aAWSTv/ACWP/w3xg=', 'Content-Length': '0'} 2022-09-03T06:40:43.725 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018e03b1a6f100053-006312f6eb-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.727 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:43.727 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:43.728 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.729 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.729 DEBUG:boto:Method: GET 2022-09-03T06:40:43.730 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.731 DEBUG:boto:Data: 2022-09-03T06:40:43.732 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.733 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:43.735 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.736 DEBUG:boto:Params: {} 2022-09-03T06:40:43.738 DEBUG:boto:Token: None 2022-09-03T06:40:43.739 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.741 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R+HQufC0kF+aAWSTv/ACWP/w3xg= 2022-09-03T06:40:43.742 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R+HQufC0kF+aAWSTv/ACWP/w3xg=', 'Content-Length': '0'} 2022-09-03T06:40:43.745 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aa0a86da0a0f449f-006312f6eb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.747 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:43.748 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:43.749 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.749 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.750 DEBUG:boto:Method: HEAD 2022-09-03T06:40:43.751 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.754 DEBUG:boto:Data: 2022-09-03T06:40:43.755 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.755 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:43.756 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.757 DEBUG:boto:Params: {} 2022-09-03T06:40:43.759 DEBUG:boto:Token: None 2022-09-03T06:40:43.760 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.760 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8Cq1Pcxl5ndIamuDJTr6UXs9F1M= 2022-09-03T06:40:43.761 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8Cq1Pcxl5ndIamuDJTr6UXs9F1M=', 'Content-Length': '0'} 2022-09-03T06:40:43.763 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000027aec1a8659de035-006312f6eb-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.764 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:43.765 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:43.765 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.766 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.766 DEBUG:boto:Method: HEAD 2022-09-03T06:40:43.767 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.767 DEBUG:boto:Data: 2022-09-03T06:40:43.769 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.771 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:43.772 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.773 DEBUG:boto:Params: {} 2022-09-03T06:40:43.774 DEBUG:boto:Token: None 2022-09-03T06:40:43.775 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.776 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8Cq1Pcxl5ndIamuDJTr6UXs9F1M= 2022-09-03T06:40:43.778 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8Cq1Pcxl5ndIamuDJTr6UXs9F1M=', 'Content-Length': '0'} 2022-09-03T06:40:43.789 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b41e20437b5176a-006312f6eb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.790 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:40:43.791 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:43.791 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:43.792 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.792 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.793 DEBUG:boto:Method: GET 2022-09-03T06:40:43.794 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.794 DEBUG:boto:Data: 2022-09-03T06:40:43.795 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.795 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:43.796 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.796 DEBUG:boto:Params: {} 2022-09-03T06:40:43.797 DEBUG:boto:Token: None 2022-09-03T06:40:43.798 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.798 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R+HQufC0kF+aAWSTv/ACWP/w3xg= 2022-09-03T06:40:43.799 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R+HQufC0kF+aAWSTv/ACWP/w3xg=', 'Content-Length': '0'} 2022-09-03T06:40:43.801 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002e520d0ce0661f91-006312f6eb-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.803 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:43.804 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:43.805 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.806 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.807 DEBUG:boto:Method: GET 2022-09-03T06:40:43.808 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.809 DEBUG:boto:Data: 2022-09-03T06:40:43.810 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.811 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:43.811 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.812 DEBUG:boto:Params: {} 2022-09-03T06:40:43.813 DEBUG:boto:Token: None 2022-09-03T06:40:43.814 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:43.815 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R+HQufC0kF+aAWSTv/ACWP/w3xg= 2022-09-03T06:40:43.815 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R+HQufC0kF+aAWSTv/ACWP/w3xg=', 'Content-Length': '0'} 2022-09-03T06:40:43.825 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000818ed43989e69f7-006312f6eb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.867 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:43.868 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:43.869 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:43.870 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:43.871 DEBUG:boto:Method: GET 2022-09-03T06:40:43.872 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:40:43.873 DEBUG:boto:Data: 2022-09-03T06:40:43.875 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.876 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:43.877 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.877 DEBUG:boto:Params: {} 2022-09-03T06:40:43.879 DEBUG:boto:Token: None 2022-09-03T06:40:43.880 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/ 2022-09-03T06:40:43.881 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZyxKsAY4RVaszhRYVHJ+W5Rx/MU= 2022-09-03T06:40:43.882 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZyxKsAY4RVaszhRYVHJ+W5Rx/MU=', 'Content-Length': '0'} 2022-09-03T06:40:43.885 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000012667e03c41ccdae-006312f6eb-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.886 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:43.888 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:43.889 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:43.889 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:43.890 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:43.891 DEBUG:boto:Method: GET 2022-09-03T06:40:43.891 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:40:43.892 DEBUG:boto:Data: 2022-09-03T06:40:43.892 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.893 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:43.895 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.896 DEBUG:boto:Params: {} 2022-09-03T06:40:43.897 DEBUG:boto:Token: None 2022-09-03T06:40:43.898 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/ 2022-09-03T06:40:43.899 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZyxKsAY4RVaszhRYVHJ+W5Rx/MU= 2022-09-03T06:40:43.900 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZyxKsAY4RVaszhRYVHJ+W5Rx/MU=', 'Content-Length': '0'} 2022-09-03T06:40:43.906 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002b341395231397ae-006312f6eb-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.909 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:43.912 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:43.913 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:43.914 DEBUG:boto:Method: HEAD 2022-09-03T06:40:43.915 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:40:43.916 DEBUG:boto:Data: 2022-09-03T06:40:43.916 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.917 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:43.917 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.918 DEBUG:boto:Params: {} 2022-09-03T06:40:43.919 DEBUG:boto:Token: None 2022-09-03T06:40:43.919 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/obj17 2022-09-03T06:40:43.920 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3Nqvtu6Uz5Gg/l2COIkANgSwuDE= 2022-09-03T06:40:43.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3Nqvtu6Uz5Gg/l2COIkANgSwuDE=', 'Content-Length': '0'} 2022-09-03T06:40:43.924 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000011466186824057bf-006312f6eb-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.925 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:43.925 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:43.926 DEBUG:boto:Method: HEAD 2022-09-03T06:40:43.926 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:40:43.927 DEBUG:boto:Data: 2022-09-03T06:40:43.928 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.928 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:43.929 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.930 DEBUG:boto:Params: {} 2022-09-03T06:40:43.931 DEBUG:boto:Token: None 2022-09-03T06:40:43.934 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/obj17 2022-09-03T06:40:43.936 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3Nqvtu6Uz5Gg/l2COIkANgSwuDE= 2022-09-03T06:40:43.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3Nqvtu6Uz5Gg/l2COIkANgSwuDE=', 'Content-Length': '0'} 2022-09-03T06:40:43.939 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a990886df99d88b-006312f6eb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.940 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:40:43.941 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:43.942 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:43.942 DEBUG:boto:Method: GET 2022-09-03T06:40:43.943 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:40:43.944 DEBUG:boto:Data: 2022-09-03T06:40:43.944 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.945 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:43.945 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.946 DEBUG:boto:Params: {} 2022-09-03T06:40:43.946 DEBUG:boto:Token: None 2022-09-03T06:40:43.947 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/obj17 2022-09-03T06:40:43.948 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bp18fVUVKyn7nHvqNGYZK7weFgw= 2022-09-03T06:40:43.950 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bp18fVUVKyn7nHvqNGYZK7weFgw=', 'Content-Length': '0'} 2022-09-03T06:40:43.953 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a2b656aa57222f5-006312f6eb-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:43.955 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:43.956 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:43.957 DEBUG:boto:Method: GET 2022-09-03T06:40:43.958 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:40:43.959 DEBUG:boto:Data: 2022-09-03T06:40:43.960 DEBUG:boto:Headers: {} 2022-09-03T06:40:43.960 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:43.961 DEBUG:boto:Port: 8000 2022-09-03T06:40:43.963 DEBUG:boto:Params: {} 2022-09-03T06:40:43.964 DEBUG:boto:Token: None 2022-09-03T06:40:43.965 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:43 GMT /dsgvcy-75/obj17 2022-09-03T06:40:43.966 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bp18fVUVKyn7nHvqNGYZK7weFgw= 2022-09-03T06:40:43.967 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bp18fVUVKyn7nHvqNGYZK7weFgw=', 'Content-Length': '0'} 2022-09-03T06:40:43.970 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d15f5795cf647a9c-006312f6eb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.010 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:44.012 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:44.013 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:44.014 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:44.015 DEBUG:boto:Method: GET 2022-09-03T06:40:44.016 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:40:44.017 DEBUG:boto:Data: 2022-09-03T06:40:44.019 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.045 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:44.048 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.049 DEBUG:boto:Params: {} 2022-09-03T06:40:44.050 DEBUG:boto:Token: None 2022-09-03T06:40:44.052 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/ 2022-09-03T06:40:44.053 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SClQtOTXAZETWaAwVwysTJD9238= 2022-09-03T06:40:44.054 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SClQtOTXAZETWaAwVwysTJD9238=', 'Content-Length': '0'} 2022-09-03T06:40:44.060 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009ea90d48deaca063-006312f6ec-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.062 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:44.064 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:44.065 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:44.066 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:44.067 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:44.068 DEBUG:boto:Method: GET 2022-09-03T06:40:44.069 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:40:44.070 DEBUG:boto:Data: 2022-09-03T06:40:44.071 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.072 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:44.073 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.074 DEBUG:boto:Params: {} 2022-09-03T06:40:44.075 DEBUG:boto:Token: None 2022-09-03T06:40:44.076 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/ 2022-09-03T06:40:44.077 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SClQtOTXAZETWaAwVwysTJD9238= 2022-09-03T06:40:44.087 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SClQtOTXAZETWaAwVwysTJD9238=', 'Content-Length': '0'} 2022-09-03T06:40:44.094 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000ab67fccfc2724e5-006312f6ec-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.096 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:44.097 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:40:44.098 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:44.098 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:44.099 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.099 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.100 DEBUG:boto:Method: GET 2022-09-03T06:40:44.101 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.102 DEBUG:boto:Data: 2022-09-03T06:40:44.103 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.103 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:44.104 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.105 DEBUG:boto:Params: {} 2022-09-03T06:40:44.108 DEBUG:boto:Token: None 2022-09-03T06:40:44.109 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.110 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cFmDk5bRFIRB7ehGiEq9irbQJG0= 2022-09-03T06:40:44.111 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cFmDk5bRFIRB7ehGiEq9irbQJG0=', 'Content-Length': '0'} 2022-09-03T06:40:44.121 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008d8617b206e54c29-006312f6ec-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.123 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:44.125 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:44.126 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.132 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.133 DEBUG:boto:Method: GET 2022-09-03T06:40:44.134 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.135 DEBUG:boto:Data: 2022-09-03T06:40:44.136 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.137 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:44.138 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.139 DEBUG:boto:Params: {} 2022-09-03T06:40:44.141 DEBUG:boto:Token: None 2022-09-03T06:40:44.142 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.143 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cFmDk5bRFIRB7ehGiEq9irbQJG0= 2022-09-03T06:40:44.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cFmDk5bRFIRB7ehGiEq9irbQJG0=', 'Content-Length': '0'} 2022-09-03T06:40:44.153 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bb73cf7872a76fcc-006312f6ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.159 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:44.161 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:44.162 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.163 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.164 DEBUG:boto:Method: HEAD 2022-09-03T06:40:44.165 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.166 DEBUG:boto:Data: 2022-09-03T06:40:44.167 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.168 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:44.169 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.170 DEBUG:boto:Params: {} 2022-09-03T06:40:44.172 DEBUG:boto:Token: None 2022-09-03T06:40:44.173 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.174 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qk3KZiVhFxWu+R+a+OirOtADLKA= 2022-09-03T06:40:44.175 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qk3KZiVhFxWu+R+a+OirOtADLKA=', 'Content-Length': '0'} 2022-09-03T06:40:44.180 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e79156844f61135-006312f6ec-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.182 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:44.183 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:44.184 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.185 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.186 DEBUG:boto:Method: HEAD 2022-09-03T06:40:44.188 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.190 DEBUG:boto:Data: 2022-09-03T06:40:44.191 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.192 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:44.193 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.194 DEBUG:boto:Params: {} 2022-09-03T06:40:44.195 DEBUG:boto:Token: None 2022-09-03T06:40:44.196 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.198 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qk3KZiVhFxWu+R+a+OirOtADLKA= 2022-09-03T06:40:44.199 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qk3KZiVhFxWu+R+a+OirOtADLKA=', 'Content-Length': '0'} 2022-09-03T06:40:44.201 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004355a16c485c38e1-006312f6ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.203 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:40:44.204 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:44.205 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:44.206 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.207 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.208 DEBUG:boto:Method: GET 2022-09-03T06:40:44.209 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.211 DEBUG:boto:Data: 2022-09-03T06:40:44.212 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.213 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:44.214 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.215 DEBUG:boto:Params: {} 2022-09-03T06:40:44.217 DEBUG:boto:Token: None 2022-09-03T06:40:44.218 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.219 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cFmDk5bRFIRB7ehGiEq9irbQJG0= 2022-09-03T06:40:44.221 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cFmDk5bRFIRB7ehGiEq9irbQJG0=', 'Content-Length': '0'} 2022-09-03T06:40:44.227 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000820f0d0b4c097c1-006312f6ec-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.229 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:44.230 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:44.231 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.232 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.233 DEBUG:boto:Method: GET 2022-09-03T06:40:44.233 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.235 DEBUG:boto:Data: 2022-09-03T06:40:44.236 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.237 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:44.238 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.239 DEBUG:boto:Params: {} 2022-09-03T06:40:44.240 DEBUG:boto:Token: None 2022-09-03T06:40:44.241 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:44.242 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cFmDk5bRFIRB7ehGiEq9irbQJG0= 2022-09-03T06:40:44.243 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cFmDk5bRFIRB7ehGiEq9irbQJG0=', 'Content-Length': '0'} 2022-09-03T06:40:44.246 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c3f6ad3de001e60-006312f6ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.255 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:44.256 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:44.257 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:44.258 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:44.259 DEBUG:boto:Method: GET 2022-09-03T06:40:44.260 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:40:44.261 DEBUG:boto:Data: 2022-09-03T06:40:44.262 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.264 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:44.265 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.266 DEBUG:boto:Params: {} 2022-09-03T06:40:44.267 DEBUG:boto:Token: None 2022-09-03T06:40:44.268 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/ 2022-09-03T06:40:44.269 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SClQtOTXAZETWaAwVwysTJD9238= 2022-09-03T06:40:44.271 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SClQtOTXAZETWaAwVwysTJD9238=', 'Content-Length': '0'} 2022-09-03T06:40:44.277 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000213739fb5ca2d8eb-006312f6ec-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.280 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:44.281 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:44.282 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:44.283 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:44.284 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:44.285 DEBUG:boto:Method: GET 2022-09-03T06:40:44.286 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:40:44.288 DEBUG:boto:Data: 2022-09-03T06:40:44.288 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.293 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:44.295 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.296 DEBUG:boto:Params: {} 2022-09-03T06:40:44.298 DEBUG:boto:Token: None 2022-09-03T06:40:44.299 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/ 2022-09-03T06:40:44.300 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SClQtOTXAZETWaAwVwysTJD9238= 2022-09-03T06:40:44.301 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SClQtOTXAZETWaAwVwysTJD9238=', 'Content-Length': '0'} 2022-09-03T06:40:44.307 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000b9c6e7767bf629a-006312f6ec-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.309 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:44.311 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:44.311 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:44.315 DEBUG:boto:Method: HEAD 2022-09-03T06:40:44.316 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:40:44.317 DEBUG:boto:Data: 2022-09-03T06:40:44.319 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.320 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:44.321 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.324 DEBUG:boto:Params: {} 2022-09-03T06:40:44.325 DEBUG:boto:Token: None 2022-09-03T06:40:44.326 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj22 2022-09-03T06:40:44.327 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7RxcRwThnrEJ2T3iACWuhBMZq6c= 2022-09-03T06:40:44.329 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7RxcRwThnrEJ2T3iACWuhBMZq6c=', 'Content-Length': '0'} 2022-09-03T06:40:44.334 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e1f58927762c542e-006312f6ec-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.335 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:44.336 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:44.337 DEBUG:boto:Method: HEAD 2022-09-03T06:40:44.343 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:40:44.344 DEBUG:boto:Data: 2022-09-03T06:40:44.345 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.346 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:44.347 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.350 DEBUG:boto:Params: {} 2022-09-03T06:40:44.352 DEBUG:boto:Token: None 2022-09-03T06:40:44.353 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj22 2022-09-03T06:40:44.354 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7RxcRwThnrEJ2T3iACWuhBMZq6c= 2022-09-03T06:40:44.355 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7RxcRwThnrEJ2T3iACWuhBMZq6c=', 'Content-Length': '0'} 2022-09-03T06:40:44.359 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000114e8c97b2a67db1-006312f6ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.360 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:40:44.360 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:44.361 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:44.361 DEBUG:boto:Method: GET 2022-09-03T06:40:44.362 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:40:44.363 DEBUG:boto:Data: 2022-09-03T06:40:44.364 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.364 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:44.365 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.366 DEBUG:boto:Params: {} 2022-09-03T06:40:44.367 DEBUG:boto:Token: None 2022-09-03T06:40:44.369 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj22 2022-09-03T06:40:44.370 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tbIpjOn1jaZty7dVcFMLngcGBxU= 2022-09-03T06:40:44.371 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tbIpjOn1jaZty7dVcFMLngcGBxU=', 'Content-Length': '0'} 2022-09-03T06:40:44.374 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009a11d5f429a65fc1-006312f6ec-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.375 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:44.376 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:44.377 DEBUG:boto:Method: GET 2022-09-03T06:40:44.378 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:40:44.379 DEBUG:boto:Data: 2022-09-03T06:40:44.380 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.381 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:44.382 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.384 DEBUG:boto:Params: {} 2022-09-03T06:40:44.385 DEBUG:boto:Token: None 2022-09-03T06:40:44.387 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj22 2022-09-03T06:40:44.388 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tbIpjOn1jaZty7dVcFMLngcGBxU= 2022-09-03T06:40:44.390 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tbIpjOn1jaZty7dVcFMLngcGBxU=', 'Content-Length': '0'} 2022-09-03T06:40:44.393 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fe3693f7e9828494-006312f6ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.396 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:44.397 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:44.398 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:44.399 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:44.400 DEBUG:boto:Method: GET 2022-09-03T06:40:44.401 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:40:44.402 DEBUG:boto:Data: 2022-09-03T06:40:44.403 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.404 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:44.405 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.406 DEBUG:boto:Params: {} 2022-09-03T06:40:44.407 DEBUG:boto:Token: None 2022-09-03T06:40:44.408 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/ 2022-09-03T06:40:44.409 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SClQtOTXAZETWaAwVwysTJD9238= 2022-09-03T06:40:44.410 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SClQtOTXAZETWaAwVwysTJD9238=', 'Content-Length': '0'} 2022-09-03T06:40:44.414 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000158346d5e5325393-006312f6ec-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.416 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:44.418 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:44.419 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:44.420 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:44.421 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:44.422 DEBUG:boto:Method: GET 2022-09-03T06:40:44.423 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:40:44.424 DEBUG:boto:Data: 2022-09-03T06:40:44.425 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.425 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:44.427 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.427 DEBUG:boto:Params: {} 2022-09-03T06:40:44.428 DEBUG:boto:Token: None 2022-09-03T06:40:44.429 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/ 2022-09-03T06:40:44.430 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SClQtOTXAZETWaAwVwysTJD9238= 2022-09-03T06:40:44.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SClQtOTXAZETWaAwVwysTJD9238=', 'Content-Length': '0'} 2022-09-03T06:40:44.435 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002bbf1981ef4b95bf-006312f6ec-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.437 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:44.438 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:40:44.439 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:44.440 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:44.440 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.441 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.441 DEBUG:boto:Method: GET 2022-09-03T06:40:44.442 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.442 DEBUG:boto:Data: 2022-09-03T06:40:44.443 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.444 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:44.444 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.445 DEBUG:boto:Params: {} 2022-09-03T06:40:44.445 DEBUG:boto:Token: None 2022-09-03T06:40:44.446 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.447 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ysDIef6Uqv6iT620yNpVqUh6eqU= 2022-09-03T06:40:44.447 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ysDIef6Uqv6iT620yNpVqUh6eqU=', 'Content-Length': '0'} 2022-09-03T06:40:44.450 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c92798c768a39efd-006312f6ec-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.451 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:44.451 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:44.452 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.453 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.453 DEBUG:boto:Method: GET 2022-09-03T06:40:44.454 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.454 DEBUG:boto:Data: 2022-09-03T06:40:44.455 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.455 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:44.462 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.463 DEBUG:boto:Params: {} 2022-09-03T06:40:44.465 DEBUG:boto:Token: None 2022-09-03T06:40:44.466 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.467 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ysDIef6Uqv6iT620yNpVqUh6eqU= 2022-09-03T06:40:44.475 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ysDIef6Uqv6iT620yNpVqUh6eqU=', 'Content-Length': '0'} 2022-09-03T06:40:44.477 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009cb8c24bd8d77a44-006312f6ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.518 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:44.520 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:44.521 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.522 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.523 DEBUG:boto:Method: HEAD 2022-09-03T06:40:44.524 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.524 DEBUG:boto:Data: 2022-09-03T06:40:44.527 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.528 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:44.530 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.531 DEBUG:boto:Params: {} 2022-09-03T06:40:44.532 DEBUG:boto:Token: None 2022-09-03T06:40:44.533 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.534 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HTF94jrNhJEcYPf88XGjlomgLgo= 2022-09-03T06:40:44.535 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HTF94jrNhJEcYPf88XGjlomgLgo=', 'Content-Length': '0'} 2022-09-03T06:40:44.542 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001309d0f006c18a8d-006312f6ec-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.544 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:44.545 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:44.546 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.547 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.548 DEBUG:boto:Method: HEAD 2022-09-03T06:40:44.549 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.549 DEBUG:boto:Data: 2022-09-03T06:40:44.550 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.551 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:44.552 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.553 DEBUG:boto:Params: {} 2022-09-03T06:40:44.554 DEBUG:boto:Token: None 2022-09-03T06:40:44.555 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.557 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HTF94jrNhJEcYPf88XGjlomgLgo= 2022-09-03T06:40:44.559 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HTF94jrNhJEcYPf88XGjlomgLgo=', 'Content-Length': '0'} 2022-09-03T06:40:44.562 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e4200a89d6ab2460-006312f6ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.563 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:40:44.564 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:44.565 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:44.566 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.568 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.569 DEBUG:boto:Method: GET 2022-09-03T06:40:44.570 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.571 DEBUG:boto:Data: 2022-09-03T06:40:44.572 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.573 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:44.573 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.574 DEBUG:boto:Params: {} 2022-09-03T06:40:44.575 DEBUG:boto:Token: None 2022-09-03T06:40:44.575 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.576 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ysDIef6Uqv6iT620yNpVqUh6eqU= 2022-09-03T06:40:44.577 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ysDIef6Uqv6iT620yNpVqUh6eqU=', 'Content-Length': '0'} 2022-09-03T06:40:44.579 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eecb6a481e3357e3-006312f6ec-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.623 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:44.624 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:44.624 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.625 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.625 DEBUG:boto:Method: GET 2022-09-03T06:40:44.626 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.627 DEBUG:boto:Data: 2022-09-03T06:40:44.627 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.628 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:44.628 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.629 DEBUG:boto:Params: {} 2022-09-03T06:40:44.631 DEBUG:boto:Token: None 2022-09-03T06:40:44.631 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:44.632 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ysDIef6Uqv6iT620yNpVqUh6eqU= 2022-09-03T06:40:44.642 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ysDIef6Uqv6iT620yNpVqUh6eqU=', 'Content-Length': '0'} 2022-09-03T06:40:44.646 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000034e6319a15ecde12-006312f6ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.648 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:44.650 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:44.651 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:44.652 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:44.653 DEBUG:boto:Method: GET 2022-09-03T06:40:44.654 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:40:44.656 DEBUG:boto:Data: 2022-09-03T06:40:44.657 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.657 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:44.658 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.659 DEBUG:boto:Params: {} 2022-09-03T06:40:44.660 DEBUG:boto:Token: None 2022-09-03T06:40:44.661 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/ 2022-09-03T06:40:44.661 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SClQtOTXAZETWaAwVwysTJD9238= 2022-09-03T06:40:44.662 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SClQtOTXAZETWaAwVwysTJD9238=', 'Content-Length': '0'} 2022-09-03T06:40:44.672 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000097ccfe51c769293-006312f6ec-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.673 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:44.675 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:44.676 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:44.677 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:44.678 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:44.679 DEBUG:boto:Method: GET 2022-09-03T06:40:44.686 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:40:44.687 DEBUG:boto:Data: 2022-09-03T06:40:44.688 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.688 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:44.689 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.691 DEBUG:boto:Params: {} 2022-09-03T06:40:44.693 DEBUG:boto:Token: None 2022-09-03T06:40:44.694 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/ 2022-09-03T06:40:44.695 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SClQtOTXAZETWaAwVwysTJD9238= 2022-09-03T06:40:44.697 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SClQtOTXAZETWaAwVwysTJD9238=', 'Content-Length': '0'} 2022-09-03T06:40:44.704 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000086de14bfa0d253bc-006312f6ec-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.704 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:44.705 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:44.706 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:44.707 DEBUG:boto:Method: HEAD 2022-09-03T06:40:44.707 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:40:44.708 DEBUG:boto:Data: 2022-09-03T06:40:44.708 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.709 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:44.709 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.710 DEBUG:boto:Params: {} 2022-09-03T06:40:44.711 DEBUG:boto:Token: None 2022-09-03T06:40:44.711 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj33 2022-09-03T06:40:44.712 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rNTxdZhVTb+n8JQmxhrDugcbv2U= 2022-09-03T06:40:44.713 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rNTxdZhVTb+n8JQmxhrDugcbv2U=', 'Content-Length': '0'} 2022-09-03T06:40:44.716 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000884e99501f2e987b-006312f6ec-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.718 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:44.719 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:44.720 DEBUG:boto:Method: HEAD 2022-09-03T06:40:44.721 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:40:44.723 DEBUG:boto:Data: 2022-09-03T06:40:44.724 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.724 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:44.725 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.726 DEBUG:boto:Params: {} 2022-09-03T06:40:44.726 DEBUG:boto:Token: None 2022-09-03T06:40:44.727 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj33 2022-09-03T06:40:44.728 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rNTxdZhVTb+n8JQmxhrDugcbv2U= 2022-09-03T06:40:44.728 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rNTxdZhVTb+n8JQmxhrDugcbv2U=', 'Content-Length': '0'} 2022-09-03T06:40:44.731 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006480e720838a203a-006312f6ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.732 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:40:44.733 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:44.734 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:44.736 DEBUG:boto:Method: GET 2022-09-03T06:40:44.737 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:40:44.738 DEBUG:boto:Data: 2022-09-03T06:40:44.739 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.740 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:44.741 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.742 DEBUG:boto:Params: {} 2022-09-03T06:40:44.745 DEBUG:boto:Token: None 2022-09-03T06:40:44.746 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj33 2022-09-03T06:40:44.747 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KD33ENEMCPo6Lw3vzFDLePD6Bzk= 2022-09-03T06:40:44.749 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KD33ENEMCPo6Lw3vzFDLePD6Bzk=', 'Content-Length': '0'} 2022-09-03T06:40:44.751 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074e102d1ed7e06d5-006312f6ec-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.753 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:44.754 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:44.756 DEBUG:boto:Method: GET 2022-09-03T06:40:44.756 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:40:44.757 DEBUG:boto:Data: 2022-09-03T06:40:44.757 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.758 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:44.759 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.761 DEBUG:boto:Params: {} 2022-09-03T06:40:44.762 DEBUG:boto:Token: None 2022-09-03T06:40:44.763 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj33 2022-09-03T06:40:44.764 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KD33ENEMCPo6Lw3vzFDLePD6Bzk= 2022-09-03T06:40:44.765 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KD33ENEMCPo6Lw3vzFDLePD6Bzk=', 'Content-Length': '0'} 2022-09-03T06:40:44.768 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048d46a103fae29ec-006312f6ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.770 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:44.771 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:44.771 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:44.772 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:44.772 DEBUG:boto:Method: GET 2022-09-03T06:40:44.773 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:40:44.773 DEBUG:boto:Data: 2022-09-03T06:40:44.774 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.775 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:44.775 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.776 DEBUG:boto:Params: {} 2022-09-03T06:40:44.777 DEBUG:boto:Token: None 2022-09-03T06:40:44.777 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/ 2022-09-03T06:40:44.778 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SClQtOTXAZETWaAwVwysTJD9238= 2022-09-03T06:40:44.779 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SClQtOTXAZETWaAwVwysTJD9238=', 'Content-Length': '0'} 2022-09-03T06:40:44.782 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003c5d1aef97b35a97-006312f6ec-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.782 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:44.783 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:44.784 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:44.784 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:44.785 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:44.786 DEBUG:boto:Method: GET 2022-09-03T06:40:44.786 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:40:44.787 DEBUG:boto:Data: 2022-09-03T06:40:44.787 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.788 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:44.789 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.789 DEBUG:boto:Params: {} 2022-09-03T06:40:44.790 DEBUG:boto:Token: None 2022-09-03T06:40:44.790 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/ 2022-09-03T06:40:44.791 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SClQtOTXAZETWaAwVwysTJD9238= 2022-09-03T06:40:44.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SClQtOTXAZETWaAwVwysTJD9238=', 'Content-Length': '0'} 2022-09-03T06:40:44.795 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fbb0d5fdbea2a9ba-006312f6ec-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.796 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:44.796 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:40:44.797 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:44.798 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:44.798 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.799 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.799 DEBUG:boto:Method: GET 2022-09-03T06:40:44.800 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.801 DEBUG:boto:Data: 2022-09-03T06:40:44.801 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.802 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:44.802 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.803 DEBUG:boto:Params: {} 2022-09-03T06:40:44.803 DEBUG:boto:Token: None 2022-09-03T06:40:44.804 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.805 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3JKfVw8pYzenZGVnsI6VbOzAa8A= 2022-09-03T06:40:44.805 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3JKfVw8pYzenZGVnsI6VbOzAa8A=', 'Content-Length': '0'} 2022-09-03T06:40:44.808 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003768c24b9fb58874-006312f6ec-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.809 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:44.809 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:44.810 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.811 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.812 DEBUG:boto:Method: GET 2022-09-03T06:40:44.812 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.813 DEBUG:boto:Data: 2022-09-03T06:40:44.813 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.814 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:44.814 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.815 DEBUG:boto:Params: {} 2022-09-03T06:40:44.815 DEBUG:boto:Token: None 2022-09-03T06:40:44.816 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.817 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3JKfVw8pYzenZGVnsI6VbOzAa8A= 2022-09-03T06:40:44.818 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3JKfVw8pYzenZGVnsI6VbOzAa8A=', 'Content-Length': '0'} 2022-09-03T06:40:44.821 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004b756036602b0740-006312f6ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.862 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:44.863 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:44.863 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.864 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.864 DEBUG:boto:Method: HEAD 2022-09-03T06:40:44.865 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.866 DEBUG:boto:Data: 2022-09-03T06:40:44.867 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.867 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:44.868 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.869 DEBUG:boto:Params: {} 2022-09-03T06:40:44.869 DEBUG:boto:Token: None 2022-09-03T06:40:44.870 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.871 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DZ0rqP5fr94wmk7+yo8KoDDLxVY= 2022-09-03T06:40:44.873 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DZ0rqP5fr94wmk7+yo8KoDDLxVY=', 'Content-Length': '0'} 2022-09-03T06:40:44.877 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c3ccba21dc704b82-006312f6ec-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.879 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:44.880 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:44.881 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.882 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.883 DEBUG:boto:Method: HEAD 2022-09-03T06:40:44.883 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.884 DEBUG:boto:Data: 2022-09-03T06:40:44.884 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.885 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:44.885 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.886 DEBUG:boto:Params: {} 2022-09-03T06:40:44.886 DEBUG:boto:Token: None 2022-09-03T06:40:44.888 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.889 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DZ0rqP5fr94wmk7+yo8KoDDLxVY= 2022-09-03T06:40:44.890 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DZ0rqP5fr94wmk7+yo8KoDDLxVY=', 'Content-Length': '0'} 2022-09-03T06:40:44.893 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ff241c8d3b10813f-006312f6ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.894 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:40:44.895 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:44.895 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:44.896 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.896 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.898 DEBUG:boto:Method: GET 2022-09-03T06:40:44.898 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.899 DEBUG:boto:Data: 2022-09-03T06:40:44.899 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.900 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:44.900 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.901 DEBUG:boto:Params: {} 2022-09-03T06:40:44.902 DEBUG:boto:Token: None 2022-09-03T06:40:44.902 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.903 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3JKfVw8pYzenZGVnsI6VbOzAa8A= 2022-09-03T06:40:44.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3JKfVw8pYzenZGVnsI6VbOzAa8A=', 'Content-Length': '0'} 2022-09-03T06:40:44.910 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cf1ba5a503c4b59f-006312f6ec-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.912 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:44.913 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:44.916 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.918 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.919 DEBUG:boto:Method: GET 2022-09-03T06:40:44.920 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.921 DEBUG:boto:Data: 2022-09-03T06:40:44.922 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.923 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:44.924 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.924 DEBUG:boto:Params: {} 2022-09-03T06:40:44.925 DEBUG:boto:Token: None 2022-09-03T06:40:44.926 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:44.926 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3JKfVw8pYzenZGVnsI6VbOzAa8A= 2022-09-03T06:40:44.927 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3JKfVw8pYzenZGVnsI6VbOzAa8A=', 'Content-Length': '0'} 2022-09-03T06:40:44.930 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a6223c628ace9392-006312f6ec-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.931 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:44.931 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:44.932 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:44.933 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:44.934 DEBUG:boto:Method: GET 2022-09-03T06:40:44.935 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:40:44.935 DEBUG:boto:Data: 2022-09-03T06:40:44.936 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.937 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:44.937 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.938 DEBUG:boto:Params: {} 2022-09-03T06:40:44.939 DEBUG:boto:Token: None 2022-09-03T06:40:44.939 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:44 GMT /dsgvcy-75/ 2022-09-03T06:40:44.940 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SClQtOTXAZETWaAwVwysTJD9238= 2022-09-03T06:40:44.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SClQtOTXAZETWaAwVwysTJD9238=', 'Content-Length': '0'} 2022-09-03T06:40:44.944 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fe918b431059a345-006312f6ec-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:44.987 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:44.990 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:44.991 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:44.992 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:44.993 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:44.994 DEBUG:boto:Method: GET 2022-09-03T06:40:44.995 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:40:44.996 DEBUG:boto:Data: 2022-09-03T06:40:44.997 DEBUG:boto:Headers: {} 2022-09-03T06:40:44.998 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:44.999 DEBUG:boto:Port: 8000 2022-09-03T06:40:44.999 DEBUG:boto:Params: {} 2022-09-03T06:40:45.000 DEBUG:boto:Token: None 2022-09-03T06:40:45.001 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/ 2022-09-03T06:40:45.002 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0= 2022-09-03T06:40:45.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0=', 'Content-Length': '0'} 2022-09-03T06:40:45.007 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dfe1a38fadef6dc2-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.008 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:45.009 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:45.010 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:45.010 DEBUG:boto:Method: HEAD 2022-09-03T06:40:45.011 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:40:45.012 DEBUG:boto:Data: 2022-09-03T06:40:45.013 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.014 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.015 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.016 DEBUG:boto:Params: {} 2022-09-03T06:40:45.017 DEBUG:boto:Token: None 2022-09-03T06:40:45.018 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj34 2022-09-03T06:40:45.019 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sP3ZTzstVRv2o6FSiHtTYQ+kSfQ= 2022-09-03T06:40:45.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sP3ZTzstVRv2o6FSiHtTYQ+kSfQ=', 'Content-Length': '0'} 2022-09-03T06:40:45.023 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000310d8027167bb5e2-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.024 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:45.025 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:45.026 DEBUG:boto:Method: HEAD 2022-09-03T06:40:45.027 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:40:45.028 DEBUG:boto:Data: 2022-09-03T06:40:45.029 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.030 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.031 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.031 DEBUG:boto:Params: {} 2022-09-03T06:40:45.034 DEBUG:boto:Token: None 2022-09-03T06:40:45.035 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj34 2022-09-03T06:40:45.036 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sP3ZTzstVRv2o6FSiHtTYQ+kSfQ= 2022-09-03T06:40:45.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sP3ZTzstVRv2o6FSiHtTYQ+kSfQ=', 'Content-Length': '0'} 2022-09-03T06:40:45.039 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d9ff72d4bb9da0ab-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.040 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:40:45.040 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:45.041 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:45.041 DEBUG:boto:Method: GET 2022-09-03T06:40:45.042 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:40:45.042 DEBUG:boto:Data: 2022-09-03T06:40:45.043 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.044 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.044 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.045 DEBUG:boto:Params: {} 2022-09-03T06:40:45.045 DEBUG:boto:Token: None 2022-09-03T06:40:45.046 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj34 2022-09-03T06:40:45.047 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Kkl6pGf2BxShMU6/8HtpxKNxxxI= 2022-09-03T06:40:45.047 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Kkl6pGf2BxShMU6/8HtpxKNxxxI=', 'Content-Length': '0'} 2022-09-03T06:40:45.049 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e641d538bf274a26-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.050 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:45.051 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:45.051 DEBUG:boto:Method: GET 2022-09-03T06:40:45.052 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:40:45.052 DEBUG:boto:Data: 2022-09-03T06:40:45.053 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.054 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.054 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.055 DEBUG:boto:Params: {} 2022-09-03T06:40:45.056 DEBUG:boto:Token: None 2022-09-03T06:40:45.056 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj34 2022-09-03T06:40:45.057 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Kkl6pGf2BxShMU6/8HtpxKNxxxI= 2022-09-03T06:40:45.058 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Kkl6pGf2BxShMU6/8HtpxKNxxxI=', 'Content-Length': '0'} 2022-09-03T06:40:45.060 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000022b81f9f416827de-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.106 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:45.107 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:45.108 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:45.109 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:45.110 DEBUG:boto:Method: GET 2022-09-03T06:40:45.111 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:40:45.111 DEBUG:boto:Data: 2022-09-03T06:40:45.112 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.112 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.113 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.114 DEBUG:boto:Params: {} 2022-09-03T06:40:45.115 DEBUG:boto:Token: None 2022-09-03T06:40:45.116 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/ 2022-09-03T06:40:45.116 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0= 2022-09-03T06:40:45.117 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0=', 'Content-Length': '0'} 2022-09-03T06:40:45.120 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a030048260328513-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.121 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:45.122 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:45.123 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:45.124 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:45.125 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:45.125 DEBUG:boto:Method: GET 2022-09-03T06:40:45.126 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:40:45.128 DEBUG:boto:Data: 2022-09-03T06:40:45.129 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.130 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.131 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.132 DEBUG:boto:Params: {} 2022-09-03T06:40:45.134 DEBUG:boto:Token: None 2022-09-03T06:40:45.135 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/ 2022-09-03T06:40:45.136 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0= 2022-09-03T06:40:45.137 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0=', 'Content-Length': '0'} 2022-09-03T06:40:45.141 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bf8cdac0f2aad075-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.142 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:45.144 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:40:45.145 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:45.146 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:45.147 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.148 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.149 DEBUG:boto:Method: GET 2022-09-03T06:40:45.150 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.151 DEBUG:boto:Data: 2022-09-03T06:40:45.151 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.152 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.153 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.153 DEBUG:boto:Params: {} 2022-09-03T06:40:45.154 DEBUG:boto:Token: None 2022-09-03T06:40:45.155 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.156 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8DyKeuiSZnGs3e3Uv5+4xGRAqjk= 2022-09-03T06:40:45.157 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8DyKeuiSZnGs3e3Uv5+4xGRAqjk=', 'Content-Length': '0'} 2022-09-03T06:40:45.160 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f5d883e6dc192415-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.162 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:45.163 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:45.164 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.165 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.166 DEBUG:boto:Method: GET 2022-09-03T06:40:45.167 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.168 DEBUG:boto:Data: 2022-09-03T06:40:45.169 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.170 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.171 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.172 DEBUG:boto:Params: {} 2022-09-03T06:40:45.172 DEBUG:boto:Token: None 2022-09-03T06:40:45.173 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.174 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8DyKeuiSZnGs3e3Uv5+4xGRAqjk= 2022-09-03T06:40:45.175 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8DyKeuiSZnGs3e3Uv5+4xGRAqjk=', 'Content-Length': '0'} 2022-09-03T06:40:45.177 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000247efed2091bc76f-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.178 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:45.178 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:45.179 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.179 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.180 DEBUG:boto:Method: HEAD 2022-09-03T06:40:45.181 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.181 DEBUG:boto:Data: 2022-09-03T06:40:45.182 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.182 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.183 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.184 DEBUG:boto:Params: {} 2022-09-03T06:40:45.184 DEBUG:boto:Token: None 2022-09-03T06:40:45.185 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.186 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a7C8VOmi0All7eTJqrMqvSV+nPQ= 2022-09-03T06:40:45.186 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a7C8VOmi0All7eTJqrMqvSV+nPQ=', 'Content-Length': '0'} 2022-09-03T06:40:45.190 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004bff1c9b83632bff-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.191 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:45.193 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:45.194 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.195 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.196 DEBUG:boto:Method: HEAD 2022-09-03T06:40:45.197 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.198 DEBUG:boto:Data: 2022-09-03T06:40:45.199 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.200 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.201 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.202 DEBUG:boto:Params: {} 2022-09-03T06:40:45.203 DEBUG:boto:Token: None 2022-09-03T06:40:45.204 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.205 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a7C8VOmi0All7eTJqrMqvSV+nPQ= 2022-09-03T06:40:45.206 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a7C8VOmi0All7eTJqrMqvSV+nPQ=', 'Content-Length': '0'} 2022-09-03T06:40:45.211 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a674480f9e82d05-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.212 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:40:45.213 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:45.213 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:45.214 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.215 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.216 DEBUG:boto:Method: GET 2022-09-03T06:40:45.216 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.217 DEBUG:boto:Data: 2022-09-03T06:40:45.218 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.218 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.219 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.220 DEBUG:boto:Params: {} 2022-09-03T06:40:45.221 DEBUG:boto:Token: None 2022-09-03T06:40:45.221 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.222 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8DyKeuiSZnGs3e3Uv5+4xGRAqjk= 2022-09-03T06:40:45.223 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8DyKeuiSZnGs3e3Uv5+4xGRAqjk=', 'Content-Length': '0'} 2022-09-03T06:40:45.225 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000017f431a361fd5dfb-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.226 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:45.227 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:45.227 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.228 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.228 DEBUG:boto:Method: GET 2022-09-03T06:40:45.229 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.230 DEBUG:boto:Data: 2022-09-03T06:40:45.231 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.232 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.233 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.234 DEBUG:boto:Params: {} 2022-09-03T06:40:45.235 DEBUG:boto:Token: None 2022-09-03T06:40:45.236 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:45.237 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8DyKeuiSZnGs3e3Uv5+4xGRAqjk= 2022-09-03T06:40:45.238 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8DyKeuiSZnGs3e3Uv5+4xGRAqjk=', 'Content-Length': '0'} 2022-09-03T06:40:45.241 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008fb0bd5179f9ccfd-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.243 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:45.244 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:45.245 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:45.246 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:45.246 DEBUG:boto:Method: GET 2022-09-03T06:40:45.247 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:40:45.247 DEBUG:boto:Data: 2022-09-03T06:40:45.248 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.248 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.249 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.250 DEBUG:boto:Params: {} 2022-09-03T06:40:45.250 DEBUG:boto:Token: None 2022-09-03T06:40:45.251 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/ 2022-09-03T06:40:45.251 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0= 2022-09-03T06:40:45.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0=', 'Content-Length': '0'} 2022-09-03T06:40:45.255 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006561c50956183fd5-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.255 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:45.256 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:45.257 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:45.257 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:45.258 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:45.259 DEBUG:boto:Method: GET 2022-09-03T06:40:45.259 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:40:45.260 DEBUG:boto:Data: 2022-09-03T06:40:45.260 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.261 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.262 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.263 DEBUG:boto:Params: {} 2022-09-03T06:40:45.263 DEBUG:boto:Token: None 2022-09-03T06:40:45.264 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/ 2022-09-03T06:40:45.264 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0= 2022-09-03T06:40:45.265 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0=', 'Content-Length': '0'} 2022-09-03T06:40:45.269 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000087755e63373c481a-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.271 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:45.272 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:45.273 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:45.273 DEBUG:boto:Method: HEAD 2022-09-03T06:40:45.274 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:40:45.275 DEBUG:boto:Data: 2022-09-03T06:40:45.276 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.277 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.278 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.279 DEBUG:boto:Params: {} 2022-09-03T06:40:45.281 DEBUG:boto:Token: None 2022-09-03T06:40:45.282 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj44 2022-09-03T06:40:45.283 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TAJHF6CzgXK08GwLzGfFG+k1hHc= 2022-09-03T06:40:45.287 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TAJHF6CzgXK08GwLzGfFG+k1hHc=', 'Content-Length': '0'} 2022-09-03T06:40:45.290 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000991af15b0b4dbde4-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.291 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:45.291 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:45.292 DEBUG:boto:Method: HEAD 2022-09-03T06:40:45.292 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:40:45.293 DEBUG:boto:Data: 2022-09-03T06:40:45.294 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.294 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.295 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.296 DEBUG:boto:Params: {} 2022-09-03T06:40:45.296 DEBUG:boto:Token: None 2022-09-03T06:40:45.297 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj44 2022-09-03T06:40:45.298 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TAJHF6CzgXK08GwLzGfFG+k1hHc= 2022-09-03T06:40:45.299 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TAJHF6CzgXK08GwLzGfFG+k1hHc=', 'Content-Length': '0'} 2022-09-03T06:40:45.301 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009b4d3366e5a0fb9d-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.302 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:40:45.303 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:45.303 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:45.304 DEBUG:boto:Method: GET 2022-09-03T06:40:45.305 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:40:45.306 DEBUG:boto:Data: 2022-09-03T06:40:45.306 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.307 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.307 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.308 DEBUG:boto:Params: {} 2022-09-03T06:40:45.309 DEBUG:boto:Token: None 2022-09-03T06:40:45.310 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj44 2022-09-03T06:40:45.310 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eqxO4zG3jqKOrdi/+H1QzgobjoY= 2022-09-03T06:40:45.311 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eqxO4zG3jqKOrdi/+H1QzgobjoY=', 'Content-Length': '0'} 2022-09-03T06:40:45.313 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009249ac72fc8d13d6-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.358 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:45.359 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:45.360 DEBUG:boto:Method: GET 2022-09-03T06:40:45.361 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:40:45.361 DEBUG:boto:Data: 2022-09-03T06:40:45.363 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.364 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.365 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.366 DEBUG:boto:Params: {} 2022-09-03T06:40:45.366 DEBUG:boto:Token: None 2022-09-03T06:40:45.367 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj44 2022-09-03T06:40:45.368 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eqxO4zG3jqKOrdi/+H1QzgobjoY= 2022-09-03T06:40:45.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eqxO4zG3jqKOrdi/+H1QzgobjoY=', 'Content-Length': '0'} 2022-09-03T06:40:45.372 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000341ef8a4ea310774-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.373 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:45.374 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:45.375 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:45.376 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:45.377 DEBUG:boto:Method: GET 2022-09-03T06:40:45.378 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:40:45.378 DEBUG:boto:Data: 2022-09-03T06:40:45.379 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.380 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.381 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.382 DEBUG:boto:Params: {} 2022-09-03T06:40:45.383 DEBUG:boto:Token: None 2022-09-03T06:40:45.384 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/ 2022-09-03T06:40:45.385 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0= 2022-09-03T06:40:45.386 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0=', 'Content-Length': '0'} 2022-09-03T06:40:45.390 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008eae89050e938857-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.391 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:45.392 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:45.393 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:45.393 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:45.394 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:45.395 DEBUG:boto:Method: GET 2022-09-03T06:40:45.396 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:40:45.397 DEBUG:boto:Data: 2022-09-03T06:40:45.399 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.399 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.400 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.401 DEBUG:boto:Params: {} 2022-09-03T06:40:45.402 DEBUG:boto:Token: None 2022-09-03T06:40:45.403 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/ 2022-09-03T06:40:45.404 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0= 2022-09-03T06:40:45.405 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0=', 'Content-Length': '0'} 2022-09-03T06:40:45.409 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002205fbf4e5a9985e-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.410 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:45.411 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:40:45.411 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:45.412 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:45.413 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.413 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.414 DEBUG:boto:Method: GET 2022-09-03T06:40:45.414 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.415 DEBUG:boto:Data: 2022-09-03T06:40:45.415 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.416 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.416 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.417 DEBUG:boto:Params: {} 2022-09-03T06:40:45.418 DEBUG:boto:Token: None 2022-09-03T06:40:45.418 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.419 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0BfuzhwkVqUlg0mgDkAXgOOkiLA= 2022-09-03T06:40:45.419 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0BfuzhwkVqUlg0mgDkAXgOOkiLA=', 'Content-Length': '0'} 2022-09-03T06:40:45.422 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000802d6e2b4c842872-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.423 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:45.423 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:45.424 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.424 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.425 DEBUG:boto:Method: GET 2022-09-03T06:40:45.425 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.426 DEBUG:boto:Data: 2022-09-03T06:40:45.426 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.427 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.427 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.428 DEBUG:boto:Params: {} 2022-09-03T06:40:45.428 DEBUG:boto:Token: None 2022-09-03T06:40:45.429 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.430 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0BfuzhwkVqUlg0mgDkAXgOOkiLA= 2022-09-03T06:40:45.430 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0BfuzhwkVqUlg0mgDkAXgOOkiLA=', 'Content-Length': '0'} 2022-09-03T06:40:45.434 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e54285294f6e439-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.435 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:45.435 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:45.436 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.436 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.437 DEBUG:boto:Method: HEAD 2022-09-03T06:40:45.438 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.438 DEBUG:boto:Data: 2022-09-03T06:40:45.439 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.439 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.440 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.440 DEBUG:boto:Params: {} 2022-09-03T06:40:45.441 DEBUG:boto:Token: None 2022-09-03T06:40:45.442 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.442 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K4M1q8vAdz1k2FOy4SCD/79L3GQ= 2022-09-03T06:40:45.443 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K4M1q8vAdz1k2FOy4SCD/79L3GQ=', 'Content-Length': '0'} 2022-09-03T06:40:45.445 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000374184a57fd1c450-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.446 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:45.447 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:45.447 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.448 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.449 DEBUG:boto:Method: HEAD 2022-09-03T06:40:45.449 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.450 DEBUG:boto:Data: 2022-09-03T06:40:45.451 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.451 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.452 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.452 DEBUG:boto:Params: {} 2022-09-03T06:40:45.453 DEBUG:boto:Token: None 2022-09-03T06:40:45.453 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.454 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K4M1q8vAdz1k2FOy4SCD/79L3GQ= 2022-09-03T06:40:45.455 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K4M1q8vAdz1k2FOy4SCD/79L3GQ=', 'Content-Length': '0'} 2022-09-03T06:40:45.457 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e661629e6dc43bf-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.458 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:40:45.459 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:45.460 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:45.461 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.461 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.462 DEBUG:boto:Method: GET 2022-09-03T06:40:45.463 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.464 DEBUG:boto:Data: 2022-09-03T06:40:45.465 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.466 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.467 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.468 DEBUG:boto:Params: {} 2022-09-03T06:40:45.471 DEBUG:boto:Token: None 2022-09-03T06:40:45.472 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.473 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0BfuzhwkVqUlg0mgDkAXgOOkiLA= 2022-09-03T06:40:45.475 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0BfuzhwkVqUlg0mgDkAXgOOkiLA=', 'Content-Length': '0'} 2022-09-03T06:40:45.486 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fc17ce780d8ceb67-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.488 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:45.489 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:45.490 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.491 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.492 DEBUG:boto:Method: GET 2022-09-03T06:40:45.493 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.495 DEBUG:boto:Data: 2022-09-03T06:40:45.496 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.497 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.498 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.500 DEBUG:boto:Params: {} 2022-09-03T06:40:45.503 DEBUG:boto:Token: None 2022-09-03T06:40:45.505 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:45.508 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0BfuzhwkVqUlg0mgDkAXgOOkiLA= 2022-09-03T06:40:45.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0BfuzhwkVqUlg0mgDkAXgOOkiLA=', 'Content-Length': '0'} 2022-09-03T06:40:45.512 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000679705503963fdc-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.514 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:45.515 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:45.516 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:45.517 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:45.518 DEBUG:boto:Method: GET 2022-09-03T06:40:45.520 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:40:45.520 DEBUG:boto:Data: 2022-09-03T06:40:45.521 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.524 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.525 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.525 DEBUG:boto:Params: {} 2022-09-03T06:40:45.527 DEBUG:boto:Token: None 2022-09-03T06:40:45.529 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/ 2022-09-03T06:40:45.530 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0= 2022-09-03T06:40:45.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0=', 'Content-Length': '0'} 2022-09-03T06:40:45.536 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000083a3afa6d0354c32-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.539 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:45.541 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:45.541 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:45.542 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:45.543 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:45.543 DEBUG:boto:Method: GET 2022-09-03T06:40:45.544 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:40:45.545 DEBUG:boto:Data: 2022-09-03T06:40:45.546 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.547 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.548 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.549 DEBUG:boto:Params: {} 2022-09-03T06:40:45.550 DEBUG:boto:Token: None 2022-09-03T06:40:45.551 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/ 2022-09-03T06:40:45.552 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0= 2022-09-03T06:40:45.553 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0=', 'Content-Length': '0'} 2022-09-03T06:40:45.556 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b56769bf5b04571e-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.558 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:45.559 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:45.560 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:45.560 DEBUG:boto:Method: HEAD 2022-09-03T06:40:45.561 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:40:45.561 DEBUG:boto:Data: 2022-09-03T06:40:45.562 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.562 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.563 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.564 DEBUG:boto:Params: {} 2022-09-03T06:40:45.565 DEBUG:boto:Token: None 2022-09-03T06:40:45.568 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj51 2022-09-03T06:40:45.570 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7pId+Keuf/vnAHvEqgdYm/0QbOo= 2022-09-03T06:40:45.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7pId+Keuf/vnAHvEqgdYm/0QbOo=', 'Content-Length': '0'} 2022-09-03T06:40:45.581 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001f828b906927ab3d-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.583 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:45.584 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:45.585 DEBUG:boto:Method: HEAD 2022-09-03T06:40:45.585 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:40:45.586 DEBUG:boto:Data: 2022-09-03T06:40:45.587 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.588 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.588 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.589 DEBUG:boto:Params: {} 2022-09-03T06:40:45.589 DEBUG:boto:Token: None 2022-09-03T06:40:45.590 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj51 2022-09-03T06:40:45.591 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7pId+Keuf/vnAHvEqgdYm/0QbOo= 2022-09-03T06:40:45.591 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7pId+Keuf/vnAHvEqgdYm/0QbOo=', 'Content-Length': '0'} 2022-09-03T06:40:45.598 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e0f795f5395170b6-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.600 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:40:45.602 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:45.604 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:45.606 DEBUG:boto:Method: GET 2022-09-03T06:40:45.607 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:40:45.609 DEBUG:boto:Data: 2022-09-03T06:40:45.610 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.611 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.613 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.615 DEBUG:boto:Params: {} 2022-09-03T06:40:45.617 DEBUG:boto:Token: None 2022-09-03T06:40:45.619 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj51 2022-09-03T06:40:45.621 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6xySY6gq/Cwv9aT1CLjo90sQYzs= 2022-09-03T06:40:45.622 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6xySY6gq/Cwv9aT1CLjo90sQYzs=', 'Content-Length': '0'} 2022-09-03T06:40:45.626 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f6efa94b04ec201b-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.628 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:45.629 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:45.631 DEBUG:boto:Method: GET 2022-09-03T06:40:45.632 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:40:45.632 DEBUG:boto:Data: 2022-09-03T06:40:45.633 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.634 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.635 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.636 DEBUG:boto:Params: {} 2022-09-03T06:40:45.637 DEBUG:boto:Token: None 2022-09-03T06:40:45.638 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj51 2022-09-03T06:40:45.639 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6xySY6gq/Cwv9aT1CLjo90sQYzs= 2022-09-03T06:40:45.640 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6xySY6gq/Cwv9aT1CLjo90sQYzs=', 'Content-Length': '0'} 2022-09-03T06:40:45.643 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fe2aaffed06a338e-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.644 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:45.645 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:45.646 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:45.647 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:45.648 DEBUG:boto:Method: GET 2022-09-03T06:40:45.648 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:40:45.649 DEBUG:boto:Data: 2022-09-03T06:40:45.649 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.650 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.650 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.651 DEBUG:boto:Params: {} 2022-09-03T06:40:45.652 DEBUG:boto:Token: None 2022-09-03T06:40:45.652 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/ 2022-09-03T06:40:45.653 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0= 2022-09-03T06:40:45.654 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0=', 'Content-Length': '0'} 2022-09-03T06:40:45.657 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cec1d358f331074b-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.698 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:45.701 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:45.702 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:45.703 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:45.704 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:45.705 DEBUG:boto:Method: GET 2022-09-03T06:40:45.706 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:40:45.707 DEBUG:boto:Data: 2022-09-03T06:40:45.708 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.709 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.710 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.711 DEBUG:boto:Params: {} 2022-09-03T06:40:45.712 DEBUG:boto:Token: None 2022-09-03T06:40:45.714 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/ 2022-09-03T06:40:45.715 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0= 2022-09-03T06:40:45.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0=', 'Content-Length': '0'} 2022-09-03T06:40:45.719 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007003b6f1c7b27318-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.720 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:45.721 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:40:45.722 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:45.723 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:45.723 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.724 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.725 DEBUG:boto:Method: GET 2022-09-03T06:40:45.725 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.726 DEBUG:boto:Data: 2022-09-03T06:40:45.727 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.727 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.728 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.729 DEBUG:boto:Params: {} 2022-09-03T06:40:45.730 DEBUG:boto:Token: None 2022-09-03T06:40:45.730 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.731 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I4Xnb0fYpO5nGoD7Jm0rxzDtRgg= 2022-09-03T06:40:45.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I4Xnb0fYpO5nGoD7Jm0rxzDtRgg=', 'Content-Length': '0'} 2022-09-03T06:40:45.734 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000043c3327ef944e1d0-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.736 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:45.736 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:45.737 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.738 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.739 DEBUG:boto:Method: GET 2022-09-03T06:40:45.740 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.741 DEBUG:boto:Data: 2022-09-03T06:40:45.741 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.742 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.743 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.743 DEBUG:boto:Params: {} 2022-09-03T06:40:45.744 DEBUG:boto:Token: None 2022-09-03T06:40:45.745 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.746 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I4Xnb0fYpO5nGoD7Jm0rxzDtRgg= 2022-09-03T06:40:45.746 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I4Xnb0fYpO5nGoD7Jm0rxzDtRgg=', 'Content-Length': '0'} 2022-09-03T06:40:45.749 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c6f232bbf1fc3ae6-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.790 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:45.791 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:45.791 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.792 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.793 DEBUG:boto:Method: HEAD 2022-09-03T06:40:45.793 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.795 DEBUG:boto:Data: 2022-09-03T06:40:45.795 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.796 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.797 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.797 DEBUG:boto:Params: {} 2022-09-03T06:40:45.798 DEBUG:boto:Token: None 2022-09-03T06:40:45.799 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.800 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iIfH0ZwiDI/XYvR74q2X7JpCT+s= 2022-09-03T06:40:45.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iIfH0ZwiDI/XYvR74q2X7JpCT+s=', 'Content-Length': '0'} 2022-09-03T06:40:45.802 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cdbf9ad785a26c53-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.803 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:45.803 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:45.804 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.804 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.805 DEBUG:boto:Method: HEAD 2022-09-03T06:40:45.805 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.806 DEBUG:boto:Data: 2022-09-03T06:40:45.807 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.807 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.808 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.808 DEBUG:boto:Params: {} 2022-09-03T06:40:45.809 DEBUG:boto:Token: None 2022-09-03T06:40:45.809 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.810 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iIfH0ZwiDI/XYvR74q2X7JpCT+s= 2022-09-03T06:40:45.811 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iIfH0ZwiDI/XYvR74q2X7JpCT+s=', 'Content-Length': '0'} 2022-09-03T06:40:45.813 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007c7ac8bdd03908d9-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.813 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:40:45.814 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:45.815 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:45.815 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.816 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.816 DEBUG:boto:Method: GET 2022-09-03T06:40:45.817 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.817 DEBUG:boto:Data: 2022-09-03T06:40:45.818 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.819 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.819 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.820 DEBUG:boto:Params: {} 2022-09-03T06:40:45.821 DEBUG:boto:Token: None 2022-09-03T06:40:45.823 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.824 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I4Xnb0fYpO5nGoD7Jm0rxzDtRgg= 2022-09-03T06:40:45.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I4Xnb0fYpO5nGoD7Jm0rxzDtRgg=', 'Content-Length': '0'} 2022-09-03T06:40:45.827 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006ca26cc904a004c6-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.829 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:45.830 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:45.831 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.832 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.832 DEBUG:boto:Method: GET 2022-09-03T06:40:45.833 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.834 DEBUG:boto:Data: 2022-09-03T06:40:45.834 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.835 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.836 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.837 DEBUG:boto:Params: {} 2022-09-03T06:40:45.837 DEBUG:boto:Token: None 2022-09-03T06:40:45.838 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:45.839 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I4Xnb0fYpO5nGoD7Jm0rxzDtRgg= 2022-09-03T06:40:45.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I4Xnb0fYpO5nGoD7Jm0rxzDtRgg=', 'Content-Length': '0'} 2022-09-03T06:40:45.843 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b8202387ab8f24a-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.844 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:45.844 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:45.845 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:45.846 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:45.847 DEBUG:boto:Method: GET 2022-09-03T06:40:45.848 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:40:45.848 DEBUG:boto:Data: 2022-09-03T06:40:45.849 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.850 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.851 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.852 DEBUG:boto:Params: {} 2022-09-03T06:40:45.853 DEBUG:boto:Token: None 2022-09-03T06:40:45.854 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/ 2022-09-03T06:40:45.855 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0= 2022-09-03T06:40:45.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0=', 'Content-Length': '0'} 2022-09-03T06:40:45.859 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000016fa6d32ca4502f6-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.902 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:45.903 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:45.904 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:45.905 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:45.905 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:45.906 DEBUG:boto:Method: GET 2022-09-03T06:40:45.906 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:40:45.907 DEBUG:boto:Data: 2022-09-03T06:40:45.908 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.908 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.909 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.910 DEBUG:boto:Params: {} 2022-09-03T06:40:45.911 DEBUG:boto:Token: None 2022-09-03T06:40:45.911 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/ 2022-09-03T06:40:45.912 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0= 2022-09-03T06:40:45.913 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0=', 'Content-Length': '0'} 2022-09-03T06:40:45.916 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006f12173bdd202584-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.917 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:45.918 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:45.919 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:45.920 DEBUG:boto:Method: HEAD 2022-09-03T06:40:45.921 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:40:45.922 DEBUG:boto:Data: 2022-09-03T06:40:45.922 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.923 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.924 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.924 DEBUG:boto:Params: {} 2022-09-03T06:40:45.925 DEBUG:boto:Token: None 2022-09-03T06:40:45.926 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj55 2022-09-03T06:40:45.926 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:geD2354cKSc8dAe6GoKv2LFq300= 2022-09-03T06:40:45.927 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:geD2354cKSc8dAe6GoKv2LFq300=', 'Content-Length': '0'} 2022-09-03T06:40:45.929 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005cf8d42bb4813eae-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.931 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:45.931 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:45.932 DEBUG:boto:Method: HEAD 2022-09-03T06:40:45.933 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:40:45.933 DEBUG:boto:Data: 2022-09-03T06:40:45.934 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.934 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.935 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.935 DEBUG:boto:Params: {} 2022-09-03T06:40:45.936 DEBUG:boto:Token: None 2022-09-03T06:40:45.937 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj55 2022-09-03T06:40:45.937 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:geD2354cKSc8dAe6GoKv2LFq300= 2022-09-03T06:40:45.938 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:geD2354cKSc8dAe6GoKv2LFq300=', 'Content-Length': '0'} 2022-09-03T06:40:45.940 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f1ccf257c544df99-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.941 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:40:45.942 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:45.943 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:45.943 DEBUG:boto:Method: GET 2022-09-03T06:40:45.944 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:40:45.944 DEBUG:boto:Data: 2022-09-03T06:40:45.945 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.945 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.946 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.946 DEBUG:boto:Params: {} 2022-09-03T06:40:45.947 DEBUG:boto:Token: None 2022-09-03T06:40:45.948 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj55 2022-09-03T06:40:45.948 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PDvnbPMHubg3aTVKxtbm+KvecZo= 2022-09-03T06:40:45.949 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PDvnbPMHubg3aTVKxtbm+KvecZo=', 'Content-Length': '0'} 2022-09-03T06:40:45.951 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c81ccc96950784c1-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.952 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:45.953 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:45.953 DEBUG:boto:Method: GET 2022-09-03T06:40:45.954 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:40:45.955 DEBUG:boto:Data: 2022-09-03T06:40:45.955 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.956 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.956 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.957 DEBUG:boto:Params: {} 2022-09-03T06:40:45.958 DEBUG:boto:Token: None 2022-09-03T06:40:45.958 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/obj55 2022-09-03T06:40:45.959 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PDvnbPMHubg3aTVKxtbm+KvecZo= 2022-09-03T06:40:45.960 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PDvnbPMHubg3aTVKxtbm+KvecZo=', 'Content-Length': '0'} 2022-09-03T06:40:45.962 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000068c41a3ea21f4545-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.964 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:45.965 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:45.966 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:45.967 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:45.968 DEBUG:boto:Method: GET 2022-09-03T06:40:45.969 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:40:45.970 DEBUG:boto:Data: 2022-09-03T06:40:45.971 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.972 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:45.973 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.974 DEBUG:boto:Params: {} 2022-09-03T06:40:45.975 DEBUG:boto:Token: None 2022-09-03T06:40:45.976 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/ 2022-09-03T06:40:45.977 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0= 2022-09-03T06:40:45.978 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0=', 'Content-Length': '0'} 2022-09-03T06:40:45.982 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007d17d7cbdc538d86-006312f6ed-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:45.983 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:45.984 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:45.985 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:45.986 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:45.987 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:45.988 DEBUG:boto:Method: GET 2022-09-03T06:40:45.988 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:40:45.989 DEBUG:boto:Data: 2022-09-03T06:40:45.990 DEBUG:boto:Headers: {} 2022-09-03T06:40:45.990 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:45.992 DEBUG:boto:Port: 8000 2022-09-03T06:40:45.992 DEBUG:boto:Params: {} 2022-09-03T06:40:45.993 DEBUG:boto:Token: None 2022-09-03T06:40:45.993 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:45 GMT /dsgvcy-75/ 2022-09-03T06:40:45.994 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0= 2022-09-03T06:40:45.995 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3dabUV90wrxZA1fy+Z1VtbRXBL0=', 'Content-Length': '0'} 2022-09-03T06:40:45.999 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000f5eb163662582ad-006312f6ed-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.000 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:46.001 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:40:46.002 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:46.003 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:46.003 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.004 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.004 DEBUG:boto:Method: GET 2022-09-03T06:40:46.005 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.005 DEBUG:boto:Data: 2022-09-03T06:40:46.006 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.007 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:46.008 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.008 DEBUG:boto:Params: {} 2022-09-03T06:40:46.009 DEBUG:boto:Token: None 2022-09-03T06:40:46.009 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.011 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A8iRVnpKrFoYwfotS+i7uOlhE7I= 2022-09-03T06:40:46.012 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A8iRVnpKrFoYwfotS+i7uOlhE7I=', 'Content-Length': '0'} 2022-09-03T06:40:46.015 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a933ebb07aa02d44-006312f6ee-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.016 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:46.017 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:46.017 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.018 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.018 DEBUG:boto:Method: GET 2022-09-03T06:40:46.019 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.019 DEBUG:boto:Data: 2022-09-03T06:40:46.020 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.022 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:46.022 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.023 DEBUG:boto:Params: {} 2022-09-03T06:40:46.023 DEBUG:boto:Token: None 2022-09-03T06:40:46.024 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.025 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A8iRVnpKrFoYwfotS+i7uOlhE7I= 2022-09-03T06:40:46.025 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A8iRVnpKrFoYwfotS+i7uOlhE7I=', 'Content-Length': '0'} 2022-09-03T06:40:46.028 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e4f95ba749feb046-006312f6ee-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.074 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:46.075 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:46.076 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.077 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.078 DEBUG:boto:Method: HEAD 2022-09-03T06:40:46.079 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.079 DEBUG:boto:Data: 2022-09-03T06:40:46.080 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.081 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:46.082 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.082 DEBUG:boto:Params: {} 2022-09-03T06:40:46.083 DEBUG:boto:Token: None 2022-09-03T06:40:46.084 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.096 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WR+fT/I5HggMSfJ1yoyAcqNPJ5I= 2022-09-03T06:40:46.097 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WR+fT/I5HggMSfJ1yoyAcqNPJ5I=', 'Content-Length': '0'} 2022-09-03T06:40:46.099 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c547a836b1d1338-006312f6ee-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.100 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:46.101 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:46.102 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.102 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.103 DEBUG:boto:Method: HEAD 2022-09-03T06:40:46.104 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.105 DEBUG:boto:Data: 2022-09-03T06:40:46.105 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.107 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:46.108 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.108 DEBUG:boto:Params: {} 2022-09-03T06:40:46.110 DEBUG:boto:Token: None 2022-09-03T06:40:46.110 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.111 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WR+fT/I5HggMSfJ1yoyAcqNPJ5I= 2022-09-03T06:40:46.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WR+fT/I5HggMSfJ1yoyAcqNPJ5I=', 'Content-Length': '0'} 2022-09-03T06:40:46.114 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000514cdff2d6d36f3b-006312f6ee-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.116 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:40:46.116 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:46.117 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:46.119 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.120 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.121 DEBUG:boto:Method: GET 2022-09-03T06:40:46.122 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.122 DEBUG:boto:Data: 2022-09-03T06:40:46.123 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.124 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:46.124 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.125 DEBUG:boto:Params: {} 2022-09-03T06:40:46.126 DEBUG:boto:Token: None 2022-09-03T06:40:46.127 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.129 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A8iRVnpKrFoYwfotS+i7uOlhE7I= 2022-09-03T06:40:46.130 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A8iRVnpKrFoYwfotS+i7uOlhE7I=', 'Content-Length': '0'} 2022-09-03T06:40:46.142 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e2bca2029171d6ee-006312f6ee-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.144 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:46.145 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:46.146 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.147 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.147 DEBUG:boto:Method: GET 2022-09-03T06:40:46.148 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.149 DEBUG:boto:Data: 2022-09-03T06:40:46.154 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.155 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:46.155 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.156 DEBUG:boto:Params: {} 2022-09-03T06:40:46.157 DEBUG:boto:Token: None 2022-09-03T06:40:46.157 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:46.158 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A8iRVnpKrFoYwfotS+i7uOlhE7I= 2022-09-03T06:40:46.159 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A8iRVnpKrFoYwfotS+i7uOlhE7I=', 'Content-Length': '0'} 2022-09-03T06:40:46.163 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000319748f940137ac9-006312f6ee-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.164 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:46.166 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:46.167 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:46.167 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:46.168 DEBUG:boto:Method: GET 2022-09-03T06:40:46.168 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:40:46.169 DEBUG:boto:Data: 2022-09-03T06:40:46.169 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.170 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:46.171 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.171 DEBUG:boto:Params: {} 2022-09-03T06:40:46.172 DEBUG:boto:Token: None 2022-09-03T06:40:46.172 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/ 2022-09-03T06:40:46.173 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WUCn4hpAd7YQr3A6vv7r/twH5NA= 2022-09-03T06:40:46.174 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WUCn4hpAd7YQr3A6vv7r/twH5NA=', 'Content-Length': '0'} 2022-09-03T06:40:46.178 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cd50b669a6b01a15-006312f6ee-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.179 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:46.180 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:46.181 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:46.181 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:46.182 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:46.183 DEBUG:boto:Method: GET 2022-09-03T06:40:46.184 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:40:46.185 DEBUG:boto:Data: 2022-09-03T06:40:46.185 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.186 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:46.187 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.188 DEBUG:boto:Params: {} 2022-09-03T06:40:46.188 DEBUG:boto:Token: None 2022-09-03T06:40:46.189 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/ 2022-09-03T06:40:46.191 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WUCn4hpAd7YQr3A6vv7r/twH5NA= 2022-09-03T06:40:46.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WUCn4hpAd7YQr3A6vv7r/twH5NA=', 'Content-Length': '0'} 2022-09-03T06:40:46.203 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e9d400f913cb74f5-006312f6ee-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.205 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:46.207 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:46.209 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:46.210 DEBUG:boto:Method: HEAD 2022-09-03T06:40:46.215 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:40:46.216 DEBUG:boto:Data: 2022-09-03T06:40:46.217 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.218 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:46.219 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.220 DEBUG:boto:Params: {} 2022-09-03T06:40:46.220 DEBUG:boto:Token: None 2022-09-03T06:40:46.221 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/obj68 2022-09-03T06:40:46.222 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QcmA+SMFPFY3pcTxSbI3MI4jN0s= 2022-09-03T06:40:46.224 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QcmA+SMFPFY3pcTxSbI3MI4jN0s=', 'Content-Length': '0'} 2022-09-03T06:40:46.226 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dd1cadbf9736eccb-006312f6ee-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.228 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:46.229 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:46.231 DEBUG:boto:Method: HEAD 2022-09-03T06:40:46.233 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:40:46.233 DEBUG:boto:Data: 2022-09-03T06:40:46.234 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.234 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:46.235 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.236 DEBUG:boto:Params: {} 2022-09-03T06:40:46.237 DEBUG:boto:Token: None 2022-09-03T06:40:46.238 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/obj68 2022-09-03T06:40:46.239 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QcmA+SMFPFY3pcTxSbI3MI4jN0s= 2022-09-03T06:40:46.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QcmA+SMFPFY3pcTxSbI3MI4jN0s=', 'Content-Length': '0'} 2022-09-03T06:40:46.243 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005130d69af5f0b3ae-006312f6ee-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.245 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:40:46.246 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:46.247 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:46.248 DEBUG:boto:Method: GET 2022-09-03T06:40:46.249 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:40:46.250 DEBUG:boto:Data: 2022-09-03T06:40:46.251 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.252 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:46.253 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.254 DEBUG:boto:Params: {} 2022-09-03T06:40:46.255 DEBUG:boto:Token: None 2022-09-03T06:40:46.256 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/obj68 2022-09-03T06:40:46.258 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YzUMQTAUxKd+Ae1gJTGKregW2XU= 2022-09-03T06:40:46.259 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YzUMQTAUxKd+Ae1gJTGKregW2XU=', 'Content-Length': '0'} 2022-09-03T06:40:46.263 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fd3907286129a115-006312f6ee-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.307 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:46.308 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:46.309 DEBUG:boto:Method: GET 2022-09-03T06:40:46.309 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:40:46.310 DEBUG:boto:Data: 2022-09-03T06:40:46.310 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.311 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:46.312 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.312 DEBUG:boto:Params: {} 2022-09-03T06:40:46.313 DEBUG:boto:Token: None 2022-09-03T06:40:46.314 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/obj68 2022-09-03T06:40:46.314 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YzUMQTAUxKd+Ae1gJTGKregW2XU= 2022-09-03T06:40:46.315 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YzUMQTAUxKd+Ae1gJTGKregW2XU=', 'Content-Length': '0'} 2022-09-03T06:40:46.318 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fa91e46c16136005-006312f6ee-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.319 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:46.319 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:46.320 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:46.320 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:46.321 DEBUG:boto:Method: GET 2022-09-03T06:40:46.322 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:40:46.322 DEBUG:boto:Data: 2022-09-03T06:40:46.323 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.323 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:46.325 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.328 DEBUG:boto:Params: {} 2022-09-03T06:40:46.329 DEBUG:boto:Token: None 2022-09-03T06:40:46.332 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/ 2022-09-03T06:40:46.332 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WUCn4hpAd7YQr3A6vv7r/twH5NA= 2022-09-03T06:40:46.333 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WUCn4hpAd7YQr3A6vv7r/twH5NA=', 'Content-Length': '0'} 2022-09-03T06:40:46.337 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c5969f4ab1108632-006312f6ee-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.338 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:46.340 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:46.341 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:46.343 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:46.344 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:46.345 DEBUG:boto:Method: GET 2022-09-03T06:40:46.346 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:40:46.347 DEBUG:boto:Data: 2022-09-03T06:40:46.348 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.349 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:46.350 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.351 DEBUG:boto:Params: {} 2022-09-03T06:40:46.352 DEBUG:boto:Token: None 2022-09-03T06:40:46.353 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/ 2022-09-03T06:40:46.354 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WUCn4hpAd7YQr3A6vv7r/twH5NA= 2022-09-03T06:40:46.355 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WUCn4hpAd7YQr3A6vv7r/twH5NA=', 'Content-Length': '0'} 2022-09-03T06:40:46.359 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000d36592821558cc3-006312f6ee-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.360 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:46.361 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:40:46.361 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:46.362 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:46.362 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.363 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.363 DEBUG:boto:Method: GET 2022-09-03T06:40:46.364 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.364 DEBUG:boto:Data: 2022-09-03T06:40:46.365 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.365 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:46.366 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.367 DEBUG:boto:Params: {} 2022-09-03T06:40:46.369 DEBUG:boto:Token: None 2022-09-03T06:40:46.370 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.370 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V/BTZ8k2z49DOvzbdJrpm5Hd9OQ= 2022-09-03T06:40:46.371 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V/BTZ8k2z49DOvzbdJrpm5Hd9OQ=', 'Content-Length': '0'} 2022-09-03T06:40:46.374 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f15b0f815d5b976-006312f6ee-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.375 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:46.376 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:46.376 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.377 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.377 DEBUG:boto:Method: GET 2022-09-03T06:40:46.378 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.378 DEBUG:boto:Data: 2022-09-03T06:40:46.379 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.379 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:46.380 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.380 DEBUG:boto:Params: {} 2022-09-03T06:40:46.381 DEBUG:boto:Token: None 2022-09-03T06:40:46.382 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.382 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V/BTZ8k2z49DOvzbdJrpm5Hd9OQ= 2022-09-03T06:40:46.383 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V/BTZ8k2z49DOvzbdJrpm5Hd9OQ=', 'Content-Length': '0'} 2022-09-03T06:40:46.390 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000042c9af9583278cd1-006312f6ee-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.391 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:46.392 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:46.392 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.392 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.393 DEBUG:boto:Method: HEAD 2022-09-03T06:40:46.394 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.394 DEBUG:boto:Data: 2022-09-03T06:40:46.395 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.395 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:46.396 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.396 DEBUG:boto:Params: {} 2022-09-03T06:40:46.397 DEBUG:boto:Token: None 2022-09-03T06:40:46.397 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.398 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rH2uNSPWnvAIS7BiUDozBIwZCZk= 2022-09-03T06:40:46.399 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rH2uNSPWnvAIS7BiUDozBIwZCZk=', 'Content-Length': '0'} 2022-09-03T06:40:46.409 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003791bafb22cf7b7f-006312f6ee-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.410 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:46.411 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:46.412 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.412 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.413 DEBUG:boto:Method: HEAD 2022-09-03T06:40:46.413 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.414 DEBUG:boto:Data: 2022-09-03T06:40:46.415 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.415 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:46.416 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.416 DEBUG:boto:Params: {} 2022-09-03T06:40:46.417 DEBUG:boto:Token: None 2022-09-03T06:40:46.418 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.419 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rH2uNSPWnvAIS7BiUDozBIwZCZk= 2022-09-03T06:40:46.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rH2uNSPWnvAIS7BiUDozBIwZCZk=', 'Content-Length': '0'} 2022-09-03T06:40:46.422 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000039fc73aaf22aaf34-006312f6ee-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.424 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:40:46.425 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:46.426 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:46.427 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.428 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.429 DEBUG:boto:Method: GET 2022-09-03T06:40:46.430 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.431 DEBUG:boto:Data: 2022-09-03T06:40:46.432 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.433 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:46.433 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.434 DEBUG:boto:Params: {} 2022-09-03T06:40:46.435 DEBUG:boto:Token: None 2022-09-03T06:40:46.436 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.436 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V/BTZ8k2z49DOvzbdJrpm5Hd9OQ= 2022-09-03T06:40:46.437 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V/BTZ8k2z49DOvzbdJrpm5Hd9OQ=', 'Content-Length': '0'} 2022-09-03T06:40:46.439 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cc0d5c30520cd950-006312f6ee-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.440 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:46.441 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:46.441 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.442 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.443 DEBUG:boto:Method: GET 2022-09-03T06:40:46.443 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.444 DEBUG:boto:Data: 2022-09-03T06:40:46.444 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.445 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:46.445 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.446 DEBUG:boto:Params: {} 2022-09-03T06:40:46.447 DEBUG:boto:Token: None 2022-09-03T06:40:46.447 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:46.448 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V/BTZ8k2z49DOvzbdJrpm5Hd9OQ= 2022-09-03T06:40:46.449 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V/BTZ8k2z49DOvzbdJrpm5Hd9OQ=', 'Content-Length': '0'} 2022-09-03T06:40:46.456 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009658d266e959ec88-006312f6ee-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.495 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:46.497 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:46.498 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:46.499 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:46.500 DEBUG:boto:Method: GET 2022-09-03T06:40:46.501 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:40:46.501 DEBUG:boto:Data: 2022-09-03T06:40:46.502 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.504 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:46.505 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.506 DEBUG:boto:Params: {} 2022-09-03T06:40:46.506 DEBUG:boto:Token: None 2022-09-03T06:40:46.507 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/ 2022-09-03T06:40:46.508 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WUCn4hpAd7YQr3A6vv7r/twH5NA= 2022-09-03T06:40:46.509 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WUCn4hpAd7YQr3A6vv7r/twH5NA=', 'Content-Length': '0'} 2022-09-03T06:40:46.514 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ba7642eb6a70a706-006312f6ee-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.520 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:46.522 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:46.523 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:46.524 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:46.525 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:46.526 DEBUG:boto:Method: GET 2022-09-03T06:40:46.527 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:40:46.527 DEBUG:boto:Data: 2022-09-03T06:40:46.528 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.528 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:46.529 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.530 DEBUG:boto:Params: {} 2022-09-03T06:40:46.530 DEBUG:boto:Token: None 2022-09-03T06:40:46.531 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/ 2022-09-03T06:40:46.532 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WUCn4hpAd7YQr3A6vv7r/twH5NA= 2022-09-03T06:40:46.533 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WUCn4hpAd7YQr3A6vv7r/twH5NA=', 'Content-Length': '0'} 2022-09-03T06:40:46.536 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009432d8aedb969324-006312f6ee-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.537 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:46.539 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:46.540 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:46.540 DEBUG:boto:Method: HEAD 2022-09-03T06:40:46.541 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:40:46.542 DEBUG:boto:Data: 2022-09-03T06:40:46.543 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.544 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:46.545 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.546 DEBUG:boto:Params: {} 2022-09-03T06:40:46.547 DEBUG:boto:Token: None 2022-09-03T06:40:46.547 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/obj85 2022-09-03T06:40:46.548 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6mqmUDCbIUyLvW8F/15YIEq8kuY= 2022-09-03T06:40:46.549 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6mqmUDCbIUyLvW8F/15YIEq8kuY=', 'Content-Length': '0'} 2022-09-03T06:40:46.556 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a245cbb09f6cb1b-006312f6ee-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.557 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:46.559 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:46.559 DEBUG:boto:Method: HEAD 2022-09-03T06:40:46.560 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:40:46.561 DEBUG:boto:Data: 2022-09-03T06:40:46.563 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.564 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:46.564 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.565 DEBUG:boto:Params: {} 2022-09-03T06:40:46.566 DEBUG:boto:Token: None 2022-09-03T06:40:46.567 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/obj85 2022-09-03T06:40:46.568 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6mqmUDCbIUyLvW8F/15YIEq8kuY= 2022-09-03T06:40:46.569 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6mqmUDCbIUyLvW8F/15YIEq8kuY=', 'Content-Length': '0'} 2022-09-03T06:40:46.573 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a5ae7d7faeab3c7d-006312f6ee-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.575 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:40:46.576 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:46.577 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:46.578 DEBUG:boto:Method: GET 2022-09-03T06:40:46.579 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:40:46.580 DEBUG:boto:Data: 2022-09-03T06:40:46.582 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.583 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:46.585 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.586 DEBUG:boto:Params: {} 2022-09-03T06:40:46.587 DEBUG:boto:Token: None 2022-09-03T06:40:46.589 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/obj85 2022-09-03T06:40:46.590 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hNV6O3jf27HfLcXaCMhEcFP36Dg= 2022-09-03T06:40:46.592 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hNV6O3jf27HfLcXaCMhEcFP36Dg=', 'Content-Length': '0'} 2022-09-03T06:40:46.597 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009375fdab00ec373a-006312f6ee-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.638 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:46.639 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:46.640 DEBUG:boto:Method: GET 2022-09-03T06:40:46.641 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:40:46.642 DEBUG:boto:Data: 2022-09-03T06:40:46.643 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.643 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:46.644 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.645 DEBUG:boto:Params: {} 2022-09-03T06:40:46.645 DEBUG:boto:Token: None 2022-09-03T06:40:46.646 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/obj85 2022-09-03T06:40:46.647 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hNV6O3jf27HfLcXaCMhEcFP36Dg= 2022-09-03T06:40:46.648 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hNV6O3jf27HfLcXaCMhEcFP36Dg=', 'Content-Length': '0'} 2022-09-03T06:40:46.657 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008c7b3d3c2bf7c2fb-006312f6ee-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.660 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:46.662 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:46.663 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:46.664 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:46.664 DEBUG:boto:Method: GET 2022-09-03T06:40:46.665 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:40:46.667 DEBUG:boto:Data: 2022-09-03T06:40:46.668 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.668 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:46.669 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.670 DEBUG:boto:Params: {} 2022-09-03T06:40:46.671 DEBUG:boto:Token: None 2022-09-03T06:40:46.672 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/ 2022-09-03T06:40:46.673 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WUCn4hpAd7YQr3A6vv7r/twH5NA= 2022-09-03T06:40:46.675 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WUCn4hpAd7YQr3A6vv7r/twH5NA=', 'Content-Length': '0'} 2022-09-03T06:40:46.688 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005351211b1965bedc-006312f6ee-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.689 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:46.691 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:46.692 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:46.693 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:46.694 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:46.695 DEBUG:boto:Method: GET 2022-09-03T06:40:46.696 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:40:46.697 DEBUG:boto:Data: 2022-09-03T06:40:46.699 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.702 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:46.703 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.704 DEBUG:boto:Params: {} 2022-09-03T06:40:46.705 DEBUG:boto:Token: None 2022-09-03T06:40:46.706 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/ 2022-09-03T06:40:46.707 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WUCn4hpAd7YQr3A6vv7r/twH5NA= 2022-09-03T06:40:46.707 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WUCn4hpAd7YQr3A6vv7r/twH5NA=', 'Content-Length': '0'} 2022-09-03T06:40:46.717 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002b5c107ea50e2fd6-006312f6ee-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.718 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:46.720 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-75 zones={test-zone2, test-zone1} 2022-09-03T06:40:46.720 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-75 zones={test-zone3, test-zone1} 2022-09-03T06:40:46.721 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:46.721 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:46.722 DEBUG:boto:Method: HEAD 2022-09-03T06:40:46.723 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:40:46.723 DEBUG:boto:Data: 2022-09-03T06:40:46.724 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.725 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:46.725 DEBUG:boto:Port: 8001 2022-09-03T06:40:46.726 DEBUG:boto:Params: {} 2022-09-03T06:40:46.727 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T06:40:46.728 DEBUG:boto:Token: None 2022-09-03T06:40:46.729 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/ 2022-09-03T06:40:46.731 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e4qZfweKoD7UNW/0Atl4spfYS4M= 2022-09-03T06:40:46.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e4qZfweKoD7UNW/0Atl4spfYS4M=', 'Content-Length': '0'} 2022-09-03T06:40:46.737 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '10'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003ea797dadf27b9de-006312f6ee-1851-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.739 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:46.740 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:46.741 DEBUG:boto:Method: HEAD 2022-09-03T06:40:46.743 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:40:46.746 DEBUG:boto:Data: 2022-09-03T06:40:46.746 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.747 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:46.747 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.748 DEBUG:boto:Params: {} 2022-09-03T06:40:46.749 DEBUG:boto:Token: None 2022-09-03T06:40:46.749 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/ 2022-09-03T06:40:46.750 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e4qZfweKoD7UNW/0Atl4spfYS4M= 2022-09-03T06:40:46.750 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e4qZfweKoD7UNW/0Atl4spfYS4M=', 'Content-Length': '0'} 2022-09-03T06:40:46.756 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '10'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b33b6c7db7b49b75-006312f6ee-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.756 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:40:46.757 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:46.758 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:46.758 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:46.759 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:46.759 DEBUG:boto:Method: GET 2022-09-03T06:40:46.760 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:46.761 DEBUG:boto:Data: 2022-09-03T06:40:46.761 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.762 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:46.762 DEBUG:boto:Port: 8001 2022-09-03T06:40:46.763 DEBUG:boto:Params: {} 2022-09-03T06:40:46.764 DEBUG:boto:Token: None 2022-09-03T06:40:46.765 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/?versions 2022-09-03T06:40:46.766 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GFt4BG3T1r0ySaoXSD3A8r5U+MA= 2022-09-03T06:40:46.767 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GFt4BG3T1r0ySaoXSD3A8r5U+MA=', 'Content-Length': '0'} 2022-09-03T06:40:46.770 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003ef55c79413fab7f-006312f6ee-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.772 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:46.776 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:40:46.777 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:40:46.778 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:40:46.781 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:40:46.782 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:40:46.785 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:40:46.787 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:40:46.787 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:40:46.788 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:40:46.789 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:40:46.790 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:40:46.791 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:46.792 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:46.793 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:46.795 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:46.796 DEBUG:boto:Method: GET 2022-09-03T06:40:46.797 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:46.798 DEBUG:boto:Data: 2022-09-03T06:40:46.799 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.800 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:46.801 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.802 DEBUG:boto:Params: {} 2022-09-03T06:40:46.803 DEBUG:boto:Token: None 2022-09-03T06:40:46.804 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/?versions 2022-09-03T06:40:46.805 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GFt4BG3T1r0ySaoXSD3A8r5U+MA= 2022-09-03T06:40:46.807 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GFt4BG3T1r0ySaoXSD3A8r5U+MA=', 'Content-Length': '0'} 2022-09-03T06:40:46.819 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a620df32d7ba13b0-006312f6ee-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.820 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:46.822 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:40:46.823 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:40:46.823 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:40:46.824 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:40:46.825 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:40:46.826 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:40:46.827 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:40:46.828 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:40:46.829 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:40:46.830 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:40:46.831 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:46.832 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:46.833 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:46.834 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:46.835 DEBUG:boto:Method: GET 2022-09-03T06:40:46.836 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:46.837 DEBUG:boto:Data: 2022-09-03T06:40:46.837 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.838 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:46.838 DEBUG:boto:Port: 8001 2022-09-03T06:40:46.839 DEBUG:boto:Params: {} 2022-09-03T06:40:46.840 DEBUG:boto:Token: None 2022-09-03T06:40:46.841 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/?versions 2022-09-03T06:40:46.842 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GFt4BG3T1r0ySaoXSD3A8r5U+MA= 2022-09-03T06:40:46.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GFt4BG3T1r0ySaoXSD3A8r5U+MA=', 'Content-Length': '0'} 2022-09-03T06:40:46.845 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008c64bbd9a140f4eb-006312f6ee-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.846 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:46.848 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:46.848 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:46.849 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:46.850 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:46.851 DEBUG:boto:Method: GET 2022-09-03T06:40:46.851 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:46.853 DEBUG:boto:Data: 2022-09-03T06:40:46.854 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.855 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:46.855 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.856 DEBUG:boto:Params: {} 2022-09-03T06:40:46.856 DEBUG:boto:Token: None 2022-09-03T06:40:46.857 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/?versions 2022-09-03T06:40:46.858 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GFt4BG3T1r0ySaoXSD3A8r5U+MA= 2022-09-03T06:40:46.859 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GFt4BG3T1r0ySaoXSD3A8r5U+MA=', 'Content-Length': '0'} 2022-09-03T06:40:46.862 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aa633a07876358a1-006312f6ee-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.863 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:46.865 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:40:46.866 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:46.867 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:46.868 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:46.869 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:46.870 DEBUG:boto:Method: GET 2022-09-03T06:40:46.871 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:46.872 DEBUG:boto:Data: 2022-09-03T06:40:46.873 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.874 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:46.875 DEBUG:boto:Port: 8001 2022-09-03T06:40:46.876 DEBUG:boto:Params: {} 2022-09-03T06:40:46.878 DEBUG:boto:Token: None 2022-09-03T06:40:46.879 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:46.880 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E5qTvmx3FodYcIf4yQEHZ1lRwyQ= 2022-09-03T06:40:46.881 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E5qTvmx3FodYcIf4yQEHZ1lRwyQ=', 'Content-Length': '0'} 2022-09-03T06:40:46.887 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e3cbcf0d23f57ec-006312f6ee-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.889 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:46.890 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:46.891 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:46.892 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:46.893 DEBUG:boto:Method: GET 2022-09-03T06:40:46.894 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:46.895 DEBUG:boto:Data: 2022-09-03T06:40:46.896 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.897 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:46.898 DEBUG:boto:Port: 8000 2022-09-03T06:40:46.899 DEBUG:boto:Params: {} 2022-09-03T06:40:46.900 DEBUG:boto:Token: None 2022-09-03T06:40:46.901 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:46.902 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E5qTvmx3FodYcIf4yQEHZ1lRwyQ= 2022-09-03T06:40:46.903 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E5qTvmx3FodYcIf4yQEHZ1lRwyQ=', 'Content-Length': '0'} 2022-09-03T06:40:46.923 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000046db8a76d351b76-006312f6ee-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:46.961 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:46.962 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:46.963 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:46.964 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:46.965 DEBUG:boto:Method: HEAD 2022-09-03T06:40:46.971 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:46.972 DEBUG:boto:Data: 2022-09-03T06:40:46.973 DEBUG:boto:Headers: {} 2022-09-03T06:40:46.980 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:46.982 DEBUG:boto:Port: 8001 2022-09-03T06:40:46.983 DEBUG:boto:Params: {} 2022-09-03T06:40:46.984 DEBUG:boto:Token: None 2022-09-03T06:40:46.985 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:46 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:46.987 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TuO6lxqs0H68+XlyPR/2+Tpb/vk= 2022-09-03T06:40:46.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TuO6lxqs0H68+XlyPR/2+Tpb/vk=', 'Content-Length': '0'} 2022-09-03T06:40:47.000 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000131a716d0dafe5c0-006312f6ee-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.002 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:47.003 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:47.004 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:47.008 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:47.009 DEBUG:boto:Method: HEAD 2022-09-03T06:40:47.010 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:47.010 DEBUG:boto:Data: 2022-09-03T06:40:47.011 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.012 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.013 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.014 DEBUG:boto:Params: {} 2022-09-03T06:40:47.015 DEBUG:boto:Token: None 2022-09-03T06:40:47.016 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:47.017 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KBjnmLq4Lx/473icnXGDTukWq1Q= 2022-09-03T06:40:47.018 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KBjnmLq4Lx/473icnXGDTukWq1Q=', 'Content-Length': '0'} 2022-09-03T06:40:47.023 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a7ffaf20668ac5f-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.026 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:40:47.028 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:47.031 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:47.032 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:47.034 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:47.035 DEBUG:boto:Method: GET 2022-09-03T06:40:47.035 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:47.036 DEBUG:boto:Data: 2022-09-03T06:40:47.037 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.037 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.038 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.038 DEBUG:boto:Params: {} 2022-09-03T06:40:47.040 DEBUG:boto:Token: None 2022-09-03T06:40:47.040 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:47.041 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:trH77WR5D5R2egtI9s6BFLU8s5c= 2022-09-03T06:40:47.042 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:trH77WR5D5R2egtI9s6BFLU8s5c=', 'Content-Length': '0'} 2022-09-03T06:40:47.054 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021ea9e0a5cb172f9-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.056 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:47.057 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:47.058 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:47.060 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:47.061 DEBUG:boto:Method: GET 2022-09-03T06:40:47.062 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:47.064 DEBUG:boto:Data: 2022-09-03T06:40:47.065 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.066 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.069 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.070 DEBUG:boto:Params: {} 2022-09-03T06:40:47.072 DEBUG:boto:Token: None 2022-09-03T06:40:47.073 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:47.075 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:trH77WR5D5R2egtI9s6BFLU8s5c= 2022-09-03T06:40:47.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:trH77WR5D5R2egtI9s6BFLU8s5c=', 'Content-Length': '0'} 2022-09-03T06:40:47.085 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001e1b27f3194b876b-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.087 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:47.088 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:47.089 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:47.089 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:47.090 DEBUG:boto:Method: GET 2022-09-03T06:40:47.091 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:40:47.092 DEBUG:boto:Data: 2022-09-03T06:40:47.093 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.094 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.095 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.096 DEBUG:boto:Params: {} 2022-09-03T06:40:47.098 DEBUG:boto:Token: None 2022-09-03T06:40:47.099 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/ 2022-09-03T06:40:47.100 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c= 2022-09-03T06:40:47.101 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c=', 'Content-Length': '0'} 2022-09-03T06:40:47.105 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005cb29d0d3200b415-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.106 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:47.108 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:47.109 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:47.110 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:47.110 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:47.111 DEBUG:boto:Method: GET 2022-09-03T06:40:47.111 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:40:47.112 DEBUG:boto:Data: 2022-09-03T06:40:47.112 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.113 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.113 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.114 DEBUG:boto:Params: {} 2022-09-03T06:40:47.114 DEBUG:boto:Token: None 2022-09-03T06:40:47.115 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/ 2022-09-03T06:40:47.116 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c= 2022-09-03T06:40:47.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c=', 'Content-Length': '0'} 2022-09-03T06:40:47.125 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b385b2e84f35760a-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.127 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:47.129 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:47.130 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:47.130 DEBUG:boto:Method: HEAD 2022-09-03T06:40:47.131 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:40:47.131 DEBUG:boto:Data: 2022-09-03T06:40:47.132 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.133 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.134 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.136 DEBUG:boto:Params: {} 2022-09-03T06:40:47.137 DEBUG:boto:Token: None 2022-09-03T06:40:47.138 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj11 2022-09-03T06:40:47.139 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zxOThnXJW5QPlDYX2oyN1Xa81ZI= 2022-09-03T06:40:47.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zxOThnXJW5QPlDYX2oyN1Xa81ZI=', 'Content-Length': '0'} 2022-09-03T06:40:47.142 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cd5176cf895be992-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.144 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:47.144 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:47.145 DEBUG:boto:Method: HEAD 2022-09-03T06:40:47.146 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:40:47.147 DEBUG:boto:Data: 2022-09-03T06:40:47.148 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.148 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.149 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.149 DEBUG:boto:Params: {} 2022-09-03T06:40:47.150 DEBUG:boto:Token: None 2022-09-03T06:40:47.151 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj11 2022-09-03T06:40:47.151 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zxOThnXJW5QPlDYX2oyN1Xa81ZI= 2022-09-03T06:40:47.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zxOThnXJW5QPlDYX2oyN1Xa81ZI=', 'Content-Length': '0'} 2022-09-03T06:40:47.154 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000044b352011ecd845a-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.155 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:40:47.156 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:47.156 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:47.157 DEBUG:boto:Method: GET 2022-09-03T06:40:47.158 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:40:47.158 DEBUG:boto:Data: 2022-09-03T06:40:47.159 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.159 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.160 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.160 DEBUG:boto:Params: {} 2022-09-03T06:40:47.161 DEBUG:boto:Token: None 2022-09-03T06:40:47.162 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj11 2022-09-03T06:40:47.162 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XuJzyLD2pM0xqBZ7X0XYwZ80FF0= 2022-09-03T06:40:47.163 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XuJzyLD2pM0xqBZ7X0XYwZ80FF0=', 'Content-Length': '0'} 2022-09-03T06:40:47.165 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000063e1085a065426b5-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.166 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:47.166 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:47.167 DEBUG:boto:Method: GET 2022-09-03T06:40:47.168 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:40:47.168 DEBUG:boto:Data: 2022-09-03T06:40:47.169 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.169 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.170 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.170 DEBUG:boto:Params: {} 2022-09-03T06:40:47.171 DEBUG:boto:Token: None 2022-09-03T06:40:47.171 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj11 2022-09-03T06:40:47.172 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XuJzyLD2pM0xqBZ7X0XYwZ80FF0= 2022-09-03T06:40:47.173 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XuJzyLD2pM0xqBZ7X0XYwZ80FF0=', 'Content-Length': '0'} 2022-09-03T06:40:47.175 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000391ebe3abe3bebf1-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.176 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:47.177 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:47.177 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:47.178 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:47.179 DEBUG:boto:Method: GET 2022-09-03T06:40:47.179 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:40:47.180 DEBUG:boto:Data: 2022-09-03T06:40:47.180 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.181 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.181 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.182 DEBUG:boto:Params: {} 2022-09-03T06:40:47.183 DEBUG:boto:Token: None 2022-09-03T06:40:47.184 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/ 2022-09-03T06:40:47.184 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c= 2022-09-03T06:40:47.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c=', 'Content-Length': '0'} 2022-09-03T06:40:47.188 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002898f5a881ece925-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.189 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:47.190 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:47.190 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:47.191 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:47.191 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:47.192 DEBUG:boto:Method: GET 2022-09-03T06:40:47.192 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:40:47.193 DEBUG:boto:Data: 2022-09-03T06:40:47.193 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.194 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.194 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.195 DEBUG:boto:Params: {} 2022-09-03T06:40:47.196 DEBUG:boto:Token: None 2022-09-03T06:40:47.196 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/ 2022-09-03T06:40:47.197 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c= 2022-09-03T06:40:47.197 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c=', 'Content-Length': '0'} 2022-09-03T06:40:47.201 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000018af02dbb10dc290-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.202 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:47.203 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:40:47.204 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:47.205 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:47.206 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.206 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.207 DEBUG:boto:Method: GET 2022-09-03T06:40:47.208 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.208 DEBUG:boto:Data: 2022-09-03T06:40:47.209 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.209 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.210 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.211 DEBUG:boto:Params: {} 2022-09-03T06:40:47.212 DEBUG:boto:Token: None 2022-09-03T06:40:47.214 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.215 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G9oeDMTf/LkWBcsNBp1PaPJg7gM= 2022-09-03T06:40:47.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G9oeDMTf/LkWBcsNBp1PaPJg7gM=', 'Content-Length': '0'} 2022-09-03T06:40:47.219 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000031965d884754991e-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.220 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:47.221 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:47.222 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.223 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.224 DEBUG:boto:Method: GET 2022-09-03T06:40:47.225 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.226 DEBUG:boto:Data: 2022-09-03T06:40:47.227 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.228 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.229 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.229 DEBUG:boto:Params: {} 2022-09-03T06:40:47.230 DEBUG:boto:Token: None 2022-09-03T06:40:47.232 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.232 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G9oeDMTf/LkWBcsNBp1PaPJg7gM= 2022-09-03T06:40:47.233 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G9oeDMTf/LkWBcsNBp1PaPJg7gM=', 'Content-Length': '0'} 2022-09-03T06:40:47.235 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000041d67ad4c8db890d-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.236 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:47.237 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:47.237 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.238 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.238 DEBUG:boto:Method: HEAD 2022-09-03T06:40:47.239 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.239 DEBUG:boto:Data: 2022-09-03T06:40:47.240 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.240 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.241 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.242 DEBUG:boto:Params: {} 2022-09-03T06:40:47.242 DEBUG:boto:Token: None 2022-09-03T06:40:47.243 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.243 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uuHdwbdeAWqt15QpPMjEZufUurQ= 2022-09-03T06:40:47.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uuHdwbdeAWqt15QpPMjEZufUurQ=', 'Content-Length': '0'} 2022-09-03T06:40:47.246 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c400c01e0a8a3c35-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.247 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:47.248 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:47.248 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.249 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.249 DEBUG:boto:Method: HEAD 2022-09-03T06:40:47.250 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.250 DEBUG:boto:Data: 2022-09-03T06:40:47.251 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.251 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.252 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.252 DEBUG:boto:Params: {} 2022-09-03T06:40:47.253 DEBUG:boto:Token: None 2022-09-03T06:40:47.253 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.254 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uuHdwbdeAWqt15QpPMjEZufUurQ= 2022-09-03T06:40:47.255 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uuHdwbdeAWqt15QpPMjEZufUurQ=', 'Content-Length': '0'} 2022-09-03T06:40:47.258 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008feb714f950ccea5-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.260 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:40:47.261 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:47.262 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:47.263 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.264 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.265 DEBUG:boto:Method: GET 2022-09-03T06:40:47.266 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.268 DEBUG:boto:Data: 2022-09-03T06:40:47.269 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.270 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.271 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.272 DEBUG:boto:Params: {} 2022-09-03T06:40:47.273 DEBUG:boto:Token: None 2022-09-03T06:40:47.274 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.275 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G9oeDMTf/LkWBcsNBp1PaPJg7gM= 2022-09-03T06:40:47.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G9oeDMTf/LkWBcsNBp1PaPJg7gM=', 'Content-Length': '0'} 2022-09-03T06:40:47.278 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b532a974cd5ea1df-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.322 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:47.323 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:47.324 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.325 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.326 DEBUG:boto:Method: GET 2022-09-03T06:40:47.327 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.328 DEBUG:boto:Data: 2022-09-03T06:40:47.328 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.329 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.329 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.330 DEBUG:boto:Params: {} 2022-09-03T06:40:47.331 DEBUG:boto:Token: None 2022-09-03T06:40:47.331 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:47.332 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G9oeDMTf/LkWBcsNBp1PaPJg7gM= 2022-09-03T06:40:47.333 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G9oeDMTf/LkWBcsNBp1PaPJg7gM=', 'Content-Length': '0'} 2022-09-03T06:40:47.335 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fe99d6925148d523-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.336 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:47.337 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:47.338 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:47.338 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:47.339 DEBUG:boto:Method: GET 2022-09-03T06:40:47.340 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:40:47.340 DEBUG:boto:Data: 2022-09-03T06:40:47.341 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.342 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.343 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.346 DEBUG:boto:Params: {} 2022-09-03T06:40:47.348 DEBUG:boto:Token: None 2022-09-03T06:40:47.349 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/ 2022-09-03T06:40:47.350 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c= 2022-09-03T06:40:47.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c=', 'Content-Length': '0'} 2022-09-03T06:40:47.355 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000304a2aa96945c653-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.357 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:47.358 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:47.373 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:47.375 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:47.376 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:47.377 DEBUG:boto:Method: GET 2022-09-03T06:40:47.378 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:40:47.379 DEBUG:boto:Data: 2022-09-03T06:40:47.380 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.381 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.382 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.384 DEBUG:boto:Params: {} 2022-09-03T06:40:47.385 DEBUG:boto:Token: None 2022-09-03T06:40:47.386 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/ 2022-09-03T06:40:47.387 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c= 2022-09-03T06:40:47.388 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c=', 'Content-Length': '0'} 2022-09-03T06:40:47.392 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008ec26764bf1cae86-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.394 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:47.395 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:47.396 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:47.397 DEBUG:boto:Method: HEAD 2022-09-03T06:40:47.398 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:40:47.399 DEBUG:boto:Data: 2022-09-03T06:40:47.400 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.401 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.402 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.403 DEBUG:boto:Params: {} 2022-09-03T06:40:47.404 DEBUG:boto:Token: None 2022-09-03T06:40:47.405 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj17 2022-09-03T06:40:47.406 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c6y1m47M2CSE0wvz2z++KbLNoLc= 2022-09-03T06:40:47.407 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c6y1m47M2CSE0wvz2z++KbLNoLc=', 'Content-Length': '0'} 2022-09-03T06:40:47.421 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000068a2383141a704bb-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.423 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:47.424 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:47.425 DEBUG:boto:Method: HEAD 2022-09-03T06:40:47.426 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:40:47.426 DEBUG:boto:Data: 2022-09-03T06:40:47.429 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.430 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.431 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.431 DEBUG:boto:Params: {} 2022-09-03T06:40:47.432 DEBUG:boto:Token: None 2022-09-03T06:40:47.433 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj17 2022-09-03T06:40:47.434 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c6y1m47M2CSE0wvz2z++KbLNoLc= 2022-09-03T06:40:47.435 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c6y1m47M2CSE0wvz2z++KbLNoLc=', 'Content-Length': '0'} 2022-09-03T06:40:47.439 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000417f7018a5343a5c-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.446 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:40:47.447 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:47.450 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:47.451 DEBUG:boto:Method: GET 2022-09-03T06:40:47.451 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:40:47.452 DEBUG:boto:Data: 2022-09-03T06:40:47.452 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.454 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.455 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.456 DEBUG:boto:Params: {} 2022-09-03T06:40:47.457 DEBUG:boto:Token: None 2022-09-03T06:40:47.458 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj17 2022-09-03T06:40:47.459 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fexm0Ejm27cqOQFfqzI6l9XlWL0= 2022-09-03T06:40:47.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fexm0Ejm27cqOQFfqzI6l9XlWL0=', 'Content-Length': '0'} 2022-09-03T06:40:47.468 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001eccde3abcf0cf4c-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.469 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:47.470 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:47.470 DEBUG:boto:Method: GET 2022-09-03T06:40:47.471 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:40:47.471 DEBUG:boto:Data: 2022-09-03T06:40:47.472 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.473 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.473 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.474 DEBUG:boto:Params: {} 2022-09-03T06:40:47.475 DEBUG:boto:Token: None 2022-09-03T06:40:47.476 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj17 2022-09-03T06:40:47.477 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fexm0Ejm27cqOQFfqzI6l9XlWL0= 2022-09-03T06:40:47.477 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fexm0Ejm27cqOQFfqzI6l9XlWL0=', 'Content-Length': '0'} 2022-09-03T06:40:47.481 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026e8850cbb4c8651-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.482 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:47.482 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:47.483 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:47.483 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:47.484 DEBUG:boto:Method: GET 2022-09-03T06:40:47.485 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:40:47.485 DEBUG:boto:Data: 2022-09-03T06:40:47.487 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.488 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.490 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.490 DEBUG:boto:Params: {} 2022-09-03T06:40:47.491 DEBUG:boto:Token: None 2022-09-03T06:40:47.492 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/ 2022-09-03T06:40:47.492 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c= 2022-09-03T06:40:47.493 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c=', 'Content-Length': '0'} 2022-09-03T06:40:47.496 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e395a4025c322a27-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.496 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:47.497 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:47.498 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:47.499 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:47.499 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:47.500 DEBUG:boto:Method: GET 2022-09-03T06:40:47.501 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:40:47.502 DEBUG:boto:Data: 2022-09-03T06:40:47.503 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.504 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.505 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.506 DEBUG:boto:Params: {} 2022-09-03T06:40:47.508 DEBUG:boto:Token: None 2022-09-03T06:40:47.509 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/ 2022-09-03T06:40:47.510 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c= 2022-09-03T06:40:47.512 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c=', 'Content-Length': '0'} 2022-09-03T06:40:47.516 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e97663e6915b95bc-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.517 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:47.519 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:40:47.521 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:47.522 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:47.522 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.523 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.524 DEBUG:boto:Method: GET 2022-09-03T06:40:47.524 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.525 DEBUG:boto:Data: 2022-09-03T06:40:47.526 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.527 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.528 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.529 DEBUG:boto:Params: {} 2022-09-03T06:40:47.529 DEBUG:boto:Token: None 2022-09-03T06:40:47.530 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.531 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bD08N6fwDV+qSVnFSnjkNiH9R4M= 2022-09-03T06:40:47.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bD08N6fwDV+qSVnFSnjkNiH9R4M=', 'Content-Length': '0'} 2022-09-03T06:40:47.534 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004317b658072b08da-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.535 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:47.536 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:47.537 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.538 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.539 DEBUG:boto:Method: GET 2022-09-03T06:40:47.540 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.541 DEBUG:boto:Data: 2022-09-03T06:40:47.542 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.542 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.543 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.544 DEBUG:boto:Params: {} 2022-09-03T06:40:47.544 DEBUG:boto:Token: None 2022-09-03T06:40:47.545 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.546 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bD08N6fwDV+qSVnFSnjkNiH9R4M= 2022-09-03T06:40:47.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bD08N6fwDV+qSVnFSnjkNiH9R4M=', 'Content-Length': '0'} 2022-09-03T06:40:47.549 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af331c2dcc347032-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.549 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:47.550 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:47.550 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.551 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.551 DEBUG:boto:Method: HEAD 2022-09-03T06:40:47.552 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.553 DEBUG:boto:Data: 2022-09-03T06:40:47.553 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.554 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.554 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.555 DEBUG:boto:Params: {} 2022-09-03T06:40:47.555 DEBUG:boto:Token: None 2022-09-03T06:40:47.556 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.556 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hgz7BQAA3w8Mjt6e7PCevDqu8VI= 2022-09-03T06:40:47.557 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hgz7BQAA3w8Mjt6e7PCevDqu8VI=', 'Content-Length': '0'} 2022-09-03T06:40:47.560 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003654c07c93001483-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.561 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:47.561 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:47.562 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.562 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.563 DEBUG:boto:Method: HEAD 2022-09-03T06:40:47.564 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.564 DEBUG:boto:Data: 2022-09-03T06:40:47.565 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.565 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.566 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.567 DEBUG:boto:Params: {} 2022-09-03T06:40:47.567 DEBUG:boto:Token: None 2022-09-03T06:40:47.568 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.568 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hgz7BQAA3w8Mjt6e7PCevDqu8VI= 2022-09-03T06:40:47.569 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hgz7BQAA3w8Mjt6e7PCevDqu8VI=', 'Content-Length': '0'} 2022-09-03T06:40:47.572 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000050b78f3d435d7b5-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.573 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:40:47.574 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:47.575 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:47.575 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.576 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.577 DEBUG:boto:Method: GET 2022-09-03T06:40:47.577 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.578 DEBUG:boto:Data: 2022-09-03T06:40:47.578 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.579 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.579 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.580 DEBUG:boto:Params: {} 2022-09-03T06:40:47.580 DEBUG:boto:Token: None 2022-09-03T06:40:47.581 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.582 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bD08N6fwDV+qSVnFSnjkNiH9R4M= 2022-09-03T06:40:47.582 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bD08N6fwDV+qSVnFSnjkNiH9R4M=', 'Content-Length': '0'} 2022-09-03T06:40:47.585 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a416f0232b86089e-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.626 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:47.628 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:47.628 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.629 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.630 DEBUG:boto:Method: GET 2022-09-03T06:40:47.630 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.631 DEBUG:boto:Data: 2022-09-03T06:40:47.631 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.632 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.633 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.634 DEBUG:boto:Params: {} 2022-09-03T06:40:47.635 DEBUG:boto:Token: None 2022-09-03T06:40:47.636 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:47.637 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bD08N6fwDV+qSVnFSnjkNiH9R4M= 2022-09-03T06:40:47.638 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bD08N6fwDV+qSVnFSnjkNiH9R4M=', 'Content-Length': '0'} 2022-09-03T06:40:47.640 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ac5bcc164bfafe57-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.641 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:47.642 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:47.642 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:47.643 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:47.644 DEBUG:boto:Method: GET 2022-09-03T06:40:47.645 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:40:47.645 DEBUG:boto:Data: 2022-09-03T06:40:47.646 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.646 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.647 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.647 DEBUG:boto:Params: {} 2022-09-03T06:40:47.648 DEBUG:boto:Token: None 2022-09-03T06:40:47.648 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/ 2022-09-03T06:40:47.649 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c= 2022-09-03T06:40:47.650 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c=', 'Content-Length': '0'} 2022-09-03T06:40:47.653 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000078cc5519fa9eabaa-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.653 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:47.654 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:47.655 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:47.655 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:47.656 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:47.656 DEBUG:boto:Method: GET 2022-09-03T06:40:47.657 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:40:47.658 DEBUG:boto:Data: 2022-09-03T06:40:47.658 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.659 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.659 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.661 DEBUG:boto:Params: {} 2022-09-03T06:40:47.661 DEBUG:boto:Token: None 2022-09-03T06:40:47.662 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/ 2022-09-03T06:40:47.662 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c= 2022-09-03T06:40:47.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c=', 'Content-Length': '0'} 2022-09-03T06:40:47.667 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b66c8910c5861a39-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.668 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:47.668 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:47.669 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:47.669 DEBUG:boto:Method: HEAD 2022-09-03T06:40:47.670 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:40:47.671 DEBUG:boto:Data: 2022-09-03T06:40:47.671 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.672 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.673 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.673 DEBUG:boto:Params: {} 2022-09-03T06:40:47.674 DEBUG:boto:Token: None 2022-09-03T06:40:47.674 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj22 2022-09-03T06:40:47.675 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4xK/ug0j96S60764prhAR+PpoN4= 2022-09-03T06:40:47.675 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4xK/ug0j96S60764prhAR+PpoN4=', 'Content-Length': '0'} 2022-09-03T06:40:47.678 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000254519c8222b7556-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.679 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:47.679 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:47.680 DEBUG:boto:Method: HEAD 2022-09-03T06:40:47.680 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:40:47.681 DEBUG:boto:Data: 2022-09-03T06:40:47.681 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.682 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.683 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.683 DEBUG:boto:Params: {} 2022-09-03T06:40:47.684 DEBUG:boto:Token: None 2022-09-03T06:40:47.684 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj22 2022-09-03T06:40:47.685 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4xK/ug0j96S60764prhAR+PpoN4= 2022-09-03T06:40:47.686 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4xK/ug0j96S60764prhAR+PpoN4=', 'Content-Length': '0'} 2022-09-03T06:40:47.688 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d24199ed58be6f4f-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.689 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:40:47.689 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:47.690 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:47.690 DEBUG:boto:Method: GET 2022-09-03T06:40:47.691 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:40:47.692 DEBUG:boto:Data: 2022-09-03T06:40:47.692 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.693 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.693 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.694 DEBUG:boto:Params: {} 2022-09-03T06:40:47.695 DEBUG:boto:Token: None 2022-09-03T06:40:47.695 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj22 2022-09-03T06:40:47.696 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EzSQUo+Kqy5WcDJ7EigcLFj9X90= 2022-09-03T06:40:47.697 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EzSQUo+Kqy5WcDJ7EigcLFj9X90=', 'Content-Length': '0'} 2022-09-03T06:40:47.699 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c85d56d12e6b696c-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.705 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:47.707 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:47.707 DEBUG:boto:Method: GET 2022-09-03T06:40:47.708 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:40:47.708 DEBUG:boto:Data: 2022-09-03T06:40:47.709 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.709 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.710 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.711 DEBUG:boto:Params: {} 2022-09-03T06:40:47.711 DEBUG:boto:Token: None 2022-09-03T06:40:47.713 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj22 2022-09-03T06:40:47.714 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EzSQUo+Kqy5WcDJ7EigcLFj9X90= 2022-09-03T06:40:47.715 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EzSQUo+Kqy5WcDJ7EigcLFj9X90=', 'Content-Length': '0'} 2022-09-03T06:40:47.718 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000083469fb516261b84-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.719 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:47.719 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:47.720 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:47.720 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:47.721 DEBUG:boto:Method: GET 2022-09-03T06:40:47.721 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:40:47.722 DEBUG:boto:Data: 2022-09-03T06:40:47.723 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.723 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.724 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.724 DEBUG:boto:Params: {} 2022-09-03T06:40:47.725 DEBUG:boto:Token: None 2022-09-03T06:40:47.726 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/ 2022-09-03T06:40:47.726 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c= 2022-09-03T06:40:47.727 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c=', 'Content-Length': '0'} 2022-09-03T06:40:47.730 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e5d81723845198a8-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.732 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:47.733 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:47.733 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:47.735 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:47.735 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:47.736 DEBUG:boto:Method: GET 2022-09-03T06:40:47.737 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:40:47.737 DEBUG:boto:Data: 2022-09-03T06:40:47.738 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.739 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.739 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.740 DEBUG:boto:Params: {} 2022-09-03T06:40:47.741 DEBUG:boto:Token: None 2022-09-03T06:40:47.741 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/ 2022-09-03T06:40:47.742 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c= 2022-09-03T06:40:47.743 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c=', 'Content-Length': '0'} 2022-09-03T06:40:47.747 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d4354fdb1f499076-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.749 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:47.750 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:40:47.751 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:47.752 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:47.753 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.753 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.754 DEBUG:boto:Method: GET 2022-09-03T06:40:47.755 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.756 DEBUG:boto:Data: 2022-09-03T06:40:47.757 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.758 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.758 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.759 DEBUG:boto:Params: {} 2022-09-03T06:40:47.760 DEBUG:boto:Token: None 2022-09-03T06:40:47.761 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.762 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bssXxF0Hwpv1hOVvKYR+xFfhkqo= 2022-09-03T06:40:47.762 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bssXxF0Hwpv1hOVvKYR+xFfhkqo=', 'Content-Length': '0'} 2022-09-03T06:40:47.765 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009c1ac41d521a34b1-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.806 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:47.807 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:47.807 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.808 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.809 DEBUG:boto:Method: GET 2022-09-03T06:40:47.810 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.812 DEBUG:boto:Data: 2022-09-03T06:40:47.812 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.813 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.814 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.815 DEBUG:boto:Params: {} 2022-09-03T06:40:47.816 DEBUG:boto:Token: None 2022-09-03T06:40:47.817 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.821 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bssXxF0Hwpv1hOVvKYR+xFfhkqo= 2022-09-03T06:40:47.821 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bssXxF0Hwpv1hOVvKYR+xFfhkqo=', 'Content-Length': '0'} 2022-09-03T06:40:47.825 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001bb0d2957cdfbdc4-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.826 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:47.826 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:47.827 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.827 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.828 DEBUG:boto:Method: HEAD 2022-09-03T06:40:47.829 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.830 DEBUG:boto:Data: 2022-09-03T06:40:47.830 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.832 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.834 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.834 DEBUG:boto:Params: {} 2022-09-03T06:40:47.835 DEBUG:boto:Token: None 2022-09-03T06:40:47.836 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.837 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4jf2wLswgPcAz6JnvcZsDzalVSM= 2022-09-03T06:40:47.838 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4jf2wLswgPcAz6JnvcZsDzalVSM=', 'Content-Length': '0'} 2022-09-03T06:40:47.857 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000085e5983cb1474bdf-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.858 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:47.859 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:47.859 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.860 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.860 DEBUG:boto:Method: HEAD 2022-09-03T06:40:47.861 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.861 DEBUG:boto:Data: 2022-09-03T06:40:47.862 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.862 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.863 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.863 DEBUG:boto:Params: {} 2022-09-03T06:40:47.864 DEBUG:boto:Token: None 2022-09-03T06:40:47.864 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.865 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4jf2wLswgPcAz6JnvcZsDzalVSM= 2022-09-03T06:40:47.866 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4jf2wLswgPcAz6JnvcZsDzalVSM=', 'Content-Length': '0'} 2022-09-03T06:40:47.868 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004999041a1355cf9a-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.869 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:40:47.870 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:47.871 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:47.871 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.872 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.872 DEBUG:boto:Method: GET 2022-09-03T06:40:47.873 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.874 DEBUG:boto:Data: 2022-09-03T06:40:47.875 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.875 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.876 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.876 DEBUG:boto:Params: {} 2022-09-03T06:40:47.877 DEBUG:boto:Token: None 2022-09-03T06:40:47.878 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.879 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bssXxF0Hwpv1hOVvKYR+xFfhkqo= 2022-09-03T06:40:47.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bssXxF0Hwpv1hOVvKYR+xFfhkqo=', 'Content-Length': '0'} 2022-09-03T06:40:47.882 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e86cf1dfef474b4b-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.884 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:47.885 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:47.885 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.887 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.887 DEBUG:boto:Method: GET 2022-09-03T06:40:47.888 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.888 DEBUG:boto:Data: 2022-09-03T06:40:47.889 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.890 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.892 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.892 DEBUG:boto:Params: {} 2022-09-03T06:40:47.894 DEBUG:boto:Token: None 2022-09-03T06:40:47.895 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:47.897 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bssXxF0Hwpv1hOVvKYR+xFfhkqo= 2022-09-03T06:40:47.898 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bssXxF0Hwpv1hOVvKYR+xFfhkqo=', 'Content-Length': '0'} 2022-09-03T06:40:47.901 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d50ee60eb60b985e-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.902 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:47.902 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:47.903 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:47.904 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:47.904 DEBUG:boto:Method: GET 2022-09-03T06:40:47.905 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:40:47.905 DEBUG:boto:Data: 2022-09-03T06:40:47.906 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.907 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.908 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.909 DEBUG:boto:Params: {} 2022-09-03T06:40:47.910 DEBUG:boto:Token: None 2022-09-03T06:40:47.911 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/ 2022-09-03T06:40:47.911 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c= 2022-09-03T06:40:47.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c=', 'Content-Length': '0'} 2022-09-03T06:40:47.915 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000de16a1f1476cd89a-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.915 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:47.917 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:47.917 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:47.918 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:47.919 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:47.920 DEBUG:boto:Method: GET 2022-09-03T06:40:47.921 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:40:47.922 DEBUG:boto:Data: 2022-09-03T06:40:47.923 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.923 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.925 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.926 DEBUG:boto:Params: {} 2022-09-03T06:40:47.927 DEBUG:boto:Token: None 2022-09-03T06:40:47.928 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/ 2022-09-03T06:40:47.929 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c= 2022-09-03T06:40:47.930 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y41mqJ9eyNtDSNegFXSSyX64Y7c=', 'Content-Length': '0'} 2022-09-03T06:40:47.934 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002e8bd34e018cb196-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.935 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:47.936 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:47.937 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:47.938 DEBUG:boto:Method: HEAD 2022-09-03T06:40:47.939 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:40:47.940 DEBUG:boto:Data: 2022-09-03T06:40:47.940 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.941 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.941 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.942 DEBUG:boto:Params: {} 2022-09-03T06:40:47.942 DEBUG:boto:Token: None 2022-09-03T06:40:47.943 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj33 2022-09-03T06:40:47.944 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+Ep2Q7PTBrRAWV0ZCS1bcfAt87g= 2022-09-03T06:40:47.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+Ep2Q7PTBrRAWV0ZCS1bcfAt87g=', 'Content-Length': '0'} 2022-09-03T06:40:47.948 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000035df2960c4af5398-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.949 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:47.949 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:47.950 DEBUG:boto:Method: HEAD 2022-09-03T06:40:47.951 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:40:47.952 DEBUG:boto:Data: 2022-09-03T06:40:47.953 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.953 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:47.955 DEBUG:boto:Port: 8000 2022-09-03T06:40:47.956 DEBUG:boto:Params: {} 2022-09-03T06:40:47.957 DEBUG:boto:Token: None 2022-09-03T06:40:47.958 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj33 2022-09-03T06:40:47.959 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+Ep2Q7PTBrRAWV0ZCS1bcfAt87g= 2022-09-03T06:40:47.959 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+Ep2Q7PTBrRAWV0ZCS1bcfAt87g=', 'Content-Length': '0'} 2022-09-03T06:40:47.961 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000090d7baf0c307bcc7-006312f6ef-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:47.962 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:40:47.963 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:47.965 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:47.965 DEBUG:boto:Method: GET 2022-09-03T06:40:47.966 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:40:47.967 DEBUG:boto:Data: 2022-09-03T06:40:47.968 DEBUG:boto:Headers: {} 2022-09-03T06:40:47.968 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:47.969 DEBUG:boto:Port: 8001 2022-09-03T06:40:47.970 DEBUG:boto:Params: {} 2022-09-03T06:40:47.971 DEBUG:boto:Token: None 2022-09-03T06:40:47.971 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:47 GMT /dsgvcy-75/obj33 2022-09-03T06:40:47.972 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZuEDw81f5JT8XDowQd5D8rkxzLo= 2022-09-03T06:40:47.973 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZuEDw81f5JT8XDowQd5D8rkxzLo=', 'Content-Length': '0'} 2022-09-03T06:40:47.975 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e127f3b6874fcaa-006312f6ef-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.022 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:48.023 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:48.024 DEBUG:boto:Method: GET 2022-09-03T06:40:48.024 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:40:48.025 DEBUG:boto:Data: 2022-09-03T06:40:48.025 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.026 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.026 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.027 DEBUG:boto:Params: {} 2022-09-03T06:40:48.028 DEBUG:boto:Token: None 2022-09-03T06:40:48.028 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj33 2022-09-03T06:40:48.029 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mHLLKTPfZ7l3/CVFHIqQeRhm8Hk= 2022-09-03T06:40:48.030 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mHLLKTPfZ7l3/CVFHIqQeRhm8Hk=', 'Content-Length': '0'} 2022-09-03T06:40:48.032 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ea62a4433a9586eb-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.033 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:48.033 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:48.034 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:48.034 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:48.035 DEBUG:boto:Method: GET 2022-09-03T06:40:48.035 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:40:48.036 DEBUG:boto:Data: 2022-09-03T06:40:48.036 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.037 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.037 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.038 DEBUG:boto:Params: {} 2022-09-03T06:40:48.038 DEBUG:boto:Token: None 2022-09-03T06:40:48.039 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/ 2022-09-03T06:40:48.040 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM= 2022-09-03T06:40:48.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM=', 'Content-Length': '0'} 2022-09-03T06:40:48.043 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000099bf2a5143ede33-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.044 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:48.045 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:48.045 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:48.046 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:48.047 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:48.047 DEBUG:boto:Method: GET 2022-09-03T06:40:48.048 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:40:48.048 DEBUG:boto:Data: 2022-09-03T06:40:48.049 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.049 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.050 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.050 DEBUG:boto:Params: {} 2022-09-03T06:40:48.051 DEBUG:boto:Token: None 2022-09-03T06:40:48.052 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/ 2022-09-03T06:40:48.052 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM= 2022-09-03T06:40:48.053 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM=', 'Content-Length': '0'} 2022-09-03T06:40:48.056 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009bce6e5c2c27a799-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.057 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:48.058 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:40:48.059 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:48.059 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:48.060 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.060 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.061 DEBUG:boto:Method: GET 2022-09-03T06:40:48.061 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.062 DEBUG:boto:Data: 2022-09-03T06:40:48.063 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.063 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.064 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.064 DEBUG:boto:Params: {} 2022-09-03T06:40:48.065 DEBUG:boto:Token: None 2022-09-03T06:40:48.065 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.066 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7y8alVzYD8Yw4dIcJkQMJb7RQR8= 2022-09-03T06:40:48.067 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7y8alVzYD8Yw4dIcJkQMJb7RQR8=', 'Content-Length': '0'} 2022-09-03T06:40:48.069 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7795b6142da6a22-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.070 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:48.071 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:48.072 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.073 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.074 DEBUG:boto:Method: GET 2022-09-03T06:40:48.075 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.075 DEBUG:boto:Data: 2022-09-03T06:40:48.076 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.077 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.078 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.079 DEBUG:boto:Params: {} 2022-09-03T06:40:48.080 DEBUG:boto:Token: None 2022-09-03T06:40:48.081 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.082 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7y8alVzYD8Yw4dIcJkQMJb7RQR8= 2022-09-03T06:40:48.083 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7y8alVzYD8Yw4dIcJkQMJb7RQR8=', 'Content-Length': '0'} 2022-09-03T06:40:48.085 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002b80368caefa5a07-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.126 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:48.127 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:48.128 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.128 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.129 DEBUG:boto:Method: HEAD 2022-09-03T06:40:48.130 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.130 DEBUG:boto:Data: 2022-09-03T06:40:48.131 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.132 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.132 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.133 DEBUG:boto:Params: {} 2022-09-03T06:40:48.134 DEBUG:boto:Token: None 2022-09-03T06:40:48.134 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.135 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0KSkj1iIAtgB/0giXJFnY0BvDP0= 2022-09-03T06:40:48.136 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0KSkj1iIAtgB/0giXJFnY0BvDP0=', 'Content-Length': '0'} 2022-09-03T06:40:48.138 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000004c97a75fbe80eab-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.140 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:48.141 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:48.142 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.143 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.144 DEBUG:boto:Method: HEAD 2022-09-03T06:40:48.145 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.146 DEBUG:boto:Data: 2022-09-03T06:40:48.147 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.148 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.149 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.151 DEBUG:boto:Params: {} 2022-09-03T06:40:48.152 DEBUG:boto:Token: None 2022-09-03T06:40:48.153 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.154 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0KSkj1iIAtgB/0giXJFnY0BvDP0= 2022-09-03T06:40:48.155 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0KSkj1iIAtgB/0giXJFnY0BvDP0=', 'Content-Length': '0'} 2022-09-03T06:40:48.158 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005ac059bfa988952f-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.159 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:40:48.160 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:48.161 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:48.162 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.162 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.164 DEBUG:boto:Method: GET 2022-09-03T06:40:48.164 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.165 DEBUG:boto:Data: 2022-09-03T06:40:48.166 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.166 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.167 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.167 DEBUG:boto:Params: {} 2022-09-03T06:40:48.168 DEBUG:boto:Token: None 2022-09-03T06:40:48.169 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.169 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7y8alVzYD8Yw4dIcJkQMJb7RQR8= 2022-09-03T06:40:48.171 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7y8alVzYD8Yw4dIcJkQMJb7RQR8=', 'Content-Length': '0'} 2022-09-03T06:40:48.178 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002af711e7599552c9-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.218 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:48.222 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:48.223 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.225 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.226 DEBUG:boto:Method: GET 2022-09-03T06:40:48.227 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.227 DEBUG:boto:Data: 2022-09-03T06:40:48.228 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.229 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.230 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.230 DEBUG:boto:Params: {} 2022-09-03T06:40:48.231 DEBUG:boto:Token: None 2022-09-03T06:40:48.232 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:48.232 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7y8alVzYD8Yw4dIcJkQMJb7RQR8= 2022-09-03T06:40:48.234 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7y8alVzYD8Yw4dIcJkQMJb7RQR8=', 'Content-Length': '0'} 2022-09-03T06:40:48.236 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005cdb075fec17a797-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.238 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:48.239 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:48.239 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:48.240 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:48.240 DEBUG:boto:Method: GET 2022-09-03T06:40:48.241 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:40:48.241 DEBUG:boto:Data: 2022-09-03T06:40:48.242 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.242 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.243 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.243 DEBUG:boto:Params: {} 2022-09-03T06:40:48.244 DEBUG:boto:Token: None 2022-09-03T06:40:48.245 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/ 2022-09-03T06:40:48.245 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM= 2022-09-03T06:40:48.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM=', 'Content-Length': '0'} 2022-09-03T06:40:48.249 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000070249803249e09-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.250 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:48.251 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:48.252 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:48.253 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:48.253 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:48.254 DEBUG:boto:Method: GET 2022-09-03T06:40:48.255 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:40:48.256 DEBUG:boto:Data: 2022-09-03T06:40:48.257 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.257 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.258 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.258 DEBUG:boto:Params: {} 2022-09-03T06:40:48.259 DEBUG:boto:Token: None 2022-09-03T06:40:48.260 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/ 2022-09-03T06:40:48.260 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM= 2022-09-03T06:40:48.261 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM=', 'Content-Length': '0'} 2022-09-03T06:40:48.264 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008296788c3f30cdf5-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.264 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:48.265 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:48.266 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:48.266 DEBUG:boto:Method: HEAD 2022-09-03T06:40:48.267 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:40:48.268 DEBUG:boto:Data: 2022-09-03T06:40:48.268 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.269 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.269 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.270 DEBUG:boto:Params: {} 2022-09-03T06:40:48.270 DEBUG:boto:Token: None 2022-09-03T06:40:48.271 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj34 2022-09-03T06:40:48.271 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+5aTRCENCQ630e7LB8QGI9AWR8Y= 2022-09-03T06:40:48.272 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+5aTRCENCQ630e7LB8QGI9AWR8Y=', 'Content-Length': '0'} 2022-09-03T06:40:48.274 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f43eda2cc3f8906-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.275 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:48.276 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:48.276 DEBUG:boto:Method: HEAD 2022-09-03T06:40:48.277 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:40:48.277 DEBUG:boto:Data: 2022-09-03T06:40:48.278 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.278 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.279 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.279 DEBUG:boto:Params: {} 2022-09-03T06:40:48.280 DEBUG:boto:Token: None 2022-09-03T06:40:48.281 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj34 2022-09-03T06:40:48.281 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+5aTRCENCQ630e7LB8QGI9AWR8Y= 2022-09-03T06:40:48.282 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+5aTRCENCQ630e7LB8QGI9AWR8Y=', 'Content-Length': '0'} 2022-09-03T06:40:48.284 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad6d9a7837bccdc4-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.285 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:40:48.285 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:48.286 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:48.287 DEBUG:boto:Method: GET 2022-09-03T06:40:48.287 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:40:48.288 DEBUG:boto:Data: 2022-09-03T06:40:48.288 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.289 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.289 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.290 DEBUG:boto:Params: {} 2022-09-03T06:40:48.291 DEBUG:boto:Token: None 2022-09-03T06:40:48.291 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj34 2022-09-03T06:40:48.292 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nS4NIs61xDSSIywlxekq02cyqMw= 2022-09-03T06:40:48.293 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nS4NIs61xDSSIywlxekq02cyqMw=', 'Content-Length': '0'} 2022-09-03T06:40:48.295 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006f25471434278207-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.296 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:48.296 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:48.297 DEBUG:boto:Method: GET 2022-09-03T06:40:48.297 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:40:48.298 DEBUG:boto:Data: 2022-09-03T06:40:48.298 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.299 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.300 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.300 DEBUG:boto:Params: {} 2022-09-03T06:40:48.301 DEBUG:boto:Token: None 2022-09-03T06:40:48.301 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj34 2022-09-03T06:40:48.302 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nS4NIs61xDSSIywlxekq02cyqMw= 2022-09-03T06:40:48.303 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nS4NIs61xDSSIywlxekq02cyqMw=', 'Content-Length': '0'} 2022-09-03T06:40:48.305 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fc5f2a7ca6441423-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.306 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:48.306 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:48.307 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:48.307 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:48.308 DEBUG:boto:Method: GET 2022-09-03T06:40:48.309 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:40:48.309 DEBUG:boto:Data: 2022-09-03T06:40:48.310 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.310 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.311 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.311 DEBUG:boto:Params: {} 2022-09-03T06:40:48.312 DEBUG:boto:Token: None 2022-09-03T06:40:48.313 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/ 2022-09-03T06:40:48.313 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM= 2022-09-03T06:40:48.314 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM=', 'Content-Length': '0'} 2022-09-03T06:40:48.317 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007aea003a45c3c6bb-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.358 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:48.359 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:48.360 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:48.360 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:48.361 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:48.362 DEBUG:boto:Method: GET 2022-09-03T06:40:48.362 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:40:48.363 DEBUG:boto:Data: 2022-09-03T06:40:48.363 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.364 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.364 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.365 DEBUG:boto:Params: {} 2022-09-03T06:40:48.365 DEBUG:boto:Token: None 2022-09-03T06:40:48.366 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/ 2022-09-03T06:40:48.367 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM= 2022-09-03T06:40:48.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM=', 'Content-Length': '0'} 2022-09-03T06:40:48.371 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000072188b388a57e51b-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.372 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:48.374 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:40:48.375 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:48.376 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:48.377 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.377 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.378 DEBUG:boto:Method: GET 2022-09-03T06:40:48.379 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.379 DEBUG:boto:Data: 2022-09-03T06:40:48.380 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.381 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.382 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.382 DEBUG:boto:Params: {} 2022-09-03T06:40:48.383 DEBUG:boto:Token: None 2022-09-03T06:40:48.384 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.384 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yN/dii8gETtQEgfE4CnpHoAwQG4= 2022-09-03T06:40:48.385 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yN/dii8gETtQEgfE4CnpHoAwQG4=', 'Content-Length': '0'} 2022-09-03T06:40:48.388 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e4ae58da51e0f9b7-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.389 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:48.390 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:48.391 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.391 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.392 DEBUG:boto:Method: GET 2022-09-03T06:40:48.392 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.393 DEBUG:boto:Data: 2022-09-03T06:40:48.393 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.394 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.394 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.395 DEBUG:boto:Params: {} 2022-09-03T06:40:48.395 DEBUG:boto:Token: None 2022-09-03T06:40:48.396 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.397 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yN/dii8gETtQEgfE4CnpHoAwQG4= 2022-09-03T06:40:48.397 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yN/dii8gETtQEgfE4CnpHoAwQG4=', 'Content-Length': '0'} 2022-09-03T06:40:48.400 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008bf11cd5d57632f2-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.442 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:48.443 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:48.443 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.444 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.444 DEBUG:boto:Method: HEAD 2022-09-03T06:40:48.445 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.445 DEBUG:boto:Data: 2022-09-03T06:40:48.446 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.447 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.447 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.448 DEBUG:boto:Params: {} 2022-09-03T06:40:48.448 DEBUG:boto:Token: None 2022-09-03T06:40:48.449 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.449 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w3SZ+RGnwEI4izeq9LB2OmmhUKY= 2022-09-03T06:40:48.450 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w3SZ+RGnwEI4izeq9LB2OmmhUKY=', 'Content-Length': '0'} 2022-09-03T06:40:48.453 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000042b152510462896b-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.455 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:48.456 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:48.457 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.458 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.459 DEBUG:boto:Method: HEAD 2022-09-03T06:40:48.460 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.461 DEBUG:boto:Data: 2022-09-03T06:40:48.462 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.463 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.466 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.467 DEBUG:boto:Params: {} 2022-09-03T06:40:48.468 DEBUG:boto:Token: None 2022-09-03T06:40:48.469 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.470 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w3SZ+RGnwEI4izeq9LB2OmmhUKY= 2022-09-03T06:40:48.471 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w3SZ+RGnwEI4izeq9LB2OmmhUKY=', 'Content-Length': '0'} 2022-09-03T06:40:48.473 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f8ed5d5d6b705f0-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.475 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:40:48.475 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:48.476 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:48.477 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.478 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.478 DEBUG:boto:Method: GET 2022-09-03T06:40:48.479 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.480 DEBUG:boto:Data: 2022-09-03T06:40:48.481 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.481 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.483 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.484 DEBUG:boto:Params: {} 2022-09-03T06:40:48.484 DEBUG:boto:Token: None 2022-09-03T06:40:48.485 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.486 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yN/dii8gETtQEgfE4CnpHoAwQG4= 2022-09-03T06:40:48.487 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yN/dii8gETtQEgfE4CnpHoAwQG4=', 'Content-Length': '0'} 2022-09-03T06:40:48.490 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000388a33971059ebf-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.491 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:48.492 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:48.493 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.494 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.494 DEBUG:boto:Method: GET 2022-09-03T06:40:48.495 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.496 DEBUG:boto:Data: 2022-09-03T06:40:48.497 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.497 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.498 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.499 DEBUG:boto:Params: {} 2022-09-03T06:40:48.499 DEBUG:boto:Token: None 2022-09-03T06:40:48.500 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:48.501 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yN/dii8gETtQEgfE4CnpHoAwQG4= 2022-09-03T06:40:48.502 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yN/dii8gETtQEgfE4CnpHoAwQG4=', 'Content-Length': '0'} 2022-09-03T06:40:48.504 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007c72eea3c958af4d-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.505 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:48.505 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:48.506 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:48.507 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:48.507 DEBUG:boto:Method: GET 2022-09-03T06:40:48.508 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:40:48.509 DEBUG:boto:Data: 2022-09-03T06:40:48.509 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.510 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.510 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.511 DEBUG:boto:Params: {} 2022-09-03T06:40:48.511 DEBUG:boto:Token: None 2022-09-03T06:40:48.512 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/ 2022-09-03T06:40:48.513 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM= 2022-09-03T06:40:48.514 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM=', 'Content-Length': '0'} 2022-09-03T06:40:48.516 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b1f1a44e96f06de7-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.517 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:48.518 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:48.519 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:48.519 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:48.520 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:48.521 DEBUG:boto:Method: GET 2022-09-03T06:40:48.521 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:40:48.522 DEBUG:boto:Data: 2022-09-03T06:40:48.522 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.523 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.523 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.524 DEBUG:boto:Params: {} 2022-09-03T06:40:48.525 DEBUG:boto:Token: None 2022-09-03T06:40:48.525 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/ 2022-09-03T06:40:48.526 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM= 2022-09-03T06:40:48.527 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM=', 'Content-Length': '0'} 2022-09-03T06:40:48.530 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000090bf7e28d4ebd2de-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.531 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:48.533 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:48.534 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:48.536 DEBUG:boto:Method: HEAD 2022-09-03T06:40:48.536 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:40:48.538 DEBUG:boto:Data: 2022-09-03T06:40:48.539 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.540 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.541 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.542 DEBUG:boto:Params: {} 2022-09-03T06:40:48.543 DEBUG:boto:Token: None 2022-09-03T06:40:48.544 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj44 2022-09-03T06:40:48.545 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Lr1rwu7FFVDNzfDOH1RfnT8NAZk= 2022-09-03T06:40:48.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Lr1rwu7FFVDNzfDOH1RfnT8NAZk=', 'Content-Length': '0'} 2022-09-03T06:40:48.548 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000066fda0618e63f291-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.549 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:48.550 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:48.550 DEBUG:boto:Method: HEAD 2022-09-03T06:40:48.551 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:40:48.552 DEBUG:boto:Data: 2022-09-03T06:40:48.552 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.553 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.554 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.555 DEBUG:boto:Params: {} 2022-09-03T06:40:48.555 DEBUG:boto:Token: None 2022-09-03T06:40:48.556 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj44 2022-09-03T06:40:48.557 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Lr1rwu7FFVDNzfDOH1RfnT8NAZk= 2022-09-03T06:40:48.557 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Lr1rwu7FFVDNzfDOH1RfnT8NAZk=', 'Content-Length': '0'} 2022-09-03T06:40:48.559 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004146df39a2b83cac-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.560 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:40:48.561 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:48.561 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:48.562 DEBUG:boto:Method: GET 2022-09-03T06:40:48.563 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:40:48.563 DEBUG:boto:Data: 2022-09-03T06:40:48.564 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.564 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.565 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.566 DEBUG:boto:Params: {} 2022-09-03T06:40:48.566 DEBUG:boto:Token: None 2022-09-03T06:40:48.567 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj44 2022-09-03T06:40:48.568 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kSJc7j17FoU/Jm1iEx/alH51Rks= 2022-09-03T06:40:48.569 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kSJc7j17FoU/Jm1iEx/alH51Rks=', 'Content-Length': '0'} 2022-09-03T06:40:48.574 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f66f3e44292dc99-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.576 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:48.577 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:48.578 DEBUG:boto:Method: GET 2022-09-03T06:40:48.580 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:40:48.581 DEBUG:boto:Data: 2022-09-03T06:40:48.582 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.584 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.585 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.586 DEBUG:boto:Params: {} 2022-09-03T06:40:48.587 DEBUG:boto:Token: None 2022-09-03T06:40:48.588 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj44 2022-09-03T06:40:48.589 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kSJc7j17FoU/Jm1iEx/alH51Rks= 2022-09-03T06:40:48.591 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kSJc7j17FoU/Jm1iEx/alH51Rks=', 'Content-Length': '0'} 2022-09-03T06:40:48.594 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000069f6bb4ae32b399d-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.595 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:48.596 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:48.597 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:48.598 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:48.599 DEBUG:boto:Method: GET 2022-09-03T06:40:48.600 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:40:48.601 DEBUG:boto:Data: 2022-09-03T06:40:48.602 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.603 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.604 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.605 DEBUG:boto:Params: {} 2022-09-03T06:40:48.606 DEBUG:boto:Token: None 2022-09-03T06:40:48.607 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/ 2022-09-03T06:40:48.608 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM= 2022-09-03T06:40:48.610 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM=', 'Content-Length': '0'} 2022-09-03T06:40:48.613 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000000e960acafee79f2-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.614 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:48.615 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:48.616 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:48.616 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:48.617 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:48.617 DEBUG:boto:Method: GET 2022-09-03T06:40:48.618 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:40:48.619 DEBUG:boto:Data: 2022-09-03T06:40:48.619 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.620 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.621 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.622 DEBUG:boto:Params: {} 2022-09-03T06:40:48.622 DEBUG:boto:Token: None 2022-09-03T06:40:48.623 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/ 2022-09-03T06:40:48.624 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM= 2022-09-03T06:40:48.624 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM=', 'Content-Length': '0'} 2022-09-03T06:40:48.627 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e501a31e08f53db5-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.628 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:48.629 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:40:48.630 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:48.631 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:48.631 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.632 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.632 DEBUG:boto:Method: GET 2022-09-03T06:40:48.633 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.633 DEBUG:boto:Data: 2022-09-03T06:40:48.634 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.634 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.635 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.636 DEBUG:boto:Params: {} 2022-09-03T06:40:48.636 DEBUG:boto:Token: None 2022-09-03T06:40:48.637 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.638 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QyU+OgFDajZkaWIsFVEzo1/yd+s= 2022-09-03T06:40:48.639 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QyU+OgFDajZkaWIsFVEzo1/yd+s=', 'Content-Length': '0'} 2022-09-03T06:40:48.642 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004496089619c48a4e-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.643 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:48.643 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:48.644 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.645 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.645 DEBUG:boto:Method: GET 2022-09-03T06:40:48.646 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.647 DEBUG:boto:Data: 2022-09-03T06:40:48.647 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.648 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.648 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.649 DEBUG:boto:Params: {} 2022-09-03T06:40:48.650 DEBUG:boto:Token: None 2022-09-03T06:40:48.651 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.651 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QyU+OgFDajZkaWIsFVEzo1/yd+s= 2022-09-03T06:40:48.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QyU+OgFDajZkaWIsFVEzo1/yd+s=', 'Content-Length': '0'} 2022-09-03T06:40:48.654 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f8612abb850d24c-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.698 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:48.700 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:48.700 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.701 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.702 DEBUG:boto:Method: HEAD 2022-09-03T06:40:48.704 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.704 DEBUG:boto:Data: 2022-09-03T06:40:48.705 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.705 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.706 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.706 DEBUG:boto:Params: {} 2022-09-03T06:40:48.707 DEBUG:boto:Token: None 2022-09-03T06:40:48.707 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.708 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HFlPjdZuezZbBOhNnJ/k2Ve2sW8= 2022-09-03T06:40:48.709 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HFlPjdZuezZbBOhNnJ/k2Ve2sW8=', 'Content-Length': '0'} 2022-09-03T06:40:48.711 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000676051362b67cde3-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.711 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:48.712 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:48.712 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.713 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.713 DEBUG:boto:Method: HEAD 2022-09-03T06:40:48.714 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.715 DEBUG:boto:Data: 2022-09-03T06:40:48.715 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.716 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.716 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.717 DEBUG:boto:Params: {} 2022-09-03T06:40:48.717 DEBUG:boto:Token: None 2022-09-03T06:40:48.718 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.718 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HFlPjdZuezZbBOhNnJ/k2Ve2sW8= 2022-09-03T06:40:48.719 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HFlPjdZuezZbBOhNnJ/k2Ve2sW8=', 'Content-Length': '0'} 2022-09-03T06:40:48.721 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000328556065590e87f-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.723 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:40:48.723 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:48.724 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:48.725 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.726 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.727 DEBUG:boto:Method: GET 2022-09-03T06:40:48.727 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.728 DEBUG:boto:Data: 2022-09-03T06:40:48.729 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.729 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.730 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.730 DEBUG:boto:Params: {} 2022-09-03T06:40:48.731 DEBUG:boto:Token: None 2022-09-03T06:40:48.732 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.733 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QyU+OgFDajZkaWIsFVEzo1/yd+s= 2022-09-03T06:40:48.734 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QyU+OgFDajZkaWIsFVEzo1/yd+s=', 'Content-Length': '0'} 2022-09-03T06:40:48.737 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009b6bc75d942f2623-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.782 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:48.783 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:48.784 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.785 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.787 DEBUG:boto:Method: GET 2022-09-03T06:40:48.788 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.789 DEBUG:boto:Data: 2022-09-03T06:40:48.790 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.791 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.791 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.792 DEBUG:boto:Params: {} 2022-09-03T06:40:48.792 DEBUG:boto:Token: None 2022-09-03T06:40:48.793 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:48.794 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QyU+OgFDajZkaWIsFVEzo1/yd+s= 2022-09-03T06:40:48.795 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QyU+OgFDajZkaWIsFVEzo1/yd+s=', 'Content-Length': '0'} 2022-09-03T06:40:48.798 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003911cd0b1b12a403-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.799 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:48.799 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:48.800 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:48.800 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:48.801 DEBUG:boto:Method: GET 2022-09-03T06:40:48.802 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:40:48.803 DEBUG:boto:Data: 2022-09-03T06:40:48.804 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.805 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.806 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.807 DEBUG:boto:Params: {} 2022-09-03T06:40:48.808 DEBUG:boto:Token: None 2022-09-03T06:40:48.809 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/ 2022-09-03T06:40:48.810 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM= 2022-09-03T06:40:48.811 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM=', 'Content-Length': '0'} 2022-09-03T06:40:48.815 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000028c45f8d45a972de-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.816 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:48.817 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:48.817 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:48.818 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:48.819 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:48.819 DEBUG:boto:Method: GET 2022-09-03T06:40:48.820 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:40:48.820 DEBUG:boto:Data: 2022-09-03T06:40:48.821 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.821 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.822 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.822 DEBUG:boto:Params: {} 2022-09-03T06:40:48.823 DEBUG:boto:Token: None 2022-09-03T06:40:48.824 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/ 2022-09-03T06:40:48.824 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM= 2022-09-03T06:40:48.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM=', 'Content-Length': '0'} 2022-09-03T06:40:48.829 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003080f58482ea7ad9-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.830 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:48.831 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:48.832 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:48.833 DEBUG:boto:Method: HEAD 2022-09-03T06:40:48.833 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:40:48.834 DEBUG:boto:Data: 2022-09-03T06:40:48.835 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.835 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.836 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.836 DEBUG:boto:Params: {} 2022-09-03T06:40:48.837 DEBUG:boto:Token: None 2022-09-03T06:40:48.837 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj51 2022-09-03T06:40:48.839 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hiyQcnGaBnLpWyhp9jQX0wKzSKE= 2022-09-03T06:40:48.839 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hiyQcnGaBnLpWyhp9jQX0wKzSKE=', 'Content-Length': '0'} 2022-09-03T06:40:48.842 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f3b323909cff0560-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.843 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:48.843 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:48.844 DEBUG:boto:Method: HEAD 2022-09-03T06:40:48.845 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:40:48.845 DEBUG:boto:Data: 2022-09-03T06:40:48.846 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.846 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.847 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.847 DEBUG:boto:Params: {} 2022-09-03T06:40:48.848 DEBUG:boto:Token: None 2022-09-03T06:40:48.849 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj51 2022-09-03T06:40:48.849 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hiyQcnGaBnLpWyhp9jQX0wKzSKE= 2022-09-03T06:40:48.850 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hiyQcnGaBnLpWyhp9jQX0wKzSKE=', 'Content-Length': '0'} 2022-09-03T06:40:48.852 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065439bccde465eb9-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.853 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:40:48.854 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:48.855 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:48.856 DEBUG:boto:Method: GET 2022-09-03T06:40:48.857 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:40:48.857 DEBUG:boto:Data: 2022-09-03T06:40:48.859 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.860 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.860 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.861 DEBUG:boto:Params: {} 2022-09-03T06:40:48.862 DEBUG:boto:Token: None 2022-09-03T06:40:48.863 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj51 2022-09-03T06:40:48.864 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fCh4WJ4LlLm3Dr8Wbe+X57tQZ0Q= 2022-09-03T06:40:48.865 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fCh4WJ4LlLm3Dr8Wbe+X57tQZ0Q=', 'Content-Length': '0'} 2022-09-03T06:40:48.868 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002fedac26f9cc6cbb-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.910 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:48.911 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:48.912 DEBUG:boto:Method: GET 2022-09-03T06:40:48.913 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:40:48.914 DEBUG:boto:Data: 2022-09-03T06:40:48.915 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.916 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.917 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.918 DEBUG:boto:Params: {} 2022-09-03T06:40:48.919 DEBUG:boto:Token: None 2022-09-03T06:40:48.919 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj51 2022-09-03T06:40:48.920 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fCh4WJ4LlLm3Dr8Wbe+X57tQZ0Q= 2022-09-03T06:40:48.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fCh4WJ4LlLm3Dr8Wbe+X57tQZ0Q=', 'Content-Length': '0'} 2022-09-03T06:40:48.924 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca01d99556e383c3-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.926 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:48.927 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:48.927 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:48.928 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:48.929 DEBUG:boto:Method: GET 2022-09-03T06:40:48.929 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:40:48.930 DEBUG:boto:Data: 2022-09-03T06:40:48.931 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.932 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.932 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.933 DEBUG:boto:Params: {} 2022-09-03T06:40:48.934 DEBUG:boto:Token: None 2022-09-03T06:40:48.935 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/ 2022-09-03T06:40:48.936 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM= 2022-09-03T06:40:48.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM=', 'Content-Length': '0'} 2022-09-03T06:40:48.940 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c0b9f49dddf0a433-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.941 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:48.942 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:48.943 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:48.943 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:48.944 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:48.945 DEBUG:boto:Method: GET 2022-09-03T06:40:48.945 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:40:48.946 DEBUG:boto:Data: 2022-09-03T06:40:48.946 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.947 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.948 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.948 DEBUG:boto:Params: {} 2022-09-03T06:40:48.950 DEBUG:boto:Token: None 2022-09-03T06:40:48.950 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/ 2022-09-03T06:40:48.951 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM= 2022-09-03T06:40:48.952 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6/StHxPZFfbhoNoRcyg4ls/2CAM=', 'Content-Length': '0'} 2022-09-03T06:40:48.955 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009019832eb583164a-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.956 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:48.957 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:40:48.957 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:48.958 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:48.958 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:48.959 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:48.960 DEBUG:boto:Method: GET 2022-09-03T06:40:48.960 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:48.961 DEBUG:boto:Data: 2022-09-03T06:40:48.961 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.962 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.962 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.963 DEBUG:boto:Params: {} 2022-09-03T06:40:48.964 DEBUG:boto:Token: None 2022-09-03T06:40:48.964 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:48.965 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EzNrM0MwcdAFKCQd18Xc0SVOF/4= 2022-09-03T06:40:48.965 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EzNrM0MwcdAFKCQd18Xc0SVOF/4=', 'Content-Length': '0'} 2022-09-03T06:40:48.968 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001074c98c84e57aa4-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.969 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:48.969 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:48.970 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:48.970 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:48.971 DEBUG:boto:Method: GET 2022-09-03T06:40:48.972 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:48.972 DEBUG:boto:Data: 2022-09-03T06:40:48.973 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.973 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.974 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.974 DEBUG:boto:Params: {} 2022-09-03T06:40:48.975 DEBUG:boto:Token: None 2022-09-03T06:40:48.976 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:48.976 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EzNrM0MwcdAFKCQd18Xc0SVOF/4= 2022-09-03T06:40:48.977 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EzNrM0MwcdAFKCQd18Xc0SVOF/4=', 'Content-Length': '0'} 2022-09-03T06:40:48.979 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000095660b08e7273063-006312f6f0-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.980 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:48.981 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:48.981 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:48.982 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:48.983 DEBUG:boto:Method: HEAD 2022-09-03T06:40:48.983 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:48.984 DEBUG:boto:Data: 2022-09-03T06:40:48.984 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.985 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:48.985 DEBUG:boto:Port: 8001 2022-09-03T06:40:48.986 DEBUG:boto:Params: {} 2022-09-03T06:40:48.986 DEBUG:boto:Token: None 2022-09-03T06:40:48.987 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:48.987 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z52mTe8v5tWSijq3j8T/vHdcuXo= 2022-09-03T06:40:48.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z52mTe8v5tWSijq3j8T/vHdcuXo=', 'Content-Length': '0'} 2022-09-03T06:40:48.990 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d58cfd0f60438084-006312f6f0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:48.991 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:48.992 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:48.992 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:48.993 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:48.993 DEBUG:boto:Method: HEAD 2022-09-03T06:40:48.994 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:48.994 DEBUG:boto:Data: 2022-09-03T06:40:48.995 DEBUG:boto:Headers: {} 2022-09-03T06:40:48.995 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:48.996 DEBUG:boto:Port: 8000 2022-09-03T06:40:48.996 DEBUG:boto:Params: {} 2022-09-03T06:40:48.997 DEBUG:boto:Token: None 2022-09-03T06:40:48.997 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:48 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:48.998 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z52mTe8v5tWSijq3j8T/vHdcuXo= 2022-09-03T06:40:48.999 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z52mTe8v5tWSijq3j8T/vHdcuXo=', 'Content-Length': '0'} 2022-09-03T06:40:49.003 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000485cd8366d7bb413-006312f6f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.004 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:40:49.004 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:49.005 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:49.005 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:49.006 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:49.007 DEBUG:boto:Method: GET 2022-09-03T06:40:49.007 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:49.008 DEBUG:boto:Data: 2022-09-03T06:40:49.008 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.009 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:49.009 DEBUG:boto:Port: 8001 2022-09-03T06:40:49.010 DEBUG:boto:Params: {} 2022-09-03T06:40:49.011 DEBUG:boto:Token: None 2022-09-03T06:40:49.011 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:49.012 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:89K1SIykRMPrcRrQEO+WWb2TPlA= 2022-09-03T06:40:49.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:89K1SIykRMPrcRrQEO+WWb2TPlA=', 'Content-Length': '0'} 2022-09-03T06:40:49.015 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c003647f3d749a7-006312f6f1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.016 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:49.017 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:49.017 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:49.018 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:49.018 DEBUG:boto:Method: GET 2022-09-03T06:40:49.019 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:49.019 DEBUG:boto:Data: 2022-09-03T06:40:49.020 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.020 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:49.022 DEBUG:boto:Port: 8000 2022-09-03T06:40:49.023 DEBUG:boto:Params: {} 2022-09-03T06:40:49.024 DEBUG:boto:Token: None 2022-09-03T06:40:49.025 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:49.025 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:89K1SIykRMPrcRrQEO+WWb2TPlA= 2022-09-03T06:40:49.026 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:89K1SIykRMPrcRrQEO+WWb2TPlA=', 'Content-Length': '0'} 2022-09-03T06:40:49.029 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2886e0f8bc51702-006312f6f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.070 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:49.071 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:49.072 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:49.073 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:49.074 DEBUG:boto:Method: GET 2022-09-03T06:40:49.075 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:40:49.076 DEBUG:boto:Data: 2022-09-03T06:40:49.077 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.077 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:49.078 DEBUG:boto:Port: 8001 2022-09-03T06:40:49.079 DEBUG:boto:Params: {} 2022-09-03T06:40:49.080 DEBUG:boto:Token: None 2022-09-03T06:40:49.081 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/ 2022-09-03T06:40:49.082 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc= 2022-09-03T06:40:49.083 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc=', 'Content-Length': '0'} 2022-09-03T06:40:49.086 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007453bd8bfa352e08-006312f6f1-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.087 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:49.088 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:49.089 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:49.089 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:49.090 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:49.090 DEBUG:boto:Method: GET 2022-09-03T06:40:49.091 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:40:49.091 DEBUG:boto:Data: 2022-09-03T06:40:49.092 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.092 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:49.093 DEBUG:boto:Port: 8000 2022-09-03T06:40:49.093 DEBUG:boto:Params: {} 2022-09-03T06:40:49.094 DEBUG:boto:Token: None 2022-09-03T06:40:49.095 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/ 2022-09-03T06:40:49.095 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc= 2022-09-03T06:40:49.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc=', 'Content-Length': '0'} 2022-09-03T06:40:49.099 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006635f29526bc0f27-006312f6f1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.100 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:49.100 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:49.101 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:49.101 DEBUG:boto:Method: HEAD 2022-09-03T06:40:49.102 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:40:49.102 DEBUG:boto:Data: 2022-09-03T06:40:49.103 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.103 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:49.104 DEBUG:boto:Port: 8001 2022-09-03T06:40:49.104 DEBUG:boto:Params: {} 2022-09-03T06:40:49.105 DEBUG:boto:Token: None 2022-09-03T06:40:49.106 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj55 2022-09-03T06:40:49.106 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j+t1l+pPKdhVDQvhhc/np9BPKeU= 2022-09-03T06:40:49.107 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j+t1l+pPKdhVDQvhhc/np9BPKeU=', 'Content-Length': '0'} 2022-09-03T06:40:49.109 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008a37cde1827e4100-006312f6f1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.110 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:49.110 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:49.111 DEBUG:boto:Method: HEAD 2022-09-03T06:40:49.111 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:40:49.112 DEBUG:boto:Data: 2022-09-03T06:40:49.112 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.113 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:49.113 DEBUG:boto:Port: 8000 2022-09-03T06:40:49.114 DEBUG:boto:Params: {} 2022-09-03T06:40:49.114 DEBUG:boto:Token: None 2022-09-03T06:40:49.115 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj55 2022-09-03T06:40:49.116 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j+t1l+pPKdhVDQvhhc/np9BPKeU= 2022-09-03T06:40:49.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j+t1l+pPKdhVDQvhhc/np9BPKeU=', 'Content-Length': '0'} 2022-09-03T06:40:49.126 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000066ed2a371d2a0ab7-006312f6f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.128 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:40:49.129 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:49.130 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:49.131 DEBUG:boto:Method: GET 2022-09-03T06:40:49.132 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:40:49.134 DEBUG:boto:Data: 2022-09-03T06:40:49.136 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.137 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:49.138 DEBUG:boto:Port: 8001 2022-09-03T06:40:49.139 DEBUG:boto:Params: {} 2022-09-03T06:40:49.140 DEBUG:boto:Token: None 2022-09-03T06:40:49.142 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj55 2022-09-03T06:40:49.143 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ktYTrwtZcGAe/hgH+eTBZjQEVaw= 2022-09-03T06:40:49.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ktYTrwtZcGAe/hgH+eTBZjQEVaw=', 'Content-Length': '0'} 2022-09-03T06:40:49.146 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c82e9f95b926ff89-006312f6f1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.147 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:49.148 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:49.149 DEBUG:boto:Method: GET 2022-09-03T06:40:49.149 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:40:49.150 DEBUG:boto:Data: 2022-09-03T06:40:49.150 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.151 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:49.152 DEBUG:boto:Port: 8000 2022-09-03T06:40:49.152 DEBUG:boto:Params: {} 2022-09-03T06:40:49.153 DEBUG:boto:Token: None 2022-09-03T06:40:49.153 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj55 2022-09-03T06:40:49.154 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ktYTrwtZcGAe/hgH+eTBZjQEVaw= 2022-09-03T06:40:49.155 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ktYTrwtZcGAe/hgH+eTBZjQEVaw=', 'Content-Length': '0'} 2022-09-03T06:40:49.157 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cd94e5d41ea94ea4-006312f6f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.198 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:49.199 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:49.200 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:49.201 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:49.201 DEBUG:boto:Method: GET 2022-09-03T06:40:49.202 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:40:49.203 DEBUG:boto:Data: 2022-09-03T06:40:49.203 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.204 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:49.204 DEBUG:boto:Port: 8001 2022-09-03T06:40:49.205 DEBUG:boto:Params: {} 2022-09-03T06:40:49.206 DEBUG:boto:Token: None 2022-09-03T06:40:49.207 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/ 2022-09-03T06:40:49.207 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc= 2022-09-03T06:40:49.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc=', 'Content-Length': '0'} 2022-09-03T06:40:49.211 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000fbb18dcc9ce98ab-006312f6f1-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.213 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:49.213 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:49.214 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:49.215 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:49.215 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:49.216 DEBUG:boto:Method: GET 2022-09-03T06:40:49.216 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:40:49.217 DEBUG:boto:Data: 2022-09-03T06:40:49.217 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.218 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:49.219 DEBUG:boto:Port: 8000 2022-09-03T06:40:49.219 DEBUG:boto:Params: {} 2022-09-03T06:40:49.220 DEBUG:boto:Token: None 2022-09-03T06:40:49.221 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/ 2022-09-03T06:40:49.222 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc= 2022-09-03T06:40:49.223 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc=', 'Content-Length': '0'} 2022-09-03T06:40:49.226 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005767e515ee43da9c-006312f6f1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.227 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:49.228 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:40:49.228 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:49.229 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:49.229 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.230 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.231 DEBUG:boto:Method: GET 2022-09-03T06:40:49.232 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.233 DEBUG:boto:Data: 2022-09-03T06:40:49.233 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.234 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:49.235 DEBUG:boto:Port: 8001 2022-09-03T06:40:49.236 DEBUG:boto:Params: {} 2022-09-03T06:40:49.238 DEBUG:boto:Token: None 2022-09-03T06:40:49.239 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.239 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0ytv9frKEXqBgmcXWjCS9ejeFNg= 2022-09-03T06:40:49.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0ytv9frKEXqBgmcXWjCS9ejeFNg=', 'Content-Length': '0'} 2022-09-03T06:40:49.243 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dbb4622ed065e06f-006312f6f1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.286 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:49.287 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:49.288 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.288 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.289 DEBUG:boto:Method: GET 2022-09-03T06:40:49.290 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.291 DEBUG:boto:Data: 2022-09-03T06:40:49.292 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.294 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:49.295 DEBUG:boto:Port: 8000 2022-09-03T06:40:49.296 DEBUG:boto:Params: {} 2022-09-03T06:40:49.297 DEBUG:boto:Token: None 2022-09-03T06:40:49.299 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.300 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0ytv9frKEXqBgmcXWjCS9ejeFNg= 2022-09-03T06:40:49.301 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0ytv9frKEXqBgmcXWjCS9ejeFNg=', 'Content-Length': '0'} 2022-09-03T06:40:49.304 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd40e7886b409645-006312f6f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.305 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:49.306 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:49.307 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.307 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.308 DEBUG:boto:Method: HEAD 2022-09-03T06:40:49.308 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.309 DEBUG:boto:Data: 2022-09-03T06:40:49.309 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.310 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:49.310 DEBUG:boto:Port: 8001 2022-09-03T06:40:49.311 DEBUG:boto:Params: {} 2022-09-03T06:40:49.312 DEBUG:boto:Token: None 2022-09-03T06:40:49.312 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.313 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AfLD6roI3PiQRrhpJ5/YRocXsdw= 2022-09-03T06:40:49.313 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AfLD6roI3PiQRrhpJ5/YRocXsdw=', 'Content-Length': '0'} 2022-09-03T06:40:49.316 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000978790ee2c5c576a-006312f6f1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.317 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:49.317 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:49.318 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.319 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.319 DEBUG:boto:Method: HEAD 2022-09-03T06:40:49.320 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.321 DEBUG:boto:Data: 2022-09-03T06:40:49.322 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.323 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:49.324 DEBUG:boto:Port: 8000 2022-09-03T06:40:49.325 DEBUG:boto:Params: {} 2022-09-03T06:40:49.326 DEBUG:boto:Token: None 2022-09-03T06:40:49.327 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.328 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AfLD6roI3PiQRrhpJ5/YRocXsdw= 2022-09-03T06:40:49.329 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AfLD6roI3PiQRrhpJ5/YRocXsdw=', 'Content-Length': '0'} 2022-09-03T06:40:49.332 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000088702a3ad60a7be6-006312f6f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.333 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:40:49.333 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:49.334 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:49.335 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.336 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.337 DEBUG:boto:Method: GET 2022-09-03T06:40:49.337 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.338 DEBUG:boto:Data: 2022-09-03T06:40:49.338 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.339 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:49.340 DEBUG:boto:Port: 8001 2022-09-03T06:40:49.340 DEBUG:boto:Params: {} 2022-09-03T06:40:49.341 DEBUG:boto:Token: None 2022-09-03T06:40:49.342 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.342 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0ytv9frKEXqBgmcXWjCS9ejeFNg= 2022-09-03T06:40:49.343 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0ytv9frKEXqBgmcXWjCS9ejeFNg=', 'Content-Length': '0'} 2022-09-03T06:40:49.346 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002223ed4240cc0164-006312f6f1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.346 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:49.347 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:49.348 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.348 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.349 DEBUG:boto:Method: GET 2022-09-03T06:40:49.350 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.351 DEBUG:boto:Data: 2022-09-03T06:40:49.352 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.352 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:49.353 DEBUG:boto:Port: 8000 2022-09-03T06:40:49.353 DEBUG:boto:Params: {} 2022-09-03T06:40:49.354 DEBUG:boto:Token: None 2022-09-03T06:40:49.355 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:49.355 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0ytv9frKEXqBgmcXWjCS9ejeFNg= 2022-09-03T06:40:49.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0ytv9frKEXqBgmcXWjCS9ejeFNg=', 'Content-Length': '0'} 2022-09-03T06:40:49.358 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e8c6e52810c57677-006312f6f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.359 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:49.360 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:49.362 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:49.362 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:49.363 DEBUG:boto:Method: GET 2022-09-03T06:40:49.364 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:40:49.364 DEBUG:boto:Data: 2022-09-03T06:40:49.365 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.366 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:49.367 DEBUG:boto:Port: 8001 2022-09-03T06:40:49.368 DEBUG:boto:Params: {} 2022-09-03T06:40:49.369 DEBUG:boto:Token: None 2022-09-03T06:40:49.370 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/ 2022-09-03T06:40:49.372 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc= 2022-09-03T06:40:49.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc=', 'Content-Length': '0'} 2022-09-03T06:40:49.376 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000028c027081b850d68-006312f6f1-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.376 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:49.377 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:49.378 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:49.379 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:49.379 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:49.380 DEBUG:boto:Method: GET 2022-09-03T06:40:49.380 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:40:49.382 DEBUG:boto:Data: 2022-09-03T06:40:49.382 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.383 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:49.383 DEBUG:boto:Port: 8000 2022-09-03T06:40:49.384 DEBUG:boto:Params: {} 2022-09-03T06:40:49.385 DEBUG:boto:Token: None 2022-09-03T06:40:49.385 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/ 2022-09-03T06:40:49.386 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc= 2022-09-03T06:40:49.387 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc=', 'Content-Length': '0'} 2022-09-03T06:40:49.390 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f4c5421fb1df4a75-006312f6f1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.391 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:49.392 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:49.393 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:49.394 DEBUG:boto:Method: HEAD 2022-09-03T06:40:49.395 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:40:49.396 DEBUG:boto:Data: 2022-09-03T06:40:49.397 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.398 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:49.398 DEBUG:boto:Port: 8001 2022-09-03T06:40:49.399 DEBUG:boto:Params: {} 2022-09-03T06:40:49.400 DEBUG:boto:Token: None 2022-09-03T06:40:49.400 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj68 2022-09-03T06:40:49.401 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:att896vSgqkIAc3zuFq6zaWO3RQ= 2022-09-03T06:40:49.402 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:att896vSgqkIAc3zuFq6zaWO3RQ=', 'Content-Length': '0'} 2022-09-03T06:40:49.406 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000014ea49a9ef6ba03e-006312f6f1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.407 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:49.408 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:49.409 DEBUG:boto:Method: HEAD 2022-09-03T06:40:49.410 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:40:49.410 DEBUG:boto:Data: 2022-09-03T06:40:49.411 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.412 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:49.413 DEBUG:boto:Port: 8000 2022-09-03T06:40:49.413 DEBUG:boto:Params: {} 2022-09-03T06:40:49.414 DEBUG:boto:Token: None 2022-09-03T06:40:49.415 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj68 2022-09-03T06:40:49.416 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:att896vSgqkIAc3zuFq6zaWO3RQ= 2022-09-03T06:40:49.416 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:att896vSgqkIAc3zuFq6zaWO3RQ=', 'Content-Length': '0'} 2022-09-03T06:40:49.418 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000006ff9736d043656-006312f6f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.420 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:40:49.421 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:49.422 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:49.423 DEBUG:boto:Method: GET 2022-09-03T06:40:49.424 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:40:49.425 DEBUG:boto:Data: 2022-09-03T06:40:49.426 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.427 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:49.428 DEBUG:boto:Port: 8001 2022-09-03T06:40:49.429 DEBUG:boto:Params: {} 2022-09-03T06:40:49.430 DEBUG:boto:Token: None 2022-09-03T06:40:49.431 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj68 2022-09-03T06:40:49.432 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UhspVKO7jEGK9BbsNxPVWl0RgvE= 2022-09-03T06:40:49.433 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UhspVKO7jEGK9BbsNxPVWl0RgvE=', 'Content-Length': '0'} 2022-09-03T06:40:49.436 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000088f85a5e2e0a9135-006312f6f1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.438 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:49.439 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:49.440 DEBUG:boto:Method: GET 2022-09-03T06:40:49.441 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:40:49.442 DEBUG:boto:Data: 2022-09-03T06:40:49.443 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.443 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:49.444 DEBUG:boto:Port: 8000 2022-09-03T06:40:49.445 DEBUG:boto:Params: {} 2022-09-03T06:40:49.446 DEBUG:boto:Token: None 2022-09-03T06:40:49.447 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj68 2022-09-03T06:40:49.448 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UhspVKO7jEGK9BbsNxPVWl0RgvE= 2022-09-03T06:40:49.449 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UhspVKO7jEGK9BbsNxPVWl0RgvE=', 'Content-Length': '0'} 2022-09-03T06:40:49.451 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000729d14d583130405-006312f6f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.494 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:49.495 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:49.495 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:49.496 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:49.496 DEBUG:boto:Method: GET 2022-09-03T06:40:49.497 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:40:49.498 DEBUG:boto:Data: 2022-09-03T06:40:49.498 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.499 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:49.500 DEBUG:boto:Port: 8001 2022-09-03T06:40:49.501 DEBUG:boto:Params: {} 2022-09-03T06:40:49.502 DEBUG:boto:Token: None 2022-09-03T06:40:49.503 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/ 2022-09-03T06:40:49.504 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc= 2022-09-03T06:40:49.505 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc=', 'Content-Length': '0'} 2022-09-03T06:40:49.509 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b16062a098576706-006312f6f1-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.510 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:49.511 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:49.512 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:49.513 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:49.514 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:49.515 DEBUG:boto:Method: GET 2022-09-03T06:40:49.516 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:40:49.518 DEBUG:boto:Data: 2022-09-03T06:40:49.519 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.519 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:49.520 DEBUG:boto:Port: 8000 2022-09-03T06:40:49.521 DEBUG:boto:Params: {} 2022-09-03T06:40:49.522 DEBUG:boto:Token: None 2022-09-03T06:40:49.523 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/ 2022-09-03T06:40:49.524 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc= 2022-09-03T06:40:49.524 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc=', 'Content-Length': '0'} 2022-09-03T06:40:49.528 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000005643b305942f30d-006312f6f1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.528 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:49.529 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:40:49.530 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:49.531 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:49.531 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.532 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.533 DEBUG:boto:Method: GET 2022-09-03T06:40:49.533 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.534 DEBUG:boto:Data: 2022-09-03T06:40:49.534 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.535 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:49.535 DEBUG:boto:Port: 8001 2022-09-03T06:40:49.536 DEBUG:boto:Params: {} 2022-09-03T06:40:49.536 DEBUG:boto:Token: None 2022-09-03T06:40:49.537 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.538 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:76GQyQpr6l72OOxy7w7lj3i9l60= 2022-09-03T06:40:49.538 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:76GQyQpr6l72OOxy7w7lj3i9l60=', 'Content-Length': '0'} 2022-09-03T06:40:49.541 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005d6e1a82d966194c-006312f6f1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.542 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:49.543 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:49.543 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.544 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.544 DEBUG:boto:Method: GET 2022-09-03T06:40:49.545 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.545 DEBUG:boto:Data: 2022-09-03T06:40:49.546 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.546 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:49.547 DEBUG:boto:Port: 8000 2022-09-03T06:40:49.547 DEBUG:boto:Params: {} 2022-09-03T06:40:49.548 DEBUG:boto:Token: None 2022-09-03T06:40:49.549 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.549 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:76GQyQpr6l72OOxy7w7lj3i9l60= 2022-09-03T06:40:49.550 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:76GQyQpr6l72OOxy7w7lj3i9l60=', 'Content-Length': '0'} 2022-09-03T06:40:49.552 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e29bf08fcd30b28e-006312f6f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.556 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:49.557 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:49.558 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.558 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.559 DEBUG:boto:Method: HEAD 2022-09-03T06:40:49.560 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.560 DEBUG:boto:Data: 2022-09-03T06:40:49.561 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.561 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:49.562 DEBUG:boto:Port: 8001 2022-09-03T06:40:49.563 DEBUG:boto:Params: {} 2022-09-03T06:40:49.564 DEBUG:boto:Token: None 2022-09-03T06:40:49.564 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.565 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rlnuy9MgoKRZ3LIUN5DGFyTJoSs= 2022-09-03T06:40:49.566 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rlnuy9MgoKRZ3LIUN5DGFyTJoSs=', 'Content-Length': '0'} 2022-09-03T06:40:49.569 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002824c838ed17b7f3-006312f6f1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.570 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:49.570 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:49.571 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.571 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.572 DEBUG:boto:Method: HEAD 2022-09-03T06:40:49.572 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.573 DEBUG:boto:Data: 2022-09-03T06:40:49.574 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.575 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:49.576 DEBUG:boto:Port: 8000 2022-09-03T06:40:49.576 DEBUG:boto:Params: {} 2022-09-03T06:40:49.577 DEBUG:boto:Token: None 2022-09-03T06:40:49.578 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.578 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rlnuy9MgoKRZ3LIUN5DGFyTJoSs= 2022-09-03T06:40:49.579 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rlnuy9MgoKRZ3LIUN5DGFyTJoSs=', 'Content-Length': '0'} 2022-09-03T06:40:49.581 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b7b34397556ad9e9-006312f6f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.582 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:40:49.583 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:49.586 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:49.586 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.587 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.587 DEBUG:boto:Method: GET 2022-09-03T06:40:49.588 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.589 DEBUG:boto:Data: 2022-09-03T06:40:49.589 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.590 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:49.590 DEBUG:boto:Port: 8001 2022-09-03T06:40:49.591 DEBUG:boto:Params: {} 2022-09-03T06:40:49.592 DEBUG:boto:Token: None 2022-09-03T06:40:49.592 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.593 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:76GQyQpr6l72OOxy7w7lj3i9l60= 2022-09-03T06:40:49.594 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:76GQyQpr6l72OOxy7w7lj3i9l60=', 'Content-Length': '0'} 2022-09-03T06:40:49.596 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007c2c0e0b6fedb96f-006312f6f1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.597 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:49.598 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:49.598 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.599 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.600 DEBUG:boto:Method: GET 2022-09-03T06:40:49.600 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.601 DEBUG:boto:Data: 2022-09-03T06:40:49.601 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.602 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:49.602 DEBUG:boto:Port: 8000 2022-09-03T06:40:49.603 DEBUG:boto:Params: {} 2022-09-03T06:40:49.604 DEBUG:boto:Token: None 2022-09-03T06:40:49.604 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:49.605 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:76GQyQpr6l72OOxy7w7lj3i9l60= 2022-09-03T06:40:49.606 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:76GQyQpr6l72OOxy7w7lj3i9l60=', 'Content-Length': '0'} 2022-09-03T06:40:49.608 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2a798d107a87791-006312f6f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.650 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:49.651 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:49.652 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:49.653 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:49.653 DEBUG:boto:Method: GET 2022-09-03T06:40:49.654 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:40:49.655 DEBUG:boto:Data: 2022-09-03T06:40:49.656 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.656 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:49.657 DEBUG:boto:Port: 8001 2022-09-03T06:40:49.657 DEBUG:boto:Params: {} 2022-09-03T06:40:49.658 DEBUG:boto:Token: None 2022-09-03T06:40:49.659 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/ 2022-09-03T06:40:49.660 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc= 2022-09-03T06:40:49.661 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc=', 'Content-Length': '0'} 2022-09-03T06:40:49.664 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000119bb9fa472f459f-006312f6f1-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.665 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:49.666 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:49.666 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:49.667 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:49.667 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:49.668 DEBUG:boto:Method: GET 2022-09-03T06:40:49.669 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:40:49.669 DEBUG:boto:Data: 2022-09-03T06:40:49.670 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.670 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:49.671 DEBUG:boto:Port: 8000 2022-09-03T06:40:49.671 DEBUG:boto:Params: {} 2022-09-03T06:40:49.672 DEBUG:boto:Token: None 2022-09-03T06:40:49.673 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/ 2022-09-03T06:40:49.673 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc= 2022-09-03T06:40:49.674 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc=', 'Content-Length': '0'} 2022-09-03T06:40:49.678 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c79b69b33a0683f9-006312f6f1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.678 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:49.679 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:49.680 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:49.680 DEBUG:boto:Method: HEAD 2022-09-03T06:40:49.681 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:40:49.681 DEBUG:boto:Data: 2022-09-03T06:40:49.682 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.683 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:49.683 DEBUG:boto:Port: 8001 2022-09-03T06:40:49.684 DEBUG:boto:Params: {} 2022-09-03T06:40:49.685 DEBUG:boto:Token: None 2022-09-03T06:40:49.685 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj85 2022-09-03T06:40:49.686 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wywIJmh/fwSmCyc9dbuoWCBphO0= 2022-09-03T06:40:49.687 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wywIJmh/fwSmCyc9dbuoWCBphO0=', 'Content-Length': '0'} 2022-09-03T06:40:49.689 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018e8c1a6ce60ca79-006312f6f1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.690 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:49.690 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:49.691 DEBUG:boto:Method: HEAD 2022-09-03T06:40:49.691 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:40:49.692 DEBUG:boto:Data: 2022-09-03T06:40:49.693 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.693 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:49.694 DEBUG:boto:Port: 8000 2022-09-03T06:40:49.694 DEBUG:boto:Params: {} 2022-09-03T06:40:49.695 DEBUG:boto:Token: None 2022-09-03T06:40:49.695 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj85 2022-09-03T06:40:49.696 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wywIJmh/fwSmCyc9dbuoWCBphO0= 2022-09-03T06:40:49.697 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wywIJmh/fwSmCyc9dbuoWCBphO0=', 'Content-Length': '0'} 2022-09-03T06:40:49.699 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000019e40fa5afeed431-006312f6f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.699 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:40:49.700 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:49.701 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:49.702 DEBUG:boto:Method: GET 2022-09-03T06:40:49.702 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:40:49.703 DEBUG:boto:Data: 2022-09-03T06:40:49.704 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.704 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:49.705 DEBUG:boto:Port: 8001 2022-09-03T06:40:49.705 DEBUG:boto:Params: {} 2022-09-03T06:40:49.706 DEBUG:boto:Token: None 2022-09-03T06:40:49.707 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj85 2022-09-03T06:40:49.707 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aDfGPCPRfkRsSuzlQYypU6ThpHY= 2022-09-03T06:40:49.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aDfGPCPRfkRsSuzlQYypU6ThpHY=', 'Content-Length': '0'} 2022-09-03T06:40:49.710 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b43ad3e1b370ab63-006312f6f1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.758 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:49.760 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:49.761 DEBUG:boto:Method: GET 2022-09-03T06:40:49.762 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:40:49.765 DEBUG:boto:Data: 2022-09-03T06:40:49.767 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.767 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:49.768 DEBUG:boto:Port: 8000 2022-09-03T06:40:49.768 DEBUG:boto:Params: {} 2022-09-03T06:40:49.769 DEBUG:boto:Token: None 2022-09-03T06:40:49.770 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj85 2022-09-03T06:40:49.771 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aDfGPCPRfkRsSuzlQYypU6ThpHY= 2022-09-03T06:40:49.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aDfGPCPRfkRsSuzlQYypU6ThpHY=', 'Content-Length': '0'} 2022-09-03T06:40:49.775 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000067dcefc414904d6f-006312f6f1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.777 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:49.778 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:49.779 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:49.780 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:49.780 DEBUG:boto:Method: GET 2022-09-03T06:40:49.782 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:40:49.784 DEBUG:boto:Data: 2022-09-03T06:40:49.784 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.785 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:49.786 DEBUG:boto:Port: 8001 2022-09-03T06:40:49.786 DEBUG:boto:Params: {} 2022-09-03T06:40:49.787 DEBUG:boto:Token: None 2022-09-03T06:40:49.787 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/ 2022-09-03T06:40:49.788 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc= 2022-09-03T06:40:49.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc=', 'Content-Length': '0'} 2022-09-03T06:40:49.792 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e3384f6f7ef0bfef-006312f6f1-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.793 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:49.794 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:49.794 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:49.795 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:49.795 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:49.796 DEBUG:boto:Method: GET 2022-09-03T06:40:49.797 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:40:49.797 DEBUG:boto:Data: 2022-09-03T06:40:49.798 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.799 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:49.799 DEBUG:boto:Port: 8000 2022-09-03T06:40:49.800 DEBUG:boto:Params: {} 2022-09-03T06:40:49.801 DEBUG:boto:Token: None 2022-09-03T06:40:49.802 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/ 2022-09-03T06:40:49.803 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc= 2022-09-03T06:40:49.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aTNC07l8bpvaCMi8xOABi/T/XYc=', 'Content-Length': '0'} 2022-09-03T06:40:49.807 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a7c735fb7038067c-006312f6f1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.850 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:49.851 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-75 zones={test-zone3, test-zone1} 2022-09-03T06:40:49.852 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-75 zones={test-zone3, test-zone2} 2022-09-03T06:40:49.852 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:49.853 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:49.854 DEBUG:boto:Method: HEAD 2022-09-03T06:40:49.855 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:40:49.856 DEBUG:boto:Data: 2022-09-03T06:40:49.856 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.858 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:49.858 DEBUG:boto:Port: 8001 2022-09-03T06:40:49.859 DEBUG:boto:Params: {} 2022-09-03T06:40:49.860 DEBUG:boto:Token: None 2022-09-03T06:40:49.861 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/ 2022-09-03T06:40:49.862 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IoRlJ7g3YoWf4tYairzpAkdu4Wk= 2022-09-03T06:40:49.864 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IoRlJ7g3YoWf4tYairzpAkdu4Wk=', 'Content-Length': '0'} 2022-09-03T06:40:49.867 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '10'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d36ac947c1e17029-006312f6f1-1851-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.868 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:49.869 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:49.869 DEBUG:boto:Method: HEAD 2022-09-03T06:40:49.870 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:40:49.870 DEBUG:boto:Data: 2022-09-03T06:40:49.871 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.872 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:49.872 DEBUG:boto:Port: 8000 2022-09-03T06:40:49.873 DEBUG:boto:Params: {} 2022-09-03T06:40:49.873 DEBUG:boto:Token: None 2022-09-03T06:40:49.874 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/ 2022-09-03T06:40:49.875 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IoRlJ7g3YoWf4tYairzpAkdu4Wk= 2022-09-03T06:40:49.876 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IoRlJ7g3YoWf4tYairzpAkdu4Wk=', 'Content-Length': '0'} 2022-09-03T06:40:49.878 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '10'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005127181c76a4e793-006312f6f1-188c-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.879 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:40:49.880 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:49.881 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:49.881 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:49.882 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:49.883 DEBUG:boto:Method: GET 2022-09-03T06:40:49.883 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:49.884 DEBUG:boto:Data: 2022-09-03T06:40:49.884 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.885 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:49.886 DEBUG:boto:Port: 8001 2022-09-03T06:40:49.886 DEBUG:boto:Params: {} 2022-09-03T06:40:49.887 DEBUG:boto:Token: None 2022-09-03T06:40:49.887 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/?versions 2022-09-03T06:40:49.889 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xNOnDEj85CUS577FGHJfWazfa9c= 2022-09-03T06:40:49.889 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xNOnDEj85CUS577FGHJfWazfa9c=', 'Content-Length': '0'} 2022-09-03T06:40:49.899 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008684b810cc8ffbca-006312f6f1-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.900 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:49.901 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:40:49.903 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:40:49.903 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:40:49.903 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:40:49.905 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:40:49.905 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:40:49.906 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:40:49.906 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:40:49.907 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:40:49.907 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:40:49.908 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:40:49.908 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:49.909 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:49.910 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:49.910 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:49.911 DEBUG:boto:Method: GET 2022-09-03T06:40:49.911 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:49.912 DEBUG:boto:Data: 2022-09-03T06:40:49.913 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.913 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:49.914 DEBUG:boto:Port: 8000 2022-09-03T06:40:49.914 DEBUG:boto:Params: {} 2022-09-03T06:40:49.915 DEBUG:boto:Token: None 2022-09-03T06:40:49.916 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/?versions 2022-09-03T06:40:49.916 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xNOnDEj85CUS577FGHJfWazfa9c= 2022-09-03T06:40:49.917 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xNOnDEj85CUS577FGHJfWazfa9c=', 'Content-Length': '0'} 2022-09-03T06:40:49.920 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f8b36adf33911458-006312f6f1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.921 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:49.923 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:40:49.923 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:40:49.924 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:40:49.924 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:40:49.925 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:40:49.926 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:40:49.926 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:40:49.927 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:40:49.927 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:40:49.928 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:40:49.929 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:49.929 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:49.930 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:49.930 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:49.931 DEBUG:boto:Method: GET 2022-09-03T06:40:49.932 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:49.932 DEBUG:boto:Data: 2022-09-03T06:40:49.933 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.933 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:49.934 DEBUG:boto:Port: 8001 2022-09-03T06:40:49.935 DEBUG:boto:Params: {} 2022-09-03T06:40:49.935 DEBUG:boto:Token: None 2022-09-03T06:40:49.936 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/?versions 2022-09-03T06:40:49.936 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xNOnDEj85CUS577FGHJfWazfa9c= 2022-09-03T06:40:49.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xNOnDEj85CUS577FGHJfWazfa9c=', 'Content-Length': '0'} 2022-09-03T06:40:49.940 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000051df4cdfeee43353-006312f6f1-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.941 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:49.942 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:49.943 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:49.943 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:49.944 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:49.945 DEBUG:boto:Method: GET 2022-09-03T06:40:49.945 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:40:49.946 DEBUG:boto:Data: 2022-09-03T06:40:49.946 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.947 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:49.947 DEBUG:boto:Port: 8000 2022-09-03T06:40:49.948 DEBUG:boto:Params: {} 2022-09-03T06:40:49.949 DEBUG:boto:Token: None 2022-09-03T06:40:49.949 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/?versions 2022-09-03T06:40:49.950 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xNOnDEj85CUS577FGHJfWazfa9c= 2022-09-03T06:40:49.950 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xNOnDEj85CUS577FGHJfWazfa9c=', 'Content-Length': '0'} 2022-09-03T06:40:49.953 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009a57b2887db7ded0-006312f6f1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.954 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:49.957 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:40:49.957 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:49.958 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:49.958 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:49.959 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:49.959 DEBUG:boto:Method: GET 2022-09-03T06:40:49.960 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:49.960 DEBUG:boto:Data: 2022-09-03T06:40:49.961 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.961 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:49.962 DEBUG:boto:Port: 8001 2022-09-03T06:40:49.963 DEBUG:boto:Params: {} 2022-09-03T06:40:49.966 DEBUG:boto:Token: None 2022-09-03T06:40:49.966 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:49.969 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c7cCLYMRn5niP6g1qrXRdiQaExw= 2022-09-03T06:40:49.971 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c7cCLYMRn5niP6g1qrXRdiQaExw=', 'Content-Length': '0'} 2022-09-03T06:40:49.974 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000663b6d1ef6061911-006312f6f1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.975 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:49.976 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:49.977 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:49.980 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:49.981 DEBUG:boto:Method: GET 2022-09-03T06:40:49.982 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:49.983 DEBUG:boto:Data: 2022-09-03T06:40:49.984 DEBUG:boto:Headers: {} 2022-09-03T06:40:49.984 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:49.985 DEBUG:boto:Port: 8000 2022-09-03T06:40:49.986 DEBUG:boto:Params: {} 2022-09-03T06:40:49.987 DEBUG:boto:Token: None 2022-09-03T06:40:49.988 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:49 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:49.990 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c7cCLYMRn5niP6g1qrXRdiQaExw= 2022-09-03T06:40:49.991 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c7cCLYMRn5niP6g1qrXRdiQaExw=', 'Content-Length': '0'} 2022-09-03T06:40:49.994 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000022de2f27c65b7efa-006312f6f1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:49.995 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:49.996 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:49.997 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:49.997 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:49.998 DEBUG:boto:Method: HEAD 2022-09-03T06:40:49.999 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:49.999 DEBUG:boto:Data: 2022-09-03T06:40:50.000 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.000 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.001 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.001 DEBUG:boto:Params: {} 2022-09-03T06:40:50.002 DEBUG:boto:Token: None 2022-09-03T06:40:50.002 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:50.003 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Pvg5tWuaMzkVG0gUebOXZTXz0WQ= 2022-09-03T06:40:50.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Pvg5tWuaMzkVG0gUebOXZTXz0WQ=', 'Content-Length': '0'} 2022-09-03T06:40:50.006 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a6cc0bcca10ca63-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.007 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:50.008 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:50.009 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:50.010 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:50.011 DEBUG:boto:Method: HEAD 2022-09-03T06:40:50.012 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:50.013 DEBUG:boto:Data: 2022-09-03T06:40:50.014 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.015 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:50.016 DEBUG:boto:Port: 8000 2022-09-03T06:40:50.017 DEBUG:boto:Params: {} 2022-09-03T06:40:50.018 DEBUG:boto:Token: None 2022-09-03T06:40:50.018 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:50.019 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Pvg5tWuaMzkVG0gUebOXZTXz0WQ= 2022-09-03T06:40:50.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Pvg5tWuaMzkVG0gUebOXZTXz0WQ=', 'Content-Length': '0'} 2022-09-03T06:40:50.022 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a8f75f6fed5aa0e1-006312f6f2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.023 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:40:50.023 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:50.024 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:50.024 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:50.025 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:50.025 DEBUG:boto:Method: GET 2022-09-03T06:40:50.026 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:50.026 DEBUG:boto:Data: 2022-09-03T06:40:50.027 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.027 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.028 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.028 DEBUG:boto:Params: {} 2022-09-03T06:40:50.029 DEBUG:boto:Token: None 2022-09-03T06:40:50.030 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:50.030 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tF9D2x2xr+Uw5Qx8XWAPHVTI6Cc= 2022-09-03T06:40:50.031 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tF9D2x2xr+Uw5Qx8XWAPHVTI6Cc=', 'Content-Length': '0'} 2022-09-03T06:40:50.033 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f8fe42ec15e71cc-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.074 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:50.075 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:50.075 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:50.076 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:50.076 DEBUG:boto:Method: GET 2022-09-03T06:40:50.077 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:50.078 DEBUG:boto:Data: 2022-09-03T06:40:50.078 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.079 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:50.079 DEBUG:boto:Port: 8000 2022-09-03T06:40:50.080 DEBUG:boto:Params: {} 2022-09-03T06:40:50.080 DEBUG:boto:Token: None 2022-09-03T06:40:50.081 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:40:50.082 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tF9D2x2xr+Uw5Qx8XWAPHVTI6Cc= 2022-09-03T06:40:50.083 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tF9D2x2xr+Uw5Qx8XWAPHVTI6Cc=', 'Content-Length': '0'} 2022-09-03T06:40:50.086 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000069f9d3a17986935-006312f6f2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.087 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:50.087 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:50.088 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:50.089 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:50.089 DEBUG:boto:Method: GET 2022-09-03T06:40:50.090 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:40:50.091 DEBUG:boto:Data: 2022-09-03T06:40:50.091 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.092 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.092 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.093 DEBUG:boto:Params: {} 2022-09-03T06:40:50.094 DEBUG:boto:Token: None 2022-09-03T06:40:50.094 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/ 2022-09-03T06:40:50.095 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY= 2022-09-03T06:40:50.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY=', 'Content-Length': '0'} 2022-09-03T06:40:50.099 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000016aeaba4f053659c-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.100 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:50.101 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:50.101 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:50.102 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:50.102 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:50.103 DEBUG:boto:Method: GET 2022-09-03T06:40:50.104 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:40:50.104 DEBUG:boto:Data: 2022-09-03T06:40:50.105 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.105 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:50.106 DEBUG:boto:Port: 8000 2022-09-03T06:40:50.107 DEBUG:boto:Params: {} 2022-09-03T06:40:50.107 DEBUG:boto:Token: None 2022-09-03T06:40:50.108 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/ 2022-09-03T06:40:50.109 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY= 2022-09-03T06:40:50.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY=', 'Content-Length': '0'} 2022-09-03T06:40:50.112 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000063dda0454715519f-006312f6f2-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.113 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:50.114 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:50.115 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:50.116 DEBUG:boto:Method: HEAD 2022-09-03T06:40:50.117 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:40:50.117 DEBUG:boto:Data: 2022-09-03T06:40:50.118 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.118 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.119 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.119 DEBUG:boto:Params: {} 2022-09-03T06:40:50.120 DEBUG:boto:Token: None 2022-09-03T06:40:50.121 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj11 2022-09-03T06:40:50.121 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bH8ZWYKwBrnrEKffBZqPatz44HQ= 2022-09-03T06:40:50.122 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bH8ZWYKwBrnrEKffBZqPatz44HQ=', 'Content-Length': '0'} 2022-09-03T06:40:50.124 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d643574c652b29e-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.125 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:50.126 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:50.126 DEBUG:boto:Method: HEAD 2022-09-03T06:40:50.127 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:40:50.127 DEBUG:boto:Data: 2022-09-03T06:40:50.128 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.129 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:50.130 DEBUG:boto:Port: 8000 2022-09-03T06:40:50.132 DEBUG:boto:Params: {} 2022-09-03T06:40:50.133 DEBUG:boto:Token: None 2022-09-03T06:40:50.135 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj11 2022-09-03T06:40:50.136 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bH8ZWYKwBrnrEKffBZqPatz44HQ= 2022-09-03T06:40:50.138 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bH8ZWYKwBrnrEKffBZqPatz44HQ=', 'Content-Length': '0'} 2022-09-03T06:40:50.140 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000befa2932b767d2d3-006312f6f2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.142 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:40:50.143 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:50.148 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:50.149 DEBUG:boto:Method: GET 2022-09-03T06:40:50.150 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:40:50.151 DEBUG:boto:Data: 2022-09-03T06:40:50.152 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.153 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.154 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.155 DEBUG:boto:Params: {} 2022-09-03T06:40:50.156 DEBUG:boto:Token: None 2022-09-03T06:40:50.157 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj11 2022-09-03T06:40:50.158 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e8np3LyMETn56rtPnW1TQ24RtRE= 2022-09-03T06:40:50.159 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e8np3LyMETn56rtPnW1TQ24RtRE=', 'Content-Length': '0'} 2022-09-03T06:40:50.162 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a4cf39d7ae860a7-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.164 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:40:50.165 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:40:50.166 DEBUG:boto:Method: GET 2022-09-03T06:40:50.167 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:40:50.168 DEBUG:boto:Data: 2022-09-03T06:40:50.169 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.170 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:50.171 DEBUG:boto:Port: 8000 2022-09-03T06:40:50.172 DEBUG:boto:Params: {} 2022-09-03T06:40:50.173 DEBUG:boto:Token: None 2022-09-03T06:40:50.174 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj11 2022-09-03T06:40:50.175 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e8np3LyMETn56rtPnW1TQ24RtRE= 2022-09-03T06:40:50.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e8np3LyMETn56rtPnW1TQ24RtRE=', 'Content-Length': '0'} 2022-09-03T06:40:50.179 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004d768c610e1b8d17-006312f6f2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.181 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:50.182 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:50.183 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:50.184 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:50.185 DEBUG:boto:Method: GET 2022-09-03T06:40:50.186 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:40:50.187 DEBUG:boto:Data: 2022-09-03T06:40:50.188 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.189 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.190 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.191 DEBUG:boto:Params: {} 2022-09-03T06:40:50.192 DEBUG:boto:Token: None 2022-09-03T06:40:50.193 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/ 2022-09-03T06:40:50.193 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY= 2022-09-03T06:40:50.194 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY=', 'Content-Length': '0'} 2022-09-03T06:40:50.197 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a5f4b7c932cbb284-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.238 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:50.239 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:50.240 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:50.240 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:50.241 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:40:50.242 DEBUG:boto:Method: GET 2022-09-03T06:40:50.242 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:40:50.243 DEBUG:boto:Data: 2022-09-03T06:40:50.244 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.244 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:50.245 DEBUG:boto:Port: 8000 2022-09-03T06:40:50.246 DEBUG:boto:Params: {} 2022-09-03T06:40:50.246 DEBUG:boto:Token: None 2022-09-03T06:40:50.247 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/ 2022-09-03T06:40:50.248 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY= 2022-09-03T06:40:50.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY=', 'Content-Length': '0'} 2022-09-03T06:40:50.251 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000025aa469e055bed9a-006312f6f2-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.252 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:50.253 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:40:50.254 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:50.255 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:50.255 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.256 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.256 DEBUG:boto:Method: GET 2022-09-03T06:40:50.257 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.258 DEBUG:boto:Data: 2022-09-03T06:40:50.258 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.259 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.259 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.260 DEBUG:boto:Params: {} 2022-09-03T06:40:50.261 DEBUG:boto:Token: None 2022-09-03T06:40:50.262 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.262 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EmKZ9sG2XKHcdd5awk4G79+3+Jw= 2022-09-03T06:40:50.263 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EmKZ9sG2XKHcdd5awk4G79+3+Jw=', 'Content-Length': '0'} 2022-09-03T06:40:50.265 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aee7b44b203f3f46-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.266 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:50.267 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:50.267 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.268 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.269 DEBUG:boto:Method: GET 2022-09-03T06:40:50.269 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.270 DEBUG:boto:Data: 2022-09-03T06:40:50.270 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.271 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:50.271 DEBUG:boto:Port: 8000 2022-09-03T06:40:50.272 DEBUG:boto:Params: {} 2022-09-03T06:40:50.273 DEBUG:boto:Token: None 2022-09-03T06:40:50.273 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.274 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EmKZ9sG2XKHcdd5awk4G79+3+Jw= 2022-09-03T06:40:50.275 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EmKZ9sG2XKHcdd5awk4G79+3+Jw=', 'Content-Length': '0'} 2022-09-03T06:40:50.277 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000151e06a4f814b201-006312f6f2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.279 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:50.281 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:50.282 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.282 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.283 DEBUG:boto:Method: HEAD 2022-09-03T06:40:50.284 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.284 DEBUG:boto:Data: 2022-09-03T06:40:50.285 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.286 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.287 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.288 DEBUG:boto:Params: {} 2022-09-03T06:40:50.289 DEBUG:boto:Token: None 2022-09-03T06:40:50.290 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.292 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:U+wK514fUIB5H4aYk42eDaTcqVU= 2022-09-03T06:40:50.293 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:U+wK514fUIB5H4aYk42eDaTcqVU=', 'Content-Length': '0'} 2022-09-03T06:40:50.295 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000876a904aaca62b7a-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.297 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:50.297 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:50.298 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.299 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.300 DEBUG:boto:Method: HEAD 2022-09-03T06:40:50.301 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.302 DEBUG:boto:Data: 2022-09-03T06:40:50.303 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.304 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:50.305 DEBUG:boto:Port: 8000 2022-09-03T06:40:50.306 DEBUG:boto:Params: {} 2022-09-03T06:40:50.308 DEBUG:boto:Token: None 2022-09-03T06:40:50.308 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.309 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:U+wK514fUIB5H4aYk42eDaTcqVU= 2022-09-03T06:40:50.310 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:U+wK514fUIB5H4aYk42eDaTcqVU=', 'Content-Length': '0'} 2022-09-03T06:40:50.313 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000099c0844cf55c0e82-006312f6f2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.314 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:40:50.315 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:50.316 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:50.317 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.318 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.319 DEBUG:boto:Method: GET 2022-09-03T06:40:50.320 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.320 DEBUG:boto:Data: 2022-09-03T06:40:50.321 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.321 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.322 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.323 DEBUG:boto:Params: {} 2022-09-03T06:40:50.323 DEBUG:boto:Token: None 2022-09-03T06:40:50.324 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.324 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EmKZ9sG2XKHcdd5awk4G79+3+Jw= 2022-09-03T06:40:50.325 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EmKZ9sG2XKHcdd5awk4G79+3+Jw=', 'Content-Length': '0'} 2022-09-03T06:40:50.327 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009481e1c56415b776-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.328 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:50.329 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:50.329 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.330 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.331 DEBUG:boto:Method: GET 2022-09-03T06:40:50.331 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.332 DEBUG:boto:Data: 2022-09-03T06:40:50.332 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.333 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:50.333 DEBUG:boto:Port: 8000 2022-09-03T06:40:50.334 DEBUG:boto:Params: {} 2022-09-03T06:40:50.335 DEBUG:boto:Token: None 2022-09-03T06:40:50.335 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:40:50.336 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EmKZ9sG2XKHcdd5awk4G79+3+Jw= 2022-09-03T06:40:50.337 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EmKZ9sG2XKHcdd5awk4G79+3+Jw=', 'Content-Length': '0'} 2022-09-03T06:40:50.342 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d5d36044f9608ec3-006312f6f2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.343 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:50.344 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:50.345 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:50.347 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:50.348 DEBUG:boto:Method: GET 2022-09-03T06:40:50.349 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:40:50.350 DEBUG:boto:Data: 2022-09-03T06:40:50.351 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.352 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.352 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.353 DEBUG:boto:Params: {} 2022-09-03T06:40:50.354 DEBUG:boto:Token: None 2022-09-03T06:40:50.355 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/ 2022-09-03T06:40:50.355 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY= 2022-09-03T06:40:50.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY=', 'Content-Length': '0'} 2022-09-03T06:40:50.360 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009aa4c8e1066f9e34-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.360 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:50.362 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:50.363 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:50.364 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:50.365 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:50.366 DEBUG:boto:Method: GET 2022-09-03T06:40:50.366 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:40:50.367 DEBUG:boto:Data: 2022-09-03T06:40:50.367 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.368 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:50.368 DEBUG:boto:Port: 8000 2022-09-03T06:40:50.369 DEBUG:boto:Params: {} 2022-09-03T06:40:50.369 DEBUG:boto:Token: None 2022-09-03T06:40:50.370 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/ 2022-09-03T06:40:50.371 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY= 2022-09-03T06:40:50.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY=', 'Content-Length': '0'} 2022-09-03T06:40:50.374 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002d19d7b7c2dbf319-006312f6f2-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.375 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:50.376 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:50.376 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:50.377 DEBUG:boto:Method: HEAD 2022-09-03T06:40:50.378 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:40:50.378 DEBUG:boto:Data: 2022-09-03T06:40:50.379 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.379 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.380 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.380 DEBUG:boto:Params: {} 2022-09-03T06:40:50.381 DEBUG:boto:Token: None 2022-09-03T06:40:50.381 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj17 2022-09-03T06:40:50.382 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/AP1lqZrvdNKs3POXhkvo+VAlGo= 2022-09-03T06:40:50.383 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/AP1lqZrvdNKs3POXhkvo+VAlGo=', 'Content-Length': '0'} 2022-09-03T06:40:50.385 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006a15f57266f43fa8-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.386 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:50.386 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:50.387 DEBUG:boto:Method: HEAD 2022-09-03T06:40:50.387 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:40:50.388 DEBUG:boto:Data: 2022-09-03T06:40:50.388 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.389 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:50.389 DEBUG:boto:Port: 8000 2022-09-03T06:40:50.390 DEBUG:boto:Params: {} 2022-09-03T06:40:50.390 DEBUG:boto:Token: None 2022-09-03T06:40:50.391 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj17 2022-09-03T06:40:50.391 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/AP1lqZrvdNKs3POXhkvo+VAlGo= 2022-09-03T06:40:50.392 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/AP1lqZrvdNKs3POXhkvo+VAlGo=', 'Content-Length': '0'} 2022-09-03T06:40:50.395 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d913b499d52071ab-006312f6f2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.396 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:40:50.397 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:50.398 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:50.398 DEBUG:boto:Method: GET 2022-09-03T06:40:50.399 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:40:50.400 DEBUG:boto:Data: 2022-09-03T06:40:50.401 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.403 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.404 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.404 DEBUG:boto:Params: {} 2022-09-03T06:40:50.405 DEBUG:boto:Token: None 2022-09-03T06:40:50.405 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj17 2022-09-03T06:40:50.406 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TLGlWCiiCRPKAxV7S5SHbUb9sZo= 2022-09-03T06:40:50.407 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TLGlWCiiCRPKAxV7S5SHbUb9sZo=', 'Content-Length': '0'} 2022-09-03T06:40:50.410 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ac01ae9414d4f067-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.411 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:40:50.413 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:40:50.414 DEBUG:boto:Method: GET 2022-09-03T06:40:50.414 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:40:50.415 DEBUG:boto:Data: 2022-09-03T06:40:50.415 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.416 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:50.416 DEBUG:boto:Port: 8000 2022-09-03T06:40:50.422 DEBUG:boto:Params: {} 2022-09-03T06:40:50.423 DEBUG:boto:Token: None 2022-09-03T06:40:50.423 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj17 2022-09-03T06:40:50.424 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TLGlWCiiCRPKAxV7S5SHbUb9sZo= 2022-09-03T06:40:50.425 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TLGlWCiiCRPKAxV7S5SHbUb9sZo=', 'Content-Length': '0'} 2022-09-03T06:40:50.427 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d4fd63ecbb649478-006312f6f2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.478 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:50.479 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:50.479 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:50.480 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:50.480 DEBUG:boto:Method: GET 2022-09-03T06:40:50.481 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:40:50.481 DEBUG:boto:Data: 2022-09-03T06:40:50.482 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.483 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.483 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.484 DEBUG:boto:Params: {} 2022-09-03T06:40:50.484 DEBUG:boto:Token: None 2022-09-03T06:40:50.485 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/ 2022-09-03T06:40:50.485 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY= 2022-09-03T06:40:50.486 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY=', 'Content-Length': '0'} 2022-09-03T06:40:50.490 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000df9ed092e8a6f95b-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.490 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:50.493 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:50.494 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:50.495 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:50.496 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:40:50.497 DEBUG:boto:Method: GET 2022-09-03T06:40:50.499 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:40:50.500 DEBUG:boto:Data: 2022-09-03T06:40:50.500 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.501 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:50.501 DEBUG:boto:Port: 8000 2022-09-03T06:40:50.502 DEBUG:boto:Params: {} 2022-09-03T06:40:50.503 DEBUG:boto:Token: None 2022-09-03T06:40:50.503 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/ 2022-09-03T06:40:50.504 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY= 2022-09-03T06:40:50.505 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY=', 'Content-Length': '0'} 2022-09-03T06:40:50.509 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004aa340c81d5864f7-006312f6f2-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.511 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:50.513 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:40:50.514 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:50.516 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:50.517 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.519 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.520 DEBUG:boto:Method: GET 2022-09-03T06:40:50.521 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.522 DEBUG:boto:Data: 2022-09-03T06:40:50.523 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.524 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.525 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.526 DEBUG:boto:Params: {} 2022-09-03T06:40:50.527 DEBUG:boto:Token: None 2022-09-03T06:40:50.529 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.530 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jJ4Kos1Is5gthkD+O7hreUowZXM= 2022-09-03T06:40:50.531 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jJ4Kos1Is5gthkD+O7hreUowZXM=', 'Content-Length': '0'} 2022-09-03T06:40:50.536 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000371f160ad744bc04-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.538 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:50.539 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:50.540 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.540 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.541 DEBUG:boto:Method: GET 2022-09-03T06:40:50.542 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.543 DEBUG:boto:Data: 2022-09-03T06:40:50.544 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.545 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:50.546 DEBUG:boto:Port: 8000 2022-09-03T06:40:50.547 DEBUG:boto:Params: {} 2022-09-03T06:40:50.548 DEBUG:boto:Token: None 2022-09-03T06:40:50.549 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.550 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jJ4Kos1Is5gthkD+O7hreUowZXM= 2022-09-03T06:40:50.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jJ4Kos1Is5gthkD+O7hreUowZXM=', 'Content-Length': '0'} 2022-09-03T06:40:50.555 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000031f6515cfcd6d26d-006312f6f2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.556 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:50.557 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:50.557 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.558 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.558 DEBUG:boto:Method: HEAD 2022-09-03T06:40:50.559 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.559 DEBUG:boto:Data: 2022-09-03T06:40:50.561 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.561 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.562 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.562 DEBUG:boto:Params: {} 2022-09-03T06:40:50.563 DEBUG:boto:Token: None 2022-09-03T06:40:50.564 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.564 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jz1S4YEDMb62MZZivpP0i2yWnJw= 2022-09-03T06:40:50.565 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jz1S4YEDMb62MZZivpP0i2yWnJw=', 'Content-Length': '0'} 2022-09-03T06:40:50.567 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eec0d457b069eed3-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.568 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:50.571 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:50.572 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.573 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.573 DEBUG:boto:Method: HEAD 2022-09-03T06:40:50.574 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.575 DEBUG:boto:Data: 2022-09-03T06:40:50.576 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.577 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:50.578 DEBUG:boto:Port: 8000 2022-09-03T06:40:50.578 DEBUG:boto:Params: {} 2022-09-03T06:40:50.579 DEBUG:boto:Token: None 2022-09-03T06:40:50.580 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.580 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jz1S4YEDMb62MZZivpP0i2yWnJw= 2022-09-03T06:40:50.581 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jz1S4YEDMb62MZZivpP0i2yWnJw=', 'Content-Length': '0'} 2022-09-03T06:40:50.583 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000015ddbd04c9d28a86-006312f6f2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.584 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:40:50.585 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:50.585 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:50.586 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.586 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.587 DEBUG:boto:Method: GET 2022-09-03T06:40:50.588 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.588 DEBUG:boto:Data: 2022-09-03T06:40:50.589 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.589 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.590 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.591 DEBUG:boto:Params: {} 2022-09-03T06:40:50.591 DEBUG:boto:Token: None 2022-09-03T06:40:50.592 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.592 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jJ4Kos1Is5gthkD+O7hreUowZXM= 2022-09-03T06:40:50.593 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jJ4Kos1Is5gthkD+O7hreUowZXM=', 'Content-Length': '0'} 2022-09-03T06:40:50.599 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a28f0b9cb04d24ab-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.642 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:50.643 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:50.643 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.644 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.645 DEBUG:boto:Method: GET 2022-09-03T06:40:50.645 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.646 DEBUG:boto:Data: 2022-09-03T06:40:50.647 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.648 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:50.649 DEBUG:boto:Port: 8000 2022-09-03T06:40:50.649 DEBUG:boto:Params: {} 2022-09-03T06:40:50.650 DEBUG:boto:Token: None 2022-09-03T06:40:50.651 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:40:50.651 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jJ4Kos1Is5gthkD+O7hreUowZXM= 2022-09-03T06:40:50.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jJ4Kos1Is5gthkD+O7hreUowZXM=', 'Content-Length': '0'} 2022-09-03T06:40:50.656 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002abe788a52be2507-006312f6f2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.657 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:50.658 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:50.659 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:50.659 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:50.660 DEBUG:boto:Method: GET 2022-09-03T06:40:50.660 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:40:50.661 DEBUG:boto:Data: 2022-09-03T06:40:50.661 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.662 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.663 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.664 DEBUG:boto:Params: {} 2022-09-03T06:40:50.665 DEBUG:boto:Token: None 2022-09-03T06:40:50.665 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/ 2022-09-03T06:40:50.667 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY= 2022-09-03T06:40:50.667 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY=', 'Content-Length': '0'} 2022-09-03T06:40:50.670 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f57ce25eddf3da67-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.671 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:50.672 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:50.673 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:50.674 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:50.675 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:50.676 DEBUG:boto:Method: GET 2022-09-03T06:40:50.677 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:40:50.679 DEBUG:boto:Data: 2022-09-03T06:40:50.680 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.681 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:50.681 DEBUG:boto:Port: 8000 2022-09-03T06:40:50.682 DEBUG:boto:Params: {} 2022-09-03T06:40:50.683 DEBUG:boto:Token: None 2022-09-03T06:40:50.685 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/ 2022-09-03T06:40:50.686 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY= 2022-09-03T06:40:50.686 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY=', 'Content-Length': '0'} 2022-09-03T06:40:50.689 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a2eb71571310fc73-006312f6f2-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.691 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:50.692 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:50.692 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:50.693 DEBUG:boto:Method: HEAD 2022-09-03T06:40:50.694 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:40:50.695 DEBUG:boto:Data: 2022-09-03T06:40:50.696 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.696 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.697 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.698 DEBUG:boto:Params: {} 2022-09-03T06:40:50.699 DEBUG:boto:Token: None 2022-09-03T06:40:50.699 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj22 2022-09-03T06:40:50.700 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mh9/ky32QyXsdW2uQDGK6Va4/L0= 2022-09-03T06:40:50.701 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mh9/ky32QyXsdW2uQDGK6Va4/L0=', 'Content-Length': '0'} 2022-09-03T06:40:50.703 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d981c35dc67f02c4-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.703 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:50.704 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:50.704 DEBUG:boto:Method: HEAD 2022-09-03T06:40:50.705 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:40:50.706 DEBUG:boto:Data: 2022-09-03T06:40:50.706 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.707 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:50.707 DEBUG:boto:Port: 8000 2022-09-03T06:40:50.708 DEBUG:boto:Params: {} 2022-09-03T06:40:50.708 DEBUG:boto:Token: None 2022-09-03T06:40:50.709 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj22 2022-09-03T06:40:50.710 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mh9/ky32QyXsdW2uQDGK6Va4/L0= 2022-09-03T06:40:50.710 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mh9/ky32QyXsdW2uQDGK6Va4/L0=', 'Content-Length': '0'} 2022-09-03T06:40:50.713 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a3cf69db7d22a12a-006312f6f2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.714 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:40:50.715 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:50.716 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:50.717 DEBUG:boto:Method: GET 2022-09-03T06:40:50.718 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:40:50.719 DEBUG:boto:Data: 2022-09-03T06:40:50.720 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.723 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.724 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.725 DEBUG:boto:Params: {} 2022-09-03T06:40:50.726 DEBUG:boto:Token: None 2022-09-03T06:40:50.727 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj22 2022-09-03T06:40:50.728 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J2bzhRH5ShQQMYlaLg6kX9RmfY0= 2022-09-03T06:40:50.729 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J2bzhRH5ShQQMYlaLg6kX9RmfY0=', 'Content-Length': '0'} 2022-09-03T06:40:50.734 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e1f472f7678a53f-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.779 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:40:50.780 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:40:50.780 DEBUG:boto:Method: GET 2022-09-03T06:40:50.781 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:40:50.782 DEBUG:boto:Data: 2022-09-03T06:40:50.783 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.783 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:50.784 DEBUG:boto:Port: 8000 2022-09-03T06:40:50.784 DEBUG:boto:Params: {} 2022-09-03T06:40:50.785 DEBUG:boto:Token: None 2022-09-03T06:40:50.786 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj22 2022-09-03T06:40:50.787 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J2bzhRH5ShQQMYlaLg6kX9RmfY0= 2022-09-03T06:40:50.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J2bzhRH5ShQQMYlaLg6kX9RmfY0=', 'Content-Length': '0'} 2022-09-03T06:40:50.791 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d6dc1640bb06254-006312f6f2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.793 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:50.794 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:50.794 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:50.795 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:50.796 DEBUG:boto:Method: GET 2022-09-03T06:40:50.797 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:40:50.797 DEBUG:boto:Data: 2022-09-03T06:40:50.799 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.800 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.801 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.802 DEBUG:boto:Params: {} 2022-09-03T06:40:50.803 DEBUG:boto:Token: None 2022-09-03T06:40:50.804 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/ 2022-09-03T06:40:50.806 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY= 2022-09-03T06:40:50.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY=', 'Content-Length': '0'} 2022-09-03T06:40:50.814 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000abf82ed135b538ff-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.817 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:50.818 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:50.819 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:50.820 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:50.821 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:40:50.825 DEBUG:boto:Method: GET 2022-09-03T06:40:50.826 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:40:50.827 DEBUG:boto:Data: 2022-09-03T06:40:50.828 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.829 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:50.830 DEBUG:boto:Port: 8000 2022-09-03T06:40:50.830 DEBUG:boto:Params: {} 2022-09-03T06:40:50.831 DEBUG:boto:Token: None 2022-09-03T06:40:50.832 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/ 2022-09-03T06:40:50.832 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY= 2022-09-03T06:40:50.833 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY=', 'Content-Length': '0'} 2022-09-03T06:40:50.838 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000036e3b916e58b013f-006312f6f2-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.839 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:50.839 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:40:50.840 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:50.842 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:50.843 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.844 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.844 DEBUG:boto:Method: GET 2022-09-03T06:40:50.845 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.847 DEBUG:boto:Data: 2022-09-03T06:40:50.848 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.849 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.850 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.851 DEBUG:boto:Params: {} 2022-09-03T06:40:50.853 DEBUG:boto:Token: None 2022-09-03T06:40:50.853 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.854 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oNukqhiTwKDzqQG0htr7Fkcbm8k= 2022-09-03T06:40:50.855 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oNukqhiTwKDzqQG0htr7Fkcbm8k=', 'Content-Length': '0'} 2022-09-03T06:40:50.857 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000014864f71a97a6202-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.858 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:50.858 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:50.859 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.860 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.860 DEBUG:boto:Method: GET 2022-09-03T06:40:50.861 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.862 DEBUG:boto:Data: 2022-09-03T06:40:50.862 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.863 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:50.863 DEBUG:boto:Port: 8000 2022-09-03T06:40:50.864 DEBUG:boto:Params: {} 2022-09-03T06:40:50.864 DEBUG:boto:Token: None 2022-09-03T06:40:50.865 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.866 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oNukqhiTwKDzqQG0htr7Fkcbm8k= 2022-09-03T06:40:50.866 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oNukqhiTwKDzqQG0htr7Fkcbm8k=', 'Content-Length': '0'} 2022-09-03T06:40:50.869 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b4874b7b6adabe3-006312f6f2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.872 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:50.875 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:50.876 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.877 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.878 DEBUG:boto:Method: HEAD 2022-09-03T06:40:50.878 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.879 DEBUG:boto:Data: 2022-09-03T06:40:50.880 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.880 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.882 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.882 DEBUG:boto:Params: {} 2022-09-03T06:40:50.883 DEBUG:boto:Token: None 2022-09-03T06:40:50.885 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.886 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wkl7KjiaVPOvvF9NbkKKlkLorL4= 2022-09-03T06:40:50.887 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wkl7KjiaVPOvvF9NbkKKlkLorL4=', 'Content-Length': '0'} 2022-09-03T06:40:50.890 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005d95e721c29c9c6b-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.890 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:50.891 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:50.891 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.892 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.893 DEBUG:boto:Method: HEAD 2022-09-03T06:40:50.893 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.894 DEBUG:boto:Data: 2022-09-03T06:40:50.894 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.895 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:50.895 DEBUG:boto:Port: 8000 2022-09-03T06:40:50.896 DEBUG:boto:Params: {} 2022-09-03T06:40:50.897 DEBUG:boto:Token: None 2022-09-03T06:40:50.897 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.898 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wkl7KjiaVPOvvF9NbkKKlkLorL4= 2022-09-03T06:40:50.899 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wkl7KjiaVPOvvF9NbkKKlkLorL4=', 'Content-Length': '0'} 2022-09-03T06:40:50.901 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d293d2908380869-006312f6f2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.902 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:40:50.903 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:50.903 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:50.904 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.904 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.905 DEBUG:boto:Method: GET 2022-09-03T06:40:50.905 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.906 DEBUG:boto:Data: 2022-09-03T06:40:50.907 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.907 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.908 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.908 DEBUG:boto:Params: {} 2022-09-03T06:40:50.909 DEBUG:boto:Token: None 2022-09-03T06:40:50.911 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.911 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oNukqhiTwKDzqQG0htr7Fkcbm8k= 2022-09-03T06:40:50.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oNukqhiTwKDzqQG0htr7Fkcbm8k=', 'Content-Length': '0'} 2022-09-03T06:40:50.914 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fb23597c2f06673a-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.915 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:50.916 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:50.917 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.918 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.919 DEBUG:boto:Method: GET 2022-09-03T06:40:50.920 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.921 DEBUG:boto:Data: 2022-09-03T06:40:50.922 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.923 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:50.924 DEBUG:boto:Port: 8000 2022-09-03T06:40:50.925 DEBUG:boto:Params: {} 2022-09-03T06:40:50.926 DEBUG:boto:Token: None 2022-09-03T06:40:50.927 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:40:50.928 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oNukqhiTwKDzqQG0htr7Fkcbm8k= 2022-09-03T06:40:50.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oNukqhiTwKDzqQG0htr7Fkcbm8k=', 'Content-Length': '0'} 2022-09-03T06:40:50.932 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a7aace9fdf1e11c6-006312f6f2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.974 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:50.975 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:50.976 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:50.977 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:50.978 DEBUG:boto:Method: GET 2022-09-03T06:40:50.978 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:40:50.979 DEBUG:boto:Data: 2022-09-03T06:40:50.979 DEBUG:boto:Headers: {} 2022-09-03T06:40:50.980 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:50.980 DEBUG:boto:Port: 8001 2022-09-03T06:40:50.981 DEBUG:boto:Params: {} 2022-09-03T06:40:50.982 DEBUG:boto:Token: None 2022-09-03T06:40:50.983 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:50 GMT /dsgvcy-75/ 2022-09-03T06:40:50.984 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY= 2022-09-03T06:40:50.985 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BM4XpN+arfnOiv1WSvAMxGT8yLY=', 'Content-Length': '0'} 2022-09-03T06:40:50.994 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003874133c37664231-006312f6f2-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:50.996 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:50.997 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:50.998 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:50.999 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:50.999 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:51.000 DEBUG:boto:Method: GET 2022-09-03T06:40:51.001 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:40:51.002 DEBUG:boto:Data: 2022-09-03T06:40:51.004 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.006 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.007 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.008 DEBUG:boto:Params: {} 2022-09-03T06:40:51.009 DEBUG:boto:Token: None 2022-09-03T06:40:51.010 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/ 2022-09-03T06:40:51.011 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ= 2022-09-03T06:40:51.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ=', 'Content-Length': '0'} 2022-09-03T06:40:51.015 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000739402fe26529637-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.016 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:51.017 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:51.018 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:51.018 DEBUG:boto:Method: HEAD 2022-09-03T06:40:51.019 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:40:51.020 DEBUG:boto:Data: 2022-09-03T06:40:51.020 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.021 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.021 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.022 DEBUG:boto:Params: {} 2022-09-03T06:40:51.023 DEBUG:boto:Token: None 2022-09-03T06:40:51.024 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj33 2022-09-03T06:40:51.025 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vLDCdOIhEZXClzbuT8r58Is1bLc= 2022-09-03T06:40:51.026 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vLDCdOIhEZXClzbuT8r58Is1bLc=', 'Content-Length': '0'} 2022-09-03T06:40:51.029 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b84b7d67e701424a-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.031 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:51.032 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:51.033 DEBUG:boto:Method: HEAD 2022-09-03T06:40:51.034 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:40:51.035 DEBUG:boto:Data: 2022-09-03T06:40:51.036 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.037 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.038 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.038 DEBUG:boto:Params: {} 2022-09-03T06:40:51.039 DEBUG:boto:Token: None 2022-09-03T06:40:51.040 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj33 2022-09-03T06:40:51.041 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vLDCdOIhEZXClzbuT8r58Is1bLc= 2022-09-03T06:40:51.041 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vLDCdOIhEZXClzbuT8r58Is1bLc=', 'Content-Length': '0'} 2022-09-03T06:40:51.043 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e28cdccd91b8b66b-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.045 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:40:51.045 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:51.046 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:51.047 DEBUG:boto:Method: GET 2022-09-03T06:40:51.047 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:40:51.048 DEBUG:boto:Data: 2022-09-03T06:40:51.049 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.049 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.050 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.050 DEBUG:boto:Params: {} 2022-09-03T06:40:51.051 DEBUG:boto:Token: None 2022-09-03T06:40:51.052 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj33 2022-09-03T06:40:51.052 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5oeEWrjNs0M5Fo0QlPKuVy4a2cw= 2022-09-03T06:40:51.053 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5oeEWrjNs0M5Fo0QlPKuVy4a2cw=', 'Content-Length': '0'} 2022-09-03T06:40:51.055 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000044134f260f169f68-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.098 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:40:51.099 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:40:51.100 DEBUG:boto:Method: GET 2022-09-03T06:40:51.100 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:40:51.101 DEBUG:boto:Data: 2022-09-03T06:40:51.102 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.102 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.103 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.103 DEBUG:boto:Params: {} 2022-09-03T06:40:51.104 DEBUG:boto:Token: None 2022-09-03T06:40:51.104 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj33 2022-09-03T06:40:51.105 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5oeEWrjNs0M5Fo0QlPKuVy4a2cw= 2022-09-03T06:40:51.106 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5oeEWrjNs0M5Fo0QlPKuVy4a2cw=', 'Content-Length': '0'} 2022-09-03T06:40:51.109 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d78c408793936b40-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.110 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:51.111 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:51.111 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:51.112 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:51.112 DEBUG:boto:Method: GET 2022-09-03T06:40:51.113 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:40:51.113 DEBUG:boto:Data: 2022-09-03T06:40:51.114 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.114 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.115 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.116 DEBUG:boto:Params: {} 2022-09-03T06:40:51.116 DEBUG:boto:Token: None 2022-09-03T06:40:51.117 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/ 2022-09-03T06:40:51.118 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ= 2022-09-03T06:40:51.118 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ=', 'Content-Length': '0'} 2022-09-03T06:40:51.121 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003f9a9c3580a14c80-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.122 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:51.123 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:51.124 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:51.125 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:51.125 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:40:51.126 DEBUG:boto:Method: GET 2022-09-03T06:40:51.126 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:40:51.127 DEBUG:boto:Data: 2022-09-03T06:40:51.127 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.129 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.130 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.130 DEBUG:boto:Params: {} 2022-09-03T06:40:51.131 DEBUG:boto:Token: None 2022-09-03T06:40:51.131 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/ 2022-09-03T06:40:51.132 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ= 2022-09-03T06:40:51.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ=', 'Content-Length': '0'} 2022-09-03T06:40:51.136 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f09ccb64c96b4323-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.137 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:51.138 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:40:51.139 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:51.140 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:51.140 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.141 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.142 DEBUG:boto:Method: GET 2022-09-03T06:40:51.143 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.144 DEBUG:boto:Data: 2022-09-03T06:40:51.145 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.145 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.146 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.147 DEBUG:boto:Params: {} 2022-09-03T06:40:51.148 DEBUG:boto:Token: None 2022-09-03T06:40:51.149 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.150 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:byLl8M7I6iyy7+A88FzEklHGK5A= 2022-09-03T06:40:51.151 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:byLl8M7I6iyy7+A88FzEklHGK5A=', 'Content-Length': '0'} 2022-09-03T06:40:51.153 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d43fd02481f84240-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.154 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:51.155 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:51.155 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.156 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.156 DEBUG:boto:Method: GET 2022-09-03T06:40:51.157 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.157 DEBUG:boto:Data: 2022-09-03T06:40:51.158 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.158 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.159 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.160 DEBUG:boto:Params: {} 2022-09-03T06:40:51.160 DEBUG:boto:Token: None 2022-09-03T06:40:51.161 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.161 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:byLl8M7I6iyy7+A88FzEklHGK5A= 2022-09-03T06:40:51.162 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:byLl8M7I6iyy7+A88FzEklHGK5A=', 'Content-Length': '0'} 2022-09-03T06:40:51.164 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c3308e13c4c064d6-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.165 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:51.166 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:51.166 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.167 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.168 DEBUG:boto:Method: HEAD 2022-09-03T06:40:51.168 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.169 DEBUG:boto:Data: 2022-09-03T06:40:51.170 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.170 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.171 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.171 DEBUG:boto:Params: {} 2022-09-03T06:40:51.172 DEBUG:boto:Token: None 2022-09-03T06:40:51.173 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.173 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:63LqmVvgMmMHVtqmbK8jdl9sVMI= 2022-09-03T06:40:51.174 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:63LqmVvgMmMHVtqmbK8jdl9sVMI=', 'Content-Length': '0'} 2022-09-03T06:40:51.176 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002e493e2cdbfdb2c8-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.176 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:51.177 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:51.177 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.178 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.179 DEBUG:boto:Method: HEAD 2022-09-03T06:40:51.179 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.179 DEBUG:boto:Data: 2022-09-03T06:40:51.180 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.180 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.181 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.181 DEBUG:boto:Params: {} 2022-09-03T06:40:51.182 DEBUG:boto:Token: None 2022-09-03T06:40:51.183 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.183 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:63LqmVvgMmMHVtqmbK8jdl9sVMI= 2022-09-03T06:40:51.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:63LqmVvgMmMHVtqmbK8jdl9sVMI=', 'Content-Length': '0'} 2022-09-03T06:40:51.186 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000883563b73177cf27-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.187 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:40:51.187 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:51.188 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:51.188 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.189 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.189 DEBUG:boto:Method: GET 2022-09-03T06:40:51.190 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.191 DEBUG:boto:Data: 2022-09-03T06:40:51.191 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.192 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.192 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.193 DEBUG:boto:Params: {} 2022-09-03T06:40:51.193 DEBUG:boto:Token: None 2022-09-03T06:40:51.194 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.195 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:byLl8M7I6iyy7+A88FzEklHGK5A= 2022-09-03T06:40:51.195 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:byLl8M7I6iyy7+A88FzEklHGK5A=', 'Content-Length': '0'} 2022-09-03T06:40:51.197 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d99890a5f82231ab-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.198 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:51.199 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:51.200 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.200 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.201 DEBUG:boto:Method: GET 2022-09-03T06:40:51.201 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.202 DEBUG:boto:Data: 2022-09-03T06:40:51.203 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.203 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.204 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.204 DEBUG:boto:Params: {} 2022-09-03T06:40:51.205 DEBUG:boto:Token: None 2022-09-03T06:40:51.205 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:40:51.206 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:byLl8M7I6iyy7+A88FzEklHGK5A= 2022-09-03T06:40:51.207 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:byLl8M7I6iyy7+A88FzEklHGK5A=', 'Content-Length': '0'} 2022-09-03T06:40:51.209 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000338d86d452e86f1a-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.210 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:51.210 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:51.211 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:51.211 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:51.212 DEBUG:boto:Method: GET 2022-09-03T06:40:51.212 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:40:51.213 DEBUG:boto:Data: 2022-09-03T06:40:51.213 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.214 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.214 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.215 DEBUG:boto:Params: {} 2022-09-03T06:40:51.216 DEBUG:boto:Token: None 2022-09-03T06:40:51.216 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/ 2022-09-03T06:40:51.217 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ= 2022-09-03T06:40:51.217 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ=', 'Content-Length': '0'} 2022-09-03T06:40:51.220 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006c864b201744cecf-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.221 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:51.222 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:51.222 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:51.223 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:51.223 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:51.224 DEBUG:boto:Method: GET 2022-09-03T06:40:51.225 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:40:51.225 DEBUG:boto:Data: 2022-09-03T06:40:51.226 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.226 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.227 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.227 DEBUG:boto:Params: {} 2022-09-03T06:40:51.228 DEBUG:boto:Token: None 2022-09-03T06:40:51.228 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/ 2022-09-03T06:40:51.229 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ= 2022-09-03T06:40:51.230 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ=', 'Content-Length': '0'} 2022-09-03T06:40:51.233 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000012d7557922fbf4f3-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.233 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:51.234 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:51.235 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:51.236 DEBUG:boto:Method: HEAD 2022-09-03T06:40:51.237 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:40:51.237 DEBUG:boto:Data: 2022-09-03T06:40:51.238 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.239 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.239 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.240 DEBUG:boto:Params: {} 2022-09-03T06:40:51.241 DEBUG:boto:Token: None 2022-09-03T06:40:51.242 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj34 2022-09-03T06:40:51.242 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C+cUOv8/oKMSNLZwPHyNmidqLjE= 2022-09-03T06:40:51.243 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C+cUOv8/oKMSNLZwPHyNmidqLjE=', 'Content-Length': '0'} 2022-09-03T06:40:51.246 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f989ea9834edabae-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.247 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:51.247 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:51.248 DEBUG:boto:Method: HEAD 2022-09-03T06:40:51.248 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:40:51.249 DEBUG:boto:Data: 2022-09-03T06:40:51.250 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.250 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.251 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.251 DEBUG:boto:Params: {} 2022-09-03T06:40:51.252 DEBUG:boto:Token: None 2022-09-03T06:40:51.253 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj34 2022-09-03T06:40:51.253 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C+cUOv8/oKMSNLZwPHyNmidqLjE= 2022-09-03T06:40:51.254 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C+cUOv8/oKMSNLZwPHyNmidqLjE=', 'Content-Length': '0'} 2022-09-03T06:40:51.256 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000014115271aa323f8a-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.256 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:40:51.257 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:51.258 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:51.258 DEBUG:boto:Method: GET 2022-09-03T06:40:51.259 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:40:51.259 DEBUG:boto:Data: 2022-09-03T06:40:51.260 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.260 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.261 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.262 DEBUG:boto:Params: {} 2022-09-03T06:40:51.262 DEBUG:boto:Token: None 2022-09-03T06:40:51.263 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj34 2022-09-03T06:40:51.263 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vVXw4RiRyC4iXg5WqnWJlM6qVRo= 2022-09-03T06:40:51.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vVXw4RiRyC4iXg5WqnWJlM6qVRo=', 'Content-Length': '0'} 2022-09-03T06:40:51.266 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ea825f8e00990bba-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.267 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:40:51.268 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:40:51.269 DEBUG:boto:Method: GET 2022-09-03T06:40:51.270 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:40:51.271 DEBUG:boto:Data: 2022-09-03T06:40:51.271 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.272 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.272 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.273 DEBUG:boto:Params: {} 2022-09-03T06:40:51.274 DEBUG:boto:Token: None 2022-09-03T06:40:51.275 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj34 2022-09-03T06:40:51.275 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vVXw4RiRyC4iXg5WqnWJlM6qVRo= 2022-09-03T06:40:51.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vVXw4RiRyC4iXg5WqnWJlM6qVRo=', 'Content-Length': '0'} 2022-09-03T06:40:51.278 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a63f1619568de5f0-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.279 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:51.280 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:51.280 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:51.281 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:51.281 DEBUG:boto:Method: GET 2022-09-03T06:40:51.282 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:40:51.282 DEBUG:boto:Data: 2022-09-03T06:40:51.283 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.284 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.285 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.286 DEBUG:boto:Params: {} 2022-09-03T06:40:51.286 DEBUG:boto:Token: None 2022-09-03T06:40:51.287 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/ 2022-09-03T06:40:51.288 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ= 2022-09-03T06:40:51.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ=', 'Content-Length': '0'} 2022-09-03T06:40:51.291 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000097e723c0301e3e80-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.338 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:51.339 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:51.340 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:51.340 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:51.341 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:40:51.342 DEBUG:boto:Method: GET 2022-09-03T06:40:51.343 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:40:51.343 DEBUG:boto:Data: 2022-09-03T06:40:51.344 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.346 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.347 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.348 DEBUG:boto:Params: {} 2022-09-03T06:40:51.349 DEBUG:boto:Token: None 2022-09-03T06:40:51.350 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/ 2022-09-03T06:40:51.351 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ= 2022-09-03T06:40:51.353 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ=', 'Content-Length': '0'} 2022-09-03T06:40:51.359 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000038ca10e1510d033b-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.360 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:51.362 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:40:51.363 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:51.364 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:51.365 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.366 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.367 DEBUG:boto:Method: GET 2022-09-03T06:40:51.370 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.371 DEBUG:boto:Data: 2022-09-03T06:40:51.372 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.372 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.373 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.374 DEBUG:boto:Params: {} 2022-09-03T06:40:51.375 DEBUG:boto:Token: None 2022-09-03T06:40:51.375 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.376 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJVYmkeiUiyz4gu8numxqky65n4= 2022-09-03T06:40:51.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJVYmkeiUiyz4gu8numxqky65n4=', 'Content-Length': '0'} 2022-09-03T06:40:51.384 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a02069bbdf7c3422-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.386 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:51.386 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:51.387 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.387 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.388 DEBUG:boto:Method: GET 2022-09-03T06:40:51.389 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.389 DEBUG:boto:Data: 2022-09-03T06:40:51.390 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.390 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.391 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.391 DEBUG:boto:Params: {} 2022-09-03T06:40:51.392 DEBUG:boto:Token: None 2022-09-03T06:40:51.393 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.393 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJVYmkeiUiyz4gu8numxqky65n4= 2022-09-03T06:40:51.394 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJVYmkeiUiyz4gu8numxqky65n4=', 'Content-Length': '0'} 2022-09-03T06:40:51.397 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000457e4d41b80c274f-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.400 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:51.402 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:51.403 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.403 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.404 DEBUG:boto:Method: HEAD 2022-09-03T06:40:51.404 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.405 DEBUG:boto:Data: 2022-09-03T06:40:51.406 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.407 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.408 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.408 DEBUG:boto:Params: {} 2022-09-03T06:40:51.409 DEBUG:boto:Token: None 2022-09-03T06:40:51.409 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.410 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aa/Qyte71cS+a8bAOIDwuOqBjrE= 2022-09-03T06:40:51.411 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aa/Qyte71cS+a8bAOIDwuOqBjrE=', 'Content-Length': '0'} 2022-09-03T06:40:51.413 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000060b809d4223bfebb-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.414 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:51.414 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:51.415 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.419 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.420 DEBUG:boto:Method: HEAD 2022-09-03T06:40:51.421 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.422 DEBUG:boto:Data: 2022-09-03T06:40:51.423 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.423 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.424 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.425 DEBUG:boto:Params: {} 2022-09-03T06:40:51.426 DEBUG:boto:Token: None 2022-09-03T06:40:51.428 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.429 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aa/Qyte71cS+a8bAOIDwuOqBjrE= 2022-09-03T06:40:51.431 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aa/Qyte71cS+a8bAOIDwuOqBjrE=', 'Content-Length': '0'} 2022-09-03T06:40:51.433 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d4994045e033d0cb-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.435 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:40:51.436 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:51.437 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:51.438 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.439 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.440 DEBUG:boto:Method: GET 2022-09-03T06:40:51.441 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.441 DEBUG:boto:Data: 2022-09-03T06:40:51.442 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.443 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.444 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.445 DEBUG:boto:Params: {} 2022-09-03T06:40:51.446 DEBUG:boto:Token: None 2022-09-03T06:40:51.447 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.448 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJVYmkeiUiyz4gu8numxqky65n4= 2022-09-03T06:40:51.449 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJVYmkeiUiyz4gu8numxqky65n4=', 'Content-Length': '0'} 2022-09-03T06:40:51.452 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e91e0e8d44261d02-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.453 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:51.453 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:51.454 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.454 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.455 DEBUG:boto:Method: GET 2022-09-03T06:40:51.456 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.456 DEBUG:boto:Data: 2022-09-03T06:40:51.457 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.457 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.458 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.458 DEBUG:boto:Params: {} 2022-09-03T06:40:51.459 DEBUG:boto:Token: None 2022-09-03T06:40:51.459 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:40:51.460 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJVYmkeiUiyz4gu8numxqky65n4= 2022-09-03T06:40:51.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJVYmkeiUiyz4gu8numxqky65n4=', 'Content-Length': '0'} 2022-09-03T06:40:51.463 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2729a255a04d5a2-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.464 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:51.464 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:51.465 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:51.465 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:51.466 DEBUG:boto:Method: GET 2022-09-03T06:40:51.466 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:40:51.467 DEBUG:boto:Data: 2022-09-03T06:40:51.467 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.468 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.469 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.469 DEBUG:boto:Params: {} 2022-09-03T06:40:51.470 DEBUG:boto:Token: None 2022-09-03T06:40:51.470 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/ 2022-09-03T06:40:51.471 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ= 2022-09-03T06:40:51.472 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ=', 'Content-Length': '0'} 2022-09-03T06:40:51.475 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000030e82c14f1e290-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.475 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:51.476 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:51.477 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:51.477 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:51.478 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:51.479 DEBUG:boto:Method: GET 2022-09-03T06:40:51.479 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:40:51.480 DEBUG:boto:Data: 2022-09-03T06:40:51.480 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.481 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.481 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.482 DEBUG:boto:Params: {} 2022-09-03T06:40:51.482 DEBUG:boto:Token: None 2022-09-03T06:40:51.483 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/ 2022-09-03T06:40:51.484 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ= 2022-09-03T06:40:51.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ=', 'Content-Length': '0'} 2022-09-03T06:40:51.487 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000061974b6721532d1e-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.534 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:51.537 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:51.538 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:51.540 DEBUG:boto:Method: HEAD 2022-09-03T06:40:51.540 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:40:51.541 DEBUG:boto:Data: 2022-09-03T06:40:51.542 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.543 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.544 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.545 DEBUG:boto:Params: {} 2022-09-03T06:40:51.547 DEBUG:boto:Token: None 2022-09-03T06:40:51.548 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj44 2022-09-03T06:40:51.548 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UWpERacGGwTDs+0Syeyom90hDhM= 2022-09-03T06:40:51.549 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UWpERacGGwTDs+0Syeyom90hDhM=', 'Content-Length': '0'} 2022-09-03T06:40:51.551 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cfc002554eb5d2cd-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.552 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:51.553 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:51.553 DEBUG:boto:Method: HEAD 2022-09-03T06:40:51.554 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:40:51.554 DEBUG:boto:Data: 2022-09-03T06:40:51.555 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.555 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.556 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.556 DEBUG:boto:Params: {} 2022-09-03T06:40:51.557 DEBUG:boto:Token: None 2022-09-03T06:40:51.557 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj44 2022-09-03T06:40:51.558 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UWpERacGGwTDs+0Syeyom90hDhM= 2022-09-03T06:40:51.559 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UWpERacGGwTDs+0Syeyom90hDhM=', 'Content-Length': '0'} 2022-09-03T06:40:51.561 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000673588793a511015-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.562 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:40:51.563 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:51.563 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:51.564 DEBUG:boto:Method: GET 2022-09-03T06:40:51.564 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:40:51.565 DEBUG:boto:Data: 2022-09-03T06:40:51.566 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.566 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.567 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.567 DEBUG:boto:Params: {} 2022-09-03T06:40:51.568 DEBUG:boto:Token: None 2022-09-03T06:40:51.568 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj44 2022-09-03T06:40:51.569 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8k5Htzrh2qa5YS9zkovCmSMavX0= 2022-09-03T06:40:51.570 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8k5Htzrh2qa5YS9zkovCmSMavX0=', 'Content-Length': '0'} 2022-09-03T06:40:51.572 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004212ab8fa7315059-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.573 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:40:51.574 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:40:51.574 DEBUG:boto:Method: GET 2022-09-03T06:40:51.575 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:40:51.576 DEBUG:boto:Data: 2022-09-03T06:40:51.576 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.577 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.578 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.578 DEBUG:boto:Params: {} 2022-09-03T06:40:51.579 DEBUG:boto:Token: None 2022-09-03T06:40:51.580 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj44 2022-09-03T06:40:51.581 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8k5Htzrh2qa5YS9zkovCmSMavX0= 2022-09-03T06:40:51.582 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8k5Htzrh2qa5YS9zkovCmSMavX0=', 'Content-Length': '0'} 2022-09-03T06:40:51.585 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008aed42cc11c60d03-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.587 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:51.588 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:51.589 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:51.591 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:51.592 DEBUG:boto:Method: GET 2022-09-03T06:40:51.593 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:40:51.595 DEBUG:boto:Data: 2022-09-03T06:40:51.596 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.596 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.597 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.598 DEBUG:boto:Params: {} 2022-09-03T06:40:51.599 DEBUG:boto:Token: None 2022-09-03T06:40:51.600 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/ 2022-09-03T06:40:51.601 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ= 2022-09-03T06:40:51.601 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ=', 'Content-Length': '0'} 2022-09-03T06:40:51.605 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004967385989ad6842-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.607 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:51.609 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:51.610 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:51.611 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:51.612 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:40:51.613 DEBUG:boto:Method: GET 2022-09-03T06:40:51.614 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:40:51.615 DEBUG:boto:Data: 2022-09-03T06:40:51.617 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.618 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.619 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.620 DEBUG:boto:Params: {} 2022-09-03T06:40:51.621 DEBUG:boto:Token: None 2022-09-03T06:40:51.623 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/ 2022-09-03T06:40:51.624 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ= 2022-09-03T06:40:51.625 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ=', 'Content-Length': '0'} 2022-09-03T06:40:51.629 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002501e9edad42d068-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.630 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:51.631 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:40:51.632 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:51.633 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:51.635 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.636 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.637 DEBUG:boto:Method: GET 2022-09-03T06:40:51.638 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.639 DEBUG:boto:Data: 2022-09-03T06:40:51.640 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.641 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.642 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.643 DEBUG:boto:Params: {} 2022-09-03T06:40:51.644 DEBUG:boto:Token: None 2022-09-03T06:40:51.645 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.647 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Sax4XvvmbaHx/iW98iJKAAVbmg4= 2022-09-03T06:40:51.648 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Sax4XvvmbaHx/iW98iJKAAVbmg4=', 'Content-Length': '0'} 2022-09-03T06:40:51.651 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000032f1aba664b2892e-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.652 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:51.655 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:51.656 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.656 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.657 DEBUG:boto:Method: GET 2022-09-03T06:40:51.658 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.659 DEBUG:boto:Data: 2022-09-03T06:40:51.660 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.661 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.661 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.662 DEBUG:boto:Params: {} 2022-09-03T06:40:51.682 DEBUG:boto:Token: None 2022-09-03T06:40:51.682 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.683 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Sax4XvvmbaHx/iW98iJKAAVbmg4= 2022-09-03T06:40:51.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Sax4XvvmbaHx/iW98iJKAAVbmg4=', 'Content-Length': '0'} 2022-09-03T06:40:51.691 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000039bb97eaaef7e972-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.693 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:51.694 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:51.695 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.696 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.697 DEBUG:boto:Method: HEAD 2022-09-03T06:40:51.697 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.698 DEBUG:boto:Data: 2022-09-03T06:40:51.698 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.699 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.701 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.702 DEBUG:boto:Params: {} 2022-09-03T06:40:51.703 DEBUG:boto:Token: None 2022-09-03T06:40:51.704 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.705 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tFQRMb6FF4lXmqCsMd8Ng0usL+8= 2022-09-03T06:40:51.706 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tFQRMb6FF4lXmqCsMd8Ng0usL+8=', 'Content-Length': '0'} 2022-09-03T06:40:51.709 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000006b4b9b7eb471fb7-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.711 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:51.712 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:51.712 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.713 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.713 DEBUG:boto:Method: HEAD 2022-09-03T06:40:51.714 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.715 DEBUG:boto:Data: 2022-09-03T06:40:51.716 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.716 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.717 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.718 DEBUG:boto:Params: {} 2022-09-03T06:40:51.719 DEBUG:boto:Token: None 2022-09-03T06:40:51.720 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.722 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tFQRMb6FF4lXmqCsMd8Ng0usL+8= 2022-09-03T06:40:51.723 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tFQRMb6FF4lXmqCsMd8Ng0usL+8=', 'Content-Length': '0'} 2022-09-03T06:40:51.727 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004b49f2471a5bb158-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.727 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:40:51.728 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:51.729 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:51.729 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.730 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.731 DEBUG:boto:Method: GET 2022-09-03T06:40:51.731 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.732 DEBUG:boto:Data: 2022-09-03T06:40:51.736 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.737 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.738 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.742 DEBUG:boto:Params: {} 2022-09-03T06:40:51.743 DEBUG:boto:Token: None 2022-09-03T06:40:51.746 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.748 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Sax4XvvmbaHx/iW98iJKAAVbmg4= 2022-09-03T06:40:51.751 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Sax4XvvmbaHx/iW98iJKAAVbmg4=', 'Content-Length': '0'} 2022-09-03T06:40:51.753 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e035331b888b9090-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.755 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:51.756 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:51.757 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.758 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.759 DEBUG:boto:Method: GET 2022-09-03T06:40:51.760 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.761 DEBUG:boto:Data: 2022-09-03T06:40:51.763 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.764 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.765 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.766 DEBUG:boto:Params: {} 2022-09-03T06:40:51.768 DEBUG:boto:Token: None 2022-09-03T06:40:51.769 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:40:51.770 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Sax4XvvmbaHx/iW98iJKAAVbmg4= 2022-09-03T06:40:51.771 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Sax4XvvmbaHx/iW98iJKAAVbmg4=', 'Content-Length': '0'} 2022-09-03T06:40:51.773 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b97735669ccbf034-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.774 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:51.775 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:51.775 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:51.776 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:51.777 DEBUG:boto:Method: GET 2022-09-03T06:40:51.777 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:40:51.778 DEBUG:boto:Data: 2022-09-03T06:40:51.779 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.779 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.780 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.780 DEBUG:boto:Params: {} 2022-09-03T06:40:51.781 DEBUG:boto:Token: None 2022-09-03T06:40:51.782 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/ 2022-09-03T06:40:51.782 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ= 2022-09-03T06:40:51.783 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ=', 'Content-Length': '0'} 2022-09-03T06:40:51.786 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f6794ea0d250ec7e-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.787 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:51.788 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:51.789 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:51.790 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:51.790 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:51.791 DEBUG:boto:Method: GET 2022-09-03T06:40:51.791 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:40:51.792 DEBUG:boto:Data: 2022-09-03T06:40:51.793 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.793 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.794 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.794 DEBUG:boto:Params: {} 2022-09-03T06:40:51.796 DEBUG:boto:Token: None 2022-09-03T06:40:51.797 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/ 2022-09-03T06:40:51.797 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ= 2022-09-03T06:40:51.798 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ=', 'Content-Length': '0'} 2022-09-03T06:40:51.801 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001a8ddd1fc529a823-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.802 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:51.803 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:51.804 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:51.804 DEBUG:boto:Method: HEAD 2022-09-03T06:40:51.805 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:40:51.805 DEBUG:boto:Data: 2022-09-03T06:40:51.806 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.807 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.807 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.808 DEBUG:boto:Params: {} 2022-09-03T06:40:51.808 DEBUG:boto:Token: None 2022-09-03T06:40:51.809 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj51 2022-09-03T06:40:51.810 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:30NqD8Z7CemJRBYMoxGfANdwmPU= 2022-09-03T06:40:51.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:30NqD8Z7CemJRBYMoxGfANdwmPU=', 'Content-Length': '0'} 2022-09-03T06:40:51.812 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009fb6106d550b3784-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.813 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:51.814 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:51.815 DEBUG:boto:Method: HEAD 2022-09-03T06:40:51.815 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:40:51.816 DEBUG:boto:Data: 2022-09-03T06:40:51.817 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.817 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.818 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.818 DEBUG:boto:Params: {} 2022-09-03T06:40:51.820 DEBUG:boto:Token: None 2022-09-03T06:40:51.820 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj51 2022-09-03T06:40:51.821 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:30NqD8Z7CemJRBYMoxGfANdwmPU= 2022-09-03T06:40:51.822 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:30NqD8Z7CemJRBYMoxGfANdwmPU=', 'Content-Length': '0'} 2022-09-03T06:40:51.824 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005ed2311777e29f26-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.825 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:40:51.826 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:51.826 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:51.827 DEBUG:boto:Method: GET 2022-09-03T06:40:51.828 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:40:51.829 DEBUG:boto:Data: 2022-09-03T06:40:51.830 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.830 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.831 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.831 DEBUG:boto:Params: {} 2022-09-03T06:40:51.832 DEBUG:boto:Token: None 2022-09-03T06:40:51.833 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj51 2022-09-03T06:40:51.833 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zpRAu0Pkd8n+X5cjlxzb3oBH8EE= 2022-09-03T06:40:51.834 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zpRAu0Pkd8n+X5cjlxzb3oBH8EE=', 'Content-Length': '0'} 2022-09-03T06:40:51.837 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000025e8f2d3a99915d1-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.839 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:40:51.839 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:40:51.840 DEBUG:boto:Method: GET 2022-09-03T06:40:51.841 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:40:51.841 DEBUG:boto:Data: 2022-09-03T06:40:51.842 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.843 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.843 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.844 DEBUG:boto:Params: {} 2022-09-03T06:40:51.845 DEBUG:boto:Token: None 2022-09-03T06:40:51.846 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj51 2022-09-03T06:40:51.847 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zpRAu0Pkd8n+X5cjlxzb3oBH8EE= 2022-09-03T06:40:51.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zpRAu0Pkd8n+X5cjlxzb3oBH8EE=', 'Content-Length': '0'} 2022-09-03T06:40:51.850 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e078f129c661aae1-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.851 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:51.852 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:51.853 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:51.853 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:51.854 DEBUG:boto:Method: GET 2022-09-03T06:40:51.855 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:40:51.856 DEBUG:boto:Data: 2022-09-03T06:40:51.856 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.857 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.857 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.858 DEBUG:boto:Params: {} 2022-09-03T06:40:51.859 DEBUG:boto:Token: None 2022-09-03T06:40:51.860 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/ 2022-09-03T06:40:51.860 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ= 2022-09-03T06:40:51.861 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ=', 'Content-Length': '0'} 2022-09-03T06:40:51.865 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004f4cddcfc1baa46a-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.866 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:51.867 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:51.867 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:51.868 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:51.869 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:40:51.869 DEBUG:boto:Method: GET 2022-09-03T06:40:51.870 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:40:51.871 DEBUG:boto:Data: 2022-09-03T06:40:51.871 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.872 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.873 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.873 DEBUG:boto:Params: {} 2022-09-03T06:40:51.874 DEBUG:boto:Token: None 2022-09-03T06:40:51.875 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/ 2022-09-03T06:40:51.876 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ= 2022-09-03T06:40:51.877 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1XZegLKnxviZf6pd1fv4zzFZUjQ=', 'Content-Length': '0'} 2022-09-03T06:40:51.879 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006b531d338df981b7-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.926 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:51.928 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:40:51.928 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:51.929 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:51.930 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:51.931 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:51.931 DEBUG:boto:Method: GET 2022-09-03T06:40:51.932 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:51.933 DEBUG:boto:Data: 2022-09-03T06:40:51.933 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.934 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:51.935 DEBUG:boto:Port: 8001 2022-09-03T06:40:51.935 DEBUG:boto:Params: {} 2022-09-03T06:40:51.936 DEBUG:boto:Token: None 2022-09-03T06:40:51.937 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:51.937 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2vZcukb4pL/0Er/+62+PuqPLQ1I= 2022-09-03T06:40:51.938 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2vZcukb4pL/0Er/+62+PuqPLQ1I=', 'Content-Length': '0'} 2022-09-03T06:40:51.941 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000950611aa2ecdbf30-006312f6f3-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.941 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:51.942 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:51.943 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:51.943 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:51.944 DEBUG:boto:Method: GET 2022-09-03T06:40:51.945 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:51.945 DEBUG:boto:Data: 2022-09-03T06:40:51.946 DEBUG:boto:Headers: {} 2022-09-03T06:40:51.946 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:51.947 DEBUG:boto:Port: 8000 2022-09-03T06:40:51.948 DEBUG:boto:Params: {} 2022-09-03T06:40:51.949 DEBUG:boto:Token: None 2022-09-03T06:40:51.951 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:51 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:51.952 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2vZcukb4pL/0Er/+62+PuqPLQ1I= 2022-09-03T06:40:51.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2vZcukb4pL/0Er/+62+PuqPLQ1I=', 'Content-Length': '0'} 2022-09-03T06:40:51.959 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007edf438a6c1b2fea-006312f6f3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:51.960 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:51.961 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:51.962 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:51.963 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:51.963 DEBUG:boto:Method: HEAD 2022-09-03T06:40:51.964 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:51.965 DEBUG:boto:Data: 2022-09-03T06:40:51.965 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.050 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:52.051 DEBUG:boto:Port: 8001 2022-09-03T06:40:52.052 DEBUG:boto:Params: {} 2022-09-03T06:40:52.055 DEBUG:boto:Token: None 2022-09-03T06:40:52.057 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:52.060 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c6lgVoAisJXzMVinw+QCs5B+img= 2022-09-03T06:40:52.063 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c6lgVoAisJXzMVinw+QCs5B+img=', 'Content-Length': '0'} 2022-09-03T06:40:52.067 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ae2f1ffb76ffb10e-006312f6f4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.068 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:52.068 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:52.069 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:52.071 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:52.071 DEBUG:boto:Method: HEAD 2022-09-03T06:40:52.072 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:52.072 DEBUG:boto:Data: 2022-09-03T06:40:52.073 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.073 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:52.075 DEBUG:boto:Port: 8000 2022-09-03T06:40:52.075 DEBUG:boto:Params: {} 2022-09-03T06:40:52.076 DEBUG:boto:Token: None 2022-09-03T06:40:52.076 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:52.077 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c6lgVoAisJXzMVinw+QCs5B+img= 2022-09-03T06:40:52.078 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c6lgVoAisJXzMVinw+QCs5B+img=', 'Content-Length': '0'} 2022-09-03T06:40:52.080 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c6938e54e87fffbf-006312f6f4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.085 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:40:52.086 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:52.087 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:52.088 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:52.089 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:52.090 DEBUG:boto:Method: GET 2022-09-03T06:40:52.091 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:52.092 DEBUG:boto:Data: 2022-09-03T06:40:52.094 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.095 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:52.095 DEBUG:boto:Port: 8001 2022-09-03T06:40:52.096 DEBUG:boto:Params: {} 2022-09-03T06:40:52.096 DEBUG:boto:Token: None 2022-09-03T06:40:52.097 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:52.097 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DFVM65CEEVtl2nNX4wICQhhBMPY= 2022-09-03T06:40:52.098 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DFVM65CEEVtl2nNX4wICQhhBMPY=', 'Content-Length': '0'} 2022-09-03T06:40:52.100 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000086fc84bdbcedad47-006312f6f4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.102 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:52.103 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:52.103 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:52.104 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:52.105 DEBUG:boto:Method: GET 2022-09-03T06:40:52.105 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:52.106 DEBUG:boto:Data: 2022-09-03T06:40:52.106 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.107 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:52.107 DEBUG:boto:Port: 8000 2022-09-03T06:40:52.108 DEBUG:boto:Params: {} 2022-09-03T06:40:52.108 DEBUG:boto:Token: None 2022-09-03T06:40:52.109 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:40:52.110 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DFVM65CEEVtl2nNX4wICQhhBMPY= 2022-09-03T06:40:52.110 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DFVM65CEEVtl2nNX4wICQhhBMPY=', 'Content-Length': '0'} 2022-09-03T06:40:52.113 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b13ab56e45e702a-006312f6f4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.114 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:52.115 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:52.115 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:52.116 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:52.116 DEBUG:boto:Method: GET 2022-09-03T06:40:52.117 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:40:52.117 DEBUG:boto:Data: 2022-09-03T06:40:52.118 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.119 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:52.119 DEBUG:boto:Port: 8001 2022-09-03T06:40:52.120 DEBUG:boto:Params: {} 2022-09-03T06:40:52.121 DEBUG:boto:Token: None 2022-09-03T06:40:52.121 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/ 2022-09-03T06:40:52.122 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg= 2022-09-03T06:40:52.123 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg=', 'Content-Length': '0'} 2022-09-03T06:40:52.126 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000081e3f910b45bc10b-006312f6f4-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.127 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:52.129 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:52.130 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:52.131 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:52.132 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:52.133 DEBUG:boto:Method: GET 2022-09-03T06:40:52.134 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:40:52.135 DEBUG:boto:Data: 2022-09-03T06:40:52.136 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.137 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:52.138 DEBUG:boto:Port: 8000 2022-09-03T06:40:52.139 DEBUG:boto:Params: {} 2022-09-03T06:40:52.140 DEBUG:boto:Token: None 2022-09-03T06:40:52.141 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/ 2022-09-03T06:40:52.142 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg= 2022-09-03T06:40:52.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg=', 'Content-Length': '0'} 2022-09-03T06:40:52.147 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cf4a70cf83422f98-006312f6f4-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.153 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:52.154 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:52.154 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:52.155 DEBUG:boto:Method: HEAD 2022-09-03T06:40:52.155 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:40:52.156 DEBUG:boto:Data: 2022-09-03T06:40:52.156 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.157 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:52.157 DEBUG:boto:Port: 8001 2022-09-03T06:40:52.167 DEBUG:boto:Params: {} 2022-09-03T06:40:52.167 DEBUG:boto:Token: None 2022-09-03T06:40:52.168 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj55 2022-09-03T06:40:52.169 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I7KE0UrH2O1MgGV25l5BRAgHy+w= 2022-09-03T06:40:52.170 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I7KE0UrH2O1MgGV25l5BRAgHy+w=', 'Content-Length': '0'} 2022-09-03T06:40:52.172 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cbea42dad1f0c5f5-006312f6f4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.173 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:52.173 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:52.174 DEBUG:boto:Method: HEAD 2022-09-03T06:40:52.174 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:40:52.175 DEBUG:boto:Data: 2022-09-03T06:40:52.176 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.176 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:52.177 DEBUG:boto:Port: 8000 2022-09-03T06:40:52.177 DEBUG:boto:Params: {} 2022-09-03T06:40:52.178 DEBUG:boto:Token: None 2022-09-03T06:40:52.179 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj55 2022-09-03T06:40:52.179 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I7KE0UrH2O1MgGV25l5BRAgHy+w= 2022-09-03T06:40:52.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I7KE0UrH2O1MgGV25l5BRAgHy+w=', 'Content-Length': '0'} 2022-09-03T06:40:52.183 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc499dc059048d97-006312f6f4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.185 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:40:52.186 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:52.187 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:52.188 DEBUG:boto:Method: GET 2022-09-03T06:40:52.190 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:40:52.190 DEBUG:boto:Data: 2022-09-03T06:40:52.191 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.192 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:52.192 DEBUG:boto:Port: 8001 2022-09-03T06:40:52.193 DEBUG:boto:Params: {} 2022-09-03T06:40:52.193 DEBUG:boto:Token: None 2022-09-03T06:40:52.195 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj55 2022-09-03T06:40:52.195 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gf9OwGx5HARw0C9i3NWy+9z/oek= 2022-09-03T06:40:52.196 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gf9OwGx5HARw0C9i3NWy+9z/oek=', 'Content-Length': '0'} 2022-09-03T06:40:52.202 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a3cdff1d431a4a97-006312f6f4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.204 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:40:52.205 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:40:52.205 DEBUG:boto:Method: GET 2022-09-03T06:40:52.206 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:40:52.206 DEBUG:boto:Data: 2022-09-03T06:40:52.207 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.207 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:52.208 DEBUG:boto:Port: 8000 2022-09-03T06:40:52.209 DEBUG:boto:Params: {} 2022-09-03T06:40:52.209 DEBUG:boto:Token: None 2022-09-03T06:40:52.210 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj55 2022-09-03T06:40:52.211 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gf9OwGx5HARw0C9i3NWy+9z/oek= 2022-09-03T06:40:52.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gf9OwGx5HARw0C9i3NWy+9z/oek=', 'Content-Length': '0'} 2022-09-03T06:40:52.215 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c661328514f6056b-006312f6f4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.258 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:52.259 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:52.260 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:52.260 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:52.261 DEBUG:boto:Method: GET 2022-09-03T06:40:52.261 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:40:52.262 DEBUG:boto:Data: 2022-09-03T06:40:52.263 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.263 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:52.264 DEBUG:boto:Port: 8001 2022-09-03T06:40:52.264 DEBUG:boto:Params: {} 2022-09-03T06:40:52.265 DEBUG:boto:Token: None 2022-09-03T06:40:52.265 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/ 2022-09-03T06:40:52.266 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg= 2022-09-03T06:40:52.267 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg=', 'Content-Length': '0'} 2022-09-03T06:40:52.274 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bc4c4d55f9a42b31-006312f6f4-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.276 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:52.277 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:52.278 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:52.279 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:52.279 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:40:52.280 DEBUG:boto:Method: GET 2022-09-03T06:40:52.280 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:40:52.281 DEBUG:boto:Data: 2022-09-03T06:40:52.281 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.282 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:52.282 DEBUG:boto:Port: 8000 2022-09-03T06:40:52.283 DEBUG:boto:Params: {} 2022-09-03T06:40:52.284 DEBUG:boto:Token: None 2022-09-03T06:40:52.284 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/ 2022-09-03T06:40:52.285 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg= 2022-09-03T06:40:52.285 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg=', 'Content-Length': '0'} 2022-09-03T06:40:52.288 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e3ca0d68bd48157e-006312f6f4-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.289 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:52.291 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:40:52.291 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:52.292 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:52.294 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.295 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.297 DEBUG:boto:Method: GET 2022-09-03T06:40:52.298 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.299 DEBUG:boto:Data: 2022-09-03T06:40:52.300 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.301 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:52.301 DEBUG:boto:Port: 8001 2022-09-03T06:40:52.302 DEBUG:boto:Params: {} 2022-09-03T06:40:52.302 DEBUG:boto:Token: None 2022-09-03T06:40:52.303 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.304 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:agfOUWOBmaeRjOYdaxwfq9GyxuA= 2022-09-03T06:40:52.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:agfOUWOBmaeRjOYdaxwfq9GyxuA=', 'Content-Length': '0'} 2022-09-03T06:40:52.307 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f05cf7890d8e7149-006312f6f4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.309 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:52.309 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:52.310 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.311 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.312 DEBUG:boto:Method: GET 2022-09-03T06:40:52.312 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.313 DEBUG:boto:Data: 2022-09-03T06:40:52.313 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.314 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:52.315 DEBUG:boto:Port: 8000 2022-09-03T06:40:52.316 DEBUG:boto:Params: {} 2022-09-03T06:40:52.317 DEBUG:boto:Token: None 2022-09-03T06:40:52.318 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.320 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:agfOUWOBmaeRjOYdaxwfq9GyxuA= 2022-09-03T06:40:52.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:agfOUWOBmaeRjOYdaxwfq9GyxuA=', 'Content-Length': '0'} 2022-09-03T06:40:52.331 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009ea13d2adceb1daf-006312f6f4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.333 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:52.334 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:52.334 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.335 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.335 DEBUG:boto:Method: HEAD 2022-09-03T06:40:52.336 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.336 DEBUG:boto:Data: 2022-09-03T06:40:52.337 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.338 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:52.338 DEBUG:boto:Port: 8001 2022-09-03T06:40:52.339 DEBUG:boto:Params: {} 2022-09-03T06:40:52.340 DEBUG:boto:Token: None 2022-09-03T06:40:52.340 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.341 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+RitssDOe5CND2sl2WLFNTWgCzc= 2022-09-03T06:40:52.342 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+RitssDOe5CND2sl2WLFNTWgCzc=', 'Content-Length': '0'} 2022-09-03T06:40:52.344 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002744a84d7bf8fa7a-006312f6f4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.345 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:52.345 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:52.346 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.347 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.347 DEBUG:boto:Method: HEAD 2022-09-03T06:40:52.348 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.348 DEBUG:boto:Data: 2022-09-03T06:40:52.349 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.349 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:52.350 DEBUG:boto:Port: 8000 2022-09-03T06:40:52.350 DEBUG:boto:Params: {} 2022-09-03T06:40:52.351 DEBUG:boto:Token: None 2022-09-03T06:40:52.351 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.352 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+RitssDOe5CND2sl2WLFNTWgCzc= 2022-09-03T06:40:52.353 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+RitssDOe5CND2sl2WLFNTWgCzc=', 'Content-Length': '0'} 2022-09-03T06:40:52.355 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000907bf64e32e17c4a-006312f6f4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.356 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:40:52.356 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:52.357 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:52.358 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.358 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.359 DEBUG:boto:Method: GET 2022-09-03T06:40:52.360 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.360 DEBUG:boto:Data: 2022-09-03T06:40:52.361 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.362 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:52.363 DEBUG:boto:Port: 8001 2022-09-03T06:40:52.364 DEBUG:boto:Params: {} 2022-09-03T06:40:52.364 DEBUG:boto:Token: None 2022-09-03T06:40:52.365 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:agfOUWOBmaeRjOYdaxwfq9GyxuA= 2022-09-03T06:40:52.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:agfOUWOBmaeRjOYdaxwfq9GyxuA=', 'Content-Length': '0'} 2022-09-03T06:40:52.369 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fe7104e0e22e632a-006312f6f4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.410 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:52.411 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:52.413 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.414 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.415 DEBUG:boto:Method: GET 2022-09-03T06:40:52.415 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.416 DEBUG:boto:Data: 2022-09-03T06:40:52.418 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.418 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:52.419 DEBUG:boto:Port: 8000 2022-09-03T06:40:52.420 DEBUG:boto:Params: {} 2022-09-03T06:40:52.420 DEBUG:boto:Token: None 2022-09-03T06:40:52.421 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:40:52.422 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:agfOUWOBmaeRjOYdaxwfq9GyxuA= 2022-09-03T06:40:52.422 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:agfOUWOBmaeRjOYdaxwfq9GyxuA=', 'Content-Length': '0'} 2022-09-03T06:40:52.426 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000decf6b018bba699d-006312f6f4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.428 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:52.429 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:52.430 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:52.431 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:52.432 DEBUG:boto:Method: GET 2022-09-03T06:40:52.433 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:40:52.434 DEBUG:boto:Data: 2022-09-03T06:40:52.435 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.436 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:52.437 DEBUG:boto:Port: 8001 2022-09-03T06:40:52.438 DEBUG:boto:Params: {} 2022-09-03T06:40:52.439 DEBUG:boto:Token: None 2022-09-03T06:40:52.440 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/ 2022-09-03T06:40:52.441 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg= 2022-09-03T06:40:52.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg=', 'Content-Length': '0'} 2022-09-03T06:40:52.445 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fb8f925f2f6b0247-006312f6f4-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.447 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:52.448 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:52.449 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:52.450 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:52.451 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:52.452 DEBUG:boto:Method: GET 2022-09-03T06:40:52.453 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:40:52.454 DEBUG:boto:Data: 2022-09-03T06:40:52.455 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.456 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:52.457 DEBUG:boto:Port: 8000 2022-09-03T06:40:52.458 DEBUG:boto:Params: {} 2022-09-03T06:40:52.459 DEBUG:boto:Token: None 2022-09-03T06:40:52.460 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/ 2022-09-03T06:40:52.462 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg= 2022-09-03T06:40:52.463 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg=', 'Content-Length': '0'} 2022-09-03T06:40:52.466 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cd896ffc3e0bb628-006312f6f4-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.467 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:52.468 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:52.468 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:52.469 DEBUG:boto:Method: HEAD 2022-09-03T06:40:52.472 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:40:52.475 DEBUG:boto:Data: 2022-09-03T06:40:52.477 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.478 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:52.479 DEBUG:boto:Port: 8001 2022-09-03T06:40:52.481 DEBUG:boto:Params: {} 2022-09-03T06:40:52.483 DEBUG:boto:Token: None 2022-09-03T06:40:52.484 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj68 2022-09-03T06:40:52.484 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r2cjz+M1QC+RNaHoW99e+x4pI5w= 2022-09-03T06:40:52.485 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r2cjz+M1QC+RNaHoW99e+x4pI5w=', 'Content-Length': '0'} 2022-09-03T06:40:52.489 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a5f770919823b5fc-006312f6f4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.490 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:52.491 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:52.493 DEBUG:boto:Method: HEAD 2022-09-03T06:40:52.494 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:40:52.494 DEBUG:boto:Data: 2022-09-03T06:40:52.495 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.495 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:52.496 DEBUG:boto:Port: 8000 2022-09-03T06:40:52.496 DEBUG:boto:Params: {} 2022-09-03T06:40:52.497 DEBUG:boto:Token: None 2022-09-03T06:40:52.497 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj68 2022-09-03T06:40:52.498 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r2cjz+M1QC+RNaHoW99e+x4pI5w= 2022-09-03T06:40:52.499 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r2cjz+M1QC+RNaHoW99e+x4pI5w=', 'Content-Length': '0'} 2022-09-03T06:40:52.502 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca7cf0b8f74fa0aa-006312f6f4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.503 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:40:52.504 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:52.504 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:52.505 DEBUG:boto:Method: GET 2022-09-03T06:40:52.506 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:40:52.507 DEBUG:boto:Data: 2022-09-03T06:40:52.507 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.508 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:52.509 DEBUG:boto:Port: 8001 2022-09-03T06:40:52.509 DEBUG:boto:Params: {} 2022-09-03T06:40:52.510 DEBUG:boto:Token: None 2022-09-03T06:40:52.511 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj68 2022-09-03T06:40:52.513 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0TbLV5ZDQtxvqE6YpmAcEakRHLo= 2022-09-03T06:40:52.514 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0TbLV5ZDQtxvqE6YpmAcEakRHLo=', 'Content-Length': '0'} 2022-09-03T06:40:52.518 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000b1039598009ad5e-006312f6f4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.519 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:40:52.520 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:40:52.522 DEBUG:boto:Method: GET 2022-09-03T06:40:52.523 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:40:52.524 DEBUG:boto:Data: 2022-09-03T06:40:52.525 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.525 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:52.526 DEBUG:boto:Port: 8000 2022-09-03T06:40:52.527 DEBUG:boto:Params: {} 2022-09-03T06:40:52.528 DEBUG:boto:Token: None 2022-09-03T06:40:52.529 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj68 2022-09-03T06:40:52.530 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0TbLV5ZDQtxvqE6YpmAcEakRHLo= 2022-09-03T06:40:52.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0TbLV5ZDQtxvqE6YpmAcEakRHLo=', 'Content-Length': '0'} 2022-09-03T06:40:52.535 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000069a39f6343dd7037-006312f6f4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.537 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:52.538 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:52.539 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:52.540 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:52.541 DEBUG:boto:Method: GET 2022-09-03T06:40:52.542 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:40:52.543 DEBUG:boto:Data: 2022-09-03T06:40:52.544 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.545 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:52.546 DEBUG:boto:Port: 8001 2022-09-03T06:40:52.547 DEBUG:boto:Params: {} 2022-09-03T06:40:52.548 DEBUG:boto:Token: None 2022-09-03T06:40:52.549 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/ 2022-09-03T06:40:52.550 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg= 2022-09-03T06:40:52.551 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg=', 'Content-Length': '0'} 2022-09-03T06:40:52.554 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c0bfe34e68cbf4be-006312f6f4-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.554 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:52.557 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:52.557 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:52.558 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:52.559 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:40:52.560 DEBUG:boto:Method: GET 2022-09-03T06:40:52.560 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:40:52.562 DEBUG:boto:Data: 2022-09-03T06:40:52.563 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.565 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:52.566 DEBUG:boto:Port: 8000 2022-09-03T06:40:52.566 DEBUG:boto:Params: {} 2022-09-03T06:40:52.568 DEBUG:boto:Token: None 2022-09-03T06:40:52.569 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/ 2022-09-03T06:40:52.570 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg= 2022-09-03T06:40:52.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg=', 'Content-Length': '0'} 2022-09-03T06:40:52.578 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000067917aa861d360af-006312f6f4-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.579 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:52.580 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:40:52.581 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:52.581 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:52.582 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.582 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.583 DEBUG:boto:Method: GET 2022-09-03T06:40:52.583 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.584 DEBUG:boto:Data: 2022-09-03T06:40:52.585 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.585 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:52.586 DEBUG:boto:Port: 8001 2022-09-03T06:40:52.586 DEBUG:boto:Params: {} 2022-09-03T06:40:52.587 DEBUG:boto:Token: None 2022-09-03T06:40:52.588 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.588 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ignbjVI/6TMT/AlQHONOAz1Lh7w= 2022-09-03T06:40:52.589 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ignbjVI/6TMT/AlQHONOAz1Lh7w=', 'Content-Length': '0'} 2022-09-03T06:40:52.591 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e13aa01a1b04635-006312f6f4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.592 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:52.593 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:52.593 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.594 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.594 DEBUG:boto:Method: GET 2022-09-03T06:40:52.595 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.595 DEBUG:boto:Data: 2022-09-03T06:40:52.596 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.596 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:52.597 DEBUG:boto:Port: 8000 2022-09-03T06:40:52.598 DEBUG:boto:Params: {} 2022-09-03T06:40:52.598 DEBUG:boto:Token: None 2022-09-03T06:40:52.599 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.600 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ignbjVI/6TMT/AlQHONOAz1Lh7w= 2022-09-03T06:40:52.600 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ignbjVI/6TMT/AlQHONOAz1Lh7w=', 'Content-Length': '0'} 2022-09-03T06:40:52.602 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000028688f53af91d561-006312f6f4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.650 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:52.650 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:52.651 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.651 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.652 DEBUG:boto:Method: HEAD 2022-09-03T06:40:52.652 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.653 DEBUG:boto:Data: 2022-09-03T06:40:52.653 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.654 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:52.655 DEBUG:boto:Port: 8001 2022-09-03T06:40:52.655 DEBUG:boto:Params: {} 2022-09-03T06:40:52.656 DEBUG:boto:Token: None 2022-09-03T06:40:52.657 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.658 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dDOpRA2pBMRQ7evfXRfVxhk0WTM= 2022-09-03T06:40:52.659 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dDOpRA2pBMRQ7evfXRfVxhk0WTM=', 'Content-Length': '0'} 2022-09-03T06:40:52.662 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004e96dee5ebad77b1-006312f6f4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.663 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:52.664 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:52.664 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.665 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.666 DEBUG:boto:Method: HEAD 2022-09-03T06:40:52.668 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.669 DEBUG:boto:Data: 2022-09-03T06:40:52.670 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.671 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:52.672 DEBUG:boto:Port: 8000 2022-09-03T06:40:52.673 DEBUG:boto:Params: {} 2022-09-03T06:40:52.674 DEBUG:boto:Token: None 2022-09-03T06:40:52.676 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.676 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dDOpRA2pBMRQ7evfXRfVxhk0WTM= 2022-09-03T06:40:52.677 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dDOpRA2pBMRQ7evfXRfVxhk0WTM=', 'Content-Length': '0'} 2022-09-03T06:40:52.679 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003d6aedc7f896d61b-006312f6f4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.680 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:40:52.680 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:52.681 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:52.682 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.682 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.683 DEBUG:boto:Method: GET 2022-09-03T06:40:52.683 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.684 DEBUG:boto:Data: 2022-09-03T06:40:52.685 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.685 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:52.686 DEBUG:boto:Port: 8001 2022-09-03T06:40:52.686 DEBUG:boto:Params: {} 2022-09-03T06:40:52.687 DEBUG:boto:Token: None 2022-09-03T06:40:52.688 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.689 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ignbjVI/6TMT/AlQHONOAz1Lh7w= 2022-09-03T06:40:52.689 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ignbjVI/6TMT/AlQHONOAz1Lh7w=', 'Content-Length': '0'} 2022-09-03T06:40:52.691 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021a0bdcaf9966378-006312f6f4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.692 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:52.693 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:52.693 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.694 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.695 DEBUG:boto:Method: GET 2022-09-03T06:40:52.695 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.696 DEBUG:boto:Data: 2022-09-03T06:40:52.696 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.697 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:52.697 DEBUG:boto:Port: 8000 2022-09-03T06:40:52.698 DEBUG:boto:Params: {} 2022-09-03T06:40:52.699 DEBUG:boto:Token: None 2022-09-03T06:40:52.699 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:40:52.700 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ignbjVI/6TMT/AlQHONOAz1Lh7w= 2022-09-03T06:40:52.701 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ignbjVI/6TMT/AlQHONOAz1Lh7w=', 'Content-Length': '0'} 2022-09-03T06:40:52.703 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003320127ab0babea5-006312f6f4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.704 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:52.704 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:52.705 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:52.705 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:52.706 DEBUG:boto:Method: GET 2022-09-03T06:40:52.707 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:40:52.707 DEBUG:boto:Data: 2022-09-03T06:40:52.708 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.709 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:52.709 DEBUG:boto:Port: 8001 2022-09-03T06:40:52.710 DEBUG:boto:Params: {} 2022-09-03T06:40:52.710 DEBUG:boto:Token: None 2022-09-03T06:40:52.711 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/ 2022-09-03T06:40:52.712 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg= 2022-09-03T06:40:52.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg=', 'Content-Length': '0'} 2022-09-03T06:40:52.715 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ef0678a0679cc735-006312f6f4-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.716 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:52.717 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:52.719 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:52.720 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:52.721 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:52.722 DEBUG:boto:Method: GET 2022-09-03T06:40:52.723 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:40:52.724 DEBUG:boto:Data: 2022-09-03T06:40:52.724 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.725 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:52.726 DEBUG:boto:Port: 8000 2022-09-03T06:40:52.727 DEBUG:boto:Params: {} 2022-09-03T06:40:52.728 DEBUG:boto:Token: None 2022-09-03T06:40:52.729 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/ 2022-09-03T06:40:52.730 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg= 2022-09-03T06:40:52.731 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg=', 'Content-Length': '0'} 2022-09-03T06:40:52.735 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d621d9f438548a4f-006312f6f4-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.778 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:52.780 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:52.781 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:52.781 DEBUG:boto:Method: HEAD 2022-09-03T06:40:52.782 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:40:52.782 DEBUG:boto:Data: 2022-09-03T06:40:52.783 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.784 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:52.785 DEBUG:boto:Port: 8001 2022-09-03T06:40:52.786 DEBUG:boto:Params: {} 2022-09-03T06:40:52.787 DEBUG:boto:Token: None 2022-09-03T06:40:52.789 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj85 2022-09-03T06:40:52.790 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oUKoVoi6q50MHgC2CWWiiK7zCQo= 2022-09-03T06:40:52.791 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oUKoVoi6q50MHgC2CWWiiK7zCQo=', 'Content-Length': '0'} 2022-09-03T06:40:52.794 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e7d9f1558b8c01c-006312f6f4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.795 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:52.796 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:52.797 DEBUG:boto:Method: HEAD 2022-09-03T06:40:52.798 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:40:52.799 DEBUG:boto:Data: 2022-09-03T06:40:52.800 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.801 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:52.802 DEBUG:boto:Port: 8000 2022-09-03T06:40:52.803 DEBUG:boto:Params: {} 2022-09-03T06:40:52.804 DEBUG:boto:Token: None 2022-09-03T06:40:52.805 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj85 2022-09-03T06:40:52.806 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oUKoVoi6q50MHgC2CWWiiK7zCQo= 2022-09-03T06:40:52.807 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oUKoVoi6q50MHgC2CWWiiK7zCQo=', 'Content-Length': '0'} 2022-09-03T06:40:52.810 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000004e6e4d191d59fbc-006312f6f4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.811 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:40:52.812 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:52.813 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:52.814 DEBUG:boto:Method: GET 2022-09-03T06:40:52.816 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:40:52.817 DEBUG:boto:Data: 2022-09-03T06:40:52.818 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.819 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:52.819 DEBUG:boto:Port: 8001 2022-09-03T06:40:52.820 DEBUG:boto:Params: {} 2022-09-03T06:40:52.821 DEBUG:boto:Token: None 2022-09-03T06:40:52.822 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj85 2022-09-03T06:40:52.822 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t0AaW1xfynqjCxyg/Ix10TNgQZw= 2022-09-03T06:40:52.823 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t0AaW1xfynqjCxyg/Ix10TNgQZw=', 'Content-Length': '0'} 2022-09-03T06:40:52.826 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aef40816f2b0dc4b-006312f6f4-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.827 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:40:52.828 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:40:52.828 DEBUG:boto:Method: GET 2022-09-03T06:40:52.829 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:40:52.830 DEBUG:boto:Data: 2022-09-03T06:40:52.830 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.831 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:52.831 DEBUG:boto:Port: 8000 2022-09-03T06:40:52.832 DEBUG:boto:Params: {} 2022-09-03T06:40:52.833 DEBUG:boto:Token: None 2022-09-03T06:40:52.833 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/obj85 2022-09-03T06:40:52.835 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t0AaW1xfynqjCxyg/Ix10TNgQZw= 2022-09-03T06:40:52.835 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t0AaW1xfynqjCxyg/Ix10TNgQZw=', 'Content-Length': '0'} 2022-09-03T06:40:52.837 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000010593dba64652a5-006312f6f4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.843 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:52.844 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:52.845 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:52.846 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:52.847 DEBUG:boto:Method: GET 2022-09-03T06:40:52.848 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:40:52.849 DEBUG:boto:Data: 2022-09-03T06:40:52.850 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.851 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:40:52.852 DEBUG:boto:Port: 8001 2022-09-03T06:40:52.854 DEBUG:boto:Params: {} 2022-09-03T06:40:52.855 DEBUG:boto:Token: None 2022-09-03T06:40:52.856 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/ 2022-09-03T06:40:52.857 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg= 2022-09-03T06:40:52.858 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg=', 'Content-Length': '0'} 2022-09-03T06:40:52.862 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d1b161dae25cc1e2-006312f6f4-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.864 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:52.866 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:52.867 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:52.868 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:52.868 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:40:52.869 DEBUG:boto:Method: GET 2022-09-03T06:40:52.870 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:40:52.871 DEBUG:boto:Data: 2022-09-03T06:40:52.872 DEBUG:boto:Headers: {} 2022-09-03T06:40:52.873 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:40:52.875 DEBUG:boto:Port: 8000 2022-09-03T06:40:52.876 DEBUG:boto:Params: {} 2022-09-03T06:40:52.877 DEBUG:boto:Token: None 2022-09-03T06:40:52.879 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:40:52 GMT /dsgvcy-75/ 2022-09-03T06:40:52.880 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg= 2022-09-03T06:40:52.881 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fMtiGEczNc2pzKl38ocoui8wYcg=', 'Content-Length': '0'} 2022-09-03T06:40:52.890 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a9276d8d9277ce19-006312f6f4-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:40:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:52.891 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:40:52.893 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-75 zones={test-zone3, test-zone2} 2022-09-03T06:40:52.893 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--num-shards', '19', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:40:52.894 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--num-shards', '19', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:40:52.895 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --num-shards 19 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T06:40:52.955 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:40:52.956 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:40:53.026 INFO:teuthology.orchestra.run.smithi181.stdout:tenant: 2022-09-03T06:40:53.027 INFO:teuthology.orchestra.run.smithi181.stdout:bucket name: dsgvcy-75 2022-09-03T06:40:53.034 INFO:teuthology.orchestra.run.smithi181.stdout:total entries: 10 2022-09-03T06:40:53.084 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:53.086 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:53.087 DEBUG:boto:Method: HEAD 2022-09-03T06:40:53.088 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:40:53.089 DEBUG:boto:Data: 2022-09-03T06:40:53.091 DEBUG:boto:Headers: {} 2022-09-03T06:40:53.092 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:53.093 DEBUG:boto:Port: 8000 2022-09-03T06:40:53.099 DEBUG:boto:Params: {} 2022-09-03T06:40:53.100 DEBUG:boto:Token: None 2022-09-03T06:40:53.101 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:53 GMT /dsgvcy-75/ 2022-09-03T06:40:53.103 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Qn85RNFBUAbPDz3XSzppljNE514= 2022-09-03T06:40:53.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Qn85RNFBUAbPDz3XSzppljNE514=', 'Content-Length': '0'} 2022-09-03T06:40:53.108 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '10'), ('X-RGW-Bytes-Used', '30'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000725c6705d76f9748-006312f6f5-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:53.110 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:40:53.111 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:40:53.112 DEBUG:boto:Method: PUT 2022-09-03T06:40:53.112 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:40:53.113 DEBUG:boto:Data: 2022-09-03T06:40:53.114 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:40:53.114 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:53.115 DEBUG:boto:Port: 8000 2022-09-03T06:40:53.116 DEBUG:boto:Params: {} 2022-09-03T06:40:53.117 DEBUG:boto:Token: None 2022-09-03T06:40:53.122 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:40:53 GMT /dsgvcy-75/obj19 2022-09-03T06:40:53.123 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hanzKndRHK47xsdGszU1ggsFaVA= 2022-09-03T06:40:53.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:40:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hanzKndRHK47xsdGszU1ggsFaVA='} 2022-09-03T06:40:53.135 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000007a47315ec6d9a821-006312f6f5-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:53.136 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:53.138 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:53.139 DEBUG:boto:Method: HEAD 2022-09-03T06:40:53.140 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:40:53.141 DEBUG:boto:Data: 2022-09-03T06:40:53.142 DEBUG:boto:Headers: {} 2022-09-03T06:40:53.143 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:53.144 DEBUG:boto:Port: 8000 2022-09-03T06:40:53.144 DEBUG:boto:Params: {} 2022-09-03T06:40:53.145 DEBUG:boto:Token: None 2022-09-03T06:40:53.146 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:53 GMT /dsgvcy-75/ 2022-09-03T06:40:53.146 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Qn85RNFBUAbPDz3XSzppljNE514= 2022-09-03T06:40:53.147 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Qn85RNFBUAbPDz3XSzppljNE514=', 'Content-Length': '0'} 2022-09-03T06:40:53.154 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '11'), ('X-RGW-Bytes-Used', '33'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e3c4456e4647178f-006312f6f5-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:53.156 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:40:53.157 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:40:53.158 DEBUG:boto:Method: PUT 2022-09-03T06:40:53.159 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:40:53.160 DEBUG:boto:Data: 2022-09-03T06:40:53.160 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:40:53.163 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:53.164 DEBUG:boto:Port: 8000 2022-09-03T06:40:53.165 DEBUG:boto:Params: {} 2022-09-03T06:40:53.166 DEBUG:boto:Token: None 2022-09-03T06:40:53.167 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:40:53 GMT /dsgvcy-75/obj38 2022-09-03T06:40:53.168 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B38iGouwGdmCKUL6TccWngC1tAI= 2022-09-03T06:40:53.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:40:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B38iGouwGdmCKUL6TccWngC1tAI='} 2022-09-03T06:40:53.236 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f3aeed7174158366-006312f6f5-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:53.238 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:53.239 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:53.240 DEBUG:boto:Method: HEAD 2022-09-03T06:40:53.242 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:40:53.243 DEBUG:boto:Data: 2022-09-03T06:40:53.253 DEBUG:boto:Headers: {} 2022-09-03T06:40:53.254 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:53.255 DEBUG:boto:Port: 8000 2022-09-03T06:40:53.256 DEBUG:boto:Params: {} 2022-09-03T06:40:53.258 DEBUG:boto:Token: None 2022-09-03T06:40:53.274 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:53 GMT /dsgvcy-75/ 2022-09-03T06:40:53.275 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Qn85RNFBUAbPDz3XSzppljNE514= 2022-09-03T06:40:53.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Qn85RNFBUAbPDz3XSzppljNE514=', 'Content-Length': '0'} 2022-09-03T06:40:53.279 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '12'), ('X-RGW-Bytes-Used', '36'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004649b1ad073a31b3-006312f6f5-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:53.280 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:40:53.281 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:40:53.282 DEBUG:boto:Method: PUT 2022-09-03T06:40:53.282 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:40:53.283 DEBUG:boto:Data: 2022-09-03T06:40:53.283 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:40:53.284 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:53.284 DEBUG:boto:Port: 8000 2022-09-03T06:40:53.285 DEBUG:boto:Params: {} 2022-09-03T06:40:53.285 DEBUG:boto:Token: None 2022-09-03T06:40:53.289 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:40:53 GMT /dsgvcy-75/obj57 2022-09-03T06:40:53.290 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KMdXG2TPkSgKNKua127FyuNGjAk= 2022-09-03T06:40:53.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:40:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KMdXG2TPkSgKNKua127FyuNGjAk='} 2022-09-03T06:40:53.298 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000000633c456486f0ade-006312f6f5-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:53.299 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:53.300 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:53.301 DEBUG:boto:Method: HEAD 2022-09-03T06:40:53.302 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:40:53.302 DEBUG:boto:Data: 2022-09-03T06:40:53.303 DEBUG:boto:Headers: {} 2022-09-03T06:40:53.305 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:53.307 DEBUG:boto:Port: 8000 2022-09-03T06:40:53.308 DEBUG:boto:Params: {} 2022-09-03T06:40:53.309 DEBUG:boto:Token: None 2022-09-03T06:40:53.310 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:53 GMT /dsgvcy-75/ 2022-09-03T06:40:53.311 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Qn85RNFBUAbPDz3XSzppljNE514= 2022-09-03T06:40:53.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Qn85RNFBUAbPDz3XSzppljNE514=', 'Content-Length': '0'} 2022-09-03T06:40:53.316 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '13'), ('X-RGW-Bytes-Used', '39'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000007cdd2b71452d78d-006312f6f5-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:53.318 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:40:53.319 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:40:53.320 DEBUG:boto:Method: PUT 2022-09-03T06:40:53.321 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:40:53.322 DEBUG:boto:Data: 2022-09-03T06:40:53.323 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:40:53.323 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:53.325 DEBUG:boto:Port: 8000 2022-09-03T06:40:53.326 DEBUG:boto:Params: {} 2022-09-03T06:40:53.327 DEBUG:boto:Token: None 2022-09-03T06:40:53.328 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:40:53 GMT /dsgvcy-75/obj76 2022-09-03T06:40:53.329 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7mxnmBaAgTbT853srzn7AWa+OIA= 2022-09-03T06:40:53.330 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:40:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7mxnmBaAgTbT853srzn7AWa+OIA='} 2022-09-03T06:40:53.379 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000a80a245243872f41-006312f6f5-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:53.380 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:40:53.381 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:40:53.381 DEBUG:boto:Method: HEAD 2022-09-03T06:40:53.382 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:40:53.383 DEBUG:boto:Data: 2022-09-03T06:40:53.383 DEBUG:boto:Headers: {} 2022-09-03T06:40:53.384 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:53.388 DEBUG:boto:Port: 8000 2022-09-03T06:40:53.389 DEBUG:boto:Params: {} 2022-09-03T06:40:53.390 DEBUG:boto:Token: None 2022-09-03T06:40:53.392 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:40:53 GMT /dsgvcy-75/ 2022-09-03T06:40:53.393 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Qn85RNFBUAbPDz3XSzppljNE514= 2022-09-03T06:40:53.394 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:40:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Qn85RNFBUAbPDz3XSzppljNE514=', 'Content-Length': '0'} 2022-09-03T06:40:53.398 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '14'), ('X-RGW-Bytes-Used', '42'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d045e89ffa42845f-006312f6f5-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:53.400 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:40:53.401 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:40:53.402 DEBUG:boto:Method: PUT 2022-09-03T06:40:53.403 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:40:53.404 DEBUG:boto:Data: 2022-09-03T06:40:53.405 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:40:53.406 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:40:53.409 DEBUG:boto:Port: 8000 2022-09-03T06:40:53.410 DEBUG:boto:Params: {} 2022-09-03T06:40:53.411 DEBUG:boto:Token: None 2022-09-03T06:40:53.412 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:40:53 GMT /dsgvcy-75/obj95 2022-09-03T06:40:53.413 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6cDV+VwRltCd0+9lh0QZhBOOjtg= 2022-09-03T06:40:53.413 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:40:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6cDV+VwRltCd0+9lh0QZhBOOjtg='} 2022-09-03T06:40:53.418 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000034489c5b4500bde0-006312f6f5-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:40:53.419 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-75 2022-09-03T06:40:53.419 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:40:53.420 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:40:53.420 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:40:53.488 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:40:53.488 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:40:53.533 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:40:53.531+0000 7fa07ebbc600 1 waiting to reach latest gen 2, on 1.. 2022-09-03T06:40:58.535 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:40:58.533+0000 7fa07ebbc600 1 waiting for incremental sync to catch up: 2022-09-03T06:40:58.536 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , , , ] 2022-09-03T06:40:58.536 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.75.5, 00000000004.69.5, , , , , , , , , , , ] 2022-09-03T06:41:03.537 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:41:03.535+0000 7fa07ebbc600 1 waiting for incremental sync to catch up: 2022-09-03T06:41:03.538 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , , , ] 2022-09-03T06:41:03.538 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.75.5, 00000000004.69.5, , , , , , , , , , , ] 2022-09-03T06:41:08.539 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:41:08.537+0000 7fa07ebbc600 1 waiting for incremental sync to catch up: 2022-09-03T06:41:08.539 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , , , ] 2022-09-03T06:41:08.540 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.75.5, 00000000004.69.5, , , , , , , , , , , ] 2022-09-03T06:41:13.541 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:41:13.539+0000 7fa07ebbc600 1 waiting for incremental sync to catch up: 2022-09-03T06:41:13.541 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , , , ] 2022-09-03T06:41:13.541 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.75.5, 00000000004.69.5, , , , , , , , , , , ] 2022-09-03T06:41:18.542 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:41:18.540+0000 7fa07ebbc600 1 bucket sync caught up with source: 2022-09-03T06:41:18.543 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000002.75.5, 00000000004.69.5, , , , , , , , , , , ] 2022-09-03T06:41:18.544 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.75.5, 00000000004.69.5, , , , , , , , , , , ] 2022-09-03T06:41:18.544 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:41:18.540+0000 7fa07ebbc600 0 bucket checkpoint complete 2022-09-03T06:41:18.562 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-75 2022-09-03T06:41:18.563 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:41:18.564 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:41:18.565 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:41:18.629 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:41:18.629 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:41:18.670 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:41:18.667+0000 7fcb19196600 1 waiting for incremental sync to catch up: 2022-09-03T06:41:18.671 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , , , , , , , , , , , , , ] 2022-09-03T06:41:18.671 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , 00000000002.75.5, 00000000004.69.5, , , , , , , , , , , ] 2022-09-03T06:41:23.672 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:41:23.669+0000 7fcb19196600 1 bucket sync caught up with source: 2022-09-03T06:41:23.673 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , 00000000002.75.5, 00000000004.69.5, , , , , , , , , , , ] 2022-09-03T06:41:23.673 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , 00000000002.75.5, 00000000004.69.5, , , , , , , , , , , ] 2022-09-03T06:41:23.673 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:41:23.669+0000 7fcb19196600 0 bucket checkpoint complete 2022-09-03T06:41:23.692 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-75 2022-09-03T06:41:23.693 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:41:23.694 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:41:23.695 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:41:23.759 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:41:23.761 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:41:23.814 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:41:23.811+0000 7fe7febfb600 1 waiting for incremental sync to catch up: 2022-09-03T06:41:23.814 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000002.75.5, , , , , , , , , , , , ] 2022-09-03T06:41:23.814 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.75.5, 00000000004.69.5, , , , , , , , , , , ] 2022-09-03T06:41:28.816 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:41:28.814+0000 7fe7febfb600 1 waiting for incremental sync to catch up: 2022-09-03T06:41:28.817 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000002.75.5, , , , , , , , , , , , ] 2022-09-03T06:41:28.817 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.75.5, 00000000004.69.5, , , , , , , , , , , ] 2022-09-03T06:41:33.817 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:41:33.815+0000 7fe7febfb600 1 waiting for incremental sync to catch up: 2022-09-03T06:41:33.818 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000002.75.5, , , , , , , , , , , , ] 2022-09-03T06:41:33.818 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.75.5, 00000000004.69.5, , , , , , , , , , , ] 2022-09-03T06:41:38.820 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:41:38.817+0000 7fe7febfb600 1 waiting for incremental sync to catch up: 2022-09-03T06:41:38.821 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000002.75.5, , , , , , , , , , , , ] 2022-09-03T06:41:38.821 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.75.5, 00000000004.69.5, , , , , , , , , , , ] 2022-09-03T06:41:43.824 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:41:43.819+0000 7fe7febfb600 1 bucket sync caught up with source: 2022-09-03T06:41:43.824 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000002.75.5, 00000000004.69.5, , , , , , , , , , , ] 2022-09-03T06:41:43.824 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.75.5, 00000000004.69.5, , , , , , , , , , , ] 2022-09-03T06:41:43.825 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:41:43.819+0000 7fe7febfb600 0 bucket checkpoint complete 2022-09-03T06:41:43.843 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-75 2022-09-03T06:41:43.844 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:41:43.845 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:41:43.845 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:41:43.913 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:41:43.913 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:41:43.954 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:41:43.952+0000 7f680e003600 1 bucket sync caught up with source: 2022-09-03T06:41:43.954 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.582.5, 00000000001.731.5, 00000000001.366.5, 00000000001.703.5, 00000000001.356.5, , , 00000000002.855.5, 00000000004.308.5, 00000000003.579.5] 2022-09-03T06:41:43.956 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.582.5, 00000000001.731.5, 00000000001.366.5, 00000000001.703.5, 00000000001.356.5, , , 00000000002.855.5, 00000000004.308.5, 00000000003.579.5] 2022-09-03T06:41:43.957 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:41:43.952+0000 7f680e003600 0 bucket checkpoint complete 2022-09-03T06:41:43.973 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-75 2022-09-03T06:41:43.974 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:41:43.975 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:41:43.976 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:41:44.036 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:41:44.037 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:41:44.076 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:41:44.074+0000 7f2eae3d0600 1 bucket sync caught up with source: 2022-09-03T06:41:44.076 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.582.5, 00000000001.731.5, 00000000001.366.5, 00000000001.703.5, 00000000001.356.5, , , 00000000002.855.5, 00000000004.308.5, 00000000003.579.5] 2022-09-03T06:41:44.077 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.582.5, 00000000001.731.5, 00000000001.366.5, 00000000001.703.5, 00000000001.356.5, , , 00000000002.855.5, 00000000004.308.5, 00000000003.579.5] 2022-09-03T06:41:44.077 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:41:44.074+0000 7f2eae3d0600 0 bucket checkpoint complete 2022-09-03T06:41:44.091 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-75 2022-09-03T06:41:44.092 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-75 2022-09-03T06:41:44.092 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:41:44.093 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:41:44.094 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:41:44.149 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:41:44.150 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:41:44.189 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:41:44.187+0000 7fba0c4ea600 1 bucket sync caught up with source: 2022-09-03T06:41:44.190 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.202.5, 00000000001.151.5, 00000000001.271.5, 00000000001.264.5, 00000000001.265.5, , , 00000000002.317.5, 00000000004.243.5, 00000000003.284.5] 2022-09-03T06:41:44.190 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.202.5, 00000000001.151.5, 00000000001.271.5, 00000000001.264.5, 00000000001.265.5, , , 00000000002.317.5, 00000000004.243.5, 00000000003.284.5] 2022-09-03T06:41:44.191 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:41:44.187+0000 7fba0c4ea600 0 bucket checkpoint complete 2022-09-03T06:41:44.205 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-75 2022-09-03T06:41:44.206 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:41:44.207 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:41:44.207 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:41:44.273 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:41:44.273 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:41:44.322 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:41:44.320+0000 7fa46cdec600 1 bucket sync caught up with source: 2022-09-03T06:41:44.323 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000002.202.5, 00000000001.151.5, 00000000001.271.5, 00000000001.264.5, 00000000001.265.5, , , 00000000002.317.5, 00000000004.243.5, 00000000003.284.5] 2022-09-03T06:41:44.324 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000002.202.5, 00000000001.151.5, 00000000001.271.5, 00000000001.264.5, 00000000001.265.5, , , 00000000002.317.5, 00000000004.243.5, 00000000003.284.5] 2022-09-03T06:41:44.324 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:41:44.320+0000 7fa46cdec600 0 bucket checkpoint complete 2022-09-03T06:41:44.341 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-75 2022-09-03T06:41:44.343 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-75 zones={test-zone2, test-zone1} 2022-09-03T06:41:44.344 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:44.345 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:44.346 DEBUG:boto:Method: HEAD 2022-09-03T06:41:44.348 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:41:44.349 DEBUG:boto:Data: 2022-09-03T06:41:44.349 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.350 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:44.350 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.351 DEBUG:boto:Params: {} 2022-09-03T06:41:44.352 DEBUG:boto:Token: None 2022-09-03T06:41:44.353 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/ 2022-09-03T06:41:44.354 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KTmG1FG3PWOvc/dgTc5ow5w3OYg= 2022-09-03T06:41:44.354 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KTmG1FG3PWOvc/dgTc5ow5w3OYg=', 'Content-Length': '0'} 2022-09-03T06:41:44.357 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '15'), ('X-RGW-Bytes-Used', '45'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000082b1da404a1a2819-006312f728-188c-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.359 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:44.359 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:44.360 DEBUG:boto:Method: HEAD 2022-09-03T06:41:44.360 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:41:44.361 DEBUG:boto:Data: 2022-09-03T06:41:44.361 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.362 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:44.362 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.363 DEBUG:boto:Params: {} 2022-09-03T06:41:44.364 DEBUG:boto:Token: None 2022-09-03T06:41:44.365 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/ 2022-09-03T06:41:44.365 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KTmG1FG3PWOvc/dgTc5ow5w3OYg= 2022-09-03T06:41:44.366 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KTmG1FG3PWOvc/dgTc5ow5w3OYg=', 'Content-Length': '0'} 2022-09-03T06:41:44.369 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '15'), ('X-RGW-Bytes-Used', '45'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005ece1f49e199513b-006312f728-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.371 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:41:44.372 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:44.373 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:44.375 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:44.376 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:44.377 DEBUG:boto:Method: GET 2022-09-03T06:41:44.378 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:44.379 DEBUG:boto:Data: 2022-09-03T06:41:44.380 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.381 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:44.382 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.383 DEBUG:boto:Params: {} 2022-09-03T06:41:44.385 DEBUG:boto:Token: None 2022-09-03T06:41:44.386 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/?versions 2022-09-03T06:41:44.387 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KMDGZ33FzUHfCfolhLttYxeuAB0= 2022-09-03T06:41:44.388 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KMDGZ33FzUHfCfolhLttYxeuAB0=', 'Content-Length': '0'} 2022-09-03T06:41:44.391 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000043cf1dcb161ec9d4-006312f728-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.392 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:44.396 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:41:44.397 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:41:44.398 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:41:44.399 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:41:44.400 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:41:44.401 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:41:44.402 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:41:44.403 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:41:44.404 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:41:44.404 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:41:44.405 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:41:44.406 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:41:44.407 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:41:44.408 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:41:44.409 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:41:44.410 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:41:44.411 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:44.412 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:44.412 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:44.413 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:44.413 DEBUG:boto:Method: GET 2022-09-03T06:41:44.414 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:44.414 DEBUG:boto:Data: 2022-09-03T06:41:44.415 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.415 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:44.416 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.416 DEBUG:boto:Params: {} 2022-09-03T06:41:44.417 DEBUG:boto:Token: None 2022-09-03T06:41:44.418 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/?versions 2022-09-03T06:41:44.418 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KMDGZ33FzUHfCfolhLttYxeuAB0= 2022-09-03T06:41:44.419 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KMDGZ33FzUHfCfolhLttYxeuAB0=', 'Content-Length': '0'} 2022-09-03T06:41:44.422 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000023f339f15af5bad5-006312f728-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.423 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:44.424 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:41:44.425 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:41:44.425 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:41:44.426 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:41:44.427 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:41:44.428 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:41:44.428 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:41:44.429 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:41:44.429 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:41:44.430 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:41:44.430 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:41:44.431 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:41:44.431 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:41:44.432 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:41:44.432 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:41:44.445 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:44.446 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:44.446 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:44.447 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:44.447 DEBUG:boto:Method: GET 2022-09-03T06:41:44.448 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:44.448 DEBUG:boto:Data: 2022-09-03T06:41:44.449 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.449 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:44.450 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.450 DEBUG:boto:Params: {} 2022-09-03T06:41:44.451 DEBUG:boto:Token: None 2022-09-03T06:41:44.452 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/?versions 2022-09-03T06:41:44.452 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KMDGZ33FzUHfCfolhLttYxeuAB0= 2022-09-03T06:41:44.453 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KMDGZ33FzUHfCfolhLttYxeuAB0=', 'Content-Length': '0'} 2022-09-03T06:41:44.455 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001c2cb33604a0ce28-006312f728-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.457 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:44.458 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:44.459 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:44.460 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:44.460 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:44.461 DEBUG:boto:Method: GET 2022-09-03T06:41:44.461 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:44.462 DEBUG:boto:Data: 2022-09-03T06:41:44.462 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.463 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:44.463 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.464 DEBUG:boto:Params: {} 2022-09-03T06:41:44.464 DEBUG:boto:Token: None 2022-09-03T06:41:44.465 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/?versions 2022-09-03T06:41:44.466 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KMDGZ33FzUHfCfolhLttYxeuAB0= 2022-09-03T06:41:44.466 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KMDGZ33FzUHfCfolhLttYxeuAB0=', 'Content-Length': '0'} 2022-09-03T06:41:44.469 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000033cfe4715bfed5c0-006312f728-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.470 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:44.471 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:41:44.472 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:44.473 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:44.473 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.474 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.474 DEBUG:boto:Method: GET 2022-09-03T06:41:44.475 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.475 DEBUG:boto:Data: 2022-09-03T06:41:44.476 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.477 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:44.477 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.478 DEBUG:boto:Params: {} 2022-09-03T06:41:44.478 DEBUG:boto:Token: None 2022-09-03T06:41:44.479 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.480 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wWFo/upK/2OjMK1qxIDleT4kW28= 2022-09-03T06:41:44.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wWFo/upK/2OjMK1qxIDleT4kW28=', 'Content-Length': '0'} 2022-09-03T06:41:44.482 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c8e9335ee8b5d508-006312f728-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.483 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:44.483 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:44.484 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.485 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.485 DEBUG:boto:Method: GET 2022-09-03T06:41:44.486 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.486 DEBUG:boto:Data: 2022-09-03T06:41:44.487 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.488 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:44.488 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.489 DEBUG:boto:Params: {} 2022-09-03T06:41:44.489 DEBUG:boto:Token: None 2022-09-03T06:41:44.490 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.491 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wWFo/upK/2OjMK1qxIDleT4kW28= 2022-09-03T06:41:44.491 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wWFo/upK/2OjMK1qxIDleT4kW28=', 'Content-Length': '0'} 2022-09-03T06:41:44.493 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004d9adbb9ad34c651-006312f728-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.494 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:44.495 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:44.495 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.496 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.496 DEBUG:boto:Method: HEAD 2022-09-03T06:41:44.497 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.498 DEBUG:boto:Data: 2022-09-03T06:41:44.498 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.499 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:44.499 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.500 DEBUG:boto:Params: {} 2022-09-03T06:41:44.500 DEBUG:boto:Token: None 2022-09-03T06:41:44.501 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.502 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EyMHCm69Ri1D33YfRqeiSMqAc78= 2022-09-03T06:41:44.502 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EyMHCm69Ri1D33YfRqeiSMqAc78=', 'Content-Length': '0'} 2022-09-03T06:41:44.504 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a5fa0dc76f57c99b-006312f728-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.505 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:44.506 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:44.506 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.507 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.507 DEBUG:boto:Method: HEAD 2022-09-03T06:41:44.508 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.508 DEBUG:boto:Data: 2022-09-03T06:41:44.509 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.509 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:44.510 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.511 DEBUG:boto:Params: {} 2022-09-03T06:41:44.511 DEBUG:boto:Token: None 2022-09-03T06:41:44.512 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.512 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EyMHCm69Ri1D33YfRqeiSMqAc78= 2022-09-03T06:41:44.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EyMHCm69Ri1D33YfRqeiSMqAc78=', 'Content-Length': '0'} 2022-09-03T06:41:44.515 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000e544e504872251e-006312f728-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.515 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:41:44.516 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:44.517 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:44.517 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.518 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.518 DEBUG:boto:Method: GET 2022-09-03T06:41:44.519 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.519 DEBUG:boto:Data: 2022-09-03T06:41:44.520 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.521 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:44.521 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.522 DEBUG:boto:Params: {} 2022-09-03T06:41:44.522 DEBUG:boto:Token: None 2022-09-03T06:41:44.523 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.524 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wWFo/upK/2OjMK1qxIDleT4kW28= 2022-09-03T06:41:44.525 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wWFo/upK/2OjMK1qxIDleT4kW28=', 'Content-Length': '0'} 2022-09-03T06:41:44.526 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009c8458e877c318cb-006312f728-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.527 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:44.528 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:44.529 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.529 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.530 DEBUG:boto:Method: GET 2022-09-03T06:41:44.530 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.531 DEBUG:boto:Data: 2022-09-03T06:41:44.531 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.532 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:44.533 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.533 DEBUG:boto:Params: {} 2022-09-03T06:41:44.534 DEBUG:boto:Token: None 2022-09-03T06:41:44.534 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:44.535 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wWFo/upK/2OjMK1qxIDleT4kW28= 2022-09-03T06:41:44.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wWFo/upK/2OjMK1qxIDleT4kW28=', 'Content-Length': '0'} 2022-09-03T06:41:44.537 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000094162f12288234a4-006312f728-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.578 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:44.579 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:44.580 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:44.580 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:44.581 DEBUG:boto:Method: GET 2022-09-03T06:41:44.581 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:41:44.582 DEBUG:boto:Data: 2022-09-03T06:41:44.583 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.583 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:44.584 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.585 DEBUG:boto:Params: {} 2022-09-03T06:41:44.586 DEBUG:boto:Token: None 2022-09-03T06:41:44.586 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/ 2022-09-03T06:41:44.587 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gr56E9x3HBPARZO89Yw8lIib1UM= 2022-09-03T06:41:44.588 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gr56E9x3HBPARZO89Yw8lIib1UM=', 'Content-Length': '0'} 2022-09-03T06:41:44.591 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ee4657e9df55ee2f-006312f728-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.592 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:44.593 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:44.593 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:44.594 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:44.594 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:44.595 DEBUG:boto:Method: GET 2022-09-03T06:41:44.595 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:41:44.596 DEBUG:boto:Data: 2022-09-03T06:41:44.596 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.597 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:44.597 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.598 DEBUG:boto:Params: {} 2022-09-03T06:41:44.599 DEBUG:boto:Token: None 2022-09-03T06:41:44.599 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/ 2022-09-03T06:41:44.600 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gr56E9x3HBPARZO89Yw8lIib1UM= 2022-09-03T06:41:44.601 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gr56E9x3HBPARZO89Yw8lIib1UM=', 'Content-Length': '0'} 2022-09-03T06:41:44.608 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008c1430a2d2bbd358-006312f728-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.609 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:44.610 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:44.610 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:44.611 DEBUG:boto:Method: HEAD 2022-09-03T06:41:44.611 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:41:44.612 DEBUG:boto:Data: 2022-09-03T06:41:44.612 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.613 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:44.613 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.614 DEBUG:boto:Params: {} 2022-09-03T06:41:44.615 DEBUG:boto:Token: None 2022-09-03T06:41:44.615 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/obj11 2022-09-03T06:41:44.616 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BaqkssBdJqz4vgCtS63jROce4+Y= 2022-09-03T06:41:44.617 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BaqkssBdJqz4vgCtS63jROce4+Y=', 'Content-Length': '0'} 2022-09-03T06:41:44.619 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000017f02fd553e86e6b-006312f728-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.621 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:44.621 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:44.622 DEBUG:boto:Method: HEAD 2022-09-03T06:41:44.622 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:41:44.623 DEBUG:boto:Data: 2022-09-03T06:41:44.623 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.624 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:44.625 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.625 DEBUG:boto:Params: {} 2022-09-03T06:41:44.626 DEBUG:boto:Token: None 2022-09-03T06:41:44.626 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/obj11 2022-09-03T06:41:44.627 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BaqkssBdJqz4vgCtS63jROce4+Y= 2022-09-03T06:41:44.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BaqkssBdJqz4vgCtS63jROce4+Y=', 'Content-Length': '0'} 2022-09-03T06:41:44.630 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008425dc7692c4b0d8-006312f728-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.632 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:41:44.633 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:44.634 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:44.635 DEBUG:boto:Method: GET 2022-09-03T06:41:44.636 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:41:44.637 DEBUG:boto:Data: 2022-09-03T06:41:44.638 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.639 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:44.640 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.641 DEBUG:boto:Params: {} 2022-09-03T06:41:44.642 DEBUG:boto:Token: None 2022-09-03T06:41:44.644 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/obj11 2022-09-03T06:41:44.645 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+rEEwKI1O3Ur85i7UNJuLNKzVTk= 2022-09-03T06:41:44.646 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+rEEwKI1O3Ur85i7UNJuLNKzVTk=', 'Content-Length': '0'} 2022-09-03T06:41:44.649 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c8aa9daed0053e42-006312f728-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.694 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:44.695 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:44.696 DEBUG:boto:Method: GET 2022-09-03T06:41:44.697 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:41:44.698 DEBUG:boto:Data: 2022-09-03T06:41:44.699 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.699 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:44.700 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.700 DEBUG:boto:Params: {} 2022-09-03T06:41:44.702 DEBUG:boto:Token: None 2022-09-03T06:41:44.703 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/obj11 2022-09-03T06:41:44.704 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+rEEwKI1O3Ur85i7UNJuLNKzVTk= 2022-09-03T06:41:44.705 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+rEEwKI1O3Ur85i7UNJuLNKzVTk=', 'Content-Length': '0'} 2022-09-03T06:41:44.708 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000243509eb9b690902-006312f728-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.709 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:44.710 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:44.711 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:44.713 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:44.714 DEBUG:boto:Method: GET 2022-09-03T06:41:44.715 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:41:44.716 DEBUG:boto:Data: 2022-09-03T06:41:44.717 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.717 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:44.718 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.719 DEBUG:boto:Params: {} 2022-09-03T06:41:44.721 DEBUG:boto:Token: None 2022-09-03T06:41:44.722 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/ 2022-09-03T06:41:44.723 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gr56E9x3HBPARZO89Yw8lIib1UM= 2022-09-03T06:41:44.725 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gr56E9x3HBPARZO89Yw8lIib1UM=', 'Content-Length': '0'} 2022-09-03T06:41:44.730 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d298c2948bbda4ee-006312f728-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.731 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:44.733 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:44.734 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:44.734 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:44.735 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:44.736 DEBUG:boto:Method: GET 2022-09-03T06:41:44.737 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:41:44.739 DEBUG:boto:Data: 2022-09-03T06:41:44.740 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.742 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:44.743 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.744 DEBUG:boto:Params: {} 2022-09-03T06:41:44.745 DEBUG:boto:Token: None 2022-09-03T06:41:44.746 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/ 2022-09-03T06:41:44.747 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gr56E9x3HBPARZO89Yw8lIib1UM= 2022-09-03T06:41:44.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gr56E9x3HBPARZO89Yw8lIib1UM=', 'Content-Length': '0'} 2022-09-03T06:41:44.751 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009c1a264b812a1ffc-006312f728-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.752 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:44.753 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:41:44.753 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:44.754 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:44.755 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.756 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.757 DEBUG:boto:Method: GET 2022-09-03T06:41:44.758 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.759 DEBUG:boto:Data: 2022-09-03T06:41:44.760 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.760 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:44.761 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.762 DEBUG:boto:Params: {} 2022-09-03T06:41:44.763 DEBUG:boto:Token: None 2022-09-03T06:41:44.763 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.764 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SSbFz3kzTrO3ynlGS70EzO8dXck= 2022-09-03T06:41:44.765 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SSbFz3kzTrO3ynlGS70EzO8dXck=', 'Content-Length': '0'} 2022-09-03T06:41:44.767 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000099b7e3f488d06b64-006312f728-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.768 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:44.768 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:44.769 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.769 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.770 DEBUG:boto:Method: GET 2022-09-03T06:41:44.771 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.771 DEBUG:boto:Data: 2022-09-03T06:41:44.772 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.772 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:44.773 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.773 DEBUG:boto:Params: {} 2022-09-03T06:41:44.774 DEBUG:boto:Token: None 2022-09-03T06:41:44.775 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.775 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SSbFz3kzTrO3ynlGS70EzO8dXck= 2022-09-03T06:41:44.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SSbFz3kzTrO3ynlGS70EzO8dXck=', 'Content-Length': '0'} 2022-09-03T06:41:44.778 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b10e52db828d08c0-006312f728-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.831 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:44.832 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:44.832 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.833 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.833 DEBUG:boto:Method: HEAD 2022-09-03T06:41:44.834 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.834 DEBUG:boto:Data: 2022-09-03T06:41:44.835 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.837 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:44.837 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.838 DEBUG:boto:Params: {} 2022-09-03T06:41:44.840 DEBUG:boto:Token: None 2022-09-03T06:41:44.842 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.844 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZFXZ1OiC2KerlNyd/IWK+wkS8zY= 2022-09-03T06:41:44.845 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZFXZ1OiC2KerlNyd/IWK+wkS8zY=', 'Content-Length': '0'} 2022-09-03T06:41:44.847 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c9b558d41ae8265-006312f728-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.848 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:44.848 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:44.849 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.850 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.851 DEBUG:boto:Method: HEAD 2022-09-03T06:41:44.851 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.852 DEBUG:boto:Data: 2022-09-03T06:41:44.853 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.853 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:44.854 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.854 DEBUG:boto:Params: {} 2022-09-03T06:41:44.855 DEBUG:boto:Token: None 2022-09-03T06:41:44.858 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.858 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZFXZ1OiC2KerlNyd/IWK+wkS8zY= 2022-09-03T06:41:44.859 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZFXZ1OiC2KerlNyd/IWK+wkS8zY=', 'Content-Length': '0'} 2022-09-03T06:41:44.861 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009b532d02f615f6bb-006312f728-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.862 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:41:44.863 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:44.864 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:44.864 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.865 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.867 DEBUG:boto:Method: GET 2022-09-03T06:41:44.869 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.869 DEBUG:boto:Data: 2022-09-03T06:41:44.872 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.872 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:44.873 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.874 DEBUG:boto:Params: {} 2022-09-03T06:41:44.876 DEBUG:boto:Token: None 2022-09-03T06:41:44.876 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.878 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SSbFz3kzTrO3ynlGS70EzO8dXck= 2022-09-03T06:41:44.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SSbFz3kzTrO3ynlGS70EzO8dXck=', 'Content-Length': '0'} 2022-09-03T06:41:44.881 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000024595e543d1c72c-006312f728-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.882 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:44.882 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:44.883 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.884 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.884 DEBUG:boto:Method: GET 2022-09-03T06:41:44.885 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.885 DEBUG:boto:Data: 2022-09-03T06:41:44.886 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.887 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:44.887 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.888 DEBUG:boto:Params: {} 2022-09-03T06:41:44.889 DEBUG:boto:Token: None 2022-09-03T06:41:44.889 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:44.891 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SSbFz3kzTrO3ynlGS70EzO8dXck= 2022-09-03T06:41:44.907 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SSbFz3kzTrO3ynlGS70EzO8dXck=', 'Content-Length': '0'} 2022-09-03T06:41:44.911 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e45e9489695cc6fb-006312f728-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.913 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:44.913 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:44.914 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:44.915 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:44.915 DEBUG:boto:Method: GET 2022-09-03T06:41:44.916 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:41:44.916 DEBUG:boto:Data: 2022-09-03T06:41:44.917 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.917 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:44.918 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.918 DEBUG:boto:Params: {} 2022-09-03T06:41:44.919 DEBUG:boto:Token: None 2022-09-03T06:41:44.920 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/ 2022-09-03T06:41:44.920 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gr56E9x3HBPARZO89Yw8lIib1UM= 2022-09-03T06:41:44.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gr56E9x3HBPARZO89Yw8lIib1UM=', 'Content-Length': '0'} 2022-09-03T06:41:44.924 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000671ec7ba0f1b9bc8-006312f728-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.926 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:44.927 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:44.928 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:44.928 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:44.929 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:44.929 DEBUG:boto:Method: GET 2022-09-03T06:41:44.930 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:41:44.932 DEBUG:boto:Data: 2022-09-03T06:41:44.933 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.934 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:44.935 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.936 DEBUG:boto:Params: {} 2022-09-03T06:41:44.937 DEBUG:boto:Token: None 2022-09-03T06:41:44.938 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/ 2022-09-03T06:41:44.939 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gr56E9x3HBPARZO89Yw8lIib1UM= 2022-09-03T06:41:44.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gr56E9x3HBPARZO89Yw8lIib1UM=', 'Content-Length': '0'} 2022-09-03T06:41:44.944 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000055a08e4e693cac9e-006312f728-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.945 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:44.946 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:44.946 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:44.947 DEBUG:boto:Method: HEAD 2022-09-03T06:41:44.947 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:41:44.948 DEBUG:boto:Data: 2022-09-03T06:41:44.948 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.949 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:44.950 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.950 DEBUG:boto:Params: {} 2022-09-03T06:41:44.951 DEBUG:boto:Token: None 2022-09-03T06:41:44.952 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/obj17 2022-09-03T06:41:44.952 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Pb9A6opanJPcbuwoTPn50At8IBs= 2022-09-03T06:41:44.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Pb9A6opanJPcbuwoTPn50At8IBs=', 'Content-Length': '0'} 2022-09-03T06:41:44.955 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000514920f2eb013937-006312f728-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.957 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:44.958 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:44.959 DEBUG:boto:Method: HEAD 2022-09-03T06:41:44.960 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:41:44.961 DEBUG:boto:Data: 2022-09-03T06:41:44.962 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.963 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:44.964 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.965 DEBUG:boto:Params: {} 2022-09-03T06:41:44.966 DEBUG:boto:Token: None 2022-09-03T06:41:44.967 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/obj17 2022-09-03T06:41:44.967 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Pb9A6opanJPcbuwoTPn50At8IBs= 2022-09-03T06:41:44.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Pb9A6opanJPcbuwoTPn50At8IBs=', 'Content-Length': '0'} 2022-09-03T06:41:44.970 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000068624eb07ab3545-006312f728-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.971 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:41:44.971 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:44.972 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:44.972 DEBUG:boto:Method: GET 2022-09-03T06:41:44.973 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:41:44.974 DEBUG:boto:Data: 2022-09-03T06:41:44.974 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.975 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:44.975 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.976 DEBUG:boto:Params: {} 2022-09-03T06:41:44.976 DEBUG:boto:Token: None 2022-09-03T06:41:44.977 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/obj17 2022-09-03T06:41:44.978 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xJH4vQEmLAv1OStE346VTwiElt4= 2022-09-03T06:41:44.978 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xJH4vQEmLAv1OStE346VTwiElt4=', 'Content-Length': '0'} 2022-09-03T06:41:44.980 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c8a5afa348c88c6a-006312f728-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:44.981 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:44.982 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:44.982 DEBUG:boto:Method: GET 2022-09-03T06:41:44.983 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:41:44.983 DEBUG:boto:Data: 2022-09-03T06:41:44.984 DEBUG:boto:Headers: {} 2022-09-03T06:41:44.984 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:44.985 DEBUG:boto:Port: 8000 2022-09-03T06:41:44.985 DEBUG:boto:Params: {} 2022-09-03T06:41:44.986 DEBUG:boto:Token: None 2022-09-03T06:41:44.987 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:44 GMT /dsgvcy-75/obj17 2022-09-03T06:41:44.987 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xJH4vQEmLAv1OStE346VTwiElt4= 2022-09-03T06:41:44.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xJH4vQEmLAv1OStE346VTwiElt4=', 'Content-Length': '0'} 2022-09-03T06:41:44.990 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b1bcca93fc3f0e48-006312f728-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.034 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:45.036 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:45.037 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:45.037 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:45.038 DEBUG:boto:Method: GET 2022-09-03T06:41:45.039 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:41:45.040 DEBUG:boto:Data: 2022-09-03T06:41:45.041 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.042 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.043 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.044 DEBUG:boto:Params: {} 2022-09-03T06:41:45.045 DEBUG:boto:Token: None 2022-09-03T06:41:45.045 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/ 2022-09-03T06:41:45.046 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0= 2022-09-03T06:41:45.047 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0=', 'Content-Length': '0'} 2022-09-03T06:41:45.050 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f2a52208a429fafb-006312f729-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.050 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:45.051 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:45.052 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:45.052 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:45.053 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:45.054 DEBUG:boto:Method: GET 2022-09-03T06:41:45.054 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:41:45.055 DEBUG:boto:Data: 2022-09-03T06:41:45.055 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.056 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.056 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.057 DEBUG:boto:Params: {} 2022-09-03T06:41:45.057 DEBUG:boto:Token: None 2022-09-03T06:41:45.058 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/ 2022-09-03T06:41:45.059 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0= 2022-09-03T06:41:45.059 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0=', 'Content-Length': '0'} 2022-09-03T06:41:45.062 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e1c8d26c07bdd0f3-006312f729-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.063 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:45.064 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:41:45.065 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:45.065 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:45.066 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.066 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.067 DEBUG:boto:Method: GET 2022-09-03T06:41:45.068 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.068 DEBUG:boto:Data: 2022-09-03T06:41:45.069 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.069 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.070 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.070 DEBUG:boto:Params: {} 2022-09-03T06:41:45.071 DEBUG:boto:Token: None 2022-09-03T06:41:45.072 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.072 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SIdoEQ6r+pZQFVswKR5+j5L3MKY= 2022-09-03T06:41:45.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SIdoEQ6r+pZQFVswKR5+j5L3MKY=', 'Content-Length': '0'} 2022-09-03T06:41:45.075 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001979e2d2bc4de5c6-006312f729-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.076 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:45.077 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:45.077 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.078 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.079 DEBUG:boto:Method: GET 2022-09-03T06:41:45.079 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.080 DEBUG:boto:Data: 2022-09-03T06:41:45.080 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.081 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.081 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.082 DEBUG:boto:Params: {} 2022-09-03T06:41:45.082 DEBUG:boto:Token: None 2022-09-03T06:41:45.083 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.084 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SIdoEQ6r+pZQFVswKR5+j5L3MKY= 2022-09-03T06:41:45.084 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SIdoEQ6r+pZQFVswKR5+j5L3MKY=', 'Content-Length': '0'} 2022-09-03T06:41:45.087 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f7492e17edc16ab9-006312f729-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.088 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:45.088 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:45.089 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.089 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.090 DEBUG:boto:Method: HEAD 2022-09-03T06:41:45.091 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.091 DEBUG:boto:Data: 2022-09-03T06:41:45.092 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.092 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.093 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.093 DEBUG:boto:Params: {} 2022-09-03T06:41:45.094 DEBUG:boto:Token: None 2022-09-03T06:41:45.094 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.095 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9obFoRpHfyZEhL91lLyAUqDrjKA= 2022-09-03T06:41:45.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9obFoRpHfyZEhL91lLyAUqDrjKA=', 'Content-Length': '0'} 2022-09-03T06:41:45.098 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bf7281c06f81d4b5-006312f729-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.099 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:45.100 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:45.100 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.101 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.102 DEBUG:boto:Method: HEAD 2022-09-03T06:41:45.102 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.103 DEBUG:boto:Data: 2022-09-03T06:41:45.104 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.105 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.105 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.106 DEBUG:boto:Params: {} 2022-09-03T06:41:45.107 DEBUG:boto:Token: None 2022-09-03T06:41:45.108 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.109 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9obFoRpHfyZEhL91lLyAUqDrjKA= 2022-09-03T06:41:45.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9obFoRpHfyZEhL91lLyAUqDrjKA=', 'Content-Length': '0'} 2022-09-03T06:41:45.112 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004f3b28e42d07d5c5-006312f729-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.113 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:41:45.114 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:45.114 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:45.115 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.116 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.116 DEBUG:boto:Method: GET 2022-09-03T06:41:45.117 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.118 DEBUG:boto:Data: 2022-09-03T06:41:45.119 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.119 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.120 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.120 DEBUG:boto:Params: {} 2022-09-03T06:41:45.121 DEBUG:boto:Token: None 2022-09-03T06:41:45.122 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.123 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SIdoEQ6r+pZQFVswKR5+j5L3MKY= 2022-09-03T06:41:45.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SIdoEQ6r+pZQFVswKR5+j5L3MKY=', 'Content-Length': '0'} 2022-09-03T06:41:45.126 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a46541d355f6b5ce-006312f729-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.166 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:45.167 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:45.168 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.169 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.169 DEBUG:boto:Method: GET 2022-09-03T06:41:45.170 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.171 DEBUG:boto:Data: 2022-09-03T06:41:45.171 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.172 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.173 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.174 DEBUG:boto:Params: {} 2022-09-03T06:41:45.174 DEBUG:boto:Token: None 2022-09-03T06:41:45.175 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:45.176 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SIdoEQ6r+pZQFVswKR5+j5L3MKY= 2022-09-03T06:41:45.176 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SIdoEQ6r+pZQFVswKR5+j5L3MKY=', 'Content-Length': '0'} 2022-09-03T06:41:45.179 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002975e919525dda0c-006312f729-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.180 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:45.180 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:45.181 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:45.182 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:45.182 DEBUG:boto:Method: GET 2022-09-03T06:41:45.183 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:41:45.184 DEBUG:boto:Data: 2022-09-03T06:41:45.185 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.185 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.186 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.187 DEBUG:boto:Params: {} 2022-09-03T06:41:45.187 DEBUG:boto:Token: None 2022-09-03T06:41:45.188 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/ 2022-09-03T06:41:45.188 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0= 2022-09-03T06:41:45.189 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0=', 'Content-Length': '0'} 2022-09-03T06:41:45.192 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cb86e9f85ebd2c96-006312f729-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.193 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:45.194 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:45.195 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:45.195 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:45.196 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:45.196 DEBUG:boto:Method: GET 2022-09-03T06:41:45.197 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:41:45.198 DEBUG:boto:Data: 2022-09-03T06:41:45.198 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.199 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.199 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.200 DEBUG:boto:Params: {} 2022-09-03T06:41:45.200 DEBUG:boto:Token: None 2022-09-03T06:41:45.201 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/ 2022-09-03T06:41:45.202 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0= 2022-09-03T06:41:45.202 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0=', 'Content-Length': '0'} 2022-09-03T06:41:45.206 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008651394763cc38d6-006312f729-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.209 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:45.210 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:45.211 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:45.212 DEBUG:boto:Method: HEAD 2022-09-03T06:41:45.213 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:41:45.213 DEBUG:boto:Data: 2022-09-03T06:41:45.214 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.214 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.215 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.216 DEBUG:boto:Params: {} 2022-09-03T06:41:45.217 DEBUG:boto:Token: None 2022-09-03T06:41:45.217 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj19 2022-09-03T06:41:45.218 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M1b0tG7X4YEHfENoCG/IwEFf73M= 2022-09-03T06:41:45.219 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M1b0tG7X4YEHfENoCG/IwEFf73M=', 'Content-Length': '0'} 2022-09-03T06:41:45.221 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004378ae4a42138b31-006312f729-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.222 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:45.222 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:45.223 DEBUG:boto:Method: HEAD 2022-09-03T06:41:45.224 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:41:45.225 DEBUG:boto:Data: 2022-09-03T06:41:45.225 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.226 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.227 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.227 DEBUG:boto:Params: {} 2022-09-03T06:41:45.228 DEBUG:boto:Token: None 2022-09-03T06:41:45.229 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj19 2022-09-03T06:41:45.229 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M1b0tG7X4YEHfENoCG/IwEFf73M= 2022-09-03T06:41:45.230 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M1b0tG7X4YEHfENoCG/IwEFf73M=', 'Content-Length': '0'} 2022-09-03T06:41:45.233 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000062c5f2ba586f0552-006312f729-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.234 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:41:45.235 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:45.236 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:45.237 DEBUG:boto:Method: GET 2022-09-03T06:41:45.238 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:41:45.239 DEBUG:boto:Data: 2022-09-03T06:41:45.240 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.241 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.241 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.242 DEBUG:boto:Params: {} 2022-09-03T06:41:45.243 DEBUG:boto:Token: None 2022-09-03T06:41:45.244 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj19 2022-09-03T06:41:45.245 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hj+7K2GPaVxkuKU+jq38GgB8e/E= 2022-09-03T06:41:45.245 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hj+7K2GPaVxkuKU+jq38GgB8e/E=', 'Content-Length': '0'} 2022-09-03T06:41:45.248 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e5fdc41d56b9456d-006312f729-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.249 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:45.250 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:45.251 DEBUG:boto:Method: GET 2022-09-03T06:41:45.251 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:41:45.252 DEBUG:boto:Data: 2022-09-03T06:41:45.253 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.254 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.254 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.255 DEBUG:boto:Params: {} 2022-09-03T06:41:45.256 DEBUG:boto:Token: None 2022-09-03T06:41:45.256 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj19 2022-09-03T06:41:45.257 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hj+7K2GPaVxkuKU+jq38GgB8e/E= 2022-09-03T06:41:45.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hj+7K2GPaVxkuKU+jq38GgB8e/E=', 'Content-Length': '0'} 2022-09-03T06:41:45.261 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001927001aba543441-006312f729-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.262 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:45.263 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:45.263 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:45.264 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:45.264 DEBUG:boto:Method: GET 2022-09-03T06:41:45.265 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:41:45.265 DEBUG:boto:Data: 2022-09-03T06:41:45.266 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.266 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.267 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.267 DEBUG:boto:Params: {} 2022-09-03T06:41:45.268 DEBUG:boto:Token: None 2022-09-03T06:41:45.270 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/ 2022-09-03T06:41:45.272 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0= 2022-09-03T06:41:45.273 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0=', 'Content-Length': '0'} 2022-09-03T06:41:45.276 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f3f6914b31a36eb4-006312f729-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.277 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:45.278 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:45.278 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:45.279 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:45.279 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:45.280 DEBUG:boto:Method: GET 2022-09-03T06:41:45.280 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:41:45.281 DEBUG:boto:Data: 2022-09-03T06:41:45.281 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.282 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.282 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.283 DEBUG:boto:Params: {} 2022-09-03T06:41:45.283 DEBUG:boto:Token: None 2022-09-03T06:41:45.284 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/ 2022-09-03T06:41:45.285 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0= 2022-09-03T06:41:45.285 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0=', 'Content-Length': '0'} 2022-09-03T06:41:45.289 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000048638268d8807c8d-006312f729-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.290 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:45.290 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:41:45.291 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:45.292 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:45.292 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.293 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.293 DEBUG:boto:Method: GET 2022-09-03T06:41:45.294 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.294 DEBUG:boto:Data: 2022-09-03T06:41:45.295 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.295 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.296 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.297 DEBUG:boto:Params: {} 2022-09-03T06:41:45.297 DEBUG:boto:Token: None 2022-09-03T06:41:45.298 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.299 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TluctP2eCd1bZHg+7UENlAfWn7M= 2022-09-03T06:41:45.299 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TluctP2eCd1bZHg+7UENlAfWn7M=', 'Content-Length': '0'} 2022-09-03T06:41:45.301 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074e06c51a04ec42f-006312f729-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.302 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:45.303 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:45.303 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.304 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.304 DEBUG:boto:Method: GET 2022-09-03T06:41:45.305 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.306 DEBUG:boto:Data: 2022-09-03T06:41:45.306 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.307 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.307 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.308 DEBUG:boto:Params: {} 2022-09-03T06:41:45.309 DEBUG:boto:Token: None 2022-09-03T06:41:45.309 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.310 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TluctP2eCd1bZHg+7UENlAfWn7M= 2022-09-03T06:41:45.311 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TluctP2eCd1bZHg+7UENlAfWn7M=', 'Content-Length': '0'} 2022-09-03T06:41:45.313 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008305cd04d798e2bc-006312f729-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.314 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:45.315 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:45.315 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.316 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.317 DEBUG:boto:Method: HEAD 2022-09-03T06:41:45.317 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.318 DEBUG:boto:Data: 2022-09-03T06:41:45.318 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.319 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.319 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.320 DEBUG:boto:Params: {} 2022-09-03T06:41:45.321 DEBUG:boto:Token: None 2022-09-03T06:41:45.321 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.322 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pAi01ojGUhSUw8JNwTrH8yT1akI= 2022-09-03T06:41:45.323 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pAi01ojGUhSUw8JNwTrH8yT1akI=', 'Content-Length': '0'} 2022-09-03T06:41:45.330 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000051be64da0ea60f75-006312f729-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.331 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:45.331 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:45.332 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.333 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.334 DEBUG:boto:Method: HEAD 2022-09-03T06:41:45.335 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.336 DEBUG:boto:Data: 2022-09-03T06:41:45.336 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.337 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.337 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.338 DEBUG:boto:Params: {} 2022-09-03T06:41:45.338 DEBUG:boto:Token: None 2022-09-03T06:41:45.339 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.340 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pAi01ojGUhSUw8JNwTrH8yT1akI= 2022-09-03T06:41:45.341 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pAi01ojGUhSUw8JNwTrH8yT1akI=', 'Content-Length': '0'} 2022-09-03T06:41:45.343 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d6d9114f2e9e1c59-006312f729-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.344 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:41:45.344 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:45.345 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:45.345 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.346 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.347 DEBUG:boto:Method: GET 2022-09-03T06:41:45.347 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.348 DEBUG:boto:Data: 2022-09-03T06:41:45.348 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.349 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.349 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.350 DEBUG:boto:Params: {} 2022-09-03T06:41:45.351 DEBUG:boto:Token: None 2022-09-03T06:41:45.352 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.352 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TluctP2eCd1bZHg+7UENlAfWn7M= 2022-09-03T06:41:45.354 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TluctP2eCd1bZHg+7UENlAfWn7M=', 'Content-Length': '0'} 2022-09-03T06:41:45.357 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000569b73c3952de071-006312f729-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.360 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:45.361 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:45.362 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.363 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.364 DEBUG:boto:Method: GET 2022-09-03T06:41:45.365 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.366 DEBUG:boto:Data: 2022-09-03T06:41:45.367 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.368 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.368 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.369 DEBUG:boto:Params: {} 2022-09-03T06:41:45.369 DEBUG:boto:Token: None 2022-09-03T06:41:45.370 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:45.371 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TluctP2eCd1bZHg+7UENlAfWn7M= 2022-09-03T06:41:45.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TluctP2eCd1bZHg+7UENlAfWn7M=', 'Content-Length': '0'} 2022-09-03T06:41:45.374 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000071c734b1d38e0e67-006312f729-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.418 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:45.419 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:45.419 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:45.420 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:45.420 DEBUG:boto:Method: GET 2022-09-03T06:41:45.421 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:41:45.421 DEBUG:boto:Data: 2022-09-03T06:41:45.422 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.422 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.423 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.424 DEBUG:boto:Params: {} 2022-09-03T06:41:45.424 DEBUG:boto:Token: None 2022-09-03T06:41:45.425 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/ 2022-09-03T06:41:45.426 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0= 2022-09-03T06:41:45.427 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0=', 'Content-Length': '0'} 2022-09-03T06:41:45.430 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000096b0aa8bf6595694-006312f729-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.430 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:45.432 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:45.432 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:45.433 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:45.433 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:45.434 DEBUG:boto:Method: GET 2022-09-03T06:41:45.435 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:41:45.435 DEBUG:boto:Data: 2022-09-03T06:41:45.436 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.436 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.437 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.437 DEBUG:boto:Params: {} 2022-09-03T06:41:45.438 DEBUG:boto:Token: None 2022-09-03T06:41:45.439 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/ 2022-09-03T06:41:45.439 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0= 2022-09-03T06:41:45.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0=', 'Content-Length': '0'} 2022-09-03T06:41:45.445 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000011b32a004c48b973-006312f729-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.445 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:45.446 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:45.447 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:45.447 DEBUG:boto:Method: HEAD 2022-09-03T06:41:45.448 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:41:45.449 DEBUG:boto:Data: 2022-09-03T06:41:45.450 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.450 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.451 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.451 DEBUG:boto:Params: {} 2022-09-03T06:41:45.452 DEBUG:boto:Token: None 2022-09-03T06:41:45.452 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj22 2022-09-03T06:41:45.453 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uP172F/VJyxVzm5b41BZvYHpkVA= 2022-09-03T06:41:45.454 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uP172F/VJyxVzm5b41BZvYHpkVA=', 'Content-Length': '0'} 2022-09-03T06:41:45.455 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000061a897da00721f6e-006312f729-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.456 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:45.457 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:45.457 DEBUG:boto:Method: HEAD 2022-09-03T06:41:45.458 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:41:45.458 DEBUG:boto:Data: 2022-09-03T06:41:45.459 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.460 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.460 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.461 DEBUG:boto:Params: {} 2022-09-03T06:41:45.461 DEBUG:boto:Token: None 2022-09-03T06:41:45.462 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj22 2022-09-03T06:41:45.462 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uP172F/VJyxVzm5b41BZvYHpkVA= 2022-09-03T06:41:45.463 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uP172F/VJyxVzm5b41BZvYHpkVA=', 'Content-Length': '0'} 2022-09-03T06:41:45.465 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aa31ae281e7d8b53-006312f729-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.466 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:41:45.467 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:45.467 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:45.468 DEBUG:boto:Method: GET 2022-09-03T06:41:45.468 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:41:45.469 DEBUG:boto:Data: 2022-09-03T06:41:45.469 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.470 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.471 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.471 DEBUG:boto:Params: {} 2022-09-03T06:41:45.472 DEBUG:boto:Token: None 2022-09-03T06:41:45.472 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj22 2022-09-03T06:41:45.473 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bNwnQliolt5CThohOsF820HXlOo= 2022-09-03T06:41:45.474 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bNwnQliolt5CThohOsF820HXlOo=', 'Content-Length': '0'} 2022-09-03T06:41:45.476 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000086cc12d0e4a07793-006312f729-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.477 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:45.477 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:45.478 DEBUG:boto:Method: GET 2022-09-03T06:41:45.479 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:41:45.479 DEBUG:boto:Data: 2022-09-03T06:41:45.480 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.480 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.481 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.481 DEBUG:boto:Params: {} 2022-09-03T06:41:45.482 DEBUG:boto:Token: None 2022-09-03T06:41:45.482 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj22 2022-09-03T06:41:45.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bNwnQliolt5CThohOsF820HXlOo= 2022-09-03T06:41:45.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bNwnQliolt5CThohOsF820HXlOo=', 'Content-Length': '0'} 2022-09-03T06:41:45.486 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000be67bc759f6fac9b-006312f729-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.487 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:45.487 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:45.488 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:45.488 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:45.489 DEBUG:boto:Method: GET 2022-09-03T06:41:45.489 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:41:45.490 DEBUG:boto:Data: 2022-09-03T06:41:45.490 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.491 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.491 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.492 DEBUG:boto:Params: {} 2022-09-03T06:41:45.493 DEBUG:boto:Token: None 2022-09-03T06:41:45.493 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/ 2022-09-03T06:41:45.494 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0= 2022-09-03T06:41:45.495 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0=', 'Content-Length': '0'} 2022-09-03T06:41:45.497 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000851c95489aa99489-006312f729-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.498 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:45.499 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:45.500 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:45.500 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:45.501 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:45.502 DEBUG:boto:Method: GET 2022-09-03T06:41:45.502 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:41:45.503 DEBUG:boto:Data: 2022-09-03T06:41:45.504 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.505 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.505 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.506 DEBUG:boto:Params: {} 2022-09-03T06:41:45.508 DEBUG:boto:Token: None 2022-09-03T06:41:45.509 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/ 2022-09-03T06:41:45.509 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0= 2022-09-03T06:41:45.510 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0=', 'Content-Length': '0'} 2022-09-03T06:41:45.513 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007bede88a90c6208b-006312f729-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.514 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:45.515 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:41:45.516 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:45.516 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:45.517 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.517 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.518 DEBUG:boto:Method: GET 2022-09-03T06:41:45.519 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.520 DEBUG:boto:Data: 2022-09-03T06:41:45.520 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.521 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.521 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.525 DEBUG:boto:Params: {} 2022-09-03T06:41:45.526 DEBUG:boto:Token: None 2022-09-03T06:41:45.526 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.527 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TSjs1OpqlaOKPHJhXYjtUz47k14= 2022-09-03T06:41:45.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TSjs1OpqlaOKPHJhXYjtUz47k14=', 'Content-Length': '0'} 2022-09-03T06:41:45.531 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004d9a6b7973a393d1-006312f729-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.533 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:45.537 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:45.538 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.539 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.540 DEBUG:boto:Method: GET 2022-09-03T06:41:45.540 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.541 DEBUG:boto:Data: 2022-09-03T06:41:45.544 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.545 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.546 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.547 DEBUG:boto:Params: {} 2022-09-03T06:41:45.548 DEBUG:boto:Token: None 2022-09-03T06:41:45.549 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.551 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TSjs1OpqlaOKPHJhXYjtUz47k14= 2022-09-03T06:41:45.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TSjs1OpqlaOKPHJhXYjtUz47k14=', 'Content-Length': '0'} 2022-09-03T06:41:45.556 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004da381a79c9d8726-006312f729-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.557 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:45.558 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:45.559 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.560 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.561 DEBUG:boto:Method: HEAD 2022-09-03T06:41:45.563 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.564 DEBUG:boto:Data: 2022-09-03T06:41:45.564 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.565 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.566 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.567 DEBUG:boto:Params: {} 2022-09-03T06:41:45.568 DEBUG:boto:Token: None 2022-09-03T06:41:45.569 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.570 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:54+7CSNdc7wX48xdbip21AeI8FA= 2022-09-03T06:41:45.571 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:54+7CSNdc7wX48xdbip21AeI8FA=', 'Content-Length': '0'} 2022-09-03T06:41:45.574 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000094da63e5c841f6b6-006312f729-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.575 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:45.575 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:45.576 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.576 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.579 DEBUG:boto:Method: HEAD 2022-09-03T06:41:45.579 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.580 DEBUG:boto:Data: 2022-09-03T06:41:45.580 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.581 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.581 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.582 DEBUG:boto:Params: {} 2022-09-03T06:41:45.582 DEBUG:boto:Token: None 2022-09-03T06:41:45.583 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.584 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:54+7CSNdc7wX48xdbip21AeI8FA= 2022-09-03T06:41:45.585 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:54+7CSNdc7wX48xdbip21AeI8FA=', 'Content-Length': '0'} 2022-09-03T06:41:45.588 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d790719f39ebb65-006312f729-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.589 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:41:45.590 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:45.590 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:45.591 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.592 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.594 DEBUG:boto:Method: GET 2022-09-03T06:41:45.595 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.595 DEBUG:boto:Data: 2022-09-03T06:41:45.596 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.597 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.597 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.598 DEBUG:boto:Params: {} 2022-09-03T06:41:45.599 DEBUG:boto:Token: None 2022-09-03T06:41:45.599 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.600 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TSjs1OpqlaOKPHJhXYjtUz47k14= 2022-09-03T06:41:45.601 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TSjs1OpqlaOKPHJhXYjtUz47k14=', 'Content-Length': '0'} 2022-09-03T06:41:45.603 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c868b33d01ff64b0-006312f729-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.604 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:45.605 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:45.605 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.606 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.606 DEBUG:boto:Method: GET 2022-09-03T06:41:45.607 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.608 DEBUG:boto:Data: 2022-09-03T06:41:45.608 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.609 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.610 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.610 DEBUG:boto:Params: {} 2022-09-03T06:41:45.611 DEBUG:boto:Token: None 2022-09-03T06:41:45.613 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:45.614 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TSjs1OpqlaOKPHJhXYjtUz47k14= 2022-09-03T06:41:45.615 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TSjs1OpqlaOKPHJhXYjtUz47k14=', 'Content-Length': '0'} 2022-09-03T06:41:45.617 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a400d82fd6e9f80-006312f729-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.618 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:45.618 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:45.620 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:45.620 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:45.621 DEBUG:boto:Method: GET 2022-09-03T06:41:45.622 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:41:45.622 DEBUG:boto:Data: 2022-09-03T06:41:45.623 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.623 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.624 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.624 DEBUG:boto:Params: {} 2022-09-03T06:41:45.625 DEBUG:boto:Token: None 2022-09-03T06:41:45.626 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/ 2022-09-03T06:41:45.626 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0= 2022-09-03T06:41:45.627 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0=', 'Content-Length': '0'} 2022-09-03T06:41:45.630 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d4015a2279871f6c-006312f729-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.674 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:45.675 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:45.676 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:45.676 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:45.677 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:45.677 DEBUG:boto:Method: GET 2022-09-03T06:41:45.678 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:41:45.679 DEBUG:boto:Data: 2022-09-03T06:41:45.679 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.680 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.681 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.681 DEBUG:boto:Params: {} 2022-09-03T06:41:45.682 DEBUG:boto:Token: None 2022-09-03T06:41:45.683 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/ 2022-09-03T06:41:45.683 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0= 2022-09-03T06:41:45.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0=', 'Content-Length': '0'} 2022-09-03T06:41:45.689 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008bf424ac63593a76-006312f729-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.689 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:45.691 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:45.691 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:45.692 DEBUG:boto:Method: HEAD 2022-09-03T06:41:45.693 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:41:45.694 DEBUG:boto:Data: 2022-09-03T06:41:45.694 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.695 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.695 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.696 DEBUG:boto:Params: {} 2022-09-03T06:41:45.697 DEBUG:boto:Token: None 2022-09-03T06:41:45.697 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj33 2022-09-03T06:41:45.698 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:foKQG760je5lVW/A5TR3n/Rhev8= 2022-09-03T06:41:45.698 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:foKQG760je5lVW/A5TR3n/Rhev8=', 'Content-Length': '0'} 2022-09-03T06:41:45.701 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000311e5ea13d34b4dd-006312f729-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.701 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:45.702 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:45.703 DEBUG:boto:Method: HEAD 2022-09-03T06:41:45.703 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:41:45.704 DEBUG:boto:Data: 2022-09-03T06:41:45.704 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.705 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.705 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.706 DEBUG:boto:Params: {} 2022-09-03T06:41:45.707 DEBUG:boto:Token: None 2022-09-03T06:41:45.707 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj33 2022-09-03T06:41:45.708 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:foKQG760je5lVW/A5TR3n/Rhev8= 2022-09-03T06:41:45.709 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:foKQG760je5lVW/A5TR3n/Rhev8=', 'Content-Length': '0'} 2022-09-03T06:41:45.711 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000253c3e8397680600-006312f729-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.712 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:41:45.712 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:45.713 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:45.714 DEBUG:boto:Method: GET 2022-09-03T06:41:45.715 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:41:45.715 DEBUG:boto:Data: 2022-09-03T06:41:45.716 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.716 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.717 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.718 DEBUG:boto:Params: {} 2022-09-03T06:41:45.719 DEBUG:boto:Token: None 2022-09-03T06:41:45.720 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj33 2022-09-03T06:41:45.721 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0N0QrSPrsDvOrFPmBCQzkMDFUi8= 2022-09-03T06:41:45.721 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0N0QrSPrsDvOrFPmBCQzkMDFUi8=', 'Content-Length': '0'} 2022-09-03T06:41:45.724 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a3d05d3cf9f87d8-006312f729-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.725 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:45.725 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:45.726 DEBUG:boto:Method: GET 2022-09-03T06:41:45.726 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:41:45.727 DEBUG:boto:Data: 2022-09-03T06:41:45.728 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.728 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.729 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.729 DEBUG:boto:Params: {} 2022-09-03T06:41:45.730 DEBUG:boto:Token: None 2022-09-03T06:41:45.730 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj33 2022-09-03T06:41:45.731 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0N0QrSPrsDvOrFPmBCQzkMDFUi8= 2022-09-03T06:41:45.733 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0N0QrSPrsDvOrFPmBCQzkMDFUi8=', 'Content-Length': '0'} 2022-09-03T06:41:45.735 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a7ee631d87a37878-006312f729-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.778 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:45.779 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:45.780 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:45.780 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:45.781 DEBUG:boto:Method: GET 2022-09-03T06:41:45.781 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:41:45.784 DEBUG:boto:Data: 2022-09-03T06:41:45.786 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.787 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.788 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.791 DEBUG:boto:Params: {} 2022-09-03T06:41:45.792 DEBUG:boto:Token: None 2022-09-03T06:41:45.793 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/ 2022-09-03T06:41:45.794 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0= 2022-09-03T06:41:45.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0=', 'Content-Length': '0'} 2022-09-03T06:41:45.799 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c0981ccd82b8d1f1-006312f729-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.800 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:45.801 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:45.802 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:45.803 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:45.804 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:45.805 DEBUG:boto:Method: GET 2022-09-03T06:41:45.806 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:41:45.808 DEBUG:boto:Data: 2022-09-03T06:41:45.809 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.811 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.813 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.814 DEBUG:boto:Params: {} 2022-09-03T06:41:45.815 DEBUG:boto:Token: None 2022-09-03T06:41:45.816 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/ 2022-09-03T06:41:45.817 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0= 2022-09-03T06:41:45.819 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0=', 'Content-Length': '0'} 2022-09-03T06:41:45.822 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a015ea6d7d4e3be6-006312f729-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.823 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:45.825 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:41:45.825 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:45.826 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:45.827 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.828 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.830 DEBUG:boto:Method: GET 2022-09-03T06:41:45.831 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.832 DEBUG:boto:Data: 2022-09-03T06:41:45.833 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.833 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.834 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.835 DEBUG:boto:Params: {} 2022-09-03T06:41:45.837 DEBUG:boto:Token: None 2022-09-03T06:41:45.837 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.838 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YPRFcCLrcAdJdRh4a0QPJR4/62I= 2022-09-03T06:41:45.838 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YPRFcCLrcAdJdRh4a0QPJR4/62I=', 'Content-Length': '0'} 2022-09-03T06:41:45.841 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008466ed02f10b5f38-006312f729-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.843 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:45.845 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:45.846 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.847 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.848 DEBUG:boto:Method: GET 2022-09-03T06:41:45.849 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.849 DEBUG:boto:Data: 2022-09-03T06:41:45.850 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.851 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.852 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.854 DEBUG:boto:Params: {} 2022-09-03T06:41:45.856 DEBUG:boto:Token: None 2022-09-03T06:41:45.857 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.858 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YPRFcCLrcAdJdRh4a0QPJR4/62I= 2022-09-03T06:41:45.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YPRFcCLrcAdJdRh4a0QPJR4/62I=', 'Content-Length': '0'} 2022-09-03T06:41:45.862 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000064f25a4bb2446c1c-006312f729-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.864 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:45.865 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:45.866 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.867 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.868 DEBUG:boto:Method: HEAD 2022-09-03T06:41:45.869 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.870 DEBUG:boto:Data: 2022-09-03T06:41:45.870 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.872 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.873 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.874 DEBUG:boto:Params: {} 2022-09-03T06:41:45.877 DEBUG:boto:Token: None 2022-09-03T06:41:45.878 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.879 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PVWzVlVpR7JArNwc0WObKuRnduU= 2022-09-03T06:41:45.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PVWzVlVpR7JArNwc0WObKuRnduU=', 'Content-Length': '0'} 2022-09-03T06:41:45.881 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b64fa63df51a901d-006312f729-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.882 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:45.883 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:45.884 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.885 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.886 DEBUG:boto:Method: HEAD 2022-09-03T06:41:45.886 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.888 DEBUG:boto:Data: 2022-09-03T06:41:45.889 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.890 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.891 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.892 DEBUG:boto:Params: {} 2022-09-03T06:41:45.893 DEBUG:boto:Token: None 2022-09-03T06:41:45.894 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.895 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PVWzVlVpR7JArNwc0WObKuRnduU= 2022-09-03T06:41:45.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PVWzVlVpR7JArNwc0WObKuRnduU=', 'Content-Length': '0'} 2022-09-03T06:41:45.900 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f52be1e5534b8a75-006312f729-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.902 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:41:45.903 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:45.904 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:45.904 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.905 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.906 DEBUG:boto:Method: GET 2022-09-03T06:41:45.906 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.915 DEBUG:boto:Data: 2022-09-03T06:41:45.916 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.917 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.918 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.919 DEBUG:boto:Params: {} 2022-09-03T06:41:45.920 DEBUG:boto:Token: None 2022-09-03T06:41:45.921 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.922 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YPRFcCLrcAdJdRh4a0QPJR4/62I= 2022-09-03T06:41:45.923 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YPRFcCLrcAdJdRh4a0QPJR4/62I=', 'Content-Length': '0'} 2022-09-03T06:41:45.926 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000b136965228b7334-006312f729-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.928 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:45.929 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:45.930 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.930 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.931 DEBUG:boto:Method: GET 2022-09-03T06:41:45.931 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.932 DEBUG:boto:Data: 2022-09-03T06:41:45.932 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.933 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.934 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.934 DEBUG:boto:Params: {} 2022-09-03T06:41:45.935 DEBUG:boto:Token: None 2022-09-03T06:41:45.935 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:45.936 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YPRFcCLrcAdJdRh4a0QPJR4/62I= 2022-09-03T06:41:45.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YPRFcCLrcAdJdRh4a0QPJR4/62I=', 'Content-Length': '0'} 2022-09-03T06:41:45.940 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000208a95715288aa56-006312f729-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.941 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:45.942 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:45.942 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:45.943 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:45.943 DEBUG:boto:Method: GET 2022-09-03T06:41:45.944 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:41:45.945 DEBUG:boto:Data: 2022-09-03T06:41:45.945 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.946 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.946 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.947 DEBUG:boto:Params: {} 2022-09-03T06:41:45.948 DEBUG:boto:Token: None 2022-09-03T06:41:45.948 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/ 2022-09-03T06:41:45.949 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0= 2022-09-03T06:41:45.950 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0=', 'Content-Length': '0'} 2022-09-03T06:41:45.952 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009938cc305aea403a-006312f729-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.954 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:45.955 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:45.956 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:45.957 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:45.958 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:45.959 DEBUG:boto:Method: GET 2022-09-03T06:41:45.960 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:41:45.961 DEBUG:boto:Data: 2022-09-03T06:41:45.962 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.963 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.964 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.964 DEBUG:boto:Params: {} 2022-09-03T06:41:45.966 DEBUG:boto:Token: None 2022-09-03T06:41:45.967 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/ 2022-09-03T06:41:45.968 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0= 2022-09-03T06:41:45.970 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/a/WDulD3olohWT36lE/0OHxWf0=', 'Content-Length': '0'} 2022-09-03T06:41:45.974 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009567f1eee0dba0e1-006312f729-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.975 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:45.977 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:45.977 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:45.978 DEBUG:boto:Method: HEAD 2022-09-03T06:41:45.979 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:41:45.979 DEBUG:boto:Data: 2022-09-03T06:41:45.980 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.980 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:45.981 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.982 DEBUG:boto:Params: {} 2022-09-03T06:41:45.982 DEBUG:boto:Token: None 2022-09-03T06:41:45.983 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj34 2022-09-03T06:41:45.984 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HFyaOa0fe0+C4xVfcvGZBTn2UDQ= 2022-09-03T06:41:45.985 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HFyaOa0fe0+C4xVfcvGZBTn2UDQ=', 'Content-Length': '0'} 2022-09-03T06:41:45.987 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b980e7f215f6ad6b-006312f729-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.988 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:45.989 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:45.989 DEBUG:boto:Method: HEAD 2022-09-03T06:41:45.990 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:41:45.990 DEBUG:boto:Data: 2022-09-03T06:41:45.991 DEBUG:boto:Headers: {} 2022-09-03T06:41:45.992 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:45.992 DEBUG:boto:Port: 8000 2022-09-03T06:41:45.993 DEBUG:boto:Params: {} 2022-09-03T06:41:45.993 DEBUG:boto:Token: None 2022-09-03T06:41:45.994 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:45 GMT /dsgvcy-75/obj34 2022-09-03T06:41:45.995 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HFyaOa0fe0+C4xVfcvGZBTn2UDQ= 2022-09-03T06:41:45.995 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HFyaOa0fe0+C4xVfcvGZBTn2UDQ=', 'Content-Length': '0'} 2022-09-03T06:41:45.998 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000f552e69855fb85f-006312f729-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:45.998 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:41:45.999 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:46.000 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:46.000 DEBUG:boto:Method: GET 2022-09-03T06:41:46.001 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:41:46.001 DEBUG:boto:Data: 2022-09-03T06:41:46.003 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.003 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.004 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.004 DEBUG:boto:Params: {} 2022-09-03T06:41:46.005 DEBUG:boto:Token: None 2022-09-03T06:41:46.006 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj34 2022-09-03T06:41:46.006 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NkoiFvHClsE++v3WcqEKMNmX/U8= 2022-09-03T06:41:46.007 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NkoiFvHClsE++v3WcqEKMNmX/U8=', 'Content-Length': '0'} 2022-09-03T06:41:46.009 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000451b45a76e882666-006312f72a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.010 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:46.011 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:46.011 DEBUG:boto:Method: GET 2022-09-03T06:41:46.012 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:41:46.012 DEBUG:boto:Data: 2022-09-03T06:41:46.013 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.014 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.014 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.015 DEBUG:boto:Params: {} 2022-09-03T06:41:46.016 DEBUG:boto:Token: None 2022-09-03T06:41:46.016 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj34 2022-09-03T06:41:46.017 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NkoiFvHClsE++v3WcqEKMNmX/U8= 2022-09-03T06:41:46.018 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NkoiFvHClsE++v3WcqEKMNmX/U8=', 'Content-Length': '0'} 2022-09-03T06:41:46.021 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a18ec56dc24ecd0e-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.069 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:46.070 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:46.071 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:46.072 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:46.072 DEBUG:boto:Method: GET 2022-09-03T06:41:46.073 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:41:46.074 DEBUG:boto:Data: 2022-09-03T06:41:46.076 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.077 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.078 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.079 DEBUG:boto:Params: {} 2022-09-03T06:41:46.080 DEBUG:boto:Token: None 2022-09-03T06:41:46.081 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/ 2022-09-03T06:41:46.081 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY= 2022-09-03T06:41:46.082 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY=', 'Content-Length': '0'} 2022-09-03T06:41:46.085 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005e4be07815cdf3a7-006312f72a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.086 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:46.087 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:46.087 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:46.088 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:46.088 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:46.089 DEBUG:boto:Method: GET 2022-09-03T06:41:46.089 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:41:46.090 DEBUG:boto:Data: 2022-09-03T06:41:46.090 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.091 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.091 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.092 DEBUG:boto:Params: {} 2022-09-03T06:41:46.092 DEBUG:boto:Token: None 2022-09-03T06:41:46.094 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/ 2022-09-03T06:41:46.095 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY= 2022-09-03T06:41:46.095 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY=', 'Content-Length': '0'} 2022-09-03T06:41:46.100 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f38a5ed334896f95-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.101 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:46.102 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:41:46.103 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:46.103 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:46.104 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.105 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.105 DEBUG:boto:Method: GET 2022-09-03T06:41:46.106 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.107 DEBUG:boto:Data: 2022-09-03T06:41:46.108 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.108 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.109 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.110 DEBUG:boto:Params: {} 2022-09-03T06:41:46.111 DEBUG:boto:Token: None 2022-09-03T06:41:46.111 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.112 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H1BLMLjjO8qGZjz09DQSfoyI3S0= 2022-09-03T06:41:46.114 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H1BLMLjjO8qGZjz09DQSfoyI3S0=', 'Content-Length': '0'} 2022-09-03T06:41:46.116 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002c7fbede3b99771c-006312f72a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.117 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:46.118 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:46.119 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.120 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.120 DEBUG:boto:Method: GET 2022-09-03T06:41:46.121 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.122 DEBUG:boto:Data: 2022-09-03T06:41:46.123 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.124 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.125 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.127 DEBUG:boto:Params: {} 2022-09-03T06:41:46.129 DEBUG:boto:Token: None 2022-09-03T06:41:46.130 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.131 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H1BLMLjjO8qGZjz09DQSfoyI3S0= 2022-09-03T06:41:46.132 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H1BLMLjjO8qGZjz09DQSfoyI3S0=', 'Content-Length': '0'} 2022-09-03T06:41:46.136 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ef4d33212d7e7222-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.139 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:46.141 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:46.143 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.144 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.144 DEBUG:boto:Method: HEAD 2022-09-03T06:41:46.145 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.146 DEBUG:boto:Data: 2022-09-03T06:41:46.148 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.148 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.149 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.150 DEBUG:boto:Params: {} 2022-09-03T06:41:46.151 DEBUG:boto:Token: None 2022-09-03T06:41:46.151 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.152 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:d1DZoTS+C2BsPbOHHSAro3KonBU= 2022-09-03T06:41:46.153 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:d1DZoTS+C2BsPbOHHSAro3KonBU=', 'Content-Length': '0'} 2022-09-03T06:41:46.155 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dd05b76a5085f77c-006312f72a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.156 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:46.156 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:46.157 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.158 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.159 DEBUG:boto:Method: HEAD 2022-09-03T06:41:46.160 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.161 DEBUG:boto:Data: 2022-09-03T06:41:46.162 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.163 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.164 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.166 DEBUG:boto:Params: {} 2022-09-03T06:41:46.167 DEBUG:boto:Token: None 2022-09-03T06:41:46.168 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.169 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:d1DZoTS+C2BsPbOHHSAro3KonBU= 2022-09-03T06:41:46.170 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:d1DZoTS+C2BsPbOHHSAro3KonBU=', 'Content-Length': '0'} 2022-09-03T06:41:46.174 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f34dd1a526c9d465-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.176 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:41:46.177 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:46.177 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:46.179 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.179 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.180 DEBUG:boto:Method: GET 2022-09-03T06:41:46.181 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.182 DEBUG:boto:Data: 2022-09-03T06:41:46.183 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.184 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.185 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.185 DEBUG:boto:Params: {} 2022-09-03T06:41:46.186 DEBUG:boto:Token: None 2022-09-03T06:41:46.187 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.188 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H1BLMLjjO8qGZjz09DQSfoyI3S0= 2022-09-03T06:41:46.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H1BLMLjjO8qGZjz09DQSfoyI3S0=', 'Content-Length': '0'} 2022-09-03T06:41:46.190 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004068cbab3e84a917-006312f72a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.234 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:46.235 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:46.236 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.237 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.237 DEBUG:boto:Method: GET 2022-09-03T06:41:46.238 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.238 DEBUG:boto:Data: 2022-09-03T06:41:46.239 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.239 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.240 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.240 DEBUG:boto:Params: {} 2022-09-03T06:41:46.241 DEBUG:boto:Token: None 2022-09-03T06:41:46.242 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:46.242 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H1BLMLjjO8qGZjz09DQSfoyI3S0= 2022-09-03T06:41:46.243 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H1BLMLjjO8qGZjz09DQSfoyI3S0=', 'Content-Length': '0'} 2022-09-03T06:41:46.246 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002c23c5ba10860656-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.248 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:46.248 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:46.249 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:46.249 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:46.250 DEBUG:boto:Method: GET 2022-09-03T06:41:46.251 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:41:46.252 DEBUG:boto:Data: 2022-09-03T06:41:46.253 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.254 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.255 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.256 DEBUG:boto:Params: {} 2022-09-03T06:41:46.257 DEBUG:boto:Token: None 2022-09-03T06:41:46.258 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/ 2022-09-03T06:41:46.259 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY= 2022-09-03T06:41:46.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY=', 'Content-Length': '0'} 2022-09-03T06:41:46.265 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d9463dac3fe088b8-006312f72a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.266 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:46.267 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:46.268 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:46.268 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:46.269 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:46.269 DEBUG:boto:Method: GET 2022-09-03T06:41:46.270 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:41:46.270 DEBUG:boto:Data: 2022-09-03T06:41:46.271 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.271 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.272 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.272 DEBUG:boto:Params: {} 2022-09-03T06:41:46.273 DEBUG:boto:Token: None 2022-09-03T06:41:46.274 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/ 2022-09-03T06:41:46.274 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY= 2022-09-03T06:41:46.275 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY=', 'Content-Length': '0'} 2022-09-03T06:41:46.279 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000062ac5826b6eedc15-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.280 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:46.281 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:46.282 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:46.283 DEBUG:boto:Method: HEAD 2022-09-03T06:41:46.284 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:41:46.285 DEBUG:boto:Data: 2022-09-03T06:41:46.286 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.287 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.288 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.290 DEBUG:boto:Params: {} 2022-09-03T06:41:46.291 DEBUG:boto:Token: None 2022-09-03T06:41:46.291 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj38 2022-09-03T06:41:46.292 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:naEWBPHUEEzkmgWoPTH9C2hJpN4= 2022-09-03T06:41:46.293 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:naEWBPHUEEzkmgWoPTH9C2hJpN4=', 'Content-Length': '0'} 2022-09-03T06:41:46.295 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ac7163b65ecff453-006312f72a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.296 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:46.297 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:46.298 DEBUG:boto:Method: HEAD 2022-09-03T06:41:46.301 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:41:46.302 DEBUG:boto:Data: 2022-09-03T06:41:46.303 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.304 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.305 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.306 DEBUG:boto:Params: {} 2022-09-03T06:41:46.307 DEBUG:boto:Token: None 2022-09-03T06:41:46.308 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj38 2022-09-03T06:41:46.309 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:naEWBPHUEEzkmgWoPTH9C2hJpN4= 2022-09-03T06:41:46.310 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:naEWBPHUEEzkmgWoPTH9C2hJpN4=', 'Content-Length': '0'} 2022-09-03T06:41:46.313 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002b42b7df2560f321-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.314 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:41:46.315 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:46.316 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:46.317 DEBUG:boto:Method: GET 2022-09-03T06:41:46.317 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:41:46.318 DEBUG:boto:Data: 2022-09-03T06:41:46.318 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.319 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.320 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.321 DEBUG:boto:Params: {} 2022-09-03T06:41:46.322 DEBUG:boto:Token: None 2022-09-03T06:41:46.323 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj38 2022-09-03T06:41:46.324 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nEuV1R8ZEUduFlmgrDSbSs9l2Mc= 2022-09-03T06:41:46.325 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nEuV1R8ZEUduFlmgrDSbSs9l2Mc=', 'Content-Length': '0'} 2022-09-03T06:41:46.327 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a576ac4e0e7c6f75-006312f72a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.370 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:46.371 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:46.372 DEBUG:boto:Method: GET 2022-09-03T06:41:46.373 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:41:46.374 DEBUG:boto:Data: 2022-09-03T06:41:46.375 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.376 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.376 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.377 DEBUG:boto:Params: {} 2022-09-03T06:41:46.377 DEBUG:boto:Token: None 2022-09-03T06:41:46.378 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj38 2022-09-03T06:41:46.379 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nEuV1R8ZEUduFlmgrDSbSs9l2Mc= 2022-09-03T06:41:46.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nEuV1R8ZEUduFlmgrDSbSs9l2Mc=', 'Content-Length': '0'} 2022-09-03T06:41:46.382 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b723db2fc989b4eb-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.383 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:46.384 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:46.384 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:46.385 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:46.385 DEBUG:boto:Method: GET 2022-09-03T06:41:46.386 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:41:46.387 DEBUG:boto:Data: 2022-09-03T06:41:46.387 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.388 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.388 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.389 DEBUG:boto:Params: {} 2022-09-03T06:41:46.390 DEBUG:boto:Token: None 2022-09-03T06:41:46.390 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/ 2022-09-03T06:41:46.391 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY= 2022-09-03T06:41:46.392 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY=', 'Content-Length': '0'} 2022-09-03T06:41:46.394 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009dd192f5dccaca88-006312f72a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.395 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:46.396 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:46.397 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:46.397 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:46.398 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:46.398 DEBUG:boto:Method: GET 2022-09-03T06:41:46.399 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:41:46.399 DEBUG:boto:Data: 2022-09-03T06:41:46.400 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.400 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.401 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.401 DEBUG:boto:Params: {} 2022-09-03T06:41:46.402 DEBUG:boto:Token: None 2022-09-03T06:41:46.403 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/ 2022-09-03T06:41:46.403 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY= 2022-09-03T06:41:46.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY=', 'Content-Length': '0'} 2022-09-03T06:41:46.407 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009db04d15cc6af7a6-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.409 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:46.411 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:41:46.412 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:46.413 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:46.414 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.415 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.416 DEBUG:boto:Method: GET 2022-09-03T06:41:46.417 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.417 DEBUG:boto:Data: 2022-09-03T06:41:46.419 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.420 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.421 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.422 DEBUG:boto:Params: {} 2022-09-03T06:41:46.423 DEBUG:boto:Token: None 2022-09-03T06:41:46.424 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.426 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/hWBNY3XsyEpGDq1H4cFsOSyO8E= 2022-09-03T06:41:46.427 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/hWBNY3XsyEpGDq1H4cFsOSyO8E=', 'Content-Length': '0'} 2022-09-03T06:41:46.430 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009ded8f66370f54aa-006312f72a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.431 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:46.432 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:46.432 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.433 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.433 DEBUG:boto:Method: GET 2022-09-03T06:41:46.434 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.435 DEBUG:boto:Data: 2022-09-03T06:41:46.435 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.436 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.436 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.437 DEBUG:boto:Params: {} 2022-09-03T06:41:46.437 DEBUG:boto:Token: None 2022-09-03T06:41:46.438 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.439 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/hWBNY3XsyEpGDq1H4cFsOSyO8E= 2022-09-03T06:41:46.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/hWBNY3XsyEpGDq1H4cFsOSyO8E=', 'Content-Length': '0'} 2022-09-03T06:41:46.442 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000055dd96bc363166a9-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.487 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:46.488 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:46.489 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.490 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.491 DEBUG:boto:Method: HEAD 2022-09-03T06:41:46.492 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.493 DEBUG:boto:Data: 2022-09-03T06:41:46.493 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.494 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.494 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.495 DEBUG:boto:Params: {} 2022-09-03T06:41:46.496 DEBUG:boto:Token: None 2022-09-03T06:41:46.496 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.497 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SkCKHNPsXUZu7I+Vd5fIWhKiABQ= 2022-09-03T06:41:46.498 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SkCKHNPsXUZu7I+Vd5fIWhKiABQ=', 'Content-Length': '0'} 2022-09-03T06:41:46.501 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d7b562cd2c0125a7-006312f72a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.502 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:46.502 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:46.503 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.503 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.504 DEBUG:boto:Method: HEAD 2022-09-03T06:41:46.505 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.505 DEBUG:boto:Data: 2022-09-03T06:41:46.506 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.506 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.507 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.508 DEBUG:boto:Params: {} 2022-09-03T06:41:46.508 DEBUG:boto:Token: None 2022-09-03T06:41:46.510 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.510 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SkCKHNPsXUZu7I+Vd5fIWhKiABQ= 2022-09-03T06:41:46.511 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SkCKHNPsXUZu7I+Vd5fIWhKiABQ=', 'Content-Length': '0'} 2022-09-03T06:41:46.513 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006eaac1a2de741be0-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.514 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:41:46.515 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:46.515 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:46.516 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.517 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.517 DEBUG:boto:Method: GET 2022-09-03T06:41:46.518 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.518 DEBUG:boto:Data: 2022-09-03T06:41:46.519 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.520 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.520 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.521 DEBUG:boto:Params: {} 2022-09-03T06:41:46.521 DEBUG:boto:Token: None 2022-09-03T06:41:46.522 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.523 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/hWBNY3XsyEpGDq1H4cFsOSyO8E= 2022-09-03T06:41:46.523 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/hWBNY3XsyEpGDq1H4cFsOSyO8E=', 'Content-Length': '0'} 2022-09-03T06:41:46.526 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007ce02a464e701436-006312f72a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.527 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:46.527 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:46.528 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.528 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.529 DEBUG:boto:Method: GET 2022-09-03T06:41:46.530 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.530 DEBUG:boto:Data: 2022-09-03T06:41:46.531 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.531 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.532 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.532 DEBUG:boto:Params: {} 2022-09-03T06:41:46.533 DEBUG:boto:Token: None 2022-09-03T06:41:46.534 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:46.534 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/hWBNY3XsyEpGDq1H4cFsOSyO8E= 2022-09-03T06:41:46.535 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/hWBNY3XsyEpGDq1H4cFsOSyO8E=', 'Content-Length': '0'} 2022-09-03T06:41:46.538 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008865a0976e9049be-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.538 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:46.539 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:46.540 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:46.540 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:46.541 DEBUG:boto:Method: GET 2022-09-03T06:41:46.541 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:41:46.542 DEBUG:boto:Data: 2022-09-03T06:41:46.543 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.543 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.544 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.544 DEBUG:boto:Params: {} 2022-09-03T06:41:46.545 DEBUG:boto:Token: None 2022-09-03T06:41:46.546 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/ 2022-09-03T06:41:46.546 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY= 2022-09-03T06:41:46.547 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY=', 'Content-Length': '0'} 2022-09-03T06:41:46.550 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a4bb444f721923a4-006312f72a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.550 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:46.551 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:46.552 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:46.553 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:46.553 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:46.554 DEBUG:boto:Method: GET 2022-09-03T06:41:46.554 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:41:46.555 DEBUG:boto:Data: 2022-09-03T06:41:46.555 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.556 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.557 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.557 DEBUG:boto:Params: {} 2022-09-03T06:41:46.558 DEBUG:boto:Token: None 2022-09-03T06:41:46.559 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/ 2022-09-03T06:41:46.560 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY= 2022-09-03T06:41:46.560 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY=', 'Content-Length': '0'} 2022-09-03T06:41:46.564 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f70b8b9b51cabcfd-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.565 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:46.566 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:46.567 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:46.568 DEBUG:boto:Method: HEAD 2022-09-03T06:41:46.568 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:41:46.569 DEBUG:boto:Data: 2022-09-03T06:41:46.570 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.570 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.571 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.571 DEBUG:boto:Params: {} 2022-09-03T06:41:46.572 DEBUG:boto:Token: None 2022-09-03T06:41:46.573 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj44 2022-09-03T06:41:46.574 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FHKSkdJAHQykhOzHz4CIIQ/Gsc8= 2022-09-03T06:41:46.574 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FHKSkdJAHQykhOzHz4CIIQ/Gsc8=', 'Content-Length': '0'} 2022-09-03T06:41:46.576 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e5a37a6808e857bf-006312f72a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.577 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:46.578 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:46.579 DEBUG:boto:Method: HEAD 2022-09-03T06:41:46.579 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:41:46.580 DEBUG:boto:Data: 2022-09-03T06:41:46.580 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.581 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.581 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.582 DEBUG:boto:Params: {} 2022-09-03T06:41:46.583 DEBUG:boto:Token: None 2022-09-03T06:41:46.583 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj44 2022-09-03T06:41:46.584 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FHKSkdJAHQykhOzHz4CIIQ/Gsc8= 2022-09-03T06:41:46.585 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FHKSkdJAHQykhOzHz4CIIQ/Gsc8=', 'Content-Length': '0'} 2022-09-03T06:41:46.588 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f7c1c771c73d02d-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.589 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:41:46.591 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:46.592 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:46.593 DEBUG:boto:Method: GET 2022-09-03T06:41:46.595 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:41:46.595 DEBUG:boto:Data: 2022-09-03T06:41:46.596 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.597 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.598 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.599 DEBUG:boto:Params: {} 2022-09-03T06:41:46.599 DEBUG:boto:Token: None 2022-09-03T06:41:46.600 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj44 2022-09-03T06:41:46.602 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SdqtjoxvDQnLkzLhzGennRULcp0= 2022-09-03T06:41:46.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SdqtjoxvDQnLkzLhzGennRULcp0=', 'Content-Length': '0'} 2022-09-03T06:41:46.605 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eb99993733d132b2-006312f72a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.606 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:46.606 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:46.607 DEBUG:boto:Method: GET 2022-09-03T06:41:46.608 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:41:46.608 DEBUG:boto:Data: 2022-09-03T06:41:46.609 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.609 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.610 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.611 DEBUG:boto:Params: {} 2022-09-03T06:41:46.611 DEBUG:boto:Token: None 2022-09-03T06:41:46.612 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj44 2022-09-03T06:41:46.613 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SdqtjoxvDQnLkzLhzGennRULcp0= 2022-09-03T06:41:46.613 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SdqtjoxvDQnLkzLhzGennRULcp0=', 'Content-Length': '0'} 2022-09-03T06:41:46.616 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000837e569ca8d97128-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.617 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:46.618 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:46.618 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:46.619 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:46.619 DEBUG:boto:Method: GET 2022-09-03T06:41:46.620 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:41:46.621 DEBUG:boto:Data: 2022-09-03T06:41:46.622 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.623 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.624 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.625 DEBUG:boto:Params: {} 2022-09-03T06:41:46.626 DEBUG:boto:Token: None 2022-09-03T06:41:46.627 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/ 2022-09-03T06:41:46.628 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY= 2022-09-03T06:41:46.629 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY=', 'Content-Length': '0'} 2022-09-03T06:41:46.632 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001373083bfc776af1-006312f72a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.633 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:46.635 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:46.635 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:46.636 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:46.637 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:46.637 DEBUG:boto:Method: GET 2022-09-03T06:41:46.638 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:41:46.639 DEBUG:boto:Data: 2022-09-03T06:41:46.639 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.640 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.641 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.642 DEBUG:boto:Params: {} 2022-09-03T06:41:46.643 DEBUG:boto:Token: None 2022-09-03T06:41:46.644 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/ 2022-09-03T06:41:46.645 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY= 2022-09-03T06:41:46.646 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY=', 'Content-Length': '0'} 2022-09-03T06:41:46.650 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000086d58f464211e78-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.651 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:46.652 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:41:46.653 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:46.654 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:46.654 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.655 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.656 DEBUG:boto:Method: GET 2022-09-03T06:41:46.657 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.658 DEBUG:boto:Data: 2022-09-03T06:41:46.659 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.660 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.661 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.662 DEBUG:boto:Params: {} 2022-09-03T06:41:46.663 DEBUG:boto:Token: None 2022-09-03T06:41:46.664 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.665 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UNBpjuFCC9cstulVIyPTB0Ia1FA= 2022-09-03T06:41:46.666 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UNBpjuFCC9cstulVIyPTB0Ia1FA=', 'Content-Length': '0'} 2022-09-03T06:41:46.669 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d6a3a608c9480641-006312f72a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.670 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:46.671 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:46.671 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.672 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.672 DEBUG:boto:Method: GET 2022-09-03T06:41:46.673 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.673 DEBUG:boto:Data: 2022-09-03T06:41:46.674 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.675 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.675 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.676 DEBUG:boto:Params: {} 2022-09-03T06:41:46.676 DEBUG:boto:Token: None 2022-09-03T06:41:46.677 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.678 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UNBpjuFCC9cstulVIyPTB0Ia1FA= 2022-09-03T06:41:46.678 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UNBpjuFCC9cstulVIyPTB0Ia1FA=', 'Content-Length': '0'} 2022-09-03T06:41:46.681 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000019bc76a2e1325f0-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.683 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:46.683 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:46.684 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.685 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.685 DEBUG:boto:Method: HEAD 2022-09-03T06:41:46.686 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.687 DEBUG:boto:Data: 2022-09-03T06:41:46.687 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.688 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.688 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.689 DEBUG:boto:Params: {} 2022-09-03T06:41:46.690 DEBUG:boto:Token: None 2022-09-03T06:41:46.690 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.691 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qq4PZrc9CMuhM5s51RcWvZbfPHk= 2022-09-03T06:41:46.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qq4PZrc9CMuhM5s51RcWvZbfPHk=', 'Content-Length': '0'} 2022-09-03T06:41:46.695 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000002da41c582bf582a-006312f72a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.696 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:46.697 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:46.698 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.699 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.700 DEBUG:boto:Method: HEAD 2022-09-03T06:41:46.701 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.702 DEBUG:boto:Data: 2022-09-03T06:41:46.703 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.704 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.705 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.706 DEBUG:boto:Params: {} 2022-09-03T06:41:46.707 DEBUG:boto:Token: None 2022-09-03T06:41:46.708 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.709 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qq4PZrc9CMuhM5s51RcWvZbfPHk= 2022-09-03T06:41:46.710 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qq4PZrc9CMuhM5s51RcWvZbfPHk=', 'Content-Length': '0'} 2022-09-03T06:41:46.714 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006438c51d0548806d-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.715 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:41:46.715 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:46.716 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:46.716 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.717 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.718 DEBUG:boto:Method: GET 2022-09-03T06:41:46.718 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.719 DEBUG:boto:Data: 2022-09-03T06:41:46.719 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.720 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.720 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.721 DEBUG:boto:Params: {} 2022-09-03T06:41:46.722 DEBUG:boto:Token: None 2022-09-03T06:41:46.722 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.730 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UNBpjuFCC9cstulVIyPTB0Ia1FA= 2022-09-03T06:41:46.731 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UNBpjuFCC9cstulVIyPTB0Ia1FA=', 'Content-Length': '0'} 2022-09-03T06:41:46.734 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000475dc8cb9edd7d1f-006312f72a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.778 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:46.779 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:46.780 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.780 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.781 DEBUG:boto:Method: GET 2022-09-03T06:41:46.781 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.782 DEBUG:boto:Data: 2022-09-03T06:41:46.783 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.784 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.784 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.785 DEBUG:boto:Params: {} 2022-09-03T06:41:46.786 DEBUG:boto:Token: None 2022-09-03T06:41:46.787 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:46.787 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UNBpjuFCC9cstulVIyPTB0Ia1FA= 2022-09-03T06:41:46.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UNBpjuFCC9cstulVIyPTB0Ia1FA=', 'Content-Length': '0'} 2022-09-03T06:41:46.791 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a8601609d70ff524-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.792 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:46.792 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:46.793 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:46.793 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:46.794 DEBUG:boto:Method: GET 2022-09-03T06:41:46.795 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:41:46.795 DEBUG:boto:Data: 2022-09-03T06:41:46.796 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.796 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.797 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.797 DEBUG:boto:Params: {} 2022-09-03T06:41:46.798 DEBUG:boto:Token: None 2022-09-03T06:41:46.799 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/ 2022-09-03T06:41:46.800 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY= 2022-09-03T06:41:46.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY=', 'Content-Length': '0'} 2022-09-03T06:41:46.803 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007c463170d798e561-006312f72a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.804 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:46.805 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:46.805 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:46.806 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:46.806 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:46.807 DEBUG:boto:Method: GET 2022-09-03T06:41:46.808 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:41:46.808 DEBUG:boto:Data: 2022-09-03T06:41:46.809 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.809 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.810 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.810 DEBUG:boto:Params: {} 2022-09-03T06:41:46.811 DEBUG:boto:Token: None 2022-09-03T06:41:46.812 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/ 2022-09-03T06:41:46.812 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY= 2022-09-03T06:41:46.813 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY=', 'Content-Length': '0'} 2022-09-03T06:41:46.816 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dd6d8a178f57a68e-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.858 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:46.860 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:46.860 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:46.861 DEBUG:boto:Method: HEAD 2022-09-03T06:41:46.861 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:41:46.863 DEBUG:boto:Data: 2022-09-03T06:41:46.864 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.865 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.865 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.866 DEBUG:boto:Params: {} 2022-09-03T06:41:46.867 DEBUG:boto:Token: None 2022-09-03T06:41:46.868 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj51 2022-09-03T06:41:46.869 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5g8AVkumsEFbD5BL63oT7iFV3Ao= 2022-09-03T06:41:46.870 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5g8AVkumsEFbD5BL63oT7iFV3Ao=', 'Content-Length': '0'} 2022-09-03T06:41:46.872 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ead711bef043a13b-006312f72a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.873 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:46.874 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:46.875 DEBUG:boto:Method: HEAD 2022-09-03T06:41:46.875 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:41:46.876 DEBUG:boto:Data: 2022-09-03T06:41:46.876 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.877 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.878 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.878 DEBUG:boto:Params: {} 2022-09-03T06:41:46.879 DEBUG:boto:Token: None 2022-09-03T06:41:46.880 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj51 2022-09-03T06:41:46.881 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5g8AVkumsEFbD5BL63oT7iFV3Ao= 2022-09-03T06:41:46.881 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5g8AVkumsEFbD5BL63oT7iFV3Ao=', 'Content-Length': '0'} 2022-09-03T06:41:46.883 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a65c61955cabd02-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.884 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:41:46.885 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:46.886 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:46.887 DEBUG:boto:Method: GET 2022-09-03T06:41:46.888 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:41:46.889 DEBUG:boto:Data: 2022-09-03T06:41:46.889 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.890 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.891 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.891 DEBUG:boto:Params: {} 2022-09-03T06:41:46.892 DEBUG:boto:Token: None 2022-09-03T06:41:46.893 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj51 2022-09-03T06:41:46.894 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VEo7B0TqYZi7/ROLVY3kPN6R8kI= 2022-09-03T06:41:46.894 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VEo7B0TqYZi7/ROLVY3kPN6R8kI=', 'Content-Length': '0'} 2022-09-03T06:41:46.897 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ae27778189928fc2-006312f72a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.897 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:46.898 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:46.899 DEBUG:boto:Method: GET 2022-09-03T06:41:46.899 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:41:46.900 DEBUG:boto:Data: 2022-09-03T06:41:46.900 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.901 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.902 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.902 DEBUG:boto:Params: {} 2022-09-03T06:41:46.903 DEBUG:boto:Token: None 2022-09-03T06:41:46.904 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj51 2022-09-03T06:41:46.904 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VEo7B0TqYZi7/ROLVY3kPN6R8kI= 2022-09-03T06:41:46.905 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VEo7B0TqYZi7/ROLVY3kPN6R8kI=', 'Content-Length': '0'} 2022-09-03T06:41:46.920 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c3cba12bcc593265-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.921 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:46.921 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:46.922 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:46.923 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:46.923 DEBUG:boto:Method: GET 2022-09-03T06:41:46.924 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:41:46.924 DEBUG:boto:Data: 2022-09-03T06:41:46.925 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.925 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.926 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.927 DEBUG:boto:Params: {} 2022-09-03T06:41:46.930 DEBUG:boto:Token: None 2022-09-03T06:41:46.931 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/ 2022-09-03T06:41:46.932 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY= 2022-09-03T06:41:46.933 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY=', 'Content-Length': '0'} 2022-09-03T06:41:46.936 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a2e19b5887b7e558-006312f72a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.937 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:46.938 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:46.939 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:46.939 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:46.940 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:46.941 DEBUG:boto:Method: GET 2022-09-03T06:41:46.941 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:41:46.942 DEBUG:boto:Data: 2022-09-03T06:41:46.942 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.943 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.943 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.944 DEBUG:boto:Params: {} 2022-09-03T06:41:46.945 DEBUG:boto:Token: None 2022-09-03T06:41:46.946 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/ 2022-09-03T06:41:46.946 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY= 2022-09-03T06:41:46.947 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ryOh/uXYsnE8KmIdR4iATVDOWSY=', 'Content-Length': '0'} 2022-09-03T06:41:46.951 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009cc46abd53683f9f-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.952 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:46.953 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:41:46.953 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:46.954 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:46.955 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:46.955 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:46.956 DEBUG:boto:Method: GET 2022-09-03T06:41:46.957 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:46.957 DEBUG:boto:Data: 2022-09-03T06:41:46.958 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.958 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:46.959 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.959 DEBUG:boto:Params: {} 2022-09-03T06:41:46.960 DEBUG:boto:Token: None 2022-09-03T06:41:46.961 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:46.961 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e/ooJgtnYU27bNZV6hGCB1mIUKU= 2022-09-03T06:41:46.962 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e/ooJgtnYU27bNZV6hGCB1mIUKU=', 'Content-Length': '0'} 2022-09-03T06:41:46.965 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000117adeb06d790419-006312f72a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:46.966 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:46.966 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:46.967 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:46.968 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:46.968 DEBUG:boto:Method: GET 2022-09-03T06:41:46.969 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:46.969 DEBUG:boto:Data: 2022-09-03T06:41:46.970 DEBUG:boto:Headers: {} 2022-09-03T06:41:46.970 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:46.971 DEBUG:boto:Port: 8000 2022-09-03T06:41:46.971 DEBUG:boto:Params: {} 2022-09-03T06:41:46.972 DEBUG:boto:Token: None 2022-09-03T06:41:46.973 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:46 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:46.973 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e/ooJgtnYU27bNZV6hGCB1mIUKU= 2022-09-03T06:41:46.974 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e/ooJgtnYU27bNZV6hGCB1mIUKU=', 'Content-Length': '0'} 2022-09-03T06:41:46.977 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000090d6cb62482c9215-006312f72a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.018 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:47.019 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:47.020 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:47.021 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:47.022 DEBUG:boto:Method: HEAD 2022-09-03T06:41:47.023 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:47.024 DEBUG:boto:Data: 2022-09-03T06:41:47.025 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.026 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:47.027 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.028 DEBUG:boto:Params: {} 2022-09-03T06:41:47.029 DEBUG:boto:Token: None 2022-09-03T06:41:47.030 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:47.032 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UaGwXDsJ1iHIZJsiBTuz1LWEtUY= 2022-09-03T06:41:47.033 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UaGwXDsJ1iHIZJsiBTuz1LWEtUY=', 'Content-Length': '0'} 2022-09-03T06:41:47.036 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e834306c53e10388-006312f72b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.038 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:47.039 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:47.040 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:47.040 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:47.041 DEBUG:boto:Method: HEAD 2022-09-03T06:41:47.042 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:47.043 DEBUG:boto:Data: 2022-09-03T06:41:47.044 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.045 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:47.046 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.047 DEBUG:boto:Params: {} 2022-09-03T06:41:47.048 DEBUG:boto:Token: None 2022-09-03T06:41:47.050 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:47.051 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UaGwXDsJ1iHIZJsiBTuz1LWEtUY= 2022-09-03T06:41:47.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UaGwXDsJ1iHIZJsiBTuz1LWEtUY=', 'Content-Length': '0'} 2022-09-03T06:41:47.055 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000294813cfd0f9961e-006312f72b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.057 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:41:47.058 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:47.059 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:47.060 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:47.061 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:47.062 DEBUG:boto:Method: GET 2022-09-03T06:41:47.063 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:47.064 DEBUG:boto:Data: 2022-09-03T06:41:47.065 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.066 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:47.067 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.068 DEBUG:boto:Params: {} 2022-09-03T06:41:47.069 DEBUG:boto:Token: None 2022-09-03T06:41:47.070 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:47.071 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pYeFD2cJlfQJE2qW0rXa7obULww= 2022-09-03T06:41:47.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pYeFD2cJlfQJE2qW0rXa7obULww=', 'Content-Length': '0'} 2022-09-03T06:41:47.075 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e689c86f25687bb1-006312f72b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.077 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:47.078 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:47.079 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:47.080 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:47.080 DEBUG:boto:Method: GET 2022-09-03T06:41:47.081 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:47.081 DEBUG:boto:Data: 2022-09-03T06:41:47.082 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.082 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:47.083 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.084 DEBUG:boto:Params: {} 2022-09-03T06:41:47.084 DEBUG:boto:Token: None 2022-09-03T06:41:47.085 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:47.085 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pYeFD2cJlfQJE2qW0rXa7obULww= 2022-09-03T06:41:47.086 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pYeFD2cJlfQJE2qW0rXa7obULww=', 'Content-Length': '0'} 2022-09-03T06:41:47.088 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008dd51a608c1b5e6b-006312f72b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.089 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:47.090 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:47.090 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:47.091 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:47.092 DEBUG:boto:Method: GET 2022-09-03T06:41:47.092 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:41:47.093 DEBUG:boto:Data: 2022-09-03T06:41:47.093 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.094 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:47.095 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.095 DEBUG:boto:Params: {} 2022-09-03T06:41:47.096 DEBUG:boto:Token: None 2022-09-03T06:41:47.097 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/ 2022-09-03T06:41:47.097 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38= 2022-09-03T06:41:47.098 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38=', 'Content-Length': '0'} 2022-09-03T06:41:47.101 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000085400982c962db71-006312f72b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.102 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:47.102 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:47.103 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:47.104 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:47.104 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:47.105 DEBUG:boto:Method: GET 2022-09-03T06:41:47.105 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:41:47.106 DEBUG:boto:Data: 2022-09-03T06:41:47.106 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.107 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:47.107 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.108 DEBUG:boto:Params: {} 2022-09-03T06:41:47.108 DEBUG:boto:Token: None 2022-09-03T06:41:47.109 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/ 2022-09-03T06:41:47.110 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38= 2022-09-03T06:41:47.110 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38=', 'Content-Length': '0'} 2022-09-03T06:41:47.114 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aaf642766f34964e-006312f72b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.115 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:47.117 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:47.118 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:47.119 DEBUG:boto:Method: HEAD 2022-09-03T06:41:47.120 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:41:47.121 DEBUG:boto:Data: 2022-09-03T06:41:47.121 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.122 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:47.123 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.124 DEBUG:boto:Params: {} 2022-09-03T06:41:47.125 DEBUG:boto:Token: None 2022-09-03T06:41:47.126 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj55 2022-09-03T06:41:47.127 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Dp1+nL8EcCzTHwfcHnF8+UnANTQ= 2022-09-03T06:41:47.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Dp1+nL8EcCzTHwfcHnF8+UnANTQ=', 'Content-Length': '0'} 2022-09-03T06:41:47.131 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000012b0b6a7548cd7bd-006312f72b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.133 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:47.133 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:47.135 DEBUG:boto:Method: HEAD 2022-09-03T06:41:47.135 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:41:47.136 DEBUG:boto:Data: 2022-09-03T06:41:47.136 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.137 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:47.138 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.138 DEBUG:boto:Params: {} 2022-09-03T06:41:47.139 DEBUG:boto:Token: None 2022-09-03T06:41:47.140 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj55 2022-09-03T06:41:47.140 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Dp1+nL8EcCzTHwfcHnF8+UnANTQ= 2022-09-03T06:41:47.141 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Dp1+nL8EcCzTHwfcHnF8+UnANTQ=', 'Content-Length': '0'} 2022-09-03T06:41:47.144 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000055690c0df18a0440-006312f72b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.144 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:41:47.145 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:47.146 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:47.146 DEBUG:boto:Method: GET 2022-09-03T06:41:47.147 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:41:47.147 DEBUG:boto:Data: 2022-09-03T06:41:47.148 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.149 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:47.149 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.150 DEBUG:boto:Params: {} 2022-09-03T06:41:47.151 DEBUG:boto:Token: None 2022-09-03T06:41:47.151 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj55 2022-09-03T06:41:47.152 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4NzxzJ2QI8REiDYmWJbPl/jko0o= 2022-09-03T06:41:47.153 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4NzxzJ2QI8REiDYmWJbPl/jko0o=', 'Content-Length': '0'} 2022-09-03T06:41:47.155 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007f080e9b57cf95fe-006312f72b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.156 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:47.156 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:47.157 DEBUG:boto:Method: GET 2022-09-03T06:41:47.158 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:41:47.158 DEBUG:boto:Data: 2022-09-03T06:41:47.159 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.159 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:47.160 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.161 DEBUG:boto:Params: {} 2022-09-03T06:41:47.162 DEBUG:boto:Token: None 2022-09-03T06:41:47.162 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj55 2022-09-03T06:41:47.163 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4NzxzJ2QI8REiDYmWJbPl/jko0o= 2022-09-03T06:41:47.163 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4NzxzJ2QI8REiDYmWJbPl/jko0o=', 'Content-Length': '0'} 2022-09-03T06:41:47.166 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000df63f59ab996c56f-006312f72b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.206 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:47.207 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:47.208 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:47.209 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:47.214 DEBUG:boto:Method: GET 2022-09-03T06:41:47.215 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:41:47.215 DEBUG:boto:Data: 2022-09-03T06:41:47.216 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.217 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:47.218 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.219 DEBUG:boto:Params: {} 2022-09-03T06:41:47.220 DEBUG:boto:Token: None 2022-09-03T06:41:47.221 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/ 2022-09-03T06:41:47.222 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38= 2022-09-03T06:41:47.223 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38=', 'Content-Length': '0'} 2022-09-03T06:41:47.227 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d085c93abca880c9-006312f72b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.228 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:47.229 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:47.230 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:47.231 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:47.232 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:47.232 DEBUG:boto:Method: GET 2022-09-03T06:41:47.233 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:41:47.234 DEBUG:boto:Data: 2022-09-03T06:41:47.234 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.235 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:47.235 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.236 DEBUG:boto:Params: {} 2022-09-03T06:41:47.237 DEBUG:boto:Token: None 2022-09-03T06:41:47.237 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/ 2022-09-03T06:41:47.238 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38= 2022-09-03T06:41:47.239 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38=', 'Content-Length': '0'} 2022-09-03T06:41:47.242 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000035bea72c98a8d2c9-006312f72b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.243 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:47.244 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:41:47.244 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:47.245 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:47.245 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.246 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.247 DEBUG:boto:Method: GET 2022-09-03T06:41:47.247 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.248 DEBUG:boto:Data: 2022-09-03T06:41:47.248 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.249 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:47.250 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.250 DEBUG:boto:Params: {} 2022-09-03T06:41:47.251 DEBUG:boto:Token: None 2022-09-03T06:41:47.251 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.252 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sbykReqneWUl/f+sCQG2CM2yZqM= 2022-09-03T06:41:47.253 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sbykReqneWUl/f+sCQG2CM2yZqM=', 'Content-Length': '0'} 2022-09-03T06:41:47.255 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c5c31dae029d90b-006312f72b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.298 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:47.300 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:47.301 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.302 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.303 DEBUG:boto:Method: GET 2022-09-03T06:41:47.304 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.305 DEBUG:boto:Data: 2022-09-03T06:41:47.307 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.308 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:47.309 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.310 DEBUG:boto:Params: {} 2022-09-03T06:41:47.311 DEBUG:boto:Token: None 2022-09-03T06:41:47.312 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.313 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sbykReqneWUl/f+sCQG2CM2yZqM= 2022-09-03T06:41:47.315 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sbykReqneWUl/f+sCQG2CM2yZqM=', 'Content-Length': '0'} 2022-09-03T06:41:47.317 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c78ed89dc8269335-006312f72b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.319 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:47.320 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:47.320 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.321 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.322 DEBUG:boto:Method: HEAD 2022-09-03T06:41:47.323 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.324 DEBUG:boto:Data: 2022-09-03T06:41:47.325 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.325 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:47.327 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.327 DEBUG:boto:Params: {} 2022-09-03T06:41:47.328 DEBUG:boto:Token: None 2022-09-03T06:41:47.329 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.329 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:okDpzJHnicbDR1/6LmDyCos3REc= 2022-09-03T06:41:47.331 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:okDpzJHnicbDR1/6LmDyCos3REc=', 'Content-Length': '0'} 2022-09-03T06:41:47.335 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006365bc29cab34eb1-006312f72b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.336 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:47.337 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:47.338 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.339 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.340 DEBUG:boto:Method: HEAD 2022-09-03T06:41:47.341 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.342 DEBUG:boto:Data: 2022-09-03T06:41:47.343 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.344 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:47.345 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.346 DEBUG:boto:Params: {} 2022-09-03T06:41:47.347 DEBUG:boto:Token: None 2022-09-03T06:41:47.348 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.349 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:okDpzJHnicbDR1/6LmDyCos3REc= 2022-09-03T06:41:47.350 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:okDpzJHnicbDR1/6LmDyCos3REc=', 'Content-Length': '0'} 2022-09-03T06:41:47.353 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aa61060f66286aef-006312f72b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.354 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:41:47.355 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:47.356 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:47.357 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.358 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.366 DEBUG:boto:Method: GET 2022-09-03T06:41:47.368 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.369 DEBUG:boto:Data: 2022-09-03T06:41:47.370 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.371 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:47.372 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.373 DEBUG:boto:Params: {} 2022-09-03T06:41:47.375 DEBUG:boto:Token: None 2022-09-03T06:41:47.376 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.377 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sbykReqneWUl/f+sCQG2CM2yZqM= 2022-09-03T06:41:47.378 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sbykReqneWUl/f+sCQG2CM2yZqM=', 'Content-Length': '0'} 2022-09-03T06:41:47.381 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d65af4b10184ba86-006312f72b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.382 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:47.383 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:47.384 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.385 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.386 DEBUG:boto:Method: GET 2022-09-03T06:41:47.386 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.387 DEBUG:boto:Data: 2022-09-03T06:41:47.387 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.388 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:47.388 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.389 DEBUG:boto:Params: {} 2022-09-03T06:41:47.390 DEBUG:boto:Token: None 2022-09-03T06:41:47.391 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:47.393 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sbykReqneWUl/f+sCQG2CM2yZqM= 2022-09-03T06:41:47.394 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sbykReqneWUl/f+sCQG2CM2yZqM=', 'Content-Length': '0'} 2022-09-03T06:41:47.402 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c4bdab37ed7710f-006312f72b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.404 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:47.404 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:47.405 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:47.405 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:47.406 DEBUG:boto:Method: GET 2022-09-03T06:41:47.407 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:41:47.408 DEBUG:boto:Data: 2022-09-03T06:41:47.409 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.409 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:47.410 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.411 DEBUG:boto:Params: {} 2022-09-03T06:41:47.413 DEBUG:boto:Token: None 2022-09-03T06:41:47.414 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/ 2022-09-03T06:41:47.415 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38= 2022-09-03T06:41:47.417 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38=', 'Content-Length': '0'} 2022-09-03T06:41:47.420 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f16e59e2c1e8ee66-006312f72b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.421 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:47.422 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:47.423 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:47.424 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:47.425 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:47.426 DEBUG:boto:Method: GET 2022-09-03T06:41:47.427 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:41:47.428 DEBUG:boto:Data: 2022-09-03T06:41:47.429 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.430 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:47.431 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.433 DEBUG:boto:Params: {} 2022-09-03T06:41:47.435 DEBUG:boto:Token: None 2022-09-03T06:41:47.436 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/ 2022-09-03T06:41:47.437 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38= 2022-09-03T06:41:47.438 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38=', 'Content-Length': '0'} 2022-09-03T06:41:47.443 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004b026d28ec90dc63-006312f72b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.444 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:47.445 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:47.445 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:47.446 DEBUG:boto:Method: HEAD 2022-09-03T06:41:47.447 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:41:47.447 DEBUG:boto:Data: 2022-09-03T06:41:47.448 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.448 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:47.449 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.449 DEBUG:boto:Params: {} 2022-09-03T06:41:47.450 DEBUG:boto:Token: None 2022-09-03T06:41:47.451 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj57 2022-09-03T06:41:47.451 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gmtL0+o/AC0nIvMQhTjUkZ9WAo4= 2022-09-03T06:41:47.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gmtL0+o/AC0nIvMQhTjUkZ9WAo4=', 'Content-Length': '0'} 2022-09-03T06:41:47.454 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c091bdebbe53be96-006312f72b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.455 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:47.456 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:47.457 DEBUG:boto:Method: HEAD 2022-09-03T06:41:47.457 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:41:47.458 DEBUG:boto:Data: 2022-09-03T06:41:47.458 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.459 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:47.459 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.462 DEBUG:boto:Params: {} 2022-09-03T06:41:47.462 DEBUG:boto:Token: None 2022-09-03T06:41:47.463 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj57 2022-09-03T06:41:47.464 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gmtL0+o/AC0nIvMQhTjUkZ9WAo4= 2022-09-03T06:41:47.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gmtL0+o/AC0nIvMQhTjUkZ9WAo4=', 'Content-Length': '0'} 2022-09-03T06:41:47.466 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c3400da93784c69d-006312f72b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.467 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:41:47.468 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:47.469 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:47.469 DEBUG:boto:Method: GET 2022-09-03T06:41:47.470 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:41:47.471 DEBUG:boto:Data: 2022-09-03T06:41:47.471 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.472 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:47.472 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.473 DEBUG:boto:Params: {} 2022-09-03T06:41:47.473 DEBUG:boto:Token: None 2022-09-03T06:41:47.474 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj57 2022-09-03T06:41:47.475 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ojOGuVp0lDR+sLG7GAmi2Ne56O0= 2022-09-03T06:41:47.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ojOGuVp0lDR+sLG7GAmi2Ne56O0=', 'Content-Length': '0'} 2022-09-03T06:41:47.479 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c91761c997cfbf80-006312f72b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.522 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:47.523 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:47.524 DEBUG:boto:Method: GET 2022-09-03T06:41:47.525 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:41:47.525 DEBUG:boto:Data: 2022-09-03T06:41:47.526 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.527 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:47.527 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.528 DEBUG:boto:Params: {} 2022-09-03T06:41:47.528 DEBUG:boto:Token: None 2022-09-03T06:41:47.529 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj57 2022-09-03T06:41:47.530 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ojOGuVp0lDR+sLG7GAmi2Ne56O0= 2022-09-03T06:41:47.531 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ojOGuVp0lDR+sLG7GAmi2Ne56O0=', 'Content-Length': '0'} 2022-09-03T06:41:47.533 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000014f14f1c093c8939-006312f72b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.534 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:47.535 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:47.536 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:47.537 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:47.538 DEBUG:boto:Method: GET 2022-09-03T06:41:47.539 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:41:47.540 DEBUG:boto:Data: 2022-09-03T06:41:47.540 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.541 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:47.542 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.543 DEBUG:boto:Params: {} 2022-09-03T06:41:47.544 DEBUG:boto:Token: None 2022-09-03T06:41:47.544 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/ 2022-09-03T06:41:47.545 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38= 2022-09-03T06:41:47.545 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38=', 'Content-Length': '0'} 2022-09-03T06:41:47.548 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000732e26b3e9270a6b-006312f72b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.549 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:47.550 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:47.551 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:47.552 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:47.552 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:47.553 DEBUG:boto:Method: GET 2022-09-03T06:41:47.554 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:41:47.555 DEBUG:boto:Data: 2022-09-03T06:41:47.556 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.556 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:47.557 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.558 DEBUG:boto:Params: {} 2022-09-03T06:41:47.559 DEBUG:boto:Token: None 2022-09-03T06:41:47.559 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/ 2022-09-03T06:41:47.560 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38= 2022-09-03T06:41:47.561 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38=', 'Content-Length': '0'} 2022-09-03T06:41:47.564 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b63b040a602352a5-006312f72b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.565 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:47.567 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:41:47.568 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:47.568 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:47.569 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.569 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.570 DEBUG:boto:Method: GET 2022-09-03T06:41:47.570 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.571 DEBUG:boto:Data: 2022-09-03T06:41:47.572 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.573 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:47.574 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.575 DEBUG:boto:Params: {} 2022-09-03T06:41:47.576 DEBUG:boto:Token: None 2022-09-03T06:41:47.577 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.578 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F7CmyCi4C6Ty1t+IYKNF8RlpTgE= 2022-09-03T06:41:47.579 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F7CmyCi4C6Ty1t+IYKNF8RlpTgE=', 'Content-Length': '0'} 2022-09-03T06:41:47.582 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006cfb17cdfdc50393-006312f72b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.584 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:47.585 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:47.585 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.586 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.587 DEBUG:boto:Method: GET 2022-09-03T06:41:47.588 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.589 DEBUG:boto:Data: 2022-09-03T06:41:47.590 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.591 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:47.592 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.593 DEBUG:boto:Params: {} 2022-09-03T06:41:47.594 DEBUG:boto:Token: None 2022-09-03T06:41:47.595 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.596 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F7CmyCi4C6Ty1t+IYKNF8RlpTgE= 2022-09-03T06:41:47.597 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F7CmyCi4C6Ty1t+IYKNF8RlpTgE=', 'Content-Length': '0'} 2022-09-03T06:41:47.600 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ae176eb6d07065a9-006312f72b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.602 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:47.603 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:47.604 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.605 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.606 DEBUG:boto:Method: HEAD 2022-09-03T06:41:47.607 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.608 DEBUG:boto:Data: 2022-09-03T06:41:47.608 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.609 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:47.610 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.610 DEBUG:boto:Params: {} 2022-09-03T06:41:47.611 DEBUG:boto:Token: None 2022-09-03T06:41:47.611 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.612 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C0f+jsQpqGMYSVvmqmMa8Sak4BU= 2022-09-03T06:41:47.613 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C0f+jsQpqGMYSVvmqmMa8Sak4BU=', 'Content-Length': '0'} 2022-09-03T06:41:47.615 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004025b15054a27984-006312f72b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.615 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:47.616 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:47.616 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.617 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.617 DEBUG:boto:Method: HEAD 2022-09-03T06:41:47.618 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.618 DEBUG:boto:Data: 2022-09-03T06:41:47.619 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.620 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:47.620 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.621 DEBUG:boto:Params: {} 2022-09-03T06:41:47.621 DEBUG:boto:Token: None 2022-09-03T06:41:47.622 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.622 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C0f+jsQpqGMYSVvmqmMa8Sak4BU= 2022-09-03T06:41:47.623 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C0f+jsQpqGMYSVvmqmMa8Sak4BU=', 'Content-Length': '0'} 2022-09-03T06:41:47.626 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e69a9da94762cc86-006312f72b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.627 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:41:47.628 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:47.629 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:47.630 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.631 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.632 DEBUG:boto:Method: GET 2022-09-03T06:41:47.633 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.633 DEBUG:boto:Data: 2022-09-03T06:41:47.634 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.635 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:47.636 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.637 DEBUG:boto:Params: {} 2022-09-03T06:41:47.638 DEBUG:boto:Token: None 2022-09-03T06:41:47.639 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.641 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F7CmyCi4C6Ty1t+IYKNF8RlpTgE= 2022-09-03T06:41:47.642 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F7CmyCi4C6Ty1t+IYKNF8RlpTgE=', 'Content-Length': '0'} 2022-09-03T06:41:47.644 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd061f3cc5c40db6-006312f72b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.686 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:47.687 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:47.688 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.688 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.689 DEBUG:boto:Method: GET 2022-09-03T06:41:47.689 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.690 DEBUG:boto:Data: 2022-09-03T06:41:47.691 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.691 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:47.692 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.692 DEBUG:boto:Params: {} 2022-09-03T06:41:47.693 DEBUG:boto:Token: None 2022-09-03T06:41:47.694 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:47.695 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F7CmyCi4C6Ty1t+IYKNF8RlpTgE= 2022-09-03T06:41:47.695 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F7CmyCi4C6Ty1t+IYKNF8RlpTgE=', 'Content-Length': '0'} 2022-09-03T06:41:47.697 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001bd4cd1ff07b0972-006312f72b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.698 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:47.699 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:47.699 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:47.700 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:47.701 DEBUG:boto:Method: GET 2022-09-03T06:41:47.701 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:41:47.702 DEBUG:boto:Data: 2022-09-03T06:41:47.702 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.703 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:47.703 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.704 DEBUG:boto:Params: {} 2022-09-03T06:41:47.705 DEBUG:boto:Token: None 2022-09-03T06:41:47.706 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/ 2022-09-03T06:41:47.707 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38= 2022-09-03T06:41:47.707 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38=', 'Content-Length': '0'} 2022-09-03T06:41:47.716 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a0be35894c6346ab-006312f72b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.717 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:47.718 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:47.719 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:47.722 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:47.723 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:47.724 DEBUG:boto:Method: GET 2022-09-03T06:41:47.726 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:41:47.727 DEBUG:boto:Data: 2022-09-03T06:41:47.728 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.728 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:47.729 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.729 DEBUG:boto:Params: {} 2022-09-03T06:41:47.731 DEBUG:boto:Token: None 2022-09-03T06:41:47.731 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/ 2022-09-03T06:41:47.732 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38= 2022-09-03T06:41:47.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38=', 'Content-Length': '0'} 2022-09-03T06:41:47.736 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000af82cd512ae28da1-006312f72b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.737 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:47.738 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:47.739 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:47.740 DEBUG:boto:Method: HEAD 2022-09-03T06:41:47.741 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:41:47.742 DEBUG:boto:Data: 2022-09-03T06:41:47.743 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.744 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:47.745 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.746 DEBUG:boto:Params: {} 2022-09-03T06:41:47.747 DEBUG:boto:Token: None 2022-09-03T06:41:47.748 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj68 2022-09-03T06:41:47.750 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LEWz5FC2sGD4TbYLScGJSzEGQrU= 2022-09-03T06:41:47.751 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LEWz5FC2sGD4TbYLScGJSzEGQrU=', 'Content-Length': '0'} 2022-09-03T06:41:47.753 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aa6bc7b8ea5e2279-006312f72b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.754 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:47.755 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:47.759 DEBUG:boto:Method: HEAD 2022-09-03T06:41:47.764 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:41:47.765 DEBUG:boto:Data: 2022-09-03T06:41:47.766 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.767 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:47.767 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.768 DEBUG:boto:Params: {} 2022-09-03T06:41:47.769 DEBUG:boto:Token: None 2022-09-03T06:41:47.770 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj68 2022-09-03T06:41:47.771 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LEWz5FC2sGD4TbYLScGJSzEGQrU= 2022-09-03T06:41:47.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LEWz5FC2sGD4TbYLScGJSzEGQrU=', 'Content-Length': '0'} 2022-09-03T06:41:47.782 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002c5ba074b598aff8-006312f72b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.784 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:41:47.785 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:47.786 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:47.787 DEBUG:boto:Method: GET 2022-09-03T06:41:47.788 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:41:47.789 DEBUG:boto:Data: 2022-09-03T06:41:47.790 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.791 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:47.792 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.793 DEBUG:boto:Params: {} 2022-09-03T06:41:47.794 DEBUG:boto:Token: None 2022-09-03T06:41:47.795 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj68 2022-09-03T06:41:47.796 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ef3XxNtdGaezwn6DH2iGGc+z1go= 2022-09-03T06:41:47.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ef3XxNtdGaezwn6DH2iGGc+z1go=', 'Content-Length': '0'} 2022-09-03T06:41:47.804 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000027b3da1502967517-006312f72b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.806 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:47.806 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:47.807 DEBUG:boto:Method: GET 2022-09-03T06:41:47.808 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:41:47.808 DEBUG:boto:Data: 2022-09-03T06:41:47.809 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.809 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:47.810 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.810 DEBUG:boto:Params: {} 2022-09-03T06:41:47.811 DEBUG:boto:Token: None 2022-09-03T06:41:47.812 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj68 2022-09-03T06:41:47.813 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ef3XxNtdGaezwn6DH2iGGc+z1go= 2022-09-03T06:41:47.814 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ef3XxNtdGaezwn6DH2iGGc+z1go=', 'Content-Length': '0'} 2022-09-03T06:41:47.816 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000036f9b4d52bb841e7-006312f72b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.858 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:47.860 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:47.861 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:47.861 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:47.862 DEBUG:boto:Method: GET 2022-09-03T06:41:47.863 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:41:47.864 DEBUG:boto:Data: 2022-09-03T06:41:47.865 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.866 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:47.866 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.868 DEBUG:boto:Params: {} 2022-09-03T06:41:47.869 DEBUG:boto:Token: None 2022-09-03T06:41:47.870 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/ 2022-09-03T06:41:47.872 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38= 2022-09-03T06:41:47.873 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38=', 'Content-Length': '0'} 2022-09-03T06:41:47.878 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009994691556aec9ca-006312f72b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.879 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:47.881 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:47.882 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:47.883 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:47.885 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:47.886 DEBUG:boto:Method: GET 2022-09-03T06:41:47.887 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:41:47.888 DEBUG:boto:Data: 2022-09-03T06:41:47.889 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.889 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:47.890 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.891 DEBUG:boto:Params: {} 2022-09-03T06:41:47.891 DEBUG:boto:Token: None 2022-09-03T06:41:47.892 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/ 2022-09-03T06:41:47.893 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38= 2022-09-03T06:41:47.893 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hi2pOnpil+QIE5MQRSelmGrcf38=', 'Content-Length': '0'} 2022-09-03T06:41:47.897 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a7574232061b714a-006312f72b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.898 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:47.899 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:41:47.899 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:47.900 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:47.901 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:47.901 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:47.902 DEBUG:boto:Method: GET 2022-09-03T06:41:47.903 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:47.903 DEBUG:boto:Data: 2022-09-03T06:41:47.904 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.904 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:47.905 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.906 DEBUG:boto:Params: {} 2022-09-03T06:41:47.907 DEBUG:boto:Token: None 2022-09-03T06:41:47.907 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:47.909 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2bpcLa7IY1cq60My0rqIJK1EdVE= 2022-09-03T06:41:47.910 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2bpcLa7IY1cq60My0rqIJK1EdVE=', 'Content-Length': '0'} 2022-09-03T06:41:47.913 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b74a7f529ab2ce8b-006312f72b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.954 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:47.956 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:47.956 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:47.957 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:47.957 DEBUG:boto:Method: GET 2022-09-03T06:41:47.958 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:47.959 DEBUG:boto:Data: 2022-09-03T06:41:47.959 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.960 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:47.961 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.961 DEBUG:boto:Params: {} 2022-09-03T06:41:47.962 DEBUG:boto:Token: None 2022-09-03T06:41:47.963 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:47.963 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2bpcLa7IY1cq60My0rqIJK1EdVE= 2022-09-03T06:41:47.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2bpcLa7IY1cq60My0rqIJK1EdVE=', 'Content-Length': '0'} 2022-09-03T06:41:47.966 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003f5fb0924904c626-006312f72b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.968 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:47.968 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:47.969 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:47.970 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:47.970 DEBUG:boto:Method: HEAD 2022-09-03T06:41:47.971 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:47.971 DEBUG:boto:Data: 2022-09-03T06:41:47.973 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.973 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:47.974 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.974 DEBUG:boto:Params: {} 2022-09-03T06:41:47.975 DEBUG:boto:Token: None 2022-09-03T06:41:47.976 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:47.976 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1r1GsUS5PF+1uQaonb36m2xJT/0= 2022-09-03T06:41:47.977 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1r1GsUS5PF+1uQaonb36m2xJT/0=', 'Content-Length': '0'} 2022-09-03T06:41:47.979 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000591c73b2d16f6cbe-006312f72b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:47.980 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:47.982 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:47.982 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:47.983 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:47.984 DEBUG:boto:Method: HEAD 2022-09-03T06:41:47.984 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:47.985 DEBUG:boto:Data: 2022-09-03T06:41:47.985 DEBUG:boto:Headers: {} 2022-09-03T06:41:47.986 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:47.987 DEBUG:boto:Port: 8000 2022-09-03T06:41:47.987 DEBUG:boto:Params: {} 2022-09-03T06:41:47.988 DEBUG:boto:Token: None 2022-09-03T06:41:47.989 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:47 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:47.989 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1r1GsUS5PF+1uQaonb36m2xJT/0= 2022-09-03T06:41:47.991 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1r1GsUS5PF+1uQaonb36m2xJT/0=', 'Content-Length': '0'} 2022-09-03T06:41:47.995 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ddf728d8a35c4de5-006312f72b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.001 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:41:48.001 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:48.003 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:48.003 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:48.004 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:48.004 DEBUG:boto:Method: GET 2022-09-03T06:41:48.005 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:48.006 DEBUG:boto:Data: 2022-09-03T06:41:48.006 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.007 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:48.008 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.009 DEBUG:boto:Params: {} 2022-09-03T06:41:48.009 DEBUG:boto:Token: None 2022-09-03T06:41:48.010 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:48.011 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PGs05fUlSDFYMVaD96lccs7YhRY= 2022-09-03T06:41:48.011 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PGs05fUlSDFYMVaD96lccs7YhRY=', 'Content-Length': '0'} 2022-09-03T06:41:48.014 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eb309611d8da5a95-006312f72c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.014 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:48.015 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:48.016 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:48.016 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:48.017 DEBUG:boto:Method: GET 2022-09-03T06:41:48.017 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:48.018 DEBUG:boto:Data: 2022-09-03T06:41:48.018 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.019 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:48.020 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.020 DEBUG:boto:Params: {} 2022-09-03T06:41:48.021 DEBUG:boto:Token: None 2022-09-03T06:41:48.021 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:48.022 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PGs05fUlSDFYMVaD96lccs7YhRY= 2022-09-03T06:41:48.023 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PGs05fUlSDFYMVaD96lccs7YhRY=', 'Content-Length': '0'} 2022-09-03T06:41:48.025 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e8ef3caf4138d8a6-006312f72c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.026 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:48.027 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:48.027 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:48.028 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:48.028 DEBUG:boto:Method: GET 2022-09-03T06:41:48.029 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:41:48.029 DEBUG:boto:Data: 2022-09-03T06:41:48.030 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.030 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:48.031 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.032 DEBUG:boto:Params: {} 2022-09-03T06:41:48.032 DEBUG:boto:Token: None 2022-09-03T06:41:48.033 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/ 2022-09-03T06:41:48.034 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs= 2022-09-03T06:41:48.034 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs=', 'Content-Length': '0'} 2022-09-03T06:41:48.037 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006e4f21ace11cfc66-006312f72c-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.038 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:48.040 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:48.040 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:48.041 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:48.042 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:48.042 DEBUG:boto:Method: GET 2022-09-03T06:41:48.043 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:41:48.044 DEBUG:boto:Data: 2022-09-03T06:41:48.044 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.045 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:48.045 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.046 DEBUG:boto:Params: {} 2022-09-03T06:41:48.046 DEBUG:boto:Token: None 2022-09-03T06:41:48.051 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/ 2022-09-03T06:41:48.053 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs= 2022-09-03T06:41:48.054 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs=', 'Content-Length': '0'} 2022-09-03T06:41:48.058 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003fd4fd32f1230e83-006312f72c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.059 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:48.061 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:48.062 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:48.063 DEBUG:boto:Method: HEAD 2022-09-03T06:41:48.064 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:41:48.065 DEBUG:boto:Data: 2022-09-03T06:41:48.066 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.068 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:48.068 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.069 DEBUG:boto:Params: {} 2022-09-03T06:41:48.071 DEBUG:boto:Token: None 2022-09-03T06:41:48.072 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj76 2022-09-03T06:41:48.073 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SaKmnJArlNKBo7z3+7Bgu9PGYe8= 2022-09-03T06:41:48.074 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SaKmnJArlNKBo7z3+7Bgu9PGYe8=', 'Content-Length': '0'} 2022-09-03T06:41:48.077 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002e5ca1854d6eb3f1-006312f72c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.078 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:48.079 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:48.079 DEBUG:boto:Method: HEAD 2022-09-03T06:41:48.080 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:41:48.080 DEBUG:boto:Data: 2022-09-03T06:41:48.081 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.082 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:48.082 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.083 DEBUG:boto:Params: {} 2022-09-03T06:41:48.083 DEBUG:boto:Token: None 2022-09-03T06:41:48.084 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj76 2022-09-03T06:41:48.085 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SaKmnJArlNKBo7z3+7Bgu9PGYe8= 2022-09-03T06:41:48.086 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SaKmnJArlNKBo7z3+7Bgu9PGYe8=', 'Content-Length': '0'} 2022-09-03T06:41:48.088 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bb736c883fea066c-006312f72c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.089 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:41:48.089 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:48.090 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:48.090 DEBUG:boto:Method: GET 2022-09-03T06:41:48.091 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:41:48.092 DEBUG:boto:Data: 2022-09-03T06:41:48.093 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.093 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:48.094 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.094 DEBUG:boto:Params: {} 2022-09-03T06:41:48.095 DEBUG:boto:Token: None 2022-09-03T06:41:48.096 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj76 2022-09-03T06:41:48.097 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w11eLrRqEjwMThNCt/ZOqKhszog= 2022-09-03T06:41:48.097 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w11eLrRqEjwMThNCt/ZOqKhszog=', 'Content-Length': '0'} 2022-09-03T06:41:48.100 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a37cc13798f1f56d-006312f72c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.101 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:48.101 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:48.102 DEBUG:boto:Method: GET 2022-09-03T06:41:48.103 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:41:48.103 DEBUG:boto:Data: 2022-09-03T06:41:48.104 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.104 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:48.105 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.106 DEBUG:boto:Params: {} 2022-09-03T06:41:48.107 DEBUG:boto:Token: None 2022-09-03T06:41:48.107 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj76 2022-09-03T06:41:48.108 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w11eLrRqEjwMThNCt/ZOqKhszog= 2022-09-03T06:41:48.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w11eLrRqEjwMThNCt/ZOqKhszog=', 'Content-Length': '0'} 2022-09-03T06:41:48.112 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c3d042c62c658fa-006312f72c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.158 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:48.159 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:48.163 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:48.163 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:48.164 DEBUG:boto:Method: GET 2022-09-03T06:41:48.165 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:41:48.166 DEBUG:boto:Data: 2022-09-03T06:41:48.166 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.167 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:48.168 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.168 DEBUG:boto:Params: {} 2022-09-03T06:41:48.169 DEBUG:boto:Token: None 2022-09-03T06:41:48.170 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/ 2022-09-03T06:41:48.171 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs= 2022-09-03T06:41:48.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs=', 'Content-Length': '0'} 2022-09-03T06:41:48.176 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000afd290c592f5ab20-006312f72c-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.176 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:48.179 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:48.180 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:48.181 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:48.182 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:48.183 DEBUG:boto:Method: GET 2022-09-03T06:41:48.184 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:41:48.185 DEBUG:boto:Data: 2022-09-03T06:41:48.185 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.186 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:48.186 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.187 DEBUG:boto:Params: {} 2022-09-03T06:41:48.187 DEBUG:boto:Token: None 2022-09-03T06:41:48.188 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/ 2022-09-03T06:41:48.189 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs= 2022-09-03T06:41:48.189 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs=', 'Content-Length': '0'} 2022-09-03T06:41:48.193 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c0800dc1ac115460-006312f72c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.194 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:48.195 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:41:48.196 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:48.197 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:48.198 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.199 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.200 DEBUG:boto:Method: GET 2022-09-03T06:41:48.201 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.202 DEBUG:boto:Data: 2022-09-03T06:41:48.203 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.204 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:48.205 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.206 DEBUG:boto:Params: {} 2022-09-03T06:41:48.207 DEBUG:boto:Token: None 2022-09-03T06:41:48.208 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.209 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iXJP/cOSUgyhGfzIABDJMw3y0SI= 2022-09-03T06:41:48.210 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iXJP/cOSUgyhGfzIABDJMw3y0SI=', 'Content-Length': '0'} 2022-09-03T06:41:48.213 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cb6cacf569ef009d-006312f72c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.216 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:48.218 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:48.219 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.221 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.222 DEBUG:boto:Method: GET 2022-09-03T06:41:48.223 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.224 DEBUG:boto:Data: 2022-09-03T06:41:48.225 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.226 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:48.227 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.228 DEBUG:boto:Params: {} 2022-09-03T06:41:48.229 DEBUG:boto:Token: None 2022-09-03T06:41:48.231 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.232 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iXJP/cOSUgyhGfzIABDJMw3y0SI= 2022-09-03T06:41:48.233 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iXJP/cOSUgyhGfzIABDJMw3y0SI=', 'Content-Length': '0'} 2022-09-03T06:41:48.235 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000de432c579f1f4de9-006312f72c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.237 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:48.238 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:48.239 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.239 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.240 DEBUG:boto:Method: HEAD 2022-09-03T06:41:48.241 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.249 DEBUG:boto:Data: 2022-09-03T06:41:48.250 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.252 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:48.252 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.253 DEBUG:boto:Params: {} 2022-09-03T06:41:48.255 DEBUG:boto:Token: None 2022-09-03T06:41:48.256 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.257 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wucT6Z7T/x9xPMrnMelzmfx7Ofc= 2022-09-03T06:41:48.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wucT6Z7T/x9xPMrnMelzmfx7Ofc=', 'Content-Length': '0'} 2022-09-03T06:41:48.260 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d72b3d55d2e62a0e-006312f72c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.262 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:48.262 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:48.263 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.263 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.264 DEBUG:boto:Method: HEAD 2022-09-03T06:41:48.265 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.266 DEBUG:boto:Data: 2022-09-03T06:41:48.266 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.267 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:48.268 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.269 DEBUG:boto:Params: {} 2022-09-03T06:41:48.270 DEBUG:boto:Token: None 2022-09-03T06:41:48.271 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.272 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wucT6Z7T/x9xPMrnMelzmfx7Ofc= 2022-09-03T06:41:48.274 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wucT6Z7T/x9xPMrnMelzmfx7Ofc=', 'Content-Length': '0'} 2022-09-03T06:41:48.282 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e3f8e8b7450ca5df-006312f72c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.283 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:41:48.284 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:48.285 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:48.285 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.286 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.286 DEBUG:boto:Method: GET 2022-09-03T06:41:48.287 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.287 DEBUG:boto:Data: 2022-09-03T06:41:48.298 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.298 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:48.299 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.299 DEBUG:boto:Params: {} 2022-09-03T06:41:48.300 DEBUG:boto:Token: None 2022-09-03T06:41:48.301 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.302 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iXJP/cOSUgyhGfzIABDJMw3y0SI= 2022-09-03T06:41:48.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iXJP/cOSUgyhGfzIABDJMw3y0SI=', 'Content-Length': '0'} 2022-09-03T06:41:48.307 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000053c8afc8e9f807a3-006312f72c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.308 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:48.309 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:48.309 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.310 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.311 DEBUG:boto:Method: GET 2022-09-03T06:41:48.311 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.312 DEBUG:boto:Data: 2022-09-03T06:41:48.312 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.313 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:48.313 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.314 DEBUG:boto:Params: {} 2022-09-03T06:41:48.314 DEBUG:boto:Token: None 2022-09-03T06:41:48.315 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:48.316 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iXJP/cOSUgyhGfzIABDJMw3y0SI= 2022-09-03T06:41:48.316 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iXJP/cOSUgyhGfzIABDJMw3y0SI=', 'Content-Length': '0'} 2022-09-03T06:41:48.319 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005602022c63b2c52e-006312f72c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.319 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:48.320 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:48.320 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:48.321 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:48.321 DEBUG:boto:Method: GET 2022-09-03T06:41:48.322 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:41:48.323 DEBUG:boto:Data: 2022-09-03T06:41:48.323 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.324 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:48.324 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.325 DEBUG:boto:Params: {} 2022-09-03T06:41:48.325 DEBUG:boto:Token: None 2022-09-03T06:41:48.326 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/ 2022-09-03T06:41:48.326 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs= 2022-09-03T06:41:48.327 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs=', 'Content-Length': '0'} 2022-09-03T06:41:48.330 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c17a04b1ddaf86c0-006312f72c-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.374 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:48.375 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:48.376 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:48.377 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:48.377 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:48.378 DEBUG:boto:Method: GET 2022-09-03T06:41:48.378 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:41:48.379 DEBUG:boto:Data: 2022-09-03T06:41:48.381 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.382 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:48.383 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.384 DEBUG:boto:Params: {} 2022-09-03T06:41:48.384 DEBUG:boto:Token: None 2022-09-03T06:41:48.385 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/ 2022-09-03T06:41:48.386 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs= 2022-09-03T06:41:48.386 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs=', 'Content-Length': '0'} 2022-09-03T06:41:48.390 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002c03f337be9a64fc-006312f72c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.391 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:48.392 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:48.392 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:48.393 DEBUG:boto:Method: HEAD 2022-09-03T06:41:48.393 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:41:48.394 DEBUG:boto:Data: 2022-09-03T06:41:48.394 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.395 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:48.396 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.396 DEBUG:boto:Params: {} 2022-09-03T06:41:48.397 DEBUG:boto:Token: None 2022-09-03T06:41:48.397 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj85 2022-09-03T06:41:48.398 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AU/KIND0A+ZSOi+sQ9Y0MGeEA3k= 2022-09-03T06:41:48.399 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AU/KIND0A+ZSOi+sQ9Y0MGeEA3k=', 'Content-Length': '0'} 2022-09-03T06:41:48.404 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001f54f6230883b28d-006312f72c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.405 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:48.406 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:48.407 DEBUG:boto:Method: HEAD 2022-09-03T06:41:48.407 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:41:48.408 DEBUG:boto:Data: 2022-09-03T06:41:48.408 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.409 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:48.410 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.410 DEBUG:boto:Params: {} 2022-09-03T06:41:48.411 DEBUG:boto:Token: None 2022-09-03T06:41:48.411 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj85 2022-09-03T06:41:48.412 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AU/KIND0A+ZSOi+sQ9Y0MGeEA3k= 2022-09-03T06:41:48.413 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AU/KIND0A+ZSOi+sQ9Y0MGeEA3k=', 'Content-Length': '0'} 2022-09-03T06:41:48.415 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d92c9e6c9bfb01c3-006312f72c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.416 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:41:48.417 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:48.418 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:48.418 DEBUG:boto:Method: GET 2022-09-03T06:41:48.419 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:41:48.419 DEBUG:boto:Data: 2022-09-03T06:41:48.420 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.421 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:48.421 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.422 DEBUG:boto:Params: {} 2022-09-03T06:41:48.422 DEBUG:boto:Token: None 2022-09-03T06:41:48.423 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj85 2022-09-03T06:41:48.424 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yu63pt8ORRLd/liu2+b4JroshcA= 2022-09-03T06:41:48.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yu63pt8ORRLd/liu2+b4JroshcA=', 'Content-Length': '0'} 2022-09-03T06:41:48.428 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000704e699016b0ef10-006312f72c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.429 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:48.434 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:48.436 DEBUG:boto:Method: GET 2022-09-03T06:41:48.437 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:41:48.438 DEBUG:boto:Data: 2022-09-03T06:41:48.444 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.445 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:48.446 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.447 DEBUG:boto:Params: {} 2022-09-03T06:41:48.449 DEBUG:boto:Token: None 2022-09-03T06:41:48.450 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj85 2022-09-03T06:41:48.450 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yu63pt8ORRLd/liu2+b4JroshcA= 2022-09-03T06:41:48.451 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yu63pt8ORRLd/liu2+b4JroshcA=', 'Content-Length': '0'} 2022-09-03T06:41:48.453 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000099c6b613229f056a-006312f72c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.496 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:48.497 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:48.501 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:48.502 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:48.502 DEBUG:boto:Method: GET 2022-09-03T06:41:48.503 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:41:48.503 DEBUG:boto:Data: 2022-09-03T06:41:48.504 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.504 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:48.505 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.506 DEBUG:boto:Params: {} 2022-09-03T06:41:48.507 DEBUG:boto:Token: None 2022-09-03T06:41:48.508 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/ 2022-09-03T06:41:48.509 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs= 2022-09-03T06:41:48.509 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs=', 'Content-Length': '0'} 2022-09-03T06:41:48.513 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c41d8193c9d948c9-006312f72c-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.514 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:48.515 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:48.516 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:48.516 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:48.517 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:48.518 DEBUG:boto:Method: GET 2022-09-03T06:41:48.519 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:41:48.519 DEBUG:boto:Data: 2022-09-03T06:41:48.520 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.521 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:48.522 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.523 DEBUG:boto:Params: {} 2022-09-03T06:41:48.524 DEBUG:boto:Token: None 2022-09-03T06:41:48.525 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/ 2022-09-03T06:41:48.526 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs= 2022-09-03T06:41:48.527 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs=', 'Content-Length': '0'} 2022-09-03T06:41:48.532 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009cfd92d0a36fcffd-006312f72c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.534 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:48.535 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:41:48.536 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:48.537 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:48.538 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.539 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.541 DEBUG:boto:Method: GET 2022-09-03T06:41:48.542 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.543 DEBUG:boto:Data: 2022-09-03T06:41:48.545 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.546 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:48.547 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.547 DEBUG:boto:Params: {} 2022-09-03T06:41:48.549 DEBUG:boto:Token: None 2022-09-03T06:41:48.551 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.552 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q1RScC8XkLzUSKDfhQO0Z7oJpMM= 2022-09-03T06:41:48.553 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q1RScC8XkLzUSKDfhQO0Z7oJpMM=', 'Content-Length': '0'} 2022-09-03T06:41:48.556 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000031b46cf231e80a3b-006312f72c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.557 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:48.559 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:48.560 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.561 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.563 DEBUG:boto:Method: GET 2022-09-03T06:41:48.564 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.564 DEBUG:boto:Data: 2022-09-03T06:41:48.565 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.567 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:48.568 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.568 DEBUG:boto:Params: {} 2022-09-03T06:41:48.569 DEBUG:boto:Token: None 2022-09-03T06:41:48.570 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.571 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q1RScC8XkLzUSKDfhQO0Z7oJpMM= 2022-09-03T06:41:48.573 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q1RScC8XkLzUSKDfhQO0Z7oJpMM=', 'Content-Length': '0'} 2022-09-03T06:41:48.575 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000085fb92d676e19e4-006312f72c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.576 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:48.577 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:48.578 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.578 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.579 DEBUG:boto:Method: HEAD 2022-09-03T06:41:48.579 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.580 DEBUG:boto:Data: 2022-09-03T06:41:48.580 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.581 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:48.581 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.583 DEBUG:boto:Params: {} 2022-09-03T06:41:48.583 DEBUG:boto:Token: None 2022-09-03T06:41:48.584 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.585 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Lx1ansLfhbHhk73dSz59856ESM0= 2022-09-03T06:41:48.585 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Lx1ansLfhbHhk73dSz59856ESM0=', 'Content-Length': '0'} 2022-09-03T06:41:48.588 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006ef734b2936e4d29-006312f72c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.588 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:48.589 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:48.589 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.590 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.591 DEBUG:boto:Method: HEAD 2022-09-03T06:41:48.591 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.592 DEBUG:boto:Data: 2022-09-03T06:41:48.593 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.594 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:48.595 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.595 DEBUG:boto:Params: {} 2022-09-03T06:41:48.596 DEBUG:boto:Token: None 2022-09-03T06:41:48.597 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.597 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Lx1ansLfhbHhk73dSz59856ESM0= 2022-09-03T06:41:48.608 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Lx1ansLfhbHhk73dSz59856ESM0=', 'Content-Length': '0'} 2022-09-03T06:41:48.611 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d98a5431ab155c15-006312f72c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.612 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:41:48.613 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:48.614 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:48.615 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.616 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.617 DEBUG:boto:Method: GET 2022-09-03T06:41:48.619 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.620 DEBUG:boto:Data: 2022-09-03T06:41:48.621 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.622 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:48.622 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.623 DEBUG:boto:Params: {} 2022-09-03T06:41:48.623 DEBUG:boto:Token: None 2022-09-03T06:41:48.624 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.625 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q1RScC8XkLzUSKDfhQO0Z7oJpMM= 2022-09-03T06:41:48.626 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q1RScC8XkLzUSKDfhQO0Z7oJpMM=', 'Content-Length': '0'} 2022-09-03T06:41:48.629 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000781d892055b70858-006312f72c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.631 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:48.632 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:48.633 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.633 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.634 DEBUG:boto:Method: GET 2022-09-03T06:41:48.635 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.635 DEBUG:boto:Data: 2022-09-03T06:41:48.636 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.636 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:48.637 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.637 DEBUG:boto:Params: {} 2022-09-03T06:41:48.638 DEBUG:boto:Token: None 2022-09-03T06:41:48.640 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:48.641 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q1RScC8XkLzUSKDfhQO0Z7oJpMM= 2022-09-03T06:41:48.641 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q1RScC8XkLzUSKDfhQO0Z7oJpMM=', 'Content-Length': '0'} 2022-09-03T06:41:48.644 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b48e96a5a85102bb-006312f72c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.692 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:48.694 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:48.695 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:48.696 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:48.697 DEBUG:boto:Method: GET 2022-09-03T06:41:48.698 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:41:48.700 DEBUG:boto:Data: 2022-09-03T06:41:48.701 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.702 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:48.703 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.704 DEBUG:boto:Params: {} 2022-09-03T06:41:48.705 DEBUG:boto:Token: None 2022-09-03T06:41:48.706 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/ 2022-09-03T06:41:48.707 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs= 2022-09-03T06:41:48.709 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs=', 'Content-Length': '0'} 2022-09-03T06:41:48.714 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009931f815da67610a-006312f72c-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.715 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:48.717 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:48.718 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:48.718 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:48.719 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:48.720 DEBUG:boto:Method: GET 2022-09-03T06:41:48.721 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:41:48.721 DEBUG:boto:Data: 2022-09-03T06:41:48.722 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.723 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:48.723 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.724 DEBUG:boto:Params: {} 2022-09-03T06:41:48.725 DEBUG:boto:Token: None 2022-09-03T06:41:48.727 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/ 2022-09-03T06:41:48.728 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs= 2022-09-03T06:41:48.729 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs=', 'Content-Length': '0'} 2022-09-03T06:41:48.733 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f4f274985fc38af8-006312f72c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.734 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:48.736 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:48.737 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:48.738 DEBUG:boto:Method: HEAD 2022-09-03T06:41:48.738 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:41:48.739 DEBUG:boto:Data: 2022-09-03T06:41:48.739 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.740 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:48.740 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.741 DEBUG:boto:Params: {} 2022-09-03T06:41:48.741 DEBUG:boto:Token: None 2022-09-03T06:41:48.742 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj95 2022-09-03T06:41:48.743 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NZ+6Qn0Mr9V9+9hH52IO8kr2JKU= 2022-09-03T06:41:48.743 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NZ+6Qn0Mr9V9+9hH52IO8kr2JKU=', 'Content-Length': '0'} 2022-09-03T06:41:48.745 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000004795cae65821a88-006312f72c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.746 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:48.747 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:48.747 DEBUG:boto:Method: HEAD 2022-09-03T06:41:48.748 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:41:48.748 DEBUG:boto:Data: 2022-09-03T06:41:48.749 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.749 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:48.750 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.750 DEBUG:boto:Params: {} 2022-09-03T06:41:48.751 DEBUG:boto:Token: None 2022-09-03T06:41:48.751 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj95 2022-09-03T06:41:48.752 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NZ+6Qn0Mr9V9+9hH52IO8kr2JKU= 2022-09-03T06:41:48.753 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NZ+6Qn0Mr9V9+9hH52IO8kr2JKU=', 'Content-Length': '0'} 2022-09-03T06:41:48.755 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000de3063c1fd0c9079-006312f72c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.756 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:41:48.756 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:48.757 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:48.758 DEBUG:boto:Method: GET 2022-09-03T06:41:48.758 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:41:48.759 DEBUG:boto:Data: 2022-09-03T06:41:48.759 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.760 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:48.760 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.761 DEBUG:boto:Params: {} 2022-09-03T06:41:48.761 DEBUG:boto:Token: None 2022-09-03T06:41:48.762 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj95 2022-09-03T06:41:48.763 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oYOC3BGYmNFPBL3ndc6qCdRC+pA= 2022-09-03T06:41:48.763 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oYOC3BGYmNFPBL3ndc6qCdRC+pA=', 'Content-Length': '0'} 2022-09-03T06:41:48.766 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d8386bc55f9d519-006312f72c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.767 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:48.768 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:48.768 DEBUG:boto:Method: GET 2022-09-03T06:41:48.769 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:41:48.769 DEBUG:boto:Data: 2022-09-03T06:41:48.770 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.771 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:48.771 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.772 DEBUG:boto:Params: {} 2022-09-03T06:41:48.773 DEBUG:boto:Token: None 2022-09-03T06:41:48.773 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj95 2022-09-03T06:41:48.774 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oYOC3BGYmNFPBL3ndc6qCdRC+pA= 2022-09-03T06:41:48.775 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oYOC3BGYmNFPBL3ndc6qCdRC+pA=', 'Content-Length': '0'} 2022-09-03T06:41:48.777 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000207a28c7662e4321-006312f72c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.778 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:48.779 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:48.782 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:48.782 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:48.783 DEBUG:boto:Method: GET 2022-09-03T06:41:48.784 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:41:48.784 DEBUG:boto:Data: 2022-09-03T06:41:48.785 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.785 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:48.786 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.787 DEBUG:boto:Params: {} 2022-09-03T06:41:48.788 DEBUG:boto:Token: None 2022-09-03T06:41:48.788 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/ 2022-09-03T06:41:48.789 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs= 2022-09-03T06:41:48.790 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs=', 'Content-Length': '0'} 2022-09-03T06:41:48.792 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000027fc502e2bf7639b-006312f72c-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.793 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:48.794 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:48.795 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:48.795 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:48.796 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:48.796 DEBUG:boto:Method: GET 2022-09-03T06:41:48.797 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:41:48.798 DEBUG:boto:Data: 2022-09-03T06:41:48.798 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.799 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:48.799 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.800 DEBUG:boto:Params: {} 2022-09-03T06:41:48.801 DEBUG:boto:Token: None 2022-09-03T06:41:48.801 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/ 2022-09-03T06:41:48.802 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs= 2022-09-03T06:41:48.803 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2fv4oe5Zs3NGeelgkMIV3RaHGCs=', 'Content-Length': '0'} 2022-09-03T06:41:48.806 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e239bd3f708bab64-006312f72c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.813 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:48.815 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-75 zones={test-zone2, test-zone1} 2022-09-03T06:41:48.815 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-75 zones={test-zone3, test-zone1} 2022-09-03T06:41:48.816 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:48.816 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:48.817 DEBUG:boto:Method: HEAD 2022-09-03T06:41:48.818 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:41:48.818 DEBUG:boto:Data: 2022-09-03T06:41:48.819 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.819 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:48.820 DEBUG:boto:Port: 8001 2022-09-03T06:41:48.820 DEBUG:boto:Params: {} 2022-09-03T06:41:48.821 DEBUG:boto:Token: None 2022-09-03T06:41:48.822 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/ 2022-09-03T06:41:48.822 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0PmUjsCLiSjfTm/D9CFxX3bkdA8= 2022-09-03T06:41:48.823 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0PmUjsCLiSjfTm/D9CFxX3bkdA8=', 'Content-Length': '0'} 2022-09-03T06:41:48.825 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '15'), ('X-RGW-Bytes-Used', '45'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e185a6d51504f208-006312f72c-1851-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.826 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:48.827 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:48.827 DEBUG:boto:Method: HEAD 2022-09-03T06:41:48.828 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:41:48.828 DEBUG:boto:Data: 2022-09-03T06:41:48.829 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.829 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:48.830 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.830 DEBUG:boto:Params: {} 2022-09-03T06:41:48.831 DEBUG:boto:Token: None 2022-09-03T06:41:48.832 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/ 2022-09-03T06:41:48.832 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0PmUjsCLiSjfTm/D9CFxX3bkdA8= 2022-09-03T06:41:48.833 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0PmUjsCLiSjfTm/D9CFxX3bkdA8=', 'Content-Length': '0'} 2022-09-03T06:41:48.837 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '15'), ('X-RGW-Bytes-Used', '45'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000005aa6179b3218c907-006312f72c-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.838 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:41:48.838 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:48.839 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:48.839 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:48.840 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:48.841 DEBUG:boto:Method: GET 2022-09-03T06:41:48.841 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:48.842 DEBUG:boto:Data: 2022-09-03T06:41:48.842 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.843 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:48.843 DEBUG:boto:Port: 8001 2022-09-03T06:41:48.844 DEBUG:boto:Params: {} 2022-09-03T06:41:48.845 DEBUG:boto:Token: None 2022-09-03T06:41:48.846 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/?versions 2022-09-03T06:41:48.847 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zg+0ybLoXC85WM6V6c1c9g3W1jc= 2022-09-03T06:41:48.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zg+0ybLoXC85WM6V6c1c9g3W1jc=', 'Content-Length': '0'} 2022-09-03T06:41:48.852 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000067b03e8aa8f4a622-006312f72c-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.853 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:48.855 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:41:48.856 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:41:48.857 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:41:48.858 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:41:48.858 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:41:48.859 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:41:48.860 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:41:48.861 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:41:48.862 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:41:48.862 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:41:48.863 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:41:48.864 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:41:48.865 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:41:48.866 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:41:48.867 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:41:48.867 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:41:48.868 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:48.869 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:48.870 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:48.871 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:48.872 DEBUG:boto:Method: GET 2022-09-03T06:41:48.873 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:48.874 DEBUG:boto:Data: 2022-09-03T06:41:48.875 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.876 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:48.877 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.878 DEBUG:boto:Params: {} 2022-09-03T06:41:48.880 DEBUG:boto:Token: None 2022-09-03T06:41:48.881 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/?versions 2022-09-03T06:41:48.882 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zg+0ybLoXC85WM6V6c1c9g3W1jc= 2022-09-03T06:41:48.883 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zg+0ybLoXC85WM6V6c1c9g3W1jc=', 'Content-Length': '0'} 2022-09-03T06:41:48.888 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000639835c4ad84c6b3-006312f72c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.889 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:48.892 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:41:48.893 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:41:48.894 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:41:48.895 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:41:48.896 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:41:48.897 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:41:48.897 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:41:48.898 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:41:48.899 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:41:48.900 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:41:48.901 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:41:48.902 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:41:48.903 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:41:48.904 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:41:48.904 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:41:48.906 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:48.907 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:48.908 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:48.909 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:48.910 DEBUG:boto:Method: GET 2022-09-03T06:41:48.911 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:48.912 DEBUG:boto:Data: 2022-09-03T06:41:48.913 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.914 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:48.915 DEBUG:boto:Port: 8001 2022-09-03T06:41:48.916 DEBUG:boto:Params: {} 2022-09-03T06:41:48.917 DEBUG:boto:Token: None 2022-09-03T06:41:48.918 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/?versions 2022-09-03T06:41:48.919 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zg+0ybLoXC85WM6V6c1c9g3W1jc= 2022-09-03T06:41:48.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zg+0ybLoXC85WM6V6c1c9g3W1jc=', 'Content-Length': '0'} 2022-09-03T06:41:48.935 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009b6e3b72564b3192-006312f72c-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.939 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:48.943 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:48.944 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:48.945 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:48.946 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:48.950 DEBUG:boto:Method: GET 2022-09-03T06:41:48.951 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:48.953 DEBUG:boto:Data: 2022-09-03T06:41:48.954 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.955 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:48.956 DEBUG:boto:Port: 8000 2022-09-03T06:41:48.956 DEBUG:boto:Params: {} 2022-09-03T06:41:48.957 DEBUG:boto:Token: None 2022-09-03T06:41:48.958 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/?versions 2022-09-03T06:41:48.959 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zg+0ybLoXC85WM6V6c1c9g3W1jc= 2022-09-03T06:41:48.959 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zg+0ybLoXC85WM6V6c1c9g3W1jc=', 'Content-Length': '0'} 2022-09-03T06:41:48.963 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006566dfecf454b1c3-006312f72c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.964 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:48.965 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:41:48.966 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:48.967 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:48.967 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:48.968 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:48.968 DEBUG:boto:Method: GET 2022-09-03T06:41:48.969 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:48.969 DEBUG:boto:Data: 2022-09-03T06:41:48.970 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.970 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:48.971 DEBUG:boto:Port: 8001 2022-09-03T06:41:48.972 DEBUG:boto:Params: {} 2022-09-03T06:41:48.972 DEBUG:boto:Token: None 2022-09-03T06:41:48.973 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:48 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:48.973 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PbX3ZspWG591ck1naFPP+VLApYw= 2022-09-03T06:41:48.974 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PbX3ZspWG591ck1naFPP+VLApYw=', 'Content-Length': '0'} 2022-09-03T06:41:48.982 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f2c8b11ce0bd16c-006312f72c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:48.985 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:48.987 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:48.988 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:48.989 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:48.991 DEBUG:boto:Method: GET 2022-09-03T06:41:48.992 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:48.993 DEBUG:boto:Data: 2022-09-03T06:41:48.993 DEBUG:boto:Headers: {} 2022-09-03T06:41:48.998 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.000 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.000 DEBUG:boto:Params: {} 2022-09-03T06:41:49.001 DEBUG:boto:Token: None 2022-09-03T06:41:49.002 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:49.002 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hwrHRMD9m3mAjKzaoYYReyWm3ZM= 2022-09-03T06:41:49.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hwrHRMD9m3mAjKzaoYYReyWm3ZM=', 'Content-Length': '0'} 2022-09-03T06:41:49.005 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000087ea86514c889d5e-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.006 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:49.007 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:49.007 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:49.008 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:49.008 DEBUG:boto:Method: HEAD 2022-09-03T06:41:49.009 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:49.009 DEBUG:boto:Data: 2022-09-03T06:41:49.010 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.010 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:49.011 DEBUG:boto:Port: 8001 2022-09-03T06:41:49.011 DEBUG:boto:Params: {} 2022-09-03T06:41:49.012 DEBUG:boto:Token: None 2022-09-03T06:41:49.012 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:49.013 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8K+vKgRgm09XHw66tXpdfdeUOk0= 2022-09-03T06:41:49.014 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8K+vKgRgm09XHw66tXpdfdeUOk0=', 'Content-Length': '0'} 2022-09-03T06:41:49.016 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fc516ee21b4bc9ec-006312f72d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.016 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:49.017 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:49.017 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:49.018 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:49.018 DEBUG:boto:Method: HEAD 2022-09-03T06:41:49.019 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:49.020 DEBUG:boto:Data: 2022-09-03T06:41:49.020 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.021 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.021 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.022 DEBUG:boto:Params: {} 2022-09-03T06:41:49.022 DEBUG:boto:Token: None 2022-09-03T06:41:49.023 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:49.024 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8K+vKgRgm09XHw66tXpdfdeUOk0= 2022-09-03T06:41:49.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8K+vKgRgm09XHw66tXpdfdeUOk0=', 'Content-Length': '0'} 2022-09-03T06:41:49.026 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bbab0dffc79ed9b0-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.027 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:41:49.028 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:49.028 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:49.029 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:49.029 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:49.030 DEBUG:boto:Method: GET 2022-09-03T06:41:49.031 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:49.031 DEBUG:boto:Data: 2022-09-03T06:41:49.032 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.033 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:49.033 DEBUG:boto:Port: 8001 2022-09-03T06:41:49.034 DEBUG:boto:Params: {} 2022-09-03T06:41:49.034 DEBUG:boto:Token: None 2022-09-03T06:41:49.035 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:49.036 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hwrHRMD9m3mAjKzaoYYReyWm3ZM= 2022-09-03T06:41:49.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hwrHRMD9m3mAjKzaoYYReyWm3ZM=', 'Content-Length': '0'} 2022-09-03T06:41:49.041 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000960b52162a49a374-006312f72d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.087 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:49.087 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:49.088 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:49.089 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:49.089 DEBUG:boto:Method: GET 2022-09-03T06:41:49.090 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:49.091 DEBUG:boto:Data: 2022-09-03T06:41:49.091 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.092 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.092 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.093 DEBUG:boto:Params: {} 2022-09-03T06:41:49.093 DEBUG:boto:Token: None 2022-09-03T06:41:49.094 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:49.095 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hwrHRMD9m3mAjKzaoYYReyWm3ZM= 2022-09-03T06:41:49.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hwrHRMD9m3mAjKzaoYYReyWm3ZM=', 'Content-Length': '0'} 2022-09-03T06:41:49.100 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007ccb6919a99677c4-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.101 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:49.101 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:49.102 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:49.103 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:49.103 DEBUG:boto:Method: GET 2022-09-03T06:41:49.104 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:41:49.105 DEBUG:boto:Data: 2022-09-03T06:41:49.107 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.108 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:49.109 DEBUG:boto:Port: 8001 2022-09-03T06:41:49.110 DEBUG:boto:Params: {} 2022-09-03T06:41:49.112 DEBUG:boto:Token: None 2022-09-03T06:41:49.113 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/ 2022-09-03T06:41:49.115 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI= 2022-09-03T06:41:49.117 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI=', 'Content-Length': '0'} 2022-09-03T06:41:49.122 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000095cb8f08770b1726-006312f72d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.123 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:49.125 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:49.129 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:49.131 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:49.132 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:49.133 DEBUG:boto:Method: GET 2022-09-03T06:41:49.133 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:41:49.134 DEBUG:boto:Data: 2022-09-03T06:41:49.135 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.136 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.136 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.137 DEBUG:boto:Params: {} 2022-09-03T06:41:49.138 DEBUG:boto:Token: None 2022-09-03T06:41:49.139 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/ 2022-09-03T06:41:49.139 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI= 2022-09-03T06:41:49.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI=', 'Content-Length': '0'} 2022-09-03T06:41:49.145 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000006c985031553935-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.146 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:49.148 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:49.148 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:49.149 DEBUG:boto:Method: HEAD 2022-09-03T06:41:49.150 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:41:49.151 DEBUG:boto:Data: 2022-09-03T06:41:49.152 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.153 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:49.154 DEBUG:boto:Port: 8001 2022-09-03T06:41:49.155 DEBUG:boto:Params: {} 2022-09-03T06:41:49.156 DEBUG:boto:Token: None 2022-09-03T06:41:49.157 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj11 2022-09-03T06:41:49.158 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/a+8QpO1FhUmGa0tL510sVt9BU4= 2022-09-03T06:41:49.159 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/a+8QpO1FhUmGa0tL510sVt9BU4=', 'Content-Length': '0'} 2022-09-03T06:41:49.162 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000002b0670c8bb97385-006312f72d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.163 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:49.164 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:49.165 DEBUG:boto:Method: HEAD 2022-09-03T06:41:49.166 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:41:49.166 DEBUG:boto:Data: 2022-09-03T06:41:49.167 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.167 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.168 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.169 DEBUG:boto:Params: {} 2022-09-03T06:41:49.169 DEBUG:boto:Token: None 2022-09-03T06:41:49.170 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj11 2022-09-03T06:41:49.171 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/a+8QpO1FhUmGa0tL510sVt9BU4= 2022-09-03T06:41:49.171 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/a+8QpO1FhUmGa0tL510sVt9BU4=', 'Content-Length': '0'} 2022-09-03T06:41:49.173 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000adfa55bc643efda2-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.174 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:41:49.175 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:49.175 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:49.176 DEBUG:boto:Method: GET 2022-09-03T06:41:49.177 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:41:49.177 DEBUG:boto:Data: 2022-09-03T06:41:49.178 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.178 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:49.179 DEBUG:boto:Port: 8001 2022-09-03T06:41:49.179 DEBUG:boto:Params: {} 2022-09-03T06:41:49.180 DEBUG:boto:Token: None 2022-09-03T06:41:49.181 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj11 2022-09-03T06:41:49.181 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hpo82i9rI3v0yfoJDolzuR8GViQ= 2022-09-03T06:41:49.182 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hpo82i9rI3v0yfoJDolzuR8GViQ=', 'Content-Length': '0'} 2022-09-03T06:41:49.184 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e006203964463366-006312f72d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.185 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:49.185 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:49.186 DEBUG:boto:Method: GET 2022-09-03T06:41:49.187 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:41:49.187 DEBUG:boto:Data: 2022-09-03T06:41:49.188 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.188 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.189 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.189 DEBUG:boto:Params: {} 2022-09-03T06:41:49.190 DEBUG:boto:Token: None 2022-09-03T06:41:49.191 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj11 2022-09-03T06:41:49.191 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hpo82i9rI3v0yfoJDolzuR8GViQ= 2022-09-03T06:41:49.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hpo82i9rI3v0yfoJDolzuR8GViQ=', 'Content-Length': '0'} 2022-09-03T06:41:49.194 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a96488c2690dcaaa-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.238 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:49.239 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:49.240 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:49.240 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:49.241 DEBUG:boto:Method: GET 2022-09-03T06:41:49.241 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:41:49.242 DEBUG:boto:Data: 2022-09-03T06:41:49.242 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.243 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:49.244 DEBUG:boto:Port: 8001 2022-09-03T06:41:49.244 DEBUG:boto:Params: {} 2022-09-03T06:41:49.245 DEBUG:boto:Token: None 2022-09-03T06:41:49.247 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/ 2022-09-03T06:41:49.248 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI= 2022-09-03T06:41:49.249 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI=', 'Content-Length': '0'} 2022-09-03T06:41:49.254 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b6f425a251d7d153-006312f72d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.255 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:49.258 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:49.259 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:49.260 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:49.262 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:49.263 DEBUG:boto:Method: GET 2022-09-03T06:41:49.265 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:41:49.266 DEBUG:boto:Data: 2022-09-03T06:41:49.267 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.268 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.269 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.270 DEBUG:boto:Params: {} 2022-09-03T06:41:49.271 DEBUG:boto:Token: None 2022-09-03T06:41:49.272 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/ 2022-09-03T06:41:49.273 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI= 2022-09-03T06:41:49.274 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI=', 'Content-Length': '0'} 2022-09-03T06:41:49.278 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000079dc176d599d9364-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.279 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:49.281 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:41:49.282 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:49.283 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:49.284 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.284 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.285 DEBUG:boto:Method: GET 2022-09-03T06:41:49.286 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.287 DEBUG:boto:Data: 2022-09-03T06:41:49.288 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.289 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:49.290 DEBUG:boto:Port: 8001 2022-09-03T06:41:49.291 DEBUG:boto:Params: {} 2022-09-03T06:41:49.292 DEBUG:boto:Token: None 2022-09-03T06:41:49.293 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.294 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KrGL0Oq4sWdiBAhhnwIJwTf9M8E= 2022-09-03T06:41:49.295 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KrGL0Oq4sWdiBAhhnwIJwTf9M8E=', 'Content-Length': '0'} 2022-09-03T06:41:49.298 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d4bf017b3cfbc94-006312f72d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.300 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:49.301 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:49.301 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.302 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.303 DEBUG:boto:Method: GET 2022-09-03T06:41:49.304 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.305 DEBUG:boto:Data: 2022-09-03T06:41:49.306 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.307 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.308 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.309 DEBUG:boto:Params: {} 2022-09-03T06:41:49.310 DEBUG:boto:Token: None 2022-09-03T06:41:49.311 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.312 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KrGL0Oq4sWdiBAhhnwIJwTf9M8E= 2022-09-03T06:41:49.313 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KrGL0Oq4sWdiBAhhnwIJwTf9M8E=', 'Content-Length': '0'} 2022-09-03T06:41:49.326 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096f1d617e278914f-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.328 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:49.329 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:49.330 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.331 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.331 DEBUG:boto:Method: HEAD 2022-09-03T06:41:49.332 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.333 DEBUG:boto:Data: 2022-09-03T06:41:49.334 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.335 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:49.336 DEBUG:boto:Port: 8001 2022-09-03T06:41:49.337 DEBUG:boto:Params: {} 2022-09-03T06:41:49.338 DEBUG:boto:Token: None 2022-09-03T06:41:49.343 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.344 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gVZn5V7NzgfnY6CsY4s4hi2TXKs= 2022-09-03T06:41:49.345 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gVZn5V7NzgfnY6CsY4s4hi2TXKs=', 'Content-Length': '0'} 2022-09-03T06:41:49.349 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000edc5164295603cbf-006312f72d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.350 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:49.351 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:49.352 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.353 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.354 DEBUG:boto:Method: HEAD 2022-09-03T06:41:49.355 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.356 DEBUG:boto:Data: 2022-09-03T06:41:49.357 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.358 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.359 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.360 DEBUG:boto:Params: {} 2022-09-03T06:41:49.361 DEBUG:boto:Token: None 2022-09-03T06:41:49.363 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.364 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gVZn5V7NzgfnY6CsY4s4hi2TXKs= 2022-09-03T06:41:49.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gVZn5V7NzgfnY6CsY4s4hi2TXKs=', 'Content-Length': '0'} 2022-09-03T06:41:49.366 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000adb52d2fec815f3e-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.367 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:41:49.368 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:49.368 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:49.369 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.382 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.383 DEBUG:boto:Method: GET 2022-09-03T06:41:49.385 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.386 DEBUG:boto:Data: 2022-09-03T06:41:49.387 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.388 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:49.389 DEBUG:boto:Port: 8001 2022-09-03T06:41:49.390 DEBUG:boto:Params: {} 2022-09-03T06:41:49.391 DEBUG:boto:Token: None 2022-09-03T06:41:49.392 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.393 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KrGL0Oq4sWdiBAhhnwIJwTf9M8E= 2022-09-03T06:41:49.395 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KrGL0Oq4sWdiBAhhnwIJwTf9M8E=', 'Content-Length': '0'} 2022-09-03T06:41:49.397 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000405fd58ca5fbe8bd-006312f72d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.399 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:49.400 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:49.401 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.402 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.403 DEBUG:boto:Method: GET 2022-09-03T06:41:49.404 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.405 DEBUG:boto:Data: 2022-09-03T06:41:49.405 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.406 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.407 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.407 DEBUG:boto:Params: {} 2022-09-03T06:41:49.408 DEBUG:boto:Token: None 2022-09-03T06:41:49.409 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:49.410 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KrGL0Oq4sWdiBAhhnwIJwTf9M8E= 2022-09-03T06:41:49.410 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KrGL0Oq4sWdiBAhhnwIJwTf9M8E=', 'Content-Length': '0'} 2022-09-03T06:41:49.415 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000050101e07640b32d-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.416 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:49.417 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:49.418 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:49.418 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:49.419 DEBUG:boto:Method: GET 2022-09-03T06:41:49.419 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:41:49.420 DEBUG:boto:Data: 2022-09-03T06:41:49.421 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.421 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:49.422 DEBUG:boto:Port: 8001 2022-09-03T06:41:49.423 DEBUG:boto:Params: {} 2022-09-03T06:41:49.423 DEBUG:boto:Token: None 2022-09-03T06:41:49.424 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/ 2022-09-03T06:41:49.425 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI= 2022-09-03T06:41:49.425 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI=', 'Content-Length': '0'} 2022-09-03T06:41:49.428 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008688d520689f926f-006312f72d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.429 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:49.430 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:49.431 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:49.432 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:49.432 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:49.433 DEBUG:boto:Method: GET 2022-09-03T06:41:49.433 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:41:49.434 DEBUG:boto:Data: 2022-09-03T06:41:49.434 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.435 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.436 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.436 DEBUG:boto:Params: {} 2022-09-03T06:41:49.437 DEBUG:boto:Token: None 2022-09-03T06:41:49.437 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/ 2022-09-03T06:41:49.438 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI= 2022-09-03T06:41:49.439 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI=', 'Content-Length': '0'} 2022-09-03T06:41:49.442 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e39bb6c9f98feffe-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.443 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:49.444 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:49.444 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:49.445 DEBUG:boto:Method: HEAD 2022-09-03T06:41:49.445 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:41:49.446 DEBUG:boto:Data: 2022-09-03T06:41:49.447 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.447 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:49.448 DEBUG:boto:Port: 8001 2022-09-03T06:41:49.448 DEBUG:boto:Params: {} 2022-09-03T06:41:49.449 DEBUG:boto:Token: None 2022-09-03T06:41:49.449 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj17 2022-09-03T06:41:49.450 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qlBzi+15UkVvyQDgTRLJRS8bDe8= 2022-09-03T06:41:49.451 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qlBzi+15UkVvyQDgTRLJRS8bDe8=', 'Content-Length': '0'} 2022-09-03T06:41:49.453 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000317969fbee554ce3-006312f72d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.454 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:49.454 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:49.455 DEBUG:boto:Method: HEAD 2022-09-03T06:41:49.455 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:41:49.456 DEBUG:boto:Data: 2022-09-03T06:41:49.456 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.457 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.457 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.458 DEBUG:boto:Params: {} 2022-09-03T06:41:49.458 DEBUG:boto:Token: None 2022-09-03T06:41:49.459 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj17 2022-09-03T06:41:49.459 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qlBzi+15UkVvyQDgTRLJRS8bDe8= 2022-09-03T06:41:49.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qlBzi+15UkVvyQDgTRLJRS8bDe8=', 'Content-Length': '0'} 2022-09-03T06:41:49.462 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000098d1e211935cb5da-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.463 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:41:49.464 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:49.464 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:49.465 DEBUG:boto:Method: GET 2022-09-03T06:41:49.465 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:41:49.466 DEBUG:boto:Data: 2022-09-03T06:41:49.466 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.467 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:49.467 DEBUG:boto:Port: 8001 2022-09-03T06:41:49.468 DEBUG:boto:Params: {} 2022-09-03T06:41:49.468 DEBUG:boto:Token: None 2022-09-03T06:41:49.469 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj17 2022-09-03T06:41:49.470 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0zcv5yO13QgNsEjrcVz9hUveWiM= 2022-09-03T06:41:49.470 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0zcv5yO13QgNsEjrcVz9hUveWiM=', 'Content-Length': '0'} 2022-09-03T06:41:49.475 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ec693f6831e49f8-006312f72d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.476 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:49.477 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:49.478 DEBUG:boto:Method: GET 2022-09-03T06:41:49.478 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:41:49.479 DEBUG:boto:Data: 2022-09-03T06:41:49.479 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.480 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.480 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.481 DEBUG:boto:Params: {} 2022-09-03T06:41:49.481 DEBUG:boto:Token: None 2022-09-03T06:41:49.482 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj17 2022-09-03T06:41:49.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0zcv5yO13QgNsEjrcVz9hUveWiM= 2022-09-03T06:41:49.483 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0zcv5yO13QgNsEjrcVz9hUveWiM=', 'Content-Length': '0'} 2022-09-03T06:41:49.486 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a73789d441980bfd-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.527 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:49.528 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:49.528 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:49.529 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:49.531 DEBUG:boto:Method: GET 2022-09-03T06:41:49.532 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:41:49.533 DEBUG:boto:Data: 2022-09-03T06:41:49.534 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.535 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:49.536 DEBUG:boto:Port: 8001 2022-09-03T06:41:49.537 DEBUG:boto:Params: {} 2022-09-03T06:41:49.538 DEBUG:boto:Token: None 2022-09-03T06:41:49.539 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/ 2022-09-03T06:41:49.540 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI= 2022-09-03T06:41:49.541 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI=', 'Content-Length': '0'} 2022-09-03T06:41:49.543 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000012a185bc0a20311a-006312f72d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.544 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:49.547 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:49.547 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:49.548 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:49.549 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:49.549 DEBUG:boto:Method: GET 2022-09-03T06:41:49.550 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:41:49.551 DEBUG:boto:Data: 2022-09-03T06:41:49.554 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.561 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.562 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.570 DEBUG:boto:Params: {} 2022-09-03T06:41:49.570 DEBUG:boto:Token: None 2022-09-03T06:41:49.571 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/ 2022-09-03T06:41:49.572 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI= 2022-09-03T06:41:49.573 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI=', 'Content-Length': '0'} 2022-09-03T06:41:49.579 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006ee5bdc19e84986d-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.580 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:49.581 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:41:49.582 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:49.583 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:49.584 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.585 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.586 DEBUG:boto:Method: GET 2022-09-03T06:41:49.587 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.588 DEBUG:boto:Data: 2022-09-03T06:41:49.590 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.591 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:49.592 DEBUG:boto:Port: 8001 2022-09-03T06:41:49.593 DEBUG:boto:Params: {} 2022-09-03T06:41:49.594 DEBUG:boto:Token: None 2022-09-03T06:41:49.595 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.597 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BrdHG3WiBOqUzh1IvMfsTYZb5fU= 2022-09-03T06:41:49.598 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BrdHG3WiBOqUzh1IvMfsTYZb5fU=', 'Content-Length': '0'} 2022-09-03T06:41:49.602 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ae268b24105a643a-006312f72d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.604 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:49.604 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:49.605 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.605 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.606 DEBUG:boto:Method: GET 2022-09-03T06:41:49.607 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.607 DEBUG:boto:Data: 2022-09-03T06:41:49.608 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.608 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.609 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.609 DEBUG:boto:Params: {} 2022-09-03T06:41:49.610 DEBUG:boto:Token: None 2022-09-03T06:41:49.611 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.611 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BrdHG3WiBOqUzh1IvMfsTYZb5fU= 2022-09-03T06:41:49.612 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BrdHG3WiBOqUzh1IvMfsTYZb5fU=', 'Content-Length': '0'} 2022-09-03T06:41:49.614 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000971c682db7880dd7-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.616 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:49.617 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:49.618 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.619 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.620 DEBUG:boto:Method: HEAD 2022-09-03T06:41:49.621 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.622 DEBUG:boto:Data: 2022-09-03T06:41:49.622 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.623 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:49.624 DEBUG:boto:Port: 8001 2022-09-03T06:41:49.624 DEBUG:boto:Params: {} 2022-09-03T06:41:49.625 DEBUG:boto:Token: None 2022-09-03T06:41:49.625 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.626 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l0uWNeQibg9lm+gJwHK4gzmIAxc= 2022-09-03T06:41:49.627 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l0uWNeQibg9lm+gJwHK4gzmIAxc=', 'Content-Length': '0'} 2022-09-03T06:41:49.630 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a34538351dc812fb-006312f72d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.631 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:49.631 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:49.632 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.633 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.634 DEBUG:boto:Method: HEAD 2022-09-03T06:41:49.634 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.635 DEBUG:boto:Data: 2022-09-03T06:41:49.635 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.636 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.636 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.637 DEBUG:boto:Params: {} 2022-09-03T06:41:49.641 DEBUG:boto:Token: None 2022-09-03T06:41:49.642 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.643 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l0uWNeQibg9lm+gJwHK4gzmIAxc= 2022-09-03T06:41:49.644 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l0uWNeQibg9lm+gJwHK4gzmIAxc=', 'Content-Length': '0'} 2022-09-03T06:41:49.646 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000187efe6910e5a020-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.648 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:41:49.649 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:49.651 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:49.652 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.652 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.653 DEBUG:boto:Method: GET 2022-09-03T06:41:49.654 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.655 DEBUG:boto:Data: 2022-09-03T06:41:49.656 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.657 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:49.657 DEBUG:boto:Port: 8001 2022-09-03T06:41:49.659 DEBUG:boto:Params: {} 2022-09-03T06:41:49.660 DEBUG:boto:Token: None 2022-09-03T06:41:49.662 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.663 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BrdHG3WiBOqUzh1IvMfsTYZb5fU= 2022-09-03T06:41:49.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BrdHG3WiBOqUzh1IvMfsTYZb5fU=', 'Content-Length': '0'} 2022-09-03T06:41:49.669 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004d3686f3b8d2f2ae-006312f72d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.671 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:49.672 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:49.673 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.675 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.676 DEBUG:boto:Method: GET 2022-09-03T06:41:49.677 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.678 DEBUG:boto:Data: 2022-09-03T06:41:49.679 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.680 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.681 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.682 DEBUG:boto:Params: {} 2022-09-03T06:41:49.683 DEBUG:boto:Token: None 2022-09-03T06:41:49.685 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:49.686 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BrdHG3WiBOqUzh1IvMfsTYZb5fU= 2022-09-03T06:41:49.688 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BrdHG3WiBOqUzh1IvMfsTYZb5fU=', 'Content-Length': '0'} 2022-09-03T06:41:49.690 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001885c7b28bdc6be6-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.731 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:49.732 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:49.733 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:49.734 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:49.735 DEBUG:boto:Method: GET 2022-09-03T06:41:49.736 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:41:49.737 DEBUG:boto:Data: 2022-09-03T06:41:49.738 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.739 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:49.740 DEBUG:boto:Port: 8001 2022-09-03T06:41:49.741 DEBUG:boto:Params: {} 2022-09-03T06:41:49.742 DEBUG:boto:Token: None 2022-09-03T06:41:49.743 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/ 2022-09-03T06:41:49.745 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI= 2022-09-03T06:41:49.746 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI=', 'Content-Length': '0'} 2022-09-03T06:41:49.750 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c710bc491b3324c7-006312f72d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.750 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:49.751 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:49.753 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:49.754 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:49.754 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:49.756 DEBUG:boto:Method: GET 2022-09-03T06:41:49.759 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:41:49.759 DEBUG:boto:Data: 2022-09-03T06:41:49.760 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.760 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.761 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.762 DEBUG:boto:Params: {} 2022-09-03T06:41:49.762 DEBUG:boto:Token: None 2022-09-03T06:41:49.763 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/ 2022-09-03T06:41:49.764 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI= 2022-09-03T06:41:49.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI=', 'Content-Length': '0'} 2022-09-03T06:41:49.777 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cae589051b7a8b0d-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.779 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:49.781 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:49.781 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:49.785 DEBUG:boto:Method: HEAD 2022-09-03T06:41:49.786 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:41:49.794 DEBUG:boto:Data: 2022-09-03T06:41:49.796 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.797 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:49.798 DEBUG:boto:Port: 8001 2022-09-03T06:41:49.799 DEBUG:boto:Params: {} 2022-09-03T06:41:49.800 DEBUG:boto:Token: None 2022-09-03T06:41:49.803 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj19 2022-09-03T06:41:49.804 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QBF7SC1PLW6hfmz9eWLPMY5h4ZQ= 2022-09-03T06:41:49.806 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QBF7SC1PLW6hfmz9eWLPMY5h4ZQ=', 'Content-Length': '0'} 2022-09-03T06:41:49.810 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d53773ac823c208e-006312f72d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.812 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:49.813 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:49.813 DEBUG:boto:Method: HEAD 2022-09-03T06:41:49.814 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:41:49.815 DEBUG:boto:Data: 2022-09-03T06:41:49.817 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.817 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.818 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.819 DEBUG:boto:Params: {} 2022-09-03T06:41:49.820 DEBUG:boto:Token: None 2022-09-03T06:41:49.821 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj19 2022-09-03T06:41:49.822 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QBF7SC1PLW6hfmz9eWLPMY5h4ZQ= 2022-09-03T06:41:49.823 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QBF7SC1PLW6hfmz9eWLPMY5h4ZQ=', 'Content-Length': '0'} 2022-09-03T06:41:49.825 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004d3aea2978dc424f-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.826 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:41:49.828 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:49.829 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:49.829 DEBUG:boto:Method: GET 2022-09-03T06:41:49.831 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:41:49.832 DEBUG:boto:Data: 2022-09-03T06:41:49.833 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.833 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:49.834 DEBUG:boto:Port: 8001 2022-09-03T06:41:49.835 DEBUG:boto:Params: {} 2022-09-03T06:41:49.835 DEBUG:boto:Token: None 2022-09-03T06:41:49.837 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj19 2022-09-03T06:41:49.837 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fsos1gmo/ycp05HLeAYrYR9VR7A= 2022-09-03T06:41:49.838 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fsos1gmo/ycp05HLeAYrYR9VR7A=', 'Content-Length': '0'} 2022-09-03T06:41:49.842 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b63c953fade8c06-006312f72d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.844 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:49.844 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:49.845 DEBUG:boto:Method: GET 2022-09-03T06:41:49.846 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:41:49.847 DEBUG:boto:Data: 2022-09-03T06:41:49.848 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.849 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.850 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.850 DEBUG:boto:Params: {} 2022-09-03T06:41:49.851 DEBUG:boto:Token: None 2022-09-03T06:41:49.852 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj19 2022-09-03T06:41:49.852 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fsos1gmo/ycp05HLeAYrYR9VR7A= 2022-09-03T06:41:49.853 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fsos1gmo/ycp05HLeAYrYR9VR7A=', 'Content-Length': '0'} 2022-09-03T06:41:49.856 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000149ddfef3f3aa087-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.857 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:49.858 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:49.858 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:49.859 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:49.860 DEBUG:boto:Method: GET 2022-09-03T06:41:49.860 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:41:49.861 DEBUG:boto:Data: 2022-09-03T06:41:49.861 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.862 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:49.862 DEBUG:boto:Port: 8001 2022-09-03T06:41:49.863 DEBUG:boto:Params: {} 2022-09-03T06:41:49.865 DEBUG:boto:Token: None 2022-09-03T06:41:49.866 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/ 2022-09-03T06:41:49.867 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI= 2022-09-03T06:41:49.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI=', 'Content-Length': '0'} 2022-09-03T06:41:49.872 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d1fef9c8ad893558-006312f72d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.873 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:49.879 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:49.880 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:49.881 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:49.881 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:49.882 DEBUG:boto:Method: GET 2022-09-03T06:41:49.883 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:41:49.884 DEBUG:boto:Data: 2022-09-03T06:41:49.885 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.885 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.886 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.886 DEBUG:boto:Params: {} 2022-09-03T06:41:49.887 DEBUG:boto:Token: None 2022-09-03T06:41:49.888 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/ 2022-09-03T06:41:49.889 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI= 2022-09-03T06:41:49.890 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y2K9b89SGNTdu7vvSzmlfnDwzsI=', 'Content-Length': '0'} 2022-09-03T06:41:49.894 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d4361d0a0883ed0d-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.895 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:49.897 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:41:49.898 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:49.899 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:49.900 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.901 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.902 DEBUG:boto:Method: GET 2022-09-03T06:41:49.903 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.903 DEBUG:boto:Data: 2022-09-03T06:41:49.904 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.905 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:49.906 DEBUG:boto:Port: 8001 2022-09-03T06:41:49.907 DEBUG:boto:Params: {} 2022-09-03T06:41:49.908 DEBUG:boto:Token: None 2022-09-03T06:41:49.909 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.910 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nsuKJeJsphRYxI2q9mQ6sMb94no= 2022-09-03T06:41:49.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nsuKJeJsphRYxI2q9mQ6sMb94no=', 'Content-Length': '0'} 2022-09-03T06:41:49.916 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000091ef83443f6a6089-006312f72d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.917 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:49.918 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:49.919 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.920 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.921 DEBUG:boto:Method: GET 2022-09-03T06:41:49.922 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.923 DEBUG:boto:Data: 2022-09-03T06:41:49.924 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.925 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.926 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.927 DEBUG:boto:Params: {} 2022-09-03T06:41:49.929 DEBUG:boto:Token: None 2022-09-03T06:41:49.930 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.932 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nsuKJeJsphRYxI2q9mQ6sMb94no= 2022-09-03T06:41:49.934 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nsuKJeJsphRYxI2q9mQ6sMb94no=', 'Content-Length': '0'} 2022-09-03T06:41:49.937 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b065441ecf47b39-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.938 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:49.939 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:49.940 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.941 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.942 DEBUG:boto:Method: HEAD 2022-09-03T06:41:49.943 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.944 DEBUG:boto:Data: 2022-09-03T06:41:49.945 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.947 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:49.948 DEBUG:boto:Port: 8001 2022-09-03T06:41:49.949 DEBUG:boto:Params: {} 2022-09-03T06:41:49.950 DEBUG:boto:Token: None 2022-09-03T06:41:49.951 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.953 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UL57rNvWz3iVrcV0i4fCERwo8RE= 2022-09-03T06:41:49.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UL57rNvWz3iVrcV0i4fCERwo8RE=', 'Content-Length': '0'} 2022-09-03T06:41:49.956 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dd00bc9042b7e673-006312f72d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.957 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:49.958 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:49.959 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.959 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.960 DEBUG:boto:Method: HEAD 2022-09-03T06:41:49.960 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.961 DEBUG:boto:Data: 2022-09-03T06:41:49.962 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.962 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.963 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.963 DEBUG:boto:Params: {} 2022-09-03T06:41:49.964 DEBUG:boto:Token: None 2022-09-03T06:41:49.965 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.965 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UL57rNvWz3iVrcV0i4fCERwo8RE= 2022-09-03T06:41:49.966 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UL57rNvWz3iVrcV0i4fCERwo8RE=', 'Content-Length': '0'} 2022-09-03T06:41:49.969 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001dae07f42260ca45-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.969 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:41:49.970 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:49.971 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:49.972 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.973 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.974 DEBUG:boto:Method: GET 2022-09-03T06:41:49.975 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.976 DEBUG:boto:Data: 2022-09-03T06:41:49.977 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.978 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:49.979 DEBUG:boto:Port: 8001 2022-09-03T06:41:49.980 DEBUG:boto:Params: {} 2022-09-03T06:41:49.982 DEBUG:boto:Token: None 2022-09-03T06:41:49.983 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.984 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nsuKJeJsphRYxI2q9mQ6sMb94no= 2022-09-03T06:41:49.985 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nsuKJeJsphRYxI2q9mQ6sMb94no=', 'Content-Length': '0'} 2022-09-03T06:41:49.987 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000479f5e9cc17cbecc-006312f72d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:49.988 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:49.988 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:49.989 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.990 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.990 DEBUG:boto:Method: GET 2022-09-03T06:41:49.992 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.992 DEBUG:boto:Data: 2022-09-03T06:41:49.993 DEBUG:boto:Headers: {} 2022-09-03T06:41:49.993 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:49.994 DEBUG:boto:Port: 8000 2022-09-03T06:41:49.994 DEBUG:boto:Params: {} 2022-09-03T06:41:49.995 DEBUG:boto:Token: None 2022-09-03T06:41:49.997 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:49 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:49.997 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nsuKJeJsphRYxI2q9mQ6sMb94no= 2022-09-03T06:41:49.998 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nsuKJeJsphRYxI2q9mQ6sMb94no=', 'Content-Length': '0'} 2022-09-03T06:41:50.000 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d792b14261261b8-006312f72d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.001 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:50.003 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:50.003 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:50.004 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:50.005 DEBUG:boto:Method: GET 2022-09-03T06:41:50.005 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:41:50.006 DEBUG:boto:Data: 2022-09-03T06:41:50.006 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.007 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:50.007 DEBUG:boto:Port: 8001 2022-09-03T06:41:50.008 DEBUG:boto:Params: {} 2022-09-03T06:41:50.008 DEBUG:boto:Token: None 2022-09-03T06:41:50.009 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/ 2022-09-03T06:41:50.010 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg= 2022-09-03T06:41:50.010 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg=', 'Content-Length': '0'} 2022-09-03T06:41:50.013 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000054369516c6133aa0-006312f72e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.058 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:50.059 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:50.060 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:50.061 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:50.062 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:50.063 DEBUG:boto:Method: GET 2022-09-03T06:41:50.064 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:41:50.064 DEBUG:boto:Data: 2022-09-03T06:41:50.065 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.066 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:50.067 DEBUG:boto:Port: 8000 2022-09-03T06:41:50.068 DEBUG:boto:Params: {} 2022-09-03T06:41:50.069 DEBUG:boto:Token: None 2022-09-03T06:41:50.069 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/ 2022-09-03T06:41:50.070 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg= 2022-09-03T06:41:50.071 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg=', 'Content-Length': '0'} 2022-09-03T06:41:50.074 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d2871c4ffea07af-006312f72e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.075 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:50.076 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:50.077 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:50.077 DEBUG:boto:Method: HEAD 2022-09-03T06:41:50.078 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:41:50.079 DEBUG:boto:Data: 2022-09-03T06:41:50.079 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.080 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:50.080 DEBUG:boto:Port: 8001 2022-09-03T06:41:50.081 DEBUG:boto:Params: {} 2022-09-03T06:41:50.081 DEBUG:boto:Token: None 2022-09-03T06:41:50.082 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj22 2022-09-03T06:41:50.083 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1synnPWQg1xqxgPVKDiYDsdqrd8= 2022-09-03T06:41:50.083 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1synnPWQg1xqxgPVKDiYDsdqrd8=', 'Content-Length': '0'} 2022-09-03T06:41:50.086 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e7ff9e04dde8a84-006312f72e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.087 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:50.087 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:50.088 DEBUG:boto:Method: HEAD 2022-09-03T06:41:50.088 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:41:50.089 DEBUG:boto:Data: 2022-09-03T06:41:50.090 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.090 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:50.091 DEBUG:boto:Port: 8000 2022-09-03T06:41:50.091 DEBUG:boto:Params: {} 2022-09-03T06:41:50.092 DEBUG:boto:Token: None 2022-09-03T06:41:50.092 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj22 2022-09-03T06:41:50.093 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1synnPWQg1xqxgPVKDiYDsdqrd8= 2022-09-03T06:41:50.094 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1synnPWQg1xqxgPVKDiYDsdqrd8=', 'Content-Length': '0'} 2022-09-03T06:41:50.096 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000049e1f860fd6cc16b-006312f72e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.097 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:41:50.097 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:50.098 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:50.099 DEBUG:boto:Method: GET 2022-09-03T06:41:50.099 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:41:50.100 DEBUG:boto:Data: 2022-09-03T06:41:50.100 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.101 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:50.101 DEBUG:boto:Port: 8001 2022-09-03T06:41:50.102 DEBUG:boto:Params: {} 2022-09-03T06:41:50.102 DEBUG:boto:Token: None 2022-09-03T06:41:50.103 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj22 2022-09-03T06:41:50.104 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oOae3IMjsOsjdQgnqx033TD+1JQ= 2022-09-03T06:41:50.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oOae3IMjsOsjdQgnqx033TD+1JQ=', 'Content-Length': '0'} 2022-09-03T06:41:50.112 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074060fada3c80fe7-006312f72e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.113 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:50.113 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:50.114 DEBUG:boto:Method: GET 2022-09-03T06:41:50.114 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:41:50.115 DEBUG:boto:Data: 2022-09-03T06:41:50.115 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.116 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:50.116 DEBUG:boto:Port: 8000 2022-09-03T06:41:50.117 DEBUG:boto:Params: {} 2022-09-03T06:41:50.118 DEBUG:boto:Token: None 2022-09-03T06:41:50.119 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj22 2022-09-03T06:41:50.119 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oOae3IMjsOsjdQgnqx033TD+1JQ= 2022-09-03T06:41:50.120 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oOae3IMjsOsjdQgnqx033TD+1JQ=', 'Content-Length': '0'} 2022-09-03T06:41:50.122 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b49ce91900a5f2ad-006312f72e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.175 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:50.176 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:50.177 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:50.178 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:50.179 DEBUG:boto:Method: GET 2022-09-03T06:41:50.181 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:41:50.183 DEBUG:boto:Data: 2022-09-03T06:41:50.184 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.185 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:50.185 DEBUG:boto:Port: 8001 2022-09-03T06:41:50.187 DEBUG:boto:Params: {} 2022-09-03T06:41:50.189 DEBUG:boto:Token: None 2022-09-03T06:41:50.190 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/ 2022-09-03T06:41:50.191 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg= 2022-09-03T06:41:50.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg=', 'Content-Length': '0'} 2022-09-03T06:41:50.196 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000010f780a08e5da2f2-006312f72e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.197 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:50.198 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:50.199 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:50.200 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:50.202 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:50.202 DEBUG:boto:Method: GET 2022-09-03T06:41:50.203 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:41:50.204 DEBUG:boto:Data: 2022-09-03T06:41:50.205 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.205 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:50.206 DEBUG:boto:Port: 8000 2022-09-03T06:41:50.207 DEBUG:boto:Params: {} 2022-09-03T06:41:50.209 DEBUG:boto:Token: None 2022-09-03T06:41:50.210 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/ 2022-09-03T06:41:50.211 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg= 2022-09-03T06:41:50.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg=', 'Content-Length': '0'} 2022-09-03T06:41:50.216 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007571fdc8c7d2f7d0-006312f72e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.218 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:50.219 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:41:50.219 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:50.220 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:50.221 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.221 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.222 DEBUG:boto:Method: GET 2022-09-03T06:41:50.223 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.224 DEBUG:boto:Data: 2022-09-03T06:41:50.225 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.226 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:50.227 DEBUG:boto:Port: 8001 2022-09-03T06:41:50.228 DEBUG:boto:Params: {} 2022-09-03T06:41:50.229 DEBUG:boto:Token: None 2022-09-03T06:41:50.230 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.231 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aKIgElFg9Pt8mgc2Q4nSIshf5CM= 2022-09-03T06:41:50.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aKIgElFg9Pt8mgc2Q4nSIshf5CM=', 'Content-Length': '0'} 2022-09-03T06:41:50.234 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000791e8a2b2389f78e-006312f72e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.236 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:50.237 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:50.238 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.239 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.240 DEBUG:boto:Method: GET 2022-09-03T06:41:50.241 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.242 DEBUG:boto:Data: 2022-09-03T06:41:50.243 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.244 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:50.245 DEBUG:boto:Port: 8000 2022-09-03T06:41:50.246 DEBUG:boto:Params: {} 2022-09-03T06:41:50.247 DEBUG:boto:Token: None 2022-09-03T06:41:50.247 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.248 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aKIgElFg9Pt8mgc2Q4nSIshf5CM= 2022-09-03T06:41:50.249 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aKIgElFg9Pt8mgc2Q4nSIshf5CM=', 'Content-Length': '0'} 2022-09-03T06:41:50.254 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001fb3b51e1fa1587c-006312f72e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.255 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:50.256 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:50.256 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.257 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.257 DEBUG:boto:Method: HEAD 2022-09-03T06:41:50.258 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.259 DEBUG:boto:Data: 2022-09-03T06:41:50.259 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.259 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:50.260 DEBUG:boto:Port: 8001 2022-09-03T06:41:50.260 DEBUG:boto:Params: {} 2022-09-03T06:41:50.261 DEBUG:boto:Token: None 2022-09-03T06:41:50.262 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.262 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RntHscIKaXQHIEo0DiKY8oJVlGg= 2022-09-03T06:41:50.263 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RntHscIKaXQHIEo0DiKY8oJVlGg=', 'Content-Length': '0'} 2022-09-03T06:41:50.265 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000588737f6c8baf3cd-006312f72e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.265 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:50.266 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:50.267 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.267 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.268 DEBUG:boto:Method: HEAD 2022-09-03T06:41:50.268 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.269 DEBUG:boto:Data: 2022-09-03T06:41:50.270 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.270 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:50.271 DEBUG:boto:Port: 8000 2022-09-03T06:41:50.271 DEBUG:boto:Params: {} 2022-09-03T06:41:50.272 DEBUG:boto:Token: None 2022-09-03T06:41:50.272 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.273 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RntHscIKaXQHIEo0DiKY8oJVlGg= 2022-09-03T06:41:50.274 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RntHscIKaXQHIEo0DiKY8oJVlGg=', 'Content-Length': '0'} 2022-09-03T06:41:50.276 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c4a0abe30e6a32cb-006312f72e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.277 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:41:50.277 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:50.278 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:50.278 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.279 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.279 DEBUG:boto:Method: GET 2022-09-03T06:41:50.280 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.280 DEBUG:boto:Data: 2022-09-03T06:41:50.281 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.281 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:50.282 DEBUG:boto:Port: 8001 2022-09-03T06:41:50.282 DEBUG:boto:Params: {} 2022-09-03T06:41:50.283 DEBUG:boto:Token: None 2022-09-03T06:41:50.284 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.284 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aKIgElFg9Pt8mgc2Q4nSIshf5CM= 2022-09-03T06:41:50.285 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aKIgElFg9Pt8mgc2Q4nSIshf5CM=', 'Content-Length': '0'} 2022-09-03T06:41:50.287 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c89da06e26ac0810-006312f72e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.288 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:50.288 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:50.289 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.290 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.290 DEBUG:boto:Method: GET 2022-09-03T06:41:50.291 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.291 DEBUG:boto:Data: 2022-09-03T06:41:50.292 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.292 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:50.293 DEBUG:boto:Port: 8000 2022-09-03T06:41:50.294 DEBUG:boto:Params: {} 2022-09-03T06:41:50.294 DEBUG:boto:Token: None 2022-09-03T06:41:50.295 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:50.295 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aKIgElFg9Pt8mgc2Q4nSIshf5CM= 2022-09-03T06:41:50.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aKIgElFg9Pt8mgc2Q4nSIshf5CM=', 'Content-Length': '0'} 2022-09-03T06:41:50.299 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000003f956083d193c09-006312f72e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.342 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:50.343 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:50.343 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:50.344 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:50.345 DEBUG:boto:Method: GET 2022-09-03T06:41:50.345 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:41:50.346 DEBUG:boto:Data: 2022-09-03T06:41:50.346 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.347 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:50.347 DEBUG:boto:Port: 8001 2022-09-03T06:41:50.348 DEBUG:boto:Params: {} 2022-09-03T06:41:50.349 DEBUG:boto:Token: None 2022-09-03T06:41:50.349 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/ 2022-09-03T06:41:50.350 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg= 2022-09-03T06:41:50.351 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg=', 'Content-Length': '0'} 2022-09-03T06:41:50.354 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c83e2b9d324d5044-006312f72e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.355 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:50.356 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:50.356 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:50.357 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:50.357 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:50.358 DEBUG:boto:Method: GET 2022-09-03T06:41:50.358 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:41:50.359 DEBUG:boto:Data: 2022-09-03T06:41:50.360 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.360 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:50.361 DEBUG:boto:Port: 8000 2022-09-03T06:41:50.362 DEBUG:boto:Params: {} 2022-09-03T06:41:50.363 DEBUG:boto:Token: None 2022-09-03T06:41:50.364 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/ 2022-09-03T06:41:50.364 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg= 2022-09-03T06:41:50.365 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg=', 'Content-Length': '0'} 2022-09-03T06:41:50.368 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b0cf9941d771d5a1-006312f72e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.369 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:50.370 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:50.371 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:50.371 DEBUG:boto:Method: HEAD 2022-09-03T06:41:50.372 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:41:50.372 DEBUG:boto:Data: 2022-09-03T06:41:50.373 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.373 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:50.374 DEBUG:boto:Port: 8001 2022-09-03T06:41:50.375 DEBUG:boto:Params: {} 2022-09-03T06:41:50.375 DEBUG:boto:Token: None 2022-09-03T06:41:50.376 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj33 2022-09-03T06:41:50.377 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ykB+CSFrMTCgY18+PFE8xUm45e8= 2022-09-03T06:41:50.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ykB+CSFrMTCgY18+PFE8xUm45e8=', 'Content-Length': '0'} 2022-09-03T06:41:50.382 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a15af905b1bc7ec9-006312f72e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.383 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:50.383 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:50.384 DEBUG:boto:Method: HEAD 2022-09-03T06:41:50.384 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:41:50.385 DEBUG:boto:Data: 2022-09-03T06:41:50.385 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.386 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:50.387 DEBUG:boto:Port: 8000 2022-09-03T06:41:50.387 DEBUG:boto:Params: {} 2022-09-03T06:41:50.388 DEBUG:boto:Token: None 2022-09-03T06:41:50.388 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj33 2022-09-03T06:41:50.389 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ykB+CSFrMTCgY18+PFE8xUm45e8= 2022-09-03T06:41:50.390 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ykB+CSFrMTCgY18+PFE8xUm45e8=', 'Content-Length': '0'} 2022-09-03T06:41:50.393 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000004246bd4d5b52ed2-006312f72e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.396 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:41:50.396 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:50.397 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:50.398 DEBUG:boto:Method: GET 2022-09-03T06:41:50.400 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:41:50.401 DEBUG:boto:Data: 2022-09-03T06:41:50.401 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.402 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:50.402 DEBUG:boto:Port: 8001 2022-09-03T06:41:50.403 DEBUG:boto:Params: {} 2022-09-03T06:41:50.404 DEBUG:boto:Token: None 2022-09-03T06:41:50.405 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj33 2022-09-03T06:41:50.405 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v2RnuoFk9bHEIZ2g0SZNxTSWZ5Y= 2022-09-03T06:41:50.406 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v2RnuoFk9bHEIZ2g0SZNxTSWZ5Y=', 'Content-Length': '0'} 2022-09-03T06:41:50.408 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e50f1f72f38ff62-006312f72e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.450 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:50.451 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:50.451 DEBUG:boto:Method: GET 2022-09-03T06:41:50.452 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:41:50.453 DEBUG:boto:Data: 2022-09-03T06:41:50.453 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.454 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:50.454 DEBUG:boto:Port: 8000 2022-09-03T06:41:50.455 DEBUG:boto:Params: {} 2022-09-03T06:41:50.456 DEBUG:boto:Token: None 2022-09-03T06:41:50.456 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj33 2022-09-03T06:41:50.457 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v2RnuoFk9bHEIZ2g0SZNxTSWZ5Y= 2022-09-03T06:41:50.458 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v2RnuoFk9bHEIZ2g0SZNxTSWZ5Y=', 'Content-Length': '0'} 2022-09-03T06:41:50.461 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e6ad2f724d5acbb3-006312f72e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.462 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:50.463 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:50.463 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:50.464 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:50.465 DEBUG:boto:Method: GET 2022-09-03T06:41:50.465 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:41:50.466 DEBUG:boto:Data: 2022-09-03T06:41:50.466 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.467 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:50.468 DEBUG:boto:Port: 8001 2022-09-03T06:41:50.468 DEBUG:boto:Params: {} 2022-09-03T06:41:50.469 DEBUG:boto:Token: None 2022-09-03T06:41:50.470 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/ 2022-09-03T06:41:50.470 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg= 2022-09-03T06:41:50.471 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg=', 'Content-Length': '0'} 2022-09-03T06:41:50.474 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007e6c8b0616864d4d-006312f72e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.475 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:50.476 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:50.477 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:50.477 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:50.478 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:50.478 DEBUG:boto:Method: GET 2022-09-03T06:41:50.479 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:41:50.479 DEBUG:boto:Data: 2022-09-03T06:41:50.480 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.480 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:50.481 DEBUG:boto:Port: 8000 2022-09-03T06:41:50.482 DEBUG:boto:Params: {} 2022-09-03T06:41:50.483 DEBUG:boto:Token: None 2022-09-03T06:41:50.483 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/ 2022-09-03T06:41:50.484 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg= 2022-09-03T06:41:50.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg=', 'Content-Length': '0'} 2022-09-03T06:41:50.488 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cee328028b263cec-006312f72e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.489 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:50.489 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:41:50.490 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:50.491 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:50.491 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.492 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.492 DEBUG:boto:Method: GET 2022-09-03T06:41:50.493 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.493 DEBUG:boto:Data: 2022-09-03T06:41:50.494 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.494 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:50.495 DEBUG:boto:Port: 8001 2022-09-03T06:41:50.495 DEBUG:boto:Params: {} 2022-09-03T06:41:50.496 DEBUG:boto:Token: None 2022-09-03T06:41:50.497 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.497 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rcQ/WANrezc0pAdxt2f44OxYmIA= 2022-09-03T06:41:50.498 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rcQ/WANrezc0pAdxt2f44OxYmIA=', 'Content-Length': '0'} 2022-09-03T06:41:50.500 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e20ea940d8f3393-006312f72e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.503 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:50.503 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:50.504 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.505 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.505 DEBUG:boto:Method: GET 2022-09-03T06:41:50.506 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.507 DEBUG:boto:Data: 2022-09-03T06:41:50.507 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.508 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:50.508 DEBUG:boto:Port: 8000 2022-09-03T06:41:50.509 DEBUG:boto:Params: {} 2022-09-03T06:41:50.510 DEBUG:boto:Token: None 2022-09-03T06:41:50.511 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.511 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rcQ/WANrezc0pAdxt2f44OxYmIA= 2022-09-03T06:41:50.512 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rcQ/WANrezc0pAdxt2f44OxYmIA=', 'Content-Length': '0'} 2022-09-03T06:41:50.515 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bab2e378021ef561-006312f72e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.558 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:50.560 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:50.561 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.562 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.563 DEBUG:boto:Method: HEAD 2022-09-03T06:41:50.564 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.564 DEBUG:boto:Data: 2022-09-03T06:41:50.565 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.565 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:50.566 DEBUG:boto:Port: 8001 2022-09-03T06:41:50.567 DEBUG:boto:Params: {} 2022-09-03T06:41:50.568 DEBUG:boto:Token: None 2022-09-03T06:41:50.569 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.570 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EufzS9RLYh77YQshGmfqhk5TPYw= 2022-09-03T06:41:50.570 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EufzS9RLYh77YQshGmfqhk5TPYw=', 'Content-Length': '0'} 2022-09-03T06:41:50.573 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee7e94b136b154ee-006312f72e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.574 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:50.574 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:50.575 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.576 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.576 DEBUG:boto:Method: HEAD 2022-09-03T06:41:50.577 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.577 DEBUG:boto:Data: 2022-09-03T06:41:50.578 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.578 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:50.579 DEBUG:boto:Port: 8000 2022-09-03T06:41:50.579 DEBUG:boto:Params: {} 2022-09-03T06:41:50.580 DEBUG:boto:Token: None 2022-09-03T06:41:50.581 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.581 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EufzS9RLYh77YQshGmfqhk5TPYw= 2022-09-03T06:41:50.582 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EufzS9RLYh77YQshGmfqhk5TPYw=', 'Content-Length': '0'} 2022-09-03T06:41:50.584 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000555bb04c97b979c3-006312f72e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.585 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:41:50.586 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:50.586 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:50.587 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.587 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.588 DEBUG:boto:Method: GET 2022-09-03T06:41:50.588 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.589 DEBUG:boto:Data: 2022-09-03T06:41:50.590 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.590 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:50.591 DEBUG:boto:Port: 8001 2022-09-03T06:41:50.592 DEBUG:boto:Params: {} 2022-09-03T06:41:50.592 DEBUG:boto:Token: None 2022-09-03T06:41:50.602 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.603 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rcQ/WANrezc0pAdxt2f44OxYmIA= 2022-09-03T06:41:50.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rcQ/WANrezc0pAdxt2f44OxYmIA=', 'Content-Length': '0'} 2022-09-03T06:41:50.606 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d3c621c01a8e6b36-006312f72e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.608 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:50.609 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:50.610 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.610 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.611 DEBUG:boto:Method: GET 2022-09-03T06:41:50.612 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.612 DEBUG:boto:Data: 2022-09-03T06:41:50.613 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.614 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:50.615 DEBUG:boto:Port: 8000 2022-09-03T06:41:50.615 DEBUG:boto:Params: {} 2022-09-03T06:41:50.616 DEBUG:boto:Token: None 2022-09-03T06:41:50.617 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:50.618 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rcQ/WANrezc0pAdxt2f44OxYmIA= 2022-09-03T06:41:50.619 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rcQ/WANrezc0pAdxt2f44OxYmIA=', 'Content-Length': '0'} 2022-09-03T06:41:50.622 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc5429dac1c29ccb-006312f72e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.626 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:50.626 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:50.627 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:50.628 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:50.628 DEBUG:boto:Method: GET 2022-09-03T06:41:50.629 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:41:50.630 DEBUG:boto:Data: 2022-09-03T06:41:50.630 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.631 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:50.635 DEBUG:boto:Port: 8001 2022-09-03T06:41:50.637 DEBUG:boto:Params: {} 2022-09-03T06:41:50.638 DEBUG:boto:Token: None 2022-09-03T06:41:50.639 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/ 2022-09-03T06:41:50.640 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg= 2022-09-03T06:41:50.641 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg=', 'Content-Length': '0'} 2022-09-03T06:41:50.650 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d0fdb79bbe4f4df4-006312f72e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.691 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:50.693 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:50.698 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:50.699 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:50.699 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:50.700 DEBUG:boto:Method: GET 2022-09-03T06:41:50.700 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:41:50.701 DEBUG:boto:Data: 2022-09-03T06:41:50.702 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.702 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:50.703 DEBUG:boto:Port: 8000 2022-09-03T06:41:50.704 DEBUG:boto:Params: {} 2022-09-03T06:41:50.705 DEBUG:boto:Token: None 2022-09-03T06:41:50.705 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/ 2022-09-03T06:41:50.706 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg= 2022-09-03T06:41:50.707 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg=', 'Content-Length': '0'} 2022-09-03T06:41:50.717 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007b9025267aaf379d-006312f72e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.718 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:50.721 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:50.721 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:50.722 DEBUG:boto:Method: HEAD 2022-09-03T06:41:50.722 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:41:50.723 DEBUG:boto:Data: 2022-09-03T06:41:50.724 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.725 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:50.726 DEBUG:boto:Port: 8001 2022-09-03T06:41:50.726 DEBUG:boto:Params: {} 2022-09-03T06:41:50.727 DEBUG:boto:Token: None 2022-09-03T06:41:50.728 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj34 2022-09-03T06:41:50.729 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CNHP3RmjpEO+cnbHivRoG6FW7+k= 2022-09-03T06:41:50.729 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CNHP3RmjpEO+cnbHivRoG6FW7+k=', 'Content-Length': '0'} 2022-09-03T06:41:50.734 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b9680331d1cde57-006312f72e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.736 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:50.738 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:50.739 DEBUG:boto:Method: HEAD 2022-09-03T06:41:50.740 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:41:50.741 DEBUG:boto:Data: 2022-09-03T06:41:50.742 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.743 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:50.746 DEBUG:boto:Port: 8000 2022-09-03T06:41:50.747 DEBUG:boto:Params: {} 2022-09-03T06:41:50.748 DEBUG:boto:Token: None 2022-09-03T06:41:50.753 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj34 2022-09-03T06:41:50.756 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CNHP3RmjpEO+cnbHivRoG6FW7+k= 2022-09-03T06:41:50.756 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CNHP3RmjpEO+cnbHivRoG6FW7+k=', 'Content-Length': '0'} 2022-09-03T06:41:50.759 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e05117fc56ba9b11-006312f72e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.760 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:41:50.760 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:50.761 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:50.762 DEBUG:boto:Method: GET 2022-09-03T06:41:50.762 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:41:50.763 DEBUG:boto:Data: 2022-09-03T06:41:50.763 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.764 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:50.764 DEBUG:boto:Port: 8001 2022-09-03T06:41:50.765 DEBUG:boto:Params: {} 2022-09-03T06:41:50.766 DEBUG:boto:Token: None 2022-09-03T06:41:50.766 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj34 2022-09-03T06:41:50.767 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hA7CUmyWdj2IDePh302OTO7Yl1A= 2022-09-03T06:41:50.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hA7CUmyWdj2IDePh302OTO7Yl1A=', 'Content-Length': '0'} 2022-09-03T06:41:50.770 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bcd7fc2e9c7dc17b-006312f72e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.771 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:50.771 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:50.772 DEBUG:boto:Method: GET 2022-09-03T06:41:50.773 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:41:50.773 DEBUG:boto:Data: 2022-09-03T06:41:50.774 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.774 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:50.775 DEBUG:boto:Port: 8000 2022-09-03T06:41:50.775 DEBUG:boto:Params: {} 2022-09-03T06:41:50.776 DEBUG:boto:Token: None 2022-09-03T06:41:50.777 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj34 2022-09-03T06:41:50.777 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hA7CUmyWdj2IDePh302OTO7Yl1A= 2022-09-03T06:41:50.778 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hA7CUmyWdj2IDePh302OTO7Yl1A=', 'Content-Length': '0'} 2022-09-03T06:41:50.792 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000033331ce77491893b-006312f72e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.838 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:50.839 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:50.840 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:50.841 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:50.842 DEBUG:boto:Method: GET 2022-09-03T06:41:50.843 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:41:50.844 DEBUG:boto:Data: 2022-09-03T06:41:50.845 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.846 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:50.847 DEBUG:boto:Port: 8001 2022-09-03T06:41:50.849 DEBUG:boto:Params: {} 2022-09-03T06:41:50.850 DEBUG:boto:Token: None 2022-09-03T06:41:50.851 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/ 2022-09-03T06:41:50.852 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg= 2022-09-03T06:41:50.853 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg=', 'Content-Length': '0'} 2022-09-03T06:41:50.862 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001326659d2f190bac-006312f72e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.867 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:50.869 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:50.870 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:50.871 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:50.872 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:50.873 DEBUG:boto:Method: GET 2022-09-03T06:41:50.874 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:41:50.875 DEBUG:boto:Data: 2022-09-03T06:41:50.876 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.877 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:50.878 DEBUG:boto:Port: 8000 2022-09-03T06:41:50.879 DEBUG:boto:Params: {} 2022-09-03T06:41:50.880 DEBUG:boto:Token: None 2022-09-03T06:41:50.881 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/ 2022-09-03T06:41:50.881 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg= 2022-09-03T06:41:50.882 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vIfqFx5rT1UpoXl1akR3lkNhvwg=', 'Content-Length': '0'} 2022-09-03T06:41:50.885 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000046da617cbb0432c2-006312f72e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.886 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:50.887 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:41:50.888 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:50.888 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:50.889 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.889 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.890 DEBUG:boto:Method: GET 2022-09-03T06:41:50.891 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.891 DEBUG:boto:Data: 2022-09-03T06:41:50.892 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.893 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:50.893 DEBUG:boto:Port: 8001 2022-09-03T06:41:50.894 DEBUG:boto:Params: {} 2022-09-03T06:41:50.894 DEBUG:boto:Token: None 2022-09-03T06:41:50.895 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.896 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PTW8XZP8+1fQr8FQJdvQZ/FkjzM= 2022-09-03T06:41:50.897 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PTW8XZP8+1fQr8FQJdvQZ/FkjzM=', 'Content-Length': '0'} 2022-09-03T06:41:50.900 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad7dac3a7578778e-006312f72e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.901 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:50.901 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:50.902 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.903 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.903 DEBUG:boto:Method: GET 2022-09-03T06:41:50.904 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.905 DEBUG:boto:Data: 2022-09-03T06:41:50.905 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.906 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:50.906 DEBUG:boto:Port: 8000 2022-09-03T06:41:50.907 DEBUG:boto:Params: {} 2022-09-03T06:41:50.908 DEBUG:boto:Token: None 2022-09-03T06:41:50.909 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.909 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PTW8XZP8+1fQr8FQJdvQZ/FkjzM= 2022-09-03T06:41:50.910 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PTW8XZP8+1fQr8FQJdvQZ/FkjzM=', 'Content-Length': '0'} 2022-09-03T06:41:50.914 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a1a7eb69a80a253-006312f72e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.915 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:50.916 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:50.917 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.918 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.919 DEBUG:boto:Method: HEAD 2022-09-03T06:41:50.927 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.928 DEBUG:boto:Data: 2022-09-03T06:41:50.929 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.929 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:50.930 DEBUG:boto:Port: 8001 2022-09-03T06:41:50.931 DEBUG:boto:Params: {} 2022-09-03T06:41:50.931 DEBUG:boto:Token: None 2022-09-03T06:41:50.932 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.933 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IhqQBdFDG99c2sTnozUQWwLFH0g= 2022-09-03T06:41:50.933 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IhqQBdFDG99c2sTnozUQWwLFH0g=', 'Content-Length': '0'} 2022-09-03T06:41:50.935 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005592e45fe754821f-006312f72e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.937 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:50.938 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:50.939 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.939 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.952 DEBUG:boto:Method: HEAD 2022-09-03T06:41:50.955 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.956 DEBUG:boto:Data: 2022-09-03T06:41:50.957 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.958 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:50.959 DEBUG:boto:Port: 8000 2022-09-03T06:41:50.959 DEBUG:boto:Params: {} 2022-09-03T06:41:50.960 DEBUG:boto:Token: None 2022-09-03T06:41:50.961 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.961 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IhqQBdFDG99c2sTnozUQWwLFH0g= 2022-09-03T06:41:50.962 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IhqQBdFDG99c2sTnozUQWwLFH0g=', 'Content-Length': '0'} 2022-09-03T06:41:50.965 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009a03fafe8e993296-006312f72e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.966 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:41:50.967 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:50.968 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:50.968 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.969 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.969 DEBUG:boto:Method: GET 2022-09-03T06:41:50.970 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.971 DEBUG:boto:Data: 2022-09-03T06:41:50.972 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.972 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:50.973 DEBUG:boto:Port: 8001 2022-09-03T06:41:50.973 DEBUG:boto:Params: {} 2022-09-03T06:41:50.974 DEBUG:boto:Token: None 2022-09-03T06:41:50.975 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.976 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PTW8XZP8+1fQr8FQJdvQZ/FkjzM= 2022-09-03T06:41:50.977 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PTW8XZP8+1fQr8FQJdvQZ/FkjzM=', 'Content-Length': '0'} 2022-09-03T06:41:50.979 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007fc5d859ace86352-006312f72e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.980 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:50.981 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:50.982 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.982 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.983 DEBUG:boto:Method: GET 2022-09-03T06:41:50.983 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.984 DEBUG:boto:Data: 2022-09-03T06:41:50.985 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.985 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:50.986 DEBUG:boto:Port: 8000 2022-09-03T06:41:50.986 DEBUG:boto:Params: {} 2022-09-03T06:41:50.987 DEBUG:boto:Token: None 2022-09-03T06:41:50.988 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:50 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:50.988 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PTW8XZP8+1fQr8FQJdvQZ/FkjzM= 2022-09-03T06:41:50.989 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:50 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PTW8XZP8+1fQr8FQJdvQZ/FkjzM=', 'Content-Length': '0'} 2022-09-03T06:41:50.992 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aa4511438fcf43da-006312f72e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:50 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:50.993 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:50.993 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:50.994 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:50.994 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:50.995 DEBUG:boto:Method: GET 2022-09-03T06:41:50.996 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:41:50.996 DEBUG:boto:Data: 2022-09-03T06:41:50.997 DEBUG:boto:Headers: {} 2022-09-03T06:41:50.997 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:50.998 DEBUG:boto:Port: 8001 2022-09-03T06:41:50.999 DEBUG:boto:Params: {} 2022-09-03T06:41:50.999 DEBUG:boto:Token: None 2022-09-03T06:41:51.000 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/ 2022-09-03T06:41:51.001 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA= 2022-09-03T06:41:51.001 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA=', 'Content-Length': '0'} 2022-09-03T06:41:51.006 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d430a4523a39cca-006312f72f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.009 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:51.010 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:51.012 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:51.014 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:51.015 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:51.016 DEBUG:boto:Method: GET 2022-09-03T06:41:51.017 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:41:51.019 DEBUG:boto:Data: 2022-09-03T06:41:51.020 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.022 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:51.023 DEBUG:boto:Port: 8000 2022-09-03T06:41:51.023 DEBUG:boto:Params: {} 2022-09-03T06:41:51.025 DEBUG:boto:Token: None 2022-09-03T06:41:51.027 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/ 2022-09-03T06:41:51.028 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA= 2022-09-03T06:41:51.030 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA=', 'Content-Length': '0'} 2022-09-03T06:41:51.034 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009288487a187dc31c-006312f72f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.034 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:51.036 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:51.036 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:51.037 DEBUG:boto:Method: HEAD 2022-09-03T06:41:51.038 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:41:51.038 DEBUG:boto:Data: 2022-09-03T06:41:51.039 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.039 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:51.040 DEBUG:boto:Port: 8001 2022-09-03T06:41:51.041 DEBUG:boto:Params: {} 2022-09-03T06:41:51.041 DEBUG:boto:Token: None 2022-09-03T06:41:51.042 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj38 2022-09-03T06:41:51.043 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uSaik5yOVpiXyBZw0S4nczxoiao= 2022-09-03T06:41:51.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uSaik5yOVpiXyBZw0S4nczxoiao=', 'Content-Length': '0'} 2022-09-03T06:41:51.045 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000937fac0277ef630f-006312f72f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.046 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:51.047 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:51.048 DEBUG:boto:Method: HEAD 2022-09-03T06:41:51.048 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:41:51.049 DEBUG:boto:Data: 2022-09-03T06:41:51.050 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.050 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:51.051 DEBUG:boto:Port: 8000 2022-09-03T06:41:51.051 DEBUG:boto:Params: {} 2022-09-03T06:41:51.052 DEBUG:boto:Token: None 2022-09-03T06:41:51.053 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj38 2022-09-03T06:41:51.053 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uSaik5yOVpiXyBZw0S4nczxoiao= 2022-09-03T06:41:51.054 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uSaik5yOVpiXyBZw0S4nczxoiao=', 'Content-Length': '0'} 2022-09-03T06:41:51.056 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000256e4dfa43ccb27-006312f72f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.057 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:41:51.058 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:51.058 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:51.059 DEBUG:boto:Method: GET 2022-09-03T06:41:51.060 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:41:51.060 DEBUG:boto:Data: 2022-09-03T06:41:51.061 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.061 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:51.062 DEBUG:boto:Port: 8001 2022-09-03T06:41:51.063 DEBUG:boto:Params: {} 2022-09-03T06:41:51.063 DEBUG:boto:Token: None 2022-09-03T06:41:51.064 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj38 2022-09-03T06:41:51.065 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nmFjGETj/awV7xSPW+RS2QZWxUY= 2022-09-03T06:41:51.065 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nmFjGETj/awV7xSPW+RS2QZWxUY=', 'Content-Length': '0'} 2022-09-03T06:41:51.069 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000050c4f9dda4cdebf8-006312f72f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.071 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:51.072 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:51.072 DEBUG:boto:Method: GET 2022-09-03T06:41:51.073 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:41:51.073 DEBUG:boto:Data: 2022-09-03T06:41:51.074 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.075 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:51.075 DEBUG:boto:Port: 8000 2022-09-03T06:41:51.076 DEBUG:boto:Params: {} 2022-09-03T06:41:51.077 DEBUG:boto:Token: None 2022-09-03T06:41:51.077 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj38 2022-09-03T06:41:51.078 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nmFjGETj/awV7xSPW+RS2QZWxUY= 2022-09-03T06:41:51.079 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nmFjGETj/awV7xSPW+RS2QZWxUY=', 'Content-Length': '0'} 2022-09-03T06:41:51.081 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d4abab3bd2429dfa-006312f72f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.082 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:51.083 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:51.083 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:51.084 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:51.084 DEBUG:boto:Method: GET 2022-09-03T06:41:51.085 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:41:51.086 DEBUG:boto:Data: 2022-09-03T06:41:51.086 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.087 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:51.087 DEBUG:boto:Port: 8001 2022-09-03T06:41:51.088 DEBUG:boto:Params: {} 2022-09-03T06:41:51.089 DEBUG:boto:Token: None 2022-09-03T06:41:51.089 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/ 2022-09-03T06:41:51.090 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA= 2022-09-03T06:41:51.091 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA=', 'Content-Length': '0'} 2022-09-03T06:41:51.097 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007e4cdb05564ce9e2-006312f72f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.142 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:51.144 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:51.145 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:51.146 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:51.147 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:51.148 DEBUG:boto:Method: GET 2022-09-03T06:41:51.150 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:41:51.151 DEBUG:boto:Data: 2022-09-03T06:41:51.153 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.154 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:51.155 DEBUG:boto:Port: 8000 2022-09-03T06:41:51.156 DEBUG:boto:Params: {} 2022-09-03T06:41:51.157 DEBUG:boto:Token: None 2022-09-03T06:41:51.159 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/ 2022-09-03T06:41:51.160 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA= 2022-09-03T06:41:51.161 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA=', 'Content-Length': '0'} 2022-09-03T06:41:51.164 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a937b27eea735079-006312f72f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.165 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:51.166 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:41:51.167 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:51.167 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:51.168 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.168 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.169 DEBUG:boto:Method: GET 2022-09-03T06:41:51.170 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.171 DEBUG:boto:Data: 2022-09-03T06:41:51.172 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.172 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:51.173 DEBUG:boto:Port: 8001 2022-09-03T06:41:51.179 DEBUG:boto:Params: {} 2022-09-03T06:41:51.181 DEBUG:boto:Token: None 2022-09-03T06:41:51.183 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.184 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2ySyV6pycu8BNqdoBQ74r95SVXk= 2022-09-03T06:41:51.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2ySyV6pycu8BNqdoBQ74r95SVXk=', 'Content-Length': '0'} 2022-09-03T06:41:51.187 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007473df2374af6a74-006312f72f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.189 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:51.190 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:51.191 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.191 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.192 DEBUG:boto:Method: GET 2022-09-03T06:41:51.193 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.194 DEBUG:boto:Data: 2022-09-03T06:41:51.196 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.196 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:51.197 DEBUG:boto:Port: 8000 2022-09-03T06:41:51.198 DEBUG:boto:Params: {} 2022-09-03T06:41:51.199 DEBUG:boto:Token: None 2022-09-03T06:41:51.200 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.200 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2ySyV6pycu8BNqdoBQ74r95SVXk= 2022-09-03T06:41:51.201 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2ySyV6pycu8BNqdoBQ74r95SVXk=', 'Content-Length': '0'} 2022-09-03T06:41:51.204 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dbb480625fe2f5d2-006312f72f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.205 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:51.206 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:51.207 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.208 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.209 DEBUG:boto:Method: HEAD 2022-09-03T06:41:51.210 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.211 DEBUG:boto:Data: 2022-09-03T06:41:51.212 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.212 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:51.213 DEBUG:boto:Port: 8001 2022-09-03T06:41:51.213 DEBUG:boto:Params: {} 2022-09-03T06:41:51.214 DEBUG:boto:Token: None 2022-09-03T06:41:51.214 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.215 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s4kTYWnjO1ZBjwoL7b7NWwtbFRI= 2022-09-03T06:41:51.215 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s4kTYWnjO1ZBjwoL7b7NWwtbFRI=', 'Content-Length': '0'} 2022-09-03T06:41:51.217 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000239c71705945c232-006312f72f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.218 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:51.219 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:51.219 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.220 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.220 DEBUG:boto:Method: HEAD 2022-09-03T06:41:51.221 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.221 DEBUG:boto:Data: 2022-09-03T06:41:51.222 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.223 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:51.223 DEBUG:boto:Port: 8000 2022-09-03T06:41:51.224 DEBUG:boto:Params: {} 2022-09-03T06:41:51.224 DEBUG:boto:Token: None 2022-09-03T06:41:51.225 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.226 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s4kTYWnjO1ZBjwoL7b7NWwtbFRI= 2022-09-03T06:41:51.227 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s4kTYWnjO1ZBjwoL7b7NWwtbFRI=', 'Content-Length': '0'} 2022-09-03T06:41:51.230 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000089f2f4f8b398525f-006312f72f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.231 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:41:51.232 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:51.233 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:51.234 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.235 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.236 DEBUG:boto:Method: GET 2022-09-03T06:41:51.236 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.237 DEBUG:boto:Data: 2022-09-03T06:41:51.237 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.238 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:51.238 DEBUG:boto:Port: 8001 2022-09-03T06:41:51.239 DEBUG:boto:Params: {} 2022-09-03T06:41:51.239 DEBUG:boto:Token: None 2022-09-03T06:41:51.240 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.241 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2ySyV6pycu8BNqdoBQ74r95SVXk= 2022-09-03T06:41:51.241 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2ySyV6pycu8BNqdoBQ74r95SVXk=', 'Content-Length': '0'} 2022-09-03T06:41:51.244 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000617b52ebc91626ce-006312f72f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.245 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:51.245 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:51.246 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.246 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.247 DEBUG:boto:Method: GET 2022-09-03T06:41:51.247 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.248 DEBUG:boto:Data: 2022-09-03T06:41:51.248 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.249 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:51.250 DEBUG:boto:Port: 8000 2022-09-03T06:41:51.251 DEBUG:boto:Params: {} 2022-09-03T06:41:51.251 DEBUG:boto:Token: None 2022-09-03T06:41:51.252 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:51.253 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2ySyV6pycu8BNqdoBQ74r95SVXk= 2022-09-03T06:41:51.253 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2ySyV6pycu8BNqdoBQ74r95SVXk=', 'Content-Length': '0'} 2022-09-03T06:41:51.256 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000240d903edcbc55d7-006312f72f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.257 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:51.258 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:51.258 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:51.259 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:51.259 DEBUG:boto:Method: GET 2022-09-03T06:41:51.260 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:41:51.261 DEBUG:boto:Data: 2022-09-03T06:41:51.262 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.263 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:51.264 DEBUG:boto:Port: 8001 2022-09-03T06:41:51.265 DEBUG:boto:Params: {} 2022-09-03T06:41:51.265 DEBUG:boto:Token: None 2022-09-03T06:41:51.267 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/ 2022-09-03T06:41:51.268 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA= 2022-09-03T06:41:51.269 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA=', 'Content-Length': '0'} 2022-09-03T06:41:51.275 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ecb7dc6402f0a5da-006312f72f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.276 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:51.277 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:51.277 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:51.278 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:51.279 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:51.279 DEBUG:boto:Method: GET 2022-09-03T06:41:51.280 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:41:51.281 DEBUG:boto:Data: 2022-09-03T06:41:51.281 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.282 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:51.283 DEBUG:boto:Port: 8000 2022-09-03T06:41:51.283 DEBUG:boto:Params: {} 2022-09-03T06:41:51.284 DEBUG:boto:Token: None 2022-09-03T06:41:51.285 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/ 2022-09-03T06:41:51.285 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA= 2022-09-03T06:41:51.286 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA=', 'Content-Length': '0'} 2022-09-03T06:41:51.290 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c8e33ac158c31d55-006312f72f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.290 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:51.292 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:51.292 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:51.293 DEBUG:boto:Method: HEAD 2022-09-03T06:41:51.294 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:41:51.294 DEBUG:boto:Data: 2022-09-03T06:41:51.295 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.296 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:51.296 DEBUG:boto:Port: 8001 2022-09-03T06:41:51.297 DEBUG:boto:Params: {} 2022-09-03T06:41:51.299 DEBUG:boto:Token: None 2022-09-03T06:41:51.300 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj44 2022-09-03T06:41:51.301 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YNIDmlhdTTEzywHHR63HikDWXc8= 2022-09-03T06:41:51.301 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YNIDmlhdTTEzywHHR63HikDWXc8=', 'Content-Length': '0'} 2022-09-03T06:41:51.308 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000028cee26fa9ad2fe2-006312f72f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.311 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:51.312 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:51.312 DEBUG:boto:Method: HEAD 2022-09-03T06:41:51.313 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:41:51.314 DEBUG:boto:Data: 2022-09-03T06:41:51.315 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.316 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:51.316 DEBUG:boto:Port: 8000 2022-09-03T06:41:51.317 DEBUG:boto:Params: {} 2022-09-03T06:41:51.318 DEBUG:boto:Token: None 2022-09-03T06:41:51.319 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj44 2022-09-03T06:41:51.320 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YNIDmlhdTTEzywHHR63HikDWXc8= 2022-09-03T06:41:51.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YNIDmlhdTTEzywHHR63HikDWXc8=', 'Content-Length': '0'} 2022-09-03T06:41:51.325 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b18d0ab37f41b5b2-006312f72f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.326 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:41:51.327 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:51.327 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:51.328 DEBUG:boto:Method: GET 2022-09-03T06:41:51.328 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:41:51.329 DEBUG:boto:Data: 2022-09-03T06:41:51.329 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.330 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:51.331 DEBUG:boto:Port: 8001 2022-09-03T06:41:51.331 DEBUG:boto:Params: {} 2022-09-03T06:41:51.339 DEBUG:boto:Token: None 2022-09-03T06:41:51.340 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj44 2022-09-03T06:41:51.342 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fPrunv9mAdWHxM8yxT5ZaYxK+fM= 2022-09-03T06:41:51.343 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fPrunv9mAdWHxM8yxT5ZaYxK+fM=', 'Content-Length': '0'} 2022-09-03T06:41:51.346 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e95935c21a4f9191-006312f72f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.386 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:51.387 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:51.388 DEBUG:boto:Method: GET 2022-09-03T06:41:51.388 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:41:51.389 DEBUG:boto:Data: 2022-09-03T06:41:51.389 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.390 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:51.391 DEBUG:boto:Port: 8000 2022-09-03T06:41:51.391 DEBUG:boto:Params: {} 2022-09-03T06:41:51.392 DEBUG:boto:Token: None 2022-09-03T06:41:51.393 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj44 2022-09-03T06:41:51.393 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fPrunv9mAdWHxM8yxT5ZaYxK+fM= 2022-09-03T06:41:51.394 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fPrunv9mAdWHxM8yxT5ZaYxK+fM=', 'Content-Length': '0'} 2022-09-03T06:41:51.397 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005a1f521f1414fcff-006312f72f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.398 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:51.399 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:51.400 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:51.401 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:51.402 DEBUG:boto:Method: GET 2022-09-03T06:41:51.403 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:41:51.404 DEBUG:boto:Data: 2022-09-03T06:41:51.405 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.406 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:51.407 DEBUG:boto:Port: 8001 2022-09-03T06:41:51.408 DEBUG:boto:Params: {} 2022-09-03T06:41:51.409 DEBUG:boto:Token: None 2022-09-03T06:41:51.411 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/ 2022-09-03T06:41:51.412 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA= 2022-09-03T06:41:51.414 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA=', 'Content-Length': '0'} 2022-09-03T06:41:51.422 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d2f031b862e331b0-006312f72f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.424 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:51.425 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:51.427 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:51.428 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:51.428 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:51.429 DEBUG:boto:Method: GET 2022-09-03T06:41:51.430 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:41:51.431 DEBUG:boto:Data: 2022-09-03T06:41:51.432 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.433 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:51.434 DEBUG:boto:Port: 8000 2022-09-03T06:41:51.435 DEBUG:boto:Params: {} 2022-09-03T06:41:51.435 DEBUG:boto:Token: None 2022-09-03T06:41:51.438 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/ 2022-09-03T06:41:51.439 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA= 2022-09-03T06:41:51.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA=', 'Content-Length': '0'} 2022-09-03T06:41:51.444 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ae67a374f29cdee7-006312f72f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.445 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:51.447 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:41:51.448 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:51.449 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:51.450 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.451 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.452 DEBUG:boto:Method: GET 2022-09-03T06:41:51.455 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.456 DEBUG:boto:Data: 2022-09-03T06:41:51.457 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.459 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:51.461 DEBUG:boto:Port: 8001 2022-09-03T06:41:51.462 DEBUG:boto:Params: {} 2022-09-03T06:41:51.464 DEBUG:boto:Token: None 2022-09-03T06:41:51.465 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.466 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3aJUiPw5gwZs3Whe4+1lM4/Q0Kc= 2022-09-03T06:41:51.467 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3aJUiPw5gwZs3Whe4+1lM4/Q0Kc=', 'Content-Length': '0'} 2022-09-03T06:41:51.471 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000b087d7d88abcbbe-006312f72f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.473 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:51.474 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:51.475 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.476 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.477 DEBUG:boto:Method: GET 2022-09-03T06:41:51.479 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.480 DEBUG:boto:Data: 2022-09-03T06:41:51.480 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.481 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:51.482 DEBUG:boto:Port: 8000 2022-09-03T06:41:51.483 DEBUG:boto:Params: {} 2022-09-03T06:41:51.483 DEBUG:boto:Token: None 2022-09-03T06:41:51.485 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.485 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3aJUiPw5gwZs3Whe4+1lM4/Q0Kc= 2022-09-03T06:41:51.486 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3aJUiPw5gwZs3Whe4+1lM4/Q0Kc=', 'Content-Length': '0'} 2022-09-03T06:41:51.488 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004495cdeafbae13e0-006312f72f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.489 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:51.491 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:51.493 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.493 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.494 DEBUG:boto:Method: HEAD 2022-09-03T06:41:51.495 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.496 DEBUG:boto:Data: 2022-09-03T06:41:51.499 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.499 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:51.500 DEBUG:boto:Port: 8001 2022-09-03T06:41:51.501 DEBUG:boto:Params: {} 2022-09-03T06:41:51.504 DEBUG:boto:Token: None 2022-09-03T06:41:51.505 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.506 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TjoqgwsoyqesT7lg97Murcv8wOw= 2022-09-03T06:41:51.507 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TjoqgwsoyqesT7lg97Murcv8wOw=', 'Content-Length': '0'} 2022-09-03T06:41:51.510 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d5fc614800c1f34-006312f72f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.512 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:51.515 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:51.515 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.516 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.517 DEBUG:boto:Method: HEAD 2022-09-03T06:41:51.518 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.519 DEBUG:boto:Data: 2022-09-03T06:41:51.519 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.520 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:51.521 DEBUG:boto:Port: 8000 2022-09-03T06:41:51.522 DEBUG:boto:Params: {} 2022-09-03T06:41:51.522 DEBUG:boto:Token: None 2022-09-03T06:41:51.523 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.524 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TjoqgwsoyqesT7lg97Murcv8wOw= 2022-09-03T06:41:51.526 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TjoqgwsoyqesT7lg97Murcv8wOw=', 'Content-Length': '0'} 2022-09-03T06:41:51.530 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003aec969d99772369-006312f72f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.531 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:41:51.532 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:51.533 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:51.533 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.534 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.535 DEBUG:boto:Method: GET 2022-09-03T06:41:51.535 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.536 DEBUG:boto:Data: 2022-09-03T06:41:51.536 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.537 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:51.537 DEBUG:boto:Port: 8001 2022-09-03T06:41:51.538 DEBUG:boto:Params: {} 2022-09-03T06:41:51.539 DEBUG:boto:Token: None 2022-09-03T06:41:51.540 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.540 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3aJUiPw5gwZs3Whe4+1lM4/Q0Kc= 2022-09-03T06:41:51.541 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3aJUiPw5gwZs3Whe4+1lM4/Q0Kc=', 'Content-Length': '0'} 2022-09-03T06:41:51.543 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d206852d0c0beded-006312f72f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.544 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:51.545 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:51.545 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.546 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.546 DEBUG:boto:Method: GET 2022-09-03T06:41:51.547 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.548 DEBUG:boto:Data: 2022-09-03T06:41:51.549 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.549 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:51.550 DEBUG:boto:Port: 8000 2022-09-03T06:41:51.550 DEBUG:boto:Params: {} 2022-09-03T06:41:51.551 DEBUG:boto:Token: None 2022-09-03T06:41:51.552 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:51.552 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3aJUiPw5gwZs3Whe4+1lM4/Q0Kc= 2022-09-03T06:41:51.553 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3aJUiPw5gwZs3Whe4+1lM4/Q0Kc=', 'Content-Length': '0'} 2022-09-03T06:41:51.556 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000066bcb351424ffa09-006312f72f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.558 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:51.559 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:51.560 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:51.561 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:51.562 DEBUG:boto:Method: GET 2022-09-03T06:41:51.563 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:41:51.564 DEBUG:boto:Data: 2022-09-03T06:41:51.565 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.566 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:51.566 DEBUG:boto:Port: 8001 2022-09-03T06:41:51.567 DEBUG:boto:Params: {} 2022-09-03T06:41:51.568 DEBUG:boto:Token: None 2022-09-03T06:41:51.568 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/ 2022-09-03T06:41:51.569 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA= 2022-09-03T06:41:51.570 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA=', 'Content-Length': '0'} 2022-09-03T06:41:51.573 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e705538c814ecb37-006312f72f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.574 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:51.575 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:51.575 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:51.576 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:51.576 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:51.577 DEBUG:boto:Method: GET 2022-09-03T06:41:51.578 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:41:51.579 DEBUG:boto:Data: 2022-09-03T06:41:51.579 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.580 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:51.580 DEBUG:boto:Port: 8000 2022-09-03T06:41:51.581 DEBUG:boto:Params: {} 2022-09-03T06:41:51.582 DEBUG:boto:Token: None 2022-09-03T06:41:51.582 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/ 2022-09-03T06:41:51.583 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA= 2022-09-03T06:41:51.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA=', 'Content-Length': '0'} 2022-09-03T06:41:51.587 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000063c6d0f114656740-006312f72f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.589 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:51.590 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:51.591 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:51.592 DEBUG:boto:Method: HEAD 2022-09-03T06:41:51.593 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:41:51.594 DEBUG:boto:Data: 2022-09-03T06:41:51.595 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.596 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:51.597 DEBUG:boto:Port: 8001 2022-09-03T06:41:51.597 DEBUG:boto:Params: {} 2022-09-03T06:41:51.599 DEBUG:boto:Token: None 2022-09-03T06:41:51.600 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj51 2022-09-03T06:41:51.601 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uiXZ0SXiemukprHJm+jbgh9B6qo= 2022-09-03T06:41:51.602 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uiXZ0SXiemukprHJm+jbgh9B6qo=', 'Content-Length': '0'} 2022-09-03T06:41:51.605 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ea25cdcf07422d25-006312f72f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.606 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:51.607 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:51.608 DEBUG:boto:Method: HEAD 2022-09-03T06:41:51.609 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:41:51.610 DEBUG:boto:Data: 2022-09-03T06:41:51.611 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.611 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:51.612 DEBUG:boto:Port: 8000 2022-09-03T06:41:51.613 DEBUG:boto:Params: {} 2022-09-03T06:41:51.615 DEBUG:boto:Token: None 2022-09-03T06:41:51.616 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj51 2022-09-03T06:41:51.617 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uiXZ0SXiemukprHJm+jbgh9B6qo= 2022-09-03T06:41:51.618 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uiXZ0SXiemukprHJm+jbgh9B6qo=', 'Content-Length': '0'} 2022-09-03T06:41:51.621 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000064477e7592625d91-006312f72f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.622 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:41:51.625 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:51.626 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:51.627 DEBUG:boto:Method: GET 2022-09-03T06:41:51.628 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:41:51.631 DEBUG:boto:Data: 2022-09-03T06:41:51.632 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.633 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:51.634 DEBUG:boto:Port: 8001 2022-09-03T06:41:51.636 DEBUG:boto:Params: {} 2022-09-03T06:41:51.637 DEBUG:boto:Token: None 2022-09-03T06:41:51.638 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj51 2022-09-03T06:41:51.639 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k41iOpha13f9W7Cn5e+1VeBnaBI= 2022-09-03T06:41:51.640 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k41iOpha13f9W7Cn5e+1VeBnaBI=', 'Content-Length': '0'} 2022-09-03T06:41:51.642 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b6db39cc09f4e1b8-006312f72f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.644 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:51.645 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:51.646 DEBUG:boto:Method: GET 2022-09-03T06:41:51.647 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:41:51.648 DEBUG:boto:Data: 2022-09-03T06:41:51.649 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.650 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:51.651 DEBUG:boto:Port: 8000 2022-09-03T06:41:51.652 DEBUG:boto:Params: {} 2022-09-03T06:41:51.653 DEBUG:boto:Token: None 2022-09-03T06:41:51.654 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj51 2022-09-03T06:41:51.655 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k41iOpha13f9W7Cn5e+1VeBnaBI= 2022-09-03T06:41:51.657 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k41iOpha13f9W7Cn5e+1VeBnaBI=', 'Content-Length': '0'} 2022-09-03T06:41:51.660 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000189d3e8a94754e6e-006312f72f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.662 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:51.663 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:51.664 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:51.665 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:51.666 DEBUG:boto:Method: GET 2022-09-03T06:41:51.667 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:41:51.668 DEBUG:boto:Data: 2022-09-03T06:41:51.668 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.669 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:51.670 DEBUG:boto:Port: 8001 2022-09-03T06:41:51.671 DEBUG:boto:Params: {} 2022-09-03T06:41:51.672 DEBUG:boto:Token: None 2022-09-03T06:41:51.673 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/ 2022-09-03T06:41:51.674 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA= 2022-09-03T06:41:51.675 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA=', 'Content-Length': '0'} 2022-09-03T06:41:51.679 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001dfe3e18c5451ea4-006312f72f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.722 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:51.725 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:51.726 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:51.727 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:51.728 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:51.729 DEBUG:boto:Method: GET 2022-09-03T06:41:51.732 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:41:51.748 DEBUG:boto:Data: 2022-09-03T06:41:51.749 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.751 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:51.752 DEBUG:boto:Port: 8000 2022-09-03T06:41:51.752 DEBUG:boto:Params: {} 2022-09-03T06:41:51.754 DEBUG:boto:Token: None 2022-09-03T06:41:51.755 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/ 2022-09-03T06:41:51.756 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA= 2022-09-03T06:41:51.757 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA=', 'Content-Length': '0'} 2022-09-03T06:41:51.761 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b8cf22176a54ac0c-006312f72f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.763 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:51.764 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:41:51.765 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:51.766 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:51.767 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.768 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.768 DEBUG:boto:Method: GET 2022-09-03T06:41:51.769 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.770 DEBUG:boto:Data: 2022-09-03T06:41:51.771 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.772 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:51.773 DEBUG:boto:Port: 8001 2022-09-03T06:41:51.773 DEBUG:boto:Params: {} 2022-09-03T06:41:51.774 DEBUG:boto:Token: None 2022-09-03T06:41:51.775 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.776 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vu31E16uyZ0dh6DMahUCdjiVLj8= 2022-09-03T06:41:51.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vu31E16uyZ0dh6DMahUCdjiVLj8=', 'Content-Length': '0'} 2022-09-03T06:41:51.781 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000339027ebac19c4c2-006312f72f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.782 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:51.783 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:51.783 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.784 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.784 DEBUG:boto:Method: GET 2022-09-03T06:41:51.785 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.785 DEBUG:boto:Data: 2022-09-03T06:41:51.786 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.795 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:51.798 DEBUG:boto:Port: 8000 2022-09-03T06:41:51.801 DEBUG:boto:Params: {} 2022-09-03T06:41:51.802 DEBUG:boto:Token: None 2022-09-03T06:41:51.812 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.815 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vu31E16uyZ0dh6DMahUCdjiVLj8= 2022-09-03T06:41:51.816 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vu31E16uyZ0dh6DMahUCdjiVLj8=', 'Content-Length': '0'} 2022-09-03T06:41:51.829 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000795224811a6e54f9-006312f72f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.831 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:51.832 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:51.833 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.834 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.835 DEBUG:boto:Method: HEAD 2022-09-03T06:41:51.836 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.838 DEBUG:boto:Data: 2022-09-03T06:41:51.839 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.840 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:51.842 DEBUG:boto:Port: 8001 2022-09-03T06:41:51.843 DEBUG:boto:Params: {} 2022-09-03T06:41:51.844 DEBUG:boto:Token: None 2022-09-03T06:41:51.845 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.846 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ia52ukPxTuDeJhDAjWZlp75X2mE= 2022-09-03T06:41:51.847 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ia52ukPxTuDeJhDAjWZlp75X2mE=', 'Content-Length': '0'} 2022-09-03T06:41:51.859 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000023098c0a836c81c5-006312f72f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.862 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:51.863 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:51.864 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.865 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.867 DEBUG:boto:Method: HEAD 2022-09-03T06:41:51.868 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.869 DEBUG:boto:Data: 2022-09-03T06:41:51.869 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.872 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:51.872 DEBUG:boto:Port: 8000 2022-09-03T06:41:51.873 DEBUG:boto:Params: {} 2022-09-03T06:41:51.874 DEBUG:boto:Token: None 2022-09-03T06:41:51.875 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.876 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ia52ukPxTuDeJhDAjWZlp75X2mE= 2022-09-03T06:41:51.877 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ia52ukPxTuDeJhDAjWZlp75X2mE=', 'Content-Length': '0'} 2022-09-03T06:41:51.883 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e2f71b698e78a3f0-006312f72f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.884 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:41:51.890 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:51.891 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:51.897 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.898 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.899 DEBUG:boto:Method: GET 2022-09-03T06:41:51.899 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.900 DEBUG:boto:Data: 2022-09-03T06:41:51.901 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.901 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:51.902 DEBUG:boto:Port: 8001 2022-09-03T06:41:51.902 DEBUG:boto:Params: {} 2022-09-03T06:41:51.903 DEBUG:boto:Token: None 2022-09-03T06:41:51.904 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.904 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vu31E16uyZ0dh6DMahUCdjiVLj8= 2022-09-03T06:41:51.905 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vu31E16uyZ0dh6DMahUCdjiVLj8=', 'Content-Length': '0'} 2022-09-03T06:41:51.908 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bfb6d0d1f7b6a48a-006312f72f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.909 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:51.909 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:51.910 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.911 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.911 DEBUG:boto:Method: GET 2022-09-03T06:41:51.912 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.912 DEBUG:boto:Data: 2022-09-03T06:41:51.913 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.913 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:51.914 DEBUG:boto:Port: 8000 2022-09-03T06:41:51.914 DEBUG:boto:Params: {} 2022-09-03T06:41:51.915 DEBUG:boto:Token: None 2022-09-03T06:41:51.937 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:51.943 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vu31E16uyZ0dh6DMahUCdjiVLj8= 2022-09-03T06:41:51.945 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vu31E16uyZ0dh6DMahUCdjiVLj8=', 'Content-Length': '0'} 2022-09-03T06:41:51.950 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000700cb59e649e567e-006312f72f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:51.952 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:51.956 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:51.957 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:51.959 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:51.959 DEBUG:boto:Method: GET 2022-09-03T06:41:51.960 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:41:51.962 DEBUG:boto:Data: 2022-09-03T06:41:51.963 DEBUG:boto:Headers: {} 2022-09-03T06:41:51.965 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:51.966 DEBUG:boto:Port: 8001 2022-09-03T06:41:51.967 DEBUG:boto:Params: {} 2022-09-03T06:41:51.968 DEBUG:boto:Token: None 2022-09-03T06:41:51.969 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:51 GMT /dsgvcy-75/ 2022-09-03T06:41:51.971 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA= 2022-09-03T06:41:51.972 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:51 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rSPPjRNIp2LoIJlAfVNQFm+UwEA=', 'Content-Length': '0'} 2022-09-03T06:41:51.989 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b2a5926a5dcb96b8-006312f72f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:51 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.002 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:52.003 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:52.013 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:52.014 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:52.016 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:52.017 DEBUG:boto:Method: GET 2022-09-03T06:41:52.018 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:41:52.019 DEBUG:boto:Data: 2022-09-03T06:41:52.020 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.021 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:52.022 DEBUG:boto:Port: 8000 2022-09-03T06:41:52.023 DEBUG:boto:Params: {} 2022-09-03T06:41:52.038 DEBUG:boto:Token: None 2022-09-03T06:41:52.042 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/ 2022-09-03T06:41:52.043 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XIc/rNOXN+NxSx7QTAy7WlMptXM= 2022-09-03T06:41:52.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XIc/rNOXN+NxSx7QTAy7WlMptXM=', 'Content-Length': '0'} 2022-09-03T06:41:52.068 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000013747e59bc54e99f-006312f730-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.070 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:52.071 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:52.072 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:52.073 DEBUG:boto:Method: HEAD 2022-09-03T06:41:52.074 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:41:52.078 DEBUG:boto:Data: 2022-09-03T06:41:52.079 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.080 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:52.081 DEBUG:boto:Port: 8001 2022-09-03T06:41:52.082 DEBUG:boto:Params: {} 2022-09-03T06:41:52.094 DEBUG:boto:Token: None 2022-09-03T06:41:52.094 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/obj55 2022-09-03T06:41:52.095 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RhFFuBgjYJjNugL+YemOd+cVTQs= 2022-09-03T06:41:52.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RhFFuBgjYJjNugL+YemOd+cVTQs=', 'Content-Length': '0'} 2022-09-03T06:41:52.105 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000068983cb7a3152712-006312f730-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.106 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:52.106 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:52.107 DEBUG:boto:Method: HEAD 2022-09-03T06:41:52.108 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:41:52.111 DEBUG:boto:Data: 2022-09-03T06:41:52.112 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.113 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:52.113 DEBUG:boto:Port: 8000 2022-09-03T06:41:52.114 DEBUG:boto:Params: {} 2022-09-03T06:41:52.114 DEBUG:boto:Token: None 2022-09-03T06:41:52.115 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/obj55 2022-09-03T06:41:52.116 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RhFFuBgjYJjNugL+YemOd+cVTQs= 2022-09-03T06:41:52.117 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RhFFuBgjYJjNugL+YemOd+cVTQs=', 'Content-Length': '0'} 2022-09-03T06:41:52.120 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000023347b0808214fb8-006312f730-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.121 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:41:52.123 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:52.123 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:52.124 DEBUG:boto:Method: GET 2022-09-03T06:41:52.125 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:41:52.126 DEBUG:boto:Data: 2022-09-03T06:41:52.126 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.127 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:52.128 DEBUG:boto:Port: 8001 2022-09-03T06:41:52.129 DEBUG:boto:Params: {} 2022-09-03T06:41:52.130 DEBUG:boto:Token: None 2022-09-03T06:41:52.132 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/obj55 2022-09-03T06:41:52.132 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JruYympk73+VR7/FIl7nH7Hx6I0= 2022-09-03T06:41:52.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JruYympk73+VR7/FIl7nH7Hx6I0=', 'Content-Length': '0'} 2022-09-03T06:41:52.152 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e406c5e6613da283-006312f730-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.154 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:52.156 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:52.158 DEBUG:boto:Method: GET 2022-09-03T06:41:52.159 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:41:52.160 DEBUG:boto:Data: 2022-09-03T06:41:52.160 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.161 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:52.162 DEBUG:boto:Port: 8000 2022-09-03T06:41:52.162 DEBUG:boto:Params: {} 2022-09-03T06:41:52.163 DEBUG:boto:Token: None 2022-09-03T06:41:52.163 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/obj55 2022-09-03T06:41:52.164 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JruYympk73+VR7/FIl7nH7Hx6I0= 2022-09-03T06:41:52.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JruYympk73+VR7/FIl7nH7Hx6I0=', 'Content-Length': '0'} 2022-09-03T06:41:52.167 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008376cf6d4c7f0ee5-006312f730-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.173 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:52.174 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:52.174 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:52.175 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:52.175 DEBUG:boto:Method: GET 2022-09-03T06:41:52.176 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:41:52.176 DEBUG:boto:Data: 2022-09-03T06:41:52.177 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.179 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:52.180 DEBUG:boto:Port: 8001 2022-09-03T06:41:52.186 DEBUG:boto:Params: {} 2022-09-03T06:41:52.187 DEBUG:boto:Token: None 2022-09-03T06:41:52.188 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/ 2022-09-03T06:41:52.188 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XIc/rNOXN+NxSx7QTAy7WlMptXM= 2022-09-03T06:41:52.189 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XIc/rNOXN+NxSx7QTAy7WlMptXM=', 'Content-Length': '0'} 2022-09-03T06:41:52.195 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009473b1f49d83f1fd-006312f730-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.196 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:52.197 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:52.198 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:52.199 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:52.199 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:52.200 DEBUG:boto:Method: GET 2022-09-03T06:41:52.200 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:41:52.204 DEBUG:boto:Data: 2022-09-03T06:41:52.205 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.206 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:52.207 DEBUG:boto:Port: 8000 2022-09-03T06:41:52.208 DEBUG:boto:Params: {} 2022-09-03T06:41:52.209 DEBUG:boto:Token: None 2022-09-03T06:41:52.210 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/ 2022-09-03T06:41:52.211 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XIc/rNOXN+NxSx7QTAy7WlMptXM= 2022-09-03T06:41:52.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XIc/rNOXN+NxSx7QTAy7WlMptXM=', 'Content-Length': '0'} 2022-09-03T06:41:52.243 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000edad1b5d490e45cf-006312f730-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.244 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:52.246 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:41:52.247 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:52.248 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:52.249 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.254 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.263 DEBUG:boto:Method: GET 2022-09-03T06:41:52.264 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.264 DEBUG:boto:Data: 2022-09-03T06:41:52.265 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.265 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:52.266 DEBUG:boto:Port: 8001 2022-09-03T06:41:52.266 DEBUG:boto:Params: {} 2022-09-03T06:41:52.267 DEBUG:boto:Token: None 2022-09-03T06:41:52.268 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.268 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dz2M6ZrvNwsA72nzRjxCJ5u5fR0= 2022-09-03T06:41:52.269 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dz2M6ZrvNwsA72nzRjxCJ5u5fR0=', 'Content-Length': '0'} 2022-09-03T06:41:52.274 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e07f9f66b3793f8a-006312f730-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.285 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:52.286 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:52.287 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.288 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.289 DEBUG:boto:Method: GET 2022-09-03T06:41:52.290 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.292 DEBUG:boto:Data: 2022-09-03T06:41:52.293 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.294 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:52.295 DEBUG:boto:Port: 8000 2022-09-03T06:41:52.296 DEBUG:boto:Params: {} 2022-09-03T06:41:52.297 DEBUG:boto:Token: None 2022-09-03T06:41:52.299 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.300 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dz2M6ZrvNwsA72nzRjxCJ5u5fR0= 2022-09-03T06:41:52.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dz2M6ZrvNwsA72nzRjxCJ5u5fR0=', 'Content-Length': '0'} 2022-09-03T06:41:52.311 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a015098909f88dfe-006312f730-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.313 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:52.314 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:52.323 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.327 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.328 DEBUG:boto:Method: HEAD 2022-09-03T06:41:52.329 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.330 DEBUG:boto:Data: 2022-09-03T06:41:52.331 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.332 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:52.333 DEBUG:boto:Port: 8001 2022-09-03T06:41:52.337 DEBUG:boto:Params: {} 2022-09-03T06:41:52.338 DEBUG:boto:Token: None 2022-09-03T06:41:52.340 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.341 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HjcvwEK5wUD5o5BGkz3zXwJTryI= 2022-09-03T06:41:52.342 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HjcvwEK5wUD5o5BGkz3zXwJTryI=', 'Content-Length': '0'} 2022-09-03T06:41:52.362 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a2bbfd83bd14469f-006312f730-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.363 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:52.365 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:52.366 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.367 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.368 DEBUG:boto:Method: HEAD 2022-09-03T06:41:52.372 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.373 DEBUG:boto:Data: 2022-09-03T06:41:52.374 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.385 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:52.386 DEBUG:boto:Port: 8000 2022-09-03T06:41:52.387 DEBUG:boto:Params: {} 2022-09-03T06:41:52.387 DEBUG:boto:Token: None 2022-09-03T06:41:52.388 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.389 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HjcvwEK5wUD5o5BGkz3zXwJTryI= 2022-09-03T06:41:52.389 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HjcvwEK5wUD5o5BGkz3zXwJTryI=', 'Content-Length': '0'} 2022-09-03T06:41:52.398 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aa1a0fe781aecb16-006312f730-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.399 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:41:52.400 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:52.401 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:52.401 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.416 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.417 DEBUG:boto:Method: GET 2022-09-03T06:41:52.427 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.428 DEBUG:boto:Data: 2022-09-03T06:41:52.441 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.443 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:52.444 DEBUG:boto:Port: 8001 2022-09-03T06:41:52.445 DEBUG:boto:Params: {} 2022-09-03T06:41:52.446 DEBUG:boto:Token: None 2022-09-03T06:41:52.447 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.448 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dz2M6ZrvNwsA72nzRjxCJ5u5fR0= 2022-09-03T06:41:52.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dz2M6ZrvNwsA72nzRjxCJ5u5fR0=', 'Content-Length': '0'} 2022-09-03T06:41:52.469 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000862d37636c6c426f-006312f730-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.470 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:52.471 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:52.472 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.473 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.474 DEBUG:boto:Method: GET 2022-09-03T06:41:52.475 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.476 DEBUG:boto:Data: 2022-09-03T06:41:52.477 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.478 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:52.487 DEBUG:boto:Port: 8000 2022-09-03T06:41:52.488 DEBUG:boto:Params: {} 2022-09-03T06:41:52.489 DEBUG:boto:Token: None 2022-09-03T06:41:52.497 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:52.498 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dz2M6ZrvNwsA72nzRjxCJ5u5fR0= 2022-09-03T06:41:52.500 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dz2M6ZrvNwsA72nzRjxCJ5u5fR0=', 'Content-Length': '0'} 2022-09-03T06:41:52.505 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1417fafa14869d3-006312f730-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.510 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:52.512 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:52.513 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:52.514 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:52.515 DEBUG:boto:Method: GET 2022-09-03T06:41:52.516 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:41:52.517 DEBUG:boto:Data: 2022-09-03T06:41:52.519 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.521 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:52.522 DEBUG:boto:Port: 8001 2022-09-03T06:41:52.523 DEBUG:boto:Params: {} 2022-09-03T06:41:52.524 DEBUG:boto:Token: None 2022-09-03T06:41:52.525 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/ 2022-09-03T06:41:52.527 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XIc/rNOXN+NxSx7QTAy7WlMptXM= 2022-09-03T06:41:52.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XIc/rNOXN+NxSx7QTAy7WlMptXM=', 'Content-Length': '0'} 2022-09-03T06:41:52.533 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cdf09b59a70f1819-006312f730-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.534 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:52.536 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:52.537 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:52.537 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:52.538 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:52.539 DEBUG:boto:Method: GET 2022-09-03T06:41:52.540 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:41:52.541 DEBUG:boto:Data: 2022-09-03T06:41:52.543 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.544 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:52.545 DEBUG:boto:Port: 8000 2022-09-03T06:41:52.545 DEBUG:boto:Params: {} 2022-09-03T06:41:52.547 DEBUG:boto:Token: None 2022-09-03T06:41:52.548 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/ 2022-09-03T06:41:52.549 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XIc/rNOXN+NxSx7QTAy7WlMptXM= 2022-09-03T06:41:52.550 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XIc/rNOXN+NxSx7QTAy7WlMptXM=', 'Content-Length': '0'} 2022-09-03T06:41:52.555 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c24838749dc746f2-006312f730-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.556 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:52.558 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:52.559 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:52.560 DEBUG:boto:Method: HEAD 2022-09-03T06:41:52.561 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:41:52.562 DEBUG:boto:Data: 2022-09-03T06:41:52.563 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.565 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:52.566 DEBUG:boto:Port: 8001 2022-09-03T06:41:52.567 DEBUG:boto:Params: {} 2022-09-03T06:41:52.568 DEBUG:boto:Token: None 2022-09-03T06:41:52.570 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/obj57 2022-09-03T06:41:52.571 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TQc02k/jJ+jFDGA28LAlLj6TpuE= 2022-09-03T06:41:52.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TQc02k/jJ+jFDGA28LAlLj6TpuE=', 'Content-Length': '0'} 2022-09-03T06:41:52.576 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000013ee0e1e1409c4c3-006312f730-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.578 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:52.579 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:52.580 DEBUG:boto:Method: HEAD 2022-09-03T06:41:52.581 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:41:52.582 DEBUG:boto:Data: 2022-09-03T06:41:52.583 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.584 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:52.585 DEBUG:boto:Port: 8000 2022-09-03T06:41:52.588 DEBUG:boto:Params: {} 2022-09-03T06:41:52.589 DEBUG:boto:Token: None 2022-09-03T06:41:52.614 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/obj57 2022-09-03T06:41:52.615 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TQc02k/jJ+jFDGA28LAlLj6TpuE= 2022-09-03T06:41:52.615 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TQc02k/jJ+jFDGA28LAlLj6TpuE=', 'Content-Length': '0'} 2022-09-03T06:41:52.618 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cd336a88a06407ed-006312f730-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.618 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:41:52.619 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:52.620 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:52.620 DEBUG:boto:Method: GET 2022-09-03T06:41:52.621 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:41:52.621 DEBUG:boto:Data: 2022-09-03T06:41:52.622 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.622 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:52.623 DEBUG:boto:Port: 8001 2022-09-03T06:41:52.623 DEBUG:boto:Params: {} 2022-09-03T06:41:52.624 DEBUG:boto:Token: None 2022-09-03T06:41:52.625 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/obj57 2022-09-03T06:41:52.625 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:x1PAxbTeJhEC+m6HkHDSzZ+e3CU= 2022-09-03T06:41:52.626 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:x1PAxbTeJhEC+m6HkHDSzZ+e3CU=', 'Content-Length': '0'} 2022-09-03T06:41:52.630 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ce3417ef43519c15-006312f730-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.631 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:52.632 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:52.633 DEBUG:boto:Method: GET 2022-09-03T06:41:52.634 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:41:52.634 DEBUG:boto:Data: 2022-09-03T06:41:52.635 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.636 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:52.636 DEBUG:boto:Port: 8000 2022-09-03T06:41:52.637 DEBUG:boto:Params: {} 2022-09-03T06:41:52.637 DEBUG:boto:Token: None 2022-09-03T06:41:52.638 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/obj57 2022-09-03T06:41:52.639 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:x1PAxbTeJhEC+m6HkHDSzZ+e3CU= 2022-09-03T06:41:52.640 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:x1PAxbTeJhEC+m6HkHDSzZ+e3CU=', 'Content-Length': '0'} 2022-09-03T06:41:52.643 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000088e752ebf24f1430-006312f730-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.644 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:52.644 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:52.645 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:52.645 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:52.646 DEBUG:boto:Method: GET 2022-09-03T06:41:52.647 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:41:52.647 DEBUG:boto:Data: 2022-09-03T06:41:52.648 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.648 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:52.649 DEBUG:boto:Port: 8001 2022-09-03T06:41:52.649 DEBUG:boto:Params: {} 2022-09-03T06:41:52.651 DEBUG:boto:Token: None 2022-09-03T06:41:52.653 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/ 2022-09-03T06:41:52.654 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XIc/rNOXN+NxSx7QTAy7WlMptXM= 2022-09-03T06:41:52.655 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XIc/rNOXN+NxSx7QTAy7WlMptXM=', 'Content-Length': '0'} 2022-09-03T06:41:52.662 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000020c9e536caa7ca2e-006312f730-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.703 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:52.704 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:52.705 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:52.706 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:52.706 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:52.707 DEBUG:boto:Method: GET 2022-09-03T06:41:52.708 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:41:52.709 DEBUG:boto:Data: 2022-09-03T06:41:52.709 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.710 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:52.711 DEBUG:boto:Port: 8000 2022-09-03T06:41:52.711 DEBUG:boto:Params: {} 2022-09-03T06:41:52.714 DEBUG:boto:Token: None 2022-09-03T06:41:52.716 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/ 2022-09-03T06:41:52.717 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XIc/rNOXN+NxSx7QTAy7WlMptXM= 2022-09-03T06:41:52.718 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XIc/rNOXN+NxSx7QTAy7WlMptXM=', 'Content-Length': '0'} 2022-09-03T06:41:52.723 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000043ecdf27d8ab042f-006312f730-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.724 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:52.725 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:41:52.726 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:52.727 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:52.728 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.729 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.730 DEBUG:boto:Method: GET 2022-09-03T06:41:52.730 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.731 DEBUG:boto:Data: 2022-09-03T06:41:52.732 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.733 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:52.735 DEBUG:boto:Port: 8001 2022-09-03T06:41:52.736 DEBUG:boto:Params: {} 2022-09-03T06:41:52.738 DEBUG:boto:Token: None 2022-09-03T06:41:52.740 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.741 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oMysYf4Q+zPbF2/R/gH33AyWHDg= 2022-09-03T06:41:52.743 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oMysYf4Q+zPbF2/R/gH33AyWHDg=', 'Content-Length': '0'} 2022-09-03T06:41:52.754 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001adaa795a6f6908e-006312f730-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.756 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:52.756 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:52.757 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.757 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.758 DEBUG:boto:Method: GET 2022-09-03T06:41:52.758 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.759 DEBUG:boto:Data: 2022-09-03T06:41:52.760 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.760 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:52.761 DEBUG:boto:Port: 8000 2022-09-03T06:41:52.761 DEBUG:boto:Params: {} 2022-09-03T06:41:52.763 DEBUG:boto:Token: None 2022-09-03T06:41:52.763 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.764 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oMysYf4Q+zPbF2/R/gH33AyWHDg= 2022-09-03T06:41:52.765 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oMysYf4Q+zPbF2/R/gH33AyWHDg=', 'Content-Length': '0'} 2022-09-03T06:41:52.782 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009759897f91406424-006312f730-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.783 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:52.784 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:52.784 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.785 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.785 DEBUG:boto:Method: HEAD 2022-09-03T06:41:52.786 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.786 DEBUG:boto:Data: 2022-09-03T06:41:52.787 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.787 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:52.792 DEBUG:boto:Port: 8001 2022-09-03T06:41:52.793 DEBUG:boto:Params: {} 2022-09-03T06:41:52.794 DEBUG:boto:Token: None 2022-09-03T06:41:52.795 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.797 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s3ezbUd7a2CaAaA1gKu3iUgJsUA= 2022-09-03T06:41:52.798 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s3ezbUd7a2CaAaA1gKu3iUgJsUA=', 'Content-Length': '0'} 2022-09-03T06:41:52.804 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000028ac870b8afa98d7-006312f730-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.806 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:52.807 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:52.807 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.808 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.809 DEBUG:boto:Method: HEAD 2022-09-03T06:41:52.810 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.811 DEBUG:boto:Data: 2022-09-03T06:41:52.811 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.812 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:52.812 DEBUG:boto:Port: 8000 2022-09-03T06:41:52.813 DEBUG:boto:Params: {} 2022-09-03T06:41:52.814 DEBUG:boto:Token: None 2022-09-03T06:41:52.815 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.815 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s3ezbUd7a2CaAaA1gKu3iUgJsUA= 2022-09-03T06:41:52.816 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s3ezbUd7a2CaAaA1gKu3iUgJsUA=', 'Content-Length': '0'} 2022-09-03T06:41:52.818 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007309f40158bf0b2d-006312f730-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.820 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:41:52.820 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:52.821 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:52.821 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.822 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.823 DEBUG:boto:Method: GET 2022-09-03T06:41:52.828 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.829 DEBUG:boto:Data: 2022-09-03T06:41:52.831 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.832 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:52.832 DEBUG:boto:Port: 8001 2022-09-03T06:41:52.833 DEBUG:boto:Params: {} 2022-09-03T06:41:52.833 DEBUG:boto:Token: None 2022-09-03T06:41:52.834 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.836 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oMysYf4Q+zPbF2/R/gH33AyWHDg= 2022-09-03T06:41:52.837 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oMysYf4Q+zPbF2/R/gH33AyWHDg=', 'Content-Length': '0'} 2022-09-03T06:41:52.844 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000936a59e108be80ca-006312f730-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.851 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:52.852 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:52.853 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.854 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.855 DEBUG:boto:Method: GET 2022-09-03T06:41:52.857 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.859 DEBUG:boto:Data: 2022-09-03T06:41:52.859 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.860 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:52.861 DEBUG:boto:Port: 8000 2022-09-03T06:41:52.862 DEBUG:boto:Params: {} 2022-09-03T06:41:52.866 DEBUG:boto:Token: None 2022-09-03T06:41:52.867 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:52.868 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oMysYf4Q+zPbF2/R/gH33AyWHDg= 2022-09-03T06:41:52.869 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oMysYf4Q+zPbF2/R/gH33AyWHDg=', 'Content-Length': '0'} 2022-09-03T06:41:52.876 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d16aecf9402f93eb-006312f730-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.878 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:52.879 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:52.879 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:52.880 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:52.880 DEBUG:boto:Method: GET 2022-09-03T06:41:52.881 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:41:52.882 DEBUG:boto:Data: 2022-09-03T06:41:52.883 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.892 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:52.892 DEBUG:boto:Port: 8001 2022-09-03T06:41:52.893 DEBUG:boto:Params: {} 2022-09-03T06:41:52.894 DEBUG:boto:Token: None 2022-09-03T06:41:52.894 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/ 2022-09-03T06:41:52.895 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XIc/rNOXN+NxSx7QTAy7WlMptXM= 2022-09-03T06:41:52.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XIc/rNOXN+NxSx7QTAy7WlMptXM=', 'Content-Length': '0'} 2022-09-03T06:41:52.898 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000017ba98e7582f0368-006312f730-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.899 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:52.900 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:52.901 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:52.901 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:52.902 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:52.903 DEBUG:boto:Method: GET 2022-09-03T06:41:52.904 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:41:52.904 DEBUG:boto:Data: 2022-09-03T06:41:52.905 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.905 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:52.906 DEBUG:boto:Port: 8000 2022-09-03T06:41:52.906 DEBUG:boto:Params: {} 2022-09-03T06:41:52.907 DEBUG:boto:Token: None 2022-09-03T06:41:52.908 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:52 GMT /dsgvcy-75/ 2022-09-03T06:41:52.908 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XIc/rNOXN+NxSx7QTAy7WlMptXM= 2022-09-03T06:41:52.909 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:52 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XIc/rNOXN+NxSx7QTAy7WlMptXM=', 'Content-Length': '0'} 2022-09-03T06:41:52.918 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000071db03a9fc30ca2b-006312f730-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:52 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:52.954 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:52.955 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:52.991 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:52.992 DEBUG:boto:Method: HEAD 2022-09-03T06:41:52.993 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:41:52.994 DEBUG:boto:Data: 2022-09-03T06:41:52.995 DEBUG:boto:Headers: {} 2022-09-03T06:41:52.996 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:52.997 DEBUG:boto:Port: 8001 2022-09-03T06:41:52.998 DEBUG:boto:Params: {} 2022-09-03T06:41:53.000 DEBUG:boto:Token: None 2022-09-03T06:41:53.001 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/obj68 2022-09-03T06:41:53.002 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L/7ZhrEDxIzWY3cg18E4l+pq96s= 2022-09-03T06:41:53.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L/7ZhrEDxIzWY3cg18E4l+pq96s=', 'Content-Length': '0'} 2022-09-03T06:41:53.007 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a1fcc55b2c11d3f-006312f731-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.008 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:53.009 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:53.018 DEBUG:boto:Method: HEAD 2022-09-03T06:41:53.019 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:41:53.020 DEBUG:boto:Data: 2022-09-03T06:41:53.022 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.026 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:53.027 DEBUG:boto:Port: 8000 2022-09-03T06:41:53.028 DEBUG:boto:Params: {} 2022-09-03T06:41:53.045 DEBUG:boto:Token: None 2022-09-03T06:41:53.046 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/obj68 2022-09-03T06:41:53.067 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L/7ZhrEDxIzWY3cg18E4l+pq96s= 2022-09-03T06:41:53.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L/7ZhrEDxIzWY3cg18E4l+pq96s=', 'Content-Length': '0'} 2022-09-03T06:41:53.105 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000047c5b86b6c87a1ca-006312f731-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.107 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:41:53.108 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:53.109 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:53.110 DEBUG:boto:Method: GET 2022-09-03T06:41:53.111 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:41:53.112 DEBUG:boto:Data: 2022-09-03T06:41:53.113 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.115 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:53.116 DEBUG:boto:Port: 8001 2022-09-03T06:41:53.117 DEBUG:boto:Params: {} 2022-09-03T06:41:53.119 DEBUG:boto:Token: None 2022-09-03T06:41:53.120 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/obj68 2022-09-03T06:41:53.121 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TUzY8YsknR96+Sw0NnFvtU7X/ic= 2022-09-03T06:41:53.122 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TUzY8YsknR96+Sw0NnFvtU7X/ic=', 'Content-Length': '0'} 2022-09-03T06:41:53.136 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bdd485c79fe6d3b8-006312f731-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.147 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:53.148 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:53.150 DEBUG:boto:Method: GET 2022-09-03T06:41:53.153 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:41:53.159 DEBUG:boto:Data: 2022-09-03T06:41:53.161 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.164 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:53.165 DEBUG:boto:Port: 8000 2022-09-03T06:41:53.167 DEBUG:boto:Params: {} 2022-09-03T06:41:53.168 DEBUG:boto:Token: None 2022-09-03T06:41:53.169 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/obj68 2022-09-03T06:41:53.170 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TUzY8YsknR96+Sw0NnFvtU7X/ic= 2022-09-03T06:41:53.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TUzY8YsknR96+Sw0NnFvtU7X/ic=', 'Content-Length': '0'} 2022-09-03T06:41:53.177 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f0ebe3ba59bc0cab-006312f731-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.194 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:53.195 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:53.196 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:53.197 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:53.198 DEBUG:boto:Method: GET 2022-09-03T06:41:53.199 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:41:53.200 DEBUG:boto:Data: 2022-09-03T06:41:53.201 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.202 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:53.203 DEBUG:boto:Port: 8001 2022-09-03T06:41:53.204 DEBUG:boto:Params: {} 2022-09-03T06:41:53.205 DEBUG:boto:Token: None 2022-09-03T06:41:53.207 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/ 2022-09-03T06:41:53.211 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:doLhH4mA0YkYPUedQZXs6xg2CFE= 2022-09-03T06:41:53.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:doLhH4mA0YkYPUedQZXs6xg2CFE=', 'Content-Length': '0'} 2022-09-03T06:41:53.229 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000522d8e549101bcd0-006312f731-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.230 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:53.232 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:53.233 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:53.234 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:53.235 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:53.236 DEBUG:boto:Method: GET 2022-09-03T06:41:53.237 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:41:53.238 DEBUG:boto:Data: 2022-09-03T06:41:53.239 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.240 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:53.262 DEBUG:boto:Port: 8000 2022-09-03T06:41:53.262 DEBUG:boto:Params: {} 2022-09-03T06:41:53.263 DEBUG:boto:Token: None 2022-09-03T06:41:53.264 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/ 2022-09-03T06:41:53.264 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:doLhH4mA0YkYPUedQZXs6xg2CFE= 2022-09-03T06:41:53.265 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:doLhH4mA0YkYPUedQZXs6xg2CFE=', 'Content-Length': '0'} 2022-09-03T06:41:53.268 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008f5c373820ebfcc7-006312f731-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.269 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:53.270 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:41:53.271 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:53.271 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:53.273 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.273 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.274 DEBUG:boto:Method: GET 2022-09-03T06:41:53.275 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.275 DEBUG:boto:Data: 2022-09-03T06:41:53.276 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.276 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:53.277 DEBUG:boto:Port: 8001 2022-09-03T06:41:53.278 DEBUG:boto:Params: {} 2022-09-03T06:41:53.278 DEBUG:boto:Token: None 2022-09-03T06:41:53.279 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.280 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dz8u4nCjmfqMN50MVXbxupXkpr0= 2022-09-03T06:41:53.281 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dz8u4nCjmfqMN50MVXbxupXkpr0=', 'Content-Length': '0'} 2022-09-03T06:41:53.284 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000047ac0d19c27374d-006312f731-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.285 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:53.285 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:53.286 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.287 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.287 DEBUG:boto:Method: GET 2022-09-03T06:41:53.288 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.288 DEBUG:boto:Data: 2022-09-03T06:41:53.289 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.290 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:53.290 DEBUG:boto:Port: 8000 2022-09-03T06:41:53.291 DEBUG:boto:Params: {} 2022-09-03T06:41:53.292 DEBUG:boto:Token: None 2022-09-03T06:41:53.293 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.294 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dz8u4nCjmfqMN50MVXbxupXkpr0= 2022-09-03T06:41:53.295 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dz8u4nCjmfqMN50MVXbxupXkpr0=', 'Content-Length': '0'} 2022-09-03T06:41:53.297 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dbc69c8ca988d9ae-006312f731-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.325 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:53.326 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:53.330 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.331 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.332 DEBUG:boto:Method: HEAD 2022-09-03T06:41:53.332 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.333 DEBUG:boto:Data: 2022-09-03T06:41:53.333 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.334 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:53.335 DEBUG:boto:Port: 8001 2022-09-03T06:41:53.335 DEBUG:boto:Params: {} 2022-09-03T06:41:53.336 DEBUG:boto:Token: None 2022-09-03T06:41:53.337 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.338 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vA7MLiSWhSRREWZHcdXyA8MDROU= 2022-09-03T06:41:53.339 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vA7MLiSWhSRREWZHcdXyA8MDROU=', 'Content-Length': '0'} 2022-09-03T06:41:53.342 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001642cf0b9f43144e-006312f731-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.343 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:53.344 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:53.345 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.346 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.347 DEBUG:boto:Method: HEAD 2022-09-03T06:41:53.348 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.349 DEBUG:boto:Data: 2022-09-03T06:41:53.350 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.351 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:53.352 DEBUG:boto:Port: 8000 2022-09-03T06:41:53.353 DEBUG:boto:Params: {} 2022-09-03T06:41:53.354 DEBUG:boto:Token: None 2022-09-03T06:41:53.355 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.356 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vA7MLiSWhSRREWZHcdXyA8MDROU= 2022-09-03T06:41:53.358 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vA7MLiSWhSRREWZHcdXyA8MDROU=', 'Content-Length': '0'} 2022-09-03T06:41:53.362 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000513669f2d1f35324-006312f731-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.364 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:41:53.365 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:53.366 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:53.367 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.368 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.369 DEBUG:boto:Method: GET 2022-09-03T06:41:53.370 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.371 DEBUG:boto:Data: 2022-09-03T06:41:53.373 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.374 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:53.375 DEBUG:boto:Port: 8001 2022-09-03T06:41:53.376 DEBUG:boto:Params: {} 2022-09-03T06:41:53.377 DEBUG:boto:Token: None 2022-09-03T06:41:53.378 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.379 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dz8u4nCjmfqMN50MVXbxupXkpr0= 2022-09-03T06:41:53.381 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dz8u4nCjmfqMN50MVXbxupXkpr0=', 'Content-Length': '0'} 2022-09-03T06:41:53.393 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b48dde46c9dd840-006312f731-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.395 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:53.398 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:53.399 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.400 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.401 DEBUG:boto:Method: GET 2022-09-03T06:41:53.402 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.403 DEBUG:boto:Data: 2022-09-03T06:41:53.404 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.405 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:53.406 DEBUG:boto:Port: 8000 2022-09-03T06:41:53.407 DEBUG:boto:Params: {} 2022-09-03T06:41:53.408 DEBUG:boto:Token: None 2022-09-03T06:41:53.409 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:53.410 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dz8u4nCjmfqMN50MVXbxupXkpr0= 2022-09-03T06:41:53.411 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dz8u4nCjmfqMN50MVXbxupXkpr0=', 'Content-Length': '0'} 2022-09-03T06:41:53.414 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001dcf260a9e590068-006312f731-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.416 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:53.417 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:53.419 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:53.420 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:53.421 DEBUG:boto:Method: GET 2022-09-03T06:41:53.422 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:41:53.423 DEBUG:boto:Data: 2022-09-03T06:41:53.425 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.426 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:53.427 DEBUG:boto:Port: 8001 2022-09-03T06:41:53.428 DEBUG:boto:Params: {} 2022-09-03T06:41:53.429 DEBUG:boto:Token: None 2022-09-03T06:41:53.430 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/ 2022-09-03T06:41:53.431 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:doLhH4mA0YkYPUedQZXs6xg2CFE= 2022-09-03T06:41:53.433 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:doLhH4mA0YkYPUedQZXs6xg2CFE=', 'Content-Length': '0'} 2022-09-03T06:41:53.443 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004cae439d56fa7947-006312f731-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.444 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:53.446 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:53.447 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:53.448 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:53.449 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:53.450 DEBUG:boto:Method: GET 2022-09-03T06:41:53.451 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:41:53.453 DEBUG:boto:Data: 2022-09-03T06:41:53.453 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.454 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:53.455 DEBUG:boto:Port: 8000 2022-09-03T06:41:53.458 DEBUG:boto:Params: {} 2022-09-03T06:41:53.460 DEBUG:boto:Token: None 2022-09-03T06:41:53.461 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/ 2022-09-03T06:41:53.462 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:doLhH4mA0YkYPUedQZXs6xg2CFE= 2022-09-03T06:41:53.463 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:doLhH4mA0YkYPUedQZXs6xg2CFE=', 'Content-Length': '0'} 2022-09-03T06:41:53.471 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007cc3010dd9ea4043-006312f731-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.473 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:53.475 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:53.476 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:53.478 DEBUG:boto:Method: HEAD 2022-09-03T06:41:53.479 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:41:53.480 DEBUG:boto:Data: 2022-09-03T06:41:53.481 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.482 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:53.483 DEBUG:boto:Port: 8001 2022-09-03T06:41:53.484 DEBUG:boto:Params: {} 2022-09-03T06:41:53.486 DEBUG:boto:Token: None 2022-09-03T06:41:53.487 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/obj76 2022-09-03T06:41:53.488 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qqs+7qTiVOO7XUDerIEXyZP/j0A= 2022-09-03T06:41:53.490 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qqs+7qTiVOO7XUDerIEXyZP/j0A=', 'Content-Length': '0'} 2022-09-03T06:41:53.497 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000050fe1067ebd74fbc-006312f731-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.499 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:53.500 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:53.501 DEBUG:boto:Method: HEAD 2022-09-03T06:41:53.502 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:41:53.504 DEBUG:boto:Data: 2022-09-03T06:41:53.505 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.506 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:53.507 DEBUG:boto:Port: 8000 2022-09-03T06:41:53.508 DEBUG:boto:Params: {} 2022-09-03T06:41:53.509 DEBUG:boto:Token: None 2022-09-03T06:41:53.510 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/obj76 2022-09-03T06:41:53.512 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qqs+7qTiVOO7XUDerIEXyZP/j0A= 2022-09-03T06:41:53.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qqs+7qTiVOO7XUDerIEXyZP/j0A=', 'Content-Length': '0'} 2022-09-03T06:41:53.516 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000037d3bd757d28a106-006312f731-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.517 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:41:53.517 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:53.518 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:53.518 DEBUG:boto:Method: GET 2022-09-03T06:41:53.519 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:41:53.520 DEBUG:boto:Data: 2022-09-03T06:41:53.520 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.521 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:53.532 DEBUG:boto:Port: 8001 2022-09-03T06:41:53.534 DEBUG:boto:Params: {} 2022-09-03T06:41:53.535 DEBUG:boto:Token: None 2022-09-03T06:41:53.536 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/obj76 2022-09-03T06:41:53.538 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EXpqHT/EZn0T6cZrEF9jSBZ4SMc= 2022-09-03T06:41:53.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EXpqHT/EZn0T6cZrEF9jSBZ4SMc=', 'Content-Length': '0'} 2022-09-03T06:41:53.547 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000067ad042c01276b4e-006312f731-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.549 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:53.550 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:53.551 DEBUG:boto:Method: GET 2022-09-03T06:41:53.552 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:41:53.553 DEBUG:boto:Data: 2022-09-03T06:41:53.556 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.557 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:53.562 DEBUG:boto:Port: 8000 2022-09-03T06:41:53.563 DEBUG:boto:Params: {} 2022-09-03T06:41:53.564 DEBUG:boto:Token: None 2022-09-03T06:41:53.565 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/obj76 2022-09-03T06:41:53.567 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EXpqHT/EZn0T6cZrEF9jSBZ4SMc= 2022-09-03T06:41:53.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EXpqHT/EZn0T6cZrEF9jSBZ4SMc=', 'Content-Length': '0'} 2022-09-03T06:41:53.573 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000007493eec966108e-006312f731-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.575 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:53.590 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:53.591 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:53.593 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:53.594 DEBUG:boto:Method: GET 2022-09-03T06:41:53.595 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:41:53.597 DEBUG:boto:Data: 2022-09-03T06:41:53.598 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.607 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:53.608 DEBUG:boto:Port: 8001 2022-09-03T06:41:53.609 DEBUG:boto:Params: {} 2022-09-03T06:41:53.611 DEBUG:boto:Token: None 2022-09-03T06:41:53.612 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/ 2022-09-03T06:41:53.614 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:doLhH4mA0YkYPUedQZXs6xg2CFE= 2022-09-03T06:41:53.617 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:doLhH4mA0YkYPUedQZXs6xg2CFE=', 'Content-Length': '0'} 2022-09-03T06:41:53.626 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000082612cd2ea08fd9f-006312f731-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.628 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:53.630 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:53.631 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:53.632 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:53.633 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:53.635 DEBUG:boto:Method: GET 2022-09-03T06:41:53.638 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:41:53.639 DEBUG:boto:Data: 2022-09-03T06:41:53.640 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.641 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:53.653 DEBUG:boto:Port: 8000 2022-09-03T06:41:53.666 DEBUG:boto:Params: {} 2022-09-03T06:41:53.668 DEBUG:boto:Token: None 2022-09-03T06:41:53.669 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/ 2022-09-03T06:41:53.671 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:doLhH4mA0YkYPUedQZXs6xg2CFE= 2022-09-03T06:41:53.673 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:doLhH4mA0YkYPUedQZXs6xg2CFE=', 'Content-Length': '0'} 2022-09-03T06:41:53.686 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009006489658f85e47-006312f731-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.687 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:53.688 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:41:53.689 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:53.689 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:53.697 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.698 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.699 DEBUG:boto:Method: GET 2022-09-03T06:41:53.700 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.701 DEBUG:boto:Data: 2022-09-03T06:41:53.702 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.703 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:53.704 DEBUG:boto:Port: 8001 2022-09-03T06:41:53.705 DEBUG:boto:Params: {} 2022-09-03T06:41:53.706 DEBUG:boto:Token: None 2022-09-03T06:41:53.708 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.710 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nFo9Xd5fETZ4B76Gyco2hH6Q93I= 2022-09-03T06:41:53.711 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nFo9Xd5fETZ4B76Gyco2hH6Q93I=', 'Content-Length': '0'} 2022-09-03T06:41:53.716 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000daf1490c4223f42e-006312f731-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.718 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:53.719 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:53.720 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.721 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.722 DEBUG:boto:Method: GET 2022-09-03T06:41:53.723 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.724 DEBUG:boto:Data: 2022-09-03T06:41:53.724 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.725 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:53.726 DEBUG:boto:Port: 8000 2022-09-03T06:41:53.727 DEBUG:boto:Params: {} 2022-09-03T06:41:53.728 DEBUG:boto:Token: None 2022-09-03T06:41:53.729 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.730 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nFo9Xd5fETZ4B76Gyco2hH6Q93I= 2022-09-03T06:41:53.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nFo9Xd5fETZ4B76Gyco2hH6Q93I=', 'Content-Length': '0'} 2022-09-03T06:41:53.736 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009c7470fe4f7d5117-006312f731-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.742 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:53.744 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:53.744 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.745 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.746 DEBUG:boto:Method: HEAD 2022-09-03T06:41:53.747 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.748 DEBUG:boto:Data: 2022-09-03T06:41:53.749 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.750 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:53.750 DEBUG:boto:Port: 8001 2022-09-03T06:41:53.751 DEBUG:boto:Params: {} 2022-09-03T06:41:53.752 DEBUG:boto:Token: None 2022-09-03T06:41:53.753 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.754 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3waGa3DcjoZuXRU5li2p6MOMpFw= 2022-09-03T06:41:53.756 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3waGa3DcjoZuXRU5li2p6MOMpFw=', 'Content-Length': '0'} 2022-09-03T06:41:53.763 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000616e7a59b28cd68d-006312f731-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.766 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:53.767 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:53.767 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.768 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.769 DEBUG:boto:Method: HEAD 2022-09-03T06:41:53.774 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.775 DEBUG:boto:Data: 2022-09-03T06:41:53.775 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.776 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:53.776 DEBUG:boto:Port: 8000 2022-09-03T06:41:53.777 DEBUG:boto:Params: {} 2022-09-03T06:41:53.777 DEBUG:boto:Token: None 2022-09-03T06:41:53.778 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.779 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3waGa3DcjoZuXRU5li2p6MOMpFw= 2022-09-03T06:41:53.779 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3waGa3DcjoZuXRU5li2p6MOMpFw=', 'Content-Length': '0'} 2022-09-03T06:41:53.781 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009fa758314883421d-006312f731-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.782 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:41:53.783 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:53.783 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:53.784 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.784 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.785 DEBUG:boto:Method: GET 2022-09-03T06:41:53.785 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.798 DEBUG:boto:Data: 2022-09-03T06:41:53.809 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.811 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:53.812 DEBUG:boto:Port: 8001 2022-09-03T06:41:53.813 DEBUG:boto:Params: {} 2022-09-03T06:41:53.814 DEBUG:boto:Token: None 2022-09-03T06:41:53.815 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.816 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nFo9Xd5fETZ4B76Gyco2hH6Q93I= 2022-09-03T06:41:53.818 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nFo9Xd5fETZ4B76Gyco2hH6Q93I=', 'Content-Length': '0'} 2022-09-03T06:41:53.821 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b25939e8366f3930-006312f731-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.822 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:53.823 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:53.825 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.826 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.827 DEBUG:boto:Method: GET 2022-09-03T06:41:53.828 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.830 DEBUG:boto:Data: 2022-09-03T06:41:53.831 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.832 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:53.833 DEBUG:boto:Port: 8000 2022-09-03T06:41:53.834 DEBUG:boto:Params: {} 2022-09-03T06:41:53.835 DEBUG:boto:Token: None 2022-09-03T06:41:53.837 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:53.838 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nFo9Xd5fETZ4B76Gyco2hH6Q93I= 2022-09-03T06:41:53.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nFo9Xd5fETZ4B76Gyco2hH6Q93I=', 'Content-Length': '0'} 2022-09-03T06:41:53.842 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c3ea9a102eaa1212-006312f731-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.844 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:53.845 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:53.846 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:53.847 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:53.848 DEBUG:boto:Method: GET 2022-09-03T06:41:53.849 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:41:53.851 DEBUG:boto:Data: 2022-09-03T06:41:53.852 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.852 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:53.853 DEBUG:boto:Port: 8001 2022-09-03T06:41:53.854 DEBUG:boto:Params: {} 2022-09-03T06:41:53.856 DEBUG:boto:Token: None 2022-09-03T06:41:53.857 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/ 2022-09-03T06:41:53.858 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:doLhH4mA0YkYPUedQZXs6xg2CFE= 2022-09-03T06:41:53.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:doLhH4mA0YkYPUedQZXs6xg2CFE=', 'Content-Length': '0'} 2022-09-03T06:41:53.864 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000062512450710b1c36-006312f731-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.865 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:53.866 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:53.867 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:53.867 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:53.868 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:53.868 DEBUG:boto:Method: GET 2022-09-03T06:41:53.869 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:41:53.869 DEBUG:boto:Data: 2022-09-03T06:41:53.870 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.870 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:53.871 DEBUG:boto:Port: 8000 2022-09-03T06:41:53.871 DEBUG:boto:Params: {} 2022-09-03T06:41:53.872 DEBUG:boto:Token: None 2022-09-03T06:41:53.873 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/ 2022-09-03T06:41:53.874 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:doLhH4mA0YkYPUedQZXs6xg2CFE= 2022-09-03T06:41:53.875 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:doLhH4mA0YkYPUedQZXs6xg2CFE=', 'Content-Length': '0'} 2022-09-03T06:41:53.879 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f948cfa2d1503ff4-006312f731-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.880 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:53.881 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:53.881 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:53.882 DEBUG:boto:Method: HEAD 2022-09-03T06:41:53.882 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:41:53.883 DEBUG:boto:Data: 2022-09-03T06:41:53.883 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.884 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:53.884 DEBUG:boto:Port: 8001 2022-09-03T06:41:53.885 DEBUG:boto:Params: {} 2022-09-03T06:41:53.886 DEBUG:boto:Token: None 2022-09-03T06:41:53.888 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/obj85 2022-09-03T06:41:53.889 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HswzWqYoJox7Yo/AX16zfkzDq80= 2022-09-03T06:41:53.890 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HswzWqYoJox7Yo/AX16zfkzDq80=', 'Content-Length': '0'} 2022-09-03T06:41:53.893 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000695b967bd27c6682-006312f731-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.895 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:53.896 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:53.897 DEBUG:boto:Method: HEAD 2022-09-03T06:41:53.902 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:41:53.903 DEBUG:boto:Data: 2022-09-03T06:41:53.904 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.905 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:53.907 DEBUG:boto:Port: 8000 2022-09-03T06:41:53.908 DEBUG:boto:Params: {} 2022-09-03T06:41:53.910 DEBUG:boto:Token: None 2022-09-03T06:41:53.911 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/obj85 2022-09-03T06:41:53.913 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HswzWqYoJox7Yo/AX16zfkzDq80= 2022-09-03T06:41:53.915 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HswzWqYoJox7Yo/AX16zfkzDq80=', 'Content-Length': '0'} 2022-09-03T06:41:53.918 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000095975704af491b6e-006312f731-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.920 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:41:53.921 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:53.970 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:53.971 DEBUG:boto:Method: GET 2022-09-03T06:41:53.972 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:41:53.972 DEBUG:boto:Data: 2022-09-03T06:41:53.973 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.974 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:53.975 DEBUG:boto:Port: 8001 2022-09-03T06:41:53.976 DEBUG:boto:Params: {} 2022-09-03T06:41:53.977 DEBUG:boto:Token: None 2022-09-03T06:41:53.977 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:53 GMT /dsgvcy-75/obj85 2022-09-03T06:41:53.978 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kdUG0Si1yovGLzAlZglEUJfJLVY= 2022-09-03T06:41:53.979 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:53 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kdUG0Si1yovGLzAlZglEUJfJLVY=', 'Content-Length': '0'} 2022-09-03T06:41:53.983 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fde98d84e8f1ce07-006312f731-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:53.984 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:53.985 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:53.987 DEBUG:boto:Method: GET 2022-09-03T06:41:53.988 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:41:53.988 DEBUG:boto:Data: 2022-09-03T06:41:53.989 DEBUG:boto:Headers: {} 2022-09-03T06:41:53.990 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:53.990 DEBUG:boto:Port: 8000 2022-09-03T06:41:53.991 DEBUG:boto:Params: {} 2022-09-03T06:41:54.002 DEBUG:boto:Token: None 2022-09-03T06:41:54.011 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj85 2022-09-03T06:41:54.012 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8p8TPBdIeRYl4pbkItdB5iTDXV8= 2022-09-03T06:41:54.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8p8TPBdIeRYl4pbkItdB5iTDXV8=', 'Content-Length': '0'} 2022-09-03T06:41:54.018 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b69fb5e6844250a4-006312f732-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.020 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:54.021 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:54.022 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:54.024 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:54.026 DEBUG:boto:Method: GET 2022-09-03T06:41:54.027 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:41:54.028 DEBUG:boto:Data: 2022-09-03T06:41:54.031 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.032 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:54.033 DEBUG:boto:Port: 8001 2022-09-03T06:41:54.034 DEBUG:boto:Params: {} 2022-09-03T06:41:54.035 DEBUG:boto:Token: None 2022-09-03T06:41:54.035 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/ 2022-09-03T06:41:54.036 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l+Up7SYRfnPdSuWeJBp+wW+jezY= 2022-09-03T06:41:54.037 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l+Up7SYRfnPdSuWeJBp+wW+jezY=', 'Content-Length': '0'} 2022-09-03T06:41:54.046 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e7547ec31b4e2cc9-006312f732-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.047 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:54.048 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:54.049 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:54.052 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:54.068 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:54.068 DEBUG:boto:Method: GET 2022-09-03T06:41:54.069 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:41:54.070 DEBUG:boto:Data: 2022-09-03T06:41:54.070 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.075 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:54.077 DEBUG:boto:Port: 8000 2022-09-03T06:41:54.077 DEBUG:boto:Params: {} 2022-09-03T06:41:54.079 DEBUG:boto:Token: None 2022-09-03T06:41:54.080 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/ 2022-09-03T06:41:54.081 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l+Up7SYRfnPdSuWeJBp+wW+jezY= 2022-09-03T06:41:54.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l+Up7SYRfnPdSuWeJBp+wW+jezY=', 'Content-Length': '0'} 2022-09-03T06:41:54.085 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000902794206c3fd24d-006312f732-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.087 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:54.088 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:41:54.089 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:54.090 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:54.090 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.091 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.091 DEBUG:boto:Method: GET 2022-09-03T06:41:54.092 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.092 DEBUG:boto:Data: 2022-09-03T06:41:54.093 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.094 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:54.094 DEBUG:boto:Port: 8001 2022-09-03T06:41:54.095 DEBUG:boto:Params: {} 2022-09-03T06:41:54.096 DEBUG:boto:Token: None 2022-09-03T06:41:54.096 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.097 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zN4Zg4FSo+9FZ3+B9jyZWBH9aHQ= 2022-09-03T06:41:54.098 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zN4Zg4FSo+9FZ3+B9jyZWBH9aHQ=', 'Content-Length': '0'} 2022-09-03T06:41:54.102 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b3418efbdd210de7-006312f732-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.104 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:54.105 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:54.106 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.107 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.110 DEBUG:boto:Method: GET 2022-09-03T06:41:54.112 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.112 DEBUG:boto:Data: 2022-09-03T06:41:54.113 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.114 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:54.115 DEBUG:boto:Port: 8000 2022-09-03T06:41:54.116 DEBUG:boto:Params: {} 2022-09-03T06:41:54.117 DEBUG:boto:Token: None 2022-09-03T06:41:54.125 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.127 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zN4Zg4FSo+9FZ3+B9jyZWBH9aHQ= 2022-09-03T06:41:54.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zN4Zg4FSo+9FZ3+B9jyZWBH9aHQ=', 'Content-Length': '0'} 2022-09-03T06:41:54.132 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000066b74ee1fe5c24ee-006312f732-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.134 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:54.136 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:54.136 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.137 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.138 DEBUG:boto:Method: HEAD 2022-09-03T06:41:54.141 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.143 DEBUG:boto:Data: 2022-09-03T06:41:54.143 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.144 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:54.144 DEBUG:boto:Port: 8001 2022-09-03T06:41:54.145 DEBUG:boto:Params: {} 2022-09-03T06:41:54.145 DEBUG:boto:Token: None 2022-09-03T06:41:54.146 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.148 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gVnRACm4jNa8fsS9K1m4vBm3wEo= 2022-09-03T06:41:54.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gVnRACm4jNa8fsS9K1m4vBm3wEo=', 'Content-Length': '0'} 2022-09-03T06:41:54.150 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e0a06fa50799af7-006312f732-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.152 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:54.153 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:54.154 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.154 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.155 DEBUG:boto:Method: HEAD 2022-09-03T06:41:54.155 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.156 DEBUG:boto:Data: 2022-09-03T06:41:54.157 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.157 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:54.158 DEBUG:boto:Port: 8000 2022-09-03T06:41:54.158 DEBUG:boto:Params: {} 2022-09-03T06:41:54.159 DEBUG:boto:Token: None 2022-09-03T06:41:54.160 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.161 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gVnRACm4jNa8fsS9K1m4vBm3wEo= 2022-09-03T06:41:54.161 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gVnRACm4jNa8fsS9K1m4vBm3wEo=', 'Content-Length': '0'} 2022-09-03T06:41:54.166 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000031904effac244a7d-006312f732-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.167 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:41:54.168 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:54.169 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:54.169 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.170 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.171 DEBUG:boto:Method: GET 2022-09-03T06:41:54.171 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.172 DEBUG:boto:Data: 2022-09-03T06:41:54.173 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.173 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:54.174 DEBUG:boto:Port: 8001 2022-09-03T06:41:54.175 DEBUG:boto:Params: {} 2022-09-03T06:41:54.176 DEBUG:boto:Token: None 2022-09-03T06:41:54.176 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.177 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zN4Zg4FSo+9FZ3+B9jyZWBH9aHQ= 2022-09-03T06:41:54.178 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zN4Zg4FSo+9FZ3+B9jyZWBH9aHQ=', 'Content-Length': '0'} 2022-09-03T06:41:54.182 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000558e5fd1163103c4-006312f732-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.184 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:54.185 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:54.186 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.187 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.189 DEBUG:boto:Method: GET 2022-09-03T06:41:54.190 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.200 DEBUG:boto:Data: 2022-09-03T06:41:54.201 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.203 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:54.204 DEBUG:boto:Port: 8000 2022-09-03T06:41:54.205 DEBUG:boto:Params: {} 2022-09-03T06:41:54.207 DEBUG:boto:Token: None 2022-09-03T06:41:54.208 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:54.209 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zN4Zg4FSo+9FZ3+B9jyZWBH9aHQ= 2022-09-03T06:41:54.210 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zN4Zg4FSo+9FZ3+B9jyZWBH9aHQ=', 'Content-Length': '0'} 2022-09-03T06:41:54.223 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000069413a82cdcd656d-006312f732-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.224 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:54.224 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:54.225 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:54.225 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:54.228 DEBUG:boto:Method: GET 2022-09-03T06:41:54.228 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:41:54.229 DEBUG:boto:Data: 2022-09-03T06:41:54.229 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.230 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:54.231 DEBUG:boto:Port: 8001 2022-09-03T06:41:54.231 DEBUG:boto:Params: {} 2022-09-03T06:41:54.232 DEBUG:boto:Token: None 2022-09-03T06:41:54.233 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/ 2022-09-03T06:41:54.234 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l+Up7SYRfnPdSuWeJBp+wW+jezY= 2022-09-03T06:41:54.234 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l+Up7SYRfnPdSuWeJBp+wW+jezY=', 'Content-Length': '0'} 2022-09-03T06:41:54.238 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000957244d62c440e57-006312f732-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.242 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:54.243 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:54.243 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:54.244 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:54.249 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:54.250 DEBUG:boto:Method: GET 2022-09-03T06:41:54.250 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:41:54.251 DEBUG:boto:Data: 2022-09-03T06:41:54.251 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.252 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:54.252 DEBUG:boto:Port: 8000 2022-09-03T06:41:54.253 DEBUG:boto:Params: {} 2022-09-03T06:41:54.253 DEBUG:boto:Token: None 2022-09-03T06:41:54.254 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/ 2022-09-03T06:41:54.255 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l+Up7SYRfnPdSuWeJBp+wW+jezY= 2022-09-03T06:41:54.256 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l+Up7SYRfnPdSuWeJBp+wW+jezY=', 'Content-Length': '0'} 2022-09-03T06:41:54.259 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e41e9db69b91b661-006312f732-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.260 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:54.260 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:54.261 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:54.262 DEBUG:boto:Method: HEAD 2022-09-03T06:41:54.262 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:41:54.263 DEBUG:boto:Data: 2022-09-03T06:41:54.263 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.264 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:54.264 DEBUG:boto:Port: 8001 2022-09-03T06:41:54.265 DEBUG:boto:Params: {} 2022-09-03T06:41:54.267 DEBUG:boto:Token: None 2022-09-03T06:41:54.267 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj95 2022-09-03T06:41:54.269 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nXkgJNp6JmRul3BINextKu3caHI= 2022-09-03T06:41:54.269 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nXkgJNp6JmRul3BINextKu3caHI=', 'Content-Length': '0'} 2022-09-03T06:41:54.271 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000922c4141e632048e-006312f732-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.272 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:54.273 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:54.274 DEBUG:boto:Method: HEAD 2022-09-03T06:41:54.274 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:41:54.275 DEBUG:boto:Data: 2022-09-03T06:41:54.287 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.288 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:54.289 DEBUG:boto:Port: 8000 2022-09-03T06:41:54.290 DEBUG:boto:Params: {} 2022-09-03T06:41:54.291 DEBUG:boto:Token: None 2022-09-03T06:41:54.292 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj95 2022-09-03T06:41:54.293 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nXkgJNp6JmRul3BINextKu3caHI= 2022-09-03T06:41:54.294 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nXkgJNp6JmRul3BINextKu3caHI=', 'Content-Length': '0'} 2022-09-03T06:41:54.296 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000425bff9e42670855-006312f732-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.297 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:41:54.298 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:54.299 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:54.300 DEBUG:boto:Method: GET 2022-09-03T06:41:54.301 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:41:54.301 DEBUG:boto:Data: 2022-09-03T06:41:54.302 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.302 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:54.303 DEBUG:boto:Port: 8001 2022-09-03T06:41:54.304 DEBUG:boto:Params: {} 2022-09-03T06:41:54.305 DEBUG:boto:Token: None 2022-09-03T06:41:54.306 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj95 2022-09-03T06:41:54.306 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8X66297A061B8YdEFevZ4XHFffQ= 2022-09-03T06:41:54.307 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8X66297A061B8YdEFevZ4XHFffQ=', 'Content-Length': '0'} 2022-09-03T06:41:54.310 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000243fba5496c5c267-006312f732-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.311 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:54.312 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:54.313 DEBUG:boto:Method: GET 2022-09-03T06:41:54.313 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:41:54.314 DEBUG:boto:Data: 2022-09-03T06:41:54.314 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.315 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:54.316 DEBUG:boto:Port: 8000 2022-09-03T06:41:54.316 DEBUG:boto:Params: {} 2022-09-03T06:41:54.317 DEBUG:boto:Token: None 2022-09-03T06:41:54.318 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj95 2022-09-03T06:41:54.318 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8X66297A061B8YdEFevZ4XHFffQ= 2022-09-03T06:41:54.319 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8X66297A061B8YdEFevZ4XHFffQ=', 'Content-Length': '0'} 2022-09-03T06:41:54.321 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000067c82533eae60fba-006312f732-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.362 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:54.364 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:54.364 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:54.368 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:54.369 DEBUG:boto:Method: GET 2022-09-03T06:41:54.370 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:41:54.371 DEBUG:boto:Data: 2022-09-03T06:41:54.372 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.373 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:54.374 DEBUG:boto:Port: 8001 2022-09-03T06:41:54.376 DEBUG:boto:Params: {} 2022-09-03T06:41:54.376 DEBUG:boto:Token: None 2022-09-03T06:41:54.377 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/ 2022-09-03T06:41:54.378 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l+Up7SYRfnPdSuWeJBp+wW+jezY= 2022-09-03T06:41:54.379 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l+Up7SYRfnPdSuWeJBp+wW+jezY=', 'Content-Length': '0'} 2022-09-03T06:41:54.381 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c36b86e803211000-006312f732-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.382 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:54.383 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:54.384 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:54.384 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:54.385 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:54.385 DEBUG:boto:Method: GET 2022-09-03T06:41:54.386 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:41:54.386 DEBUG:boto:Data: 2022-09-03T06:41:54.387 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.388 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:54.388 DEBUG:boto:Port: 8000 2022-09-03T06:41:54.389 DEBUG:boto:Params: {} 2022-09-03T06:41:54.389 DEBUG:boto:Token: None 2022-09-03T06:41:54.390 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/ 2022-09-03T06:41:54.391 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l+Up7SYRfnPdSuWeJBp+wW+jezY= 2022-09-03T06:41:54.391 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l+Up7SYRfnPdSuWeJBp+wW+jezY=', 'Content-Length': '0'} 2022-09-03T06:41:54.395 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001e1f23fba46abb64-006312f732-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.396 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:54.397 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-75 zones={test-zone3, test-zone1} 2022-09-03T06:41:54.397 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-75 zones={test-zone3, test-zone2} 2022-09-03T06:41:54.398 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:54.398 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:54.399 DEBUG:boto:Method: HEAD 2022-09-03T06:41:54.399 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:41:54.400 DEBUG:boto:Data: 2022-09-03T06:41:54.401 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.402 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:54.402 DEBUG:boto:Port: 8001 2022-09-03T06:41:54.403 DEBUG:boto:Params: {} 2022-09-03T06:41:54.403 DEBUG:boto:Token: None 2022-09-03T06:41:54.404 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/ 2022-09-03T06:41:54.405 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qdpzM5+T4httjosfuKhsc8Pf+ao= 2022-09-03T06:41:54.406 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qdpzM5+T4httjosfuKhsc8Pf+ao=', 'Content-Length': '0'} 2022-09-03T06:41:54.409 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '15'), ('X-RGW-Bytes-Used', '45'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a93825862571257f-006312f732-1851-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.410 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:54.411 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:54.414 DEBUG:boto:Method: HEAD 2022-09-03T06:41:54.416 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:41:54.416 DEBUG:boto:Data: 2022-09-03T06:41:54.417 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.418 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:54.419 DEBUG:boto:Port: 8000 2022-09-03T06:41:54.422 DEBUG:boto:Params: {} 2022-09-03T06:41:54.423 DEBUG:boto:Token: None 2022-09-03T06:41:54.424 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/ 2022-09-03T06:41:54.426 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qdpzM5+T4httjosfuKhsc8Pf+ao= 2022-09-03T06:41:54.427 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qdpzM5+T4httjosfuKhsc8Pf+ao=', 'Content-Length': '0'} 2022-09-03T06:41:54.430 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '15'), ('X-RGW-Bytes-Used', '45'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000481174d843d79ef4-006312f732-188c-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.430 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:41:54.431 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:54.431 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:54.432 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:54.432 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:54.433 DEBUG:boto:Method: GET 2022-09-03T06:41:54.434 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:54.435 DEBUG:boto:Data: 2022-09-03T06:41:54.435 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.436 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:54.437 DEBUG:boto:Port: 8001 2022-09-03T06:41:54.437 DEBUG:boto:Params: {} 2022-09-03T06:41:54.438 DEBUG:boto:Token: None 2022-09-03T06:41:54.439 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/?versions 2022-09-03T06:41:54.440 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cLX/2ex4pT0peA3OPZInfmH1z0E= 2022-09-03T06:41:54.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cLX/2ex4pT0peA3OPZInfmH1z0E=', 'Content-Length': '0'} 2022-09-03T06:41:54.444 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000c5de1571d72f67b-006312f732-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.445 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:54.448 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:41:54.449 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:41:54.450 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:41:54.451 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:41:54.452 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:41:54.453 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:41:54.454 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:41:54.456 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:41:54.457 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:41:54.458 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:41:54.458 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:41:54.459 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:41:54.460 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:41:54.461 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:41:54.462 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:41:54.463 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:41:54.464 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:54.465 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:54.466 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:54.467 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:54.468 DEBUG:boto:Method: GET 2022-09-03T06:41:54.469 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:54.470 DEBUG:boto:Data: 2022-09-03T06:41:54.471 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.472 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:54.473 DEBUG:boto:Port: 8000 2022-09-03T06:41:54.474 DEBUG:boto:Params: {} 2022-09-03T06:41:54.475 DEBUG:boto:Token: None 2022-09-03T06:41:54.476 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/?versions 2022-09-03T06:41:54.477 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cLX/2ex4pT0peA3OPZInfmH1z0E= 2022-09-03T06:41:54.478 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cLX/2ex4pT0peA3OPZInfmH1z0E=', 'Content-Length': '0'} 2022-09-03T06:41:54.481 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000830fe29f0394031-006312f732-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.482 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:54.484 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:41:54.485 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:41:54.485 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:41:54.485 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:41:54.486 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:41:54.487 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:41:54.487 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:41:54.488 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:41:54.489 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:41:54.490 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:41:54.492 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:41:54.493 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:41:54.493 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:41:54.494 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:41:54.495 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:41:54.496 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:54.497 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:54.498 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:54.499 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:54.500 DEBUG:boto:Method: GET 2022-09-03T06:41:54.501 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:54.502 DEBUG:boto:Data: 2022-09-03T06:41:54.503 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.504 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:54.505 DEBUG:boto:Port: 8001 2022-09-03T06:41:54.506 DEBUG:boto:Params: {} 2022-09-03T06:41:54.507 DEBUG:boto:Token: None 2022-09-03T06:41:54.508 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/?versions 2022-09-03T06:41:54.509 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cLX/2ex4pT0peA3OPZInfmH1z0E= 2022-09-03T06:41:54.510 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cLX/2ex4pT0peA3OPZInfmH1z0E=', 'Content-Length': '0'} 2022-09-03T06:41:54.514 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001f6507a4461dc4e7-006312f732-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.515 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:54.517 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:54.517 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:54.518 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:54.519 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:54.519 DEBUG:boto:Method: GET 2022-09-03T06:41:54.520 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:41:54.520 DEBUG:boto:Data: 2022-09-03T06:41:54.521 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.522 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:54.522 DEBUG:boto:Port: 8000 2022-09-03T06:41:54.523 DEBUG:boto:Params: {} 2022-09-03T06:41:54.524 DEBUG:boto:Token: None 2022-09-03T06:41:54.526 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/?versions 2022-09-03T06:41:54.528 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cLX/2ex4pT0peA3OPZInfmH1z0E= 2022-09-03T06:41:54.529 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cLX/2ex4pT0peA3OPZInfmH1z0E=', 'Content-Length': '0'} 2022-09-03T06:41:54.534 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c5b46f21fb09e11c-006312f732-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.537 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:54.538 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:41:54.539 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:54.539 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:54.540 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.540 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.541 DEBUG:boto:Method: GET 2022-09-03T06:41:54.542 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.542 DEBUG:boto:Data: 2022-09-03T06:41:54.543 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.544 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:54.544 DEBUG:boto:Port: 8001 2022-09-03T06:41:54.545 DEBUG:boto:Params: {} 2022-09-03T06:41:54.547 DEBUG:boto:Token: None 2022-09-03T06:41:54.548 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.549 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hmwX8P2xDkcL7lGbSorZKob9Cpc= 2022-09-03T06:41:54.549 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hmwX8P2xDkcL7lGbSorZKob9Cpc=', 'Content-Length': '0'} 2022-09-03T06:41:54.552 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000083e288660b49f24e-006312f732-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.596 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:54.597 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:54.597 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.598 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.599 DEBUG:boto:Method: GET 2022-09-03T06:41:54.599 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.600 DEBUG:boto:Data: 2022-09-03T06:41:54.600 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.601 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:54.601 DEBUG:boto:Port: 8000 2022-09-03T06:41:54.602 DEBUG:boto:Params: {} 2022-09-03T06:41:54.603 DEBUG:boto:Token: None 2022-09-03T06:41:54.604 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.606 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hmwX8P2xDkcL7lGbSorZKob9Cpc= 2022-09-03T06:41:54.608 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hmwX8P2xDkcL7lGbSorZKob9Cpc=', 'Content-Length': '0'} 2022-09-03T06:41:54.610 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000beeb21f182593554-006312f732-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.611 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:54.612 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:54.612 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.613 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.613 DEBUG:boto:Method: HEAD 2022-09-03T06:41:54.614 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.615 DEBUG:boto:Data: 2022-09-03T06:41:54.615 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.616 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:54.617 DEBUG:boto:Port: 8001 2022-09-03T06:41:54.617 DEBUG:boto:Params: {} 2022-09-03T06:41:54.618 DEBUG:boto:Token: None 2022-09-03T06:41:54.619 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.619 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sUt6JfroPJBFbFr+JsEjN3i041E= 2022-09-03T06:41:54.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sUt6JfroPJBFbFr+JsEjN3i041E=', 'Content-Length': '0'} 2022-09-03T06:41:54.622 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008a73d442cb1bbc47-006312f732-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.624 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:54.625 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:54.626 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.627 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.628 DEBUG:boto:Method: HEAD 2022-09-03T06:41:54.638 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.639 DEBUG:boto:Data: 2022-09-03T06:41:54.640 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.641 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:54.643 DEBUG:boto:Port: 8000 2022-09-03T06:41:54.644 DEBUG:boto:Params: {} 2022-09-03T06:41:54.645 DEBUG:boto:Token: None 2022-09-03T06:41:54.646 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.648 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sUt6JfroPJBFbFr+JsEjN3i041E= 2022-09-03T06:41:54.649 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sUt6JfroPJBFbFr+JsEjN3i041E=', 'Content-Length': '0'} 2022-09-03T06:41:54.652 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000070a5720eb9321f7a-006312f732-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.653 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:41:54.654 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:54.656 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:54.657 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.658 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.659 DEBUG:boto:Method: GET 2022-09-03T06:41:54.660 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.661 DEBUG:boto:Data: 2022-09-03T06:41:54.661 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.662 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:54.663 DEBUG:boto:Port: 8001 2022-09-03T06:41:54.663 DEBUG:boto:Params: {} 2022-09-03T06:41:54.664 DEBUG:boto:Token: None 2022-09-03T06:41:54.664 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.665 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hmwX8P2xDkcL7lGbSorZKob9Cpc= 2022-09-03T06:41:54.666 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hmwX8P2xDkcL7lGbSorZKob9Cpc=', 'Content-Length': '0'} 2022-09-03T06:41:54.668 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000043c6dde45b091c05-006312f732-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.669 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:54.670 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:54.670 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.671 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.672 DEBUG:boto:Method: GET 2022-09-03T06:41:54.672 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.673 DEBUG:boto:Data: 2022-09-03T06:41:54.673 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.674 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:54.674 DEBUG:boto:Port: 8000 2022-09-03T06:41:54.675 DEBUG:boto:Params: {} 2022-09-03T06:41:54.676 DEBUG:boto:Token: None 2022-09-03T06:41:54.676 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:41:54.677 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hmwX8P2xDkcL7lGbSorZKob9Cpc= 2022-09-03T06:41:54.678 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hmwX8P2xDkcL7lGbSorZKob9Cpc=', 'Content-Length': '0'} 2022-09-03T06:41:54.680 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c26b441716dfdf5-006312f732-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.683 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:54.684 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:54.685 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:54.686 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:54.687 DEBUG:boto:Method: GET 2022-09-03T06:41:54.688 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:41:54.690 DEBUG:boto:Data: 2022-09-03T06:41:54.691 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.692 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:54.695 DEBUG:boto:Port: 8001 2022-09-03T06:41:54.696 DEBUG:boto:Params: {} 2022-09-03T06:41:54.697 DEBUG:boto:Token: None 2022-09-03T06:41:54.698 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/ 2022-09-03T06:41:54.699 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l+Up7SYRfnPdSuWeJBp+wW+jezY= 2022-09-03T06:41:54.699 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l+Up7SYRfnPdSuWeJBp+wW+jezY=', 'Content-Length': '0'} 2022-09-03T06:41:54.704 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008bc96667f7a12cfd-006312f732-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.705 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:54.713 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:54.715 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:54.716 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:54.717 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:54.718 DEBUG:boto:Method: GET 2022-09-03T06:41:54.719 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:41:54.721 DEBUG:boto:Data: 2022-09-03T06:41:54.723 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.724 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:54.726 DEBUG:boto:Port: 8000 2022-09-03T06:41:54.728 DEBUG:boto:Params: {} 2022-09-03T06:41:54.730 DEBUG:boto:Token: None 2022-09-03T06:41:54.732 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/ 2022-09-03T06:41:54.733 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l+Up7SYRfnPdSuWeJBp+wW+jezY= 2022-09-03T06:41:54.736 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l+Up7SYRfnPdSuWeJBp+wW+jezY=', 'Content-Length': '0'} 2022-09-03T06:41:54.740 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a669c23c38f9913a-006312f732-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.741 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:54.742 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:54.742 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:54.743 DEBUG:boto:Method: HEAD 2022-09-03T06:41:54.743 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:41:54.744 DEBUG:boto:Data: 2022-09-03T06:41:54.745 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.745 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:54.746 DEBUG:boto:Port: 8001 2022-09-03T06:41:54.746 DEBUG:boto:Params: {} 2022-09-03T06:41:54.747 DEBUG:boto:Token: None 2022-09-03T06:41:54.747 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj11 2022-09-03T06:41:54.748 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rn89KTDHET96o8bBtw/+QN2hVik= 2022-09-03T06:41:54.749 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rn89KTDHET96o8bBtw/+QN2hVik=', 'Content-Length': '0'} 2022-09-03T06:41:54.751 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c4c6b3930f577bd2-006312f732-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.752 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:54.752 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:54.753 DEBUG:boto:Method: HEAD 2022-09-03T06:41:54.753 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:41:54.754 DEBUG:boto:Data: 2022-09-03T06:41:54.754 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.755 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:54.755 DEBUG:boto:Port: 8000 2022-09-03T06:41:54.756 DEBUG:boto:Params: {} 2022-09-03T06:41:54.756 DEBUG:boto:Token: None 2022-09-03T06:41:54.757 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj11 2022-09-03T06:41:54.757 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rn89KTDHET96o8bBtw/+QN2hVik= 2022-09-03T06:41:54.758 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rn89KTDHET96o8bBtw/+QN2hVik=', 'Content-Length': '0'} 2022-09-03T06:41:54.769 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006185e8a0b6c10c2b-006312f732-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.772 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:41:54.774 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:54.776 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:54.777 DEBUG:boto:Method: GET 2022-09-03T06:41:54.778 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:41:54.779 DEBUG:boto:Data: 2022-09-03T06:41:54.780 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.780 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:54.781 DEBUG:boto:Port: 8001 2022-09-03T06:41:54.784 DEBUG:boto:Params: {} 2022-09-03T06:41:54.785 DEBUG:boto:Token: None 2022-09-03T06:41:54.793 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj11 2022-09-03T06:41:54.794 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HBQQ1qtdoZqqAZkHBJoXMxkjIIQ= 2022-09-03T06:41:54.795 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HBQQ1qtdoZqqAZkHBJoXMxkjIIQ=', 'Content-Length': '0'} 2022-09-03T06:41:54.804 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000035cc8c2cb8dc467d-006312f732-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.816 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:41:54.817 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:41:54.819 DEBUG:boto:Method: GET 2022-09-03T06:41:54.820 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:41:54.822 DEBUG:boto:Data: 2022-09-03T06:41:54.823 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.824 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:54.825 DEBUG:boto:Port: 8000 2022-09-03T06:41:54.826 DEBUG:boto:Params: {} 2022-09-03T06:41:54.827 DEBUG:boto:Token: None 2022-09-03T06:41:54.828 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj11 2022-09-03T06:41:54.829 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HBQQ1qtdoZqqAZkHBJoXMxkjIIQ= 2022-09-03T06:41:54.831 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HBQQ1qtdoZqqAZkHBJoXMxkjIIQ=', 'Content-Length': '0'} 2022-09-03T06:41:54.835 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000462e852b97990fa8-006312f732-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.837 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:54.838 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:54.839 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:54.840 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:54.841 DEBUG:boto:Method: GET 2022-09-03T06:41:54.842 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:41:54.843 DEBUG:boto:Data: 2022-09-03T06:41:54.845 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.846 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:54.846 DEBUG:boto:Port: 8001 2022-09-03T06:41:54.847 DEBUG:boto:Params: {} 2022-09-03T06:41:54.848 DEBUG:boto:Token: None 2022-09-03T06:41:54.848 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/ 2022-09-03T06:41:54.849 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l+Up7SYRfnPdSuWeJBp+wW+jezY= 2022-09-03T06:41:54.850 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l+Up7SYRfnPdSuWeJBp+wW+jezY=', 'Content-Length': '0'} 2022-09-03T06:41:54.854 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b65d620b802d0212-006312f732-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.854 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:54.855 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:54.856 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:54.856 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:54.857 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:41:54.858 DEBUG:boto:Method: GET 2022-09-03T06:41:54.858 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:41:54.859 DEBUG:boto:Data: 2022-09-03T06:41:54.860 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.860 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:54.861 DEBUG:boto:Port: 8000 2022-09-03T06:41:54.861 DEBUG:boto:Params: {} 2022-09-03T06:41:54.862 DEBUG:boto:Token: None 2022-09-03T06:41:54.863 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/ 2022-09-03T06:41:54.863 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l+Up7SYRfnPdSuWeJBp+wW+jezY= 2022-09-03T06:41:54.864 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l+Up7SYRfnPdSuWeJBp+wW+jezY=', 'Content-Length': '0'} 2022-09-03T06:41:54.868 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000faa236a974e05cb7-006312f732-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.869 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:54.870 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:41:54.871 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:54.871 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:54.872 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.872 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.873 DEBUG:boto:Method: GET 2022-09-03T06:41:54.874 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.874 DEBUG:boto:Data: 2022-09-03T06:41:54.875 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.875 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:54.876 DEBUG:boto:Port: 8001 2022-09-03T06:41:54.876 DEBUG:boto:Params: {} 2022-09-03T06:41:54.877 DEBUG:boto:Token: None 2022-09-03T06:41:54.878 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.878 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3OVcInsod/a3mG56cNhrQCFU8zk= 2022-09-03T06:41:54.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3OVcInsod/a3mG56cNhrQCFU8zk=', 'Content-Length': '0'} 2022-09-03T06:41:54.881 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd7fe21bb7ae7fa9-006312f732-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.882 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:54.883 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:54.883 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.884 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.884 DEBUG:boto:Method: GET 2022-09-03T06:41:54.885 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.885 DEBUG:boto:Data: 2022-09-03T06:41:54.886 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.887 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:54.887 DEBUG:boto:Port: 8000 2022-09-03T06:41:54.888 DEBUG:boto:Params: {} 2022-09-03T06:41:54.890 DEBUG:boto:Token: None 2022-09-03T06:41:54.892 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.893 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3OVcInsod/a3mG56cNhrQCFU8zk= 2022-09-03T06:41:54.894 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3OVcInsod/a3mG56cNhrQCFU8zk=', 'Content-Length': '0'} 2022-09-03T06:41:54.897 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000016cbbde8d9137962-006312f732-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.899 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:54.899 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:54.900 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.901 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.902 DEBUG:boto:Method: HEAD 2022-09-03T06:41:54.903 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.904 DEBUG:boto:Data: 2022-09-03T06:41:54.905 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.905 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:54.906 DEBUG:boto:Port: 8001 2022-09-03T06:41:54.907 DEBUG:boto:Params: {} 2022-09-03T06:41:54.908 DEBUG:boto:Token: None 2022-09-03T06:41:54.909 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.910 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xs2ZhMKPQuhEHj9Q15zWFLQ+10M= 2022-09-03T06:41:54.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xs2ZhMKPQuhEHj9Q15zWFLQ+10M=', 'Content-Length': '0'} 2022-09-03T06:41:54.914 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005a307ba5614ad172-006312f732-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.915 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:54.915 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:54.916 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.916 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.917 DEBUG:boto:Method: HEAD 2022-09-03T06:41:54.917 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.918 DEBUG:boto:Data: 2022-09-03T06:41:54.919 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.919 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:54.920 DEBUG:boto:Port: 8000 2022-09-03T06:41:54.920 DEBUG:boto:Params: {} 2022-09-03T06:41:54.921 DEBUG:boto:Token: None 2022-09-03T06:41:54.921 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.922 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xs2ZhMKPQuhEHj9Q15zWFLQ+10M= 2022-09-03T06:41:54.923 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xs2ZhMKPQuhEHj9Q15zWFLQ+10M=', 'Content-Length': '0'} 2022-09-03T06:41:54.926 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000098e5ad25ba00e75c-006312f732-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.927 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:41:54.928 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:54.929 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:54.929 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.931 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.931 DEBUG:boto:Method: GET 2022-09-03T06:41:54.933 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.934 DEBUG:boto:Data: 2022-09-03T06:41:54.935 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.935 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:54.936 DEBUG:boto:Port: 8001 2022-09-03T06:41:54.937 DEBUG:boto:Params: {} 2022-09-03T06:41:54.938 DEBUG:boto:Token: None 2022-09-03T06:41:54.939 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.941 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3OVcInsod/a3mG56cNhrQCFU8zk= 2022-09-03T06:41:54.943 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3OVcInsod/a3mG56cNhrQCFU8zk=', 'Content-Length': '0'} 2022-09-03T06:41:54.947 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007e5c7cf05aacf8e0-006312f732-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.949 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:54.950 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:54.951 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.952 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.953 DEBUG:boto:Method: GET 2022-09-03T06:41:54.954 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.955 DEBUG:boto:Data: 2022-09-03T06:41:54.956 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.957 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:54.958 DEBUG:boto:Port: 8000 2022-09-03T06:41:54.958 DEBUG:boto:Params: {} 2022-09-03T06:41:54.960 DEBUG:boto:Token: None 2022-09-03T06:41:54.961 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:41:54.962 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3OVcInsod/a3mG56cNhrQCFU8zk= 2022-09-03T06:41:54.963 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3OVcInsod/a3mG56cNhrQCFU8zk=', 'Content-Length': '0'} 2022-09-03T06:41:54.966 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008aab6f36cf796308-006312f732-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.967 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:54.968 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:54.969 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:54.970 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:54.971 DEBUG:boto:Method: GET 2022-09-03T06:41:54.972 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:41:54.973 DEBUG:boto:Data: 2022-09-03T06:41:54.974 DEBUG:boto:Headers: {} 2022-09-03T06:41:54.975 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:54.976 DEBUG:boto:Port: 8001 2022-09-03T06:41:54.986 DEBUG:boto:Params: {} 2022-09-03T06:41:54.987 DEBUG:boto:Token: None 2022-09-03T06:41:54.988 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:54 GMT /dsgvcy-75/ 2022-09-03T06:41:54.989 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l+Up7SYRfnPdSuWeJBp+wW+jezY= 2022-09-03T06:41:54.991 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:54 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l+Up7SYRfnPdSuWeJBp+wW+jezY=', 'Content-Length': '0'} 2022-09-03T06:41:54.995 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000082f1f5c12ab51ea5-006312f732-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:54 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:54.996 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:54.998 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:54.999 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:55.000 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:55.001 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:55.012 DEBUG:boto:Method: GET 2022-09-03T06:41:55.013 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:41:55.014 DEBUG:boto:Data: 2022-09-03T06:41:55.015 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.016 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:55.017 DEBUG:boto:Port: 8000 2022-09-03T06:41:55.018 DEBUG:boto:Params: {} 2022-09-03T06:41:55.019 DEBUG:boto:Token: None 2022-09-03T06:41:55.019 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/ 2022-09-03T06:41:55.020 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CS5Lv5znWlAFI9gP/qsRGfmAjGA= 2022-09-03T06:41:55.021 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CS5Lv5znWlAFI9gP/qsRGfmAjGA=', 'Content-Length': '0'} 2022-09-03T06:41:55.029 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000af522c0bba4d5ed8-006312f733-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.031 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:55.032 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:55.033 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:55.033 DEBUG:boto:Method: HEAD 2022-09-03T06:41:55.034 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:41:55.035 DEBUG:boto:Data: 2022-09-03T06:41:55.035 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.036 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:55.037 DEBUG:boto:Port: 8001 2022-09-03T06:41:55.037 DEBUG:boto:Params: {} 2022-09-03T06:41:55.038 DEBUG:boto:Token: None 2022-09-03T06:41:55.039 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj17 2022-09-03T06:41:55.039 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cnpEhjyR9prte81Ejn0w8kO47I0= 2022-09-03T06:41:55.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cnpEhjyR9prte81Ejn0w8kO47I0=', 'Content-Length': '0'} 2022-09-03T06:41:55.047 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003ac87af194dddde6-006312f733-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.049 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:55.050 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:55.051 DEBUG:boto:Method: HEAD 2022-09-03T06:41:55.052 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:41:55.053 DEBUG:boto:Data: 2022-09-03T06:41:55.054 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.055 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:55.056 DEBUG:boto:Port: 8000 2022-09-03T06:41:55.058 DEBUG:boto:Params: {} 2022-09-03T06:41:55.058 DEBUG:boto:Token: None 2022-09-03T06:41:55.059 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj17 2022-09-03T06:41:55.060 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cnpEhjyR9prte81Ejn0w8kO47I0= 2022-09-03T06:41:55.061 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cnpEhjyR9prte81Ejn0w8kO47I0=', 'Content-Length': '0'} 2022-09-03T06:41:55.068 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a835058bc12cb68c-006312f733-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.070 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:41:55.071 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:55.072 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:55.072 DEBUG:boto:Method: GET 2022-09-03T06:41:55.073 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:41:55.083 DEBUG:boto:Data: 2022-09-03T06:41:55.084 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.085 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:55.086 DEBUG:boto:Port: 8001 2022-09-03T06:41:55.087 DEBUG:boto:Params: {} 2022-09-03T06:41:55.088 DEBUG:boto:Token: None 2022-09-03T06:41:55.089 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj17 2022-09-03T06:41:55.090 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F+9MSxled3Tg0Z4hZnW/hLm/tM4= 2022-09-03T06:41:55.091 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F+9MSxled3Tg0Z4hZnW/hLm/tM4=', 'Content-Length': '0'} 2022-09-03T06:41:55.094 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ed2319ac024c3518-006312f733-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.095 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:41:55.096 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:41:55.097 DEBUG:boto:Method: GET 2022-09-03T06:41:55.098 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:41:55.099 DEBUG:boto:Data: 2022-09-03T06:41:55.100 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.101 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:55.102 DEBUG:boto:Port: 8000 2022-09-03T06:41:55.103 DEBUG:boto:Params: {} 2022-09-03T06:41:55.104 DEBUG:boto:Token: None 2022-09-03T06:41:55.106 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj17 2022-09-03T06:41:55.107 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F+9MSxled3Tg0Z4hZnW/hLm/tM4= 2022-09-03T06:41:55.108 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F+9MSxled3Tg0Z4hZnW/hLm/tM4=', 'Content-Length': '0'} 2022-09-03T06:41:55.111 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e4a4d5947108faf-006312f733-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.113 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:55.114 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:55.115 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:55.116 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:55.117 DEBUG:boto:Method: GET 2022-09-03T06:41:55.118 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:41:55.119 DEBUG:boto:Data: 2022-09-03T06:41:55.119 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.120 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:55.120 DEBUG:boto:Port: 8001 2022-09-03T06:41:55.121 DEBUG:boto:Params: {} 2022-09-03T06:41:55.122 DEBUG:boto:Token: None 2022-09-03T06:41:55.123 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/ 2022-09-03T06:41:55.124 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CS5Lv5znWlAFI9gP/qsRGfmAjGA= 2022-09-03T06:41:55.126 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CS5Lv5znWlAFI9gP/qsRGfmAjGA=', 'Content-Length': '0'} 2022-09-03T06:41:55.129 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000080dd1ac5e49f1d4f-006312f733-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.130 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:55.132 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:55.133 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:55.134 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:55.135 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:41:55.136 DEBUG:boto:Method: GET 2022-09-03T06:41:55.137 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:41:55.138 DEBUG:boto:Data: 2022-09-03T06:41:55.138 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.139 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:55.140 DEBUG:boto:Port: 8000 2022-09-03T06:41:55.141 DEBUG:boto:Params: {} 2022-09-03T06:41:55.142 DEBUG:boto:Token: None 2022-09-03T06:41:55.143 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/ 2022-09-03T06:41:55.144 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CS5Lv5znWlAFI9gP/qsRGfmAjGA= 2022-09-03T06:41:55.145 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CS5Lv5znWlAFI9gP/qsRGfmAjGA=', 'Content-Length': '0'} 2022-09-03T06:41:55.149 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fb4c3104a11f8e59-006312f733-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.151 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:55.152 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:41:55.153 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:55.161 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:55.162 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.163 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.164 DEBUG:boto:Method: GET 2022-09-03T06:41:55.167 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.168 DEBUG:boto:Data: 2022-09-03T06:41:55.169 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.169 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:55.170 DEBUG:boto:Port: 8001 2022-09-03T06:41:55.171 DEBUG:boto:Params: {} 2022-09-03T06:41:55.172 DEBUG:boto:Token: None 2022-09-03T06:41:55.173 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.175 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:roki1pscvg74FjrOFoliMvnAwms= 2022-09-03T06:41:55.176 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:roki1pscvg74FjrOFoliMvnAwms=', 'Content-Length': '0'} 2022-09-03T06:41:55.183 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000922631c7ca861d44-006312f733-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.185 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:55.186 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:55.187 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.188 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.190 DEBUG:boto:Method: GET 2022-09-03T06:41:55.191 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.191 DEBUG:boto:Data: 2022-09-03T06:41:55.192 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.193 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:55.194 DEBUG:boto:Port: 8000 2022-09-03T06:41:55.194 DEBUG:boto:Params: {} 2022-09-03T06:41:55.195 DEBUG:boto:Token: None 2022-09-03T06:41:55.196 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.197 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:roki1pscvg74FjrOFoliMvnAwms= 2022-09-03T06:41:55.198 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:roki1pscvg74FjrOFoliMvnAwms=', 'Content-Length': '0'} 2022-09-03T06:41:55.205 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a807c1d80a7249fb-006312f733-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.209 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:55.210 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:55.212 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.213 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.214 DEBUG:boto:Method: HEAD 2022-09-03T06:41:55.215 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.217 DEBUG:boto:Data: 2022-09-03T06:41:55.218 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.219 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:55.220 DEBUG:boto:Port: 8001 2022-09-03T06:41:55.222 DEBUG:boto:Params: {} 2022-09-03T06:41:55.223 DEBUG:boto:Token: None 2022-09-03T06:41:55.225 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.226 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1Tg6c+wvbEqDhBB4yM7q4B7T5iE= 2022-09-03T06:41:55.227 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1Tg6c+wvbEqDhBB4yM7q4B7T5iE=', 'Content-Length': '0'} 2022-09-03T06:41:55.231 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008f626fe7d1ba6fae-006312f733-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.233 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:55.234 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:55.234 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.235 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.236 DEBUG:boto:Method: HEAD 2022-09-03T06:41:55.236 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.237 DEBUG:boto:Data: 2022-09-03T06:41:55.238 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.238 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:55.239 DEBUG:boto:Port: 8000 2022-09-03T06:41:55.239 DEBUG:boto:Params: {} 2022-09-03T06:41:55.240 DEBUG:boto:Token: None 2022-09-03T06:41:55.240 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.241 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1Tg6c+wvbEqDhBB4yM7q4B7T5iE= 2022-09-03T06:41:55.242 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1Tg6c+wvbEqDhBB4yM7q4B7T5iE=', 'Content-Length': '0'} 2022-09-03T06:41:55.245 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000909826b7a7d007e-006312f733-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.246 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:41:55.247 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:55.248 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:55.257 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.258 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.258 DEBUG:boto:Method: GET 2022-09-03T06:41:55.259 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.259 DEBUG:boto:Data: 2022-09-03T06:41:55.260 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.260 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:55.261 DEBUG:boto:Port: 8001 2022-09-03T06:41:55.262 DEBUG:boto:Params: {} 2022-09-03T06:41:55.262 DEBUG:boto:Token: None 2022-09-03T06:41:55.263 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.264 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:roki1pscvg74FjrOFoliMvnAwms= 2022-09-03T06:41:55.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:roki1pscvg74FjrOFoliMvnAwms=', 'Content-Length': '0'} 2022-09-03T06:41:55.270 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003ffcbad2d2a8beec-006312f733-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.272 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:55.273 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:55.274 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.275 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.275 DEBUG:boto:Method: GET 2022-09-03T06:41:55.276 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.277 DEBUG:boto:Data: 2022-09-03T06:41:55.277 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.278 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:55.279 DEBUG:boto:Port: 8000 2022-09-03T06:41:55.279 DEBUG:boto:Params: {} 2022-09-03T06:41:55.281 DEBUG:boto:Token: None 2022-09-03T06:41:55.282 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:41:55.283 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:roki1pscvg74FjrOFoliMvnAwms= 2022-09-03T06:41:55.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:roki1pscvg74FjrOFoliMvnAwms=', 'Content-Length': '0'} 2022-09-03T06:41:55.287 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000236a4cd7220ce2c5-006312f733-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.289 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:55.290 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:55.291 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:55.292 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:55.293 DEBUG:boto:Method: GET 2022-09-03T06:41:55.295 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:41:55.295 DEBUG:boto:Data: 2022-09-03T06:41:55.297 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.298 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:55.299 DEBUG:boto:Port: 8001 2022-09-03T06:41:55.300 DEBUG:boto:Params: {} 2022-09-03T06:41:55.301 DEBUG:boto:Token: None 2022-09-03T06:41:55.302 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/ 2022-09-03T06:41:55.303 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CS5Lv5znWlAFI9gP/qsRGfmAjGA= 2022-09-03T06:41:55.305 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CS5Lv5znWlAFI9gP/qsRGfmAjGA=', 'Content-Length': '0'} 2022-09-03T06:41:55.308 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d5e93e85306dceff-006312f733-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.309 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:55.310 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:55.311 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:55.312 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:55.313 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:55.314 DEBUG:boto:Method: GET 2022-09-03T06:41:55.315 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:41:55.315 DEBUG:boto:Data: 2022-09-03T06:41:55.316 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.317 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:55.318 DEBUG:boto:Port: 8000 2022-09-03T06:41:55.319 DEBUG:boto:Params: {} 2022-09-03T06:41:55.320 DEBUG:boto:Token: None 2022-09-03T06:41:55.322 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/ 2022-09-03T06:41:55.323 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CS5Lv5znWlAFI9gP/qsRGfmAjGA= 2022-09-03T06:41:55.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CS5Lv5znWlAFI9gP/qsRGfmAjGA=', 'Content-Length': '0'} 2022-09-03T06:41:55.327 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e5ea16513f19a461-006312f733-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.375 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:55.377 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:55.378 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:55.378 DEBUG:boto:Method: HEAD 2022-09-03T06:41:55.379 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:41:55.379 DEBUG:boto:Data: 2022-09-03T06:41:55.380 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.380 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:55.381 DEBUG:boto:Port: 8001 2022-09-03T06:41:55.382 DEBUG:boto:Params: {} 2022-09-03T06:41:55.382 DEBUG:boto:Token: None 2022-09-03T06:41:55.383 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj19 2022-09-03T06:41:55.384 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I0d2QFI8k17VHTdUQdYuEvC1+5Y= 2022-09-03T06:41:55.384 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I0d2QFI8k17VHTdUQdYuEvC1+5Y=', 'Content-Length': '0'} 2022-09-03T06:41:55.386 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000025cf4b0e8154c7c5-006312f733-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.387 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:55.388 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:55.388 DEBUG:boto:Method: HEAD 2022-09-03T06:41:55.389 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:41:55.389 DEBUG:boto:Data: 2022-09-03T06:41:55.390 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.390 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:55.391 DEBUG:boto:Port: 8000 2022-09-03T06:41:55.391 DEBUG:boto:Params: {} 2022-09-03T06:41:55.392 DEBUG:boto:Token: None 2022-09-03T06:41:55.393 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj19 2022-09-03T06:41:55.393 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I0d2QFI8k17VHTdUQdYuEvC1+5Y= 2022-09-03T06:41:55.394 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I0d2QFI8k17VHTdUQdYuEvC1+5Y=', 'Content-Length': '0'} 2022-09-03T06:41:55.397 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005398883f37d13d33-006312f733-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.399 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:41:55.400 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:55.401 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:55.402 DEBUG:boto:Method: GET 2022-09-03T06:41:55.403 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:41:55.404 DEBUG:boto:Data: 2022-09-03T06:41:55.405 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.406 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:55.407 DEBUG:boto:Port: 8001 2022-09-03T06:41:55.409 DEBUG:boto:Params: {} 2022-09-03T06:41:55.410 DEBUG:boto:Token: None 2022-09-03T06:41:55.411 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj19 2022-09-03T06:41:55.412 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kx6bZIuZYoZZA5WO1ykBZ2GpMyw= 2022-09-03T06:41:55.413 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kx6bZIuZYoZZA5WO1ykBZ2GpMyw=', 'Content-Length': '0'} 2022-09-03T06:41:55.416 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003319062693623256-006312f733-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.458 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:41:55.459 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:41:55.460 DEBUG:boto:Method: GET 2022-09-03T06:41:55.461 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:41:55.462 DEBUG:boto:Data: 2022-09-03T06:41:55.463 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.464 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:55.465 DEBUG:boto:Port: 8000 2022-09-03T06:41:55.466 DEBUG:boto:Params: {} 2022-09-03T06:41:55.467 DEBUG:boto:Token: None 2022-09-03T06:41:55.468 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj19 2022-09-03T06:41:55.469 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kx6bZIuZYoZZA5WO1ykBZ2GpMyw= 2022-09-03T06:41:55.470 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kx6bZIuZYoZZA5WO1ykBZ2GpMyw=', 'Content-Length': '0'} 2022-09-03T06:41:55.473 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a29046a151db8179-006312f733-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.475 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:55.476 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:55.477 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:55.478 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:55.479 DEBUG:boto:Method: GET 2022-09-03T06:41:55.480 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:41:55.481 DEBUG:boto:Data: 2022-09-03T06:41:55.482 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.483 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:55.484 DEBUG:boto:Port: 8001 2022-09-03T06:41:55.485 DEBUG:boto:Params: {} 2022-09-03T06:41:55.486 DEBUG:boto:Token: None 2022-09-03T06:41:55.487 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/ 2022-09-03T06:41:55.488 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CS5Lv5znWlAFI9gP/qsRGfmAjGA= 2022-09-03T06:41:55.489 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CS5Lv5znWlAFI9gP/qsRGfmAjGA=', 'Content-Length': '0'} 2022-09-03T06:41:55.493 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cb91491db06c9c52-006312f733-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.494 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:55.496 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:55.497 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:55.498 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:55.499 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:41:55.500 DEBUG:boto:Method: GET 2022-09-03T06:41:55.501 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:41:55.502 DEBUG:boto:Data: 2022-09-03T06:41:55.503 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.504 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:55.505 DEBUG:boto:Port: 8000 2022-09-03T06:41:55.506 DEBUG:boto:Params: {} 2022-09-03T06:41:55.507 DEBUG:boto:Token: None 2022-09-03T06:41:55.509 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/ 2022-09-03T06:41:55.510 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CS5Lv5znWlAFI9gP/qsRGfmAjGA= 2022-09-03T06:41:55.511 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CS5Lv5znWlAFI9gP/qsRGfmAjGA=', 'Content-Length': '0'} 2022-09-03T06:41:55.515 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000002ae717b8a7f4499-006312f733-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.516 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:55.517 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:41:55.518 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:55.519 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:55.520 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.520 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.521 DEBUG:boto:Method: GET 2022-09-03T06:41:55.522 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.523 DEBUG:boto:Data: 2022-09-03T06:41:55.524 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.525 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:55.526 DEBUG:boto:Port: 8001 2022-09-03T06:41:55.527 DEBUG:boto:Params: {} 2022-09-03T06:41:55.528 DEBUG:boto:Token: None 2022-09-03T06:41:55.529 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.530 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f8amEzfSNUD6V3CYQAxorrl7BIM= 2022-09-03T06:41:55.531 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f8amEzfSNUD6V3CYQAxorrl7BIM=', 'Content-Length': '0'} 2022-09-03T06:41:55.534 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000903939aa790f4a43-006312f733-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.536 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:55.536 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:55.537 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.537 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.538 DEBUG:boto:Method: GET 2022-09-03T06:41:55.539 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.539 DEBUG:boto:Data: 2022-09-03T06:41:55.540 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.541 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:55.542 DEBUG:boto:Port: 8000 2022-09-03T06:41:55.543 DEBUG:boto:Params: {} 2022-09-03T06:41:55.544 DEBUG:boto:Token: None 2022-09-03T06:41:55.545 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.546 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f8amEzfSNUD6V3CYQAxorrl7BIM= 2022-09-03T06:41:55.547 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f8amEzfSNUD6V3CYQAxorrl7BIM=', 'Content-Length': '0'} 2022-09-03T06:41:55.550 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000044de2c2b98d4544f-006312f733-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.552 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:55.552 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:55.554 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.555 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.556 DEBUG:boto:Method: HEAD 2022-09-03T06:41:55.557 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.557 DEBUG:boto:Data: 2022-09-03T06:41:55.558 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.559 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:55.560 DEBUG:boto:Port: 8001 2022-09-03T06:41:55.561 DEBUG:boto:Params: {} 2022-09-03T06:41:55.562 DEBUG:boto:Token: None 2022-09-03T06:41:55.563 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.564 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2p5HpixVv2L7h3nm2Bbo/vbWljw= 2022-09-03T06:41:55.565 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2p5HpixVv2L7h3nm2Bbo/vbWljw=', 'Content-Length': '0'} 2022-09-03T06:41:55.568 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000078216c6dd4276543-006312f733-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.569 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:55.571 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:55.577 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.577 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.578 DEBUG:boto:Method: HEAD 2022-09-03T06:41:55.578 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.579 DEBUG:boto:Data: 2022-09-03T06:41:55.580 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.580 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:55.581 DEBUG:boto:Port: 8000 2022-09-03T06:41:55.582 DEBUG:boto:Params: {} 2022-09-03T06:41:55.582 DEBUG:boto:Token: None 2022-09-03T06:41:55.583 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.584 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2p5HpixVv2L7h3nm2Bbo/vbWljw= 2022-09-03T06:41:55.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2p5HpixVv2L7h3nm2Bbo/vbWljw=', 'Content-Length': '0'} 2022-09-03T06:41:55.587 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000764ccb8dfe5e42a2-006312f733-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.587 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:41:55.588 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:55.589 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:55.589 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.590 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.591 DEBUG:boto:Method: GET 2022-09-03T06:41:55.591 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.592 DEBUG:boto:Data: 2022-09-03T06:41:55.592 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.593 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:55.593 DEBUG:boto:Port: 8001 2022-09-03T06:41:55.595 DEBUG:boto:Params: {} 2022-09-03T06:41:55.596 DEBUG:boto:Token: None 2022-09-03T06:41:55.597 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.598 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f8amEzfSNUD6V3CYQAxorrl7BIM= 2022-09-03T06:41:55.598 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f8amEzfSNUD6V3CYQAxorrl7BIM=', 'Content-Length': '0'} 2022-09-03T06:41:55.601 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a8db88d0267ab684-006312f733-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.602 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:55.602 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:55.603 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.603 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.604 DEBUG:boto:Method: GET 2022-09-03T06:41:55.604 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.605 DEBUG:boto:Data: 2022-09-03T06:41:55.605 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.606 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:55.608 DEBUG:boto:Port: 8000 2022-09-03T06:41:55.609 DEBUG:boto:Params: {} 2022-09-03T06:41:55.610 DEBUG:boto:Token: None 2022-09-03T06:41:55.610 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:41:55.611 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f8amEzfSNUD6V3CYQAxorrl7BIM= 2022-09-03T06:41:55.612 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f8amEzfSNUD6V3CYQAxorrl7BIM=', 'Content-Length': '0'} 2022-09-03T06:41:55.614 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d16544ee598f5fb7-006312f733-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.654 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:55.656 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:55.657 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:55.658 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:55.659 DEBUG:boto:Method: GET 2022-09-03T06:41:55.660 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:41:55.661 DEBUG:boto:Data: 2022-09-03T06:41:55.662 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.663 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:55.664 DEBUG:boto:Port: 8001 2022-09-03T06:41:55.665 DEBUG:boto:Params: {} 2022-09-03T06:41:55.667 DEBUG:boto:Token: None 2022-09-03T06:41:55.668 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/ 2022-09-03T06:41:55.669 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CS5Lv5znWlAFI9gP/qsRGfmAjGA= 2022-09-03T06:41:55.670 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CS5Lv5znWlAFI9gP/qsRGfmAjGA=', 'Content-Length': '0'} 2022-09-03T06:41:55.674 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008229c67700a49474-006312f733-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.675 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:55.676 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:55.677 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:55.678 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:55.679 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:55.680 DEBUG:boto:Method: GET 2022-09-03T06:41:55.681 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:41:55.682 DEBUG:boto:Data: 2022-09-03T06:41:55.683 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.684 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:55.685 DEBUG:boto:Port: 8000 2022-09-03T06:41:55.687 DEBUG:boto:Params: {} 2022-09-03T06:41:55.688 DEBUG:boto:Token: None 2022-09-03T06:41:55.689 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/ 2022-09-03T06:41:55.690 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CS5Lv5znWlAFI9gP/qsRGfmAjGA= 2022-09-03T06:41:55.691 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CS5Lv5znWlAFI9gP/qsRGfmAjGA=', 'Content-Length': '0'} 2022-09-03T06:41:55.695 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005cccb81615621fca-006312f733-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.696 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:55.697 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:55.698 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:55.699 DEBUG:boto:Method: HEAD 2022-09-03T06:41:55.700 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:41:55.701 DEBUG:boto:Data: 2022-09-03T06:41:55.702 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.703 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:55.704 DEBUG:boto:Port: 8001 2022-09-03T06:41:55.705 DEBUG:boto:Params: {} 2022-09-03T06:41:55.706 DEBUG:boto:Token: None 2022-09-03T06:41:55.707 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj22 2022-09-03T06:41:55.708 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xsiLNvGCEDWITGuzkA8v4ghTlWc= 2022-09-03T06:41:55.709 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xsiLNvGCEDWITGuzkA8v4ghTlWc=', 'Content-Length': '0'} 2022-09-03T06:41:55.715 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000005935dd4059ce2a6-006312f733-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.716 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:55.717 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:55.717 DEBUG:boto:Method: HEAD 2022-09-03T06:41:55.718 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:41:55.718 DEBUG:boto:Data: 2022-09-03T06:41:55.719 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.719 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:55.720 DEBUG:boto:Port: 8000 2022-09-03T06:41:55.720 DEBUG:boto:Params: {} 2022-09-03T06:41:55.721 DEBUG:boto:Token: None 2022-09-03T06:41:55.721 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj22 2022-09-03T06:41:55.722 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xsiLNvGCEDWITGuzkA8v4ghTlWc= 2022-09-03T06:41:55.723 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xsiLNvGCEDWITGuzkA8v4ghTlWc=', 'Content-Length': '0'} 2022-09-03T06:41:55.726 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002e540ea6e7cc6417-006312f733-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.727 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:41:55.729 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:55.729 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:55.730 DEBUG:boto:Method: GET 2022-09-03T06:41:55.730 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:41:55.731 DEBUG:boto:Data: 2022-09-03T06:41:55.732 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.733 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:55.733 DEBUG:boto:Port: 8001 2022-09-03T06:41:55.734 DEBUG:boto:Params: {} 2022-09-03T06:41:55.735 DEBUG:boto:Token: None 2022-09-03T06:41:55.736 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj22 2022-09-03T06:41:55.737 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NKo9WexwkIdc7UkINOaPy/6X/bk= 2022-09-03T06:41:55.737 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NKo9WexwkIdc7UkINOaPy/6X/bk=', 'Content-Length': '0'} 2022-09-03T06:41:55.740 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c00ee827251e65c3-006312f733-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.783 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:41:55.784 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:41:55.785 DEBUG:boto:Method: GET 2022-09-03T06:41:55.785 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:41:55.786 DEBUG:boto:Data: 2022-09-03T06:41:55.787 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.787 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:55.788 DEBUG:boto:Port: 8000 2022-09-03T06:41:55.788 DEBUG:boto:Params: {} 2022-09-03T06:41:55.789 DEBUG:boto:Token: None 2022-09-03T06:41:55.790 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj22 2022-09-03T06:41:55.791 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NKo9WexwkIdc7UkINOaPy/6X/bk= 2022-09-03T06:41:55.791 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NKo9WexwkIdc7UkINOaPy/6X/bk=', 'Content-Length': '0'} 2022-09-03T06:41:55.794 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000bc00062a5b2d1ac-006312f733-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.796 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:55.797 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:55.798 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:55.799 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:55.800 DEBUG:boto:Method: GET 2022-09-03T06:41:55.800 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:41:55.801 DEBUG:boto:Data: 2022-09-03T06:41:55.802 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.803 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:55.804 DEBUG:boto:Port: 8001 2022-09-03T06:41:55.805 DEBUG:boto:Params: {} 2022-09-03T06:41:55.805 DEBUG:boto:Token: None 2022-09-03T06:41:55.806 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/ 2022-09-03T06:41:55.807 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CS5Lv5znWlAFI9gP/qsRGfmAjGA= 2022-09-03T06:41:55.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CS5Lv5znWlAFI9gP/qsRGfmAjGA=', 'Content-Length': '0'} 2022-09-03T06:41:55.812 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f7569c59510ece36-006312f733-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.814 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:55.816 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:55.817 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:55.818 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:55.819 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:41:55.820 DEBUG:boto:Method: GET 2022-09-03T06:41:55.821 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:41:55.822 DEBUG:boto:Data: 2022-09-03T06:41:55.825 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.826 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:55.827 DEBUG:boto:Port: 8000 2022-09-03T06:41:55.827 DEBUG:boto:Params: {} 2022-09-03T06:41:55.828 DEBUG:boto:Token: None 2022-09-03T06:41:55.829 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/ 2022-09-03T06:41:55.829 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CS5Lv5znWlAFI9gP/qsRGfmAjGA= 2022-09-03T06:41:55.831 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CS5Lv5znWlAFI9gP/qsRGfmAjGA=', 'Content-Length': '0'} 2022-09-03T06:41:55.842 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003da124f846809bd8-006312f733-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.879 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:55.886 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:41:55.892 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:55.893 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:55.894 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:55.896 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:55.897 DEBUG:boto:Method: GET 2022-09-03T06:41:55.898 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:55.899 DEBUG:boto:Data: 2022-09-03T06:41:55.901 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.903 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:55.905 DEBUG:boto:Port: 8001 2022-09-03T06:41:55.906 DEBUG:boto:Params: {} 2022-09-03T06:41:55.907 DEBUG:boto:Token: None 2022-09-03T06:41:55.909 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:55.910 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hTJ/qsQZ8RdQMeHczLFHi+unLow= 2022-09-03T06:41:55.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hTJ/qsQZ8RdQMeHczLFHi+unLow=', 'Content-Length': '0'} 2022-09-03T06:41:55.924 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003eae51bfd239a517-006312f733-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.924 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:55.925 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:55.926 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:55.926 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:55.927 DEBUG:boto:Method: GET 2022-09-03T06:41:55.927 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:55.928 DEBUG:boto:Data: 2022-09-03T06:41:55.928 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.929 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:55.929 DEBUG:boto:Port: 8000 2022-09-03T06:41:55.930 DEBUG:boto:Params: {} 2022-09-03T06:41:55.931 DEBUG:boto:Token: None 2022-09-03T06:41:55.934 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:55.935 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hTJ/qsQZ8RdQMeHczLFHi+unLow= 2022-09-03T06:41:55.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hTJ/qsQZ8RdQMeHczLFHi+unLow=', 'Content-Length': '0'} 2022-09-03T06:41:55.942 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000313dc981d9fa7b62-006312f733-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.943 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:55.944 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:55.945 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:55.946 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:55.947 DEBUG:boto:Method: HEAD 2022-09-03T06:41:55.947 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:55.948 DEBUG:boto:Data: 2022-09-03T06:41:55.948 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.949 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:55.949 DEBUG:boto:Port: 8001 2022-09-03T06:41:55.950 DEBUG:boto:Params: {} 2022-09-03T06:41:55.951 DEBUG:boto:Token: None 2022-09-03T06:41:55.952 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:55.953 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A7DqjPXtS5YHpKiwyu8Wm3cppdA= 2022-09-03T06:41:55.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A7DqjPXtS5YHpKiwyu8Wm3cppdA=', 'Content-Length': '0'} 2022-09-03T06:41:55.955 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b33177c39c986e01-006312f733-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.956 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:55.957 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:55.958 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:55.958 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:55.959 DEBUG:boto:Method: HEAD 2022-09-03T06:41:55.960 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:55.960 DEBUG:boto:Data: 2022-09-03T06:41:55.962 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.962 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:55.963 DEBUG:boto:Port: 8000 2022-09-03T06:41:55.963 DEBUG:boto:Params: {} 2022-09-03T06:41:55.964 DEBUG:boto:Token: None 2022-09-03T06:41:55.964 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:55.965 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A7DqjPXtS5YHpKiwyu8Wm3cppdA= 2022-09-03T06:41:55.966 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A7DqjPXtS5YHpKiwyu8Wm3cppdA=', 'Content-Length': '0'} 2022-09-03T06:41:55.970 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af5ba0cf13c929e5-006312f733-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.971 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:41:55.972 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:55.972 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:55.973 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:55.974 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:55.974 DEBUG:boto:Method: GET 2022-09-03T06:41:55.975 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:55.975 DEBUG:boto:Data: 2022-09-03T06:41:55.977 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.978 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:55.978 DEBUG:boto:Port: 8001 2022-09-03T06:41:55.979 DEBUG:boto:Params: {} 2022-09-03T06:41:55.980 DEBUG:boto:Token: None 2022-09-03T06:41:55.980 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:55 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:55.981 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hTJ/qsQZ8RdQMeHczLFHi+unLow= 2022-09-03T06:41:55.982 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:55 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hTJ/qsQZ8RdQMeHczLFHi+unLow=', 'Content-Length': '0'} 2022-09-03T06:41:55.984 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000089311f41cb33e5b2-006312f733-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:55 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:55.985 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:55.986 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:55.986 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:55.987 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:55.987 DEBUG:boto:Method: GET 2022-09-03T06:41:55.994 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:55.995 DEBUG:boto:Data: 2022-09-03T06:41:55.995 DEBUG:boto:Headers: {} 2022-09-03T06:41:55.996 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:55.997 DEBUG:boto:Port: 8000 2022-09-03T06:41:55.999 DEBUG:boto:Params: {} 2022-09-03T06:41:56.000 DEBUG:boto:Token: None 2022-09-03T06:41:56.001 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:41:56.002 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tGJ8Ee2I1fZn3EFtIyPcW2BYysU= 2022-09-03T06:41:56.002 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tGJ8Ee2I1fZn3EFtIyPcW2BYysU=', 'Content-Length': '0'} 2022-09-03T06:41:56.005 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000027742cfdda9b658a-006312f734-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.007 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:56.008 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:56.009 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:56.010 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:56.011 DEBUG:boto:Method: GET 2022-09-03T06:41:56.019 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:41:56.021 DEBUG:boto:Data: 2022-09-03T06:41:56.021 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.025 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:56.026 DEBUG:boto:Port: 8001 2022-09-03T06:41:56.027 DEBUG:boto:Params: {} 2022-09-03T06:41:56.028 DEBUG:boto:Token: None 2022-09-03T06:41:56.029 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/ 2022-09-03T06:41:56.030 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c= 2022-09-03T06:41:56.031 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c=', 'Content-Length': '0'} 2022-09-03T06:41:56.034 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002a0e26bce5ea9811-006312f734-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.035 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:56.036 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:56.037 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:56.038 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:56.038 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:56.039 DEBUG:boto:Method: GET 2022-09-03T06:41:56.039 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:41:56.040 DEBUG:boto:Data: 2022-09-03T06:41:56.040 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.041 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:56.042 DEBUG:boto:Port: 8000 2022-09-03T06:41:56.042 DEBUG:boto:Params: {} 2022-09-03T06:41:56.043 DEBUG:boto:Token: None 2022-09-03T06:41:56.044 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/ 2022-09-03T06:41:56.044 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c= 2022-09-03T06:41:56.045 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c=', 'Content-Length': '0'} 2022-09-03T06:41:56.049 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000009e5d1267675746e-006312f734-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.054 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:56.055 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:56.055 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:56.056 DEBUG:boto:Method: HEAD 2022-09-03T06:41:56.056 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:41:56.057 DEBUG:boto:Data: 2022-09-03T06:41:56.057 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.058 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:56.058 DEBUG:boto:Port: 8001 2022-09-03T06:41:56.059 DEBUG:boto:Params: {} 2022-09-03T06:41:56.060 DEBUG:boto:Token: None 2022-09-03T06:41:56.060 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj33 2022-09-03T06:41:56.061 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VQ7+iIgAFZKkmtsNikmHJ+D4g+c= 2022-09-03T06:41:56.066 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VQ7+iIgAFZKkmtsNikmHJ+D4g+c=', 'Content-Length': '0'} 2022-09-03T06:41:56.069 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000019bf7c8d6e16c576-006312f734-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.070 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:56.071 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:56.072 DEBUG:boto:Method: HEAD 2022-09-03T06:41:56.073 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:41:56.074 DEBUG:boto:Data: 2022-09-03T06:41:56.075 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.076 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:56.077 DEBUG:boto:Port: 8000 2022-09-03T06:41:56.078 DEBUG:boto:Params: {} 2022-09-03T06:41:56.079 DEBUG:boto:Token: None 2022-09-03T06:41:56.080 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj33 2022-09-03T06:41:56.081 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VQ7+iIgAFZKkmtsNikmHJ+D4g+c= 2022-09-03T06:41:56.082 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VQ7+iIgAFZKkmtsNikmHJ+D4g+c=', 'Content-Length': '0'} 2022-09-03T06:41:56.086 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000056ad4372b79ab7bf-006312f734-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.087 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:41:56.090 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:56.091 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:56.091 DEBUG:boto:Method: GET 2022-09-03T06:41:56.092 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:41:56.093 DEBUG:boto:Data: 2022-09-03T06:41:56.093 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.094 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:56.095 DEBUG:boto:Port: 8001 2022-09-03T06:41:56.096 DEBUG:boto:Params: {} 2022-09-03T06:41:56.097 DEBUG:boto:Token: None 2022-09-03T06:41:56.098 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj33 2022-09-03T06:41:56.099 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4p5qSCm8c8HLoEjwQe/W9Cvjm+Q= 2022-09-03T06:41:56.100 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4p5qSCm8c8HLoEjwQe/W9Cvjm+Q=', 'Content-Length': '0'} 2022-09-03T06:41:56.102 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007aa61a86f3988612-006312f734-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.103 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:41:56.104 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:41:56.104 DEBUG:boto:Method: GET 2022-09-03T06:41:56.105 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:41:56.105 DEBUG:boto:Data: 2022-09-03T06:41:56.106 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.106 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:56.107 DEBUG:boto:Port: 8000 2022-09-03T06:41:56.107 DEBUG:boto:Params: {} 2022-09-03T06:41:56.108 DEBUG:boto:Token: None 2022-09-03T06:41:56.109 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj33 2022-09-03T06:41:56.109 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4p5qSCm8c8HLoEjwQe/W9Cvjm+Q= 2022-09-03T06:41:56.110 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4p5qSCm8c8HLoEjwQe/W9Cvjm+Q=', 'Content-Length': '0'} 2022-09-03T06:41:56.112 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000076d4934d20d4bd63-006312f734-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.113 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:56.113 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:56.114 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:56.114 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:56.115 DEBUG:boto:Method: GET 2022-09-03T06:41:56.115 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:41:56.116 DEBUG:boto:Data: 2022-09-03T06:41:56.117 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.117 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:56.118 DEBUG:boto:Port: 8001 2022-09-03T06:41:56.118 DEBUG:boto:Params: {} 2022-09-03T06:41:56.119 DEBUG:boto:Token: None 2022-09-03T06:41:56.119 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/ 2022-09-03T06:41:56.120 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c= 2022-09-03T06:41:56.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c=', 'Content-Length': '0'} 2022-09-03T06:41:56.124 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000051d6479fce98eeca-006312f734-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.170 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:56.171 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:56.172 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:56.173 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:56.173 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:41:56.174 DEBUG:boto:Method: GET 2022-09-03T06:41:56.175 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:41:56.176 DEBUG:boto:Data: 2022-09-03T06:41:56.177 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.178 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:56.181 DEBUG:boto:Port: 8000 2022-09-03T06:41:56.181 DEBUG:boto:Params: {} 2022-09-03T06:41:56.182 DEBUG:boto:Token: None 2022-09-03T06:41:56.183 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/ 2022-09-03T06:41:56.183 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c= 2022-09-03T06:41:56.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c=', 'Content-Length': '0'} 2022-09-03T06:41:56.192 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000adcc5884ce6064e8-006312f734-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.193 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:56.195 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:41:56.196 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:56.197 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:56.198 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.199 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.200 DEBUG:boto:Method: GET 2022-09-03T06:41:56.201 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.202 DEBUG:boto:Data: 2022-09-03T06:41:56.202 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.203 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:56.204 DEBUG:boto:Port: 8001 2022-09-03T06:41:56.205 DEBUG:boto:Params: {} 2022-09-03T06:41:56.206 DEBUG:boto:Token: None 2022-09-03T06:41:56.207 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.208 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tyKTo8uzY1VHzOfwQT/1XQMYgHA= 2022-09-03T06:41:56.209 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tyKTo8uzY1VHzOfwQT/1XQMYgHA=', 'Content-Length': '0'} 2022-09-03T06:41:56.212 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eace84590d6f8fb6-006312f734-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.213 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:56.214 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:56.215 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.216 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.217 DEBUG:boto:Method: GET 2022-09-03T06:41:56.218 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.219 DEBUG:boto:Data: 2022-09-03T06:41:56.221 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.223 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:56.223 DEBUG:boto:Port: 8000 2022-09-03T06:41:56.224 DEBUG:boto:Params: {} 2022-09-03T06:41:56.225 DEBUG:boto:Token: None 2022-09-03T06:41:56.226 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.227 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tyKTo8uzY1VHzOfwQT/1XQMYgHA= 2022-09-03T06:41:56.228 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tyKTo8uzY1VHzOfwQT/1XQMYgHA=', 'Content-Length': '0'} 2022-09-03T06:41:56.232 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d445f8a3aa78f3fc-006312f734-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.234 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:56.235 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:56.236 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.237 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.240 DEBUG:boto:Method: HEAD 2022-09-03T06:41:56.242 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.244 DEBUG:boto:Data: 2022-09-03T06:41:56.244 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.246 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:56.247 DEBUG:boto:Port: 8001 2022-09-03T06:41:56.248 DEBUG:boto:Params: {} 2022-09-03T06:41:56.249 DEBUG:boto:Token: None 2022-09-03T06:41:56.250 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.251 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6hBWa/6iO7nVLAw24eg81xn31C4= 2022-09-03T06:41:56.253 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6hBWa/6iO7nVLAw24eg81xn31C4=', 'Content-Length': '0'} 2022-09-03T06:41:56.255 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000362c4037cda7f30d-006312f734-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.256 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:56.257 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:56.258 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.258 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.259 DEBUG:boto:Method: HEAD 2022-09-03T06:41:56.259 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.260 DEBUG:boto:Data: 2022-09-03T06:41:56.260 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.261 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:56.261 DEBUG:boto:Port: 8000 2022-09-03T06:41:56.262 DEBUG:boto:Params: {} 2022-09-03T06:41:56.263 DEBUG:boto:Token: None 2022-09-03T06:41:56.264 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.266 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6hBWa/6iO7nVLAw24eg81xn31C4= 2022-09-03T06:41:56.267 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6hBWa/6iO7nVLAw24eg81xn31C4=', 'Content-Length': '0'} 2022-09-03T06:41:56.269 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018a52248a0479f08-006312f734-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.270 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:41:56.271 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:56.273 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:56.273 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.274 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.274 DEBUG:boto:Method: GET 2022-09-03T06:41:56.275 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.275 DEBUG:boto:Data: 2022-09-03T06:41:56.276 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.277 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:56.278 DEBUG:boto:Port: 8001 2022-09-03T06:41:56.279 DEBUG:boto:Params: {} 2022-09-03T06:41:56.280 DEBUG:boto:Token: None 2022-09-03T06:41:56.281 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.282 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tyKTo8uzY1VHzOfwQT/1XQMYgHA= 2022-09-03T06:41:56.283 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tyKTo8uzY1VHzOfwQT/1XQMYgHA=', 'Content-Length': '0'} 2022-09-03T06:41:56.286 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002eb0249421a301f7-006312f734-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.287 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:56.287 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:56.288 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.288 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.289 DEBUG:boto:Method: GET 2022-09-03T06:41:56.289 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.290 DEBUG:boto:Data: 2022-09-03T06:41:56.291 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.291 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:56.292 DEBUG:boto:Port: 8000 2022-09-03T06:41:56.292 DEBUG:boto:Params: {} 2022-09-03T06:41:56.293 DEBUG:boto:Token: None 2022-09-03T06:41:56.294 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:41:56.295 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tyKTo8uzY1VHzOfwQT/1XQMYgHA= 2022-09-03T06:41:56.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tyKTo8uzY1VHzOfwQT/1XQMYgHA=', 'Content-Length': '0'} 2022-09-03T06:41:56.298 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000032a709ed9e5e1e67-006312f734-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.298 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:56.299 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:56.300 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:56.301 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:56.302 DEBUG:boto:Method: GET 2022-09-03T06:41:56.303 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:41:56.304 DEBUG:boto:Data: 2022-09-03T06:41:56.304 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.305 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:56.305 DEBUG:boto:Port: 8001 2022-09-03T06:41:56.306 DEBUG:boto:Params: {} 2022-09-03T06:41:56.306 DEBUG:boto:Token: None 2022-09-03T06:41:56.307 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/ 2022-09-03T06:41:56.308 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c= 2022-09-03T06:41:56.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c=', 'Content-Length': '0'} 2022-09-03T06:41:56.311 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000034659fe0a630d16f-006312f734-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.312 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:56.313 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:56.313 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:56.314 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:56.315 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:56.315 DEBUG:boto:Method: GET 2022-09-03T06:41:56.316 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:41:56.316 DEBUG:boto:Data: 2022-09-03T06:41:56.317 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.317 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:56.318 DEBUG:boto:Port: 8000 2022-09-03T06:41:56.318 DEBUG:boto:Params: {} 2022-09-03T06:41:56.319 DEBUG:boto:Token: None 2022-09-03T06:41:56.320 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/ 2022-09-03T06:41:56.321 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c= 2022-09-03T06:41:56.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c=', 'Content-Length': '0'} 2022-09-03T06:41:56.324 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c08f161fe1b1d99b-006312f734-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.325 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:56.326 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:56.326 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:56.327 DEBUG:boto:Method: HEAD 2022-09-03T06:41:56.327 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:41:56.328 DEBUG:boto:Data: 2022-09-03T06:41:56.328 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.329 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:56.330 DEBUG:boto:Port: 8001 2022-09-03T06:41:56.330 DEBUG:boto:Params: {} 2022-09-03T06:41:56.331 DEBUG:boto:Token: None 2022-09-03T06:41:56.332 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj34 2022-09-03T06:41:56.332 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B6+E8iYoXS05PrHEf5kuiW6JiQs= 2022-09-03T06:41:56.333 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B6+E8iYoXS05PrHEf5kuiW6JiQs=', 'Content-Length': '0'} 2022-09-03T06:41:56.335 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c4f0d584aae3e04c-006312f734-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.336 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:56.337 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:56.337 DEBUG:boto:Method: HEAD 2022-09-03T06:41:56.338 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:41:56.339 DEBUG:boto:Data: 2022-09-03T06:41:56.339 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.340 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:56.341 DEBUG:boto:Port: 8000 2022-09-03T06:41:56.341 DEBUG:boto:Params: {} 2022-09-03T06:41:56.342 DEBUG:boto:Token: None 2022-09-03T06:41:56.343 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj34 2022-09-03T06:41:56.343 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B6+E8iYoXS05PrHEf5kuiW6JiQs= 2022-09-03T06:41:56.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B6+E8iYoXS05PrHEf5kuiW6JiQs=', 'Content-Length': '0'} 2022-09-03T06:41:56.346 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000048bc08253e871e3-006312f734-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.348 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:41:56.349 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:56.350 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:56.351 DEBUG:boto:Method: GET 2022-09-03T06:41:56.352 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:41:56.353 DEBUG:boto:Data: 2022-09-03T06:41:56.354 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.355 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:56.356 DEBUG:boto:Port: 8001 2022-09-03T06:41:56.357 DEBUG:boto:Params: {} 2022-09-03T06:41:56.358 DEBUG:boto:Token: None 2022-09-03T06:41:56.358 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj34 2022-09-03T06:41:56.359 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zGiHv1kvNglVY4brWeFwrO2QisI= 2022-09-03T06:41:56.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zGiHv1kvNglVY4brWeFwrO2QisI=', 'Content-Length': '0'} 2022-09-03T06:41:56.364 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007cbc9a2a116a17fa-006312f734-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.366 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:41:56.371 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:41:56.373 DEBUG:boto:Method: GET 2022-09-03T06:41:56.374 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:41:56.375 DEBUG:boto:Data: 2022-09-03T06:41:56.376 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.377 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:56.378 DEBUG:boto:Port: 8000 2022-09-03T06:41:56.379 DEBUG:boto:Params: {} 2022-09-03T06:41:56.383 DEBUG:boto:Token: None 2022-09-03T06:41:56.384 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj34 2022-09-03T06:41:56.386 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zGiHv1kvNglVY4brWeFwrO2QisI= 2022-09-03T06:41:56.386 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zGiHv1kvNglVY4brWeFwrO2QisI=', 'Content-Length': '0'} 2022-09-03T06:41:56.389 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a8e48aa0c502ee52-006312f734-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.434 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:56.435 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:56.435 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:56.436 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:56.437 DEBUG:boto:Method: GET 2022-09-03T06:41:56.437 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:41:56.438 DEBUG:boto:Data: 2022-09-03T06:41:56.438 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.439 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:56.439 DEBUG:boto:Port: 8001 2022-09-03T06:41:56.440 DEBUG:boto:Params: {} 2022-09-03T06:41:56.441 DEBUG:boto:Token: None 2022-09-03T06:41:56.441 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/ 2022-09-03T06:41:56.442 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c= 2022-09-03T06:41:56.443 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c=', 'Content-Length': '0'} 2022-09-03T06:41:56.445 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002c1bb9f1702b04d9-006312f734-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.446 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:56.448 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:56.448 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:56.449 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:56.450 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:41:56.450 DEBUG:boto:Method: GET 2022-09-03T06:41:56.451 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:41:56.452 DEBUG:boto:Data: 2022-09-03T06:41:56.452 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.453 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:56.454 DEBUG:boto:Port: 8000 2022-09-03T06:41:56.454 DEBUG:boto:Params: {} 2022-09-03T06:41:56.455 DEBUG:boto:Token: None 2022-09-03T06:41:56.456 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/ 2022-09-03T06:41:56.456 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c= 2022-09-03T06:41:56.457 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c=', 'Content-Length': '0'} 2022-09-03T06:41:56.470 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000d3559541e221d7a-006312f734-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.471 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:56.472 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:41:56.472 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:56.473 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:56.474 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.474 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.475 DEBUG:boto:Method: GET 2022-09-03T06:41:56.475 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.478 DEBUG:boto:Data: 2022-09-03T06:41:56.479 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.479 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:56.480 DEBUG:boto:Port: 8001 2022-09-03T06:41:56.480 DEBUG:boto:Params: {} 2022-09-03T06:41:56.481 DEBUG:boto:Token: None 2022-09-03T06:41:56.482 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F+jx6FitkMQZcrZaCQPuxl0sIPQ= 2022-09-03T06:41:56.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F+jx6FitkMQZcrZaCQPuxl0sIPQ=', 'Content-Length': '0'} 2022-09-03T06:41:56.487 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000023b6f367f1713bfe-006312f734-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.489 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:56.490 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:56.492 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.493 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.494 DEBUG:boto:Method: GET 2022-09-03T06:41:56.495 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.497 DEBUG:boto:Data: 2022-09-03T06:41:56.499 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.500 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:56.501 DEBUG:boto:Port: 8000 2022-09-03T06:41:56.502 DEBUG:boto:Params: {} 2022-09-03T06:41:56.504 DEBUG:boto:Token: None 2022-09-03T06:41:56.505 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.506 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F+jx6FitkMQZcrZaCQPuxl0sIPQ= 2022-09-03T06:41:56.507 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F+jx6FitkMQZcrZaCQPuxl0sIPQ=', 'Content-Length': '0'} 2022-09-03T06:41:56.511 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bbc9ad267be98e70-006312f734-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.512 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:56.513 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:56.514 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.515 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.516 DEBUG:boto:Method: HEAD 2022-09-03T06:41:56.517 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.519 DEBUG:boto:Data: 2022-09-03T06:41:56.520 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.521 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:56.521 DEBUG:boto:Port: 8001 2022-09-03T06:41:56.522 DEBUG:boto:Params: {} 2022-09-03T06:41:56.523 DEBUG:boto:Token: None 2022-09-03T06:41:56.523 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.524 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WsqK2qdFh+A/K09HOmNKlv99jxY= 2022-09-03T06:41:56.524 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WsqK2qdFh+A/K09HOmNKlv99jxY=', 'Content-Length': '0'} 2022-09-03T06:41:56.527 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000050f5d545b333903a-006312f734-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.528 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:56.529 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:56.529 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.530 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.530 DEBUG:boto:Method: HEAD 2022-09-03T06:41:56.531 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.531 DEBUG:boto:Data: 2022-09-03T06:41:56.532 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.532 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:56.533 DEBUG:boto:Port: 8000 2022-09-03T06:41:56.533 DEBUG:boto:Params: {} 2022-09-03T06:41:56.534 DEBUG:boto:Token: None 2022-09-03T06:41:56.534 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.535 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WsqK2qdFh+A/K09HOmNKlv99jxY= 2022-09-03T06:41:56.535 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WsqK2qdFh+A/K09HOmNKlv99jxY=', 'Content-Length': '0'} 2022-09-03T06:41:56.538 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000079506a13378868d1-006312f734-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.538 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:41:56.539 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:56.539 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:56.540 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.540 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.541 DEBUG:boto:Method: GET 2022-09-03T06:41:56.541 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.542 DEBUG:boto:Data: 2022-09-03T06:41:56.543 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.544 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:56.545 DEBUG:boto:Port: 8001 2022-09-03T06:41:56.546 DEBUG:boto:Params: {} 2022-09-03T06:41:56.547 DEBUG:boto:Token: None 2022-09-03T06:41:56.548 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.549 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F+jx6FitkMQZcrZaCQPuxl0sIPQ= 2022-09-03T06:41:56.550 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F+jx6FitkMQZcrZaCQPuxl0sIPQ=', 'Content-Length': '0'} 2022-09-03T06:41:56.553 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d607c243d6aa5faf-006312f734-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.601 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:56.603 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:56.604 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.605 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.610 DEBUG:boto:Method: GET 2022-09-03T06:41:56.610 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.611 DEBUG:boto:Data: 2022-09-03T06:41:56.612 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.612 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:56.613 DEBUG:boto:Port: 8000 2022-09-03T06:41:56.613 DEBUG:boto:Params: {} 2022-09-03T06:41:56.614 DEBUG:boto:Token: None 2022-09-03T06:41:56.615 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:41:56.615 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F+jx6FitkMQZcrZaCQPuxl0sIPQ= 2022-09-03T06:41:56.616 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F+jx6FitkMQZcrZaCQPuxl0sIPQ=', 'Content-Length': '0'} 2022-09-03T06:41:56.618 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000079cf7ad9e1d8d634-006312f734-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.619 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:56.620 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:56.620 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:56.621 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:56.621 DEBUG:boto:Method: GET 2022-09-03T06:41:56.622 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:41:56.623 DEBUG:boto:Data: 2022-09-03T06:41:56.623 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.624 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:56.625 DEBUG:boto:Port: 8001 2022-09-03T06:41:56.625 DEBUG:boto:Params: {} 2022-09-03T06:41:56.626 DEBUG:boto:Token: None 2022-09-03T06:41:56.627 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/ 2022-09-03T06:41:56.627 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c= 2022-09-03T06:41:56.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c=', 'Content-Length': '0'} 2022-09-03T06:41:56.632 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bdd04e23f635d24b-006312f734-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.633 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:56.634 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:56.635 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:56.635 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:56.636 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:56.637 DEBUG:boto:Method: GET 2022-09-03T06:41:56.637 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:41:56.638 DEBUG:boto:Data: 2022-09-03T06:41:56.638 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.639 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:56.639 DEBUG:boto:Port: 8000 2022-09-03T06:41:56.640 DEBUG:boto:Params: {} 2022-09-03T06:41:56.640 DEBUG:boto:Token: None 2022-09-03T06:41:56.641 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/ 2022-09-03T06:41:56.642 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c= 2022-09-03T06:41:56.642 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c=', 'Content-Length': '0'} 2022-09-03T06:41:56.649 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c82b38e87aa0ca84-006312f734-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.650 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:56.651 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:56.652 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:56.652 DEBUG:boto:Method: HEAD 2022-09-03T06:41:56.653 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:41:56.653 DEBUG:boto:Data: 2022-09-03T06:41:56.654 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.654 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:56.655 DEBUG:boto:Port: 8001 2022-09-03T06:41:56.656 DEBUG:boto:Params: {} 2022-09-03T06:41:56.656 DEBUG:boto:Token: None 2022-09-03T06:41:56.658 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj38 2022-09-03T06:41:56.658 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/+nT2uFJn+TNu53ja+K6ZaQSig= 2022-09-03T06:41:56.659 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/+nT2uFJn+TNu53ja+K6ZaQSig=', 'Content-Length': '0'} 2022-09-03T06:41:56.661 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000431e52556ae69702-006312f734-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.663 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:56.664 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:56.665 DEBUG:boto:Method: HEAD 2022-09-03T06:41:56.666 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:41:56.667 DEBUG:boto:Data: 2022-09-03T06:41:56.668 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.669 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:56.670 DEBUG:boto:Port: 8000 2022-09-03T06:41:56.671 DEBUG:boto:Params: {} 2022-09-03T06:41:56.672 DEBUG:boto:Token: None 2022-09-03T06:41:56.674 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj38 2022-09-03T06:41:56.675 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/+nT2uFJn+TNu53ja+K6ZaQSig= 2022-09-03T06:41:56.676 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/+nT2uFJn+TNu53ja+K6ZaQSig=', 'Content-Length': '0'} 2022-09-03T06:41:56.678 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000094b96ac95f405ed7-006312f734-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.679 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:41:56.680 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:56.680 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:56.681 DEBUG:boto:Method: GET 2022-09-03T06:41:56.682 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:41:56.682 DEBUG:boto:Data: 2022-09-03T06:41:56.683 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.683 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:56.684 DEBUG:boto:Port: 8001 2022-09-03T06:41:56.685 DEBUG:boto:Params: {} 2022-09-03T06:41:56.686 DEBUG:boto:Token: None 2022-09-03T06:41:56.687 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj38 2022-09-03T06:41:56.688 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QuT+rMK1f+oEufHBWkdqbB5u2x0= 2022-09-03T06:41:56.689 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QuT+rMK1f+oEufHBWkdqbB5u2x0=', 'Content-Length': '0'} 2022-09-03T06:41:56.692 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f5375861343d3e59-006312f734-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.694 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:41:56.695 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:41:56.696 DEBUG:boto:Method: GET 2022-09-03T06:41:56.697 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:41:56.697 DEBUG:boto:Data: 2022-09-03T06:41:56.698 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.699 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:56.700 DEBUG:boto:Port: 8000 2022-09-03T06:41:56.701 DEBUG:boto:Params: {} 2022-09-03T06:41:56.702 DEBUG:boto:Token: None 2022-09-03T06:41:56.703 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj38 2022-09-03T06:41:56.704 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QuT+rMK1f+oEufHBWkdqbB5u2x0= 2022-09-03T06:41:56.705 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QuT+rMK1f+oEufHBWkdqbB5u2x0=', 'Content-Length': '0'} 2022-09-03T06:41:56.708 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000786061c31de4cf15-006312f734-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.754 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:56.755 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:56.755 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:56.756 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:56.757 DEBUG:boto:Method: GET 2022-09-03T06:41:56.757 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:41:56.758 DEBUG:boto:Data: 2022-09-03T06:41:56.758 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.760 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:56.760 DEBUG:boto:Port: 8001 2022-09-03T06:41:56.761 DEBUG:boto:Params: {} 2022-09-03T06:41:56.761 DEBUG:boto:Token: None 2022-09-03T06:41:56.763 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/ 2022-09-03T06:41:56.763 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c= 2022-09-03T06:41:56.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c=', 'Content-Length': '0'} 2022-09-03T06:41:56.770 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e1a134c9369fd6ac-006312f734-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.772 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:56.772 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:56.773 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:56.774 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:56.775 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:41:56.776 DEBUG:boto:Method: GET 2022-09-03T06:41:56.777 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:41:56.778 DEBUG:boto:Data: 2022-09-03T06:41:56.779 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.780 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:56.782 DEBUG:boto:Port: 8000 2022-09-03T06:41:56.784 DEBUG:boto:Params: {} 2022-09-03T06:41:56.785 DEBUG:boto:Token: None 2022-09-03T06:41:56.786 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/ 2022-09-03T06:41:56.787 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c= 2022-09-03T06:41:56.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c=', 'Content-Length': '0'} 2022-09-03T06:41:56.794 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003c81ec51797cc56b-006312f734-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.795 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:56.796 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:41:56.797 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:56.798 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:56.799 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.800 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.800 DEBUG:boto:Method: GET 2022-09-03T06:41:56.801 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.802 DEBUG:boto:Data: 2022-09-03T06:41:56.803 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.804 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:56.806 DEBUG:boto:Port: 8001 2022-09-03T06:41:56.807 DEBUG:boto:Params: {} 2022-09-03T06:41:56.808 DEBUG:boto:Token: None 2022-09-03T06:41:56.809 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.811 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KlhfSPgTTw3KiHzQ/q/dwI1XxjU= 2022-09-03T06:41:56.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KlhfSPgTTw3KiHzQ/q/dwI1XxjU=', 'Content-Length': '0'} 2022-09-03T06:41:56.817 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000077ea7191a1e44d73-006312f734-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.820 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:56.821 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:56.822 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.824 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.826 DEBUG:boto:Method: GET 2022-09-03T06:41:56.827 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.829 DEBUG:boto:Data: 2022-09-03T06:41:56.832 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.833 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:56.834 DEBUG:boto:Port: 8000 2022-09-03T06:41:56.834 DEBUG:boto:Params: {} 2022-09-03T06:41:56.835 DEBUG:boto:Token: None 2022-09-03T06:41:56.836 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.836 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KlhfSPgTTw3KiHzQ/q/dwI1XxjU= 2022-09-03T06:41:56.837 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KlhfSPgTTw3KiHzQ/q/dwI1XxjU=', 'Content-Length': '0'} 2022-09-03T06:41:56.846 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000291e4711611f2d08-006312f734-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.847 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:56.849 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:56.849 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.850 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.850 DEBUG:boto:Method: HEAD 2022-09-03T06:41:56.851 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.851 DEBUG:boto:Data: 2022-09-03T06:41:56.852 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.853 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:56.853 DEBUG:boto:Port: 8001 2022-09-03T06:41:56.854 DEBUG:boto:Params: {} 2022-09-03T06:41:56.855 DEBUG:boto:Token: None 2022-09-03T06:41:56.855 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.856 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wp83fIENhC4ne5reCpFzV0tZvt0= 2022-09-03T06:41:56.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wp83fIENhC4ne5reCpFzV0tZvt0=', 'Content-Length': '0'} 2022-09-03T06:41:56.868 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006de03a13808a7d15-006312f734-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.870 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:56.871 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:56.872 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.872 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.873 DEBUG:boto:Method: HEAD 2022-09-03T06:41:56.874 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.874 DEBUG:boto:Data: 2022-09-03T06:41:56.875 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.875 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:56.876 DEBUG:boto:Port: 8000 2022-09-03T06:41:56.876 DEBUG:boto:Params: {} 2022-09-03T06:41:56.877 DEBUG:boto:Token: None 2022-09-03T06:41:56.878 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.878 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wp83fIENhC4ne5reCpFzV0tZvt0= 2022-09-03T06:41:56.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wp83fIENhC4ne5reCpFzV0tZvt0=', 'Content-Length': '0'} 2022-09-03T06:41:56.881 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ed19cba63cae9552-006312f734-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.883 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:41:56.883 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:56.884 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:56.889 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.891 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.893 DEBUG:boto:Method: GET 2022-09-03T06:41:56.894 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.895 DEBUG:boto:Data: 2022-09-03T06:41:56.896 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.899 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:56.900 DEBUG:boto:Port: 8001 2022-09-03T06:41:56.901 DEBUG:boto:Params: {} 2022-09-03T06:41:56.903 DEBUG:boto:Token: None 2022-09-03T06:41:56.904 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.905 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KlhfSPgTTw3KiHzQ/q/dwI1XxjU= 2022-09-03T06:41:56.907 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KlhfSPgTTw3KiHzQ/q/dwI1XxjU=', 'Content-Length': '0'} 2022-09-03T06:41:56.911 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000345ce8c7adfa192c-006312f734-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.912 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:56.913 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:56.913 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.914 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.914 DEBUG:boto:Method: GET 2022-09-03T06:41:56.915 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.915 DEBUG:boto:Data: 2022-09-03T06:41:56.916 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.917 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:56.918 DEBUG:boto:Port: 8000 2022-09-03T06:41:56.920 DEBUG:boto:Params: {} 2022-09-03T06:41:56.922 DEBUG:boto:Token: None 2022-09-03T06:41:56.923 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:41:56.925 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KlhfSPgTTw3KiHzQ/q/dwI1XxjU= 2022-09-03T06:41:56.926 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KlhfSPgTTw3KiHzQ/q/dwI1XxjU=', 'Content-Length': '0'} 2022-09-03T06:41:56.929 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bac5d5bdb992cfb1-006312f734-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.931 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:56.932 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:56.933 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:56.934 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:56.935 DEBUG:boto:Method: GET 2022-09-03T06:41:56.936 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:41:56.937 DEBUG:boto:Data: 2022-09-03T06:41:56.938 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.939 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:56.941 DEBUG:boto:Port: 8001 2022-09-03T06:41:56.942 DEBUG:boto:Params: {} 2022-09-03T06:41:56.943 DEBUG:boto:Token: None 2022-09-03T06:41:56.944 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/ 2022-09-03T06:41:56.945 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c= 2022-09-03T06:41:56.946 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c=', 'Content-Length': '0'} 2022-09-03T06:41:56.949 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ac740622b8b02b8d-006312f734-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.951 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:56.952 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:56.953 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:56.954 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:56.955 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:56.956 DEBUG:boto:Method: GET 2022-09-03T06:41:56.957 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:41:56.958 DEBUG:boto:Data: 2022-09-03T06:41:56.959 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.960 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:56.961 DEBUG:boto:Port: 8000 2022-09-03T06:41:56.962 DEBUG:boto:Params: {} 2022-09-03T06:41:56.963 DEBUG:boto:Token: None 2022-09-03T06:41:56.964 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/ 2022-09-03T06:41:56.965 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c= 2022-09-03T06:41:56.966 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eo3cxdbxaPw4gY98jaQFPBOur9c=', 'Content-Length': '0'} 2022-09-03T06:41:56.976 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000038a6ae502b62522d-006312f734-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:56.980 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:56.984 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:56.985 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:56.985 DEBUG:boto:Method: HEAD 2022-09-03T06:41:56.986 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:41:56.987 DEBUG:boto:Data: 2022-09-03T06:41:56.987 DEBUG:boto:Headers: {} 2022-09-03T06:41:56.988 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:56.988 DEBUG:boto:Port: 8001 2022-09-03T06:41:56.989 DEBUG:boto:Params: {} 2022-09-03T06:41:56.989 DEBUG:boto:Token: None 2022-09-03T06:41:56.990 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:56 GMT /dsgvcy-75/obj44 2022-09-03T06:41:56.991 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lJkrHnSJvfKoorPNE95FDDpINUs= 2022-09-03T06:41:56.991 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:56 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lJkrHnSJvfKoorPNE95FDDpINUs=', 'Content-Length': '0'} 2022-09-03T06:41:57.001 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000661a5733fe3e8201-006312f734-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:56 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.004 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:57.004 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:57.005 DEBUG:boto:Method: HEAD 2022-09-03T06:41:57.005 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:41:57.006 DEBUG:boto:Data: 2022-09-03T06:41:57.007 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.008 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:57.009 DEBUG:boto:Port: 8000 2022-09-03T06:41:57.010 DEBUG:boto:Params: {} 2022-09-03T06:41:57.011 DEBUG:boto:Token: None 2022-09-03T06:41:57.012 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj44 2022-09-03T06:41:57.012 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8BKcfLWrKZ19S29ZDbe2quzv28Q= 2022-09-03T06:41:57.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8BKcfLWrKZ19S29ZDbe2quzv28Q=', 'Content-Length': '0'} 2022-09-03T06:41:57.018 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000553092ba7b213763-006312f735-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.030 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:41:57.031 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:57.032 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:57.032 DEBUG:boto:Method: GET 2022-09-03T06:41:57.033 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:41:57.033 DEBUG:boto:Data: 2022-09-03T06:41:57.034 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.035 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:57.035 DEBUG:boto:Port: 8001 2022-09-03T06:41:57.036 DEBUG:boto:Params: {} 2022-09-03T06:41:57.036 DEBUG:boto:Token: None 2022-09-03T06:41:57.039 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj44 2022-09-03T06:41:57.040 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bh0BOOsLVnoyr467yakGf5K797c= 2022-09-03T06:41:57.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bh0BOOsLVnoyr467yakGf5K797c=', 'Content-Length': '0'} 2022-09-03T06:41:57.044 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005639428bb99c3f82-006312f735-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.046 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:41:57.047 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:41:57.048 DEBUG:boto:Method: GET 2022-09-03T06:41:57.049 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:41:57.050 DEBUG:boto:Data: 2022-09-03T06:41:57.050 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.051 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:57.051 DEBUG:boto:Port: 8000 2022-09-03T06:41:57.052 DEBUG:boto:Params: {} 2022-09-03T06:41:57.053 DEBUG:boto:Token: None 2022-09-03T06:41:57.054 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj44 2022-09-03T06:41:57.055 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bh0BOOsLVnoyr467yakGf5K797c= 2022-09-03T06:41:57.057 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bh0BOOsLVnoyr467yakGf5K797c=', 'Content-Length': '0'} 2022-09-03T06:41:57.059 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ef2d3269c3e0981c-006312f735-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.103 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:57.105 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:57.106 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:57.108 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:57.109 DEBUG:boto:Method: GET 2022-09-03T06:41:57.110 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:41:57.111 DEBUG:boto:Data: 2022-09-03T06:41:57.112 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.113 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:57.114 DEBUG:boto:Port: 8001 2022-09-03T06:41:57.115 DEBUG:boto:Params: {} 2022-09-03T06:41:57.116 DEBUG:boto:Token: None 2022-09-03T06:41:57.117 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/ 2022-09-03T06:41:57.118 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM= 2022-09-03T06:41:57.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM=', 'Content-Length': '0'} 2022-09-03T06:41:57.124 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000053831a7d3a753d09-006312f735-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.125 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:57.127 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:57.128 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:57.129 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:57.130 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:41:57.130 DEBUG:boto:Method: GET 2022-09-03T06:41:57.131 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:41:57.131 DEBUG:boto:Data: 2022-09-03T06:41:57.132 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.133 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:57.134 DEBUG:boto:Port: 8000 2022-09-03T06:41:57.135 DEBUG:boto:Params: {} 2022-09-03T06:41:57.136 DEBUG:boto:Token: None 2022-09-03T06:41:57.137 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/ 2022-09-03T06:41:57.138 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM= 2022-09-03T06:41:57.139 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM=', 'Content-Length': '0'} 2022-09-03T06:41:57.144 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000014e84944e8a3d40a-006312f735-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.145 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:57.148 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:41:57.149 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:57.150 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:57.151 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.152 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.154 DEBUG:boto:Method: GET 2022-09-03T06:41:57.156 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.157 DEBUG:boto:Data: 2022-09-03T06:41:57.158 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.160 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:57.161 DEBUG:boto:Port: 8001 2022-09-03T06:41:57.162 DEBUG:boto:Params: {} 2022-09-03T06:41:57.164 DEBUG:boto:Token: None 2022-09-03T06:41:57.165 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.166 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yX9hqF9RKArCCCZ1KWa10XXWZls= 2022-09-03T06:41:57.167 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yX9hqF9RKArCCCZ1KWa10XXWZls=', 'Content-Length': '0'} 2022-09-03T06:41:57.171 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000044132525f98c7b68-006312f735-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.173 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:57.174 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:57.175 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.176 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.177 DEBUG:boto:Method: GET 2022-09-03T06:41:57.177 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.179 DEBUG:boto:Data: 2022-09-03T06:41:57.179 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.180 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:57.190 DEBUG:boto:Port: 8000 2022-09-03T06:41:57.191 DEBUG:boto:Params: {} 2022-09-03T06:41:57.193 DEBUG:boto:Token: None 2022-09-03T06:41:57.195 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.196 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yX9hqF9RKArCCCZ1KWa10XXWZls= 2022-09-03T06:41:57.197 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yX9hqF9RKArCCCZ1KWa10XXWZls=', 'Content-Length': '0'} 2022-09-03T06:41:57.200 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000752aadf5b55ceb89-006312f735-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.201 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:57.203 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:57.204 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.204 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.205 DEBUG:boto:Method: HEAD 2022-09-03T06:41:57.205 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.206 DEBUG:boto:Data: 2022-09-03T06:41:57.206 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.207 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:57.208 DEBUG:boto:Port: 8001 2022-09-03T06:41:57.208 DEBUG:boto:Params: {} 2022-09-03T06:41:57.209 DEBUG:boto:Token: None 2022-09-03T06:41:57.210 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.210 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aHar8FLaT4w2a+X29Re8lW2rSoM= 2022-09-03T06:41:57.211 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aHar8FLaT4w2a+X29Re8lW2rSoM=', 'Content-Length': '0'} 2022-09-03T06:41:57.215 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008aa625be19597cae-006312f735-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.218 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:57.218 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:57.219 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.219 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.220 DEBUG:boto:Method: HEAD 2022-09-03T06:41:57.220 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.221 DEBUG:boto:Data: 2022-09-03T06:41:57.222 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.223 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:57.224 DEBUG:boto:Port: 8000 2022-09-03T06:41:57.227 DEBUG:boto:Params: {} 2022-09-03T06:41:57.228 DEBUG:boto:Token: None 2022-09-03T06:41:57.229 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.230 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aHar8FLaT4w2a+X29Re8lW2rSoM= 2022-09-03T06:41:57.231 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aHar8FLaT4w2a+X29Re8lW2rSoM=', 'Content-Length': '0'} 2022-09-03T06:41:57.235 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fd85e217cb69791f-006312f735-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.236 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:41:57.237 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:57.237 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:57.238 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.239 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.239 DEBUG:boto:Method: GET 2022-09-03T06:41:57.240 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.240 DEBUG:boto:Data: 2022-09-03T06:41:57.241 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.242 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:57.242 DEBUG:boto:Port: 8001 2022-09-03T06:41:57.243 DEBUG:boto:Params: {} 2022-09-03T06:41:57.243 DEBUG:boto:Token: None 2022-09-03T06:41:57.245 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.246 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yX9hqF9RKArCCCZ1KWa10XXWZls= 2022-09-03T06:41:57.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yX9hqF9RKArCCCZ1KWa10XXWZls=', 'Content-Length': '0'} 2022-09-03T06:41:57.248 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000050d691df2aaca0f9-006312f735-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.250 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:57.250 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:57.251 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.251 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.252 DEBUG:boto:Method: GET 2022-09-03T06:41:57.253 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.256 DEBUG:boto:Data: 2022-09-03T06:41:57.257 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.258 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:57.258 DEBUG:boto:Port: 8000 2022-09-03T06:41:57.259 DEBUG:boto:Params: {} 2022-09-03T06:41:57.260 DEBUG:boto:Token: None 2022-09-03T06:41:57.261 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:41:57.263 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yX9hqF9RKArCCCZ1KWa10XXWZls= 2022-09-03T06:41:57.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yX9hqF9RKArCCCZ1KWa10XXWZls=', 'Content-Length': '0'} 2022-09-03T06:41:57.267 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a322e82204e3345a-006312f735-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.268 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:57.269 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:57.270 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:57.271 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:57.273 DEBUG:boto:Method: GET 2022-09-03T06:41:57.290 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:41:57.291 DEBUG:boto:Data: 2022-09-03T06:41:57.291 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.292 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:57.292 DEBUG:boto:Port: 8001 2022-09-03T06:41:57.293 DEBUG:boto:Params: {} 2022-09-03T06:41:57.299 DEBUG:boto:Token: None 2022-09-03T06:41:57.299 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/ 2022-09-03T06:41:57.301 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM= 2022-09-03T06:41:57.301 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM=', 'Content-Length': '0'} 2022-09-03T06:41:57.304 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000200a52b10b4d1e53-006312f735-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.305 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:57.306 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:57.306 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:57.307 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:57.308 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:57.308 DEBUG:boto:Method: GET 2022-09-03T06:41:57.309 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:41:57.309 DEBUG:boto:Data: 2022-09-03T06:41:57.310 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.310 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:57.311 DEBUG:boto:Port: 8000 2022-09-03T06:41:57.311 DEBUG:boto:Params: {} 2022-09-03T06:41:57.312 DEBUG:boto:Token: None 2022-09-03T06:41:57.312 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/ 2022-09-03T06:41:57.313 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM= 2022-09-03T06:41:57.314 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM=', 'Content-Length': '0'} 2022-09-03T06:41:57.317 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000019e28169e3b302d7-006312f735-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.318 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:57.319 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:57.320 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:57.322 DEBUG:boto:Method: HEAD 2022-09-03T06:41:57.323 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:41:57.324 DEBUG:boto:Data: 2022-09-03T06:41:57.325 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.326 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:57.327 DEBUG:boto:Port: 8001 2022-09-03T06:41:57.328 DEBUG:boto:Params: {} 2022-09-03T06:41:57.329 DEBUG:boto:Token: None 2022-09-03T06:41:57.330 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj51 2022-09-03T06:41:57.331 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4yc111p0KU5WGttFjQSpNQJKogg= 2022-09-03T06:41:57.332 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4yc111p0KU5WGttFjQSpNQJKogg=', 'Content-Length': '0'} 2022-09-03T06:41:57.340 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f01f6ea87eb9fea5-006312f735-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.341 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:57.342 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:57.342 DEBUG:boto:Method: HEAD 2022-09-03T06:41:57.343 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:41:57.343 DEBUG:boto:Data: 2022-09-03T06:41:57.344 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.345 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:57.345 DEBUG:boto:Port: 8000 2022-09-03T06:41:57.346 DEBUG:boto:Params: {} 2022-09-03T06:41:57.347 DEBUG:boto:Token: None 2022-09-03T06:41:57.348 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj51 2022-09-03T06:41:57.349 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4yc111p0KU5WGttFjQSpNQJKogg= 2022-09-03T06:41:57.350 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4yc111p0KU5WGttFjQSpNQJKogg=', 'Content-Length': '0'} 2022-09-03T06:41:57.352 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a1dfc89e4e5fa54-006312f735-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.353 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:41:57.354 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:57.354 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:57.355 DEBUG:boto:Method: GET 2022-09-03T06:41:57.357 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:41:57.359 DEBUG:boto:Data: 2022-09-03T06:41:57.360 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.360 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:57.361 DEBUG:boto:Port: 8001 2022-09-03T06:41:57.361 DEBUG:boto:Params: {} 2022-09-03T06:41:57.362 DEBUG:boto:Token: None 2022-09-03T06:41:57.363 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj51 2022-09-03T06:41:57.364 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DyZihji8m8xA7ZUoGsSXZje7jkk= 2022-09-03T06:41:57.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DyZihji8m8xA7ZUoGsSXZje7jkk=', 'Content-Length': '0'} 2022-09-03T06:41:57.367 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fcd5637847993fcf-006312f735-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.369 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:41:57.369 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:41:57.370 DEBUG:boto:Method: GET 2022-09-03T06:41:57.371 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:41:57.372 DEBUG:boto:Data: 2022-09-03T06:41:57.373 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.374 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:57.375 DEBUG:boto:Port: 8000 2022-09-03T06:41:57.376 DEBUG:boto:Params: {} 2022-09-03T06:41:57.377 DEBUG:boto:Token: None 2022-09-03T06:41:57.378 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj51 2022-09-03T06:41:57.379 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DyZihji8m8xA7ZUoGsSXZje7jkk= 2022-09-03T06:41:57.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DyZihji8m8xA7ZUoGsSXZje7jkk=', 'Content-Length': '0'} 2022-09-03T06:41:57.384 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004ab67ba9fa09105f-006312f735-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.385 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:57.386 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:57.387 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:57.387 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:57.388 DEBUG:boto:Method: GET 2022-09-03T06:41:57.389 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:41:57.391 DEBUG:boto:Data: 2022-09-03T06:41:57.392 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.393 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:57.394 DEBUG:boto:Port: 8001 2022-09-03T06:41:57.395 DEBUG:boto:Params: {} 2022-09-03T06:41:57.396 DEBUG:boto:Token: None 2022-09-03T06:41:57.397 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/ 2022-09-03T06:41:57.399 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM= 2022-09-03T06:41:57.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM=', 'Content-Length': '0'} 2022-09-03T06:41:57.404 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003b26d3d0f79ded62-006312f735-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.446 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:57.448 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:57.449 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:57.450 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:57.451 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:41:57.452 DEBUG:boto:Method: GET 2022-09-03T06:41:57.453 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:41:57.454 DEBUG:boto:Data: 2022-09-03T06:41:57.455 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.456 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:57.457 DEBUG:boto:Port: 8000 2022-09-03T06:41:57.458 DEBUG:boto:Params: {} 2022-09-03T06:41:57.459 DEBUG:boto:Token: None 2022-09-03T06:41:57.460 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/ 2022-09-03T06:41:57.461 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM= 2022-09-03T06:41:57.462 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM=', 'Content-Length': '0'} 2022-09-03T06:41:57.470 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000918cff8d398924f-006312f735-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.471 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:57.473 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:41:57.474 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:57.475 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:57.476 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.477 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.478 DEBUG:boto:Method: GET 2022-09-03T06:41:57.479 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.480 DEBUG:boto:Data: 2022-09-03T06:41:57.481 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.482 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:57.484 DEBUG:boto:Port: 8001 2022-09-03T06:41:57.485 DEBUG:boto:Params: {} 2022-09-03T06:41:57.486 DEBUG:boto:Token: None 2022-09-03T06:41:57.487 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.488 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YcjP3aFY6EYre+2Jlw67Q2xx004= 2022-09-03T06:41:57.490 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YcjP3aFY6EYre+2Jlw67Q2xx004=', 'Content-Length': '0'} 2022-09-03T06:41:57.493 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007cd1331ea1a36c3c-006312f735-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.494 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:57.496 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:57.497 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.498 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.499 DEBUG:boto:Method: GET 2022-09-03T06:41:57.500 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.500 DEBUG:boto:Data: 2022-09-03T06:41:57.501 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.502 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:57.503 DEBUG:boto:Port: 8000 2022-09-03T06:41:57.503 DEBUG:boto:Params: {} 2022-09-03T06:41:57.504 DEBUG:boto:Token: None 2022-09-03T06:41:57.505 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.506 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YcjP3aFY6EYre+2Jlw67Q2xx004= 2022-09-03T06:41:57.507 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YcjP3aFY6EYre+2Jlw67Q2xx004=', 'Content-Length': '0'} 2022-09-03T06:41:57.509 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009906e2e4c21b2db4-006312f735-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.510 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:57.511 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:57.511 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.512 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.512 DEBUG:boto:Method: HEAD 2022-09-03T06:41:57.513 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.513 DEBUG:boto:Data: 2022-09-03T06:41:57.514 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.514 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:57.515 DEBUG:boto:Port: 8001 2022-09-03T06:41:57.515 DEBUG:boto:Params: {} 2022-09-03T06:41:57.516 DEBUG:boto:Token: None 2022-09-03T06:41:57.517 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.518 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cmnC4QioQAN3WMi0ZOYj++VSdZI= 2022-09-03T06:41:57.518 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cmnC4QioQAN3WMi0ZOYj++VSdZI=', 'Content-Length': '0'} 2022-09-03T06:41:57.521 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000985da8e878843489-006312f735-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.522 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:57.522 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:57.523 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.523 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.524 DEBUG:boto:Method: HEAD 2022-09-03T06:41:57.524 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.525 DEBUG:boto:Data: 2022-09-03T06:41:57.525 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.526 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:57.527 DEBUG:boto:Port: 8000 2022-09-03T06:41:57.527 DEBUG:boto:Params: {} 2022-09-03T06:41:57.528 DEBUG:boto:Token: None 2022-09-03T06:41:57.529 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.529 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cmnC4QioQAN3WMi0ZOYj++VSdZI= 2022-09-03T06:41:57.530 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cmnC4QioQAN3WMi0ZOYj++VSdZI=', 'Content-Length': '0'} 2022-09-03T06:41:57.533 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000dd45cac2acf1d8d-006312f735-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.534 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:41:57.535 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:57.535 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:57.536 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.536 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.537 DEBUG:boto:Method: GET 2022-09-03T06:41:57.537 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.538 DEBUG:boto:Data: 2022-09-03T06:41:57.539 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.540 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:57.540 DEBUG:boto:Port: 8001 2022-09-03T06:41:57.541 DEBUG:boto:Params: {} 2022-09-03T06:41:57.542 DEBUG:boto:Token: None 2022-09-03T06:41:57.543 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.544 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YcjP3aFY6EYre+2Jlw67Q2xx004= 2022-09-03T06:41:57.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YcjP3aFY6EYre+2Jlw67Q2xx004=', 'Content-Length': '0'} 2022-09-03T06:41:57.547 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000075e3c83f2351e060-006312f735-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.548 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:57.549 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:57.549 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.550 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.551 DEBUG:boto:Method: GET 2022-09-03T06:41:57.551 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.552 DEBUG:boto:Data: 2022-09-03T06:41:57.552 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.553 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:57.554 DEBUG:boto:Port: 8000 2022-09-03T06:41:57.554 DEBUG:boto:Params: {} 2022-09-03T06:41:57.555 DEBUG:boto:Token: None 2022-09-03T06:41:57.555 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:41:57.556 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YcjP3aFY6EYre+2Jlw67Q2xx004= 2022-09-03T06:41:57.557 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YcjP3aFY6EYre+2Jlw67Q2xx004=', 'Content-Length': '0'} 2022-09-03T06:41:57.559 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008015acf69dc871ad-006312f735-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.560 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:57.561 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:57.561 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:57.562 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:57.562 DEBUG:boto:Method: GET 2022-09-03T06:41:57.563 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:41:57.564 DEBUG:boto:Data: 2022-09-03T06:41:57.564 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.565 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:57.565 DEBUG:boto:Port: 8001 2022-09-03T06:41:57.566 DEBUG:boto:Params: {} 2022-09-03T06:41:57.567 DEBUG:boto:Token: None 2022-09-03T06:41:57.567 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/ 2022-09-03T06:41:57.568 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM= 2022-09-03T06:41:57.569 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM=', 'Content-Length': '0'} 2022-09-03T06:41:57.572 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004c3a4eb9df25448d-006312f735-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.573 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:57.584 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:57.585 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:57.585 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:57.586 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:57.586 DEBUG:boto:Method: GET 2022-09-03T06:41:57.587 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:41:57.587 DEBUG:boto:Data: 2022-09-03T06:41:57.588 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.588 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:57.589 DEBUG:boto:Port: 8000 2022-09-03T06:41:57.590 DEBUG:boto:Params: {} 2022-09-03T06:41:57.591 DEBUG:boto:Token: None 2022-09-03T06:41:57.592 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/ 2022-09-03T06:41:57.594 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM= 2022-09-03T06:41:57.595 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM=', 'Content-Length': '0'} 2022-09-03T06:41:57.602 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eb460352d5f43275-006312f735-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.642 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:57.645 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:57.646 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:57.647 DEBUG:boto:Method: HEAD 2022-09-03T06:41:57.648 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:41:57.649 DEBUG:boto:Data: 2022-09-03T06:41:57.650 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.651 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:57.652 DEBUG:boto:Port: 8001 2022-09-03T06:41:57.653 DEBUG:boto:Params: {} 2022-09-03T06:41:57.655 DEBUG:boto:Token: None 2022-09-03T06:41:57.656 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj55 2022-09-03T06:41:57.656 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sriZu3g7uYW25Q3Ib19KevVIAVM= 2022-09-03T06:41:57.657 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sriZu3g7uYW25Q3Ib19KevVIAVM=', 'Content-Length': '0'} 2022-09-03T06:41:57.664 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a60720f623f02099-006312f735-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.665 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:57.666 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:57.668 DEBUG:boto:Method: HEAD 2022-09-03T06:41:57.669 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:41:57.670 DEBUG:boto:Data: 2022-09-03T06:41:57.671 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.671 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:57.672 DEBUG:boto:Port: 8000 2022-09-03T06:41:57.673 DEBUG:boto:Params: {} 2022-09-03T06:41:57.674 DEBUG:boto:Token: None 2022-09-03T06:41:57.675 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj55 2022-09-03T06:41:57.676 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sriZu3g7uYW25Q3Ib19KevVIAVM= 2022-09-03T06:41:57.677 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sriZu3g7uYW25Q3Ib19KevVIAVM=', 'Content-Length': '0'} 2022-09-03T06:41:57.680 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e6ad0c1bbb3b5132-006312f735-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.682 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:41:57.684 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:57.685 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:57.686 DEBUG:boto:Method: GET 2022-09-03T06:41:57.686 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:41:57.687 DEBUG:boto:Data: 2022-09-03T06:41:57.688 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.689 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:57.690 DEBUG:boto:Port: 8001 2022-09-03T06:41:57.691 DEBUG:boto:Params: {} 2022-09-03T06:41:57.692 DEBUG:boto:Token: None 2022-09-03T06:41:57.693 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj55 2022-09-03T06:41:57.694 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GRO/rk+ujwQ0WYiak3xO/DcJz04= 2022-09-03T06:41:57.695 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GRO/rk+ujwQ0WYiak3xO/DcJz04=', 'Content-Length': '0'} 2022-09-03T06:41:57.698 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000081344a741b39319a-006312f735-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.699 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:41:57.700 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:41:57.701 DEBUG:boto:Method: GET 2022-09-03T06:41:57.702 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:41:57.703 DEBUG:boto:Data: 2022-09-03T06:41:57.703 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.704 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:57.705 DEBUG:boto:Port: 8000 2022-09-03T06:41:57.706 DEBUG:boto:Params: {} 2022-09-03T06:41:57.708 DEBUG:boto:Token: None 2022-09-03T06:41:57.709 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj55 2022-09-03T06:41:57.710 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GRO/rk+ujwQ0WYiak3xO/DcJz04= 2022-09-03T06:41:57.711 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GRO/rk+ujwQ0WYiak3xO/DcJz04=', 'Content-Length': '0'} 2022-09-03T06:41:57.716 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca4401dd44a000d7-006312f735-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.717 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:57.718 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:57.720 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:57.721 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:57.724 DEBUG:boto:Method: GET 2022-09-03T06:41:57.724 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:41:57.725 DEBUG:boto:Data: 2022-09-03T06:41:57.725 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.726 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:57.726 DEBUG:boto:Port: 8001 2022-09-03T06:41:57.727 DEBUG:boto:Params: {} 2022-09-03T06:41:57.728 DEBUG:boto:Token: None 2022-09-03T06:41:57.728 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/ 2022-09-03T06:41:57.729 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM= 2022-09-03T06:41:57.730 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM=', 'Content-Length': '0'} 2022-09-03T06:41:57.735 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b51679e0d8e024fa-006312f735-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.736 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:57.737 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:57.737 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:57.738 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:57.738 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:41:57.739 DEBUG:boto:Method: GET 2022-09-03T06:41:57.739 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:41:57.740 DEBUG:boto:Data: 2022-09-03T06:41:57.740 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.741 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:57.742 DEBUG:boto:Port: 8000 2022-09-03T06:41:57.742 DEBUG:boto:Params: {} 2022-09-03T06:41:57.743 DEBUG:boto:Token: None 2022-09-03T06:41:57.744 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/ 2022-09-03T06:41:57.745 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM= 2022-09-03T06:41:57.745 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM=', 'Content-Length': '0'} 2022-09-03T06:41:57.748 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000307e07988251f09e-006312f735-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.750 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:57.751 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:41:57.753 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:57.754 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:57.755 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.757 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.758 DEBUG:boto:Method: GET 2022-09-03T06:41:57.759 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.760 DEBUG:boto:Data: 2022-09-03T06:41:57.761 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.762 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:57.763 DEBUG:boto:Port: 8001 2022-09-03T06:41:57.763 DEBUG:boto:Params: {} 2022-09-03T06:41:57.764 DEBUG:boto:Token: None 2022-09-03T06:41:57.764 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.765 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:poH/ceG3H3M3tJOtutjvHvyv30Q= 2022-09-03T06:41:57.766 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:poH/ceG3H3M3tJOtutjvHvyv30Q=', 'Content-Length': '0'} 2022-09-03T06:41:57.768 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000df51d0f76b9a4f2-006312f735-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.770 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:57.771 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:57.773 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.775 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.775 DEBUG:boto:Method: GET 2022-09-03T06:41:57.776 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.777 DEBUG:boto:Data: 2022-09-03T06:41:57.777 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.778 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:57.778 DEBUG:boto:Port: 8000 2022-09-03T06:41:57.779 DEBUG:boto:Params: {} 2022-09-03T06:41:57.780 DEBUG:boto:Token: None 2022-09-03T06:41:57.780 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.781 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:poH/ceG3H3M3tJOtutjvHvyv30Q= 2022-09-03T06:41:57.782 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:poH/ceG3H3M3tJOtutjvHvyv30Q=', 'Content-Length': '0'} 2022-09-03T06:41:57.785 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dbc83037788dc878-006312f735-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.786 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:57.787 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:57.787 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.788 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.789 DEBUG:boto:Method: HEAD 2022-09-03T06:41:57.789 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.790 DEBUG:boto:Data: 2022-09-03T06:41:57.791 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.791 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:57.793 DEBUG:boto:Port: 8001 2022-09-03T06:41:57.794 DEBUG:boto:Params: {} 2022-09-03T06:41:57.795 DEBUG:boto:Token: None 2022-09-03T06:41:57.796 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.798 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EZqbbDLyUXmJDQxWYsJbLEQZYXk= 2022-09-03T06:41:57.799 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EZqbbDLyUXmJDQxWYsJbLEQZYXk=', 'Content-Length': '0'} 2022-09-03T06:41:57.802 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000209437b9c35cacd5-006312f735-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.803 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:57.805 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:57.806 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.807 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.808 DEBUG:boto:Method: HEAD 2022-09-03T06:41:57.809 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.810 DEBUG:boto:Data: 2022-09-03T06:41:57.811 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.812 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:57.813 DEBUG:boto:Port: 8000 2022-09-03T06:41:57.814 DEBUG:boto:Params: {} 2022-09-03T06:41:57.815 DEBUG:boto:Token: None 2022-09-03T06:41:57.816 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.818 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EZqbbDLyUXmJDQxWYsJbLEQZYXk= 2022-09-03T06:41:57.819 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EZqbbDLyUXmJDQxWYsJbLEQZYXk=', 'Content-Length': '0'} 2022-09-03T06:41:57.823 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000778e3bc0d7b2cbaa-006312f735-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.825 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:41:57.827 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:57.828 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:57.829 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.830 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.831 DEBUG:boto:Method: GET 2022-09-03T06:41:57.832 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.833 DEBUG:boto:Data: 2022-09-03T06:41:57.834 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.835 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:57.837 DEBUG:boto:Port: 8001 2022-09-03T06:41:57.838 DEBUG:boto:Params: {} 2022-09-03T06:41:57.839 DEBUG:boto:Token: None 2022-09-03T06:41:57.840 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.841 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:poH/ceG3H3M3tJOtutjvHvyv30Q= 2022-09-03T06:41:57.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:poH/ceG3H3M3tJOtutjvHvyv30Q=', 'Content-Length': '0'} 2022-09-03T06:41:57.844 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e072021fb80e73f6-006312f735-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.845 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:57.846 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:57.846 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.847 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.847 DEBUG:boto:Method: GET 2022-09-03T06:41:57.848 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.848 DEBUG:boto:Data: 2022-09-03T06:41:57.850 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.850 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:57.851 DEBUG:boto:Port: 8000 2022-09-03T06:41:57.851 DEBUG:boto:Params: {} 2022-09-03T06:41:57.852 DEBUG:boto:Token: None 2022-09-03T06:41:57.853 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:41:57.854 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:poH/ceG3H3M3tJOtutjvHvyv30Q= 2022-09-03T06:41:57.855 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:poH/ceG3H3M3tJOtutjvHvyv30Q=', 'Content-Length': '0'} 2022-09-03T06:41:57.857 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bf8f48e2b1ba0c4c-006312f735-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.858 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:57.858 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:57.859 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:57.859 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:57.860 DEBUG:boto:Method: GET 2022-09-03T06:41:57.860 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:41:57.861 DEBUG:boto:Data: 2022-09-03T06:41:57.862 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.862 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:57.863 DEBUG:boto:Port: 8001 2022-09-03T06:41:57.864 DEBUG:boto:Params: {} 2022-09-03T06:41:57.865 DEBUG:boto:Token: None 2022-09-03T06:41:57.867 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/ 2022-09-03T06:41:57.868 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM= 2022-09-03T06:41:57.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM=', 'Content-Length': '0'} 2022-09-03T06:41:57.871 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ac1fcf810863bc08-006312f735-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.872 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:57.873 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:57.874 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:57.874 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:57.875 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:57.875 DEBUG:boto:Method: GET 2022-09-03T06:41:57.876 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:41:57.876 DEBUG:boto:Data: 2022-09-03T06:41:57.877 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.877 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:57.878 DEBUG:boto:Port: 8000 2022-09-03T06:41:57.878 DEBUG:boto:Params: {} 2022-09-03T06:41:57.879 DEBUG:boto:Token: None 2022-09-03T06:41:57.880 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/ 2022-09-03T06:41:57.880 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM= 2022-09-03T06:41:57.881 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM=', 'Content-Length': '0'} 2022-09-03T06:41:57.884 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000018a5055d9172c074-006312f735-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.884 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:57.885 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:57.886 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:57.886 DEBUG:boto:Method: HEAD 2022-09-03T06:41:57.887 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:41:57.887 DEBUG:boto:Data: 2022-09-03T06:41:57.888 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.888 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:57.889 DEBUG:boto:Port: 8001 2022-09-03T06:41:57.889 DEBUG:boto:Params: {} 2022-09-03T06:41:57.890 DEBUG:boto:Token: None 2022-09-03T06:41:57.891 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj57 2022-09-03T06:41:57.891 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:trvg7oNnaUH7YDk7NRCfqbmDMGk= 2022-09-03T06:41:57.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:trvg7oNnaUH7YDk7NRCfqbmDMGk=', 'Content-Length': '0'} 2022-09-03T06:41:57.894 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a906c947d4f87fbb-006312f735-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.895 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:57.895 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:57.896 DEBUG:boto:Method: HEAD 2022-09-03T06:41:57.896 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:41:57.897 DEBUG:boto:Data: 2022-09-03T06:41:57.897 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.898 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:57.898 DEBUG:boto:Port: 8000 2022-09-03T06:41:57.899 DEBUG:boto:Params: {} 2022-09-03T06:41:57.899 DEBUG:boto:Token: None 2022-09-03T06:41:57.900 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj57 2022-09-03T06:41:57.900 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:trvg7oNnaUH7YDk7NRCfqbmDMGk= 2022-09-03T06:41:57.901 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:trvg7oNnaUH7YDk7NRCfqbmDMGk=', 'Content-Length': '0'} 2022-09-03T06:41:57.903 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e06ac27380610837-006312f735-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.904 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:41:57.904 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:57.905 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:57.905 DEBUG:boto:Method: GET 2022-09-03T06:41:57.906 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:41:57.906 DEBUG:boto:Data: 2022-09-03T06:41:57.907 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.908 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:57.908 DEBUG:boto:Port: 8001 2022-09-03T06:41:57.909 DEBUG:boto:Params: {} 2022-09-03T06:41:57.909 DEBUG:boto:Token: None 2022-09-03T06:41:57.910 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj57 2022-09-03T06:41:57.911 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sbdH1kXQ/lSIv8vFu5lCJqS7sHQ= 2022-09-03T06:41:57.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sbdH1kXQ/lSIv8vFu5lCJqS7sHQ=', 'Content-Length': '0'} 2022-09-03T06:41:57.915 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d040a85dc8dc2b46-006312f735-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.959 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:41:57.961 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:41:57.963 DEBUG:boto:Method: GET 2022-09-03T06:41:57.964 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:41:57.967 DEBUG:boto:Data: 2022-09-03T06:41:57.968 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.969 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:57.970 DEBUG:boto:Port: 8000 2022-09-03T06:41:57.971 DEBUG:boto:Params: {} 2022-09-03T06:41:57.972 DEBUG:boto:Token: None 2022-09-03T06:41:57.973 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/obj57 2022-09-03T06:41:57.975 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sbdH1kXQ/lSIv8vFu5lCJqS7sHQ= 2022-09-03T06:41:57.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sbdH1kXQ/lSIv8vFu5lCJqS7sHQ=', 'Content-Length': '0'} 2022-09-03T06:41:57.979 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000012de93b559edee9-006312f735-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:57.982 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:57.983 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:57.984 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:57.985 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:57.986 DEBUG:boto:Method: GET 2022-09-03T06:41:57.987 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:41:57.988 DEBUG:boto:Data: 2022-09-03T06:41:57.989 DEBUG:boto:Headers: {} 2022-09-03T06:41:57.990 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:57.991 DEBUG:boto:Port: 8001 2022-09-03T06:41:57.992 DEBUG:boto:Params: {} 2022-09-03T06:41:57.994 DEBUG:boto:Token: None 2022-09-03T06:41:57.995 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:57 GMT /dsgvcy-75/ 2022-09-03T06:41:57.996 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM= 2022-09-03T06:41:57.997 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:57 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Haho5YaDbxuCtUuDriCAPwUqAIM=', 'Content-Length': '0'} 2022-09-03T06:41:58.001 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000afe5fa9787b8b7e8-006312f735-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:57 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.002 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:58.004 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:58.005 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:58.006 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:58.008 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:41:58.009 DEBUG:boto:Method: GET 2022-09-03T06:41:58.010 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:41:58.011 DEBUG:boto:Data: 2022-09-03T06:41:58.014 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.015 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.016 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.016 DEBUG:boto:Params: {} 2022-09-03T06:41:58.017 DEBUG:boto:Token: None 2022-09-03T06:41:58.018 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/ 2022-09-03T06:41:58.019 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA= 2022-09-03T06:41:58.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA=', 'Content-Length': '0'} 2022-09-03T06:41:58.025 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008b8bfb8af83131dc-006312f736-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.026 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:58.028 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:41:58.029 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:58.030 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:58.031 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.033 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.034 DEBUG:boto:Method: GET 2022-09-03T06:41:58.035 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.037 DEBUG:boto:Data: 2022-09-03T06:41:58.038 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.039 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.040 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.041 DEBUG:boto:Params: {} 2022-09-03T06:41:58.041 DEBUG:boto:Token: None 2022-09-03T06:41:58.042 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.043 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rt1OJzhCCtXN9arVdKEYpqYHd28= 2022-09-03T06:41:58.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rt1OJzhCCtXN9arVdKEYpqYHd28=', 'Content-Length': '0'} 2022-09-03T06:41:58.046 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000076a91d07d6bf3e45-006312f736-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.048 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:58.049 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:58.050 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.051 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.052 DEBUG:boto:Method: GET 2022-09-03T06:41:58.056 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.057 DEBUG:boto:Data: 2022-09-03T06:41:58.058 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.059 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.060 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.061 DEBUG:boto:Params: {} 2022-09-03T06:41:58.063 DEBUG:boto:Token: None 2022-09-03T06:41:58.067 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.068 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rt1OJzhCCtXN9arVdKEYpqYHd28= 2022-09-03T06:41:58.070 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rt1OJzhCCtXN9arVdKEYpqYHd28=', 'Content-Length': '0'} 2022-09-03T06:41:58.074 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000288f1b17f43bc9d6-006312f736-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.075 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:58.076 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:58.077 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.077 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.078 DEBUG:boto:Method: HEAD 2022-09-03T06:41:58.078 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.079 DEBUG:boto:Data: 2022-09-03T06:41:58.080 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.080 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.081 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.081 DEBUG:boto:Params: {} 2022-09-03T06:41:58.082 DEBUG:boto:Token: None 2022-09-03T06:41:58.082 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.083 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:46wRvSw1eyqZga6IlQ0VQwnfU3U= 2022-09-03T06:41:58.084 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:46wRvSw1eyqZga6IlQ0VQwnfU3U=', 'Content-Length': '0'} 2022-09-03T06:41:58.085 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006da2926b61ed0e46-006312f736-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.086 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:58.087 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:58.087 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.088 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.089 DEBUG:boto:Method: HEAD 2022-09-03T06:41:58.089 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.090 DEBUG:boto:Data: 2022-09-03T06:41:58.090 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.091 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.092 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.092 DEBUG:boto:Params: {} 2022-09-03T06:41:58.094 DEBUG:boto:Token: None 2022-09-03T06:41:58.095 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.096 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:46wRvSw1eyqZga6IlQ0VQwnfU3U= 2022-09-03T06:41:58.097 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:46wRvSw1eyqZga6IlQ0VQwnfU3U=', 'Content-Length': '0'} 2022-09-03T06:41:58.100 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006c0ce77bd63114eb-006312f736-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.102 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:41:58.103 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:58.104 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:58.105 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.105 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.106 DEBUG:boto:Method: GET 2022-09-03T06:41:58.107 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.108 DEBUG:boto:Data: 2022-09-03T06:41:58.108 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.109 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.110 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.112 DEBUG:boto:Params: {} 2022-09-03T06:41:58.113 DEBUG:boto:Token: None 2022-09-03T06:41:58.114 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.115 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rt1OJzhCCtXN9arVdKEYpqYHd28= 2022-09-03T06:41:58.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rt1OJzhCCtXN9arVdKEYpqYHd28=', 'Content-Length': '0'} 2022-09-03T06:41:58.120 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e6f56474556eea1-006312f736-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.126 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:58.127 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:58.127 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.128 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.129 DEBUG:boto:Method: GET 2022-09-03T06:41:58.130 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.136 DEBUG:boto:Data: 2022-09-03T06:41:58.137 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.138 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.138 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.139 DEBUG:boto:Params: {} 2022-09-03T06:41:58.143 DEBUG:boto:Token: None 2022-09-03T06:41:58.144 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:41:58.145 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rt1OJzhCCtXN9arVdKEYpqYHd28= 2022-09-03T06:41:58.145 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rt1OJzhCCtXN9arVdKEYpqYHd28=', 'Content-Length': '0'} 2022-09-03T06:41:58.148 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f37ecc7a4563214-006312f736-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.149 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:58.149 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:58.150 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:58.151 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:58.151 DEBUG:boto:Method: GET 2022-09-03T06:41:58.152 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:41:58.152 DEBUG:boto:Data: 2022-09-03T06:41:58.153 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.153 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.154 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.155 DEBUG:boto:Params: {} 2022-09-03T06:41:58.155 DEBUG:boto:Token: None 2022-09-03T06:41:58.156 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/ 2022-09-03T06:41:58.157 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA= 2022-09-03T06:41:58.157 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA=', 'Content-Length': '0'} 2022-09-03T06:41:58.160 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000523e33439a37b6f7-006312f736-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.161 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:58.162 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:58.163 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:58.163 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:58.164 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:58.164 DEBUG:boto:Method: GET 2022-09-03T06:41:58.165 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:41:58.165 DEBUG:boto:Data: 2022-09-03T06:41:58.166 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.166 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.167 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.168 DEBUG:boto:Params: {} 2022-09-03T06:41:58.168 DEBUG:boto:Token: None 2022-09-03T06:41:58.169 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/ 2022-09-03T06:41:58.170 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA= 2022-09-03T06:41:58.170 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA=', 'Content-Length': '0'} 2022-09-03T06:41:58.173 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cbd01f03ea0b64b4-006312f736-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.174 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:58.175 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:58.175 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:58.176 DEBUG:boto:Method: HEAD 2022-09-03T06:41:58.181 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:41:58.181 DEBUG:boto:Data: 2022-09-03T06:41:58.182 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.183 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.183 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.184 DEBUG:boto:Params: {} 2022-09-03T06:41:58.184 DEBUG:boto:Token: None 2022-09-03T06:41:58.185 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj68 2022-09-03T06:41:58.186 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+nRNYwX0gNsljwNU4I7JP+7a1L8= 2022-09-03T06:41:58.186 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+nRNYwX0gNsljwNU4I7JP+7a1L8=', 'Content-Length': '0'} 2022-09-03T06:41:58.188 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fb6d18dc8145d945-006312f736-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.189 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:58.190 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:58.190 DEBUG:boto:Method: HEAD 2022-09-03T06:41:58.191 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:41:58.191 DEBUG:boto:Data: 2022-09-03T06:41:58.192 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.193 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.193 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.194 DEBUG:boto:Params: {} 2022-09-03T06:41:58.194 DEBUG:boto:Token: None 2022-09-03T06:41:58.195 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj68 2022-09-03T06:41:58.195 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+nRNYwX0gNsljwNU4I7JP+7a1L8= 2022-09-03T06:41:58.196 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+nRNYwX0gNsljwNU4I7JP+7a1L8=', 'Content-Length': '0'} 2022-09-03T06:41:58.198 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b979eeca82737179-006312f736-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.199 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:41:58.200 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:58.200 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:58.201 DEBUG:boto:Method: GET 2022-09-03T06:41:58.202 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:41:58.202 DEBUG:boto:Data: 2022-09-03T06:41:58.203 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.203 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.204 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.204 DEBUG:boto:Params: {} 2022-09-03T06:41:58.205 DEBUG:boto:Token: None 2022-09-03T06:41:58.205 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj68 2022-09-03T06:41:58.206 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vYVkazyj+mY1Ugh5bhS/KlTIWKg= 2022-09-03T06:41:58.207 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vYVkazyj+mY1Ugh5bhS/KlTIWKg=', 'Content-Length': '0'} 2022-09-03T06:41:58.209 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000047bf386c1726e368-006312f736-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.210 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:41:58.211 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:41:58.211 DEBUG:boto:Method: GET 2022-09-03T06:41:58.212 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:41:58.212 DEBUG:boto:Data: 2022-09-03T06:41:58.213 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.213 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.214 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.214 DEBUG:boto:Params: {} 2022-09-03T06:41:58.215 DEBUG:boto:Token: None 2022-09-03T06:41:58.216 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj68 2022-09-03T06:41:58.216 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vYVkazyj+mY1Ugh5bhS/KlTIWKg= 2022-09-03T06:41:58.217 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vYVkazyj+mY1Ugh5bhS/KlTIWKg=', 'Content-Length': '0'} 2022-09-03T06:41:58.220 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002dcd8e2b6d2c0d5a-006312f736-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.266 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:58.267 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:58.278 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:58.278 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:58.279 DEBUG:boto:Method: GET 2022-09-03T06:41:58.280 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:41:58.280 DEBUG:boto:Data: 2022-09-03T06:41:58.281 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.281 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.285 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.291 DEBUG:boto:Params: {} 2022-09-03T06:41:58.292 DEBUG:boto:Token: None 2022-09-03T06:41:58.293 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/ 2022-09-03T06:41:58.294 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA= 2022-09-03T06:41:58.294 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA=', 'Content-Length': '0'} 2022-09-03T06:41:58.298 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000af705b7135498cb-006312f736-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.299 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:58.301 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:58.301 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:58.302 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:58.303 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:41:58.304 DEBUG:boto:Method: GET 2022-09-03T06:41:58.305 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:41:58.306 DEBUG:boto:Data: 2022-09-03T06:41:58.307 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.309 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.309 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.311 DEBUG:boto:Params: {} 2022-09-03T06:41:58.312 DEBUG:boto:Token: None 2022-09-03T06:41:58.313 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/ 2022-09-03T06:41:58.314 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA= 2022-09-03T06:41:58.315 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA=', 'Content-Length': '0'} 2022-09-03T06:41:58.319 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d2bdf2811e0384f1-006312f736-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.320 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:58.321 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:41:58.322 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:58.323 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:58.324 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.325 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.325 DEBUG:boto:Method: GET 2022-09-03T06:41:58.326 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.327 DEBUG:boto:Data: 2022-09-03T06:41:58.328 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.329 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.330 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.331 DEBUG:boto:Params: {} 2022-09-03T06:41:58.332 DEBUG:boto:Token: None 2022-09-03T06:41:58.332 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.333 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WS3cxHcL82XCh5Q8mjBhZ+elxxE= 2022-09-03T06:41:58.335 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WS3cxHcL82XCh5Q8mjBhZ+elxxE=', 'Content-Length': '0'} 2022-09-03T06:41:58.337 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000800fb0a20bee43ba-006312f736-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.338 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:58.339 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:58.339 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.340 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.340 DEBUG:boto:Method: GET 2022-09-03T06:41:58.341 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.342 DEBUG:boto:Data: 2022-09-03T06:41:58.343 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.343 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.344 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.344 DEBUG:boto:Params: {} 2022-09-03T06:41:58.345 DEBUG:boto:Token: None 2022-09-03T06:41:58.346 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.346 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WS3cxHcL82XCh5Q8mjBhZ+elxxE= 2022-09-03T06:41:58.347 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WS3cxHcL82XCh5Q8mjBhZ+elxxE=', 'Content-Length': '0'} 2022-09-03T06:41:58.350 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eed1d2ece79a069c-006312f736-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.352 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:58.352 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:58.353 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.353 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.354 DEBUG:boto:Method: HEAD 2022-09-03T06:41:58.354 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.355 DEBUG:boto:Data: 2022-09-03T06:41:58.355 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.357 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.358 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.358 DEBUG:boto:Params: {} 2022-09-03T06:41:58.359 DEBUG:boto:Token: None 2022-09-03T06:41:58.359 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.360 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O6c58ReDadkuFN+b4kvlbARDBV4= 2022-09-03T06:41:58.361 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O6c58ReDadkuFN+b4kvlbARDBV4=', 'Content-Length': '0'} 2022-09-03T06:41:58.363 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e8383999e5e46a7-006312f736-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.364 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:58.364 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:58.365 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.365 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.366 DEBUG:boto:Method: HEAD 2022-09-03T06:41:58.367 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.367 DEBUG:boto:Data: 2022-09-03T06:41:58.368 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.368 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.369 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.369 DEBUG:boto:Params: {} 2022-09-03T06:41:58.370 DEBUG:boto:Token: None 2022-09-03T06:41:58.371 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.371 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O6c58ReDadkuFN+b4kvlbARDBV4= 2022-09-03T06:41:58.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O6c58ReDadkuFN+b4kvlbARDBV4=', 'Content-Length': '0'} 2022-09-03T06:41:58.375 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000045de3ebe6c5aecd4-006312f736-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.376 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:41:58.377 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:58.377 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:58.378 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.379 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.379 DEBUG:boto:Method: GET 2022-09-03T06:41:58.380 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.380 DEBUG:boto:Data: 2022-09-03T06:41:58.381 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.381 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.382 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.383 DEBUG:boto:Params: {} 2022-09-03T06:41:58.383 DEBUG:boto:Token: None 2022-09-03T06:41:58.384 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.385 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WS3cxHcL82XCh5Q8mjBhZ+elxxE= 2022-09-03T06:41:58.385 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WS3cxHcL82XCh5Q8mjBhZ+elxxE=', 'Content-Length': '0'} 2022-09-03T06:41:58.388 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000794a56081984baf7-006312f736-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.434 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:58.435 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:58.436 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.437 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.439 DEBUG:boto:Method: GET 2022-09-03T06:41:58.440 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.441 DEBUG:boto:Data: 2022-09-03T06:41:58.442 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.442 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.443 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.444 DEBUG:boto:Params: {} 2022-09-03T06:41:58.444 DEBUG:boto:Token: None 2022-09-03T06:41:58.445 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:41:58.449 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WS3cxHcL82XCh5Q8mjBhZ+elxxE= 2022-09-03T06:41:58.450 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WS3cxHcL82XCh5Q8mjBhZ+elxxE=', 'Content-Length': '0'} 2022-09-03T06:41:58.452 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000077463eb46c664043-006312f736-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.453 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:58.454 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:58.454 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:58.455 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:58.456 DEBUG:boto:Method: GET 2022-09-03T06:41:58.456 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:41:58.457 DEBUG:boto:Data: 2022-09-03T06:41:58.458 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.458 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.459 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.460 DEBUG:boto:Params: {} 2022-09-03T06:41:58.460 DEBUG:boto:Token: None 2022-09-03T06:41:58.462 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/ 2022-09-03T06:41:58.463 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA= 2022-09-03T06:41:58.466 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA=', 'Content-Length': '0'} 2022-09-03T06:41:58.470 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c9b71df20344dd59-006312f736-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.471 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:58.473 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:58.474 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:58.474 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:58.475 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:58.475 DEBUG:boto:Method: GET 2022-09-03T06:41:58.476 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:41:58.477 DEBUG:boto:Data: 2022-09-03T06:41:58.477 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.478 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.479 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.481 DEBUG:boto:Params: {} 2022-09-03T06:41:58.482 DEBUG:boto:Token: None 2022-09-03T06:41:58.483 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/ 2022-09-03T06:41:58.484 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA= 2022-09-03T06:41:58.486 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA=', 'Content-Length': '0'} 2022-09-03T06:41:58.490 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001086b2f0b4811d9e-006312f736-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.491 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:58.493 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:58.494 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:58.495 DEBUG:boto:Method: HEAD 2022-09-03T06:41:58.496 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:41:58.497 DEBUG:boto:Data: 2022-09-03T06:41:58.499 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.500 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.501 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.502 DEBUG:boto:Params: {} 2022-09-03T06:41:58.503 DEBUG:boto:Token: None 2022-09-03T06:41:58.504 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj76 2022-09-03T06:41:58.504 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w5Yu/5tqBMLuZMbTvYL0L+7HqUU= 2022-09-03T06:41:58.505 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w5Yu/5tqBMLuZMbTvYL0L+7HqUU=', 'Content-Length': '0'} 2022-09-03T06:41:58.508 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003d3a1ef2f0a40d57-006312f736-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.508 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:58.509 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:58.510 DEBUG:boto:Method: HEAD 2022-09-03T06:41:58.511 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:41:58.512 DEBUG:boto:Data: 2022-09-03T06:41:58.513 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.514 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.515 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.516 DEBUG:boto:Params: {} 2022-09-03T06:41:58.517 DEBUG:boto:Token: None 2022-09-03T06:41:58.519 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj76 2022-09-03T06:41:58.520 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w5Yu/5tqBMLuZMbTvYL0L+7HqUU= 2022-09-03T06:41:58.521 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w5Yu/5tqBMLuZMbTvYL0L+7HqUU=', 'Content-Length': '0'} 2022-09-03T06:41:58.526 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a90a3bf399a5a977-006312f736-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.527 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:41:58.529 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:58.531 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:58.532 DEBUG:boto:Method: GET 2022-09-03T06:41:58.533 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:41:58.535 DEBUG:boto:Data: 2022-09-03T06:41:58.536 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.537 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.538 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.540 DEBUG:boto:Params: {} 2022-09-03T06:41:58.541 DEBUG:boto:Token: None 2022-09-03T06:41:58.542 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj76 2022-09-03T06:41:58.543 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NVymui8wVOkKFZQqvjbkgm2zZ3w= 2022-09-03T06:41:58.543 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NVymui8wVOkKFZQqvjbkgm2zZ3w=', 'Content-Length': '0'} 2022-09-03T06:41:58.546 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b6449483c7d5424e-006312f736-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.547 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:41:58.547 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:41:58.548 DEBUG:boto:Method: GET 2022-09-03T06:41:58.548 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:41:58.549 DEBUG:boto:Data: 2022-09-03T06:41:58.550 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.550 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.551 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.551 DEBUG:boto:Params: {} 2022-09-03T06:41:58.552 DEBUG:boto:Token: None 2022-09-03T06:41:58.553 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj76 2022-09-03T06:41:58.553 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NVymui8wVOkKFZQqvjbkgm2zZ3w= 2022-09-03T06:41:58.554 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NVymui8wVOkKFZQqvjbkgm2zZ3w=', 'Content-Length': '0'} 2022-09-03T06:41:58.556 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ab2f6207badee972-006312f736-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.557 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:58.557 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:58.558 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:58.559 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:58.559 DEBUG:boto:Method: GET 2022-09-03T06:41:58.560 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:41:58.560 DEBUG:boto:Data: 2022-09-03T06:41:58.561 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.561 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.562 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.562 DEBUG:boto:Params: {} 2022-09-03T06:41:58.563 DEBUG:boto:Token: None 2022-09-03T06:41:58.564 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/ 2022-09-03T06:41:58.565 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA= 2022-09-03T06:41:58.565 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA=', 'Content-Length': '0'} 2022-09-03T06:41:58.568 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001c2c51ead2bb4d7a-006312f736-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.569 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:58.569 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:58.570 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:58.571 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:58.571 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:41:58.572 DEBUG:boto:Method: GET 2022-09-03T06:41:58.572 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:41:58.574 DEBUG:boto:Data: 2022-09-03T06:41:58.574 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.575 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.576 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.576 DEBUG:boto:Params: {} 2022-09-03T06:41:58.577 DEBUG:boto:Token: None 2022-09-03T06:41:58.578 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/ 2022-09-03T06:41:58.579 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA= 2022-09-03T06:41:58.579 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA=', 'Content-Length': '0'} 2022-09-03T06:41:58.584 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e74ed8c8b5bd02da-006312f736-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.586 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:58.587 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:41:58.587 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:58.588 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:58.589 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.590 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.590 DEBUG:boto:Method: GET 2022-09-03T06:41:58.591 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.591 DEBUG:boto:Data: 2022-09-03T06:41:58.592 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.592 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.593 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.594 DEBUG:boto:Params: {} 2022-09-03T06:41:58.595 DEBUG:boto:Token: None 2022-09-03T06:41:58.595 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.596 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q9OlCYXBH5NzXKSPFoYxu+v6lik= 2022-09-03T06:41:58.597 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q9OlCYXBH5NzXKSPFoYxu+v6lik=', 'Content-Length': '0'} 2022-09-03T06:41:58.599 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bf5378e48500f2a3-006312f736-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.600 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:58.600 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:58.601 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.601 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.602 DEBUG:boto:Method: GET 2022-09-03T06:41:58.603 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.603 DEBUG:boto:Data: 2022-09-03T06:41:58.604 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.605 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.605 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.606 DEBUG:boto:Params: {} 2022-09-03T06:41:58.607 DEBUG:boto:Token: None 2022-09-03T06:41:58.607 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.608 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q9OlCYXBH5NzXKSPFoYxu+v6lik= 2022-09-03T06:41:58.609 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q9OlCYXBH5NzXKSPFoYxu+v6lik=', 'Content-Length': '0'} 2022-09-03T06:41:58.612 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ece0f857982054e7-006312f736-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.654 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:58.655 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:58.656 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.657 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.658 DEBUG:boto:Method: HEAD 2022-09-03T06:41:58.658 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.659 DEBUG:boto:Data: 2022-09-03T06:41:58.660 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.661 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.661 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.662 DEBUG:boto:Params: {} 2022-09-03T06:41:58.663 DEBUG:boto:Token: None 2022-09-03T06:41:58.663 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.664 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ovG+aXPDbEf+w+gZ9TEhFBK5wEI= 2022-09-03T06:41:58.665 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ovG+aXPDbEf+w+gZ9TEhFBK5wEI=', 'Content-Length': '0'} 2022-09-03T06:41:58.667 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000e79fef41cbadf07-006312f736-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.668 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:58.669 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:58.670 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.672 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.672 DEBUG:boto:Method: HEAD 2022-09-03T06:41:58.673 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.674 DEBUG:boto:Data: 2022-09-03T06:41:58.675 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.676 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.677 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.678 DEBUG:boto:Params: {} 2022-09-03T06:41:58.678 DEBUG:boto:Token: None 2022-09-03T06:41:58.679 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.680 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ovG+aXPDbEf+w+gZ9TEhFBK5wEI= 2022-09-03T06:41:58.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ovG+aXPDbEf+w+gZ9TEhFBK5wEI=', 'Content-Length': '0'} 2022-09-03T06:41:58.683 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007c9edefbd20930ea-006312f736-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.684 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:41:58.684 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:58.685 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:58.685 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.686 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.686 DEBUG:boto:Method: GET 2022-09-03T06:41:58.687 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.688 DEBUG:boto:Data: 2022-09-03T06:41:58.689 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.690 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.691 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.692 DEBUG:boto:Params: {} 2022-09-03T06:41:58.693 DEBUG:boto:Token: None 2022-09-03T06:41:58.693 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.694 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q9OlCYXBH5NzXKSPFoYxu+v6lik= 2022-09-03T06:41:58.695 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q9OlCYXBH5NzXKSPFoYxu+v6lik=', 'Content-Length': '0'} 2022-09-03T06:41:58.697 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004819baa601f43689-006312f736-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.698 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:58.699 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:58.700 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.701 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.702 DEBUG:boto:Method: GET 2022-09-03T06:41:58.703 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.704 DEBUG:boto:Data: 2022-09-03T06:41:58.705 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.706 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.707 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.717 DEBUG:boto:Params: {} 2022-09-03T06:41:58.718 DEBUG:boto:Token: None 2022-09-03T06:41:58.719 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:41:58.720 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q9OlCYXBH5NzXKSPFoYxu+v6lik= 2022-09-03T06:41:58.721 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q9OlCYXBH5NzXKSPFoYxu+v6lik=', 'Content-Length': '0'} 2022-09-03T06:41:58.724 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000050d7c3fe47ab3c96-006312f736-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.725 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:58.726 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:58.727 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:58.727 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:58.728 DEBUG:boto:Method: GET 2022-09-03T06:41:58.728 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:41:58.729 DEBUG:boto:Data: 2022-09-03T06:41:58.730 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.731 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.732 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.733 DEBUG:boto:Params: {} 2022-09-03T06:41:58.734 DEBUG:boto:Token: None 2022-09-03T06:41:58.735 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/ 2022-09-03T06:41:58.735 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA= 2022-09-03T06:41:58.736 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA=', 'Content-Length': '0'} 2022-09-03T06:41:58.738 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e48c91d37f451e34-006312f736-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.739 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:58.740 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:58.740 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:58.741 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:58.741 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:58.742 DEBUG:boto:Method: GET 2022-09-03T06:41:58.743 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:41:58.743 DEBUG:boto:Data: 2022-09-03T06:41:58.744 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.744 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.745 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.745 DEBUG:boto:Params: {} 2022-09-03T06:41:58.746 DEBUG:boto:Token: None 2022-09-03T06:41:58.747 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/ 2022-09-03T06:41:58.747 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA= 2022-09-03T06:41:58.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA=', 'Content-Length': '0'} 2022-09-03T06:41:58.753 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005e102430f330f03e-006312f736-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.753 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:58.754 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:58.755 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:58.755 DEBUG:boto:Method: HEAD 2022-09-03T06:41:58.756 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:41:58.756 DEBUG:boto:Data: 2022-09-03T06:41:58.757 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.758 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.758 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.759 DEBUG:boto:Params: {} 2022-09-03T06:41:58.759 DEBUG:boto:Token: None 2022-09-03T06:41:58.760 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj85 2022-09-03T06:41:58.760 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UKYBemOa5cWuUNxHUR2aVSiGXhs= 2022-09-03T06:41:58.761 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UKYBemOa5cWuUNxHUR2aVSiGXhs=', 'Content-Length': '0'} 2022-09-03T06:41:58.764 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b037f5c512f39be3-006312f736-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.765 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:58.766 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:58.767 DEBUG:boto:Method: HEAD 2022-09-03T06:41:58.768 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:41:58.769 DEBUG:boto:Data: 2022-09-03T06:41:58.770 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.771 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.772 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.773 DEBUG:boto:Params: {} 2022-09-03T06:41:58.774 DEBUG:boto:Token: None 2022-09-03T06:41:58.775 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj85 2022-09-03T06:41:58.776 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UKYBemOa5cWuUNxHUR2aVSiGXhs= 2022-09-03T06:41:58.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UKYBemOa5cWuUNxHUR2aVSiGXhs=', 'Content-Length': '0'} 2022-09-03T06:41:58.784 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000085602ab4c3a90a06-006312f736-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.786 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:41:58.787 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:58.788 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:58.789 DEBUG:boto:Method: GET 2022-09-03T06:41:58.790 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:41:58.799 DEBUG:boto:Data: 2022-09-03T06:41:58.801 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.802 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.803 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.804 DEBUG:boto:Params: {} 2022-09-03T06:41:58.805 DEBUG:boto:Token: None 2022-09-03T06:41:58.806 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj85 2022-09-03T06:41:58.807 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:swtBs/sUt2UjLhtZSXQIYtfbQVg= 2022-09-03T06:41:58.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:swtBs/sUt2UjLhtZSXQIYtfbQVg=', 'Content-Length': '0'} 2022-09-03T06:41:58.815 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bb978c28468a0ad1-006312f736-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.817 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:41:58.818 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:41:58.819 DEBUG:boto:Method: GET 2022-09-03T06:41:58.827 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:41:58.828 DEBUG:boto:Data: 2022-09-03T06:41:58.828 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.829 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.829 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.830 DEBUG:boto:Params: {} 2022-09-03T06:41:58.831 DEBUG:boto:Token: None 2022-09-03T06:41:58.832 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj85 2022-09-03T06:41:58.832 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:swtBs/sUt2UjLhtZSXQIYtfbQVg= 2022-09-03T06:41:58.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:swtBs/sUt2UjLhtZSXQIYtfbQVg=', 'Content-Length': '0'} 2022-09-03T06:41:58.841 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096a690f448556a3f-006312f736-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.842 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:58.843 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:58.843 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:58.844 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:58.845 DEBUG:boto:Method: GET 2022-09-03T06:41:58.845 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:41:58.846 DEBUG:boto:Data: 2022-09-03T06:41:58.846 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.848 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.848 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.849 DEBUG:boto:Params: {} 2022-09-03T06:41:58.850 DEBUG:boto:Token: None 2022-09-03T06:41:58.850 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/ 2022-09-03T06:41:58.851 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA= 2022-09-03T06:41:58.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA=', 'Content-Length': '0'} 2022-09-03T06:41:58.856 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000af240bf9128ea1d0-006312f736-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.858 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:58.861 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:58.862 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:58.863 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:58.864 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:41:58.866 DEBUG:boto:Method: GET 2022-09-03T06:41:58.867 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:41:58.868 DEBUG:boto:Data: 2022-09-03T06:41:58.870 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.871 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.872 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.874 DEBUG:boto:Params: {} 2022-09-03T06:41:58.875 DEBUG:boto:Token: None 2022-09-03T06:41:58.876 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/ 2022-09-03T06:41:58.877 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA= 2022-09-03T06:41:58.878 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v3wKtdOqeLsNa3/0VOwtzkbdPOA=', 'Content-Length': '0'} 2022-09-03T06:41:58.882 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001adb596582272509-006312f736-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.884 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:58.885 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:41:58.887 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:58.888 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:58.889 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:58.890 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:58.891 DEBUG:boto:Method: GET 2022-09-03T06:41:58.892 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:58.898 DEBUG:boto:Data: 2022-09-03T06:41:58.899 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.900 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.901 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.902 DEBUG:boto:Params: {} 2022-09-03T06:41:58.903 DEBUG:boto:Token: None 2022-09-03T06:41:58.904 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:58.905 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bE/zgtQZnvG2xPmvB//h2MKXYYY= 2022-09-03T06:41:58.907 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bE/zgtQZnvG2xPmvB//h2MKXYYY=', 'Content-Length': '0'} 2022-09-03T06:41:58.913 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074cdc20f0dc98563-006312f736-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.914 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:58.915 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:58.916 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:58.917 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:58.918 DEBUG:boto:Method: GET 2022-09-03T06:41:58.919 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:58.920 DEBUG:boto:Data: 2022-09-03T06:41:58.921 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.922 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.923 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.924 DEBUG:boto:Params: {} 2022-09-03T06:41:58.925 DEBUG:boto:Token: None 2022-09-03T06:41:58.926 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:58.927 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bE/zgtQZnvG2xPmvB//h2MKXYYY= 2022-09-03T06:41:58.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bE/zgtQZnvG2xPmvB//h2MKXYYY=', 'Content-Length': '0'} 2022-09-03T06:41:58.931 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003d9a8a37e3675eac-006312f736-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.932 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:58.933 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:58.934 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:58.935 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:58.935 DEBUG:boto:Method: HEAD 2022-09-03T06:41:58.936 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:58.937 DEBUG:boto:Data: 2022-09-03T06:41:58.938 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.939 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.939 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.940 DEBUG:boto:Params: {} 2022-09-03T06:41:58.941 DEBUG:boto:Token: None 2022-09-03T06:41:58.941 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:58.942 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w3bFAg4jiJyWVpaA0PfqEcBxlGc= 2022-09-03T06:41:58.943 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w3bFAg4jiJyWVpaA0PfqEcBxlGc=', 'Content-Length': '0'} 2022-09-03T06:41:58.947 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a036d8c160c81c50-006312f736-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.948 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:58.949 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:58.949 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:58.950 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:58.950 DEBUG:boto:Method: HEAD 2022-09-03T06:41:58.951 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:58.953 DEBUG:boto:Data: 2022-09-03T06:41:58.954 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.955 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:58.955 DEBUG:boto:Port: 8000 2022-09-03T06:41:58.956 DEBUG:boto:Params: {} 2022-09-03T06:41:58.957 DEBUG:boto:Token: None 2022-09-03T06:41:58.957 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:58.960 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w3bFAg4jiJyWVpaA0PfqEcBxlGc= 2022-09-03T06:41:58.960 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w3bFAg4jiJyWVpaA0PfqEcBxlGc=', 'Content-Length': '0'} 2022-09-03T06:41:58.962 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c4ff1eeb7e94cd21-006312f736-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.964 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:41:58.966 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:58.967 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:58.967 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:58.969 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:58.970 DEBUG:boto:Method: GET 2022-09-03T06:41:58.971 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:58.972 DEBUG:boto:Data: 2022-09-03T06:41:58.974 DEBUG:boto:Headers: {} 2022-09-03T06:41:58.975 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:58.976 DEBUG:boto:Port: 8001 2022-09-03T06:41:58.977 DEBUG:boto:Params: {} 2022-09-03T06:41:58.978 DEBUG:boto:Token: None 2022-09-03T06:41:58.979 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:58 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:58.980 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bE/zgtQZnvG2xPmvB//h2MKXYYY= 2022-09-03T06:41:58.990 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:58 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bE/zgtQZnvG2xPmvB//h2MKXYYY=', 'Content-Length': '0'} 2022-09-03T06:41:58.994 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004b6fb50777ee7ea4-006312f736-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:58 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:58.995 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:58.996 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:58.996 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:58.997 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:58.997 DEBUG:boto:Method: GET 2022-09-03T06:41:58.998 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:58.999 DEBUG:boto:Data: 2022-09-03T06:41:58.999 DEBUG:boto:Headers: {} 2022-09-03T06:41:59.000 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:59.000 DEBUG:boto:Port: 8000 2022-09-03T06:41:59.001 DEBUG:boto:Params: {} 2022-09-03T06:41:59.001 DEBUG:boto:Token: None 2022-09-03T06:41:59.002 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:59 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:41:59.003 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:drOeiKZrZ/HowWFXknKmkAN7Lh4= 2022-09-03T06:41:59.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:drOeiKZrZ/HowWFXknKmkAN7Lh4=', 'Content-Length': '0'} 2022-09-03T06:41:59.006 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f63968f08d4f35b7-006312f737-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:59.006 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:59.007 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:59.008 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:59.008 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:59.009 DEBUG:boto:Method: GET 2022-09-03T06:41:59.009 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:41:59.010 DEBUG:boto:Data: 2022-09-03T06:41:59.011 DEBUG:boto:Headers: {} 2022-09-03T06:41:59.011 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:59.012 DEBUG:boto:Port: 8001 2022-09-03T06:41:59.012 DEBUG:boto:Params: {} 2022-09-03T06:41:59.013 DEBUG:boto:Token: None 2022-09-03T06:41:59.013 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:59 GMT /dsgvcy-75/ 2022-09-03T06:41:59.014 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:egqFJBkgJyA7K4E0EJjFzfmmS/o= 2022-09-03T06:41:59.015 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:egqFJBkgJyA7K4E0EJjFzfmmS/o=', 'Content-Length': '0'} 2022-09-03T06:41:59.020 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003a4f0fffd00b5f1b-006312f737-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:59.021 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:59.022 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:59.023 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:59.024 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:59.024 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:59.025 DEBUG:boto:Method: GET 2022-09-03T06:41:59.025 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:41:59.032 DEBUG:boto:Data: 2022-09-03T06:41:59.034 DEBUG:boto:Headers: {} 2022-09-03T06:41:59.036 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:59.037 DEBUG:boto:Port: 8000 2022-09-03T06:41:59.038 DEBUG:boto:Params: {} 2022-09-03T06:41:59.079 DEBUG:boto:Token: None 2022-09-03T06:41:59.080 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:59 GMT /dsgvcy-75/ 2022-09-03T06:41:59.081 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:egqFJBkgJyA7K4E0EJjFzfmmS/o= 2022-09-03T06:41:59.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:egqFJBkgJyA7K4E0EJjFzfmmS/o=', 'Content-Length': '0'} 2022-09-03T06:41:59.109 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d97952997259dc6e-006312f737-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:59.110 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:59.120 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:59.121 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:59.122 DEBUG:boto:Method: HEAD 2022-09-03T06:41:59.122 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:41:59.123 DEBUG:boto:Data: 2022-09-03T06:41:59.124 DEBUG:boto:Headers: {} 2022-09-03T06:41:59.124 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:59.126 DEBUG:boto:Port: 8001 2022-09-03T06:41:59.127 DEBUG:boto:Params: {} 2022-09-03T06:41:59.128 DEBUG:boto:Token: None 2022-09-03T06:41:59.129 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:59 GMT /dsgvcy-75/obj95 2022-09-03T06:41:59.129 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fxTFEXQp0Wb0i/X5J/s81vQz/ls= 2022-09-03T06:41:59.131 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fxTFEXQp0Wb0i/X5J/s81vQz/ls=', 'Content-Length': '0'} 2022-09-03T06:41:59.133 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002b1a71ff7dfb61dd-006312f737-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:59.134 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:59.138 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:59.139 DEBUG:boto:Method: HEAD 2022-09-03T06:41:59.140 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:41:59.141 DEBUG:boto:Data: 2022-09-03T06:41:59.151 DEBUG:boto:Headers: {} 2022-09-03T06:41:59.152 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:59.153 DEBUG:boto:Port: 8000 2022-09-03T06:41:59.154 DEBUG:boto:Params: {} 2022-09-03T06:41:59.155 DEBUG:boto:Token: None 2022-09-03T06:41:59.156 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:59 GMT /dsgvcy-75/obj95 2022-09-03T06:41:59.157 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fxTFEXQp0Wb0i/X5J/s81vQz/ls= 2022-09-03T06:41:59.158 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fxTFEXQp0Wb0i/X5J/s81vQz/ls=', 'Content-Length': '0'} 2022-09-03T06:41:59.175 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000968794cee45956c8-006312f737-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:59.176 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:41:59.177 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:59.178 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:59.180 DEBUG:boto:Method: GET 2022-09-03T06:41:59.187 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:41:59.188 DEBUG:boto:Data: 2022-09-03T06:41:59.188 DEBUG:boto:Headers: {} 2022-09-03T06:41:59.189 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:59.190 DEBUG:boto:Port: 8001 2022-09-03T06:41:59.192 DEBUG:boto:Params: {} 2022-09-03T06:41:59.193 DEBUG:boto:Token: None 2022-09-03T06:41:59.203 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:59 GMT /dsgvcy-75/obj95 2022-09-03T06:41:59.204 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q4ZrLvFkJ9Gln/80i+2RIHrYx38= 2022-09-03T06:41:59.206 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q4ZrLvFkJ9Gln/80i+2RIHrYx38=', 'Content-Length': '0'} 2022-09-03T06:41:59.225 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b555502512ca0f9-006312f737-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:59.236 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:41:59.239 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:41:59.242 DEBUG:boto:Method: GET 2022-09-03T06:41:59.244 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:41:59.253 DEBUG:boto:Data: 2022-09-03T06:41:59.255 DEBUG:boto:Headers: {} 2022-09-03T06:41:59.257 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:59.258 DEBUG:boto:Port: 8000 2022-09-03T06:41:59.258 DEBUG:boto:Params: {} 2022-09-03T06:41:59.259 DEBUG:boto:Token: None 2022-09-03T06:41:59.260 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:59 GMT /dsgvcy-75/obj95 2022-09-03T06:41:59.260 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q4ZrLvFkJ9Gln/80i+2RIHrYx38= 2022-09-03T06:41:59.261 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q4ZrLvFkJ9Gln/80i+2RIHrYx38=', 'Content-Length': '0'} 2022-09-03T06:41:59.264 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003f0a38d8740bfc0a-006312f737-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:59.265 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:59.267 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:59.269 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:59.270 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:59.271 DEBUG:boto:Method: GET 2022-09-03T06:41:59.272 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:41:59.273 DEBUG:boto:Data: 2022-09-03T06:41:59.274 DEBUG:boto:Headers: {} 2022-09-03T06:41:59.275 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:41:59.276 DEBUG:boto:Port: 8001 2022-09-03T06:41:59.277 DEBUG:boto:Params: {} 2022-09-03T06:41:59.279 DEBUG:boto:Token: None 2022-09-03T06:41:59.280 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:59 GMT /dsgvcy-75/ 2022-09-03T06:41:59.281 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:egqFJBkgJyA7K4E0EJjFzfmmS/o= 2022-09-03T06:41:59.282 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:egqFJBkgJyA7K4E0EJjFzfmmS/o=', 'Content-Length': '0'} 2022-09-03T06:41:59.293 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ab208fb720f6ec9e-006312f737-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:59.304 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:59.314 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:59.319 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:59.322 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:59.331 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:41:59.333 DEBUG:boto:Method: GET 2022-09-03T06:41:59.334 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:41:59.335 DEBUG:boto:Data: 2022-09-03T06:41:59.336 DEBUG:boto:Headers: {} 2022-09-03T06:41:59.337 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:41:59.338 DEBUG:boto:Port: 8000 2022-09-03T06:41:59.339 DEBUG:boto:Params: {} 2022-09-03T06:41:59.340 DEBUG:boto:Token: None 2022-09-03T06:41:59.341 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:41:59 GMT /dsgvcy-75/ 2022-09-03T06:41:59.342 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:egqFJBkgJyA7K4E0EJjFzfmmS/o= 2022-09-03T06:41:59.343 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:egqFJBkgJyA7K4E0EJjFzfmmS/o=', 'Content-Length': '0'} 2022-09-03T06:41:59.348 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f84bb61784147a1d-006312f737-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:59.354 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:41:59.357 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-75 zones={test-zone3, test-zone2} 2022-09-03T06:41:59.358 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--num-shards', '23', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:41:59.359 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--num-shards', '23', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:41:59.361 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --num-shards 23 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T06:41:59.423 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:41:59.424 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:41:59.489 INFO:teuthology.orchestra.run.smithi181.stdout:tenant: 2022-09-03T06:41:59.491 INFO:teuthology.orchestra.run.smithi181.stdout:bucket name: dsgvcy-75 2022-09-03T06:41:59.501 INFO:teuthology.orchestra.run.smithi181.stdout:total entries: 15 2022-09-03T06:41:59.561 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:59.562 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:59.565 DEBUG:boto:Method: HEAD 2022-09-03T06:41:59.568 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:41:59.571 DEBUG:boto:Data: 2022-09-03T06:41:59.572 DEBUG:boto:Headers: {} 2022-09-03T06:41:59.573 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:59.574 DEBUG:boto:Port: 8000 2022-09-03T06:41:59.575 DEBUG:boto:Params: {} 2022-09-03T06:41:59.578 DEBUG:boto:Token: None 2022-09-03T06:41:59.591 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:59 GMT /dsgvcy-75/ 2022-09-03T06:41:59.591 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t9T9U1+zz6kPjTQvOar0kWEVNCQ= 2022-09-03T06:41:59.592 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t9T9U1+zz6kPjTQvOar0kWEVNCQ=', 'Content-Length': '0'} 2022-09-03T06:41:59.614 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '15'), ('X-RGW-Bytes-Used', '45'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000009ba7b20e4e994ee5-006312f737-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:59.616 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:41:59.617 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:41:59.619 DEBUG:boto:Method: PUT 2022-09-03T06:41:59.620 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:41:59.621 DEBUG:boto:Data: 2022-09-03T06:41:59.623 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:41:59.625 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:59.626 DEBUG:boto:Port: 8000 2022-09-03T06:41:59.627 DEBUG:boto:Params: {} 2022-09-03T06:41:59.628 DEBUG:boto:Token: None 2022-09-03T06:41:59.629 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:41:59 GMT /dsgvcy-75/obj23 2022-09-03T06:41:59.629 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tIPg0OdSfBFlMYNJ5Xv4jZDKa6k= 2022-09-03T06:41:59.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:41:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tIPg0OdSfBFlMYNJ5Xv4jZDKa6k='} 2022-09-03T06:41:59.638 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000007f7d610702e6adaf-006312f737-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:59.639 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:59.640 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:59.640 DEBUG:boto:Method: HEAD 2022-09-03T06:41:59.641 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:41:59.641 DEBUG:boto:Data: 2022-09-03T06:41:59.643 DEBUG:boto:Headers: {} 2022-09-03T06:41:59.645 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:59.647 DEBUG:boto:Port: 8000 2022-09-03T06:41:59.647 DEBUG:boto:Params: {} 2022-09-03T06:41:59.648 DEBUG:boto:Token: None 2022-09-03T06:41:59.649 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:59 GMT /dsgvcy-75/ 2022-09-03T06:41:59.650 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t9T9U1+zz6kPjTQvOar0kWEVNCQ= 2022-09-03T06:41:59.651 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t9T9U1+zz6kPjTQvOar0kWEVNCQ=', 'Content-Length': '0'} 2022-09-03T06:41:59.655 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '16'), ('X-RGW-Bytes-Used', '48'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000388f197213636e43-006312f737-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:59.656 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:41:59.657 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:41:59.657 DEBUG:boto:Method: PUT 2022-09-03T06:41:59.658 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:41:59.659 DEBUG:boto:Data: 2022-09-03T06:41:59.659 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:41:59.660 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:59.660 DEBUG:boto:Port: 8000 2022-09-03T06:41:59.661 DEBUG:boto:Params: {} 2022-09-03T06:41:59.661 DEBUG:boto:Token: None 2022-09-03T06:41:59.664 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:41:59 GMT /dsgvcy-75/obj46 2022-09-03T06:41:59.666 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UfxNJe6S7ce018BOwKn5UCOhWgU= 2022-09-03T06:41:59.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:41:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UfxNJe6S7ce018BOwKn5UCOhWgU='} 2022-09-03T06:41:59.734 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000571c3fcfa2d3006d-006312f737-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:59.739 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:59.740 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:59.741 DEBUG:boto:Method: HEAD 2022-09-03T06:41:59.742 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:41:59.743 DEBUG:boto:Data: 2022-09-03T06:41:59.744 DEBUG:boto:Headers: {} 2022-09-03T06:41:59.756 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:59.758 DEBUG:boto:Port: 8000 2022-09-03T06:41:59.785 DEBUG:boto:Params: {} 2022-09-03T06:41:59.787 DEBUG:boto:Token: None 2022-09-03T06:41:59.788 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:59 GMT /dsgvcy-75/ 2022-09-03T06:41:59.789 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t9T9U1+zz6kPjTQvOar0kWEVNCQ= 2022-09-03T06:41:59.791 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t9T9U1+zz6kPjTQvOar0kWEVNCQ=', 'Content-Length': '0'} 2022-09-03T06:41:59.800 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '17'), ('X-RGW-Bytes-Used', '51'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000004e1f30c50afa3eac-006312f737-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:59.801 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:41:59.803 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:41:59.804 DEBUG:boto:Method: PUT 2022-09-03T06:41:59.805 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:41:59.805 DEBUG:boto:Data: 2022-09-03T06:41:59.806 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:41:59.807 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:59.807 DEBUG:boto:Port: 8000 2022-09-03T06:41:59.808 DEBUG:boto:Params: {} 2022-09-03T06:41:59.809 DEBUG:boto:Token: None 2022-09-03T06:41:59.809 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:41:59 GMT /dsgvcy-75/obj69 2022-09-03T06:41:59.814 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+s4XiwxD8pTeWwVXQZHuxd16x9U= 2022-09-03T06:41:59.816 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:41:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+s4XiwxD8pTeWwVXQZHuxd16x9U='} 2022-09-03T06:41:59.826 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003943af24b0314dbb-006312f737-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:59.837 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:59.838 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:59.840 DEBUG:boto:Method: HEAD 2022-09-03T06:41:59.842 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:41:59.843 DEBUG:boto:Data: 2022-09-03T06:41:59.844 DEBUG:boto:Headers: {} 2022-09-03T06:41:59.849 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:59.851 DEBUG:boto:Port: 8000 2022-09-03T06:41:59.862 DEBUG:boto:Params: {} 2022-09-03T06:41:59.863 DEBUG:boto:Token: None 2022-09-03T06:41:59.864 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:59 GMT /dsgvcy-75/ 2022-09-03T06:41:59.865 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t9T9U1+zz6kPjTQvOar0kWEVNCQ= 2022-09-03T06:41:59.866 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t9T9U1+zz6kPjTQvOar0kWEVNCQ=', 'Content-Length': '0'} 2022-09-03T06:41:59.875 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '18'), ('X-RGW-Bytes-Used', '54'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000045a365a43e3f9b6e-006312f737-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:59.877 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:41:59.878 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:41:59.879 DEBUG:boto:Method: PUT 2022-09-03T06:41:59.880 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:41:59.881 DEBUG:boto:Data: 2022-09-03T06:41:59.882 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:41:59.883 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:59.883 DEBUG:boto:Port: 8000 2022-09-03T06:41:59.884 DEBUG:boto:Params: {} 2022-09-03T06:41:59.885 DEBUG:boto:Token: None 2022-09-03T06:41:59.887 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:41:59 GMT /dsgvcy-75/obj92 2022-09-03T06:41:59.888 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iiwqhEYm+PQhWFVFavyXoXGjQW0= 2022-09-03T06:41:59.889 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:41:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iiwqhEYm+PQhWFVFavyXoXGjQW0='} 2022-09-03T06:41:59.897 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003132fa074eaed26e-006312f737-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:59.898 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:41:59.899 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:41:59.900 DEBUG:boto:Method: HEAD 2022-09-03T06:41:59.900 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:41:59.901 DEBUG:boto:Data: 2022-09-03T06:41:59.901 DEBUG:boto:Headers: {} 2022-09-03T06:41:59.902 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:59.902 DEBUG:boto:Port: 8000 2022-09-03T06:41:59.903 DEBUG:boto:Params: {} 2022-09-03T06:41:59.904 DEBUG:boto:Token: None 2022-09-03T06:41:59.904 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:41:59 GMT /dsgvcy-75/ 2022-09-03T06:41:59.905 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t9T9U1+zz6kPjTQvOar0kWEVNCQ= 2022-09-03T06:41:59.906 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:41:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t9T9U1+zz6kPjTQvOar0kWEVNCQ=', 'Content-Length': '0'} 2022-09-03T06:41:59.909 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '19'), ('X-RGW-Bytes-Used', '57'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f7fbbbaa7fd04f49-006312f737-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:59.910 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:41:59.910 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:41:59.911 DEBUG:boto:Method: PUT 2022-09-03T06:41:59.912 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:41:59.912 DEBUG:boto:Data: 2022-09-03T06:41:59.913 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:41:59.913 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:41:59.914 DEBUG:boto:Port: 8000 2022-09-03T06:41:59.915 DEBUG:boto:Params: {} 2022-09-03T06:41:59.916 DEBUG:boto:Token: None 2022-09-03T06:41:59.916 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:41:59 GMT /dsgvcy-75/obj115 2022-09-03T06:41:59.917 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LL4wuB75yaOQ9NrYadsVqgM4wcM= 2022-09-03T06:41:59.919 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:41:59 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LL4wuB75yaOQ9NrYadsVqgM4wcM='} 2022-09-03T06:41:59.974 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000009a752a9a9aae0bc8-006312f737-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:41:59.977 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-75 2022-09-03T06:41:59.978 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:41:59.979 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:41:59.980 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:42:00.055 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:42:00.056 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:42:00.107 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:42:00.104+0000 7fa53a60a600 1 waiting to reach latest gen 3, on 2.. 2022-09-03T06:42:05.109 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:42:05.105+0000 7fa53a60a600 1 waiting to reach latest gen 3, on 2.. 2022-09-03T06:42:10.107 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:42:10.105+0000 7fa53a60a600 1 waiting to reach latest gen 3, on 2.. 2022-09-03T06:42:15.110 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:42:15.105+0000 7fa53a60a600 1 waiting to reach latest gen 3, on 2.. 2022-09-03T06:42:20.111 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:42:20.109+0000 7fa53a60a600 1 waiting for incremental sync to catch up: 2022-09-03T06:42:20.112 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , ] 2022-09-03T06:42:20.112 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:42:25.120 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:42:25.111+0000 7fa53a60a600 1 waiting for incremental sync to catch up: 2022-09-03T06:42:25.120 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , ] 2022-09-03T06:42:25.121 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:42:30.115 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:42:30.113+0000 7fa53a60a600 1 waiting for incremental sync to catch up: 2022-09-03T06:42:30.116 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , ] 2022-09-03T06:42:30.117 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:42:35.117 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:42:35.115+0000 7fa53a60a600 1 waiting for incremental sync to catch up: 2022-09-03T06:42:35.118 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , ] 2022-09-03T06:42:35.118 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:42:40.121 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:42:40.117+0000 7fa53a60a600 1 waiting for incremental sync to catch up: 2022-09-03T06:42:40.121 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000001.66.5, , , , , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:42:40.121 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:42:45.121 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:42:45.119+0000 7fa53a60a600 1 waiting for incremental sync to catch up: 2022-09-03T06:42:45.122 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000001.66.5, , , , , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:42:45.122 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:42:50.124 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:42:50.121+0000 7fa53a60a600 1 waiting for incremental sync to catch up: 2022-09-03T06:42:50.124 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000001.66.5, , , , , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:42:50.125 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:42:55.125 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:42:55.123+0000 7fa53a60a600 1 waiting for incremental sync to catch up: 2022-09-03T06:42:55.126 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000001.66.5, , , , , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:42:55.126 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:00.127 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:43:00.125+0000 7fa53a60a600 1 waiting for incremental sync to catch up: 2022-09-03T06:43:00.128 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000001.66.5, , , , , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:00.128 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:05.130 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:43:05.127+0000 7fa53a60a600 1 waiting for incremental sync to catch up: 2022-09-03T06:43:05.130 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000001.66.5, , , , , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:05.131 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:10.132 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:43:10.130+0000 7fa53a60a600 1 waiting for incremental sync to catch up: 2022-09-03T06:43:10.132 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000001.66.5, , , , , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:10.133 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:15.135 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:43:15.132+0000 7fa53a60a600 1 waiting for incremental sync to catch up: 2022-09-03T06:43:15.135 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000001.66.5, , , , , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:15.135 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:20.136 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:43:20.134+0000 7fa53a60a600 1 waiting for incremental sync to catch up: 2022-09-03T06:43:20.137 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000001.66.5, , , , , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:20.138 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:25.139 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:43:25.136+0000 7fa53a60a600 1 waiting for incremental sync to catch up: 2022-09-03T06:43:25.141 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000001.66.5, , , , , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:25.142 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:30.140 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:43:30.138+0000 7fa53a60a600 1 waiting for incremental sync to catch up: 2022-09-03T06:43:30.141 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000001.66.5, , , , , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:30.141 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:35.142 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:43:35.140+0000 7fa53a60a600 1 waiting for incremental sync to catch up: 2022-09-03T06:43:35.143 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000001.66.5, , , , , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:35.143 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:40.145 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:43:40.143+0000 7fa53a60a600 1 waiting for incremental sync to catch up: 2022-09-03T06:43:40.146 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000001.66.5, , , , , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:40.146 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:45.147 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:43:45.144+0000 7fa53a60a600 1 waiting for incremental sync to catch up: 2022-09-03T06:43:45.147 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000001.66.5, , , , , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:45.148 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:50.147 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:43:50.145+0000 7fa53a60a600 1 waiting for incremental sync to catch up: 2022-09-03T06:43:50.148 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000001.66.5, , , , , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:50.149 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:55.150 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:43:55.148+0000 7fa53a60a600 1 waiting for incremental sync to catch up: 2022-09-03T06:43:55.151 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000001.66.5, , , , , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:43:55.151 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:44:00.152 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:44:00.148+0000 7fa53a60a600 1 bucket sync caught up with source: 2022-09-03T06:44:00.152 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:44:00.153 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:44:00.153 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:44:00.148+0000 7fa53a60a600 0 bucket checkpoint complete 2022-09-03T06:44:00.171 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-75 2022-09-03T06:44:00.172 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:44:00.174 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:44:00.175 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:44:00.243 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:44:00.243 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:44:00.305 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:44:00.304+0000 7fea1c179600 1 bucket sync caught up with source: 2022-09-03T06:44:00.306 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:44:00.306 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:44:00.307 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:44:00.304+0000 7fea1c179600 0 bucket checkpoint complete 2022-09-03T06:44:00.330 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-75 2022-09-03T06:44:00.331 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:44:00.331 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:44:00.332 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:44:00.397 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:44:00.398 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:44:00.438 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:44:00.436+0000 7fb345bbb600 1 bucket sync caught up with source: 2022-09-03T06:44:00.439 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:44:00.440 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , 00000000001.66.5, , , 00000000002.75.5, , 00000000002.70.5, , , , , , 00000000001.77.5, , 00000000001.60.5, , , , ] 2022-09-03T06:44:00.440 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:44:00.436+0000 7fb345bbb600 0 bucket checkpoint complete 2022-09-03T06:44:00.460 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-75 2022-09-03T06:44:00.461 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:44:00.462 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:44:00.463 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:44:00.520 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:44:00.521 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:44:00.567 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:44:00.565+0000 7f43cceaf600 1 bucket sync caught up with source: 2022-09-03T06:44:00.568 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.582.5, 00000000002.739.5, 00000000001.366.5, 00000000002.707.5, 00000000002.372.5, , , 00000000002.855.5, 00000000005.312.5, 00000000004.587.5] 2022-09-03T06:44:00.568 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.582.5, 00000000002.739.5, 00000000001.366.5, 00000000002.707.5, 00000000002.372.5, , , 00000000002.855.5, 00000000005.312.5, 00000000004.587.5] 2022-09-03T06:44:00.569 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:44:00.565+0000 7f43cceaf600 0 bucket checkpoint complete 2022-09-03T06:44:00.582 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-75 2022-09-03T06:44:00.583 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:44:00.584 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:44:00.585 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:44:00.644 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:44:00.645 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:44:00.702 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:44:00.700+0000 7fd54cca4600 1 bucket sync caught up with source: 2022-09-03T06:44:00.702 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.582.5, 00000000002.739.5, 00000000001.366.5, 00000000002.707.5, 00000000002.372.5, , , 00000000002.855.5, 00000000005.312.5, 00000000004.587.5] 2022-09-03T06:44:00.702 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.582.5, 00000000002.739.5, 00000000001.366.5, 00000000002.707.5, 00000000002.372.5, , , 00000000002.855.5, 00000000005.312.5, 00000000004.587.5] 2022-09-03T06:44:00.703 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:44:00.700+0000 7fd54cca4600 0 bucket checkpoint complete 2022-09-03T06:44:00.716 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-75 2022-09-03T06:44:00.717 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-75 2022-09-03T06:44:00.718 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:44:00.719 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:44:00.719 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:44:00.776 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:44:00.776 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:44:00.811 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:44:00.809+0000 7fc27a6a7600 1 bucket sync caught up with source: 2022-09-03T06:44:00.811 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.202.5, 00000000003.158.5, 00000000001.271.5, 00000000002.268.5, 00000000002.277.5, , , 00000000002.317.5, 00000000005.247.5, 00000000004.290.5] 2022-09-03T06:44:00.811 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.202.5, 00000000003.158.5, 00000000001.271.5, 00000000002.268.5, 00000000002.277.5, , , 00000000002.317.5, 00000000005.247.5, 00000000004.290.5] 2022-09-03T06:44:00.812 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:44:00.809+0000 7fc27a6a7600 0 bucket checkpoint complete 2022-09-03T06:44:00.825 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-75 2022-09-03T06:44:00.826 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:44:00.827 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:44:00.829 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:44:00.887 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:44:00.888 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:44:00.926 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:44:00.923+0000 7fda61a64600 1 bucket sync caught up with source: 2022-09-03T06:44:00.927 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000002.202.5, 00000000003.158.5, 00000000001.271.5, 00000000002.268.5, 00000000002.277.5, , , 00000000002.317.5, 00000000005.247.5, 00000000004.290.5] 2022-09-03T06:44:00.927 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000002.202.5, 00000000003.158.5, 00000000001.271.5, 00000000002.268.5, 00000000002.277.5, , , 00000000002.317.5, 00000000005.247.5, 00000000004.290.5] 2022-09-03T06:44:00.927 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:44:00.923+0000 7fda61a64600 0 bucket checkpoint complete 2022-09-03T06:44:00.942 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-75 2022-09-03T06:44:00.944 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-75 zones={test-zone2, test-zone1} 2022-09-03T06:44:00.945 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:00.946 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:00.947 DEBUG:boto:Method: HEAD 2022-09-03T06:44:00.948 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:44:00.949 DEBUG:boto:Data: 2022-09-03T06:44:00.950 DEBUG:boto:Headers: {} 2022-09-03T06:44:00.951 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:00.952 DEBUG:boto:Port: 8000 2022-09-03T06:44:00.953 DEBUG:boto:Params: {} 2022-09-03T06:44:00.955 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:44:00.956 DEBUG:boto:Token: None 2022-09-03T06:44:00.957 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:00 GMT /dsgvcy-75/ 2022-09-03T06:44:00.958 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jX9QujkE4gxC5YlXJpT5joBjRRw= 2022-09-03T06:44:00.959 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jX9QujkE4gxC5YlXJpT5joBjRRw=', 'Content-Length': '0'} 2022-09-03T06:44:00.965 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '20'), ('X-RGW-Bytes-Used', '60'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ffd99b114f44a021-006312f7b0-188c-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:44:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:00.966 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:00.966 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:00.967 DEBUG:boto:Method: HEAD 2022-09-03T06:44:00.968 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:44:00.968 DEBUG:boto:Data: 2022-09-03T06:44:00.969 DEBUG:boto:Headers: {} 2022-09-03T06:44:00.969 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:00.970 DEBUG:boto:Port: 8000 2022-09-03T06:44:00.970 DEBUG:boto:Params: {} 2022-09-03T06:44:00.971 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:44:00.972 DEBUG:boto:Token: None 2022-09-03T06:44:00.972 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:00 GMT /dsgvcy-75/ 2022-09-03T06:44:00.973 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jX9QujkE4gxC5YlXJpT5joBjRRw= 2022-09-03T06:44:00.974 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jX9QujkE4gxC5YlXJpT5joBjRRw=', 'Content-Length': '0'} 2022-09-03T06:44:00.979 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '20'), ('X-RGW-Bytes-Used', '60'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b73bb4c984c827e2-006312f7b0-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:44:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:00.980 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:44:00.981 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:00.982 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:00.982 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:00.983 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:00.983 DEBUG:boto:Method: GET 2022-09-03T06:44:00.984 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:00.984 DEBUG:boto:Data: 2022-09-03T06:44:00.985 DEBUG:boto:Headers: {} 2022-09-03T06:44:00.985 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:00.986 DEBUG:boto:Port: 8000 2022-09-03T06:44:00.986 DEBUG:boto:Params: {} 2022-09-03T06:44:00.987 DEBUG:boto:Token: None 2022-09-03T06:44:00.989 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:00 GMT /dsgvcy-75/?versions 2022-09-03T06:44:00.990 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A+qxDp/FJsWIB8bsYl7lm9gACEo= 2022-09-03T06:44:00.991 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A+qxDp/FJsWIB8bsYl7lm9gACEo=', 'Content-Length': '0'} 2022-09-03T06:44:00.993 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000db266a4e018415d3-006312f7b0-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:00.995 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:01.000 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:44:01.001 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:44:01.002 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:44:01.002 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:44:01.003 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:44:01.004 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:44:01.004 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:44:01.005 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:44:01.006 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:44:01.007 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:44:01.008 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:44:01.008 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:44:01.009 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:44:01.010 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:44:01.011 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:44:01.012 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:44:01.013 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:44:01.014 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:44:01.015 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:44:01.015 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:44:01.016 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:44:01.017 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:01.017 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:01.018 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:01.018 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:01.019 DEBUG:boto:Method: GET 2022-09-03T06:44:01.020 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:01.020 DEBUG:boto:Data: 2022-09-03T06:44:01.022 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.023 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:01.024 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.026 DEBUG:boto:Params: {} 2022-09-03T06:44:01.027 DEBUG:boto:Token: None 2022-09-03T06:44:01.027 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/?versions 2022-09-03T06:44:01.028 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JBExptCq0DgvsRktEIpomBWAdD0= 2022-09-03T06:44:01.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JBExptCq0DgvsRktEIpomBWAdD0=', 'Content-Length': '0'} 2022-09-03T06:44:01.033 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d3e9c18d5fa0c542-006312f7b1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.034 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:01.037 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:44:01.038 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:44:01.039 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:44:01.040 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:44:01.041 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:44:01.047 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:44:01.048 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:44:01.049 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:44:01.050 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:44:01.051 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:44:01.052 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:44:01.053 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:44:01.054 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:44:01.055 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:44:01.055 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:44:01.056 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:44:01.057 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:44:01.058 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:44:01.059 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:44:01.059 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:44:01.060 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:01.061 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:01.062 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:01.062 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:01.063 DEBUG:boto:Method: GET 2022-09-03T06:44:01.064 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:01.065 DEBUG:boto:Data: 2022-09-03T06:44:01.066 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.067 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:01.068 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.069 DEBUG:boto:Params: {} 2022-09-03T06:44:01.071 DEBUG:boto:Token: None 2022-09-03T06:44:01.072 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/?versions 2022-09-03T06:44:01.073 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JBExptCq0DgvsRktEIpomBWAdD0= 2022-09-03T06:44:01.074 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JBExptCq0DgvsRktEIpomBWAdD0=', 'Content-Length': '0'} 2022-09-03T06:44:01.077 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008f2a27d474939f51-006312f7b1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.078 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:01.080 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:01.081 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:01.081 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:01.086 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:01.087 DEBUG:boto:Method: GET 2022-09-03T06:44:01.087 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:01.088 DEBUG:boto:Data: 2022-09-03T06:44:01.088 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.089 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:01.089 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.090 DEBUG:boto:Params: {} 2022-09-03T06:44:01.091 DEBUG:boto:Token: None 2022-09-03T06:44:01.091 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/?versions 2022-09-03T06:44:01.092 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JBExptCq0DgvsRktEIpomBWAdD0= 2022-09-03T06:44:01.093 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JBExptCq0DgvsRktEIpomBWAdD0=', 'Content-Length': '0'} 2022-09-03T06:44:01.099 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000030892c50b954b5d9-006312f7b1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.100 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:01.104 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:44:01.106 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:01.107 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:01.108 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.109 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.110 DEBUG:boto:Method: GET 2022-09-03T06:44:01.110 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.111 DEBUG:boto:Data: 2022-09-03T06:44:01.112 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.112 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:01.113 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.115 DEBUG:boto:Params: {} 2022-09-03T06:44:01.116 DEBUG:boto:Token: None 2022-09-03T06:44:01.117 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.118 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F+0BdSv6/6X8dyxMA6VUxNmGzAw= 2022-09-03T06:44:01.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F+0BdSv6/6X8dyxMA6VUxNmGzAw=', 'Content-Length': '0'} 2022-09-03T06:44:01.121 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e71140481ee52bf6-006312f7b1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.122 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:01.123 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:01.124 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.125 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.126 DEBUG:boto:Method: GET 2022-09-03T06:44:01.126 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.127 DEBUG:boto:Data: 2022-09-03T06:44:01.128 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.129 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:01.130 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.131 DEBUG:boto:Params: {} 2022-09-03T06:44:01.131 DEBUG:boto:Token: None 2022-09-03T06:44:01.132 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.133 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F+0BdSv6/6X8dyxMA6VUxNmGzAw= 2022-09-03T06:44:01.134 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F+0BdSv6/6X8dyxMA6VUxNmGzAw=', 'Content-Length': '0'} 2022-09-03T06:44:01.136 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000036c7c42f8f23bfb7-006312f7b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.137 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:01.138 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:01.138 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.139 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.139 DEBUG:boto:Method: HEAD 2022-09-03T06:44:01.140 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.140 DEBUG:boto:Data: 2022-09-03T06:44:01.141 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.141 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:01.142 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.142 DEBUG:boto:Params: {} 2022-09-03T06:44:01.143 DEBUG:boto:Token: None 2022-09-03T06:44:01.144 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.144 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:g6wUInf/LtH6d0iPtoEk+Z8lIvI= 2022-09-03T06:44:01.145 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:g6wUInf/LtH6d0iPtoEk+Z8lIvI=', 'Content-Length': '0'} 2022-09-03T06:44:01.147 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000024a524f999a90581-006312f7b1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.148 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:01.148 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:01.149 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.150 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.150 DEBUG:boto:Method: HEAD 2022-09-03T06:44:01.151 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.151 DEBUG:boto:Data: 2022-09-03T06:44:01.152 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.152 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:01.153 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.154 DEBUG:boto:Params: {} 2022-09-03T06:44:01.154 DEBUG:boto:Token: None 2022-09-03T06:44:01.155 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.156 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:g6wUInf/LtH6d0iPtoEk+Z8lIvI= 2022-09-03T06:44:01.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:g6wUInf/LtH6d0iPtoEk+Z8lIvI=', 'Content-Length': '0'} 2022-09-03T06:44:01.158 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007e983cb5ff1478b4-006312f7b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.159 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:44:01.159 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:01.160 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:01.160 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.161 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.161 DEBUG:boto:Method: GET 2022-09-03T06:44:01.162 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.164 DEBUG:boto:Data: 2022-09-03T06:44:01.165 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.165 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:01.166 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.167 DEBUG:boto:Params: {} 2022-09-03T06:44:01.167 DEBUG:boto:Token: None 2022-09-03T06:44:01.168 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.169 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F+0BdSv6/6X8dyxMA6VUxNmGzAw= 2022-09-03T06:44:01.170 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F+0BdSv6/6X8dyxMA6VUxNmGzAw=', 'Content-Length': '0'} 2022-09-03T06:44:01.173 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021578ee7f1706ae9-006312f7b1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.174 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:01.175 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:01.176 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.177 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.178 DEBUG:boto:Method: GET 2022-09-03T06:44:01.179 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.179 DEBUG:boto:Data: 2022-09-03T06:44:01.180 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.180 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:01.181 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.182 DEBUG:boto:Params: {} 2022-09-03T06:44:01.182 DEBUG:boto:Token: None 2022-09-03T06:44:01.183 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:01.183 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F+0BdSv6/6X8dyxMA6VUxNmGzAw= 2022-09-03T06:44:01.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F+0BdSv6/6X8dyxMA6VUxNmGzAw=', 'Content-Length': '0'} 2022-09-03T06:44:01.186 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003bd5a2a59dfcb6a8-006312f7b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.187 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:01.188 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:01.188 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:01.189 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:01.189 DEBUG:boto:Method: GET 2022-09-03T06:44:01.190 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:44:01.190 DEBUG:boto:Data: 2022-09-03T06:44:01.191 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.191 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:01.192 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.192 DEBUG:boto:Params: {} 2022-09-03T06:44:01.193 DEBUG:boto:Token: None 2022-09-03T06:44:01.194 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/ 2022-09-03T06:44:01.194 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4= 2022-09-03T06:44:01.195 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4=', 'Content-Length': '0'} 2022-09-03T06:44:01.197 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009cf4a0478cbd995a-006312f7b1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.198 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:01.199 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:01.200 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:01.200 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:01.201 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:01.201 DEBUG:boto:Method: GET 2022-09-03T06:44:01.202 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:44:01.202 DEBUG:boto:Data: 2022-09-03T06:44:01.203 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.203 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:01.204 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.204 DEBUG:boto:Params: {} 2022-09-03T06:44:01.205 DEBUG:boto:Token: None 2022-09-03T06:44:01.206 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/ 2022-09-03T06:44:01.206 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4= 2022-09-03T06:44:01.207 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4=', 'Content-Length': '0'} 2022-09-03T06:44:01.211 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000049209024cd2da9a7-006312f7b1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.258 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:01.259 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:01.260 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:01.261 DEBUG:boto:Method: HEAD 2022-09-03T06:44:01.261 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:44:01.262 DEBUG:boto:Data: 2022-09-03T06:44:01.262 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.263 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:01.264 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.265 DEBUG:boto:Params: {} 2022-09-03T06:44:01.267 DEBUG:boto:Token: None 2022-09-03T06:44:01.268 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj11 2022-09-03T06:44:01.269 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:teTKvE9eIHKskcXU2kvxPSU4cLI= 2022-09-03T06:44:01.271 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:teTKvE9eIHKskcXU2kvxPSU4cLI=', 'Content-Length': '0'} 2022-09-03T06:44:01.273 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000243f2aaacd183d42-006312f7b1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.274 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:01.275 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:01.276 DEBUG:boto:Method: HEAD 2022-09-03T06:44:01.277 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:44:01.277 DEBUG:boto:Data: 2022-09-03T06:44:01.278 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.279 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:01.279 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.282 DEBUG:boto:Params: {} 2022-09-03T06:44:01.283 DEBUG:boto:Token: None 2022-09-03T06:44:01.283 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj11 2022-09-03T06:44:01.284 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:teTKvE9eIHKskcXU2kvxPSU4cLI= 2022-09-03T06:44:01.285 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:teTKvE9eIHKskcXU2kvxPSU4cLI=', 'Content-Length': '0'} 2022-09-03T06:44:01.288 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dd6718aa416fd460-006312f7b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.289 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:44:01.290 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:01.291 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:01.292 DEBUG:boto:Method: GET 2022-09-03T06:44:01.294 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:44:01.295 DEBUG:boto:Data: 2022-09-03T06:44:01.296 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.297 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:01.299 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.300 DEBUG:boto:Params: {} 2022-09-03T06:44:01.301 DEBUG:boto:Token: None 2022-09-03T06:44:01.302 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj11 2022-09-03T06:44:01.304 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m67pn9I3PAjzWI3mj92LX6goJ20= 2022-09-03T06:44:01.305 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m67pn9I3PAjzWI3mj92LX6goJ20=', 'Content-Length': '0'} 2022-09-03T06:44:01.307 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000038485c79cc3de64b-006312f7b1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.309 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:01.310 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:01.311 DEBUG:boto:Method: GET 2022-09-03T06:44:01.312 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:44:01.313 DEBUG:boto:Data: 2022-09-03T06:44:01.313 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.314 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:01.315 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.316 DEBUG:boto:Params: {} 2022-09-03T06:44:01.317 DEBUG:boto:Token: None 2022-09-03T06:44:01.318 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj11 2022-09-03T06:44:01.319 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m67pn9I3PAjzWI3mj92LX6goJ20= 2022-09-03T06:44:01.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m67pn9I3PAjzWI3mj92LX6goJ20=', 'Content-Length': '0'} 2022-09-03T06:44:01.323 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cde22f33418c2169-006312f7b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.325 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:01.326 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:01.327 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:01.328 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:01.329 DEBUG:boto:Method: GET 2022-09-03T06:44:01.330 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:44:01.331 DEBUG:boto:Data: 2022-09-03T06:44:01.332 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.333 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:01.334 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.335 DEBUG:boto:Params: {} 2022-09-03T06:44:01.336 DEBUG:boto:Token: None 2022-09-03T06:44:01.338 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/ 2022-09-03T06:44:01.339 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4= 2022-09-03T06:44:01.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4=', 'Content-Length': '0'} 2022-09-03T06:44:01.344 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002e9d4528f3daabf6-006312f7b1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.345 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:01.346 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:01.347 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:01.347 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:01.348 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:01.349 DEBUG:boto:Method: GET 2022-09-03T06:44:01.349 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:44:01.350 DEBUG:boto:Data: 2022-09-03T06:44:01.350 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.351 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:01.352 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.353 DEBUG:boto:Params: {} 2022-09-03T06:44:01.354 DEBUG:boto:Token: None 2022-09-03T06:44:01.355 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/ 2022-09-03T06:44:01.356 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4= 2022-09-03T06:44:01.358 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4=', 'Content-Length': '0'} 2022-09-03T06:44:01.362 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000032e39480aac76623-006312f7b1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.363 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:01.365 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:44:01.365 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:01.366 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:01.367 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.368 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.369 DEBUG:boto:Method: GET 2022-09-03T06:44:01.369 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.370 DEBUG:boto:Data: 2022-09-03T06:44:01.371 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.372 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:01.373 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.374 DEBUG:boto:Params: {} 2022-09-03T06:44:01.376 DEBUG:boto:Token: None 2022-09-03T06:44:01.377 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.378 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TY8OrNvSmSxhslfXapLolkL0Bd4= 2022-09-03T06:44:01.379 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TY8OrNvSmSxhslfXapLolkL0Bd4=', 'Content-Length': '0'} 2022-09-03T06:44:01.387 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ebf445ec3467be61-006312f7b1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.389 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:01.389 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:01.390 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.391 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.391 DEBUG:boto:Method: GET 2022-09-03T06:44:01.392 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.393 DEBUG:boto:Data: 2022-09-03T06:44:01.393 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.394 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:01.395 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.395 DEBUG:boto:Params: {} 2022-09-03T06:44:01.396 DEBUG:boto:Token: None 2022-09-03T06:44:01.397 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.398 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TY8OrNvSmSxhslfXapLolkL0Bd4= 2022-09-03T06:44:01.399 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TY8OrNvSmSxhslfXapLolkL0Bd4=', 'Content-Length': '0'} 2022-09-03T06:44:01.401 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b06c0febcb65205f-006312f7b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.402 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:01.403 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:01.403 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.404 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.404 DEBUG:boto:Method: HEAD 2022-09-03T06:44:01.405 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.406 DEBUG:boto:Data: 2022-09-03T06:44:01.407 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.407 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:01.408 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.409 DEBUG:boto:Params: {} 2022-09-03T06:44:01.410 DEBUG:boto:Token: None 2022-09-03T06:44:01.410 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.411 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VomnIR+yOLBKbhNPO9Yd5wuQFUk= 2022-09-03T06:44:01.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VomnIR+yOLBKbhNPO9Yd5wuQFUk=', 'Content-Length': '0'} 2022-09-03T06:44:01.415 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000070f573e4414329a9-006312f7b1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.416 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:01.418 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:01.419 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.420 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.420 DEBUG:boto:Method: HEAD 2022-09-03T06:44:01.421 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.422 DEBUG:boto:Data: 2022-09-03T06:44:01.424 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.426 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:01.427 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.427 DEBUG:boto:Params: {} 2022-09-03T06:44:01.429 DEBUG:boto:Token: None 2022-09-03T06:44:01.430 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.431 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VomnIR+yOLBKbhNPO9Yd5wuQFUk= 2022-09-03T06:44:01.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VomnIR+yOLBKbhNPO9Yd5wuQFUk=', 'Content-Length': '0'} 2022-09-03T06:44:01.443 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000add22a684a767b31-006312f7b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.445 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:44:01.446 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:01.447 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:01.448 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.449 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.449 DEBUG:boto:Method: GET 2022-09-03T06:44:01.450 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.451 DEBUG:boto:Data: 2022-09-03T06:44:01.452 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.453 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:01.453 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.454 DEBUG:boto:Params: {} 2022-09-03T06:44:01.455 DEBUG:boto:Token: None 2022-09-03T06:44:01.455 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.456 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TY8OrNvSmSxhslfXapLolkL0Bd4= 2022-09-03T06:44:01.457 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TY8OrNvSmSxhslfXapLolkL0Bd4=', 'Content-Length': '0'} 2022-09-03T06:44:01.459 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006acba01511c8badb-006312f7b1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.502 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:01.503 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:01.504 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.505 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.505 DEBUG:boto:Method: GET 2022-09-03T06:44:01.506 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.507 DEBUG:boto:Data: 2022-09-03T06:44:01.507 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.508 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:01.509 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.510 DEBUG:boto:Params: {} 2022-09-03T06:44:01.510 DEBUG:boto:Token: None 2022-09-03T06:44:01.511 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:01.512 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TY8OrNvSmSxhslfXapLolkL0Bd4= 2022-09-03T06:44:01.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TY8OrNvSmSxhslfXapLolkL0Bd4=', 'Content-Length': '0'} 2022-09-03T06:44:01.515 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009ef5e7c5c8b1bfd4-006312f7b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.516 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:01.516 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:01.517 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:01.518 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:01.518 DEBUG:boto:Method: GET 2022-09-03T06:44:01.519 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:44:01.520 DEBUG:boto:Data: 2022-09-03T06:44:01.520 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.521 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:01.522 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.523 DEBUG:boto:Params: {} 2022-09-03T06:44:01.524 DEBUG:boto:Token: None 2022-09-03T06:44:01.526 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/ 2022-09-03T06:44:01.527 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4= 2022-09-03T06:44:01.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4=', 'Content-Length': '0'} 2022-09-03T06:44:01.530 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000083bbca902cf2904a-006312f7b1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.531 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:01.532 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:01.533 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:01.533 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:01.534 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:01.535 DEBUG:boto:Method: GET 2022-09-03T06:44:01.536 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:44:01.537 DEBUG:boto:Data: 2022-09-03T06:44:01.537 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.538 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:01.540 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.540 DEBUG:boto:Params: {} 2022-09-03T06:44:01.541 DEBUG:boto:Token: None 2022-09-03T06:44:01.542 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/ 2022-09-03T06:44:01.543 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4= 2022-09-03T06:44:01.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4=', 'Content-Length': '0'} 2022-09-03T06:44:01.548 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000739cf498bc8abe0f-006312f7b1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.549 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:01.550 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:01.551 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:01.552 DEBUG:boto:Method: HEAD 2022-09-03T06:44:01.552 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:44:01.553 DEBUG:boto:Data: 2022-09-03T06:44:01.553 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.554 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:01.554 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.555 DEBUG:boto:Params: {} 2022-09-03T06:44:01.555 DEBUG:boto:Token: None 2022-09-03T06:44:01.556 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj115 2022-09-03T06:44:01.557 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r20zLSMlMUaQqNciWWQDNsOwkRo= 2022-09-03T06:44:01.558 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r20zLSMlMUaQqNciWWQDNsOwkRo=', 'Content-Length': '0'} 2022-09-03T06:44:01.560 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000035fe088119c932f7-006312f7b1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.560 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:01.561 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:01.562 DEBUG:boto:Method: HEAD 2022-09-03T06:44:01.562 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:44:01.563 DEBUG:boto:Data: 2022-09-03T06:44:01.563 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.564 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:01.565 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.565 DEBUG:boto:Params: {} 2022-09-03T06:44:01.567 DEBUG:boto:Token: None 2022-09-03T06:44:01.567 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj115 2022-09-03T06:44:01.568 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r20zLSMlMUaQqNciWWQDNsOwkRo= 2022-09-03T06:44:01.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r20zLSMlMUaQqNciWWQDNsOwkRo=', 'Content-Length': '0'} 2022-09-03T06:44:01.570 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dfd6ad4d8ec078a3-006312f7b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.571 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:44:01.572 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:01.572 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:01.573 DEBUG:boto:Method: GET 2022-09-03T06:44:01.574 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:44:01.574 DEBUG:boto:Data: 2022-09-03T06:44:01.575 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.576 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:01.577 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.577 DEBUG:boto:Params: {} 2022-09-03T06:44:01.578 DEBUG:boto:Token: None 2022-09-03T06:44:01.579 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj115 2022-09-03T06:44:01.580 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ce7m43IE3BbTo7FQAyqAhO3qchc= 2022-09-03T06:44:01.581 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ce7m43IE3BbTo7FQAyqAhO3qchc=', 'Content-Length': '0'} 2022-09-03T06:44:01.583 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000732b9b429b0fa719-006312f7b1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.584 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:01.585 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:01.585 DEBUG:boto:Method: GET 2022-09-03T06:44:01.586 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:44:01.587 DEBUG:boto:Data: 2022-09-03T06:44:01.587 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.588 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:01.588 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.589 DEBUG:boto:Params: {} 2022-09-03T06:44:01.590 DEBUG:boto:Token: None 2022-09-03T06:44:01.591 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj115 2022-09-03T06:44:01.592 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ce7m43IE3BbTo7FQAyqAhO3qchc= 2022-09-03T06:44:01.592 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ce7m43IE3BbTo7FQAyqAhO3qchc=', 'Content-Length': '0'} 2022-09-03T06:44:01.595 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b0f51646915bb05-006312f7b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.642 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:01.643 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:01.643 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:01.644 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:01.645 DEBUG:boto:Method: GET 2022-09-03T06:44:01.645 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:44:01.646 DEBUG:boto:Data: 2022-09-03T06:44:01.646 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.647 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:01.648 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.648 DEBUG:boto:Params: {} 2022-09-03T06:44:01.649 DEBUG:boto:Token: None 2022-09-03T06:44:01.650 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/ 2022-09-03T06:44:01.651 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4= 2022-09-03T06:44:01.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4=', 'Content-Length': '0'} 2022-09-03T06:44:01.654 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000031cd2f19e46aa1e5-006312f7b1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.655 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:01.656 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:01.657 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:01.658 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:01.658 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:01.659 DEBUG:boto:Method: GET 2022-09-03T06:44:01.659 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:44:01.660 DEBUG:boto:Data: 2022-09-03T06:44:01.661 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.661 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:01.662 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.663 DEBUG:boto:Params: {} 2022-09-03T06:44:01.665 DEBUG:boto:Token: None 2022-09-03T06:44:01.666 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/ 2022-09-03T06:44:01.667 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4= 2022-09-03T06:44:01.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4=', 'Content-Length': '0'} 2022-09-03T06:44:01.671 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000068b378921a970f6-006312f7b1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.672 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:01.673 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:44:01.674 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:01.675 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:01.676 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.676 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.677 DEBUG:boto:Method: GET 2022-09-03T06:44:01.677 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.678 DEBUG:boto:Data: 2022-09-03T06:44:01.678 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.679 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:01.680 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.682 DEBUG:boto:Params: {} 2022-09-03T06:44:01.684 DEBUG:boto:Token: None 2022-09-03T06:44:01.685 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.686 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tc6+4fA5M1sQi8ioXWlo08WPn20= 2022-09-03T06:44:01.688 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tc6+4fA5M1sQi8ioXWlo08WPn20=', 'Content-Length': '0'} 2022-09-03T06:44:01.690 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000076d5d14b43e9e476-006312f7b1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.734 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:01.735 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:01.736 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.737 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.737 DEBUG:boto:Method: GET 2022-09-03T06:44:01.738 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.739 DEBUG:boto:Data: 2022-09-03T06:44:01.739 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.740 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:01.741 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.741 DEBUG:boto:Params: {} 2022-09-03T06:44:01.742 DEBUG:boto:Token: None 2022-09-03T06:44:01.743 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.743 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tc6+4fA5M1sQi8ioXWlo08WPn20= 2022-09-03T06:44:01.744 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tc6+4fA5M1sQi8ioXWlo08WPn20=', 'Content-Length': '0'} 2022-09-03T06:44:01.747 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f245b19c320bf5bc-006312f7b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.748 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:01.748 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:01.749 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.750 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.750 DEBUG:boto:Method: HEAD 2022-09-03T06:44:01.751 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.751 DEBUG:boto:Data: 2022-09-03T06:44:01.752 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.752 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:01.753 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.754 DEBUG:boto:Params: {} 2022-09-03T06:44:01.754 DEBUG:boto:Token: None 2022-09-03T06:44:01.755 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.756 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bmAUlRCki8Y55SLVgIp54vxSJXI= 2022-09-03T06:44:01.757 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bmAUlRCki8Y55SLVgIp54vxSJXI=', 'Content-Length': '0'} 2022-09-03T06:44:01.759 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009a7db3607b22a567-006312f7b1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.759 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:01.760 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:01.761 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.761 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.762 DEBUG:boto:Method: HEAD 2022-09-03T06:44:01.762 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.763 DEBUG:boto:Data: 2022-09-03T06:44:01.763 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.764 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:01.764 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.765 DEBUG:boto:Params: {} 2022-09-03T06:44:01.766 DEBUG:boto:Token: None 2022-09-03T06:44:01.766 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.767 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bmAUlRCki8Y55SLVgIp54vxSJXI= 2022-09-03T06:44:01.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bmAUlRCki8Y55SLVgIp54vxSJXI=', 'Content-Length': '0'} 2022-09-03T06:44:01.770 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000091d4f0a19eb2944f-006312f7b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.771 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:44:01.771 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:01.772 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:01.773 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.773 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.774 DEBUG:boto:Method: GET 2022-09-03T06:44:01.774 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.775 DEBUG:boto:Data: 2022-09-03T06:44:01.776 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.777 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:01.778 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.779 DEBUG:boto:Params: {} 2022-09-03T06:44:01.780 DEBUG:boto:Token: None 2022-09-03T06:44:01.781 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.781 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tc6+4fA5M1sQi8ioXWlo08WPn20= 2022-09-03T06:44:01.782 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tc6+4fA5M1sQi8ioXWlo08WPn20=', 'Content-Length': '0'} 2022-09-03T06:44:01.785 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b601f1f59b6ddce-006312f7b1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.787 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:01.788 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:01.790 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.791 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.792 DEBUG:boto:Method: GET 2022-09-03T06:44:01.793 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.795 DEBUG:boto:Data: 2022-09-03T06:44:01.796 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.797 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:01.798 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.800 DEBUG:boto:Params: {} 2022-09-03T06:44:01.801 DEBUG:boto:Token: None 2022-09-03T06:44:01.802 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:01.804 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tc6+4fA5M1sQi8ioXWlo08WPn20= 2022-09-03T06:44:01.805 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tc6+4fA5M1sQi8ioXWlo08WPn20=', 'Content-Length': '0'} 2022-09-03T06:44:01.808 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008dcc7b37a02b6188-006312f7b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.809 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:01.810 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:01.811 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:01.812 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:01.813 DEBUG:boto:Method: GET 2022-09-03T06:44:01.814 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:44:01.816 DEBUG:boto:Data: 2022-09-03T06:44:01.817 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.818 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:01.819 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.820 DEBUG:boto:Params: {} 2022-09-03T06:44:01.821 DEBUG:boto:Token: None 2022-09-03T06:44:01.822 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/ 2022-09-03T06:44:01.823 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4= 2022-09-03T06:44:01.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4=', 'Content-Length': '0'} 2022-09-03T06:44:01.827 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005601e04c52535a6e-006312f7b1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.829 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:01.830 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:01.831 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:01.832 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:01.833 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:01.834 DEBUG:boto:Method: GET 2022-09-03T06:44:01.835 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:44:01.836 DEBUG:boto:Data: 2022-09-03T06:44:01.837 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.839 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:01.840 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.841 DEBUG:boto:Params: {} 2022-09-03T06:44:01.842 DEBUG:boto:Token: None 2022-09-03T06:44:01.843 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/ 2022-09-03T06:44:01.844 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4= 2022-09-03T06:44:01.846 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4=', 'Content-Length': '0'} 2022-09-03T06:44:01.850 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e511f39f66de860f-006312f7b1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.851 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:01.853 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:01.854 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:01.855 DEBUG:boto:Method: HEAD 2022-09-03T06:44:01.856 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:44:01.857 DEBUG:boto:Data: 2022-09-03T06:44:01.858 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.859 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:01.860 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.861 DEBUG:boto:Params: {} 2022-09-03T06:44:01.862 DEBUG:boto:Token: None 2022-09-03T06:44:01.863 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj17 2022-09-03T06:44:01.864 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vfDpYVra/FkdtPeK+jLEb+VZr2I= 2022-09-03T06:44:01.865 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vfDpYVra/FkdtPeK+jLEb+VZr2I=', 'Content-Length': '0'} 2022-09-03T06:44:01.868 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000de6baf0565b3dcb9-006312f7b1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.869 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:01.870 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:01.871 DEBUG:boto:Method: HEAD 2022-09-03T06:44:01.872 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:44:01.873 DEBUG:boto:Data: 2022-09-03T06:44:01.874 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.875 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:01.876 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.877 DEBUG:boto:Params: {} 2022-09-03T06:44:01.878 DEBUG:boto:Token: None 2022-09-03T06:44:01.879 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj17 2022-09-03T06:44:01.880 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vfDpYVra/FkdtPeK+jLEb+VZr2I= 2022-09-03T06:44:01.882 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vfDpYVra/FkdtPeK+jLEb+VZr2I=', 'Content-Length': '0'} 2022-09-03T06:44:01.884 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a7745a2cf1cf753-006312f7b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.886 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:44:01.887 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:01.888 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:01.889 DEBUG:boto:Method: GET 2022-09-03T06:44:01.890 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:44:01.891 DEBUG:boto:Data: 2022-09-03T06:44:01.892 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.893 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:01.894 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.895 DEBUG:boto:Params: {} 2022-09-03T06:44:01.896 DEBUG:boto:Token: None 2022-09-03T06:44:01.898 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj17 2022-09-03T06:44:01.899 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J74/OfKbQyNPehNDDFb7bi9Cy2g= 2022-09-03T06:44:01.900 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J74/OfKbQyNPehNDDFb7bi9Cy2g=', 'Content-Length': '0'} 2022-09-03T06:44:01.903 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a871816cc611882-006312f7b1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.904 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:01.905 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:01.906 DEBUG:boto:Method: GET 2022-09-03T06:44:01.907 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:44:01.907 DEBUG:boto:Data: 2022-09-03T06:44:01.908 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.908 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:01.909 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.909 DEBUG:boto:Params: {} 2022-09-03T06:44:01.910 DEBUG:boto:Token: None 2022-09-03T06:44:01.911 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/obj17 2022-09-03T06:44:01.911 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J74/OfKbQyNPehNDDFb7bi9Cy2g= 2022-09-03T06:44:01.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J74/OfKbQyNPehNDDFb7bi9Cy2g=', 'Content-Length': '0'} 2022-09-03T06:44:01.915 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001bb3cd830423ceee-006312f7b1-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.962 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:01.963 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:01.964 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:01.964 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:01.965 DEBUG:boto:Method: GET 2022-09-03T06:44:01.966 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:44:01.966 DEBUG:boto:Data: 2022-09-03T06:44:01.967 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.967 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:01.968 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.968 DEBUG:boto:Params: {} 2022-09-03T06:44:01.969 DEBUG:boto:Token: None 2022-09-03T06:44:01.970 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/ 2022-09-03T06:44:01.971 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4= 2022-09-03T06:44:01.972 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4=', 'Content-Length': '0'} 2022-09-03T06:44:01.975 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000039f2f32892a8c1f1-006312f7b1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.976 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:01.977 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:01.978 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:01.979 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:01.979 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:01.980 DEBUG:boto:Method: GET 2022-09-03T06:44:01.981 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:44:01.981 DEBUG:boto:Data: 2022-09-03T06:44:01.982 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.982 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:01.983 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.984 DEBUG:boto:Params: {} 2022-09-03T06:44:01.984 DEBUG:boto:Token: None 2022-09-03T06:44:01.985 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:01 GMT /dsgvcy-75/ 2022-09-03T06:44:01.986 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4= 2022-09-03T06:44:01.987 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aDyu01EoUFo2hPmLNETUJd1XrH4=', 'Content-Length': '0'} 2022-09-03T06:44:01.990 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000490027862e465035-006312f7b1-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:01.991 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:01.992 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:44:01.993 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:01.993 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:01.994 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:01.994 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:01.995 DEBUG:boto:Method: GET 2022-09-03T06:44:01.996 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:01.996 DEBUG:boto:Data: 2022-09-03T06:44:01.997 DEBUG:boto:Headers: {} 2022-09-03T06:44:01.997 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:01.998 DEBUG:boto:Port: 8000 2022-09-03T06:44:01.998 DEBUG:boto:Params: {} 2022-09-03T06:44:01.999 DEBUG:boto:Token: None 2022-09-03T06:44:02.000 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:02.001 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJTE7SDia5hBuf56oUss2FTaMaI= 2022-09-03T06:44:02.001 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJTE7SDia5hBuf56oUss2FTaMaI=', 'Content-Length': '0'} 2022-09-03T06:44:02.003 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000044895c8e1a12b7f1-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.046 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:02.048 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:02.049 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:02.050 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:02.051 DEBUG:boto:Method: GET 2022-09-03T06:44:02.052 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:02.053 DEBUG:boto:Data: 2022-09-03T06:44:02.054 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.055 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.056 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.057 DEBUG:boto:Params: {} 2022-09-03T06:44:02.058 DEBUG:boto:Token: None 2022-09-03T06:44:02.058 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:02.059 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJTE7SDia5hBuf56oUss2FTaMaI= 2022-09-03T06:44:02.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJTE7SDia5hBuf56oUss2FTaMaI=', 'Content-Length': '0'} 2022-09-03T06:44:02.063 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000ba3bff6a890ee57-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.065 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:02.066 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:02.067 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:02.068 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:02.068 DEBUG:boto:Method: HEAD 2022-09-03T06:44:02.069 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:02.070 DEBUG:boto:Data: 2022-09-03T06:44:02.071 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.072 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:02.073 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.074 DEBUG:boto:Params: {} 2022-09-03T06:44:02.075 DEBUG:boto:Token: None 2022-09-03T06:44:02.076 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:02.077 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bxvppivY+RK1a1eU1HTpK6f1j64= 2022-09-03T06:44:02.078 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bxvppivY+RK1a1eU1HTpK6f1j64=', 'Content-Length': '0'} 2022-09-03T06:44:02.080 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096515b6b954ed54d-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.080 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:02.081 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:02.081 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:02.082 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:02.083 DEBUG:boto:Method: HEAD 2022-09-03T06:44:02.083 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:02.084 DEBUG:boto:Data: 2022-09-03T06:44:02.084 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.085 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.085 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.086 DEBUG:boto:Params: {} 2022-09-03T06:44:02.087 DEBUG:boto:Token: None 2022-09-03T06:44:02.087 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:02.088 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bxvppivY+RK1a1eU1HTpK6f1j64= 2022-09-03T06:44:02.089 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bxvppivY+RK1a1eU1HTpK6f1j64=', 'Content-Length': '0'} 2022-09-03T06:44:02.091 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001db9c4802740eec9-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.092 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:44:02.093 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:02.094 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:02.095 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:02.096 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:02.097 DEBUG:boto:Method: GET 2022-09-03T06:44:02.098 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:02.099 DEBUG:boto:Data: 2022-09-03T06:44:02.100 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.101 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:02.102 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.102 DEBUG:boto:Params: {} 2022-09-03T06:44:02.103 DEBUG:boto:Token: None 2022-09-03T06:44:02.104 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:02.104 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJTE7SDia5hBuf56oUss2FTaMaI= 2022-09-03T06:44:02.105 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJTE7SDia5hBuf56oUss2FTaMaI=', 'Content-Length': '0'} 2022-09-03T06:44:02.107 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f1b8df0179b87c4-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.108 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:02.108 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:02.109 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:02.109 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:02.110 DEBUG:boto:Method: GET 2022-09-03T06:44:02.111 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:02.111 DEBUG:boto:Data: 2022-09-03T06:44:02.112 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.112 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.113 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.113 DEBUG:boto:Params: {} 2022-09-03T06:44:02.114 DEBUG:boto:Token: None 2022-09-03T06:44:02.115 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:02.115 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JJTE7SDia5hBuf56oUss2FTaMaI= 2022-09-03T06:44:02.117 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JJTE7SDia5hBuf56oUss2FTaMaI=', 'Content-Length': '0'} 2022-09-03T06:44:02.119 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bba17b1bdeafb7be-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.162 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:02.164 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:02.164 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:02.165 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:02.166 DEBUG:boto:Method: GET 2022-09-03T06:44:02.167 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:44:02.169 DEBUG:boto:Data: 2022-09-03T06:44:02.170 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.172 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:02.173 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.174 DEBUG:boto:Params: {} 2022-09-03T06:44:02.175 DEBUG:boto:Token: None 2022-09-03T06:44:02.177 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/ 2022-09-03T06:44:02.178 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0= 2022-09-03T06:44:02.179 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0=', 'Content-Length': '0'} 2022-09-03T06:44:02.188 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000043580d65064fd9de-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.190 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:02.191 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:02.191 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:02.192 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:02.192 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:02.193 DEBUG:boto:Method: GET 2022-09-03T06:44:02.193 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:44:02.194 DEBUG:boto:Data: 2022-09-03T06:44:02.194 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.195 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.196 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.196 DEBUG:boto:Params: {} 2022-09-03T06:44:02.197 DEBUG:boto:Token: None 2022-09-03T06:44:02.197 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/ 2022-09-03T06:44:02.198 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0= 2022-09-03T06:44:02.199 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0=', 'Content-Length': '0'} 2022-09-03T06:44:02.202 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000051e4e80eceec1bef-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.204 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:02.205 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:02.206 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:02.207 DEBUG:boto:Method: HEAD 2022-09-03T06:44:02.208 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:44:02.208 DEBUG:boto:Data: 2022-09-03T06:44:02.209 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.210 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:02.211 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.212 DEBUG:boto:Params: {} 2022-09-03T06:44:02.213 DEBUG:boto:Token: None 2022-09-03T06:44:02.214 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj19 2022-09-03T06:44:02.215 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2eyYX512AgyRJtLmXCAxGjyyKl8= 2022-09-03T06:44:02.217 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2eyYX512AgyRJtLmXCAxGjyyKl8=', 'Content-Length': '0'} 2022-09-03T06:44:02.219 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000070f8f2bb8c2dd8d2-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.220 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:02.221 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:02.221 DEBUG:boto:Method: HEAD 2022-09-03T06:44:02.222 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:44:02.222 DEBUG:boto:Data: 2022-09-03T06:44:02.223 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.223 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.224 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.224 DEBUG:boto:Params: {} 2022-09-03T06:44:02.225 DEBUG:boto:Token: None 2022-09-03T06:44:02.225 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj19 2022-09-03T06:44:02.226 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2eyYX512AgyRJtLmXCAxGjyyKl8= 2022-09-03T06:44:02.227 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2eyYX512AgyRJtLmXCAxGjyyKl8=', 'Content-Length': '0'} 2022-09-03T06:44:02.229 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000afa9001a41ea5888-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.229 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:44:02.230 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:02.231 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:02.231 DEBUG:boto:Method: GET 2022-09-03T06:44:02.232 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:44:02.232 DEBUG:boto:Data: 2022-09-03T06:44:02.233 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.233 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:02.234 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.235 DEBUG:boto:Params: {} 2022-09-03T06:44:02.235 DEBUG:boto:Token: None 2022-09-03T06:44:02.236 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj19 2022-09-03T06:44:02.237 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8lu8y0LeTaifzxPuzbUgJyHlAQA= 2022-09-03T06:44:02.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8lu8y0LeTaifzxPuzbUgJyHlAQA=', 'Content-Length': '0'} 2022-09-03T06:44:02.239 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e679145d146f386c-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.240 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:02.241 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:02.241 DEBUG:boto:Method: GET 2022-09-03T06:44:02.242 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:44:02.242 DEBUG:boto:Data: 2022-09-03T06:44:02.243 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.244 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.244 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.245 DEBUG:boto:Params: {} 2022-09-03T06:44:02.246 DEBUG:boto:Token: None 2022-09-03T06:44:02.246 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj19 2022-09-03T06:44:02.247 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8lu8y0LeTaifzxPuzbUgJyHlAQA= 2022-09-03T06:44:02.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8lu8y0LeTaifzxPuzbUgJyHlAQA=', 'Content-Length': '0'} 2022-09-03T06:44:02.249 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a67e8b55e1e3bf2e-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.251 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:02.252 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:02.252 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:02.253 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:02.253 DEBUG:boto:Method: GET 2022-09-03T06:44:02.254 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:44:02.255 DEBUG:boto:Data: 2022-09-03T06:44:02.255 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.256 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:02.256 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.257 DEBUG:boto:Params: {} 2022-09-03T06:44:02.257 DEBUG:boto:Token: None 2022-09-03T06:44:02.258 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/ 2022-09-03T06:44:02.259 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0= 2022-09-03T06:44:02.259 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0=', 'Content-Length': '0'} 2022-09-03T06:44:02.263 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000780e12dce7ba4e45-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.270 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:02.272 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:02.275 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:02.275 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:02.276 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:02.276 DEBUG:boto:Method: GET 2022-09-03T06:44:02.277 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:44:02.278 DEBUG:boto:Data: 2022-09-03T06:44:02.279 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.280 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.282 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.282 DEBUG:boto:Params: {} 2022-09-03T06:44:02.283 DEBUG:boto:Token: None 2022-09-03T06:44:02.284 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/ 2022-09-03T06:44:02.284 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0= 2022-09-03T06:44:02.285 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0=', 'Content-Length': '0'} 2022-09-03T06:44:02.293 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000088081b23783d59d6-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.294 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:02.295 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:44:02.296 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:02.296 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:02.297 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.297 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.298 DEBUG:boto:Method: GET 2022-09-03T06:44:02.298 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.299 DEBUG:boto:Data: 2022-09-03T06:44:02.300 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.300 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:02.301 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.302 DEBUG:boto:Params: {} 2022-09-03T06:44:02.303 DEBUG:boto:Token: None 2022-09-03T06:44:02.303 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.304 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OZnfgX28pjWxfm6FP3yUWEqsUN8= 2022-09-03T06:44:02.305 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OZnfgX28pjWxfm6FP3yUWEqsUN8=', 'Content-Length': '0'} 2022-09-03T06:44:02.313 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ef093f6f54262c0e-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.355 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:02.355 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:02.356 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.357 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.358 DEBUG:boto:Method: GET 2022-09-03T06:44:02.359 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.360 DEBUG:boto:Data: 2022-09-03T06:44:02.361 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.362 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.363 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.364 DEBUG:boto:Params: {} 2022-09-03T06:44:02.364 DEBUG:boto:Token: None 2022-09-03T06:44:02.365 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OZnfgX28pjWxfm6FP3yUWEqsUN8= 2022-09-03T06:44:02.366 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OZnfgX28pjWxfm6FP3yUWEqsUN8=', 'Content-Length': '0'} 2022-09-03T06:44:02.369 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000b6f98ef6985111c-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.370 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:02.370 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:02.371 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.372 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.372 DEBUG:boto:Method: HEAD 2022-09-03T06:44:02.374 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.375 DEBUG:boto:Data: 2022-09-03T06:44:02.376 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.377 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:02.378 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.379 DEBUG:boto:Params: {} 2022-09-03T06:44:02.380 DEBUG:boto:Token: None 2022-09-03T06:44:02.381 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.382 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uuc5I/+yEKQ2dVAdBijdGAL9R7o= 2022-09-03T06:44:02.383 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uuc5I/+yEKQ2dVAdBijdGAL9R7o=', 'Content-Length': '0'} 2022-09-03T06:44:02.385 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f50714bbf3abffd0-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.386 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:02.387 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:02.388 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.388 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.389 DEBUG:boto:Method: HEAD 2022-09-03T06:44:02.389 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.390 DEBUG:boto:Data: 2022-09-03T06:44:02.390 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.391 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.391 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.392 DEBUG:boto:Params: {} 2022-09-03T06:44:02.392 DEBUG:boto:Token: None 2022-09-03T06:44:02.393 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.394 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uuc5I/+yEKQ2dVAdBijdGAL9R7o= 2022-09-03T06:44:02.394 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uuc5I/+yEKQ2dVAdBijdGAL9R7o=', 'Content-Length': '0'} 2022-09-03T06:44:02.397 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002809dd441f146e05-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.397 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:44:02.398 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:02.399 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:02.399 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.400 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.400 DEBUG:boto:Method: GET 2022-09-03T06:44:02.401 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.401 DEBUG:boto:Data: 2022-09-03T06:44:02.402 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.403 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:02.403 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.404 DEBUG:boto:Params: {} 2022-09-03T06:44:02.404 DEBUG:boto:Token: None 2022-09-03T06:44:02.405 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.406 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OZnfgX28pjWxfm6FP3yUWEqsUN8= 2022-09-03T06:44:02.406 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OZnfgX28pjWxfm6FP3yUWEqsUN8=', 'Content-Length': '0'} 2022-09-03T06:44:02.408 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d7023876c98ca46e-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.409 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:02.410 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:02.411 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.411 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.412 DEBUG:boto:Method: GET 2022-09-03T06:44:02.412 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.413 DEBUG:boto:Data: 2022-09-03T06:44:02.414 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.415 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.415 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.416 DEBUG:boto:Params: {} 2022-09-03T06:44:02.416 DEBUG:boto:Token: None 2022-09-03T06:44:02.417 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:02.418 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OZnfgX28pjWxfm6FP3yUWEqsUN8= 2022-09-03T06:44:02.419 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OZnfgX28pjWxfm6FP3yUWEqsUN8=', 'Content-Length': '0'} 2022-09-03T06:44:02.421 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000079054b9031348da6-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.422 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:02.423 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:02.423 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:02.424 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:02.424 DEBUG:boto:Method: GET 2022-09-03T06:44:02.425 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:44:02.425 DEBUG:boto:Data: 2022-09-03T06:44:02.426 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.426 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:02.427 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.427 DEBUG:boto:Params: {} 2022-09-03T06:44:02.428 DEBUG:boto:Token: None 2022-09-03T06:44:02.429 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/ 2022-09-03T06:44:02.429 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0= 2022-09-03T06:44:02.430 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0=', 'Content-Length': '0'} 2022-09-03T06:44:02.433 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007806d2fcaff2377e-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.433 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:02.434 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:02.435 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:02.435 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:02.436 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:02.436 DEBUG:boto:Method: GET 2022-09-03T06:44:02.437 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:44:02.437 DEBUG:boto:Data: 2022-09-03T06:44:02.438 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.439 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.439 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.440 DEBUG:boto:Params: {} 2022-09-03T06:44:02.440 DEBUG:boto:Token: None 2022-09-03T06:44:02.441 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/ 2022-09-03T06:44:02.441 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0= 2022-09-03T06:44:02.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0=', 'Content-Length': '0'} 2022-09-03T06:44:02.446 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000076eada348d8db2e5-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.446 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:02.447 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:02.448 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:02.448 DEBUG:boto:Method: HEAD 2022-09-03T06:44:02.449 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:44:02.449 DEBUG:boto:Data: 2022-09-03T06:44:02.450 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.451 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:02.451 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.452 DEBUG:boto:Params: {} 2022-09-03T06:44:02.452 DEBUG:boto:Token: None 2022-09-03T06:44:02.453 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj22 2022-09-03T06:44:02.453 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n/hRlloPG18vQzpa1aL+I/ow+Hk= 2022-09-03T06:44:02.454 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n/hRlloPG18vQzpa1aL+I/ow+Hk=', 'Content-Length': '0'} 2022-09-03T06:44:02.456 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a07bfca7d62370b8-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.458 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:02.459 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:02.459 DEBUG:boto:Method: HEAD 2022-09-03T06:44:02.460 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:44:02.461 DEBUG:boto:Data: 2022-09-03T06:44:02.462 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.463 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.464 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.464 DEBUG:boto:Params: {} 2022-09-03T06:44:02.497 DEBUG:boto:Token: None 2022-09-03T06:44:02.498 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj22 2022-09-03T06:44:02.499 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n/hRlloPG18vQzpa1aL+I/ow+Hk= 2022-09-03T06:44:02.499 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n/hRlloPG18vQzpa1aL+I/ow+Hk=', 'Content-Length': '0'} 2022-09-03T06:44:02.502 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000098832c9335bcc51e-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.503 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:44:02.504 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:02.504 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:02.505 DEBUG:boto:Method: GET 2022-09-03T06:44:02.505 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:44:02.506 DEBUG:boto:Data: 2022-09-03T06:44:02.507 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.507 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:02.508 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.509 DEBUG:boto:Params: {} 2022-09-03T06:44:02.509 DEBUG:boto:Token: None 2022-09-03T06:44:02.510 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj22 2022-09-03T06:44:02.511 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dWRrychddbyBFIKVW4w4rUGnsLs= 2022-09-03T06:44:02.511 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dWRrychddbyBFIKVW4w4rUGnsLs=', 'Content-Length': '0'} 2022-09-03T06:44:02.514 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005766e01ee86e0635-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.515 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:02.516 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:02.516 DEBUG:boto:Method: GET 2022-09-03T06:44:02.517 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:44:02.517 DEBUG:boto:Data: 2022-09-03T06:44:02.518 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.518 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.519 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.519 DEBUG:boto:Params: {} 2022-09-03T06:44:02.520 DEBUG:boto:Token: None 2022-09-03T06:44:02.521 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj22 2022-09-03T06:44:02.521 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dWRrychddbyBFIKVW4w4rUGnsLs= 2022-09-03T06:44:02.522 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dWRrychddbyBFIKVW4w4rUGnsLs=', 'Content-Length': '0'} 2022-09-03T06:44:02.524 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f6327f7481d543d8-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.525 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:02.526 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:02.526 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:02.527 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:02.528 DEBUG:boto:Method: GET 2022-09-03T06:44:02.528 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:44:02.529 DEBUG:boto:Data: 2022-09-03T06:44:02.529 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.530 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:02.531 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.531 DEBUG:boto:Params: {} 2022-09-03T06:44:02.532 DEBUG:boto:Token: None 2022-09-03T06:44:02.533 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/ 2022-09-03T06:44:02.533 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0= 2022-09-03T06:44:02.534 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0=', 'Content-Length': '0'} 2022-09-03T06:44:02.537 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d1556c50b921e7fa-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.538 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:02.539 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:02.539 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:02.540 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:02.540 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:02.541 DEBUG:boto:Method: GET 2022-09-03T06:44:02.541 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:44:02.542 DEBUG:boto:Data: 2022-09-03T06:44:02.543 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.543 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.544 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.544 DEBUG:boto:Params: {} 2022-09-03T06:44:02.545 DEBUG:boto:Token: None 2022-09-03T06:44:02.545 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/ 2022-09-03T06:44:02.546 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0= 2022-09-03T06:44:02.547 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0=', 'Content-Length': '0'} 2022-09-03T06:44:02.550 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001951c25fa819ab5f-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.551 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:02.552 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:44:02.553 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:02.554 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:02.554 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.555 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.555 DEBUG:boto:Method: GET 2022-09-03T06:44:02.556 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.556 DEBUG:boto:Data: 2022-09-03T06:44:02.557 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.557 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:02.558 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.559 DEBUG:boto:Params: {} 2022-09-03T06:44:02.561 DEBUG:boto:Token: None 2022-09-03T06:44:02.562 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.563 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WsYgvWOO0ILWEzvGPCbfgfYcWlY= 2022-09-03T06:44:02.564 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WsYgvWOO0ILWEzvGPCbfgfYcWlY=', 'Content-Length': '0'} 2022-09-03T06:44:02.568 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000842cde862ad937da-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.572 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:02.573 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:02.574 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.574 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.575 DEBUG:boto:Method: GET 2022-09-03T06:44:02.575 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.576 DEBUG:boto:Data: 2022-09-03T06:44:02.577 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.577 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.578 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.578 DEBUG:boto:Params: {} 2022-09-03T06:44:02.579 DEBUG:boto:Token: None 2022-09-03T06:44:02.580 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.581 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WsYgvWOO0ILWEzvGPCbfgfYcWlY= 2022-09-03T06:44:02.581 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WsYgvWOO0ILWEzvGPCbfgfYcWlY=', 'Content-Length': '0'} 2022-09-03T06:44:02.584 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000027fb34411a5e1de-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.627 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:02.628 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:02.629 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.630 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.631 DEBUG:boto:Method: HEAD 2022-09-03T06:44:02.631 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.632 DEBUG:boto:Data: 2022-09-03T06:44:02.633 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.634 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:02.636 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.637 DEBUG:boto:Params: {} 2022-09-03T06:44:02.638 DEBUG:boto:Token: None 2022-09-03T06:44:02.639 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.640 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ryH6D62Wnb0MFnYtzgjePaoCOZc= 2022-09-03T06:44:02.641 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ryH6D62Wnb0MFnYtzgjePaoCOZc=', 'Content-Length': '0'} 2022-09-03T06:44:02.643 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a40ae5fdfeda2ae0-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.645 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:02.646 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:02.647 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.648 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.648 DEBUG:boto:Method: HEAD 2022-09-03T06:44:02.649 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.650 DEBUG:boto:Data: 2022-09-03T06:44:02.651 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.652 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.653 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.654 DEBUG:boto:Params: {} 2022-09-03T06:44:02.655 DEBUG:boto:Token: None 2022-09-03T06:44:02.656 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.657 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ryH6D62Wnb0MFnYtzgjePaoCOZc= 2022-09-03T06:44:02.658 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ryH6D62Wnb0MFnYtzgjePaoCOZc=', 'Content-Length': '0'} 2022-09-03T06:44:02.664 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006a820ed6d73204a4-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.665 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:44:02.666 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:02.667 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:02.668 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.669 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.670 DEBUG:boto:Method: GET 2022-09-03T06:44:02.671 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.674 DEBUG:boto:Data: 2022-09-03T06:44:02.675 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.676 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:02.678 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.679 DEBUG:boto:Params: {} 2022-09-03T06:44:02.680 DEBUG:boto:Token: None 2022-09-03T06:44:02.681 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.682 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WsYgvWOO0ILWEzvGPCbfgfYcWlY= 2022-09-03T06:44:02.683 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WsYgvWOO0ILWEzvGPCbfgfYcWlY=', 'Content-Length': '0'} 2022-09-03T06:44:02.686 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b9cd3cd63b782547-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.687 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:02.688 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:02.689 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.690 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.690 DEBUG:boto:Method: GET 2022-09-03T06:44:02.691 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.691 DEBUG:boto:Data: 2022-09-03T06:44:02.692 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.693 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.696 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.699 DEBUG:boto:Params: {} 2022-09-03T06:44:02.700 DEBUG:boto:Token: None 2022-09-03T06:44:02.701 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:02.702 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WsYgvWOO0ILWEzvGPCbfgfYcWlY= 2022-09-03T06:44:02.703 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WsYgvWOO0ILWEzvGPCbfgfYcWlY=', 'Content-Length': '0'} 2022-09-03T06:44:02.707 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000031f1ce6bac876419-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.708 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:02.710 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:02.711 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:02.712 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:02.712 DEBUG:boto:Method: GET 2022-09-03T06:44:02.713 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:44:02.714 DEBUG:boto:Data: 2022-09-03T06:44:02.715 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.716 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:02.719 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.720 DEBUG:boto:Params: {} 2022-09-03T06:44:02.722 DEBUG:boto:Token: None 2022-09-03T06:44:02.723 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/ 2022-09-03T06:44:02.724 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0= 2022-09-03T06:44:02.727 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0=', 'Content-Length': '0'} 2022-09-03T06:44:02.730 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d07f5f88f2b81ef4-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.770 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:02.772 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:02.774 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:02.776 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:02.777 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:02.778 DEBUG:boto:Method: GET 2022-09-03T06:44:02.779 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:44:02.780 DEBUG:boto:Data: 2022-09-03T06:44:02.781 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.782 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.783 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.784 DEBUG:boto:Params: {} 2022-09-03T06:44:02.785 DEBUG:boto:Token: None 2022-09-03T06:44:02.786 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/ 2022-09-03T06:44:02.787 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0= 2022-09-03T06:44:02.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0=', 'Content-Length': '0'} 2022-09-03T06:44:02.792 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009f65bf732463e0ee-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.793 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:02.794 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:02.795 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:02.796 DEBUG:boto:Method: HEAD 2022-09-03T06:44:02.797 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:44:02.798 DEBUG:boto:Data: 2022-09-03T06:44:02.799 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.800 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:02.801 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.802 DEBUG:boto:Params: {} 2022-09-03T06:44:02.803 DEBUG:boto:Token: None 2022-09-03T06:44:02.804 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj23 2022-09-03T06:44:02.804 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SkR4XqnXxnC1t2lKGo1rNWQcLgw= 2022-09-03T06:44:02.805 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SkR4XqnXxnC1t2lKGo1rNWQcLgw=', 'Content-Length': '0'} 2022-09-03T06:44:02.808 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a312d717c78f64b-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.809 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:02.809 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:02.810 DEBUG:boto:Method: HEAD 2022-09-03T06:44:02.810 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:44:02.811 DEBUG:boto:Data: 2022-09-03T06:44:02.812 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.812 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.813 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.813 DEBUG:boto:Params: {} 2022-09-03T06:44:02.814 DEBUG:boto:Token: None 2022-09-03T06:44:02.814 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj23 2022-09-03T06:44:02.815 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SkR4XqnXxnC1t2lKGo1rNWQcLgw= 2022-09-03T06:44:02.816 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SkR4XqnXxnC1t2lKGo1rNWQcLgw=', 'Content-Length': '0'} 2022-09-03T06:44:02.819 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000210fd4d4287ca67d-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.820 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:44:02.821 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:02.822 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:02.823 DEBUG:boto:Method: GET 2022-09-03T06:44:02.824 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:44:02.825 DEBUG:boto:Data: 2022-09-03T06:44:02.826 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.827 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:02.829 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.830 DEBUG:boto:Params: {} 2022-09-03T06:44:02.831 DEBUG:boto:Token: None 2022-09-03T06:44:02.832 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj23 2022-09-03T06:44:02.833 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7ckVG0FI0xBpQF6+nrWCLEWh7zg= 2022-09-03T06:44:02.834 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7ckVG0FI0xBpQF6+nrWCLEWh7zg=', 'Content-Length': '0'} 2022-09-03T06:44:02.836 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000024b2bb191168892a-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.837 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:02.838 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:02.838 DEBUG:boto:Method: GET 2022-09-03T06:44:02.839 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:44:02.840 DEBUG:boto:Data: 2022-09-03T06:44:02.841 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.842 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.843 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.844 DEBUG:boto:Params: {} 2022-09-03T06:44:02.845 DEBUG:boto:Token: None 2022-09-03T06:44:02.846 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj23 2022-09-03T06:44:02.848 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7ckVG0FI0xBpQF6+nrWCLEWh7zg= 2022-09-03T06:44:02.849 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7ckVG0FI0xBpQF6+nrWCLEWh7zg=', 'Content-Length': '0'} 2022-09-03T06:44:02.853 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000272d5c838974b3fe-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.898 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:02.899 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:02.900 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:02.901 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:02.901 DEBUG:boto:Method: GET 2022-09-03T06:44:02.902 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:44:02.903 DEBUG:boto:Data: 2022-09-03T06:44:02.903 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.904 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:02.905 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.905 DEBUG:boto:Params: {} 2022-09-03T06:44:02.906 DEBUG:boto:Token: None 2022-09-03T06:44:02.907 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/ 2022-09-03T06:44:02.908 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0= 2022-09-03T06:44:02.908 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0=', 'Content-Length': '0'} 2022-09-03T06:44:02.911 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000501bcbb8e149e73e-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.912 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:02.913 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:02.913 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:02.914 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:02.915 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:02.915 DEBUG:boto:Method: GET 2022-09-03T06:44:02.916 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:44:02.917 DEBUG:boto:Data: 2022-09-03T06:44:02.917 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.918 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.918 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.919 DEBUG:boto:Params: {} 2022-09-03T06:44:02.920 DEBUG:boto:Token: None 2022-09-03T06:44:02.920 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/ 2022-09-03T06:44:02.921 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0= 2022-09-03T06:44:02.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7fp6B7IWd0GTYqDWXITFMe5UYS0=', 'Content-Length': '0'} 2022-09-03T06:44:02.925 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cbdb10db86f6938a-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.926 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:02.927 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:44:02.927 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:02.928 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:02.928 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.929 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.929 DEBUG:boto:Method: GET 2022-09-03T06:44:02.930 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.930 DEBUG:boto:Data: 2022-09-03T06:44:02.931 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.931 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:02.932 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.932 DEBUG:boto:Params: {} 2022-09-03T06:44:02.933 DEBUG:boto:Token: None 2022-09-03T06:44:02.934 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.934 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:d+n0vADVNxI1zoqgpqxpiBDPtCQ= 2022-09-03T06:44:02.935 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:d+n0vADVNxI1zoqgpqxpiBDPtCQ=', 'Content-Length': '0'} 2022-09-03T06:44:02.937 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000926b4167357afe80-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.938 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:02.939 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:02.939 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.940 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.940 DEBUG:boto:Method: GET 2022-09-03T06:44:02.941 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.941 DEBUG:boto:Data: 2022-09-03T06:44:02.942 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.942 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.943 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.943 DEBUG:boto:Params: {} 2022-09-03T06:44:02.944 DEBUG:boto:Token: None 2022-09-03T06:44:02.945 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.945 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:d+n0vADVNxI1zoqgpqxpiBDPtCQ= 2022-09-03T06:44:02.946 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:d+n0vADVNxI1zoqgpqxpiBDPtCQ=', 'Content-Length': '0'} 2022-09-03T06:44:02.948 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000083166ef9df48129f-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.949 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:02.950 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:02.951 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.951 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.952 DEBUG:boto:Method: HEAD 2022-09-03T06:44:02.952 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.953 DEBUG:boto:Data: 2022-09-03T06:44:02.953 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.954 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:02.955 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.955 DEBUG:boto:Params: {} 2022-09-03T06:44:02.956 DEBUG:boto:Token: None 2022-09-03T06:44:02.956 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.957 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z4xBx3Nyswj7WA1Z03sRPp8OXZs= 2022-09-03T06:44:02.958 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z4xBx3Nyswj7WA1Z03sRPp8OXZs=', 'Content-Length': '0'} 2022-09-03T06:44:02.960 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006ae02e1f9c692979-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.961 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:02.962 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:02.963 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.964 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.965 DEBUG:boto:Method: HEAD 2022-09-03T06:44:02.966 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.966 DEBUG:boto:Data: 2022-09-03T06:44:02.967 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.967 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.968 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.968 DEBUG:boto:Params: {} 2022-09-03T06:44:02.969 DEBUG:boto:Token: None 2022-09-03T06:44:02.970 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.970 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z4xBx3Nyswj7WA1Z03sRPp8OXZs= 2022-09-03T06:44:02.971 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z4xBx3Nyswj7WA1Z03sRPp8OXZs=', 'Content-Length': '0'} 2022-09-03T06:44:02.973 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e4c8f6e878741584-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.974 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:44:02.975 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:02.976 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:02.976 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.977 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.977 DEBUG:boto:Method: GET 2022-09-03T06:44:02.978 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.978 DEBUG:boto:Data: 2022-09-03T06:44:02.979 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.979 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:02.980 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.981 DEBUG:boto:Params: {} 2022-09-03T06:44:02.981 DEBUG:boto:Token: None 2022-09-03T06:44:02.982 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.983 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:d+n0vADVNxI1zoqgpqxpiBDPtCQ= 2022-09-03T06:44:02.983 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:d+n0vADVNxI1zoqgpqxpiBDPtCQ=', 'Content-Length': '0'} 2022-09-03T06:44:02.986 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003f46684f51bb99ae-006312f7b2-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:02.987 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:02.988 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:02.989 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.990 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.991 DEBUG:boto:Method: GET 2022-09-03T06:44:02.991 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.992 DEBUG:boto:Data: 2022-09-03T06:44:02.993 DEBUG:boto:Headers: {} 2022-09-03T06:44:02.993 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:02.994 DEBUG:boto:Port: 8000 2022-09-03T06:44:02.994 DEBUG:boto:Params: {} 2022-09-03T06:44:02.995 DEBUG:boto:Token: None 2022-09-03T06:44:02.996 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:02 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:02.996 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:d+n0vADVNxI1zoqgpqxpiBDPtCQ= 2022-09-03T06:44:02.997 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:d+n0vADVNxI1zoqgpqxpiBDPtCQ=', 'Content-Length': '0'} 2022-09-03T06:44:03.000 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000051f4f7f58bb3a820-006312f7b2-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.001 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:03.002 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:03.002 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:03.003 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:03.003 DEBUG:boto:Method: GET 2022-09-03T06:44:03.004 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:44:03.005 DEBUG:boto:Data: 2022-09-03T06:44:03.006 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.007 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.007 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.008 DEBUG:boto:Params: {} 2022-09-03T06:44:03.008 DEBUG:boto:Token: None 2022-09-03T06:44:03.009 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/ 2022-09-03T06:44:03.010 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY= 2022-09-03T06:44:03.011 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY=', 'Content-Length': '0'} 2022-09-03T06:44:03.013 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c9b1c6f8c6380cb4-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.015 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:03.015 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:03.016 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:03.017 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:03.017 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:03.018 DEBUG:boto:Method: GET 2022-09-03T06:44:03.019 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:44:03.019 DEBUG:boto:Data: 2022-09-03T06:44:03.020 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.021 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.021 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.022 DEBUG:boto:Params: {} 2022-09-03T06:44:03.022 DEBUG:boto:Token: None 2022-09-03T06:44:03.023 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/ 2022-09-03T06:44:03.024 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY= 2022-09-03T06:44:03.025 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY=', 'Content-Length': '0'} 2022-09-03T06:44:03.029 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d036ebfe4aa6d921-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.031 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:03.032 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:03.033 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:03.034 DEBUG:boto:Method: HEAD 2022-09-03T06:44:03.035 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:44:03.035 DEBUG:boto:Data: 2022-09-03T06:44:03.036 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.037 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.038 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.039 DEBUG:boto:Params: {} 2022-09-03T06:44:03.039 DEBUG:boto:Token: None 2022-09-03T06:44:03.041 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj33 2022-09-03T06:44:03.041 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HCZ2JI8e9Z5kA3JbMtHyMfKVaBo= 2022-09-03T06:44:03.042 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HCZ2JI8e9Z5kA3JbMtHyMfKVaBo=', 'Content-Length': '0'} 2022-09-03T06:44:03.044 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e30b4d9069f8d6a5-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.045 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:03.046 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:03.046 DEBUG:boto:Method: HEAD 2022-09-03T06:44:03.047 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:44:03.047 DEBUG:boto:Data: 2022-09-03T06:44:03.048 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.048 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.049 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.049 DEBUG:boto:Params: {} 2022-09-03T06:44:03.050 DEBUG:boto:Token: None 2022-09-03T06:44:03.051 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj33 2022-09-03T06:44:03.051 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HCZ2JI8e9Z5kA3JbMtHyMfKVaBo= 2022-09-03T06:44:03.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HCZ2JI8e9Z5kA3JbMtHyMfKVaBo=', 'Content-Length': '0'} 2022-09-03T06:44:03.054 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f0fc59f34afd6436-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.055 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:44:03.056 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:03.057 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:03.058 DEBUG:boto:Method: GET 2022-09-03T06:44:03.059 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:44:03.060 DEBUG:boto:Data: 2022-09-03T06:44:03.061 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.062 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.063 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.064 DEBUG:boto:Params: {} 2022-09-03T06:44:03.065 DEBUG:boto:Token: None 2022-09-03T06:44:03.066 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj33 2022-09-03T06:44:03.067 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sbJU1Ua9Wc9WkBfiBvh9jNvK7Q0= 2022-09-03T06:44:03.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sbJU1Ua9Wc9WkBfiBvh9jNvK7Q0=', 'Content-Length': '0'} 2022-09-03T06:44:03.070 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007457ef7aa0f8a87d-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.071 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:03.072 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:03.072 DEBUG:boto:Method: GET 2022-09-03T06:44:03.073 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:44:03.073 DEBUG:boto:Data: 2022-09-03T06:44:03.074 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.075 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.075 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.076 DEBUG:boto:Params: {} 2022-09-03T06:44:03.076 DEBUG:boto:Token: None 2022-09-03T06:44:03.077 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj33 2022-09-03T06:44:03.078 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sbJU1Ua9Wc9WkBfiBvh9jNvK7Q0= 2022-09-03T06:44:03.079 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sbJU1Ua9Wc9WkBfiBvh9jNvK7Q0=', 'Content-Length': '0'} 2022-09-03T06:44:03.082 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000f62c488fd1f3c08-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.083 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:03.083 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:03.084 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:03.084 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:03.085 DEBUG:boto:Method: GET 2022-09-03T06:44:03.085 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:44:03.086 DEBUG:boto:Data: 2022-09-03T06:44:03.086 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.087 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.088 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.089 DEBUG:boto:Params: {} 2022-09-03T06:44:03.090 DEBUG:boto:Token: None 2022-09-03T06:44:03.090 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/ 2022-09-03T06:44:03.091 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY= 2022-09-03T06:44:03.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY=', 'Content-Length': '0'} 2022-09-03T06:44:03.095 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f57fd61ee1898fea-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.138 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:03.139 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:03.140 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:03.140 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:03.141 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:03.142 DEBUG:boto:Method: GET 2022-09-03T06:44:03.142 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:44:03.143 DEBUG:boto:Data: 2022-09-03T06:44:03.146 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.148 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.149 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.153 DEBUG:boto:Params: {} 2022-09-03T06:44:03.155 DEBUG:boto:Token: None 2022-09-03T06:44:03.155 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/ 2022-09-03T06:44:03.156 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY= 2022-09-03T06:44:03.157 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY=', 'Content-Length': '0'} 2022-09-03T06:44:03.160 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000edad2b46d1379d74-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.161 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:03.162 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:44:03.163 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:03.164 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:03.165 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.166 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.168 DEBUG:boto:Method: GET 2022-09-03T06:44:03.169 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.170 DEBUG:boto:Data: 2022-09-03T06:44:03.171 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.172 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.172 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.173 DEBUG:boto:Params: {} 2022-09-03T06:44:03.173 DEBUG:boto:Token: None 2022-09-03T06:44:03.174 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.175 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t6HmN4Kpr5F5RX/SLtCIjMpGsrE= 2022-09-03T06:44:03.175 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t6HmN4Kpr5F5RX/SLtCIjMpGsrE=', 'Content-Length': '0'} 2022-09-03T06:44:03.178 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006eccd1fad6a9dd73-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.179 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:03.180 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:03.181 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.182 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.183 DEBUG:boto:Method: GET 2022-09-03T06:44:03.184 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.185 DEBUG:boto:Data: 2022-09-03T06:44:03.186 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.187 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.187 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.188 DEBUG:boto:Params: {} 2022-09-03T06:44:03.190 DEBUG:boto:Token: None 2022-09-03T06:44:03.191 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.192 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t6HmN4Kpr5F5RX/SLtCIjMpGsrE= 2022-09-03T06:44:03.193 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t6HmN4Kpr5F5RX/SLtCIjMpGsrE=', 'Content-Length': '0'} 2022-09-03T06:44:03.196 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ae17130ea16ead73-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.238 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:03.239 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:03.240 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.240 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.241 DEBUG:boto:Method: HEAD 2022-09-03T06:44:03.242 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.243 DEBUG:boto:Data: 2022-09-03T06:44:03.244 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.246 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.246 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.247 DEBUG:boto:Params: {} 2022-09-03T06:44:03.248 DEBUG:boto:Token: None 2022-09-03T06:44:03.249 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.250 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uCAwH+u/mMtUN8jhZkEGikg4/Hc= 2022-09-03T06:44:03.251 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uCAwH+u/mMtUN8jhZkEGikg4/Hc=', 'Content-Length': '0'} 2022-09-03T06:44:03.254 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000969144eb5c44d48f-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.255 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:03.256 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:03.256 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.257 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.258 DEBUG:boto:Method: HEAD 2022-09-03T06:44:03.259 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.259 DEBUG:boto:Data: 2022-09-03T06:44:03.260 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.260 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.261 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.261 DEBUG:boto:Params: {} 2022-09-03T06:44:03.262 DEBUG:boto:Token: None 2022-09-03T06:44:03.263 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.264 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uCAwH+u/mMtUN8jhZkEGikg4/Hc= 2022-09-03T06:44:03.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uCAwH+u/mMtUN8jhZkEGikg4/Hc=', 'Content-Length': '0'} 2022-09-03T06:44:03.267 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000ae86ef297f6766f-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.268 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:44:03.269 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:03.270 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:03.270 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.271 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.272 DEBUG:boto:Method: GET 2022-09-03T06:44:03.272 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.273 DEBUG:boto:Data: 2022-09-03T06:44:03.274 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.275 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.277 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.278 DEBUG:boto:Params: {} 2022-09-03T06:44:03.279 DEBUG:boto:Token: None 2022-09-03T06:44:03.280 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.281 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t6HmN4Kpr5F5RX/SLtCIjMpGsrE= 2022-09-03T06:44:03.283 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t6HmN4Kpr5F5RX/SLtCIjMpGsrE=', 'Content-Length': '0'} 2022-09-03T06:44:03.286 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d09cc08583d75d8f-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.286 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:03.287 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:03.288 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.288 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.289 DEBUG:boto:Method: GET 2022-09-03T06:44:03.290 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.291 DEBUG:boto:Data: 2022-09-03T06:44:03.292 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.293 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.293 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.294 DEBUG:boto:Params: {} 2022-09-03T06:44:03.296 DEBUG:boto:Token: None 2022-09-03T06:44:03.297 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:03.298 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t6HmN4Kpr5F5RX/SLtCIjMpGsrE= 2022-09-03T06:44:03.299 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t6HmN4Kpr5F5RX/SLtCIjMpGsrE=', 'Content-Length': '0'} 2022-09-03T06:44:03.302 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007d07c3db9d79fa16-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.303 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:03.304 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:03.305 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:03.306 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:03.307 DEBUG:boto:Method: GET 2022-09-03T06:44:03.307 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:44:03.308 DEBUG:boto:Data: 2022-09-03T06:44:03.308 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.309 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.310 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.310 DEBUG:boto:Params: {} 2022-09-03T06:44:03.311 DEBUG:boto:Token: None 2022-09-03T06:44:03.312 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/ 2022-09-03T06:44:03.312 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY= 2022-09-03T06:44:03.313 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY=', 'Content-Length': '0'} 2022-09-03T06:44:03.316 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000074955f47a2f0e0fe-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.317 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:03.318 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:03.319 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:03.319 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:03.320 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:03.321 DEBUG:boto:Method: GET 2022-09-03T06:44:03.321 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:44:03.322 DEBUG:boto:Data: 2022-09-03T06:44:03.323 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.323 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.324 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.324 DEBUG:boto:Params: {} 2022-09-03T06:44:03.325 DEBUG:boto:Token: None 2022-09-03T06:44:03.326 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/ 2022-09-03T06:44:03.326 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY= 2022-09-03T06:44:03.327 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY=', 'Content-Length': '0'} 2022-09-03T06:44:03.332 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f3bf731eed43f01e-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.333 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:03.335 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:03.336 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:03.337 DEBUG:boto:Method: HEAD 2022-09-03T06:44:03.338 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:44:03.338 DEBUG:boto:Data: 2022-09-03T06:44:03.339 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.339 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.340 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.340 DEBUG:boto:Params: {} 2022-09-03T06:44:03.341 DEBUG:boto:Token: None 2022-09-03T06:44:03.342 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj34 2022-09-03T06:44:03.343 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tJ/GM6VHWjw5J4qlo6uFGa45TYw= 2022-09-03T06:44:03.343 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tJ/GM6VHWjw5J4qlo6uFGa45TYw=', 'Content-Length': '0'} 2022-09-03T06:44:03.345 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006a9c7f05e63e7f55-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.346 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:03.347 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:03.348 DEBUG:boto:Method: HEAD 2022-09-03T06:44:03.349 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:44:03.349 DEBUG:boto:Data: 2022-09-03T06:44:03.350 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.351 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.352 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.353 DEBUG:boto:Params: {} 2022-09-03T06:44:03.354 DEBUG:boto:Token: None 2022-09-03T06:44:03.356 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj34 2022-09-03T06:44:03.357 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tJ/GM6VHWjw5J4qlo6uFGa45TYw= 2022-09-03T06:44:03.358 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tJ/GM6VHWjw5J4qlo6uFGa45TYw=', 'Content-Length': '0'} 2022-09-03T06:44:03.360 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000062e390b608d6a449-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.361 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:44:03.361 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:03.362 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:03.362 DEBUG:boto:Method: GET 2022-09-03T06:44:03.363 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:44:03.363 DEBUG:boto:Data: 2022-09-03T06:44:03.364 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.364 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.365 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.365 DEBUG:boto:Params: {} 2022-09-03T06:44:03.366 DEBUG:boto:Token: None 2022-09-03T06:44:03.367 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj34 2022-09-03T06:44:03.367 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:embWZ77NZ4ZGNj/k+ONkxh6OXMA= 2022-09-03T06:44:03.368 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:embWZ77NZ4ZGNj/k+ONkxh6OXMA=', 'Content-Length': '0'} 2022-09-03T06:44:03.370 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e49202fd49c010b8-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.371 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:03.372 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:03.372 DEBUG:boto:Method: GET 2022-09-03T06:44:03.373 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:44:03.373 DEBUG:boto:Data: 2022-09-03T06:44:03.374 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.375 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.376 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.376 DEBUG:boto:Params: {} 2022-09-03T06:44:03.377 DEBUG:boto:Token: None 2022-09-03T06:44:03.378 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj34 2022-09-03T06:44:03.378 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:embWZ77NZ4ZGNj/k+ONkxh6OXMA= 2022-09-03T06:44:03.379 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:embWZ77NZ4ZGNj/k+ONkxh6OXMA=', 'Content-Length': '0'} 2022-09-03T06:44:03.382 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000068e3e486efd2d7db-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.422 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:03.423 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:03.423 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:03.424 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:03.424 DEBUG:boto:Method: GET 2022-09-03T06:44:03.425 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:44:03.425 DEBUG:boto:Data: 2022-09-03T06:44:03.426 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.427 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.427 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.428 DEBUG:boto:Params: {} 2022-09-03T06:44:03.429 DEBUG:boto:Token: None 2022-09-03T06:44:03.430 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/ 2022-09-03T06:44:03.431 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY= 2022-09-03T06:44:03.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY=', 'Content-Length': '0'} 2022-09-03T06:44:03.436 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000a3891cfe6f7e242-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.437 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:03.439 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:03.439 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:03.440 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:03.441 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:03.442 DEBUG:boto:Method: GET 2022-09-03T06:44:03.443 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:44:03.444 DEBUG:boto:Data: 2022-09-03T06:44:03.444 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.445 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.445 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.446 DEBUG:boto:Params: {} 2022-09-03T06:44:03.447 DEBUG:boto:Token: None 2022-09-03T06:44:03.447 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/ 2022-09-03T06:44:03.448 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY= 2022-09-03T06:44:03.448 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY=', 'Content-Length': '0'} 2022-09-03T06:44:03.452 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003c242b11647f288b-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.453 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:03.454 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:44:03.454 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:03.455 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:03.456 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.456 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.457 DEBUG:boto:Method: GET 2022-09-03T06:44:03.457 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.458 DEBUG:boto:Data: 2022-09-03T06:44:03.458 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.459 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.459 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.460 DEBUG:boto:Params: {} 2022-09-03T06:44:03.461 DEBUG:boto:Token: None 2022-09-03T06:44:03.462 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.462 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M/xal1GO9tq4IvVz4SgaNmxf1JQ= 2022-09-03T06:44:03.463 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M/xal1GO9tq4IvVz4SgaNmxf1JQ=', 'Content-Length': '0'} 2022-09-03T06:44:03.465 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000afc68ea76a67b468-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.466 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:03.467 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:03.467 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.468 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.468 DEBUG:boto:Method: GET 2022-09-03T06:44:03.469 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.469 DEBUG:boto:Data: 2022-09-03T06:44:03.470 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.471 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.471 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.472 DEBUG:boto:Params: {} 2022-09-03T06:44:03.473 DEBUG:boto:Token: None 2022-09-03T06:44:03.473 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.474 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M/xal1GO9tq4IvVz4SgaNmxf1JQ= 2022-09-03T06:44:03.475 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M/xal1GO9tq4IvVz4SgaNmxf1JQ=', 'Content-Length': '0'} 2022-09-03T06:44:03.477 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fa305e2e9230d7b4-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.478 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:03.478 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:03.479 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.479 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.480 DEBUG:boto:Method: HEAD 2022-09-03T06:44:03.480 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.481 DEBUG:boto:Data: 2022-09-03T06:44:03.482 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.482 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.483 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.483 DEBUG:boto:Params: {} 2022-09-03T06:44:03.484 DEBUG:boto:Token: None 2022-09-03T06:44:03.484 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.485 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HtEU6+0PzpjE6urgiKMEQRtpqZk= 2022-09-03T06:44:03.486 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HtEU6+0PzpjE6urgiKMEQRtpqZk=', 'Content-Length': '0'} 2022-09-03T06:44:03.488 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000070354f26b6eb4ff7-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.489 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:03.489 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:03.490 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.490 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.491 DEBUG:boto:Method: HEAD 2022-09-03T06:44:03.491 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.492 DEBUG:boto:Data: 2022-09-03T06:44:03.492 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.493 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.493 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.494 DEBUG:boto:Params: {} 2022-09-03T06:44:03.494 DEBUG:boto:Token: None 2022-09-03T06:44:03.495 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.496 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HtEU6+0PzpjE6urgiKMEQRtpqZk= 2022-09-03T06:44:03.496 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HtEU6+0PzpjE6urgiKMEQRtpqZk=', 'Content-Length': '0'} 2022-09-03T06:44:03.498 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000046f4f6165cbdb3bc-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.499 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:44:03.500 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:03.501 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:03.502 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.502 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.503 DEBUG:boto:Method: GET 2022-09-03T06:44:03.504 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.504 DEBUG:boto:Data: 2022-09-03T06:44:03.505 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.506 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.506 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.507 DEBUG:boto:Params: {} 2022-09-03T06:44:03.508 DEBUG:boto:Token: None 2022-09-03T06:44:03.508 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.509 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M/xal1GO9tq4IvVz4SgaNmxf1JQ= 2022-09-03T06:44:03.510 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M/xal1GO9tq4IvVz4SgaNmxf1JQ=', 'Content-Length': '0'} 2022-09-03T06:44:03.512 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000028d2e829b9c505aa-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.513 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:03.514 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:03.514 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.515 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.515 DEBUG:boto:Method: GET 2022-09-03T06:44:03.516 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.516 DEBUG:boto:Data: 2022-09-03T06:44:03.517 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.518 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.518 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.519 DEBUG:boto:Params: {} 2022-09-03T06:44:03.519 DEBUG:boto:Token: None 2022-09-03T06:44:03.520 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:03.520 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M/xal1GO9tq4IvVz4SgaNmxf1JQ= 2022-09-03T06:44:03.521 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M/xal1GO9tq4IvVz4SgaNmxf1JQ=', 'Content-Length': '0'} 2022-09-03T06:44:03.538 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002cdaaec4dea60540-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.582 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:03.583 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:03.584 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:03.584 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:03.585 DEBUG:boto:Method: GET 2022-09-03T06:44:03.586 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:44:03.587 DEBUG:boto:Data: 2022-09-03T06:44:03.588 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.589 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.591 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.592 DEBUG:boto:Params: {} 2022-09-03T06:44:03.593 DEBUG:boto:Token: None 2022-09-03T06:44:03.593 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/ 2022-09-03T06:44:03.594 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY= 2022-09-03T06:44:03.595 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY=', 'Content-Length': '0'} 2022-09-03T06:44:03.599 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009dcedb9ff830c9a9-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.600 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:03.601 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:03.602 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:03.603 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:03.603 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:03.604 DEBUG:boto:Method: GET 2022-09-03T06:44:03.604 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:44:03.605 DEBUG:boto:Data: 2022-09-03T06:44:03.606 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.606 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.607 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.607 DEBUG:boto:Params: {} 2022-09-03T06:44:03.608 DEBUG:boto:Token: None 2022-09-03T06:44:03.609 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/ 2022-09-03T06:44:03.609 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY= 2022-09-03T06:44:03.610 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY=', 'Content-Length': '0'} 2022-09-03T06:44:03.613 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000017fba7f4b5b14612-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.614 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:03.615 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:03.616 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:03.617 DEBUG:boto:Method: HEAD 2022-09-03T06:44:03.618 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:44:03.619 DEBUG:boto:Data: 2022-09-03T06:44:03.620 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.621 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.622 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.623 DEBUG:boto:Params: {} 2022-09-03T06:44:03.624 DEBUG:boto:Token: None 2022-09-03T06:44:03.625 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj38 2022-09-03T06:44:03.626 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:08uWL/OK+rLDVNpukY+TmD13ZNQ= 2022-09-03T06:44:03.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:08uWL/OK+rLDVNpukY+TmD13ZNQ=', 'Content-Length': '0'} 2022-09-03T06:44:03.631 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000506a0cdb033d2162-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.632 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:03.633 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:03.634 DEBUG:boto:Method: HEAD 2022-09-03T06:44:03.634 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:44:03.635 DEBUG:boto:Data: 2022-09-03T06:44:03.636 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.636 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.637 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.638 DEBUG:boto:Params: {} 2022-09-03T06:44:03.639 DEBUG:boto:Token: None 2022-09-03T06:44:03.639 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj38 2022-09-03T06:44:03.640 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:08uWL/OK+rLDVNpukY+TmD13ZNQ= 2022-09-03T06:44:03.641 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:08uWL/OK+rLDVNpukY+TmD13ZNQ=', 'Content-Length': '0'} 2022-09-03T06:44:03.643 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ec449908e6279c16-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.644 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:44:03.645 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:03.646 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:03.647 DEBUG:boto:Method: GET 2022-09-03T06:44:03.647 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:44:03.648 DEBUG:boto:Data: 2022-09-03T06:44:03.649 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.650 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.651 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.652 DEBUG:boto:Params: {} 2022-09-03T06:44:03.653 DEBUG:boto:Token: None 2022-09-03T06:44:03.654 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj38 2022-09-03T06:44:03.655 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:psOQV3R/J+5HNl9v2Y2eeAoJ1bo= 2022-09-03T06:44:03.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:psOQV3R/J+5HNl9v2Y2eeAoJ1bo=', 'Content-Length': '0'} 2022-09-03T06:44:03.659 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cee6783abe28151f-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.708 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:03.709 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:03.710 DEBUG:boto:Method: GET 2022-09-03T06:44:03.711 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:44:03.711 DEBUG:boto:Data: 2022-09-03T06:44:03.712 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.712 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.713 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.713 DEBUG:boto:Params: {} 2022-09-03T06:44:03.714 DEBUG:boto:Token: None 2022-09-03T06:44:03.715 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj38 2022-09-03T06:44:03.715 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:psOQV3R/J+5HNl9v2Y2eeAoJ1bo= 2022-09-03T06:44:03.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:psOQV3R/J+5HNl9v2Y2eeAoJ1bo=', 'Content-Length': '0'} 2022-09-03T06:44:03.719 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000efeb8cfd63404202-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.721 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:03.722 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:03.723 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:03.723 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:03.724 DEBUG:boto:Method: GET 2022-09-03T06:44:03.725 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:44:03.726 DEBUG:boto:Data: 2022-09-03T06:44:03.727 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.728 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.729 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.729 DEBUG:boto:Params: {} 2022-09-03T06:44:03.730 DEBUG:boto:Token: None 2022-09-03T06:44:03.731 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/ 2022-09-03T06:44:03.731 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY= 2022-09-03T06:44:03.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY=', 'Content-Length': '0'} 2022-09-03T06:44:03.735 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002ba6880d55c2ec79-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.736 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:03.737 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:03.737 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:03.738 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:03.739 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:03.740 DEBUG:boto:Method: GET 2022-09-03T06:44:03.740 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:44:03.741 DEBUG:boto:Data: 2022-09-03T06:44:03.742 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.742 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.743 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.744 DEBUG:boto:Params: {} 2022-09-03T06:44:03.744 DEBUG:boto:Token: None 2022-09-03T06:44:03.745 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/ 2022-09-03T06:44:03.745 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY= 2022-09-03T06:44:03.746 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY=', 'Content-Length': '0'} 2022-09-03T06:44:03.750 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b56f94898b46138a-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.751 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:03.752 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:44:03.752 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:03.753 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:03.753 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.754 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.754 DEBUG:boto:Method: GET 2022-09-03T06:44:03.755 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.756 DEBUG:boto:Data: 2022-09-03T06:44:03.756 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.757 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.758 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.758 DEBUG:boto:Params: {} 2022-09-03T06:44:03.759 DEBUG:boto:Token: None 2022-09-03T06:44:03.761 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.762 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:84T2brAzTKJcqKRHjFdwPyV0LqE= 2022-09-03T06:44:03.763 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:84T2brAzTKJcqKRHjFdwPyV0LqE=', 'Content-Length': '0'} 2022-09-03T06:44:03.765 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b8ee0be2c4e7637f-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.767 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:03.768 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:03.768 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.769 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.769 DEBUG:boto:Method: GET 2022-09-03T06:44:03.770 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.771 DEBUG:boto:Data: 2022-09-03T06:44:03.772 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.772 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.773 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.774 DEBUG:boto:Params: {} 2022-09-03T06:44:03.775 DEBUG:boto:Token: None 2022-09-03T06:44:03.775 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.777 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:84T2brAzTKJcqKRHjFdwPyV0LqE= 2022-09-03T06:44:03.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:84T2brAzTKJcqKRHjFdwPyV0LqE=', 'Content-Length': '0'} 2022-09-03T06:44:03.780 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f8f73864f955cdb4-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.781 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:03.782 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:03.782 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.783 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.783 DEBUG:boto:Method: HEAD 2022-09-03T06:44:03.784 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.784 DEBUG:boto:Data: 2022-09-03T06:44:03.785 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.785 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.786 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.787 DEBUG:boto:Params: {} 2022-09-03T06:44:03.787 DEBUG:boto:Token: None 2022-09-03T06:44:03.788 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.789 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hCITr3SSqNDf+bIqa/iqChBodzk= 2022-09-03T06:44:03.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hCITr3SSqNDf+bIqa/iqChBodzk=', 'Content-Length': '0'} 2022-09-03T06:44:03.791 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a08f5fbfc786161-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.792 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:03.793 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:03.794 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.795 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.796 DEBUG:boto:Method: HEAD 2022-09-03T06:44:03.796 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.797 DEBUG:boto:Data: 2022-09-03T06:44:03.798 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.799 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.800 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.804 DEBUG:boto:Params: {} 2022-09-03T06:44:03.806 DEBUG:boto:Token: None 2022-09-03T06:44:03.807 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.808 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hCITr3SSqNDf+bIqa/iqChBodzk= 2022-09-03T06:44:03.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hCITr3SSqNDf+bIqa/iqChBodzk=', 'Content-Length': '0'} 2022-09-03T06:44:03.813 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d3036971bdfade2-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.813 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:44:03.814 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:03.815 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:03.816 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.816 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.817 DEBUG:boto:Method: GET 2022-09-03T06:44:03.818 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.818 DEBUG:boto:Data: 2022-09-03T06:44:03.819 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.819 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.820 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.821 DEBUG:boto:Params: {} 2022-09-03T06:44:03.821 DEBUG:boto:Token: None 2022-09-03T06:44:03.823 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.824 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:84T2brAzTKJcqKRHjFdwPyV0LqE= 2022-09-03T06:44:03.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:84T2brAzTKJcqKRHjFdwPyV0LqE=', 'Content-Length': '0'} 2022-09-03T06:44:03.826 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000529a469d52bc5d5a-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.827 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:03.828 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:03.829 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.829 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.830 DEBUG:boto:Method: GET 2022-09-03T06:44:03.830 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.831 DEBUG:boto:Data: 2022-09-03T06:44:03.832 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.832 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.833 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.834 DEBUG:boto:Params: {} 2022-09-03T06:44:03.834 DEBUG:boto:Token: None 2022-09-03T06:44:03.835 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:03.836 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:84T2brAzTKJcqKRHjFdwPyV0LqE= 2022-09-03T06:44:03.837 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:84T2brAzTKJcqKRHjFdwPyV0LqE=', 'Content-Length': '0'} 2022-09-03T06:44:03.839 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ec24c095912ff012-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.840 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:03.841 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:03.842 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:03.842 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:03.843 DEBUG:boto:Method: GET 2022-09-03T06:44:03.843 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:44:03.844 DEBUG:boto:Data: 2022-09-03T06:44:03.845 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.845 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.846 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.846 DEBUG:boto:Params: {} 2022-09-03T06:44:03.847 DEBUG:boto:Token: None 2022-09-03T06:44:03.848 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/ 2022-09-03T06:44:03.848 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY= 2022-09-03T06:44:03.849 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY=', 'Content-Length': '0'} 2022-09-03T06:44:03.852 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007f19cff472e204ca-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.894 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:03.895 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:03.896 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:03.897 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:03.897 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:03.898 DEBUG:boto:Method: GET 2022-09-03T06:44:03.898 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:44:03.899 DEBUG:boto:Data: 2022-09-03T06:44:03.900 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.900 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.901 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.902 DEBUG:boto:Params: {} 2022-09-03T06:44:03.902 DEBUG:boto:Token: None 2022-09-03T06:44:03.903 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/ 2022-09-03T06:44:03.904 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY= 2022-09-03T06:44:03.905 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY=', 'Content-Length': '0'} 2022-09-03T06:44:03.910 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000255eae47c08da0cb-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.911 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:03.913 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:03.913 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:03.914 DEBUG:boto:Method: HEAD 2022-09-03T06:44:03.915 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:44:03.916 DEBUG:boto:Data: 2022-09-03T06:44:03.917 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.918 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.919 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.919 DEBUG:boto:Params: {} 2022-09-03T06:44:03.920 DEBUG:boto:Token: None 2022-09-03T06:44:03.921 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj44 2022-09-03T06:44:03.921 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:occmbu1z/urL6hhtlveoYH1/XhM= 2022-09-03T06:44:03.922 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:occmbu1z/urL6hhtlveoYH1/XhM=', 'Content-Length': '0'} 2022-09-03T06:44:03.924 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000971dccc56e68a823-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.926 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:03.927 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:03.928 DEBUG:boto:Method: HEAD 2022-09-03T06:44:03.929 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:44:03.930 DEBUG:boto:Data: 2022-09-03T06:44:03.930 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.931 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.931 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.932 DEBUG:boto:Params: {} 2022-09-03T06:44:03.933 DEBUG:boto:Token: None 2022-09-03T06:44:03.933 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj44 2022-09-03T06:44:03.934 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:occmbu1z/urL6hhtlveoYH1/XhM= 2022-09-03T06:44:03.934 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:occmbu1z/urL6hhtlveoYH1/XhM=', 'Content-Length': '0'} 2022-09-03T06:44:03.937 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db792d617343744b-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.938 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:44:03.939 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:03.940 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:03.940 DEBUG:boto:Method: GET 2022-09-03T06:44:03.941 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:44:03.942 DEBUG:boto:Data: 2022-09-03T06:44:03.943 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.944 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.944 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.945 DEBUG:boto:Params: {} 2022-09-03T06:44:03.946 DEBUG:boto:Token: None 2022-09-03T06:44:03.947 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj44 2022-09-03T06:44:03.948 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r4hpxIfV2xbMkRR79CQX++Fr/fs= 2022-09-03T06:44:03.949 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r4hpxIfV2xbMkRR79CQX++Fr/fs=', 'Content-Length': '0'} 2022-09-03T06:44:03.952 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008252ccf38ad22a1b-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.953 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:03.953 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:03.954 DEBUG:boto:Method: GET 2022-09-03T06:44:03.955 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:44:03.955 DEBUG:boto:Data: 2022-09-03T06:44:03.956 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.957 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.957 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.958 DEBUG:boto:Params: {} 2022-09-03T06:44:03.959 DEBUG:boto:Token: None 2022-09-03T06:44:03.959 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj44 2022-09-03T06:44:03.960 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r4hpxIfV2xbMkRR79CQX++Fr/fs= 2022-09-03T06:44:03.961 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r4hpxIfV2xbMkRR79CQX++Fr/fs=', 'Content-Length': '0'} 2022-09-03T06:44:03.963 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009526f0758a5006ff-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.964 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:03.965 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:03.965 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:03.966 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:03.967 DEBUG:boto:Method: GET 2022-09-03T06:44:03.967 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:44:03.968 DEBUG:boto:Data: 2022-09-03T06:44:03.968 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.969 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.969 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.970 DEBUG:boto:Params: {} 2022-09-03T06:44:03.971 DEBUG:boto:Token: None 2022-09-03T06:44:03.971 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/ 2022-09-03T06:44:03.972 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY= 2022-09-03T06:44:03.973 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY=', 'Content-Length': '0'} 2022-09-03T06:44:03.976 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000945fee6bfabe92f4-006312f7b3-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.977 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:03.978 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:03.978 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:03.979 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:03.980 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:03.980 DEBUG:boto:Method: GET 2022-09-03T06:44:03.981 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:44:03.981 DEBUG:boto:Data: 2022-09-03T06:44:03.982 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.983 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:03.983 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.984 DEBUG:boto:Params: {} 2022-09-03T06:44:03.984 DEBUG:boto:Token: None 2022-09-03T06:44:03.985 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/ 2022-09-03T06:44:03.986 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY= 2022-09-03T06:44:03.986 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zTAEmHK0wWJi+P+6t/ZA+z4VsdY=', 'Content-Length': '0'} 2022-09-03T06:44:03.990 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e5b020538577d05f-006312f7b3-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:03.990 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:03.991 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:44:03.992 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:03.993 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:03.993 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:03.994 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:03.994 DEBUG:boto:Method: GET 2022-09-03T06:44:03.995 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:03.995 DEBUG:boto:Data: 2022-09-03T06:44:03.996 DEBUG:boto:Headers: {} 2022-09-03T06:44:03.996 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:03.997 DEBUG:boto:Port: 8000 2022-09-03T06:44:03.998 DEBUG:boto:Params: {} 2022-09-03T06:44:03.998 DEBUG:boto:Token: None 2022-09-03T06:44:03.999 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:03 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:04.000 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LiCjrJsQ5K5h0aLKfiPs72SbwNU= 2022-09-03T06:44:04.000 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LiCjrJsQ5K5h0aLKfiPs72SbwNU=', 'Content-Length': '0'} 2022-09-03T06:44:04.003 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cad45fea4e7bd82d-006312f7b4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.004 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:04.004 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:04.005 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:04.005 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:04.006 DEBUG:boto:Method: GET 2022-09-03T06:44:04.007 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:04.007 DEBUG:boto:Data: 2022-09-03T06:44:04.008 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.009 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:04.009 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.010 DEBUG:boto:Params: {} 2022-09-03T06:44:04.011 DEBUG:boto:Token: None 2022-09-03T06:44:04.011 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:04.012 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OXqmUN86vGrtVY7sIHhvGqjtK6k= 2022-09-03T06:44:04.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OXqmUN86vGrtVY7sIHhvGqjtK6k=', 'Content-Length': '0'} 2022-09-03T06:44:04.015 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002e51b40d32c16590-006312f7b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.016 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:04.016 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:04.017 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:04.018 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:04.018 DEBUG:boto:Method: HEAD 2022-09-03T06:44:04.019 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:04.019 DEBUG:boto:Data: 2022-09-03T06:44:04.020 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.021 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:04.022 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.022 DEBUG:boto:Params: {} 2022-09-03T06:44:04.023 DEBUG:boto:Token: None 2022-09-03T06:44:04.023 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:04.024 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JnNKnbzWmJJf30Y9uOfNu0LJt3g= 2022-09-03T06:44:04.025 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JnNKnbzWmJJf30Y9uOfNu0LJt3g=', 'Content-Length': '0'} 2022-09-03T06:44:04.027 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a14c3d91238db18d-006312f7b4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.028 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:04.029 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:04.029 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:04.030 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:04.030 DEBUG:boto:Method: HEAD 2022-09-03T06:44:04.031 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:04.031 DEBUG:boto:Data: 2022-09-03T06:44:04.032 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.032 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:04.033 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.033 DEBUG:boto:Params: {} 2022-09-03T06:44:04.034 DEBUG:boto:Token: None 2022-09-03T06:44:04.035 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:04.035 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JnNKnbzWmJJf30Y9uOfNu0LJt3g= 2022-09-03T06:44:04.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JnNKnbzWmJJf30Y9uOfNu0LJt3g=', 'Content-Length': '0'} 2022-09-03T06:44:04.038 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b44e35e42448971-006312f7b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.039 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:44:04.039 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:04.040 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:04.041 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:04.042 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:04.043 DEBUG:boto:Method: GET 2022-09-03T06:44:04.044 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:04.045 DEBUG:boto:Data: 2022-09-03T06:44:04.046 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.047 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:04.048 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.049 DEBUG:boto:Params: {} 2022-09-03T06:44:04.050 DEBUG:boto:Token: None 2022-09-03T06:44:04.050 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:04.051 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OXqmUN86vGrtVY7sIHhvGqjtK6k= 2022-09-03T06:44:04.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OXqmUN86vGrtVY7sIHhvGqjtK6k=', 'Content-Length': '0'} 2022-09-03T06:44:04.054 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008fbe8fac909d2c59-006312f7b4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.055 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:04.056 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:04.056 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:04.057 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:04.058 DEBUG:boto:Method: GET 2022-09-03T06:44:04.059 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:04.060 DEBUG:boto:Data: 2022-09-03T06:44:04.061 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.062 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:04.063 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.064 DEBUG:boto:Params: {} 2022-09-03T06:44:04.065 DEBUG:boto:Token: None 2022-09-03T06:44:04.066 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:04.067 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OXqmUN86vGrtVY7sIHhvGqjtK6k= 2022-09-03T06:44:04.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OXqmUN86vGrtVY7sIHhvGqjtK6k=', 'Content-Length': '0'} 2022-09-03T06:44:04.070 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005615e55396af6388-006312f7b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.110 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:04.111 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:04.112 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:04.113 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:04.114 DEBUG:boto:Method: GET 2022-09-03T06:44:04.114 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:44:04.115 DEBUG:boto:Data: 2022-09-03T06:44:04.116 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.116 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:04.117 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.117 DEBUG:boto:Params: {} 2022-09-03T06:44:04.118 DEBUG:boto:Token: None 2022-09-03T06:44:04.119 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/ 2022-09-03T06:44:04.119 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws= 2022-09-03T06:44:04.120 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws=', 'Content-Length': '0'} 2022-09-03T06:44:04.123 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000021177dd816308dad-006312f7b4-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.125 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:04.126 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:04.127 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:04.128 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:04.129 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:04.130 DEBUG:boto:Method: GET 2022-09-03T06:44:04.131 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:44:04.132 DEBUG:boto:Data: 2022-09-03T06:44:04.133 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.133 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:04.134 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.136 DEBUG:boto:Params: {} 2022-09-03T06:44:04.137 DEBUG:boto:Token: None 2022-09-03T06:44:04.137 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/ 2022-09-03T06:44:04.138 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws= 2022-09-03T06:44:04.139 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws=', 'Content-Length': '0'} 2022-09-03T06:44:04.144 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000064bcfc13d3a087b5-006312f7b4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.145 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:04.147 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:04.148 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:04.149 DEBUG:boto:Method: HEAD 2022-09-03T06:44:04.150 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:44:04.150 DEBUG:boto:Data: 2022-09-03T06:44:04.151 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.151 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:04.152 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.152 DEBUG:boto:Params: {} 2022-09-03T06:44:04.153 DEBUG:boto:Token: None 2022-09-03T06:44:04.154 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj46 2022-09-03T06:44:04.154 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:puwig/BPp7BwCpec6WqCWJb37Ts= 2022-09-03T06:44:04.155 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:puwig/BPp7BwCpec6WqCWJb37Ts=', 'Content-Length': '0'} 2022-09-03T06:44:04.158 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d3d596eb87fb8c59-006312f7b4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.159 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:04.160 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:04.160 DEBUG:boto:Method: HEAD 2022-09-03T06:44:04.161 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:44:04.161 DEBUG:boto:Data: 2022-09-03T06:44:04.162 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.162 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:04.163 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.164 DEBUG:boto:Params: {} 2022-09-03T06:44:04.164 DEBUG:boto:Token: None 2022-09-03T06:44:04.165 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj46 2022-09-03T06:44:04.165 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:puwig/BPp7BwCpec6WqCWJb37Ts= 2022-09-03T06:44:04.166 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:puwig/BPp7BwCpec6WqCWJb37Ts=', 'Content-Length': '0'} 2022-09-03T06:44:04.168 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a3baa8b7d84faf8-006312f7b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.169 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:44:04.170 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:04.171 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:04.171 DEBUG:boto:Method: GET 2022-09-03T06:44:04.172 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:44:04.173 DEBUG:boto:Data: 2022-09-03T06:44:04.173 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.174 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:04.175 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.175 DEBUG:boto:Params: {} 2022-09-03T06:44:04.176 DEBUG:boto:Token: None 2022-09-03T06:44:04.177 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj46 2022-09-03T06:44:04.177 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CrIzivIe8lj1mi8tc7AuFP67+iw= 2022-09-03T06:44:04.178 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CrIzivIe8lj1mi8tc7AuFP67+iw=', 'Content-Length': '0'} 2022-09-03T06:44:04.180 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a5204401be23237f-006312f7b4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.222 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:04.223 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:04.224 DEBUG:boto:Method: GET 2022-09-03T06:44:04.225 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:44:04.226 DEBUG:boto:Data: 2022-09-03T06:44:04.227 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.229 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:04.230 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.231 DEBUG:boto:Params: {} 2022-09-03T06:44:04.232 DEBUG:boto:Token: None 2022-09-03T06:44:04.233 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj46 2022-09-03T06:44:04.234 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CrIzivIe8lj1mi8tc7AuFP67+iw= 2022-09-03T06:44:04.235 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CrIzivIe8lj1mi8tc7AuFP67+iw=', 'Content-Length': '0'} 2022-09-03T06:44:04.238 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000590c41ec834875cc-006312f7b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.238 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:04.239 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:04.240 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:04.241 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:04.241 DEBUG:boto:Method: GET 2022-09-03T06:44:04.242 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:44:04.242 DEBUG:boto:Data: 2022-09-03T06:44:04.243 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.244 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:04.244 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.245 DEBUG:boto:Params: {} 2022-09-03T06:44:04.245 DEBUG:boto:Token: None 2022-09-03T06:44:04.246 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/ 2022-09-03T06:44:04.247 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws= 2022-09-03T06:44:04.247 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws=', 'Content-Length': '0'} 2022-09-03T06:44:04.250 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000652cb7ffa1169e60-006312f7b4-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.251 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:04.253 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:04.254 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:04.255 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:04.255 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:04.256 DEBUG:boto:Method: GET 2022-09-03T06:44:04.257 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:44:04.257 DEBUG:boto:Data: 2022-09-03T06:44:04.258 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.259 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:04.260 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.261 DEBUG:boto:Params: {} 2022-09-03T06:44:04.263 DEBUG:boto:Token: None 2022-09-03T06:44:04.264 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/ 2022-09-03T06:44:04.264 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws= 2022-09-03T06:44:04.265 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws=', 'Content-Length': '0'} 2022-09-03T06:44:04.270 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000038b12ead0e1ac274-006312f7b4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.271 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:04.273 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:44:04.274 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:04.275 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:04.276 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.277 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.278 DEBUG:boto:Method: GET 2022-09-03T06:44:04.279 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.280 DEBUG:boto:Data: 2022-09-03T06:44:04.280 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.281 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:04.282 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.282 DEBUG:boto:Params: {} 2022-09-03T06:44:04.283 DEBUG:boto:Token: None 2022-09-03T06:44:04.283 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.284 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OaczJJ3Ssl0GLMNWdh8uCuSphXQ= 2022-09-03T06:44:04.285 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OaczJJ3Ssl0GLMNWdh8uCuSphXQ=', 'Content-Length': '0'} 2022-09-03T06:44:04.288 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006df7b51a8b48a233-006312f7b4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.290 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:04.292 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:04.295 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.296 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.297 DEBUG:boto:Method: GET 2022-09-03T06:44:04.298 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.299 DEBUG:boto:Data: 2022-09-03T06:44:04.299 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.300 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:04.301 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.301 DEBUG:boto:Params: {} 2022-09-03T06:44:04.302 DEBUG:boto:Token: None 2022-09-03T06:44:04.303 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.304 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OaczJJ3Ssl0GLMNWdh8uCuSphXQ= 2022-09-03T06:44:04.305 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OaczJJ3Ssl0GLMNWdh8uCuSphXQ=', 'Content-Length': '0'} 2022-09-03T06:44:04.309 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005946f930a0097d56-006312f7b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.311 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:04.311 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:04.312 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.313 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.314 DEBUG:boto:Method: HEAD 2022-09-03T06:44:04.315 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.316 DEBUG:boto:Data: 2022-09-03T06:44:04.318 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.319 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:04.320 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.321 DEBUG:boto:Params: {} 2022-09-03T06:44:04.322 DEBUG:boto:Token: None 2022-09-03T06:44:04.323 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.324 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MLh/gyONDLIT3Jt2oFsTJCM/2OM= 2022-09-03T06:44:04.325 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MLh/gyONDLIT3Jt2oFsTJCM/2OM=', 'Content-Length': '0'} 2022-09-03T06:44:04.334 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d7aff877652a0be-006312f7b4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.335 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:04.336 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:04.337 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.338 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.339 DEBUG:boto:Method: HEAD 2022-09-03T06:44:04.340 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.342 DEBUG:boto:Data: 2022-09-03T06:44:04.343 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.344 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:04.344 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.345 DEBUG:boto:Params: {} 2022-09-03T06:44:04.346 DEBUG:boto:Token: None 2022-09-03T06:44:04.347 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.348 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MLh/gyONDLIT3Jt2oFsTJCM/2OM= 2022-09-03T06:44:04.349 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MLh/gyONDLIT3Jt2oFsTJCM/2OM=', 'Content-Length': '0'} 2022-09-03T06:44:04.352 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006d151acb54d19975-006312f7b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.353 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:44:04.354 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:04.355 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:04.356 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.357 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.358 DEBUG:boto:Method: GET 2022-09-03T06:44:04.359 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.360 DEBUG:boto:Data: 2022-09-03T06:44:04.361 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.362 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:04.363 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.363 DEBUG:boto:Params: {} 2022-09-03T06:44:04.365 DEBUG:boto:Token: None 2022-09-03T06:44:04.365 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OaczJJ3Ssl0GLMNWdh8uCuSphXQ= 2022-09-03T06:44:04.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OaczJJ3Ssl0GLMNWdh8uCuSphXQ=', 'Content-Length': '0'} 2022-09-03T06:44:04.369 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ef1ecdb962312b88-006312f7b4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.414 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:04.420 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:04.421 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.422 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.424 DEBUG:boto:Method: GET 2022-09-03T06:44:04.425 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.426 DEBUG:boto:Data: 2022-09-03T06:44:04.427 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.428 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:04.428 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.429 DEBUG:boto:Params: {} 2022-09-03T06:44:04.430 DEBUG:boto:Token: None 2022-09-03T06:44:04.431 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:04.432 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OaczJJ3Ssl0GLMNWdh8uCuSphXQ= 2022-09-03T06:44:04.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OaczJJ3Ssl0GLMNWdh8uCuSphXQ=', 'Content-Length': '0'} 2022-09-03T06:44:04.436 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f0654a28523df1ed-006312f7b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.436 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:04.437 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:04.438 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:04.438 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:04.439 DEBUG:boto:Method: GET 2022-09-03T06:44:04.439 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:44:04.440 DEBUG:boto:Data: 2022-09-03T06:44:04.440 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.441 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:04.442 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.443 DEBUG:boto:Params: {} 2022-09-03T06:44:04.443 DEBUG:boto:Token: None 2022-09-03T06:44:04.444 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/ 2022-09-03T06:44:04.444 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws= 2022-09-03T06:44:04.446 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws=', 'Content-Length': '0'} 2022-09-03T06:44:04.449 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bbb3a10b2d583d8c-006312f7b4-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.450 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:04.452 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:04.453 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:04.454 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:04.455 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:04.456 DEBUG:boto:Method: GET 2022-09-03T06:44:04.456 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:44:04.457 DEBUG:boto:Data: 2022-09-03T06:44:04.457 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.458 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:04.459 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.459 DEBUG:boto:Params: {} 2022-09-03T06:44:04.460 DEBUG:boto:Token: None 2022-09-03T06:44:04.460 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/ 2022-09-03T06:44:04.461 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws= 2022-09-03T06:44:04.462 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws=', 'Content-Length': '0'} 2022-09-03T06:44:04.465 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e447a84b27d8c2f8-006312f7b4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.513 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:04.530 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:04.530 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:04.531 DEBUG:boto:Method: HEAD 2022-09-03T06:44:04.531 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:44:04.532 DEBUG:boto:Data: 2022-09-03T06:44:04.532 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.533 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:04.533 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.538 DEBUG:boto:Params: {} 2022-09-03T06:44:04.539 DEBUG:boto:Token: None 2022-09-03T06:44:04.540 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj51 2022-09-03T06:44:04.541 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7WX3FvXLuuzzzYnlft8YGiNZjq4= 2022-09-03T06:44:04.542 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7WX3FvXLuuzzzYnlft8YGiNZjq4=', 'Content-Length': '0'} 2022-09-03T06:44:04.545 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f57d3a820b273d85-006312f7b4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.546 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:04.547 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:04.548 DEBUG:boto:Method: HEAD 2022-09-03T06:44:04.549 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:44:04.550 DEBUG:boto:Data: 2022-09-03T06:44:04.551 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.552 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:04.554 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.554 DEBUG:boto:Params: {} 2022-09-03T06:44:04.555 DEBUG:boto:Token: None 2022-09-03T06:44:04.556 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj51 2022-09-03T06:44:04.557 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7WX3FvXLuuzzzYnlft8YGiNZjq4= 2022-09-03T06:44:04.558 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7WX3FvXLuuzzzYnlft8YGiNZjq4=', 'Content-Length': '0'} 2022-09-03T06:44:04.561 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c43fff89fa46d5ae-006312f7b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.562 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:44:04.562 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:04.563 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:04.563 DEBUG:boto:Method: GET 2022-09-03T06:44:04.564 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:44:04.565 DEBUG:boto:Data: 2022-09-03T06:44:04.565 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.566 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:04.567 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.568 DEBUG:boto:Params: {} 2022-09-03T06:44:04.569 DEBUG:boto:Token: None 2022-09-03T06:44:04.570 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj51 2022-09-03T06:44:04.571 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KmdWnUiLf5IZP6vgJv9ny7KwV50= 2022-09-03T06:44:04.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KmdWnUiLf5IZP6vgJv9ny7KwV50=', 'Content-Length': '0'} 2022-09-03T06:44:04.575 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f9a5953c653eeb32-006312f7b4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.576 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:04.576 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:04.577 DEBUG:boto:Method: GET 2022-09-03T06:44:04.577 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:44:04.578 DEBUG:boto:Data: 2022-09-03T06:44:04.578 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.579 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:04.580 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.580 DEBUG:boto:Params: {} 2022-09-03T06:44:04.581 DEBUG:boto:Token: None 2022-09-03T06:44:04.582 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj51 2022-09-03T06:44:04.583 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KmdWnUiLf5IZP6vgJv9ny7KwV50= 2022-09-03T06:44:04.583 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KmdWnUiLf5IZP6vgJv9ny7KwV50=', 'Content-Length': '0'} 2022-09-03T06:44:04.586 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b1dee8957b767f5a-006312f7b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.587 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:04.588 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:04.588 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:04.589 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:04.589 DEBUG:boto:Method: GET 2022-09-03T06:44:04.590 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:44:04.591 DEBUG:boto:Data: 2022-09-03T06:44:04.591 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.592 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:04.593 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.593 DEBUG:boto:Params: {} 2022-09-03T06:44:04.594 DEBUG:boto:Token: None 2022-09-03T06:44:04.595 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/ 2022-09-03T06:44:04.595 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws= 2022-09-03T06:44:04.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws=', 'Content-Length': '0'} 2022-09-03T06:44:04.599 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eb4e2e7406628d07-006312f7b4-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.600 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:04.601 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:04.602 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:04.603 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:04.604 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:04.605 DEBUG:boto:Method: GET 2022-09-03T06:44:04.606 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:44:04.607 DEBUG:boto:Data: 2022-09-03T06:44:04.608 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.609 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:04.610 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.611 DEBUG:boto:Params: {} 2022-09-03T06:44:04.612 DEBUG:boto:Token: None 2022-09-03T06:44:04.613 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/ 2022-09-03T06:44:04.614 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws= 2022-09-03T06:44:04.615 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws=', 'Content-Length': '0'} 2022-09-03T06:44:04.619 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ea085f3e6f2c45c2-006312f7b4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.621 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:04.622 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:44:04.623 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:04.624 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:04.625 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.626 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.627 DEBUG:boto:Method: GET 2022-09-03T06:44:04.628 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.629 DEBUG:boto:Data: 2022-09-03T06:44:04.629 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.630 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:04.631 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.632 DEBUG:boto:Params: {} 2022-09-03T06:44:04.633 DEBUG:boto:Token: None 2022-09-03T06:44:04.634 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.635 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:URA7fjpWgiMfMdtRfP+M5x8xtEM= 2022-09-03T06:44:04.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:URA7fjpWgiMfMdtRfP+M5x8xtEM=', 'Content-Length': '0'} 2022-09-03T06:44:04.649 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c5c121f77e38d360-006312f7b4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.690 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:04.693 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:04.694 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.696 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.698 DEBUG:boto:Method: GET 2022-09-03T06:44:04.698 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.699 DEBUG:boto:Data: 2022-09-03T06:44:04.700 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.700 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:04.701 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.702 DEBUG:boto:Params: {} 2022-09-03T06:44:04.703 DEBUG:boto:Token: None 2022-09-03T06:44:04.704 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.705 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:URA7fjpWgiMfMdtRfP+M5x8xtEM= 2022-09-03T06:44:04.706 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:URA7fjpWgiMfMdtRfP+M5x8xtEM=', 'Content-Length': '0'} 2022-09-03T06:44:04.710 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000794cd3ab6087b6bd-006312f7b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.711 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:04.712 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:04.712 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.713 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.713 DEBUG:boto:Method: HEAD 2022-09-03T06:44:04.714 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.714 DEBUG:boto:Data: 2022-09-03T06:44:04.715 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.715 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:04.716 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.716 DEBUG:boto:Params: {} 2022-09-03T06:44:04.717 DEBUG:boto:Token: None 2022-09-03T06:44:04.718 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.718 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L4lH2NEuOUt/N8eQ6zh9hL7c6OY= 2022-09-03T06:44:04.719 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L4lH2NEuOUt/N8eQ6zh9hL7c6OY=', 'Content-Length': '0'} 2022-09-03T06:44:04.723 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bdeafcbba5441257-006312f7b4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.725 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:04.726 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:04.726 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.727 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.728 DEBUG:boto:Method: HEAD 2022-09-03T06:44:04.728 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.729 DEBUG:boto:Data: 2022-09-03T06:44:04.730 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.730 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:04.731 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.731 DEBUG:boto:Params: {} 2022-09-03T06:44:04.732 DEBUG:boto:Token: None 2022-09-03T06:44:04.733 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.733 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L4lH2NEuOUt/N8eQ6zh9hL7c6OY= 2022-09-03T06:44:04.734 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L4lH2NEuOUt/N8eQ6zh9hL7c6OY=', 'Content-Length': '0'} 2022-09-03T06:44:04.739 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b02117ded8cac550-006312f7b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.740 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:44:04.740 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:04.741 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:04.741 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.742 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.743 DEBUG:boto:Method: GET 2022-09-03T06:44:04.743 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.744 DEBUG:boto:Data: 2022-09-03T06:44:04.745 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.746 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:04.747 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.747 DEBUG:boto:Params: {} 2022-09-03T06:44:04.748 DEBUG:boto:Token: None 2022-09-03T06:44:04.750 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.751 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:URA7fjpWgiMfMdtRfP+M5x8xtEM= 2022-09-03T06:44:04.751 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:URA7fjpWgiMfMdtRfP+M5x8xtEM=', 'Content-Length': '0'} 2022-09-03T06:44:04.753 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000025d1df6d54010d3-006312f7b4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.754 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:04.755 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:04.755 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.756 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.756 DEBUG:boto:Method: GET 2022-09-03T06:44:04.757 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.757 DEBUG:boto:Data: 2022-09-03T06:44:04.758 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.758 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:04.759 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.759 DEBUG:boto:Params: {} 2022-09-03T06:44:04.760 DEBUG:boto:Token: None 2022-09-03T06:44:04.761 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:04.761 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:URA7fjpWgiMfMdtRfP+M5x8xtEM= 2022-09-03T06:44:04.762 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:URA7fjpWgiMfMdtRfP+M5x8xtEM=', 'Content-Length': '0'} 2022-09-03T06:44:04.764 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b259db5f893e0c3c-006312f7b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.766 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:04.766 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:04.767 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:04.768 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:04.768 DEBUG:boto:Method: GET 2022-09-03T06:44:04.769 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:44:04.769 DEBUG:boto:Data: 2022-09-03T06:44:04.770 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.771 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:04.771 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.772 DEBUG:boto:Params: {} 2022-09-03T06:44:04.773 DEBUG:boto:Token: None 2022-09-03T06:44:04.773 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/ 2022-09-03T06:44:04.774 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws= 2022-09-03T06:44:04.775 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws=', 'Content-Length': '0'} 2022-09-03T06:44:04.778 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000053b00c6fc74244d0-006312f7b4-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.779 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:04.780 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:04.781 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:04.782 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:04.783 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:04.783 DEBUG:boto:Method: GET 2022-09-03T06:44:04.784 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:44:04.784 DEBUG:boto:Data: 2022-09-03T06:44:04.785 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.786 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:04.786 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.787 DEBUG:boto:Params: {} 2022-09-03T06:44:04.787 DEBUG:boto:Token: None 2022-09-03T06:44:04.788 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/ 2022-09-03T06:44:04.789 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws= 2022-09-03T06:44:04.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws=', 'Content-Length': '0'} 2022-09-03T06:44:04.793 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006a299dc17bc35392-006312f7b4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.794 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:04.795 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:04.795 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:04.796 DEBUG:boto:Method: HEAD 2022-09-03T06:44:04.796 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:44:04.797 DEBUG:boto:Data: 2022-09-03T06:44:04.797 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.798 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:04.798 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.799 DEBUG:boto:Params: {} 2022-09-03T06:44:04.800 DEBUG:boto:Token: None 2022-09-03T06:44:04.800 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj55 2022-09-03T06:44:04.801 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hXrcmJjWW3Pikl/c4FfJce14/m4= 2022-09-03T06:44:04.802 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hXrcmJjWW3Pikl/c4FfJce14/m4=', 'Content-Length': '0'} 2022-09-03T06:44:04.804 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cdc2d9b6afbeb5cc-006312f7b4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.805 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:04.805 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:04.806 DEBUG:boto:Method: HEAD 2022-09-03T06:44:04.806 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:44:04.807 DEBUG:boto:Data: 2022-09-03T06:44:04.808 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.809 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:04.810 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.811 DEBUG:boto:Params: {} 2022-09-03T06:44:04.812 DEBUG:boto:Token: None 2022-09-03T06:44:04.813 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj55 2022-09-03T06:44:04.814 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hXrcmJjWW3Pikl/c4FfJce14/m4= 2022-09-03T06:44:04.815 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hXrcmJjWW3Pikl/c4FfJce14/m4=', 'Content-Length': '0'} 2022-09-03T06:44:04.818 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000000afa1f012a2e1ba-006312f7b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.819 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:44:04.820 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:04.820 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:04.821 DEBUG:boto:Method: GET 2022-09-03T06:44:04.821 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:44:04.822 DEBUG:boto:Data: 2022-09-03T06:44:04.823 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.823 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:04.824 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.824 DEBUG:boto:Params: {} 2022-09-03T06:44:04.825 DEBUG:boto:Token: None 2022-09-03T06:44:04.826 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj55 2022-09-03T06:44:04.827 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZyzcgNkxLwBFaIw5R96OvVOz0Y0= 2022-09-03T06:44:04.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZyzcgNkxLwBFaIw5R96OvVOz0Y0=', 'Content-Length': '0'} 2022-09-03T06:44:04.831 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000294188fef60cc453-006312f7b4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.878 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:04.879 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:04.880 DEBUG:boto:Method: GET 2022-09-03T06:44:04.880 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:44:04.881 DEBUG:boto:Data: 2022-09-03T06:44:04.881 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.882 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:04.882 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.883 DEBUG:boto:Params: {} 2022-09-03T06:44:04.884 DEBUG:boto:Token: None 2022-09-03T06:44:04.884 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj55 2022-09-03T06:44:04.885 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZyzcgNkxLwBFaIw5R96OvVOz0Y0= 2022-09-03T06:44:04.886 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZyzcgNkxLwBFaIw5R96OvVOz0Y0=', 'Content-Length': '0'} 2022-09-03T06:44:04.888 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d03aea06d6cca11a-006312f7b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.889 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:04.890 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:04.891 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:04.891 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:04.892 DEBUG:boto:Method: GET 2022-09-03T06:44:04.892 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:44:04.893 DEBUG:boto:Data: 2022-09-03T06:44:04.893 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.894 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:04.895 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.895 DEBUG:boto:Params: {} 2022-09-03T06:44:04.896 DEBUG:boto:Token: None 2022-09-03T06:44:04.897 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/ 2022-09-03T06:44:04.897 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws= 2022-09-03T06:44:04.898 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws=', 'Content-Length': '0'} 2022-09-03T06:44:04.900 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006281a43b50855756-006312f7b4-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.901 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:04.902 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:04.903 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:04.903 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:04.904 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:04.904 DEBUG:boto:Method: GET 2022-09-03T06:44:04.905 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:44:04.905 DEBUG:boto:Data: 2022-09-03T06:44:04.906 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.907 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:04.907 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.908 DEBUG:boto:Params: {} 2022-09-03T06:44:04.909 DEBUG:boto:Token: None 2022-09-03T06:44:04.909 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/ 2022-09-03T06:44:04.910 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws= 2022-09-03T06:44:04.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5av7mjfSnHLHQu7Yze3kJ20FUws=', 'Content-Length': '0'} 2022-09-03T06:44:04.915 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c9e66ca346362fb3-006312f7b4-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.915 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:04.916 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:44:04.917 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:04.918 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:04.918 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:04.919 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:04.919 DEBUG:boto:Method: GET 2022-09-03T06:44:04.920 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:04.920 DEBUG:boto:Data: 2022-09-03T06:44:04.921 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.922 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:04.922 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.923 DEBUG:boto:Params: {} 2022-09-03T06:44:04.923 DEBUG:boto:Token: None 2022-09-03T06:44:04.924 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:04.925 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jD6Rjn6dPjaP9kTzVLq+p5cEPp0= 2022-09-03T06:44:04.925 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jD6Rjn6dPjaP9kTzVLq+p5cEPp0=', 'Content-Length': '0'} 2022-09-03T06:44:04.928 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004ea20d64a7b3a3cd-006312f7b4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.929 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:04.930 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:04.930 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:04.931 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:04.931 DEBUG:boto:Method: GET 2022-09-03T06:44:04.932 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:04.932 DEBUG:boto:Data: 2022-09-03T06:44:04.933 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.933 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:04.934 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.934 DEBUG:boto:Params: {} 2022-09-03T06:44:04.935 DEBUG:boto:Token: None 2022-09-03T06:44:04.936 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:04.936 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jD6Rjn6dPjaP9kTzVLq+p5cEPp0= 2022-09-03T06:44:04.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jD6Rjn6dPjaP9kTzVLq+p5cEPp0=', 'Content-Length': '0'} 2022-09-03T06:44:04.939 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000571a4dcfa36d0c6b-006312f7b4-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.982 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:04.984 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:04.984 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:04.985 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:04.986 DEBUG:boto:Method: HEAD 2022-09-03T06:44:04.987 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:04.988 DEBUG:boto:Data: 2022-09-03T06:44:04.988 DEBUG:boto:Headers: {} 2022-09-03T06:44:04.989 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:04.989 DEBUG:boto:Port: 8000 2022-09-03T06:44:04.990 DEBUG:boto:Params: {} 2022-09-03T06:44:04.990 DEBUG:boto:Token: None 2022-09-03T06:44:04.991 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:04 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:04.992 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/HiysFLZoU6EFjmwVOjY/uM5mfU= 2022-09-03T06:44:04.993 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/HiysFLZoU6EFjmwVOjY/uM5mfU=', 'Content-Length': '0'} 2022-09-03T06:44:04.995 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004352de4b5db5ee54-006312f7b4-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:04.996 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:04.996 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:04.997 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:04.997 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:04.998 DEBUG:boto:Method: HEAD 2022-09-03T06:44:04.998 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:04.999 DEBUG:boto:Data: 2022-09-03T06:44:04.999 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.000 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.000 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.001 DEBUG:boto:Params: {} 2022-09-03T06:44:05.002 DEBUG:boto:Token: None 2022-09-03T06:44:05.003 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:05.004 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qmEyJXrh/C7LbpbzAqpwXkdeFvI= 2022-09-03T06:44:05.005 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qmEyJXrh/C7LbpbzAqpwXkdeFvI=', 'Content-Length': '0'} 2022-09-03T06:44:05.007 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007eeda4204bafdd55-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.008 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:44:05.009 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:05.009 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:05.010 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:05.011 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:05.011 DEBUG:boto:Method: GET 2022-09-03T06:44:05.012 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:05.013 DEBUG:boto:Data: 2022-09-03T06:44:05.013 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.014 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.015 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.015 DEBUG:boto:Params: {} 2022-09-03T06:44:05.016 DEBUG:boto:Token: None 2022-09-03T06:44:05.016 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:05.017 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5yZY/xZSseEn3unMIYfnmNaWKt4= 2022-09-03T06:44:05.018 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5yZY/xZSseEn3unMIYfnmNaWKt4=', 'Content-Length': '0'} 2022-09-03T06:44:05.020 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b3e2bcc9062832ce-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.021 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:05.021 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:05.022 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:05.023 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:05.024 DEBUG:boto:Method: GET 2022-09-03T06:44:05.025 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:05.025 DEBUG:boto:Data: 2022-09-03T06:44:05.026 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.027 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.027 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.028 DEBUG:boto:Params: {} 2022-09-03T06:44:05.028 DEBUG:boto:Token: None 2022-09-03T06:44:05.029 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:05.030 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5yZY/xZSseEn3unMIYfnmNaWKt4= 2022-09-03T06:44:05.030 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5yZY/xZSseEn3unMIYfnmNaWKt4=', 'Content-Length': '0'} 2022-09-03T06:44:05.033 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000069c6e34c59cf8e5b-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.034 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:05.035 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:05.035 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:05.036 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:05.036 DEBUG:boto:Method: GET 2022-09-03T06:44:05.037 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:44:05.038 DEBUG:boto:Data: 2022-09-03T06:44:05.039 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.040 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.041 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.042 DEBUG:boto:Params: {} 2022-09-03T06:44:05.043 DEBUG:boto:Token: None 2022-09-03T06:44:05.044 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/ 2022-09-03T06:44:05.045 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms= 2022-09-03T06:44:05.046 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms=', 'Content-Length': '0'} 2022-09-03T06:44:05.049 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000082a3bbcf8c71ca06-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.051 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:05.052 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:05.053 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:05.054 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:05.055 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:05.055 DEBUG:boto:Method: GET 2022-09-03T06:44:05.056 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:44:05.057 DEBUG:boto:Data: 2022-09-03T06:44:05.058 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.059 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.060 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.061 DEBUG:boto:Params: {} 2022-09-03T06:44:05.062 DEBUG:boto:Token: None 2022-09-03T06:44:05.063 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/ 2022-09-03T06:44:05.064 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms= 2022-09-03T06:44:05.065 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms=', 'Content-Length': '0'} 2022-09-03T06:44:05.068 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002f53e3dd9bdaea5e-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.069 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:05.070 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:05.071 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:05.071 DEBUG:boto:Method: HEAD 2022-09-03T06:44:05.072 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:44:05.072 DEBUG:boto:Data: 2022-09-03T06:44:05.073 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.073 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.074 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.074 DEBUG:boto:Params: {} 2022-09-03T06:44:05.075 DEBUG:boto:Token: None 2022-09-03T06:44:05.075 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj57 2022-09-03T06:44:05.076 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dL+OCHmD+ZohocBfZ0/MkJZmvHM= 2022-09-03T06:44:05.077 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dL+OCHmD+ZohocBfZ0/MkJZmvHM=', 'Content-Length': '0'} 2022-09-03T06:44:05.079 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d8d6750bebe66860-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.079 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:05.080 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:05.080 DEBUG:boto:Method: HEAD 2022-09-03T06:44:05.081 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:44:05.081 DEBUG:boto:Data: 2022-09-03T06:44:05.082 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.082 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.083 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.083 DEBUG:boto:Params: {} 2022-09-03T06:44:05.084 DEBUG:boto:Token: None 2022-09-03T06:44:05.085 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj57 2022-09-03T06:44:05.085 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dL+OCHmD+ZohocBfZ0/MkJZmvHM= 2022-09-03T06:44:05.086 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dL+OCHmD+ZohocBfZ0/MkJZmvHM=', 'Content-Length': '0'} 2022-09-03T06:44:05.089 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c64aeccba8d7f135-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.090 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:44:05.091 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:05.091 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:05.092 DEBUG:boto:Method: GET 2022-09-03T06:44:05.093 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:44:05.093 DEBUG:boto:Data: 2022-09-03T06:44:05.094 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.095 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.095 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.096 DEBUG:boto:Params: {} 2022-09-03T06:44:05.097 DEBUG:boto:Token: None 2022-09-03T06:44:05.097 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj57 2022-09-03T06:44:05.098 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xZLlFVN+7jMn3IDYS3X5vrTiqW4= 2022-09-03T06:44:05.100 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xZLlFVN+7jMn3IDYS3X5vrTiqW4=', 'Content-Length': '0'} 2022-09-03T06:44:05.103 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000050dbb797ce7a474-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.147 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:05.148 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:05.148 DEBUG:boto:Method: GET 2022-09-03T06:44:05.149 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:44:05.149 DEBUG:boto:Data: 2022-09-03T06:44:05.150 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.150 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.151 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.152 DEBUG:boto:Params: {} 2022-09-03T06:44:05.153 DEBUG:boto:Token: None 2022-09-03T06:44:05.154 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj57 2022-09-03T06:44:05.154 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xZLlFVN+7jMn3IDYS3X5vrTiqW4= 2022-09-03T06:44:05.155 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xZLlFVN+7jMn3IDYS3X5vrTiqW4=', 'Content-Length': '0'} 2022-09-03T06:44:05.157 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fc12fbd1145cd3c2-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.158 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:05.159 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:05.160 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:05.160 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:05.161 DEBUG:boto:Method: GET 2022-09-03T06:44:05.161 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:44:05.162 DEBUG:boto:Data: 2022-09-03T06:44:05.163 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.164 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.165 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.166 DEBUG:boto:Params: {} 2022-09-03T06:44:05.167 DEBUG:boto:Token: None 2022-09-03T06:44:05.168 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/ 2022-09-03T06:44:05.169 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms= 2022-09-03T06:44:05.170 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms=', 'Content-Length': '0'} 2022-09-03T06:44:05.176 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000162dcd1a5932318f-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.177 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:05.179 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:05.179 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:05.180 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:05.181 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:05.182 DEBUG:boto:Method: GET 2022-09-03T06:44:05.183 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:44:05.183 DEBUG:boto:Data: 2022-09-03T06:44:05.184 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.185 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.186 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.186 DEBUG:boto:Params: {} 2022-09-03T06:44:05.187 DEBUG:boto:Token: None 2022-09-03T06:44:05.188 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/ 2022-09-03T06:44:05.189 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms= 2022-09-03T06:44:05.190 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms=', 'Content-Length': '0'} 2022-09-03T06:44:05.194 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cee8955c53ace6f6-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.195 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:05.197 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:44:05.198 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:05.199 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:05.200 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.200 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.201 DEBUG:boto:Method: GET 2022-09-03T06:44:05.201 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.202 DEBUG:boto:Data: 2022-09-03T06:44:05.203 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.203 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.204 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.204 DEBUG:boto:Params: {} 2022-09-03T06:44:05.205 DEBUG:boto:Token: None 2022-09-03T06:44:05.206 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.206 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XzGPi+Fp7zHPH6TeQpXuzW5UHso= 2022-09-03T06:44:05.207 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XzGPi+Fp7zHPH6TeQpXuzW5UHso=', 'Content-Length': '0'} 2022-09-03T06:44:05.210 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006f32ac9792f92498-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.212 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:05.212 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:05.213 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.214 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.214 DEBUG:boto:Method: GET 2022-09-03T06:44:05.215 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.215 DEBUG:boto:Data: 2022-09-03T06:44:05.216 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.216 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.217 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.217 DEBUG:boto:Params: {} 2022-09-03T06:44:05.218 DEBUG:boto:Token: None 2022-09-03T06:44:05.219 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.219 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XzGPi+Fp7zHPH6TeQpXuzW5UHso= 2022-09-03T06:44:05.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XzGPi+Fp7zHPH6TeQpXuzW5UHso=', 'Content-Length': '0'} 2022-09-03T06:44:05.222 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000444509678fc5fb0a-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.223 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:05.224 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:05.224 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.225 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.225 DEBUG:boto:Method: HEAD 2022-09-03T06:44:05.226 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.226 DEBUG:boto:Data: 2022-09-03T06:44:05.227 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.227 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.228 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.228 DEBUG:boto:Params: {} 2022-09-03T06:44:05.229 DEBUG:boto:Token: None 2022-09-03T06:44:05.230 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.230 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nMchQhv0jtSNbrFDbaDOZsCM/XE= 2022-09-03T06:44:05.231 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nMchQhv0jtSNbrFDbaDOZsCM/XE=', 'Content-Length': '0'} 2022-09-03T06:44:05.233 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004d5183b40ff683ef-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.234 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:05.234 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:05.235 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.235 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.236 DEBUG:boto:Method: HEAD 2022-09-03T06:44:05.237 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.237 DEBUG:boto:Data: 2022-09-03T06:44:05.238 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.238 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.239 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.239 DEBUG:boto:Params: {} 2022-09-03T06:44:05.240 DEBUG:boto:Token: None 2022-09-03T06:44:05.240 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.241 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nMchQhv0jtSNbrFDbaDOZsCM/XE= 2022-09-03T06:44:05.242 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nMchQhv0jtSNbrFDbaDOZsCM/XE=', 'Content-Length': '0'} 2022-09-03T06:44:05.244 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ffc6664375b1f281-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.245 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:44:05.246 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:05.247 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:05.248 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.248 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.249 DEBUG:boto:Method: GET 2022-09-03T06:44:05.249 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.250 DEBUG:boto:Data: 2022-09-03T06:44:05.251 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.252 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.253 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.253 DEBUG:boto:Params: {} 2022-09-03T06:44:05.254 DEBUG:boto:Token: None 2022-09-03T06:44:05.255 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.256 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XzGPi+Fp7zHPH6TeQpXuzW5UHso= 2022-09-03T06:44:05.257 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XzGPi+Fp7zHPH6TeQpXuzW5UHso=', 'Content-Length': '0'} 2022-09-03T06:44:05.259 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007ea96c48cf647244-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.306 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:05.307 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:05.308 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.309 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.309 DEBUG:boto:Method: GET 2022-09-03T06:44:05.310 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.311 DEBUG:boto:Data: 2022-09-03T06:44:05.311 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.312 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.313 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.313 DEBUG:boto:Params: {} 2022-09-03T06:44:05.314 DEBUG:boto:Token: None 2022-09-03T06:44:05.315 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:05.316 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XzGPi+Fp7zHPH6TeQpXuzW5UHso= 2022-09-03T06:44:05.317 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XzGPi+Fp7zHPH6TeQpXuzW5UHso=', 'Content-Length': '0'} 2022-09-03T06:44:05.321 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006ff82f6dde04bae4-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.322 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:05.323 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:05.324 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:05.325 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:05.326 DEBUG:boto:Method: GET 2022-09-03T06:44:05.327 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:44:05.328 DEBUG:boto:Data: 2022-09-03T06:44:05.328 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.329 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.329 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.330 DEBUG:boto:Params: {} 2022-09-03T06:44:05.331 DEBUG:boto:Token: None 2022-09-03T06:44:05.332 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/ 2022-09-03T06:44:05.332 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms= 2022-09-03T06:44:05.333 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms=', 'Content-Length': '0'} 2022-09-03T06:44:05.338 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000071c16c21dc4064c1-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.339 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:05.340 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:05.341 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:05.341 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:05.342 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:05.342 DEBUG:boto:Method: GET 2022-09-03T06:44:05.343 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:44:05.344 DEBUG:boto:Data: 2022-09-03T06:44:05.344 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.345 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.346 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.346 DEBUG:boto:Params: {} 2022-09-03T06:44:05.347 DEBUG:boto:Token: None 2022-09-03T06:44:05.348 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/ 2022-09-03T06:44:05.348 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms= 2022-09-03T06:44:05.349 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms=', 'Content-Length': '0'} 2022-09-03T06:44:05.353 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003fe4f59221d9b8af-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.398 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:05.399 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:05.400 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:05.400 DEBUG:boto:Method: HEAD 2022-09-03T06:44:05.401 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:44:05.401 DEBUG:boto:Data: 2022-09-03T06:44:05.402 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.402 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.403 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.403 DEBUG:boto:Params: {} 2022-09-03T06:44:05.404 DEBUG:boto:Token: None 2022-09-03T06:44:05.405 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj68 2022-09-03T06:44:05.405 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cBuIH67YHgoV4l1/THWcqlpWCKU= 2022-09-03T06:44:05.406 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cBuIH67YHgoV4l1/THWcqlpWCKU=', 'Content-Length': '0'} 2022-09-03T06:44:05.408 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d493ac8f04038ca-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.408 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:05.409 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:05.410 DEBUG:boto:Method: HEAD 2022-09-03T06:44:05.411 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:44:05.412 DEBUG:boto:Data: 2022-09-03T06:44:05.413 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.413 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.414 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.415 DEBUG:boto:Params: {} 2022-09-03T06:44:05.418 DEBUG:boto:Token: None 2022-09-03T06:44:05.418 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj68 2022-09-03T06:44:05.419 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cBuIH67YHgoV4l1/THWcqlpWCKU= 2022-09-03T06:44:05.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cBuIH67YHgoV4l1/THWcqlpWCKU=', 'Content-Length': '0'} 2022-09-03T06:44:05.422 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007e8e75c07b715533-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.423 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:44:05.424 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:05.424 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:05.425 DEBUG:boto:Method: GET 2022-09-03T06:44:05.426 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:44:05.427 DEBUG:boto:Data: 2022-09-03T06:44:05.428 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.428 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.429 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.429 DEBUG:boto:Params: {} 2022-09-03T06:44:05.430 DEBUG:boto:Token: None 2022-09-03T06:44:05.431 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj68 2022-09-03T06:44:05.432 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J9phqZtvK22nuo8MpbzndgNqfbY= 2022-09-03T06:44:05.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J9phqZtvK22nuo8MpbzndgNqfbY=', 'Content-Length': '0'} 2022-09-03T06:44:05.434 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d826ac654569340b-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.435 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:05.436 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:05.437 DEBUG:boto:Method: GET 2022-09-03T06:44:05.438 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:44:05.438 DEBUG:boto:Data: 2022-09-03T06:44:05.439 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.440 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.440 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.441 DEBUG:boto:Params: {} 2022-09-03T06:44:05.442 DEBUG:boto:Token: None 2022-09-03T06:44:05.442 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj68 2022-09-03T06:44:05.443 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J9phqZtvK22nuo8MpbzndgNqfbY= 2022-09-03T06:44:05.444 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J9phqZtvK22nuo8MpbzndgNqfbY=', 'Content-Length': '0'} 2022-09-03T06:44:05.446 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f6c5ee191e116d7c-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.447 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:05.447 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:05.448 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:05.449 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:05.449 DEBUG:boto:Method: GET 2022-09-03T06:44:05.450 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:44:05.451 DEBUG:boto:Data: 2022-09-03T06:44:05.451 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.452 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.452 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.453 DEBUG:boto:Params: {} 2022-09-03T06:44:05.454 DEBUG:boto:Token: None 2022-09-03T06:44:05.455 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/ 2022-09-03T06:44:05.456 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms= 2022-09-03T06:44:05.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms=', 'Content-Length': '0'} 2022-09-03T06:44:05.459 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000acbb5c0f4e474959-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.461 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:05.462 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:05.463 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:05.464 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:05.464 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:05.465 DEBUG:boto:Method: GET 2022-09-03T06:44:05.466 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:44:05.466 DEBUG:boto:Data: 2022-09-03T06:44:05.467 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.467 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.468 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.469 DEBUG:boto:Params: {} 2022-09-03T06:44:05.469 DEBUG:boto:Token: None 2022-09-03T06:44:05.471 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/ 2022-09-03T06:44:05.472 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms= 2022-09-03T06:44:05.472 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms=', 'Content-Length': '0'} 2022-09-03T06:44:05.476 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001eb79a78d51ef322-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.522 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:05.524 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:44:05.525 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:05.526 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:05.528 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.528 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.529 DEBUG:boto:Method: GET 2022-09-03T06:44:05.530 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.531 DEBUG:boto:Data: 2022-09-03T06:44:05.531 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.532 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.533 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.534 DEBUG:boto:Params: {} 2022-09-03T06:44:05.534 DEBUG:boto:Token: None 2022-09-03T06:44:05.536 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.537 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hUFo0ts+SkWcJ/kdWcvx//ibXEU= 2022-09-03T06:44:05.538 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hUFo0ts+SkWcJ/kdWcvx//ibXEU=', 'Content-Length': '0'} 2022-09-03T06:44:05.541 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2ddffbb6e0bf56c-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.542 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:05.543 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:05.544 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.545 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.546 DEBUG:boto:Method: GET 2022-09-03T06:44:05.547 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.548 DEBUG:boto:Data: 2022-09-03T06:44:05.549 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.550 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.551 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.552 DEBUG:boto:Params: {} 2022-09-03T06:44:05.553 DEBUG:boto:Token: None 2022-09-03T06:44:05.554 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.555 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hUFo0ts+SkWcJ/kdWcvx//ibXEU= 2022-09-03T06:44:05.556 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hUFo0ts+SkWcJ/kdWcvx//ibXEU=', 'Content-Length': '0'} 2022-09-03T06:44:05.558 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000017db8fc7c00b41ef-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.559 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:05.560 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:05.561 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.561 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.562 DEBUG:boto:Method: HEAD 2022-09-03T06:44:05.563 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.564 DEBUG:boto:Data: 2022-09-03T06:44:05.565 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.565 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.566 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.566 DEBUG:boto:Params: {} 2022-09-03T06:44:05.567 DEBUG:boto:Token: None 2022-09-03T06:44:05.567 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.568 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:29n199K+2g3Bw76Va+caHBgVZjw= 2022-09-03T06:44:05.569 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:29n199K+2g3Bw76Va+caHBgVZjw=', 'Content-Length': '0'} 2022-09-03T06:44:05.571 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d8ff220ec830f468-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.572 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:05.573 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:05.574 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.575 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.576 DEBUG:boto:Method: HEAD 2022-09-03T06:44:05.577 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.578 DEBUG:boto:Data: 2022-09-03T06:44:05.579 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.580 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.581 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.582 DEBUG:boto:Params: {} 2022-09-03T06:44:05.583 DEBUG:boto:Token: None 2022-09-03T06:44:05.584 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.586 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:29n199K+2g3Bw76Va+caHBgVZjw= 2022-09-03T06:44:05.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:29n199K+2g3Bw76Va+caHBgVZjw=', 'Content-Length': '0'} 2022-09-03T06:44:05.589 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c96df7a27f851b93-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.591 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:44:05.591 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:05.592 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:05.593 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.594 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.595 DEBUG:boto:Method: GET 2022-09-03T06:44:05.596 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.597 DEBUG:boto:Data: 2022-09-03T06:44:05.598 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.598 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.599 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.600 DEBUG:boto:Params: {} 2022-09-03T06:44:05.601 DEBUG:boto:Token: None 2022-09-03T06:44:05.601 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.602 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hUFo0ts+SkWcJ/kdWcvx//ibXEU= 2022-09-03T06:44:05.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hUFo0ts+SkWcJ/kdWcvx//ibXEU=', 'Content-Length': '0'} 2022-09-03T06:44:05.605 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008bbd5cf5cb50da57-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.646 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:05.648 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:05.650 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.651 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.652 DEBUG:boto:Method: GET 2022-09-03T06:44:05.653 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.654 DEBUG:boto:Data: 2022-09-03T06:44:05.655 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.656 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.657 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.658 DEBUG:boto:Params: {} 2022-09-03T06:44:05.659 DEBUG:boto:Token: None 2022-09-03T06:44:05.660 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:05.662 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hUFo0ts+SkWcJ/kdWcvx//ibXEU= 2022-09-03T06:44:05.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hUFo0ts+SkWcJ/kdWcvx//ibXEU=', 'Content-Length': '0'} 2022-09-03T06:44:05.665 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c5ca1de199d0b27-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.667 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:05.667 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:05.668 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:05.668 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:05.669 DEBUG:boto:Method: GET 2022-09-03T06:44:05.669 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:44:05.670 DEBUG:boto:Data: 2022-09-03T06:44:05.670 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.671 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.671 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.672 DEBUG:boto:Params: {} 2022-09-03T06:44:05.673 DEBUG:boto:Token: None 2022-09-03T06:44:05.673 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/ 2022-09-03T06:44:05.674 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms= 2022-09-03T06:44:05.675 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms=', 'Content-Length': '0'} 2022-09-03T06:44:05.677 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000acb3f4e58258f92d-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.678 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:05.679 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:05.680 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:05.680 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:05.681 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:05.681 DEBUG:boto:Method: GET 2022-09-03T06:44:05.682 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:44:05.682 DEBUG:boto:Data: 2022-09-03T06:44:05.683 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.683 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.684 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.684 DEBUG:boto:Params: {} 2022-09-03T06:44:05.685 DEBUG:boto:Token: None 2022-09-03T06:44:05.686 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/ 2022-09-03T06:44:05.686 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms= 2022-09-03T06:44:05.687 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms=', 'Content-Length': '0'} 2022-09-03T06:44:05.690 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000daf0236a2b8d2d59-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.691 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:05.692 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:05.693 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:05.693 DEBUG:boto:Method: HEAD 2022-09-03T06:44:05.694 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:44:05.694 DEBUG:boto:Data: 2022-09-03T06:44:05.695 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.695 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.696 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.696 DEBUG:boto:Params: {} 2022-09-03T06:44:05.697 DEBUG:boto:Token: None 2022-09-03T06:44:05.698 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj69 2022-09-03T06:44:05.698 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KfC6ibpVscwLH0rxyos2vYv42Zs= 2022-09-03T06:44:05.699 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KfC6ibpVscwLH0rxyos2vYv42Zs=', 'Content-Length': '0'} 2022-09-03T06:44:05.701 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a37152163bc5b132-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.702 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:05.703 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:05.703 DEBUG:boto:Method: HEAD 2022-09-03T06:44:05.704 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:44:05.704 DEBUG:boto:Data: 2022-09-03T06:44:05.705 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.705 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.706 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.706 DEBUG:boto:Params: {} 2022-09-03T06:44:05.707 DEBUG:boto:Token: None 2022-09-03T06:44:05.708 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj69 2022-09-03T06:44:05.708 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KfC6ibpVscwLH0rxyos2vYv42Zs= 2022-09-03T06:44:05.709 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KfC6ibpVscwLH0rxyos2vYv42Zs=', 'Content-Length': '0'} 2022-09-03T06:44:05.711 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003cdde8c3a6425663-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.712 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:44:05.712 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:05.713 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:05.714 DEBUG:boto:Method: GET 2022-09-03T06:44:05.715 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:44:05.716 DEBUG:boto:Data: 2022-09-03T06:44:05.717 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.718 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.719 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.720 DEBUG:boto:Params: {} 2022-09-03T06:44:05.721 DEBUG:boto:Token: None 2022-09-03T06:44:05.722 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj69 2022-09-03T06:44:05.723 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5UKpkJeTXDDrLE19YaKAHJkv7g8= 2022-09-03T06:44:05.724 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5UKpkJeTXDDrLE19YaKAHJkv7g8=', 'Content-Length': '0'} 2022-09-03T06:44:05.727 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006fb49c82364d7e66-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.728 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:05.729 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:05.730 DEBUG:boto:Method: GET 2022-09-03T06:44:05.731 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:44:05.733 DEBUG:boto:Data: 2022-09-03T06:44:05.734 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.735 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.736 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.737 DEBUG:boto:Params: {} 2022-09-03T06:44:05.738 DEBUG:boto:Token: None 2022-09-03T06:44:05.739 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj69 2022-09-03T06:44:05.740 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5UKpkJeTXDDrLE19YaKAHJkv7g8= 2022-09-03T06:44:05.741 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5UKpkJeTXDDrLE19YaKAHJkv7g8=', 'Content-Length': '0'} 2022-09-03T06:44:05.744 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002270ec463d8cc4db-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.786 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:05.788 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:05.789 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:05.789 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:05.790 DEBUG:boto:Method: GET 2022-09-03T06:44:05.791 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:44:05.792 DEBUG:boto:Data: 2022-09-03T06:44:05.793 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.794 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.795 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.796 DEBUG:boto:Params: {} 2022-09-03T06:44:05.797 DEBUG:boto:Token: None 2022-09-03T06:44:05.799 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/ 2022-09-03T06:44:05.800 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms= 2022-09-03T06:44:05.801 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms=', 'Content-Length': '0'} 2022-09-03T06:44:05.804 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000092137089ac37459f-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.804 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:05.805 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:05.806 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:05.806 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:05.809 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:05.810 DEBUG:boto:Method: GET 2022-09-03T06:44:05.811 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:44:05.811 DEBUG:boto:Data: 2022-09-03T06:44:05.812 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.813 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.814 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.814 DEBUG:boto:Params: {} 2022-09-03T06:44:05.815 DEBUG:boto:Token: None 2022-09-03T06:44:05.816 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/ 2022-09-03T06:44:05.817 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms= 2022-09-03T06:44:05.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms=', 'Content-Length': '0'} 2022-09-03T06:44:05.822 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a29fb4823b8b6793-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.823 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:05.823 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:44:05.824 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:05.825 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:05.825 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.826 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.826 DEBUG:boto:Method: GET 2022-09-03T06:44:05.827 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.828 DEBUG:boto:Data: 2022-09-03T06:44:05.828 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.829 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.830 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.830 DEBUG:boto:Params: {} 2022-09-03T06:44:05.831 DEBUG:boto:Token: None 2022-09-03T06:44:05.832 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.833 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ld4/7gDzODnWuJ/S0RnPymjYi64= 2022-09-03T06:44:05.835 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ld4/7gDzODnWuJ/S0RnPymjYi64=', 'Content-Length': '0'} 2022-09-03T06:44:05.837 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000016ffa861f9112b91-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.838 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:05.838 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:05.839 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.840 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.841 DEBUG:boto:Method: GET 2022-09-03T06:44:05.841 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.842 DEBUG:boto:Data: 2022-09-03T06:44:05.843 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.844 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.844 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.845 DEBUG:boto:Params: {} 2022-09-03T06:44:05.845 DEBUG:boto:Token: None 2022-09-03T06:44:05.846 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.847 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ld4/7gDzODnWuJ/S0RnPymjYi64= 2022-09-03T06:44:05.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ld4/7gDzODnWuJ/S0RnPymjYi64=', 'Content-Length': '0'} 2022-09-03T06:44:05.850 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad1a759993d6499e-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.851 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:05.851 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:05.852 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.852 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.853 DEBUG:boto:Method: HEAD 2022-09-03T06:44:05.854 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.854 DEBUG:boto:Data: 2022-09-03T06:44:05.855 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.855 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.856 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.856 DEBUG:boto:Params: {} 2022-09-03T06:44:05.857 DEBUG:boto:Token: None 2022-09-03T06:44:05.857 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.858 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8BIdIz5KWnkibdYfDxvQwxgSmks= 2022-09-03T06:44:05.859 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8BIdIz5KWnkibdYfDxvQwxgSmks=', 'Content-Length': '0'} 2022-09-03T06:44:05.861 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000036a7c44625ef8b3d-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.861 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:05.862 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:05.862 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.863 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.863 DEBUG:boto:Method: HEAD 2022-09-03T06:44:05.864 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.865 DEBUG:boto:Data: 2022-09-03T06:44:05.865 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.866 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.866 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.867 DEBUG:boto:Params: {} 2022-09-03T06:44:05.867 DEBUG:boto:Token: None 2022-09-03T06:44:05.868 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.869 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8BIdIz5KWnkibdYfDxvQwxgSmks= 2022-09-03T06:44:05.869 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8BIdIz5KWnkibdYfDxvQwxgSmks=', 'Content-Length': '0'} 2022-09-03T06:44:05.871 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001de656cd03005e93-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.872 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:44:05.872 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:05.873 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:05.874 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.874 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.875 DEBUG:boto:Method: GET 2022-09-03T06:44:05.875 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.876 DEBUG:boto:Data: 2022-09-03T06:44:05.876 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.877 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.878 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.878 DEBUG:boto:Params: {} 2022-09-03T06:44:05.879 DEBUG:boto:Token: None 2022-09-03T06:44:05.879 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.880 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ld4/7gDzODnWuJ/S0RnPymjYi64= 2022-09-03T06:44:05.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ld4/7gDzODnWuJ/S0RnPymjYi64=', 'Content-Length': '0'} 2022-09-03T06:44:05.882 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cd2f66b19143e6c3-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.883 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:05.884 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:05.884 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.885 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.886 DEBUG:boto:Method: GET 2022-09-03T06:44:05.886 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.887 DEBUG:boto:Data: 2022-09-03T06:44:05.888 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.888 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.889 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.889 DEBUG:boto:Params: {} 2022-09-03T06:44:05.890 DEBUG:boto:Token: None 2022-09-03T06:44:05.890 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:05.891 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ld4/7gDzODnWuJ/S0RnPymjYi64= 2022-09-03T06:44:05.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ld4/7gDzODnWuJ/S0RnPymjYi64=', 'Content-Length': '0'} 2022-09-03T06:44:05.894 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000554270e524fb2f3e-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.895 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:05.896 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:05.897 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:05.897 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:05.898 DEBUG:boto:Method: GET 2022-09-03T06:44:05.898 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:44:05.899 DEBUG:boto:Data: 2022-09-03T06:44:05.899 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.900 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.901 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.901 DEBUG:boto:Params: {} 2022-09-03T06:44:05.902 DEBUG:boto:Token: None 2022-09-03T06:44:05.902 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/ 2022-09-03T06:44:05.903 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms= 2022-09-03T06:44:05.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms=', 'Content-Length': '0'} 2022-09-03T06:44:05.906 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000874189612240af51-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.907 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:05.908 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:05.908 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:05.909 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:05.909 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:05.910 DEBUG:boto:Method: GET 2022-09-03T06:44:05.911 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:44:05.911 DEBUG:boto:Data: 2022-09-03T06:44:05.912 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.912 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.913 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.914 DEBUG:boto:Params: {} 2022-09-03T06:44:05.914 DEBUG:boto:Token: None 2022-09-03T06:44:05.915 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/ 2022-09-03T06:44:05.915 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms= 2022-09-03T06:44:05.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KEENHD2l6faIjxog/LM8u+TlFms=', 'Content-Length': '0'} 2022-09-03T06:44:05.920 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c88dff1ba29a2ea7-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.921 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:05.922 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:05.922 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:05.923 DEBUG:boto:Method: HEAD 2022-09-03T06:44:05.923 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:44:05.924 DEBUG:boto:Data: 2022-09-03T06:44:05.925 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.925 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.926 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.926 DEBUG:boto:Params: {} 2022-09-03T06:44:05.927 DEBUG:boto:Token: None 2022-09-03T06:44:05.927 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj76 2022-09-03T06:44:05.928 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ChoCPQ6pg+Zg3m1fxJVZb0p9lqw= 2022-09-03T06:44:05.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ChoCPQ6pg+Zg3m1fxJVZb0p9lqw=', 'Content-Length': '0'} 2022-09-03T06:44:05.931 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007521c52623a72f5f-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.931 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:05.932 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:05.933 DEBUG:boto:Method: HEAD 2022-09-03T06:44:05.933 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:44:05.934 DEBUG:boto:Data: 2022-09-03T06:44:05.935 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.935 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.936 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.936 DEBUG:boto:Params: {} 2022-09-03T06:44:05.937 DEBUG:boto:Token: None 2022-09-03T06:44:05.937 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj76 2022-09-03T06:44:05.938 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ChoCPQ6pg+Zg3m1fxJVZb0p9lqw= 2022-09-03T06:44:05.939 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ChoCPQ6pg+Zg3m1fxJVZb0p9lqw=', 'Content-Length': '0'} 2022-09-03T06:44:05.941 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000033fadb4ee4bd8a73-006312f7b5-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.942 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:44:05.943 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:05.944 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:05.944 DEBUG:boto:Method: GET 2022-09-03T06:44:05.945 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:44:05.946 DEBUG:boto:Data: 2022-09-03T06:44:05.946 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.947 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:05.948 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.948 DEBUG:boto:Params: {} 2022-09-03T06:44:05.949 DEBUG:boto:Token: None 2022-09-03T06:44:05.949 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:05 GMT /dsgvcy-75/obj76 2022-09-03T06:44:05.950 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZTzCccC6TzSwfdMAOs6jdFXQd9Y= 2022-09-03T06:44:05.951 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZTzCccC6TzSwfdMAOs6jdFXQd9Y=', 'Content-Length': '0'} 2022-09-03T06:44:05.953 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f4da49a4facc85fc-006312f7b5-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:05.994 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:05.995 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:05.995 DEBUG:boto:Method: GET 2022-09-03T06:44:05.996 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:44:05.996 DEBUG:boto:Data: 2022-09-03T06:44:05.997 DEBUG:boto:Headers: {} 2022-09-03T06:44:05.997 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:05.998 DEBUG:boto:Port: 8000 2022-09-03T06:44:05.999 DEBUG:boto:Params: {} 2022-09-03T06:44:05.999 DEBUG:boto:Token: None 2022-09-03T06:44:06.000 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj76 2022-09-03T06:44:06.001 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SnpNvl9aNM2LwhyYDRF/L4YsM2I= 2022-09-03T06:44:06.001 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SnpNvl9aNM2LwhyYDRF/L4YsM2I=', 'Content-Length': '0'} 2022-09-03T06:44:06.003 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000080f1f85698085c4f-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.004 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:06.005 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:06.005 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:06.006 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:06.006 DEBUG:boto:Method: GET 2022-09-03T06:44:06.007 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:44:06.008 DEBUG:boto:Data: 2022-09-03T06:44:06.008 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.009 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:06.009 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.010 DEBUG:boto:Params: {} 2022-09-03T06:44:06.011 DEBUG:boto:Token: None 2022-09-03T06:44:06.011 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/ 2022-09-03T06:44:06.012 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw= 2022-09-03T06:44:06.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw=', 'Content-Length': '0'} 2022-09-03T06:44:06.016 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ca9b852ba61d2162-006312f7b6-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.016 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:06.017 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:06.018 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:06.018 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:06.019 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:06.019 DEBUG:boto:Method: GET 2022-09-03T06:44:06.020 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:44:06.021 DEBUG:boto:Data: 2022-09-03T06:44:06.021 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.022 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.022 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.023 DEBUG:boto:Params: {} 2022-09-03T06:44:06.024 DEBUG:boto:Token: None 2022-09-03T06:44:06.024 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/ 2022-09-03T06:44:06.025 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw= 2022-09-03T06:44:06.026 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw=', 'Content-Length': '0'} 2022-09-03T06:44:06.029 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006fcf4b03e5507616-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.031 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:06.032 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:44:06.032 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:06.033 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:06.034 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.034 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.035 DEBUG:boto:Method: GET 2022-09-03T06:44:06.035 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.036 DEBUG:boto:Data: 2022-09-03T06:44:06.036 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.037 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:06.038 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.038 DEBUG:boto:Params: {} 2022-09-03T06:44:06.039 DEBUG:boto:Token: None 2022-09-03T06:44:06.039 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.040 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QLGvg8JsZSeSg/DUmnEJvW61WKQ= 2022-09-03T06:44:06.041 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QLGvg8JsZSeSg/DUmnEJvW61WKQ=', 'Content-Length': '0'} 2022-09-03T06:44:06.043 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ed94eeaee334516b-006312f7b6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.044 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:06.044 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:06.045 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.045 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.046 DEBUG:boto:Method: GET 2022-09-03T06:44:06.046 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.047 DEBUG:boto:Data: 2022-09-03T06:44:06.048 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.048 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.049 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.049 DEBUG:boto:Params: {} 2022-09-03T06:44:06.050 DEBUG:boto:Token: None 2022-09-03T06:44:06.051 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.051 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QLGvg8JsZSeSg/DUmnEJvW61WKQ= 2022-09-03T06:44:06.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QLGvg8JsZSeSg/DUmnEJvW61WKQ=', 'Content-Length': '0'} 2022-09-03T06:44:06.054 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007873322ec3fc5261-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.055 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:06.056 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:06.056 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.057 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.057 DEBUG:boto:Method: HEAD 2022-09-03T06:44:06.058 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.059 DEBUG:boto:Data: 2022-09-03T06:44:06.059 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.060 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:06.060 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.061 DEBUG:boto:Params: {} 2022-09-03T06:44:06.061 DEBUG:boto:Token: None 2022-09-03T06:44:06.062 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.063 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CwvxWanjj+BW++OKuy5lDtahcHU= 2022-09-03T06:44:06.063 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CwvxWanjj+BW++OKuy5lDtahcHU=', 'Content-Length': '0'} 2022-09-03T06:44:06.065 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007d2691116963ac2c-006312f7b6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.066 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:06.067 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:06.067 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.068 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.068 DEBUG:boto:Method: HEAD 2022-09-03T06:44:06.069 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.069 DEBUG:boto:Data: 2022-09-03T06:44:06.070 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.070 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.071 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.071 DEBUG:boto:Params: {} 2022-09-03T06:44:06.072 DEBUG:boto:Token: None 2022-09-03T06:44:06.072 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.073 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CwvxWanjj+BW++OKuy5lDtahcHU= 2022-09-03T06:44:06.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CwvxWanjj+BW++OKuy5lDtahcHU=', 'Content-Length': '0'} 2022-09-03T06:44:06.076 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000be39be1b88fc1868-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.076 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:44:06.077 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:06.078 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:06.078 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.079 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.079 DEBUG:boto:Method: GET 2022-09-03T06:44:06.080 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.080 DEBUG:boto:Data: 2022-09-03T06:44:06.081 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.081 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:06.082 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.083 DEBUG:boto:Params: {} 2022-09-03T06:44:06.083 DEBUG:boto:Token: None 2022-09-03T06:44:06.084 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.084 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QLGvg8JsZSeSg/DUmnEJvW61WKQ= 2022-09-03T06:44:06.085 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QLGvg8JsZSeSg/DUmnEJvW61WKQ=', 'Content-Length': '0'} 2022-09-03T06:44:06.087 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af2af8719f90a62c-006312f7b6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.134 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:06.135 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:06.136 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.136 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.137 DEBUG:boto:Method: GET 2022-09-03T06:44:06.138 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.138 DEBUG:boto:Data: 2022-09-03T06:44:06.139 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.139 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.140 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.141 DEBUG:boto:Params: {} 2022-09-03T06:44:06.142 DEBUG:boto:Token: None 2022-09-03T06:44:06.143 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:06.143 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QLGvg8JsZSeSg/DUmnEJvW61WKQ= 2022-09-03T06:44:06.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QLGvg8JsZSeSg/DUmnEJvW61WKQ=', 'Content-Length': '0'} 2022-09-03T06:44:06.147 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d0cf34a663d9d24d-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.148 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:06.148 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:06.149 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:06.150 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:06.150 DEBUG:boto:Method: GET 2022-09-03T06:44:06.151 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:44:06.152 DEBUG:boto:Data: 2022-09-03T06:44:06.152 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.153 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:06.154 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.154 DEBUG:boto:Params: {} 2022-09-03T06:44:06.155 DEBUG:boto:Token: None 2022-09-03T06:44:06.156 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/ 2022-09-03T06:44:06.156 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw= 2022-09-03T06:44:06.157 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw=', 'Content-Length': '0'} 2022-09-03T06:44:06.160 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004bb9d6f37cb81128-006312f7b6-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.160 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:06.162 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:06.162 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:06.163 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:06.164 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:06.165 DEBUG:boto:Method: GET 2022-09-03T06:44:06.166 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:44:06.167 DEBUG:boto:Data: 2022-09-03T06:44:06.168 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.168 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.169 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.169 DEBUG:boto:Params: {} 2022-09-03T06:44:06.170 DEBUG:boto:Token: None 2022-09-03T06:44:06.171 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/ 2022-09-03T06:44:06.171 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw= 2022-09-03T06:44:06.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw=', 'Content-Length': '0'} 2022-09-03T06:44:06.176 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000098b51b365149296f-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.177 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:06.179 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:06.180 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:06.181 DEBUG:boto:Method: HEAD 2022-09-03T06:44:06.181 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:44:06.182 DEBUG:boto:Data: 2022-09-03T06:44:06.184 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.185 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:06.185 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.186 DEBUG:boto:Params: {} 2022-09-03T06:44:06.187 DEBUG:boto:Token: None 2022-09-03T06:44:06.188 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj85 2022-09-03T06:44:06.189 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YWyOjzLpTg9cC+BabjYOslnYtJ8= 2022-09-03T06:44:06.190 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YWyOjzLpTg9cC+BabjYOslnYtJ8=', 'Content-Length': '0'} 2022-09-03T06:44:06.192 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a2b31b1a2ea6d797-006312f7b6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.193 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:06.193 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:06.194 DEBUG:boto:Method: HEAD 2022-09-03T06:44:06.194 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:44:06.195 DEBUG:boto:Data: 2022-09-03T06:44:06.195 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.196 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.197 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.197 DEBUG:boto:Params: {} 2022-09-03T06:44:06.198 DEBUG:boto:Token: None 2022-09-03T06:44:06.199 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj85 2022-09-03T06:44:06.199 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YWyOjzLpTg9cC+BabjYOslnYtJ8= 2022-09-03T06:44:06.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YWyOjzLpTg9cC+BabjYOslnYtJ8=', 'Content-Length': '0'} 2022-09-03T06:44:06.202 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db1e4bc157895bae-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.203 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:44:06.204 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:06.204 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:06.205 DEBUG:boto:Method: GET 2022-09-03T06:44:06.205 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:44:06.206 DEBUG:boto:Data: 2022-09-03T06:44:06.207 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.208 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:06.208 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.209 DEBUG:boto:Params: {} 2022-09-03T06:44:06.209 DEBUG:boto:Token: None 2022-09-03T06:44:06.210 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj85 2022-09-03T06:44:06.211 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b9igB510uDzStK6sHo3t+TQrfuI= 2022-09-03T06:44:06.211 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b9igB510uDzStK6sHo3t+TQrfuI=', 'Content-Length': '0'} 2022-09-03T06:44:06.214 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004d0eaea2c99cf81b-006312f7b6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.215 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:06.217 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:06.218 DEBUG:boto:Method: GET 2022-09-03T06:44:06.219 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:44:06.220 DEBUG:boto:Data: 2022-09-03T06:44:06.221 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.222 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.223 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.224 DEBUG:boto:Params: {} 2022-09-03T06:44:06.225 DEBUG:boto:Token: None 2022-09-03T06:44:06.226 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj85 2022-09-03T06:44:06.226 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b9igB510uDzStK6sHo3t+TQrfuI= 2022-09-03T06:44:06.227 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b9igB510uDzStK6sHo3t+TQrfuI=', 'Content-Length': '0'} 2022-09-03T06:44:06.230 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f9a52ca3af9aa59d-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.275 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:06.275 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:06.276 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:06.277 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:06.278 DEBUG:boto:Method: GET 2022-09-03T06:44:06.278 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:44:06.279 DEBUG:boto:Data: 2022-09-03T06:44:06.280 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.280 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:06.281 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.281 DEBUG:boto:Params: {} 2022-09-03T06:44:06.282 DEBUG:boto:Token: None 2022-09-03T06:44:06.283 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/ 2022-09-03T06:44:06.283 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw= 2022-09-03T06:44:06.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw=', 'Content-Length': '0'} 2022-09-03T06:44:06.286 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006113562374ac7aee-006312f7b6-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.288 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:06.289 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:06.289 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:06.290 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:06.292 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:06.293 DEBUG:boto:Method: GET 2022-09-03T06:44:06.294 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:44:06.294 DEBUG:boto:Data: 2022-09-03T06:44:06.295 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.296 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.296 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.297 DEBUG:boto:Params: {} 2022-09-03T06:44:06.298 DEBUG:boto:Token: None 2022-09-03T06:44:06.299 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/ 2022-09-03T06:44:06.299 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw= 2022-09-03T06:44:06.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw=', 'Content-Length': '0'} 2022-09-03T06:44:06.304 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000052d0aa6937869a8a-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.305 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:06.306 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:44:06.307 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:06.308 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:06.309 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.310 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.311 DEBUG:boto:Method: GET 2022-09-03T06:44:06.312 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.313 DEBUG:boto:Data: 2022-09-03T06:44:06.315 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.315 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:06.316 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.316 DEBUG:boto:Params: {} 2022-09-03T06:44:06.317 DEBUG:boto:Token: None 2022-09-03T06:44:06.318 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.318 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j8vqCv3KHOIRA1AoNHhaCMIX9WA= 2022-09-03T06:44:06.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j8vqCv3KHOIRA1AoNHhaCMIX9WA=', 'Content-Length': '0'} 2022-09-03T06:44:06.322 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cb05c89c255731d4-006312f7b6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.323 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:06.324 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:06.324 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.325 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.326 DEBUG:boto:Method: GET 2022-09-03T06:44:06.327 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.327 DEBUG:boto:Data: 2022-09-03T06:44:06.328 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.328 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.329 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.329 DEBUG:boto:Params: {} 2022-09-03T06:44:06.330 DEBUG:boto:Token: None 2022-09-03T06:44:06.331 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.332 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j8vqCv3KHOIRA1AoNHhaCMIX9WA= 2022-09-03T06:44:06.332 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j8vqCv3KHOIRA1AoNHhaCMIX9WA=', 'Content-Length': '0'} 2022-09-03T06:44:06.335 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bf2b879b88fcd860-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.335 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:06.336 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:06.337 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.338 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.339 DEBUG:boto:Method: HEAD 2022-09-03T06:44:06.339 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.340 DEBUG:boto:Data: 2022-09-03T06:44:06.341 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.342 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:06.343 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.344 DEBUG:boto:Params: {} 2022-09-03T06:44:06.345 DEBUG:boto:Token: None 2022-09-03T06:44:06.346 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.347 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HdcP1zXnZeEDypI/o1XxmmMJ93Y= 2022-09-03T06:44:06.349 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HdcP1zXnZeEDypI/o1XxmmMJ93Y=', 'Content-Length': '0'} 2022-09-03T06:44:06.351 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e25ac3eb60819d3e-006312f7b6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.352 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:06.353 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:06.354 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.354 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.355 DEBUG:boto:Method: HEAD 2022-09-03T06:44:06.356 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.357 DEBUG:boto:Data: 2022-09-03T06:44:06.357 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.358 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.358 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.359 DEBUG:boto:Params: {} 2022-09-03T06:44:06.360 DEBUG:boto:Token: None 2022-09-03T06:44:06.360 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.361 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HdcP1zXnZeEDypI/o1XxmmMJ93Y= 2022-09-03T06:44:06.362 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HdcP1zXnZeEDypI/o1XxmmMJ93Y=', 'Content-Length': '0'} 2022-09-03T06:44:06.364 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a35fed135013998-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.365 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:44:06.366 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:06.367 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:06.368 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.369 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.369 DEBUG:boto:Method: GET 2022-09-03T06:44:06.370 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.370 DEBUG:boto:Data: 2022-09-03T06:44:06.371 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.372 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:06.372 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.373 DEBUG:boto:Params: {} 2022-09-03T06:44:06.373 DEBUG:boto:Token: None 2022-09-03T06:44:06.374 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.375 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j8vqCv3KHOIRA1AoNHhaCMIX9WA= 2022-09-03T06:44:06.376 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j8vqCv3KHOIRA1AoNHhaCMIX9WA=', 'Content-Length': '0'} 2022-09-03T06:44:06.378 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d68b389d423a6295-006312f7b6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.422 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:06.424 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:06.425 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.426 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.427 DEBUG:boto:Method: GET 2022-09-03T06:44:06.428 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.429 DEBUG:boto:Data: 2022-09-03T06:44:06.429 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.430 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.431 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.431 DEBUG:boto:Params: {} 2022-09-03T06:44:06.432 DEBUG:boto:Token: None 2022-09-03T06:44:06.433 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:06.434 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j8vqCv3KHOIRA1AoNHhaCMIX9WA= 2022-09-03T06:44:06.434 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j8vqCv3KHOIRA1AoNHhaCMIX9WA=', 'Content-Length': '0'} 2022-09-03T06:44:06.439 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008a7dfec601cd50d5-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.440 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:06.440 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:06.441 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:06.441 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:06.442 DEBUG:boto:Method: GET 2022-09-03T06:44:06.443 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:44:06.443 DEBUG:boto:Data: 2022-09-03T06:44:06.445 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.446 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:06.446 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.447 DEBUG:boto:Params: {} 2022-09-03T06:44:06.448 DEBUG:boto:Token: None 2022-09-03T06:44:06.449 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/ 2022-09-03T06:44:06.449 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw= 2022-09-03T06:44:06.450 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw=', 'Content-Length': '0'} 2022-09-03T06:44:06.453 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000d5c53d09b943b1f-006312f7b6-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.455 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:06.456 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:06.457 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:06.458 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:06.458 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:06.459 DEBUG:boto:Method: GET 2022-09-03T06:44:06.460 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:44:06.460 DEBUG:boto:Data: 2022-09-03T06:44:06.461 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.461 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.462 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.463 DEBUG:boto:Params: {} 2022-09-03T06:44:06.464 DEBUG:boto:Token: None 2022-09-03T06:44:06.464 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/ 2022-09-03T06:44:06.466 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw= 2022-09-03T06:44:06.466 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw=', 'Content-Length': '0'} 2022-09-03T06:44:06.470 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000568457de3b39a128-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.471 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:06.473 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:06.474 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:06.475 DEBUG:boto:Method: HEAD 2022-09-03T06:44:06.476 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:44:06.478 DEBUG:boto:Data: 2022-09-03T06:44:06.479 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.480 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:06.482 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.483 DEBUG:boto:Params: {} 2022-09-03T06:44:06.484 DEBUG:boto:Token: None 2022-09-03T06:44:06.485 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj92 2022-09-03T06:44:06.487 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qzRx7GWvMNXeP8C/zHcQdTp1VN8= 2022-09-03T06:44:06.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qzRx7GWvMNXeP8C/zHcQdTp1VN8=', 'Content-Length': '0'} 2022-09-03T06:44:06.491 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001e317cca0d2a12a8-006312f7b6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.492 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:06.493 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:06.494 DEBUG:boto:Method: HEAD 2022-09-03T06:44:06.495 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:44:06.496 DEBUG:boto:Data: 2022-09-03T06:44:06.497 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.498 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.499 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.500 DEBUG:boto:Params: {} 2022-09-03T06:44:06.501 DEBUG:boto:Token: None 2022-09-03T06:44:06.503 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj92 2022-09-03T06:44:06.504 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qzRx7GWvMNXeP8C/zHcQdTp1VN8= 2022-09-03T06:44:06.505 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qzRx7GWvMNXeP8C/zHcQdTp1VN8=', 'Content-Length': '0'} 2022-09-03T06:44:06.508 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001f0c6ab8d433e5f9-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.509 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:44:06.510 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:06.511 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:06.512 DEBUG:boto:Method: GET 2022-09-03T06:44:06.513 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:44:06.514 DEBUG:boto:Data: 2022-09-03T06:44:06.515 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.516 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:06.518 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.519 DEBUG:boto:Params: {} 2022-09-03T06:44:06.520 DEBUG:boto:Token: None 2022-09-03T06:44:06.521 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj92 2022-09-03T06:44:06.522 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8a+c48qgwamTox3hrmBF8SSOxJo= 2022-09-03T06:44:06.523 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8a+c48qgwamTox3hrmBF8SSOxJo=', 'Content-Length': '0'} 2022-09-03T06:44:06.525 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d8c7db30eefa0c2-006312f7b6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.526 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:06.527 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:06.529 DEBUG:boto:Method: GET 2022-09-03T06:44:06.530 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:44:06.531 DEBUG:boto:Data: 2022-09-03T06:44:06.531 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.532 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.533 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.534 DEBUG:boto:Params: {} 2022-09-03T06:44:06.535 DEBUG:boto:Token: None 2022-09-03T06:44:06.535 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj92 2022-09-03T06:44:06.536 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8a+c48qgwamTox3hrmBF8SSOxJo= 2022-09-03T06:44:06.537 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8a+c48qgwamTox3hrmBF8SSOxJo=', 'Content-Length': '0'} 2022-09-03T06:44:06.540 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048f4e1a8e6440748-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.541 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:06.542 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:06.542 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:06.543 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:06.543 DEBUG:boto:Method: GET 2022-09-03T06:44:06.544 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:44:06.544 DEBUG:boto:Data: 2022-09-03T06:44:06.545 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.546 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:06.546 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.547 DEBUG:boto:Params: {} 2022-09-03T06:44:06.548 DEBUG:boto:Token: None 2022-09-03T06:44:06.548 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/ 2022-09-03T06:44:06.549 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw= 2022-09-03T06:44:06.550 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw=', 'Content-Length': '0'} 2022-09-03T06:44:06.553 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bbea5decbfd23b73-006312f7b6-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.554 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:06.556 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:06.558 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:06.559 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:06.560 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:06.561 DEBUG:boto:Method: GET 2022-09-03T06:44:06.561 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:44:06.563 DEBUG:boto:Data: 2022-09-03T06:44:06.563 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.564 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.564 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.565 DEBUG:boto:Params: {} 2022-09-03T06:44:06.565 DEBUG:boto:Token: None 2022-09-03T06:44:06.567 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/ 2022-09-03T06:44:06.568 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw= 2022-09-03T06:44:06.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw=', 'Content-Length': '0'} 2022-09-03T06:44:06.572 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000acc18ec57c0513fb-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.573 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:06.574 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:44:06.574 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:06.575 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:06.575 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.576 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.577 DEBUG:boto:Method: GET 2022-09-03T06:44:06.577 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.578 DEBUG:boto:Data: 2022-09-03T06:44:06.578 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.579 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:06.580 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.580 DEBUG:boto:Params: {} 2022-09-03T06:44:06.581 DEBUG:boto:Token: None 2022-09-03T06:44:06.582 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.583 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HOicHJWcRzrbo10O9MMMG10/Jh8= 2022-09-03T06:44:06.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HOicHJWcRzrbo10O9MMMG10/Jh8=', 'Content-Length': '0'} 2022-09-03T06:44:06.586 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ea24f10e960bdbff-006312f7b6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.587 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:06.587 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:06.588 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.588 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.589 DEBUG:boto:Method: GET 2022-09-03T06:44:06.589 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.590 DEBUG:boto:Data: 2022-09-03T06:44:06.591 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.591 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.592 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.593 DEBUG:boto:Params: {} 2022-09-03T06:44:06.593 DEBUG:boto:Token: None 2022-09-03T06:44:06.594 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.595 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HOicHJWcRzrbo10O9MMMG10/Jh8= 2022-09-03T06:44:06.595 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HOicHJWcRzrbo10O9MMMG10/Jh8=', 'Content-Length': '0'} 2022-09-03T06:44:06.597 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000034d53b36e2f0fc63-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.642 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:06.643 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:06.644 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.645 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.645 DEBUG:boto:Method: HEAD 2022-09-03T06:44:06.646 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.647 DEBUG:boto:Data: 2022-09-03T06:44:06.647 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.648 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:06.648 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.649 DEBUG:boto:Params: {} 2022-09-03T06:44:06.650 DEBUG:boto:Token: None 2022-09-03T06:44:06.650 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.651 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O2YZgYHeyWtlMu0XJq4NxATGJdM= 2022-09-03T06:44:06.653 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O2YZgYHeyWtlMu0XJq4NxATGJdM=', 'Content-Length': '0'} 2022-09-03T06:44:06.655 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000013a46f7d5295c893-006312f7b6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.656 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:06.657 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:06.657 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.658 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.658 DEBUG:boto:Method: HEAD 2022-09-03T06:44:06.659 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.659 DEBUG:boto:Data: 2022-09-03T06:44:06.660 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.660 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.661 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.661 DEBUG:boto:Params: {} 2022-09-03T06:44:06.662 DEBUG:boto:Token: None 2022-09-03T06:44:06.663 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.664 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O2YZgYHeyWtlMu0XJq4NxATGJdM= 2022-09-03T06:44:06.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O2YZgYHeyWtlMu0XJq4NxATGJdM=', 'Content-Length': '0'} 2022-09-03T06:44:06.667 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000052c0f1b52776d8cc-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.668 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:44:06.668 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:06.669 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:06.669 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.670 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.671 DEBUG:boto:Method: GET 2022-09-03T06:44:06.671 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.672 DEBUG:boto:Data: 2022-09-03T06:44:06.672 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.673 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:06.673 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.674 DEBUG:boto:Params: {} 2022-09-03T06:44:06.675 DEBUG:boto:Token: None 2022-09-03T06:44:06.676 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.677 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HOicHJWcRzrbo10O9MMMG10/Jh8= 2022-09-03T06:44:06.678 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HOicHJWcRzrbo10O9MMMG10/Jh8=', 'Content-Length': '0'} 2022-09-03T06:44:06.688 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a1164294f49416b-006312f7b6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.693 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:06.694 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:06.695 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.696 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.697 DEBUG:boto:Method: GET 2022-09-03T06:44:06.698 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.699 DEBUG:boto:Data: 2022-09-03T06:44:06.700 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.701 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.702 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.703 DEBUG:boto:Params: {} 2022-09-03T06:44:06.704 DEBUG:boto:Token: None 2022-09-03T06:44:06.705 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:06.707 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HOicHJWcRzrbo10O9MMMG10/Jh8= 2022-09-03T06:44:06.709 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HOicHJWcRzrbo10O9MMMG10/Jh8=', 'Content-Length': '0'} 2022-09-03T06:44:06.713 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000dfd662eb689a4f0-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.715 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:06.716 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:06.717 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:06.718 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:06.719 DEBUG:boto:Method: GET 2022-09-03T06:44:06.720 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:44:06.722 DEBUG:boto:Data: 2022-09-03T06:44:06.723 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.724 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:06.725 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.726 DEBUG:boto:Params: {} 2022-09-03T06:44:06.727 DEBUG:boto:Token: None 2022-09-03T06:44:06.728 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/ 2022-09-03T06:44:06.729 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw= 2022-09-03T06:44:06.730 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw=', 'Content-Length': '0'} 2022-09-03T06:44:06.734 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000035da2cce35ff9b6-006312f7b6-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.735 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:06.737 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:06.738 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:06.739 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:06.740 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:06.741 DEBUG:boto:Method: GET 2022-09-03T06:44:06.742 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:44:06.743 DEBUG:boto:Data: 2022-09-03T06:44:06.744 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.745 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.746 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.755 DEBUG:boto:Params: {} 2022-09-03T06:44:06.756 DEBUG:boto:Token: None 2022-09-03T06:44:06.757 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/ 2022-09-03T06:44:06.758 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw= 2022-09-03T06:44:06.759 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw=', 'Content-Length': '0'} 2022-09-03T06:44:06.764 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007670085590a7eaf1-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.765 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:06.767 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:06.768 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:06.769 DEBUG:boto:Method: HEAD 2022-09-03T06:44:06.770 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:44:06.771 DEBUG:boto:Data: 2022-09-03T06:44:06.772 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.773 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:06.774 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.775 DEBUG:boto:Params: {} 2022-09-03T06:44:06.776 DEBUG:boto:Token: None 2022-09-03T06:44:06.777 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj95 2022-09-03T06:44:06.778 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:siU89GY6bIHlbacTs3TITG+OibE= 2022-09-03T06:44:06.779 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:siU89GY6bIHlbacTs3TITG+OibE=', 'Content-Length': '0'} 2022-09-03T06:44:06.782 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d9ec2ceed946ba1b-006312f7b6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.783 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:06.784 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:06.785 DEBUG:boto:Method: HEAD 2022-09-03T06:44:06.786 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:44:06.787 DEBUG:boto:Data: 2022-09-03T06:44:06.788 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.789 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.790 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.791 DEBUG:boto:Params: {} 2022-09-03T06:44:06.792 DEBUG:boto:Token: None 2022-09-03T06:44:06.793 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj95 2022-09-03T06:44:06.794 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:siU89GY6bIHlbacTs3TITG+OibE= 2022-09-03T06:44:06.795 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:siU89GY6bIHlbacTs3TITG+OibE=', 'Content-Length': '0'} 2022-09-03T06:44:06.797 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d593bb4d55903877-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.798 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:44:06.799 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:06.799 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:06.800 DEBUG:boto:Method: GET 2022-09-03T06:44:06.800 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:44:06.801 DEBUG:boto:Data: 2022-09-03T06:44:06.801 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.802 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:06.803 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.803 DEBUG:boto:Params: {} 2022-09-03T06:44:06.804 DEBUG:boto:Token: None 2022-09-03T06:44:06.804 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj95 2022-09-03T06:44:06.805 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i1zd2NtfPv3xFn/w7lRWt228fw8= 2022-09-03T06:44:06.806 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i1zd2NtfPv3xFn/w7lRWt228fw8=', 'Content-Length': '0'} 2022-09-03T06:44:06.808 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000129dd401a51c61ad-006312f7b6-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.809 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:06.809 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:06.810 DEBUG:boto:Method: GET 2022-09-03T06:44:06.810 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:44:06.811 DEBUG:boto:Data: 2022-09-03T06:44:06.812 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.812 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.813 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.813 DEBUG:boto:Params: {} 2022-09-03T06:44:06.814 DEBUG:boto:Token: None 2022-09-03T06:44:06.815 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/obj95 2022-09-03T06:44:06.816 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i1zd2NtfPv3xFn/w7lRWt228fw8= 2022-09-03T06:44:06.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i1zd2NtfPv3xFn/w7lRWt228fw8=', 'Content-Length': '0'} 2022-09-03T06:44:06.819 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000528d68f9b999a18a-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.820 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:06.821 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:06.822 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:06.822 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:06.823 DEBUG:boto:Method: GET 2022-09-03T06:44:06.823 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:44:06.824 DEBUG:boto:Data: 2022-09-03T06:44:06.825 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.826 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:06.827 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.828 DEBUG:boto:Params: {} 2022-09-03T06:44:06.830 DEBUG:boto:Token: None 2022-09-03T06:44:06.831 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/ 2022-09-03T06:44:06.832 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw= 2022-09-03T06:44:06.833 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw=', 'Content-Length': '0'} 2022-09-03T06:44:06.838 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000000d81adbd4365876-006312f7b6-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.840 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:06.841 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:06.842 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:06.843 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:06.844 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:06.845 DEBUG:boto:Method: GET 2022-09-03T06:44:06.845 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:44:06.846 DEBUG:boto:Data: 2022-09-03T06:44:06.847 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.848 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.848 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.849 DEBUG:boto:Params: {} 2022-09-03T06:44:06.850 DEBUG:boto:Token: None 2022-09-03T06:44:06.850 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/ 2022-09-03T06:44:06.851 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw= 2022-09-03T06:44:06.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F9z9dW/za++VX1BmkqTEdQCFYEw=', 'Content-Length': '0'} 2022-09-03T06:44:06.855 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007bb93b020cbedbba-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.902 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:06.903 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-75 zones={test-zone2, test-zone1} 2022-09-03T06:44:06.904 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-75 zones={test-zone3, test-zone1} 2022-09-03T06:44:06.904 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:06.905 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:06.905 DEBUG:boto:Method: HEAD 2022-09-03T06:44:06.906 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:44:06.906 DEBUG:boto:Data: 2022-09-03T06:44:06.907 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.908 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:06.908 DEBUG:boto:Port: 8001 2022-09-03T06:44:06.909 DEBUG:boto:Params: {} 2022-09-03T06:44:06.911 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T06:44:06.912 DEBUG:boto:Token: None 2022-09-03T06:44:06.913 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/ 2022-09-03T06:44:06.914 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4f/KDLkUJSu6+3F9fjR5Fx7nrW0= 2022-09-03T06:44:06.915 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4f/KDLkUJSu6+3F9fjR5Fx7nrW0=', 'Content-Length': '0'} 2022-09-03T06:44:06.920 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '20'), ('X-RGW-Bytes-Used', '60'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000a140a86ab2e1e0e0-006312f7b6-1851-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.920 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:06.921 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:06.921 DEBUG:boto:Method: HEAD 2022-09-03T06:44:06.922 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:44:06.923 DEBUG:boto:Data: 2022-09-03T06:44:06.923 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.924 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.924 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.925 DEBUG:boto:Params: {} 2022-09-03T06:44:06.925 DEBUG:boto:Token: None 2022-09-03T06:44:06.926 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/ 2022-09-03T06:44:06.927 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4f/KDLkUJSu6+3F9fjR5Fx7nrW0= 2022-09-03T06:44:06.927 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4f/KDLkUJSu6+3F9fjR5Fx7nrW0=', 'Content-Length': '0'} 2022-09-03T06:44:06.931 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '20'), ('X-RGW-Bytes-Used', '60'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000f4aea7519a1edf2c-006312f7b6-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.931 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:44:06.932 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:06.933 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:06.933 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:06.934 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:06.934 DEBUG:boto:Method: GET 2022-09-03T06:44:06.935 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:06.935 DEBUG:boto:Data: 2022-09-03T06:44:06.936 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.936 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:06.937 DEBUG:boto:Port: 8001 2022-09-03T06:44:06.937 DEBUG:boto:Params: {} 2022-09-03T06:44:06.938 DEBUG:boto:Token: None 2022-09-03T06:44:06.939 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/?versions 2022-09-03T06:44:06.939 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v1OcbAVAhCV6sorJHMa+xkJEoFk= 2022-09-03T06:44:06.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v1OcbAVAhCV6sorJHMa+xkJEoFk=', 'Content-Length': '0'} 2022-09-03T06:44:06.943 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bab101b08b0a9b34-006312f7b6-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.944 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:06.946 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:44:06.946 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:44:06.947 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:44:06.947 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:44:06.948 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:44:06.948 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:44:06.949 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:44:06.949 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:44:06.950 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:44:06.950 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:44:06.951 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:44:06.951 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:44:06.952 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:44:06.952 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:44:06.953 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:44:06.953 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:44:06.954 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:44:06.955 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:44:06.955 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:44:06.956 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:44:06.956 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:44:06.957 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:06.957 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:06.958 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:06.959 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:06.959 DEBUG:boto:Method: GET 2022-09-03T06:44:06.960 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:06.960 DEBUG:boto:Data: 2022-09-03T06:44:06.961 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.962 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:06.962 DEBUG:boto:Port: 8000 2022-09-03T06:44:06.963 DEBUG:boto:Params: {} 2022-09-03T06:44:06.964 DEBUG:boto:Token: None 2022-09-03T06:44:06.964 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/?versions 2022-09-03T06:44:06.965 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v1OcbAVAhCV6sorJHMa+xkJEoFk= 2022-09-03T06:44:06.965 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v1OcbAVAhCV6sorJHMa+xkJEoFk=', 'Content-Length': '0'} 2022-09-03T06:44:06.969 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007c6d7784fbcf6351-006312f7b6-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.970 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:06.972 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:44:06.973 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:44:06.973 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:44:06.974 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:44:06.974 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:44:06.975 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:44:06.976 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:44:06.976 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:44:06.977 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:44:06.977 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:44:06.978 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:44:06.978 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:44:06.979 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:44:06.979 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:44:06.980 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:44:06.980 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:44:06.981 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:44:06.981 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:44:06.982 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:44:06.982 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:44:06.983 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:06.984 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:06.984 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:06.985 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:06.985 DEBUG:boto:Method: GET 2022-09-03T06:44:06.986 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:06.986 DEBUG:boto:Data: 2022-09-03T06:44:06.987 DEBUG:boto:Headers: {} 2022-09-03T06:44:06.987 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:06.988 DEBUG:boto:Port: 8001 2022-09-03T06:44:06.989 DEBUG:boto:Params: {} 2022-09-03T06:44:06.989 DEBUG:boto:Token: None 2022-09-03T06:44:06.990 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:06 GMT /dsgvcy-75/?versions 2022-09-03T06:44:06.990 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v1OcbAVAhCV6sorJHMa+xkJEoFk= 2022-09-03T06:44:06.991 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v1OcbAVAhCV6sorJHMa+xkJEoFk=', 'Content-Length': '0'} 2022-09-03T06:44:06.994 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008e8b8aca9c5e1bca-006312f7b6-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:06.995 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:06.997 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:06.997 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:06.998 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:06.998 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:06.999 DEBUG:boto:Method: GET 2022-09-03T06:44:06.999 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:07.000 DEBUG:boto:Data: 2022-09-03T06:44:07.001 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.001 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.002 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.002 DEBUG:boto:Params: {} 2022-09-03T06:44:07.003 DEBUG:boto:Token: None 2022-09-03T06:44:07.003 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/?versions 2022-09-03T06:44:07.004 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aiS85mC3jHB/tnYOp4IQRhxKu5M= 2022-09-03T06:44:07.005 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aiS85mC3jHB/tnYOp4IQRhxKu5M=', 'Content-Length': '0'} 2022-09-03T06:44:07.009 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000030655dd45c5191dd-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.009 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:07.011 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:44:07.012 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:07.012 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:07.013 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.013 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.014 DEBUG:boto:Method: GET 2022-09-03T06:44:07.014 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.015 DEBUG:boto:Data: 2022-09-03T06:44:07.016 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.016 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.017 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.017 DEBUG:boto:Params: {} 2022-09-03T06:44:07.018 DEBUG:boto:Token: None 2022-09-03T06:44:07.019 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.019 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dsnKmd/7Rb0Z/zRgC1fMWGEM8zA= 2022-09-03T06:44:07.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dsnKmd/7Rb0Z/zRgC1fMWGEM8zA=', 'Content-Length': '0'} 2022-09-03T06:44:07.022 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e336c626b561960-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.023 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:07.024 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:07.025 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.025 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.026 DEBUG:boto:Method: GET 2022-09-03T06:44:07.026 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.027 DEBUG:boto:Data: 2022-09-03T06:44:07.027 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.028 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.028 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.029 DEBUG:boto:Params: {} 2022-09-03T06:44:07.030 DEBUG:boto:Token: None 2022-09-03T06:44:07.030 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.031 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dsnKmd/7Rb0Z/zRgC1fMWGEM8zA= 2022-09-03T06:44:07.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dsnKmd/7Rb0Z/zRgC1fMWGEM8zA=', 'Content-Length': '0'} 2022-09-03T06:44:07.034 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ecee38f664065feb-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.074 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:07.075 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:07.076 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.076 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.077 DEBUG:boto:Method: HEAD 2022-09-03T06:44:07.077 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.078 DEBUG:boto:Data: 2022-09-03T06:44:07.078 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.079 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.079 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.080 DEBUG:boto:Params: {} 2022-09-03T06:44:07.081 DEBUG:boto:Token: None 2022-09-03T06:44:07.081 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.082 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gyr1a4gvFdBSrv0CbAU1dUKQIA8= 2022-09-03T06:44:07.083 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gyr1a4gvFdBSrv0CbAU1dUKQIA8=', 'Content-Length': '0'} 2022-09-03T06:44:07.085 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db3e4ccf4153d3db-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.086 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:07.087 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:07.087 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.088 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.088 DEBUG:boto:Method: HEAD 2022-09-03T06:44:07.089 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.089 DEBUG:boto:Data: 2022-09-03T06:44:07.090 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.091 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.091 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.092 DEBUG:boto:Params: {} 2022-09-03T06:44:07.092 DEBUG:boto:Token: None 2022-09-03T06:44:07.093 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.094 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gyr1a4gvFdBSrv0CbAU1dUKQIA8= 2022-09-03T06:44:07.094 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gyr1a4gvFdBSrv0CbAU1dUKQIA8=', 'Content-Length': '0'} 2022-09-03T06:44:07.096 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b361b16ad820e37-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.097 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:44:07.098 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:07.098 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:07.099 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.099 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.100 DEBUG:boto:Method: GET 2022-09-03T06:44:07.101 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.101 DEBUG:boto:Data: 2022-09-03T06:44:07.102 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.102 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.103 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.104 DEBUG:boto:Params: {} 2022-09-03T06:44:07.104 DEBUG:boto:Token: None 2022-09-03T06:44:07.105 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.106 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dsnKmd/7Rb0Z/zRgC1fMWGEM8zA= 2022-09-03T06:44:07.106 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dsnKmd/7Rb0Z/zRgC1fMWGEM8zA=', 'Content-Length': '0'} 2022-09-03T06:44:07.109 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f74629616178aa75-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.110 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:07.110 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:07.111 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.111 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.112 DEBUG:boto:Method: GET 2022-09-03T06:44:07.113 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.113 DEBUG:boto:Data: 2022-09-03T06:44:07.114 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.114 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.115 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.116 DEBUG:boto:Params: {} 2022-09-03T06:44:07.116 DEBUG:boto:Token: None 2022-09-03T06:44:07.117 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:07.118 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dsnKmd/7Rb0Z/zRgC1fMWGEM8zA= 2022-09-03T06:44:07.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dsnKmd/7Rb0Z/zRgC1fMWGEM8zA=', 'Content-Length': '0'} 2022-09-03T06:44:07.121 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006485a344f12f304c-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.122 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:07.122 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:07.123 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:07.124 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:07.124 DEBUG:boto:Method: GET 2022-09-03T06:44:07.125 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:44:07.125 DEBUG:boto:Data: 2022-09-03T06:44:07.126 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.127 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.127 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.128 DEBUG:boto:Params: {} 2022-09-03T06:44:07.128 DEBUG:boto:Token: None 2022-09-03T06:44:07.129 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/ 2022-09-03T06:44:07.130 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc= 2022-09-03T06:44:07.130 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc=', 'Content-Length': '0'} 2022-09-03T06:44:07.133 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003ddf7516bb836069-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.134 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:07.135 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:07.136 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:07.137 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:07.137 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:07.138 DEBUG:boto:Method: GET 2022-09-03T06:44:07.138 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:44:07.139 DEBUG:boto:Data: 2022-09-03T06:44:07.140 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.140 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.141 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.141 DEBUG:boto:Params: {} 2022-09-03T06:44:07.142 DEBUG:boto:Token: None 2022-09-03T06:44:07.143 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/ 2022-09-03T06:44:07.143 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc= 2022-09-03T06:44:07.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc=', 'Content-Length': '0'} 2022-09-03T06:44:07.148 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000795b616657435fda-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.148 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:07.149 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:07.150 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:07.151 DEBUG:boto:Method: HEAD 2022-09-03T06:44:07.151 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:44:07.152 DEBUG:boto:Data: 2022-09-03T06:44:07.152 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.153 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.153 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.154 DEBUG:boto:Params: {} 2022-09-03T06:44:07.155 DEBUG:boto:Token: None 2022-09-03T06:44:07.155 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj11 2022-09-03T06:44:07.156 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OZu+NLoetIN6tbWbQqbgaIpZkcA= 2022-09-03T06:44:07.157 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OZu+NLoetIN6tbWbQqbgaIpZkcA=', 'Content-Length': '0'} 2022-09-03T06:44:07.159 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000353b36e0b033b9f0-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.159 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:07.160 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:07.161 DEBUG:boto:Method: HEAD 2022-09-03T06:44:07.161 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:44:07.162 DEBUG:boto:Data: 2022-09-03T06:44:07.162 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.163 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.163 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.164 DEBUG:boto:Params: {} 2022-09-03T06:44:07.165 DEBUG:boto:Token: None 2022-09-03T06:44:07.165 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj11 2022-09-03T06:44:07.166 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OZu+NLoetIN6tbWbQqbgaIpZkcA= 2022-09-03T06:44:07.167 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OZu+NLoetIN6tbWbQqbgaIpZkcA=', 'Content-Length': '0'} 2022-09-03T06:44:07.170 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f4c4d0b7f7b64a64-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.172 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:44:07.173 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:07.174 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:07.175 DEBUG:boto:Method: GET 2022-09-03T06:44:07.176 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:44:07.177 DEBUG:boto:Data: 2022-09-03T06:44:07.178 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.179 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.180 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.181 DEBUG:boto:Params: {} 2022-09-03T06:44:07.182 DEBUG:boto:Token: None 2022-09-03T06:44:07.182 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj11 2022-09-03T06:44:07.183 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/7l4hUfYXQk3gSe1IsTBNHIfGKk= 2022-09-03T06:44:07.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/7l4hUfYXQk3gSe1IsTBNHIfGKk=', 'Content-Length': '0'} 2022-09-03T06:44:07.186 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000bcada8bb678dc91-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.230 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:07.231 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:07.232 DEBUG:boto:Method: GET 2022-09-03T06:44:07.232 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:44:07.233 DEBUG:boto:Data: 2022-09-03T06:44:07.233 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.234 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.235 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.235 DEBUG:boto:Params: {} 2022-09-03T06:44:07.236 DEBUG:boto:Token: None 2022-09-03T06:44:07.237 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj11 2022-09-03T06:44:07.238 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/7l4hUfYXQk3gSe1IsTBNHIfGKk= 2022-09-03T06:44:07.238 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/7l4hUfYXQk3gSe1IsTBNHIfGKk=', 'Content-Length': '0'} 2022-09-03T06:44:07.241 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d917aaf0dd542bd5-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.241 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:07.242 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:07.243 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:07.244 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:07.244 DEBUG:boto:Method: GET 2022-09-03T06:44:07.245 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:44:07.245 DEBUG:boto:Data: 2022-09-03T06:44:07.246 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.246 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.247 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.248 DEBUG:boto:Params: {} 2022-09-03T06:44:07.248 DEBUG:boto:Token: None 2022-09-03T06:44:07.249 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/ 2022-09-03T06:44:07.250 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc= 2022-09-03T06:44:07.250 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc=', 'Content-Length': '0'} 2022-09-03T06:44:07.253 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000072d93382d826aaea-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.254 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:07.255 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:07.256 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:07.256 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:07.257 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:07.258 DEBUG:boto:Method: GET 2022-09-03T06:44:07.258 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:44:07.259 DEBUG:boto:Data: 2022-09-03T06:44:07.259 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.260 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.261 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.261 DEBUG:boto:Params: {} 2022-09-03T06:44:07.262 DEBUG:boto:Token: None 2022-09-03T06:44:07.263 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/ 2022-09-03T06:44:07.264 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc= 2022-09-03T06:44:07.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc=', 'Content-Length': '0'} 2022-09-03T06:44:07.268 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e68075595d32354b-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.269 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:07.270 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:44:07.271 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:07.272 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:07.273 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.274 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.275 DEBUG:boto:Method: GET 2022-09-03T06:44:07.276 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.277 DEBUG:boto:Data: 2022-09-03T06:44:07.277 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.278 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.278 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.279 DEBUG:boto:Params: {} 2022-09-03T06:44:07.280 DEBUG:boto:Token: None 2022-09-03T06:44:07.281 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.281 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Nso6AaH2SsW+QVAayzmcC0/rQFs= 2022-09-03T06:44:07.282 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Nso6AaH2SsW+QVAayzmcC0/rQFs=', 'Content-Length': '0'} 2022-09-03T06:44:07.285 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000891ffe2982510cb1-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.286 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:07.287 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:07.288 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.289 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.290 DEBUG:boto:Method: GET 2022-09-03T06:44:07.291 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.292 DEBUG:boto:Data: 2022-09-03T06:44:07.292 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.293 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.294 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.294 DEBUG:boto:Params: {} 2022-09-03T06:44:07.295 DEBUG:boto:Token: None 2022-09-03T06:44:07.296 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.296 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Nso6AaH2SsW+QVAayzmcC0/rQFs= 2022-09-03T06:44:07.297 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Nso6AaH2SsW+QVAayzmcC0/rQFs=', 'Content-Length': '0'} 2022-09-03T06:44:07.300 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001422980405ca5375-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.342 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:07.343 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:07.344 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.344 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.345 DEBUG:boto:Method: HEAD 2022-09-03T06:44:07.345 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.346 DEBUG:boto:Data: 2022-09-03T06:44:07.347 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.347 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.348 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.349 DEBUG:boto:Params: {} 2022-09-03T06:44:07.351 DEBUG:boto:Token: None 2022-09-03T06:44:07.352 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.353 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dQwhCh4l0hUDSmcRZrhPNtr4Nf0= 2022-09-03T06:44:07.354 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dQwhCh4l0hUDSmcRZrhPNtr4Nf0=', 'Content-Length': '0'} 2022-09-03T06:44:07.357 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dfbd8a362a874d3e-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.358 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:07.358 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:07.359 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.359 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.360 DEBUG:boto:Method: HEAD 2022-09-03T06:44:07.361 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.362 DEBUG:boto:Data: 2022-09-03T06:44:07.363 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.363 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.364 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.365 DEBUG:boto:Params: {} 2022-09-03T06:44:07.366 DEBUG:boto:Token: None 2022-09-03T06:44:07.367 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.368 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dQwhCh4l0hUDSmcRZrhPNtr4Nf0= 2022-09-03T06:44:07.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dQwhCh4l0hUDSmcRZrhPNtr4Nf0=', 'Content-Length': '0'} 2022-09-03T06:44:07.371 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000095e1f0baf181ac56-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.372 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:44:07.373 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:07.373 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:07.374 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.375 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.376 DEBUG:boto:Method: GET 2022-09-03T06:44:07.376 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.377 DEBUG:boto:Data: 2022-09-03T06:44:07.377 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.378 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.379 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.379 DEBUG:boto:Params: {} 2022-09-03T06:44:07.380 DEBUG:boto:Token: None 2022-09-03T06:44:07.380 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.381 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Nso6AaH2SsW+QVAayzmcC0/rQFs= 2022-09-03T06:44:07.382 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Nso6AaH2SsW+QVAayzmcC0/rQFs=', 'Content-Length': '0'} 2022-09-03T06:44:07.384 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bf002e9f4debe9b6-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.385 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:07.386 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:07.387 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.387 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.388 DEBUG:boto:Method: GET 2022-09-03T06:44:07.388 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.389 DEBUG:boto:Data: 2022-09-03T06:44:07.389 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.390 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.391 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.391 DEBUG:boto:Params: {} 2022-09-03T06:44:07.392 DEBUG:boto:Token: None 2022-09-03T06:44:07.393 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:07.393 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Nso6AaH2SsW+QVAayzmcC0/rQFs= 2022-09-03T06:44:07.394 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Nso6AaH2SsW+QVAayzmcC0/rQFs=', 'Content-Length': '0'} 2022-09-03T06:44:07.397 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aec4e3b84573fd4a-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.397 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:07.398 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:07.399 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:07.399 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:07.400 DEBUG:boto:Method: GET 2022-09-03T06:44:07.400 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:44:07.401 DEBUG:boto:Data: 2022-09-03T06:44:07.401 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.402 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.403 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.403 DEBUG:boto:Params: {} 2022-09-03T06:44:07.404 DEBUG:boto:Token: None 2022-09-03T06:44:07.405 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/ 2022-09-03T06:44:07.406 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc= 2022-09-03T06:44:07.407 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc=', 'Content-Length': '0'} 2022-09-03T06:44:07.410 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b10a591ef89b62bf-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.410 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:07.412 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:07.412 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:07.413 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:07.413 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:07.414 DEBUG:boto:Method: GET 2022-09-03T06:44:07.414 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:44:07.415 DEBUG:boto:Data: 2022-09-03T06:44:07.415 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.416 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.416 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.417 DEBUG:boto:Params: {} 2022-09-03T06:44:07.418 DEBUG:boto:Token: None 2022-09-03T06:44:07.420 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/ 2022-09-03T06:44:07.421 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc= 2022-09-03T06:44:07.422 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc=', 'Content-Length': '0'} 2022-09-03T06:44:07.426 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006513a500c51ea390-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.427 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:07.428 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:07.429 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:07.429 DEBUG:boto:Method: HEAD 2022-09-03T06:44:07.430 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:44:07.431 DEBUG:boto:Data: 2022-09-03T06:44:07.431 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.432 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.432 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.433 DEBUG:boto:Params: {} 2022-09-03T06:44:07.433 DEBUG:boto:Token: None 2022-09-03T06:44:07.434 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj115 2022-09-03T06:44:07.435 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r3tZTGCr8WxGFN5ATVLPh5NM5fk= 2022-09-03T06:44:07.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r3tZTGCr8WxGFN5ATVLPh5NM5fk=', 'Content-Length': '0'} 2022-09-03T06:44:07.438 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca6f6e7009bc9c82-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.439 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:07.440 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:07.440 DEBUG:boto:Method: HEAD 2022-09-03T06:44:07.441 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:44:07.442 DEBUG:boto:Data: 2022-09-03T06:44:07.442 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.443 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.443 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.444 DEBUG:boto:Params: {} 2022-09-03T06:44:07.444 DEBUG:boto:Token: None 2022-09-03T06:44:07.445 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj115 2022-09-03T06:44:07.446 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r3tZTGCr8WxGFN5ATVLPh5NM5fk= 2022-09-03T06:44:07.446 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r3tZTGCr8WxGFN5ATVLPh5NM5fk=', 'Content-Length': '0'} 2022-09-03T06:44:07.448 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000748d8fdaa18a2474-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.449 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:44:07.450 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:07.450 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:07.451 DEBUG:boto:Method: GET 2022-09-03T06:44:07.452 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:44:07.452 DEBUG:boto:Data: 2022-09-03T06:44:07.453 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.453 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.454 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.454 DEBUG:boto:Params: {} 2022-09-03T06:44:07.455 DEBUG:boto:Token: None 2022-09-03T06:44:07.456 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj115 2022-09-03T06:44:07.456 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OBLHEJXx99KninmtX91oGeykadk= 2022-09-03T06:44:07.457 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OBLHEJXx99KninmtX91oGeykadk=', 'Content-Length': '0'} 2022-09-03T06:44:07.459 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d69874b98bd016ba-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.460 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:07.461 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:07.461 DEBUG:boto:Method: GET 2022-09-03T06:44:07.462 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:44:07.462 DEBUG:boto:Data: 2022-09-03T06:44:07.463 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.463 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.464 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.464 DEBUG:boto:Params: {} 2022-09-03T06:44:07.465 DEBUG:boto:Token: None 2022-09-03T06:44:07.466 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj115 2022-09-03T06:44:07.466 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OBLHEJXx99KninmtX91oGeykadk= 2022-09-03T06:44:07.467 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OBLHEJXx99KninmtX91oGeykadk=', 'Content-Length': '0'} 2022-09-03T06:44:07.469 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ddcad33ee6bdc133-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.470 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:07.471 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:07.471 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:07.472 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:07.472 DEBUG:boto:Method: GET 2022-09-03T06:44:07.473 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:44:07.473 DEBUG:boto:Data: 2022-09-03T06:44:07.474 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.474 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.475 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.475 DEBUG:boto:Params: {} 2022-09-03T06:44:07.476 DEBUG:boto:Token: None 2022-09-03T06:44:07.476 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/ 2022-09-03T06:44:07.477 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc= 2022-09-03T06:44:07.478 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc=', 'Content-Length': '0'} 2022-09-03T06:44:07.480 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cf8c65f256c3f4fa-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.481 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:07.482 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:07.483 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:07.483 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:07.484 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:07.484 DEBUG:boto:Method: GET 2022-09-03T06:44:07.485 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:44:07.485 DEBUG:boto:Data: 2022-09-03T06:44:07.486 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.486 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.487 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.487 DEBUG:boto:Params: {} 2022-09-03T06:44:07.488 DEBUG:boto:Token: None 2022-09-03T06:44:07.488 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/ 2022-09-03T06:44:07.489 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc= 2022-09-03T06:44:07.490 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc=', 'Content-Length': '0'} 2022-09-03T06:44:07.494 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000072222c477a8b207b-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.494 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:07.495 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:44:07.496 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:07.496 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:07.497 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.497 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.498 DEBUG:boto:Method: GET 2022-09-03T06:44:07.498 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.499 DEBUG:boto:Data: 2022-09-03T06:44:07.499 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.500 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.500 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.501 DEBUG:boto:Params: {} 2022-09-03T06:44:07.502 DEBUG:boto:Token: None 2022-09-03T06:44:07.502 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.503 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2MMRB05vzO+Y9zuB1CtzJXF7j00= 2022-09-03T06:44:07.504 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2MMRB05vzO+Y9zuB1CtzJXF7j00=', 'Content-Length': '0'} 2022-09-03T06:44:07.506 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009aa619893317286b-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.506 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:07.507 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:07.508 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.508 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.509 DEBUG:boto:Method: GET 2022-09-03T06:44:07.509 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.510 DEBUG:boto:Data: 2022-09-03T06:44:07.510 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.511 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.511 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.512 DEBUG:boto:Params: {} 2022-09-03T06:44:07.512 DEBUG:boto:Token: None 2022-09-03T06:44:07.513 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.514 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2MMRB05vzO+Y9zuB1CtzJXF7j00= 2022-09-03T06:44:07.514 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2MMRB05vzO+Y9zuB1CtzJXF7j00=', 'Content-Length': '0'} 2022-09-03T06:44:07.517 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b0c77fb03004bb64-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.517 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:07.518 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:07.518 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.519 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.519 DEBUG:boto:Method: HEAD 2022-09-03T06:44:07.520 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.521 DEBUG:boto:Data: 2022-09-03T06:44:07.521 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.522 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.522 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.523 DEBUG:boto:Params: {} 2022-09-03T06:44:07.523 DEBUG:boto:Token: None 2022-09-03T06:44:07.524 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.524 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m4iPyD3fc8XEtd2g3iFdZWh97bY= 2022-09-03T06:44:07.525 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m4iPyD3fc8XEtd2g3iFdZWh97bY=', 'Content-Length': '0'} 2022-09-03T06:44:07.527 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a4386787d54af94d-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.528 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:07.528 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:07.529 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.529 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.530 DEBUG:boto:Method: HEAD 2022-09-03T06:44:07.531 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.531 DEBUG:boto:Data: 2022-09-03T06:44:07.532 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.532 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.533 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.533 DEBUG:boto:Params: {} 2022-09-03T06:44:07.534 DEBUG:boto:Token: None 2022-09-03T06:44:07.535 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.536 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m4iPyD3fc8XEtd2g3iFdZWh97bY= 2022-09-03T06:44:07.537 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m4iPyD3fc8XEtd2g3iFdZWh97bY=', 'Content-Length': '0'} 2022-09-03T06:44:07.539 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d4294623f2d61c84-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.540 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:44:07.541 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:07.541 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:07.542 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.543 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.543 DEBUG:boto:Method: GET 2022-09-03T06:44:07.544 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.545 DEBUG:boto:Data: 2022-09-03T06:44:07.545 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.546 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.547 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.547 DEBUG:boto:Params: {} 2022-09-03T06:44:07.548 DEBUG:boto:Token: None 2022-09-03T06:44:07.549 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.550 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2MMRB05vzO+Y9zuB1CtzJXF7j00= 2022-09-03T06:44:07.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2MMRB05vzO+Y9zuB1CtzJXF7j00=', 'Content-Length': '0'} 2022-09-03T06:44:07.554 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000071cdd2c6b85c3dda-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.556 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:07.557 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:07.558 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.559 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.560 DEBUG:boto:Method: GET 2022-09-03T06:44:07.561 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.562 DEBUG:boto:Data: 2022-09-03T06:44:07.563 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.563 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.564 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.565 DEBUG:boto:Params: {} 2022-09-03T06:44:07.566 DEBUG:boto:Token: None 2022-09-03T06:44:07.567 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:07.569 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2MMRB05vzO+Y9zuB1CtzJXF7j00= 2022-09-03T06:44:07.570 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2MMRB05vzO+Y9zuB1CtzJXF7j00=', 'Content-Length': '0'} 2022-09-03T06:44:07.572 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000020d014a076b47b33-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.573 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:07.574 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:07.574 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:07.575 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:07.576 DEBUG:boto:Method: GET 2022-09-03T06:44:07.576 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:44:07.577 DEBUG:boto:Data: 2022-09-03T06:44:07.577 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.578 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.578 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.579 DEBUG:boto:Params: {} 2022-09-03T06:44:07.579 DEBUG:boto:Token: None 2022-09-03T06:44:07.580 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/ 2022-09-03T06:44:07.581 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc= 2022-09-03T06:44:07.581 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc=', 'Content-Length': '0'} 2022-09-03T06:44:07.584 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008cd7b5aa98531ecb-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.626 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:07.627 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:07.628 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:07.629 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:07.630 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:07.630 DEBUG:boto:Method: GET 2022-09-03T06:44:07.631 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:44:07.631 DEBUG:boto:Data: 2022-09-03T06:44:07.632 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.633 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.633 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.634 DEBUG:boto:Params: {} 2022-09-03T06:44:07.635 DEBUG:boto:Token: None 2022-09-03T06:44:07.635 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/ 2022-09-03T06:44:07.636 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc= 2022-09-03T06:44:07.637 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc=', 'Content-Length': '0'} 2022-09-03T06:44:07.641 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c10b8ab3b55348e3-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.642 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:07.644 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:07.644 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:07.645 DEBUG:boto:Method: HEAD 2022-09-03T06:44:07.645 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:44:07.646 DEBUG:boto:Data: 2022-09-03T06:44:07.647 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.647 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.648 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.648 DEBUG:boto:Params: {} 2022-09-03T06:44:07.649 DEBUG:boto:Token: None 2022-09-03T06:44:07.650 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj17 2022-09-03T06:44:07.650 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+8zt5rNSf+w7cyx2XhmzhfNT3Xs= 2022-09-03T06:44:07.651 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+8zt5rNSf+w7cyx2XhmzhfNT3Xs=', 'Content-Length': '0'} 2022-09-03T06:44:07.653 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c73ee529c775d75b-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.654 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:07.655 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:07.656 DEBUG:boto:Method: HEAD 2022-09-03T06:44:07.656 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:44:07.657 DEBUG:boto:Data: 2022-09-03T06:44:07.657 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.658 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.659 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.659 DEBUG:boto:Params: {} 2022-09-03T06:44:07.660 DEBUG:boto:Token: None 2022-09-03T06:44:07.661 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj17 2022-09-03T06:44:07.661 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+8zt5rNSf+w7cyx2XhmzhfNT3Xs= 2022-09-03T06:44:07.662 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+8zt5rNSf+w7cyx2XhmzhfNT3Xs=', 'Content-Length': '0'} 2022-09-03T06:44:07.664 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f63336ad5ff0cf6-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.665 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:44:07.666 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:07.667 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:07.667 DEBUG:boto:Method: GET 2022-09-03T06:44:07.668 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:44:07.669 DEBUG:boto:Data: 2022-09-03T06:44:07.669 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.670 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.671 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.671 DEBUG:boto:Params: {} 2022-09-03T06:44:07.672 DEBUG:boto:Token: None 2022-09-03T06:44:07.673 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj17 2022-09-03T06:44:07.673 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ptvltVQTMiiMTx+Moj73qRwmKPM= 2022-09-03T06:44:07.674 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ptvltVQTMiiMTx+Moj73qRwmKPM=', 'Content-Length': '0'} 2022-09-03T06:44:07.676 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000290e46072bf6806e-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.677 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:07.678 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:07.679 DEBUG:boto:Method: GET 2022-09-03T06:44:07.680 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:44:07.680 DEBUG:boto:Data: 2022-09-03T06:44:07.681 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.681 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.682 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.683 DEBUG:boto:Params: {} 2022-09-03T06:44:07.683 DEBUG:boto:Token: None 2022-09-03T06:44:07.684 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj17 2022-09-03T06:44:07.685 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ptvltVQTMiiMTx+Moj73qRwmKPM= 2022-09-03T06:44:07.686 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ptvltVQTMiiMTx+Moj73qRwmKPM=', 'Content-Length': '0'} 2022-09-03T06:44:07.688 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b5c8fecf66809e22-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.689 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:07.690 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:07.691 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:07.691 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:07.692 DEBUG:boto:Method: GET 2022-09-03T06:44:07.693 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:44:07.693 DEBUG:boto:Data: 2022-09-03T06:44:07.694 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.695 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.696 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.697 DEBUG:boto:Params: {} 2022-09-03T06:44:07.698 DEBUG:boto:Token: None 2022-09-03T06:44:07.698 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/ 2022-09-03T06:44:07.699 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc= 2022-09-03T06:44:07.700 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc=', 'Content-Length': '0'} 2022-09-03T06:44:07.703 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000712ee31c35c6b66e-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.704 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:07.705 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:07.706 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:07.707 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:07.707 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:07.708 DEBUG:boto:Method: GET 2022-09-03T06:44:07.709 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:44:07.710 DEBUG:boto:Data: 2022-09-03T06:44:07.711 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.712 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.712 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.713 DEBUG:boto:Params: {} 2022-09-03T06:44:07.714 DEBUG:boto:Token: None 2022-09-03T06:44:07.714 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/ 2022-09-03T06:44:07.715 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc= 2022-09-03T06:44:07.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc=', 'Content-Length': '0'} 2022-09-03T06:44:07.721 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007d91bb315eec39da-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.722 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:07.723 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:44:07.723 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:07.724 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:07.724 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.725 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.726 DEBUG:boto:Method: GET 2022-09-03T06:44:07.726 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.727 DEBUG:boto:Data: 2022-09-03T06:44:07.727 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.728 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.729 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.729 DEBUG:boto:Params: {} 2022-09-03T06:44:07.730 DEBUG:boto:Token: None 2022-09-03T06:44:07.731 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.731 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:czpb+U3tjNGZeu9O0jPZNbgL+og= 2022-09-03T06:44:07.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:czpb+U3tjNGZeu9O0jPZNbgL+og=', 'Content-Length': '0'} 2022-09-03T06:44:07.735 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000051a94e7b3e1b3c8f-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.736 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:07.737 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:07.737 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.738 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.739 DEBUG:boto:Method: GET 2022-09-03T06:44:07.739 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.740 DEBUG:boto:Data: 2022-09-03T06:44:07.740 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.741 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.742 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.743 DEBUG:boto:Params: {} 2022-09-03T06:44:07.743 DEBUG:boto:Token: None 2022-09-03T06:44:07.744 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.745 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:czpb+U3tjNGZeu9O0jPZNbgL+og= 2022-09-03T06:44:07.746 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:czpb+U3tjNGZeu9O0jPZNbgL+og=', 'Content-Length': '0'} 2022-09-03T06:44:07.749 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004ee35b892590e81d-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.791 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:07.793 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:07.794 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.794 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.795 DEBUG:boto:Method: HEAD 2022-09-03T06:44:07.796 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.797 DEBUG:boto:Data: 2022-09-03T06:44:07.798 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.799 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.801 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.802 DEBUG:boto:Params: {} 2022-09-03T06:44:07.802 DEBUG:boto:Token: None 2022-09-03T06:44:07.803 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.804 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:akVhz+AY3RkAsxiU9gDONlQvDqc= 2022-09-03T06:44:07.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:akVhz+AY3RkAsxiU9gDONlQvDqc=', 'Content-Length': '0'} 2022-09-03T06:44:07.807 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000829387bf3d211dd-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.808 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:07.808 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:07.809 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.810 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.810 DEBUG:boto:Method: HEAD 2022-09-03T06:44:07.811 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.812 DEBUG:boto:Data: 2022-09-03T06:44:07.812 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.813 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.813 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.814 DEBUG:boto:Params: {} 2022-09-03T06:44:07.815 DEBUG:boto:Token: None 2022-09-03T06:44:07.816 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.816 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:akVhz+AY3RkAsxiU9gDONlQvDqc= 2022-09-03T06:44:07.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:akVhz+AY3RkAsxiU9gDONlQvDqc=', 'Content-Length': '0'} 2022-09-03T06:44:07.819 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008bc48aef79eb12a3-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.820 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:44:07.821 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:07.822 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:07.822 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.823 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.823 DEBUG:boto:Method: GET 2022-09-03T06:44:07.824 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.825 DEBUG:boto:Data: 2022-09-03T06:44:07.825 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.826 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.827 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.827 DEBUG:boto:Params: {} 2022-09-03T06:44:07.828 DEBUG:boto:Token: None 2022-09-03T06:44:07.829 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.829 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:czpb+U3tjNGZeu9O0jPZNbgL+og= 2022-09-03T06:44:07.830 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:czpb+U3tjNGZeu9O0jPZNbgL+og=', 'Content-Length': '0'} 2022-09-03T06:44:07.832 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b9deb9e83386828d-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.833 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:07.834 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:07.834 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.835 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.836 DEBUG:boto:Method: GET 2022-09-03T06:44:07.836 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.837 DEBUG:boto:Data: 2022-09-03T06:44:07.837 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.838 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.839 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.839 DEBUG:boto:Params: {} 2022-09-03T06:44:07.840 DEBUG:boto:Token: None 2022-09-03T06:44:07.840 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:07.841 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:czpb+U3tjNGZeu9O0jPZNbgL+og= 2022-09-03T06:44:07.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:czpb+U3tjNGZeu9O0jPZNbgL+og=', 'Content-Length': '0'} 2022-09-03T06:44:07.844 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b28c4c57e7a1ddc5-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.845 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:07.846 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:07.847 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:07.847 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:07.848 DEBUG:boto:Method: GET 2022-09-03T06:44:07.848 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:44:07.849 DEBUG:boto:Data: 2022-09-03T06:44:07.850 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.850 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.851 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.851 DEBUG:boto:Params: {} 2022-09-03T06:44:07.852 DEBUG:boto:Token: None 2022-09-03T06:44:07.853 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/ 2022-09-03T06:44:07.853 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc= 2022-09-03T06:44:07.854 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc=', 'Content-Length': '0'} 2022-09-03T06:44:07.858 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000062671f00b6ade4d2-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.859 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:07.860 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:07.861 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:07.862 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:07.863 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:07.863 DEBUG:boto:Method: GET 2022-09-03T06:44:07.864 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:44:07.864 DEBUG:boto:Data: 2022-09-03T06:44:07.865 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.866 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.866 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.867 DEBUG:boto:Params: {} 2022-09-03T06:44:07.868 DEBUG:boto:Token: None 2022-09-03T06:44:07.868 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/ 2022-09-03T06:44:07.869 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc= 2022-09-03T06:44:07.870 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc=', 'Content-Length': '0'} 2022-09-03T06:44:07.874 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c34bef20cf5419b4-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.874 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:07.875 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:07.876 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:07.876 DEBUG:boto:Method: HEAD 2022-09-03T06:44:07.877 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:44:07.877 DEBUG:boto:Data: 2022-09-03T06:44:07.878 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.878 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.879 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.879 DEBUG:boto:Params: {} 2022-09-03T06:44:07.880 DEBUG:boto:Token: None 2022-09-03T06:44:07.881 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj19 2022-09-03T06:44:07.881 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KY1xcn5lgWyr8qfPwQJPQLmhA0Q= 2022-09-03T06:44:07.882 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KY1xcn5lgWyr8qfPwQJPQLmhA0Q=', 'Content-Length': '0'} 2022-09-03T06:44:07.884 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000058a0c85fe5046d41-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.885 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:07.885 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:07.886 DEBUG:boto:Method: HEAD 2022-09-03T06:44:07.887 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:44:07.887 DEBUG:boto:Data: 2022-09-03T06:44:07.888 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.888 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.889 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.889 DEBUG:boto:Params: {} 2022-09-03T06:44:07.890 DEBUG:boto:Token: None 2022-09-03T06:44:07.891 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj19 2022-09-03T06:44:07.892 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KY1xcn5lgWyr8qfPwQJPQLmhA0Q= 2022-09-03T06:44:07.893 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KY1xcn5lgWyr8qfPwQJPQLmhA0Q=', 'Content-Length': '0'} 2022-09-03T06:44:07.895 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008c1aef86c85d6ba1-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.896 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:44:07.897 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:07.897 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:07.898 DEBUG:boto:Method: GET 2022-09-03T06:44:07.898 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:44:07.899 DEBUG:boto:Data: 2022-09-03T06:44:07.900 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.900 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.901 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.901 DEBUG:boto:Params: {} 2022-09-03T06:44:07.902 DEBUG:boto:Token: None 2022-09-03T06:44:07.903 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj19 2022-09-03T06:44:07.903 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3j9X4D1BbCacujfpKQtM0wCRFwY= 2022-09-03T06:44:07.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3j9X4D1BbCacujfpKQtM0wCRFwY=', 'Content-Length': '0'} 2022-09-03T06:44:07.906 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cb5e5d4213031c2b-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.950 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:07.951 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:07.952 DEBUG:boto:Method: GET 2022-09-03T06:44:07.954 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:44:07.955 DEBUG:boto:Data: 2022-09-03T06:44:07.957 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.958 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.959 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.960 DEBUG:boto:Params: {} 2022-09-03T06:44:07.960 DEBUG:boto:Token: None 2022-09-03T06:44:07.962 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/obj19 2022-09-03T06:44:07.963 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3j9X4D1BbCacujfpKQtM0wCRFwY= 2022-09-03T06:44:07.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3j9X4D1BbCacujfpKQtM0wCRFwY=', 'Content-Length': '0'} 2022-09-03T06:44:07.966 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b834234fed26ef42-006312f7b7-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.968 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:07.970 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:07.970 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:07.971 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:07.971 DEBUG:boto:Method: GET 2022-09-03T06:44:07.972 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:44:07.973 DEBUG:boto:Data: 2022-09-03T06:44:07.973 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.974 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:07.975 DEBUG:boto:Port: 8001 2022-09-03T06:44:07.976 DEBUG:boto:Params: {} 2022-09-03T06:44:07.977 DEBUG:boto:Token: None 2022-09-03T06:44:07.978 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/ 2022-09-03T06:44:07.979 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc= 2022-09-03T06:44:07.980 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc=', 'Content-Length': '0'} 2022-09-03T06:44:07.983 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ce315a565b81c47e-006312f7b7-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:07.984 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:07.986 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:07.987 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:07.988 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:07.989 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:07.990 DEBUG:boto:Method: GET 2022-09-03T06:44:07.991 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:44:07.992 DEBUG:boto:Data: 2022-09-03T06:44:07.993 DEBUG:boto:Headers: {} 2022-09-03T06:44:07.994 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:07.995 DEBUG:boto:Port: 8000 2022-09-03T06:44:07.996 DEBUG:boto:Params: {} 2022-09-03T06:44:07.997 DEBUG:boto:Token: None 2022-09-03T06:44:07.998 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:07 GMT /dsgvcy-75/ 2022-09-03T06:44:07.999 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc= 2022-09-03T06:44:08.000 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6N4g4swRcu03ODyA9Vu957iMDuc=', 'Content-Length': '0'} 2022-09-03T06:44:08.003 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c266c00e45fe05f0-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.004 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:08.005 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:44:08.006 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:08.007 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:08.007 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.008 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.008 DEBUG:boto:Method: GET 2022-09-03T06:44:08.009 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.009 DEBUG:boto:Data: 2022-09-03T06:44:08.010 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.011 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.011 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.012 DEBUG:boto:Params: {} 2022-09-03T06:44:08.013 DEBUG:boto:Token: None 2022-09-03T06:44:08.013 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.014 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D6IRpvCGxaPNp4cg9qjXwfdrpPg= 2022-09-03T06:44:08.015 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D6IRpvCGxaPNp4cg9qjXwfdrpPg=', 'Content-Length': '0'} 2022-09-03T06:44:08.017 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aed957f342d785c7-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.018 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:08.019 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:08.020 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.020 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.021 DEBUG:boto:Method: GET 2022-09-03T06:44:08.022 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.022 DEBUG:boto:Data: 2022-09-03T06:44:08.023 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.023 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.024 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.025 DEBUG:boto:Params: {} 2022-09-03T06:44:08.025 DEBUG:boto:Token: None 2022-09-03T06:44:08.026 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.027 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D6IRpvCGxaPNp4cg9qjXwfdrpPg= 2022-09-03T06:44:08.027 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D6IRpvCGxaPNp4cg9qjXwfdrpPg=', 'Content-Length': '0'} 2022-09-03T06:44:08.030 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e110e413339b5f7c-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.070 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:08.071 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:08.071 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.072 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.073 DEBUG:boto:Method: HEAD 2022-09-03T06:44:08.073 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.074 DEBUG:boto:Data: 2022-09-03T06:44:08.075 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.075 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.076 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.077 DEBUG:boto:Params: {} 2022-09-03T06:44:08.078 DEBUG:boto:Token: None 2022-09-03T06:44:08.079 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.080 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WuYP/fnKOueeOMS/ccYv+O59SwQ= 2022-09-03T06:44:08.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WuYP/fnKOueeOMS/ccYv+O59SwQ=', 'Content-Length': '0'} 2022-09-03T06:44:08.084 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d86174ac6ae36dcd-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.086 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:08.087 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:08.088 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.089 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.090 DEBUG:boto:Method: HEAD 2022-09-03T06:44:08.090 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.091 DEBUG:boto:Data: 2022-09-03T06:44:08.092 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.093 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.094 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.095 DEBUG:boto:Params: {} 2022-09-03T06:44:08.096 DEBUG:boto:Token: None 2022-09-03T06:44:08.097 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.098 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WuYP/fnKOueeOMS/ccYv+O59SwQ= 2022-09-03T06:44:08.099 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WuYP/fnKOueeOMS/ccYv+O59SwQ=', 'Content-Length': '0'} 2022-09-03T06:44:08.102 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005322705aa7742587-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.103 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:44:08.104 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:08.105 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:08.106 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.107 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.108 DEBUG:boto:Method: GET 2022-09-03T06:44:08.108 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.109 DEBUG:boto:Data: 2022-09-03T06:44:08.110 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.111 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.112 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.113 DEBUG:boto:Params: {} 2022-09-03T06:44:08.114 DEBUG:boto:Token: None 2022-09-03T06:44:08.115 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.116 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D6IRpvCGxaPNp4cg9qjXwfdrpPg= 2022-09-03T06:44:08.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D6IRpvCGxaPNp4cg9qjXwfdrpPg=', 'Content-Length': '0'} 2022-09-03T06:44:08.119 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f9023aa71eac7c79-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.121 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:08.122 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:08.124 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.125 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.126 DEBUG:boto:Method: GET 2022-09-03T06:44:08.127 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.128 DEBUG:boto:Data: 2022-09-03T06:44:08.129 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.130 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.131 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.132 DEBUG:boto:Params: {} 2022-09-03T06:44:08.133 DEBUG:boto:Token: None 2022-09-03T06:44:08.134 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:08.135 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D6IRpvCGxaPNp4cg9qjXwfdrpPg= 2022-09-03T06:44:08.136 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D6IRpvCGxaPNp4cg9qjXwfdrpPg=', 'Content-Length': '0'} 2022-09-03T06:44:08.139 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000e3b91f718e9e6a5-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.141 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:08.142 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:08.143 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:08.144 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:08.145 DEBUG:boto:Method: GET 2022-09-03T06:44:08.146 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:44:08.147 DEBUG:boto:Data: 2022-09-03T06:44:08.148 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.149 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.149 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.150 DEBUG:boto:Params: {} 2022-09-03T06:44:08.151 DEBUG:boto:Token: None 2022-09-03T06:44:08.151 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/ 2022-09-03T06:44:08.152 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I= 2022-09-03T06:44:08.153 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I=', 'Content-Length': '0'} 2022-09-03T06:44:08.159 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003d63fc0668deefa5-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.202 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:08.204 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:08.204 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:08.205 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:08.206 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:08.207 DEBUG:boto:Method: GET 2022-09-03T06:44:08.208 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:44:08.209 DEBUG:boto:Data: 2022-09-03T06:44:08.209 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.210 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.211 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.212 DEBUG:boto:Params: {} 2022-09-03T06:44:08.213 DEBUG:boto:Token: None 2022-09-03T06:44:08.214 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/ 2022-09-03T06:44:08.216 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I= 2022-09-03T06:44:08.217 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I=', 'Content-Length': '0'} 2022-09-03T06:44:08.221 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000083953f7868a11332-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.221 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:08.222 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:08.223 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:08.224 DEBUG:boto:Method: HEAD 2022-09-03T06:44:08.224 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:44:08.225 DEBUG:boto:Data: 2022-09-03T06:44:08.225 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.226 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.226 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.227 DEBUG:boto:Params: {} 2022-09-03T06:44:08.228 DEBUG:boto:Token: None 2022-09-03T06:44:08.228 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj22 2022-09-03T06:44:08.229 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bmkgqi+oD1jQ2fgAZAA8ZA7kbSM= 2022-09-03T06:44:08.230 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bmkgqi+oD1jQ2fgAZAA8ZA7kbSM=', 'Content-Length': '0'} 2022-09-03T06:44:08.232 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e32f2f52d0ff738-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.233 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:08.234 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:08.235 DEBUG:boto:Method: HEAD 2022-09-03T06:44:08.235 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:44:08.236 DEBUG:boto:Data: 2022-09-03T06:44:08.236 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.237 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.238 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.238 DEBUG:boto:Params: {} 2022-09-03T06:44:08.239 DEBUG:boto:Token: None 2022-09-03T06:44:08.240 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj22 2022-09-03T06:44:08.241 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bmkgqi+oD1jQ2fgAZAA8ZA7kbSM= 2022-09-03T06:44:08.241 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bmkgqi+oD1jQ2fgAZAA8ZA7kbSM=', 'Content-Length': '0'} 2022-09-03T06:44:08.244 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065423d31aa77b4ea-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.245 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:44:08.246 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:08.246 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:08.247 DEBUG:boto:Method: GET 2022-09-03T06:44:08.248 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:44:08.248 DEBUG:boto:Data: 2022-09-03T06:44:08.249 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.249 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.250 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.251 DEBUG:boto:Params: {} 2022-09-03T06:44:08.252 DEBUG:boto:Token: None 2022-09-03T06:44:08.253 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj22 2022-09-03T06:44:08.253 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hi33PkdLdWvOIFcWlMLaVBZIQyI= 2022-09-03T06:44:08.254 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hi33PkdLdWvOIFcWlMLaVBZIQyI=', 'Content-Length': '0'} 2022-09-03T06:44:08.256 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007f5b3474fc36d3b2-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.257 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:08.258 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:08.258 DEBUG:boto:Method: GET 2022-09-03T06:44:08.259 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:44:08.259 DEBUG:boto:Data: 2022-09-03T06:44:08.260 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.261 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.261 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.262 DEBUG:boto:Params: {} 2022-09-03T06:44:08.263 DEBUG:boto:Token: None 2022-09-03T06:44:08.263 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj22 2022-09-03T06:44:08.264 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hi33PkdLdWvOIFcWlMLaVBZIQyI= 2022-09-03T06:44:08.265 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hi33PkdLdWvOIFcWlMLaVBZIQyI=', 'Content-Length': '0'} 2022-09-03T06:44:08.267 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db38e3c77b045f85-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.268 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:08.269 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:08.269 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:08.270 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:08.271 DEBUG:boto:Method: GET 2022-09-03T06:44:08.271 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:44:08.272 DEBUG:boto:Data: 2022-09-03T06:44:08.272 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.273 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.273 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.274 DEBUG:boto:Params: {} 2022-09-03T06:44:08.274 DEBUG:boto:Token: None 2022-09-03T06:44:08.275 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/ 2022-09-03T06:44:08.276 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I= 2022-09-03T06:44:08.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I=', 'Content-Length': '0'} 2022-09-03T06:44:08.280 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001305b41cead13c8b-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.280 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:08.281 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:08.282 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:08.282 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:08.283 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:08.283 DEBUG:boto:Method: GET 2022-09-03T06:44:08.284 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:44:08.284 DEBUG:boto:Data: 2022-09-03T06:44:08.285 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.286 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.286 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.287 DEBUG:boto:Params: {} 2022-09-03T06:44:08.287 DEBUG:boto:Token: None 2022-09-03T06:44:08.288 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/ 2022-09-03T06:44:08.289 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I= 2022-09-03T06:44:08.289 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I=', 'Content-Length': '0'} 2022-09-03T06:44:08.293 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000068aea6c5a1d4f935-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.294 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:08.294 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:44:08.295 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:08.296 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:08.296 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.297 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.297 DEBUG:boto:Method: GET 2022-09-03T06:44:08.298 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.298 DEBUG:boto:Data: 2022-09-03T06:44:08.299 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.300 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.300 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.301 DEBUG:boto:Params: {} 2022-09-03T06:44:08.302 DEBUG:boto:Token: None 2022-09-03T06:44:08.302 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.303 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+SR8TldER/LEvJYoJaeJbDzI9Xs= 2022-09-03T06:44:08.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+SR8TldER/LEvJYoJaeJbDzI9Xs=', 'Content-Length': '0'} 2022-09-03T06:44:08.306 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eb080281099c7f81-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.307 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:08.308 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:08.309 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.309 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.310 DEBUG:boto:Method: GET 2022-09-03T06:44:08.311 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.311 DEBUG:boto:Data: 2022-09-03T06:44:08.312 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.313 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.314 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.314 DEBUG:boto:Params: {} 2022-09-03T06:44:08.315 DEBUG:boto:Token: None 2022-09-03T06:44:08.316 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.317 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+SR8TldER/LEvJYoJaeJbDzI9Xs= 2022-09-03T06:44:08.319 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+SR8TldER/LEvJYoJaeJbDzI9Xs=', 'Content-Length': '0'} 2022-09-03T06:44:08.322 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000932c6f19c7b4ce05-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.324 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:08.325 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:08.326 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.327 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.329 DEBUG:boto:Method: HEAD 2022-09-03T06:44:08.331 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.332 DEBUG:boto:Data: 2022-09-03T06:44:08.333 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.335 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.336 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.337 DEBUG:boto:Params: {} 2022-09-03T06:44:08.338 DEBUG:boto:Token: None 2022-09-03T06:44:08.339 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.340 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sGbXErrY3PiLOmo7UMgxxoEyh+c= 2022-09-03T06:44:08.341 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sGbXErrY3PiLOmo7UMgxxoEyh+c=', 'Content-Length': '0'} 2022-09-03T06:44:08.346 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000519db8f24d3ce7e9-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.348 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:08.349 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:08.350 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.351 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.352 DEBUG:boto:Method: HEAD 2022-09-03T06:44:08.353 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.354 DEBUG:boto:Data: 2022-09-03T06:44:08.354 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.355 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.356 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.357 DEBUG:boto:Params: {} 2022-09-03T06:44:08.359 DEBUG:boto:Token: None 2022-09-03T06:44:08.360 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.361 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sGbXErrY3PiLOmo7UMgxxoEyh+c= 2022-09-03T06:44:08.362 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sGbXErrY3PiLOmo7UMgxxoEyh+c=', 'Content-Length': '0'} 2022-09-03T06:44:08.364 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d3f629c0bc893eb2-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.366 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:44:08.367 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:08.368 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:08.369 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.370 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.371 DEBUG:boto:Method: GET 2022-09-03T06:44:08.373 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.374 DEBUG:boto:Data: 2022-09-03T06:44:08.375 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.376 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.377 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.378 DEBUG:boto:Params: {} 2022-09-03T06:44:08.379 DEBUG:boto:Token: None 2022-09-03T06:44:08.380 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.381 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+SR8TldER/LEvJYoJaeJbDzI9Xs= 2022-09-03T06:44:08.383 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+SR8TldER/LEvJYoJaeJbDzI9Xs=', 'Content-Length': '0'} 2022-09-03T06:44:08.385 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000652498eef6d83213-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.427 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:08.428 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:08.429 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.429 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.430 DEBUG:boto:Method: GET 2022-09-03T06:44:08.431 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.432 DEBUG:boto:Data: 2022-09-03T06:44:08.433 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.434 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.435 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.435 DEBUG:boto:Params: {} 2022-09-03T06:44:08.436 DEBUG:boto:Token: None 2022-09-03T06:44:08.437 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:08.438 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+SR8TldER/LEvJYoJaeJbDzI9Xs= 2022-09-03T06:44:08.438 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+SR8TldER/LEvJYoJaeJbDzI9Xs=', 'Content-Length': '0'} 2022-09-03T06:44:08.441 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004af92ee1af06452e-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.442 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:08.444 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:08.444 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:08.445 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:08.446 DEBUG:boto:Method: GET 2022-09-03T06:44:08.447 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:44:08.448 DEBUG:boto:Data: 2022-09-03T06:44:08.449 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.449 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.450 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.450 DEBUG:boto:Params: {} 2022-09-03T06:44:08.451 DEBUG:boto:Token: None 2022-09-03T06:44:08.451 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/ 2022-09-03T06:44:08.452 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I= 2022-09-03T06:44:08.453 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I=', 'Content-Length': '0'} 2022-09-03T06:44:08.456 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000368a41cf83a4efb8-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.457 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:08.458 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:08.459 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:08.460 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:08.461 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:08.462 DEBUG:boto:Method: GET 2022-09-03T06:44:08.464 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:44:08.465 DEBUG:boto:Data: 2022-09-03T06:44:08.465 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.466 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.467 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.468 DEBUG:boto:Params: {} 2022-09-03T06:44:08.469 DEBUG:boto:Token: None 2022-09-03T06:44:08.471 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/ 2022-09-03T06:44:08.472 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I= 2022-09-03T06:44:08.474 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I=', 'Content-Length': '0'} 2022-09-03T06:44:08.477 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000080a2838dd84dbee3-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.478 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:08.480 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:08.480 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:08.481 DEBUG:boto:Method: HEAD 2022-09-03T06:44:08.482 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:44:08.483 DEBUG:boto:Data: 2022-09-03T06:44:08.484 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.484 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.485 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.485 DEBUG:boto:Params: {} 2022-09-03T06:44:08.486 DEBUG:boto:Token: None 2022-09-03T06:44:08.486 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj23 2022-09-03T06:44:08.487 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+M3do8k2oudfzEXvtEw74mN4GQE= 2022-09-03T06:44:08.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+M3do8k2oudfzEXvtEw74mN4GQE=', 'Content-Length': '0'} 2022-09-03T06:44:08.490 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000ee0bfa84f0af11e-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.491 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:08.492 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:08.492 DEBUG:boto:Method: HEAD 2022-09-03T06:44:08.493 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:44:08.494 DEBUG:boto:Data: 2022-09-03T06:44:08.494 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.495 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.496 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.496 DEBUG:boto:Params: {} 2022-09-03T06:44:08.497 DEBUG:boto:Token: None 2022-09-03T06:44:08.498 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj23 2022-09-03T06:44:08.499 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+M3do8k2oudfzEXvtEw74mN4GQE= 2022-09-03T06:44:08.500 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+M3do8k2oudfzEXvtEw74mN4GQE=', 'Content-Length': '0'} 2022-09-03T06:44:08.502 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d29e519fe58be0c7-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.503 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:44:08.503 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:08.504 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:08.505 DEBUG:boto:Method: GET 2022-09-03T06:44:08.505 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:44:08.506 DEBUG:boto:Data: 2022-09-03T06:44:08.506 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.507 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.507 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.508 DEBUG:boto:Params: {} 2022-09-03T06:44:08.509 DEBUG:boto:Token: None 2022-09-03T06:44:08.509 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj23 2022-09-03T06:44:08.510 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WpR7jeb1vhIOlnMQWM7AsK/sCKU= 2022-09-03T06:44:08.510 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WpR7jeb1vhIOlnMQWM7AsK/sCKU=', 'Content-Length': '0'} 2022-09-03T06:44:08.513 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000043ef0d55f6ee2923-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.514 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:08.514 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:08.515 DEBUG:boto:Method: GET 2022-09-03T06:44:08.515 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:44:08.516 DEBUG:boto:Data: 2022-09-03T06:44:08.516 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.517 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.517 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.518 DEBUG:boto:Params: {} 2022-09-03T06:44:08.518 DEBUG:boto:Token: None 2022-09-03T06:44:08.519 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj23 2022-09-03T06:44:08.520 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WpR7jeb1vhIOlnMQWM7AsK/sCKU= 2022-09-03T06:44:08.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WpR7jeb1vhIOlnMQWM7AsK/sCKU=', 'Content-Length': '0'} 2022-09-03T06:44:08.523 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b4683ab1fc2cd3b1-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.524 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:08.524 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:08.525 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:08.525 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:08.526 DEBUG:boto:Method: GET 2022-09-03T06:44:08.526 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:44:08.527 DEBUG:boto:Data: 2022-09-03T06:44:08.527 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.528 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.528 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.529 DEBUG:boto:Params: {} 2022-09-03T06:44:08.529 DEBUG:boto:Token: None 2022-09-03T06:44:08.530 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/ 2022-09-03T06:44:08.531 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I= 2022-09-03T06:44:08.531 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I=', 'Content-Length': '0'} 2022-09-03T06:44:08.534 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003dfd9d56b2d3c308-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.535 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:08.546 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:08.546 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:08.547 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:08.547 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:08.548 DEBUG:boto:Method: GET 2022-09-03T06:44:08.548 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:44:08.549 DEBUG:boto:Data: 2022-09-03T06:44:08.549 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.550 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.550 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.551 DEBUG:boto:Params: {} 2022-09-03T06:44:08.551 DEBUG:boto:Token: None 2022-09-03T06:44:08.552 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/ 2022-09-03T06:44:08.553 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I= 2022-09-03T06:44:08.553 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I=', 'Content-Length': '0'} 2022-09-03T06:44:08.557 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007de67e18d729079c-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.598 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:08.599 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:44:08.600 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:08.600 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:08.601 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.601 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.602 DEBUG:boto:Method: GET 2022-09-03T06:44:08.603 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.603 DEBUG:boto:Data: 2022-09-03T06:44:08.604 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.605 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.605 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.606 DEBUG:boto:Params: {} 2022-09-03T06:44:08.607 DEBUG:boto:Token: None 2022-09-03T06:44:08.607 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.608 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:igv8qQZFDRl5JoZmatQihBA67kY= 2022-09-03T06:44:08.609 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:igv8qQZFDRl5JoZmatQihBA67kY=', 'Content-Length': '0'} 2022-09-03T06:44:08.611 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b4a57fe2684aa575-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.612 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:08.613 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:08.613 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.614 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.615 DEBUG:boto:Method: GET 2022-09-03T06:44:08.616 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.616 DEBUG:boto:Data: 2022-09-03T06:44:08.617 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.618 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.618 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.619 DEBUG:boto:Params: {} 2022-09-03T06:44:08.619 DEBUG:boto:Token: None 2022-09-03T06:44:08.620 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.621 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:igv8qQZFDRl5JoZmatQihBA67kY= 2022-09-03T06:44:08.621 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:igv8qQZFDRl5JoZmatQihBA67kY=', 'Content-Length': '0'} 2022-09-03T06:44:08.624 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f358a534342d6a6c-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.624 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:08.625 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:08.625 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.626 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.626 DEBUG:boto:Method: HEAD 2022-09-03T06:44:08.627 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.627 DEBUG:boto:Data: 2022-09-03T06:44:08.628 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.628 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.629 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.629 DEBUG:boto:Params: {} 2022-09-03T06:44:08.630 DEBUG:boto:Token: None 2022-09-03T06:44:08.631 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.631 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y5ra6al26nrGr/beXVqDe1bk8CQ= 2022-09-03T06:44:08.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y5ra6al26nrGr/beXVqDe1bk8CQ=', 'Content-Length': '0'} 2022-09-03T06:44:08.634 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e3041208346b094e-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.635 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:08.635 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:08.636 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.636 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.637 DEBUG:boto:Method: HEAD 2022-09-03T06:44:08.637 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.638 DEBUG:boto:Data: 2022-09-03T06:44:08.638 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.639 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.639 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.640 DEBUG:boto:Params: {} 2022-09-03T06:44:08.640 DEBUG:boto:Token: None 2022-09-03T06:44:08.641 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.641 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y5ra6al26nrGr/beXVqDe1bk8CQ= 2022-09-03T06:44:08.642 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y5ra6al26nrGr/beXVqDe1bk8CQ=', 'Content-Length': '0'} 2022-09-03T06:44:08.644 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f6e7efc20ebf6379-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.645 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:44:08.645 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:08.646 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:08.646 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.647 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.648 DEBUG:boto:Method: GET 2022-09-03T06:44:08.648 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.649 DEBUG:boto:Data: 2022-09-03T06:44:08.649 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.650 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.650 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.651 DEBUG:boto:Params: {} 2022-09-03T06:44:08.651 DEBUG:boto:Token: None 2022-09-03T06:44:08.652 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.653 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:igv8qQZFDRl5JoZmatQihBA67kY= 2022-09-03T06:44:08.653 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:igv8qQZFDRl5JoZmatQihBA67kY=', 'Content-Length': '0'} 2022-09-03T06:44:08.656 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d67f57152a6b92e-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.698 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:08.699 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:08.700 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.700 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.701 DEBUG:boto:Method: GET 2022-09-03T06:44:08.701 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.702 DEBUG:boto:Data: 2022-09-03T06:44:08.702 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.703 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.703 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.704 DEBUG:boto:Params: {} 2022-09-03T06:44:08.705 DEBUG:boto:Token: None 2022-09-03T06:44:08.707 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:08.708 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:igv8qQZFDRl5JoZmatQihBA67kY= 2022-09-03T06:44:08.709 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:igv8qQZFDRl5JoZmatQihBA67kY=', 'Content-Length': '0'} 2022-09-03T06:44:08.712 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cbb53a279f28675b-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.713 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:08.714 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:08.715 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:08.716 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:08.717 DEBUG:boto:Method: GET 2022-09-03T06:44:08.718 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:44:08.719 DEBUG:boto:Data: 2022-09-03T06:44:08.720 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.721 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.722 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.723 DEBUG:boto:Params: {} 2022-09-03T06:44:08.724 DEBUG:boto:Token: None 2022-09-03T06:44:08.725 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/ 2022-09-03T06:44:08.726 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I= 2022-09-03T06:44:08.727 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I=', 'Content-Length': '0'} 2022-09-03T06:44:08.731 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e10ab6b28899e011-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.739 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:08.740 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:08.741 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:08.742 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:08.743 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:08.744 DEBUG:boto:Method: GET 2022-09-03T06:44:08.745 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:44:08.746 DEBUG:boto:Data: 2022-09-03T06:44:08.747 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.748 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.749 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.750 DEBUG:boto:Params: {} 2022-09-03T06:44:08.751 DEBUG:boto:Token: None 2022-09-03T06:44:08.751 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/ 2022-09-03T06:44:08.752 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I= 2022-09-03T06:44:08.753 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I=', 'Content-Length': '0'} 2022-09-03T06:44:08.757 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cce3078ce680b868-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.758 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:08.759 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:08.759 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:08.760 DEBUG:boto:Method: HEAD 2022-09-03T06:44:08.761 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:44:08.761 DEBUG:boto:Data: 2022-09-03T06:44:08.761 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.762 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.763 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.763 DEBUG:boto:Params: {} 2022-09-03T06:44:08.764 DEBUG:boto:Token: None 2022-09-03T06:44:08.764 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj33 2022-09-03T06:44:08.765 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HFhBIMwGyPv0sbSqY4ni0pS+vmo= 2022-09-03T06:44:08.766 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HFhBIMwGyPv0sbSqY4ni0pS+vmo=', 'Content-Length': '0'} 2022-09-03T06:44:08.768 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006965fb34a7082f07-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.769 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:08.770 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:08.771 DEBUG:boto:Method: HEAD 2022-09-03T06:44:08.772 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:44:08.773 DEBUG:boto:Data: 2022-09-03T06:44:08.774 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.775 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.776 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.777 DEBUG:boto:Params: {} 2022-09-03T06:44:08.778 DEBUG:boto:Token: None 2022-09-03T06:44:08.779 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj33 2022-09-03T06:44:08.780 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HFhBIMwGyPv0sbSqY4ni0pS+vmo= 2022-09-03T06:44:08.781 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HFhBIMwGyPv0sbSqY4ni0pS+vmo=', 'Content-Length': '0'} 2022-09-03T06:44:08.784 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c8df999e7c55a0c-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.784 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:44:08.785 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:08.785 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:08.786 DEBUG:boto:Method: GET 2022-09-03T06:44:08.787 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:44:08.787 DEBUG:boto:Data: 2022-09-03T06:44:08.788 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.788 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.789 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.790 DEBUG:boto:Params: {} 2022-09-03T06:44:08.790 DEBUG:boto:Token: None 2022-09-03T06:44:08.791 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj33 2022-09-03T06:44:08.792 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SESkV7QdGRVxf0BEdAYjWWcSEgc= 2022-09-03T06:44:08.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SESkV7QdGRVxf0BEdAYjWWcSEgc=', 'Content-Length': '0'} 2022-09-03T06:44:08.794 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e7e05dd5d6e32031-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.795 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:08.796 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:08.796 DEBUG:boto:Method: GET 2022-09-03T06:44:08.797 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:44:08.797 DEBUG:boto:Data: 2022-09-03T06:44:08.798 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.799 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.799 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.800 DEBUG:boto:Params: {} 2022-09-03T06:44:08.800 DEBUG:boto:Token: None 2022-09-03T06:44:08.801 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj33 2022-09-03T06:44:08.802 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SESkV7QdGRVxf0BEdAYjWWcSEgc= 2022-09-03T06:44:08.803 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SESkV7QdGRVxf0BEdAYjWWcSEgc=', 'Content-Length': '0'} 2022-09-03T06:44:08.805 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000307f4aa8a800833f-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.806 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:08.807 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:08.808 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:08.809 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:08.810 DEBUG:boto:Method: GET 2022-09-03T06:44:08.811 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:44:08.811 DEBUG:boto:Data: 2022-09-03T06:44:08.812 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.813 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.813 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.814 DEBUG:boto:Params: {} 2022-09-03T06:44:08.815 DEBUG:boto:Token: None 2022-09-03T06:44:08.815 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/ 2022-09-03T06:44:08.816 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I= 2022-09-03T06:44:08.818 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I=', 'Content-Length': '0'} 2022-09-03T06:44:08.821 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008a7f7782c302acd3-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.862 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:08.863 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:08.864 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:08.864 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:08.865 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:08.865 DEBUG:boto:Method: GET 2022-09-03T06:44:08.866 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:44:08.867 DEBUG:boto:Data: 2022-09-03T06:44:08.868 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.868 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.869 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.869 DEBUG:boto:Params: {} 2022-09-03T06:44:08.870 DEBUG:boto:Token: None 2022-09-03T06:44:08.871 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/ 2022-09-03T06:44:08.872 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I= 2022-09-03T06:44:08.873 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I=', 'Content-Length': '0'} 2022-09-03T06:44:08.877 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c56e95d6a8a76e81-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.878 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:08.879 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:44:08.880 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:08.881 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:08.881 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.882 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.882 DEBUG:boto:Method: GET 2022-09-03T06:44:08.883 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.884 DEBUG:boto:Data: 2022-09-03T06:44:08.884 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.885 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.885 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.886 DEBUG:boto:Params: {} 2022-09-03T06:44:08.886 DEBUG:boto:Token: None 2022-09-03T06:44:08.887 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.888 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TVrTSs4FqRrbsrzdWUVMLCPvSAk= 2022-09-03T06:44:08.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TVrTSs4FqRrbsrzdWUVMLCPvSAk=', 'Content-Length': '0'} 2022-09-03T06:44:08.891 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d41df64425b165b8-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.891 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:08.892 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:08.893 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.893 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.894 DEBUG:boto:Method: GET 2022-09-03T06:44:08.894 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.895 DEBUG:boto:Data: 2022-09-03T06:44:08.895 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.896 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.896 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.897 DEBUG:boto:Params: {} 2022-09-03T06:44:08.898 DEBUG:boto:Token: None 2022-09-03T06:44:08.898 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.899 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TVrTSs4FqRrbsrzdWUVMLCPvSAk= 2022-09-03T06:44:08.899 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TVrTSs4FqRrbsrzdWUVMLCPvSAk=', 'Content-Length': '0'} 2022-09-03T06:44:08.902 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c0b63995f4277bdf-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.902 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:08.903 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:08.903 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.904 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.904 DEBUG:boto:Method: HEAD 2022-09-03T06:44:08.905 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.905 DEBUG:boto:Data: 2022-09-03T06:44:08.906 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.906 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.907 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.907 DEBUG:boto:Params: {} 2022-09-03T06:44:08.908 DEBUG:boto:Token: None 2022-09-03T06:44:08.909 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.909 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n3H8Ww+kmeMWwUsLWw3HaJ6UF8g= 2022-09-03T06:44:08.910 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n3H8Ww+kmeMWwUsLWw3HaJ6UF8g=', 'Content-Length': '0'} 2022-09-03T06:44:08.912 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000099f1d6ef40e7192e-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.913 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:08.914 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:08.914 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.915 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.915 DEBUG:boto:Method: HEAD 2022-09-03T06:44:08.916 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.916 DEBUG:boto:Data: 2022-09-03T06:44:08.917 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.917 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.918 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.919 DEBUG:boto:Params: {} 2022-09-03T06:44:08.920 DEBUG:boto:Token: None 2022-09-03T06:44:08.920 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.921 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n3H8Ww+kmeMWwUsLWw3HaJ6UF8g= 2022-09-03T06:44:08.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n3H8Ww+kmeMWwUsLWw3HaJ6UF8g=', 'Content-Length': '0'} 2022-09-03T06:44:08.926 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021a9cb951e42bde5-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.927 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:44:08.927 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:08.928 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:08.928 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.929 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.930 DEBUG:boto:Method: GET 2022-09-03T06:44:08.931 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.931 DEBUG:boto:Data: 2022-09-03T06:44:08.932 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.933 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.933 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.935 DEBUG:boto:Params: {} 2022-09-03T06:44:08.936 DEBUG:boto:Token: None 2022-09-03T06:44:08.936 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.937 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TVrTSs4FqRrbsrzdWUVMLCPvSAk= 2022-09-03T06:44:08.938 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TVrTSs4FqRrbsrzdWUVMLCPvSAk=', 'Content-Length': '0'} 2022-09-03T06:44:08.941 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000077fd7747634fd47d-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.942 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:08.943 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:08.944 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.945 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.946 DEBUG:boto:Method: GET 2022-09-03T06:44:08.948 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.949 DEBUG:boto:Data: 2022-09-03T06:44:08.950 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.951 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.952 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.953 DEBUG:boto:Params: {} 2022-09-03T06:44:08.954 DEBUG:boto:Token: None 2022-09-03T06:44:08.955 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:08.958 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TVrTSs4FqRrbsrzdWUVMLCPvSAk= 2022-09-03T06:44:08.959 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TVrTSs4FqRrbsrzdWUVMLCPvSAk=', 'Content-Length': '0'} 2022-09-03T06:44:08.961 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e08d94e85c237b87-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.963 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:08.964 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:08.965 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:08.966 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:08.966 DEBUG:boto:Method: GET 2022-09-03T06:44:08.967 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:44:08.968 DEBUG:boto:Data: 2022-09-03T06:44:08.968 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.969 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.969 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.970 DEBUG:boto:Params: {} 2022-09-03T06:44:08.971 DEBUG:boto:Token: None 2022-09-03T06:44:08.971 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/ 2022-09-03T06:44:08.972 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I= 2022-09-03T06:44:08.973 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I=', 'Content-Length': '0'} 2022-09-03T06:44:08.976 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009d99b10c34090050-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.977 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:08.978 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:08.978 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:08.979 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:08.979 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:08.980 DEBUG:boto:Method: GET 2022-09-03T06:44:08.981 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:44:08.981 DEBUG:boto:Data: 2022-09-03T06:44:08.982 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.982 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:08.983 DEBUG:boto:Port: 8000 2022-09-03T06:44:08.983 DEBUG:boto:Params: {} 2022-09-03T06:44:08.984 DEBUG:boto:Token: None 2022-09-03T06:44:08.985 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/ 2022-09-03T06:44:08.985 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I= 2022-09-03T06:44:08.986 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9ej+pbAqvqTqhyXIS/k+ujAoO6I=', 'Content-Length': '0'} 2022-09-03T06:44:08.990 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000576ca90fdfed6ee5-006312f7b8-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:08.990 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:08.991 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:08.992 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:08.993 DEBUG:boto:Method: HEAD 2022-09-03T06:44:08.993 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:44:08.994 DEBUG:boto:Data: 2022-09-03T06:44:08.994 DEBUG:boto:Headers: {} 2022-09-03T06:44:08.995 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:08.995 DEBUG:boto:Port: 8001 2022-09-03T06:44:08.996 DEBUG:boto:Params: {} 2022-09-03T06:44:08.997 DEBUG:boto:Token: None 2022-09-03T06:44:08.997 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:08 GMT /dsgvcy-75/obj34 2022-09-03T06:44:08.998 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JvMkJxA7Fxnb9Ts3Z8IQfnsPwPM= 2022-09-03T06:44:08.998 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JvMkJxA7Fxnb9Ts3Z8IQfnsPwPM=', 'Content-Length': '0'} 2022-09-03T06:44:09.001 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ec722967c433b315-006312f7b8-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.001 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:09.002 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:09.003 DEBUG:boto:Method: HEAD 2022-09-03T06:44:09.003 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:44:09.004 DEBUG:boto:Data: 2022-09-03T06:44:09.004 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.005 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:09.005 DEBUG:boto:Port: 8000 2022-09-03T06:44:09.006 DEBUG:boto:Params: {} 2022-09-03T06:44:09.006 DEBUG:boto:Token: None 2022-09-03T06:44:09.007 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj34 2022-09-03T06:44:09.007 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:T8PHC6vYGTZTti47fydchJjfEd8= 2022-09-03T06:44:09.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:T8PHC6vYGTZTti47fydchJjfEd8=', 'Content-Length': '0'} 2022-09-03T06:44:09.011 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c3d67c075d7ac863-006312f7b9-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.012 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:44:09.013 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:09.014 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:09.015 DEBUG:boto:Method: GET 2022-09-03T06:44:09.016 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:44:09.016 DEBUG:boto:Data: 2022-09-03T06:44:09.017 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.018 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:09.019 DEBUG:boto:Port: 8001 2022-09-03T06:44:09.020 DEBUG:boto:Params: {} 2022-09-03T06:44:09.021 DEBUG:boto:Token: None 2022-09-03T06:44:09.023 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj34 2022-09-03T06:44:09.025 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Co1UANLLV7kYVdctrC6DEaVS58o= 2022-09-03T06:44:09.026 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Co1UANLLV7kYVdctrC6DEaVS58o=', 'Content-Length': '0'} 2022-09-03T06:44:09.028 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004159a38ac29a6f29-006312f7b9-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.070 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:09.071 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:09.072 DEBUG:boto:Method: GET 2022-09-03T06:44:09.073 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:44:09.074 DEBUG:boto:Data: 2022-09-03T06:44:09.075 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.075 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:09.076 DEBUG:boto:Port: 8000 2022-09-03T06:44:09.076 DEBUG:boto:Params: {} 2022-09-03T06:44:09.078 DEBUG:boto:Token: None 2022-09-03T06:44:09.079 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj34 2022-09-03T06:44:09.080 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Co1UANLLV7kYVdctrC6DEaVS58o= 2022-09-03T06:44:09.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Co1UANLLV7kYVdctrC6DEaVS58o=', 'Content-Length': '0'} 2022-09-03T06:44:09.084 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000120365a4f1ff24d2-006312f7b9-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.086 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:09.086 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:09.087 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:09.089 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:09.089 DEBUG:boto:Method: GET 2022-09-03T06:44:09.090 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:44:09.091 DEBUG:boto:Data: 2022-09-03T06:44:09.092 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.093 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:09.094 DEBUG:boto:Port: 8001 2022-09-03T06:44:09.095 DEBUG:boto:Params: {} 2022-09-03T06:44:09.095 DEBUG:boto:Token: None 2022-09-03T06:44:09.096 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/ 2022-09-03T06:44:09.096 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM= 2022-09-03T06:44:09.097 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM=', 'Content-Length': '0'} 2022-09-03T06:44:09.101 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005eb925241f87d1cf-006312f7b9-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.101 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:09.103 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:09.103 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:09.104 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:09.104 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:09.105 DEBUG:boto:Method: GET 2022-09-03T06:44:09.106 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:44:09.107 DEBUG:boto:Data: 2022-09-03T06:44:09.107 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.108 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:09.108 DEBUG:boto:Port: 8000 2022-09-03T06:44:09.109 DEBUG:boto:Params: {} 2022-09-03T06:44:09.109 DEBUG:boto:Token: None 2022-09-03T06:44:09.110 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/ 2022-09-03T06:44:09.111 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM= 2022-09-03T06:44:09.111 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM=', 'Content-Length': '0'} 2022-09-03T06:44:09.116 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f12b386d546221aa-006312f7b9-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.117 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:09.119 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:44:09.120 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:09.121 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:09.122 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.123 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.124 DEBUG:boto:Method: GET 2022-09-03T06:44:09.125 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.126 DEBUG:boto:Data: 2022-09-03T06:44:09.127 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.128 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:09.129 DEBUG:boto:Port: 8001 2022-09-03T06:44:09.129 DEBUG:boto:Params: {} 2022-09-03T06:44:09.130 DEBUG:boto:Token: None 2022-09-03T06:44:09.131 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.131 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Xykxcss029D0rLhnMAOHN37UAjg= 2022-09-03T06:44:09.132 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Xykxcss029D0rLhnMAOHN37UAjg=', 'Content-Length': '0'} 2022-09-03T06:44:09.135 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c451a9ab897127ef-006312f7b9-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.136 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:09.137 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:09.137 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.138 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.139 DEBUG:boto:Method: GET 2022-09-03T06:44:09.140 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.141 DEBUG:boto:Data: 2022-09-03T06:44:09.142 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.143 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:09.143 DEBUG:boto:Port: 8000 2022-09-03T06:44:09.144 DEBUG:boto:Params: {} 2022-09-03T06:44:09.146 DEBUG:boto:Token: None 2022-09-03T06:44:09.147 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.148 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Xykxcss029D0rLhnMAOHN37UAjg= 2022-09-03T06:44:09.149 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Xykxcss029D0rLhnMAOHN37UAjg=', 'Content-Length': '0'} 2022-09-03T06:44:09.152 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000014f54799128a545f-006312f7b9-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.153 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:09.153 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:09.154 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.155 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.156 DEBUG:boto:Method: HEAD 2022-09-03T06:44:09.156 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.157 DEBUG:boto:Data: 2022-09-03T06:44:09.158 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.158 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:09.159 DEBUG:boto:Port: 8001 2022-09-03T06:44:09.159 DEBUG:boto:Params: {} 2022-09-03T06:44:09.160 DEBUG:boto:Token: None 2022-09-03T06:44:09.161 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.162 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:T/ByWCkbeHASuuser6xeV9qY0Uk= 2022-09-03T06:44:09.163 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:T/ByWCkbeHASuuser6xeV9qY0Uk=', 'Content-Length': '0'} 2022-09-03T06:44:09.165 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000052d8558aa64ff51b-006312f7b9-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.166 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:09.167 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:09.168 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.169 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.170 DEBUG:boto:Method: HEAD 2022-09-03T06:44:09.171 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.172 DEBUG:boto:Data: 2022-09-03T06:44:09.173 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.174 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:09.174 DEBUG:boto:Port: 8000 2022-09-03T06:44:09.175 DEBUG:boto:Params: {} 2022-09-03T06:44:09.177 DEBUG:boto:Token: None 2022-09-03T06:44:09.178 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.179 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:T/ByWCkbeHASuuser6xeV9qY0Uk= 2022-09-03T06:44:09.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:T/ByWCkbeHASuuser6xeV9qY0Uk=', 'Content-Length': '0'} 2022-09-03T06:44:09.182 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000209a9e108b981c7-006312f7b9-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.183 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:44:09.184 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:09.184 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:09.185 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.185 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.186 DEBUG:boto:Method: GET 2022-09-03T06:44:09.187 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.187 DEBUG:boto:Data: 2022-09-03T06:44:09.188 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.188 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:09.189 DEBUG:boto:Port: 8001 2022-09-03T06:44:09.189 DEBUG:boto:Params: {} 2022-09-03T06:44:09.190 DEBUG:boto:Token: None 2022-09-03T06:44:09.191 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.191 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Xykxcss029D0rLhnMAOHN37UAjg= 2022-09-03T06:44:09.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Xykxcss029D0rLhnMAOHN37UAjg=', 'Content-Length': '0'} 2022-09-03T06:44:09.194 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fdb019ba7215b5c3-006312f7b9-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.195 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:09.196 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:09.197 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.197 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.198 DEBUG:boto:Method: GET 2022-09-03T06:44:09.199 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.199 DEBUG:boto:Data: 2022-09-03T06:44:09.200 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.200 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:09.201 DEBUG:boto:Port: 8000 2022-09-03T06:44:09.201 DEBUG:boto:Params: {} 2022-09-03T06:44:09.202 DEBUG:boto:Token: None 2022-09-03T06:44:09.203 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:09.203 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Xykxcss029D0rLhnMAOHN37UAjg= 2022-09-03T06:44:09.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Xykxcss029D0rLhnMAOHN37UAjg=', 'Content-Length': '0'} 2022-09-03T06:44:09.206 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b3551db664ec292-006312f7b9-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.250 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:09.251 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:09.252 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:09.253 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:09.254 DEBUG:boto:Method: GET 2022-09-03T06:44:09.255 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:44:09.257 DEBUG:boto:Data: 2022-09-03T06:44:09.258 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.259 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:09.261 DEBUG:boto:Port: 8001 2022-09-03T06:44:09.262 DEBUG:boto:Params: {} 2022-09-03T06:44:09.263 DEBUG:boto:Token: None 2022-09-03T06:44:09.264 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/ 2022-09-03T06:44:09.265 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM= 2022-09-03T06:44:09.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM=', 'Content-Length': '0'} 2022-09-03T06:44:09.271 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004deb2358590fd79a-006312f7b9-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.273 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:09.273 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:09.274 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:09.275 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:09.276 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:09.276 DEBUG:boto:Method: GET 2022-09-03T06:44:09.277 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:44:09.277 DEBUG:boto:Data: 2022-09-03T06:44:09.278 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.278 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:09.279 DEBUG:boto:Port: 8000 2022-09-03T06:44:09.280 DEBUG:boto:Params: {} 2022-09-03T06:44:09.280 DEBUG:boto:Token: None 2022-09-03T06:44:09.281 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/ 2022-09-03T06:44:09.282 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM= 2022-09-03T06:44:09.283 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM=', 'Content-Length': '0'} 2022-09-03T06:44:09.287 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f7b3d9fb228c856a-006312f7b9-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.288 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:09.289 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:09.290 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:09.290 DEBUG:boto:Method: HEAD 2022-09-03T06:44:09.291 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:44:09.292 DEBUG:boto:Data: 2022-09-03T06:44:09.292 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.293 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:09.293 DEBUG:boto:Port: 8001 2022-09-03T06:44:09.294 DEBUG:boto:Params: {} 2022-09-03T06:44:09.294 DEBUG:boto:Token: None 2022-09-03T06:44:09.295 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj38 2022-09-03T06:44:09.296 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oECc10+e9w49pDgtzuovW87Z2cM= 2022-09-03T06:44:09.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oECc10+e9w49pDgtzuovW87Z2cM=', 'Content-Length': '0'} 2022-09-03T06:44:09.299 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000102010c091d7ab55-006312f7b9-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.300 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:09.300 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:09.301 DEBUG:boto:Method: HEAD 2022-09-03T06:44:09.301 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:44:09.302 DEBUG:boto:Data: 2022-09-03T06:44:09.302 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.303 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:09.304 DEBUG:boto:Port: 8000 2022-09-03T06:44:09.304 DEBUG:boto:Params: {} 2022-09-03T06:44:09.305 DEBUG:boto:Token: None 2022-09-03T06:44:09.305 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj38 2022-09-03T06:44:09.306 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oECc10+e9w49pDgtzuovW87Z2cM= 2022-09-03T06:44:09.306 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oECc10+e9w49pDgtzuovW87Z2cM=', 'Content-Length': '0'} 2022-09-03T06:44:09.309 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e62b9850578eb55a-006312f7b9-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.309 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:44:09.310 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:09.310 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:09.311 DEBUG:boto:Method: GET 2022-09-03T06:44:09.312 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:44:09.312 DEBUG:boto:Data: 2022-09-03T06:44:09.313 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.313 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:09.314 DEBUG:boto:Port: 8001 2022-09-03T06:44:09.314 DEBUG:boto:Params: {} 2022-09-03T06:44:09.315 DEBUG:boto:Token: None 2022-09-03T06:44:09.316 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj38 2022-09-03T06:44:09.316 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gcjBMhtnpY1cVmL3xt880w1amNY= 2022-09-03T06:44:09.317 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gcjBMhtnpY1cVmL3xt880w1amNY=', 'Content-Length': '0'} 2022-09-03T06:44:09.319 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000052ab3f0106bf14ae-006312f7b9-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.362 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:09.364 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:09.365 DEBUG:boto:Method: GET 2022-09-03T06:44:09.366 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:44:09.367 DEBUG:boto:Data: 2022-09-03T06:44:09.368 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.368 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:09.369 DEBUG:boto:Port: 8000 2022-09-03T06:44:09.370 DEBUG:boto:Params: {} 2022-09-03T06:44:09.371 DEBUG:boto:Token: None 2022-09-03T06:44:09.372 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj38 2022-09-03T06:44:09.373 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gcjBMhtnpY1cVmL3xt880w1amNY= 2022-09-03T06:44:09.374 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gcjBMhtnpY1cVmL3xt880w1amNY=', 'Content-Length': '0'} 2022-09-03T06:44:09.377 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005432ca60543f3e5b-006312f7b9-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.378 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:09.379 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:09.380 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:09.381 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:09.382 DEBUG:boto:Method: GET 2022-09-03T06:44:09.383 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:44:09.384 DEBUG:boto:Data: 2022-09-03T06:44:09.385 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.386 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:09.387 DEBUG:boto:Port: 8001 2022-09-03T06:44:09.388 DEBUG:boto:Params: {} 2022-09-03T06:44:09.389 DEBUG:boto:Token: None 2022-09-03T06:44:09.390 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/ 2022-09-03T06:44:09.391 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM= 2022-09-03T06:44:09.393 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM=', 'Content-Length': '0'} 2022-09-03T06:44:09.396 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004abcced5b86c5d78-006312f7b9-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.397 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:09.398 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:09.399 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:09.400 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:09.400 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:09.401 DEBUG:boto:Method: GET 2022-09-03T06:44:09.401 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:44:09.402 DEBUG:boto:Data: 2022-09-03T06:44:09.402 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.403 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:09.404 DEBUG:boto:Port: 8000 2022-09-03T06:44:09.404 DEBUG:boto:Params: {} 2022-09-03T06:44:09.405 DEBUG:boto:Token: None 2022-09-03T06:44:09.406 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/ 2022-09-03T06:44:09.407 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM= 2022-09-03T06:44:09.407 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM=', 'Content-Length': '0'} 2022-09-03T06:44:09.412 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ea8964f01a61c015-006312f7b9-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.413 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:09.415 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:44:09.416 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:09.417 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:09.417 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.418 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.419 DEBUG:boto:Method: GET 2022-09-03T06:44:09.419 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.420 DEBUG:boto:Data: 2022-09-03T06:44:09.421 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.421 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:09.422 DEBUG:boto:Port: 8001 2022-09-03T06:44:09.423 DEBUG:boto:Params: {} 2022-09-03T06:44:09.424 DEBUG:boto:Token: None 2022-09-03T06:44:09.424 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.425 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HCwVHouHO02GwEhpMBo/ev6/7vE= 2022-09-03T06:44:09.426 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HCwVHouHO02GwEhpMBo/ev6/7vE=', 'Content-Length': '0'} 2022-09-03T06:44:09.428 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c790661113fb911a-006312f7b9-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.429 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:09.429 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:09.430 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.431 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.432 DEBUG:boto:Method: GET 2022-09-03T06:44:09.432 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.433 DEBUG:boto:Data: 2022-09-03T06:44:09.433 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.434 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:09.435 DEBUG:boto:Port: 8000 2022-09-03T06:44:09.436 DEBUG:boto:Params: {} 2022-09-03T06:44:09.436 DEBUG:boto:Token: None 2022-09-03T06:44:09.437 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.438 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HCwVHouHO02GwEhpMBo/ev6/7vE= 2022-09-03T06:44:09.439 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HCwVHouHO02GwEhpMBo/ev6/7vE=', 'Content-Length': '0'} 2022-09-03T06:44:09.441 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f9be6adaa1bcc69d-006312f7b9-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.482 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:09.483 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:09.484 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.484 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.485 DEBUG:boto:Method: HEAD 2022-09-03T06:44:09.486 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.487 DEBUG:boto:Data: 2022-09-03T06:44:09.488 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.489 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:09.490 DEBUG:boto:Port: 8001 2022-09-03T06:44:09.491 DEBUG:boto:Params: {} 2022-09-03T06:44:09.492 DEBUG:boto:Token: None 2022-09-03T06:44:09.493 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.494 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J/mOAzjcbfbt5ZUHH2w8KeXRzrE= 2022-09-03T06:44:09.495 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J/mOAzjcbfbt5ZUHH2w8KeXRzrE=', 'Content-Length': '0'} 2022-09-03T06:44:09.497 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f969e60049528f9d-006312f7b9-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.498 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:09.498 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:09.499 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.499 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.500 DEBUG:boto:Method: HEAD 2022-09-03T06:44:09.500 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.501 DEBUG:boto:Data: 2022-09-03T06:44:09.501 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.502 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:09.503 DEBUG:boto:Port: 8000 2022-09-03T06:44:09.503 DEBUG:boto:Params: {} 2022-09-03T06:44:09.504 DEBUG:boto:Token: None 2022-09-03T06:44:09.504 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.505 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J/mOAzjcbfbt5ZUHH2w8KeXRzrE= 2022-09-03T06:44:09.505 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J/mOAzjcbfbt5ZUHH2w8KeXRzrE=', 'Content-Length': '0'} 2022-09-03T06:44:09.508 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f502e3074d06441f-006312f7b9-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.508 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:44:09.509 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:09.510 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:09.510 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.511 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.511 DEBUG:boto:Method: GET 2022-09-03T06:44:09.512 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.512 DEBUG:boto:Data: 2022-09-03T06:44:09.513 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.513 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:09.514 DEBUG:boto:Port: 8001 2022-09-03T06:44:09.514 DEBUG:boto:Params: {} 2022-09-03T06:44:09.515 DEBUG:boto:Token: None 2022-09-03T06:44:09.516 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.516 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HCwVHouHO02GwEhpMBo/ev6/7vE= 2022-09-03T06:44:09.517 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HCwVHouHO02GwEhpMBo/ev6/7vE=', 'Content-Length': '0'} 2022-09-03T06:44:09.519 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000da8ed562c47ec84a-006312f7b9-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.520 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:09.521 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:09.522 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.522 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.523 DEBUG:boto:Method: GET 2022-09-03T06:44:09.524 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.525 DEBUG:boto:Data: 2022-09-03T06:44:09.526 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.527 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:09.527 DEBUG:boto:Port: 8000 2022-09-03T06:44:09.528 DEBUG:boto:Params: {} 2022-09-03T06:44:09.529 DEBUG:boto:Token: None 2022-09-03T06:44:09.530 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:09.530 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HCwVHouHO02GwEhpMBo/ev6/7vE= 2022-09-03T06:44:09.531 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HCwVHouHO02GwEhpMBo/ev6/7vE=', 'Content-Length': '0'} 2022-09-03T06:44:09.533 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a327f5039bf0520e-006312f7b9-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.534 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:09.535 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:09.535 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:09.536 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:09.537 DEBUG:boto:Method: GET 2022-09-03T06:44:09.538 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:44:09.539 DEBUG:boto:Data: 2022-09-03T06:44:09.540 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.541 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:09.542 DEBUG:boto:Port: 8001 2022-09-03T06:44:09.543 DEBUG:boto:Params: {} 2022-09-03T06:44:09.544 DEBUG:boto:Token: None 2022-09-03T06:44:09.545 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/ 2022-09-03T06:44:09.546 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM= 2022-09-03T06:44:09.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM=', 'Content-Length': '0'} 2022-09-03T06:44:09.549 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002173242496c7c5a8-006312f7b9-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.550 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:09.551 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:09.552 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:09.552 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:09.553 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:09.553 DEBUG:boto:Method: GET 2022-09-03T06:44:09.554 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:44:09.555 DEBUG:boto:Data: 2022-09-03T06:44:09.555 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.556 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:09.556 DEBUG:boto:Port: 8000 2022-09-03T06:44:09.557 DEBUG:boto:Params: {} 2022-09-03T06:44:09.557 DEBUG:boto:Token: None 2022-09-03T06:44:09.558 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/ 2022-09-03T06:44:09.558 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM= 2022-09-03T06:44:09.559 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM=', 'Content-Length': '0'} 2022-09-03T06:44:09.563 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001f5c2fbdbd1468bd-006312f7b9-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.563 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:09.564 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:09.565 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:09.565 DEBUG:boto:Method: HEAD 2022-09-03T06:44:09.566 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:44:09.566 DEBUG:boto:Data: 2022-09-03T06:44:09.567 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.567 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:09.568 DEBUG:boto:Port: 8001 2022-09-03T06:44:09.568 DEBUG:boto:Params: {} 2022-09-03T06:44:09.569 DEBUG:boto:Token: None 2022-09-03T06:44:09.569 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj44 2022-09-03T06:44:09.570 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VEQ+YWU0hzPPXL26hYLk2T0V/iQ= 2022-09-03T06:44:09.571 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VEQ+YWU0hzPPXL26hYLk2T0V/iQ=', 'Content-Length': '0'} 2022-09-03T06:44:09.573 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d16ca8f7a3e2e558-006312f7b9-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.573 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:09.574 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:09.575 DEBUG:boto:Method: HEAD 2022-09-03T06:44:09.575 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:44:09.576 DEBUG:boto:Data: 2022-09-03T06:44:09.576 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.577 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:09.577 DEBUG:boto:Port: 8000 2022-09-03T06:44:09.578 DEBUG:boto:Params: {} 2022-09-03T06:44:09.578 DEBUG:boto:Token: None 2022-09-03T06:44:09.579 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj44 2022-09-03T06:44:09.579 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VEQ+YWU0hzPPXL26hYLk2T0V/iQ= 2022-09-03T06:44:09.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VEQ+YWU0hzPPXL26hYLk2T0V/iQ=', 'Content-Length': '0'} 2022-09-03T06:44:09.582 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bcc09f4ca135420f-006312f7b9-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.583 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:44:09.583 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:09.584 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:09.584 DEBUG:boto:Method: GET 2022-09-03T06:44:09.585 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:44:09.585 DEBUG:boto:Data: 2022-09-03T06:44:09.586 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.586 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:09.587 DEBUG:boto:Port: 8001 2022-09-03T06:44:09.587 DEBUG:boto:Params: {} 2022-09-03T06:44:09.588 DEBUG:boto:Token: None 2022-09-03T06:44:09.589 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj44 2022-09-03T06:44:09.589 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jYZ8F72n1FSkmNK1E8LuKxdmR18= 2022-09-03T06:44:09.590 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jYZ8F72n1FSkmNK1E8LuKxdmR18=', 'Content-Length': '0'} 2022-09-03T06:44:09.592 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee1e0a623402ecb5-006312f7b9-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.593 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:09.593 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:09.594 DEBUG:boto:Method: GET 2022-09-03T06:44:09.595 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:44:09.595 DEBUG:boto:Data: 2022-09-03T06:44:09.596 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.597 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:09.598 DEBUG:boto:Port: 8000 2022-09-03T06:44:09.599 DEBUG:boto:Params: {} 2022-09-03T06:44:09.600 DEBUG:boto:Token: None 2022-09-03T06:44:09.601 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj44 2022-09-03T06:44:09.602 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jYZ8F72n1FSkmNK1E8LuKxdmR18= 2022-09-03T06:44:09.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jYZ8F72n1FSkmNK1E8LuKxdmR18=', 'Content-Length': '0'} 2022-09-03T06:44:09.606 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a86a14e2775caae9-006312f7b9-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.651 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:09.652 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:09.652 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:09.653 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:09.654 DEBUG:boto:Method: GET 2022-09-03T06:44:09.654 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:44:09.655 DEBUG:boto:Data: 2022-09-03T06:44:09.656 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.657 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:09.658 DEBUG:boto:Port: 8001 2022-09-03T06:44:09.659 DEBUG:boto:Params: {} 2022-09-03T06:44:09.660 DEBUG:boto:Token: None 2022-09-03T06:44:09.661 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/ 2022-09-03T06:44:09.661 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM= 2022-09-03T06:44:09.662 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM=', 'Content-Length': '0'} 2022-09-03T06:44:09.666 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e2bbccc824174484-006312f7b9-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.673 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:09.675 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:09.676 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:09.677 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:09.679 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:09.680 DEBUG:boto:Method: GET 2022-09-03T06:44:09.681 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:44:09.682 DEBUG:boto:Data: 2022-09-03T06:44:09.683 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.684 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:09.685 DEBUG:boto:Port: 8000 2022-09-03T06:44:09.686 DEBUG:boto:Params: {} 2022-09-03T06:44:09.688 DEBUG:boto:Token: None 2022-09-03T06:44:09.689 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/ 2022-09-03T06:44:09.690 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM= 2022-09-03T06:44:09.691 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM=', 'Content-Length': '0'} 2022-09-03T06:44:09.704 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e388d4a3ca60dd49-006312f7b9-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.707 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:09.708 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:44:09.709 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:09.710 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:09.711 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.711 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.712 DEBUG:boto:Method: GET 2022-09-03T06:44:09.713 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.713 DEBUG:boto:Data: 2022-09-03T06:44:09.714 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.715 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:09.716 DEBUG:boto:Port: 8001 2022-09-03T06:44:09.717 DEBUG:boto:Params: {} 2022-09-03T06:44:09.719 DEBUG:boto:Token: None 2022-09-03T06:44:09.719 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.721 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fePNo2rrFSkP/X/BmCTzODdhetw= 2022-09-03T06:44:09.721 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fePNo2rrFSkP/X/BmCTzODdhetw=', 'Content-Length': '0'} 2022-09-03T06:44:09.727 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cfc5095fe2c43b5b-006312f7b9-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.728 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:09.729 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:09.730 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.731 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.732 DEBUG:boto:Method: GET 2022-09-03T06:44:09.733 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.734 DEBUG:boto:Data: 2022-09-03T06:44:09.735 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.736 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:09.737 DEBUG:boto:Port: 8000 2022-09-03T06:44:09.738 DEBUG:boto:Params: {} 2022-09-03T06:44:09.739 DEBUG:boto:Token: None 2022-09-03T06:44:09.741 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.742 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fePNo2rrFSkP/X/BmCTzODdhetw= 2022-09-03T06:44:09.744 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fePNo2rrFSkP/X/BmCTzODdhetw=', 'Content-Length': '0'} 2022-09-03T06:44:09.755 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002c32b50eb94f7252-006312f7b9-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.763 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:09.764 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:09.764 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.765 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.766 DEBUG:boto:Method: HEAD 2022-09-03T06:44:09.767 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.768 DEBUG:boto:Data: 2022-09-03T06:44:09.769 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.769 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:09.770 DEBUG:boto:Port: 8001 2022-09-03T06:44:09.771 DEBUG:boto:Params: {} 2022-09-03T06:44:09.772 DEBUG:boto:Token: None 2022-09-03T06:44:09.774 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.775 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VL0dLR7xr6dKaKchLiEwjzwR1ZQ= 2022-09-03T06:44:09.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VL0dLR7xr6dKaKchLiEwjzwR1ZQ=', 'Content-Length': '0'} 2022-09-03T06:44:09.779 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b525bfc3ae0d81a8-006312f7b9-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.780 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:09.781 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:09.781 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.782 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.783 DEBUG:boto:Method: HEAD 2022-09-03T06:44:09.783 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.784 DEBUG:boto:Data: 2022-09-03T06:44:09.784 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.785 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:09.785 DEBUG:boto:Port: 8000 2022-09-03T06:44:09.786 DEBUG:boto:Params: {} 2022-09-03T06:44:09.786 DEBUG:boto:Token: None 2022-09-03T06:44:09.787 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.788 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VL0dLR7xr6dKaKchLiEwjzwR1ZQ= 2022-09-03T06:44:09.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VL0dLR7xr6dKaKchLiEwjzwR1ZQ=', 'Content-Length': '0'} 2022-09-03T06:44:09.791 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000282d9944729d07cf-006312f7b9-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.792 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:44:09.793 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:09.794 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:09.795 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.798 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.799 DEBUG:boto:Method: GET 2022-09-03T06:44:09.800 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.800 DEBUG:boto:Data: 2022-09-03T06:44:09.801 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.802 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:09.803 DEBUG:boto:Port: 8001 2022-09-03T06:44:09.804 DEBUG:boto:Params: {} 2022-09-03T06:44:09.805 DEBUG:boto:Token: None 2022-09-03T06:44:09.807 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.807 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fePNo2rrFSkP/X/BmCTzODdhetw= 2022-09-03T06:44:09.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fePNo2rrFSkP/X/BmCTzODdhetw=', 'Content-Length': '0'} 2022-09-03T06:44:09.812 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000654d7e186feeacbc-006312f7b9-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.855 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:09.857 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:09.858 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.859 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.861 DEBUG:boto:Method: GET 2022-09-03T06:44:09.862 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.864 DEBUG:boto:Data: 2022-09-03T06:44:09.865 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.866 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:09.867 DEBUG:boto:Port: 8000 2022-09-03T06:44:09.868 DEBUG:boto:Params: {} 2022-09-03T06:44:09.869 DEBUG:boto:Token: None 2022-09-03T06:44:09.871 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:09.872 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fePNo2rrFSkP/X/BmCTzODdhetw= 2022-09-03T06:44:09.874 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fePNo2rrFSkP/X/BmCTzODdhetw=', 'Content-Length': '0'} 2022-09-03T06:44:09.880 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001518d89750047995-006312f7b9-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.888 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:09.890 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:09.891 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:09.892 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:09.893 DEBUG:boto:Method: GET 2022-09-03T06:44:09.893 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:44:09.895 DEBUG:boto:Data: 2022-09-03T06:44:09.896 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.897 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:09.898 DEBUG:boto:Port: 8001 2022-09-03T06:44:09.900 DEBUG:boto:Params: {} 2022-09-03T06:44:09.901 DEBUG:boto:Token: None 2022-09-03T06:44:09.902 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/ 2022-09-03T06:44:09.903 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM= 2022-09-03T06:44:09.905 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM=', 'Content-Length': '0'} 2022-09-03T06:44:09.914 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a4f85a5346054b65-006312f7b9-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.916 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:09.917 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:09.918 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:09.918 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:09.919 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:09.919 DEBUG:boto:Method: GET 2022-09-03T06:44:09.920 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:44:09.921 DEBUG:boto:Data: 2022-09-03T06:44:09.922 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.923 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:09.923 DEBUG:boto:Port: 8000 2022-09-03T06:44:09.924 DEBUG:boto:Params: {} 2022-09-03T06:44:09.926 DEBUG:boto:Token: None 2022-09-03T06:44:09.927 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/ 2022-09-03T06:44:09.928 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM= 2022-09-03T06:44:09.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZmKIYLTv3SecfEFYWcdvpmEQyBM=', 'Content-Length': '0'} 2022-09-03T06:44:09.933 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000f25391f6cfdf2ca-006312f7b9-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.934 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:09.935 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:09.935 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:09.936 DEBUG:boto:Method: HEAD 2022-09-03T06:44:09.937 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:44:09.937 DEBUG:boto:Data: 2022-09-03T06:44:09.938 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.939 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:09.939 DEBUG:boto:Port: 8001 2022-09-03T06:44:09.940 DEBUG:boto:Params: {} 2022-09-03T06:44:09.940 DEBUG:boto:Token: None 2022-09-03T06:44:09.941 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj46 2022-09-03T06:44:09.941 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xjWAdkWkUA5/znkZy7+1rJzA1s4= 2022-09-03T06:44:09.942 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xjWAdkWkUA5/znkZy7+1rJzA1s4=', 'Content-Length': '0'} 2022-09-03T06:44:09.944 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000be8214fc225a1717-006312f7b9-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.945 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:09.946 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:09.947 DEBUG:boto:Method: HEAD 2022-09-03T06:44:09.947 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:44:09.948 DEBUG:boto:Data: 2022-09-03T06:44:09.948 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.949 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:09.949 DEBUG:boto:Port: 8000 2022-09-03T06:44:09.950 DEBUG:boto:Params: {} 2022-09-03T06:44:09.951 DEBUG:boto:Token: None 2022-09-03T06:44:09.951 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj46 2022-09-03T06:44:09.952 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xjWAdkWkUA5/znkZy7+1rJzA1s4= 2022-09-03T06:44:09.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xjWAdkWkUA5/znkZy7+1rJzA1s4=', 'Content-Length': '0'} 2022-09-03T06:44:09.956 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2323853b559c69b-006312f7b9-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.957 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:44:09.958 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:09.959 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:09.959 DEBUG:boto:Method: GET 2022-09-03T06:44:09.960 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:44:09.961 DEBUG:boto:Data: 2022-09-03T06:44:09.961 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.962 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:09.963 DEBUG:boto:Port: 8001 2022-09-03T06:44:09.964 DEBUG:boto:Params: {} 2022-09-03T06:44:09.965 DEBUG:boto:Token: None 2022-09-03T06:44:09.966 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj46 2022-09-03T06:44:09.967 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ytocTntK0hnJV1237944SLIohOk= 2022-09-03T06:44:09.969 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ytocTntK0hnJV1237944SLIohOk=', 'Content-Length': '0'} 2022-09-03T06:44:09.978 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fd4b3faa39587cbc-006312f7b9-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:09.984 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:09.985 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:09.986 DEBUG:boto:Method: GET 2022-09-03T06:44:09.987 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:44:09.988 DEBUG:boto:Data: 2022-09-03T06:44:09.989 DEBUG:boto:Headers: {} 2022-09-03T06:44:09.990 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:09.991 DEBUG:boto:Port: 8000 2022-09-03T06:44:09.992 DEBUG:boto:Params: {} 2022-09-03T06:44:09.993 DEBUG:boto:Token: None 2022-09-03T06:44:09.994 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:09 GMT /dsgvcy-75/obj46 2022-09-03T06:44:09.995 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ytocTntK0hnJV1237944SLIohOk= 2022-09-03T06:44:09.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ytocTntK0hnJV1237944SLIohOk=', 'Content-Length': '0'} 2022-09-03T06:44:10.001 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000601df0e31aeebdb-006312f7b9-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.042 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:10.044 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:10.045 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:10.045 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:10.046 DEBUG:boto:Method: GET 2022-09-03T06:44:10.047 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:44:10.048 DEBUG:boto:Data: 2022-09-03T06:44:10.049 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.050 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.051 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.051 DEBUG:boto:Params: {} 2022-09-03T06:44:10.052 DEBUG:boto:Token: None 2022-09-03T06:44:10.053 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/ 2022-09-03T06:44:10.055 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU= 2022-09-03T06:44:10.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU=', 'Content-Length': '0'} 2022-09-03T06:44:10.059 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b0505855146fc99c-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.060 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:10.061 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:10.062 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:10.062 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:10.063 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:10.063 DEBUG:boto:Method: GET 2022-09-03T06:44:10.064 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:44:10.064 DEBUG:boto:Data: 2022-09-03T06:44:10.065 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.066 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.066 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.067 DEBUG:boto:Params: {} 2022-09-03T06:44:10.068 DEBUG:boto:Token: None 2022-09-03T06:44:10.068 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/ 2022-09-03T06:44:10.069 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU= 2022-09-03T06:44:10.069 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU=', 'Content-Length': '0'} 2022-09-03T06:44:10.074 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007829cff921848e1f-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.076 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:10.077 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:44:10.078 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:10.079 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:10.080 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.081 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.081 DEBUG:boto:Method: GET 2022-09-03T06:44:10.082 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.083 DEBUG:boto:Data: 2022-09-03T06:44:10.084 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.084 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.085 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.087 DEBUG:boto:Params: {} 2022-09-03T06:44:10.088 DEBUG:boto:Token: None 2022-09-03T06:44:10.089 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.090 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gZSy1eIH31uDQlspmOgcddkbutU= 2022-09-03T06:44:10.091 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gZSy1eIH31uDQlspmOgcddkbutU=', 'Content-Length': '0'} 2022-09-03T06:44:10.094 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b1926a0740b1267-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.097 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:10.098 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:10.098 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.099 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.099 DEBUG:boto:Method: GET 2022-09-03T06:44:10.100 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.100 DEBUG:boto:Data: 2022-09-03T06:44:10.101 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.101 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.102 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.103 DEBUG:boto:Params: {} 2022-09-03T06:44:10.103 DEBUG:boto:Token: None 2022-09-03T06:44:10.104 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.105 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gZSy1eIH31uDQlspmOgcddkbutU= 2022-09-03T06:44:10.105 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gZSy1eIH31uDQlspmOgcddkbutU=', 'Content-Length': '0'} 2022-09-03T06:44:10.108 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c1a3ce98cc1518b1-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.109 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:10.110 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:10.110 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.111 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.111 DEBUG:boto:Method: HEAD 2022-09-03T06:44:10.112 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.112 DEBUG:boto:Data: 2022-09-03T06:44:10.113 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.113 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.114 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.114 DEBUG:boto:Params: {} 2022-09-03T06:44:10.115 DEBUG:boto:Token: None 2022-09-03T06:44:10.115 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.116 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/WSeP4AqCglSICDD07G1Y7C3D8E= 2022-09-03T06:44:10.117 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/WSeP4AqCglSICDD07G1Y7C3D8E=', 'Content-Length': '0'} 2022-09-03T06:44:10.119 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000004070daf460ac75d-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.120 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:10.120 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:10.121 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.121 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.122 DEBUG:boto:Method: HEAD 2022-09-03T06:44:10.122 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.123 DEBUG:boto:Data: 2022-09-03T06:44:10.123 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.124 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.124 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.125 DEBUG:boto:Params: {} 2022-09-03T06:44:10.125 DEBUG:boto:Token: None 2022-09-03T06:44:10.126 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.127 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/WSeP4AqCglSICDD07G1Y7C3D8E= 2022-09-03T06:44:10.127 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/WSeP4AqCglSICDD07G1Y7C3D8E=', 'Content-Length': '0'} 2022-09-03T06:44:10.134 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000066f1c2c4f88ecdc8-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.135 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:44:10.136 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:10.136 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:10.137 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.137 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.138 DEBUG:boto:Method: GET 2022-09-03T06:44:10.139 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.139 DEBUG:boto:Data: 2022-09-03T06:44:10.140 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.140 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.141 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.141 DEBUG:boto:Params: {} 2022-09-03T06:44:10.142 DEBUG:boto:Token: None 2022-09-03T06:44:10.143 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.143 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gZSy1eIH31uDQlspmOgcddkbutU= 2022-09-03T06:44:10.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gZSy1eIH31uDQlspmOgcddkbutU=', 'Content-Length': '0'} 2022-09-03T06:44:10.146 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006dbf3d1d958f7bd7-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.194 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:10.195 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:10.196 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.196 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.197 DEBUG:boto:Method: GET 2022-09-03T06:44:10.198 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.198 DEBUG:boto:Data: 2022-09-03T06:44:10.199 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.200 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.200 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.202 DEBUG:boto:Params: {} 2022-09-03T06:44:10.202 DEBUG:boto:Token: None 2022-09-03T06:44:10.203 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:10.204 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gZSy1eIH31uDQlspmOgcddkbutU= 2022-09-03T06:44:10.205 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gZSy1eIH31uDQlspmOgcddkbutU=', 'Content-Length': '0'} 2022-09-03T06:44:10.208 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000063a54698e0e82008-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.209 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:10.210 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:10.211 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:10.212 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:10.212 DEBUG:boto:Method: GET 2022-09-03T06:44:10.213 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:44:10.213 DEBUG:boto:Data: 2022-09-03T06:44:10.214 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.214 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.215 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.215 DEBUG:boto:Params: {} 2022-09-03T06:44:10.216 DEBUG:boto:Token: None 2022-09-03T06:44:10.216 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/ 2022-09-03T06:44:10.217 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU= 2022-09-03T06:44:10.218 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU=', 'Content-Length': '0'} 2022-09-03T06:44:10.221 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e09b57c8ead6b1fa-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.222 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:10.222 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:10.223 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:10.224 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:10.224 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:10.225 DEBUG:boto:Method: GET 2022-09-03T06:44:10.225 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:44:10.226 DEBUG:boto:Data: 2022-09-03T06:44:10.226 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.227 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.227 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.228 DEBUG:boto:Params: {} 2022-09-03T06:44:10.228 DEBUG:boto:Token: None 2022-09-03T06:44:10.229 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/ 2022-09-03T06:44:10.230 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU= 2022-09-03T06:44:10.230 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU=', 'Content-Length': '0'} 2022-09-03T06:44:10.234 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a9d0e86cfd55b1e2-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.235 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:10.235 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:10.236 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:10.237 DEBUG:boto:Method: HEAD 2022-09-03T06:44:10.237 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:44:10.238 DEBUG:boto:Data: 2022-09-03T06:44:10.238 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.239 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.239 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.240 DEBUG:boto:Params: {} 2022-09-03T06:44:10.241 DEBUG:boto:Token: None 2022-09-03T06:44:10.241 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj51 2022-09-03T06:44:10.242 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/fWDvIypbUDgd5wnvMbLOfU6ZPU= 2022-09-03T06:44:10.242 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/fWDvIypbUDgd5wnvMbLOfU6ZPU=', 'Content-Length': '0'} 2022-09-03T06:44:10.245 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aa51fa452ec662d6-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.246 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:10.246 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:10.247 DEBUG:boto:Method: HEAD 2022-09-03T06:44:10.247 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:44:10.248 DEBUG:boto:Data: 2022-09-03T06:44:10.248 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.249 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.249 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.250 DEBUG:boto:Params: {} 2022-09-03T06:44:10.250 DEBUG:boto:Token: None 2022-09-03T06:44:10.251 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj51 2022-09-03T06:44:10.251 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/fWDvIypbUDgd5wnvMbLOfU6ZPU= 2022-09-03T06:44:10.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/fWDvIypbUDgd5wnvMbLOfU6ZPU=', 'Content-Length': '0'} 2022-09-03T06:44:10.255 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000da6b13786dedbadc-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.256 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:44:10.257 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:10.257 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:10.258 DEBUG:boto:Method: GET 2022-09-03T06:44:10.259 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:44:10.260 DEBUG:boto:Data: 2022-09-03T06:44:10.261 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.261 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.262 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.263 DEBUG:boto:Params: {} 2022-09-03T06:44:10.264 DEBUG:boto:Token: None 2022-09-03T06:44:10.265 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj51 2022-09-03T06:44:10.266 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GDjOTkXwWFHBI0TM8NE1S5V2Rvc= 2022-09-03T06:44:10.267 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GDjOTkXwWFHBI0TM8NE1S5V2Rvc=', 'Content-Length': '0'} 2022-09-03T06:44:10.270 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fdac65e50bfdbb2e-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.271 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:10.272 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:10.272 DEBUG:boto:Method: GET 2022-09-03T06:44:10.273 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:44:10.273 DEBUG:boto:Data: 2022-09-03T06:44:10.274 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.275 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.276 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.276 DEBUG:boto:Params: {} 2022-09-03T06:44:10.277 DEBUG:boto:Token: None 2022-09-03T06:44:10.278 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj51 2022-09-03T06:44:10.278 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GDjOTkXwWFHBI0TM8NE1S5V2Rvc= 2022-09-03T06:44:10.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GDjOTkXwWFHBI0TM8NE1S5V2Rvc=', 'Content-Length': '0'} 2022-09-03T06:44:10.282 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a99f8505a944be3-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.330 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:10.331 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:10.332 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:10.333 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:10.333 DEBUG:boto:Method: GET 2022-09-03T06:44:10.334 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:44:10.335 DEBUG:boto:Data: 2022-09-03T06:44:10.336 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.337 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.337 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.338 DEBUG:boto:Params: {} 2022-09-03T06:44:10.339 DEBUG:boto:Token: None 2022-09-03T06:44:10.342 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/ 2022-09-03T06:44:10.344 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU= 2022-09-03T06:44:10.345 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU=', 'Content-Length': '0'} 2022-09-03T06:44:10.352 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005e5f8efb2dcb6e99-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.354 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:10.356 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:10.357 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:10.357 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:10.358 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:10.358 DEBUG:boto:Method: GET 2022-09-03T06:44:10.359 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:44:10.359 DEBUG:boto:Data: 2022-09-03T06:44:10.360 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.363 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.364 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.367 DEBUG:boto:Params: {} 2022-09-03T06:44:10.368 DEBUG:boto:Token: None 2022-09-03T06:44:10.369 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/ 2022-09-03T06:44:10.369 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU= 2022-09-03T06:44:10.370 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU=', 'Content-Length': '0'} 2022-09-03T06:44:10.376 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000491e1055050d892a-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.379 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:10.380 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:44:10.381 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:10.382 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:10.383 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.384 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.385 DEBUG:boto:Method: GET 2022-09-03T06:44:10.386 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.389 DEBUG:boto:Data: 2022-09-03T06:44:10.390 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.391 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.392 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.393 DEBUG:boto:Params: {} 2022-09-03T06:44:10.395 DEBUG:boto:Token: None 2022-09-03T06:44:10.396 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.397 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wMep4pSY0Kl0X/9ssVHAxQwFUAU= 2022-09-03T06:44:10.398 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wMep4pSY0Kl0X/9ssVHAxQwFUAU=', 'Content-Length': '0'} 2022-09-03T06:44:10.401 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000001df81f1ea1a67e0-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.403 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:10.404 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:10.405 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.405 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.406 DEBUG:boto:Method: GET 2022-09-03T06:44:10.407 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.407 DEBUG:boto:Data: 2022-09-03T06:44:10.408 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.408 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.409 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.410 DEBUG:boto:Params: {} 2022-09-03T06:44:10.411 DEBUG:boto:Token: None 2022-09-03T06:44:10.413 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.414 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wMep4pSY0Kl0X/9ssVHAxQwFUAU= 2022-09-03T06:44:10.415 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wMep4pSY0Kl0X/9ssVHAxQwFUAU=', 'Content-Length': '0'} 2022-09-03T06:44:10.422 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000086db973dd9e08625-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.426 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:10.428 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:10.429 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.430 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.431 DEBUG:boto:Method: HEAD 2022-09-03T06:44:10.433 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.435 DEBUG:boto:Data: 2022-09-03T06:44:10.436 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.437 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.438 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.438 DEBUG:boto:Params: {} 2022-09-03T06:44:10.439 DEBUG:boto:Token: None 2022-09-03T06:44:10.440 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.441 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3xKwUD3r4u5ua4baSxpymEiy70Q= 2022-09-03T06:44:10.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3xKwUD3r4u5ua4baSxpymEiy70Q=', 'Content-Length': '0'} 2022-09-03T06:44:10.448 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000047819f0a7fdba87a-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.449 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:10.449 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:10.450 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.452 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.453 DEBUG:boto:Method: HEAD 2022-09-03T06:44:10.453 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.454 DEBUG:boto:Data: 2022-09-03T06:44:10.455 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.455 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.456 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.457 DEBUG:boto:Params: {} 2022-09-03T06:44:10.458 DEBUG:boto:Token: None 2022-09-03T06:44:10.458 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.459 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3xKwUD3r4u5ua4baSxpymEiy70Q= 2022-09-03T06:44:10.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3xKwUD3r4u5ua4baSxpymEiy70Q=', 'Content-Length': '0'} 2022-09-03T06:44:10.464 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008cd5801a7646f329-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.466 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:44:10.467 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:10.467 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:10.468 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.469 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.470 DEBUG:boto:Method: GET 2022-09-03T06:44:10.471 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.472 DEBUG:boto:Data: 2022-09-03T06:44:10.472 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.473 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.474 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.475 DEBUG:boto:Params: {} 2022-09-03T06:44:10.476 DEBUG:boto:Token: None 2022-09-03T06:44:10.477 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.478 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wMep4pSY0Kl0X/9ssVHAxQwFUAU= 2022-09-03T06:44:10.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wMep4pSY0Kl0X/9ssVHAxQwFUAU=', 'Content-Length': '0'} 2022-09-03T06:44:10.485 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ebfeaf040b31675c-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.486 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:10.487 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:10.488 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.489 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.490 DEBUG:boto:Method: GET 2022-09-03T06:44:10.491 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.492 DEBUG:boto:Data: 2022-09-03T06:44:10.493 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.494 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.495 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.495 DEBUG:boto:Params: {} 2022-09-03T06:44:10.496 DEBUG:boto:Token: None 2022-09-03T06:44:10.497 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:10.497 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wMep4pSY0Kl0X/9ssVHAxQwFUAU= 2022-09-03T06:44:10.498 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wMep4pSY0Kl0X/9ssVHAxQwFUAU=', 'Content-Length': '0'} 2022-09-03T06:44:10.501 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001450f1734af43c60-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.503 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:10.504 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:10.505 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:10.505 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:10.506 DEBUG:boto:Method: GET 2022-09-03T06:44:10.507 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:44:10.507 DEBUG:boto:Data: 2022-09-03T06:44:10.508 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.508 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.509 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.510 DEBUG:boto:Params: {} 2022-09-03T06:44:10.510 DEBUG:boto:Token: None 2022-09-03T06:44:10.511 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/ 2022-09-03T06:44:10.512 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU= 2022-09-03T06:44:10.512 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU=', 'Content-Length': '0'} 2022-09-03T06:44:10.515 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000091bc3b4df2c46f93-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.561 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:10.563 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:10.564 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:10.565 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:10.566 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:10.567 DEBUG:boto:Method: GET 2022-09-03T06:44:10.568 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:44:10.568 DEBUG:boto:Data: 2022-09-03T06:44:10.569 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.570 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.570 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.571 DEBUG:boto:Params: {} 2022-09-03T06:44:10.571 DEBUG:boto:Token: None 2022-09-03T06:44:10.572 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/ 2022-09-03T06:44:10.573 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU= 2022-09-03T06:44:10.573 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU=', 'Content-Length': '0'} 2022-09-03T06:44:10.578 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c3dc3be650535bdf-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.579 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:10.580 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:10.581 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:10.581 DEBUG:boto:Method: HEAD 2022-09-03T06:44:10.582 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:44:10.582 DEBUG:boto:Data: 2022-09-03T06:44:10.583 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.584 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.585 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.586 DEBUG:boto:Params: {} 2022-09-03T06:44:10.587 DEBUG:boto:Token: None 2022-09-03T06:44:10.588 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj55 2022-09-03T06:44:10.589 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tNGzQaSr/zNW/ZRIINCgN0HJ2oI= 2022-09-03T06:44:10.590 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tNGzQaSr/zNW/ZRIINCgN0HJ2oI=', 'Content-Length': '0'} 2022-09-03T06:44:10.594 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d7d91bcf7e00a65f-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.595 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:10.596 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:10.597 DEBUG:boto:Method: HEAD 2022-09-03T06:44:10.598 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:44:10.599 DEBUG:boto:Data: 2022-09-03T06:44:10.599 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.600 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.600 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.601 DEBUG:boto:Params: {} 2022-09-03T06:44:10.601 DEBUG:boto:Token: None 2022-09-03T06:44:10.602 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj55 2022-09-03T06:44:10.603 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tNGzQaSr/zNW/ZRIINCgN0HJ2oI= 2022-09-03T06:44:10.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tNGzQaSr/zNW/ZRIINCgN0HJ2oI=', 'Content-Length': '0'} 2022-09-03T06:44:10.606 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001024f6c214f4ec49-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.607 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:44:10.608 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:10.608 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:10.609 DEBUG:boto:Method: GET 2022-09-03T06:44:10.609 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:44:10.610 DEBUG:boto:Data: 2022-09-03T06:44:10.611 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.612 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.612 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.613 DEBUG:boto:Params: {} 2022-09-03T06:44:10.614 DEBUG:boto:Token: None 2022-09-03T06:44:10.614 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj55 2022-09-03T06:44:10.615 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KpT/ZfjH5u2Ml9Darf3KabTJ88c= 2022-09-03T06:44:10.615 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KpT/ZfjH5u2Ml9Darf3KabTJ88c=', 'Content-Length': '0'} 2022-09-03T06:44:10.618 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021e88130c751cf98-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.620 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:10.620 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:10.621 DEBUG:boto:Method: GET 2022-09-03T06:44:10.622 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:44:10.622 DEBUG:boto:Data: 2022-09-03T06:44:10.623 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.623 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.624 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.624 DEBUG:boto:Params: {} 2022-09-03T06:44:10.625 DEBUG:boto:Token: None 2022-09-03T06:44:10.626 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj55 2022-09-03T06:44:10.626 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KpT/ZfjH5u2Ml9Darf3KabTJ88c= 2022-09-03T06:44:10.627 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KpT/ZfjH5u2Ml9Darf3KabTJ88c=', 'Content-Length': '0'} 2022-09-03T06:44:10.630 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000915e02f520b29a78-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.672 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:10.673 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:10.673 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:10.674 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:10.675 DEBUG:boto:Method: GET 2022-09-03T06:44:10.675 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:44:10.676 DEBUG:boto:Data: 2022-09-03T06:44:10.676 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.677 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.677 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.678 DEBUG:boto:Params: {} 2022-09-03T06:44:10.679 DEBUG:boto:Token: None 2022-09-03T06:44:10.679 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/ 2022-09-03T06:44:10.680 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU= 2022-09-03T06:44:10.681 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU=', 'Content-Length': '0'} 2022-09-03T06:44:10.684 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006fd6351871189d8a-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.685 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:10.686 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:10.687 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:10.687 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:10.688 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:10.688 DEBUG:boto:Method: GET 2022-09-03T06:44:10.689 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:44:10.690 DEBUG:boto:Data: 2022-09-03T06:44:10.690 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.691 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.691 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.692 DEBUG:boto:Params: {} 2022-09-03T06:44:10.693 DEBUG:boto:Token: None 2022-09-03T06:44:10.693 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/ 2022-09-03T06:44:10.694 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU= 2022-09-03T06:44:10.694 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU=', 'Content-Length': '0'} 2022-09-03T06:44:10.698 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000023f73649663db449-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.699 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:10.700 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:44:10.701 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:10.702 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:10.702 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.703 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.704 DEBUG:boto:Method: GET 2022-09-03T06:44:10.705 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.706 DEBUG:boto:Data: 2022-09-03T06:44:10.707 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.708 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.709 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.710 DEBUG:boto:Params: {} 2022-09-03T06:44:10.711 DEBUG:boto:Token: None 2022-09-03T06:44:10.712 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.713 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DfF9SB3H0WIES39lvvityTqAcho= 2022-09-03T06:44:10.714 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DfF9SB3H0WIES39lvvityTqAcho=', 'Content-Length': '0'} 2022-09-03T06:44:10.716 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000306befc2bae48aaf-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.717 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:10.717 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:10.718 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.718 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.719 DEBUG:boto:Method: GET 2022-09-03T06:44:10.720 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.720 DEBUG:boto:Data: 2022-09-03T06:44:10.721 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.721 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.722 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.722 DEBUG:boto:Params: {} 2022-09-03T06:44:10.723 DEBUG:boto:Token: None 2022-09-03T06:44:10.723 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.724 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DfF9SB3H0WIES39lvvityTqAcho= 2022-09-03T06:44:10.725 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DfF9SB3H0WIES39lvvityTqAcho=', 'Content-Length': '0'} 2022-09-03T06:44:10.727 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000884a476be807408f-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.728 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:10.728 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:10.729 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.729 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.730 DEBUG:boto:Method: HEAD 2022-09-03T06:44:10.731 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.731 DEBUG:boto:Data: 2022-09-03T06:44:10.732 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.732 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.733 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.733 DEBUG:boto:Params: {} 2022-09-03T06:44:10.734 DEBUG:boto:Token: None 2022-09-03T06:44:10.734 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.735 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4lRV3Df2LTU/nN/HLRUlyDDlxKM= 2022-09-03T06:44:10.736 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4lRV3Df2LTU/nN/HLRUlyDDlxKM=', 'Content-Length': '0'} 2022-09-03T06:44:10.738 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000662142038a669083-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.739 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:10.740 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:10.741 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.742 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.743 DEBUG:boto:Method: HEAD 2022-09-03T06:44:10.743 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.744 DEBUG:boto:Data: 2022-09-03T06:44:10.745 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.746 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.747 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.748 DEBUG:boto:Params: {} 2022-09-03T06:44:10.749 DEBUG:boto:Token: None 2022-09-03T06:44:10.750 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.751 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4lRV3Df2LTU/nN/HLRUlyDDlxKM= 2022-09-03T06:44:10.752 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4lRV3Df2LTU/nN/HLRUlyDDlxKM=', 'Content-Length': '0'} 2022-09-03T06:44:10.755 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f38763d92d3eb3a8-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.756 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:44:10.757 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:10.758 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:10.759 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.760 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.761 DEBUG:boto:Method: GET 2022-09-03T06:44:10.762 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.763 DEBUG:boto:Data: 2022-09-03T06:44:10.764 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.765 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.765 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.766 DEBUG:boto:Params: {} 2022-09-03T06:44:10.767 DEBUG:boto:Token: None 2022-09-03T06:44:10.768 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.769 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DfF9SB3H0WIES39lvvityTqAcho= 2022-09-03T06:44:10.770 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DfF9SB3H0WIES39lvvityTqAcho=', 'Content-Length': '0'} 2022-09-03T06:44:10.772 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e247de6caf9b8b1a-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.773 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:10.774 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:10.774 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.775 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.775 DEBUG:boto:Method: GET 2022-09-03T06:44:10.776 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.776 DEBUG:boto:Data: 2022-09-03T06:44:10.777 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.778 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.786 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.786 DEBUG:boto:Params: {} 2022-09-03T06:44:10.787 DEBUG:boto:Token: None 2022-09-03T06:44:10.788 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:10.788 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DfF9SB3H0WIES39lvvityTqAcho= 2022-09-03T06:44:10.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DfF9SB3H0WIES39lvvityTqAcho=', 'Content-Length': '0'} 2022-09-03T06:44:10.791 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ab5e5e30d37d0a22-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.838 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:10.839 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:10.839 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:10.840 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:10.841 DEBUG:boto:Method: GET 2022-09-03T06:44:10.841 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:44:10.842 DEBUG:boto:Data: 2022-09-03T06:44:10.842 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.843 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.843 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.844 DEBUG:boto:Params: {} 2022-09-03T06:44:10.845 DEBUG:boto:Token: None 2022-09-03T06:44:10.845 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/ 2022-09-03T06:44:10.846 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU= 2022-09-03T06:44:10.847 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU=', 'Content-Length': '0'} 2022-09-03T06:44:10.849 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000058337303d2bcb2d-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.850 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:10.851 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:10.852 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:10.853 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:10.854 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:10.854 DEBUG:boto:Method: GET 2022-09-03T06:44:10.855 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:44:10.856 DEBUG:boto:Data: 2022-09-03T06:44:10.856 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.857 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.858 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.859 DEBUG:boto:Params: {} 2022-09-03T06:44:10.859 DEBUG:boto:Token: None 2022-09-03T06:44:10.860 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/ 2022-09-03T06:44:10.861 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU= 2022-09-03T06:44:10.862 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU=', 'Content-Length': '0'} 2022-09-03T06:44:10.865 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a583151b6bb3e369-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.866 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:10.867 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:10.868 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:10.868 DEBUG:boto:Method: HEAD 2022-09-03T06:44:10.869 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:44:10.870 DEBUG:boto:Data: 2022-09-03T06:44:10.870 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.871 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.872 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.872 DEBUG:boto:Params: {} 2022-09-03T06:44:10.873 DEBUG:boto:Token: None 2022-09-03T06:44:10.874 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj57 2022-09-03T06:44:10.874 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+8O+VLiiEkJKjp0gsjXv7U1rDP4= 2022-09-03T06:44:10.875 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+8O+VLiiEkJKjp0gsjXv7U1rDP4=', 'Content-Length': '0'} 2022-09-03T06:44:10.878 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e71aa6bd710aea77-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.878 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:10.879 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:10.880 DEBUG:boto:Method: HEAD 2022-09-03T06:44:10.880 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:44:10.881 DEBUG:boto:Data: 2022-09-03T06:44:10.881 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.882 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.883 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.883 DEBUG:boto:Params: {} 2022-09-03T06:44:10.884 DEBUG:boto:Token: None 2022-09-03T06:44:10.884 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj57 2022-09-03T06:44:10.885 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+8O+VLiiEkJKjp0gsjXv7U1rDP4= 2022-09-03T06:44:10.885 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+8O+VLiiEkJKjp0gsjXv7U1rDP4=', 'Content-Length': '0'} 2022-09-03T06:44:10.888 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000136244e0e3a76192-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.889 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:44:10.889 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:10.890 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:10.891 DEBUG:boto:Method: GET 2022-09-03T06:44:10.891 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:44:10.892 DEBUG:boto:Data: 2022-09-03T06:44:10.893 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.894 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.894 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.895 DEBUG:boto:Params: {} 2022-09-03T06:44:10.896 DEBUG:boto:Token: None 2022-09-03T06:44:10.896 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj57 2022-09-03T06:44:10.897 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XkkLCC/YHub9dcC2jRLFkh8L484= 2022-09-03T06:44:10.897 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XkkLCC/YHub9dcC2jRLFkh8L484=', 'Content-Length': '0'} 2022-09-03T06:44:10.900 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bb9488a6badfc2df-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.900 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:10.901 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:10.901 DEBUG:boto:Method: GET 2022-09-03T06:44:10.902 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:44:10.903 DEBUG:boto:Data: 2022-09-03T06:44:10.904 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.904 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.905 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.906 DEBUG:boto:Params: {} 2022-09-03T06:44:10.907 DEBUG:boto:Token: None 2022-09-03T06:44:10.908 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj57 2022-09-03T06:44:10.908 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XkkLCC/YHub9dcC2jRLFkh8L484= 2022-09-03T06:44:10.909 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XkkLCC/YHub9dcC2jRLFkh8L484=', 'Content-Length': '0'} 2022-09-03T06:44:10.912 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000287fbfc594102c6d-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.912 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:10.914 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:10.915 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:10.915 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:10.916 DEBUG:boto:Method: GET 2022-09-03T06:44:10.916 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:44:10.917 DEBUG:boto:Data: 2022-09-03T06:44:10.917 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.918 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.918 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.919 DEBUG:boto:Params: {} 2022-09-03T06:44:10.919 DEBUG:boto:Token: None 2022-09-03T06:44:10.920 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/ 2022-09-03T06:44:10.921 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU= 2022-09-03T06:44:10.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU=', 'Content-Length': '0'} 2022-09-03T06:44:10.924 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001d3dd577981c7c55-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.925 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:10.926 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:10.927 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:10.928 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:10.928 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:10.929 DEBUG:boto:Method: GET 2022-09-03T06:44:10.929 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:44:10.930 DEBUG:boto:Data: 2022-09-03T06:44:10.930 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.931 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.931 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.932 DEBUG:boto:Params: {} 2022-09-03T06:44:10.933 DEBUG:boto:Token: None 2022-09-03T06:44:10.933 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/ 2022-09-03T06:44:10.934 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU= 2022-09-03T06:44:10.935 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wqqw71QfCIIcTxQwA+NCQBqJaCU=', 'Content-Length': '0'} 2022-09-03T06:44:10.938 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003db55792a23c4c16-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.939 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:10.940 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:44:10.941 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:10.941 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:10.942 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:10.943 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:10.943 DEBUG:boto:Method: GET 2022-09-03T06:44:10.944 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:10.945 DEBUG:boto:Data: 2022-09-03T06:44:10.945 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.946 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.947 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.947 DEBUG:boto:Params: {} 2022-09-03T06:44:10.948 DEBUG:boto:Token: None 2022-09-03T06:44:10.949 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:10.949 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ye9xit1MZ0x9GdBIemZadayrV9I= 2022-09-03T06:44:10.950 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ye9xit1MZ0x9GdBIemZadayrV9I=', 'Content-Length': '0'} 2022-09-03T06:44:10.953 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f671c0cdf28ef478-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.954 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:10.954 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:10.955 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:10.956 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:10.956 DEBUG:boto:Method: GET 2022-09-03T06:44:10.957 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:10.958 DEBUG:boto:Data: 2022-09-03T06:44:10.958 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.959 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.959 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.960 DEBUG:boto:Params: {} 2022-09-03T06:44:10.960 DEBUG:boto:Token: None 2022-09-03T06:44:10.961 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:10.962 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ye9xit1MZ0x9GdBIemZadayrV9I= 2022-09-03T06:44:10.963 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ye9xit1MZ0x9GdBIemZadayrV9I=', 'Content-Length': '0'} 2022-09-03T06:44:10.965 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b91a58fe7d2ac01f-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.967 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:10.967 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:10.968 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:10.969 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:10.969 DEBUG:boto:Method: HEAD 2022-09-03T06:44:10.970 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:10.971 DEBUG:boto:Data: 2022-09-03T06:44:10.971 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.972 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.972 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.973 DEBUG:boto:Params: {} 2022-09-03T06:44:10.974 DEBUG:boto:Token: None 2022-09-03T06:44:10.975 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:10.975 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SBp51N97NQKzFkZsdL0kyAVsQmM= 2022-09-03T06:44:10.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SBp51N97NQKzFkZsdL0kyAVsQmM=', 'Content-Length': '0'} 2022-09-03T06:44:10.978 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000023b4f0e8e99544a0-006312f7ba-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.979 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:10.980 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:10.980 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:10.981 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:10.982 DEBUG:boto:Method: HEAD 2022-09-03T06:44:10.982 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:10.983 DEBUG:boto:Data: 2022-09-03T06:44:10.983 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.984 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:10.984 DEBUG:boto:Port: 8000 2022-09-03T06:44:10.985 DEBUG:boto:Params: {} 2022-09-03T06:44:10.986 DEBUG:boto:Token: None 2022-09-03T06:44:10.986 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:10 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:10.987 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SBp51N97NQKzFkZsdL0kyAVsQmM= 2022-09-03T06:44:10.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SBp51N97NQKzFkZsdL0kyAVsQmM=', 'Content-Length': '0'} 2022-09-03T06:44:10.990 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007771ef3fd45eab43-006312f7ba-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:10.991 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:44:10.992 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:10.992 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:10.993 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:10.994 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:10.994 DEBUG:boto:Method: GET 2022-09-03T06:44:10.995 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:10.996 DEBUG:boto:Data: 2022-09-03T06:44:10.997 DEBUG:boto:Headers: {} 2022-09-03T06:44:10.997 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:10.998 DEBUG:boto:Port: 8001 2022-09-03T06:44:10.999 DEBUG:boto:Params: {} 2022-09-03T06:44:10.999 DEBUG:boto:Token: None 2022-09-03T06:44:11.000 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:11.001 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+MCCw/QBH6XA3+jPmUbrBma1SCA= 2022-09-03T06:44:11.002 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+MCCw/QBH6XA3+jPmUbrBma1SCA=', 'Content-Length': '0'} 2022-09-03T06:44:11.004 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000214825c70a3065f5-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.005 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:11.006 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:11.006 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:11.007 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:11.008 DEBUG:boto:Method: GET 2022-09-03T06:44:11.008 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:11.009 DEBUG:boto:Data: 2022-09-03T06:44:11.009 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.010 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.011 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.011 DEBUG:boto:Params: {} 2022-09-03T06:44:11.012 DEBUG:boto:Token: None 2022-09-03T06:44:11.013 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:11.013 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+MCCw/QBH6XA3+jPmUbrBma1SCA= 2022-09-03T06:44:11.014 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+MCCw/QBH6XA3+jPmUbrBma1SCA=', 'Content-Length': '0'} 2022-09-03T06:44:11.017 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e8df772f2a994196-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.058 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:11.059 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:11.059 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:11.060 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:11.060 DEBUG:boto:Method: GET 2022-09-03T06:44:11.061 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:44:11.062 DEBUG:boto:Data: 2022-09-03T06:44:11.063 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.064 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.066 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.067 DEBUG:boto:Params: {} 2022-09-03T06:44:11.067 DEBUG:boto:Token: None 2022-09-03T06:44:11.068 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/ 2022-09-03T06:44:11.069 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg= 2022-09-03T06:44:11.069 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg=', 'Content-Length': '0'} 2022-09-03T06:44:11.073 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e0e0101e2907bb78-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.073 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:11.074 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:11.075 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:11.075 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:11.077 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:11.077 DEBUG:boto:Method: GET 2022-09-03T06:44:11.078 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:44:11.078 DEBUG:boto:Data: 2022-09-03T06:44:11.079 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.080 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.081 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.082 DEBUG:boto:Params: {} 2022-09-03T06:44:11.083 DEBUG:boto:Token: None 2022-09-03T06:44:11.084 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/ 2022-09-03T06:44:11.085 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg= 2022-09-03T06:44:11.086 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg=', 'Content-Length': '0'} 2022-09-03T06:44:11.089 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000040ff737001eaf7a4-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.090 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:11.091 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:11.092 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:11.092 DEBUG:boto:Method: HEAD 2022-09-03T06:44:11.093 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:44:11.093 DEBUG:boto:Data: 2022-09-03T06:44:11.094 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.094 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.095 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.095 DEBUG:boto:Params: {} 2022-09-03T06:44:11.096 DEBUG:boto:Token: None 2022-09-03T06:44:11.096 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj68 2022-09-03T06:44:11.097 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PIgOiehCW9FbK1YagEYyrHlqvRk= 2022-09-03T06:44:11.098 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PIgOiehCW9FbK1YagEYyrHlqvRk=', 'Content-Length': '0'} 2022-09-03T06:44:11.100 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2ee4d77aa894fc4-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.101 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:11.102 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:11.102 DEBUG:boto:Method: HEAD 2022-09-03T06:44:11.103 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:44:11.103 DEBUG:boto:Data: 2022-09-03T06:44:11.104 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.104 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.105 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.105 DEBUG:boto:Params: {} 2022-09-03T06:44:11.106 DEBUG:boto:Token: None 2022-09-03T06:44:11.107 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj68 2022-09-03T06:44:11.107 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PIgOiehCW9FbK1YagEYyrHlqvRk= 2022-09-03T06:44:11.108 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PIgOiehCW9FbK1YagEYyrHlqvRk=', 'Content-Length': '0'} 2022-09-03T06:44:11.110 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007ddca6e54b31782f-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.111 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:44:11.111 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:11.112 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:11.112 DEBUG:boto:Method: GET 2022-09-03T06:44:11.113 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:44:11.113 DEBUG:boto:Data: 2022-09-03T06:44:11.114 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.115 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.115 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.116 DEBUG:boto:Params: {} 2022-09-03T06:44:11.116 DEBUG:boto:Token: None 2022-09-03T06:44:11.117 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj68 2022-09-03T06:44:11.117 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G5nZfE+eDK43Id5m6TcDsIxQYVI= 2022-09-03T06:44:11.118 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G5nZfE+eDK43Id5m6TcDsIxQYVI=', 'Content-Length': '0'} 2022-09-03T06:44:11.121 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b248d04ccbb1268-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.122 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:11.123 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:11.124 DEBUG:boto:Method: GET 2022-09-03T06:44:11.124 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:44:11.125 DEBUG:boto:Data: 2022-09-03T06:44:11.126 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.127 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.128 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.128 DEBUG:boto:Params: {} 2022-09-03T06:44:11.129 DEBUG:boto:Token: None 2022-09-03T06:44:11.130 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj68 2022-09-03T06:44:11.131 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G5nZfE+eDK43Id5m6TcDsIxQYVI= 2022-09-03T06:44:11.131 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G5nZfE+eDK43Id5m6TcDsIxQYVI=', 'Content-Length': '0'} 2022-09-03T06:44:11.133 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000209b6462f14a10b6-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.134 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:11.135 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:11.135 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:11.136 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:11.137 DEBUG:boto:Method: GET 2022-09-03T06:44:11.138 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:44:11.139 DEBUG:boto:Data: 2022-09-03T06:44:11.140 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.141 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.142 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.142 DEBUG:boto:Params: {} 2022-09-03T06:44:11.143 DEBUG:boto:Token: None 2022-09-03T06:44:11.143 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/ 2022-09-03T06:44:11.144 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg= 2022-09-03T06:44:11.145 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg=', 'Content-Length': '0'} 2022-09-03T06:44:11.148 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008cc831d3235fbbf0-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.148 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:11.149 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:11.150 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:11.150 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:11.151 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:11.151 DEBUG:boto:Method: GET 2022-09-03T06:44:11.152 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:44:11.152 DEBUG:boto:Data: 2022-09-03T06:44:11.153 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.153 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.154 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.155 DEBUG:boto:Params: {} 2022-09-03T06:44:11.156 DEBUG:boto:Token: None 2022-09-03T06:44:11.157 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/ 2022-09-03T06:44:11.158 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg= 2022-09-03T06:44:11.159 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg=', 'Content-Length': '0'} 2022-09-03T06:44:11.163 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000adb5a51029fba5fe-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.164 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:11.165 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:44:11.166 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:11.166 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:11.167 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.167 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.168 DEBUG:boto:Method: GET 2022-09-03T06:44:11.168 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.169 DEBUG:boto:Data: 2022-09-03T06:44:11.170 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.170 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.171 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.171 DEBUG:boto:Params: {} 2022-09-03T06:44:11.172 DEBUG:boto:Token: None 2022-09-03T06:44:11.173 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.173 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:56Knag6P3uV5NQNAmpejwa6WirM= 2022-09-03T06:44:11.174 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:56Knag6P3uV5NQNAmpejwa6WirM=', 'Content-Length': '0'} 2022-09-03T06:44:11.176 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f01d0441419487dd-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.222 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:11.223 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:11.224 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.225 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.226 DEBUG:boto:Method: GET 2022-09-03T06:44:11.226 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.227 DEBUG:boto:Data: 2022-09-03T06:44:11.228 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.228 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.229 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.230 DEBUG:boto:Params: {} 2022-09-03T06:44:11.231 DEBUG:boto:Token: None 2022-09-03T06:44:11.231 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.232 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:56Knag6P3uV5NQNAmpejwa6WirM= 2022-09-03T06:44:11.233 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:56Knag6P3uV5NQNAmpejwa6WirM=', 'Content-Length': '0'} 2022-09-03T06:44:11.236 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eeb284cad9920aa8-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.237 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:11.237 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:11.238 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.239 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.239 DEBUG:boto:Method: HEAD 2022-09-03T06:44:11.240 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.240 DEBUG:boto:Data: 2022-09-03T06:44:11.241 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.242 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.242 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.243 DEBUG:boto:Params: {} 2022-09-03T06:44:11.244 DEBUG:boto:Token: None 2022-09-03T06:44:11.244 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.245 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0YcnNcx55G+HrL8m5cVdQdtdwXE= 2022-09-03T06:44:11.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0YcnNcx55G+HrL8m5cVdQdtdwXE=', 'Content-Length': '0'} 2022-09-03T06:44:11.248 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f088bf3d817b89b-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.249 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:11.250 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:11.250 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.251 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.251 DEBUG:boto:Method: HEAD 2022-09-03T06:44:11.252 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.253 DEBUG:boto:Data: 2022-09-03T06:44:11.253 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.254 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.255 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.255 DEBUG:boto:Params: {} 2022-09-03T06:44:11.256 DEBUG:boto:Token: None 2022-09-03T06:44:11.257 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.258 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0YcnNcx55G+HrL8m5cVdQdtdwXE= 2022-09-03T06:44:11.259 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0YcnNcx55G+HrL8m5cVdQdtdwXE=', 'Content-Length': '0'} 2022-09-03T06:44:11.262 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f5029a98a7e08708-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.263 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:44:11.264 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:11.265 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:11.266 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.266 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.267 DEBUG:boto:Method: GET 2022-09-03T06:44:11.268 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.269 DEBUG:boto:Data: 2022-09-03T06:44:11.270 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.271 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.272 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.273 DEBUG:boto:Params: {} 2022-09-03T06:44:11.274 DEBUG:boto:Token: None 2022-09-03T06:44:11.275 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.276 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:56Knag6P3uV5NQNAmpejwa6WirM= 2022-09-03T06:44:11.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:56Knag6P3uV5NQNAmpejwa6WirM=', 'Content-Length': '0'} 2022-09-03T06:44:11.279 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000321d5e6683cc73d4-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.280 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:11.280 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:11.281 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.282 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.282 DEBUG:boto:Method: GET 2022-09-03T06:44:11.283 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.283 DEBUG:boto:Data: 2022-09-03T06:44:11.284 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.284 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.285 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.285 DEBUG:boto:Params: {} 2022-09-03T06:44:11.286 DEBUG:boto:Token: None 2022-09-03T06:44:11.287 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:11.287 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:56Knag6P3uV5NQNAmpejwa6WirM= 2022-09-03T06:44:11.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:56Knag6P3uV5NQNAmpejwa6WirM=', 'Content-Length': '0'} 2022-09-03T06:44:11.290 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000088570ddaf54369b2-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.334 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:11.335 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:11.336 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:11.336 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:11.337 DEBUG:boto:Method: GET 2022-09-03T06:44:11.338 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:44:11.338 DEBUG:boto:Data: 2022-09-03T06:44:11.339 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.339 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.340 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.341 DEBUG:boto:Params: {} 2022-09-03T06:44:11.341 DEBUG:boto:Token: None 2022-09-03T06:44:11.342 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/ 2022-09-03T06:44:11.343 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg= 2022-09-03T06:44:11.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg=', 'Content-Length': '0'} 2022-09-03T06:44:11.348 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000073db65ac42746872-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.349 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:11.350 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:11.351 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:11.352 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:11.353 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:11.354 DEBUG:boto:Method: GET 2022-09-03T06:44:11.355 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:44:11.356 DEBUG:boto:Data: 2022-09-03T06:44:11.356 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.357 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.358 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.358 DEBUG:boto:Params: {} 2022-09-03T06:44:11.359 DEBUG:boto:Token: None 2022-09-03T06:44:11.359 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/ 2022-09-03T06:44:11.360 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg= 2022-09-03T06:44:11.361 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg=', 'Content-Length': '0'} 2022-09-03T06:44:11.365 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003e53ab8cd4f980fc-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.366 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:11.367 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:11.368 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:11.369 DEBUG:boto:Method: HEAD 2022-09-03T06:44:11.370 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:44:11.371 DEBUG:boto:Data: 2022-09-03T06:44:11.372 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.372 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.373 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.374 DEBUG:boto:Params: {} 2022-09-03T06:44:11.374 DEBUG:boto:Token: None 2022-09-03T06:44:11.375 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj69 2022-09-03T06:44:11.376 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4E5W3zX56/MeVsvXD80FbN8P8kQ= 2022-09-03T06:44:11.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4E5W3zX56/MeVsvXD80FbN8P8kQ=', 'Content-Length': '0'} 2022-09-03T06:44:11.380 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c418492b1c8f0b8-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.388 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:11.389 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:11.390 DEBUG:boto:Method: HEAD 2022-09-03T06:44:11.391 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:44:11.392 DEBUG:boto:Data: 2022-09-03T06:44:11.393 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.394 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.394 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.395 DEBUG:boto:Params: {} 2022-09-03T06:44:11.396 DEBUG:boto:Token: None 2022-09-03T06:44:11.397 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj69 2022-09-03T06:44:11.398 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4E5W3zX56/MeVsvXD80FbN8P8kQ= 2022-09-03T06:44:11.399 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4E5W3zX56/MeVsvXD80FbN8P8kQ=', 'Content-Length': '0'} 2022-09-03T06:44:11.402 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db10259215e8450a-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.403 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:44:11.403 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:11.404 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:11.405 DEBUG:boto:Method: GET 2022-09-03T06:44:11.405 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:44:11.406 DEBUG:boto:Data: 2022-09-03T06:44:11.407 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.407 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.408 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.408 DEBUG:boto:Params: {} 2022-09-03T06:44:11.409 DEBUG:boto:Token: None 2022-09-03T06:44:11.410 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj69 2022-09-03T06:44:11.410 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kIrd01L+9nFhhKvQNImP9oXgllw= 2022-09-03T06:44:11.411 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kIrd01L+9nFhhKvQNImP9oXgllw=', 'Content-Length': '0'} 2022-09-03T06:44:11.413 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dcf027e9c1dbbac1-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.414 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:11.415 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:11.415 DEBUG:boto:Method: GET 2022-09-03T06:44:11.416 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:44:11.417 DEBUG:boto:Data: 2022-09-03T06:44:11.417 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.418 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.418 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.419 DEBUG:boto:Params: {} 2022-09-03T06:44:11.420 DEBUG:boto:Token: None 2022-09-03T06:44:11.420 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj69 2022-09-03T06:44:11.421 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kIrd01L+9nFhhKvQNImP9oXgllw= 2022-09-03T06:44:11.422 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kIrd01L+9nFhhKvQNImP9oXgllw=', 'Content-Length': '0'} 2022-09-03T06:44:11.424 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000660a61b64f021184-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.425 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:11.426 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:11.427 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:11.428 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:11.429 DEBUG:boto:Method: GET 2022-09-03T06:44:11.430 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:44:11.431 DEBUG:boto:Data: 2022-09-03T06:44:11.432 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.432 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.433 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.433 DEBUG:boto:Params: {} 2022-09-03T06:44:11.434 DEBUG:boto:Token: None 2022-09-03T06:44:11.435 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/ 2022-09-03T06:44:11.435 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg= 2022-09-03T06:44:11.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg=', 'Content-Length': '0'} 2022-09-03T06:44:11.439 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000432ff97c788a1f00-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.440 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:11.441 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:11.442 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:11.442 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:11.443 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:11.443 DEBUG:boto:Method: GET 2022-09-03T06:44:11.444 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:44:11.444 DEBUG:boto:Data: 2022-09-03T06:44:11.445 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.446 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.446 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.447 DEBUG:boto:Params: {} 2022-09-03T06:44:11.448 DEBUG:boto:Token: None 2022-09-03T06:44:11.448 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/ 2022-09-03T06:44:11.449 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg= 2022-09-03T06:44:11.450 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg=', 'Content-Length': '0'} 2022-09-03T06:44:11.453 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bfc70bf04aa856ec-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.454 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:11.455 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:44:11.456 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:11.456 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:11.457 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.457 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.458 DEBUG:boto:Method: GET 2022-09-03T06:44:11.459 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.459 DEBUG:boto:Data: 2022-09-03T06:44:11.460 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.461 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.461 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.462 DEBUG:boto:Params: {} 2022-09-03T06:44:11.462 DEBUG:boto:Token: None 2022-09-03T06:44:11.463 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.464 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oflVcvyTd4KUAU/OnWGK8VKckG4= 2022-09-03T06:44:11.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oflVcvyTd4KUAU/OnWGK8VKckG4=', 'Content-Length': '0'} 2022-09-03T06:44:11.466 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dcca66ab650ee309-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.467 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:11.468 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:11.468 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.469 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.469 DEBUG:boto:Method: GET 2022-09-03T06:44:11.470 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.471 DEBUG:boto:Data: 2022-09-03T06:44:11.471 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.472 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.472 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.473 DEBUG:boto:Params: {} 2022-09-03T06:44:11.474 DEBUG:boto:Token: None 2022-09-03T06:44:11.474 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.475 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oflVcvyTd4KUAU/OnWGK8VKckG4= 2022-09-03T06:44:11.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oflVcvyTd4KUAU/OnWGK8VKckG4=', 'Content-Length': '0'} 2022-09-03T06:44:11.478 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000009bc7e9ec54800f3-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.479 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:11.479 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:11.480 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.480 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.481 DEBUG:boto:Method: HEAD 2022-09-03T06:44:11.481 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.482 DEBUG:boto:Data: 2022-09-03T06:44:11.483 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.483 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.484 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.484 DEBUG:boto:Params: {} 2022-09-03T06:44:11.485 DEBUG:boto:Token: None 2022-09-03T06:44:11.485 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.486 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rWF4RfK2oW3rhIwDQl25iTkhUpM= 2022-09-03T06:44:11.487 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rWF4RfK2oW3rhIwDQl25iTkhUpM=', 'Content-Length': '0'} 2022-09-03T06:44:11.489 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a27176c88497a345-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.489 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:11.490 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:11.490 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.491 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.491 DEBUG:boto:Method: HEAD 2022-09-03T06:44:11.492 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.493 DEBUG:boto:Data: 2022-09-03T06:44:11.493 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.494 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.494 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.495 DEBUG:boto:Params: {} 2022-09-03T06:44:11.495 DEBUG:boto:Token: None 2022-09-03T06:44:11.496 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.497 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rWF4RfK2oW3rhIwDQl25iTkhUpM= 2022-09-03T06:44:11.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rWF4RfK2oW3rhIwDQl25iTkhUpM=', 'Content-Length': '0'} 2022-09-03T06:44:11.499 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fea2ef8a7bb0f4dc-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.500 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:44:11.501 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:11.501 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:11.502 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.503 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.503 DEBUG:boto:Method: GET 2022-09-03T06:44:11.504 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.504 DEBUG:boto:Data: 2022-09-03T06:44:11.505 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.505 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.506 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.507 DEBUG:boto:Params: {} 2022-09-03T06:44:11.507 DEBUG:boto:Token: None 2022-09-03T06:44:11.508 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.508 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oflVcvyTd4KUAU/OnWGK8VKckG4= 2022-09-03T06:44:11.509 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oflVcvyTd4KUAU/OnWGK8VKckG4=', 'Content-Length': '0'} 2022-09-03T06:44:11.511 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008710abc012b73b95-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.558 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:11.559 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:11.560 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.560 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.561 DEBUG:boto:Method: GET 2022-09-03T06:44:11.562 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.562 DEBUG:boto:Data: 2022-09-03T06:44:11.563 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.564 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.564 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.565 DEBUG:boto:Params: {} 2022-09-03T06:44:11.566 DEBUG:boto:Token: None 2022-09-03T06:44:11.567 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:11.568 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oflVcvyTd4KUAU/OnWGK8VKckG4= 2022-09-03T06:44:11.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oflVcvyTd4KUAU/OnWGK8VKckG4=', 'Content-Length': '0'} 2022-09-03T06:44:11.571 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fbd7c965b8733df2-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.572 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:11.573 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:11.574 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:11.575 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:11.576 DEBUG:boto:Method: GET 2022-09-03T06:44:11.576 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:44:11.577 DEBUG:boto:Data: 2022-09-03T06:44:11.578 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.579 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.579 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.580 DEBUG:boto:Params: {} 2022-09-03T06:44:11.580 DEBUG:boto:Token: None 2022-09-03T06:44:11.581 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/ 2022-09-03T06:44:11.582 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg= 2022-09-03T06:44:11.583 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg=', 'Content-Length': '0'} 2022-09-03T06:44:11.586 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008a1d30e55c77ec0d-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.587 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:11.588 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:11.588 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:11.589 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:11.589 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:11.590 DEBUG:boto:Method: GET 2022-09-03T06:44:11.591 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:44:11.591 DEBUG:boto:Data: 2022-09-03T06:44:11.592 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.593 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.594 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.594 DEBUG:boto:Params: {} 2022-09-03T06:44:11.595 DEBUG:boto:Token: None 2022-09-03T06:44:11.596 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/ 2022-09-03T06:44:11.597 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg= 2022-09-03T06:44:11.597 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg=', 'Content-Length': '0'} 2022-09-03T06:44:11.601 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cc769a5d4b499818-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.642 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:11.644 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:11.644 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:11.645 DEBUG:boto:Method: HEAD 2022-09-03T06:44:11.646 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:44:11.646 DEBUG:boto:Data: 2022-09-03T06:44:11.647 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.648 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.648 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.649 DEBUG:boto:Params: {} 2022-09-03T06:44:11.650 DEBUG:boto:Token: None 2022-09-03T06:44:11.650 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj76 2022-09-03T06:44:11.651 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IlADMc9xV19r9ESghxQ9om+UXiU= 2022-09-03T06:44:11.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IlADMc9xV19r9ESghxQ9om+UXiU=', 'Content-Length': '0'} 2022-09-03T06:44:11.654 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000606820b7a6d906ac-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.655 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:11.656 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:11.656 DEBUG:boto:Method: HEAD 2022-09-03T06:44:11.657 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:44:11.658 DEBUG:boto:Data: 2022-09-03T06:44:11.658 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.659 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.659 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.660 DEBUG:boto:Params: {} 2022-09-03T06:44:11.661 DEBUG:boto:Token: None 2022-09-03T06:44:11.661 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj76 2022-09-03T06:44:11.662 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IlADMc9xV19r9ESghxQ9om+UXiU= 2022-09-03T06:44:11.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IlADMc9xV19r9ESghxQ9om+UXiU=', 'Content-Length': '0'} 2022-09-03T06:44:11.665 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f65f0bc6da840b5f-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.666 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:44:11.667 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:11.667 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:11.668 DEBUG:boto:Method: GET 2022-09-03T06:44:11.669 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:44:11.669 DEBUG:boto:Data: 2022-09-03T06:44:11.670 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.670 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.671 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.672 DEBUG:boto:Params: {} 2022-09-03T06:44:11.672 DEBUG:boto:Token: None 2022-09-03T06:44:11.673 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj76 2022-09-03T06:44:11.674 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EFMjYNDBqgAfPd2kcvGx8GrinS8= 2022-09-03T06:44:11.675 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EFMjYNDBqgAfPd2kcvGx8GrinS8=', 'Content-Length': '0'} 2022-09-03T06:44:11.677 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca7ed156edcb6858-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.678 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:11.678 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:11.679 DEBUG:boto:Method: GET 2022-09-03T06:44:11.679 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:44:11.680 DEBUG:boto:Data: 2022-09-03T06:44:11.681 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.681 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.682 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.682 DEBUG:boto:Params: {} 2022-09-03T06:44:11.683 DEBUG:boto:Token: None 2022-09-03T06:44:11.684 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj76 2022-09-03T06:44:11.684 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EFMjYNDBqgAfPd2kcvGx8GrinS8= 2022-09-03T06:44:11.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EFMjYNDBqgAfPd2kcvGx8GrinS8=', 'Content-Length': '0'} 2022-09-03T06:44:11.687 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006aa2fbea3a074543-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.688 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:11.689 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:11.690 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:11.690 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:11.691 DEBUG:boto:Method: GET 2022-09-03T06:44:11.692 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:44:11.693 DEBUG:boto:Data: 2022-09-03T06:44:11.694 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.695 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.696 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.696 DEBUG:boto:Params: {} 2022-09-03T06:44:11.697 DEBUG:boto:Token: None 2022-09-03T06:44:11.698 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/ 2022-09-03T06:44:11.699 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg= 2022-09-03T06:44:11.699 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg=', 'Content-Length': '0'} 2022-09-03T06:44:11.702 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005a38aa57938d66ee-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.703 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:11.705 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:11.706 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:11.707 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:11.708 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:11.709 DEBUG:boto:Method: GET 2022-09-03T06:44:11.709 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:44:11.710 DEBUG:boto:Data: 2022-09-03T06:44:11.711 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.711 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.712 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.713 DEBUG:boto:Params: {} 2022-09-03T06:44:11.713 DEBUG:boto:Token: None 2022-09-03T06:44:11.714 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/ 2022-09-03T06:44:11.715 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg= 2022-09-03T06:44:11.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg=', 'Content-Length': '0'} 2022-09-03T06:44:11.719 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000060947c2fe679df89-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.720 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:11.721 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:44:11.722 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:11.722 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:11.723 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.724 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.724 DEBUG:boto:Method: GET 2022-09-03T06:44:11.725 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.725 DEBUG:boto:Data: 2022-09-03T06:44:11.726 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.726 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.727 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.727 DEBUG:boto:Params: {} 2022-09-03T06:44:11.728 DEBUG:boto:Token: None 2022-09-03T06:44:11.728 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.729 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/TNTZog3htxSriOHbxV0h/pymZY= 2022-09-03T06:44:11.730 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/TNTZog3htxSriOHbxV0h/pymZY=', 'Content-Length': '0'} 2022-09-03T06:44:11.732 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f15ae09c4538cc71-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.774 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:11.775 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:11.775 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.776 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.777 DEBUG:boto:Method: GET 2022-09-03T06:44:11.777 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.778 DEBUG:boto:Data: 2022-09-03T06:44:11.779 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.780 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.780 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.781 DEBUG:boto:Params: {} 2022-09-03T06:44:11.782 DEBUG:boto:Token: None 2022-09-03T06:44:11.782 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.783 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/TNTZog3htxSriOHbxV0h/pymZY= 2022-09-03T06:44:11.784 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/TNTZog3htxSriOHbxV0h/pymZY=', 'Content-Length': '0'} 2022-09-03T06:44:11.787 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd63d42f547273af-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.788 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:11.789 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:11.789 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.790 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.790 DEBUG:boto:Method: HEAD 2022-09-03T06:44:11.791 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.791 DEBUG:boto:Data: 2022-09-03T06:44:11.792 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.792 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.793 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.793 DEBUG:boto:Params: {} 2022-09-03T06:44:11.794 DEBUG:boto:Token: None 2022-09-03T06:44:11.795 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.795 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aAmjswEVjTgpEbFNg4/skEhxz/A= 2022-09-03T06:44:11.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aAmjswEVjTgpEbFNg4/skEhxz/A=', 'Content-Length': '0'} 2022-09-03T06:44:11.798 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c4831b28e4ceed43-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.799 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:11.800 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:11.800 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.801 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.802 DEBUG:boto:Method: HEAD 2022-09-03T06:44:11.802 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.803 DEBUG:boto:Data: 2022-09-03T06:44:11.803 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.804 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.805 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.805 DEBUG:boto:Params: {} 2022-09-03T06:44:11.806 DEBUG:boto:Token: None 2022-09-03T06:44:11.807 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.807 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aAmjswEVjTgpEbFNg4/skEhxz/A= 2022-09-03T06:44:11.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aAmjswEVjTgpEbFNg4/skEhxz/A=', 'Content-Length': '0'} 2022-09-03T06:44:11.810 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000782ea1499a04ff43-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.811 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:44:11.812 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:11.813 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:11.814 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.815 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.817 DEBUG:boto:Method: GET 2022-09-03T06:44:11.818 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.819 DEBUG:boto:Data: 2022-09-03T06:44:11.820 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.821 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.822 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.823 DEBUG:boto:Params: {} 2022-09-03T06:44:11.823 DEBUG:boto:Token: None 2022-09-03T06:44:11.824 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.825 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/TNTZog3htxSriOHbxV0h/pymZY= 2022-09-03T06:44:11.826 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/TNTZog3htxSriOHbxV0h/pymZY=', 'Content-Length': '0'} 2022-09-03T06:44:11.828 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000359dd063c1422333-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.829 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:11.829 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:11.830 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.830 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.831 DEBUG:boto:Method: GET 2022-09-03T06:44:11.832 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.832 DEBUG:boto:Data: 2022-09-03T06:44:11.833 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.833 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.834 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.835 DEBUG:boto:Params: {} 2022-09-03T06:44:11.835 DEBUG:boto:Token: None 2022-09-03T06:44:11.836 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:11.837 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/TNTZog3htxSriOHbxV0h/pymZY= 2022-09-03T06:44:11.837 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/TNTZog3htxSriOHbxV0h/pymZY=', 'Content-Length': '0'} 2022-09-03T06:44:11.839 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001086360d7474db00-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.882 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:11.883 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:11.884 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:11.884 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:11.885 DEBUG:boto:Method: GET 2022-09-03T06:44:11.886 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:44:11.886 DEBUG:boto:Data: 2022-09-03T06:44:11.887 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.887 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.888 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.888 DEBUG:boto:Params: {} 2022-09-03T06:44:11.889 DEBUG:boto:Token: None 2022-09-03T06:44:11.890 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/ 2022-09-03T06:44:11.890 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg= 2022-09-03T06:44:11.891 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg=', 'Content-Length': '0'} 2022-09-03T06:44:11.894 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000027f176e79ba581af-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.895 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:11.896 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:11.896 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:11.897 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:11.897 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:11.898 DEBUG:boto:Method: GET 2022-09-03T06:44:11.899 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:44:11.900 DEBUG:boto:Data: 2022-09-03T06:44:11.901 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.902 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.903 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.904 DEBUG:boto:Params: {} 2022-09-03T06:44:11.905 DEBUG:boto:Token: None 2022-09-03T06:44:11.907 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/ 2022-09-03T06:44:11.908 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg= 2022-09-03T06:44:11.909 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Aa8DNBNg4a1vr3G5q8UAEUYhJZg=', 'Content-Length': '0'} 2022-09-03T06:44:11.913 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d537cf33312caa95-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.914 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:11.915 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:11.915 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:11.916 DEBUG:boto:Method: HEAD 2022-09-03T06:44:11.916 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:44:11.917 DEBUG:boto:Data: 2022-09-03T06:44:11.917 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.918 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.918 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.919 DEBUG:boto:Params: {} 2022-09-03T06:44:11.919 DEBUG:boto:Token: None 2022-09-03T06:44:11.920 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj85 2022-09-03T06:44:11.921 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pyafRJr61+sD+Oca0L8cEkXdkrI= 2022-09-03T06:44:11.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pyafRJr61+sD+Oca0L8cEkXdkrI=', 'Content-Length': '0'} 2022-09-03T06:44:11.923 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f1b0828d2febf00-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.924 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:11.925 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:11.925 DEBUG:boto:Method: HEAD 2022-09-03T06:44:11.926 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:44:11.926 DEBUG:boto:Data: 2022-09-03T06:44:11.927 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.927 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.928 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.928 DEBUG:boto:Params: {} 2022-09-03T06:44:11.929 DEBUG:boto:Token: None 2022-09-03T06:44:11.929 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj85 2022-09-03T06:44:11.930 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pyafRJr61+sD+Oca0L8cEkXdkrI= 2022-09-03T06:44:11.931 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pyafRJr61+sD+Oca0L8cEkXdkrI=', 'Content-Length': '0'} 2022-09-03T06:44:11.933 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000665b377043be079b-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.933 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:44:11.934 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:11.935 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:11.935 DEBUG:boto:Method: GET 2022-09-03T06:44:11.936 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:44:11.936 DEBUG:boto:Data: 2022-09-03T06:44:11.937 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.937 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:11.938 DEBUG:boto:Port: 8001 2022-09-03T06:44:11.938 DEBUG:boto:Params: {} 2022-09-03T06:44:11.939 DEBUG:boto:Token: None 2022-09-03T06:44:11.940 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj85 2022-09-03T06:44:11.940 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lsoDu8xlOHQrsYWEK2Vbxk/Q0oU= 2022-09-03T06:44:11.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lsoDu8xlOHQrsYWEK2Vbxk/Q0oU=', 'Content-Length': '0'} 2022-09-03T06:44:11.943 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ff5080b7ff273535-006312f7bb-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:11.991 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:11.992 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:11.992 DEBUG:boto:Method: GET 2022-09-03T06:44:11.993 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:44:11.994 DEBUG:boto:Data: 2022-09-03T06:44:11.994 DEBUG:boto:Headers: {} 2022-09-03T06:44:11.995 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:11.995 DEBUG:boto:Port: 8000 2022-09-03T06:44:11.996 DEBUG:boto:Params: {} 2022-09-03T06:44:11.996 DEBUG:boto:Token: None 2022-09-03T06:44:11.997 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:11 GMT /dsgvcy-75/obj85 2022-09-03T06:44:11.998 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lsoDu8xlOHQrsYWEK2Vbxk/Q0oU= 2022-09-03T06:44:11.998 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lsoDu8xlOHQrsYWEK2Vbxk/Q0oU=', 'Content-Length': '0'} 2022-09-03T06:44:12.001 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007504d9124e7cf71f-006312f7bb-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.002 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:12.002 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:12.003 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:12.003 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:12.004 DEBUG:boto:Method: GET 2022-09-03T06:44:12.004 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:44:12.005 DEBUG:boto:Data: 2022-09-03T06:44:12.008 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.009 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.010 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.010 DEBUG:boto:Params: {} 2022-09-03T06:44:12.011 DEBUG:boto:Token: None 2022-09-03T06:44:12.011 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/ 2022-09-03T06:44:12.012 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE= 2022-09-03T06:44:12.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE=', 'Content-Length': '0'} 2022-09-03T06:44:12.016 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006ee77e0004662600-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.017 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:12.017 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:12.018 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:12.019 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:12.019 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:12.020 DEBUG:boto:Method: GET 2022-09-03T06:44:12.020 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:44:12.021 DEBUG:boto:Data: 2022-09-03T06:44:12.021 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.022 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:12.023 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.023 DEBUG:boto:Params: {} 2022-09-03T06:44:12.024 DEBUG:boto:Token: None 2022-09-03T06:44:12.024 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/ 2022-09-03T06:44:12.025 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE= 2022-09-03T06:44:12.026 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE=', 'Content-Length': '0'} 2022-09-03T06:44:12.029 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d7923950a39c7206-006312f7bc-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.030 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:12.031 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:44:12.032 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:12.033 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:12.034 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.034 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.035 DEBUG:boto:Method: GET 2022-09-03T06:44:12.037 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.038 DEBUG:boto:Data: 2022-09-03T06:44:12.039 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.039 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.040 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.041 DEBUG:boto:Params: {} 2022-09-03T06:44:12.042 DEBUG:boto:Token: None 2022-09-03T06:44:12.043 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.043 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:io71VIlwoqZO/3BKOxKcUpgeMk4= 2022-09-03T06:44:12.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:io71VIlwoqZO/3BKOxKcUpgeMk4=', 'Content-Length': '0'} 2022-09-03T06:44:12.047 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005a1e81fa295624f2-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.048 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:12.048 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:12.049 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.050 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.050 DEBUG:boto:Method: GET 2022-09-03T06:44:12.051 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.052 DEBUG:boto:Data: 2022-09-03T06:44:12.052 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.053 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:12.054 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.054 DEBUG:boto:Params: {} 2022-09-03T06:44:12.055 DEBUG:boto:Token: None 2022-09-03T06:44:12.056 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.056 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:io71VIlwoqZO/3BKOxKcUpgeMk4= 2022-09-03T06:44:12.057 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:io71VIlwoqZO/3BKOxKcUpgeMk4=', 'Content-Length': '0'} 2022-09-03T06:44:12.060 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002c1684e24482bfd8-006312f7bc-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.061 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:12.062 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:12.063 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.063 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.064 DEBUG:boto:Method: HEAD 2022-09-03T06:44:12.064 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.065 DEBUG:boto:Data: 2022-09-03T06:44:12.066 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.066 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.067 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.067 DEBUG:boto:Params: {} 2022-09-03T06:44:12.068 DEBUG:boto:Token: None 2022-09-03T06:44:12.069 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.070 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A8fdNyPbXTx63tHVN3dYrGpvMVs= 2022-09-03T06:44:12.070 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A8fdNyPbXTx63tHVN3dYrGpvMVs=', 'Content-Length': '0'} 2022-09-03T06:44:12.072 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000009ae247969010333-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.073 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:12.074 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:12.075 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.076 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.076 DEBUG:boto:Method: HEAD 2022-09-03T06:44:12.077 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.077 DEBUG:boto:Data: 2022-09-03T06:44:12.078 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.079 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:12.079 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.080 DEBUG:boto:Params: {} 2022-09-03T06:44:12.081 DEBUG:boto:Token: None 2022-09-03T06:44:12.082 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.082 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A8fdNyPbXTx63tHVN3dYrGpvMVs= 2022-09-03T06:44:12.083 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A8fdNyPbXTx63tHVN3dYrGpvMVs=', 'Content-Length': '0'} 2022-09-03T06:44:12.085 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c0c62dad22ac113b-006312f7bc-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.086 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:44:12.087 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:12.088 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:12.088 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.089 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.090 DEBUG:boto:Method: GET 2022-09-03T06:44:12.091 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.092 DEBUG:boto:Data: 2022-09-03T06:44:12.092 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.093 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.093 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.094 DEBUG:boto:Params: {} 2022-09-03T06:44:12.095 DEBUG:boto:Token: None 2022-09-03T06:44:12.096 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.097 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:io71VIlwoqZO/3BKOxKcUpgeMk4= 2022-09-03T06:44:12.097 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:io71VIlwoqZO/3BKOxKcUpgeMk4=', 'Content-Length': '0'} 2022-09-03T06:44:12.100 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e8f572c070af6e3c-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.146 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:12.147 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:12.147 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.148 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.149 DEBUG:boto:Method: GET 2022-09-03T06:44:12.149 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.150 DEBUG:boto:Data: 2022-09-03T06:44:12.150 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.151 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:12.152 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.152 DEBUG:boto:Params: {} 2022-09-03T06:44:12.153 DEBUG:boto:Token: None 2022-09-03T06:44:12.154 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:12.154 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:io71VIlwoqZO/3BKOxKcUpgeMk4= 2022-09-03T06:44:12.155 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:io71VIlwoqZO/3BKOxKcUpgeMk4=', 'Content-Length': '0'} 2022-09-03T06:44:12.157 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000078f87d2a1fe43c0-006312f7bc-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.158 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:12.159 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:12.160 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:12.160 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:12.161 DEBUG:boto:Method: GET 2022-09-03T06:44:12.161 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:44:12.162 DEBUG:boto:Data: 2022-09-03T06:44:12.162 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.163 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.164 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.164 DEBUG:boto:Params: {} 2022-09-03T06:44:12.165 DEBUG:boto:Token: None 2022-09-03T06:44:12.165 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/ 2022-09-03T06:44:12.166 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE= 2022-09-03T06:44:12.167 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE=', 'Content-Length': '0'} 2022-09-03T06:44:12.170 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008f84adb7e141bdf1-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.171 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:12.171 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:12.172 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:12.173 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:12.173 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:12.174 DEBUG:boto:Method: GET 2022-09-03T06:44:12.174 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:44:12.175 DEBUG:boto:Data: 2022-09-03T06:44:12.176 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.177 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:12.177 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.178 DEBUG:boto:Params: {} 2022-09-03T06:44:12.179 DEBUG:boto:Token: None 2022-09-03T06:44:12.179 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/ 2022-09-03T06:44:12.180 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE= 2022-09-03T06:44:12.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE=', 'Content-Length': '0'} 2022-09-03T06:44:12.184 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002094fbe7a7f8ec22-006312f7bc-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.185 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:12.186 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:12.187 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:12.187 DEBUG:boto:Method: HEAD 2022-09-03T06:44:12.188 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:44:12.188 DEBUG:boto:Data: 2022-09-03T06:44:12.189 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.189 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.190 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.190 DEBUG:boto:Params: {} 2022-09-03T06:44:12.191 DEBUG:boto:Token: None 2022-09-03T06:44:12.192 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj92 2022-09-03T06:44:12.192 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dZudMCd90l61rofsQPJzq33PjOA= 2022-09-03T06:44:12.193 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dZudMCd90l61rofsQPJzq33PjOA=', 'Content-Length': '0'} 2022-09-03T06:44:12.195 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b8b7c8b770f1961-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.196 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:12.197 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:12.197 DEBUG:boto:Method: HEAD 2022-09-03T06:44:12.198 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:44:12.198 DEBUG:boto:Data: 2022-09-03T06:44:12.199 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.200 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:12.200 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.201 DEBUG:boto:Params: {} 2022-09-03T06:44:12.201 DEBUG:boto:Token: None 2022-09-03T06:44:12.202 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj92 2022-09-03T06:44:12.203 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dZudMCd90l61rofsQPJzq33PjOA= 2022-09-03T06:44:12.203 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dZudMCd90l61rofsQPJzq33PjOA=', 'Content-Length': '0'} 2022-09-03T06:44:12.206 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000802392a2be53d973-006312f7bc-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.206 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:44:12.207 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:12.208 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:12.208 DEBUG:boto:Method: GET 2022-09-03T06:44:12.209 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:44:12.210 DEBUG:boto:Data: 2022-09-03T06:44:12.210 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.211 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.211 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.212 DEBUG:boto:Params: {} 2022-09-03T06:44:12.213 DEBUG:boto:Token: None 2022-09-03T06:44:12.213 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj92 2022-09-03T06:44:12.214 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4prXVKoFJB0coZJo4qMEBx+ed8I= 2022-09-03T06:44:12.215 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4prXVKoFJB0coZJo4qMEBx+ed8I=', 'Content-Length': '0'} 2022-09-03T06:44:12.217 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007a228f007d2c5d2a-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.262 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:12.263 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:12.264 DEBUG:boto:Method: GET 2022-09-03T06:44:12.265 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:44:12.265 DEBUG:boto:Data: 2022-09-03T06:44:12.266 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.266 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:12.267 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.268 DEBUG:boto:Params: {} 2022-09-03T06:44:12.268 DEBUG:boto:Token: None 2022-09-03T06:44:12.269 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj92 2022-09-03T06:44:12.270 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4prXVKoFJB0coZJo4qMEBx+ed8I= 2022-09-03T06:44:12.270 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4prXVKoFJB0coZJo4qMEBx+ed8I=', 'Content-Length': '0'} 2022-09-03T06:44:12.273 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000028a67288ecb9e398-006312f7bc-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.274 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:12.275 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:12.276 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:12.277 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:12.278 DEBUG:boto:Method: GET 2022-09-03T06:44:12.279 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:44:12.279 DEBUG:boto:Data: 2022-09-03T06:44:12.280 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.280 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.281 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.282 DEBUG:boto:Params: {} 2022-09-03T06:44:12.282 DEBUG:boto:Token: None 2022-09-03T06:44:12.283 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/ 2022-09-03T06:44:12.284 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE= 2022-09-03T06:44:12.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE=', 'Content-Length': '0'} 2022-09-03T06:44:12.287 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006490553eed594ef5-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.288 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:12.289 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:12.290 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:12.290 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:12.291 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:12.291 DEBUG:boto:Method: GET 2022-09-03T06:44:12.292 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:44:12.293 DEBUG:boto:Data: 2022-09-03T06:44:12.293 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.294 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:12.295 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.295 DEBUG:boto:Params: {} 2022-09-03T06:44:12.296 DEBUG:boto:Token: None 2022-09-03T06:44:12.296 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/ 2022-09-03T06:44:12.297 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE= 2022-09-03T06:44:12.298 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE=', 'Content-Length': '0'} 2022-09-03T06:44:12.301 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000c7a43222bb21c46-006312f7bc-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.302 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:12.303 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:44:12.304 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:12.304 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:12.305 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.305 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.306 DEBUG:boto:Method: GET 2022-09-03T06:44:12.306 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.307 DEBUG:boto:Data: 2022-09-03T06:44:12.307 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.308 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.308 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.309 DEBUG:boto:Params: {} 2022-09-03T06:44:12.310 DEBUG:boto:Token: None 2022-09-03T06:44:12.310 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.311 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dSnAAmJqkBfv1rFlSDfC9AvSqEo= 2022-09-03T06:44:12.311 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dSnAAmJqkBfv1rFlSDfC9AvSqEo=', 'Content-Length': '0'} 2022-09-03T06:44:12.313 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000b68f5ac820e96b7-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.314 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:12.315 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:12.315 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.316 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.316 DEBUG:boto:Method: GET 2022-09-03T06:44:12.317 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.317 DEBUG:boto:Data: 2022-09-03T06:44:12.318 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.318 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:12.319 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.319 DEBUG:boto:Params: {} 2022-09-03T06:44:12.320 DEBUG:boto:Token: None 2022-09-03T06:44:12.321 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.321 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dSnAAmJqkBfv1rFlSDfC9AvSqEo= 2022-09-03T06:44:12.322 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dSnAAmJqkBfv1rFlSDfC9AvSqEo=', 'Content-Length': '0'} 2022-09-03T06:44:12.324 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a51474f72fc6eaea-006312f7bc-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.325 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:12.326 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:12.326 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.327 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.327 DEBUG:boto:Method: HEAD 2022-09-03T06:44:12.328 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.328 DEBUG:boto:Data: 2022-09-03T06:44:12.329 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.329 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.330 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.330 DEBUG:boto:Params: {} 2022-09-03T06:44:12.331 DEBUG:boto:Token: None 2022-09-03T06:44:12.331 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.332 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r3wz+Ed/NXjsAT/ZU248kC599ME= 2022-09-03T06:44:12.332 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r3wz+Ed/NXjsAT/ZU248kC599ME=', 'Content-Length': '0'} 2022-09-03T06:44:12.334 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000097f68c4d17125cac-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.335 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:12.336 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:12.336 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.337 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.337 DEBUG:boto:Method: HEAD 2022-09-03T06:44:12.338 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.338 DEBUG:boto:Data: 2022-09-03T06:44:12.339 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.339 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:12.340 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.340 DEBUG:boto:Params: {} 2022-09-03T06:44:12.341 DEBUG:boto:Token: None 2022-09-03T06:44:12.341 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.342 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r3wz+Ed/NXjsAT/ZU248kC599ME= 2022-09-03T06:44:12.343 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r3wz+Ed/NXjsAT/ZU248kC599ME=', 'Content-Length': '0'} 2022-09-03T06:44:12.345 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c6ebe877b829b592-006312f7bc-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.347 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:44:12.347 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:12.348 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:12.349 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.350 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.351 DEBUG:boto:Method: GET 2022-09-03T06:44:12.351 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.352 DEBUG:boto:Data: 2022-09-03T06:44:12.353 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.353 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.354 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.354 DEBUG:boto:Params: {} 2022-09-03T06:44:12.355 DEBUG:boto:Token: None 2022-09-03T06:44:12.356 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.356 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dSnAAmJqkBfv1rFlSDfC9AvSqEo= 2022-09-03T06:44:12.357 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dSnAAmJqkBfv1rFlSDfC9AvSqEo=', 'Content-Length': '0'} 2022-09-03T06:44:12.359 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065c9d67935199aa0-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.402 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:12.403 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:12.403 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.404 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.405 DEBUG:boto:Method: GET 2022-09-03T06:44:12.405 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.406 DEBUG:boto:Data: 2022-09-03T06:44:12.406 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.407 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:12.407 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.408 DEBUG:boto:Params: {} 2022-09-03T06:44:12.408 DEBUG:boto:Token: None 2022-09-03T06:44:12.409 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:12.410 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dSnAAmJqkBfv1rFlSDfC9AvSqEo= 2022-09-03T06:44:12.410 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dSnAAmJqkBfv1rFlSDfC9AvSqEo=', 'Content-Length': '0'} 2022-09-03T06:44:12.413 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007061cc9a07f9ae08-006312f7bc-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.414 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:12.414 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:12.415 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:12.415 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:12.416 DEBUG:boto:Method: GET 2022-09-03T06:44:12.417 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:44:12.417 DEBUG:boto:Data: 2022-09-03T06:44:12.418 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.418 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.419 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.419 DEBUG:boto:Params: {} 2022-09-03T06:44:12.420 DEBUG:boto:Token: None 2022-09-03T06:44:12.420 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/ 2022-09-03T06:44:12.421 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE= 2022-09-03T06:44:12.422 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE=', 'Content-Length': '0'} 2022-09-03T06:44:12.424 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c727c6976c224b10-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.425 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:12.426 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:12.426 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:12.427 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:12.427 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:12.428 DEBUG:boto:Method: GET 2022-09-03T06:44:12.428 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:44:12.429 DEBUG:boto:Data: 2022-09-03T06:44:12.429 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.430 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:12.430 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.431 DEBUG:boto:Params: {} 2022-09-03T06:44:12.432 DEBUG:boto:Token: None 2022-09-03T06:44:12.432 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/ 2022-09-03T06:44:12.433 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE= 2022-09-03T06:44:12.433 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE=', 'Content-Length': '0'} 2022-09-03T06:44:12.437 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002674aabbd117f770-006312f7bc-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.438 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:12.439 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:12.439 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:12.440 DEBUG:boto:Method: HEAD 2022-09-03T06:44:12.440 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:44:12.441 DEBUG:boto:Data: 2022-09-03T06:44:12.441 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.442 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.442 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.443 DEBUG:boto:Params: {} 2022-09-03T06:44:12.444 DEBUG:boto:Token: None 2022-09-03T06:44:12.444 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj95 2022-09-03T06:44:12.445 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:x/0zfLSyVwnBoN6F35YRf0U4PLA= 2022-09-03T06:44:12.445 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:x/0zfLSyVwnBoN6F35YRf0U4PLA=', 'Content-Length': '0'} 2022-09-03T06:44:12.447 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f597f386bf8e4813-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.448 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:12.449 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:12.449 DEBUG:boto:Method: HEAD 2022-09-03T06:44:12.450 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:44:12.450 DEBUG:boto:Data: 2022-09-03T06:44:12.451 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.451 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:12.452 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.452 DEBUG:boto:Params: {} 2022-09-03T06:44:12.453 DEBUG:boto:Token: None 2022-09-03T06:44:12.453 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj95 2022-09-03T06:44:12.454 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:x/0zfLSyVwnBoN6F35YRf0U4PLA= 2022-09-03T06:44:12.455 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:x/0zfLSyVwnBoN6F35YRf0U4PLA=', 'Content-Length': '0'} 2022-09-03T06:44:12.457 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009460ade9dcf36761-006312f7bc-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.458 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:44:12.458 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:12.459 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:12.460 DEBUG:boto:Method: GET 2022-09-03T06:44:12.460 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:44:12.461 DEBUG:boto:Data: 2022-09-03T06:44:12.461 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.462 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.462 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.463 DEBUG:boto:Params: {} 2022-09-03T06:44:12.464 DEBUG:boto:Token: None 2022-09-03T06:44:12.464 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj95 2022-09-03T06:44:12.465 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ux9pP9ERJTBaEvnduNRm99yk8po= 2022-09-03T06:44:12.465 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ux9pP9ERJTBaEvnduNRm99yk8po=', 'Content-Length': '0'} 2022-09-03T06:44:12.468 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c7d8141356ca2c7-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.468 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:12.469 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:12.469 DEBUG:boto:Method: GET 2022-09-03T06:44:12.470 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:44:12.471 DEBUG:boto:Data: 2022-09-03T06:44:12.471 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.472 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:12.473 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.473 DEBUG:boto:Params: {} 2022-09-03T06:44:12.474 DEBUG:boto:Token: None 2022-09-03T06:44:12.475 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj95 2022-09-03T06:44:12.475 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ux9pP9ERJTBaEvnduNRm99yk8po= 2022-09-03T06:44:12.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ux9pP9ERJTBaEvnduNRm99yk8po=', 'Content-Length': '0'} 2022-09-03T06:44:12.478 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006927f49d57bac07c-006312f7bc-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.479 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:12.479 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:12.480 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:12.480 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:12.481 DEBUG:boto:Method: GET 2022-09-03T06:44:12.481 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:44:12.482 DEBUG:boto:Data: 2022-09-03T06:44:12.483 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.483 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.484 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.484 DEBUG:boto:Params: {} 2022-09-03T06:44:12.485 DEBUG:boto:Token: None 2022-09-03T06:44:12.485 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/ 2022-09-03T06:44:12.486 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE= 2022-09-03T06:44:12.487 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE=', 'Content-Length': '0'} 2022-09-03T06:44:12.489 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c891fb70ade09e3d-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.490 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:12.491 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:12.492 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:12.492 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:12.493 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:12.494 DEBUG:boto:Method: GET 2022-09-03T06:44:12.494 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:44:12.495 DEBUG:boto:Data: 2022-09-03T06:44:12.495 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.496 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:12.496 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.497 DEBUG:boto:Params: {} 2022-09-03T06:44:12.497 DEBUG:boto:Token: None 2022-09-03T06:44:12.498 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/ 2022-09-03T06:44:12.499 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE= 2022-09-03T06:44:12.499 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE=', 'Content-Length': '0'} 2022-09-03T06:44:12.503 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f605f26528c5d055-006312f7bc-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.550 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:12.552 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-75 zones={test-zone3, test-zone1} 2022-09-03T06:44:12.553 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-75 zones={test-zone3, test-zone2} 2022-09-03T06:44:12.553 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:12.554 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:12.556 DEBUG:boto:Method: HEAD 2022-09-03T06:44:12.556 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:44:12.557 DEBUG:boto:Data: 2022-09-03T06:44:12.558 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.559 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.559 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.560 DEBUG:boto:Params: {} 2022-09-03T06:44:12.561 DEBUG:boto:Token: None 2022-09-03T06:44:12.562 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/ 2022-09-03T06:44:12.562 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6WtioHzG9QIDX8glxPr7Gt1t2Tk= 2022-09-03T06:44:12.563 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6WtioHzG9QIDX8glxPr7Gt1t2Tk=', 'Content-Length': '0'} 2022-09-03T06:44:12.566 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '20'), ('X-RGW-Bytes-Used', '60'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006fc71152ad69f43e-006312f7bc-1851-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.567 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:12.568 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:12.569 DEBUG:boto:Method: HEAD 2022-09-03T06:44:12.569 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:44:12.570 DEBUG:boto:Data: 2022-09-03T06:44:12.570 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.571 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:12.571 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.572 DEBUG:boto:Params: {} 2022-09-03T06:44:12.573 DEBUG:boto:Token: None 2022-09-03T06:44:12.573 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/ 2022-09-03T06:44:12.574 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6WtioHzG9QIDX8glxPr7Gt1t2Tk= 2022-09-03T06:44:12.575 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6WtioHzG9QIDX8glxPr7Gt1t2Tk=', 'Content-Length': '0'} 2022-09-03T06:44:12.577 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '20'), ('X-RGW-Bytes-Used', '60'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000eaa0ea502a6e6ded-006312f7bc-188c-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.578 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:44:12.579 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:12.579 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:12.580 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:12.580 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:12.581 DEBUG:boto:Method: GET 2022-09-03T06:44:12.581 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:12.582 DEBUG:boto:Data: 2022-09-03T06:44:12.583 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.583 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.584 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.584 DEBUG:boto:Params: {} 2022-09-03T06:44:12.585 DEBUG:boto:Token: None 2022-09-03T06:44:12.586 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/?versions 2022-09-03T06:44:12.586 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E2L8YwMcOi9zAhOiGVdWGLNffgA= 2022-09-03T06:44:12.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E2L8YwMcOi9zAhOiGVdWGLNffgA=', 'Content-Length': '0'} 2022-09-03T06:44:12.590 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bc6b7832f595a633-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.591 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:12.593 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:44:12.593 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:44:12.594 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:44:12.594 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:44:12.595 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:44:12.595 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:44:12.596 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:44:12.596 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:44:12.597 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:44:12.598 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:44:12.598 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:44:12.599 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:44:12.600 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:44:12.602 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:44:12.602 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:44:12.603 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:44:12.604 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:44:12.605 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:44:12.606 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:44:12.607 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:44:12.608 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:44:12.608 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:12.609 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:12.610 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:12.611 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:12.611 DEBUG:boto:Method: GET 2022-09-03T06:44:12.612 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:12.612 DEBUG:boto:Data: 2022-09-03T06:44:12.613 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.613 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:12.614 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.614 DEBUG:boto:Params: {} 2022-09-03T06:44:12.615 DEBUG:boto:Token: None 2022-09-03T06:44:12.615 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/?versions 2022-09-03T06:44:12.616 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E2L8YwMcOi9zAhOiGVdWGLNffgA= 2022-09-03T06:44:12.617 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E2L8YwMcOi9zAhOiGVdWGLNffgA=', 'Content-Length': '0'} 2022-09-03T06:44:12.620 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000096422d0b4c304d7e-006312f7bc-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.621 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:12.622 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:44:12.623 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:44:12.623 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:44:12.624 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:44:12.624 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:44:12.625 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:44:12.625 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:44:12.626 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:44:12.626 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:44:12.627 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:44:12.627 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:44:12.628 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:44:12.629 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:44:12.629 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:44:12.630 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:44:12.630 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:44:12.631 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:44:12.632 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:44:12.633 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:44:12.634 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:44:12.635 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:12.635 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:12.636 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:12.637 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:12.638 DEBUG:boto:Method: GET 2022-09-03T06:44:12.639 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:12.640 DEBUG:boto:Data: 2022-09-03T06:44:12.641 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.642 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.643 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.644 DEBUG:boto:Params: {} 2022-09-03T06:44:12.644 DEBUG:boto:Token: None 2022-09-03T06:44:12.645 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/?versions 2022-09-03T06:44:12.645 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E2L8YwMcOi9zAhOiGVdWGLNffgA= 2022-09-03T06:44:12.646 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E2L8YwMcOi9zAhOiGVdWGLNffgA=', 'Content-Length': '0'} 2022-09-03T06:44:12.649 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a9557678ae56a336-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.650 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:12.652 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:12.653 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:12.654 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:12.655 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:12.655 DEBUG:boto:Method: GET 2022-09-03T06:44:12.656 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:44:12.657 DEBUG:boto:Data: 2022-09-03T06:44:12.659 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.660 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:12.660 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.661 DEBUG:boto:Params: {} 2022-09-03T06:44:12.662 DEBUG:boto:Token: None 2022-09-03T06:44:12.663 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/?versions 2022-09-03T06:44:12.664 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E2L8YwMcOi9zAhOiGVdWGLNffgA= 2022-09-03T06:44:12.665 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E2L8YwMcOi9zAhOiGVdWGLNffgA=', 'Content-Length': '0'} 2022-09-03T06:44:12.670 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bfc44339dd24eea8-006312f7bc-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.671 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:12.674 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:44:12.675 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:12.675 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:12.676 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.676 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.677 DEBUG:boto:Method: GET 2022-09-03T06:44:12.677 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.678 DEBUG:boto:Data: 2022-09-03T06:44:12.678 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.679 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.680 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.680 DEBUG:boto:Params: {} 2022-09-03T06:44:12.681 DEBUG:boto:Token: None 2022-09-03T06:44:12.682 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.682 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k0moPg8tYvpddYKHGA6cf0O2muw= 2022-09-03T06:44:12.683 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k0moPg8tYvpddYKHGA6cf0O2muw=', 'Content-Length': '0'} 2022-09-03T06:44:12.686 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a8a311b5b135b921-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.687 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:12.687 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:12.688 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.688 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.689 DEBUG:boto:Method: GET 2022-09-03T06:44:12.690 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.690 DEBUG:boto:Data: 2022-09-03T06:44:12.691 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.691 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:12.692 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.692 DEBUG:boto:Params: {} 2022-09-03T06:44:12.693 DEBUG:boto:Token: None 2022-09-03T06:44:12.694 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.694 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k0moPg8tYvpddYKHGA6cf0O2muw= 2022-09-03T06:44:12.695 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k0moPg8tYvpddYKHGA6cf0O2muw=', 'Content-Length': '0'} 2022-09-03T06:44:12.698 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000034842a824ff5ca63-006312f7bc-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.699 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:12.699 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:12.700 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.700 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.701 DEBUG:boto:Method: HEAD 2022-09-03T06:44:12.701 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.702 DEBUG:boto:Data: 2022-09-03T06:44:12.702 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.703 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.703 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.704 DEBUG:boto:Params: {} 2022-09-03T06:44:12.704 DEBUG:boto:Token: None 2022-09-03T06:44:12.705 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.706 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KPEQBl0+WmhJ4HkTgIpgGPiv6fo= 2022-09-03T06:44:12.707 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KPEQBl0+WmhJ4HkTgIpgGPiv6fo=', 'Content-Length': '0'} 2022-09-03T06:44:12.709 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e2ac9f79db9b0e0-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.709 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:12.710 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:12.711 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.711 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.712 DEBUG:boto:Method: HEAD 2022-09-03T06:44:12.713 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.713 DEBUG:boto:Data: 2022-09-03T06:44:12.714 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.714 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:12.715 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.715 DEBUG:boto:Params: {} 2022-09-03T06:44:12.716 DEBUG:boto:Token: None 2022-09-03T06:44:12.716 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.717 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KPEQBl0+WmhJ4HkTgIpgGPiv6fo= 2022-09-03T06:44:12.718 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KPEQBl0+WmhJ4HkTgIpgGPiv6fo=', 'Content-Length': '0'} 2022-09-03T06:44:12.720 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003f9efa753e1f4b05-006312f7bc-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.721 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:44:12.722 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:12.722 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:12.723 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.723 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.724 DEBUG:boto:Method: GET 2022-09-03T06:44:12.725 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.725 DEBUG:boto:Data: 2022-09-03T06:44:12.726 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.726 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.727 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.728 DEBUG:boto:Params: {} 2022-09-03T06:44:12.730 DEBUG:boto:Token: None 2022-09-03T06:44:12.731 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.732 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k0moPg8tYvpddYKHGA6cf0O2muw= 2022-09-03T06:44:12.733 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k0moPg8tYvpddYKHGA6cf0O2muw=', 'Content-Length': '0'} 2022-09-03T06:44:12.736 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e74c4364e295e9d-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.738 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:12.739 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:12.739 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.740 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.740 DEBUG:boto:Method: GET 2022-09-03T06:44:12.741 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.741 DEBUG:boto:Data: 2022-09-03T06:44:12.742 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.742 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:12.743 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.743 DEBUG:boto:Params: {} 2022-09-03T06:44:12.744 DEBUG:boto:Token: None 2022-09-03T06:44:12.744 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:44:12.745 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k0moPg8tYvpddYKHGA6cf0O2muw= 2022-09-03T06:44:12.746 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k0moPg8tYvpddYKHGA6cf0O2muw=', 'Content-Length': '0'} 2022-09-03T06:44:12.748 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000576220d9792e2b99-006312f7bc-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.790 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:12.791 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:12.792 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:12.792 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:12.795 DEBUG:boto:Method: GET 2022-09-03T06:44:12.806 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:44:12.806 DEBUG:boto:Data: 2022-09-03T06:44:12.807 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.808 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.808 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.809 DEBUG:boto:Params: {} 2022-09-03T06:44:12.809 DEBUG:boto:Token: None 2022-09-03T06:44:12.810 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/ 2022-09-03T06:44:12.811 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE= 2022-09-03T06:44:12.811 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE=', 'Content-Length': '0'} 2022-09-03T06:44:12.814 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f86d28672b3e7560-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.815 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:12.815 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:12.816 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:12.817 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:12.817 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:12.818 DEBUG:boto:Method: GET 2022-09-03T06:44:12.818 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:44:12.819 DEBUG:boto:Data: 2022-09-03T06:44:12.819 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.820 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:12.820 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.821 DEBUG:boto:Params: {} 2022-09-03T06:44:12.821 DEBUG:boto:Token: None 2022-09-03T06:44:12.822 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/ 2022-09-03T06:44:12.823 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE= 2022-09-03T06:44:12.823 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE=', 'Content-Length': '0'} 2022-09-03T06:44:12.826 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000046fa7925b06684bd-006312f7bc-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.828 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:12.829 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:12.830 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:12.830 DEBUG:boto:Method: HEAD 2022-09-03T06:44:12.831 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:44:12.832 DEBUG:boto:Data: 2022-09-03T06:44:12.832 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.833 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.833 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.834 DEBUG:boto:Params: {} 2022-09-03T06:44:12.835 DEBUG:boto:Token: None 2022-09-03T06:44:12.835 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj11 2022-09-03T06:44:12.836 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Sy8pT6StpNxT/2o2w2QC4ZrBGxM= 2022-09-03T06:44:12.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Sy8pT6StpNxT/2o2w2QC4ZrBGxM=', 'Content-Length': '0'} 2022-09-03T06:44:12.839 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000023c341e8c12baeaf-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.840 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:12.840 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:12.841 DEBUG:boto:Method: HEAD 2022-09-03T06:44:12.841 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:44:12.842 DEBUG:boto:Data: 2022-09-03T06:44:12.842 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.843 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:12.843 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.844 DEBUG:boto:Params: {} 2022-09-03T06:44:12.844 DEBUG:boto:Token: None 2022-09-03T06:44:12.845 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj11 2022-09-03T06:44:12.846 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Sy8pT6StpNxT/2o2w2QC4ZrBGxM= 2022-09-03T06:44:12.846 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Sy8pT6StpNxT/2o2w2QC4ZrBGxM=', 'Content-Length': '0'} 2022-09-03T06:44:12.849 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bc2d295e013bdd61-006312f7bc-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.850 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:44:12.851 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:12.851 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:12.852 DEBUG:boto:Method: GET 2022-09-03T06:44:12.853 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:44:12.853 DEBUG:boto:Data: 2022-09-03T06:44:12.854 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.854 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.855 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.856 DEBUG:boto:Params: {} 2022-09-03T06:44:12.856 DEBUG:boto:Token: None 2022-09-03T06:44:12.857 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj11 2022-09-03T06:44:12.858 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qErWQmR+PkWiagAPS1u24sMnXgw= 2022-09-03T06:44:12.858 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qErWQmR+PkWiagAPS1u24sMnXgw=', 'Content-Length': '0'} 2022-09-03T06:44:12.861 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e176dccaff70b5e-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.902 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:44:12.903 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:44:12.904 DEBUG:boto:Method: GET 2022-09-03T06:44:12.905 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:44:12.907 DEBUG:boto:Data: 2022-09-03T06:44:12.908 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.908 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:12.909 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.910 DEBUG:boto:Params: {} 2022-09-03T06:44:12.911 DEBUG:boto:Token: None 2022-09-03T06:44:12.912 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj11 2022-09-03T06:44:12.913 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qErWQmR+PkWiagAPS1u24sMnXgw= 2022-09-03T06:44:12.915 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qErWQmR+PkWiagAPS1u24sMnXgw=', 'Content-Length': '0'} 2022-09-03T06:44:12.917 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fa79e631480bb1de-006312f7bc-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.918 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:12.919 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:12.920 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:12.920 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:12.921 DEBUG:boto:Method: GET 2022-09-03T06:44:12.921 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:44:12.922 DEBUG:boto:Data: 2022-09-03T06:44:12.922 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.923 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.923 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.924 DEBUG:boto:Params: {} 2022-09-03T06:44:12.925 DEBUG:boto:Token: None 2022-09-03T06:44:12.925 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/ 2022-09-03T06:44:12.926 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE= 2022-09-03T06:44:12.927 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE=', 'Content-Length': '0'} 2022-09-03T06:44:12.930 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007434da8fa3205b6b-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.931 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:12.931 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:12.932 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:12.933 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:12.933 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:44:12.934 DEBUG:boto:Method: GET 2022-09-03T06:44:12.934 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:44:12.935 DEBUG:boto:Data: 2022-09-03T06:44:12.936 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.936 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:12.937 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.937 DEBUG:boto:Params: {} 2022-09-03T06:44:12.938 DEBUG:boto:Token: None 2022-09-03T06:44:12.939 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/ 2022-09-03T06:44:12.940 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE= 2022-09-03T06:44:12.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lLSVNDIZwwpmWlquzvMVepg5VBE=', 'Content-Length': '0'} 2022-09-03T06:44:12.944 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000047463f5b0cbb3102-006312f7bc-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.945 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:12.946 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:44:12.947 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:12.947 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:12.948 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:12.948 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:12.949 DEBUG:boto:Method: GET 2022-09-03T06:44:12.949 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:12.950 DEBUG:boto:Data: 2022-09-03T06:44:12.951 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.951 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.952 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.952 DEBUG:boto:Params: {} 2022-09-03T06:44:12.953 DEBUG:boto:Token: None 2022-09-03T06:44:12.954 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:12.955 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fZ90wur2U/WoZbNtmJzC5gdnlCI= 2022-09-03T06:44:12.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fZ90wur2U/WoZbNtmJzC5gdnlCI=', 'Content-Length': '0'} 2022-09-03T06:44:12.959 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002c10c433513fc1e2-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.960 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:12.961 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:12.962 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:12.963 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:12.964 DEBUG:boto:Method: GET 2022-09-03T06:44:12.965 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:12.966 DEBUG:boto:Data: 2022-09-03T06:44:12.967 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.968 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:12.969 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.970 DEBUG:boto:Params: {} 2022-09-03T06:44:12.971 DEBUG:boto:Token: None 2022-09-03T06:44:12.972 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:12.973 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fZ90wur2U/WoZbNtmJzC5gdnlCI= 2022-09-03T06:44:12.973 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fZ90wur2U/WoZbNtmJzC5gdnlCI=', 'Content-Length': '0'} 2022-09-03T06:44:12.976 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000260e3b93c89a5d77-006312f7bc-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.976 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:12.977 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:12.977 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:12.978 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:12.979 DEBUG:boto:Method: HEAD 2022-09-03T06:44:12.980 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:12.980 DEBUG:boto:Data: 2022-09-03T06:44:12.981 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.981 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:12.982 DEBUG:boto:Port: 8001 2022-09-03T06:44:12.982 DEBUG:boto:Params: {} 2022-09-03T06:44:12.983 DEBUG:boto:Token: None 2022-09-03T06:44:12.983 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:12.984 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z1ePOhenBMLdZb2iHSqtjNu33aI= 2022-09-03T06:44:12.985 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z1ePOhenBMLdZb2iHSqtjNu33aI=', 'Content-Length': '0'} 2022-09-03T06:44:12.987 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f4ac73a8f90cb6f5-006312f7bc-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:12.988 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:12.989 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:12.989 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:12.990 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:12.991 DEBUG:boto:Method: HEAD 2022-09-03T06:44:12.991 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:12.992 DEBUG:boto:Data: 2022-09-03T06:44:12.992 DEBUG:boto:Headers: {} 2022-09-03T06:44:12.993 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:12.993 DEBUG:boto:Port: 8000 2022-09-03T06:44:12.994 DEBUG:boto:Params: {} 2022-09-03T06:44:12.995 DEBUG:boto:Token: None 2022-09-03T06:44:12.995 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:12 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:12.996 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z1ePOhenBMLdZb2iHSqtjNu33aI= 2022-09-03T06:44:12.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z1ePOhenBMLdZb2iHSqtjNu33aI=', 'Content-Length': '0'} 2022-09-03T06:44:12.999 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000989c2e0b08b75842-006312f7bc-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.000 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:44:13.001 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:13.002 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:13.003 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:13.004 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:13.005 DEBUG:boto:Method: GET 2022-09-03T06:44:13.005 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:13.006 DEBUG:boto:Data: 2022-09-03T06:44:13.007 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.008 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.009 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.010 DEBUG:boto:Params: {} 2022-09-03T06:44:13.011 DEBUG:boto:Token: None 2022-09-03T06:44:13.011 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:13.012 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hQawoWbQilOpcHSc1uYhSYELaec= 2022-09-03T06:44:13.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hQawoWbQilOpcHSc1uYhSYELaec=', 'Content-Length': '0'} 2022-09-03T06:44:13.015 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006ddbbb9bb2868660-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.062 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:13.063 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:13.064 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:13.064 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:13.065 DEBUG:boto:Method: GET 2022-09-03T06:44:13.065 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:13.066 DEBUG:boto:Data: 2022-09-03T06:44:13.066 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.067 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.068 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.068 DEBUG:boto:Params: {} 2022-09-03T06:44:13.069 DEBUG:boto:Token: None 2022-09-03T06:44:13.070 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:44:13.070 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hQawoWbQilOpcHSc1uYhSYELaec= 2022-09-03T06:44:13.071 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hQawoWbQilOpcHSc1uYhSYELaec=', 'Content-Length': '0'} 2022-09-03T06:44:13.073 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c527d80961146851-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.075 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:13.075 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:13.077 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:13.078 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:13.079 DEBUG:boto:Method: GET 2022-09-03T06:44:13.081 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:44:13.081 DEBUG:boto:Data: 2022-09-03T06:44:13.083 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.084 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.085 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.085 DEBUG:boto:Params: {} 2022-09-03T06:44:13.086 DEBUG:boto:Token: None 2022-09-03T06:44:13.087 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/ 2022-09-03T06:44:13.087 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c= 2022-09-03T06:44:13.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c=', 'Content-Length': '0'} 2022-09-03T06:44:13.091 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a0cf1c2df74c1fc9-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.092 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:13.093 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:13.094 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:13.094 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:13.095 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:13.095 DEBUG:boto:Method: GET 2022-09-03T06:44:13.096 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:44:13.097 DEBUG:boto:Data: 2022-09-03T06:44:13.098 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.099 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.100 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.100 DEBUG:boto:Params: {} 2022-09-03T06:44:13.101 DEBUG:boto:Token: None 2022-09-03T06:44:13.102 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/ 2022-09-03T06:44:13.103 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c= 2022-09-03T06:44:13.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c=', 'Content-Length': '0'} 2022-09-03T06:44:13.107 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000438a083fa8202cba-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.109 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:13.110 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:13.111 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:13.111 DEBUG:boto:Method: HEAD 2022-09-03T06:44:13.112 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:44:13.113 DEBUG:boto:Data: 2022-09-03T06:44:13.114 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.114 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.116 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.116 DEBUG:boto:Params: {} 2022-09-03T06:44:13.117 DEBUG:boto:Token: None 2022-09-03T06:44:13.118 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj115 2022-09-03T06:44:13.119 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QoC3sF6u3Di/gQ0PqIPX6qr5duk= 2022-09-03T06:44:13.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QoC3sF6u3Di/gQ0PqIPX6qr5duk=', 'Content-Length': '0'} 2022-09-03T06:44:13.124 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000cc639f9a0f7bd5f-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.125 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:13.126 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:13.127 DEBUG:boto:Method: HEAD 2022-09-03T06:44:13.128 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:44:13.128 DEBUG:boto:Data: 2022-09-03T06:44:13.129 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.130 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.131 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.131 DEBUG:boto:Params: {} 2022-09-03T06:44:13.132 DEBUG:boto:Token: None 2022-09-03T06:44:13.133 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj115 2022-09-03T06:44:13.134 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QoC3sF6u3Di/gQ0PqIPX6qr5duk= 2022-09-03T06:44:13.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QoC3sF6u3Di/gQ0PqIPX6qr5duk=', 'Content-Length': '0'} 2022-09-03T06:44:13.139 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cef2ada90107129d-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.139 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:44:13.140 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:13.141 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:13.142 DEBUG:boto:Method: GET 2022-09-03T06:44:13.144 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:44:13.145 DEBUG:boto:Data: 2022-09-03T06:44:13.146 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.147 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.148 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.149 DEBUG:boto:Params: {} 2022-09-03T06:44:13.150 DEBUG:boto:Token: None 2022-09-03T06:44:13.151 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj115 2022-09-03T06:44:13.152 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:apn/qbJ4NKpA6YIarbaBWKOEDSk= 2022-09-03T06:44:13.153 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:apn/qbJ4NKpA6YIarbaBWKOEDSk=', 'Content-Length': '0'} 2022-09-03T06:44:13.156 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b59c072456c48586-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.157 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:44:13.158 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:44:13.158 DEBUG:boto:Method: GET 2022-09-03T06:44:13.159 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:44:13.160 DEBUG:boto:Data: 2022-09-03T06:44:13.161 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.161 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.162 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.163 DEBUG:boto:Params: {} 2022-09-03T06:44:13.163 DEBUG:boto:Token: None 2022-09-03T06:44:13.164 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj115 2022-09-03T06:44:13.165 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:apn/qbJ4NKpA6YIarbaBWKOEDSk= 2022-09-03T06:44:13.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:apn/qbJ4NKpA6YIarbaBWKOEDSk=', 'Content-Length': '0'} 2022-09-03T06:44:13.167 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006d90a637b0e02b2b-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.210 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:13.211 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:13.211 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:13.212 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:13.213 DEBUG:boto:Method: GET 2022-09-03T06:44:13.213 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:44:13.214 DEBUG:boto:Data: 2022-09-03T06:44:13.215 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.215 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.216 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.216 DEBUG:boto:Params: {} 2022-09-03T06:44:13.217 DEBUG:boto:Token: None 2022-09-03T06:44:13.218 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/ 2022-09-03T06:44:13.218 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c= 2022-09-03T06:44:13.219 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c=', 'Content-Length': '0'} 2022-09-03T06:44:13.222 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008cb6f5674c7900a6-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.223 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:13.224 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:13.224 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:13.225 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:13.225 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:44:13.226 DEBUG:boto:Method: GET 2022-09-03T06:44:13.227 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:44:13.227 DEBUG:boto:Data: 2022-09-03T06:44:13.228 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.228 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.229 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.229 DEBUG:boto:Params: {} 2022-09-03T06:44:13.230 DEBUG:boto:Token: None 2022-09-03T06:44:13.231 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/ 2022-09-03T06:44:13.232 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c= 2022-09-03T06:44:13.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c=', 'Content-Length': '0'} 2022-09-03T06:44:13.235 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000554d20671fe170dd-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.236 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:13.237 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:44:13.238 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:13.238 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:13.239 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.240 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.240 DEBUG:boto:Method: GET 2022-09-03T06:44:13.241 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.241 DEBUG:boto:Data: 2022-09-03T06:44:13.242 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.242 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.243 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.243 DEBUG:boto:Params: {} 2022-09-03T06:44:13.244 DEBUG:boto:Token: None 2022-09-03T06:44:13.245 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.245 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KwKA+h8EbXb1t8olSBVZOi6H1K4= 2022-09-03T06:44:13.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KwKA+h8EbXb1t8olSBVZOi6H1K4=', 'Content-Length': '0'} 2022-09-03T06:44:13.248 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd6a2f3796de1dab-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.249 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:13.250 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:13.250 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.251 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.251 DEBUG:boto:Method: GET 2022-09-03T06:44:13.252 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.253 DEBUG:boto:Data: 2022-09-03T06:44:13.253 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.254 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.254 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.255 DEBUG:boto:Params: {} 2022-09-03T06:44:13.256 DEBUG:boto:Token: None 2022-09-03T06:44:13.256 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.257 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KwKA+h8EbXb1t8olSBVZOi6H1K4= 2022-09-03T06:44:13.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KwKA+h8EbXb1t8olSBVZOi6H1K4=', 'Content-Length': '0'} 2022-09-03T06:44:13.260 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d7cdb4e0a510080-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.261 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:13.262 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:13.262 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.263 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.264 DEBUG:boto:Method: HEAD 2022-09-03T06:44:13.265 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.265 DEBUG:boto:Data: 2022-09-03T06:44:13.266 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.266 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.267 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.267 DEBUG:boto:Params: {} 2022-09-03T06:44:13.268 DEBUG:boto:Token: None 2022-09-03T06:44:13.269 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.271 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ew8S3qEfi98fQKQtVg8V2n/wSfI= 2022-09-03T06:44:13.271 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ew8S3qEfi98fQKQtVg8V2n/wSfI=', 'Content-Length': '0'} 2022-09-03T06:44:13.273 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021d74075ee356878-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.274 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:13.275 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:13.276 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.276 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.277 DEBUG:boto:Method: HEAD 2022-09-03T06:44:13.277 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.278 DEBUG:boto:Data: 2022-09-03T06:44:13.278 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.279 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.279 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.280 DEBUG:boto:Params: {} 2022-09-03T06:44:13.280 DEBUG:boto:Token: None 2022-09-03T06:44:13.281 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.283 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ew8S3qEfi98fQKQtVg8V2n/wSfI= 2022-09-03T06:44:13.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ew8S3qEfi98fQKQtVg8V2n/wSfI=', 'Content-Length': '0'} 2022-09-03T06:44:13.286 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b6b1c5eac7ad3f84-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.287 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:44:13.287 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:13.288 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:13.288 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.289 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.289 DEBUG:boto:Method: GET 2022-09-03T06:44:13.290 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.291 DEBUG:boto:Data: 2022-09-03T06:44:13.291 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.292 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.292 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.293 DEBUG:boto:Params: {} 2022-09-03T06:44:13.293 DEBUG:boto:Token: None 2022-09-03T06:44:13.294 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.295 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KwKA+h8EbXb1t8olSBVZOi6H1K4= 2022-09-03T06:44:13.295 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KwKA+h8EbXb1t8olSBVZOi6H1K4=', 'Content-Length': '0'} 2022-09-03T06:44:13.297 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a33b7e8120f148e9-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.298 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:13.299 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:13.299 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.300 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.300 DEBUG:boto:Method: GET 2022-09-03T06:44:13.301 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.301 DEBUG:boto:Data: 2022-09-03T06:44:13.302 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.303 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.303 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.304 DEBUG:boto:Params: {} 2022-09-03T06:44:13.304 DEBUG:boto:Token: None 2022-09-03T06:44:13.305 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:44:13.305 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KwKA+h8EbXb1t8olSBVZOi6H1K4= 2022-09-03T06:44:13.306 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KwKA+h8EbXb1t8olSBVZOi6H1K4=', 'Content-Length': '0'} 2022-09-03T06:44:13.308 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000060345c5d9a683673-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.309 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:13.310 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:13.311 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:13.311 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:13.312 DEBUG:boto:Method: GET 2022-09-03T06:44:13.313 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:44:13.313 DEBUG:boto:Data: 2022-09-03T06:44:13.314 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.314 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.315 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.316 DEBUG:boto:Params: {} 2022-09-03T06:44:13.316 DEBUG:boto:Token: None 2022-09-03T06:44:13.317 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/ 2022-09-03T06:44:13.318 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c= 2022-09-03T06:44:13.319 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c=', 'Content-Length': '0'} 2022-09-03T06:44:13.323 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000dc368ed792e2ceb-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.324 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:13.324 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:13.325 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:13.326 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:13.326 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:13.327 DEBUG:boto:Method: GET 2022-09-03T06:44:13.328 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:44:13.328 DEBUG:boto:Data: 2022-09-03T06:44:13.329 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.329 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.330 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.330 DEBUG:boto:Params: {} 2022-09-03T06:44:13.331 DEBUG:boto:Token: None 2022-09-03T06:44:13.332 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/ 2022-09-03T06:44:13.332 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c= 2022-09-03T06:44:13.333 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c=', 'Content-Length': '0'} 2022-09-03T06:44:13.336 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eee19a947f547397-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.336 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:13.337 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:13.338 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:13.338 DEBUG:boto:Method: HEAD 2022-09-03T06:44:13.339 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:44:13.339 DEBUG:boto:Data: 2022-09-03T06:44:13.340 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.340 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.341 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.341 DEBUG:boto:Params: {} 2022-09-03T06:44:13.342 DEBUG:boto:Token: None 2022-09-03T06:44:13.342 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj17 2022-09-03T06:44:13.343 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N77ma6zGbhHk/j7uPDbLp+DD4aM= 2022-09-03T06:44:13.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N77ma6zGbhHk/j7uPDbLp+DD4aM=', 'Content-Length': '0'} 2022-09-03T06:44:13.346 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000014d0c10f572313fb-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.346 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:13.347 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:13.347 DEBUG:boto:Method: HEAD 2022-09-03T06:44:13.348 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:44:13.348 DEBUG:boto:Data: 2022-09-03T06:44:13.349 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.349 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.350 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.350 DEBUG:boto:Params: {} 2022-09-03T06:44:13.351 DEBUG:boto:Token: None 2022-09-03T06:44:13.352 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj17 2022-09-03T06:44:13.352 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N77ma6zGbhHk/j7uPDbLp+DD4aM= 2022-09-03T06:44:13.353 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N77ma6zGbhHk/j7uPDbLp+DD4aM=', 'Content-Length': '0'} 2022-09-03T06:44:13.355 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee925097d58cdac9-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.356 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:44:13.356 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:13.357 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:13.357 DEBUG:boto:Method: GET 2022-09-03T06:44:13.358 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:44:13.359 DEBUG:boto:Data: 2022-09-03T06:44:13.359 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.360 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.360 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.361 DEBUG:boto:Params: {} 2022-09-03T06:44:13.361 DEBUG:boto:Token: None 2022-09-03T06:44:13.362 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj17 2022-09-03T06:44:13.363 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4X/9dsu+Mt/ps+blypcpKoh452U= 2022-09-03T06:44:13.363 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4X/9dsu+Mt/ps+blypcpKoh452U=', 'Content-Length': '0'} 2022-09-03T06:44:13.365 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d2e45b2050a761b4-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.410 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:44:13.411 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:44:13.411 DEBUG:boto:Method: GET 2022-09-03T06:44:13.412 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:44:13.413 DEBUG:boto:Data: 2022-09-03T06:44:13.413 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.414 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.414 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.415 DEBUG:boto:Params: {} 2022-09-03T06:44:13.415 DEBUG:boto:Token: None 2022-09-03T06:44:13.416 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj17 2022-09-03T06:44:13.417 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4X/9dsu+Mt/ps+blypcpKoh452U= 2022-09-03T06:44:13.417 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4X/9dsu+Mt/ps+blypcpKoh452U=', 'Content-Length': '0'} 2022-09-03T06:44:13.419 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000be35fadc9afc6b2e-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.420 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:13.421 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:13.421 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:13.422 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:13.422 DEBUG:boto:Method: GET 2022-09-03T06:44:13.423 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:44:13.423 DEBUG:boto:Data: 2022-09-03T06:44:13.424 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.424 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.425 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.426 DEBUG:boto:Params: {} 2022-09-03T06:44:13.426 DEBUG:boto:Token: None 2022-09-03T06:44:13.427 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/ 2022-09-03T06:44:13.428 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c= 2022-09-03T06:44:13.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c=', 'Content-Length': '0'} 2022-09-03T06:44:13.431 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008c212db07bfae1cc-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.432 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:13.433 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:13.433 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:13.434 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:13.435 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:44:13.435 DEBUG:boto:Method: GET 2022-09-03T06:44:13.436 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:44:13.437 DEBUG:boto:Data: 2022-09-03T06:44:13.437 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.438 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.438 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.439 DEBUG:boto:Params: {} 2022-09-03T06:44:13.440 DEBUG:boto:Token: None 2022-09-03T06:44:13.440 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/ 2022-09-03T06:44:13.441 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c= 2022-09-03T06:44:13.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c=', 'Content-Length': '0'} 2022-09-03T06:44:13.445 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006549cfed8cbe534a-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.446 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:13.447 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:44:13.448 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:13.449 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:13.450 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.451 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.452 DEBUG:boto:Method: GET 2022-09-03T06:44:13.452 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.453 DEBUG:boto:Data: 2022-09-03T06:44:13.454 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.454 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.455 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.455 DEBUG:boto:Params: {} 2022-09-03T06:44:13.456 DEBUG:boto:Token: None 2022-09-03T06:44:13.457 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.458 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oaZBB6Z9ESv627REjVuAMorNDnE= 2022-09-03T06:44:13.459 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oaZBB6Z9ESv627REjVuAMorNDnE=', 'Content-Length': '0'} 2022-09-03T06:44:13.462 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ec4e429a5ab814af-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.463 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:13.463 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:13.464 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.465 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.465 DEBUG:boto:Method: GET 2022-09-03T06:44:13.466 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.467 DEBUG:boto:Data: 2022-09-03T06:44:13.467 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.468 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.469 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.469 DEBUG:boto:Params: {} 2022-09-03T06:44:13.470 DEBUG:boto:Token: None 2022-09-03T06:44:13.471 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.472 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oaZBB6Z9ESv627REjVuAMorNDnE= 2022-09-03T06:44:13.473 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oaZBB6Z9ESv627REjVuAMorNDnE=', 'Content-Length': '0'} 2022-09-03T06:44:13.475 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000ce7720baa4328b5-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.476 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:13.477 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:13.478 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.479 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.480 DEBUG:boto:Method: HEAD 2022-09-03T06:44:13.481 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.482 DEBUG:boto:Data: 2022-09-03T06:44:13.482 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.484 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.484 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.485 DEBUG:boto:Params: {} 2022-09-03T06:44:13.486 DEBUG:boto:Token: None 2022-09-03T06:44:13.487 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.488 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r2KNqTzuuUYd2VdyOXERWaN/DHs= 2022-09-03T06:44:13.489 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r2KNqTzuuUYd2VdyOXERWaN/DHs=', 'Content-Length': '0'} 2022-09-03T06:44:13.492 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002243c3481484f141-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.493 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:13.493 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:13.494 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.495 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.495 DEBUG:boto:Method: HEAD 2022-09-03T06:44:13.496 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.496 DEBUG:boto:Data: 2022-09-03T06:44:13.497 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.497 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.498 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.498 DEBUG:boto:Params: {} 2022-09-03T06:44:13.499 DEBUG:boto:Token: None 2022-09-03T06:44:13.499 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.500 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r2KNqTzuuUYd2VdyOXERWaN/DHs= 2022-09-03T06:44:13.500 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r2KNqTzuuUYd2VdyOXERWaN/DHs=', 'Content-Length': '0'} 2022-09-03T06:44:13.503 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e7a2da7c4ae45b93-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.503 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:44:13.504 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:13.505 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:13.505 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.506 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.506 DEBUG:boto:Method: GET 2022-09-03T06:44:13.507 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.508 DEBUG:boto:Data: 2022-09-03T06:44:13.508 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.509 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.509 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.510 DEBUG:boto:Params: {} 2022-09-03T06:44:13.511 DEBUG:boto:Token: None 2022-09-03T06:44:13.512 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.512 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oaZBB6Z9ESv627REjVuAMorNDnE= 2022-09-03T06:44:13.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oaZBB6Z9ESv627REjVuAMorNDnE=', 'Content-Length': '0'} 2022-09-03T06:44:13.519 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e9bbd21c2cd2d1fa-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.519 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:13.520 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:13.521 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.521 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.522 DEBUG:boto:Method: GET 2022-09-03T06:44:13.523 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.524 DEBUG:boto:Data: 2022-09-03T06:44:13.525 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.526 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.527 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.528 DEBUG:boto:Params: {} 2022-09-03T06:44:13.529 DEBUG:boto:Token: None 2022-09-03T06:44:13.530 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:44:13.531 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oaZBB6Z9ESv627REjVuAMorNDnE= 2022-09-03T06:44:13.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oaZBB6Z9ESv627REjVuAMorNDnE=', 'Content-Length': '0'} 2022-09-03T06:44:13.535 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c9c94212510b7a7-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.578 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:13.579 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:13.580 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:13.581 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:13.582 DEBUG:boto:Method: GET 2022-09-03T06:44:13.583 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:44:13.583 DEBUG:boto:Data: 2022-09-03T06:44:13.584 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.585 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.585 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.586 DEBUG:boto:Params: {} 2022-09-03T06:44:13.586 DEBUG:boto:Token: None 2022-09-03T06:44:13.587 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/ 2022-09-03T06:44:13.587 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c= 2022-09-03T06:44:13.588 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c=', 'Content-Length': '0'} 2022-09-03T06:44:13.591 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000f5e74a246fd2fe3-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.592 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:13.593 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:13.593 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:13.594 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:13.595 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:13.596 DEBUG:boto:Method: GET 2022-09-03T06:44:13.596 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:44:13.597 DEBUG:boto:Data: 2022-09-03T06:44:13.597 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.598 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.598 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.599 DEBUG:boto:Params: {} 2022-09-03T06:44:13.600 DEBUG:boto:Token: None 2022-09-03T06:44:13.600 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/ 2022-09-03T06:44:13.601 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c= 2022-09-03T06:44:13.602 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c=', 'Content-Length': '0'} 2022-09-03T06:44:13.606 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002afe9685b49e827c-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.607 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:13.608 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:13.608 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:13.609 DEBUG:boto:Method: HEAD 2022-09-03T06:44:13.610 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:44:13.611 DEBUG:boto:Data: 2022-09-03T06:44:13.611 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.612 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.613 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.613 DEBUG:boto:Params: {} 2022-09-03T06:44:13.614 DEBUG:boto:Token: None 2022-09-03T06:44:13.615 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj19 2022-09-03T06:44:13.616 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SD2VkTT8OeOc1ghnwtji7SQbXAA= 2022-09-03T06:44:13.618 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SD2VkTT8OeOc1ghnwtji7SQbXAA=', 'Content-Length': '0'} 2022-09-03T06:44:13.626 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009cbf00c00c9ae3d2-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.627 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:13.628 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:13.628 DEBUG:boto:Method: HEAD 2022-09-03T06:44:13.629 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:44:13.629 DEBUG:boto:Data: 2022-09-03T06:44:13.630 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.630 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.631 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.631 DEBUG:boto:Params: {} 2022-09-03T06:44:13.632 DEBUG:boto:Token: None 2022-09-03T06:44:13.632 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj19 2022-09-03T06:44:13.633 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SD2VkTT8OeOc1ghnwtji7SQbXAA= 2022-09-03T06:44:13.634 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SD2VkTT8OeOc1ghnwtji7SQbXAA=', 'Content-Length': '0'} 2022-09-03T06:44:13.636 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003727f17f6220c93a-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.636 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:44:13.637 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:13.637 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:13.638 DEBUG:boto:Method: GET 2022-09-03T06:44:13.639 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:44:13.639 DEBUG:boto:Data: 2022-09-03T06:44:13.640 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.640 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.641 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.641 DEBUG:boto:Params: {} 2022-09-03T06:44:13.642 DEBUG:boto:Token: None 2022-09-03T06:44:13.643 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj19 2022-09-03T06:44:13.644 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hh//iy3b+9beGu7RE6urXsJunvg= 2022-09-03T06:44:13.644 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hh//iy3b+9beGu7RE6urXsJunvg=', 'Content-Length': '0'} 2022-09-03T06:44:13.646 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a57ae53038d086cd-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.690 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:44:13.692 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:44:13.693 DEBUG:boto:Method: GET 2022-09-03T06:44:13.694 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:44:13.695 DEBUG:boto:Data: 2022-09-03T06:44:13.696 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.697 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.697 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.698 DEBUG:boto:Params: {} 2022-09-03T06:44:13.700 DEBUG:boto:Token: None 2022-09-03T06:44:13.701 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj19 2022-09-03T06:44:13.701 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hh//iy3b+9beGu7RE6urXsJunvg= 2022-09-03T06:44:13.702 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hh//iy3b+9beGu7RE6urXsJunvg=', 'Content-Length': '0'} 2022-09-03T06:44:13.704 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000497939964583b95-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.705 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:13.706 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:13.707 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:13.707 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:13.708 DEBUG:boto:Method: GET 2022-09-03T06:44:13.708 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:44:13.709 DEBUG:boto:Data: 2022-09-03T06:44:13.711 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.711 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.712 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.712 DEBUG:boto:Params: {} 2022-09-03T06:44:13.713 DEBUG:boto:Token: None 2022-09-03T06:44:13.713 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/ 2022-09-03T06:44:13.715 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c= 2022-09-03T06:44:13.715 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c=', 'Content-Length': '0'} 2022-09-03T06:44:13.718 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000025ae9268b96275a7-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.719 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:13.719 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:13.720 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:13.721 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:13.721 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:44:13.722 DEBUG:boto:Method: GET 2022-09-03T06:44:13.723 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:44:13.723 DEBUG:boto:Data: 2022-09-03T06:44:13.724 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.724 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.725 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.726 DEBUG:boto:Params: {} 2022-09-03T06:44:13.727 DEBUG:boto:Token: None 2022-09-03T06:44:13.727 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/ 2022-09-03T06:44:13.728 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c= 2022-09-03T06:44:13.729 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c=', 'Content-Length': '0'} 2022-09-03T06:44:13.731 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000083a5ba3bdc369a46-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.732 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:13.733 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:44:13.734 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:13.735 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:13.735 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.736 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.737 DEBUG:boto:Method: GET 2022-09-03T06:44:13.738 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.738 DEBUG:boto:Data: 2022-09-03T06:44:13.739 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.739 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.740 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.740 DEBUG:boto:Params: {} 2022-09-03T06:44:13.741 DEBUG:boto:Token: None 2022-09-03T06:44:13.742 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.742 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vgc0wMEN5UXpqfEP6Dws3vaVdJc= 2022-09-03T06:44:13.743 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vgc0wMEN5UXpqfEP6Dws3vaVdJc=', 'Content-Length': '0'} 2022-09-03T06:44:13.745 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006d005169abc23d0b-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.746 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:13.747 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:13.748 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.749 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.750 DEBUG:boto:Method: GET 2022-09-03T06:44:13.751 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.752 DEBUG:boto:Data: 2022-09-03T06:44:13.753 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.754 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.755 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.756 DEBUG:boto:Params: {} 2022-09-03T06:44:13.757 DEBUG:boto:Token: None 2022-09-03T06:44:13.758 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.759 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vgc0wMEN5UXpqfEP6Dws3vaVdJc= 2022-09-03T06:44:13.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vgc0wMEN5UXpqfEP6Dws3vaVdJc=', 'Content-Length': '0'} 2022-09-03T06:44:13.763 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000037ca0412807b615e-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.764 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:13.765 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:13.766 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.766 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.767 DEBUG:boto:Method: HEAD 2022-09-03T06:44:13.768 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.769 DEBUG:boto:Data: 2022-09-03T06:44:13.770 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.770 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.771 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.771 DEBUG:boto:Params: {} 2022-09-03T06:44:13.772 DEBUG:boto:Token: None 2022-09-03T06:44:13.773 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.773 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LMt1gj10aKARWl1MXFAQAQBhK7c= 2022-09-03T06:44:13.774 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LMt1gj10aKARWl1MXFAQAQBhK7c=', 'Content-Length': '0'} 2022-09-03T06:44:13.776 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005aa71151322919a4-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.777 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:13.777 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:13.778 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.779 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.779 DEBUG:boto:Method: HEAD 2022-09-03T06:44:13.780 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.780 DEBUG:boto:Data: 2022-09-03T06:44:13.781 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.781 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.782 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.783 DEBUG:boto:Params: {} 2022-09-03T06:44:13.783 DEBUG:boto:Token: None 2022-09-03T06:44:13.784 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.784 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LMt1gj10aKARWl1MXFAQAQBhK7c= 2022-09-03T06:44:13.785 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LMt1gj10aKARWl1MXFAQAQBhK7c=', 'Content-Length': '0'} 2022-09-03T06:44:13.787 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000502e7753747f917a-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.788 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:44:13.789 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:13.789 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:13.790 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.790 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.791 DEBUG:boto:Method: GET 2022-09-03T06:44:13.791 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.792 DEBUG:boto:Data: 2022-09-03T06:44:13.792 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.793 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.793 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.794 DEBUG:boto:Params: {} 2022-09-03T06:44:13.794 DEBUG:boto:Token: None 2022-09-03T06:44:13.795 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.796 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vgc0wMEN5UXpqfEP6Dws3vaVdJc= 2022-09-03T06:44:13.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vgc0wMEN5UXpqfEP6Dws3vaVdJc=', 'Content-Length': '0'} 2022-09-03T06:44:13.798 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009cb0f2eed778f25a-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.799 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:13.800 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:13.800 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.801 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.801 DEBUG:boto:Method: GET 2022-09-03T06:44:13.802 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.802 DEBUG:boto:Data: 2022-09-03T06:44:13.803 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.803 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.804 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.805 DEBUG:boto:Params: {} 2022-09-03T06:44:13.805 DEBUG:boto:Token: None 2022-09-03T06:44:13.806 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:44:13.806 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vgc0wMEN5UXpqfEP6Dws3vaVdJc= 2022-09-03T06:44:13.807 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vgc0wMEN5UXpqfEP6Dws3vaVdJc=', 'Content-Length': '0'} 2022-09-03T06:44:13.809 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000098c19a5739f3cc06-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.854 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:13.855 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:13.856 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:13.857 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:13.858 DEBUG:boto:Method: GET 2022-09-03T06:44:13.859 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:44:13.860 DEBUG:boto:Data: 2022-09-03T06:44:13.862 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.863 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.864 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.865 DEBUG:boto:Params: {} 2022-09-03T06:44:13.867 DEBUG:boto:Token: None 2022-09-03T06:44:13.868 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/ 2022-09-03T06:44:13.869 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c= 2022-09-03T06:44:13.870 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c=', 'Content-Length': '0'} 2022-09-03T06:44:13.873 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000009e0bb785ae5261d-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.875 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:13.876 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:13.877 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:13.878 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:13.879 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:13.880 DEBUG:boto:Method: GET 2022-09-03T06:44:13.881 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:44:13.882 DEBUG:boto:Data: 2022-09-03T06:44:13.882 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.883 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.883 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.884 DEBUG:boto:Params: {} 2022-09-03T06:44:13.885 DEBUG:boto:Token: None 2022-09-03T06:44:13.885 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/ 2022-09-03T06:44:13.886 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c= 2022-09-03T06:44:13.887 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c=', 'Content-Length': '0'} 2022-09-03T06:44:13.889 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004e6bbb2a4bec30d2-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.891 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:13.891 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:13.892 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:13.893 DEBUG:boto:Method: HEAD 2022-09-03T06:44:13.893 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:44:13.894 DEBUG:boto:Data: 2022-09-03T06:44:13.894 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.895 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.895 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.896 DEBUG:boto:Params: {} 2022-09-03T06:44:13.897 DEBUG:boto:Token: None 2022-09-03T06:44:13.897 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj22 2022-09-03T06:44:13.898 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:P3wjvXNHSbTr6i51TvHE1RSFFK8= 2022-09-03T06:44:13.899 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:P3wjvXNHSbTr6i51TvHE1RSFFK8=', 'Content-Length': '0'} 2022-09-03T06:44:13.901 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d067c8d559d1ae2-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.901 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:13.902 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:13.903 DEBUG:boto:Method: HEAD 2022-09-03T06:44:13.903 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:44:13.904 DEBUG:boto:Data: 2022-09-03T06:44:13.904 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.905 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.905 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.906 DEBUG:boto:Params: {} 2022-09-03T06:44:13.907 DEBUG:boto:Token: None 2022-09-03T06:44:13.907 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj22 2022-09-03T06:44:13.908 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:P3wjvXNHSbTr6i51TvHE1RSFFK8= 2022-09-03T06:44:13.908 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:P3wjvXNHSbTr6i51TvHE1RSFFK8=', 'Content-Length': '0'} 2022-09-03T06:44:13.913 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a15a598bdff960b-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.915 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:44:13.916 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:13.917 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:13.918 DEBUG:boto:Method: GET 2022-09-03T06:44:13.919 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:44:13.920 DEBUG:boto:Data: 2022-09-03T06:44:13.921 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.922 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.923 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.924 DEBUG:boto:Params: {} 2022-09-03T06:44:13.925 DEBUG:boto:Token: None 2022-09-03T06:44:13.926 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj22 2022-09-03T06:44:13.928 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:13jmFVEf4iypvUK1w/aLRwONtq4= 2022-09-03T06:44:13.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:13jmFVEf4iypvUK1w/aLRwONtq4=', 'Content-Length': '0'} 2022-09-03T06:44:13.932 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000748c1c86e31b13d0-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.933 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:44:13.934 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:44:13.935 DEBUG:boto:Method: GET 2022-09-03T06:44:13.937 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:44:13.937 DEBUG:boto:Data: 2022-09-03T06:44:13.939 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.940 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.941 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.942 DEBUG:boto:Params: {} 2022-09-03T06:44:13.943 DEBUG:boto:Token: None 2022-09-03T06:44:13.944 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj22 2022-09-03T06:44:13.945 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:13jmFVEf4iypvUK1w/aLRwONtq4= 2022-09-03T06:44:13.945 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:13jmFVEf4iypvUK1w/aLRwONtq4=', 'Content-Length': '0'} 2022-09-03T06:44:13.947 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000054668dc1e0309d3f-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.948 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:13.949 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:13.949 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:13.950 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:13.950 DEBUG:boto:Method: GET 2022-09-03T06:44:13.951 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:44:13.951 DEBUG:boto:Data: 2022-09-03T06:44:13.952 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.952 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.953 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.953 DEBUG:boto:Params: {} 2022-09-03T06:44:13.954 DEBUG:boto:Token: None 2022-09-03T06:44:13.955 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/ 2022-09-03T06:44:13.955 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c= 2022-09-03T06:44:13.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c=', 'Content-Length': '0'} 2022-09-03T06:44:13.958 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e6171eca57740289-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.959 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:13.960 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:13.960 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:13.961 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:13.962 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:44:13.962 DEBUG:boto:Method: GET 2022-09-03T06:44:13.963 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:44:13.963 DEBUG:boto:Data: 2022-09-03T06:44:13.964 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.964 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:13.965 DEBUG:boto:Port: 8000 2022-09-03T06:44:13.965 DEBUG:boto:Params: {} 2022-09-03T06:44:13.966 DEBUG:boto:Token: None 2022-09-03T06:44:13.967 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/ 2022-09-03T06:44:13.967 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c= 2022-09-03T06:44:13.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h0iSj7k6H/UJyUucxhDdAfur94c=', 'Content-Length': '0'} 2022-09-03T06:44:13.971 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e327f6439c957526-006312f7bd-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:13.971 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:13.972 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:44:13.973 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:13.974 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:13.974 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:13.975 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:13.975 DEBUG:boto:Method: GET 2022-09-03T06:44:13.976 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:13.977 DEBUG:boto:Data: 2022-09-03T06:44:13.977 DEBUG:boto:Headers: {} 2022-09-03T06:44:13.978 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:13.978 DEBUG:boto:Port: 8001 2022-09-03T06:44:13.979 DEBUG:boto:Params: {} 2022-09-03T06:44:13.980 DEBUG:boto:Token: None 2022-09-03T06:44:13.980 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:13 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:13.981 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZXqUstti42pMcqD3LETsY6AstFk= 2022-09-03T06:44:13.982 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZXqUstti42pMcqD3LETsY6AstFk=', 'Content-Length': '0'} 2022-09-03T06:44:13.984 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c1981e28cd6ac845-006312f7bd-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.026 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:14.027 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:14.027 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:14.028 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:14.029 DEBUG:boto:Method: GET 2022-09-03T06:44:14.029 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:14.030 DEBUG:boto:Data: 2022-09-03T06:44:14.030 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.031 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.031 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.032 DEBUG:boto:Params: {} 2022-09-03T06:44:14.033 DEBUG:boto:Token: None 2022-09-03T06:44:14.033 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:14.034 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9yhWrPf6h9VDd7PL0mnAawmprp8= 2022-09-03T06:44:14.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9yhWrPf6h9VDd7PL0mnAawmprp8=', 'Content-Length': '0'} 2022-09-03T06:44:14.038 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008727e14979d211ec-006312f7be-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.040 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:14.041 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:14.042 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:14.043 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:14.043 DEBUG:boto:Method: HEAD 2022-09-03T06:44:14.044 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:14.045 DEBUG:boto:Data: 2022-09-03T06:44:14.046 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.046 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.047 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.047 DEBUG:boto:Params: {} 2022-09-03T06:44:14.048 DEBUG:boto:Token: None 2022-09-03T06:44:14.049 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:14.050 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bu9vRI9QoLbbOqs3vCTkg78Iq4Q= 2022-09-03T06:44:14.051 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bu9vRI9QoLbbOqs3vCTkg78Iq4Q=', 'Content-Length': '0'} 2022-09-03T06:44:14.054 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000794e9d7d4b528373-006312f7be-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.055 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:14.056 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:14.056 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:14.057 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:14.058 DEBUG:boto:Method: HEAD 2022-09-03T06:44:14.058 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:14.059 DEBUG:boto:Data: 2022-09-03T06:44:14.059 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.060 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.060 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.061 DEBUG:boto:Params: {} 2022-09-03T06:44:14.061 DEBUG:boto:Token: None 2022-09-03T06:44:14.062 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:14.063 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bu9vRI9QoLbbOqs3vCTkg78Iq4Q= 2022-09-03T06:44:14.063 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bu9vRI9QoLbbOqs3vCTkg78Iq4Q=', 'Content-Length': '0'} 2022-09-03T06:44:14.066 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000609ff1f1c70c2fa9-006312f7be-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.066 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:44:14.067 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:14.068 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:14.068 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:14.069 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:14.070 DEBUG:boto:Method: GET 2022-09-03T06:44:14.071 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:14.071 DEBUG:boto:Data: 2022-09-03T06:44:14.072 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.073 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.074 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.075 DEBUG:boto:Params: {} 2022-09-03T06:44:14.076 DEBUG:boto:Token: None 2022-09-03T06:44:14.077 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:14.078 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9yhWrPf6h9VDd7PL0mnAawmprp8= 2022-09-03T06:44:14.079 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9yhWrPf6h9VDd7PL0mnAawmprp8=', 'Content-Length': '0'} 2022-09-03T06:44:14.082 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f80bf8cbd3ca1f89-006312f7be-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.083 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:14.084 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:14.085 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:14.086 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:14.087 DEBUG:boto:Method: GET 2022-09-03T06:44:14.088 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:14.089 DEBUG:boto:Data: 2022-09-03T06:44:14.089 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.090 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.091 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.091 DEBUG:boto:Params: {} 2022-09-03T06:44:14.092 DEBUG:boto:Token: None 2022-09-03T06:44:14.093 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:44:14.093 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9yhWrPf6h9VDd7PL0mnAawmprp8= 2022-09-03T06:44:14.094 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9yhWrPf6h9VDd7PL0mnAawmprp8=', 'Content-Length': '0'} 2022-09-03T06:44:14.096 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d60bfa8c00af7c83-006312f7be-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.138 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:14.139 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:14.140 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:14.140 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:14.141 DEBUG:boto:Method: GET 2022-09-03T06:44:14.142 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:44:14.142 DEBUG:boto:Data: 2022-09-03T06:44:14.143 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.143 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.144 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.145 DEBUG:boto:Params: {} 2022-09-03T06:44:14.145 DEBUG:boto:Token: None 2022-09-03T06:44:14.146 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/ 2022-09-03T06:44:14.147 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks= 2022-09-03T06:44:14.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks=', 'Content-Length': '0'} 2022-09-03T06:44:14.151 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a4b6e31733151fca-006312f7be-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.152 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:14.153 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:14.153 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:14.154 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:14.154 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:14.155 DEBUG:boto:Method: GET 2022-09-03T06:44:14.155 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:44:14.156 DEBUG:boto:Data: 2022-09-03T06:44:14.157 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.158 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.158 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.159 DEBUG:boto:Params: {} 2022-09-03T06:44:14.159 DEBUG:boto:Token: None 2022-09-03T06:44:14.160 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/ 2022-09-03T06:44:14.161 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks= 2022-09-03T06:44:14.162 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks=', 'Content-Length': '0'} 2022-09-03T06:44:14.165 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000334f58036ef9f370-006312f7be-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.166 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:14.167 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:14.168 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:14.169 DEBUG:boto:Method: HEAD 2022-09-03T06:44:14.171 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:44:14.172 DEBUG:boto:Data: 2022-09-03T06:44:14.173 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.173 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.174 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.176 DEBUG:boto:Params: {} 2022-09-03T06:44:14.176 DEBUG:boto:Token: None 2022-09-03T06:44:14.177 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj23 2022-09-03T06:44:14.178 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2DpKezX1h1wzFGvhGeEllADe6Dk= 2022-09-03T06:44:14.179 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2DpKezX1h1wzFGvhGeEllADe6Dk=', 'Content-Length': '0'} 2022-09-03T06:44:14.182 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b441631790a7ee5-006312f7be-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.183 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:14.183 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:14.184 DEBUG:boto:Method: HEAD 2022-09-03T06:44:14.185 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:44:14.186 DEBUG:boto:Data: 2022-09-03T06:44:14.186 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.187 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.188 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.188 DEBUG:boto:Params: {} 2022-09-03T06:44:14.189 DEBUG:boto:Token: None 2022-09-03T06:44:14.189 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj23 2022-09-03T06:44:14.190 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2DpKezX1h1wzFGvhGeEllADe6Dk= 2022-09-03T06:44:14.191 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2DpKezX1h1wzFGvhGeEllADe6Dk=', 'Content-Length': '0'} 2022-09-03T06:44:14.193 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005a8e2f99ef76f3ff-006312f7be-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.194 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:44:14.195 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:14.195 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:14.196 DEBUG:boto:Method: GET 2022-09-03T06:44:14.197 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:44:14.198 DEBUG:boto:Data: 2022-09-03T06:44:14.198 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.199 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.199 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.200 DEBUG:boto:Params: {} 2022-09-03T06:44:14.200 DEBUG:boto:Token: None 2022-09-03T06:44:14.201 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj23 2022-09-03T06:44:14.202 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dJYsGiW3rQ7zLTPJtOIv1FIa38k= 2022-09-03T06:44:14.203 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dJYsGiW3rQ7zLTPJtOIv1FIa38k=', 'Content-Length': '0'} 2022-09-03T06:44:14.207 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca839d656399d00e-006312f7be-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.209 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:44:14.209 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:44:14.210 DEBUG:boto:Method: GET 2022-09-03T06:44:14.210 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:44:14.211 DEBUG:boto:Data: 2022-09-03T06:44:14.212 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.214 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.215 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.216 DEBUG:boto:Params: {} 2022-09-03T06:44:14.217 DEBUG:boto:Token: None 2022-09-03T06:44:14.217 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj23 2022-09-03T06:44:14.218 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dJYsGiW3rQ7zLTPJtOIv1FIa38k= 2022-09-03T06:44:14.219 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dJYsGiW3rQ7zLTPJtOIv1FIa38k=', 'Content-Length': '0'} 2022-09-03T06:44:14.221 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004c86c671168284c1-006312f7be-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.222 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:14.222 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:14.223 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:14.223 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:14.224 DEBUG:boto:Method: GET 2022-09-03T06:44:14.224 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:44:14.225 DEBUG:boto:Data: 2022-09-03T06:44:14.226 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.227 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.227 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.228 DEBUG:boto:Params: {} 2022-09-03T06:44:14.229 DEBUG:boto:Token: None 2022-09-03T06:44:14.229 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/ 2022-09-03T06:44:14.230 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks= 2022-09-03T06:44:14.230 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks=', 'Content-Length': '0'} 2022-09-03T06:44:14.233 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e07b3f10f4e964a9-006312f7be-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.234 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:14.235 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:14.236 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:14.236 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:14.237 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:44:14.237 DEBUG:boto:Method: GET 2022-09-03T06:44:14.238 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:44:14.239 DEBUG:boto:Data: 2022-09-03T06:44:14.239 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.240 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.241 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.241 DEBUG:boto:Params: {} 2022-09-03T06:44:14.242 DEBUG:boto:Token: None 2022-09-03T06:44:14.243 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/ 2022-09-03T06:44:14.244 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks= 2022-09-03T06:44:14.245 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks=', 'Content-Length': '0'} 2022-09-03T06:44:14.248 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005ff9aa2c00415f68-006312f7be-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.248 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:14.249 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:44:14.250 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:14.251 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:14.251 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.252 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.253 DEBUG:boto:Method: GET 2022-09-03T06:44:14.253 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.254 DEBUG:boto:Data: 2022-09-03T06:44:14.255 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.255 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.256 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.257 DEBUG:boto:Params: {} 2022-09-03T06:44:14.258 DEBUG:boto:Token: None 2022-09-03T06:44:14.258 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.259 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8jBlv8rDlgGJ7M/qovgrQ6w72mA= 2022-09-03T06:44:14.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8jBlv8rDlgGJ7M/qovgrQ6w72mA=', 'Content-Length': '0'} 2022-09-03T06:44:14.262 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001720000a0f457a42-006312f7be-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.263 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:14.263 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:14.264 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.264 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.265 DEBUG:boto:Method: GET 2022-09-03T06:44:14.266 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.266 DEBUG:boto:Data: 2022-09-03T06:44:14.267 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.267 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.268 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.269 DEBUG:boto:Params: {} 2022-09-03T06:44:14.269 DEBUG:boto:Token: None 2022-09-03T06:44:14.270 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.271 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8jBlv8rDlgGJ7M/qovgrQ6w72mA= 2022-09-03T06:44:14.271 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8jBlv8rDlgGJ7M/qovgrQ6w72mA=', 'Content-Length': '0'} 2022-09-03T06:44:14.274 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000adb92376f4ace4c8-006312f7be-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.275 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:14.276 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:14.277 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.278 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.279 DEBUG:boto:Method: HEAD 2022-09-03T06:44:14.280 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.281 DEBUG:boto:Data: 2022-09-03T06:44:14.282 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.283 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.284 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.285 DEBUG:boto:Params: {} 2022-09-03T06:44:14.287 DEBUG:boto:Token: None 2022-09-03T06:44:14.298 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.300 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AnAXO4EM5ah+mc24ObX6auPNJuc= 2022-09-03T06:44:14.301 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AnAXO4EM5ah+mc24ObX6auPNJuc=', 'Content-Length': '0'} 2022-09-03T06:44:14.304 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000013ce4c004a8a26b-006312f7be-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.305 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:14.306 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:14.307 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.308 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.309 DEBUG:boto:Method: HEAD 2022-09-03T06:44:14.310 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.311 DEBUG:boto:Data: 2022-09-03T06:44:14.312 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.313 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.314 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.315 DEBUG:boto:Params: {} 2022-09-03T06:44:14.316 DEBUG:boto:Token: None 2022-09-03T06:44:14.318 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.320 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AnAXO4EM5ah+mc24ObX6auPNJuc= 2022-09-03T06:44:14.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AnAXO4EM5ah+mc24ObX6auPNJuc=', 'Content-Length': '0'} 2022-09-03T06:44:14.324 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000153bd3e1c0b83cc2-006312f7be-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.325 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:44:14.326 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:14.327 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:14.328 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.329 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.330 DEBUG:boto:Method: GET 2022-09-03T06:44:14.331 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.332 DEBUG:boto:Data: 2022-09-03T06:44:14.333 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.334 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.335 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.335 DEBUG:boto:Params: {} 2022-09-03T06:44:14.336 DEBUG:boto:Token: None 2022-09-03T06:44:14.337 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.339 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8jBlv8rDlgGJ7M/qovgrQ6w72mA= 2022-09-03T06:44:14.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8jBlv8rDlgGJ7M/qovgrQ6w72mA=', 'Content-Length': '0'} 2022-09-03T06:44:14.347 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a0523d6aed35f1eb-006312f7be-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.349 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:14.349 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:14.350 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.351 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.352 DEBUG:boto:Method: GET 2022-09-03T06:44:14.353 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.354 DEBUG:boto:Data: 2022-09-03T06:44:14.355 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.356 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.357 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.358 DEBUG:boto:Params: {} 2022-09-03T06:44:14.359 DEBUG:boto:Token: None 2022-09-03T06:44:14.361 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:44:14.362 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8jBlv8rDlgGJ7M/qovgrQ6w72mA= 2022-09-03T06:44:14.363 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8jBlv8rDlgGJ7M/qovgrQ6w72mA=', 'Content-Length': '0'} 2022-09-03T06:44:14.373 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000be21fca65a1f0bd8-006312f7be-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.375 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:14.375 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:14.376 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:14.377 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:14.377 DEBUG:boto:Method: GET 2022-09-03T06:44:14.378 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:44:14.379 DEBUG:boto:Data: 2022-09-03T06:44:14.380 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.380 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.381 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.382 DEBUG:boto:Params: {} 2022-09-03T06:44:14.383 DEBUG:boto:Token: None 2022-09-03T06:44:14.385 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/ 2022-09-03T06:44:14.386 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks= 2022-09-03T06:44:14.387 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks=', 'Content-Length': '0'} 2022-09-03T06:44:14.391 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cc2bd199f0efc66e-006312f7be-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.392 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:14.394 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:14.394 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:14.395 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:14.395 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:14.396 DEBUG:boto:Method: GET 2022-09-03T06:44:14.397 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:44:14.397 DEBUG:boto:Data: 2022-09-03T06:44:14.398 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.399 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.399 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.400 DEBUG:boto:Params: {} 2022-09-03T06:44:14.401 DEBUG:boto:Token: None 2022-09-03T06:44:14.401 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/ 2022-09-03T06:44:14.402 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks= 2022-09-03T06:44:14.403 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks=', 'Content-Length': '0'} 2022-09-03T06:44:14.406 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000933848d45b8c2d7a-006312f7be-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.454 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:14.455 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:14.456 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:14.457 DEBUG:boto:Method: HEAD 2022-09-03T06:44:14.457 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:44:14.458 DEBUG:boto:Data: 2022-09-03T06:44:14.459 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.460 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.460 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.461 DEBUG:boto:Params: {} 2022-09-03T06:44:14.462 DEBUG:boto:Token: None 2022-09-03T06:44:14.463 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj33 2022-09-03T06:44:14.463 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L65VzxSM3ukVTAd0uYzf7ZvI5hA= 2022-09-03T06:44:14.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L65VzxSM3ukVTAd0uYzf7ZvI5hA=', 'Content-Length': '0'} 2022-09-03T06:44:14.467 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002aa3c8dea3229ad9-006312f7be-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.468 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:14.468 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:14.469 DEBUG:boto:Method: HEAD 2022-09-03T06:44:14.469 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:44:14.470 DEBUG:boto:Data: 2022-09-03T06:44:14.471 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.471 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.472 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.473 DEBUG:boto:Params: {} 2022-09-03T06:44:14.475 DEBUG:boto:Token: None 2022-09-03T06:44:14.476 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj33 2022-09-03T06:44:14.477 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L65VzxSM3ukVTAd0uYzf7ZvI5hA= 2022-09-03T06:44:14.478 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L65VzxSM3ukVTAd0uYzf7ZvI5hA=', 'Content-Length': '0'} 2022-09-03T06:44:14.488 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005632489f25b3b9f8-006312f7be-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.489 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:44:14.490 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:14.491 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:14.491 DEBUG:boto:Method: GET 2022-09-03T06:44:14.492 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:44:14.492 DEBUG:boto:Data: 2022-09-03T06:44:14.493 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.494 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.494 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.495 DEBUG:boto:Params: {} 2022-09-03T06:44:14.496 DEBUG:boto:Token: None 2022-09-03T06:44:14.497 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj33 2022-09-03T06:44:14.498 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yJL1xnw0haWWE1m9jXvZ1PLqIuc= 2022-09-03T06:44:14.498 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yJL1xnw0haWWE1m9jXvZ1PLqIuc=', 'Content-Length': '0'} 2022-09-03T06:44:14.500 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c21d233205721a0d-006312f7be-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.546 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:44:14.547 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:44:14.547 DEBUG:boto:Method: GET 2022-09-03T06:44:14.548 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:44:14.548 DEBUG:boto:Data: 2022-09-03T06:44:14.549 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.550 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.550 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.551 DEBUG:boto:Params: {} 2022-09-03T06:44:14.552 DEBUG:boto:Token: None 2022-09-03T06:44:14.552 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj33 2022-09-03T06:44:14.553 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yJL1xnw0haWWE1m9jXvZ1PLqIuc= 2022-09-03T06:44:14.554 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yJL1xnw0haWWE1m9jXvZ1PLqIuc=', 'Content-Length': '0'} 2022-09-03T06:44:14.556 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009c23637c3e25e330-006312f7be-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.556 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:14.557 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:14.558 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:14.558 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:14.559 DEBUG:boto:Method: GET 2022-09-03T06:44:14.559 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:44:14.560 DEBUG:boto:Data: 2022-09-03T06:44:14.560 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.561 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.561 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.562 DEBUG:boto:Params: {} 2022-09-03T06:44:14.563 DEBUG:boto:Token: None 2022-09-03T06:44:14.563 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/ 2022-09-03T06:44:14.564 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks= 2022-09-03T06:44:14.565 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks=', 'Content-Length': '0'} 2022-09-03T06:44:14.568 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002cb44f1c309ae33e-006312f7be-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.569 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:14.570 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:14.570 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:14.571 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:14.571 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:44:14.572 DEBUG:boto:Method: GET 2022-09-03T06:44:14.572 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:44:14.573 DEBUG:boto:Data: 2022-09-03T06:44:14.573 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.574 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.575 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.575 DEBUG:boto:Params: {} 2022-09-03T06:44:14.576 DEBUG:boto:Token: None 2022-09-03T06:44:14.577 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/ 2022-09-03T06:44:14.577 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks= 2022-09-03T06:44:14.578 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks=', 'Content-Length': '0'} 2022-09-03T06:44:14.581 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009cbd249a986a521f-006312f7be-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.581 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:14.582 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:44:14.583 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:14.583 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:14.584 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.584 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.585 DEBUG:boto:Method: GET 2022-09-03T06:44:14.585 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.586 DEBUG:boto:Data: 2022-09-03T06:44:14.587 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.587 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.588 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.588 DEBUG:boto:Params: {} 2022-09-03T06:44:14.589 DEBUG:boto:Token: None 2022-09-03T06:44:14.589 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.590 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+6X9qm1Q4EKHd0fvbee/LExLTfk= 2022-09-03T06:44:14.591 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+6X9qm1Q4EKHd0fvbee/LExLTfk=', 'Content-Length': '0'} 2022-09-03T06:44:14.593 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000027ad4f7ba7c5d831-006312f7be-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.594 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:14.594 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:14.595 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.595 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.596 DEBUG:boto:Method: GET 2022-09-03T06:44:14.597 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.597 DEBUG:boto:Data: 2022-09-03T06:44:14.598 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.598 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.599 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.599 DEBUG:boto:Params: {} 2022-09-03T06:44:14.600 DEBUG:boto:Token: None 2022-09-03T06:44:14.601 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.601 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+6X9qm1Q4EKHd0fvbee/LExLTfk= 2022-09-03T06:44:14.602 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+6X9qm1Q4EKHd0fvbee/LExLTfk=', 'Content-Length': '0'} 2022-09-03T06:44:14.604 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d8ce5a2ec3cf8d76-006312f7be-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.605 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:14.605 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:14.606 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.606 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.607 DEBUG:boto:Method: HEAD 2022-09-03T06:44:14.607 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.608 DEBUG:boto:Data: 2022-09-03T06:44:14.608 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.609 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.609 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.610 DEBUG:boto:Params: {} 2022-09-03T06:44:14.610 DEBUG:boto:Token: None 2022-09-03T06:44:14.611 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.612 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oxJxpZRCwze7goC9zc5/dy9yWYU= 2022-09-03T06:44:14.612 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oxJxpZRCwze7goC9zc5/dy9yWYU=', 'Content-Length': '0'} 2022-09-03T06:44:14.614 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000001c0f4eb3f1b5060-006312f7be-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.615 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:14.616 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:14.616 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.617 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.617 DEBUG:boto:Method: HEAD 2022-09-03T06:44:14.618 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.618 DEBUG:boto:Data: 2022-09-03T06:44:14.619 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.619 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.620 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.620 DEBUG:boto:Params: {} 2022-09-03T06:44:14.621 DEBUG:boto:Token: None 2022-09-03T06:44:14.621 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.622 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oxJxpZRCwze7goC9zc5/dy9yWYU= 2022-09-03T06:44:14.623 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oxJxpZRCwze7goC9zc5/dy9yWYU=', 'Content-Length': '0'} 2022-09-03T06:44:14.625 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001412054d85ec8723-006312f7be-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.625 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:44:14.626 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:14.627 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:14.627 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.628 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.628 DEBUG:boto:Method: GET 2022-09-03T06:44:14.629 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.629 DEBUG:boto:Data: 2022-09-03T06:44:14.630 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.630 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.631 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.631 DEBUG:boto:Params: {} 2022-09-03T06:44:14.632 DEBUG:boto:Token: None 2022-09-03T06:44:14.633 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.633 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+6X9qm1Q4EKHd0fvbee/LExLTfk= 2022-09-03T06:44:14.634 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+6X9qm1Q4EKHd0fvbee/LExLTfk=', 'Content-Length': '0'} 2022-09-03T06:44:14.636 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018ac6aae5c6e7ce6-006312f7be-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.637 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:14.637 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:14.638 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.638 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.639 DEBUG:boto:Method: GET 2022-09-03T06:44:14.640 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.640 DEBUG:boto:Data: 2022-09-03T06:44:14.641 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.641 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.642 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.642 DEBUG:boto:Params: {} 2022-09-03T06:44:14.643 DEBUG:boto:Token: None 2022-09-03T06:44:14.643 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:44:14.644 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+6X9qm1Q4EKHd0fvbee/LExLTfk= 2022-09-03T06:44:14.645 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+6X9qm1Q4EKHd0fvbee/LExLTfk=', 'Content-Length': '0'} 2022-09-03T06:44:14.647 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000dff336ba3152f94-006312f7be-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.648 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:14.648 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:14.649 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:14.649 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:14.650 DEBUG:boto:Method: GET 2022-09-03T06:44:14.650 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:44:14.651 DEBUG:boto:Data: 2022-09-03T06:44:14.651 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.652 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.652 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.653 DEBUG:boto:Params: {} 2022-09-03T06:44:14.653 DEBUG:boto:Token: None 2022-09-03T06:44:14.654 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/ 2022-09-03T06:44:14.655 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks= 2022-09-03T06:44:14.655 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks=', 'Content-Length': '0'} 2022-09-03T06:44:14.658 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bbf5cfe81e74e0b2-006312f7be-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.659 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:14.660 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:14.660 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:14.661 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:14.661 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:14.662 DEBUG:boto:Method: GET 2022-09-03T06:44:14.663 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:44:14.663 DEBUG:boto:Data: 2022-09-03T06:44:14.664 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.664 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.665 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.665 DEBUG:boto:Params: {} 2022-09-03T06:44:14.666 DEBUG:boto:Token: None 2022-09-03T06:44:14.667 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/ 2022-09-03T06:44:14.667 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks= 2022-09-03T06:44:14.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks=', 'Content-Length': '0'} 2022-09-03T06:44:14.671 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004943bef8584f6d6e-006312f7be-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.671 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:14.672 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:14.673 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:14.673 DEBUG:boto:Method: HEAD 2022-09-03T06:44:14.674 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:44:14.674 DEBUG:boto:Data: 2022-09-03T06:44:14.675 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.675 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.676 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.676 DEBUG:boto:Params: {} 2022-09-03T06:44:14.677 DEBUG:boto:Token: None 2022-09-03T06:44:14.678 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj34 2022-09-03T06:44:14.678 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BT9HiHXRlaUagu9HNarb8jgXsS4= 2022-09-03T06:44:14.679 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BT9HiHXRlaUagu9HNarb8jgXsS4=', 'Content-Length': '0'} 2022-09-03T06:44:14.681 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e44a7c689b352c02-006312f7be-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.682 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:14.682 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:14.683 DEBUG:boto:Method: HEAD 2022-09-03T06:44:14.683 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:44:14.684 DEBUG:boto:Data: 2022-09-03T06:44:14.684 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.685 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.685 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.686 DEBUG:boto:Params: {} 2022-09-03T06:44:14.686 DEBUG:boto:Token: None 2022-09-03T06:44:14.687 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj34 2022-09-03T06:44:14.687 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BT9HiHXRlaUagu9HNarb8jgXsS4= 2022-09-03T06:44:14.688 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BT9HiHXRlaUagu9HNarb8jgXsS4=', 'Content-Length': '0'} 2022-09-03T06:44:14.690 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a8f28b22f35da76-006312f7be-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.691 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:44:14.691 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:14.692 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:14.692 DEBUG:boto:Method: GET 2022-09-03T06:44:14.693 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:44:14.694 DEBUG:boto:Data: 2022-09-03T06:44:14.694 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.695 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.695 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.696 DEBUG:boto:Params: {} 2022-09-03T06:44:14.696 DEBUG:boto:Token: None 2022-09-03T06:44:14.697 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj34 2022-09-03T06:44:14.698 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CJuTZMhiy6500wzXo30d5YgLZUM= 2022-09-03T06:44:14.698 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CJuTZMhiy6500wzXo30d5YgLZUM=', 'Content-Length': '0'} 2022-09-03T06:44:14.700 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b3aadcc05e75733-006312f7be-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.743 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:44:14.744 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:44:14.745 DEBUG:boto:Method: GET 2022-09-03T06:44:14.745 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:44:14.746 DEBUG:boto:Data: 2022-09-03T06:44:14.746 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.747 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.748 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.748 DEBUG:boto:Params: {} 2022-09-03T06:44:14.749 DEBUG:boto:Token: None 2022-09-03T06:44:14.750 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj34 2022-09-03T06:44:14.750 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CJuTZMhiy6500wzXo30d5YgLZUM= 2022-09-03T06:44:14.751 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CJuTZMhiy6500wzXo30d5YgLZUM=', 'Content-Length': '0'} 2022-09-03T06:44:14.753 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008f2af4f65dc75126-006312f7be-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.755 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:14.756 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:14.756 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:14.757 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:14.757 DEBUG:boto:Method: GET 2022-09-03T06:44:14.758 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:44:14.759 DEBUG:boto:Data: 2022-09-03T06:44:14.760 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.761 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.762 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.762 DEBUG:boto:Params: {} 2022-09-03T06:44:14.763 DEBUG:boto:Token: None 2022-09-03T06:44:14.764 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/ 2022-09-03T06:44:14.765 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks= 2022-09-03T06:44:14.766 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks=', 'Content-Length': '0'} 2022-09-03T06:44:14.769 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000023490839125232f7-006312f7be-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.770 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:14.771 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:14.772 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:14.772 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:14.773 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:44:14.774 DEBUG:boto:Method: GET 2022-09-03T06:44:14.774 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:44:14.775 DEBUG:boto:Data: 2022-09-03T06:44:14.775 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.776 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.776 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.777 DEBUG:boto:Params: {} 2022-09-03T06:44:14.778 DEBUG:boto:Token: None 2022-09-03T06:44:14.778 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/ 2022-09-03T06:44:14.779 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks= 2022-09-03T06:44:14.780 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks=', 'Content-Length': '0'} 2022-09-03T06:44:14.783 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f40928a2ba74f492-006312f7be-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.784 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:14.785 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:44:14.785 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:14.786 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:14.786 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.787 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.787 DEBUG:boto:Method: GET 2022-09-03T06:44:14.788 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.788 DEBUG:boto:Data: 2022-09-03T06:44:14.789 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.789 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.790 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.791 DEBUG:boto:Params: {} 2022-09-03T06:44:14.791 DEBUG:boto:Token: None 2022-09-03T06:44:14.792 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.793 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:99qMHfwThdeDlcWFNCSEj1XldLk= 2022-09-03T06:44:14.793 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:99qMHfwThdeDlcWFNCSEj1XldLk=', 'Content-Length': '0'} 2022-09-03T06:44:14.796 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000050b56570a7593d66-006312f7be-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.796 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:14.797 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:14.798 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.798 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.799 DEBUG:boto:Method: GET 2022-09-03T06:44:14.800 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.801 DEBUG:boto:Data: 2022-09-03T06:44:14.802 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.803 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.804 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.805 DEBUG:boto:Params: {} 2022-09-03T06:44:14.806 DEBUG:boto:Token: None 2022-09-03T06:44:14.807 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.807 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:99qMHfwThdeDlcWFNCSEj1XldLk= 2022-09-03T06:44:14.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:99qMHfwThdeDlcWFNCSEj1XldLk=', 'Content-Length': '0'} 2022-09-03T06:44:14.821 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c8f554e5a15f5975-006312f7be-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.823 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:14.824 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:14.825 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.825 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.826 DEBUG:boto:Method: HEAD 2022-09-03T06:44:14.827 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.828 DEBUG:boto:Data: 2022-09-03T06:44:14.829 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.830 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.831 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.832 DEBUG:boto:Params: {} 2022-09-03T06:44:14.833 DEBUG:boto:Token: None 2022-09-03T06:44:14.834 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.835 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:blrn6y8eCDWvUt5cprirKYg4+HI= 2022-09-03T06:44:14.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:blrn6y8eCDWvUt5cprirKYg4+HI=', 'Content-Length': '0'} 2022-09-03T06:44:14.839 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008382ef64845bf2fd-006312f7be-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.840 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:14.840 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:14.841 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.841 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.842 DEBUG:boto:Method: HEAD 2022-09-03T06:44:14.843 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.843 DEBUG:boto:Data: 2022-09-03T06:44:14.844 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.844 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.845 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.845 DEBUG:boto:Params: {} 2022-09-03T06:44:14.846 DEBUG:boto:Token: None 2022-09-03T06:44:14.847 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.847 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:blrn6y8eCDWvUt5cprirKYg4+HI= 2022-09-03T06:44:14.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:blrn6y8eCDWvUt5cprirKYg4+HI=', 'Content-Length': '0'} 2022-09-03T06:44:14.850 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a767f23d9504372e-006312f7be-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.851 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:44:14.851 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:14.852 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:14.853 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.853 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.854 DEBUG:boto:Method: GET 2022-09-03T06:44:14.855 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.855 DEBUG:boto:Data: 2022-09-03T06:44:14.856 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.857 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.857 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.858 DEBUG:boto:Params: {} 2022-09-03T06:44:14.859 DEBUG:boto:Token: None 2022-09-03T06:44:14.859 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.860 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:99qMHfwThdeDlcWFNCSEj1XldLk= 2022-09-03T06:44:14.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:99qMHfwThdeDlcWFNCSEj1XldLk=', 'Content-Length': '0'} 2022-09-03T06:44:14.863 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006764be59708884ab-006312f7be-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.864 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:14.864 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:14.865 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.865 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.866 DEBUG:boto:Method: GET 2022-09-03T06:44:14.867 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.867 DEBUG:boto:Data: 2022-09-03T06:44:14.868 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.868 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.869 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.869 DEBUG:boto:Params: {} 2022-09-03T06:44:14.870 DEBUG:boto:Token: None 2022-09-03T06:44:14.871 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:44:14.871 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:99qMHfwThdeDlcWFNCSEj1XldLk= 2022-09-03T06:44:14.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:99qMHfwThdeDlcWFNCSEj1XldLk=', 'Content-Length': '0'} 2022-09-03T06:44:14.874 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bf69e12082d1c812-006312f7be-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.918 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:14.919 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:14.920 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:14.920 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:14.921 DEBUG:boto:Method: GET 2022-09-03T06:44:14.921 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:44:14.922 DEBUG:boto:Data: 2022-09-03T06:44:14.923 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.923 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.924 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.925 DEBUG:boto:Params: {} 2022-09-03T06:44:14.925 DEBUG:boto:Token: None 2022-09-03T06:44:14.926 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/ 2022-09-03T06:44:14.927 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks= 2022-09-03T06:44:14.927 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks=', 'Content-Length': '0'} 2022-09-03T06:44:14.930 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009b6105948fe90558-006312f7be-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.931 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:14.932 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:14.933 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:14.933 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:14.934 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:14.934 DEBUG:boto:Method: GET 2022-09-03T06:44:14.935 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:44:14.936 DEBUG:boto:Data: 2022-09-03T06:44:14.936 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.937 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.938 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.939 DEBUG:boto:Params: {} 2022-09-03T06:44:14.940 DEBUG:boto:Token: None 2022-09-03T06:44:14.940 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/ 2022-09-03T06:44:14.941 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks= 2022-09-03T06:44:14.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E3QUCn5F9yug4LQq/zz1gu8WAks=', 'Content-Length': '0'} 2022-09-03T06:44:14.944 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000044b04ef5135f7edf-006312f7be-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.945 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:14.946 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:14.947 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:14.947 DEBUG:boto:Method: HEAD 2022-09-03T06:44:14.949 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:44:14.949 DEBUG:boto:Data: 2022-09-03T06:44:14.950 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.950 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.951 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.951 DEBUG:boto:Params: {} 2022-09-03T06:44:14.952 DEBUG:boto:Token: None 2022-09-03T06:44:14.952 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj38 2022-09-03T06:44:14.953 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+T2zjZecZgg90CbsgTD6xNP+FY0= 2022-09-03T06:44:14.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+T2zjZecZgg90CbsgTD6xNP+FY0=', 'Content-Length': '0'} 2022-09-03T06:44:14.956 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000134458440aeca511-006312f7be-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.956 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:14.957 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:14.958 DEBUG:boto:Method: HEAD 2022-09-03T06:44:14.959 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:44:14.959 DEBUG:boto:Data: 2022-09-03T06:44:14.960 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.960 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:14.961 DEBUG:boto:Port: 8000 2022-09-03T06:44:14.961 DEBUG:boto:Params: {} 2022-09-03T06:44:14.962 DEBUG:boto:Token: None 2022-09-03T06:44:14.962 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj38 2022-09-03T06:44:14.963 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+T2zjZecZgg90CbsgTD6xNP+FY0= 2022-09-03T06:44:14.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+T2zjZecZgg90CbsgTD6xNP+FY0=', 'Content-Length': '0'} 2022-09-03T06:44:14.966 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000490338d441043ac4-006312f7be-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:14.967 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:44:14.968 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:14.968 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:14.969 DEBUG:boto:Method: GET 2022-09-03T06:44:14.969 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:44:14.970 DEBUG:boto:Data: 2022-09-03T06:44:14.971 DEBUG:boto:Headers: {} 2022-09-03T06:44:14.972 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:14.972 DEBUG:boto:Port: 8001 2022-09-03T06:44:14.973 DEBUG:boto:Params: {} 2022-09-03T06:44:14.973 DEBUG:boto:Token: None 2022-09-03T06:44:14.974 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:14 GMT /dsgvcy-75/obj38 2022-09-03T06:44:14.975 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:65FFNjl7VxaL94yMPEvki758GXU= 2022-09-03T06:44:14.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:65FFNjl7VxaL94yMPEvki758GXU=', 'Content-Length': '0'} 2022-09-03T06:44:14.978 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000b1a7d43c3c20d0c-006312f7be-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.031 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:44:15.031 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:44:15.032 DEBUG:boto:Method: GET 2022-09-03T06:44:15.033 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:44:15.033 DEBUG:boto:Data: 2022-09-03T06:44:15.034 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.035 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.035 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.036 DEBUG:boto:Params: {} 2022-09-03T06:44:15.037 DEBUG:boto:Token: None 2022-09-03T06:44:15.037 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj38 2022-09-03T06:44:15.038 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z9UcDyIRZHXh5p22hPiBTkgwF4Q= 2022-09-03T06:44:15.039 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z9UcDyIRZHXh5p22hPiBTkgwF4Q=', 'Content-Length': '0'} 2022-09-03T06:44:15.041 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002990b6a4114b6556-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.042 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:15.043 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:15.043 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:15.044 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:15.045 DEBUG:boto:Method: GET 2022-09-03T06:44:15.045 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:44:15.046 DEBUG:boto:Data: 2022-09-03T06:44:15.046 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.047 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.047 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.048 DEBUG:boto:Params: {} 2022-09-03T06:44:15.049 DEBUG:boto:Token: None 2022-09-03T06:44:15.049 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/ 2022-09-03T06:44:15.050 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw= 2022-09-03T06:44:15.051 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw=', 'Content-Length': '0'} 2022-09-03T06:44:15.053 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b715dbbf9bc74cd7-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.054 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:15.055 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:15.056 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:15.057 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:15.058 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:44:15.058 DEBUG:boto:Method: GET 2022-09-03T06:44:15.059 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:44:15.060 DEBUG:boto:Data: 2022-09-03T06:44:15.061 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.062 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.063 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.064 DEBUG:boto:Params: {} 2022-09-03T06:44:15.065 DEBUG:boto:Token: None 2022-09-03T06:44:15.066 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/ 2022-09-03T06:44:15.067 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw= 2022-09-03T06:44:15.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw=', 'Content-Length': '0'} 2022-09-03T06:44:15.071 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006e3ac69a70a03178-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.071 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:15.072 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:44:15.073 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:15.073 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:15.074 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.075 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.075 DEBUG:boto:Method: GET 2022-09-03T06:44:15.076 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.076 DEBUG:boto:Data: 2022-09-03T06:44:15.077 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.077 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.078 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.078 DEBUG:boto:Params: {} 2022-09-03T06:44:15.079 DEBUG:boto:Token: None 2022-09-03T06:44:15.080 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.080 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ghK9ET23tTy2kH3JQhkZ9n7o7no= 2022-09-03T06:44:15.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ghK9ET23tTy2kH3JQhkZ9n7o7no=', 'Content-Length': '0'} 2022-09-03T06:44:15.083 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000092a238a065e0e1d8-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.084 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:15.085 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:15.086 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.086 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.087 DEBUG:boto:Method: GET 2022-09-03T06:44:15.087 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.088 DEBUG:boto:Data: 2022-09-03T06:44:15.089 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.089 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.090 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.090 DEBUG:boto:Params: {} 2022-09-03T06:44:15.091 DEBUG:boto:Token: None 2022-09-03T06:44:15.092 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.093 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ghK9ET23tTy2kH3JQhkZ9n7o7no= 2022-09-03T06:44:15.093 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ghK9ET23tTy2kH3JQhkZ9n7o7no=', 'Content-Length': '0'} 2022-09-03T06:44:15.095 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000413aa2dddc4bb607-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.096 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:15.097 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:15.098 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.098 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.099 DEBUG:boto:Method: HEAD 2022-09-03T06:44:15.100 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.100 DEBUG:boto:Data: 2022-09-03T06:44:15.101 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.102 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.102 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.103 DEBUG:boto:Params: {} 2022-09-03T06:44:15.103 DEBUG:boto:Token: None 2022-09-03T06:44:15.104 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.105 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ykWZZPihTiEPwTx+Ibzbr+7ihYI= 2022-09-03T06:44:15.105 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ykWZZPihTiEPwTx+Ibzbr+7ihYI=', 'Content-Length': '0'} 2022-09-03T06:44:15.108 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f0787e912b9c91b5-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.109 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:15.110 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:15.110 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.111 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.111 DEBUG:boto:Method: HEAD 2022-09-03T06:44:15.112 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.112 DEBUG:boto:Data: 2022-09-03T06:44:15.113 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.113 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.114 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.114 DEBUG:boto:Params: {} 2022-09-03T06:44:15.115 DEBUG:boto:Token: None 2022-09-03T06:44:15.116 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.116 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ykWZZPihTiEPwTx+Ibzbr+7ihYI= 2022-09-03T06:44:15.117 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ykWZZPihTiEPwTx+Ibzbr+7ihYI=', 'Content-Length': '0'} 2022-09-03T06:44:15.119 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c8bdfb513e94c5e-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.121 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:44:15.121 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:15.122 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:15.123 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.123 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.124 DEBUG:boto:Method: GET 2022-09-03T06:44:15.124 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.125 DEBUG:boto:Data: 2022-09-03T06:44:15.125 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.126 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.127 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.127 DEBUG:boto:Params: {} 2022-09-03T06:44:15.128 DEBUG:boto:Token: None 2022-09-03T06:44:15.128 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.129 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ghK9ET23tTy2kH3JQhkZ9n7o7no= 2022-09-03T06:44:15.130 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ghK9ET23tTy2kH3JQhkZ9n7o7no=', 'Content-Length': '0'} 2022-09-03T06:44:15.133 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f6f7b0a7a3029660-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.134 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:15.134 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:15.135 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.136 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.136 DEBUG:boto:Method: GET 2022-09-03T06:44:15.137 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.138 DEBUG:boto:Data: 2022-09-03T06:44:15.139 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.140 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.141 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.142 DEBUG:boto:Params: {} 2022-09-03T06:44:15.144 DEBUG:boto:Token: None 2022-09-03T06:44:15.145 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:44:15.146 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ghK9ET23tTy2kH3JQhkZ9n7o7no= 2022-09-03T06:44:15.147 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ghK9ET23tTy2kH3JQhkZ9n7o7no=', 'Content-Length': '0'} 2022-09-03T06:44:15.149 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048ab0002027057b6-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.150 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:15.151 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:15.152 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:15.152 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:15.153 DEBUG:boto:Method: GET 2022-09-03T06:44:15.153 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:44:15.154 DEBUG:boto:Data: 2022-09-03T06:44:15.154 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.155 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.155 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.156 DEBUG:boto:Params: {} 2022-09-03T06:44:15.157 DEBUG:boto:Token: None 2022-09-03T06:44:15.157 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/ 2022-09-03T06:44:15.158 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw= 2022-09-03T06:44:15.158 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw=', 'Content-Length': '0'} 2022-09-03T06:44:15.163 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007c0f55cc35583a1c-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.164 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:15.165 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:15.166 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:15.167 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:15.167 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:15.168 DEBUG:boto:Method: GET 2022-09-03T06:44:15.168 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:44:15.169 DEBUG:boto:Data: 2022-09-03T06:44:15.170 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.171 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.172 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.173 DEBUG:boto:Params: {} 2022-09-03T06:44:15.174 DEBUG:boto:Token: None 2022-09-03T06:44:15.175 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/ 2022-09-03T06:44:15.176 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw= 2022-09-03T06:44:15.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw=', 'Content-Length': '0'} 2022-09-03T06:44:15.180 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000073b68e755dad0d66-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.181 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:15.181 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:15.182 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:15.183 DEBUG:boto:Method: HEAD 2022-09-03T06:44:15.183 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:44:15.184 DEBUG:boto:Data: 2022-09-03T06:44:15.184 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.185 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.185 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.186 DEBUG:boto:Params: {} 2022-09-03T06:44:15.187 DEBUG:boto:Token: None 2022-09-03T06:44:15.187 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj44 2022-09-03T06:44:15.188 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+aliMrrXrD2Ty9bvNnfzUBpjUG0= 2022-09-03T06:44:15.189 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+aliMrrXrD2Ty9bvNnfzUBpjUG0=', 'Content-Length': '0'} 2022-09-03T06:44:15.192 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000466f86366debfe19-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.193 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:15.194 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:15.195 DEBUG:boto:Method: HEAD 2022-09-03T06:44:15.196 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:44:15.196 DEBUG:boto:Data: 2022-09-03T06:44:15.197 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.198 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.199 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.200 DEBUG:boto:Params: {} 2022-09-03T06:44:15.200 DEBUG:boto:Token: None 2022-09-03T06:44:15.201 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj44 2022-09-03T06:44:15.202 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+aliMrrXrD2Ty9bvNnfzUBpjUG0= 2022-09-03T06:44:15.203 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+aliMrrXrD2Ty9bvNnfzUBpjUG0=', 'Content-Length': '0'} 2022-09-03T06:44:15.205 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bbe7b98ee4f29e44-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.207 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:44:15.208 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:15.209 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:15.210 DEBUG:boto:Method: GET 2022-09-03T06:44:15.210 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:44:15.211 DEBUG:boto:Data: 2022-09-03T06:44:15.212 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.213 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.213 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.214 DEBUG:boto:Params: {} 2022-09-03T06:44:15.214 DEBUG:boto:Token: None 2022-09-03T06:44:15.215 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj44 2022-09-03T06:44:15.215 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QFpTeocwtX1WF3EMcFmH/lyxPhE= 2022-09-03T06:44:15.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QFpTeocwtX1WF3EMcFmH/lyxPhE=', 'Content-Length': '0'} 2022-09-03T06:44:15.218 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000056f15b60a3882b2b-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.219 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:44:15.220 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:44:15.220 DEBUG:boto:Method: GET 2022-09-03T06:44:15.221 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:44:15.222 DEBUG:boto:Data: 2022-09-03T06:44:15.223 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.225 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.225 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.226 DEBUG:boto:Params: {} 2022-09-03T06:44:15.227 DEBUG:boto:Token: None 2022-09-03T06:44:15.227 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj44 2022-09-03T06:44:15.228 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QFpTeocwtX1WF3EMcFmH/lyxPhE= 2022-09-03T06:44:15.228 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QFpTeocwtX1WF3EMcFmH/lyxPhE=', 'Content-Length': '0'} 2022-09-03T06:44:15.231 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f606b5c88cf9f3f3-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.232 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:15.232 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:15.233 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:15.234 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:15.235 DEBUG:boto:Method: GET 2022-09-03T06:44:15.235 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:44:15.236 DEBUG:boto:Data: 2022-09-03T06:44:15.236 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.237 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.237 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.238 DEBUG:boto:Params: {} 2022-09-03T06:44:15.238 DEBUG:boto:Token: None 2022-09-03T06:44:15.239 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/ 2022-09-03T06:44:15.240 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw= 2022-09-03T06:44:15.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw=', 'Content-Length': '0'} 2022-09-03T06:44:15.243 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006ec53ab5f20fd479-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.286 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:15.288 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:15.289 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:15.290 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:15.291 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:44:15.292 DEBUG:boto:Method: GET 2022-09-03T06:44:15.293 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:44:15.294 DEBUG:boto:Data: 2022-09-03T06:44:15.295 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.296 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.297 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.298 DEBUG:boto:Params: {} 2022-09-03T06:44:15.299 DEBUG:boto:Token: None 2022-09-03T06:44:15.300 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/ 2022-09-03T06:44:15.301 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw= 2022-09-03T06:44:15.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw=', 'Content-Length': '0'} 2022-09-03T06:44:15.305 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f4022d1a25b923f0-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.306 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:15.307 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:44:15.308 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:15.308 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:15.309 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.309 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.310 DEBUG:boto:Method: GET 2022-09-03T06:44:15.310 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.311 DEBUG:boto:Data: 2022-09-03T06:44:15.312 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.312 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.313 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.313 DEBUG:boto:Params: {} 2022-09-03T06:44:15.314 DEBUG:boto:Token: None 2022-09-03T06:44:15.315 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.316 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:baijtFNsYbf4KEDEAobQ/eX/A1Y= 2022-09-03T06:44:15.316 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:baijtFNsYbf4KEDEAobQ/eX/A1Y=', 'Content-Length': '0'} 2022-09-03T06:44:15.318 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009eff1b8900d77f3b-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.319 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:15.320 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:15.321 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.321 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.322 DEBUG:boto:Method: GET 2022-09-03T06:44:15.323 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.323 DEBUG:boto:Data: 2022-09-03T06:44:15.324 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.324 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.325 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.325 DEBUG:boto:Params: {} 2022-09-03T06:44:15.326 DEBUG:boto:Token: None 2022-09-03T06:44:15.327 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.327 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:baijtFNsYbf4KEDEAobQ/eX/A1Y= 2022-09-03T06:44:15.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:baijtFNsYbf4KEDEAobQ/eX/A1Y=', 'Content-Length': '0'} 2022-09-03T06:44:15.330 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004264bce23dc78a06-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.331 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:15.332 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:15.332 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.333 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.333 DEBUG:boto:Method: HEAD 2022-09-03T06:44:15.334 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.335 DEBUG:boto:Data: 2022-09-03T06:44:15.335 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.336 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.336 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.337 DEBUG:boto:Params: {} 2022-09-03T06:44:15.337 DEBUG:boto:Token: None 2022-09-03T06:44:15.338 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.339 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xwANpoIfRpmuND52TqDFx20S3To= 2022-09-03T06:44:15.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xwANpoIfRpmuND52TqDFx20S3To=', 'Content-Length': '0'} 2022-09-03T06:44:15.342 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7bed916e644e306-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.343 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:15.343 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:15.344 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.344 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.345 DEBUG:boto:Method: HEAD 2022-09-03T06:44:15.345 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.346 DEBUG:boto:Data: 2022-09-03T06:44:15.346 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.347 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.347 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.348 DEBUG:boto:Params: {} 2022-09-03T06:44:15.349 DEBUG:boto:Token: None 2022-09-03T06:44:15.349 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.350 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xwANpoIfRpmuND52TqDFx20S3To= 2022-09-03T06:44:15.351 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xwANpoIfRpmuND52TqDFx20S3To=', 'Content-Length': '0'} 2022-09-03T06:44:15.353 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007d11309f72db7522-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.354 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:44:15.355 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:15.356 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:15.357 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.357 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.359 DEBUG:boto:Method: GET 2022-09-03T06:44:15.359 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.360 DEBUG:boto:Data: 2022-09-03T06:44:15.361 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.362 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.363 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.364 DEBUG:boto:Params: {} 2022-09-03T06:44:15.365 DEBUG:boto:Token: None 2022-09-03T06:44:15.365 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:baijtFNsYbf4KEDEAobQ/eX/A1Y= 2022-09-03T06:44:15.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:baijtFNsYbf4KEDEAobQ/eX/A1Y=', 'Content-Length': '0'} 2022-09-03T06:44:15.369 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dbad7c20c6ae365c-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.410 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:15.411 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:15.412 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.412 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.413 DEBUG:boto:Method: GET 2022-09-03T06:44:15.415 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.415 DEBUG:boto:Data: 2022-09-03T06:44:15.416 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.416 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.417 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.418 DEBUG:boto:Params: {} 2022-09-03T06:44:15.418 DEBUG:boto:Token: None 2022-09-03T06:44:15.419 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:44:15.420 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:baijtFNsYbf4KEDEAobQ/eX/A1Y= 2022-09-03T06:44:15.421 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:baijtFNsYbf4KEDEAobQ/eX/A1Y=', 'Content-Length': '0'} 2022-09-03T06:44:15.425 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000546a049626ff728a-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.426 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:15.427 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:15.427 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:15.428 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:15.429 DEBUG:boto:Method: GET 2022-09-03T06:44:15.430 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:44:15.431 DEBUG:boto:Data: 2022-09-03T06:44:15.432 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.433 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.434 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.436 DEBUG:boto:Params: {} 2022-09-03T06:44:15.437 DEBUG:boto:Token: None 2022-09-03T06:44:15.439 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/ 2022-09-03T06:44:15.440 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw= 2022-09-03T06:44:15.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw=', 'Content-Length': '0'} 2022-09-03T06:44:15.444 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bb676cbd188d34b2-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.445 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:15.446 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:15.447 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:15.448 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:15.448 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:15.449 DEBUG:boto:Method: GET 2022-09-03T06:44:15.449 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:44:15.450 DEBUG:boto:Data: 2022-09-03T06:44:15.451 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.452 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.452 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.453 DEBUG:boto:Params: {} 2022-09-03T06:44:15.453 DEBUG:boto:Token: None 2022-09-03T06:44:15.454 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/ 2022-09-03T06:44:15.455 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw= 2022-09-03T06:44:15.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw=', 'Content-Length': '0'} 2022-09-03T06:44:15.458 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c4267caf64f0f145-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.459 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:15.460 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:15.461 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:15.463 DEBUG:boto:Method: HEAD 2022-09-03T06:44:15.463 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:44:15.464 DEBUG:boto:Data: 2022-09-03T06:44:15.465 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.465 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.466 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.466 DEBUG:boto:Params: {} 2022-09-03T06:44:15.467 DEBUG:boto:Token: None 2022-09-03T06:44:15.468 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj46 2022-09-03T06:44:15.469 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BLqlSaGfJno3I1aBamQMkzKyhr4= 2022-09-03T06:44:15.469 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BLqlSaGfJno3I1aBamQMkzKyhr4=', 'Content-Length': '0'} 2022-09-03T06:44:15.471 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b7fcdee195352c5-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.472 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:15.474 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:15.474 DEBUG:boto:Method: HEAD 2022-09-03T06:44:15.475 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:44:15.475 DEBUG:boto:Data: 2022-09-03T06:44:15.476 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.476 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.477 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.477 DEBUG:boto:Params: {} 2022-09-03T06:44:15.478 DEBUG:boto:Token: None 2022-09-03T06:44:15.479 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj46 2022-09-03T06:44:15.479 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BLqlSaGfJno3I1aBamQMkzKyhr4= 2022-09-03T06:44:15.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BLqlSaGfJno3I1aBamQMkzKyhr4=', 'Content-Length': '0'} 2022-09-03T06:44:15.482 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000041b480f35c2ae93b-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.483 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:44:15.484 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:15.485 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:15.485 DEBUG:boto:Method: GET 2022-09-03T06:44:15.487 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:44:15.487 DEBUG:boto:Data: 2022-09-03T06:44:15.489 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.490 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.491 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.492 DEBUG:boto:Params: {} 2022-09-03T06:44:15.494 DEBUG:boto:Token: None 2022-09-03T06:44:15.495 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj46 2022-09-03T06:44:15.495 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mbc49V/U/edXQdSFJDSqdS/O2Ls= 2022-09-03T06:44:15.496 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mbc49V/U/edXQdSFJDSqdS/O2Ls=', 'Content-Length': '0'} 2022-09-03T06:44:15.498 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006c60da21e3a02b3d-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.499 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:44:15.500 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:44:15.500 DEBUG:boto:Method: GET 2022-09-03T06:44:15.501 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:44:15.501 DEBUG:boto:Data: 2022-09-03T06:44:15.502 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.502 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.503 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.504 DEBUG:boto:Params: {} 2022-09-03T06:44:15.505 DEBUG:boto:Token: None 2022-09-03T06:44:15.505 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj46 2022-09-03T06:44:15.506 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mbc49V/U/edXQdSFJDSqdS/O2Ls= 2022-09-03T06:44:15.507 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mbc49V/U/edXQdSFJDSqdS/O2Ls=', 'Content-Length': '0'} 2022-09-03T06:44:15.510 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca4501a8c7cf142b-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.514 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:15.515 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:15.516 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:15.517 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:15.518 DEBUG:boto:Method: GET 2022-09-03T06:44:15.518 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:44:15.519 DEBUG:boto:Data: 2022-09-03T06:44:15.520 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.521 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.522 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.522 DEBUG:boto:Params: {} 2022-09-03T06:44:15.523 DEBUG:boto:Token: None 2022-09-03T06:44:15.524 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/ 2022-09-03T06:44:15.524 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw= 2022-09-03T06:44:15.525 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw=', 'Content-Length': '0'} 2022-09-03T06:44:15.528 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fc4ef2a254ff1f69-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.529 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:15.530 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:15.530 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:15.531 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:15.532 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:44:15.533 DEBUG:boto:Method: GET 2022-09-03T06:44:15.533 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:44:15.534 DEBUG:boto:Data: 2022-09-03T06:44:15.534 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.535 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.536 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.536 DEBUG:boto:Params: {} 2022-09-03T06:44:15.537 DEBUG:boto:Token: None 2022-09-03T06:44:15.537 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/ 2022-09-03T06:44:15.538 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw= 2022-09-03T06:44:15.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw=', 'Content-Length': '0'} 2022-09-03T06:44:15.543 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000501087df9a4ffdd3-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.544 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:15.545 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:44:15.546 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:15.547 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:15.547 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.548 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.549 DEBUG:boto:Method: GET 2022-09-03T06:44:15.550 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.550 DEBUG:boto:Data: 2022-09-03T06:44:15.551 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.552 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.553 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.554 DEBUG:boto:Params: {} 2022-09-03T06:44:15.555 DEBUG:boto:Token: None 2022-09-03T06:44:15.556 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.557 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LcXuY0tOpfSzb455WkOB3RdBRpk= 2022-09-03T06:44:15.558 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LcXuY0tOpfSzb455WkOB3RdBRpk=', 'Content-Length': '0'} 2022-09-03T06:44:15.561 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000da1db1073e2f72e3-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.563 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:15.564 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:15.565 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.565 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.567 DEBUG:boto:Method: GET 2022-09-03T06:44:15.568 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.569 DEBUG:boto:Data: 2022-09-03T06:44:15.570 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.571 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.572 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.573 DEBUG:boto:Params: {} 2022-09-03T06:44:15.574 DEBUG:boto:Token: None 2022-09-03T06:44:15.575 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.576 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LcXuY0tOpfSzb455WkOB3RdBRpk= 2022-09-03T06:44:15.577 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LcXuY0tOpfSzb455WkOB3RdBRpk=', 'Content-Length': '0'} 2022-09-03T06:44:15.579 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000032ad57c5da066883-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.622 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:15.623 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:15.624 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.625 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.626 DEBUG:boto:Method: HEAD 2022-09-03T06:44:15.626 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.627 DEBUG:boto:Data: 2022-09-03T06:44:15.627 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.628 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.628 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.629 DEBUG:boto:Params: {} 2022-09-03T06:44:15.630 DEBUG:boto:Token: None 2022-09-03T06:44:15.630 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.631 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q648LTQfh3A7fkhujAOF63KB244= 2022-09-03T06:44:15.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q648LTQfh3A7fkhujAOF63KB244=', 'Content-Length': '0'} 2022-09-03T06:44:15.634 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000df3e9e0412f97048-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.635 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:15.635 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:15.636 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.636 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.637 DEBUG:boto:Method: HEAD 2022-09-03T06:44:15.637 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.638 DEBUG:boto:Data: 2022-09-03T06:44:15.639 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.640 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.641 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.643 DEBUG:boto:Params: {} 2022-09-03T06:44:15.644 DEBUG:boto:Token: None 2022-09-03T06:44:15.645 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.646 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q648LTQfh3A7fkhujAOF63KB244= 2022-09-03T06:44:15.647 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q648LTQfh3A7fkhujAOF63KB244=', 'Content-Length': '0'} 2022-09-03T06:44:15.651 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ae4d14fdd408cb66-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.652 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:44:15.653 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:15.654 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:15.654 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.655 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.656 DEBUG:boto:Method: GET 2022-09-03T06:44:15.657 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.658 DEBUG:boto:Data: 2022-09-03T06:44:15.659 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.659 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.660 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.661 DEBUG:boto:Params: {} 2022-09-03T06:44:15.662 DEBUG:boto:Token: None 2022-09-03T06:44:15.662 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.663 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LcXuY0tOpfSzb455WkOB3RdBRpk= 2022-09-03T06:44:15.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LcXuY0tOpfSzb455WkOB3RdBRpk=', 'Content-Length': '0'} 2022-09-03T06:44:15.667 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ed7f2b36b9e63ac-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.714 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:15.715 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:15.715 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.716 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.716 DEBUG:boto:Method: GET 2022-09-03T06:44:15.717 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.718 DEBUG:boto:Data: 2022-09-03T06:44:15.719 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.719 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.720 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.720 DEBUG:boto:Params: {} 2022-09-03T06:44:15.721 DEBUG:boto:Token: None 2022-09-03T06:44:15.726 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:44:15.727 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LcXuY0tOpfSzb455WkOB3RdBRpk= 2022-09-03T06:44:15.727 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LcXuY0tOpfSzb455WkOB3RdBRpk=', 'Content-Length': '0'} 2022-09-03T06:44:15.730 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dff1491034260677-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.731 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:15.731 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:15.732 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:15.732 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:15.733 DEBUG:boto:Method: GET 2022-09-03T06:44:15.733 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:44:15.734 DEBUG:boto:Data: 2022-09-03T06:44:15.734 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.735 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.735 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.736 DEBUG:boto:Params: {} 2022-09-03T06:44:15.737 DEBUG:boto:Token: None 2022-09-03T06:44:15.738 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/ 2022-09-03T06:44:15.739 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw= 2022-09-03T06:44:15.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw=', 'Content-Length': '0'} 2022-09-03T06:44:15.742 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000099332412f4408f2d-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.743 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:15.744 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:15.745 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:15.745 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:15.746 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:15.746 DEBUG:boto:Method: GET 2022-09-03T06:44:15.747 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:44:15.748 DEBUG:boto:Data: 2022-09-03T06:44:15.749 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.750 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.751 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.752 DEBUG:boto:Params: {} 2022-09-03T06:44:15.753 DEBUG:boto:Token: None 2022-09-03T06:44:15.754 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/ 2022-09-03T06:44:15.755 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw= 2022-09-03T06:44:15.756 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw=', 'Content-Length': '0'} 2022-09-03T06:44:15.759 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000173f9aac448adec-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.761 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:15.762 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:15.763 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:15.764 DEBUG:boto:Method: HEAD 2022-09-03T06:44:15.765 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:44:15.766 DEBUG:boto:Data: 2022-09-03T06:44:15.767 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.768 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.768 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.769 DEBUG:boto:Params: {} 2022-09-03T06:44:15.770 DEBUG:boto:Token: None 2022-09-03T06:44:15.770 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj51 2022-09-03T06:44:15.771 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:g9CvkusenONz2gujYO+wXU1mAn8= 2022-09-03T06:44:15.771 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:g9CvkusenONz2gujYO+wXU1mAn8=', 'Content-Length': '0'} 2022-09-03T06:44:15.774 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000094535ee709ac0775-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.774 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:15.775 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:15.775 DEBUG:boto:Method: HEAD 2022-09-03T06:44:15.776 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:44:15.776 DEBUG:boto:Data: 2022-09-03T06:44:15.777 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.777 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.778 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.778 DEBUG:boto:Params: {} 2022-09-03T06:44:15.779 DEBUG:boto:Token: None 2022-09-03T06:44:15.780 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj51 2022-09-03T06:44:15.780 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:g9CvkusenONz2gujYO+wXU1mAn8= 2022-09-03T06:44:15.781 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:g9CvkusenONz2gujYO+wXU1mAn8=', 'Content-Length': '0'} 2022-09-03T06:44:15.783 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000091a1b5f1d8dbcee0-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.783 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:44:15.784 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:15.785 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:15.785 DEBUG:boto:Method: GET 2022-09-03T06:44:15.786 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:44:15.786 DEBUG:boto:Data: 2022-09-03T06:44:15.787 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.788 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.788 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.789 DEBUG:boto:Params: {} 2022-09-03T06:44:15.789 DEBUG:boto:Token: None 2022-09-03T06:44:15.790 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj51 2022-09-03T06:44:15.791 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m6xfZTdu+PWfbnoOSxV8Vo+bK2I= 2022-09-03T06:44:15.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m6xfZTdu+PWfbnoOSxV8Vo+bK2I=', 'Content-Length': '0'} 2022-09-03T06:44:15.794 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e91cb7411416fd09-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.795 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:44:15.795 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:44:15.796 DEBUG:boto:Method: GET 2022-09-03T06:44:15.796 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:44:15.797 DEBUG:boto:Data: 2022-09-03T06:44:15.797 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.798 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.799 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.799 DEBUG:boto:Params: {} 2022-09-03T06:44:15.800 DEBUG:boto:Token: None 2022-09-03T06:44:15.800 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj51 2022-09-03T06:44:15.801 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m6xfZTdu+PWfbnoOSxV8Vo+bK2I= 2022-09-03T06:44:15.802 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m6xfZTdu+PWfbnoOSxV8Vo+bK2I=', 'Content-Length': '0'} 2022-09-03T06:44:15.804 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d1817bbcd1c2e305-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.846 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:15.847 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:15.848 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:15.848 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:15.849 DEBUG:boto:Method: GET 2022-09-03T06:44:15.850 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:44:15.850 DEBUG:boto:Data: 2022-09-03T06:44:15.851 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.852 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.853 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.854 DEBUG:boto:Params: {} 2022-09-03T06:44:15.855 DEBUG:boto:Token: None 2022-09-03T06:44:15.856 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/ 2022-09-03T06:44:15.857 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw= 2022-09-03T06:44:15.857 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw=', 'Content-Length': '0'} 2022-09-03T06:44:15.861 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b1ffca7bd890bf9d-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.861 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:15.862 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:15.863 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:15.864 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:15.864 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:44:15.865 DEBUG:boto:Method: GET 2022-09-03T06:44:15.865 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:44:15.866 DEBUG:boto:Data: 2022-09-03T06:44:15.866 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.867 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.867 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.868 DEBUG:boto:Params: {} 2022-09-03T06:44:15.869 DEBUG:boto:Token: None 2022-09-03T06:44:15.869 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/ 2022-09-03T06:44:15.870 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw= 2022-09-03T06:44:15.871 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gk0kpkV1gMLuEdBJMJKUH9UAmQw=', 'Content-Length': '0'} 2022-09-03T06:44:15.874 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005be3f2ff353e6135-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.876 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:15.877 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:44:15.878 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:15.878 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:15.879 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:15.880 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:15.880 DEBUG:boto:Method: GET 2022-09-03T06:44:15.881 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:15.881 DEBUG:boto:Data: 2022-09-03T06:44:15.883 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.884 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.885 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.886 DEBUG:boto:Params: {} 2022-09-03T06:44:15.886 DEBUG:boto:Token: None 2022-09-03T06:44:15.887 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:15.888 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RLeyfYjPAjcVsBCpAwOjJroJKhE= 2022-09-03T06:44:15.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RLeyfYjPAjcVsBCpAwOjJroJKhE=', 'Content-Length': '0'} 2022-09-03T06:44:15.891 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000005684f1df0a444d-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.892 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:15.892 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:15.893 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:15.894 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:15.894 DEBUG:boto:Method: GET 2022-09-03T06:44:15.895 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:15.895 DEBUG:boto:Data: 2022-09-03T06:44:15.896 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.896 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.897 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.897 DEBUG:boto:Params: {} 2022-09-03T06:44:15.898 DEBUG:boto:Token: None 2022-09-03T06:44:15.899 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:15.899 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RLeyfYjPAjcVsBCpAwOjJroJKhE= 2022-09-03T06:44:15.900 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RLeyfYjPAjcVsBCpAwOjJroJKhE=', 'Content-Length': '0'} 2022-09-03T06:44:15.902 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002eb3d7df077436a9-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.903 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:15.903 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:15.904 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:15.905 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:15.905 DEBUG:boto:Method: HEAD 2022-09-03T06:44:15.906 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:15.907 DEBUG:boto:Data: 2022-09-03T06:44:15.907 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.908 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.908 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.909 DEBUG:boto:Params: {} 2022-09-03T06:44:15.909 DEBUG:boto:Token: None 2022-09-03T06:44:15.910 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:15.911 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z9+LEotcz6y36BPBKdavmux2hic= 2022-09-03T06:44:15.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z9+LEotcz6y36BPBKdavmux2hic=', 'Content-Length': '0'} 2022-09-03T06:44:15.914 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca98e91c4f801664-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.914 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:15.915 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:15.916 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:15.916 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:15.917 DEBUG:boto:Method: HEAD 2022-09-03T06:44:15.917 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:15.918 DEBUG:boto:Data: 2022-09-03T06:44:15.918 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.919 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:15.919 DEBUG:boto:Port: 8000 2022-09-03T06:44:15.920 DEBUG:boto:Params: {} 2022-09-03T06:44:15.920 DEBUG:boto:Token: None 2022-09-03T06:44:15.921 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:15.922 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z9+LEotcz6y36BPBKdavmux2hic= 2022-09-03T06:44:15.923 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z9+LEotcz6y36BPBKdavmux2hic=', 'Content-Length': '0'} 2022-09-03T06:44:15.925 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a58e814ec75f4332-006312f7bf-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.927 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:44:15.928 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:15.929 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:15.929 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:15.930 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:15.931 DEBUG:boto:Method: GET 2022-09-03T06:44:15.932 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:15.933 DEBUG:boto:Data: 2022-09-03T06:44:15.934 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.935 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:15.935 DEBUG:boto:Port: 8001 2022-09-03T06:44:15.936 DEBUG:boto:Params: {} 2022-09-03T06:44:15.937 DEBUG:boto:Token: None 2022-09-03T06:44:15.937 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:15 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:15.938 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RLeyfYjPAjcVsBCpAwOjJroJKhE= 2022-09-03T06:44:15.939 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RLeyfYjPAjcVsBCpAwOjJroJKhE=', 'Content-Length': '0'} 2022-09-03T06:44:15.941 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000022e7f2cc484b3a11-006312f7bf-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:15.994 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:15.995 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:15.996 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:15.996 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:15.997 DEBUG:boto:Method: GET 2022-09-03T06:44:15.998 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:15.998 DEBUG:boto:Data: 2022-09-03T06:44:15.999 DEBUG:boto:Headers: {} 2022-09-03T06:44:15.999 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.000 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.001 DEBUG:boto:Params: {} 2022-09-03T06:44:16.001 DEBUG:boto:Token: None 2022-09-03T06:44:16.002 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:44:16.003 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XRd9KNYreHHY2KOc9fOjYmyjde8= 2022-09-03T06:44:16.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XRd9KNYreHHY2KOc9fOjYmyjde8=', 'Content-Length': '0'} 2022-09-03T06:44:16.006 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000221f9c930a4321ca-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.007 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:16.007 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:16.008 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:16.009 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:16.009 DEBUG:boto:Method: GET 2022-09-03T06:44:16.010 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:44:16.010 DEBUG:boto:Data: 2022-09-03T06:44:16.011 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.011 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.012 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.012 DEBUG:boto:Params: {} 2022-09-03T06:44:16.013 DEBUG:boto:Token: None 2022-09-03T06:44:16.014 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/ 2022-09-03T06:44:16.014 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo= 2022-09-03T06:44:16.015 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo=', 'Content-Length': '0'} 2022-09-03T06:44:16.018 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fcfe1b152953aa5b-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.018 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:16.019 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:16.020 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:16.021 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:16.021 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:16.022 DEBUG:boto:Method: GET 2022-09-03T06:44:16.022 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:44:16.023 DEBUG:boto:Data: 2022-09-03T06:44:16.024 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.024 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.025 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.025 DEBUG:boto:Params: {} 2022-09-03T06:44:16.026 DEBUG:boto:Token: None 2022-09-03T06:44:16.026 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/ 2022-09-03T06:44:16.027 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo= 2022-09-03T06:44:16.028 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo=', 'Content-Length': '0'} 2022-09-03T06:44:16.030 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001ab3913e1b3b549b-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.031 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:16.032 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:16.033 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:16.034 DEBUG:boto:Method: HEAD 2022-09-03T06:44:16.035 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:44:16.035 DEBUG:boto:Data: 2022-09-03T06:44:16.036 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.036 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.037 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.038 DEBUG:boto:Params: {} 2022-09-03T06:44:16.038 DEBUG:boto:Token: None 2022-09-03T06:44:16.039 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj55 2022-09-03T06:44:16.039 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EK/4qfl1wkR005u7EwQWYxAL9SM= 2022-09-03T06:44:16.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EK/4qfl1wkR005u7EwQWYxAL9SM=', 'Content-Length': '0'} 2022-09-03T06:44:16.042 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d5ed22e92f6cffeb-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.043 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:16.043 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:16.044 DEBUG:boto:Method: HEAD 2022-09-03T06:44:16.045 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:44:16.045 DEBUG:boto:Data: 2022-09-03T06:44:16.046 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.046 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.047 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.047 DEBUG:boto:Params: {} 2022-09-03T06:44:16.048 DEBUG:boto:Token: None 2022-09-03T06:44:16.048 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj55 2022-09-03T06:44:16.049 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EK/4qfl1wkR005u7EwQWYxAL9SM= 2022-09-03T06:44:16.050 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EK/4qfl1wkR005u7EwQWYxAL9SM=', 'Content-Length': '0'} 2022-09-03T06:44:16.051 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009968012490a86af2-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.053 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:44:16.054 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:16.054 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:16.055 DEBUG:boto:Method: GET 2022-09-03T06:44:16.056 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:44:16.057 DEBUG:boto:Data: 2022-09-03T06:44:16.057 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.058 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.059 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.060 DEBUG:boto:Params: {} 2022-09-03T06:44:16.060 DEBUG:boto:Token: None 2022-09-03T06:44:16.061 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj55 2022-09-03T06:44:16.062 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tHf01B8cdGU3rW/W6tQ8/XrLM3U= 2022-09-03T06:44:16.062 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tHf01B8cdGU3rW/W6tQ8/XrLM3U=', 'Content-Length': '0'} 2022-09-03T06:44:16.065 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000025c5854cfe4f994c-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.065 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:44:16.066 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:44:16.067 DEBUG:boto:Method: GET 2022-09-03T06:44:16.067 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:44:16.068 DEBUG:boto:Data: 2022-09-03T06:44:16.068 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.069 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.069 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.070 DEBUG:boto:Params: {} 2022-09-03T06:44:16.071 DEBUG:boto:Token: None 2022-09-03T06:44:16.071 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj55 2022-09-03T06:44:16.072 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tHf01B8cdGU3rW/W6tQ8/XrLM3U= 2022-09-03T06:44:16.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tHf01B8cdGU3rW/W6tQ8/XrLM3U=', 'Content-Length': '0'} 2022-09-03T06:44:16.075 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d5d803545a289fc3-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.076 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:16.076 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:16.077 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:16.078 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:16.078 DEBUG:boto:Method: GET 2022-09-03T06:44:16.079 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:44:16.079 DEBUG:boto:Data: 2022-09-03T06:44:16.080 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.080 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.081 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.081 DEBUG:boto:Params: {} 2022-09-03T06:44:16.082 DEBUG:boto:Token: None 2022-09-03T06:44:16.083 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/ 2022-09-03T06:44:16.083 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo= 2022-09-03T06:44:16.084 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo=', 'Content-Length': '0'} 2022-09-03T06:44:16.087 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009c496b54bf27f16c-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.088 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:16.089 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:16.090 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:16.090 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:16.091 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:44:16.092 DEBUG:boto:Method: GET 2022-09-03T06:44:16.093 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:44:16.094 DEBUG:boto:Data: 2022-09-03T06:44:16.095 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.096 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.096 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.097 DEBUG:boto:Params: {} 2022-09-03T06:44:16.098 DEBUG:boto:Token: None 2022-09-03T06:44:16.099 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/ 2022-09-03T06:44:16.100 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo= 2022-09-03T06:44:16.101 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo=', 'Content-Length': '0'} 2022-09-03T06:44:16.106 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007a83ed5eec9c005f-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.107 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:16.108 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:44:16.109 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:16.110 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:16.111 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.112 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.112 DEBUG:boto:Method: GET 2022-09-03T06:44:16.113 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.114 DEBUG:boto:Data: 2022-09-03T06:44:16.115 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.116 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.116 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.118 DEBUG:boto:Params: {} 2022-09-03T06:44:16.119 DEBUG:boto:Token: None 2022-09-03T06:44:16.119 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.120 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0x3iZYFj0/Q5VkQoblt8XT/D7uU= 2022-09-03T06:44:16.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0x3iZYFj0/Q5VkQoblt8XT/D7uU=', 'Content-Length': '0'} 2022-09-03T06:44:16.123 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009fd54ed25b00efd6-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.166 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:16.167 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:16.168 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.169 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.169 DEBUG:boto:Method: GET 2022-09-03T06:44:16.170 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.170 DEBUG:boto:Data: 2022-09-03T06:44:16.171 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.172 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.172 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.173 DEBUG:boto:Params: {} 2022-09-03T06:44:16.174 DEBUG:boto:Token: None 2022-09-03T06:44:16.175 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.175 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0x3iZYFj0/Q5VkQoblt8XT/D7uU= 2022-09-03T06:44:16.176 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0x3iZYFj0/Q5VkQoblt8XT/D7uU=', 'Content-Length': '0'} 2022-09-03T06:44:16.178 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000de1d8d57d3e257c2-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.179 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:16.180 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:16.180 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.181 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.181 DEBUG:boto:Method: HEAD 2022-09-03T06:44:16.182 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.183 DEBUG:boto:Data: 2022-09-03T06:44:16.184 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.184 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.185 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.185 DEBUG:boto:Params: {} 2022-09-03T06:44:16.186 DEBUG:boto:Token: None 2022-09-03T06:44:16.187 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.188 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VN519aDJW0E8acYjR7R39RPNWFc= 2022-09-03T06:44:16.189 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VN519aDJW0E8acYjR7R39RPNWFc=', 'Content-Length': '0'} 2022-09-03T06:44:16.191 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008372300d6d9add20-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.191 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:16.192 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:16.193 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.193 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.194 DEBUG:boto:Method: HEAD 2022-09-03T06:44:16.194 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.195 DEBUG:boto:Data: 2022-09-03T06:44:16.195 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.196 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.196 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.197 DEBUG:boto:Params: {} 2022-09-03T06:44:16.198 DEBUG:boto:Token: None 2022-09-03T06:44:16.199 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.199 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VN519aDJW0E8acYjR7R39RPNWFc= 2022-09-03T06:44:16.201 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VN519aDJW0E8acYjR7R39RPNWFc=', 'Content-Length': '0'} 2022-09-03T06:44:16.203 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bec69aadc692912c-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.203 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:44:16.204 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:16.205 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:16.205 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.206 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.206 DEBUG:boto:Method: GET 2022-09-03T06:44:16.207 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.208 DEBUG:boto:Data: 2022-09-03T06:44:16.209 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.210 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.210 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.211 DEBUG:boto:Params: {} 2022-09-03T06:44:16.211 DEBUG:boto:Token: None 2022-09-03T06:44:16.212 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.213 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0x3iZYFj0/Q5VkQoblt8XT/D7uU= 2022-09-03T06:44:16.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0x3iZYFj0/Q5VkQoblt8XT/D7uU=', 'Content-Length': '0'} 2022-09-03T06:44:16.216 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fa1d8cc8eb995fca-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.217 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:16.218 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:16.218 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.219 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.220 DEBUG:boto:Method: GET 2022-09-03T06:44:16.220 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.221 DEBUG:boto:Data: 2022-09-03T06:44:16.221 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.222 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.223 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.223 DEBUG:boto:Params: {} 2022-09-03T06:44:16.224 DEBUG:boto:Token: None 2022-09-03T06:44:16.224 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:44:16.225 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0x3iZYFj0/Q5VkQoblt8XT/D7uU= 2022-09-03T06:44:16.226 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0x3iZYFj0/Q5VkQoblt8XT/D7uU=', 'Content-Length': '0'} 2022-09-03T06:44:16.227 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000045873d37cd6c6674-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.274 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:16.275 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:16.276 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:16.276 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:16.277 DEBUG:boto:Method: GET 2022-09-03T06:44:16.277 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:44:16.278 DEBUG:boto:Data: 2022-09-03T06:44:16.278 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.279 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.280 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.280 DEBUG:boto:Params: {} 2022-09-03T06:44:16.281 DEBUG:boto:Token: None 2022-09-03T06:44:16.282 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/ 2022-09-03T06:44:16.282 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo= 2022-09-03T06:44:16.283 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo=', 'Content-Length': '0'} 2022-09-03T06:44:16.285 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f5e2ace89204c30e-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.286 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:16.287 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:16.288 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:16.288 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:16.290 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:16.290 DEBUG:boto:Method: GET 2022-09-03T06:44:16.292 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:44:16.293 DEBUG:boto:Data: 2022-09-03T06:44:16.293 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.294 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.295 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.296 DEBUG:boto:Params: {} 2022-09-03T06:44:16.297 DEBUG:boto:Token: None 2022-09-03T06:44:16.298 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/ 2022-09-03T06:44:16.298 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo= 2022-09-03T06:44:16.299 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo=', 'Content-Length': '0'} 2022-09-03T06:44:16.302 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e2f9d600f4be1a3e-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.304 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:16.305 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:16.306 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:16.307 DEBUG:boto:Method: HEAD 2022-09-03T06:44:16.308 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:44:16.309 DEBUG:boto:Data: 2022-09-03T06:44:16.310 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.311 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.312 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.313 DEBUG:boto:Params: {} 2022-09-03T06:44:16.314 DEBUG:boto:Token: None 2022-09-03T06:44:16.315 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj57 2022-09-03T06:44:16.316 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:W59JDpXCqcW0sa8YjsuXtbCZoWk= 2022-09-03T06:44:16.317 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:W59JDpXCqcW0sa8YjsuXtbCZoWk=', 'Content-Length': '0'} 2022-09-03T06:44:16.320 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007609d6cb27c3b8ff-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.321 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:16.322 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:16.323 DEBUG:boto:Method: HEAD 2022-09-03T06:44:16.324 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:44:16.325 DEBUG:boto:Data: 2022-09-03T06:44:16.326 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.327 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.328 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.329 DEBUG:boto:Params: {} 2022-09-03T06:44:16.330 DEBUG:boto:Token: None 2022-09-03T06:44:16.331 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj57 2022-09-03T06:44:16.332 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:W59JDpXCqcW0sa8YjsuXtbCZoWk= 2022-09-03T06:44:16.333 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:W59JDpXCqcW0sa8YjsuXtbCZoWk=', 'Content-Length': '0'} 2022-09-03T06:44:16.335 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a04055f13e95dc3a-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.337 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:44:16.338 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:16.339 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:16.340 DEBUG:boto:Method: GET 2022-09-03T06:44:16.341 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:44:16.342 DEBUG:boto:Data: 2022-09-03T06:44:16.343 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.344 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.344 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.345 DEBUG:boto:Params: {} 2022-09-03T06:44:16.347 DEBUG:boto:Token: None 2022-09-03T06:44:16.348 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj57 2022-09-03T06:44:16.349 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RiiVI+SbI1swLY11GNmcCaIntBw= 2022-09-03T06:44:16.350 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RiiVI+SbI1swLY11GNmcCaIntBw=', 'Content-Length': '0'} 2022-09-03T06:44:16.353 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000098a66b56c90af445-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.354 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:44:16.355 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:44:16.356 DEBUG:boto:Method: GET 2022-09-03T06:44:16.357 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:44:16.358 DEBUG:boto:Data: 2022-09-03T06:44:16.359 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.360 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.361 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.362 DEBUG:boto:Params: {} 2022-09-03T06:44:16.363 DEBUG:boto:Token: None 2022-09-03T06:44:16.363 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj57 2022-09-03T06:44:16.364 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RiiVI+SbI1swLY11GNmcCaIntBw= 2022-09-03T06:44:16.365 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RiiVI+SbI1swLY11GNmcCaIntBw=', 'Content-Length': '0'} 2022-09-03T06:44:16.368 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a76b0bca2c08ce6-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.370 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:16.370 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:16.371 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:16.372 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:16.372 DEBUG:boto:Method: GET 2022-09-03T06:44:16.373 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:44:16.374 DEBUG:boto:Data: 2022-09-03T06:44:16.375 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.375 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.376 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.377 DEBUG:boto:Params: {} 2022-09-03T06:44:16.378 DEBUG:boto:Token: None 2022-09-03T06:44:16.378 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/ 2022-09-03T06:44:16.379 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo= 2022-09-03T06:44:16.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo=', 'Content-Length': '0'} 2022-09-03T06:44:16.383 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000443e771b31028b66-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.385 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:16.387 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:16.388 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:16.389 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:16.389 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:44:16.390 DEBUG:boto:Method: GET 2022-09-03T06:44:16.392 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:44:16.393 DEBUG:boto:Data: 2022-09-03T06:44:16.394 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.395 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.396 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.397 DEBUG:boto:Params: {} 2022-09-03T06:44:16.399 DEBUG:boto:Token: None 2022-09-03T06:44:16.400 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/ 2022-09-03T06:44:16.401 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo= 2022-09-03T06:44:16.403 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo=', 'Content-Length': '0'} 2022-09-03T06:44:16.405 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001b467a3d72ed2be7-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.406 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:16.407 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:44:16.408 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:16.409 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:16.410 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.410 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.411 DEBUG:boto:Method: GET 2022-09-03T06:44:16.412 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.412 DEBUG:boto:Data: 2022-09-03T06:44:16.413 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.414 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.416 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.416 DEBUG:boto:Params: {} 2022-09-03T06:44:16.418 DEBUG:boto:Token: None 2022-09-03T06:44:16.419 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.420 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c09y3jwLMFUnugTO8vSiLdshCF8= 2022-09-03T06:44:16.421 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c09y3jwLMFUnugTO8vSiLdshCF8=', 'Content-Length': '0'} 2022-09-03T06:44:16.425 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a90085fe1ebe8a5-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.474 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:16.474 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:16.475 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.476 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.476 DEBUG:boto:Method: GET 2022-09-03T06:44:16.477 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.477 DEBUG:boto:Data: 2022-09-03T06:44:16.478 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.479 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.479 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.480 DEBUG:boto:Params: {} 2022-09-03T06:44:16.481 DEBUG:boto:Token: None 2022-09-03T06:44:16.481 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.482 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c09y3jwLMFUnugTO8vSiLdshCF8= 2022-09-03T06:44:16.483 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c09y3jwLMFUnugTO8vSiLdshCF8=', 'Content-Length': '0'} 2022-09-03T06:44:16.485 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f5a8fe2a7ced05e7-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.486 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:16.487 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:16.487 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.488 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.488 DEBUG:boto:Method: HEAD 2022-09-03T06:44:16.489 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.489 DEBUG:boto:Data: 2022-09-03T06:44:16.490 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.490 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.491 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.491 DEBUG:boto:Params: {} 2022-09-03T06:44:16.492 DEBUG:boto:Token: None 2022-09-03T06:44:16.493 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.493 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4jgBcISDvppzF7oMLylQ4DdDR8I= 2022-09-03T06:44:16.494 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4jgBcISDvppzF7oMLylQ4DdDR8I=', 'Content-Length': '0'} 2022-09-03T06:44:16.497 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000013976f04a795ba8b-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.498 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:16.500 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:16.501 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.502 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.503 DEBUG:boto:Method: HEAD 2022-09-03T06:44:16.504 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.505 DEBUG:boto:Data: 2022-09-03T06:44:16.506 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.507 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.507 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.508 DEBUG:boto:Params: {} 2022-09-03T06:44:16.508 DEBUG:boto:Token: None 2022-09-03T06:44:16.509 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.510 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4jgBcISDvppzF7oMLylQ4DdDR8I= 2022-09-03T06:44:16.510 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4jgBcISDvppzF7oMLylQ4DdDR8I=', 'Content-Length': '0'} 2022-09-03T06:44:16.513 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000036b16db99a705f84-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.513 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:44:16.514 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:16.515 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:16.515 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.516 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.516 DEBUG:boto:Method: GET 2022-09-03T06:44:16.517 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.517 DEBUG:boto:Data: 2022-09-03T06:44:16.518 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.519 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.519 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.520 DEBUG:boto:Params: {} 2022-09-03T06:44:16.520 DEBUG:boto:Token: None 2022-09-03T06:44:16.521 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.522 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c09y3jwLMFUnugTO8vSiLdshCF8= 2022-09-03T06:44:16.522 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c09y3jwLMFUnugTO8vSiLdshCF8=', 'Content-Length': '0'} 2022-09-03T06:44:16.524 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a1c5f3950eeb206-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.525 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:16.526 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:16.527 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.528 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.528 DEBUG:boto:Method: GET 2022-09-03T06:44:16.529 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.530 DEBUG:boto:Data: 2022-09-03T06:44:16.531 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.531 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.532 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.533 DEBUG:boto:Params: {} 2022-09-03T06:44:16.534 DEBUG:boto:Token: None 2022-09-03T06:44:16.534 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:44:16.535 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c09y3jwLMFUnugTO8vSiLdshCF8= 2022-09-03T06:44:16.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c09y3jwLMFUnugTO8vSiLdshCF8=', 'Content-Length': '0'} 2022-09-03T06:44:16.538 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003d93753e419fa206-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.539 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:16.540 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:16.541 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:16.544 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:16.545 DEBUG:boto:Method: GET 2022-09-03T06:44:16.546 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:44:16.547 DEBUG:boto:Data: 2022-09-03T06:44:16.548 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.549 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.551 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.551 DEBUG:boto:Params: {} 2022-09-03T06:44:16.552 DEBUG:boto:Token: None 2022-09-03T06:44:16.553 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/ 2022-09-03T06:44:16.554 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo= 2022-09-03T06:44:16.555 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo=', 'Content-Length': '0'} 2022-09-03T06:44:16.558 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b209aa4ae8f25e7d-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.559 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:16.560 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:16.560 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:16.561 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:16.562 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:16.562 DEBUG:boto:Method: GET 2022-09-03T06:44:16.563 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:44:16.563 DEBUG:boto:Data: 2022-09-03T06:44:16.564 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.565 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.565 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.566 DEBUG:boto:Params: {} 2022-09-03T06:44:16.567 DEBUG:boto:Token: None 2022-09-03T06:44:16.567 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/ 2022-09-03T06:44:16.568 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo= 2022-09-03T06:44:16.569 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo=', 'Content-Length': '0'} 2022-09-03T06:44:16.572 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000060331c1f762ba8e0-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.618 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:16.619 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:16.620 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:16.620 DEBUG:boto:Method: HEAD 2022-09-03T06:44:16.621 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:44:16.622 DEBUG:boto:Data: 2022-09-03T06:44:16.622 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.623 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.624 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.625 DEBUG:boto:Params: {} 2022-09-03T06:44:16.627 DEBUG:boto:Token: None 2022-09-03T06:44:16.628 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj68 2022-09-03T06:44:16.629 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/npxp82kUvUHSnzN6L6+B5V+YUg= 2022-09-03T06:44:16.630 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/npxp82kUvUHSnzN6L6+B5V+YUg=', 'Content-Length': '0'} 2022-09-03T06:44:16.634 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000563985dd5c226991-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.635 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:16.636 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:16.637 DEBUG:boto:Method: HEAD 2022-09-03T06:44:16.638 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:44:16.639 DEBUG:boto:Data: 2022-09-03T06:44:16.641 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.642 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.643 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.644 DEBUG:boto:Params: {} 2022-09-03T06:44:16.645 DEBUG:boto:Token: None 2022-09-03T06:44:16.647 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj68 2022-09-03T06:44:16.648 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/npxp82kUvUHSnzN6L6+B5V+YUg= 2022-09-03T06:44:16.650 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/npxp82kUvUHSnzN6L6+B5V+YUg=', 'Content-Length': '0'} 2022-09-03T06:44:16.652 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000712308b2219555be-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.653 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:44:16.653 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:16.654 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:16.655 DEBUG:boto:Method: GET 2022-09-03T06:44:16.655 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:44:16.656 DEBUG:boto:Data: 2022-09-03T06:44:16.656 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.657 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.658 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.658 DEBUG:boto:Params: {} 2022-09-03T06:44:16.659 DEBUG:boto:Token: None 2022-09-03T06:44:16.660 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj68 2022-09-03T06:44:16.661 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hr9/eLN61sDa4C38pXex4kWo9EU= 2022-09-03T06:44:16.661 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hr9/eLN61sDa4C38pXex4kWo9EU=', 'Content-Length': '0'} 2022-09-03T06:44:16.664 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b5fea416e995fb84-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.665 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:44:16.665 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:44:16.666 DEBUG:boto:Method: GET 2022-09-03T06:44:16.667 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:44:16.668 DEBUG:boto:Data: 2022-09-03T06:44:16.668 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.669 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.669 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.670 DEBUG:boto:Params: {} 2022-09-03T06:44:16.671 DEBUG:boto:Token: None 2022-09-03T06:44:16.671 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj68 2022-09-03T06:44:16.672 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hr9/eLN61sDa4C38pXex4kWo9EU= 2022-09-03T06:44:16.673 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hr9/eLN61sDa4C38pXex4kWo9EU=', 'Content-Length': '0'} 2022-09-03T06:44:16.675 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000029b26f69dc9820b5-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.676 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:16.676 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:16.677 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:16.678 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:16.678 DEBUG:boto:Method: GET 2022-09-03T06:44:16.679 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:44:16.679 DEBUG:boto:Data: 2022-09-03T06:44:16.680 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.680 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.681 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.681 DEBUG:boto:Params: {} 2022-09-03T06:44:16.682 DEBUG:boto:Token: None 2022-09-03T06:44:16.683 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/ 2022-09-03T06:44:16.683 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo= 2022-09-03T06:44:16.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo=', 'Content-Length': '0'} 2022-09-03T06:44:16.686 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009e853c9e008acdd0-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.687 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:16.688 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:16.689 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:16.689 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:16.690 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:44:16.690 DEBUG:boto:Method: GET 2022-09-03T06:44:16.691 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:44:16.692 DEBUG:boto:Data: 2022-09-03T06:44:16.692 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.693 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.693 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.694 DEBUG:boto:Params: {} 2022-09-03T06:44:16.694 DEBUG:boto:Token: None 2022-09-03T06:44:16.695 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/ 2022-09-03T06:44:16.696 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo= 2022-09-03T06:44:16.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo=', 'Content-Length': '0'} 2022-09-03T06:44:16.699 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000793109dfa03c3f4e-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.700 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:16.701 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:44:16.702 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:16.703 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:16.705 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.705 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.706 DEBUG:boto:Method: GET 2022-09-03T06:44:16.707 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.708 DEBUG:boto:Data: 2022-09-03T06:44:16.709 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.710 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.711 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.712 DEBUG:boto:Params: {} 2022-09-03T06:44:16.713 DEBUG:boto:Token: None 2022-09-03T06:44:16.714 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.715 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BHqyIdMMC2MFH5696U/eiCNH5M8= 2022-09-03T06:44:16.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BHqyIdMMC2MFH5696U/eiCNH5M8=', 'Content-Length': '0'} 2022-09-03T06:44:16.719 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000368e59641ce4be29-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.720 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:16.721 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:16.721 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.722 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.722 DEBUG:boto:Method: GET 2022-09-03T06:44:16.724 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.725 DEBUG:boto:Data: 2022-09-03T06:44:16.725 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.727 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.728 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.728 DEBUG:boto:Params: {} 2022-09-03T06:44:16.729 DEBUG:boto:Token: None 2022-09-03T06:44:16.730 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.730 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BHqyIdMMC2MFH5696U/eiCNH5M8= 2022-09-03T06:44:16.731 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BHqyIdMMC2MFH5696U/eiCNH5M8=', 'Content-Length': '0'} 2022-09-03T06:44:16.733 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004619c911f559a3b4-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.734 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:16.735 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:16.736 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.736 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.737 DEBUG:boto:Method: HEAD 2022-09-03T06:44:16.738 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.739 DEBUG:boto:Data: 2022-09-03T06:44:16.740 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.741 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.742 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.743 DEBUG:boto:Params: {} 2022-09-03T06:44:16.744 DEBUG:boto:Token: None 2022-09-03T06:44:16.745 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.746 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w7k/AIzv5UMR1GPz2ZQWdYwU4qE= 2022-09-03T06:44:16.747 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w7k/AIzv5UMR1GPz2ZQWdYwU4qE=', 'Content-Length': '0'} 2022-09-03T06:44:16.750 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000633dab3b3e745e17-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.751 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:16.751 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:16.752 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.752 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.753 DEBUG:boto:Method: HEAD 2022-09-03T06:44:16.753 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.754 DEBUG:boto:Data: 2022-09-03T06:44:16.755 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.755 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.756 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.756 DEBUG:boto:Params: {} 2022-09-03T06:44:16.757 DEBUG:boto:Token: None 2022-09-03T06:44:16.758 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.758 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w7k/AIzv5UMR1GPz2ZQWdYwU4qE= 2022-09-03T06:44:16.759 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w7k/AIzv5UMR1GPz2ZQWdYwU4qE=', 'Content-Length': '0'} 2022-09-03T06:44:16.761 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009680829843fc2f80-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.762 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:44:16.764 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:16.765 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:16.766 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.767 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.768 DEBUG:boto:Method: GET 2022-09-03T06:44:16.769 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.770 DEBUG:boto:Data: 2022-09-03T06:44:16.771 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.772 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.773 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.774 DEBUG:boto:Params: {} 2022-09-03T06:44:16.775 DEBUG:boto:Token: None 2022-09-03T06:44:16.776 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.777 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BHqyIdMMC2MFH5696U/eiCNH5M8= 2022-09-03T06:44:16.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BHqyIdMMC2MFH5696U/eiCNH5M8=', 'Content-Length': '0'} 2022-09-03T06:44:16.779 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a4cabcec614a1670-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.826 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:16.827 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:16.828 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.829 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.829 DEBUG:boto:Method: GET 2022-09-03T06:44:16.839 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.840 DEBUG:boto:Data: 2022-09-03T06:44:16.841 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.841 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.842 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.842 DEBUG:boto:Params: {} 2022-09-03T06:44:16.843 DEBUG:boto:Token: None 2022-09-03T06:44:16.844 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:44:16.844 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BHqyIdMMC2MFH5696U/eiCNH5M8= 2022-09-03T06:44:16.845 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BHqyIdMMC2MFH5696U/eiCNH5M8=', 'Content-Length': '0'} 2022-09-03T06:44:16.847 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001fa431caaaf1a7c9-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.848 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:16.848 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:16.849 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:16.850 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:16.850 DEBUG:boto:Method: GET 2022-09-03T06:44:16.851 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:44:16.851 DEBUG:boto:Data: 2022-09-03T06:44:16.852 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.853 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.853 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.854 DEBUG:boto:Params: {} 2022-09-03T06:44:16.854 DEBUG:boto:Token: None 2022-09-03T06:44:16.855 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/ 2022-09-03T06:44:16.856 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo= 2022-09-03T06:44:16.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo=', 'Content-Length': '0'} 2022-09-03T06:44:16.859 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f98db97828e41872-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.860 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:16.861 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:16.862 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:16.862 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:16.863 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:16.864 DEBUG:boto:Method: GET 2022-09-03T06:44:16.864 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:44:16.865 DEBUG:boto:Data: 2022-09-03T06:44:16.865 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.866 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.866 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.867 DEBUG:boto:Params: {} 2022-09-03T06:44:16.868 DEBUG:boto:Token: None 2022-09-03T06:44:16.868 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/ 2022-09-03T06:44:16.869 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo= 2022-09-03T06:44:16.870 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo=', 'Content-Length': '0'} 2022-09-03T06:44:16.872 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d1576154ba13c724-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.873 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:16.874 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:16.874 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:16.875 DEBUG:boto:Method: HEAD 2022-09-03T06:44:16.876 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:44:16.876 DEBUG:boto:Data: 2022-09-03T06:44:16.877 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.877 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.878 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.879 DEBUG:boto:Params: {} 2022-09-03T06:44:16.879 DEBUG:boto:Token: None 2022-09-03T06:44:16.880 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj69 2022-09-03T06:44:16.881 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nOYXpUkU2Wx/E5BvnMZh4ieD5TU= 2022-09-03T06:44:16.881 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nOYXpUkU2Wx/E5BvnMZh4ieD5TU=', 'Content-Length': '0'} 2022-09-03T06:44:16.883 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000061010f349517ee6b-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.884 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:16.885 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:16.885 DEBUG:boto:Method: HEAD 2022-09-03T06:44:16.886 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:44:16.887 DEBUG:boto:Data: 2022-09-03T06:44:16.887 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.888 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.889 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.889 DEBUG:boto:Params: {} 2022-09-03T06:44:16.890 DEBUG:boto:Token: None 2022-09-03T06:44:16.890 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj69 2022-09-03T06:44:16.891 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nOYXpUkU2Wx/E5BvnMZh4ieD5TU= 2022-09-03T06:44:16.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nOYXpUkU2Wx/E5BvnMZh4ieD5TU=', 'Content-Length': '0'} 2022-09-03T06:44:16.894 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cb3f6a7bb3d55b2a-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.895 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:44:16.895 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:16.896 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:16.897 DEBUG:boto:Method: GET 2022-09-03T06:44:16.897 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:44:16.898 DEBUG:boto:Data: 2022-09-03T06:44:16.898 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.899 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.900 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.900 DEBUG:boto:Params: {} 2022-09-03T06:44:16.901 DEBUG:boto:Token: None 2022-09-03T06:44:16.902 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj69 2022-09-03T06:44:16.903 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZjxBEBXrE+7mkQ69Jaz2Kgi/XKI= 2022-09-03T06:44:16.903 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZjxBEBXrE+7mkQ69Jaz2Kgi/XKI=', 'Content-Length': '0'} 2022-09-03T06:44:16.906 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a4c1009f631e4124-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.907 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:44:16.908 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:44:16.909 DEBUG:boto:Method: GET 2022-09-03T06:44:16.909 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:44:16.910 DEBUG:boto:Data: 2022-09-03T06:44:16.910 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.911 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.911 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.912 DEBUG:boto:Params: {} 2022-09-03T06:44:16.913 DEBUG:boto:Token: None 2022-09-03T06:44:16.914 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj69 2022-09-03T06:44:16.915 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZjxBEBXrE+7mkQ69Jaz2Kgi/XKI= 2022-09-03T06:44:16.915 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZjxBEBXrE+7mkQ69Jaz2Kgi/XKI=', 'Content-Length': '0'} 2022-09-03T06:44:16.917 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000147e2b797cb8144a-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.919 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:16.919 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:16.920 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:16.921 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:16.921 DEBUG:boto:Method: GET 2022-09-03T06:44:16.922 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:44:16.923 DEBUG:boto:Data: 2022-09-03T06:44:16.923 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.924 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.925 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.926 DEBUG:boto:Params: {} 2022-09-03T06:44:16.927 DEBUG:boto:Token: None 2022-09-03T06:44:16.927 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/ 2022-09-03T06:44:16.928 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo= 2022-09-03T06:44:16.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo=', 'Content-Length': '0'} 2022-09-03T06:44:16.932 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009e4a1416a9b4018e-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.933 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:16.934 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:16.935 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:16.936 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:16.936 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:44:16.937 DEBUG:boto:Method: GET 2022-09-03T06:44:16.937 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:44:16.938 DEBUG:boto:Data: 2022-09-03T06:44:16.939 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.940 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.940 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.941 DEBUG:boto:Params: {} 2022-09-03T06:44:16.942 DEBUG:boto:Token: None 2022-09-03T06:44:16.942 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/ 2022-09-03T06:44:16.943 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo= 2022-09-03T06:44:16.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G/vt+o1mwNe7MDY8gmp+gioRdvo=', 'Content-Length': '0'} 2022-09-03T06:44:16.947 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000964b7fb0942ca039-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.948 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:16.949 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:44:16.949 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:16.950 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:16.950 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:16.951 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:16.952 DEBUG:boto:Method: GET 2022-09-03T06:44:16.952 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:16.953 DEBUG:boto:Data: 2022-09-03T06:44:16.954 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.954 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:16.955 DEBUG:boto:Port: 8001 2022-09-03T06:44:16.956 DEBUG:boto:Params: {} 2022-09-03T06:44:16.956 DEBUG:boto:Token: None 2022-09-03T06:44:16.957 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:16.958 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kGlxQokUisrWki1Xb9KPXgdPq9M= 2022-09-03T06:44:16.959 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kGlxQokUisrWki1Xb9KPXgdPq9M=', 'Content-Length': '0'} 2022-09-03T06:44:16.961 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b79e7e82ae906467-006312f7c0-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:16.962 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:16.963 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:16.963 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:16.964 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:16.965 DEBUG:boto:Method: GET 2022-09-03T06:44:16.965 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:16.966 DEBUG:boto:Data: 2022-09-03T06:44:16.967 DEBUG:boto:Headers: {} 2022-09-03T06:44:16.967 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:16.968 DEBUG:boto:Port: 8000 2022-09-03T06:44:16.968 DEBUG:boto:Params: {} 2022-09-03T06:44:16.969 DEBUG:boto:Token: None 2022-09-03T06:44:16.970 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:16 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:16.970 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kGlxQokUisrWki1Xb9KPXgdPq9M= 2022-09-03T06:44:16.971 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kGlxQokUisrWki1Xb9KPXgdPq9M=', 'Content-Length': '0'} 2022-09-03T06:44:16.973 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c867d73a912a8476-006312f7c0-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.020 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:17.022 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:17.023 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:17.024 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:17.025 DEBUG:boto:Method: HEAD 2022-09-03T06:44:17.026 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:17.027 DEBUG:boto:Data: 2022-09-03T06:44:17.028 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.029 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.030 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.031 DEBUG:boto:Params: {} 2022-09-03T06:44:17.032 DEBUG:boto:Token: None 2022-09-03T06:44:17.033 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:17.034 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qtDyJUsP8HJOfJiI/vxVEzK5flk= 2022-09-03T06:44:17.035 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qtDyJUsP8HJOfJiI/vxVEzK5flk=', 'Content-Length': '0'} 2022-09-03T06:44:17.038 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009801801eb5014d6c-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.043 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:17.044 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:17.046 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:17.047 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:17.048 DEBUG:boto:Method: HEAD 2022-09-03T06:44:17.050 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:17.051 DEBUG:boto:Data: 2022-09-03T06:44:17.052 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.054 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.055 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.056 DEBUG:boto:Params: {} 2022-09-03T06:44:17.057 DEBUG:boto:Token: None 2022-09-03T06:44:17.058 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:17.058 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qtDyJUsP8HJOfJiI/vxVEzK5flk= 2022-09-03T06:44:17.059 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qtDyJUsP8HJOfJiI/vxVEzK5flk=', 'Content-Length': '0'} 2022-09-03T06:44:17.061 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f1c3f1d827693a8f-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.062 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:44:17.062 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:17.064 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:17.064 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:17.065 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:17.065 DEBUG:boto:Method: GET 2022-09-03T06:44:17.066 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:17.068 DEBUG:boto:Data: 2022-09-03T06:44:17.068 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.069 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.070 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.070 DEBUG:boto:Params: {} 2022-09-03T06:44:17.072 DEBUG:boto:Token: None 2022-09-03T06:44:17.072 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:17.073 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fUoaMU80WhseWXQla8MTs+4jeeo= 2022-09-03T06:44:17.074 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fUoaMU80WhseWXQla8MTs+4jeeo=', 'Content-Length': '0'} 2022-09-03T06:44:17.076 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000acf1848994a56ad4-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.077 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:17.077 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:17.078 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:17.078 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:17.079 DEBUG:boto:Method: GET 2022-09-03T06:44:17.080 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:17.080 DEBUG:boto:Data: 2022-09-03T06:44:17.081 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.082 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.082 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.083 DEBUG:boto:Params: {} 2022-09-03T06:44:17.083 DEBUG:boto:Token: None 2022-09-03T06:44:17.084 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:44:17.085 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fUoaMU80WhseWXQla8MTs+4jeeo= 2022-09-03T06:44:17.086 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fUoaMU80WhseWXQla8MTs+4jeeo=', 'Content-Length': '0'} 2022-09-03T06:44:17.088 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004fec839a21b73117-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.090 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:17.091 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:17.092 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:17.093 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:17.093 DEBUG:boto:Method: GET 2022-09-03T06:44:17.094 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:44:17.095 DEBUG:boto:Data: 2022-09-03T06:44:17.096 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.097 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.097 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.098 DEBUG:boto:Params: {} 2022-09-03T06:44:17.099 DEBUG:boto:Token: None 2022-09-03T06:44:17.101 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/ 2022-09-03T06:44:17.103 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck= 2022-09-03T06:44:17.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck=', 'Content-Length': '0'} 2022-09-03T06:44:17.110 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000061c938ec3c4ce4c3-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.112 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:17.113 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:17.114 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:17.115 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:17.115 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:17.116 DEBUG:boto:Method: GET 2022-09-03T06:44:17.116 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:44:17.117 DEBUG:boto:Data: 2022-09-03T06:44:17.118 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.118 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.119 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.119 DEBUG:boto:Params: {} 2022-09-03T06:44:17.120 DEBUG:boto:Token: None 2022-09-03T06:44:17.121 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/ 2022-09-03T06:44:17.122 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck= 2022-09-03T06:44:17.123 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck=', 'Content-Length': '0'} 2022-09-03T06:44:17.126 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b0ebdef0d7468754-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.127 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:17.129 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:17.130 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:17.131 DEBUG:boto:Method: HEAD 2022-09-03T06:44:17.131 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:44:17.132 DEBUG:boto:Data: 2022-09-03T06:44:17.134 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.134 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.135 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.135 DEBUG:boto:Params: {} 2022-09-03T06:44:17.136 DEBUG:boto:Token: None 2022-09-03T06:44:17.136 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj76 2022-09-03T06:44:17.137 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wYwix9VfuCwt4MhvI3LLlDpsrKg= 2022-09-03T06:44:17.138 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wYwix9VfuCwt4MhvI3LLlDpsrKg=', 'Content-Length': '0'} 2022-09-03T06:44:17.140 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000dd6d6ecf015b75a-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.141 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:17.141 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:17.142 DEBUG:boto:Method: HEAD 2022-09-03T06:44:17.142 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:44:17.143 DEBUG:boto:Data: 2022-09-03T06:44:17.144 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.145 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.145 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.146 DEBUG:boto:Params: {} 2022-09-03T06:44:17.148 DEBUG:boto:Token: None 2022-09-03T06:44:17.149 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj76 2022-09-03T06:44:17.150 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wYwix9VfuCwt4MhvI3LLlDpsrKg= 2022-09-03T06:44:17.151 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wYwix9VfuCwt4MhvI3LLlDpsrKg=', 'Content-Length': '0'} 2022-09-03T06:44:17.153 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000916c74970905595b-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.154 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:44:17.156 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:17.157 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:17.158 DEBUG:boto:Method: GET 2022-09-03T06:44:17.159 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:44:17.159 DEBUG:boto:Data: 2022-09-03T06:44:17.160 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.161 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.162 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.163 DEBUG:boto:Params: {} 2022-09-03T06:44:17.165 DEBUG:boto:Token: None 2022-09-03T06:44:17.166 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj76 2022-09-03T06:44:17.167 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/UclEVgbH1BWL2wPJdBYNLstWF8= 2022-09-03T06:44:17.168 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/UclEVgbH1BWL2wPJdBYNLstWF8=', 'Content-Length': '0'} 2022-09-03T06:44:17.170 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048191e9031e7d62c-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.214 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:44:17.215 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:44:17.215 DEBUG:boto:Method: GET 2022-09-03T06:44:17.216 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:44:17.216 DEBUG:boto:Data: 2022-09-03T06:44:17.217 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.218 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.218 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.219 DEBUG:boto:Params: {} 2022-09-03T06:44:17.219 DEBUG:boto:Token: None 2022-09-03T06:44:17.220 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj76 2022-09-03T06:44:17.221 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/UclEVgbH1BWL2wPJdBYNLstWF8= 2022-09-03T06:44:17.221 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/UclEVgbH1BWL2wPJdBYNLstWF8=', 'Content-Length': '0'} 2022-09-03T06:44:17.224 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fb55b3dd81d42a77-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.225 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:17.225 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:17.226 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:17.226 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:17.227 DEBUG:boto:Method: GET 2022-09-03T06:44:17.227 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:44:17.228 DEBUG:boto:Data: 2022-09-03T06:44:17.229 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.230 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.230 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.231 DEBUG:boto:Params: {} 2022-09-03T06:44:17.231 DEBUG:boto:Token: None 2022-09-03T06:44:17.232 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/ 2022-09-03T06:44:17.233 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck= 2022-09-03T06:44:17.233 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck=', 'Content-Length': '0'} 2022-09-03T06:44:17.240 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007a52f31ace62dc85-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.241 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:17.243 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:17.244 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:17.244 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:17.245 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:44:17.246 DEBUG:boto:Method: GET 2022-09-03T06:44:17.247 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:44:17.247 DEBUG:boto:Data: 2022-09-03T06:44:17.248 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.249 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.250 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.250 DEBUG:boto:Params: {} 2022-09-03T06:44:17.251 DEBUG:boto:Token: None 2022-09-03T06:44:17.252 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/ 2022-09-03T06:44:17.252 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck= 2022-09-03T06:44:17.253 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck=', 'Content-Length': '0'} 2022-09-03T06:44:17.255 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000ab5a3ec0b6372da-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.256 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:17.257 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:44:17.257 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:17.258 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:17.259 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.259 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.260 DEBUG:boto:Method: GET 2022-09-03T06:44:17.261 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.262 DEBUG:boto:Data: 2022-09-03T06:44:17.262 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.263 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.263 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.264 DEBUG:boto:Params: {} 2022-09-03T06:44:17.264 DEBUG:boto:Token: None 2022-09-03T06:44:17.265 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.266 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:udMg977rbUH6oR6LJ+F/Ti3StXk= 2022-09-03T06:44:17.267 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:udMg977rbUH6oR6LJ+F/Ti3StXk=', 'Content-Length': '0'} 2022-09-03T06:44:17.269 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a117eb7c4e6d3845-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.271 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:17.272 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:17.273 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.274 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.275 DEBUG:boto:Method: GET 2022-09-03T06:44:17.276 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.277 DEBUG:boto:Data: 2022-09-03T06:44:17.278 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.281 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.282 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.283 DEBUG:boto:Params: {} 2022-09-03T06:44:17.284 DEBUG:boto:Token: None 2022-09-03T06:44:17.285 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.286 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:udMg977rbUH6oR6LJ+F/Ti3StXk= 2022-09-03T06:44:17.287 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:udMg977rbUH6oR6LJ+F/Ti3StXk=', 'Content-Length': '0'} 2022-09-03T06:44:17.289 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a46a0a3dc9e7241c-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.290 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:17.290 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:17.291 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.292 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.292 DEBUG:boto:Method: HEAD 2022-09-03T06:44:17.293 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.293 DEBUG:boto:Data: 2022-09-03T06:44:17.294 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.294 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.295 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.296 DEBUG:boto:Params: {} 2022-09-03T06:44:17.296 DEBUG:boto:Token: None 2022-09-03T06:44:17.297 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.297 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+kv5XxFfHjCwsAvzCsNdp+LG7LU= 2022-09-03T06:44:17.298 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+kv5XxFfHjCwsAvzCsNdp+LG7LU=', 'Content-Length': '0'} 2022-09-03T06:44:17.300 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008653bc2c2d813e0b-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.301 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:17.301 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:17.302 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.303 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.303 DEBUG:boto:Method: HEAD 2022-09-03T06:44:17.304 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.304 DEBUG:boto:Data: 2022-09-03T06:44:17.305 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.305 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.306 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.306 DEBUG:boto:Params: {} 2022-09-03T06:44:17.307 DEBUG:boto:Token: None 2022-09-03T06:44:17.308 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.308 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+kv5XxFfHjCwsAvzCsNdp+LG7LU= 2022-09-03T06:44:17.309 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+kv5XxFfHjCwsAvzCsNdp+LG7LU=', 'Content-Length': '0'} 2022-09-03T06:44:17.311 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e9129fb6e720911b-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.312 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:44:17.313 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:17.314 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:17.314 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.315 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.315 DEBUG:boto:Method: GET 2022-09-03T06:44:17.316 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.316 DEBUG:boto:Data: 2022-09-03T06:44:17.317 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.317 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.318 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.319 DEBUG:boto:Params: {} 2022-09-03T06:44:17.319 DEBUG:boto:Token: None 2022-09-03T06:44:17.320 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.321 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:udMg977rbUH6oR6LJ+F/Ti3StXk= 2022-09-03T06:44:17.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:udMg977rbUH6oR6LJ+F/Ti3StXk=', 'Content-Length': '0'} 2022-09-03T06:44:17.324 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e5b892775a50f92a-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.325 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:17.326 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:17.327 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.328 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.329 DEBUG:boto:Method: GET 2022-09-03T06:44:17.330 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.331 DEBUG:boto:Data: 2022-09-03T06:44:17.332 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.333 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.334 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.335 DEBUG:boto:Params: {} 2022-09-03T06:44:17.336 DEBUG:boto:Token: None 2022-09-03T06:44:17.337 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:44:17.338 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:udMg977rbUH6oR6LJ+F/Ti3StXk= 2022-09-03T06:44:17.342 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:udMg977rbUH6oR6LJ+F/Ti3StXk=', 'Content-Length': '0'} 2022-09-03T06:44:17.344 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000662ff83695e35450-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.345 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:17.346 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:17.347 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:17.348 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:17.349 DEBUG:boto:Method: GET 2022-09-03T06:44:17.350 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:44:17.350 DEBUG:boto:Data: 2022-09-03T06:44:17.352 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.352 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.354 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.355 DEBUG:boto:Params: {} 2022-09-03T06:44:17.355 DEBUG:boto:Token: None 2022-09-03T06:44:17.357 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/ 2022-09-03T06:44:17.358 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck= 2022-09-03T06:44:17.358 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck=', 'Content-Length': '0'} 2022-09-03T06:44:17.361 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ea814ac6f2a73533-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.362 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:17.363 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:17.363 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:17.364 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:17.364 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:17.365 DEBUG:boto:Method: GET 2022-09-03T06:44:17.366 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:44:17.366 DEBUG:boto:Data: 2022-09-03T06:44:17.368 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.369 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.370 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.371 DEBUG:boto:Params: {} 2022-09-03T06:44:17.372 DEBUG:boto:Token: None 2022-09-03T06:44:17.373 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/ 2022-09-03T06:44:17.374 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck= 2022-09-03T06:44:17.375 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck=', 'Content-Length': '0'} 2022-09-03T06:44:17.379 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b3a23fc8738ccee7-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.380 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:17.381 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:17.382 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:17.384 DEBUG:boto:Method: HEAD 2022-09-03T06:44:17.385 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:44:17.386 DEBUG:boto:Data: 2022-09-03T06:44:17.387 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.388 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.388 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.389 DEBUG:boto:Params: {} 2022-09-03T06:44:17.389 DEBUG:boto:Token: None 2022-09-03T06:44:17.390 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj85 2022-09-03T06:44:17.391 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yjMi8GQdi7VgvoxF0115nYnaLK0= 2022-09-03T06:44:17.391 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yjMi8GQdi7VgvoxF0115nYnaLK0=', 'Content-Length': '0'} 2022-09-03T06:44:17.393 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000173ed60bdfb25cf6-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.394 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:17.395 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:17.395 DEBUG:boto:Method: HEAD 2022-09-03T06:44:17.396 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:44:17.396 DEBUG:boto:Data: 2022-09-03T06:44:17.397 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.397 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.398 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.398 DEBUG:boto:Params: {} 2022-09-03T06:44:17.399 DEBUG:boto:Token: None 2022-09-03T06:44:17.399 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj85 2022-09-03T06:44:17.400 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yjMi8GQdi7VgvoxF0115nYnaLK0= 2022-09-03T06:44:17.401 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yjMi8GQdi7VgvoxF0115nYnaLK0=', 'Content-Length': '0'} 2022-09-03T06:44:17.402 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c0749e4dcebac0cc-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.403 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:44:17.404 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:17.404 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:17.405 DEBUG:boto:Method: GET 2022-09-03T06:44:17.406 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:44:17.406 DEBUG:boto:Data: 2022-09-03T06:44:17.407 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.408 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.409 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.410 DEBUG:boto:Params: {} 2022-09-03T06:44:17.411 DEBUG:boto:Token: None 2022-09-03T06:44:17.412 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj85 2022-09-03T06:44:17.413 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jiqZrG/AAvWTC+GVLyA9svK1WXc= 2022-09-03T06:44:17.414 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jiqZrG/AAvWTC+GVLyA9svK1WXc=', 'Content-Length': '0'} 2022-09-03T06:44:17.416 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000511d8d5411de3014-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.418 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:44:17.418 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:44:17.419 DEBUG:boto:Method: GET 2022-09-03T06:44:17.420 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:44:17.420 DEBUG:boto:Data: 2022-09-03T06:44:17.421 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.421 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.422 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.423 DEBUG:boto:Params: {} 2022-09-03T06:44:17.423 DEBUG:boto:Token: None 2022-09-03T06:44:17.424 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj85 2022-09-03T06:44:17.425 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jiqZrG/AAvWTC+GVLyA9svK1WXc= 2022-09-03T06:44:17.426 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jiqZrG/AAvWTC+GVLyA9svK1WXc=', 'Content-Length': '0'} 2022-09-03T06:44:17.428 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000468a0f3d89b62f3a-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.474 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:17.475 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:17.476 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:17.477 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:17.478 DEBUG:boto:Method: GET 2022-09-03T06:44:17.479 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:44:17.480 DEBUG:boto:Data: 2022-09-03T06:44:17.481 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.482 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.483 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.484 DEBUG:boto:Params: {} 2022-09-03T06:44:17.485 DEBUG:boto:Token: None 2022-09-03T06:44:17.486 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/ 2022-09-03T06:44:17.487 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck= 2022-09-03T06:44:17.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck=', 'Content-Length': '0'} 2022-09-03T06:44:17.491 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c3046d2b2327e7f4-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.493 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:17.494 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:17.495 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:17.496 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:17.497 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:44:17.498 DEBUG:boto:Method: GET 2022-09-03T06:44:17.499 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:44:17.500 DEBUG:boto:Data: 2022-09-03T06:44:17.501 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.501 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.502 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.503 DEBUG:boto:Params: {} 2022-09-03T06:44:17.504 DEBUG:boto:Token: None 2022-09-03T06:44:17.505 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/ 2022-09-03T06:44:17.506 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck= 2022-09-03T06:44:17.507 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck=', 'Content-Length': '0'} 2022-09-03T06:44:17.510 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c6b93ff3bd497336-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.511 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:17.512 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:44:17.513 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:17.514 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:17.515 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.516 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.517 DEBUG:boto:Method: GET 2022-09-03T06:44:17.518 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.519 DEBUG:boto:Data: 2022-09-03T06:44:17.520 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.521 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.522 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.523 DEBUG:boto:Params: {} 2022-09-03T06:44:17.523 DEBUG:boto:Token: None 2022-09-03T06:44:17.524 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.525 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1wkSk5yFhNR5zByoIQiEmad6eJg= 2022-09-03T06:44:17.525 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1wkSk5yFhNR5zByoIQiEmad6eJg=', 'Content-Length': '0'} 2022-09-03T06:44:17.527 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006c1f410bbf5366f8-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.574 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:17.575 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:17.576 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.577 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.578 DEBUG:boto:Method: GET 2022-09-03T06:44:17.579 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.580 DEBUG:boto:Data: 2022-09-03T06:44:17.581 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.582 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.583 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.584 DEBUG:boto:Params: {} 2022-09-03T06:44:17.585 DEBUG:boto:Token: None 2022-09-03T06:44:17.587 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.588 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1wkSk5yFhNR5zByoIQiEmad6eJg= 2022-09-03T06:44:17.589 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1wkSk5yFhNR5zByoIQiEmad6eJg=', 'Content-Length': '0'} 2022-09-03T06:44:17.592 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003725d15ecb12de53-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.593 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:17.594 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:17.595 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.596 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.597 DEBUG:boto:Method: HEAD 2022-09-03T06:44:17.598 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.599 DEBUG:boto:Data: 2022-09-03T06:44:17.600 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.601 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.602 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.603 DEBUG:boto:Params: {} 2022-09-03T06:44:17.604 DEBUG:boto:Token: None 2022-09-03T06:44:17.605 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.606 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GzvDSbMeXo4mPKK9X5iGSEaa9ao= 2022-09-03T06:44:17.607 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GzvDSbMeXo4mPKK9X5iGSEaa9ao=', 'Content-Length': '0'} 2022-09-03T06:44:17.610 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd17da3c5d94da23-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.612 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:17.612 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:17.613 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.614 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.614 DEBUG:boto:Method: HEAD 2022-09-03T06:44:17.615 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.615 DEBUG:boto:Data: 2022-09-03T06:44:17.616 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.616 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.617 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.618 DEBUG:boto:Params: {} 2022-09-03T06:44:17.618 DEBUG:boto:Token: None 2022-09-03T06:44:17.619 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.620 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GzvDSbMeXo4mPKK9X5iGSEaa9ao= 2022-09-03T06:44:17.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GzvDSbMeXo4mPKK9X5iGSEaa9ao=', 'Content-Length': '0'} 2022-09-03T06:44:17.622 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009425b5981ed5a970-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.624 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:44:17.624 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:17.625 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:17.625 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.626 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.627 DEBUG:boto:Method: GET 2022-09-03T06:44:17.627 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.628 DEBUG:boto:Data: 2022-09-03T06:44:17.628 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.629 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.629 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.630 DEBUG:boto:Params: {} 2022-09-03T06:44:17.631 DEBUG:boto:Token: None 2022-09-03T06:44:17.631 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.632 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1wkSk5yFhNR5zByoIQiEmad6eJg= 2022-09-03T06:44:17.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1wkSk5yFhNR5zByoIQiEmad6eJg=', 'Content-Length': '0'} 2022-09-03T06:44:17.635 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c0bd1e927d5886a-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.635 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:17.636 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:17.637 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.637 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.638 DEBUG:boto:Method: GET 2022-09-03T06:44:17.638 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.639 DEBUG:boto:Data: 2022-09-03T06:44:17.639 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.640 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.640 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.641 DEBUG:boto:Params: {} 2022-09-03T06:44:17.641 DEBUG:boto:Token: None 2022-09-03T06:44:17.642 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:44:17.643 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1wkSk5yFhNR5zByoIQiEmad6eJg= 2022-09-03T06:44:17.643 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1wkSk5yFhNR5zByoIQiEmad6eJg=', 'Content-Length': '0'} 2022-09-03T06:44:17.645 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000283635685d00549a-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.646 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:17.647 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:17.647 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:17.648 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:17.648 DEBUG:boto:Method: GET 2022-09-03T06:44:17.649 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:44:17.649 DEBUG:boto:Data: 2022-09-03T06:44:17.650 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.650 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.651 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.651 DEBUG:boto:Params: {} 2022-09-03T06:44:17.652 DEBUG:boto:Token: None 2022-09-03T06:44:17.653 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/ 2022-09-03T06:44:17.653 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck= 2022-09-03T06:44:17.654 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck=', 'Content-Length': '0'} 2022-09-03T06:44:17.657 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f1b8660abb88ea42-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.702 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:17.703 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:17.704 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:17.705 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:17.705 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:17.706 DEBUG:boto:Method: GET 2022-09-03T06:44:17.706 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:44:17.707 DEBUG:boto:Data: 2022-09-03T06:44:17.707 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.708 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.708 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.709 DEBUG:boto:Params: {} 2022-09-03T06:44:17.709 DEBUG:boto:Token: None 2022-09-03T06:44:17.710 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/ 2022-09-03T06:44:17.711 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck= 2022-09-03T06:44:17.711 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck=', 'Content-Length': '0'} 2022-09-03T06:44:17.714 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003c5c961fd3968676-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.715 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:17.716 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:17.717 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:17.718 DEBUG:boto:Method: HEAD 2022-09-03T06:44:17.719 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:44:17.719 DEBUG:boto:Data: 2022-09-03T06:44:17.720 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.721 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.722 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.723 DEBUG:boto:Params: {} 2022-09-03T06:44:17.724 DEBUG:boto:Token: None 2022-09-03T06:44:17.724 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj92 2022-09-03T06:44:17.725 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hl5VaNN0tD/6hghErUzYz5dQ2CQ= 2022-09-03T06:44:17.726 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hl5VaNN0tD/6hghErUzYz5dQ2CQ=', 'Content-Length': '0'} 2022-09-03T06:44:17.729 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e8716c9b3b6b7d3-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.730 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:17.730 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:17.731 DEBUG:boto:Method: HEAD 2022-09-03T06:44:17.731 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:44:17.732 DEBUG:boto:Data: 2022-09-03T06:44:17.732 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.733 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.733 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.734 DEBUG:boto:Params: {} 2022-09-03T06:44:17.735 DEBUG:boto:Token: None 2022-09-03T06:44:17.735 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj92 2022-09-03T06:44:17.736 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hl5VaNN0tD/6hghErUzYz5dQ2CQ= 2022-09-03T06:44:17.737 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hl5VaNN0tD/6hghErUzYz5dQ2CQ=', 'Content-Length': '0'} 2022-09-03T06:44:17.739 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a32f1218c1e07927-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.740 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:44:17.740 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:17.741 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:17.741 DEBUG:boto:Method: GET 2022-09-03T06:44:17.742 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:44:17.742 DEBUG:boto:Data: 2022-09-03T06:44:17.743 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.743 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.744 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.744 DEBUG:boto:Params: {} 2022-09-03T06:44:17.745 DEBUG:boto:Token: None 2022-09-03T06:44:17.746 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj92 2022-09-03T06:44:17.746 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VCAWzAgPd9TlSRfG4auoyHwcJQs= 2022-09-03T06:44:17.747 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VCAWzAgPd9TlSRfG4auoyHwcJQs=', 'Content-Length': '0'} 2022-09-03T06:44:17.750 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000de0eb1f0b55136ed-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.751 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:44:17.751 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:44:17.752 DEBUG:boto:Method: GET 2022-09-03T06:44:17.752 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:44:17.753 DEBUG:boto:Data: 2022-09-03T06:44:17.753 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.754 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.755 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.755 DEBUG:boto:Params: {} 2022-09-03T06:44:17.756 DEBUG:boto:Token: None 2022-09-03T06:44:17.757 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj92 2022-09-03T06:44:17.757 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VCAWzAgPd9TlSRfG4auoyHwcJQs= 2022-09-03T06:44:17.758 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VCAWzAgPd9TlSRfG4auoyHwcJQs=', 'Content-Length': '0'} 2022-09-03T06:44:17.760 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a11e1ee4fc8c73e-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.761 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:17.761 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:17.762 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:17.763 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:17.763 DEBUG:boto:Method: GET 2022-09-03T06:44:17.764 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:44:17.764 DEBUG:boto:Data: 2022-09-03T06:44:17.765 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.766 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.767 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.767 DEBUG:boto:Params: {} 2022-09-03T06:44:17.768 DEBUG:boto:Token: None 2022-09-03T06:44:17.768 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/ 2022-09-03T06:44:17.769 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck= 2022-09-03T06:44:17.769 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck=', 'Content-Length': '0'} 2022-09-03T06:44:17.772 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bbb0ec694a2120b5-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.773 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:17.774 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:17.775 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:17.775 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:17.776 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:44:17.777 DEBUG:boto:Method: GET 2022-09-03T06:44:17.778 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:44:17.778 DEBUG:boto:Data: 2022-09-03T06:44:17.779 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.779 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.780 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.780 DEBUG:boto:Params: {} 2022-09-03T06:44:17.781 DEBUG:boto:Token: None 2022-09-03T06:44:17.782 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/ 2022-09-03T06:44:17.783 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck= 2022-09-03T06:44:17.783 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck=', 'Content-Length': '0'} 2022-09-03T06:44:17.786 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aabf2070a8dbae59-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.787 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:17.788 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:44:17.789 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:17.789 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:17.790 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.791 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.792 DEBUG:boto:Method: GET 2022-09-03T06:44:17.793 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.794 DEBUG:boto:Data: 2022-09-03T06:44:17.795 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.795 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.796 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.797 DEBUG:boto:Params: {} 2022-09-03T06:44:17.798 DEBUG:boto:Token: None 2022-09-03T06:44:17.799 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.799 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tdvIX4yOdNefJU8/Rb/luuMbpgg= 2022-09-03T06:44:17.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tdvIX4yOdNefJU8/Rb/luuMbpgg=', 'Content-Length': '0'} 2022-09-03T06:44:17.802 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e532cfd6410e5317-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.803 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:17.804 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:17.804 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.805 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.805 DEBUG:boto:Method: GET 2022-09-03T06:44:17.806 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.806 DEBUG:boto:Data: 2022-09-03T06:44:17.807 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.808 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.808 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.809 DEBUG:boto:Params: {} 2022-09-03T06:44:17.810 DEBUG:boto:Token: None 2022-09-03T06:44:17.810 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.811 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tdvIX4yOdNefJU8/Rb/luuMbpgg= 2022-09-03T06:44:17.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tdvIX4yOdNefJU8/Rb/luuMbpgg=', 'Content-Length': '0'} 2022-09-03T06:44:17.814 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006cff5c73ae218bee-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.815 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:17.816 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:17.816 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.817 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.817 DEBUG:boto:Method: HEAD 2022-09-03T06:44:17.818 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.818 DEBUG:boto:Data: 2022-09-03T06:44:17.819 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.819 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.820 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.820 DEBUG:boto:Params: {} 2022-09-03T06:44:17.821 DEBUG:boto:Token: None 2022-09-03T06:44:17.822 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.822 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3rUqiFjFTLxcaHCb7qqfgbr1Gow= 2022-09-03T06:44:17.823 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3rUqiFjFTLxcaHCb7qqfgbr1Gow=', 'Content-Length': '0'} 2022-09-03T06:44:17.825 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f38fba65f3f1df73-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.826 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:17.827 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:17.827 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.828 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.829 DEBUG:boto:Method: HEAD 2022-09-03T06:44:17.829 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.830 DEBUG:boto:Data: 2022-09-03T06:44:17.830 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.831 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.831 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.832 DEBUG:boto:Params: {} 2022-09-03T06:44:17.832 DEBUG:boto:Token: None 2022-09-03T06:44:17.833 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.833 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3rUqiFjFTLxcaHCb7qqfgbr1Gow= 2022-09-03T06:44:17.834 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3rUqiFjFTLxcaHCb7qqfgbr1Gow=', 'Content-Length': '0'} 2022-09-03T06:44:17.836 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ead29696a3f24e01-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.836 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:44:17.837 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:17.837 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:17.838 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.839 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.839 DEBUG:boto:Method: GET 2022-09-03T06:44:17.840 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.840 DEBUG:boto:Data: 2022-09-03T06:44:17.841 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.841 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.842 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.842 DEBUG:boto:Params: {} 2022-09-03T06:44:17.843 DEBUG:boto:Token: None 2022-09-03T06:44:17.844 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.844 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tdvIX4yOdNefJU8/Rb/luuMbpgg= 2022-09-03T06:44:17.845 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tdvIX4yOdNefJU8/Rb/luuMbpgg=', 'Content-Length': '0'} 2022-09-03T06:44:17.847 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a25afc3dfd6357a5-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.848 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:17.848 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:17.849 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.849 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.850 DEBUG:boto:Method: GET 2022-09-03T06:44:17.850 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.851 DEBUG:boto:Data: 2022-09-03T06:44:17.851 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.852 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.852 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.853 DEBUG:boto:Params: {} 2022-09-03T06:44:17.854 DEBUG:boto:Token: None 2022-09-03T06:44:17.854 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:44:17.855 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tdvIX4yOdNefJU8/Rb/luuMbpgg= 2022-09-03T06:44:17.855 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tdvIX4yOdNefJU8/Rb/luuMbpgg=', 'Content-Length': '0'} 2022-09-03T06:44:17.857 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d31d424cff690e1-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.858 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:17.859 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:17.859 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:17.860 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:17.860 DEBUG:boto:Method: GET 2022-09-03T06:44:17.861 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:44:17.861 DEBUG:boto:Data: 2022-09-03T06:44:17.862 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.862 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.863 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.863 DEBUG:boto:Params: {} 2022-09-03T06:44:17.864 DEBUG:boto:Token: None 2022-09-03T06:44:17.864 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/ 2022-09-03T06:44:17.865 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck= 2022-09-03T06:44:17.866 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck=', 'Content-Length': '0'} 2022-09-03T06:44:17.868 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000052694ffeb661f7d7-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.912 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:17.913 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:17.914 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:17.915 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:17.915 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:17.916 DEBUG:boto:Method: GET 2022-09-03T06:44:17.916 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:44:17.917 DEBUG:boto:Data: 2022-09-03T06:44:17.917 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.918 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.919 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.919 DEBUG:boto:Params: {} 2022-09-03T06:44:17.920 DEBUG:boto:Token: None 2022-09-03T06:44:17.922 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/ 2022-09-03T06:44:17.923 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck= 2022-09-03T06:44:17.923 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck=', 'Content-Length': '0'} 2022-09-03T06:44:17.925 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e7afc7df6c1ae4e4-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.926 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:17.928 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:17.928 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:17.929 DEBUG:boto:Method: HEAD 2022-09-03T06:44:17.930 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:44:17.930 DEBUG:boto:Data: 2022-09-03T06:44:17.931 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.931 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.932 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.932 DEBUG:boto:Params: {} 2022-09-03T06:44:17.933 DEBUG:boto:Token: None 2022-09-03T06:44:17.933 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj95 2022-09-03T06:44:17.934 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AKZXKU1IdTK0jOZmlQQnrUTraGY= 2022-09-03T06:44:17.934 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AKZXKU1IdTK0jOZmlQQnrUTraGY=', 'Content-Length': '0'} 2022-09-03T06:44:17.937 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000022f046f90dc6961c-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.937 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:17.938 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:17.938 DEBUG:boto:Method: HEAD 2022-09-03T06:44:17.939 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:44:17.939 DEBUG:boto:Data: 2022-09-03T06:44:17.940 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.940 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.941 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.941 DEBUG:boto:Params: {} 2022-09-03T06:44:17.942 DEBUG:boto:Token: None 2022-09-03T06:44:17.943 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj95 2022-09-03T06:44:17.943 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AKZXKU1IdTK0jOZmlQQnrUTraGY= 2022-09-03T06:44:17.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AKZXKU1IdTK0jOZmlQQnrUTraGY=', 'Content-Length': '0'} 2022-09-03T06:44:17.946 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003686e06dc7d5e506-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.946 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:44:17.947 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:17.948 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:17.948 DEBUG:boto:Method: GET 2022-09-03T06:44:17.949 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:44:17.949 DEBUG:boto:Data: 2022-09-03T06:44:17.950 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.950 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.951 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.951 DEBUG:boto:Params: {} 2022-09-03T06:44:17.952 DEBUG:boto:Token: None 2022-09-03T06:44:17.952 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj95 2022-09-03T06:44:17.953 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/aLgAcbxPvS0ajJrIuzJfmTEFLA= 2022-09-03T06:44:17.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/aLgAcbxPvS0ajJrIuzJfmTEFLA=', 'Content-Length': '0'} 2022-09-03T06:44:17.956 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008871b9a1a2dfa4ae-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.957 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:44:17.957 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:44:17.958 DEBUG:boto:Method: GET 2022-09-03T06:44:17.958 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:44:17.959 DEBUG:boto:Data: 2022-09-03T06:44:17.959 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.960 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.961 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.961 DEBUG:boto:Params: {} 2022-09-03T06:44:17.962 DEBUG:boto:Token: None 2022-09-03T06:44:17.962 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/obj95 2022-09-03T06:44:17.963 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/aLgAcbxPvS0ajJrIuzJfmTEFLA= 2022-09-03T06:44:17.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/aLgAcbxPvS0ajJrIuzJfmTEFLA=', 'Content-Length': '0'} 2022-09-03T06:44:17.966 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001f353ca1436a8cd8-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.967 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:17.967 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:17.968 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:17.969 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:17.969 DEBUG:boto:Method: GET 2022-09-03T06:44:17.970 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:44:17.971 DEBUG:boto:Data: 2022-09-03T06:44:17.971 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.972 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:44:17.973 DEBUG:boto:Port: 8001 2022-09-03T06:44:17.973 DEBUG:boto:Params: {} 2022-09-03T06:44:17.974 DEBUG:boto:Token: None 2022-09-03T06:44:17.975 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/ 2022-09-03T06:44:17.976 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck= 2022-09-03T06:44:17.977 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck=', 'Content-Length': '0'} 2022-09-03T06:44:17.980 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004ccfd58d8e5c78c9-006312f7c1-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.981 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:17.982 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:17.982 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:17.983 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:17.984 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:44:17.984 DEBUG:boto:Method: GET 2022-09-03T06:44:17.985 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:44:17.986 DEBUG:boto:Data: 2022-09-03T06:44:17.987 DEBUG:boto:Headers: {} 2022-09-03T06:44:17.987 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:44:17.988 DEBUG:boto:Port: 8000 2022-09-03T06:44:17.989 DEBUG:boto:Params: {} 2022-09-03T06:44:17.990 DEBUG:boto:Token: None 2022-09-03T06:44:17.990 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:44:17 GMT /dsgvcy-75/ 2022-09-03T06:44:17.991 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck= 2022-09-03T06:44:17.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wmhRKcyWTpJ1YAVzUzS6Kg49Tck=', 'Content-Length': '0'} 2022-09-03T06:44:17.995 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000051d18ec562a3b01c-006312f7c1-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:44:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:17.996 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:44:17.997 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-75 zones={test-zone3, test-zone2} 2022-09-03T06:44:17.997 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--num-shards', '29', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:44:17.998 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--num-shards', '29', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:44:17.999 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --num-shards 29 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T06:44:18.060 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:44:18.060 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:44:18.129 INFO:teuthology.orchestra.run.smithi181.stdout:tenant: 2022-09-03T06:44:18.130 INFO:teuthology.orchestra.run.smithi181.stdout:bucket name: dsgvcy-75 2022-09-03T06:44:18.142 INFO:teuthology.orchestra.run.smithi181.stdout:total entries: 20 2022-09-03T06:44:18.208 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:18.210 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:18.210 DEBUG:boto:Method: HEAD 2022-09-03T06:44:18.211 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:44:18.211 DEBUG:boto:Data: 2022-09-03T06:44:18.212 DEBUG:boto:Headers: {} 2022-09-03T06:44:18.213 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:18.214 DEBUG:boto:Port: 8000 2022-09-03T06:44:18.215 DEBUG:boto:Params: {} 2022-09-03T06:44:18.216 DEBUG:boto:Token: None 2022-09-03T06:44:18.216 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:18 GMT /dsgvcy-75/ 2022-09-03T06:44:18.217 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BacqyZSYxhs2SdY0KXcqibtPJO8= 2022-09-03T06:44:18.218 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BacqyZSYxhs2SdY0KXcqibtPJO8=', 'Content-Length': '0'} 2022-09-03T06:44:18.222 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '20'), ('X-RGW-Bytes-Used', '60'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003b3458ee66a48077-006312f7c2-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:18.223 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:44:18.223 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:44:18.224 DEBUG:boto:Method: PUT 2022-09-03T06:44:18.224 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:44:18.225 DEBUG:boto:Data: 2022-09-03T06:44:18.225 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:44:18.226 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:18.226 DEBUG:boto:Port: 8000 2022-09-03T06:44:18.227 DEBUG:boto:Params: {} 2022-09-03T06:44:18.228 DEBUG:boto:Token: None 2022-09-03T06:44:18.228 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:44:18 GMT /dsgvcy-75/obj29 2022-09-03T06:44:18.229 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tA2KvmMHnL5JVfvIsd67DEZ4nOU= 2022-09-03T06:44:18.229 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:44:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tA2KvmMHnL5JVfvIsd67DEZ4nOU='} 2022-09-03T06:44:18.236 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000007bbdf9ca7bbe9b34-006312f7c2-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:18.237 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:18.239 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:18.241 DEBUG:boto:Method: HEAD 2022-09-03T06:44:18.242 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:44:18.243 DEBUG:boto:Data: 2022-09-03T06:44:18.244 DEBUG:boto:Headers: {} 2022-09-03T06:44:18.245 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:18.246 DEBUG:boto:Port: 8000 2022-09-03T06:44:18.247 DEBUG:boto:Params: {} 2022-09-03T06:44:18.248 DEBUG:boto:Token: None 2022-09-03T06:44:18.249 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:18 GMT /dsgvcy-75/ 2022-09-03T06:44:18.249 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BacqyZSYxhs2SdY0KXcqibtPJO8= 2022-09-03T06:44:18.251 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BacqyZSYxhs2SdY0KXcqibtPJO8=', 'Content-Length': '0'} 2022-09-03T06:44:18.255 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '21'), ('X-RGW-Bytes-Used', '63'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c7faca7b241ed62b-006312f7c2-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:18.257 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:44:18.258 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:44:18.259 DEBUG:boto:Method: PUT 2022-09-03T06:44:18.260 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:44:18.261 DEBUG:boto:Data: 2022-09-03T06:44:18.265 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:44:18.265 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:18.266 DEBUG:boto:Port: 8000 2022-09-03T06:44:18.266 DEBUG:boto:Params: {} 2022-09-03T06:44:18.267 DEBUG:boto:Token: None 2022-09-03T06:44:18.268 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:44:18 GMT /dsgvcy-75/obj58 2022-09-03T06:44:18.269 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YWOX8j/mX8eu0oASxvVLoBmG+Qg= 2022-09-03T06:44:18.269 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:44:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YWOX8j/mX8eu0oASxvVLoBmG+Qg='} 2022-09-03T06:44:18.275 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000004ffe9eec6aee4e83-006312f7c2-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:18.276 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:18.276 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:18.277 DEBUG:boto:Method: HEAD 2022-09-03T06:44:18.278 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:44:18.278 DEBUG:boto:Data: 2022-09-03T06:44:18.279 DEBUG:boto:Headers: {} 2022-09-03T06:44:18.279 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:18.280 DEBUG:boto:Port: 8000 2022-09-03T06:44:18.280 DEBUG:boto:Params: {} 2022-09-03T06:44:18.281 DEBUG:boto:Token: None 2022-09-03T06:44:18.282 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:18 GMT /dsgvcy-75/ 2022-09-03T06:44:18.283 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BacqyZSYxhs2SdY0KXcqibtPJO8= 2022-09-03T06:44:18.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BacqyZSYxhs2SdY0KXcqibtPJO8=', 'Content-Length': '0'} 2022-09-03T06:44:18.287 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '22'), ('X-RGW-Bytes-Used', '66'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000431082b755aab948-006312f7c2-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:18.288 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:44:18.289 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:44:18.289 DEBUG:boto:Method: PUT 2022-09-03T06:44:18.290 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:44:18.291 DEBUG:boto:Data: 2022-09-03T06:44:18.291 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:44:18.292 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:18.293 DEBUG:boto:Port: 8000 2022-09-03T06:44:18.293 DEBUG:boto:Params: {} 2022-09-03T06:44:18.294 DEBUG:boto:Token: None 2022-09-03T06:44:18.295 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:44:18 GMT /dsgvcy-75/obj87 2022-09-03T06:44:18.296 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YTBVh3EubijEsSwQRJiJUD07BJA= 2022-09-03T06:44:18.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:44:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YTBVh3EubijEsSwQRJiJUD07BJA='} 2022-09-03T06:44:18.338 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000961fcdb9ab7a0b4c-006312f7c2-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:18.339 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:18.340 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:18.340 DEBUG:boto:Method: HEAD 2022-09-03T06:44:18.341 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:44:18.341 DEBUG:boto:Data: 2022-09-03T06:44:18.342 DEBUG:boto:Headers: {} 2022-09-03T06:44:18.342 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:18.343 DEBUG:boto:Port: 8000 2022-09-03T06:44:18.344 DEBUG:boto:Params: {} 2022-09-03T06:44:18.345 DEBUG:boto:Token: None 2022-09-03T06:44:18.345 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:18 GMT /dsgvcy-75/ 2022-09-03T06:44:18.346 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BacqyZSYxhs2SdY0KXcqibtPJO8= 2022-09-03T06:44:18.347 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BacqyZSYxhs2SdY0KXcqibtPJO8=', 'Content-Length': '0'} 2022-09-03T06:44:18.350 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '23'), ('X-RGW-Bytes-Used', '69'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000013cc64475065e92-006312f7c2-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:18.351 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:44:18.351 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:44:18.352 DEBUG:boto:Method: PUT 2022-09-03T06:44:18.353 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:44:18.353 DEBUG:boto:Data: 2022-09-03T06:44:18.355 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:44:18.356 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:18.357 DEBUG:boto:Port: 8000 2022-09-03T06:44:18.359 DEBUG:boto:Params: {} 2022-09-03T06:44:18.360 DEBUG:boto:Token: None 2022-09-03T06:44:18.361 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:44:18 GMT /dsgvcy-75/obj116 2022-09-03T06:44:18.361 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ga8YCsNMOunkGKG+7oTWDbcFqXU= 2022-09-03T06:44:18.362 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:44:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ga8YCsNMOunkGKG+7oTWDbcFqXU='} 2022-09-03T06:44:18.370 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000377a2f8f0fa8a889-006312f7c2-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:18.371 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:44:18.372 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:44:18.372 DEBUG:boto:Method: HEAD 2022-09-03T06:44:18.373 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:44:18.373 DEBUG:boto:Data: 2022-09-03T06:44:18.374 DEBUG:boto:Headers: {} 2022-09-03T06:44:18.374 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:18.375 DEBUG:boto:Port: 8000 2022-09-03T06:44:18.375 DEBUG:boto:Params: {} 2022-09-03T06:44:18.376 DEBUG:boto:Token: None 2022-09-03T06:44:18.377 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:44:18 GMT /dsgvcy-75/ 2022-09-03T06:44:18.377 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BacqyZSYxhs2SdY0KXcqibtPJO8= 2022-09-03T06:44:18.378 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:44:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BacqyZSYxhs2SdY0KXcqibtPJO8=', 'Content-Length': '0'} 2022-09-03T06:44:18.381 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '24'), ('X-RGW-Bytes-Used', '72'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000fc43f9963461fde6-006312f7c2-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:18.382 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:44:18.382 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:44:18.383 DEBUG:boto:Method: PUT 2022-09-03T06:44:18.383 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:44:18.384 DEBUG:boto:Data: 2022-09-03T06:44:18.385 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:44:18.385 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:44:18.386 DEBUG:boto:Port: 8000 2022-09-03T06:44:18.387 DEBUG:boto:Params: {} 2022-09-03T06:44:18.388 DEBUG:boto:Token: None 2022-09-03T06:44:18.388 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:44:18 GMT /dsgvcy-75/obj145 2022-09-03T06:44:18.389 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k8JodlDJAId9OzFnUysNf2RkfBg= 2022-09-03T06:44:18.389 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:44:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k8JodlDJAId9OzFnUysNf2RkfBg='} 2022-09-03T06:44:18.401 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000002ca275620907c568-006312f7c2-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:44:18.402 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-75 2022-09-03T06:44:18.402 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:44:18.403 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:44:18.404 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:44:18.466 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:44:18.466 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:44:18.513 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:44:18.512+0000 7fd31e309600 1 waiting to reach latest gen 4, on 3.. 2022-09-03T06:44:23.514 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:44:23.512+0000 7fd31e309600 1 waiting to reach latest gen 4, on 3.. 2022-09-03T06:44:28.518 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:44:28.513+0000 7fd31e309600 1 waiting to reach latest gen 4, on 3.. 2022-09-03T06:44:33.516 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:44:33.514+0000 7fd31e309600 1 waiting to reach latest gen 4, on 3.. 2022-09-03T06:44:38.520 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:44:38.516+0000 7fd31e309600 1 waiting for incremental sync to catch up: 2022-09-03T06:44:38.520 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2022-09-03T06:44:38.520 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000002.76.5, 00000000001.78.5, , , , , 00000000001.80.5, , , , , , , , , , , 00000000002.105.5, 00000000001.84.5, , , , , , , , , , ] 2022-09-03T06:44:43.521 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:44:43.519+0000 7fd31e309600 1 waiting for incremental sync to catch up: 2022-09-03T06:44:43.522 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2022-09-03T06:44:43.522 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000002.76.5, 00000000001.78.5, , , , , 00000000001.80.5, , , , , , , , , , , 00000000002.105.5, 00000000001.84.5, , , , , , , , , , ] 2022-09-03T06:44:48.523 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:44:48.520+0000 7fd31e309600 1 waiting for incremental sync to catch up: 2022-09-03T06:44:48.523 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2022-09-03T06:44:48.524 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000002.76.5, 00000000001.78.5, , , , , 00000000001.80.5, , , , , , , , , , , 00000000002.105.5, 00000000001.84.5, , , , , , , , , , ] 2022-09-03T06:44:53.526 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:44:53.523+0000 7fd31e309600 1 waiting for incremental sync to catch up: 2022-09-03T06:44:53.526 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2022-09-03T06:44:53.527 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000002.76.5, 00000000001.78.5, , , , , 00000000001.80.5, , , , , , , , , , , 00000000002.105.5, 00000000001.84.5, , , , , , , , , , ] 2022-09-03T06:44:58.528 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:44:58.526+0000 7fd31e309600 1 bucket sync caught up with source: 2022-09-03T06:44:58.528 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000002.76.5, 00000000001.78.5, , , , , 00000000001.80.5, , , , , , , , , , , 00000000002.105.5, 00000000001.84.5, , , , , , , , , , ] 2022-09-03T06:44:58.529 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000002.76.5, 00000000001.78.5, , , , , 00000000001.80.5, , , , , , , , , , , 00000000002.105.5, 00000000001.84.5, , , , , , , , , , ] 2022-09-03T06:44:58.529 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:44:58.526+0000 7fd31e309600 0 bucket checkpoint complete 2022-09-03T06:44:58.547 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-75 2022-09-03T06:44:58.548 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:44:58.549 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:44:58.550 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:44:58.614 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:44:58.615 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:44:58.660 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:44:58.659+0000 7f02cedc9600 1 bucket sync caught up with source: 2022-09-03T06:44:58.661 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [00000000002.76.5, 00000000001.78.5, , , , , 00000000001.80.5, , , , , , , , , , , 00000000002.105.5, 00000000001.84.5, , , , , , , , , , ] 2022-09-03T06:44:58.662 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [00000000002.76.5, 00000000001.78.5, , , , , 00000000001.80.5, , , , , , , , , , , 00000000002.105.5, 00000000001.84.5, , , , , , , , , , ] 2022-09-03T06:44:58.662 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:44:58.659+0000 7f02cedc9600 0 bucket checkpoint complete 2022-09-03T06:44:58.678 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-75 2022-09-03T06:44:58.680 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:44:58.681 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:44:58.682 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:44:58.747 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:44:58.748 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:44:58.797 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:44:58.795+0000 7f85adf23600 1 waiting for incremental sync to catch up: 2022-09-03T06:44:58.798 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000002.76.5, , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2022-09-03T06:44:58.798 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000002.76.5, 00000000001.78.5, , , , , 00000000001.80.5, , , , , , , , , , , 00000000002.105.5, 00000000001.84.5, , , , , , , , , , ] 2022-09-03T06:45:03.800 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:45:03.798+0000 7f85adf23600 1 waiting for incremental sync to catch up: 2022-09-03T06:45:03.800 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000002.76.5, 00000000001.78.5, , , , , , , , , , , , , , , , 00000000002.105.5, , , , , , , , , , , ] 2022-09-03T06:45:03.801 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000002.76.5, 00000000001.78.5, , , , , 00000000001.80.5, , , , , , , , , , , 00000000002.105.5, 00000000001.84.5, , , , , , , , , , ] 2022-09-03T06:45:08.802 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:45:08.800+0000 7f85adf23600 1 waiting for incremental sync to catch up: 2022-09-03T06:45:08.802 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000002.76.5, 00000000001.78.5, , , , , , , , , , , , , , , , 00000000002.105.5, , , , , , , , , , , ] 2022-09-03T06:45:08.803 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000002.76.5, 00000000001.78.5, , , , , 00000000001.80.5, , , , , , , , , , , 00000000002.105.5, 00000000001.84.5, , , , , , , , , , ] 2022-09-03T06:45:13.804 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:45:13.802+0000 7f85adf23600 1 waiting for incremental sync to catch up: 2022-09-03T06:45:13.805 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000002.76.5, 00000000001.78.5, , , , , , , , , , , , , , , , 00000000002.105.5, , , , , , , , , , , ] 2022-09-03T06:45:13.806 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000002.76.5, 00000000001.78.5, , , , , 00000000001.80.5, , , , , , , , , , , 00000000002.105.5, 00000000001.84.5, , , , , , , , , , ] 2022-09-03T06:45:18.806 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:45:18.804+0000 7f85adf23600 1 waiting for incremental sync to catch up: 2022-09-03T06:45:18.807 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000002.76.5, 00000000001.78.5, , , , , , , , , , , , , , , , 00000000002.105.5, , , , , , , , , , , ] 2022-09-03T06:45:18.808 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000002.76.5, 00000000001.78.5, , , , , 00000000001.80.5, , , , , , , , , , , 00000000002.105.5, 00000000001.84.5, , , , , , , , , , ] 2022-09-03T06:45:23.809 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:45:23.807+0000 7f85adf23600 1 bucket sync caught up with source: 2022-09-03T06:45:23.810 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [00000000002.76.5, 00000000001.78.5, , , , , 00000000001.80.5, , , , , , , , , , , 00000000002.105.5, 00000000001.84.5, , , , , , , , , , ] 2022-09-03T06:45:23.810 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [00000000002.76.5, 00000000001.78.5, , , , , 00000000001.80.5, , , , , , , , , , , 00000000002.105.5, 00000000001.84.5, , , , , , , , , , ] 2022-09-03T06:45:23.810 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:45:23.807+0000 7f85adf23600 0 bucket checkpoint complete 2022-09-03T06:45:23.831 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-75 2022-09-03T06:45:23.832 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:45:23.834 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:45:23.835 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:45:23.906 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:45:23.906 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:45:23.960 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:45:23.957+0000 7f76d0c6e600 1 bucket sync caught up with source: 2022-09-03T06:45:23.960 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.582.5, 00000000002.739.5, 00000000001.366.5, 00000000003.711.5, 00000000004.380.5, 00000000001.364.5, 00000000001.865.5, 00000000002.855.5, 00000000005.312.5, 00000000004.587.5] 2022-09-03T06:45:23.961 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.582.5, 00000000002.739.5, 00000000001.366.5, 00000000003.711.5, 00000000004.380.5, 00000000001.364.5, 00000000001.865.5, 00000000002.855.5, 00000000005.312.5, 00000000004.587.5] 2022-09-03T06:45:23.961 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:45:23.957+0000 7f76d0c6e600 0 bucket checkpoint complete 2022-09-03T06:45:23.978 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-75 2022-09-03T06:45:23.978 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:45:23.979 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:45:23.980 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:45:24.037 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:45:24.037 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:45:24.080 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:45:24.078+0000 7f93c0861600 1 bucket sync caught up with source: 2022-09-03T06:45:24.081 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.582.5, 00000000002.739.5, 00000000001.366.5, 00000000003.711.5, 00000000004.380.5, 00000000001.364.5, 00000000001.865.5, 00000000002.855.5, 00000000005.312.5, 00000000004.587.5] 2022-09-03T06:45:24.081 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.582.5, 00000000002.739.5, 00000000001.366.5, 00000000003.711.5, 00000000004.380.5, 00000000001.364.5, 00000000001.865.5, 00000000002.855.5, 00000000005.312.5, 00000000004.587.5] 2022-09-03T06:45:24.081 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:45:24.078+0000 7f93c0861600 0 bucket checkpoint complete 2022-09-03T06:45:24.096 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-75 2022-09-03T06:45:24.097 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-75 2022-09-03T06:45:24.097 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:45:24.098 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:45:24.099 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:45:24.156 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:45:24.157 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:45:24.192 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:45:24.189+0000 7f439d0c4600 1 bucket sync caught up with source: 2022-09-03T06:45:24.192 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000002.202.5, 00000000003.158.5, 00000000001.271.5, 00000000003.272.5, 00000000004.285.5, 00000000001.249.5, 00000000001.327.5, 00000000002.317.5, 00000000005.247.5, 00000000004.290.5] 2022-09-03T06:45:24.193 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000002.202.5, 00000000003.158.5, 00000000001.271.5, 00000000003.272.5, 00000000004.285.5, 00000000001.249.5, 00000000001.327.5, 00000000002.317.5, 00000000005.247.5, 00000000004.290.5] 2022-09-03T06:45:24.193 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:45:24.189+0000 7f439d0c4600 0 bucket checkpoint complete 2022-09-03T06:45:24.206 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-75 2022-09-03T06:45:24.207 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:45:24.208 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:45:24.209 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:45:24.274 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:45:24.275 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:45:24.313 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:45:24.311+0000 7f37d9b01600 1 bucket sync caught up with source: 2022-09-03T06:45:24.314 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000002.202.5, 00000000003.158.5, 00000000001.271.5, 00000000003.272.5, 00000000004.285.5, 00000000001.249.5, 00000000001.327.5, 00000000002.317.5, 00000000005.247.5, 00000000004.290.5] 2022-09-03T06:45:24.314 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000002.202.5, 00000000003.158.5, 00000000001.271.5, 00000000003.272.5, 00000000004.285.5, 00000000001.249.5, 00000000001.327.5, 00000000002.317.5, 00000000005.247.5, 00000000004.290.5] 2022-09-03T06:45:24.314 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:45:24.311+0000 7f37d9b01600 0 bucket checkpoint complete 2022-09-03T06:45:24.328 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-75 2022-09-03T06:45:24.328 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-75 zones={test-zone2, test-zone1} 2022-09-03T06:45:24.329 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:24.330 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:24.331 DEBUG:boto:Method: HEAD 2022-09-03T06:45:24.331 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:45:24.332 DEBUG:boto:Data: 2022-09-03T06:45:24.332 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.333 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:24.333 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.334 DEBUG:boto:Params: {} 2022-09-03T06:45:24.335 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:45:24.336 DEBUG:boto:Token: None 2022-09-03T06:45:24.337 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/ 2022-09-03T06:45:24.337 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:izEsfkT2hGXlA4FGvKfnISsr3JY= 2022-09-03T06:45:24.338 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:izEsfkT2hGXlA4FGvKfnISsr3JY=', 'Content-Length': '0'} 2022-09-03T06:45:24.343 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '25'), ('X-RGW-Bytes-Used', '75'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000bd8009e74c1722fc-006312f804-188c-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.344 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:24.344 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:24.345 DEBUG:boto:Method: HEAD 2022-09-03T06:45:24.345 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:45:24.346 DEBUG:boto:Data: 2022-09-03T06:45:24.347 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.347 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:24.348 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.348 DEBUG:boto:Params: {} 2022-09-03T06:45:24.349 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:45:24.350 DEBUG:boto:Token: None 2022-09-03T06:45:24.351 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/ 2022-09-03T06:45:24.351 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:izEsfkT2hGXlA4FGvKfnISsr3JY= 2022-09-03T06:45:24.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:izEsfkT2hGXlA4FGvKfnISsr3JY=', 'Content-Length': '0'} 2022-09-03T06:45:24.356 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '25'), ('X-RGW-Bytes-Used', '75'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000d05d2445e47fb473-006312f804-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.357 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:45:24.358 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:24.358 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:24.359 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:24.360 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:24.361 DEBUG:boto:Method: GET 2022-09-03T06:45:24.362 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:24.362 DEBUG:boto:Data: 2022-09-03T06:45:24.363 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.363 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:24.364 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.364 DEBUG:boto:Params: {} 2022-09-03T06:45:24.365 DEBUG:boto:Token: None 2022-09-03T06:45:24.366 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/?versions 2022-09-03T06:45:24.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B1gydb/H5BveMeCdG9ao/TOsW4E= 2022-09-03T06:45:24.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B1gydb/H5BveMeCdG9ao/TOsW4E=', 'Content-Length': '0'} 2022-09-03T06:45:24.370 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bf8b722fb9c0638a-006312f804-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.371 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:24.377 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:45:24.378 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:45:24.379 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2022-09-03T06:45:24.380 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2022-09-03T06:45:24.381 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:45:24.381 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:45:24.383 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:45:24.384 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:45:24.384 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2022-09-03T06:45:24.385 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:45:24.385 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:45:24.386 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:45:24.387 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:45:24.387 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:45:24.388 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:45:24.388 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:45:24.389 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:45:24.390 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2022-09-03T06:45:24.390 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:45:24.391 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:45:24.391 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:45:24.391 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:45:24.392 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2022-09-03T06:45:24.392 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:45:24.393 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:45:24.393 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:45:24.394 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:24.395 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:24.395 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:24.396 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:24.396 DEBUG:boto:Method: GET 2022-09-03T06:45:24.397 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:24.398 DEBUG:boto:Data: 2022-09-03T06:45:24.398 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.399 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:24.399 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.400 DEBUG:boto:Params: {} 2022-09-03T06:45:24.401 DEBUG:boto:Token: None 2022-09-03T06:45:24.401 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/?versions 2022-09-03T06:45:24.402 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B1gydb/H5BveMeCdG9ao/TOsW4E= 2022-09-03T06:45:24.403 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B1gydb/H5BveMeCdG9ao/TOsW4E=', 'Content-Length': '0'} 2022-09-03T06:45:24.406 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000056419e860317c3ce-006312f804-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.408 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:24.413 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:45:24.414 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:45:24.415 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2022-09-03T06:45:24.416 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2022-09-03T06:45:24.417 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:45:24.418 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:45:24.419 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:45:24.420 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:45:24.420 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2022-09-03T06:45:24.421 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:45:24.422 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:45:24.423 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:45:24.424 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:45:24.425 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:45:24.426 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:45:24.427 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:45:24.428 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:45:24.429 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2022-09-03T06:45:24.430 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:45:24.431 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:45:24.432 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:45:24.433 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:45:24.434 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2022-09-03T06:45:24.435 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:45:24.436 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:45:24.437 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:24.438 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:24.439 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:24.440 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:24.441 DEBUG:boto:Method: GET 2022-09-03T06:45:24.442 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:24.443 DEBUG:boto:Data: 2022-09-03T06:45:24.444 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.445 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:24.446 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.447 DEBUG:boto:Params: {} 2022-09-03T06:45:24.448 DEBUG:boto:Token: None 2022-09-03T06:45:24.449 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/?versions 2022-09-03T06:45:24.450 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B1gydb/H5BveMeCdG9ao/TOsW4E= 2022-09-03T06:45:24.451 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B1gydb/H5BveMeCdG9ao/TOsW4E=', 'Content-Length': '0'} 2022-09-03T06:45:24.454 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005d4c56fd88964412-006312f804-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.456 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:24.459 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:24.461 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:24.461 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:24.462 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:24.464 DEBUG:boto:Method: GET 2022-09-03T06:45:24.464 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:24.465 DEBUG:boto:Data: 2022-09-03T06:45:24.466 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.467 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:24.468 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.469 DEBUG:boto:Params: {} 2022-09-03T06:45:24.470 DEBUG:boto:Token: None 2022-09-03T06:45:24.471 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/?versions 2022-09-03T06:45:24.473 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B1gydb/H5BveMeCdG9ao/TOsW4E= 2022-09-03T06:45:24.474 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B1gydb/H5BveMeCdG9ao/TOsW4E=', 'Content-Length': '0'} 2022-09-03T06:45:24.478 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000042dbec015d580676-006312f804-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.479 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:24.482 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:45:24.482 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:24.483 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:24.484 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.484 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.485 DEBUG:boto:Method: GET 2022-09-03T06:45:24.485 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.486 DEBUG:boto:Data: 2022-09-03T06:45:24.486 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.487 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:24.487 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.488 DEBUG:boto:Params: {} 2022-09-03T06:45:24.489 DEBUG:boto:Token: None 2022-09-03T06:45:24.489 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.490 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QkKkIBaWfchCqu4QzEZwifJql3c= 2022-09-03T06:45:24.491 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QkKkIBaWfchCqu4QzEZwifJql3c=', 'Content-Length': '0'} 2022-09-03T06:45:24.493 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a5a42336df79c858-006312f804-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.494 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:24.494 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:24.495 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.495 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.496 DEBUG:boto:Method: GET 2022-09-03T06:45:24.496 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.497 DEBUG:boto:Data: 2022-09-03T06:45:24.497 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.498 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:24.498 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.499 DEBUG:boto:Params: {} 2022-09-03T06:45:24.500 DEBUG:boto:Token: None 2022-09-03T06:45:24.500 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.501 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QkKkIBaWfchCqu4QzEZwifJql3c= 2022-09-03T06:45:24.502 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QkKkIBaWfchCqu4QzEZwifJql3c=', 'Content-Length': '0'} 2022-09-03T06:45:24.503 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000653599b22f263382-006312f804-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.504 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:24.505 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:24.506 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.507 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.507 DEBUG:boto:Method: HEAD 2022-09-03T06:45:24.508 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.508 DEBUG:boto:Data: 2022-09-03T06:45:24.509 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.509 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:24.510 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.510 DEBUG:boto:Params: {} 2022-09-03T06:45:24.511 DEBUG:boto:Token: None 2022-09-03T06:45:24.512 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.512 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7zeU3AO24LNRI6hgZPBGSZ3oM04= 2022-09-03T06:45:24.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7zeU3AO24LNRI6hgZPBGSZ3oM04=', 'Content-Length': '0'} 2022-09-03T06:45:24.516 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004af4bf5de8adbea7-006312f804-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.517 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:24.517 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:24.518 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.519 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.519 DEBUG:boto:Method: HEAD 2022-09-03T06:45:24.520 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.521 DEBUG:boto:Data: 2022-09-03T06:45:24.522 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.523 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:24.524 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.525 DEBUG:boto:Params: {} 2022-09-03T06:45:24.526 DEBUG:boto:Token: None 2022-09-03T06:45:24.527 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.528 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7zeU3AO24LNRI6hgZPBGSZ3oM04= 2022-09-03T06:45:24.530 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7zeU3AO24LNRI6hgZPBGSZ3oM04=', 'Content-Length': '0'} 2022-09-03T06:45:24.532 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000035681df0b76bb4be-006312f804-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.534 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:45:24.535 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:24.536 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:24.536 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.538 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.539 DEBUG:boto:Method: GET 2022-09-03T06:45:24.539 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.540 DEBUG:boto:Data: 2022-09-03T06:45:24.541 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.541 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:24.542 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.542 DEBUG:boto:Params: {} 2022-09-03T06:45:24.543 DEBUG:boto:Token: None 2022-09-03T06:45:24.544 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.545 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QkKkIBaWfchCqu4QzEZwifJql3c= 2022-09-03T06:45:24.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QkKkIBaWfchCqu4QzEZwifJql3c=', 'Content-Length': '0'} 2022-09-03T06:45:24.550 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000176fa6a1688b28df-006312f804-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.551 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:24.552 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:24.552 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.553 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.553 DEBUG:boto:Method: GET 2022-09-03T06:45:24.554 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.555 DEBUG:boto:Data: 2022-09-03T06:45:24.555 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.556 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:24.557 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.557 DEBUG:boto:Params: {} 2022-09-03T06:45:24.558 DEBUG:boto:Token: None 2022-09-03T06:45:24.559 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:24.560 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QkKkIBaWfchCqu4QzEZwifJql3c= 2022-09-03T06:45:24.560 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QkKkIBaWfchCqu4QzEZwifJql3c=', 'Content-Length': '0'} 2022-09-03T06:45:24.563 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a6dd252965f3b2d8-006312f804-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.564 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:24.564 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:24.565 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:24.566 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:24.567 DEBUG:boto:Method: GET 2022-09-03T06:45:24.568 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:45:24.569 DEBUG:boto:Data: 2022-09-03T06:45:24.569 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.570 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:24.571 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.572 DEBUG:boto:Params: {} 2022-09-03T06:45:24.572 DEBUG:boto:Token: None 2022-09-03T06:45:24.573 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/ 2022-09-03T06:45:24.574 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5SMFy0jsOpAxGwGAhdVR+44rihs= 2022-09-03T06:45:24.574 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5SMFy0jsOpAxGwGAhdVR+44rihs=', 'Content-Length': '0'} 2022-09-03T06:45:24.577 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000750e18bf0cbea485-006312f804-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.578 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:24.579 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:24.580 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:24.581 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:24.581 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:24.582 DEBUG:boto:Method: GET 2022-09-03T06:45:24.583 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:45:24.583 DEBUG:boto:Data: 2022-09-03T06:45:24.584 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.584 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:24.585 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.586 DEBUG:boto:Params: {} 2022-09-03T06:45:24.587 DEBUG:boto:Token: None 2022-09-03T06:45:24.587 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/ 2022-09-03T06:45:24.588 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5SMFy0jsOpAxGwGAhdVR+44rihs= 2022-09-03T06:45:24.589 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5SMFy0jsOpAxGwGAhdVR+44rihs=', 'Content-Length': '0'} 2022-09-03T06:45:24.592 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005e5a6a768a9f12c7-006312f804-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.593 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:24.594 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:24.595 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:24.595 DEBUG:boto:Method: HEAD 2022-09-03T06:45:24.596 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:45:24.597 DEBUG:boto:Data: 2022-09-03T06:45:24.597 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.598 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:24.598 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.599 DEBUG:boto:Params: {} 2022-09-03T06:45:24.599 DEBUG:boto:Token: None 2022-09-03T06:45:24.600 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/obj11 2022-09-03T06:45:24.601 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yF6gCIkwppb70rEHddrNyTww/CE= 2022-09-03T06:45:24.601 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yF6gCIkwppb70rEHddrNyTww/CE=', 'Content-Length': '0'} 2022-09-03T06:45:24.603 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000066783480219e5e59-006312f804-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.604 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:24.605 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:24.605 DEBUG:boto:Method: HEAD 2022-09-03T06:45:24.606 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:45:24.606 DEBUG:boto:Data: 2022-09-03T06:45:24.607 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.607 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:24.608 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.608 DEBUG:boto:Params: {} 2022-09-03T06:45:24.609 DEBUG:boto:Token: None 2022-09-03T06:45:24.609 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/obj11 2022-09-03T06:45:24.610 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yF6gCIkwppb70rEHddrNyTww/CE= 2022-09-03T06:45:24.611 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yF6gCIkwppb70rEHddrNyTww/CE=', 'Content-Length': '0'} 2022-09-03T06:45:24.613 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bedf90de13c48ccb-006312f804-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.613 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:45:24.614 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:24.615 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:24.615 DEBUG:boto:Method: GET 2022-09-03T06:45:24.616 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:45:24.617 DEBUG:boto:Data: 2022-09-03T06:45:24.617 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.618 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:24.619 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.619 DEBUG:boto:Params: {} 2022-09-03T06:45:24.620 DEBUG:boto:Token: None 2022-09-03T06:45:24.620 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/obj11 2022-09-03T06:45:24.621 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uf7y/WMbUlbi7r5Xvm3geak+qxs= 2022-09-03T06:45:24.622 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uf7y/WMbUlbi7r5Xvm3geak+qxs=', 'Content-Length': '0'} 2022-09-03T06:45:24.624 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009cf483d37e25c22c-006312f804-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.625 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:24.626 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:24.627 DEBUG:boto:Method: GET 2022-09-03T06:45:24.627 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:45:24.628 DEBUG:boto:Data: 2022-09-03T06:45:24.628 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.629 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:24.630 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.630 DEBUG:boto:Params: {} 2022-09-03T06:45:24.631 DEBUG:boto:Token: None 2022-09-03T06:45:24.632 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/obj11 2022-09-03T06:45:24.632 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uf7y/WMbUlbi7r5Xvm3geak+qxs= 2022-09-03T06:45:24.633 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uf7y/WMbUlbi7r5Xvm3geak+qxs=', 'Content-Length': '0'} 2022-09-03T06:45:24.635 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ddfe31633ec3a575-006312f804-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.679 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:24.680 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:24.681 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:24.682 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:24.684 DEBUG:boto:Method: GET 2022-09-03T06:45:24.685 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:45:24.686 DEBUG:boto:Data: 2022-09-03T06:45:24.687 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.688 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:24.690 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.691 DEBUG:boto:Params: {} 2022-09-03T06:45:24.692 DEBUG:boto:Token: None 2022-09-03T06:45:24.694 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/ 2022-09-03T06:45:24.695 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5SMFy0jsOpAxGwGAhdVR+44rihs= 2022-09-03T06:45:24.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5SMFy0jsOpAxGwGAhdVR+44rihs=', 'Content-Length': '0'} 2022-09-03T06:45:24.700 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000987b53e72f981f5e-006312f804-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.701 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:24.703 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:24.704 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:24.705 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:24.706 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:24.707 DEBUG:boto:Method: GET 2022-09-03T06:45:24.708 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:45:24.710 DEBUG:boto:Data: 2022-09-03T06:45:24.710 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.711 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:24.712 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.712 DEBUG:boto:Params: {} 2022-09-03T06:45:24.713 DEBUG:boto:Token: None 2022-09-03T06:45:24.714 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/ 2022-09-03T06:45:24.715 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5SMFy0jsOpAxGwGAhdVR+44rihs= 2022-09-03T06:45:24.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5SMFy0jsOpAxGwGAhdVR+44rihs=', 'Content-Length': '0'} 2022-09-03T06:45:24.720 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000de99ef9b9e1f8bea-006312f804-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.721 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:24.723 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:45:24.724 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:24.725 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:24.726 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.727 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.728 DEBUG:boto:Method: GET 2022-09-03T06:45:24.729 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.730 DEBUG:boto:Data: 2022-09-03T06:45:24.731 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.732 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:24.733 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.734 DEBUG:boto:Params: {} 2022-09-03T06:45:24.735 DEBUG:boto:Token: None 2022-09-03T06:45:24.736 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.737 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lO0K/78ZpwXOgb7odwUQwEJR+ok= 2022-09-03T06:45:24.738 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lO0K/78ZpwXOgb7odwUQwEJR+ok=', 'Content-Length': '0'} 2022-09-03T06:45:24.741 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000079ebe056f20078e-006312f804-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.743 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:24.743 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:24.744 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.745 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.746 DEBUG:boto:Method: GET 2022-09-03T06:45:24.747 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.748 DEBUG:boto:Data: 2022-09-03T06:45:24.748 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.749 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:24.750 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.750 DEBUG:boto:Params: {} 2022-09-03T06:45:24.751 DEBUG:boto:Token: None 2022-09-03T06:45:24.752 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.753 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lO0K/78ZpwXOgb7odwUQwEJR+ok= 2022-09-03T06:45:24.753 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lO0K/78ZpwXOgb7odwUQwEJR+ok=', 'Content-Length': '0'} 2022-09-03T06:45:24.756 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f005e8bae7d0bf64-006312f804-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.757 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:24.757 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:24.758 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.758 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.759 DEBUG:boto:Method: HEAD 2022-09-03T06:45:24.760 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.760 DEBUG:boto:Data: 2022-09-03T06:45:24.761 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.761 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:24.762 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.762 DEBUG:boto:Params: {} 2022-09-03T06:45:24.763 DEBUG:boto:Token: None 2022-09-03T06:45:24.764 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.764 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NaXipBAcTPO6a7rDUpM+WOIA3q0= 2022-09-03T06:45:24.765 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NaXipBAcTPO6a7rDUpM+WOIA3q0=', 'Content-Length': '0'} 2022-09-03T06:45:24.767 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008f9906afd0aa5db6-006312f804-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.768 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:24.768 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:24.769 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.769 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.770 DEBUG:boto:Method: HEAD 2022-09-03T06:45:24.770 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.771 DEBUG:boto:Data: 2022-09-03T06:45:24.771 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.772 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:24.772 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.773 DEBUG:boto:Params: {} 2022-09-03T06:45:24.774 DEBUG:boto:Token: None 2022-09-03T06:45:24.774 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.775 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NaXipBAcTPO6a7rDUpM+WOIA3q0= 2022-09-03T06:45:24.775 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NaXipBAcTPO6a7rDUpM+WOIA3q0=', 'Content-Length': '0'} 2022-09-03T06:45:24.777 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000539a84ec3cb0ce37-006312f804-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.778 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:45:24.779 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:24.779 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:24.780 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.780 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.781 DEBUG:boto:Method: GET 2022-09-03T06:45:24.781 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.782 DEBUG:boto:Data: 2022-09-03T06:45:24.782 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.783 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:24.783 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.784 DEBUG:boto:Params: {} 2022-09-03T06:45:24.785 DEBUG:boto:Token: None 2022-09-03T06:45:24.785 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.786 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lO0K/78ZpwXOgb7odwUQwEJR+ok= 2022-09-03T06:45:24.786 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lO0K/78ZpwXOgb7odwUQwEJR+ok=', 'Content-Length': '0'} 2022-09-03T06:45:24.789 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005bc45ee603548535-006312f804-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.790 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:24.790 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:24.791 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.791 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.792 DEBUG:boto:Method: GET 2022-09-03T06:45:24.792 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.793 DEBUG:boto:Data: 2022-09-03T06:45:24.794 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.794 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:24.795 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.795 DEBUG:boto:Params: {} 2022-09-03T06:45:24.796 DEBUG:boto:Token: None 2022-09-03T06:45:24.797 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:24.797 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lO0K/78ZpwXOgb7odwUQwEJR+ok= 2022-09-03T06:45:24.798 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lO0K/78ZpwXOgb7odwUQwEJR+ok=', 'Content-Length': '0'} 2022-09-03T06:45:24.800 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000941bfc1ca21bd933-006312f804-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.801 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:24.802 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:24.802 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:24.803 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:24.803 DEBUG:boto:Method: GET 2022-09-03T06:45:24.804 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:45:24.804 DEBUG:boto:Data: 2022-09-03T06:45:24.805 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.806 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:24.806 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.807 DEBUG:boto:Params: {} 2022-09-03T06:45:24.808 DEBUG:boto:Token: None 2022-09-03T06:45:24.808 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/ 2022-09-03T06:45:24.810 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5SMFy0jsOpAxGwGAhdVR+44rihs= 2022-09-03T06:45:24.811 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5SMFy0jsOpAxGwGAhdVR+44rihs=', 'Content-Length': '0'} 2022-09-03T06:45:24.813 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000da61e57ab8bdb70c-006312f804-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.815 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:24.816 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:24.817 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:24.818 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:24.819 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:24.820 DEBUG:boto:Method: GET 2022-09-03T06:45:24.821 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:45:24.822 DEBUG:boto:Data: 2022-09-03T06:45:24.823 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.824 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:24.825 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.826 DEBUG:boto:Params: {} 2022-09-03T06:45:24.827 DEBUG:boto:Token: None 2022-09-03T06:45:24.827 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/ 2022-09-03T06:45:24.828 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5SMFy0jsOpAxGwGAhdVR+44rihs= 2022-09-03T06:45:24.829 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5SMFy0jsOpAxGwGAhdVR+44rihs=', 'Content-Length': '0'} 2022-09-03T06:45:24.833 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fd76505dcf572b4b-006312f804-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.834 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:24.835 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:24.836 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:24.837 DEBUG:boto:Method: HEAD 2022-09-03T06:45:24.838 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:45:24.838 DEBUG:boto:Data: 2022-09-03T06:45:24.839 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.840 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:24.840 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.841 DEBUG:boto:Params: {} 2022-09-03T06:45:24.842 DEBUG:boto:Token: None 2022-09-03T06:45:24.842 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/obj115 2022-09-03T06:45:24.843 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LmEyaGHKq302tpHYc11fQqTb+zI= 2022-09-03T06:45:24.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LmEyaGHKq302tpHYc11fQqTb+zI=', 'Content-Length': '0'} 2022-09-03T06:45:24.846 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d64badf2abe70d8b-006312f804-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.847 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:24.848 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:24.848 DEBUG:boto:Method: HEAD 2022-09-03T06:45:24.849 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:45:24.849 DEBUG:boto:Data: 2022-09-03T06:45:24.850 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.850 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:24.851 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.852 DEBUG:boto:Params: {} 2022-09-03T06:45:24.852 DEBUG:boto:Token: None 2022-09-03T06:45:24.853 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/obj115 2022-09-03T06:45:24.854 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LmEyaGHKq302tpHYc11fQqTb+zI= 2022-09-03T06:45:24.854 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LmEyaGHKq302tpHYc11fQqTb+zI=', 'Content-Length': '0'} 2022-09-03T06:45:24.856 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a9c4fd16e080679-006312f804-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.857 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:45:24.858 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:24.859 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:24.860 DEBUG:boto:Method: GET 2022-09-03T06:45:24.861 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:45:24.862 DEBUG:boto:Data: 2022-09-03T06:45:24.863 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.864 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:24.865 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.865 DEBUG:boto:Params: {} 2022-09-03T06:45:24.866 DEBUG:boto:Token: None 2022-09-03T06:45:24.867 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/obj115 2022-09-03T06:45:24.868 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zvWEBnJ4Rdlu5UEMG5IlL8WUc6I= 2022-09-03T06:45:24.870 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zvWEBnJ4Rdlu5UEMG5IlL8WUc6I=', 'Content-Length': '0'} 2022-09-03T06:45:24.872 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003dc1c73e02f91490-006312f804-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.914 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:24.915 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:24.916 DEBUG:boto:Method: GET 2022-09-03T06:45:24.917 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:45:24.918 DEBUG:boto:Data: 2022-09-03T06:45:24.920 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.921 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:24.921 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.922 DEBUG:boto:Params: {} 2022-09-03T06:45:24.923 DEBUG:boto:Token: None 2022-09-03T06:45:24.924 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/obj115 2022-09-03T06:45:24.925 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zvWEBnJ4Rdlu5UEMG5IlL8WUc6I= 2022-09-03T06:45:24.926 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zvWEBnJ4Rdlu5UEMG5IlL8WUc6I=', 'Content-Length': '0'} 2022-09-03T06:45:24.928 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a114d16bb7313dfd-006312f804-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.929 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:24.929 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:24.930 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:24.931 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:24.931 DEBUG:boto:Method: GET 2022-09-03T06:45:24.932 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:45:24.933 DEBUG:boto:Data: 2022-09-03T06:45:24.934 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.935 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:24.935 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.936 DEBUG:boto:Params: {} 2022-09-03T06:45:24.937 DEBUG:boto:Token: None 2022-09-03T06:45:24.937 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/ 2022-09-03T06:45:24.938 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5SMFy0jsOpAxGwGAhdVR+44rihs= 2022-09-03T06:45:24.939 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5SMFy0jsOpAxGwGAhdVR+44rihs=', 'Content-Length': '0'} 2022-09-03T06:45:24.943 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f5f2e23161f88539-006312f804-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.945 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:24.946 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:24.947 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:24.948 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:24.949 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:24.950 DEBUG:boto:Method: GET 2022-09-03T06:45:24.951 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:45:24.953 DEBUG:boto:Data: 2022-09-03T06:45:24.954 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.954 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:24.955 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.956 DEBUG:boto:Params: {} 2022-09-03T06:45:24.958 DEBUG:boto:Token: None 2022-09-03T06:45:24.959 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/ 2022-09-03T06:45:24.960 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5SMFy0jsOpAxGwGAhdVR+44rihs= 2022-09-03T06:45:24.961 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5SMFy0jsOpAxGwGAhdVR+44rihs=', 'Content-Length': '0'} 2022-09-03T06:45:24.966 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000868a9e7bd32d57be-006312f804-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.967 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:24.968 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:45:24.969 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:24.969 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:24.970 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:24.971 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:24.971 DEBUG:boto:Method: GET 2022-09-03T06:45:24.972 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:24.973 DEBUG:boto:Data: 2022-09-03T06:45:24.973 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.974 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:24.975 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.975 DEBUG:boto:Params: {} 2022-09-03T06:45:24.976 DEBUG:boto:Token: None 2022-09-03T06:45:24.977 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:24.977 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qFo+nypCBH+ar1CofFE4er6v2OM= 2022-09-03T06:45:24.978 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qFo+nypCBH+ar1CofFE4er6v2OM=', 'Content-Length': '0'} 2022-09-03T06:45:24.980 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000051b622c8527401de-006312f804-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.981 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:24.982 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:24.982 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:24.983 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:24.984 DEBUG:boto:Method: GET 2022-09-03T06:45:24.985 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:24.986 DEBUG:boto:Data: 2022-09-03T06:45:24.987 DEBUG:boto:Headers: {} 2022-09-03T06:45:24.987 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:24.988 DEBUG:boto:Port: 8000 2022-09-03T06:45:24.989 DEBUG:boto:Params: {} 2022-09-03T06:45:24.991 DEBUG:boto:Token: None 2022-09-03T06:45:24.992 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:24 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:24.993 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qFo+nypCBH+ar1CofFE4er6v2OM= 2022-09-03T06:45:24.994 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qFo+nypCBH+ar1CofFE4er6v2OM=', 'Content-Length': '0'} 2022-09-03T06:45:24.997 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009b4b76f201b0a2c9-006312f804-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:24.999 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:25.000 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:25.001 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:25.002 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:25.003 DEBUG:boto:Method: HEAD 2022-09-03T06:45:25.005 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:25.006 DEBUG:boto:Data: 2022-09-03T06:45:25.007 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.008 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.009 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.009 DEBUG:boto:Params: {} 2022-09-03T06:45:25.010 DEBUG:boto:Token: None 2022-09-03T06:45:25.011 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:25.012 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:amyUt1t+zrZs9df7qYKEs7ymPQY= 2022-09-03T06:45:25.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:amyUt1t+zrZs9df7qYKEs7ymPQY=', 'Content-Length': '0'} 2022-09-03T06:45:25.016 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d30148a8a6b3c4f-006312f805-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.017 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:25.018 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:25.019 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:25.020 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:25.021 DEBUG:boto:Method: HEAD 2022-09-03T06:45:25.021 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:25.022 DEBUG:boto:Data: 2022-09-03T06:45:25.023 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.024 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.025 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.026 DEBUG:boto:Params: {} 2022-09-03T06:45:25.027 DEBUG:boto:Token: None 2022-09-03T06:45:25.029 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:25.030 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:amyUt1t+zrZs9df7qYKEs7ymPQY= 2022-09-03T06:45:25.031 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:amyUt1t+zrZs9df7qYKEs7ymPQY=', 'Content-Length': '0'} 2022-09-03T06:45:25.034 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b95b63acd86f2253-006312f805-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.035 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:45:25.036 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:25.037 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:25.038 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:25.039 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:25.040 DEBUG:boto:Method: GET 2022-09-03T06:45:25.041 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:25.042 DEBUG:boto:Data: 2022-09-03T06:45:25.043 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.044 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.045 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.046 DEBUG:boto:Params: {} 2022-09-03T06:45:25.047 DEBUG:boto:Token: None 2022-09-03T06:45:25.048 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:25.049 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1oVk3OP0UkBnGTAyE6gz4glnPws= 2022-09-03T06:45:25.050 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1oVk3OP0UkBnGTAyE6gz4glnPws=', 'Content-Length': '0'} 2022-09-03T06:45:25.052 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008acde7ab7d1413fe-006312f805-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.054 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:25.054 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:25.055 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:25.056 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:25.057 DEBUG:boto:Method: GET 2022-09-03T06:45:25.058 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:25.059 DEBUG:boto:Data: 2022-09-03T06:45:25.060 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.061 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.062 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.063 DEBUG:boto:Params: {} 2022-09-03T06:45:25.064 DEBUG:boto:Token: None 2022-09-03T06:45:25.065 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:25.066 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1oVk3OP0UkBnGTAyE6gz4glnPws= 2022-09-03T06:45:25.067 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1oVk3OP0UkBnGTAyE6gz4glnPws=', 'Content-Length': '0'} 2022-09-03T06:45:25.072 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000549c17a45539606f-006312f805-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.074 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:25.074 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:25.075 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:25.075 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:25.076 DEBUG:boto:Method: GET 2022-09-03T06:45:25.077 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:45:25.078 DEBUG:boto:Data: 2022-09-03T06:45:25.078 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.079 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.079 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.080 DEBUG:boto:Params: {} 2022-09-03T06:45:25.081 DEBUG:boto:Token: None 2022-09-03T06:45:25.081 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/ 2022-09-03T06:45:25.082 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc= 2022-09-03T06:45:25.084 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc=', 'Content-Length': '0'} 2022-09-03T06:45:25.087 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aa28c4188c23f44f-006312f805-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.088 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:25.090 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:25.091 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:25.091 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:25.092 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:25.093 DEBUG:boto:Method: GET 2022-09-03T06:45:25.094 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:45:25.095 DEBUG:boto:Data: 2022-09-03T06:45:25.096 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.097 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.097 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.098 DEBUG:boto:Params: {} 2022-09-03T06:45:25.099 DEBUG:boto:Token: None 2022-09-03T06:45:25.100 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/ 2022-09-03T06:45:25.102 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc= 2022-09-03T06:45:25.103 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc=', 'Content-Length': '0'} 2022-09-03T06:45:25.107 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f52f36c988b2c060-006312f805-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.108 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:25.109 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:25.110 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:25.111 DEBUG:boto:Method: HEAD 2022-09-03T06:45:25.111 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:45:25.112 DEBUG:boto:Data: 2022-09-03T06:45:25.113 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.113 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.114 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.115 DEBUG:boto:Params: {} 2022-09-03T06:45:25.116 DEBUG:boto:Token: None 2022-09-03T06:45:25.117 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj116 2022-09-03T06:45:25.119 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6GFq46mTNyo7nm7YMN5HGz0VGv0= 2022-09-03T06:45:25.120 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6GFq46mTNyo7nm7YMN5HGz0VGv0=', 'Content-Length': '0'} 2022-09-03T06:45:25.122 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cbc41f67c2c66bea-006312f805-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.123 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:25.124 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:25.125 DEBUG:boto:Method: HEAD 2022-09-03T06:45:25.126 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:45:25.127 DEBUG:boto:Data: 2022-09-03T06:45:25.128 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.129 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.130 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.131 DEBUG:boto:Params: {} 2022-09-03T06:45:25.132 DEBUG:boto:Token: None 2022-09-03T06:45:25.133 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj116 2022-09-03T06:45:25.134 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6GFq46mTNyo7nm7YMN5HGz0VGv0= 2022-09-03T06:45:25.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6GFq46mTNyo7nm7YMN5HGz0VGv0=', 'Content-Length': '0'} 2022-09-03T06:45:25.138 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bdabfaa4d4fbb270-006312f805-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.139 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:45:25.140 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:25.141 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:25.142 DEBUG:boto:Method: GET 2022-09-03T06:45:25.143 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:45:25.144 DEBUG:boto:Data: 2022-09-03T06:45:25.145 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.146 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.147 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.148 DEBUG:boto:Params: {} 2022-09-03T06:45:25.149 DEBUG:boto:Token: None 2022-09-03T06:45:25.150 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj116 2022-09-03T06:45:25.152 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q/nlggLRr5q2ZTnwX1VpZ8FRF5U= 2022-09-03T06:45:25.153 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q/nlggLRr5q2ZTnwX1VpZ8FRF5U=', 'Content-Length': '0'} 2022-09-03T06:45:25.156 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000013603e8ad9d38ab4-006312f805-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.202 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:25.203 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:25.204 DEBUG:boto:Method: GET 2022-09-03T06:45:25.205 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:45:25.206 DEBUG:boto:Data: 2022-09-03T06:45:25.207 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.208 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.209 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.210 DEBUG:boto:Params: {} 2022-09-03T06:45:25.212 DEBUG:boto:Token: None 2022-09-03T06:45:25.214 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj116 2022-09-03T06:45:25.215 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q/nlggLRr5q2ZTnwX1VpZ8FRF5U= 2022-09-03T06:45:25.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q/nlggLRr5q2ZTnwX1VpZ8FRF5U=', 'Content-Length': '0'} 2022-09-03T06:45:25.218 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d2acef87649aef5-006312f805-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.219 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:25.220 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:25.221 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:25.222 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:25.223 DEBUG:boto:Method: GET 2022-09-03T06:45:25.224 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:45:25.225 DEBUG:boto:Data: 2022-09-03T06:45:25.226 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.226 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.227 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.228 DEBUG:boto:Params: {} 2022-09-03T06:45:25.229 DEBUG:boto:Token: None 2022-09-03T06:45:25.229 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/ 2022-09-03T06:45:25.230 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc= 2022-09-03T06:45:25.231 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc=', 'Content-Length': '0'} 2022-09-03T06:45:25.234 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ecae59cca25d5f85-006312f805-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.235 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:25.236 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:25.236 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:25.237 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:25.237 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:25.238 DEBUG:boto:Method: GET 2022-09-03T06:45:25.238 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:45:25.239 DEBUG:boto:Data: 2022-09-03T06:45:25.239 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.240 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.241 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.241 DEBUG:boto:Params: {} 2022-09-03T06:45:25.242 DEBUG:boto:Token: None 2022-09-03T06:45:25.243 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/ 2022-09-03T06:45:25.243 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc= 2022-09-03T06:45:25.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc=', 'Content-Length': '0'} 2022-09-03T06:45:25.247 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003f201796ab59dab2-006312f805-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.248 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:25.249 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:45:25.249 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:25.250 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:25.250 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.251 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.252 DEBUG:boto:Method: GET 2022-09-03T06:45:25.252 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.253 DEBUG:boto:Data: 2022-09-03T06:45:25.253 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.254 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.255 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.255 DEBUG:boto:Params: {} 2022-09-03T06:45:25.256 DEBUG:boto:Token: None 2022-09-03T06:45:25.256 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.257 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ej5LZhhwIy3r30Ux+jLyEPKIiZ4= 2022-09-03T06:45:25.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ej5LZhhwIy3r30Ux+jLyEPKIiZ4=', 'Content-Length': '0'} 2022-09-03T06:45:25.261 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007efa156ead43c993-006312f805-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.262 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:25.262 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:25.263 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.263 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.264 DEBUG:boto:Method: GET 2022-09-03T06:45:25.264 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.265 DEBUG:boto:Data: 2022-09-03T06:45:25.265 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.266 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.267 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.267 DEBUG:boto:Params: {} 2022-09-03T06:45:25.268 DEBUG:boto:Token: None 2022-09-03T06:45:25.270 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.271 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ej5LZhhwIy3r30Ux+jLyEPKIiZ4= 2022-09-03T06:45:25.272 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ej5LZhhwIy3r30Ux+jLyEPKIiZ4=', 'Content-Length': '0'} 2022-09-03T06:45:25.274 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d6681aad3fc1eb11-006312f805-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.275 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:25.276 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:25.276 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.277 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.277 DEBUG:boto:Method: HEAD 2022-09-03T06:45:25.278 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.278 DEBUG:boto:Data: 2022-09-03T06:45:25.279 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.279 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.280 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.280 DEBUG:boto:Params: {} 2022-09-03T06:45:25.281 DEBUG:boto:Token: None 2022-09-03T06:45:25.282 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.282 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s9IRdnImThLpMVUl1SVEg1rs5aY= 2022-09-03T06:45:25.283 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s9IRdnImThLpMVUl1SVEg1rs5aY=', 'Content-Length': '0'} 2022-09-03T06:45:25.285 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004e2b7fd0dafc3c31-006312f805-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.286 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:25.287 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:25.287 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.288 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.288 DEBUG:boto:Method: HEAD 2022-09-03T06:45:25.289 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.289 DEBUG:boto:Data: 2022-09-03T06:45:25.290 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.290 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.291 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.291 DEBUG:boto:Params: {} 2022-09-03T06:45:25.292 DEBUG:boto:Token: None 2022-09-03T06:45:25.293 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.293 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s9IRdnImThLpMVUl1SVEg1rs5aY= 2022-09-03T06:45:25.294 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s9IRdnImThLpMVUl1SVEg1rs5aY=', 'Content-Length': '0'} 2022-09-03T06:45:25.296 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e9e56769003e6167-006312f805-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.297 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:45:25.297 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:25.298 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:25.298 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.299 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.300 DEBUG:boto:Method: GET 2022-09-03T06:45:25.302 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.303 DEBUG:boto:Data: 2022-09-03T06:45:25.303 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.304 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.304 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.305 DEBUG:boto:Params: {} 2022-09-03T06:45:25.305 DEBUG:boto:Token: None 2022-09-03T06:45:25.306 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.307 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ej5LZhhwIy3r30Ux+jLyEPKIiZ4= 2022-09-03T06:45:25.307 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ej5LZhhwIy3r30Ux+jLyEPKIiZ4=', 'Content-Length': '0'} 2022-09-03T06:45:25.310 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bdfe95c6323eaef4-006312f805-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.311 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:25.312 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:25.312 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.313 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.313 DEBUG:boto:Method: GET 2022-09-03T06:45:25.314 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.315 DEBUG:boto:Data: 2022-09-03T06:45:25.316 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.317 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.317 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.318 DEBUG:boto:Params: {} 2022-09-03T06:45:25.319 DEBUG:boto:Token: None 2022-09-03T06:45:25.319 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:25.320 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ej5LZhhwIy3r30Ux+jLyEPKIiZ4= 2022-09-03T06:45:25.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ej5LZhhwIy3r30Ux+jLyEPKIiZ4=', 'Content-Length': '0'} 2022-09-03T06:45:25.323 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ac816e0498b8d8ef-006312f805-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.324 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:25.325 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:25.326 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:25.326 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:25.327 DEBUG:boto:Method: GET 2022-09-03T06:45:25.328 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:45:25.329 DEBUG:boto:Data: 2022-09-03T06:45:25.330 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.331 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.332 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.332 DEBUG:boto:Params: {} 2022-09-03T06:45:25.333 DEBUG:boto:Token: None 2022-09-03T06:45:25.334 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/ 2022-09-03T06:45:25.335 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc= 2022-09-03T06:45:25.335 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc=', 'Content-Length': '0'} 2022-09-03T06:45:25.338 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009757024e45573cf8-006312f805-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.339 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:25.339 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:25.340 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:25.340 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:25.341 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:25.341 DEBUG:boto:Method: GET 2022-09-03T06:45:25.342 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:45:25.343 DEBUG:boto:Data: 2022-09-03T06:45:25.343 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.344 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.344 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.345 DEBUG:boto:Params: {} 2022-09-03T06:45:25.346 DEBUG:boto:Token: None 2022-09-03T06:45:25.352 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/ 2022-09-03T06:45:25.353 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc= 2022-09-03T06:45:25.354 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc=', 'Content-Length': '0'} 2022-09-03T06:45:25.358 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000311ea81e62e4e9ff-006312f805-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.408 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:25.410 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:25.411 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:25.411 DEBUG:boto:Method: HEAD 2022-09-03T06:45:25.412 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:45:25.412 DEBUG:boto:Data: 2022-09-03T06:45:25.413 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.413 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.414 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.415 DEBUG:boto:Params: {} 2022-09-03T06:45:25.416 DEBUG:boto:Token: None 2022-09-03T06:45:25.416 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj145 2022-09-03T06:45:25.417 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CT/1CydvC7DY8labKEr05gADoHA= 2022-09-03T06:45:25.418 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CT/1CydvC7DY8labKEr05gADoHA=', 'Content-Length': '0'} 2022-09-03T06:45:25.420 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b11bfef18590092a-006312f805-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.422 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:25.423 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:25.423 DEBUG:boto:Method: HEAD 2022-09-03T06:45:25.424 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:45:25.425 DEBUG:boto:Data: 2022-09-03T06:45:25.426 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.427 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.428 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.429 DEBUG:boto:Params: {} 2022-09-03T06:45:25.429 DEBUG:boto:Token: None 2022-09-03T06:45:25.430 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj145 2022-09-03T06:45:25.431 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CT/1CydvC7DY8labKEr05gADoHA= 2022-09-03T06:45:25.431 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CT/1CydvC7DY8labKEr05gADoHA=', 'Content-Length': '0'} 2022-09-03T06:45:25.435 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000037b10b37b1eba29c-006312f805-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.436 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:45:25.437 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:25.439 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:25.440 DEBUG:boto:Method: GET 2022-09-03T06:45:25.441 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:45:25.442 DEBUG:boto:Data: 2022-09-03T06:45:25.444 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.445 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.446 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.447 DEBUG:boto:Params: {} 2022-09-03T06:45:25.448 DEBUG:boto:Token: None 2022-09-03T06:45:25.449 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj145 2022-09-03T06:45:25.451 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ivafvtl7lsTShsRnQPDbVXuy9ms= 2022-09-03T06:45:25.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ivafvtl7lsTShsRnQPDbVXuy9ms=', 'Content-Length': '0'} 2022-09-03T06:45:25.454 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000205e232007302bbf-006312f805-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.455 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:25.456 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:25.456 DEBUG:boto:Method: GET 2022-09-03T06:45:25.457 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:45:25.457 DEBUG:boto:Data: 2022-09-03T06:45:25.458 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.459 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.460 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.461 DEBUG:boto:Params: {} 2022-09-03T06:45:25.461 DEBUG:boto:Token: None 2022-09-03T06:45:25.462 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj145 2022-09-03T06:45:25.463 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ivafvtl7lsTShsRnQPDbVXuy9ms= 2022-09-03T06:45:25.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ivafvtl7lsTShsRnQPDbVXuy9ms=', 'Content-Length': '0'} 2022-09-03T06:45:25.466 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000030128c8ce3497c5d-006312f805-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.467 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:25.468 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:25.468 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:25.469 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:25.469 DEBUG:boto:Method: GET 2022-09-03T06:45:25.470 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:45:25.471 DEBUG:boto:Data: 2022-09-03T06:45:25.473 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.474 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.475 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.476 DEBUG:boto:Params: {} 2022-09-03T06:45:25.477 DEBUG:boto:Token: None 2022-09-03T06:45:25.479 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/ 2022-09-03T06:45:25.480 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc= 2022-09-03T06:45:25.481 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc=', 'Content-Length': '0'} 2022-09-03T06:45:25.485 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006cebf723bbae52ba-006312f805-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.530 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:25.531 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:25.532 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:25.533 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:25.545 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:25.545 DEBUG:boto:Method: GET 2022-09-03T06:45:25.546 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:45:25.547 DEBUG:boto:Data: 2022-09-03T06:45:25.547 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.548 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.548 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.549 DEBUG:boto:Params: {} 2022-09-03T06:45:25.550 DEBUG:boto:Token: None 2022-09-03T06:45:25.550 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/ 2022-09-03T06:45:25.551 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc= 2022-09-03T06:45:25.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc=', 'Content-Length': '0'} 2022-09-03T06:45:25.562 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002f43dde7642c4f32-006312f805-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.564 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:25.565 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:45:25.565 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:25.566 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:25.567 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.567 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.568 DEBUG:boto:Method: GET 2022-09-03T06:45:25.569 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.569 DEBUG:boto:Data: 2022-09-03T06:45:25.570 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.570 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.571 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.571 DEBUG:boto:Params: {} 2022-09-03T06:45:25.572 DEBUG:boto:Token: None 2022-09-03T06:45:25.573 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.574 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Je7qWe+8ZtXz7yRD8SBnJXaey9g= 2022-09-03T06:45:25.574 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Je7qWe+8ZtXz7yRD8SBnJXaey9g=', 'Content-Length': '0'} 2022-09-03T06:45:25.576 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000015258906d1f8885d-006312f805-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.577 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:25.578 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:25.579 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.579 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.580 DEBUG:boto:Method: GET 2022-09-03T06:45:25.580 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.581 DEBUG:boto:Data: 2022-09-03T06:45:25.581 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.582 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.582 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.583 DEBUG:boto:Params: {} 2022-09-03T06:45:25.584 DEBUG:boto:Token: None 2022-09-03T06:45:25.584 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.585 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Je7qWe+8ZtXz7yRD8SBnJXaey9g= 2022-09-03T06:45:25.586 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Je7qWe+8ZtXz7yRD8SBnJXaey9g=', 'Content-Length': '0'} 2022-09-03T06:45:25.588 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000c80b067777842e4-006312f805-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.589 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:25.590 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:25.591 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.591 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.592 DEBUG:boto:Method: HEAD 2022-09-03T06:45:25.593 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.594 DEBUG:boto:Data: 2022-09-03T06:45:25.595 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.596 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.597 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.598 DEBUG:boto:Params: {} 2022-09-03T06:45:25.599 DEBUG:boto:Token: None 2022-09-03T06:45:25.600 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.601 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t4T/CBkHeIVgRtySFlfn1HqwCcQ= 2022-09-03T06:45:25.602 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t4T/CBkHeIVgRtySFlfn1HqwCcQ=', 'Content-Length': '0'} 2022-09-03T06:45:25.605 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f9d723cecf87193e-006312f805-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.607 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:25.608 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:25.608 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.609 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.610 DEBUG:boto:Method: HEAD 2022-09-03T06:45:25.611 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.612 DEBUG:boto:Data: 2022-09-03T06:45:25.613 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.614 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.615 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.616 DEBUG:boto:Params: {} 2022-09-03T06:45:25.617 DEBUG:boto:Token: None 2022-09-03T06:45:25.618 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.619 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t4T/CBkHeIVgRtySFlfn1HqwCcQ= 2022-09-03T06:45:25.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t4T/CBkHeIVgRtySFlfn1HqwCcQ=', 'Content-Length': '0'} 2022-09-03T06:45:25.623 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000abc0047ce7d0007e-006312f805-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.624 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:45:25.625 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:25.626 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:25.628 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.629 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.630 DEBUG:boto:Method: GET 2022-09-03T06:45:25.631 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.632 DEBUG:boto:Data: 2022-09-03T06:45:25.633 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.634 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.635 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.635 DEBUG:boto:Params: {} 2022-09-03T06:45:25.636 DEBUG:boto:Token: None 2022-09-03T06:45:25.637 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.638 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Je7qWe+8ZtXz7yRD8SBnJXaey9g= 2022-09-03T06:45:25.639 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Je7qWe+8ZtXz7yRD8SBnJXaey9g=', 'Content-Length': '0'} 2022-09-03T06:45:25.641 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d1d0aea66fb51755-006312f805-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.642 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:25.643 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:25.643 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.644 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.645 DEBUG:boto:Method: GET 2022-09-03T06:45:25.646 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.647 DEBUG:boto:Data: 2022-09-03T06:45:25.648 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.648 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.649 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.649 DEBUG:boto:Params: {} 2022-09-03T06:45:25.650 DEBUG:boto:Token: None 2022-09-03T06:45:25.651 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:25.652 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Je7qWe+8ZtXz7yRD8SBnJXaey9g= 2022-09-03T06:45:25.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Je7qWe+8ZtXz7yRD8SBnJXaey9g=', 'Content-Length': '0'} 2022-09-03T06:45:25.654 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000055b4664b967f92d4-006312f805-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.702 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:25.703 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:25.703 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:25.704 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:25.704 DEBUG:boto:Method: GET 2022-09-03T06:45:25.705 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:45:25.706 DEBUG:boto:Data: 2022-09-03T06:45:25.706 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.707 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.707 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.708 DEBUG:boto:Params: {} 2022-09-03T06:45:25.708 DEBUG:boto:Token: None 2022-09-03T06:45:25.709 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/ 2022-09-03T06:45:25.709 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc= 2022-09-03T06:45:25.710 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc=', 'Content-Length': '0'} 2022-09-03T06:45:25.713 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e576c9ff9db7dfdc-006312f805-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.714 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:25.715 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:25.715 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:25.716 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:25.716 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:25.717 DEBUG:boto:Method: GET 2022-09-03T06:45:25.717 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:45:25.718 DEBUG:boto:Data: 2022-09-03T06:45:25.719 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.719 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.720 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.720 DEBUG:boto:Params: {} 2022-09-03T06:45:25.721 DEBUG:boto:Token: None 2022-09-03T06:45:25.721 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/ 2022-09-03T06:45:25.722 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc= 2022-09-03T06:45:25.723 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc=', 'Content-Length': '0'} 2022-09-03T06:45:25.726 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000175e7b3a9a5ee807-006312f805-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.728 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:25.729 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:25.730 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:25.731 DEBUG:boto:Method: HEAD 2022-09-03T06:45:25.732 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:45:25.733 DEBUG:boto:Data: 2022-09-03T06:45:25.734 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.735 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.736 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.737 DEBUG:boto:Params: {} 2022-09-03T06:45:25.738 DEBUG:boto:Token: None 2022-09-03T06:45:25.739 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj17 2022-09-03T06:45:25.740 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a8QGvE2JhHRvocOKee7lJWNPL0g= 2022-09-03T06:45:25.741 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a8QGvE2JhHRvocOKee7lJWNPL0g=', 'Content-Length': '0'} 2022-09-03T06:45:25.744 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b8ff6240293ed1a-006312f805-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.746 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:25.746 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:25.747 DEBUG:boto:Method: HEAD 2022-09-03T06:45:25.748 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:45:25.748 DEBUG:boto:Data: 2022-09-03T06:45:25.749 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.749 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.750 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.750 DEBUG:boto:Params: {} 2022-09-03T06:45:25.751 DEBUG:boto:Token: None 2022-09-03T06:45:25.751 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj17 2022-09-03T06:45:25.752 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a8QGvE2JhHRvocOKee7lJWNPL0g= 2022-09-03T06:45:25.752 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a8QGvE2JhHRvocOKee7lJWNPL0g=', 'Content-Length': '0'} 2022-09-03T06:45:25.754 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d73bd78ad69bdc71-006312f805-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.755 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:45:25.756 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:25.756 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:25.757 DEBUG:boto:Method: GET 2022-09-03T06:45:25.757 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:45:25.758 DEBUG:boto:Data: 2022-09-03T06:45:25.758 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.759 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.759 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.760 DEBUG:boto:Params: {} 2022-09-03T06:45:25.760 DEBUG:boto:Token: None 2022-09-03T06:45:25.761 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj17 2022-09-03T06:45:25.762 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0KJ6ROKwuaAqxE+HZuwROCJgqF8= 2022-09-03T06:45:25.762 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0KJ6ROKwuaAqxE+HZuwROCJgqF8=', 'Content-Length': '0'} 2022-09-03T06:45:25.765 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f3c7918b972e4103-006312f805-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.766 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:25.767 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:25.767 DEBUG:boto:Method: GET 2022-09-03T06:45:25.768 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:45:25.769 DEBUG:boto:Data: 2022-09-03T06:45:25.769 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.770 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.770 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.771 DEBUG:boto:Params: {} 2022-09-03T06:45:25.772 DEBUG:boto:Token: None 2022-09-03T06:45:25.772 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj17 2022-09-03T06:45:25.773 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0KJ6ROKwuaAqxE+HZuwROCJgqF8= 2022-09-03T06:45:25.774 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0KJ6ROKwuaAqxE+HZuwROCJgqF8=', 'Content-Length': '0'} 2022-09-03T06:45:25.775 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000038b195bec2f6c3a9-006312f805-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.823 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:25.825 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:25.826 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:25.828 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:25.829 DEBUG:boto:Method: GET 2022-09-03T06:45:25.830 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:45:25.831 DEBUG:boto:Data: 2022-09-03T06:45:25.832 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.833 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.834 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.835 DEBUG:boto:Params: {} 2022-09-03T06:45:25.836 DEBUG:boto:Token: None 2022-09-03T06:45:25.837 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/ 2022-09-03T06:45:25.838 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc= 2022-09-03T06:45:25.839 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc=', 'Content-Length': '0'} 2022-09-03T06:45:25.843 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f8f6dadec95f32f8-006312f805-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.844 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:25.846 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:25.847 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:25.848 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:25.849 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:25.850 DEBUG:boto:Method: GET 2022-09-03T06:45:25.851 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:45:25.852 DEBUG:boto:Data: 2022-09-03T06:45:25.853 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.854 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.855 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.856 DEBUG:boto:Params: {} 2022-09-03T06:45:25.858 DEBUG:boto:Token: None 2022-09-03T06:45:25.859 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/ 2022-09-03T06:45:25.860 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc= 2022-09-03T06:45:25.861 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc=', 'Content-Length': '0'} 2022-09-03T06:45:25.866 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000005abeb139cfd359a-006312f805-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.867 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:25.869 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:45:25.870 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:25.870 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:25.871 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.871 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.872 DEBUG:boto:Method: GET 2022-09-03T06:45:25.873 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.874 DEBUG:boto:Data: 2022-09-03T06:45:25.875 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.875 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.877 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.878 DEBUG:boto:Params: {} 2022-09-03T06:45:25.880 DEBUG:boto:Token: None 2022-09-03T06:45:25.881 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.882 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7nAcM4zXhkemHqUwQqLFS8BLdLQ= 2022-09-03T06:45:25.884 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7nAcM4zXhkemHqUwQqLFS8BLdLQ=', 'Content-Length': '0'} 2022-09-03T06:45:25.887 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004823a11595bbe14e-006312f805-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.888 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:25.889 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:25.891 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.892 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.893 DEBUG:boto:Method: GET 2022-09-03T06:45:25.894 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.895 DEBUG:boto:Data: 2022-09-03T06:45:25.896 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.898 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.898 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.899 DEBUG:boto:Params: {} 2022-09-03T06:45:25.900 DEBUG:boto:Token: None 2022-09-03T06:45:25.901 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.901 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7nAcM4zXhkemHqUwQqLFS8BLdLQ= 2022-09-03T06:45:25.902 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7nAcM4zXhkemHqUwQqLFS8BLdLQ=', 'Content-Length': '0'} 2022-09-03T06:45:25.904 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ada619883119cc6-006312f805-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.905 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:25.906 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:25.906 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.907 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.907 DEBUG:boto:Method: HEAD 2022-09-03T06:45:25.908 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.908 DEBUG:boto:Data: 2022-09-03T06:45:25.909 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.909 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.910 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.911 DEBUG:boto:Params: {} 2022-09-03T06:45:25.911 DEBUG:boto:Token: None 2022-09-03T06:45:25.912 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.912 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+x+7d+zr+4wqTXeo4OzEMn7ktRQ= 2022-09-03T06:45:25.913 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+x+7d+zr+4wqTXeo4OzEMn7ktRQ=', 'Content-Length': '0'} 2022-09-03T06:45:25.915 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b8cff58e56c7b173-006312f805-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.916 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:25.917 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:25.917 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.918 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.919 DEBUG:boto:Method: HEAD 2022-09-03T06:45:25.920 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.921 DEBUG:boto:Data: 2022-09-03T06:45:25.922 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.923 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.924 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.925 DEBUG:boto:Params: {} 2022-09-03T06:45:25.926 DEBUG:boto:Token: None 2022-09-03T06:45:25.926 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.928 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+x+7d+zr+4wqTXeo4OzEMn7ktRQ= 2022-09-03T06:45:25.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+x+7d+zr+4wqTXeo4OzEMn7ktRQ=', 'Content-Length': '0'} 2022-09-03T06:45:25.931 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005a85f3d63680a2f4-006312f805-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.932 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:45:25.933 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:25.934 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:25.935 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.935 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.936 DEBUG:boto:Method: GET 2022-09-03T06:45:25.936 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.937 DEBUG:boto:Data: 2022-09-03T06:45:25.937 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.938 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.939 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.940 DEBUG:boto:Params: {} 2022-09-03T06:45:25.941 DEBUG:boto:Token: None 2022-09-03T06:45:25.942 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.944 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7nAcM4zXhkemHqUwQqLFS8BLdLQ= 2022-09-03T06:45:25.945 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7nAcM4zXhkemHqUwQqLFS8BLdLQ=', 'Content-Length': '0'} 2022-09-03T06:45:25.948 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c8591de384116c41-006312f805-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.952 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:25.953 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:25.954 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.955 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.955 DEBUG:boto:Method: GET 2022-09-03T06:45:25.956 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.957 DEBUG:boto:Data: 2022-09-03T06:45:25.957 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.958 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.958 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.959 DEBUG:boto:Params: {} 2022-09-03T06:45:25.960 DEBUG:boto:Token: None 2022-09-03T06:45:25.960 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:25.961 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7nAcM4zXhkemHqUwQqLFS8BLdLQ= 2022-09-03T06:45:25.962 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7nAcM4zXhkemHqUwQqLFS8BLdLQ=', 'Content-Length': '0'} 2022-09-03T06:45:25.964 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000f2d2aeb0004fa25-006312f805-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.966 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:25.967 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:25.967 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:25.968 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:25.968 DEBUG:boto:Method: GET 2022-09-03T06:45:25.969 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:45:25.970 DEBUG:boto:Data: 2022-09-03T06:45:25.970 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.971 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:25.971 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.972 DEBUG:boto:Params: {} 2022-09-03T06:45:25.973 DEBUG:boto:Token: None 2022-09-03T06:45:25.973 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/ 2022-09-03T06:45:25.975 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc= 2022-09-03T06:45:25.975 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc=', 'Content-Length': '0'} 2022-09-03T06:45:25.978 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b70a5702b38678cf-006312f805-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:25.980 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:25.981 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:25.982 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:25.984 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:25.985 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:25.986 DEBUG:boto:Method: GET 2022-09-03T06:45:25.988 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:45:25.989 DEBUG:boto:Data: 2022-09-03T06:45:25.989 DEBUG:boto:Headers: {} 2022-09-03T06:45:25.990 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:25.990 DEBUG:boto:Port: 8000 2022-09-03T06:45:25.991 DEBUG:boto:Params: {} 2022-09-03T06:45:25.991 DEBUG:boto:Token: None 2022-09-03T06:45:25.992 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:25 GMT /dsgvcy-75/ 2022-09-03T06:45:25.993 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc= 2022-09-03T06:45:25.993 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HPZ/bu865rd3KYgcVt5Citd7Vvc=', 'Content-Length': '0'} 2022-09-03T06:45:25.998 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000036312daad4966f50-006312f805-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.000 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:26.001 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:26.003 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:26.004 DEBUG:boto:Method: HEAD 2022-09-03T06:45:26.004 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:45:26.006 DEBUG:boto:Data: 2022-09-03T06:45:26.006 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.007 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.007 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.008 DEBUG:boto:Params: {} 2022-09-03T06:45:26.008 DEBUG:boto:Token: None 2022-09-03T06:45:26.009 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj19 2022-09-03T06:45:26.010 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m6ECyHPs/Midoh+cRcbMHbwGe+w= 2022-09-03T06:45:26.010 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m6ECyHPs/Midoh+cRcbMHbwGe+w=', 'Content-Length': '0'} 2022-09-03T06:45:26.012 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065a8b490b72bd3c9-006312f806-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.014 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:26.015 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:26.016 DEBUG:boto:Method: HEAD 2022-09-03T06:45:26.017 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:45:26.018 DEBUG:boto:Data: 2022-09-03T06:45:26.019 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.019 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:26.020 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.020 DEBUG:boto:Params: {} 2022-09-03T06:45:26.021 DEBUG:boto:Token: None 2022-09-03T06:45:26.021 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj19 2022-09-03T06:45:26.022 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m6ECyHPs/Midoh+cRcbMHbwGe+w= 2022-09-03T06:45:26.023 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m6ECyHPs/Midoh+cRcbMHbwGe+w=', 'Content-Length': '0'} 2022-09-03T06:45:26.025 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000075c538e6f72268d1-006312f806-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.026 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:45:26.027 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:26.027 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:26.028 DEBUG:boto:Method: GET 2022-09-03T06:45:26.029 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:45:26.029 DEBUG:boto:Data: 2022-09-03T06:45:26.030 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.031 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.032 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.033 DEBUG:boto:Params: {} 2022-09-03T06:45:26.035 DEBUG:boto:Token: None 2022-09-03T06:45:26.036 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj19 2022-09-03T06:45:26.037 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e3PaJZOpjcQXLydjcy0idY4m4Ic= 2022-09-03T06:45:26.038 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e3PaJZOpjcQXLydjcy0idY4m4Ic=', 'Content-Length': '0'} 2022-09-03T06:45:26.041 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d50ffb30c433485-006312f806-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.042 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:26.043 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:26.044 DEBUG:boto:Method: GET 2022-09-03T06:45:26.046 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:45:26.047 DEBUG:boto:Data: 2022-09-03T06:45:26.048 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.049 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:26.050 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.051 DEBUG:boto:Params: {} 2022-09-03T06:45:26.052 DEBUG:boto:Token: None 2022-09-03T06:45:26.053 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj19 2022-09-03T06:45:26.053 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e3PaJZOpjcQXLydjcy0idY4m4Ic= 2022-09-03T06:45:26.054 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e3PaJZOpjcQXLydjcy0idY4m4Ic=', 'Content-Length': '0'} 2022-09-03T06:45:26.057 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b728f85268ecc869-006312f806-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.058 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:26.059 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:26.059 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:26.060 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:26.061 DEBUG:boto:Method: GET 2022-09-03T06:45:26.062 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:45:26.063 DEBUG:boto:Data: 2022-09-03T06:45:26.064 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.065 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.066 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.067 DEBUG:boto:Params: {} 2022-09-03T06:45:26.068 DEBUG:boto:Token: None 2022-09-03T06:45:26.068 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/ 2022-09-03T06:45:26.070 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E= 2022-09-03T06:45:26.071 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E=', 'Content-Length': '0'} 2022-09-03T06:45:26.076 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009c923de897f68c7f-006312f806-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.077 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:26.078 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:26.078 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:26.079 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:26.080 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:26.080 DEBUG:boto:Method: GET 2022-09-03T06:45:26.081 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:45:26.081 DEBUG:boto:Data: 2022-09-03T06:45:26.082 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.082 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:26.083 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.083 DEBUG:boto:Params: {} 2022-09-03T06:45:26.084 DEBUG:boto:Token: None 2022-09-03T06:45:26.085 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/ 2022-09-03T06:45:26.085 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E= 2022-09-03T06:45:26.086 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E=', 'Content-Length': '0'} 2022-09-03T06:45:26.090 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b59e8934200c0c83-006312f806-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.091 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:26.093 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:45:26.094 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:26.095 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:26.096 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.097 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.099 DEBUG:boto:Method: GET 2022-09-03T06:45:26.100 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.100 DEBUG:boto:Data: 2022-09-03T06:45:26.101 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.102 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.102 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.103 DEBUG:boto:Params: {} 2022-09-03T06:45:26.105 DEBUG:boto:Token: None 2022-09-03T06:45:26.105 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.106 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kk2Zurt8ZwqC/okSK3ze6vqrMGs= 2022-09-03T06:45:26.107 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kk2Zurt8ZwqC/okSK3ze6vqrMGs=', 'Content-Length': '0'} 2022-09-03T06:45:26.109 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b3169032505830e-006312f806-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.154 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:26.155 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:26.155 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.156 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.157 DEBUG:boto:Method: GET 2022-09-03T06:45:26.157 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.159 DEBUG:boto:Data: 2022-09-03T06:45:26.160 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.161 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:26.162 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.163 DEBUG:boto:Params: {} 2022-09-03T06:45:26.164 DEBUG:boto:Token: None 2022-09-03T06:45:26.166 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.167 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kk2Zurt8ZwqC/okSK3ze6vqrMGs= 2022-09-03T06:45:26.168 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kk2Zurt8ZwqC/okSK3ze6vqrMGs=', 'Content-Length': '0'} 2022-09-03T06:45:26.171 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074185f9d07fab9b5-006312f806-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.173 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:26.175 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:26.177 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.178 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.179 DEBUG:boto:Method: HEAD 2022-09-03T06:45:26.180 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.181 DEBUG:boto:Data: 2022-09-03T06:45:26.182 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.183 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.184 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.186 DEBUG:boto:Params: {} 2022-09-03T06:45:26.187 DEBUG:boto:Token: None 2022-09-03T06:45:26.188 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.189 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IGBG2Cmvvh0Hh4oCoI/j4hzej+0= 2022-09-03T06:45:26.190 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IGBG2Cmvvh0Hh4oCoI/j4hzej+0=', 'Content-Length': '0'} 2022-09-03T06:45:26.193 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c4913b29234a968-006312f806-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.194 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:26.194 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:26.195 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.195 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.196 DEBUG:boto:Method: HEAD 2022-09-03T06:45:26.197 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.197 DEBUG:boto:Data: 2022-09-03T06:45:26.198 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.198 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:26.199 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.199 DEBUG:boto:Params: {} 2022-09-03T06:45:26.200 DEBUG:boto:Token: None 2022-09-03T06:45:26.200 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.201 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IGBG2Cmvvh0Hh4oCoI/j4hzej+0= 2022-09-03T06:45:26.202 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IGBG2Cmvvh0Hh4oCoI/j4hzej+0=', 'Content-Length': '0'} 2022-09-03T06:45:26.204 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000377ed0156f23e2ef-006312f806-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.204 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:45:26.205 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:26.206 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:26.206 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.207 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.207 DEBUG:boto:Method: GET 2022-09-03T06:45:26.208 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.208 DEBUG:boto:Data: 2022-09-03T06:45:26.209 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.209 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.210 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.211 DEBUG:boto:Params: {} 2022-09-03T06:45:26.211 DEBUG:boto:Token: None 2022-09-03T06:45:26.212 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.212 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kk2Zurt8ZwqC/okSK3ze6vqrMGs= 2022-09-03T06:45:26.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kk2Zurt8ZwqC/okSK3ze6vqrMGs=', 'Content-Length': '0'} 2022-09-03T06:45:26.216 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eb5b53ffc465c9e7-006312f806-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.218 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:26.218 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:26.219 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.220 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.220 DEBUG:boto:Method: GET 2022-09-03T06:45:26.221 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.222 DEBUG:boto:Data: 2022-09-03T06:45:26.222 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.223 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:26.224 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.224 DEBUG:boto:Params: {} 2022-09-03T06:45:26.225 DEBUG:boto:Token: None 2022-09-03T06:45:26.226 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:26.227 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kk2Zurt8ZwqC/okSK3ze6vqrMGs= 2022-09-03T06:45:26.228 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kk2Zurt8ZwqC/okSK3ze6vqrMGs=', 'Content-Length': '0'} 2022-09-03T06:45:26.230 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000091f7ec560f9cb420-006312f806-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.231 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:26.232 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:26.233 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:26.234 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:26.234 DEBUG:boto:Method: GET 2022-09-03T06:45:26.235 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:45:26.236 DEBUG:boto:Data: 2022-09-03T06:45:26.236 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.237 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.238 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.238 DEBUG:boto:Params: {} 2022-09-03T06:45:26.239 DEBUG:boto:Token: None 2022-09-03T06:45:26.240 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/ 2022-09-03T06:45:26.241 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E= 2022-09-03T06:45:26.241 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E=', 'Content-Length': '0'} 2022-09-03T06:45:26.244 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005470a5020d0115c9-006312f806-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.245 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:26.246 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:26.246 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:26.247 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:26.247 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:26.248 DEBUG:boto:Method: GET 2022-09-03T06:45:26.249 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:45:26.249 DEBUG:boto:Data: 2022-09-03T06:45:26.250 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.251 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:26.252 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.252 DEBUG:boto:Params: {} 2022-09-03T06:45:26.253 DEBUG:boto:Token: None 2022-09-03T06:45:26.253 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/ 2022-09-03T06:45:26.254 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E= 2022-09-03T06:45:26.255 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E=', 'Content-Length': '0'} 2022-09-03T06:45:26.260 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e11d13d373906d08-006312f806-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.302 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:26.304 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:26.304 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:26.305 DEBUG:boto:Method: HEAD 2022-09-03T06:45:26.305 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:45:26.306 DEBUG:boto:Data: 2022-09-03T06:45:26.307 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.307 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.308 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.308 DEBUG:boto:Params: {} 2022-09-03T06:45:26.309 DEBUG:boto:Token: None 2022-09-03T06:45:26.309 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj22 2022-09-03T06:45:26.310 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sc56cjqbxgbqPB6oUtmgayuouJc= 2022-09-03T06:45:26.311 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sc56cjqbxgbqPB6oUtmgayuouJc=', 'Content-Length': '0'} 2022-09-03T06:45:26.313 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006fc5ff6a20f0cd88-006312f806-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.314 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:26.315 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:26.316 DEBUG:boto:Method: HEAD 2022-09-03T06:45:26.317 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:45:26.318 DEBUG:boto:Data: 2022-09-03T06:45:26.319 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.320 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:26.320 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.321 DEBUG:boto:Params: {} 2022-09-03T06:45:26.323 DEBUG:boto:Token: None 2022-09-03T06:45:26.325 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj22 2022-09-03T06:45:26.326 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sc56cjqbxgbqPB6oUtmgayuouJc= 2022-09-03T06:45:26.327 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sc56cjqbxgbqPB6oUtmgayuouJc=', 'Content-Length': '0'} 2022-09-03T06:45:26.330 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000097bc2289a2110113-006312f806-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.331 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:45:26.333 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:26.334 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:26.335 DEBUG:boto:Method: GET 2022-09-03T06:45:26.337 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:45:26.338 DEBUG:boto:Data: 2022-09-03T06:45:26.339 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.340 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.341 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.342 DEBUG:boto:Params: {} 2022-09-03T06:45:26.343 DEBUG:boto:Token: None 2022-09-03T06:45:26.344 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj22 2022-09-03T06:45:26.345 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pJzU1UfET+uXzEkZjfizlVxHEiw= 2022-09-03T06:45:26.346 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pJzU1UfET+uXzEkZjfizlVxHEiw=', 'Content-Length': '0'} 2022-09-03T06:45:26.349 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000942277f9f55dffc1-006312f806-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.351 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:26.351 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:26.352 DEBUG:boto:Method: GET 2022-09-03T06:45:26.353 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:45:26.355 DEBUG:boto:Data: 2022-09-03T06:45:26.355 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.356 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:26.357 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.358 DEBUG:boto:Params: {} 2022-09-03T06:45:26.360 DEBUG:boto:Token: None 2022-09-03T06:45:26.361 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj22 2022-09-03T06:45:26.362 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pJzU1UfET+uXzEkZjfizlVxHEiw= 2022-09-03T06:45:26.363 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pJzU1UfET+uXzEkZjfizlVxHEiw=', 'Content-Length': '0'} 2022-09-03T06:45:26.366 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005d68c1ed1b85fd90-006312f806-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.368 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:26.368 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:26.369 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:26.370 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:26.371 DEBUG:boto:Method: GET 2022-09-03T06:45:26.371 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:45:26.372 DEBUG:boto:Data: 2022-09-03T06:45:26.373 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.373 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.374 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.375 DEBUG:boto:Params: {} 2022-09-03T06:45:26.376 DEBUG:boto:Token: None 2022-09-03T06:45:26.376 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/ 2022-09-03T06:45:26.378 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E= 2022-09-03T06:45:26.379 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E=', 'Content-Length': '0'} 2022-09-03T06:45:26.384 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001db5e36629f18f41-006312f806-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.386 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:26.387 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:26.388 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:26.389 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:26.390 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:26.391 DEBUG:boto:Method: GET 2022-09-03T06:45:26.392 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:45:26.393 DEBUG:boto:Data: 2022-09-03T06:45:26.394 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.395 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:26.396 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.397 DEBUG:boto:Params: {} 2022-09-03T06:45:26.399 DEBUG:boto:Token: None 2022-09-03T06:45:26.400 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/ 2022-09-03T06:45:26.401 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E= 2022-09-03T06:45:26.402 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E=', 'Content-Length': '0'} 2022-09-03T06:45:26.407 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a9a55b64a79393c0-006312f806-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.408 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:26.411 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:45:26.411 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:26.413 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:26.414 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.415 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.416 DEBUG:boto:Method: GET 2022-09-03T06:45:26.417 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.418 DEBUG:boto:Data: 2022-09-03T06:45:26.419 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.420 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.421 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.423 DEBUG:boto:Params: {} 2022-09-03T06:45:26.424 DEBUG:boto:Token: None 2022-09-03T06:45:26.425 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.427 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bbv9d7jsvThDegdl5DSRP1OYBuw= 2022-09-03T06:45:26.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bbv9d7jsvThDegdl5DSRP1OYBuw=', 'Content-Length': '0'} 2022-09-03T06:45:26.438 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ec475f32525c2098-006312f806-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.447 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:26.447 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:26.448 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.449 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.450 DEBUG:boto:Method: GET 2022-09-03T06:45:26.450 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.451 DEBUG:boto:Data: 2022-09-03T06:45:26.451 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.452 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:26.453 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.453 DEBUG:boto:Params: {} 2022-09-03T06:45:26.454 DEBUG:boto:Token: None 2022-09-03T06:45:26.454 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.455 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bbv9d7jsvThDegdl5DSRP1OYBuw= 2022-09-03T06:45:26.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bbv9d7jsvThDegdl5DSRP1OYBuw=', 'Content-Length': '0'} 2022-09-03T06:45:26.459 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a07684bc08314420-006312f806-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.460 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:26.460 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:26.461 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.462 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.462 DEBUG:boto:Method: HEAD 2022-09-03T06:45:26.463 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.463 DEBUG:boto:Data: 2022-09-03T06:45:26.464 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.464 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.465 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.466 DEBUG:boto:Params: {} 2022-09-03T06:45:26.467 DEBUG:boto:Token: None 2022-09-03T06:45:26.468 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.469 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TY3b07eoTbrYKrnlOB5y1LGzzzA= 2022-09-03T06:45:26.470 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TY3b07eoTbrYKrnlOB5y1LGzzzA=', 'Content-Length': '0'} 2022-09-03T06:45:26.473 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000111c75910263821d-006312f806-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.474 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:26.474 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:26.475 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.476 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.476 DEBUG:boto:Method: HEAD 2022-09-03T06:45:26.477 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.477 DEBUG:boto:Data: 2022-09-03T06:45:26.478 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.478 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:26.479 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.480 DEBUG:boto:Params: {} 2022-09-03T06:45:26.480 DEBUG:boto:Token: None 2022-09-03T06:45:26.481 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TY3b07eoTbrYKrnlOB5y1LGzzzA= 2022-09-03T06:45:26.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TY3b07eoTbrYKrnlOB5y1LGzzzA=', 'Content-Length': '0'} 2022-09-03T06:45:26.488 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008924e4b485b4d1e4-006312f806-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.489 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:45:26.490 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:26.491 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:26.491 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.492 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.492 DEBUG:boto:Method: GET 2022-09-03T06:45:26.493 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.493 DEBUG:boto:Data: 2022-09-03T06:45:26.494 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.495 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.495 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.496 DEBUG:boto:Params: {} 2022-09-03T06:45:26.497 DEBUG:boto:Token: None 2022-09-03T06:45:26.497 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.498 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bbv9d7jsvThDegdl5DSRP1OYBuw= 2022-09-03T06:45:26.499 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bbv9d7jsvThDegdl5DSRP1OYBuw=', 'Content-Length': '0'} 2022-09-03T06:45:26.502 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db82ad7fd29218a6-006312f806-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.550 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:26.551 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:26.551 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.552 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.552 DEBUG:boto:Method: GET 2022-09-03T06:45:26.553 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.554 DEBUG:boto:Data: 2022-09-03T06:45:26.555 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.555 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:26.556 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.557 DEBUG:boto:Params: {} 2022-09-03T06:45:26.558 DEBUG:boto:Token: None 2022-09-03T06:45:26.559 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:26.559 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bbv9d7jsvThDegdl5DSRP1OYBuw= 2022-09-03T06:45:26.560 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bbv9d7jsvThDegdl5DSRP1OYBuw=', 'Content-Length': '0'} 2022-09-03T06:45:26.562 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f1f565e98419bc51-006312f806-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.564 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:26.565 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:26.566 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:26.568 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:26.569 DEBUG:boto:Method: GET 2022-09-03T06:45:26.570 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:45:26.573 DEBUG:boto:Data: 2022-09-03T06:45:26.574 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.575 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.576 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.577 DEBUG:boto:Params: {} 2022-09-03T06:45:26.578 DEBUG:boto:Token: None 2022-09-03T06:45:26.579 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/ 2022-09-03T06:45:26.580 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E= 2022-09-03T06:45:26.581 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E=', 'Content-Length': '0'} 2022-09-03T06:45:26.585 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000515a623d44673852-006312f806-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.586 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:26.588 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:26.589 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:26.590 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:26.591 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:26.592 DEBUG:boto:Method: GET 2022-09-03T06:45:26.592 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:45:26.594 DEBUG:boto:Data: 2022-09-03T06:45:26.594 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.595 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:26.596 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.597 DEBUG:boto:Params: {} 2022-09-03T06:45:26.598 DEBUG:boto:Token: None 2022-09-03T06:45:26.600 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/ 2022-09-03T06:45:26.601 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E= 2022-09-03T06:45:26.602 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E=', 'Content-Length': '0'} 2022-09-03T06:45:26.606 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000038e7f8f0a7720655-006312f806-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.608 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:26.608 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:26.609 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:26.609 DEBUG:boto:Method: HEAD 2022-09-03T06:45:26.610 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:45:26.612 DEBUG:boto:Data: 2022-09-03T06:45:26.612 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.613 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.613 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.614 DEBUG:boto:Params: {} 2022-09-03T06:45:26.614 DEBUG:boto:Token: None 2022-09-03T06:45:26.615 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj23 2022-09-03T06:45:26.616 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TC8pemo2wEMOzo8I69KflguRNCE= 2022-09-03T06:45:26.616 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TC8pemo2wEMOzo8I69KflguRNCE=', 'Content-Length': '0'} 2022-09-03T06:45:26.619 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000072d1953beff4955c-006312f806-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.620 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:26.621 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:26.622 DEBUG:boto:Method: HEAD 2022-09-03T06:45:26.623 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:45:26.624 DEBUG:boto:Data: 2022-09-03T06:45:26.625 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.626 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:26.627 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.628 DEBUG:boto:Params: {} 2022-09-03T06:45:26.629 DEBUG:boto:Token: None 2022-09-03T06:45:26.630 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj23 2022-09-03T06:45:26.631 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TC8pemo2wEMOzo8I69KflguRNCE= 2022-09-03T06:45:26.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TC8pemo2wEMOzo8I69KflguRNCE=', 'Content-Length': '0'} 2022-09-03T06:45:26.635 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000098062e111b5e6b64-006312f806-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.637 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:45:26.638 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:26.639 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:26.640 DEBUG:boto:Method: GET 2022-09-03T06:45:26.642 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:45:26.643 DEBUG:boto:Data: 2022-09-03T06:45:26.644 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.645 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.645 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.646 DEBUG:boto:Params: {} 2022-09-03T06:45:26.647 DEBUG:boto:Token: None 2022-09-03T06:45:26.648 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj23 2022-09-03T06:45:26.649 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bT27Bj6dHmlWP9vfhDkFiRG3tzk= 2022-09-03T06:45:26.650 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bT27Bj6dHmlWP9vfhDkFiRG3tzk=', 'Content-Length': '0'} 2022-09-03T06:45:26.652 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006742006b1f4c9216-006312f806-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.653 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:26.654 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:26.654 DEBUG:boto:Method: GET 2022-09-03T06:45:26.655 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:45:26.656 DEBUG:boto:Data: 2022-09-03T06:45:26.656 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.657 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:26.657 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.658 DEBUG:boto:Params: {} 2022-09-03T06:45:26.659 DEBUG:boto:Token: None 2022-09-03T06:45:26.659 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj23 2022-09-03T06:45:26.660 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bT27Bj6dHmlWP9vfhDkFiRG3tzk= 2022-09-03T06:45:26.661 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bT27Bj6dHmlWP9vfhDkFiRG3tzk=', 'Content-Length': '0'} 2022-09-03T06:45:26.664 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ac1c9d4264c13a74-006312f806-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.665 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:26.665 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:26.666 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:26.667 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:26.668 DEBUG:boto:Method: GET 2022-09-03T06:45:26.668 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:45:26.669 DEBUG:boto:Data: 2022-09-03T06:45:26.670 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.670 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.671 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.672 DEBUG:boto:Params: {} 2022-09-03T06:45:26.672 DEBUG:boto:Token: None 2022-09-03T06:45:26.673 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/ 2022-09-03T06:45:26.674 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E= 2022-09-03T06:45:26.675 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E=', 'Content-Length': '0'} 2022-09-03T06:45:26.678 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006527df7ca4e4ff7d-006312f806-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.679 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:26.680 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:26.680 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:26.681 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:26.681 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:26.682 DEBUG:boto:Method: GET 2022-09-03T06:45:26.683 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:45:26.683 DEBUG:boto:Data: 2022-09-03T06:45:26.684 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.684 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:26.685 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.685 DEBUG:boto:Params: {} 2022-09-03T06:45:26.686 DEBUG:boto:Token: None 2022-09-03T06:45:26.686 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/ 2022-09-03T06:45:26.687 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E= 2022-09-03T06:45:26.688 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E=', 'Content-Length': '0'} 2022-09-03T06:45:26.693 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000069ff4fe5c9b13dce-006312f806-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.738 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:26.740 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:45:26.740 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:26.741 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:26.741 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.742 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.743 DEBUG:boto:Method: GET 2022-09-03T06:45:26.743 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.744 DEBUG:boto:Data: 2022-09-03T06:45:26.745 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.745 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.746 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.746 DEBUG:boto:Params: {} 2022-09-03T06:45:26.747 DEBUG:boto:Token: None 2022-09-03T06:45:26.748 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.748 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QCIy84fkAz9hEtasQZig65rBGuU= 2022-09-03T06:45:26.749 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QCIy84fkAz9hEtasQZig65rBGuU=', 'Content-Length': '0'} 2022-09-03T06:45:26.752 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000033bb0375a6d4d8d2-006312f806-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.753 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:26.754 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:26.755 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.755 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.756 DEBUG:boto:Method: GET 2022-09-03T06:45:26.757 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.758 DEBUG:boto:Data: 2022-09-03T06:45:26.759 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.759 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:26.760 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.761 DEBUG:boto:Params: {} 2022-09-03T06:45:26.762 DEBUG:boto:Token: None 2022-09-03T06:45:26.763 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.764 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QCIy84fkAz9hEtasQZig65rBGuU= 2022-09-03T06:45:26.765 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QCIy84fkAz9hEtasQZig65rBGuU=', 'Content-Length': '0'} 2022-09-03T06:45:26.767 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001870a45534a79bef-006312f806-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.768 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:26.769 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:26.769 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.770 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.771 DEBUG:boto:Method: HEAD 2022-09-03T06:45:26.772 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.772 DEBUG:boto:Data: 2022-09-03T06:45:26.773 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.773 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.774 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.774 DEBUG:boto:Params: {} 2022-09-03T06:45:26.775 DEBUG:boto:Token: None 2022-09-03T06:45:26.777 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.777 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mKtIOzSV8d6uQXHAaevjq3oFMcE= 2022-09-03T06:45:26.778 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mKtIOzSV8d6uQXHAaevjq3oFMcE=', 'Content-Length': '0'} 2022-09-03T06:45:26.781 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000007845db06834afe1-006312f806-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.783 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:26.784 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:26.785 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.785 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.786 DEBUG:boto:Method: HEAD 2022-09-03T06:45:26.788 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.789 DEBUG:boto:Data: 2022-09-03T06:45:26.790 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.791 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:26.792 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.793 DEBUG:boto:Params: {} 2022-09-03T06:45:26.796 DEBUG:boto:Token: None 2022-09-03T06:45:26.797 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.798 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mKtIOzSV8d6uQXHAaevjq3oFMcE= 2022-09-03T06:45:26.799 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mKtIOzSV8d6uQXHAaevjq3oFMcE=', 'Content-Length': '0'} 2022-09-03T06:45:26.802 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b4f1bd1201c05c9-006312f806-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.803 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:45:26.803 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:26.804 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:26.805 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.806 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.807 DEBUG:boto:Method: GET 2022-09-03T06:45:26.807 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.808 DEBUG:boto:Data: 2022-09-03T06:45:26.808 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.809 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.810 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.811 DEBUG:boto:Params: {} 2022-09-03T06:45:26.812 DEBUG:boto:Token: None 2022-09-03T06:45:26.812 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.813 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QCIy84fkAz9hEtasQZig65rBGuU= 2022-09-03T06:45:26.814 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QCIy84fkAz9hEtasQZig65rBGuU=', 'Content-Length': '0'} 2022-09-03T06:45:26.816 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d6957e5f06e4910e-006312f806-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.817 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:26.818 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:26.818 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.819 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.819 DEBUG:boto:Method: GET 2022-09-03T06:45:26.820 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.821 DEBUG:boto:Data: 2022-09-03T06:45:26.821 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.822 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:26.822 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.823 DEBUG:boto:Params: {} 2022-09-03T06:45:26.823 DEBUG:boto:Token: None 2022-09-03T06:45:26.824 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:26.825 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QCIy84fkAz9hEtasQZig65rBGuU= 2022-09-03T06:45:26.827 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QCIy84fkAz9hEtasQZig65rBGuU=', 'Content-Length': '0'} 2022-09-03T06:45:26.829 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006184de1cae127602-006312f806-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.831 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:26.832 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:26.833 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:26.833 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:26.834 DEBUG:boto:Method: GET 2022-09-03T06:45:26.835 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:45:26.836 DEBUG:boto:Data: 2022-09-03T06:45:26.837 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.838 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.839 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.840 DEBUG:boto:Params: {} 2022-09-03T06:45:26.841 DEBUG:boto:Token: None 2022-09-03T06:45:26.842 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/ 2022-09-03T06:45:26.844 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E= 2022-09-03T06:45:26.845 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E=', 'Content-Length': '0'} 2022-09-03T06:45:26.849 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006f0fc662ad157a22-006312f806-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.851 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:26.852 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:26.853 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:26.854 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:26.855 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:26.856 DEBUG:boto:Method: GET 2022-09-03T06:45:26.858 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:45:26.859 DEBUG:boto:Data: 2022-09-03T06:45:26.860 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.860 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:26.861 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.861 DEBUG:boto:Params: {} 2022-09-03T06:45:26.864 DEBUG:boto:Token: None 2022-09-03T06:45:26.865 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/ 2022-09-03T06:45:26.866 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E= 2022-09-03T06:45:26.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E=', 'Content-Length': '0'} 2022-09-03T06:45:26.872 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000049ea4060bca94fce-006312f806-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.873 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:26.874 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:26.875 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:26.876 DEBUG:boto:Method: HEAD 2022-09-03T06:45:26.876 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:45:26.877 DEBUG:boto:Data: 2022-09-03T06:45:26.878 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.879 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.880 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.881 DEBUG:boto:Params: {} 2022-09-03T06:45:26.882 DEBUG:boto:Token: None 2022-09-03T06:45:26.884 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj29 2022-09-03T06:45:26.885 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yoj6nmZLYjMN8qp9znHUKslyqew= 2022-09-03T06:45:26.886 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yoj6nmZLYjMN8qp9znHUKslyqew=', 'Content-Length': '0'} 2022-09-03T06:45:26.899 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cdd8bdc4e9c60d94-006312f806-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.901 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:26.901 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:26.902 DEBUG:boto:Method: HEAD 2022-09-03T06:45:26.903 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:45:26.903 DEBUG:boto:Data: 2022-09-03T06:45:26.904 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.905 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:26.906 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.907 DEBUG:boto:Params: {} 2022-09-03T06:45:26.908 DEBUG:boto:Token: None 2022-09-03T06:45:26.909 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj29 2022-09-03T06:45:26.909 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yoj6nmZLYjMN8qp9znHUKslyqew= 2022-09-03T06:45:26.910 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yoj6nmZLYjMN8qp9znHUKslyqew=', 'Content-Length': '0'} 2022-09-03T06:45:26.913 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b9653bca64eac0d-006312f806-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.915 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:45:26.916 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:26.917 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:26.918 DEBUG:boto:Method: GET 2022-09-03T06:45:26.919 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:45:26.921 DEBUG:boto:Data: 2022-09-03T06:45:26.922 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.923 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.923 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.924 DEBUG:boto:Params: {} 2022-09-03T06:45:26.925 DEBUG:boto:Token: None 2022-09-03T06:45:26.926 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj29 2022-09-03T06:45:26.927 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZCpZRcZeYx0iicyQ4aZek1K7UDc= 2022-09-03T06:45:26.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZCpZRcZeYx0iicyQ4aZek1K7UDc=', 'Content-Length': '0'} 2022-09-03T06:45:26.930 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e82228ae94b4ce2f-006312f806-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.974 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:26.975 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:26.976 DEBUG:boto:Method: GET 2022-09-03T06:45:26.976 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:45:26.977 DEBUG:boto:Data: 2022-09-03T06:45:26.977 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.978 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:26.979 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.979 DEBUG:boto:Params: {} 2022-09-03T06:45:26.980 DEBUG:boto:Token: None 2022-09-03T06:45:26.981 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/obj29 2022-09-03T06:45:26.982 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZCpZRcZeYx0iicyQ4aZek1K7UDc= 2022-09-03T06:45:26.982 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZCpZRcZeYx0iicyQ4aZek1K7UDc=', 'Content-Length': '0'} 2022-09-03T06:45:26.985 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fde0854f47220e07-006312f806-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:26.986 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:26.987 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:26.987 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:26.988 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:26.988 DEBUG:boto:Method: GET 2022-09-03T06:45:26.990 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:45:26.991 DEBUG:boto:Data: 2022-09-03T06:45:26.994 DEBUG:boto:Headers: {} 2022-09-03T06:45:26.995 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:26.995 DEBUG:boto:Port: 8000 2022-09-03T06:45:26.996 DEBUG:boto:Params: {} 2022-09-03T06:45:26.997 DEBUG:boto:Token: None 2022-09-03T06:45:26.998 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:26 GMT /dsgvcy-75/ 2022-09-03T06:45:26.999 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E= 2022-09-03T06:45:27.000 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:giAPaiy6XT1K/AnXvPbjzmHgX5E=', 'Content-Length': '0'} 2022-09-03T06:45:27.004 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b6c8cee497ed843a-006312f807-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.006 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:27.009 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:27.010 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:27.011 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:27.011 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:27.012 DEBUG:boto:Method: GET 2022-09-03T06:45:27.013 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:45:27.013 DEBUG:boto:Data: 2022-09-03T06:45:27.015 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.015 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.016 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.017 DEBUG:boto:Params: {} 2022-09-03T06:45:27.017 DEBUG:boto:Token: None 2022-09-03T06:45:27.018 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/ 2022-09-03T06:45:27.019 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio= 2022-09-03T06:45:27.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio=', 'Content-Length': '0'} 2022-09-03T06:45:27.024 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005214a327c975fc5a-006312f807-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.025 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:27.026 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:45:27.026 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:27.027 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:27.027 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.028 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.028 DEBUG:boto:Method: GET 2022-09-03T06:45:27.029 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.029 DEBUG:boto:Data: 2022-09-03T06:45:27.030 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.031 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.032 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.033 DEBUG:boto:Params: {} 2022-09-03T06:45:27.033 DEBUG:boto:Token: None 2022-09-03T06:45:27.034 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.034 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2xm/DmbZ5Tk6JrElSa3jNkXWS+g= 2022-09-03T06:45:27.035 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2xm/DmbZ5Tk6JrElSa3jNkXWS+g=', 'Content-Length': '0'} 2022-09-03T06:45:27.040 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009a2fb9cc7de1c44f-006312f807-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.041 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:27.042 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:27.043 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.043 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.044 DEBUG:boto:Method: GET 2022-09-03T06:45:27.044 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.045 DEBUG:boto:Data: 2022-09-03T06:45:27.045 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.046 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.046 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.047 DEBUG:boto:Params: {} 2022-09-03T06:45:27.048 DEBUG:boto:Token: None 2022-09-03T06:45:27.048 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.049 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2xm/DmbZ5Tk6JrElSa3jNkXWS+g= 2022-09-03T06:45:27.049 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2xm/DmbZ5Tk6JrElSa3jNkXWS+g=', 'Content-Length': '0'} 2022-09-03T06:45:27.055 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a9dee59d29f071d0-006312f807-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.057 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:27.058 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:27.059 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.060 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.061 DEBUG:boto:Method: HEAD 2022-09-03T06:45:27.062 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.063 DEBUG:boto:Data: 2022-09-03T06:45:27.064 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.065 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.066 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.067 DEBUG:boto:Params: {} 2022-09-03T06:45:27.068 DEBUG:boto:Token: None 2022-09-03T06:45:27.069 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.070 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H9xyEv8Edvq6eSPtuy/SkYHJ2pY= 2022-09-03T06:45:27.071 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H9xyEv8Edvq6eSPtuy/SkYHJ2pY=', 'Content-Length': '0'} 2022-09-03T06:45:27.073 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a47893413e3789ae-006312f807-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.075 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:27.076 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:27.077 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.078 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.079 DEBUG:boto:Method: HEAD 2022-09-03T06:45:27.080 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.081 DEBUG:boto:Data: 2022-09-03T06:45:27.081 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.082 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.083 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.085 DEBUG:boto:Params: {} 2022-09-03T06:45:27.086 DEBUG:boto:Token: None 2022-09-03T06:45:27.087 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.088 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H9xyEv8Edvq6eSPtuy/SkYHJ2pY= 2022-09-03T06:45:27.090 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H9xyEv8Edvq6eSPtuy/SkYHJ2pY=', 'Content-Length': '0'} 2022-09-03T06:45:27.093 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000801dc21142eb1ade-006312f807-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.095 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:45:27.096 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:27.097 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:27.098 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.099 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.100 DEBUG:boto:Method: GET 2022-09-03T06:45:27.101 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.102 DEBUG:boto:Data: 2022-09-03T06:45:27.103 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.105 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.106 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.107 DEBUG:boto:Params: {} 2022-09-03T06:45:27.108 DEBUG:boto:Token: None 2022-09-03T06:45:27.109 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.111 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2xm/DmbZ5Tk6JrElSa3jNkXWS+g= 2022-09-03T06:45:27.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2xm/DmbZ5Tk6JrElSa3jNkXWS+g=', 'Content-Length': '0'} 2022-09-03T06:45:27.116 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000031a8b6102fef86db-006312f807-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.118 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:27.119 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:27.120 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.122 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.124 DEBUG:boto:Method: GET 2022-09-03T06:45:27.125 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.126 DEBUG:boto:Data: 2022-09-03T06:45:27.128 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.129 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.130 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.132 DEBUG:boto:Params: {} 2022-09-03T06:45:27.134 DEBUG:boto:Token: None 2022-09-03T06:45:27.136 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:27.137 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2xm/DmbZ5Tk6JrElSa3jNkXWS+g= 2022-09-03T06:45:27.138 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2xm/DmbZ5Tk6JrElSa3jNkXWS+g=', 'Content-Length': '0'} 2022-09-03T06:45:27.142 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001e8b0d3f7bf9b3d5-006312f807-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.143 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:27.145 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:27.146 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:27.147 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:27.148 DEBUG:boto:Method: GET 2022-09-03T06:45:27.149 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:45:27.150 DEBUG:boto:Data: 2022-09-03T06:45:27.151 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.151 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.152 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.152 DEBUG:boto:Params: {} 2022-09-03T06:45:27.153 DEBUG:boto:Token: None 2022-09-03T06:45:27.154 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/ 2022-09-03T06:45:27.154 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio= 2022-09-03T06:45:27.155 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio=', 'Content-Length': '0'} 2022-09-03T06:45:27.161 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000053b7d4a0c4f33602-006312f807-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.163 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:27.164 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:27.164 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:27.165 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:27.165 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:27.166 DEBUG:boto:Method: GET 2022-09-03T06:45:27.167 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:45:27.168 DEBUG:boto:Data: 2022-09-03T06:45:27.168 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.169 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.170 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.175 DEBUG:boto:Params: {} 2022-09-03T06:45:27.177 DEBUG:boto:Token: None 2022-09-03T06:45:27.178 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/ 2022-09-03T06:45:27.179 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio= 2022-09-03T06:45:27.181 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio=', 'Content-Length': '0'} 2022-09-03T06:45:27.193 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000350f897049d305be-006312f807-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.195 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:27.197 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:27.197 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:27.199 DEBUG:boto:Method: HEAD 2022-09-03T06:45:27.200 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:45:27.200 DEBUG:boto:Data: 2022-09-03T06:45:27.201 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.203 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.204 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.205 DEBUG:boto:Params: {} 2022-09-03T06:45:27.206 DEBUG:boto:Token: None 2022-09-03T06:45:27.207 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj33 2022-09-03T06:45:27.209 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JlocBMLhPDiy1XTUNbVizAWxeF0= 2022-09-03T06:45:27.210 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JlocBMLhPDiy1XTUNbVizAWxeF0=', 'Content-Length': '0'} 2022-09-03T06:45:27.213 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007d5bc3beb1b31b0a-006312f807-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.215 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:27.217 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:27.218 DEBUG:boto:Method: HEAD 2022-09-03T06:45:27.219 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:45:27.220 DEBUG:boto:Data: 2022-09-03T06:45:27.221 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.222 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.222 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.223 DEBUG:boto:Params: {} 2022-09-03T06:45:27.224 DEBUG:boto:Token: None 2022-09-03T06:45:27.225 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj33 2022-09-03T06:45:27.226 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JlocBMLhPDiy1XTUNbVizAWxeF0= 2022-09-03T06:45:27.227 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JlocBMLhPDiy1XTUNbVizAWxeF0=', 'Content-Length': '0'} 2022-09-03T06:45:27.233 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fa14e83b7ce5665a-006312f807-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.235 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:45:27.237 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:27.238 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:27.241 DEBUG:boto:Method: GET 2022-09-03T06:45:27.241 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:45:27.242 DEBUG:boto:Data: 2022-09-03T06:45:27.242 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.243 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.244 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.244 DEBUG:boto:Params: {} 2022-09-03T06:45:27.245 DEBUG:boto:Token: None 2022-09-03T06:45:27.246 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj33 2022-09-03T06:45:27.248 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B1jNUmx7qtfui6i7YMlW/7YHe70= 2022-09-03T06:45:27.249 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B1jNUmx7qtfui6i7YMlW/7YHe70=', 'Content-Length': '0'} 2022-09-03T06:45:27.251 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001675280ff15e1d4f-006312f807-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.252 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:27.253 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:27.253 DEBUG:boto:Method: GET 2022-09-03T06:45:27.254 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:45:27.255 DEBUG:boto:Data: 2022-09-03T06:45:27.255 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.256 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.257 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.257 DEBUG:boto:Params: {} 2022-09-03T06:45:27.258 DEBUG:boto:Token: None 2022-09-03T06:45:27.259 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj33 2022-09-03T06:45:27.259 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B1jNUmx7qtfui6i7YMlW/7YHe70= 2022-09-03T06:45:27.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B1jNUmx7qtfui6i7YMlW/7YHe70=', 'Content-Length': '0'} 2022-09-03T06:45:27.266 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000001ad45b726c30f44-006312f807-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.306 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:27.307 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:27.308 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:27.308 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:27.309 DEBUG:boto:Method: GET 2022-09-03T06:45:27.310 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:45:27.310 DEBUG:boto:Data: 2022-09-03T06:45:27.311 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.312 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.313 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.313 DEBUG:boto:Params: {} 2022-09-03T06:45:27.314 DEBUG:boto:Token: None 2022-09-03T06:45:27.315 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/ 2022-09-03T06:45:27.315 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio= 2022-09-03T06:45:27.316 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio=', 'Content-Length': '0'} 2022-09-03T06:45:27.319 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000020efcd753b74e7e6-006312f807-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.320 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:27.321 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:27.322 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:27.323 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:27.323 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:27.324 DEBUG:boto:Method: GET 2022-09-03T06:45:27.334 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:45:27.335 DEBUG:boto:Data: 2022-09-03T06:45:27.336 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.338 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.339 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.340 DEBUG:boto:Params: {} 2022-09-03T06:45:27.341 DEBUG:boto:Token: None 2022-09-03T06:45:27.342 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/ 2022-09-03T06:45:27.344 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio= 2022-09-03T06:45:27.346 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio=', 'Content-Length': '0'} 2022-09-03T06:45:27.350 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bc3e9a06f2ece94d-006312f807-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.351 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:27.352 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:45:27.353 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:27.353 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:27.354 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.355 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.356 DEBUG:boto:Method: GET 2022-09-03T06:45:27.358 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.359 DEBUG:boto:Data: 2022-09-03T06:45:27.360 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.361 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.361 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.362 DEBUG:boto:Params: {} 2022-09-03T06:45:27.363 DEBUG:boto:Token: None 2022-09-03T06:45:27.363 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.364 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WgiTfbF4yln/uHyI8rWO1ppFe+E= 2022-09-03T06:45:27.365 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WgiTfbF4yln/uHyI8rWO1ppFe+E=', 'Content-Length': '0'} 2022-09-03T06:45:27.367 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e5f4b98d33b487c4-006312f807-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.368 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:27.369 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:27.369 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.370 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.370 DEBUG:boto:Method: GET 2022-09-03T06:45:27.371 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.372 DEBUG:boto:Data: 2022-09-03T06:45:27.372 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.373 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.373 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.374 DEBUG:boto:Params: {} 2022-09-03T06:45:27.375 DEBUG:boto:Token: None 2022-09-03T06:45:27.376 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.376 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WgiTfbF4yln/uHyI8rWO1ppFe+E= 2022-09-03T06:45:27.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WgiTfbF4yln/uHyI8rWO1ppFe+E=', 'Content-Length': '0'} 2022-09-03T06:45:27.383 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000092e26d80537ecada-006312f807-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.388 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:27.388 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:27.389 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.389 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.390 DEBUG:boto:Method: HEAD 2022-09-03T06:45:27.391 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.392 DEBUG:boto:Data: 2022-09-03T06:45:27.393 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.394 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.394 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.395 DEBUG:boto:Params: {} 2022-09-03T06:45:27.396 DEBUG:boto:Token: None 2022-09-03T06:45:27.397 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.398 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WudwKzAIK1XxO+4q5QvFjEs+oQ0= 2022-09-03T06:45:27.399 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WudwKzAIK1XxO+4q5QvFjEs+oQ0=', 'Content-Length': '0'} 2022-09-03T06:45:27.402 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e8f62c0acfb5dba0-006312f807-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.403 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:27.404 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:27.405 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.405 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.406 DEBUG:boto:Method: HEAD 2022-09-03T06:45:27.407 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.407 DEBUG:boto:Data: 2022-09-03T06:45:27.407 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.408 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.408 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.409 DEBUG:boto:Params: {} 2022-09-03T06:45:27.410 DEBUG:boto:Token: None 2022-09-03T06:45:27.410 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.411 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WudwKzAIK1XxO+4q5QvFjEs+oQ0= 2022-09-03T06:45:27.411 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WudwKzAIK1XxO+4q5QvFjEs+oQ0=', 'Content-Length': '0'} 2022-09-03T06:45:27.414 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d5698e893476fba4-006312f807-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.415 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:45:27.416 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:27.417 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:27.417 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.418 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.419 DEBUG:boto:Method: GET 2022-09-03T06:45:27.419 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.420 DEBUG:boto:Data: 2022-09-03T06:45:27.420 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.421 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.422 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.422 DEBUG:boto:Params: {} 2022-09-03T06:45:27.423 DEBUG:boto:Token: None 2022-09-03T06:45:27.424 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.424 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WgiTfbF4yln/uHyI8rWO1ppFe+E= 2022-09-03T06:45:27.425 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WgiTfbF4yln/uHyI8rWO1ppFe+E=', 'Content-Length': '0'} 2022-09-03T06:45:27.428 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005cc5440642d66322-006312f807-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.429 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:27.430 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:27.431 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.431 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.432 DEBUG:boto:Method: GET 2022-09-03T06:45:27.433 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.434 DEBUG:boto:Data: 2022-09-03T06:45:27.435 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.436 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.437 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.438 DEBUG:boto:Params: {} 2022-09-03T06:45:27.439 DEBUG:boto:Token: None 2022-09-03T06:45:27.440 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:27.441 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WgiTfbF4yln/uHyI8rWO1ppFe+E= 2022-09-03T06:45:27.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WgiTfbF4yln/uHyI8rWO1ppFe+E=', 'Content-Length': '0'} 2022-09-03T06:45:27.445 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000041f2c1f5c96dfbb4-006312f807-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.486 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:27.487 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:27.492 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:27.493 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:27.494 DEBUG:boto:Method: GET 2022-09-03T06:45:27.495 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:45:27.496 DEBUG:boto:Data: 2022-09-03T06:45:27.497 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.498 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.499 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.500 DEBUG:boto:Params: {} 2022-09-03T06:45:27.501 DEBUG:boto:Token: None 2022-09-03T06:45:27.502 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/ 2022-09-03T06:45:27.503 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio= 2022-09-03T06:45:27.504 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio=', 'Content-Length': '0'} 2022-09-03T06:45:27.508 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009f65d75d30a78892-006312f807-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.509 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:27.511 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:27.511 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:27.512 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:27.513 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:27.514 DEBUG:boto:Method: GET 2022-09-03T06:45:27.515 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:45:27.516 DEBUG:boto:Data: 2022-09-03T06:45:27.517 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.517 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.519 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.519 DEBUG:boto:Params: {} 2022-09-03T06:45:27.520 DEBUG:boto:Token: None 2022-09-03T06:45:27.520 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/ 2022-09-03T06:45:27.521 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio= 2022-09-03T06:45:27.522 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio=', 'Content-Length': '0'} 2022-09-03T06:45:27.526 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000016dc4374d230b7eb-006312f807-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.527 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:27.528 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:27.528 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:27.529 DEBUG:boto:Method: HEAD 2022-09-03T06:45:27.529 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:45:27.530 DEBUG:boto:Data: 2022-09-03T06:45:27.531 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.532 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.533 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.534 DEBUG:boto:Params: {} 2022-09-03T06:45:27.535 DEBUG:boto:Token: None 2022-09-03T06:45:27.536 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj34 2022-09-03T06:45:27.537 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PbRP54s4x0H9XiVuSbLXYGTrWdo= 2022-09-03T06:45:27.538 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PbRP54s4x0H9XiVuSbLXYGTrWdo=', 'Content-Length': '0'} 2022-09-03T06:45:27.540 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dcc5a0df7276520c-006312f807-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.541 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:27.542 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:27.543 DEBUG:boto:Method: HEAD 2022-09-03T06:45:27.543 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:45:27.544 DEBUG:boto:Data: 2022-09-03T06:45:27.544 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.545 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.545 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.546 DEBUG:boto:Params: {} 2022-09-03T06:45:27.547 DEBUG:boto:Token: None 2022-09-03T06:45:27.547 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj34 2022-09-03T06:45:27.548 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PbRP54s4x0H9XiVuSbLXYGTrWdo= 2022-09-03T06:45:27.548 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PbRP54s4x0H9XiVuSbLXYGTrWdo=', 'Content-Length': '0'} 2022-09-03T06:45:27.550 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000088c0e82abe933399-006312f807-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.551 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:45:27.552 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:27.553 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:27.554 DEBUG:boto:Method: GET 2022-09-03T06:45:27.555 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:45:27.555 DEBUG:boto:Data: 2022-09-03T06:45:27.556 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.557 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.558 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.559 DEBUG:boto:Params: {} 2022-09-03T06:45:27.559 DEBUG:boto:Token: None 2022-09-03T06:45:27.560 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj34 2022-09-03T06:45:27.561 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GkJQd1IUfr7O9a/sBy820tfFtos= 2022-09-03T06:45:27.561 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GkJQd1IUfr7O9a/sBy820tfFtos=', 'Content-Length': '0'} 2022-09-03T06:45:27.564 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004441f28dfa3013a8-006312f807-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.565 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:27.565 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:27.566 DEBUG:boto:Method: GET 2022-09-03T06:45:27.566 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:45:27.567 DEBUG:boto:Data: 2022-09-03T06:45:27.567 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.568 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.569 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.569 DEBUG:boto:Params: {} 2022-09-03T06:45:27.570 DEBUG:boto:Token: None 2022-09-03T06:45:27.570 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj34 2022-09-03T06:45:27.571 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GkJQd1IUfr7O9a/sBy820tfFtos= 2022-09-03T06:45:27.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GkJQd1IUfr7O9a/sBy820tfFtos=', 'Content-Length': '0'} 2022-09-03T06:45:27.578 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d74fb68e077c737-006312f807-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.618 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:27.619 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:27.620 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:27.620 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:27.621 DEBUG:boto:Method: GET 2022-09-03T06:45:27.621 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:45:27.622 DEBUG:boto:Data: 2022-09-03T06:45:27.622 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.624 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.625 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.626 DEBUG:boto:Params: {} 2022-09-03T06:45:27.627 DEBUG:boto:Token: None 2022-09-03T06:45:27.628 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/ 2022-09-03T06:45:27.629 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio= 2022-09-03T06:45:27.630 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio=', 'Content-Length': '0'} 2022-09-03T06:45:27.633 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009c7c5c0edf10b564-006312f807-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.634 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:27.635 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:27.636 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:27.637 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:27.637 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:27.638 DEBUG:boto:Method: GET 2022-09-03T06:45:27.638 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:45:27.639 DEBUG:boto:Data: 2022-09-03T06:45:27.639 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.640 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.641 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.641 DEBUG:boto:Params: {} 2022-09-03T06:45:27.642 DEBUG:boto:Token: None 2022-09-03T06:45:27.642 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/ 2022-09-03T06:45:27.644 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio= 2022-09-03T06:45:27.645 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio=', 'Content-Length': '0'} 2022-09-03T06:45:27.650 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002a259b9eed82077d-006312f807-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.651 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:27.653 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:45:27.654 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:27.655 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:27.656 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.657 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.657 DEBUG:boto:Method: GET 2022-09-03T06:45:27.658 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.659 DEBUG:boto:Data: 2022-09-03T06:45:27.660 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.661 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.662 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.662 DEBUG:boto:Params: {} 2022-09-03T06:45:27.663 DEBUG:boto:Token: None 2022-09-03T06:45:27.664 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.664 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fIjdU9teSM5nEca3phQLBvpHXi0= 2022-09-03T06:45:27.665 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fIjdU9teSM5nEca3phQLBvpHXi0=', 'Content-Length': '0'} 2022-09-03T06:45:27.667 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009a61600a744d8c6e-006312f807-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.668 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:27.669 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:27.669 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.670 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.670 DEBUG:boto:Method: GET 2022-09-03T06:45:27.671 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.672 DEBUG:boto:Data: 2022-09-03T06:45:27.673 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.673 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.674 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.674 DEBUG:boto:Params: {} 2022-09-03T06:45:27.675 DEBUG:boto:Token: None 2022-09-03T06:45:27.676 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.677 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fIjdU9teSM5nEca3phQLBvpHXi0= 2022-09-03T06:45:27.677 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fIjdU9teSM5nEca3phQLBvpHXi0=', 'Content-Length': '0'} 2022-09-03T06:45:27.680 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000293a4fe2c1bcac8e-006312f807-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.681 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:27.681 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:27.682 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.682 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.683 DEBUG:boto:Method: HEAD 2022-09-03T06:45:27.684 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.684 DEBUG:boto:Data: 2022-09-03T06:45:27.685 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.685 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.686 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.686 DEBUG:boto:Params: {} 2022-09-03T06:45:27.687 DEBUG:boto:Token: None 2022-09-03T06:45:27.688 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.689 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aN+d3plU6Tcibf9kQGD6n7SWYro= 2022-09-03T06:45:27.689 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aN+d3plU6Tcibf9kQGD6n7SWYro=', 'Content-Length': '0'} 2022-09-03T06:45:27.692 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b66eae6b5f59089-006312f807-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.694 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:27.695 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:27.696 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.697 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.698 DEBUG:boto:Method: HEAD 2022-09-03T06:45:27.699 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.700 DEBUG:boto:Data: 2022-09-03T06:45:27.700 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.701 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.701 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.702 DEBUG:boto:Params: {} 2022-09-03T06:45:27.703 DEBUG:boto:Token: None 2022-09-03T06:45:27.704 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.705 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aN+d3plU6Tcibf9kQGD6n7SWYro= 2022-09-03T06:45:27.705 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aN+d3plU6Tcibf9kQGD6n7SWYro=', 'Content-Length': '0'} 2022-09-03T06:45:27.707 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000091b097ddfac4e127-006312f807-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.708 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:45:27.709 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:27.709 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:27.710 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.710 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.711 DEBUG:boto:Method: GET 2022-09-03T06:45:27.712 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.713 DEBUG:boto:Data: 2022-09-03T06:45:27.714 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.715 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.716 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.717 DEBUG:boto:Params: {} 2022-09-03T06:45:27.718 DEBUG:boto:Token: None 2022-09-03T06:45:27.719 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.720 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fIjdU9teSM5nEca3phQLBvpHXi0= 2022-09-03T06:45:27.721 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fIjdU9teSM5nEca3phQLBvpHXi0=', 'Content-Length': '0'} 2022-09-03T06:45:27.724 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ff23a711fed0590b-006312f807-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.726 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:27.726 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:27.727 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.728 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.729 DEBUG:boto:Method: GET 2022-09-03T06:45:27.729 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.730 DEBUG:boto:Data: 2022-09-03T06:45:27.730 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.731 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.731 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.732 DEBUG:boto:Params: {} 2022-09-03T06:45:27.733 DEBUG:boto:Token: None 2022-09-03T06:45:27.733 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:27.734 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fIjdU9teSM5nEca3phQLBvpHXi0= 2022-09-03T06:45:27.735 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fIjdU9teSM5nEca3phQLBvpHXi0=', 'Content-Length': '0'} 2022-09-03T06:45:27.737 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000008b6cb1f8ec82031-006312f807-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.738 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:27.738 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:27.739 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:27.739 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:27.740 DEBUG:boto:Method: GET 2022-09-03T06:45:27.741 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:45:27.741 DEBUG:boto:Data: 2022-09-03T06:45:27.742 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.742 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.743 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.743 DEBUG:boto:Params: {} 2022-09-03T06:45:27.744 DEBUG:boto:Token: None 2022-09-03T06:45:27.745 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/ 2022-09-03T06:45:27.745 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio= 2022-09-03T06:45:27.746 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio=', 'Content-Length': '0'} 2022-09-03T06:45:27.749 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004f848f6dd1b817f5-006312f807-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.750 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:27.751 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:27.751 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:27.752 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:27.753 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:27.754 DEBUG:boto:Method: GET 2022-09-03T06:45:27.755 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:45:27.755 DEBUG:boto:Data: 2022-09-03T06:45:27.756 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.756 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.757 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.757 DEBUG:boto:Params: {} 2022-09-03T06:45:27.758 DEBUG:boto:Token: None 2022-09-03T06:45:27.759 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/ 2022-09-03T06:45:27.760 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio= 2022-09-03T06:45:27.761 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio=', 'Content-Length': '0'} 2022-09-03T06:45:27.766 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f56d504361af8be4-006312f807-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.767 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:27.768 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:27.769 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:27.770 DEBUG:boto:Method: HEAD 2022-09-03T06:45:27.771 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:45:27.772 DEBUG:boto:Data: 2022-09-03T06:45:27.773 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.773 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.774 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.775 DEBUG:boto:Params: {} 2022-09-03T06:45:27.775 DEBUG:boto:Token: None 2022-09-03T06:45:27.776 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj38 2022-09-03T06:45:27.776 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ca81PvJp+G2WMy/rZkDqewsPCck= 2022-09-03T06:45:27.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ca81PvJp+G2WMy/rZkDqewsPCck=', 'Content-Length': '0'} 2022-09-03T06:45:27.779 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000036b8940ec51f34c3-006312f807-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.781 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:27.782 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:27.783 DEBUG:boto:Method: HEAD 2022-09-03T06:45:27.784 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:45:27.784 DEBUG:boto:Data: 2022-09-03T06:45:27.785 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.786 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.787 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.787 DEBUG:boto:Params: {} 2022-09-03T06:45:27.788 DEBUG:boto:Token: None 2022-09-03T06:45:27.789 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj38 2022-09-03T06:45:27.790 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ca81PvJp+G2WMy/rZkDqewsPCck= 2022-09-03T06:45:27.790 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ca81PvJp+G2WMy/rZkDqewsPCck=', 'Content-Length': '0'} 2022-09-03T06:45:27.792 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cb99d61513080266-006312f807-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.796 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:45:27.797 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:27.797 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:27.798 DEBUG:boto:Method: GET 2022-09-03T06:45:27.798 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:45:27.799 DEBUG:boto:Data: 2022-09-03T06:45:27.800 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.800 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.801 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.801 DEBUG:boto:Params: {} 2022-09-03T06:45:27.802 DEBUG:boto:Token: None 2022-09-03T06:45:27.803 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj38 2022-09-03T06:45:27.803 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CzznWrNz69Oy0gtdkNqkktzMIjc= 2022-09-03T06:45:27.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CzznWrNz69Oy0gtdkNqkktzMIjc=', 'Content-Length': '0'} 2022-09-03T06:45:27.806 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eac5aa32833ecd0c-006312f807-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.807 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:27.808 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:27.808 DEBUG:boto:Method: GET 2022-09-03T06:45:27.809 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:45:27.809 DEBUG:boto:Data: 2022-09-03T06:45:27.810 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.810 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.811 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.811 DEBUG:boto:Params: {} 2022-09-03T06:45:27.812 DEBUG:boto:Token: None 2022-09-03T06:45:27.813 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj38 2022-09-03T06:45:27.813 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CzznWrNz69Oy0gtdkNqkktzMIjc= 2022-09-03T06:45:27.815 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CzznWrNz69Oy0gtdkNqkktzMIjc=', 'Content-Length': '0'} 2022-09-03T06:45:27.817 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f558d856e266dbf2-006312f807-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.818 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:27.819 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:27.820 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:27.820 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:27.821 DEBUG:boto:Method: GET 2022-09-03T06:45:27.821 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:45:27.822 DEBUG:boto:Data: 2022-09-03T06:45:27.823 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.823 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.824 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.825 DEBUG:boto:Params: {} 2022-09-03T06:45:27.826 DEBUG:boto:Token: None 2022-09-03T06:45:27.826 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/ 2022-09-03T06:45:27.827 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio= 2022-09-03T06:45:27.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio=', 'Content-Length': '0'} 2022-09-03T06:45:27.831 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000000d22d3bc9e8d56-006312f807-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.874 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:27.875 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:27.876 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:27.876 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:27.877 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:27.877 DEBUG:boto:Method: GET 2022-09-03T06:45:27.878 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:45:27.879 DEBUG:boto:Data: 2022-09-03T06:45:27.879 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.880 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.881 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.881 DEBUG:boto:Params: {} 2022-09-03T06:45:27.882 DEBUG:boto:Token: None 2022-09-03T06:45:27.883 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/ 2022-09-03T06:45:27.883 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio= 2022-09-03T06:45:27.884 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N3YU6dXqqw7/bHtBiLSyRgqFiio=', 'Content-Length': '0'} 2022-09-03T06:45:27.888 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a595e4cd916c02f0-006312f807-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.889 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:27.890 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:45:27.890 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:27.891 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:27.891 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.892 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.893 DEBUG:boto:Method: GET 2022-09-03T06:45:27.893 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.894 DEBUG:boto:Data: 2022-09-03T06:45:27.894 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.895 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.896 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.897 DEBUG:boto:Params: {} 2022-09-03T06:45:27.898 DEBUG:boto:Token: None 2022-09-03T06:45:27.898 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.899 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XWwTrVldPQ7iIMIOuL72nD9iTfQ= 2022-09-03T06:45:27.900 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XWwTrVldPQ7iIMIOuL72nD9iTfQ=', 'Content-Length': '0'} 2022-09-03T06:45:27.902 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002b3d0460b8e1550a-006312f807-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.903 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:27.904 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:27.904 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.905 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.905 DEBUG:boto:Method: GET 2022-09-03T06:45:27.906 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.906 DEBUG:boto:Data: 2022-09-03T06:45:27.907 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.907 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.908 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.908 DEBUG:boto:Params: {} 2022-09-03T06:45:27.909 DEBUG:boto:Token: None 2022-09-03T06:45:27.910 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.910 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XWwTrVldPQ7iIMIOuL72nD9iTfQ= 2022-09-03T06:45:27.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XWwTrVldPQ7iIMIOuL72nD9iTfQ=', 'Content-Length': '0'} 2022-09-03T06:45:27.913 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008945ec85f6414915-006312f807-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.914 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:27.915 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:27.915 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.916 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.916 DEBUG:boto:Method: HEAD 2022-09-03T06:45:27.917 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.917 DEBUG:boto:Data: 2022-09-03T06:45:27.918 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.918 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.919 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.919 DEBUG:boto:Params: {} 2022-09-03T06:45:27.920 DEBUG:boto:Token: None 2022-09-03T06:45:27.921 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.921 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mQSMuQl2OOqyCV363Q55EU/n/rg= 2022-09-03T06:45:27.922 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mQSMuQl2OOqyCV363Q55EU/n/rg=', 'Content-Length': '0'} 2022-09-03T06:45:27.924 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000be8c7c6df84b1270-006312f807-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.925 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:27.925 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:27.926 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.926 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.927 DEBUG:boto:Method: HEAD 2022-09-03T06:45:27.927 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.928 DEBUG:boto:Data: 2022-09-03T06:45:27.928 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.929 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.929 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.930 DEBUG:boto:Params: {} 2022-09-03T06:45:27.930 DEBUG:boto:Token: None 2022-09-03T06:45:27.931 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.931 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mQSMuQl2OOqyCV363Q55EU/n/rg= 2022-09-03T06:45:27.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mQSMuQl2OOqyCV363Q55EU/n/rg=', 'Content-Length': '0'} 2022-09-03T06:45:27.934 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007f42721ec99ca4bd-006312f807-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.935 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:45:27.936 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:27.936 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:27.937 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.937 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.938 DEBUG:boto:Method: GET 2022-09-03T06:45:27.938 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.939 DEBUG:boto:Data: 2022-09-03T06:45:27.939 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.940 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:27.940 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.941 DEBUG:boto:Params: {} 2022-09-03T06:45:27.941 DEBUG:boto:Token: None 2022-09-03T06:45:27.942 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.943 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XWwTrVldPQ7iIMIOuL72nD9iTfQ= 2022-09-03T06:45:27.943 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XWwTrVldPQ7iIMIOuL72nD9iTfQ=', 'Content-Length': '0'} 2022-09-03T06:45:27.945 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f579802ca1b9a04a-006312f807-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.946 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:27.947 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:27.947 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.948 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.948 DEBUG:boto:Method: GET 2022-09-03T06:45:27.949 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.949 DEBUG:boto:Data: 2022-09-03T06:45:27.950 DEBUG:boto:Headers: {} 2022-09-03T06:45:27.950 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:27.951 DEBUG:boto:Port: 8000 2022-09-03T06:45:27.951 DEBUG:boto:Params: {} 2022-09-03T06:45:27.952 DEBUG:boto:Token: None 2022-09-03T06:45:27.953 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:27 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:27.953 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XWwTrVldPQ7iIMIOuL72nD9iTfQ= 2022-09-03T06:45:27.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XWwTrVldPQ7iIMIOuL72nD9iTfQ=', 'Content-Length': '0'} 2022-09-03T06:45:27.956 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d6c71f7ffad46b4-006312f807-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:27.998 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:28.000 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:28.001 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:28.002 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:28.003 DEBUG:boto:Method: GET 2022-09-03T06:45:28.004 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:45:28.005 DEBUG:boto:Data: 2022-09-03T06:45:28.005 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.006 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.006 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.007 DEBUG:boto:Params: {} 2022-09-03T06:45:28.008 DEBUG:boto:Token: None 2022-09-03T06:45:28.008 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/ 2022-09-03T06:45:28.009 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg= 2022-09-03T06:45:28.009 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg=', 'Content-Length': '0'} 2022-09-03T06:45:28.012 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000010be7765ce28084f-006312f808-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.014 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:28.015 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:28.015 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:28.016 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:28.017 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:28.018 DEBUG:boto:Method: GET 2022-09-03T06:45:28.019 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:45:28.019 DEBUG:boto:Data: 2022-09-03T06:45:28.020 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.020 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.021 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.021 DEBUG:boto:Params: {} 2022-09-03T06:45:28.022 DEBUG:boto:Token: None 2022-09-03T06:45:28.023 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/ 2022-09-03T06:45:28.024 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg= 2022-09-03T06:45:28.025 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg=', 'Content-Length': '0'} 2022-09-03T06:45:28.029 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000081f699f44212ad61-006312f808-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.031 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:28.032 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:28.033 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:28.034 DEBUG:boto:Method: HEAD 2022-09-03T06:45:28.035 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:45:28.035 DEBUG:boto:Data: 2022-09-03T06:45:28.036 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.037 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.037 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.038 DEBUG:boto:Params: {} 2022-09-03T06:45:28.039 DEBUG:boto:Token: None 2022-09-03T06:45:28.039 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj44 2022-09-03T06:45:28.040 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9pVy/HD2knJTvanc55pjxitfv/w= 2022-09-03T06:45:28.041 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9pVy/HD2knJTvanc55pjxitfv/w=', 'Content-Length': '0'} 2022-09-03T06:45:28.044 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e3ef3e665019f76b-006312f808-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.045 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:28.045 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:28.046 DEBUG:boto:Method: HEAD 2022-09-03T06:45:28.047 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:45:28.047 DEBUG:boto:Data: 2022-09-03T06:45:28.048 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.049 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.049 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.050 DEBUG:boto:Params: {} 2022-09-03T06:45:28.051 DEBUG:boto:Token: None 2022-09-03T06:45:28.052 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj44 2022-09-03T06:45:28.053 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9pVy/HD2knJTvanc55pjxitfv/w= 2022-09-03T06:45:28.054 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9pVy/HD2knJTvanc55pjxitfv/w=', 'Content-Length': '0'} 2022-09-03T06:45:28.057 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000807f4524452db7c0-006312f808-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.058 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:45:28.059 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:28.060 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:28.060 DEBUG:boto:Method: GET 2022-09-03T06:45:28.061 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:45:28.061 DEBUG:boto:Data: 2022-09-03T06:45:28.062 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.063 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.063 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.064 DEBUG:boto:Params: {} 2022-09-03T06:45:28.065 DEBUG:boto:Token: None 2022-09-03T06:45:28.065 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj44 2022-09-03T06:45:28.066 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0f6qU3Xwg5Q70JHWJHxU46b99bs= 2022-09-03T06:45:28.067 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0f6qU3Xwg5Q70JHWJHxU46b99bs=', 'Content-Length': '0'} 2022-09-03T06:45:28.069 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d896138367615d29-006312f808-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.070 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:28.070 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:28.071 DEBUG:boto:Method: GET 2022-09-03T06:45:28.071 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:45:28.072 DEBUG:boto:Data: 2022-09-03T06:45:28.073 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.073 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.074 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.074 DEBUG:boto:Params: {} 2022-09-03T06:45:28.075 DEBUG:boto:Token: None 2022-09-03T06:45:28.076 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj44 2022-09-03T06:45:28.077 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0f6qU3Xwg5Q70JHWJHxU46b99bs= 2022-09-03T06:45:28.077 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0f6qU3Xwg5Q70JHWJHxU46b99bs=', 'Content-Length': '0'} 2022-09-03T06:45:28.079 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008f8ce1b7163c42af-006312f808-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.080 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:28.081 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:28.081 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:28.082 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:28.082 DEBUG:boto:Method: GET 2022-09-03T06:45:28.083 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:45:28.084 DEBUG:boto:Data: 2022-09-03T06:45:28.084 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.085 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.085 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.086 DEBUG:boto:Params: {} 2022-09-03T06:45:28.086 DEBUG:boto:Token: None 2022-09-03T06:45:28.087 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/ 2022-09-03T06:45:28.087 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg= 2022-09-03T06:45:28.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg=', 'Content-Length': '0'} 2022-09-03T06:45:28.092 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000097bcd4d7fd82759f-006312f808-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.094 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:28.095 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:28.096 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:28.097 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:28.098 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:28.098 DEBUG:boto:Method: GET 2022-09-03T06:45:28.099 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:45:28.099 DEBUG:boto:Data: 2022-09-03T06:45:28.100 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.100 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.101 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.102 DEBUG:boto:Params: {} 2022-09-03T06:45:28.102 DEBUG:boto:Token: None 2022-09-03T06:45:28.103 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/ 2022-09-03T06:45:28.104 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg= 2022-09-03T06:45:28.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg=', 'Content-Length': '0'} 2022-09-03T06:45:28.108 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b2321504a33e24c5-006312f808-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.109 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:28.110 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:45:28.111 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:28.112 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:28.112 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.113 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.113 DEBUG:boto:Method: GET 2022-09-03T06:45:28.114 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.115 DEBUG:boto:Data: 2022-09-03T06:45:28.115 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.116 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.116 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.117 DEBUG:boto:Params: {} 2022-09-03T06:45:28.118 DEBUG:boto:Token: None 2022-09-03T06:45:28.118 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.119 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RqXJ7a4mcMXh2wltdD3icIRqnSI= 2022-09-03T06:45:28.120 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RqXJ7a4mcMXh2wltdD3icIRqnSI=', 'Content-Length': '0'} 2022-09-03T06:45:28.124 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ce29e5b408aae5fd-006312f808-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.172 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:28.174 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:28.175 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.176 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.177 DEBUG:boto:Method: GET 2022-09-03T06:45:28.178 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.179 DEBUG:boto:Data: 2022-09-03T06:45:28.180 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.181 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.182 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.183 DEBUG:boto:Params: {} 2022-09-03T06:45:28.184 DEBUG:boto:Token: None 2022-09-03T06:45:28.185 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.186 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RqXJ7a4mcMXh2wltdD3icIRqnSI= 2022-09-03T06:45:28.187 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RqXJ7a4mcMXh2wltdD3icIRqnSI=', 'Content-Length': '0'} 2022-09-03T06:45:28.190 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b3023adfe2c8698e-006312f808-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.192 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:28.193 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:28.193 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.194 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.195 DEBUG:boto:Method: HEAD 2022-09-03T06:45:28.195 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.196 DEBUG:boto:Data: 2022-09-03T06:45:28.196 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.197 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.198 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.198 DEBUG:boto:Params: {} 2022-09-03T06:45:28.199 DEBUG:boto:Token: None 2022-09-03T06:45:28.200 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.201 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vx2pjEGNE1CS4gZnaSLpT5LNJ+8= 2022-09-03T06:45:28.201 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vx2pjEGNE1CS4gZnaSLpT5LNJ+8=', 'Content-Length': '0'} 2022-09-03T06:45:28.203 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003171fd9f1fbc62f9-006312f808-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.204 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:28.205 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:28.205 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.206 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.206 DEBUG:boto:Method: HEAD 2022-09-03T06:45:28.207 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.208 DEBUG:boto:Data: 2022-09-03T06:45:28.208 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.209 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.209 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.210 DEBUG:boto:Params: {} 2022-09-03T06:45:28.210 DEBUG:boto:Token: None 2022-09-03T06:45:28.211 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.211 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vx2pjEGNE1CS4gZnaSLpT5LNJ+8= 2022-09-03T06:45:28.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vx2pjEGNE1CS4gZnaSLpT5LNJ+8=', 'Content-Length': '0'} 2022-09-03T06:45:28.215 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000089905e6b240e6493-006312f808-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.216 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:45:28.217 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:28.218 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:28.219 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.220 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.221 DEBUG:boto:Method: GET 2022-09-03T06:45:28.222 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.223 DEBUG:boto:Data: 2022-09-03T06:45:28.224 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.225 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.225 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.226 DEBUG:boto:Params: {} 2022-09-03T06:45:28.227 DEBUG:boto:Token: None 2022-09-03T06:45:28.228 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.229 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RqXJ7a4mcMXh2wltdD3icIRqnSI= 2022-09-03T06:45:28.230 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RqXJ7a4mcMXh2wltdD3icIRqnSI=', 'Content-Length': '0'} 2022-09-03T06:45:28.232 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000079d205b2afa203f1-006312f808-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.234 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:28.235 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:28.236 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.236 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.237 DEBUG:boto:Method: GET 2022-09-03T06:45:28.237 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.238 DEBUG:boto:Data: 2022-09-03T06:45:28.238 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.239 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.239 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.240 DEBUG:boto:Params: {} 2022-09-03T06:45:28.241 DEBUG:boto:Token: None 2022-09-03T06:45:28.241 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:28.242 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RqXJ7a4mcMXh2wltdD3icIRqnSI= 2022-09-03T06:45:28.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RqXJ7a4mcMXh2wltdD3icIRqnSI=', 'Content-Length': '0'} 2022-09-03T06:45:28.246 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006a9ee07c1ece0d73-006312f808-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.290 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:28.291 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:28.292 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:28.293 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:28.293 DEBUG:boto:Method: GET 2022-09-03T06:45:28.294 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:45:28.295 DEBUG:boto:Data: 2022-09-03T06:45:28.296 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.296 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.297 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.297 DEBUG:boto:Params: {} 2022-09-03T06:45:28.298 DEBUG:boto:Token: None 2022-09-03T06:45:28.299 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/ 2022-09-03T06:45:28.300 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg= 2022-09-03T06:45:28.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg=', 'Content-Length': '0'} 2022-09-03T06:45:28.303 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a74d0aefc8e07086-006312f808-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.304 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:28.306 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:28.306 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:28.307 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:28.307 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:28.308 DEBUG:boto:Method: GET 2022-09-03T06:45:28.309 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:45:28.310 DEBUG:boto:Data: 2022-09-03T06:45:28.312 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.313 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.314 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.315 DEBUG:boto:Params: {} 2022-09-03T06:45:28.317 DEBUG:boto:Token: None 2022-09-03T06:45:28.318 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/ 2022-09-03T06:45:28.319 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg= 2022-09-03T06:45:28.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg=', 'Content-Length': '0'} 2022-09-03T06:45:28.325 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cf24865b1152cc44-006312f808-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.326 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:28.327 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:28.327 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:28.328 DEBUG:boto:Method: HEAD 2022-09-03T06:45:28.328 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:45:28.329 DEBUG:boto:Data: 2022-09-03T06:45:28.329 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.330 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.330 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.331 DEBUG:boto:Params: {} 2022-09-03T06:45:28.331 DEBUG:boto:Token: None 2022-09-03T06:45:28.332 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj46 2022-09-03T06:45:28.332 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pFf8BVS0fUNBy+hW1VP75PbkMu0= 2022-09-03T06:45:28.333 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pFf8BVS0fUNBy+hW1VP75PbkMu0=', 'Content-Length': '0'} 2022-09-03T06:45:28.335 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e0c1f6363926453-006312f808-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.336 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:28.336 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:28.337 DEBUG:boto:Method: HEAD 2022-09-03T06:45:28.338 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:45:28.339 DEBUG:boto:Data: 2022-09-03T06:45:28.339 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.340 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.340 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.340 DEBUG:boto:Params: {} 2022-09-03T06:45:28.341 DEBUG:boto:Token: None 2022-09-03T06:45:28.342 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj46 2022-09-03T06:45:28.342 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pFf8BVS0fUNBy+hW1VP75PbkMu0= 2022-09-03T06:45:28.343 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pFf8BVS0fUNBy+hW1VP75PbkMu0=', 'Content-Length': '0'} 2022-09-03T06:45:28.345 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b4848baa81cb6523-006312f808-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.346 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:45:28.347 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:28.347 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:28.348 DEBUG:boto:Method: GET 2022-09-03T06:45:28.348 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:45:28.349 DEBUG:boto:Data: 2022-09-03T06:45:28.349 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.350 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.350 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.351 DEBUG:boto:Params: {} 2022-09-03T06:45:28.352 DEBUG:boto:Token: None 2022-09-03T06:45:28.352 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj46 2022-09-03T06:45:28.353 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:P1wx+lL6OV69CeUwFIaptepwzjU= 2022-09-03T06:45:28.354 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:P1wx+lL6OV69CeUwFIaptepwzjU=', 'Content-Length': '0'} 2022-09-03T06:45:28.356 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000004dd709eb8a4734-006312f808-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.357 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:28.357 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:28.358 DEBUG:boto:Method: GET 2022-09-03T06:45:28.358 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:45:28.359 DEBUG:boto:Data: 2022-09-03T06:45:28.359 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.360 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.361 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.361 DEBUG:boto:Params: {} 2022-09-03T06:45:28.362 DEBUG:boto:Token: None 2022-09-03T06:45:28.362 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj46 2022-09-03T06:45:28.363 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:P1wx+lL6OV69CeUwFIaptepwzjU= 2022-09-03T06:45:28.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:P1wx+lL6OV69CeUwFIaptepwzjU=', 'Content-Length': '0'} 2022-09-03T06:45:28.366 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002bee7eb3bdc72abe-006312f808-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.366 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:28.367 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:28.367 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:28.368 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:28.369 DEBUG:boto:Method: GET 2022-09-03T06:45:28.369 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:45:28.370 DEBUG:boto:Data: 2022-09-03T06:45:28.371 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.371 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.372 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.372 DEBUG:boto:Params: {} 2022-09-03T06:45:28.373 DEBUG:boto:Token: None 2022-09-03T06:45:28.373 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/ 2022-09-03T06:45:28.374 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg= 2022-09-03T06:45:28.374 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg=', 'Content-Length': '0'} 2022-09-03T06:45:28.377 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000070ac30be2eb90cbc-006312f808-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.378 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:28.379 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:28.380 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:28.380 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:28.381 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:28.381 DEBUG:boto:Method: GET 2022-09-03T06:45:28.382 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:45:28.382 DEBUG:boto:Data: 2022-09-03T06:45:28.383 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.383 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.384 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.385 DEBUG:boto:Params: {} 2022-09-03T06:45:28.385 DEBUG:boto:Token: None 2022-09-03T06:45:28.386 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/ 2022-09-03T06:45:28.387 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg= 2022-09-03T06:45:28.387 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg=', 'Content-Length': '0'} 2022-09-03T06:45:28.392 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b5741ba85db9ff85-006312f808-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.393 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:28.394 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:45:28.395 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:28.395 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:28.396 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.397 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.399 DEBUG:boto:Method: GET 2022-09-03T06:45:28.399 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.400 DEBUG:boto:Data: 2022-09-03T06:45:28.402 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.403 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.403 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.404 DEBUG:boto:Params: {} 2022-09-03T06:45:28.405 DEBUG:boto:Token: None 2022-09-03T06:45:28.405 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.406 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HMs4JqxVSX+GLoXXuEP/GC4i6Lg= 2022-09-03T06:45:28.407 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HMs4JqxVSX+GLoXXuEP/GC4i6Lg=', 'Content-Length': '0'} 2022-09-03T06:45:28.409 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000546c50f354bd5e14-006312f808-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.410 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:28.410 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:28.411 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.411 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.412 DEBUG:boto:Method: GET 2022-09-03T06:45:28.412 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.413 DEBUG:boto:Data: 2022-09-03T06:45:28.413 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.414 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.414 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.415 DEBUG:boto:Params: {} 2022-09-03T06:45:28.416 DEBUG:boto:Token: None 2022-09-03T06:45:28.416 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.417 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HMs4JqxVSX+GLoXXuEP/GC4i6Lg= 2022-09-03T06:45:28.417 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HMs4JqxVSX+GLoXXuEP/GC4i6Lg=', 'Content-Length': '0'} 2022-09-03T06:45:28.420 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000026bf31b709b5aef-006312f808-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.466 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:28.467 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:28.468 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.468 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.469 DEBUG:boto:Method: HEAD 2022-09-03T06:45:28.469 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.470 DEBUG:boto:Data: 2022-09-03T06:45:28.471 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.472 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.482 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.483 DEBUG:boto:Params: {} 2022-09-03T06:45:28.484 DEBUG:boto:Token: None 2022-09-03T06:45:28.484 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.485 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6BaF9Hcw4uDOQVib5RMjxEyYKo0= 2022-09-03T06:45:28.485 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6BaF9Hcw4uDOQVib5RMjxEyYKo0=', 'Content-Length': '0'} 2022-09-03T06:45:28.488 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000061cddf709edad495-006312f808-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.489 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:28.489 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:28.490 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.491 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.491 DEBUG:boto:Method: HEAD 2022-09-03T06:45:28.492 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.494 DEBUG:boto:Data: 2022-09-03T06:45:28.495 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.498 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.499 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.500 DEBUG:boto:Params: {} 2022-09-03T06:45:28.501 DEBUG:boto:Token: None 2022-09-03T06:45:28.502 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.503 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6BaF9Hcw4uDOQVib5RMjxEyYKo0= 2022-09-03T06:45:28.504 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6BaF9Hcw4uDOQVib5RMjxEyYKo0=', 'Content-Length': '0'} 2022-09-03T06:45:28.507 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc15423dff0e5228-006312f808-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.508 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:45:28.508 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:28.509 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:28.510 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.511 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.512 DEBUG:boto:Method: GET 2022-09-03T06:45:28.513 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.513 DEBUG:boto:Data: 2022-09-03T06:45:28.514 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.515 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.516 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.516 DEBUG:boto:Params: {} 2022-09-03T06:45:28.517 DEBUG:boto:Token: None 2022-09-03T06:45:28.518 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.519 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HMs4JqxVSX+GLoXXuEP/GC4i6Lg= 2022-09-03T06:45:28.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HMs4JqxVSX+GLoXXuEP/GC4i6Lg=', 'Content-Length': '0'} 2022-09-03T06:45:28.531 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000043a0662a20a6521a-006312f808-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.532 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:28.533 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:28.535 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.536 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.537 DEBUG:boto:Method: GET 2022-09-03T06:45:28.538 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.539 DEBUG:boto:Data: 2022-09-03T06:45:28.540 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.541 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.542 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.543 DEBUG:boto:Params: {} 2022-09-03T06:45:28.544 DEBUG:boto:Token: None 2022-09-03T06:45:28.546 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:28.546 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HMs4JqxVSX+GLoXXuEP/GC4i6Lg= 2022-09-03T06:45:28.547 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HMs4JqxVSX+GLoXXuEP/GC4i6Lg=', 'Content-Length': '0'} 2022-09-03T06:45:28.550 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f4912318f6f4a9b-006312f808-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.551 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:28.552 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:28.553 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:28.553 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:28.555 DEBUG:boto:Method: GET 2022-09-03T06:45:28.555 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:45:28.556 DEBUG:boto:Data: 2022-09-03T06:45:28.557 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.558 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.559 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.560 DEBUG:boto:Params: {} 2022-09-03T06:45:28.561 DEBUG:boto:Token: None 2022-09-03T06:45:28.562 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/ 2022-09-03T06:45:28.563 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg= 2022-09-03T06:45:28.564 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg=', 'Content-Length': '0'} 2022-09-03T06:45:28.567 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a09ced95baf2dfa6-006312f808-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.610 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:28.611 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:28.613 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:28.614 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:28.614 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:28.615 DEBUG:boto:Method: GET 2022-09-03T06:45:28.615 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:45:28.616 DEBUG:boto:Data: 2022-09-03T06:45:28.616 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.617 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.617 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.618 DEBUG:boto:Params: {} 2022-09-03T06:45:28.619 DEBUG:boto:Token: None 2022-09-03T06:45:28.619 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/ 2022-09-03T06:45:28.620 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg= 2022-09-03T06:45:28.621 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg=', 'Content-Length': '0'} 2022-09-03T06:45:28.626 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000822210b18c65f8d5-006312f808-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.627 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:28.628 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:28.628 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:28.629 DEBUG:boto:Method: HEAD 2022-09-03T06:45:28.631 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:45:28.632 DEBUG:boto:Data: 2022-09-03T06:45:28.633 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.633 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.634 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.635 DEBUG:boto:Params: {} 2022-09-03T06:45:28.635 DEBUG:boto:Token: None 2022-09-03T06:45:28.636 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj51 2022-09-03T06:45:28.636 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dLL0gOocjyYfISRaTT5XSOsxQ38= 2022-09-03T06:45:28.637 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dLL0gOocjyYfISRaTT5XSOsxQ38=', 'Content-Length': '0'} 2022-09-03T06:45:28.642 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2bda91b385fbf93-006312f808-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.643 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:28.644 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:28.645 DEBUG:boto:Method: HEAD 2022-09-03T06:45:28.646 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:45:28.647 DEBUG:boto:Data: 2022-09-03T06:45:28.648 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.649 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.650 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.651 DEBUG:boto:Params: {} 2022-09-03T06:45:28.652 DEBUG:boto:Token: None 2022-09-03T06:45:28.653 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj51 2022-09-03T06:45:28.655 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dLL0gOocjyYfISRaTT5XSOsxQ38= 2022-09-03T06:45:28.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dLL0gOocjyYfISRaTT5XSOsxQ38=', 'Content-Length': '0'} 2022-09-03T06:45:28.669 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006cc3b84317df442c-006312f808-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.670 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:45:28.671 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:28.672 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:28.673 DEBUG:boto:Method: GET 2022-09-03T06:45:28.674 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:45:28.675 DEBUG:boto:Data: 2022-09-03T06:45:28.676 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.677 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.677 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.678 DEBUG:boto:Params: {} 2022-09-03T06:45:28.681 DEBUG:boto:Token: None 2022-09-03T06:45:28.682 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj51 2022-09-03T06:45:28.684 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EwOGO7U5qewtBxcIuql900d7T88= 2022-09-03T06:45:28.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EwOGO7U5qewtBxcIuql900d7T88=', 'Content-Length': '0'} 2022-09-03T06:45:28.688 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000965101e8e7672e14-006312f808-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.690 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:28.692 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:28.692 DEBUG:boto:Method: GET 2022-09-03T06:45:28.693 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:45:28.694 DEBUG:boto:Data: 2022-09-03T06:45:28.695 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.695 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.696 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.697 DEBUG:boto:Params: {} 2022-09-03T06:45:28.699 DEBUG:boto:Token: None 2022-09-03T06:45:28.702 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj51 2022-09-03T06:45:28.703 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EwOGO7U5qewtBxcIuql900d7T88= 2022-09-03T06:45:28.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EwOGO7U5qewtBxcIuql900d7T88=', 'Content-Length': '0'} 2022-09-03T06:45:28.708 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ed2e3665b5356f42-006312f808-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.709 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:28.710 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:28.711 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:28.711 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:28.712 DEBUG:boto:Method: GET 2022-09-03T06:45:28.713 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:45:28.714 DEBUG:boto:Data: 2022-09-03T06:45:28.715 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.716 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.717 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.718 DEBUG:boto:Params: {} 2022-09-03T06:45:28.719 DEBUG:boto:Token: None 2022-09-03T06:45:28.720 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/ 2022-09-03T06:45:28.721 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg= 2022-09-03T06:45:28.723 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg=', 'Content-Length': '0'} 2022-09-03T06:45:28.726 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e91f59e9ee167ce6-006312f808-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.727 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:28.729 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:28.730 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:28.731 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:28.732 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:28.733 DEBUG:boto:Method: GET 2022-09-03T06:45:28.734 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:45:28.735 DEBUG:boto:Data: 2022-09-03T06:45:28.736 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.737 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.738 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.739 DEBUG:boto:Params: {} 2022-09-03T06:45:28.740 DEBUG:boto:Token: None 2022-09-03T06:45:28.741 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/ 2022-09-03T06:45:28.742 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg= 2022-09-03T06:45:28.744 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg=', 'Content-Length': '0'} 2022-09-03T06:45:28.749 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c650d3552f84f17a-006312f808-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.750 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:28.752 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:45:28.753 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:28.754 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:28.755 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.757 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.758 DEBUG:boto:Method: GET 2022-09-03T06:45:28.759 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.760 DEBUG:boto:Data: 2022-09-03T06:45:28.761 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.763 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.764 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.765 DEBUG:boto:Params: {} 2022-09-03T06:45:28.766 DEBUG:boto:Token: None 2022-09-03T06:45:28.768 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.769 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q8UFHzguYUvCu3lctxtnPPsn1/g= 2022-09-03T06:45:28.770 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q8UFHzguYUvCu3lctxtnPPsn1/g=', 'Content-Length': '0'} 2022-09-03T06:45:28.774 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003280c61afe99cbe7-006312f808-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.775 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:28.776 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:28.777 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.778 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.779 DEBUG:boto:Method: GET 2022-09-03T06:45:28.779 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.780 DEBUG:boto:Data: 2022-09-03T06:45:28.780 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.781 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.782 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.782 DEBUG:boto:Params: {} 2022-09-03T06:45:28.783 DEBUG:boto:Token: None 2022-09-03T06:45:28.784 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.784 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q8UFHzguYUvCu3lctxtnPPsn1/g= 2022-09-03T06:45:28.785 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q8UFHzguYUvCu3lctxtnPPsn1/g=', 'Content-Length': '0'} 2022-09-03T06:45:28.788 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000004e961915863ad99-006312f808-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.789 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:28.789 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:28.790 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.790 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.791 DEBUG:boto:Method: HEAD 2022-09-03T06:45:28.791 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.792 DEBUG:boto:Data: 2022-09-03T06:45:28.793 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.793 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.794 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.795 DEBUG:boto:Params: {} 2022-09-03T06:45:28.795 DEBUG:boto:Token: None 2022-09-03T06:45:28.796 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.797 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KWknMWphn4CZmpb+0Y59dwbDHOU= 2022-09-03T06:45:28.798 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KWknMWphn4CZmpb+0Y59dwbDHOU=', 'Content-Length': '0'} 2022-09-03T06:45:28.800 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000201afeac182045f2-006312f808-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.801 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:28.801 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:28.802 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.802 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.803 DEBUG:boto:Method: HEAD 2022-09-03T06:45:28.803 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.804 DEBUG:boto:Data: 2022-09-03T06:45:28.805 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.805 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.806 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.806 DEBUG:boto:Params: {} 2022-09-03T06:45:28.807 DEBUG:boto:Token: None 2022-09-03T06:45:28.808 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.808 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KWknMWphn4CZmpb+0Y59dwbDHOU= 2022-09-03T06:45:28.809 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KWknMWphn4CZmpb+0Y59dwbDHOU=', 'Content-Length': '0'} 2022-09-03T06:45:28.811 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e9d5a4236ac35ec0-006312f808-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.812 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:45:28.813 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:28.813 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:28.814 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.814 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.815 DEBUG:boto:Method: GET 2022-09-03T06:45:28.815 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.816 DEBUG:boto:Data: 2022-09-03T06:45:28.816 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.817 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.817 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.818 DEBUG:boto:Params: {} 2022-09-03T06:45:28.819 DEBUG:boto:Token: None 2022-09-03T06:45:28.819 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.820 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q8UFHzguYUvCu3lctxtnPPsn1/g= 2022-09-03T06:45:28.820 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q8UFHzguYUvCu3lctxtnPPsn1/g=', 'Content-Length': '0'} 2022-09-03T06:45:28.823 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000709bc0ae40a0b9b1-006312f808-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.825 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:28.826 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:28.827 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.827 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.828 DEBUG:boto:Method: GET 2022-09-03T06:45:28.829 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.830 DEBUG:boto:Data: 2022-09-03T06:45:28.831 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.832 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.832 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.833 DEBUG:boto:Params: {} 2022-09-03T06:45:28.834 DEBUG:boto:Token: None 2022-09-03T06:45:28.834 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:28.835 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q8UFHzguYUvCu3lctxtnPPsn1/g= 2022-09-03T06:45:28.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q8UFHzguYUvCu3lctxtnPPsn1/g=', 'Content-Length': '0'} 2022-09-03T06:45:28.838 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000428b6fce16860666-006312f808-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.839 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:28.839 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:28.840 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:28.841 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:28.841 DEBUG:boto:Method: GET 2022-09-03T06:45:28.842 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:45:28.842 DEBUG:boto:Data: 2022-09-03T06:45:28.843 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.843 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.844 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.845 DEBUG:boto:Params: {} 2022-09-03T06:45:28.846 DEBUG:boto:Token: None 2022-09-03T06:45:28.847 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/ 2022-09-03T06:45:28.848 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg= 2022-09-03T06:45:28.849 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg=', 'Content-Length': '0'} 2022-09-03T06:45:28.852 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003060f2c31840aa49-006312f808-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.894 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:28.897 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:28.898 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:28.899 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:28.900 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:28.901 DEBUG:boto:Method: GET 2022-09-03T06:45:28.903 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:45:28.904 DEBUG:boto:Data: 2022-09-03T06:45:28.905 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.905 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.906 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.907 DEBUG:boto:Params: {} 2022-09-03T06:45:28.908 DEBUG:boto:Token: None 2022-09-03T06:45:28.909 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/ 2022-09-03T06:45:28.910 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg= 2022-09-03T06:45:28.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VY7U0FDp44SZsh6UgqZVIBWPbWg=', 'Content-Length': '0'} 2022-09-03T06:45:28.920 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000700dd27976b4becd-006312f808-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.921 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:28.922 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:28.923 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:28.923 DEBUG:boto:Method: HEAD 2022-09-03T06:45:28.924 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:45:28.924 DEBUG:boto:Data: 2022-09-03T06:45:28.925 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.926 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.926 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.927 DEBUG:boto:Params: {} 2022-09-03T06:45:28.927 DEBUG:boto:Token: None 2022-09-03T06:45:28.928 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj55 2022-09-03T06:45:28.929 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Sgx742n+xlWlu7OzIJ3Xhsv6Sp0= 2022-09-03T06:45:28.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Sgx742n+xlWlu7OzIJ3Xhsv6Sp0=', 'Content-Length': '0'} 2022-09-03T06:45:28.931 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006a94dff0fda2c43b-006312f808-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.932 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:28.933 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:28.933 DEBUG:boto:Method: HEAD 2022-09-03T06:45:28.934 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:45:28.934 DEBUG:boto:Data: 2022-09-03T06:45:28.935 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.935 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.936 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.937 DEBUG:boto:Params: {} 2022-09-03T06:45:28.937 DEBUG:boto:Token: None 2022-09-03T06:45:28.938 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj55 2022-09-03T06:45:28.938 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Sgx742n+xlWlu7OzIJ3Xhsv6Sp0= 2022-09-03T06:45:28.939 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Sgx742n+xlWlu7OzIJ3Xhsv6Sp0=', 'Content-Length': '0'} 2022-09-03T06:45:28.941 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f04c4cfa81763887-006312f808-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.942 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:45:28.943 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:28.943 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:28.944 DEBUG:boto:Method: GET 2022-09-03T06:45:28.944 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:45:28.945 DEBUG:boto:Data: 2022-09-03T06:45:28.945 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.946 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:28.947 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.947 DEBUG:boto:Params: {} 2022-09-03T06:45:28.948 DEBUG:boto:Token: None 2022-09-03T06:45:28.949 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj55 2022-09-03T06:45:28.949 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9mRW7Bu3kWFzJSTfOaUlecs9qYM= 2022-09-03T06:45:28.950 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9mRW7Bu3kWFzJSTfOaUlecs9qYM=', 'Content-Length': '0'} 2022-09-03T06:45:28.953 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006403fc4e7344c906-006312f808-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:28.954 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:28.954 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:28.955 DEBUG:boto:Method: GET 2022-09-03T06:45:28.956 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:45:28.956 DEBUG:boto:Data: 2022-09-03T06:45:28.957 DEBUG:boto:Headers: {} 2022-09-03T06:45:28.957 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:28.958 DEBUG:boto:Port: 8000 2022-09-03T06:45:28.959 DEBUG:boto:Params: {} 2022-09-03T06:45:28.959 DEBUG:boto:Token: None 2022-09-03T06:45:28.960 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:28 GMT /dsgvcy-75/obj55 2022-09-03T06:45:28.961 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9mRW7Bu3kWFzJSTfOaUlecs9qYM= 2022-09-03T06:45:28.962 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9mRW7Bu3kWFzJSTfOaUlecs9qYM=', 'Content-Length': '0'} 2022-09-03T06:45:28.964 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000111787147aecdc9b-006312f808-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.006 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:29.007 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:29.008 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:29.009 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:29.010 DEBUG:boto:Method: GET 2022-09-03T06:45:29.011 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:45:29.012 DEBUG:boto:Data: 2022-09-03T06:45:29.013 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.014 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.015 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.016 DEBUG:boto:Params: {} 2022-09-03T06:45:29.017 DEBUG:boto:Token: None 2022-09-03T06:45:29.019 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/ 2022-09-03T06:45:29.020 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA= 2022-09-03T06:45:29.021 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA=', 'Content-Length': '0'} 2022-09-03T06:45:29.025 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000272d92eb255b6469-006312f809-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.027 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:29.030 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:29.031 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:29.032 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:29.033 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:29.034 DEBUG:boto:Method: GET 2022-09-03T06:45:29.036 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:45:29.037 DEBUG:boto:Data: 2022-09-03T06:45:29.038 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.039 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.040 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.041 DEBUG:boto:Params: {} 2022-09-03T06:45:29.043 DEBUG:boto:Token: None 2022-09-03T06:45:29.044 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/ 2022-09-03T06:45:29.045 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA= 2022-09-03T06:45:29.047 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA=', 'Content-Length': '0'} 2022-09-03T06:45:29.051 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fe0a12a91679f836-006312f809-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.053 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:29.054 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:45:29.055 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:29.056 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:29.056 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.057 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.057 DEBUG:boto:Method: GET 2022-09-03T06:45:29.058 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.058 DEBUG:boto:Data: 2022-09-03T06:45:29.059 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.060 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.060 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.061 DEBUG:boto:Params: {} 2022-09-03T06:45:29.061 DEBUG:boto:Token: None 2022-09-03T06:45:29.062 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.063 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vrV3FI1l9FhxIvc2gJVCvwfjYXQ= 2022-09-03T06:45:29.063 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vrV3FI1l9FhxIvc2gJVCvwfjYXQ=', 'Content-Length': '0'} 2022-09-03T06:45:29.066 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000092d33b288857355a-006312f809-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.067 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:29.068 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:29.069 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.069 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.070 DEBUG:boto:Method: GET 2022-09-03T06:45:29.071 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.072 DEBUG:boto:Data: 2022-09-03T06:45:29.074 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.075 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.076 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.077 DEBUG:boto:Params: {} 2022-09-03T06:45:29.078 DEBUG:boto:Token: None 2022-09-03T06:45:29.079 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.080 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vrV3FI1l9FhxIvc2gJVCvwfjYXQ= 2022-09-03T06:45:29.082 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vrV3FI1l9FhxIvc2gJVCvwfjYXQ=', 'Content-Length': '0'} 2022-09-03T06:45:29.085 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000de224b2542615dae-006312f809-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.087 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:29.088 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:29.089 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.090 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.091 DEBUG:boto:Method: HEAD 2022-09-03T06:45:29.092 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.093 DEBUG:boto:Data: 2022-09-03T06:45:29.094 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.095 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.096 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.096 DEBUG:boto:Params: {} 2022-09-03T06:45:29.097 DEBUG:boto:Token: None 2022-09-03T06:45:29.099 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.100 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ecnrOVhj7apkLNx0yCD5c5N/mvE= 2022-09-03T06:45:29.101 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ecnrOVhj7apkLNx0yCD5c5N/mvE=', 'Content-Length': '0'} 2022-09-03T06:45:29.103 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000b15a1d2460f89b8-006312f809-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.105 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:29.106 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:29.107 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.108 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.109 DEBUG:boto:Method: HEAD 2022-09-03T06:45:29.110 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.111 DEBUG:boto:Data: 2022-09-03T06:45:29.112 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.112 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.113 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.114 DEBUG:boto:Params: {} 2022-09-03T06:45:29.115 DEBUG:boto:Token: None 2022-09-03T06:45:29.116 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.117 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ecnrOVhj7apkLNx0yCD5c5N/mvE= 2022-09-03T06:45:29.118 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ecnrOVhj7apkLNx0yCD5c5N/mvE=', 'Content-Length': '0'} 2022-09-03T06:45:29.120 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000017c9cdb224982726-006312f809-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.120 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:45:29.121 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:29.122 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:29.123 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.124 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.125 DEBUG:boto:Method: GET 2022-09-03T06:45:29.126 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.127 DEBUG:boto:Data: 2022-09-03T06:45:29.128 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.129 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.130 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.130 DEBUG:boto:Params: {} 2022-09-03T06:45:29.131 DEBUG:boto:Token: None 2022-09-03T06:45:29.132 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.132 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vrV3FI1l9FhxIvc2gJVCvwfjYXQ= 2022-09-03T06:45:29.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vrV3FI1l9FhxIvc2gJVCvwfjYXQ=', 'Content-Length': '0'} 2022-09-03T06:45:29.135 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002b5562da3844d0c2-006312f809-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.136 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:29.137 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:29.138 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.139 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.140 DEBUG:boto:Method: GET 2022-09-03T06:45:29.141 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.142 DEBUG:boto:Data: 2022-09-03T06:45:29.142 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.143 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.144 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.145 DEBUG:boto:Params: {} 2022-09-03T06:45:29.147 DEBUG:boto:Token: None 2022-09-03T06:45:29.148 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:29.149 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vrV3FI1l9FhxIvc2gJVCvwfjYXQ= 2022-09-03T06:45:29.150 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vrV3FI1l9FhxIvc2gJVCvwfjYXQ=', 'Content-Length': '0'} 2022-09-03T06:45:29.153 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000031432198179933c3-006312f809-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.155 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:29.156 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:29.157 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:29.158 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:29.159 DEBUG:boto:Method: GET 2022-09-03T06:45:29.159 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:45:29.160 DEBUG:boto:Data: 2022-09-03T06:45:29.161 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.162 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.163 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.164 DEBUG:boto:Params: {} 2022-09-03T06:45:29.165 DEBUG:boto:Token: None 2022-09-03T06:45:29.167 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/ 2022-09-03T06:45:29.168 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA= 2022-09-03T06:45:29.169 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA=', 'Content-Length': '0'} 2022-09-03T06:45:29.173 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ea2cd1081fef46a6-006312f809-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.175 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:29.176 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:29.177 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:29.178 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:29.179 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:29.180 DEBUG:boto:Method: GET 2022-09-03T06:45:29.181 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:45:29.182 DEBUG:boto:Data: 2022-09-03T06:45:29.183 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.184 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.185 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.186 DEBUG:boto:Params: {} 2022-09-03T06:45:29.187 DEBUG:boto:Token: None 2022-09-03T06:45:29.188 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/ 2022-09-03T06:45:29.188 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA= 2022-09-03T06:45:29.189 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA=', 'Content-Length': '0'} 2022-09-03T06:45:29.193 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cfb6dee09eecb3f1-006312f809-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.194 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:29.196 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:29.197 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:29.198 DEBUG:boto:Method: HEAD 2022-09-03T06:45:29.199 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:45:29.199 DEBUG:boto:Data: 2022-09-03T06:45:29.200 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.201 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.202 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.203 DEBUG:boto:Params: {} 2022-09-03T06:45:29.204 DEBUG:boto:Token: None 2022-09-03T06:45:29.205 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj57 2022-09-03T06:45:29.206 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FILYtM7PVsNlddHbsWLe4Dd47p4= 2022-09-03T06:45:29.207 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FILYtM7PVsNlddHbsWLe4Dd47p4=', 'Content-Length': '0'} 2022-09-03T06:45:29.210 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ea76135232084e55-006312f809-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.211 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:29.212 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:29.212 DEBUG:boto:Method: HEAD 2022-09-03T06:45:29.213 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:45:29.214 DEBUG:boto:Data: 2022-09-03T06:45:29.214 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.215 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.216 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.217 DEBUG:boto:Params: {} 2022-09-03T06:45:29.219 DEBUG:boto:Token: None 2022-09-03T06:45:29.219 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj57 2022-09-03T06:45:29.220 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FILYtM7PVsNlddHbsWLe4Dd47p4= 2022-09-03T06:45:29.221 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FILYtM7PVsNlddHbsWLe4Dd47p4=', 'Content-Length': '0'} 2022-09-03T06:45:29.223 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f0eda4e6c017a90a-006312f809-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.223 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:45:29.224 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:29.225 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:29.225 DEBUG:boto:Method: GET 2022-09-03T06:45:29.226 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:45:29.226 DEBUG:boto:Data: 2022-09-03T06:45:29.227 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.228 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.228 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.229 DEBUG:boto:Params: {} 2022-09-03T06:45:29.229 DEBUG:boto:Token: None 2022-09-03T06:45:29.230 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj57 2022-09-03T06:45:29.231 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hQNrojH9ApKbdJOOi8+xgw+FKJE= 2022-09-03T06:45:29.231 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hQNrojH9ApKbdJOOi8+xgw+FKJE=', 'Content-Length': '0'} 2022-09-03T06:45:29.233 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000160db028e11a3f1f-006312f809-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.234 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:29.235 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:29.235 DEBUG:boto:Method: GET 2022-09-03T06:45:29.236 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:45:29.237 DEBUG:boto:Data: 2022-09-03T06:45:29.237 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.238 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.239 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.239 DEBUG:boto:Params: {} 2022-09-03T06:45:29.240 DEBUG:boto:Token: None 2022-09-03T06:45:29.240 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj57 2022-09-03T06:45:29.241 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hQNrojH9ApKbdJOOi8+xgw+FKJE= 2022-09-03T06:45:29.242 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hQNrojH9ApKbdJOOi8+xgw+FKJE=', 'Content-Length': '0'} 2022-09-03T06:45:29.244 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003870820ce312fbf2-006312f809-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.245 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:29.246 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:29.247 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:29.247 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:29.248 DEBUG:boto:Method: GET 2022-09-03T06:45:29.249 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:45:29.249 DEBUG:boto:Data: 2022-09-03T06:45:29.250 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.250 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.251 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.252 DEBUG:boto:Params: {} 2022-09-03T06:45:29.252 DEBUG:boto:Token: None 2022-09-03T06:45:29.253 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/ 2022-09-03T06:45:29.254 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA= 2022-09-03T06:45:29.254 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA=', 'Content-Length': '0'} 2022-09-03T06:45:29.257 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003369be6df9040dd4-006312f809-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.258 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:29.259 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:29.260 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:29.260 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:29.261 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:29.261 DEBUG:boto:Method: GET 2022-09-03T06:45:29.262 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:45:29.263 DEBUG:boto:Data: 2022-09-03T06:45:29.263 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.264 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.265 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.265 DEBUG:boto:Params: {} 2022-09-03T06:45:29.266 DEBUG:boto:Token: None 2022-09-03T06:45:29.266 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/ 2022-09-03T06:45:29.267 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA= 2022-09-03T06:45:29.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA=', 'Content-Length': '0'} 2022-09-03T06:45:29.272 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d21ec91c7dd31fc1-006312f809-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.273 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:29.275 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:45:29.275 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:29.276 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:29.276 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.277 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.277 DEBUG:boto:Method: GET 2022-09-03T06:45:29.278 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.279 DEBUG:boto:Data: 2022-09-03T06:45:29.279 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.280 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.280 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.281 DEBUG:boto:Params: {} 2022-09-03T06:45:29.281 DEBUG:boto:Token: None 2022-09-03T06:45:29.282 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.283 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JIS3TqroTouaURB5FxSzD+34pNE= 2022-09-03T06:45:29.283 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JIS3TqroTouaURB5FxSzD+34pNE=', 'Content-Length': '0'} 2022-09-03T06:45:29.286 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d745dd58fa79f407-006312f809-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.287 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:29.288 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:29.288 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.289 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.289 DEBUG:boto:Method: GET 2022-09-03T06:45:29.290 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.291 DEBUG:boto:Data: 2022-09-03T06:45:29.291 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.292 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.292 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.293 DEBUG:boto:Params: {} 2022-09-03T06:45:29.294 DEBUG:boto:Token: None 2022-09-03T06:45:29.295 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.296 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JIS3TqroTouaURB5FxSzD+34pNE= 2022-09-03T06:45:29.297 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JIS3TqroTouaURB5FxSzD+34pNE=', 'Content-Length': '0'} 2022-09-03T06:45:29.299 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000071ee7fe8fa66f175-006312f809-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.343 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:29.345 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:29.345 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.346 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.347 DEBUG:boto:Method: HEAD 2022-09-03T06:45:29.347 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.348 DEBUG:boto:Data: 2022-09-03T06:45:29.348 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.349 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.349 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.350 DEBUG:boto:Params: {} 2022-09-03T06:45:29.351 DEBUG:boto:Token: None 2022-09-03T06:45:29.351 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.353 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OSjUg/v4chPdlbK8Rc6n7HPt3sE= 2022-09-03T06:45:29.354 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OSjUg/v4chPdlbK8Rc6n7HPt3sE=', 'Content-Length': '0'} 2022-09-03T06:45:29.357 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000086ba57c44cc90064-006312f809-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.358 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:29.358 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:29.359 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.359 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.360 DEBUG:boto:Method: HEAD 2022-09-03T06:45:29.361 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.361 DEBUG:boto:Data: 2022-09-03T06:45:29.362 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.362 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.363 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.363 DEBUG:boto:Params: {} 2022-09-03T06:45:29.364 DEBUG:boto:Token: None 2022-09-03T06:45:29.365 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OSjUg/v4chPdlbK8Rc6n7HPt3sE= 2022-09-03T06:45:29.366 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OSjUg/v4chPdlbK8Rc6n7HPt3sE=', 'Content-Length': '0'} 2022-09-03T06:45:29.368 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000de898956b50cace7-006312f809-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.370 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:45:29.371 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:29.371 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:29.372 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.373 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.373 DEBUG:boto:Method: GET 2022-09-03T06:45:29.374 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.375 DEBUG:boto:Data: 2022-09-03T06:45:29.375 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.376 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.377 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.377 DEBUG:boto:Params: {} 2022-09-03T06:45:29.378 DEBUG:boto:Token: None 2022-09-03T06:45:29.379 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.381 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JIS3TqroTouaURB5FxSzD+34pNE= 2022-09-03T06:45:29.382 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JIS3TqroTouaURB5FxSzD+34pNE=', 'Content-Length': '0'} 2022-09-03T06:45:29.384 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c0e3760d272c1d19-006312f809-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.385 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:29.386 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:29.387 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.388 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.388 DEBUG:boto:Method: GET 2022-09-03T06:45:29.389 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.390 DEBUG:boto:Data: 2022-09-03T06:45:29.391 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.391 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.392 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.393 DEBUG:boto:Params: {} 2022-09-03T06:45:29.393 DEBUG:boto:Token: None 2022-09-03T06:45:29.394 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:29.395 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JIS3TqroTouaURB5FxSzD+34pNE= 2022-09-03T06:45:29.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JIS3TqroTouaURB5FxSzD+34pNE=', 'Content-Length': '0'} 2022-09-03T06:45:29.398 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001e8361e28d11d591-006312f809-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.399 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:29.400 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:29.400 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:29.401 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:29.401 DEBUG:boto:Method: GET 2022-09-03T06:45:29.402 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:45:29.403 DEBUG:boto:Data: 2022-09-03T06:45:29.403 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.404 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.404 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.405 DEBUG:boto:Params: {} 2022-09-03T06:45:29.405 DEBUG:boto:Token: None 2022-09-03T06:45:29.406 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/ 2022-09-03T06:45:29.407 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA= 2022-09-03T06:45:29.407 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA=', 'Content-Length': '0'} 2022-09-03T06:45:29.410 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e0f5c5df4b500858-006312f809-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.411 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:29.413 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:29.413 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:29.414 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:29.414 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:29.415 DEBUG:boto:Method: GET 2022-09-03T06:45:29.415 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:45:29.416 DEBUG:boto:Data: 2022-09-03T06:45:29.416 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.417 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.417 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.418 DEBUG:boto:Params: {} 2022-09-03T06:45:29.419 DEBUG:boto:Token: None 2022-09-03T06:45:29.419 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/ 2022-09-03T06:45:29.420 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA= 2022-09-03T06:45:29.420 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA=', 'Content-Length': '0'} 2022-09-03T06:45:29.425 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b2c7f52900439d6c-006312f809-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.426 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:29.428 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:29.429 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:29.429 DEBUG:boto:Method: HEAD 2022-09-03T06:45:29.430 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:45:29.431 DEBUG:boto:Data: 2022-09-03T06:45:29.431 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.432 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.433 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.434 DEBUG:boto:Params: {} 2022-09-03T06:45:29.435 DEBUG:boto:Token: None 2022-09-03T06:45:29.436 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj58 2022-09-03T06:45:29.436 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iXcrBuKHL19lDkHDo26HntmHimc= 2022-09-03T06:45:29.437 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iXcrBuKHL19lDkHDo26HntmHimc=', 'Content-Length': '0'} 2022-09-03T06:45:29.439 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b0279eaf31aa2d76-006312f809-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.440 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:29.441 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:29.441 DEBUG:boto:Method: HEAD 2022-09-03T06:45:29.442 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:45:29.443 DEBUG:boto:Data: 2022-09-03T06:45:29.443 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.444 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.445 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.446 DEBUG:boto:Params: {} 2022-09-03T06:45:29.446 DEBUG:boto:Token: None 2022-09-03T06:45:29.447 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj58 2022-09-03T06:45:29.448 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iXcrBuKHL19lDkHDo26HntmHimc= 2022-09-03T06:45:29.450 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iXcrBuKHL19lDkHDo26HntmHimc=', 'Content-Length': '0'} 2022-09-03T06:45:29.452 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000092b77e0d4e8dba70-006312f809-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.454 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:45:29.455 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:29.456 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:29.457 DEBUG:boto:Method: GET 2022-09-03T06:45:29.458 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:45:29.459 DEBUG:boto:Data: 2022-09-03T06:45:29.460 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.461 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.462 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.463 DEBUG:boto:Params: {} 2022-09-03T06:45:29.464 DEBUG:boto:Token: None 2022-09-03T06:45:29.465 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj58 2022-09-03T06:45:29.466 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bgb1lNX9rwcSC4TdC2yxc3M3zpg= 2022-09-03T06:45:29.467 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bgb1lNX9rwcSC4TdC2yxc3M3zpg=', 'Content-Length': '0'} 2022-09-03T06:45:29.469 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000feb82ed361379b0c-006312f809-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.510 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:29.511 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:29.512 DEBUG:boto:Method: GET 2022-09-03T06:45:29.512 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:45:29.513 DEBUG:boto:Data: 2022-09-03T06:45:29.513 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.514 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.515 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.516 DEBUG:boto:Params: {} 2022-09-03T06:45:29.516 DEBUG:boto:Token: None 2022-09-03T06:45:29.518 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj58 2022-09-03T06:45:29.518 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bgb1lNX9rwcSC4TdC2yxc3M3zpg= 2022-09-03T06:45:29.519 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bgb1lNX9rwcSC4TdC2yxc3M3zpg=', 'Content-Length': '0'} 2022-09-03T06:45:29.522 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000f4b7cfe0cc86cc2-006312f809-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.523 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:29.523 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:29.524 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:29.524 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:29.525 DEBUG:boto:Method: GET 2022-09-03T06:45:29.526 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:45:29.526 DEBUG:boto:Data: 2022-09-03T06:45:29.527 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.527 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.528 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.528 DEBUG:boto:Params: {} 2022-09-03T06:45:29.529 DEBUG:boto:Token: None 2022-09-03T06:45:29.530 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/ 2022-09-03T06:45:29.530 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA= 2022-09-03T06:45:29.531 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA=', 'Content-Length': '0'} 2022-09-03T06:45:29.534 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cf7a960934395ee8-006312f809-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.534 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:29.535 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:29.536 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:29.536 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:29.537 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:29.538 DEBUG:boto:Method: GET 2022-09-03T06:45:29.538 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:45:29.539 DEBUG:boto:Data: 2022-09-03T06:45:29.539 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.540 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.540 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.541 DEBUG:boto:Params: {} 2022-09-03T06:45:29.541 DEBUG:boto:Token: None 2022-09-03T06:45:29.542 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/ 2022-09-03T06:45:29.543 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA= 2022-09-03T06:45:29.543 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA=', 'Content-Length': '0'} 2022-09-03T06:45:29.548 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000db5fd9baf6231290-006312f809-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.549 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:29.549 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:45:29.550 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:29.551 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:29.551 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.552 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.552 DEBUG:boto:Method: GET 2022-09-03T06:45:29.553 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.553 DEBUG:boto:Data: 2022-09-03T06:45:29.554 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.555 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.555 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.556 DEBUG:boto:Params: {} 2022-09-03T06:45:29.556 DEBUG:boto:Token: None 2022-09-03T06:45:29.557 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.558 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C4eH2IBvbgbnLMrhekByyKbfLhI= 2022-09-03T06:45:29.558 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C4eH2IBvbgbnLMrhekByyKbfLhI=', 'Content-Length': '0'} 2022-09-03T06:45:29.561 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000015a153584e48b8eb-006312f809-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.562 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:29.562 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:29.563 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.563 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.564 DEBUG:boto:Method: GET 2022-09-03T06:45:29.564 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.565 DEBUG:boto:Data: 2022-09-03T06:45:29.565 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.566 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.567 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.567 DEBUG:boto:Params: {} 2022-09-03T06:45:29.568 DEBUG:boto:Token: None 2022-09-03T06:45:29.568 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.569 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C4eH2IBvbgbnLMrhekByyKbfLhI= 2022-09-03T06:45:29.570 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C4eH2IBvbgbnLMrhekByyKbfLhI=', 'Content-Length': '0'} 2022-09-03T06:45:29.572 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c54b119c71b98799-006312f809-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.573 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:29.573 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:29.574 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.575 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.575 DEBUG:boto:Method: HEAD 2022-09-03T06:45:29.576 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.576 DEBUG:boto:Data: 2022-09-03T06:45:29.577 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.578 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.578 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.579 DEBUG:boto:Params: {} 2022-09-03T06:45:29.580 DEBUG:boto:Token: None 2022-09-03T06:45:29.580 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.581 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n8gVgbIytAWjXZFQD6Vio9FDlqg= 2022-09-03T06:45:29.581 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n8gVgbIytAWjXZFQD6Vio9FDlqg=', 'Content-Length': '0'} 2022-09-03T06:45:29.584 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f3b9b8c27d44e36-006312f809-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.585 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:29.585 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:29.586 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.586 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.587 DEBUG:boto:Method: HEAD 2022-09-03T06:45:29.588 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.588 DEBUG:boto:Data: 2022-09-03T06:45:29.589 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.589 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.590 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.591 DEBUG:boto:Params: {} 2022-09-03T06:45:29.591 DEBUG:boto:Token: None 2022-09-03T06:45:29.592 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.593 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n8gVgbIytAWjXZFQD6Vio9FDlqg= 2022-09-03T06:45:29.593 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n8gVgbIytAWjXZFQD6Vio9FDlqg=', 'Content-Length': '0'} 2022-09-03T06:45:29.604 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006040c7b2f8ebbc11-006312f809-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.605 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:45:29.606 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:29.606 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:29.607 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.607 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.608 DEBUG:boto:Method: GET 2022-09-03T06:45:29.609 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.609 DEBUG:boto:Data: 2022-09-03T06:45:29.610 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.610 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.611 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.612 DEBUG:boto:Params: {} 2022-09-03T06:45:29.612 DEBUG:boto:Token: None 2022-09-03T06:45:29.613 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.614 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C4eH2IBvbgbnLMrhekByyKbfLhI= 2022-09-03T06:45:29.615 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C4eH2IBvbgbnLMrhekByyKbfLhI=', 'Content-Length': '0'} 2022-09-03T06:45:29.617 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e62f9d02671f26d2-006312f809-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.618 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:29.618 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:29.619 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.620 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.620 DEBUG:boto:Method: GET 2022-09-03T06:45:29.621 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.621 DEBUG:boto:Data: 2022-09-03T06:45:29.622 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.623 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.623 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.624 DEBUG:boto:Params: {} 2022-09-03T06:45:29.624 DEBUG:boto:Token: None 2022-09-03T06:45:29.625 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:29.626 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C4eH2IBvbgbnLMrhekByyKbfLhI= 2022-09-03T06:45:29.627 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C4eH2IBvbgbnLMrhekByyKbfLhI=', 'Content-Length': '0'} 2022-09-03T06:45:29.629 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a64f394a9d3b2bc-006312f809-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.630 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:29.631 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:29.631 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:29.632 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:29.633 DEBUG:boto:Method: GET 2022-09-03T06:45:29.633 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:45:29.634 DEBUG:boto:Data: 2022-09-03T06:45:29.634 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.635 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.636 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.637 DEBUG:boto:Params: {} 2022-09-03T06:45:29.638 DEBUG:boto:Token: None 2022-09-03T06:45:29.638 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/ 2022-09-03T06:45:29.639 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA= 2022-09-03T06:45:29.640 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA=', 'Content-Length': '0'} 2022-09-03T06:45:29.642 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000020716f2e6caf51de-006312f809-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.643 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:29.644 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:29.645 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:29.645 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:29.646 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:29.646 DEBUG:boto:Method: GET 2022-09-03T06:45:29.647 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:45:29.648 DEBUG:boto:Data: 2022-09-03T06:45:29.648 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.649 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.649 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.650 DEBUG:boto:Params: {} 2022-09-03T06:45:29.651 DEBUG:boto:Token: None 2022-09-03T06:45:29.652 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/ 2022-09-03T06:45:29.653 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA= 2022-09-03T06:45:29.654 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA=', 'Content-Length': '0'} 2022-09-03T06:45:29.659 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009a7b4db5406ac0f7-006312f809-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.660 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:29.661 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:29.662 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:29.662 DEBUG:boto:Method: HEAD 2022-09-03T06:45:29.663 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:45:29.664 DEBUG:boto:Data: 2022-09-03T06:45:29.664 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.665 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.665 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.666 DEBUG:boto:Params: {} 2022-09-03T06:45:29.667 DEBUG:boto:Token: None 2022-09-03T06:45:29.667 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj68 2022-09-03T06:45:29.668 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+f7eMvItDS20wRYHQ+VOC/UAr+U= 2022-09-03T06:45:29.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+f7eMvItDS20wRYHQ+VOC/UAr+U=', 'Content-Length': '0'} 2022-09-03T06:45:29.671 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004ccc536d41c239e0-006312f809-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.672 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:29.672 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:29.673 DEBUG:boto:Method: HEAD 2022-09-03T06:45:29.673 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:45:29.674 DEBUG:boto:Data: 2022-09-03T06:45:29.675 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.675 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.676 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.676 DEBUG:boto:Params: {} 2022-09-03T06:45:29.677 DEBUG:boto:Token: None 2022-09-03T06:45:29.677 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj68 2022-09-03T06:45:29.678 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+f7eMvItDS20wRYHQ+VOC/UAr+U= 2022-09-03T06:45:29.679 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+f7eMvItDS20wRYHQ+VOC/UAr+U=', 'Content-Length': '0'} 2022-09-03T06:45:29.681 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e3f9f7d750186520-006312f809-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.682 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:45:29.683 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:29.684 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:29.685 DEBUG:boto:Method: GET 2022-09-03T06:45:29.686 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:45:29.687 DEBUG:boto:Data: 2022-09-03T06:45:29.688 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.688 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.689 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.690 DEBUG:boto:Params: {} 2022-09-03T06:45:29.690 DEBUG:boto:Token: None 2022-09-03T06:45:29.691 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj68 2022-09-03T06:45:29.692 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+9gbiQfWc10eZElmwh7gOI9v7AE= 2022-09-03T06:45:29.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+9gbiQfWc10eZElmwh7gOI9v7AE=', 'Content-Length': '0'} 2022-09-03T06:45:29.695 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc32d21ba8388273-006312f809-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.738 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:29.739 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:29.740 DEBUG:boto:Method: GET 2022-09-03T06:45:29.742 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:45:29.743 DEBUG:boto:Data: 2022-09-03T06:45:29.744 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.745 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.746 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.747 DEBUG:boto:Params: {} 2022-09-03T06:45:29.747 DEBUG:boto:Token: None 2022-09-03T06:45:29.748 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj68 2022-09-03T06:45:29.749 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+9gbiQfWc10eZElmwh7gOI9v7AE= 2022-09-03T06:45:29.749 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+9gbiQfWc10eZElmwh7gOI9v7AE=', 'Content-Length': '0'} 2022-09-03T06:45:29.755 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a90d046a870880e0-006312f809-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.757 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:29.758 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:29.759 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:29.760 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:29.761 DEBUG:boto:Method: GET 2022-09-03T06:45:29.762 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:45:29.763 DEBUG:boto:Data: 2022-09-03T06:45:29.764 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.764 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.766 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.766 DEBUG:boto:Params: {} 2022-09-03T06:45:29.768 DEBUG:boto:Token: None 2022-09-03T06:45:29.769 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/ 2022-09-03T06:45:29.770 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA= 2022-09-03T06:45:29.771 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA=', 'Content-Length': '0'} 2022-09-03T06:45:29.775 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dc38647183b68faf-006312f809-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.776 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:29.778 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:29.779 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:29.780 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:29.781 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:29.782 DEBUG:boto:Method: GET 2022-09-03T06:45:29.783 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:45:29.784 DEBUG:boto:Data: 2022-09-03T06:45:29.785 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.786 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.787 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.787 DEBUG:boto:Params: {} 2022-09-03T06:45:29.788 DEBUG:boto:Token: None 2022-09-03T06:45:29.789 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/ 2022-09-03T06:45:29.791 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA= 2022-09-03T06:45:29.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA=', 'Content-Length': '0'} 2022-09-03T06:45:29.796 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007dadc22d3e9932cf-006312f809-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.838 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:29.840 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:45:29.841 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:29.842 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:29.844 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.845 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.846 DEBUG:boto:Method: GET 2022-09-03T06:45:29.847 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.848 DEBUG:boto:Data: 2022-09-03T06:45:29.848 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.849 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.850 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.851 DEBUG:boto:Params: {} 2022-09-03T06:45:29.851 DEBUG:boto:Token: None 2022-09-03T06:45:29.852 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.853 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wW6Wg1YYPIwmhrkmXp9T8qnhvU0= 2022-09-03T06:45:29.853 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wW6Wg1YYPIwmhrkmXp9T8qnhvU0=', 'Content-Length': '0'} 2022-09-03T06:45:29.856 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b645aa3ed6443b5d-006312f809-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.858 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:29.859 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:29.859 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.860 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.861 DEBUG:boto:Method: GET 2022-09-03T06:45:29.861 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.862 DEBUG:boto:Data: 2022-09-03T06:45:29.862 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.863 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.863 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.864 DEBUG:boto:Params: {} 2022-09-03T06:45:29.865 DEBUG:boto:Token: None 2022-09-03T06:45:29.866 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.866 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wW6Wg1YYPIwmhrkmXp9T8qnhvU0= 2022-09-03T06:45:29.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wW6Wg1YYPIwmhrkmXp9T8qnhvU0=', 'Content-Length': '0'} 2022-09-03T06:45:29.870 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b3bcc6b58e673ef-006312f809-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.871 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:29.871 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:29.872 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.872 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.873 DEBUG:boto:Method: HEAD 2022-09-03T06:45:29.874 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.874 DEBUG:boto:Data: 2022-09-03T06:45:29.875 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.875 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.876 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.877 DEBUG:boto:Params: {} 2022-09-03T06:45:29.878 DEBUG:boto:Token: None 2022-09-03T06:45:29.878 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.879 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aEJdBLrBD5d4N3rH/7VhK27fJ8k= 2022-09-03T06:45:29.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aEJdBLrBD5d4N3rH/7VhK27fJ8k=', 'Content-Length': '0'} 2022-09-03T06:45:29.882 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f4806bc564490485-006312f809-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.883 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:29.884 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:29.885 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.886 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.886 DEBUG:boto:Method: HEAD 2022-09-03T06:45:29.887 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.888 DEBUG:boto:Data: 2022-09-03T06:45:29.888 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.889 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.890 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.891 DEBUG:boto:Params: {} 2022-09-03T06:45:29.892 DEBUG:boto:Token: None 2022-09-03T06:45:29.893 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.894 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aEJdBLrBD5d4N3rH/7VhK27fJ8k= 2022-09-03T06:45:29.894 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aEJdBLrBD5d4N3rH/7VhK27fJ8k=', 'Content-Length': '0'} 2022-09-03T06:45:29.897 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008eac7fcf03463cb9-006312f809-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.897 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:45:29.898 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:29.899 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:29.899 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.900 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.901 DEBUG:boto:Method: GET 2022-09-03T06:45:29.902 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.903 DEBUG:boto:Data: 2022-09-03T06:45:29.903 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.904 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.905 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.906 DEBUG:boto:Params: {} 2022-09-03T06:45:29.907 DEBUG:boto:Token: None 2022-09-03T06:45:29.908 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.909 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wW6Wg1YYPIwmhrkmXp9T8qnhvU0= 2022-09-03T06:45:29.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wW6Wg1YYPIwmhrkmXp9T8qnhvU0=', 'Content-Length': '0'} 2022-09-03T06:45:29.914 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000033ce27ff781006ca-006312f809-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.915 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:29.916 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:29.917 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.918 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.919 DEBUG:boto:Method: GET 2022-09-03T06:45:29.919 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.920 DEBUG:boto:Data: 2022-09-03T06:45:29.921 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.921 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.922 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.923 DEBUG:boto:Params: {} 2022-09-03T06:45:29.924 DEBUG:boto:Token: None 2022-09-03T06:45:29.925 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:29.925 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wW6Wg1YYPIwmhrkmXp9T8qnhvU0= 2022-09-03T06:45:29.926 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wW6Wg1YYPIwmhrkmXp9T8qnhvU0=', 'Content-Length': '0'} 2022-09-03T06:45:29.928 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000ba597899a6977d7-006312f809-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.930 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:29.931 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:29.931 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:29.932 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:29.932 DEBUG:boto:Method: GET 2022-09-03T06:45:29.933 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:45:29.933 DEBUG:boto:Data: 2022-09-03T06:45:29.934 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.934 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:29.935 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.936 DEBUG:boto:Params: {} 2022-09-03T06:45:29.936 DEBUG:boto:Token: None 2022-09-03T06:45:29.937 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/ 2022-09-03T06:45:29.937 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA= 2022-09-03T06:45:29.938 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA=', 'Content-Length': '0'} 2022-09-03T06:45:29.941 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f193d621fb5315c2-006312f809-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:29.982 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:29.984 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:29.985 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:29.986 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:29.987 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:29.988 DEBUG:boto:Method: GET 2022-09-03T06:45:29.989 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:45:29.990 DEBUG:boto:Data: 2022-09-03T06:45:29.991 DEBUG:boto:Headers: {} 2022-09-03T06:45:29.992 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:29.993 DEBUG:boto:Port: 8000 2022-09-03T06:45:29.994 DEBUG:boto:Params: {} 2022-09-03T06:45:29.995 DEBUG:boto:Token: None 2022-09-03T06:45:29.997 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:29 GMT /dsgvcy-75/ 2022-09-03T06:45:29.998 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA= 2022-09-03T06:45:29.999 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ok2Xq0bdJDQhyRlcv/MpfjEI1bA=', 'Content-Length': '0'} 2022-09-03T06:45:30.004 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009851bf5cdc904faa-006312f809-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.005 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:30.006 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:30.008 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:30.009 DEBUG:boto:Method: HEAD 2022-09-03T06:45:30.010 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:45:30.011 DEBUG:boto:Data: 2022-09-03T06:45:30.011 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.012 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.013 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.013 DEBUG:boto:Params: {} 2022-09-03T06:45:30.015 DEBUG:boto:Token: None 2022-09-03T06:45:30.015 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj69 2022-09-03T06:45:30.016 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XEdPTIdG0Qrh2lYvw7QsEt+XCGA= 2022-09-03T06:45:30.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XEdPTIdG0Qrh2lYvw7QsEt+XCGA=', 'Content-Length': '0'} 2022-09-03T06:45:30.019 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d10140a18e72ba0e-006312f80a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.020 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:30.021 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:30.022 DEBUG:boto:Method: HEAD 2022-09-03T06:45:30.023 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:45:30.024 DEBUG:boto:Data: 2022-09-03T06:45:30.025 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.026 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.027 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.028 DEBUG:boto:Params: {} 2022-09-03T06:45:30.029 DEBUG:boto:Token: None 2022-09-03T06:45:30.031 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj69 2022-09-03T06:45:30.032 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XEdPTIdG0Qrh2lYvw7QsEt+XCGA= 2022-09-03T06:45:30.033 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XEdPTIdG0Qrh2lYvw7QsEt+XCGA=', 'Content-Length': '0'} 2022-09-03T06:45:30.036 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002522353cbd22025f-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.037 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:45:30.038 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:30.041 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:30.042 DEBUG:boto:Method: GET 2022-09-03T06:45:30.043 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:45:30.044 DEBUG:boto:Data: 2022-09-03T06:45:30.044 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.045 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.045 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.046 DEBUG:boto:Params: {} 2022-09-03T06:45:30.047 DEBUG:boto:Token: None 2022-09-03T06:45:30.047 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj69 2022-09-03T06:45:30.048 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:T6mxIyX3W+cGlgrlQHv7akuo+oM= 2022-09-03T06:45:30.049 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:T6mxIyX3W+cGlgrlQHv7akuo+oM=', 'Content-Length': '0'} 2022-09-03T06:45:30.051 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a9a3c48f56eeb987-006312f80a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.052 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:30.053 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:30.054 DEBUG:boto:Method: GET 2022-09-03T06:45:30.054 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:45:30.055 DEBUG:boto:Data: 2022-09-03T06:45:30.056 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.056 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.057 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.057 DEBUG:boto:Params: {} 2022-09-03T06:45:30.058 DEBUG:boto:Token: None 2022-09-03T06:45:30.059 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj69 2022-09-03T06:45:30.059 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:T6mxIyX3W+cGlgrlQHv7akuo+oM= 2022-09-03T06:45:30.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:T6mxIyX3W+cGlgrlQHv7akuo+oM=', 'Content-Length': '0'} 2022-09-03T06:45:30.066 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009b03c0a4baac893e-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.067 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:30.068 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:30.069 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:30.069 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:30.070 DEBUG:boto:Method: GET 2022-09-03T06:45:30.070 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:45:30.071 DEBUG:boto:Data: 2022-09-03T06:45:30.071 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.072 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.072 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.073 DEBUG:boto:Params: {} 2022-09-03T06:45:30.074 DEBUG:boto:Token: None 2022-09-03T06:45:30.074 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/ 2022-09-03T06:45:30.075 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q= 2022-09-03T06:45:30.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q=', 'Content-Length': '0'} 2022-09-03T06:45:30.079 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009ba0850220792566-006312f80a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.079 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:30.080 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:30.081 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:30.082 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:30.082 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:30.083 DEBUG:boto:Method: GET 2022-09-03T06:45:30.083 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:45:30.084 DEBUG:boto:Data: 2022-09-03T06:45:30.085 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.085 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.086 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.086 DEBUG:boto:Params: {} 2022-09-03T06:45:30.087 DEBUG:boto:Token: None 2022-09-03T06:45:30.088 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/ 2022-09-03T06:45:30.088 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q= 2022-09-03T06:45:30.089 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q=', 'Content-Length': '0'} 2022-09-03T06:45:30.093 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005cb3a657ba0e831f-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.142 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:30.144 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:45:30.145 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:30.146 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:30.147 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.148 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.151 DEBUG:boto:Method: GET 2022-09-03T06:45:30.152 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.153 DEBUG:boto:Data: 2022-09-03T06:45:30.154 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.155 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.156 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.157 DEBUG:boto:Params: {} 2022-09-03T06:45:30.158 DEBUG:boto:Token: None 2022-09-03T06:45:30.159 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.161 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qEDgjDToYZEUsSWaHdsMBahmIfs= 2022-09-03T06:45:30.161 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qEDgjDToYZEUsSWaHdsMBahmIfs=', 'Content-Length': '0'} 2022-09-03T06:45:30.164 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ae248689076c4ee-006312f80a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.164 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:30.165 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:30.166 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.166 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.167 DEBUG:boto:Method: GET 2022-09-03T06:45:30.169 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.170 DEBUG:boto:Data: 2022-09-03T06:45:30.170 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.171 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.172 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.173 DEBUG:boto:Params: {} 2022-09-03T06:45:30.174 DEBUG:boto:Token: None 2022-09-03T06:45:30.175 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.175 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qEDgjDToYZEUsSWaHdsMBahmIfs= 2022-09-03T06:45:30.176 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qEDgjDToYZEUsSWaHdsMBahmIfs=', 'Content-Length': '0'} 2022-09-03T06:45:30.183 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001551249b2f307ef7-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.185 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:30.187 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:30.188 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.189 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.189 DEBUG:boto:Method: HEAD 2022-09-03T06:45:30.190 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.191 DEBUG:boto:Data: 2022-09-03T06:45:30.192 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.193 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.194 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.196 DEBUG:boto:Params: {} 2022-09-03T06:45:30.197 DEBUG:boto:Token: None 2022-09-03T06:45:30.198 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.199 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v3npUDxA1fDYLGEvZBjZ+QdjX3Q= 2022-09-03T06:45:30.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v3npUDxA1fDYLGEvZBjZ+QdjX3Q=', 'Content-Length': '0'} 2022-09-03T06:45:30.202 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c8a36dcd2071028c-006312f80a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.204 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:30.205 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:30.206 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.207 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.209 DEBUG:boto:Method: HEAD 2022-09-03T06:45:30.210 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.211 DEBUG:boto:Data: 2022-09-03T06:45:30.212 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.213 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.214 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.215 DEBUG:boto:Params: {} 2022-09-03T06:45:30.215 DEBUG:boto:Token: None 2022-09-03T06:45:30.216 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.217 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v3npUDxA1fDYLGEvZBjZ+QdjX3Q= 2022-09-03T06:45:30.217 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v3npUDxA1fDYLGEvZBjZ+QdjX3Q=', 'Content-Length': '0'} 2022-09-03T06:45:30.221 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a2cd971443bf9bf6-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.222 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:45:30.223 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:30.223 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:30.224 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.225 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.226 DEBUG:boto:Method: GET 2022-09-03T06:45:30.226 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.227 DEBUG:boto:Data: 2022-09-03T06:45:30.229 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.230 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.231 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.232 DEBUG:boto:Params: {} 2022-09-03T06:45:30.232 DEBUG:boto:Token: None 2022-09-03T06:45:30.233 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.233 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qEDgjDToYZEUsSWaHdsMBahmIfs= 2022-09-03T06:45:30.234 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qEDgjDToYZEUsSWaHdsMBahmIfs=', 'Content-Length': '0'} 2022-09-03T06:45:30.237 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004bc7755878b58968-006312f80a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.238 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:30.239 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:30.239 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.240 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.241 DEBUG:boto:Method: GET 2022-09-03T06:45:30.241 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.242 DEBUG:boto:Data: 2022-09-03T06:45:30.243 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.244 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.245 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.246 DEBUG:boto:Params: {} 2022-09-03T06:45:30.247 DEBUG:boto:Token: None 2022-09-03T06:45:30.248 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:30.249 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qEDgjDToYZEUsSWaHdsMBahmIfs= 2022-09-03T06:45:30.250 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qEDgjDToYZEUsSWaHdsMBahmIfs=', 'Content-Length': '0'} 2022-09-03T06:45:30.253 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a563334191eb548-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.254 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:30.255 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:30.256 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:30.258 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:30.259 DEBUG:boto:Method: GET 2022-09-03T06:45:30.260 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:45:30.261 DEBUG:boto:Data: 2022-09-03T06:45:30.262 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.263 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.264 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.265 DEBUG:boto:Params: {} 2022-09-03T06:45:30.266 DEBUG:boto:Token: None 2022-09-03T06:45:30.268 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/ 2022-09-03T06:45:30.269 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q= 2022-09-03T06:45:30.270 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q=', 'Content-Length': '0'} 2022-09-03T06:45:30.273 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008a834581b685b58d-006312f80a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.275 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:30.276 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:30.276 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:30.277 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:30.278 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:30.278 DEBUG:boto:Method: GET 2022-09-03T06:45:30.279 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:45:30.279 DEBUG:boto:Data: 2022-09-03T06:45:30.280 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.280 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.281 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.281 DEBUG:boto:Params: {} 2022-09-03T06:45:30.282 DEBUG:boto:Token: None 2022-09-03T06:45:30.283 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/ 2022-09-03T06:45:30.283 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q= 2022-09-03T06:45:30.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q=', 'Content-Length': '0'} 2022-09-03T06:45:30.288 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e1bbb3d5cd4bf5e4-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.289 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:30.290 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:30.290 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:30.291 DEBUG:boto:Method: HEAD 2022-09-03T06:45:30.291 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:45:30.292 DEBUG:boto:Data: 2022-09-03T06:45:30.292 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.293 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.293 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.294 DEBUG:boto:Params: {} 2022-09-03T06:45:30.295 DEBUG:boto:Token: None 2022-09-03T06:45:30.295 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj76 2022-09-03T06:45:30.296 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+AIMt7yGF2UGBsTiiDq+PIfAobQ= 2022-09-03T06:45:30.297 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+AIMt7yGF2UGBsTiiDq+PIfAobQ=', 'Content-Length': '0'} 2022-09-03T06:45:30.299 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d5693adf898ece7f-006312f80a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.299 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:30.300 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:30.301 DEBUG:boto:Method: HEAD 2022-09-03T06:45:30.301 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:45:30.302 DEBUG:boto:Data: 2022-09-03T06:45:30.302 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.303 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.303 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.304 DEBUG:boto:Params: {} 2022-09-03T06:45:30.304 DEBUG:boto:Token: None 2022-09-03T06:45:30.305 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj76 2022-09-03T06:45:30.306 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+AIMt7yGF2UGBsTiiDq+PIfAobQ= 2022-09-03T06:45:30.306 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+AIMt7yGF2UGBsTiiDq+PIfAobQ=', 'Content-Length': '0'} 2022-09-03T06:45:30.309 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b0400f6bba6b398c-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.310 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:45:30.311 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:30.312 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:30.313 DEBUG:boto:Method: GET 2022-09-03T06:45:30.315 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:45:30.316 DEBUG:boto:Data: 2022-09-03T06:45:30.317 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.318 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.319 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.319 DEBUG:boto:Params: {} 2022-09-03T06:45:30.320 DEBUG:boto:Token: None 2022-09-03T06:45:30.322 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj76 2022-09-03T06:45:30.323 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uVlfylhw9b4FoKLxArc/BXu+NcU= 2022-09-03T06:45:30.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uVlfylhw9b4FoKLxArc/BXu+NcU=', 'Content-Length': '0'} 2022-09-03T06:45:30.327 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f751e1646405645f-006312f80a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.374 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:30.375 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:30.375 DEBUG:boto:Method: GET 2022-09-03T06:45:30.376 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:45:30.376 DEBUG:boto:Data: 2022-09-03T06:45:30.377 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.377 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.378 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.379 DEBUG:boto:Params: {} 2022-09-03T06:45:30.380 DEBUG:boto:Token: None 2022-09-03T06:45:30.380 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj76 2022-09-03T06:45:30.381 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uVlfylhw9b4FoKLxArc/BXu+NcU= 2022-09-03T06:45:30.381 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uVlfylhw9b4FoKLxArc/BXu+NcU=', 'Content-Length': '0'} 2022-09-03T06:45:30.384 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008f61e2770930ba03-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.385 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:30.385 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:30.386 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:30.387 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:30.387 DEBUG:boto:Method: GET 2022-09-03T06:45:30.388 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:45:30.389 DEBUG:boto:Data: 2022-09-03T06:45:30.389 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.390 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.390 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.391 DEBUG:boto:Params: {} 2022-09-03T06:45:30.391 DEBUG:boto:Token: None 2022-09-03T06:45:30.392 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/ 2022-09-03T06:45:30.393 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q= 2022-09-03T06:45:30.393 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q=', 'Content-Length': '0'} 2022-09-03T06:45:30.396 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e64159197f7f8c30-006312f80a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.397 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:30.398 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:30.399 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:30.399 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:30.400 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:30.400 DEBUG:boto:Method: GET 2022-09-03T06:45:30.401 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:45:30.401 DEBUG:boto:Data: 2022-09-03T06:45:30.402 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.402 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.403 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.403 DEBUG:boto:Params: {} 2022-09-03T06:45:30.404 DEBUG:boto:Token: None 2022-09-03T06:45:30.405 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/ 2022-09-03T06:45:30.405 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q= 2022-09-03T06:45:30.406 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q=', 'Content-Length': '0'} 2022-09-03T06:45:30.410 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000006480e9db04f1a1e-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.411 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:30.412 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:45:30.413 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:30.413 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:30.414 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.415 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.415 DEBUG:boto:Method: GET 2022-09-03T06:45:30.416 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.417 DEBUG:boto:Data: 2022-09-03T06:45:30.417 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.418 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.419 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.420 DEBUG:boto:Params: {} 2022-09-03T06:45:30.420 DEBUG:boto:Token: None 2022-09-03T06:45:30.421 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.422 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f7M22TzkENOgljVEQXJt9mz8BS8= 2022-09-03T06:45:30.423 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f7M22TzkENOgljVEQXJt9mz8BS8=', 'Content-Length': '0'} 2022-09-03T06:45:30.427 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000f028be824547a1b-006312f80a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.428 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:30.429 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:30.429 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.430 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.431 DEBUG:boto:Method: GET 2022-09-03T06:45:30.431 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.432 DEBUG:boto:Data: 2022-09-03T06:45:30.433 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.434 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.435 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.435 DEBUG:boto:Params: {} 2022-09-03T06:45:30.436 DEBUG:boto:Token: None 2022-09-03T06:45:30.437 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.437 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f7M22TzkENOgljVEQXJt9mz8BS8= 2022-09-03T06:45:30.438 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f7M22TzkENOgljVEQXJt9mz8BS8=', 'Content-Length': '0'} 2022-09-03T06:45:30.440 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2d46545813b2eec-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.441 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:30.442 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:30.443 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.444 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.445 DEBUG:boto:Method: HEAD 2022-09-03T06:45:30.446 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.446 DEBUG:boto:Data: 2022-09-03T06:45:30.448 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.448 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.449 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.449 DEBUG:boto:Params: {} 2022-09-03T06:45:30.450 DEBUG:boto:Token: None 2022-09-03T06:45:30.451 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.452 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NpnCtjTwzxOWdquHVCNiO+Bj3i4= 2022-09-03T06:45:30.453 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NpnCtjTwzxOWdquHVCNiO+Bj3i4=', 'Content-Length': '0'} 2022-09-03T06:45:30.456 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f6010f45a722c90f-006312f80a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.457 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:30.458 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:30.459 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.460 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.461 DEBUG:boto:Method: HEAD 2022-09-03T06:45:30.461 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.462 DEBUG:boto:Data: 2022-09-03T06:45:30.463 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.464 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.465 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.465 DEBUG:boto:Params: {} 2022-09-03T06:45:30.466 DEBUG:boto:Token: None 2022-09-03T06:45:30.467 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.468 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NpnCtjTwzxOWdquHVCNiO+Bj3i4= 2022-09-03T06:45:30.469 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NpnCtjTwzxOWdquHVCNiO+Bj3i4=', 'Content-Length': '0'} 2022-09-03T06:45:30.471 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001fe3edeb5aa2aaef-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.472 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:45:30.472 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:30.473 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:30.474 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.474 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.475 DEBUG:boto:Method: GET 2022-09-03T06:45:30.476 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.476 DEBUG:boto:Data: 2022-09-03T06:45:30.477 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.478 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.478 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.479 DEBUG:boto:Params: {} 2022-09-03T06:45:30.480 DEBUG:boto:Token: None 2022-09-03T06:45:30.481 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.482 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f7M22TzkENOgljVEQXJt9mz8BS8= 2022-09-03T06:45:30.483 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f7M22TzkENOgljVEQXJt9mz8BS8=', 'Content-Length': '0'} 2022-09-03T06:45:30.486 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000394f7542b9f9ec73-006312f80a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.530 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:30.531 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:30.532 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.533 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.534 DEBUG:boto:Method: GET 2022-09-03T06:45:30.535 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.535 DEBUG:boto:Data: 2022-09-03T06:45:30.536 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.537 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.538 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.539 DEBUG:boto:Params: {} 2022-09-03T06:45:30.540 DEBUG:boto:Token: None 2022-09-03T06:45:30.541 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:30.541 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f7M22TzkENOgljVEQXJt9mz8BS8= 2022-09-03T06:45:30.542 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f7M22TzkENOgljVEQXJt9mz8BS8=', 'Content-Length': '0'} 2022-09-03T06:45:30.545 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007ecf6fc6ce2aeed7-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.547 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:30.548 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:30.549 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:30.550 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:30.551 DEBUG:boto:Method: GET 2022-09-03T06:45:30.552 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:45:30.553 DEBUG:boto:Data: 2022-09-03T06:45:30.553 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.554 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.555 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.556 DEBUG:boto:Params: {} 2022-09-03T06:45:30.557 DEBUG:boto:Token: None 2022-09-03T06:45:30.558 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/ 2022-09-03T06:45:30.559 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q= 2022-09-03T06:45:30.560 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q=', 'Content-Length': '0'} 2022-09-03T06:45:30.563 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000091fbcb488bd13dbd-006312f80a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.564 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:30.565 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:30.565 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:30.566 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:30.567 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:30.568 DEBUG:boto:Method: GET 2022-09-03T06:45:30.568 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:45:30.569 DEBUG:boto:Data: 2022-09-03T06:45:30.570 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.571 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.571 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.572 DEBUG:boto:Params: {} 2022-09-03T06:45:30.573 DEBUG:boto:Token: None 2022-09-03T06:45:30.574 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/ 2022-09-03T06:45:30.575 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q= 2022-09-03T06:45:30.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q=', 'Content-Length': '0'} 2022-09-03T06:45:30.581 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003d5c223c62279d39-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.622 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:30.623 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:30.624 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:30.624 DEBUG:boto:Method: HEAD 2022-09-03T06:45:30.625 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:45:30.626 DEBUG:boto:Data: 2022-09-03T06:45:30.627 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.627 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.628 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.629 DEBUG:boto:Params: {} 2022-09-03T06:45:30.630 DEBUG:boto:Token: None 2022-09-03T06:45:30.631 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj85 2022-09-03T06:45:30.632 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/HQPQMB2giuPy4Nsrlg7i4PVdHA= 2022-09-03T06:45:30.634 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/HQPQMB2giuPy4Nsrlg7i4PVdHA=', 'Content-Length': '0'} 2022-09-03T06:45:30.637 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000090a58b41061ee3b3-006312f80a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.638 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:30.639 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:30.640 DEBUG:boto:Method: HEAD 2022-09-03T06:45:30.641 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:45:30.642 DEBUG:boto:Data: 2022-09-03T06:45:30.643 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.644 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.645 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.646 DEBUG:boto:Params: {} 2022-09-03T06:45:30.647 DEBUG:boto:Token: None 2022-09-03T06:45:30.648 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj85 2022-09-03T06:45:30.649 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/HQPQMB2giuPy4Nsrlg7i4PVdHA= 2022-09-03T06:45:30.650 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/HQPQMB2giuPy4Nsrlg7i4PVdHA=', 'Content-Length': '0'} 2022-09-03T06:45:30.652 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000054a56c541e41a9b3-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.653 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:45:30.654 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:30.655 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:30.656 DEBUG:boto:Method: GET 2022-09-03T06:45:30.657 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:45:30.657 DEBUG:boto:Data: 2022-09-03T06:45:30.658 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.659 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.659 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.660 DEBUG:boto:Params: {} 2022-09-03T06:45:30.661 DEBUG:boto:Token: None 2022-09-03T06:45:30.662 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj85 2022-09-03T06:45:30.663 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wmI7RUBlcN7UC/DL6yCs/bjxubQ= 2022-09-03T06:45:30.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wmI7RUBlcN7UC/DL6yCs/bjxubQ=', 'Content-Length': '0'} 2022-09-03T06:45:30.666 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000067d037f58ed4803-006312f80a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.667 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:30.668 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:30.669 DEBUG:boto:Method: GET 2022-09-03T06:45:30.670 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:45:30.671 DEBUG:boto:Data: 2022-09-03T06:45:30.672 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.673 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.673 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.674 DEBUG:boto:Params: {} 2022-09-03T06:45:30.675 DEBUG:boto:Token: None 2022-09-03T06:45:30.675 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj85 2022-09-03T06:45:30.676 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wmI7RUBlcN7UC/DL6yCs/bjxubQ= 2022-09-03T06:45:30.677 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wmI7RUBlcN7UC/DL6yCs/bjxubQ=', 'Content-Length': '0'} 2022-09-03T06:45:30.679 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e5bab8e3f499dbaf-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.680 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:30.681 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:30.682 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:30.683 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:30.684 DEBUG:boto:Method: GET 2022-09-03T06:45:30.685 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:45:30.686 DEBUG:boto:Data: 2022-09-03T06:45:30.687 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.688 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.689 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.690 DEBUG:boto:Params: {} 2022-09-03T06:45:30.691 DEBUG:boto:Token: None 2022-09-03T06:45:30.691 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/ 2022-09-03T06:45:30.692 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q= 2022-09-03T06:45:30.693 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q=', 'Content-Length': '0'} 2022-09-03T06:45:30.696 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000efa1b6350502646-006312f80a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.697 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:30.698 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:30.699 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:30.699 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:30.700 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:30.700 DEBUG:boto:Method: GET 2022-09-03T06:45:30.701 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:45:30.701 DEBUG:boto:Data: 2022-09-03T06:45:30.702 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.702 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.703 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.704 DEBUG:boto:Params: {} 2022-09-03T06:45:30.704 DEBUG:boto:Token: None 2022-09-03T06:45:30.705 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/ 2022-09-03T06:45:30.706 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q= 2022-09-03T06:45:30.707 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q=', 'Content-Length': '0'} 2022-09-03T06:45:30.711 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b99c12038457cb16-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.712 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:30.713 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:45:30.714 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:30.715 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:30.715 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.716 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.716 DEBUG:boto:Method: GET 2022-09-03T06:45:30.717 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.718 DEBUG:boto:Data: 2022-09-03T06:45:30.718 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.719 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.719 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.720 DEBUG:boto:Params: {} 2022-09-03T06:45:30.720 DEBUG:boto:Token: None 2022-09-03T06:45:30.721 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.722 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VAYLWNETnXKrLlJ8ZDPyb5Aw4Zo= 2022-09-03T06:45:30.723 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VAYLWNETnXKrLlJ8ZDPyb5Aw4Zo=', 'Content-Length': '0'} 2022-09-03T06:45:30.725 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca22d1213f17b65d-006312f80a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.726 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:30.727 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:30.727 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.728 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.728 DEBUG:boto:Method: GET 2022-09-03T06:45:30.729 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.729 DEBUG:boto:Data: 2022-09-03T06:45:30.730 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.731 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.732 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.732 DEBUG:boto:Params: {} 2022-09-03T06:45:30.733 DEBUG:boto:Token: None 2022-09-03T06:45:30.734 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.734 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VAYLWNETnXKrLlJ8ZDPyb5Aw4Zo= 2022-09-03T06:45:30.735 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VAYLWNETnXKrLlJ8ZDPyb5Aw4Zo=', 'Content-Length': '0'} 2022-09-03T06:45:30.738 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000791b3c8d71ddadde-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.739 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:30.739 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:30.740 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.740 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.741 DEBUG:boto:Method: HEAD 2022-09-03T06:45:30.741 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.742 DEBUG:boto:Data: 2022-09-03T06:45:30.743 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.743 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.744 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.744 DEBUG:boto:Params: {} 2022-09-03T06:45:30.745 DEBUG:boto:Token: None 2022-09-03T06:45:30.745 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.746 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ewUlEdSeiZ6YyKHg2UtgK99g2d4= 2022-09-03T06:45:30.747 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ewUlEdSeiZ6YyKHg2UtgK99g2d4=', 'Content-Length': '0'} 2022-09-03T06:45:30.749 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000036037b7ae5325d41-006312f80a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.750 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:30.751 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:30.752 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.752 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.753 DEBUG:boto:Method: HEAD 2022-09-03T06:45:30.753 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.754 DEBUG:boto:Data: 2022-09-03T06:45:30.755 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.755 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.756 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.757 DEBUG:boto:Params: {} 2022-09-03T06:45:30.758 DEBUG:boto:Token: None 2022-09-03T06:45:30.758 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.759 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ewUlEdSeiZ6YyKHg2UtgK99g2d4= 2022-09-03T06:45:30.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ewUlEdSeiZ6YyKHg2UtgK99g2d4=', 'Content-Length': '0'} 2022-09-03T06:45:30.762 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000006aabfc24d7bfc90-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.762 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:45:30.763 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:30.764 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:30.764 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.765 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.766 DEBUG:boto:Method: GET 2022-09-03T06:45:30.766 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.767 DEBUG:boto:Data: 2022-09-03T06:45:30.768 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.768 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.769 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.769 DEBUG:boto:Params: {} 2022-09-03T06:45:30.770 DEBUG:boto:Token: None 2022-09-03T06:45:30.771 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.771 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VAYLWNETnXKrLlJ8ZDPyb5Aw4Zo= 2022-09-03T06:45:30.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VAYLWNETnXKrLlJ8ZDPyb5Aw4Zo=', 'Content-Length': '0'} 2022-09-03T06:45:30.774 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000990515c4d59e8272-006312f80a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.775 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:30.776 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:30.776 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.777 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.777 DEBUG:boto:Method: GET 2022-09-03T06:45:30.778 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.778 DEBUG:boto:Data: 2022-09-03T06:45:30.779 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.780 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.780 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.781 DEBUG:boto:Params: {} 2022-09-03T06:45:30.781 DEBUG:boto:Token: None 2022-09-03T06:45:30.782 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:30.783 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VAYLWNETnXKrLlJ8ZDPyb5Aw4Zo= 2022-09-03T06:45:30.784 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VAYLWNETnXKrLlJ8ZDPyb5Aw4Zo=', 'Content-Length': '0'} 2022-09-03T06:45:30.786 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000064c663ffa68655f7-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.830 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:30.831 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:30.831 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:30.832 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:30.832 DEBUG:boto:Method: GET 2022-09-03T06:45:30.833 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:45:30.834 DEBUG:boto:Data: 2022-09-03T06:45:30.834 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.835 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.835 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.836 DEBUG:boto:Params: {} 2022-09-03T06:45:30.836 DEBUG:boto:Token: None 2022-09-03T06:45:30.837 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/ 2022-09-03T06:45:30.837 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q= 2022-09-03T06:45:30.838 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q=', 'Content-Length': '0'} 2022-09-03T06:45:30.841 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008174335be1e0fb46-006312f80a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.842 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:30.842 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:30.843 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:30.844 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:30.844 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:30.845 DEBUG:boto:Method: GET 2022-09-03T06:45:30.845 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:45:30.846 DEBUG:boto:Data: 2022-09-03T06:45:30.846 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.847 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.847 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.848 DEBUG:boto:Params: {} 2022-09-03T06:45:30.848 DEBUG:boto:Token: None 2022-09-03T06:45:30.849 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/ 2022-09-03T06:45:30.849 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q= 2022-09-03T06:45:30.850 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q=', 'Content-Length': '0'} 2022-09-03T06:45:30.854 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003667d5c65c6919fb-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.855 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:30.856 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:30.856 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:30.857 DEBUG:boto:Method: HEAD 2022-09-03T06:45:30.857 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:45:30.858 DEBUG:boto:Data: 2022-09-03T06:45:30.858 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.859 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.859 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.860 DEBUG:boto:Params: {} 2022-09-03T06:45:30.860 DEBUG:boto:Token: None 2022-09-03T06:45:30.861 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj87 2022-09-03T06:45:30.862 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X4eA9kBkaKHnB9JUXik5F5X9F+0= 2022-09-03T06:45:30.862 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X4eA9kBkaKHnB9JUXik5F5X9F+0=', 'Content-Length': '0'} 2022-09-03T06:45:30.864 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000581eeccfcc20451c-006312f80a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.865 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:30.866 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:30.866 DEBUG:boto:Method: HEAD 2022-09-03T06:45:30.867 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:45:30.867 DEBUG:boto:Data: 2022-09-03T06:45:30.868 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.868 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.869 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.869 DEBUG:boto:Params: {} 2022-09-03T06:45:30.870 DEBUG:boto:Token: None 2022-09-03T06:45:30.871 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj87 2022-09-03T06:45:30.871 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X4eA9kBkaKHnB9JUXik5F5X9F+0= 2022-09-03T06:45:30.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X4eA9kBkaKHnB9JUXik5F5X9F+0=', 'Content-Length': '0'} 2022-09-03T06:45:30.874 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006d8d0b7757a52fa8-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.875 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:45:30.875 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:30.876 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:30.877 DEBUG:boto:Method: GET 2022-09-03T06:45:30.877 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:45:30.878 DEBUG:boto:Data: 2022-09-03T06:45:30.879 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.879 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.880 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.880 DEBUG:boto:Params: {} 2022-09-03T06:45:30.881 DEBUG:boto:Token: None 2022-09-03T06:45:30.881 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj87 2022-09-03T06:45:30.882 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/YB6SQ31s0MQeYax+CU4CV0wbVs= 2022-09-03T06:45:30.883 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/YB6SQ31s0MQeYax+CU4CV0wbVs=', 'Content-Length': '0'} 2022-09-03T06:45:30.885 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001cca399fcdf1e76e-006312f80a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.886 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:30.887 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:30.887 DEBUG:boto:Method: GET 2022-09-03T06:45:30.888 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:45:30.889 DEBUG:boto:Data: 2022-09-03T06:45:30.889 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.890 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.890 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.891 DEBUG:boto:Params: {} 2022-09-03T06:45:30.891 DEBUG:boto:Token: None 2022-09-03T06:45:30.892 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj87 2022-09-03T06:45:30.892 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/YB6SQ31s0MQeYax+CU4CV0wbVs= 2022-09-03T06:45:30.893 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/YB6SQ31s0MQeYax+CU4CV0wbVs=', 'Content-Length': '0'} 2022-09-03T06:45:30.895 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ea1e74c1c29086b6-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.897 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:30.897 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:30.898 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:30.898 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:30.899 DEBUG:boto:Method: GET 2022-09-03T06:45:30.900 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:45:30.900 DEBUG:boto:Data: 2022-09-03T06:45:30.901 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.902 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.902 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.903 DEBUG:boto:Params: {} 2022-09-03T06:45:30.904 DEBUG:boto:Token: None 2022-09-03T06:45:30.904 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/ 2022-09-03T06:45:30.905 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q= 2022-09-03T06:45:30.906 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q=', 'Content-Length': '0'} 2022-09-03T06:45:30.909 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c1a16b6912a31e74-006312f80a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.910 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:30.911 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:30.912 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:30.912 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:30.913 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:30.914 DEBUG:boto:Method: GET 2022-09-03T06:45:30.914 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:45:30.915 DEBUG:boto:Data: 2022-09-03T06:45:30.915 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.916 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.917 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.918 DEBUG:boto:Params: {} 2022-09-03T06:45:30.919 DEBUG:boto:Token: None 2022-09-03T06:45:30.920 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/ 2022-09-03T06:45:30.920 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q= 2022-09-03T06:45:30.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8Q05HSKweHkwUIE3+cnniYIwU2Q=', 'Content-Length': '0'} 2022-09-03T06:45:30.925 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004ce394332ea04534-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.926 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:30.927 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:45:30.928 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:30.928 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:30.929 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:30.930 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:30.931 DEBUG:boto:Method: GET 2022-09-03T06:45:30.931 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:30.932 DEBUG:boto:Data: 2022-09-03T06:45:30.932 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.933 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.933 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.934 DEBUG:boto:Params: {} 2022-09-03T06:45:30.935 DEBUG:boto:Token: None 2022-09-03T06:45:30.936 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:30.937 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QyxGD3eUGuMICo9/9lm9owzwvig= 2022-09-03T06:45:30.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QyxGD3eUGuMICo9/9lm9owzwvig=', 'Content-Length': '0'} 2022-09-03T06:45:30.940 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004027f28cfcab4a83-006312f80a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.941 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:30.942 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:30.943 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:30.944 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:30.945 DEBUG:boto:Method: GET 2022-09-03T06:45:30.946 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:30.947 DEBUG:boto:Data: 2022-09-03T06:45:30.948 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.949 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.950 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.951 DEBUG:boto:Params: {} 2022-09-03T06:45:30.953 DEBUG:boto:Token: None 2022-09-03T06:45:30.954 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:30.955 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QyxGD3eUGuMICo9/9lm9owzwvig= 2022-09-03T06:45:30.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QyxGD3eUGuMICo9/9lm9owzwvig=', 'Content-Length': '0'} 2022-09-03T06:45:30.959 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000016a503ec7cb580bd-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.960 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:30.961 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:30.962 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:30.963 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:30.964 DEBUG:boto:Method: HEAD 2022-09-03T06:45:30.964 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:30.965 DEBUG:boto:Data: 2022-09-03T06:45:30.966 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.967 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.968 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.969 DEBUG:boto:Params: {} 2022-09-03T06:45:30.969 DEBUG:boto:Token: None 2022-09-03T06:45:30.970 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:30.971 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XbGFtlO3Uyd2WgWQ2+kqdvMJC0U= 2022-09-03T06:45:30.971 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XbGFtlO3Uyd2WgWQ2+kqdvMJC0U=', 'Content-Length': '0'} 2022-09-03T06:45:30.973 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065cf786798977517-006312f80a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.974 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:30.975 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:30.976 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:30.976 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:30.977 DEBUG:boto:Method: HEAD 2022-09-03T06:45:30.977 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:30.978 DEBUG:boto:Data: 2022-09-03T06:45:30.978 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.979 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:30.979 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.980 DEBUG:boto:Params: {} 2022-09-03T06:45:30.981 DEBUG:boto:Token: None 2022-09-03T06:45:30.981 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:30.982 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XbGFtlO3Uyd2WgWQ2+kqdvMJC0U= 2022-09-03T06:45:30.983 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XbGFtlO3Uyd2WgWQ2+kqdvMJC0U=', 'Content-Length': '0'} 2022-09-03T06:45:30.985 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a028fdd845c199e8-006312f80a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:30.986 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:45:30.987 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:30.988 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:30.989 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:30.989 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:30.990 DEBUG:boto:Method: GET 2022-09-03T06:45:30.990 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:30.991 DEBUG:boto:Data: 2022-09-03T06:45:30.991 DEBUG:boto:Headers: {} 2022-09-03T06:45:30.992 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:30.992 DEBUG:boto:Port: 8000 2022-09-03T06:45:30.993 DEBUG:boto:Params: {} 2022-09-03T06:45:30.994 DEBUG:boto:Token: None 2022-09-03T06:45:30.994 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:30 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:30.995 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QyxGD3eUGuMICo9/9lm9owzwvig= 2022-09-03T06:45:30.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QyxGD3eUGuMICo9/9lm9owzwvig=', 'Content-Length': '0'} 2022-09-03T06:45:31.000 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000819d38c1eb7fbcec-006312f80a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.002 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:31.002 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:31.003 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:31.004 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:31.004 DEBUG:boto:Method: GET 2022-09-03T06:45:31.005 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:31.005 DEBUG:boto:Data: 2022-09-03T06:45:31.006 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.007 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:31.007 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.008 DEBUG:boto:Params: {} 2022-09-03T06:45:31.009 DEBUG:boto:Token: None 2022-09-03T06:45:31.009 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:31.010 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eC4uUUpNsybGCBvZA63nPc9Ui58= 2022-09-03T06:45:31.010 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eC4uUUpNsybGCBvZA63nPc9Ui58=', 'Content-Length': '0'} 2022-09-03T06:45:31.013 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b218cc8c2be4d0d-006312f80b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.058 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:31.059 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:31.060 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:31.061 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:31.062 DEBUG:boto:Method: GET 2022-09-03T06:45:31.063 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:45:31.063 DEBUG:boto:Data: 2022-09-03T06:45:31.064 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.064 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:31.066 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.067 DEBUG:boto:Params: {} 2022-09-03T06:45:31.068 DEBUG:boto:Token: None 2022-09-03T06:45:31.068 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/ 2022-09-03T06:45:31.069 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M= 2022-09-03T06:45:31.070 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M=', 'Content-Length': '0'} 2022-09-03T06:45:31.073 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c123d600aae1743e-006312f80b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.074 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:31.076 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:31.077 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:31.078 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:31.079 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:31.081 DEBUG:boto:Method: GET 2022-09-03T06:45:31.082 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:45:31.083 DEBUG:boto:Data: 2022-09-03T06:45:31.084 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.085 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:31.086 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.088 DEBUG:boto:Params: {} 2022-09-03T06:45:31.089 DEBUG:boto:Token: None 2022-09-03T06:45:31.090 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/ 2022-09-03T06:45:31.091 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M= 2022-09-03T06:45:31.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M=', 'Content-Length': '0'} 2022-09-03T06:45:31.100 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000011e10af4d3a8ddcf-006312f80b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.102 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:31.103 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:31.104 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:31.105 DEBUG:boto:Method: HEAD 2022-09-03T06:45:31.106 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:45:31.107 DEBUG:boto:Data: 2022-09-03T06:45:31.108 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.110 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:31.111 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.112 DEBUG:boto:Params: {} 2022-09-03T06:45:31.113 DEBUG:boto:Token: None 2022-09-03T06:45:31.114 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj92 2022-09-03T06:45:31.115 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C8E4KdqJ/Oc6sRbjQ+K/YJNj7po= 2022-09-03T06:45:31.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C8E4KdqJ/Oc6sRbjQ+K/YJNj7po=', 'Content-Length': '0'} 2022-09-03T06:45:31.120 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e672aa85ca6b599a-006312f80b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.122 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:31.123 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:31.124 DEBUG:boto:Method: HEAD 2022-09-03T06:45:31.125 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:45:31.126 DEBUG:boto:Data: 2022-09-03T06:45:31.127 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.128 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:31.129 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.130 DEBUG:boto:Params: {} 2022-09-03T06:45:31.131 DEBUG:boto:Token: None 2022-09-03T06:45:31.132 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj92 2022-09-03T06:45:31.133 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C8E4KdqJ/Oc6sRbjQ+K/YJNj7po= 2022-09-03T06:45:31.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C8E4KdqJ/Oc6sRbjQ+K/YJNj7po=', 'Content-Length': '0'} 2022-09-03T06:45:31.138 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ef38cae336409621-006312f80b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.139 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:45:31.140 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:31.141 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:31.143 DEBUG:boto:Method: GET 2022-09-03T06:45:31.145 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:45:31.146 DEBUG:boto:Data: 2022-09-03T06:45:31.147 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.148 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:31.149 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.150 DEBUG:boto:Params: {} 2022-09-03T06:45:31.151 DEBUG:boto:Token: None 2022-09-03T06:45:31.152 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj92 2022-09-03T06:45:31.153 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WoyA5sSGARkQ5g3EVkotUUIQ7rs= 2022-09-03T06:45:31.155 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WoyA5sSGARkQ5g3EVkotUUIQ7rs=', 'Content-Length': '0'} 2022-09-03T06:45:31.159 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca4b87f1a069404c-006312f80b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.160 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:31.161 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:31.162 DEBUG:boto:Method: GET 2022-09-03T06:45:31.163 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:45:31.164 DEBUG:boto:Data: 2022-09-03T06:45:31.167 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.168 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:31.169 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.171 DEBUG:boto:Params: {} 2022-09-03T06:45:31.172 DEBUG:boto:Token: None 2022-09-03T06:45:31.172 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj92 2022-09-03T06:45:31.173 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WoyA5sSGARkQ5g3EVkotUUIQ7rs= 2022-09-03T06:45:31.174 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WoyA5sSGARkQ5g3EVkotUUIQ7rs=', 'Content-Length': '0'} 2022-09-03T06:45:31.179 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b1fcf9e5e70df42-006312f80b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.180 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:31.181 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:31.181 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:31.183 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:31.184 DEBUG:boto:Method: GET 2022-09-03T06:45:31.185 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:45:31.186 DEBUG:boto:Data: 2022-09-03T06:45:31.187 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.188 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:31.189 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.190 DEBUG:boto:Params: {} 2022-09-03T06:45:31.191 DEBUG:boto:Token: None 2022-09-03T06:45:31.192 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/ 2022-09-03T06:45:31.194 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M= 2022-09-03T06:45:31.195 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M=', 'Content-Length': '0'} 2022-09-03T06:45:31.198 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000db9a29d35dfac361-006312f80b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.199 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:31.201 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:31.202 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:31.203 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:31.204 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:31.205 DEBUG:boto:Method: GET 2022-09-03T06:45:31.206 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:45:31.207 DEBUG:boto:Data: 2022-09-03T06:45:31.208 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.209 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:31.210 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.211 DEBUG:boto:Params: {} 2022-09-03T06:45:31.212 DEBUG:boto:Token: None 2022-09-03T06:45:31.213 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/ 2022-09-03T06:45:31.214 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M= 2022-09-03T06:45:31.215 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M=', 'Content-Length': '0'} 2022-09-03T06:45:31.224 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000041e56fd7f48ee4fa-006312f80b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.225 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:31.226 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:45:31.227 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:31.228 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:31.228 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.229 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.230 DEBUG:boto:Method: GET 2022-09-03T06:45:31.230 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.231 DEBUG:boto:Data: 2022-09-03T06:45:31.231 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.232 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:31.233 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.233 DEBUG:boto:Params: {} 2022-09-03T06:45:31.234 DEBUG:boto:Token: None 2022-09-03T06:45:31.234 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.235 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gTm61tW9vVbLzHHBC/UmVXeWscc= 2022-09-03T06:45:31.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gTm61tW9vVbLzHHBC/UmVXeWscc=', 'Content-Length': '0'} 2022-09-03T06:45:31.240 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d71f67c641320007-006312f80b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.282 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:31.283 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:31.284 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.285 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.287 DEBUG:boto:Method: GET 2022-09-03T06:45:31.288 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.289 DEBUG:boto:Data: 2022-09-03T06:45:31.290 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.291 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:31.292 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.292 DEBUG:boto:Params: {} 2022-09-03T06:45:31.294 DEBUG:boto:Token: None 2022-09-03T06:45:31.295 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.296 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gTm61tW9vVbLzHHBC/UmVXeWscc= 2022-09-03T06:45:31.297 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gTm61tW9vVbLzHHBC/UmVXeWscc=', 'Content-Length': '0'} 2022-09-03T06:45:31.301 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c31d089fc1d1753a-006312f80b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.302 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:31.302 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:31.304 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.304 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.305 DEBUG:boto:Method: HEAD 2022-09-03T06:45:31.306 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.306 DEBUG:boto:Data: 2022-09-03T06:45:31.307 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.307 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:31.308 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.309 DEBUG:boto:Params: {} 2022-09-03T06:45:31.309 DEBUG:boto:Token: None 2022-09-03T06:45:31.310 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.310 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ih/rHEvpjCHoOypYCQP6v2k5kj8= 2022-09-03T06:45:31.311 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ih/rHEvpjCHoOypYCQP6v2k5kj8=', 'Content-Length': '0'} 2022-09-03T06:45:31.313 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005d5d43fcb6c1847f-006312f80b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.314 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:31.315 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:31.315 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.316 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.316 DEBUG:boto:Method: HEAD 2022-09-03T06:45:31.317 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.317 DEBUG:boto:Data: 2022-09-03T06:45:31.318 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.318 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:31.319 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.319 DEBUG:boto:Params: {} 2022-09-03T06:45:31.320 DEBUG:boto:Token: None 2022-09-03T06:45:31.320 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.321 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ih/rHEvpjCHoOypYCQP6v2k5kj8= 2022-09-03T06:45:31.322 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ih/rHEvpjCHoOypYCQP6v2k5kj8=', 'Content-Length': '0'} 2022-09-03T06:45:31.325 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000044ced6263b60a628-006312f80b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.326 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:45:31.327 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:31.327 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:31.328 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.329 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.329 DEBUG:boto:Method: GET 2022-09-03T06:45:31.330 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.331 DEBUG:boto:Data: 2022-09-03T06:45:31.331 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.332 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:31.332 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.333 DEBUG:boto:Params: {} 2022-09-03T06:45:31.333 DEBUG:boto:Token: None 2022-09-03T06:45:31.334 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.335 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gTm61tW9vVbLzHHBC/UmVXeWscc= 2022-09-03T06:45:31.335 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gTm61tW9vVbLzHHBC/UmVXeWscc=', 'Content-Length': '0'} 2022-09-03T06:45:31.338 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eabdcbd1fa2d239f-006312f80b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.339 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:31.339 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:31.340 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.340 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.341 DEBUG:boto:Method: GET 2022-09-03T06:45:31.342 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.343 DEBUG:boto:Data: 2022-09-03T06:45:31.344 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.344 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:31.345 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.346 DEBUG:boto:Params: {} 2022-09-03T06:45:31.347 DEBUG:boto:Token: None 2022-09-03T06:45:31.349 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:31.350 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gTm61tW9vVbLzHHBC/UmVXeWscc= 2022-09-03T06:45:31.351 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gTm61tW9vVbLzHHBC/UmVXeWscc=', 'Content-Length': '0'} 2022-09-03T06:45:31.354 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007aaa56f21fbcffaa-006312f80b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.355 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:31.356 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:31.357 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:31.358 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:31.359 DEBUG:boto:Method: GET 2022-09-03T06:45:31.360 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:45:31.361 DEBUG:boto:Data: 2022-09-03T06:45:31.361 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.362 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:31.362 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.364 DEBUG:boto:Params: {} 2022-09-03T06:45:31.364 DEBUG:boto:Token: None 2022-09-03T06:45:31.365 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/ 2022-09-03T06:45:31.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M= 2022-09-03T06:45:31.366 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M=', 'Content-Length': '0'} 2022-09-03T06:45:31.369 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000074547468d7ff6fc-006312f80b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.370 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:31.371 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:31.371 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:31.372 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:31.372 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:31.373 DEBUG:boto:Method: GET 2022-09-03T06:45:31.373 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:45:31.374 DEBUG:boto:Data: 2022-09-03T06:45:31.374 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.375 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:31.375 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.376 DEBUG:boto:Params: {} 2022-09-03T06:45:31.377 DEBUG:boto:Token: None 2022-09-03T06:45:31.377 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/ 2022-09-03T06:45:31.378 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M= 2022-09-03T06:45:31.379 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M=', 'Content-Length': '0'} 2022-09-03T06:45:31.383 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008f417537e7e9e8ac-006312f80b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.430 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:31.431 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:31.433 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:31.433 DEBUG:boto:Method: HEAD 2022-09-03T06:45:31.435 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:45:31.435 DEBUG:boto:Data: 2022-09-03T06:45:31.436 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.436 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:31.437 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.437 DEBUG:boto:Params: {} 2022-09-03T06:45:31.438 DEBUG:boto:Token: None 2022-09-03T06:45:31.439 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj95 2022-09-03T06:45:31.439 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JyrsD/3Dw6sWKNQBN/uxtcKIxls= 2022-09-03T06:45:31.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JyrsD/3Dw6sWKNQBN/uxtcKIxls=', 'Content-Length': '0'} 2022-09-03T06:45:31.442 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d5aa7bef80b6cd62-006312f80b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.443 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:31.444 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:31.445 DEBUG:boto:Method: HEAD 2022-09-03T06:45:31.446 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:45:31.447 DEBUG:boto:Data: 2022-09-03T06:45:31.448 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.449 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:31.450 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.451 DEBUG:boto:Params: {} 2022-09-03T06:45:31.452 DEBUG:boto:Token: None 2022-09-03T06:45:31.453 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj95 2022-09-03T06:45:31.454 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JyrsD/3Dw6sWKNQBN/uxtcKIxls= 2022-09-03T06:45:31.454 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JyrsD/3Dw6sWKNQBN/uxtcKIxls=', 'Content-Length': '0'} 2022-09-03T06:45:31.457 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ed3b6cba7bd4a2e2-006312f80b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.458 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:45:31.459 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:31.460 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:31.461 DEBUG:boto:Method: GET 2022-09-03T06:45:31.462 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:45:31.463 DEBUG:boto:Data: 2022-09-03T06:45:31.465 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.466 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:31.466 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.467 DEBUG:boto:Params: {} 2022-09-03T06:45:31.468 DEBUG:boto:Token: None 2022-09-03T06:45:31.468 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj95 2022-09-03T06:45:31.469 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8uwIuxJKL8UQ9gjgyy33ffyAXZs= 2022-09-03T06:45:31.470 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8uwIuxJKL8UQ9gjgyy33ffyAXZs=', 'Content-Length': '0'} 2022-09-03T06:45:31.472 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c11c7be765af97a-006312f80b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.473 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:31.474 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:31.474 DEBUG:boto:Method: GET 2022-09-03T06:45:31.475 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:45:31.476 DEBUG:boto:Data: 2022-09-03T06:45:31.476 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.477 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:31.477 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.478 DEBUG:boto:Params: {} 2022-09-03T06:45:31.479 DEBUG:boto:Token: None 2022-09-03T06:45:31.479 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj95 2022-09-03T06:45:31.480 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8uwIuxJKL8UQ9gjgyy33ffyAXZs= 2022-09-03T06:45:31.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8uwIuxJKL8UQ9gjgyy33ffyAXZs=', 'Content-Length': '0'} 2022-09-03T06:45:31.483 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000011b3032c385ec285-006312f80b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.484 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:31.485 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:31.485 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:31.486 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:31.486 DEBUG:boto:Method: GET 2022-09-03T06:45:31.487 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:45:31.488 DEBUG:boto:Data: 2022-09-03T06:45:31.488 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.489 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:31.489 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.490 DEBUG:boto:Params: {} 2022-09-03T06:45:31.491 DEBUG:boto:Token: None 2022-09-03T06:45:31.491 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/ 2022-09-03T06:45:31.492 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M= 2022-09-03T06:45:31.493 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M=', 'Content-Length': '0'} 2022-09-03T06:45:31.496 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000680079cc4e784be3-006312f80b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.497 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:31.497 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:31.498 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:31.499 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:31.499 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:31.501 DEBUG:boto:Method: GET 2022-09-03T06:45:31.501 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:45:31.502 DEBUG:boto:Data: 2022-09-03T06:45:31.503 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.504 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:31.504 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.505 DEBUG:boto:Params: {} 2022-09-03T06:45:31.506 DEBUG:boto:Token: None 2022-09-03T06:45:31.506 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/ 2022-09-03T06:45:31.507 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M= 2022-09-03T06:45:31.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M=', 'Content-Length': '0'} 2022-09-03T06:45:31.512 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aecf4a4f1af4a855-006312f80b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.513 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:31.515 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-75 zones={test-zone2, test-zone1} 2022-09-03T06:45:31.515 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-75 zones={test-zone3, test-zone1} 2022-09-03T06:45:31.516 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:31.517 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:31.517 DEBUG:boto:Method: HEAD 2022-09-03T06:45:31.518 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:45:31.519 DEBUG:boto:Data: 2022-09-03T06:45:31.519 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.520 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:31.521 DEBUG:boto:Port: 8001 2022-09-03T06:45:31.522 DEBUG:boto:Params: {} 2022-09-03T06:45:31.522 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T06:45:31.523 DEBUG:boto:Token: None 2022-09-03T06:45:31.524 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/ 2022-09-03T06:45:31.525 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Tz9VjhGu87SsVqGuRirLvHHp0ww= 2022-09-03T06:45:31.525 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Tz9VjhGu87SsVqGuRirLvHHp0ww=', 'Content-Length': '0'} 2022-09-03T06:45:31.529 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '25'), ('X-RGW-Bytes-Used', '75'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000060c068bfed39f156-006312f80b-1851-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.530 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:31.530 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:31.531 DEBUG:boto:Method: HEAD 2022-09-03T06:45:31.532 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:45:31.532 DEBUG:boto:Data: 2022-09-03T06:45:31.533 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.533 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:31.534 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.534 DEBUG:boto:Params: {} 2022-09-03T06:45:31.535 DEBUG:boto:Token: None 2022-09-03T06:45:31.536 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/ 2022-09-03T06:45:31.537 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Tz9VjhGu87SsVqGuRirLvHHp0ww= 2022-09-03T06:45:31.537 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Tz9VjhGu87SsVqGuRirLvHHp0ww=', 'Content-Length': '0'} 2022-09-03T06:45:31.542 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '25'), ('X-RGW-Bytes-Used', '75'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000070ac3f17b614ff1b-006312f80b-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.543 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:45:31.545 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:31.546 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:31.547 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:31.548 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:31.550 DEBUG:boto:Method: GET 2022-09-03T06:45:31.551 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:31.551 DEBUG:boto:Data: 2022-09-03T06:45:31.552 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.552 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:31.553 DEBUG:boto:Port: 8001 2022-09-03T06:45:31.553 DEBUG:boto:Params: {} 2022-09-03T06:45:31.554 DEBUG:boto:Token: None 2022-09-03T06:45:31.555 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/?versions 2022-09-03T06:45:31.555 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:On5weOpTanCNDwsTs7oRHUR2GB8= 2022-09-03T06:45:31.556 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:On5weOpTanCNDwsTs7oRHUR2GB8=', 'Content-Length': '0'} 2022-09-03T06:45:31.561 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003e3983e56b1f53f6-006312f80b-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.564 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:31.568 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:45:31.569 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:45:31.570 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2022-09-03T06:45:31.571 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2022-09-03T06:45:31.572 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:45:31.572 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:45:31.573 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:45:31.574 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:45:31.575 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2022-09-03T06:45:31.575 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:45:31.576 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:45:31.576 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:45:31.577 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:45:31.577 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:45:31.578 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:45:31.578 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:45:31.579 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:45:31.579 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2022-09-03T06:45:31.580 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:45:31.580 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:45:31.581 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:45:31.581 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:45:31.582 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2022-09-03T06:45:31.582 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:45:31.582 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:45:31.583 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:45:31.584 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:31.584 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:31.585 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:31.585 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:31.586 DEBUG:boto:Method: GET 2022-09-03T06:45:31.586 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:31.587 DEBUG:boto:Data: 2022-09-03T06:45:31.587 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.588 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:31.588 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.589 DEBUG:boto:Params: {} 2022-09-03T06:45:31.589 DEBUG:boto:Token: None 2022-09-03T06:45:31.590 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/?versions 2022-09-03T06:45:31.591 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:On5weOpTanCNDwsTs7oRHUR2GB8= 2022-09-03T06:45:31.591 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:On5weOpTanCNDwsTs7oRHUR2GB8=', 'Content-Length': '0'} 2022-09-03T06:45:31.596 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aab9296a99ba3497-006312f80b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.596 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:31.599 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:45:31.599 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:45:31.600 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2022-09-03T06:45:31.601 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2022-09-03T06:45:31.602 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:45:31.602 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:45:31.603 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:45:31.604 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:45:31.605 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2022-09-03T06:45:31.606 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:45:31.607 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:45:31.608 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:45:31.609 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:45:31.610 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:45:31.611 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:45:31.612 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:45:31.612 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:45:31.613 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2022-09-03T06:45:31.623 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:45:31.624 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:45:31.625 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:45:31.625 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:45:31.626 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2022-09-03T06:45:31.626 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:45:31.627 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:45:31.628 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:31.628 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:31.629 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:31.629 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:31.630 DEBUG:boto:Method: GET 2022-09-03T06:45:31.630 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:31.631 DEBUG:boto:Data: 2022-09-03T06:45:31.631 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.632 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:31.632 DEBUG:boto:Port: 8001 2022-09-03T06:45:31.633 DEBUG:boto:Params: {} 2022-09-03T06:45:31.634 DEBUG:boto:Token: None 2022-09-03T06:45:31.634 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/?versions 2022-09-03T06:45:31.635 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:On5weOpTanCNDwsTs7oRHUR2GB8= 2022-09-03T06:45:31.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:On5weOpTanCNDwsTs7oRHUR2GB8=', 'Content-Length': '0'} 2022-09-03T06:45:31.640 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000111d507872a73f89-006312f80b-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.641 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:31.645 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:31.646 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:31.647 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:31.648 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:31.649 DEBUG:boto:Method: GET 2022-09-03T06:45:31.649 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:31.650 DEBUG:boto:Data: 2022-09-03T06:45:31.650 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.651 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:31.652 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.652 DEBUG:boto:Params: {} 2022-09-03T06:45:31.653 DEBUG:boto:Token: None 2022-09-03T06:45:31.653 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/?versions 2022-09-03T06:45:31.654 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:On5weOpTanCNDwsTs7oRHUR2GB8= 2022-09-03T06:45:31.655 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:On5weOpTanCNDwsTs7oRHUR2GB8=', 'Content-Length': '0'} 2022-09-03T06:45:31.660 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ec005cd6e7a09f5c-006312f80b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.662 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:31.666 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:45:31.667 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:31.668 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:31.668 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.669 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.670 DEBUG:boto:Method: GET 2022-09-03T06:45:31.671 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.672 DEBUG:boto:Data: 2022-09-03T06:45:31.673 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.674 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:31.675 DEBUG:boto:Port: 8001 2022-09-03T06:45:31.676 DEBUG:boto:Params: {} 2022-09-03T06:45:31.677 DEBUG:boto:Token: None 2022-09-03T06:45:31.678 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.679 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rHOeQLH2hQA+dTYjwED9h5DC540= 2022-09-03T06:45:31.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rHOeQLH2hQA+dTYjwED9h5DC540=', 'Content-Length': '0'} 2022-09-03T06:45:31.683 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003d3f37b90dcaca3e-006312f80b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.684 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:31.685 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:31.685 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.686 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.686 DEBUG:boto:Method: GET 2022-09-03T06:45:31.687 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.687 DEBUG:boto:Data: 2022-09-03T06:45:31.688 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.689 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:31.690 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.691 DEBUG:boto:Params: {} 2022-09-03T06:45:31.691 DEBUG:boto:Token: None 2022-09-03T06:45:31.692 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.692 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rHOeQLH2hQA+dTYjwED9h5DC540= 2022-09-03T06:45:31.693 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rHOeQLH2hQA+dTYjwED9h5DC540=', 'Content-Length': '0'} 2022-09-03T06:45:31.695 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000002558e44f4887157-006312f80b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.696 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:31.697 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:31.697 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.698 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.698 DEBUG:boto:Method: HEAD 2022-09-03T06:45:31.699 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.699 DEBUG:boto:Data: 2022-09-03T06:45:31.700 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.701 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:31.701 DEBUG:boto:Port: 8001 2022-09-03T06:45:31.702 DEBUG:boto:Params: {} 2022-09-03T06:45:31.703 DEBUG:boto:Token: None 2022-09-03T06:45:31.704 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.704 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/5pJIsNf3A0izugm7Z7zYj2hoI4= 2022-09-03T06:45:31.705 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/5pJIsNf3A0izugm7Z7zYj2hoI4=', 'Content-Length': '0'} 2022-09-03T06:45:31.707 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004bd82943899acc67-006312f80b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.708 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:31.709 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:31.709 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.710 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.710 DEBUG:boto:Method: HEAD 2022-09-03T06:45:31.711 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.711 DEBUG:boto:Data: 2022-09-03T06:45:31.712 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.713 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:31.713 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.714 DEBUG:boto:Params: {} 2022-09-03T06:45:31.714 DEBUG:boto:Token: None 2022-09-03T06:45:31.715 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.715 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/5pJIsNf3A0izugm7Z7zYj2hoI4= 2022-09-03T06:45:31.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/5pJIsNf3A0izugm7Z7zYj2hoI4=', 'Content-Length': '0'} 2022-09-03T06:45:31.718 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000608bef9c0411c25e-006312f80b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.719 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:45:31.719 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:31.720 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:31.721 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.721 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.722 DEBUG:boto:Method: GET 2022-09-03T06:45:31.723 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.723 DEBUG:boto:Data: 2022-09-03T06:45:31.724 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.724 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:31.725 DEBUG:boto:Port: 8001 2022-09-03T06:45:31.725 DEBUG:boto:Params: {} 2022-09-03T06:45:31.726 DEBUG:boto:Token: None 2022-09-03T06:45:31.727 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.727 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rHOeQLH2hQA+dTYjwED9h5DC540= 2022-09-03T06:45:31.728 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rHOeQLH2hQA+dTYjwED9h5DC540=', 'Content-Length': '0'} 2022-09-03T06:45:31.730 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cac593d937c82834-006312f80b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.731 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:31.732 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:31.732 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.733 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.733 DEBUG:boto:Method: GET 2022-09-03T06:45:31.734 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.734 DEBUG:boto:Data: 2022-09-03T06:45:31.735 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.735 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:31.736 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.736 DEBUG:boto:Params: {} 2022-09-03T06:45:31.737 DEBUG:boto:Token: None 2022-09-03T06:45:31.738 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:31.738 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rHOeQLH2hQA+dTYjwED9h5DC540= 2022-09-03T06:45:31.739 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rHOeQLH2hQA+dTYjwED9h5DC540=', 'Content-Length': '0'} 2022-09-03T06:45:31.742 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000025bcd7e18b0f75b0-006312f80b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.743 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:31.743 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:31.744 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:31.744 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:31.745 DEBUG:boto:Method: GET 2022-09-03T06:45:31.745 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:45:31.746 DEBUG:boto:Data: 2022-09-03T06:45:31.746 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.747 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:31.748 DEBUG:boto:Port: 8001 2022-09-03T06:45:31.748 DEBUG:boto:Params: {} 2022-09-03T06:45:31.749 DEBUG:boto:Token: None 2022-09-03T06:45:31.749 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/ 2022-09-03T06:45:31.750 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M= 2022-09-03T06:45:31.751 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M=', 'Content-Length': '0'} 2022-09-03T06:45:31.754 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000857844291fa7034c-006312f80b-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.799 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:31.801 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:31.802 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:31.803 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:31.804 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:31.805 DEBUG:boto:Method: GET 2022-09-03T06:45:31.806 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:45:31.808 DEBUG:boto:Data: 2022-09-03T06:45:31.809 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.810 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:31.812 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.813 DEBUG:boto:Params: {} 2022-09-03T06:45:31.814 DEBUG:boto:Token: None 2022-09-03T06:45:31.815 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/ 2022-09-03T06:45:31.819 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M= 2022-09-03T06:45:31.821 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M=', 'Content-Length': '0'} 2022-09-03T06:45:31.826 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002ce3a6f97840a1f6-006312f80b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.827 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:31.828 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:31.828 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:31.829 DEBUG:boto:Method: HEAD 2022-09-03T06:45:31.829 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:45:31.830 DEBUG:boto:Data: 2022-09-03T06:45:31.831 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.831 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:31.832 DEBUG:boto:Port: 8001 2022-09-03T06:45:31.832 DEBUG:boto:Params: {} 2022-09-03T06:45:31.833 DEBUG:boto:Token: None 2022-09-03T06:45:31.834 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj11 2022-09-03T06:45:31.834 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J3yL2WkaLLNsQkcq4lNLYzqGlkM= 2022-09-03T06:45:31.835 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J3yL2WkaLLNsQkcq4lNLYzqGlkM=', 'Content-Length': '0'} 2022-09-03T06:45:31.837 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000028aa41f2e0f9237-006312f80b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.838 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:31.839 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:31.839 DEBUG:boto:Method: HEAD 2022-09-03T06:45:31.840 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:45:31.840 DEBUG:boto:Data: 2022-09-03T06:45:31.841 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.842 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:31.842 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.843 DEBUG:boto:Params: {} 2022-09-03T06:45:31.844 DEBUG:boto:Token: None 2022-09-03T06:45:31.844 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj11 2022-09-03T06:45:31.845 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J3yL2WkaLLNsQkcq4lNLYzqGlkM= 2022-09-03T06:45:31.846 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J3yL2WkaLLNsQkcq4lNLYzqGlkM=', 'Content-Length': '0'} 2022-09-03T06:45:31.850 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003da11418bdd95cbd-006312f80b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.852 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:45:31.853 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:31.854 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:31.855 DEBUG:boto:Method: GET 2022-09-03T06:45:31.856 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:45:31.857 DEBUG:boto:Data: 2022-09-03T06:45:31.858 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.860 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:31.861 DEBUG:boto:Port: 8001 2022-09-03T06:45:31.862 DEBUG:boto:Params: {} 2022-09-03T06:45:31.863 DEBUG:boto:Token: None 2022-09-03T06:45:31.864 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj11 2022-09-03T06:45:31.865 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+mM9Ry9ykYcf3mJQWCOF6vU2+t8= 2022-09-03T06:45:31.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+mM9Ry9ykYcf3mJQWCOF6vU2+t8=', 'Content-Length': '0'} 2022-09-03T06:45:31.869 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074b5077b5f2a1c27-006312f80b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.871 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:31.872 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:31.873 DEBUG:boto:Method: GET 2022-09-03T06:45:31.874 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:45:31.875 DEBUG:boto:Data: 2022-09-03T06:45:31.877 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.877 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:31.878 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.879 DEBUG:boto:Params: {} 2022-09-03T06:45:31.881 DEBUG:boto:Token: None 2022-09-03T06:45:31.882 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj11 2022-09-03T06:45:31.882 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+mM9Ry9ykYcf3mJQWCOF6vU2+t8= 2022-09-03T06:45:31.883 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+mM9Ry9ykYcf3mJQWCOF6vU2+t8=', 'Content-Length': '0'} 2022-09-03T06:45:31.885 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000666b8c3023d4c72c-006312f80b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.887 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:31.887 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:31.888 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:31.889 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:31.889 DEBUG:boto:Method: GET 2022-09-03T06:45:31.890 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:45:31.890 DEBUG:boto:Data: 2022-09-03T06:45:31.891 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.892 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:31.892 DEBUG:boto:Port: 8001 2022-09-03T06:45:31.893 DEBUG:boto:Params: {} 2022-09-03T06:45:31.894 DEBUG:boto:Token: None 2022-09-03T06:45:31.894 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/ 2022-09-03T06:45:31.895 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M= 2022-09-03T06:45:31.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M=', 'Content-Length': '0'} 2022-09-03T06:45:31.899 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b017cfea16499974-006312f80b-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.904 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:31.906 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:31.907 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:31.908 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:31.910 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:31.911 DEBUG:boto:Method: GET 2022-09-03T06:45:31.913 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:45:31.914 DEBUG:boto:Data: 2022-09-03T06:45:31.915 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.916 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:31.917 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.918 DEBUG:boto:Params: {} 2022-09-03T06:45:31.918 DEBUG:boto:Token: None 2022-09-03T06:45:31.919 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/ 2022-09-03T06:45:31.920 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M= 2022-09-03T06:45:31.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2pNBnVNtH36zt8k2iDtdquoN85M=', 'Content-Length': '0'} 2022-09-03T06:45:31.924 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f3100b9378939e97-006312f80b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.925 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:31.927 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:45:31.928 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:31.928 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:31.929 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:31.929 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:31.930 DEBUG:boto:Method: GET 2022-09-03T06:45:31.931 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:31.931 DEBUG:boto:Data: 2022-09-03T06:45:31.932 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.933 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:31.933 DEBUG:boto:Port: 8001 2022-09-03T06:45:31.934 DEBUG:boto:Params: {} 2022-09-03T06:45:31.935 DEBUG:boto:Token: None 2022-09-03T06:45:31.936 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:31.937 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gSd5CMYoOIyANipDeE9fldTd170= 2022-09-03T06:45:31.938 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gSd5CMYoOIyANipDeE9fldTd170=', 'Content-Length': '0'} 2022-09-03T06:45:31.941 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b604244f3bc9f55f-006312f80b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:31.942 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:31.943 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:31.943 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:31.944 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:31.945 DEBUG:boto:Method: GET 2022-09-03T06:45:31.945 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:31.946 DEBUG:boto:Data: 2022-09-03T06:45:31.947 DEBUG:boto:Headers: {} 2022-09-03T06:45:31.947 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:31.948 DEBUG:boto:Port: 8000 2022-09-03T06:45:31.948 DEBUG:boto:Params: {} 2022-09-03T06:45:31.949 DEBUG:boto:Token: None 2022-09-03T06:45:31.950 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:31 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:31.950 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gSd5CMYoOIyANipDeE9fldTd170= 2022-09-03T06:45:31.951 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gSd5CMYoOIyANipDeE9fldTd170=', 'Content-Length': '0'} 2022-09-03T06:45:31.954 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000033765b33f0e2b11d-006312f80b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.006 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:32.007 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:32.008 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:32.009 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:32.010 DEBUG:boto:Method: HEAD 2022-09-03T06:45:32.011 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:32.012 DEBUG:boto:Data: 2022-09-03T06:45:32.013 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.014 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:32.015 DEBUG:boto:Port: 8001 2022-09-03T06:45:32.016 DEBUG:boto:Params: {} 2022-09-03T06:45:32.017 DEBUG:boto:Token: None 2022-09-03T06:45:32.018 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:32.019 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6Pl56pm6A/dv4blTmPrYrednfsk= 2022-09-03T06:45:32.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6Pl56pm6A/dv4blTmPrYrednfsk=', 'Content-Length': '0'} 2022-09-03T06:45:32.023 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cd8e761851a00feb-006312f80c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.024 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:32.025 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:32.026 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:32.027 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:32.028 DEBUG:boto:Method: HEAD 2022-09-03T06:45:32.029 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:32.030 DEBUG:boto:Data: 2022-09-03T06:45:32.031 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.032 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:32.033 DEBUG:boto:Port: 8000 2022-09-03T06:45:32.034 DEBUG:boto:Params: {} 2022-09-03T06:45:32.035 DEBUG:boto:Token: None 2022-09-03T06:45:32.035 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:32.036 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6Pl56pm6A/dv4blTmPrYrednfsk= 2022-09-03T06:45:32.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6Pl56pm6A/dv4blTmPrYrednfsk=', 'Content-Length': '0'} 2022-09-03T06:45:32.038 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c547aeaeb532e3b-006312f80c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.039 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:45:32.040 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:32.040 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:32.041 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:32.041 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:32.042 DEBUG:boto:Method: GET 2022-09-03T06:45:32.043 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:32.043 DEBUG:boto:Data: 2022-09-03T06:45:32.044 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.044 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:32.045 DEBUG:boto:Port: 8001 2022-09-03T06:45:32.045 DEBUG:boto:Params: {} 2022-09-03T06:45:32.046 DEBUG:boto:Token: None 2022-09-03T06:45:32.047 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:32.047 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yd/urNfAVA5xDX1mIT4nweEJpBc= 2022-09-03T06:45:32.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yd/urNfAVA5xDX1mIT4nweEJpBc=', 'Content-Length': '0'} 2022-09-03T06:45:32.050 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e581d0a9c55c96d-006312f80c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.094 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:32.095 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:32.096 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:32.096 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:32.097 DEBUG:boto:Method: GET 2022-09-03T06:45:32.098 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:32.099 DEBUG:boto:Data: 2022-09-03T06:45:32.100 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.100 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:32.102 DEBUG:boto:Port: 8000 2022-09-03T06:45:32.103 DEBUG:boto:Params: {} 2022-09-03T06:45:32.105 DEBUG:boto:Token: None 2022-09-03T06:45:32.106 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:32.107 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yd/urNfAVA5xDX1mIT4nweEJpBc= 2022-09-03T06:45:32.108 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yd/urNfAVA5xDX1mIT4nweEJpBc=', 'Content-Length': '0'} 2022-09-03T06:45:32.111 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000033c4979d1069e2c7-006312f80c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.113 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:32.115 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:32.115 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:32.116 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:32.117 DEBUG:boto:Method: GET 2022-09-03T06:45:32.117 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:45:32.118 DEBUG:boto:Data: 2022-09-03T06:45:32.118 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.119 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:32.120 DEBUG:boto:Port: 8001 2022-09-03T06:45:32.120 DEBUG:boto:Params: {} 2022-09-03T06:45:32.121 DEBUG:boto:Token: None 2022-09-03T06:45:32.121 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/ 2022-09-03T06:45:32.122 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ= 2022-09-03T06:45:32.123 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ=', 'Content-Length': '0'} 2022-09-03T06:45:32.129 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000004ee91b90188112f-006312f80c-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.131 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:32.133 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:32.134 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:32.135 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:32.136 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:32.138 DEBUG:boto:Method: GET 2022-09-03T06:45:32.139 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:45:32.140 DEBUG:boto:Data: 2022-09-03T06:45:32.141 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.142 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:32.143 DEBUG:boto:Port: 8000 2022-09-03T06:45:32.144 DEBUG:boto:Params: {} 2022-09-03T06:45:32.145 DEBUG:boto:Token: None 2022-09-03T06:45:32.146 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/ 2022-09-03T06:45:32.146 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ= 2022-09-03T06:45:32.147 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ=', 'Content-Length': '0'} 2022-09-03T06:45:32.152 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006fc25e2a4460b2aa-006312f80c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.155 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:32.156 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:32.157 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:32.159 DEBUG:boto:Method: HEAD 2022-09-03T06:45:32.159 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:45:32.160 DEBUG:boto:Data: 2022-09-03T06:45:32.160 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.161 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:32.161 DEBUG:boto:Port: 8001 2022-09-03T06:45:32.162 DEBUG:boto:Params: {} 2022-09-03T06:45:32.163 DEBUG:boto:Token: None 2022-09-03T06:45:32.164 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj115 2022-09-03T06:45:32.165 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AdldibDJlP1fGVEjezyzI/OEZAM= 2022-09-03T06:45:32.166 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AdldibDJlP1fGVEjezyzI/OEZAM=', 'Content-Length': '0'} 2022-09-03T06:45:32.169 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001580a9b88e0ad089-006312f80c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.171 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:32.172 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:32.173 DEBUG:boto:Method: HEAD 2022-09-03T06:45:32.173 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:45:32.174 DEBUG:boto:Data: 2022-09-03T06:45:32.175 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.175 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:32.176 DEBUG:boto:Port: 8000 2022-09-03T06:45:32.176 DEBUG:boto:Params: {} 2022-09-03T06:45:32.177 DEBUG:boto:Token: None 2022-09-03T06:45:32.178 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj115 2022-09-03T06:45:32.180 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AdldibDJlP1fGVEjezyzI/OEZAM= 2022-09-03T06:45:32.181 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AdldibDJlP1fGVEjezyzI/OEZAM=', 'Content-Length': '0'} 2022-09-03T06:45:32.184 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f55846c117bd0495-006312f80c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.186 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:45:32.187 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:32.188 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:32.189 DEBUG:boto:Method: GET 2022-09-03T06:45:32.190 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:45:32.191 DEBUG:boto:Data: 2022-09-03T06:45:32.192 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.192 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:32.193 DEBUG:boto:Port: 8001 2022-09-03T06:45:32.194 DEBUG:boto:Params: {} 2022-09-03T06:45:32.194 DEBUG:boto:Token: None 2022-09-03T06:45:32.195 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj115 2022-09-03T06:45:32.195 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SbHM2P9k59EOX2r+BZ64ZxlN+94= 2022-09-03T06:45:32.196 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SbHM2P9k59EOX2r+BZ64ZxlN+94=', 'Content-Length': '0'} 2022-09-03T06:45:32.200 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ab7d5ea5b9aec96e-006312f80c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.201 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:32.202 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:32.203 DEBUG:boto:Method: GET 2022-09-03T06:45:32.204 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:45:32.205 DEBUG:boto:Data: 2022-09-03T06:45:32.206 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.207 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:32.208 DEBUG:boto:Port: 8000 2022-09-03T06:45:32.209 DEBUG:boto:Params: {} 2022-09-03T06:45:32.210 DEBUG:boto:Token: None 2022-09-03T06:45:32.211 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj115 2022-09-03T06:45:32.213 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SbHM2P9k59EOX2r+BZ64ZxlN+94= 2022-09-03T06:45:32.214 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SbHM2P9k59EOX2r+BZ64ZxlN+94=', 'Content-Length': '0'} 2022-09-03T06:45:32.219 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d1104aafc4f10dcf-006312f80c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.266 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:32.267 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:32.268 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:32.269 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:32.274 DEBUG:boto:Method: GET 2022-09-03T06:45:32.276 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:45:32.277 DEBUG:boto:Data: 2022-09-03T06:45:32.278 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.279 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:32.280 DEBUG:boto:Port: 8001 2022-09-03T06:45:32.281 DEBUG:boto:Params: {} 2022-09-03T06:45:32.282 DEBUG:boto:Token: None 2022-09-03T06:45:32.283 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/ 2022-09-03T06:45:32.285 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ= 2022-09-03T06:45:32.286 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ=', 'Content-Length': '0'} 2022-09-03T06:45:32.289 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003a28954a942105fc-006312f80c-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.291 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:32.292 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:32.293 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:32.294 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:32.295 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:32.296 DEBUG:boto:Method: GET 2022-09-03T06:45:32.298 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:45:32.299 DEBUG:boto:Data: 2022-09-03T06:45:32.300 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.301 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:32.302 DEBUG:boto:Port: 8000 2022-09-03T06:45:32.303 DEBUG:boto:Params: {} 2022-09-03T06:45:32.304 DEBUG:boto:Token: None 2022-09-03T06:45:32.305 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/ 2022-09-03T06:45:32.306 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ= 2022-09-03T06:45:32.307 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ=', 'Content-Length': '0'} 2022-09-03T06:45:32.313 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005bbd4cdc630e71ef-006312f80c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.315 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:32.318 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:45:32.320 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:32.321 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:32.322 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.323 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.324 DEBUG:boto:Method: GET 2022-09-03T06:45:32.325 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.326 DEBUG:boto:Data: 2022-09-03T06:45:32.327 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.329 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:32.331 DEBUG:boto:Port: 8001 2022-09-03T06:45:32.332 DEBUG:boto:Params: {} 2022-09-03T06:45:32.333 DEBUG:boto:Token: None 2022-09-03T06:45:32.334 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.335 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qyBHeiE+4TiH4+6q/nfPDaV3rbY= 2022-09-03T06:45:32.337 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qyBHeiE+4TiH4+6q/nfPDaV3rbY=', 'Content-Length': '0'} 2022-09-03T06:45:32.341 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000608ffd3a9835cdb3-006312f80c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.343 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:32.344 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:32.345 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.346 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.347 DEBUG:boto:Method: GET 2022-09-03T06:45:32.348 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.349 DEBUG:boto:Data: 2022-09-03T06:45:32.350 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.352 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:32.352 DEBUG:boto:Port: 8000 2022-09-03T06:45:32.353 DEBUG:boto:Params: {} 2022-09-03T06:45:32.355 DEBUG:boto:Token: None 2022-09-03T06:45:32.356 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.357 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qyBHeiE+4TiH4+6q/nfPDaV3rbY= 2022-09-03T06:45:32.358 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qyBHeiE+4TiH4+6q/nfPDaV3rbY=', 'Content-Length': '0'} 2022-09-03T06:45:32.360 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000039d8e9e6b8659068-006312f80c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.361 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:32.362 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:32.363 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.363 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.364 DEBUG:boto:Method: HEAD 2022-09-03T06:45:32.364 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.365 DEBUG:boto:Data: 2022-09-03T06:45:32.366 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.367 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:32.368 DEBUG:boto:Port: 8001 2022-09-03T06:45:32.369 DEBUG:boto:Params: {} 2022-09-03T06:45:32.370 DEBUG:boto:Token: None 2022-09-03T06:45:32.370 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.371 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QGPZJzmZfTnPIXKyRoUrr8WIAqc= 2022-09-03T06:45:32.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QGPZJzmZfTnPIXKyRoUrr8WIAqc=', 'Content-Length': '0'} 2022-09-03T06:45:32.374 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1c13902220933af-006312f80c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.375 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:32.375 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:32.376 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.376 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.377 DEBUG:boto:Method: HEAD 2022-09-03T06:45:32.377 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.378 DEBUG:boto:Data: 2022-09-03T06:45:32.379 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.379 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:32.380 DEBUG:boto:Port: 8000 2022-09-03T06:45:32.380 DEBUG:boto:Params: {} 2022-09-03T06:45:32.381 DEBUG:boto:Token: None 2022-09-03T06:45:32.381 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.382 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QGPZJzmZfTnPIXKyRoUrr8WIAqc= 2022-09-03T06:45:32.383 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QGPZJzmZfTnPIXKyRoUrr8WIAqc=', 'Content-Length': '0'} 2022-09-03T06:45:32.385 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000005de341ce6e5ed67-006312f80c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.386 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:45:32.387 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:32.387 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:32.388 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.388 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.389 DEBUG:boto:Method: GET 2022-09-03T06:45:32.390 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.390 DEBUG:boto:Data: 2022-09-03T06:45:32.391 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.391 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:32.392 DEBUG:boto:Port: 8001 2022-09-03T06:45:32.393 DEBUG:boto:Params: {} 2022-09-03T06:45:32.394 DEBUG:boto:Token: None 2022-09-03T06:45:32.395 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.396 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qyBHeiE+4TiH4+6q/nfPDaV3rbY= 2022-09-03T06:45:32.397 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qyBHeiE+4TiH4+6q/nfPDaV3rbY=', 'Content-Length': '0'} 2022-09-03T06:45:32.400 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000042a491ba7a25da5d-006312f80c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.402 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:32.403 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:32.404 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.406 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.407 DEBUG:boto:Method: GET 2022-09-03T06:45:32.408 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.409 DEBUG:boto:Data: 2022-09-03T06:45:32.410 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.411 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:32.412 DEBUG:boto:Port: 8000 2022-09-03T06:45:32.423 DEBUG:boto:Params: {} 2022-09-03T06:45:32.424 DEBUG:boto:Token: None 2022-09-03T06:45:32.425 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:32.426 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qyBHeiE+4TiH4+6q/nfPDaV3rbY= 2022-09-03T06:45:32.427 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qyBHeiE+4TiH4+6q/nfPDaV3rbY=', 'Content-Length': '0'} 2022-09-03T06:45:32.431 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008f29600a0e1d3f47-006312f80c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.432 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:32.433 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:32.434 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:32.435 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:32.436 DEBUG:boto:Method: GET 2022-09-03T06:45:32.437 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:45:32.438 DEBUG:boto:Data: 2022-09-03T06:45:32.439 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.440 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:32.441 DEBUG:boto:Port: 8001 2022-09-03T06:45:32.442 DEBUG:boto:Params: {} 2022-09-03T06:45:32.444 DEBUG:boto:Token: None 2022-09-03T06:45:32.445 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/ 2022-09-03T06:45:32.447 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ= 2022-09-03T06:45:32.448 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ=', 'Content-Length': '0'} 2022-09-03T06:45:32.451 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000be8126cca832eedb-006312f80c-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.453 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:32.454 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:32.455 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:32.456 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:32.457 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:32.458 DEBUG:boto:Method: GET 2022-09-03T06:45:32.459 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:45:32.460 DEBUG:boto:Data: 2022-09-03T06:45:32.461 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.462 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:32.463 DEBUG:boto:Port: 8000 2022-09-03T06:45:32.463 DEBUG:boto:Params: {} 2022-09-03T06:45:32.464 DEBUG:boto:Token: None 2022-09-03T06:45:32.465 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/ 2022-09-03T06:45:32.466 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ= 2022-09-03T06:45:32.467 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ=', 'Content-Length': '0'} 2022-09-03T06:45:32.471 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009ee4b3eaf62fb174-006312f80c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.472 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:32.473 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:32.475 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:32.476 DEBUG:boto:Method: HEAD 2022-09-03T06:45:32.477 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:45:32.478 DEBUG:boto:Data: 2022-09-03T06:45:32.478 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.479 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:32.480 DEBUG:boto:Port: 8001 2022-09-03T06:45:32.480 DEBUG:boto:Params: {} 2022-09-03T06:45:32.481 DEBUG:boto:Token: None 2022-09-03T06:45:32.482 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj116 2022-09-03T06:45:32.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6P4p8NL4BJEihVQtmi2bu2BDUP0= 2022-09-03T06:45:32.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6P4p8NL4BJEihVQtmi2bu2BDUP0=', 'Content-Length': '0'} 2022-09-03T06:45:32.487 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7f6d8631e189337-006312f80c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.488 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:32.489 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:32.490 DEBUG:boto:Method: HEAD 2022-09-03T06:45:32.491 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:45:32.492 DEBUG:boto:Data: 2022-09-03T06:45:32.493 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.494 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:32.495 DEBUG:boto:Port: 8000 2022-09-03T06:45:32.496 DEBUG:boto:Params: {} 2022-09-03T06:45:32.497 DEBUG:boto:Token: None 2022-09-03T06:45:32.498 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj116 2022-09-03T06:45:32.499 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6P4p8NL4BJEihVQtmi2bu2BDUP0= 2022-09-03T06:45:32.500 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6P4p8NL4BJEihVQtmi2bu2BDUP0=', 'Content-Length': '0'} 2022-09-03T06:45:32.503 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026a0ef3084eba070-006312f80c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.505 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:45:32.506 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:32.507 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:32.508 DEBUG:boto:Method: GET 2022-09-03T06:45:32.508 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:45:32.509 DEBUG:boto:Data: 2022-09-03T06:45:32.510 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.510 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:32.511 DEBUG:boto:Port: 8001 2022-09-03T06:45:32.511 DEBUG:boto:Params: {} 2022-09-03T06:45:32.512 DEBUG:boto:Token: None 2022-09-03T06:45:32.513 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj116 2022-09-03T06:45:32.514 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6ZW1/SO959mCCZRIuXJ6Ypqbxn8= 2022-09-03T06:45:32.515 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6ZW1/SO959mCCZRIuXJ6Ypqbxn8=', 'Content-Length': '0'} 2022-09-03T06:45:32.517 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002eafe621019c1bb9-006312f80c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.562 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:32.564 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:32.565 DEBUG:boto:Method: GET 2022-09-03T06:45:32.565 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:45:32.566 DEBUG:boto:Data: 2022-09-03T06:45:32.567 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.567 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:32.568 DEBUG:boto:Port: 8000 2022-09-03T06:45:32.569 DEBUG:boto:Params: {} 2022-09-03T06:45:32.570 DEBUG:boto:Token: None 2022-09-03T06:45:32.570 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj116 2022-09-03T06:45:32.571 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6ZW1/SO959mCCZRIuXJ6Ypqbxn8= 2022-09-03T06:45:32.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6ZW1/SO959mCCZRIuXJ6Ypqbxn8=', 'Content-Length': '0'} 2022-09-03T06:45:32.575 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002fc971276522a03b-006312f80c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.577 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:32.578 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:32.580 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:32.581 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:32.582 DEBUG:boto:Method: GET 2022-09-03T06:45:32.583 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:45:32.583 DEBUG:boto:Data: 2022-09-03T06:45:32.585 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.586 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:32.587 DEBUG:boto:Port: 8001 2022-09-03T06:45:32.587 DEBUG:boto:Params: {} 2022-09-03T06:45:32.588 DEBUG:boto:Token: None 2022-09-03T06:45:32.589 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/ 2022-09-03T06:45:32.589 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ= 2022-09-03T06:45:32.590 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ=', 'Content-Length': '0'} 2022-09-03T06:45:32.593 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a0e49685fb1c854f-006312f80c-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.595 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:32.597 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:32.598 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:32.599 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:32.600 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:32.601 DEBUG:boto:Method: GET 2022-09-03T06:45:32.602 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:45:32.603 DEBUG:boto:Data: 2022-09-03T06:45:32.604 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.605 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:32.606 DEBUG:boto:Port: 8000 2022-09-03T06:45:32.607 DEBUG:boto:Params: {} 2022-09-03T06:45:32.608 DEBUG:boto:Token: None 2022-09-03T06:45:32.609 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/ 2022-09-03T06:45:32.609 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ= 2022-09-03T06:45:32.610 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ=', 'Content-Length': '0'} 2022-09-03T06:45:32.614 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003a1e51aa508e99b2-006312f80c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.662 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:32.663 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:45:32.664 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:32.664 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:32.665 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.666 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.667 DEBUG:boto:Method: GET 2022-09-03T06:45:32.668 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.668 DEBUG:boto:Data: 2022-09-03T06:45:32.669 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.670 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:32.670 DEBUG:boto:Port: 8001 2022-09-03T06:45:32.671 DEBUG:boto:Params: {} 2022-09-03T06:45:32.671 DEBUG:boto:Token: None 2022-09-03T06:45:32.673 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.673 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BJeEuNPjioR16AxoOnMtE1oTcjw= 2022-09-03T06:45:32.674 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BJeEuNPjioR16AxoOnMtE1oTcjw=', 'Content-Length': '0'} 2022-09-03T06:45:32.676 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000094969c2305716f12-006312f80c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.677 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:32.678 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:32.678 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.679 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.680 DEBUG:boto:Method: GET 2022-09-03T06:45:32.680 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.681 DEBUG:boto:Data: 2022-09-03T06:45:32.681 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.683 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:32.683 DEBUG:boto:Port: 8000 2022-09-03T06:45:32.684 DEBUG:boto:Params: {} 2022-09-03T06:45:32.685 DEBUG:boto:Token: None 2022-09-03T06:45:32.686 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.686 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BJeEuNPjioR16AxoOnMtE1oTcjw= 2022-09-03T06:45:32.687 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BJeEuNPjioR16AxoOnMtE1oTcjw=', 'Content-Length': '0'} 2022-09-03T06:45:32.689 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca1a0bc7bf91ddf5-006312f80c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.690 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:32.691 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:32.691 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.692 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.693 DEBUG:boto:Method: HEAD 2022-09-03T06:45:32.694 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.694 DEBUG:boto:Data: 2022-09-03T06:45:32.695 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.696 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:32.696 DEBUG:boto:Port: 8001 2022-09-03T06:45:32.697 DEBUG:boto:Params: {} 2022-09-03T06:45:32.697 DEBUG:boto:Token: None 2022-09-03T06:45:32.698 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.699 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:db9Nmv3BMX2QemXAbyte1z6wHYg= 2022-09-03T06:45:32.699 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:db9Nmv3BMX2QemXAbyte1z6wHYg=', 'Content-Length': '0'} 2022-09-03T06:45:32.701 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d07363774d444d59-006312f80c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.702 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:32.703 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:32.704 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.704 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.705 DEBUG:boto:Method: HEAD 2022-09-03T06:45:32.706 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.706 DEBUG:boto:Data: 2022-09-03T06:45:32.707 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.707 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:32.708 DEBUG:boto:Port: 8000 2022-09-03T06:45:32.709 DEBUG:boto:Params: {} 2022-09-03T06:45:32.709 DEBUG:boto:Token: None 2022-09-03T06:45:32.710 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.711 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:db9Nmv3BMX2QemXAbyte1z6wHYg= 2022-09-03T06:45:32.711 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:db9Nmv3BMX2QemXAbyte1z6wHYg=', 'Content-Length': '0'} 2022-09-03T06:45:32.714 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fd39f8205cbe30c6-006312f80c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.715 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:45:32.716 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:32.716 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:32.717 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.717 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.718 DEBUG:boto:Method: GET 2022-09-03T06:45:32.719 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.719 DEBUG:boto:Data: 2022-09-03T06:45:32.720 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.720 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:32.721 DEBUG:boto:Port: 8001 2022-09-03T06:45:32.721 DEBUG:boto:Params: {} 2022-09-03T06:45:32.722 DEBUG:boto:Token: None 2022-09-03T06:45:32.723 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.723 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BJeEuNPjioR16AxoOnMtE1oTcjw= 2022-09-03T06:45:32.724 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BJeEuNPjioR16AxoOnMtE1oTcjw=', 'Content-Length': '0'} 2022-09-03T06:45:32.727 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000071145eab1c569aa9-006312f80c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.728 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:32.728 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:32.729 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.729 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.730 DEBUG:boto:Method: GET 2022-09-03T06:45:32.730 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.731 DEBUG:boto:Data: 2022-09-03T06:45:32.732 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.732 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:32.733 DEBUG:boto:Port: 8000 2022-09-03T06:45:32.733 DEBUG:boto:Params: {} 2022-09-03T06:45:32.734 DEBUG:boto:Token: None 2022-09-03T06:45:32.735 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:32.735 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BJeEuNPjioR16AxoOnMtE1oTcjw= 2022-09-03T06:45:32.736 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BJeEuNPjioR16AxoOnMtE1oTcjw=', 'Content-Length': '0'} 2022-09-03T06:45:32.738 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009071e408889ea3a9-006312f80c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.739 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:32.740 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:32.740 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:32.741 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:32.741 DEBUG:boto:Method: GET 2022-09-03T06:45:32.742 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:45:32.743 DEBUG:boto:Data: 2022-09-03T06:45:32.743 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.744 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:32.744 DEBUG:boto:Port: 8001 2022-09-03T06:45:32.745 DEBUG:boto:Params: {} 2022-09-03T06:45:32.745 DEBUG:boto:Token: None 2022-09-03T06:45:32.746 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/ 2022-09-03T06:45:32.747 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ= 2022-09-03T06:45:32.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ=', 'Content-Length': '0'} 2022-09-03T06:45:32.751 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000008815e1ba88d90d2-006312f80c-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.751 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:32.752 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:32.753 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:32.753 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:32.754 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:32.755 DEBUG:boto:Method: GET 2022-09-03T06:45:32.755 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:45:32.756 DEBUG:boto:Data: 2022-09-03T06:45:32.756 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.757 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:32.757 DEBUG:boto:Port: 8000 2022-09-03T06:45:32.758 DEBUG:boto:Params: {} 2022-09-03T06:45:32.758 DEBUG:boto:Token: None 2022-09-03T06:45:32.759 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/ 2022-09-03T06:45:32.760 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ= 2022-09-03T06:45:32.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ=', 'Content-Length': '0'} 2022-09-03T06:45:32.764 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000211d0d36f0295a83-006312f80c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.765 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:32.766 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:32.767 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:32.767 DEBUG:boto:Method: HEAD 2022-09-03T06:45:32.768 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:45:32.768 DEBUG:boto:Data: 2022-09-03T06:45:32.769 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.769 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:32.770 DEBUG:boto:Port: 8001 2022-09-03T06:45:32.770 DEBUG:boto:Params: {} 2022-09-03T06:45:32.771 DEBUG:boto:Token: None 2022-09-03T06:45:32.772 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj145 2022-09-03T06:45:32.772 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H2BE3N0Hv3FwIWDWqczB169SrQ4= 2022-09-03T06:45:32.773 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H2BE3N0Hv3FwIWDWqczB169SrQ4=', 'Content-Length': '0'} 2022-09-03T06:45:32.775 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018c8914bcc543f0a-006312f80c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.776 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:32.776 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:32.777 DEBUG:boto:Method: HEAD 2022-09-03T06:45:32.777 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:45:32.778 DEBUG:boto:Data: 2022-09-03T06:45:32.778 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.779 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:32.779 DEBUG:boto:Port: 8000 2022-09-03T06:45:32.780 DEBUG:boto:Params: {} 2022-09-03T06:45:32.780 DEBUG:boto:Token: None 2022-09-03T06:45:32.781 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj145 2022-09-03T06:45:32.782 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H2BE3N0Hv3FwIWDWqczB169SrQ4= 2022-09-03T06:45:32.782 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H2BE3N0Hv3FwIWDWqczB169SrQ4=', 'Content-Length': '0'} 2022-09-03T06:45:32.785 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000000cd5dafd24e25b3-006312f80c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.785 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:45:32.786 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:32.787 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:32.787 DEBUG:boto:Method: GET 2022-09-03T06:45:32.788 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:45:32.788 DEBUG:boto:Data: 2022-09-03T06:45:32.789 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.789 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:32.790 DEBUG:boto:Port: 8001 2022-09-03T06:45:32.790 DEBUG:boto:Params: {} 2022-09-03T06:45:32.791 DEBUG:boto:Token: None 2022-09-03T06:45:32.792 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj145 2022-09-03T06:45:32.792 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VYlpwLeGrDaxAJiPJ5979Jlfbjc= 2022-09-03T06:45:32.793 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VYlpwLeGrDaxAJiPJ5979Jlfbjc=', 'Content-Length': '0'} 2022-09-03T06:45:32.795 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e2a51756feadfec3-006312f80c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.843 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:32.844 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:32.844 DEBUG:boto:Method: GET 2022-09-03T06:45:32.845 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:45:32.846 DEBUG:boto:Data: 2022-09-03T06:45:32.846 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.847 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:32.847 DEBUG:boto:Port: 8000 2022-09-03T06:45:32.848 DEBUG:boto:Params: {} 2022-09-03T06:45:32.849 DEBUG:boto:Token: None 2022-09-03T06:45:32.849 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj145 2022-09-03T06:45:32.851 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VYlpwLeGrDaxAJiPJ5979Jlfbjc= 2022-09-03T06:45:32.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VYlpwLeGrDaxAJiPJ5979Jlfbjc=', 'Content-Length': '0'} 2022-09-03T06:45:32.854 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ddb581f5f7ff7e1-006312f80c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.855 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:32.856 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:32.856 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:32.857 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:32.858 DEBUG:boto:Method: GET 2022-09-03T06:45:32.858 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:45:32.859 DEBUG:boto:Data: 2022-09-03T06:45:32.859 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.861 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:32.862 DEBUG:boto:Port: 8001 2022-09-03T06:45:32.863 DEBUG:boto:Params: {} 2022-09-03T06:45:32.865 DEBUG:boto:Token: None 2022-09-03T06:45:32.866 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/ 2022-09-03T06:45:32.867 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ= 2022-09-03T06:45:32.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ=', 'Content-Length': '0'} 2022-09-03T06:45:32.872 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e279c1534523156e-006312f80c-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.873 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:32.874 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:32.875 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:32.876 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:32.877 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:32.877 DEBUG:boto:Method: GET 2022-09-03T06:45:32.878 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:45:32.878 DEBUG:boto:Data: 2022-09-03T06:45:32.879 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.880 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:32.880 DEBUG:boto:Port: 8000 2022-09-03T06:45:32.881 DEBUG:boto:Params: {} 2022-09-03T06:45:32.881 DEBUG:boto:Token: None 2022-09-03T06:45:32.882 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/ 2022-09-03T06:45:32.883 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ= 2022-09-03T06:45:32.883 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rc4z38FTRlqCk9fS0pp+rkUteDQ=', 'Content-Length': '0'} 2022-09-03T06:45:32.887 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a7b81373ad2c731c-006312f80c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.888 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:32.889 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:45:32.890 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:32.890 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:32.891 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.892 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.892 DEBUG:boto:Method: GET 2022-09-03T06:45:32.893 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.894 DEBUG:boto:Data: 2022-09-03T06:45:32.895 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.895 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:32.896 DEBUG:boto:Port: 8001 2022-09-03T06:45:32.897 DEBUG:boto:Params: {} 2022-09-03T06:45:32.898 DEBUG:boto:Token: None 2022-09-03T06:45:32.899 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.900 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EzfD8g2ZaYIJ1NXahA3GqPKBSaQ= 2022-09-03T06:45:32.901 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EzfD8g2ZaYIJ1NXahA3GqPKBSaQ=', 'Content-Length': '0'} 2022-09-03T06:45:32.904 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000e7cb93b82d74e35-006312f80c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.905 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:32.906 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:32.907 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.908 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.909 DEBUG:boto:Method: GET 2022-09-03T06:45:32.910 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.910 DEBUG:boto:Data: 2022-09-03T06:45:32.911 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.912 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:32.912 DEBUG:boto:Port: 8000 2022-09-03T06:45:32.913 DEBUG:boto:Params: {} 2022-09-03T06:45:32.914 DEBUG:boto:Token: None 2022-09-03T06:45:32.915 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.915 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EzfD8g2ZaYIJ1NXahA3GqPKBSaQ= 2022-09-03T06:45:32.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EzfD8g2ZaYIJ1NXahA3GqPKBSaQ=', 'Content-Length': '0'} 2022-09-03T06:45:32.918 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000265c3c9549aa6cb6-006312f80c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.919 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:32.920 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:32.920 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.921 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.921 DEBUG:boto:Method: HEAD 2022-09-03T06:45:32.922 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.922 DEBUG:boto:Data: 2022-09-03T06:45:32.923 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.924 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:32.924 DEBUG:boto:Port: 8001 2022-09-03T06:45:32.925 DEBUG:boto:Params: {} 2022-09-03T06:45:32.926 DEBUG:boto:Token: None 2022-09-03T06:45:32.926 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.927 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ItLXYKwD6Yp9aSP+7xtY4FL6oHQ= 2022-09-03T06:45:32.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ItLXYKwD6Yp9aSP+7xtY4FL6oHQ=', 'Content-Length': '0'} 2022-09-03T06:45:32.930 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000060d419e80692e0a0-006312f80c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.931 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:32.931 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:32.932 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.932 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.933 DEBUG:boto:Method: HEAD 2022-09-03T06:45:32.933 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.934 DEBUG:boto:Data: 2022-09-03T06:45:32.934 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.935 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:32.935 DEBUG:boto:Port: 8000 2022-09-03T06:45:32.936 DEBUG:boto:Params: {} 2022-09-03T06:45:32.937 DEBUG:boto:Token: None 2022-09-03T06:45:32.938 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.938 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ItLXYKwD6Yp9aSP+7xtY4FL6oHQ= 2022-09-03T06:45:32.939 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ItLXYKwD6Yp9aSP+7xtY4FL6oHQ=', 'Content-Length': '0'} 2022-09-03T06:45:32.941 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000032c6d9a5eedb935d-006312f80c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.942 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:45:32.943 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:32.943 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:32.944 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.944 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.945 DEBUG:boto:Method: GET 2022-09-03T06:45:32.945 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.946 DEBUG:boto:Data: 2022-09-03T06:45:32.947 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.947 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:32.948 DEBUG:boto:Port: 8001 2022-09-03T06:45:32.948 DEBUG:boto:Params: {} 2022-09-03T06:45:32.949 DEBUG:boto:Token: None 2022-09-03T06:45:32.949 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.950 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EzfD8g2ZaYIJ1NXahA3GqPKBSaQ= 2022-09-03T06:45:32.951 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EzfD8g2ZaYIJ1NXahA3GqPKBSaQ=', 'Content-Length': '0'} 2022-09-03T06:45:32.953 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000024effc817b95eb1e-006312f80c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:32.954 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:32.954 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:32.955 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.956 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.956 DEBUG:boto:Method: GET 2022-09-03T06:45:32.957 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.957 DEBUG:boto:Data: 2022-09-03T06:45:32.958 DEBUG:boto:Headers: {} 2022-09-03T06:45:32.958 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:32.959 DEBUG:boto:Port: 8000 2022-09-03T06:45:32.959 DEBUG:boto:Params: {} 2022-09-03T06:45:32.960 DEBUG:boto:Token: None 2022-09-03T06:45:32.961 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:32 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:32.961 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EzfD8g2ZaYIJ1NXahA3GqPKBSaQ= 2022-09-03T06:45:32.962 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EzfD8g2ZaYIJ1NXahA3GqPKBSaQ=', 'Content-Length': '0'} 2022-09-03T06:45:32.964 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a1017b724cad276-006312f80c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.006 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:33.008 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:33.008 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:33.010 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:33.011 DEBUG:boto:Method: GET 2022-09-03T06:45:33.012 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:45:33.012 DEBUG:boto:Data: 2022-09-03T06:45:33.013 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.013 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.014 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.015 DEBUG:boto:Params: {} 2022-09-03T06:45:33.015 DEBUG:boto:Token: None 2022-09-03T06:45:33.016 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/ 2022-09-03T06:45:33.016 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk= 2022-09-03T06:45:33.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk=', 'Content-Length': '0'} 2022-09-03T06:45:33.021 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cce6d3f9d93596ee-006312f80d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.021 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:33.022 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:33.023 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:33.024 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:33.024 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:33.025 DEBUG:boto:Method: GET 2022-09-03T06:45:33.026 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:45:33.027 DEBUG:boto:Data: 2022-09-03T06:45:33.027 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.028 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.028 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.029 DEBUG:boto:Params: {} 2022-09-03T06:45:33.030 DEBUG:boto:Token: None 2022-09-03T06:45:33.030 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/ 2022-09-03T06:45:33.031 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk= 2022-09-03T06:45:33.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk=', 'Content-Length': '0'} 2022-09-03T06:45:33.036 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000057ec426cd04a5b4a-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.037 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:33.038 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:33.038 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:33.039 DEBUG:boto:Method: HEAD 2022-09-03T06:45:33.040 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:45:33.040 DEBUG:boto:Data: 2022-09-03T06:45:33.041 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.041 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.042 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.042 DEBUG:boto:Params: {} 2022-09-03T06:45:33.043 DEBUG:boto:Token: None 2022-09-03T06:45:33.043 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj17 2022-09-03T06:45:33.044 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0ZGHpWGJ0//LLSbHKSZFqKOXQKw= 2022-09-03T06:45:33.045 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0ZGHpWGJ0//LLSbHKSZFqKOXQKw=', 'Content-Length': '0'} 2022-09-03T06:45:33.047 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000086d5792636b5ccd4-006312f80d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.048 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:33.049 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:33.049 DEBUG:boto:Method: HEAD 2022-09-03T06:45:33.050 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:45:33.051 DEBUG:boto:Data: 2022-09-03T06:45:33.051 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.052 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.052 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.053 DEBUG:boto:Params: {} 2022-09-03T06:45:33.053 DEBUG:boto:Token: None 2022-09-03T06:45:33.054 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj17 2022-09-03T06:45:33.054 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0ZGHpWGJ0//LLSbHKSZFqKOXQKw= 2022-09-03T06:45:33.055 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0ZGHpWGJ0//LLSbHKSZFqKOXQKw=', 'Content-Length': '0'} 2022-09-03T06:45:33.057 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004685aecf4b19c3c4-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.058 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:45:33.058 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:33.059 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:33.059 DEBUG:boto:Method: GET 2022-09-03T06:45:33.060 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:45:33.061 DEBUG:boto:Data: 2022-09-03T06:45:33.061 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.062 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.062 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.063 DEBUG:boto:Params: {} 2022-09-03T06:45:33.063 DEBUG:boto:Token: None 2022-09-03T06:45:33.064 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj17 2022-09-03T06:45:33.065 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y1rcKdfRdOICmNkx8mmJCwBVJEc= 2022-09-03T06:45:33.065 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y1rcKdfRdOICmNkx8mmJCwBVJEc=', 'Content-Length': '0'} 2022-09-03T06:45:33.068 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000586b4ea2f8438568-006312f80d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.110 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:33.111 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:33.112 DEBUG:boto:Method: GET 2022-09-03T06:45:33.112 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:45:33.113 DEBUG:boto:Data: 2022-09-03T06:45:33.113 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.114 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.114 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.115 DEBUG:boto:Params: {} 2022-09-03T06:45:33.116 DEBUG:boto:Token: None 2022-09-03T06:45:33.116 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj17 2022-09-03T06:45:33.117 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y1rcKdfRdOICmNkx8mmJCwBVJEc= 2022-09-03T06:45:33.118 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y1rcKdfRdOICmNkx8mmJCwBVJEc=', 'Content-Length': '0'} 2022-09-03T06:45:33.120 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000076e8de4a0911e19a-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.121 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:33.122 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:33.122 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:33.123 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:33.124 DEBUG:boto:Method: GET 2022-09-03T06:45:33.125 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:45:33.126 DEBUG:boto:Data: 2022-09-03T06:45:33.127 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.128 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.128 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.129 DEBUG:boto:Params: {} 2022-09-03T06:45:33.129 DEBUG:boto:Token: None 2022-09-03T06:45:33.130 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/ 2022-09-03T06:45:33.131 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk= 2022-09-03T06:45:33.132 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk=', 'Content-Length': '0'} 2022-09-03T06:45:33.135 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b521633ff173377b-006312f80d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.136 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:33.138 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:33.139 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:33.140 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:33.141 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:33.142 DEBUG:boto:Method: GET 2022-09-03T06:45:33.143 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:45:33.144 DEBUG:boto:Data: 2022-09-03T06:45:33.144 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.145 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.147 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.148 DEBUG:boto:Params: {} 2022-09-03T06:45:33.149 DEBUG:boto:Token: None 2022-09-03T06:45:33.150 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/ 2022-09-03T06:45:33.151 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk= 2022-09-03T06:45:33.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk=', 'Content-Length': '0'} 2022-09-03T06:45:33.156 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e5275b366110bccb-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.157 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:33.158 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:45:33.159 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:33.160 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:33.161 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.161 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.162 DEBUG:boto:Method: GET 2022-09-03T06:45:33.163 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.163 DEBUG:boto:Data: 2022-09-03T06:45:33.164 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.165 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.166 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.168 DEBUG:boto:Params: {} 2022-09-03T06:45:33.169 DEBUG:boto:Token: None 2022-09-03T06:45:33.170 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.171 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9puC5sepOVyGHVZYmJJJMK41AMg= 2022-09-03T06:45:33.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9puC5sepOVyGHVZYmJJJMK41AMg=', 'Content-Length': '0'} 2022-09-03T06:45:33.174 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000010f70b53523553b0-006312f80d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.175 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:33.176 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:33.176 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.177 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.178 DEBUG:boto:Method: GET 2022-09-03T06:45:33.179 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.179 DEBUG:boto:Data: 2022-09-03T06:45:33.180 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.181 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.181 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.182 DEBUG:boto:Params: {} 2022-09-03T06:45:33.182 DEBUG:boto:Token: None 2022-09-03T06:45:33.183 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.184 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9puC5sepOVyGHVZYmJJJMK41AMg= 2022-09-03T06:45:33.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9puC5sepOVyGHVZYmJJJMK41AMg=', 'Content-Length': '0'} 2022-09-03T06:45:33.187 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003f8db838643cee51-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.230 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:33.231 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:33.231 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.232 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.232 DEBUG:boto:Method: HEAD 2022-09-03T06:45:33.233 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.233 DEBUG:boto:Data: 2022-09-03T06:45:33.234 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.234 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.235 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.235 DEBUG:boto:Params: {} 2022-09-03T06:45:33.236 DEBUG:boto:Token: None 2022-09-03T06:45:33.237 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.237 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EECvegnFDSHBwqonLqT4VRDztVc= 2022-09-03T06:45:33.238 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EECvegnFDSHBwqonLqT4VRDztVc=', 'Content-Length': '0'} 2022-09-03T06:45:33.240 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004ee342d818e731f3-006312f80d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.241 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:33.242 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:33.242 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.243 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.243 DEBUG:boto:Method: HEAD 2022-09-03T06:45:33.244 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.244 DEBUG:boto:Data: 2022-09-03T06:45:33.245 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.245 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.246 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.246 DEBUG:boto:Params: {} 2022-09-03T06:45:33.247 DEBUG:boto:Token: None 2022-09-03T06:45:33.247 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.248 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EECvegnFDSHBwqonLqT4VRDztVc= 2022-09-03T06:45:33.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EECvegnFDSHBwqonLqT4VRDztVc=', 'Content-Length': '0'} 2022-09-03T06:45:33.250 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000063c9bb239314f15c-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.251 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:45:33.252 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:33.252 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:33.253 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.253 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.254 DEBUG:boto:Method: GET 2022-09-03T06:45:33.255 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.255 DEBUG:boto:Data: 2022-09-03T06:45:33.256 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.256 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.257 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.258 DEBUG:boto:Params: {} 2022-09-03T06:45:33.259 DEBUG:boto:Token: None 2022-09-03T06:45:33.259 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.260 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9puC5sepOVyGHVZYmJJJMK41AMg= 2022-09-03T06:45:33.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9puC5sepOVyGHVZYmJJJMK41AMg=', 'Content-Length': '0'} 2022-09-03T06:45:33.263 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c15e5311a9c94e64-006312f80d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.265 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:33.266 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:33.267 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.267 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.268 DEBUG:boto:Method: GET 2022-09-03T06:45:33.269 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.269 DEBUG:boto:Data: 2022-09-03T06:45:33.270 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.271 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.271 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.272 DEBUG:boto:Params: {} 2022-09-03T06:45:33.272 DEBUG:boto:Token: None 2022-09-03T06:45:33.273 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:33.273 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9puC5sepOVyGHVZYmJJJMK41AMg= 2022-09-03T06:45:33.274 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9puC5sepOVyGHVZYmJJJMK41AMg=', 'Content-Length': '0'} 2022-09-03T06:45:33.276 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096701ae45a2d1967-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.277 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:33.278 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:33.279 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:33.280 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:33.281 DEBUG:boto:Method: GET 2022-09-03T06:45:33.282 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:45:33.283 DEBUG:boto:Data: 2022-09-03T06:45:33.284 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.285 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.286 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.287 DEBUG:boto:Params: {} 2022-09-03T06:45:33.288 DEBUG:boto:Token: None 2022-09-03T06:45:33.289 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/ 2022-09-03T06:45:33.290 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk= 2022-09-03T06:45:33.291 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk=', 'Content-Length': '0'} 2022-09-03T06:45:33.294 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009363b98ca2c8ead6-006312f80d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.338 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:33.339 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:33.340 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:33.340 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:33.341 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:33.342 DEBUG:boto:Method: GET 2022-09-03T06:45:33.343 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:45:33.343 DEBUG:boto:Data: 2022-09-03T06:45:33.344 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.344 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.345 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.345 DEBUG:boto:Params: {} 2022-09-03T06:45:33.346 DEBUG:boto:Token: None 2022-09-03T06:45:33.347 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/ 2022-09-03T06:45:33.348 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk= 2022-09-03T06:45:33.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk=', 'Content-Length': '0'} 2022-09-03T06:45:33.352 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d5cc4520546f08af-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.353 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:33.354 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:33.354 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:33.355 DEBUG:boto:Method: HEAD 2022-09-03T06:45:33.355 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:45:33.356 DEBUG:boto:Data: 2022-09-03T06:45:33.356 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.357 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.357 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.358 DEBUG:boto:Params: {} 2022-09-03T06:45:33.359 DEBUG:boto:Token: None 2022-09-03T06:45:33.359 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj19 2022-09-03T06:45:33.360 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4hkvAtnS2bjI53u55OESQk6s0vM= 2022-09-03T06:45:33.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4hkvAtnS2bjI53u55OESQk6s0vM=', 'Content-Length': '0'} 2022-09-03T06:45:33.363 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005a5deeecb06bbacc-006312f80d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.364 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:33.364 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:33.365 DEBUG:boto:Method: HEAD 2022-09-03T06:45:33.365 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:45:33.366 DEBUG:boto:Data: 2022-09-03T06:45:33.366 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.367 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.367 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.368 DEBUG:boto:Params: {} 2022-09-03T06:45:33.369 DEBUG:boto:Token: None 2022-09-03T06:45:33.369 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj19 2022-09-03T06:45:33.370 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4hkvAtnS2bjI53u55OESQk6s0vM= 2022-09-03T06:45:33.370 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4hkvAtnS2bjI53u55OESQk6s0vM=', 'Content-Length': '0'} 2022-09-03T06:45:33.373 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f947d6555175870c-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.374 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:45:33.374 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:33.375 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:33.376 DEBUG:boto:Method: GET 2022-09-03T06:45:33.377 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:45:33.378 DEBUG:boto:Data: 2022-09-03T06:45:33.379 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.380 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.381 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.382 DEBUG:boto:Params: {} 2022-09-03T06:45:33.383 DEBUG:boto:Token: None 2022-09-03T06:45:33.384 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj19 2022-09-03T06:45:33.384 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/SQRY/+2VfN0l3R4SG1PLdib0sI= 2022-09-03T06:45:33.385 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/SQRY/+2VfN0l3R4SG1PLdib0sI=', 'Content-Length': '0'} 2022-09-03T06:45:33.387 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cdc3af569d4d560d-006312f80d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.388 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:33.389 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:33.389 DEBUG:boto:Method: GET 2022-09-03T06:45:33.390 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:45:33.390 DEBUG:boto:Data: 2022-09-03T06:45:33.391 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.391 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.392 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.392 DEBUG:boto:Params: {} 2022-09-03T06:45:33.393 DEBUG:boto:Token: None 2022-09-03T06:45:33.394 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj19 2022-09-03T06:45:33.394 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/SQRY/+2VfN0l3R4SG1PLdib0sI= 2022-09-03T06:45:33.395 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/SQRY/+2VfN0l3R4SG1PLdib0sI=', 'Content-Length': '0'} 2022-09-03T06:45:33.397 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b546679aed610aef-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.438 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:33.439 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:33.439 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:33.440 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:33.440 DEBUG:boto:Method: GET 2022-09-03T06:45:33.441 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:45:33.442 DEBUG:boto:Data: 2022-09-03T06:45:33.442 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.443 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.443 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.444 DEBUG:boto:Params: {} 2022-09-03T06:45:33.444 DEBUG:boto:Token: None 2022-09-03T06:45:33.445 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/ 2022-09-03T06:45:33.446 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk= 2022-09-03T06:45:33.446 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk=', 'Content-Length': '0'} 2022-09-03T06:45:33.449 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dd309c0f4fec7e5a-006312f80d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.450 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:33.451 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:33.452 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:33.452 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:33.453 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:33.453 DEBUG:boto:Method: GET 2022-09-03T06:45:33.455 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:45:33.456 DEBUG:boto:Data: 2022-09-03T06:45:33.457 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.458 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.459 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.460 DEBUG:boto:Params: {} 2022-09-03T06:45:33.461 DEBUG:boto:Token: None 2022-09-03T06:45:33.462 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/ 2022-09-03T06:45:33.463 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk= 2022-09-03T06:45:33.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk=', 'Content-Length': '0'} 2022-09-03T06:45:33.468 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000076eec7d091473601-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.468 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:33.469 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:45:33.470 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:33.470 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:33.471 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.473 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.474 DEBUG:boto:Method: GET 2022-09-03T06:45:33.475 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.476 DEBUG:boto:Data: 2022-09-03T06:45:33.477 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.478 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.479 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.479 DEBUG:boto:Params: {} 2022-09-03T06:45:33.480 DEBUG:boto:Token: None 2022-09-03T06:45:33.481 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.482 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m9qTKjVjdHFHrEn1cfYkxWY3mGg= 2022-09-03T06:45:33.483 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m9qTKjVjdHFHrEn1cfYkxWY3mGg=', 'Content-Length': '0'} 2022-09-03T06:45:33.485 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d17b233d90a40fa3-006312f80d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.486 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:33.486 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:33.487 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.488 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.489 DEBUG:boto:Method: GET 2022-09-03T06:45:33.491 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.491 DEBUG:boto:Data: 2022-09-03T06:45:33.492 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.493 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.494 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.495 DEBUG:boto:Params: {} 2022-09-03T06:45:33.496 DEBUG:boto:Token: None 2022-09-03T06:45:33.497 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.499 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m9qTKjVjdHFHrEn1cfYkxWY3mGg= 2022-09-03T06:45:33.500 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m9qTKjVjdHFHrEn1cfYkxWY3mGg=', 'Content-Length': '0'} 2022-09-03T06:45:33.503 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bfefc2dc2c07fb4e-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.505 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:33.505 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:33.506 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.507 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.508 DEBUG:boto:Method: HEAD 2022-09-03T06:45:33.509 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.510 DEBUG:boto:Data: 2022-09-03T06:45:33.511 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.511 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.512 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.513 DEBUG:boto:Params: {} 2022-09-03T06:45:33.514 DEBUG:boto:Token: None 2022-09-03T06:45:33.515 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.516 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BCs46v7EVlLAezG+e9d84rt/Efg= 2022-09-03T06:45:33.517 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BCs46v7EVlLAezG+e9d84rt/Efg=', 'Content-Length': '0'} 2022-09-03T06:45:33.519 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000174a2fa631d4532b-006312f80d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.520 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:33.521 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:33.522 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.522 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.523 DEBUG:boto:Method: HEAD 2022-09-03T06:45:33.523 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.524 DEBUG:boto:Data: 2022-09-03T06:45:33.524 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.525 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.525 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.526 DEBUG:boto:Params: {} 2022-09-03T06:45:33.526 DEBUG:boto:Token: None 2022-09-03T06:45:33.527 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.528 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BCs46v7EVlLAezG+e9d84rt/Efg= 2022-09-03T06:45:33.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BCs46v7EVlLAezG+e9d84rt/Efg=', 'Content-Length': '0'} 2022-09-03T06:45:33.530 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a5358b2f27b5ff3c-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.531 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:45:33.532 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:33.532 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:33.533 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.534 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.534 DEBUG:boto:Method: GET 2022-09-03T06:45:33.535 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.535 DEBUG:boto:Data: 2022-09-03T06:45:33.536 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.536 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.537 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.537 DEBUG:boto:Params: {} 2022-09-03T06:45:33.538 DEBUG:boto:Token: None 2022-09-03T06:45:33.539 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.539 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m9qTKjVjdHFHrEn1cfYkxWY3mGg= 2022-09-03T06:45:33.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m9qTKjVjdHFHrEn1cfYkxWY3mGg=', 'Content-Length': '0'} 2022-09-03T06:45:33.542 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005ce0fb98f9c96b17-006312f80d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.544 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:33.545 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:33.546 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.546 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.547 DEBUG:boto:Method: GET 2022-09-03T06:45:33.548 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.549 DEBUG:boto:Data: 2022-09-03T06:45:33.549 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.550 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.551 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.551 DEBUG:boto:Params: {} 2022-09-03T06:45:33.552 DEBUG:boto:Token: None 2022-09-03T06:45:33.553 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:33.553 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m9qTKjVjdHFHrEn1cfYkxWY3mGg= 2022-09-03T06:45:33.554 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m9qTKjVjdHFHrEn1cfYkxWY3mGg=', 'Content-Length': '0'} 2022-09-03T06:45:33.556 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009a69c643430a43f1-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.602 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:33.603 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:33.604 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:33.604 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:33.606 DEBUG:boto:Method: GET 2022-09-03T06:45:33.607 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:45:33.608 DEBUG:boto:Data: 2022-09-03T06:45:33.608 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.609 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.609 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.610 DEBUG:boto:Params: {} 2022-09-03T06:45:33.611 DEBUG:boto:Token: None 2022-09-03T06:45:33.611 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/ 2022-09-03T06:45:33.612 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk= 2022-09-03T06:45:33.612 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk=', 'Content-Length': '0'} 2022-09-03T06:45:33.615 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000556ef9f1fbfc742e-006312f80d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.616 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:33.618 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:33.619 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:33.620 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:33.621 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:33.622 DEBUG:boto:Method: GET 2022-09-03T06:45:33.623 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:45:33.623 DEBUG:boto:Data: 2022-09-03T06:45:33.624 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.625 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.626 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.626 DEBUG:boto:Params: {} 2022-09-03T06:45:33.627 DEBUG:boto:Token: None 2022-09-03T06:45:33.628 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/ 2022-09-03T06:45:33.629 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk= 2022-09-03T06:45:33.630 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk=', 'Content-Length': '0'} 2022-09-03T06:45:33.644 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aeeb5ebc81869899-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.645 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:33.646 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:33.647 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:33.647 DEBUG:boto:Method: HEAD 2022-09-03T06:45:33.648 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:45:33.648 DEBUG:boto:Data: 2022-09-03T06:45:33.649 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.649 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.650 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.650 DEBUG:boto:Params: {} 2022-09-03T06:45:33.651 DEBUG:boto:Token: None 2022-09-03T06:45:33.651 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj22 2022-09-03T06:45:33.652 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1sNAFePYPv0AqfVeCmekPu++E/M= 2022-09-03T06:45:33.653 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1sNAFePYPv0AqfVeCmekPu++E/M=', 'Content-Length': '0'} 2022-09-03T06:45:33.655 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f848ebd915c80889-006312f80d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.656 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:33.656 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:33.657 DEBUG:boto:Method: HEAD 2022-09-03T06:45:33.657 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:45:33.658 DEBUG:boto:Data: 2022-09-03T06:45:33.658 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.659 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.659 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.660 DEBUG:boto:Params: {} 2022-09-03T06:45:33.660 DEBUG:boto:Token: None 2022-09-03T06:45:33.661 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj22 2022-09-03T06:45:33.662 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1sNAFePYPv0AqfVeCmekPu++E/M= 2022-09-03T06:45:33.662 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1sNAFePYPv0AqfVeCmekPu++E/M=', 'Content-Length': '0'} 2022-09-03T06:45:33.665 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000041273d47e9b49697-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.665 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:45:33.666 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:33.666 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:33.667 DEBUG:boto:Method: GET 2022-09-03T06:45:33.667 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:45:33.668 DEBUG:boto:Data: 2022-09-03T06:45:33.669 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.669 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.670 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.670 DEBUG:boto:Params: {} 2022-09-03T06:45:33.671 DEBUG:boto:Token: None 2022-09-03T06:45:33.671 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj22 2022-09-03T06:45:33.672 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jtNhPEuB5qjfJLyOZIXxowS4Too= 2022-09-03T06:45:33.673 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jtNhPEuB5qjfJLyOZIXxowS4Too=', 'Content-Length': '0'} 2022-09-03T06:45:33.675 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b1c2bf910a555670-006312f80d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.676 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:33.677 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:33.677 DEBUG:boto:Method: GET 2022-09-03T06:45:33.678 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:45:33.678 DEBUG:boto:Data: 2022-09-03T06:45:33.679 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.679 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.680 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.680 DEBUG:boto:Params: {} 2022-09-03T06:45:33.681 DEBUG:boto:Token: None 2022-09-03T06:45:33.681 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj22 2022-09-03T06:45:33.682 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jtNhPEuB5qjfJLyOZIXxowS4Too= 2022-09-03T06:45:33.683 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jtNhPEuB5qjfJLyOZIXxowS4Too=', 'Content-Length': '0'} 2022-09-03T06:45:33.685 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000feba14945a55c7d8-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.686 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:33.686 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:33.687 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:33.687 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:33.688 DEBUG:boto:Method: GET 2022-09-03T06:45:33.689 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:45:33.689 DEBUG:boto:Data: 2022-09-03T06:45:33.690 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.690 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.691 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.691 DEBUG:boto:Params: {} 2022-09-03T06:45:33.692 DEBUG:boto:Token: None 2022-09-03T06:45:33.693 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/ 2022-09-03T06:45:33.693 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk= 2022-09-03T06:45:33.694 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk=', 'Content-Length': '0'} 2022-09-03T06:45:33.697 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006d51da3c7c57c35d-006312f80d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.699 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:33.700 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:33.701 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:33.702 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:33.703 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:33.704 DEBUG:boto:Method: GET 2022-09-03T06:45:33.705 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:45:33.706 DEBUG:boto:Data: 2022-09-03T06:45:33.707 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.708 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.709 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.710 DEBUG:boto:Params: {} 2022-09-03T06:45:33.711 DEBUG:boto:Token: None 2022-09-03T06:45:33.712 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/ 2022-09-03T06:45:33.714 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk= 2022-09-03T06:45:33.715 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk=', 'Content-Length': '0'} 2022-09-03T06:45:33.719 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dd8bd58c98b6557f-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.721 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:33.722 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:45:33.723 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:33.724 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:33.725 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.726 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.727 DEBUG:boto:Method: GET 2022-09-03T06:45:33.728 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.729 DEBUG:boto:Data: 2022-09-03T06:45:33.729 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.730 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.731 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.732 DEBUG:boto:Params: {} 2022-09-03T06:45:33.733 DEBUG:boto:Token: None 2022-09-03T06:45:33.734 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.735 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CoT6sLgBk72Qqw7XXB2kAxl4nAA= 2022-09-03T06:45:33.736 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CoT6sLgBk72Qqw7XXB2kAxl4nAA=', 'Content-Length': '0'} 2022-09-03T06:45:33.738 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000075c12e8ae4aa5635-006312f80d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.740 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:33.741 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:33.742 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.743 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.744 DEBUG:boto:Method: GET 2022-09-03T06:45:33.745 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.746 DEBUG:boto:Data: 2022-09-03T06:45:33.747 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.748 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.749 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.750 DEBUG:boto:Params: {} 2022-09-03T06:45:33.751 DEBUG:boto:Token: None 2022-09-03T06:45:33.752 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.753 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CoT6sLgBk72Qqw7XXB2kAxl4nAA= 2022-09-03T06:45:33.754 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CoT6sLgBk72Qqw7XXB2kAxl4nAA=', 'Content-Length': '0'} 2022-09-03T06:45:33.756 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000004d553d08f8e3260-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.802 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:33.803 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:33.804 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.805 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.806 DEBUG:boto:Method: HEAD 2022-09-03T06:45:33.807 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.808 DEBUG:boto:Data: 2022-09-03T06:45:33.809 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.810 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.811 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.812 DEBUG:boto:Params: {} 2022-09-03T06:45:33.813 DEBUG:boto:Token: None 2022-09-03T06:45:33.814 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.815 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e2abDwP/T8Kp0/OgP6sM61tmCj0= 2022-09-03T06:45:33.816 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e2abDwP/T8Kp0/OgP6sM61tmCj0=', 'Content-Length': '0'} 2022-09-03T06:45:33.818 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000ff3225386d8f8a5-006312f80d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.819 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:33.820 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:33.820 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.821 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.822 DEBUG:boto:Method: HEAD 2022-09-03T06:45:33.823 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.824 DEBUG:boto:Data: 2022-09-03T06:45:33.825 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.826 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.826 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.827 DEBUG:boto:Params: {} 2022-09-03T06:45:33.827 DEBUG:boto:Token: None 2022-09-03T06:45:33.828 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.829 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e2abDwP/T8Kp0/OgP6sM61tmCj0= 2022-09-03T06:45:33.830 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e2abDwP/T8Kp0/OgP6sM61tmCj0=', 'Content-Length': '0'} 2022-09-03T06:45:33.832 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fdfdefc95d1f4369-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.833 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:45:33.833 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:33.834 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:33.835 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.836 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.837 DEBUG:boto:Method: GET 2022-09-03T06:45:33.838 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.839 DEBUG:boto:Data: 2022-09-03T06:45:33.840 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.841 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.842 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.843 DEBUG:boto:Params: {} 2022-09-03T06:45:33.846 DEBUG:boto:Token: None 2022-09-03T06:45:33.847 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.848 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CoT6sLgBk72Qqw7XXB2kAxl4nAA= 2022-09-03T06:45:33.849 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CoT6sLgBk72Qqw7XXB2kAxl4nAA=', 'Content-Length': '0'} 2022-09-03T06:45:33.852 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002dded55549951d90-006312f80d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.894 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:33.895 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:33.896 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.897 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.898 DEBUG:boto:Method: GET 2022-09-03T06:45:33.899 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.900 DEBUG:boto:Data: 2022-09-03T06:45:33.901 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.902 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.903 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.904 DEBUG:boto:Params: {} 2022-09-03T06:45:33.905 DEBUG:boto:Token: None 2022-09-03T06:45:33.906 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:33.907 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CoT6sLgBk72Qqw7XXB2kAxl4nAA= 2022-09-03T06:45:33.908 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CoT6sLgBk72Qqw7XXB2kAxl4nAA=', 'Content-Length': '0'} 2022-09-03T06:45:33.910 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c93445fe68f0d5f0-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.911 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:33.912 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:33.913 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:33.913 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:33.914 DEBUG:boto:Method: GET 2022-09-03T06:45:33.915 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:45:33.915 DEBUG:boto:Data: 2022-09-03T06:45:33.916 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.916 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.917 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.918 DEBUG:boto:Params: {} 2022-09-03T06:45:33.918 DEBUG:boto:Token: None 2022-09-03T06:45:33.919 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/ 2022-09-03T06:45:33.920 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk= 2022-09-03T06:45:33.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk=', 'Content-Length': '0'} 2022-09-03T06:45:33.923 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002cb5d9f87b1bdc7d-006312f80d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.924 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:33.925 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:33.926 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:33.926 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:33.927 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:33.928 DEBUG:boto:Method: GET 2022-09-03T06:45:33.928 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:45:33.929 DEBUG:boto:Data: 2022-09-03T06:45:33.929 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.930 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.930 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.931 DEBUG:boto:Params: {} 2022-09-03T06:45:33.931 DEBUG:boto:Token: None 2022-09-03T06:45:33.932 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/ 2022-09-03T06:45:33.933 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk= 2022-09-03T06:45:33.934 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk=', 'Content-Length': '0'} 2022-09-03T06:45:33.938 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c28bd76fcbadacc5-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.940 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:33.941 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:33.942 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:33.943 DEBUG:boto:Method: HEAD 2022-09-03T06:45:33.944 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:45:33.945 DEBUG:boto:Data: 2022-09-03T06:45:33.946 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.947 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.948 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.949 DEBUG:boto:Params: {} 2022-09-03T06:45:33.950 DEBUG:boto:Token: None 2022-09-03T06:45:33.951 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj23 2022-09-03T06:45:33.952 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tXvWMVTJXNsAlakuwpY/CfymvkM= 2022-09-03T06:45:33.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tXvWMVTJXNsAlakuwpY/CfymvkM=', 'Content-Length': '0'} 2022-09-03T06:45:33.957 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b766248ac83c7a7d-006312f80d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.958 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:33.959 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:33.960 DEBUG:boto:Method: HEAD 2022-09-03T06:45:33.960 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:45:33.961 DEBUG:boto:Data: 2022-09-03T06:45:33.961 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.962 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.962 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.963 DEBUG:boto:Params: {} 2022-09-03T06:45:33.963 DEBUG:boto:Token: None 2022-09-03T06:45:33.964 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj23 2022-09-03T06:45:33.964 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tXvWMVTJXNsAlakuwpY/CfymvkM= 2022-09-03T06:45:33.965 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tXvWMVTJXNsAlakuwpY/CfymvkM=', 'Content-Length': '0'} 2022-09-03T06:45:33.967 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000070c9e4196a16b7ce-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.968 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:45:33.969 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:33.969 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:33.970 DEBUG:boto:Method: GET 2022-09-03T06:45:33.971 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:45:33.971 DEBUG:boto:Data: 2022-09-03T06:45:33.972 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.972 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.973 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.973 DEBUG:boto:Params: {} 2022-09-03T06:45:33.974 DEBUG:boto:Token: None 2022-09-03T06:45:33.974 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj23 2022-09-03T06:45:33.975 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hh5zlHaiCUv6N7RervFLYjCQUFQ= 2022-09-03T06:45:33.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hh5zlHaiCUv6N7RervFLYjCQUFQ=', 'Content-Length': '0'} 2022-09-03T06:45:33.978 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006373e1608bd535a6-006312f80d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.979 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:33.979 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:33.980 DEBUG:boto:Method: GET 2022-09-03T06:45:33.981 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:45:33.981 DEBUG:boto:Data: 2022-09-03T06:45:33.982 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.982 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:33.983 DEBUG:boto:Port: 8000 2022-09-03T06:45:33.983 DEBUG:boto:Params: {} 2022-09-03T06:45:33.984 DEBUG:boto:Token: None 2022-09-03T06:45:33.984 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/obj23 2022-09-03T06:45:33.985 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hh5zlHaiCUv6N7RervFLYjCQUFQ= 2022-09-03T06:45:33.986 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hh5zlHaiCUv6N7RervFLYjCQUFQ=', 'Content-Length': '0'} 2022-09-03T06:45:33.988 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c092679715578caa-006312f80d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:33.989 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:33.989 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:33.990 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:33.991 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:33.991 DEBUG:boto:Method: GET 2022-09-03T06:45:33.992 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:45:33.992 DEBUG:boto:Data: 2022-09-03T06:45:33.993 DEBUG:boto:Headers: {} 2022-09-03T06:45:33.993 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:33.994 DEBUG:boto:Port: 8001 2022-09-03T06:45:33.994 DEBUG:boto:Params: {} 2022-09-03T06:45:33.995 DEBUG:boto:Token: None 2022-09-03T06:45:33.995 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:33 GMT /dsgvcy-75/ 2022-09-03T06:45:33.996 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk= 2022-09-03T06:45:33.997 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8zLBEoHgbXgGGkfpjTMI5tzDhnk=', 'Content-Length': '0'} 2022-09-03T06:45:34.001 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009f09420bb57baa24-006312f80d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.043 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:34.044 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:34.044 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:34.045 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:34.045 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:34.046 DEBUG:boto:Method: GET 2022-09-03T06:45:34.047 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:45:34.048 DEBUG:boto:Data: 2022-09-03T06:45:34.048 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.049 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.050 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.050 DEBUG:boto:Params: {} 2022-09-03T06:45:34.051 DEBUG:boto:Token: None 2022-09-03T06:45:34.052 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/ 2022-09-03T06:45:34.053 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY= 2022-09-03T06:45:34.054 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY=', 'Content-Length': '0'} 2022-09-03T06:45:34.059 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008b666912b3613160-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.060 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:34.061 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:45:34.062 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:34.063 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:34.064 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.064 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.065 DEBUG:boto:Method: GET 2022-09-03T06:45:34.065 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.066 DEBUG:boto:Data: 2022-09-03T06:45:34.068 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.069 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.070 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.071 DEBUG:boto:Params: {} 2022-09-03T06:45:34.072 DEBUG:boto:Token: None 2022-09-03T06:45:34.073 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.074 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LoS6jCDABvWrUUOnVLr+Tuiik9Y= 2022-09-03T06:45:34.074 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LoS6jCDABvWrUUOnVLr+Tuiik9Y=', 'Content-Length': '0'} 2022-09-03T06:45:34.077 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d325aed1d15d67c6-006312f80e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.078 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:34.078 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:34.079 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.080 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.080 DEBUG:boto:Method: GET 2022-09-03T06:45:34.081 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.081 DEBUG:boto:Data: 2022-09-03T06:45:34.082 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.083 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.084 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.085 DEBUG:boto:Params: {} 2022-09-03T06:45:34.086 DEBUG:boto:Token: None 2022-09-03T06:45:34.087 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.087 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LoS6jCDABvWrUUOnVLr+Tuiik9Y= 2022-09-03T06:45:34.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LoS6jCDABvWrUUOnVLr+Tuiik9Y=', 'Content-Length': '0'} 2022-09-03T06:45:34.091 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bfcfa914b58f7102-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.134 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:34.135 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:34.136 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.137 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.138 DEBUG:boto:Method: HEAD 2022-09-03T06:45:34.139 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.139 DEBUG:boto:Data: 2022-09-03T06:45:34.140 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.141 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.142 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.143 DEBUG:boto:Params: {} 2022-09-03T06:45:34.144 DEBUG:boto:Token: None 2022-09-03T06:45:34.145 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.146 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xckwIvnVWoZYDtEVArOnGg6O8OE= 2022-09-03T06:45:34.147 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xckwIvnVWoZYDtEVArOnGg6O8OE=', 'Content-Length': '0'} 2022-09-03T06:45:34.150 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000902d61c36627ac71-006312f80e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.151 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:34.152 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:34.153 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.154 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.155 DEBUG:boto:Method: HEAD 2022-09-03T06:45:34.155 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.156 DEBUG:boto:Data: 2022-09-03T06:45:34.156 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.157 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.157 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.158 DEBUG:boto:Params: {} 2022-09-03T06:45:34.159 DEBUG:boto:Token: None 2022-09-03T06:45:34.159 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.160 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xckwIvnVWoZYDtEVArOnGg6O8OE= 2022-09-03T06:45:34.161 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xckwIvnVWoZYDtEVArOnGg6O8OE=', 'Content-Length': '0'} 2022-09-03T06:45:34.163 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b1137bb327f11d90-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.165 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:45:34.165 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:34.166 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:34.167 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.168 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.169 DEBUG:boto:Method: GET 2022-09-03T06:45:34.169 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.170 DEBUG:boto:Data: 2022-09-03T06:45:34.170 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.171 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.171 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.172 DEBUG:boto:Params: {} 2022-09-03T06:45:34.172 DEBUG:boto:Token: None 2022-09-03T06:45:34.173 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.174 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LoS6jCDABvWrUUOnVLr+Tuiik9Y= 2022-09-03T06:45:34.175 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LoS6jCDABvWrUUOnVLr+Tuiik9Y=', 'Content-Length': '0'} 2022-09-03T06:45:34.178 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eefe13729968fde0-006312f80e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.180 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:34.181 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:34.181 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.183 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.184 DEBUG:boto:Method: GET 2022-09-03T06:45:34.185 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.186 DEBUG:boto:Data: 2022-09-03T06:45:34.186 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.187 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.188 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.188 DEBUG:boto:Params: {} 2022-09-03T06:45:34.189 DEBUG:boto:Token: None 2022-09-03T06:45:34.190 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:34.191 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LoS6jCDABvWrUUOnVLr+Tuiik9Y= 2022-09-03T06:45:34.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LoS6jCDABvWrUUOnVLr+Tuiik9Y=', 'Content-Length': '0'} 2022-09-03T06:45:34.195 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003870c50d856f9b97-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.196 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:34.196 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:34.197 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:34.198 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:34.198 DEBUG:boto:Method: GET 2022-09-03T06:45:34.199 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:45:34.199 DEBUG:boto:Data: 2022-09-03T06:45:34.200 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.201 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.201 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.202 DEBUG:boto:Params: {} 2022-09-03T06:45:34.203 DEBUG:boto:Token: None 2022-09-03T06:45:34.203 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/ 2022-09-03T06:45:34.204 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY= 2022-09-03T06:45:34.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY=', 'Content-Length': '0'} 2022-09-03T06:45:34.207 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007d6e38eaa8fe7dd5-006312f80e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.209 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:34.210 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:34.211 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:34.212 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:34.213 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:34.214 DEBUG:boto:Method: GET 2022-09-03T06:45:34.215 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:45:34.216 DEBUG:boto:Data: 2022-09-03T06:45:34.216 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.217 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.218 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.219 DEBUG:boto:Params: {} 2022-09-03T06:45:34.220 DEBUG:boto:Token: None 2022-09-03T06:45:34.220 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/ 2022-09-03T06:45:34.221 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY= 2022-09-03T06:45:34.221 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY=', 'Content-Length': '0'} 2022-09-03T06:45:34.226 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b9a639f139b61fed-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.227 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:34.228 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:34.228 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:34.229 DEBUG:boto:Method: HEAD 2022-09-03T06:45:34.229 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:45:34.230 DEBUG:boto:Data: 2022-09-03T06:45:34.231 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.232 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.232 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.233 DEBUG:boto:Params: {} 2022-09-03T06:45:34.234 DEBUG:boto:Token: None 2022-09-03T06:45:34.234 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj29 2022-09-03T06:45:34.235 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8SUjn/LAB7KTEOf+mik95PLbQRQ= 2022-09-03T06:45:34.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8SUjn/LAB7KTEOf+mik95PLbQRQ=', 'Content-Length': '0'} 2022-09-03T06:45:34.238 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000449e8b1076fef140-006312f80e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.239 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:34.240 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:34.240 DEBUG:boto:Method: HEAD 2022-09-03T06:45:34.241 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:45:34.242 DEBUG:boto:Data: 2022-09-03T06:45:34.242 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.243 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.243 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.244 DEBUG:boto:Params: {} 2022-09-03T06:45:34.244 DEBUG:boto:Token: None 2022-09-03T06:45:34.245 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj29 2022-09-03T06:45:34.245 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8SUjn/LAB7KTEOf+mik95PLbQRQ= 2022-09-03T06:45:34.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8SUjn/LAB7KTEOf+mik95PLbQRQ=', 'Content-Length': '0'} 2022-09-03T06:45:34.249 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000726076ad6c9ed437-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.250 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:45:34.250 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:34.251 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:34.252 DEBUG:boto:Method: GET 2022-09-03T06:45:34.252 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:45:34.253 DEBUG:boto:Data: 2022-09-03T06:45:34.254 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.255 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.255 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.256 DEBUG:boto:Params: {} 2022-09-03T06:45:34.257 DEBUG:boto:Token: None 2022-09-03T06:45:34.258 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj29 2022-09-03T06:45:34.259 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pmnxyKT2R8I+Aog24wxM5M5TLUg= 2022-09-03T06:45:34.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pmnxyKT2R8I+Aog24wxM5M5TLUg=', 'Content-Length': '0'} 2022-09-03T06:45:34.262 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004dd4e45c7066aeab-006312f80e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.306 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:34.307 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:34.308 DEBUG:boto:Method: GET 2022-09-03T06:45:34.308 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:45:34.309 DEBUG:boto:Data: 2022-09-03T06:45:34.310 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.310 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.311 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.311 DEBUG:boto:Params: {} 2022-09-03T06:45:34.312 DEBUG:boto:Token: None 2022-09-03T06:45:34.313 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj29 2022-09-03T06:45:34.314 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pmnxyKT2R8I+Aog24wxM5M5TLUg= 2022-09-03T06:45:34.315 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pmnxyKT2R8I+Aog24wxM5M5TLUg=', 'Content-Length': '0'} 2022-09-03T06:45:34.317 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000058729ae4b6655684-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.318 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:34.318 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:34.319 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:34.319 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:34.320 DEBUG:boto:Method: GET 2022-09-03T06:45:34.321 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:45:34.321 DEBUG:boto:Data: 2022-09-03T06:45:34.322 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.322 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.323 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.324 DEBUG:boto:Params: {} 2022-09-03T06:45:34.325 DEBUG:boto:Token: None 2022-09-03T06:45:34.326 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/ 2022-09-03T06:45:34.327 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY= 2022-09-03T06:45:34.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY=', 'Content-Length': '0'} 2022-09-03T06:45:34.331 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002e795b8208e8ac8b-006312f80e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.332 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:34.333 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:34.333 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:34.334 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:34.335 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:34.336 DEBUG:boto:Method: GET 2022-09-03T06:45:34.336 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:45:34.337 DEBUG:boto:Data: 2022-09-03T06:45:34.337 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.338 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.339 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.340 DEBUG:boto:Params: {} 2022-09-03T06:45:34.340 DEBUG:boto:Token: None 2022-09-03T06:45:34.341 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/ 2022-09-03T06:45:34.341 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY= 2022-09-03T06:45:34.342 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY=', 'Content-Length': '0'} 2022-09-03T06:45:34.346 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000122e3d766ad1bc0d-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.347 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:34.348 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:45:34.349 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:34.350 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:34.351 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.352 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.353 DEBUG:boto:Method: GET 2022-09-03T06:45:34.354 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.354 DEBUG:boto:Data: 2022-09-03T06:45:34.355 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.355 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.356 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.357 DEBUG:boto:Params: {} 2022-09-03T06:45:34.357 DEBUG:boto:Token: None 2022-09-03T06:45:34.358 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.359 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FrjifvJOglJdtQJuIhjnM5d5Czw= 2022-09-03T06:45:34.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FrjifvJOglJdtQJuIhjnM5d5Czw=', 'Content-Length': '0'} 2022-09-03T06:45:34.363 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009220bdd47e9cfe5b-006312f80e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.364 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:34.365 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:34.366 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.367 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.368 DEBUG:boto:Method: GET 2022-09-03T06:45:34.369 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.370 DEBUG:boto:Data: 2022-09-03T06:45:34.372 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.372 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.373 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.374 DEBUG:boto:Params: {} 2022-09-03T06:45:34.375 DEBUG:boto:Token: None 2022-09-03T06:45:34.376 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.378 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FrjifvJOglJdtQJuIhjnM5d5Czw= 2022-09-03T06:45:34.379 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FrjifvJOglJdtQJuIhjnM5d5Czw=', 'Content-Length': '0'} 2022-09-03T06:45:34.381 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eea9cfc0e26997bb-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.383 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:34.383 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:34.384 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.384 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.385 DEBUG:boto:Method: HEAD 2022-09-03T06:45:34.385 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.386 DEBUG:boto:Data: 2022-09-03T06:45:34.387 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.388 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.389 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.390 DEBUG:boto:Params: {} 2022-09-03T06:45:34.391 DEBUG:boto:Token: None 2022-09-03T06:45:34.392 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.394 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6MNQ6+XOCu0rpqSr7K1TyEpxBxI= 2022-09-03T06:45:34.395 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6MNQ6+XOCu0rpqSr7K1TyEpxBxI=', 'Content-Length': '0'} 2022-09-03T06:45:34.398 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee861666903cb3ed-006312f80e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.399 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:34.400 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:34.401 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.402 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.402 DEBUG:boto:Method: HEAD 2022-09-03T06:45:34.403 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.404 DEBUG:boto:Data: 2022-09-03T06:45:34.404 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.405 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.406 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.407 DEBUG:boto:Params: {} 2022-09-03T06:45:34.407 DEBUG:boto:Token: None 2022-09-03T06:45:34.409 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.410 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6MNQ6+XOCu0rpqSr7K1TyEpxBxI= 2022-09-03T06:45:34.411 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6MNQ6+XOCu0rpqSr7K1TyEpxBxI=', 'Content-Length': '0'} 2022-09-03T06:45:34.421 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004c88e43d2c5e85bb-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.422 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:45:34.423 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:34.424 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:34.425 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.426 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.427 DEBUG:boto:Method: GET 2022-09-03T06:45:34.428 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.429 DEBUG:boto:Data: 2022-09-03T06:45:34.430 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.431 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.432 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.433 DEBUG:boto:Params: {} 2022-09-03T06:45:34.434 DEBUG:boto:Token: None 2022-09-03T06:45:34.435 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.436 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FrjifvJOglJdtQJuIhjnM5d5Czw= 2022-09-03T06:45:34.437 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FrjifvJOglJdtQJuIhjnM5d5Czw=', 'Content-Length': '0'} 2022-09-03T06:45:34.441 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b4e2a6d445064f0-006312f80e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.482 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:34.483 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:34.484 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.484 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.485 DEBUG:boto:Method: GET 2022-09-03T06:45:34.486 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.490 DEBUG:boto:Data: 2022-09-03T06:45:34.491 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.491 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.492 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.492 DEBUG:boto:Params: {} 2022-09-03T06:45:34.493 DEBUG:boto:Token: None 2022-09-03T06:45:34.493 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:34.494 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FrjifvJOglJdtQJuIhjnM5d5Czw= 2022-09-03T06:45:34.495 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FrjifvJOglJdtQJuIhjnM5d5Czw=', 'Content-Length': '0'} 2022-09-03T06:45:34.497 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000036ee879970577a83-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.499 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:34.499 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:34.500 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:34.501 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:34.502 DEBUG:boto:Method: GET 2022-09-03T06:45:34.502 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:45:34.503 DEBUG:boto:Data: 2022-09-03T06:45:34.505 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.505 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.506 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.507 DEBUG:boto:Params: {} 2022-09-03T06:45:34.508 DEBUG:boto:Token: None 2022-09-03T06:45:34.509 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/ 2022-09-03T06:45:34.510 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY= 2022-09-03T06:45:34.510 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY=', 'Content-Length': '0'} 2022-09-03T06:45:34.514 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f4fa658e725d141c-006312f80e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.515 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:34.517 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:34.518 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:34.519 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:34.520 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:34.521 DEBUG:boto:Method: GET 2022-09-03T06:45:34.522 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:45:34.523 DEBUG:boto:Data: 2022-09-03T06:45:34.523 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.525 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.526 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.527 DEBUG:boto:Params: {} 2022-09-03T06:45:34.528 DEBUG:boto:Token: None 2022-09-03T06:45:34.529 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/ 2022-09-03T06:45:34.530 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY= 2022-09-03T06:45:34.531 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY=', 'Content-Length': '0'} 2022-09-03T06:45:34.536 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000db95da6d281d700-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.539 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:34.540 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:34.541 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:34.541 DEBUG:boto:Method: HEAD 2022-09-03T06:45:34.542 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:45:34.542 DEBUG:boto:Data: 2022-09-03T06:45:34.543 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.544 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.544 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.545 DEBUG:boto:Params: {} 2022-09-03T06:45:34.546 DEBUG:boto:Token: None 2022-09-03T06:45:34.546 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj33 2022-09-03T06:45:34.547 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1/tc6Wait9xWLPmzB2dX9zdaE0g= 2022-09-03T06:45:34.548 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1/tc6Wait9xWLPmzB2dX9zdaE0g=', 'Content-Length': '0'} 2022-09-03T06:45:34.551 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e31363600f7894ce-006312f80e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.552 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:34.553 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:34.553 DEBUG:boto:Method: HEAD 2022-09-03T06:45:34.554 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:45:34.554 DEBUG:boto:Data: 2022-09-03T06:45:34.555 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.555 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.556 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.556 DEBUG:boto:Params: {} 2022-09-03T06:45:34.557 DEBUG:boto:Token: None 2022-09-03T06:45:34.558 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj33 2022-09-03T06:45:34.559 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1/tc6Wait9xWLPmzB2dX9zdaE0g= 2022-09-03T06:45:34.559 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1/tc6Wait9xWLPmzB2dX9zdaE0g=', 'Content-Length': '0'} 2022-09-03T06:45:34.561 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001cac97091b2de44f-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.563 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:45:34.563 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:34.564 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:34.565 DEBUG:boto:Method: GET 2022-09-03T06:45:34.566 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:45:34.567 DEBUG:boto:Data: 2022-09-03T06:45:34.568 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.569 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.570 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.571 DEBUG:boto:Params: {} 2022-09-03T06:45:34.572 DEBUG:boto:Token: None 2022-09-03T06:45:34.573 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj33 2022-09-03T06:45:34.575 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OK5aYidAp9TUeet2xlef188/Lcc= 2022-09-03T06:45:34.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OK5aYidAp9TUeet2xlef188/Lcc=', 'Content-Length': '0'} 2022-09-03T06:45:34.578 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003ba6bf2e92091112-006312f80e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.579 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:34.579 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:34.580 DEBUG:boto:Method: GET 2022-09-03T06:45:34.580 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:45:34.581 DEBUG:boto:Data: 2022-09-03T06:45:34.581 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.582 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.583 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.583 DEBUG:boto:Params: {} 2022-09-03T06:45:34.584 DEBUG:boto:Token: None 2022-09-03T06:45:34.585 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj33 2022-09-03T06:45:34.585 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OK5aYidAp9TUeet2xlef188/Lcc= 2022-09-03T06:45:34.586 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OK5aYidAp9TUeet2xlef188/Lcc=', 'Content-Length': '0'} 2022-09-03T06:45:34.588 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e8a65ac7e08e5696-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.589 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:34.590 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:34.590 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:34.591 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:34.591 DEBUG:boto:Method: GET 2022-09-03T06:45:34.592 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:45:34.592 DEBUG:boto:Data: 2022-09-03T06:45:34.593 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.594 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.594 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.595 DEBUG:boto:Params: {} 2022-09-03T06:45:34.595 DEBUG:boto:Token: None 2022-09-03T06:45:34.596 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/ 2022-09-03T06:45:34.597 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY= 2022-09-03T06:45:34.597 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY=', 'Content-Length': '0'} 2022-09-03T06:45:34.600 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002450e710e5a0e4d0-006312f80e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.601 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:34.602 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:34.602 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:34.603 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:34.603 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:34.604 DEBUG:boto:Method: GET 2022-09-03T06:45:34.604 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:45:34.605 DEBUG:boto:Data: 2022-09-03T06:45:34.605 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.606 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.607 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.607 DEBUG:boto:Params: {} 2022-09-03T06:45:34.608 DEBUG:boto:Token: None 2022-09-03T06:45:34.609 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/ 2022-09-03T06:45:34.610 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY= 2022-09-03T06:45:34.611 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY=', 'Content-Length': '0'} 2022-09-03T06:45:34.620 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ec5eb255c8a511cd-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.621 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:34.622 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:45:34.623 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:34.624 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:34.624 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.625 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.625 DEBUG:boto:Method: GET 2022-09-03T06:45:34.626 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.626 DEBUG:boto:Data: 2022-09-03T06:45:34.627 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.627 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.628 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.629 DEBUG:boto:Params: {} 2022-09-03T06:45:34.629 DEBUG:boto:Token: None 2022-09-03T06:45:34.630 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.631 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wU+SYmTT0j/xnk5inpmeetmYA+o= 2022-09-03T06:45:34.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wU+SYmTT0j/xnk5inpmeetmYA+o=', 'Content-Length': '0'} 2022-09-03T06:45:34.635 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a40f05529c1376d1-006312f80e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.636 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:34.637 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:34.638 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.639 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.639 DEBUG:boto:Method: GET 2022-09-03T06:45:34.640 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.640 DEBUG:boto:Data: 2022-09-03T06:45:34.641 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.641 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.642 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.643 DEBUG:boto:Params: {} 2022-09-03T06:45:34.643 DEBUG:boto:Token: None 2022-09-03T06:45:34.644 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.645 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wU+SYmTT0j/xnk5inpmeetmYA+o= 2022-09-03T06:45:34.645 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wU+SYmTT0j/xnk5inpmeetmYA+o=', 'Content-Length': '0'} 2022-09-03T06:45:34.648 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a271d72036004310-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.690 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:34.691 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:34.692 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.692 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.693 DEBUG:boto:Method: HEAD 2022-09-03T06:45:34.693 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.694 DEBUG:boto:Data: 2022-09-03T06:45:34.694 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.695 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.695 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.696 DEBUG:boto:Params: {} 2022-09-03T06:45:34.696 DEBUG:boto:Token: None 2022-09-03T06:45:34.697 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.698 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C+en3KCGW153uFsJN1EJ02S66u0= 2022-09-03T06:45:34.698 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C+en3KCGW153uFsJN1EJ02S66u0=', 'Content-Length': '0'} 2022-09-03T06:45:34.700 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e23470c138df14f3-006312f80e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.701 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:34.702 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:34.702 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.704 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.705 DEBUG:boto:Method: HEAD 2022-09-03T06:45:34.707 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.707 DEBUG:boto:Data: 2022-09-03T06:45:34.708 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.709 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.710 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.711 DEBUG:boto:Params: {} 2022-09-03T06:45:34.712 DEBUG:boto:Token: None 2022-09-03T06:45:34.714 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.715 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C+en3KCGW153uFsJN1EJ02S66u0= 2022-09-03T06:45:34.716 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C+en3KCGW153uFsJN1EJ02S66u0=', 'Content-Length': '0'} 2022-09-03T06:45:34.718 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a741ae96755633c1-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.719 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:45:34.720 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:34.721 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:34.722 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.723 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.724 DEBUG:boto:Method: GET 2022-09-03T06:45:34.726 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.726 DEBUG:boto:Data: 2022-09-03T06:45:34.727 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.728 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.729 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.730 DEBUG:boto:Params: {} 2022-09-03T06:45:34.731 DEBUG:boto:Token: None 2022-09-03T06:45:34.732 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.733 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wU+SYmTT0j/xnk5inpmeetmYA+o= 2022-09-03T06:45:34.734 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wU+SYmTT0j/xnk5inpmeetmYA+o=', 'Content-Length': '0'} 2022-09-03T06:45:34.736 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009b90bf1d09a5807e-006312f80e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.737 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:34.738 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:34.738 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.739 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.739 DEBUG:boto:Method: GET 2022-09-03T06:45:34.740 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.741 DEBUG:boto:Data: 2022-09-03T06:45:34.741 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.742 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.742 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.743 DEBUG:boto:Params: {} 2022-09-03T06:45:34.744 DEBUG:boto:Token: None 2022-09-03T06:45:34.744 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:34.745 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wU+SYmTT0j/xnk5inpmeetmYA+o= 2022-09-03T06:45:34.746 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wU+SYmTT0j/xnk5inpmeetmYA+o=', 'Content-Length': '0'} 2022-09-03T06:45:34.748 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009373e3b9ed0a648e-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.749 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:34.750 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:34.751 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:34.751 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:34.752 DEBUG:boto:Method: GET 2022-09-03T06:45:34.752 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:45:34.753 DEBUG:boto:Data: 2022-09-03T06:45:34.753 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.754 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.754 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.755 DEBUG:boto:Params: {} 2022-09-03T06:45:34.756 DEBUG:boto:Token: None 2022-09-03T06:45:34.756 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/ 2022-09-03T06:45:34.757 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY= 2022-09-03T06:45:34.757 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY=', 'Content-Length': '0'} 2022-09-03T06:45:34.761 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000113c7db3efcfc55c-006312f80e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.762 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:34.764 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:34.765 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:34.765 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:34.766 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:34.767 DEBUG:boto:Method: GET 2022-09-03T06:45:34.768 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:45:34.769 DEBUG:boto:Data: 2022-09-03T06:45:34.770 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.771 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.773 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.773 DEBUG:boto:Params: {} 2022-09-03T06:45:34.774 DEBUG:boto:Token: None 2022-09-03T06:45:34.775 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/ 2022-09-03T06:45:34.776 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY= 2022-09-03T06:45:34.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY=', 'Content-Length': '0'} 2022-09-03T06:45:34.786 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000004412d3a14fc06c7-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.788 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:34.788 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:34.789 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:34.790 DEBUG:boto:Method: HEAD 2022-09-03T06:45:34.790 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:45:34.791 DEBUG:boto:Data: 2022-09-03T06:45:34.791 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.792 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.792 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.793 DEBUG:boto:Params: {} 2022-09-03T06:45:34.793 DEBUG:boto:Token: None 2022-09-03T06:45:34.794 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj34 2022-09-03T06:45:34.794 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IfCiN1sm1TH5A3hI1szMxOwB034= 2022-09-03T06:45:34.795 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IfCiN1sm1TH5A3hI1szMxOwB034=', 'Content-Length': '0'} 2022-09-03T06:45:34.799 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000588026eb28f723f3-006312f80e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.799 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:34.800 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:34.801 DEBUG:boto:Method: HEAD 2022-09-03T06:45:34.801 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:45:34.802 DEBUG:boto:Data: 2022-09-03T06:45:34.803 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.803 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.804 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.804 DEBUG:boto:Params: {} 2022-09-03T06:45:34.805 DEBUG:boto:Token: None 2022-09-03T06:45:34.806 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj34 2022-09-03T06:45:34.806 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IfCiN1sm1TH5A3hI1szMxOwB034= 2022-09-03T06:45:34.807 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IfCiN1sm1TH5A3hI1szMxOwB034=', 'Content-Length': '0'} 2022-09-03T06:45:34.809 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000be9790bda6e3e19c-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.811 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:45:34.812 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:34.813 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:34.814 DEBUG:boto:Method: GET 2022-09-03T06:45:34.815 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:45:34.816 DEBUG:boto:Data: 2022-09-03T06:45:34.817 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.817 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.818 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.818 DEBUG:boto:Params: {} 2022-09-03T06:45:34.819 DEBUG:boto:Token: None 2022-09-03T06:45:34.819 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj34 2022-09-03T06:45:34.820 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wrrMNFFT0WDEEabzls/5d9gnWSI= 2022-09-03T06:45:34.821 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wrrMNFFT0WDEEabzls/5d9gnWSI=', 'Content-Length': '0'} 2022-09-03T06:45:34.823 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009eb1d3f75ee15a5d-006312f80e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.824 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:34.825 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:34.825 DEBUG:boto:Method: GET 2022-09-03T06:45:34.826 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:45:34.827 DEBUG:boto:Data: 2022-09-03T06:45:34.827 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.828 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.828 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.829 DEBUG:boto:Params: {} 2022-09-03T06:45:34.830 DEBUG:boto:Token: None 2022-09-03T06:45:34.830 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj34 2022-09-03T06:45:34.831 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wrrMNFFT0WDEEabzls/5d9gnWSI= 2022-09-03T06:45:34.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wrrMNFFT0WDEEabzls/5d9gnWSI=', 'Content-Length': '0'} 2022-09-03T06:45:34.834 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000b607ccfa9c28fcc-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.835 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:34.836 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:34.836 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:34.837 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:34.838 DEBUG:boto:Method: GET 2022-09-03T06:45:34.838 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:45:34.839 DEBUG:boto:Data: 2022-09-03T06:45:34.840 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.840 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.841 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.841 DEBUG:boto:Params: {} 2022-09-03T06:45:34.842 DEBUG:boto:Token: None 2022-09-03T06:45:34.842 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/ 2022-09-03T06:45:34.843 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY= 2022-09-03T06:45:34.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY=', 'Content-Length': '0'} 2022-09-03T06:45:34.847 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000088c1d2af36d8cb67-006312f80e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.848 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:34.849 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:34.851 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:34.851 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:34.852 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:34.853 DEBUG:boto:Method: GET 2022-09-03T06:45:34.854 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:45:34.855 DEBUG:boto:Data: 2022-09-03T06:45:34.855 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.856 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.857 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.858 DEBUG:boto:Params: {} 2022-09-03T06:45:34.858 DEBUG:boto:Token: None 2022-09-03T06:45:34.859 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/ 2022-09-03T06:45:34.860 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY= 2022-09-03T06:45:34.861 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NWOzhbilAf7uhYuiVhYUDJlR9zY=', 'Content-Length': '0'} 2022-09-03T06:45:34.865 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000189b2a8e83a216e-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.908 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:34.909 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:45:34.910 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:34.910 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:34.911 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:34.912 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:34.912 DEBUG:boto:Method: GET 2022-09-03T06:45:34.913 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:34.913 DEBUG:boto:Data: 2022-09-03T06:45:34.914 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.915 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.916 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.917 DEBUG:boto:Params: {} 2022-09-03T06:45:34.918 DEBUG:boto:Token: None 2022-09-03T06:45:34.919 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:34.920 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XOI7m2G0ZjujxNVNIYAoluj6f0g= 2022-09-03T06:45:34.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XOI7m2G0ZjujxNVNIYAoluj6f0g=', 'Content-Length': '0'} 2022-09-03T06:45:34.924 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d195765290c3d2d0-006312f80e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.927 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:34.928 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:34.929 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:34.929 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:34.930 DEBUG:boto:Method: GET 2022-09-03T06:45:34.931 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:34.932 DEBUG:boto:Data: 2022-09-03T06:45:34.932 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.933 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.933 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.934 DEBUG:boto:Params: {} 2022-09-03T06:45:34.934 DEBUG:boto:Token: None 2022-09-03T06:45:34.935 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:34.936 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XOI7m2G0ZjujxNVNIYAoluj6f0g= 2022-09-03T06:45:34.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XOI7m2G0ZjujxNVNIYAoluj6f0g=', 'Content-Length': '0'} 2022-09-03T06:45:34.938 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000077080b4b861daf4-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.939 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:34.940 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:34.940 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:34.941 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:34.942 DEBUG:boto:Method: HEAD 2022-09-03T06:45:34.942 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:34.943 DEBUG:boto:Data: 2022-09-03T06:45:34.943 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.944 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.944 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.945 DEBUG:boto:Params: {} 2022-09-03T06:45:34.946 DEBUG:boto:Token: None 2022-09-03T06:45:34.947 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:34.947 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t9AufCetNX66vZIzo+odu7ln1I4= 2022-09-03T06:45:34.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t9AufCetNX66vZIzo+odu7ln1I4=', 'Content-Length': '0'} 2022-09-03T06:45:34.950 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000563f25acbe89ca12-006312f80e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.951 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:34.952 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:34.952 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:34.953 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:34.953 DEBUG:boto:Method: HEAD 2022-09-03T06:45:34.954 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:34.954 DEBUG:boto:Data: 2022-09-03T06:45:34.955 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.955 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:34.956 DEBUG:boto:Port: 8000 2022-09-03T06:45:34.956 DEBUG:boto:Params: {} 2022-09-03T06:45:34.957 DEBUG:boto:Token: None 2022-09-03T06:45:34.958 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:34.959 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t9AufCetNX66vZIzo+odu7ln1I4= 2022-09-03T06:45:34.960 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t9AufCetNX66vZIzo+odu7ln1I4=', 'Content-Length': '0'} 2022-09-03T06:45:34.963 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000748c1753e0707a12-006312f80e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:34.964 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:45:34.965 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:34.966 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:34.967 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:34.968 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:34.969 DEBUG:boto:Method: GET 2022-09-03T06:45:34.970 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:34.972 DEBUG:boto:Data: 2022-09-03T06:45:34.973 DEBUG:boto:Headers: {} 2022-09-03T06:45:34.974 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:34.975 DEBUG:boto:Port: 8001 2022-09-03T06:45:34.976 DEBUG:boto:Params: {} 2022-09-03T06:45:34.977 DEBUG:boto:Token: None 2022-09-03T06:45:34.978 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:34 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:34.979 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XOI7m2G0ZjujxNVNIYAoluj6f0g= 2022-09-03T06:45:34.981 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XOI7m2G0ZjujxNVNIYAoluj6f0g=', 'Content-Length': '0'} 2022-09-03T06:45:34.983 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000056fe97f76d5cf3df-006312f80e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.026 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:35.027 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:35.028 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:35.029 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:35.029 DEBUG:boto:Method: GET 2022-09-03T06:45:35.030 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:35.031 DEBUG:boto:Data: 2022-09-03T06:45:35.031 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.032 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:35.033 DEBUG:boto:Port: 8000 2022-09-03T06:45:35.033 DEBUG:boto:Params: {} 2022-09-03T06:45:35.034 DEBUG:boto:Token: None 2022-09-03T06:45:35.035 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:35.036 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AACBDb2Kt0OUTydtyLtk/st8SVc= 2022-09-03T06:45:35.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AACBDb2Kt0OUTydtyLtk/st8SVc=', 'Content-Length': '0'} 2022-09-03T06:45:35.039 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b365dd4efbb42b0a-006312f80f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.040 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:35.041 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:35.042 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:35.043 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:35.044 DEBUG:boto:Method: GET 2022-09-03T06:45:35.046 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:45:35.047 DEBUG:boto:Data: 2022-09-03T06:45:35.048 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.049 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:35.050 DEBUG:boto:Port: 8001 2022-09-03T06:45:35.052 DEBUG:boto:Params: {} 2022-09-03T06:45:35.053 DEBUG:boto:Token: None 2022-09-03T06:45:35.054 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/ 2022-09-03T06:45:35.056 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU= 2022-09-03T06:45:35.057 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU=', 'Content-Length': '0'} 2022-09-03T06:45:35.061 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000003d6fc521f432b4a-006312f80f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.062 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:35.064 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:35.065 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:35.066 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:35.067 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:35.069 DEBUG:boto:Method: GET 2022-09-03T06:45:35.070 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:45:35.071 DEBUG:boto:Data: 2022-09-03T06:45:35.071 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.072 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:35.073 DEBUG:boto:Port: 8000 2022-09-03T06:45:35.073 DEBUG:boto:Params: {} 2022-09-03T06:45:35.074 DEBUG:boto:Token: None 2022-09-03T06:45:35.075 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/ 2022-09-03T06:45:35.075 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU= 2022-09-03T06:45:35.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU=', 'Content-Length': '0'} 2022-09-03T06:45:35.080 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d98c4709aaea1687-006312f80f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.081 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:35.083 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:35.083 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:35.084 DEBUG:boto:Method: HEAD 2022-09-03T06:45:35.084 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:45:35.085 DEBUG:boto:Data: 2022-09-03T06:45:35.085 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.086 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:35.086 DEBUG:boto:Port: 8001 2022-09-03T06:45:35.087 DEBUG:boto:Params: {} 2022-09-03T06:45:35.088 DEBUG:boto:Token: None 2022-09-03T06:45:35.090 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj38 2022-09-03T06:45:35.090 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N5A++uuSqkTpW2ZMzC9mEjDpf9c= 2022-09-03T06:45:35.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N5A++uuSqkTpW2ZMzC9mEjDpf9c=', 'Content-Length': '0'} 2022-09-03T06:45:35.095 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000631f1de0934a324f-006312f80f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.097 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:35.098 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:35.100 DEBUG:boto:Method: HEAD 2022-09-03T06:45:35.101 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:45:35.102 DEBUG:boto:Data: 2022-09-03T06:45:35.103 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.104 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:35.105 DEBUG:boto:Port: 8000 2022-09-03T06:45:35.106 DEBUG:boto:Params: {} 2022-09-03T06:45:35.107 DEBUG:boto:Token: None 2022-09-03T06:45:35.108 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj38 2022-09-03T06:45:35.109 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N5A++uuSqkTpW2ZMzC9mEjDpf9c= 2022-09-03T06:45:35.111 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N5A++uuSqkTpW2ZMzC9mEjDpf9c=', 'Content-Length': '0'} 2022-09-03T06:45:35.113 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004807c7564d41f156-006312f80f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.115 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:45:35.115 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:35.116 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:35.117 DEBUG:boto:Method: GET 2022-09-03T06:45:35.118 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:45:35.119 DEBUG:boto:Data: 2022-09-03T06:45:35.119 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.120 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:35.121 DEBUG:boto:Port: 8001 2022-09-03T06:45:35.121 DEBUG:boto:Params: {} 2022-09-03T06:45:35.122 DEBUG:boto:Token: None 2022-09-03T06:45:35.123 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj38 2022-09-03T06:45:35.124 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WVf7IEkxkFNVNhUxJmiLT9XWmGA= 2022-09-03T06:45:35.126 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WVf7IEkxkFNVNhUxJmiLT9XWmGA=', 'Content-Length': '0'} 2022-09-03T06:45:35.129 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000000b700a65a3b645d-006312f80f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.130 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:35.131 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:35.131 DEBUG:boto:Method: GET 2022-09-03T06:45:35.132 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:45:35.133 DEBUG:boto:Data: 2022-09-03T06:45:35.134 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.135 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:35.135 DEBUG:boto:Port: 8000 2022-09-03T06:45:35.136 DEBUG:boto:Params: {} 2022-09-03T06:45:35.137 DEBUG:boto:Token: None 2022-09-03T06:45:35.138 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj38 2022-09-03T06:45:35.139 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WVf7IEkxkFNVNhUxJmiLT9XWmGA= 2022-09-03T06:45:35.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WVf7IEkxkFNVNhUxJmiLT9XWmGA=', 'Content-Length': '0'} 2022-09-03T06:45:35.143 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000740316081d2c358d-006312f80f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.190 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:35.191 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:35.192 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:35.192 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:35.193 DEBUG:boto:Method: GET 2022-09-03T06:45:35.193 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:45:35.194 DEBUG:boto:Data: 2022-09-03T06:45:35.195 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.195 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:35.196 DEBUG:boto:Port: 8001 2022-09-03T06:45:35.196 DEBUG:boto:Params: {} 2022-09-03T06:45:35.197 DEBUG:boto:Token: None 2022-09-03T06:45:35.198 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/ 2022-09-03T06:45:35.199 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU= 2022-09-03T06:45:35.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU=', 'Content-Length': '0'} 2022-09-03T06:45:35.203 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005113e81a9f8800eb-006312f80f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.204 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:35.205 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:35.205 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:35.206 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:35.207 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:35.207 DEBUG:boto:Method: GET 2022-09-03T06:45:35.208 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:45:35.208 DEBUG:boto:Data: 2022-09-03T06:45:35.209 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.210 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:35.210 DEBUG:boto:Port: 8000 2022-09-03T06:45:35.211 DEBUG:boto:Params: {} 2022-09-03T06:45:35.213 DEBUG:boto:Token: None 2022-09-03T06:45:35.213 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/ 2022-09-03T06:45:35.214 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU= 2022-09-03T06:45:35.215 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU=', 'Content-Length': '0'} 2022-09-03T06:45:35.219 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b8c00feeafaeb2c8-006312f80f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.220 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:35.221 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:45:35.222 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:35.222 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:35.223 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.224 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.224 DEBUG:boto:Method: GET 2022-09-03T06:45:35.225 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.226 DEBUG:boto:Data: 2022-09-03T06:45:35.226 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.227 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:35.228 DEBUG:boto:Port: 8001 2022-09-03T06:45:35.228 DEBUG:boto:Params: {} 2022-09-03T06:45:35.229 DEBUG:boto:Token: None 2022-09-03T06:45:35.230 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.230 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SxXFYR7eqIzqxvJ3XKIw1Aqnb7w= 2022-09-03T06:45:35.231 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SxXFYR7eqIzqxvJ3XKIw1Aqnb7w=', 'Content-Length': '0'} 2022-09-03T06:45:35.234 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000c72a7f24515b05e-006312f80f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.274 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:35.276 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:35.277 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.278 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.279 DEBUG:boto:Method: GET 2022-09-03T06:45:35.280 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.281 DEBUG:boto:Data: 2022-09-03T06:45:35.282 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.283 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:35.284 DEBUG:boto:Port: 8000 2022-09-03T06:45:35.285 DEBUG:boto:Params: {} 2022-09-03T06:45:35.286 DEBUG:boto:Token: None 2022-09-03T06:45:35.287 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.289 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SxXFYR7eqIzqxvJ3XKIw1Aqnb7w= 2022-09-03T06:45:35.290 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SxXFYR7eqIzqxvJ3XKIw1Aqnb7w=', 'Content-Length': '0'} 2022-09-03T06:45:35.294 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074f12b9d3709f439-006312f80f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.295 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:35.296 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:35.297 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.298 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.299 DEBUG:boto:Method: HEAD 2022-09-03T06:45:35.300 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.301 DEBUG:boto:Data: 2022-09-03T06:45:35.302 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.303 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:35.304 DEBUG:boto:Port: 8001 2022-09-03T06:45:35.304 DEBUG:boto:Params: {} 2022-09-03T06:45:35.305 DEBUG:boto:Token: None 2022-09-03T06:45:35.307 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.308 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BAHnvmmvzA/ePQ4cKTbdciiIuQk= 2022-09-03T06:45:35.309 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BAHnvmmvzA/ePQ4cKTbdciiIuQk=', 'Content-Length': '0'} 2022-09-03T06:45:35.311 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000015ad389c680aca9f-006312f80f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.312 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:35.312 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:35.313 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.314 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.314 DEBUG:boto:Method: HEAD 2022-09-03T06:45:35.315 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.315 DEBUG:boto:Data: 2022-09-03T06:45:35.316 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.316 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:35.317 DEBUG:boto:Port: 8000 2022-09-03T06:45:35.317 DEBUG:boto:Params: {} 2022-09-03T06:45:35.318 DEBUG:boto:Token: None 2022-09-03T06:45:35.318 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.319 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BAHnvmmvzA/ePQ4cKTbdciiIuQk= 2022-09-03T06:45:35.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BAHnvmmvzA/ePQ4cKTbdciiIuQk=', 'Content-Length': '0'} 2022-09-03T06:45:35.322 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000067ff99455b524fdc-006312f80f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.323 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:45:35.323 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:35.324 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:35.324 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.325 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.325 DEBUG:boto:Method: GET 2022-09-03T06:45:35.326 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.326 DEBUG:boto:Data: 2022-09-03T06:45:35.327 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.328 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:35.328 DEBUG:boto:Port: 8001 2022-09-03T06:45:35.329 DEBUG:boto:Params: {} 2022-09-03T06:45:35.329 DEBUG:boto:Token: None 2022-09-03T06:45:35.330 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.331 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SxXFYR7eqIzqxvJ3XKIw1Aqnb7w= 2022-09-03T06:45:35.332 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SxXFYR7eqIzqxvJ3XKIw1Aqnb7w=', 'Content-Length': '0'} 2022-09-03T06:45:35.334 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a6384dd25c1bac63-006312f80f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.335 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:35.335 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:35.336 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.336 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.337 DEBUG:boto:Method: GET 2022-09-03T06:45:35.338 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.338 DEBUG:boto:Data: 2022-09-03T06:45:35.339 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.339 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:35.340 DEBUG:boto:Port: 8000 2022-09-03T06:45:35.340 DEBUG:boto:Params: {} 2022-09-03T06:45:35.341 DEBUG:boto:Token: None 2022-09-03T06:45:35.342 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:35.342 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SxXFYR7eqIzqxvJ3XKIw1Aqnb7w= 2022-09-03T06:45:35.343 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SxXFYR7eqIzqxvJ3XKIw1Aqnb7w=', 'Content-Length': '0'} 2022-09-03T06:45:35.345 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000031ee96f0063527e3-006312f80f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.346 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:35.347 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:35.347 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:35.348 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:35.349 DEBUG:boto:Method: GET 2022-09-03T06:45:35.349 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:45:35.350 DEBUG:boto:Data: 2022-09-03T06:45:35.350 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.351 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:35.351 DEBUG:boto:Port: 8001 2022-09-03T06:45:35.352 DEBUG:boto:Params: {} 2022-09-03T06:45:35.352 DEBUG:boto:Token: None 2022-09-03T06:45:35.353 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/ 2022-09-03T06:45:35.354 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU= 2022-09-03T06:45:35.354 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU=', 'Content-Length': '0'} 2022-09-03T06:45:35.357 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004f2e09b7f2494f00-006312f80f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.358 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:35.359 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:35.360 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:35.361 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:35.361 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:35.362 DEBUG:boto:Method: GET 2022-09-03T06:45:35.362 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:45:35.363 DEBUG:boto:Data: 2022-09-03T06:45:35.364 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.365 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:35.365 DEBUG:boto:Port: 8000 2022-09-03T06:45:35.366 DEBUG:boto:Params: {} 2022-09-03T06:45:35.367 DEBUG:boto:Token: None 2022-09-03T06:45:35.368 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/ 2022-09-03T06:45:35.369 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU= 2022-09-03T06:45:35.370 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU=', 'Content-Length': '0'} 2022-09-03T06:45:35.375 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009f3458feceaea58e-006312f80f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.376 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:35.377 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:35.378 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:35.379 DEBUG:boto:Method: HEAD 2022-09-03T06:45:35.380 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:45:35.381 DEBUG:boto:Data: 2022-09-03T06:45:35.381 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.382 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:35.382 DEBUG:boto:Port: 8001 2022-09-03T06:45:35.383 DEBUG:boto:Params: {} 2022-09-03T06:45:35.383 DEBUG:boto:Token: None 2022-09-03T06:45:35.384 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj44 2022-09-03T06:45:35.384 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3CGDu9VhEeppYmTwyIxhxrsLa5g= 2022-09-03T06:45:35.385 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3CGDu9VhEeppYmTwyIxhxrsLa5g=', 'Content-Length': '0'} 2022-09-03T06:45:35.387 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b05b0ac010d41d95-006312f80f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.388 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:35.389 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:35.389 DEBUG:boto:Method: HEAD 2022-09-03T06:45:35.390 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:45:35.393 DEBUG:boto:Data: 2022-09-03T06:45:35.394 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.395 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:35.396 DEBUG:boto:Port: 8000 2022-09-03T06:45:35.397 DEBUG:boto:Params: {} 2022-09-03T06:45:35.398 DEBUG:boto:Token: None 2022-09-03T06:45:35.399 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj44 2022-09-03T06:45:35.400 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3CGDu9VhEeppYmTwyIxhxrsLa5g= 2022-09-03T06:45:35.401 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3CGDu9VhEeppYmTwyIxhxrsLa5g=', 'Content-Length': '0'} 2022-09-03T06:45:35.404 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e439c0ed8693737-006312f80f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.405 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:45:35.407 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:35.408 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:35.409 DEBUG:boto:Method: GET 2022-09-03T06:45:35.410 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:45:35.411 DEBUG:boto:Data: 2022-09-03T06:45:35.412 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.413 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:35.414 DEBUG:boto:Port: 8001 2022-09-03T06:45:35.415 DEBUG:boto:Params: {} 2022-09-03T06:45:35.416 DEBUG:boto:Token: None 2022-09-03T06:45:35.417 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj44 2022-09-03T06:45:35.418 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G+DvoF7umIlJ/zlroFXUrfTE9gg= 2022-09-03T06:45:35.419 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G+DvoF7umIlJ/zlroFXUrfTE9gg=', 'Content-Length': '0'} 2022-09-03T06:45:35.422 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ce285fc440b4fc4-006312f80f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.423 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:35.424 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:35.425 DEBUG:boto:Method: GET 2022-09-03T06:45:35.427 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:45:35.428 DEBUG:boto:Data: 2022-09-03T06:45:35.429 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.430 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:35.431 DEBUG:boto:Port: 8000 2022-09-03T06:45:35.432 DEBUG:boto:Params: {} 2022-09-03T06:45:35.433 DEBUG:boto:Token: None 2022-09-03T06:45:35.434 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj44 2022-09-03T06:45:35.435 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G+DvoF7umIlJ/zlroFXUrfTE9gg= 2022-09-03T06:45:35.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G+DvoF7umIlJ/zlroFXUrfTE9gg=', 'Content-Length': '0'} 2022-09-03T06:45:35.440 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd2fbefe1e8c4050-006312f80f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.441 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:35.442 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:35.443 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:35.444 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:35.445 DEBUG:boto:Method: GET 2022-09-03T06:45:35.446 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:45:35.447 DEBUG:boto:Data: 2022-09-03T06:45:35.448 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.450 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:35.451 DEBUG:boto:Port: 8001 2022-09-03T06:45:35.452 DEBUG:boto:Params: {} 2022-09-03T06:45:35.453 DEBUG:boto:Token: None 2022-09-03T06:45:35.454 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/ 2022-09-03T06:45:35.455 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU= 2022-09-03T06:45:35.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU=', 'Content-Length': '0'} 2022-09-03T06:45:35.460 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004fe404bbc656eae3-006312f80f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.461 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:35.462 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:35.463 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:35.466 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:35.467 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:35.469 DEBUG:boto:Method: GET 2022-09-03T06:45:35.470 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:45:35.471 DEBUG:boto:Data: 2022-09-03T06:45:35.471 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.472 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:35.472 DEBUG:boto:Port: 8000 2022-09-03T06:45:35.473 DEBUG:boto:Params: {} 2022-09-03T06:45:35.475 DEBUG:boto:Token: None 2022-09-03T06:45:35.476 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/ 2022-09-03T06:45:35.477 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU= 2022-09-03T06:45:35.478 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU=', 'Content-Length': '0'} 2022-09-03T06:45:35.485 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ca1c32d0da2b0337-006312f80f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.487 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:35.492 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:45:35.494 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:35.495 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:35.496 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.498 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.499 DEBUG:boto:Method: GET 2022-09-03T06:45:35.500 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.501 DEBUG:boto:Data: 2022-09-03T06:45:35.502 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.503 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:35.504 DEBUG:boto:Port: 8001 2022-09-03T06:45:35.506 DEBUG:boto:Params: {} 2022-09-03T06:45:35.507 DEBUG:boto:Token: None 2022-09-03T06:45:35.508 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.510 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CgRwaZMNdqFRAL4SaFgpgXakk4A= 2022-09-03T06:45:35.511 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CgRwaZMNdqFRAL4SaFgpgXakk4A=', 'Content-Length': '0'} 2022-09-03T06:45:35.514 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000037cd991cd6c8d5eb-006312f80f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.516 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:35.517 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:35.518 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.520 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.521 DEBUG:boto:Method: GET 2022-09-03T06:45:35.522 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.523 DEBUG:boto:Data: 2022-09-03T06:45:35.524 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.525 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:35.526 DEBUG:boto:Port: 8000 2022-09-03T06:45:35.527 DEBUG:boto:Params: {} 2022-09-03T06:45:35.528 DEBUG:boto:Token: None 2022-09-03T06:45:35.529 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.530 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CgRwaZMNdqFRAL4SaFgpgXakk4A= 2022-09-03T06:45:35.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CgRwaZMNdqFRAL4SaFgpgXakk4A=', 'Content-Length': '0'} 2022-09-03T06:45:35.535 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000ae59cb1258e2afd-006312f80f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.536 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:35.537 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:35.538 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.539 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.540 DEBUG:boto:Method: HEAD 2022-09-03T06:45:35.541 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.542 DEBUG:boto:Data: 2022-09-03T06:45:35.543 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.544 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:35.545 DEBUG:boto:Port: 8001 2022-09-03T06:45:35.546 DEBUG:boto:Params: {} 2022-09-03T06:45:35.547 DEBUG:boto:Token: None 2022-09-03T06:45:35.548 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.550 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rVIb9zgwlYe0kauDcAuBJMcpgD0= 2022-09-03T06:45:35.551 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rVIb9zgwlYe0kauDcAuBJMcpgD0=', 'Content-Length': '0'} 2022-09-03T06:45:35.555 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e72cd1d5c258a779-006312f80f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.556 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:35.557 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:35.558 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.559 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.560 DEBUG:boto:Method: HEAD 2022-09-03T06:45:35.561 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.562 DEBUG:boto:Data: 2022-09-03T06:45:35.563 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.564 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:35.565 DEBUG:boto:Port: 8000 2022-09-03T06:45:35.566 DEBUG:boto:Params: {} 2022-09-03T06:45:35.567 DEBUG:boto:Token: None 2022-09-03T06:45:35.568 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.570 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rVIb9zgwlYe0kauDcAuBJMcpgD0= 2022-09-03T06:45:35.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rVIb9zgwlYe0kauDcAuBJMcpgD0=', 'Content-Length': '0'} 2022-09-03T06:45:35.575 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000051ac931e6b2c7cb0-006312f80f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.576 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:45:35.577 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:35.578 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:35.579 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.580 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.581 DEBUG:boto:Method: GET 2022-09-03T06:45:35.582 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.583 DEBUG:boto:Data: 2022-09-03T06:45:35.584 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.585 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:35.586 DEBUG:boto:Port: 8001 2022-09-03T06:45:35.587 DEBUG:boto:Params: {} 2022-09-03T06:45:35.589 DEBUG:boto:Token: None 2022-09-03T06:45:35.590 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.591 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CgRwaZMNdqFRAL4SaFgpgXakk4A= 2022-09-03T06:45:35.592 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CgRwaZMNdqFRAL4SaFgpgXakk4A=', 'Content-Length': '0'} 2022-09-03T06:45:35.595 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b4a2f7f79c021a84-006312f80f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.642 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:35.651 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:35.652 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.653 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.654 DEBUG:boto:Method: GET 2022-09-03T06:45:35.655 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.662 DEBUG:boto:Data: 2022-09-03T06:45:35.664 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.665 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:35.665 DEBUG:boto:Port: 8000 2022-09-03T06:45:35.671 DEBUG:boto:Params: {} 2022-09-03T06:45:35.674 DEBUG:boto:Token: None 2022-09-03T06:45:35.675 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:35.678 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CgRwaZMNdqFRAL4SaFgpgXakk4A= 2022-09-03T06:45:35.679 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CgRwaZMNdqFRAL4SaFgpgXakk4A=', 'Content-Length': '0'} 2022-09-03T06:45:35.681 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ec2203fb29e25be6-006312f80f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.683 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:35.683 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:35.685 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:35.686 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:35.687 DEBUG:boto:Method: GET 2022-09-03T06:45:35.688 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:45:35.688 DEBUG:boto:Data: 2022-09-03T06:45:35.689 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.690 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:35.691 DEBUG:boto:Port: 8001 2022-09-03T06:45:35.692 DEBUG:boto:Params: {} 2022-09-03T06:45:35.693 DEBUG:boto:Token: None 2022-09-03T06:45:35.694 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/ 2022-09-03T06:45:35.695 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU= 2022-09-03T06:45:35.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU=', 'Content-Length': '0'} 2022-09-03T06:45:35.700 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000131da91e341dd393-006312f80f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.700 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:35.702 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:35.702 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:35.703 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:35.704 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:35.704 DEBUG:boto:Method: GET 2022-09-03T06:45:35.705 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:45:35.706 DEBUG:boto:Data: 2022-09-03T06:45:35.707 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.708 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:35.709 DEBUG:boto:Port: 8000 2022-09-03T06:45:35.710 DEBUG:boto:Params: {} 2022-09-03T06:45:35.711 DEBUG:boto:Token: None 2022-09-03T06:45:35.712 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/ 2022-09-03T06:45:35.713 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU= 2022-09-03T06:45:35.714 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU=', 'Content-Length': '0'} 2022-09-03T06:45:35.719 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000065e62466b928a9e4-006312f80f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.720 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:35.721 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:35.721 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:35.722 DEBUG:boto:Method: HEAD 2022-09-03T06:45:35.722 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:45:35.723 DEBUG:boto:Data: 2022-09-03T06:45:35.723 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.724 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:35.725 DEBUG:boto:Port: 8001 2022-09-03T06:45:35.725 DEBUG:boto:Params: {} 2022-09-03T06:45:35.726 DEBUG:boto:Token: None 2022-09-03T06:45:35.727 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj46 2022-09-03T06:45:35.728 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+XABF3/R2SXZtrXsj2YKPtF0VjU= 2022-09-03T06:45:35.729 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+XABF3/R2SXZtrXsj2YKPtF0VjU=', 'Content-Length': '0'} 2022-09-03T06:45:35.734 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a55527715dcef070-006312f80f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.735 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:35.736 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:35.737 DEBUG:boto:Method: HEAD 2022-09-03T06:45:35.738 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:45:35.738 DEBUG:boto:Data: 2022-09-03T06:45:35.739 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.740 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:35.741 DEBUG:boto:Port: 8000 2022-09-03T06:45:35.742 DEBUG:boto:Params: {} 2022-09-03T06:45:35.742 DEBUG:boto:Token: None 2022-09-03T06:45:35.743 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj46 2022-09-03T06:45:35.744 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+XABF3/R2SXZtrXsj2YKPtF0VjU= 2022-09-03T06:45:35.745 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+XABF3/R2SXZtrXsj2YKPtF0VjU=', 'Content-Length': '0'} 2022-09-03T06:45:35.747 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000092fa4de212378ba1-006312f80f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.748 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:45:35.748 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:35.749 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:35.749 DEBUG:boto:Method: GET 2022-09-03T06:45:35.750 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:45:35.751 DEBUG:boto:Data: 2022-09-03T06:45:35.752 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.752 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:35.753 DEBUG:boto:Port: 8001 2022-09-03T06:45:35.753 DEBUG:boto:Params: {} 2022-09-03T06:45:35.755 DEBUG:boto:Token: None 2022-09-03T06:45:35.755 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj46 2022-09-03T06:45:35.756 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:78C7muwHU34kn9Q7F9SwXDYXfRg= 2022-09-03T06:45:35.757 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:78C7muwHU34kn9Q7F9SwXDYXfRg=', 'Content-Length': '0'} 2022-09-03T06:45:35.760 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e25182bb219ee7a-006312f80f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.760 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:35.761 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:35.762 DEBUG:boto:Method: GET 2022-09-03T06:45:35.762 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:45:35.763 DEBUG:boto:Data: 2022-09-03T06:45:35.763 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.764 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:35.765 DEBUG:boto:Port: 8000 2022-09-03T06:45:35.765 DEBUG:boto:Params: {} 2022-09-03T06:45:35.766 DEBUG:boto:Token: None 2022-09-03T06:45:35.766 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj46 2022-09-03T06:45:35.767 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:78C7muwHU34kn9Q7F9SwXDYXfRg= 2022-09-03T06:45:35.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:78C7muwHU34kn9Q7F9SwXDYXfRg=', 'Content-Length': '0'} 2022-09-03T06:45:35.770 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fdf7c112316e1d7d-006312f80f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.814 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:35.816 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:35.816 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:35.817 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:35.818 DEBUG:boto:Method: GET 2022-09-03T06:45:35.819 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:45:35.820 DEBUG:boto:Data: 2022-09-03T06:45:35.821 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.822 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:35.823 DEBUG:boto:Port: 8001 2022-09-03T06:45:35.824 DEBUG:boto:Params: {} 2022-09-03T06:45:35.826 DEBUG:boto:Token: None 2022-09-03T06:45:35.827 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/ 2022-09-03T06:45:35.828 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU= 2022-09-03T06:45:35.829 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU=', 'Content-Length': '0'} 2022-09-03T06:45:35.833 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004f624013b931d828-006312f80f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.837 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:35.838 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:35.839 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:35.840 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:35.841 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:35.842 DEBUG:boto:Method: GET 2022-09-03T06:45:35.843 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:45:35.843 DEBUG:boto:Data: 2022-09-03T06:45:35.844 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.845 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:35.846 DEBUG:boto:Port: 8000 2022-09-03T06:45:35.847 DEBUG:boto:Params: {} 2022-09-03T06:45:35.848 DEBUG:boto:Token: None 2022-09-03T06:45:35.849 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/ 2022-09-03T06:45:35.851 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU= 2022-09-03T06:45:35.851 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU=', 'Content-Length': '0'} 2022-09-03T06:45:35.855 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a0e21fe5b7b25b0d-006312f80f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.856 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:35.857 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:45:35.858 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:35.859 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:35.860 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.860 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.861 DEBUG:boto:Method: GET 2022-09-03T06:45:35.861 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.862 DEBUG:boto:Data: 2022-09-03T06:45:35.863 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.863 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:35.864 DEBUG:boto:Port: 8001 2022-09-03T06:45:35.864 DEBUG:boto:Params: {} 2022-09-03T06:45:35.865 DEBUG:boto:Token: None 2022-09-03T06:45:35.866 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.867 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pcJW90oOFPGU3xC4ShuBa95AR/g= 2022-09-03T06:45:35.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pcJW90oOFPGU3xC4ShuBa95AR/g=', 'Content-Length': '0'} 2022-09-03T06:45:35.872 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cd9e2ae8dfb105c0-006312f80f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.874 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:35.875 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:35.876 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.877 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.878 DEBUG:boto:Method: GET 2022-09-03T06:45:35.879 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.881 DEBUG:boto:Data: 2022-09-03T06:45:35.882 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.883 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:35.884 DEBUG:boto:Port: 8000 2022-09-03T06:45:35.886 DEBUG:boto:Params: {} 2022-09-03T06:45:35.886 DEBUG:boto:Token: None 2022-09-03T06:45:35.888 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.889 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pcJW90oOFPGU3xC4ShuBa95AR/g= 2022-09-03T06:45:35.889 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pcJW90oOFPGU3xC4ShuBa95AR/g=', 'Content-Length': '0'} 2022-09-03T06:45:35.892 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b326688e70a7e9e-006312f80f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.893 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:35.894 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:35.894 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.895 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.895 DEBUG:boto:Method: HEAD 2022-09-03T06:45:35.896 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.896 DEBUG:boto:Data: 2022-09-03T06:45:35.897 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.898 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:35.898 DEBUG:boto:Port: 8001 2022-09-03T06:45:35.899 DEBUG:boto:Params: {} 2022-09-03T06:45:35.900 DEBUG:boto:Token: None 2022-09-03T06:45:35.900 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.901 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UmmStvQQ8IWzvdtX3efIbUY2PxA= 2022-09-03T06:45:35.902 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UmmStvQQ8IWzvdtX3efIbUY2PxA=', 'Content-Length': '0'} 2022-09-03T06:45:35.904 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ddbde7f7469c27fb-006312f80f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.905 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:35.905 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:35.906 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.907 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.907 DEBUG:boto:Method: HEAD 2022-09-03T06:45:35.908 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.908 DEBUG:boto:Data: 2022-09-03T06:45:35.909 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.909 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:35.910 DEBUG:boto:Port: 8000 2022-09-03T06:45:35.910 DEBUG:boto:Params: {} 2022-09-03T06:45:35.911 DEBUG:boto:Token: None 2022-09-03T06:45:35.911 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.912 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UmmStvQQ8IWzvdtX3efIbUY2PxA= 2022-09-03T06:45:35.913 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UmmStvQQ8IWzvdtX3efIbUY2PxA=', 'Content-Length': '0'} 2022-09-03T06:45:35.915 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c5bd568a0c06fd35-006312f80f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.916 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:45:35.916 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:35.917 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:35.917 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.918 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.918 DEBUG:boto:Method: GET 2022-09-03T06:45:35.919 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.920 DEBUG:boto:Data: 2022-09-03T06:45:35.920 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.921 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:35.921 DEBUG:boto:Port: 8001 2022-09-03T06:45:35.922 DEBUG:boto:Params: {} 2022-09-03T06:45:35.922 DEBUG:boto:Token: None 2022-09-03T06:45:35.923 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.924 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pcJW90oOFPGU3xC4ShuBa95AR/g= 2022-09-03T06:45:35.924 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pcJW90oOFPGU3xC4ShuBa95AR/g=', 'Content-Length': '0'} 2022-09-03T06:45:35.929 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000097bc7251cafb68e3-006312f80f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.930 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:35.931 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:35.931 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.932 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.932 DEBUG:boto:Method: GET 2022-09-03T06:45:35.933 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.933 DEBUG:boto:Data: 2022-09-03T06:45:35.934 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.934 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:35.935 DEBUG:boto:Port: 8000 2022-09-03T06:45:35.935 DEBUG:boto:Params: {} 2022-09-03T06:45:35.936 DEBUG:boto:Token: None 2022-09-03T06:45:35.937 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:35.937 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pcJW90oOFPGU3xC4ShuBa95AR/g= 2022-09-03T06:45:35.938 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pcJW90oOFPGU3xC4ShuBa95AR/g=', 'Content-Length': '0'} 2022-09-03T06:45:35.940 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000059a53591f42c94c0-006312f80f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:35.986 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:35.987 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:35.988 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:35.988 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:35.989 DEBUG:boto:Method: GET 2022-09-03T06:45:35.990 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:45:35.990 DEBUG:boto:Data: 2022-09-03T06:45:35.991 DEBUG:boto:Headers: {} 2022-09-03T06:45:35.991 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:35.992 DEBUG:boto:Port: 8001 2022-09-03T06:45:35.992 DEBUG:boto:Params: {} 2022-09-03T06:45:35.994 DEBUG:boto:Token: None 2022-09-03T06:45:35.994 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:35 GMT /dsgvcy-75/ 2022-09-03T06:45:35.995 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU= 2022-09-03T06:45:35.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zgtadttlZcHPh9Wgy/55dkliJKU=', 'Content-Length': '0'} 2022-09-03T06:45:35.999 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000115215b182d46d39-006312f80f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.000 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:36.001 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:36.001 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:36.002 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:36.002 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:36.003 DEBUG:boto:Method: GET 2022-09-03T06:45:36.003 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:45:36.004 DEBUG:boto:Data: 2022-09-03T06:45:36.005 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.005 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.006 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.006 DEBUG:boto:Params: {} 2022-09-03T06:45:36.007 DEBUG:boto:Token: None 2022-09-03T06:45:36.009 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/ 2022-09-03T06:45:36.010 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns= 2022-09-03T06:45:36.011 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns=', 'Content-Length': '0'} 2022-09-03T06:45:36.016 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c8a920d60cb45e2c-006312f810-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.017 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:36.017 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:36.018 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:36.019 DEBUG:boto:Method: HEAD 2022-09-03T06:45:36.019 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:45:36.020 DEBUG:boto:Data: 2022-09-03T06:45:36.020 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.021 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:36.021 DEBUG:boto:Port: 8001 2022-09-03T06:45:36.022 DEBUG:boto:Params: {} 2022-09-03T06:45:36.023 DEBUG:boto:Token: None 2022-09-03T06:45:36.023 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj51 2022-09-03T06:45:36.024 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vhDG0PRIOmGetAhvhJP3a8/YMAo= 2022-09-03T06:45:36.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vhDG0PRIOmGetAhvhJP3a8/YMAo=', 'Content-Length': '0'} 2022-09-03T06:45:36.027 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a4c4f34e697c36a-006312f810-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.028 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:36.029 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:36.031 DEBUG:boto:Method: HEAD 2022-09-03T06:45:36.033 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:45:36.034 DEBUG:boto:Data: 2022-09-03T06:45:36.035 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.036 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.037 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.037 DEBUG:boto:Params: {} 2022-09-03T06:45:36.038 DEBUG:boto:Token: None 2022-09-03T06:45:36.038 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj51 2022-09-03T06:45:36.039 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vhDG0PRIOmGetAhvhJP3a8/YMAo= 2022-09-03T06:45:36.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vhDG0PRIOmGetAhvhJP3a8/YMAo=', 'Content-Length': '0'} 2022-09-03T06:45:36.042 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000044329a7e1bb06ef8-006312f810-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.045 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:45:36.046 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:36.047 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:36.047 DEBUG:boto:Method: GET 2022-09-03T06:45:36.048 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:45:36.048 DEBUG:boto:Data: 2022-09-03T06:45:36.049 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.050 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:36.051 DEBUG:boto:Port: 8001 2022-09-03T06:45:36.051 DEBUG:boto:Params: {} 2022-09-03T06:45:36.052 DEBUG:boto:Token: None 2022-09-03T06:45:36.053 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj51 2022-09-03T06:45:36.054 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1CJegoFP1XSXRLh3aoIqs+VeuSs= 2022-09-03T06:45:36.055 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1CJegoFP1XSXRLh3aoIqs+VeuSs=', 'Content-Length': '0'} 2022-09-03T06:45:36.058 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000088b349fb42b1f04-006312f810-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.103 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:36.106 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:36.107 DEBUG:boto:Method: GET 2022-09-03T06:45:36.109 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:45:36.110 DEBUG:boto:Data: 2022-09-03T06:45:36.111 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.112 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.114 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.116 DEBUG:boto:Params: {} 2022-09-03T06:45:36.117 DEBUG:boto:Token: None 2022-09-03T06:45:36.118 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj51 2022-09-03T06:45:36.118 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1CJegoFP1XSXRLh3aoIqs+VeuSs= 2022-09-03T06:45:36.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1CJegoFP1XSXRLh3aoIqs+VeuSs=', 'Content-Length': '0'} 2022-09-03T06:45:36.121 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000047932e240129126b-006312f810-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.122 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:36.123 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:36.124 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:36.125 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:36.127 DEBUG:boto:Method: GET 2022-09-03T06:45:36.127 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:45:36.128 DEBUG:boto:Data: 2022-09-03T06:45:36.129 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.129 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:36.130 DEBUG:boto:Port: 8001 2022-09-03T06:45:36.130 DEBUG:boto:Params: {} 2022-09-03T06:45:36.131 DEBUG:boto:Token: None 2022-09-03T06:45:36.132 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/ 2022-09-03T06:45:36.133 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns= 2022-09-03T06:45:36.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns=', 'Content-Length': '0'} 2022-09-03T06:45:36.140 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000899e0f8a29a67497-006312f810-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.141 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:36.143 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:36.143 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:36.144 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:36.145 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:36.146 DEBUG:boto:Method: GET 2022-09-03T06:45:36.147 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:45:36.148 DEBUG:boto:Data: 2022-09-03T06:45:36.149 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.150 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.151 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.152 DEBUG:boto:Params: {} 2022-09-03T06:45:36.153 DEBUG:boto:Token: None 2022-09-03T06:45:36.154 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/ 2022-09-03T06:45:36.155 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns= 2022-09-03T06:45:36.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns=', 'Content-Length': '0'} 2022-09-03T06:45:36.161 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000319a1dedb3dde553-006312f810-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.162 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:36.163 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:45:36.164 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:36.165 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:36.166 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.166 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.167 DEBUG:boto:Method: GET 2022-09-03T06:45:36.167 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.169 DEBUG:boto:Data: 2022-09-03T06:45:36.170 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.171 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:36.172 DEBUG:boto:Port: 8001 2022-09-03T06:45:36.173 DEBUG:boto:Params: {} 2022-09-03T06:45:36.174 DEBUG:boto:Token: None 2022-09-03T06:45:36.175 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.176 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3nfs05pEMChcK5W480LEJy18yew= 2022-09-03T06:45:36.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3nfs05pEMChcK5W480LEJy18yew=', 'Content-Length': '0'} 2022-09-03T06:45:36.179 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f6f56ee57ab54e4a-006312f810-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.180 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:36.180 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:36.181 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.182 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.182 DEBUG:boto:Method: GET 2022-09-03T06:45:36.183 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.183 DEBUG:boto:Data: 2022-09-03T06:45:36.184 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.184 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.185 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.185 DEBUG:boto:Params: {} 2022-09-03T06:45:36.186 DEBUG:boto:Token: None 2022-09-03T06:45:36.187 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.188 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3nfs05pEMChcK5W480LEJy18yew= 2022-09-03T06:45:36.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3nfs05pEMChcK5W480LEJy18yew=', 'Content-Length': '0'} 2022-09-03T06:45:36.191 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009577651e4d443b2c-006312f810-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.238 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:36.239 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:36.240 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.241 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.242 DEBUG:boto:Method: HEAD 2022-09-03T06:45:36.243 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.244 DEBUG:boto:Data: 2022-09-03T06:45:36.245 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.245 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:36.246 DEBUG:boto:Port: 8001 2022-09-03T06:45:36.247 DEBUG:boto:Params: {} 2022-09-03T06:45:36.248 DEBUG:boto:Token: None 2022-09-03T06:45:36.249 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.249 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WNj1S17icL9fGLRskSRN7Bo7Uv4= 2022-09-03T06:45:36.250 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WNj1S17icL9fGLRskSRN7Bo7Uv4=', 'Content-Length': '0'} 2022-09-03T06:45:36.253 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d0397b609ac37177-006312f810-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.253 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:36.254 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:36.255 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.255 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.256 DEBUG:boto:Method: HEAD 2022-09-03T06:45:36.257 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.258 DEBUG:boto:Data: 2022-09-03T06:45:36.259 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.260 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.261 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.262 DEBUG:boto:Params: {} 2022-09-03T06:45:36.263 DEBUG:boto:Token: None 2022-09-03T06:45:36.263 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.264 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WNj1S17icL9fGLRskSRN7Bo7Uv4= 2022-09-03T06:45:36.265 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WNj1S17icL9fGLRskSRN7Bo7Uv4=', 'Content-Length': '0'} 2022-09-03T06:45:36.267 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001229c951a2193a56-006312f810-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.269 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:45:36.269 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:36.270 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:36.271 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.272 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.273 DEBUG:boto:Method: GET 2022-09-03T06:45:36.274 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.275 DEBUG:boto:Data: 2022-09-03T06:45:36.276 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.276 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:36.277 DEBUG:boto:Port: 8001 2022-09-03T06:45:36.278 DEBUG:boto:Params: {} 2022-09-03T06:45:36.279 DEBUG:boto:Token: None 2022-09-03T06:45:36.280 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.281 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3nfs05pEMChcK5W480LEJy18yew= 2022-09-03T06:45:36.282 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3nfs05pEMChcK5W480LEJy18yew=', 'Content-Length': '0'} 2022-09-03T06:45:36.285 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007c090fc6fdb32bfe-006312f810-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.286 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:36.287 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:36.288 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.288 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.289 DEBUG:boto:Method: GET 2022-09-03T06:45:36.290 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.291 DEBUG:boto:Data: 2022-09-03T06:45:36.291 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.292 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.292 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.293 DEBUG:boto:Params: {} 2022-09-03T06:45:36.294 DEBUG:boto:Token: None 2022-09-03T06:45:36.295 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:36.296 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3nfs05pEMChcK5W480LEJy18yew= 2022-09-03T06:45:36.297 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3nfs05pEMChcK5W480LEJy18yew=', 'Content-Length': '0'} 2022-09-03T06:45:36.300 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009b394bf2306f765c-006312f810-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.300 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:36.301 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:36.302 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:36.302 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:36.303 DEBUG:boto:Method: GET 2022-09-03T06:45:36.303 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:45:36.304 DEBUG:boto:Data: 2022-09-03T06:45:36.304 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.305 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:36.305 DEBUG:boto:Port: 8001 2022-09-03T06:45:36.306 DEBUG:boto:Params: {} 2022-09-03T06:45:36.307 DEBUG:boto:Token: None 2022-09-03T06:45:36.307 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/ 2022-09-03T06:45:36.308 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns= 2022-09-03T06:45:36.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns=', 'Content-Length': '0'} 2022-09-03T06:45:36.311 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d59dbb777689c2a5-006312f810-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.312 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:36.313 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:36.314 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:36.314 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:36.315 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:36.315 DEBUG:boto:Method: GET 2022-09-03T06:45:36.316 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:45:36.316 DEBUG:boto:Data: 2022-09-03T06:45:36.317 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.317 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.318 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.318 DEBUG:boto:Params: {} 2022-09-03T06:45:36.319 DEBUG:boto:Token: None 2022-09-03T06:45:36.319 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/ 2022-09-03T06:45:36.320 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns= 2022-09-03T06:45:36.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns=', 'Content-Length': '0'} 2022-09-03T06:45:36.325 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eb0e6d51d42052f2-006312f810-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.326 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:36.327 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:36.327 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:36.328 DEBUG:boto:Method: HEAD 2022-09-03T06:45:36.328 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:45:36.329 DEBUG:boto:Data: 2022-09-03T06:45:36.330 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.330 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:36.331 DEBUG:boto:Port: 8001 2022-09-03T06:45:36.332 DEBUG:boto:Params: {} 2022-09-03T06:45:36.332 DEBUG:boto:Token: None 2022-09-03T06:45:36.333 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj55 2022-09-03T06:45:36.333 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M91IyESFf/qQK3boLtRQkasq8yU= 2022-09-03T06:45:36.334 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M91IyESFf/qQK3boLtRQkasq8yU=', 'Content-Length': '0'} 2022-09-03T06:45:36.336 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007f9b4fdedcc5fe98-006312f810-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.337 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:36.338 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:36.338 DEBUG:boto:Method: HEAD 2022-09-03T06:45:36.339 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:45:36.339 DEBUG:boto:Data: 2022-09-03T06:45:36.340 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.340 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.341 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.341 DEBUG:boto:Params: {} 2022-09-03T06:45:36.342 DEBUG:boto:Token: None 2022-09-03T06:45:36.342 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj55 2022-09-03T06:45:36.343 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:M91IyESFf/qQK3boLtRQkasq8yU= 2022-09-03T06:45:36.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:M91IyESFf/qQK3boLtRQkasq8yU=', 'Content-Length': '0'} 2022-09-03T06:45:36.346 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007bfe94959da17cca-006312f810-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.347 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:45:36.348 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:36.348 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:36.349 DEBUG:boto:Method: GET 2022-09-03T06:45:36.349 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:45:36.350 DEBUG:boto:Data: 2022-09-03T06:45:36.350 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.351 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:36.352 DEBUG:boto:Port: 8001 2022-09-03T06:45:36.352 DEBUG:boto:Params: {} 2022-09-03T06:45:36.353 DEBUG:boto:Token: None 2022-09-03T06:45:36.354 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj55 2022-09-03T06:45:36.354 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+hurDZm/pOHZ/bdMZ+ZNQ+r21Tk= 2022-09-03T06:45:36.355 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+hurDZm/pOHZ/bdMZ+ZNQ+r21Tk=', 'Content-Length': '0'} 2022-09-03T06:45:36.358 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c1c3e593411decbb-006312f810-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.398 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:36.399 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:36.399 DEBUG:boto:Method: GET 2022-09-03T06:45:36.400 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:45:36.401 DEBUG:boto:Data: 2022-09-03T06:45:36.401 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.402 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.402 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.403 DEBUG:boto:Params: {} 2022-09-03T06:45:36.404 DEBUG:boto:Token: None 2022-09-03T06:45:36.404 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj55 2022-09-03T06:45:36.405 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+hurDZm/pOHZ/bdMZ+ZNQ+r21Tk= 2022-09-03T06:45:36.406 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+hurDZm/pOHZ/bdMZ+ZNQ+r21Tk=', 'Content-Length': '0'} 2022-09-03T06:45:36.408 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000049c6f1fd2157bc67-006312f810-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.419 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:36.420 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:36.420 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:36.421 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:36.422 DEBUG:boto:Method: GET 2022-09-03T06:45:36.422 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:45:36.423 DEBUG:boto:Data: 2022-09-03T06:45:36.423 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.424 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:36.424 DEBUG:boto:Port: 8001 2022-09-03T06:45:36.425 DEBUG:boto:Params: {} 2022-09-03T06:45:36.425 DEBUG:boto:Token: None 2022-09-03T06:45:36.426 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/ 2022-09-03T06:45:36.427 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns= 2022-09-03T06:45:36.427 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns=', 'Content-Length': '0'} 2022-09-03T06:45:36.430 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ab532ce71607ecb2-006312f810-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.431 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:36.432 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:36.432 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:36.433 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:36.433 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:36.434 DEBUG:boto:Method: GET 2022-09-03T06:45:36.435 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:45:36.435 DEBUG:boto:Data: 2022-09-03T06:45:36.436 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.436 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.437 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.437 DEBUG:boto:Params: {} 2022-09-03T06:45:36.438 DEBUG:boto:Token: None 2022-09-03T06:45:36.438 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/ 2022-09-03T06:45:36.439 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns= 2022-09-03T06:45:36.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns=', 'Content-Length': '0'} 2022-09-03T06:45:36.443 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000514b377ffd25e4fb-006312f810-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.486 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:36.487 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:45:36.488 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:36.488 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:36.489 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.490 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.490 DEBUG:boto:Method: GET 2022-09-03T06:45:36.491 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.491 DEBUG:boto:Data: 2022-09-03T06:45:36.492 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.493 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:36.493 DEBUG:boto:Port: 8001 2022-09-03T06:45:36.495 DEBUG:boto:Params: {} 2022-09-03T06:45:36.497 DEBUG:boto:Token: None 2022-09-03T06:45:36.498 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.499 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y21ll+NEbTqWW5aJ0rff3BQiDZ0= 2022-09-03T06:45:36.500 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y21ll+NEbTqWW5aJ0rff3BQiDZ0=', 'Content-Length': '0'} 2022-09-03T06:45:36.503 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aa8d1ac78180904c-006312f810-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.504 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:36.505 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:36.506 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.507 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.508 DEBUG:boto:Method: GET 2022-09-03T06:45:36.509 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.509 DEBUG:boto:Data: 2022-09-03T06:45:36.510 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.510 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.511 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.512 DEBUG:boto:Params: {} 2022-09-03T06:45:36.513 DEBUG:boto:Token: None 2022-09-03T06:45:36.513 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.514 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y21ll+NEbTqWW5aJ0rff3BQiDZ0= 2022-09-03T06:45:36.515 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y21ll+NEbTqWW5aJ0rff3BQiDZ0=', 'Content-Length': '0'} 2022-09-03T06:45:36.517 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e07841406c2a54e1-006312f810-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.518 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:36.519 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:36.520 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.520 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.521 DEBUG:boto:Method: HEAD 2022-09-03T06:45:36.522 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.522 DEBUG:boto:Data: 2022-09-03T06:45:36.523 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.524 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:36.524 DEBUG:boto:Port: 8001 2022-09-03T06:45:36.525 DEBUG:boto:Params: {} 2022-09-03T06:45:36.526 DEBUG:boto:Token: None 2022-09-03T06:45:36.527 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.527 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KJBqmjgsttJD+0ZQr7a+aa0CMHU= 2022-09-03T06:45:36.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KJBqmjgsttJD+0ZQr7a+aa0CMHU=', 'Content-Length': '0'} 2022-09-03T06:45:36.530 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c103607205578d6c-006312f810-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.531 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:36.532 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:36.533 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.533 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.534 DEBUG:boto:Method: HEAD 2022-09-03T06:45:36.535 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.535 DEBUG:boto:Data: 2022-09-03T06:45:36.536 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.537 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.537 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.538 DEBUG:boto:Params: {} 2022-09-03T06:45:36.538 DEBUG:boto:Token: None 2022-09-03T06:45:36.539 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.540 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KJBqmjgsttJD+0ZQr7a+aa0CMHU= 2022-09-03T06:45:36.541 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KJBqmjgsttJD+0ZQr7a+aa0CMHU=', 'Content-Length': '0'} 2022-09-03T06:45:36.543 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000441c7791416b0f5f-006312f810-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.544 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:45:36.545 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:36.545 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:36.546 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.546 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.547 DEBUG:boto:Method: GET 2022-09-03T06:45:36.547 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.548 DEBUG:boto:Data: 2022-09-03T06:45:36.549 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.549 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:36.550 DEBUG:boto:Port: 8001 2022-09-03T06:45:36.550 DEBUG:boto:Params: {} 2022-09-03T06:45:36.551 DEBUG:boto:Token: None 2022-09-03T06:45:36.552 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.552 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y21ll+NEbTqWW5aJ0rff3BQiDZ0= 2022-09-03T06:45:36.553 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y21ll+NEbTqWW5aJ0rff3BQiDZ0=', 'Content-Length': '0'} 2022-09-03T06:45:36.555 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b6dd91233d53e786-006312f810-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.598 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:36.599 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:36.600 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.600 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.601 DEBUG:boto:Method: GET 2022-09-03T06:45:36.601 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.602 DEBUG:boto:Data: 2022-09-03T06:45:36.602 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.603 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.604 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.604 DEBUG:boto:Params: {} 2022-09-03T06:45:36.605 DEBUG:boto:Token: None 2022-09-03T06:45:36.606 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:36.607 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y21ll+NEbTqWW5aJ0rff3BQiDZ0= 2022-09-03T06:45:36.607 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y21ll+NEbTqWW5aJ0rff3BQiDZ0=', 'Content-Length': '0'} 2022-09-03T06:45:36.609 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b60e600afb502c1c-006312f810-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.610 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:36.611 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:36.612 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:36.612 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:36.613 DEBUG:boto:Method: GET 2022-09-03T06:45:36.613 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:45:36.614 DEBUG:boto:Data: 2022-09-03T06:45:36.615 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.616 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:36.617 DEBUG:boto:Port: 8001 2022-09-03T06:45:36.618 DEBUG:boto:Params: {} 2022-09-03T06:45:36.619 DEBUG:boto:Token: None 2022-09-03T06:45:36.620 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/ 2022-09-03T06:45:36.621 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns= 2022-09-03T06:45:36.622 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns=', 'Content-Length': '0'} 2022-09-03T06:45:36.626 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000674dd5ef30fad64a-006312f810-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.627 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:36.628 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:36.629 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:36.629 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:36.630 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:36.630 DEBUG:boto:Method: GET 2022-09-03T06:45:36.631 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:45:36.631 DEBUG:boto:Data: 2022-09-03T06:45:36.632 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.633 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.633 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.634 DEBUG:boto:Params: {} 2022-09-03T06:45:36.634 DEBUG:boto:Token: None 2022-09-03T06:45:36.635 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/ 2022-09-03T06:45:36.636 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns= 2022-09-03T06:45:36.637 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns=', 'Content-Length': '0'} 2022-09-03T06:45:36.641 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000041ebcaf6b705d75-006312f810-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.642 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:36.642 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:36.643 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:36.644 DEBUG:boto:Method: HEAD 2022-09-03T06:45:36.644 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:45:36.645 DEBUG:boto:Data: 2022-09-03T06:45:36.645 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.646 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:36.646 DEBUG:boto:Port: 8001 2022-09-03T06:45:36.647 DEBUG:boto:Params: {} 2022-09-03T06:45:36.647 DEBUG:boto:Token: None 2022-09-03T06:45:36.648 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj57 2022-09-03T06:45:36.649 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gPFxkG/qzfiSzGP27ylfMlzjY8Y= 2022-09-03T06:45:36.649 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gPFxkG/qzfiSzGP27ylfMlzjY8Y=', 'Content-Length': '0'} 2022-09-03T06:45:36.651 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a357732ff73b909c-006312f810-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.652 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:36.653 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:36.653 DEBUG:boto:Method: HEAD 2022-09-03T06:45:36.654 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:45:36.654 DEBUG:boto:Data: 2022-09-03T06:45:36.655 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.655 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.656 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.657 DEBUG:boto:Params: {} 2022-09-03T06:45:36.657 DEBUG:boto:Token: None 2022-09-03T06:45:36.658 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj57 2022-09-03T06:45:36.658 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gPFxkG/qzfiSzGP27ylfMlzjY8Y= 2022-09-03T06:45:36.659 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gPFxkG/qzfiSzGP27ylfMlzjY8Y=', 'Content-Length': '0'} 2022-09-03T06:45:36.662 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cfb2ed54192dfbd0-006312f810-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.662 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:45:36.663 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:36.664 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:36.664 DEBUG:boto:Method: GET 2022-09-03T06:45:36.665 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:45:36.665 DEBUG:boto:Data: 2022-09-03T06:45:36.666 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.666 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:36.667 DEBUG:boto:Port: 8001 2022-09-03T06:45:36.667 DEBUG:boto:Params: {} 2022-09-03T06:45:36.668 DEBUG:boto:Token: None 2022-09-03T06:45:36.669 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj57 2022-09-03T06:45:36.669 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m2G8htp4CzlJKKkoBYaAIOQi3XM= 2022-09-03T06:45:36.670 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m2G8htp4CzlJKKkoBYaAIOQi3XM=', 'Content-Length': '0'} 2022-09-03T06:45:36.672 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000245fe7b8974d0d70-006312f810-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.673 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:36.673 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:36.674 DEBUG:boto:Method: GET 2022-09-03T06:45:36.675 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:45:36.675 DEBUG:boto:Data: 2022-09-03T06:45:36.676 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.677 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.677 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.678 DEBUG:boto:Params: {} 2022-09-03T06:45:36.678 DEBUG:boto:Token: None 2022-09-03T06:45:36.679 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj57 2022-09-03T06:45:36.680 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m2G8htp4CzlJKKkoBYaAIOQi3XM= 2022-09-03T06:45:36.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m2G8htp4CzlJKKkoBYaAIOQi3XM=', 'Content-Length': '0'} 2022-09-03T06:45:36.683 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004e202c318beb718a-006312f810-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.684 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:36.685 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:36.686 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:36.687 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:36.688 DEBUG:boto:Method: GET 2022-09-03T06:45:36.689 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:45:36.690 DEBUG:boto:Data: 2022-09-03T06:45:36.691 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.693 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:36.694 DEBUG:boto:Port: 8001 2022-09-03T06:45:36.695 DEBUG:boto:Params: {} 2022-09-03T06:45:36.696 DEBUG:boto:Token: None 2022-09-03T06:45:36.696 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/ 2022-09-03T06:45:36.697 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns= 2022-09-03T06:45:36.698 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns=', 'Content-Length': '0'} 2022-09-03T06:45:36.701 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005f608159f417da71-006312f810-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.702 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:36.703 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:36.704 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:36.704 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:36.705 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:36.705 DEBUG:boto:Method: GET 2022-09-03T06:45:36.706 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:45:36.707 DEBUG:boto:Data: 2022-09-03T06:45:36.707 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.708 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.709 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.709 DEBUG:boto:Params: {} 2022-09-03T06:45:36.710 DEBUG:boto:Token: None 2022-09-03T06:45:36.711 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/ 2022-09-03T06:45:36.711 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns= 2022-09-03T06:45:36.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns=', 'Content-Length': '0'} 2022-09-03T06:45:36.716 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d30c6ad85c47150d-006312f810-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.758 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:36.759 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:45:36.760 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:36.761 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:36.761 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.762 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.763 DEBUG:boto:Method: GET 2022-09-03T06:45:36.763 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.764 DEBUG:boto:Data: 2022-09-03T06:45:36.764 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.765 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:36.766 DEBUG:boto:Port: 8001 2022-09-03T06:45:36.766 DEBUG:boto:Params: {} 2022-09-03T06:45:36.768 DEBUG:boto:Token: None 2022-09-03T06:45:36.769 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.770 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1sIQBHKgefWdD1L9eozoY/RJKEE= 2022-09-03T06:45:36.771 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1sIQBHKgefWdD1L9eozoY/RJKEE=', 'Content-Length': '0'} 2022-09-03T06:45:36.776 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a2c725aaf5da1896-006312f810-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.781 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:36.782 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:36.783 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.784 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.785 DEBUG:boto:Method: GET 2022-09-03T06:45:36.786 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.787 DEBUG:boto:Data: 2022-09-03T06:45:36.788 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.789 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.791 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.792 DEBUG:boto:Params: {} 2022-09-03T06:45:36.793 DEBUG:boto:Token: None 2022-09-03T06:45:36.795 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.796 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1sIQBHKgefWdD1L9eozoY/RJKEE= 2022-09-03T06:45:36.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1sIQBHKgefWdD1L9eozoY/RJKEE=', 'Content-Length': '0'} 2022-09-03T06:45:36.800 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af2776e7d343d7ea-006312f810-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.801 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:36.802 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:36.803 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.804 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.805 DEBUG:boto:Method: HEAD 2022-09-03T06:45:36.806 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.807 DEBUG:boto:Data: 2022-09-03T06:45:36.808 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.809 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:36.810 DEBUG:boto:Port: 8001 2022-09-03T06:45:36.811 DEBUG:boto:Params: {} 2022-09-03T06:45:36.812 DEBUG:boto:Token: None 2022-09-03T06:45:36.813 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.814 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zxJ8b3jBsaFAg7RF7rM7E7Vmcp4= 2022-09-03T06:45:36.814 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zxJ8b3jBsaFAg7RF7rM7E7Vmcp4=', 'Content-Length': '0'} 2022-09-03T06:45:36.817 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a4b585c27fede7dc-006312f810-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.818 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:36.819 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:36.820 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.820 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.821 DEBUG:boto:Method: HEAD 2022-09-03T06:45:36.822 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.822 DEBUG:boto:Data: 2022-09-03T06:45:36.823 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.823 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.824 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.824 DEBUG:boto:Params: {} 2022-09-03T06:45:36.825 DEBUG:boto:Token: None 2022-09-03T06:45:36.826 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.827 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zxJ8b3jBsaFAg7RF7rM7E7Vmcp4= 2022-09-03T06:45:36.827 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zxJ8b3jBsaFAg7RF7rM7E7Vmcp4=', 'Content-Length': '0'} 2022-09-03T06:45:36.830 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000037c0b154b8727e0e-006312f810-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.831 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:45:36.832 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:36.833 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:36.834 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.836 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.837 DEBUG:boto:Method: GET 2022-09-03T06:45:36.838 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.839 DEBUG:boto:Data: 2022-09-03T06:45:36.840 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.840 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:36.841 DEBUG:boto:Port: 8001 2022-09-03T06:45:36.842 DEBUG:boto:Params: {} 2022-09-03T06:45:36.843 DEBUG:boto:Token: None 2022-09-03T06:45:36.843 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.844 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1sIQBHKgefWdD1L9eozoY/RJKEE= 2022-09-03T06:45:36.845 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1sIQBHKgefWdD1L9eozoY/RJKEE=', 'Content-Length': '0'} 2022-09-03T06:45:36.847 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000050c4f429a1e129f0-006312f810-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.848 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:36.849 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:36.850 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.850 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.851 DEBUG:boto:Method: GET 2022-09-03T06:45:36.852 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.852 DEBUG:boto:Data: 2022-09-03T06:45:36.853 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.854 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.855 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.855 DEBUG:boto:Params: {} 2022-09-03T06:45:36.856 DEBUG:boto:Token: None 2022-09-03T06:45:36.856 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:36.857 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1sIQBHKgefWdD1L9eozoY/RJKEE= 2022-09-03T06:45:36.858 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1sIQBHKgefWdD1L9eozoY/RJKEE=', 'Content-Length': '0'} 2022-09-03T06:45:36.861 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000abe6492b0804e4f1-006312f810-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.863 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:36.864 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:36.865 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:36.866 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:36.867 DEBUG:boto:Method: GET 2022-09-03T06:45:36.868 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:45:36.869 DEBUG:boto:Data: 2022-09-03T06:45:36.870 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.871 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:36.872 DEBUG:boto:Port: 8001 2022-09-03T06:45:36.873 DEBUG:boto:Params: {} 2022-09-03T06:45:36.874 DEBUG:boto:Token: None 2022-09-03T06:45:36.876 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/ 2022-09-03T06:45:36.877 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns= 2022-09-03T06:45:36.878 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns=', 'Content-Length': '0'} 2022-09-03T06:45:36.882 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d35be6bd9fa71490-006312f810-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.883 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:36.884 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:36.884 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:36.885 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:36.885 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:36.886 DEBUG:boto:Method: GET 2022-09-03T06:45:36.886 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:45:36.887 DEBUG:boto:Data: 2022-09-03T06:45:36.887 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.888 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.888 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.889 DEBUG:boto:Params: {} 2022-09-03T06:45:36.890 DEBUG:boto:Token: None 2022-09-03T06:45:36.891 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/ 2022-09-03T06:45:36.891 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns= 2022-09-03T06:45:36.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CIo/0JIaiyE+xiDRBtSqgqUeXns=', 'Content-Length': '0'} 2022-09-03T06:45:36.896 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002ae1f6cb869c289b-006312f810-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.897 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:36.898 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:36.899 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:36.900 DEBUG:boto:Method: HEAD 2022-09-03T06:45:36.900 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:45:36.901 DEBUG:boto:Data: 2022-09-03T06:45:36.901 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.902 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:36.903 DEBUG:boto:Port: 8001 2022-09-03T06:45:36.904 DEBUG:boto:Params: {} 2022-09-03T06:45:36.905 DEBUG:boto:Token: None 2022-09-03T06:45:36.907 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj58 2022-09-03T06:45:36.908 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ff7Cqu3Dws9912uHgaF2gMsE06Q= 2022-09-03T06:45:36.909 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ff7Cqu3Dws9912uHgaF2gMsE06Q=', 'Content-Length': '0'} 2022-09-03T06:45:36.912 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000663cd5f72f14ef82-006312f810-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.913 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:36.914 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:36.915 DEBUG:boto:Method: HEAD 2022-09-03T06:45:36.916 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:45:36.917 DEBUG:boto:Data: 2022-09-03T06:45:36.918 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.919 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.919 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.920 DEBUG:boto:Params: {} 2022-09-03T06:45:36.922 DEBUG:boto:Token: None 2022-09-03T06:45:36.922 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj58 2022-09-03T06:45:36.924 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ff7Cqu3Dws9912uHgaF2gMsE06Q= 2022-09-03T06:45:36.924 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ff7Cqu3Dws9912uHgaF2gMsE06Q=', 'Content-Length': '0'} 2022-09-03T06:45:36.927 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000563b7a805ce289b1-006312f810-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.927 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:45:36.928 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:36.929 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:36.930 DEBUG:boto:Method: GET 2022-09-03T06:45:36.931 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:45:36.932 DEBUG:boto:Data: 2022-09-03T06:45:36.932 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.933 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:36.935 DEBUG:boto:Port: 8001 2022-09-03T06:45:36.936 DEBUG:boto:Params: {} 2022-09-03T06:45:36.937 DEBUG:boto:Token: None 2022-09-03T06:45:36.939 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj58 2022-09-03T06:45:36.941 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wS+SWFSQR3qSfS4QrDTu06OwKfc= 2022-09-03T06:45:36.942 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wS+SWFSQR3qSfS4QrDTu06OwKfc=', 'Content-Length': '0'} 2022-09-03T06:45:36.945 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000054ff0b32947571e0-006312f810-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:36.986 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:36.987 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:36.988 DEBUG:boto:Method: GET 2022-09-03T06:45:36.988 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:45:36.989 DEBUG:boto:Data: 2022-09-03T06:45:36.990 DEBUG:boto:Headers: {} 2022-09-03T06:45:36.990 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:36.991 DEBUG:boto:Port: 8000 2022-09-03T06:45:36.991 DEBUG:boto:Params: {} 2022-09-03T06:45:36.992 DEBUG:boto:Token: None 2022-09-03T06:45:36.993 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:36 GMT /dsgvcy-75/obj58 2022-09-03T06:45:36.993 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wS+SWFSQR3qSfS4QrDTu06OwKfc= 2022-09-03T06:45:36.994 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wS+SWFSQR3qSfS4QrDTu06OwKfc=', 'Content-Length': '0'} 2022-09-03T06:45:36.997 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000109808ef98857666-006312f810-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.000 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:37.001 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:37.002 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:37.003 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:37.004 DEBUG:boto:Method: GET 2022-09-03T06:45:37.005 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:45:37.006 DEBUG:boto:Data: 2022-09-03T06:45:37.007 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.008 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:37.009 DEBUG:boto:Port: 8001 2022-09-03T06:45:37.010 DEBUG:boto:Params: {} 2022-09-03T06:45:37.012 DEBUG:boto:Token: None 2022-09-03T06:45:37.013 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/ 2022-09-03T06:45:37.014 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU= 2022-09-03T06:45:37.015 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU=', 'Content-Length': '0'} 2022-09-03T06:45:37.019 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000964697a79004f29a-006312f811-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.020 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:37.020 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:37.021 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:37.022 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:37.022 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:37.023 DEBUG:boto:Method: GET 2022-09-03T06:45:37.024 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:45:37.024 DEBUG:boto:Data: 2022-09-03T06:45:37.025 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.026 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:37.026 DEBUG:boto:Port: 8000 2022-09-03T06:45:37.027 DEBUG:boto:Params: {} 2022-09-03T06:45:37.028 DEBUG:boto:Token: None 2022-09-03T06:45:37.029 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/ 2022-09-03T06:45:37.029 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU= 2022-09-03T06:45:37.030 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU=', 'Content-Length': '0'} 2022-09-03T06:45:37.044 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007974383a145fb396-006312f811-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.046 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:37.047 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:45:37.048 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:37.049 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:37.050 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.051 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.051 DEBUG:boto:Method: GET 2022-09-03T06:45:37.052 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.052 DEBUG:boto:Data: 2022-09-03T06:45:37.053 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.054 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:37.054 DEBUG:boto:Port: 8001 2022-09-03T06:45:37.055 DEBUG:boto:Params: {} 2022-09-03T06:45:37.056 DEBUG:boto:Token: None 2022-09-03T06:45:37.056 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.057 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:u3apF+1bSFTUc5q8FQsen35Yptg= 2022-09-03T06:45:37.058 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:u3apF+1bSFTUc5q8FQsen35Yptg=', 'Content-Length': '0'} 2022-09-03T06:45:37.060 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ada98060efb37e5c-006312f811-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.065 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:37.065 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:37.066 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.066 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.067 DEBUG:boto:Method: GET 2022-09-03T06:45:37.068 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.068 DEBUG:boto:Data: 2022-09-03T06:45:37.069 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.069 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:37.070 DEBUG:boto:Port: 8000 2022-09-03T06:45:37.070 DEBUG:boto:Params: {} 2022-09-03T06:45:37.071 DEBUG:boto:Token: None 2022-09-03T06:45:37.072 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.072 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:u3apF+1bSFTUc5q8FQsen35Yptg= 2022-09-03T06:45:37.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:u3apF+1bSFTUc5q8FQsen35Yptg=', 'Content-Length': '0'} 2022-09-03T06:45:37.075 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005083a30b227d1b5c-006312f811-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.118 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:37.120 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:37.120 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.121 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.122 DEBUG:boto:Method: HEAD 2022-09-03T06:45:37.123 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.124 DEBUG:boto:Data: 2022-09-03T06:45:37.125 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.125 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:37.126 DEBUG:boto:Port: 8001 2022-09-03T06:45:37.127 DEBUG:boto:Params: {} 2022-09-03T06:45:37.128 DEBUG:boto:Token: None 2022-09-03T06:45:37.130 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.130 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ou/oROiE4xl+bKtlebz7+9TEVBs= 2022-09-03T06:45:37.131 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ou/oROiE4xl+bKtlebz7+9TEVBs=', 'Content-Length': '0'} 2022-09-03T06:45:37.133 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000507347b2bf04abdc-006312f811-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.134 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:37.135 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:37.135 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.136 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.137 DEBUG:boto:Method: HEAD 2022-09-03T06:45:37.137 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.138 DEBUG:boto:Data: 2022-09-03T06:45:37.138 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.139 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:37.139 DEBUG:boto:Port: 8000 2022-09-03T06:45:37.140 DEBUG:boto:Params: {} 2022-09-03T06:45:37.141 DEBUG:boto:Token: None 2022-09-03T06:45:37.141 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.142 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ou/oROiE4xl+bKtlebz7+9TEVBs= 2022-09-03T06:45:37.143 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ou/oROiE4xl+bKtlebz7+9TEVBs=', 'Content-Length': '0'} 2022-09-03T06:45:37.145 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008c7482b33891bc5a-006312f811-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.146 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:45:37.147 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:37.147 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:37.148 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.148 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.149 DEBUG:boto:Method: GET 2022-09-03T06:45:37.150 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.150 DEBUG:boto:Data: 2022-09-03T06:45:37.151 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.152 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:37.153 DEBUG:boto:Port: 8001 2022-09-03T06:45:37.154 DEBUG:boto:Params: {} 2022-09-03T06:45:37.155 DEBUG:boto:Token: None 2022-09-03T06:45:37.156 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.157 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:u3apF+1bSFTUc5q8FQsen35Yptg= 2022-09-03T06:45:37.158 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:u3apF+1bSFTUc5q8FQsen35Yptg=', 'Content-Length': '0'} 2022-09-03T06:45:37.160 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008f0380e0a1f880a1-006312f811-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.161 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:37.162 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:37.163 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.163 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.164 DEBUG:boto:Method: GET 2022-09-03T06:45:37.164 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.165 DEBUG:boto:Data: 2022-09-03T06:45:37.165 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.166 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:37.166 DEBUG:boto:Port: 8000 2022-09-03T06:45:37.167 DEBUG:boto:Params: {} 2022-09-03T06:45:37.168 DEBUG:boto:Token: None 2022-09-03T06:45:37.168 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:37.169 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:u3apF+1bSFTUc5q8FQsen35Yptg= 2022-09-03T06:45:37.170 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:u3apF+1bSFTUc5q8FQsen35Yptg=', 'Content-Length': '0'} 2022-09-03T06:45:37.172 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002c0da8d139566fe5-006312f811-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.173 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:37.174 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:37.174 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:37.175 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:37.175 DEBUG:boto:Method: GET 2022-09-03T06:45:37.176 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:45:37.176 DEBUG:boto:Data: 2022-09-03T06:45:37.177 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.178 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:37.178 DEBUG:boto:Port: 8001 2022-09-03T06:45:37.179 DEBUG:boto:Params: {} 2022-09-03T06:45:37.179 DEBUG:boto:Token: None 2022-09-03T06:45:37.180 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/ 2022-09-03T06:45:37.181 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU= 2022-09-03T06:45:37.181 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU=', 'Content-Length': '0'} 2022-09-03T06:45:37.184 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004327502a87643743-006312f811-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.185 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:37.186 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:37.187 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:37.188 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:37.188 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:37.189 DEBUG:boto:Method: GET 2022-09-03T06:45:37.189 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:45:37.190 DEBUG:boto:Data: 2022-09-03T06:45:37.190 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.191 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:37.192 DEBUG:boto:Port: 8000 2022-09-03T06:45:37.192 DEBUG:boto:Params: {} 2022-09-03T06:45:37.193 DEBUG:boto:Token: None 2022-09-03T06:45:37.194 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/ 2022-09-03T06:45:37.194 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU= 2022-09-03T06:45:37.195 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU=', 'Content-Length': '0'} 2022-09-03T06:45:37.199 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008f3d5347c95a4948-006312f811-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.200 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:37.201 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:37.201 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:37.202 DEBUG:boto:Method: HEAD 2022-09-03T06:45:37.202 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:45:37.203 DEBUG:boto:Data: 2022-09-03T06:45:37.204 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.204 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:37.205 DEBUG:boto:Port: 8001 2022-09-03T06:45:37.206 DEBUG:boto:Params: {} 2022-09-03T06:45:37.207 DEBUG:boto:Token: None 2022-09-03T06:45:37.208 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj68 2022-09-03T06:45:37.209 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sZzVTAhT/nTgZrUW5DBN3FT9DQM= 2022-09-03T06:45:37.210 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sZzVTAhT/nTgZrUW5DBN3FT9DQM=', 'Content-Length': '0'} 2022-09-03T06:45:37.213 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f682163ed3007a33-006312f811-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.214 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:37.214 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:37.215 DEBUG:boto:Method: HEAD 2022-09-03T06:45:37.216 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:45:37.216 DEBUG:boto:Data: 2022-09-03T06:45:37.217 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.217 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:37.218 DEBUG:boto:Port: 8000 2022-09-03T06:45:37.218 DEBUG:boto:Params: {} 2022-09-03T06:45:37.219 DEBUG:boto:Token: None 2022-09-03T06:45:37.220 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj68 2022-09-03T06:45:37.220 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sZzVTAhT/nTgZrUW5DBN3FT9DQM= 2022-09-03T06:45:37.221 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sZzVTAhT/nTgZrUW5DBN3FT9DQM=', 'Content-Length': '0'} 2022-09-03T06:45:37.223 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009edd997d519c0546-006312f811-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.224 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:45:37.224 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:37.225 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:37.226 DEBUG:boto:Method: GET 2022-09-03T06:45:37.227 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:45:37.228 DEBUG:boto:Data: 2022-09-03T06:45:37.228 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.229 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:37.229 DEBUG:boto:Port: 8001 2022-09-03T06:45:37.230 DEBUG:boto:Params: {} 2022-09-03T06:45:37.232 DEBUG:boto:Token: None 2022-09-03T06:45:37.233 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj68 2022-09-03T06:45:37.234 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mdK/RGhK9H+JPte+d7kES88NmjM= 2022-09-03T06:45:37.235 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mdK/RGhK9H+JPte+d7kES88NmjM=', 'Content-Length': '0'} 2022-09-03T06:45:37.238 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000031b0a2cc405e6ae1-006312f811-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.239 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:37.239 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:37.240 DEBUG:boto:Method: GET 2022-09-03T06:45:37.240 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:45:37.241 DEBUG:boto:Data: 2022-09-03T06:45:37.242 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.242 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:37.243 DEBUG:boto:Port: 8000 2022-09-03T06:45:37.243 DEBUG:boto:Params: {} 2022-09-03T06:45:37.244 DEBUG:boto:Token: None 2022-09-03T06:45:37.245 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj68 2022-09-03T06:45:37.245 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mdK/RGhK9H+JPte+d7kES88NmjM= 2022-09-03T06:45:37.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mdK/RGhK9H+JPte+d7kES88NmjM=', 'Content-Length': '0'} 2022-09-03T06:45:37.248 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002cb489b24ea6d640-006312f811-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.250 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:37.251 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:37.252 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:37.253 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:37.253 DEBUG:boto:Method: GET 2022-09-03T06:45:37.254 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:45:37.255 DEBUG:boto:Data: 2022-09-03T06:45:37.256 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.257 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:37.258 DEBUG:boto:Port: 8001 2022-09-03T06:45:37.258 DEBUG:boto:Params: {} 2022-09-03T06:45:37.259 DEBUG:boto:Token: None 2022-09-03T06:45:37.260 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/ 2022-09-03T06:45:37.261 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU= 2022-09-03T06:45:37.262 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU=', 'Content-Length': '0'} 2022-09-03T06:45:37.266 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ec01baac9b09ff94-006312f811-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.266 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:37.267 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:37.268 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:37.268 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:37.269 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:37.269 DEBUG:boto:Method: GET 2022-09-03T06:45:37.270 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:45:37.270 DEBUG:boto:Data: 2022-09-03T06:45:37.271 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.271 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:37.272 DEBUG:boto:Port: 8000 2022-09-03T06:45:37.272 DEBUG:boto:Params: {} 2022-09-03T06:45:37.273 DEBUG:boto:Token: None 2022-09-03T06:45:37.274 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/ 2022-09-03T06:45:37.274 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU= 2022-09-03T06:45:37.275 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU=', 'Content-Length': '0'} 2022-09-03T06:45:37.279 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000078ae0596ef2ba6ad-006312f811-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.280 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:37.282 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:45:37.282 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:37.283 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:37.284 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.284 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.285 DEBUG:boto:Method: GET 2022-09-03T06:45:37.285 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.286 DEBUG:boto:Data: 2022-09-03T06:45:37.287 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.288 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:37.288 DEBUG:boto:Port: 8001 2022-09-03T06:45:37.289 DEBUG:boto:Params: {} 2022-09-03T06:45:37.290 DEBUG:boto:Token: None 2022-09-03T06:45:37.290 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.291 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:g+01W0a1Ut/lBkADvFdo4ko1GW8= 2022-09-03T06:45:37.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:g+01W0a1Ut/lBkADvFdo4ko1GW8=', 'Content-Length': '0'} 2022-09-03T06:45:37.294 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000739d2e445a1e2c22-006312f811-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.295 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:37.296 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:37.296 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.297 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.299 DEBUG:boto:Method: GET 2022-09-03T06:45:37.300 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.301 DEBUG:boto:Data: 2022-09-03T06:45:37.301 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.302 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:37.303 DEBUG:boto:Port: 8000 2022-09-03T06:45:37.304 DEBUG:boto:Params: {} 2022-09-03T06:45:37.305 DEBUG:boto:Token: None 2022-09-03T06:45:37.306 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.307 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:g+01W0a1Ut/lBkADvFdo4ko1GW8= 2022-09-03T06:45:37.307 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:g+01W0a1Ut/lBkADvFdo4ko1GW8=', 'Content-Length': '0'} 2022-09-03T06:45:37.311 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000019245493dbc88a38-006312f811-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.354 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:37.355 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:37.355 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.356 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.356 DEBUG:boto:Method: HEAD 2022-09-03T06:45:37.357 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.357 DEBUG:boto:Data: 2022-09-03T06:45:37.358 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.359 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:37.359 DEBUG:boto:Port: 8001 2022-09-03T06:45:37.360 DEBUG:boto:Params: {} 2022-09-03T06:45:37.360 DEBUG:boto:Token: None 2022-09-03T06:45:37.361 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.361 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2gZhEUvoY9R4oXwFgNTRD7+X32w= 2022-09-03T06:45:37.363 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2gZhEUvoY9R4oXwFgNTRD7+X32w=', 'Content-Length': '0'} 2022-09-03T06:45:37.365 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d73aadb53ddd8439-006312f811-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.366 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:37.368 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:37.368 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.369 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.369 DEBUG:boto:Method: HEAD 2022-09-03T06:45:37.370 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.371 DEBUG:boto:Data: 2022-09-03T06:45:37.371 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.372 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:37.372 DEBUG:boto:Port: 8000 2022-09-03T06:45:37.373 DEBUG:boto:Params: {} 2022-09-03T06:45:37.373 DEBUG:boto:Token: None 2022-09-03T06:45:37.374 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.375 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2gZhEUvoY9R4oXwFgNTRD7+X32w= 2022-09-03T06:45:37.375 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2gZhEUvoY9R4oXwFgNTRD7+X32w=', 'Content-Length': '0'} 2022-09-03T06:45:37.377 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b7db1ba84488fba-006312f811-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.378 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:45:37.379 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:37.380 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:37.380 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.381 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.381 DEBUG:boto:Method: GET 2022-09-03T06:45:37.382 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.383 DEBUG:boto:Data: 2022-09-03T06:45:37.383 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.384 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:37.384 DEBUG:boto:Port: 8001 2022-09-03T06:45:37.385 DEBUG:boto:Params: {} 2022-09-03T06:45:37.386 DEBUG:boto:Token: None 2022-09-03T06:45:37.386 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.387 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:g+01W0a1Ut/lBkADvFdo4ko1GW8= 2022-09-03T06:45:37.388 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:g+01W0a1Ut/lBkADvFdo4ko1GW8=', 'Content-Length': '0'} 2022-09-03T06:45:37.391 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001acb19ff2d039f7d-006312f811-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.392 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:37.392 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:37.393 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.393 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.394 DEBUG:boto:Method: GET 2022-09-03T06:45:37.395 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.395 DEBUG:boto:Data: 2022-09-03T06:45:37.396 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.396 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:37.397 DEBUG:boto:Port: 8000 2022-09-03T06:45:37.397 DEBUG:boto:Params: {} 2022-09-03T06:45:37.398 DEBUG:boto:Token: None 2022-09-03T06:45:37.399 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:37.399 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:g+01W0a1Ut/lBkADvFdo4ko1GW8= 2022-09-03T06:45:37.401 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:g+01W0a1Ut/lBkADvFdo4ko1GW8=', 'Content-Length': '0'} 2022-09-03T06:45:37.404 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b4993ced1e7ba9f1-006312f811-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.405 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:37.405 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:37.406 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:37.407 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:37.407 DEBUG:boto:Method: GET 2022-09-03T06:45:37.470 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:45:37.471 DEBUG:boto:Data: 2022-09-03T06:45:37.472 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.472 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:37.473 DEBUG:boto:Port: 8001 2022-09-03T06:45:37.473 DEBUG:boto:Params: {} 2022-09-03T06:45:37.478 DEBUG:boto:Token: None 2022-09-03T06:45:37.481 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/ 2022-09-03T06:45:37.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU= 2022-09-03T06:45:37.487 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU=', 'Content-Length': '0'} 2022-09-03T06:45:37.491 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000058691bffd7b77948-006312f811-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.493 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:37.494 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:37.495 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:37.496 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:37.497 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:37.497 DEBUG:boto:Method: GET 2022-09-03T06:45:37.499 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:45:37.500 DEBUG:boto:Data: 2022-09-03T06:45:37.500 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.501 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:37.502 DEBUG:boto:Port: 8000 2022-09-03T06:45:37.502 DEBUG:boto:Params: {} 2022-09-03T06:45:37.503 DEBUG:boto:Token: None 2022-09-03T06:45:37.504 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/ 2022-09-03T06:45:37.504 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU= 2022-09-03T06:45:37.505 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU=', 'Content-Length': '0'} 2022-09-03T06:45:37.514 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000edb33da96e615573-006312f811-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.515 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:37.516 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:37.517 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:37.517 DEBUG:boto:Method: HEAD 2022-09-03T06:45:37.518 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:45:37.519 DEBUG:boto:Data: 2022-09-03T06:45:37.519 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.520 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:37.520 DEBUG:boto:Port: 8001 2022-09-03T06:45:37.521 DEBUG:boto:Params: {} 2022-09-03T06:45:37.521 DEBUG:boto:Token: None 2022-09-03T06:45:37.522 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj69 2022-09-03T06:45:37.522 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:15sf+TQKU9wiAeWvwGPIGGeyLCE= 2022-09-03T06:45:37.523 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:15sf+TQKU9wiAeWvwGPIGGeyLCE=', 'Content-Length': '0'} 2022-09-03T06:45:37.542 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f587ae0b674bffcf-006312f811-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.544 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:37.545 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:37.546 DEBUG:boto:Method: HEAD 2022-09-03T06:45:37.546 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:45:37.547 DEBUG:boto:Data: 2022-09-03T06:45:37.548 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.549 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:37.550 DEBUG:boto:Port: 8000 2022-09-03T06:45:37.551 DEBUG:boto:Params: {} 2022-09-03T06:45:37.552 DEBUG:boto:Token: None 2022-09-03T06:45:37.553 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj69 2022-09-03T06:45:37.555 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:15sf+TQKU9wiAeWvwGPIGGeyLCE= 2022-09-03T06:45:37.556 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:15sf+TQKU9wiAeWvwGPIGGeyLCE=', 'Content-Length': '0'} 2022-09-03T06:45:37.558 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003351f3704543a7f5-006312f811-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.559 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:45:37.560 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:37.561 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:37.562 DEBUG:boto:Method: GET 2022-09-03T06:45:37.563 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:45:37.564 DEBUG:boto:Data: 2022-09-03T06:45:37.565 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.566 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:37.567 DEBUG:boto:Port: 8001 2022-09-03T06:45:37.567 DEBUG:boto:Params: {} 2022-09-03T06:45:37.568 DEBUG:boto:Token: None 2022-09-03T06:45:37.568 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj69 2022-09-03T06:45:37.569 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yX17xqF8C6Q5mAx2jKVE5HK8pcg= 2022-09-03T06:45:37.570 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yX17xqF8C6Q5mAx2jKVE5HK8pcg=', 'Content-Length': '0'} 2022-09-03T06:45:37.574 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b4febadd2b43e1b9-006312f811-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.618 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:37.619 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:37.619 DEBUG:boto:Method: GET 2022-09-03T06:45:37.620 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:45:37.621 DEBUG:boto:Data: 2022-09-03T06:45:37.621 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.622 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:37.622 DEBUG:boto:Port: 8000 2022-09-03T06:45:37.623 DEBUG:boto:Params: {} 2022-09-03T06:45:37.624 DEBUG:boto:Token: None 2022-09-03T06:45:37.626 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj69 2022-09-03T06:45:37.626 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yX17xqF8C6Q5mAx2jKVE5HK8pcg= 2022-09-03T06:45:37.627 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yX17xqF8C6Q5mAx2jKVE5HK8pcg=', 'Content-Length': '0'} 2022-09-03T06:45:37.630 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000071373de3fea54c47-006312f811-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.631 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:37.632 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:37.632 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:37.633 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:37.634 DEBUG:boto:Method: GET 2022-09-03T06:45:37.634 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:45:37.635 DEBUG:boto:Data: 2022-09-03T06:45:37.635 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.636 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:37.637 DEBUG:boto:Port: 8001 2022-09-03T06:45:37.637 DEBUG:boto:Params: {} 2022-09-03T06:45:37.638 DEBUG:boto:Token: None 2022-09-03T06:45:37.639 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/ 2022-09-03T06:45:37.639 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU= 2022-09-03T06:45:37.640 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU=', 'Content-Length': '0'} 2022-09-03T06:45:37.643 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a3cabba996a4b0b1-006312f811-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.644 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:37.646 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:37.647 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:37.648 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:37.648 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:37.649 DEBUG:boto:Method: GET 2022-09-03T06:45:37.649 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:45:37.650 DEBUG:boto:Data: 2022-09-03T06:45:37.650 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.651 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:37.652 DEBUG:boto:Port: 8000 2022-09-03T06:45:37.652 DEBUG:boto:Params: {} 2022-09-03T06:45:37.653 DEBUG:boto:Token: None 2022-09-03T06:45:37.653 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/ 2022-09-03T06:45:37.654 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU= 2022-09-03T06:45:37.655 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU=', 'Content-Length': '0'} 2022-09-03T06:45:37.670 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000003fe61e0539e307e-006312f811-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.678 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:37.680 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:45:37.682 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:37.683 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:37.683 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.685 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.686 DEBUG:boto:Method: GET 2022-09-03T06:45:37.687 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.688 DEBUG:boto:Data: 2022-09-03T06:45:37.688 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.689 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:37.689 DEBUG:boto:Port: 8001 2022-09-03T06:45:37.691 DEBUG:boto:Params: {} 2022-09-03T06:45:37.692 DEBUG:boto:Token: None 2022-09-03T06:45:37.693 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.695 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hm5F5x+DUB2dFWgLh1f5VGw6Cdc= 2022-09-03T06:45:37.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hm5F5x+DUB2dFWgLh1f5VGw6Cdc=', 'Content-Length': '0'} 2022-09-03T06:45:37.715 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ce8e7a409dc1bc81-006312f811-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.718 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:37.719 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:37.719 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.720 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.720 DEBUG:boto:Method: GET 2022-09-03T06:45:37.721 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.721 DEBUG:boto:Data: 2022-09-03T06:45:37.722 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.723 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:37.723 DEBUG:boto:Port: 8000 2022-09-03T06:45:37.724 DEBUG:boto:Params: {} 2022-09-03T06:45:37.724 DEBUG:boto:Token: None 2022-09-03T06:45:37.725 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.726 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hm5F5x+DUB2dFWgLh1f5VGw6Cdc= 2022-09-03T06:45:37.726 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hm5F5x+DUB2dFWgLh1f5VGw6Cdc=', 'Content-Length': '0'} 2022-09-03T06:45:37.729 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000efbcfed2830b2864-006312f811-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.729 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:37.730 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:37.731 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.731 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.732 DEBUG:boto:Method: HEAD 2022-09-03T06:45:37.732 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.733 DEBUG:boto:Data: 2022-09-03T06:45:37.734 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.734 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:37.735 DEBUG:boto:Port: 8001 2022-09-03T06:45:37.735 DEBUG:boto:Params: {} 2022-09-03T06:45:37.736 DEBUG:boto:Token: None 2022-09-03T06:45:37.737 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.737 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PqDN9dy8JhU+f3mblzq0AKNF0x4= 2022-09-03T06:45:37.738 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PqDN9dy8JhU+f3mblzq0AKNF0x4=', 'Content-Length': '0'} 2022-09-03T06:45:37.740 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000089bc03cb864c9425-006312f811-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.741 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:37.742 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:37.743 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.743 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.744 DEBUG:boto:Method: HEAD 2022-09-03T06:45:37.745 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.745 DEBUG:boto:Data: 2022-09-03T06:45:37.746 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.747 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:37.747 DEBUG:boto:Port: 8000 2022-09-03T06:45:37.748 DEBUG:boto:Params: {} 2022-09-03T06:45:37.748 DEBUG:boto:Token: None 2022-09-03T06:45:37.749 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.750 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PqDN9dy8JhU+f3mblzq0AKNF0x4= 2022-09-03T06:45:37.750 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PqDN9dy8JhU+f3mblzq0AKNF0x4=', 'Content-Length': '0'} 2022-09-03T06:45:37.753 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b777eb6f6626e61c-006312f811-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.753 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:45:37.754 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:37.755 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:37.756 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.757 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.757 DEBUG:boto:Method: GET 2022-09-03T06:45:37.758 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.758 DEBUG:boto:Data: 2022-09-03T06:45:37.759 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.760 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:37.761 DEBUG:boto:Port: 8001 2022-09-03T06:45:37.761 DEBUG:boto:Params: {} 2022-09-03T06:45:37.762 DEBUG:boto:Token: None 2022-09-03T06:45:37.763 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.763 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hm5F5x+DUB2dFWgLh1f5VGw6Cdc= 2022-09-03T06:45:37.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hm5F5x+DUB2dFWgLh1f5VGw6Cdc=', 'Content-Length': '0'} 2022-09-03T06:45:37.767 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009ea795b71e109bd3-006312f811-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.768 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:37.768 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:37.769 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.770 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.770 DEBUG:boto:Method: GET 2022-09-03T06:45:37.771 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.772 DEBUG:boto:Data: 2022-09-03T06:45:37.772 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.773 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:37.774 DEBUG:boto:Port: 8000 2022-09-03T06:45:37.776 DEBUG:boto:Params: {} 2022-09-03T06:45:37.777 DEBUG:boto:Token: None 2022-09-03T06:45:37.778 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:37.778 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hm5F5x+DUB2dFWgLh1f5VGw6Cdc= 2022-09-03T06:45:37.779 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hm5F5x+DUB2dFWgLh1f5VGw6Cdc=', 'Content-Length': '0'} 2022-09-03T06:45:37.782 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006d3206142993b607-006312f811-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.783 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:37.784 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:37.785 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:37.785 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:37.786 DEBUG:boto:Method: GET 2022-09-03T06:45:37.787 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:45:37.787 DEBUG:boto:Data: 2022-09-03T06:45:37.788 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.788 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:37.789 DEBUG:boto:Port: 8001 2022-09-03T06:45:37.790 DEBUG:boto:Params: {} 2022-09-03T06:45:37.790 DEBUG:boto:Token: None 2022-09-03T06:45:37.791 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/ 2022-09-03T06:45:37.792 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU= 2022-09-03T06:45:37.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU=', 'Content-Length': '0'} 2022-09-03T06:45:37.796 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a9c6e6a25b868dcf-006312f811-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.842 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:37.844 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:37.845 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:37.845 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:37.846 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:37.846 DEBUG:boto:Method: GET 2022-09-03T06:45:37.847 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:45:37.848 DEBUG:boto:Data: 2022-09-03T06:45:37.848 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.849 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:37.850 DEBUG:boto:Port: 8000 2022-09-03T06:45:37.850 DEBUG:boto:Params: {} 2022-09-03T06:45:37.851 DEBUG:boto:Token: None 2022-09-03T06:45:37.852 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/ 2022-09-03T06:45:37.852 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU= 2022-09-03T06:45:37.853 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU=', 'Content-Length': '0'} 2022-09-03T06:45:37.857 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000081b2da8b038a9dc2-006312f811-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.858 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:37.859 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:37.860 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:37.860 DEBUG:boto:Method: HEAD 2022-09-03T06:45:37.861 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:45:37.861 DEBUG:boto:Data: 2022-09-03T06:45:37.862 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.863 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:37.863 DEBUG:boto:Port: 8001 2022-09-03T06:45:37.864 DEBUG:boto:Params: {} 2022-09-03T06:45:37.865 DEBUG:boto:Token: None 2022-09-03T06:45:37.865 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj76 2022-09-03T06:45:37.866 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ctAIS7cdoIgZMU8HA8FXCYjT4yY= 2022-09-03T06:45:37.866 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ctAIS7cdoIgZMU8HA8FXCYjT4yY=', 'Content-Length': '0'} 2022-09-03T06:45:37.869 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006d412ac36dba1f52-006312f811-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.869 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:37.870 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:37.871 DEBUG:boto:Method: HEAD 2022-09-03T06:45:37.872 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:45:37.873 DEBUG:boto:Data: 2022-09-03T06:45:37.874 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.875 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:37.876 DEBUG:boto:Port: 8000 2022-09-03T06:45:37.877 DEBUG:boto:Params: {} 2022-09-03T06:45:37.878 DEBUG:boto:Token: None 2022-09-03T06:45:37.879 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj76 2022-09-03T06:45:37.881 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ctAIS7cdoIgZMU8HA8FXCYjT4yY= 2022-09-03T06:45:37.882 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ctAIS7cdoIgZMU8HA8FXCYjT4yY=', 'Content-Length': '0'} 2022-09-03T06:45:37.885 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af5e01a94c08f891-006312f811-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.886 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:45:37.887 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:37.887 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:37.888 DEBUG:boto:Method: GET 2022-09-03T06:45:37.889 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:45:37.889 DEBUG:boto:Data: 2022-09-03T06:45:37.890 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.891 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:37.892 DEBUG:boto:Port: 8001 2022-09-03T06:45:37.893 DEBUG:boto:Params: {} 2022-09-03T06:45:37.894 DEBUG:boto:Token: None 2022-09-03T06:45:37.896 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj76 2022-09-03T06:45:37.897 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OHPLQwO+nMDY31n2qnlMCIdpKcU= 2022-09-03T06:45:37.898 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OHPLQwO+nMDY31n2qnlMCIdpKcU=', 'Content-Length': '0'} 2022-09-03T06:45:37.901 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000119c1ebc7adbabb3-006312f811-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.902 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:37.904 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:37.904 DEBUG:boto:Method: GET 2022-09-03T06:45:37.905 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:45:37.906 DEBUG:boto:Data: 2022-09-03T06:45:37.907 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.908 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:37.909 DEBUG:boto:Port: 8000 2022-09-03T06:45:37.909 DEBUG:boto:Params: {} 2022-09-03T06:45:37.911 DEBUG:boto:Token: None 2022-09-03T06:45:37.911 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj76 2022-09-03T06:45:37.912 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OHPLQwO+nMDY31n2qnlMCIdpKcU= 2022-09-03T06:45:37.913 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OHPLQwO+nMDY31n2qnlMCIdpKcU=', 'Content-Length': '0'} 2022-09-03T06:45:37.920 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000072f1cb5274f1bb7e-006312f811-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.922 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:37.923 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:37.924 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:37.924 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:37.925 DEBUG:boto:Method: GET 2022-09-03T06:45:37.926 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:45:37.926 DEBUG:boto:Data: 2022-09-03T06:45:37.927 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.927 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:37.928 DEBUG:boto:Port: 8001 2022-09-03T06:45:37.928 DEBUG:boto:Params: {} 2022-09-03T06:45:37.929 DEBUG:boto:Token: None 2022-09-03T06:45:37.930 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/ 2022-09-03T06:45:37.930 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU= 2022-09-03T06:45:37.931 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU=', 'Content-Length': '0'} 2022-09-03T06:45:37.934 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ecb766d2c6a5354a-006312f811-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.935 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:37.936 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:37.937 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:37.938 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:37.939 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:37.940 DEBUG:boto:Method: GET 2022-09-03T06:45:37.941 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:45:37.943 DEBUG:boto:Data: 2022-09-03T06:45:37.943 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.944 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:37.945 DEBUG:boto:Port: 8000 2022-09-03T06:45:37.946 DEBUG:boto:Params: {} 2022-09-03T06:45:37.946 DEBUG:boto:Token: None 2022-09-03T06:45:37.947 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/ 2022-09-03T06:45:37.947 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU= 2022-09-03T06:45:37.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:67WkJj/rSUsN2fB6gI90mPwZjUU=', 'Content-Length': '0'} 2022-09-03T06:45:37.952 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ed3069955609cb25-006312f811-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.953 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:37.954 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:45:37.955 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:37.956 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:37.956 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:37.957 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:37.957 DEBUG:boto:Method: GET 2022-09-03T06:45:37.958 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:37.959 DEBUG:boto:Data: 2022-09-03T06:45:37.959 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.960 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:37.960 DEBUG:boto:Port: 8001 2022-09-03T06:45:37.962 DEBUG:boto:Params: {} 2022-09-03T06:45:37.962 DEBUG:boto:Token: None 2022-09-03T06:45:37.963 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:37.964 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3IE1imTap/XizWiYoAoZGyMcD6E= 2022-09-03T06:45:37.966 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3IE1imTap/XizWiYoAoZGyMcD6E=', 'Content-Length': '0'} 2022-09-03T06:45:37.968 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002368e335689673ef-006312f811-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:37.970 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:37.971 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:37.972 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:37.974 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:37.975 DEBUG:boto:Method: GET 2022-09-03T06:45:37.975 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:37.976 DEBUG:boto:Data: 2022-09-03T06:45:37.977 DEBUG:boto:Headers: {} 2022-09-03T06:45:37.977 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:37.978 DEBUG:boto:Port: 8000 2022-09-03T06:45:37.979 DEBUG:boto:Params: {} 2022-09-03T06:45:37.979 DEBUG:boto:Token: None 2022-09-03T06:45:37.980 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:37 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:37.981 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3IE1imTap/XizWiYoAoZGyMcD6E= 2022-09-03T06:45:37.982 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3IE1imTap/XizWiYoAoZGyMcD6E=', 'Content-Length': '0'} 2022-09-03T06:45:37.985 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e6fa4cf559bf5ce0-006312f811-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.028 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:38.030 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:38.031 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:38.033 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:38.033 DEBUG:boto:Method: HEAD 2022-09-03T06:45:38.034 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:38.035 DEBUG:boto:Data: 2022-09-03T06:45:38.036 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.037 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.038 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.039 DEBUG:boto:Params: {} 2022-09-03T06:45:38.040 DEBUG:boto:Token: None 2022-09-03T06:45:38.041 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:38.043 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DX7P/g5I9OnKf3ATSyG8x9Z8COc= 2022-09-03T06:45:38.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DX7P/g5I9OnKf3ATSyG8x9Z8COc=', 'Content-Length': '0'} 2022-09-03T06:45:38.047 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003953ee8b0554d347-006312f812-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.048 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:38.049 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:38.050 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:38.051 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:38.052 DEBUG:boto:Method: HEAD 2022-09-03T06:45:38.053 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:38.054 DEBUG:boto:Data: 2022-09-03T06:45:38.055 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.056 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.057 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.058 DEBUG:boto:Params: {} 2022-09-03T06:45:38.059 DEBUG:boto:Token: None 2022-09-03T06:45:38.060 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:38.061 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DX7P/g5I9OnKf3ATSyG8x9Z8COc= 2022-09-03T06:45:38.063 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DX7P/g5I9OnKf3ATSyG8x9Z8COc=', 'Content-Length': '0'} 2022-09-03T06:45:38.069 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000406fb461c999515f-006312f812-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.071 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:45:38.072 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:38.073 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:38.074 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:38.075 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:38.076 DEBUG:boto:Method: GET 2022-09-03T06:45:38.078 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:38.079 DEBUG:boto:Data: 2022-09-03T06:45:38.079 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.080 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.081 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.082 DEBUG:boto:Params: {} 2022-09-03T06:45:38.083 DEBUG:boto:Token: None 2022-09-03T06:45:38.084 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:38.085 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4lIScBQTsPXWxJWiYbjyk9lIao0= 2022-09-03T06:45:38.086 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4lIScBQTsPXWxJWiYbjyk9lIao0=', 'Content-Length': '0'} 2022-09-03T06:45:38.088 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000092d6906c3defa706-006312f812-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.089 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:38.090 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:38.090 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:38.091 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:38.091 DEBUG:boto:Method: GET 2022-09-03T06:45:38.092 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:38.093 DEBUG:boto:Data: 2022-09-03T06:45:38.093 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.094 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.095 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.095 DEBUG:boto:Params: {} 2022-09-03T06:45:38.096 DEBUG:boto:Token: None 2022-09-03T06:45:38.097 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:38.097 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4lIScBQTsPXWxJWiYbjyk9lIao0= 2022-09-03T06:45:38.098 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4lIScBQTsPXWxJWiYbjyk9lIao0=', 'Content-Length': '0'} 2022-09-03T06:45:38.100 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000090c5d22413b98a72-006312f812-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.101 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:38.102 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:38.103 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:38.104 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:38.105 DEBUG:boto:Method: GET 2022-09-03T06:45:38.106 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:45:38.107 DEBUG:boto:Data: 2022-09-03T06:45:38.108 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.108 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.109 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.110 DEBUG:boto:Params: {} 2022-09-03T06:45:38.111 DEBUG:boto:Token: None 2022-09-03T06:45:38.112 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/ 2022-09-03T06:45:38.113 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0= 2022-09-03T06:45:38.114 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0=', 'Content-Length': '0'} 2022-09-03T06:45:38.117 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ad672802694d9747-006312f812-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.118 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:38.119 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:38.120 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:38.121 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:38.121 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:38.122 DEBUG:boto:Method: GET 2022-09-03T06:45:38.123 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:45:38.123 DEBUG:boto:Data: 2022-09-03T06:45:38.124 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.125 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.126 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.126 DEBUG:boto:Params: {} 2022-09-03T06:45:38.127 DEBUG:boto:Token: None 2022-09-03T06:45:38.127 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/ 2022-09-03T06:45:38.128 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0= 2022-09-03T06:45:38.129 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0=', 'Content-Length': '0'} 2022-09-03T06:45:38.133 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000066c60a080c62aff9-006312f812-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.134 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:38.135 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:38.136 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:38.136 DEBUG:boto:Method: HEAD 2022-09-03T06:45:38.137 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:45:38.137 DEBUG:boto:Data: 2022-09-03T06:45:38.138 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.138 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.139 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.139 DEBUG:boto:Params: {} 2022-09-03T06:45:38.140 DEBUG:boto:Token: None 2022-09-03T06:45:38.141 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj85 2022-09-03T06:45:38.141 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rbegbp2WiRxTxWR3uhA+Vi1qu3g= 2022-09-03T06:45:38.142 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rbegbp2WiRxTxWR3uhA+Vi1qu3g=', 'Content-Length': '0'} 2022-09-03T06:45:38.144 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c99dab6df6503717-006312f812-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.145 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:38.146 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:38.146 DEBUG:boto:Method: HEAD 2022-09-03T06:45:38.147 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:45:38.148 DEBUG:boto:Data: 2022-09-03T06:45:38.149 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.149 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.150 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.150 DEBUG:boto:Params: {} 2022-09-03T06:45:38.151 DEBUG:boto:Token: None 2022-09-03T06:45:38.152 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj85 2022-09-03T06:45:38.152 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rbegbp2WiRxTxWR3uhA+Vi1qu3g= 2022-09-03T06:45:38.153 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rbegbp2WiRxTxWR3uhA+Vi1qu3g=', 'Content-Length': '0'} 2022-09-03T06:45:38.155 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002665983396410411-006312f812-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.156 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:45:38.157 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:38.157 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:38.158 DEBUG:boto:Method: GET 2022-09-03T06:45:38.158 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:45:38.159 DEBUG:boto:Data: 2022-09-03T06:45:38.159 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.160 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.160 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.161 DEBUG:boto:Params: {} 2022-09-03T06:45:38.162 DEBUG:boto:Token: None 2022-09-03T06:45:38.162 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj85 2022-09-03T06:45:38.163 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eKYE10aGrwEtHzKRHc5R67aLTwE= 2022-09-03T06:45:38.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eKYE10aGrwEtHzKRHc5R67aLTwE=', 'Content-Length': '0'} 2022-09-03T06:45:38.166 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000019759b990a9ea0fd-006312f812-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.167 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:38.168 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:38.169 DEBUG:boto:Method: GET 2022-09-03T06:45:38.170 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:45:38.170 DEBUG:boto:Data: 2022-09-03T06:45:38.171 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.171 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.172 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.172 DEBUG:boto:Params: {} 2022-09-03T06:45:38.173 DEBUG:boto:Token: None 2022-09-03T06:45:38.174 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj85 2022-09-03T06:45:38.175 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eKYE10aGrwEtHzKRHc5R67aLTwE= 2022-09-03T06:45:38.175 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eKYE10aGrwEtHzKRHc5R67aLTwE=', 'Content-Length': '0'} 2022-09-03T06:45:38.177 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096816edbbd270584-006312f812-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.222 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:38.223 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:38.224 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:38.224 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:38.225 DEBUG:boto:Method: GET 2022-09-03T06:45:38.225 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:45:38.226 DEBUG:boto:Data: 2022-09-03T06:45:38.226 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.227 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.227 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.228 DEBUG:boto:Params: {} 2022-09-03T06:45:38.229 DEBUG:boto:Token: None 2022-09-03T06:45:38.229 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/ 2022-09-03T06:45:38.230 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0= 2022-09-03T06:45:38.231 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0=', 'Content-Length': '0'} 2022-09-03T06:45:38.234 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a509390a6cdd9302-006312f812-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.234 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:38.235 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:38.236 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:38.236 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:38.237 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:38.237 DEBUG:boto:Method: GET 2022-09-03T06:45:38.238 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:45:38.239 DEBUG:boto:Data: 2022-09-03T06:45:38.239 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.240 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.241 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.241 DEBUG:boto:Params: {} 2022-09-03T06:45:38.242 DEBUG:boto:Token: None 2022-09-03T06:45:38.242 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/ 2022-09-03T06:45:38.243 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0= 2022-09-03T06:45:38.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0=', 'Content-Length': '0'} 2022-09-03T06:45:38.248 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002a06b70d959d80ce-006312f812-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.249 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:38.250 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:45:38.250 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:38.251 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:38.251 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.252 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.252 DEBUG:boto:Method: GET 2022-09-03T06:45:38.253 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.254 DEBUG:boto:Data: 2022-09-03T06:45:38.254 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.255 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.255 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.256 DEBUG:boto:Params: {} 2022-09-03T06:45:38.256 DEBUG:boto:Token: None 2022-09-03T06:45:38.257 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.258 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KTCTQd6JIunu4fmDYrG9jdKLPs0= 2022-09-03T06:45:38.259 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KTCTQd6JIunu4fmDYrG9jdKLPs0=', 'Content-Length': '0'} 2022-09-03T06:45:38.261 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004ba172717a3c7d7b-006312f812-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.262 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:38.263 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:38.263 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.264 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.264 DEBUG:boto:Method: GET 2022-09-03T06:45:38.265 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.265 DEBUG:boto:Data: 2022-09-03T06:45:38.266 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.266 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.267 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.268 DEBUG:boto:Params: {} 2022-09-03T06:45:38.268 DEBUG:boto:Token: None 2022-09-03T06:45:38.269 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.269 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KTCTQd6JIunu4fmDYrG9jdKLPs0= 2022-09-03T06:45:38.270 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KTCTQd6JIunu4fmDYrG9jdKLPs0=', 'Content-Length': '0'} 2022-09-03T06:45:38.273 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001bda1737a6ecbc68-006312f812-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.273 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:38.274 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:38.275 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.275 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.276 DEBUG:boto:Method: HEAD 2022-09-03T06:45:38.276 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.277 DEBUG:boto:Data: 2022-09-03T06:45:38.277 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.278 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.278 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.279 DEBUG:boto:Params: {} 2022-09-03T06:45:38.279 DEBUG:boto:Token: None 2022-09-03T06:45:38.280 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.280 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MOLZG0pdMrSreeuLCCJYNPA0Q00= 2022-09-03T06:45:38.281 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MOLZG0pdMrSreeuLCCJYNPA0Q00=', 'Content-Length': '0'} 2022-09-03T06:45:38.283 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000016e18ac18733798d-006312f812-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.284 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:38.285 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:38.285 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.286 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.286 DEBUG:boto:Method: HEAD 2022-09-03T06:45:38.287 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.287 DEBUG:boto:Data: 2022-09-03T06:45:38.288 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.288 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.289 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.289 DEBUG:boto:Params: {} 2022-09-03T06:45:38.290 DEBUG:boto:Token: None 2022-09-03T06:45:38.291 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.291 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MOLZG0pdMrSreeuLCCJYNPA0Q00= 2022-09-03T06:45:38.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MOLZG0pdMrSreeuLCCJYNPA0Q00=', 'Content-Length': '0'} 2022-09-03T06:45:38.295 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004129b3b0ad028bf4-006312f812-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.295 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:45:38.296 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:38.297 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:38.297 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.298 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.299 DEBUG:boto:Method: GET 2022-09-03T06:45:38.299 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.300 DEBUG:boto:Data: 2022-09-03T06:45:38.300 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.301 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.301 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.302 DEBUG:boto:Params: {} 2022-09-03T06:45:38.303 DEBUG:boto:Token: None 2022-09-03T06:45:38.303 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.304 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KTCTQd6JIunu4fmDYrG9jdKLPs0= 2022-09-03T06:45:38.305 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KTCTQd6JIunu4fmDYrG9jdKLPs0=', 'Content-Length': '0'} 2022-09-03T06:45:38.307 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b3ecf548c40607d9-006312f812-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.354 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:38.355 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:38.356 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.357 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.358 DEBUG:boto:Method: GET 2022-09-03T06:45:38.359 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.360 DEBUG:boto:Data: 2022-09-03T06:45:38.360 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.361 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.362 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.362 DEBUG:boto:Params: {} 2022-09-03T06:45:38.363 DEBUG:boto:Token: None 2022-09-03T06:45:38.363 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:38.364 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KTCTQd6JIunu4fmDYrG9jdKLPs0= 2022-09-03T06:45:38.365 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KTCTQd6JIunu4fmDYrG9jdKLPs0=', 'Content-Length': '0'} 2022-09-03T06:45:38.367 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008dc1877702456a58-006312f812-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.368 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:38.369 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:38.369 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:38.370 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:38.371 DEBUG:boto:Method: GET 2022-09-03T06:45:38.372 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:45:38.372 DEBUG:boto:Data: 2022-09-03T06:45:38.373 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.374 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.375 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.375 DEBUG:boto:Params: {} 2022-09-03T06:45:38.376 DEBUG:boto:Token: None 2022-09-03T06:45:38.376 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/ 2022-09-03T06:45:38.377 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0= 2022-09-03T06:45:38.378 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0=', 'Content-Length': '0'} 2022-09-03T06:45:38.381 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000ef0506bca9ed431-006312f812-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.381 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:38.382 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:38.383 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:38.384 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:38.384 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:38.385 DEBUG:boto:Method: GET 2022-09-03T06:45:38.385 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:45:38.386 DEBUG:boto:Data: 2022-09-03T06:45:38.386 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.387 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.387 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.388 DEBUG:boto:Params: {} 2022-09-03T06:45:38.389 DEBUG:boto:Token: None 2022-09-03T06:45:38.389 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/ 2022-09-03T06:45:38.390 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0= 2022-09-03T06:45:38.391 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0=', 'Content-Length': '0'} 2022-09-03T06:45:38.394 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dbc3ab5fa7451ba9-006312f812-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.438 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:38.439 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:38.440 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:38.441 DEBUG:boto:Method: HEAD 2022-09-03T06:45:38.441 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:45:38.442 DEBUG:boto:Data: 2022-09-03T06:45:38.442 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.443 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.443 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.444 DEBUG:boto:Params: {} 2022-09-03T06:45:38.444 DEBUG:boto:Token: None 2022-09-03T06:45:38.445 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj87 2022-09-03T06:45:38.446 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G5ZjOsn10aInegK9JA4wOhZWblo= 2022-09-03T06:45:38.446 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G5ZjOsn10aInegK9JA4wOhZWblo=', 'Content-Length': '0'} 2022-09-03T06:45:38.449 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a8ce3b1a53ef8df-006312f812-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.450 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:38.450 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:38.451 DEBUG:boto:Method: HEAD 2022-09-03T06:45:38.451 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:45:38.452 DEBUG:boto:Data: 2022-09-03T06:45:38.452 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.453 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.453 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.454 DEBUG:boto:Params: {} 2022-09-03T06:45:38.455 DEBUG:boto:Token: None 2022-09-03T06:45:38.455 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj87 2022-09-03T06:45:38.456 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G5ZjOsn10aInegK9JA4wOhZWblo= 2022-09-03T06:45:38.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G5ZjOsn10aInegK9JA4wOhZWblo=', 'Content-Length': '0'} 2022-09-03T06:45:38.459 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eefd43442ea504ee-006312f812-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.459 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:45:38.460 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:38.461 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:38.461 DEBUG:boto:Method: GET 2022-09-03T06:45:38.462 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:45:38.462 DEBUG:boto:Data: 2022-09-03T06:45:38.463 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.463 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.464 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.465 DEBUG:boto:Params: {} 2022-09-03T06:45:38.465 DEBUG:boto:Token: None 2022-09-03T06:45:38.466 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj87 2022-09-03T06:45:38.467 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rJcY9TECdrbOCVGDharE5+iQ8dY= 2022-09-03T06:45:38.467 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rJcY9TECdrbOCVGDharE5+iQ8dY=', 'Content-Length': '0'} 2022-09-03T06:45:38.469 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e18cf88e55f43c89-006312f812-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.470 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:38.471 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:38.472 DEBUG:boto:Method: GET 2022-09-03T06:45:38.472 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:45:38.473 DEBUG:boto:Data: 2022-09-03T06:45:38.473 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.474 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.474 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.475 DEBUG:boto:Params: {} 2022-09-03T06:45:38.476 DEBUG:boto:Token: None 2022-09-03T06:45:38.476 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj87 2022-09-03T06:45:38.477 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rJcY9TECdrbOCVGDharE5+iQ8dY= 2022-09-03T06:45:38.478 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rJcY9TECdrbOCVGDharE5+iQ8dY=', 'Content-Length': '0'} 2022-09-03T06:45:38.480 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d9c8cb9bb43f1ef9-006312f812-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.481 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:38.481 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:38.482 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:38.482 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:38.483 DEBUG:boto:Method: GET 2022-09-03T06:45:38.483 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:45:38.484 DEBUG:boto:Data: 2022-09-03T06:45:38.485 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.485 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.486 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.486 DEBUG:boto:Params: {} 2022-09-03T06:45:38.487 DEBUG:boto:Token: None 2022-09-03T06:45:38.488 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/ 2022-09-03T06:45:38.488 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0= 2022-09-03T06:45:38.489 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0=', 'Content-Length': '0'} 2022-09-03T06:45:38.492 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000508f243c88ecc837-006312f812-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.493 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:38.493 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:38.494 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:38.495 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:38.495 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:38.496 DEBUG:boto:Method: GET 2022-09-03T06:45:38.496 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:45:38.497 DEBUG:boto:Data: 2022-09-03T06:45:38.497 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.498 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.498 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.499 DEBUG:boto:Params: {} 2022-09-03T06:45:38.500 DEBUG:boto:Token: None 2022-09-03T06:45:38.500 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/ 2022-09-03T06:45:38.501 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0= 2022-09-03T06:45:38.502 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0=', 'Content-Length': '0'} 2022-09-03T06:45:38.505 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009935efd1e3523166-006312f812-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.506 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:38.507 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:45:38.508 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:38.508 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:38.509 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.509 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.510 DEBUG:boto:Method: GET 2022-09-03T06:45:38.511 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.511 DEBUG:boto:Data: 2022-09-03T06:45:38.512 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.512 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.513 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.513 DEBUG:boto:Params: {} 2022-09-03T06:45:38.514 DEBUG:boto:Token: None 2022-09-03T06:45:38.515 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.516 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FYg+k6IbbhOIouhFW0HXwIlwUdc= 2022-09-03T06:45:38.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FYg+k6IbbhOIouhFW0HXwIlwUdc=', 'Content-Length': '0'} 2022-09-03T06:45:38.518 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000027cb43da047ad721-006312f812-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.519 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:38.520 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:38.520 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.521 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.521 DEBUG:boto:Method: GET 2022-09-03T06:45:38.522 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.523 DEBUG:boto:Data: 2022-09-03T06:45:38.523 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.524 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.524 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.525 DEBUG:boto:Params: {} 2022-09-03T06:45:38.526 DEBUG:boto:Token: None 2022-09-03T06:45:38.526 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.527 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FYg+k6IbbhOIouhFW0HXwIlwUdc= 2022-09-03T06:45:38.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FYg+k6IbbhOIouhFW0HXwIlwUdc=', 'Content-Length': '0'} 2022-09-03T06:45:38.530 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d274e1a425438dc4-006312f812-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.531 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:38.532 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:38.533 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.534 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.535 DEBUG:boto:Method: HEAD 2022-09-03T06:45:38.536 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.537 DEBUG:boto:Data: 2022-09-03T06:45:38.538 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.538 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.539 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.540 DEBUG:boto:Params: {} 2022-09-03T06:45:38.541 DEBUG:boto:Token: None 2022-09-03T06:45:38.543 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.544 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:g7umpcO1KO26fJb410Dm32BUcvg= 2022-09-03T06:45:38.545 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:g7umpcO1KO26fJb410Dm32BUcvg=', 'Content-Length': '0'} 2022-09-03T06:45:38.548 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006f21bfa1e4d30510-006312f812-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.549 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:38.550 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:38.551 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.552 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.553 DEBUG:boto:Method: HEAD 2022-09-03T06:45:38.553 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.554 DEBUG:boto:Data: 2022-09-03T06:45:38.555 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.556 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.557 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.558 DEBUG:boto:Params: {} 2022-09-03T06:45:38.559 DEBUG:boto:Token: None 2022-09-03T06:45:38.559 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.560 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:g7umpcO1KO26fJb410Dm32BUcvg= 2022-09-03T06:45:38.561 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:g7umpcO1KO26fJb410Dm32BUcvg=', 'Content-Length': '0'} 2022-09-03T06:45:38.563 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000063af4068fea007d8-006312f812-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.564 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:45:38.565 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:38.566 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:38.566 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.567 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.567 DEBUG:boto:Method: GET 2022-09-03T06:45:38.568 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.568 DEBUG:boto:Data: 2022-09-03T06:45:38.569 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.569 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.570 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.570 DEBUG:boto:Params: {} 2022-09-03T06:45:38.571 DEBUG:boto:Token: None 2022-09-03T06:45:38.572 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.572 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FYg+k6IbbhOIouhFW0HXwIlwUdc= 2022-09-03T06:45:38.573 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FYg+k6IbbhOIouhFW0HXwIlwUdc=', 'Content-Length': '0'} 2022-09-03T06:45:38.575 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000025f3eb6a49f9f4dd-006312f812-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.576 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:38.577 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:38.577 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.578 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.578 DEBUG:boto:Method: GET 2022-09-03T06:45:38.579 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.579 DEBUG:boto:Data: 2022-09-03T06:45:38.580 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.580 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.581 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.581 DEBUG:boto:Params: {} 2022-09-03T06:45:38.582 DEBUG:boto:Token: None 2022-09-03T06:45:38.583 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:38.583 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FYg+k6IbbhOIouhFW0HXwIlwUdc= 2022-09-03T06:45:38.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FYg+k6IbbhOIouhFW0HXwIlwUdc=', 'Content-Length': '0'} 2022-09-03T06:45:38.586 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000035dfbb258f7103e5-006312f812-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.634 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:38.635 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:38.635 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:38.636 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:38.637 DEBUG:boto:Method: GET 2022-09-03T06:45:38.638 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:45:38.638 DEBUG:boto:Data: 2022-09-03T06:45:38.639 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.639 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.640 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.641 DEBUG:boto:Params: {} 2022-09-03T06:45:38.642 DEBUG:boto:Token: None 2022-09-03T06:45:38.643 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/ 2022-09-03T06:45:38.644 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0= 2022-09-03T06:45:38.645 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0=', 'Content-Length': '0'} 2022-09-03T06:45:38.649 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dcd23a48e89ba718-006312f812-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.649 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:38.650 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:38.651 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:38.652 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:38.652 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:38.653 DEBUG:boto:Method: GET 2022-09-03T06:45:38.653 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:45:38.654 DEBUG:boto:Data: 2022-09-03T06:45:38.654 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.655 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.655 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.656 DEBUG:boto:Params: {} 2022-09-03T06:45:38.657 DEBUG:boto:Token: None 2022-09-03T06:45:38.657 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/ 2022-09-03T06:45:38.658 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0= 2022-09-03T06:45:38.659 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0=', 'Content-Length': '0'} 2022-09-03T06:45:38.662 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008bd8c2d3d85d2845-006312f812-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.663 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:38.664 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:38.665 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:38.665 DEBUG:boto:Method: HEAD 2022-09-03T06:45:38.666 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:45:38.666 DEBUG:boto:Data: 2022-09-03T06:45:38.667 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.667 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.679 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.680 DEBUG:boto:Params: {} 2022-09-03T06:45:38.680 DEBUG:boto:Token: None 2022-09-03T06:45:38.681 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj92 2022-09-03T06:45:38.682 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i1JI+QYFwMYyGxw5vouvaCNfNGo= 2022-09-03T06:45:38.682 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i1JI+QYFwMYyGxw5vouvaCNfNGo=', 'Content-Length': '0'} 2022-09-03T06:45:38.685 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000679361e8dbb5f930-006312f812-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.686 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:38.686 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:38.687 DEBUG:boto:Method: HEAD 2022-09-03T06:45:38.687 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:45:38.688 DEBUG:boto:Data: 2022-09-03T06:45:38.688 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.689 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.690 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.691 DEBUG:boto:Params: {} 2022-09-03T06:45:38.691 DEBUG:boto:Token: None 2022-09-03T06:45:38.692 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj92 2022-09-03T06:45:38.692 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i1JI+QYFwMYyGxw5vouvaCNfNGo= 2022-09-03T06:45:38.693 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i1JI+QYFwMYyGxw5vouvaCNfNGo=', 'Content-Length': '0'} 2022-09-03T06:45:38.696 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000112210a7625eda24-006312f812-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.696 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:45:38.697 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:38.698 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:38.698 DEBUG:boto:Method: GET 2022-09-03T06:45:38.699 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:45:38.699 DEBUG:boto:Data: 2022-09-03T06:45:38.700 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.700 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.701 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.702 DEBUG:boto:Params: {} 2022-09-03T06:45:38.702 DEBUG:boto:Token: None 2022-09-03T06:45:38.703 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj92 2022-09-03T06:45:38.704 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ypTbYkTxZLe98bed8+tr4lpDyT8= 2022-09-03T06:45:38.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ypTbYkTxZLe98bed8+tr4lpDyT8=', 'Content-Length': '0'} 2022-09-03T06:45:38.706 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d58dde1f53d3644-006312f812-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.707 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:38.708 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:38.708 DEBUG:boto:Method: GET 2022-09-03T06:45:38.709 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:45:38.709 DEBUG:boto:Data: 2022-09-03T06:45:38.710 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.711 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.711 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.712 DEBUG:boto:Params: {} 2022-09-03T06:45:38.712 DEBUG:boto:Token: None 2022-09-03T06:45:38.713 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj92 2022-09-03T06:45:38.714 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ypTbYkTxZLe98bed8+tr4lpDyT8= 2022-09-03T06:45:38.714 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ypTbYkTxZLe98bed8+tr4lpDyT8=', 'Content-Length': '0'} 2022-09-03T06:45:38.716 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000004398e222c5ec1cf-006312f812-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.717 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:38.718 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:38.718 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:38.719 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:38.719 DEBUG:boto:Method: GET 2022-09-03T06:45:38.720 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:45:38.720 DEBUG:boto:Data: 2022-09-03T06:45:38.721 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.721 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.722 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.723 DEBUG:boto:Params: {} 2022-09-03T06:45:38.723 DEBUG:boto:Token: None 2022-09-03T06:45:38.724 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/ 2022-09-03T06:45:38.724 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0= 2022-09-03T06:45:38.725 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0=', 'Content-Length': '0'} 2022-09-03T06:45:38.728 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001d3ae21e94bb9998-006312f812-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.770 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:38.771 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:38.772 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:38.772 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:38.773 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:38.773 DEBUG:boto:Method: GET 2022-09-03T06:45:38.774 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:45:38.775 DEBUG:boto:Data: 2022-09-03T06:45:38.775 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.776 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.777 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.777 DEBUG:boto:Params: {} 2022-09-03T06:45:38.778 DEBUG:boto:Token: None 2022-09-03T06:45:38.779 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/ 2022-09-03T06:45:38.780 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0= 2022-09-03T06:45:38.780 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0=', 'Content-Length': '0'} 2022-09-03T06:45:38.784 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dfb7a32a23975fc0-006312f812-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.785 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:38.786 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:45:38.787 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:38.788 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:38.788 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.789 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.789 DEBUG:boto:Method: GET 2022-09-03T06:45:38.790 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.790 DEBUG:boto:Data: 2022-09-03T06:45:38.791 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.792 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.792 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.793 DEBUG:boto:Params: {} 2022-09-03T06:45:38.793 DEBUG:boto:Token: None 2022-09-03T06:45:38.794 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.795 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WgYdejc69GV1KX9GqzPX1L2dL24= 2022-09-03T06:45:38.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WgYdejc69GV1KX9GqzPX1L2dL24=', 'Content-Length': '0'} 2022-09-03T06:45:38.798 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009217e2f9f40feba9-006312f812-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.799 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:38.799 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:38.800 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.801 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.801 DEBUG:boto:Method: GET 2022-09-03T06:45:38.802 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.802 DEBUG:boto:Data: 2022-09-03T06:45:38.803 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.803 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.804 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.804 DEBUG:boto:Params: {} 2022-09-03T06:45:38.805 DEBUG:boto:Token: None 2022-09-03T06:45:38.806 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.806 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WgYdejc69GV1KX9GqzPX1L2dL24= 2022-09-03T06:45:38.807 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WgYdejc69GV1KX9GqzPX1L2dL24=', 'Content-Length': '0'} 2022-09-03T06:45:38.810 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004e1fcd495b3f4a7a-006312f812-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.858 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:38.859 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:38.860 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.860 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.861 DEBUG:boto:Method: HEAD 2022-09-03T06:45:38.861 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.862 DEBUG:boto:Data: 2022-09-03T06:45:38.862 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.863 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.864 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.864 DEBUG:boto:Params: {} 2022-09-03T06:45:38.865 DEBUG:boto:Token: None 2022-09-03T06:45:38.865 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.866 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iKhosb3MFlkODv72vHeqFvuq1sc= 2022-09-03T06:45:38.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iKhosb3MFlkODv72vHeqFvuq1sc=', 'Content-Length': '0'} 2022-09-03T06:45:38.869 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000030a4a4dd4ff5b5b8-006312f812-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.870 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:38.871 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:38.872 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.872 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.873 DEBUG:boto:Method: HEAD 2022-09-03T06:45:38.874 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.875 DEBUG:boto:Data: 2022-09-03T06:45:38.876 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.877 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.878 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.879 DEBUG:boto:Params: {} 2022-09-03T06:45:38.880 DEBUG:boto:Token: None 2022-09-03T06:45:38.881 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.882 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iKhosb3MFlkODv72vHeqFvuq1sc= 2022-09-03T06:45:38.883 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iKhosb3MFlkODv72vHeqFvuq1sc=', 'Content-Length': '0'} 2022-09-03T06:45:38.885 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c8121e3d1abd6cb-006312f812-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.886 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:45:38.887 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:38.888 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:38.888 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.889 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.890 DEBUG:boto:Method: GET 2022-09-03T06:45:38.891 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.891 DEBUG:boto:Data: 2022-09-03T06:45:38.892 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.893 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.893 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.894 DEBUG:boto:Params: {} 2022-09-03T06:45:38.895 DEBUG:boto:Token: None 2022-09-03T06:45:38.896 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.896 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WgYdejc69GV1KX9GqzPX1L2dL24= 2022-09-03T06:45:38.897 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WgYdejc69GV1KX9GqzPX1L2dL24=', 'Content-Length': '0'} 2022-09-03T06:45:38.900 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000076bdc810f03e49f4-006312f812-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.901 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:38.901 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:38.902 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.902 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.903 DEBUG:boto:Method: GET 2022-09-03T06:45:38.904 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.905 DEBUG:boto:Data: 2022-09-03T06:45:38.905 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.906 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.906 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.907 DEBUG:boto:Params: {} 2022-09-03T06:45:38.908 DEBUG:boto:Token: None 2022-09-03T06:45:38.909 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:38.909 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WgYdejc69GV1KX9GqzPX1L2dL24= 2022-09-03T06:45:38.910 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WgYdejc69GV1KX9GqzPX1L2dL24=', 'Content-Length': '0'} 2022-09-03T06:45:38.913 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000092ff5a73a91ba235-006312f812-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.913 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:38.914 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:38.915 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:38.915 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:38.916 DEBUG:boto:Method: GET 2022-09-03T06:45:38.916 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:45:38.917 DEBUG:boto:Data: 2022-09-03T06:45:38.917 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.918 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.918 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.919 DEBUG:boto:Params: {} 2022-09-03T06:45:38.919 DEBUG:boto:Token: None 2022-09-03T06:45:38.920 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/ 2022-09-03T06:45:38.921 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0= 2022-09-03T06:45:38.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0=', 'Content-Length': '0'} 2022-09-03T06:45:38.928 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000da8543885c17cf97-006312f812-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.930 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:38.932 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:38.932 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:38.933 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:38.933 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:38.934 DEBUG:boto:Method: GET 2022-09-03T06:45:38.935 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:45:38.935 DEBUG:boto:Data: 2022-09-03T06:45:38.936 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.937 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.938 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.938 DEBUG:boto:Params: {} 2022-09-03T06:45:38.939 DEBUG:boto:Token: None 2022-09-03T06:45:38.940 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/ 2022-09-03T06:45:38.940 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0= 2022-09-03T06:45:38.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j3vN8ZGIwHHWX6x1qhCFJb8b6v0=', 'Content-Length': '0'} 2022-09-03T06:45:38.946 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003f918ee85c350d35-006312f812-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.947 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:38.948 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:38.948 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:38.949 DEBUG:boto:Method: HEAD 2022-09-03T06:45:38.950 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:45:38.950 DEBUG:boto:Data: 2022-09-03T06:45:38.951 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.951 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.952 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.953 DEBUG:boto:Params: {} 2022-09-03T06:45:38.954 DEBUG:boto:Token: None 2022-09-03T06:45:38.955 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj95 2022-09-03T06:45:38.955 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O39r4WbQjLgOul0VEDTKAvcetPA= 2022-09-03T06:45:38.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O39r4WbQjLgOul0VEDTKAvcetPA=', 'Content-Length': '0'} 2022-09-03T06:45:38.959 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000097ea8085e990201a-006312f812-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.961 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:38.962 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:38.963 DEBUG:boto:Method: HEAD 2022-09-03T06:45:38.964 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:45:38.964 DEBUG:boto:Data: 2022-09-03T06:45:38.965 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.967 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:38.967 DEBUG:boto:Port: 8000 2022-09-03T06:45:38.968 DEBUG:boto:Params: {} 2022-09-03T06:45:38.969 DEBUG:boto:Token: None 2022-09-03T06:45:38.971 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj95 2022-09-03T06:45:38.972 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O39r4WbQjLgOul0VEDTKAvcetPA= 2022-09-03T06:45:38.973 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O39r4WbQjLgOul0VEDTKAvcetPA=', 'Content-Length': '0'} 2022-09-03T06:45:38.976 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a91d04f53bef8879-006312f812-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:38.977 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:45:38.978 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:38.979 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:38.980 DEBUG:boto:Method: GET 2022-09-03T06:45:38.981 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:45:38.982 DEBUG:boto:Data: 2022-09-03T06:45:38.983 DEBUG:boto:Headers: {} 2022-09-03T06:45:38.984 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:38.985 DEBUG:boto:Port: 8001 2022-09-03T06:45:38.986 DEBUG:boto:Params: {} 2022-09-03T06:45:38.987 DEBUG:boto:Token: None 2022-09-03T06:45:38.988 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:38 GMT /dsgvcy-75/obj95 2022-09-03T06:45:38.989 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7K1u43PQwLJQjUhCZDUl6IW1fb8= 2022-09-03T06:45:38.991 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7K1u43PQwLJQjUhCZDUl6IW1fb8=', 'Content-Length': '0'} 2022-09-03T06:45:38.994 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a80b96820160c123-006312f812-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.034 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:39.035 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:39.036 DEBUG:boto:Method: GET 2022-09-03T06:45:39.036 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:45:39.037 DEBUG:boto:Data: 2022-09-03T06:45:39.038 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.039 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:39.039 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.040 DEBUG:boto:Params: {} 2022-09-03T06:45:39.041 DEBUG:boto:Token: None 2022-09-03T06:45:39.041 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj95 2022-09-03T06:45:39.042 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vCyUFtti6jA6AWXjX8UDFrtbzMI= 2022-09-03T06:45:39.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vCyUFtti6jA6AWXjX8UDFrtbzMI=', 'Content-Length': '0'} 2022-09-03T06:45:39.045 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e3d3768990cc5c12-006312f813-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.047 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:39.047 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:39.048 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:39.048 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:39.049 DEBUG:boto:Method: GET 2022-09-03T06:45:39.050 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:45:39.051 DEBUG:boto:Data: 2022-09-03T06:45:39.051 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.052 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.052 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.053 DEBUG:boto:Params: {} 2022-09-03T06:45:39.054 DEBUG:boto:Token: None 2022-09-03T06:45:39.055 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/ 2022-09-03T06:45:39.056 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM= 2022-09-03T06:45:39.057 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM=', 'Content-Length': '0'} 2022-09-03T06:45:39.060 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e47bbc615d2dd389-006312f813-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.061 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:39.062 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:39.063 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:39.064 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:39.065 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:39.066 DEBUG:boto:Method: GET 2022-09-03T06:45:39.067 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:45:39.068 DEBUG:boto:Data: 2022-09-03T06:45:39.068 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.069 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:39.069 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.070 DEBUG:boto:Params: {} 2022-09-03T06:45:39.071 DEBUG:boto:Token: None 2022-09-03T06:45:39.071 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/ 2022-09-03T06:45:39.072 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM= 2022-09-03T06:45:39.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM=', 'Content-Length': '0'} 2022-09-03T06:45:39.077 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c70c5f885e7991f8-006312f813-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.078 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:39.080 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-75 zones={test-zone3, test-zone1} 2022-09-03T06:45:39.081 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-75 zones={test-zone3, test-zone2} 2022-09-03T06:45:39.082 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:39.083 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:39.084 DEBUG:boto:Method: HEAD 2022-09-03T06:45:39.085 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:45:39.086 DEBUG:boto:Data: 2022-09-03T06:45:39.087 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.088 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.089 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.090 DEBUG:boto:Params: {} 2022-09-03T06:45:39.092 DEBUG:boto:Token: None 2022-09-03T06:45:39.093 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/ 2022-09-03T06:45:39.095 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jf2mfQ8N3agwGSUoutWq8JdqVBY= 2022-09-03T06:45:39.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jf2mfQ8N3agwGSUoutWq8JdqVBY=', 'Content-Length': '0'} 2022-09-03T06:45:39.102 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '25'), ('X-RGW-Bytes-Used', '75'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000007db035458033bfc1-006312f813-1851-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.103 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:39.105 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:39.105 DEBUG:boto:Method: HEAD 2022-09-03T06:45:39.106 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:45:39.106 DEBUG:boto:Data: 2022-09-03T06:45:39.107 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.108 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:39.108 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.109 DEBUG:boto:Params: {} 2022-09-03T06:45:39.109 DEBUG:boto:Token: None 2022-09-03T06:45:39.110 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/ 2022-09-03T06:45:39.111 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jf2mfQ8N3agwGSUoutWq8JdqVBY= 2022-09-03T06:45:39.111 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jf2mfQ8N3agwGSUoutWq8JdqVBY=', 'Content-Length': '0'} 2022-09-03T06:45:39.114 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '25'), ('X-RGW-Bytes-Used', '75'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000cf6144873832493a-006312f813-188c-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.116 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:45:39.117 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:39.118 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:39.118 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:39.119 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:39.120 DEBUG:boto:Method: GET 2022-09-03T06:45:39.121 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:39.122 DEBUG:boto:Data: 2022-09-03T06:45:39.123 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.124 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.124 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.126 DEBUG:boto:Params: {} 2022-09-03T06:45:39.128 DEBUG:boto:Token: None 2022-09-03T06:45:39.129 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/?versions 2022-09-03T06:45:39.131 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+cLO1Yo5lDeqbsgBHHYsKoN+Y10= 2022-09-03T06:45:39.132 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+cLO1Yo5lDeqbsgBHHYsKoN+Y10=', 'Content-Length': '0'} 2022-09-03T06:45:39.135 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000054ffc85bf5785c60-006312f813-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.137 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:39.140 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:45:39.141 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:45:39.142 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2022-09-03T06:45:39.143 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2022-09-03T06:45:39.144 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:45:39.145 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:45:39.146 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:45:39.147 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:45:39.147 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2022-09-03T06:45:39.149 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:45:39.150 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:45:39.151 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:45:39.152 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:45:39.153 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:45:39.154 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:45:39.155 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:45:39.158 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:45:39.159 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2022-09-03T06:45:39.160 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:45:39.160 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:45:39.161 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:45:39.162 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:45:39.163 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2022-09-03T06:45:39.165 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:45:39.165 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:45:39.167 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:45:39.167 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:39.168 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:39.169 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:39.169 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:39.170 DEBUG:boto:Method: GET 2022-09-03T06:45:39.170 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:39.171 DEBUG:boto:Data: 2022-09-03T06:45:39.172 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.172 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:39.173 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.173 DEBUG:boto:Params: {} 2022-09-03T06:45:39.174 DEBUG:boto:Token: None 2022-09-03T06:45:39.175 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/?versions 2022-09-03T06:45:39.176 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+cLO1Yo5lDeqbsgBHHYsKoN+Y10= 2022-09-03T06:45:39.176 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+cLO1Yo5lDeqbsgBHHYsKoN+Y10=', 'Content-Length': '0'} 2022-09-03T06:45:39.180 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e70be425df32e277-006312f813-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.180 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:39.183 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:45:39.184 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:45:39.184 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2022-09-03T06:45:39.185 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2022-09-03T06:45:39.185 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:45:39.186 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:45:39.187 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:45:39.188 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:45:39.189 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2022-09-03T06:45:39.190 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:45:39.190 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:45:39.191 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:45:39.191 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:45:39.192 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:45:39.192 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:45:39.193 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:45:39.193 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:45:39.194 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2022-09-03T06:45:39.195 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:45:39.196 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:45:39.197 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:45:39.198 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:45:39.199 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2022-09-03T06:45:39.200 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:45:39.200 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:45:39.201 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:39.202 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:39.203 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:39.204 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:39.205 DEBUG:boto:Method: GET 2022-09-03T06:45:39.205 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:39.206 DEBUG:boto:Data: 2022-09-03T06:45:39.207 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.207 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.208 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.208 DEBUG:boto:Params: {} 2022-09-03T06:45:39.209 DEBUG:boto:Token: None 2022-09-03T06:45:39.210 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/?versions 2022-09-03T06:45:39.210 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+cLO1Yo5lDeqbsgBHHYsKoN+Y10= 2022-09-03T06:45:39.211 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+cLO1Yo5lDeqbsgBHHYsKoN+Y10=', 'Content-Length': '0'} 2022-09-03T06:45:39.214 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000005113411449e06c2-006312f813-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.217 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:39.222 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:39.223 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:39.224 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:39.225 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:39.226 DEBUG:boto:Method: GET 2022-09-03T06:45:39.227 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:45:39.228 DEBUG:boto:Data: 2022-09-03T06:45:39.229 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.230 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:39.231 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.232 DEBUG:boto:Params: {} 2022-09-03T06:45:39.233 DEBUG:boto:Token: None 2022-09-03T06:45:39.234 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/?versions 2022-09-03T06:45:39.235 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+cLO1Yo5lDeqbsgBHHYsKoN+Y10= 2022-09-03T06:45:39.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+cLO1Yo5lDeqbsgBHHYsKoN+Y10=', 'Content-Length': '0'} 2022-09-03T06:45:39.241 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000045f45687c0e9a804-006312f813-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.242 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:39.244 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:45:39.245 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:39.246 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:39.246 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.247 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.249 DEBUG:boto:Method: GET 2022-09-03T06:45:39.250 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.251 DEBUG:boto:Data: 2022-09-03T06:45:39.253 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.254 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.255 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.256 DEBUG:boto:Params: {} 2022-09-03T06:45:39.257 DEBUG:boto:Token: None 2022-09-03T06:45:39.258 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.260 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xOFkb3zIkiwcBkSp6KCuNkiSfb0= 2022-09-03T06:45:39.261 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xOFkb3zIkiwcBkSp6KCuNkiSfb0=', 'Content-Length': '0'} 2022-09-03T06:45:39.266 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000647879d6312d6928-006312f813-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.268 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:39.271 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:39.272 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.272 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.273 DEBUG:boto:Method: GET 2022-09-03T06:45:39.274 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.276 DEBUG:boto:Data: 2022-09-03T06:45:39.277 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.277 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:39.278 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.279 DEBUG:boto:Params: {} 2022-09-03T06:45:39.280 DEBUG:boto:Token: None 2022-09-03T06:45:39.281 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.281 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xOFkb3zIkiwcBkSp6KCuNkiSfb0= 2022-09-03T06:45:39.282 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xOFkb3zIkiwcBkSp6KCuNkiSfb0=', 'Content-Length': '0'} 2022-09-03T06:45:39.285 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a1ca4eb14ac38f3-006312f813-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.286 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:39.287 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:39.287 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.288 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.288 DEBUG:boto:Method: HEAD 2022-09-03T06:45:39.289 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.290 DEBUG:boto:Data: 2022-09-03T06:45:39.290 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.291 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.292 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.292 DEBUG:boto:Params: {} 2022-09-03T06:45:39.293 DEBUG:boto:Token: None 2022-09-03T06:45:39.294 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.295 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sSawu5T+bspGDgqxNrvktoINxpM= 2022-09-03T06:45:39.295 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sSawu5T+bspGDgqxNrvktoINxpM=', 'Content-Length': '0'} 2022-09-03T06:45:39.297 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b9881bb8478d0a7-006312f813-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.298 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:39.299 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:39.300 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.300 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.301 DEBUG:boto:Method: HEAD 2022-09-03T06:45:39.301 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.302 DEBUG:boto:Data: 2022-09-03T06:45:39.302 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.303 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:39.304 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.304 DEBUG:boto:Params: {} 2022-09-03T06:45:39.305 DEBUG:boto:Token: None 2022-09-03T06:45:39.306 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.306 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sSawu5T+bspGDgqxNrvktoINxpM= 2022-09-03T06:45:39.307 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sSawu5T+bspGDgqxNrvktoINxpM=', 'Content-Length': '0'} 2022-09-03T06:45:39.309 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f023edaa4d04bb3e-006312f813-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.310 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:45:39.310 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:39.311 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:39.312 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.312 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.313 DEBUG:boto:Method: GET 2022-09-03T06:45:39.313 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.314 DEBUG:boto:Data: 2022-09-03T06:45:39.314 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.315 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.315 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.316 DEBUG:boto:Params: {} 2022-09-03T06:45:39.317 DEBUG:boto:Token: None 2022-09-03T06:45:39.318 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.318 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xOFkb3zIkiwcBkSp6KCuNkiSfb0= 2022-09-03T06:45:39.319 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xOFkb3zIkiwcBkSp6KCuNkiSfb0=', 'Content-Length': '0'} 2022-09-03T06:45:39.322 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e6e02edaf4cb5b42-006312f813-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.325 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:39.326 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:39.327 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.328 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.329 DEBUG:boto:Method: GET 2022-09-03T06:45:39.330 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.332 DEBUG:boto:Data: 2022-09-03T06:45:39.333 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.334 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:39.335 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.336 DEBUG:boto:Params: {} 2022-09-03T06:45:39.337 DEBUG:boto:Token: None 2022-09-03T06:45:39.339 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:45:39.341 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xOFkb3zIkiwcBkSp6KCuNkiSfb0= 2022-09-03T06:45:39.342 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xOFkb3zIkiwcBkSp6KCuNkiSfb0=', 'Content-Length': '0'} 2022-09-03T06:45:39.346 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e5f6db214b80f88e-006312f813-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.347 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:39.348 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:39.349 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:39.350 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:39.350 DEBUG:boto:Method: GET 2022-09-03T06:45:39.351 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:45:39.352 DEBUG:boto:Data: 2022-09-03T06:45:39.352 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.353 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.354 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.354 DEBUG:boto:Params: {} 2022-09-03T06:45:39.355 DEBUG:boto:Token: None 2022-09-03T06:45:39.357 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/ 2022-09-03T06:45:39.358 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM= 2022-09-03T06:45:39.358 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM=', 'Content-Length': '0'} 2022-09-03T06:45:39.364 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002a027e0feb3b4f03-006312f813-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.367 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:39.368 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:39.369 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:39.369 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:39.370 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:39.371 DEBUG:boto:Method: GET 2022-09-03T06:45:39.372 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:45:39.373 DEBUG:boto:Data: 2022-09-03T06:45:39.374 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.375 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:39.375 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.376 DEBUG:boto:Params: {} 2022-09-03T06:45:39.379 DEBUG:boto:Token: None 2022-09-03T06:45:39.379 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/ 2022-09-03T06:45:39.380 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM= 2022-09-03T06:45:39.381 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM=', 'Content-Length': '0'} 2022-09-03T06:45:39.387 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f6b00ed3667e52cf-006312f813-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.388 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:39.390 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:39.390 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:39.391 DEBUG:boto:Method: HEAD 2022-09-03T06:45:39.392 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:45:39.393 DEBUG:boto:Data: 2022-09-03T06:45:39.394 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.395 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.399 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.400 DEBUG:boto:Params: {} 2022-09-03T06:45:39.400 DEBUG:boto:Token: None 2022-09-03T06:45:39.401 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj11 2022-09-03T06:45:39.402 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1vL773NqJj4SFYhCsW/f0epXFcI= 2022-09-03T06:45:39.402 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1vL773NqJj4SFYhCsW/f0epXFcI=', 'Content-Length': '0'} 2022-09-03T06:45:39.406 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dca2be598e7686d2-006312f813-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.407 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:39.407 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:39.408 DEBUG:boto:Method: HEAD 2022-09-03T06:45:39.410 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:45:39.411 DEBUG:boto:Data: 2022-09-03T06:45:39.411 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.412 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:39.412 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.413 DEBUG:boto:Params: {} 2022-09-03T06:45:39.414 DEBUG:boto:Token: None 2022-09-03T06:45:39.415 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj11 2022-09-03T06:45:39.416 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1vL773NqJj4SFYhCsW/f0epXFcI= 2022-09-03T06:45:39.416 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1vL773NqJj4SFYhCsW/f0epXFcI=', 'Content-Length': '0'} 2022-09-03T06:45:39.422 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e852a0d0f0312fda-006312f813-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.424 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:45:39.425 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:39.426 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:39.427 DEBUG:boto:Method: GET 2022-09-03T06:45:39.428 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:45:39.428 DEBUG:boto:Data: 2022-09-03T06:45:39.429 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.430 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.431 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.432 DEBUG:boto:Params: {} 2022-09-03T06:45:39.432 DEBUG:boto:Token: None 2022-09-03T06:45:39.433 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj11 2022-09-03T06:45:39.434 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N52tk3gvCA5KQmMdtIcih8wfiIU= 2022-09-03T06:45:39.434 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N52tk3gvCA5KQmMdtIcih8wfiIU=', 'Content-Length': '0'} 2022-09-03T06:45:39.436 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000ba7547cfe9ab0bc-006312f813-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.437 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:45:39.438 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:45:39.439 DEBUG:boto:Method: GET 2022-09-03T06:45:39.439 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:45:39.440 DEBUG:boto:Data: 2022-09-03T06:45:39.441 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.442 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:39.443 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.443 DEBUG:boto:Params: {} 2022-09-03T06:45:39.444 DEBUG:boto:Token: None 2022-09-03T06:45:39.445 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj11 2022-09-03T06:45:39.445 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N52tk3gvCA5KQmMdtIcih8wfiIU= 2022-09-03T06:45:39.446 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N52tk3gvCA5KQmMdtIcih8wfiIU=', 'Content-Length': '0'} 2022-09-03T06:45:39.448 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b0291cfc75ba585f-006312f813-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.494 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:39.496 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:39.497 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:39.497 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:39.499 DEBUG:boto:Method: GET 2022-09-03T06:45:39.500 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:45:39.500 DEBUG:boto:Data: 2022-09-03T06:45:39.502 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.504 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.504 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.505 DEBUG:boto:Params: {} 2022-09-03T06:45:39.506 DEBUG:boto:Token: None 2022-09-03T06:45:39.507 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/ 2022-09-03T06:45:39.508 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM= 2022-09-03T06:45:39.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM=', 'Content-Length': '0'} 2022-09-03T06:45:39.511 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007ec8094b0295daf7-006312f813-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.513 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:39.514 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:39.515 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:39.516 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:39.516 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:45:39.517 DEBUG:boto:Method: GET 2022-09-03T06:45:39.517 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:45:39.518 DEBUG:boto:Data: 2022-09-03T06:45:39.518 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.519 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:39.519 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.520 DEBUG:boto:Params: {} 2022-09-03T06:45:39.521 DEBUG:boto:Token: None 2022-09-03T06:45:39.521 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/ 2022-09-03T06:45:39.522 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM= 2022-09-03T06:45:39.523 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM=', 'Content-Length': '0'} 2022-09-03T06:45:39.530 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e9392660292bf91c-006312f813-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.532 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:39.533 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:45:39.534 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:39.535 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:39.535 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.536 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.537 DEBUG:boto:Method: GET 2022-09-03T06:45:39.537 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.538 DEBUG:boto:Data: 2022-09-03T06:45:39.538 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.539 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.539 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.540 DEBUG:boto:Params: {} 2022-09-03T06:45:39.541 DEBUG:boto:Token: None 2022-09-03T06:45:39.541 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.542 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TG4rtoNqEXjh8IakdVdKBavZteI= 2022-09-03T06:45:39.543 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TG4rtoNqEXjh8IakdVdKBavZteI=', 'Content-Length': '0'} 2022-09-03T06:45:39.545 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009cb278b09f5be270-006312f813-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.546 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:39.546 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:39.547 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.547 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.548 DEBUG:boto:Method: GET 2022-09-03T06:45:39.548 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.549 DEBUG:boto:Data: 2022-09-03T06:45:39.549 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.550 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:39.551 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.552 DEBUG:boto:Params: {} 2022-09-03T06:45:39.552 DEBUG:boto:Token: None 2022-09-03T06:45:39.553 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.554 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TG4rtoNqEXjh8IakdVdKBavZteI= 2022-09-03T06:45:39.555 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TG4rtoNqEXjh8IakdVdKBavZteI=', 'Content-Length': '0'} 2022-09-03T06:45:39.557 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000084ff2443815159f9-006312f813-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.558 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:39.558 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:39.559 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.560 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.561 DEBUG:boto:Method: HEAD 2022-09-03T06:45:39.562 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.563 DEBUG:boto:Data: 2022-09-03T06:45:39.565 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.565 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.566 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.567 DEBUG:boto:Params: {} 2022-09-03T06:45:39.568 DEBUG:boto:Token: None 2022-09-03T06:45:39.570 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.571 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FlyLSb5IJXiOjvhA0HFjLAfM1E8= 2022-09-03T06:45:39.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FlyLSb5IJXiOjvhA0HFjLAfM1E8=', 'Content-Length': '0'} 2022-09-03T06:45:39.574 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bc133c76b5832d81-006312f813-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.575 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:39.575 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:39.576 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.576 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.577 DEBUG:boto:Method: HEAD 2022-09-03T06:45:39.577 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.579 DEBUG:boto:Data: 2022-09-03T06:45:39.580 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.581 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:39.581 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.582 DEBUG:boto:Params: {} 2022-09-03T06:45:39.583 DEBUG:boto:Token: None 2022-09-03T06:45:39.584 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.585 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FlyLSb5IJXiOjvhA0HFjLAfM1E8= 2022-09-03T06:45:39.586 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FlyLSb5IJXiOjvhA0HFjLAfM1E8=', 'Content-Length': '0'} 2022-09-03T06:45:39.589 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c726039a84d876ec-006312f813-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.590 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:45:39.591 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:39.592 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:39.592 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.593 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.593 DEBUG:boto:Method: GET 2022-09-03T06:45:39.594 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.594 DEBUG:boto:Data: 2022-09-03T06:45:39.595 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.595 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.596 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.597 DEBUG:boto:Params: {} 2022-09-03T06:45:39.598 DEBUG:boto:Token: None 2022-09-03T06:45:39.599 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.600 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TG4rtoNqEXjh8IakdVdKBavZteI= 2022-09-03T06:45:39.600 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TG4rtoNqEXjh8IakdVdKBavZteI=', 'Content-Length': '0'} 2022-09-03T06:45:39.602 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b34a717c6b9a4895-006312f813-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.603 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:39.604 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:39.605 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.605 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.606 DEBUG:boto:Method: GET 2022-09-03T06:45:39.607 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.608 DEBUG:boto:Data: 2022-09-03T06:45:39.609 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.610 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:39.611 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.612 DEBUG:boto:Params: {} 2022-09-03T06:45:39.613 DEBUG:boto:Token: None 2022-09-03T06:45:39.614 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:45:39.615 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TG4rtoNqEXjh8IakdVdKBavZteI= 2022-09-03T06:45:39.617 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TG4rtoNqEXjh8IakdVdKBavZteI=', 'Content-Length': '0'} 2022-09-03T06:45:39.619 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ef08ac06088c930e-006312f813-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.620 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:39.621 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:39.621 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:39.622 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:39.622 DEBUG:boto:Method: GET 2022-09-03T06:45:39.623 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:45:39.623 DEBUG:boto:Data: 2022-09-03T06:45:39.624 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.624 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.625 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.625 DEBUG:boto:Params: {} 2022-09-03T06:45:39.626 DEBUG:boto:Token: None 2022-09-03T06:45:39.627 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/ 2022-09-03T06:45:39.627 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM= 2022-09-03T06:45:39.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM=', 'Content-Length': '0'} 2022-09-03T06:45:39.632 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000018a057f82b5df23e-006312f813-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.632 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:39.634 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:39.634 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:39.635 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:39.636 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:39.636 DEBUG:boto:Method: GET 2022-09-03T06:45:39.637 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:45:39.637 DEBUG:boto:Data: 2022-09-03T06:45:39.638 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.639 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:39.639 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.640 DEBUG:boto:Params: {} 2022-09-03T06:45:39.641 DEBUG:boto:Token: None 2022-09-03T06:45:39.641 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/ 2022-09-03T06:45:39.642 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM= 2022-09-03T06:45:39.643 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM=', 'Content-Length': '0'} 2022-09-03T06:45:39.646 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c80dd047659628dc-006312f813-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.694 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:39.695 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:39.696 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:39.696 DEBUG:boto:Method: HEAD 2022-09-03T06:45:39.697 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:45:39.697 DEBUG:boto:Data: 2022-09-03T06:45:39.698 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.698 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.699 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.699 DEBUG:boto:Params: {} 2022-09-03T06:45:39.700 DEBUG:boto:Token: None 2022-09-03T06:45:39.701 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj115 2022-09-03T06:45:39.702 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IS0307vstIkKQ2/x0GA5nNwk8t4= 2022-09-03T06:45:39.702 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IS0307vstIkKQ2/x0GA5nNwk8t4=', 'Content-Length': '0'} 2022-09-03T06:45:39.705 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000025cd752c105a0cca-006312f813-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.705 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:39.706 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:39.707 DEBUG:boto:Method: HEAD 2022-09-03T06:45:39.707 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:45:39.708 DEBUG:boto:Data: 2022-09-03T06:45:39.708 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.709 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:39.709 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.710 DEBUG:boto:Params: {} 2022-09-03T06:45:39.711 DEBUG:boto:Token: None 2022-09-03T06:45:39.711 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj115 2022-09-03T06:45:39.712 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IS0307vstIkKQ2/x0GA5nNwk8t4= 2022-09-03T06:45:39.713 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IS0307vstIkKQ2/x0GA5nNwk8t4=', 'Content-Length': '0'} 2022-09-03T06:45:39.717 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000301d030040ecd44b-006312f813-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.719 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:45:39.720 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:39.721 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:39.722 DEBUG:boto:Method: GET 2022-09-03T06:45:39.723 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:45:39.724 DEBUG:boto:Data: 2022-09-03T06:45:39.725 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.726 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.727 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.728 DEBUG:boto:Params: {} 2022-09-03T06:45:39.729 DEBUG:boto:Token: None 2022-09-03T06:45:39.730 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj115 2022-09-03T06:45:39.731 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3eR/TxHcjdaIwaoZHWG8Wnl2NgU= 2022-09-03T06:45:39.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3eR/TxHcjdaIwaoZHWG8Wnl2NgU=', 'Content-Length': '0'} 2022-09-03T06:45:39.735 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a9ae8923dca3cba5-006312f813-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.737 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:45:39.738 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:45:39.739 DEBUG:boto:Method: GET 2022-09-03T06:45:39.739 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:45:39.740 DEBUG:boto:Data: 2022-09-03T06:45:39.741 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.741 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:39.742 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.743 DEBUG:boto:Params: {} 2022-09-03T06:45:39.744 DEBUG:boto:Token: None 2022-09-03T06:45:39.745 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj115 2022-09-03T06:45:39.746 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3eR/TxHcjdaIwaoZHWG8Wnl2NgU= 2022-09-03T06:45:39.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3eR/TxHcjdaIwaoZHWG8Wnl2NgU=', 'Content-Length': '0'} 2022-09-03T06:45:39.751 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074bd169b013b99ca-006312f813-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.752 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:39.753 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:39.753 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:39.754 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:39.754 DEBUG:boto:Method: GET 2022-09-03T06:45:39.755 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:45:39.756 DEBUG:boto:Data: 2022-09-03T06:45:39.757 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.757 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.758 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.759 DEBUG:boto:Params: {} 2022-09-03T06:45:39.760 DEBUG:boto:Token: None 2022-09-03T06:45:39.760 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/ 2022-09-03T06:45:39.761 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM= 2022-09-03T06:45:39.762 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM=', 'Content-Length': '0'} 2022-09-03T06:45:39.764 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006f1c0063f24d031e-006312f813-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.765 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:39.767 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:39.767 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:39.768 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:39.768 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:45:39.769 DEBUG:boto:Method: GET 2022-09-03T06:45:39.770 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:45:39.770 DEBUG:boto:Data: 2022-09-03T06:45:39.771 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.772 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:39.773 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.774 DEBUG:boto:Params: {} 2022-09-03T06:45:39.775 DEBUG:boto:Token: None 2022-09-03T06:45:39.776 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/ 2022-09-03T06:45:39.777 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM= 2022-09-03T06:45:39.778 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM=', 'Content-Length': '0'} 2022-09-03T06:45:39.788 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001a5cf298af02c7eb-006312f813-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.834 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:39.835 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:45:39.836 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:39.836 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:39.837 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.838 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.838 DEBUG:boto:Method: GET 2022-09-03T06:45:39.839 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.840 DEBUG:boto:Data: 2022-09-03T06:45:39.840 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.841 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.841 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.842 DEBUG:boto:Params: {} 2022-09-03T06:45:39.842 DEBUG:boto:Token: None 2022-09-03T06:45:39.843 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.844 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N4Bipq4lEecHMqT4C8AtWoFZ+xM= 2022-09-03T06:45:39.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N4Bipq4lEecHMqT4C8AtWoFZ+xM=', 'Content-Length': '0'} 2022-09-03T06:45:39.847 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065444cafb8fda958-006312f813-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.847 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:39.848 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:39.848 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.849 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.850 DEBUG:boto:Method: GET 2022-09-03T06:45:39.850 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.851 DEBUG:boto:Data: 2022-09-03T06:45:39.851 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.852 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:39.852 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.853 DEBUG:boto:Params: {} 2022-09-03T06:45:39.853 DEBUG:boto:Token: None 2022-09-03T06:45:39.854 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.855 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N4Bipq4lEecHMqT4C8AtWoFZ+xM= 2022-09-03T06:45:39.855 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N4Bipq4lEecHMqT4C8AtWoFZ+xM=', 'Content-Length': '0'} 2022-09-03T06:45:39.857 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000810f8ac7c2848417-006312f813-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.858 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:39.859 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:39.859 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.860 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.860 DEBUG:boto:Method: HEAD 2022-09-03T06:45:39.861 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.861 DEBUG:boto:Data: 2022-09-03T06:45:39.862 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.863 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.863 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.866 DEBUG:boto:Params: {} 2022-09-03T06:45:39.867 DEBUG:boto:Token: None 2022-09-03T06:45:39.868 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.868 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3g+qtTnoMNNkL1sYFLYsFLromdg= 2022-09-03T06:45:39.869 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3g+qtTnoMNNkL1sYFLYsFLromdg=', 'Content-Length': '0'} 2022-09-03T06:45:39.871 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007bc14874adf49801-006312f813-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.872 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:39.873 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:39.873 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.874 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.874 DEBUG:boto:Method: HEAD 2022-09-03T06:45:39.875 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.875 DEBUG:boto:Data: 2022-09-03T06:45:39.876 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.877 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:39.877 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.878 DEBUG:boto:Params: {} 2022-09-03T06:45:39.878 DEBUG:boto:Token: None 2022-09-03T06:45:39.879 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.880 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3g+qtTnoMNNkL1sYFLYsFLromdg= 2022-09-03T06:45:39.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3g+qtTnoMNNkL1sYFLYsFLromdg=', 'Content-Length': '0'} 2022-09-03T06:45:39.882 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000034a0ccdd7f520ec-006312f813-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.883 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:45:39.884 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:39.884 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:39.885 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.885 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.886 DEBUG:boto:Method: GET 2022-09-03T06:45:39.887 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.887 DEBUG:boto:Data: 2022-09-03T06:45:39.888 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.888 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.889 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.889 DEBUG:boto:Params: {} 2022-09-03T06:45:39.890 DEBUG:boto:Token: None 2022-09-03T06:45:39.891 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.891 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N4Bipq4lEecHMqT4C8AtWoFZ+xM= 2022-09-03T06:45:39.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N4Bipq4lEecHMqT4C8AtWoFZ+xM=', 'Content-Length': '0'} 2022-09-03T06:45:39.894 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000246ec643b77a82ad-006312f813-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.938 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:39.939 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:39.939 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.940 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.941 DEBUG:boto:Method: GET 2022-09-03T06:45:39.941 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.942 DEBUG:boto:Data: 2022-09-03T06:45:39.942 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.943 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:39.943 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.944 DEBUG:boto:Params: {} 2022-09-03T06:45:39.944 DEBUG:boto:Token: None 2022-09-03T06:45:39.945 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:45:39.946 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N4Bipq4lEecHMqT4C8AtWoFZ+xM= 2022-09-03T06:45:39.946 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N4Bipq4lEecHMqT4C8AtWoFZ+xM=', 'Content-Length': '0'} 2022-09-03T06:45:39.949 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000027135819b1ad2f3f-006312f813-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.949 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:39.950 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:39.951 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:39.951 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:39.952 DEBUG:boto:Method: GET 2022-09-03T06:45:39.952 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:45:39.953 DEBUG:boto:Data: 2022-09-03T06:45:39.953 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.954 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.955 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.956 DEBUG:boto:Params: {} 2022-09-03T06:45:39.956 DEBUG:boto:Token: None 2022-09-03T06:45:39.957 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/ 2022-09-03T06:45:39.957 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM= 2022-09-03T06:45:39.958 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM=', 'Content-Length': '0'} 2022-09-03T06:45:39.961 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000762f1864c31a7daa-006312f813-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.962 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:39.963 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:39.963 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:39.964 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:39.965 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:39.965 DEBUG:boto:Method: GET 2022-09-03T06:45:39.966 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:45:39.966 DEBUG:boto:Data: 2022-09-03T06:45:39.967 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.967 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:39.968 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.968 DEBUG:boto:Params: {} 2022-09-03T06:45:39.969 DEBUG:boto:Token: None 2022-09-03T06:45:39.970 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/ 2022-09-03T06:45:39.970 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM= 2022-09-03T06:45:39.971 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w2w9LYcGNpI8I7fd/ejDc8/4yGM=', 'Content-Length': '0'} 2022-09-03T06:45:39.974 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000049081f28444bcb11-006312f813-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.974 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:39.975 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:39.976 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:39.976 DEBUG:boto:Method: HEAD 2022-09-03T06:45:39.977 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:45:39.977 DEBUG:boto:Data: 2022-09-03T06:45:39.978 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.978 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.979 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.979 DEBUG:boto:Params: {} 2022-09-03T06:45:39.980 DEBUG:boto:Token: None 2022-09-03T06:45:39.981 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj116 2022-09-03T06:45:39.981 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R8B99TN7/xxobF1oCJ1nsEEhhmw= 2022-09-03T06:45:39.982 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R8B99TN7/xxobF1oCJ1nsEEhhmw=', 'Content-Length': '0'} 2022-09-03T06:45:39.984 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000edd07af6ed33b678-006312f813-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.985 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:39.985 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:39.986 DEBUG:boto:Method: HEAD 2022-09-03T06:45:39.986 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:45:39.987 DEBUG:boto:Data: 2022-09-03T06:45:39.987 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.988 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:39.988 DEBUG:boto:Port: 8000 2022-09-03T06:45:39.989 DEBUG:boto:Params: {} 2022-09-03T06:45:39.989 DEBUG:boto:Token: None 2022-09-03T06:45:39.990 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:39 GMT /dsgvcy-75/obj116 2022-09-03T06:45:39.991 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R8B99TN7/xxobF1oCJ1nsEEhhmw= 2022-09-03T06:45:39.991 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R8B99TN7/xxobF1oCJ1nsEEhhmw=', 'Content-Length': '0'} 2022-09-03T06:45:39.993 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c9b1be2641b1c85b-006312f813-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:39.994 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:45:39.995 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:39.995 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:39.996 DEBUG:boto:Method: GET 2022-09-03T06:45:39.996 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:45:39.997 DEBUG:boto:Data: 2022-09-03T06:45:39.997 DEBUG:boto:Headers: {} 2022-09-03T06:45:39.998 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:39.999 DEBUG:boto:Port: 8001 2022-09-03T06:45:39.999 DEBUG:boto:Params: {} 2022-09-03T06:45:40.000 DEBUG:boto:Token: None 2022-09-03T06:45:40.000 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj116 2022-09-03T06:45:40.001 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N5n//Gs2Kva/f2Qs2rCSoHv/GDA= 2022-09-03T06:45:40.002 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N5n//Gs2Kva/f2Qs2rCSoHv/GDA=', 'Content-Length': '0'} 2022-09-03T06:45:40.004 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f90c66ad562d004-006312f814-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.004 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:45:40.005 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:45:40.006 DEBUG:boto:Method: GET 2022-09-03T06:45:40.006 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:45:40.007 DEBUG:boto:Data: 2022-09-03T06:45:40.007 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.008 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.009 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.009 DEBUG:boto:Params: {} 2022-09-03T06:45:40.010 DEBUG:boto:Token: None 2022-09-03T06:45:40.010 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj116 2022-09-03T06:45:40.011 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N5n//Gs2Kva/f2Qs2rCSoHv/GDA= 2022-09-03T06:45:40.012 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N5n//Gs2Kva/f2Qs2rCSoHv/GDA=', 'Content-Length': '0'} 2022-09-03T06:45:40.014 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fd3d5b3f1971b4c0-006312f814-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.015 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:40.015 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:40.016 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:40.016 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:40.017 DEBUG:boto:Method: GET 2022-09-03T06:45:40.017 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:45:40.018 DEBUG:boto:Data: 2022-09-03T06:45:40.018 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.019 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.020 DEBUG:boto:Port: 8001 2022-09-03T06:45:40.020 DEBUG:boto:Params: {} 2022-09-03T06:45:40.021 DEBUG:boto:Token: None 2022-09-03T06:45:40.021 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/ 2022-09-03T06:45:40.022 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0= 2022-09-03T06:45:40.023 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0=', 'Content-Length': '0'} 2022-09-03T06:45:40.025 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001109623f39fcf655-006312f814-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.026 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:40.027 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:40.028 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:40.028 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:40.029 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:45:40.029 DEBUG:boto:Method: GET 2022-09-03T06:45:40.030 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:45:40.031 DEBUG:boto:Data: 2022-09-03T06:45:40.031 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.032 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.032 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.033 DEBUG:boto:Params: {} 2022-09-03T06:45:40.033 DEBUG:boto:Token: None 2022-09-03T06:45:40.034 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/ 2022-09-03T06:45:40.035 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0= 2022-09-03T06:45:40.035 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0=', 'Content-Length': '0'} 2022-09-03T06:45:40.038 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009987599623fa5207-006312f814-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.039 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:40.040 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:45:40.040 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:40.041 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:40.041 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.042 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.043 DEBUG:boto:Method: GET 2022-09-03T06:45:40.043 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.044 DEBUG:boto:Data: 2022-09-03T06:45:40.044 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.045 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.045 DEBUG:boto:Port: 8001 2022-09-03T06:45:40.046 DEBUG:boto:Params: {} 2022-09-03T06:45:40.047 DEBUG:boto:Token: None 2022-09-03T06:45:40.047 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.048 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lPVN2MyJA02pyeuFnG6x9DUDsoU= 2022-09-03T06:45:40.049 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lPVN2MyJA02pyeuFnG6x9DUDsoU=', 'Content-Length': '0'} 2022-09-03T06:45:40.051 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000edda8d5997856e7b-006312f814-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.052 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:40.052 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:40.053 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.053 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.054 DEBUG:boto:Method: GET 2022-09-03T06:45:40.055 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.055 DEBUG:boto:Data: 2022-09-03T06:45:40.056 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.056 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.057 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.057 DEBUG:boto:Params: {} 2022-09-03T06:45:40.058 DEBUG:boto:Token: None 2022-09-03T06:45:40.059 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.059 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lPVN2MyJA02pyeuFnG6x9DUDsoU= 2022-09-03T06:45:40.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lPVN2MyJA02pyeuFnG6x9DUDsoU=', 'Content-Length': '0'} 2022-09-03T06:45:40.062 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000033d872c83f4e7fef-006312f814-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.106 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:40.107 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:40.109 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.110 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.111 DEBUG:boto:Method: HEAD 2022-09-03T06:45:40.112 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.113 DEBUG:boto:Data: 2022-09-03T06:45:40.114 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.116 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.117 DEBUG:boto:Port: 8001 2022-09-03T06:45:40.117 DEBUG:boto:Params: {} 2022-09-03T06:45:40.119 DEBUG:boto:Token: None 2022-09-03T06:45:40.119 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.120 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sSysw+PQbTScJNiGQ61XDRQXtA8= 2022-09-03T06:45:40.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sSysw+PQbTScJNiGQ61XDRQXtA8=', 'Content-Length': '0'} 2022-09-03T06:45:40.123 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004f8991ef637315e9-006312f814-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.124 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:40.125 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:40.125 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.126 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.127 DEBUG:boto:Method: HEAD 2022-09-03T06:45:40.127 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.128 DEBUG:boto:Data: 2022-09-03T06:45:40.128 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.129 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.129 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.130 DEBUG:boto:Params: {} 2022-09-03T06:45:40.130 DEBUG:boto:Token: None 2022-09-03T06:45:40.131 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.131 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sSysw+PQbTScJNiGQ61XDRQXtA8= 2022-09-03T06:45:40.132 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sSysw+PQbTScJNiGQ61XDRQXtA8=', 'Content-Length': '0'} 2022-09-03T06:45:40.134 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a4055b54274b17b6-006312f814-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.135 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:45:40.136 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:40.137 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:40.137 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.138 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.139 DEBUG:boto:Method: GET 2022-09-03T06:45:40.141 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.142 DEBUG:boto:Data: 2022-09-03T06:45:40.143 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.144 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.144 DEBUG:boto:Port: 8001 2022-09-03T06:45:40.145 DEBUG:boto:Params: {} 2022-09-03T06:45:40.146 DEBUG:boto:Token: None 2022-09-03T06:45:40.147 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.147 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lPVN2MyJA02pyeuFnG6x9DUDsoU= 2022-09-03T06:45:40.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lPVN2MyJA02pyeuFnG6x9DUDsoU=', 'Content-Length': '0'} 2022-09-03T06:45:40.150 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f10d0c10daf5f94e-006312f814-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.152 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:40.153 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:40.154 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.155 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.156 DEBUG:boto:Method: GET 2022-09-03T06:45:40.156 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.157 DEBUG:boto:Data: 2022-09-03T06:45:40.158 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.158 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.159 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.159 DEBUG:boto:Params: {} 2022-09-03T06:45:40.161 DEBUG:boto:Token: None 2022-09-03T06:45:40.162 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:45:40.163 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lPVN2MyJA02pyeuFnG6x9DUDsoU= 2022-09-03T06:45:40.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lPVN2MyJA02pyeuFnG6x9DUDsoU=', 'Content-Length': '0'} 2022-09-03T06:45:40.168 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f304d82bba1d91a-006312f814-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.170 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:40.172 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:40.174 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:40.175 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:40.176 DEBUG:boto:Method: GET 2022-09-03T06:45:40.177 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:45:40.180 DEBUG:boto:Data: 2022-09-03T06:45:40.181 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.181 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.182 DEBUG:boto:Port: 8001 2022-09-03T06:45:40.183 DEBUG:boto:Params: {} 2022-09-03T06:45:40.183 DEBUG:boto:Token: None 2022-09-03T06:45:40.184 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/ 2022-09-03T06:45:40.184 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0= 2022-09-03T06:45:40.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0=', 'Content-Length': '0'} 2022-09-03T06:45:40.188 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000041ac2864147022dd-006312f814-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.189 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:40.194 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:40.195 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:40.196 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:40.197 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:40.207 DEBUG:boto:Method: GET 2022-09-03T06:45:40.209 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:45:40.209 DEBUG:boto:Data: 2022-09-03T06:45:40.210 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.210 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.211 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.212 DEBUG:boto:Params: {} 2022-09-03T06:45:40.212 DEBUG:boto:Token: None 2022-09-03T06:45:40.213 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/ 2022-09-03T06:45:40.213 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0= 2022-09-03T06:45:40.214 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0=', 'Content-Length': '0'} 2022-09-03T06:45:40.217 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000007d055d4de24d696-006312f814-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.218 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:40.219 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:40.219 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:40.220 DEBUG:boto:Method: HEAD 2022-09-03T06:45:40.220 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:45:40.221 DEBUG:boto:Data: 2022-09-03T06:45:40.222 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.222 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.223 DEBUG:boto:Port: 8001 2022-09-03T06:45:40.223 DEBUG:boto:Params: {} 2022-09-03T06:45:40.224 DEBUG:boto:Token: None 2022-09-03T06:45:40.225 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj145 2022-09-03T06:45:40.225 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Umumga2al7PWEgBGx3Khn3nqAaE= 2022-09-03T06:45:40.226 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Umumga2al7PWEgBGx3Khn3nqAaE=', 'Content-Length': '0'} 2022-09-03T06:45:40.228 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000047c432e224c1ab89-006312f814-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.229 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:40.230 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:40.230 DEBUG:boto:Method: HEAD 2022-09-03T06:45:40.231 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:45:40.231 DEBUG:boto:Data: 2022-09-03T06:45:40.232 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.232 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.233 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.233 DEBUG:boto:Params: {} 2022-09-03T06:45:40.234 DEBUG:boto:Token: None 2022-09-03T06:45:40.235 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj145 2022-09-03T06:45:40.235 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Umumga2al7PWEgBGx3Khn3nqAaE= 2022-09-03T06:45:40.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Umumga2al7PWEgBGx3Khn3nqAaE=', 'Content-Length': '0'} 2022-09-03T06:45:40.238 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c8da9c13ed1755f-006312f814-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.239 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:45:40.239 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:40.240 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:40.240 DEBUG:boto:Method: GET 2022-09-03T06:45:40.241 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:45:40.242 DEBUG:boto:Data: 2022-09-03T06:45:40.243 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.244 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.244 DEBUG:boto:Port: 8001 2022-09-03T06:45:40.245 DEBUG:boto:Params: {} 2022-09-03T06:45:40.245 DEBUG:boto:Token: None 2022-09-03T06:45:40.246 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj145 2022-09-03T06:45:40.247 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Qcy/xqBH4CDEUHgy0EHZvn3dzGk= 2022-09-03T06:45:40.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Qcy/xqBH4CDEUHgy0EHZvn3dzGk=', 'Content-Length': '0'} 2022-09-03T06:45:40.252 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000042398a4e96692f1-006312f814-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.253 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:45:40.254 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:45:40.254 DEBUG:boto:Method: GET 2022-09-03T06:45:40.255 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:45:40.256 DEBUG:boto:Data: 2022-09-03T06:45:40.256 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.257 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.257 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.258 DEBUG:boto:Params: {} 2022-09-03T06:45:40.259 DEBUG:boto:Token: None 2022-09-03T06:45:40.259 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj145 2022-09-03T06:45:40.260 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Qcy/xqBH4CDEUHgy0EHZvn3dzGk= 2022-09-03T06:45:40.261 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Qcy/xqBH4CDEUHgy0EHZvn3dzGk=', 'Content-Length': '0'} 2022-09-03T06:45:40.263 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000885a264c8baa8f04-006312f814-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.264 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:40.265 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:40.265 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:40.266 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:40.266 DEBUG:boto:Method: GET 2022-09-03T06:45:40.267 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:45:40.267 DEBUG:boto:Data: 2022-09-03T06:45:40.268 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.269 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.269 DEBUG:boto:Port: 8001 2022-09-03T06:45:40.270 DEBUG:boto:Params: {} 2022-09-03T06:45:40.271 DEBUG:boto:Token: None 2022-09-03T06:45:40.272 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/ 2022-09-03T06:45:40.273 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0= 2022-09-03T06:45:40.273 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0=', 'Content-Length': '0'} 2022-09-03T06:45:40.277 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e22b43e58443caf7-006312f814-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.318 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:40.320 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:40.322 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:40.323 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:40.324 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:45:40.325 DEBUG:boto:Method: GET 2022-09-03T06:45:40.327 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:45:40.328 DEBUG:boto:Data: 2022-09-03T06:45:40.329 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.332 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.333 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.333 DEBUG:boto:Params: {} 2022-09-03T06:45:40.334 DEBUG:boto:Token: None 2022-09-03T06:45:40.335 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/ 2022-09-03T06:45:40.336 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0= 2022-09-03T06:45:40.337 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0=', 'Content-Length': '0'} 2022-09-03T06:45:40.345 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000c3c7c0f7cf8192e-006312f814-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.348 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:40.351 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:45:40.352 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:40.353 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:40.353 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.354 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.354 DEBUG:boto:Method: GET 2022-09-03T06:45:40.355 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.356 DEBUG:boto:Data: 2022-09-03T06:45:40.356 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.357 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.357 DEBUG:boto:Port: 8001 2022-09-03T06:45:40.358 DEBUG:boto:Params: {} 2022-09-03T06:45:40.359 DEBUG:boto:Token: None 2022-09-03T06:45:40.359 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.360 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FT4/96Afcs8iuzrdhv0lVGBKMuU= 2022-09-03T06:45:40.361 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FT4/96Afcs8iuzrdhv0lVGBKMuU=', 'Content-Length': '0'} 2022-09-03T06:45:40.363 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003607c29516d138c9-006312f814-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.364 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:40.365 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:40.366 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.367 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.367 DEBUG:boto:Method: GET 2022-09-03T06:45:40.368 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.369 DEBUG:boto:Data: 2022-09-03T06:45:40.370 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.370 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.371 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.371 DEBUG:boto:Params: {} 2022-09-03T06:45:40.372 DEBUG:boto:Token: None 2022-09-03T06:45:40.373 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.374 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FT4/96Afcs8iuzrdhv0lVGBKMuU= 2022-09-03T06:45:40.374 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FT4/96Afcs8iuzrdhv0lVGBKMuU=', 'Content-Length': '0'} 2022-09-03T06:45:40.377 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000486b917efa8d5c97-006312f814-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.418 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:40.419 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:40.421 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.421 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.422 DEBUG:boto:Method: HEAD 2022-09-03T06:45:40.423 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.423 DEBUG:boto:Data: 2022-09-03T06:45:40.424 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.424 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.425 DEBUG:boto:Port: 8001 2022-09-03T06:45:40.426 DEBUG:boto:Params: {} 2022-09-03T06:45:40.426 DEBUG:boto:Token: None 2022-09-03T06:45:40.427 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.427 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0snJz+BhTn31xYPkJWb6L9k9w8g= 2022-09-03T06:45:40.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0snJz+BhTn31xYPkJWb6L9k9w8g=', 'Content-Length': '0'} 2022-09-03T06:45:40.430 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a4e219e85c9feb60-006312f814-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.431 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:40.431 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:40.432 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.432 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.433 DEBUG:boto:Method: HEAD 2022-09-03T06:45:40.433 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.434 DEBUG:boto:Data: 2022-09-03T06:45:40.435 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.435 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.436 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.436 DEBUG:boto:Params: {} 2022-09-03T06:45:40.437 DEBUG:boto:Token: None 2022-09-03T06:45:40.438 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.438 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0snJz+BhTn31xYPkJWb6L9k9w8g= 2022-09-03T06:45:40.439 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0snJz+BhTn31xYPkJWb6L9k9w8g=', 'Content-Length': '0'} 2022-09-03T06:45:40.441 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b8d071fd96987689-006312f814-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.442 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:45:40.443 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:40.443 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:40.444 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.445 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.445 DEBUG:boto:Method: GET 2022-09-03T06:45:40.446 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.447 DEBUG:boto:Data: 2022-09-03T06:45:40.448 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.448 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.449 DEBUG:boto:Port: 8001 2022-09-03T06:45:40.450 DEBUG:boto:Params: {} 2022-09-03T06:45:40.450 DEBUG:boto:Token: None 2022-09-03T06:45:40.451 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.452 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FT4/96Afcs8iuzrdhv0lVGBKMuU= 2022-09-03T06:45:40.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FT4/96Afcs8iuzrdhv0lVGBKMuU=', 'Content-Length': '0'} 2022-09-03T06:45:40.454 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aa69b8ed5c33b924-006312f814-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.455 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:40.456 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:40.457 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.458 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.460 DEBUG:boto:Method: GET 2022-09-03T06:45:40.461 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.462 DEBUG:boto:Data: 2022-09-03T06:45:40.462 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.463 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.463 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.464 DEBUG:boto:Params: {} 2022-09-03T06:45:40.465 DEBUG:boto:Token: None 2022-09-03T06:45:40.465 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:45:40.466 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FT4/96Afcs8iuzrdhv0lVGBKMuU= 2022-09-03T06:45:40.467 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FT4/96Afcs8iuzrdhv0lVGBKMuU=', 'Content-Length': '0'} 2022-09-03T06:45:40.469 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000054a377179d32d0b4-006312f814-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.470 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:40.471 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:40.472 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:40.472 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:40.473 DEBUG:boto:Method: GET 2022-09-03T06:45:40.473 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:45:40.474 DEBUG:boto:Data: 2022-09-03T06:45:40.475 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.475 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.476 DEBUG:boto:Port: 8001 2022-09-03T06:45:40.477 DEBUG:boto:Params: {} 2022-09-03T06:45:40.477 DEBUG:boto:Token: None 2022-09-03T06:45:40.478 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/ 2022-09-03T06:45:40.479 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0= 2022-09-03T06:45:40.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0=', 'Content-Length': '0'} 2022-09-03T06:45:40.483 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000b3c2b9809b3af1c-006312f814-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.484 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:40.485 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:40.487 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:40.488 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:40.488 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:40.489 DEBUG:boto:Method: GET 2022-09-03T06:45:40.490 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:45:40.491 DEBUG:boto:Data: 2022-09-03T06:45:40.492 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.493 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.494 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.495 DEBUG:boto:Params: {} 2022-09-03T06:45:40.496 DEBUG:boto:Token: None 2022-09-03T06:45:40.497 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/ 2022-09-03T06:45:40.499 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0= 2022-09-03T06:45:40.500 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0=', 'Content-Length': '0'} 2022-09-03T06:45:40.503 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002cc21988c32cec29-006312f814-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.505 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:40.507 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:40.508 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:40.509 DEBUG:boto:Method: HEAD 2022-09-03T06:45:40.510 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:45:40.510 DEBUG:boto:Data: 2022-09-03T06:45:40.511 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.513 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.513 DEBUG:boto:Port: 8001 2022-09-03T06:45:40.514 DEBUG:boto:Params: {} 2022-09-03T06:45:40.515 DEBUG:boto:Token: None 2022-09-03T06:45:40.515 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj17 2022-09-03T06:45:40.516 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pNwFxHd0TPutVl4dax4tdADsoZw= 2022-09-03T06:45:40.517 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pNwFxHd0TPutVl4dax4tdADsoZw=', 'Content-Length': '0'} 2022-09-03T06:45:40.519 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000082b61cddf96c43ca-006312f814-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.521 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:40.522 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:40.522 DEBUG:boto:Method: HEAD 2022-09-03T06:45:40.523 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:45:40.524 DEBUG:boto:Data: 2022-09-03T06:45:40.525 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.525 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.526 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.527 DEBUG:boto:Params: {} 2022-09-03T06:45:40.528 DEBUG:boto:Token: None 2022-09-03T06:45:40.529 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj17 2022-09-03T06:45:40.529 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pNwFxHd0TPutVl4dax4tdADsoZw= 2022-09-03T06:45:40.530 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pNwFxHd0TPutVl4dax4tdADsoZw=', 'Content-Length': '0'} 2022-09-03T06:45:40.544 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008759b47e874d3e71-006312f814-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.545 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:45:40.546 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:40.547 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:40.548 DEBUG:boto:Method: GET 2022-09-03T06:45:40.550 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:45:40.551 DEBUG:boto:Data: 2022-09-03T06:45:40.552 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.553 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.555 DEBUG:boto:Port: 8001 2022-09-03T06:45:40.556 DEBUG:boto:Params: {} 2022-09-03T06:45:40.557 DEBUG:boto:Token: None 2022-09-03T06:45:40.558 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj17 2022-09-03T06:45:40.559 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v6b/UvW74YE7FFPv8nt3D0cvT7U= 2022-09-03T06:45:40.561 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v6b/UvW74YE7FFPv8nt3D0cvT7U=', 'Content-Length': '0'} 2022-09-03T06:45:40.564 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2da5c28f04b6a17-006312f814-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.565 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:45:40.566 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:45:40.567 DEBUG:boto:Method: GET 2022-09-03T06:45:40.568 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:45:40.570 DEBUG:boto:Data: 2022-09-03T06:45:40.571 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.572 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.573 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.575 DEBUG:boto:Params: {} 2022-09-03T06:45:40.576 DEBUG:boto:Token: None 2022-09-03T06:45:40.577 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj17 2022-09-03T06:45:40.578 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v6b/UvW74YE7FFPv8nt3D0cvT7U= 2022-09-03T06:45:40.580 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v6b/UvW74YE7FFPv8nt3D0cvT7U=', 'Content-Length': '0'} 2022-09-03T06:45:40.583 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000725702eb13a6d674-006312f814-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.586 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:40.587 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:40.588 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:40.589 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:40.591 DEBUG:boto:Method: GET 2022-09-03T06:45:40.598 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:45:40.599 DEBUG:boto:Data: 2022-09-03T06:45:40.599 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.600 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.601 DEBUG:boto:Port: 8001 2022-09-03T06:45:40.601 DEBUG:boto:Params: {} 2022-09-03T06:45:40.602 DEBUG:boto:Token: None 2022-09-03T06:45:40.603 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/ 2022-09-03T06:45:40.603 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0= 2022-09-03T06:45:40.604 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0=', 'Content-Length': '0'} 2022-09-03T06:45:40.607 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009338761df2c5e139-006312f814-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.608 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:40.610 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:40.610 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:40.611 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:40.612 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:45:40.612 DEBUG:boto:Method: GET 2022-09-03T06:45:40.613 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:45:40.613 DEBUG:boto:Data: 2022-09-03T06:45:40.614 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.615 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.615 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.616 DEBUG:boto:Params: {} 2022-09-03T06:45:40.618 DEBUG:boto:Token: None 2022-09-03T06:45:40.619 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/ 2022-09-03T06:45:40.620 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0= 2022-09-03T06:45:40.622 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0=', 'Content-Length': '0'} 2022-09-03T06:45:40.628 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007dd4a8d3b1df1298-006312f814-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.674 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:40.676 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:45:40.678 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:40.679 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:40.680 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.682 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.683 DEBUG:boto:Method: GET 2022-09-03T06:45:40.684 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.686 DEBUG:boto:Data: 2022-09-03T06:45:40.687 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.688 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.690 DEBUG:boto:Port: 8001 2022-09-03T06:45:40.692 DEBUG:boto:Params: {} 2022-09-03T06:45:40.694 DEBUG:boto:Token: None 2022-09-03T06:45:40.695 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.696 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cCFeRI3/RXBGof7XCzGhbZNEVsg= 2022-09-03T06:45:40.697 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cCFeRI3/RXBGof7XCzGhbZNEVsg=', 'Content-Length': '0'} 2022-09-03T06:45:40.701 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000068125d7d7b27f1c5-006312f814-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.703 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:40.704 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:40.704 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.705 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.705 DEBUG:boto:Method: GET 2022-09-03T06:45:40.706 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.707 DEBUG:boto:Data: 2022-09-03T06:45:40.707 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.708 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.708 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.709 DEBUG:boto:Params: {} 2022-09-03T06:45:40.709 DEBUG:boto:Token: None 2022-09-03T06:45:40.710 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.711 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cCFeRI3/RXBGof7XCzGhbZNEVsg= 2022-09-03T06:45:40.711 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cCFeRI3/RXBGof7XCzGhbZNEVsg=', 'Content-Length': '0'} 2022-09-03T06:45:40.714 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000035a4a198c048bb8c-006312f814-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.715 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:40.715 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:40.716 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.717 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.717 DEBUG:boto:Method: HEAD 2022-09-03T06:45:40.718 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.718 DEBUG:boto:Data: 2022-09-03T06:45:40.719 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.719 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.720 DEBUG:boto:Port: 8001 2022-09-03T06:45:40.720 DEBUG:boto:Params: {} 2022-09-03T06:45:40.721 DEBUG:boto:Token: None 2022-09-03T06:45:40.721 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.722 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vqfYq1OQP3NgD8zdkuCApJHL2rk= 2022-09-03T06:45:40.723 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vqfYq1OQP3NgD8zdkuCApJHL2rk=', 'Content-Length': '0'} 2022-09-03T06:45:40.726 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008062dba632fbbf37-006312f814-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.727 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:40.728 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:40.729 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.729 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.730 DEBUG:boto:Method: HEAD 2022-09-03T06:45:40.730 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.731 DEBUG:boto:Data: 2022-09-03T06:45:40.731 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.732 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.732 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.733 DEBUG:boto:Params: {} 2022-09-03T06:45:40.734 DEBUG:boto:Token: None 2022-09-03T06:45:40.734 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.735 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vqfYq1OQP3NgD8zdkuCApJHL2rk= 2022-09-03T06:45:40.736 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vqfYq1OQP3NgD8zdkuCApJHL2rk=', 'Content-Length': '0'} 2022-09-03T06:45:40.738 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f56d851a7cc8d70a-006312f814-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.739 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:45:40.740 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:40.740 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:40.741 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.741 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.742 DEBUG:boto:Method: GET 2022-09-03T06:45:40.743 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.743 DEBUG:boto:Data: 2022-09-03T06:45:40.744 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.744 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.745 DEBUG:boto:Port: 8001 2022-09-03T06:45:40.745 DEBUG:boto:Params: {} 2022-09-03T06:45:40.746 DEBUG:boto:Token: None 2022-09-03T06:45:40.747 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.748 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cCFeRI3/RXBGof7XCzGhbZNEVsg= 2022-09-03T06:45:40.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cCFeRI3/RXBGof7XCzGhbZNEVsg=', 'Content-Length': '0'} 2022-09-03T06:45:40.751 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000056e2d3d72ffd37f-006312f814-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.752 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:40.752 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:40.753 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.753 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.754 DEBUG:boto:Method: GET 2022-09-03T06:45:40.754 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.755 DEBUG:boto:Data: 2022-09-03T06:45:40.756 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.756 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.757 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.757 DEBUG:boto:Params: {} 2022-09-03T06:45:40.758 DEBUG:boto:Token: None 2022-09-03T06:45:40.758 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:45:40.759 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cCFeRI3/RXBGof7XCzGhbZNEVsg= 2022-09-03T06:45:40.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cCFeRI3/RXBGof7XCzGhbZNEVsg=', 'Content-Length': '0'} 2022-09-03T06:45:40.762 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c7d1dabc259f697-006312f814-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.806 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:40.807 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:40.808 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:40.809 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:40.811 DEBUG:boto:Method: GET 2022-09-03T06:45:40.812 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:45:40.813 DEBUG:boto:Data: 2022-09-03T06:45:40.814 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.815 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.816 DEBUG:boto:Port: 8001 2022-09-03T06:45:40.817 DEBUG:boto:Params: {} 2022-09-03T06:45:40.819 DEBUG:boto:Token: None 2022-09-03T06:45:40.820 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/ 2022-09-03T06:45:40.821 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0= 2022-09-03T06:45:40.821 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0=', 'Content-Length': '0'} 2022-09-03T06:45:40.825 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b620dd2f76b2063c-006312f814-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.826 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:40.827 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:40.827 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:40.828 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:40.829 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:40.829 DEBUG:boto:Method: GET 2022-09-03T06:45:40.831 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:45:40.831 DEBUG:boto:Data: 2022-09-03T06:45:40.832 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.833 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.833 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.834 DEBUG:boto:Params: {} 2022-09-03T06:45:40.834 DEBUG:boto:Token: None 2022-09-03T06:45:40.835 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/ 2022-09-03T06:45:40.836 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0= 2022-09-03T06:45:40.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0=', 'Content-Length': '0'} 2022-09-03T06:45:40.839 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001c8243d7e65f8354-006312f814-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.840 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:40.840 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:40.841 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:40.842 DEBUG:boto:Method: HEAD 2022-09-03T06:45:40.842 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:45:40.843 DEBUG:boto:Data: 2022-09-03T06:45:40.844 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.844 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.845 DEBUG:boto:Port: 8001 2022-09-03T06:45:40.845 DEBUG:boto:Params: {} 2022-09-03T06:45:40.846 DEBUG:boto:Token: None 2022-09-03T06:45:40.847 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj19 2022-09-03T06:45:40.847 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PMfjg6hE3+MK0mSq3hF8n+wiUDI= 2022-09-03T06:45:40.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PMfjg6hE3+MK0mSq3hF8n+wiUDI=', 'Content-Length': '0'} 2022-09-03T06:45:40.850 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a45bc8897a290a06-006312f814-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.851 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:40.852 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:40.852 DEBUG:boto:Method: HEAD 2022-09-03T06:45:40.853 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:45:40.853 DEBUG:boto:Data: 2022-09-03T06:45:40.854 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.855 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.856 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.856 DEBUG:boto:Params: {} 2022-09-03T06:45:40.857 DEBUG:boto:Token: None 2022-09-03T06:45:40.857 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj19 2022-09-03T06:45:40.858 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PMfjg6hE3+MK0mSq3hF8n+wiUDI= 2022-09-03T06:45:40.859 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PMfjg6hE3+MK0mSq3hF8n+wiUDI=', 'Content-Length': '0'} 2022-09-03T06:45:40.861 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000973d76ccab91a4ef-006312f814-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.862 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:45:40.862 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:40.863 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:40.863 DEBUG:boto:Method: GET 2022-09-03T06:45:40.864 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:45:40.864 DEBUG:boto:Data: 2022-09-03T06:45:40.865 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.866 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.866 DEBUG:boto:Port: 8001 2022-09-03T06:45:40.867 DEBUG:boto:Params: {} 2022-09-03T06:45:40.868 DEBUG:boto:Token: None 2022-09-03T06:45:40.869 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj19 2022-09-03T06:45:40.869 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eU/T20Vo/cYJGqrUGXrR8lIoEtc= 2022-09-03T06:45:40.870 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eU/T20Vo/cYJGqrUGXrR8lIoEtc=', 'Content-Length': '0'} 2022-09-03T06:45:40.872 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000068e75c763732cff4-006312f814-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.873 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:45:40.873 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:45:40.874 DEBUG:boto:Method: GET 2022-09-03T06:45:40.874 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:45:40.875 DEBUG:boto:Data: 2022-09-03T06:45:40.876 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.877 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.877 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.878 DEBUG:boto:Params: {} 2022-09-03T06:45:40.879 DEBUG:boto:Token: None 2022-09-03T06:45:40.879 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj19 2022-09-03T06:45:40.880 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eU/T20Vo/cYJGqrUGXrR8lIoEtc= 2022-09-03T06:45:40.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eU/T20Vo/cYJGqrUGXrR8lIoEtc=', 'Content-Length': '0'} 2022-09-03T06:45:40.882 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000feb67ac7019729ed-006312f814-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.883 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:40.884 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:40.884 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:40.885 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:40.885 DEBUG:boto:Method: GET 2022-09-03T06:45:40.886 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:45:40.887 DEBUG:boto:Data: 2022-09-03T06:45:40.887 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.888 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.889 DEBUG:boto:Port: 8001 2022-09-03T06:45:40.889 DEBUG:boto:Params: {} 2022-09-03T06:45:40.890 DEBUG:boto:Token: None 2022-09-03T06:45:40.890 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/ 2022-09-03T06:45:40.891 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0= 2022-09-03T06:45:40.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0=', 'Content-Length': '0'} 2022-09-03T06:45:40.894 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c804440a0010268b-006312f814-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.896 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:40.897 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:40.897 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:40.898 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:40.898 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:45:40.899 DEBUG:boto:Method: GET 2022-09-03T06:45:40.900 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:45:40.900 DEBUG:boto:Data: 2022-09-03T06:45:40.901 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.901 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.902 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.902 DEBUG:boto:Params: {} 2022-09-03T06:45:40.903 DEBUG:boto:Token: None 2022-09-03T06:45:40.904 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/ 2022-09-03T06:45:40.905 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0= 2022-09-03T06:45:40.906 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f5/Cd7iMuW0Qwclv67WOAPMgaM0=', 'Content-Length': '0'} 2022-09-03T06:45:40.909 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005a27aa085b7ae307-006312f814-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.954 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:40.956 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:45:40.957 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:40.959 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:40.960 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:40.960 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:40.961 DEBUG:boto:Method: GET 2022-09-03T06:45:40.962 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:40.963 DEBUG:boto:Data: 2022-09-03T06:45:40.964 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.965 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.966 DEBUG:boto:Port: 8001 2022-09-03T06:45:40.967 DEBUG:boto:Params: {} 2022-09-03T06:45:40.968 DEBUG:boto:Token: None 2022-09-03T06:45:40.969 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:40.970 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k+qG/pjRryf71ZAfgZ6DCvBhK8M= 2022-09-03T06:45:40.971 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k+qG/pjRryf71ZAfgZ6DCvBhK8M=', 'Content-Length': '0'} 2022-09-03T06:45:40.974 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e557cd4c397d7ce-006312f814-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.975 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:40.976 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:40.977 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:40.978 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:40.978 DEBUG:boto:Method: GET 2022-09-03T06:45:40.979 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:40.980 DEBUG:boto:Data: 2022-09-03T06:45:40.980 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.981 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:40.982 DEBUG:boto:Port: 8000 2022-09-03T06:45:40.983 DEBUG:boto:Params: {} 2022-09-03T06:45:40.984 DEBUG:boto:Token: None 2022-09-03T06:45:40.984 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:40 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:40.986 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k+qG/pjRryf71ZAfgZ6DCvBhK8M= 2022-09-03T06:45:40.986 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k+qG/pjRryf71ZAfgZ6DCvBhK8M=', 'Content-Length': '0'} 2022-09-03T06:45:40.989 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000511c2fe85eab2c80-006312f814-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:40.990 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:40.991 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:40.992 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:40.993 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:40.994 DEBUG:boto:Method: HEAD 2022-09-03T06:45:40.995 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:40.996 DEBUG:boto:Data: 2022-09-03T06:45:40.997 DEBUG:boto:Headers: {} 2022-09-03T06:45:40.997 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:40.998 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.000 DEBUG:boto:Params: {} 2022-09-03T06:45:41.001 DEBUG:boto:Token: None 2022-09-03T06:45:41.001 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:41.002 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:01CZ8/pJci0CBdpiKzezdnSfgLQ= 2022-09-03T06:45:41.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:01CZ8/pJci0CBdpiKzezdnSfgLQ=', 'Content-Length': '0'} 2022-09-03T06:45:41.005 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007698a954e6c93376-006312f815-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.006 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:41.006 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:41.007 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:41.008 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:41.009 DEBUG:boto:Method: HEAD 2022-09-03T06:45:41.009 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:41.010 DEBUG:boto:Data: 2022-09-03T06:45:41.011 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.011 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.012 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.012 DEBUG:boto:Params: {} 2022-09-03T06:45:41.013 DEBUG:boto:Token: None 2022-09-03T06:45:41.014 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:41.014 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:01CZ8/pJci0CBdpiKzezdnSfgLQ= 2022-09-03T06:45:41.015 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:01CZ8/pJci0CBdpiKzezdnSfgLQ=', 'Content-Length': '0'} 2022-09-03T06:45:41.017 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000016396f85d0acec2-006312f815-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.018 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:45:41.018 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:41.019 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:41.019 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:41.020 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:41.020 DEBUG:boto:Method: GET 2022-09-03T06:45:41.021 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:41.021 DEBUG:boto:Data: 2022-09-03T06:45:41.022 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.022 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.023 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.023 DEBUG:boto:Params: {} 2022-09-03T06:45:41.024 DEBUG:boto:Token: None 2022-09-03T06:45:41.025 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:41.025 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:48i9YrzvVDJjv8z6F0PsJu4bkmI= 2022-09-03T06:45:41.026 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:48i9YrzvVDJjv8z6F0PsJu4bkmI=', 'Content-Length': '0'} 2022-09-03T06:45:41.028 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005bed003859a05eb8-006312f815-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.029 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:41.030 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:41.030 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:41.031 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:41.031 DEBUG:boto:Method: GET 2022-09-03T06:45:41.032 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:41.032 DEBUG:boto:Data: 2022-09-03T06:45:41.033 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.034 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.034 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.035 DEBUG:boto:Params: {} 2022-09-03T06:45:41.035 DEBUG:boto:Token: None 2022-09-03T06:45:41.036 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:45:41.037 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:48i9YrzvVDJjv8z6F0PsJu4bkmI= 2022-09-03T06:45:41.037 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:48i9YrzvVDJjv8z6F0PsJu4bkmI=', 'Content-Length': '0'} 2022-09-03T06:45:41.039 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000991926f7a0389445-006312f815-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.041 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:41.041 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:41.042 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:41.042 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:41.043 DEBUG:boto:Method: GET 2022-09-03T06:45:41.044 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:45:41.044 DEBUG:boto:Data: 2022-09-03T06:45:41.045 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.045 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.046 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.046 DEBUG:boto:Params: {} 2022-09-03T06:45:41.047 DEBUG:boto:Token: None 2022-09-03T06:45:41.048 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/ 2022-09-03T06:45:41.048 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE= 2022-09-03T06:45:41.049 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE=', 'Content-Length': '0'} 2022-09-03T06:45:41.052 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000096e5a683229a562-006312f815-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.052 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:41.053 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:41.054 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:41.054 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:41.055 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:41.056 DEBUG:boto:Method: GET 2022-09-03T06:45:41.056 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:45:41.057 DEBUG:boto:Data: 2022-09-03T06:45:41.057 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.058 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.058 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.059 DEBUG:boto:Params: {} 2022-09-03T06:45:41.059 DEBUG:boto:Token: None 2022-09-03T06:45:41.060 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/ 2022-09-03T06:45:41.061 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE= 2022-09-03T06:45:41.062 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE=', 'Content-Length': '0'} 2022-09-03T06:45:41.064 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006334ad0721c64706-006312f815-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.065 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:41.066 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:41.067 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:41.067 DEBUG:boto:Method: HEAD 2022-09-03T06:45:41.068 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:45:41.068 DEBUG:boto:Data: 2022-09-03T06:45:41.069 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.070 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.070 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.071 DEBUG:boto:Params: {} 2022-09-03T06:45:41.072 DEBUG:boto:Token: None 2022-09-03T06:45:41.072 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj22 2022-09-03T06:45:41.073 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6g8a429d+sCPOu0Aznx2Ir+8DsA= 2022-09-03T06:45:41.074 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6g8a429d+sCPOu0Aznx2Ir+8DsA=', 'Content-Length': '0'} 2022-09-03T06:45:41.076 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007075b19ebbeb2ee9-006312f815-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.076 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:41.077 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:41.078 DEBUG:boto:Method: HEAD 2022-09-03T06:45:41.078 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:45:41.079 DEBUG:boto:Data: 2022-09-03T06:45:41.079 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.080 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.080 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.081 DEBUG:boto:Params: {} 2022-09-03T06:45:41.082 DEBUG:boto:Token: None 2022-09-03T06:45:41.082 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj22 2022-09-03T06:45:41.083 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6g8a429d+sCPOu0Aznx2Ir+8DsA= 2022-09-03T06:45:41.083 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6g8a429d+sCPOu0Aznx2Ir+8DsA=', 'Content-Length': '0'} 2022-09-03T06:45:41.085 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000809874562f9bf24e-006312f815-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.086 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:45:41.087 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:41.087 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:41.088 DEBUG:boto:Method: GET 2022-09-03T06:45:41.088 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:45:41.089 DEBUG:boto:Data: 2022-09-03T06:45:41.090 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.090 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.091 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.091 DEBUG:boto:Params: {} 2022-09-03T06:45:41.092 DEBUG:boto:Token: None 2022-09-03T06:45:41.093 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj22 2022-09-03T06:45:41.093 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tVhqAeLGrYEshhzoFsaREALBOCM= 2022-09-03T06:45:41.094 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tVhqAeLGrYEshhzoFsaREALBOCM=', 'Content-Length': '0'} 2022-09-03T06:45:41.096 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000983c76e8f256020c-006312f815-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.097 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:45:41.098 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:45:41.099 DEBUG:boto:Method: GET 2022-09-03T06:45:41.099 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:45:41.100 DEBUG:boto:Data: 2022-09-03T06:45:41.100 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.101 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.102 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.102 DEBUG:boto:Params: {} 2022-09-03T06:45:41.103 DEBUG:boto:Token: None 2022-09-03T06:45:41.103 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj22 2022-09-03T06:45:41.104 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tVhqAeLGrYEshhzoFsaREALBOCM= 2022-09-03T06:45:41.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tVhqAeLGrYEshhzoFsaREALBOCM=', 'Content-Length': '0'} 2022-09-03T06:45:41.107 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000223903ad915d7f0d-006312f815-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.108 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:41.109 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:41.109 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:41.110 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:41.110 DEBUG:boto:Method: GET 2022-09-03T06:45:41.111 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:45:41.111 DEBUG:boto:Data: 2022-09-03T06:45:41.112 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.112 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.113 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.114 DEBUG:boto:Params: {} 2022-09-03T06:45:41.114 DEBUG:boto:Token: None 2022-09-03T06:45:41.115 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/ 2022-09-03T06:45:41.116 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE= 2022-09-03T06:45:41.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE=', 'Content-Length': '0'} 2022-09-03T06:45:41.119 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009e78ce0f205fed13-006312f815-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.119 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:41.120 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:41.121 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:41.121 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:41.122 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:45:41.123 DEBUG:boto:Method: GET 2022-09-03T06:45:41.123 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:45:41.124 DEBUG:boto:Data: 2022-09-03T06:45:41.124 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.125 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.125 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.126 DEBUG:boto:Params: {} 2022-09-03T06:45:41.126 DEBUG:boto:Token: None 2022-09-03T06:45:41.127 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/ 2022-09-03T06:45:41.128 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE= 2022-09-03T06:45:41.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE=', 'Content-Length': '0'} 2022-09-03T06:45:41.131 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000031fccdf3d0781b1a-006312f815-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.132 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:41.132 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:45:41.133 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:41.134 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:41.134 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.135 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.135 DEBUG:boto:Method: GET 2022-09-03T06:45:41.136 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.137 DEBUG:boto:Data: 2022-09-03T06:45:41.137 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.138 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.138 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.139 DEBUG:boto:Params: {} 2022-09-03T06:45:41.139 DEBUG:boto:Token: None 2022-09-03T06:45:41.140 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.141 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WlAShUPFz4mpONwcXUAiKQnUL5Y= 2022-09-03T06:45:41.141 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WlAShUPFz4mpONwcXUAiKQnUL5Y=', 'Content-Length': '0'} 2022-09-03T06:45:41.144 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b637b8e12fd6980-006312f815-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.190 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:41.191 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:41.192 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.192 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.193 DEBUG:boto:Method: GET 2022-09-03T06:45:41.193 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.194 DEBUG:boto:Data: 2022-09-03T06:45:41.194 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.195 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.196 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.196 DEBUG:boto:Params: {} 2022-09-03T06:45:41.197 DEBUG:boto:Token: None 2022-09-03T06:45:41.198 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.199 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WlAShUPFz4mpONwcXUAiKQnUL5Y= 2022-09-03T06:45:41.199 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WlAShUPFz4mpONwcXUAiKQnUL5Y=', 'Content-Length': '0'} 2022-09-03T06:45:41.202 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d0027017e01414ca-006312f815-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.203 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:41.203 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:41.204 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.204 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.205 DEBUG:boto:Method: HEAD 2022-09-03T06:45:41.205 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.206 DEBUG:boto:Data: 2022-09-03T06:45:41.206 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.207 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.207 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.208 DEBUG:boto:Params: {} 2022-09-03T06:45:41.209 DEBUG:boto:Token: None 2022-09-03T06:45:41.210 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.210 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jwHf5m9JxUaus2L4yWN1So4U8oE= 2022-09-03T06:45:41.211 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jwHf5m9JxUaus2L4yWN1So4U8oE=', 'Content-Length': '0'} 2022-09-03T06:45:41.213 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d3f319d6aa81a565-006312f815-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.213 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:41.214 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:41.215 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.215 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.216 DEBUG:boto:Method: HEAD 2022-09-03T06:45:41.216 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.217 DEBUG:boto:Data: 2022-09-03T06:45:41.217 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.218 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.219 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.219 DEBUG:boto:Params: {} 2022-09-03T06:45:41.220 DEBUG:boto:Token: None 2022-09-03T06:45:41.220 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.221 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jwHf5m9JxUaus2L4yWN1So4U8oE= 2022-09-03T06:45:41.222 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jwHf5m9JxUaus2L4yWN1So4U8oE=', 'Content-Length': '0'} 2022-09-03T06:45:41.224 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065d8c880df45675a-006312f815-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.225 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:45:41.225 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:41.226 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:41.227 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.227 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.228 DEBUG:boto:Method: GET 2022-09-03T06:45:41.228 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.229 DEBUG:boto:Data: 2022-09-03T06:45:41.230 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.230 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.231 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.231 DEBUG:boto:Params: {} 2022-09-03T06:45:41.232 DEBUG:boto:Token: None 2022-09-03T06:45:41.233 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.233 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WlAShUPFz4mpONwcXUAiKQnUL5Y= 2022-09-03T06:45:41.234 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WlAShUPFz4mpONwcXUAiKQnUL5Y=', 'Content-Length': '0'} 2022-09-03T06:45:41.236 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006bb7e6ccbe451358-006312f815-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.237 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:41.238 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:41.238 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.239 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.239 DEBUG:boto:Method: GET 2022-09-03T06:45:41.240 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.240 DEBUG:boto:Data: 2022-09-03T06:45:41.241 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.241 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.242 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.243 DEBUG:boto:Params: {} 2022-09-03T06:45:41.243 DEBUG:boto:Token: None 2022-09-03T06:45:41.244 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:45:41.244 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WlAShUPFz4mpONwcXUAiKQnUL5Y= 2022-09-03T06:45:41.245 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WlAShUPFz4mpONwcXUAiKQnUL5Y=', 'Content-Length': '0'} 2022-09-03T06:45:41.247 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a9e8dcf7466db7e-006312f815-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.248 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:41.249 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:41.250 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:41.250 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:41.251 DEBUG:boto:Method: GET 2022-09-03T06:45:41.251 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:45:41.252 DEBUG:boto:Data: 2022-09-03T06:45:41.252 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.253 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.253 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.254 DEBUG:boto:Params: {} 2022-09-03T06:45:41.254 DEBUG:boto:Token: None 2022-09-03T06:45:41.255 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/ 2022-09-03T06:45:41.256 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE= 2022-09-03T06:45:41.256 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE=', 'Content-Length': '0'} 2022-09-03T06:45:41.259 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000acaa5655136d31de-006312f815-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.260 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:41.261 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:41.262 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:41.262 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:41.263 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:41.263 DEBUG:boto:Method: GET 2022-09-03T06:45:41.264 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:45:41.264 DEBUG:boto:Data: 2022-09-03T06:45:41.265 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.265 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.266 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.267 DEBUG:boto:Params: {} 2022-09-03T06:45:41.267 DEBUG:boto:Token: None 2022-09-03T06:45:41.268 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/ 2022-09-03T06:45:41.268 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE= 2022-09-03T06:45:41.269 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE=', 'Content-Length': '0'} 2022-09-03T06:45:41.272 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004573a50fc0e5f2d0-006312f815-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.272 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:41.273 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:41.274 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:41.274 DEBUG:boto:Method: HEAD 2022-09-03T06:45:41.275 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:45:41.275 DEBUG:boto:Data: 2022-09-03T06:45:41.276 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.276 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.277 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.277 DEBUG:boto:Params: {} 2022-09-03T06:45:41.278 DEBUG:boto:Token: None 2022-09-03T06:45:41.279 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj23 2022-09-03T06:45:41.280 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v493vgoTyRtVSlI5hfQPx3D43kc= 2022-09-03T06:45:41.280 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v493vgoTyRtVSlI5hfQPx3D43kc=', 'Content-Length': '0'} 2022-09-03T06:45:41.282 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009825c85850739ab0-006312f815-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.283 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:41.283 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:41.284 DEBUG:boto:Method: HEAD 2022-09-03T06:45:41.284 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:45:41.285 DEBUG:boto:Data: 2022-09-03T06:45:41.285 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.286 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.286 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.287 DEBUG:boto:Params: {} 2022-09-03T06:45:41.287 DEBUG:boto:Token: None 2022-09-03T06:45:41.288 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj23 2022-09-03T06:45:41.289 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v493vgoTyRtVSlI5hfQPx3D43kc= 2022-09-03T06:45:41.289 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v493vgoTyRtVSlI5hfQPx3D43kc=', 'Content-Length': '0'} 2022-09-03T06:45:41.291 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e176cc8677036a22-006312f815-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.292 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:45:41.293 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:41.293 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:41.294 DEBUG:boto:Method: GET 2022-09-03T06:45:41.295 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:45:41.295 DEBUG:boto:Data: 2022-09-03T06:45:41.296 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.296 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.297 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.297 DEBUG:boto:Params: {} 2022-09-03T06:45:41.298 DEBUG:boto:Token: None 2022-09-03T06:45:41.298 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj23 2022-09-03T06:45:41.299 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XedlcOqCnkeeAEpdZLfeRNNuB34= 2022-09-03T06:45:41.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XedlcOqCnkeeAEpdZLfeRNNuB34=', 'Content-Length': '0'} 2022-09-03T06:45:41.302 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000756201509502536a-006312f815-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.350 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:45:41.351 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:45:41.352 DEBUG:boto:Method: GET 2022-09-03T06:45:41.352 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:45:41.353 DEBUG:boto:Data: 2022-09-03T06:45:41.353 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.355 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.355 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.356 DEBUG:boto:Params: {} 2022-09-03T06:45:41.356 DEBUG:boto:Token: None 2022-09-03T06:45:41.357 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj23 2022-09-03T06:45:41.358 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XedlcOqCnkeeAEpdZLfeRNNuB34= 2022-09-03T06:45:41.359 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XedlcOqCnkeeAEpdZLfeRNNuB34=', 'Content-Length': '0'} 2022-09-03T06:45:41.361 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004e774653b77865b7-006312f815-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.362 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:41.362 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:41.363 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:41.363 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:41.364 DEBUG:boto:Method: GET 2022-09-03T06:45:41.365 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:45:41.365 DEBUG:boto:Data: 2022-09-03T06:45:41.366 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.367 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.367 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.368 DEBUG:boto:Params: {} 2022-09-03T06:45:41.368 DEBUG:boto:Token: None 2022-09-03T06:45:41.369 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/ 2022-09-03T06:45:41.370 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE= 2022-09-03T06:45:41.370 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE=', 'Content-Length': '0'} 2022-09-03T06:45:41.373 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003d54a9dd0d6634bc-006312f815-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.374 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:41.375 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:41.376 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:41.376 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:41.377 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:45:41.378 DEBUG:boto:Method: GET 2022-09-03T06:45:41.379 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:45:41.380 DEBUG:boto:Data: 2022-09-03T06:45:41.381 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.382 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.383 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.384 DEBUG:boto:Params: {} 2022-09-03T06:45:41.385 DEBUG:boto:Token: None 2022-09-03T06:45:41.386 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/ 2022-09-03T06:45:41.386 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE= 2022-09-03T06:45:41.387 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE=', 'Content-Length': '0'} 2022-09-03T06:45:41.390 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000096dead9a8f36a474-006312f815-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.390 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:41.391 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:45:41.392 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:41.392 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:41.393 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.394 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.394 DEBUG:boto:Method: GET 2022-09-03T06:45:41.395 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.396 DEBUG:boto:Data: 2022-09-03T06:45:41.397 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.397 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.398 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.399 DEBUG:boto:Params: {} 2022-09-03T06:45:41.400 DEBUG:boto:Token: None 2022-09-03T06:45:41.401 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.403 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hbfpRrXx/pB2usOwuLcnjdAvhVk= 2022-09-03T06:45:41.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hbfpRrXx/pB2usOwuLcnjdAvhVk=', 'Content-Length': '0'} 2022-09-03T06:45:41.407 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000046f2803f99bae039-006312f815-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.408 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:41.409 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:41.409 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.410 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.411 DEBUG:boto:Method: GET 2022-09-03T06:45:41.412 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.412 DEBUG:boto:Data: 2022-09-03T06:45:41.413 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.413 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.414 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.414 DEBUG:boto:Params: {} 2022-09-03T06:45:41.415 DEBUG:boto:Token: None 2022-09-03T06:45:41.416 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.416 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hbfpRrXx/pB2usOwuLcnjdAvhVk= 2022-09-03T06:45:41.417 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hbfpRrXx/pB2usOwuLcnjdAvhVk=', 'Content-Length': '0'} 2022-09-03T06:45:41.420 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008fa995784e843d3c-006312f815-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.420 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:41.421 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:41.422 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.422 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.423 DEBUG:boto:Method: HEAD 2022-09-03T06:45:41.423 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.424 DEBUG:boto:Data: 2022-09-03T06:45:41.424 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.425 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.425 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.426 DEBUG:boto:Params: {} 2022-09-03T06:45:41.427 DEBUG:boto:Token: None 2022-09-03T06:45:41.428 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.429 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gxUten/EePN5ytGCKRz+aS02c8A= 2022-09-03T06:45:41.429 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gxUten/EePN5ytGCKRz+aS02c8A=', 'Content-Length': '0'} 2022-09-03T06:45:41.431 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f791ba24a2648dc6-006312f815-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.432 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:41.433 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:41.433 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.434 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.434 DEBUG:boto:Method: HEAD 2022-09-03T06:45:41.435 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.435 DEBUG:boto:Data: 2022-09-03T06:45:41.436 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.437 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.438 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.439 DEBUG:boto:Params: {} 2022-09-03T06:45:41.439 DEBUG:boto:Token: None 2022-09-03T06:45:41.440 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.440 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gxUten/EePN5ytGCKRz+aS02c8A= 2022-09-03T06:45:41.441 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gxUten/EePN5ytGCKRz+aS02c8A=', 'Content-Length': '0'} 2022-09-03T06:45:41.443 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000004aacde440bb7657-006312f815-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.444 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:45:41.445 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:41.445 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:41.446 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.446 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.447 DEBUG:boto:Method: GET 2022-09-03T06:45:41.448 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.448 DEBUG:boto:Data: 2022-09-03T06:45:41.449 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.449 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.450 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.451 DEBUG:boto:Params: {} 2022-09-03T06:45:41.451 DEBUG:boto:Token: None 2022-09-03T06:45:41.452 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.453 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hbfpRrXx/pB2usOwuLcnjdAvhVk= 2022-09-03T06:45:41.454 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hbfpRrXx/pB2usOwuLcnjdAvhVk=', 'Content-Length': '0'} 2022-09-03T06:45:41.457 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fd5a88288fdc2cdd-006312f815-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.458 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:41.459 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:41.460 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.461 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.462 DEBUG:boto:Method: GET 2022-09-03T06:45:41.463 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.463 DEBUG:boto:Data: 2022-09-03T06:45:41.464 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.464 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.465 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.465 DEBUG:boto:Params: {} 2022-09-03T06:45:41.466 DEBUG:boto:Token: None 2022-09-03T06:45:41.467 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:45:41.467 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hbfpRrXx/pB2usOwuLcnjdAvhVk= 2022-09-03T06:45:41.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hbfpRrXx/pB2usOwuLcnjdAvhVk=', 'Content-Length': '0'} 2022-09-03T06:45:41.470 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a921d1a3320fbcca-006312f815-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.472 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:41.473 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:41.474 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:41.475 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:41.476 DEBUG:boto:Method: GET 2022-09-03T06:45:41.477 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:45:41.477 DEBUG:boto:Data: 2022-09-03T06:45:41.479 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.479 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.480 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.481 DEBUG:boto:Params: {} 2022-09-03T06:45:41.482 DEBUG:boto:Token: None 2022-09-03T06:45:41.482 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/ 2022-09-03T06:45:41.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE= 2022-09-03T06:45:41.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE=', 'Content-Length': '0'} 2022-09-03T06:45:41.486 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e8b0dded44b42e52-006312f815-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.487 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:41.488 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:41.489 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:41.489 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:41.490 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:41.491 DEBUG:boto:Method: GET 2022-09-03T06:45:41.492 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:45:41.493 DEBUG:boto:Data: 2022-09-03T06:45:41.494 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.495 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.496 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.497 DEBUG:boto:Params: {} 2022-09-03T06:45:41.499 DEBUG:boto:Token: None 2022-09-03T06:45:41.500 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/ 2022-09-03T06:45:41.501 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE= 2022-09-03T06:45:41.502 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE=', 'Content-Length': '0'} 2022-09-03T06:45:41.505 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009fb047c1ddaf736b-006312f815-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.546 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:41.548 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:41.549 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:41.550 DEBUG:boto:Method: HEAD 2022-09-03T06:45:41.551 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:45:41.552 DEBUG:boto:Data: 2022-09-03T06:45:41.554 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.554 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.555 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.555 DEBUG:boto:Params: {} 2022-09-03T06:45:41.556 DEBUG:boto:Token: None 2022-09-03T06:45:41.557 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj29 2022-09-03T06:45:41.557 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:T0FpbXcSnhcoy9uyN+3aaHZdSn4= 2022-09-03T06:45:41.558 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:T0FpbXcSnhcoy9uyN+3aaHZdSn4=', 'Content-Length': '0'} 2022-09-03T06:45:41.560 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d00e60147f55a953-006312f815-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.561 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:41.561 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:41.562 DEBUG:boto:Method: HEAD 2022-09-03T06:45:41.562 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:45:41.563 DEBUG:boto:Data: 2022-09-03T06:45:41.564 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.564 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.565 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.565 DEBUG:boto:Params: {} 2022-09-03T06:45:41.566 DEBUG:boto:Token: None 2022-09-03T06:45:41.567 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj29 2022-09-03T06:45:41.567 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:T0FpbXcSnhcoy9uyN+3aaHZdSn4= 2022-09-03T06:45:41.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:T0FpbXcSnhcoy9uyN+3aaHZdSn4=', 'Content-Length': '0'} 2022-09-03T06:45:41.570 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000888043a617b3d78a-006312f815-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.571 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:45:41.571 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:41.572 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:41.573 DEBUG:boto:Method: GET 2022-09-03T06:45:41.573 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:45:41.574 DEBUG:boto:Data: 2022-09-03T06:45:41.574 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.575 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.576 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.576 DEBUG:boto:Params: {} 2022-09-03T06:45:41.577 DEBUG:boto:Token: None 2022-09-03T06:45:41.578 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj29 2022-09-03T06:45:41.578 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LOJAHQkfZKMGd/HhU+v5ULOfQRQ= 2022-09-03T06:45:41.579 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LOJAHQkfZKMGd/HhU+v5ULOfQRQ=', 'Content-Length': '0'} 2022-09-03T06:45:41.583 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007ae369f464f68478-006312f815-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.584 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:45:41.584 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:45:41.585 DEBUG:boto:Method: GET 2022-09-03T06:45:41.585 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:45:41.586 DEBUG:boto:Data: 2022-09-03T06:45:41.586 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.588 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.589 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.589 DEBUG:boto:Params: {} 2022-09-03T06:45:41.590 DEBUG:boto:Token: None 2022-09-03T06:45:41.591 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj29 2022-09-03T06:45:41.591 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LOJAHQkfZKMGd/HhU+v5ULOfQRQ= 2022-09-03T06:45:41.592 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LOJAHQkfZKMGd/HhU+v5ULOfQRQ=', 'Content-Length': '0'} 2022-09-03T06:45:41.594 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b0268340a051862-006312f815-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.595 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:41.596 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:41.596 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:41.597 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:41.597 DEBUG:boto:Method: GET 2022-09-03T06:45:41.599 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:45:41.599 DEBUG:boto:Data: 2022-09-03T06:45:41.600 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.600 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.601 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.601 DEBUG:boto:Params: {} 2022-09-03T06:45:41.602 DEBUG:boto:Token: None 2022-09-03T06:45:41.603 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/ 2022-09-03T06:45:41.603 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE= 2022-09-03T06:45:41.605 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE=', 'Content-Length': '0'} 2022-09-03T06:45:41.608 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fe75c31150e02eb4-006312f815-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.609 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:41.610 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:41.611 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:41.611 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:41.612 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:45:41.612 DEBUG:boto:Method: GET 2022-09-03T06:45:41.613 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:45:41.613 DEBUG:boto:Data: 2022-09-03T06:45:41.614 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.614 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.615 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.615 DEBUG:boto:Params: {} 2022-09-03T06:45:41.616 DEBUG:boto:Token: None 2022-09-03T06:45:41.616 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/ 2022-09-03T06:45:41.618 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE= 2022-09-03T06:45:41.621 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE=', 'Content-Length': '0'} 2022-09-03T06:45:41.623 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000268440e64f711f49-006312f815-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.670 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:41.671 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:45:41.672 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:41.673 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:41.673 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.674 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.675 DEBUG:boto:Method: GET 2022-09-03T06:45:41.675 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.676 DEBUG:boto:Data: 2022-09-03T06:45:41.677 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.677 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.678 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.678 DEBUG:boto:Params: {} 2022-09-03T06:45:41.679 DEBUG:boto:Token: None 2022-09-03T06:45:41.680 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.681 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hUPOD4v3IbJ7kHldomuUJ+c7qBY= 2022-09-03T06:45:41.681 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hUPOD4v3IbJ7kHldomuUJ+c7qBY=', 'Content-Length': '0'} 2022-09-03T06:45:41.684 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b8e50000a13e062-006312f815-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.686 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:41.686 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:41.687 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.688 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.688 DEBUG:boto:Method: GET 2022-09-03T06:45:41.689 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.690 DEBUG:boto:Data: 2022-09-03T06:45:41.690 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.698 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.699 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.699 DEBUG:boto:Params: {} 2022-09-03T06:45:41.700 DEBUG:boto:Token: None 2022-09-03T06:45:41.701 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.701 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hUPOD4v3IbJ7kHldomuUJ+c7qBY= 2022-09-03T06:45:41.702 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hUPOD4v3IbJ7kHldomuUJ+c7qBY=', 'Content-Length': '0'} 2022-09-03T06:45:41.704 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000100ee4ab43522e32-006312f815-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.705 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:41.706 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:41.706 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.707 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.707 DEBUG:boto:Method: HEAD 2022-09-03T06:45:41.708 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.708 DEBUG:boto:Data: 2022-09-03T06:45:41.709 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.709 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.710 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.710 DEBUG:boto:Params: {} 2022-09-03T06:45:41.711 DEBUG:boto:Token: None 2022-09-03T06:45:41.712 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.712 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O8M9WkPUGJVdrFxcwo502s0jYeI= 2022-09-03T06:45:41.714 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O8M9WkPUGJVdrFxcwo502s0jYeI=', 'Content-Length': '0'} 2022-09-03T06:45:41.716 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008344a28711046236-006312f815-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.717 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:41.718 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:41.718 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.719 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.720 DEBUG:boto:Method: HEAD 2022-09-03T06:45:41.720 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.721 DEBUG:boto:Data: 2022-09-03T06:45:41.721 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.722 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.722 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.723 DEBUG:boto:Params: {} 2022-09-03T06:45:41.723 DEBUG:boto:Token: None 2022-09-03T06:45:41.724 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.724 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O8M9WkPUGJVdrFxcwo502s0jYeI= 2022-09-03T06:45:41.725 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O8M9WkPUGJVdrFxcwo502s0jYeI=', 'Content-Length': '0'} 2022-09-03T06:45:41.727 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000fe8fa1cd155755a-006312f815-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.729 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:45:41.729 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:41.730 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:41.730 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.731 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.731 DEBUG:boto:Method: GET 2022-09-03T06:45:41.732 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.733 DEBUG:boto:Data: 2022-09-03T06:45:41.733 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.734 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.734 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.735 DEBUG:boto:Params: {} 2022-09-03T06:45:41.735 DEBUG:boto:Token: None 2022-09-03T06:45:41.736 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.737 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hUPOD4v3IbJ7kHldomuUJ+c7qBY= 2022-09-03T06:45:41.737 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hUPOD4v3IbJ7kHldomuUJ+c7qBY=', 'Content-Length': '0'} 2022-09-03T06:45:41.739 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dd3e66fce2c0af8f-006312f815-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.740 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:41.741 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:41.741 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.742 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.742 DEBUG:boto:Method: GET 2022-09-03T06:45:41.743 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.744 DEBUG:boto:Data: 2022-09-03T06:45:41.744 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.745 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.745 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.746 DEBUG:boto:Params: {} 2022-09-03T06:45:41.746 DEBUG:boto:Token: None 2022-09-03T06:45:41.747 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:45:41.748 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hUPOD4v3IbJ7kHldomuUJ+c7qBY= 2022-09-03T06:45:41.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hUPOD4v3IbJ7kHldomuUJ+c7qBY=', 'Content-Length': '0'} 2022-09-03T06:45:41.751 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074f8af6791a66aa4-006312f815-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.752 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:41.752 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:41.753 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:41.753 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:41.754 DEBUG:boto:Method: GET 2022-09-03T06:45:41.754 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:45:41.755 DEBUG:boto:Data: 2022-09-03T06:45:41.755 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.756 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.756 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.757 DEBUG:boto:Params: {} 2022-09-03T06:45:41.758 DEBUG:boto:Token: None 2022-09-03T06:45:41.759 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/ 2022-09-03T06:45:41.759 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE= 2022-09-03T06:45:41.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE=', 'Content-Length': '0'} 2022-09-03T06:45:41.764 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007530953a3c69d2dc-006312f815-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.765 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:41.766 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:41.766 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:41.767 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:41.768 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:41.768 DEBUG:boto:Method: GET 2022-09-03T06:45:41.769 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:45:41.769 DEBUG:boto:Data: 2022-09-03T06:45:41.770 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.770 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.771 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.771 DEBUG:boto:Params: {} 2022-09-03T06:45:41.772 DEBUG:boto:Token: None 2022-09-03T06:45:41.773 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/ 2022-09-03T06:45:41.773 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE= 2022-09-03T06:45:41.774 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE=', 'Content-Length': '0'} 2022-09-03T06:45:41.777 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000095276183444adf0c-006312f815-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.818 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:41.819 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:41.820 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:41.820 DEBUG:boto:Method: HEAD 2022-09-03T06:45:41.821 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:45:41.821 DEBUG:boto:Data: 2022-09-03T06:45:41.822 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.822 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.823 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.824 DEBUG:boto:Params: {} 2022-09-03T06:45:41.824 DEBUG:boto:Token: None 2022-09-03T06:45:41.825 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj33 2022-09-03T06:45:41.826 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HM96VuyJqegX/rHfpm0ZJuNUpZI= 2022-09-03T06:45:41.826 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HM96VuyJqegX/rHfpm0ZJuNUpZI=', 'Content-Length': '0'} 2022-09-03T06:45:41.829 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e95f59d7c08d004e-006312f815-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.830 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:41.831 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:41.832 DEBUG:boto:Method: HEAD 2022-09-03T06:45:41.832 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:45:41.833 DEBUG:boto:Data: 2022-09-03T06:45:41.833 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.834 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.834 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.836 DEBUG:boto:Params: {} 2022-09-03T06:45:41.837 DEBUG:boto:Token: None 2022-09-03T06:45:41.837 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj33 2022-09-03T06:45:41.838 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HM96VuyJqegX/rHfpm0ZJuNUpZI= 2022-09-03T06:45:41.839 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HM96VuyJqegX/rHfpm0ZJuNUpZI=', 'Content-Length': '0'} 2022-09-03T06:45:41.841 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002134e816ac0f3f5c-006312f815-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.842 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:45:41.843 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:41.843 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:41.844 DEBUG:boto:Method: GET 2022-09-03T06:45:41.844 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:45:41.845 DEBUG:boto:Data: 2022-09-03T06:45:41.847 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.847 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.849 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.849 DEBUG:boto:Params: {} 2022-09-03T06:45:41.850 DEBUG:boto:Token: None 2022-09-03T06:45:41.851 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj33 2022-09-03T06:45:41.852 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RLUJuHHaZPJCQ9ZKwwiN4+svMB0= 2022-09-03T06:45:41.853 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RLUJuHHaZPJCQ9ZKwwiN4+svMB0=', 'Content-Length': '0'} 2022-09-03T06:45:41.855 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000544640044f122bd7-006312f815-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.857 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:45:41.858 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:45:41.859 DEBUG:boto:Method: GET 2022-09-03T06:45:41.860 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:45:41.861 DEBUG:boto:Data: 2022-09-03T06:45:41.861 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.862 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.863 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.864 DEBUG:boto:Params: {} 2022-09-03T06:45:41.864 DEBUG:boto:Token: None 2022-09-03T06:45:41.867 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj33 2022-09-03T06:45:41.868 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RLUJuHHaZPJCQ9ZKwwiN4+svMB0= 2022-09-03T06:45:41.869 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RLUJuHHaZPJCQ9ZKwwiN4+svMB0=', 'Content-Length': '0'} 2022-09-03T06:45:41.871 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aa74f22a1dbc3a2c-006312f815-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.872 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:41.872 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:41.873 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:41.874 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:41.874 DEBUG:boto:Method: GET 2022-09-03T06:45:41.875 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:45:41.875 DEBUG:boto:Data: 2022-09-03T06:45:41.876 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.876 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.877 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.877 DEBUG:boto:Params: {} 2022-09-03T06:45:41.878 DEBUG:boto:Token: None 2022-09-03T06:45:41.879 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/ 2022-09-03T06:45:41.879 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE= 2022-09-03T06:45:41.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE=', 'Content-Length': '0'} 2022-09-03T06:45:41.883 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a26121ad2451f6c1-006312f815-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.884 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:41.885 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:41.886 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:41.886 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:41.887 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:45:41.887 DEBUG:boto:Method: GET 2022-09-03T06:45:41.888 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:45:41.888 DEBUG:boto:Data: 2022-09-03T06:45:41.889 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.889 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.890 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.891 DEBUG:boto:Params: {} 2022-09-03T06:45:41.891 DEBUG:boto:Token: None 2022-09-03T06:45:41.892 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/ 2022-09-03T06:45:41.892 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE= 2022-09-03T06:45:41.893 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LJOsAMpBb4ApvgB87RAuby9S7oE=', 'Content-Length': '0'} 2022-09-03T06:45:41.896 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003591d8e79e27e3a9-006312f815-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.896 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:41.897 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:45:41.898 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:41.899 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:41.900 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:41.901 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:41.901 DEBUG:boto:Method: GET 2022-09-03T06:45:41.902 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:41.902 DEBUG:boto:Data: 2022-09-03T06:45:41.903 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.903 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.904 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.904 DEBUG:boto:Params: {} 2022-09-03T06:45:41.905 DEBUG:boto:Token: None 2022-09-03T06:45:41.906 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:41.906 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Agdgs1ypUaQveGuUW8sw3Ne0J1g= 2022-09-03T06:45:41.907 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Agdgs1ypUaQveGuUW8sw3Ne0J1g=', 'Content-Length': '0'} 2022-09-03T06:45:41.915 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f966c739afac16b9-006312f815-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.958 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:41.959 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:41.960 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:41.961 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:41.961 DEBUG:boto:Method: GET 2022-09-03T06:45:41.962 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:41.963 DEBUG:boto:Data: 2022-09-03T06:45:41.964 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.964 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.965 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.966 DEBUG:boto:Params: {} 2022-09-03T06:45:41.967 DEBUG:boto:Token: None 2022-09-03T06:45:41.967 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:41.968 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Agdgs1ypUaQveGuUW8sw3Ne0J1g= 2022-09-03T06:45:41.969 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Agdgs1ypUaQveGuUW8sw3Ne0J1g=', 'Content-Length': '0'} 2022-09-03T06:45:41.972 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b7d4d1970aeb07e-006312f815-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.973 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:41.974 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:41.974 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:41.975 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:41.976 DEBUG:boto:Method: HEAD 2022-09-03T06:45:41.976 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:41.977 DEBUG:boto:Data: 2022-09-03T06:45:41.978 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.978 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:41.979 DEBUG:boto:Port: 8001 2022-09-03T06:45:41.980 DEBUG:boto:Params: {} 2022-09-03T06:45:41.981 DEBUG:boto:Token: None 2022-09-03T06:45:41.981 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:41 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:41.982 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1hk7qaVIqI6DsKSNkJkWptKr+AI= 2022-09-03T06:45:41.983 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1hk7qaVIqI6DsKSNkJkWptKr+AI=', 'Content-Length': '0'} 2022-09-03T06:45:41.986 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e7d4b849ddcea1ad-006312f815-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:41.988 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:41.989 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:41.990 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:41.992 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:41.993 DEBUG:boto:Method: HEAD 2022-09-03T06:45:41.994 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:41.995 DEBUG:boto:Data: 2022-09-03T06:45:41.996 DEBUG:boto:Headers: {} 2022-09-03T06:45:41.997 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:41.998 DEBUG:boto:Port: 8000 2022-09-03T06:45:41.999 DEBUG:boto:Params: {} 2022-09-03T06:45:42.001 DEBUG:boto:Token: None 2022-09-03T06:45:42.002 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:42.003 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KWEBMRR5K8T3TAkjUrrsPbquhVI= 2022-09-03T06:45:42.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KWEBMRR5K8T3TAkjUrrsPbquhVI=', 'Content-Length': '0'} 2022-09-03T06:45:42.006 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000086077ad11204c86f-006312f816-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.007 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:45:42.008 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:42.009 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:42.010 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:42.011 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:42.011 DEBUG:boto:Method: GET 2022-09-03T06:45:42.012 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:42.013 DEBUG:boto:Data: 2022-09-03T06:45:42.014 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.015 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:42.015 DEBUG:boto:Port: 8001 2022-09-03T06:45:42.016 DEBUG:boto:Params: {} 2022-09-03T06:45:42.017 DEBUG:boto:Token: None 2022-09-03T06:45:42.018 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:42.019 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OfUYWqxWjnNhEJHlNcifCm7DlDY= 2022-09-03T06:45:42.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OfUYWqxWjnNhEJHlNcifCm7DlDY=', 'Content-Length': '0'} 2022-09-03T06:45:42.022 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000228682accfc1fa5-006312f816-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.023 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:42.024 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:42.025 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:42.025 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:42.026 DEBUG:boto:Method: GET 2022-09-03T06:45:42.028 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:42.029 DEBUG:boto:Data: 2022-09-03T06:45:42.029 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.030 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:42.031 DEBUG:boto:Port: 8000 2022-09-03T06:45:42.031 DEBUG:boto:Params: {} 2022-09-03T06:45:42.032 DEBUG:boto:Token: None 2022-09-03T06:45:42.033 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:45:42.034 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OfUYWqxWjnNhEJHlNcifCm7DlDY= 2022-09-03T06:45:42.034 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OfUYWqxWjnNhEJHlNcifCm7DlDY=', 'Content-Length': '0'} 2022-09-03T06:45:42.037 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ac179f3b55523769-006312f816-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.078 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:42.079 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:42.080 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:42.081 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:42.081 DEBUG:boto:Method: GET 2022-09-03T06:45:42.082 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:45:42.083 DEBUG:boto:Data: 2022-09-03T06:45:42.083 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.084 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:42.085 DEBUG:boto:Port: 8001 2022-09-03T06:45:42.085 DEBUG:boto:Params: {} 2022-09-03T06:45:42.086 DEBUG:boto:Token: None 2022-09-03T06:45:42.087 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/ 2022-09-03T06:45:42.087 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY= 2022-09-03T06:45:42.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY=', 'Content-Length': '0'} 2022-09-03T06:45:42.093 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000869c1ec04cdf791d-006312f816-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.094 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:42.095 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:42.096 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:42.096 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:42.097 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:42.097 DEBUG:boto:Method: GET 2022-09-03T06:45:42.098 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:45:42.099 DEBUG:boto:Data: 2022-09-03T06:45:42.099 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.100 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:42.100 DEBUG:boto:Port: 8000 2022-09-03T06:45:42.101 DEBUG:boto:Params: {} 2022-09-03T06:45:42.101 DEBUG:boto:Token: None 2022-09-03T06:45:42.102 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/ 2022-09-03T06:45:42.103 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY= 2022-09-03T06:45:42.103 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY=', 'Content-Length': '0'} 2022-09-03T06:45:42.106 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000084db9cf8c8d0dd26-006312f816-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.107 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:42.108 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:42.109 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:42.110 DEBUG:boto:Method: HEAD 2022-09-03T06:45:42.111 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:45:42.111 DEBUG:boto:Data: 2022-09-03T06:45:42.112 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.113 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:42.113 DEBUG:boto:Port: 8001 2022-09-03T06:45:42.115 DEBUG:boto:Params: {} 2022-09-03T06:45:42.115 DEBUG:boto:Token: None 2022-09-03T06:45:42.116 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj34 2022-09-03T06:45:42.117 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zre+OHcb7hXvRBe73l/f2FAEmJA= 2022-09-03T06:45:42.118 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zre+OHcb7hXvRBe73l/f2FAEmJA=', 'Content-Length': '0'} 2022-09-03T06:45:42.121 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fd3158d8f0be8f92-006312f816-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.122 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:42.123 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:42.123 DEBUG:boto:Method: HEAD 2022-09-03T06:45:42.125 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:45:42.125 DEBUG:boto:Data: 2022-09-03T06:45:42.126 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.127 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:42.127 DEBUG:boto:Port: 8000 2022-09-03T06:45:42.128 DEBUG:boto:Params: {} 2022-09-03T06:45:42.129 DEBUG:boto:Token: None 2022-09-03T06:45:42.129 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj34 2022-09-03T06:45:42.130 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Zre+OHcb7hXvRBe73l/f2FAEmJA= 2022-09-03T06:45:42.131 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Zre+OHcb7hXvRBe73l/f2FAEmJA=', 'Content-Length': '0'} 2022-09-03T06:45:42.134 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fb4ed593fc184cb1-006312f816-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.136 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:45:42.137 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:42.138 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:42.139 DEBUG:boto:Method: GET 2022-09-03T06:45:42.140 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:45:42.141 DEBUG:boto:Data: 2022-09-03T06:45:42.142 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.144 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:42.145 DEBUG:boto:Port: 8001 2022-09-03T06:45:42.146 DEBUG:boto:Params: {} 2022-09-03T06:45:42.147 DEBUG:boto:Token: None 2022-09-03T06:45:42.148 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj34 2022-09-03T06:45:42.149 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cJ4zvUt32IbYy3VHwKNdfSDWItg= 2022-09-03T06:45:42.150 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cJ4zvUt32IbYy3VHwKNdfSDWItg=', 'Content-Length': '0'} 2022-09-03T06:45:42.153 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e95ce375fd66057-006312f816-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.198 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:45:42.200 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:45:42.201 DEBUG:boto:Method: GET 2022-09-03T06:45:42.202 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:45:42.203 DEBUG:boto:Data: 2022-09-03T06:45:42.203 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.204 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:42.204 DEBUG:boto:Port: 8000 2022-09-03T06:45:42.205 DEBUG:boto:Params: {} 2022-09-03T06:45:42.206 DEBUG:boto:Token: None 2022-09-03T06:45:42.206 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj34 2022-09-03T06:45:42.207 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cJ4zvUt32IbYy3VHwKNdfSDWItg= 2022-09-03T06:45:42.208 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cJ4zvUt32IbYy3VHwKNdfSDWItg=', 'Content-Length': '0'} 2022-09-03T06:45:42.210 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000059276040eba7085-006312f816-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.211 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:42.212 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:42.213 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:42.214 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:42.215 DEBUG:boto:Method: GET 2022-09-03T06:45:42.215 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:45:42.216 DEBUG:boto:Data: 2022-09-03T06:45:42.216 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.217 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:42.218 DEBUG:boto:Port: 8001 2022-09-03T06:45:42.218 DEBUG:boto:Params: {} 2022-09-03T06:45:42.219 DEBUG:boto:Token: None 2022-09-03T06:45:42.219 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/ 2022-09-03T06:45:42.220 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY= 2022-09-03T06:45:42.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY=', 'Content-Length': '0'} 2022-09-03T06:45:42.223 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000014688860e401659f-006312f816-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.224 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:42.225 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:42.225 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:42.226 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:42.226 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:45:42.227 DEBUG:boto:Method: GET 2022-09-03T06:45:42.227 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:45:42.228 DEBUG:boto:Data: 2022-09-03T06:45:42.229 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.229 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:42.230 DEBUG:boto:Port: 8000 2022-09-03T06:45:42.230 DEBUG:boto:Params: {} 2022-09-03T06:45:42.231 DEBUG:boto:Token: None 2022-09-03T06:45:42.231 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/ 2022-09-03T06:45:42.232 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY= 2022-09-03T06:45:42.233 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY=', 'Content-Length': '0'} 2022-09-03T06:45:42.236 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a553239b55435292-006312f816-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.282 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:42.283 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:45:42.284 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:42.284 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:42.285 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.285 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.286 DEBUG:boto:Method: GET 2022-09-03T06:45:42.287 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.287 DEBUG:boto:Data: 2022-09-03T06:45:42.288 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.288 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:42.289 DEBUG:boto:Port: 8001 2022-09-03T06:45:42.290 DEBUG:boto:Params: {} 2022-09-03T06:45:42.290 DEBUG:boto:Token: None 2022-09-03T06:45:42.292 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.292 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8aXja1xByQzXkg2wTqw8uB7DiBI= 2022-09-03T06:45:42.293 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8aXja1xByQzXkg2wTqw8uB7DiBI=', 'Content-Length': '0'} 2022-09-03T06:45:42.295 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e42823947dd30297-006312f816-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.297 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:42.298 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:42.299 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.300 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.301 DEBUG:boto:Method: GET 2022-09-03T06:45:42.301 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.303 DEBUG:boto:Data: 2022-09-03T06:45:42.304 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.305 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:42.305 DEBUG:boto:Port: 8000 2022-09-03T06:45:42.306 DEBUG:boto:Params: {} 2022-09-03T06:45:42.307 DEBUG:boto:Token: None 2022-09-03T06:45:42.308 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.308 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8aXja1xByQzXkg2wTqw8uB7DiBI= 2022-09-03T06:45:42.309 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8aXja1xByQzXkg2wTqw8uB7DiBI=', 'Content-Length': '0'} 2022-09-03T06:45:42.311 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000078c76bea75dff56a-006312f816-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.313 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:42.313 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:42.314 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.314 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.315 DEBUG:boto:Method: HEAD 2022-09-03T06:45:42.316 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.317 DEBUG:boto:Data: 2022-09-03T06:45:42.318 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.319 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:42.319 DEBUG:boto:Port: 8001 2022-09-03T06:45:42.320 DEBUG:boto:Params: {} 2022-09-03T06:45:42.321 DEBUG:boto:Token: None 2022-09-03T06:45:42.322 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.323 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4g5Uzx/Ch3CR7R9F3/UFNuopj4Y= 2022-09-03T06:45:42.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4g5Uzx/Ch3CR7R9F3/UFNuopj4Y=', 'Content-Length': '0'} 2022-09-03T06:45:42.326 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e0eadfa8b45b990c-006312f816-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.327 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:42.327 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:42.328 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.328 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.329 DEBUG:boto:Method: HEAD 2022-09-03T06:45:42.329 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.330 DEBUG:boto:Data: 2022-09-03T06:45:42.331 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.331 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:42.332 DEBUG:boto:Port: 8000 2022-09-03T06:45:42.332 DEBUG:boto:Params: {} 2022-09-03T06:45:42.333 DEBUG:boto:Token: None 2022-09-03T06:45:42.333 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.334 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4g5Uzx/Ch3CR7R9F3/UFNuopj4Y= 2022-09-03T06:45:42.335 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4g5Uzx/Ch3CR7R9F3/UFNuopj4Y=', 'Content-Length': '0'} 2022-09-03T06:45:42.337 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000cc08f7bb915faed-006312f816-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.338 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:45:42.339 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:42.340 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:42.341 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.342 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.343 DEBUG:boto:Method: GET 2022-09-03T06:45:42.343 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.344 DEBUG:boto:Data: 2022-09-03T06:45:42.344 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.345 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:42.345 DEBUG:boto:Port: 8001 2022-09-03T06:45:42.346 DEBUG:boto:Params: {} 2022-09-03T06:45:42.346 DEBUG:boto:Token: None 2022-09-03T06:45:42.347 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.348 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8aXja1xByQzXkg2wTqw8uB7DiBI= 2022-09-03T06:45:42.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8aXja1xByQzXkg2wTqw8uB7DiBI=', 'Content-Length': '0'} 2022-09-03T06:45:42.351 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d95ac4f4c709bd42-006312f816-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.353 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:42.353 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:42.354 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.355 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.355 DEBUG:boto:Method: GET 2022-09-03T06:45:42.356 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.356 DEBUG:boto:Data: 2022-09-03T06:45:42.357 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.357 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:42.358 DEBUG:boto:Port: 8000 2022-09-03T06:45:42.359 DEBUG:boto:Params: {} 2022-09-03T06:45:42.359 DEBUG:boto:Token: None 2022-09-03T06:45:42.360 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:45:42.361 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8aXja1xByQzXkg2wTqw8uB7DiBI= 2022-09-03T06:45:42.361 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8aXja1xByQzXkg2wTqw8uB7DiBI=', 'Content-Length': '0'} 2022-09-03T06:45:42.363 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db88ac8465cd1d79-006312f816-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.364 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:42.365 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:42.366 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:42.366 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:42.367 DEBUG:boto:Method: GET 2022-09-03T06:45:42.367 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:45:42.368 DEBUG:boto:Data: 2022-09-03T06:45:42.368 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.369 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:42.370 DEBUG:boto:Port: 8001 2022-09-03T06:45:42.370 DEBUG:boto:Params: {} 2022-09-03T06:45:42.371 DEBUG:boto:Token: None 2022-09-03T06:45:42.371 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/ 2022-09-03T06:45:42.372 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY= 2022-09-03T06:45:42.373 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY=', 'Content-Length': '0'} 2022-09-03T06:45:42.375 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dbe2e1b1a916c57d-006312f816-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.418 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:42.419 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:42.420 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:42.420 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:42.421 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:42.422 DEBUG:boto:Method: GET 2022-09-03T06:45:42.422 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:45:42.423 DEBUG:boto:Data: 2022-09-03T06:45:42.423 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.424 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:42.424 DEBUG:boto:Port: 8000 2022-09-03T06:45:42.425 DEBUG:boto:Params: {} 2022-09-03T06:45:42.426 DEBUG:boto:Token: None 2022-09-03T06:45:42.429 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/ 2022-09-03T06:45:42.433 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY= 2022-09-03T06:45:42.435 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY=', 'Content-Length': '0'} 2022-09-03T06:45:42.439 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000013246305273083ba-006312f816-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.440 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:42.441 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:42.442 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:42.442 DEBUG:boto:Method: HEAD 2022-09-03T06:45:42.443 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:45:42.443 DEBUG:boto:Data: 2022-09-03T06:45:42.444 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.445 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:42.445 DEBUG:boto:Port: 8001 2022-09-03T06:45:42.446 DEBUG:boto:Params: {} 2022-09-03T06:45:42.446 DEBUG:boto:Token: None 2022-09-03T06:45:42.448 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj38 2022-09-03T06:45:42.448 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q0lMjV5LVT74yx05LwABRIFi/kM= 2022-09-03T06:45:42.449 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q0lMjV5LVT74yx05LwABRIFi/kM=', 'Content-Length': '0'} 2022-09-03T06:45:42.451 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000055003d41b16bd8dd-006312f816-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.452 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:42.453 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:42.453 DEBUG:boto:Method: HEAD 2022-09-03T06:45:42.454 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:45:42.455 DEBUG:boto:Data: 2022-09-03T06:45:42.455 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.456 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:42.457 DEBUG:boto:Port: 8000 2022-09-03T06:45:42.458 DEBUG:boto:Params: {} 2022-09-03T06:45:42.459 DEBUG:boto:Token: None 2022-09-03T06:45:42.460 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj38 2022-09-03T06:45:42.461 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q0lMjV5LVT74yx05LwABRIFi/kM= 2022-09-03T06:45:42.462 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q0lMjV5LVT74yx05LwABRIFi/kM=', 'Content-Length': '0'} 2022-09-03T06:45:42.464 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cdc7f9d5d0aa1d6d-006312f816-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.466 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:45:42.467 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:42.468 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:42.469 DEBUG:boto:Method: GET 2022-09-03T06:45:42.470 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:45:42.471 DEBUG:boto:Data: 2022-09-03T06:45:42.473 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.474 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:42.475 DEBUG:boto:Port: 8001 2022-09-03T06:45:42.476 DEBUG:boto:Params: {} 2022-09-03T06:45:42.477 DEBUG:boto:Token: None 2022-09-03T06:45:42.478 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj38 2022-09-03T06:45:42.480 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XVbABljh437Er4munjNrlRgeMw8= 2022-09-03T06:45:42.481 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XVbABljh437Er4munjNrlRgeMw8=', 'Content-Length': '0'} 2022-09-03T06:45:42.484 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000108493a59de0b0ee-006312f816-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.486 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:45:42.487 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:45:42.488 DEBUG:boto:Method: GET 2022-09-03T06:45:42.489 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:45:42.491 DEBUG:boto:Data: 2022-09-03T06:45:42.492 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.493 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:42.494 DEBUG:boto:Port: 8000 2022-09-03T06:45:42.495 DEBUG:boto:Params: {} 2022-09-03T06:45:42.496 DEBUG:boto:Token: None 2022-09-03T06:45:42.497 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj38 2022-09-03T06:45:42.499 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XVbABljh437Er4munjNrlRgeMw8= 2022-09-03T06:45:42.501 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XVbABljh437Er4munjNrlRgeMw8=', 'Content-Length': '0'} 2022-09-03T06:45:42.504 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c44c8f81eb9419c5-006312f816-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.505 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:42.506 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:42.507 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:42.508 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:42.509 DEBUG:boto:Method: GET 2022-09-03T06:45:42.510 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:45:42.510 DEBUG:boto:Data: 2022-09-03T06:45:42.511 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.513 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:42.515 DEBUG:boto:Port: 8001 2022-09-03T06:45:42.515 DEBUG:boto:Params: {} 2022-09-03T06:45:42.516 DEBUG:boto:Token: None 2022-09-03T06:45:42.517 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/ 2022-09-03T06:45:42.518 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY= 2022-09-03T06:45:42.519 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY=', 'Content-Length': '0'} 2022-09-03T06:45:42.522 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cfa562c9a75f1039-006312f816-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.523 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:42.525 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:42.525 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:42.527 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:42.528 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:45:42.528 DEBUG:boto:Method: GET 2022-09-03T06:45:42.529 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:45:42.530 DEBUG:boto:Data: 2022-09-03T06:45:42.531 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.532 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:42.533 DEBUG:boto:Port: 8000 2022-09-03T06:45:42.534 DEBUG:boto:Params: {} 2022-09-03T06:45:42.535 DEBUG:boto:Token: None 2022-09-03T06:45:42.536 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/ 2022-09-03T06:45:42.538 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY= 2022-09-03T06:45:42.539 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY=', 'Content-Length': '0'} 2022-09-03T06:45:42.542 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a6b9365f785f4e4f-006312f816-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.547 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:42.549 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:45:42.550 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:42.551 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:42.552 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.554 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.557 DEBUG:boto:Method: GET 2022-09-03T06:45:42.558 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.559 DEBUG:boto:Data: 2022-09-03T06:45:42.560 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.561 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:42.562 DEBUG:boto:Port: 8001 2022-09-03T06:45:42.564 DEBUG:boto:Params: {} 2022-09-03T06:45:42.565 DEBUG:boto:Token: None 2022-09-03T06:45:42.566 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.567 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mlrrSgUBftFHctQz31uf/NUSXXk= 2022-09-03T06:45:42.569 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mlrrSgUBftFHctQz31uf/NUSXXk=', 'Content-Length': '0'} 2022-09-03T06:45:42.583 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001befd8014edd933d-006312f816-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.594 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:42.595 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:42.595 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.596 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.597 DEBUG:boto:Method: GET 2022-09-03T06:45:42.598 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.598 DEBUG:boto:Data: 2022-09-03T06:45:42.599 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.599 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:42.600 DEBUG:boto:Port: 8000 2022-09-03T06:45:42.600 DEBUG:boto:Params: {} 2022-09-03T06:45:42.601 DEBUG:boto:Token: None 2022-09-03T06:45:42.602 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.603 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mlrrSgUBftFHctQz31uf/NUSXXk= 2022-09-03T06:45:42.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mlrrSgUBftFHctQz31uf/NUSXXk=', 'Content-Length': '0'} 2022-09-03T06:45:42.606 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fbd55a4478e0f9c0-006312f816-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.607 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:42.608 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:42.609 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.609 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.610 DEBUG:boto:Method: HEAD 2022-09-03T06:45:42.610 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.611 DEBUG:boto:Data: 2022-09-03T06:45:42.612 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.612 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:42.613 DEBUG:boto:Port: 8001 2022-09-03T06:45:42.613 DEBUG:boto:Params: {} 2022-09-03T06:45:42.614 DEBUG:boto:Token: None 2022-09-03T06:45:42.614 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.615 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XfVAkl17DAQJmIow1ticVd384QY= 2022-09-03T06:45:42.616 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XfVAkl17DAQJmIow1ticVd384QY=', 'Content-Length': '0'} 2022-09-03T06:45:42.618 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e1b5696e13677d73-006312f816-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.619 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:42.619 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:42.620 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.620 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.621 DEBUG:boto:Method: HEAD 2022-09-03T06:45:42.622 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.622 DEBUG:boto:Data: 2022-09-03T06:45:42.623 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.623 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:42.624 DEBUG:boto:Port: 8000 2022-09-03T06:45:42.624 DEBUG:boto:Params: {} 2022-09-03T06:45:42.625 DEBUG:boto:Token: None 2022-09-03T06:45:42.625 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.626 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XfVAkl17DAQJmIow1ticVd384QY= 2022-09-03T06:45:42.627 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XfVAkl17DAQJmIow1ticVd384QY=', 'Content-Length': '0'} 2022-09-03T06:45:42.629 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002e83865bbfc65bf4-006312f816-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.630 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:45:42.631 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:42.631 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:42.632 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.632 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.633 DEBUG:boto:Method: GET 2022-09-03T06:45:42.633 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.634 DEBUG:boto:Data: 2022-09-03T06:45:42.635 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.635 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:42.636 DEBUG:boto:Port: 8001 2022-09-03T06:45:42.636 DEBUG:boto:Params: {} 2022-09-03T06:45:42.637 DEBUG:boto:Token: None 2022-09-03T06:45:42.638 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.638 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mlrrSgUBftFHctQz31uf/NUSXXk= 2022-09-03T06:45:42.639 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mlrrSgUBftFHctQz31uf/NUSXXk=', 'Content-Length': '0'} 2022-09-03T06:45:42.642 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d67c8abcbb023e21-006312f816-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.690 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:42.691 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:42.692 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.692 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.693 DEBUG:boto:Method: GET 2022-09-03T06:45:42.694 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.695 DEBUG:boto:Data: 2022-09-03T06:45:42.695 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.696 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:42.697 DEBUG:boto:Port: 8000 2022-09-03T06:45:42.698 DEBUG:boto:Params: {} 2022-09-03T06:45:42.700 DEBUG:boto:Token: None 2022-09-03T06:45:42.701 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:45:42.702 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mlrrSgUBftFHctQz31uf/NUSXXk= 2022-09-03T06:45:42.703 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mlrrSgUBftFHctQz31uf/NUSXXk=', 'Content-Length': '0'} 2022-09-03T06:45:42.711 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000774e0d38ed55c018-006312f816-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.713 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:42.714 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:42.715 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:42.716 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:42.719 DEBUG:boto:Method: GET 2022-09-03T06:45:42.720 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:45:42.721 DEBUG:boto:Data: 2022-09-03T06:45:42.722 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.723 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:42.724 DEBUG:boto:Port: 8001 2022-09-03T06:45:42.725 DEBUG:boto:Params: {} 2022-09-03T06:45:42.727 DEBUG:boto:Token: None 2022-09-03T06:45:42.728 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/ 2022-09-03T06:45:42.729 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY= 2022-09-03T06:45:42.729 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY=', 'Content-Length': '0'} 2022-09-03T06:45:42.733 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f3e8a8f70fbb08b8-006312f816-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.735 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:42.736 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:42.737 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:42.737 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:42.739 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:42.741 DEBUG:boto:Method: GET 2022-09-03T06:45:42.742 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:45:42.743 DEBUG:boto:Data: 2022-09-03T06:45:42.744 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.744 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:42.745 DEBUG:boto:Port: 8000 2022-09-03T06:45:42.746 DEBUG:boto:Params: {} 2022-09-03T06:45:42.747 DEBUG:boto:Token: None 2022-09-03T06:45:42.748 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/ 2022-09-03T06:45:42.748 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY= 2022-09-03T06:45:42.749 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY=', 'Content-Length': '0'} 2022-09-03T06:45:42.753 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000037a9f11b8d8180d7-006312f816-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.754 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:42.755 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:42.755 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:42.756 DEBUG:boto:Method: HEAD 2022-09-03T06:45:42.757 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:45:42.757 DEBUG:boto:Data: 2022-09-03T06:45:42.758 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.758 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:42.759 DEBUG:boto:Port: 8001 2022-09-03T06:45:42.759 DEBUG:boto:Params: {} 2022-09-03T06:45:42.760 DEBUG:boto:Token: None 2022-09-03T06:45:42.760 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj44 2022-09-03T06:45:42.761 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JmMl1eXm/laE++vM0JQNdHOQDp8= 2022-09-03T06:45:42.762 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JmMl1eXm/laE++vM0JQNdHOQDp8=', 'Content-Length': '0'} 2022-09-03T06:45:42.767 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e6392b17cd9ff19-006312f816-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.768 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:42.769 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:42.770 DEBUG:boto:Method: HEAD 2022-09-03T06:45:42.770 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:45:42.771 DEBUG:boto:Data: 2022-09-03T06:45:42.771 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.772 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:42.772 DEBUG:boto:Port: 8000 2022-09-03T06:45:42.773 DEBUG:boto:Params: {} 2022-09-03T06:45:42.773 DEBUG:boto:Token: None 2022-09-03T06:45:42.774 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj44 2022-09-03T06:45:42.775 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JmMl1eXm/laE++vM0JQNdHOQDp8= 2022-09-03T06:45:42.775 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JmMl1eXm/laE++vM0JQNdHOQDp8=', 'Content-Length': '0'} 2022-09-03T06:45:42.778 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000503a55430a7f4741-006312f816-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.780 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:45:42.781 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:42.784 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:42.785 DEBUG:boto:Method: GET 2022-09-03T06:45:42.787 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:45:42.788 DEBUG:boto:Data: 2022-09-03T06:45:42.790 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.791 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:42.791 DEBUG:boto:Port: 8001 2022-09-03T06:45:42.792 DEBUG:boto:Params: {} 2022-09-03T06:45:42.793 DEBUG:boto:Token: None 2022-09-03T06:45:42.793 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj44 2022-09-03T06:45:42.794 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v7ploE4rEntQfUz7YvLi1+CWXVE= 2022-09-03T06:45:42.795 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v7ploE4rEntQfUz7YvLi1+CWXVE=', 'Content-Length': '0'} 2022-09-03T06:45:42.798 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cdaf0ff189c693c6-006312f816-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.800 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:45:42.801 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:45:42.802 DEBUG:boto:Method: GET 2022-09-03T06:45:42.803 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:45:42.803 DEBUG:boto:Data: 2022-09-03T06:45:42.804 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.805 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:42.805 DEBUG:boto:Port: 8000 2022-09-03T06:45:42.806 DEBUG:boto:Params: {} 2022-09-03T06:45:42.807 DEBUG:boto:Token: None 2022-09-03T06:45:42.808 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj44 2022-09-03T06:45:42.809 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v7ploE4rEntQfUz7YvLi1+CWXVE= 2022-09-03T06:45:42.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v7ploE4rEntQfUz7YvLi1+CWXVE=', 'Content-Length': '0'} 2022-09-03T06:45:42.817 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d6ff507bcc0024dd-006312f816-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.823 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:42.824 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:42.825 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:42.825 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:42.826 DEBUG:boto:Method: GET 2022-09-03T06:45:42.827 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:45:42.827 DEBUG:boto:Data: 2022-09-03T06:45:42.828 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.828 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:42.829 DEBUG:boto:Port: 8001 2022-09-03T06:45:42.830 DEBUG:boto:Params: {} 2022-09-03T06:45:42.830 DEBUG:boto:Token: None 2022-09-03T06:45:42.831 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/ 2022-09-03T06:45:42.833 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY= 2022-09-03T06:45:42.834 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY=', 'Content-Length': '0'} 2022-09-03T06:45:42.844 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cd5f2a5d10333907-006312f816-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.846 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:42.847 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:42.849 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:42.851 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:42.852 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:45:42.854 DEBUG:boto:Method: GET 2022-09-03T06:45:42.855 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:45:42.856 DEBUG:boto:Data: 2022-09-03T06:45:42.857 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.858 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:42.859 DEBUG:boto:Port: 8000 2022-09-03T06:45:42.860 DEBUG:boto:Params: {} 2022-09-03T06:45:42.861 DEBUG:boto:Token: None 2022-09-03T06:45:42.863 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/ 2022-09-03T06:45:42.864 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY= 2022-09-03T06:45:42.865 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/efCoWQOUu0PjvfeDbcVW2YYYSY=', 'Content-Length': '0'} 2022-09-03T06:45:42.869 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006005297365b57ef9-006312f816-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.871 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:42.873 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:45:42.875 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:42.876 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:42.877 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:42.878 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:42.879 DEBUG:boto:Method: GET 2022-09-03T06:45:42.880 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:42.880 DEBUG:boto:Data: 2022-09-03T06:45:42.882 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.883 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:42.884 DEBUG:boto:Port: 8001 2022-09-03T06:45:42.886 DEBUG:boto:Params: {} 2022-09-03T06:45:42.887 DEBUG:boto:Token: None 2022-09-03T06:45:42.888 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:42.890 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:45f/uDCiZtpj+uYHdkdVDiIGHbU= 2022-09-03T06:45:42.891 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:45f/uDCiZtpj+uYHdkdVDiIGHbU=', 'Content-Length': '0'} 2022-09-03T06:45:42.896 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000068e0921ebbb18c22-006312f816-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.900 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:42.903 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:42.904 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:42.905 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:42.907 DEBUG:boto:Method: GET 2022-09-03T06:45:42.908 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:42.910 DEBUG:boto:Data: 2022-09-03T06:45:42.912 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.915 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:42.916 DEBUG:boto:Port: 8000 2022-09-03T06:45:42.916 DEBUG:boto:Params: {} 2022-09-03T06:45:42.917 DEBUG:boto:Token: None 2022-09-03T06:45:42.918 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:42.919 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:45f/uDCiZtpj+uYHdkdVDiIGHbU= 2022-09-03T06:45:42.919 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:45f/uDCiZtpj+uYHdkdVDiIGHbU=', 'Content-Length': '0'} 2022-09-03T06:45:42.923 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c6c22910cf26aca1-006312f816-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.929 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:42.930 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:42.931 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:42.936 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:42.939 DEBUG:boto:Method: HEAD 2022-09-03T06:45:42.941 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:42.942 DEBUG:boto:Data: 2022-09-03T06:45:42.943 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.944 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:42.945 DEBUG:boto:Port: 8001 2022-09-03T06:45:42.946 DEBUG:boto:Params: {} 2022-09-03T06:45:42.947 DEBUG:boto:Token: None 2022-09-03T06:45:42.950 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:42.951 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ICFY+CToyHH7bLpNBnWYQtGDuSo= 2022-09-03T06:45:42.952 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ICFY+CToyHH7bLpNBnWYQtGDuSo=', 'Content-Length': '0'} 2022-09-03T06:45:42.960 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000090db6e9c290a2d38-006312f816-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.962 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:42.965 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:42.966 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:42.967 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:42.967 DEBUG:boto:Method: HEAD 2022-09-03T06:45:42.968 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:42.969 DEBUG:boto:Data: 2022-09-03T06:45:42.970 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.970 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:42.971 DEBUG:boto:Port: 8000 2022-09-03T06:45:42.972 DEBUG:boto:Params: {} 2022-09-03T06:45:42.973 DEBUG:boto:Token: None 2022-09-03T06:45:42.973 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:42.975 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ICFY+CToyHH7bLpNBnWYQtGDuSo= 2022-09-03T06:45:42.977 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ICFY+CToyHH7bLpNBnWYQtGDuSo=', 'Content-Length': '0'} 2022-09-03T06:45:42.980 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000024a5615db8a1a669-006312f816-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.981 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:45:42.982 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:42.983 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:42.983 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:42.984 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:42.984 DEBUG:boto:Method: GET 2022-09-03T06:45:42.985 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:42.986 DEBUG:boto:Data: 2022-09-03T06:45:42.986 DEBUG:boto:Headers: {} 2022-09-03T06:45:42.987 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:42.988 DEBUG:boto:Port: 8001 2022-09-03T06:45:42.989 DEBUG:boto:Params: {} 2022-09-03T06:45:42.989 DEBUG:boto:Token: None 2022-09-03T06:45:42.990 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:42 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:42.991 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:45f/uDCiZtpj+uYHdkdVDiIGHbU= 2022-09-03T06:45:42.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:45f/uDCiZtpj+uYHdkdVDiIGHbU=', 'Content-Length': '0'} 2022-09-03T06:45:42.994 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000055538f376c722aa9-006312f816-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:42.997 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:42.998 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:42.999 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:43.000 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:43.000 DEBUG:boto:Method: GET 2022-09-03T06:45:43.001 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:43.003 DEBUG:boto:Data: 2022-09-03T06:45:43.004 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.005 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:43.006 DEBUG:boto:Port: 8000 2022-09-03T06:45:43.006 DEBUG:boto:Params: {} 2022-09-03T06:45:43.007 DEBUG:boto:Token: None 2022-09-03T06:45:43.008 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:45:43.009 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b0B7t4NgrOTgZzOLzVPw5MBI3TQ= 2022-09-03T06:45:43.010 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b0B7t4NgrOTgZzOLzVPw5MBI3TQ=', 'Content-Length': '0'} 2022-09-03T06:45:43.013 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003f9697a7f1239315-006312f817-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.015 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:43.015 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:43.016 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:43.017 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:43.018 DEBUG:boto:Method: GET 2022-09-03T06:45:43.019 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:45:43.019 DEBUG:boto:Data: 2022-09-03T06:45:43.020 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.021 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:43.021 DEBUG:boto:Port: 8001 2022-09-03T06:45:43.022 DEBUG:boto:Params: {} 2022-09-03T06:45:43.023 DEBUG:boto:Token: None 2022-09-03T06:45:43.024 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/ 2022-09-03T06:45:43.025 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng= 2022-09-03T06:45:43.026 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng=', 'Content-Length': '0'} 2022-09-03T06:45:43.031 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003a8244b592d17c46-006312f817-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.080 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:43.081 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:43.082 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:43.083 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:43.084 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:43.085 DEBUG:boto:Method: GET 2022-09-03T06:45:43.087 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:45:43.088 DEBUG:boto:Data: 2022-09-03T06:45:43.092 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.095 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:43.098 DEBUG:boto:Port: 8000 2022-09-03T06:45:43.098 DEBUG:boto:Params: {} 2022-09-03T06:45:43.100 DEBUG:boto:Token: None 2022-09-03T06:45:43.100 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/ 2022-09-03T06:45:43.101 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng= 2022-09-03T06:45:43.102 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng=', 'Content-Length': '0'} 2022-09-03T06:45:43.108 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006907a01ac3238b22-006312f817-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.110 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:43.114 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:43.115 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:43.117 DEBUG:boto:Method: HEAD 2022-09-03T06:45:43.119 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:45:43.120 DEBUG:boto:Data: 2022-09-03T06:45:43.122 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.123 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:43.124 DEBUG:boto:Port: 8001 2022-09-03T06:45:43.125 DEBUG:boto:Params: {} 2022-09-03T06:45:43.127 DEBUG:boto:Token: None 2022-09-03T06:45:43.128 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj46 2022-09-03T06:45:43.129 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ukvgE6Br+bfIM6QD2JSLpVcwcls= 2022-09-03T06:45:43.131 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ukvgE6Br+bfIM6QD2JSLpVcwcls=', 'Content-Length': '0'} 2022-09-03T06:45:43.134 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d6674379d1645bad-006312f817-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.135 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:43.135 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:43.136 DEBUG:boto:Method: HEAD 2022-09-03T06:45:43.137 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:45:43.137 DEBUG:boto:Data: 2022-09-03T06:45:43.138 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.139 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:43.139 DEBUG:boto:Port: 8000 2022-09-03T06:45:43.140 DEBUG:boto:Params: {} 2022-09-03T06:45:43.141 DEBUG:boto:Token: None 2022-09-03T06:45:43.142 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj46 2022-09-03T06:45:43.143 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ukvgE6Br+bfIM6QD2JSLpVcwcls= 2022-09-03T06:45:43.143 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ukvgE6Br+bfIM6QD2JSLpVcwcls=', 'Content-Length': '0'} 2022-09-03T06:45:43.146 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ccc30783fd3c1aad-006312f817-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.147 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:45:43.148 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:43.149 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:43.150 DEBUG:boto:Method: GET 2022-09-03T06:45:43.151 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:45:43.152 DEBUG:boto:Data: 2022-09-03T06:45:43.153 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.154 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:43.154 DEBUG:boto:Port: 8001 2022-09-03T06:45:43.155 DEBUG:boto:Params: {} 2022-09-03T06:45:43.156 DEBUG:boto:Token: None 2022-09-03T06:45:43.157 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj46 2022-09-03T06:45:43.158 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7KajH6hl/2kUGIcjoaoekufCBFU= 2022-09-03T06:45:43.158 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7KajH6hl/2kUGIcjoaoekufCBFU=', 'Content-Length': '0'} 2022-09-03T06:45:43.161 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000068fba34f9d509768-006312f817-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.163 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:45:43.164 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:45:43.165 DEBUG:boto:Method: GET 2022-09-03T06:45:43.166 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:45:43.167 DEBUG:boto:Data: 2022-09-03T06:45:43.167 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.168 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:43.169 DEBUG:boto:Port: 8000 2022-09-03T06:45:43.170 DEBUG:boto:Params: {} 2022-09-03T06:45:43.171 DEBUG:boto:Token: None 2022-09-03T06:45:43.172 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj46 2022-09-03T06:45:43.173 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7KajH6hl/2kUGIcjoaoekufCBFU= 2022-09-03T06:45:43.174 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7KajH6hl/2kUGIcjoaoekufCBFU=', 'Content-Length': '0'} 2022-09-03T06:45:43.177 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000174bd50b89f0d15e-006312f817-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.179 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:43.180 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:43.181 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:43.181 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:43.182 DEBUG:boto:Method: GET 2022-09-03T06:45:43.183 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:45:43.184 DEBUG:boto:Data: 2022-09-03T06:45:43.185 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.186 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:43.187 DEBUG:boto:Port: 8001 2022-09-03T06:45:43.188 DEBUG:boto:Params: {} 2022-09-03T06:45:43.189 DEBUG:boto:Token: None 2022-09-03T06:45:43.190 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/ 2022-09-03T06:45:43.191 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng= 2022-09-03T06:45:43.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng=', 'Content-Length': '0'} 2022-09-03T06:45:43.196 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c1bb3dd4d0c6f421-006312f817-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.199 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:43.201 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:43.202 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:43.204 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:43.205 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:45:43.208 DEBUG:boto:Method: GET 2022-09-03T06:45:43.208 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:45:43.212 DEBUG:boto:Data: 2022-09-03T06:45:43.213 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.215 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:43.216 DEBUG:boto:Port: 8000 2022-09-03T06:45:43.217 DEBUG:boto:Params: {} 2022-09-03T06:45:43.219 DEBUG:boto:Token: None 2022-09-03T06:45:43.220 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/ 2022-09-03T06:45:43.221 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng= 2022-09-03T06:45:43.222 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng=', 'Content-Length': '0'} 2022-09-03T06:45:43.226 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f6243ca8318790d2-006312f817-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.228 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:43.229 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:45:43.231 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:43.232 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:43.233 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.235 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.236 DEBUG:boto:Method: GET 2022-09-03T06:45:43.237 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.239 DEBUG:boto:Data: 2022-09-03T06:45:43.240 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.242 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:43.243 DEBUG:boto:Port: 8001 2022-09-03T06:45:43.245 DEBUG:boto:Params: {} 2022-09-03T06:45:43.246 DEBUG:boto:Token: None 2022-09-03T06:45:43.248 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.249 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TOYEUhftakDb5aCGQJ4UT+QA1SQ= 2022-09-03T06:45:43.250 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TOYEUhftakDb5aCGQJ4UT+QA1SQ=', 'Content-Length': '0'} 2022-09-03T06:45:43.254 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a463a5e682c8d99-006312f817-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.256 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:43.258 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:43.259 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.260 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.261 DEBUG:boto:Method: GET 2022-09-03T06:45:43.262 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.266 DEBUG:boto:Data: 2022-09-03T06:45:43.268 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.269 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:43.270 DEBUG:boto:Port: 8000 2022-09-03T06:45:43.271 DEBUG:boto:Params: {} 2022-09-03T06:45:43.273 DEBUG:boto:Token: None 2022-09-03T06:45:43.274 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.275 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TOYEUhftakDb5aCGQJ4UT+QA1SQ= 2022-09-03T06:45:43.277 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TOYEUhftakDb5aCGQJ4UT+QA1SQ=', 'Content-Length': '0'} 2022-09-03T06:45:43.280 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009cd76d1e931ed462-006312f817-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.284 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:43.285 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:43.287 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.288 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.289 DEBUG:boto:Method: HEAD 2022-09-03T06:45:43.290 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.290 DEBUG:boto:Data: 2022-09-03T06:45:43.291 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.292 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:43.293 DEBUG:boto:Port: 8001 2022-09-03T06:45:43.293 DEBUG:boto:Params: {} 2022-09-03T06:45:43.295 DEBUG:boto:Token: None 2022-09-03T06:45:43.296 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.297 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:txd9Ys0KbAZbVImM0SIHxzC3cEo= 2022-09-03T06:45:43.298 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:txd9Ys0KbAZbVImM0SIHxzC3cEo=', 'Content-Length': '0'} 2022-09-03T06:45:43.302 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000002fa37196685fced-006312f817-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.303 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:43.304 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:43.305 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.306 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.307 DEBUG:boto:Method: HEAD 2022-09-03T06:45:43.308 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.309 DEBUG:boto:Data: 2022-09-03T06:45:43.309 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.310 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:43.311 DEBUG:boto:Port: 8000 2022-09-03T06:45:43.312 DEBUG:boto:Params: {} 2022-09-03T06:45:43.313 DEBUG:boto:Token: None 2022-09-03T06:45:43.314 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.315 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:txd9Ys0KbAZbVImM0SIHxzC3cEo= 2022-09-03T06:45:43.315 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:txd9Ys0KbAZbVImM0SIHxzC3cEo=', 'Content-Length': '0'} 2022-09-03T06:45:43.322 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007873ad5e83293c9c-006312f817-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.323 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:45:43.324 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:43.324 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:43.325 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.326 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.327 DEBUG:boto:Method: GET 2022-09-03T06:45:43.327 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.328 DEBUG:boto:Data: 2022-09-03T06:45:43.329 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.330 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:43.331 DEBUG:boto:Port: 8001 2022-09-03T06:45:43.331 DEBUG:boto:Params: {} 2022-09-03T06:45:43.332 DEBUG:boto:Token: None 2022-09-03T06:45:43.333 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.333 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TOYEUhftakDb5aCGQJ4UT+QA1SQ= 2022-09-03T06:45:43.334 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TOYEUhftakDb5aCGQJ4UT+QA1SQ=', 'Content-Length': '0'} 2022-09-03T06:45:43.337 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d8b127914a785351-006312f817-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.339 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:43.339 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:43.340 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.341 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.341 DEBUG:boto:Method: GET 2022-09-03T06:45:43.342 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.342 DEBUG:boto:Data: 2022-09-03T06:45:43.343 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.343 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:43.344 DEBUG:boto:Port: 8000 2022-09-03T06:45:43.344 DEBUG:boto:Params: {} 2022-09-03T06:45:43.345 DEBUG:boto:Token: None 2022-09-03T06:45:43.346 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:45:43.346 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TOYEUhftakDb5aCGQJ4UT+QA1SQ= 2022-09-03T06:45:43.347 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TOYEUhftakDb5aCGQJ4UT+QA1SQ=', 'Content-Length': '0'} 2022-09-03T06:45:43.351 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2c791f0cd624dd4-006312f817-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.390 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:43.391 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:43.392 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:43.393 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:43.394 DEBUG:boto:Method: GET 2022-09-03T06:45:43.396 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:45:43.397 DEBUG:boto:Data: 2022-09-03T06:45:43.399 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.399 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:43.400 DEBUG:boto:Port: 8001 2022-09-03T06:45:43.400 DEBUG:boto:Params: {} 2022-09-03T06:45:43.401 DEBUG:boto:Token: None 2022-09-03T06:45:43.402 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/ 2022-09-03T06:45:43.403 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng= 2022-09-03T06:45:43.403 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng=', 'Content-Length': '0'} 2022-09-03T06:45:43.417 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008b54b9a2a17e2aee-006312f817-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.419 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:43.421 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:43.422 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:43.423 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:43.424 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:43.426 DEBUG:boto:Method: GET 2022-09-03T06:45:43.427 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:45:43.428 DEBUG:boto:Data: 2022-09-03T06:45:43.429 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.431 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:43.432 DEBUG:boto:Port: 8000 2022-09-03T06:45:43.433 DEBUG:boto:Params: {} 2022-09-03T06:45:43.434 DEBUG:boto:Token: None 2022-09-03T06:45:43.435 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/ 2022-09-03T06:45:43.435 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng= 2022-09-03T06:45:43.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng=', 'Content-Length': '0'} 2022-09-03T06:45:43.439 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009869ca76313191a2-006312f817-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.440 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:43.441 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:43.442 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:43.442 DEBUG:boto:Method: HEAD 2022-09-03T06:45:43.443 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:45:43.444 DEBUG:boto:Data: 2022-09-03T06:45:43.444 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.445 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:43.446 DEBUG:boto:Port: 8001 2022-09-03T06:45:43.446 DEBUG:boto:Params: {} 2022-09-03T06:45:43.447 DEBUG:boto:Token: None 2022-09-03T06:45:43.448 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj51 2022-09-03T06:45:43.449 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s9lhMyjeNvFnYz8QVXHXvLj6q1E= 2022-09-03T06:45:43.450 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s9lhMyjeNvFnYz8QVXHXvLj6q1E=', 'Content-Length': '0'} 2022-09-03T06:45:43.452 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000688e11316bdf004c-006312f817-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.454 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:43.455 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:43.456 DEBUG:boto:Method: HEAD 2022-09-03T06:45:43.457 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:45:43.458 DEBUG:boto:Data: 2022-09-03T06:45:43.459 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.460 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:43.461 DEBUG:boto:Port: 8000 2022-09-03T06:45:43.462 DEBUG:boto:Params: {} 2022-09-03T06:45:43.463 DEBUG:boto:Token: None 2022-09-03T06:45:43.464 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj51 2022-09-03T06:45:43.465 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s9lhMyjeNvFnYz8QVXHXvLj6q1E= 2022-09-03T06:45:43.466 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s9lhMyjeNvFnYz8QVXHXvLj6q1E=', 'Content-Length': '0'} 2022-09-03T06:45:43.469 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d2240114aaf93576-006312f817-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.471 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:45:43.471 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:43.472 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:43.473 DEBUG:boto:Method: GET 2022-09-03T06:45:43.473 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:45:43.474 DEBUG:boto:Data: 2022-09-03T06:45:43.474 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.475 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:43.475 DEBUG:boto:Port: 8001 2022-09-03T06:45:43.476 DEBUG:boto:Params: {} 2022-09-03T06:45:43.477 DEBUG:boto:Token: None 2022-09-03T06:45:43.477 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj51 2022-09-03T06:45:43.478 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V93028iGs/8XOHiXwGplDWGtXX0= 2022-09-03T06:45:43.479 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V93028iGs/8XOHiXwGplDWGtXX0=', 'Content-Length': '0'} 2022-09-03T06:45:43.482 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000011e406865b410a61-006312f817-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.483 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:45:43.484 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:45:43.485 DEBUG:boto:Method: GET 2022-09-03T06:45:43.486 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:45:43.487 DEBUG:boto:Data: 2022-09-03T06:45:43.488 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.489 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:43.490 DEBUG:boto:Port: 8000 2022-09-03T06:45:43.491 DEBUG:boto:Params: {} 2022-09-03T06:45:43.492 DEBUG:boto:Token: None 2022-09-03T06:45:43.493 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj51 2022-09-03T06:45:43.495 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V93028iGs/8XOHiXwGplDWGtXX0= 2022-09-03T06:45:43.496 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V93028iGs/8XOHiXwGplDWGtXX0=', 'Content-Length': '0'} 2022-09-03T06:45:43.499 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000071b41252836f4038-006312f817-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.500 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:43.501 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:43.502 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:43.503 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:43.504 DEBUG:boto:Method: GET 2022-09-03T06:45:43.505 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:45:43.506 DEBUG:boto:Data: 2022-09-03T06:45:43.507 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.508 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:43.508 DEBUG:boto:Port: 8001 2022-09-03T06:45:43.509 DEBUG:boto:Params: {} 2022-09-03T06:45:43.510 DEBUG:boto:Token: None 2022-09-03T06:45:43.511 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/ 2022-09-03T06:45:43.512 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng= 2022-09-03T06:45:43.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng=', 'Content-Length': '0'} 2022-09-03T06:45:43.522 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a9d6b61577405e1d-006312f817-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.558 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:43.562 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:43.563 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:43.564 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:43.564 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:45:43.565 DEBUG:boto:Method: GET 2022-09-03T06:45:43.565 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:45:43.566 DEBUG:boto:Data: 2022-09-03T06:45:43.567 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.567 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:43.568 DEBUG:boto:Port: 8000 2022-09-03T06:45:43.568 DEBUG:boto:Params: {} 2022-09-03T06:45:43.569 DEBUG:boto:Token: None 2022-09-03T06:45:43.570 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/ 2022-09-03T06:45:43.571 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng= 2022-09-03T06:45:43.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng=', 'Content-Length': '0'} 2022-09-03T06:45:43.575 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b94718d2f043f13a-006312f817-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.576 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:43.577 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:45:43.578 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:43.579 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:43.580 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.581 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.582 DEBUG:boto:Method: GET 2022-09-03T06:45:43.583 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.584 DEBUG:boto:Data: 2022-09-03T06:45:43.584 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.585 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:43.585 DEBUG:boto:Port: 8001 2022-09-03T06:45:43.586 DEBUG:boto:Params: {} 2022-09-03T06:45:43.587 DEBUG:boto:Token: None 2022-09-03T06:45:43.588 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.589 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MiQOqfR8ztsYmPRQnuQa/X4b1Z0= 2022-09-03T06:45:43.589 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MiQOqfR8ztsYmPRQnuQa/X4b1Z0=', 'Content-Length': '0'} 2022-09-03T06:45:43.593 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a69c48c7226d969c-006312f817-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.594 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:43.595 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:43.596 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.598 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.599 DEBUG:boto:Method: GET 2022-09-03T06:45:43.599 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.600 DEBUG:boto:Data: 2022-09-03T06:45:43.600 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.601 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:43.602 DEBUG:boto:Port: 8000 2022-09-03T06:45:43.602 DEBUG:boto:Params: {} 2022-09-03T06:45:43.603 DEBUG:boto:Token: None 2022-09-03T06:45:43.604 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.604 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MiQOqfR8ztsYmPRQnuQa/X4b1Z0= 2022-09-03T06:45:43.605 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MiQOqfR8ztsYmPRQnuQa/X4b1Z0=', 'Content-Length': '0'} 2022-09-03T06:45:43.608 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000056dc3a764d94e972-006312f817-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.609 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:43.609 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:43.610 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.611 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.611 DEBUG:boto:Method: HEAD 2022-09-03T06:45:43.612 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.613 DEBUG:boto:Data: 2022-09-03T06:45:43.614 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.615 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:43.616 DEBUG:boto:Port: 8001 2022-09-03T06:45:43.616 DEBUG:boto:Params: {} 2022-09-03T06:45:43.617 DEBUG:boto:Token: None 2022-09-03T06:45:43.617 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.618 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qtkeHwA6nccyPPDRtP2sTmED4KA= 2022-09-03T06:45:43.619 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qtkeHwA6nccyPPDRtP2sTmED4KA=', 'Content-Length': '0'} 2022-09-03T06:45:43.622 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d1c01e6bf122148-006312f817-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.623 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:43.624 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:43.625 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.626 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.626 DEBUG:boto:Method: HEAD 2022-09-03T06:45:43.627 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.628 DEBUG:boto:Data: 2022-09-03T06:45:43.629 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.630 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:43.631 DEBUG:boto:Port: 8000 2022-09-03T06:45:43.632 DEBUG:boto:Params: {} 2022-09-03T06:45:43.632 DEBUG:boto:Token: None 2022-09-03T06:45:43.633 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.634 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qtkeHwA6nccyPPDRtP2sTmED4KA= 2022-09-03T06:45:43.635 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qtkeHwA6nccyPPDRtP2sTmED4KA=', 'Content-Length': '0'} 2022-09-03T06:45:43.639 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000de91d46075d83a2e-006312f817-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.640 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:45:43.641 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:43.643 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:43.643 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.644 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.645 DEBUG:boto:Method: GET 2022-09-03T06:45:43.645 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.646 DEBUG:boto:Data: 2022-09-03T06:45:43.646 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.647 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:43.647 DEBUG:boto:Port: 8001 2022-09-03T06:45:43.648 DEBUG:boto:Params: {} 2022-09-03T06:45:43.649 DEBUG:boto:Token: None 2022-09-03T06:45:43.649 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.650 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MiQOqfR8ztsYmPRQnuQa/X4b1Z0= 2022-09-03T06:45:43.651 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MiQOqfR8ztsYmPRQnuQa/X4b1Z0=', 'Content-Length': '0'} 2022-09-03T06:45:43.654 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000278bbb786f538631-006312f817-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.656 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:43.657 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:43.658 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.659 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.660 DEBUG:boto:Method: GET 2022-09-03T06:45:43.661 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.662 DEBUG:boto:Data: 2022-09-03T06:45:43.663 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.664 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:43.665 DEBUG:boto:Port: 8000 2022-09-03T06:45:43.666 DEBUG:boto:Params: {} 2022-09-03T06:45:43.666 DEBUG:boto:Token: None 2022-09-03T06:45:43.667 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:45:43.668 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MiQOqfR8ztsYmPRQnuQa/X4b1Z0= 2022-09-03T06:45:43.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MiQOqfR8ztsYmPRQnuQa/X4b1Z0=', 'Content-Length': '0'} 2022-09-03T06:45:43.671 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e7b1a3295fdae4a4-006312f817-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.714 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:43.716 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:43.716 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:43.717 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:43.718 DEBUG:boto:Method: GET 2022-09-03T06:45:43.718 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:45:43.722 DEBUG:boto:Data: 2022-09-03T06:45:43.723 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.723 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:43.724 DEBUG:boto:Port: 8001 2022-09-03T06:45:43.724 DEBUG:boto:Params: {} 2022-09-03T06:45:43.725 DEBUG:boto:Token: None 2022-09-03T06:45:43.726 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/ 2022-09-03T06:45:43.726 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng= 2022-09-03T06:45:43.727 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng=', 'Content-Length': '0'} 2022-09-03T06:45:43.730 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000087bdfa19f3e7f04b-006312f817-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.731 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:43.732 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:43.733 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:43.733 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:43.734 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:43.735 DEBUG:boto:Method: GET 2022-09-03T06:45:43.735 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:45:43.736 DEBUG:boto:Data: 2022-09-03T06:45:43.736 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.737 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:43.737 DEBUG:boto:Port: 8000 2022-09-03T06:45:43.738 DEBUG:boto:Params: {} 2022-09-03T06:45:43.739 DEBUG:boto:Token: None 2022-09-03T06:45:43.739 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/ 2022-09-03T06:45:43.740 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng= 2022-09-03T06:45:43.741 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng=', 'Content-Length': '0'} 2022-09-03T06:45:43.744 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000986fca9a45000810-006312f817-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.745 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:43.747 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:43.748 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:43.749 DEBUG:boto:Method: HEAD 2022-09-03T06:45:43.750 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:45:43.750 DEBUG:boto:Data: 2022-09-03T06:45:43.751 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.751 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:43.752 DEBUG:boto:Port: 8001 2022-09-03T06:45:43.752 DEBUG:boto:Params: {} 2022-09-03T06:45:43.753 DEBUG:boto:Token: None 2022-09-03T06:45:43.753 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj55 2022-09-03T06:45:43.754 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pw1bH92rOi4I+MWR5b5+X2708i8= 2022-09-03T06:45:43.755 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pw1bH92rOi4I+MWR5b5+X2708i8=', 'Content-Length': '0'} 2022-09-03T06:45:43.757 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bb32f48c2fe16352-006312f817-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.758 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:43.759 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:43.759 DEBUG:boto:Method: HEAD 2022-09-03T06:45:43.760 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:45:43.760 DEBUG:boto:Data: 2022-09-03T06:45:43.761 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.761 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:43.762 DEBUG:boto:Port: 8000 2022-09-03T06:45:43.763 DEBUG:boto:Params: {} 2022-09-03T06:45:43.763 DEBUG:boto:Token: None 2022-09-03T06:45:43.764 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj55 2022-09-03T06:45:43.765 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pw1bH92rOi4I+MWR5b5+X2708i8= 2022-09-03T06:45:43.765 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pw1bH92rOi4I+MWR5b5+X2708i8=', 'Content-Length': '0'} 2022-09-03T06:45:43.768 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cfb5e2c61da244c7-006312f817-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.768 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:45:43.769 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:43.770 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:43.770 DEBUG:boto:Method: GET 2022-09-03T06:45:43.771 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:45:43.772 DEBUG:boto:Data: 2022-09-03T06:45:43.772 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.773 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:43.774 DEBUG:boto:Port: 8001 2022-09-03T06:45:43.775 DEBUG:boto:Params: {} 2022-09-03T06:45:43.776 DEBUG:boto:Token: None 2022-09-03T06:45:43.776 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj55 2022-09-03T06:45:43.777 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sxIouuZdXQ+/BmciPb3KRm4nD7k= 2022-09-03T06:45:43.778 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sxIouuZdXQ+/BmciPb3KRm4nD7k=', 'Content-Length': '0'} 2022-09-03T06:45:43.780 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000071c60c28bb7a3bcc-006312f817-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.781 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:45:43.782 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:45:43.783 DEBUG:boto:Method: GET 2022-09-03T06:45:43.784 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:45:43.785 DEBUG:boto:Data: 2022-09-03T06:45:43.786 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.787 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:43.788 DEBUG:boto:Port: 8000 2022-09-03T06:45:43.789 DEBUG:boto:Params: {} 2022-09-03T06:45:43.790 DEBUG:boto:Token: None 2022-09-03T06:45:43.791 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj55 2022-09-03T06:45:43.792 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sxIouuZdXQ+/BmciPb3KRm4nD7k= 2022-09-03T06:45:43.793 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sxIouuZdXQ+/BmciPb3KRm4nD7k=', 'Content-Length': '0'} 2022-09-03T06:45:43.796 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ba06e45b4cdba0f4-006312f817-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.798 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:43.799 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:43.801 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:43.801 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:43.802 DEBUG:boto:Method: GET 2022-09-03T06:45:43.803 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:45:43.804 DEBUG:boto:Data: 2022-09-03T06:45:43.804 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.806 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:43.806 DEBUG:boto:Port: 8001 2022-09-03T06:45:43.807 DEBUG:boto:Params: {} 2022-09-03T06:45:43.808 DEBUG:boto:Token: None 2022-09-03T06:45:43.809 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/ 2022-09-03T06:45:43.809 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng= 2022-09-03T06:45:43.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng=', 'Content-Length': '0'} 2022-09-03T06:45:43.814 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e38fc22057e838cd-006312f817-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.815 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:43.817 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:43.817 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:43.818 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:43.818 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:45:43.819 DEBUG:boto:Method: GET 2022-09-03T06:45:43.820 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:45:43.820 DEBUG:boto:Data: 2022-09-03T06:45:43.821 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.822 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:43.823 DEBUG:boto:Port: 8000 2022-09-03T06:45:43.824 DEBUG:boto:Params: {} 2022-09-03T06:45:43.825 DEBUG:boto:Token: None 2022-09-03T06:45:43.826 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/ 2022-09-03T06:45:43.827 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng= 2022-09-03T06:45:43.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng=', 'Content-Length': '0'} 2022-09-03T06:45:43.831 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ac1cf0470f548741-006312f817-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.874 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:43.875 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:45:43.877 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:43.877 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:43.878 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.878 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.879 DEBUG:boto:Method: GET 2022-09-03T06:45:43.879 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.882 DEBUG:boto:Data: 2022-09-03T06:45:43.882 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.883 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:43.884 DEBUG:boto:Port: 8001 2022-09-03T06:45:43.884 DEBUG:boto:Params: {} 2022-09-03T06:45:43.885 DEBUG:boto:Token: None 2022-09-03T06:45:43.885 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.886 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VG7NNOkMDlm09lzOx8iOTPXSPkI= 2022-09-03T06:45:43.887 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VG7NNOkMDlm09lzOx8iOTPXSPkI=', 'Content-Length': '0'} 2022-09-03T06:45:43.889 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000044eccc0418da5273-006312f817-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.890 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:43.890 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:43.891 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.891 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.892 DEBUG:boto:Method: GET 2022-09-03T06:45:43.893 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.893 DEBUG:boto:Data: 2022-09-03T06:45:43.894 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.895 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:43.895 DEBUG:boto:Port: 8000 2022-09-03T06:45:43.896 DEBUG:boto:Params: {} 2022-09-03T06:45:43.897 DEBUG:boto:Token: None 2022-09-03T06:45:43.898 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.899 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VG7NNOkMDlm09lzOx8iOTPXSPkI= 2022-09-03T06:45:43.900 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VG7NNOkMDlm09lzOx8iOTPXSPkI=', 'Content-Length': '0'} 2022-09-03T06:45:43.902 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000069c0ef3dcc3354fe-006312f817-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.903 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:43.903 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:43.904 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.905 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.905 DEBUG:boto:Method: HEAD 2022-09-03T06:45:43.906 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.906 DEBUG:boto:Data: 2022-09-03T06:45:43.907 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.907 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:43.908 DEBUG:boto:Port: 8001 2022-09-03T06:45:43.908 DEBUG:boto:Params: {} 2022-09-03T06:45:43.909 DEBUG:boto:Token: None 2022-09-03T06:45:43.910 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.911 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zCVHj+XXBirdTagj8IGqROsxxrs= 2022-09-03T06:45:43.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zCVHj+XXBirdTagj8IGqROsxxrs=', 'Content-Length': '0'} 2022-09-03T06:45:43.914 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000437df6f6f7cb1439-006312f817-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.915 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:43.915 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:43.916 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.916 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.917 DEBUG:boto:Method: HEAD 2022-09-03T06:45:43.918 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.918 DEBUG:boto:Data: 2022-09-03T06:45:43.919 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.919 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:43.920 DEBUG:boto:Port: 8000 2022-09-03T06:45:43.920 DEBUG:boto:Params: {} 2022-09-03T06:45:43.921 DEBUG:boto:Token: None 2022-09-03T06:45:43.922 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.922 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zCVHj+XXBirdTagj8IGqROsxxrs= 2022-09-03T06:45:43.925 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zCVHj+XXBirdTagj8IGqROsxxrs=', 'Content-Length': '0'} 2022-09-03T06:45:43.928 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eaf7299065bab86b-006312f817-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.929 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:45:43.930 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:43.931 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:43.932 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.933 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.934 DEBUG:boto:Method: GET 2022-09-03T06:45:43.935 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.937 DEBUG:boto:Data: 2022-09-03T06:45:43.942 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.943 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:43.944 DEBUG:boto:Port: 8001 2022-09-03T06:45:43.945 DEBUG:boto:Params: {} 2022-09-03T06:45:43.946 DEBUG:boto:Token: None 2022-09-03T06:45:43.947 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.948 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VG7NNOkMDlm09lzOx8iOTPXSPkI= 2022-09-03T06:45:43.949 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VG7NNOkMDlm09lzOx8iOTPXSPkI=', 'Content-Length': '0'} 2022-09-03T06:45:43.952 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000289c219222501727-006312f817-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.955 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:43.956 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:43.958 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.959 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.959 DEBUG:boto:Method: GET 2022-09-03T06:45:43.960 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.960 DEBUG:boto:Data: 2022-09-03T06:45:43.961 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.961 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:43.962 DEBUG:boto:Port: 8000 2022-09-03T06:45:43.963 DEBUG:boto:Params: {} 2022-09-03T06:45:43.964 DEBUG:boto:Token: None 2022-09-03T06:45:43.965 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:45:43.966 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VG7NNOkMDlm09lzOx8iOTPXSPkI= 2022-09-03T06:45:43.966 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VG7NNOkMDlm09lzOx8iOTPXSPkI=', 'Content-Length': '0'} 2022-09-03T06:45:43.969 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ede1344a80df032c-006312f817-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.970 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:43.970 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:43.971 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:43.972 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:43.973 DEBUG:boto:Method: GET 2022-09-03T06:45:43.973 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:45:43.974 DEBUG:boto:Data: 2022-09-03T06:45:43.975 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.975 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:43.976 DEBUG:boto:Port: 8001 2022-09-03T06:45:43.976 DEBUG:boto:Params: {} 2022-09-03T06:45:43.977 DEBUG:boto:Token: None 2022-09-03T06:45:43.978 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/ 2022-09-03T06:45:43.978 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng= 2022-09-03T06:45:43.979 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng=', 'Content-Length': '0'} 2022-09-03T06:45:43.982 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fe75056def9eb2f0-006312f817-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:43.983 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:43.985 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:43.987 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:43.988 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:43.988 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:43.989 DEBUG:boto:Method: GET 2022-09-03T06:45:43.990 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:45:43.990 DEBUG:boto:Data: 2022-09-03T06:45:43.991 DEBUG:boto:Headers: {} 2022-09-03T06:45:43.992 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:43.992 DEBUG:boto:Port: 8000 2022-09-03T06:45:43.993 DEBUG:boto:Params: {} 2022-09-03T06:45:43.996 DEBUG:boto:Token: None 2022-09-03T06:45:43.998 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:43 GMT /dsgvcy-75/ 2022-09-03T06:45:44.000 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng= 2022-09-03T06:45:44.001 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PQ+uLKKwrq9YCOILm0JTAYwcRng=', 'Content-Length': '0'} 2022-09-03T06:45:44.004 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009c33d4ec2cf2fda1-006312f818-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.005 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:44.006 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:44.007 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:44.007 DEBUG:boto:Method: HEAD 2022-09-03T06:45:44.008 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:45:44.009 DEBUG:boto:Data: 2022-09-03T06:45:44.009 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.010 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.010 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.011 DEBUG:boto:Params: {} 2022-09-03T06:45:44.011 DEBUG:boto:Token: None 2022-09-03T06:45:44.012 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj57 2022-09-03T06:45:44.013 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fERTVckY8zdcqaBInE4o0i8BKRc= 2022-09-03T06:45:44.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fERTVckY8zdcqaBInE4o0i8BKRc=', 'Content-Length': '0'} 2022-09-03T06:45:44.015 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000879538ac8d24fdfa-006312f818-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.017 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:44.017 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:44.018 DEBUG:boto:Method: HEAD 2022-09-03T06:45:44.019 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:45:44.019 DEBUG:boto:Data: 2022-09-03T06:45:44.020 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.020 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.021 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.021 DEBUG:boto:Params: {} 2022-09-03T06:45:44.022 DEBUG:boto:Token: None 2022-09-03T06:45:44.023 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj57 2022-09-03T06:45:44.023 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fERTVckY8zdcqaBInE4o0i8BKRc= 2022-09-03T06:45:44.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fERTVckY8zdcqaBInE4o0i8BKRc=', 'Content-Length': '0'} 2022-09-03T06:45:44.026 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000116475da6f593426-006312f818-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.027 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:45:44.028 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:44.029 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:44.030 DEBUG:boto:Method: GET 2022-09-03T06:45:44.031 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:45:44.031 DEBUG:boto:Data: 2022-09-03T06:45:44.032 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.032 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.033 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.034 DEBUG:boto:Params: {} 2022-09-03T06:45:44.035 DEBUG:boto:Token: None 2022-09-03T06:45:44.035 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj57 2022-09-03T06:45:44.036 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hpv4YUdnmYXifVo4gWYwMxp1zmY= 2022-09-03T06:45:44.037 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hpv4YUdnmYXifVo4gWYwMxp1zmY=', 'Content-Length': '0'} 2022-09-03T06:45:44.040 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000066a2fe769355e346-006312f818-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.084 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:45:44.086 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:45:44.087 DEBUG:boto:Method: GET 2022-09-03T06:45:44.088 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:45:44.089 DEBUG:boto:Data: 2022-09-03T06:45:44.090 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.091 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.092 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.093 DEBUG:boto:Params: {} 2022-09-03T06:45:44.094 DEBUG:boto:Token: None 2022-09-03T06:45:44.095 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj57 2022-09-03T06:45:44.096 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hpv4YUdnmYXifVo4gWYwMxp1zmY= 2022-09-03T06:45:44.097 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hpv4YUdnmYXifVo4gWYwMxp1zmY=', 'Content-Length': '0'} 2022-09-03T06:45:44.101 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005755463fe19a6ead-006312f818-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.102 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:44.103 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:44.104 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:44.105 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:44.106 DEBUG:boto:Method: GET 2022-09-03T06:45:44.107 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:45:44.108 DEBUG:boto:Data: 2022-09-03T06:45:44.109 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.110 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.111 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.112 DEBUG:boto:Params: {} 2022-09-03T06:45:44.113 DEBUG:boto:Token: None 2022-09-03T06:45:44.115 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/ 2022-09-03T06:45:44.116 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8= 2022-09-03T06:45:44.117 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8=', 'Content-Length': '0'} 2022-09-03T06:45:44.120 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000031b9b9db69cbc240-006312f818-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.121 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:44.123 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:44.124 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:44.125 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:44.126 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:45:44.127 DEBUG:boto:Method: GET 2022-09-03T06:45:44.128 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:45:44.130 DEBUG:boto:Data: 2022-09-03T06:45:44.131 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.133 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.134 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.134 DEBUG:boto:Params: {} 2022-09-03T06:45:44.135 DEBUG:boto:Token: None 2022-09-03T06:45:44.136 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/ 2022-09-03T06:45:44.136 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8= 2022-09-03T06:45:44.138 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8=', 'Content-Length': '0'} 2022-09-03T06:45:44.142 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d7277b4a65843037-006312f818-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.143 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:44.144 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:45:44.145 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:44.146 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:44.146 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.147 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.148 DEBUG:boto:Method: GET 2022-09-03T06:45:44.148 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.149 DEBUG:boto:Data: 2022-09-03T06:45:44.149 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.150 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.150 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.151 DEBUG:boto:Params: {} 2022-09-03T06:45:44.152 DEBUG:boto:Token: None 2022-09-03T06:45:44.152 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.154 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4VLPBKnbfCoG9aJ7uDi7hbxvyBE= 2022-09-03T06:45:44.155 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4VLPBKnbfCoG9aJ7uDi7hbxvyBE=', 'Content-Length': '0'} 2022-09-03T06:45:44.157 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003485bded1ed9e6ad-006312f818-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.158 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:44.159 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:44.159 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.160 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.161 DEBUG:boto:Method: GET 2022-09-03T06:45:44.161 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.162 DEBUG:boto:Data: 2022-09-03T06:45:44.163 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.163 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.164 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.165 DEBUG:boto:Params: {} 2022-09-03T06:45:44.166 DEBUG:boto:Token: None 2022-09-03T06:45:44.167 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.168 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4VLPBKnbfCoG9aJ7uDi7hbxvyBE= 2022-09-03T06:45:44.168 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4VLPBKnbfCoG9aJ7uDi7hbxvyBE=', 'Content-Length': '0'} 2022-09-03T06:45:44.171 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000388e6105b4cedb0b-006312f818-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.172 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:44.173 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:44.175 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.175 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.176 DEBUG:boto:Method: HEAD 2022-09-03T06:45:44.177 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.177 DEBUG:boto:Data: 2022-09-03T06:45:44.178 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.179 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.179 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.180 DEBUG:boto:Params: {} 2022-09-03T06:45:44.181 DEBUG:boto:Token: None 2022-09-03T06:45:44.181 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.182 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wkVIxKal5QMbzk8O0MxY1MQBBiE= 2022-09-03T06:45:44.182 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wkVIxKal5QMbzk8O0MxY1MQBBiE=', 'Content-Length': '0'} 2022-09-03T06:45:44.188 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003cc6e9d5db28db7f-006312f818-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.189 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:44.192 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:44.193 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.193 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.194 DEBUG:boto:Method: HEAD 2022-09-03T06:45:44.196 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.196 DEBUG:boto:Data: 2022-09-03T06:45:44.197 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.198 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.198 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.199 DEBUG:boto:Params: {} 2022-09-03T06:45:44.199 DEBUG:boto:Token: None 2022-09-03T06:45:44.200 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.200 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wkVIxKal5QMbzk8O0MxY1MQBBiE= 2022-09-03T06:45:44.203 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wkVIxKal5QMbzk8O0MxY1MQBBiE=', 'Content-Length': '0'} 2022-09-03T06:45:44.207 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cf53bb58c71617e1-006312f818-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.210 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:45:44.211 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:44.212 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:44.213 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.214 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.214 DEBUG:boto:Method: GET 2022-09-03T06:45:44.215 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.215 DEBUG:boto:Data: 2022-09-03T06:45:44.216 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.216 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.217 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.217 DEBUG:boto:Params: {} 2022-09-03T06:45:44.218 DEBUG:boto:Token: None 2022-09-03T06:45:44.219 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.219 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4VLPBKnbfCoG9aJ7uDi7hbxvyBE= 2022-09-03T06:45:44.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4VLPBKnbfCoG9aJ7uDi7hbxvyBE=', 'Content-Length': '0'} 2022-09-03T06:45:44.223 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd6d25b49bdb54d0-006312f818-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.225 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:44.226 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:44.226 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.227 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.227 DEBUG:boto:Method: GET 2022-09-03T06:45:44.228 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.229 DEBUG:boto:Data: 2022-09-03T06:45:44.229 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.230 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.231 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.232 DEBUG:boto:Params: {} 2022-09-03T06:45:44.233 DEBUG:boto:Token: None 2022-09-03T06:45:44.234 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:45:44.236 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4VLPBKnbfCoG9aJ7uDi7hbxvyBE= 2022-09-03T06:45:44.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4VLPBKnbfCoG9aJ7uDi7hbxvyBE=', 'Content-Length': '0'} 2022-09-03T06:45:44.240 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000497a02da2cbbb2fa-006312f818-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.291 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:44.292 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:44.293 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:44.294 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:44.295 DEBUG:boto:Method: GET 2022-09-03T06:45:44.296 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:45:44.297 DEBUG:boto:Data: 2022-09-03T06:45:44.299 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.300 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.301 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.302 DEBUG:boto:Params: {} 2022-09-03T06:45:44.303 DEBUG:boto:Token: None 2022-09-03T06:45:44.304 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/ 2022-09-03T06:45:44.305 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8= 2022-09-03T06:45:44.306 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8=', 'Content-Length': '0'} 2022-09-03T06:45:44.310 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aa9f1bfbbd528566-006312f818-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.311 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:44.313 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:44.314 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:44.316 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:44.317 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:44.320 DEBUG:boto:Method: GET 2022-09-03T06:45:44.321 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:45:44.322 DEBUG:boto:Data: 2022-09-03T06:45:44.322 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.323 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.323 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.324 DEBUG:boto:Params: {} 2022-09-03T06:45:44.324 DEBUG:boto:Token: None 2022-09-03T06:45:44.325 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/ 2022-09-03T06:45:44.326 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8= 2022-09-03T06:45:44.326 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8=', 'Content-Length': '0'} 2022-09-03T06:45:44.331 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000005c76d08b8712588-006312f818-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.331 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:44.332 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:44.333 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:44.334 DEBUG:boto:Method: HEAD 2022-09-03T06:45:44.334 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:45:44.335 DEBUG:boto:Data: 2022-09-03T06:45:44.335 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.336 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.336 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.337 DEBUG:boto:Params: {} 2022-09-03T06:45:44.338 DEBUG:boto:Token: None 2022-09-03T06:45:44.338 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj58 2022-09-03T06:45:44.339 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mjv93pJfwf/i337Yycp40R1zNDk= 2022-09-03T06:45:44.339 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mjv93pJfwf/i337Yycp40R1zNDk=', 'Content-Length': '0'} 2022-09-03T06:45:44.342 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cd90f372187d184c-006312f818-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.343 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:44.344 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:44.344 DEBUG:boto:Method: HEAD 2022-09-03T06:45:44.345 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:45:44.345 DEBUG:boto:Data: 2022-09-03T06:45:44.346 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.346 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.347 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.347 DEBUG:boto:Params: {} 2022-09-03T06:45:44.348 DEBUG:boto:Token: None 2022-09-03T06:45:44.349 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj58 2022-09-03T06:45:44.349 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mjv93pJfwf/i337Yycp40R1zNDk= 2022-09-03T06:45:44.350 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mjv93pJfwf/i337Yycp40R1zNDk=', 'Content-Length': '0'} 2022-09-03T06:45:44.352 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b681cfafaecfe145-006312f818-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.353 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:45:44.353 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:44.354 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:44.355 DEBUG:boto:Method: GET 2022-09-03T06:45:44.355 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:45:44.356 DEBUG:boto:Data: 2022-09-03T06:45:44.356 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.357 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.357 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.358 DEBUG:boto:Params: {} 2022-09-03T06:45:44.359 DEBUG:boto:Token: None 2022-09-03T06:45:44.359 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj58 2022-09-03T06:45:44.360 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b6x/Mg6eJoLJHJpkvdqT0bbpmjI= 2022-09-03T06:45:44.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b6x/Mg6eJoLJHJpkvdqT0bbpmjI=', 'Content-Length': '0'} 2022-09-03T06:45:44.363 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b6abe82ac598fa5-006312f818-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.364 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:45:44.364 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:45:44.370 DEBUG:boto:Method: GET 2022-09-03T06:45:44.371 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:45:44.372 DEBUG:boto:Data: 2022-09-03T06:45:44.372 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.373 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.375 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.376 DEBUG:boto:Params: {} 2022-09-03T06:45:44.376 DEBUG:boto:Token: None 2022-09-03T06:45:44.377 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj58 2022-09-03T06:45:44.378 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b6x/Mg6eJoLJHJpkvdqT0bbpmjI= 2022-09-03T06:45:44.378 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b6x/Mg6eJoLJHJpkvdqT0bbpmjI=', 'Content-Length': '0'} 2022-09-03T06:45:44.381 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000058170f34c373e546-006312f818-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.381 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:44.382 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:44.383 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:44.383 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:44.384 DEBUG:boto:Method: GET 2022-09-03T06:45:44.384 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:45:44.385 DEBUG:boto:Data: 2022-09-03T06:45:44.385 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.386 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.387 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.387 DEBUG:boto:Params: {} 2022-09-03T06:45:44.388 DEBUG:boto:Token: None 2022-09-03T06:45:44.388 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/ 2022-09-03T06:45:44.389 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8= 2022-09-03T06:45:44.389 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8=', 'Content-Length': '0'} 2022-09-03T06:45:44.392 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000008f93994a081ed47-006312f818-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.438 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:44.439 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:44.440 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:44.440 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:44.441 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:45:44.441 DEBUG:boto:Method: GET 2022-09-03T06:45:44.442 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:45:44.442 DEBUG:boto:Data: 2022-09-03T06:45:44.443 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.443 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.444 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.444 DEBUG:boto:Params: {} 2022-09-03T06:45:44.445 DEBUG:boto:Token: None 2022-09-03T06:45:44.446 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/ 2022-09-03T06:45:44.447 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8= 2022-09-03T06:45:44.448 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8=', 'Content-Length': '0'} 2022-09-03T06:45:44.451 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005d1f82f48b205e4a-006312f818-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.452 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:44.453 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:45:44.453 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:44.454 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:44.454 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.455 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.456 DEBUG:boto:Method: GET 2022-09-03T06:45:44.457 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.458 DEBUG:boto:Data: 2022-09-03T06:45:44.458 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.459 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.460 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.460 DEBUG:boto:Params: {} 2022-09-03T06:45:44.461 DEBUG:boto:Token: None 2022-09-03T06:45:44.462 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.463 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ph+k1mnqtdUYLe5ep1vp/cPl4/E= 2022-09-03T06:45:44.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ph+k1mnqtdUYLe5ep1vp/cPl4/E=', 'Content-Length': '0'} 2022-09-03T06:45:44.468 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a17171ee381a6b95-006312f818-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.470 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:44.471 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:44.472 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.473 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.474 DEBUG:boto:Method: GET 2022-09-03T06:45:44.475 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.475 DEBUG:boto:Data: 2022-09-03T06:45:44.476 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.478 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.479 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.480 DEBUG:boto:Params: {} 2022-09-03T06:45:44.481 DEBUG:boto:Token: None 2022-09-03T06:45:44.482 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.484 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ph+k1mnqtdUYLe5ep1vp/cPl4/E= 2022-09-03T06:45:44.485 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ph+k1mnqtdUYLe5ep1vp/cPl4/E=', 'Content-Length': '0'} 2022-09-03T06:45:44.489 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b3f810cd7650c9b1-006312f818-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.530 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:44.531 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:44.531 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.532 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.532 DEBUG:boto:Method: HEAD 2022-09-03T06:45:44.533 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.533 DEBUG:boto:Data: 2022-09-03T06:45:44.534 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.535 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.535 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.536 DEBUG:boto:Params: {} 2022-09-03T06:45:44.536 DEBUG:boto:Token: None 2022-09-03T06:45:44.537 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.538 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K2DScELR+lTwrXP2MT81KPD+xKY= 2022-09-03T06:45:44.539 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K2DScELR+lTwrXP2MT81KPD+xKY=', 'Content-Length': '0'} 2022-09-03T06:45:44.541 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c366e54c23224a1-006312f818-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.542 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:44.542 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:44.543 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.544 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.544 DEBUG:boto:Method: HEAD 2022-09-03T06:45:44.545 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.545 DEBUG:boto:Data: 2022-09-03T06:45:44.546 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.547 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.547 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.548 DEBUG:boto:Params: {} 2022-09-03T06:45:44.548 DEBUG:boto:Token: None 2022-09-03T06:45:44.549 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.550 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K2DScELR+lTwrXP2MT81KPD+xKY= 2022-09-03T06:45:44.550 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K2DScELR+lTwrXP2MT81KPD+xKY=', 'Content-Length': '0'} 2022-09-03T06:45:44.553 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000005c3c8a25a432c63-006312f818-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.553 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:45:44.554 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:44.555 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:44.555 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.556 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.557 DEBUG:boto:Method: GET 2022-09-03T06:45:44.558 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.558 DEBUG:boto:Data: 2022-09-03T06:45:44.559 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.559 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.560 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.560 DEBUG:boto:Params: {} 2022-09-03T06:45:44.562 DEBUG:boto:Token: None 2022-09-03T06:45:44.562 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.563 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ph+k1mnqtdUYLe5ep1vp/cPl4/E= 2022-09-03T06:45:44.564 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ph+k1mnqtdUYLe5ep1vp/cPl4/E=', 'Content-Length': '0'} 2022-09-03T06:45:44.566 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d45876b2a6597a9-006312f818-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.568 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:44.569 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:44.570 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.571 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.571 DEBUG:boto:Method: GET 2022-09-03T06:45:44.572 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.574 DEBUG:boto:Data: 2022-09-03T06:45:44.574 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.575 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.575 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.576 DEBUG:boto:Params: {} 2022-09-03T06:45:44.577 DEBUG:boto:Token: None 2022-09-03T06:45:44.577 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:45:44.578 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ph+k1mnqtdUYLe5ep1vp/cPl4/E= 2022-09-03T06:45:44.579 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ph+k1mnqtdUYLe5ep1vp/cPl4/E=', 'Content-Length': '0'} 2022-09-03T06:45:44.582 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c78cd629c05081ab-006312f818-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.583 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:44.583 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:44.584 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:44.585 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:44.585 DEBUG:boto:Method: GET 2022-09-03T06:45:44.586 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:45:44.586 DEBUG:boto:Data: 2022-09-03T06:45:44.587 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.589 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.589 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.590 DEBUG:boto:Params: {} 2022-09-03T06:45:44.591 DEBUG:boto:Token: None 2022-09-03T06:45:44.592 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/ 2022-09-03T06:45:44.592 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8= 2022-09-03T06:45:44.593 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8=', 'Content-Length': '0'} 2022-09-03T06:45:44.596 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001acaa31bf00ad739-006312f818-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.597 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:44.598 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:44.598 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:44.599 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:44.599 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:44.600 DEBUG:boto:Method: GET 2022-09-03T06:45:44.601 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:45:44.602 DEBUG:boto:Data: 2022-09-03T06:45:44.602 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.603 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.603 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.604 DEBUG:boto:Params: {} 2022-09-03T06:45:44.605 DEBUG:boto:Token: None 2022-09-03T06:45:44.605 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/ 2022-09-03T06:45:44.606 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8= 2022-09-03T06:45:44.607 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8=', 'Content-Length': '0'} 2022-09-03T06:45:44.609 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000085b44214e3edec1c-006312f818-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.610 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:44.611 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:44.612 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:44.613 DEBUG:boto:Method: HEAD 2022-09-03T06:45:44.613 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:45:44.614 DEBUG:boto:Data: 2022-09-03T06:45:44.614 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.615 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.615 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.616 DEBUG:boto:Params: {} 2022-09-03T06:45:44.617 DEBUG:boto:Token: None 2022-09-03T06:45:44.618 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj68 2022-09-03T06:45:44.619 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iHNX+z1nV222RatJgOHSfLQt2wM= 2022-09-03T06:45:44.619 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iHNX+z1nV222RatJgOHSfLQt2wM=', 'Content-Length': '0'} 2022-09-03T06:45:44.621 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000439d59369023f934-006312f818-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.623 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:44.623 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:44.624 DEBUG:boto:Method: HEAD 2022-09-03T06:45:44.624 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:45:44.625 DEBUG:boto:Data: 2022-09-03T06:45:44.626 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.628 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.629 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.630 DEBUG:boto:Params: {} 2022-09-03T06:45:44.631 DEBUG:boto:Token: None 2022-09-03T06:45:44.631 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj68 2022-09-03T06:45:44.636 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iHNX+z1nV222RatJgOHSfLQt2wM= 2022-09-03T06:45:44.637 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iHNX+z1nV222RatJgOHSfLQt2wM=', 'Content-Length': '0'} 2022-09-03T06:45:44.640 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2cca12be5a2df4f-006312f818-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.641 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:45:44.642 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:44.643 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:44.643 DEBUG:boto:Method: GET 2022-09-03T06:45:44.644 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:45:44.645 DEBUG:boto:Data: 2022-09-03T06:45:44.645 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.646 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.647 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.647 DEBUG:boto:Params: {} 2022-09-03T06:45:44.648 DEBUG:boto:Token: None 2022-09-03T06:45:44.649 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj68 2022-09-03T06:45:44.649 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cvafEi9T+Kv2TJ3+0ZRh6xVa7Iw= 2022-09-03T06:45:44.650 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cvafEi9T+Kv2TJ3+0ZRh6xVa7Iw=', 'Content-Length': '0'} 2022-09-03T06:45:44.652 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000825d04bd485266e4-006312f818-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.653 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:45:44.654 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:45:44.654 DEBUG:boto:Method: GET 2022-09-03T06:45:44.655 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:45:44.656 DEBUG:boto:Data: 2022-09-03T06:45:44.656 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.657 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.658 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.658 DEBUG:boto:Params: {} 2022-09-03T06:45:44.659 DEBUG:boto:Token: None 2022-09-03T06:45:44.659 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj68 2022-09-03T06:45:44.660 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cvafEi9T+Kv2TJ3+0ZRh6xVa7Iw= 2022-09-03T06:45:44.661 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cvafEi9T+Kv2TJ3+0ZRh6xVa7Iw=', 'Content-Length': '0'} 2022-09-03T06:45:44.667 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000079b6af8b2aa4e5ea-006312f818-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.668 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:44.669 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:44.669 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:44.670 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:44.671 DEBUG:boto:Method: GET 2022-09-03T06:45:44.671 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:45:44.672 DEBUG:boto:Data: 2022-09-03T06:45:44.673 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.673 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.674 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.675 DEBUG:boto:Params: {} 2022-09-03T06:45:44.675 DEBUG:boto:Token: None 2022-09-03T06:45:44.676 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/ 2022-09-03T06:45:44.678 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8= 2022-09-03T06:45:44.678 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8=', 'Content-Length': '0'} 2022-09-03T06:45:44.681 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a64980f62e3fe0dd-006312f818-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.682 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:44.683 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:44.684 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:44.684 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:44.685 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:45:44.686 DEBUG:boto:Method: GET 2022-09-03T06:45:44.686 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:45:44.687 DEBUG:boto:Data: 2022-09-03T06:45:44.687 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.688 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.689 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.689 DEBUG:boto:Params: {} 2022-09-03T06:45:44.690 DEBUG:boto:Token: None 2022-09-03T06:45:44.691 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/ 2022-09-03T06:45:44.691 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8= 2022-09-03T06:45:44.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8=', 'Content-Length': '0'} 2022-09-03T06:45:44.695 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001c92a4714c89fc44-006312f818-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.738 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:44.739 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:45:44.740 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:44.741 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:44.741 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.742 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.742 DEBUG:boto:Method: GET 2022-09-03T06:45:44.743 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.744 DEBUG:boto:Data: 2022-09-03T06:45:44.744 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.745 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.745 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.746 DEBUG:boto:Params: {} 2022-09-03T06:45:44.747 DEBUG:boto:Token: None 2022-09-03T06:45:44.747 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.748 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WCtoS66Yb5nsjbYak2pOndBeBZk= 2022-09-03T06:45:44.749 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WCtoS66Yb5nsjbYak2pOndBeBZk=', 'Content-Length': '0'} 2022-09-03T06:45:44.751 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000759f6c29146fffbe-006312f818-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.752 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:44.753 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:44.753 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.754 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.754 DEBUG:boto:Method: GET 2022-09-03T06:45:44.755 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.756 DEBUG:boto:Data: 2022-09-03T06:45:44.756 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.757 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.757 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.758 DEBUG:boto:Params: {} 2022-09-03T06:45:44.759 DEBUG:boto:Token: None 2022-09-03T06:45:44.760 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.761 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WCtoS66Yb5nsjbYak2pOndBeBZk= 2022-09-03T06:45:44.761 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WCtoS66Yb5nsjbYak2pOndBeBZk=', 'Content-Length': '0'} 2022-09-03T06:45:44.764 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002be3fe4b5b08d4a4-006312f818-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.765 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:44.766 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:44.766 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.767 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.768 DEBUG:boto:Method: HEAD 2022-09-03T06:45:44.768 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.769 DEBUG:boto:Data: 2022-09-03T06:45:44.769 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.770 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.770 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.772 DEBUG:boto:Params: {} 2022-09-03T06:45:44.772 DEBUG:boto:Token: None 2022-09-03T06:45:44.773 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.774 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z4jJ57ThfPRaypaeA2lEe+2zWvA= 2022-09-03T06:45:44.774 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z4jJ57ThfPRaypaeA2lEe+2zWvA=', 'Content-Length': '0'} 2022-09-03T06:45:44.776 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f375ac2917208ab6-006312f818-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.777 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:44.778 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:44.779 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.779 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.780 DEBUG:boto:Method: HEAD 2022-09-03T06:45:44.781 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.781 DEBUG:boto:Data: 2022-09-03T06:45:44.782 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.783 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.783 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.784 DEBUG:boto:Params: {} 2022-09-03T06:45:44.785 DEBUG:boto:Token: None 2022-09-03T06:45:44.785 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.786 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z4jJ57ThfPRaypaeA2lEe+2zWvA= 2022-09-03T06:45:44.787 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z4jJ57ThfPRaypaeA2lEe+2zWvA=', 'Content-Length': '0'} 2022-09-03T06:45:44.790 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000127b5c1ef7ccd291-006312f818-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.791 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:45:44.792 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:44.793 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:44.794 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.795 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.795 DEBUG:boto:Method: GET 2022-09-03T06:45:44.796 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.796 DEBUG:boto:Data: 2022-09-03T06:45:44.797 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.797 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.798 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.799 DEBUG:boto:Params: {} 2022-09-03T06:45:44.799 DEBUG:boto:Token: None 2022-09-03T06:45:44.800 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.801 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WCtoS66Yb5nsjbYak2pOndBeBZk= 2022-09-03T06:45:44.801 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WCtoS66Yb5nsjbYak2pOndBeBZk=', 'Content-Length': '0'} 2022-09-03T06:45:44.804 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fc841e18199b11bc-006312f818-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.807 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:44.808 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:44.809 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.810 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.811 DEBUG:boto:Method: GET 2022-09-03T06:45:44.812 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.813 DEBUG:boto:Data: 2022-09-03T06:45:44.814 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.815 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.816 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.816 DEBUG:boto:Params: {} 2022-09-03T06:45:44.818 DEBUG:boto:Token: None 2022-09-03T06:45:44.819 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:45:44.820 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WCtoS66Yb5nsjbYak2pOndBeBZk= 2022-09-03T06:45:44.820 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WCtoS66Yb5nsjbYak2pOndBeBZk=', 'Content-Length': '0'} 2022-09-03T06:45:44.823 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b742c428a3b07805-006312f818-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.824 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:44.825 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:44.825 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:44.826 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:44.827 DEBUG:boto:Method: GET 2022-09-03T06:45:44.827 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:45:44.828 DEBUG:boto:Data: 2022-09-03T06:45:44.829 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.829 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.830 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.831 DEBUG:boto:Params: {} 2022-09-03T06:45:44.832 DEBUG:boto:Token: None 2022-09-03T06:45:44.833 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/ 2022-09-03T06:45:44.833 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8= 2022-09-03T06:45:44.834 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8=', 'Content-Length': '0'} 2022-09-03T06:45:44.836 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000058355d7cce45e18-006312f818-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.837 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:44.839 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:44.840 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:44.840 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:44.841 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:44.842 DEBUG:boto:Method: GET 2022-09-03T06:45:44.842 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:45:44.843 DEBUG:boto:Data: 2022-09-03T06:45:44.843 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.844 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.844 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.845 DEBUG:boto:Params: {} 2022-09-03T06:45:44.845 DEBUG:boto:Token: None 2022-09-03T06:45:44.846 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/ 2022-09-03T06:45:44.847 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8= 2022-09-03T06:45:44.847 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8=', 'Content-Length': '0'} 2022-09-03T06:45:44.850 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000071911e7b168d7d49-006312f818-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.851 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:44.852 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:44.852 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:44.853 DEBUG:boto:Method: HEAD 2022-09-03T06:45:44.854 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:45:44.854 DEBUG:boto:Data: 2022-09-03T06:45:44.855 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.856 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.856 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.857 DEBUG:boto:Params: {} 2022-09-03T06:45:44.857 DEBUG:boto:Token: None 2022-09-03T06:45:44.858 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj69 2022-09-03T06:45:44.858 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0eashJs3c18iQRcqoURdLD4z67k= 2022-09-03T06:45:44.859 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0eashJs3c18iQRcqoURdLD4z67k=', 'Content-Length': '0'} 2022-09-03T06:45:44.861 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004ac2a70137bb1c91-006312f818-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.862 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:44.862 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:44.863 DEBUG:boto:Method: HEAD 2022-09-03T06:45:44.863 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:45:44.864 DEBUG:boto:Data: 2022-09-03T06:45:44.864 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.865 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.865 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.866 DEBUG:boto:Params: {} 2022-09-03T06:45:44.867 DEBUG:boto:Token: None 2022-09-03T06:45:44.867 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj69 2022-09-03T06:45:44.868 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0eashJs3c18iQRcqoURdLD4z67k= 2022-09-03T06:45:44.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0eashJs3c18iQRcqoURdLD4z67k=', 'Content-Length': '0'} 2022-09-03T06:45:44.870 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e14a959e4a22d2e5-006312f818-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.871 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:45:44.871 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:44.872 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:44.873 DEBUG:boto:Method: GET 2022-09-03T06:45:44.873 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:45:44.874 DEBUG:boto:Data: 2022-09-03T06:45:44.875 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.876 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.877 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.877 DEBUG:boto:Params: {} 2022-09-03T06:45:44.878 DEBUG:boto:Token: None 2022-09-03T06:45:44.879 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj69 2022-09-03T06:45:44.879 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SmbM7NFVq9GH49Vr8YuZPyVqTqc= 2022-09-03T06:45:44.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SmbM7NFVq9GH49Vr8YuZPyVqTqc=', 'Content-Length': '0'} 2022-09-03T06:45:44.886 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001bdf0254aca889cf-006312f818-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.887 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:45:44.888 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:45:44.889 DEBUG:boto:Method: GET 2022-09-03T06:45:44.890 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:45:44.891 DEBUG:boto:Data: 2022-09-03T06:45:44.892 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.892 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.893 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.894 DEBUG:boto:Params: {} 2022-09-03T06:45:44.895 DEBUG:boto:Token: None 2022-09-03T06:45:44.896 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj69 2022-09-03T06:45:44.897 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SmbM7NFVq9GH49Vr8YuZPyVqTqc= 2022-09-03T06:45:44.898 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SmbM7NFVq9GH49Vr8YuZPyVqTqc=', 'Content-Length': '0'} 2022-09-03T06:45:44.901 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002fcd82e4fb63b986-006312f818-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.946 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:44.947 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:44.948 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:44.949 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:44.949 DEBUG:boto:Method: GET 2022-09-03T06:45:44.950 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:45:44.950 DEBUG:boto:Data: 2022-09-03T06:45:44.951 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.952 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.952 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.953 DEBUG:boto:Params: {} 2022-09-03T06:45:44.953 DEBUG:boto:Token: None 2022-09-03T06:45:44.954 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/ 2022-09-03T06:45:44.955 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8= 2022-09-03T06:45:44.955 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8=', 'Content-Length': '0'} 2022-09-03T06:45:44.958 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002bdcd6072f5c234f-006312f818-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.959 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:44.960 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:44.960 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:44.961 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:44.961 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:45:44.962 DEBUG:boto:Method: GET 2022-09-03T06:45:44.963 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:45:44.963 DEBUG:boto:Data: 2022-09-03T06:45:44.964 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.964 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:44.965 DEBUG:boto:Port: 8000 2022-09-03T06:45:44.965 DEBUG:boto:Params: {} 2022-09-03T06:45:44.966 DEBUG:boto:Token: None 2022-09-03T06:45:44.967 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/ 2022-09-03T06:45:44.968 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8= 2022-09-03T06:45:44.969 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r7iwjRsZz9LKnB44mV6mvGvDdy8=', 'Content-Length': '0'} 2022-09-03T06:45:44.972 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f082f46c08f1b40e-006312f818-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:44.972 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:44.973 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:45:44.974 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:44.974 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:44.975 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:44.975 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:44.976 DEBUG:boto:Method: GET 2022-09-03T06:45:44.977 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:44.978 DEBUG:boto:Data: 2022-09-03T06:45:44.978 DEBUG:boto:Headers: {} 2022-09-03T06:45:44.979 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:44.980 DEBUG:boto:Port: 8001 2022-09-03T06:45:44.980 DEBUG:boto:Params: {} 2022-09-03T06:45:44.981 DEBUG:boto:Token: None 2022-09-03T06:45:44.981 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:44 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:44.982 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JCj4ztVV9aNOBOUopuTdZV8CWUc= 2022-09-03T06:45:44.983 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JCj4ztVV9aNOBOUopuTdZV8CWUc=', 'Content-Length': '0'} 2022-09-03T06:45:44.985 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b241aaa0a14e37ed-006312f818-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.026 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:45.028 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:45.029 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:45.030 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:45.033 DEBUG:boto:Method: GET 2022-09-03T06:45:45.034 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:45.035 DEBUG:boto:Data: 2022-09-03T06:45:45.036 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.037 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:45.038 DEBUG:boto:Port: 8000 2022-09-03T06:45:45.039 DEBUG:boto:Params: {} 2022-09-03T06:45:45.041 DEBUG:boto:Token: None 2022-09-03T06:45:45.043 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:45.044 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dKY8QEXrSxZSOYgM/1BtAn4MPFA= 2022-09-03T06:45:45.045 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dKY8QEXrSxZSOYgM/1BtAn4MPFA=', 'Content-Length': '0'} 2022-09-03T06:45:45.050 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005a96f9baf1704492-006312f819-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.051 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:45.053 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:45.054 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:45.055 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:45.056 DEBUG:boto:Method: HEAD 2022-09-03T06:45:45.057 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:45.058 DEBUG:boto:Data: 2022-09-03T06:45:45.059 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.061 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:45.062 DEBUG:boto:Port: 8001 2022-09-03T06:45:45.063 DEBUG:boto:Params: {} 2022-09-03T06:45:45.064 DEBUG:boto:Token: None 2022-09-03T06:45:45.065 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:45.067 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8JE79T6EJeFVsXLTn1MHWtWaA30= 2022-09-03T06:45:45.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8JE79T6EJeFVsXLTn1MHWtWaA30=', 'Content-Length': '0'} 2022-09-03T06:45:45.077 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000001585a9619e53463-006312f819-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.078 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:45.079 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:45.079 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:45.080 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:45.080 DEBUG:boto:Method: HEAD 2022-09-03T06:45:45.081 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:45.081 DEBUG:boto:Data: 2022-09-03T06:45:45.082 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.082 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:45.083 DEBUG:boto:Port: 8000 2022-09-03T06:45:45.084 DEBUG:boto:Params: {} 2022-09-03T06:45:45.084 DEBUG:boto:Token: None 2022-09-03T06:45:45.085 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:45.086 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8JE79T6EJeFVsXLTn1MHWtWaA30= 2022-09-03T06:45:45.087 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8JE79T6EJeFVsXLTn1MHWtWaA30=', 'Content-Length': '0'} 2022-09-03T06:45:45.089 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004b21a363ee56ba91-006312f819-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.091 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:45:45.092 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:45.094 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:45.096 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:45.098 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:45.100 DEBUG:boto:Method: GET 2022-09-03T06:45:45.101 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:45.103 DEBUG:boto:Data: 2022-09-03T06:45:45.104 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.105 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:45.106 DEBUG:boto:Port: 8001 2022-09-03T06:45:45.107 DEBUG:boto:Params: {} 2022-09-03T06:45:45.108 DEBUG:boto:Token: None 2022-09-03T06:45:45.109 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:45.110 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dKY8QEXrSxZSOYgM/1BtAn4MPFA= 2022-09-03T06:45:45.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dKY8QEXrSxZSOYgM/1BtAn4MPFA=', 'Content-Length': '0'} 2022-09-03T06:45:45.115 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a3caaf791064592c-006312f819-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.116 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:45.117 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:45.118 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:45.120 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:45.121 DEBUG:boto:Method: GET 2022-09-03T06:45:45.122 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:45.123 DEBUG:boto:Data: 2022-09-03T06:45:45.124 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.125 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:45.125 DEBUG:boto:Port: 8000 2022-09-03T06:45:45.126 DEBUG:boto:Params: {} 2022-09-03T06:45:45.127 DEBUG:boto:Token: None 2022-09-03T06:45:45.128 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:45:45.129 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dKY8QEXrSxZSOYgM/1BtAn4MPFA= 2022-09-03T06:45:45.130 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dKY8QEXrSxZSOYgM/1BtAn4MPFA=', 'Content-Length': '0'} 2022-09-03T06:45:45.137 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008bad6e739fa962b3-006312f819-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.138 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:45.139 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:45.140 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:45.140 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:45.141 DEBUG:boto:Method: GET 2022-09-03T06:45:45.141 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:45:45.142 DEBUG:boto:Data: 2022-09-03T06:45:45.142 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.143 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:45.144 DEBUG:boto:Port: 8001 2022-09-03T06:45:45.144 DEBUG:boto:Params: {} 2022-09-03T06:45:45.145 DEBUG:boto:Token: None 2022-09-03T06:45:45.146 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/ 2022-09-03T06:45:45.147 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BExC764FO51WrJEvR/WwlFK8xC0= 2022-09-03T06:45:45.147 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BExC764FO51WrJEvR/WwlFK8xC0=', 'Content-Length': '0'} 2022-09-03T06:45:45.153 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008b39323f03411d8f-006312f819-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.154 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:45.156 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:45.157 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:45.158 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:45.160 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:45.162 DEBUG:boto:Method: GET 2022-09-03T06:45:45.162 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:45:45.163 DEBUG:boto:Data: 2022-09-03T06:45:45.165 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.165 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:45.166 DEBUG:boto:Port: 8000 2022-09-03T06:45:45.167 DEBUG:boto:Params: {} 2022-09-03T06:45:45.168 DEBUG:boto:Token: None 2022-09-03T06:45:45.169 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/ 2022-09-03T06:45:45.171 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BExC764FO51WrJEvR/WwlFK8xC0= 2022-09-03T06:45:45.173 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BExC764FO51WrJEvR/WwlFK8xC0=', 'Content-Length': '0'} 2022-09-03T06:45:45.181 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002de1a069b84f0d5a-006312f819-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.226 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:45.227 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:45.228 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:45.228 DEBUG:boto:Method: HEAD 2022-09-03T06:45:45.229 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:45:45.229 DEBUG:boto:Data: 2022-09-03T06:45:45.230 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.231 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:45.232 DEBUG:boto:Port: 8001 2022-09-03T06:45:45.232 DEBUG:boto:Params: {} 2022-09-03T06:45:45.233 DEBUG:boto:Token: None 2022-09-03T06:45:45.234 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj76 2022-09-03T06:45:45.234 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s38BeEY74XZszY98t7K1A05Z5u8= 2022-09-03T06:45:45.235 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s38BeEY74XZszY98t7K1A05Z5u8=', 'Content-Length': '0'} 2022-09-03T06:45:45.238 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000046c9918a1bbeff18-006312f819-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.239 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:45.240 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:45.241 DEBUG:boto:Method: HEAD 2022-09-03T06:45:45.241 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:45:45.242 DEBUG:boto:Data: 2022-09-03T06:45:45.243 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.243 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:45.244 DEBUG:boto:Port: 8000 2022-09-03T06:45:45.244 DEBUG:boto:Params: {} 2022-09-03T06:45:45.245 DEBUG:boto:Token: None 2022-09-03T06:45:45.246 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj76 2022-09-03T06:45:45.246 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s38BeEY74XZszY98t7K1A05Z5u8= 2022-09-03T06:45:45.247 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s38BeEY74XZszY98t7K1A05Z5u8=', 'Content-Length': '0'} 2022-09-03T06:45:45.250 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b75734071ed76845-006312f819-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.251 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:45:45.251 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:45.252 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:45.253 DEBUG:boto:Method: GET 2022-09-03T06:45:45.253 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:45:45.254 DEBUG:boto:Data: 2022-09-03T06:45:45.255 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.257 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:45.257 DEBUG:boto:Port: 8001 2022-09-03T06:45:45.258 DEBUG:boto:Params: {} 2022-09-03T06:45:45.259 DEBUG:boto:Token: None 2022-09-03T06:45:45.260 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj76 2022-09-03T06:45:45.261 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ftdVUrqiCPuXhaR0Ts8gld7vmKQ= 2022-09-03T06:45:45.261 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ftdVUrqiCPuXhaR0Ts8gld7vmKQ=', 'Content-Length': '0'} 2022-09-03T06:45:45.264 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000221e2fe34f3ba6e8-006312f819-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.264 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:45:45.265 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:45:45.266 DEBUG:boto:Method: GET 2022-09-03T06:45:45.266 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:45:45.267 DEBUG:boto:Data: 2022-09-03T06:45:45.267 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.268 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:45.269 DEBUG:boto:Port: 8000 2022-09-03T06:45:45.269 DEBUG:boto:Params: {} 2022-09-03T06:45:45.270 DEBUG:boto:Token: None 2022-09-03T06:45:45.271 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj76 2022-09-03T06:45:45.272 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ftdVUrqiCPuXhaR0Ts8gld7vmKQ= 2022-09-03T06:45:45.272 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ftdVUrqiCPuXhaR0Ts8gld7vmKQ=', 'Content-Length': '0'} 2022-09-03T06:45:45.274 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004009eafb961de77b-006312f819-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.275 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:45.276 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:45.277 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:45.277 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:45.278 DEBUG:boto:Method: GET 2022-09-03T06:45:45.278 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:45:45.279 DEBUG:boto:Data: 2022-09-03T06:45:45.279 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.280 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:45.281 DEBUG:boto:Port: 8001 2022-09-03T06:45:45.282 DEBUG:boto:Params: {} 2022-09-03T06:45:45.282 DEBUG:boto:Token: None 2022-09-03T06:45:45.283 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/ 2022-09-03T06:45:45.284 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BExC764FO51WrJEvR/WwlFK8xC0= 2022-09-03T06:45:45.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BExC764FO51WrJEvR/WwlFK8xC0=', 'Content-Length': '0'} 2022-09-03T06:45:45.287 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007445d3df579c5c9d-006312f819-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.288 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:45.289 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:45.289 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:45.290 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:45.290 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:45:45.291 DEBUG:boto:Method: GET 2022-09-03T06:45:45.292 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:45:45.293 DEBUG:boto:Data: 2022-09-03T06:45:45.293 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.294 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:45.294 DEBUG:boto:Port: 8000 2022-09-03T06:45:45.295 DEBUG:boto:Params: {} 2022-09-03T06:45:45.296 DEBUG:boto:Token: None 2022-09-03T06:45:45.296 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/ 2022-09-03T06:45:45.297 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BExC764FO51WrJEvR/WwlFK8xC0= 2022-09-03T06:45:45.297 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BExC764FO51WrJEvR/WwlFK8xC0=', 'Content-Length': '0'} 2022-09-03T06:45:45.301 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d58d339cfc512b3-006312f819-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.302 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:45.303 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:45:45.304 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:45.304 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:45.305 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.306 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.306 DEBUG:boto:Method: GET 2022-09-03T06:45:45.307 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.308 DEBUG:boto:Data: 2022-09-03T06:45:45.308 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.309 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:45.309 DEBUG:boto:Port: 8001 2022-09-03T06:45:45.310 DEBUG:boto:Params: {} 2022-09-03T06:45:45.310 DEBUG:boto:Token: None 2022-09-03T06:45:45.311 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.312 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fv6+LX2RJ0mPBHyIFypaLYsCJ48= 2022-09-03T06:45:45.313 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fv6+LX2RJ0mPBHyIFypaLYsCJ48=', 'Content-Length': '0'} 2022-09-03T06:45:45.315 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000656ea9b172d49218-006312f819-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.362 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:45.363 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:45.364 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.365 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.366 DEBUG:boto:Method: GET 2022-09-03T06:45:45.367 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.367 DEBUG:boto:Data: 2022-09-03T06:45:45.368 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.368 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:45.369 DEBUG:boto:Port: 8000 2022-09-03T06:45:45.369 DEBUG:boto:Params: {} 2022-09-03T06:45:45.370 DEBUG:boto:Token: None 2022-09-03T06:45:45.371 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.371 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fv6+LX2RJ0mPBHyIFypaLYsCJ48= 2022-09-03T06:45:45.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fv6+LX2RJ0mPBHyIFypaLYsCJ48=', 'Content-Length': '0'} 2022-09-03T06:45:45.374 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fde5bd44744699de-006312f819-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.375 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:45.377 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:45.377 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.378 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.380 DEBUG:boto:Method: HEAD 2022-09-03T06:45:45.380 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.381 DEBUG:boto:Data: 2022-09-03T06:45:45.381 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.382 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:45.382 DEBUG:boto:Port: 8001 2022-09-03T06:45:45.383 DEBUG:boto:Params: {} 2022-09-03T06:45:45.383 DEBUG:boto:Token: None 2022-09-03T06:45:45.384 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.385 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JXVUA9uL/Qa0j6J96CSLAv4TE8I= 2022-09-03T06:45:45.385 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JXVUA9uL/Qa0j6J96CSLAv4TE8I=', 'Content-Length': '0'} 2022-09-03T06:45:45.388 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000019d3c22bd39300dc-006312f819-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.389 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:45.389 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:45.390 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.391 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.392 DEBUG:boto:Method: HEAD 2022-09-03T06:45:45.392 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.393 DEBUG:boto:Data: 2022-09-03T06:45:45.393 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.394 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:45.395 DEBUG:boto:Port: 8000 2022-09-03T06:45:45.396 DEBUG:boto:Params: {} 2022-09-03T06:45:45.397 DEBUG:boto:Token: None 2022-09-03T06:45:45.398 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.399 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JXVUA9uL/Qa0j6J96CSLAv4TE8I= 2022-09-03T06:45:45.399 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JXVUA9uL/Qa0j6J96CSLAv4TE8I=', 'Content-Length': '0'} 2022-09-03T06:45:45.402 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000da65091bd4e16699-006312f819-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.403 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:45:45.404 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:45.404 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:45.405 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.406 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.406 DEBUG:boto:Method: GET 2022-09-03T06:45:45.407 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.407 DEBUG:boto:Data: 2022-09-03T06:45:45.408 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.409 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:45.409 DEBUG:boto:Port: 8001 2022-09-03T06:45:45.410 DEBUG:boto:Params: {} 2022-09-03T06:45:45.410 DEBUG:boto:Token: None 2022-09-03T06:45:45.411 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.412 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fv6+LX2RJ0mPBHyIFypaLYsCJ48= 2022-09-03T06:45:45.413 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fv6+LX2RJ0mPBHyIFypaLYsCJ48=', 'Content-Length': '0'} 2022-09-03T06:45:45.415 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db371e149256a0c5-006312f819-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.416 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:45.416 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:45.417 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.417 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.418 DEBUG:boto:Method: GET 2022-09-03T06:45:45.419 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.419 DEBUG:boto:Data: 2022-09-03T06:45:45.420 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.420 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:45.421 DEBUG:boto:Port: 8000 2022-09-03T06:45:45.422 DEBUG:boto:Params: {} 2022-09-03T06:45:45.422 DEBUG:boto:Token: None 2022-09-03T06:45:45.423 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:45:45.424 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fv6+LX2RJ0mPBHyIFypaLYsCJ48= 2022-09-03T06:45:45.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fv6+LX2RJ0mPBHyIFypaLYsCJ48=', 'Content-Length': '0'} 2022-09-03T06:45:45.426 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e37d63e1ec5eff29-006312f819-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.474 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:45.476 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:45.477 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:45.478 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:45.479 DEBUG:boto:Method: GET 2022-09-03T06:45:45.481 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:45:45.482 DEBUG:boto:Data: 2022-09-03T06:45:45.483 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.483 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:45.484 DEBUG:boto:Port: 8001 2022-09-03T06:45:45.484 DEBUG:boto:Params: {} 2022-09-03T06:45:45.485 DEBUG:boto:Token: None 2022-09-03T06:45:45.486 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/ 2022-09-03T06:45:45.487 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BExC764FO51WrJEvR/WwlFK8xC0= 2022-09-03T06:45:45.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BExC764FO51WrJEvR/WwlFK8xC0=', 'Content-Length': '0'} 2022-09-03T06:45:45.491 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000623056431e202d9-006312f819-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.491 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:45.492 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:45.493 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:45.494 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:45.494 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:45.495 DEBUG:boto:Method: GET 2022-09-03T06:45:45.495 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:45:45.496 DEBUG:boto:Data: 2022-09-03T06:45:45.496 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.497 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:45.498 DEBUG:boto:Port: 8000 2022-09-03T06:45:45.498 DEBUG:boto:Params: {} 2022-09-03T06:45:45.499 DEBUG:boto:Token: None 2022-09-03T06:45:45.500 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/ 2022-09-03T06:45:45.500 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BExC764FO51WrJEvR/WwlFK8xC0= 2022-09-03T06:45:45.501 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BExC764FO51WrJEvR/WwlFK8xC0=', 'Content-Length': '0'} 2022-09-03T06:45:45.504 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d429828b11d0474a-006312f819-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.504 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:45.505 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:45.506 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:45.506 DEBUG:boto:Method: HEAD 2022-09-03T06:45:45.507 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:45:45.507 DEBUG:boto:Data: 2022-09-03T06:45:45.508 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.508 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:45.509 DEBUG:boto:Port: 8001 2022-09-03T06:45:45.509 DEBUG:boto:Params: {} 2022-09-03T06:45:45.510 DEBUG:boto:Token: None 2022-09-03T06:45:45.511 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj85 2022-09-03T06:45:45.512 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7ym4Asse2tlc+0pfKMcJ2aTY8S4= 2022-09-03T06:45:45.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7ym4Asse2tlc+0pfKMcJ2aTY8S4=', 'Content-Length': '0'} 2022-09-03T06:45:45.515 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2ec5aee4add6b39-006312f819-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.516 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:45.517 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:45.518 DEBUG:boto:Method: HEAD 2022-09-03T06:45:45.519 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:45:45.519 DEBUG:boto:Data: 2022-09-03T06:45:45.520 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.520 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:45.522 DEBUG:boto:Port: 8000 2022-09-03T06:45:45.523 DEBUG:boto:Params: {} 2022-09-03T06:45:45.524 DEBUG:boto:Token: None 2022-09-03T06:45:45.525 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj85 2022-09-03T06:45:45.526 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7ym4Asse2tlc+0pfKMcJ2aTY8S4= 2022-09-03T06:45:45.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7ym4Asse2tlc+0pfKMcJ2aTY8S4=', 'Content-Length': '0'} 2022-09-03T06:45:45.530 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000044fb40078265a605-006312f819-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.532 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:45:45.533 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:45.534 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:45.535 DEBUG:boto:Method: GET 2022-09-03T06:45:45.536 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:45:45.537 DEBUG:boto:Data: 2022-09-03T06:45:45.538 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.539 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:45.540 DEBUG:boto:Port: 8001 2022-09-03T06:45:45.541 DEBUG:boto:Params: {} 2022-09-03T06:45:45.542 DEBUG:boto:Token: None 2022-09-03T06:45:45.543 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj85 2022-09-03T06:45:45.544 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aLem092NonlVsBPo8OEuA2akP7g= 2022-09-03T06:45:45.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aLem092NonlVsBPo8OEuA2akP7g=', 'Content-Length': '0'} 2022-09-03T06:45:45.547 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000060f0b308b132c5a2-006312f819-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.594 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:45:45.595 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:45:45.595 DEBUG:boto:Method: GET 2022-09-03T06:45:45.596 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:45:45.597 DEBUG:boto:Data: 2022-09-03T06:45:45.597 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.598 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:45.598 DEBUG:boto:Port: 8000 2022-09-03T06:45:45.599 DEBUG:boto:Params: {} 2022-09-03T06:45:45.599 DEBUG:boto:Token: None 2022-09-03T06:45:45.600 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj85 2022-09-03T06:45:45.601 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aLem092NonlVsBPo8OEuA2akP7g= 2022-09-03T06:45:45.601 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aLem092NonlVsBPo8OEuA2akP7g=', 'Content-Length': '0'} 2022-09-03T06:45:45.609 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000059997bb470ac822d-006312f819-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.611 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:45.611 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:45.612 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:45.612 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:45.613 DEBUG:boto:Method: GET 2022-09-03T06:45:45.613 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:45:45.614 DEBUG:boto:Data: 2022-09-03T06:45:45.615 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.615 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:45.616 DEBUG:boto:Port: 8001 2022-09-03T06:45:45.617 DEBUG:boto:Params: {} 2022-09-03T06:45:45.618 DEBUG:boto:Token: None 2022-09-03T06:45:45.618 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/ 2022-09-03T06:45:45.619 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BExC764FO51WrJEvR/WwlFK8xC0= 2022-09-03T06:45:45.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BExC764FO51WrJEvR/WwlFK8xC0=', 'Content-Length': '0'} 2022-09-03T06:45:45.623 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e47c66df349d95f6-006312f819-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.624 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:45.625 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:45.625 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:45.626 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:45.627 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:45:45.628 DEBUG:boto:Method: GET 2022-09-03T06:45:45.628 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:45:45.629 DEBUG:boto:Data: 2022-09-03T06:45:45.630 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.631 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:45.632 DEBUG:boto:Port: 8000 2022-09-03T06:45:45.632 DEBUG:boto:Params: {} 2022-09-03T06:45:45.633 DEBUG:boto:Token: None 2022-09-03T06:45:45.634 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/ 2022-09-03T06:45:45.634 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BExC764FO51WrJEvR/WwlFK8xC0= 2022-09-03T06:45:45.635 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BExC764FO51WrJEvR/WwlFK8xC0=', 'Content-Length': '0'} 2022-09-03T06:45:45.638 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fd87db87ce9fe381-006312f819-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.640 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:45.641 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:45:45.642 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:45.643 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:45.643 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.644 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.644 DEBUG:boto:Method: GET 2022-09-03T06:45:45.645 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.646 DEBUG:boto:Data: 2022-09-03T06:45:45.647 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.649 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:45.650 DEBUG:boto:Port: 8001 2022-09-03T06:45:45.651 DEBUG:boto:Params: {} 2022-09-03T06:45:45.652 DEBUG:boto:Token: None 2022-09-03T06:45:45.653 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.655 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FKRzl382q/oEVaG+ULa2we6W2GU= 2022-09-03T06:45:45.658 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FKRzl382q/oEVaG+ULa2we6W2GU=', 'Content-Length': '0'} 2022-09-03T06:45:45.661 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001719aebbd1d70fb0-006312f819-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.662 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:45.663 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:45.663 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.664 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.666 DEBUG:boto:Method: GET 2022-09-03T06:45:45.667 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.667 DEBUG:boto:Data: 2022-09-03T06:45:45.668 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.668 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:45.669 DEBUG:boto:Port: 8000 2022-09-03T06:45:45.669 DEBUG:boto:Params: {} 2022-09-03T06:45:45.670 DEBUG:boto:Token: None 2022-09-03T06:45:45.671 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.671 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FKRzl382q/oEVaG+ULa2we6W2GU= 2022-09-03T06:45:45.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FKRzl382q/oEVaG+ULa2we6W2GU=', 'Content-Length': '0'} 2022-09-03T06:45:45.674 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000044d51cb9f34ec8f7-006312f819-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.720 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:45.721 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:45.722 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.723 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.723 DEBUG:boto:Method: HEAD 2022-09-03T06:45:45.724 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.724 DEBUG:boto:Data: 2022-09-03T06:45:45.725 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.726 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:45.726 DEBUG:boto:Port: 8001 2022-09-03T06:45:45.742 DEBUG:boto:Params: {} 2022-09-03T06:45:45.743 DEBUG:boto:Token: None 2022-09-03T06:45:45.743 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.744 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9nZl+YhCHB9Hh2CL8n0uxVSPR7A= 2022-09-03T06:45:45.745 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9nZl+YhCHB9Hh2CL8n0uxVSPR7A=', 'Content-Length': '0'} 2022-09-03T06:45:45.747 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000059c7bc27a9063e40-006312f819-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.748 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:45.749 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:45.749 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.750 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.750 DEBUG:boto:Method: HEAD 2022-09-03T06:45:45.751 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.752 DEBUG:boto:Data: 2022-09-03T06:45:45.752 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.753 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:45.753 DEBUG:boto:Port: 8000 2022-09-03T06:45:45.754 DEBUG:boto:Params: {} 2022-09-03T06:45:45.755 DEBUG:boto:Token: None 2022-09-03T06:45:45.755 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.756 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9nZl+YhCHB9Hh2CL8n0uxVSPR7A= 2022-09-03T06:45:45.756 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9nZl+YhCHB9Hh2CL8n0uxVSPR7A=', 'Content-Length': '0'} 2022-09-03T06:45:45.759 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000485c3a541c842fb-006312f819-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.759 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:45:45.760 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:45.761 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:45.761 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.762 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.762 DEBUG:boto:Method: GET 2022-09-03T06:45:45.763 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.764 DEBUG:boto:Data: 2022-09-03T06:45:45.764 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.765 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:45.765 DEBUG:boto:Port: 8001 2022-09-03T06:45:45.766 DEBUG:boto:Params: {} 2022-09-03T06:45:45.766 DEBUG:boto:Token: None 2022-09-03T06:45:45.767 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.768 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FKRzl382q/oEVaG+ULa2we6W2GU= 2022-09-03T06:45:45.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FKRzl382q/oEVaG+ULa2we6W2GU=', 'Content-Length': '0'} 2022-09-03T06:45:45.771 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ed0cc2350d15079b-006312f819-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.772 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:45.772 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:45.773 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.774 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.774 DEBUG:boto:Method: GET 2022-09-03T06:45:45.775 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.775 DEBUG:boto:Data: 2022-09-03T06:45:45.776 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.777 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:45.777 DEBUG:boto:Port: 8000 2022-09-03T06:45:45.778 DEBUG:boto:Params: {} 2022-09-03T06:45:45.779 DEBUG:boto:Token: None 2022-09-03T06:45:45.779 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:45:45.780 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FKRzl382q/oEVaG+ULa2we6W2GU= 2022-09-03T06:45:45.781 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FKRzl382q/oEVaG+ULa2we6W2GU=', 'Content-Length': '0'} 2022-09-03T06:45:45.784 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004fa76bad327fff4e-006312f819-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.785 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:45.786 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:45.787 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:45.788 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:45.789 DEBUG:boto:Method: GET 2022-09-03T06:45:45.790 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:45:45.791 DEBUG:boto:Data: 2022-09-03T06:45:45.792 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.793 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:45.794 DEBUG:boto:Port: 8001 2022-09-03T06:45:45.795 DEBUG:boto:Params: {} 2022-09-03T06:45:45.796 DEBUG:boto:Token: None 2022-09-03T06:45:45.797 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/ 2022-09-03T06:45:45.797 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BExC764FO51WrJEvR/WwlFK8xC0= 2022-09-03T06:45:45.798 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BExC764FO51WrJEvR/WwlFK8xC0=', 'Content-Length': '0'} 2022-09-03T06:45:45.801 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000828632ef66dd5782-006312f819-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.847 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:45.851 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:45.852 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:45.853 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:45.854 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:45.855 DEBUG:boto:Method: GET 2022-09-03T06:45:45.857 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:45:45.858 DEBUG:boto:Data: 2022-09-03T06:45:45.859 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.860 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:45.861 DEBUG:boto:Port: 8000 2022-09-03T06:45:45.862 DEBUG:boto:Params: {} 2022-09-03T06:45:45.862 DEBUG:boto:Token: None 2022-09-03T06:45:45.863 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/ 2022-09-03T06:45:45.864 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BExC764FO51WrJEvR/WwlFK8xC0= 2022-09-03T06:45:45.864 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BExC764FO51WrJEvR/WwlFK8xC0=', 'Content-Length': '0'} 2022-09-03T06:45:45.868 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000000121a21489e07b-006312f819-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.870 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:45.871 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:45.872 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:45.873 DEBUG:boto:Method: HEAD 2022-09-03T06:45:45.874 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:45:45.874 DEBUG:boto:Data: 2022-09-03T06:45:45.875 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.876 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:45.877 DEBUG:boto:Port: 8001 2022-09-03T06:45:45.879 DEBUG:boto:Params: {} 2022-09-03T06:45:45.880 DEBUG:boto:Token: None 2022-09-03T06:45:45.881 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj87 2022-09-03T06:45:45.882 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VAsuRHbW2FWgSpAGr064tPOQOIM= 2022-09-03T06:45:45.883 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VAsuRHbW2FWgSpAGr064tPOQOIM=', 'Content-Length': '0'} 2022-09-03T06:45:45.886 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000173aedee5eca2beb-006312f819-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.887 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:45.888 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:45.890 DEBUG:boto:Method: HEAD 2022-09-03T06:45:45.892 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:45:45.893 DEBUG:boto:Data: 2022-09-03T06:45:45.894 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.895 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:45.896 DEBUG:boto:Port: 8000 2022-09-03T06:45:45.897 DEBUG:boto:Params: {} 2022-09-03T06:45:45.900 DEBUG:boto:Token: None 2022-09-03T06:45:45.900 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj87 2022-09-03T06:45:45.901 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VAsuRHbW2FWgSpAGr064tPOQOIM= 2022-09-03T06:45:45.902 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VAsuRHbW2FWgSpAGr064tPOQOIM=', 'Content-Length': '0'} 2022-09-03T06:45:45.906 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007a99aebf94da47da-006312f819-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.908 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:45:45.909 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:45.909 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:45.911 DEBUG:boto:Method: GET 2022-09-03T06:45:45.912 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:45:45.913 DEBUG:boto:Data: 2022-09-03T06:45:45.914 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.915 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:45.916 DEBUG:boto:Port: 8001 2022-09-03T06:45:45.918 DEBUG:boto:Params: {} 2022-09-03T06:45:45.920 DEBUG:boto:Token: None 2022-09-03T06:45:45.922 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj87 2022-09-03T06:45:45.924 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ODvGALM4ZKlN3Hp+0CKIlRgUmXU= 2022-09-03T06:45:45.925 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ODvGALM4ZKlN3Hp+0CKIlRgUmXU=', 'Content-Length': '0'} 2022-09-03T06:45:45.927 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003aaa91fc3e0327f6-006312f819-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.930 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:45:45.930 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:45:45.931 DEBUG:boto:Method: GET 2022-09-03T06:45:45.932 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:45:45.932 DEBUG:boto:Data: 2022-09-03T06:45:45.933 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.933 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:45.934 DEBUG:boto:Port: 8000 2022-09-03T06:45:45.934 DEBUG:boto:Params: {} 2022-09-03T06:45:45.935 DEBUG:boto:Token: None 2022-09-03T06:45:45.936 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/obj87 2022-09-03T06:45:45.937 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ODvGALM4ZKlN3Hp+0CKIlRgUmXU= 2022-09-03T06:45:45.938 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ODvGALM4ZKlN3Hp+0CKIlRgUmXU=', 'Content-Length': '0'} 2022-09-03T06:45:45.940 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007e60cea411bbb2ed-006312f819-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:45.982 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:45.983 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:45.983 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:45.984 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:45.985 DEBUG:boto:Method: GET 2022-09-03T06:45:45.986 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:45:45.987 DEBUG:boto:Data: 2022-09-03T06:45:45.988 DEBUG:boto:Headers: {} 2022-09-03T06:45:45.989 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:45.990 DEBUG:boto:Port: 8001 2022-09-03T06:45:45.991 DEBUG:boto:Params: {} 2022-09-03T06:45:45.993 DEBUG:boto:Token: None 2022-09-03T06:45:45.994 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:45 GMT /dsgvcy-75/ 2022-09-03T06:45:45.995 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BExC764FO51WrJEvR/WwlFK8xC0= 2022-09-03T06:45:45.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BExC764FO51WrJEvR/WwlFK8xC0=', 'Content-Length': '0'} 2022-09-03T06:45:46.000 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000622c241bf73e2f63-006312f819-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.002 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:46.003 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:46.004 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:46.005 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:46.007 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:45:46.008 DEBUG:boto:Method: GET 2022-09-03T06:45:46.009 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:45:46.010 DEBUG:boto:Data: 2022-09-03T06:45:46.010 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.011 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:46.012 DEBUG:boto:Port: 8000 2022-09-03T06:45:46.012 DEBUG:boto:Params: {} 2022-09-03T06:45:46.013 DEBUG:boto:Token: None 2022-09-03T06:45:46.014 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/ 2022-09-03T06:45:46.015 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0UOR7ER9dzeetPDULGBgciZEtEs= 2022-09-03T06:45:46.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0UOR7ER9dzeetPDULGBgciZEtEs=', 'Content-Length': '0'} 2022-09-03T06:45:46.019 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000088dfde8c1f9a5068-006312f81a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.020 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:46.021 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:45:46.022 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:46.023 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:46.023 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.024 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.024 DEBUG:boto:Method: GET 2022-09-03T06:45:46.025 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.025 DEBUG:boto:Data: 2022-09-03T06:45:46.026 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.027 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:46.027 DEBUG:boto:Port: 8001 2022-09-03T06:45:46.028 DEBUG:boto:Params: {} 2022-09-03T06:45:46.031 DEBUG:boto:Token: None 2022-09-03T06:45:46.032 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.033 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yReVELscUkL+pC0bfs6DUYS0hkI= 2022-09-03T06:45:46.035 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yReVELscUkL+pC0bfs6DUYS0hkI=', 'Content-Length': '0'} 2022-09-03T06:45:46.038 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d35fb19011896ced-006312f81a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.039 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:46.040 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:46.041 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.042 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.043 DEBUG:boto:Method: GET 2022-09-03T06:45:46.045 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.046 DEBUG:boto:Data: 2022-09-03T06:45:46.047 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.048 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:46.050 DEBUG:boto:Port: 8000 2022-09-03T06:45:46.051 DEBUG:boto:Params: {} 2022-09-03T06:45:46.052 DEBUG:boto:Token: None 2022-09-03T06:45:46.054 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.055 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yReVELscUkL+pC0bfs6DUYS0hkI= 2022-09-03T06:45:46.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yReVELscUkL+pC0bfs6DUYS0hkI=', 'Content-Length': '0'} 2022-09-03T06:45:46.059 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b3f2e30cce680e87-006312f81a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.060 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:46.061 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:46.062 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.063 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.064 DEBUG:boto:Method: HEAD 2022-09-03T06:45:46.065 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.066 DEBUG:boto:Data: 2022-09-03T06:45:46.067 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.068 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:46.069 DEBUG:boto:Port: 8001 2022-09-03T06:45:46.070 DEBUG:boto:Params: {} 2022-09-03T06:45:46.071 DEBUG:boto:Token: None 2022-09-03T06:45:46.072 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.073 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hu0ukQ4OrTh+unCIGlzdLwBOSEk= 2022-09-03T06:45:46.074 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hu0ukQ4OrTh+unCIGlzdLwBOSEk=', 'Content-Length': '0'} 2022-09-03T06:45:46.076 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004990476e9843837e-006312f81a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.077 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:46.078 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:46.078 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.079 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.079 DEBUG:boto:Method: HEAD 2022-09-03T06:45:46.080 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.080 DEBUG:boto:Data: 2022-09-03T06:45:46.081 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.081 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:46.082 DEBUG:boto:Port: 8000 2022-09-03T06:45:46.083 DEBUG:boto:Params: {} 2022-09-03T06:45:46.083 DEBUG:boto:Token: None 2022-09-03T06:45:46.084 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.084 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hu0ukQ4OrTh+unCIGlzdLwBOSEk= 2022-09-03T06:45:46.085 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hu0ukQ4OrTh+unCIGlzdLwBOSEk=', 'Content-Length': '0'} 2022-09-03T06:45:46.088 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000914f61569fc3c718-006312f81a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.088 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:45:46.089 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:46.090 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:46.090 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.091 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.092 DEBUG:boto:Method: GET 2022-09-03T06:45:46.092 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.093 DEBUG:boto:Data: 2022-09-03T06:45:46.093 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.094 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:46.095 DEBUG:boto:Port: 8001 2022-09-03T06:45:46.095 DEBUG:boto:Params: {} 2022-09-03T06:45:46.096 DEBUG:boto:Token: None 2022-09-03T06:45:46.097 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.098 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yReVELscUkL+pC0bfs6DUYS0hkI= 2022-09-03T06:45:46.099 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yReVELscUkL+pC0bfs6DUYS0hkI=', 'Content-Length': '0'} 2022-09-03T06:45:46.101 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c9ca93747e069ca6-006312f81a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.102 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:46.102 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:46.103 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.103 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.104 DEBUG:boto:Method: GET 2022-09-03T06:45:46.105 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.105 DEBUG:boto:Data: 2022-09-03T06:45:46.106 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.106 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:46.107 DEBUG:boto:Port: 8000 2022-09-03T06:45:46.107 DEBUG:boto:Params: {} 2022-09-03T06:45:46.108 DEBUG:boto:Token: None 2022-09-03T06:45:46.109 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:45:46.110 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yReVELscUkL+pC0bfs6DUYS0hkI= 2022-09-03T06:45:46.110 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yReVELscUkL+pC0bfs6DUYS0hkI=', 'Content-Length': '0'} 2022-09-03T06:45:46.112 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002771d06c248e2ef9-006312f81a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.154 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:46.155 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:46.156 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:46.157 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:46.158 DEBUG:boto:Method: GET 2022-09-03T06:45:46.159 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:45:46.160 DEBUG:boto:Data: 2022-09-03T06:45:46.161 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.162 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:46.163 DEBUG:boto:Port: 8001 2022-09-03T06:45:46.164 DEBUG:boto:Params: {} 2022-09-03T06:45:46.165 DEBUG:boto:Token: None 2022-09-03T06:45:46.166 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/ 2022-09-03T06:45:46.167 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0UOR7ER9dzeetPDULGBgciZEtEs= 2022-09-03T06:45:46.168 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0UOR7ER9dzeetPDULGBgciZEtEs=', 'Content-Length': '0'} 2022-09-03T06:45:46.171 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e9bc933c0fde053b-006312f81a-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.172 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:46.174 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:46.175 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:46.176 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:46.177 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:46.178 DEBUG:boto:Method: GET 2022-09-03T06:45:46.179 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:45:46.180 DEBUG:boto:Data: 2022-09-03T06:45:46.181 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.182 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:46.182 DEBUG:boto:Port: 8000 2022-09-03T06:45:46.183 DEBUG:boto:Params: {} 2022-09-03T06:45:46.183 DEBUG:boto:Token: None 2022-09-03T06:45:46.184 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/ 2022-09-03T06:45:46.185 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0UOR7ER9dzeetPDULGBgciZEtEs= 2022-09-03T06:45:46.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0UOR7ER9dzeetPDULGBgciZEtEs=', 'Content-Length': '0'} 2022-09-03T06:45:46.188 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005fb6ac22a4e513e4-006312f81a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.189 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:46.191 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:46.192 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:46.193 DEBUG:boto:Method: HEAD 2022-09-03T06:45:46.194 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:45:46.195 DEBUG:boto:Data: 2022-09-03T06:45:46.196 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.197 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:46.198 DEBUG:boto:Port: 8001 2022-09-03T06:45:46.198 DEBUG:boto:Params: {} 2022-09-03T06:45:46.200 DEBUG:boto:Token: None 2022-09-03T06:45:46.201 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj92 2022-09-03T06:45:46.202 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fozYtV4DvX2wq4OH3/xRRJ+SKHA= 2022-09-03T06:45:46.203 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fozYtV4DvX2wq4OH3/xRRJ+SKHA=', 'Content-Length': '0'} 2022-09-03T06:45:46.205 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ccb3b7a9a2f235f4-006312f81a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.207 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:46.208 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:46.209 DEBUG:boto:Method: HEAD 2022-09-03T06:45:46.209 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:45:46.210 DEBUG:boto:Data: 2022-09-03T06:45:46.211 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.211 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:46.212 DEBUG:boto:Port: 8000 2022-09-03T06:45:46.213 DEBUG:boto:Params: {} 2022-09-03T06:45:46.213 DEBUG:boto:Token: None 2022-09-03T06:45:46.214 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj92 2022-09-03T06:45:46.215 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fozYtV4DvX2wq4OH3/xRRJ+SKHA= 2022-09-03T06:45:46.215 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fozYtV4DvX2wq4OH3/xRRJ+SKHA=', 'Content-Length': '0'} 2022-09-03T06:45:46.218 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fa135beb31a5e739-006312f81a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.219 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:45:46.220 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:46.220 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:46.221 DEBUG:boto:Method: GET 2022-09-03T06:45:46.221 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:45:46.222 DEBUG:boto:Data: 2022-09-03T06:45:46.223 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.223 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:46.224 DEBUG:boto:Port: 8001 2022-09-03T06:45:46.225 DEBUG:boto:Params: {} 2022-09-03T06:45:46.225 DEBUG:boto:Token: None 2022-09-03T06:45:46.226 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj92 2022-09-03T06:45:46.227 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RPi/C/myUo93L2hXvi2hXiAmxck= 2022-09-03T06:45:46.227 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RPi/C/myUo93L2hXvi2hXiAmxck=', 'Content-Length': '0'} 2022-09-03T06:45:46.230 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000089c72390bac59222-006312f81a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.231 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:45:46.232 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:45:46.232 DEBUG:boto:Method: GET 2022-09-03T06:45:46.233 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:45:46.233 DEBUG:boto:Data: 2022-09-03T06:45:46.234 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.234 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:46.235 DEBUG:boto:Port: 8000 2022-09-03T06:45:46.235 DEBUG:boto:Params: {} 2022-09-03T06:45:46.236 DEBUG:boto:Token: None 2022-09-03T06:45:46.237 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj92 2022-09-03T06:45:46.237 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RPi/C/myUo93L2hXvi2hXiAmxck= 2022-09-03T06:45:46.238 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RPi/C/myUo93L2hXvi2hXiAmxck=', 'Content-Length': '0'} 2022-09-03T06:45:46.240 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c0805c5ed46264c2-006312f81a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.241 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:46.242 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:46.242 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:46.243 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:46.243 DEBUG:boto:Method: GET 2022-09-03T06:45:46.244 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:45:46.245 DEBUG:boto:Data: 2022-09-03T06:45:46.245 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.246 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:46.246 DEBUG:boto:Port: 8001 2022-09-03T06:45:46.247 DEBUG:boto:Params: {} 2022-09-03T06:45:46.248 DEBUG:boto:Token: None 2022-09-03T06:45:46.250 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/ 2022-09-03T06:45:46.251 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0UOR7ER9dzeetPDULGBgciZEtEs= 2022-09-03T06:45:46.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0UOR7ER9dzeetPDULGBgciZEtEs=', 'Content-Length': '0'} 2022-09-03T06:45:46.256 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d8037efdbb021d7c-006312f81a-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.257 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:46.259 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:46.260 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:46.261 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:46.262 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:45:46.263 DEBUG:boto:Method: GET 2022-09-03T06:45:46.264 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:45:46.265 DEBUG:boto:Data: 2022-09-03T06:45:46.266 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.267 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:46.268 DEBUG:boto:Port: 8000 2022-09-03T06:45:46.269 DEBUG:boto:Params: {} 2022-09-03T06:45:46.270 DEBUG:boto:Token: None 2022-09-03T06:45:46.271 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/ 2022-09-03T06:45:46.272 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0UOR7ER9dzeetPDULGBgciZEtEs= 2022-09-03T06:45:46.273 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0UOR7ER9dzeetPDULGBgciZEtEs=', 'Content-Length': '0'} 2022-09-03T06:45:46.277 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000052c100c8af322eef-006312f81a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.278 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:46.279 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:45:46.280 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:46.281 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:46.282 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.283 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.284 DEBUG:boto:Method: GET 2022-09-03T06:45:46.285 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.288 DEBUG:boto:Data: 2022-09-03T06:45:46.289 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.290 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:46.291 DEBUG:boto:Port: 8001 2022-09-03T06:45:46.292 DEBUG:boto:Params: {} 2022-09-03T06:45:46.293 DEBUG:boto:Token: None 2022-09-03T06:45:46.294 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.295 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZU/BC6z2J/L9YIqwL8Y6K7nKyXQ= 2022-09-03T06:45:46.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZU/BC6z2J/L9YIqwL8Y6K7nKyXQ=', 'Content-Length': '0'} 2022-09-03T06:45:46.298 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a05c3e810ea8c5f7-006312f81a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.346 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:46.347 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:46.348 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.348 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.349 DEBUG:boto:Method: GET 2022-09-03T06:45:46.349 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.350 DEBUG:boto:Data: 2022-09-03T06:45:46.351 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.351 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:46.352 DEBUG:boto:Port: 8000 2022-09-03T06:45:46.352 DEBUG:boto:Params: {} 2022-09-03T06:45:46.353 DEBUG:boto:Token: None 2022-09-03T06:45:46.354 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.354 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZU/BC6z2J/L9YIqwL8Y6K7nKyXQ= 2022-09-03T06:45:46.355 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZU/BC6z2J/L9YIqwL8Y6K7nKyXQ=', 'Content-Length': '0'} 2022-09-03T06:45:46.358 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004ccc54a3c2617721-006312f81a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.359 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:46.359 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:46.360 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.360 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.361 DEBUG:boto:Method: HEAD 2022-09-03T06:45:46.362 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.362 DEBUG:boto:Data: 2022-09-03T06:45:46.363 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.363 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:46.364 DEBUG:boto:Port: 8001 2022-09-03T06:45:46.364 DEBUG:boto:Params: {} 2022-09-03T06:45:46.365 DEBUG:boto:Token: None 2022-09-03T06:45:46.366 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zhMvvVs6PwoY2Y789+ufiDOXHbc= 2022-09-03T06:45:46.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zhMvvVs6PwoY2Y789+ufiDOXHbc=', 'Content-Length': '0'} 2022-09-03T06:45:46.370 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004c75a30d4c721dc7-006312f81a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.371 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:46.372 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:46.373 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.374 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.375 DEBUG:boto:Method: HEAD 2022-09-03T06:45:46.376 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.376 DEBUG:boto:Data: 2022-09-03T06:45:46.377 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.378 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:46.379 DEBUG:boto:Port: 8000 2022-09-03T06:45:46.379 DEBUG:boto:Params: {} 2022-09-03T06:45:46.380 DEBUG:boto:Token: None 2022-09-03T06:45:46.384 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.385 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zhMvvVs6PwoY2Y789+ufiDOXHbc= 2022-09-03T06:45:46.386 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zhMvvVs6PwoY2Y789+ufiDOXHbc=', 'Content-Length': '0'} 2022-09-03T06:45:46.389 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c80d057a7daa6eb-006312f81a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.390 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:45:46.390 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:46.391 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:46.392 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.392 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.393 DEBUG:boto:Method: GET 2022-09-03T06:45:46.393 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.394 DEBUG:boto:Data: 2022-09-03T06:45:46.395 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.395 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:46.396 DEBUG:boto:Port: 8001 2022-09-03T06:45:46.396 DEBUG:boto:Params: {} 2022-09-03T06:45:46.397 DEBUG:boto:Token: None 2022-09-03T06:45:46.398 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.398 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZU/BC6z2J/L9YIqwL8Y6K7nKyXQ= 2022-09-03T06:45:46.399 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZU/BC6z2J/L9YIqwL8Y6K7nKyXQ=', 'Content-Length': '0'} 2022-09-03T06:45:46.401 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000141193de4bbff00d-006312f81a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.402 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:46.403 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:46.403 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.404 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.405 DEBUG:boto:Method: GET 2022-09-03T06:45:46.405 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.406 DEBUG:boto:Data: 2022-09-03T06:45:46.406 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.407 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:46.407 DEBUG:boto:Port: 8000 2022-09-03T06:45:46.408 DEBUG:boto:Params: {} 2022-09-03T06:45:46.408 DEBUG:boto:Token: None 2022-09-03T06:45:46.409 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:45:46.410 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZU/BC6z2J/L9YIqwL8Y6K7nKyXQ= 2022-09-03T06:45:46.410 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZU/BC6z2J/L9YIqwL8Y6K7nKyXQ=', 'Content-Length': '0'} 2022-09-03T06:45:46.413 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000166976c672444184-006312f81a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.414 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:46.414 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:46.415 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:46.415 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:46.416 DEBUG:boto:Method: GET 2022-09-03T06:45:46.417 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:45:46.417 DEBUG:boto:Data: 2022-09-03T06:45:46.418 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.419 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:46.420 DEBUG:boto:Port: 8001 2022-09-03T06:45:46.421 DEBUG:boto:Params: {} 2022-09-03T06:45:46.422 DEBUG:boto:Token: None 2022-09-03T06:45:46.423 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/ 2022-09-03T06:45:46.424 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0UOR7ER9dzeetPDULGBgciZEtEs= 2022-09-03T06:45:46.425 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0UOR7ER9dzeetPDULGBgciZEtEs=', 'Content-Length': '0'} 2022-09-03T06:45:46.429 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e93e88c06c11ca87-006312f81a-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.430 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:46.431 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:46.432 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:46.433 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:46.434 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:46.434 DEBUG:boto:Method: GET 2022-09-03T06:45:46.435 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:45:46.436 DEBUG:boto:Data: 2022-09-03T06:45:46.437 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.438 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:46.439 DEBUG:boto:Port: 8000 2022-09-03T06:45:46.439 DEBUG:boto:Params: {} 2022-09-03T06:45:46.441 DEBUG:boto:Token: None 2022-09-03T06:45:46.442 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/ 2022-09-03T06:45:46.443 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0UOR7ER9dzeetPDULGBgciZEtEs= 2022-09-03T06:45:46.444 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0UOR7ER9dzeetPDULGBgciZEtEs=', 'Content-Length': '0'} 2022-09-03T06:45:46.447 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000244f3afc8a16eb07-006312f81a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.448 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:46.450 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:46.451 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:46.452 DEBUG:boto:Method: HEAD 2022-09-03T06:45:46.453 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:45:46.454 DEBUG:boto:Data: 2022-09-03T06:45:46.455 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.456 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:46.456 DEBUG:boto:Port: 8001 2022-09-03T06:45:46.457 DEBUG:boto:Params: {} 2022-09-03T06:45:46.458 DEBUG:boto:Token: None 2022-09-03T06:45:46.459 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj95 2022-09-03T06:45:46.460 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:02+MVDKnRZl0ml2tKsTAu8kDIuU= 2022-09-03T06:45:46.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:02+MVDKnRZl0ml2tKsTAu8kDIuU=', 'Content-Length': '0'} 2022-09-03T06:45:46.463 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000051f96a515381cabd-006312f81a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.465 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:46.465 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:46.466 DEBUG:boto:Method: HEAD 2022-09-03T06:45:46.467 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:45:46.467 DEBUG:boto:Data: 2022-09-03T06:45:46.468 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.468 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:46.469 DEBUG:boto:Port: 8000 2022-09-03T06:45:46.469 DEBUG:boto:Params: {} 2022-09-03T06:45:46.470 DEBUG:boto:Token: None 2022-09-03T06:45:46.470 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj95 2022-09-03T06:45:46.471 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:02+MVDKnRZl0ml2tKsTAu8kDIuU= 2022-09-03T06:45:46.472 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:02+MVDKnRZl0ml2tKsTAu8kDIuU=', 'Content-Length': '0'} 2022-09-03T06:45:46.474 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008f92e38d2ae94c91-006312f81a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.475 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:45:46.475 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:46.476 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:46.476 DEBUG:boto:Method: GET 2022-09-03T06:45:46.477 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:45:46.477 DEBUG:boto:Data: 2022-09-03T06:45:46.478 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.479 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:46.480 DEBUG:boto:Port: 8001 2022-09-03T06:45:46.480 DEBUG:boto:Params: {} 2022-09-03T06:45:46.481 DEBUG:boto:Token: None 2022-09-03T06:45:46.482 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj95 2022-09-03T06:45:46.482 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9VzPLOTcxy2E9mM5pzC5Z3+OqCs= 2022-09-03T06:45:46.483 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9VzPLOTcxy2E9mM5pzC5Z3+OqCs=', 'Content-Length': '0'} 2022-09-03T06:45:46.486 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db3779dcc168c23b-006312f81a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.487 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:45:46.488 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:45:46.488 DEBUG:boto:Method: GET 2022-09-03T06:45:46.489 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:45:46.490 DEBUG:boto:Data: 2022-09-03T06:45:46.490 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.491 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:46.491 DEBUG:boto:Port: 8000 2022-09-03T06:45:46.492 DEBUG:boto:Params: {} 2022-09-03T06:45:46.492 DEBUG:boto:Token: None 2022-09-03T06:45:46.493 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj95 2022-09-03T06:45:46.494 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9VzPLOTcxy2E9mM5pzC5Z3+OqCs= 2022-09-03T06:45:46.495 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9VzPLOTcxy2E9mM5pzC5Z3+OqCs=', 'Content-Length': '0'} 2022-09-03T06:45:46.498 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ed7b7f1957c08f9a-006312f81a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.499 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:46.500 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:46.501 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:46.501 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:46.502 DEBUG:boto:Method: GET 2022-09-03T06:45:46.503 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:45:46.504 DEBUG:boto:Data: 2022-09-03T06:45:46.504 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.505 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:45:46.506 DEBUG:boto:Port: 8001 2022-09-03T06:45:46.506 DEBUG:boto:Params: {} 2022-09-03T06:45:46.507 DEBUG:boto:Token: None 2022-09-03T06:45:46.508 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/ 2022-09-03T06:45:46.509 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0UOR7ER9dzeetPDULGBgciZEtEs= 2022-09-03T06:45:46.510 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0UOR7ER9dzeetPDULGBgciZEtEs=', 'Content-Length': '0'} 2022-09-03T06:45:46.512 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000014ff6d10c6a5911a-006312f81a-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.558 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:46.560 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:46.562 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:46.563 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:46.564 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:45:46.564 DEBUG:boto:Method: GET 2022-09-03T06:45:46.565 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:45:46.566 DEBUG:boto:Data: 2022-09-03T06:45:46.567 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.568 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:45:46.569 DEBUG:boto:Port: 8000 2022-09-03T06:45:46.569 DEBUG:boto:Params: {} 2022-09-03T06:45:46.570 DEBUG:boto:Token: None 2022-09-03T06:45:46.571 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/ 2022-09-03T06:45:46.572 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0UOR7ER9dzeetPDULGBgciZEtEs= 2022-09-03T06:45:46.573 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0UOR7ER9dzeetPDULGBgciZEtEs=', 'Content-Length': '0'} 2022-09-03T06:45:46.576 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d838dc2e191e9d92-006312f81a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.578 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:45:46.579 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-75 zones={test-zone3, test-zone2} 2022-09-03T06:45:46.580 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--num-shards', '31', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:45:46.581 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--num-shards', '31', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:45:46.582 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --num-shards 31 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T06:45:46.643 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:45:46.643 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:45:46.699 INFO:teuthology.orchestra.run.smithi181.stdout:tenant: 2022-09-03T06:45:46.699 INFO:teuthology.orchestra.run.smithi181.stdout:bucket name: dsgvcy-75 2022-09-03T06:45:46.712 INFO:teuthology.orchestra.run.smithi181.stdout:total entries: 25 2022-09-03T06:45:46.780 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:46.781 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:46.782 DEBUG:boto:Method: HEAD 2022-09-03T06:45:46.783 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:45:46.783 DEBUG:boto:Data: 2022-09-03T06:45:46.784 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.785 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:46.786 DEBUG:boto:Port: 8000 2022-09-03T06:45:46.787 DEBUG:boto:Params: {} 2022-09-03T06:45:46.788 DEBUG:boto:Token: None 2022-09-03T06:45:46.789 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/ 2022-09-03T06:45:46.790 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:thOv+50a6lvn1GTsDUh3hCffMN0= 2022-09-03T06:45:46.791 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:thOv+50a6lvn1GTsDUh3hCffMN0=', 'Content-Length': '0'} 2022-09-03T06:45:46.803 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '25'), ('X-RGW-Bytes-Used', '75'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000527005dc2762a0bf-006312f81a-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.804 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:45:46.805 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:45:46.805 DEBUG:boto:Method: PUT 2022-09-03T06:45:46.806 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:45:46.806 DEBUG:boto:Data: 2022-09-03T06:45:46.807 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:45:46.807 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:46.808 DEBUG:boto:Port: 8000 2022-09-03T06:45:46.808 DEBUG:boto:Params: {} 2022-09-03T06:45:46.809 DEBUG:boto:Token: None 2022-09-03T06:45:46.810 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj31 2022-09-03T06:45:46.810 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V9tflFfdUNpJLtbJhawGgoPYdoI= 2022-09-03T06:45:46.811 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V9tflFfdUNpJLtbJhawGgoPYdoI='} 2022-09-03T06:45:46.818 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000005c414ba394962ea2-006312f81a-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.820 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:46.821 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:46.822 DEBUG:boto:Method: HEAD 2022-09-03T06:45:46.823 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:45:46.824 DEBUG:boto:Data: 2022-09-03T06:45:46.825 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.826 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:46.827 DEBUG:boto:Port: 8000 2022-09-03T06:45:46.828 DEBUG:boto:Params: {} 2022-09-03T06:45:46.830 DEBUG:boto:Token: None 2022-09-03T06:45:46.831 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/ 2022-09-03T06:45:46.832 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:thOv+50a6lvn1GTsDUh3hCffMN0= 2022-09-03T06:45:46.833 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:thOv+50a6lvn1GTsDUh3hCffMN0=', 'Content-Length': '0'} 2022-09-03T06:45:46.838 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '26'), ('X-RGW-Bytes-Used', '78'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b1fe0f33753a8326-006312f81a-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.839 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:45:46.840 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:45:46.841 DEBUG:boto:Method: PUT 2022-09-03T06:45:46.842 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:45:46.843 DEBUG:boto:Data: 2022-09-03T06:45:46.844 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:45:46.844 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:46.845 DEBUG:boto:Port: 8000 2022-09-03T06:45:46.845 DEBUG:boto:Params: {} 2022-09-03T06:45:46.846 DEBUG:boto:Token: None 2022-09-03T06:45:46.847 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj62 2022-09-03T06:45:46.847 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZeRe/ON94vj8Q04HTy9h3UFYw9M= 2022-09-03T06:45:46.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZeRe/ON94vj8Q04HTy9h3UFYw9M='} 2022-09-03T06:45:46.890 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000df027bfcbc2dc35f-006312f81a-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.892 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:46.893 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:46.893 DEBUG:boto:Method: HEAD 2022-09-03T06:45:46.894 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:45:46.895 DEBUG:boto:Data: 2022-09-03T06:45:46.895 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.896 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:46.897 DEBUG:boto:Port: 8000 2022-09-03T06:45:46.897 DEBUG:boto:Params: {} 2022-09-03T06:45:46.904 DEBUG:boto:Token: None 2022-09-03T06:45:46.905 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/ 2022-09-03T06:45:46.906 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:thOv+50a6lvn1GTsDUh3hCffMN0= 2022-09-03T06:45:46.907 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:thOv+50a6lvn1GTsDUh3hCffMN0=', 'Content-Length': '0'} 2022-09-03T06:45:46.911 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '27'), ('X-RGW-Bytes-Used', '81'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000e097d2fc9cddeaff-006312f81a-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.913 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:45:46.914 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:45:46.914 DEBUG:boto:Method: PUT 2022-09-03T06:45:46.915 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:45:46.916 DEBUG:boto:Data: 2022-09-03T06:45:46.917 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:45:46.919 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:46.920 DEBUG:boto:Port: 8000 2022-09-03T06:45:46.921 DEBUG:boto:Params: {} 2022-09-03T06:45:46.922 DEBUG:boto:Token: None 2022-09-03T06:45:46.923 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj93 2022-09-03T06:45:46.924 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D8r7sUh2XZEFm8vtYJf/WmKNPzk= 2022-09-03T06:45:46.924 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D8r7sUh2XZEFm8vtYJf/WmKNPzk='} 2022-09-03T06:45:46.937 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000fa2824bdda6baba7-006312f81a-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.939 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:46.939 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:46.940 DEBUG:boto:Method: HEAD 2022-09-03T06:45:46.941 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:45:46.942 DEBUG:boto:Data: 2022-09-03T06:45:46.943 DEBUG:boto:Headers: {} 2022-09-03T06:45:46.944 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:46.944 DEBUG:boto:Port: 8000 2022-09-03T06:45:46.945 DEBUG:boto:Params: {} 2022-09-03T06:45:46.945 DEBUG:boto:Token: None 2022-09-03T06:45:46.946 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/ 2022-09-03T06:45:46.946 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:thOv+50a6lvn1GTsDUh3hCffMN0= 2022-09-03T06:45:46.947 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:thOv+50a6lvn1GTsDUh3hCffMN0=', 'Content-Length': '0'} 2022-09-03T06:45:46.952 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '28'), ('X-RGW-Bytes-Used', '84'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000247e05ad0955d3bc-006312f81a-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:46.953 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:45:46.954 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:45:46.955 DEBUG:boto:Method: PUT 2022-09-03T06:45:46.955 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:45:46.956 DEBUG:boto:Data: 2022-09-03T06:45:46.957 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:45:46.957 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:46.958 DEBUG:boto:Port: 8000 2022-09-03T06:45:46.959 DEBUG:boto:Params: {} 2022-09-03T06:45:46.959 DEBUG:boto:Token: None 2022-09-03T06:45:46.960 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:45:46 GMT /dsgvcy-75/obj124 2022-09-03T06:45:46.961 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uT6oJhfnZQhJnfYztKECeDb0nTk= 2022-09-03T06:45:46.962 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:45:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uT6oJhfnZQhJnfYztKECeDb0nTk='} 2022-09-03T06:45:47.010 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000f8bf70275dbe7334-006312f81a-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:47.012 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:45:47.013 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:45:47.013 DEBUG:boto:Method: HEAD 2022-09-03T06:45:47.014 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:45:47.014 DEBUG:boto:Data: 2022-09-03T06:45:47.015 DEBUG:boto:Headers: {} 2022-09-03T06:45:47.016 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:47.016 DEBUG:boto:Port: 8000 2022-09-03T06:45:47.017 DEBUG:boto:Params: {} 2022-09-03T06:45:47.018 DEBUG:boto:Token: None 2022-09-03T06:45:47.020 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:45:47 GMT /dsgvcy-75/ 2022-09-03T06:45:47.021 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O66bjR6Ksns09ofVYNaZy7orloI= 2022-09-03T06:45:47.022 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:45:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O66bjR6Ksns09ofVYNaZy7orloI=', 'Content-Length': '0'} 2022-09-03T06:45:47.026 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '29'), ('X-RGW-Bytes-Used', '87'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000962481ca14fa47f0-006312f81b-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:47.027 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:45:47.029 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:45:47.030 DEBUG:boto:Method: PUT 2022-09-03T06:45:47.031 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:45:47.031 DEBUG:boto:Data: 2022-09-03T06:45:47.032 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:45:47.032 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:45:47.033 DEBUG:boto:Port: 8000 2022-09-03T06:45:47.033 DEBUG:boto:Params: {} 2022-09-03T06:45:47.034 DEBUG:boto:Token: None 2022-09-03T06:45:47.034 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:45:47 GMT /dsgvcy-75/obj155 2022-09-03T06:45:47.035 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bhQ6k1pEjvGONQm+fKVh1w5uB+w= 2022-09-03T06:45:47.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:45:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bhQ6k1pEjvGONQm+fKVh1w5uB+w='} 2022-09-03T06:45:47.042 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000ce3afeaf4d64d142-006312f81b-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:45:47.044 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-75 2022-09-03T06:45:47.045 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:45:47.046 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:45:47.047 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:45:47.115 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:45:47.116 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:45:47.155 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:45:47.153+0000 7fd0ba577600 1 waiting to reach latest gen 5, on 4.. 2022-09-03T06:45:52.156 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:45:52.153+0000 7fd0ba577600 1 waiting to reach latest gen 5, on 4.. 2022-09-03T06:45:57.158 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:45:57.156+0000 7fd0ba577600 1 waiting for incremental sync to catch up: 2022-09-03T06:45:57.159 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2022-09-03T06:45:57.160 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , 00000000002.91.5, , 00000000001.115.5, , 00000000001.54.5, , , , , , , , , , 00000000001.86.5, , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:46:02.160 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:46:02.158+0000 7fd0ba577600 1 waiting for incremental sync to catch up: 2022-09-03T06:46:02.161 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2022-09-03T06:46:02.161 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , 00000000002.91.5, , 00000000001.115.5, , 00000000001.54.5, , , , , , , , , , 00000000001.86.5, , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:46:07.162 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:46:07.160+0000 7fd0ba577600 1 waiting for incremental sync to catch up: 2022-09-03T06:46:07.163 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2022-09-03T06:46:07.163 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , 00000000002.91.5, , 00000000001.115.5, , 00000000001.54.5, , , , , , , , , , 00000000001.86.5, , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:46:12.168 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:46:12.163+0000 7fd0ba577600 1 waiting for incremental sync to catch up: 2022-09-03T06:46:12.169 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2022-09-03T06:46:12.169 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , 00000000002.91.5, , 00000000001.115.5, , 00000000001.54.5, , , , , , , , , , 00000000001.86.5, , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:46:17.167 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:46:17.165+0000 7fd0ba577600 1 waiting for incremental sync to catch up: 2022-09-03T06:46:17.168 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000002.91.5, , , , 00000000001.54.5, , , , , , , , , , , , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:46:17.168 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , 00000000002.91.5, , 00000000001.115.5, , 00000000001.54.5, , , , , , , , , , 00000000001.86.5, , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:46:22.174 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:46:22.168+0000 7fd0ba577600 1 waiting for incremental sync to catch up: 2022-09-03T06:46:22.175 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000002.91.5, , , , 00000000001.54.5, , , , , , , , , , , , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:46:22.175 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , 00000000002.91.5, , 00000000001.115.5, , 00000000001.54.5, , , , , , , , , , 00000000001.86.5, , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:46:27.173 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:46:27.171+0000 7fd0ba577600 1 waiting for incremental sync to catch up: 2022-09-03T06:46:27.174 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000002.91.5, , , , 00000000001.54.5, , , , , , , , , , , , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:46:27.174 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , 00000000002.91.5, , 00000000001.115.5, , 00000000001.54.5, , , , , , , , , , 00000000001.86.5, , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:46:32.176 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:46:32.172+0000 7fd0ba577600 1 waiting for incremental sync to catch up: 2022-09-03T06:46:32.177 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000002.91.5, , , , 00000000001.54.5, , , , , , , , , , , , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:46:32.177 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , 00000000002.91.5, , 00000000001.115.5, , 00000000001.54.5, , , , , , , , , , 00000000001.86.5, , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:46:37.178 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:46:37.176+0000 7fd0ba577600 1 waiting for incremental sync to catch up: 2022-09-03T06:46:37.178 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000002.91.5, , , , 00000000001.54.5, , , , , , , , , , , , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:46:37.178 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , 00000000002.91.5, , 00000000001.115.5, , 00000000001.54.5, , , , , , , , , , 00000000001.86.5, , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:46:42.180 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:46:42.178+0000 7fd0ba577600 1 waiting for incremental sync to catch up: 2022-09-03T06:46:42.181 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000002.91.5, , , , 00000000001.54.5, , , , , , , , , , , , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:46:42.181 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , 00000000002.91.5, , 00000000001.115.5, , 00000000001.54.5, , , , , , , , , , 00000000001.86.5, , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:46:47.183 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:46:47.181+0000 7fd0ba577600 1 waiting for incremental sync to catch up: 2022-09-03T06:46:47.184 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000002.91.5, , , , 00000000001.54.5, , , , , , , , , , , , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:46:47.184 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , 00000000002.91.5, , 00000000001.115.5, , 00000000001.54.5, , , , , , , , , , 00000000001.86.5, , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:46:52.185 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:46:52.183+0000 7fd0ba577600 1 waiting for incremental sync to catch up: 2022-09-03T06:46:52.186 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000002.91.5, , , , 00000000001.54.5, , , , , , , , , , , , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:46:52.187 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , 00000000002.91.5, , 00000000001.115.5, , 00000000001.54.5, , , , , , , , , , 00000000001.86.5, , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:46:57.188 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:46:57.186+0000 7fd0ba577600 1 waiting for incremental sync to catch up: 2022-09-03T06:46:57.189 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000002.91.5, , , , 00000000001.54.5, , , , , , , , , , , , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:46:57.189 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , 00000000002.91.5, , 00000000001.115.5, , 00000000001.54.5, , , , , , , , , , 00000000001.86.5, , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:47:02.193 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:47:02.188+0000 7fd0ba577600 1 waiting for incremental sync to catch up: 2022-09-03T06:47:02.195 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000002.91.5, , , , 00000000001.54.5, , , , , , , , , , , , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:47:02.196 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , 00000000002.91.5, , 00000000001.115.5, , 00000000001.54.5, , , , , , , , , , 00000000001.86.5, , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:47:07.193 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:47:07.191+0000 7fd0ba577600 1 waiting for incremental sync to catch up: 2022-09-03T06:47:07.193 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000002.91.5, , , , 00000000001.54.5, , , , , , , , , , , , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:47:07.194 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , 00000000002.91.5, , 00000000001.115.5, , 00000000001.54.5, , , , , , , , , , 00000000001.86.5, , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:47:12.196 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:47:12.193+0000 7fd0ba577600 1 waiting for incremental sync to catch up: 2022-09-03T06:47:12.196 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000002.91.5, , , , 00000000001.54.5, , , , , , , , , , , , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:47:12.196 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , 00000000002.91.5, , 00000000001.115.5, , 00000000001.54.5, , , , , , , , , , 00000000001.86.5, , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:47:17.198 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:47:17.195+0000 7fd0ba577600 1 bucket sync caught up with source: 2022-09-03T06:47:17.199 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000002.91.5, , 00000000001.115.5, , 00000000001.54.5, , , , , , , , , , 00000000001.86.5, , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:47:17.199 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , 00000000002.91.5, , 00000000001.115.5, , 00000000001.54.5, , , , , , , , , , 00000000001.86.5, , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:47:17.200 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:47:17.195+0000 7fd0ba577600 0 bucket checkpoint complete 2022-09-03T06:47:17.218 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-75 2022-09-03T06:47:17.220 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:47:17.221 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:47:17.222 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:47:17.287 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:47:17.287 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:47:17.350 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:47:17.348+0000 7fecf5ee5600 1 bucket sync caught up with source: 2022-09-03T06:47:17.350 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , 00000000002.91.5, , 00000000001.115.5, , 00000000001.54.5, , , , , , , , , , 00000000001.86.5, , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:47:17.351 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , 00000000002.91.5, , 00000000001.115.5, , 00000000001.54.5, , , , , , , , , , 00000000001.86.5, , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:47:17.351 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:47:17.348+0000 7fecf5ee5600 0 bucket checkpoint complete 2022-09-03T06:47:17.375 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-75 2022-09-03T06:47:17.376 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:47:17.376 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:47:17.377 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:47:17.441 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:47:17.441 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:47:17.503 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:47:17.501+0000 7f269ce00600 1 bucket sync caught up with source: 2022-09-03T06:47:17.504 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , 00000000002.91.5, , 00000000001.115.5, , 00000000001.54.5, , , , , , , , , , 00000000001.86.5, , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:47:17.504 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , 00000000002.91.5, , 00000000001.115.5, , 00000000001.54.5, , , , , , , , , , 00000000001.86.5, , , , , , , , , , , , , , 00000000001.153.5] 2022-09-03T06:47:17.504 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:47:17.501+0000 7f269ce00600 0 bucket checkpoint complete 2022-09-03T06:47:17.524 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-75 2022-09-03T06:47:17.525 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:47:17.527 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:47:17.528 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:47:17.589 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:47:17.590 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:47:17.628 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:47:17.626+0000 7fa448ccc600 1 bucket sync caught up with source: 2022-09-03T06:47:17.629 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000003.590.5, 00000000002.739.5, 00000000002.386.5, 00000000004.715.5, 00000000004.380.5, 00000000001.364.5, 00000000001.865.5, 00000000003.871.5, 00000000006.318.5, 00000000004.587.5] 2022-09-03T06:47:17.630 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000003.590.5, 00000000002.739.5, 00000000002.386.5, 00000000004.715.5, 00000000004.380.5, 00000000001.364.5, 00000000001.865.5, 00000000003.871.5, 00000000006.318.5, 00000000004.587.5] 2022-09-03T06:47:17.630 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:47:17.626+0000 7fa448ccc600 0 bucket checkpoint complete 2022-09-03T06:47:17.642 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-75 2022-09-03T06:47:17.643 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:47:17.643 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:47:17.644 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:47:17.701 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:47:17.702 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:47:17.744 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:47:17.742+0000 7fb1ccfaa600 1 waiting for incremental sync to catch up: 2022-09-03T06:47:17.744 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000003.590.5, 00000000002.739.5, 00000000001.366.5, 00000000004.715.5, 00000000004.380.5, 00000000001.364.5, 00000000001.865.5, 00000000002.855.5, 00000000006.318.5, 00000000004.587.5] 2022-09-03T06:47:17.745 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000003.590.5, 00000000002.739.5, 00000000002.386.5, 00000000004.715.5, 00000000004.380.5, 00000000001.364.5, 00000000001.865.5, 00000000003.871.5, 00000000006.318.5, 00000000004.587.5] 2022-09-03T06:47:22.745 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:47:22.744+0000 7fb1ccfaa600 1 bucket sync caught up with source: 2022-09-03T06:47:22.746 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000003.590.5, 00000000002.739.5, 00000000002.386.5, 00000000004.715.5, 00000000004.380.5, 00000000001.364.5, 00000000001.865.5, 00000000003.871.5, 00000000006.318.5, 00000000004.587.5] 2022-09-03T06:47:22.746 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000003.590.5, 00000000002.739.5, 00000000002.386.5, 00000000004.715.5, 00000000004.380.5, 00000000001.364.5, 00000000001.865.5, 00000000003.871.5, 00000000006.318.5, 00000000004.587.5] 2022-09-03T06:47:22.746 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:47:22.744+0000 7fb1ccfaa600 0 bucket checkpoint complete 2022-09-03T06:47:22.765 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-75 2022-09-03T06:47:22.766 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-75 2022-09-03T06:47:22.767 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:47:22.767 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:47:22.768 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:47:22.830 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:47:22.831 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:47:22.875 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:47:22.866+0000 7eff1a248600 1 bucket sync caught up with source: 2022-09-03T06:47:22.876 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000003.210.5, 00000000003.158.5, 00000000002.291.5, 00000000004.276.5, 00000000004.285.5, 00000000001.249.5, 00000000001.327.5, 00000000003.333.5, 00000000006.253.5, 00000000004.290.5] 2022-09-03T06:47:22.876 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000003.210.5, 00000000003.158.5, 00000000002.291.5, 00000000004.276.5, 00000000004.285.5, 00000000001.249.5, 00000000001.327.5, 00000000003.333.5, 00000000006.253.5, 00000000004.290.5] 2022-09-03T06:47:22.876 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:47:22.866+0000 7eff1a248600 0 bucket checkpoint complete 2022-09-03T06:47:22.884 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-75 2022-09-03T06:47:22.885 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:47:22.886 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:47:22.886 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:47:22.954 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:47:22.954 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:47:22.992 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:47:22.988+0000 7f2dd30e6600 1 bucket sync caught up with source: 2022-09-03T06:47:22.993 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000003.210.5, 00000000003.158.5, 00000000002.291.5, 00000000004.276.5, 00000000004.285.5, 00000000001.249.5, 00000000001.327.5, 00000000003.333.5, 00000000006.253.5, 00000000004.290.5] 2022-09-03T06:47:22.994 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000003.210.5, 00000000003.158.5, 00000000002.291.5, 00000000004.276.5, 00000000004.285.5, 00000000001.249.5, 00000000001.327.5, 00000000003.333.5, 00000000006.253.5, 00000000004.290.5] 2022-09-03T06:47:22.994 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:47:22.988+0000 7f2dd30e6600 0 bucket checkpoint complete 2022-09-03T06:47:23.006 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-75 2022-09-03T06:47:23.008 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-75 zones={test-zone2, test-zone1} 2022-09-03T06:47:23.009 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:23.010 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:23.011 DEBUG:boto:Method: HEAD 2022-09-03T06:47:23.012 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:47:23.012 DEBUG:boto:Data: 2022-09-03T06:47:23.013 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.014 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.015 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.016 DEBUG:boto:Params: {} 2022-09-03T06:47:23.018 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:47:23.019 DEBUG:boto:Token: None 2022-09-03T06:47:23.020 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/ 2022-09-03T06:47:23.021 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IOZDZcxQa2H4KGZl/0I0f43NI0w= 2022-09-03T06:47:23.021 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IOZDZcxQa2H4KGZl/0I0f43NI0w=', 'Content-Length': '0'} 2022-09-03T06:47:23.026 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '30'), ('X-RGW-Bytes-Used', '90'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000467205a4b5da1de7-006312f87b-188c-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.027 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:23.027 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:23.028 DEBUG:boto:Method: HEAD 2022-09-03T06:47:23.029 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:47:23.029 DEBUG:boto:Data: 2022-09-03T06:47:23.030 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.030 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.031 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.032 DEBUG:boto:Params: {} 2022-09-03T06:47:23.032 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:47:23.033 DEBUG:boto:Token: None 2022-09-03T06:47:23.034 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/ 2022-09-03T06:47:23.035 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IOZDZcxQa2H4KGZl/0I0f43NI0w= 2022-09-03T06:47:23.035 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IOZDZcxQa2H4KGZl/0I0f43NI0w=', 'Content-Length': '0'} 2022-09-03T06:47:23.043 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '30'), ('X-RGW-Bytes-Used', '90'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000685d601a732d0a74-006312f87b-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.044 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:47:23.045 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:23.046 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:23.046 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:23.047 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:23.047 DEBUG:boto:Method: GET 2022-09-03T06:47:23.048 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:23.049 DEBUG:boto:Data: 2022-09-03T06:47:23.049 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.051 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.051 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.052 DEBUG:boto:Params: {} 2022-09-03T06:47:23.053 DEBUG:boto:Token: None 2022-09-03T06:47:23.053 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/?versions 2022-09-03T06:47:23.054 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4/w31qOCSLYjDP2qqS6I7i0Hcco= 2022-09-03T06:47:23.055 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4/w31qOCSLYjDP2qqS6I7i0Hcco=', 'Content-Length': '0'} 2022-09-03T06:47:23.058 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b9ea4bdaa28dca32-006312f87b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.059 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:23.062 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:47:23.062 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:47:23.063 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2022-09-03T06:47:23.063 DEBUG:tasks.rgw_multi.zone_rados:o=obj124 2022-09-03T06:47:23.064 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2022-09-03T06:47:23.065 DEBUG:tasks.rgw_multi.zone_rados:o=obj155 2022-09-03T06:47:23.065 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:47:23.066 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:47:23.066 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:47:23.067 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:47:23.068 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2022-09-03T06:47:23.068 DEBUG:tasks.rgw_multi.zone_rados:o=obj31 2022-09-03T06:47:23.069 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:47:23.069 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:47:23.070 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:47:23.070 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:47:23.071 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:47:23.071 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:47:23.072 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:47:23.072 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:47:23.073 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2022-09-03T06:47:23.074 DEBUG:tasks.rgw_multi.zone_rados:o=obj62 2022-09-03T06:47:23.074 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:47:23.075 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:47:23.076 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:47:23.076 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:47:23.077 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2022-09-03T06:47:23.078 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:47:23.078 DEBUG:tasks.rgw_multi.zone_rados:o=obj93 2022-09-03T06:47:23.079 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:47:23.080 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:47:23.080 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:23.081 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:23.081 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:23.082 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:23.083 DEBUG:boto:Method: GET 2022-09-03T06:47:23.083 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:23.084 DEBUG:boto:Data: 2022-09-03T06:47:23.084 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.085 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.085 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.086 DEBUG:boto:Params: {} 2022-09-03T06:47:23.086 DEBUG:boto:Token: None 2022-09-03T06:47:23.087 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/?versions 2022-09-03T06:47:23.087 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4/w31qOCSLYjDP2qqS6I7i0Hcco= 2022-09-03T06:47:23.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4/w31qOCSLYjDP2qqS6I7i0Hcco=', 'Content-Length': '0'} 2022-09-03T06:47:23.093 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000514e7b30a9bac783-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.094 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:23.097 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:47:23.098 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:47:23.099 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2022-09-03T06:47:23.099 DEBUG:tasks.rgw_multi.zone_rados:o=obj124 2022-09-03T06:47:23.100 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2022-09-03T06:47:23.100 DEBUG:tasks.rgw_multi.zone_rados:o=obj155 2022-09-03T06:47:23.101 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:47:23.101 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:47:23.102 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:47:23.103 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:47:23.104 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2022-09-03T06:47:23.104 DEBUG:tasks.rgw_multi.zone_rados:o=obj31 2022-09-03T06:47:23.105 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:47:23.106 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:47:23.107 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:47:23.107 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:47:23.108 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:47:23.109 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:47:23.109 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:47:23.110 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:47:23.110 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2022-09-03T06:47:23.111 DEBUG:tasks.rgw_multi.zone_rados:o=obj62 2022-09-03T06:47:23.111 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:47:23.112 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:47:23.112 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:47:23.113 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:47:23.113 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2022-09-03T06:47:23.114 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:47:23.114 DEBUG:tasks.rgw_multi.zone_rados:o=obj93 2022-09-03T06:47:23.115 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:47:23.116 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:23.117 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:23.117 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:23.118 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:23.119 DEBUG:boto:Method: GET 2022-09-03T06:47:23.120 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:23.121 DEBUG:boto:Data: 2022-09-03T06:47:23.122 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.123 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.124 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.125 DEBUG:boto:Params: {} 2022-09-03T06:47:23.126 DEBUG:boto:Token: None 2022-09-03T06:47:23.127 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/?versions 2022-09-03T06:47:23.128 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4/w31qOCSLYjDP2qqS6I7i0Hcco= 2022-09-03T06:47:23.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4/w31qOCSLYjDP2qqS6I7i0Hcco=', 'Content-Length': '0'} 2022-09-03T06:47:23.132 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ebe0e8ea4636d5b9-006312f87b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.133 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:23.135 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:23.136 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:23.136 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:23.137 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:23.138 DEBUG:boto:Method: GET 2022-09-03T06:47:23.139 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:23.140 DEBUG:boto:Data: 2022-09-03T06:47:23.141 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.142 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.143 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.144 DEBUG:boto:Params: {} 2022-09-03T06:47:23.145 DEBUG:boto:Token: None 2022-09-03T06:47:23.146 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/?versions 2022-09-03T06:47:23.148 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4/w31qOCSLYjDP2qqS6I7i0Hcco= 2022-09-03T06:47:23.149 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4/w31qOCSLYjDP2qqS6I7i0Hcco=', 'Content-Length': '0'} 2022-09-03T06:47:23.155 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d77398bdee96ce68-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.156 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:23.164 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:47:23.165 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:23.166 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:23.166 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.167 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.171 DEBUG:boto:Method: GET 2022-09-03T06:47:23.172 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.172 DEBUG:boto:Data: 2022-09-03T06:47:23.173 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.174 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.174 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.175 DEBUG:boto:Params: {} 2022-09-03T06:47:23.176 DEBUG:boto:Token: None 2022-09-03T06:47:23.176 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.177 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+sSg/hN+4OkVim9aqn5Z3pggHcY= 2022-09-03T06:47:23.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+sSg/hN+4OkVim9aqn5Z3pggHcY=', 'Content-Length': '0'} 2022-09-03T06:47:23.181 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cafb4c4944b0c8cb-006312f87b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.183 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:23.184 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:23.186 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.187 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.188 DEBUG:boto:Method: GET 2022-09-03T06:47:23.189 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.191 DEBUG:boto:Data: 2022-09-03T06:47:23.192 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.193 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.193 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.194 DEBUG:boto:Params: {} 2022-09-03T06:47:23.195 DEBUG:boto:Token: None 2022-09-03T06:47:23.195 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.196 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+sSg/hN+4OkVim9aqn5Z3pggHcY= 2022-09-03T06:47:23.197 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+sSg/hN+4OkVim9aqn5Z3pggHcY=', 'Content-Length': '0'} 2022-09-03T06:47:23.204 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f1117d7c22488340-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.205 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:23.205 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:23.206 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.207 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.207 DEBUG:boto:Method: HEAD 2022-09-03T06:47:23.208 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.208 DEBUG:boto:Data: 2022-09-03T06:47:23.209 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.210 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.211 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.211 DEBUG:boto:Params: {} 2022-09-03T06:47:23.212 DEBUG:boto:Token: None 2022-09-03T06:47:23.213 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.214 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Uy5nDPrUipSkdZjm5wU+FOpZt5I= 2022-09-03T06:47:23.215 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Uy5nDPrUipSkdZjm5wU+FOpZt5I=', 'Content-Length': '0'} 2022-09-03T06:47:23.217 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004c4e0a96fdfba983-006312f87b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.218 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:23.218 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:23.219 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.219 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.220 DEBUG:boto:Method: HEAD 2022-09-03T06:47:23.220 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.221 DEBUG:boto:Data: 2022-09-03T06:47:23.221 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.222 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.222 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.223 DEBUG:boto:Params: {} 2022-09-03T06:47:23.224 DEBUG:boto:Token: None 2022-09-03T06:47:23.225 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.226 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Uy5nDPrUipSkdZjm5wU+FOpZt5I= 2022-09-03T06:47:23.226 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Uy5nDPrUipSkdZjm5wU+FOpZt5I=', 'Content-Length': '0'} 2022-09-03T06:47:23.229 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c681c09947135354-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.230 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:47:23.238 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:23.239 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:23.239 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.240 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.240 DEBUG:boto:Method: GET 2022-09-03T06:47:23.241 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.241 DEBUG:boto:Data: 2022-09-03T06:47:23.242 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.243 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.243 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.244 DEBUG:boto:Params: {} 2022-09-03T06:47:23.244 DEBUG:boto:Token: None 2022-09-03T06:47:23.245 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.245 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+sSg/hN+4OkVim9aqn5Z3pggHcY= 2022-09-03T06:47:23.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+sSg/hN+4OkVim9aqn5Z3pggHcY=', 'Content-Length': '0'} 2022-09-03T06:47:23.248 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d75dc6b23dcc347f-006312f87b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.249 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:23.250 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:23.250 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.251 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.251 DEBUG:boto:Method: GET 2022-09-03T06:47:23.252 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.253 DEBUG:boto:Data: 2022-09-03T06:47:23.253 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.254 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.254 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.255 DEBUG:boto:Params: {} 2022-09-03T06:47:23.256 DEBUG:boto:Token: None 2022-09-03T06:47:23.256 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:23.257 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+sSg/hN+4OkVim9aqn5Z3pggHcY= 2022-09-03T06:47:23.257 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+sSg/hN+4OkVim9aqn5Z3pggHcY=', 'Content-Length': '0'} 2022-09-03T06:47:23.260 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001671334ae9f88d59-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.261 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:23.262 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:23.263 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:23.264 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:23.265 DEBUG:boto:Method: GET 2022-09-03T06:47:23.265 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:47:23.266 DEBUG:boto:Data: 2022-09-03T06:47:23.267 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.268 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.269 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.270 DEBUG:boto:Params: {} 2022-09-03T06:47:23.271 DEBUG:boto:Token: None 2022-09-03T06:47:23.271 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/ 2022-09-03T06:47:23.272 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo= 2022-09-03T06:47:23.272 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo=', 'Content-Length': '0'} 2022-09-03T06:47:23.275 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000078c49db28589b137-006312f87b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.276 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:23.277 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:23.278 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:23.278 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:23.279 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:23.279 DEBUG:boto:Method: GET 2022-09-03T06:47:23.280 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:47:23.281 DEBUG:boto:Data: 2022-09-03T06:47:23.282 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.282 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.283 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.283 DEBUG:boto:Params: {} 2022-09-03T06:47:23.284 DEBUG:boto:Token: None 2022-09-03T06:47:23.285 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/ 2022-09-03T06:47:23.286 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo= 2022-09-03T06:47:23.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo=', 'Content-Length': '0'} 2022-09-03T06:47:23.292 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ade56024a483379c-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.293 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:23.294 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:23.295 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:23.296 DEBUG:boto:Method: HEAD 2022-09-03T06:47:23.297 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:47:23.298 DEBUG:boto:Data: 2022-09-03T06:47:23.299 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.299 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.300 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.301 DEBUG:boto:Params: {} 2022-09-03T06:47:23.302 DEBUG:boto:Token: None 2022-09-03T06:47:23.303 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj11 2022-09-03T06:47:23.304 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jNrUdnFsxRpBMGO7d65xMT2OvWU= 2022-09-03T06:47:23.305 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jNrUdnFsxRpBMGO7d65xMT2OvWU=', 'Content-Length': '0'} 2022-09-03T06:47:23.308 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000027a8f551ec608bae-006312f87b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.309 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:23.310 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:23.311 DEBUG:boto:Method: HEAD 2022-09-03T06:47:23.311 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:47:23.312 DEBUG:boto:Data: 2022-09-03T06:47:23.313 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.314 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.315 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.316 DEBUG:boto:Params: {} 2022-09-03T06:47:23.317 DEBUG:boto:Token: None 2022-09-03T06:47:23.318 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj11 2022-09-03T06:47:23.319 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jNrUdnFsxRpBMGO7d65xMT2OvWU= 2022-09-03T06:47:23.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jNrUdnFsxRpBMGO7d65xMT2OvWU=', 'Content-Length': '0'} 2022-09-03T06:47:23.323 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ace5faafa9c503c4-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.324 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:47:23.325 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:23.325 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:23.326 DEBUG:boto:Method: GET 2022-09-03T06:47:23.327 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:47:23.327 DEBUG:boto:Data: 2022-09-03T06:47:23.328 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.328 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.329 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.330 DEBUG:boto:Params: {} 2022-09-03T06:47:23.330 DEBUG:boto:Token: None 2022-09-03T06:47:23.331 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj11 2022-09-03T06:47:23.332 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NL3NT29V2UPa+y+crfKYRB3hdA0= 2022-09-03T06:47:23.332 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NL3NT29V2UPa+y+crfKYRB3hdA0=', 'Content-Length': '0'} 2022-09-03T06:47:23.335 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e694f1c65ec9f3e2-006312f87b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.336 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:23.337 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:23.338 DEBUG:boto:Method: GET 2022-09-03T06:47:23.339 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:47:23.340 DEBUG:boto:Data: 2022-09-03T06:47:23.341 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.342 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.343 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.344 DEBUG:boto:Params: {} 2022-09-03T06:47:23.346 DEBUG:boto:Token: None 2022-09-03T06:47:23.347 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj11 2022-09-03T06:47:23.348 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NL3NT29V2UPa+y+crfKYRB3hdA0= 2022-09-03T06:47:23.350 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NL3NT29V2UPa+y+crfKYRB3hdA0=', 'Content-Length': '0'} 2022-09-03T06:47:23.353 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005a710133f8495865-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.398 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:23.400 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:23.401 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:23.401 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:23.403 DEBUG:boto:Method: GET 2022-09-03T06:47:23.404 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:47:23.405 DEBUG:boto:Data: 2022-09-03T06:47:23.406 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.409 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.410 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.411 DEBUG:boto:Params: {} 2022-09-03T06:47:23.411 DEBUG:boto:Token: None 2022-09-03T06:47:23.412 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/ 2022-09-03T06:47:23.413 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo= 2022-09-03T06:47:23.413 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo=', 'Content-Length': '0'} 2022-09-03T06:47:23.417 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000457e73ac4bbfc198-006312f87b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.417 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:23.419 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:23.420 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:23.421 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:23.421 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:23.422 DEBUG:boto:Method: GET 2022-09-03T06:47:23.422 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:47:23.423 DEBUG:boto:Data: 2022-09-03T06:47:23.424 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.424 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.425 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.426 DEBUG:boto:Params: {} 2022-09-03T06:47:23.426 DEBUG:boto:Token: None 2022-09-03T06:47:23.427 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/ 2022-09-03T06:47:23.428 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo= 2022-09-03T06:47:23.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo=', 'Content-Length': '0'} 2022-09-03T06:47:23.433 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d7714cd74115ea45-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.433 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:23.434 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:47:23.435 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:23.436 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:23.436 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.437 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.437 DEBUG:boto:Method: GET 2022-09-03T06:47:23.438 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.438 DEBUG:boto:Data: 2022-09-03T06:47:23.439 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.439 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.440 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.440 DEBUG:boto:Params: {} 2022-09-03T06:47:23.441 DEBUG:boto:Token: None 2022-09-03T06:47:23.442 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.442 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S3MQObNfaLa6sLCk0Q/aNyjv/H0= 2022-09-03T06:47:23.443 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S3MQObNfaLa6sLCk0Q/aNyjv/H0=', 'Content-Length': '0'} 2022-09-03T06:47:23.445 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e8efe7f0192fbe4-006312f87b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.446 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:23.447 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:23.448 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.448 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.449 DEBUG:boto:Method: GET 2022-09-03T06:47:23.450 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.450 DEBUG:boto:Data: 2022-09-03T06:47:23.451 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.451 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.452 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.453 DEBUG:boto:Params: {} 2022-09-03T06:47:23.453 DEBUG:boto:Token: None 2022-09-03T06:47:23.454 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.455 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S3MQObNfaLa6sLCk0Q/aNyjv/H0= 2022-09-03T06:47:23.455 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S3MQObNfaLa6sLCk0Q/aNyjv/H0=', 'Content-Length': '0'} 2022-09-03T06:47:23.458 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fb76a0bc648ec00f-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.458 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:23.459 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:23.460 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.460 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.461 DEBUG:boto:Method: HEAD 2022-09-03T06:47:23.461 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.462 DEBUG:boto:Data: 2022-09-03T06:47:23.462 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.463 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.464 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.464 DEBUG:boto:Params: {} 2022-09-03T06:47:23.465 DEBUG:boto:Token: None 2022-09-03T06:47:23.465 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.466 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7i8Xw6HtflmIk4xbGoy2GpNhXn4= 2022-09-03T06:47:23.467 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7i8Xw6HtflmIk4xbGoy2GpNhXn4=', 'Content-Length': '0'} 2022-09-03T06:47:23.469 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b89fe19caead58f3-006312f87b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.469 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:23.470 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:23.471 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.472 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.472 DEBUG:boto:Method: HEAD 2022-09-03T06:47:23.473 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.473 DEBUG:boto:Data: 2022-09-03T06:47:23.474 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.475 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.475 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.476 DEBUG:boto:Params: {} 2022-09-03T06:47:23.477 DEBUG:boto:Token: None 2022-09-03T06:47:23.477 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.478 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7i8Xw6HtflmIk4xbGoy2GpNhXn4= 2022-09-03T06:47:23.478 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7i8Xw6HtflmIk4xbGoy2GpNhXn4=', 'Content-Length': '0'} 2022-09-03T06:47:23.480 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aae3dd84803b9247-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.481 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:47:23.482 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:23.482 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:23.483 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.483 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.484 DEBUG:boto:Method: GET 2022-09-03T06:47:23.484 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.485 DEBUG:boto:Data: 2022-09-03T06:47:23.485 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.487 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.487 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.488 DEBUG:boto:Params: {} 2022-09-03T06:47:23.488 DEBUG:boto:Token: None 2022-09-03T06:47:23.489 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.490 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S3MQObNfaLa6sLCk0Q/aNyjv/H0= 2022-09-03T06:47:23.491 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S3MQObNfaLa6sLCk0Q/aNyjv/H0=', 'Content-Length': '0'} 2022-09-03T06:47:23.493 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000076e18a8947182a6e-006312f87b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.494 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:23.495 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:23.496 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.496 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.497 DEBUG:boto:Method: GET 2022-09-03T06:47:23.498 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.498 DEBUG:boto:Data: 2022-09-03T06:47:23.499 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.499 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.500 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.500 DEBUG:boto:Params: {} 2022-09-03T06:47:23.501 DEBUG:boto:Token: None 2022-09-03T06:47:23.502 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:23.502 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S3MQObNfaLa6sLCk0Q/aNyjv/H0= 2022-09-03T06:47:23.503 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S3MQObNfaLa6sLCk0Q/aNyjv/H0=', 'Content-Length': '0'} 2022-09-03T06:47:23.507 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d80a08d7957835f-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.508 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:23.508 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:23.509 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:23.509 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:23.510 DEBUG:boto:Method: GET 2022-09-03T06:47:23.510 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:47:23.511 DEBUG:boto:Data: 2022-09-03T06:47:23.512 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.512 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.513 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.513 DEBUG:boto:Params: {} 2022-09-03T06:47:23.514 DEBUG:boto:Token: None 2022-09-03T06:47:23.514 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/ 2022-09-03T06:47:23.515 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo= 2022-09-03T06:47:23.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo=', 'Content-Length': '0'} 2022-09-03T06:47:23.518 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008a6e5e2ffe1c2369-006312f87b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.519 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:23.520 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:23.520 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:23.521 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:23.521 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:23.522 DEBUG:boto:Method: GET 2022-09-03T06:47:23.522 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:47:23.523 DEBUG:boto:Data: 2022-09-03T06:47:23.523 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.524 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.525 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.525 DEBUG:boto:Params: {} 2022-09-03T06:47:23.526 DEBUG:boto:Token: None 2022-09-03T06:47:23.527 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/ 2022-09-03T06:47:23.527 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo= 2022-09-03T06:47:23.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo=', 'Content-Length': '0'} 2022-09-03T06:47:23.532 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005748b4cbef8a0718-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.574 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:23.575 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:23.576 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:23.577 DEBUG:boto:Method: HEAD 2022-09-03T06:47:23.577 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:47:23.578 DEBUG:boto:Data: 2022-09-03T06:47:23.578 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.579 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.579 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.580 DEBUG:boto:Params: {} 2022-09-03T06:47:23.580 DEBUG:boto:Token: None 2022-09-03T06:47:23.581 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj115 2022-09-03T06:47:23.582 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zTh+4/R97uz2eJBEBSo8cuddx1w= 2022-09-03T06:47:23.583 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zTh+4/R97uz2eJBEBSo8cuddx1w=', 'Content-Length': '0'} 2022-09-03T06:47:23.585 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c5ab27bcb8d24d3b-006312f87b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.586 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:23.586 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:23.587 DEBUG:boto:Method: HEAD 2022-09-03T06:47:23.587 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:47:23.588 DEBUG:boto:Data: 2022-09-03T06:47:23.588 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.589 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.589 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.590 DEBUG:boto:Params: {} 2022-09-03T06:47:23.590 DEBUG:boto:Token: None 2022-09-03T06:47:23.591 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj115 2022-09-03T06:47:23.592 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zTh+4/R97uz2eJBEBSo8cuddx1w= 2022-09-03T06:47:23.593 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zTh+4/R97uz2eJBEBSo8cuddx1w=', 'Content-Length': '0'} 2022-09-03T06:47:23.595 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f64346551c73a24e-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.596 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:47:23.596 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:23.597 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:23.598 DEBUG:boto:Method: GET 2022-09-03T06:47:23.599 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:47:23.600 DEBUG:boto:Data: 2022-09-03T06:47:23.601 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.601 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.602 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.603 DEBUG:boto:Params: {} 2022-09-03T06:47:23.604 DEBUG:boto:Token: None 2022-09-03T06:47:23.605 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj115 2022-09-03T06:47:23.605 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FaLrKFY7Zh/Jppu9UKzOO7zJ/bQ= 2022-09-03T06:47:23.606 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FaLrKFY7Zh/Jppu9UKzOO7zJ/bQ=', 'Content-Length': '0'} 2022-09-03T06:47:23.608 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000765aaf151dd01cb6-006312f87b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.610 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:23.611 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:23.611 DEBUG:boto:Method: GET 2022-09-03T06:47:23.612 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:47:23.612 DEBUG:boto:Data: 2022-09-03T06:47:23.613 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.614 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.614 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.615 DEBUG:boto:Params: {} 2022-09-03T06:47:23.616 DEBUG:boto:Token: None 2022-09-03T06:47:23.616 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj115 2022-09-03T06:47:23.617 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FaLrKFY7Zh/Jppu9UKzOO7zJ/bQ= 2022-09-03T06:47:23.618 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FaLrKFY7Zh/Jppu9UKzOO7zJ/bQ=', 'Content-Length': '0'} 2022-09-03T06:47:23.621 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004393a8f19823b638-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.622 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:23.624 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:23.624 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:23.625 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:23.625 DEBUG:boto:Method: GET 2022-09-03T06:47:23.626 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:47:23.627 DEBUG:boto:Data: 2022-09-03T06:47:23.628 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.629 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.630 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.631 DEBUG:boto:Params: {} 2022-09-03T06:47:23.632 DEBUG:boto:Token: None 2022-09-03T06:47:23.633 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/ 2022-09-03T06:47:23.634 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo= 2022-09-03T06:47:23.635 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo=', 'Content-Length': '0'} 2022-09-03T06:47:23.638 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005b07863229da0611-006312f87b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.639 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:23.640 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:23.641 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:23.641 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:23.642 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:23.642 DEBUG:boto:Method: GET 2022-09-03T06:47:23.643 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:47:23.644 DEBUG:boto:Data: 2022-09-03T06:47:23.644 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.645 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.646 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.646 DEBUG:boto:Params: {} 2022-09-03T06:47:23.647 DEBUG:boto:Token: None 2022-09-03T06:47:23.648 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/ 2022-09-03T06:47:23.648 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo= 2022-09-03T06:47:23.649 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo=', 'Content-Length': '0'} 2022-09-03T06:47:23.653 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000091e66c95962e44e9-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.654 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:23.655 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:47:23.655 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:23.656 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:23.657 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.657 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.658 DEBUG:boto:Method: GET 2022-09-03T06:47:23.658 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.659 DEBUG:boto:Data: 2022-09-03T06:47:23.660 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.660 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.661 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.661 DEBUG:boto:Params: {} 2022-09-03T06:47:23.662 DEBUG:boto:Token: None 2022-09-03T06:47:23.663 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.663 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ck0SypQEvtwpszqnfVcDinCOe/w= 2022-09-03T06:47:23.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ck0SypQEvtwpszqnfVcDinCOe/w=', 'Content-Length': '0'} 2022-09-03T06:47:23.666 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000758b94946762b13d-006312f87b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.667 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:23.668 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:23.669 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.669 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.670 DEBUG:boto:Method: GET 2022-09-03T06:47:23.670 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.671 DEBUG:boto:Data: 2022-09-03T06:47:23.671 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.672 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.672 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.673 DEBUG:boto:Params: {} 2022-09-03T06:47:23.674 DEBUG:boto:Token: None 2022-09-03T06:47:23.674 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.675 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ck0SypQEvtwpszqnfVcDinCOe/w= 2022-09-03T06:47:23.676 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ck0SypQEvtwpszqnfVcDinCOe/w=', 'Content-Length': '0'} 2022-09-03T06:47:23.678 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7de6427138c45c6-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.679 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:23.679 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:23.680 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.681 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.682 DEBUG:boto:Method: HEAD 2022-09-03T06:47:23.682 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.683 DEBUG:boto:Data: 2022-09-03T06:47:23.683 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.684 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.685 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.685 DEBUG:boto:Params: {} 2022-09-03T06:47:23.686 DEBUG:boto:Token: None 2022-09-03T06:47:23.686 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.689 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:moIuxUkSTIZSa7vLZlAutMVE4aw= 2022-09-03T06:47:23.689 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:moIuxUkSTIZSa7vLZlAutMVE4aw=', 'Content-Length': '0'} 2022-09-03T06:47:23.692 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000055defa28ec78a987-006312f87b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.693 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:23.693 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:23.694 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.695 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.696 DEBUG:boto:Method: HEAD 2022-09-03T06:47:23.696 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.697 DEBUG:boto:Data: 2022-09-03T06:47:23.697 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.698 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.699 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.699 DEBUG:boto:Params: {} 2022-09-03T06:47:23.700 DEBUG:boto:Token: None 2022-09-03T06:47:23.701 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.702 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:moIuxUkSTIZSa7vLZlAutMVE4aw= 2022-09-03T06:47:23.702 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:moIuxUkSTIZSa7vLZlAutMVE4aw=', 'Content-Length': '0'} 2022-09-03T06:47:23.705 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e94933d364b1f710-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.707 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:47:23.707 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:23.708 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:23.709 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.709 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.710 DEBUG:boto:Method: GET 2022-09-03T06:47:23.714 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.715 DEBUG:boto:Data: 2022-09-03T06:47:23.717 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.719 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.720 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.721 DEBUG:boto:Params: {} 2022-09-03T06:47:23.722 DEBUG:boto:Token: None 2022-09-03T06:47:23.723 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.725 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ck0SypQEvtwpszqnfVcDinCOe/w= 2022-09-03T06:47:23.726 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ck0SypQEvtwpszqnfVcDinCOe/w=', 'Content-Length': '0'} 2022-09-03T06:47:23.732 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e13ce5c0452f62ab-006312f87b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.778 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:23.779 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:23.780 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.782 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.783 DEBUG:boto:Method: GET 2022-09-03T06:47:23.784 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.785 DEBUG:boto:Data: 2022-09-03T06:47:23.787 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.788 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.789 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.790 DEBUG:boto:Params: {} 2022-09-03T06:47:23.790 DEBUG:boto:Token: None 2022-09-03T06:47:23.791 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:23.792 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ck0SypQEvtwpszqnfVcDinCOe/w= 2022-09-03T06:47:23.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ck0SypQEvtwpszqnfVcDinCOe/w=', 'Content-Length': '0'} 2022-09-03T06:47:23.795 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000588aa7c88bf44d0-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.796 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:23.797 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:23.797 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:23.798 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:23.798 DEBUG:boto:Method: GET 2022-09-03T06:47:23.799 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:47:23.800 DEBUG:boto:Data: 2022-09-03T06:47:23.800 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.801 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.801 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.802 DEBUG:boto:Params: {} 2022-09-03T06:47:23.803 DEBUG:boto:Token: None 2022-09-03T06:47:23.803 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/ 2022-09-03T06:47:23.804 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo= 2022-09-03T06:47:23.805 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo=', 'Content-Length': '0'} 2022-09-03T06:47:23.807 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000945dcd8d634f3721-006312f87b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.808 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:23.809 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:23.810 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:23.811 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:23.811 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:23.812 DEBUG:boto:Method: GET 2022-09-03T06:47:23.812 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:47:23.813 DEBUG:boto:Data: 2022-09-03T06:47:23.813 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.814 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.814 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.815 DEBUG:boto:Params: {} 2022-09-03T06:47:23.815 DEBUG:boto:Token: None 2022-09-03T06:47:23.816 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/ 2022-09-03T06:47:23.817 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo= 2022-09-03T06:47:23.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo=', 'Content-Length': '0'} 2022-09-03T06:47:23.821 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006fd71c9fe7d5e136-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.862 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:23.863 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:23.864 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:23.865 DEBUG:boto:Method: HEAD 2022-09-03T06:47:23.865 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:47:23.872 DEBUG:boto:Data: 2022-09-03T06:47:23.873 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.874 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.874 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.875 DEBUG:boto:Params: {} 2022-09-03T06:47:23.876 DEBUG:boto:Token: None 2022-09-03T06:47:23.876 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj116 2022-09-03T06:47:23.877 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LuSCEYtANqZe69sqRMjT5iNkQ5s= 2022-09-03T06:47:23.878 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LuSCEYtANqZe69sqRMjT5iNkQ5s=', 'Content-Length': '0'} 2022-09-03T06:47:23.880 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000036bd13b813bf96a1-006312f87b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.881 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:23.882 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:23.883 DEBUG:boto:Method: HEAD 2022-09-03T06:47:23.884 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:47:23.885 DEBUG:boto:Data: 2022-09-03T06:47:23.885 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.886 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.886 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.887 DEBUG:boto:Params: {} 2022-09-03T06:47:23.887 DEBUG:boto:Token: None 2022-09-03T06:47:23.888 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj116 2022-09-03T06:47:23.889 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LuSCEYtANqZe69sqRMjT5iNkQ5s= 2022-09-03T06:47:23.889 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LuSCEYtANqZe69sqRMjT5iNkQ5s=', 'Content-Length': '0'} 2022-09-03T06:47:23.892 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d53cec45d0878973-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.893 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:47:23.893 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:23.894 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:23.895 DEBUG:boto:Method: GET 2022-09-03T06:47:23.895 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:47:23.896 DEBUG:boto:Data: 2022-09-03T06:47:23.897 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.897 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.898 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.898 DEBUG:boto:Params: {} 2022-09-03T06:47:23.899 DEBUG:boto:Token: None 2022-09-03T06:47:23.900 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj116 2022-09-03T06:47:23.901 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vfs5Rqa2COfO5lYyTGOhd2nTzxk= 2022-09-03T06:47:23.902 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vfs5Rqa2COfO5lYyTGOhd2nTzxk=', 'Content-Length': '0'} 2022-09-03T06:47:23.905 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005bf241528b757e41-006312f87b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.906 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:23.907 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:23.908 DEBUG:boto:Method: GET 2022-09-03T06:47:23.908 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:47:23.909 DEBUG:boto:Data: 2022-09-03T06:47:23.909 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.910 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.911 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.911 DEBUG:boto:Params: {} 2022-09-03T06:47:23.912 DEBUG:boto:Token: None 2022-09-03T06:47:23.913 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj116 2022-09-03T06:47:23.913 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vfs5Rqa2COfO5lYyTGOhd2nTzxk= 2022-09-03T06:47:23.914 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vfs5Rqa2COfO5lYyTGOhd2nTzxk=', 'Content-Length': '0'} 2022-09-03T06:47:23.917 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000059d338bd8d18c202-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.919 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:23.919 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:23.920 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:23.921 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:23.921 DEBUG:boto:Method: GET 2022-09-03T06:47:23.922 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:47:23.923 DEBUG:boto:Data: 2022-09-03T06:47:23.923 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.924 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.924 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.925 DEBUG:boto:Params: {} 2022-09-03T06:47:23.926 DEBUG:boto:Token: None 2022-09-03T06:47:23.926 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/ 2022-09-03T06:47:23.927 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo= 2022-09-03T06:47:23.927 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo=', 'Content-Length': '0'} 2022-09-03T06:47:23.930 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009245ea878f3b833f-006312f87b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.931 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:23.932 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:23.933 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:23.933 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:23.934 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:23.934 DEBUG:boto:Method: GET 2022-09-03T06:47:23.935 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:47:23.936 DEBUG:boto:Data: 2022-09-03T06:47:23.937 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.937 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.938 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.939 DEBUG:boto:Params: {} 2022-09-03T06:47:23.939 DEBUG:boto:Token: None 2022-09-03T06:47:23.940 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/ 2022-09-03T06:47:23.940 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo= 2022-09-03T06:47:23.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TiuwBE0GCZUNNV1fivYrggQCVqo=', 'Content-Length': '0'} 2022-09-03T06:47:23.945 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005f570a3099616d8e-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.946 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:23.947 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2022-09-03T06:47:23.948 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:23.948 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:23.949 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:23.949 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:23.950 DEBUG:boto:Method: GET 2022-09-03T06:47:23.950 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:23.951 DEBUG:boto:Data: 2022-09-03T06:47:23.952 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.952 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.953 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.953 DEBUG:boto:Params: {} 2022-09-03T06:47:23.954 DEBUG:boto:Token: None 2022-09-03T06:47:23.955 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:23.955 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WlEx469gfk7drqRAlTQPGW2cvhg= 2022-09-03T06:47:23.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WlEx469gfk7drqRAlTQPGW2cvhg=', 'Content-Length': '0'} 2022-09-03T06:47:23.959 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000860b53eb0b39d7f7-006312f87b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.960 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:23.961 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:23.962 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:23.963 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:23.964 DEBUG:boto:Method: GET 2022-09-03T06:47:23.965 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:23.966 DEBUG:boto:Data: 2022-09-03T06:47:23.967 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.967 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.968 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.969 DEBUG:boto:Params: {} 2022-09-03T06:47:23.970 DEBUG:boto:Token: None 2022-09-03T06:47:23.971 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:23.972 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WlEx469gfk7drqRAlTQPGW2cvhg= 2022-09-03T06:47:23.973 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WlEx469gfk7drqRAlTQPGW2cvhg=', 'Content-Length': '0'} 2022-09-03T06:47:23.976 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003bc88dfce8a242c0-006312f87b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.977 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:23.978 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:23.979 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:23.980 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:23.981 DEBUG:boto:Method: HEAD 2022-09-03T06:47:23.981 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:23.982 DEBUG:boto:Data: 2022-09-03T06:47:23.982 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.983 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:23.983 DEBUG:boto:Port: 8000 2022-09-03T06:47:23.984 DEBUG:boto:Params: {} 2022-09-03T06:47:23.984 DEBUG:boto:Token: None 2022-09-03T06:47:23.985 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:23 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:23.985 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wP6i81H3QfxkOEP6yZXcGEWEpX4= 2022-09-03T06:47:23.986 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wP6i81H3QfxkOEP6yZXcGEWEpX4=', 'Content-Length': '0'} 2022-09-03T06:47:23.990 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002059514a8e6b4383-006312f87b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:23.991 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:23.992 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:23.993 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:23.994 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:23.995 DEBUG:boto:Method: HEAD 2022-09-03T06:47:23.996 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:23.997 DEBUG:boto:Data: 2022-09-03T06:47:23.997 DEBUG:boto:Headers: {} 2022-09-03T06:47:23.998 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:23.999 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.001 DEBUG:boto:Params: {} 2022-09-03T06:47:24.002 DEBUG:boto:Token: None 2022-09-03T06:47:24.003 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:24.004 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VdHrMC5IjsiMDp8zFJSjYjGKFOE= 2022-09-03T06:47:24.005 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VdHrMC5IjsiMDp8zFJSjYjGKFOE=', 'Content-Length': '0'} 2022-09-03T06:47:24.008 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000facdeca95fa4f8e4-006312f87c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.009 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2022-09-03T06:47:24.010 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:24.011 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:24.012 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:24.013 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:24.014 DEBUG:boto:Method: GET 2022-09-03T06:47:24.015 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:24.016 DEBUG:boto:Data: 2022-09-03T06:47:24.017 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.018 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:24.019 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.020 DEBUG:boto:Params: {} 2022-09-03T06:47:24.021 DEBUG:boto:Token: None 2022-09-03T06:47:24.023 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:24.024 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wsf7N/I6hZjSG5COxRsfM94gcDU= 2022-09-03T06:47:24.025 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wsf7N/I6hZjSG5COxRsfM94gcDU=', 'Content-Length': '0'} 2022-09-03T06:47:24.028 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cda82c768160e131-006312f87c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.070 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:24.071 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:24.071 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:24.072 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:24.073 DEBUG:boto:Method: GET 2022-09-03T06:47:24.073 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:24.074 DEBUG:boto:Data: 2022-09-03T06:47:24.074 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.075 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:24.076 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.076 DEBUG:boto:Params: {} 2022-09-03T06:47:24.077 DEBUG:boto:Token: None 2022-09-03T06:47:24.078 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:24.078 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wsf7N/I6hZjSG5COxRsfM94gcDU= 2022-09-03T06:47:24.079 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wsf7N/I6hZjSG5COxRsfM94gcDU=', 'Content-Length': '0'} 2022-09-03T06:47:24.081 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006e794d2d81905136-006312f87c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.082 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:24.083 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:24.084 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:24.084 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:24.085 DEBUG:boto:Method: GET 2022-09-03T06:47:24.085 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:47:24.086 DEBUG:boto:Data: 2022-09-03T06:47:24.088 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.088 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:24.089 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.089 DEBUG:boto:Params: {} 2022-09-03T06:47:24.090 DEBUG:boto:Token: None 2022-09-03T06:47:24.091 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/ 2022-09-03T06:47:24.092 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk= 2022-09-03T06:47:24.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk=', 'Content-Length': '0'} 2022-09-03T06:47:24.095 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000059487c0066697bd2-006312f87c-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.096 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:24.097 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:24.098 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:24.098 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:24.099 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:24.100 DEBUG:boto:Method: GET 2022-09-03T06:47:24.100 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:47:24.101 DEBUG:boto:Data: 2022-09-03T06:47:24.102 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.102 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:24.103 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.104 DEBUG:boto:Params: {} 2022-09-03T06:47:24.105 DEBUG:boto:Token: None 2022-09-03T06:47:24.105 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/ 2022-09-03T06:47:24.106 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk= 2022-09-03T06:47:24.107 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk=', 'Content-Length': '0'} 2022-09-03T06:47:24.111 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000088440741b362f4bf-006312f87c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.113 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:24.114 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:24.114 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:24.115 DEBUG:boto:Method: HEAD 2022-09-03T06:47:24.115 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:47:24.116 DEBUG:boto:Data: 2022-09-03T06:47:24.117 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.119 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:24.120 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.121 DEBUG:boto:Params: {} 2022-09-03T06:47:24.122 DEBUG:boto:Token: None 2022-09-03T06:47:24.124 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj124 2022-09-03T06:47:24.125 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oyhT2zACgl0RlpC29lpHNgyrjK8= 2022-09-03T06:47:24.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oyhT2zACgl0RlpC29lpHNgyrjK8=', 'Content-Length': '0'} 2022-09-03T06:47:24.131 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d183e5bd0ed8746-006312f87c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.132 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:24.133 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:24.135 DEBUG:boto:Method: HEAD 2022-09-03T06:47:24.136 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:47:24.136 DEBUG:boto:Data: 2022-09-03T06:47:24.137 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.137 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:24.138 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.139 DEBUG:boto:Params: {} 2022-09-03T06:47:24.140 DEBUG:boto:Token: None 2022-09-03T06:47:24.141 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj124 2022-09-03T06:47:24.142 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oyhT2zACgl0RlpC29lpHNgyrjK8= 2022-09-03T06:47:24.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oyhT2zACgl0RlpC29lpHNgyrjK8=', 'Content-Length': '0'} 2022-09-03T06:47:24.148 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002bd71f21c11dd472-006312f87c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.150 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2022-09-03T06:47:24.151 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:24.152 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:24.153 DEBUG:boto:Method: GET 2022-09-03T06:47:24.154 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:47:24.155 DEBUG:boto:Data: 2022-09-03T06:47:24.156 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.157 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:24.158 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.159 DEBUG:boto:Params: {} 2022-09-03T06:47:24.160 DEBUG:boto:Token: None 2022-09-03T06:47:24.160 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj124 2022-09-03T06:47:24.161 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ec1M6pfnS7+xLx9dcmuDTxB+Jzc= 2022-09-03T06:47:24.162 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ec1M6pfnS7+xLx9dcmuDTxB+Jzc=', 'Content-Length': '0'} 2022-09-03T06:47:24.164 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000536f51a394f59ff0-006312f87c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.207 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:24.207 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:24.208 DEBUG:boto:Method: GET 2022-09-03T06:47:24.208 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:47:24.209 DEBUG:boto:Data: 2022-09-03T06:47:24.210 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.211 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:24.212 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.212 DEBUG:boto:Params: {} 2022-09-03T06:47:24.213 DEBUG:boto:Token: None 2022-09-03T06:47:24.214 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj124 2022-09-03T06:47:24.215 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ec1M6pfnS7+xLx9dcmuDTxB+Jzc= 2022-09-03T06:47:24.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ec1M6pfnS7+xLx9dcmuDTxB+Jzc=', 'Content-Length': '0'} 2022-09-03T06:47:24.218 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002bfa092028f2b3a7-006312f87c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.219 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:24.220 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:24.221 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:24.222 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:24.223 DEBUG:boto:Method: GET 2022-09-03T06:47:24.224 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:47:24.225 DEBUG:boto:Data: 2022-09-03T06:47:24.226 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.228 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:24.229 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.232 DEBUG:boto:Params: {} 2022-09-03T06:47:24.233 DEBUG:boto:Token: None 2022-09-03T06:47:24.235 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/ 2022-09-03T06:47:24.236 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk= 2022-09-03T06:47:24.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk=', 'Content-Length': '0'} 2022-09-03T06:47:24.241 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004b147c53af8bfe3e-006312f87c-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.242 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:24.244 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:24.245 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:24.245 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:24.246 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:24.247 DEBUG:boto:Method: GET 2022-09-03T06:47:24.248 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:47:24.249 DEBUG:boto:Data: 2022-09-03T06:47:24.250 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.250 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:24.251 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.252 DEBUG:boto:Params: {} 2022-09-03T06:47:24.252 DEBUG:boto:Token: None 2022-09-03T06:47:24.253 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/ 2022-09-03T06:47:24.254 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk= 2022-09-03T06:47:24.255 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk=', 'Content-Length': '0'} 2022-09-03T06:47:24.259 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003c7dd2df5f481473-006312f87c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.302 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:24.303 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:47:24.304 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:24.304 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:24.305 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.305 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.306 DEBUG:boto:Method: GET 2022-09-03T06:47:24.306 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.307 DEBUG:boto:Data: 2022-09-03T06:47:24.308 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.308 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:24.309 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.309 DEBUG:boto:Params: {} 2022-09-03T06:47:24.310 DEBUG:boto:Token: None 2022-09-03T06:47:24.311 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.312 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KMy5QB538bAR8Ot3XyAUy2jHFWo= 2022-09-03T06:47:24.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KMy5QB538bAR8Ot3XyAUy2jHFWo=', 'Content-Length': '0'} 2022-09-03T06:47:24.314 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c59a28718145207c-006312f87c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.315 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:24.316 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:24.317 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.317 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.318 DEBUG:boto:Method: GET 2022-09-03T06:47:24.319 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.319 DEBUG:boto:Data: 2022-09-03T06:47:24.320 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.320 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:24.321 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.322 DEBUG:boto:Params: {} 2022-09-03T06:47:24.322 DEBUG:boto:Token: None 2022-09-03T06:47:24.323 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.324 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KMy5QB538bAR8Ot3XyAUy2jHFWo= 2022-09-03T06:47:24.325 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KMy5QB538bAR8Ot3XyAUy2jHFWo=', 'Content-Length': '0'} 2022-09-03T06:47:24.328 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c4fa9d26aedf7b2a-006312f87c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.329 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:24.330 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:24.331 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.331 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.332 DEBUG:boto:Method: HEAD 2022-09-03T06:47:24.332 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.333 DEBUG:boto:Data: 2022-09-03T06:47:24.334 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.334 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:24.335 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.336 DEBUG:boto:Params: {} 2022-09-03T06:47:24.337 DEBUG:boto:Token: None 2022-09-03T06:47:24.337 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.339 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IuCMNcpK/hKQriwVgVja/Mg+0tI= 2022-09-03T06:47:24.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IuCMNcpK/hKQriwVgVja/Mg+0tI=', 'Content-Length': '0'} 2022-09-03T06:47:24.343 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000012fc9c9b41292c93-006312f87c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.344 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:24.344 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:24.345 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.345 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.346 DEBUG:boto:Method: HEAD 2022-09-03T06:47:24.346 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.347 DEBUG:boto:Data: 2022-09-03T06:47:24.348 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.349 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:24.350 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.351 DEBUG:boto:Params: {} 2022-09-03T06:47:24.352 DEBUG:boto:Token: None 2022-09-03T06:47:24.353 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.355 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IuCMNcpK/hKQriwVgVja/Mg+0tI= 2022-09-03T06:47:24.355 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IuCMNcpK/hKQriwVgVja/Mg+0tI=', 'Content-Length': '0'} 2022-09-03T06:47:24.358 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000abb45a8c6c7acf6d-006312f87c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.359 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:47:24.359 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:24.360 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:24.361 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.362 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.362 DEBUG:boto:Method: GET 2022-09-03T06:47:24.363 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.364 DEBUG:boto:Data: 2022-09-03T06:47:24.364 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.365 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:24.365 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.366 DEBUG:boto:Params: {} 2022-09-03T06:47:24.367 DEBUG:boto:Token: None 2022-09-03T06:47:24.367 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.368 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KMy5QB538bAR8Ot3XyAUy2jHFWo= 2022-09-03T06:47:24.370 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KMy5QB538bAR8Ot3XyAUy2jHFWo=', 'Content-Length': '0'} 2022-09-03T06:47:24.373 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000000a12417a40cb796-006312f87c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.421 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:24.422 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:24.423 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.424 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.425 DEBUG:boto:Method: GET 2022-09-03T06:47:24.426 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.427 DEBUG:boto:Data: 2022-09-03T06:47:24.428 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.429 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:24.431 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.432 DEBUG:boto:Params: {} 2022-09-03T06:47:24.433 DEBUG:boto:Token: None 2022-09-03T06:47:24.434 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:24.435 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KMy5QB538bAR8Ot3XyAUy2jHFWo= 2022-09-03T06:47:24.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KMy5QB538bAR8Ot3XyAUy2jHFWo=', 'Content-Length': '0'} 2022-09-03T06:47:24.439 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f7d678dde3dc5a86-006312f87c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.440 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:24.441 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:24.442 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:24.443 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:24.443 DEBUG:boto:Method: GET 2022-09-03T06:47:24.444 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:47:24.445 DEBUG:boto:Data: 2022-09-03T06:47:24.446 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.447 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:24.448 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.448 DEBUG:boto:Params: {} 2022-09-03T06:47:24.449 DEBUG:boto:Token: None 2022-09-03T06:47:24.450 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/ 2022-09-03T06:47:24.451 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk= 2022-09-03T06:47:24.453 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk=', 'Content-Length': '0'} 2022-09-03T06:47:24.456 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b96e9529b04e6789-006312f87c-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.458 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:24.460 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:24.461 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:24.462 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:24.463 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:24.464 DEBUG:boto:Method: GET 2022-09-03T06:47:24.465 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:47:24.465 DEBUG:boto:Data: 2022-09-03T06:47:24.466 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.467 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:24.468 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.469 DEBUG:boto:Params: {} 2022-09-03T06:47:24.471 DEBUG:boto:Token: None 2022-09-03T06:47:24.472 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/ 2022-09-03T06:47:24.473 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk= 2022-09-03T06:47:24.474 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk=', 'Content-Length': '0'} 2022-09-03T06:47:24.490 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a9dfcc06a01b08d0-006312f87c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.491 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:24.493 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:24.494 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:24.495 DEBUG:boto:Method: HEAD 2022-09-03T06:47:24.495 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:47:24.496 DEBUG:boto:Data: 2022-09-03T06:47:24.497 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.497 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:24.498 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.498 DEBUG:boto:Params: {} 2022-09-03T06:47:24.499 DEBUG:boto:Token: None 2022-09-03T06:47:24.500 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj145 2022-09-03T06:47:24.500 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GgkZRSym6a3ULzsAlPWYtU7pPac= 2022-09-03T06:47:24.501 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GgkZRSym6a3ULzsAlPWYtU7pPac=', 'Content-Length': '0'} 2022-09-03T06:47:24.503 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009651cfdd5106d442-006312f87c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.504 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:24.505 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:24.506 DEBUG:boto:Method: HEAD 2022-09-03T06:47:24.507 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:47:24.507 DEBUG:boto:Data: 2022-09-03T06:47:24.508 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.509 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:24.510 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.511 DEBUG:boto:Params: {} 2022-09-03T06:47:24.512 DEBUG:boto:Token: None 2022-09-03T06:47:24.513 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj145 2022-09-03T06:47:24.514 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GgkZRSym6a3ULzsAlPWYtU7pPac= 2022-09-03T06:47:24.515 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GgkZRSym6a3ULzsAlPWYtU7pPac=', 'Content-Length': '0'} 2022-09-03T06:47:24.518 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000749345850f26e743-006312f87c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.520 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:47:24.520 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:24.521 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:24.521 DEBUG:boto:Method: GET 2022-09-03T06:47:24.522 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:47:24.523 DEBUG:boto:Data: 2022-09-03T06:47:24.523 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.524 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:24.525 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.525 DEBUG:boto:Params: {} 2022-09-03T06:47:24.526 DEBUG:boto:Token: None 2022-09-03T06:47:24.526 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj145 2022-09-03T06:47:24.527 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eduQEF1wlLKcL/VeP3+83FxlJRw= 2022-09-03T06:47:24.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eduQEF1wlLKcL/VeP3+83FxlJRw=', 'Content-Length': '0'} 2022-09-03T06:47:24.530 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000067fad5a4e9d6e398-006312f87c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.531 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:24.532 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:24.533 DEBUG:boto:Method: GET 2022-09-03T06:47:24.534 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:47:24.535 DEBUG:boto:Data: 2022-09-03T06:47:24.536 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.537 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:24.538 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.538 DEBUG:boto:Params: {} 2022-09-03T06:47:24.540 DEBUG:boto:Token: None 2022-09-03T06:47:24.542 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj145 2022-09-03T06:47:24.543 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eduQEF1wlLKcL/VeP3+83FxlJRw= 2022-09-03T06:47:24.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eduQEF1wlLKcL/VeP3+83FxlJRw=', 'Content-Length': '0'} 2022-09-03T06:47:24.548 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002369bfd5d9bc5bd5-006312f87c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.590 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:24.592 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:24.592 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:24.593 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:24.597 DEBUG:boto:Method: GET 2022-09-03T06:47:24.598 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:47:24.599 DEBUG:boto:Data: 2022-09-03T06:47:24.602 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.603 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:24.603 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.604 DEBUG:boto:Params: {} 2022-09-03T06:47:24.605 DEBUG:boto:Token: None 2022-09-03T06:47:24.605 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/ 2022-09-03T06:47:24.606 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk= 2022-09-03T06:47:24.607 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk=', 'Content-Length': '0'} 2022-09-03T06:47:24.613 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f76551075bc41291-006312f87c-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.614 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:24.615 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:24.616 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:24.618 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:24.619 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:24.620 DEBUG:boto:Method: GET 2022-09-03T06:47:24.621 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:47:24.622 DEBUG:boto:Data: 2022-09-03T06:47:24.623 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.624 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:24.625 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.625 DEBUG:boto:Params: {} 2022-09-03T06:47:24.626 DEBUG:boto:Token: None 2022-09-03T06:47:24.627 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/ 2022-09-03T06:47:24.628 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk= 2022-09-03T06:47:24.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk=', 'Content-Length': '0'} 2022-09-03T06:47:24.633 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002b6ef266541d34fa-006312f87c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.634 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:24.635 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2022-09-03T06:47:24.636 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:24.636 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:24.637 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.637 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.638 DEBUG:boto:Method: GET 2022-09-03T06:47:24.638 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.639 DEBUG:boto:Data: 2022-09-03T06:47:24.640 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.640 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:24.641 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.642 DEBUG:boto:Params: {} 2022-09-03T06:47:24.642 DEBUG:boto:Token: None 2022-09-03T06:47:24.643 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.644 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V6k1gqVTKYa5N7BcQ3T2cnpHrBA= 2022-09-03T06:47:24.644 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V6k1gqVTKYa5N7BcQ3T2cnpHrBA=', 'Content-Length': '0'} 2022-09-03T06:47:24.647 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000624ccdceb785d643-006312f87c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.648 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:24.649 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:24.650 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.650 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.651 DEBUG:boto:Method: GET 2022-09-03T06:47:24.651 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.653 DEBUG:boto:Data: 2022-09-03T06:47:24.653 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.654 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:24.654 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.655 DEBUG:boto:Params: {} 2022-09-03T06:47:24.656 DEBUG:boto:Token: None 2022-09-03T06:47:24.657 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.657 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V6k1gqVTKYa5N7BcQ3T2cnpHrBA= 2022-09-03T06:47:24.658 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V6k1gqVTKYa5N7BcQ3T2cnpHrBA=', 'Content-Length': '0'} 2022-09-03T06:47:24.661 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000011c4aa5ac606232e-006312f87c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.662 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:24.664 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:24.664 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.665 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.665 DEBUG:boto:Method: HEAD 2022-09-03T06:47:24.678 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.679 DEBUG:boto:Data: 2022-09-03T06:47:24.679 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.680 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:24.681 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.681 DEBUG:boto:Params: {} 2022-09-03T06:47:24.682 DEBUG:boto:Token: None 2022-09-03T06:47:24.683 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.683 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gAQ3Z91KYPbsdhyjm0/ygLlj3Ek= 2022-09-03T06:47:24.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gAQ3Z91KYPbsdhyjm0/ygLlj3Ek=', 'Content-Length': '0'} 2022-09-03T06:47:24.691 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006830362d49aa631e-006312f87c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.693 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:24.694 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:24.695 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.696 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.696 DEBUG:boto:Method: HEAD 2022-09-03T06:47:24.698 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.699 DEBUG:boto:Data: 2022-09-03T06:47:24.700 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.701 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:24.702 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.703 DEBUG:boto:Params: {} 2022-09-03T06:47:24.703 DEBUG:boto:Token: None 2022-09-03T06:47:24.704 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.704 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gAQ3Z91KYPbsdhyjm0/ygLlj3Ek= 2022-09-03T06:47:24.705 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gAQ3Z91KYPbsdhyjm0/ygLlj3Ek=', 'Content-Length': '0'} 2022-09-03T06:47:24.707 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008f1cf2922e81328a-006312f87c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.708 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2022-09-03T06:47:24.708 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:24.709 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:24.710 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.710 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.711 DEBUG:boto:Method: GET 2022-09-03T06:47:24.712 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.713 DEBUG:boto:Data: 2022-09-03T06:47:24.713 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.714 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:24.715 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.716 DEBUG:boto:Params: {} 2022-09-03T06:47:24.716 DEBUG:boto:Token: None 2022-09-03T06:47:24.717 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.718 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V6k1gqVTKYa5N7BcQ3T2cnpHrBA= 2022-09-03T06:47:24.719 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V6k1gqVTKYa5N7BcQ3T2cnpHrBA=', 'Content-Length': '0'} 2022-09-03T06:47:24.722 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005ee1cdccf996cc2d-006312f87c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.768 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:24.770 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:24.771 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.772 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.773 DEBUG:boto:Method: GET 2022-09-03T06:47:24.774 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.775 DEBUG:boto:Data: 2022-09-03T06:47:24.776 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.777 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:24.778 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.779 DEBUG:boto:Params: {} 2022-09-03T06:47:24.781 DEBUG:boto:Token: None 2022-09-03T06:47:24.782 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:24.783 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V6k1gqVTKYa5N7BcQ3T2cnpHrBA= 2022-09-03T06:47:24.784 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V6k1gqVTKYa5N7BcQ3T2cnpHrBA=', 'Content-Length': '0'} 2022-09-03T06:47:24.787 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2e410126a28e188-006312f87c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.788 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:24.789 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:24.789 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:24.790 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:24.791 DEBUG:boto:Method: GET 2022-09-03T06:47:24.792 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:47:24.793 DEBUG:boto:Data: 2022-09-03T06:47:24.793 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.794 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:24.795 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.796 DEBUG:boto:Params: {} 2022-09-03T06:47:24.796 DEBUG:boto:Token: None 2022-09-03T06:47:24.797 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/ 2022-09-03T06:47:24.798 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk= 2022-09-03T06:47:24.798 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk=', 'Content-Length': '0'} 2022-09-03T06:47:24.801 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c047b32a456b9ddb-006312f87c-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.802 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:24.803 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:24.803 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:24.804 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:24.804 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:24.805 DEBUG:boto:Method: GET 2022-09-03T06:47:24.805 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:47:24.806 DEBUG:boto:Data: 2022-09-03T06:47:24.807 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.807 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:24.808 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.809 DEBUG:boto:Params: {} 2022-09-03T06:47:24.810 DEBUG:boto:Token: None 2022-09-03T06:47:24.811 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/ 2022-09-03T06:47:24.812 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk= 2022-09-03T06:47:24.813 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk=', 'Content-Length': '0'} 2022-09-03T06:47:24.818 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000398c4b6e6d2817c4-006312f87c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.859 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:24.860 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:24.861 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:24.861 DEBUG:boto:Method: HEAD 2022-09-03T06:47:24.862 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:47:24.862 DEBUG:boto:Data: 2022-09-03T06:47:24.863 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.863 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:24.864 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.864 DEBUG:boto:Params: {} 2022-09-03T06:47:24.865 DEBUG:boto:Token: None 2022-09-03T06:47:24.866 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj155 2022-09-03T06:47:24.866 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eGMcPuq0DCj3Z2F1IrDZfgvLNd4= 2022-09-03T06:47:24.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eGMcPuq0DCj3Z2F1IrDZfgvLNd4=', 'Content-Length': '0'} 2022-09-03T06:47:24.869 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000766893c9bddcd817-006312f87c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.871 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:24.872 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:24.872 DEBUG:boto:Method: HEAD 2022-09-03T06:47:24.873 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:47:24.875 DEBUG:boto:Data: 2022-09-03T06:47:24.876 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.877 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:24.878 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.879 DEBUG:boto:Params: {} 2022-09-03T06:47:24.880 DEBUG:boto:Token: None 2022-09-03T06:47:24.881 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj155 2022-09-03T06:47:24.882 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eGMcPuq0DCj3Z2F1IrDZfgvLNd4= 2022-09-03T06:47:24.883 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eGMcPuq0DCj3Z2F1IrDZfgvLNd4=', 'Content-Length': '0'} 2022-09-03T06:47:24.886 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000024d5643357f5b8d1-006312f87c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.886 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2022-09-03T06:47:24.887 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:24.888 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:24.888 DEBUG:boto:Method: GET 2022-09-03T06:47:24.889 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:47:24.889 DEBUG:boto:Data: 2022-09-03T06:47:24.890 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.890 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:24.891 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.891 DEBUG:boto:Params: {} 2022-09-03T06:47:24.892 DEBUG:boto:Token: None 2022-09-03T06:47:24.893 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj155 2022-09-03T06:47:24.893 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xeRvHXuC4F+MzxGgjh0WUPteAsI= 2022-09-03T06:47:24.894 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xeRvHXuC4F+MzxGgjh0WUPteAsI=', 'Content-Length': '0'} 2022-09-03T06:47:24.896 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bcf76ec6bd38a98b-006312f87c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.897 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:24.898 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:24.899 DEBUG:boto:Method: GET 2022-09-03T06:47:24.900 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:47:24.901 DEBUG:boto:Data: 2022-09-03T06:47:24.902 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.903 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:24.904 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.905 DEBUG:boto:Params: {} 2022-09-03T06:47:24.906 DEBUG:boto:Token: None 2022-09-03T06:47:24.907 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj155 2022-09-03T06:47:24.908 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xeRvHXuC4F+MzxGgjh0WUPteAsI= 2022-09-03T06:47:24.909 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xeRvHXuC4F+MzxGgjh0WUPteAsI=', 'Content-Length': '0'} 2022-09-03T06:47:24.912 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b7ca017c7f29ed32-006312f87c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.913 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:24.915 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:24.916 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:24.917 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:24.918 DEBUG:boto:Method: GET 2022-09-03T06:47:24.919 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:47:24.920 DEBUG:boto:Data: 2022-09-03T06:47:24.921 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.922 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:24.923 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.924 DEBUG:boto:Params: {} 2022-09-03T06:47:24.925 DEBUG:boto:Token: None 2022-09-03T06:47:24.926 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/ 2022-09-03T06:47:24.927 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk= 2022-09-03T06:47:24.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk=', 'Content-Length': '0'} 2022-09-03T06:47:24.932 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000ecf25c7c83694a7-006312f87c-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.933 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:24.935 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:24.936 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:24.937 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:24.938 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:24.939 DEBUG:boto:Method: GET 2022-09-03T06:47:24.940 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:47:24.941 DEBUG:boto:Data: 2022-09-03T06:47:24.941 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.942 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:24.943 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.944 DEBUG:boto:Params: {} 2022-09-03T06:47:24.945 DEBUG:boto:Token: None 2022-09-03T06:47:24.946 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/ 2022-09-03T06:47:24.948 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk= 2022-09-03T06:47:24.949 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J4B6YhGrykJwA2n+nUgrFEcJSJk=', 'Content-Length': '0'} 2022-09-03T06:47:24.953 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000001d05e13265f2e00-006312f87c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.954 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:24.956 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:47:24.957 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:24.958 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:24.959 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:24.961 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:24.962 DEBUG:boto:Method: GET 2022-09-03T06:47:24.963 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:24.964 DEBUG:boto:Data: 2022-09-03T06:47:24.964 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.965 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:24.966 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.967 DEBUG:boto:Params: {} 2022-09-03T06:47:24.968 DEBUG:boto:Token: None 2022-09-03T06:47:24.968 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:24.969 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:922k+1fVNrkRCFMpAbB7Pf+bta0= 2022-09-03T06:47:24.970 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:922k+1fVNrkRCFMpAbB7Pf+bta0=', 'Content-Length': '0'} 2022-09-03T06:47:24.972 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000036f919ad5c25f056-006312f87c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.973 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:24.974 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:24.974 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:24.975 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:24.976 DEBUG:boto:Method: GET 2022-09-03T06:47:24.976 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:24.977 DEBUG:boto:Data: 2022-09-03T06:47:24.977 DEBUG:boto:Headers: {} 2022-09-03T06:47:24.978 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:24.979 DEBUG:boto:Port: 8000 2022-09-03T06:47:24.980 DEBUG:boto:Params: {} 2022-09-03T06:47:24.981 DEBUG:boto:Token: None 2022-09-03T06:47:24.982 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:24 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:24.983 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:922k+1fVNrkRCFMpAbB7Pf+bta0= 2022-09-03T06:47:24.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:922k+1fVNrkRCFMpAbB7Pf+bta0=', 'Content-Length': '0'} 2022-09-03T06:47:24.987 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d34671b59e90282a-006312f87c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:24.997 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:24.999 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:24.999 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:25.000 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:25.001 DEBUG:boto:Method: HEAD 2022-09-03T06:47:25.002 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:25.003 DEBUG:boto:Data: 2022-09-03T06:47:25.004 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.005 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:25.006 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.007 DEBUG:boto:Params: {} 2022-09-03T06:47:25.008 DEBUG:boto:Token: None 2022-09-03T06:47:25.009 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:25.010 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cRS+o81JBt/hFBBgTGMa/wsg1ts= 2022-09-03T06:47:25.011 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cRS+o81JBt/hFBBgTGMa/wsg1ts=', 'Content-Length': '0'} 2022-09-03T06:47:25.014 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008fc6437303d5b742-006312f87d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.015 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:25.016 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:25.018 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:25.019 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:25.020 DEBUG:boto:Method: HEAD 2022-09-03T06:47:25.021 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:25.021 DEBUG:boto:Data: 2022-09-03T06:47:25.022 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.023 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:25.024 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.027 DEBUG:boto:Params: {} 2022-09-03T06:47:25.028 DEBUG:boto:Token: None 2022-09-03T06:47:25.029 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:25.030 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cRS+o81JBt/hFBBgTGMa/wsg1ts= 2022-09-03T06:47:25.031 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cRS+o81JBt/hFBBgTGMa/wsg1ts=', 'Content-Length': '0'} 2022-09-03T06:47:25.040 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000006b9b9d67bc8c40d-006312f87d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.042 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:47:25.043 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:25.044 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:25.045 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:25.046 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:25.047 DEBUG:boto:Method: GET 2022-09-03T06:47:25.048 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:25.049 DEBUG:boto:Data: 2022-09-03T06:47:25.050 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.051 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:25.052 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.053 DEBUG:boto:Params: {} 2022-09-03T06:47:25.054 DEBUG:boto:Token: None 2022-09-03T06:47:25.055 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:25.056 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:niLEYqv69/trZFUd8zGHI/fo/Oc= 2022-09-03T06:47:25.057 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:niLEYqv69/trZFUd8zGHI/fo/Oc=', 'Content-Length': '0'} 2022-09-03T06:47:25.060 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ebb17039edc1454-006312f87d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.061 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:25.063 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:25.064 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:25.065 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:25.065 DEBUG:boto:Method: GET 2022-09-03T06:47:25.066 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:25.067 DEBUG:boto:Data: 2022-09-03T06:47:25.067 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.068 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:25.068 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.069 DEBUG:boto:Params: {} 2022-09-03T06:47:25.069 DEBUG:boto:Token: None 2022-09-03T06:47:25.070 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:25.071 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:niLEYqv69/trZFUd8zGHI/fo/Oc= 2022-09-03T06:47:25.071 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:niLEYqv69/trZFUd8zGHI/fo/Oc=', 'Content-Length': '0'} 2022-09-03T06:47:25.075 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000085143a07d18f813-006312f87d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.076 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:25.077 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:25.078 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:25.079 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:25.079 DEBUG:boto:Method: GET 2022-09-03T06:47:25.080 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:47:25.081 DEBUG:boto:Data: 2022-09-03T06:47:25.081 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.082 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:25.082 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.083 DEBUG:boto:Params: {} 2022-09-03T06:47:25.092 DEBUG:boto:Token: None 2022-09-03T06:47:25.092 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/ 2022-09-03T06:47:25.093 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc= 2022-09-03T06:47:25.094 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc=', 'Content-Length': '0'} 2022-09-03T06:47:25.097 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006b77a4f2c9ff7c12-006312f87d-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.097 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:25.098 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:25.099 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:25.099 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:25.100 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:25.100 DEBUG:boto:Method: GET 2022-09-03T06:47:25.101 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:47:25.101 DEBUG:boto:Data: 2022-09-03T06:47:25.102 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.103 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:25.104 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.104 DEBUG:boto:Params: {} 2022-09-03T06:47:25.105 DEBUG:boto:Token: None 2022-09-03T06:47:25.106 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/ 2022-09-03T06:47:25.107 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc= 2022-09-03T06:47:25.107 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc=', 'Content-Length': '0'} 2022-09-03T06:47:25.112 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003ac86de60e2d3613-006312f87d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.154 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:25.156 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:25.157 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:25.158 DEBUG:boto:Method: HEAD 2022-09-03T06:47:25.159 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:47:25.160 DEBUG:boto:Data: 2022-09-03T06:47:25.160 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.161 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:25.162 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.163 DEBUG:boto:Params: {} 2022-09-03T06:47:25.164 DEBUG:boto:Token: None 2022-09-03T06:47:25.165 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj17 2022-09-03T06:47:25.166 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FDz3u1/kxCNqawrwWGuRQ9pMDgA= 2022-09-03T06:47:25.168 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FDz3u1/kxCNqawrwWGuRQ9pMDgA=', 'Content-Length': '0'} 2022-09-03T06:47:25.171 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e1b4d882c93b0ad-006312f87d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.173 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:25.174 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:25.176 DEBUG:boto:Method: HEAD 2022-09-03T06:47:25.176 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:47:25.177 DEBUG:boto:Data: 2022-09-03T06:47:25.178 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.179 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:25.179 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.180 DEBUG:boto:Params: {} 2022-09-03T06:47:25.181 DEBUG:boto:Token: None 2022-09-03T06:47:25.182 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj17 2022-09-03T06:47:25.183 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FDz3u1/kxCNqawrwWGuRQ9pMDgA= 2022-09-03T06:47:25.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FDz3u1/kxCNqawrwWGuRQ9pMDgA=', 'Content-Length': '0'} 2022-09-03T06:47:25.187 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7059e7867bad4f7-006312f87d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.188 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:47:25.188 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:25.189 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:25.189 DEBUG:boto:Method: GET 2022-09-03T06:47:25.190 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:47:25.191 DEBUG:boto:Data: 2022-09-03T06:47:25.191 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.192 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:25.192 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.193 DEBUG:boto:Params: {} 2022-09-03T06:47:25.194 DEBUG:boto:Token: None 2022-09-03T06:47:25.195 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj17 2022-09-03T06:47:25.196 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wIXsqOxcHPrlCa1ZztFtNaCYuqc= 2022-09-03T06:47:25.196 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wIXsqOxcHPrlCa1ZztFtNaCYuqc=', 'Content-Length': '0'} 2022-09-03T06:47:25.199 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000539605c8ee54e36f-006312f87d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.200 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:25.201 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:25.202 DEBUG:boto:Method: GET 2022-09-03T06:47:25.203 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:47:25.205 DEBUG:boto:Data: 2022-09-03T06:47:25.206 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.207 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:25.208 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.209 DEBUG:boto:Params: {} 2022-09-03T06:47:25.211 DEBUG:boto:Token: None 2022-09-03T06:47:25.212 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj17 2022-09-03T06:47:25.214 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wIXsqOxcHPrlCa1ZztFtNaCYuqc= 2022-09-03T06:47:25.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wIXsqOxcHPrlCa1ZztFtNaCYuqc=', 'Content-Length': '0'} 2022-09-03T06:47:25.220 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d5c91bf6b574e62d-006312f87d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.221 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:25.221 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:25.222 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:25.223 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:25.223 DEBUG:boto:Method: GET 2022-09-03T06:47:25.224 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:47:25.225 DEBUG:boto:Data: 2022-09-03T06:47:25.225 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.226 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:25.226 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.227 DEBUG:boto:Params: {} 2022-09-03T06:47:25.227 DEBUG:boto:Token: None 2022-09-03T06:47:25.228 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/ 2022-09-03T06:47:25.229 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc= 2022-09-03T06:47:25.230 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc=', 'Content-Length': '0'} 2022-09-03T06:47:25.233 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000acd6184584515310-006312f87d-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.278 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:25.280 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:25.281 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:25.281 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:25.282 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:25.283 DEBUG:boto:Method: GET 2022-09-03T06:47:25.283 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:47:25.284 DEBUG:boto:Data: 2022-09-03T06:47:25.285 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.287 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:25.288 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.289 DEBUG:boto:Params: {} 2022-09-03T06:47:25.290 DEBUG:boto:Token: None 2022-09-03T06:47:25.291 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/ 2022-09-03T06:47:25.292 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc= 2022-09-03T06:47:25.293 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc=', 'Content-Length': '0'} 2022-09-03T06:47:25.298 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a12bd8f486b486af-006312f87d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.300 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:25.301 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:47:25.302 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:25.303 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:25.304 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.305 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.306 DEBUG:boto:Method: GET 2022-09-03T06:47:25.307 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.307 DEBUG:boto:Data: 2022-09-03T06:47:25.308 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.309 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:25.309 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.310 DEBUG:boto:Params: {} 2022-09-03T06:47:25.311 DEBUG:boto:Token: None 2022-09-03T06:47:25.312 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.313 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0dHrBV1OHoW5f7RisshnS4GRKhQ= 2022-09-03T06:47:25.314 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0dHrBV1OHoW5f7RisshnS4GRKhQ=', 'Content-Length': '0'} 2022-09-03T06:47:25.317 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000e1a7da121f203a7-006312f87d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.319 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:25.320 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:25.321 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.322 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.323 DEBUG:boto:Method: GET 2022-09-03T06:47:25.324 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.325 DEBUG:boto:Data: 2022-09-03T06:47:25.326 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.327 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:25.328 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.329 DEBUG:boto:Params: {} 2022-09-03T06:47:25.330 DEBUG:boto:Token: None 2022-09-03T06:47:25.330 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.331 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0dHrBV1OHoW5f7RisshnS4GRKhQ= 2022-09-03T06:47:25.331 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0dHrBV1OHoW5f7RisshnS4GRKhQ=', 'Content-Length': '0'} 2022-09-03T06:47:25.334 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee56757c23203073-006312f87d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.382 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:25.383 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:25.384 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.385 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.386 DEBUG:boto:Method: HEAD 2022-09-03T06:47:25.387 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.388 DEBUG:boto:Data: 2022-09-03T06:47:25.389 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.391 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:25.392 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.393 DEBUG:boto:Params: {} 2022-09-03T06:47:25.394 DEBUG:boto:Token: None 2022-09-03T06:47:25.395 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.396 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/g+oo888k2kQt5jRiQybvA8rGng= 2022-09-03T06:47:25.397 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/g+oo888k2kQt5jRiQybvA8rGng=', 'Content-Length': '0'} 2022-09-03T06:47:25.400 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096b7809577da11da-006312f87d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.402 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:25.403 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:25.404 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.405 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.409 DEBUG:boto:Method: HEAD 2022-09-03T06:47:25.410 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.410 DEBUG:boto:Data: 2022-09-03T06:47:25.411 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.411 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:25.412 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.413 DEBUG:boto:Params: {} 2022-09-03T06:47:25.413 DEBUG:boto:Token: None 2022-09-03T06:47:25.414 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.415 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/g+oo888k2kQt5jRiQybvA8rGng= 2022-09-03T06:47:25.415 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/g+oo888k2kQt5jRiQybvA8rGng=', 'Content-Length': '0'} 2022-09-03T06:47:25.418 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ae56a36abaad7c4c-006312f87d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.419 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:47:25.419 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:25.420 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:25.421 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.422 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.422 DEBUG:boto:Method: GET 2022-09-03T06:47:25.423 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.423 DEBUG:boto:Data: 2022-09-03T06:47:25.424 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.425 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:25.425 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.426 DEBUG:boto:Params: {} 2022-09-03T06:47:25.427 DEBUG:boto:Token: None 2022-09-03T06:47:25.427 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.428 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0dHrBV1OHoW5f7RisshnS4GRKhQ= 2022-09-03T06:47:25.429 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0dHrBV1OHoW5f7RisshnS4GRKhQ=', 'Content-Length': '0'} 2022-09-03T06:47:25.431 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fd53d1c19e94be10-006312f87d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.433 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:25.434 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:25.435 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.436 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.437 DEBUG:boto:Method: GET 2022-09-03T06:47:25.438 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.439 DEBUG:boto:Data: 2022-09-03T06:47:25.440 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.441 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:25.442 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.443 DEBUG:boto:Params: {} 2022-09-03T06:47:25.444 DEBUG:boto:Token: None 2022-09-03T06:47:25.445 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:25.446 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0dHrBV1OHoW5f7RisshnS4GRKhQ= 2022-09-03T06:47:25.447 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0dHrBV1OHoW5f7RisshnS4GRKhQ=', 'Content-Length': '0'} 2022-09-03T06:47:25.450 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ddca03cb7c59af4f-006312f87d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.452 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:25.452 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:25.453 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:25.454 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:25.455 DEBUG:boto:Method: GET 2022-09-03T06:47:25.456 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:47:25.457 DEBUG:boto:Data: 2022-09-03T06:47:25.458 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.459 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:25.460 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.461 DEBUG:boto:Params: {} 2022-09-03T06:47:25.462 DEBUG:boto:Token: None 2022-09-03T06:47:25.463 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/ 2022-09-03T06:47:25.464 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc= 2022-09-03T06:47:25.465 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc=', 'Content-Length': '0'} 2022-09-03T06:47:25.469 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cbf2f84dc94ed096-006312f87d-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.470 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:25.472 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:25.473 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:25.474 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:25.475 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:25.476 DEBUG:boto:Method: GET 2022-09-03T06:47:25.476 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:47:25.477 DEBUG:boto:Data: 2022-09-03T06:47:25.478 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.479 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:25.480 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.480 DEBUG:boto:Params: {} 2022-09-03T06:47:25.482 DEBUG:boto:Token: None 2022-09-03T06:47:25.482 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/ 2022-09-03T06:47:25.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc= 2022-09-03T06:47:25.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc=', 'Content-Length': '0'} 2022-09-03T06:47:25.488 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000015b633d80332dc5b-006312f87d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.489 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:25.490 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:25.490 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:25.491 DEBUG:boto:Method: HEAD 2022-09-03T06:47:25.491 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:47:25.492 DEBUG:boto:Data: 2022-09-03T06:47:25.492 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.493 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:25.493 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.494 DEBUG:boto:Params: {} 2022-09-03T06:47:25.494 DEBUG:boto:Token: None 2022-09-03T06:47:25.495 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj19 2022-09-03T06:47:25.496 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PO67te/SfwcMRXf082G0eQUAtbc= 2022-09-03T06:47:25.496 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PO67te/SfwcMRXf082G0eQUAtbc=', 'Content-Length': '0'} 2022-09-03T06:47:25.498 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000033e4f59529dbfb3-006312f87d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.499 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:25.500 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:25.500 DEBUG:boto:Method: HEAD 2022-09-03T06:47:25.501 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:47:25.501 DEBUG:boto:Data: 2022-09-03T06:47:25.502 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.502 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:25.503 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.503 DEBUG:boto:Params: {} 2022-09-03T06:47:25.504 DEBUG:boto:Token: None 2022-09-03T06:47:25.504 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj19 2022-09-03T06:47:25.505 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PO67te/SfwcMRXf082G0eQUAtbc= 2022-09-03T06:47:25.506 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PO67te/SfwcMRXf082G0eQUAtbc=', 'Content-Length': '0'} 2022-09-03T06:47:25.508 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f0412b234bed6b56-006312f87d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.508 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:47:25.509 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:25.509 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:25.510 DEBUG:boto:Method: GET 2022-09-03T06:47:25.511 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:47:25.511 DEBUG:boto:Data: 2022-09-03T06:47:25.512 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.512 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:25.513 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.513 DEBUG:boto:Params: {} 2022-09-03T06:47:25.514 DEBUG:boto:Token: None 2022-09-03T06:47:25.514 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj19 2022-09-03T06:47:25.515 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3U8FuLvvFCxRTFtM+yFhCOxh3+U= 2022-09-03T06:47:25.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3U8FuLvvFCxRTFtM+yFhCOxh3+U=', 'Content-Length': '0'} 2022-09-03T06:47:25.518 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d4b3fa84953ba277-006312f87d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.519 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:25.519 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:25.520 DEBUG:boto:Method: GET 2022-09-03T06:47:25.521 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:47:25.521 DEBUG:boto:Data: 2022-09-03T06:47:25.522 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.522 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:25.523 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.523 DEBUG:boto:Params: {} 2022-09-03T06:47:25.524 DEBUG:boto:Token: None 2022-09-03T06:47:25.524 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj19 2022-09-03T06:47:25.525 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3U8FuLvvFCxRTFtM+yFhCOxh3+U= 2022-09-03T06:47:25.526 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3U8FuLvvFCxRTFtM+yFhCOxh3+U=', 'Content-Length': '0'} 2022-09-03T06:47:25.528 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d1fc8b60ee5654eb-006312f87d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.575 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:25.576 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:25.577 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:25.577 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:25.578 DEBUG:boto:Method: GET 2022-09-03T06:47:25.578 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:47:25.579 DEBUG:boto:Data: 2022-09-03T06:47:25.579 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.580 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:25.581 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.582 DEBUG:boto:Params: {} 2022-09-03T06:47:25.582 DEBUG:boto:Token: None 2022-09-03T06:47:25.583 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/ 2022-09-03T06:47:25.583 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc= 2022-09-03T06:47:25.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc=', 'Content-Length': '0'} 2022-09-03T06:47:25.587 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000419191ac71a06991-006312f87d-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.587 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:25.589 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:25.589 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:25.590 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:25.591 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:25.591 DEBUG:boto:Method: GET 2022-09-03T06:47:25.592 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:47:25.592 DEBUG:boto:Data: 2022-09-03T06:47:25.593 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.593 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:25.594 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.595 DEBUG:boto:Params: {} 2022-09-03T06:47:25.595 DEBUG:boto:Token: None 2022-09-03T06:47:25.596 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/ 2022-09-03T06:47:25.597 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc= 2022-09-03T06:47:25.598 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc=', 'Content-Length': '0'} 2022-09-03T06:47:25.602 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006af1c54321d6044e-006312f87d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.603 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:25.605 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:47:25.606 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:25.607 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:25.607 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.608 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.608 DEBUG:boto:Method: GET 2022-09-03T06:47:25.609 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.609 DEBUG:boto:Data: 2022-09-03T06:47:25.610 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.610 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:25.611 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.611 DEBUG:boto:Params: {} 2022-09-03T06:47:25.612 DEBUG:boto:Token: None 2022-09-03T06:47:25.613 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.613 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:81h2ejjevPizMpxNTPYS8BimJrs= 2022-09-03T06:47:25.614 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:81h2ejjevPizMpxNTPYS8BimJrs=', 'Content-Length': '0'} 2022-09-03T06:47:25.616 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d14db007418c1fd9-006312f87d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.617 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:25.617 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:25.618 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.618 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.619 DEBUG:boto:Method: GET 2022-09-03T06:47:25.619 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.620 DEBUG:boto:Data: 2022-09-03T06:47:25.620 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.621 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:25.621 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.622 DEBUG:boto:Params: {} 2022-09-03T06:47:25.623 DEBUG:boto:Token: None 2022-09-03T06:47:25.623 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.624 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:81h2ejjevPizMpxNTPYS8BimJrs= 2022-09-03T06:47:25.624 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:81h2ejjevPizMpxNTPYS8BimJrs=', 'Content-Length': '0'} 2022-09-03T06:47:25.627 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005fc33c5b9653ef6b-006312f87d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.628 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:25.628 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:25.629 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.629 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.630 DEBUG:boto:Method: HEAD 2022-09-03T06:47:25.630 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.631 DEBUG:boto:Data: 2022-09-03T06:47:25.631 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.632 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:25.632 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.633 DEBUG:boto:Params: {} 2022-09-03T06:47:25.633 DEBUG:boto:Token: None 2022-09-03T06:47:25.634 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.635 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:954UAQXDvDtc8n/lxOE5Yk8gJ8Q= 2022-09-03T06:47:25.635 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:954UAQXDvDtc8n/lxOE5Yk8gJ8Q=', 'Content-Length': '0'} 2022-09-03T06:47:25.637 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000095bbd62166e16f50-006312f87d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.638 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:25.639 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:25.639 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.640 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.640 DEBUG:boto:Method: HEAD 2022-09-03T06:47:25.641 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.641 DEBUG:boto:Data: 2022-09-03T06:47:25.642 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.642 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:25.643 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.644 DEBUG:boto:Params: {} 2022-09-03T06:47:25.644 DEBUG:boto:Token: None 2022-09-03T06:47:25.645 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.645 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:954UAQXDvDtc8n/lxOE5Yk8gJ8Q= 2022-09-03T06:47:25.646 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:954UAQXDvDtc8n/lxOE5Yk8gJ8Q=', 'Content-Length': '0'} 2022-09-03T06:47:25.648 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a9b219d130b6b52-006312f87d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.649 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:47:25.650 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:25.650 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:25.651 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.652 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.652 DEBUG:boto:Method: GET 2022-09-03T06:47:25.653 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.653 DEBUG:boto:Data: 2022-09-03T06:47:25.654 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.654 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:25.655 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.656 DEBUG:boto:Params: {} 2022-09-03T06:47:25.656 DEBUG:boto:Token: None 2022-09-03T06:47:25.657 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.658 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:81h2ejjevPizMpxNTPYS8BimJrs= 2022-09-03T06:47:25.658 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:81h2ejjevPizMpxNTPYS8BimJrs=', 'Content-Length': '0'} 2022-09-03T06:47:25.660 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000091435d0af617e6b9-006312f87d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.661 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:25.662 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:25.663 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.663 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.664 DEBUG:boto:Method: GET 2022-09-03T06:47:25.665 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.666 DEBUG:boto:Data: 2022-09-03T06:47:25.666 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.667 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:25.667 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.668 DEBUG:boto:Params: {} 2022-09-03T06:47:25.668 DEBUG:boto:Token: None 2022-09-03T06:47:25.669 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:25.670 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:81h2ejjevPizMpxNTPYS8BimJrs= 2022-09-03T06:47:25.670 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:81h2ejjevPizMpxNTPYS8BimJrs=', 'Content-Length': '0'} 2022-09-03T06:47:25.673 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000052004a79da334ff9-006312f87d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.714 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:25.715 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:25.716 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:25.716 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:25.717 DEBUG:boto:Method: GET 2022-09-03T06:47:25.718 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:47:25.718 DEBUG:boto:Data: 2022-09-03T06:47:25.719 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.720 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:25.720 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.721 DEBUG:boto:Params: {} 2022-09-03T06:47:25.723 DEBUG:boto:Token: None 2022-09-03T06:47:25.723 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/ 2022-09-03T06:47:25.724 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc= 2022-09-03T06:47:25.725 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc=', 'Content-Length': '0'} 2022-09-03T06:47:25.728 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000047d32c4452b2e7ec-006312f87d-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.729 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:25.730 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:25.731 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:25.731 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:25.733 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:25.733 DEBUG:boto:Method: GET 2022-09-03T06:47:25.734 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:47:25.735 DEBUG:boto:Data: 2022-09-03T06:47:25.735 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.736 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:25.736 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.737 DEBUG:boto:Params: {} 2022-09-03T06:47:25.737 DEBUG:boto:Token: None 2022-09-03T06:47:25.738 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/ 2022-09-03T06:47:25.739 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc= 2022-09-03T06:47:25.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc=', 'Content-Length': '0'} 2022-09-03T06:47:25.744 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d80ef3311ac7cd45-006312f87d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.745 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:25.746 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:25.746 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:25.747 DEBUG:boto:Method: HEAD 2022-09-03T06:47:25.748 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:47:25.749 DEBUG:boto:Data: 2022-09-03T06:47:25.750 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.752 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:25.753 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.754 DEBUG:boto:Params: {} 2022-09-03T06:47:25.755 DEBUG:boto:Token: None 2022-09-03T06:47:25.755 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj22 2022-09-03T06:47:25.756 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tBWqaeGg7p0/D6tkCb1uhVH9hTg= 2022-09-03T06:47:25.757 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tBWqaeGg7p0/D6tkCb1uhVH9hTg=', 'Content-Length': '0'} 2022-09-03T06:47:25.759 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000331a882d595c225-006312f87d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.760 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:25.761 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:25.761 DEBUG:boto:Method: HEAD 2022-09-03T06:47:25.762 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:47:25.763 DEBUG:boto:Data: 2022-09-03T06:47:25.764 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.765 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:25.766 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.766 DEBUG:boto:Params: {} 2022-09-03T06:47:25.767 DEBUG:boto:Token: None 2022-09-03T06:47:25.769 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj22 2022-09-03T06:47:25.769 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tBWqaeGg7p0/D6tkCb1uhVH9hTg= 2022-09-03T06:47:25.771 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tBWqaeGg7p0/D6tkCb1uhVH9hTg=', 'Content-Length': '0'} 2022-09-03T06:47:25.774 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e76a9a6871d7db37-006312f87d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.775 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:47:25.776 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:25.776 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:25.777 DEBUG:boto:Method: GET 2022-09-03T06:47:25.777 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:47:25.778 DEBUG:boto:Data: 2022-09-03T06:47:25.778 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.779 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:25.779 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.780 DEBUG:boto:Params: {} 2022-09-03T06:47:25.780 DEBUG:boto:Token: None 2022-09-03T06:47:25.781 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj22 2022-09-03T06:47:25.782 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w6ooIifcI7ri6ayAQXdV4qtZdsQ= 2022-09-03T06:47:25.782 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w6ooIifcI7ri6ayAQXdV4qtZdsQ=', 'Content-Length': '0'} 2022-09-03T06:47:25.788 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cc92d188c3accd48-006312f87d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.832 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:25.833 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:25.834 DEBUG:boto:Method: GET 2022-09-03T06:47:25.835 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:47:25.836 DEBUG:boto:Data: 2022-09-03T06:47:25.837 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.838 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:25.839 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.840 DEBUG:boto:Params: {} 2022-09-03T06:47:25.842 DEBUG:boto:Token: None 2022-09-03T06:47:25.847 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj22 2022-09-03T06:47:25.849 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w6ooIifcI7ri6ayAQXdV4qtZdsQ= 2022-09-03T06:47:25.854 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w6ooIifcI7ri6ayAQXdV4qtZdsQ=', 'Content-Length': '0'} 2022-09-03T06:47:25.857 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000907be837aba7e649-006312f87d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.859 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:25.860 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:25.862 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:25.863 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:25.864 DEBUG:boto:Method: GET 2022-09-03T06:47:25.865 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:47:25.865 DEBUG:boto:Data: 2022-09-03T06:47:25.866 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.867 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:25.868 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.869 DEBUG:boto:Params: {} 2022-09-03T06:47:25.869 DEBUG:boto:Token: None 2022-09-03T06:47:25.870 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/ 2022-09-03T06:47:25.871 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc= 2022-09-03T06:47:25.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc=', 'Content-Length': '0'} 2022-09-03T06:47:25.875 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005c0f644e7cc88f37-006312f87d-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.876 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:25.877 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:25.878 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:25.878 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:25.879 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:25.879 DEBUG:boto:Method: GET 2022-09-03T06:47:25.880 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:47:25.880 DEBUG:boto:Data: 2022-09-03T06:47:25.881 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.881 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:25.882 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.883 DEBUG:boto:Params: {} 2022-09-03T06:47:25.884 DEBUG:boto:Token: None 2022-09-03T06:47:25.884 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/ 2022-09-03T06:47:25.885 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc= 2022-09-03T06:47:25.886 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mGSv8h9nzpDD5/rx4MpTHiUz0zc=', 'Content-Length': '0'} 2022-09-03T06:47:25.892 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000cd1181ab3c6d5f4-006312f87d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.893 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:25.895 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:47:25.896 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:25.897 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:25.898 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:25.899 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:25.900 DEBUG:boto:Method: GET 2022-09-03T06:47:25.901 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:25.902 DEBUG:boto:Data: 2022-09-03T06:47:25.903 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.904 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:25.905 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.906 DEBUG:boto:Params: {} 2022-09-03T06:47:25.907 DEBUG:boto:Token: None 2022-09-03T06:47:25.908 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:25.909 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WziiLteIVpAHkVGHr7nMAnS0Y4w= 2022-09-03T06:47:25.910 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WziiLteIVpAHkVGHr7nMAnS0Y4w=', 'Content-Length': '0'} 2022-09-03T06:47:25.913 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000016428822fae59b99-006312f87d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.914 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:25.914 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:25.916 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:25.917 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:25.917 DEBUG:boto:Method: GET 2022-09-03T06:47:25.918 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:25.918 DEBUG:boto:Data: 2022-09-03T06:47:25.919 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.919 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:25.920 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.920 DEBUG:boto:Params: {} 2022-09-03T06:47:25.921 DEBUG:boto:Token: None 2022-09-03T06:47:25.922 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:25.922 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WziiLteIVpAHkVGHr7nMAnS0Y4w= 2022-09-03T06:47:25.923 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WziiLteIVpAHkVGHr7nMAnS0Y4w=', 'Content-Length': '0'} 2022-09-03T06:47:25.926 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000abb814c7d5db1724-006312f87d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.966 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:25.968 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:25.969 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:25.970 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:25.971 DEBUG:boto:Method: HEAD 2022-09-03T06:47:25.972 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:25.973 DEBUG:boto:Data: 2022-09-03T06:47:25.974 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.975 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:25.975 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.976 DEBUG:boto:Params: {} 2022-09-03T06:47:25.977 DEBUG:boto:Token: None 2022-09-03T06:47:25.977 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:25.978 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J+bsMmUL5A0uIwRs7ziX+OKcRMU= 2022-09-03T06:47:25.979 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J+bsMmUL5A0uIwRs7ziX+OKcRMU=', 'Content-Length': '0'} 2022-09-03T06:47:25.981 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c1d493f14f2a6df8-006312f87d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.982 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:25.982 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:25.983 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:25.983 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:25.984 DEBUG:boto:Method: HEAD 2022-09-03T06:47:25.985 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:25.985 DEBUG:boto:Data: 2022-09-03T06:47:25.986 DEBUG:boto:Headers: {} 2022-09-03T06:47:25.986 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:25.987 DEBUG:boto:Port: 8000 2022-09-03T06:47:25.987 DEBUG:boto:Params: {} 2022-09-03T06:47:25.988 DEBUG:boto:Token: None 2022-09-03T06:47:25.988 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:25 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:25.989 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J+bsMmUL5A0uIwRs7ziX+OKcRMU= 2022-09-03T06:47:25.990 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J+bsMmUL5A0uIwRs7ziX+OKcRMU=', 'Content-Length': '0'} 2022-09-03T06:47:25.992 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b32f94a8a4d5f731-006312f87d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:25.993 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:47:25.994 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:25.995 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:25.995 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:25.996 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:25.997 DEBUG:boto:Method: GET 2022-09-03T06:47:25.998 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:25.999 DEBUG:boto:Data: 2022-09-03T06:47:26.000 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.001 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.002 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.003 DEBUG:boto:Params: {} 2022-09-03T06:47:26.004 DEBUG:boto:Token: None 2022-09-03T06:47:26.004 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:26.005 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pwoSUTWYqfDB6Rzfwkt5wsdn2tI= 2022-09-03T06:47:26.005 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pwoSUTWYqfDB6Rzfwkt5wsdn2tI=', 'Content-Length': '0'} 2022-09-03T06:47:26.008 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000038a6804b93c875ba-006312f87e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.009 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:26.010 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:26.010 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:26.011 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:26.011 DEBUG:boto:Method: GET 2022-09-03T06:47:26.012 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:26.013 DEBUG:boto:Data: 2022-09-03T06:47:26.013 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.014 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.014 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.015 DEBUG:boto:Params: {} 2022-09-03T06:47:26.015 DEBUG:boto:Token: None 2022-09-03T06:47:26.016 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:26.017 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pwoSUTWYqfDB6Rzfwkt5wsdn2tI= 2022-09-03T06:47:26.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pwoSUTWYqfDB6Rzfwkt5wsdn2tI=', 'Content-Length': '0'} 2022-09-03T06:47:26.019 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009ccb41fd55883826-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.020 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:26.021 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:26.021 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:26.022 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:26.023 DEBUG:boto:Method: GET 2022-09-03T06:47:26.023 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:47:26.024 DEBUG:boto:Data: 2022-09-03T06:47:26.024 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.025 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.025 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.026 DEBUG:boto:Params: {} 2022-09-03T06:47:26.027 DEBUG:boto:Token: None 2022-09-03T06:47:26.027 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/ 2022-09-03T06:47:26.028 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg= 2022-09-03T06:47:26.028 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg=', 'Content-Length': '0'} 2022-09-03T06:47:26.031 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004bf9583d8e3c4ecd-006312f87e-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.032 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:26.034 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:26.035 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:26.036 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:26.037 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:26.037 DEBUG:boto:Method: GET 2022-09-03T06:47:26.038 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:47:26.039 DEBUG:boto:Data: 2022-09-03T06:47:26.040 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.040 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.041 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.041 DEBUG:boto:Params: {} 2022-09-03T06:47:26.042 DEBUG:boto:Token: None 2022-09-03T06:47:26.043 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/ 2022-09-03T06:47:26.044 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg= 2022-09-03T06:47:26.045 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg=', 'Content-Length': '0'} 2022-09-03T06:47:26.049 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ef6a32c28caffb6f-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.050 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:26.051 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:26.052 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:26.052 DEBUG:boto:Method: HEAD 2022-09-03T06:47:26.053 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:47:26.053 DEBUG:boto:Data: 2022-09-03T06:47:26.054 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.054 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.055 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.055 DEBUG:boto:Params: {} 2022-09-03T06:47:26.056 DEBUG:boto:Token: None 2022-09-03T06:47:26.056 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj23 2022-09-03T06:47:26.057 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X+b7Six+Zjj+lauQ0fle4lQFbA8= 2022-09-03T06:47:26.058 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X+b7Six+Zjj+lauQ0fle4lQFbA8=', 'Content-Length': '0'} 2022-09-03T06:47:26.061 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c6e142e4425b5e64-006312f87e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.062 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:26.063 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:26.064 DEBUG:boto:Method: HEAD 2022-09-03T06:47:26.065 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:47:26.066 DEBUG:boto:Data: 2022-09-03T06:47:26.067 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.068 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.068 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.069 DEBUG:boto:Params: {} 2022-09-03T06:47:26.070 DEBUG:boto:Token: None 2022-09-03T06:47:26.070 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj23 2022-09-03T06:47:26.071 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X+b7Six+Zjj+lauQ0fle4lQFbA8= 2022-09-03T06:47:26.071 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X+b7Six+Zjj+lauQ0fle4lQFbA8=', 'Content-Length': '0'} 2022-09-03T06:47:26.073 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d5b6a7ba3ea498c-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.074 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:47:26.075 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:26.075 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:26.076 DEBUG:boto:Method: GET 2022-09-03T06:47:26.076 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:47:26.077 DEBUG:boto:Data: 2022-09-03T06:47:26.078 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.078 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.079 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.079 DEBUG:boto:Params: {} 2022-09-03T06:47:26.080 DEBUG:boto:Token: None 2022-09-03T06:47:26.081 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj23 2022-09-03T06:47:26.081 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:miEbKhV//l11SPruNGwWXlvSdGg= 2022-09-03T06:47:26.082 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:miEbKhV//l11SPruNGwWXlvSdGg=', 'Content-Length': '0'} 2022-09-03T06:47:26.084 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a897c42865422754-006312f87e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.085 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:26.086 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:26.086 DEBUG:boto:Method: GET 2022-09-03T06:47:26.087 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:47:26.087 DEBUG:boto:Data: 2022-09-03T06:47:26.088 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.088 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.089 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.090 DEBUG:boto:Params: {} 2022-09-03T06:47:26.090 DEBUG:boto:Token: None 2022-09-03T06:47:26.091 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj23 2022-09-03T06:47:26.091 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:miEbKhV//l11SPruNGwWXlvSdGg= 2022-09-03T06:47:26.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:miEbKhV//l11SPruNGwWXlvSdGg=', 'Content-Length': '0'} 2022-09-03T06:47:26.094 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000013922faaf6a69e27-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.095 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:26.095 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:26.096 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:26.096 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:26.097 DEBUG:boto:Method: GET 2022-09-03T06:47:26.098 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:47:26.098 DEBUG:boto:Data: 2022-09-03T06:47:26.099 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.099 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.100 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.100 DEBUG:boto:Params: {} 2022-09-03T06:47:26.101 DEBUG:boto:Token: None 2022-09-03T06:47:26.101 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/ 2022-09-03T06:47:26.102 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg= 2022-09-03T06:47:26.103 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg=', 'Content-Length': '0'} 2022-09-03T06:47:26.105 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000721330a24b70f3ba-006312f87e-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.146 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:26.147 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:26.148 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:26.148 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:26.149 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:26.149 DEBUG:boto:Method: GET 2022-09-03T06:47:26.150 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:47:26.151 DEBUG:boto:Data: 2022-09-03T06:47:26.151 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.152 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.152 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.153 DEBUG:boto:Params: {} 2022-09-03T06:47:26.154 DEBUG:boto:Token: None 2022-09-03T06:47:26.154 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/ 2022-09-03T06:47:26.155 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg= 2022-09-03T06:47:26.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg=', 'Content-Length': '0'} 2022-09-03T06:47:26.160 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000220d198c0bdcacb1-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.161 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:26.162 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:47:26.162 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:26.163 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:26.164 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.164 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.165 DEBUG:boto:Method: GET 2022-09-03T06:47:26.165 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.166 DEBUG:boto:Data: 2022-09-03T06:47:26.166 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.167 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.167 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.168 DEBUG:boto:Params: {} 2022-09-03T06:47:26.169 DEBUG:boto:Token: None 2022-09-03T06:47:26.169 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.170 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2nDs3x1W2GU9YbxPWMqM7xii+PY= 2022-09-03T06:47:26.170 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2nDs3x1W2GU9YbxPWMqM7xii+PY=', 'Content-Length': '0'} 2022-09-03T06:47:26.173 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fc13597a4465619c-006312f87e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.174 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:26.174 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:26.175 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.175 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.176 DEBUG:boto:Method: GET 2022-09-03T06:47:26.176 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.177 DEBUG:boto:Data: 2022-09-03T06:47:26.177 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.178 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.178 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.179 DEBUG:boto:Params: {} 2022-09-03T06:47:26.179 DEBUG:boto:Token: None 2022-09-03T06:47:26.180 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.181 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2nDs3x1W2GU9YbxPWMqM7xii+PY= 2022-09-03T06:47:26.181 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2nDs3x1W2GU9YbxPWMqM7xii+PY=', 'Content-Length': '0'} 2022-09-03T06:47:26.184 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006744c8ac0c93b7e3-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.226 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:26.227 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:26.228 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.229 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.230 DEBUG:boto:Method: HEAD 2022-09-03T06:47:26.230 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.231 DEBUG:boto:Data: 2022-09-03T06:47:26.231 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.232 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.233 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.233 DEBUG:boto:Params: {} 2022-09-03T06:47:26.234 DEBUG:boto:Token: None 2022-09-03T06:47:26.235 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.236 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gx/7J20WNQ49PhK/Uzz2o8upiak= 2022-09-03T06:47:26.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gx/7J20WNQ49PhK/Uzz2o8upiak=', 'Content-Length': '0'} 2022-09-03T06:47:26.239 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b02c74f7acd11e6-006312f87e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.240 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:26.240 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:26.241 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.241 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.242 DEBUG:boto:Method: HEAD 2022-09-03T06:47:26.242 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.244 DEBUG:boto:Data: 2022-09-03T06:47:26.244 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.245 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.246 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.246 DEBUG:boto:Params: {} 2022-09-03T06:47:26.247 DEBUG:boto:Token: None 2022-09-03T06:47:26.247 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.248 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gx/7J20WNQ49PhK/Uzz2o8upiak= 2022-09-03T06:47:26.249 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gx/7J20WNQ49PhK/Uzz2o8upiak=', 'Content-Length': '0'} 2022-09-03T06:47:26.251 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000925461567a35c824-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.252 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:47:26.252 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:26.253 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:26.253 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.254 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.254 DEBUG:boto:Method: GET 2022-09-03T06:47:26.255 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.256 DEBUG:boto:Data: 2022-09-03T06:47:26.256 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.257 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.257 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.258 DEBUG:boto:Params: {} 2022-09-03T06:47:26.259 DEBUG:boto:Token: None 2022-09-03T06:47:26.259 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.260 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2nDs3x1W2GU9YbxPWMqM7xii+PY= 2022-09-03T06:47:26.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2nDs3x1W2GU9YbxPWMqM7xii+PY=', 'Content-Length': '0'} 2022-09-03T06:47:26.263 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2d2ef69417f47ad-006312f87e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.263 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:26.264 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:26.265 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.265 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.266 DEBUG:boto:Method: GET 2022-09-03T06:47:26.267 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.267 DEBUG:boto:Data: 2022-09-03T06:47:26.268 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.268 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.269 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.269 DEBUG:boto:Params: {} 2022-09-03T06:47:26.270 DEBUG:boto:Token: None 2022-09-03T06:47:26.271 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:26.271 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2nDs3x1W2GU9YbxPWMqM7xii+PY= 2022-09-03T06:47:26.272 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2nDs3x1W2GU9YbxPWMqM7xii+PY=', 'Content-Length': '0'} 2022-09-03T06:47:26.274 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000023d049f3700d6145-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.275 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:26.276 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:26.276 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:26.277 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:26.277 DEBUG:boto:Method: GET 2022-09-03T06:47:26.278 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:47:26.278 DEBUG:boto:Data: 2022-09-03T06:47:26.279 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.279 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.280 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.281 DEBUG:boto:Params: {} 2022-09-03T06:47:26.281 DEBUG:boto:Token: None 2022-09-03T06:47:26.282 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/ 2022-09-03T06:47:26.282 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg= 2022-09-03T06:47:26.283 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg=', 'Content-Length': '0'} 2022-09-03T06:47:26.286 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000648173d54bb329e7-006312f87e-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.330 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:26.331 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:26.332 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:26.332 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:26.333 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:26.334 DEBUG:boto:Method: GET 2022-09-03T06:47:26.335 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:47:26.336 DEBUG:boto:Data: 2022-09-03T06:47:26.337 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.340 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.341 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.342 DEBUG:boto:Params: {} 2022-09-03T06:47:26.344 DEBUG:boto:Token: None 2022-09-03T06:47:26.345 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/ 2022-09-03T06:47:26.346 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg= 2022-09-03T06:47:26.347 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg=', 'Content-Length': '0'} 2022-09-03T06:47:26.353 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d0ee48dfd42f178c-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.354 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:26.356 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:26.357 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:26.358 DEBUG:boto:Method: HEAD 2022-09-03T06:47:26.359 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:47:26.360 DEBUG:boto:Data: 2022-09-03T06:47:26.361 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.362 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.363 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.364 DEBUG:boto:Params: {} 2022-09-03T06:47:26.365 DEBUG:boto:Token: None 2022-09-03T06:47:26.366 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj29 2022-09-03T06:47:26.367 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DmxuxUlPHsuTeqzqXlGfmDryHf4= 2022-09-03T06:47:26.368 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DmxuxUlPHsuTeqzqXlGfmDryHf4=', 'Content-Length': '0'} 2022-09-03T06:47:26.371 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000791a554ffb465d96-006312f87e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.372 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:26.373 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:26.373 DEBUG:boto:Method: HEAD 2022-09-03T06:47:26.374 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:47:26.374 DEBUG:boto:Data: 2022-09-03T06:47:26.375 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.375 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.376 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.376 DEBUG:boto:Params: {} 2022-09-03T06:47:26.377 DEBUG:boto:Token: None 2022-09-03T06:47:26.377 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj29 2022-09-03T06:47:26.378 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DmxuxUlPHsuTeqzqXlGfmDryHf4= 2022-09-03T06:47:26.379 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DmxuxUlPHsuTeqzqXlGfmDryHf4=', 'Content-Length': '0'} 2022-09-03T06:47:26.381 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000069e56cfd8318eea4-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.384 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:47:26.386 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:26.387 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:26.388 DEBUG:boto:Method: GET 2022-09-03T06:47:26.389 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:47:26.390 DEBUG:boto:Data: 2022-09-03T06:47:26.391 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.392 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.393 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.394 DEBUG:boto:Params: {} 2022-09-03T06:47:26.395 DEBUG:boto:Token: None 2022-09-03T06:47:26.396 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj29 2022-09-03T06:47:26.397 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7eIUDvN6vVpc+IytZcTMCqUblj4= 2022-09-03T06:47:26.398 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7eIUDvN6vVpc+IytZcTMCqUblj4=', 'Content-Length': '0'} 2022-09-03T06:47:26.402 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dcee62bf5973fcf6-006312f87e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.404 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:26.405 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:26.405 DEBUG:boto:Method: GET 2022-09-03T06:47:26.406 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:47:26.406 DEBUG:boto:Data: 2022-09-03T06:47:26.407 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.407 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.408 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.409 DEBUG:boto:Params: {} 2022-09-03T06:47:26.409 DEBUG:boto:Token: None 2022-09-03T06:47:26.410 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj29 2022-09-03T06:47:26.411 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7eIUDvN6vVpc+IytZcTMCqUblj4= 2022-09-03T06:47:26.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7eIUDvN6vVpc+IytZcTMCqUblj4=', 'Content-Length': '0'} 2022-09-03T06:47:26.420 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000002b89d1341be64ee-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.462 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:26.464 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:26.465 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:26.465 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:26.466 DEBUG:boto:Method: GET 2022-09-03T06:47:26.466 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:47:26.467 DEBUG:boto:Data: 2022-09-03T06:47:26.467 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.468 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.469 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.469 DEBUG:boto:Params: {} 2022-09-03T06:47:26.470 DEBUG:boto:Token: None 2022-09-03T06:47:26.470 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/ 2022-09-03T06:47:26.471 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg= 2022-09-03T06:47:26.472 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg=', 'Content-Length': '0'} 2022-09-03T06:47:26.475 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a5cf84959ed78afb-006312f87e-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.475 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:26.476 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:26.477 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:26.477 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:26.478 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:26.478 DEBUG:boto:Method: GET 2022-09-03T06:47:26.479 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:47:26.479 DEBUG:boto:Data: 2022-09-03T06:47:26.480 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.480 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.481 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.481 DEBUG:boto:Params: {} 2022-09-03T06:47:26.482 DEBUG:boto:Token: None 2022-09-03T06:47:26.483 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/ 2022-09-03T06:47:26.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg= 2022-09-03T06:47:26.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg=', 'Content-Length': '0'} 2022-09-03T06:47:26.488 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000041e6adb61793a2cd-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.489 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:26.490 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2022-09-03T06:47:26.491 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:26.492 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:26.492 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.493 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.494 DEBUG:boto:Method: GET 2022-09-03T06:47:26.494 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.495 DEBUG:boto:Data: 2022-09-03T06:47:26.496 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.496 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.497 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.497 DEBUG:boto:Params: {} 2022-09-03T06:47:26.498 DEBUG:boto:Token: None 2022-09-03T06:47:26.499 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.500 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tj6pACvr5mcFJPuoyL1FB2DF5hw= 2022-09-03T06:47:26.500 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tj6pACvr5mcFJPuoyL1FB2DF5hw=', 'Content-Length': '0'} 2022-09-03T06:47:26.503 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f099a5712c166e97-006312f87e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.550 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:26.551 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:26.552 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.552 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.553 DEBUG:boto:Method: GET 2022-09-03T06:47:26.553 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.554 DEBUG:boto:Data: 2022-09-03T06:47:26.555 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.555 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.556 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.557 DEBUG:boto:Params: {} 2022-09-03T06:47:26.557 DEBUG:boto:Token: None 2022-09-03T06:47:26.558 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.559 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tj6pACvr5mcFJPuoyL1FB2DF5hw= 2022-09-03T06:47:26.559 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tj6pACvr5mcFJPuoyL1FB2DF5hw=', 'Content-Length': '0'} 2022-09-03T06:47:26.562 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e7a81f427e9bd07a-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.563 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:26.564 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:26.564 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.566 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.567 DEBUG:boto:Method: HEAD 2022-09-03T06:47:26.568 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.569 DEBUG:boto:Data: 2022-09-03T06:47:26.570 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.570 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.571 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.571 DEBUG:boto:Params: {} 2022-09-03T06:47:26.572 DEBUG:boto:Token: None 2022-09-03T06:47:26.573 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.573 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YtozDG8VZSKHIfwQanbJ8b3w0ZY= 2022-09-03T06:47:26.574 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YtozDG8VZSKHIfwQanbJ8b3w0ZY=', 'Content-Length': '0'} 2022-09-03T06:47:26.576 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000093c4e3c7f77b92ed-006312f87e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.578 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:26.579 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:26.580 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.580 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.581 DEBUG:boto:Method: HEAD 2022-09-03T06:47:26.582 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.582 DEBUG:boto:Data: 2022-09-03T06:47:26.583 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.583 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.584 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.584 DEBUG:boto:Params: {} 2022-09-03T06:47:26.585 DEBUG:boto:Token: None 2022-09-03T06:47:26.585 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.586 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YtozDG8VZSKHIfwQanbJ8b3w0ZY= 2022-09-03T06:47:26.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YtozDG8VZSKHIfwQanbJ8b3w0ZY=', 'Content-Length': '0'} 2022-09-03T06:47:26.589 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000073e937e759075e8e-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.589 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2022-09-03T06:47:26.590 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:26.590 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:26.591 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.592 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.592 DEBUG:boto:Method: GET 2022-09-03T06:47:26.593 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.593 DEBUG:boto:Data: 2022-09-03T06:47:26.594 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.594 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.595 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.595 DEBUG:boto:Params: {} 2022-09-03T06:47:26.596 DEBUG:boto:Token: None 2022-09-03T06:47:26.597 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.597 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tj6pACvr5mcFJPuoyL1FB2DF5hw= 2022-09-03T06:47:26.598 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tj6pACvr5mcFJPuoyL1FB2DF5hw=', 'Content-Length': '0'} 2022-09-03T06:47:26.600 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006a91ade95a1c778d-006312f87e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.601 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:26.602 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:26.602 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.603 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.603 DEBUG:boto:Method: GET 2022-09-03T06:47:26.604 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.604 DEBUG:boto:Data: 2022-09-03T06:47:26.605 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.605 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.606 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.606 DEBUG:boto:Params: {} 2022-09-03T06:47:26.607 DEBUG:boto:Token: None 2022-09-03T06:47:26.608 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:26.609 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tj6pACvr5mcFJPuoyL1FB2DF5hw= 2022-09-03T06:47:26.609 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tj6pACvr5mcFJPuoyL1FB2DF5hw=', 'Content-Length': '0'} 2022-09-03T06:47:26.611 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007414d1e6b56fa116-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.612 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:26.613 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:26.613 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:26.614 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:26.614 DEBUG:boto:Method: GET 2022-09-03T06:47:26.615 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:47:26.615 DEBUG:boto:Data: 2022-09-03T06:47:26.616 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.616 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.617 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.617 DEBUG:boto:Params: {} 2022-09-03T06:47:26.618 DEBUG:boto:Token: None 2022-09-03T06:47:26.619 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/ 2022-09-03T06:47:26.619 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg= 2022-09-03T06:47:26.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg=', 'Content-Length': '0'} 2022-09-03T06:47:26.623 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e16c33ca07cb0fc0-006312f87e-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.624 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:26.625 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:26.625 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:26.626 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:26.627 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:26.627 DEBUG:boto:Method: GET 2022-09-03T06:47:26.628 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:47:26.628 DEBUG:boto:Data: 2022-09-03T06:47:26.629 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.629 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.630 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.630 DEBUG:boto:Params: {} 2022-09-03T06:47:26.631 DEBUG:boto:Token: None 2022-09-03T06:47:26.631 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/ 2022-09-03T06:47:26.632 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg= 2022-09-03T06:47:26.633 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg=', 'Content-Length': '0'} 2022-09-03T06:47:26.637 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000df914c4dfa0f6e5b-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.638 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:26.638 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:26.639 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:26.640 DEBUG:boto:Method: HEAD 2022-09-03T06:47:26.640 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:47:26.641 DEBUG:boto:Data: 2022-09-03T06:47:26.641 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.642 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.642 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.643 DEBUG:boto:Params: {} 2022-09-03T06:47:26.643 DEBUG:boto:Token: None 2022-09-03T06:47:26.644 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj31 2022-09-03T06:47:26.644 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f28kf+476FxNmVKrtHTtUVOV+Kc= 2022-09-03T06:47:26.645 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f28kf+476FxNmVKrtHTtUVOV+Kc=', 'Content-Length': '0'} 2022-09-03T06:47:26.647 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bc2ba311b9122768-006312f87e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.648 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:26.648 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:26.649 DEBUG:boto:Method: HEAD 2022-09-03T06:47:26.649 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:47:26.650 DEBUG:boto:Data: 2022-09-03T06:47:26.650 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.651 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.651 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.652 DEBUG:boto:Params: {} 2022-09-03T06:47:26.652 DEBUG:boto:Token: None 2022-09-03T06:47:26.653 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj31 2022-09-03T06:47:26.653 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:f28kf+476FxNmVKrtHTtUVOV+Kc= 2022-09-03T06:47:26.654 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:f28kf+476FxNmVKrtHTtUVOV+Kc=', 'Content-Length': '0'} 2022-09-03T06:47:26.656 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f84ca0f17c85eb22-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.657 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2022-09-03T06:47:26.658 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:26.658 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:26.659 DEBUG:boto:Method: GET 2022-09-03T06:47:26.659 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:47:26.660 DEBUG:boto:Data: 2022-09-03T06:47:26.660 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.661 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.662 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.662 DEBUG:boto:Params: {} 2022-09-03T06:47:26.663 DEBUG:boto:Token: None 2022-09-03T06:47:26.663 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj31 2022-09-03T06:47:26.664 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yjiRpHszhHOtpBn5ONJvbyYVAXM= 2022-09-03T06:47:26.665 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yjiRpHszhHOtpBn5ONJvbyYVAXM=', 'Content-Length': '0'} 2022-09-03T06:47:26.667 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e550ec6ab28366a-006312f87e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.714 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:26.715 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:26.716 DEBUG:boto:Method: GET 2022-09-03T06:47:26.717 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:47:26.718 DEBUG:boto:Data: 2022-09-03T06:47:26.720 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.720 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.721 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.721 DEBUG:boto:Params: {} 2022-09-03T06:47:26.722 DEBUG:boto:Token: None 2022-09-03T06:47:26.723 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj31 2022-09-03T06:47:26.723 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yjiRpHszhHOtpBn5ONJvbyYVAXM= 2022-09-03T06:47:26.724 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yjiRpHszhHOtpBn5ONJvbyYVAXM=', 'Content-Length': '0'} 2022-09-03T06:47:26.727 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d6ff607754327610-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.728 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:26.729 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:26.730 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:26.731 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:26.732 DEBUG:boto:Method: GET 2022-09-03T06:47:26.733 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:47:26.734 DEBUG:boto:Data: 2022-09-03T06:47:26.734 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.735 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.736 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.736 DEBUG:boto:Params: {} 2022-09-03T06:47:26.737 DEBUG:boto:Token: None 2022-09-03T06:47:26.739 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/ 2022-09-03T06:47:26.740 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg= 2022-09-03T06:47:26.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg=', 'Content-Length': '0'} 2022-09-03T06:47:26.743 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001663342d3c60b644-006312f87e-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.744 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:26.745 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:26.746 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:26.746 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:26.747 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:26.748 DEBUG:boto:Method: GET 2022-09-03T06:47:26.748 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:47:26.749 DEBUG:boto:Data: 2022-09-03T06:47:26.749 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.750 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.751 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.752 DEBUG:boto:Params: {} 2022-09-03T06:47:26.754 DEBUG:boto:Token: None 2022-09-03T06:47:26.755 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/ 2022-09-03T06:47:26.756 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg= 2022-09-03T06:47:26.757 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg=', 'Content-Length': '0'} 2022-09-03T06:47:26.762 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001652f7eb68d9851f-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.764 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:26.765 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:47:26.765 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:26.766 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:26.767 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.767 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.768 DEBUG:boto:Method: GET 2022-09-03T06:47:26.768 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.769 DEBUG:boto:Data: 2022-09-03T06:47:26.769 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.770 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.771 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.773 DEBUG:boto:Params: {} 2022-09-03T06:47:26.773 DEBUG:boto:Token: None 2022-09-03T06:47:26.774 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.775 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iDCNX+nNzvt+T+ys4j6Et504G4c= 2022-09-03T06:47:26.775 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iDCNX+nNzvt+T+ys4j6Et504G4c=', 'Content-Length': '0'} 2022-09-03T06:47:26.778 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d1fec81a140f10b-006312f87e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.779 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:26.779 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:26.780 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.781 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.782 DEBUG:boto:Method: GET 2022-09-03T06:47:26.783 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.783 DEBUG:boto:Data: 2022-09-03T06:47:26.784 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.785 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.785 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.786 DEBUG:boto:Params: {} 2022-09-03T06:47:26.787 DEBUG:boto:Token: None 2022-09-03T06:47:26.788 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.789 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iDCNX+nNzvt+T+ys4j6Et504G4c= 2022-09-03T06:47:26.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iDCNX+nNzvt+T+ys4j6Et504G4c=', 'Content-Length': '0'} 2022-09-03T06:47:26.794 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cb12950db5a85f30-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.796 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:26.797 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:26.797 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.798 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.799 DEBUG:boto:Method: HEAD 2022-09-03T06:47:26.800 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.801 DEBUG:boto:Data: 2022-09-03T06:47:26.802 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.803 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.804 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.805 DEBUG:boto:Params: {} 2022-09-03T06:47:26.806 DEBUG:boto:Token: None 2022-09-03T06:47:26.807 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.808 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CQWCBNR17TRU8K1BO5PAr08PfUE= 2022-09-03T06:47:26.809 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CQWCBNR17TRU8K1BO5PAr08PfUE=', 'Content-Length': '0'} 2022-09-03T06:47:26.812 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000710d276068f57f99-006312f87e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.813 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:26.814 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:26.815 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.816 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.817 DEBUG:boto:Method: HEAD 2022-09-03T06:47:26.818 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.819 DEBUG:boto:Data: 2022-09-03T06:47:26.820 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.821 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.821 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.822 DEBUG:boto:Params: {} 2022-09-03T06:47:26.823 DEBUG:boto:Token: None 2022-09-03T06:47:26.825 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.826 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CQWCBNR17TRU8K1BO5PAr08PfUE= 2022-09-03T06:47:26.827 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CQWCBNR17TRU8K1BO5PAr08PfUE=', 'Content-Length': '0'} 2022-09-03T06:47:26.830 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d0f858047cd725a3-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.831 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:47:26.832 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:26.833 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:26.833 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.834 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.835 DEBUG:boto:Method: GET 2022-09-03T06:47:26.835 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.836 DEBUG:boto:Data: 2022-09-03T06:47:26.836 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.837 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.837 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.838 DEBUG:boto:Params: {} 2022-09-03T06:47:26.839 DEBUG:boto:Token: None 2022-09-03T06:47:26.839 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.840 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iDCNX+nNzvt+T+ys4j6Et504G4c= 2022-09-03T06:47:26.841 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iDCNX+nNzvt+T+ys4j6Et504G4c=', 'Content-Length': '0'} 2022-09-03T06:47:26.843 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e19b6051393703f6-006312f87e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.844 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:26.844 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:26.845 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.845 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.846 DEBUG:boto:Method: GET 2022-09-03T06:47:26.847 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.847 DEBUG:boto:Data: 2022-09-03T06:47:26.848 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.849 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.850 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.850 DEBUG:boto:Params: {} 2022-09-03T06:47:26.851 DEBUG:boto:Token: None 2022-09-03T06:47:26.851 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:26.852 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iDCNX+nNzvt+T+ys4j6Et504G4c= 2022-09-03T06:47:26.853 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iDCNX+nNzvt+T+ys4j6Et504G4c=', 'Content-Length': '0'} 2022-09-03T06:47:26.855 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000397bdad0fec6f1ac-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.856 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:26.857 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:26.857 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:26.858 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:26.858 DEBUG:boto:Method: GET 2022-09-03T06:47:26.859 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:47:26.860 DEBUG:boto:Data: 2022-09-03T06:47:26.860 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.861 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.861 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.862 DEBUG:boto:Params: {} 2022-09-03T06:47:26.863 DEBUG:boto:Token: None 2022-09-03T06:47:26.863 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/ 2022-09-03T06:47:26.864 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg= 2022-09-03T06:47:26.865 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg=', 'Content-Length': '0'} 2022-09-03T06:47:26.872 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005773c07f82f843a7-006312f87e-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.914 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:26.916 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:26.917 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:26.918 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:26.918 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:26.919 DEBUG:boto:Method: GET 2022-09-03T06:47:26.921 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:47:26.922 DEBUG:boto:Data: 2022-09-03T06:47:26.922 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.923 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.924 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.925 DEBUG:boto:Params: {} 2022-09-03T06:47:26.927 DEBUG:boto:Token: None 2022-09-03T06:47:26.928 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/ 2022-09-03T06:47:26.929 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg= 2022-09-03T06:47:26.930 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/oiCkF/OasmTwO/NaeogrBWyKg=', 'Content-Length': '0'} 2022-09-03T06:47:26.935 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ec7c71bf0a3162eb-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.936 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:26.937 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:26.938 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:26.938 DEBUG:boto:Method: HEAD 2022-09-03T06:47:26.939 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:47:26.940 DEBUG:boto:Data: 2022-09-03T06:47:26.940 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.941 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.942 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.942 DEBUG:boto:Params: {} 2022-09-03T06:47:26.943 DEBUG:boto:Token: None 2022-09-03T06:47:26.944 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj33 2022-09-03T06:47:26.944 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N537Xlen84LGbPQW5G1nsqaWt8o= 2022-09-03T06:47:26.945 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N537Xlen84LGbPQW5G1nsqaWt8o=', 'Content-Length': '0'} 2022-09-03T06:47:26.947 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dfe662f09d3d62d1-006312f87e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.948 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:26.949 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:26.949 DEBUG:boto:Method: HEAD 2022-09-03T06:47:26.950 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:47:26.950 DEBUG:boto:Data: 2022-09-03T06:47:26.951 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.951 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.952 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.952 DEBUG:boto:Params: {} 2022-09-03T06:47:26.953 DEBUG:boto:Token: None 2022-09-03T06:47:26.954 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj33 2022-09-03T06:47:26.954 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N537Xlen84LGbPQW5G1nsqaWt8o= 2022-09-03T06:47:26.955 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N537Xlen84LGbPQW5G1nsqaWt8o=', 'Content-Length': '0'} 2022-09-03T06:47:26.957 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000099e8755e7143b23a-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.958 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:47:26.959 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:26.959 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:26.960 DEBUG:boto:Method: GET 2022-09-03T06:47:26.960 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:47:26.961 DEBUG:boto:Data: 2022-09-03T06:47:26.961 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.962 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:26.963 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.963 DEBUG:boto:Params: {} 2022-09-03T06:47:26.964 DEBUG:boto:Token: None 2022-09-03T06:47:26.964 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj33 2022-09-03T06:47:26.965 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6KRiSi5JNsSEXE8EZ9Y3R/7k5gQ= 2022-09-03T06:47:26.966 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6KRiSi5JNsSEXE8EZ9Y3R/7k5gQ=', 'Content-Length': '0'} 2022-09-03T06:47:26.968 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f605780dea07ac89-006312f87e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:26.969 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:26.970 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:26.970 DEBUG:boto:Method: GET 2022-09-03T06:47:26.971 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:47:26.971 DEBUG:boto:Data: 2022-09-03T06:47:26.973 DEBUG:boto:Headers: {} 2022-09-03T06:47:26.973 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:26.974 DEBUG:boto:Port: 8000 2022-09-03T06:47:26.975 DEBUG:boto:Params: {} 2022-09-03T06:47:26.975 DEBUG:boto:Token: None 2022-09-03T06:47:26.976 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:26 GMT /dsgvcy-75/obj33 2022-09-03T06:47:26.977 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6KRiSi5JNsSEXE8EZ9Y3R/7k5gQ= 2022-09-03T06:47:26.978 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6KRiSi5JNsSEXE8EZ9Y3R/7k5gQ=', 'Content-Length': '0'} 2022-09-03T06:47:26.980 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074ba8b5eb33fa670-006312f87e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.022 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:27.023 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:27.024 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:27.024 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:27.025 DEBUG:boto:Method: GET 2022-09-03T06:47:27.026 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:47:27.027 DEBUG:boto:Data: 2022-09-03T06:47:27.028 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.029 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.029 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.031 DEBUG:boto:Params: {} 2022-09-03T06:47:27.033 DEBUG:boto:Token: None 2022-09-03T06:47:27.033 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/ 2022-09-03T06:47:27.034 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw= 2022-09-03T06:47:27.035 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw=', 'Content-Length': '0'} 2022-09-03T06:47:27.039 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c218247a75a850d7-006312f87f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.039 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:27.041 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:27.042 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:27.042 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:27.043 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:27.044 DEBUG:boto:Method: GET 2022-09-03T06:47:27.044 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:47:27.045 DEBUG:boto:Data: 2022-09-03T06:47:27.045 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.046 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.047 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.047 DEBUG:boto:Params: {} 2022-09-03T06:47:27.048 DEBUG:boto:Token: None 2022-09-03T06:47:27.048 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/ 2022-09-03T06:47:27.049 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw= 2022-09-03T06:47:27.050 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw=', 'Content-Length': '0'} 2022-09-03T06:47:27.054 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003a074049fbff3235-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.055 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:27.056 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:47:27.056 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:27.057 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:27.057 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.058 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.059 DEBUG:boto:Method: GET 2022-09-03T06:47:27.059 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.060 DEBUG:boto:Data: 2022-09-03T06:47:27.060 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.061 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.062 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.063 DEBUG:boto:Params: {} 2022-09-03T06:47:27.063 DEBUG:boto:Token: None 2022-09-03T06:47:27.064 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.065 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k/+tgOdLrIvMGUVLM569/LOhL3Q= 2022-09-03T06:47:27.065 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k/+tgOdLrIvMGUVLM569/LOhL3Q=', 'Content-Length': '0'} 2022-09-03T06:47:27.068 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bc916ab8eba85316-006312f87f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.070 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:27.071 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:27.072 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.073 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.074 DEBUG:boto:Method: GET 2022-09-03T06:47:27.075 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.076 DEBUG:boto:Data: 2022-09-03T06:47:27.077 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.078 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.078 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.079 DEBUG:boto:Params: {} 2022-09-03T06:47:27.081 DEBUG:boto:Token: None 2022-09-03T06:47:27.082 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.083 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k/+tgOdLrIvMGUVLM569/LOhL3Q= 2022-09-03T06:47:27.084 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k/+tgOdLrIvMGUVLM569/LOhL3Q=', 'Content-Length': '0'} 2022-09-03T06:47:27.087 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a84257af960ee102-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.089 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:27.090 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:27.090 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.091 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.092 DEBUG:boto:Method: HEAD 2022-09-03T06:47:27.093 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.093 DEBUG:boto:Data: 2022-09-03T06:47:27.094 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.095 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.095 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.096 DEBUG:boto:Params: {} 2022-09-03T06:47:27.097 DEBUG:boto:Token: None 2022-09-03T06:47:27.098 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.098 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p5ZM1bo77AylWsn6i+a/dNObN3I= 2022-09-03T06:47:27.099 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p5ZM1bo77AylWsn6i+a/dNObN3I=', 'Content-Length': '0'} 2022-09-03T06:47:27.101 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e2e1caa31ff9b6f9-006312f87f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.102 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:27.103 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:27.104 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.105 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.106 DEBUG:boto:Method: HEAD 2022-09-03T06:47:27.107 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.108 DEBUG:boto:Data: 2022-09-03T06:47:27.109 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.110 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.111 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.111 DEBUG:boto:Params: {} 2022-09-03T06:47:27.112 DEBUG:boto:Token: None 2022-09-03T06:47:27.113 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.113 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p5ZM1bo77AylWsn6i+a/dNObN3I= 2022-09-03T06:47:27.114 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p5ZM1bo77AylWsn6i+a/dNObN3I=', 'Content-Length': '0'} 2022-09-03T06:47:27.117 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d477128b58ec293f-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.118 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:47:27.118 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:27.119 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:27.119 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.120 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.121 DEBUG:boto:Method: GET 2022-09-03T06:47:27.122 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.123 DEBUG:boto:Data: 2022-09-03T06:47:27.124 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.125 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.126 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.127 DEBUG:boto:Params: {} 2022-09-03T06:47:27.128 DEBUG:boto:Token: None 2022-09-03T06:47:27.129 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.130 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k/+tgOdLrIvMGUVLM569/LOhL3Q= 2022-09-03T06:47:27.131 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k/+tgOdLrIvMGUVLM569/LOhL3Q=', 'Content-Length': '0'} 2022-09-03T06:47:27.133 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f9ed110d0e446740-006312f87f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.135 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:27.135 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:27.137 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.138 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.139 DEBUG:boto:Method: GET 2022-09-03T06:47:27.139 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.140 DEBUG:boto:Data: 2022-09-03T06:47:27.140 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.141 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.142 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.142 DEBUG:boto:Params: {} 2022-09-03T06:47:27.143 DEBUG:boto:Token: None 2022-09-03T06:47:27.144 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:27.144 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k/+tgOdLrIvMGUVLM569/LOhL3Q= 2022-09-03T06:47:27.145 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k/+tgOdLrIvMGUVLM569/LOhL3Q=', 'Content-Length': '0'} 2022-09-03T06:47:27.147 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b8b601a0878dc7b-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.190 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:27.191 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:27.192 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:27.192 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:27.193 DEBUG:boto:Method: GET 2022-09-03T06:47:27.194 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:47:27.194 DEBUG:boto:Data: 2022-09-03T06:47:27.195 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.196 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.196 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.197 DEBUG:boto:Params: {} 2022-09-03T06:47:27.198 DEBUG:boto:Token: None 2022-09-03T06:47:27.199 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/ 2022-09-03T06:47:27.200 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw= 2022-09-03T06:47:27.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw=', 'Content-Length': '0'} 2022-09-03T06:47:27.204 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c830ec3467068282-006312f87f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.205 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:27.206 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:27.207 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:27.208 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:27.209 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:27.210 DEBUG:boto:Method: GET 2022-09-03T06:47:27.211 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:47:27.212 DEBUG:boto:Data: 2022-09-03T06:47:27.213 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.214 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.215 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.216 DEBUG:boto:Params: {} 2022-09-03T06:47:27.217 DEBUG:boto:Token: None 2022-09-03T06:47:27.218 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/ 2022-09-03T06:47:27.219 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw= 2022-09-03T06:47:27.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw=', 'Content-Length': '0'} 2022-09-03T06:47:27.225 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004f9f8f9a2a9ca8c8-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.226 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:27.227 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:27.227 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:27.228 DEBUG:boto:Method: HEAD 2022-09-03T06:47:27.228 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:47:27.229 DEBUG:boto:Data: 2022-09-03T06:47:27.229 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.230 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.231 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.231 DEBUG:boto:Params: {} 2022-09-03T06:47:27.232 DEBUG:boto:Token: None 2022-09-03T06:47:27.233 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj34 2022-09-03T06:47:27.234 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/5y2MwHXc5XVfo/o5pBToHM3cbU= 2022-09-03T06:47:27.235 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/5y2MwHXc5XVfo/o5pBToHM3cbU=', 'Content-Length': '0'} 2022-09-03T06:47:27.237 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d7dab92bd1fe789e-006312f87f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.238 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:27.239 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:27.239 DEBUG:boto:Method: HEAD 2022-09-03T06:47:27.240 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:47:27.240 DEBUG:boto:Data: 2022-09-03T06:47:27.241 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.241 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.242 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.242 DEBUG:boto:Params: {} 2022-09-03T06:47:27.243 DEBUG:boto:Token: None 2022-09-03T06:47:27.244 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj34 2022-09-03T06:47:27.244 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/5y2MwHXc5XVfo/o5pBToHM3cbU= 2022-09-03T06:47:27.245 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/5y2MwHXc5XVfo/o5pBToHM3cbU=', 'Content-Length': '0'} 2022-09-03T06:47:27.247 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000266816d25dc4b964-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.248 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:47:27.249 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:27.249 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:27.250 DEBUG:boto:Method: GET 2022-09-03T06:47:27.251 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:47:27.251 DEBUG:boto:Data: 2022-09-03T06:47:27.252 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.252 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.253 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.253 DEBUG:boto:Params: {} 2022-09-03T06:47:27.254 DEBUG:boto:Token: None 2022-09-03T06:47:27.255 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj34 2022-09-03T06:47:27.255 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8UBSxA20NHOE+jtlhNUmO6/xo4k= 2022-09-03T06:47:27.256 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8UBSxA20NHOE+jtlhNUmO6/xo4k=', 'Content-Length': '0'} 2022-09-03T06:47:27.259 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000befb799b63ee8a89-006312f87f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.302 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:27.303 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:27.304 DEBUG:boto:Method: GET 2022-09-03T06:47:27.304 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:47:27.305 DEBUG:boto:Data: 2022-09-03T06:47:27.306 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.307 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.308 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.309 DEBUG:boto:Params: {} 2022-09-03T06:47:27.309 DEBUG:boto:Token: None 2022-09-03T06:47:27.311 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj34 2022-09-03T06:47:27.312 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8UBSxA20NHOE+jtlhNUmO6/xo4k= 2022-09-03T06:47:27.313 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8UBSxA20NHOE+jtlhNUmO6/xo4k=', 'Content-Length': '0'} 2022-09-03T06:47:27.315 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000377614c1f4c20243-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.316 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:27.317 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:27.318 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:27.319 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:27.319 DEBUG:boto:Method: GET 2022-09-03T06:47:27.320 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:47:27.321 DEBUG:boto:Data: 2022-09-03T06:47:27.322 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.323 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.324 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.324 DEBUG:boto:Params: {} 2022-09-03T06:47:27.325 DEBUG:boto:Token: None 2022-09-03T06:47:27.326 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/ 2022-09-03T06:47:27.327 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw= 2022-09-03T06:47:27.327 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw=', 'Content-Length': '0'} 2022-09-03T06:47:27.330 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005be4a7c011f16400-006312f87f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.331 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:27.332 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:27.333 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:27.333 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:27.335 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:27.335 DEBUG:boto:Method: GET 2022-09-03T06:47:27.336 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:47:27.337 DEBUG:boto:Data: 2022-09-03T06:47:27.337 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.338 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.339 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.340 DEBUG:boto:Params: {} 2022-09-03T06:47:27.341 DEBUG:boto:Token: None 2022-09-03T06:47:27.342 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/ 2022-09-03T06:47:27.343 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw= 2022-09-03T06:47:27.344 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw=', 'Content-Length': '0'} 2022-09-03T06:47:27.351 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009d076cf2b0e843db-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.352 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:27.353 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:47:27.354 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:27.355 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:27.356 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.356 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.357 DEBUG:boto:Method: GET 2022-09-03T06:47:27.358 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.358 DEBUG:boto:Data: 2022-09-03T06:47:27.359 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.360 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.361 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.361 DEBUG:boto:Params: {} 2022-09-03T06:47:27.362 DEBUG:boto:Token: None 2022-09-03T06:47:27.363 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.364 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fiMBtCjPyqFY0F9Th/+zbmhjt1g= 2022-09-03T06:47:27.365 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fiMBtCjPyqFY0F9Th/+zbmhjt1g=', 'Content-Length': '0'} 2022-09-03T06:47:27.367 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065e2e2380d59bcf0-006312f87f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.368 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:27.368 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:27.369 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.369 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.370 DEBUG:boto:Method: GET 2022-09-03T06:47:27.371 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.372 DEBUG:boto:Data: 2022-09-03T06:47:27.372 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.373 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.373 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.374 DEBUG:boto:Params: {} 2022-09-03T06:47:27.374 DEBUG:boto:Token: None 2022-09-03T06:47:27.375 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.376 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fiMBtCjPyqFY0F9Th/+zbmhjt1g= 2022-09-03T06:47:27.376 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fiMBtCjPyqFY0F9Th/+zbmhjt1g=', 'Content-Length': '0'} 2022-09-03T06:47:27.379 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008c6e96e6af31edb7-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.422 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:27.423 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:27.424 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.425 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.426 DEBUG:boto:Method: HEAD 2022-09-03T06:47:27.427 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.428 DEBUG:boto:Data: 2022-09-03T06:47:27.430 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.431 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.432 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.433 DEBUG:boto:Params: {} 2022-09-03T06:47:27.434 DEBUG:boto:Token: None 2022-09-03T06:47:27.435 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.436 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nf22LqHXbB541wMKrtBvLLJ9/kw= 2022-09-03T06:47:27.437 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nf22LqHXbB541wMKrtBvLLJ9/kw=', 'Content-Length': '0'} 2022-09-03T06:47:27.440 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d68c5291d2d9a50-006312f87f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.442 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:27.443 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:27.444 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.445 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.446 DEBUG:boto:Method: HEAD 2022-09-03T06:47:27.447 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.448 DEBUG:boto:Data: 2022-09-03T06:47:27.448 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.449 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.450 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.451 DEBUG:boto:Params: {} 2022-09-03T06:47:27.452 DEBUG:boto:Token: None 2022-09-03T06:47:27.453 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.454 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nf22LqHXbB541wMKrtBvLLJ9/kw= 2022-09-03T06:47:27.455 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nf22LqHXbB541wMKrtBvLLJ9/kw=', 'Content-Length': '0'} 2022-09-03T06:47:27.458 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007509263a8336bcb3-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.459 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:47:27.460 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:27.461 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:27.461 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.462 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.463 DEBUG:boto:Method: GET 2022-09-03T06:47:27.463 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.464 DEBUG:boto:Data: 2022-09-03T06:47:27.465 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.465 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.466 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.466 DEBUG:boto:Params: {} 2022-09-03T06:47:27.467 DEBUG:boto:Token: None 2022-09-03T06:47:27.468 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.468 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fiMBtCjPyqFY0F9Th/+zbmhjt1g= 2022-09-03T06:47:27.469 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fiMBtCjPyqFY0F9Th/+zbmhjt1g=', 'Content-Length': '0'} 2022-09-03T06:47:27.472 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cb881ae9fb20e6e6-006312f87f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.473 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:27.473 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:27.474 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.475 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.475 DEBUG:boto:Method: GET 2022-09-03T06:47:27.476 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.477 DEBUG:boto:Data: 2022-09-03T06:47:27.477 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.478 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.478 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.479 DEBUG:boto:Params: {} 2022-09-03T06:47:27.480 DEBUG:boto:Token: None 2022-09-03T06:47:27.480 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:27.482 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fiMBtCjPyqFY0F9Th/+zbmhjt1g= 2022-09-03T06:47:27.483 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fiMBtCjPyqFY0F9Th/+zbmhjt1g=', 'Content-Length': '0'} 2022-09-03T06:47:27.485 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000814ade96af90bef-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.487 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:27.487 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:27.488 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:27.489 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:27.490 DEBUG:boto:Method: GET 2022-09-03T06:47:27.491 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:47:27.492 DEBUG:boto:Data: 2022-09-03T06:47:27.492 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.493 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.494 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.495 DEBUG:boto:Params: {} 2022-09-03T06:47:27.496 DEBUG:boto:Token: None 2022-09-03T06:47:27.497 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/ 2022-09-03T06:47:27.498 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw= 2022-09-03T06:47:27.500 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw=', 'Content-Length': '0'} 2022-09-03T06:47:27.503 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000076bf348cab2b19ce-006312f87f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.504 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:27.505 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:27.506 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:27.506 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:27.507 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:27.508 DEBUG:boto:Method: GET 2022-09-03T06:47:27.508 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:47:27.509 DEBUG:boto:Data: 2022-09-03T06:47:27.510 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.511 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.512 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.513 DEBUG:boto:Params: {} 2022-09-03T06:47:27.514 DEBUG:boto:Token: None 2022-09-03T06:47:27.515 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/ 2022-09-03T06:47:27.516 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw= 2022-09-03T06:47:27.517 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw=', 'Content-Length': '0'} 2022-09-03T06:47:27.522 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f14f6c9ae58c7f76-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.523 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:27.524 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:27.525 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:27.526 DEBUG:boto:Method: HEAD 2022-09-03T06:47:27.527 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:47:27.528 DEBUG:boto:Data: 2022-09-03T06:47:27.529 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.530 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.531 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.531 DEBUG:boto:Params: {} 2022-09-03T06:47:27.532 DEBUG:boto:Token: None 2022-09-03T06:47:27.533 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj38 2022-09-03T06:47:27.533 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/g1hwy3Q4mXQlJENK1J+x0CKLOI= 2022-09-03T06:47:27.534 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/g1hwy3Q4mXQlJENK1J+x0CKLOI=', 'Content-Length': '0'} 2022-09-03T06:47:27.536 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2e63ed688b8b232-006312f87f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.537 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:27.538 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:27.538 DEBUG:boto:Method: HEAD 2022-09-03T06:47:27.539 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:47:27.540 DEBUG:boto:Data: 2022-09-03T06:47:27.540 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.541 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.541 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.542 DEBUG:boto:Params: {} 2022-09-03T06:47:27.543 DEBUG:boto:Token: None 2022-09-03T06:47:27.543 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj38 2022-09-03T06:47:27.544 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/g1hwy3Q4mXQlJENK1J+x0CKLOI= 2022-09-03T06:47:27.545 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/g1hwy3Q4mXQlJENK1J+x0CKLOI=', 'Content-Length': '0'} 2022-09-03T06:47:27.547 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b8a508d8533e22ab-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.548 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:47:27.549 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:27.550 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:27.550 DEBUG:boto:Method: GET 2022-09-03T06:47:27.551 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:47:27.551 DEBUG:boto:Data: 2022-09-03T06:47:27.552 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.552 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.553 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.554 DEBUG:boto:Params: {} 2022-09-03T06:47:27.554 DEBUG:boto:Token: None 2022-09-03T06:47:27.555 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj38 2022-09-03T06:47:27.556 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V6FKAaD5HW3GB6n6JlKuDrd2vOM= 2022-09-03T06:47:27.556 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V6FKAaD5HW3GB6n6JlKuDrd2vOM=', 'Content-Length': '0'} 2022-09-03T06:47:27.558 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006e681ac22fcabb91-006312f87f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.559 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:27.560 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:27.560 DEBUG:boto:Method: GET 2022-09-03T06:47:27.561 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:47:27.562 DEBUG:boto:Data: 2022-09-03T06:47:27.562 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.563 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.563 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.564 DEBUG:boto:Params: {} 2022-09-03T06:47:27.565 DEBUG:boto:Token: None 2022-09-03T06:47:27.565 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj38 2022-09-03T06:47:27.566 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V6FKAaD5HW3GB6n6JlKuDrd2vOM= 2022-09-03T06:47:27.567 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V6FKAaD5HW3GB6n6JlKuDrd2vOM=', 'Content-Length': '0'} 2022-09-03T06:47:27.569 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000013e9d1e0e7fd241d-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.570 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:27.571 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:27.571 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:27.572 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:27.572 DEBUG:boto:Method: GET 2022-09-03T06:47:27.573 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:47:27.573 DEBUG:boto:Data: 2022-09-03T06:47:27.574 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.574 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.575 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.576 DEBUG:boto:Params: {} 2022-09-03T06:47:27.576 DEBUG:boto:Token: None 2022-09-03T06:47:27.577 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/ 2022-09-03T06:47:27.577 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw= 2022-09-03T06:47:27.578 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw=', 'Content-Length': '0'} 2022-09-03T06:47:27.583 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000429ddf7a0aff5231-006312f87f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.630 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:27.632 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:27.632 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:27.633 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:27.633 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:27.634 DEBUG:boto:Method: GET 2022-09-03T06:47:27.635 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:47:27.636 DEBUG:boto:Data: 2022-09-03T06:47:27.636 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.637 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.637 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.638 DEBUG:boto:Params: {} 2022-09-03T06:47:27.639 DEBUG:boto:Token: None 2022-09-03T06:47:27.640 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/ 2022-09-03T06:47:27.641 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw= 2022-09-03T06:47:27.641 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw=', 'Content-Length': '0'} 2022-09-03T06:47:27.648 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000439cdd36508bdf8e-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.649 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:27.650 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:47:27.651 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:27.651 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:27.652 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.652 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.653 DEBUG:boto:Method: GET 2022-09-03T06:47:27.654 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.654 DEBUG:boto:Data: 2022-09-03T06:47:27.655 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.655 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.656 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.657 DEBUG:boto:Params: {} 2022-09-03T06:47:27.657 DEBUG:boto:Token: None 2022-09-03T06:47:27.658 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.659 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TNABhsU0iXYpmmfwtQqnJ4CNd20= 2022-09-03T06:47:27.659 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TNABhsU0iXYpmmfwtQqnJ4CNd20=', 'Content-Length': '0'} 2022-09-03T06:47:27.662 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000846338360a7434f2-006312f87f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.663 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:27.664 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:27.665 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.666 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.666 DEBUG:boto:Method: GET 2022-09-03T06:47:27.667 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.668 DEBUG:boto:Data: 2022-09-03T06:47:27.668 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.669 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.670 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.670 DEBUG:boto:Params: {} 2022-09-03T06:47:27.671 DEBUG:boto:Token: None 2022-09-03T06:47:27.672 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.673 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TNABhsU0iXYpmmfwtQqnJ4CNd20= 2022-09-03T06:47:27.674 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TNABhsU0iXYpmmfwtQqnJ4CNd20=', 'Content-Length': '0'} 2022-09-03T06:47:27.677 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ec202c7538107d28-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.718 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:27.719 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:27.719 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.720 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.720 DEBUG:boto:Method: HEAD 2022-09-03T06:47:27.721 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.721 DEBUG:boto:Data: 2022-09-03T06:47:27.722 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.722 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.723 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.723 DEBUG:boto:Params: {} 2022-09-03T06:47:27.724 DEBUG:boto:Token: None 2022-09-03T06:47:27.725 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.725 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jL6Eag37tsk5+4x/jkZEd16sJWI= 2022-09-03T06:47:27.726 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jL6Eag37tsk5+4x/jkZEd16sJWI=', 'Content-Length': '0'} 2022-09-03T06:47:27.728 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d7938768cef9748-006312f87f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.730 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:27.731 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:27.731 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.732 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.733 DEBUG:boto:Method: HEAD 2022-09-03T06:47:27.734 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.735 DEBUG:boto:Data: 2022-09-03T06:47:27.736 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.736 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.736 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.737 DEBUG:boto:Params: {} 2022-09-03T06:47:27.738 DEBUG:boto:Token: None 2022-09-03T06:47:27.738 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.739 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jL6Eag37tsk5+4x/jkZEd16sJWI= 2022-09-03T06:47:27.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jL6Eag37tsk5+4x/jkZEd16sJWI=', 'Content-Length': '0'} 2022-09-03T06:47:27.742 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fbd5f333474f7bd7-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.744 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:47:27.745 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:27.746 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:27.747 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.747 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.748 DEBUG:boto:Method: GET 2022-09-03T06:47:27.749 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.750 DEBUG:boto:Data: 2022-09-03T06:47:27.751 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.752 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.753 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.754 DEBUG:boto:Params: {} 2022-09-03T06:47:27.755 DEBUG:boto:Token: None 2022-09-03T06:47:27.755 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.756 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TNABhsU0iXYpmmfwtQqnJ4CNd20= 2022-09-03T06:47:27.757 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TNABhsU0iXYpmmfwtQqnJ4CNd20=', 'Content-Length': '0'} 2022-09-03T06:47:27.760 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000093bc8db231056e88-006312f87f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.761 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:27.762 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:27.763 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.764 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.765 DEBUG:boto:Method: GET 2022-09-03T06:47:27.766 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.767 DEBUG:boto:Data: 2022-09-03T06:47:27.768 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.769 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.769 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.770 DEBUG:boto:Params: {} 2022-09-03T06:47:27.771 DEBUG:boto:Token: None 2022-09-03T06:47:27.772 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:27.774 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TNABhsU0iXYpmmfwtQqnJ4CNd20= 2022-09-03T06:47:27.775 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TNABhsU0iXYpmmfwtQqnJ4CNd20=', 'Content-Length': '0'} 2022-09-03T06:47:27.778 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000003115bf345123209-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.780 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:27.780 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:27.781 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:27.782 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:27.783 DEBUG:boto:Method: GET 2022-09-03T06:47:27.784 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:47:27.785 DEBUG:boto:Data: 2022-09-03T06:47:27.785 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.786 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.786 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.787 DEBUG:boto:Params: {} 2022-09-03T06:47:27.787 DEBUG:boto:Token: None 2022-09-03T06:47:27.788 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/ 2022-09-03T06:47:27.788 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw= 2022-09-03T06:47:27.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw=', 'Content-Length': '0'} 2022-09-03T06:47:27.792 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a00f016cf213a03a-006312f87f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.793 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:27.794 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:27.794 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:27.795 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:27.795 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:27.796 DEBUG:boto:Method: GET 2022-09-03T06:47:27.796 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:47:27.797 DEBUG:boto:Data: 2022-09-03T06:47:27.797 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.798 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.799 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.799 DEBUG:boto:Params: {} 2022-09-03T06:47:27.800 DEBUG:boto:Token: None 2022-09-03T06:47:27.800 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/ 2022-09-03T06:47:27.801 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw= 2022-09-03T06:47:27.802 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw=', 'Content-Length': '0'} 2022-09-03T06:47:27.806 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000019c1976802705821-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.846 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:27.847 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:27.848 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:27.849 DEBUG:boto:Method: HEAD 2022-09-03T06:47:27.849 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:47:27.850 DEBUG:boto:Data: 2022-09-03T06:47:27.850 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.851 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.852 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.853 DEBUG:boto:Params: {} 2022-09-03T06:47:27.853 DEBUG:boto:Token: None 2022-09-03T06:47:27.854 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj44 2022-09-03T06:47:27.855 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oLzkZrlj7aJlZ/0yizLUUt5qJHU= 2022-09-03T06:47:27.855 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oLzkZrlj7aJlZ/0yizLUUt5qJHU=', 'Content-Length': '0'} 2022-09-03T06:47:27.858 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b674c2c87d6fafcc-006312f87f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.859 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:27.859 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:27.860 DEBUG:boto:Method: HEAD 2022-09-03T06:47:27.860 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:47:27.861 DEBUG:boto:Data: 2022-09-03T06:47:27.861 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.862 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.862 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.863 DEBUG:boto:Params: {} 2022-09-03T06:47:27.863 DEBUG:boto:Token: None 2022-09-03T06:47:27.864 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj44 2022-09-03T06:47:27.865 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oLzkZrlj7aJlZ/0yizLUUt5qJHU= 2022-09-03T06:47:27.865 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oLzkZrlj7aJlZ/0yizLUUt5qJHU=', 'Content-Length': '0'} 2022-09-03T06:47:27.868 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000196f51d6d10f08e3-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.869 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:47:27.869 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:27.870 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:27.871 DEBUG:boto:Method: GET 2022-09-03T06:47:27.871 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:47:27.872 DEBUG:boto:Data: 2022-09-03T06:47:27.872 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.873 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.873 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.874 DEBUG:boto:Params: {} 2022-09-03T06:47:27.875 DEBUG:boto:Token: None 2022-09-03T06:47:27.875 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj44 2022-09-03T06:47:27.876 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BNqJ0C0OblJLd2Dsve3ykmmNtU0= 2022-09-03T06:47:27.877 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BNqJ0C0OblJLd2Dsve3ykmmNtU0=', 'Content-Length': '0'} 2022-09-03T06:47:27.879 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e4cfea548906b6c3-006312f87f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.880 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:27.880 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:27.881 DEBUG:boto:Method: GET 2022-09-03T06:47:27.881 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:47:27.882 DEBUG:boto:Data: 2022-09-03T06:47:27.882 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.883 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.883 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.884 DEBUG:boto:Params: {} 2022-09-03T06:47:27.885 DEBUG:boto:Token: None 2022-09-03T06:47:27.885 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj44 2022-09-03T06:47:27.886 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BNqJ0C0OblJLd2Dsve3ykmmNtU0= 2022-09-03T06:47:27.887 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BNqJ0C0OblJLd2Dsve3ykmmNtU0=', 'Content-Length': '0'} 2022-09-03T06:47:27.889 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d2f29c7120596f1-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.889 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:27.890 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:27.891 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:27.891 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:27.892 DEBUG:boto:Method: GET 2022-09-03T06:47:27.892 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:47:27.893 DEBUG:boto:Data: 2022-09-03T06:47:27.893 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.894 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.894 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.895 DEBUG:boto:Params: {} 2022-09-03T06:47:27.895 DEBUG:boto:Token: None 2022-09-03T06:47:27.896 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/ 2022-09-03T06:47:27.897 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw= 2022-09-03T06:47:27.897 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw=', 'Content-Length': '0'} 2022-09-03T06:47:27.900 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000023a22858c7859aed-006312f87f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.901 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:27.902 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:27.902 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:27.903 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:27.904 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:27.904 DEBUG:boto:Method: GET 2022-09-03T06:47:27.905 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:47:27.905 DEBUG:boto:Data: 2022-09-03T06:47:27.906 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.906 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.907 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.908 DEBUG:boto:Params: {} 2022-09-03T06:47:27.909 DEBUG:boto:Token: None 2022-09-03T06:47:27.909 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/ 2022-09-03T06:47:27.910 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw= 2022-09-03T06:47:27.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/pWESqgJxa1w3x7E3Zyrm7OADw=', 'Content-Length': '0'} 2022-09-03T06:47:27.915 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003a56310878024517-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.916 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:27.917 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:47:27.917 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:27.918 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:27.918 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.919 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.920 DEBUG:boto:Method: GET 2022-09-03T06:47:27.920 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.921 DEBUG:boto:Data: 2022-09-03T06:47:27.922 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.922 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.923 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.923 DEBUG:boto:Params: {} 2022-09-03T06:47:27.924 DEBUG:boto:Token: None 2022-09-03T06:47:27.925 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.925 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ak88msHFfJigAoULE7aUmaC9wCI= 2022-09-03T06:47:27.926 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ak88msHFfJigAoULE7aUmaC9wCI=', 'Content-Length': '0'} 2022-09-03T06:47:27.928 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003db02a937ab68bb5-006312f87f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.929 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:27.930 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:27.931 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.931 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.932 DEBUG:boto:Method: GET 2022-09-03T06:47:27.932 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.933 DEBUG:boto:Data: 2022-09-03T06:47:27.934 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.935 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.936 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.937 DEBUG:boto:Params: {} 2022-09-03T06:47:27.938 DEBUG:boto:Token: None 2022-09-03T06:47:27.939 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.940 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ak88msHFfJigAoULE7aUmaC9wCI= 2022-09-03T06:47:27.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ak88msHFfJigAoULE7aUmaC9wCI=', 'Content-Length': '0'} 2022-09-03T06:47:27.943 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001fb38058849106e5-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.944 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:27.945 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:27.945 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.946 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.947 DEBUG:boto:Method: HEAD 2022-09-03T06:47:27.947 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.948 DEBUG:boto:Data: 2022-09-03T06:47:27.949 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.951 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.952 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.952 DEBUG:boto:Params: {} 2022-09-03T06:47:27.953 DEBUG:boto:Token: None 2022-09-03T06:47:27.954 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.955 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1j4A/1IfCAYYUEUMWIdlFbNKaNk= 2022-09-03T06:47:27.955 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1j4A/1IfCAYYUEUMWIdlFbNKaNk=', 'Content-Length': '0'} 2022-09-03T06:47:27.958 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005bff15bc1fdd22f3-006312f87f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.958 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:27.959 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:27.959 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.960 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.961 DEBUG:boto:Method: HEAD 2022-09-03T06:47:27.961 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.962 DEBUG:boto:Data: 2022-09-03T06:47:27.963 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.963 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.964 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.964 DEBUG:boto:Params: {} 2022-09-03T06:47:27.966 DEBUG:boto:Token: None 2022-09-03T06:47:27.967 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.967 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1j4A/1IfCAYYUEUMWIdlFbNKaNk= 2022-09-03T06:47:27.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1j4A/1IfCAYYUEUMWIdlFbNKaNk=', 'Content-Length': '0'} 2022-09-03T06:47:27.970 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bdad86369fd9c0a6-006312f87f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.971 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:47:27.971 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:27.972 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:27.972 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.973 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.973 DEBUG:boto:Method: GET 2022-09-03T06:47:27.974 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.974 DEBUG:boto:Data: 2022-09-03T06:47:27.975 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.976 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:27.976 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.977 DEBUG:boto:Params: {} 2022-09-03T06:47:27.978 DEBUG:boto:Token: None 2022-09-03T06:47:27.979 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.980 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ak88msHFfJigAoULE7aUmaC9wCI= 2022-09-03T06:47:27.980 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ak88msHFfJigAoULE7aUmaC9wCI=', 'Content-Length': '0'} 2022-09-03T06:47:27.983 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f29147ef75d224ad-006312f87f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:27.984 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:27.986 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:27.987 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.988 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.989 DEBUG:boto:Method: GET 2022-09-03T06:47:27.990 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.991 DEBUG:boto:Data: 2022-09-03T06:47:27.991 DEBUG:boto:Headers: {} 2022-09-03T06:47:27.992 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:27.993 DEBUG:boto:Port: 8000 2022-09-03T06:47:27.994 DEBUG:boto:Params: {} 2022-09-03T06:47:27.995 DEBUG:boto:Token: None 2022-09-03T06:47:27.996 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:27 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:27.997 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ak88msHFfJigAoULE7aUmaC9wCI= 2022-09-03T06:47:27.999 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ak88msHFfJigAoULE7aUmaC9wCI=', 'Content-Length': '0'} 2022-09-03T06:47:28.002 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000053551eab254dba80-006312f880-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.042 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:28.043 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:28.043 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:28.044 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:28.044 DEBUG:boto:Method: GET 2022-09-03T06:47:28.045 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:47:28.046 DEBUG:boto:Data: 2022-09-03T06:47:28.046 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.047 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.047 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.048 DEBUG:boto:Params: {} 2022-09-03T06:47:28.048 DEBUG:boto:Token: None 2022-09-03T06:47:28.049 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/ 2022-09-03T06:47:28.050 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4= 2022-09-03T06:47:28.050 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4=', 'Content-Length': '0'} 2022-09-03T06:47:28.053 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000086d409fffa6b39d7-006312f880-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.054 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:28.056 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:28.056 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:28.057 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:28.057 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:28.058 DEBUG:boto:Method: GET 2022-09-03T06:47:28.059 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:47:28.059 DEBUG:boto:Data: 2022-09-03T06:47:28.060 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.060 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.061 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.061 DEBUG:boto:Params: {} 2022-09-03T06:47:28.062 DEBUG:boto:Token: None 2022-09-03T06:47:28.063 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/ 2022-09-03T06:47:28.063 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4= 2022-09-03T06:47:28.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4=', 'Content-Length': '0'} 2022-09-03T06:47:28.068 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007266367adf6e6e24-006312f880-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.069 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:28.070 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:28.071 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:28.071 DEBUG:boto:Method: HEAD 2022-09-03T06:47:28.072 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:47:28.072 DEBUG:boto:Data: 2022-09-03T06:47:28.073 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.073 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.074 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.074 DEBUG:boto:Params: {} 2022-09-03T06:47:28.075 DEBUG:boto:Token: None 2022-09-03T06:47:28.075 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj46 2022-09-03T06:47:28.076 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D6ymE1sVcUt3umUnQp08N9LtTWE= 2022-09-03T06:47:28.077 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D6ymE1sVcUt3umUnQp08N9LtTWE=', 'Content-Length': '0'} 2022-09-03T06:47:28.079 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a15ac038149aeae2-006312f880-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.080 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:28.080 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:28.081 DEBUG:boto:Method: HEAD 2022-09-03T06:47:28.081 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:47:28.082 DEBUG:boto:Data: 2022-09-03T06:47:28.082 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.083 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.083 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.084 DEBUG:boto:Params: {} 2022-09-03T06:47:28.084 DEBUG:boto:Token: None 2022-09-03T06:47:28.085 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj46 2022-09-03T06:47:28.086 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D6ymE1sVcUt3umUnQp08N9LtTWE= 2022-09-03T06:47:28.087 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D6ymE1sVcUt3umUnQp08N9LtTWE=', 'Content-Length': '0'} 2022-09-03T06:47:28.090 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000032e8a036c13b2c6a-006312f880-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.091 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:47:28.092 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:28.093 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:28.094 DEBUG:boto:Method: GET 2022-09-03T06:47:28.095 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:47:28.096 DEBUG:boto:Data: 2022-09-03T06:47:28.096 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.097 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.098 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.099 DEBUG:boto:Params: {} 2022-09-03T06:47:28.100 DEBUG:boto:Token: None 2022-09-03T06:47:28.101 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj46 2022-09-03T06:47:28.102 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wGKyO4VKjJBj5uBdebRiO65WZWg= 2022-09-03T06:47:28.103 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wGKyO4VKjJBj5uBdebRiO65WZWg=', 'Content-Length': '0'} 2022-09-03T06:47:28.105 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001e59647e844428d8-006312f880-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.150 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:28.151 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:28.152 DEBUG:boto:Method: GET 2022-09-03T06:47:28.152 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:47:28.153 DEBUG:boto:Data: 2022-09-03T06:47:28.154 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.155 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.155 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.156 DEBUG:boto:Params: {} 2022-09-03T06:47:28.157 DEBUG:boto:Token: None 2022-09-03T06:47:28.157 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj46 2022-09-03T06:47:28.158 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wGKyO4VKjJBj5uBdebRiO65WZWg= 2022-09-03T06:47:28.159 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wGKyO4VKjJBj5uBdebRiO65WZWg=', 'Content-Length': '0'} 2022-09-03T06:47:28.161 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000271afdec9dbe1745-006312f880-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.162 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:28.163 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:28.164 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:28.165 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:28.166 DEBUG:boto:Method: GET 2022-09-03T06:47:28.166 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:47:28.167 DEBUG:boto:Data: 2022-09-03T06:47:28.167 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.168 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.168 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.169 DEBUG:boto:Params: {} 2022-09-03T06:47:28.170 DEBUG:boto:Token: None 2022-09-03T06:47:28.171 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/ 2022-09-03T06:47:28.171 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4= 2022-09-03T06:47:28.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4=', 'Content-Length': '0'} 2022-09-03T06:47:28.175 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000dbd0525b245fd14-006312f880-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.175 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:28.177 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:28.179 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:28.180 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:28.181 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:28.182 DEBUG:boto:Method: GET 2022-09-03T06:47:28.183 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:47:28.184 DEBUG:boto:Data: 2022-09-03T06:47:28.185 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.186 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.188 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.189 DEBUG:boto:Params: {} 2022-09-03T06:47:28.190 DEBUG:boto:Token: None 2022-09-03T06:47:28.191 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/ 2022-09-03T06:47:28.192 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4= 2022-09-03T06:47:28.193 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4=', 'Content-Length': '0'} 2022-09-03T06:47:28.198 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000067c3021dc87567b3-006312f880-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.199 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:28.200 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:47:28.200 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:28.201 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:28.201 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.203 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.204 DEBUG:boto:Method: GET 2022-09-03T06:47:28.205 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.206 DEBUG:boto:Data: 2022-09-03T06:47:28.206 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.207 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.208 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.209 DEBUG:boto:Params: {} 2022-09-03T06:47:28.210 DEBUG:boto:Token: None 2022-09-03T06:47:28.211 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.212 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TO6lAlQXjCmDP94ftiqdV+qNAjA= 2022-09-03T06:47:28.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TO6lAlQXjCmDP94ftiqdV+qNAjA=', 'Content-Length': '0'} 2022-09-03T06:47:28.215 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f0911e504c79bdaa-006312f880-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.216 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:28.217 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:28.217 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.218 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.219 DEBUG:boto:Method: GET 2022-09-03T06:47:28.220 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.221 DEBUG:boto:Data: 2022-09-03T06:47:28.222 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.223 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.224 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.226 DEBUG:boto:Params: {} 2022-09-03T06:47:28.227 DEBUG:boto:Token: None 2022-09-03T06:47:28.228 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.229 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TO6lAlQXjCmDP94ftiqdV+qNAjA= 2022-09-03T06:47:28.230 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TO6lAlQXjCmDP94ftiqdV+qNAjA=', 'Content-Length': '0'} 2022-09-03T06:47:28.232 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000990cb942c07d3ee6-006312f880-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.233 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:28.234 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:28.235 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.236 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.237 DEBUG:boto:Method: HEAD 2022-09-03T06:47:28.238 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.239 DEBUG:boto:Data: 2022-09-03T06:47:28.240 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.240 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.241 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.243 DEBUG:boto:Params: {} 2022-09-03T06:47:28.244 DEBUG:boto:Token: None 2022-09-03T06:47:28.245 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.246 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iiuARHDaghnBtD5PVT7mKAlaCP8= 2022-09-03T06:47:28.247 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iiuARHDaghnBtD5PVT7mKAlaCP8=', 'Content-Length': '0'} 2022-09-03T06:47:28.249 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000568dcb57140bab86-006312f880-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.250 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:28.251 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:28.252 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.252 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.253 DEBUG:boto:Method: HEAD 2022-09-03T06:47:28.253 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.254 DEBUG:boto:Data: 2022-09-03T06:47:28.254 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.255 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.256 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.256 DEBUG:boto:Params: {} 2022-09-03T06:47:28.257 DEBUG:boto:Token: None 2022-09-03T06:47:28.257 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.258 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iiuARHDaghnBtD5PVT7mKAlaCP8= 2022-09-03T06:47:28.259 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iiuARHDaghnBtD5PVT7mKAlaCP8=', 'Content-Length': '0'} 2022-09-03T06:47:28.261 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000733fca64f503eea7-006312f880-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.262 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:47:28.262 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:28.263 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:28.263 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.264 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.264 DEBUG:boto:Method: GET 2022-09-03T06:47:28.265 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.266 DEBUG:boto:Data: 2022-09-03T06:47:28.266 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.267 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.267 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.268 DEBUG:boto:Params: {} 2022-09-03T06:47:28.268 DEBUG:boto:Token: None 2022-09-03T06:47:28.269 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.270 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TO6lAlQXjCmDP94ftiqdV+qNAjA= 2022-09-03T06:47:28.270 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TO6lAlQXjCmDP94ftiqdV+qNAjA=', 'Content-Length': '0'} 2022-09-03T06:47:28.273 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000219ce8448236d713-006312f880-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.314 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:28.315 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:28.315 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.316 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.316 DEBUG:boto:Method: GET 2022-09-03T06:47:28.317 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.318 DEBUG:boto:Data: 2022-09-03T06:47:28.318 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.319 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.319 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.320 DEBUG:boto:Params: {} 2022-09-03T06:47:28.321 DEBUG:boto:Token: None 2022-09-03T06:47:28.321 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:28.322 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TO6lAlQXjCmDP94ftiqdV+qNAjA= 2022-09-03T06:47:28.323 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TO6lAlQXjCmDP94ftiqdV+qNAjA=', 'Content-Length': '0'} 2022-09-03T06:47:28.325 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c02ce9308865477-006312f880-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.326 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:28.326 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:28.327 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:28.328 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:28.330 DEBUG:boto:Method: GET 2022-09-03T06:47:28.331 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:47:28.332 DEBUG:boto:Data: 2022-09-03T06:47:28.333 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.334 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.335 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.336 DEBUG:boto:Params: {} 2022-09-03T06:47:28.337 DEBUG:boto:Token: None 2022-09-03T06:47:28.339 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/ 2022-09-03T06:47:28.340 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4= 2022-09-03T06:47:28.341 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4=', 'Content-Length': '0'} 2022-09-03T06:47:28.344 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000019ac3f65903b9fd2-006312f880-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.345 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:28.346 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:28.346 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:28.347 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:28.347 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:28.348 DEBUG:boto:Method: GET 2022-09-03T06:47:28.349 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:47:28.349 DEBUG:boto:Data: 2022-09-03T06:47:28.350 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.350 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.351 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.351 DEBUG:boto:Params: {} 2022-09-03T06:47:28.353 DEBUG:boto:Token: None 2022-09-03T06:47:28.353 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/ 2022-09-03T06:47:28.354 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4= 2022-09-03T06:47:28.355 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4=', 'Content-Length': '0'} 2022-09-03T06:47:28.359 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003f6c59c4474244a7-006312f880-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.360 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:28.361 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:28.362 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:28.363 DEBUG:boto:Method: HEAD 2022-09-03T06:47:28.363 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:47:28.364 DEBUG:boto:Data: 2022-09-03T06:47:28.364 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.365 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.365 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.366 DEBUG:boto:Params: {} 2022-09-03T06:47:28.367 DEBUG:boto:Token: None 2022-09-03T06:47:28.367 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj51 2022-09-03T06:47:28.368 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BvWl1mYZi+N6KahbZh0gS0/q9Ms= 2022-09-03T06:47:28.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BvWl1mYZi+N6KahbZh0gS0/q9Ms=', 'Content-Length': '0'} 2022-09-03T06:47:28.371 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009936061ceec10414-006312f880-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.372 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:28.372 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:28.373 DEBUG:boto:Method: HEAD 2022-09-03T06:47:28.374 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:47:28.374 DEBUG:boto:Data: 2022-09-03T06:47:28.375 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.375 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.376 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.376 DEBUG:boto:Params: {} 2022-09-03T06:47:28.377 DEBUG:boto:Token: None 2022-09-03T06:47:28.377 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj51 2022-09-03T06:47:28.378 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BvWl1mYZi+N6KahbZh0gS0/q9Ms= 2022-09-03T06:47:28.379 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BvWl1mYZi+N6KahbZh0gS0/q9Ms=', 'Content-Length': '0'} 2022-09-03T06:47:28.381 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d4b70cc82877747d-006312f880-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.382 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:47:28.382 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:28.383 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:28.383 DEBUG:boto:Method: GET 2022-09-03T06:47:28.384 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:47:28.384 DEBUG:boto:Data: 2022-09-03T06:47:28.385 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.386 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.386 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.387 DEBUG:boto:Params: {} 2022-09-03T06:47:28.388 DEBUG:boto:Token: None 2022-09-03T06:47:28.388 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj51 2022-09-03T06:47:28.389 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GMVEZD9ruKn0LRzhc9I9PSZWx04= 2022-09-03T06:47:28.390 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GMVEZD9ruKn0LRzhc9I9PSZWx04=', 'Content-Length': '0'} 2022-09-03T06:47:28.392 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ea8e9e6cc8532df-006312f880-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.393 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:28.394 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:28.394 DEBUG:boto:Method: GET 2022-09-03T06:47:28.395 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:47:28.396 DEBUG:boto:Data: 2022-09-03T06:47:28.396 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.397 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.398 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.398 DEBUG:boto:Params: {} 2022-09-03T06:47:28.399 DEBUG:boto:Token: None 2022-09-03T06:47:28.400 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj51 2022-09-03T06:47:28.400 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GMVEZD9ruKn0LRzhc9I9PSZWx04= 2022-09-03T06:47:28.401 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GMVEZD9ruKn0LRzhc9I9PSZWx04=', 'Content-Length': '0'} 2022-09-03T06:47:28.403 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000047ca507747a977a1-006312f880-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.446 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:28.447 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:28.448 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:28.448 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:28.449 DEBUG:boto:Method: GET 2022-09-03T06:47:28.450 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:47:28.450 DEBUG:boto:Data: 2022-09-03T06:47:28.451 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.451 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.452 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.453 DEBUG:boto:Params: {} 2022-09-03T06:47:28.453 DEBUG:boto:Token: None 2022-09-03T06:47:28.454 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/ 2022-09-03T06:47:28.455 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4= 2022-09-03T06:47:28.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4=', 'Content-Length': '0'} 2022-09-03T06:47:28.459 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002d2fac0a74d1cc2e-006312f880-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.461 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:28.462 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:28.463 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:28.464 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:28.466 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:28.467 DEBUG:boto:Method: GET 2022-09-03T06:47:28.467 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:47:28.468 DEBUG:boto:Data: 2022-09-03T06:47:28.468 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.469 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.469 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.470 DEBUG:boto:Params: {} 2022-09-03T06:47:28.471 DEBUG:boto:Token: None 2022-09-03T06:47:28.471 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/ 2022-09-03T06:47:28.472 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4= 2022-09-03T06:47:28.473 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4=', 'Content-Length': '0'} 2022-09-03T06:47:28.477 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fb857cc60373375d-006312f880-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.478 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:28.479 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:47:28.479 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:28.480 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:28.480 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.481 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.482 DEBUG:boto:Method: GET 2022-09-03T06:47:28.482 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.483 DEBUG:boto:Data: 2022-09-03T06:47:28.484 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.484 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.485 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.485 DEBUG:boto:Params: {} 2022-09-03T06:47:28.486 DEBUG:boto:Token: None 2022-09-03T06:47:28.486 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.487 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EQMagnIW8DJ7xemaGaze/0J1np4= 2022-09-03T06:47:28.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EQMagnIW8DJ7xemaGaze/0J1np4=', 'Content-Length': '0'} 2022-09-03T06:47:28.491 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000031d444d18693cf77-006312f880-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.534 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:28.535 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:28.535 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.536 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.536 DEBUG:boto:Method: GET 2022-09-03T06:47:28.537 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.538 DEBUG:boto:Data: 2022-09-03T06:47:28.539 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.539 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.540 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.540 DEBUG:boto:Params: {} 2022-09-03T06:47:28.541 DEBUG:boto:Token: None 2022-09-03T06:47:28.542 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.542 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EQMagnIW8DJ7xemaGaze/0J1np4= 2022-09-03T06:47:28.543 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EQMagnIW8DJ7xemaGaze/0J1np4=', 'Content-Length': '0'} 2022-09-03T06:47:28.546 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c4571dd0d573a9a6-006312f880-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.547 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:28.548 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:28.549 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.550 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.551 DEBUG:boto:Method: HEAD 2022-09-03T06:47:28.552 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.553 DEBUG:boto:Data: 2022-09-03T06:47:28.554 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.556 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.556 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.557 DEBUG:boto:Params: {} 2022-09-03T06:47:28.558 DEBUG:boto:Token: None 2022-09-03T06:47:28.559 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.561 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JlNA3+zAp6U3Y7d1d1MnU/RkukI= 2022-09-03T06:47:28.561 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JlNA3+zAp6U3Y7d1d1MnU/RkukI=', 'Content-Length': '0'} 2022-09-03T06:47:28.564 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003d0679f0744612e9-006312f880-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.564 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:28.565 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:28.566 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.566 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.567 DEBUG:boto:Method: HEAD 2022-09-03T06:47:28.568 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.568 DEBUG:boto:Data: 2022-09-03T06:47:28.569 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.569 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.570 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.571 DEBUG:boto:Params: {} 2022-09-03T06:47:28.572 DEBUG:boto:Token: None 2022-09-03T06:47:28.572 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.573 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JlNA3+zAp6U3Y7d1d1MnU/RkukI= 2022-09-03T06:47:28.574 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JlNA3+zAp6U3Y7d1d1MnU/RkukI=', 'Content-Length': '0'} 2022-09-03T06:47:28.576 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000071f051e36960d360-006312f880-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.577 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:47:28.578 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:28.579 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:28.580 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.581 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.582 DEBUG:boto:Method: GET 2022-09-03T06:47:28.583 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.584 DEBUG:boto:Data: 2022-09-03T06:47:28.585 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.586 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.587 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.588 DEBUG:boto:Params: {} 2022-09-03T06:47:28.589 DEBUG:boto:Token: None 2022-09-03T06:47:28.590 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.592 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EQMagnIW8DJ7xemaGaze/0J1np4= 2022-09-03T06:47:28.593 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EQMagnIW8DJ7xemaGaze/0J1np4=', 'Content-Length': '0'} 2022-09-03T06:47:28.596 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b3074d587cd6087-006312f880-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.597 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:28.598 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:28.598 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.599 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.600 DEBUG:boto:Method: GET 2022-09-03T06:47:28.600 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.601 DEBUG:boto:Data: 2022-09-03T06:47:28.601 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.602 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.602 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.603 DEBUG:boto:Params: {} 2022-09-03T06:47:28.604 DEBUG:boto:Token: None 2022-09-03T06:47:28.607 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:28.609 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EQMagnIW8DJ7xemaGaze/0J1np4= 2022-09-03T06:47:28.610 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EQMagnIW8DJ7xemaGaze/0J1np4=', 'Content-Length': '0'} 2022-09-03T06:47:28.612 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bb530beda51f0a22-006312f880-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.613 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:28.614 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:28.615 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:28.615 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:28.616 DEBUG:boto:Method: GET 2022-09-03T06:47:28.616 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:47:28.617 DEBUG:boto:Data: 2022-09-03T06:47:28.617 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.618 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.618 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.619 DEBUG:boto:Params: {} 2022-09-03T06:47:28.620 DEBUG:boto:Token: None 2022-09-03T06:47:28.620 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/ 2022-09-03T06:47:28.621 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4= 2022-09-03T06:47:28.621 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4=', 'Content-Length': '0'} 2022-09-03T06:47:28.625 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000695624ef87cef005-006312f880-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.626 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:28.628 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:28.628 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:28.629 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:28.630 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:28.630 DEBUG:boto:Method: GET 2022-09-03T06:47:28.631 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:47:28.631 DEBUG:boto:Data: 2022-09-03T06:47:28.632 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.632 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.633 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.633 DEBUG:boto:Params: {} 2022-09-03T06:47:28.634 DEBUG:boto:Token: None 2022-09-03T06:47:28.635 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/ 2022-09-03T06:47:28.635 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4= 2022-09-03T06:47:28.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4=', 'Content-Length': '0'} 2022-09-03T06:47:28.640 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ab89985ffec1e8e4-006312f880-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.641 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:28.643 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:28.644 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:28.644 DEBUG:boto:Method: HEAD 2022-09-03T06:47:28.645 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:47:28.646 DEBUG:boto:Data: 2022-09-03T06:47:28.647 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.648 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.649 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.650 DEBUG:boto:Params: {} 2022-09-03T06:47:28.651 DEBUG:boto:Token: None 2022-09-03T06:47:28.652 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj55 2022-09-03T06:47:28.653 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5BgbhpMrCXvHQtGOEXAc6p/kugg= 2022-09-03T06:47:28.654 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5BgbhpMrCXvHQtGOEXAc6p/kugg=', 'Content-Length': '0'} 2022-09-03T06:47:28.657 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000841ee2a7feebcfa0-006312f880-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.658 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:28.659 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:28.660 DEBUG:boto:Method: HEAD 2022-09-03T06:47:28.661 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:47:28.662 DEBUG:boto:Data: 2022-09-03T06:47:28.663 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.664 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.664 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.665 DEBUG:boto:Params: {} 2022-09-03T06:47:28.666 DEBUG:boto:Token: None 2022-09-03T06:47:28.667 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj55 2022-09-03T06:47:28.668 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5BgbhpMrCXvHQtGOEXAc6p/kugg= 2022-09-03T06:47:28.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5BgbhpMrCXvHQtGOEXAc6p/kugg=', 'Content-Length': '0'} 2022-09-03T06:47:28.671 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009ff36837cab337cc-006312f880-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.673 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:47:28.674 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:28.675 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:28.675 DEBUG:boto:Method: GET 2022-09-03T06:47:28.676 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:47:28.677 DEBUG:boto:Data: 2022-09-03T06:47:28.678 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.679 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.680 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.681 DEBUG:boto:Params: {} 2022-09-03T06:47:28.682 DEBUG:boto:Token: None 2022-09-03T06:47:28.683 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj55 2022-09-03T06:47:28.684 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z0LFqFXrVG4GvvFD+YJ9LPtv/3s= 2022-09-03T06:47:28.686 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z0LFqFXrVG4GvvFD+YJ9LPtv/3s=', 'Content-Length': '0'} 2022-09-03T06:47:28.688 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d819dfe7b51990a-006312f880-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.689 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:28.690 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:28.691 DEBUG:boto:Method: GET 2022-09-03T06:47:28.692 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:47:28.693 DEBUG:boto:Data: 2022-09-03T06:47:28.694 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.695 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.696 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.697 DEBUG:boto:Params: {} 2022-09-03T06:47:28.709 DEBUG:boto:Token: None 2022-09-03T06:47:28.710 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj55 2022-09-03T06:47:28.711 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z0LFqFXrVG4GvvFD+YJ9LPtv/3s= 2022-09-03T06:47:28.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z0LFqFXrVG4GvvFD+YJ9LPtv/3s=', 'Content-Length': '0'} 2022-09-03T06:47:28.715 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000088ef4d2209a8521-006312f880-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.716 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:28.717 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:28.717 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:28.718 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:28.718 DEBUG:boto:Method: GET 2022-09-03T06:47:28.719 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:47:28.720 DEBUG:boto:Data: 2022-09-03T06:47:28.721 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.721 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.722 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.722 DEBUG:boto:Params: {} 2022-09-03T06:47:28.723 DEBUG:boto:Token: None 2022-09-03T06:47:28.723 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/ 2022-09-03T06:47:28.724 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4= 2022-09-03T06:47:28.725 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4=', 'Content-Length': '0'} 2022-09-03T06:47:28.728 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d7a8d5d6bc3e42d9-006312f880-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.729 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:28.730 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:28.731 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:28.732 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:28.733 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:28.734 DEBUG:boto:Method: GET 2022-09-03T06:47:28.735 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:47:28.736 DEBUG:boto:Data: 2022-09-03T06:47:28.737 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.738 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.738 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.739 DEBUG:boto:Params: {} 2022-09-03T06:47:28.739 DEBUG:boto:Token: None 2022-09-03T06:47:28.740 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/ 2022-09-03T06:47:28.741 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4= 2022-09-03T06:47:28.742 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4=', 'Content-Length': '0'} 2022-09-03T06:47:28.746 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003504414380a121b0-006312f880-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.795 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:28.796 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:47:28.797 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:28.798 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:28.799 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.799 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.800 DEBUG:boto:Method: GET 2022-09-03T06:47:28.801 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.801 DEBUG:boto:Data: 2022-09-03T06:47:28.802 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.803 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.803 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.804 DEBUG:boto:Params: {} 2022-09-03T06:47:28.804 DEBUG:boto:Token: None 2022-09-03T06:47:28.806 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.807 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JBELdGmo8OOGD3wKizvpqgZaEOw= 2022-09-03T06:47:28.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JBELdGmo8OOGD3wKizvpqgZaEOw=', 'Content-Length': '0'} 2022-09-03T06:47:28.810 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f9d6c922a106df86-006312f880-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.811 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:28.812 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:28.812 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.813 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.814 DEBUG:boto:Method: GET 2022-09-03T06:47:28.814 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.816 DEBUG:boto:Data: 2022-09-03T06:47:28.817 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.818 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.819 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.819 DEBUG:boto:Params: {} 2022-09-03T06:47:28.820 DEBUG:boto:Token: None 2022-09-03T06:47:28.820 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.821 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JBELdGmo8OOGD3wKizvpqgZaEOw= 2022-09-03T06:47:28.822 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JBELdGmo8OOGD3wKizvpqgZaEOw=', 'Content-Length': '0'} 2022-09-03T06:47:28.824 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2a51f2ca2b18ccb-006312f880-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.825 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:28.825 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:28.826 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.827 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.827 DEBUG:boto:Method: HEAD 2022-09-03T06:47:28.828 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.828 DEBUG:boto:Data: 2022-09-03T06:47:28.829 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.829 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.829 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.830 DEBUG:boto:Params: {} 2022-09-03T06:47:28.831 DEBUG:boto:Token: None 2022-09-03T06:47:28.831 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.832 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wn79lu9KmKm6aSGJM3wfiFEV7bo= 2022-09-03T06:47:28.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wn79lu9KmKm6aSGJM3wfiFEV7bo=', 'Content-Length': '0'} 2022-09-03T06:47:28.834 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d58165baee7fac1-006312f880-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.835 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:28.836 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:28.836 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.837 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.837 DEBUG:boto:Method: HEAD 2022-09-03T06:47:28.838 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.838 DEBUG:boto:Data: 2022-09-03T06:47:28.839 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.839 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.840 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.840 DEBUG:boto:Params: {} 2022-09-03T06:47:28.841 DEBUG:boto:Token: None 2022-09-03T06:47:28.841 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.842 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Wn79lu9KmKm6aSGJM3wfiFEV7bo= 2022-09-03T06:47:28.843 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Wn79lu9KmKm6aSGJM3wfiFEV7bo=', 'Content-Length': '0'} 2022-09-03T06:47:28.845 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000052ab9080b08ae209-006312f880-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.846 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:47:28.846 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:28.847 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:28.847 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.848 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.848 DEBUG:boto:Method: GET 2022-09-03T06:47:28.849 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.849 DEBUG:boto:Data: 2022-09-03T06:47:28.850 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.850 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.851 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.851 DEBUG:boto:Params: {} 2022-09-03T06:47:28.852 DEBUG:boto:Token: None 2022-09-03T06:47:28.853 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.853 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JBELdGmo8OOGD3wKizvpqgZaEOw= 2022-09-03T06:47:28.854 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JBELdGmo8OOGD3wKizvpqgZaEOw=', 'Content-Length': '0'} 2022-09-03T06:47:28.856 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000529a8698738c4ce8-006312f880-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.898 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:28.899 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:28.899 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.900 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.901 DEBUG:boto:Method: GET 2022-09-03T06:47:28.901 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.902 DEBUG:boto:Data: 2022-09-03T06:47:28.903 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.904 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.905 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.905 DEBUG:boto:Params: {} 2022-09-03T06:47:28.906 DEBUG:boto:Token: None 2022-09-03T06:47:28.907 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:28.907 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JBELdGmo8OOGD3wKizvpqgZaEOw= 2022-09-03T06:47:28.908 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JBELdGmo8OOGD3wKizvpqgZaEOw=', 'Content-Length': '0'} 2022-09-03T06:47:28.911 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e23f5369ce00d333-006312f880-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.912 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:28.912 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:28.913 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:28.914 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:28.914 DEBUG:boto:Method: GET 2022-09-03T06:47:28.915 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:47:28.915 DEBUG:boto:Data: 2022-09-03T06:47:28.916 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.916 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.917 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.917 DEBUG:boto:Params: {} 2022-09-03T06:47:28.918 DEBUG:boto:Token: None 2022-09-03T06:47:28.919 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/ 2022-09-03T06:47:28.921 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4= 2022-09-03T06:47:28.922 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4=', 'Content-Length': '0'} 2022-09-03T06:47:28.925 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fbcf3e72d6e0baab-006312f880-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.926 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:28.927 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:28.928 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:28.929 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:28.930 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:28.931 DEBUG:boto:Method: GET 2022-09-03T06:47:28.932 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:47:28.933 DEBUG:boto:Data: 2022-09-03T06:47:28.934 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.934 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.935 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.936 DEBUG:boto:Params: {} 2022-09-03T06:47:28.938 DEBUG:boto:Token: None 2022-09-03T06:47:28.939 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/ 2022-09-03T06:47:28.940 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4= 2022-09-03T06:47:28.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/yEFJL07kyuJBB6Db4BTtAg3tX4=', 'Content-Length': '0'} 2022-09-03T06:47:28.946 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002d1fd2c5540eaf2c-006312f880-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.947 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:28.948 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:28.949 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:28.949 DEBUG:boto:Method: HEAD 2022-09-03T06:47:28.950 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:47:28.950 DEBUG:boto:Data: 2022-09-03T06:47:28.951 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.951 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.952 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.952 DEBUG:boto:Params: {} 2022-09-03T06:47:28.953 DEBUG:boto:Token: None 2022-09-03T06:47:28.953 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj57 2022-09-03T06:47:28.954 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Abs1DJIPwTeuXyIwpc8JN6lRD6A= 2022-09-03T06:47:28.955 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Abs1DJIPwTeuXyIwpc8JN6lRD6A=', 'Content-Length': '0'} 2022-09-03T06:47:28.957 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003d42c74b614a0075-006312f880-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.958 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:28.958 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:28.959 DEBUG:boto:Method: HEAD 2022-09-03T06:47:28.959 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:47:28.960 DEBUG:boto:Data: 2022-09-03T06:47:28.960 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.961 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.961 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.962 DEBUG:boto:Params: {} 2022-09-03T06:47:28.962 DEBUG:boto:Token: None 2022-09-03T06:47:28.963 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj57 2022-09-03T06:47:28.964 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Abs1DJIPwTeuXyIwpc8JN6lRD6A= 2022-09-03T06:47:28.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Abs1DJIPwTeuXyIwpc8JN6lRD6A=', 'Content-Length': '0'} 2022-09-03T06:47:28.966 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000436f901f1500d1ba-006312f880-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.967 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:47:28.968 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:28.968 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:28.969 DEBUG:boto:Method: GET 2022-09-03T06:47:28.969 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:47:28.970 DEBUG:boto:Data: 2022-09-03T06:47:28.971 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.971 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:28.972 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.972 DEBUG:boto:Params: {} 2022-09-03T06:47:28.973 DEBUG:boto:Token: None 2022-09-03T06:47:28.974 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj57 2022-09-03T06:47:28.975 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8/y8iYNzbYd2/sCxZnQvLx3aHig= 2022-09-03T06:47:28.975 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8/y8iYNzbYd2/sCxZnQvLx3aHig=', 'Content-Length': '0'} 2022-09-03T06:47:28.977 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005725849ee3b712f2-006312f880-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:28.978 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:28.979 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:28.980 DEBUG:boto:Method: GET 2022-09-03T06:47:28.980 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:47:28.981 DEBUG:boto:Data: 2022-09-03T06:47:28.981 DEBUG:boto:Headers: {} 2022-09-03T06:47:28.982 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:28.982 DEBUG:boto:Port: 8000 2022-09-03T06:47:28.983 DEBUG:boto:Params: {} 2022-09-03T06:47:28.983 DEBUG:boto:Token: None 2022-09-03T06:47:28.984 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:28 GMT /dsgvcy-75/obj57 2022-09-03T06:47:28.985 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8/y8iYNzbYd2/sCxZnQvLx3aHig= 2022-09-03T06:47:28.985 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8/y8iYNzbYd2/sCxZnQvLx3aHig=', 'Content-Length': '0'} 2022-09-03T06:47:28.987 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000f8fc33d1076f83e-006312f880-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.035 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:29.037 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:29.038 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:29.039 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:29.041 DEBUG:boto:Method: GET 2022-09-03T06:47:29.041 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:47:29.042 DEBUG:boto:Data: 2022-09-03T06:47:29.043 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.043 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.044 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.044 DEBUG:boto:Params: {} 2022-09-03T06:47:29.045 DEBUG:boto:Token: None 2022-09-03T06:47:29.046 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/ 2022-09-03T06:47:29.047 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk= 2022-09-03T06:47:29.047 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk=', 'Content-Length': '0'} 2022-09-03T06:47:29.050 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c43ff31973e5aba9-006312f881-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.052 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:29.054 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:29.055 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:29.056 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:29.056 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:29.057 DEBUG:boto:Method: GET 2022-09-03T06:47:29.058 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:47:29.059 DEBUG:boto:Data: 2022-09-03T06:47:29.060 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.061 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.062 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.063 DEBUG:boto:Params: {} 2022-09-03T06:47:29.064 DEBUG:boto:Token: None 2022-09-03T06:47:29.065 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/ 2022-09-03T06:47:29.066 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk= 2022-09-03T06:47:29.067 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk=', 'Content-Length': '0'} 2022-09-03T06:47:29.072 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004ecebc26920b6e0f-006312f881-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.073 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:29.074 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:47:29.075 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:29.076 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:29.076 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.077 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.077 DEBUG:boto:Method: GET 2022-09-03T06:47:29.078 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.078 DEBUG:boto:Data: 2022-09-03T06:47:29.079 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.079 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.080 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.080 DEBUG:boto:Params: {} 2022-09-03T06:47:29.081 DEBUG:boto:Token: None 2022-09-03T06:47:29.082 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.082 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fm/X/TLRVStrKPEdi0XhFpVTfwE= 2022-09-03T06:47:29.083 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fm/X/TLRVStrKPEdi0XhFpVTfwE=', 'Content-Length': '0'} 2022-09-03T06:47:29.085 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000095e4eed77a6a469-006312f881-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.086 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:29.087 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:29.087 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.088 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.088 DEBUG:boto:Method: GET 2022-09-03T06:47:29.089 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.090 DEBUG:boto:Data: 2022-09-03T06:47:29.090 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.091 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.091 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.092 DEBUG:boto:Params: {} 2022-09-03T06:47:29.093 DEBUG:boto:Token: None 2022-09-03T06:47:29.093 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.094 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fm/X/TLRVStrKPEdi0XhFpVTfwE= 2022-09-03T06:47:29.095 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fm/X/TLRVStrKPEdi0XhFpVTfwE=', 'Content-Length': '0'} 2022-09-03T06:47:29.097 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d5be6184b4f2ab2e-006312f881-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.098 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:29.098 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:29.099 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.099 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.100 DEBUG:boto:Method: HEAD 2022-09-03T06:47:29.101 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.101 DEBUG:boto:Data: 2022-09-03T06:47:29.102 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.103 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.103 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.104 DEBUG:boto:Params: {} 2022-09-03T06:47:29.104 DEBUG:boto:Token: None 2022-09-03T06:47:29.105 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.105 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oyuULJrKCK6RjVCBfgmpdv1y0Qg= 2022-09-03T06:47:29.106 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oyuULJrKCK6RjVCBfgmpdv1y0Qg=', 'Content-Length': '0'} 2022-09-03T06:47:29.108 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006691a21cac0f2330-006312f881-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.109 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:29.110 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:29.110 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.111 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.111 DEBUG:boto:Method: HEAD 2022-09-03T06:47:29.112 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.112 DEBUG:boto:Data: 2022-09-03T06:47:29.113 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.113 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.114 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.114 DEBUG:boto:Params: {} 2022-09-03T06:47:29.115 DEBUG:boto:Token: None 2022-09-03T06:47:29.115 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.116 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oyuULJrKCK6RjVCBfgmpdv1y0Qg= 2022-09-03T06:47:29.117 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oyuULJrKCK6RjVCBfgmpdv1y0Qg=', 'Content-Length': '0'} 2022-09-03T06:47:29.119 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000ecceafca37b7351-006312f881-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.120 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:47:29.120 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:29.121 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:29.122 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.122 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.123 DEBUG:boto:Method: GET 2022-09-03T06:47:29.123 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.124 DEBUG:boto:Data: 2022-09-03T06:47:29.124 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.125 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.126 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.126 DEBUG:boto:Params: {} 2022-09-03T06:47:29.127 DEBUG:boto:Token: None 2022-09-03T06:47:29.127 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.129 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fm/X/TLRVStrKPEdi0XhFpVTfwE= 2022-09-03T06:47:29.129 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fm/X/TLRVStrKPEdi0XhFpVTfwE=', 'Content-Length': '0'} 2022-09-03T06:47:29.132 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000b69d4b9f472f91f-006312f881-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.132 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:29.133 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:29.134 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.134 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.135 DEBUG:boto:Method: GET 2022-09-03T06:47:29.135 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.136 DEBUG:boto:Data: 2022-09-03T06:47:29.137 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.137 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.138 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.139 DEBUG:boto:Params: {} 2022-09-03T06:47:29.139 DEBUG:boto:Token: None 2022-09-03T06:47:29.140 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:29.140 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fm/X/TLRVStrKPEdi0XhFpVTfwE= 2022-09-03T06:47:29.141 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fm/X/TLRVStrKPEdi0XhFpVTfwE=', 'Content-Length': '0'} 2022-09-03T06:47:29.144 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000052a560e704659324-006312f881-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.186 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:29.187 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:29.188 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:29.188 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:29.189 DEBUG:boto:Method: GET 2022-09-03T06:47:29.190 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:47:29.191 DEBUG:boto:Data: 2022-09-03T06:47:29.191 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.192 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.193 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.194 DEBUG:boto:Params: {} 2022-09-03T06:47:29.195 DEBUG:boto:Token: None 2022-09-03T06:47:29.197 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/ 2022-09-03T06:47:29.198 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk= 2022-09-03T06:47:29.199 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk=', 'Content-Length': '0'} 2022-09-03T06:47:29.204 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b51a60e7d9398204-006312f881-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.205 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:29.207 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:29.208 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:29.209 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:29.210 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:29.210 DEBUG:boto:Method: GET 2022-09-03T06:47:29.211 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:47:29.212 DEBUG:boto:Data: 2022-09-03T06:47:29.212 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.214 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.214 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.215 DEBUG:boto:Params: {} 2022-09-03T06:47:29.216 DEBUG:boto:Token: None 2022-09-03T06:47:29.217 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/ 2022-09-03T06:47:29.217 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk= 2022-09-03T06:47:29.218 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk=', 'Content-Length': '0'} 2022-09-03T06:47:29.223 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000635d615301cd4928-006312f881-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.224 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:29.225 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:29.225 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:29.226 DEBUG:boto:Method: HEAD 2022-09-03T06:47:29.227 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:47:29.227 DEBUG:boto:Data: 2022-09-03T06:47:29.228 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.228 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.229 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.231 DEBUG:boto:Params: {} 2022-09-03T06:47:29.232 DEBUG:boto:Token: None 2022-09-03T06:47:29.233 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj58 2022-09-03T06:47:29.234 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c3NrIGexJt7TbuyRsiIOp8P1YCY= 2022-09-03T06:47:29.234 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c3NrIGexJt7TbuyRsiIOp8P1YCY=', 'Content-Length': '0'} 2022-09-03T06:47:29.238 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000002154e3c01319521-006312f881-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.240 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:29.241 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:29.244 DEBUG:boto:Method: HEAD 2022-09-03T06:47:29.244 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:47:29.245 DEBUG:boto:Data: 2022-09-03T06:47:29.246 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.247 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.247 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.248 DEBUG:boto:Params: {} 2022-09-03T06:47:29.249 DEBUG:boto:Token: None 2022-09-03T06:47:29.250 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj58 2022-09-03T06:47:29.251 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c3NrIGexJt7TbuyRsiIOp8P1YCY= 2022-09-03T06:47:29.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c3NrIGexJt7TbuyRsiIOp8P1YCY=', 'Content-Length': '0'} 2022-09-03T06:47:29.256 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000687c337fc242d4c5-006312f881-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.259 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:47:29.260 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:29.260 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:29.261 DEBUG:boto:Method: GET 2022-09-03T06:47:29.261 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:47:29.262 DEBUG:boto:Data: 2022-09-03T06:47:29.263 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.264 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.264 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.265 DEBUG:boto:Params: {} 2022-09-03T06:47:29.265 DEBUG:boto:Token: None 2022-09-03T06:47:29.266 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj58 2022-09-03T06:47:29.267 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7an6qNgtMgFFZfPh+G3lcPzog3U= 2022-09-03T06:47:29.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7an6qNgtMgFFZfPh+G3lcPzog3U=', 'Content-Length': '0'} 2022-09-03T06:47:29.270 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000486b52ff5a7fcb96-006312f881-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.314 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:29.315 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:29.316 DEBUG:boto:Method: GET 2022-09-03T06:47:29.316 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:47:29.317 DEBUG:boto:Data: 2022-09-03T06:47:29.317 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.318 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.319 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.319 DEBUG:boto:Params: {} 2022-09-03T06:47:29.320 DEBUG:boto:Token: None 2022-09-03T06:47:29.321 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj58 2022-09-03T06:47:29.321 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7an6qNgtMgFFZfPh+G3lcPzog3U= 2022-09-03T06:47:29.322 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7an6qNgtMgFFZfPh+G3lcPzog3U=', 'Content-Length': '0'} 2022-09-03T06:47:29.324 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000057533595527a5de2-006312f881-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.325 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:29.326 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:29.326 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:29.327 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:29.327 DEBUG:boto:Method: GET 2022-09-03T06:47:29.328 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:47:29.328 DEBUG:boto:Data: 2022-09-03T06:47:29.329 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.329 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.330 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.331 DEBUG:boto:Params: {} 2022-09-03T06:47:29.331 DEBUG:boto:Token: None 2022-09-03T06:47:29.332 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/ 2022-09-03T06:47:29.333 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk= 2022-09-03T06:47:29.333 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk=', 'Content-Length': '0'} 2022-09-03T06:47:29.337 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009523f2f981bc4d24-006312f881-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.338 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:29.339 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:29.339 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:29.340 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:29.341 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:29.342 DEBUG:boto:Method: GET 2022-09-03T06:47:29.343 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:47:29.343 DEBUG:boto:Data: 2022-09-03T06:47:29.344 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.344 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.345 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.346 DEBUG:boto:Params: {} 2022-09-03T06:47:29.346 DEBUG:boto:Token: None 2022-09-03T06:47:29.347 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/ 2022-09-03T06:47:29.348 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk= 2022-09-03T06:47:29.349 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk=', 'Content-Length': '0'} 2022-09-03T06:47:29.354 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006f36c3df2c1175c7-006312f881-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.355 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:29.357 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2022-09-03T06:47:29.358 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:29.359 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:29.360 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.360 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.361 DEBUG:boto:Method: GET 2022-09-03T06:47:29.362 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.363 DEBUG:boto:Data: 2022-09-03T06:47:29.364 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.365 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.367 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.367 DEBUG:boto:Params: {} 2022-09-03T06:47:29.368 DEBUG:boto:Token: None 2022-09-03T06:47:29.369 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.370 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GZpzDqCHO0qEtqGnpek5Lpbdlb4= 2022-09-03T06:47:29.370 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GZpzDqCHO0qEtqGnpek5Lpbdlb4=', 'Content-Length': '0'} 2022-09-03T06:47:29.373 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007faf1240fef45500-006312f881-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.374 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:29.374 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:29.375 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.375 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.376 DEBUG:boto:Method: GET 2022-09-03T06:47:29.376 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.377 DEBUG:boto:Data: 2022-09-03T06:47:29.377 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.378 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.378 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.379 DEBUG:boto:Params: {} 2022-09-03T06:47:29.379 DEBUG:boto:Token: None 2022-09-03T06:47:29.380 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.381 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GZpzDqCHO0qEtqGnpek5Lpbdlb4= 2022-09-03T06:47:29.381 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GZpzDqCHO0qEtqGnpek5Lpbdlb4=', 'Content-Length': '0'} 2022-09-03T06:47:29.384 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a0a10ef359375d31-006312f881-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.385 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:29.385 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:29.386 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.386 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.387 DEBUG:boto:Method: HEAD 2022-09-03T06:47:29.387 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.388 DEBUG:boto:Data: 2022-09-03T06:47:29.388 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.389 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.389 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.390 DEBUG:boto:Params: {} 2022-09-03T06:47:29.390 DEBUG:boto:Token: None 2022-09-03T06:47:29.391 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.392 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5JMYH5Ws6h7uKiaO168K3PiYLPk= 2022-09-03T06:47:29.392 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5JMYH5Ws6h7uKiaO168K3PiYLPk=', 'Content-Length': '0'} 2022-09-03T06:47:29.395 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ace04eefc13e1e3e-006312f881-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.396 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:29.396 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:29.397 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.398 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.399 DEBUG:boto:Method: HEAD 2022-09-03T06:47:29.400 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.400 DEBUG:boto:Data: 2022-09-03T06:47:29.401 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.402 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.403 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.404 DEBUG:boto:Params: {} 2022-09-03T06:47:29.406 DEBUG:boto:Token: None 2022-09-03T06:47:29.407 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.408 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5JMYH5Ws6h7uKiaO168K3PiYLPk= 2022-09-03T06:47:29.409 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5JMYH5Ws6h7uKiaO168K3PiYLPk=', 'Content-Length': '0'} 2022-09-03T06:47:29.411 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b66040e9f90d49e6-006312f881-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.412 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2022-09-03T06:47:29.413 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:29.413 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:29.414 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.414 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.415 DEBUG:boto:Method: GET 2022-09-03T06:47:29.415 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.416 DEBUG:boto:Data: 2022-09-03T06:47:29.416 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.417 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.417 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.418 DEBUG:boto:Params: {} 2022-09-03T06:47:29.419 DEBUG:boto:Token: None 2022-09-03T06:47:29.419 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.420 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GZpzDqCHO0qEtqGnpek5Lpbdlb4= 2022-09-03T06:47:29.421 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GZpzDqCHO0qEtqGnpek5Lpbdlb4=', 'Content-Length': '0'} 2022-09-03T06:47:29.423 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007fa4b2b4ad10127e-006312f881-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.424 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:29.424 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:29.425 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.425 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.426 DEBUG:boto:Method: GET 2022-09-03T06:47:29.426 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.427 DEBUG:boto:Data: 2022-09-03T06:47:29.427 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.428 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.429 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.429 DEBUG:boto:Params: {} 2022-09-03T06:47:29.430 DEBUG:boto:Token: None 2022-09-03T06:47:29.430 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:29.431 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GZpzDqCHO0qEtqGnpek5Lpbdlb4= 2022-09-03T06:47:29.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GZpzDqCHO0qEtqGnpek5Lpbdlb4=', 'Content-Length': '0'} 2022-09-03T06:47:29.434 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c41585c5b1ba157-006312f881-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.435 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:29.435 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:29.436 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:29.436 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:29.437 DEBUG:boto:Method: GET 2022-09-03T06:47:29.438 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:47:29.438 DEBUG:boto:Data: 2022-09-03T06:47:29.439 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.439 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.440 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.440 DEBUG:boto:Params: {} 2022-09-03T06:47:29.441 DEBUG:boto:Token: None 2022-09-03T06:47:29.442 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/ 2022-09-03T06:47:29.442 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk= 2022-09-03T06:47:29.443 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk=', 'Content-Length': '0'} 2022-09-03T06:47:29.446 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b732a86460d2e3a7-006312f881-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.486 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:29.487 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:29.488 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:29.488 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:29.489 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:29.489 DEBUG:boto:Method: GET 2022-09-03T06:47:29.490 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:47:29.490 DEBUG:boto:Data: 2022-09-03T06:47:29.491 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.491 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.492 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.493 DEBUG:boto:Params: {} 2022-09-03T06:47:29.493 DEBUG:boto:Token: None 2022-09-03T06:47:29.494 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/ 2022-09-03T06:47:29.494 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk= 2022-09-03T06:47:29.495 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk=', 'Content-Length': '0'} 2022-09-03T06:47:29.499 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000099d896ee4ddd299-006312f881-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.500 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:29.501 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:29.501 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:29.502 DEBUG:boto:Method: HEAD 2022-09-03T06:47:29.502 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:47:29.503 DEBUG:boto:Data: 2022-09-03T06:47:29.503 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.504 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.504 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.505 DEBUG:boto:Params: {} 2022-09-03T06:47:29.506 DEBUG:boto:Token: None 2022-09-03T06:47:29.506 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj62 2022-09-03T06:47:29.507 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CNvUfIIwUJomW1bOWZaWKNd0JHg= 2022-09-03T06:47:29.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CNvUfIIwUJomW1bOWZaWKNd0JHg=', 'Content-Length': '0'} 2022-09-03T06:47:29.510 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e3f1ad8d0faf0db4-006312f881-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.511 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:29.511 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:29.512 DEBUG:boto:Method: HEAD 2022-09-03T06:47:29.512 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:47:29.513 DEBUG:boto:Data: 2022-09-03T06:47:29.513 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.514 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.514 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.515 DEBUG:boto:Params: {} 2022-09-03T06:47:29.516 DEBUG:boto:Token: None 2022-09-03T06:47:29.516 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj62 2022-09-03T06:47:29.517 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CNvUfIIwUJomW1bOWZaWKNd0JHg= 2022-09-03T06:47:29.517 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CNvUfIIwUJomW1bOWZaWKNd0JHg=', 'Content-Length': '0'} 2022-09-03T06:47:29.520 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cab7cfeda88c0393-006312f881-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.521 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2022-09-03T06:47:29.522 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:29.523 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:29.524 DEBUG:boto:Method: GET 2022-09-03T06:47:29.524 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:47:29.525 DEBUG:boto:Data: 2022-09-03T06:47:29.525 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.526 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.527 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.527 DEBUG:boto:Params: {} 2022-09-03T06:47:29.528 DEBUG:boto:Token: None 2022-09-03T06:47:29.529 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj62 2022-09-03T06:47:29.529 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s277PSUUE9xDt5k6LGXnj1KYrBs= 2022-09-03T06:47:29.530 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s277PSUUE9xDt5k6LGXnj1KYrBs=', 'Content-Length': '0'} 2022-09-03T06:47:29.532 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000da7edc6293758c94-006312f881-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.533 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:29.534 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:29.535 DEBUG:boto:Method: GET 2022-09-03T06:47:29.535 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:47:29.536 DEBUG:boto:Data: 2022-09-03T06:47:29.536 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.537 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.537 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.538 DEBUG:boto:Params: {} 2022-09-03T06:47:29.539 DEBUG:boto:Token: None 2022-09-03T06:47:29.539 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj62 2022-09-03T06:47:29.540 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s277PSUUE9xDt5k6LGXnj1KYrBs= 2022-09-03T06:47:29.541 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s277PSUUE9xDt5k6LGXnj1KYrBs=', 'Content-Length': '0'} 2022-09-03T06:47:29.544 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005ca0042b2864dcae-006312f881-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.586 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:29.587 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:29.588 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:29.588 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:29.589 DEBUG:boto:Method: GET 2022-09-03T06:47:29.590 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:47:29.590 DEBUG:boto:Data: 2022-09-03T06:47:29.591 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.591 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.592 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.592 DEBUG:boto:Params: {} 2022-09-03T06:47:29.593 DEBUG:boto:Token: None 2022-09-03T06:47:29.594 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/ 2022-09-03T06:47:29.595 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk= 2022-09-03T06:47:29.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk=', 'Content-Length': '0'} 2022-09-03T06:47:29.600 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000073749af78b0330b6-006312f881-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.601 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:29.602 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:29.603 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:29.604 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:29.605 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:29.605 DEBUG:boto:Method: GET 2022-09-03T06:47:29.606 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:47:29.607 DEBUG:boto:Data: 2022-09-03T06:47:29.608 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.608 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.609 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.610 DEBUG:boto:Params: {} 2022-09-03T06:47:29.610 DEBUG:boto:Token: None 2022-09-03T06:47:29.611 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/ 2022-09-03T06:47:29.612 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk= 2022-09-03T06:47:29.613 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk=', 'Content-Length': '0'} 2022-09-03T06:47:29.618 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009a235dd4a73f8838-006312f881-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.618 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:29.619 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:47:29.620 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:29.621 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:29.622 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.623 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.624 DEBUG:boto:Method: GET 2022-09-03T06:47:29.625 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.626 DEBUG:boto:Data: 2022-09-03T06:47:29.627 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.628 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.629 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.630 DEBUG:boto:Params: {} 2022-09-03T06:47:29.631 DEBUG:boto:Token: None 2022-09-03T06:47:29.633 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.633 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1i5E9UPBoC5QgYzb41qmw5ZDzIE= 2022-09-03T06:47:29.634 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1i5E9UPBoC5QgYzb41qmw5ZDzIE=', 'Content-Length': '0'} 2022-09-03T06:47:29.636 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005be0d6e59cf73553-006312f881-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.637 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:29.638 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:29.638 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.639 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.640 DEBUG:boto:Method: GET 2022-09-03T06:47:29.640 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.641 DEBUG:boto:Data: 2022-09-03T06:47:29.642 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.642 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.643 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.644 DEBUG:boto:Params: {} 2022-09-03T06:47:29.644 DEBUG:boto:Token: None 2022-09-03T06:47:29.645 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.646 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1i5E9UPBoC5QgYzb41qmw5ZDzIE= 2022-09-03T06:47:29.647 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1i5E9UPBoC5QgYzb41qmw5ZDzIE=', 'Content-Length': '0'} 2022-09-03T06:47:29.649 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000053ffd883b3c17eea-006312f881-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.650 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:29.651 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:29.651 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.652 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.652 DEBUG:boto:Method: HEAD 2022-09-03T06:47:29.653 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.653 DEBUG:boto:Data: 2022-09-03T06:47:29.654 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.654 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.655 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.655 DEBUG:boto:Params: {} 2022-09-03T06:47:29.656 DEBUG:boto:Token: None 2022-09-03T06:47:29.657 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.657 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BdHJ1dnJhX+DkX7iw5+BepOVqLk= 2022-09-03T06:47:29.658 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BdHJ1dnJhX+DkX7iw5+BepOVqLk=', 'Content-Length': '0'} 2022-09-03T06:47:29.660 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c0602963d0fd3713-006312f881-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.661 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:29.661 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:29.662 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.663 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.663 DEBUG:boto:Method: HEAD 2022-09-03T06:47:29.664 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.664 DEBUG:boto:Data: 2022-09-03T06:47:29.665 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.665 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.666 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.666 DEBUG:boto:Params: {} 2022-09-03T06:47:29.667 DEBUG:boto:Token: None 2022-09-03T06:47:29.667 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.668 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BdHJ1dnJhX+DkX7iw5+BepOVqLk= 2022-09-03T06:47:29.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BdHJ1dnJhX+DkX7iw5+BepOVqLk=', 'Content-Length': '0'} 2022-09-03T06:47:29.671 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e21c0705b3d8f5a-006312f881-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.672 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:47:29.672 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:29.673 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:29.673 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.674 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.675 DEBUG:boto:Method: GET 2022-09-03T06:47:29.675 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.676 DEBUG:boto:Data: 2022-09-03T06:47:29.676 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.677 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.678 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.678 DEBUG:boto:Params: {} 2022-09-03T06:47:29.679 DEBUG:boto:Token: None 2022-09-03T06:47:29.680 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.680 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1i5E9UPBoC5QgYzb41qmw5ZDzIE= 2022-09-03T06:47:29.681 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1i5E9UPBoC5QgYzb41qmw5ZDzIE=', 'Content-Length': '0'} 2022-09-03T06:47:29.683 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e5f1a9f521fae2f-006312f881-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.684 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:29.685 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:29.685 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.686 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.687 DEBUG:boto:Method: GET 2022-09-03T06:47:29.687 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.688 DEBUG:boto:Data: 2022-09-03T06:47:29.688 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.689 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.689 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.690 DEBUG:boto:Params: {} 2022-09-03T06:47:29.691 DEBUG:boto:Token: None 2022-09-03T06:47:29.691 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:29.692 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1i5E9UPBoC5QgYzb41qmw5ZDzIE= 2022-09-03T06:47:29.693 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1i5E9UPBoC5QgYzb41qmw5ZDzIE=', 'Content-Length': '0'} 2022-09-03T06:47:29.695 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002950c18dc4640913-006312f881-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.696 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:29.696 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:29.697 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:29.698 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:29.698 DEBUG:boto:Method: GET 2022-09-03T06:47:29.699 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:47:29.699 DEBUG:boto:Data: 2022-09-03T06:47:29.700 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.700 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.701 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.702 DEBUG:boto:Params: {} 2022-09-03T06:47:29.702 DEBUG:boto:Token: None 2022-09-03T06:47:29.703 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/ 2022-09-03T06:47:29.704 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk= 2022-09-03T06:47:29.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk=', 'Content-Length': '0'} 2022-09-03T06:47:29.707 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fee2861118198d24-006312f881-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.708 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:29.709 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:29.709 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:29.710 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:29.711 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:29.712 DEBUG:boto:Method: GET 2022-09-03T06:47:29.713 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:47:29.713 DEBUG:boto:Data: 2022-09-03T06:47:29.714 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.715 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.716 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.716 DEBUG:boto:Params: {} 2022-09-03T06:47:29.717 DEBUG:boto:Token: None 2022-09-03T06:47:29.718 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/ 2022-09-03T06:47:29.719 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk= 2022-09-03T06:47:29.719 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk=', 'Content-Length': '0'} 2022-09-03T06:47:29.723 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e5201d252a3618da-006312f881-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.766 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:29.768 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:29.769 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:29.770 DEBUG:boto:Method: HEAD 2022-09-03T06:47:29.771 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:47:29.772 DEBUG:boto:Data: 2022-09-03T06:47:29.773 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.774 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.775 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.776 DEBUG:boto:Params: {} 2022-09-03T06:47:29.776 DEBUG:boto:Token: None 2022-09-03T06:47:29.777 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj68 2022-09-03T06:47:29.778 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3zewQ2pq+n+Mr/LuB4PetN0tSds= 2022-09-03T06:47:29.779 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3zewQ2pq+n+Mr/LuB4PetN0tSds=', 'Content-Length': '0'} 2022-09-03T06:47:29.782 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005626fce9dab2389b-006312f881-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.783 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:29.783 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:29.784 DEBUG:boto:Method: HEAD 2022-09-03T06:47:29.784 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:47:29.785 DEBUG:boto:Data: 2022-09-03T06:47:29.785 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.786 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.787 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.787 DEBUG:boto:Params: {} 2022-09-03T06:47:29.788 DEBUG:boto:Token: None 2022-09-03T06:47:29.788 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj68 2022-09-03T06:47:29.789 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3zewQ2pq+n+Mr/LuB4PetN0tSds= 2022-09-03T06:47:29.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3zewQ2pq+n+Mr/LuB4PetN0tSds=', 'Content-Length': '0'} 2022-09-03T06:47:29.792 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065782e2c545f8d0a-006312f881-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.793 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:47:29.793 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:29.794 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:29.795 DEBUG:boto:Method: GET 2022-09-03T06:47:29.795 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:47:29.796 DEBUG:boto:Data: 2022-09-03T06:47:29.796 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.797 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.797 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.798 DEBUG:boto:Params: {} 2022-09-03T06:47:29.798 DEBUG:boto:Token: None 2022-09-03T06:47:29.799 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj68 2022-09-03T06:47:29.800 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bzEWP/Xo9z1AHY1L098WoNx9/Ko= 2022-09-03T06:47:29.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bzEWP/Xo9z1AHY1L098WoNx9/Ko=', 'Content-Length': '0'} 2022-09-03T06:47:29.803 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d3e0c10541c9ff9-006312f881-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.850 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:29.851 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:29.851 DEBUG:boto:Method: GET 2022-09-03T06:47:29.852 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:47:29.852 DEBUG:boto:Data: 2022-09-03T06:47:29.853 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.853 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.854 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.855 DEBUG:boto:Params: {} 2022-09-03T06:47:29.855 DEBUG:boto:Token: None 2022-09-03T06:47:29.856 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj68 2022-09-03T06:47:29.856 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bzEWP/Xo9z1AHY1L098WoNx9/Ko= 2022-09-03T06:47:29.857 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bzEWP/Xo9z1AHY1L098WoNx9/Ko=', 'Content-Length': '0'} 2022-09-03T06:47:29.859 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fff44fc88176c3c8-006312f881-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.860 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:29.861 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:29.861 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:29.862 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:29.862 DEBUG:boto:Method: GET 2022-09-03T06:47:29.863 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:47:29.863 DEBUG:boto:Data: 2022-09-03T06:47:29.864 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.865 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.865 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.866 DEBUG:boto:Params: {} 2022-09-03T06:47:29.866 DEBUG:boto:Token: None 2022-09-03T06:47:29.867 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/ 2022-09-03T06:47:29.868 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk= 2022-09-03T06:47:29.869 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk=', 'Content-Length': '0'} 2022-09-03T06:47:29.874 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000728c3da3e0167b25-006312f881-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.875 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:29.877 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:29.878 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:29.879 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:29.880 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:29.880 DEBUG:boto:Method: GET 2022-09-03T06:47:29.881 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:47:29.881 DEBUG:boto:Data: 2022-09-03T06:47:29.882 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.882 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.883 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.884 DEBUG:boto:Params: {} 2022-09-03T06:47:29.884 DEBUG:boto:Token: None 2022-09-03T06:47:29.885 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/ 2022-09-03T06:47:29.885 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk= 2022-09-03T06:47:29.886 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFgfIEhWIbM9Iv7AvOxYvtCBOVk=', 'Content-Length': '0'} 2022-09-03T06:47:29.891 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d3b2d122b2b8c6c1-006312f881-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.891 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:29.892 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:47:29.893 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:29.894 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:29.894 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.895 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.895 DEBUG:boto:Method: GET 2022-09-03T06:47:29.896 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.897 DEBUG:boto:Data: 2022-09-03T06:47:29.897 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.898 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.898 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.899 DEBUG:boto:Params: {} 2022-09-03T06:47:29.900 DEBUG:boto:Token: None 2022-09-03T06:47:29.901 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.902 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZPDUZzvJv9rvCzhClCTQMsIzet8= 2022-09-03T06:47:29.903 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZPDUZzvJv9rvCzhClCTQMsIzet8=', 'Content-Length': '0'} 2022-09-03T06:47:29.906 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aa08581eeb84118f-006312f881-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.907 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:29.908 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:29.909 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.910 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.911 DEBUG:boto:Method: GET 2022-09-03T06:47:29.911 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.912 DEBUG:boto:Data: 2022-09-03T06:47:29.912 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.913 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.913 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.914 DEBUG:boto:Params: {} 2022-09-03T06:47:29.915 DEBUG:boto:Token: None 2022-09-03T06:47:29.915 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.916 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZPDUZzvJv9rvCzhClCTQMsIzet8= 2022-09-03T06:47:29.917 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZPDUZzvJv9rvCzhClCTQMsIzet8=', 'Content-Length': '0'} 2022-09-03T06:47:29.919 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000081c5bee4b3da6c34-006312f881-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.920 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:29.921 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:29.922 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.922 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.924 DEBUG:boto:Method: HEAD 2022-09-03T06:47:29.924 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.925 DEBUG:boto:Data: 2022-09-03T06:47:29.926 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.926 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.927 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.928 DEBUG:boto:Params: {} 2022-09-03T06:47:29.928 DEBUG:boto:Token: None 2022-09-03T06:47:29.929 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.929 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zjk86oWFIGINmO6wXMzwgoPB9+M= 2022-09-03T06:47:29.930 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zjk86oWFIGINmO6wXMzwgoPB9+M=', 'Content-Length': '0'} 2022-09-03T06:47:29.932 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd2bde6f218429f6-006312f881-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.934 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:29.934 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:29.935 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.935 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.936 DEBUG:boto:Method: HEAD 2022-09-03T06:47:29.936 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.937 DEBUG:boto:Data: 2022-09-03T06:47:29.937 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.938 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.938 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.939 DEBUG:boto:Params: {} 2022-09-03T06:47:29.940 DEBUG:boto:Token: None 2022-09-03T06:47:29.940 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.941 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zjk86oWFIGINmO6wXMzwgoPB9+M= 2022-09-03T06:47:29.942 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zjk86oWFIGINmO6wXMzwgoPB9+M=', 'Content-Length': '0'} 2022-09-03T06:47:29.944 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c64087f7543e0e3d-006312f881-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.945 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:47:29.946 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:29.947 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:29.948 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.949 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.950 DEBUG:boto:Method: GET 2022-09-03T06:47:29.950 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.951 DEBUG:boto:Data: 2022-09-03T06:47:29.952 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.953 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:29.955 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.956 DEBUG:boto:Params: {} 2022-09-03T06:47:29.957 DEBUG:boto:Token: None 2022-09-03T06:47:29.958 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.959 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZPDUZzvJv9rvCzhClCTQMsIzet8= 2022-09-03T06:47:29.960 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZPDUZzvJv9rvCzhClCTQMsIzet8=', 'Content-Length': '0'} 2022-09-03T06:47:29.963 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007a2bbbc5b1bc5ecb-006312f881-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:29.964 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:29.965 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:29.966 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.967 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.968 DEBUG:boto:Method: GET 2022-09-03T06:47:29.969 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.970 DEBUG:boto:Data: 2022-09-03T06:47:29.971 DEBUG:boto:Headers: {} 2022-09-03T06:47:29.971 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:29.972 DEBUG:boto:Port: 8000 2022-09-03T06:47:29.973 DEBUG:boto:Params: {} 2022-09-03T06:47:29.973 DEBUG:boto:Token: None 2022-09-03T06:47:29.974 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:29 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:29.975 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZPDUZzvJv9rvCzhClCTQMsIzet8= 2022-09-03T06:47:29.975 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZPDUZzvJv9rvCzhClCTQMsIzet8=', 'Content-Length': '0'} 2022-09-03T06:47:29.977 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000054b0bb946b1493e4-006312f881-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.018 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:30.020 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:30.021 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:30.022 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:30.023 DEBUG:boto:Method: GET 2022-09-03T06:47:30.024 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:47:30.024 DEBUG:boto:Data: 2022-09-03T06:47:30.025 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.025 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.026 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.027 DEBUG:boto:Params: {} 2022-09-03T06:47:30.027 DEBUG:boto:Token: None 2022-09-03T06:47:30.028 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/ 2022-09-03T06:47:30.029 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4= 2022-09-03T06:47:30.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4=', 'Content-Length': '0'} 2022-09-03T06:47:30.033 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d3a6a69cf571e9f8-006312f882-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.034 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:30.035 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:30.036 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:30.036 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:30.037 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:30.037 DEBUG:boto:Method: GET 2022-09-03T06:47:30.038 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:47:30.039 DEBUG:boto:Data: 2022-09-03T06:47:30.039 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.040 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.040 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.041 DEBUG:boto:Params: {} 2022-09-03T06:47:30.041 DEBUG:boto:Token: None 2022-09-03T06:47:30.042 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/ 2022-09-03T06:47:30.043 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4= 2022-09-03T06:47:30.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4=', 'Content-Length': '0'} 2022-09-03T06:47:30.047 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000020063ce472f873f-006312f882-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.048 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:30.049 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:30.050 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:30.050 DEBUG:boto:Method: HEAD 2022-09-03T06:47:30.051 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:47:30.051 DEBUG:boto:Data: 2022-09-03T06:47:30.052 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.052 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.053 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.053 DEBUG:boto:Params: {} 2022-09-03T06:47:30.054 DEBUG:boto:Token: None 2022-09-03T06:47:30.055 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj69 2022-09-03T06:47:30.058 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mSW+KEej/CwCoqb4c0kW8U0s8o4= 2022-09-03T06:47:30.059 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mSW+KEej/CwCoqb4c0kW8U0s8o4=', 'Content-Length': '0'} 2022-09-03T06:47:30.061 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000adadc8b1a1f19725-006312f882-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.063 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:30.064 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:30.065 DEBUG:boto:Method: HEAD 2022-09-03T06:47:30.066 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:47:30.068 DEBUG:boto:Data: 2022-09-03T06:47:30.069 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.070 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.071 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.072 DEBUG:boto:Params: {} 2022-09-03T06:47:30.073 DEBUG:boto:Token: None 2022-09-03T06:47:30.073 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj69 2022-09-03T06:47:30.074 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mSW+KEej/CwCoqb4c0kW8U0s8o4= 2022-09-03T06:47:30.075 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mSW+KEej/CwCoqb4c0kW8U0s8o4=', 'Content-Length': '0'} 2022-09-03T06:47:30.078 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b7722b8073461e00-006312f882-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.079 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:47:30.080 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:30.081 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:30.082 DEBUG:boto:Method: GET 2022-09-03T06:47:30.083 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:47:30.084 DEBUG:boto:Data: 2022-09-03T06:47:30.085 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.086 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.087 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.088 DEBUG:boto:Params: {} 2022-09-03T06:47:30.089 DEBUG:boto:Token: None 2022-09-03T06:47:30.090 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj69 2022-09-03T06:47:30.091 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HCP9fhZPwcPYRox+H7ZMwGzoOHQ= 2022-09-03T06:47:30.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HCP9fhZPwcPYRox+H7ZMwGzoOHQ=', 'Content-Length': '0'} 2022-09-03T06:47:30.094 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000397d2cd9ac234348-006312f882-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.094 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:30.095 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:30.096 DEBUG:boto:Method: GET 2022-09-03T06:47:30.096 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:47:30.097 DEBUG:boto:Data: 2022-09-03T06:47:30.097 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.098 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.098 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.099 DEBUG:boto:Params: {} 2022-09-03T06:47:30.099 DEBUG:boto:Token: None 2022-09-03T06:47:30.100 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj69 2022-09-03T06:47:30.101 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HCP9fhZPwcPYRox+H7ZMwGzoOHQ= 2022-09-03T06:47:30.101 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HCP9fhZPwcPYRox+H7ZMwGzoOHQ=', 'Content-Length': '0'} 2022-09-03T06:47:30.104 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000877b57624760bed6-006312f882-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.105 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:30.107 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:30.108 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:30.108 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:30.109 DEBUG:boto:Method: GET 2022-09-03T06:47:30.110 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:47:30.111 DEBUG:boto:Data: 2022-09-03T06:47:30.112 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.113 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.114 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.115 DEBUG:boto:Params: {} 2022-09-03T06:47:30.116 DEBUG:boto:Token: None 2022-09-03T06:47:30.117 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/ 2022-09-03T06:47:30.118 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4= 2022-09-03T06:47:30.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4=', 'Content-Length': '0'} 2022-09-03T06:47:30.122 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002b42134850f81a7a-006312f882-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.123 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:30.124 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:30.125 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:30.125 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:30.126 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:30.127 DEBUG:boto:Method: GET 2022-09-03T06:47:30.127 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:47:30.128 DEBUG:boto:Data: 2022-09-03T06:47:30.129 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.130 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.130 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.131 DEBUG:boto:Params: {} 2022-09-03T06:47:30.131 DEBUG:boto:Token: None 2022-09-03T06:47:30.132 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/ 2022-09-03T06:47:30.133 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4= 2022-09-03T06:47:30.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4=', 'Content-Length': '0'} 2022-09-03T06:47:30.138 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000063ea50cdc91632a1-006312f882-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.139 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:30.141 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:47:30.141 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:30.142 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:30.143 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.143 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.144 DEBUG:boto:Method: GET 2022-09-03T06:47:30.145 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.146 DEBUG:boto:Data: 2022-09-03T06:47:30.147 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.148 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.149 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.150 DEBUG:boto:Params: {} 2022-09-03T06:47:30.151 DEBUG:boto:Token: None 2022-09-03T06:47:30.153 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.153 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XKFi4yoYWgOxoH7rf/HaBM6Z+Ng= 2022-09-03T06:47:30.154 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XKFi4yoYWgOxoH7rf/HaBM6Z+Ng=', 'Content-Length': '0'} 2022-09-03T06:47:30.156 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f31e75c8dd2e020-006312f882-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.198 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:30.199 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:30.200 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.200 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.201 DEBUG:boto:Method: GET 2022-09-03T06:47:30.202 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.203 DEBUG:boto:Data: 2022-09-03T06:47:30.203 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.204 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.204 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.205 DEBUG:boto:Params: {} 2022-09-03T06:47:30.205 DEBUG:boto:Token: None 2022-09-03T06:47:30.206 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.208 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XKFi4yoYWgOxoH7rf/HaBM6Z+Ng= 2022-09-03T06:47:30.209 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XKFi4yoYWgOxoH7rf/HaBM6Z+Ng=', 'Content-Length': '0'} 2022-09-03T06:47:30.212 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005167b8969b71b919-006312f882-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.213 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:30.214 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:30.215 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.215 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.216 DEBUG:boto:Method: HEAD 2022-09-03T06:47:30.216 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.217 DEBUG:boto:Data: 2022-09-03T06:47:30.217 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.218 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.219 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.219 DEBUG:boto:Params: {} 2022-09-03T06:47:30.220 DEBUG:boto:Token: None 2022-09-03T06:47:30.220 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.221 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hTXf/Ed5/jI4YF3wARQoELyFGzQ= 2022-09-03T06:47:30.222 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hTXf/Ed5/jI4YF3wARQoELyFGzQ=', 'Content-Length': '0'} 2022-09-03T06:47:30.224 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000bcf1b4a64d550cc-006312f882-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.224 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:30.225 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:30.226 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.226 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.227 DEBUG:boto:Method: HEAD 2022-09-03T06:47:30.227 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.228 DEBUG:boto:Data: 2022-09-03T06:47:30.229 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.229 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.230 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.231 DEBUG:boto:Params: {} 2022-09-03T06:47:30.231 DEBUG:boto:Token: None 2022-09-03T06:47:30.232 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.232 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hTXf/Ed5/jI4YF3wARQoELyFGzQ= 2022-09-03T06:47:30.233 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hTXf/Ed5/jI4YF3wARQoELyFGzQ=', 'Content-Length': '0'} 2022-09-03T06:47:30.235 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008957386c5517eba2-006312f882-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.236 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:47:30.237 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:30.237 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:30.238 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.238 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.239 DEBUG:boto:Method: GET 2022-09-03T06:47:30.239 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.240 DEBUG:boto:Data: 2022-09-03T06:47:30.241 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.241 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.242 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.242 DEBUG:boto:Params: {} 2022-09-03T06:47:30.243 DEBUG:boto:Token: None 2022-09-03T06:47:30.243 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.244 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XKFi4yoYWgOxoH7rf/HaBM6Z+Ng= 2022-09-03T06:47:30.245 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XKFi4yoYWgOxoH7rf/HaBM6Z+Ng=', 'Content-Length': '0'} 2022-09-03T06:47:30.247 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003ea1ea5475b7bd77-006312f882-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.248 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:30.249 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:30.249 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.250 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.250 DEBUG:boto:Method: GET 2022-09-03T06:47:30.251 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.252 DEBUG:boto:Data: 2022-09-03T06:47:30.252 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.253 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.253 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.254 DEBUG:boto:Params: {} 2022-09-03T06:47:30.255 DEBUG:boto:Token: None 2022-09-03T06:47:30.255 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:30.256 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XKFi4yoYWgOxoH7rf/HaBM6Z+Ng= 2022-09-03T06:47:30.257 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XKFi4yoYWgOxoH7rf/HaBM6Z+Ng=', 'Content-Length': '0'} 2022-09-03T06:47:30.259 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f9ee3d33a8331f88-006312f882-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.260 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:30.261 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:30.261 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:30.262 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:30.263 DEBUG:boto:Method: GET 2022-09-03T06:47:30.263 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:47:30.264 DEBUG:boto:Data: 2022-09-03T06:47:30.264 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.265 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.265 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.266 DEBUG:boto:Params: {} 2022-09-03T06:47:30.267 DEBUG:boto:Token: None 2022-09-03T06:47:30.268 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/ 2022-09-03T06:47:30.268 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4= 2022-09-03T06:47:30.269 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4=', 'Content-Length': '0'} 2022-09-03T06:47:30.275 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004edc837f23095732-006312f882-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.275 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:30.276 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:30.278 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:30.278 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:30.279 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:30.280 DEBUG:boto:Method: GET 2022-09-03T06:47:30.281 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:47:30.282 DEBUG:boto:Data: 2022-09-03T06:47:30.283 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.284 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.285 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.286 DEBUG:boto:Params: {} 2022-09-03T06:47:30.287 DEBUG:boto:Token: None 2022-09-03T06:47:30.288 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/ 2022-09-03T06:47:30.289 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4= 2022-09-03T06:47:30.290 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4=', 'Content-Length': '0'} 2022-09-03T06:47:30.294 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003d321da355a7fdd4-006312f882-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.295 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:30.296 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:30.296 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:30.297 DEBUG:boto:Method: HEAD 2022-09-03T06:47:30.298 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:47:30.298 DEBUG:boto:Data: 2022-09-03T06:47:30.299 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.299 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.300 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.300 DEBUG:boto:Params: {} 2022-09-03T06:47:30.301 DEBUG:boto:Token: None 2022-09-03T06:47:30.302 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj76 2022-09-03T06:47:30.302 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bFa+PZpioCzyI94VsVkmiLyWB5M= 2022-09-03T06:47:30.303 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bFa+PZpioCzyI94VsVkmiLyWB5M=', 'Content-Length': '0'} 2022-09-03T06:47:30.305 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a4a551c4b0a3e65f-006312f882-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.306 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:30.307 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:30.307 DEBUG:boto:Method: HEAD 2022-09-03T06:47:30.308 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:47:30.308 DEBUG:boto:Data: 2022-09-03T06:47:30.309 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.309 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.310 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.310 DEBUG:boto:Params: {} 2022-09-03T06:47:30.311 DEBUG:boto:Token: None 2022-09-03T06:47:30.312 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj76 2022-09-03T06:47:30.312 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bFa+PZpioCzyI94VsVkmiLyWB5M= 2022-09-03T06:47:30.313 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bFa+PZpioCzyI94VsVkmiLyWB5M=', 'Content-Length': '0'} 2022-09-03T06:47:30.315 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a58a62372a453fe3-006312f882-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.316 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:47:30.317 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:30.317 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:30.318 DEBUG:boto:Method: GET 2022-09-03T06:47:30.319 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:47:30.319 DEBUG:boto:Data: 2022-09-03T06:47:30.320 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.320 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.321 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.321 DEBUG:boto:Params: {} 2022-09-03T06:47:30.322 DEBUG:boto:Token: None 2022-09-03T06:47:30.323 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj76 2022-09-03T06:47:30.323 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rKijOlzSeWKEUAkj+3j3OHCugi8= 2022-09-03T06:47:30.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rKijOlzSeWKEUAkj+3j3OHCugi8=', 'Content-Length': '0'} 2022-09-03T06:47:30.327 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000964cb054b068dd48-006312f882-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.374 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:30.375 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:30.376 DEBUG:boto:Method: GET 2022-09-03T06:47:30.376 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:47:30.377 DEBUG:boto:Data: 2022-09-03T06:47:30.377 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.378 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.379 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.380 DEBUG:boto:Params: {} 2022-09-03T06:47:30.380 DEBUG:boto:Token: None 2022-09-03T06:47:30.381 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj76 2022-09-03T06:47:30.382 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rKijOlzSeWKEUAkj+3j3OHCugi8= 2022-09-03T06:47:30.383 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rKijOlzSeWKEUAkj+3j3OHCugi8=', 'Content-Length': '0'} 2022-09-03T06:47:30.385 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d9fa85123486a09-006312f882-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.386 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:30.387 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:30.388 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:30.388 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:30.389 DEBUG:boto:Method: GET 2022-09-03T06:47:30.390 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:47:30.390 DEBUG:boto:Data: 2022-09-03T06:47:30.391 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.391 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.392 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.393 DEBUG:boto:Params: {} 2022-09-03T06:47:30.394 DEBUG:boto:Token: None 2022-09-03T06:47:30.394 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/ 2022-09-03T06:47:30.395 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4= 2022-09-03T06:47:30.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4=', 'Content-Length': '0'} 2022-09-03T06:47:30.399 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d591e8f8e6682e88-006312f882-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.400 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:30.401 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:30.401 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:30.402 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:30.402 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:30.403 DEBUG:boto:Method: GET 2022-09-03T06:47:30.404 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:47:30.404 DEBUG:boto:Data: 2022-09-03T06:47:30.405 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.405 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.406 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.407 DEBUG:boto:Params: {} 2022-09-03T06:47:30.407 DEBUG:boto:Token: None 2022-09-03T06:47:30.408 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/ 2022-09-03T06:47:30.409 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4= 2022-09-03T06:47:30.409 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4=', 'Content-Length': '0'} 2022-09-03T06:47:30.414 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005ebffb53a11c0a11-006312f882-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.415 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:30.415 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:47:30.416 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:30.417 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:30.417 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.418 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.418 DEBUG:boto:Method: GET 2022-09-03T06:47:30.419 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.419 DEBUG:boto:Data: 2022-09-03T06:47:30.420 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.420 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.421 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.421 DEBUG:boto:Params: {} 2022-09-03T06:47:30.422 DEBUG:boto:Token: None 2022-09-03T06:47:30.423 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.423 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q32J1fwn0S73Ye85WdZMS0QCKag= 2022-09-03T06:47:30.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q32J1fwn0S73Ye85WdZMS0QCKag=', 'Content-Length': '0'} 2022-09-03T06:47:30.426 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d9a1de86bdd0be8-006312f882-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.427 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:30.427 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:30.428 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.429 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.429 DEBUG:boto:Method: GET 2022-09-03T06:47:30.430 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.430 DEBUG:boto:Data: 2022-09-03T06:47:30.431 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.431 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.432 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.432 DEBUG:boto:Params: {} 2022-09-03T06:47:30.433 DEBUG:boto:Token: None 2022-09-03T06:47:30.434 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.434 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q32J1fwn0S73Ye85WdZMS0QCKag= 2022-09-03T06:47:30.435 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q32J1fwn0S73Ye85WdZMS0QCKag=', 'Content-Length': '0'} 2022-09-03T06:47:30.437 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009b777ad3146626a6-006312f882-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.438 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:30.438 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:30.439 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.440 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.440 DEBUG:boto:Method: HEAD 2022-09-03T06:47:30.441 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.442 DEBUG:boto:Data: 2022-09-03T06:47:30.442 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.443 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.443 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.444 DEBUG:boto:Params: {} 2022-09-03T06:47:30.444 DEBUG:boto:Token: None 2022-09-03T06:47:30.445 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.445 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8d2VB45tfSbQiGqZGRWFDaSGqwY= 2022-09-03T06:47:30.446 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8d2VB45tfSbQiGqZGRWFDaSGqwY=', 'Content-Length': '0'} 2022-09-03T06:47:30.448 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004ff40283f9d39ec3-006312f882-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.449 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:30.449 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:30.450 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.451 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.451 DEBUG:boto:Method: HEAD 2022-09-03T06:47:30.452 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.452 DEBUG:boto:Data: 2022-09-03T06:47:30.453 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.453 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.454 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.454 DEBUG:boto:Params: {} 2022-09-03T06:47:30.455 DEBUG:boto:Token: None 2022-09-03T06:47:30.455 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.456 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8d2VB45tfSbQiGqZGRWFDaSGqwY= 2022-09-03T06:47:30.457 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8d2VB45tfSbQiGqZGRWFDaSGqwY=', 'Content-Length': '0'} 2022-09-03T06:47:30.459 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000556c1358ff2580f1-006312f882-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.460 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:47:30.460 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:30.461 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:30.461 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.462 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.463 DEBUG:boto:Method: GET 2022-09-03T06:47:30.463 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.464 DEBUG:boto:Data: 2022-09-03T06:47:30.464 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.465 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.465 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.466 DEBUG:boto:Params: {} 2022-09-03T06:47:30.466 DEBUG:boto:Token: None 2022-09-03T06:47:30.467 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.468 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q32J1fwn0S73Ye85WdZMS0QCKag= 2022-09-03T06:47:30.469 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q32J1fwn0S73Ye85WdZMS0QCKag=', 'Content-Length': '0'} 2022-09-03T06:47:30.471 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e8bc7193349ace5a-006312f882-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.518 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:30.519 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:30.520 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.520 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.522 DEBUG:boto:Method: GET 2022-09-03T06:47:30.523 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.524 DEBUG:boto:Data: 2022-09-03T06:47:30.524 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.525 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.526 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.527 DEBUG:boto:Params: {} 2022-09-03T06:47:30.528 DEBUG:boto:Token: None 2022-09-03T06:47:30.529 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:30.530 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q32J1fwn0S73Ye85WdZMS0QCKag= 2022-09-03T06:47:30.531 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q32J1fwn0S73Ye85WdZMS0QCKag=', 'Content-Length': '0'} 2022-09-03T06:47:30.534 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eb538f09c85cf8c3-006312f882-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.536 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:30.536 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:30.537 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:30.538 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:30.539 DEBUG:boto:Method: GET 2022-09-03T06:47:30.539 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:47:30.540 DEBUG:boto:Data: 2022-09-03T06:47:30.540 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.541 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.541 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.542 DEBUG:boto:Params: {} 2022-09-03T06:47:30.543 DEBUG:boto:Token: None 2022-09-03T06:47:30.544 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/ 2022-09-03T06:47:30.544 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4= 2022-09-03T06:47:30.545 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4=', 'Content-Length': '0'} 2022-09-03T06:47:30.548 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bfefb6cea5d996e7-006312f882-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.549 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:30.550 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:30.551 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:30.552 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:30.552 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:30.553 DEBUG:boto:Method: GET 2022-09-03T06:47:30.554 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:47:30.555 DEBUG:boto:Data: 2022-09-03T06:47:30.556 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.557 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.557 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.558 DEBUG:boto:Params: {} 2022-09-03T06:47:30.559 DEBUG:boto:Token: None 2022-09-03T06:47:30.561 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/ 2022-09-03T06:47:30.562 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4= 2022-09-03T06:47:30.563 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4=', 'Content-Length': '0'} 2022-09-03T06:47:30.568 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000064d65cc65771d2e8-006312f882-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.568 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:30.569 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:30.570 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:30.571 DEBUG:boto:Method: HEAD 2022-09-03T06:47:30.571 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:47:30.572 DEBUG:boto:Data: 2022-09-03T06:47:30.572 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.573 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.573 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.574 DEBUG:boto:Params: {} 2022-09-03T06:47:30.574 DEBUG:boto:Token: None 2022-09-03T06:47:30.575 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj85 2022-09-03T06:47:30.576 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HszI8kCHSdVgUREqHUtV+QUEX/M= 2022-09-03T06:47:30.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HszI8kCHSdVgUREqHUtV+QUEX/M=', 'Content-Length': '0'} 2022-09-03T06:47:30.578 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000168419821b3d3e52-006312f882-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.579 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:30.580 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:30.580 DEBUG:boto:Method: HEAD 2022-09-03T06:47:30.581 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:47:30.582 DEBUG:boto:Data: 2022-09-03T06:47:30.582 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.583 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.583 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.584 DEBUG:boto:Params: {} 2022-09-03T06:47:30.585 DEBUG:boto:Token: None 2022-09-03T06:47:30.585 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj85 2022-09-03T06:47:30.586 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HszI8kCHSdVgUREqHUtV+QUEX/M= 2022-09-03T06:47:30.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HszI8kCHSdVgUREqHUtV+QUEX/M=', 'Content-Length': '0'} 2022-09-03T06:47:30.589 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000140550c601df016b-006312f882-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.589 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:47:30.590 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:30.591 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:30.591 DEBUG:boto:Method: GET 2022-09-03T06:47:30.592 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:47:30.592 DEBUG:boto:Data: 2022-09-03T06:47:30.593 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.593 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.594 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.594 DEBUG:boto:Params: {} 2022-09-03T06:47:30.595 DEBUG:boto:Token: None 2022-09-03T06:47:30.596 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj85 2022-09-03T06:47:30.597 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VmjDgSgStJDt8xym0wXVmYv5Sfk= 2022-09-03T06:47:30.597 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VmjDgSgStJDt8xym0wXVmYv5Sfk=', 'Content-Length': '0'} 2022-09-03T06:47:30.600 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e4f66fa44a60ee65-006312f882-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.601 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:30.602 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:30.602 DEBUG:boto:Method: GET 2022-09-03T06:47:30.603 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:47:30.603 DEBUG:boto:Data: 2022-09-03T06:47:30.604 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.605 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.606 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.607 DEBUG:boto:Params: {} 2022-09-03T06:47:30.608 DEBUG:boto:Token: None 2022-09-03T06:47:30.608 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj85 2022-09-03T06:47:30.609 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VmjDgSgStJDt8xym0wXVmYv5Sfk= 2022-09-03T06:47:30.609 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VmjDgSgStJDt8xym0wXVmYv5Sfk=', 'Content-Length': '0'} 2022-09-03T06:47:30.612 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004814a42565f7f3df-006312f882-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.613 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:30.613 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:30.614 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:30.614 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:30.615 DEBUG:boto:Method: GET 2022-09-03T06:47:30.616 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:47:30.616 DEBUG:boto:Data: 2022-09-03T06:47:30.617 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.617 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.618 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.618 DEBUG:boto:Params: {} 2022-09-03T06:47:30.619 DEBUG:boto:Token: None 2022-09-03T06:47:30.620 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/ 2022-09-03T06:47:30.620 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4= 2022-09-03T06:47:30.621 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4=', 'Content-Length': '0'} 2022-09-03T06:47:30.624 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000012225046ccffddc6-006312f882-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.625 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:30.626 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:30.627 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:30.627 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:30.628 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:30.629 DEBUG:boto:Method: GET 2022-09-03T06:47:30.629 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:47:30.630 DEBUG:boto:Data: 2022-09-03T06:47:30.630 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.631 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.631 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.632 DEBUG:boto:Params: {} 2022-09-03T06:47:30.632 DEBUG:boto:Token: None 2022-09-03T06:47:30.633 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/ 2022-09-03T06:47:30.634 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4= 2022-09-03T06:47:30.634 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4=', 'Content-Length': '0'} 2022-09-03T06:47:30.639 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000064b26198e382ef1c-006312f882-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.640 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:30.641 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:47:30.642 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:30.643 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:30.643 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.644 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.644 DEBUG:boto:Method: GET 2022-09-03T06:47:30.645 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.645 DEBUG:boto:Data: 2022-09-03T06:47:30.646 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.647 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.647 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.648 DEBUG:boto:Params: {} 2022-09-03T06:47:30.648 DEBUG:boto:Token: None 2022-09-03T06:47:30.649 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.650 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NERXIs+HrNvg4b6KQ1p2H1lAVuc= 2022-09-03T06:47:30.650 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NERXIs+HrNvg4b6KQ1p2H1lAVuc=', 'Content-Length': '0'} 2022-09-03T06:47:30.653 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000f6f3d3ee27f42b1-006312f882-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.653 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:30.654 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:30.655 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.655 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.656 DEBUG:boto:Method: GET 2022-09-03T06:47:30.657 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.657 DEBUG:boto:Data: 2022-09-03T06:47:30.658 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.658 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.659 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.660 DEBUG:boto:Params: {} 2022-09-03T06:47:30.660 DEBUG:boto:Token: None 2022-09-03T06:47:30.661 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.662 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NERXIs+HrNvg4b6KQ1p2H1lAVuc= 2022-09-03T06:47:30.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NERXIs+HrNvg4b6KQ1p2H1lAVuc=', 'Content-Length': '0'} 2022-09-03T06:47:30.667 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cd07b833b780fc2e-006312f882-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.710 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:30.711 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:30.712 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.712 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.713 DEBUG:boto:Method: HEAD 2022-09-03T06:47:30.714 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.714 DEBUG:boto:Data: 2022-09-03T06:47:30.715 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.722 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.725 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.726 DEBUG:boto:Params: {} 2022-09-03T06:47:30.727 DEBUG:boto:Token: None 2022-09-03T06:47:30.727 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.728 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E8RN255NiOLkv6xUSuEtk4Eik7g= 2022-09-03T06:47:30.729 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E8RN255NiOLkv6xUSuEtk4Eik7g=', 'Content-Length': '0'} 2022-09-03T06:47:30.731 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000880e19c98b47e7d8-006312f882-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.732 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:30.734 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:30.735 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.736 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.737 DEBUG:boto:Method: HEAD 2022-09-03T06:47:30.738 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.739 DEBUG:boto:Data: 2022-09-03T06:47:30.740 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.741 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.742 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.743 DEBUG:boto:Params: {} 2022-09-03T06:47:30.743 DEBUG:boto:Token: None 2022-09-03T06:47:30.745 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.747 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E8RN255NiOLkv6xUSuEtk4Eik7g= 2022-09-03T06:47:30.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E8RN255NiOLkv6xUSuEtk4Eik7g=', 'Content-Length': '0'} 2022-09-03T06:47:30.751 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000339ca4fd93b62203-006312f882-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.752 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:47:30.753 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:30.754 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:30.754 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.755 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.756 DEBUG:boto:Method: GET 2022-09-03T06:47:30.757 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.758 DEBUG:boto:Data: 2022-09-03T06:47:30.759 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.760 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.761 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.762 DEBUG:boto:Params: {} 2022-09-03T06:47:30.764 DEBUG:boto:Token: None 2022-09-03T06:47:30.765 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.766 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NERXIs+HrNvg4b6KQ1p2H1lAVuc= 2022-09-03T06:47:30.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NERXIs+HrNvg4b6KQ1p2H1lAVuc=', 'Content-Length': '0'} 2022-09-03T06:47:30.770 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003cea56fa063fd40b-006312f882-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.820 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:30.822 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:30.823 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.824 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.825 DEBUG:boto:Method: GET 2022-09-03T06:47:30.826 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.827 DEBUG:boto:Data: 2022-09-03T06:47:30.828 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.829 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.830 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.831 DEBUG:boto:Params: {} 2022-09-03T06:47:30.831 DEBUG:boto:Token: None 2022-09-03T06:47:30.832 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:30.833 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NERXIs+HrNvg4b6KQ1p2H1lAVuc= 2022-09-03T06:47:30.834 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NERXIs+HrNvg4b6KQ1p2H1lAVuc=', 'Content-Length': '0'} 2022-09-03T06:47:30.837 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000046c83cbdecf8181f-006312f882-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.838 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:30.839 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:30.840 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:30.841 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:30.842 DEBUG:boto:Method: GET 2022-09-03T06:47:30.843 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:47:30.844 DEBUG:boto:Data: 2022-09-03T06:47:30.845 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.846 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.847 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.848 DEBUG:boto:Params: {} 2022-09-03T06:47:30.849 DEBUG:boto:Token: None 2022-09-03T06:47:30.851 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/ 2022-09-03T06:47:30.852 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4= 2022-09-03T06:47:30.853 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4=', 'Content-Length': '0'} 2022-09-03T06:47:30.856 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001796619145660267-006312f882-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.857 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:30.858 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:30.859 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:30.859 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:30.860 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:30.860 DEBUG:boto:Method: GET 2022-09-03T06:47:30.861 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:47:30.862 DEBUG:boto:Data: 2022-09-03T06:47:30.864 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.865 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.865 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.866 DEBUG:boto:Params: {} 2022-09-03T06:47:30.867 DEBUG:boto:Token: None 2022-09-03T06:47:30.867 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/ 2022-09-03T06:47:30.868 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4= 2022-09-03T06:47:30.869 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4=', 'Content-Length': '0'} 2022-09-03T06:47:30.873 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000014031480dc624677-006312f882-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.874 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:30.876 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:30.876 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:30.877 DEBUG:boto:Method: HEAD 2022-09-03T06:47:30.877 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:47:30.878 DEBUG:boto:Data: 2022-09-03T06:47:30.878 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.879 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.879 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.880 DEBUG:boto:Params: {} 2022-09-03T06:47:30.880 DEBUG:boto:Token: None 2022-09-03T06:47:30.881 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj87 2022-09-03T06:47:30.882 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:idrXBR7JGU24tFaMHQfyFK+Nwjw= 2022-09-03T06:47:30.882 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:idrXBR7JGU24tFaMHQfyFK+Nwjw=', 'Content-Length': '0'} 2022-09-03T06:47:30.884 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006aa8f5f8cab6f8fa-006312f882-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.885 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:30.886 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:30.886 DEBUG:boto:Method: HEAD 2022-09-03T06:47:30.887 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:47:30.887 DEBUG:boto:Data: 2022-09-03T06:47:30.888 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.889 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.889 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.890 DEBUG:boto:Params: {} 2022-09-03T06:47:30.890 DEBUG:boto:Token: None 2022-09-03T06:47:30.891 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj87 2022-09-03T06:47:30.891 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:idrXBR7JGU24tFaMHQfyFK+Nwjw= 2022-09-03T06:47:30.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:idrXBR7JGU24tFaMHQfyFK+Nwjw=', 'Content-Length': '0'} 2022-09-03T06:47:30.894 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000078f2c89eb2d91f44-006312f882-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.895 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:47:30.896 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:30.897 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:30.897 DEBUG:boto:Method: GET 2022-09-03T06:47:30.898 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:47:30.899 DEBUG:boto:Data: 2022-09-03T06:47:30.899 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.900 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.900 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.901 DEBUG:boto:Params: {} 2022-09-03T06:47:30.902 DEBUG:boto:Token: None 2022-09-03T06:47:30.902 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj87 2022-09-03T06:47:30.903 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ByPk39wqynTefTsNArDu4fXeLAs= 2022-09-03T06:47:30.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ByPk39wqynTefTsNArDu4fXeLAs=', 'Content-Length': '0'} 2022-09-03T06:47:30.906 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001adb840918056472-006312f882-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.907 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:30.907 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:30.908 DEBUG:boto:Method: GET 2022-09-03T06:47:30.908 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:47:30.909 DEBUG:boto:Data: 2022-09-03T06:47:30.909 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.910 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.911 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.911 DEBUG:boto:Params: {} 2022-09-03T06:47:30.912 DEBUG:boto:Token: None 2022-09-03T06:47:30.912 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj87 2022-09-03T06:47:30.913 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ByPk39wqynTefTsNArDu4fXeLAs= 2022-09-03T06:47:30.914 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ByPk39wqynTefTsNArDu4fXeLAs=', 'Content-Length': '0'} 2022-09-03T06:47:30.916 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a028576af06775c0-006312f882-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.958 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:30.959 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:30.959 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:30.960 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:30.960 DEBUG:boto:Method: GET 2022-09-03T06:47:30.961 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:47:30.961 DEBUG:boto:Data: 2022-09-03T06:47:30.962 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.963 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.963 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.964 DEBUG:boto:Params: {} 2022-09-03T06:47:30.965 DEBUG:boto:Token: None 2022-09-03T06:47:30.965 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/ 2022-09-03T06:47:30.966 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4= 2022-09-03T06:47:30.966 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4=', 'Content-Length': '0'} 2022-09-03T06:47:30.969 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f909a9efeec4c0f5-006312f882-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.970 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:30.971 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:30.971 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:30.972 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:30.973 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:30.973 DEBUG:boto:Method: GET 2022-09-03T06:47:30.974 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:47:30.975 DEBUG:boto:Data: 2022-09-03T06:47:30.976 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.977 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:30.977 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.978 DEBUG:boto:Params: {} 2022-09-03T06:47:30.979 DEBUG:boto:Token: None 2022-09-03T06:47:30.980 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/ 2022-09-03T06:47:30.981 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4= 2022-09-03T06:47:30.981 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fAaeOLMRzCfS8cVmOIqeY+sVwC4=', 'Content-Length': '0'} 2022-09-03T06:47:30.988 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000924301d4115064d9-006312f882-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:30.990 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:30.991 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:47:30.992 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:30.993 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:30.994 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:30.994 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:30.995 DEBUG:boto:Method: GET 2022-09-03T06:47:30.995 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:30.996 DEBUG:boto:Data: 2022-09-03T06:47:30.996 DEBUG:boto:Headers: {} 2022-09-03T06:47:30.997 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:30.997 DEBUG:boto:Port: 8000 2022-09-03T06:47:30.998 DEBUG:boto:Params: {} 2022-09-03T06:47:30.999 DEBUG:boto:Token: None 2022-09-03T06:47:30.999 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:30 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:31.000 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5wiGb9WxxuUhldSzZQuCXAKqcD8= 2022-09-03T06:47:31.001 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5wiGb9WxxuUhldSzZQuCXAKqcD8=', 'Content-Length': '0'} 2022-09-03T06:47:31.004 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d727042b286ce441-006312f883-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.005 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:31.006 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:31.007 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:31.008 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:31.009 DEBUG:boto:Method: GET 2022-09-03T06:47:31.010 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:31.011 DEBUG:boto:Data: 2022-09-03T06:47:31.012 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.013 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.013 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.014 DEBUG:boto:Params: {} 2022-09-03T06:47:31.015 DEBUG:boto:Token: None 2022-09-03T06:47:31.016 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:31.016 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QgYcEIO7PZDPRm7ym1imbRR8eL4= 2022-09-03T06:47:31.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QgYcEIO7PZDPRm7ym1imbRR8eL4=', 'Content-Length': '0'} 2022-09-03T06:47:31.019 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c0f4c5b407ef9871-006312f883-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.021 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:31.021 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:31.022 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:31.023 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:31.023 DEBUG:boto:Method: HEAD 2022-09-03T06:47:31.024 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:31.025 DEBUG:boto:Data: 2022-09-03T06:47:31.027 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.028 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:31.028 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.029 DEBUG:boto:Params: {} 2022-09-03T06:47:31.029 DEBUG:boto:Token: None 2022-09-03T06:47:31.030 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:31.031 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cdUTayq5TGkPrx3+RxYUBhi2QXA= 2022-09-03T06:47:31.031 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cdUTayq5TGkPrx3+RxYUBhi2QXA=', 'Content-Length': '0'} 2022-09-03T06:47:31.033 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a968016f30d8b0a-006312f883-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.034 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:31.035 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:31.035 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:31.036 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:31.036 DEBUG:boto:Method: HEAD 2022-09-03T06:47:31.037 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:31.037 DEBUG:boto:Data: 2022-09-03T06:47:31.038 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.038 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.039 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.039 DEBUG:boto:Params: {} 2022-09-03T06:47:31.040 DEBUG:boto:Token: None 2022-09-03T06:47:31.040 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:31.041 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cdUTayq5TGkPrx3+RxYUBhi2QXA= 2022-09-03T06:47:31.041 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cdUTayq5TGkPrx3+RxYUBhi2QXA=', 'Content-Length': '0'} 2022-09-03T06:47:31.043 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a272b731a8e9fca0-006312f883-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.044 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:47:31.045 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:31.045 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:31.046 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:31.046 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:31.047 DEBUG:boto:Method: GET 2022-09-03T06:47:31.048 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:31.048 DEBUG:boto:Data: 2022-09-03T06:47:31.049 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.049 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:31.050 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.050 DEBUG:boto:Params: {} 2022-09-03T06:47:31.051 DEBUG:boto:Token: None 2022-09-03T06:47:31.052 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:31.052 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QgYcEIO7PZDPRm7ym1imbRR8eL4= 2022-09-03T06:47:31.053 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QgYcEIO7PZDPRm7ym1imbRR8eL4=', 'Content-Length': '0'} 2022-09-03T06:47:31.055 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e35ae5f5bbf6d4ce-006312f883-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.056 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:31.057 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:31.057 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:31.058 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:31.059 DEBUG:boto:Method: GET 2022-09-03T06:47:31.059 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:31.060 DEBUG:boto:Data: 2022-09-03T06:47:31.060 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.061 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.061 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.062 DEBUG:boto:Params: {} 2022-09-03T06:47:31.063 DEBUG:boto:Token: None 2022-09-03T06:47:31.063 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:31.064 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QgYcEIO7PZDPRm7ym1imbRR8eL4= 2022-09-03T06:47:31.065 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QgYcEIO7PZDPRm7ym1imbRR8eL4=', 'Content-Length': '0'} 2022-09-03T06:47:31.067 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002dbd9904be1c3b03-006312f883-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.114 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:31.115 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:31.116 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:31.116 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:31.117 DEBUG:boto:Method: GET 2022-09-03T06:47:31.117 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:47:31.118 DEBUG:boto:Data: 2022-09-03T06:47:31.119 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.119 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:31.120 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.121 DEBUG:boto:Params: {} 2022-09-03T06:47:31.121 DEBUG:boto:Token: None 2022-09-03T06:47:31.122 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/ 2022-09-03T06:47:31.123 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8= 2022-09-03T06:47:31.123 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8=', 'Content-Length': '0'} 2022-09-03T06:47:31.126 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007ecd32eda8624d1b-006312f883-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.127 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:31.128 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:31.128 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:31.129 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:31.129 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:31.130 DEBUG:boto:Method: GET 2022-09-03T06:47:31.131 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:47:31.132 DEBUG:boto:Data: 2022-09-03T06:47:31.132 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.133 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.133 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.134 DEBUG:boto:Params: {} 2022-09-03T06:47:31.135 DEBUG:boto:Token: None 2022-09-03T06:47:31.135 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/ 2022-09-03T06:47:31.136 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8= 2022-09-03T06:47:31.137 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8=', 'Content-Length': '0'} 2022-09-03T06:47:31.141 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a2859943d80e8be7-006312f883-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.142 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:31.143 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:31.143 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:31.144 DEBUG:boto:Method: HEAD 2022-09-03T06:47:31.144 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:47:31.145 DEBUG:boto:Data: 2022-09-03T06:47:31.145 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.146 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:31.147 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.147 DEBUG:boto:Params: {} 2022-09-03T06:47:31.148 DEBUG:boto:Token: None 2022-09-03T06:47:31.148 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj92 2022-09-03T06:47:31.149 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QV/DZobLSqxfr34YFcqpCmKWiUY= 2022-09-03T06:47:31.150 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QV/DZobLSqxfr34YFcqpCmKWiUY=', 'Content-Length': '0'} 2022-09-03T06:47:31.152 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ff605ca52147506d-006312f883-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.153 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:31.154 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:31.154 DEBUG:boto:Method: HEAD 2022-09-03T06:47:31.155 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:47:31.155 DEBUG:boto:Data: 2022-09-03T06:47:31.156 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.157 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.157 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.158 DEBUG:boto:Params: {} 2022-09-03T06:47:31.158 DEBUG:boto:Token: None 2022-09-03T06:47:31.159 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj92 2022-09-03T06:47:31.159 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QV/DZobLSqxfr34YFcqpCmKWiUY= 2022-09-03T06:47:31.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QV/DZobLSqxfr34YFcqpCmKWiUY=', 'Content-Length': '0'} 2022-09-03T06:47:31.163 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008c2e1a701cdd4fb4-006312f883-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.163 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:47:31.164 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:31.164 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:31.165 DEBUG:boto:Method: GET 2022-09-03T06:47:31.166 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:47:31.166 DEBUG:boto:Data: 2022-09-03T06:47:31.167 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.167 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:31.168 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.168 DEBUG:boto:Params: {} 2022-09-03T06:47:31.169 DEBUG:boto:Token: None 2022-09-03T06:47:31.170 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj92 2022-09-03T06:47:31.170 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jBKhMH1Xuqz3b6F64sOcU9ekx1g= 2022-09-03T06:47:31.171 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jBKhMH1Xuqz3b6F64sOcU9ekx1g=', 'Content-Length': '0'} 2022-09-03T06:47:31.173 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000768ce28fdcaaef78-006312f883-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.174 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:31.175 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:31.176 DEBUG:boto:Method: GET 2022-09-03T06:47:31.176 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:47:31.177 DEBUG:boto:Data: 2022-09-03T06:47:31.178 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.178 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.179 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.179 DEBUG:boto:Params: {} 2022-09-03T06:47:31.180 DEBUG:boto:Token: None 2022-09-03T06:47:31.181 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj92 2022-09-03T06:47:31.182 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jBKhMH1Xuqz3b6F64sOcU9ekx1g= 2022-09-03T06:47:31.182 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jBKhMH1Xuqz3b6F64sOcU9ekx1g=', 'Content-Length': '0'} 2022-09-03T06:47:31.185 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a90277369dba13cd-006312f883-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.230 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:31.231 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:31.232 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:31.233 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:31.233 DEBUG:boto:Method: GET 2022-09-03T06:47:31.234 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:47:31.234 DEBUG:boto:Data: 2022-09-03T06:47:31.235 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.235 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:31.236 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.236 DEBUG:boto:Params: {} 2022-09-03T06:47:31.237 DEBUG:boto:Token: None 2022-09-03T06:47:31.238 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/ 2022-09-03T06:47:31.238 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8= 2022-09-03T06:47:31.239 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8=', 'Content-Length': '0'} 2022-09-03T06:47:31.241 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000925c49a5885f5c02-006312f883-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.242 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:31.243 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:31.244 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:31.244 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:31.245 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:31.245 DEBUG:boto:Method: GET 2022-09-03T06:47:31.246 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:47:31.246 DEBUG:boto:Data: 2022-09-03T06:47:31.247 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.247 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.248 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.248 DEBUG:boto:Params: {} 2022-09-03T06:47:31.249 DEBUG:boto:Token: None 2022-09-03T06:47:31.250 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/ 2022-09-03T06:47:31.250 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8= 2022-09-03T06:47:31.251 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8=', 'Content-Length': '0'} 2022-09-03T06:47:31.255 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002698e366e48fda03-006312f883-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.256 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:31.258 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2022-09-03T06:47:31.259 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:31.260 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:31.261 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.261 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.262 DEBUG:boto:Method: GET 2022-09-03T06:47:31.263 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.264 DEBUG:boto:Data: 2022-09-03T06:47:31.265 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.266 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:31.267 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.267 DEBUG:boto:Params: {} 2022-09-03T06:47:31.268 DEBUG:boto:Token: None 2022-09-03T06:47:31.268 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.269 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TgJGV0NoLDe0HeKV3BCNjlXmqP4= 2022-09-03T06:47:31.270 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TgJGV0NoLDe0HeKV3BCNjlXmqP4=', 'Content-Length': '0'} 2022-09-03T06:47:31.272 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000360deb933f934de1-006312f883-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.273 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:31.273 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:31.274 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.274 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.275 DEBUG:boto:Method: GET 2022-09-03T06:47:31.275 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.276 DEBUG:boto:Data: 2022-09-03T06:47:31.276 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.277 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.278 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.278 DEBUG:boto:Params: {} 2022-09-03T06:47:31.279 DEBUG:boto:Token: None 2022-09-03T06:47:31.280 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.281 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TgJGV0NoLDe0HeKV3BCNjlXmqP4= 2022-09-03T06:47:31.282 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TgJGV0NoLDe0HeKV3BCNjlXmqP4=', 'Content-Length': '0'} 2022-09-03T06:47:31.285 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f3c65767f613483f-006312f883-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.286 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:31.287 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:31.287 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.288 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.288 DEBUG:boto:Method: HEAD 2022-09-03T06:47:31.289 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.289 DEBUG:boto:Data: 2022-09-03T06:47:31.290 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.291 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:31.291 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.292 DEBUG:boto:Params: {} 2022-09-03T06:47:31.292 DEBUG:boto:Token: None 2022-09-03T06:47:31.293 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.294 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L4nxqkIFG4DTClgfc/DN6bzKZTw= 2022-09-03T06:47:31.295 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L4nxqkIFG4DTClgfc/DN6bzKZTw=', 'Content-Length': '0'} 2022-09-03T06:47:31.298 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000886ebfa09c14dd61-006312f883-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.299 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:31.300 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:31.301 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.302 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.302 DEBUG:boto:Method: HEAD 2022-09-03T06:47:31.303 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.303 DEBUG:boto:Data: 2022-09-03T06:47:31.304 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.305 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.306 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.307 DEBUG:boto:Params: {} 2022-09-03T06:47:31.308 DEBUG:boto:Token: None 2022-09-03T06:47:31.309 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.310 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L4nxqkIFG4DTClgfc/DN6bzKZTw= 2022-09-03T06:47:31.311 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L4nxqkIFG4DTClgfc/DN6bzKZTw=', 'Content-Length': '0'} 2022-09-03T06:47:31.314 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bba0261a2fcd60ce-006312f883-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.315 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2022-09-03T06:47:31.316 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:31.317 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:31.318 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.319 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.319 DEBUG:boto:Method: GET 2022-09-03T06:47:31.320 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.321 DEBUG:boto:Data: 2022-09-03T06:47:31.321 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.322 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:31.322 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.323 DEBUG:boto:Params: {} 2022-09-03T06:47:31.324 DEBUG:boto:Token: None 2022-09-03T06:47:31.324 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.325 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TgJGV0NoLDe0HeKV3BCNjlXmqP4= 2022-09-03T06:47:31.326 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TgJGV0NoLDe0HeKV3BCNjlXmqP4=', 'Content-Length': '0'} 2022-09-03T06:47:31.328 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b9f4ace775373d59-006312f883-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.329 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:31.329 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:31.330 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.330 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.331 DEBUG:boto:Method: GET 2022-09-03T06:47:31.331 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.332 DEBUG:boto:Data: 2022-09-03T06:47:31.333 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.333 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.334 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.335 DEBUG:boto:Params: {} 2022-09-03T06:47:31.335 DEBUG:boto:Token: None 2022-09-03T06:47:31.336 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:31.337 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TgJGV0NoLDe0HeKV3BCNjlXmqP4= 2022-09-03T06:47:31.337 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TgJGV0NoLDe0HeKV3BCNjlXmqP4=', 'Content-Length': '0'} 2022-09-03T06:47:31.339 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c9cd5f786d4e5a2-006312f883-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.340 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:31.341 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:31.342 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:31.342 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:31.343 DEBUG:boto:Method: GET 2022-09-03T06:47:31.343 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:47:31.344 DEBUG:boto:Data: 2022-09-03T06:47:31.344 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.345 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:31.346 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.346 DEBUG:boto:Params: {} 2022-09-03T06:47:31.347 DEBUG:boto:Token: None 2022-09-03T06:47:31.347 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/ 2022-09-03T06:47:31.348 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8= 2022-09-03T06:47:31.349 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8=', 'Content-Length': '0'} 2022-09-03T06:47:31.351 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000007c0178d89948100-006312f883-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.352 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:31.353 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:31.354 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:31.354 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:31.355 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:31.356 DEBUG:boto:Method: GET 2022-09-03T06:47:31.356 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:47:31.357 DEBUG:boto:Data: 2022-09-03T06:47:31.357 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.358 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.359 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.360 DEBUG:boto:Params: {} 2022-09-03T06:47:31.361 DEBUG:boto:Token: None 2022-09-03T06:47:31.362 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/ 2022-09-03T06:47:31.363 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8= 2022-09-03T06:47:31.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8=', 'Content-Length': '0'} 2022-09-03T06:47:31.369 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000056cef974817c18d0-006312f883-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.370 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:31.372 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:31.373 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:31.373 DEBUG:boto:Method: HEAD 2022-09-03T06:47:31.374 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:47:31.375 DEBUG:boto:Data: 2022-09-03T06:47:31.376 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.377 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:31.378 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.378 DEBUG:boto:Params: {} 2022-09-03T06:47:31.379 DEBUG:boto:Token: None 2022-09-03T06:47:31.380 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj93 2022-09-03T06:47:31.381 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6eik+200zx2LIjPz8FEGN37u+8A= 2022-09-03T06:47:31.382 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6eik+200zx2LIjPz8FEGN37u+8A=', 'Content-Length': '0'} 2022-09-03T06:47:31.384 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f669fa2d75ab688b-006312f883-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.385 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:31.386 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:31.387 DEBUG:boto:Method: HEAD 2022-09-03T06:47:31.388 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:47:31.389 DEBUG:boto:Data: 2022-09-03T06:47:31.390 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.391 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.392 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.393 DEBUG:boto:Params: {} 2022-09-03T06:47:31.394 DEBUG:boto:Token: None 2022-09-03T06:47:31.395 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj93 2022-09-03T06:47:31.396 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6eik+200zx2LIjPz8FEGN37u+8A= 2022-09-03T06:47:31.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6eik+200zx2LIjPz8FEGN37u+8A=', 'Content-Length': '0'} 2022-09-03T06:47:31.399 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000030ee9ba83ea4432-006312f883-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.399 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2022-09-03T06:47:31.400 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:31.401 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:31.401 DEBUG:boto:Method: GET 2022-09-03T06:47:31.402 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:47:31.402 DEBUG:boto:Data: 2022-09-03T06:47:31.403 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.403 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:31.404 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.404 DEBUG:boto:Params: {} 2022-09-03T06:47:31.405 DEBUG:boto:Token: None 2022-09-03T06:47:31.406 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj93 2022-09-03T06:47:31.407 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OqJ4pP5P8zCVABTe9OttWLKWUOQ= 2022-09-03T06:47:31.408 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OqJ4pP5P8zCVABTe9OttWLKWUOQ=', 'Content-Length': '0'} 2022-09-03T06:47:31.409 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a19d1aae1c137342-006312f883-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.456 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:31.458 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:31.459 DEBUG:boto:Method: GET 2022-09-03T06:47:31.460 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:47:31.461 DEBUG:boto:Data: 2022-09-03T06:47:31.462 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.463 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.464 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.465 DEBUG:boto:Params: {} 2022-09-03T06:47:31.466 DEBUG:boto:Token: None 2022-09-03T06:47:31.467 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj93 2022-09-03T06:47:31.468 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OqJ4pP5P8zCVABTe9OttWLKWUOQ= 2022-09-03T06:47:31.469 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OqJ4pP5P8zCVABTe9OttWLKWUOQ=', 'Content-Length': '0'} 2022-09-03T06:47:31.476 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009b7482748e71be2c-006312f883-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.477 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:31.478 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:31.479 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:31.479 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:31.480 DEBUG:boto:Method: GET 2022-09-03T06:47:31.480 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:47:31.481 DEBUG:boto:Data: 2022-09-03T06:47:31.481 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.482 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:31.483 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.483 DEBUG:boto:Params: {} 2022-09-03T06:47:31.484 DEBUG:boto:Token: None 2022-09-03T06:47:31.485 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/ 2022-09-03T06:47:31.486 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8= 2022-09-03T06:47:31.487 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8=', 'Content-Length': '0'} 2022-09-03T06:47:31.490 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000973e8356b549e2d4-006312f883-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.491 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:31.492 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:31.493 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:31.494 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:31.495 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:31.496 DEBUG:boto:Method: GET 2022-09-03T06:47:31.496 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:47:31.497 DEBUG:boto:Data: 2022-09-03T06:47:31.497 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.498 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.499 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.499 DEBUG:boto:Params: {} 2022-09-03T06:47:31.500 DEBUG:boto:Token: None 2022-09-03T06:47:31.500 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/ 2022-09-03T06:47:31.501 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8= 2022-09-03T06:47:31.502 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8=', 'Content-Length': '0'} 2022-09-03T06:47:31.512 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b7e816646c8eb061-006312f883-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.513 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:31.513 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:47:31.514 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:31.515 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:31.515 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.516 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.516 DEBUG:boto:Method: GET 2022-09-03T06:47:31.517 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.517 DEBUG:boto:Data: 2022-09-03T06:47:31.518 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.519 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:31.519 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.520 DEBUG:boto:Params: {} 2022-09-03T06:47:31.521 DEBUG:boto:Token: None 2022-09-03T06:47:31.521 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.522 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UtlQA9WV9P+1em0UDPg+8QYSJJU= 2022-09-03T06:47:31.523 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UtlQA9WV9P+1em0UDPg+8QYSJJU=', 'Content-Length': '0'} 2022-09-03T06:47:31.525 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000043b0db5f1165e99-006312f883-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.526 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:31.526 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:31.527 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.528 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.528 DEBUG:boto:Method: GET 2022-09-03T06:47:31.529 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.529 DEBUG:boto:Data: 2022-09-03T06:47:31.530 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.530 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.531 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.532 DEBUG:boto:Params: {} 2022-09-03T06:47:31.532 DEBUG:boto:Token: None 2022-09-03T06:47:31.533 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.534 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UtlQA9WV9P+1em0UDPg+8QYSJJU= 2022-09-03T06:47:31.535 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UtlQA9WV9P+1em0UDPg+8QYSJJU=', 'Content-Length': '0'} 2022-09-03T06:47:31.537 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000222296c4112686f7-006312f883-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.538 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:31.538 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:31.539 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.540 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.540 DEBUG:boto:Method: HEAD 2022-09-03T06:47:31.541 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.541 DEBUG:boto:Data: 2022-09-03T06:47:31.542 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.542 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:31.543 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.544 DEBUG:boto:Params: {} 2022-09-03T06:47:31.544 DEBUG:boto:Token: None 2022-09-03T06:47:31.545 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.546 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZZVmLBDg6t7FkpgG8vWBs5eTLHs= 2022-09-03T06:47:31.547 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZZVmLBDg6t7FkpgG8vWBs5eTLHs=', 'Content-Length': '0'} 2022-09-03T06:47:31.549 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006a68b23e557af54f-006312f883-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.550 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:31.551 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:31.552 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.553 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.554 DEBUG:boto:Method: HEAD 2022-09-03T06:47:31.555 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.556 DEBUG:boto:Data: 2022-09-03T06:47:31.557 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.559 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.560 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.561 DEBUG:boto:Params: {} 2022-09-03T06:47:31.562 DEBUG:boto:Token: None 2022-09-03T06:47:31.563 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.564 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZZVmLBDg6t7FkpgG8vWBs5eTLHs= 2022-09-03T06:47:31.565 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZZVmLBDg6t7FkpgG8vWBs5eTLHs=', 'Content-Length': '0'} 2022-09-03T06:47:31.567 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a50e128a6990b061-006312f883-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.568 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:47:31.569 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:31.570 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:31.571 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.572 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.573 DEBUG:boto:Method: GET 2022-09-03T06:47:31.574 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.575 DEBUG:boto:Data: 2022-09-03T06:47:31.576 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.577 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:31.578 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.579 DEBUG:boto:Params: {} 2022-09-03T06:47:31.579 DEBUG:boto:Token: None 2022-09-03T06:47:31.580 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.581 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UtlQA9WV9P+1em0UDPg+8QYSJJU= 2022-09-03T06:47:31.581 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UtlQA9WV9P+1em0UDPg+8QYSJJU=', 'Content-Length': '0'} 2022-09-03T06:47:31.584 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad275733af014e2f-006312f883-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.585 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:31.586 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:31.587 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.590 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.591 DEBUG:boto:Method: GET 2022-09-03T06:47:31.591 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.592 DEBUG:boto:Data: 2022-09-03T06:47:31.592 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.593 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.593 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.594 DEBUG:boto:Params: {} 2022-09-03T06:47:31.595 DEBUG:boto:Token: None 2022-09-03T06:47:31.595 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:31.596 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UtlQA9WV9P+1em0UDPg+8QYSJJU= 2022-09-03T06:47:31.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UtlQA9WV9P+1em0UDPg+8QYSJJU=', 'Content-Length': '0'} 2022-09-03T06:47:31.599 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eabffdc89a60b862-006312f883-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.600 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:31.601 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:31.601 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:31.602 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:31.602 DEBUG:boto:Method: GET 2022-09-03T06:47:31.603 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:47:31.604 DEBUG:boto:Data: 2022-09-03T06:47:31.605 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.606 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:31.606 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.607 DEBUG:boto:Params: {} 2022-09-03T06:47:31.608 DEBUG:boto:Token: None 2022-09-03T06:47:31.608 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/ 2022-09-03T06:47:31.609 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8= 2022-09-03T06:47:31.610 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8=', 'Content-Length': '0'} 2022-09-03T06:47:31.614 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009d1f481e6cf4e106-006312f883-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.615 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:31.617 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:31.617 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:31.618 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:31.619 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:31.620 DEBUG:boto:Method: GET 2022-09-03T06:47:31.622 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:47:31.622 DEBUG:boto:Data: 2022-09-03T06:47:31.623 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.624 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.624 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.625 DEBUG:boto:Params: {} 2022-09-03T06:47:31.626 DEBUG:boto:Token: None 2022-09-03T06:47:31.626 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/ 2022-09-03T06:47:31.627 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8= 2022-09-03T06:47:31.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8=', 'Content-Length': '0'} 2022-09-03T06:47:31.632 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008253a6347a3d39e6-006312f883-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.677 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:31.679 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:31.680 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:31.681 DEBUG:boto:Method: HEAD 2022-09-03T06:47:31.682 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:47:31.686 DEBUG:boto:Data: 2022-09-03T06:47:31.687 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.687 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:31.688 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.689 DEBUG:boto:Params: {} 2022-09-03T06:47:31.689 DEBUG:boto:Token: None 2022-09-03T06:47:31.690 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj95 2022-09-03T06:47:31.691 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HR2HDFyNRIftrNOwd/oTJRlHhLQ= 2022-09-03T06:47:31.691 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HR2HDFyNRIftrNOwd/oTJRlHhLQ=', 'Content-Length': '0'} 2022-09-03T06:47:31.695 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000000419a3524dff43b-006312f883-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.698 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:31.699 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:31.699 DEBUG:boto:Method: HEAD 2022-09-03T06:47:31.700 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:47:31.701 DEBUG:boto:Data: 2022-09-03T06:47:31.702 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.703 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.704 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.705 DEBUG:boto:Params: {} 2022-09-03T06:47:31.707 DEBUG:boto:Token: None 2022-09-03T06:47:31.708 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj95 2022-09-03T06:47:31.709 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HR2HDFyNRIftrNOwd/oTJRlHhLQ= 2022-09-03T06:47:31.710 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HR2HDFyNRIftrNOwd/oTJRlHhLQ=', 'Content-Length': '0'} 2022-09-03T06:47:31.714 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e0411d8ca4f52846-006312f883-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.715 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:47:31.716 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:31.717 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:31.719 DEBUG:boto:Method: GET 2022-09-03T06:47:31.720 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:47:31.721 DEBUG:boto:Data: 2022-09-03T06:47:31.722 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.723 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:31.724 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.725 DEBUG:boto:Params: {} 2022-09-03T06:47:31.726 DEBUG:boto:Token: None 2022-09-03T06:47:31.727 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj95 2022-09-03T06:47:31.728 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y6p5hiJAxB41+TDwNFcOPRDooO4= 2022-09-03T06:47:31.729 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y6p5hiJAxB41+TDwNFcOPRDooO4=', 'Content-Length': '0'} 2022-09-03T06:47:31.732 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000007f89b1a053b6d57-006312f883-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.779 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:31.780 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:31.782 DEBUG:boto:Method: GET 2022-09-03T06:47:31.784 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:47:31.785 DEBUG:boto:Data: 2022-09-03T06:47:31.787 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.787 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.788 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.789 DEBUG:boto:Params: {} 2022-09-03T06:47:31.790 DEBUG:boto:Token: None 2022-09-03T06:47:31.790 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj95 2022-09-03T06:47:31.793 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y6p5hiJAxB41+TDwNFcOPRDooO4= 2022-09-03T06:47:31.793 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y6p5hiJAxB41+TDwNFcOPRDooO4=', 'Content-Length': '0'} 2022-09-03T06:47:31.796 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003aa80f4bf4690f6e-006312f883-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.798 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:31.799 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:31.800 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:31.801 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:31.801 DEBUG:boto:Method: GET 2022-09-03T06:47:31.802 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:47:31.802 DEBUG:boto:Data: 2022-09-03T06:47:31.803 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.803 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:31.804 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.804 DEBUG:boto:Params: {} 2022-09-03T06:47:31.805 DEBUG:boto:Token: None 2022-09-03T06:47:31.806 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/ 2022-09-03T06:47:31.806 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8= 2022-09-03T06:47:31.807 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8=', 'Content-Length': '0'} 2022-09-03T06:47:31.811 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ddc97948b550de16-006312f883-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.812 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:31.813 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:31.813 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:31.814 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:31.815 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:31.815 DEBUG:boto:Method: GET 2022-09-03T06:47:31.816 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:47:31.816 DEBUG:boto:Data: 2022-09-03T06:47:31.817 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.817 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.818 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.819 DEBUG:boto:Params: {} 2022-09-03T06:47:31.820 DEBUG:boto:Token: None 2022-09-03T06:47:31.821 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/ 2022-09-03T06:47:31.823 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8= 2022-09-03T06:47:31.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vJyElEPRqPP82J4MvLMboM667z8=', 'Content-Length': '0'} 2022-09-03T06:47:31.829 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002602545b2dd79d96-006312f883-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.829 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:31.831 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-75 zones={test-zone2, test-zone1} 2022-09-03T06:47:31.831 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-75 zones={test-zone3, test-zone1} 2022-09-03T06:47:31.832 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:31.833 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:31.833 DEBUG:boto:Method: HEAD 2022-09-03T06:47:31.834 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:47:31.835 DEBUG:boto:Data: 2022-09-03T06:47:31.835 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.836 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:31.836 DEBUG:boto:Port: 8001 2022-09-03T06:47:31.837 DEBUG:boto:Params: {} 2022-09-03T06:47:31.838 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T06:47:31.839 DEBUG:boto:Token: None 2022-09-03T06:47:31.839 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/ 2022-09-03T06:47:31.840 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CHUFFQ4pAh0/V8t02c1N4FplMWU= 2022-09-03T06:47:31.841 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CHUFFQ4pAh0/V8t02c1N4FplMWU=', 'Content-Length': '0'} 2022-09-03T06:47:31.848 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '30'), ('X-RGW-Bytes-Used', '90'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000d2baf1a293b3aac-006312f883-1851-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.849 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:31.850 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:31.850 DEBUG:boto:Method: HEAD 2022-09-03T06:47:31.851 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:47:31.851 DEBUG:boto:Data: 2022-09-03T06:47:31.852 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.853 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.854 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.855 DEBUG:boto:Params: {} 2022-09-03T06:47:31.856 DEBUG:boto:Token: None 2022-09-03T06:47:31.857 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/ 2022-09-03T06:47:31.858 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CHUFFQ4pAh0/V8t02c1N4FplMWU= 2022-09-03T06:47:31.859 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CHUFFQ4pAh0/V8t02c1N4FplMWU=', 'Content-Length': '0'} 2022-09-03T06:47:31.864 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '30'), ('X-RGW-Bytes-Used', '90'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000012adb14e47f0c100-006312f883-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.864 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:47:31.865 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:31.866 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:31.867 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:31.867 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:31.868 DEBUG:boto:Method: GET 2022-09-03T06:47:31.869 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:31.870 DEBUG:boto:Data: 2022-09-03T06:47:31.871 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.871 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:31.872 DEBUG:boto:Port: 8001 2022-09-03T06:47:31.873 DEBUG:boto:Params: {} 2022-09-03T06:47:31.874 DEBUG:boto:Token: None 2022-09-03T06:47:31.876 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/?versions 2022-09-03T06:47:31.877 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hqOyDuYrKP02fgviil0DDCc9MkU= 2022-09-03T06:47:31.878 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hqOyDuYrKP02fgviil0DDCc9MkU=', 'Content-Length': '0'} 2022-09-03T06:47:31.881 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d7ffab394489592b-006312f883-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.883 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:31.886 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:47:31.886 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:47:31.887 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2022-09-03T06:47:31.887 DEBUG:tasks.rgw_multi.zone_rados:o=obj124 2022-09-03T06:47:31.888 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2022-09-03T06:47:31.889 DEBUG:tasks.rgw_multi.zone_rados:o=obj155 2022-09-03T06:47:31.889 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:47:31.890 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:47:31.890 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:47:31.891 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:47:31.891 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2022-09-03T06:47:31.892 DEBUG:tasks.rgw_multi.zone_rados:o=obj31 2022-09-03T06:47:31.892 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:47:31.893 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:47:31.893 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:47:31.894 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:47:31.894 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:47:31.895 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:47:31.895 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:47:31.896 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:47:31.896 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2022-09-03T06:47:31.897 DEBUG:tasks.rgw_multi.zone_rados:o=obj62 2022-09-03T06:47:31.897 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:47:31.898 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:47:31.898 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:47:31.899 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:47:31.899 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2022-09-03T06:47:31.900 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:47:31.900 DEBUG:tasks.rgw_multi.zone_rados:o=obj93 2022-09-03T06:47:31.901 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:47:31.901 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:47:31.902 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:31.903 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:31.903 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:31.904 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:31.904 DEBUG:boto:Method: GET 2022-09-03T06:47:31.905 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:31.905 DEBUG:boto:Data: 2022-09-03T06:47:31.906 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.907 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.907 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.908 DEBUG:boto:Params: {} 2022-09-03T06:47:31.909 DEBUG:boto:Token: None 2022-09-03T06:47:31.910 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/?versions 2022-09-03T06:47:31.910 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hqOyDuYrKP02fgviil0DDCc9MkU= 2022-09-03T06:47:31.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hqOyDuYrKP02fgviil0DDCc9MkU=', 'Content-Length': '0'} 2022-09-03T06:47:31.915 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009b9a60e87a3967db-006312f883-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.916 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:31.918 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:47:31.919 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:47:31.920 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2022-09-03T06:47:31.920 DEBUG:tasks.rgw_multi.zone_rados:o=obj124 2022-09-03T06:47:31.921 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2022-09-03T06:47:31.921 DEBUG:tasks.rgw_multi.zone_rados:o=obj155 2022-09-03T06:47:31.922 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:47:31.922 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:47:31.923 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:47:31.923 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:47:31.924 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2022-09-03T06:47:31.924 DEBUG:tasks.rgw_multi.zone_rados:o=obj31 2022-09-03T06:47:31.925 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:47:31.925 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:47:31.926 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:47:31.926 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:47:31.927 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:47:31.927 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:47:31.928 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:47:31.928 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:47:31.929 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2022-09-03T06:47:31.929 DEBUG:tasks.rgw_multi.zone_rados:o=obj62 2022-09-03T06:47:31.930 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:47:31.930 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:47:31.931 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:47:31.931 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:47:31.932 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2022-09-03T06:47:31.932 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:47:31.933 DEBUG:tasks.rgw_multi.zone_rados:o=obj93 2022-09-03T06:47:31.933 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:47:31.934 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:31.934 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:31.935 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:31.936 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:31.937 DEBUG:boto:Method: GET 2022-09-03T06:47:31.938 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:31.939 DEBUG:boto:Data: 2022-09-03T06:47:31.940 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.941 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:31.942 DEBUG:boto:Port: 8001 2022-09-03T06:47:31.944 DEBUG:boto:Params: {} 2022-09-03T06:47:31.945 DEBUG:boto:Token: None 2022-09-03T06:47:31.945 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/?versions 2022-09-03T06:47:31.946 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hqOyDuYrKP02fgviil0DDCc9MkU= 2022-09-03T06:47:31.947 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hqOyDuYrKP02fgviil0DDCc9MkU=', 'Content-Length': '0'} 2022-09-03T06:47:31.950 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003412bf21a53a1ce4-006312f883-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.953 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:31.958 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:31.959 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:31.960 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:31.961 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:31.962 DEBUG:boto:Method: GET 2022-09-03T06:47:31.963 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:31.964 DEBUG:boto:Data: 2022-09-03T06:47:31.965 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.966 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.966 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.967 DEBUG:boto:Params: {} 2022-09-03T06:47:31.967 DEBUG:boto:Token: None 2022-09-03T06:47:31.968 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/?versions 2022-09-03T06:47:31.969 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hqOyDuYrKP02fgviil0DDCc9MkU= 2022-09-03T06:47:31.969 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hqOyDuYrKP02fgviil0DDCc9MkU=', 'Content-Length': '0'} 2022-09-03T06:47:31.974 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000006f343bd7dfbd60e-006312f883-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.975 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:31.977 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:47:31.978 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:31.979 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:31.979 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:31.980 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:31.980 DEBUG:boto:Method: GET 2022-09-03T06:47:31.981 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:31.981 DEBUG:boto:Data: 2022-09-03T06:47:31.982 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.983 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:31.983 DEBUG:boto:Port: 8001 2022-09-03T06:47:31.984 DEBUG:boto:Params: {} 2022-09-03T06:47:31.985 DEBUG:boto:Token: None 2022-09-03T06:47:31.986 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:31.986 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:afaNDx+8gPBezXvB30F6j52Hs+A= 2022-09-03T06:47:31.987 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:afaNDx+8gPBezXvB30F6j52Hs+A=', 'Content-Length': '0'} 2022-09-03T06:47:31.990 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000355fdc54fb75e99d-006312f883-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:31.991 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:31.991 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:31.992 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:31.993 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:31.994 DEBUG:boto:Method: GET 2022-09-03T06:47:31.994 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:31.995 DEBUG:boto:Data: 2022-09-03T06:47:31.996 DEBUG:boto:Headers: {} 2022-09-03T06:47:31.996 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:31.997 DEBUG:boto:Port: 8000 2022-09-03T06:47:31.997 DEBUG:boto:Params: {} 2022-09-03T06:47:31.998 DEBUG:boto:Token: None 2022-09-03T06:47:31.999 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:31 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:31.999 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:afaNDx+8gPBezXvB30F6j52Hs+A= 2022-09-03T06:47:32.000 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:afaNDx+8gPBezXvB30F6j52Hs+A=', 'Content-Length': '0'} 2022-09-03T06:47:32.002 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008139cf176fb0f1bd-006312f884-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.046 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:32.047 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:32.048 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:32.048 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:32.049 DEBUG:boto:Method: HEAD 2022-09-03T06:47:32.049 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:32.050 DEBUG:boto:Data: 2022-09-03T06:47:32.050 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.051 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:32.051 DEBUG:boto:Port: 8001 2022-09-03T06:47:32.052 DEBUG:boto:Params: {} 2022-09-03T06:47:32.052 DEBUG:boto:Token: None 2022-09-03T06:47:32.053 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:32.054 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/ZxjXSu+zAdLsJa4m1a46T/ECOY= 2022-09-03T06:47:32.054 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/ZxjXSu+zAdLsJa4m1a46T/ECOY=', 'Content-Length': '0'} 2022-09-03T06:47:32.057 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cef9b13519debd1c-006312f884-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.060 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:32.061 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:32.063 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:32.063 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:32.064 DEBUG:boto:Method: HEAD 2022-09-03T06:47:32.064 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:32.065 DEBUG:boto:Data: 2022-09-03T06:47:32.065 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.066 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:32.066 DEBUG:boto:Port: 8000 2022-09-03T06:47:32.067 DEBUG:boto:Params: {} 2022-09-03T06:47:32.068 DEBUG:boto:Token: None 2022-09-03T06:47:32.068 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:32.069 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/ZxjXSu+zAdLsJa4m1a46T/ECOY= 2022-09-03T06:47:32.069 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/ZxjXSu+zAdLsJa4m1a46T/ECOY=', 'Content-Length': '0'} 2022-09-03T06:47:32.072 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000089020faba8a0257-006312f884-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.072 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:47:32.073 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:32.074 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:32.074 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:32.075 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:32.075 DEBUG:boto:Method: GET 2022-09-03T06:47:32.076 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:32.076 DEBUG:boto:Data: 2022-09-03T06:47:32.077 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.077 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:32.078 DEBUG:boto:Port: 8001 2022-09-03T06:47:32.078 DEBUG:boto:Params: {} 2022-09-03T06:47:32.079 DEBUG:boto:Token: None 2022-09-03T06:47:32.080 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:32.080 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Uw9hfLkJB1GM3RjH8he0vu7+gcg= 2022-09-03T06:47:32.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Uw9hfLkJB1GM3RjH8he0vu7+gcg=', 'Content-Length': '0'} 2022-09-03T06:47:32.090 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b822897f665ac76b-006312f884-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.091 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:32.092 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:32.092 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:32.093 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:32.093 DEBUG:boto:Method: GET 2022-09-03T06:47:32.094 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:32.094 DEBUG:boto:Data: 2022-09-03T06:47:32.095 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.096 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:32.097 DEBUG:boto:Port: 8000 2022-09-03T06:47:32.097 DEBUG:boto:Params: {} 2022-09-03T06:47:32.099 DEBUG:boto:Token: None 2022-09-03T06:47:32.099 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:32.100 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Uw9hfLkJB1GM3RjH8he0vu7+gcg= 2022-09-03T06:47:32.101 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Uw9hfLkJB1GM3RjH8he0vu7+gcg=', 'Content-Length': '0'} 2022-09-03T06:47:32.104 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000084a8b8c02f51fb1d-006312f884-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.105 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:32.106 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:32.107 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:32.107 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:32.108 DEBUG:boto:Method: GET 2022-09-03T06:47:32.109 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:47:32.110 DEBUG:boto:Data: 2022-09-03T06:47:32.111 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.111 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:32.112 DEBUG:boto:Port: 8001 2022-09-03T06:47:32.113 DEBUG:boto:Params: {} 2022-09-03T06:47:32.114 DEBUG:boto:Token: None 2022-09-03T06:47:32.115 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/ 2022-09-03T06:47:32.116 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM= 2022-09-03T06:47:32.117 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM=', 'Content-Length': '0'} 2022-09-03T06:47:32.120 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003850dc9ca844d972-006312f884-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.121 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:32.122 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:32.122 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:32.123 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:32.124 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:32.125 DEBUG:boto:Method: GET 2022-09-03T06:47:32.126 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:47:32.127 DEBUG:boto:Data: 2022-09-03T06:47:32.128 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.128 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:32.129 DEBUG:boto:Port: 8000 2022-09-03T06:47:32.130 DEBUG:boto:Params: {} 2022-09-03T06:47:32.132 DEBUG:boto:Token: None 2022-09-03T06:47:32.133 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/ 2022-09-03T06:47:32.134 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM= 2022-09-03T06:47:32.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM=', 'Content-Length': '0'} 2022-09-03T06:47:32.141 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000091d7cb0e154b3d2f-006312f884-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.142 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:32.143 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:32.143 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:32.144 DEBUG:boto:Method: HEAD 2022-09-03T06:47:32.144 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:47:32.145 DEBUG:boto:Data: 2022-09-03T06:47:32.145 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.146 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:32.147 DEBUG:boto:Port: 8001 2022-09-03T06:47:32.148 DEBUG:boto:Params: {} 2022-09-03T06:47:32.149 DEBUG:boto:Token: None 2022-09-03T06:47:32.150 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj11 2022-09-03T06:47:32.151 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L08zD3HKa7ior8AxXdzdzkFcZ0s= 2022-09-03T06:47:32.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L08zD3HKa7ior8AxXdzdzkFcZ0s=', 'Content-Length': '0'} 2022-09-03T06:47:32.155 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003535e9a4f825b292-006312f884-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.156 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:32.157 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:32.158 DEBUG:boto:Method: HEAD 2022-09-03T06:47:32.159 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:47:32.159 DEBUG:boto:Data: 2022-09-03T06:47:32.160 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.161 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:32.162 DEBUG:boto:Port: 8000 2022-09-03T06:47:32.163 DEBUG:boto:Params: {} 2022-09-03T06:47:32.164 DEBUG:boto:Token: None 2022-09-03T06:47:32.165 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj11 2022-09-03T06:47:32.166 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L08zD3HKa7ior8AxXdzdzkFcZ0s= 2022-09-03T06:47:32.167 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L08zD3HKa7ior8AxXdzdzkFcZ0s=', 'Content-Length': '0'} 2022-09-03T06:47:32.169 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e468145dc88577f2-006312f884-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.170 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:47:32.171 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:32.171 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:32.172 DEBUG:boto:Method: GET 2022-09-03T06:47:32.172 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:47:32.173 DEBUG:boto:Data: 2022-09-03T06:47:32.174 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.175 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:32.175 DEBUG:boto:Port: 8001 2022-09-03T06:47:32.176 DEBUG:boto:Params: {} 2022-09-03T06:47:32.176 DEBUG:boto:Token: None 2022-09-03T06:47:32.177 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj11 2022-09-03T06:47:32.178 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QicpRjSDGgXXOBr1UQSA6BaPxug= 2022-09-03T06:47:32.179 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QicpRjSDGgXXOBr1UQSA6BaPxug=', 'Content-Length': '0'} 2022-09-03T06:47:32.186 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fcbd1114ce688473-006312f884-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.235 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:32.236 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:32.237 DEBUG:boto:Method: GET 2022-09-03T06:47:32.239 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:47:32.239 DEBUG:boto:Data: 2022-09-03T06:47:32.240 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.241 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:32.243 DEBUG:boto:Port: 8000 2022-09-03T06:47:32.245 DEBUG:boto:Params: {} 2022-09-03T06:47:32.246 DEBUG:boto:Token: None 2022-09-03T06:47:32.248 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj11 2022-09-03T06:47:32.249 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QicpRjSDGgXXOBr1UQSA6BaPxug= 2022-09-03T06:47:32.251 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QicpRjSDGgXXOBr1UQSA6BaPxug=', 'Content-Length': '0'} 2022-09-03T06:47:32.253 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000128ab39c56015568-006312f884-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.256 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:32.257 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:32.258 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:32.258 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:32.259 DEBUG:boto:Method: GET 2022-09-03T06:47:32.260 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:47:32.260 DEBUG:boto:Data: 2022-09-03T06:47:32.261 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.261 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:32.262 DEBUG:boto:Port: 8001 2022-09-03T06:47:32.262 DEBUG:boto:Params: {} 2022-09-03T06:47:32.263 DEBUG:boto:Token: None 2022-09-03T06:47:32.264 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/ 2022-09-03T06:47:32.264 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM= 2022-09-03T06:47:32.265 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM=', 'Content-Length': '0'} 2022-09-03T06:47:32.269 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000aa844f52fed9a66-006312f884-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.270 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:32.272 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:32.274 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:32.276 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:32.277 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:32.278 DEBUG:boto:Method: GET 2022-09-03T06:47:32.279 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:47:32.279 DEBUG:boto:Data: 2022-09-03T06:47:32.280 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.280 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:32.281 DEBUG:boto:Port: 8000 2022-09-03T06:47:32.281 DEBUG:boto:Params: {} 2022-09-03T06:47:32.282 DEBUG:boto:Token: None 2022-09-03T06:47:32.283 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/ 2022-09-03T06:47:32.283 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM= 2022-09-03T06:47:32.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM=', 'Content-Length': '0'} 2022-09-03T06:47:32.288 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000156200c454b91de2-006312f884-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.331 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:32.334 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:47:32.335 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:32.336 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:32.337 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.342 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.343 DEBUG:boto:Method: GET 2022-09-03T06:47:32.344 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.345 DEBUG:boto:Data: 2022-09-03T06:47:32.346 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.347 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:32.348 DEBUG:boto:Port: 8001 2022-09-03T06:47:32.349 DEBUG:boto:Params: {} 2022-09-03T06:47:32.350 DEBUG:boto:Token: None 2022-09-03T06:47:32.351 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.352 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E4DoMBMvcWHffqqgc+unlLEEWGc= 2022-09-03T06:47:32.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E4DoMBMvcWHffqqgc+unlLEEWGc=', 'Content-Length': '0'} 2022-09-03T06:47:32.355 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f5e8bc7aadbd3aee-006312f884-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.357 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:32.358 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:32.358 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.359 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.359 DEBUG:boto:Method: GET 2022-09-03T06:47:32.360 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.361 DEBUG:boto:Data: 2022-09-03T06:47:32.361 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.362 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:32.362 DEBUG:boto:Port: 8000 2022-09-03T06:47:32.363 DEBUG:boto:Params: {} 2022-09-03T06:47:32.364 DEBUG:boto:Token: None 2022-09-03T06:47:32.364 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.365 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E4DoMBMvcWHffqqgc+unlLEEWGc= 2022-09-03T06:47:32.366 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E4DoMBMvcWHffqqgc+unlLEEWGc=', 'Content-Length': '0'} 2022-09-03T06:47:32.368 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001f842d6e3f750f6b-006312f884-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.369 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:32.370 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:32.371 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.371 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.372 DEBUG:boto:Method: HEAD 2022-09-03T06:47:32.372 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.373 DEBUG:boto:Data: 2022-09-03T06:47:32.373 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.374 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:32.374 DEBUG:boto:Port: 8001 2022-09-03T06:47:32.375 DEBUG:boto:Params: {} 2022-09-03T06:47:32.375 DEBUG:boto:Token: None 2022-09-03T06:47:32.376 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.377 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AWDUKjTCTNmRKD6DApzvUeeJ93Q= 2022-09-03T06:47:32.378 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AWDUKjTCTNmRKD6DApzvUeeJ93Q=', 'Content-Length': '0'} 2022-09-03T06:47:32.381 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e3d26600050737d2-006312f884-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.384 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:32.384 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:32.385 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.386 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.388 DEBUG:boto:Method: HEAD 2022-09-03T06:47:32.388 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.389 DEBUG:boto:Data: 2022-09-03T06:47:32.390 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.392 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:32.393 DEBUG:boto:Port: 8000 2022-09-03T06:47:32.394 DEBUG:boto:Params: {} 2022-09-03T06:47:32.395 DEBUG:boto:Token: None 2022-09-03T06:47:32.396 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.397 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AWDUKjTCTNmRKD6DApzvUeeJ93Q= 2022-09-03T06:47:32.398 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AWDUKjTCTNmRKD6DApzvUeeJ93Q=', 'Content-Length': '0'} 2022-09-03T06:47:32.401 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b9a072c7c3205168-006312f884-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.403 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:47:32.403 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:32.404 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:32.405 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.405 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.406 DEBUG:boto:Method: GET 2022-09-03T06:47:32.406 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.407 DEBUG:boto:Data: 2022-09-03T06:47:32.408 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.408 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:32.409 DEBUG:boto:Port: 8001 2022-09-03T06:47:32.410 DEBUG:boto:Params: {} 2022-09-03T06:47:32.411 DEBUG:boto:Token: None 2022-09-03T06:47:32.412 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.413 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E4DoMBMvcWHffqqgc+unlLEEWGc= 2022-09-03T06:47:32.414 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E4DoMBMvcWHffqqgc+unlLEEWGc=', 'Content-Length': '0'} 2022-09-03T06:47:32.416 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005a2f24ca3175a523-006312f884-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.417 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:32.418 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:32.419 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.420 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.421 DEBUG:boto:Method: GET 2022-09-03T06:47:32.421 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.422 DEBUG:boto:Data: 2022-09-03T06:47:32.422 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.424 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:32.424 DEBUG:boto:Port: 8000 2022-09-03T06:47:32.425 DEBUG:boto:Params: {} 2022-09-03T06:47:32.426 DEBUG:boto:Token: None 2022-09-03T06:47:32.427 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:32.428 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E4DoMBMvcWHffqqgc+unlLEEWGc= 2022-09-03T06:47:32.429 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E4DoMBMvcWHffqqgc+unlLEEWGc=', 'Content-Length': '0'} 2022-09-03T06:47:32.432 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a4a796ab5a9368c-006312f884-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.434 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:32.435 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:32.436 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:32.437 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:32.438 DEBUG:boto:Method: GET 2022-09-03T06:47:32.438 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:47:32.439 DEBUG:boto:Data: 2022-09-03T06:47:32.439 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.440 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:32.441 DEBUG:boto:Port: 8001 2022-09-03T06:47:32.441 DEBUG:boto:Params: {} 2022-09-03T06:47:32.442 DEBUG:boto:Token: None 2022-09-03T06:47:32.443 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/ 2022-09-03T06:47:32.444 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM= 2022-09-03T06:47:32.444 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM=', 'Content-Length': '0'} 2022-09-03T06:47:32.447 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000487b10abed1c26a9-006312f884-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.448 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:32.449 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:32.450 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:32.451 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:32.451 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:32.452 DEBUG:boto:Method: GET 2022-09-03T06:47:32.452 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:47:32.453 DEBUG:boto:Data: 2022-09-03T06:47:32.453 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.454 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:32.455 DEBUG:boto:Port: 8000 2022-09-03T06:47:32.456 DEBUG:boto:Params: {} 2022-09-03T06:47:32.457 DEBUG:boto:Token: None 2022-09-03T06:47:32.458 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/ 2022-09-03T06:47:32.458 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM= 2022-09-03T06:47:32.459 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM=', 'Content-Length': '0'} 2022-09-03T06:47:32.464 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ac7bc602b21d4255-006312f884-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.465 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:32.466 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:32.466 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:32.467 DEBUG:boto:Method: HEAD 2022-09-03T06:47:32.468 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:47:32.468 DEBUG:boto:Data: 2022-09-03T06:47:32.469 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.469 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:32.470 DEBUG:boto:Port: 8001 2022-09-03T06:47:32.471 DEBUG:boto:Params: {} 2022-09-03T06:47:32.472 DEBUG:boto:Token: None 2022-09-03T06:47:32.473 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj115 2022-09-03T06:47:32.474 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/ifDuKYWWLSxdaWE9Xcrpm00sIo= 2022-09-03T06:47:32.475 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/ifDuKYWWLSxdaWE9Xcrpm00sIo=', 'Content-Length': '0'} 2022-09-03T06:47:32.478 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c75b6284e3d7e66c-006312f884-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.479 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:32.480 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:32.481 DEBUG:boto:Method: HEAD 2022-09-03T06:47:32.482 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:47:32.483 DEBUG:boto:Data: 2022-09-03T06:47:32.484 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.485 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:32.486 DEBUG:boto:Port: 8000 2022-09-03T06:47:32.487 DEBUG:boto:Params: {} 2022-09-03T06:47:32.488 DEBUG:boto:Token: None 2022-09-03T06:47:32.489 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj115 2022-09-03T06:47:32.490 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/ifDuKYWWLSxdaWE9Xcrpm00sIo= 2022-09-03T06:47:32.491 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/ifDuKYWWLSxdaWE9Xcrpm00sIo=', 'Content-Length': '0'} 2022-09-03T06:47:32.498 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b1eb566edb67f04-006312f884-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.499 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:47:32.500 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:32.501 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:32.501 DEBUG:boto:Method: GET 2022-09-03T06:47:32.502 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:47:32.503 DEBUG:boto:Data: 2022-09-03T06:47:32.503 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.504 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:32.504 DEBUG:boto:Port: 8001 2022-09-03T06:47:32.505 DEBUG:boto:Params: {} 2022-09-03T06:47:32.506 DEBUG:boto:Token: None 2022-09-03T06:47:32.507 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj115 2022-09-03T06:47:32.507 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X2f1tIlQRFNQPydBfu47Iierl6I= 2022-09-03T06:47:32.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X2f1tIlQRFNQPydBfu47Iierl6I=', 'Content-Length': '0'} 2022-09-03T06:47:32.511 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009188de8a9b73945b-006312f884-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.554 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:32.555 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:32.556 DEBUG:boto:Method: GET 2022-09-03T06:47:32.556 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:47:32.557 DEBUG:boto:Data: 2022-09-03T06:47:32.558 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.558 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:32.559 DEBUG:boto:Port: 8000 2022-09-03T06:47:32.559 DEBUG:boto:Params: {} 2022-09-03T06:47:32.560 DEBUG:boto:Token: None 2022-09-03T06:47:32.561 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj115 2022-09-03T06:47:32.562 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X2f1tIlQRFNQPydBfu47Iierl6I= 2022-09-03T06:47:32.562 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X2f1tIlQRFNQPydBfu47Iierl6I=', 'Content-Length': '0'} 2022-09-03T06:47:32.565 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008f90542a5acc7296-006312f884-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.566 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:32.566 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:32.567 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:32.567 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:32.568 DEBUG:boto:Method: GET 2022-09-03T06:47:32.569 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:47:32.569 DEBUG:boto:Data: 2022-09-03T06:47:32.570 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.570 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:32.571 DEBUG:boto:Port: 8001 2022-09-03T06:47:32.572 DEBUG:boto:Params: {} 2022-09-03T06:47:32.572 DEBUG:boto:Token: None 2022-09-03T06:47:32.573 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/ 2022-09-03T06:47:32.574 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM= 2022-09-03T06:47:32.574 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM=', 'Content-Length': '0'} 2022-09-03T06:47:32.577 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000078d90b307f78d1b8-006312f884-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.578 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:32.579 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:32.580 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:32.580 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:32.581 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:32.581 DEBUG:boto:Method: GET 2022-09-03T06:47:32.582 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:47:32.582 DEBUG:boto:Data: 2022-09-03T06:47:32.583 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.583 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:32.584 DEBUG:boto:Port: 8000 2022-09-03T06:47:32.584 DEBUG:boto:Params: {} 2022-09-03T06:47:32.585 DEBUG:boto:Token: None 2022-09-03T06:47:32.586 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/ 2022-09-03T06:47:32.586 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM= 2022-09-03T06:47:32.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM=', 'Content-Length': '0'} 2022-09-03T06:47:32.591 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fd65648921c5424d-006312f884-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.592 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:32.593 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:47:32.593 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:32.594 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:32.595 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.595 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.596 DEBUG:boto:Method: GET 2022-09-03T06:47:32.597 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.597 DEBUG:boto:Data: 2022-09-03T06:47:32.598 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.598 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:32.599 DEBUG:boto:Port: 8001 2022-09-03T06:47:32.600 DEBUG:boto:Params: {} 2022-09-03T06:47:32.601 DEBUG:boto:Token: None 2022-09-03T06:47:32.601 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.602 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DwFoV0ZLXkkkDbpRi5r0VUxMUew= 2022-09-03T06:47:32.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DwFoV0ZLXkkkDbpRi5r0VUxMUew=', 'Content-Length': '0'} 2022-09-03T06:47:32.605 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006f968414471a6621-006312f884-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.606 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:32.607 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:32.608 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.608 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.609 DEBUG:boto:Method: GET 2022-09-03T06:47:32.609 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.610 DEBUG:boto:Data: 2022-09-03T06:47:32.610 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.611 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:32.612 DEBUG:boto:Port: 8000 2022-09-03T06:47:32.612 DEBUG:boto:Params: {} 2022-09-03T06:47:32.613 DEBUG:boto:Token: None 2022-09-03T06:47:32.613 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.614 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DwFoV0ZLXkkkDbpRi5r0VUxMUew= 2022-09-03T06:47:32.615 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DwFoV0ZLXkkkDbpRi5r0VUxMUew=', 'Content-Length': '0'} 2022-09-03T06:47:32.617 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c3b45fb8df226e0f-006312f884-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.658 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:32.660 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:32.661 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.662 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.663 DEBUG:boto:Method: HEAD 2022-09-03T06:47:32.663 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.664 DEBUG:boto:Data: 2022-09-03T06:47:32.665 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.665 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:32.666 DEBUG:boto:Port: 8001 2022-09-03T06:47:32.666 DEBUG:boto:Params: {} 2022-09-03T06:47:32.667 DEBUG:boto:Token: None 2022-09-03T06:47:32.668 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.669 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RyGbgL/tz9h3y2Z93Wm48H+9iZE= 2022-09-03T06:47:32.670 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RyGbgL/tz9h3y2Z93Wm48H+9iZE=', 'Content-Length': '0'} 2022-09-03T06:47:32.672 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003060f3dc98d10828-006312f884-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.674 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:32.675 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:32.675 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.676 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.677 DEBUG:boto:Method: HEAD 2022-09-03T06:47:32.678 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.679 DEBUG:boto:Data: 2022-09-03T06:47:32.679 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.680 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:32.681 DEBUG:boto:Port: 8000 2022-09-03T06:47:32.681 DEBUG:boto:Params: {} 2022-09-03T06:47:32.682 DEBUG:boto:Token: None 2022-09-03T06:47:32.683 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.685 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RyGbgL/tz9h3y2Z93Wm48H+9iZE= 2022-09-03T06:47:32.686 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RyGbgL/tz9h3y2Z93Wm48H+9iZE=', 'Content-Length': '0'} 2022-09-03T06:47:32.688 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000082e59455d146f305-006312f884-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.690 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:47:32.690 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:32.691 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:32.691 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.692 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.693 DEBUG:boto:Method: GET 2022-09-03T06:47:32.693 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.694 DEBUG:boto:Data: 2022-09-03T06:47:32.694 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.695 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:32.695 DEBUG:boto:Port: 8001 2022-09-03T06:47:32.696 DEBUG:boto:Params: {} 2022-09-03T06:47:32.697 DEBUG:boto:Token: None 2022-09-03T06:47:32.697 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.698 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DwFoV0ZLXkkkDbpRi5r0VUxMUew= 2022-09-03T06:47:32.699 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DwFoV0ZLXkkkDbpRi5r0VUxMUew=', 'Content-Length': '0'} 2022-09-03T06:47:32.701 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f629a3f8770bef0-006312f884-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.702 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:32.702 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:32.703 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.704 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.704 DEBUG:boto:Method: GET 2022-09-03T06:47:32.705 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.705 DEBUG:boto:Data: 2022-09-03T06:47:32.706 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.707 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:32.707 DEBUG:boto:Port: 8000 2022-09-03T06:47:32.708 DEBUG:boto:Params: {} 2022-09-03T06:47:32.709 DEBUG:boto:Token: None 2022-09-03T06:47:32.709 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:32.710 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DwFoV0ZLXkkkDbpRi5r0VUxMUew= 2022-09-03T06:47:32.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DwFoV0ZLXkkkDbpRi5r0VUxMUew=', 'Content-Length': '0'} 2022-09-03T06:47:32.715 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006c57f806d55afe66-006312f884-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.716 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:32.717 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:32.718 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:32.718 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:32.719 DEBUG:boto:Method: GET 2022-09-03T06:47:32.720 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:47:32.720 DEBUG:boto:Data: 2022-09-03T06:47:32.721 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.722 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:32.723 DEBUG:boto:Port: 8001 2022-09-03T06:47:32.724 DEBUG:boto:Params: {} 2022-09-03T06:47:32.724 DEBUG:boto:Token: None 2022-09-03T06:47:32.725 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/ 2022-09-03T06:47:32.726 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM= 2022-09-03T06:47:32.727 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM=', 'Content-Length': '0'} 2022-09-03T06:47:32.740 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cf2674ed0ef6787a-006312f884-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.741 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:32.742 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:32.743 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:32.743 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:32.744 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:32.745 DEBUG:boto:Method: GET 2022-09-03T06:47:32.746 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:47:32.747 DEBUG:boto:Data: 2022-09-03T06:47:32.749 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.750 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:32.752 DEBUG:boto:Port: 8000 2022-09-03T06:47:32.753 DEBUG:boto:Params: {} 2022-09-03T06:47:32.754 DEBUG:boto:Token: None 2022-09-03T06:47:32.756 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/ 2022-09-03T06:47:32.757 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM= 2022-09-03T06:47:32.758 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM=', 'Content-Length': '0'} 2022-09-03T06:47:32.762 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000d82a713c34395a4-006312f884-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.764 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:32.767 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:32.770 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:32.771 DEBUG:boto:Method: HEAD 2022-09-03T06:47:32.772 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:47:32.774 DEBUG:boto:Data: 2022-09-03T06:47:32.775 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.776 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:32.777 DEBUG:boto:Port: 8001 2022-09-03T06:47:32.779 DEBUG:boto:Params: {} 2022-09-03T06:47:32.779 DEBUG:boto:Token: None 2022-09-03T06:47:32.780 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj116 2022-09-03T06:47:32.781 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z+BIzcOnb99nYfPkchzLRTSwaaU= 2022-09-03T06:47:32.781 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z+BIzcOnb99nYfPkchzLRTSwaaU=', 'Content-Length': '0'} 2022-09-03T06:47:32.783 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000126d3c17b953fb7d-006312f884-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.784 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:32.785 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:32.785 DEBUG:boto:Method: HEAD 2022-09-03T06:47:32.786 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:47:32.786 DEBUG:boto:Data: 2022-09-03T06:47:32.787 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.787 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:32.788 DEBUG:boto:Port: 8000 2022-09-03T06:47:32.788 DEBUG:boto:Params: {} 2022-09-03T06:47:32.789 DEBUG:boto:Token: None 2022-09-03T06:47:32.790 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj116 2022-09-03T06:47:32.790 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z+BIzcOnb99nYfPkchzLRTSwaaU= 2022-09-03T06:47:32.791 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z+BIzcOnb99nYfPkchzLRTSwaaU=', 'Content-Length': '0'} 2022-09-03T06:47:32.793 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ebfd7a9e6d085184-006312f884-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.794 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:47:32.795 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:32.795 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:32.796 DEBUG:boto:Method: GET 2022-09-03T06:47:32.797 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:47:32.797 DEBUG:boto:Data: 2022-09-03T06:47:32.798 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.798 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:32.799 DEBUG:boto:Port: 8001 2022-09-03T06:47:32.800 DEBUG:boto:Params: {} 2022-09-03T06:47:32.801 DEBUG:boto:Token: None 2022-09-03T06:47:32.802 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj116 2022-09-03T06:47:32.803 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fkxfBn/3q2hf6L6dNnHgHVXOmOI= 2022-09-03T06:47:32.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fkxfBn/3q2hf6L6dNnHgHVXOmOI=', 'Content-Length': '0'} 2022-09-03T06:47:32.806 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc46fdb9005e74e9-006312f884-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.807 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:32.808 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:32.808 DEBUG:boto:Method: GET 2022-09-03T06:47:32.809 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:47:32.810 DEBUG:boto:Data: 2022-09-03T06:47:32.810 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.811 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:32.811 DEBUG:boto:Port: 8000 2022-09-03T06:47:32.812 DEBUG:boto:Params: {} 2022-09-03T06:47:32.813 DEBUG:boto:Token: None 2022-09-03T06:47:32.813 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj116 2022-09-03T06:47:32.814 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fkxfBn/3q2hf6L6dNnHgHVXOmOI= 2022-09-03T06:47:32.815 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fkxfBn/3q2hf6L6dNnHgHVXOmOI=', 'Content-Length': '0'} 2022-09-03T06:47:32.817 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003ea304f806ab5ce1-006312f884-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.818 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:32.818 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:32.819 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:32.819 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:32.820 DEBUG:boto:Method: GET 2022-09-03T06:47:32.821 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:47:32.821 DEBUG:boto:Data: 2022-09-03T06:47:32.822 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.822 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:32.823 DEBUG:boto:Port: 8001 2022-09-03T06:47:32.823 DEBUG:boto:Params: {} 2022-09-03T06:47:32.824 DEBUG:boto:Token: None 2022-09-03T06:47:32.825 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/ 2022-09-03T06:47:32.825 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM= 2022-09-03T06:47:32.826 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM=', 'Content-Length': '0'} 2022-09-03T06:47:32.830 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000053ae5547e609db3e-006312f884-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.831 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:32.833 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:32.834 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:32.835 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:32.836 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:32.837 DEBUG:boto:Method: GET 2022-09-03T06:47:32.838 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:47:32.839 DEBUG:boto:Data: 2022-09-03T06:47:32.840 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.841 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:32.841 DEBUG:boto:Port: 8000 2022-09-03T06:47:32.842 DEBUG:boto:Params: {} 2022-09-03T06:47:32.843 DEBUG:boto:Token: None 2022-09-03T06:47:32.844 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/ 2022-09-03T06:47:32.845 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM= 2022-09-03T06:47:32.846 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bc8Y/zFhWvXfRiRUjlB3V97wOUM=', 'Content-Length': '0'} 2022-09-03T06:47:32.850 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b310e5eee88e88f3-006312f884-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.851 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:32.852 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2022-09-03T06:47:32.853 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:32.853 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:32.854 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.855 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.855 DEBUG:boto:Method: GET 2022-09-03T06:47:32.856 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.856 DEBUG:boto:Data: 2022-09-03T06:47:32.857 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.858 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:32.859 DEBUG:boto:Port: 8001 2022-09-03T06:47:32.860 DEBUG:boto:Params: {} 2022-09-03T06:47:32.860 DEBUG:boto:Token: None 2022-09-03T06:47:32.861 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.864 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RJbes2wuWTfq2cdQViQaCuQrUJA= 2022-09-03T06:47:32.866 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RJbes2wuWTfq2cdQViQaCuQrUJA=', 'Content-Length': '0'} 2022-09-03T06:47:32.869 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000565f8105d6a2bced-006312f884-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.910 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:32.911 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:32.912 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.913 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.914 DEBUG:boto:Method: GET 2022-09-03T06:47:32.916 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.916 DEBUG:boto:Data: 2022-09-03T06:47:32.917 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.918 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:32.919 DEBUG:boto:Port: 8000 2022-09-03T06:47:32.919 DEBUG:boto:Params: {} 2022-09-03T06:47:32.920 DEBUG:boto:Token: None 2022-09-03T06:47:32.921 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.922 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RJbes2wuWTfq2cdQViQaCuQrUJA= 2022-09-03T06:47:32.923 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RJbes2wuWTfq2cdQViQaCuQrUJA=', 'Content-Length': '0'} 2022-09-03T06:47:32.926 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a8c05519332fd68-006312f884-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.927 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:32.927 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:32.928 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.928 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.929 DEBUG:boto:Method: HEAD 2022-09-03T06:47:32.929 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.930 DEBUG:boto:Data: 2022-09-03T06:47:32.931 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.931 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:32.932 DEBUG:boto:Port: 8001 2022-09-03T06:47:32.932 DEBUG:boto:Params: {} 2022-09-03T06:47:32.933 DEBUG:boto:Token: None 2022-09-03T06:47:32.934 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.935 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HoFnWhh1b5viIfScp6SqCjw1Ais= 2022-09-03T06:47:32.935 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HoFnWhh1b5viIfScp6SqCjw1Ais=', 'Content-Length': '0'} 2022-09-03T06:47:32.938 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000068a7cfce1fda2f23-006312f884-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.939 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:32.939 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:32.940 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.940 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.941 DEBUG:boto:Method: HEAD 2022-09-03T06:47:32.941 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.942 DEBUG:boto:Data: 2022-09-03T06:47:32.943 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.944 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:32.944 DEBUG:boto:Port: 8000 2022-09-03T06:47:32.945 DEBUG:boto:Params: {} 2022-09-03T06:47:32.946 DEBUG:boto:Token: None 2022-09-03T06:47:32.946 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.947 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HoFnWhh1b5viIfScp6SqCjw1Ais= 2022-09-03T06:47:32.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HoFnWhh1b5viIfScp6SqCjw1Ais=', 'Content-Length': '0'} 2022-09-03T06:47:32.951 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bb83dd272e4bf173-006312f884-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.951 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2022-09-03T06:47:32.952 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:32.953 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:32.954 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.955 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.956 DEBUG:boto:Method: GET 2022-09-03T06:47:32.956 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.957 DEBUG:boto:Data: 2022-09-03T06:47:32.958 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.959 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:32.960 DEBUG:boto:Port: 8001 2022-09-03T06:47:32.961 DEBUG:boto:Params: {} 2022-09-03T06:47:32.962 DEBUG:boto:Token: None 2022-09-03T06:47:32.965 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.966 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RJbes2wuWTfq2cdQViQaCuQrUJA= 2022-09-03T06:47:32.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RJbes2wuWTfq2cdQViQaCuQrUJA=', 'Content-Length': '0'} 2022-09-03T06:47:32.971 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a37b1087bb0ad28b-006312f884-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:32.972 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:32.973 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:32.974 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.975 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.976 DEBUG:boto:Method: GET 2022-09-03T06:47:32.977 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.978 DEBUG:boto:Data: 2022-09-03T06:47:32.978 DEBUG:boto:Headers: {} 2022-09-03T06:47:32.979 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:32.979 DEBUG:boto:Port: 8000 2022-09-03T06:47:32.980 DEBUG:boto:Params: {} 2022-09-03T06:47:32.981 DEBUG:boto:Token: None 2022-09-03T06:47:32.982 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:32 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:32.982 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RJbes2wuWTfq2cdQViQaCuQrUJA= 2022-09-03T06:47:32.983 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RJbes2wuWTfq2cdQViQaCuQrUJA=', 'Content-Length': '0'} 2022-09-03T06:47:32.985 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096271a8869e3ea84-006312f884-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.026 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:33.027 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:33.028 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:33.029 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:33.030 DEBUG:boto:Method: GET 2022-09-03T06:47:33.031 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:47:33.032 DEBUG:boto:Data: 2022-09-03T06:47:33.033 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.034 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.034 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.035 DEBUG:boto:Params: {} 2022-09-03T06:47:33.036 DEBUG:boto:Token: None 2022-09-03T06:47:33.037 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/ 2022-09-03T06:47:33.038 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8= 2022-09-03T06:47:33.039 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8=', 'Content-Length': '0'} 2022-09-03T06:47:33.042 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000edd8ca7fb04d8d75-006312f885-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.043 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:33.044 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:33.044 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:33.045 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:33.045 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:33.046 DEBUG:boto:Method: GET 2022-09-03T06:47:33.047 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:47:33.047 DEBUG:boto:Data: 2022-09-03T06:47:33.048 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.049 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.049 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.050 DEBUG:boto:Params: {} 2022-09-03T06:47:33.050 DEBUG:boto:Token: None 2022-09-03T06:47:33.051 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/ 2022-09-03T06:47:33.052 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8= 2022-09-03T06:47:33.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8=', 'Content-Length': '0'} 2022-09-03T06:47:33.057 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006c88bcefd83724f2-006312f885-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.057 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:33.058 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:33.059 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:33.060 DEBUG:boto:Method: HEAD 2022-09-03T06:47:33.060 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:47:33.061 DEBUG:boto:Data: 2022-09-03T06:47:33.061 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.062 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.063 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.063 DEBUG:boto:Params: {} 2022-09-03T06:47:33.064 DEBUG:boto:Token: None 2022-09-03T06:47:33.064 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj124 2022-09-03T06:47:33.065 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wwFMj+Nufjos0B5YM/hgFLReUIo= 2022-09-03T06:47:33.067 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wwFMj+Nufjos0B5YM/hgFLReUIo=', 'Content-Length': '0'} 2022-09-03T06:47:33.069 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007c20a2b314b2a9f4-006312f885-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.071 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:33.071 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:33.072 DEBUG:boto:Method: HEAD 2022-09-03T06:47:33.072 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:47:33.073 DEBUG:boto:Data: 2022-09-03T06:47:33.074 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.075 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.076 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.077 DEBUG:boto:Params: {} 2022-09-03T06:47:33.078 DEBUG:boto:Token: None 2022-09-03T06:47:33.079 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj124 2022-09-03T06:47:33.080 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wwFMj+Nufjos0B5YM/hgFLReUIo= 2022-09-03T06:47:33.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wwFMj+Nufjos0B5YM/hgFLReUIo=', 'Content-Length': '0'} 2022-09-03T06:47:33.084 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000033af8c60c594ef5a-006312f885-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.085 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2022-09-03T06:47:33.086 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:33.086 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:33.087 DEBUG:boto:Method: GET 2022-09-03T06:47:33.088 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:47:33.089 DEBUG:boto:Data: 2022-09-03T06:47:33.089 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.090 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.091 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.092 DEBUG:boto:Params: {} 2022-09-03T06:47:33.092 DEBUG:boto:Token: None 2022-09-03T06:47:33.093 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj124 2022-09-03T06:47:33.094 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eVXQ5+F0G1UhW2TooQ+EHuAQVK8= 2022-09-03T06:47:33.095 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eVXQ5+F0G1UhW2TooQ+EHuAQVK8=', 'Content-Length': '0'} 2022-09-03T06:47:33.097 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b067ff6a35cbfc5-006312f885-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.098 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:33.099 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:33.100 DEBUG:boto:Method: GET 2022-09-03T06:47:33.101 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:47:33.102 DEBUG:boto:Data: 2022-09-03T06:47:33.103 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.104 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.105 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.106 DEBUG:boto:Params: {} 2022-09-03T06:47:33.107 DEBUG:boto:Token: None 2022-09-03T06:47:33.108 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj124 2022-09-03T06:47:33.109 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eVXQ5+F0G1UhW2TooQ+EHuAQVK8= 2022-09-03T06:47:33.110 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eVXQ5+F0G1UhW2TooQ+EHuAQVK8=', 'Content-Length': '0'} 2022-09-03T06:47:33.113 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000592a62b4f6c48960-006312f885-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.114 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:33.115 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:33.115 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:33.116 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:33.116 DEBUG:boto:Method: GET 2022-09-03T06:47:33.117 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:47:33.118 DEBUG:boto:Data: 2022-09-03T06:47:33.118 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.119 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.120 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.120 DEBUG:boto:Params: {} 2022-09-03T06:47:33.121 DEBUG:boto:Token: None 2022-09-03T06:47:33.122 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/ 2022-09-03T06:47:33.124 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8= 2022-09-03T06:47:33.125 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8=', 'Content-Length': '0'} 2022-09-03T06:47:33.128 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002259719ab2fd8cb0-006312f885-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.129 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:33.130 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:33.131 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:33.131 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:33.132 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:33.133 DEBUG:boto:Method: GET 2022-09-03T06:47:33.133 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:47:33.134 DEBUG:boto:Data: 2022-09-03T06:47:33.135 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.135 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.136 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.136 DEBUG:boto:Params: {} 2022-09-03T06:47:33.137 DEBUG:boto:Token: None 2022-09-03T06:47:33.138 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/ 2022-09-03T06:47:33.138 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8= 2022-09-03T06:47:33.139 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8=', 'Content-Length': '0'} 2022-09-03T06:47:33.143 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004839776648b70629-006312f885-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.144 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:33.155 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:47:33.156 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:33.157 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:33.158 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.159 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.160 DEBUG:boto:Method: GET 2022-09-03T06:47:33.161 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.162 DEBUG:boto:Data: 2022-09-03T06:47:33.163 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.164 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.165 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.166 DEBUG:boto:Params: {} 2022-09-03T06:47:33.167 DEBUG:boto:Token: None 2022-09-03T06:47:33.168 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.169 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6mV393tU5wKvMQ9ARcMbKaOve40= 2022-09-03T06:47:33.170 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6mV393tU5wKvMQ9ARcMbKaOve40=', 'Content-Length': '0'} 2022-09-03T06:47:33.173 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000368ef4206c3fee27-006312f885-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.175 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:33.175 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:33.176 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.177 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.178 DEBUG:boto:Method: GET 2022-09-03T06:47:33.179 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.180 DEBUG:boto:Data: 2022-09-03T06:47:33.180 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.181 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.182 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.183 DEBUG:boto:Params: {} 2022-09-03T06:47:33.184 DEBUG:boto:Token: None 2022-09-03T06:47:33.185 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.186 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6mV393tU5wKvMQ9ARcMbKaOve40= 2022-09-03T06:47:33.187 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6mV393tU5wKvMQ9ARcMbKaOve40=', 'Content-Length': '0'} 2022-09-03T06:47:33.189 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cb5441fe6423b01e-006312f885-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.191 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:33.191 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:33.192 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.193 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.194 DEBUG:boto:Method: HEAD 2022-09-03T06:47:33.195 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.196 DEBUG:boto:Data: 2022-09-03T06:47:33.196 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.197 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.198 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.199 DEBUG:boto:Params: {} 2022-09-03T06:47:33.199 DEBUG:boto:Token: None 2022-09-03T06:47:33.200 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.201 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bubvJ3+LtCKdZq5hX8a785pI1vo= 2022-09-03T06:47:33.201 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bubvJ3+LtCKdZq5hX8a785pI1vo=', 'Content-Length': '0'} 2022-09-03T06:47:33.203 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004233c5b521240fd2-006312f885-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.204 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:33.205 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:33.205 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.206 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.206 DEBUG:boto:Method: HEAD 2022-09-03T06:47:33.207 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.207 DEBUG:boto:Data: 2022-09-03T06:47:33.208 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.208 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.209 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.210 DEBUG:boto:Params: {} 2022-09-03T06:47:33.210 DEBUG:boto:Token: None 2022-09-03T06:47:33.211 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.212 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bubvJ3+LtCKdZq5hX8a785pI1vo= 2022-09-03T06:47:33.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bubvJ3+LtCKdZq5hX8a785pI1vo=', 'Content-Length': '0'} 2022-09-03T06:47:33.215 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c563af4311834d53-006312f885-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.215 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:47:33.216 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:33.217 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:33.217 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.218 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.218 DEBUG:boto:Method: GET 2022-09-03T06:47:33.219 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.219 DEBUG:boto:Data: 2022-09-03T06:47:33.220 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.220 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.221 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.222 DEBUG:boto:Params: {} 2022-09-03T06:47:33.222 DEBUG:boto:Token: None 2022-09-03T06:47:33.223 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.224 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6mV393tU5wKvMQ9ARcMbKaOve40= 2022-09-03T06:47:33.225 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6mV393tU5wKvMQ9ARcMbKaOve40=', 'Content-Length': '0'} 2022-09-03T06:47:33.227 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bbd15f6aee5c23c5-006312f885-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.229 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:33.230 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:33.231 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.232 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.233 DEBUG:boto:Method: GET 2022-09-03T06:47:33.234 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.235 DEBUG:boto:Data: 2022-09-03T06:47:33.236 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.237 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.238 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.238 DEBUG:boto:Params: {} 2022-09-03T06:47:33.239 DEBUG:boto:Token: None 2022-09-03T06:47:33.241 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:33.242 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6mV393tU5wKvMQ9ARcMbKaOve40= 2022-09-03T06:47:33.242 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6mV393tU5wKvMQ9ARcMbKaOve40=', 'Content-Length': '0'} 2022-09-03T06:47:33.245 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a8d3989414409124-006312f885-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.286 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:33.287 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:33.288 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:33.288 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:33.289 DEBUG:boto:Method: GET 2022-09-03T06:47:33.289 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:47:33.290 DEBUG:boto:Data: 2022-09-03T06:47:33.290 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.291 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.292 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.292 DEBUG:boto:Params: {} 2022-09-03T06:47:33.293 DEBUG:boto:Token: None 2022-09-03T06:47:33.293 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/ 2022-09-03T06:47:33.294 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8= 2022-09-03T06:47:33.295 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8=', 'Content-Length': '0'} 2022-09-03T06:47:33.298 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000058f4ec408bb2adfc-006312f885-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.299 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:33.300 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:33.300 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:33.301 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:33.301 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:33.302 DEBUG:boto:Method: GET 2022-09-03T06:47:33.302 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:47:33.303 DEBUG:boto:Data: 2022-09-03T06:47:33.304 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.304 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.305 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.305 DEBUG:boto:Params: {} 2022-09-03T06:47:33.306 DEBUG:boto:Token: None 2022-09-03T06:47:33.307 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/ 2022-09-03T06:47:33.307 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8= 2022-09-03T06:47:33.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8=', 'Content-Length': '0'} 2022-09-03T06:47:33.312 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c51dcae6b973f4ce-006312f885-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.313 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:33.314 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:33.315 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:33.315 DEBUG:boto:Method: HEAD 2022-09-03T06:47:33.316 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:47:33.317 DEBUG:boto:Data: 2022-09-03T06:47:33.318 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.318 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.319 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.319 DEBUG:boto:Params: {} 2022-09-03T06:47:33.320 DEBUG:boto:Token: None 2022-09-03T06:47:33.320 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj145 2022-09-03T06:47:33.321 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O4s6ii+fEYVkrh2VdUREvh2JczI= 2022-09-03T06:47:33.322 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O4s6ii+fEYVkrh2VdUREvh2JczI=', 'Content-Length': '0'} 2022-09-03T06:47:33.324 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000091503e5c911dbcd5-006312f885-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.325 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:33.326 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:33.326 DEBUG:boto:Method: HEAD 2022-09-03T06:47:33.327 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:47:33.327 DEBUG:boto:Data: 2022-09-03T06:47:33.328 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.328 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.329 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.329 DEBUG:boto:Params: {} 2022-09-03T06:47:33.330 DEBUG:boto:Token: None 2022-09-03T06:47:33.331 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj145 2022-09-03T06:47:33.332 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O4s6ii+fEYVkrh2VdUREvh2JczI= 2022-09-03T06:47:33.332 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O4s6ii+fEYVkrh2VdUREvh2JczI=', 'Content-Length': '0'} 2022-09-03T06:47:33.335 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000579da4e8fef646d1-006312f885-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.335 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:47:33.336 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:33.336 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:33.337 DEBUG:boto:Method: GET 2022-09-03T06:47:33.338 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:47:33.338 DEBUG:boto:Data: 2022-09-03T06:47:33.339 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.339 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.340 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.340 DEBUG:boto:Params: {} 2022-09-03T06:47:33.341 DEBUG:boto:Token: None 2022-09-03T06:47:33.342 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj145 2022-09-03T06:47:33.343 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bupZd/WUqfQA6FSXdz9mM1orUTA= 2022-09-03T06:47:33.343 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bupZd/WUqfQA6FSXdz9mM1orUTA=', 'Content-Length': '0'} 2022-09-03T06:47:33.345 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000088642991ae3bd000-006312f885-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.386 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:33.387 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:33.387 DEBUG:boto:Method: GET 2022-09-03T06:47:33.388 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:47:33.389 DEBUG:boto:Data: 2022-09-03T06:47:33.390 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.391 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.392 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.393 DEBUG:boto:Params: {} 2022-09-03T06:47:33.394 DEBUG:boto:Token: None 2022-09-03T06:47:33.395 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj145 2022-09-03T06:47:33.396 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bupZd/WUqfQA6FSXdz9mM1orUTA= 2022-09-03T06:47:33.397 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bupZd/WUqfQA6FSXdz9mM1orUTA=', 'Content-Length': '0'} 2022-09-03T06:47:33.403 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e5433795aae34f03-006312f885-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.404 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:33.405 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:33.406 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:33.407 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:33.408 DEBUG:boto:Method: GET 2022-09-03T06:47:33.409 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:47:33.409 DEBUG:boto:Data: 2022-09-03T06:47:33.410 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.411 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.412 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.413 DEBUG:boto:Params: {} 2022-09-03T06:47:33.413 DEBUG:boto:Token: None 2022-09-03T06:47:33.414 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/ 2022-09-03T06:47:33.415 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8= 2022-09-03T06:47:33.416 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8=', 'Content-Length': '0'} 2022-09-03T06:47:33.419 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000077a5e5eafa9d79e6-006312f885-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.420 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:33.421 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:33.421 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:33.422 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:33.423 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:33.424 DEBUG:boto:Method: GET 2022-09-03T06:47:33.424 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:47:33.425 DEBUG:boto:Data: 2022-09-03T06:47:33.425 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.426 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.427 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.427 DEBUG:boto:Params: {} 2022-09-03T06:47:33.428 DEBUG:boto:Token: None 2022-09-03T06:47:33.428 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/ 2022-09-03T06:47:33.429 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8= 2022-09-03T06:47:33.430 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8=', 'Content-Length': '0'} 2022-09-03T06:47:33.434 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000072d7696ced38fe8c-006312f885-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.434 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:33.435 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2022-09-03T06:47:33.436 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:33.436 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:33.437 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.438 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.439 DEBUG:boto:Method: GET 2022-09-03T06:47:33.440 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.441 DEBUG:boto:Data: 2022-09-03T06:47:33.441 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.442 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.443 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.443 DEBUG:boto:Params: {} 2022-09-03T06:47:33.444 DEBUG:boto:Token: None 2022-09-03T06:47:33.445 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.446 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hrdgBqNWRndLCYL9OslCVi97/+s= 2022-09-03T06:47:33.447 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hrdgBqNWRndLCYL9OslCVi97/+s=', 'Content-Length': '0'} 2022-09-03T06:47:33.450 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c5124ba2d4015f1a-006312f885-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.451 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:33.451 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:33.452 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.452 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.453 DEBUG:boto:Method: GET 2022-09-03T06:47:33.453 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.454 DEBUG:boto:Data: 2022-09-03T06:47:33.454 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.455 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.456 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.456 DEBUG:boto:Params: {} 2022-09-03T06:47:33.457 DEBUG:boto:Token: None 2022-09-03T06:47:33.457 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.458 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hrdgBqNWRndLCYL9OslCVi97/+s= 2022-09-03T06:47:33.458 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hrdgBqNWRndLCYL9OslCVi97/+s=', 'Content-Length': '0'} 2022-09-03T06:47:33.461 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b7c39d503811469-006312f885-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.461 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:33.462 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:33.462 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.463 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.463 DEBUG:boto:Method: HEAD 2022-09-03T06:47:33.464 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.464 DEBUG:boto:Data: 2022-09-03T06:47:33.465 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.465 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.466 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.466 DEBUG:boto:Params: {} 2022-09-03T06:47:33.467 DEBUG:boto:Token: None 2022-09-03T06:47:33.468 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.468 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0rKJAT6OXr3m9G7wHGO8kfgSqgo= 2022-09-03T06:47:33.469 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0rKJAT6OXr3m9G7wHGO8kfgSqgo=', 'Content-Length': '0'} 2022-09-03T06:47:33.471 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003937b55c8be98039-006312f885-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.472 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:33.472 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:33.473 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.473 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.474 DEBUG:boto:Method: HEAD 2022-09-03T06:47:33.474 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.475 DEBUG:boto:Data: 2022-09-03T06:47:33.475 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.476 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.477 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.477 DEBUG:boto:Params: {} 2022-09-03T06:47:33.478 DEBUG:boto:Token: None 2022-09-03T06:47:33.478 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.479 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0rKJAT6OXr3m9G7wHGO8kfgSqgo= 2022-09-03T06:47:33.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0rKJAT6OXr3m9G7wHGO8kfgSqgo=', 'Content-Length': '0'} 2022-09-03T06:47:33.482 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008153db89cb972271-006312f885-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.482 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2022-09-03T06:47:33.483 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:33.484 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:33.484 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.485 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.485 DEBUG:boto:Method: GET 2022-09-03T06:47:33.486 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.486 DEBUG:boto:Data: 2022-09-03T06:47:33.487 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.487 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.488 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.488 DEBUG:boto:Params: {} 2022-09-03T06:47:33.489 DEBUG:boto:Token: None 2022-09-03T06:47:33.490 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.490 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hrdgBqNWRndLCYL9OslCVi97/+s= 2022-09-03T06:47:33.491 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hrdgBqNWRndLCYL9OslCVi97/+s=', 'Content-Length': '0'} 2022-09-03T06:47:33.493 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000031632fd0882439b9-006312f885-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.534 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:33.535 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:33.536 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.537 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.538 DEBUG:boto:Method: GET 2022-09-03T06:47:33.539 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.540 DEBUG:boto:Data: 2022-09-03T06:47:33.541 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.543 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.544 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.545 DEBUG:boto:Params: {} 2022-09-03T06:47:33.546 DEBUG:boto:Token: None 2022-09-03T06:47:33.546 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:33.548 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hrdgBqNWRndLCYL9OslCVi97/+s= 2022-09-03T06:47:33.549 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hrdgBqNWRndLCYL9OslCVi97/+s=', 'Content-Length': '0'} 2022-09-03T06:47:33.552 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000144734fdccb3a1df-006312f885-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.553 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:33.554 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:33.555 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:33.556 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:33.556 DEBUG:boto:Method: GET 2022-09-03T06:47:33.557 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:47:33.558 DEBUG:boto:Data: 2022-09-03T06:47:33.559 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.560 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.561 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.562 DEBUG:boto:Params: {} 2022-09-03T06:47:33.563 DEBUG:boto:Token: None 2022-09-03T06:47:33.565 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/ 2022-09-03T06:47:33.566 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8= 2022-09-03T06:47:33.567 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8=', 'Content-Length': '0'} 2022-09-03T06:47:33.573 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000096fde07e8f46067c-006312f885-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.574 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:33.574 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:33.575 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:33.576 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:33.576 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:33.577 DEBUG:boto:Method: GET 2022-09-03T06:47:33.578 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:47:33.578 DEBUG:boto:Data: 2022-09-03T06:47:33.579 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.580 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.581 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.581 DEBUG:boto:Params: {} 2022-09-03T06:47:33.582 DEBUG:boto:Token: None 2022-09-03T06:47:33.583 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/ 2022-09-03T06:47:33.583 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8= 2022-09-03T06:47:33.584 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8=', 'Content-Length': '0'} 2022-09-03T06:47:33.588 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cf5ac8bde4faa61a-006312f885-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.590 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:33.591 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:33.592 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:33.593 DEBUG:boto:Method: HEAD 2022-09-03T06:47:33.593 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:47:33.594 DEBUG:boto:Data: 2022-09-03T06:47:33.594 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.595 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.596 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.597 DEBUG:boto:Params: {} 2022-09-03T06:47:33.597 DEBUG:boto:Token: None 2022-09-03T06:47:33.598 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj155 2022-09-03T06:47:33.598 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r9LKKv2XEKuIFhbyU/7XCtU+ccQ= 2022-09-03T06:47:33.599 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r9LKKv2XEKuIFhbyU/7XCtU+ccQ=', 'Content-Length': '0'} 2022-09-03T06:47:33.610 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026e79151be4d6142-006312f885-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.612 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:33.613 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:33.615 DEBUG:boto:Method: HEAD 2022-09-03T06:47:33.616 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:47:33.617 DEBUG:boto:Data: 2022-09-03T06:47:33.617 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.618 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.619 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.620 DEBUG:boto:Params: {} 2022-09-03T06:47:33.621 DEBUG:boto:Token: None 2022-09-03T06:47:33.621 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj155 2022-09-03T06:47:33.622 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r9LKKv2XEKuIFhbyU/7XCtU+ccQ= 2022-09-03T06:47:33.623 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r9LKKv2XEKuIFhbyU/7XCtU+ccQ=', 'Content-Length': '0'} 2022-09-03T06:47:33.627 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002453958045aa5f8e-006312f885-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.628 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2022-09-03T06:47:33.629 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:33.629 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:33.630 DEBUG:boto:Method: GET 2022-09-03T06:47:33.631 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:47:33.632 DEBUG:boto:Data: 2022-09-03T06:47:33.632 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.633 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.633 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.634 DEBUG:boto:Params: {} 2022-09-03T06:47:33.634 DEBUG:boto:Token: None 2022-09-03T06:47:33.635 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj155 2022-09-03T06:47:33.636 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:02Iie2ferql1f7nnXLGGEQuUhEY= 2022-09-03T06:47:33.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:02Iie2ferql1f7nnXLGGEQuUhEY=', 'Content-Length': '0'} 2022-09-03T06:47:33.639 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005ef62de0eca4ce0a-006312f885-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.683 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:33.684 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:33.684 DEBUG:boto:Method: GET 2022-09-03T06:47:33.685 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:47:33.685 DEBUG:boto:Data: 2022-09-03T06:47:33.686 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.687 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.688 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.688 DEBUG:boto:Params: {} 2022-09-03T06:47:33.689 DEBUG:boto:Token: None 2022-09-03T06:47:33.690 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj155 2022-09-03T06:47:33.690 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:02Iie2ferql1f7nnXLGGEQuUhEY= 2022-09-03T06:47:33.691 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:02Iie2ferql1f7nnXLGGEQuUhEY=', 'Content-Length': '0'} 2022-09-03T06:47:33.699 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001f6b8a451cd7ea78-006312f885-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.700 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:33.700 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:33.701 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:33.701 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:33.702 DEBUG:boto:Method: GET 2022-09-03T06:47:33.703 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:47:33.703 DEBUG:boto:Data: 2022-09-03T06:47:33.704 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.704 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.705 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.705 DEBUG:boto:Params: {} 2022-09-03T06:47:33.706 DEBUG:boto:Token: None 2022-09-03T06:47:33.707 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/ 2022-09-03T06:47:33.707 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8= 2022-09-03T06:47:33.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8=', 'Content-Length': '0'} 2022-09-03T06:47:33.711 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000662e735cce775f4a-006312f885-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.712 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:33.712 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:33.713 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:33.714 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:33.715 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:33.715 DEBUG:boto:Method: GET 2022-09-03T06:47:33.716 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:47:33.716 DEBUG:boto:Data: 2022-09-03T06:47:33.717 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.718 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.718 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.719 DEBUG:boto:Params: {} 2022-09-03T06:47:33.720 DEBUG:boto:Token: None 2022-09-03T06:47:33.720 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/ 2022-09-03T06:47:33.721 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8= 2022-09-03T06:47:33.723 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8=', 'Content-Length': '0'} 2022-09-03T06:47:33.736 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ec1268ec2906dc80-006312f885-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.738 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:33.740 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:47:33.741 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:33.742 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:33.743 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.743 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.744 DEBUG:boto:Method: GET 2022-09-03T06:47:33.745 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.747 DEBUG:boto:Data: 2022-09-03T06:47:33.749 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.750 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.751 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.753 DEBUG:boto:Params: {} 2022-09-03T06:47:33.755 DEBUG:boto:Token: None 2022-09-03T06:47:33.756 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.757 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CH/1N6FiQQ+nmv+Tp12EY/UE81c= 2022-09-03T06:47:33.758 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CH/1N6FiQQ+nmv+Tp12EY/UE81c=', 'Content-Length': '0'} 2022-09-03T06:47:33.766 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c595cb22c44e44c-006312f885-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.768 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:33.769 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:33.769 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.770 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.770 DEBUG:boto:Method: GET 2022-09-03T06:47:33.771 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.772 DEBUG:boto:Data: 2022-09-03T06:47:33.772 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.773 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.773 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.774 DEBUG:boto:Params: {} 2022-09-03T06:47:33.776 DEBUG:boto:Token: None 2022-09-03T06:47:33.777 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.778 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CH/1N6FiQQ+nmv+Tp12EY/UE81c= 2022-09-03T06:47:33.778 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CH/1N6FiQQ+nmv+Tp12EY/UE81c=', 'Content-Length': '0'} 2022-09-03T06:47:33.781 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b5f2727928ac0b71-006312f885-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.783 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:33.783 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:33.784 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.785 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.785 DEBUG:boto:Method: HEAD 2022-09-03T06:47:33.786 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.786 DEBUG:boto:Data: 2022-09-03T06:47:33.787 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.787 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.788 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.789 DEBUG:boto:Params: {} 2022-09-03T06:47:33.789 DEBUG:boto:Token: None 2022-09-03T06:47:33.790 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.791 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WouF40c2lw0Lm/iciXWI+bC4doc= 2022-09-03T06:47:33.791 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WouF40c2lw0Lm/iciXWI+bC4doc=', 'Content-Length': '0'} 2022-09-03T06:47:33.794 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000010e219fc14a7c9cf-006312f885-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.794 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:33.795 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:33.796 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.796 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.797 DEBUG:boto:Method: HEAD 2022-09-03T06:47:33.797 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.798 DEBUG:boto:Data: 2022-09-03T06:47:33.798 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.799 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.800 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.800 DEBUG:boto:Params: {} 2022-09-03T06:47:33.801 DEBUG:boto:Token: None 2022-09-03T06:47:33.802 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.802 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WouF40c2lw0Lm/iciXWI+bC4doc= 2022-09-03T06:47:33.803 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WouF40c2lw0Lm/iciXWI+bC4doc=', 'Content-Length': '0'} 2022-09-03T06:47:33.806 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b48608f8455887b1-006312f885-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.808 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:47:33.809 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:33.810 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:33.811 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.812 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.813 DEBUG:boto:Method: GET 2022-09-03T06:47:33.813 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.814 DEBUG:boto:Data: 2022-09-03T06:47:33.815 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.815 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.816 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.816 DEBUG:boto:Params: {} 2022-09-03T06:47:33.817 DEBUG:boto:Token: None 2022-09-03T06:47:33.817 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.818 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CH/1N6FiQQ+nmv+Tp12EY/UE81c= 2022-09-03T06:47:33.819 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CH/1N6FiQQ+nmv+Tp12EY/UE81c=', 'Content-Length': '0'} 2022-09-03T06:47:33.821 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000339067c1ccd6fc44-006312f885-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.822 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:33.822 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:33.823 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.824 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.824 DEBUG:boto:Method: GET 2022-09-03T06:47:33.825 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.825 DEBUG:boto:Data: 2022-09-03T06:47:33.826 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.826 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.827 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.827 DEBUG:boto:Params: {} 2022-09-03T06:47:33.828 DEBUG:boto:Token: None 2022-09-03T06:47:33.829 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:33.829 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CH/1N6FiQQ+nmv+Tp12EY/UE81c= 2022-09-03T06:47:33.830 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CH/1N6FiQQ+nmv+Tp12EY/UE81c=', 'Content-Length': '0'} 2022-09-03T06:47:33.833 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008ff6078b82fcf008-006312f885-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.834 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:33.834 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:33.835 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:33.835 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:33.836 DEBUG:boto:Method: GET 2022-09-03T06:47:33.837 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:47:33.837 DEBUG:boto:Data: 2022-09-03T06:47:33.838 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.838 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.839 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.840 DEBUG:boto:Params: {} 2022-09-03T06:47:33.840 DEBUG:boto:Token: None 2022-09-03T06:47:33.841 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/ 2022-09-03T06:47:33.842 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8= 2022-09-03T06:47:33.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8=', 'Content-Length': '0'} 2022-09-03T06:47:33.845 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004beb3af3512af95b-006312f885-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.846 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:33.847 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:33.848 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:33.848 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:33.849 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:33.850 DEBUG:boto:Method: GET 2022-09-03T06:47:33.850 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:47:33.851 DEBUG:boto:Data: 2022-09-03T06:47:33.851 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.852 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.853 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.853 DEBUG:boto:Params: {} 2022-09-03T06:47:33.854 DEBUG:boto:Token: None 2022-09-03T06:47:33.854 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/ 2022-09-03T06:47:33.855 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8= 2022-09-03T06:47:33.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8=', 'Content-Length': '0'} 2022-09-03T06:47:33.860 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000030b795eb86c98dd7-006312f885-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.861 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:33.862 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:33.863 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:33.863 DEBUG:boto:Method: HEAD 2022-09-03T06:47:33.864 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:47:33.865 DEBUG:boto:Data: 2022-09-03T06:47:33.865 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.866 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.866 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.867 DEBUG:boto:Params: {} 2022-09-03T06:47:33.867 DEBUG:boto:Token: None 2022-09-03T06:47:33.868 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj17 2022-09-03T06:47:33.869 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IrVBxeh3dRpDuhmdORiD42+JkZs= 2022-09-03T06:47:33.869 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IrVBxeh3dRpDuhmdORiD42+JkZs=', 'Content-Length': '0'} 2022-09-03T06:47:33.871 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000051a6fc8a0d80f83d-006312f885-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.872 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:33.873 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:33.873 DEBUG:boto:Method: HEAD 2022-09-03T06:47:33.874 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:47:33.874 DEBUG:boto:Data: 2022-09-03T06:47:33.875 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.876 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.876 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.877 DEBUG:boto:Params: {} 2022-09-03T06:47:33.877 DEBUG:boto:Token: None 2022-09-03T06:47:33.878 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj17 2022-09-03T06:47:33.879 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IrVBxeh3dRpDuhmdORiD42+JkZs= 2022-09-03T06:47:33.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IrVBxeh3dRpDuhmdORiD42+JkZs=', 'Content-Length': '0'} 2022-09-03T06:47:33.889 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a59ad55c19f277a6-006312f885-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.889 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:47:33.890 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:33.891 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:33.891 DEBUG:boto:Method: GET 2022-09-03T06:47:33.892 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:47:33.893 DEBUG:boto:Data: 2022-09-03T06:47:33.893 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.894 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.894 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.895 DEBUG:boto:Params: {} 2022-09-03T06:47:33.896 DEBUG:boto:Token: None 2022-09-03T06:47:33.896 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj17 2022-09-03T06:47:33.897 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:g+ReQwlJWQ7i64zBHk5W4OpXQxA= 2022-09-03T06:47:33.898 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:g+ReQwlJWQ7i64zBHk5W4OpXQxA=', 'Content-Length': '0'} 2022-09-03T06:47:33.900 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000037fc75d65f47903a-006312f885-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.903 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:33.904 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:33.904 DEBUG:boto:Method: GET 2022-09-03T06:47:33.905 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:47:33.906 DEBUG:boto:Data: 2022-09-03T06:47:33.906 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.907 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.908 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.909 DEBUG:boto:Params: {} 2022-09-03T06:47:33.910 DEBUG:boto:Token: None 2022-09-03T06:47:33.912 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj17 2022-09-03T06:47:33.913 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:g+ReQwlJWQ7i64zBHk5W4OpXQxA= 2022-09-03T06:47:33.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:g+ReQwlJWQ7i64zBHk5W4OpXQxA=', 'Content-Length': '0'} 2022-09-03T06:47:33.918 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000500ad1542c3fd6fa-006312f885-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.919 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:33.920 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:33.920 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:33.921 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:33.922 DEBUG:boto:Method: GET 2022-09-03T06:47:33.922 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:47:33.923 DEBUG:boto:Data: 2022-09-03T06:47:33.924 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.924 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.925 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.926 DEBUG:boto:Params: {} 2022-09-03T06:47:33.927 DEBUG:boto:Token: None 2022-09-03T06:47:33.927 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/ 2022-09-03T06:47:33.928 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8= 2022-09-03T06:47:33.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8=', 'Content-Length': '0'} 2022-09-03T06:47:33.932 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000020a0a575a31b993a-006312f885-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.933 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:33.934 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:33.934 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:33.935 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:33.936 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:33.936 DEBUG:boto:Method: GET 2022-09-03T06:47:33.937 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:47:33.937 DEBUG:boto:Data: 2022-09-03T06:47:33.938 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.939 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.939 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.940 DEBUG:boto:Params: {} 2022-09-03T06:47:33.941 DEBUG:boto:Token: None 2022-09-03T06:47:33.941 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/ 2022-09-03T06:47:33.942 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8= 2022-09-03T06:47:33.943 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wIJI/TeOBFbiIGgKEQJHkPIL5b8=', 'Content-Length': '0'} 2022-09-03T06:47:33.947 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000b47120a20c6a21b-006312f885-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.947 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:33.948 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:47:33.949 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:33.949 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:33.950 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:33.950 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:33.951 DEBUG:boto:Method: GET 2022-09-03T06:47:33.952 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:33.952 DEBUG:boto:Data: 2022-09-03T06:47:33.953 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.953 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:33.954 DEBUG:boto:Port: 8001 2022-09-03T06:47:33.954 DEBUG:boto:Params: {} 2022-09-03T06:47:33.955 DEBUG:boto:Token: None 2022-09-03T06:47:33.956 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:33.956 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bEpEev57loktarx1xLIrrBN1HdU= 2022-09-03T06:47:33.957 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bEpEev57loktarx1xLIrrBN1HdU=', 'Content-Length': '0'} 2022-09-03T06:47:33.959 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b9f2f042912e46a5-006312f885-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:33.960 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:33.961 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:33.962 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:33.962 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:33.963 DEBUG:boto:Method: GET 2022-09-03T06:47:33.964 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:33.964 DEBUG:boto:Data: 2022-09-03T06:47:33.965 DEBUG:boto:Headers: {} 2022-09-03T06:47:33.965 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:33.966 DEBUG:boto:Port: 8000 2022-09-03T06:47:33.966 DEBUG:boto:Params: {} 2022-09-03T06:47:33.967 DEBUG:boto:Token: None 2022-09-03T06:47:33.968 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:33 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:33.968 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bEpEev57loktarx1xLIrrBN1HdU= 2022-09-03T06:47:33.969 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bEpEev57loktarx1xLIrrBN1HdU=', 'Content-Length': '0'} 2022-09-03T06:47:33.972 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d9e1c9c7858bac4-006312f885-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.018 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:34.019 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:34.020 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:34.021 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:34.022 DEBUG:boto:Method: HEAD 2022-09-03T06:47:34.022 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:34.023 DEBUG:boto:Data: 2022-09-03T06:47:34.023 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.024 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:34.024 DEBUG:boto:Port: 8001 2022-09-03T06:47:34.025 DEBUG:boto:Params: {} 2022-09-03T06:47:34.025 DEBUG:boto:Token: None 2022-09-03T06:47:34.026 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:34.028 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EJunwoIGL1GVgLTSgq7uiwSFKN4= 2022-09-03T06:47:34.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EJunwoIGL1GVgLTSgq7uiwSFKN4=', 'Content-Length': '0'} 2022-09-03T06:47:34.031 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000987616987fb706b-006312f886-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.032 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:34.032 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:34.033 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:34.033 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:34.034 DEBUG:boto:Method: HEAD 2022-09-03T06:47:34.034 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:34.035 DEBUG:boto:Data: 2022-09-03T06:47:34.035 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.036 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:34.036 DEBUG:boto:Port: 8000 2022-09-03T06:47:34.037 DEBUG:boto:Params: {} 2022-09-03T06:47:34.038 DEBUG:boto:Token: None 2022-09-03T06:47:34.038 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:34.039 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EJunwoIGL1GVgLTSgq7uiwSFKN4= 2022-09-03T06:47:34.039 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EJunwoIGL1GVgLTSgq7uiwSFKN4=', 'Content-Length': '0'} 2022-09-03T06:47:34.042 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007dfa39f776003181-006312f886-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.043 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:47:34.043 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:34.044 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:34.044 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:34.045 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:34.045 DEBUG:boto:Method: GET 2022-09-03T06:47:34.046 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:34.046 DEBUG:boto:Data: 2022-09-03T06:47:34.047 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.047 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:34.048 DEBUG:boto:Port: 8001 2022-09-03T06:47:34.049 DEBUG:boto:Params: {} 2022-09-03T06:47:34.050 DEBUG:boto:Token: None 2022-09-03T06:47:34.050 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:34.051 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cWyl1hFjMcb+aZSyOqA+KqEDe9Q= 2022-09-03T06:47:34.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cWyl1hFjMcb+aZSyOqA+KqEDe9Q=', 'Content-Length': '0'} 2022-09-03T06:47:34.054 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b25b8a82666fe4c-006312f886-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.094 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:34.095 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:34.095 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:34.096 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:34.096 DEBUG:boto:Method: GET 2022-09-03T06:47:34.097 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:34.098 DEBUG:boto:Data: 2022-09-03T06:47:34.098 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.099 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:34.099 DEBUG:boto:Port: 8000 2022-09-03T06:47:34.100 DEBUG:boto:Params: {} 2022-09-03T06:47:34.100 DEBUG:boto:Token: None 2022-09-03T06:47:34.101 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:34.101 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cWyl1hFjMcb+aZSyOqA+KqEDe9Q= 2022-09-03T06:47:34.102 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cWyl1hFjMcb+aZSyOqA+KqEDe9Q=', 'Content-Length': '0'} 2022-09-03T06:47:34.105 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a3a925e4dd4b969c-006312f886-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.106 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:34.107 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:34.108 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:34.109 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:34.109 DEBUG:boto:Method: GET 2022-09-03T06:47:34.113 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:47:34.114 DEBUG:boto:Data: 2022-09-03T06:47:34.115 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.116 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:34.117 DEBUG:boto:Port: 8001 2022-09-03T06:47:34.118 DEBUG:boto:Params: {} 2022-09-03T06:47:34.119 DEBUG:boto:Token: None 2022-09-03T06:47:34.120 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/ 2022-09-03T06:47:34.121 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo= 2022-09-03T06:47:34.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo=', 'Content-Length': '0'} 2022-09-03T06:47:34.125 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d59de5ca0c3e35fe-006312f886-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.126 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:34.127 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:34.128 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:34.128 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:34.129 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:34.129 DEBUG:boto:Method: GET 2022-09-03T06:47:34.130 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:47:34.130 DEBUG:boto:Data: 2022-09-03T06:47:34.131 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.132 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:34.133 DEBUG:boto:Port: 8000 2022-09-03T06:47:34.133 DEBUG:boto:Params: {} 2022-09-03T06:47:34.134 DEBUG:boto:Token: None 2022-09-03T06:47:34.135 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/ 2022-09-03T06:47:34.135 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo= 2022-09-03T06:47:34.136 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo=', 'Content-Length': '0'} 2022-09-03T06:47:34.140 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ef5db185a2f068c0-006312f886-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.142 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:34.143 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:34.144 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:34.145 DEBUG:boto:Method: HEAD 2022-09-03T06:47:34.146 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:47:34.147 DEBUG:boto:Data: 2022-09-03T06:47:34.148 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.149 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:34.150 DEBUG:boto:Port: 8001 2022-09-03T06:47:34.151 DEBUG:boto:Params: {} 2022-09-03T06:47:34.152 DEBUG:boto:Token: None 2022-09-03T06:47:34.152 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj19 2022-09-03T06:47:34.153 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A0RJtMcpADcAbaSB4GWqf9PxfL8= 2022-09-03T06:47:34.154 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A0RJtMcpADcAbaSB4GWqf9PxfL8=', 'Content-Length': '0'} 2022-09-03T06:47:34.156 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008d49bcc192e9f0e6-006312f886-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.157 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:34.158 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:34.159 DEBUG:boto:Method: HEAD 2022-09-03T06:47:34.159 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:47:34.160 DEBUG:boto:Data: 2022-09-03T06:47:34.160 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.161 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:34.161 DEBUG:boto:Port: 8000 2022-09-03T06:47:34.162 DEBUG:boto:Params: {} 2022-09-03T06:47:34.163 DEBUG:boto:Token: None 2022-09-03T06:47:34.163 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj19 2022-09-03T06:47:34.164 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A0RJtMcpADcAbaSB4GWqf9PxfL8= 2022-09-03T06:47:34.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A0RJtMcpADcAbaSB4GWqf9PxfL8=', 'Content-Length': '0'} 2022-09-03T06:47:34.167 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c05513a4a1a830d2-006312f886-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.169 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:47:34.170 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:34.170 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:34.171 DEBUG:boto:Method: GET 2022-09-03T06:47:34.171 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:47:34.172 DEBUG:boto:Data: 2022-09-03T06:47:34.173 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.173 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:34.175 DEBUG:boto:Port: 8001 2022-09-03T06:47:34.176 DEBUG:boto:Params: {} 2022-09-03T06:47:34.177 DEBUG:boto:Token: None 2022-09-03T06:47:34.179 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj19 2022-09-03T06:47:34.179 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ph1OyusaWHi6yKk51Jz7OX3Z3kE= 2022-09-03T06:47:34.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ph1OyusaWHi6yKk51Jz7OX3Z3kE=', 'Content-Length': '0'} 2022-09-03T06:47:34.182 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fa8364c0a68db5e3-006312f886-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.183 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:34.184 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:34.185 DEBUG:boto:Method: GET 2022-09-03T06:47:34.185 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:47:34.186 DEBUG:boto:Data: 2022-09-03T06:47:34.186 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.187 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:34.188 DEBUG:boto:Port: 8000 2022-09-03T06:47:34.188 DEBUG:boto:Params: {} 2022-09-03T06:47:34.189 DEBUG:boto:Token: None 2022-09-03T06:47:34.190 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj19 2022-09-03T06:47:34.191 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ph1OyusaWHi6yKk51Jz7OX3Z3kE= 2022-09-03T06:47:34.191 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ph1OyusaWHi6yKk51Jz7OX3Z3kE=', 'Content-Length': '0'} 2022-09-03T06:47:34.194 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000406c1c88f1db61a1-006312f886-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.238 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:34.239 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:34.240 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:34.240 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:34.241 DEBUG:boto:Method: GET 2022-09-03T06:47:34.241 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:47:34.242 DEBUG:boto:Data: 2022-09-03T06:47:34.243 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.243 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:34.244 DEBUG:boto:Port: 8001 2022-09-03T06:47:34.244 DEBUG:boto:Params: {} 2022-09-03T06:47:34.245 DEBUG:boto:Token: None 2022-09-03T06:47:34.245 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/ 2022-09-03T06:47:34.246 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo= 2022-09-03T06:47:34.247 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo=', 'Content-Length': '0'} 2022-09-03T06:47:34.250 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ca72a2952e589e37-006312f886-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.251 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:34.253 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:34.254 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:34.255 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:34.256 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:34.257 DEBUG:boto:Method: GET 2022-09-03T06:47:34.258 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:47:34.259 DEBUG:boto:Data: 2022-09-03T06:47:34.259 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.260 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:34.261 DEBUG:boto:Port: 8000 2022-09-03T06:47:34.262 DEBUG:boto:Params: {} 2022-09-03T06:47:34.263 DEBUG:boto:Token: None 2022-09-03T06:47:34.265 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/ 2022-09-03T06:47:34.266 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo= 2022-09-03T06:47:34.267 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo=', 'Content-Length': '0'} 2022-09-03T06:47:34.271 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a2e433ea5c8a73e9-006312f886-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.272 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:34.274 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:47:34.275 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:34.277 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:34.277 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.278 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.279 DEBUG:boto:Method: GET 2022-09-03T06:47:34.280 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.281 DEBUG:boto:Data: 2022-09-03T06:47:34.282 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.283 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:34.283 DEBUG:boto:Port: 8001 2022-09-03T06:47:34.284 DEBUG:boto:Params: {} 2022-09-03T06:47:34.285 DEBUG:boto:Token: None 2022-09-03T06:47:34.286 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.286 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Esm+0GyBfg3AhmlMPY748QyDDBE= 2022-09-03T06:47:34.287 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Esm+0GyBfg3AhmlMPY748QyDDBE=', 'Content-Length': '0'} 2022-09-03T06:47:34.290 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000be72554e8eb6e660-006312f886-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.291 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:34.292 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:34.292 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.293 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.294 DEBUG:boto:Method: GET 2022-09-03T06:47:34.294 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.295 DEBUG:boto:Data: 2022-09-03T06:47:34.295 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.296 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:34.297 DEBUG:boto:Port: 8000 2022-09-03T06:47:34.297 DEBUG:boto:Params: {} 2022-09-03T06:47:34.299 DEBUG:boto:Token: None 2022-09-03T06:47:34.300 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.300 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Esm+0GyBfg3AhmlMPY748QyDDBE= 2022-09-03T06:47:34.301 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Esm+0GyBfg3AhmlMPY748QyDDBE=', 'Content-Length': '0'} 2022-09-03T06:47:34.303 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000015dfef70ea4261c0-006312f886-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.304 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:34.305 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:34.305 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.306 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.307 DEBUG:boto:Method: HEAD 2022-09-03T06:47:34.307 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.308 DEBUG:boto:Data: 2022-09-03T06:47:34.308 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.309 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:34.309 DEBUG:boto:Port: 8001 2022-09-03T06:47:34.310 DEBUG:boto:Params: {} 2022-09-03T06:47:34.311 DEBUG:boto:Token: None 2022-09-03T06:47:34.312 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.312 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BKdaap3XitKOTKCrj2+kGIOocr8= 2022-09-03T06:47:34.313 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BKdaap3XitKOTKCrj2+kGIOocr8=', 'Content-Length': '0'} 2022-09-03T06:47:34.316 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e103dc1a04c2a675-006312f886-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.316 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:34.317 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:34.318 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.318 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.319 DEBUG:boto:Method: HEAD 2022-09-03T06:47:34.320 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.320 DEBUG:boto:Data: 2022-09-03T06:47:34.321 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.321 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:34.322 DEBUG:boto:Port: 8000 2022-09-03T06:47:34.323 DEBUG:boto:Params: {} 2022-09-03T06:47:34.324 DEBUG:boto:Token: None 2022-09-03T06:47:34.325 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.326 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BKdaap3XitKOTKCrj2+kGIOocr8= 2022-09-03T06:47:34.327 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BKdaap3XitKOTKCrj2+kGIOocr8=', 'Content-Length': '0'} 2022-09-03T06:47:34.330 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c9fb1c60621ef510-006312f886-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.331 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:47:34.332 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:34.333 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:34.334 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.334 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.335 DEBUG:boto:Method: GET 2022-09-03T06:47:34.335 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.336 DEBUG:boto:Data: 2022-09-03T06:47:34.337 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.337 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:34.338 DEBUG:boto:Port: 8001 2022-09-03T06:47:34.338 DEBUG:boto:Params: {} 2022-09-03T06:47:34.339 DEBUG:boto:Token: None 2022-09-03T06:47:34.340 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.341 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Esm+0GyBfg3AhmlMPY748QyDDBE= 2022-09-03T06:47:34.342 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Esm+0GyBfg3AhmlMPY748QyDDBE=', 'Content-Length': '0'} 2022-09-03T06:47:34.344 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000039bbd788cf98271e-006312f886-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.386 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:34.388 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:34.388 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.389 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.390 DEBUG:boto:Method: GET 2022-09-03T06:47:34.391 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.391 DEBUG:boto:Data: 2022-09-03T06:47:34.392 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.393 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:34.393 DEBUG:boto:Port: 8000 2022-09-03T06:47:34.394 DEBUG:boto:Params: {} 2022-09-03T06:47:34.394 DEBUG:boto:Token: None 2022-09-03T06:47:34.395 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:34.396 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Esm+0GyBfg3AhmlMPY748QyDDBE= 2022-09-03T06:47:34.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Esm+0GyBfg3AhmlMPY748QyDDBE=', 'Content-Length': '0'} 2022-09-03T06:47:34.399 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d725dc6810be5b3c-006312f886-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.400 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:34.400 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:34.401 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:34.401 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:34.402 DEBUG:boto:Method: GET 2022-09-03T06:47:34.403 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:47:34.403 DEBUG:boto:Data: 2022-09-03T06:47:34.404 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.404 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:34.405 DEBUG:boto:Port: 8001 2022-09-03T06:47:34.406 DEBUG:boto:Params: {} 2022-09-03T06:47:34.407 DEBUG:boto:Token: None 2022-09-03T06:47:34.407 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/ 2022-09-03T06:47:34.408 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo= 2022-09-03T06:47:34.409 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo=', 'Content-Length': '0'} 2022-09-03T06:47:34.413 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000037334454b0c9fb7d-006312f886-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.414 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:34.416 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:34.417 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:34.418 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:34.419 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:34.419 DEBUG:boto:Method: GET 2022-09-03T06:47:34.420 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:47:34.421 DEBUG:boto:Data: 2022-09-03T06:47:34.421 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.422 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:34.422 DEBUG:boto:Port: 8000 2022-09-03T06:47:34.423 DEBUG:boto:Params: {} 2022-09-03T06:47:34.423 DEBUG:boto:Token: None 2022-09-03T06:47:34.424 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/ 2022-09-03T06:47:34.425 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo= 2022-09-03T06:47:34.425 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo=', 'Content-Length': '0'} 2022-09-03T06:47:34.430 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008b90402f288cf548-006312f886-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.431 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:34.432 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:34.433 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:34.434 DEBUG:boto:Method: HEAD 2022-09-03T06:47:34.434 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:47:34.435 DEBUG:boto:Data: 2022-09-03T06:47:34.435 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.436 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:34.437 DEBUG:boto:Port: 8001 2022-09-03T06:47:34.437 DEBUG:boto:Params: {} 2022-09-03T06:47:34.438 DEBUG:boto:Token: None 2022-09-03T06:47:34.439 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj22 2022-09-03T06:47:34.439 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XREqiYyz1py08J0GsyXMGY3ve9k= 2022-09-03T06:47:34.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XREqiYyz1py08J0GsyXMGY3ve9k=', 'Content-Length': '0'} 2022-09-03T06:47:34.443 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b1bf41723fc9461c-006312f886-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.444 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:34.445 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:34.445 DEBUG:boto:Method: HEAD 2022-09-03T06:47:34.446 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:47:34.447 DEBUG:boto:Data: 2022-09-03T06:47:34.448 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.449 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:34.450 DEBUG:boto:Port: 8000 2022-09-03T06:47:34.451 DEBUG:boto:Params: {} 2022-09-03T06:47:34.452 DEBUG:boto:Token: None 2022-09-03T06:47:34.453 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj22 2022-09-03T06:47:34.454 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XREqiYyz1py08J0GsyXMGY3ve9k= 2022-09-03T06:47:34.455 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XREqiYyz1py08J0GsyXMGY3ve9k=', 'Content-Length': '0'} 2022-09-03T06:47:34.458 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f9526c373f4a7a6-006312f886-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.459 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:47:34.461 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:34.462 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:34.463 DEBUG:boto:Method: GET 2022-09-03T06:47:34.464 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:47:34.466 DEBUG:boto:Data: 2022-09-03T06:47:34.467 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.468 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:34.469 DEBUG:boto:Port: 8001 2022-09-03T06:47:34.470 DEBUG:boto:Params: {} 2022-09-03T06:47:34.471 DEBUG:boto:Token: None 2022-09-03T06:47:34.472 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj22 2022-09-03T06:47:34.473 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b+ca0GfNCwJwoVdijQCNWVdNZgA= 2022-09-03T06:47:34.475 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b+ca0GfNCwJwoVdijQCNWVdNZgA=', 'Content-Length': '0'} 2022-09-03T06:47:34.477 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000058176e6e7449d865-006312f886-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.478 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:34.479 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:34.480 DEBUG:boto:Method: GET 2022-09-03T06:47:34.481 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:47:34.481 DEBUG:boto:Data: 2022-09-03T06:47:34.482 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.483 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:34.483 DEBUG:boto:Port: 8000 2022-09-03T06:47:34.484 DEBUG:boto:Params: {} 2022-09-03T06:47:34.485 DEBUG:boto:Token: None 2022-09-03T06:47:34.486 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj22 2022-09-03T06:47:34.487 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b+ca0GfNCwJwoVdijQCNWVdNZgA= 2022-09-03T06:47:34.487 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b+ca0GfNCwJwoVdijQCNWVdNZgA=', 'Content-Length': '0'} 2022-09-03T06:47:34.493 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b68aa7e4b8017fa0-006312f886-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.534 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:34.535 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:34.536 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:34.536 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:34.537 DEBUG:boto:Method: GET 2022-09-03T06:47:34.537 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:47:34.538 DEBUG:boto:Data: 2022-09-03T06:47:34.539 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.539 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:34.540 DEBUG:boto:Port: 8001 2022-09-03T06:47:34.540 DEBUG:boto:Params: {} 2022-09-03T06:47:34.542 DEBUG:boto:Token: None 2022-09-03T06:47:34.543 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/ 2022-09-03T06:47:34.544 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo= 2022-09-03T06:47:34.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo=', 'Content-Length': '0'} 2022-09-03T06:47:34.550 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c11617421e4cd59f-006312f886-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.551 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:34.553 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:34.553 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:34.554 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:34.556 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:34.557 DEBUG:boto:Method: GET 2022-09-03T06:47:34.559 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:47:34.560 DEBUG:boto:Data: 2022-09-03T06:47:34.561 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.561 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:34.562 DEBUG:boto:Port: 8000 2022-09-03T06:47:34.563 DEBUG:boto:Params: {} 2022-09-03T06:47:34.564 DEBUG:boto:Token: None 2022-09-03T06:47:34.565 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/ 2022-09-03T06:47:34.565 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo= 2022-09-03T06:47:34.566 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo=', 'Content-Length': '0'} 2022-09-03T06:47:34.570 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d76b788f43c5b56f-006312f886-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.571 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:34.572 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:47:34.573 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:34.573 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:34.574 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.575 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.577 DEBUG:boto:Method: GET 2022-09-03T06:47:34.579 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.582 DEBUG:boto:Data: 2022-09-03T06:47:34.583 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.584 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:34.584 DEBUG:boto:Port: 8001 2022-09-03T06:47:34.585 DEBUG:boto:Params: {} 2022-09-03T06:47:34.586 DEBUG:boto:Token: None 2022-09-03T06:47:34.587 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.587 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2sFx41MRxTrW5w2x8Olg8x+7b3A= 2022-09-03T06:47:34.588 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2sFx41MRxTrW5w2x8Olg8x+7b3A=', 'Content-Length': '0'} 2022-09-03T06:47:34.608 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ab7cb56c62ea934-006312f886-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.652 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:34.653 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:34.654 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.655 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.656 DEBUG:boto:Method: GET 2022-09-03T06:47:34.657 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.658 DEBUG:boto:Data: 2022-09-03T06:47:34.659 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.660 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:34.661 DEBUG:boto:Port: 8000 2022-09-03T06:47:34.662 DEBUG:boto:Params: {} 2022-09-03T06:47:34.663 DEBUG:boto:Token: None 2022-09-03T06:47:34.664 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.665 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2sFx41MRxTrW5w2x8Olg8x+7b3A= 2022-09-03T06:47:34.666 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2sFx41MRxTrW5w2x8Olg8x+7b3A=', 'Content-Length': '0'} 2022-09-03T06:47:34.676 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e1e5f654f98a6d6b-006312f886-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.678 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:34.679 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:34.680 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.680 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.681 DEBUG:boto:Method: HEAD 2022-09-03T06:47:34.682 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.683 DEBUG:boto:Data: 2022-09-03T06:47:34.684 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.685 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:34.686 DEBUG:boto:Port: 8001 2022-09-03T06:47:34.687 DEBUG:boto:Params: {} 2022-09-03T06:47:34.688 DEBUG:boto:Token: None 2022-09-03T06:47:34.689 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.691 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jiP2YdEMTG/6bo6omxEXjqPnPPE= 2022-09-03T06:47:34.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jiP2YdEMTG/6bo6omxEXjqPnPPE=', 'Content-Length': '0'} 2022-09-03T06:47:34.695 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018963e087d2ca2d3-006312f886-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.697 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:34.697 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:34.698 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.698 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.699 DEBUG:boto:Method: HEAD 2022-09-03T06:47:34.700 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.700 DEBUG:boto:Data: 2022-09-03T06:47:34.701 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.702 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:34.702 DEBUG:boto:Port: 8000 2022-09-03T06:47:34.703 DEBUG:boto:Params: {} 2022-09-03T06:47:34.704 DEBUG:boto:Token: None 2022-09-03T06:47:34.704 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.705 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jiP2YdEMTG/6bo6omxEXjqPnPPE= 2022-09-03T06:47:34.706 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jiP2YdEMTG/6bo6omxEXjqPnPPE=', 'Content-Length': '0'} 2022-09-03T06:47:34.710 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cfbdb798a1cd8128-006312f886-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.711 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:47:34.712 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:34.713 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:34.716 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.717 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.718 DEBUG:boto:Method: GET 2022-09-03T06:47:34.719 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.720 DEBUG:boto:Data: 2022-09-03T06:47:34.721 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.722 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:34.723 DEBUG:boto:Port: 8001 2022-09-03T06:47:34.724 DEBUG:boto:Params: {} 2022-09-03T06:47:34.725 DEBUG:boto:Token: None 2022-09-03T06:47:34.726 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.728 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2sFx41MRxTrW5w2x8Olg8x+7b3A= 2022-09-03T06:47:34.729 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2sFx41MRxTrW5w2x8Olg8x+7b3A=', 'Content-Length': '0'} 2022-09-03T06:47:34.733 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007eca845e4848691d-006312f886-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.734 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:34.735 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:34.735 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.736 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.736 DEBUG:boto:Method: GET 2022-09-03T06:47:34.737 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.738 DEBUG:boto:Data: 2022-09-03T06:47:34.738 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.739 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:34.740 DEBUG:boto:Port: 8000 2022-09-03T06:47:34.740 DEBUG:boto:Params: {} 2022-09-03T06:47:34.741 DEBUG:boto:Token: None 2022-09-03T06:47:34.742 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:34.743 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2sFx41MRxTrW5w2x8Olg8x+7b3A= 2022-09-03T06:47:34.744 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2sFx41MRxTrW5w2x8Olg8x+7b3A=', 'Content-Length': '0'} 2022-09-03T06:47:34.757 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018d58b06f3df2025-006312f886-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.759 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:34.760 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:34.760 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:34.761 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:34.761 DEBUG:boto:Method: GET 2022-09-03T06:47:34.762 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:47:34.763 DEBUG:boto:Data: 2022-09-03T06:47:34.763 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.764 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:34.764 DEBUG:boto:Port: 8001 2022-09-03T06:47:34.765 DEBUG:boto:Params: {} 2022-09-03T06:47:34.767 DEBUG:boto:Token: None 2022-09-03T06:47:34.767 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/ 2022-09-03T06:47:34.768 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo= 2022-09-03T06:47:34.769 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo=', 'Content-Length': '0'} 2022-09-03T06:47:34.772 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000073ec134dd0203db2-006312f886-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.774 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:34.775 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:34.776 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:34.777 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:34.778 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:34.779 DEBUG:boto:Method: GET 2022-09-03T06:47:34.780 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:47:34.781 DEBUG:boto:Data: 2022-09-03T06:47:34.782 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.782 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:34.783 DEBUG:boto:Port: 8000 2022-09-03T06:47:34.784 DEBUG:boto:Params: {} 2022-09-03T06:47:34.786 DEBUG:boto:Token: None 2022-09-03T06:47:34.787 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/ 2022-09-03T06:47:34.788 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo= 2022-09-03T06:47:34.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo=', 'Content-Length': '0'} 2022-09-03T06:47:34.794 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009f9f19a20e1dffbf-006312f886-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.796 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:34.797 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:34.798 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:34.799 DEBUG:boto:Method: HEAD 2022-09-03T06:47:34.800 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:47:34.801 DEBUG:boto:Data: 2022-09-03T06:47:34.802 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.804 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:34.804 DEBUG:boto:Port: 8001 2022-09-03T06:47:34.805 DEBUG:boto:Params: {} 2022-09-03T06:47:34.805 DEBUG:boto:Token: None 2022-09-03T06:47:34.806 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj23 2022-09-03T06:47:34.807 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tTyaEqeIx1lz+ygj5OW9bsxCDUc= 2022-09-03T06:47:34.807 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tTyaEqeIx1lz+ygj5OW9bsxCDUc=', 'Content-Length': '0'} 2022-09-03T06:47:34.810 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007701d079d4367394-006312f886-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.811 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:34.812 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:34.812 DEBUG:boto:Method: HEAD 2022-09-03T06:47:34.813 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:47:34.813 DEBUG:boto:Data: 2022-09-03T06:47:34.814 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.815 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:34.815 DEBUG:boto:Port: 8000 2022-09-03T06:47:34.816 DEBUG:boto:Params: {} 2022-09-03T06:47:34.816 DEBUG:boto:Token: None 2022-09-03T06:47:34.817 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj23 2022-09-03T06:47:34.818 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tTyaEqeIx1lz+ygj5OW9bsxCDUc= 2022-09-03T06:47:34.818 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tTyaEqeIx1lz+ygj5OW9bsxCDUc=', 'Content-Length': '0'} 2022-09-03T06:47:34.821 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e19999d2d041bae-006312f886-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.822 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:47:34.822 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:34.823 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:34.823 DEBUG:boto:Method: GET 2022-09-03T06:47:34.824 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:47:34.825 DEBUG:boto:Data: 2022-09-03T06:47:34.825 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.826 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:34.827 DEBUG:boto:Port: 8001 2022-09-03T06:47:34.828 DEBUG:boto:Params: {} 2022-09-03T06:47:34.829 DEBUG:boto:Token: None 2022-09-03T06:47:34.830 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj23 2022-09-03T06:47:34.832 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Pwl3MbVxT6Qq+iYK7+C4Og73cPo= 2022-09-03T06:47:34.833 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Pwl3MbVxT6Qq+iYK7+C4Og73cPo=', 'Content-Length': '0'} 2022-09-03T06:47:34.835 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000028a7b8bb940405b2-006312f886-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.836 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:34.837 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:34.837 DEBUG:boto:Method: GET 2022-09-03T06:47:34.838 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:47:34.839 DEBUG:boto:Data: 2022-09-03T06:47:34.839 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.840 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:34.841 DEBUG:boto:Port: 8000 2022-09-03T06:47:34.841 DEBUG:boto:Params: {} 2022-09-03T06:47:34.842 DEBUG:boto:Token: None 2022-09-03T06:47:34.843 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj23 2022-09-03T06:47:34.843 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Pwl3MbVxT6Qq+iYK7+C4Og73cPo= 2022-09-03T06:47:34.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Pwl3MbVxT6Qq+iYK7+C4Og73cPo=', 'Content-Length': '0'} 2022-09-03T06:47:34.846 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c8bcc3229e6e6ddd-006312f886-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.894 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:34.896 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:34.897 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:34.898 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:34.899 DEBUG:boto:Method: GET 2022-09-03T06:47:34.900 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:47:34.903 DEBUG:boto:Data: 2022-09-03T06:47:34.904 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.905 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:34.906 DEBUG:boto:Port: 8001 2022-09-03T06:47:34.907 DEBUG:boto:Params: {} 2022-09-03T06:47:34.908 DEBUG:boto:Token: None 2022-09-03T06:47:34.909 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/ 2022-09-03T06:47:34.909 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo= 2022-09-03T06:47:34.910 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo=', 'Content-Length': '0'} 2022-09-03T06:47:34.913 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c920d8bcf26abc6d-006312f886-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.915 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:34.917 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:34.920 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:34.922 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:34.923 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:34.924 DEBUG:boto:Method: GET 2022-09-03T06:47:34.924 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:47:34.925 DEBUG:boto:Data: 2022-09-03T06:47:34.925 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.926 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:34.927 DEBUG:boto:Port: 8000 2022-09-03T06:47:34.928 DEBUG:boto:Params: {} 2022-09-03T06:47:34.929 DEBUG:boto:Token: None 2022-09-03T06:47:34.930 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/ 2022-09-03T06:47:34.931 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo= 2022-09-03T06:47:34.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:caF3y/enjlRS0SNlSXKYrQ0LOQo=', 'Content-Length': '0'} 2022-09-03T06:47:34.936 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b2d49c907d95f7e4-006312f886-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.945 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:34.947 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:47:34.949 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:34.950 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:34.951 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:34.952 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:34.953 DEBUG:boto:Method: GET 2022-09-03T06:47:34.954 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:34.955 DEBUG:boto:Data: 2022-09-03T06:47:34.956 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.958 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:34.959 DEBUG:boto:Port: 8001 2022-09-03T06:47:34.960 DEBUG:boto:Params: {} 2022-09-03T06:47:34.962 DEBUG:boto:Token: None 2022-09-03T06:47:34.962 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:34.963 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aVD8Kaznvl6W1JRkm/crCLY5d8Q= 2022-09-03T06:47:34.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aVD8Kaznvl6W1JRkm/crCLY5d8Q=', 'Content-Length': '0'} 2022-09-03T06:47:34.966 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c57a788941184da6-006312f886-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.967 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:34.968 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:34.968 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:34.969 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:34.970 DEBUG:boto:Method: GET 2022-09-03T06:47:34.973 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:34.973 DEBUG:boto:Data: 2022-09-03T06:47:34.974 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.974 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:34.975 DEBUG:boto:Port: 8000 2022-09-03T06:47:34.976 DEBUG:boto:Params: {} 2022-09-03T06:47:34.976 DEBUG:boto:Token: None 2022-09-03T06:47:34.978 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:34.978 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aVD8Kaznvl6W1JRkm/crCLY5d8Q= 2022-09-03T06:47:34.979 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aVD8Kaznvl6W1JRkm/crCLY5d8Q=', 'Content-Length': '0'} 2022-09-03T06:47:34.981 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c33bab1b77d1e132-006312f886-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:34 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:34.983 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:34.984 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:34.985 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:34.986 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:34.988 DEBUG:boto:Method: HEAD 2022-09-03T06:47:34.988 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:34.989 DEBUG:boto:Data: 2022-09-03T06:47:34.990 DEBUG:boto:Headers: {} 2022-09-03T06:47:34.991 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:34.992 DEBUG:boto:Port: 8001 2022-09-03T06:47:34.993 DEBUG:boto:Params: {} 2022-09-03T06:47:34.994 DEBUG:boto:Token: None 2022-09-03T06:47:34.995 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:34 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:34.997 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5vzNUqWw7aZiksoSWqU2PBT7+BE= 2022-09-03T06:47:34.999 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:34 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5vzNUqWw7aZiksoSWqU2PBT7+BE=', 'Content-Length': '0'} 2022-09-03T06:47:35.002 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007a2aafafa83f9e6e-006312f887-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.004 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:35.005 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:35.006 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:35.007 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:35.008 DEBUG:boto:Method: HEAD 2022-09-03T06:47:35.009 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:35.010 DEBUG:boto:Data: 2022-09-03T06:47:35.011 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.012 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:35.013 DEBUG:boto:Port: 8000 2022-09-03T06:47:35.014 DEBUG:boto:Params: {} 2022-09-03T06:47:35.015 DEBUG:boto:Token: None 2022-09-03T06:47:35.016 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:35.018 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:obgYJPQbbM95GjZA81rjZH/em5Y= 2022-09-03T06:47:35.019 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:obgYJPQbbM95GjZA81rjZH/em5Y=', 'Content-Length': '0'} 2022-09-03T06:47:35.021 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e8d5e7ddc37f1f7d-006312f887-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.022 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:47:35.023 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:35.023 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:35.024 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:35.024 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:35.025 DEBUG:boto:Method: GET 2022-09-03T06:47:35.026 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:35.026 DEBUG:boto:Data: 2022-09-03T06:47:35.027 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.027 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:35.028 DEBUG:boto:Port: 8001 2022-09-03T06:47:35.029 DEBUG:boto:Params: {} 2022-09-03T06:47:35.029 DEBUG:boto:Token: None 2022-09-03T06:47:35.030 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:35.031 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yr8jBtVgFy4l8a0gewhmX+VlBEw= 2022-09-03T06:47:35.031 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yr8jBtVgFy4l8a0gewhmX+VlBEw=', 'Content-Length': '0'} 2022-09-03T06:47:35.034 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c4b569c031912bc2-006312f887-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.078 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:35.080 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:35.081 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:35.082 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:35.086 DEBUG:boto:Method: GET 2022-09-03T06:47:35.087 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:35.087 DEBUG:boto:Data: 2022-09-03T06:47:35.089 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.091 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:35.094 DEBUG:boto:Port: 8000 2022-09-03T06:47:35.096 DEBUG:boto:Params: {} 2022-09-03T06:47:35.097 DEBUG:boto:Token: None 2022-09-03T06:47:35.098 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:35.099 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yr8jBtVgFy4l8a0gewhmX+VlBEw= 2022-09-03T06:47:35.100 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yr8jBtVgFy4l8a0gewhmX+VlBEw=', 'Content-Length': '0'} 2022-09-03T06:47:35.103 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007aa478de1d6691dc-006312f887-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.104 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:35.105 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:35.105 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:35.106 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:35.107 DEBUG:boto:Method: GET 2022-09-03T06:47:35.107 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:47:35.108 DEBUG:boto:Data: 2022-09-03T06:47:35.109 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.110 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:35.111 DEBUG:boto:Port: 8001 2022-09-03T06:47:35.113 DEBUG:boto:Params: {} 2022-09-03T06:47:35.114 DEBUG:boto:Token: None 2022-09-03T06:47:35.115 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/ 2022-09-03T06:47:35.116 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I= 2022-09-03T06:47:35.117 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I=', 'Content-Length': '0'} 2022-09-03T06:47:35.122 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000062ee407b03df4246-006312f887-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.123 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:35.124 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:35.125 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:35.126 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:35.127 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:35.128 DEBUG:boto:Method: GET 2022-09-03T06:47:35.129 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:47:35.129 DEBUG:boto:Data: 2022-09-03T06:47:35.130 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.131 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:35.131 DEBUG:boto:Port: 8000 2022-09-03T06:47:35.132 DEBUG:boto:Params: {} 2022-09-03T06:47:35.132 DEBUG:boto:Token: None 2022-09-03T06:47:35.133 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/ 2022-09-03T06:47:35.134 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I= 2022-09-03T06:47:35.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I=', 'Content-Length': '0'} 2022-09-03T06:47:35.140 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002174dabbf7b5d24f-006312f887-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.141 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:35.142 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:35.143 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:35.144 DEBUG:boto:Method: HEAD 2022-09-03T06:47:35.145 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:47:35.146 DEBUG:boto:Data: 2022-09-03T06:47:35.147 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.149 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:35.150 DEBUG:boto:Port: 8001 2022-09-03T06:47:35.151 DEBUG:boto:Params: {} 2022-09-03T06:47:35.152 DEBUG:boto:Token: None 2022-09-03T06:47:35.152 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj29 2022-09-03T06:47:35.153 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NE/MnfjfWTNCRXKpShb6MKxan2c= 2022-09-03T06:47:35.154 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NE/MnfjfWTNCRXKpShb6MKxan2c=', 'Content-Length': '0'} 2022-09-03T06:47:35.156 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ac9ba7e08be926a2-006312f887-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.157 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:35.158 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:35.158 DEBUG:boto:Method: HEAD 2022-09-03T06:47:35.159 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:47:35.159 DEBUG:boto:Data: 2022-09-03T06:47:35.160 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.160 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:35.161 DEBUG:boto:Port: 8000 2022-09-03T06:47:35.161 DEBUG:boto:Params: {} 2022-09-03T06:47:35.162 DEBUG:boto:Token: None 2022-09-03T06:47:35.163 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj29 2022-09-03T06:47:35.163 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NE/MnfjfWTNCRXKpShb6MKxan2c= 2022-09-03T06:47:35.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NE/MnfjfWTNCRXKpShb6MKxan2c=', 'Content-Length': '0'} 2022-09-03T06:47:35.166 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000094d8dcc8ffad1152-006312f887-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.168 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:47:35.169 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:35.171 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:35.172 DEBUG:boto:Method: GET 2022-09-03T06:47:35.173 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:47:35.174 DEBUG:boto:Data: 2022-09-03T06:47:35.175 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.176 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:35.178 DEBUG:boto:Port: 8001 2022-09-03T06:47:35.179 DEBUG:boto:Params: {} 2022-09-03T06:47:35.179 DEBUG:boto:Token: None 2022-09-03T06:47:35.180 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj29 2022-09-03T06:47:35.181 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OFVXscgnFY77Uczbl7yiofkGGSU= 2022-09-03T06:47:35.181 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OFVXscgnFY77Uczbl7yiofkGGSU=', 'Content-Length': '0'} 2022-09-03T06:47:35.183 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e0378e66bdf15540-006312f887-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.184 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:35.185 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:35.185 DEBUG:boto:Method: GET 2022-09-03T06:47:35.186 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:47:35.187 DEBUG:boto:Data: 2022-09-03T06:47:35.187 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.188 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:35.188 DEBUG:boto:Port: 8000 2022-09-03T06:47:35.189 DEBUG:boto:Params: {} 2022-09-03T06:47:35.189 DEBUG:boto:Token: None 2022-09-03T06:47:35.190 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj29 2022-09-03T06:47:35.191 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OFVXscgnFY77Uczbl7yiofkGGSU= 2022-09-03T06:47:35.191 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OFVXscgnFY77Uczbl7yiofkGGSU=', 'Content-Length': '0'} 2022-09-03T06:47:35.193 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000010b2cdaf52f09602-006312f887-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.234 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:35.235 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:35.236 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:35.237 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:35.238 DEBUG:boto:Method: GET 2022-09-03T06:47:35.240 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:47:35.241 DEBUG:boto:Data: 2022-09-03T06:47:35.242 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.243 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:35.244 DEBUG:boto:Port: 8001 2022-09-03T06:47:35.245 DEBUG:boto:Params: {} 2022-09-03T06:47:35.246 DEBUG:boto:Token: None 2022-09-03T06:47:35.247 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/ 2022-09-03T06:47:35.248 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I= 2022-09-03T06:47:35.250 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I=', 'Content-Length': '0'} 2022-09-03T06:47:35.254 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009cbbb165da0ae5cb-006312f887-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.255 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:35.256 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:35.257 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:35.257 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:35.258 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:35.258 DEBUG:boto:Method: GET 2022-09-03T06:47:35.259 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:47:35.260 DEBUG:boto:Data: 2022-09-03T06:47:35.260 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.261 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:35.262 DEBUG:boto:Port: 8000 2022-09-03T06:47:35.263 DEBUG:boto:Params: {} 2022-09-03T06:47:35.264 DEBUG:boto:Token: None 2022-09-03T06:47:35.266 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/ 2022-09-03T06:47:35.267 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I= 2022-09-03T06:47:35.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I=', 'Content-Length': '0'} 2022-09-03T06:47:35.281 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000033eb24fad5e5da1b-006312f887-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.284 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:35.286 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2022-09-03T06:47:35.287 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:35.288 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:35.288 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.289 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.290 DEBUG:boto:Method: GET 2022-09-03T06:47:35.291 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.292 DEBUG:boto:Data: 2022-09-03T06:47:35.293 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.294 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:35.295 DEBUG:boto:Port: 8001 2022-09-03T06:47:35.296 DEBUG:boto:Params: {} 2022-09-03T06:47:35.297 DEBUG:boto:Token: None 2022-09-03T06:47:35.298 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.299 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+GQPGLaKiKTG4chhjPltuJdAAcM= 2022-09-03T06:47:35.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+GQPGLaKiKTG4chhjPltuJdAAcM=', 'Content-Length': '0'} 2022-09-03T06:47:35.304 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003270a60d39899f70-006312f887-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.306 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:35.307 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:35.308 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.309 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.310 DEBUG:boto:Method: GET 2022-09-03T06:47:35.311 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.312 DEBUG:boto:Data: 2022-09-03T06:47:35.313 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.314 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:35.315 DEBUG:boto:Port: 8000 2022-09-03T06:47:35.316 DEBUG:boto:Params: {} 2022-09-03T06:47:35.318 DEBUG:boto:Token: None 2022-09-03T06:47:35.319 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.320 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+GQPGLaKiKTG4chhjPltuJdAAcM= 2022-09-03T06:47:35.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+GQPGLaKiKTG4chhjPltuJdAAcM=', 'Content-Length': '0'} 2022-09-03T06:47:35.324 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a084668726bf69dc-006312f887-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.326 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:35.327 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:35.328 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.329 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.330 DEBUG:boto:Method: HEAD 2022-09-03T06:47:35.331 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.331 DEBUG:boto:Data: 2022-09-03T06:47:35.332 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.333 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:35.335 DEBUG:boto:Port: 8001 2022-09-03T06:47:35.336 DEBUG:boto:Params: {} 2022-09-03T06:47:35.338 DEBUG:boto:Token: None 2022-09-03T06:47:35.339 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.339 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BH4cO89kbI1dFmP0Jw0LQGARjMA= 2022-09-03T06:47:35.340 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BH4cO89kbI1dFmP0Jw0LQGARjMA=', 'Content-Length': '0'} 2022-09-03T06:47:35.342 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000046dc039f1174b83d-006312f887-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.344 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:35.345 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:35.346 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.347 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.348 DEBUG:boto:Method: HEAD 2022-09-03T06:47:35.349 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.350 DEBUG:boto:Data: 2022-09-03T06:47:35.352 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.352 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:35.353 DEBUG:boto:Port: 8000 2022-09-03T06:47:35.355 DEBUG:boto:Params: {} 2022-09-03T06:47:35.356 DEBUG:boto:Token: None 2022-09-03T06:47:35.356 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.357 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BH4cO89kbI1dFmP0Jw0LQGARjMA= 2022-09-03T06:47:35.357 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BH4cO89kbI1dFmP0Jw0LQGARjMA=', 'Content-Length': '0'} 2022-09-03T06:47:35.360 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000086a9ac0d4018b86d-006312f887-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.361 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2022-09-03T06:47:35.362 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:35.363 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:35.364 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.365 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.366 DEBUG:boto:Method: GET 2022-09-03T06:47:35.366 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.367 DEBUG:boto:Data: 2022-09-03T06:47:35.367 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.368 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:35.368 DEBUG:boto:Port: 8001 2022-09-03T06:47:35.369 DEBUG:boto:Params: {} 2022-09-03T06:47:35.369 DEBUG:boto:Token: None 2022-09-03T06:47:35.370 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.371 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+GQPGLaKiKTG4chhjPltuJdAAcM= 2022-09-03T06:47:35.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+GQPGLaKiKTG4chhjPltuJdAAcM=', 'Content-Length': '0'} 2022-09-03T06:47:35.374 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f7c6bcbc521f582-006312f887-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.420 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:35.421 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:35.422 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.423 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.423 DEBUG:boto:Method: GET 2022-09-03T06:47:35.424 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.424 DEBUG:boto:Data: 2022-09-03T06:47:35.425 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.426 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:35.426 DEBUG:boto:Port: 8000 2022-09-03T06:47:35.427 DEBUG:boto:Params: {} 2022-09-03T06:47:35.427 DEBUG:boto:Token: None 2022-09-03T06:47:35.430 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:35.431 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+GQPGLaKiKTG4chhjPltuJdAAcM= 2022-09-03T06:47:35.435 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+GQPGLaKiKTG4chhjPltuJdAAcM=', 'Content-Length': '0'} 2022-09-03T06:47:35.438 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f0e10d5acae73685-006312f887-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.439 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:35.439 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:35.440 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:35.442 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:35.443 DEBUG:boto:Method: GET 2022-09-03T06:47:35.445 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:47:35.446 DEBUG:boto:Data: 2022-09-03T06:47:35.447 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.448 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:35.450 DEBUG:boto:Port: 8001 2022-09-03T06:47:35.451 DEBUG:boto:Params: {} 2022-09-03T06:47:35.452 DEBUG:boto:Token: None 2022-09-03T06:47:35.452 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/ 2022-09-03T06:47:35.453 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I= 2022-09-03T06:47:35.454 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I=', 'Content-Length': '0'} 2022-09-03T06:47:35.457 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000051bc6e240cd59cd7-006312f887-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.458 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:35.459 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:35.460 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:35.461 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:35.461 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:35.463 DEBUG:boto:Method: GET 2022-09-03T06:47:35.464 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:47:35.465 DEBUG:boto:Data: 2022-09-03T06:47:35.466 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.467 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:35.468 DEBUG:boto:Port: 8000 2022-09-03T06:47:35.469 DEBUG:boto:Params: {} 2022-09-03T06:47:35.470 DEBUG:boto:Token: None 2022-09-03T06:47:35.471 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/ 2022-09-03T06:47:35.472 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I= 2022-09-03T06:47:35.473 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I=', 'Content-Length': '0'} 2022-09-03T06:47:35.478 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007e9da68945cc4a5d-006312f887-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.480 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:35.482 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:35.483 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:35.484 DEBUG:boto:Method: HEAD 2022-09-03T06:47:35.485 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:47:35.486 DEBUG:boto:Data: 2022-09-03T06:47:35.487 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.488 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:35.488 DEBUG:boto:Port: 8001 2022-09-03T06:47:35.489 DEBUG:boto:Params: {} 2022-09-03T06:47:35.490 DEBUG:boto:Token: None 2022-09-03T06:47:35.491 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj31 2022-09-03T06:47:35.492 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FfK3scJrgqNb0sJsn1bwDLY5uN0= 2022-09-03T06:47:35.493 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FfK3scJrgqNb0sJsn1bwDLY5uN0=', 'Content-Length': '0'} 2022-09-03T06:47:35.496 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006c2aada19b318c60-006312f887-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.498 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:35.499 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:35.500 DEBUG:boto:Method: HEAD 2022-09-03T06:47:35.501 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:47:35.502 DEBUG:boto:Data: 2022-09-03T06:47:35.503 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.504 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:35.505 DEBUG:boto:Port: 8000 2022-09-03T06:47:35.506 DEBUG:boto:Params: {} 2022-09-03T06:47:35.507 DEBUG:boto:Token: None 2022-09-03T06:47:35.507 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj31 2022-09-03T06:47:35.508 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FfK3scJrgqNb0sJsn1bwDLY5uN0= 2022-09-03T06:47:35.509 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FfK3scJrgqNb0sJsn1bwDLY5uN0=', 'Content-Length': '0'} 2022-09-03T06:47:35.513 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c5e2e9c3d139f87a-006312f887-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.515 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2022-09-03T06:47:35.515 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:35.516 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:35.517 DEBUG:boto:Method: GET 2022-09-03T06:47:35.518 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:47:35.519 DEBUG:boto:Data: 2022-09-03T06:47:35.519 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.520 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:35.521 DEBUG:boto:Port: 8001 2022-09-03T06:47:35.521 DEBUG:boto:Params: {} 2022-09-03T06:47:35.523 DEBUG:boto:Token: None 2022-09-03T06:47:35.524 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj31 2022-09-03T06:47:35.525 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bqcS328QDs0LEeQBRMFnlQ9m3ms= 2022-09-03T06:47:35.526 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bqcS328QDs0LEeQBRMFnlQ9m3ms=', 'Content-Length': '0'} 2022-09-03T06:47:35.528 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001f3899757ce3de23-006312f887-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.529 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:35.529 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:35.530 DEBUG:boto:Method: GET 2022-09-03T06:47:35.531 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:47:35.532 DEBUG:boto:Data: 2022-09-03T06:47:35.532 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.533 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:35.534 DEBUG:boto:Port: 8000 2022-09-03T06:47:35.535 DEBUG:boto:Params: {} 2022-09-03T06:47:35.535 DEBUG:boto:Token: None 2022-09-03T06:47:35.537 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj31 2022-09-03T06:47:35.538 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bqcS328QDs0LEeQBRMFnlQ9m3ms= 2022-09-03T06:47:35.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bqcS328QDs0LEeQBRMFnlQ9m3ms=', 'Content-Length': '0'} 2022-09-03T06:47:35.543 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000041cfa0e34559f4b7-006312f887-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.544 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:35.544 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:35.545 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:35.545 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:35.546 DEBUG:boto:Method: GET 2022-09-03T06:47:35.547 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:47:35.547 DEBUG:boto:Data: 2022-09-03T06:47:35.548 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.548 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:35.549 DEBUG:boto:Port: 8001 2022-09-03T06:47:35.550 DEBUG:boto:Params: {} 2022-09-03T06:47:35.551 DEBUG:boto:Token: None 2022-09-03T06:47:35.552 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/ 2022-09-03T06:47:35.552 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I= 2022-09-03T06:47:35.553 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I=', 'Content-Length': '0'} 2022-09-03T06:47:35.556 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007158f5e4498cbe0f-006312f887-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.557 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:35.559 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:35.560 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:35.561 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:35.561 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:35.562 DEBUG:boto:Method: GET 2022-09-03T06:47:35.563 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:47:35.564 DEBUG:boto:Data: 2022-09-03T06:47:35.565 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.566 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:35.568 DEBUG:boto:Port: 8000 2022-09-03T06:47:35.568 DEBUG:boto:Params: {} 2022-09-03T06:47:35.569 DEBUG:boto:Token: None 2022-09-03T06:47:35.570 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/ 2022-09-03T06:47:35.570 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I= 2022-09-03T06:47:35.571 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I=', 'Content-Length': '0'} 2022-09-03T06:47:35.576 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f9045f16b7f01336-006312f887-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.577 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:35.579 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:47:35.580 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:35.580 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:35.581 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.582 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.583 DEBUG:boto:Method: GET 2022-09-03T06:47:35.584 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.585 DEBUG:boto:Data: 2022-09-03T06:47:35.586 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.587 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:35.587 DEBUG:boto:Port: 8001 2022-09-03T06:47:35.588 DEBUG:boto:Params: {} 2022-09-03T06:47:35.588 DEBUG:boto:Token: None 2022-09-03T06:47:35.589 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.590 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LlNKFVyib0i1iG+6QeYNUdpzit8= 2022-09-03T06:47:35.590 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LlNKFVyib0i1iG+6QeYNUdpzit8=', 'Content-Length': '0'} 2022-09-03T06:47:35.592 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005df1a880d1161f3f-006312f887-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.594 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:35.595 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:35.597 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.597 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.598 DEBUG:boto:Method: GET 2022-09-03T06:47:35.598 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.599 DEBUG:boto:Data: 2022-09-03T06:47:35.599 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.600 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:35.602 DEBUG:boto:Port: 8000 2022-09-03T06:47:35.602 DEBUG:boto:Params: {} 2022-09-03T06:47:35.603 DEBUG:boto:Token: None 2022-09-03T06:47:35.604 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.604 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LlNKFVyib0i1iG+6QeYNUdpzit8= 2022-09-03T06:47:35.605 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LlNKFVyib0i1iG+6QeYNUdpzit8=', 'Content-Length': '0'} 2022-09-03T06:47:35.607 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000051fc45e7cab347c1-006312f887-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.653 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:35.654 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:35.655 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.656 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.658 DEBUG:boto:Method: HEAD 2022-09-03T06:47:35.661 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.662 DEBUG:boto:Data: 2022-09-03T06:47:35.662 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.663 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:35.663 DEBUG:boto:Port: 8001 2022-09-03T06:47:35.664 DEBUG:boto:Params: {} 2022-09-03T06:47:35.665 DEBUG:boto:Token: None 2022-09-03T06:47:35.665 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.666 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CM9vn3ybJlrIVEEdN0BZrWfnrrc= 2022-09-03T06:47:35.667 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CM9vn3ybJlrIVEEdN0BZrWfnrrc=', 'Content-Length': '0'} 2022-09-03T06:47:35.670 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000041b1665a44a8edfe-006312f887-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.671 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:35.672 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:35.673 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.673 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.674 DEBUG:boto:Method: HEAD 2022-09-03T06:47:35.675 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.676 DEBUG:boto:Data: 2022-09-03T06:47:35.677 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.678 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:35.679 DEBUG:boto:Port: 8000 2022-09-03T06:47:35.679 DEBUG:boto:Params: {} 2022-09-03T06:47:35.742 DEBUG:boto:Token: None 2022-09-03T06:47:35.743 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.744 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CM9vn3ybJlrIVEEdN0BZrWfnrrc= 2022-09-03T06:47:35.745 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CM9vn3ybJlrIVEEdN0BZrWfnrrc=', 'Content-Length': '0'} 2022-09-03T06:47:35.748 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000719c8f4bf497d1b-006312f887-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.750 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:47:35.751 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:35.752 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:35.752 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.753 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.754 DEBUG:boto:Method: GET 2022-09-03T06:47:35.755 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.756 DEBUG:boto:Data: 2022-09-03T06:47:35.757 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.758 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:35.759 DEBUG:boto:Port: 8001 2022-09-03T06:47:35.760 DEBUG:boto:Params: {} 2022-09-03T06:47:35.761 DEBUG:boto:Token: None 2022-09-03T06:47:35.762 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.763 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LlNKFVyib0i1iG+6QeYNUdpzit8= 2022-09-03T06:47:35.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LlNKFVyib0i1iG+6QeYNUdpzit8=', 'Content-Length': '0'} 2022-09-03T06:47:35.767 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000064ea85b62e38785d-006312f887-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.769 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:35.770 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:35.771 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.771 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.772 DEBUG:boto:Method: GET 2022-09-03T06:47:35.772 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.773 DEBUG:boto:Data: 2022-09-03T06:47:35.773 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.774 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:35.774 DEBUG:boto:Port: 8000 2022-09-03T06:47:35.775 DEBUG:boto:Params: {} 2022-09-03T06:47:35.776 DEBUG:boto:Token: None 2022-09-03T06:47:35.776 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:35.777 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LlNKFVyib0i1iG+6QeYNUdpzit8= 2022-09-03T06:47:35.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LlNKFVyib0i1iG+6QeYNUdpzit8=', 'Content-Length': '0'} 2022-09-03T06:47:35.780 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d696691cff4ab9c-006312f887-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.781 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:35.781 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:35.782 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:35.782 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:35.783 DEBUG:boto:Method: GET 2022-09-03T06:47:35.784 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:47:35.784 DEBUG:boto:Data: 2022-09-03T06:47:35.785 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.785 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:35.786 DEBUG:boto:Port: 8001 2022-09-03T06:47:35.786 DEBUG:boto:Params: {} 2022-09-03T06:47:35.787 DEBUG:boto:Token: None 2022-09-03T06:47:35.788 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/ 2022-09-03T06:47:35.788 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I= 2022-09-03T06:47:35.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I=', 'Content-Length': '0'} 2022-09-03T06:47:35.792 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000262a61ba38431835-006312f887-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.793 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:35.794 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:35.794 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:35.795 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:35.795 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:35.796 DEBUG:boto:Method: GET 2022-09-03T06:47:35.796 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:47:35.797 DEBUG:boto:Data: 2022-09-03T06:47:35.798 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.799 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:35.800 DEBUG:boto:Port: 8000 2022-09-03T06:47:35.801 DEBUG:boto:Params: {} 2022-09-03T06:47:35.801 DEBUG:boto:Token: None 2022-09-03T06:47:35.802 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/ 2022-09-03T06:47:35.804 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I= 2022-09-03T06:47:35.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I=', 'Content-Length': '0'} 2022-09-03T06:47:35.809 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000432608da5f87687c-006312f887-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.810 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:35.811 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:35.811 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:35.812 DEBUG:boto:Method: HEAD 2022-09-03T06:47:35.812 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:47:35.813 DEBUG:boto:Data: 2022-09-03T06:47:35.813 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.814 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:35.814 DEBUG:boto:Port: 8001 2022-09-03T06:47:35.815 DEBUG:boto:Params: {} 2022-09-03T06:47:35.819 DEBUG:boto:Token: None 2022-09-03T06:47:35.819 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj33 2022-09-03T06:47:35.820 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SMP9Td4Q2fxhj/u0klf5yWrSpc4= 2022-09-03T06:47:35.821 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SMP9Td4Q2fxhj/u0klf5yWrSpc4=', 'Content-Length': '0'} 2022-09-03T06:47:35.823 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000803bd56baf8402a6-006312f887-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.824 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:35.825 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:35.825 DEBUG:boto:Method: HEAD 2022-09-03T06:47:35.827 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:47:35.828 DEBUG:boto:Data: 2022-09-03T06:47:35.830 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.831 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:35.832 DEBUG:boto:Port: 8000 2022-09-03T06:47:35.833 DEBUG:boto:Params: {} 2022-09-03T06:47:35.834 DEBUG:boto:Token: None 2022-09-03T06:47:35.835 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj33 2022-09-03T06:47:35.836 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SMP9Td4Q2fxhj/u0klf5yWrSpc4= 2022-09-03T06:47:35.837 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SMP9Td4Q2fxhj/u0klf5yWrSpc4=', 'Content-Length': '0'} 2022-09-03T06:47:35.840 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000799474089c4bea8e-006312f887-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.841 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:47:35.841 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:35.842 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:35.843 DEBUG:boto:Method: GET 2022-09-03T06:47:35.843 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:47:35.844 DEBUG:boto:Data: 2022-09-03T06:47:35.844 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.845 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:35.845 DEBUG:boto:Port: 8001 2022-09-03T06:47:35.846 DEBUG:boto:Params: {} 2022-09-03T06:47:35.847 DEBUG:boto:Token: None 2022-09-03T06:47:35.847 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj33 2022-09-03T06:47:35.848 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3ls30g6m1776gav81xJ/SZfr+54= 2022-09-03T06:47:35.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3ls30g6m1776gav81xJ/SZfr+54=', 'Content-Length': '0'} 2022-09-03T06:47:35.850 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000000f6cc6091ecb808-006312f887-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.894 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:35.895 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:35.896 DEBUG:boto:Method: GET 2022-09-03T06:47:35.896 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:47:35.897 DEBUG:boto:Data: 2022-09-03T06:47:35.898 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.898 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:35.899 DEBUG:boto:Port: 8000 2022-09-03T06:47:35.899 DEBUG:boto:Params: {} 2022-09-03T06:47:35.900 DEBUG:boto:Token: None 2022-09-03T06:47:35.901 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj33 2022-09-03T06:47:35.901 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3ls30g6m1776gav81xJ/SZfr+54= 2022-09-03T06:47:35.902 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3ls30g6m1776gav81xJ/SZfr+54=', 'Content-Length': '0'} 2022-09-03T06:47:35.905 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005d4f1492f06491cc-006312f887-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.905 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:35.906 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:35.907 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:35.907 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:35.908 DEBUG:boto:Method: GET 2022-09-03T06:47:35.909 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:47:35.909 DEBUG:boto:Data: 2022-09-03T06:47:35.910 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.911 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:35.911 DEBUG:boto:Port: 8001 2022-09-03T06:47:35.912 DEBUG:boto:Params: {} 2022-09-03T06:47:35.912 DEBUG:boto:Token: None 2022-09-03T06:47:35.913 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/ 2022-09-03T06:47:35.914 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I= 2022-09-03T06:47:35.915 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I=', 'Content-Length': '0'} 2022-09-03T06:47:35.919 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e4bf748e180017d1-006312f887-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.920 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:35.922 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:35.923 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:35.924 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:35.925 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:35.926 DEBUG:boto:Method: GET 2022-09-03T06:47:35.927 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:47:35.928 DEBUG:boto:Data: 2022-09-03T06:47:35.929 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.930 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:35.931 DEBUG:boto:Port: 8000 2022-09-03T06:47:35.931 DEBUG:boto:Params: {} 2022-09-03T06:47:35.933 DEBUG:boto:Token: None 2022-09-03T06:47:35.933 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/ 2022-09-03T06:47:35.934 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I= 2022-09-03T06:47:35.935 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jk5hEt1wq2BChRl0jGOW5vvnF2I=', 'Content-Length': '0'} 2022-09-03T06:47:35.940 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000046d20641f1eae451-006312f887-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.942 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:35.943 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:47:35.944 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:35.944 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:35.945 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:35.945 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:35.946 DEBUG:boto:Method: GET 2022-09-03T06:47:35.947 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:35.947 DEBUG:boto:Data: 2022-09-03T06:47:35.948 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.948 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:35.949 DEBUG:boto:Port: 8001 2022-09-03T06:47:35.949 DEBUG:boto:Params: {} 2022-09-03T06:47:35.950 DEBUG:boto:Token: None 2022-09-03T06:47:35.951 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:35.951 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jKpkMxnPv+yGX79AXBVvzU3SRsc= 2022-09-03T06:47:35.952 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jKpkMxnPv+yGX79AXBVvzU3SRsc=', 'Content-Length': '0'} 2022-09-03T06:47:35.954 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000933396c5f31349c4-006312f887-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.955 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:35.956 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:35.956 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:35.957 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:35.957 DEBUG:boto:Method: GET 2022-09-03T06:47:35.958 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:35.958 DEBUG:boto:Data: 2022-09-03T06:47:35.959 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.959 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:35.960 DEBUG:boto:Port: 8000 2022-09-03T06:47:35.960 DEBUG:boto:Params: {} 2022-09-03T06:47:35.961 DEBUG:boto:Token: None 2022-09-03T06:47:35.962 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:35.963 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jKpkMxnPv+yGX79AXBVvzU3SRsc= 2022-09-03T06:47:35.963 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jKpkMxnPv+yGX79AXBVvzU3SRsc=', 'Content-Length': '0'} 2022-09-03T06:47:35.967 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bb1c129cf3a6857f-006312f887-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.968 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:35.968 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:35.969 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:35.969 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:35.970 DEBUG:boto:Method: HEAD 2022-09-03T06:47:35.970 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:35.971 DEBUG:boto:Data: 2022-09-03T06:47:35.971 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.972 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:35.972 DEBUG:boto:Port: 8001 2022-09-03T06:47:35.973 DEBUG:boto:Params: {} 2022-09-03T06:47:35.974 DEBUG:boto:Token: None 2022-09-03T06:47:35.974 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:35.975 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vUtC0rXweKx2mj6iB66QYhiSQNg= 2022-09-03T06:47:35.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vUtC0rXweKx2mj6iB66QYhiSQNg=', 'Content-Length': '0'} 2022-09-03T06:47:35.978 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad4cd454e8df9e85-006312f887-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.979 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:35.980 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:35.980 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:35.981 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:35.982 DEBUG:boto:Method: HEAD 2022-09-03T06:47:35.982 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:35.983 DEBUG:boto:Data: 2022-09-03T06:47:35.984 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.984 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:35.985 DEBUG:boto:Port: 8000 2022-09-03T06:47:35.986 DEBUG:boto:Params: {} 2022-09-03T06:47:35.986 DEBUG:boto:Token: None 2022-09-03T06:47:35.987 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:35 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:35.987 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vUtC0rXweKx2mj6iB66QYhiSQNg= 2022-09-03T06:47:35.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:35 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vUtC0rXweKx2mj6iB66QYhiSQNg=', 'Content-Length': '0'} 2022-09-03T06:47:35.990 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000045db33b75cd4e625-006312f887-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:35 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:35.992 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:47:35.993 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:35.994 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:35.995 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:35.996 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:35.997 DEBUG:boto:Method: GET 2022-09-03T06:47:35.998 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:35.998 DEBUG:boto:Data: 2022-09-03T06:47:35.999 DEBUG:boto:Headers: {} 2022-09-03T06:47:35.999 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:36.000 DEBUG:boto:Port: 8001 2022-09-03T06:47:36.000 DEBUG:boto:Params: {} 2022-09-03T06:47:36.001 DEBUG:boto:Token: None 2022-09-03T06:47:36.002 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:36.002 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fKa7nZKPO8gk/eUjOSMtxQObV1g= 2022-09-03T06:47:36.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fKa7nZKPO8gk/eUjOSMtxQObV1g=', 'Content-Length': '0'} 2022-09-03T06:47:36.005 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008c1fa8b2b47c765e-006312f888-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.046 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:36.047 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:36.047 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:36.048 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:36.048 DEBUG:boto:Method: GET 2022-09-03T06:47:36.049 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:36.049 DEBUG:boto:Data: 2022-09-03T06:47:36.050 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.051 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:36.051 DEBUG:boto:Port: 8000 2022-09-03T06:47:36.052 DEBUG:boto:Params: {} 2022-09-03T06:47:36.052 DEBUG:boto:Token: None 2022-09-03T06:47:36.053 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:36.054 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fKa7nZKPO8gk/eUjOSMtxQObV1g= 2022-09-03T06:47:36.054 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fKa7nZKPO8gk/eUjOSMtxQObV1g=', 'Content-Length': '0'} 2022-09-03T06:47:36.057 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f8f58e3b1ec76134-006312f888-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.059 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:36.059 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:36.060 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:36.060 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:36.061 DEBUG:boto:Method: GET 2022-09-03T06:47:36.061 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:47:36.062 DEBUG:boto:Data: 2022-09-03T06:47:36.063 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.063 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:36.064 DEBUG:boto:Port: 8001 2022-09-03T06:47:36.065 DEBUG:boto:Params: {} 2022-09-03T06:47:36.066 DEBUG:boto:Token: None 2022-09-03T06:47:36.067 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/ 2022-09-03T06:47:36.068 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg= 2022-09-03T06:47:36.069 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg=', 'Content-Length': '0'} 2022-09-03T06:47:36.072 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cb86dc9157dd97d0-006312f888-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.074 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:36.075 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:36.076 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:36.077 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:36.078 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:36.079 DEBUG:boto:Method: GET 2022-09-03T06:47:36.079 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:47:36.080 DEBUG:boto:Data: 2022-09-03T06:47:36.080 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.081 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:36.081 DEBUG:boto:Port: 8000 2022-09-03T06:47:36.083 DEBUG:boto:Params: {} 2022-09-03T06:47:36.084 DEBUG:boto:Token: None 2022-09-03T06:47:36.085 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/ 2022-09-03T06:47:36.086 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg= 2022-09-03T06:47:36.087 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg=', 'Content-Length': '0'} 2022-09-03T06:47:36.091 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005535c60b25e43e2d-006312f888-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.092 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:36.093 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:36.094 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:36.094 DEBUG:boto:Method: HEAD 2022-09-03T06:47:36.095 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:47:36.095 DEBUG:boto:Data: 2022-09-03T06:47:36.096 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.096 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:36.097 DEBUG:boto:Port: 8001 2022-09-03T06:47:36.097 DEBUG:boto:Params: {} 2022-09-03T06:47:36.098 DEBUG:boto:Token: None 2022-09-03T06:47:36.099 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj34 2022-09-03T06:47:36.099 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p3n9cl5/W+J1VbtXfB/pyIOqxGE= 2022-09-03T06:47:36.100 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p3n9cl5/W+J1VbtXfB/pyIOqxGE=', 'Content-Length': '0'} 2022-09-03T06:47:36.102 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e6f6de41171b40d5-006312f888-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.103 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:36.103 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:36.104 DEBUG:boto:Method: HEAD 2022-09-03T06:47:36.104 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:47:36.105 DEBUG:boto:Data: 2022-09-03T06:47:36.106 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.106 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:36.107 DEBUG:boto:Port: 8000 2022-09-03T06:47:36.107 DEBUG:boto:Params: {} 2022-09-03T06:47:36.108 DEBUG:boto:Token: None 2022-09-03T06:47:36.108 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj34 2022-09-03T06:47:36.109 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p3n9cl5/W+J1VbtXfB/pyIOqxGE= 2022-09-03T06:47:36.110 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p3n9cl5/W+J1VbtXfB/pyIOqxGE=', 'Content-Length': '0'} 2022-09-03T06:47:36.112 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018efa32b55c4c84d-006312f888-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.113 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:47:36.113 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:36.114 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:36.114 DEBUG:boto:Method: GET 2022-09-03T06:47:36.115 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:47:36.116 DEBUG:boto:Data: 2022-09-03T06:47:36.116 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.117 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:36.117 DEBUG:boto:Port: 8001 2022-09-03T06:47:36.118 DEBUG:boto:Params: {} 2022-09-03T06:47:36.119 DEBUG:boto:Token: None 2022-09-03T06:47:36.119 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj34 2022-09-03T06:47:36.120 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xTgBWgruraV9tSLIYLYH5BVqGKk= 2022-09-03T06:47:36.120 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xTgBWgruraV9tSLIYLYH5BVqGKk=', 'Content-Length': '0'} 2022-09-03T06:47:36.123 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b0a07685ab0b5573-006312f888-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.123 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:36.124 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:36.124 DEBUG:boto:Method: GET 2022-09-03T06:47:36.125 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:47:36.126 DEBUG:boto:Data: 2022-09-03T06:47:36.126 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.127 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:36.128 DEBUG:boto:Port: 8000 2022-09-03T06:47:36.128 DEBUG:boto:Params: {} 2022-09-03T06:47:36.129 DEBUG:boto:Token: None 2022-09-03T06:47:36.129 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj34 2022-09-03T06:47:36.130 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xTgBWgruraV9tSLIYLYH5BVqGKk= 2022-09-03T06:47:36.131 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xTgBWgruraV9tSLIYLYH5BVqGKk=', 'Content-Length': '0'} 2022-09-03T06:47:36.133 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000038e0e8e2082fb4f7-006312f888-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.133 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:36.134 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:36.135 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:36.135 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:36.136 DEBUG:boto:Method: GET 2022-09-03T06:47:36.136 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:47:36.137 DEBUG:boto:Data: 2022-09-03T06:47:36.137 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.138 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:36.139 DEBUG:boto:Port: 8001 2022-09-03T06:47:36.139 DEBUG:boto:Params: {} 2022-09-03T06:47:36.142 DEBUG:boto:Token: None 2022-09-03T06:47:36.142 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/ 2022-09-03T06:47:36.143 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg= 2022-09-03T06:47:36.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg=', 'Content-Length': '0'} 2022-09-03T06:47:36.147 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000cc66085b4bc85d7-006312f888-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.195 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:36.197 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:36.199 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:36.200 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:36.201 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:36.202 DEBUG:boto:Method: GET 2022-09-03T06:47:36.203 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:47:36.204 DEBUG:boto:Data: 2022-09-03T06:47:36.205 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.208 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:36.209 DEBUG:boto:Port: 8000 2022-09-03T06:47:36.210 DEBUG:boto:Params: {} 2022-09-03T06:47:36.211 DEBUG:boto:Token: None 2022-09-03T06:47:36.212 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/ 2022-09-03T06:47:36.213 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg= 2022-09-03T06:47:36.214 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg=', 'Content-Length': '0'} 2022-09-03T06:47:36.219 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ed5a0bffb02efbf3-006312f888-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.220 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:36.222 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:47:36.223 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:36.224 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:36.225 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.226 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.227 DEBUG:boto:Method: GET 2022-09-03T06:47:36.228 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.229 DEBUG:boto:Data: 2022-09-03T06:47:36.230 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.231 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:36.232 DEBUG:boto:Port: 8001 2022-09-03T06:47:36.233 DEBUG:boto:Params: {} 2022-09-03T06:47:36.234 DEBUG:boto:Token: None 2022-09-03T06:47:36.236 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.237 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oYLJU1/VcGRxei4BWfsvC3M82cQ= 2022-09-03T06:47:36.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oYLJU1/VcGRxei4BWfsvC3M82cQ=', 'Content-Length': '0'} 2022-09-03T06:47:36.240 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dddfadba18921b7c-006312f888-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.241 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:36.241 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:36.242 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.243 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.243 DEBUG:boto:Method: GET 2022-09-03T06:47:36.244 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.245 DEBUG:boto:Data: 2022-09-03T06:47:36.245 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.246 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:36.247 DEBUG:boto:Port: 8000 2022-09-03T06:47:36.247 DEBUG:boto:Params: {} 2022-09-03T06:47:36.248 DEBUG:boto:Token: None 2022-09-03T06:47:36.249 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.249 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oYLJU1/VcGRxei4BWfsvC3M82cQ= 2022-09-03T06:47:36.250 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oYLJU1/VcGRxei4BWfsvC3M82cQ=', 'Content-Length': '0'} 2022-09-03T06:47:36.252 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000046b7f2a51257e1d4-006312f888-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.294 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:36.295 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:36.296 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.296 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.297 DEBUG:boto:Method: HEAD 2022-09-03T06:47:36.298 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.299 DEBUG:boto:Data: 2022-09-03T06:47:36.300 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.302 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:36.304 DEBUG:boto:Port: 8001 2022-09-03T06:47:36.305 DEBUG:boto:Params: {} 2022-09-03T06:47:36.305 DEBUG:boto:Token: None 2022-09-03T06:47:36.306 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.307 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mUhetrC6QK6i+c8fqiWWnUXF4Kc= 2022-09-03T06:47:36.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mUhetrC6QK6i+c8fqiWWnUXF4Kc=', 'Content-Length': '0'} 2022-09-03T06:47:36.310 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006cab45e18e464f64-006312f888-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.311 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:36.311 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:36.312 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.312 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.314 DEBUG:boto:Method: HEAD 2022-09-03T06:47:36.316 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.316 DEBUG:boto:Data: 2022-09-03T06:47:36.317 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.317 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:36.318 DEBUG:boto:Port: 8000 2022-09-03T06:47:36.319 DEBUG:boto:Params: {} 2022-09-03T06:47:36.319 DEBUG:boto:Token: None 2022-09-03T06:47:36.320 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.321 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mUhetrC6QK6i+c8fqiWWnUXF4Kc= 2022-09-03T06:47:36.322 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mUhetrC6QK6i+c8fqiWWnUXF4Kc=', 'Content-Length': '0'} 2022-09-03T06:47:36.324 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000077e1fe31cc68d3aa-006312f888-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.325 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:47:36.325 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:36.326 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:36.327 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.328 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.329 DEBUG:boto:Method: GET 2022-09-03T06:47:36.330 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.331 DEBUG:boto:Data: 2022-09-03T06:47:36.331 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.332 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:36.332 DEBUG:boto:Port: 8001 2022-09-03T06:47:36.333 DEBUG:boto:Params: {} 2022-09-03T06:47:36.334 DEBUG:boto:Token: None 2022-09-03T06:47:36.335 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.335 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oYLJU1/VcGRxei4BWfsvC3M82cQ= 2022-09-03T06:47:36.336 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oYLJU1/VcGRxei4BWfsvC3M82cQ=', 'Content-Length': '0'} 2022-09-03T06:47:36.339 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db06b844c0400419-006312f888-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.340 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:36.340 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:36.341 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.342 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.343 DEBUG:boto:Method: GET 2022-09-03T06:47:36.343 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.344 DEBUG:boto:Data: 2022-09-03T06:47:36.344 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.345 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:36.346 DEBUG:boto:Port: 8000 2022-09-03T06:47:36.347 DEBUG:boto:Params: {} 2022-09-03T06:47:36.353 DEBUG:boto:Token: None 2022-09-03T06:47:36.354 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:36.354 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oYLJU1/VcGRxei4BWfsvC3M82cQ= 2022-09-03T06:47:36.355 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oYLJU1/VcGRxei4BWfsvC3M82cQ=', 'Content-Length': '0'} 2022-09-03T06:47:36.362 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008fa5ca535c779586-006312f888-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.364 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:36.364 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:36.365 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:36.366 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:36.366 DEBUG:boto:Method: GET 2022-09-03T06:47:36.367 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:47:36.367 DEBUG:boto:Data: 2022-09-03T06:47:36.368 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.369 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:36.369 DEBUG:boto:Port: 8001 2022-09-03T06:47:36.370 DEBUG:boto:Params: {} 2022-09-03T06:47:36.371 DEBUG:boto:Token: None 2022-09-03T06:47:36.371 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/ 2022-09-03T06:47:36.372 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg= 2022-09-03T06:47:36.373 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg=', 'Content-Length': '0'} 2022-09-03T06:47:36.376 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e4306d2c3de0ceea-006312f888-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.377 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:36.378 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:36.379 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:36.380 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:36.380 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:36.381 DEBUG:boto:Method: GET 2022-09-03T06:47:36.381 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:47:36.382 DEBUG:boto:Data: 2022-09-03T06:47:36.382 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.383 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:36.383 DEBUG:boto:Port: 8000 2022-09-03T06:47:36.384 DEBUG:boto:Params: {} 2022-09-03T06:47:36.384 DEBUG:boto:Token: None 2022-09-03T06:47:36.385 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/ 2022-09-03T06:47:36.386 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg= 2022-09-03T06:47:36.386 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg=', 'Content-Length': '0'} 2022-09-03T06:47:36.391 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e1df9420b3712cb7-006312f888-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.392 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:36.393 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:36.394 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:36.395 DEBUG:boto:Method: HEAD 2022-09-03T06:47:36.395 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:47:36.396 DEBUG:boto:Data: 2022-09-03T06:47:36.396 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.397 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:36.397 DEBUG:boto:Port: 8001 2022-09-03T06:47:36.398 DEBUG:boto:Params: {} 2022-09-03T06:47:36.399 DEBUG:boto:Token: None 2022-09-03T06:47:36.399 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj38 2022-09-03T06:47:36.400 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aGOXMb0Mm4k0e8R8/bpZsWNYk50= 2022-09-03T06:47:36.401 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aGOXMb0Mm4k0e8R8/bpZsWNYk50=', 'Content-Length': '0'} 2022-09-03T06:47:36.403 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001419c4e4ff346f76-006312f888-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.404 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:36.405 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:36.405 DEBUG:boto:Method: HEAD 2022-09-03T06:47:36.406 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:47:36.407 DEBUG:boto:Data: 2022-09-03T06:47:36.407 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.408 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:36.408 DEBUG:boto:Port: 8000 2022-09-03T06:47:36.409 DEBUG:boto:Params: {} 2022-09-03T06:47:36.411 DEBUG:boto:Token: None 2022-09-03T06:47:36.412 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj38 2022-09-03T06:47:36.412 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aGOXMb0Mm4k0e8R8/bpZsWNYk50= 2022-09-03T06:47:36.413 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aGOXMb0Mm4k0e8R8/bpZsWNYk50=', 'Content-Length': '0'} 2022-09-03T06:47:36.415 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000137c394f333c951e-006312f888-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.416 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:47:36.417 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:36.417 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:36.418 DEBUG:boto:Method: GET 2022-09-03T06:47:36.419 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:47:36.419 DEBUG:boto:Data: 2022-09-03T06:47:36.420 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.420 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:36.421 DEBUG:boto:Port: 8001 2022-09-03T06:47:36.421 DEBUG:boto:Params: {} 2022-09-03T06:47:36.422 DEBUG:boto:Token: None 2022-09-03T06:47:36.422 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj38 2022-09-03T06:47:36.423 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:np1Pj2gXMVuqV+XKEEnW1ZnmKeQ= 2022-09-03T06:47:36.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:np1Pj2gXMVuqV+XKEEnW1ZnmKeQ=', 'Content-Length': '0'} 2022-09-03T06:47:36.426 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007f23ba0b8889b620-006312f888-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.428 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:36.429 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:36.430 DEBUG:boto:Method: GET 2022-09-03T06:47:36.431 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:47:36.432 DEBUG:boto:Data: 2022-09-03T06:47:36.432 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.433 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:36.434 DEBUG:boto:Port: 8000 2022-09-03T06:47:36.434 DEBUG:boto:Params: {} 2022-09-03T06:47:36.435 DEBUG:boto:Token: None 2022-09-03T06:47:36.436 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj38 2022-09-03T06:47:36.437 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:np1Pj2gXMVuqV+XKEEnW1ZnmKeQ= 2022-09-03T06:47:36.439 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:np1Pj2gXMVuqV+XKEEnW1ZnmKeQ=', 'Content-Length': '0'} 2022-09-03T06:47:36.442 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ebd1c2770128df6c-006312f888-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.486 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:36.487 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:36.488 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:36.489 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:36.489 DEBUG:boto:Method: GET 2022-09-03T06:47:36.490 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:47:36.491 DEBUG:boto:Data: 2022-09-03T06:47:36.491 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.493 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:36.493 DEBUG:boto:Port: 8001 2022-09-03T06:47:36.494 DEBUG:boto:Params: {} 2022-09-03T06:47:36.495 DEBUG:boto:Token: None 2022-09-03T06:47:36.496 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/ 2022-09-03T06:47:36.497 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg= 2022-09-03T06:47:36.498 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg=', 'Content-Length': '0'} 2022-09-03T06:47:36.501 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ca6fc99d387efe84-006312f888-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.502 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:36.503 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:36.504 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:36.504 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:36.505 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:36.506 DEBUG:boto:Method: GET 2022-09-03T06:47:36.506 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:47:36.507 DEBUG:boto:Data: 2022-09-03T06:47:36.508 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.508 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:36.509 DEBUG:boto:Port: 8000 2022-09-03T06:47:36.509 DEBUG:boto:Params: {} 2022-09-03T06:47:36.510 DEBUG:boto:Token: None 2022-09-03T06:47:36.511 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/ 2022-09-03T06:47:36.512 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg= 2022-09-03T06:47:36.514 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg=', 'Content-Length': '0'} 2022-09-03T06:47:36.519 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000006e21b43b14fb4b8-006312f888-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.521 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:36.524 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:47:36.527 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:36.527 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:36.528 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.528 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.529 DEBUG:boto:Method: GET 2022-09-03T06:47:36.530 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.531 DEBUG:boto:Data: 2022-09-03T06:47:36.531 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.532 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:36.533 DEBUG:boto:Port: 8001 2022-09-03T06:47:36.534 DEBUG:boto:Params: {} 2022-09-03T06:47:36.535 DEBUG:boto:Token: None 2022-09-03T06:47:36.536 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.536 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3DxeeohQAq0rUqVL/KoM6XRalms= 2022-09-03T06:47:36.537 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3DxeeohQAq0rUqVL/KoM6XRalms=', 'Content-Length': '0'} 2022-09-03T06:47:36.540 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004076b7dc6268024f-006312f888-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.541 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:36.542 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:36.543 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.544 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.545 DEBUG:boto:Method: GET 2022-09-03T06:47:36.546 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.547 DEBUG:boto:Data: 2022-09-03T06:47:36.548 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.549 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:36.550 DEBUG:boto:Port: 8000 2022-09-03T06:47:36.551 DEBUG:boto:Params: {} 2022-09-03T06:47:36.552 DEBUG:boto:Token: None 2022-09-03T06:47:36.553 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.555 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3DxeeohQAq0rUqVL/KoM6XRalms= 2022-09-03T06:47:36.556 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3DxeeohQAq0rUqVL/KoM6XRalms=', 'Content-Length': '0'} 2022-09-03T06:47:36.559 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ebd4cf6211a6dc5d-006312f888-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.561 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:36.562 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:36.563 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.563 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.564 DEBUG:boto:Method: HEAD 2022-09-03T06:47:36.564 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.565 DEBUG:boto:Data: 2022-09-03T06:47:36.565 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.566 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:36.566 DEBUG:boto:Port: 8001 2022-09-03T06:47:36.567 DEBUG:boto:Params: {} 2022-09-03T06:47:36.568 DEBUG:boto:Token: None 2022-09-03T06:47:36.568 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.569 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B0VnKD8MA5f0WI4AXuKvkOYjnOg= 2022-09-03T06:47:36.570 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B0VnKD8MA5f0WI4AXuKvkOYjnOg=', 'Content-Length': '0'} 2022-09-03T06:47:36.572 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000075bc997e0b11996e-006312f888-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.574 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:36.575 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:36.575 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.576 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.577 DEBUG:boto:Method: HEAD 2022-09-03T06:47:36.578 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.579 DEBUG:boto:Data: 2022-09-03T06:47:36.580 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.581 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:36.582 DEBUG:boto:Port: 8000 2022-09-03T06:47:36.583 DEBUG:boto:Params: {} 2022-09-03T06:47:36.584 DEBUG:boto:Token: None 2022-09-03T06:47:36.585 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.586 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B0VnKD8MA5f0WI4AXuKvkOYjnOg= 2022-09-03T06:47:36.588 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B0VnKD8MA5f0WI4AXuKvkOYjnOg=', 'Content-Length': '0'} 2022-09-03T06:47:36.595 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001283e44057ed50c9-006312f888-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.596 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:47:36.597 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:36.598 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:36.598 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.600 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.601 DEBUG:boto:Method: GET 2022-09-03T06:47:36.602 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.603 DEBUG:boto:Data: 2022-09-03T06:47:36.604 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.605 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:36.606 DEBUG:boto:Port: 8001 2022-09-03T06:47:36.607 DEBUG:boto:Params: {} 2022-09-03T06:47:36.608 DEBUG:boto:Token: None 2022-09-03T06:47:36.609 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.611 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3DxeeohQAq0rUqVL/KoM6XRalms= 2022-09-03T06:47:36.613 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3DxeeohQAq0rUqVL/KoM6XRalms=', 'Content-Length': '0'} 2022-09-03T06:47:36.615 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af850d2cd236973c-006312f888-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.616 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:36.617 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:36.619 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.620 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.620 DEBUG:boto:Method: GET 2022-09-03T06:47:36.621 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.621 DEBUG:boto:Data: 2022-09-03T06:47:36.622 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.623 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:36.623 DEBUG:boto:Port: 8000 2022-09-03T06:47:36.624 DEBUG:boto:Params: {} 2022-09-03T06:47:36.625 DEBUG:boto:Token: None 2022-09-03T06:47:36.625 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:36.626 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3DxeeohQAq0rUqVL/KoM6XRalms= 2022-09-03T06:47:36.627 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3DxeeohQAq0rUqVL/KoM6XRalms=', 'Content-Length': '0'} 2022-09-03T06:47:36.631 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e3987a5ad0d95d3-006312f888-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.678 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:36.679 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:36.680 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:36.680 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:36.681 DEBUG:boto:Method: GET 2022-09-03T06:47:36.681 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:47:36.682 DEBUG:boto:Data: 2022-09-03T06:47:36.683 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.683 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:36.684 DEBUG:boto:Port: 8001 2022-09-03T06:47:36.685 DEBUG:boto:Params: {} 2022-09-03T06:47:36.686 DEBUG:boto:Token: None 2022-09-03T06:47:36.687 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/ 2022-09-03T06:47:36.688 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg= 2022-09-03T06:47:36.689 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg=', 'Content-Length': '0'} 2022-09-03T06:47:36.693 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fd3f2b950793adf8-006312f888-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.694 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:36.696 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:36.697 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:36.697 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:36.698 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:36.699 DEBUG:boto:Method: GET 2022-09-03T06:47:36.699 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:47:36.700 DEBUG:boto:Data: 2022-09-03T06:47:36.700 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.701 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:36.701 DEBUG:boto:Port: 8000 2022-09-03T06:47:36.702 DEBUG:boto:Params: {} 2022-09-03T06:47:36.703 DEBUG:boto:Token: None 2022-09-03T06:47:36.704 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/ 2022-09-03T06:47:36.704 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg= 2022-09-03T06:47:36.705 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg=', 'Content-Length': '0'} 2022-09-03T06:47:36.709 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000be43ec63c19c514-006312f888-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.710 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:36.712 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:36.713 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:36.714 DEBUG:boto:Method: HEAD 2022-09-03T06:47:36.714 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:47:36.715 DEBUG:boto:Data: 2022-09-03T06:47:36.716 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.717 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:36.718 DEBUG:boto:Port: 8001 2022-09-03T06:47:36.719 DEBUG:boto:Params: {} 2022-09-03T06:47:36.719 DEBUG:boto:Token: None 2022-09-03T06:47:36.720 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj44 2022-09-03T06:47:36.721 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DUC74xUedp5RsPiskpxB9W0eWmY= 2022-09-03T06:47:36.721 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DUC74xUedp5RsPiskpxB9W0eWmY=', 'Content-Length': '0'} 2022-09-03T06:47:36.723 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad1e1bfcf3094ce5-006312f888-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.724 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:36.725 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:36.726 DEBUG:boto:Method: HEAD 2022-09-03T06:47:36.727 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:47:36.728 DEBUG:boto:Data: 2022-09-03T06:47:36.728 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.729 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:36.729 DEBUG:boto:Port: 8000 2022-09-03T06:47:36.730 DEBUG:boto:Params: {} 2022-09-03T06:47:36.730 DEBUG:boto:Token: None 2022-09-03T06:47:36.731 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj44 2022-09-03T06:47:36.731 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DUC74xUedp5RsPiskpxB9W0eWmY= 2022-09-03T06:47:36.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DUC74xUedp5RsPiskpxB9W0eWmY=', 'Content-Length': '0'} 2022-09-03T06:47:36.734 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009405aabdb004f84c-006312f888-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.735 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:47:36.736 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:36.736 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:36.737 DEBUG:boto:Method: GET 2022-09-03T06:47:36.737 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:47:36.738 DEBUG:boto:Data: 2022-09-03T06:47:36.738 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.739 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:36.740 DEBUG:boto:Port: 8001 2022-09-03T06:47:36.740 DEBUG:boto:Params: {} 2022-09-03T06:47:36.741 DEBUG:boto:Token: None 2022-09-03T06:47:36.741 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj44 2022-09-03T06:47:36.742 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qz0LyGwcylyi95S2twhMpfTVZDo= 2022-09-03T06:47:36.743 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qz0LyGwcylyi95S2twhMpfTVZDo=', 'Content-Length': '0'} 2022-09-03T06:47:36.745 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021f6eec0ad8e2b47-006312f888-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.746 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:36.746 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:36.747 DEBUG:boto:Method: GET 2022-09-03T06:47:36.748 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:47:36.748 DEBUG:boto:Data: 2022-09-03T06:47:36.749 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.749 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:36.750 DEBUG:boto:Port: 8000 2022-09-03T06:47:36.750 DEBUG:boto:Params: {} 2022-09-03T06:47:36.751 DEBUG:boto:Token: None 2022-09-03T06:47:36.752 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj44 2022-09-03T06:47:36.752 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qz0LyGwcylyi95S2twhMpfTVZDo= 2022-09-03T06:47:36.753 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qz0LyGwcylyi95S2twhMpfTVZDo=', 'Content-Length': '0'} 2022-09-03T06:47:36.755 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b6cf0e81cc498048-006312f888-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.756 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:36.756 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:36.757 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:36.758 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:36.758 DEBUG:boto:Method: GET 2022-09-03T06:47:36.759 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:47:36.759 DEBUG:boto:Data: 2022-09-03T06:47:36.760 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.761 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:36.761 DEBUG:boto:Port: 8001 2022-09-03T06:47:36.762 DEBUG:boto:Params: {} 2022-09-03T06:47:36.770 DEBUG:boto:Token: None 2022-09-03T06:47:36.771 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/ 2022-09-03T06:47:36.772 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg= 2022-09-03T06:47:36.773 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg=', 'Content-Length': '0'} 2022-09-03T06:47:36.776 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000001204a9dd9846cd4-006312f888-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.777 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:36.778 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:36.779 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:36.780 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:36.780 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:36.781 DEBUG:boto:Method: GET 2022-09-03T06:47:36.781 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:47:36.782 DEBUG:boto:Data: 2022-09-03T06:47:36.782 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.783 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:36.783 DEBUG:boto:Port: 8000 2022-09-03T06:47:36.784 DEBUG:boto:Params: {} 2022-09-03T06:47:36.785 DEBUG:boto:Token: None 2022-09-03T06:47:36.785 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/ 2022-09-03T06:47:36.786 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg= 2022-09-03T06:47:36.787 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg=', 'Content-Length': '0'} 2022-09-03T06:47:36.792 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c9a899be916c4be7-006312f888-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.834 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:36.835 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:47:36.836 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:36.837 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:36.838 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.839 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.840 DEBUG:boto:Method: GET 2022-09-03T06:47:36.841 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.841 DEBUG:boto:Data: 2022-09-03T06:47:36.842 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.843 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:36.843 DEBUG:boto:Port: 8001 2022-09-03T06:47:36.844 DEBUG:boto:Params: {} 2022-09-03T06:47:36.845 DEBUG:boto:Token: None 2022-09-03T06:47:36.846 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.847 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xxX9ZCdUxoNHmN/Rnj59bm1s1+Q= 2022-09-03T06:47:36.848 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xxX9ZCdUxoNHmN/Rnj59bm1s1+Q=', 'Content-Length': '0'} 2022-09-03T06:47:36.850 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7cf44b35214d22e-006312f888-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.852 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:36.853 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:36.854 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.855 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.856 DEBUG:boto:Method: GET 2022-09-03T06:47:36.857 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.858 DEBUG:boto:Data: 2022-09-03T06:47:36.860 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.861 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:36.862 DEBUG:boto:Port: 8000 2022-09-03T06:47:36.863 DEBUG:boto:Params: {} 2022-09-03T06:47:36.864 DEBUG:boto:Token: None 2022-09-03T06:47:36.865 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.866 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xxX9ZCdUxoNHmN/Rnj59bm1s1+Q= 2022-09-03T06:47:36.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xxX9ZCdUxoNHmN/Rnj59bm1s1+Q=', 'Content-Length': '0'} 2022-09-03T06:47:36.870 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009c23fa1f8b66d74e-006312f888-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.871 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:36.872 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:36.873 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.873 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.874 DEBUG:boto:Method: HEAD 2022-09-03T06:47:36.874 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.875 DEBUG:boto:Data: 2022-09-03T06:47:36.876 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.877 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:36.877 DEBUG:boto:Port: 8001 2022-09-03T06:47:36.878 DEBUG:boto:Params: {} 2022-09-03T06:47:36.879 DEBUG:boto:Token: None 2022-09-03T06:47:36.880 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.881 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D0+WjBjVKMZ5lEyKC3L1+/EyeUM= 2022-09-03T06:47:36.882 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D0+WjBjVKMZ5lEyKC3L1+/EyeUM=', 'Content-Length': '0'} 2022-09-03T06:47:36.884 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000028f63b84d7a2af1c-006312f888-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.885 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:36.885 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:36.886 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.886 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.887 DEBUG:boto:Method: HEAD 2022-09-03T06:47:36.888 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.888 DEBUG:boto:Data: 2022-09-03T06:47:36.889 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.889 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:36.890 DEBUG:boto:Port: 8000 2022-09-03T06:47:36.890 DEBUG:boto:Params: {} 2022-09-03T06:47:36.891 DEBUG:boto:Token: None 2022-09-03T06:47:36.891 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.892 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D0+WjBjVKMZ5lEyKC3L1+/EyeUM= 2022-09-03T06:47:36.893 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D0+WjBjVKMZ5lEyKC3L1+/EyeUM=', 'Content-Length': '0'} 2022-09-03T06:47:36.895 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b9ee5f74b9c7e06-006312f888-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.896 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:47:36.896 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:36.897 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:36.897 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.898 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.898 DEBUG:boto:Method: GET 2022-09-03T06:47:36.899 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.900 DEBUG:boto:Data: 2022-09-03T06:47:36.900 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.901 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:36.901 DEBUG:boto:Port: 8001 2022-09-03T06:47:36.902 DEBUG:boto:Params: {} 2022-09-03T06:47:36.903 DEBUG:boto:Token: None 2022-09-03T06:47:36.903 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.904 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xxX9ZCdUxoNHmN/Rnj59bm1s1+Q= 2022-09-03T06:47:36.905 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xxX9ZCdUxoNHmN/Rnj59bm1s1+Q=', 'Content-Length': '0'} 2022-09-03T06:47:36.907 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d36ae740d147b34-006312f888-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.951 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:36.952 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:36.953 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.955 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.956 DEBUG:boto:Method: GET 2022-09-03T06:47:36.957 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.958 DEBUG:boto:Data: 2022-09-03T06:47:36.959 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.960 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:36.961 DEBUG:boto:Port: 8000 2022-09-03T06:47:36.962 DEBUG:boto:Params: {} 2022-09-03T06:47:36.964 DEBUG:boto:Token: None 2022-09-03T06:47:36.965 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:36.966 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xxX9ZCdUxoNHmN/Rnj59bm1s1+Q= 2022-09-03T06:47:36.967 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xxX9ZCdUxoNHmN/Rnj59bm1s1+Q=', 'Content-Length': '0'} 2022-09-03T06:47:36.970 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021efa6c5719daa0c-006312f888-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.972 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:36.973 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:36.974 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:36.974 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:36.975 DEBUG:boto:Method: GET 2022-09-03T06:47:36.976 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:47:36.977 DEBUG:boto:Data: 2022-09-03T06:47:36.978 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.979 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:36.979 DEBUG:boto:Port: 8001 2022-09-03T06:47:36.980 DEBUG:boto:Params: {} 2022-09-03T06:47:36.981 DEBUG:boto:Token: None 2022-09-03T06:47:36.982 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/ 2022-09-03T06:47:36.983 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg= 2022-09-03T06:47:36.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg=', 'Content-Length': '0'} 2022-09-03T06:47:36.987 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000340ea7d88c2106c6-006312f888-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:36 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:36.987 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:36.988 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:36.989 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:36.990 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:36.990 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:36.991 DEBUG:boto:Method: GET 2022-09-03T06:47:36.992 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:47:36.993 DEBUG:boto:Data: 2022-09-03T06:47:36.994 DEBUG:boto:Headers: {} 2022-09-03T06:47:36.995 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:36.996 DEBUG:boto:Port: 8000 2022-09-03T06:47:36.997 DEBUG:boto:Params: {} 2022-09-03T06:47:36.998 DEBUG:boto:Token: None 2022-09-03T06:47:36.999 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:36 GMT /dsgvcy-75/ 2022-09-03T06:47:37.000 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg= 2022-09-03T06:47:37.001 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:36 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qzmybQx/KV2S6qf9DluITGTBdcg=', 'Content-Length': '0'} 2022-09-03T06:47:37.008 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000450c730d547a362d-006312f889-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.009 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:37.010 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:37.011 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:37.011 DEBUG:boto:Method: HEAD 2022-09-03T06:47:37.012 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:47:37.013 DEBUG:boto:Data: 2022-09-03T06:47:37.013 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.014 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.015 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.016 DEBUG:boto:Params: {} 2022-09-03T06:47:37.016 DEBUG:boto:Token: None 2022-09-03T06:47:37.017 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj46 2022-09-03T06:47:37.018 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SESxs6Vn6ZZJqfqmvG0tD/Irjyg= 2022-09-03T06:47:37.019 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SESxs6Vn6ZZJqfqmvG0tD/Irjyg=', 'Content-Length': '0'} 2022-09-03T06:47:37.021 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c1465aaadebecb85-006312f889-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.022 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:37.023 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:37.023 DEBUG:boto:Method: HEAD 2022-09-03T06:47:37.024 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:47:37.024 DEBUG:boto:Data: 2022-09-03T06:47:37.025 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.025 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.026 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.027 DEBUG:boto:Params: {} 2022-09-03T06:47:37.027 DEBUG:boto:Token: None 2022-09-03T06:47:37.028 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj46 2022-09-03T06:47:37.029 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SESxs6Vn6ZZJqfqmvG0tD/Irjyg= 2022-09-03T06:47:37.030 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SESxs6Vn6ZZJqfqmvG0tD/Irjyg=', 'Content-Length': '0'} 2022-09-03T06:47:37.033 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000015d5c93635c3654d-006312f889-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.034 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:47:37.035 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:37.037 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:37.037 DEBUG:boto:Method: GET 2022-09-03T06:47:37.038 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:47:37.039 DEBUG:boto:Data: 2022-09-03T06:47:37.040 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.041 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.042 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.043 DEBUG:boto:Params: {} 2022-09-03T06:47:37.043 DEBUG:boto:Token: None 2022-09-03T06:47:37.044 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj46 2022-09-03T06:47:37.045 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kiFDPVwMOuk/1MEY1mRoxH7genY= 2022-09-03T06:47:37.046 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kiFDPVwMOuk/1MEY1mRoxH7genY=', 'Content-Length': '0'} 2022-09-03T06:47:37.051 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001612ae43f09383f2-006312f889-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.053 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:37.053 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:37.054 DEBUG:boto:Method: GET 2022-09-03T06:47:37.055 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:47:37.055 DEBUG:boto:Data: 2022-09-03T06:47:37.056 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.056 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.057 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.058 DEBUG:boto:Params: {} 2022-09-03T06:47:37.058 DEBUG:boto:Token: None 2022-09-03T06:47:37.059 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj46 2022-09-03T06:47:37.060 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kiFDPVwMOuk/1MEY1mRoxH7genY= 2022-09-03T06:47:37.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kiFDPVwMOuk/1MEY1mRoxH7genY=', 'Content-Length': '0'} 2022-09-03T06:47:37.064 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005a25383f9a8c6590-006312f889-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.107 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:37.108 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:37.109 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:37.109 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:37.110 DEBUG:boto:Method: GET 2022-09-03T06:47:37.111 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:47:37.112 DEBUG:boto:Data: 2022-09-03T06:47:37.112 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.113 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.114 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.115 DEBUG:boto:Params: {} 2022-09-03T06:47:37.115 DEBUG:boto:Token: None 2022-09-03T06:47:37.116 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/ 2022-09-03T06:47:37.117 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI= 2022-09-03T06:47:37.118 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI=', 'Content-Length': '0'} 2022-09-03T06:47:37.130 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cdcb934ca93c1d31-006312f889-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.132 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:37.133 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:37.134 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:37.135 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:37.136 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:37.137 DEBUG:boto:Method: GET 2022-09-03T06:47:37.138 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:47:37.139 DEBUG:boto:Data: 2022-09-03T06:47:37.141 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.142 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.143 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.144 DEBUG:boto:Params: {} 2022-09-03T06:47:37.146 DEBUG:boto:Token: None 2022-09-03T06:47:37.147 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/ 2022-09-03T06:47:37.148 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI= 2022-09-03T06:47:37.149 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI=', 'Content-Length': '0'} 2022-09-03T06:47:37.154 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d0e0774d3df28ba5-006312f889-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.156 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:37.158 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:47:37.160 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:37.160 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:37.161 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.162 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.163 DEBUG:boto:Method: GET 2022-09-03T06:47:37.163 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.164 DEBUG:boto:Data: 2022-09-03T06:47:37.164 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.165 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.166 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.166 DEBUG:boto:Params: {} 2022-09-03T06:47:37.167 DEBUG:boto:Token: None 2022-09-03T06:47:37.168 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.168 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lDtoCa8KvzD6TwVm33+ppR7Ja0E= 2022-09-03T06:47:37.169 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lDtoCa8KvzD6TwVm33+ppR7Ja0E=', 'Content-Length': '0'} 2022-09-03T06:47:37.174 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee50f7e35244d8d9-006312f889-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.175 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:37.175 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:37.176 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.177 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.178 DEBUG:boto:Method: GET 2022-09-03T06:47:37.179 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.180 DEBUG:boto:Data: 2022-09-03T06:47:37.180 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.181 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.182 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.183 DEBUG:boto:Params: {} 2022-09-03T06:47:37.183 DEBUG:boto:Token: None 2022-09-03T06:47:37.184 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.185 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lDtoCa8KvzD6TwVm33+ppR7Ja0E= 2022-09-03T06:47:37.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lDtoCa8KvzD6TwVm33+ppR7Ja0E=', 'Content-Length': '0'} 2022-09-03T06:47:37.188 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c359beb0648887bd-006312f889-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.189 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:37.190 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:37.191 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.191 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.192 DEBUG:boto:Method: HEAD 2022-09-03T06:47:37.192 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.193 DEBUG:boto:Data: 2022-09-03T06:47:37.194 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.194 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.195 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.196 DEBUG:boto:Params: {} 2022-09-03T06:47:37.197 DEBUG:boto:Token: None 2022-09-03T06:47:37.198 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.199 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tk+lAe6piVX/PVztJNfdKEuj3LI= 2022-09-03T06:47:37.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tk+lAe6piVX/PVztJNfdKEuj3LI=', 'Content-Length': '0'} 2022-09-03T06:47:37.210 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e3c022f1276f2869-006312f889-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.212 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:37.212 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:37.213 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.214 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.215 DEBUG:boto:Method: HEAD 2022-09-03T06:47:37.216 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.217 DEBUG:boto:Data: 2022-09-03T06:47:37.218 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.219 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.220 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.220 DEBUG:boto:Params: {} 2022-09-03T06:47:37.221 DEBUG:boto:Token: None 2022-09-03T06:47:37.222 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.223 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tk+lAe6piVX/PVztJNfdKEuj3LI= 2022-09-03T06:47:37.223 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tk+lAe6piVX/PVztJNfdKEuj3LI=', 'Content-Length': '0'} 2022-09-03T06:47:37.226 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000058ea23a203eb94a0-006312f889-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.227 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:47:37.227 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:37.228 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:37.229 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.231 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.232 DEBUG:boto:Method: GET 2022-09-03T06:47:37.233 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.234 DEBUG:boto:Data: 2022-09-03T06:47:37.234 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.235 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.235 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.237 DEBUG:boto:Params: {} 2022-09-03T06:47:37.237 DEBUG:boto:Token: None 2022-09-03T06:47:37.238 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.239 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lDtoCa8KvzD6TwVm33+ppR7Ja0E= 2022-09-03T06:47:37.239 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lDtoCa8KvzD6TwVm33+ppR7Ja0E=', 'Content-Length': '0'} 2022-09-03T06:47:37.242 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bbe594cb4b83f33f-006312f889-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.244 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:37.244 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:37.245 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.246 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.247 DEBUG:boto:Method: GET 2022-09-03T06:47:37.248 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.248 DEBUG:boto:Data: 2022-09-03T06:47:37.249 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.250 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.251 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.251 DEBUG:boto:Params: {} 2022-09-03T06:47:37.252 DEBUG:boto:Token: None 2022-09-03T06:47:37.253 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:37.253 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lDtoCa8KvzD6TwVm33+ppR7Ja0E= 2022-09-03T06:47:37.254 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lDtoCa8KvzD6TwVm33+ppR7Ja0E=', 'Content-Length': '0'} 2022-09-03T06:47:37.257 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000925d956e3286dd78-006312f889-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.258 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:37.258 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:37.259 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:37.260 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:37.260 DEBUG:boto:Method: GET 2022-09-03T06:47:37.261 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:47:37.261 DEBUG:boto:Data: 2022-09-03T06:47:37.262 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.262 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.263 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.263 DEBUG:boto:Params: {} 2022-09-03T06:47:37.264 DEBUG:boto:Token: None 2022-09-03T06:47:37.265 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/ 2022-09-03T06:47:37.265 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI= 2022-09-03T06:47:37.266 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI=', 'Content-Length': '0'} 2022-09-03T06:47:37.278 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006d3ae8ebb8398edb-006312f889-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.280 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:37.282 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:37.283 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:37.284 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:37.285 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:37.286 DEBUG:boto:Method: GET 2022-09-03T06:47:37.288 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:47:37.289 DEBUG:boto:Data: 2022-09-03T06:47:37.290 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.291 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.292 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.295 DEBUG:boto:Params: {} 2022-09-03T06:47:37.296 DEBUG:boto:Token: None 2022-09-03T06:47:37.297 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/ 2022-09-03T06:47:37.298 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI= 2022-09-03T06:47:37.299 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI=', 'Content-Length': '0'} 2022-09-03T06:47:37.308 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006e823b4f16004b6e-006312f889-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.309 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:37.312 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:37.313 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:37.314 DEBUG:boto:Method: HEAD 2022-09-03T06:47:37.314 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:47:37.315 DEBUG:boto:Data: 2022-09-03T06:47:37.315 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.316 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.317 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.317 DEBUG:boto:Params: {} 2022-09-03T06:47:37.318 DEBUG:boto:Token: None 2022-09-03T06:47:37.318 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj51 2022-09-03T06:47:37.319 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/Oh9K6yxntdXQAB/0fq2iZQ5Le8= 2022-09-03T06:47:37.319 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/Oh9K6yxntdXQAB/0fq2iZQ5Le8=', 'Content-Length': '0'} 2022-09-03T06:47:37.322 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026d1d8a69a867ebe-006312f889-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.322 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:37.323 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:37.324 DEBUG:boto:Method: HEAD 2022-09-03T06:47:37.324 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:47:37.325 DEBUG:boto:Data: 2022-09-03T06:47:37.325 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.326 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.326 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.327 DEBUG:boto:Params: {} 2022-09-03T06:47:37.327 DEBUG:boto:Token: None 2022-09-03T06:47:37.328 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj51 2022-09-03T06:47:37.329 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/Oh9K6yxntdXQAB/0fq2iZQ5Le8= 2022-09-03T06:47:37.329 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/Oh9K6yxntdXQAB/0fq2iZQ5Le8=', 'Content-Length': '0'} 2022-09-03T06:47:37.331 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000052b7e0ed0141a6f3-006312f889-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.332 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:47:37.333 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:37.333 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:37.334 DEBUG:boto:Method: GET 2022-09-03T06:47:37.334 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:47:37.335 DEBUG:boto:Data: 2022-09-03T06:47:37.335 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.337 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.337 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.339 DEBUG:boto:Params: {} 2022-09-03T06:47:37.339 DEBUG:boto:Token: None 2022-09-03T06:47:37.340 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj51 2022-09-03T06:47:37.341 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VGW/+Woi0rwLsn70GO8iomOm9MQ= 2022-09-03T06:47:37.341 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VGW/+Woi0rwLsn70GO8iomOm9MQ=', 'Content-Length': '0'} 2022-09-03T06:47:37.344 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bfb2edc5f967df0e-006312f889-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.345 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:37.345 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:37.347 DEBUG:boto:Method: GET 2022-09-03T06:47:37.348 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:47:37.349 DEBUG:boto:Data: 2022-09-03T06:47:37.350 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.351 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.352 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.353 DEBUG:boto:Params: {} 2022-09-03T06:47:37.355 DEBUG:boto:Token: None 2022-09-03T06:47:37.356 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj51 2022-09-03T06:47:37.357 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VGW/+Woi0rwLsn70GO8iomOm9MQ= 2022-09-03T06:47:37.358 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VGW/+Woi0rwLsn70GO8iomOm9MQ=', 'Content-Length': '0'} 2022-09-03T06:47:37.362 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f808ca6f32256349-006312f889-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.363 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:37.364 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:37.366 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:37.366 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:37.367 DEBUG:boto:Method: GET 2022-09-03T06:47:37.368 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:47:37.368 DEBUG:boto:Data: 2022-09-03T06:47:37.369 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.369 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.370 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.370 DEBUG:boto:Params: {} 2022-09-03T06:47:37.371 DEBUG:boto:Token: None 2022-09-03T06:47:37.372 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/ 2022-09-03T06:47:37.372 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI= 2022-09-03T06:47:37.373 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI=', 'Content-Length': '0'} 2022-09-03T06:47:37.376 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000944f91ea96ad90f4-006312f889-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.377 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:37.378 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:37.378 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:37.379 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:37.380 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:37.380 DEBUG:boto:Method: GET 2022-09-03T06:47:37.381 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:47:37.382 DEBUG:boto:Data: 2022-09-03T06:47:37.384 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.384 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.385 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.386 DEBUG:boto:Params: {} 2022-09-03T06:47:37.387 DEBUG:boto:Token: None 2022-09-03T06:47:37.388 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/ 2022-09-03T06:47:37.389 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI= 2022-09-03T06:47:37.389 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI=', 'Content-Length': '0'} 2022-09-03T06:47:37.393 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b6c939c5cebdcdea-006312f889-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.394 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:37.395 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:47:37.397 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:37.397 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:37.398 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.398 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.399 DEBUG:boto:Method: GET 2022-09-03T06:47:37.399 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.400 DEBUG:boto:Data: 2022-09-03T06:47:37.400 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.401 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.401 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.402 DEBUG:boto:Params: {} 2022-09-03T06:47:37.404 DEBUG:boto:Token: None 2022-09-03T06:47:37.404 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.405 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NHiWW3nsC6EW0/iCCSVPi2SxssY= 2022-09-03T06:47:37.406 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NHiWW3nsC6EW0/iCCSVPi2SxssY=', 'Content-Length': '0'} 2022-09-03T06:47:37.408 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bedb3994d9d8b507-006312f889-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.409 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:37.409 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:37.410 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.410 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.411 DEBUG:boto:Method: GET 2022-09-03T06:47:37.412 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.412 DEBUG:boto:Data: 2022-09-03T06:47:37.413 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.413 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.414 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.414 DEBUG:boto:Params: {} 2022-09-03T06:47:37.415 DEBUG:boto:Token: None 2022-09-03T06:47:37.415 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.416 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NHiWW3nsC6EW0/iCCSVPi2SxssY= 2022-09-03T06:47:37.417 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NHiWW3nsC6EW0/iCCSVPi2SxssY=', 'Content-Length': '0'} 2022-09-03T06:47:37.419 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e8aa4fc468354a7-006312f889-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.420 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:37.421 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:37.421 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.422 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.422 DEBUG:boto:Method: HEAD 2022-09-03T06:47:37.423 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.423 DEBUG:boto:Data: 2022-09-03T06:47:37.424 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.424 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.425 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.425 DEBUG:boto:Params: {} 2022-09-03T06:47:37.426 DEBUG:boto:Token: None 2022-09-03T06:47:37.427 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.427 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dSwi37bHKTowVAOOLM/+sPx/yHU= 2022-09-03T06:47:37.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dSwi37bHKTowVAOOLM/+sPx/yHU=', 'Content-Length': '0'} 2022-09-03T06:47:37.430 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db5a13157fd0f633-006312f889-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.431 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:37.432 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:37.432 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.433 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.434 DEBUG:boto:Method: HEAD 2022-09-03T06:47:37.434 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.435 DEBUG:boto:Data: 2022-09-03T06:47:37.435 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.436 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.437 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.438 DEBUG:boto:Params: {} 2022-09-03T06:47:37.439 DEBUG:boto:Token: None 2022-09-03T06:47:37.440 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.441 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dSwi37bHKTowVAOOLM/+sPx/yHU= 2022-09-03T06:47:37.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dSwi37bHKTowVAOOLM/+sPx/yHU=', 'Content-Length': '0'} 2022-09-03T06:47:37.446 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000040ef846a55ffcdd7-006312f889-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.447 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:47:37.448 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:37.449 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:37.450 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.451 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.452 DEBUG:boto:Method: GET 2022-09-03T06:47:37.452 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.459 DEBUG:boto:Data: 2022-09-03T06:47:37.460 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.464 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.465 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.466 DEBUG:boto:Params: {} 2022-09-03T06:47:37.467 DEBUG:boto:Token: None 2022-09-03T06:47:37.469 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.470 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NHiWW3nsC6EW0/iCCSVPi2SxssY= 2022-09-03T06:47:37.471 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NHiWW3nsC6EW0/iCCSVPi2SxssY=', 'Content-Length': '0'} 2022-09-03T06:47:37.474 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2a35ca6bba6084b-006312f889-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.475 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:37.476 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:37.477 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.478 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.479 DEBUG:boto:Method: GET 2022-09-03T06:47:37.479 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.480 DEBUG:boto:Data: 2022-09-03T06:47:37.481 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.482 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.483 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.484 DEBUG:boto:Params: {} 2022-09-03T06:47:37.485 DEBUG:boto:Token: None 2022-09-03T06:47:37.486 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:37.487 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NHiWW3nsC6EW0/iCCSVPi2SxssY= 2022-09-03T06:47:37.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NHiWW3nsC6EW0/iCCSVPi2SxssY=', 'Content-Length': '0'} 2022-09-03T06:47:37.490 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004b5846e4f3cd0113-006312f889-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.491 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:37.491 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:37.492 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:37.493 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:37.493 DEBUG:boto:Method: GET 2022-09-03T06:47:37.494 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:47:37.495 DEBUG:boto:Data: 2022-09-03T06:47:37.496 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.497 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.498 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.498 DEBUG:boto:Params: {} 2022-09-03T06:47:37.499 DEBUG:boto:Token: None 2022-09-03T06:47:37.500 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/ 2022-09-03T06:47:37.500 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI= 2022-09-03T06:47:37.501 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI=', 'Content-Length': '0'} 2022-09-03T06:47:37.504 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000050578293811e9224-006312f889-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.505 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:37.505 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:37.506 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:37.507 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:37.507 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:37.508 DEBUG:boto:Method: GET 2022-09-03T06:47:37.508 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:47:37.509 DEBUG:boto:Data: 2022-09-03T06:47:37.509 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.510 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.510 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.511 DEBUG:boto:Params: {} 2022-09-03T06:47:37.511 DEBUG:boto:Token: None 2022-09-03T06:47:37.512 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/ 2022-09-03T06:47:37.513 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI= 2022-09-03T06:47:37.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI=', 'Content-Length': '0'} 2022-09-03T06:47:37.518 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fc1e2be1d1312a12-006312f889-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.519 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:37.519 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:37.520 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:37.520 DEBUG:boto:Method: HEAD 2022-09-03T06:47:37.521 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:47:37.522 DEBUG:boto:Data: 2022-09-03T06:47:37.522 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.523 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.523 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.524 DEBUG:boto:Params: {} 2022-09-03T06:47:37.524 DEBUG:boto:Token: None 2022-09-03T06:47:37.525 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj55 2022-09-03T06:47:37.526 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RaiH70+2qVOS4PgLJrMUfb31UvU= 2022-09-03T06:47:37.527 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RaiH70+2qVOS4PgLJrMUfb31UvU=', 'Content-Length': '0'} 2022-09-03T06:47:37.530 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000228e59ed552aec3f-006312f889-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.531 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:37.531 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:37.532 DEBUG:boto:Method: HEAD 2022-09-03T06:47:37.532 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:47:37.533 DEBUG:boto:Data: 2022-09-03T06:47:37.533 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.534 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.534 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.535 DEBUG:boto:Params: {} 2022-09-03T06:47:37.536 DEBUG:boto:Token: None 2022-09-03T06:47:37.536 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj55 2022-09-03T06:47:37.537 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RaiH70+2qVOS4PgLJrMUfb31UvU= 2022-09-03T06:47:37.537 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RaiH70+2qVOS4PgLJrMUfb31UvU=', 'Content-Length': '0'} 2022-09-03T06:47:37.540 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000057255ea3d03ca764-006312f889-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.541 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:47:37.541 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:37.542 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:37.542 DEBUG:boto:Method: GET 2022-09-03T06:47:37.543 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:47:37.543 DEBUG:boto:Data: 2022-09-03T06:47:37.544 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.545 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.545 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.546 DEBUG:boto:Params: {} 2022-09-03T06:47:37.546 DEBUG:boto:Token: None 2022-09-03T06:47:37.547 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj55 2022-09-03T06:47:37.548 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fLmq7kVTJ21qc5/TanWNxQV0ijI= 2022-09-03T06:47:37.548 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fLmq7kVTJ21qc5/TanWNxQV0ijI=', 'Content-Length': '0'} 2022-09-03T06:47:37.550 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a5cdd7af42e19cd8-006312f889-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.594 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:37.595 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:37.596 DEBUG:boto:Method: GET 2022-09-03T06:47:37.597 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:47:37.598 DEBUG:boto:Data: 2022-09-03T06:47:37.599 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.602 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.605 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.607 DEBUG:boto:Params: {} 2022-09-03T06:47:37.608 DEBUG:boto:Token: None 2022-09-03T06:47:37.610 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj55 2022-09-03T06:47:37.611 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fLmq7kVTJ21qc5/TanWNxQV0ijI= 2022-09-03T06:47:37.613 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fLmq7kVTJ21qc5/TanWNxQV0ijI=', 'Content-Length': '0'} 2022-09-03T06:47:37.616 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001f7f35f9b364ea39-006312f889-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.617 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:37.618 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:37.619 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:37.620 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:37.621 DEBUG:boto:Method: GET 2022-09-03T06:47:37.621 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:47:37.622 DEBUG:boto:Data: 2022-09-03T06:47:37.622 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.623 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.624 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.625 DEBUG:boto:Params: {} 2022-09-03T06:47:37.626 DEBUG:boto:Token: None 2022-09-03T06:47:37.627 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/ 2022-09-03T06:47:37.629 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI= 2022-09-03T06:47:37.630 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI=', 'Content-Length': '0'} 2022-09-03T06:47:37.634 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000178ec0365b9fcd11-006312f889-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.635 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:37.636 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:37.637 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:37.638 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:37.639 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:37.639 DEBUG:boto:Method: GET 2022-09-03T06:47:37.640 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:47:37.640 DEBUG:boto:Data: 2022-09-03T06:47:37.641 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.642 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.642 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.643 DEBUG:boto:Params: {} 2022-09-03T06:47:37.644 DEBUG:boto:Token: None 2022-09-03T06:47:37.645 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/ 2022-09-03T06:47:37.645 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI= 2022-09-03T06:47:37.646 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI=', 'Content-Length': '0'} 2022-09-03T06:47:37.651 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cb96549e46c4b0fc-006312f889-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.652 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:37.654 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:47:37.655 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:37.656 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:37.657 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.658 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.659 DEBUG:boto:Method: GET 2022-09-03T06:47:37.660 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.661 DEBUG:boto:Data: 2022-09-03T06:47:37.662 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.663 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.664 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.665 DEBUG:boto:Params: {} 2022-09-03T06:47:37.666 DEBUG:boto:Token: None 2022-09-03T06:47:37.667 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.668 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L4IR7hw6CFr0P/MTfj6UWvPKA3c= 2022-09-03T06:47:37.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L4IR7hw6CFr0P/MTfj6UWvPKA3c=', 'Content-Length': '0'} 2022-09-03T06:47:37.672 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001e3101fd4e2fac65-006312f889-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.674 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:37.675 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:37.676 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.677 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.678 DEBUG:boto:Method: GET 2022-09-03T06:47:37.679 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.680 DEBUG:boto:Data: 2022-09-03T06:47:37.681 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.682 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.683 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.684 DEBUG:boto:Params: {} 2022-09-03T06:47:37.686 DEBUG:boto:Token: None 2022-09-03T06:47:37.687 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.688 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L4IR7hw6CFr0P/MTfj6UWvPKA3c= 2022-09-03T06:47:37.690 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L4IR7hw6CFr0P/MTfj6UWvPKA3c=', 'Content-Length': '0'} 2022-09-03T06:47:37.693 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000023c0348e30748303-006312f889-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.740 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:37.742 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:37.743 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.744 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.745 DEBUG:boto:Method: HEAD 2022-09-03T06:47:37.745 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.746 DEBUG:boto:Data: 2022-09-03T06:47:37.747 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.748 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.748 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.749 DEBUG:boto:Params: {} 2022-09-03T06:47:37.750 DEBUG:boto:Token: None 2022-09-03T06:47:37.751 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.752 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bkcr/cJiJ+OP+lOCZYAfPiS2skc= 2022-09-03T06:47:37.753 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bkcr/cJiJ+OP+lOCZYAfPiS2skc=', 'Content-Length': '0'} 2022-09-03T06:47:37.755 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f5f6fa2e7b6deb5f-006312f889-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.757 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:37.758 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:37.759 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.759 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.760 DEBUG:boto:Method: HEAD 2022-09-03T06:47:37.762 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.763 DEBUG:boto:Data: 2022-09-03T06:47:37.764 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.765 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.766 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.767 DEBUG:boto:Params: {} 2022-09-03T06:47:37.768 DEBUG:boto:Token: None 2022-09-03T06:47:37.769 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.771 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bkcr/cJiJ+OP+lOCZYAfPiS2skc= 2022-09-03T06:47:37.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bkcr/cJiJ+OP+lOCZYAfPiS2skc=', 'Content-Length': '0'} 2022-09-03T06:47:37.775 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000093fd7133783276ba-006312f889-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.776 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:47:37.777 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:37.777 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:37.787 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.789 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.790 DEBUG:boto:Method: GET 2022-09-03T06:47:37.791 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.792 DEBUG:boto:Data: 2022-09-03T06:47:37.793 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.794 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.795 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.796 DEBUG:boto:Params: {} 2022-09-03T06:47:37.797 DEBUG:boto:Token: None 2022-09-03T06:47:37.798 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.800 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L4IR7hw6CFr0P/MTfj6UWvPKA3c= 2022-09-03T06:47:37.801 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L4IR7hw6CFr0P/MTfj6UWvPKA3c=', 'Content-Length': '0'} 2022-09-03T06:47:37.804 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000098dd4ded6edaf43-006312f889-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.805 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:37.806 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:37.807 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.807 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.808 DEBUG:boto:Method: GET 2022-09-03T06:47:37.809 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.809 DEBUG:boto:Data: 2022-09-03T06:47:37.810 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.811 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.811 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.812 DEBUG:boto:Params: {} 2022-09-03T06:47:37.813 DEBUG:boto:Token: None 2022-09-03T06:47:37.814 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:37.815 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:L4IR7hw6CFr0P/MTfj6UWvPKA3c= 2022-09-03T06:47:37.815 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:L4IR7hw6CFr0P/MTfj6UWvPKA3c=', 'Content-Length': '0'} 2022-09-03T06:47:37.818 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d29fb75a1d01768c-006312f889-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.819 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:37.820 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:37.820 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:37.821 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:37.822 DEBUG:boto:Method: GET 2022-09-03T06:47:37.822 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:47:37.823 DEBUG:boto:Data: 2022-09-03T06:47:37.824 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.824 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.825 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.825 DEBUG:boto:Params: {} 2022-09-03T06:47:37.826 DEBUG:boto:Token: None 2022-09-03T06:47:37.827 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/ 2022-09-03T06:47:37.828 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI= 2022-09-03T06:47:37.828 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI=', 'Content-Length': '0'} 2022-09-03T06:47:37.831 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000062d7da0986f990e0-006312f889-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.832 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:37.833 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:37.834 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:37.835 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:37.835 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:37.836 DEBUG:boto:Method: GET 2022-09-03T06:47:37.836 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:47:37.837 DEBUG:boto:Data: 2022-09-03T06:47:37.838 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.838 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.839 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.840 DEBUG:boto:Params: {} 2022-09-03T06:47:37.841 DEBUG:boto:Token: None 2022-09-03T06:47:37.842 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/ 2022-09-03T06:47:37.842 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI= 2022-09-03T06:47:37.843 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI=', 'Content-Length': '0'} 2022-09-03T06:47:37.848 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009750bf148f21202d-006312f889-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.894 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:37.897 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:37.899 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:37.900 DEBUG:boto:Method: HEAD 2022-09-03T06:47:37.901 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:47:37.902 DEBUG:boto:Data: 2022-09-03T06:47:37.903 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.903 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.904 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.905 DEBUG:boto:Params: {} 2022-09-03T06:47:37.907 DEBUG:boto:Token: None 2022-09-03T06:47:37.908 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj57 2022-09-03T06:47:37.909 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bewy26OMXj1WKQ7pvB5sA85MYbA= 2022-09-03T06:47:37.910 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bewy26OMXj1WKQ7pvB5sA85MYbA=', 'Content-Length': '0'} 2022-09-03T06:47:37.913 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002898846f47dbc205-006312f889-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.914 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:37.915 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:37.915 DEBUG:boto:Method: HEAD 2022-09-03T06:47:37.916 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:47:37.916 DEBUG:boto:Data: 2022-09-03T06:47:37.917 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.917 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.918 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.919 DEBUG:boto:Params: {} 2022-09-03T06:47:37.920 DEBUG:boto:Token: None 2022-09-03T06:47:37.921 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj57 2022-09-03T06:47:37.921 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bewy26OMXj1WKQ7pvB5sA85MYbA= 2022-09-03T06:47:37.922 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bewy26OMXj1WKQ7pvB5sA85MYbA=', 'Content-Length': '0'} 2022-09-03T06:47:37.924 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f47bd6018a526dd8-006312f889-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.925 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:47:37.926 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:37.927 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:37.927 DEBUG:boto:Method: GET 2022-09-03T06:47:37.928 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:47:37.928 DEBUG:boto:Data: 2022-09-03T06:47:37.929 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.930 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.930 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.931 DEBUG:boto:Params: {} 2022-09-03T06:47:37.931 DEBUG:boto:Token: None 2022-09-03T06:47:37.932 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj57 2022-09-03T06:47:37.933 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jDuxYIk9kmIQew+oCyb+RQbKOfw= 2022-09-03T06:47:37.933 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jDuxYIk9kmIQew+oCyb+RQbKOfw=', 'Content-Length': '0'} 2022-09-03T06:47:37.936 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af41efe366d38338-006312f889-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.936 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:37.937 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:37.938 DEBUG:boto:Method: GET 2022-09-03T06:47:37.939 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:47:37.939 DEBUG:boto:Data: 2022-09-03T06:47:37.940 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.940 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.941 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.941 DEBUG:boto:Params: {} 2022-09-03T06:47:37.942 DEBUG:boto:Token: None 2022-09-03T06:47:37.943 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/obj57 2022-09-03T06:47:37.943 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jDuxYIk9kmIQew+oCyb+RQbKOfw= 2022-09-03T06:47:37.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jDuxYIk9kmIQew+oCyb+RQbKOfw=', 'Content-Length': '0'} 2022-09-03T06:47:37.946 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c13e10481e48e384-006312f889-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.947 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:37.948 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:37.949 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:37.950 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:37.951 DEBUG:boto:Method: GET 2022-09-03T06:47:37.953 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:47:37.955 DEBUG:boto:Data: 2022-09-03T06:47:37.955 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.956 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.957 DEBUG:boto:Port: 8001 2022-09-03T06:47:37.958 DEBUG:boto:Params: {} 2022-09-03T06:47:37.959 DEBUG:boto:Token: None 2022-09-03T06:47:37.961 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/ 2022-09-03T06:47:37.963 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI= 2022-09-03T06:47:37.965 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI=', 'Content-Length': '0'} 2022-09-03T06:47:37.969 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f40725d7b27244e2-006312f889-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.970 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:37.971 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:37.972 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:37.973 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:37.974 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:37.975 DEBUG:boto:Method: GET 2022-09-03T06:47:37.975 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:47:37.976 DEBUG:boto:Data: 2022-09-03T06:47:37.978 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.979 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:37.980 DEBUG:boto:Port: 8000 2022-09-03T06:47:37.981 DEBUG:boto:Params: {} 2022-09-03T06:47:37.982 DEBUG:boto:Token: None 2022-09-03T06:47:37.983 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:37 GMT /dsgvcy-75/ 2022-09-03T06:47:37.984 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI= 2022-09-03T06:47:37.986 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FhsJOaKU5R6uMILukRJlsAnHIgI=', 'Content-Length': '0'} 2022-09-03T06:47:37.991 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001a59a152fb7677ab-006312f889-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:37.992 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:37.993 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:47:37.994 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:37.995 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:37.995 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:37.996 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:37.996 DEBUG:boto:Method: GET 2022-09-03T06:47:37.997 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:37.997 DEBUG:boto:Data: 2022-09-03T06:47:37.998 DEBUG:boto:Headers: {} 2022-09-03T06:47:37.998 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:37.999 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.000 DEBUG:boto:Params: {} 2022-09-03T06:47:38.000 DEBUG:boto:Token: None 2022-09-03T06:47:38.001 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:38.001 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DCG7dd+tCGNUX6xeDtUItF+oyJo= 2022-09-03T06:47:38.002 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DCG7dd+tCGNUX6xeDtUItF+oyJo=', 'Content-Length': '0'} 2022-09-03T06:47:38.004 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005ae4cca75ade42bf-006312f88a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.005 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:38.006 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:38.006 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:38.007 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:38.007 DEBUG:boto:Method: GET 2022-09-03T06:47:38.008 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:38.009 DEBUG:boto:Data: 2022-09-03T06:47:38.009 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.010 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.010 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.011 DEBUG:boto:Params: {} 2022-09-03T06:47:38.012 DEBUG:boto:Token: None 2022-09-03T06:47:38.012 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:38.013 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DCG7dd+tCGNUX6xeDtUItF+oyJo= 2022-09-03T06:47:38.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DCG7dd+tCGNUX6xeDtUItF+oyJo=', 'Content-Length': '0'} 2022-09-03T06:47:38.016 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000043674e2dd8df2d62-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.062 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:38.063 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:38.064 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:38.065 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:38.067 DEBUG:boto:Method: HEAD 2022-09-03T06:47:38.068 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:38.069 DEBUG:boto:Data: 2022-09-03T06:47:38.070 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.071 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:38.071 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.072 DEBUG:boto:Params: {} 2022-09-03T06:47:38.073 DEBUG:boto:Token: None 2022-09-03T06:47:38.074 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:38.075 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CI9qGm8zkTfCN3cYs90MJIWgAeA= 2022-09-03T06:47:38.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CI9qGm8zkTfCN3cYs90MJIWgAeA=', 'Content-Length': '0'} 2022-09-03T06:47:38.079 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000939c14dc89cd4030-006312f88a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.080 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:38.081 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:38.082 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:38.083 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:38.084 DEBUG:boto:Method: HEAD 2022-09-03T06:47:38.085 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:38.086 DEBUG:boto:Data: 2022-09-03T06:47:38.090 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.092 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.093 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.094 DEBUG:boto:Params: {} 2022-09-03T06:47:38.095 DEBUG:boto:Token: None 2022-09-03T06:47:38.096 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:38.097 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CI9qGm8zkTfCN3cYs90MJIWgAeA= 2022-09-03T06:47:38.098 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CI9qGm8zkTfCN3cYs90MJIWgAeA=', 'Content-Length': '0'} 2022-09-03T06:47:38.102 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000069bf3fcb6ef04ea-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.103 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:47:38.104 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:38.105 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:38.106 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:38.107 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:38.108 DEBUG:boto:Method: GET 2022-09-03T06:47:38.109 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:38.110 DEBUG:boto:Data: 2022-09-03T06:47:38.111 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.112 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:38.114 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.115 DEBUG:boto:Params: {} 2022-09-03T06:47:38.116 DEBUG:boto:Token: None 2022-09-03T06:47:38.117 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:38.118 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DCG7dd+tCGNUX6xeDtUItF+oyJo= 2022-09-03T06:47:38.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DCG7dd+tCGNUX6xeDtUItF+oyJo=', 'Content-Length': '0'} 2022-09-03T06:47:38.121 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ab4a7cd9216dde9d-006312f88a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.122 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:38.123 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:38.123 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:38.124 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:38.125 DEBUG:boto:Method: GET 2022-09-03T06:47:38.125 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:38.126 DEBUG:boto:Data: 2022-09-03T06:47:38.126 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.127 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.128 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.129 DEBUG:boto:Params: {} 2022-09-03T06:47:38.129 DEBUG:boto:Token: None 2022-09-03T06:47:38.130 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:38.131 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DCG7dd+tCGNUX6xeDtUItF+oyJo= 2022-09-03T06:47:38.132 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DCG7dd+tCGNUX6xeDtUItF+oyJo=', 'Content-Length': '0'} 2022-09-03T06:47:38.137 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000baf515c174fe63a-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.142 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:38.143 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:38.143 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:38.144 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:38.145 DEBUG:boto:Method: GET 2022-09-03T06:47:38.145 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:47:38.146 DEBUG:boto:Data: 2022-09-03T06:47:38.147 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.147 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:38.148 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.148 DEBUG:boto:Params: {} 2022-09-03T06:47:38.149 DEBUG:boto:Token: None 2022-09-03T06:47:38.150 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/ 2022-09-03T06:47:38.150 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo= 2022-09-03T06:47:38.151 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo=', 'Content-Length': '0'} 2022-09-03T06:47:38.154 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cc27e4096d3c2925-006312f88a-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.155 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:38.156 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:38.157 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:38.157 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:38.158 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:38.159 DEBUG:boto:Method: GET 2022-09-03T06:47:38.159 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:47:38.160 DEBUG:boto:Data: 2022-09-03T06:47:38.160 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.161 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.162 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.162 DEBUG:boto:Params: {} 2022-09-03T06:47:38.163 DEBUG:boto:Token: None 2022-09-03T06:47:38.164 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/ 2022-09-03T06:47:38.164 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo= 2022-09-03T06:47:38.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo=', 'Content-Length': '0'} 2022-09-03T06:47:38.170 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000905be5b1411c996d-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.171 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:38.172 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:38.173 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:38.175 DEBUG:boto:Method: HEAD 2022-09-03T06:47:38.175 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:47:38.176 DEBUG:boto:Data: 2022-09-03T06:47:38.176 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.177 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:38.178 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.179 DEBUG:boto:Params: {} 2022-09-03T06:47:38.180 DEBUG:boto:Token: None 2022-09-03T06:47:38.181 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj58 2022-09-03T06:47:38.182 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i4eqAWO84JDrEs8KKY/CIvqvo0U= 2022-09-03T06:47:38.183 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i4eqAWO84JDrEs8KKY/CIvqvo0U=', 'Content-Length': '0'} 2022-09-03T06:47:38.186 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b22655671ccb28b2-006312f88a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.188 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:38.189 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:38.190 DEBUG:boto:Method: HEAD 2022-09-03T06:47:38.191 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:47:38.192 DEBUG:boto:Data: 2022-09-03T06:47:38.193 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.194 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.195 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.196 DEBUG:boto:Params: {} 2022-09-03T06:47:38.197 DEBUG:boto:Token: None 2022-09-03T06:47:38.198 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj58 2022-09-03T06:47:38.199 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i4eqAWO84JDrEs8KKY/CIvqvo0U= 2022-09-03T06:47:38.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i4eqAWO84JDrEs8KKY/CIvqvo0U=', 'Content-Length': '0'} 2022-09-03T06:47:38.203 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b3dcaae76940a319-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.204 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:47:38.206 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:38.207 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:38.208 DEBUG:boto:Method: GET 2022-09-03T06:47:38.209 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:47:38.210 DEBUG:boto:Data: 2022-09-03T06:47:38.211 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.212 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:38.213 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.214 DEBUG:boto:Params: {} 2022-09-03T06:47:38.215 DEBUG:boto:Token: None 2022-09-03T06:47:38.216 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj58 2022-09-03T06:47:38.217 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r+eo8RWC0xsYadcYnCgpSvcxuHE= 2022-09-03T06:47:38.218 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r+eo8RWC0xsYadcYnCgpSvcxuHE=', 'Content-Length': '0'} 2022-09-03T06:47:38.221 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000086e19506ec79feeb-006312f88a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.262 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:38.263 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:38.264 DEBUG:boto:Method: GET 2022-09-03T06:47:38.264 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:47:38.265 DEBUG:boto:Data: 2022-09-03T06:47:38.266 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.266 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.267 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.267 DEBUG:boto:Params: {} 2022-09-03T06:47:38.268 DEBUG:boto:Token: None 2022-09-03T06:47:38.269 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj58 2022-09-03T06:47:38.269 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r+eo8RWC0xsYadcYnCgpSvcxuHE= 2022-09-03T06:47:38.270 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r+eo8RWC0xsYadcYnCgpSvcxuHE=', 'Content-Length': '0'} 2022-09-03T06:47:38.272 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003bdf7ffead1b3d6c-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.273 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:38.274 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:38.274 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:38.275 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:38.276 DEBUG:boto:Method: GET 2022-09-03T06:47:38.276 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:47:38.277 DEBUG:boto:Data: 2022-09-03T06:47:38.277 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.278 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:38.278 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.279 DEBUG:boto:Params: {} 2022-09-03T06:47:38.279 DEBUG:boto:Token: None 2022-09-03T06:47:38.280 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/ 2022-09-03T06:47:38.281 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo= 2022-09-03T06:47:38.281 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo=', 'Content-Length': '0'} 2022-09-03T06:47:38.284 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d1ccac036132eb33-006312f88a-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.285 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:38.286 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:38.287 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:38.287 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:38.288 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:38.288 DEBUG:boto:Method: GET 2022-09-03T06:47:38.289 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:47:38.290 DEBUG:boto:Data: 2022-09-03T06:47:38.290 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.291 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.291 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.292 DEBUG:boto:Params: {} 2022-09-03T06:47:38.292 DEBUG:boto:Token: None 2022-09-03T06:47:38.293 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/ 2022-09-03T06:47:38.294 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo= 2022-09-03T06:47:38.295 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo=', 'Content-Length': '0'} 2022-09-03T06:47:38.299 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bfe3e9f7189e1a10-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.300 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:38.301 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2022-09-03T06:47:38.302 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:38.303 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:38.303 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.304 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.305 DEBUG:boto:Method: GET 2022-09-03T06:47:38.305 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.306 DEBUG:boto:Data: 2022-09-03T06:47:38.307 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.308 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:38.308 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.309 DEBUG:boto:Params: {} 2022-09-03T06:47:38.310 DEBUG:boto:Token: None 2022-09-03T06:47:38.310 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.311 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MieydkLAwqQ2ovjjRjrVJvT2HeQ= 2022-09-03T06:47:38.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MieydkLAwqQ2ovjjRjrVJvT2HeQ=', 'Content-Length': '0'} 2022-09-03T06:47:38.314 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000002e6ccebf1940332-006312f88a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.315 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:38.315 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:38.316 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.316 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.317 DEBUG:boto:Method: GET 2022-09-03T06:47:38.318 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.319 DEBUG:boto:Data: 2022-09-03T06:47:38.319 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.320 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.320 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.321 DEBUG:boto:Params: {} 2022-09-03T06:47:38.322 DEBUG:boto:Token: None 2022-09-03T06:47:38.323 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.324 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MieydkLAwqQ2ovjjRjrVJvT2HeQ= 2022-09-03T06:47:38.325 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MieydkLAwqQ2ovjjRjrVJvT2HeQ=', 'Content-Length': '0'} 2022-09-03T06:47:38.329 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000034c2781ac3ad4238-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.330 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:38.331 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:38.332 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.333 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.333 DEBUG:boto:Method: HEAD 2022-09-03T06:47:38.334 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.334 DEBUG:boto:Data: 2022-09-03T06:47:38.335 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.335 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:38.336 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.336 DEBUG:boto:Params: {} 2022-09-03T06:47:38.337 DEBUG:boto:Token: None 2022-09-03T06:47:38.338 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.338 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q/KKZ7GpSKYBFCnNju/DGA9+C/c= 2022-09-03T06:47:38.339 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q/KKZ7GpSKYBFCnNju/DGA9+C/c=', 'Content-Length': '0'} 2022-09-03T06:47:38.341 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000788b3c87983a998e-006312f88a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.342 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:38.343 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:38.344 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.345 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.346 DEBUG:boto:Method: HEAD 2022-09-03T06:47:38.348 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.349 DEBUG:boto:Data: 2022-09-03T06:47:38.349 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.350 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.351 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.352 DEBUG:boto:Params: {} 2022-09-03T06:47:38.353 DEBUG:boto:Token: None 2022-09-03T06:47:38.354 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.355 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q/KKZ7GpSKYBFCnNju/DGA9+C/c= 2022-09-03T06:47:38.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q/KKZ7GpSKYBFCnNju/DGA9+C/c=', 'Content-Length': '0'} 2022-09-03T06:47:38.359 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c573274d006d38a-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.360 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2022-09-03T06:47:38.361 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:38.362 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:38.363 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.363 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.364 DEBUG:boto:Method: GET 2022-09-03T06:47:38.365 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.366 DEBUG:boto:Data: 2022-09-03T06:47:38.366 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.367 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:38.368 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.368 DEBUG:boto:Params: {} 2022-09-03T06:47:38.369 DEBUG:boto:Token: None 2022-09-03T06:47:38.369 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.370 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MieydkLAwqQ2ovjjRjrVJvT2HeQ= 2022-09-03T06:47:38.371 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MieydkLAwqQ2ovjjRjrVJvT2HeQ=', 'Content-Length': '0'} 2022-09-03T06:47:38.374 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002ba7a51e67f50664-006312f88a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.375 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:38.376 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:38.376 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.377 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.378 DEBUG:boto:Method: GET 2022-09-03T06:47:38.378 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.379 DEBUG:boto:Data: 2022-09-03T06:47:38.379 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.380 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.380 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.381 DEBUG:boto:Params: {} 2022-09-03T06:47:38.381 DEBUG:boto:Token: None 2022-09-03T06:47:38.382 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:38.383 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MieydkLAwqQ2ovjjRjrVJvT2HeQ= 2022-09-03T06:47:38.383 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MieydkLAwqQ2ovjjRjrVJvT2HeQ=', 'Content-Length': '0'} 2022-09-03T06:47:38.386 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000084f76c838c0efc71-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.430 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:38.431 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:38.432 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:38.432 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:38.433 DEBUG:boto:Method: GET 2022-09-03T06:47:38.433 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:47:38.434 DEBUG:boto:Data: 2022-09-03T06:47:38.435 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.435 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:38.436 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.436 DEBUG:boto:Params: {} 2022-09-03T06:47:38.437 DEBUG:boto:Token: None 2022-09-03T06:47:38.438 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/ 2022-09-03T06:47:38.438 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo= 2022-09-03T06:47:38.439 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo=', 'Content-Length': '0'} 2022-09-03T06:47:38.442 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f19f0848b939486e-006312f88a-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.443 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:38.444 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:38.445 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:38.446 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:38.446 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:38.447 DEBUG:boto:Method: GET 2022-09-03T06:47:38.448 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:47:38.448 DEBUG:boto:Data: 2022-09-03T06:47:38.449 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.449 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.450 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.451 DEBUG:boto:Params: {} 2022-09-03T06:47:38.451 DEBUG:boto:Token: None 2022-09-03T06:47:38.452 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/ 2022-09-03T06:47:38.453 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo= 2022-09-03T06:47:38.453 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo=', 'Content-Length': '0'} 2022-09-03T06:47:38.457 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000070257fa61b2a9342-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.458 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:38.459 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:38.460 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:38.460 DEBUG:boto:Method: HEAD 2022-09-03T06:47:38.461 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:47:38.461 DEBUG:boto:Data: 2022-09-03T06:47:38.462 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.462 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:38.463 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.463 DEBUG:boto:Params: {} 2022-09-03T06:47:38.464 DEBUG:boto:Token: None 2022-09-03T06:47:38.465 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj62 2022-09-03T06:47:38.465 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a1dNX+5b/jPrIsErXr45cEfdsLU= 2022-09-03T06:47:38.466 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a1dNX+5b/jPrIsErXr45cEfdsLU=', 'Content-Length': '0'} 2022-09-03T06:47:38.468 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000028b5092a680612d6-006312f88a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.469 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:38.469 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:38.470 DEBUG:boto:Method: HEAD 2022-09-03T06:47:38.470 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:47:38.471 DEBUG:boto:Data: 2022-09-03T06:47:38.471 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.472 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.473 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.473 DEBUG:boto:Params: {} 2022-09-03T06:47:38.474 DEBUG:boto:Token: None 2022-09-03T06:47:38.474 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj62 2022-09-03T06:47:38.475 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a1dNX+5b/jPrIsErXr45cEfdsLU= 2022-09-03T06:47:38.475 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a1dNX+5b/jPrIsErXr45cEfdsLU=', 'Content-Length': '0'} 2022-09-03T06:47:38.477 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004d5f5c9d29f67257-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.478 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2022-09-03T06:47:38.479 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:38.480 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:38.480 DEBUG:boto:Method: GET 2022-09-03T06:47:38.481 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:47:38.482 DEBUG:boto:Data: 2022-09-03T06:47:38.482 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.483 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:38.483 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.484 DEBUG:boto:Params: {} 2022-09-03T06:47:38.484 DEBUG:boto:Token: None 2022-09-03T06:47:38.485 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj62 2022-09-03T06:47:38.486 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DpjOwhGed7mIefpYt4ZTuYo+ruQ= 2022-09-03T06:47:38.486 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DpjOwhGed7mIefpYt4ZTuYo+ruQ=', 'Content-Length': '0'} 2022-09-03T06:47:38.489 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f54c25f3df72a247-006312f88a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.531 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:38.533 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:38.534 DEBUG:boto:Method: GET 2022-09-03T06:47:38.535 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:47:38.536 DEBUG:boto:Data: 2022-09-03T06:47:38.537 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.539 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.541 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.542 DEBUG:boto:Params: {} 2022-09-03T06:47:38.543 DEBUG:boto:Token: None 2022-09-03T06:47:38.543 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj62 2022-09-03T06:47:38.544 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DpjOwhGed7mIefpYt4ZTuYo+ruQ= 2022-09-03T06:47:38.545 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DpjOwhGed7mIefpYt4ZTuYo+ruQ=', 'Content-Length': '0'} 2022-09-03T06:47:38.549 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b1c64b2edbcd8c20-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.551 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:38.552 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:38.553 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:38.554 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:38.555 DEBUG:boto:Method: GET 2022-09-03T06:47:38.556 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:47:38.557 DEBUG:boto:Data: 2022-09-03T06:47:38.558 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.560 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:38.561 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.561 DEBUG:boto:Params: {} 2022-09-03T06:47:38.562 DEBUG:boto:Token: None 2022-09-03T06:47:38.563 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/ 2022-09-03T06:47:38.564 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo= 2022-09-03T06:47:38.564 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo=', 'Content-Length': '0'} 2022-09-03T06:47:38.567 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a2636313cc7051e4-006312f88a-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.572 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:38.573 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:38.574 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:38.575 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:38.576 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:38.577 DEBUG:boto:Method: GET 2022-09-03T06:47:38.578 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:47:38.579 DEBUG:boto:Data: 2022-09-03T06:47:38.580 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.580 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.581 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.582 DEBUG:boto:Params: {} 2022-09-03T06:47:38.583 DEBUG:boto:Token: None 2022-09-03T06:47:38.584 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/ 2022-09-03T06:47:38.585 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo= 2022-09-03T06:47:38.586 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo=', 'Content-Length': '0'} 2022-09-03T06:47:38.590 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e710c62f604715bb-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.592 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:38.593 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:47:38.594 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:38.595 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:38.596 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.597 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.598 DEBUG:boto:Method: GET 2022-09-03T06:47:38.599 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.601 DEBUG:boto:Data: 2022-09-03T06:47:38.602 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.602 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:38.603 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.603 DEBUG:boto:Params: {} 2022-09-03T06:47:38.604 DEBUG:boto:Token: None 2022-09-03T06:47:38.605 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.605 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ujaur+is085d1wSyIBsXK8LciTk= 2022-09-03T06:47:38.606 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ujaur+is085d1wSyIBsXK8LciTk=', 'Content-Length': '0'} 2022-09-03T06:47:38.608 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000062d9751a59c9d526-006312f88a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.609 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:38.610 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:38.610 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.611 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.611 DEBUG:boto:Method: GET 2022-09-03T06:47:38.612 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.613 DEBUG:boto:Data: 2022-09-03T06:47:38.613 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.614 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.615 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.615 DEBUG:boto:Params: {} 2022-09-03T06:47:38.616 DEBUG:boto:Token: None 2022-09-03T06:47:38.617 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.618 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ujaur+is085d1wSyIBsXK8LciTk= 2022-09-03T06:47:38.619 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ujaur+is085d1wSyIBsXK8LciTk=', 'Content-Length': '0'} 2022-09-03T06:47:38.621 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008ed26e7b775f803d-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.622 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:38.623 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:38.623 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.624 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.624 DEBUG:boto:Method: HEAD 2022-09-03T06:47:38.625 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.626 DEBUG:boto:Data: 2022-09-03T06:47:38.626 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.627 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:38.628 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.629 DEBUG:boto:Params: {} 2022-09-03T06:47:38.630 DEBUG:boto:Token: None 2022-09-03T06:47:38.631 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.632 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TMz3GMAnn4NLYQ7MmkQyG+oAlgo= 2022-09-03T06:47:38.633 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TMz3GMAnn4NLYQ7MmkQyG+oAlgo=', 'Content-Length': '0'} 2022-09-03T06:47:38.635 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000400da0377cedbd31-006312f88a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.636 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:38.636 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:38.637 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.637 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.638 DEBUG:boto:Method: HEAD 2022-09-03T06:47:38.638 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.639 DEBUG:boto:Data: 2022-09-03T06:47:38.639 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.640 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.640 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.641 DEBUG:boto:Params: {} 2022-09-03T06:47:38.641 DEBUG:boto:Token: None 2022-09-03T06:47:38.642 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.643 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TMz3GMAnn4NLYQ7MmkQyG+oAlgo= 2022-09-03T06:47:38.643 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TMz3GMAnn4NLYQ7MmkQyG+oAlgo=', 'Content-Length': '0'} 2022-09-03T06:47:38.646 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f584686eb9bacb0f-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.647 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:47:38.647 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:38.648 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:38.648 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.649 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.650 DEBUG:boto:Method: GET 2022-09-03T06:47:38.650 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.651 DEBUG:boto:Data: 2022-09-03T06:47:38.651 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.652 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:38.652 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.653 DEBUG:boto:Params: {} 2022-09-03T06:47:38.653 DEBUG:boto:Token: None 2022-09-03T06:47:38.654 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.655 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ujaur+is085d1wSyIBsXK8LciTk= 2022-09-03T06:47:38.655 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ujaur+is085d1wSyIBsXK8LciTk=', 'Content-Length': '0'} 2022-09-03T06:47:38.658 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002779ed69cdbc48c8-006312f88a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.703 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:38.704 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:38.704 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.705 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.705 DEBUG:boto:Method: GET 2022-09-03T06:47:38.706 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.707 DEBUG:boto:Data: 2022-09-03T06:47:38.707 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.709 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.710 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.711 DEBUG:boto:Params: {} 2022-09-03T06:47:38.713 DEBUG:boto:Token: None 2022-09-03T06:47:38.714 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:38.714 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ujaur+is085d1wSyIBsXK8LciTk= 2022-09-03T06:47:38.715 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ujaur+is085d1wSyIBsXK8LciTk=', 'Content-Length': '0'} 2022-09-03T06:47:38.717 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000062de16463999efe3-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.718 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:38.720 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:38.721 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:38.722 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:38.724 DEBUG:boto:Method: GET 2022-09-03T06:47:38.725 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:47:38.726 DEBUG:boto:Data: 2022-09-03T06:47:38.727 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.728 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:38.729 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.729 DEBUG:boto:Params: {} 2022-09-03T06:47:38.730 DEBUG:boto:Token: None 2022-09-03T06:47:38.731 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/ 2022-09-03T06:47:38.732 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo= 2022-09-03T06:47:38.733 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo=', 'Content-Length': '0'} 2022-09-03T06:47:38.737 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000576fe3ed07fb89ca-006312f88a-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.738 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:38.740 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:38.740 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:38.741 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:38.741 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:38.742 DEBUG:boto:Method: GET 2022-09-03T06:47:38.742 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:47:38.743 DEBUG:boto:Data: 2022-09-03T06:47:38.743 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.744 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.745 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.746 DEBUG:boto:Params: {} 2022-09-03T06:47:38.746 DEBUG:boto:Token: None 2022-09-03T06:47:38.747 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/ 2022-09-03T06:47:38.747 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo= 2022-09-03T06:47:38.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo=', 'Content-Length': '0'} 2022-09-03T06:47:38.752 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007236e34f9a3120e0-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.753 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:38.755 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:38.759 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:38.760 DEBUG:boto:Method: HEAD 2022-09-03T06:47:38.761 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:47:38.762 DEBUG:boto:Data: 2022-09-03T06:47:38.763 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.764 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:38.764 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.765 DEBUG:boto:Params: {} 2022-09-03T06:47:38.765 DEBUG:boto:Token: None 2022-09-03T06:47:38.766 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj68 2022-09-03T06:47:38.766 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t3O3eOE5M83Jz42La7bCchZTpM8= 2022-09-03T06:47:38.767 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t3O3eOE5M83Jz42La7bCchZTpM8=', 'Content-Length': '0'} 2022-09-03T06:47:38.769 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad7629801b673e1d-006312f88a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.770 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:38.771 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:38.771 DEBUG:boto:Method: HEAD 2022-09-03T06:47:38.772 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:47:38.772 DEBUG:boto:Data: 2022-09-03T06:47:38.773 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.773 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.774 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.774 DEBUG:boto:Params: {} 2022-09-03T06:47:38.775 DEBUG:boto:Token: None 2022-09-03T06:47:38.775 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj68 2022-09-03T06:47:38.776 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t3O3eOE5M83Jz42La7bCchZTpM8= 2022-09-03T06:47:38.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t3O3eOE5M83Jz42La7bCchZTpM8=', 'Content-Length': '0'} 2022-09-03T06:47:38.779 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ca90b7219b8e7473-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.779 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:47:38.781 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:38.781 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:38.782 DEBUG:boto:Method: GET 2022-09-03T06:47:38.790 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:47:38.790 DEBUG:boto:Data: 2022-09-03T06:47:38.791 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.792 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:38.792 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.793 DEBUG:boto:Params: {} 2022-09-03T06:47:38.794 DEBUG:boto:Token: None 2022-09-03T06:47:38.795 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj68 2022-09-03T06:47:38.795 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6VFh6uEmI+yPTjb1DD7xgu/GfsA= 2022-09-03T06:47:38.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6VFh6uEmI+yPTjb1DD7xgu/GfsA=', 'Content-Length': '0'} 2022-09-03T06:47:38.799 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b66844df95650f65-006312f88a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.800 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:38.801 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:38.801 DEBUG:boto:Method: GET 2022-09-03T06:47:38.802 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:47:38.803 DEBUG:boto:Data: 2022-09-03T06:47:38.803 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.804 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.804 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.805 DEBUG:boto:Params: {} 2022-09-03T06:47:38.805 DEBUG:boto:Token: None 2022-09-03T06:47:38.806 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj68 2022-09-03T06:47:38.807 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6VFh6uEmI+yPTjb1DD7xgu/GfsA= 2022-09-03T06:47:38.807 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6VFh6uEmI+yPTjb1DD7xgu/GfsA=', 'Content-Length': '0'} 2022-09-03T06:47:38.809 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002db505bbd8438e86-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.850 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:38.852 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:38.853 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:38.855 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:38.856 DEBUG:boto:Method: GET 2022-09-03T06:47:38.858 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:47:38.859 DEBUG:boto:Data: 2022-09-03T06:47:38.859 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.860 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:38.861 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.862 DEBUG:boto:Params: {} 2022-09-03T06:47:38.863 DEBUG:boto:Token: None 2022-09-03T06:47:38.864 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/ 2022-09-03T06:47:38.865 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo= 2022-09-03T06:47:38.866 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo=', 'Content-Length': '0'} 2022-09-03T06:47:38.870 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ba66df6c0613a59b-006312f88a-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.871 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:38.872 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:38.873 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:38.874 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:38.875 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:38.876 DEBUG:boto:Method: GET 2022-09-03T06:47:38.877 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:47:38.878 DEBUG:boto:Data: 2022-09-03T06:47:38.879 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.880 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.881 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.882 DEBUG:boto:Params: {} 2022-09-03T06:47:38.883 DEBUG:boto:Token: None 2022-09-03T06:47:38.884 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/ 2022-09-03T06:47:38.885 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo= 2022-09-03T06:47:38.886 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:82+S9/hKJ7duc4P3IfBH3SXGfWo=', 'Content-Length': '0'} 2022-09-03T06:47:38.899 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000255a5e2a0a4f5fc0-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.900 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:38.902 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:47:38.903 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:38.904 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:38.904 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.905 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.906 DEBUG:boto:Method: GET 2022-09-03T06:47:38.907 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.908 DEBUG:boto:Data: 2022-09-03T06:47:38.909 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.910 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:38.911 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.911 DEBUG:boto:Params: {} 2022-09-03T06:47:38.912 DEBUG:boto:Token: None 2022-09-03T06:47:38.913 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.914 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kzqtDUyMWtBoXedvesgVimGoF5c= 2022-09-03T06:47:38.914 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kzqtDUyMWtBoXedvesgVimGoF5c=', 'Content-Length': '0'} 2022-09-03T06:47:38.918 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000050cb05e63eb60199-006312f88a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.920 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:38.921 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:38.922 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.923 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.924 DEBUG:boto:Method: GET 2022-09-03T06:47:38.925 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.926 DEBUG:boto:Data: 2022-09-03T06:47:38.926 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.927 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.927 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.928 DEBUG:boto:Params: {} 2022-09-03T06:47:38.929 DEBUG:boto:Token: None 2022-09-03T06:47:38.929 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.930 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kzqtDUyMWtBoXedvesgVimGoF5c= 2022-09-03T06:47:38.931 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kzqtDUyMWtBoXedvesgVimGoF5c=', 'Content-Length': '0'} 2022-09-03T06:47:38.934 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a70d39f38e7d136d-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.935 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:38.936 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:38.937 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.938 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.939 DEBUG:boto:Method: HEAD 2022-09-03T06:47:38.939 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.940 DEBUG:boto:Data: 2022-09-03T06:47:38.940 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.942 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:38.943 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.944 DEBUG:boto:Params: {} 2022-09-03T06:47:38.944 DEBUG:boto:Token: None 2022-09-03T06:47:38.945 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.946 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hy5LzRFsTa2ZESOvqR2iClLk3fY= 2022-09-03T06:47:38.946 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hy5LzRFsTa2ZESOvqR2iClLk3fY=', 'Content-Length': '0'} 2022-09-03T06:47:38.949 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd0d418d6e632a82-006312f88a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.949 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:38.950 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:38.951 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.951 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.952 DEBUG:boto:Method: HEAD 2022-09-03T06:47:38.953 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.953 DEBUG:boto:Data: 2022-09-03T06:47:38.954 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.954 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.955 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.955 DEBUG:boto:Params: {} 2022-09-03T06:47:38.956 DEBUG:boto:Token: None 2022-09-03T06:47:38.956 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.957 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hy5LzRFsTa2ZESOvqR2iClLk3fY= 2022-09-03T06:47:38.958 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hy5LzRFsTa2ZESOvqR2iClLk3fY=', 'Content-Length': '0'} 2022-09-03T06:47:38.960 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d3a957f6d6fd4fc-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.961 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:47:38.961 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:38.962 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:38.963 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.963 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.964 DEBUG:boto:Method: GET 2022-09-03T06:47:38.965 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.965 DEBUG:boto:Data: 2022-09-03T06:47:38.966 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.966 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:38.967 DEBUG:boto:Port: 8001 2022-09-03T06:47:38.967 DEBUG:boto:Params: {} 2022-09-03T06:47:38.968 DEBUG:boto:Token: None 2022-09-03T06:47:38.968 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.969 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kzqtDUyMWtBoXedvesgVimGoF5c= 2022-09-03T06:47:38.970 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kzqtDUyMWtBoXedvesgVimGoF5c=', 'Content-Length': '0'} 2022-09-03T06:47:38.972 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dcc108329701eaab-006312f88a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:38.973 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:38.974 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:38.975 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.975 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.976 DEBUG:boto:Method: GET 2022-09-03T06:47:38.976 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.977 DEBUG:boto:Data: 2022-09-03T06:47:38.978 DEBUG:boto:Headers: {} 2022-09-03T06:47:38.979 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:38.980 DEBUG:boto:Port: 8000 2022-09-03T06:47:38.980 DEBUG:boto:Params: {} 2022-09-03T06:47:38.981 DEBUG:boto:Token: None 2022-09-03T06:47:38.982 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:38 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:38.982 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kzqtDUyMWtBoXedvesgVimGoF5c= 2022-09-03T06:47:38.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:38 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kzqtDUyMWtBoXedvesgVimGoF5c=', 'Content-Length': '0'} 2022-09-03T06:47:38.994 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008a0427c98591f5db-006312f88a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:38 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.030 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:39.031 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:39.032 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:39.033 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:39.035 DEBUG:boto:Method: GET 2022-09-03T06:47:39.036 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:47:39.037 DEBUG:boto:Data: 2022-09-03T06:47:39.039 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.040 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:39.041 DEBUG:boto:Port: 8001 2022-09-03T06:47:39.042 DEBUG:boto:Params: {} 2022-09-03T06:47:39.043 DEBUG:boto:Token: None 2022-09-03T06:47:39.044 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/ 2022-09-03T06:47:39.045 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo= 2022-09-03T06:47:39.046 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo=', 'Content-Length': '0'} 2022-09-03T06:47:39.052 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f3c0764cf22f656b-006312f88b-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.054 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:39.055 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:39.056 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:39.057 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:39.058 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:39.059 DEBUG:boto:Method: GET 2022-09-03T06:47:39.060 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:47:39.061 DEBUG:boto:Data: 2022-09-03T06:47:39.062 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.063 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:39.065 DEBUG:boto:Port: 8000 2022-09-03T06:47:39.066 DEBUG:boto:Params: {} 2022-09-03T06:47:39.067 DEBUG:boto:Token: None 2022-09-03T06:47:39.068 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/ 2022-09-03T06:47:39.069 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo= 2022-09-03T06:47:39.070 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo=', 'Content-Length': '0'} 2022-09-03T06:47:39.076 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ac392fa50bebf28a-006312f88b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.077 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:39.078 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:39.078 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:39.079 DEBUG:boto:Method: HEAD 2022-09-03T06:47:39.079 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:47:39.080 DEBUG:boto:Data: 2022-09-03T06:47:39.081 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.082 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:39.082 DEBUG:boto:Port: 8001 2022-09-03T06:47:39.083 DEBUG:boto:Params: {} 2022-09-03T06:47:39.083 DEBUG:boto:Token: None 2022-09-03T06:47:39.085 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj69 2022-09-03T06:47:39.085 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XjWwgtAldRef9YssEtt4JaEAbOU= 2022-09-03T06:47:39.086 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XjWwgtAldRef9YssEtt4JaEAbOU=', 'Content-Length': '0'} 2022-09-03T06:47:39.088 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006c1facc276912edf-006312f88b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.089 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:39.089 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:39.090 DEBUG:boto:Method: HEAD 2022-09-03T06:47:39.091 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:47:39.091 DEBUG:boto:Data: 2022-09-03T06:47:39.092 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.092 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:39.093 DEBUG:boto:Port: 8000 2022-09-03T06:47:39.093 DEBUG:boto:Params: {} 2022-09-03T06:47:39.094 DEBUG:boto:Token: None 2022-09-03T06:47:39.095 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj69 2022-09-03T06:47:39.096 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XjWwgtAldRef9YssEtt4JaEAbOU= 2022-09-03T06:47:39.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XjWwgtAldRef9YssEtt4JaEAbOU=', 'Content-Length': '0'} 2022-09-03T06:47:39.100 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005272ab8902ee0bc4-006312f88b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.101 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:47:39.102 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:39.103 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:39.104 DEBUG:boto:Method: GET 2022-09-03T06:47:39.105 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:47:39.106 DEBUG:boto:Data: 2022-09-03T06:47:39.109 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.110 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:39.111 DEBUG:boto:Port: 8001 2022-09-03T06:47:39.112 DEBUG:boto:Params: {} 2022-09-03T06:47:39.113 DEBUG:boto:Token: None 2022-09-03T06:47:39.114 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj69 2022-09-03T06:47:39.116 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WUeq3gqwHvf8KFKakdzJeJl9yFs= 2022-09-03T06:47:39.118 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WUeq3gqwHvf8KFKakdzJeJl9yFs=', 'Content-Length': '0'} 2022-09-03T06:47:39.125 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005323dbe261e3d2d9-006312f88b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.126 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:39.127 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:39.129 DEBUG:boto:Method: GET 2022-09-03T06:47:39.130 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:47:39.131 DEBUG:boto:Data: 2022-09-03T06:47:39.132 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.133 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:39.134 DEBUG:boto:Port: 8000 2022-09-03T06:47:39.135 DEBUG:boto:Params: {} 2022-09-03T06:47:39.136 DEBUG:boto:Token: None 2022-09-03T06:47:39.137 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj69 2022-09-03T06:47:39.138 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WUeq3gqwHvf8KFKakdzJeJl9yFs= 2022-09-03T06:47:39.139 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WUeq3gqwHvf8KFKakdzJeJl9yFs=', 'Content-Length': '0'} 2022-09-03T06:47:39.142 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d029ec019fe727e-006312f88b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.143 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:39.144 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:39.145 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:39.145 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:39.146 DEBUG:boto:Method: GET 2022-09-03T06:47:39.147 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:47:39.148 DEBUG:boto:Data: 2022-09-03T06:47:39.149 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.150 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:39.151 DEBUG:boto:Port: 8001 2022-09-03T06:47:39.152 DEBUG:boto:Params: {} 2022-09-03T06:47:39.153 DEBUG:boto:Token: None 2022-09-03T06:47:39.154 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/ 2022-09-03T06:47:39.155 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo= 2022-09-03T06:47:39.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo=', 'Content-Length': '0'} 2022-09-03T06:47:39.160 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000042ac158ad4c56484-006312f88b-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.202 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:39.204 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:39.205 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:39.205 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:39.206 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:39.207 DEBUG:boto:Method: GET 2022-09-03T06:47:39.208 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:47:39.208 DEBUG:boto:Data: 2022-09-03T06:47:39.209 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.209 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:39.210 DEBUG:boto:Port: 8000 2022-09-03T06:47:39.210 DEBUG:boto:Params: {} 2022-09-03T06:47:39.211 DEBUG:boto:Token: None 2022-09-03T06:47:39.212 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/ 2022-09-03T06:47:39.213 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo= 2022-09-03T06:47:39.214 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo=', 'Content-Length': '0'} 2022-09-03T06:47:39.218 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000088b2a495eddead1e-006312f88b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.224 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:39.225 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:47:39.226 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:39.227 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:39.228 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.229 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.230 DEBUG:boto:Method: GET 2022-09-03T06:47:39.231 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.232 DEBUG:boto:Data: 2022-09-03T06:47:39.233 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.234 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:39.235 DEBUG:boto:Port: 8001 2022-09-03T06:47:39.236 DEBUG:boto:Params: {} 2022-09-03T06:47:39.237 DEBUG:boto:Token: None 2022-09-03T06:47:39.238 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.239 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wgJB1rYvbV3V8I5mR8h8YtMVydE= 2022-09-03T06:47:39.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wgJB1rYvbV3V8I5mR8h8YtMVydE=', 'Content-Length': '0'} 2022-09-03T06:47:39.243 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000083e4dd71bd7b6a23-006312f88b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.244 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:39.245 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:39.246 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.247 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.248 DEBUG:boto:Method: GET 2022-09-03T06:47:39.249 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.250 DEBUG:boto:Data: 2022-09-03T06:47:39.251 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.252 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:39.253 DEBUG:boto:Port: 8000 2022-09-03T06:47:39.254 DEBUG:boto:Params: {} 2022-09-03T06:47:39.255 DEBUG:boto:Token: None 2022-09-03T06:47:39.256 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.257 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wgJB1rYvbV3V8I5mR8h8YtMVydE= 2022-09-03T06:47:39.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wgJB1rYvbV3V8I5mR8h8YtMVydE=', 'Content-Length': '0'} 2022-09-03T06:47:39.260 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000066c44db1a881fb76-006312f88b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.261 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:39.261 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:39.262 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.262 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.263 DEBUG:boto:Method: HEAD 2022-09-03T06:47:39.263 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.264 DEBUG:boto:Data: 2022-09-03T06:47:39.264 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.265 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:39.265 DEBUG:boto:Port: 8001 2022-09-03T06:47:39.266 DEBUG:boto:Params: {} 2022-09-03T06:47:39.267 DEBUG:boto:Token: None 2022-09-03T06:47:39.267 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.268 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pCXUdgdCd5hMzgVInBthU+ShA9Q= 2022-09-03T06:47:39.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pCXUdgdCd5hMzgVInBthU+ShA9Q=', 'Content-Length': '0'} 2022-09-03T06:47:39.270 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e3cb6e64f273ffb-006312f88b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.271 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:39.272 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:39.272 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.273 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.273 DEBUG:boto:Method: HEAD 2022-09-03T06:47:39.274 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.274 DEBUG:boto:Data: 2022-09-03T06:47:39.275 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.276 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:39.276 DEBUG:boto:Port: 8000 2022-09-03T06:47:39.277 DEBUG:boto:Params: {} 2022-09-03T06:47:39.277 DEBUG:boto:Token: None 2022-09-03T06:47:39.278 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.278 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pCXUdgdCd5hMzgVInBthU+ShA9Q= 2022-09-03T06:47:39.279 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pCXUdgdCd5hMzgVInBthU+ShA9Q=', 'Content-Length': '0'} 2022-09-03T06:47:39.289 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000084f4aca3a313e5ae-006312f88b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.290 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:47:39.291 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:39.292 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:39.293 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.294 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.295 DEBUG:boto:Method: GET 2022-09-03T06:47:39.296 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.297 DEBUG:boto:Data: 2022-09-03T06:47:39.298 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.299 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:39.300 DEBUG:boto:Port: 8001 2022-09-03T06:47:39.301 DEBUG:boto:Params: {} 2022-09-03T06:47:39.302 DEBUG:boto:Token: None 2022-09-03T06:47:39.304 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.305 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wgJB1rYvbV3V8I5mR8h8YtMVydE= 2022-09-03T06:47:39.306 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wgJB1rYvbV3V8I5mR8h8YtMVydE=', 'Content-Length': '0'} 2022-09-03T06:47:39.309 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000401a4628798007e8-006312f88b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.350 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:39.352 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:39.353 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.354 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.355 DEBUG:boto:Method: GET 2022-09-03T06:47:39.356 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.357 DEBUG:boto:Data: 2022-09-03T06:47:39.358 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.359 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:39.361 DEBUG:boto:Port: 8000 2022-09-03T06:47:39.362 DEBUG:boto:Params: {} 2022-09-03T06:47:39.363 DEBUG:boto:Token: None 2022-09-03T06:47:39.365 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:39.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wgJB1rYvbV3V8I5mR8h8YtMVydE= 2022-09-03T06:47:39.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wgJB1rYvbV3V8I5mR8h8YtMVydE=', 'Content-Length': '0'} 2022-09-03T06:47:39.370 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002c87420715e35bfe-006312f88b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.372 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:39.373 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:39.373 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:39.374 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:39.375 DEBUG:boto:Method: GET 2022-09-03T06:47:39.375 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:47:39.376 DEBUG:boto:Data: 2022-09-03T06:47:39.376 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.377 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:39.377 DEBUG:boto:Port: 8001 2022-09-03T06:47:39.378 DEBUG:boto:Params: {} 2022-09-03T06:47:39.378 DEBUG:boto:Token: None 2022-09-03T06:47:39.379 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/ 2022-09-03T06:47:39.380 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo= 2022-09-03T06:47:39.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo=', 'Content-Length': '0'} 2022-09-03T06:47:39.383 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c2c43d6b382ada32-006312f88b-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.384 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:39.385 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:39.386 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:39.386 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:39.387 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:39.387 DEBUG:boto:Method: GET 2022-09-03T06:47:39.388 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:47:39.389 DEBUG:boto:Data: 2022-09-03T06:47:39.389 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.390 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:39.390 DEBUG:boto:Port: 8000 2022-09-03T06:47:39.391 DEBUG:boto:Params: {} 2022-09-03T06:47:39.391 DEBUG:boto:Token: None 2022-09-03T06:47:39.392 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/ 2022-09-03T06:47:39.393 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo= 2022-09-03T06:47:39.393 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo=', 'Content-Length': '0'} 2022-09-03T06:47:39.399 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006652002789c9f31d-006312f88b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.446 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:39.448 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:39.449 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:39.450 DEBUG:boto:Method: HEAD 2022-09-03T06:47:39.451 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:47:39.454 DEBUG:boto:Data: 2022-09-03T06:47:39.455 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.455 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:39.456 DEBUG:boto:Port: 8001 2022-09-03T06:47:39.457 DEBUG:boto:Params: {} 2022-09-03T06:47:39.457 DEBUG:boto:Token: None 2022-09-03T06:47:39.458 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj76 2022-09-03T06:47:39.459 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k6xSaH0IUXnumGSAXYedXrVMXh4= 2022-09-03T06:47:39.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k6xSaH0IUXnumGSAXYedXrVMXh4=', 'Content-Length': '0'} 2022-09-03T06:47:39.463 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a2407525e2aa7fe-006312f88b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.465 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:39.466 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:39.467 DEBUG:boto:Method: HEAD 2022-09-03T06:47:39.467 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:47:39.468 DEBUG:boto:Data: 2022-09-03T06:47:39.468 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.469 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:39.469 DEBUG:boto:Port: 8000 2022-09-03T06:47:39.470 DEBUG:boto:Params: {} 2022-09-03T06:47:39.470 DEBUG:boto:Token: None 2022-09-03T06:47:39.471 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj76 2022-09-03T06:47:39.472 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k6xSaH0IUXnumGSAXYedXrVMXh4= 2022-09-03T06:47:39.472 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k6xSaH0IUXnumGSAXYedXrVMXh4=', 'Content-Length': '0'} 2022-09-03T06:47:39.474 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d5e7000789cd4490-006312f88b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.475 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:47:39.476 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:39.476 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:39.477 DEBUG:boto:Method: GET 2022-09-03T06:47:39.477 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:47:39.478 DEBUG:boto:Data: 2022-09-03T06:47:39.478 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.479 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:39.480 DEBUG:boto:Port: 8001 2022-09-03T06:47:39.481 DEBUG:boto:Params: {} 2022-09-03T06:47:39.482 DEBUG:boto:Token: None 2022-09-03T06:47:39.483 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj76 2022-09-03T06:47:39.485 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zSdYpVapDTxlU5aqmLWJtV8hyQc= 2022-09-03T06:47:39.486 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zSdYpVapDTxlU5aqmLWJtV8hyQc=', 'Content-Length': '0'} 2022-09-03T06:47:39.489 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d090f06874ee5e21-006312f88b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.491 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:39.492 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:39.493 DEBUG:boto:Method: GET 2022-09-03T06:47:39.494 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:47:39.495 DEBUG:boto:Data: 2022-09-03T06:47:39.496 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.497 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:39.498 DEBUG:boto:Port: 8000 2022-09-03T06:47:39.499 DEBUG:boto:Params: {} 2022-09-03T06:47:39.500 DEBUG:boto:Token: None 2022-09-03T06:47:39.501 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj76 2022-09-03T06:47:39.502 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zSdYpVapDTxlU5aqmLWJtV8hyQc= 2022-09-03T06:47:39.503 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zSdYpVapDTxlU5aqmLWJtV8hyQc=', 'Content-Length': '0'} 2022-09-03T06:47:39.506 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000064607ad5e2c0716a-006312f88b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.508 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:39.509 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:39.510 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:39.510 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:39.511 DEBUG:boto:Method: GET 2022-09-03T06:47:39.511 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:47:39.512 DEBUG:boto:Data: 2022-09-03T06:47:39.513 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.513 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:39.514 DEBUG:boto:Port: 8001 2022-09-03T06:47:39.515 DEBUG:boto:Params: {} 2022-09-03T06:47:39.516 DEBUG:boto:Token: None 2022-09-03T06:47:39.516 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/ 2022-09-03T06:47:39.517 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo= 2022-09-03T06:47:39.518 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo=', 'Content-Length': '0'} 2022-09-03T06:47:39.521 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d0cb78f179b7f0f7-006312f88b-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.522 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:39.522 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:39.523 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:39.524 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:39.526 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:39.527 DEBUG:boto:Method: GET 2022-09-03T06:47:39.528 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:47:39.529 DEBUG:boto:Data: 2022-09-03T06:47:39.530 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.531 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:39.531 DEBUG:boto:Port: 8000 2022-09-03T06:47:39.532 DEBUG:boto:Params: {} 2022-09-03T06:47:39.533 DEBUG:boto:Token: None 2022-09-03T06:47:39.534 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/ 2022-09-03T06:47:39.535 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo= 2022-09-03T06:47:39.535 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo=', 'Content-Length': '0'} 2022-09-03T06:47:39.540 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000106bb44c5c65961b-006312f88b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.586 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:39.587 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:47:39.588 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:39.588 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:39.589 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.589 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.590 DEBUG:boto:Method: GET 2022-09-03T06:47:39.591 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.591 DEBUG:boto:Data: 2022-09-03T06:47:39.592 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.592 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:39.593 DEBUG:boto:Port: 8001 2022-09-03T06:47:39.593 DEBUG:boto:Params: {} 2022-09-03T06:47:39.594 DEBUG:boto:Token: None 2022-09-03T06:47:39.595 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.595 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:swm3k1LHrIr3ndYL6GCcv27cAIg= 2022-09-03T06:47:39.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:swm3k1LHrIr3ndYL6GCcv27cAIg=', 'Content-Length': '0'} 2022-09-03T06:47:39.599 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a8896e6f8b3fb4cb-006312f88b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.600 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:39.601 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:39.602 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.603 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.603 DEBUG:boto:Method: GET 2022-09-03T06:47:39.604 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.605 DEBUG:boto:Data: 2022-09-03T06:47:39.605 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.606 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:39.606 DEBUG:boto:Port: 8000 2022-09-03T06:47:39.607 DEBUG:boto:Params: {} 2022-09-03T06:47:39.607 DEBUG:boto:Token: None 2022-09-03T06:47:39.608 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.609 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:swm3k1LHrIr3ndYL6GCcv27cAIg= 2022-09-03T06:47:39.609 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:swm3k1LHrIr3ndYL6GCcv27cAIg=', 'Content-Length': '0'} 2022-09-03T06:47:39.612 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b6ac147f5c3df3eb-006312f88b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.614 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:39.614 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:39.615 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.615 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.616 DEBUG:boto:Method: HEAD 2022-09-03T06:47:39.617 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.617 DEBUG:boto:Data: 2022-09-03T06:47:39.618 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.618 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:39.619 DEBUG:boto:Port: 8001 2022-09-03T06:47:39.620 DEBUG:boto:Params: {} 2022-09-03T06:47:39.620 DEBUG:boto:Token: None 2022-09-03T06:47:39.621 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.621 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v6yT46UFNZzkKJkOx/jUiw33eGc= 2022-09-03T06:47:39.622 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v6yT46UFNZzkKJkOx/jUiw33eGc=', 'Content-Length': '0'} 2022-09-03T06:47:39.625 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f677c9648ab2737f-006312f88b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.625 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:39.626 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:39.627 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.627 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.628 DEBUG:boto:Method: HEAD 2022-09-03T06:47:39.628 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.629 DEBUG:boto:Data: 2022-09-03T06:47:39.629 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.630 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:39.630 DEBUG:boto:Port: 8000 2022-09-03T06:47:39.631 DEBUG:boto:Params: {} 2022-09-03T06:47:39.631 DEBUG:boto:Token: None 2022-09-03T06:47:39.632 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.632 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v6yT46UFNZzkKJkOx/jUiw33eGc= 2022-09-03T06:47:39.633 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v6yT46UFNZzkKJkOx/jUiw33eGc=', 'Content-Length': '0'} 2022-09-03T06:47:39.635 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b1a9c78d3c58034-006312f88b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.637 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:47:39.637 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:39.638 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:39.639 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.639 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.640 DEBUG:boto:Method: GET 2022-09-03T06:47:39.640 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.641 DEBUG:boto:Data: 2022-09-03T06:47:39.642 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.643 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:39.643 DEBUG:boto:Port: 8001 2022-09-03T06:47:39.644 DEBUG:boto:Params: {} 2022-09-03T06:47:39.644 DEBUG:boto:Token: None 2022-09-03T06:47:39.645 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.646 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:swm3k1LHrIr3ndYL6GCcv27cAIg= 2022-09-03T06:47:39.646 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:swm3k1LHrIr3ndYL6GCcv27cAIg=', 'Content-Length': '0'} 2022-09-03T06:47:39.649 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005856a8e4617b20c2-006312f88b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.694 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:39.696 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:39.697 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.698 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.699 DEBUG:boto:Method: GET 2022-09-03T06:47:39.700 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.701 DEBUG:boto:Data: 2022-09-03T06:47:39.702 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.704 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:39.706 DEBUG:boto:Port: 8000 2022-09-03T06:47:39.707 DEBUG:boto:Params: {} 2022-09-03T06:47:39.708 DEBUG:boto:Token: None 2022-09-03T06:47:39.708 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:39.709 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:swm3k1LHrIr3ndYL6GCcv27cAIg= 2022-09-03T06:47:39.710 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:swm3k1LHrIr3ndYL6GCcv27cAIg=', 'Content-Length': '0'} 2022-09-03T06:47:39.713 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000297cb36dcddd9400-006312f88b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.714 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:39.714 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:39.715 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:39.715 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:39.716 DEBUG:boto:Method: GET 2022-09-03T06:47:39.717 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:47:39.717 DEBUG:boto:Data: 2022-09-03T06:47:39.718 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.718 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:39.719 DEBUG:boto:Port: 8001 2022-09-03T06:47:39.719 DEBUG:boto:Params: {} 2022-09-03T06:47:39.720 DEBUG:boto:Token: None 2022-09-03T06:47:39.720 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/ 2022-09-03T06:47:39.721 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo= 2022-09-03T06:47:39.722 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo=', 'Content-Length': '0'} 2022-09-03T06:47:39.725 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000b7d8e4412183917-006312f88b-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.726 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:39.727 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:39.728 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:39.729 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:39.730 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:39.731 DEBUG:boto:Method: GET 2022-09-03T06:47:39.732 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:47:39.733 DEBUG:boto:Data: 2022-09-03T06:47:39.734 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.735 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:39.736 DEBUG:boto:Port: 8000 2022-09-03T06:47:39.737 DEBUG:boto:Params: {} 2022-09-03T06:47:39.738 DEBUG:boto:Token: None 2022-09-03T06:47:39.740 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/ 2022-09-03T06:47:39.741 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo= 2022-09-03T06:47:39.742 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo=', 'Content-Length': '0'} 2022-09-03T06:47:39.746 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000821cdb39e0881c1f-006312f88b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.747 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:39.749 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:39.750 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:39.751 DEBUG:boto:Method: HEAD 2022-09-03T06:47:39.752 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:47:39.753 DEBUG:boto:Data: 2022-09-03T06:47:39.754 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.755 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:39.757 DEBUG:boto:Port: 8001 2022-09-03T06:47:39.758 DEBUG:boto:Params: {} 2022-09-03T06:47:39.759 DEBUG:boto:Token: None 2022-09-03T06:47:39.759 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj85 2022-09-03T06:47:39.760 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AnDozvJ7kPrGJcEB2hYK0+t4gsg= 2022-09-03T06:47:39.762 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AnDozvJ7kPrGJcEB2hYK0+t4gsg=', 'Content-Length': '0'} 2022-09-03T06:47:39.764 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000056791fe3fa564333-006312f88b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.765 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:39.766 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:39.766 DEBUG:boto:Method: HEAD 2022-09-03T06:47:39.767 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:47:39.768 DEBUG:boto:Data: 2022-09-03T06:47:39.768 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.769 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:39.769 DEBUG:boto:Port: 8000 2022-09-03T06:47:39.770 DEBUG:boto:Params: {} 2022-09-03T06:47:39.770 DEBUG:boto:Token: None 2022-09-03T06:47:39.771 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj85 2022-09-03T06:47:39.772 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AnDozvJ7kPrGJcEB2hYK0+t4gsg= 2022-09-03T06:47:39.772 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AnDozvJ7kPrGJcEB2hYK0+t4gsg=', 'Content-Length': '0'} 2022-09-03T06:47:39.775 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000008f72a39920debf7-006312f88b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.775 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:47:39.776 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:39.777 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:39.778 DEBUG:boto:Method: GET 2022-09-03T06:47:39.778 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:47:39.779 DEBUG:boto:Data: 2022-09-03T06:47:39.780 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.781 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:39.782 DEBUG:boto:Port: 8001 2022-09-03T06:47:39.782 DEBUG:boto:Params: {} 2022-09-03T06:47:39.783 DEBUG:boto:Token: None 2022-09-03T06:47:39.783 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj85 2022-09-03T06:47:39.785 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BoKsl9SGqr73NhSek2CY95bs+Gs= 2022-09-03T06:47:39.786 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BoKsl9SGqr73NhSek2CY95bs+Gs=', 'Content-Length': '0'} 2022-09-03T06:47:39.789 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003de8a38cf0f1308e-006312f88b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.790 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:39.791 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:39.791 DEBUG:boto:Method: GET 2022-09-03T06:47:39.792 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:47:39.792 DEBUG:boto:Data: 2022-09-03T06:47:39.793 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.793 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:39.794 DEBUG:boto:Port: 8000 2022-09-03T06:47:39.794 DEBUG:boto:Params: {} 2022-09-03T06:47:39.795 DEBUG:boto:Token: None 2022-09-03T06:47:39.796 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj85 2022-09-03T06:47:39.796 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BoKsl9SGqr73NhSek2CY95bs+Gs= 2022-09-03T06:47:39.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BoKsl9SGqr73NhSek2CY95bs+Gs=', 'Content-Length': '0'} 2022-09-03T06:47:39.799 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000353d9dc56b8c657f-006312f88b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.842 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:39.843 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:39.844 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:39.844 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:39.845 DEBUG:boto:Method: GET 2022-09-03T06:47:39.846 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:47:39.847 DEBUG:boto:Data: 2022-09-03T06:47:39.848 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.848 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:39.849 DEBUG:boto:Port: 8001 2022-09-03T06:47:39.850 DEBUG:boto:Params: {} 2022-09-03T06:47:39.851 DEBUG:boto:Token: None 2022-09-03T06:47:39.851 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/ 2022-09-03T06:47:39.852 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo= 2022-09-03T06:47:39.853 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo=', 'Content-Length': '0'} 2022-09-03T06:47:39.856 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001a8750066910d23d-006312f88b-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.857 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:39.858 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:39.858 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:39.859 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:39.860 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:39.860 DEBUG:boto:Method: GET 2022-09-03T06:47:39.861 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:47:39.861 DEBUG:boto:Data: 2022-09-03T06:47:39.862 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.862 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:39.863 DEBUG:boto:Port: 8000 2022-09-03T06:47:39.863 DEBUG:boto:Params: {} 2022-09-03T06:47:39.864 DEBUG:boto:Token: None 2022-09-03T06:47:39.865 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/ 2022-09-03T06:47:39.865 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo= 2022-09-03T06:47:39.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i/AA9qbs6p71Mdd3H17H2TzgXBo=', 'Content-Length': '0'} 2022-09-03T06:47:39.871 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002232b310d449daa4-006312f88b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.873 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:39.874 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:47:39.875 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:39.875 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:39.876 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.877 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.877 DEBUG:boto:Method: GET 2022-09-03T06:47:39.878 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.879 DEBUG:boto:Data: 2022-09-03T06:47:39.880 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.881 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:39.882 DEBUG:boto:Port: 8001 2022-09-03T06:47:39.883 DEBUG:boto:Params: {} 2022-09-03T06:47:39.884 DEBUG:boto:Token: None 2022-09-03T06:47:39.885 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.886 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ivL0blPsx9D42JGmQqmgGtUwu3M= 2022-09-03T06:47:39.886 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ivL0blPsx9D42JGmQqmgGtUwu3M=', 'Content-Length': '0'} 2022-09-03T06:47:39.888 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000472cb0d3f168e8e2-006312f88b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.889 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:39.890 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:39.890 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.891 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.892 DEBUG:boto:Method: GET 2022-09-03T06:47:39.892 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.893 DEBUG:boto:Data: 2022-09-03T06:47:39.893 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.894 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:39.894 DEBUG:boto:Port: 8000 2022-09-03T06:47:39.895 DEBUG:boto:Params: {} 2022-09-03T06:47:39.896 DEBUG:boto:Token: None 2022-09-03T06:47:39.896 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.897 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ivL0blPsx9D42JGmQqmgGtUwu3M= 2022-09-03T06:47:39.897 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ivL0blPsx9D42JGmQqmgGtUwu3M=', 'Content-Length': '0'} 2022-09-03T06:47:39.900 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e9342e611cf6645-006312f88b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.901 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:39.902 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:39.903 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.903 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.904 DEBUG:boto:Method: HEAD 2022-09-03T06:47:39.904 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.905 DEBUG:boto:Data: 2022-09-03T06:47:39.905 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.906 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:39.906 DEBUG:boto:Port: 8001 2022-09-03T06:47:39.907 DEBUG:boto:Params: {} 2022-09-03T06:47:39.908 DEBUG:boto:Token: None 2022-09-03T06:47:39.908 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.909 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6hF+WTDiutHunfvbBzbq4LnSvXM= 2022-09-03T06:47:39.910 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6hF+WTDiutHunfvbBzbq4LnSvXM=', 'Content-Length': '0'} 2022-09-03T06:47:39.912 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000073f24cec0168bf64-006312f88b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.912 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:39.913 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:39.913 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.914 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.915 DEBUG:boto:Method: HEAD 2022-09-03T06:47:39.915 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.916 DEBUG:boto:Data: 2022-09-03T06:47:39.916 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.917 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:39.917 DEBUG:boto:Port: 8000 2022-09-03T06:47:39.918 DEBUG:boto:Params: {} 2022-09-03T06:47:39.918 DEBUG:boto:Token: None 2022-09-03T06:47:39.919 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.919 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6hF+WTDiutHunfvbBzbq4LnSvXM= 2022-09-03T06:47:39.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6hF+WTDiutHunfvbBzbq4LnSvXM=', 'Content-Length': '0'} 2022-09-03T06:47:39.922 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007ead46c3c94e8ad6-006312f88b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.923 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:47:39.924 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:39.924 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:39.925 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.926 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.926 DEBUG:boto:Method: GET 2022-09-03T06:47:39.927 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.928 DEBUG:boto:Data: 2022-09-03T06:47:39.929 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.930 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:39.931 DEBUG:boto:Port: 8001 2022-09-03T06:47:39.931 DEBUG:boto:Params: {} 2022-09-03T06:47:39.932 DEBUG:boto:Token: None 2022-09-03T06:47:39.933 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.933 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ivL0blPsx9D42JGmQqmgGtUwu3M= 2022-09-03T06:47:39.935 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ivL0blPsx9D42JGmQqmgGtUwu3M=', 'Content-Length': '0'} 2022-09-03T06:47:39.937 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000282ec7c38dc1411-006312f88b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.938 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:39.939 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:39.939 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.940 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.941 DEBUG:boto:Method: GET 2022-09-03T06:47:39.942 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.942 DEBUG:boto:Data: 2022-09-03T06:47:39.943 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.944 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:39.945 DEBUG:boto:Port: 8000 2022-09-03T06:47:39.946 DEBUG:boto:Params: {} 2022-09-03T06:47:39.947 DEBUG:boto:Token: None 2022-09-03T06:47:39.948 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:39 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:39.949 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ivL0blPsx9D42JGmQqmgGtUwu3M= 2022-09-03T06:47:39.950 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ivL0blPsx9D42JGmQqmgGtUwu3M=', 'Content-Length': '0'} 2022-09-03T06:47:39.953 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e8c74a6944d4963-006312f88b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:39.994 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:39.995 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:39.996 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:39.996 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:39.997 DEBUG:boto:Method: GET 2022-09-03T06:47:39.997 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:47:39.998 DEBUG:boto:Data: 2022-09-03T06:47:39.998 DEBUG:boto:Headers: {} 2022-09-03T06:47:39.999 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.000 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.000 DEBUG:boto:Params: {} 2022-09-03T06:47:40.001 DEBUG:boto:Token: None 2022-09-03T06:47:40.001 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/ 2022-09-03T06:47:40.002 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24= 2022-09-03T06:47:40.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24=', 'Content-Length': '0'} 2022-09-03T06:47:40.005 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000794006ce4fff67ce-006312f88c-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.006 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:40.007 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:40.008 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:40.008 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:40.009 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:40.010 DEBUG:boto:Method: GET 2022-09-03T06:47:40.010 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:47:40.011 DEBUG:boto:Data: 2022-09-03T06:47:40.011 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.012 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.013 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.013 DEBUG:boto:Params: {} 2022-09-03T06:47:40.014 DEBUG:boto:Token: None 2022-09-03T06:47:40.015 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/ 2022-09-03T06:47:40.016 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24= 2022-09-03T06:47:40.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24=', 'Content-Length': '0'} 2022-09-03T06:47:40.020 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c715366066f87617-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.021 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:40.022 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:40.023 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:40.023 DEBUG:boto:Method: HEAD 2022-09-03T06:47:40.024 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:47:40.025 DEBUG:boto:Data: 2022-09-03T06:47:40.026 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.026 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.027 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.028 DEBUG:boto:Params: {} 2022-09-03T06:47:40.028 DEBUG:boto:Token: None 2022-09-03T06:47:40.029 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj87 2022-09-03T06:47:40.029 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IvH2Mfk6mzp+D/fog7LbfVresSE= 2022-09-03T06:47:40.030 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IvH2Mfk6mzp+D/fog7LbfVresSE=', 'Content-Length': '0'} 2022-09-03T06:47:40.032 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ae541141692aaaa-006312f88c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.033 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:40.034 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:40.034 DEBUG:boto:Method: HEAD 2022-09-03T06:47:40.035 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:47:40.036 DEBUG:boto:Data: 2022-09-03T06:47:40.037 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.038 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.040 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.041 DEBUG:boto:Params: {} 2022-09-03T06:47:40.043 DEBUG:boto:Token: None 2022-09-03T06:47:40.044 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj87 2022-09-03T06:47:40.045 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IvH2Mfk6mzp+D/fog7LbfVresSE= 2022-09-03T06:47:40.047 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IvH2Mfk6mzp+D/fog7LbfVresSE=', 'Content-Length': '0'} 2022-09-03T06:47:40.051 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a5c107a9b0cc5c7-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.052 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:47:40.052 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:40.053 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:40.053 DEBUG:boto:Method: GET 2022-09-03T06:47:40.054 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:47:40.054 DEBUG:boto:Data: 2022-09-03T06:47:40.055 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.056 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.056 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.057 DEBUG:boto:Params: {} 2022-09-03T06:47:40.057 DEBUG:boto:Token: None 2022-09-03T06:47:40.058 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj87 2022-09-03T06:47:40.059 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Dww8gbcpAaVGrbdU4fYoZaDYowY= 2022-09-03T06:47:40.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Dww8gbcpAaVGrbdU4fYoZaDYowY=', 'Content-Length': '0'} 2022-09-03T06:47:40.063 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000024375e47cd407f9c-006312f88c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.065 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:40.066 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:40.067 DEBUG:boto:Method: GET 2022-09-03T06:47:40.068 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:47:40.069 DEBUG:boto:Data: 2022-09-03T06:47:40.070 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.071 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.072 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.073 DEBUG:boto:Params: {} 2022-09-03T06:47:40.073 DEBUG:boto:Token: None 2022-09-03T06:47:40.074 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj87 2022-09-03T06:47:40.075 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Dww8gbcpAaVGrbdU4fYoZaDYowY= 2022-09-03T06:47:40.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Dww8gbcpAaVGrbdU4fYoZaDYowY=', 'Content-Length': '0'} 2022-09-03T06:47:40.078 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026ac2898c68a708b-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.079 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:40.080 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:40.081 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:40.081 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:40.082 DEBUG:boto:Method: GET 2022-09-03T06:47:40.083 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:47:40.084 DEBUG:boto:Data: 2022-09-03T06:47:40.084 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.085 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.086 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.087 DEBUG:boto:Params: {} 2022-09-03T06:47:40.088 DEBUG:boto:Token: None 2022-09-03T06:47:40.089 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/ 2022-09-03T06:47:40.090 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24= 2022-09-03T06:47:40.091 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24=', 'Content-Length': '0'} 2022-09-03T06:47:40.094 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a4ae193a313eaedf-006312f88c-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.096 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:40.097 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:40.097 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:40.098 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:40.099 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:40.100 DEBUG:boto:Method: GET 2022-09-03T06:47:40.101 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:47:40.102 DEBUG:boto:Data: 2022-09-03T06:47:40.103 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.104 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.106 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.107 DEBUG:boto:Params: {} 2022-09-03T06:47:40.108 DEBUG:boto:Token: None 2022-09-03T06:47:40.108 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/ 2022-09-03T06:47:40.109 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24= 2022-09-03T06:47:40.110 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24=', 'Content-Length': '0'} 2022-09-03T06:47:40.115 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b6a09df1fb434678-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.162 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:40.163 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:47:40.164 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:40.165 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:40.166 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.166 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.167 DEBUG:boto:Method: GET 2022-09-03T06:47:40.167 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.168 DEBUG:boto:Data: 2022-09-03T06:47:40.168 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.169 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.169 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.170 DEBUG:boto:Params: {} 2022-09-03T06:47:40.171 DEBUG:boto:Token: None 2022-09-03T06:47:40.171 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.172 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7piugc8JagWWytav/FPCRLMFU0w= 2022-09-03T06:47:40.173 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7piugc8JagWWytav/FPCRLMFU0w=', 'Content-Length': '0'} 2022-09-03T06:47:40.176 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ae008b101354224b-006312f88c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.177 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:40.178 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:40.179 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.180 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.181 DEBUG:boto:Method: GET 2022-09-03T06:47:40.182 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.183 DEBUG:boto:Data: 2022-09-03T06:47:40.183 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.184 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.184 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.185 DEBUG:boto:Params: {} 2022-09-03T06:47:40.186 DEBUG:boto:Token: None 2022-09-03T06:47:40.187 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.187 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7piugc8JagWWytav/FPCRLMFU0w= 2022-09-03T06:47:40.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7piugc8JagWWytav/FPCRLMFU0w=', 'Content-Length': '0'} 2022-09-03T06:47:40.191 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000023cb7fd07b3b4bc8-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.192 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:40.192 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:40.193 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.193 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.194 DEBUG:boto:Method: HEAD 2022-09-03T06:47:40.194 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.195 DEBUG:boto:Data: 2022-09-03T06:47:40.196 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.196 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.197 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.197 DEBUG:boto:Params: {} 2022-09-03T06:47:40.198 DEBUG:boto:Token: None 2022-09-03T06:47:40.198 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.199 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y7p0utequPj6GkjmVanKTNnPlkc= 2022-09-03T06:47:40.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y7p0utequPj6GkjmVanKTNnPlkc=', 'Content-Length': '0'} 2022-09-03T06:47:40.201 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e783db466556357d-006312f88c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.202 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:40.203 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:40.204 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.204 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.205 DEBUG:boto:Method: HEAD 2022-09-03T06:47:40.205 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.206 DEBUG:boto:Data: 2022-09-03T06:47:40.206 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.207 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.207 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.208 DEBUG:boto:Params: {} 2022-09-03T06:47:40.208 DEBUG:boto:Token: None 2022-09-03T06:47:40.209 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.209 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y7p0utequPj6GkjmVanKTNnPlkc= 2022-09-03T06:47:40.210 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y7p0utequPj6GkjmVanKTNnPlkc=', 'Content-Length': '0'} 2022-09-03T06:47:40.212 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000066f072f94c5858ce-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.213 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:47:40.214 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:40.214 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:40.215 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.215 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.216 DEBUG:boto:Method: GET 2022-09-03T06:47:40.217 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.217 DEBUG:boto:Data: 2022-09-03T06:47:40.218 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.218 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.219 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.220 DEBUG:boto:Params: {} 2022-09-03T06:47:40.220 DEBUG:boto:Token: None 2022-09-03T06:47:40.221 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.222 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7piugc8JagWWytav/FPCRLMFU0w= 2022-09-03T06:47:40.222 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7piugc8JagWWytav/FPCRLMFU0w=', 'Content-Length': '0'} 2022-09-03T06:47:40.225 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000095db8092a4418db-006312f88c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.225 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:40.226 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:40.227 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.227 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.228 DEBUG:boto:Method: GET 2022-09-03T06:47:40.228 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.229 DEBUG:boto:Data: 2022-09-03T06:47:40.229 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.230 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.230 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.231 DEBUG:boto:Params: {} 2022-09-03T06:47:40.232 DEBUG:boto:Token: None 2022-09-03T06:47:40.232 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:40.233 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7piugc8JagWWytav/FPCRLMFU0w= 2022-09-03T06:47:40.234 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7piugc8JagWWytav/FPCRLMFU0w=', 'Content-Length': '0'} 2022-09-03T06:47:40.236 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e7ebdcf9bc454ca5-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.237 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:40.238 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:40.238 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:40.239 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:40.239 DEBUG:boto:Method: GET 2022-09-03T06:47:40.240 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:47:40.241 DEBUG:boto:Data: 2022-09-03T06:47:40.241 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.242 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.242 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.243 DEBUG:boto:Params: {} 2022-09-03T06:47:40.244 DEBUG:boto:Token: None 2022-09-03T06:47:40.244 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/ 2022-09-03T06:47:40.245 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24= 2022-09-03T06:47:40.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24=', 'Content-Length': '0'} 2022-09-03T06:47:40.249 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f0449f0511394f08-006312f88c-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.249 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:40.251 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:40.252 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:40.252 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:40.253 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:40.254 DEBUG:boto:Method: GET 2022-09-03T06:47:40.254 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:47:40.255 DEBUG:boto:Data: 2022-09-03T06:47:40.255 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.256 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.257 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.258 DEBUG:boto:Params: {} 2022-09-03T06:47:40.258 DEBUG:boto:Token: None 2022-09-03T06:47:40.259 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/ 2022-09-03T06:47:40.260 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24= 2022-09-03T06:47:40.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24=', 'Content-Length': '0'} 2022-09-03T06:47:40.265 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dd75dba353799b69-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.266 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:40.267 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:40.268 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:40.269 DEBUG:boto:Method: HEAD 2022-09-03T06:47:40.270 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:47:40.270 DEBUG:boto:Data: 2022-09-03T06:47:40.271 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.271 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.272 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.273 DEBUG:boto:Params: {} 2022-09-03T06:47:40.273 DEBUG:boto:Token: None 2022-09-03T06:47:40.274 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj92 2022-09-03T06:47:40.274 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:U6K22tIkV3AdKCem14Is+5JF9nk= 2022-09-03T06:47:40.275 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:U6K22tIkV3AdKCem14Is+5JF9nk=', 'Content-Length': '0'} 2022-09-03T06:47:40.278 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002acf7e500d9e2255-006312f88c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.279 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:40.282 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:40.283 DEBUG:boto:Method: HEAD 2022-09-03T06:47:40.284 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:47:40.285 DEBUG:boto:Data: 2022-09-03T06:47:40.286 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.286 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.287 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.288 DEBUG:boto:Params: {} 2022-09-03T06:47:40.289 DEBUG:boto:Token: None 2022-09-03T06:47:40.289 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj92 2022-09-03T06:47:40.293 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:U6K22tIkV3AdKCem14Is+5JF9nk= 2022-09-03T06:47:40.294 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:U6K22tIkV3AdKCem14Is+5JF9nk=', 'Content-Length': '0'} 2022-09-03T06:47:40.297 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000045856685dc0740fe-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.298 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:47:40.299 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:40.300 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:40.301 DEBUG:boto:Method: GET 2022-09-03T06:47:40.301 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:47:40.302 DEBUG:boto:Data: 2022-09-03T06:47:40.303 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.303 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.304 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.304 DEBUG:boto:Params: {} 2022-09-03T06:47:40.305 DEBUG:boto:Token: None 2022-09-03T06:47:40.306 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj92 2022-09-03T06:47:40.307 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uphspZArO7SDUkY1i9y6pwS7tGI= 2022-09-03T06:47:40.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uphspZArO7SDUkY1i9y6pwS7tGI=', 'Content-Length': '0'} 2022-09-03T06:47:40.310 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e01f83cce99176a3-006312f88c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.311 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:40.312 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:40.313 DEBUG:boto:Method: GET 2022-09-03T06:47:40.313 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:47:40.314 DEBUG:boto:Data: 2022-09-03T06:47:40.314 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.315 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.316 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.316 DEBUG:boto:Params: {} 2022-09-03T06:47:40.317 DEBUG:boto:Token: None 2022-09-03T06:47:40.318 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj92 2022-09-03T06:47:40.319 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uphspZArO7SDUkY1i9y6pwS7tGI= 2022-09-03T06:47:40.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uphspZArO7SDUkY1i9y6pwS7tGI=', 'Content-Length': '0'} 2022-09-03T06:47:40.322 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000161562d8d3c3f607-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.377 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:40.379 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:40.380 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:40.381 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:40.382 DEBUG:boto:Method: GET 2022-09-03T06:47:40.382 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:47:40.383 DEBUG:boto:Data: 2022-09-03T06:47:40.383 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.384 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.385 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.385 DEBUG:boto:Params: {} 2022-09-03T06:47:40.386 DEBUG:boto:Token: None 2022-09-03T06:47:40.387 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/ 2022-09-03T06:47:40.388 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24= 2022-09-03T06:47:40.388 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24=', 'Content-Length': '0'} 2022-09-03T06:47:40.392 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009b10360e1e5b764a-006312f88c-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.393 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:40.395 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:40.396 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:40.397 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:40.398 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:40.399 DEBUG:boto:Method: GET 2022-09-03T06:47:40.400 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:47:40.401 DEBUG:boto:Data: 2022-09-03T06:47:40.402 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.403 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.405 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.405 DEBUG:boto:Params: {} 2022-09-03T06:47:40.406 DEBUG:boto:Token: None 2022-09-03T06:47:40.407 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/ 2022-09-03T06:47:40.408 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24= 2022-09-03T06:47:40.409 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24=', 'Content-Length': '0'} 2022-09-03T06:47:40.414 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aa9327d19bec7336-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.415 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:40.416 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2022-09-03T06:47:40.416 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:40.417 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:40.417 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.418 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.419 DEBUG:boto:Method: GET 2022-09-03T06:47:40.420 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.420 DEBUG:boto:Data: 2022-09-03T06:47:40.421 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.421 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.422 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.423 DEBUG:boto:Params: {} 2022-09-03T06:47:40.423 DEBUG:boto:Token: None 2022-09-03T06:47:40.424 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.425 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aapHa5VWMTE2WZU3/jYYZzMyjEE= 2022-09-03T06:47:40.425 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aapHa5VWMTE2WZU3/jYYZzMyjEE=', 'Content-Length': '0'} 2022-09-03T06:47:40.429 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad9aa2f0d0172bed-006312f88c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.430 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:40.431 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:40.432 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.433 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.435 DEBUG:boto:Method: GET 2022-09-03T06:47:40.436 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.437 DEBUG:boto:Data: 2022-09-03T06:47:40.438 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.439 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.439 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.440 DEBUG:boto:Params: {} 2022-09-03T06:47:40.441 DEBUG:boto:Token: None 2022-09-03T06:47:40.442 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.443 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aapHa5VWMTE2WZU3/jYYZzMyjEE= 2022-09-03T06:47:40.444 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aapHa5VWMTE2WZU3/jYYZzMyjEE=', 'Content-Length': '0'} 2022-09-03T06:47:40.446 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000046ce07786562218e-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.447 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:40.447 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:40.448 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.449 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.449 DEBUG:boto:Method: HEAD 2022-09-03T06:47:40.450 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.450 DEBUG:boto:Data: 2022-09-03T06:47:40.451 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.451 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.452 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.453 DEBUG:boto:Params: {} 2022-09-03T06:47:40.453 DEBUG:boto:Token: None 2022-09-03T06:47:40.454 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.455 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8+pqGOKXTuM3XORDOea4Iy6t6eM= 2022-09-03T06:47:40.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8+pqGOKXTuM3XORDOea4Iy6t6eM=', 'Content-Length': '0'} 2022-09-03T06:47:40.459 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000362d70d0e9de4a48-006312f88c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.461 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:40.462 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:40.463 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.464 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.464 DEBUG:boto:Method: HEAD 2022-09-03T06:47:40.465 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.466 DEBUG:boto:Data: 2022-09-03T06:47:40.467 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.468 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.468 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.469 DEBUG:boto:Params: {} 2022-09-03T06:47:40.470 DEBUG:boto:Token: None 2022-09-03T06:47:40.470 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.471 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8+pqGOKXTuM3XORDOea4Iy6t6eM= 2022-09-03T06:47:40.472 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8+pqGOKXTuM3XORDOea4Iy6t6eM=', 'Content-Length': '0'} 2022-09-03T06:47:40.474 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000adc1892c4325ea22-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.475 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2022-09-03T06:47:40.476 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:40.477 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:40.478 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.479 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.480 DEBUG:boto:Method: GET 2022-09-03T06:47:40.481 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.482 DEBUG:boto:Data: 2022-09-03T06:47:40.483 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.484 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.485 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.486 DEBUG:boto:Params: {} 2022-09-03T06:47:40.487 DEBUG:boto:Token: None 2022-09-03T06:47:40.488 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.489 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aapHa5VWMTE2WZU3/jYYZzMyjEE= 2022-09-03T06:47:40.490 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aapHa5VWMTE2WZU3/jYYZzMyjEE=', 'Content-Length': '0'} 2022-09-03T06:47:40.492 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e8d0227fcdf85bc-006312f88c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.494 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:40.495 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:40.496 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.496 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.497 DEBUG:boto:Method: GET 2022-09-03T06:47:40.498 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.499 DEBUG:boto:Data: 2022-09-03T06:47:40.501 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.502 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.502 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.503 DEBUG:boto:Params: {} 2022-09-03T06:47:40.505 DEBUG:boto:Token: None 2022-09-03T06:47:40.506 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:40.506 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aapHa5VWMTE2WZU3/jYYZzMyjEE= 2022-09-03T06:47:40.507 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aapHa5VWMTE2WZU3/jYYZzMyjEE=', 'Content-Length': '0'} 2022-09-03T06:47:40.514 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e84164f699b8f9e-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.554 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:40.555 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:40.556 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:40.557 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:40.558 DEBUG:boto:Method: GET 2022-09-03T06:47:40.559 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:47:40.560 DEBUG:boto:Data: 2022-09-03T06:47:40.561 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.563 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.564 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.565 DEBUG:boto:Params: {} 2022-09-03T06:47:40.566 DEBUG:boto:Token: None 2022-09-03T06:47:40.566 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/ 2022-09-03T06:47:40.567 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24= 2022-09-03T06:47:40.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24=', 'Content-Length': '0'} 2022-09-03T06:47:40.571 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002f39d41ebe43a00d-006312f88c-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.572 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:40.573 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:40.573 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:40.574 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:40.575 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:40.575 DEBUG:boto:Method: GET 2022-09-03T06:47:40.576 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:47:40.576 DEBUG:boto:Data: 2022-09-03T06:47:40.577 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.577 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.578 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.578 DEBUG:boto:Params: {} 2022-09-03T06:47:40.579 DEBUG:boto:Token: None 2022-09-03T06:47:40.580 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/ 2022-09-03T06:47:40.580 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24= 2022-09-03T06:47:40.581 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24=', 'Content-Length': '0'} 2022-09-03T06:47:40.585 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003324780d71fa5428-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.586 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:40.587 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:40.587 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:40.588 DEBUG:boto:Method: HEAD 2022-09-03T06:47:40.589 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:47:40.589 DEBUG:boto:Data: 2022-09-03T06:47:40.590 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.590 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.591 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.591 DEBUG:boto:Params: {} 2022-09-03T06:47:40.592 DEBUG:boto:Token: None 2022-09-03T06:47:40.592 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj93 2022-09-03T06:47:40.593 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Md0SLIstA7vBeYOoh5rML/De7YE= 2022-09-03T06:47:40.593 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Md0SLIstA7vBeYOoh5rML/De7YE=', 'Content-Length': '0'} 2022-09-03T06:47:40.595 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c8928dd5f8f4e015-006312f88c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.596 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:40.597 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:40.597 DEBUG:boto:Method: HEAD 2022-09-03T06:47:40.598 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:47:40.598 DEBUG:boto:Data: 2022-09-03T06:47:40.599 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.600 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.600 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.601 DEBUG:boto:Params: {} 2022-09-03T06:47:40.602 DEBUG:boto:Token: None 2022-09-03T06:47:40.602 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj93 2022-09-03T06:47:40.604 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Md0SLIstA7vBeYOoh5rML/De7YE= 2022-09-03T06:47:40.604 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Md0SLIstA7vBeYOoh5rML/De7YE=', 'Content-Length': '0'} 2022-09-03T06:47:40.607 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a2515ade8d09a5a-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.608 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2022-09-03T06:47:40.609 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:40.609 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:40.610 DEBUG:boto:Method: GET 2022-09-03T06:47:40.610 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:47:40.611 DEBUG:boto:Data: 2022-09-03T06:47:40.612 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.612 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.613 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.613 DEBUG:boto:Params: {} 2022-09-03T06:47:40.614 DEBUG:boto:Token: None 2022-09-03T06:47:40.615 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj93 2022-09-03T06:47:40.615 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EIFaojOz1S1fck9SN5IqV4LofS4= 2022-09-03T06:47:40.616 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EIFaojOz1S1fck9SN5IqV4LofS4=', 'Content-Length': '0'} 2022-09-03T06:47:40.619 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b6d9ef9c60aa285-006312f88c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.662 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:40.663 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:40.664 DEBUG:boto:Method: GET 2022-09-03T06:47:40.664 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:47:40.665 DEBUG:boto:Data: 2022-09-03T06:47:40.665 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.666 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.667 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.667 DEBUG:boto:Params: {} 2022-09-03T06:47:40.668 DEBUG:boto:Token: None 2022-09-03T06:47:40.670 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj93 2022-09-03T06:47:40.670 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EIFaojOz1S1fck9SN5IqV4LofS4= 2022-09-03T06:47:40.671 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EIFaojOz1S1fck9SN5IqV4LofS4=', 'Content-Length': '0'} 2022-09-03T06:47:40.679 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000083e7bc37b64f2162-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.681 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:40.682 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:40.683 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:40.683 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:40.684 DEBUG:boto:Method: GET 2022-09-03T06:47:40.684 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:47:40.685 DEBUG:boto:Data: 2022-09-03T06:47:40.685 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.686 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.686 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.687 DEBUG:boto:Params: {} 2022-09-03T06:47:40.687 DEBUG:boto:Token: None 2022-09-03T06:47:40.688 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/ 2022-09-03T06:47:40.688 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24= 2022-09-03T06:47:40.689 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24=', 'Content-Length': '0'} 2022-09-03T06:47:40.693 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002b95b68635ae04bc-006312f88c-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.693 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:40.695 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:40.695 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:40.696 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:40.697 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:40.697 DEBUG:boto:Method: GET 2022-09-03T06:47:40.698 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:47:40.699 DEBUG:boto:Data: 2022-09-03T06:47:40.700 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.700 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.701 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.702 DEBUG:boto:Params: {} 2022-09-03T06:47:40.704 DEBUG:boto:Token: None 2022-09-03T06:47:40.705 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/ 2022-09-03T06:47:40.706 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24= 2022-09-03T06:47:40.707 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24=', 'Content-Length': '0'} 2022-09-03T06:47:40.711 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000053aa563fd9610aea-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.758 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:40.759 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:47:40.760 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:40.760 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:40.761 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.761 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.762 DEBUG:boto:Method: GET 2022-09-03T06:47:40.763 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.763 DEBUG:boto:Data: 2022-09-03T06:47:40.764 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.766 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.767 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.767 DEBUG:boto:Params: {} 2022-09-03T06:47:40.768 DEBUG:boto:Token: None 2022-09-03T06:47:40.768 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.769 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KMyU3NSCL1FCVOHbV/nToKZ/byE= 2022-09-03T06:47:40.770 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KMyU3NSCL1FCVOHbV/nToKZ/byE=', 'Content-Length': '0'} 2022-09-03T06:47:40.772 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000340700c69f6cc73-006312f88c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.773 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:40.774 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:40.774 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.775 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.775 DEBUG:boto:Method: GET 2022-09-03T06:47:40.776 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.777 DEBUG:boto:Data: 2022-09-03T06:47:40.777 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.778 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.778 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.779 DEBUG:boto:Params: {} 2022-09-03T06:47:40.780 DEBUG:boto:Token: None 2022-09-03T06:47:40.780 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.781 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KMyU3NSCL1FCVOHbV/nToKZ/byE= 2022-09-03T06:47:40.782 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KMyU3NSCL1FCVOHbV/nToKZ/byE=', 'Content-Length': '0'} 2022-09-03T06:47:40.784 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c6ff85104709fd3-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.785 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:40.785 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:40.786 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.786 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.787 DEBUG:boto:Method: HEAD 2022-09-03T06:47:40.787 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.788 DEBUG:boto:Data: 2022-09-03T06:47:40.788 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.789 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.789 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.790 DEBUG:boto:Params: {} 2022-09-03T06:47:40.791 DEBUG:boto:Token: None 2022-09-03T06:47:40.791 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.792 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MhgKWqNvijkhLeFisGlXV2eMrN4= 2022-09-03T06:47:40.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MhgKWqNvijkhLeFisGlXV2eMrN4=', 'Content-Length': '0'} 2022-09-03T06:47:40.794 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004f60b5c26ef869bd-006312f88c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.801 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:40.802 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:40.803 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.803 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.804 DEBUG:boto:Method: HEAD 2022-09-03T06:47:40.804 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.805 DEBUG:boto:Data: 2022-09-03T06:47:40.805 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.806 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.807 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.807 DEBUG:boto:Params: {} 2022-09-03T06:47:40.808 DEBUG:boto:Token: None 2022-09-03T06:47:40.808 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.809 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MhgKWqNvijkhLeFisGlXV2eMrN4= 2022-09-03T06:47:40.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MhgKWqNvijkhLeFisGlXV2eMrN4=', 'Content-Length': '0'} 2022-09-03T06:47:40.812 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a3f0be1694df2b5a-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.814 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:47:40.815 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:40.816 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:40.816 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.817 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.820 DEBUG:boto:Method: GET 2022-09-03T06:47:40.821 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.821 DEBUG:boto:Data: 2022-09-03T06:47:40.822 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.823 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.824 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.825 DEBUG:boto:Params: {} 2022-09-03T06:47:40.826 DEBUG:boto:Token: None 2022-09-03T06:47:40.827 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.828 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KMyU3NSCL1FCVOHbV/nToKZ/byE= 2022-09-03T06:47:40.829 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KMyU3NSCL1FCVOHbV/nToKZ/byE=', 'Content-Length': '0'} 2022-09-03T06:47:40.832 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e4a68b795186f8a5-006312f88c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.833 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:40.834 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:40.834 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.835 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.835 DEBUG:boto:Method: GET 2022-09-03T06:47:40.836 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.836 DEBUG:boto:Data: 2022-09-03T06:47:40.837 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.837 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.838 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.839 DEBUG:boto:Params: {} 2022-09-03T06:47:40.839 DEBUG:boto:Token: None 2022-09-03T06:47:40.840 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:40.840 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KMyU3NSCL1FCVOHbV/nToKZ/byE= 2022-09-03T06:47:40.841 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KMyU3NSCL1FCVOHbV/nToKZ/byE=', 'Content-Length': '0'} 2022-09-03T06:47:40.843 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007f520989c89a2abb-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.844 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:40.845 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:40.845 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:40.846 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:40.847 DEBUG:boto:Method: GET 2022-09-03T06:47:40.847 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:47:40.848 DEBUG:boto:Data: 2022-09-03T06:47:40.848 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.849 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.850 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.850 DEBUG:boto:Params: {} 2022-09-03T06:47:40.851 DEBUG:boto:Token: None 2022-09-03T06:47:40.852 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/ 2022-09-03T06:47:40.852 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24= 2022-09-03T06:47:40.853 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24=', 'Content-Length': '0'} 2022-09-03T06:47:40.856 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e6d364ef65c57aa2-006312f88c-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.857 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:40.858 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:40.859 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:40.859 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:40.860 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:40.860 DEBUG:boto:Method: GET 2022-09-03T06:47:40.861 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:47:40.861 DEBUG:boto:Data: 2022-09-03T06:47:40.862 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.862 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.863 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.863 DEBUG:boto:Params: {} 2022-09-03T06:47:40.864 DEBUG:boto:Token: None 2022-09-03T06:47:40.865 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/ 2022-09-03T06:47:40.865 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24= 2022-09-03T06:47:40.866 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24=', 'Content-Length': '0'} 2022-09-03T06:47:40.870 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e951c9d94bf2252f-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.871 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:40.872 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:40.872 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:40.873 DEBUG:boto:Method: HEAD 2022-09-03T06:47:40.873 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:47:40.874 DEBUG:boto:Data: 2022-09-03T06:47:40.874 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.875 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.875 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.876 DEBUG:boto:Params: {} 2022-09-03T06:47:40.876 DEBUG:boto:Token: None 2022-09-03T06:47:40.877 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj95 2022-09-03T06:47:40.877 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SQfNLKId6ar5ICl1xNl4uoQ/BnE= 2022-09-03T06:47:40.878 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SQfNLKId6ar5ICl1xNl4uoQ/BnE=', 'Content-Length': '0'} 2022-09-03T06:47:40.880 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d325abe7cfded222-006312f88c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.881 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:40.881 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:40.882 DEBUG:boto:Method: HEAD 2022-09-03T06:47:40.882 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:47:40.883 DEBUG:boto:Data: 2022-09-03T06:47:40.883 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.884 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.884 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.885 DEBUG:boto:Params: {} 2022-09-03T06:47:40.885 DEBUG:boto:Token: None 2022-09-03T06:47:40.886 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj95 2022-09-03T06:47:40.886 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SQfNLKId6ar5ICl1xNl4uoQ/BnE= 2022-09-03T06:47:40.887 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SQfNLKId6ar5ICl1xNl4uoQ/BnE=', 'Content-Length': '0'} 2022-09-03T06:47:40.889 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000058824968c13b54bc-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.890 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:47:40.891 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:40.891 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:40.892 DEBUG:boto:Method: GET 2022-09-03T06:47:40.892 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:47:40.893 DEBUG:boto:Data: 2022-09-03T06:47:40.893 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.894 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.894 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.895 DEBUG:boto:Params: {} 2022-09-03T06:47:40.895 DEBUG:boto:Token: None 2022-09-03T06:47:40.896 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj95 2022-09-03T06:47:40.897 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bkSVtDSSd8oIY8ra6mOF430Ftkc= 2022-09-03T06:47:40.897 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bkSVtDSSd8oIY8ra6mOF430Ftkc=', 'Content-Length': '0'} 2022-09-03T06:47:40.899 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000be4d3d8e88d6c8ba-006312f88c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.900 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:40.901 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:40.901 DEBUG:boto:Method: GET 2022-09-03T06:47:40.902 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:47:40.902 DEBUG:boto:Data: 2022-09-03T06:47:40.903 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.903 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.904 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.904 DEBUG:boto:Params: {} 2022-09-03T06:47:40.905 DEBUG:boto:Token: None 2022-09-03T06:47:40.906 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/obj95 2022-09-03T06:47:40.906 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bkSVtDSSd8oIY8ra6mOF430Ftkc= 2022-09-03T06:47:40.907 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bkSVtDSSd8oIY8ra6mOF430Ftkc=', 'Content-Length': '0'} 2022-09-03T06:47:40.909 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c75ee443303fa15b-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.910 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:40.910 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:40.911 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:40.911 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:40.912 DEBUG:boto:Method: GET 2022-09-03T06:47:40.912 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:47:40.913 DEBUG:boto:Data: 2022-09-03T06:47:40.913 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.914 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.914 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.915 DEBUG:boto:Params: {} 2022-09-03T06:47:40.915 DEBUG:boto:Token: None 2022-09-03T06:47:40.916 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/ 2022-09-03T06:47:40.917 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24= 2022-09-03T06:47:40.917 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24=', 'Content-Length': '0'} 2022-09-03T06:47:40.920 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000614e07cf0e8edae3-006312f88c-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.921 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:40.922 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:40.922 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:40.923 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:40.923 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:40.924 DEBUG:boto:Method: GET 2022-09-03T06:47:40.925 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:47:40.925 DEBUG:boto:Data: 2022-09-03T06:47:40.926 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.926 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:40.927 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.927 DEBUG:boto:Params: {} 2022-09-03T06:47:40.928 DEBUG:boto:Token: None 2022-09-03T06:47:40.928 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/ 2022-09-03T06:47:40.929 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24= 2022-09-03T06:47:40.930 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sXLgpogPkXV4jnSDuBetd8CSM24=', 'Content-Length': '0'} 2022-09-03T06:47:40.934 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000073239be6011380d4-006312f88c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.934 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:40.935 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-75 zones={test-zone3, test-zone1} 2022-09-03T06:47:40.936 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-75 zones={test-zone3, test-zone2} 2022-09-03T06:47:40.936 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:40.937 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:40.937 DEBUG:boto:Method: HEAD 2022-09-03T06:47:40.938 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:47:40.939 DEBUG:boto:Data: 2022-09-03T06:47:40.939 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.940 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.940 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.941 DEBUG:boto:Params: {} 2022-09-03T06:47:40.941 DEBUG:boto:Token: None 2022-09-03T06:47:40.942 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/ 2022-09-03T06:47:40.942 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c3K8OXLk/UrieAhpSvDgiCXglxU= 2022-09-03T06:47:40.943 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c3K8OXLk/UrieAhpSvDgiCXglxU=', 'Content-Length': '0'} 2022-09-03T06:47:40.946 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '30'), ('X-RGW-Bytes-Used', '90'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003d75b72180bdd685-006312f88c-1851-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.947 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:40.947 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:40.948 DEBUG:boto:Method: HEAD 2022-09-03T06:47:40.948 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:47:40.949 DEBUG:boto:Data: 2022-09-03T06:47:40.949 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.950 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:40.950 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.951 DEBUG:boto:Params: {} 2022-09-03T06:47:40.951 DEBUG:boto:Token: None 2022-09-03T06:47:40.952 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/ 2022-09-03T06:47:40.952 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c3K8OXLk/UrieAhpSvDgiCXglxU= 2022-09-03T06:47:40.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c3K8OXLk/UrieAhpSvDgiCXglxU=', 'Content-Length': '0'} 2022-09-03T06:47:40.956 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '30'), ('X-RGW-Bytes-Used', '90'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000091da04b439701ad3-006312f88c-188c-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.956 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:47:40.957 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:40.957 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:40.958 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:40.958 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:40.959 DEBUG:boto:Method: GET 2022-09-03T06:47:40.960 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:40.960 DEBUG:boto:Data: 2022-09-03T06:47:40.961 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.961 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:40.962 DEBUG:boto:Port: 8001 2022-09-03T06:47:40.962 DEBUG:boto:Params: {} 2022-09-03T06:47:40.963 DEBUG:boto:Token: None 2022-09-03T06:47:40.963 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/?versions 2022-09-03T06:47:40.964 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HHBQ5gsUGdvZxuZMheAzz28usoo= 2022-09-03T06:47:40.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HHBQ5gsUGdvZxuZMheAzz28usoo=', 'Content-Length': '0'} 2022-09-03T06:47:40.968 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000042fec6d53d65c3e2-006312f88c-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:40.968 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:40.971 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:47:40.971 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:47:40.972 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2022-09-03T06:47:40.973 DEBUG:tasks.rgw_multi.zone_rados:o=obj124 2022-09-03T06:47:40.973 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2022-09-03T06:47:40.974 DEBUG:tasks.rgw_multi.zone_rados:o=obj155 2022-09-03T06:47:40.974 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:47:40.975 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:47:40.975 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:47:40.976 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:47:40.976 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2022-09-03T06:47:40.977 DEBUG:tasks.rgw_multi.zone_rados:o=obj31 2022-09-03T06:47:40.977 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:47:40.978 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:47:40.978 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:47:40.979 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:47:40.979 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:47:40.980 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:47:40.981 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:47:40.981 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:47:40.982 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2022-09-03T06:47:40.983 DEBUG:tasks.rgw_multi.zone_rados:o=obj62 2022-09-03T06:47:40.984 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:47:40.984 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:47:40.985 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:47:40.986 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:47:40.987 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2022-09-03T06:47:40.987 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:47:40.988 DEBUG:tasks.rgw_multi.zone_rados:o=obj93 2022-09-03T06:47:40.988 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:47:40.989 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:47:40.989 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:40.990 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:40.991 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:40.991 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:40.992 DEBUG:boto:Method: GET 2022-09-03T06:47:40.992 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:40.993 DEBUG:boto:Data: 2022-09-03T06:47:40.993 DEBUG:boto:Headers: {} 2022-09-03T06:47:40.994 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:40.994 DEBUG:boto:Port: 8000 2022-09-03T06:47:40.995 DEBUG:boto:Params: {} 2022-09-03T06:47:40.995 DEBUG:boto:Token: None 2022-09-03T06:47:40.996 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:40 GMT /dsgvcy-75/?versions 2022-09-03T06:47:40.997 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HHBQ5gsUGdvZxuZMheAzz28usoo= 2022-09-03T06:47:40.997 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:40 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HHBQ5gsUGdvZxuZMheAzz28usoo=', 'Content-Length': '0'} 2022-09-03T06:47:41.000 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008772327996d91a18-006312f88c-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:40 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.001 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:41.003 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:47:41.004 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:47:41.004 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2022-09-03T06:47:41.005 DEBUG:tasks.rgw_multi.zone_rados:o=obj124 2022-09-03T06:47:41.005 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2022-09-03T06:47:41.005 DEBUG:tasks.rgw_multi.zone_rados:o=obj155 2022-09-03T06:47:41.006 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:47:41.006 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:47:41.007 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:47:41.007 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:47:41.008 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2022-09-03T06:47:41.008 DEBUG:tasks.rgw_multi.zone_rados:o=obj31 2022-09-03T06:47:41.009 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:47:41.009 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:47:41.010 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:47:41.010 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:47:41.011 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:47:41.011 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:47:41.012 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:47:41.012 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:47:41.013 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2022-09-03T06:47:41.013 DEBUG:tasks.rgw_multi.zone_rados:o=obj62 2022-09-03T06:47:41.013 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:47:41.014 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:47:41.014 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:47:41.015 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:47:41.015 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2022-09-03T06:47:41.016 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:47:41.016 DEBUG:tasks.rgw_multi.zone_rados:o=obj93 2022-09-03T06:47:41.017 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:47:41.017 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:41.018 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:41.018 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:41.019 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:41.020 DEBUG:boto:Method: GET 2022-09-03T06:47:41.020 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:41.021 DEBUG:boto:Data: 2022-09-03T06:47:41.021 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.022 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.022 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.023 DEBUG:boto:Params: {} 2022-09-03T06:47:41.024 DEBUG:boto:Token: None 2022-09-03T06:47:41.024 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/?versions 2022-09-03T06:47:41.025 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bp6Rpn40Y8+4R/D95KoC2vvJU0w= 2022-09-03T06:47:41.025 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bp6Rpn40Y8+4R/D95KoC2vvJU0w=', 'Content-Length': '0'} 2022-09-03T06:47:41.029 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d9f3b589365b464c-006312f88d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.029 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:41.032 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:41.032 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:41.033 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:41.033 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:41.034 DEBUG:boto:Method: GET 2022-09-03T06:47:41.035 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:47:41.035 DEBUG:boto:Data: 2022-09-03T06:47:41.036 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.036 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.037 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.037 DEBUG:boto:Params: {} 2022-09-03T06:47:41.038 DEBUG:boto:Token: None 2022-09-03T06:47:41.039 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/?versions 2022-09-03T06:47:41.039 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Bp6Rpn40Y8+4R/D95KoC2vvJU0w= 2022-09-03T06:47:41.040 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Bp6Rpn40Y8+4R/D95KoC2vvJU0w=', 'Content-Length': '0'} 2022-09-03T06:47:41.043 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000362c4710512e0c9-006312f88d-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.044 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:41.046 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:47:41.046 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:41.047 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:41.047 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.048 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.049 DEBUG:boto:Method: GET 2022-09-03T06:47:41.049 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.050 DEBUG:boto:Data: 2022-09-03T06:47:41.050 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.051 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.051 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.052 DEBUG:boto:Params: {} 2022-09-03T06:47:41.053 DEBUG:boto:Token: None 2022-09-03T06:47:41.053 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.054 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GQEnSYTLc7CuqhNFXNMkGlLKVds= 2022-09-03T06:47:41.055 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GQEnSYTLc7CuqhNFXNMkGlLKVds=', 'Content-Length': '0'} 2022-09-03T06:47:41.060 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e8d70545d5ddd78e-006312f88d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.061 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:41.061 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:41.062 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.063 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.063 DEBUG:boto:Method: GET 2022-09-03T06:47:41.064 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.064 DEBUG:boto:Data: 2022-09-03T06:47:41.065 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.065 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.066 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.066 DEBUG:boto:Params: {} 2022-09-03T06:47:41.067 DEBUG:boto:Token: None 2022-09-03T06:47:41.067 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.068 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GQEnSYTLc7CuqhNFXNMkGlLKVds= 2022-09-03T06:47:41.069 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GQEnSYTLc7CuqhNFXNMkGlLKVds=', 'Content-Length': '0'} 2022-09-03T06:47:41.071 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000299a9f8e2658fa5-006312f88d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.072 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:41.073 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:41.073 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.074 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.074 DEBUG:boto:Method: HEAD 2022-09-03T06:47:41.075 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.075 DEBUG:boto:Data: 2022-09-03T06:47:41.076 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.076 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.077 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.078 DEBUG:boto:Params: {} 2022-09-03T06:47:41.079 DEBUG:boto:Token: None 2022-09-03T06:47:41.079 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.080 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WY8Kpyb4Fh2qZwId2tkE2UspmDY= 2022-09-03T06:47:41.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WY8Kpyb4Fh2qZwId2tkE2UspmDY=', 'Content-Length': '0'} 2022-09-03T06:47:41.083 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f1a2979c0c987115-006312f88d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.084 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:41.084 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:41.085 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.085 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.086 DEBUG:boto:Method: HEAD 2022-09-03T06:47:41.087 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.088 DEBUG:boto:Data: 2022-09-03T06:47:41.089 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.089 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.090 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.091 DEBUG:boto:Params: {} 2022-09-03T06:47:41.092 DEBUG:boto:Token: None 2022-09-03T06:47:41.093 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.094 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WY8Kpyb4Fh2qZwId2tkE2UspmDY= 2022-09-03T06:47:41.095 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WY8Kpyb4Fh2qZwId2tkE2UspmDY=', 'Content-Length': '0'} 2022-09-03T06:47:41.098 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008d6a1fcdaef5b67f-006312f88d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.099 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:47:41.100 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:41.101 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:41.102 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.103 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.104 DEBUG:boto:Method: GET 2022-09-03T06:47:41.105 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.106 DEBUG:boto:Data: 2022-09-03T06:47:41.107 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.108 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.109 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.110 DEBUG:boto:Params: {} 2022-09-03T06:47:41.111 DEBUG:boto:Token: None 2022-09-03T06:47:41.112 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.114 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GQEnSYTLc7CuqhNFXNMkGlLKVds= 2022-09-03T06:47:41.115 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GQEnSYTLc7CuqhNFXNMkGlLKVds=', 'Content-Length': '0'} 2022-09-03T06:47:41.117 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009cc1d5c2806cbe0e-006312f88d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.162 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:41.164 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:41.165 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.166 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.167 DEBUG:boto:Method: GET 2022-09-03T06:47:41.168 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.169 DEBUG:boto:Data: 2022-09-03T06:47:41.170 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.171 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.171 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.172 DEBUG:boto:Params: {} 2022-09-03T06:47:41.173 DEBUG:boto:Token: None 2022-09-03T06:47:41.174 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:47:41.175 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GQEnSYTLc7CuqhNFXNMkGlLKVds= 2022-09-03T06:47:41.175 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GQEnSYTLc7CuqhNFXNMkGlLKVds=', 'Content-Length': '0'} 2022-09-03T06:47:41.178 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000daeadc16f686d4b0-006312f88d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.179 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:41.179 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:41.180 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:41.181 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:41.181 DEBUG:boto:Method: GET 2022-09-03T06:47:41.182 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:47:41.182 DEBUG:boto:Data: 2022-09-03T06:47:41.183 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.184 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.184 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.185 DEBUG:boto:Params: {} 2022-09-03T06:47:41.186 DEBUG:boto:Token: None 2022-09-03T06:47:41.186 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/ 2022-09-03T06:47:41.187 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA= 2022-09-03T06:47:41.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA=', 'Content-Length': '0'} 2022-09-03T06:47:41.191 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006c28e64e31ae33bb-006312f88d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.192 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:41.193 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:41.194 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:41.194 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:41.195 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:41.195 DEBUG:boto:Method: GET 2022-09-03T06:47:41.196 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:47:41.196 DEBUG:boto:Data: 2022-09-03T06:47:41.197 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.197 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.198 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.198 DEBUG:boto:Params: {} 2022-09-03T06:47:41.199 DEBUG:boto:Token: None 2022-09-03T06:47:41.200 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/ 2022-09-03T06:47:41.200 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA= 2022-09-03T06:47:41.201 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA=', 'Content-Length': '0'} 2022-09-03T06:47:41.206 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000095fcfd89d0e14d48-006312f88d-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.206 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:41.207 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:41.208 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:41.208 DEBUG:boto:Method: HEAD 2022-09-03T06:47:41.209 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:47:41.209 DEBUG:boto:Data: 2022-09-03T06:47:41.210 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.210 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.211 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.211 DEBUG:boto:Params: {} 2022-09-03T06:47:41.212 DEBUG:boto:Token: None 2022-09-03T06:47:41.213 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj11 2022-09-03T06:47:41.213 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AH0npWbCrFS063/dasoKC3B7RJk= 2022-09-03T06:47:41.214 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AH0npWbCrFS063/dasoKC3B7RJk=', 'Content-Length': '0'} 2022-09-03T06:47:41.216 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000024163d944ff13117-006312f88d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.217 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:41.217 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:41.219 DEBUG:boto:Method: HEAD 2022-09-03T06:47:41.220 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:47:41.221 DEBUG:boto:Data: 2022-09-03T06:47:41.221 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.222 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.223 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.224 DEBUG:boto:Params: {} 2022-09-03T06:47:41.225 DEBUG:boto:Token: None 2022-09-03T06:47:41.226 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj11 2022-09-03T06:47:41.226 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AH0npWbCrFS063/dasoKC3B7RJk= 2022-09-03T06:47:41.227 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AH0npWbCrFS063/dasoKC3B7RJk=', 'Content-Length': '0'} 2022-09-03T06:47:41.229 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ce46dbe363045e56-006312f88d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.230 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:47:41.231 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:41.231 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:41.232 DEBUG:boto:Method: GET 2022-09-03T06:47:41.232 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:47:41.233 DEBUG:boto:Data: 2022-09-03T06:47:41.234 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.234 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.235 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.236 DEBUG:boto:Params: {} 2022-09-03T06:47:41.237 DEBUG:boto:Token: None 2022-09-03T06:47:41.239 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj11 2022-09-03T06:47:41.240 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wshw7acSNlWEN0B2QvOxWP7aK68= 2022-09-03T06:47:41.241 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wshw7acSNlWEN0B2QvOxWP7aK68=', 'Content-Length': '0'} 2022-09-03T06:47:41.244 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000095df6255a1e6927c-006312f88d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.246 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:47:41.246 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:47:41.247 DEBUG:boto:Method: GET 2022-09-03T06:47:41.248 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:47:41.248 DEBUG:boto:Data: 2022-09-03T06:47:41.249 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.249 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.250 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.250 DEBUG:boto:Params: {} 2022-09-03T06:47:41.251 DEBUG:boto:Token: None 2022-09-03T06:47:41.252 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj11 2022-09-03T06:47:41.252 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wshw7acSNlWEN0B2QvOxWP7aK68= 2022-09-03T06:47:41.253 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wshw7acSNlWEN0B2QvOxWP7aK68=', 'Content-Length': '0'} 2022-09-03T06:47:41.255 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000b711684ad369d2f-006312f88d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.258 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:41.259 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:41.260 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:41.261 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:41.262 DEBUG:boto:Method: GET 2022-09-03T06:47:41.263 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:47:41.264 DEBUG:boto:Data: 2022-09-03T06:47:41.265 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.266 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.268 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.269 DEBUG:boto:Params: {} 2022-09-03T06:47:41.270 DEBUG:boto:Token: None 2022-09-03T06:47:41.271 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/ 2022-09-03T06:47:41.273 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA= 2022-09-03T06:47:41.274 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA=', 'Content-Length': '0'} 2022-09-03T06:47:41.278 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006fae90c79c8daaf7-006312f88d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.279 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:41.280 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:41.280 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:41.281 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:41.282 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:47:41.282 DEBUG:boto:Method: GET 2022-09-03T06:47:41.283 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:47:41.284 DEBUG:boto:Data: 2022-09-03T06:47:41.284 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.285 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.285 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.286 DEBUG:boto:Params: {} 2022-09-03T06:47:41.287 DEBUG:boto:Token: None 2022-09-03T06:47:41.288 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/ 2022-09-03T06:47:41.288 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA= 2022-09-03T06:47:41.289 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA=', 'Content-Length': '0'} 2022-09-03T06:47:41.292 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a8a4423a4d618cef-006312f88d-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.293 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:41.294 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:47:41.295 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:41.296 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:41.296 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.297 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.297 DEBUG:boto:Method: GET 2022-09-03T06:47:41.298 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.298 DEBUG:boto:Data: 2022-09-03T06:47:41.299 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.300 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.300 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.301 DEBUG:boto:Params: {} 2022-09-03T06:47:41.302 DEBUG:boto:Token: None 2022-09-03T06:47:41.302 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.303 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:veRE37sV+IxQ+ourCnjgUpnaV5k= 2022-09-03T06:47:41.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:veRE37sV+IxQ+ourCnjgUpnaV5k=', 'Content-Length': '0'} 2022-09-03T06:47:41.306 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006eb11a371f9d72ae-006312f88d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.307 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:41.308 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:41.309 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.309 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.310 DEBUG:boto:Method: GET 2022-09-03T06:47:41.310 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.311 DEBUG:boto:Data: 2022-09-03T06:47:41.312 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.312 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.313 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.313 DEBUG:boto:Params: {} 2022-09-03T06:47:41.314 DEBUG:boto:Token: None 2022-09-03T06:47:41.315 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.315 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:veRE37sV+IxQ+ourCnjgUpnaV5k= 2022-09-03T06:47:41.316 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:veRE37sV+IxQ+ourCnjgUpnaV5k=', 'Content-Length': '0'} 2022-09-03T06:47:41.318 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000729019fadaf4b14f-006312f88d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.319 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:41.319 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:41.320 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.321 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.321 DEBUG:boto:Method: HEAD 2022-09-03T06:47:41.322 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.322 DEBUG:boto:Data: 2022-09-03T06:47:41.323 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.324 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.324 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.325 DEBUG:boto:Params: {} 2022-09-03T06:47:41.325 DEBUG:boto:Token: None 2022-09-03T06:47:41.326 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.327 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9ZWKcRN2+tdEX+7N4L0UlfM/nnU= 2022-09-03T06:47:41.327 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9ZWKcRN2+tdEX+7N4L0UlfM/nnU=', 'Content-Length': '0'} 2022-09-03T06:47:41.329 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e1c555033da41e6-006312f88d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.330 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:41.331 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:41.331 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.332 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.333 DEBUG:boto:Method: HEAD 2022-09-03T06:47:41.333 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.334 DEBUG:boto:Data: 2022-09-03T06:47:41.335 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.335 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.336 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.336 DEBUG:boto:Params: {} 2022-09-03T06:47:41.337 DEBUG:boto:Token: None 2022-09-03T06:47:41.338 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.339 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9ZWKcRN2+tdEX+7N4L0UlfM/nnU= 2022-09-03T06:47:41.339 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9ZWKcRN2+tdEX+7N4L0UlfM/nnU=', 'Content-Length': '0'} 2022-09-03T06:47:41.343 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000041bbe84ba8311258-006312f88d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.344 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:47:41.345 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:41.346 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:41.346 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.347 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.347 DEBUG:boto:Method: GET 2022-09-03T06:47:41.348 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.349 DEBUG:boto:Data: 2022-09-03T06:47:41.349 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.350 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.350 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.351 DEBUG:boto:Params: {} 2022-09-03T06:47:41.352 DEBUG:boto:Token: None 2022-09-03T06:47:41.352 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.353 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:veRE37sV+IxQ+ourCnjgUpnaV5k= 2022-09-03T06:47:41.353 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:veRE37sV+IxQ+ourCnjgUpnaV5k=', 'Content-Length': '0'} 2022-09-03T06:47:41.356 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b5d118eb8e347e42-006312f88d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.357 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:41.357 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:41.358 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.359 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.359 DEBUG:boto:Method: GET 2022-09-03T06:47:41.360 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.360 DEBUG:boto:Data: 2022-09-03T06:47:41.361 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.362 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.363 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.363 DEBUG:boto:Params: {} 2022-09-03T06:47:41.364 DEBUG:boto:Token: None 2022-09-03T06:47:41.365 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:47:41.365 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:veRE37sV+IxQ+ourCnjgUpnaV5k= 2022-09-03T06:47:41.366 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:veRE37sV+IxQ+ourCnjgUpnaV5k=', 'Content-Length': '0'} 2022-09-03T06:47:41.368 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000023ba7a764092700d-006312f88d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.369 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:41.370 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:41.370 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:41.371 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:41.372 DEBUG:boto:Method: GET 2022-09-03T06:47:41.372 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:47:41.373 DEBUG:boto:Data: 2022-09-03T06:47:41.373 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.374 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.375 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.376 DEBUG:boto:Params: {} 2022-09-03T06:47:41.376 DEBUG:boto:Token: None 2022-09-03T06:47:41.377 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/ 2022-09-03T06:47:41.377 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA= 2022-09-03T06:47:41.378 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA=', 'Content-Length': '0'} 2022-09-03T06:47:41.381 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000037b8de85354f5f8b-006312f88d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.381 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:41.382 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:41.383 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:41.383 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:41.384 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:41.384 DEBUG:boto:Method: GET 2022-09-03T06:47:41.385 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:47:41.385 DEBUG:boto:Data: 2022-09-03T06:47:41.386 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.387 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.387 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.388 DEBUG:boto:Params: {} 2022-09-03T06:47:41.388 DEBUG:boto:Token: None 2022-09-03T06:47:41.389 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/ 2022-09-03T06:47:41.390 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA= 2022-09-03T06:47:41.390 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA=', 'Content-Length': '0'} 2022-09-03T06:47:41.394 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000042fab506271bb108-006312f88d-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.396 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:41.407 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:41.409 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:41.410 DEBUG:boto:Method: HEAD 2022-09-03T06:47:41.410 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:47:41.411 DEBUG:boto:Data: 2022-09-03T06:47:41.412 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.412 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.413 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.413 DEBUG:boto:Params: {} 2022-09-03T06:47:41.414 DEBUG:boto:Token: None 2022-09-03T06:47:41.414 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj115 2022-09-03T06:47:41.415 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tC2bybAC/jx+FQnCApB1qMSUNFc= 2022-09-03T06:47:41.416 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tC2bybAC/jx+FQnCApB1qMSUNFc=', 'Content-Length': '0'} 2022-09-03T06:47:41.418 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000095612288e2c27b5e-006312f88d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.420 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:41.422 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:41.423 DEBUG:boto:Method: HEAD 2022-09-03T06:47:41.423 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:47:41.424 DEBUG:boto:Data: 2022-09-03T06:47:41.426 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.426 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.427 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.428 DEBUG:boto:Params: {} 2022-09-03T06:47:41.428 DEBUG:boto:Token: None 2022-09-03T06:47:41.429 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj115 2022-09-03T06:47:41.429 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tC2bybAC/jx+FQnCApB1qMSUNFc= 2022-09-03T06:47:41.430 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tC2bybAC/jx+FQnCApB1qMSUNFc=', 'Content-Length': '0'} 2022-09-03T06:47:41.432 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000087393c7ee0828e4f-006312f88d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.433 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:47:41.434 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:41.435 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:41.436 DEBUG:boto:Method: GET 2022-09-03T06:47:41.436 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:47:41.437 DEBUG:boto:Data: 2022-09-03T06:47:41.437 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.438 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.438 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.439 DEBUG:boto:Params: {} 2022-09-03T06:47:41.440 DEBUG:boto:Token: None 2022-09-03T06:47:41.440 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj115 2022-09-03T06:47:41.441 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N5pVAN6zCcjho7pk4rpu7ujhons= 2022-09-03T06:47:41.441 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N5pVAN6zCcjho7pk4rpu7ujhons=', 'Content-Length': '0'} 2022-09-03T06:47:41.444 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007cf3e49fec7fbea9-006312f88d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.446 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:47:41.447 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:47:41.448 DEBUG:boto:Method: GET 2022-09-03T06:47:41.449 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:47:41.451 DEBUG:boto:Data: 2022-09-03T06:47:41.452 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.453 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.454 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.454 DEBUG:boto:Params: {} 2022-09-03T06:47:41.455 DEBUG:boto:Token: None 2022-09-03T06:47:41.456 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj115 2022-09-03T06:47:41.456 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N5pVAN6zCcjho7pk4rpu7ujhons= 2022-09-03T06:47:41.457 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N5pVAN6zCcjho7pk4rpu7ujhons=', 'Content-Length': '0'} 2022-09-03T06:47:41.459 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7cc3ae6c330578e-006312f88d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.502 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:41.503 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:41.504 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:41.504 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:41.505 DEBUG:boto:Method: GET 2022-09-03T06:47:41.506 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:47:41.506 DEBUG:boto:Data: 2022-09-03T06:47:41.507 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.508 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.508 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.510 DEBUG:boto:Params: {} 2022-09-03T06:47:41.511 DEBUG:boto:Token: None 2022-09-03T06:47:41.511 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/ 2022-09-03T06:47:41.512 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA= 2022-09-03T06:47:41.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA=', 'Content-Length': '0'} 2022-09-03T06:47:41.515 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001bb0e7c807beb5dd-006312f88d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.516 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:41.517 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:41.518 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:41.518 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:41.519 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:47:41.519 DEBUG:boto:Method: GET 2022-09-03T06:47:41.520 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:47:41.521 DEBUG:boto:Data: 2022-09-03T06:47:41.522 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.523 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.524 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.525 DEBUG:boto:Params: {} 2022-09-03T06:47:41.526 DEBUG:boto:Token: None 2022-09-03T06:47:41.526 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/ 2022-09-03T06:47:41.527 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA= 2022-09-03T06:47:41.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA=', 'Content-Length': '0'} 2022-09-03T06:47:41.537 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000397baa5d65edd5c9-006312f88d-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.538 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:41.539 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:47:41.540 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:41.541 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:41.541 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.542 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.543 DEBUG:boto:Method: GET 2022-09-03T06:47:41.544 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.545 DEBUG:boto:Data: 2022-09-03T06:47:41.545 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.546 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.547 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.548 DEBUG:boto:Params: {} 2022-09-03T06:47:41.548 DEBUG:boto:Token: None 2022-09-03T06:47:41.549 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.550 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8o5dOl06U+YoIgcCI9C7TM7ysKA= 2022-09-03T06:47:41.551 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8o5dOl06U+YoIgcCI9C7TM7ysKA=', 'Content-Length': '0'} 2022-09-03T06:47:41.553 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a15f5c3771c37f9c-006312f88d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.599 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:41.601 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:41.602 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.603 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.603 DEBUG:boto:Method: GET 2022-09-03T06:47:41.604 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.604 DEBUG:boto:Data: 2022-09-03T06:47:41.605 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.605 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.606 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.607 DEBUG:boto:Params: {} 2022-09-03T06:47:41.607 DEBUG:boto:Token: None 2022-09-03T06:47:41.608 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.609 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8o5dOl06U+YoIgcCI9C7TM7ysKA= 2022-09-03T06:47:41.610 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8o5dOl06U+YoIgcCI9C7TM7ysKA=', 'Content-Length': '0'} 2022-09-03T06:47:41.614 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a2457906f5060303-006312f88d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.615 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:41.616 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:41.617 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.618 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.618 DEBUG:boto:Method: HEAD 2022-09-03T06:47:41.619 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.619 DEBUG:boto:Data: 2022-09-03T06:47:41.620 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.620 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.621 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.621 DEBUG:boto:Params: {} 2022-09-03T06:47:41.622 DEBUG:boto:Token: None 2022-09-03T06:47:41.623 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.623 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xnsplzVcgnHE94WKSTE7GM2CX/Y= 2022-09-03T06:47:41.624 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xnsplzVcgnHE94WKSTE7GM2CX/Y=', 'Content-Length': '0'} 2022-09-03T06:47:41.626 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000113f01afba139e91-006312f88d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.627 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:41.628 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:41.629 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.629 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.630 DEBUG:boto:Method: HEAD 2022-09-03T06:47:41.630 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.631 DEBUG:boto:Data: 2022-09-03T06:47:41.631 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.632 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.632 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.634 DEBUG:boto:Params: {} 2022-09-03T06:47:41.634 DEBUG:boto:Token: None 2022-09-03T06:47:41.635 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.636 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xnsplzVcgnHE94WKSTE7GM2CX/Y= 2022-09-03T06:47:41.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xnsplzVcgnHE94WKSTE7GM2CX/Y=', 'Content-Length': '0'} 2022-09-03T06:47:41.639 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bb7d7308e423a91b-006312f88d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.640 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:47:41.641 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:41.641 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:41.642 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.642 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.644 DEBUG:boto:Method: GET 2022-09-03T06:47:41.645 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.645 DEBUG:boto:Data: 2022-09-03T06:47:41.646 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.646 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.648 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.648 DEBUG:boto:Params: {} 2022-09-03T06:47:41.649 DEBUG:boto:Token: None 2022-09-03T06:47:41.649 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.650 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8o5dOl06U+YoIgcCI9C7TM7ysKA= 2022-09-03T06:47:41.651 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8o5dOl06U+YoIgcCI9C7TM7ysKA=', 'Content-Length': '0'} 2022-09-03T06:47:41.653 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000066b56f3d886051df-006312f88d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.654 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:41.655 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:41.655 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.656 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.657 DEBUG:boto:Method: GET 2022-09-03T06:47:41.657 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.658 DEBUG:boto:Data: 2022-09-03T06:47:41.659 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.659 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.660 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.660 DEBUG:boto:Params: {} 2022-09-03T06:47:41.661 DEBUG:boto:Token: None 2022-09-03T06:47:41.662 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:47:41.662 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8o5dOl06U+YoIgcCI9C7TM7ysKA= 2022-09-03T06:47:41.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8o5dOl06U+YoIgcCI9C7TM7ysKA=', 'Content-Length': '0'} 2022-09-03T06:47:41.665 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000387f3099eaa397c3-006312f88d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.666 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:41.667 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:41.668 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:41.668 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:41.669 DEBUG:boto:Method: GET 2022-09-03T06:47:41.670 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:47:41.670 DEBUG:boto:Data: 2022-09-03T06:47:41.671 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.672 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.673 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.674 DEBUG:boto:Params: {} 2022-09-03T06:47:41.675 DEBUG:boto:Token: None 2022-09-03T06:47:41.676 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/ 2022-09-03T06:47:41.677 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA= 2022-09-03T06:47:41.678 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA=', 'Content-Length': '0'} 2022-09-03T06:47:41.681 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d927699d70738f42-006312f88d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.682 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:41.683 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:41.684 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:41.684 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:41.685 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:41.686 DEBUG:boto:Method: GET 2022-09-03T06:47:41.686 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:47:41.687 DEBUG:boto:Data: 2022-09-03T06:47:41.688 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.688 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.689 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.690 DEBUG:boto:Params: {} 2022-09-03T06:47:41.691 DEBUG:boto:Token: None 2022-09-03T06:47:41.692 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/ 2022-09-03T06:47:41.693 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA= 2022-09-03T06:47:41.693 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA=', 'Content-Length': '0'} 2022-09-03T06:47:41.696 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bbcda1be83380a64-006312f88d-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.697 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:41.699 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:41.699 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:41.700 DEBUG:boto:Method: HEAD 2022-09-03T06:47:41.700 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:47:41.701 DEBUG:boto:Data: 2022-09-03T06:47:41.701 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.702 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.702 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.703 DEBUG:boto:Params: {} 2022-09-03T06:47:41.703 DEBUG:boto:Token: None 2022-09-03T06:47:41.704 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj116 2022-09-03T06:47:41.704 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pJp81N6wRZuevbP2mZRJwJN1Au0= 2022-09-03T06:47:41.705 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pJp81N6wRZuevbP2mZRJwJN1Au0=', 'Content-Length': '0'} 2022-09-03T06:47:41.708 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b40268d7c45b9c45-006312f88d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.708 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:41.709 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:41.709 DEBUG:boto:Method: HEAD 2022-09-03T06:47:41.710 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:47:41.711 DEBUG:boto:Data: 2022-09-03T06:47:41.711 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.712 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.712 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.713 DEBUG:boto:Params: {} 2022-09-03T06:47:41.713 DEBUG:boto:Token: None 2022-09-03T06:47:41.714 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj116 2022-09-03T06:47:41.715 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pJp81N6wRZuevbP2mZRJwJN1Au0= 2022-09-03T06:47:41.715 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pJp81N6wRZuevbP2mZRJwJN1Au0=', 'Content-Length': '0'} 2022-09-03T06:47:41.718 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000283ff30bdbdbd067-006312f88d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.719 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:47:41.720 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:41.720 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:41.721 DEBUG:boto:Method: GET 2022-09-03T06:47:41.722 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:47:41.723 DEBUG:boto:Data: 2022-09-03T06:47:41.723 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.724 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.725 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.726 DEBUG:boto:Params: {} 2022-09-03T06:47:41.726 DEBUG:boto:Token: None 2022-09-03T06:47:41.728 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj116 2022-09-03T06:47:41.728 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N+z6jiM+yBvnn91omxIbGuiWRMU= 2022-09-03T06:47:41.729 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N+z6jiM+yBvnn91omxIbGuiWRMU=', 'Content-Length': '0'} 2022-09-03T06:47:41.733 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005a2d0ed71f9ff3e3-006312f88d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.734 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:47:41.734 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:47:41.735 DEBUG:boto:Method: GET 2022-09-03T06:47:41.735 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:47:41.736 DEBUG:boto:Data: 2022-09-03T06:47:41.736 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.737 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.737 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.738 DEBUG:boto:Params: {} 2022-09-03T06:47:41.738 DEBUG:boto:Token: None 2022-09-03T06:47:41.739 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj116 2022-09-03T06:47:41.740 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N+z6jiM+yBvnn91omxIbGuiWRMU= 2022-09-03T06:47:41.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N+z6jiM+yBvnn91omxIbGuiWRMU=', 'Content-Length': '0'} 2022-09-03T06:47:41.742 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ba4dc7c527b89be1-006312f88d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.743 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:41.744 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:41.744 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:41.745 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:41.745 DEBUG:boto:Method: GET 2022-09-03T06:47:41.746 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:47:41.746 DEBUG:boto:Data: 2022-09-03T06:47:41.747 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.747 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.748 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.749 DEBUG:boto:Params: {} 2022-09-03T06:47:41.749 DEBUG:boto:Token: None 2022-09-03T06:47:41.750 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/ 2022-09-03T06:47:41.750 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA= 2022-09-03T06:47:41.751 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA=', 'Content-Length': '0'} 2022-09-03T06:47:41.754 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000037fec22f8d032cf6-006312f88d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.755 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:41.756 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:41.757 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:41.758 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:41.759 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:47:41.760 DEBUG:boto:Method: GET 2022-09-03T06:47:41.760 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:47:41.761 DEBUG:boto:Data: 2022-09-03T06:47:41.762 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.762 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.763 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.764 DEBUG:boto:Params: {} 2022-09-03T06:47:41.765 DEBUG:boto:Token: None 2022-09-03T06:47:41.766 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/ 2022-09-03T06:47:41.767 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA= 2022-09-03T06:47:41.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA=', 'Content-Length': '0'} 2022-09-03T06:47:41.771 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000046d6d07166c3a491-006312f88d-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.814 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:41.815 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2022-09-03T06:47:41.816 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:41.817 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:41.817 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.818 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.818 DEBUG:boto:Method: GET 2022-09-03T06:47:41.819 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.820 DEBUG:boto:Data: 2022-09-03T06:47:41.820 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.821 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.822 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.822 DEBUG:boto:Params: {} 2022-09-03T06:47:41.823 DEBUG:boto:Token: None 2022-09-03T06:47:41.824 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.825 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J6f3xCFESIaPWZ8oNZvvugiGL4A= 2022-09-03T06:47:41.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J6f3xCFESIaPWZ8oNZvvugiGL4A=', 'Content-Length': '0'} 2022-09-03T06:47:41.828 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000043fdd4f80b5bc406-006312f88d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.829 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:41.829 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:41.830 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.830 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.831 DEBUG:boto:Method: GET 2022-09-03T06:47:41.831 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.832 DEBUG:boto:Data: 2022-09-03T06:47:41.832 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.833 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.834 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.835 DEBUG:boto:Params: {} 2022-09-03T06:47:41.835 DEBUG:boto:Token: None 2022-09-03T06:47:41.836 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.837 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J6f3xCFESIaPWZ8oNZvvugiGL4A= 2022-09-03T06:47:41.837 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J6f3xCFESIaPWZ8oNZvvugiGL4A=', 'Content-Length': '0'} 2022-09-03T06:47:41.840 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000053a33f2d3685e80a-006312f88d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.840 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:41.841 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:41.842 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.842 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.843 DEBUG:boto:Method: HEAD 2022-09-03T06:47:41.843 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.844 DEBUG:boto:Data: 2022-09-03T06:47:41.845 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.845 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.846 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.846 DEBUG:boto:Params: {} 2022-09-03T06:47:41.847 DEBUG:boto:Token: None 2022-09-03T06:47:41.847 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.848 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZHbV4gvMm0ydzh9MpDdkeOVm0KA= 2022-09-03T06:47:41.849 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZHbV4gvMm0ydzh9MpDdkeOVm0KA=', 'Content-Length': '0'} 2022-09-03T06:47:41.851 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003f5d5a3b96298da3-006312f88d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.852 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:41.852 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:41.853 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.853 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.854 DEBUG:boto:Method: HEAD 2022-09-03T06:47:41.855 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.856 DEBUG:boto:Data: 2022-09-03T06:47:41.856 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.857 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.857 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.858 DEBUG:boto:Params: {} 2022-09-03T06:47:41.858 DEBUG:boto:Token: None 2022-09-03T06:47:41.859 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.860 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZHbV4gvMm0ydzh9MpDdkeOVm0KA= 2022-09-03T06:47:41.861 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZHbV4gvMm0ydzh9MpDdkeOVm0KA=', 'Content-Length': '0'} 2022-09-03T06:47:41.864 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d23c87393eb4eb42-006312f88d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.864 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2022-09-03T06:47:41.865 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:41.866 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:41.866 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.867 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.868 DEBUG:boto:Method: GET 2022-09-03T06:47:41.868 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.869 DEBUG:boto:Data: 2022-09-03T06:47:41.869 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.870 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.870 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.871 DEBUG:boto:Params: {} 2022-09-03T06:47:41.872 DEBUG:boto:Token: None 2022-09-03T06:47:41.872 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.873 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J6f3xCFESIaPWZ8oNZvvugiGL4A= 2022-09-03T06:47:41.874 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J6f3xCFESIaPWZ8oNZvvugiGL4A=', 'Content-Length': '0'} 2022-09-03T06:47:41.876 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a74be5023222f15e-006312f88d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.923 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:41.924 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:41.926 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.927 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.928 DEBUG:boto:Method: GET 2022-09-03T06:47:41.929 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.929 DEBUG:boto:Data: 2022-09-03T06:47:41.930 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.931 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.932 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.932 DEBUG:boto:Params: {} 2022-09-03T06:47:41.933 DEBUG:boto:Token: None 2022-09-03T06:47:41.934 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:47:41.935 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:J6f3xCFESIaPWZ8oNZvvugiGL4A= 2022-09-03T06:47:41.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:J6f3xCFESIaPWZ8oNZvvugiGL4A=', 'Content-Length': '0'} 2022-09-03T06:47:41.938 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008937b4d3aba4d7ed-006312f88d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.940 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:41.940 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:41.941 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:41.942 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:41.943 DEBUG:boto:Method: GET 2022-09-03T06:47:41.944 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:47:41.944 DEBUG:boto:Data: 2022-09-03T06:47:41.945 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.946 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.947 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.949 DEBUG:boto:Params: {} 2022-09-03T06:47:41.950 DEBUG:boto:Token: None 2022-09-03T06:47:41.951 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/ 2022-09-03T06:47:41.952 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA= 2022-09-03T06:47:41.953 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA=', 'Content-Length': '0'} 2022-09-03T06:47:41.959 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e170fcc17c03cdd0-006312f88d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.960 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:41.961 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:41.962 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:41.963 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:41.963 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:41.964 DEBUG:boto:Method: GET 2022-09-03T06:47:41.964 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:47:41.965 DEBUG:boto:Data: 2022-09-03T06:47:41.965 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.966 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.966 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.967 DEBUG:boto:Params: {} 2022-09-03T06:47:41.968 DEBUG:boto:Token: None 2022-09-03T06:47:41.968 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/ 2022-09-03T06:47:41.969 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA= 2022-09-03T06:47:41.969 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9TIwrkHjj7iTPF+BUq/T5H0FPVA=', 'Content-Length': '0'} 2022-09-03T06:47:41.972 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004b05c3bc22d51be4-006312f88d-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.973 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:41.974 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:41.975 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:41.976 DEBUG:boto:Method: HEAD 2022-09-03T06:47:41.977 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:47:41.978 DEBUG:boto:Data: 2022-09-03T06:47:41.979 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.980 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:41.981 DEBUG:boto:Port: 8001 2022-09-03T06:47:41.982 DEBUG:boto:Params: {} 2022-09-03T06:47:41.983 DEBUG:boto:Token: None 2022-09-03T06:47:41.984 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj124 2022-09-03T06:47:41.985 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Qi1B6q8+bprTfT7Dnf9kyH1lBiQ= 2022-09-03T06:47:41.986 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Qi1B6q8+bprTfT7Dnf9kyH1lBiQ=', 'Content-Length': '0'} 2022-09-03T06:47:41.989 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006fbfdd0d7b958945-006312f88d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:41.990 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:41.990 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:41.991 DEBUG:boto:Method: HEAD 2022-09-03T06:47:41.992 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:47:41.992 DEBUG:boto:Data: 2022-09-03T06:47:41.993 DEBUG:boto:Headers: {} 2022-09-03T06:47:41.993 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:41.994 DEBUG:boto:Port: 8000 2022-09-03T06:47:41.995 DEBUG:boto:Params: {} 2022-09-03T06:47:41.995 DEBUG:boto:Token: None 2022-09-03T06:47:41.996 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:41 GMT /dsgvcy-75/obj124 2022-09-03T06:47:41.997 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Qi1B6q8+bprTfT7Dnf9kyH1lBiQ= 2022-09-03T06:47:41.997 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:41 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Qi1B6q8+bprTfT7Dnf9kyH1lBiQ=', 'Content-Length': '0'} 2022-09-03T06:47:41.999 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000830048900338fb8a-006312f88d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:41 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.000 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2022-09-03T06:47:42.001 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:42.001 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:42.002 DEBUG:boto:Method: GET 2022-09-03T06:47:42.003 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:47:42.003 DEBUG:boto:Data: 2022-09-03T06:47:42.004 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.004 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.005 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.005 DEBUG:boto:Params: {} 2022-09-03T06:47:42.006 DEBUG:boto:Token: None 2022-09-03T06:47:42.006 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj124 2022-09-03T06:47:42.007 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gl7qRE7DNpdjYzERIJo0JDoHOGU= 2022-09-03T06:47:42.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gl7qRE7DNpdjYzERIJo0JDoHOGU=', 'Content-Length': '0'} 2022-09-03T06:47:42.010 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e756e476562e53f-006312f88e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.011 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:47:42.011 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:47:42.012 DEBUG:boto:Method: GET 2022-09-03T06:47:42.012 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:47:42.013 DEBUG:boto:Data: 2022-09-03T06:47:42.013 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.014 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.014 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.015 DEBUG:boto:Params: {} 2022-09-03T06:47:42.015 DEBUG:boto:Token: None 2022-09-03T06:47:42.016 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj124 2022-09-03T06:47:42.017 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gl7qRE7DNpdjYzERIJo0JDoHOGU= 2022-09-03T06:47:42.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gl7qRE7DNpdjYzERIJo0JDoHOGU=', 'Content-Length': '0'} 2022-09-03T06:47:42.020 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000084fdaa8dbf9c095b-006312f88e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.020 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:42.021 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:42.022 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:42.022 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:42.023 DEBUG:boto:Method: GET 2022-09-03T06:47:42.024 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:47:42.024 DEBUG:boto:Data: 2022-09-03T06:47:42.025 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.026 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.026 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.027 DEBUG:boto:Params: {} 2022-09-03T06:47:42.027 DEBUG:boto:Token: None 2022-09-03T06:47:42.028 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/ 2022-09-03T06:47:42.029 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y= 2022-09-03T06:47:42.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y=', 'Content-Length': '0'} 2022-09-03T06:47:42.032 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000080a00384100fdfa2-006312f88e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.033 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:42.033 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:42.034 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:42.035 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:42.035 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:47:42.036 DEBUG:boto:Method: GET 2022-09-03T06:47:42.036 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:47:42.037 DEBUG:boto:Data: 2022-09-03T06:47:42.037 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.038 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.038 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.039 DEBUG:boto:Params: {} 2022-09-03T06:47:42.040 DEBUG:boto:Token: None 2022-09-03T06:47:42.040 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/ 2022-09-03T06:47:42.041 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y= 2022-09-03T06:47:42.041 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y=', 'Content-Length': '0'} 2022-09-03T06:47:42.044 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007298e4dcdd50fcac-006312f88e-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.045 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:42.046 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:47:42.047 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:42.047 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:42.048 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.048 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.049 DEBUG:boto:Method: GET 2022-09-03T06:47:42.049 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.050 DEBUG:boto:Data: 2022-09-03T06:47:42.050 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.051 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.052 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.052 DEBUG:boto:Params: {} 2022-09-03T06:47:42.053 DEBUG:boto:Token: None 2022-09-03T06:47:42.053 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.054 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1vTDnK/LI31i7tg9Wt9Vr/7fwTg= 2022-09-03T06:47:42.055 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1vTDnK/LI31i7tg9Wt9Vr/7fwTg=', 'Content-Length': '0'} 2022-09-03T06:47:42.057 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e8a64fd65afad39-006312f88e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.058 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:42.058 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:42.059 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.059 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.060 DEBUG:boto:Method: GET 2022-09-03T06:47:42.060 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.061 DEBUG:boto:Data: 2022-09-03T06:47:42.061 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.062 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.062 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.063 DEBUG:boto:Params: {} 2022-09-03T06:47:42.064 DEBUG:boto:Token: None 2022-09-03T06:47:42.064 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.065 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1vTDnK/LI31i7tg9Wt9Vr/7fwTg= 2022-09-03T06:47:42.065 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1vTDnK/LI31i7tg9Wt9Vr/7fwTg=', 'Content-Length': '0'} 2022-09-03T06:47:42.068 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002425d69bcfc243d2-006312f88e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.069 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:42.069 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:42.070 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.070 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.071 DEBUG:boto:Method: HEAD 2022-09-03T06:47:42.071 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.072 DEBUG:boto:Data: 2022-09-03T06:47:42.072 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.073 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.073 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.074 DEBUG:boto:Params: {} 2022-09-03T06:47:42.075 DEBUG:boto:Token: None 2022-09-03T06:47:42.075 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.076 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s1nCS64bs9KhbuKuW6szHyfj+uk= 2022-09-03T06:47:42.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s1nCS64bs9KhbuKuW6szHyfj+uk=', 'Content-Length': '0'} 2022-09-03T06:47:42.079 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c6e70373b5a3bf3-006312f88e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.079 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:42.080 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:42.080 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.081 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.081 DEBUG:boto:Method: HEAD 2022-09-03T06:47:42.082 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.083 DEBUG:boto:Data: 2022-09-03T06:47:42.083 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.084 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.084 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.085 DEBUG:boto:Params: {} 2022-09-03T06:47:42.085 DEBUG:boto:Token: None 2022-09-03T06:47:42.086 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.086 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s1nCS64bs9KhbuKuW6szHyfj+uk= 2022-09-03T06:47:42.087 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s1nCS64bs9KhbuKuW6szHyfj+uk=', 'Content-Length': '0'} 2022-09-03T06:47:42.089 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005feb6d96b3235ba2-006312f88e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.090 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:47:42.090 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:42.091 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:42.091 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.092 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.092 DEBUG:boto:Method: GET 2022-09-03T06:47:42.093 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.094 DEBUG:boto:Data: 2022-09-03T06:47:42.094 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.095 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.095 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.096 DEBUG:boto:Params: {} 2022-09-03T06:47:42.096 DEBUG:boto:Token: None 2022-09-03T06:47:42.097 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.097 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1vTDnK/LI31i7tg9Wt9Vr/7fwTg= 2022-09-03T06:47:42.098 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1vTDnK/LI31i7tg9Wt9Vr/7fwTg=', 'Content-Length': '0'} 2022-09-03T06:47:42.100 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000098454ff77bf72bc6-006312f88e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.142 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:42.143 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:42.144 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.145 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.146 DEBUG:boto:Method: GET 2022-09-03T06:47:42.147 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.148 DEBUG:boto:Data: 2022-09-03T06:47:42.149 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.151 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.151 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.152 DEBUG:boto:Params: {} 2022-09-03T06:47:42.152 DEBUG:boto:Token: None 2022-09-03T06:47:42.153 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:47:42.154 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1vTDnK/LI31i7tg9Wt9Vr/7fwTg= 2022-09-03T06:47:42.155 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1vTDnK/LI31i7tg9Wt9Vr/7fwTg=', 'Content-Length': '0'} 2022-09-03T06:47:42.157 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000309a20c46b52bb73-006312f88e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.158 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:42.159 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:42.160 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:42.160 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:42.161 DEBUG:boto:Method: GET 2022-09-03T06:47:42.161 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:47:42.162 DEBUG:boto:Data: 2022-09-03T06:47:42.162 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.163 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.163 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.164 DEBUG:boto:Params: {} 2022-09-03T06:47:42.164 DEBUG:boto:Token: None 2022-09-03T06:47:42.165 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/ 2022-09-03T06:47:42.166 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y= 2022-09-03T06:47:42.166 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y=', 'Content-Length': '0'} 2022-09-03T06:47:42.169 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a3f7639c7d7e961d-006312f88e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.170 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:42.171 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:42.171 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:42.172 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:42.172 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:42.173 DEBUG:boto:Method: GET 2022-09-03T06:47:42.174 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:47:42.174 DEBUG:boto:Data: 2022-09-03T06:47:42.175 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.176 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.176 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.177 DEBUG:boto:Params: {} 2022-09-03T06:47:42.177 DEBUG:boto:Token: None 2022-09-03T06:47:42.178 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/ 2022-09-03T06:47:42.179 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y= 2022-09-03T06:47:42.179 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y=', 'Content-Length': '0'} 2022-09-03T06:47:42.183 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d5e768bcbf806988-006312f88e-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.183 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:42.184 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:42.185 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:42.185 DEBUG:boto:Method: HEAD 2022-09-03T06:47:42.186 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:47:42.186 DEBUG:boto:Data: 2022-09-03T06:47:42.187 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.187 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.188 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.188 DEBUG:boto:Params: {} 2022-09-03T06:47:42.189 DEBUG:boto:Token: None 2022-09-03T06:47:42.189 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj145 2022-09-03T06:47:42.190 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zxFXOb5McwqS+ahVQDbkXuh2hNk= 2022-09-03T06:47:42.191 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zxFXOb5McwqS+ahVQDbkXuh2hNk=', 'Content-Length': '0'} 2022-09-03T06:47:42.193 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c87a97642eeb4d08-006312f88e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.193 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:42.194 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:42.194 DEBUG:boto:Method: HEAD 2022-09-03T06:47:42.195 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:47:42.195 DEBUG:boto:Data: 2022-09-03T06:47:42.196 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.196 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.197 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.198 DEBUG:boto:Params: {} 2022-09-03T06:47:42.198 DEBUG:boto:Token: None 2022-09-03T06:47:42.199 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj145 2022-09-03T06:47:42.200 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zxFXOb5McwqS+ahVQDbkXuh2hNk= 2022-09-03T06:47:42.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zxFXOb5McwqS+ahVQDbkXuh2hNk=', 'Content-Length': '0'} 2022-09-03T06:47:42.202 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000038f9f0205e20c8a2-006312f88e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.203 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:47:42.204 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:42.205 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:42.205 DEBUG:boto:Method: GET 2022-09-03T06:47:42.206 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:47:42.206 DEBUG:boto:Data: 2022-09-03T06:47:42.207 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.207 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.208 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.209 DEBUG:boto:Params: {} 2022-09-03T06:47:42.209 DEBUG:boto:Token: None 2022-09-03T06:47:42.210 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj145 2022-09-03T06:47:42.211 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y8L6PXnRZuSw5zUJY7mKLHjPmxs= 2022-09-03T06:47:42.211 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y8L6PXnRZuSw5zUJY7mKLHjPmxs=', 'Content-Length': '0'} 2022-09-03T06:47:42.215 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007a803d1c885905a2-006312f88e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.215 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:47:42.216 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:47:42.217 DEBUG:boto:Method: GET 2022-09-03T06:47:42.217 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:47:42.218 DEBUG:boto:Data: 2022-09-03T06:47:42.218 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.219 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.219 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.220 DEBUG:boto:Params: {} 2022-09-03T06:47:42.221 DEBUG:boto:Token: None 2022-09-03T06:47:42.221 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj145 2022-09-03T06:47:42.222 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y8L6PXnRZuSw5zUJY7mKLHjPmxs= 2022-09-03T06:47:42.223 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y8L6PXnRZuSw5zUJY7mKLHjPmxs=', 'Content-Length': '0'} 2022-09-03T06:47:42.225 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c6c7d1cb3620e210-006312f88e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.266 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:42.267 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:42.267 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:42.268 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:42.269 DEBUG:boto:Method: GET 2022-09-03T06:47:42.270 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:47:42.271 DEBUG:boto:Data: 2022-09-03T06:47:42.271 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.272 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.272 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.273 DEBUG:boto:Params: {} 2022-09-03T06:47:42.274 DEBUG:boto:Token: None 2022-09-03T06:47:42.275 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/ 2022-09-03T06:47:42.276 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y= 2022-09-03T06:47:42.277 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y=', 'Content-Length': '0'} 2022-09-03T06:47:42.280 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b910e4f3240d93e8-006312f88e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.280 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:42.281 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:42.282 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:42.282 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:42.283 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:47:42.283 DEBUG:boto:Method: GET 2022-09-03T06:47:42.284 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:47:42.284 DEBUG:boto:Data: 2022-09-03T06:47:42.285 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.286 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.286 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.287 DEBUG:boto:Params: {} 2022-09-03T06:47:42.288 DEBUG:boto:Token: None 2022-09-03T06:47:42.289 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/ 2022-09-03T06:47:42.290 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y= 2022-09-03T06:47:42.290 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y=', 'Content-Length': '0'} 2022-09-03T06:47:42.293 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001e863fb505686a47-006312f88e-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.294 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:42.296 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2022-09-03T06:47:42.296 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:42.297 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:42.298 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.299 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.300 DEBUG:boto:Method: GET 2022-09-03T06:47:42.301 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.302 DEBUG:boto:Data: 2022-09-03T06:47:42.303 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.303 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.304 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.304 DEBUG:boto:Params: {} 2022-09-03T06:47:42.305 DEBUG:boto:Token: None 2022-09-03T06:47:42.306 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.307 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tjYIexaus+3lIHdzWmyUKeEeLP8= 2022-09-03T06:47:42.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tjYIexaus+3lIHdzWmyUKeEeLP8=', 'Content-Length': '0'} 2022-09-03T06:47:42.310 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b776a94040ebb785-006312f88e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.311 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:42.312 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:42.312 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.313 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.313 DEBUG:boto:Method: GET 2022-09-03T06:47:42.314 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.316 DEBUG:boto:Data: 2022-09-03T06:47:42.317 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.318 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.319 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.321 DEBUG:boto:Params: {} 2022-09-03T06:47:42.322 DEBUG:boto:Token: None 2022-09-03T06:47:42.324 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.325 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tjYIexaus+3lIHdzWmyUKeEeLP8= 2022-09-03T06:47:42.326 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tjYIexaus+3lIHdzWmyUKeEeLP8=', 'Content-Length': '0'} 2022-09-03T06:47:42.329 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009fe8abf6c94389f7-006312f88e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.330 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:42.331 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:42.332 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.333 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.334 DEBUG:boto:Method: HEAD 2022-09-03T06:47:42.335 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.336 DEBUG:boto:Data: 2022-09-03T06:47:42.337 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.338 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.338 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.339 DEBUG:boto:Params: {} 2022-09-03T06:47:42.340 DEBUG:boto:Token: None 2022-09-03T06:47:42.341 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.342 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bVoz4BLGk9bglIOLzceOHCGVSrI= 2022-09-03T06:47:42.343 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bVoz4BLGk9bglIOLzceOHCGVSrI=', 'Content-Length': '0'} 2022-09-03T06:47:42.346 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fcbaf8fa3b77fd48-006312f88e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.347 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:42.348 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:42.349 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.350 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.350 DEBUG:boto:Method: HEAD 2022-09-03T06:47:42.351 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.351 DEBUG:boto:Data: 2022-09-03T06:47:42.352 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.352 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.353 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.353 DEBUG:boto:Params: {} 2022-09-03T06:47:42.354 DEBUG:boto:Token: None 2022-09-03T06:47:42.354 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.355 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bVoz4BLGk9bglIOLzceOHCGVSrI= 2022-09-03T06:47:42.355 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bVoz4BLGk9bglIOLzceOHCGVSrI=', 'Content-Length': '0'} 2022-09-03T06:47:42.358 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000090c4bb4e04b96e95-006312f88e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.359 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2022-09-03T06:47:42.359 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:42.360 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:42.360 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.361 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.362 DEBUG:boto:Method: GET 2022-09-03T06:47:42.362 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.363 DEBUG:boto:Data: 2022-09-03T06:47:42.363 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.364 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.364 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.365 DEBUG:boto:Params: {} 2022-09-03T06:47:42.365 DEBUG:boto:Token: None 2022-09-03T06:47:42.366 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tjYIexaus+3lIHdzWmyUKeEeLP8= 2022-09-03T06:47:42.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tjYIexaus+3lIHdzWmyUKeEeLP8=', 'Content-Length': '0'} 2022-09-03T06:47:42.369 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d284fc7fe7a9e56d-006312f88e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.414 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:42.415 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:42.416 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.416 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.417 DEBUG:boto:Method: GET 2022-09-03T06:47:42.417 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.418 DEBUG:boto:Data: 2022-09-03T06:47:42.419 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.419 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.420 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.421 DEBUG:boto:Params: {} 2022-09-03T06:47:42.421 DEBUG:boto:Token: None 2022-09-03T06:47:42.422 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:47:42.423 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tjYIexaus+3lIHdzWmyUKeEeLP8= 2022-09-03T06:47:42.423 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tjYIexaus+3lIHdzWmyUKeEeLP8=', 'Content-Length': '0'} 2022-09-03T06:47:42.426 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009306ece28b3ac353-006312f88e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.428 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:42.429 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:42.430 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:42.431 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:42.432 DEBUG:boto:Method: GET 2022-09-03T06:47:42.432 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:47:42.433 DEBUG:boto:Data: 2022-09-03T06:47:42.433 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.434 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.435 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.436 DEBUG:boto:Params: {} 2022-09-03T06:47:42.437 DEBUG:boto:Token: None 2022-09-03T06:47:42.438 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/ 2022-09-03T06:47:42.439 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y= 2022-09-03T06:47:42.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y=', 'Content-Length': '0'} 2022-09-03T06:47:42.443 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007da6227455a95c4d-006312f88e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.444 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:42.445 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:42.446 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:42.446 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:42.447 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:42.447 DEBUG:boto:Method: GET 2022-09-03T06:47:42.449 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:47:42.449 DEBUG:boto:Data: 2022-09-03T06:47:42.450 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.451 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.452 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.453 DEBUG:boto:Params: {} 2022-09-03T06:47:42.454 DEBUG:boto:Token: None 2022-09-03T06:47:42.455 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/ 2022-09-03T06:47:42.456 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y= 2022-09-03T06:47:42.457 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y=', 'Content-Length': '0'} 2022-09-03T06:47:42.459 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005f9fca54e8bb6440-006312f88e-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.460 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:42.462 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:42.463 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:42.464 DEBUG:boto:Method: HEAD 2022-09-03T06:47:42.465 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:47:42.466 DEBUG:boto:Data: 2022-09-03T06:47:42.467 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.467 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.468 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.469 DEBUG:boto:Params: {} 2022-09-03T06:47:42.470 DEBUG:boto:Token: None 2022-09-03T06:47:42.471 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj155 2022-09-03T06:47:42.472 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iuAk8ioRoLoEyVo6xOZV6owFKuY= 2022-09-03T06:47:42.474 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iuAk8ioRoLoEyVo6xOZV6owFKuY=', 'Content-Length': '0'} 2022-09-03T06:47:42.476 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b6b3f78167af9887-006312f88e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.478 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:42.479 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:42.480 DEBUG:boto:Method: HEAD 2022-09-03T06:47:42.481 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:47:42.482 DEBUG:boto:Data: 2022-09-03T06:47:42.483 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.484 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.485 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.486 DEBUG:boto:Params: {} 2022-09-03T06:47:42.487 DEBUG:boto:Token: None 2022-09-03T06:47:42.488 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj155 2022-09-03T06:47:42.489 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iuAk8ioRoLoEyVo6xOZV6owFKuY= 2022-09-03T06:47:42.490 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iuAk8ioRoLoEyVo6xOZV6owFKuY=', 'Content-Length': '0'} 2022-09-03T06:47:42.494 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c62586dcfadf6e9e-006312f88e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.495 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2022-09-03T06:47:42.496 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:42.497 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:42.498 DEBUG:boto:Method: GET 2022-09-03T06:47:42.499 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:47:42.500 DEBUG:boto:Data: 2022-09-03T06:47:42.500 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.501 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.502 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.503 DEBUG:boto:Params: {} 2022-09-03T06:47:42.504 DEBUG:boto:Token: None 2022-09-03T06:47:42.505 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj155 2022-09-03T06:47:42.506 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DJYmfmFEetg+E6XQ4pRb8xjIgiE= 2022-09-03T06:47:42.506 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DJYmfmFEetg+E6XQ4pRb8xjIgiE=', 'Content-Length': '0'} 2022-09-03T06:47:42.508 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002bc036418153e39b-006312f88e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.509 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:47:42.510 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:47:42.510 DEBUG:boto:Method: GET 2022-09-03T06:47:42.511 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:47:42.511 DEBUG:boto:Data: 2022-09-03T06:47:42.512 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.513 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.514 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.515 DEBUG:boto:Params: {} 2022-09-03T06:47:42.516 DEBUG:boto:Token: None 2022-09-03T06:47:42.517 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj155 2022-09-03T06:47:42.518 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DJYmfmFEetg+E6XQ4pRb8xjIgiE= 2022-09-03T06:47:42.519 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DJYmfmFEetg+E6XQ4pRb8xjIgiE=', 'Content-Length': '0'} 2022-09-03T06:47:42.522 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b09d7415a002169-006312f88e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.524 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:42.524 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:42.525 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:42.525 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:42.526 DEBUG:boto:Method: GET 2022-09-03T06:47:42.527 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:47:42.527 DEBUG:boto:Data: 2022-09-03T06:47:42.528 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.528 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.529 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.529 DEBUG:boto:Params: {} 2022-09-03T06:47:42.530 DEBUG:boto:Token: None 2022-09-03T06:47:42.531 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/ 2022-09-03T06:47:42.531 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y= 2022-09-03T06:47:42.533 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y=', 'Content-Length': '0'} 2022-09-03T06:47:42.536 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004c502ab31507459d-006312f88e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.537 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:42.538 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:42.539 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:42.540 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:42.541 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:47:42.542 DEBUG:boto:Method: GET 2022-09-03T06:47:42.543 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:47:42.544 DEBUG:boto:Data: 2022-09-03T06:47:42.544 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.545 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.545 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.546 DEBUG:boto:Params: {} 2022-09-03T06:47:42.547 DEBUG:boto:Token: None 2022-09-03T06:47:42.547 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/ 2022-09-03T06:47:42.548 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y= 2022-09-03T06:47:42.549 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y=', 'Content-Length': '0'} 2022-09-03T06:47:42.552 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f9393051ce45bf7a-006312f88e-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.598 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:42.599 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:47:42.600 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:42.600 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:42.601 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.601 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.602 DEBUG:boto:Method: GET 2022-09-03T06:47:42.603 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.603 DEBUG:boto:Data: 2022-09-03T06:47:42.604 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.604 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.605 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.605 DEBUG:boto:Params: {} 2022-09-03T06:47:42.606 DEBUG:boto:Token: None 2022-09-03T06:47:42.607 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.607 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mghi7DWGQYCSHX59PC7wD1xEuvs= 2022-09-03T06:47:42.608 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mghi7DWGQYCSHX59PC7wD1xEuvs=', 'Content-Length': '0'} 2022-09-03T06:47:42.610 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000255b4bef84402b58-006312f88e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.611 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:42.612 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:42.612 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.613 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.613 DEBUG:boto:Method: GET 2022-09-03T06:47:42.614 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.614 DEBUG:boto:Data: 2022-09-03T06:47:42.615 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.616 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.616 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.617 DEBUG:boto:Params: {} 2022-09-03T06:47:42.617 DEBUG:boto:Token: None 2022-09-03T06:47:42.618 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.618 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mghi7DWGQYCSHX59PC7wD1xEuvs= 2022-09-03T06:47:42.619 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mghi7DWGQYCSHX59PC7wD1xEuvs=', 'Content-Length': '0'} 2022-09-03T06:47:42.621 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fffe09317e26d586-006312f88e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.622 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:42.623 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:42.624 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.625 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.626 DEBUG:boto:Method: HEAD 2022-09-03T06:47:42.627 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.627 DEBUG:boto:Data: 2022-09-03T06:47:42.628 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.628 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.629 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.630 DEBUG:boto:Params: {} 2022-09-03T06:47:42.630 DEBUG:boto:Token: None 2022-09-03T06:47:42.631 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.632 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5Nf+BYgMKthIFKQ+S9NReaNVsD4= 2022-09-03T06:47:42.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5Nf+BYgMKthIFKQ+S9NReaNVsD4=', 'Content-Length': '0'} 2022-09-03T06:47:42.635 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000f0f645bbace260b-006312f88e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.637 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:42.638 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:42.639 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.639 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.640 DEBUG:boto:Method: HEAD 2022-09-03T06:47:42.640 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.641 DEBUG:boto:Data: 2022-09-03T06:47:42.642 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.642 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.643 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.643 DEBUG:boto:Params: {} 2022-09-03T06:47:42.644 DEBUG:boto:Token: None 2022-09-03T06:47:42.645 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.645 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5Nf+BYgMKthIFKQ+S9NReaNVsD4= 2022-09-03T06:47:42.646 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5Nf+BYgMKthIFKQ+S9NReaNVsD4=', 'Content-Length': '0'} 2022-09-03T06:47:42.648 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b2ab7b1565aa3fa-006312f88e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.649 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:47:42.650 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:42.650 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:42.651 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.651 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.652 DEBUG:boto:Method: GET 2022-09-03T06:47:42.652 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.653 DEBUG:boto:Data: 2022-09-03T06:47:42.653 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.654 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.654 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.655 DEBUG:boto:Params: {} 2022-09-03T06:47:42.656 DEBUG:boto:Token: None 2022-09-03T06:47:42.656 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.657 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mghi7DWGQYCSHX59PC7wD1xEuvs= 2022-09-03T06:47:42.658 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mghi7DWGQYCSHX59PC7wD1xEuvs=', 'Content-Length': '0'} 2022-09-03T06:47:42.660 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000049bad07fe8b62b0-006312f88e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.702 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:42.703 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:42.704 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.705 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.706 DEBUG:boto:Method: GET 2022-09-03T06:47:42.707 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.708 DEBUG:boto:Data: 2022-09-03T06:47:42.709 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.710 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.712 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.713 DEBUG:boto:Params: {} 2022-09-03T06:47:42.714 DEBUG:boto:Token: None 2022-09-03T06:47:42.715 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:47:42.716 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mghi7DWGQYCSHX59PC7wD1xEuvs= 2022-09-03T06:47:42.718 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mghi7DWGQYCSHX59PC7wD1xEuvs=', 'Content-Length': '0'} 2022-09-03T06:47:42.720 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004ae7a83c320ea8c5-006312f88e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.722 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:42.724 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:42.724 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:42.725 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:42.725 DEBUG:boto:Method: GET 2022-09-03T06:47:42.726 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:47:42.727 DEBUG:boto:Data: 2022-09-03T06:47:42.727 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.728 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.728 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.729 DEBUG:boto:Params: {} 2022-09-03T06:47:42.730 DEBUG:boto:Token: None 2022-09-03T06:47:42.730 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/ 2022-09-03T06:47:42.731 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y= 2022-09-03T06:47:42.732 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y=', 'Content-Length': '0'} 2022-09-03T06:47:42.736 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ddc522a8fbbbf3d1-006312f88e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.736 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:42.737 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:42.738 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:42.739 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:42.739 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:42.740 DEBUG:boto:Method: GET 2022-09-03T06:47:42.740 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:47:42.741 DEBUG:boto:Data: 2022-09-03T06:47:42.741 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.742 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.743 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.744 DEBUG:boto:Params: {} 2022-09-03T06:47:42.744 DEBUG:boto:Token: None 2022-09-03T06:47:42.745 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/ 2022-09-03T06:47:42.746 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y= 2022-09-03T06:47:42.746 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y=', 'Content-Length': '0'} 2022-09-03T06:47:42.749 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000087b0c5d42e570201-006312f88e-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.750 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:42.751 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:42.751 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:42.752 DEBUG:boto:Method: HEAD 2022-09-03T06:47:42.752 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:47:42.753 DEBUG:boto:Data: 2022-09-03T06:47:42.754 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.754 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.755 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.755 DEBUG:boto:Params: {} 2022-09-03T06:47:42.756 DEBUG:boto:Token: None 2022-09-03T06:47:42.757 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj17 2022-09-03T06:47:42.758 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:10afJFAPgnjud0Hu6tGMAQ6rJh8= 2022-09-03T06:47:42.758 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:10afJFAPgnjud0Hu6tGMAQ6rJh8=', 'Content-Length': '0'} 2022-09-03T06:47:42.760 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000241dbc0127cd83a1-006312f88e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.761 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:42.762 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:42.763 DEBUG:boto:Method: HEAD 2022-09-03T06:47:42.764 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:47:42.765 DEBUG:boto:Data: 2022-09-03T06:47:42.766 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.767 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.768 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.769 DEBUG:boto:Params: {} 2022-09-03T06:47:42.770 DEBUG:boto:Token: None 2022-09-03T06:47:42.771 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj17 2022-09-03T06:47:42.772 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:10afJFAPgnjud0Hu6tGMAQ6rJh8= 2022-09-03T06:47:42.773 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:10afJFAPgnjud0Hu6tGMAQ6rJh8=', 'Content-Length': '0'} 2022-09-03T06:47:42.775 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e471082bdfde5b87-006312f88e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.777 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:47:42.777 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:42.778 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:42.779 DEBUG:boto:Method: GET 2022-09-03T06:47:42.780 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:47:42.781 DEBUG:boto:Data: 2022-09-03T06:47:42.782 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.783 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.784 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.784 DEBUG:boto:Params: {} 2022-09-03T06:47:42.785 DEBUG:boto:Token: None 2022-09-03T06:47:42.786 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj17 2022-09-03T06:47:42.787 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pVqOacIACbz+aLkTl3qADzG1uFM= 2022-09-03T06:47:42.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pVqOacIACbz+aLkTl3qADzG1uFM=', 'Content-Length': '0'} 2022-09-03T06:47:42.790 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000519c04dced97d930-006312f88e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.791 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:47:42.792 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:47:42.793 DEBUG:boto:Method: GET 2022-09-03T06:47:42.793 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:47:42.794 DEBUG:boto:Data: 2022-09-03T06:47:42.794 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.795 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.796 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.796 DEBUG:boto:Params: {} 2022-09-03T06:47:42.797 DEBUG:boto:Token: None 2022-09-03T06:47:42.798 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj17 2022-09-03T06:47:42.798 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pVqOacIACbz+aLkTl3qADzG1uFM= 2022-09-03T06:47:42.799 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pVqOacIACbz+aLkTl3qADzG1uFM=', 'Content-Length': '0'} 2022-09-03T06:47:42.801 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f497754234f2adb-006312f88e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.802 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:42.803 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:42.803 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:42.804 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:42.804 DEBUG:boto:Method: GET 2022-09-03T06:47:42.805 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:47:42.805 DEBUG:boto:Data: 2022-09-03T06:47:42.806 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.806 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.807 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.808 DEBUG:boto:Params: {} 2022-09-03T06:47:42.808 DEBUG:boto:Token: None 2022-09-03T06:47:42.809 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/ 2022-09-03T06:47:42.809 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y= 2022-09-03T06:47:42.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y=', 'Content-Length': '0'} 2022-09-03T06:47:42.820 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009c686acca4d8323c-006312f88e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.862 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:42.863 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:42.864 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:42.864 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:42.865 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:47:42.865 DEBUG:boto:Method: GET 2022-09-03T06:47:42.866 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:47:42.866 DEBUG:boto:Data: 2022-09-03T06:47:42.867 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.868 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.868 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.869 DEBUG:boto:Params: {} 2022-09-03T06:47:42.869 DEBUG:boto:Token: None 2022-09-03T06:47:42.871 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/ 2022-09-03T06:47:42.871 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y= 2022-09-03T06:47:42.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y+XVtjythtd8d6NWG37zOe54W5Y=', 'Content-Length': '0'} 2022-09-03T06:47:42.875 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006dd3a66f8d5ae3a0-006312f88e-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.876 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:42.877 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:47:42.877 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:42.878 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:42.879 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.879 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.880 DEBUG:boto:Method: GET 2022-09-03T06:47:42.881 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.881 DEBUG:boto:Data: 2022-09-03T06:47:42.882 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.882 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.883 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.884 DEBUG:boto:Params: {} 2022-09-03T06:47:42.884 DEBUG:boto:Token: None 2022-09-03T06:47:42.885 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.886 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j8XWwd2iosrYFiB8K7HNRZ6BPRA= 2022-09-03T06:47:42.887 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j8XWwd2iosrYFiB8K7HNRZ6BPRA=', 'Content-Length': '0'} 2022-09-03T06:47:42.889 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c6f7ec51e19b60a2-006312f88e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.890 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:42.891 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:42.892 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.893 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.895 DEBUG:boto:Method: GET 2022-09-03T06:47:42.896 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.896 DEBUG:boto:Data: 2022-09-03T06:47:42.897 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.899 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.900 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.901 DEBUG:boto:Params: {} 2022-09-03T06:47:42.902 DEBUG:boto:Token: None 2022-09-03T06:47:42.903 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.903 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j8XWwd2iosrYFiB8K7HNRZ6BPRA= 2022-09-03T06:47:42.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j8XWwd2iosrYFiB8K7HNRZ6BPRA=', 'Content-Length': '0'} 2022-09-03T06:47:42.906 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000be87aaa17f8ac7ce-006312f88e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.907 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:42.908 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:42.908 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.909 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.909 DEBUG:boto:Method: HEAD 2022-09-03T06:47:42.910 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.910 DEBUG:boto:Data: 2022-09-03T06:47:42.911 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.911 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.912 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.913 DEBUG:boto:Params: {} 2022-09-03T06:47:42.914 DEBUG:boto:Token: None 2022-09-03T06:47:42.914 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.915 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ug/bxXVEO8jhAfjjW9VIxW8h7d8= 2022-09-03T06:47:42.915 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ug/bxXVEO8jhAfjjW9VIxW8h7d8=', 'Content-Length': '0'} 2022-09-03T06:47:42.918 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b8a9cbd8c467c255-006312f88e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.919 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:42.920 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:42.920 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.921 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.921 DEBUG:boto:Method: HEAD 2022-09-03T06:47:42.922 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.922 DEBUG:boto:Data: 2022-09-03T06:47:42.923 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.924 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.925 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.925 DEBUG:boto:Params: {} 2022-09-03T06:47:42.926 DEBUG:boto:Token: None 2022-09-03T06:47:42.927 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.927 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ug/bxXVEO8jhAfjjW9VIxW8h7d8= 2022-09-03T06:47:42.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ug/bxXVEO8jhAfjjW9VIxW8h7d8=', 'Content-Length': '0'} 2022-09-03T06:47:42.930 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d0e688e45ab3e651-006312f88e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.931 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:47:42.931 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:42.932 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:42.932 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.933 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.934 DEBUG:boto:Method: GET 2022-09-03T06:47:42.934 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.935 DEBUG:boto:Data: 2022-09-03T06:47:42.935 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.936 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:42.937 DEBUG:boto:Port: 8001 2022-09-03T06:47:42.937 DEBUG:boto:Params: {} 2022-09-03T06:47:42.938 DEBUG:boto:Token: None 2022-09-03T06:47:42.938 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.939 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j8XWwd2iosrYFiB8K7HNRZ6BPRA= 2022-09-03T06:47:42.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j8XWwd2iosrYFiB8K7HNRZ6BPRA=', 'Content-Length': '0'} 2022-09-03T06:47:42.943 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c909a22e8aca0ba4-006312f88e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:42.944 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:42.945 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:42.946 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.947 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.948 DEBUG:boto:Method: GET 2022-09-03T06:47:42.949 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.949 DEBUG:boto:Data: 2022-09-03T06:47:42.950 DEBUG:boto:Headers: {} 2022-09-03T06:47:42.950 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:42.951 DEBUG:boto:Port: 8000 2022-09-03T06:47:42.951 DEBUG:boto:Params: {} 2022-09-03T06:47:42.952 DEBUG:boto:Token: None 2022-09-03T06:47:42.953 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:42 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:47:42.954 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j8XWwd2iosrYFiB8K7HNRZ6BPRA= 2022-09-03T06:47:42.955 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:42 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j8XWwd2iosrYFiB8K7HNRZ6BPRA=', 'Content-Length': '0'} 2022-09-03T06:47:42.957 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000008cd59baa046aa16-006312f88e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:42 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.002 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:43.003 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:43.004 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:43.004 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:43.005 DEBUG:boto:Method: GET 2022-09-03T06:47:43.005 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:47:43.006 DEBUG:boto:Data: 2022-09-03T06:47:43.007 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.007 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.008 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.008 DEBUG:boto:Params: {} 2022-09-03T06:47:43.009 DEBUG:boto:Token: None 2022-09-03T06:47:43.010 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/ 2022-09-03T06:47:43.010 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0= 2022-09-03T06:47:43.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0=', 'Content-Length': '0'} 2022-09-03T06:47:43.016 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000602a5672f67b3f7a-006312f88f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.017 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:43.018 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:43.018 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:43.019 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:43.019 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:43.020 DEBUG:boto:Method: GET 2022-09-03T06:47:43.020 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:47:43.021 DEBUG:boto:Data: 2022-09-03T06:47:43.021 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.022 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.022 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.023 DEBUG:boto:Params: {} 2022-09-03T06:47:43.024 DEBUG:boto:Token: None 2022-09-03T06:47:43.025 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/ 2022-09-03T06:47:43.026 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0= 2022-09-03T06:47:43.027 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0=', 'Content-Length': '0'} 2022-09-03T06:47:43.031 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000047a12c7dae5c64d8-006312f88f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.031 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:43.032 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:43.034 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:43.035 DEBUG:boto:Method: HEAD 2022-09-03T06:47:43.035 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:47:43.037 DEBUG:boto:Data: 2022-09-03T06:47:43.038 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.039 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.040 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.040 DEBUG:boto:Params: {} 2022-09-03T06:47:43.041 DEBUG:boto:Token: None 2022-09-03T06:47:43.042 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj19 2022-09-03T06:47:43.043 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QPenADgj6jBJyuwBNnS3cBkz85o= 2022-09-03T06:47:43.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QPenADgj6jBJyuwBNnS3cBkz85o=', 'Content-Length': '0'} 2022-09-03T06:47:43.047 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b45f2c1857160ec9-006312f88f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.048 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:43.049 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:43.050 DEBUG:boto:Method: HEAD 2022-09-03T06:47:43.051 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:47:43.051 DEBUG:boto:Data: 2022-09-03T06:47:43.052 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.052 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.053 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.053 DEBUG:boto:Params: {} 2022-09-03T06:47:43.054 DEBUG:boto:Token: None 2022-09-03T06:47:43.055 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj19 2022-09-03T06:47:43.055 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QPenADgj6jBJyuwBNnS3cBkz85o= 2022-09-03T06:47:43.056 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QPenADgj6jBJyuwBNnS3cBkz85o=', 'Content-Length': '0'} 2022-09-03T06:47:43.058 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006a70594ec22eb19c-006312f88f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.059 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:47:43.060 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:43.060 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:43.061 DEBUG:boto:Method: GET 2022-09-03T06:47:43.062 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:47:43.062 DEBUG:boto:Data: 2022-09-03T06:47:43.063 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.064 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.064 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.065 DEBUG:boto:Params: {} 2022-09-03T06:47:43.066 DEBUG:boto:Token: None 2022-09-03T06:47:43.066 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj19 2022-09-03T06:47:43.067 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WKd+8eUaWlXcY5+4JIWWi/9kbKw= 2022-09-03T06:47:43.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WKd+8eUaWlXcY5+4JIWWi/9kbKw=', 'Content-Length': '0'} 2022-09-03T06:47:43.070 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001939263e89176516-006312f88f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.071 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:47:43.072 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:47:43.072 DEBUG:boto:Method: GET 2022-09-03T06:47:43.073 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:47:43.074 DEBUG:boto:Data: 2022-09-03T06:47:43.074 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.075 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.076 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.076 DEBUG:boto:Params: {} 2022-09-03T06:47:43.077 DEBUG:boto:Token: None 2022-09-03T06:47:43.078 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj19 2022-09-03T06:47:43.079 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WKd+8eUaWlXcY5+4JIWWi/9kbKw= 2022-09-03T06:47:43.079 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WKd+8eUaWlXcY5+4JIWWi/9kbKw=', 'Content-Length': '0'} 2022-09-03T06:47:43.081 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000951bfdb76ceea208-006312f88f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.122 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:43.123 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:43.123 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:43.124 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:43.125 DEBUG:boto:Method: GET 2022-09-03T06:47:43.125 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:47:43.126 DEBUG:boto:Data: 2022-09-03T06:47:43.127 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.127 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.128 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.129 DEBUG:boto:Params: {} 2022-09-03T06:47:43.129 DEBUG:boto:Token: None 2022-09-03T06:47:43.130 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/ 2022-09-03T06:47:43.131 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0= 2022-09-03T06:47:43.132 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0=', 'Content-Length': '0'} 2022-09-03T06:47:43.135 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000480890313446a22-006312f88f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.136 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:43.137 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:43.138 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:43.139 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:43.140 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:47:43.141 DEBUG:boto:Method: GET 2022-09-03T06:47:43.142 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:47:43.143 DEBUG:boto:Data: 2022-09-03T06:47:43.144 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.145 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.145 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.146 DEBUG:boto:Params: {} 2022-09-03T06:47:43.147 DEBUG:boto:Token: None 2022-09-03T06:47:43.148 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/ 2022-09-03T06:47:43.149 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0= 2022-09-03T06:47:43.150 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0=', 'Content-Length': '0'} 2022-09-03T06:47:43.153 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b988919103fa1269-006312f88f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.153 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:43.154 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:47:43.155 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:43.155 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:43.156 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.157 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.157 DEBUG:boto:Method: GET 2022-09-03T06:47:43.158 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.158 DEBUG:boto:Data: 2022-09-03T06:47:43.159 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.159 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.160 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.160 DEBUG:boto:Params: {} 2022-09-03T06:47:43.161 DEBUG:boto:Token: None 2022-09-03T06:47:43.161 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.162 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/wBwZWjBvE2uDNasrt7TFqEgFQ4= 2022-09-03T06:47:43.163 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/wBwZWjBvE2uDNasrt7TFqEgFQ4=', 'Content-Length': '0'} 2022-09-03T06:47:43.165 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008753a8b6ed86c040-006312f88f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.206 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:43.207 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:43.208 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.208 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.209 DEBUG:boto:Method: GET 2022-09-03T06:47:43.209 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.210 DEBUG:boto:Data: 2022-09-03T06:47:43.211 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.212 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.212 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.213 DEBUG:boto:Params: {} 2022-09-03T06:47:43.213 DEBUG:boto:Token: None 2022-09-03T06:47:43.214 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.215 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/wBwZWjBvE2uDNasrt7TFqEgFQ4= 2022-09-03T06:47:43.215 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/wBwZWjBvE2uDNasrt7TFqEgFQ4=', 'Content-Length': '0'} 2022-09-03T06:47:43.218 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f67fa89d1432e35c-006312f88f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.218 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:43.219 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:43.220 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.220 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.221 DEBUG:boto:Method: HEAD 2022-09-03T06:47:43.221 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.222 DEBUG:boto:Data: 2022-09-03T06:47:43.223 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.223 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.224 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.224 DEBUG:boto:Params: {} 2022-09-03T06:47:43.225 DEBUG:boto:Token: None 2022-09-03T06:47:43.226 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.226 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aptOjv9SbxvVGCs/TcqWi2iAHEo= 2022-09-03T06:47:43.227 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aptOjv9SbxvVGCs/TcqWi2iAHEo=', 'Content-Length': '0'} 2022-09-03T06:47:43.229 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001dc8a53ec88f6169-006312f88f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.230 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:43.230 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:43.231 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.231 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.233 DEBUG:boto:Method: HEAD 2022-09-03T06:47:43.233 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.234 DEBUG:boto:Data: 2022-09-03T06:47:43.234 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.235 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.235 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.236 DEBUG:boto:Params: {} 2022-09-03T06:47:43.236 DEBUG:boto:Token: None 2022-09-03T06:47:43.237 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.238 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aptOjv9SbxvVGCs/TcqWi2iAHEo= 2022-09-03T06:47:43.238 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aptOjv9SbxvVGCs/TcqWi2iAHEo=', 'Content-Length': '0'} 2022-09-03T06:47:43.240 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aabb4fa1b3e7657f-006312f88f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.241 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:47:43.242 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:43.242 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:43.243 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.244 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.244 DEBUG:boto:Method: GET 2022-09-03T06:47:43.245 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.245 DEBUG:boto:Data: 2022-09-03T06:47:43.246 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.246 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.247 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.248 DEBUG:boto:Params: {} 2022-09-03T06:47:43.248 DEBUG:boto:Token: None 2022-09-03T06:47:43.249 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.250 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/wBwZWjBvE2uDNasrt7TFqEgFQ4= 2022-09-03T06:47:43.251 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/wBwZWjBvE2uDNasrt7TFqEgFQ4=', 'Content-Length': '0'} 2022-09-03T06:47:43.253 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ce0edf8a2cb39fbd-006312f88f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.254 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:43.254 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:43.255 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.255 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.256 DEBUG:boto:Method: GET 2022-09-03T06:47:43.257 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.257 DEBUG:boto:Data: 2022-09-03T06:47:43.258 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.258 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.259 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.260 DEBUG:boto:Params: {} 2022-09-03T06:47:43.260 DEBUG:boto:Token: None 2022-09-03T06:47:43.261 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:47:43.261 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/wBwZWjBvE2uDNasrt7TFqEgFQ4= 2022-09-03T06:47:43.262 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/wBwZWjBvE2uDNasrt7TFqEgFQ4=', 'Content-Length': '0'} 2022-09-03T06:47:43.265 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000085f2918f589c9545-006312f88f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.266 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:43.267 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:43.268 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:43.269 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:43.270 DEBUG:boto:Method: GET 2022-09-03T06:47:43.271 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:47:43.271 DEBUG:boto:Data: 2022-09-03T06:47:43.272 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.273 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.274 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.275 DEBUG:boto:Params: {} 2022-09-03T06:47:43.275 DEBUG:boto:Token: None 2022-09-03T06:47:43.276 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/ 2022-09-03T06:47:43.277 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0= 2022-09-03T06:47:43.278 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0=', 'Content-Length': '0'} 2022-09-03T06:47:43.281 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002786d303eaf4b6ff-006312f88f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.282 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:43.283 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:43.283 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:43.284 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:43.284 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:43.285 DEBUG:boto:Method: GET 2022-09-03T06:47:43.285 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:47:43.286 DEBUG:boto:Data: 2022-09-03T06:47:43.286 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.287 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.287 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.288 DEBUG:boto:Params: {} 2022-09-03T06:47:43.288 DEBUG:boto:Token: None 2022-09-03T06:47:43.289 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/ 2022-09-03T06:47:43.290 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0= 2022-09-03T06:47:43.290 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0=', 'Content-Length': '0'} 2022-09-03T06:47:43.293 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a16c94eab999be0d-006312f88f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.294 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:43.295 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:43.296 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:43.296 DEBUG:boto:Method: HEAD 2022-09-03T06:47:43.297 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:47:43.297 DEBUG:boto:Data: 2022-09-03T06:47:43.298 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.298 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.299 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.299 DEBUG:boto:Params: {} 2022-09-03T06:47:43.300 DEBUG:boto:Token: None 2022-09-03T06:47:43.301 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj22 2022-09-03T06:47:43.302 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FHF5OYAJiRRLzfwOD/TfFfr7BHg= 2022-09-03T06:47:43.303 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FHF5OYAJiRRLzfwOD/TfFfr7BHg=', 'Content-Length': '0'} 2022-09-03T06:47:43.306 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e086f14cf47f9e0d-006312f88f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.307 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:43.308 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:43.309 DEBUG:boto:Method: HEAD 2022-09-03T06:47:43.310 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:47:43.311 DEBUG:boto:Data: 2022-09-03T06:47:43.312 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.313 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.314 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.315 DEBUG:boto:Params: {} 2022-09-03T06:47:43.316 DEBUG:boto:Token: None 2022-09-03T06:47:43.316 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj22 2022-09-03T06:47:43.317 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FHF5OYAJiRRLzfwOD/TfFfr7BHg= 2022-09-03T06:47:43.317 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FHF5OYAJiRRLzfwOD/TfFfr7BHg=', 'Content-Length': '0'} 2022-09-03T06:47:43.320 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d6dc7857ed533e51-006312f88f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.320 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:47:43.321 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:43.322 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:43.322 DEBUG:boto:Method: GET 2022-09-03T06:47:43.323 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:47:43.324 DEBUG:boto:Data: 2022-09-03T06:47:43.324 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.325 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.325 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.326 DEBUG:boto:Params: {} 2022-09-03T06:47:43.326 DEBUG:boto:Token: None 2022-09-03T06:47:43.327 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj22 2022-09-03T06:47:43.328 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s2N591jcqfGatD7XNaPBz9tMvSU= 2022-09-03T06:47:43.328 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s2N591jcqfGatD7XNaPBz9tMvSU=', 'Content-Length': '0'} 2022-09-03T06:47:43.330 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a7adbeac60d9236-006312f88f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.331 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:47:43.332 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:47:43.332 DEBUG:boto:Method: GET 2022-09-03T06:47:43.333 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:47:43.333 DEBUG:boto:Data: 2022-09-03T06:47:43.334 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.335 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.336 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.336 DEBUG:boto:Params: {} 2022-09-03T06:47:43.337 DEBUG:boto:Token: None 2022-09-03T06:47:43.338 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj22 2022-09-03T06:47:43.338 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s2N591jcqfGatD7XNaPBz9tMvSU= 2022-09-03T06:47:43.339 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s2N591jcqfGatD7XNaPBz9tMvSU=', 'Content-Length': '0'} 2022-09-03T06:47:43.341 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000383ad19b5a43e66c-006312f88f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.342 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:43.343 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:43.343 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:43.344 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:43.346 DEBUG:boto:Method: GET 2022-09-03T06:47:43.347 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:47:43.347 DEBUG:boto:Data: 2022-09-03T06:47:43.348 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.348 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.349 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.349 DEBUG:boto:Params: {} 2022-09-03T06:47:43.350 DEBUG:boto:Token: None 2022-09-03T06:47:43.350 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/ 2022-09-03T06:47:43.351 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0= 2022-09-03T06:47:43.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0=', 'Content-Length': '0'} 2022-09-03T06:47:43.354 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002d00272216126748-006312f88f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.355 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:43.356 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:43.356 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:43.357 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:43.357 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:47:43.358 DEBUG:boto:Method: GET 2022-09-03T06:47:43.358 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:47:43.359 DEBUG:boto:Data: 2022-09-03T06:47:43.359 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.360 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.360 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.361 DEBUG:boto:Params: {} 2022-09-03T06:47:43.361 DEBUG:boto:Token: None 2022-09-03T06:47:43.362 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/ 2022-09-03T06:47:43.363 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0= 2022-09-03T06:47:43.363 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0=', 'Content-Length': '0'} 2022-09-03T06:47:43.371 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cfe89b7520cd3491-006312f88f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.372 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:43.373 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:47:43.374 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:43.374 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:43.375 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.375 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.376 DEBUG:boto:Method: GET 2022-09-03T06:47:43.376 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.377 DEBUG:boto:Data: 2022-09-03T06:47:43.377 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.378 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.379 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.379 DEBUG:boto:Params: {} 2022-09-03T06:47:43.380 DEBUG:boto:Token: None 2022-09-03T06:47:43.380 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.381 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kxzdVUMM7dIzrl9+X72zpbPFaoY= 2022-09-03T06:47:43.382 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kxzdVUMM7dIzrl9+X72zpbPFaoY=', 'Content-Length': '0'} 2022-09-03T06:47:43.384 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000b47c6557d3d626f-006312f88f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.430 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:43.431 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:43.432 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.433 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.434 DEBUG:boto:Method: GET 2022-09-03T06:47:43.435 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.436 DEBUG:boto:Data: 2022-09-03T06:47:43.437 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.437 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.438 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.439 DEBUG:boto:Params: {} 2022-09-03T06:47:43.440 DEBUG:boto:Token: None 2022-09-03T06:47:43.441 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.442 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kxzdVUMM7dIzrl9+X72zpbPFaoY= 2022-09-03T06:47:43.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kxzdVUMM7dIzrl9+X72zpbPFaoY=', 'Content-Length': '0'} 2022-09-03T06:47:43.445 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a346ad885d45aab-006312f88f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.446 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:43.446 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:43.447 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.447 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.448 DEBUG:boto:Method: HEAD 2022-09-03T06:47:43.448 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.449 DEBUG:boto:Data: 2022-09-03T06:47:43.450 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.451 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.452 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.453 DEBUG:boto:Params: {} 2022-09-03T06:47:43.454 DEBUG:boto:Token: None 2022-09-03T06:47:43.455 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.456 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LdPdEIR7U0qc2hSTP6d6uqaxMZQ= 2022-09-03T06:47:43.457 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LdPdEIR7U0qc2hSTP6d6uqaxMZQ=', 'Content-Length': '0'} 2022-09-03T06:47:43.460 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000502b1b483b6d13a3-006312f88f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.461 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:43.462 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:43.462 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.463 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.463 DEBUG:boto:Method: HEAD 2022-09-03T06:47:43.464 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.464 DEBUG:boto:Data: 2022-09-03T06:47:43.465 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.465 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.466 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.467 DEBUG:boto:Params: {} 2022-09-03T06:47:43.467 DEBUG:boto:Token: None 2022-09-03T06:47:43.468 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.468 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LdPdEIR7U0qc2hSTP6d6uqaxMZQ= 2022-09-03T06:47:43.469 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LdPdEIR7U0qc2hSTP6d6uqaxMZQ=', 'Content-Length': '0'} 2022-09-03T06:47:43.471 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b33b40032106253-006312f88f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.472 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:47:43.472 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:43.473 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:43.473 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.474 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.475 DEBUG:boto:Method: GET 2022-09-03T06:47:43.475 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.476 DEBUG:boto:Data: 2022-09-03T06:47:43.476 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.477 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.477 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.478 DEBUG:boto:Params: {} 2022-09-03T06:47:43.478 DEBUG:boto:Token: None 2022-09-03T06:47:43.479 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.480 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kxzdVUMM7dIzrl9+X72zpbPFaoY= 2022-09-03T06:47:43.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kxzdVUMM7dIzrl9+X72zpbPFaoY=', 'Content-Length': '0'} 2022-09-03T06:47:43.482 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006a705d86c45891cc-006312f88f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.483 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:43.484 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:43.484 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.485 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.485 DEBUG:boto:Method: GET 2022-09-03T06:47:43.486 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.486 DEBUG:boto:Data: 2022-09-03T06:47:43.487 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.488 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.488 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.489 DEBUG:boto:Params: {} 2022-09-03T06:47:43.489 DEBUG:boto:Token: None 2022-09-03T06:47:43.490 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:47:43.490 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kxzdVUMM7dIzrl9+X72zpbPFaoY= 2022-09-03T06:47:43.491 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kxzdVUMM7dIzrl9+X72zpbPFaoY=', 'Content-Length': '0'} 2022-09-03T06:47:43.493 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d944314f32f55ed4-006312f88f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.494 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:43.495 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:43.495 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:43.496 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:43.497 DEBUG:boto:Method: GET 2022-09-03T06:47:43.497 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:47:43.498 DEBUG:boto:Data: 2022-09-03T06:47:43.499 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.500 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.501 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.502 DEBUG:boto:Params: {} 2022-09-03T06:47:43.503 DEBUG:boto:Token: None 2022-09-03T06:47:43.504 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/ 2022-09-03T06:47:43.505 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0= 2022-09-03T06:47:43.507 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0=', 'Content-Length': '0'} 2022-09-03T06:47:43.510 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cf34b6a95eb8ee86-006312f88f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.511 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:43.513 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:43.514 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:43.515 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:43.515 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:43.516 DEBUG:boto:Method: GET 2022-09-03T06:47:43.517 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:47:43.518 DEBUG:boto:Data: 2022-09-03T06:47:43.519 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.521 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.522 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.522 DEBUG:boto:Params: {} 2022-09-03T06:47:43.524 DEBUG:boto:Token: None 2022-09-03T06:47:43.524 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/ 2022-09-03T06:47:43.525 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0= 2022-09-03T06:47:43.525 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0=', 'Content-Length': '0'} 2022-09-03T06:47:43.528 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001245d27202df7f58-006312f88f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.529 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:43.530 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:43.530 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:43.531 DEBUG:boto:Method: HEAD 2022-09-03T06:47:43.531 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:47:43.532 DEBUG:boto:Data: 2022-09-03T06:47:43.532 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.533 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.533 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.534 DEBUG:boto:Params: {} 2022-09-03T06:47:43.534 DEBUG:boto:Token: None 2022-09-03T06:47:43.535 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj23 2022-09-03T06:47:43.536 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VrCPjbNhzl4qx47ndMnASnVv1hg= 2022-09-03T06:47:43.536 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VrCPjbNhzl4qx47ndMnASnVv1hg=', 'Content-Length': '0'} 2022-09-03T06:47:43.538 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006e83776d381f061b-006312f88f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.539 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:43.540 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:43.541 DEBUG:boto:Method: HEAD 2022-09-03T06:47:43.542 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:47:43.543 DEBUG:boto:Data: 2022-09-03T06:47:43.544 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.545 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.546 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.546 DEBUG:boto:Params: {} 2022-09-03T06:47:43.547 DEBUG:boto:Token: None 2022-09-03T06:47:43.548 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj23 2022-09-03T06:47:43.549 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VrCPjbNhzl4qx47ndMnASnVv1hg= 2022-09-03T06:47:43.550 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VrCPjbNhzl4qx47ndMnASnVv1hg=', 'Content-Length': '0'} 2022-09-03T06:47:43.552 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005646ed7b3c08374b-006312f88f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.553 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:47:43.553 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:43.554 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:43.555 DEBUG:boto:Method: GET 2022-09-03T06:47:43.555 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:47:43.556 DEBUG:boto:Data: 2022-09-03T06:47:43.557 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.558 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.558 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.559 DEBUG:boto:Params: {} 2022-09-03T06:47:43.560 DEBUG:boto:Token: None 2022-09-03T06:47:43.561 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj23 2022-09-03T06:47:43.562 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XGNovLd+ju8FdqpqgpU/DqXyYBQ= 2022-09-03T06:47:43.562 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XGNovLd+ju8FdqpqgpU/DqXyYBQ=', 'Content-Length': '0'} 2022-09-03T06:47:43.564 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000071b0328606cae997-006312f88f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.610 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:47:43.611 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:47:43.612 DEBUG:boto:Method: GET 2022-09-03T06:47:43.613 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:47:43.614 DEBUG:boto:Data: 2022-09-03T06:47:43.615 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.615 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.616 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.617 DEBUG:boto:Params: {} 2022-09-03T06:47:43.619 DEBUG:boto:Token: None 2022-09-03T06:47:43.620 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj23 2022-09-03T06:47:43.621 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XGNovLd+ju8FdqpqgpU/DqXyYBQ= 2022-09-03T06:47:43.622 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XGNovLd+ju8FdqpqgpU/DqXyYBQ=', 'Content-Length': '0'} 2022-09-03T06:47:43.624 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000da66db9bc41fbf9b-006312f88f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.626 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:43.627 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:43.628 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:43.629 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:43.629 DEBUG:boto:Method: GET 2022-09-03T06:47:43.630 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:47:43.631 DEBUG:boto:Data: 2022-09-03T06:47:43.632 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.633 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.634 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.635 DEBUG:boto:Params: {} 2022-09-03T06:47:43.636 DEBUG:boto:Token: None 2022-09-03T06:47:43.637 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/ 2022-09-03T06:47:43.638 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0= 2022-09-03T06:47:43.639 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0=', 'Content-Length': '0'} 2022-09-03T06:47:43.642 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001b69f9bb276d8078-006312f88f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.643 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:43.644 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:43.645 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:43.645 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:43.646 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:47:43.646 DEBUG:boto:Method: GET 2022-09-03T06:47:43.647 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:47:43.647 DEBUG:boto:Data: 2022-09-03T06:47:43.648 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.649 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.650 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.650 DEBUG:boto:Params: {} 2022-09-03T06:47:43.651 DEBUG:boto:Token: None 2022-09-03T06:47:43.652 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/ 2022-09-03T06:47:43.652 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0= 2022-09-03T06:47:43.653 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0=', 'Content-Length': '0'} 2022-09-03T06:47:43.656 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cc4df7647425529a-006312f88f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.657 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:43.658 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:47:43.658 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:43.659 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:43.659 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.660 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.660 DEBUG:boto:Method: GET 2022-09-03T06:47:43.661 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.661 DEBUG:boto:Data: 2022-09-03T06:47:43.662 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.662 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.663 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.664 DEBUG:boto:Params: {} 2022-09-03T06:47:43.665 DEBUG:boto:Token: None 2022-09-03T06:47:43.666 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.667 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mIWOS6QBENiQnxTIs0crmrhVu4s= 2022-09-03T06:47:43.668 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mIWOS6QBENiQnxTIs0crmrhVu4s=', 'Content-Length': '0'} 2022-09-03T06:47:43.670 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004676acc798b9570a-006312f88f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.672 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:43.673 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:43.674 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.675 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.675 DEBUG:boto:Method: GET 2022-09-03T06:47:43.676 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.677 DEBUG:boto:Data: 2022-09-03T06:47:43.678 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.679 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.680 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.681 DEBUG:boto:Params: {} 2022-09-03T06:47:43.681 DEBUG:boto:Token: None 2022-09-03T06:47:43.682 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.682 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mIWOS6QBENiQnxTIs0crmrhVu4s= 2022-09-03T06:47:43.683 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mIWOS6QBENiQnxTIs0crmrhVu4s=', 'Content-Length': '0'} 2022-09-03T06:47:43.685 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008f0b1f178a22618f-006312f88f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.686 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:43.687 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:43.687 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.688 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.688 DEBUG:boto:Method: HEAD 2022-09-03T06:47:43.689 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.689 DEBUG:boto:Data: 2022-09-03T06:47:43.690 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.690 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.691 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.692 DEBUG:boto:Params: {} 2022-09-03T06:47:43.693 DEBUG:boto:Token: None 2022-09-03T06:47:43.694 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.695 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h7u9szCC4/lGlAgGUBapyk494Rs= 2022-09-03T06:47:43.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h7u9szCC4/lGlAgGUBapyk494Rs=', 'Content-Length': '0'} 2022-09-03T06:47:43.699 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000efcb3cae50d09e0b-006312f88f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.700 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:43.701 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:43.702 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.703 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.704 DEBUG:boto:Method: HEAD 2022-09-03T06:47:43.704 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.705 DEBUG:boto:Data: 2022-09-03T06:47:43.706 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.707 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.708 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.709 DEBUG:boto:Params: {} 2022-09-03T06:47:43.710 DEBUG:boto:Token: None 2022-09-03T06:47:43.711 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.712 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h7u9szCC4/lGlAgGUBapyk494Rs= 2022-09-03T06:47:43.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h7u9szCC4/lGlAgGUBapyk494Rs=', 'Content-Length': '0'} 2022-09-03T06:47:43.715 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009fbbc86dc3daa92f-006312f88f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.716 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:47:43.717 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:43.717 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:43.718 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.718 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.719 DEBUG:boto:Method: GET 2022-09-03T06:47:43.719 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.720 DEBUG:boto:Data: 2022-09-03T06:47:43.720 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.721 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.721 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.722 DEBUG:boto:Params: {} 2022-09-03T06:47:43.723 DEBUG:boto:Token: None 2022-09-03T06:47:43.723 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.724 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mIWOS6QBENiQnxTIs0crmrhVu4s= 2022-09-03T06:47:43.724 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mIWOS6QBENiQnxTIs0crmrhVu4s=', 'Content-Length': '0'} 2022-09-03T06:47:43.730 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004637879eb6749def-006312f88f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.770 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:43.772 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:43.772 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.773 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.774 DEBUG:boto:Method: GET 2022-09-03T06:47:43.775 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.776 DEBUG:boto:Data: 2022-09-03T06:47:43.776 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.777 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.777 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.778 DEBUG:boto:Params: {} 2022-09-03T06:47:43.779 DEBUG:boto:Token: None 2022-09-03T06:47:43.780 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:47:43.781 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mIWOS6QBENiQnxTIs0crmrhVu4s= 2022-09-03T06:47:43.782 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mIWOS6QBENiQnxTIs0crmrhVu4s=', 'Content-Length': '0'} 2022-09-03T06:47:43.784 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008f9e0e7fe51e0fdb-006312f88f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.785 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:43.786 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:43.786 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:43.787 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:43.788 DEBUG:boto:Method: GET 2022-09-03T06:47:43.789 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:47:43.790 DEBUG:boto:Data: 2022-09-03T06:47:43.791 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.792 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.793 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.793 DEBUG:boto:Params: {} 2022-09-03T06:47:43.794 DEBUG:boto:Token: None 2022-09-03T06:47:43.795 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/ 2022-09-03T06:47:43.796 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0= 2022-09-03T06:47:43.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0=', 'Content-Length': '0'} 2022-09-03T06:47:43.801 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dbfebac371377928-006312f88f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.802 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:43.804 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:43.805 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:43.806 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:43.807 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:43.808 DEBUG:boto:Method: GET 2022-09-03T06:47:43.809 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:47:43.810 DEBUG:boto:Data: 2022-09-03T06:47:43.811 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.812 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.813 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.814 DEBUG:boto:Params: {} 2022-09-03T06:47:43.814 DEBUG:boto:Token: None 2022-09-03T06:47:43.815 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/ 2022-09-03T06:47:43.816 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0= 2022-09-03T06:47:43.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0=', 'Content-Length': '0'} 2022-09-03T06:47:43.820 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a19a849cc8891db0-006312f88f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.821 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:43.822 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:43.823 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:43.824 DEBUG:boto:Method: HEAD 2022-09-03T06:47:43.825 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:47:43.825 DEBUG:boto:Data: 2022-09-03T06:47:43.826 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.827 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.827 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.828 DEBUG:boto:Params: {} 2022-09-03T06:47:43.828 DEBUG:boto:Token: None 2022-09-03T06:47:43.829 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj29 2022-09-03T06:47:43.830 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lFUweKhnq/HKEmFq95dRNUwQcOY= 2022-09-03T06:47:43.830 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lFUweKhnq/HKEmFq95dRNUwQcOY=', 'Content-Length': '0'} 2022-09-03T06:47:43.833 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b8a36f9a015cdb18-006312f88f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.834 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:43.835 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:43.836 DEBUG:boto:Method: HEAD 2022-09-03T06:47:43.837 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:47:43.837 DEBUG:boto:Data: 2022-09-03T06:47:43.838 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.839 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.840 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.840 DEBUG:boto:Params: {} 2022-09-03T06:47:43.841 DEBUG:boto:Token: None 2022-09-03T06:47:43.841 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj29 2022-09-03T06:47:43.842 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lFUweKhnq/HKEmFq95dRNUwQcOY= 2022-09-03T06:47:43.843 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lFUweKhnq/HKEmFq95dRNUwQcOY=', 'Content-Length': '0'} 2022-09-03T06:47:43.846 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006a5eaeb932c8e1d3-006312f88f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.847 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:47:43.847 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:43.848 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:43.848 DEBUG:boto:Method: GET 2022-09-03T06:47:43.849 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:47:43.850 DEBUG:boto:Data: 2022-09-03T06:47:43.851 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.852 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.853 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.855 DEBUG:boto:Params: {} 2022-09-03T06:47:43.856 DEBUG:boto:Token: None 2022-09-03T06:47:43.857 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj29 2022-09-03T06:47:43.858 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fWfm3CCgqvNy2zOq+HXHsiSIIe8= 2022-09-03T06:47:43.859 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fWfm3CCgqvNy2zOq+HXHsiSIIe8=', 'Content-Length': '0'} 2022-09-03T06:47:43.862 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000efff16b4e85ef7a1-006312f88f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.863 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:47:43.863 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:47:43.864 DEBUG:boto:Method: GET 2022-09-03T06:47:43.864 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:47:43.865 DEBUG:boto:Data: 2022-09-03T06:47:43.866 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.866 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.867 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.867 DEBUG:boto:Params: {} 2022-09-03T06:47:43.868 DEBUG:boto:Token: None 2022-09-03T06:47:43.869 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj29 2022-09-03T06:47:43.869 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fWfm3CCgqvNy2zOq+HXHsiSIIe8= 2022-09-03T06:47:43.871 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fWfm3CCgqvNy2zOq+HXHsiSIIe8=', 'Content-Length': '0'} 2022-09-03T06:47:43.874 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cc3936becf90f22e-006312f88f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.875 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:43.876 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:43.877 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:43.878 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:43.879 DEBUG:boto:Method: GET 2022-09-03T06:47:43.879 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:47:43.880 DEBUG:boto:Data: 2022-09-03T06:47:43.880 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.881 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.882 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.882 DEBUG:boto:Params: {} 2022-09-03T06:47:43.883 DEBUG:boto:Token: None 2022-09-03T06:47:43.884 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/ 2022-09-03T06:47:43.885 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0= 2022-09-03T06:47:43.886 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0=', 'Content-Length': '0'} 2022-09-03T06:47:43.888 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000014396a11b9e03599-006312f88f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.890 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:43.891 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:43.892 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:43.893 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:43.894 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:47:43.895 DEBUG:boto:Method: GET 2022-09-03T06:47:43.896 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:47:43.897 DEBUG:boto:Data: 2022-09-03T06:47:43.898 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.898 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.905 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.906 DEBUG:boto:Params: {} 2022-09-03T06:47:43.908 DEBUG:boto:Token: None 2022-09-03T06:47:43.908 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/ 2022-09-03T06:47:43.909 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0= 2022-09-03T06:47:43.910 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hTSSE+uWVC7ZpwxoF0anIhQhGe0=', 'Content-Length': '0'} 2022-09-03T06:47:43.913 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004b8d43c92086c731-006312f88f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.914 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:43.915 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2022-09-03T06:47:43.916 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:43.917 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:43.918 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:43.919 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:43.920 DEBUG:boto:Method: GET 2022-09-03T06:47:43.920 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:43.921 DEBUG:boto:Data: 2022-09-03T06:47:43.922 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.923 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.924 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.925 DEBUG:boto:Params: {} 2022-09-03T06:47:43.926 DEBUG:boto:Token: None 2022-09-03T06:47:43.927 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:43.928 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q8CkhRQcA8s/j82JCL8CGrt+cpE= 2022-09-03T06:47:43.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q8CkhRQcA8s/j82JCL8CGrt+cpE=', 'Content-Length': '0'} 2022-09-03T06:47:43.932 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fe81fb1e15170983-006312f88f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.934 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:43.935 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:43.936 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:43.937 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:43.938 DEBUG:boto:Method: GET 2022-09-03T06:47:43.939 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:43.940 DEBUG:boto:Data: 2022-09-03T06:47:43.941 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.942 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.943 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.944 DEBUG:boto:Params: {} 2022-09-03T06:47:43.945 DEBUG:boto:Token: None 2022-09-03T06:47:43.946 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:43.947 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q8CkhRQcA8s/j82JCL8CGrt+cpE= 2022-09-03T06:47:43.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q8CkhRQcA8s/j82JCL8CGrt+cpE=', 'Content-Length': '0'} 2022-09-03T06:47:43.951 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cc62ec85b3a41139-006312f88f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.952 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:43.953 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:43.954 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:43.955 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:43.956 DEBUG:boto:Method: HEAD 2022-09-03T06:47:43.957 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:43.962 DEBUG:boto:Data: 2022-09-03T06:47:43.963 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.964 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.965 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.965 DEBUG:boto:Params: {} 2022-09-03T06:47:43.966 DEBUG:boto:Token: None 2022-09-03T06:47:43.967 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:43.968 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5Jp1JgpoQGWFZqqDz/eX1TOao5A= 2022-09-03T06:47:43.969 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5Jp1JgpoQGWFZqqDz/eX1TOao5A=', 'Content-Length': '0'} 2022-09-03T06:47:43.971 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004045f41e60253331-006312f88f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.972 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:43.973 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:43.973 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:43.974 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:43.974 DEBUG:boto:Method: HEAD 2022-09-03T06:47:43.975 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:43.975 DEBUG:boto:Data: 2022-09-03T06:47:43.976 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.976 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:43.977 DEBUG:boto:Port: 8000 2022-09-03T06:47:43.977 DEBUG:boto:Params: {} 2022-09-03T06:47:43.978 DEBUG:boto:Token: None 2022-09-03T06:47:43.978 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:43.979 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5Jp1JgpoQGWFZqqDz/eX1TOao5A= 2022-09-03T06:47:43.980 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5Jp1JgpoQGWFZqqDz/eX1TOao5A=', 'Content-Length': '0'} 2022-09-03T06:47:43.982 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006af73ae86e9db6ab-006312f88f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.983 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2022-09-03T06:47:43.983 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:43.984 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:43.984 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:43.985 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:43.985 DEBUG:boto:Method: GET 2022-09-03T06:47:43.986 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:43.986 DEBUG:boto:Data: 2022-09-03T06:47:43.987 DEBUG:boto:Headers: {} 2022-09-03T06:47:43.987 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:43.988 DEBUG:boto:Port: 8001 2022-09-03T06:47:43.989 DEBUG:boto:Params: {} 2022-09-03T06:47:43.989 DEBUG:boto:Token: None 2022-09-03T06:47:43.990 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:43 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:43.990 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q8CkhRQcA8s/j82JCL8CGrt+cpE= 2022-09-03T06:47:43.991 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:43 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q8CkhRQcA8s/j82JCL8CGrt+cpE=', 'Content-Length': '0'} 2022-09-03T06:47:43.993 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002bbe025ee023fca0-006312f88f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:43 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:43.994 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:43.995 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:43.996 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:43.996 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:43.998 DEBUG:boto:Method: GET 2022-09-03T06:47:43.999 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:43.999 DEBUG:boto:Data: 2022-09-03T06:47:44.000 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.001 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.002 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.002 DEBUG:boto:Params: {} 2022-09-03T06:47:44.004 DEBUG:boto:Token: None 2022-09-03T06:47:44.004 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:47:44.005 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BjKB46qSs9OEXl2rX5BFuBE/N+U= 2022-09-03T06:47:44.006 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BjKB46qSs9OEXl2rX5BFuBE/N+U=', 'Content-Length': '0'} 2022-09-03T06:47:44.009 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a074b1821893875a-006312f890-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.011 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:44.012 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:44.013 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:44.014 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:44.015 DEBUG:boto:Method: GET 2022-09-03T06:47:44.016 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:47:44.017 DEBUG:boto:Data: 2022-09-03T06:47:44.018 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.018 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.019 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.020 DEBUG:boto:Params: {} 2022-09-03T06:47:44.021 DEBUG:boto:Token: None 2022-09-03T06:47:44.022 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/ 2022-09-03T06:47:44.023 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU= 2022-09-03T06:47:44.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU=', 'Content-Length': '0'} 2022-09-03T06:47:44.028 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001dd33b4ab666e4ec-006312f890-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.029 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:44.031 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:44.032 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:44.033 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:44.033 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:44.034 DEBUG:boto:Method: GET 2022-09-03T06:47:44.035 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:47:44.036 DEBUG:boto:Data: 2022-09-03T06:47:44.038 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.039 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.040 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.041 DEBUG:boto:Params: {} 2022-09-03T06:47:44.042 DEBUG:boto:Token: None 2022-09-03T06:47:44.043 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/ 2022-09-03T06:47:44.044 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU= 2022-09-03T06:47:44.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU=', 'Content-Length': '0'} 2022-09-03T06:47:44.047 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000923bb4c3d2c4ee32-006312f890-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.048 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:44.049 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:44.049 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:44.050 DEBUG:boto:Method: HEAD 2022-09-03T06:47:44.051 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:47:44.052 DEBUG:boto:Data: 2022-09-03T06:47:44.053 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.054 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.055 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.056 DEBUG:boto:Params: {} 2022-09-03T06:47:44.057 DEBUG:boto:Token: None 2022-09-03T06:47:44.058 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj31 2022-09-03T06:47:44.059 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ieg4wzfsJCkpAEcweSzdHIq4sy0= 2022-09-03T06:47:44.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ieg4wzfsJCkpAEcweSzdHIq4sy0=', 'Content-Length': '0'} 2022-09-03T06:47:44.063 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cc583e6fb9787cd2-006312f890-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.064 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:44.066 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:44.067 DEBUG:boto:Method: HEAD 2022-09-03T06:47:44.067 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:47:44.068 DEBUG:boto:Data: 2022-09-03T06:47:44.069 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.070 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.071 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.072 DEBUG:boto:Params: {} 2022-09-03T06:47:44.073 DEBUG:boto:Token: None 2022-09-03T06:47:44.074 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj31 2022-09-03T06:47:44.075 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ieg4wzfsJCkpAEcweSzdHIq4sy0= 2022-09-03T06:47:44.076 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ieg4wzfsJCkpAEcweSzdHIq4sy0=', 'Content-Length': '0'} 2022-09-03T06:47:44.079 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b3599e18b3d14362-006312f890-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.080 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2022-09-03T06:47:44.081 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:44.082 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:44.083 DEBUG:boto:Method: GET 2022-09-03T06:47:44.084 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:47:44.085 DEBUG:boto:Data: 2022-09-03T06:47:44.085 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.086 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.086 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.087 DEBUG:boto:Params: {} 2022-09-03T06:47:44.087 DEBUG:boto:Token: None 2022-09-03T06:47:44.088 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj31 2022-09-03T06:47:44.089 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Pji3pRPLAdeZibKqN3O3xOnGkP4= 2022-09-03T06:47:44.090 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Pji3pRPLAdeZibKqN3O3xOnGkP4=', 'Content-Length': '0'} 2022-09-03T06:47:44.093 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ea3126e6d9213621-006312f890-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.094 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:47:44.095 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:47:44.096 DEBUG:boto:Method: GET 2022-09-03T06:47:44.097 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:47:44.098 DEBUG:boto:Data: 2022-09-03T06:47:44.100 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.101 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.102 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.103 DEBUG:boto:Params: {} 2022-09-03T06:47:44.104 DEBUG:boto:Token: None 2022-09-03T06:47:44.104 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj31 2022-09-03T06:47:44.106 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Pji3pRPLAdeZibKqN3O3xOnGkP4= 2022-09-03T06:47:44.107 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Pji3pRPLAdeZibKqN3O3xOnGkP4=', 'Content-Length': '0'} 2022-09-03T06:47:44.109 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b618fc906c79bb3e-006312f890-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.150 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:44.151 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:44.151 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:44.152 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:44.153 DEBUG:boto:Method: GET 2022-09-03T06:47:44.153 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:47:44.154 DEBUG:boto:Data: 2022-09-03T06:47:44.155 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.155 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.156 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.156 DEBUG:boto:Params: {} 2022-09-03T06:47:44.157 DEBUG:boto:Token: None 2022-09-03T06:47:44.158 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/ 2022-09-03T06:47:44.158 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU= 2022-09-03T06:47:44.159 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU=', 'Content-Length': '0'} 2022-09-03T06:47:44.162 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000adcee2ce9bdffaa9-006312f890-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.163 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:44.164 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:44.165 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:44.165 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:44.166 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:47:44.166 DEBUG:boto:Method: GET 2022-09-03T06:47:44.167 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:47:44.167 DEBUG:boto:Data: 2022-09-03T06:47:44.168 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.169 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.170 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.170 DEBUG:boto:Params: {} 2022-09-03T06:47:44.171 DEBUG:boto:Token: None 2022-09-03T06:47:44.172 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/ 2022-09-03T06:47:44.172 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU= 2022-09-03T06:47:44.173 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU=', 'Content-Length': '0'} 2022-09-03T06:47:44.176 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000005775828a69c7c76-006312f890-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.177 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:44.179 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:47:44.179 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:44.180 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:44.181 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.181 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.182 DEBUG:boto:Method: GET 2022-09-03T06:47:44.182 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.183 DEBUG:boto:Data: 2022-09-03T06:47:44.184 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.184 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.185 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.185 DEBUG:boto:Params: {} 2022-09-03T06:47:44.186 DEBUG:boto:Token: None 2022-09-03T06:47:44.187 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.187 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JtWtJEzH55nFwbqLIXt9sDl+k64= 2022-09-03T06:47:44.188 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JtWtJEzH55nFwbqLIXt9sDl+k64=', 'Content-Length': '0'} 2022-09-03T06:47:44.190 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007eb674d5fe891213-006312f890-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.234 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:44.235 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:44.235 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.236 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.237 DEBUG:boto:Method: GET 2022-09-03T06:47:44.237 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.238 DEBUG:boto:Data: 2022-09-03T06:47:44.238 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.239 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.239 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.240 DEBUG:boto:Params: {} 2022-09-03T06:47:44.241 DEBUG:boto:Token: None 2022-09-03T06:47:44.241 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.242 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JtWtJEzH55nFwbqLIXt9sDl+k64= 2022-09-03T06:47:44.243 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JtWtJEzH55nFwbqLIXt9sDl+k64=', 'Content-Length': '0'} 2022-09-03T06:47:44.245 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b46d46f212ed45c-006312f890-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.246 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:44.247 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:44.248 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.249 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.250 DEBUG:boto:Method: HEAD 2022-09-03T06:47:44.251 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.251 DEBUG:boto:Data: 2022-09-03T06:47:44.252 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.253 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.254 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.255 DEBUG:boto:Params: {} 2022-09-03T06:47:44.256 DEBUG:boto:Token: None 2022-09-03T06:47:44.257 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.259 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eeP0cLY5QkY1gwBTaH6q4qnyNJg= 2022-09-03T06:47:44.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eeP0cLY5QkY1gwBTaH6q4qnyNJg=', 'Content-Length': '0'} 2022-09-03T06:47:44.263 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000033e2bfd187b45f93-006312f890-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.264 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:44.264 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:44.265 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.265 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.266 DEBUG:boto:Method: HEAD 2022-09-03T06:47:44.267 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.268 DEBUG:boto:Data: 2022-09-03T06:47:44.268 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.269 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.270 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.271 DEBUG:boto:Params: {} 2022-09-03T06:47:44.272 DEBUG:boto:Token: None 2022-09-03T06:47:44.272 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.273 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eeP0cLY5QkY1gwBTaH6q4qnyNJg= 2022-09-03T06:47:44.274 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eeP0cLY5QkY1gwBTaH6q4qnyNJg=', 'Content-Length': '0'} 2022-09-03T06:47:44.277 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000097762b154036db9e-006312f890-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.277 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:47:44.278 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:44.279 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:44.279 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.280 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.281 DEBUG:boto:Method: GET 2022-09-03T06:47:44.282 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.283 DEBUG:boto:Data: 2022-09-03T06:47:44.284 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.284 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.285 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.285 DEBUG:boto:Params: {} 2022-09-03T06:47:44.286 DEBUG:boto:Token: None 2022-09-03T06:47:44.287 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.288 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JtWtJEzH55nFwbqLIXt9sDl+k64= 2022-09-03T06:47:44.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JtWtJEzH55nFwbqLIXt9sDl+k64=', 'Content-Length': '0'} 2022-09-03T06:47:44.290 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e8b7f72b68ac75eb-006312f890-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.292 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:44.293 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:44.293 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.294 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.295 DEBUG:boto:Method: GET 2022-09-03T06:47:44.295 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.296 DEBUG:boto:Data: 2022-09-03T06:47:44.296 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.297 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.298 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.299 DEBUG:boto:Params: {} 2022-09-03T06:47:44.300 DEBUG:boto:Token: None 2022-09-03T06:47:44.301 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:47:44.301 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JtWtJEzH55nFwbqLIXt9sDl+k64= 2022-09-03T06:47:44.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JtWtJEzH55nFwbqLIXt9sDl+k64=', 'Content-Length': '0'} 2022-09-03T06:47:44.305 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d6bf085bd4a85c0-006312f890-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.307 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:44.307 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:44.308 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:44.309 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:44.310 DEBUG:boto:Method: GET 2022-09-03T06:47:44.311 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:47:44.312 DEBUG:boto:Data: 2022-09-03T06:47:44.313 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.314 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.315 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.316 DEBUG:boto:Params: {} 2022-09-03T06:47:44.317 DEBUG:boto:Token: None 2022-09-03T06:47:44.318 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/ 2022-09-03T06:47:44.319 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU= 2022-09-03T06:47:44.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU=', 'Content-Length': '0'} 2022-09-03T06:47:44.323 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009fa151b6b174b32e-006312f890-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.366 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:44.368 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:44.369 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:44.370 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:44.371 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:44.371 DEBUG:boto:Method: GET 2022-09-03T06:47:44.372 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:47:44.372 DEBUG:boto:Data: 2022-09-03T06:47:44.373 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.374 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.374 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.375 DEBUG:boto:Params: {} 2022-09-03T06:47:44.375 DEBUG:boto:Token: None 2022-09-03T06:47:44.376 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/ 2022-09-03T06:47:44.376 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU= 2022-09-03T06:47:44.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU=', 'Content-Length': '0'} 2022-09-03T06:47:44.380 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002db5a7f4ab755c5b-006312f890-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.382 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:44.383 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:44.384 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:44.385 DEBUG:boto:Method: HEAD 2022-09-03T06:47:44.386 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:47:44.386 DEBUG:boto:Data: 2022-09-03T06:47:44.387 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.387 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.388 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.389 DEBUG:boto:Params: {} 2022-09-03T06:47:44.389 DEBUG:boto:Token: None 2022-09-03T06:47:44.390 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj33 2022-09-03T06:47:44.390 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/hw86yvaJcVAVwVwcCiOo8f1Zg= 2022-09-03T06:47:44.391 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/hw86yvaJcVAVwVwcCiOo8f1Zg=', 'Content-Length': '0'} 2022-09-03T06:47:44.393 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db9eac4c2e0c6560-006312f890-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.394 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:44.395 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:44.395 DEBUG:boto:Method: HEAD 2022-09-03T06:47:44.396 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:47:44.396 DEBUG:boto:Data: 2022-09-03T06:47:44.397 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.397 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.398 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.398 DEBUG:boto:Params: {} 2022-09-03T06:47:44.399 DEBUG:boto:Token: None 2022-09-03T06:47:44.400 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj33 2022-09-03T06:47:44.401 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3/hw86yvaJcVAVwVwcCiOo8f1Zg= 2022-09-03T06:47:44.401 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3/hw86yvaJcVAVwVwcCiOo8f1Zg=', 'Content-Length': '0'} 2022-09-03T06:47:44.404 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ab686d9c7c4981bf-006312f890-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.405 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:47:44.406 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:44.407 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:44.408 DEBUG:boto:Method: GET 2022-09-03T06:47:44.409 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:47:44.410 DEBUG:boto:Data: 2022-09-03T06:47:44.410 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.411 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.412 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.413 DEBUG:boto:Params: {} 2022-09-03T06:47:44.414 DEBUG:boto:Token: None 2022-09-03T06:47:44.415 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj33 2022-09-03T06:47:44.416 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5BdrdjxJpaPNcff5XEvH0uToS5A= 2022-09-03T06:47:44.417 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5BdrdjxJpaPNcff5XEvH0uToS5A=', 'Content-Length': '0'} 2022-09-03T06:47:44.419 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000017fb23b5f1decedb-006312f890-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.420 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:47:44.421 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:47:44.422 DEBUG:boto:Method: GET 2022-09-03T06:47:44.423 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:47:44.424 DEBUG:boto:Data: 2022-09-03T06:47:44.425 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.426 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.427 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.428 DEBUG:boto:Params: {} 2022-09-03T06:47:44.429 DEBUG:boto:Token: None 2022-09-03T06:47:44.430 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj33 2022-09-03T06:47:44.431 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5BdrdjxJpaPNcff5XEvH0uToS5A= 2022-09-03T06:47:44.433 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5BdrdjxJpaPNcff5XEvH0uToS5A=', 'Content-Length': '0'} 2022-09-03T06:47:44.436 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b396cdb014f4ef1-006312f890-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.437 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:44.438 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:44.439 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:44.440 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:44.441 DEBUG:boto:Method: GET 2022-09-03T06:47:44.442 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:47:44.443 DEBUG:boto:Data: 2022-09-03T06:47:44.444 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.445 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.446 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.447 DEBUG:boto:Params: {} 2022-09-03T06:47:44.448 DEBUG:boto:Token: None 2022-09-03T06:47:44.448 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/ 2022-09-03T06:47:44.449 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU= 2022-09-03T06:47:44.449 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU=', 'Content-Length': '0'} 2022-09-03T06:47:44.452 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000649f099577e1f52b-006312f890-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.453 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:44.454 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:44.455 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:44.455 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:44.456 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:47:44.456 DEBUG:boto:Method: GET 2022-09-03T06:47:44.457 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:47:44.458 DEBUG:boto:Data: 2022-09-03T06:47:44.458 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.459 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.459 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.460 DEBUG:boto:Params: {} 2022-09-03T06:47:44.461 DEBUG:boto:Token: None 2022-09-03T06:47:44.461 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/ 2022-09-03T06:47:44.462 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU= 2022-09-03T06:47:44.463 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU=', 'Content-Length': '0'} 2022-09-03T06:47:44.465 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000076f2a9cd0f0abe82-006312f890-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.467 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:44.468 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:47:44.469 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:44.470 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:44.471 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.472 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.473 DEBUG:boto:Method: GET 2022-09-03T06:47:44.474 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.475 DEBUG:boto:Data: 2022-09-03T06:47:44.476 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.477 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.478 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.479 DEBUG:boto:Params: {} 2022-09-03T06:47:44.480 DEBUG:boto:Token: None 2022-09-03T06:47:44.481 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.482 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VPTLrbA41PDtMz1oCRlIeHIpwV8= 2022-09-03T06:47:44.483 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VPTLrbA41PDtMz1oCRlIeHIpwV8=', 'Content-Length': '0'} 2022-09-03T06:47:44.486 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000651dd354fdc54588-006312f890-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.487 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:44.488 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:44.489 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.490 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.491 DEBUG:boto:Method: GET 2022-09-03T06:47:44.491 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.492 DEBUG:boto:Data: 2022-09-03T06:47:44.492 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.493 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.494 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.494 DEBUG:boto:Params: {} 2022-09-03T06:47:44.495 DEBUG:boto:Token: None 2022-09-03T06:47:44.496 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.496 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VPTLrbA41PDtMz1oCRlIeHIpwV8= 2022-09-03T06:47:44.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VPTLrbA41PDtMz1oCRlIeHIpwV8=', 'Content-Length': '0'} 2022-09-03T06:47:44.499 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008ba90d9360d2c1ef-006312f890-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.500 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:44.500 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:44.501 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.502 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.502 DEBUG:boto:Method: HEAD 2022-09-03T06:47:44.503 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.503 DEBUG:boto:Data: 2022-09-03T06:47:44.504 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.504 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.505 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.505 DEBUG:boto:Params: {} 2022-09-03T06:47:44.506 DEBUG:boto:Token: None 2022-09-03T06:47:44.506 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.507 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j4lHkY2y0NLpdDI/ZweSDrlsnd4= 2022-09-03T06:47:44.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j4lHkY2y0NLpdDI/ZweSDrlsnd4=', 'Content-Length': '0'} 2022-09-03T06:47:44.510 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e15b62cb67fc28f8-006312f890-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.511 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:44.511 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:44.512 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.512 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.513 DEBUG:boto:Method: HEAD 2022-09-03T06:47:44.513 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.514 DEBUG:boto:Data: 2022-09-03T06:47:44.514 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.515 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.515 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.516 DEBUG:boto:Params: {} 2022-09-03T06:47:44.516 DEBUG:boto:Token: None 2022-09-03T06:47:44.517 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.518 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j4lHkY2y0NLpdDI/ZweSDrlsnd4= 2022-09-03T06:47:44.518 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j4lHkY2y0NLpdDI/ZweSDrlsnd4=', 'Content-Length': '0'} 2022-09-03T06:47:44.520 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000085361958b09df143-006312f890-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.522 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:47:44.523 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:44.523 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:44.524 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.525 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.525 DEBUG:boto:Method: GET 2022-09-03T06:47:44.526 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.526 DEBUG:boto:Data: 2022-09-03T06:47:44.527 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.527 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.528 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.528 DEBUG:boto:Params: {} 2022-09-03T06:47:44.529 DEBUG:boto:Token: None 2022-09-03T06:47:44.531 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.531 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VPTLrbA41PDtMz1oCRlIeHIpwV8= 2022-09-03T06:47:44.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VPTLrbA41PDtMz1oCRlIeHIpwV8=', 'Content-Length': '0'} 2022-09-03T06:47:44.535 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fb5d39765d5e13b9-006312f890-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.536 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:44.537 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:44.537 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.538 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.539 DEBUG:boto:Method: GET 2022-09-03T06:47:44.539 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.540 DEBUG:boto:Data: 2022-09-03T06:47:44.540 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.541 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.542 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.542 DEBUG:boto:Params: {} 2022-09-03T06:47:44.543 DEBUG:boto:Token: None 2022-09-03T06:47:44.543 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:47:44.544 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VPTLrbA41PDtMz1oCRlIeHIpwV8= 2022-09-03T06:47:44.545 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VPTLrbA41PDtMz1oCRlIeHIpwV8=', 'Content-Length': '0'} 2022-09-03T06:47:44.547 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad0871d3d6b321f5-006312f890-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.548 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:44.548 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:44.549 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:44.549 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:44.550 DEBUG:boto:Method: GET 2022-09-03T06:47:44.550 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:47:44.551 DEBUG:boto:Data: 2022-09-03T06:47:44.551 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.552 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.552 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.553 DEBUG:boto:Params: {} 2022-09-03T06:47:44.554 DEBUG:boto:Token: None 2022-09-03T06:47:44.554 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/ 2022-09-03T06:47:44.555 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU= 2022-09-03T06:47:44.555 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU=', 'Content-Length': '0'} 2022-09-03T06:47:44.558 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000afe89976dfc70306-006312f890-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.559 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:44.560 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:44.560 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:44.561 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:44.561 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:44.562 DEBUG:boto:Method: GET 2022-09-03T06:47:44.562 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:47:44.563 DEBUG:boto:Data: 2022-09-03T06:47:44.564 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.564 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.565 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.565 DEBUG:boto:Params: {} 2022-09-03T06:47:44.566 DEBUG:boto:Token: None 2022-09-03T06:47:44.566 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/ 2022-09-03T06:47:44.567 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU= 2022-09-03T06:47:44.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU=', 'Content-Length': '0'} 2022-09-03T06:47:44.570 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eacdc8cc2d4031f3-006312f890-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.619 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:44.621 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:44.622 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:44.623 DEBUG:boto:Method: HEAD 2022-09-03T06:47:44.624 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:47:44.624 DEBUG:boto:Data: 2022-09-03T06:47:44.625 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.625 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.626 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.626 DEBUG:boto:Params: {} 2022-09-03T06:47:44.627 DEBUG:boto:Token: None 2022-09-03T06:47:44.627 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj34 2022-09-03T06:47:44.628 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:elR9MlBWsmNUqYsbNbSZ+o+rQmw= 2022-09-03T06:47:44.629 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:elR9MlBWsmNUqYsbNbSZ+o+rQmw=', 'Content-Length': '0'} 2022-09-03T06:47:44.631 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002fd9fd9561c78218-006312f890-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.632 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:44.632 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:44.633 DEBUG:boto:Method: HEAD 2022-09-03T06:47:44.633 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:47:44.634 DEBUG:boto:Data: 2022-09-03T06:47:44.634 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.635 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.635 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.636 DEBUG:boto:Params: {} 2022-09-03T06:47:44.636 DEBUG:boto:Token: None 2022-09-03T06:47:44.637 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj34 2022-09-03T06:47:44.638 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:elR9MlBWsmNUqYsbNbSZ+o+rQmw= 2022-09-03T06:47:44.639 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:elR9MlBWsmNUqYsbNbSZ+o+rQmw=', 'Content-Length': '0'} 2022-09-03T06:47:44.641 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b8108b0cabf2a2c0-006312f890-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.642 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:47:44.642 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:44.643 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:44.644 DEBUG:boto:Method: GET 2022-09-03T06:47:44.644 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:47:44.645 DEBUG:boto:Data: 2022-09-03T06:47:44.645 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.646 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.647 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.648 DEBUG:boto:Params: {} 2022-09-03T06:47:44.648 DEBUG:boto:Token: None 2022-09-03T06:47:44.649 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj34 2022-09-03T06:47:44.650 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c1pWPMqOPsiYrWMMIaVgSrUaEec= 2022-09-03T06:47:44.650 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c1pWPMqOPsiYrWMMIaVgSrUaEec=', 'Content-Length': '0'} 2022-09-03T06:47:44.652 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007d80e1543a9128e7-006312f890-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.698 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:47:44.699 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:47:44.700 DEBUG:boto:Method: GET 2022-09-03T06:47:44.701 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:47:44.702 DEBUG:boto:Data: 2022-09-03T06:47:44.702 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.703 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.704 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.704 DEBUG:boto:Params: {} 2022-09-03T06:47:44.705 DEBUG:boto:Token: None 2022-09-03T06:47:44.706 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj34 2022-09-03T06:47:44.707 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c1pWPMqOPsiYrWMMIaVgSrUaEec= 2022-09-03T06:47:44.707 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c1pWPMqOPsiYrWMMIaVgSrUaEec=', 'Content-Length': '0'} 2022-09-03T06:47:44.710 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005fa1bf7be9bf3688-006312f890-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.711 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:44.711 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:44.712 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:44.712 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:44.713 DEBUG:boto:Method: GET 2022-09-03T06:47:44.714 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:47:44.714 DEBUG:boto:Data: 2022-09-03T06:47:44.715 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.716 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.716 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.717 DEBUG:boto:Params: {} 2022-09-03T06:47:44.718 DEBUG:boto:Token: None 2022-09-03T06:47:44.718 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/ 2022-09-03T06:47:44.719 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU= 2022-09-03T06:47:44.720 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU=', 'Content-Length': '0'} 2022-09-03T06:47:44.723 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002ed0986d464ccb78-006312f890-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.724 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:44.725 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:44.725 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:44.726 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:44.727 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:47:44.727 DEBUG:boto:Method: GET 2022-09-03T06:47:44.728 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:47:44.729 DEBUG:boto:Data: 2022-09-03T06:47:44.730 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.731 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.732 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.733 DEBUG:boto:Params: {} 2022-09-03T06:47:44.733 DEBUG:boto:Token: None 2022-09-03T06:47:44.734 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/ 2022-09-03T06:47:44.735 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU= 2022-09-03T06:47:44.736 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU=', 'Content-Length': '0'} 2022-09-03T06:47:44.739 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007808878b0fd5128b-006312f890-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.739 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:44.740 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:47:44.741 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:44.741 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:44.742 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.743 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.743 DEBUG:boto:Method: GET 2022-09-03T06:47:44.744 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.744 DEBUG:boto:Data: 2022-09-03T06:47:44.745 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.745 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.746 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.746 DEBUG:boto:Params: {} 2022-09-03T06:47:44.747 DEBUG:boto:Token: None 2022-09-03T06:47:44.748 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.748 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K9i43U8N+k2jM7g1JYNyGuoqUpU= 2022-09-03T06:47:44.749 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K9i43U8N+k2jM7g1JYNyGuoqUpU=', 'Content-Length': '0'} 2022-09-03T06:47:44.751 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b843dbfb7d321365-006312f890-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.753 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:44.754 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:44.755 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.756 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.757 DEBUG:boto:Method: GET 2022-09-03T06:47:44.758 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.759 DEBUG:boto:Data: 2022-09-03T06:47:44.760 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.761 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.762 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.763 DEBUG:boto:Params: {} 2022-09-03T06:47:44.763 DEBUG:boto:Token: None 2022-09-03T06:47:44.764 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.765 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K9i43U8N+k2jM7g1JYNyGuoqUpU= 2022-09-03T06:47:44.765 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K9i43U8N+k2jM7g1JYNyGuoqUpU=', 'Content-Length': '0'} 2022-09-03T06:47:44.768 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000006d6ca48e399db1e-006312f890-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.769 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:44.770 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:44.771 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.771 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.772 DEBUG:boto:Method: HEAD 2022-09-03T06:47:44.772 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.773 DEBUG:boto:Data: 2022-09-03T06:47:44.774 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.774 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.775 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.775 DEBUG:boto:Params: {} 2022-09-03T06:47:44.776 DEBUG:boto:Token: None 2022-09-03T06:47:44.777 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.777 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ofb2yPr02dwOT8GsSh/KeCBHg+I= 2022-09-03T06:47:44.778 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ofb2yPr02dwOT8GsSh/KeCBHg+I=', 'Content-Length': '0'} 2022-09-03T06:47:44.780 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e72c0fa3dec0ef04-006312f890-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.781 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:44.781 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:44.782 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.782 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.783 DEBUG:boto:Method: HEAD 2022-09-03T06:47:44.783 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.784 DEBUG:boto:Data: 2022-09-03T06:47:44.784 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.785 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.785 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.786 DEBUG:boto:Params: {} 2022-09-03T06:47:44.786 DEBUG:boto:Token: None 2022-09-03T06:47:44.787 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.788 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ofb2yPr02dwOT8GsSh/KeCBHg+I= 2022-09-03T06:47:44.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ofb2yPr02dwOT8GsSh/KeCBHg+I=', 'Content-Length': '0'} 2022-09-03T06:47:44.791 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000099bc8c5aec558c8a-006312f890-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.792 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:47:44.793 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:44.794 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:44.795 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.796 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.797 DEBUG:boto:Method: GET 2022-09-03T06:47:44.798 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.799 DEBUG:boto:Data: 2022-09-03T06:47:44.800 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.801 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.802 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.803 DEBUG:boto:Params: {} 2022-09-03T06:47:44.804 DEBUG:boto:Token: None 2022-09-03T06:47:44.805 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.806 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K9i43U8N+k2jM7g1JYNyGuoqUpU= 2022-09-03T06:47:44.807 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K9i43U8N+k2jM7g1JYNyGuoqUpU=', 'Content-Length': '0'} 2022-09-03T06:47:44.810 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2e2aac7d2eccc8e-006312f890-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.811 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:44.812 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:44.813 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.814 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.815 DEBUG:boto:Method: GET 2022-09-03T06:47:44.816 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.817 DEBUG:boto:Data: 2022-09-03T06:47:44.818 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.819 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.819 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.820 DEBUG:boto:Params: {} 2022-09-03T06:47:44.822 DEBUG:boto:Token: None 2022-09-03T06:47:44.823 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:47:44.824 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:K9i43U8N+k2jM7g1JYNyGuoqUpU= 2022-09-03T06:47:44.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:K9i43U8N+k2jM7g1JYNyGuoqUpU=', 'Content-Length': '0'} 2022-09-03T06:47:44.833 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e025908383d2c890-006312f890-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.874 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:44.875 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:44.875 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:44.876 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:44.877 DEBUG:boto:Method: GET 2022-09-03T06:47:44.877 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:47:44.878 DEBUG:boto:Data: 2022-09-03T06:47:44.878 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.879 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.879 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.880 DEBUG:boto:Params: {} 2022-09-03T06:47:44.880 DEBUG:boto:Token: None 2022-09-03T06:47:44.881 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/ 2022-09-03T06:47:44.882 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU= 2022-09-03T06:47:44.883 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU=', 'Content-Length': '0'} 2022-09-03T06:47:44.886 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ca754b390b6949a2-006312f890-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.887 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:44.888 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:44.888 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:44.889 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:44.889 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:44.890 DEBUG:boto:Method: GET 2022-09-03T06:47:44.891 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:47:44.891 DEBUG:boto:Data: 2022-09-03T06:47:44.892 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.893 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.893 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.894 DEBUG:boto:Params: {} 2022-09-03T06:47:44.894 DEBUG:boto:Token: None 2022-09-03T06:47:44.895 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/ 2022-09-03T06:47:44.896 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU= 2022-09-03T06:47:44.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU=', 'Content-Length': '0'} 2022-09-03T06:47:44.899 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fe9220f9dd214d86-006312f890-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.900 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:44.901 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:44.901 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:44.902 DEBUG:boto:Method: HEAD 2022-09-03T06:47:44.902 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:47:44.903 DEBUG:boto:Data: 2022-09-03T06:47:44.904 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.904 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.905 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.905 DEBUG:boto:Params: {} 2022-09-03T06:47:44.906 DEBUG:boto:Token: None 2022-09-03T06:47:44.906 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj38 2022-09-03T06:47:44.907 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7AqYiDXYjWPqCQot4e0VVHUMps8= 2022-09-03T06:47:44.908 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7AqYiDXYjWPqCQot4e0VVHUMps8=', 'Content-Length': '0'} 2022-09-03T06:47:44.910 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008f8b2631b2c6bd96-006312f890-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.911 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:44.911 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:44.912 DEBUG:boto:Method: HEAD 2022-09-03T06:47:44.912 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:47:44.913 DEBUG:boto:Data: 2022-09-03T06:47:44.914 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.915 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.916 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.916 DEBUG:boto:Params: {} 2022-09-03T06:47:44.917 DEBUG:boto:Token: None 2022-09-03T06:47:44.917 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj38 2022-09-03T06:47:44.918 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7AqYiDXYjWPqCQot4e0VVHUMps8= 2022-09-03T06:47:44.919 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7AqYiDXYjWPqCQot4e0VVHUMps8=', 'Content-Length': '0'} 2022-09-03T06:47:44.921 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b6911e5f88806e8f-006312f890-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.922 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:47:44.922 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:44.923 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:44.924 DEBUG:boto:Method: GET 2022-09-03T06:47:44.924 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:47:44.925 DEBUG:boto:Data: 2022-09-03T06:47:44.926 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.926 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.927 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.928 DEBUG:boto:Params: {} 2022-09-03T06:47:44.928 DEBUG:boto:Token: None 2022-09-03T06:47:44.929 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj38 2022-09-03T06:47:44.930 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s0rOyMSMNcqZO4kfd81EwFTpteA= 2022-09-03T06:47:44.930 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s0rOyMSMNcqZO4kfd81EwFTpteA=', 'Content-Length': '0'} 2022-09-03T06:47:44.933 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003f0fb96ec6702b11-006312f890-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.974 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:47:44.976 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:47:44.977 DEBUG:boto:Method: GET 2022-09-03T06:47:44.978 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:47:44.979 DEBUG:boto:Data: 2022-09-03T06:47:44.980 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.980 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:44.981 DEBUG:boto:Port: 8000 2022-09-03T06:47:44.982 DEBUG:boto:Params: {} 2022-09-03T06:47:44.983 DEBUG:boto:Token: None 2022-09-03T06:47:44.984 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/obj38 2022-09-03T06:47:44.984 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s0rOyMSMNcqZO4kfd81EwFTpteA= 2022-09-03T06:47:44.985 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s0rOyMSMNcqZO4kfd81EwFTpteA=', 'Content-Length': '0'} 2022-09-03T06:47:44.987 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ec4c97b87fcb0959-006312f890-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:44 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:44.988 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:44.989 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:44.989 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:44.990 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:44.991 DEBUG:boto:Method: GET 2022-09-03T06:47:44.991 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:47:44.992 DEBUG:boto:Data: 2022-09-03T06:47:44.992 DEBUG:boto:Headers: {} 2022-09-03T06:47:44.993 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:44.994 DEBUG:boto:Port: 8001 2022-09-03T06:47:44.995 DEBUG:boto:Params: {} 2022-09-03T06:47:44.996 DEBUG:boto:Token: None 2022-09-03T06:47:44.998 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:44 GMT /dsgvcy-75/ 2022-09-03T06:47:44.999 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU= 2022-09-03T06:47:45.000 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:44 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vFSzbBek/1CbRzBhVhuN9H8paBU=', 'Content-Length': '0'} 2022-09-03T06:47:45.003 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009ed9c2f759b97086-006312f891-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.005 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:45.006 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:45.007 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:45.008 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:45.008 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:47:45.009 DEBUG:boto:Method: GET 2022-09-03T06:47:45.009 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:47:45.010 DEBUG:boto:Data: 2022-09-03T06:47:45.011 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.012 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:45.012 DEBUG:boto:Port: 8000 2022-09-03T06:47:45.013 DEBUG:boto:Params: {} 2022-09-03T06:47:45.013 DEBUG:boto:Token: None 2022-09-03T06:47:45.014 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/ 2022-09-03T06:47:45.015 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A= 2022-09-03T06:47:45.015 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A=', 'Content-Length': '0'} 2022-09-03T06:47:45.018 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000870d2f50c90834d6-006312f891-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.019 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:45.020 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:47:45.020 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:45.021 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:45.021 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.022 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.022 DEBUG:boto:Method: GET 2022-09-03T06:47:45.023 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.023 DEBUG:boto:Data: 2022-09-03T06:47:45.024 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.025 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:45.025 DEBUG:boto:Port: 8001 2022-09-03T06:47:45.026 DEBUG:boto:Params: {} 2022-09-03T06:47:45.027 DEBUG:boto:Token: None 2022-09-03T06:47:45.027 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.028 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Pry/3Cpr/dNIGRQ9/BgJtrHcDaw= 2022-09-03T06:47:45.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Pry/3Cpr/dNIGRQ9/BgJtrHcDaw=', 'Content-Length': '0'} 2022-09-03T06:47:45.031 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000950075d2ce6f6c5f-006312f891-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.032 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:45.033 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:45.033 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.034 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.034 DEBUG:boto:Method: GET 2022-09-03T06:47:45.036 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.037 DEBUG:boto:Data: 2022-09-03T06:47:45.038 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.039 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:45.040 DEBUG:boto:Port: 8000 2022-09-03T06:47:45.041 DEBUG:boto:Params: {} 2022-09-03T06:47:45.042 DEBUG:boto:Token: None 2022-09-03T06:47:45.043 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.044 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Pry/3Cpr/dNIGRQ9/BgJtrHcDaw= 2022-09-03T06:47:45.045 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Pry/3Cpr/dNIGRQ9/BgJtrHcDaw=', 'Content-Length': '0'} 2022-09-03T06:47:45.047 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007e13b6e08849c99b-006312f891-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.094 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:45.095 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:45.097 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.098 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.098 DEBUG:boto:Method: HEAD 2022-09-03T06:47:45.099 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.101 DEBUG:boto:Data: 2022-09-03T06:47:45.102 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.102 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:45.103 DEBUG:boto:Port: 8001 2022-09-03T06:47:45.104 DEBUG:boto:Params: {} 2022-09-03T06:47:45.105 DEBUG:boto:Token: None 2022-09-03T06:47:45.107 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.108 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VQ9yZ7APiSxecqwjoyNEjElcMlc= 2022-09-03T06:47:45.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VQ9yZ7APiSxecqwjoyNEjElcMlc=', 'Content-Length': '0'} 2022-09-03T06:47:45.112 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee64d4df8217adc5-006312f891-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.113 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:45.113 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:45.114 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.114 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.115 DEBUG:boto:Method: HEAD 2022-09-03T06:47:45.116 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.116 DEBUG:boto:Data: 2022-09-03T06:47:45.117 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.118 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:45.119 DEBUG:boto:Port: 8000 2022-09-03T06:47:45.120 DEBUG:boto:Params: {} 2022-09-03T06:47:45.121 DEBUG:boto:Token: None 2022-09-03T06:47:45.122 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.123 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VQ9yZ7APiSxecqwjoyNEjElcMlc= 2022-09-03T06:47:45.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VQ9yZ7APiSxecqwjoyNEjElcMlc=', 'Content-Length': '0'} 2022-09-03T06:47:45.126 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000011527f80b917f00f-006312f891-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.127 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:47:45.127 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:45.128 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:45.129 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.129 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.130 DEBUG:boto:Method: GET 2022-09-03T06:47:45.131 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.131 DEBUG:boto:Data: 2022-09-03T06:47:45.132 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.132 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:45.133 DEBUG:boto:Port: 8001 2022-09-03T06:47:45.133 DEBUG:boto:Params: {} 2022-09-03T06:47:45.134 DEBUG:boto:Token: None 2022-09-03T06:47:45.135 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.136 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Pry/3Cpr/dNIGRQ9/BgJtrHcDaw= 2022-09-03T06:47:45.136 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Pry/3Cpr/dNIGRQ9/BgJtrHcDaw=', 'Content-Length': '0'} 2022-09-03T06:47:45.139 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aaa2a5d38de1e5da-006312f891-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.140 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:45.140 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:45.141 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.142 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.142 DEBUG:boto:Method: GET 2022-09-03T06:47:45.143 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.143 DEBUG:boto:Data: 2022-09-03T06:47:45.144 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.144 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:45.145 DEBUG:boto:Port: 8000 2022-09-03T06:47:45.145 DEBUG:boto:Params: {} 2022-09-03T06:47:45.146 DEBUG:boto:Token: None 2022-09-03T06:47:45.147 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:47:45.147 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Pry/3Cpr/dNIGRQ9/BgJtrHcDaw= 2022-09-03T06:47:45.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Pry/3Cpr/dNIGRQ9/BgJtrHcDaw=', 'Content-Length': '0'} 2022-09-03T06:47:45.150 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000e952e740de51a23-006312f891-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.151 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:45.152 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:45.153 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:45.154 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:45.155 DEBUG:boto:Method: GET 2022-09-03T06:47:45.156 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:47:45.157 DEBUG:boto:Data: 2022-09-03T06:47:45.158 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.159 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:45.161 DEBUG:boto:Port: 8001 2022-09-03T06:47:45.162 DEBUG:boto:Params: {} 2022-09-03T06:47:45.162 DEBUG:boto:Token: None 2022-09-03T06:47:45.163 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/ 2022-09-03T06:47:45.164 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A= 2022-09-03T06:47:45.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A=', 'Content-Length': '0'} 2022-09-03T06:47:45.169 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006a7f5c36f5028261-006312f891-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.210 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:45.212 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:45.213 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:45.214 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:45.215 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:45.216 DEBUG:boto:Method: GET 2022-09-03T06:47:45.217 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:47:45.218 DEBUG:boto:Data: 2022-09-03T06:47:45.219 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.220 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:45.221 DEBUG:boto:Port: 8000 2022-09-03T06:47:45.222 DEBUG:boto:Params: {} 2022-09-03T06:47:45.223 DEBUG:boto:Token: None 2022-09-03T06:47:45.224 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/ 2022-09-03T06:47:45.225 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A= 2022-09-03T06:47:45.227 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A=', 'Content-Length': '0'} 2022-09-03T06:47:45.230 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e5fd5ae3a0c26e4e-006312f891-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.231 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:45.232 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:45.233 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:45.234 DEBUG:boto:Method: HEAD 2022-09-03T06:47:45.235 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:47:45.236 DEBUG:boto:Data: 2022-09-03T06:47:45.237 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.238 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:45.239 DEBUG:boto:Port: 8001 2022-09-03T06:47:45.240 DEBUG:boto:Params: {} 2022-09-03T06:47:45.241 DEBUG:boto:Token: None 2022-09-03T06:47:45.242 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj44 2022-09-03T06:47:45.243 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2vg8tth0ipdVV8M2LQkP6XPiyqY= 2022-09-03T06:47:45.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2vg8tth0ipdVV8M2LQkP6XPiyqY=', 'Content-Length': '0'} 2022-09-03T06:47:45.246 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d5338f3f033a1d9-006312f891-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.246 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:45.247 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:45.247 DEBUG:boto:Method: HEAD 2022-09-03T06:47:45.248 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:47:45.248 DEBUG:boto:Data: 2022-09-03T06:47:45.249 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.249 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:45.250 DEBUG:boto:Port: 8000 2022-09-03T06:47:45.250 DEBUG:boto:Params: {} 2022-09-03T06:47:45.251 DEBUG:boto:Token: None 2022-09-03T06:47:45.252 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj44 2022-09-03T06:47:45.252 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2vg8tth0ipdVV8M2LQkP6XPiyqY= 2022-09-03T06:47:45.253 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2vg8tth0ipdVV8M2LQkP6XPiyqY=', 'Content-Length': '0'} 2022-09-03T06:47:45.255 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007ad5171b8c6d86af-006312f891-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.255 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:47:45.256 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:45.257 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:45.257 DEBUG:boto:Method: GET 2022-09-03T06:47:45.258 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:47:45.258 DEBUG:boto:Data: 2022-09-03T06:47:45.259 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.259 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:45.260 DEBUG:boto:Port: 8001 2022-09-03T06:47:45.260 DEBUG:boto:Params: {} 2022-09-03T06:47:45.261 DEBUG:boto:Token: None 2022-09-03T06:47:45.262 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj44 2022-09-03T06:47:45.262 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i7jv7M413uHkNDnwG3WM7ym2G6w= 2022-09-03T06:47:45.263 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i7jv7M413uHkNDnwG3WM7ym2G6w=', 'Content-Length': '0'} 2022-09-03T06:47:45.265 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f35b999e381a4f13-006312f891-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.266 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:47:45.266 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:47:45.267 DEBUG:boto:Method: GET 2022-09-03T06:47:45.268 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:47:45.268 DEBUG:boto:Data: 2022-09-03T06:47:45.269 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.269 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:45.270 DEBUG:boto:Port: 8000 2022-09-03T06:47:45.270 DEBUG:boto:Params: {} 2022-09-03T06:47:45.271 DEBUG:boto:Token: None 2022-09-03T06:47:45.271 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj44 2022-09-03T06:47:45.272 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i7jv7M413uHkNDnwG3WM7ym2G6w= 2022-09-03T06:47:45.272 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i7jv7M413uHkNDnwG3WM7ym2G6w=', 'Content-Length': '0'} 2022-09-03T06:47:45.275 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000788cf9b67c8a5626-006312f891-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.318 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:45.319 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:45.319 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:45.320 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:45.320 DEBUG:boto:Method: GET 2022-09-03T06:47:45.321 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:47:45.321 DEBUG:boto:Data: 2022-09-03T06:47:45.322 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.322 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:45.323 DEBUG:boto:Port: 8001 2022-09-03T06:47:45.323 DEBUG:boto:Params: {} 2022-09-03T06:47:45.324 DEBUG:boto:Token: None 2022-09-03T06:47:45.325 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/ 2022-09-03T06:47:45.325 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A= 2022-09-03T06:47:45.326 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A=', 'Content-Length': '0'} 2022-09-03T06:47:45.328 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d980ed0b5894d76a-006312f891-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.329 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:45.330 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:45.331 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:45.331 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:45.332 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:47:45.332 DEBUG:boto:Method: GET 2022-09-03T06:47:45.333 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:47:45.333 DEBUG:boto:Data: 2022-09-03T06:47:45.334 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.334 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:45.335 DEBUG:boto:Port: 8000 2022-09-03T06:47:45.335 DEBUG:boto:Params: {} 2022-09-03T06:47:45.336 DEBUG:boto:Token: None 2022-09-03T06:47:45.337 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/ 2022-09-03T06:47:45.337 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A= 2022-09-03T06:47:45.338 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A=', 'Content-Length': '0'} 2022-09-03T06:47:45.341 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000821eef33f15ad065-006312f891-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.341 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:45.342 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:47:45.343 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:45.343 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:45.344 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.344 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.345 DEBUG:boto:Method: GET 2022-09-03T06:47:45.346 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.346 DEBUG:boto:Data: 2022-09-03T06:47:45.347 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.348 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:45.348 DEBUG:boto:Port: 8001 2022-09-03T06:47:45.349 DEBUG:boto:Params: {} 2022-09-03T06:47:45.349 DEBUG:boto:Token: None 2022-09-03T06:47:45.350 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.351 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qvXIW29udK9jof47AaKq3n964b0= 2022-09-03T06:47:45.351 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qvXIW29udK9jof47AaKq3n964b0=', 'Content-Length': '0'} 2022-09-03T06:47:45.354 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004c661c95562618ad-006312f891-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.355 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:45.355 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:45.357 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.357 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.358 DEBUG:boto:Method: GET 2022-09-03T06:47:45.358 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.359 DEBUG:boto:Data: 2022-09-03T06:47:45.359 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.360 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:45.360 DEBUG:boto:Port: 8000 2022-09-03T06:47:45.361 DEBUG:boto:Params: {} 2022-09-03T06:47:45.361 DEBUG:boto:Token: None 2022-09-03T06:47:45.362 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.363 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qvXIW29udK9jof47AaKq3n964b0= 2022-09-03T06:47:45.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qvXIW29udK9jof47AaKq3n964b0=', 'Content-Length': '0'} 2022-09-03T06:47:45.366 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002374f7de9aa7297b-006312f891-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.367 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:45.367 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:45.368 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.368 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.369 DEBUG:boto:Method: HEAD 2022-09-03T06:47:45.369 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.370 DEBUG:boto:Data: 2022-09-03T06:47:45.371 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.371 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:45.372 DEBUG:boto:Port: 8001 2022-09-03T06:47:45.372 DEBUG:boto:Params: {} 2022-09-03T06:47:45.373 DEBUG:boto:Token: None 2022-09-03T06:47:45.374 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.374 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:js2FrM08WJ7knTZoxjHkBGp0UVc= 2022-09-03T06:47:45.375 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:js2FrM08WJ7knTZoxjHkBGp0UVc=', 'Content-Length': '0'} 2022-09-03T06:47:45.377 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008a381eba2bf46915-006312f891-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.378 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:45.378 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:45.379 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.380 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.380 DEBUG:boto:Method: HEAD 2022-09-03T06:47:45.381 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.381 DEBUG:boto:Data: 2022-09-03T06:47:45.382 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.382 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:45.383 DEBUG:boto:Port: 8000 2022-09-03T06:47:45.383 DEBUG:boto:Params: {} 2022-09-03T06:47:45.384 DEBUG:boto:Token: None 2022-09-03T06:47:45.385 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.385 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:js2FrM08WJ7knTZoxjHkBGp0UVc= 2022-09-03T06:47:45.386 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:js2FrM08WJ7knTZoxjHkBGp0UVc=', 'Content-Length': '0'} 2022-09-03T06:47:45.388 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002ea1dcc8cec5603b-006312f891-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.389 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:47:45.390 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:45.390 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:45.391 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.392 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.392 DEBUG:boto:Method: GET 2022-09-03T06:47:45.393 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.394 DEBUG:boto:Data: 2022-09-03T06:47:45.394 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.395 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:45.396 DEBUG:boto:Port: 8001 2022-09-03T06:47:45.396 DEBUG:boto:Params: {} 2022-09-03T06:47:45.397 DEBUG:boto:Token: None 2022-09-03T06:47:45.398 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.398 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qvXIW29udK9jof47AaKq3n964b0= 2022-09-03T06:47:45.399 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qvXIW29udK9jof47AaKq3n964b0=', 'Content-Length': '0'} 2022-09-03T06:47:45.401 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a17f6d12e3b9e02-006312f891-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.402 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:45.403 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:45.403 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.404 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.404 DEBUG:boto:Method: GET 2022-09-03T06:47:45.405 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.406 DEBUG:boto:Data: 2022-09-03T06:47:45.406 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.407 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:45.408 DEBUG:boto:Port: 8000 2022-09-03T06:47:45.409 DEBUG:boto:Params: {} 2022-09-03T06:47:45.410 DEBUG:boto:Token: None 2022-09-03T06:47:45.411 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:47:45.412 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qvXIW29udK9jof47AaKq3n964b0= 2022-09-03T06:47:45.414 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qvXIW29udK9jof47AaKq3n964b0=', 'Content-Length': '0'} 2022-09-03T06:47:45.416 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2d16a4679ee2433-006312f891-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.418 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:45.420 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:45.420 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:45.421 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:45.422 DEBUG:boto:Method: GET 2022-09-03T06:47:45.423 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:47:45.424 DEBUG:boto:Data: 2022-09-03T06:47:45.425 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.426 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:45.437 DEBUG:boto:Port: 8001 2022-09-03T06:47:45.438 DEBUG:boto:Params: {} 2022-09-03T06:47:45.439 DEBUG:boto:Token: None 2022-09-03T06:47:45.440 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/ 2022-09-03T06:47:45.442 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A= 2022-09-03T06:47:45.445 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A=', 'Content-Length': '0'} 2022-09-03T06:47:45.448 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004f99f83ccc8a50e1-006312f891-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.450 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:45.451 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:45.451 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:45.452 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:45.453 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:45.453 DEBUG:boto:Method: GET 2022-09-03T06:47:45.454 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:47:45.455 DEBUG:boto:Data: 2022-09-03T06:47:45.455 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.456 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:45.457 DEBUG:boto:Port: 8000 2022-09-03T06:47:45.457 DEBUG:boto:Params: {} 2022-09-03T06:47:45.458 DEBUG:boto:Token: None 2022-09-03T06:47:45.459 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/ 2022-09-03T06:47:45.460 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A= 2022-09-03T06:47:45.461 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A=', 'Content-Length': '0'} 2022-09-03T06:47:45.464 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000081df53be526abfad-006312f891-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.465 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:45.466 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:45.466 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:45.467 DEBUG:boto:Method: HEAD 2022-09-03T06:47:45.468 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:47:45.468 DEBUG:boto:Data: 2022-09-03T06:47:45.469 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.470 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:45.470 DEBUG:boto:Port: 8001 2022-09-03T06:47:45.471 DEBUG:boto:Params: {} 2022-09-03T06:47:45.473 DEBUG:boto:Token: None 2022-09-03T06:47:45.474 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj46 2022-09-03T06:47:45.475 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aHq2xcjKrQDzhmvwMgEszPLU6aY= 2022-09-03T06:47:45.477 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aHq2xcjKrQDzhmvwMgEszPLU6aY=', 'Content-Length': '0'} 2022-09-03T06:47:45.481 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004b11f09c6150f656-006312f891-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.482 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:45.483 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:45.485 DEBUG:boto:Method: HEAD 2022-09-03T06:47:45.486 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:47:45.487 DEBUG:boto:Data: 2022-09-03T06:47:45.488 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.491 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:45.492 DEBUG:boto:Port: 8000 2022-09-03T06:47:45.493 DEBUG:boto:Params: {} 2022-09-03T06:47:45.494 DEBUG:boto:Token: None 2022-09-03T06:47:45.495 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj46 2022-09-03T06:47:45.496 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aHq2xcjKrQDzhmvwMgEszPLU6aY= 2022-09-03T06:47:45.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aHq2xcjKrQDzhmvwMgEszPLU6aY=', 'Content-Length': '0'} 2022-09-03T06:47:45.500 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db6c0e46b88de715-006312f891-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.502 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:47:45.503 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:45.504 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:45.505 DEBUG:boto:Method: GET 2022-09-03T06:47:45.506 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:47:45.507 DEBUG:boto:Data: 2022-09-03T06:47:45.507 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.508 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:45.508 DEBUG:boto:Port: 8001 2022-09-03T06:47:45.509 DEBUG:boto:Params: {} 2022-09-03T06:47:45.510 DEBUG:boto:Token: None 2022-09-03T06:47:45.511 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj46 2022-09-03T06:47:45.512 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ACu+xd33w7PjFUaY/DlxrKCcdbc= 2022-09-03T06:47:45.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ACu+xd33w7PjFUaY/DlxrKCcdbc=', 'Content-Length': '0'} 2022-09-03T06:47:45.516 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000031a0d5072580df39-006312f891-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.518 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:47:45.519 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:47:45.520 DEBUG:boto:Method: GET 2022-09-03T06:47:45.521 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:47:45.522 DEBUG:boto:Data: 2022-09-03T06:47:45.522 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.523 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:45.524 DEBUG:boto:Port: 8000 2022-09-03T06:47:45.525 DEBUG:boto:Params: {} 2022-09-03T06:47:45.526 DEBUG:boto:Token: None 2022-09-03T06:47:45.527 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj46 2022-09-03T06:47:45.528 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ACu+xd33w7PjFUaY/DlxrKCcdbc= 2022-09-03T06:47:45.530 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ACu+xd33w7PjFUaY/DlxrKCcdbc=', 'Content-Length': '0'} 2022-09-03T06:47:45.532 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000b89eafae91cab0a-006312f891-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.581 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:45.587 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:45.588 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:45.588 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:45.589 DEBUG:boto:Method: GET 2022-09-03T06:47:45.589 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:47:45.590 DEBUG:boto:Data: 2022-09-03T06:47:45.591 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.591 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:45.592 DEBUG:boto:Port: 8001 2022-09-03T06:47:45.592 DEBUG:boto:Params: {} 2022-09-03T06:47:45.593 DEBUG:boto:Token: None 2022-09-03T06:47:45.594 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/ 2022-09-03T06:47:45.594 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A= 2022-09-03T06:47:45.595 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A=', 'Content-Length': '0'} 2022-09-03T06:47:45.598 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004ab9d22eb045e02f-006312f891-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.599 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:45.600 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:45.601 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:45.601 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:45.602 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:47:45.603 DEBUG:boto:Method: GET 2022-09-03T06:47:45.603 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:47:45.604 DEBUG:boto:Data: 2022-09-03T06:47:45.604 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.605 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:45.606 DEBUG:boto:Port: 8000 2022-09-03T06:47:45.607 DEBUG:boto:Params: {} 2022-09-03T06:47:45.607 DEBUG:boto:Token: None 2022-09-03T06:47:45.608 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/ 2022-09-03T06:47:45.609 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A= 2022-09-03T06:47:45.609 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A=', 'Content-Length': '0'} 2022-09-03T06:47:45.612 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000461be58a0a874666-006312f891-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.613 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:45.614 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:47:45.615 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:45.616 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:45.616 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.617 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.618 DEBUG:boto:Method: GET 2022-09-03T06:47:45.618 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.619 DEBUG:boto:Data: 2022-09-03T06:47:45.620 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.620 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:45.621 DEBUG:boto:Port: 8001 2022-09-03T06:47:45.621 DEBUG:boto:Params: {} 2022-09-03T06:47:45.622 DEBUG:boto:Token: None 2022-09-03T06:47:45.622 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.623 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fm4tfTr/OeznPoN6AFasAIOyVAk= 2022-09-03T06:47:45.624 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fm4tfTr/OeznPoN6AFasAIOyVAk=', 'Content-Length': '0'} 2022-09-03T06:47:45.627 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000031932fed2612d7a7-006312f891-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.666 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:45.667 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:45.667 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.668 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.669 DEBUG:boto:Method: GET 2022-09-03T06:47:45.669 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.670 DEBUG:boto:Data: 2022-09-03T06:47:45.670 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.671 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:45.672 DEBUG:boto:Port: 8000 2022-09-03T06:47:45.672 DEBUG:boto:Params: {} 2022-09-03T06:47:45.673 DEBUG:boto:Token: None 2022-09-03T06:47:45.674 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.674 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fm4tfTr/OeznPoN6AFasAIOyVAk= 2022-09-03T06:47:45.675 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fm4tfTr/OeznPoN6AFasAIOyVAk=', 'Content-Length': '0'} 2022-09-03T06:47:45.679 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000039d045e7c9faaba9-006312f891-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.680 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:45.680 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:45.681 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.681 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.682 DEBUG:boto:Method: HEAD 2022-09-03T06:47:45.683 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.683 DEBUG:boto:Data: 2022-09-03T06:47:45.684 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.684 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:45.685 DEBUG:boto:Port: 8001 2022-09-03T06:47:45.685 DEBUG:boto:Params: {} 2022-09-03T06:47:45.686 DEBUG:boto:Token: None 2022-09-03T06:47:45.687 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.687 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RNstq7gHAeLeRz3+NjcJCj1kK9o= 2022-09-03T06:47:45.688 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RNstq7gHAeLeRz3+NjcJCj1kK9o=', 'Content-Length': '0'} 2022-09-03T06:47:45.690 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d59af24f88cc2512-006312f891-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.691 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:45.692 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:45.692 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.693 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.694 DEBUG:boto:Method: HEAD 2022-09-03T06:47:45.694 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.695 DEBUG:boto:Data: 2022-09-03T06:47:45.695 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.696 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:45.697 DEBUG:boto:Port: 8000 2022-09-03T06:47:45.697 DEBUG:boto:Params: {} 2022-09-03T06:47:45.698 DEBUG:boto:Token: None 2022-09-03T06:47:45.702 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.704 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RNstq7gHAeLeRz3+NjcJCj1kK9o= 2022-09-03T06:47:45.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RNstq7gHAeLeRz3+NjcJCj1kK9o=', 'Content-Length': '0'} 2022-09-03T06:47:45.706 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b76f5c0d0252a9c-006312f891-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.707 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:47:45.708 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:45.708 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:45.709 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.709 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.710 DEBUG:boto:Method: GET 2022-09-03T06:47:45.711 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.711 DEBUG:boto:Data: 2022-09-03T06:47:45.712 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.712 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:45.713 DEBUG:boto:Port: 8001 2022-09-03T06:47:45.713 DEBUG:boto:Params: {} 2022-09-03T06:47:45.714 DEBUG:boto:Token: None 2022-09-03T06:47:45.715 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.716 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fm4tfTr/OeznPoN6AFasAIOyVAk= 2022-09-03T06:47:45.717 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fm4tfTr/OeznPoN6AFasAIOyVAk=', 'Content-Length': '0'} 2022-09-03T06:47:45.720 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7d17c2ef51d5009-006312f891-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.722 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:45.723 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:45.724 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.724 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.725 DEBUG:boto:Method: GET 2022-09-03T06:47:45.725 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.726 DEBUG:boto:Data: 2022-09-03T06:47:45.727 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.728 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:45.729 DEBUG:boto:Port: 8000 2022-09-03T06:47:45.730 DEBUG:boto:Params: {} 2022-09-03T06:47:45.731 DEBUG:boto:Token: None 2022-09-03T06:47:45.732 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:47:45.733 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Fm4tfTr/OeznPoN6AFasAIOyVAk= 2022-09-03T06:47:45.735 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Fm4tfTr/OeznPoN6AFasAIOyVAk=', 'Content-Length': '0'} 2022-09-03T06:47:45.739 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a54454f73d3dab1b-006312f891-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.787 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:45.788 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:45.789 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:45.790 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:45.790 DEBUG:boto:Method: GET 2022-09-03T06:47:45.792 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:47:45.793 DEBUG:boto:Data: 2022-09-03T06:47:45.794 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.795 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:45.800 DEBUG:boto:Port: 8001 2022-09-03T06:47:45.801 DEBUG:boto:Params: {} 2022-09-03T06:47:45.801 DEBUG:boto:Token: None 2022-09-03T06:47:45.802 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/ 2022-09-03T06:47:45.803 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A= 2022-09-03T06:47:45.803 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A=', 'Content-Length': '0'} 2022-09-03T06:47:45.807 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006e7a1ff96daba42f-006312f891-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.808 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:45.812 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:45.812 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:45.813 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:45.813 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:45.814 DEBUG:boto:Method: GET 2022-09-03T06:47:45.815 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:47:45.815 DEBUG:boto:Data: 2022-09-03T06:47:45.816 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.816 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:45.817 DEBUG:boto:Port: 8000 2022-09-03T06:47:45.817 DEBUG:boto:Params: {} 2022-09-03T06:47:45.818 DEBUG:boto:Token: None 2022-09-03T06:47:45.819 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/ 2022-09-03T06:47:45.820 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A= 2022-09-03T06:47:45.821 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A=', 'Content-Length': '0'} 2022-09-03T06:47:45.823 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ebae07a62e3bebdd-006312f891-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.824 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:45.825 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:45.826 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:45.827 DEBUG:boto:Method: HEAD 2022-09-03T06:47:45.827 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:47:45.828 DEBUG:boto:Data: 2022-09-03T06:47:45.828 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.829 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:45.829 DEBUG:boto:Port: 8001 2022-09-03T06:47:45.830 DEBUG:boto:Params: {} 2022-09-03T06:47:45.831 DEBUG:boto:Token: None 2022-09-03T06:47:45.832 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj51 2022-09-03T06:47:45.833 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ovt23GRxz9vfDM1vBIB4aNHHQ/0= 2022-09-03T06:47:45.833 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ovt23GRxz9vfDM1vBIB4aNHHQ/0=', 'Content-Length': '0'} 2022-09-03T06:47:45.836 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000000b4efeecb14a8d0-006312f891-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.838 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:45.839 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:45.839 DEBUG:boto:Method: HEAD 2022-09-03T06:47:45.840 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:47:45.841 DEBUG:boto:Data: 2022-09-03T06:47:45.843 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.844 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:45.845 DEBUG:boto:Port: 8000 2022-09-03T06:47:45.846 DEBUG:boto:Params: {} 2022-09-03T06:47:45.848 DEBUG:boto:Token: None 2022-09-03T06:47:45.849 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj51 2022-09-03T06:47:45.850 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ovt23GRxz9vfDM1vBIB4aNHHQ/0= 2022-09-03T06:47:45.851 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ovt23GRxz9vfDM1vBIB4aNHHQ/0=', 'Content-Length': '0'} 2022-09-03T06:47:45.854 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aa9bb85a005c1434-006312f891-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.855 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:47:45.856 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:45.856 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:45.857 DEBUG:boto:Method: GET 2022-09-03T06:47:45.858 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:47:45.858 DEBUG:boto:Data: 2022-09-03T06:47:45.859 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.860 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:45.860 DEBUG:boto:Port: 8001 2022-09-03T06:47:45.861 DEBUG:boto:Params: {} 2022-09-03T06:47:45.862 DEBUG:boto:Token: None 2022-09-03T06:47:45.863 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj51 2022-09-03T06:47:45.863 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fzewoElE4FmjH4sIfngNuLVt5JM= 2022-09-03T06:47:45.864 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fzewoElE4FmjH4sIfngNuLVt5JM=', 'Content-Length': '0'} 2022-09-03T06:47:45.866 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008a70ec9d4dbb15ed-006312f891-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.911 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:47:45.912 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:47:45.913 DEBUG:boto:Method: GET 2022-09-03T06:47:45.914 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:47:45.915 DEBUG:boto:Data: 2022-09-03T06:47:45.916 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.917 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:45.918 DEBUG:boto:Port: 8000 2022-09-03T06:47:45.919 DEBUG:boto:Params: {} 2022-09-03T06:47:45.920 DEBUG:boto:Token: None 2022-09-03T06:47:45.922 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/obj51 2022-09-03T06:47:45.923 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fzewoElE4FmjH4sIfngNuLVt5JM= 2022-09-03T06:47:45.924 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fzewoElE4FmjH4sIfngNuLVt5JM=', 'Content-Length': '0'} 2022-09-03T06:47:45.928 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d9a0138ed24b440d-006312f891-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.929 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:45.930 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:45.930 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:45.931 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:45.931 DEBUG:boto:Method: GET 2022-09-03T06:47:45.932 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:47:45.934 DEBUG:boto:Data: 2022-09-03T06:47:45.935 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.937 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:45.938 DEBUG:boto:Port: 8001 2022-09-03T06:47:45.938 DEBUG:boto:Params: {} 2022-09-03T06:47:45.939 DEBUG:boto:Token: None 2022-09-03T06:47:45.940 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/ 2022-09-03T06:47:45.942 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A= 2022-09-03T06:47:45.942 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A=', 'Content-Length': '0'} 2022-09-03T06:47:45.945 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e16266e6811d6b73-006312f891-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:45.946 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:45.947 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:45.948 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:45.949 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:45.949 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:47:45.950 DEBUG:boto:Method: GET 2022-09-03T06:47:45.951 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:47:45.951 DEBUG:boto:Data: 2022-09-03T06:47:45.952 DEBUG:boto:Headers: {} 2022-09-03T06:47:45.952 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:45.953 DEBUG:boto:Port: 8000 2022-09-03T06:47:45.953 DEBUG:boto:Params: {} 2022-09-03T06:47:45.954 DEBUG:boto:Token: None 2022-09-03T06:47:45.955 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:45 GMT /dsgvcy-75/ 2022-09-03T06:47:45.955 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A= 2022-09-03T06:47:45.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:45 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IV458o+9mCBGXMT3gcVqXK1xT9A=', 'Content-Length': '0'} 2022-09-03T06:47:45.959 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006d0abb4e02f4c6a3-006312f891-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:45 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.002 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:46.004 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:47:46.004 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:46.005 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:46.006 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.007 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.007 DEBUG:boto:Method: GET 2022-09-03T06:47:46.008 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.008 DEBUG:boto:Data: 2022-09-03T06:47:46.009 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.010 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.010 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.011 DEBUG:boto:Params: {} 2022-09-03T06:47:46.011 DEBUG:boto:Token: None 2022-09-03T06:47:46.012 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.013 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gksOWcpWGw+BZIlmL1zwZ2DNk4k= 2022-09-03T06:47:46.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gksOWcpWGw+BZIlmL1zwZ2DNk4k=', 'Content-Length': '0'} 2022-09-03T06:47:46.017 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a91f5527ebad04f-006312f892-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.018 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:46.019 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:46.020 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.020 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.021 DEBUG:boto:Method: GET 2022-09-03T06:47:46.021 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.022 DEBUG:boto:Data: 2022-09-03T06:47:46.022 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.023 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.023 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.024 DEBUG:boto:Params: {} 2022-09-03T06:47:46.025 DEBUG:boto:Token: None 2022-09-03T06:47:46.025 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.026 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gksOWcpWGw+BZIlmL1zwZ2DNk4k= 2022-09-03T06:47:46.027 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gksOWcpWGw+BZIlmL1zwZ2DNk4k=', 'Content-Length': '0'} 2022-09-03T06:47:46.030 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b01e9d089340797-006312f892-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.031 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:46.031 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:46.032 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.032 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.033 DEBUG:boto:Method: HEAD 2022-09-03T06:47:46.033 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.034 DEBUG:boto:Data: 2022-09-03T06:47:46.034 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.035 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.035 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.036 DEBUG:boto:Params: {} 2022-09-03T06:47:46.037 DEBUG:boto:Token: None 2022-09-03T06:47:46.037 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.038 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NpVK0VCuVUcHMljL1hjKbUGKZWY= 2022-09-03T06:47:46.039 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NpVK0VCuVUcHMljL1hjKbUGKZWY=', 'Content-Length': '0'} 2022-09-03T06:47:46.041 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000678754b82d7a579f-006312f892-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.042 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:46.042 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:46.043 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.044 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.044 DEBUG:boto:Method: HEAD 2022-09-03T06:47:46.045 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.045 DEBUG:boto:Data: 2022-09-03T06:47:46.046 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.046 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.047 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.048 DEBUG:boto:Params: {} 2022-09-03T06:47:46.048 DEBUG:boto:Token: None 2022-09-03T06:47:46.049 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.049 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NpVK0VCuVUcHMljL1hjKbUGKZWY= 2022-09-03T06:47:46.050 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NpVK0VCuVUcHMljL1hjKbUGKZWY=', 'Content-Length': '0'} 2022-09-03T06:47:46.052 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007693214b1092c28c-006312f892-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.053 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:47:46.054 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:46.055 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:46.055 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.056 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.056 DEBUG:boto:Method: GET 2022-09-03T06:47:46.058 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.058 DEBUG:boto:Data: 2022-09-03T06:47:46.059 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.059 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.060 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.060 DEBUG:boto:Params: {} 2022-09-03T06:47:46.061 DEBUG:boto:Token: None 2022-09-03T06:47:46.062 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.062 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gksOWcpWGw+BZIlmL1zwZ2DNk4k= 2022-09-03T06:47:46.063 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gksOWcpWGw+BZIlmL1zwZ2DNk4k=', 'Content-Length': '0'} 2022-09-03T06:47:46.065 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a6391bf945d538aa-006312f892-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.066 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:46.067 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:46.069 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.070 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.071 DEBUG:boto:Method: GET 2022-09-03T06:47:46.072 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.073 DEBUG:boto:Data: 2022-09-03T06:47:46.074 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.075 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.076 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.076 DEBUG:boto:Params: {} 2022-09-03T06:47:46.077 DEBUG:boto:Token: None 2022-09-03T06:47:46.079 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:47:46.080 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gksOWcpWGw+BZIlmL1zwZ2DNk4k= 2022-09-03T06:47:46.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gksOWcpWGw+BZIlmL1zwZ2DNk4k=', 'Content-Length': '0'} 2022-09-03T06:47:46.083 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009c30efcb30dcdf14-006312f892-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.084 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:46.085 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:46.085 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:46.086 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:46.086 DEBUG:boto:Method: GET 2022-09-03T06:47:46.087 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:47:46.087 DEBUG:boto:Data: 2022-09-03T06:47:46.088 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.089 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.090 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.091 DEBUG:boto:Params: {} 2022-09-03T06:47:46.092 DEBUG:boto:Token: None 2022-09-03T06:47:46.093 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/ 2022-09-03T06:47:46.095 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g= 2022-09-03T06:47:46.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g=', 'Content-Length': '0'} 2022-09-03T06:47:46.100 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003cbfa90da2e9c9a8-006312f892-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.102 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:46.103 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:46.104 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:46.105 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:46.106 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:46.107 DEBUG:boto:Method: GET 2022-09-03T06:47:46.108 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:47:46.109 DEBUG:boto:Data: 2022-09-03T06:47:46.110 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.111 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.112 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.113 DEBUG:boto:Params: {} 2022-09-03T06:47:46.114 DEBUG:boto:Token: None 2022-09-03T06:47:46.115 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/ 2022-09-03T06:47:46.116 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g= 2022-09-03T06:47:46.118 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g=', 'Content-Length': '0'} 2022-09-03T06:47:46.123 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007beab5b0ecd3df83-006312f892-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.124 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:46.124 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:46.125 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:46.125 DEBUG:boto:Method: HEAD 2022-09-03T06:47:46.127 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:47:46.127 DEBUG:boto:Data: 2022-09-03T06:47:46.128 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.129 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.134 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.135 DEBUG:boto:Params: {} 2022-09-03T06:47:46.135 DEBUG:boto:Token: None 2022-09-03T06:47:46.136 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj55 2022-09-03T06:47:46.136 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8Y2qE5qcI4hh0KukX/USTOpMYUY= 2022-09-03T06:47:46.137 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8Y2qE5qcI4hh0KukX/USTOpMYUY=', 'Content-Length': '0'} 2022-09-03T06:47:46.140 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bce7a7b1a5c95e69-006312f892-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.142 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:46.143 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:46.144 DEBUG:boto:Method: HEAD 2022-09-03T06:47:46.145 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:47:46.146 DEBUG:boto:Data: 2022-09-03T06:47:46.146 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.147 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.147 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.148 DEBUG:boto:Params: {} 2022-09-03T06:47:46.150 DEBUG:boto:Token: None 2022-09-03T06:47:46.151 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj55 2022-09-03T06:47:46.152 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8Y2qE5qcI4hh0KukX/USTOpMYUY= 2022-09-03T06:47:46.153 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8Y2qE5qcI4hh0KukX/USTOpMYUY=', 'Content-Length': '0'} 2022-09-03T06:47:46.156 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000cb748e2f747894f-006312f892-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.157 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:47:46.158 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:46.159 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:46.160 DEBUG:boto:Method: GET 2022-09-03T06:47:46.161 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:47:46.162 DEBUG:boto:Data: 2022-09-03T06:47:46.164 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.165 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.166 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.167 DEBUG:boto:Params: {} 2022-09-03T06:47:46.167 DEBUG:boto:Token: None 2022-09-03T06:47:46.168 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj55 2022-09-03T06:47:46.169 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uPYiupwKGHnyi3tPv3EaiWlGtko= 2022-09-03T06:47:46.170 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uPYiupwKGHnyi3tPv3EaiWlGtko=', 'Content-Length': '0'} 2022-09-03T06:47:46.173 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000083dbd5ea1c04e66f-006312f892-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.174 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:47:46.175 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:47:46.176 DEBUG:boto:Method: GET 2022-09-03T06:47:46.177 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:47:46.177 DEBUG:boto:Data: 2022-09-03T06:47:46.178 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.179 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.180 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.181 DEBUG:boto:Params: {} 2022-09-03T06:47:46.181 DEBUG:boto:Token: None 2022-09-03T06:47:46.182 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj55 2022-09-03T06:47:46.182 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uPYiupwKGHnyi3tPv3EaiWlGtko= 2022-09-03T06:47:46.183 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uPYiupwKGHnyi3tPv3EaiWlGtko=', 'Content-Length': '0'} 2022-09-03T06:47:46.185 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006f84da9738442f47-006312f892-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.232 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:46.233 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:46.234 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:46.235 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:46.236 DEBUG:boto:Method: GET 2022-09-03T06:47:46.237 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:47:46.237 DEBUG:boto:Data: 2022-09-03T06:47:46.238 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.238 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.239 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.240 DEBUG:boto:Params: {} 2022-09-03T06:47:46.240 DEBUG:boto:Token: None 2022-09-03T06:47:46.241 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/ 2022-09-03T06:47:46.242 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g= 2022-09-03T06:47:46.247 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g=', 'Content-Length': '0'} 2022-09-03T06:47:46.250 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d772fdce1f342f0-006312f892-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.251 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:46.252 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:46.253 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:46.253 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:46.254 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:47:46.254 DEBUG:boto:Method: GET 2022-09-03T06:47:46.255 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:47:46.255 DEBUG:boto:Data: 2022-09-03T06:47:46.256 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.256 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.257 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.257 DEBUG:boto:Params: {} 2022-09-03T06:47:46.258 DEBUG:boto:Token: None 2022-09-03T06:47:46.259 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/ 2022-09-03T06:47:46.259 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g= 2022-09-03T06:47:46.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g=', 'Content-Length': '0'} 2022-09-03T06:47:46.263 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b0928a64873030db-006312f892-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.264 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:46.265 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:47:46.265 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:46.266 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:46.267 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.267 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.268 DEBUG:boto:Method: GET 2022-09-03T06:47:46.268 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.269 DEBUG:boto:Data: 2022-09-03T06:47:46.269 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.270 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.270 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.271 DEBUG:boto:Params: {} 2022-09-03T06:47:46.272 DEBUG:boto:Token: None 2022-09-03T06:47:46.272 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.273 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j4FgMTC+nmILYJ0I0Co4EnXhkuY= 2022-09-03T06:47:46.274 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j4FgMTC+nmILYJ0I0Co4EnXhkuY=', 'Content-Length': '0'} 2022-09-03T06:47:46.276 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d9febc165de8a78-006312f892-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.318 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:46.320 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:46.321 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.322 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.323 DEBUG:boto:Method: GET 2022-09-03T06:47:46.324 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.326 DEBUG:boto:Data: 2022-09-03T06:47:46.327 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.328 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.332 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.333 DEBUG:boto:Params: {} 2022-09-03T06:47:46.335 DEBUG:boto:Token: None 2022-09-03T06:47:46.336 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.337 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j4FgMTC+nmILYJ0I0Co4EnXhkuY= 2022-09-03T06:47:46.338 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j4FgMTC+nmILYJ0I0Co4EnXhkuY=', 'Content-Length': '0'} 2022-09-03T06:47:46.340 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008d016a377bc576b8-006312f892-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.342 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:46.343 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:46.344 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.345 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.346 DEBUG:boto:Method: HEAD 2022-09-03T06:47:46.347 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.348 DEBUG:boto:Data: 2022-09-03T06:47:46.349 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.350 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.351 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.352 DEBUG:boto:Params: {} 2022-09-03T06:47:46.353 DEBUG:boto:Token: None 2022-09-03T06:47:46.356 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.358 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wEPP8uqGWScJhB0CMvHd6F7vu5Q= 2022-09-03T06:47:46.359 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wEPP8uqGWScJhB0CMvHd6F7vu5Q=', 'Content-Length': '0'} 2022-09-03T06:47:46.361 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000027f87e75b68b5ac2-006312f892-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.362 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:46.364 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:46.365 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.366 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.367 DEBUG:boto:Method: HEAD 2022-09-03T06:47:46.368 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.369 DEBUG:boto:Data: 2022-09-03T06:47:46.370 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.371 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.372 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.372 DEBUG:boto:Params: {} 2022-09-03T06:47:46.373 DEBUG:boto:Token: None 2022-09-03T06:47:46.374 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.375 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wEPP8uqGWScJhB0CMvHd6F7vu5Q= 2022-09-03T06:47:46.375 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wEPP8uqGWScJhB0CMvHd6F7vu5Q=', 'Content-Length': '0'} 2022-09-03T06:47:46.378 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008418009c774295c6-006312f892-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.378 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:47:46.379 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:46.380 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:46.380 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.381 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.381 DEBUG:boto:Method: GET 2022-09-03T06:47:46.382 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.382 DEBUG:boto:Data: 2022-09-03T06:47:46.383 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.384 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.384 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.385 DEBUG:boto:Params: {} 2022-09-03T06:47:46.386 DEBUG:boto:Token: None 2022-09-03T06:47:46.386 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.387 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j4FgMTC+nmILYJ0I0Co4EnXhkuY= 2022-09-03T06:47:46.388 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j4FgMTC+nmILYJ0I0Co4EnXhkuY=', 'Content-Length': '0'} 2022-09-03T06:47:46.390 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e7ba0d443095617b-006312f892-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.391 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:46.391 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:46.392 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.393 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.393 DEBUG:boto:Method: GET 2022-09-03T06:47:46.394 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.394 DEBUG:boto:Data: 2022-09-03T06:47:46.395 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.395 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.396 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.397 DEBUG:boto:Params: {} 2022-09-03T06:47:46.397 DEBUG:boto:Token: None 2022-09-03T06:47:46.398 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:47:46.398 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j4FgMTC+nmILYJ0I0Co4EnXhkuY= 2022-09-03T06:47:46.399 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j4FgMTC+nmILYJ0I0Co4EnXhkuY=', 'Content-Length': '0'} 2022-09-03T06:47:46.402 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006e7671ad0a6a1c8e-006312f892-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.402 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:46.403 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:46.404 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:46.404 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:46.405 DEBUG:boto:Method: GET 2022-09-03T06:47:46.405 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:47:46.406 DEBUG:boto:Data: 2022-09-03T06:47:46.406 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.407 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.408 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.408 DEBUG:boto:Params: {} 2022-09-03T06:47:46.409 DEBUG:boto:Token: None 2022-09-03T06:47:46.409 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/ 2022-09-03T06:47:46.410 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g= 2022-09-03T06:47:46.411 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g=', 'Content-Length': '0'} 2022-09-03T06:47:46.414 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ae5e425ca3374448-006312f892-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.414 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:46.415 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:46.416 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:46.416 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:46.417 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:46.417 DEBUG:boto:Method: GET 2022-09-03T06:47:46.418 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:47:46.418 DEBUG:boto:Data: 2022-09-03T06:47:46.419 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.419 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.420 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.421 DEBUG:boto:Params: {} 2022-09-03T06:47:46.421 DEBUG:boto:Token: None 2022-09-03T06:47:46.422 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/ 2022-09-03T06:47:46.422 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g= 2022-09-03T06:47:46.423 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g=', 'Content-Length': '0'} 2022-09-03T06:47:46.426 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ed48791e2cf12232-006312f892-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.427 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:46.429 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:46.430 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:46.431 DEBUG:boto:Method: HEAD 2022-09-03T06:47:46.431 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:47:46.432 DEBUG:boto:Data: 2022-09-03T06:47:46.433 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.434 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.435 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.436 DEBUG:boto:Params: {} 2022-09-03T06:47:46.437 DEBUG:boto:Token: None 2022-09-03T06:47:46.438 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj57 2022-09-03T06:47:46.439 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:x5wXcYbZC9D1K6gtcw5Z7N9mQO4= 2022-09-03T06:47:46.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:x5wXcYbZC9D1K6gtcw5Z7N9mQO4=', 'Content-Length': '0'} 2022-09-03T06:47:46.443 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a2eb8df421bd98d5-006312f892-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.444 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:46.445 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:46.446 DEBUG:boto:Method: HEAD 2022-09-03T06:47:46.447 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:47:46.448 DEBUG:boto:Data: 2022-09-03T06:47:46.449 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.450 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.451 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.451 DEBUG:boto:Params: {} 2022-09-03T06:47:46.453 DEBUG:boto:Token: None 2022-09-03T06:47:46.455 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj57 2022-09-03T06:47:46.456 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:x5wXcYbZC9D1K6gtcw5Z7N9mQO4= 2022-09-03T06:47:46.457 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:x5wXcYbZC9D1K6gtcw5Z7N9mQO4=', 'Content-Length': '0'} 2022-09-03T06:47:46.459 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000002bee68c7b28ce94-006312f892-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.460 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:47:46.461 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:46.462 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:46.463 DEBUG:boto:Method: GET 2022-09-03T06:47:46.463 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:47:46.464 DEBUG:boto:Data: 2022-09-03T06:47:46.465 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.466 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.467 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.467 DEBUG:boto:Params: {} 2022-09-03T06:47:46.468 DEBUG:boto:Token: None 2022-09-03T06:47:46.469 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj57 2022-09-03T06:47:46.470 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SbdQuxQqMgyZZ+WJ4YZaJb9qbjg= 2022-09-03T06:47:46.471 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SbdQuxQqMgyZZ+WJ4YZaJb9qbjg=', 'Content-Length': '0'} 2022-09-03T06:47:46.474 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002e4f11cdd8f84258-006312f892-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.476 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:47:46.477 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:47:46.478 DEBUG:boto:Method: GET 2022-09-03T06:47:46.479 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:47:46.480 DEBUG:boto:Data: 2022-09-03T06:47:46.481 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.482 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.483 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.483 DEBUG:boto:Params: {} 2022-09-03T06:47:46.485 DEBUG:boto:Token: None 2022-09-03T06:47:46.486 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj57 2022-09-03T06:47:46.487 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SbdQuxQqMgyZZ+WJ4YZaJb9qbjg= 2022-09-03T06:47:46.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SbdQuxQqMgyZZ+WJ4YZaJb9qbjg=', 'Content-Length': '0'} 2022-09-03T06:47:46.490 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a912babf750009e-006312f892-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.492 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:46.492 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:46.493 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:46.494 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:46.494 DEBUG:boto:Method: GET 2022-09-03T06:47:46.495 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:47:46.496 DEBUG:boto:Data: 2022-09-03T06:47:46.496 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.497 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.498 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.498 DEBUG:boto:Params: {} 2022-09-03T06:47:46.499 DEBUG:boto:Token: None 2022-09-03T06:47:46.500 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/ 2022-09-03T06:47:46.501 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g= 2022-09-03T06:47:46.502 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g=', 'Content-Length': '0'} 2022-09-03T06:47:46.505 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bf209c70ef7ddce6-006312f892-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.546 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:46.547 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:46.548 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:46.549 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:46.550 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:47:46.550 DEBUG:boto:Method: GET 2022-09-03T06:47:46.551 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:47:46.552 DEBUG:boto:Data: 2022-09-03T06:47:46.553 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.554 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.555 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.556 DEBUG:boto:Params: {} 2022-09-03T06:47:46.557 DEBUG:boto:Token: None 2022-09-03T06:47:46.558 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/ 2022-09-03T06:47:46.560 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g= 2022-09-03T06:47:46.561 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g=', 'Content-Length': '0'} 2022-09-03T06:47:46.564 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000da65ac30a88449ac-006312f892-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.565 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:46.567 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:47:46.568 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:46.569 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:46.570 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.570 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.571 DEBUG:boto:Method: GET 2022-09-03T06:47:46.572 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.573 DEBUG:boto:Data: 2022-09-03T06:47:46.574 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.575 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.576 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.577 DEBUG:boto:Params: {} 2022-09-03T06:47:46.577 DEBUG:boto:Token: None 2022-09-03T06:47:46.578 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.580 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zlnOMu5sZGGY8w6/MjVw2Jn9XSs= 2022-09-03T06:47:46.581 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zlnOMu5sZGGY8w6/MjVw2Jn9XSs=', 'Content-Length': '0'} 2022-09-03T06:47:46.584 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d7e43ff934ba988f-006312f892-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.585 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:46.586 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:46.587 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.588 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.589 DEBUG:boto:Method: GET 2022-09-03T06:47:46.590 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.591 DEBUG:boto:Data: 2022-09-03T06:47:46.592 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.593 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.594 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.595 DEBUG:boto:Params: {} 2022-09-03T06:47:46.596 DEBUG:boto:Token: None 2022-09-03T06:47:46.598 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.599 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zlnOMu5sZGGY8w6/MjVw2Jn9XSs= 2022-09-03T06:47:46.600 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zlnOMu5sZGGY8w6/MjVw2Jn9XSs=', 'Content-Length': '0'} 2022-09-03T06:47:46.603 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000383f3dcc9ee83f9a-006312f892-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.604 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:46.605 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:46.606 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.607 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.608 DEBUG:boto:Method: HEAD 2022-09-03T06:47:46.608 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.609 DEBUG:boto:Data: 2022-09-03T06:47:46.610 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.611 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.612 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.612 DEBUG:boto:Params: {} 2022-09-03T06:47:46.614 DEBUG:boto:Token: None 2022-09-03T06:47:46.615 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.616 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lRhzappR/jj3rv6P7qcu6xFbt9o= 2022-09-03T06:47:46.617 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lRhzappR/jj3rv6P7qcu6xFbt9o=', 'Content-Length': '0'} 2022-09-03T06:47:46.619 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a14e1f471a99122b-006312f892-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.620 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:46.621 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:46.621 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.622 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.622 DEBUG:boto:Method: HEAD 2022-09-03T06:47:46.623 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.623 DEBUG:boto:Data: 2022-09-03T06:47:46.624 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.624 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.625 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.625 DEBUG:boto:Params: {} 2022-09-03T06:47:46.626 DEBUG:boto:Token: None 2022-09-03T06:47:46.627 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.627 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lRhzappR/jj3rv6P7qcu6xFbt9o= 2022-09-03T06:47:46.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lRhzappR/jj3rv6P7qcu6xFbt9o=', 'Content-Length': '0'} 2022-09-03T06:47:46.630 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee952338c8262056-006312f892-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.631 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:47:46.631 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:46.632 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:46.633 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.634 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.634 DEBUG:boto:Method: GET 2022-09-03T06:47:46.635 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.636 DEBUG:boto:Data: 2022-09-03T06:47:46.636 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.637 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.637 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.638 DEBUG:boto:Params: {} 2022-09-03T06:47:46.638 DEBUG:boto:Token: None 2022-09-03T06:47:46.639 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.640 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zlnOMu5sZGGY8w6/MjVw2Jn9XSs= 2022-09-03T06:47:46.640 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zlnOMu5sZGGY8w6/MjVw2Jn9XSs=', 'Content-Length': '0'} 2022-09-03T06:47:46.643 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006834f4693862147b-006312f892-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.644 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:46.645 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:46.646 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.647 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.648 DEBUG:boto:Method: GET 2022-09-03T06:47:46.649 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.650 DEBUG:boto:Data: 2022-09-03T06:47:46.651 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.652 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.653 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.653 DEBUG:boto:Params: {} 2022-09-03T06:47:46.654 DEBUG:boto:Token: None 2022-09-03T06:47:46.655 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:47:46.655 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zlnOMu5sZGGY8w6/MjVw2Jn9XSs= 2022-09-03T06:47:46.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zlnOMu5sZGGY8w6/MjVw2Jn9XSs=', 'Content-Length': '0'} 2022-09-03T06:47:46.658 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b5c858f41186e4a-006312f892-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.702 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:46.703 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:46.704 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:46.704 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:46.705 DEBUG:boto:Method: GET 2022-09-03T06:47:46.706 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:47:46.707 DEBUG:boto:Data: 2022-09-03T06:47:46.708 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.709 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.709 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.710 DEBUG:boto:Params: {} 2022-09-03T06:47:46.711 DEBUG:boto:Token: None 2022-09-03T06:47:46.711 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/ 2022-09-03T06:47:46.712 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g= 2022-09-03T06:47:46.713 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g=', 'Content-Length': '0'} 2022-09-03T06:47:46.716 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f47c070f237a843f-006312f892-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.717 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:46.718 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:46.719 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:46.719 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:46.720 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:46.720 DEBUG:boto:Method: GET 2022-09-03T06:47:46.721 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:47:46.721 DEBUG:boto:Data: 2022-09-03T06:47:46.722 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.722 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.723 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.723 DEBUG:boto:Params: {} 2022-09-03T06:47:46.724 DEBUG:boto:Token: None 2022-09-03T06:47:46.725 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/ 2022-09-03T06:47:46.725 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g= 2022-09-03T06:47:46.726 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g=', 'Content-Length': '0'} 2022-09-03T06:47:46.729 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000697cfecb74857dc5-006312f892-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.730 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:46.731 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:46.731 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:46.732 DEBUG:boto:Method: HEAD 2022-09-03T06:47:46.732 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:47:46.733 DEBUG:boto:Data: 2022-09-03T06:47:46.734 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.734 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.735 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.735 DEBUG:boto:Params: {} 2022-09-03T06:47:46.736 DEBUG:boto:Token: None 2022-09-03T06:47:46.737 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj58 2022-09-03T06:47:46.737 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i6zdlZ+s2sxGJku0LbkGcOncTGQ= 2022-09-03T06:47:46.738 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i6zdlZ+s2sxGJku0LbkGcOncTGQ=', 'Content-Length': '0'} 2022-09-03T06:47:46.740 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001f955d040fc692d9-006312f892-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.740 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:46.741 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:46.741 DEBUG:boto:Method: HEAD 2022-09-03T06:47:46.742 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:47:46.743 DEBUG:boto:Data: 2022-09-03T06:47:46.743 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.744 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.744 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.744 DEBUG:boto:Params: {} 2022-09-03T06:47:46.745 DEBUG:boto:Token: None 2022-09-03T06:47:46.746 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj58 2022-09-03T06:47:46.746 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i6zdlZ+s2sxGJku0LbkGcOncTGQ= 2022-09-03T06:47:46.747 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i6zdlZ+s2sxGJku0LbkGcOncTGQ=', 'Content-Length': '0'} 2022-09-03T06:47:46.749 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e81883c92289b34e-006312f892-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.749 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:47:46.750 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:46.751 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:46.751 DEBUG:boto:Method: GET 2022-09-03T06:47:46.752 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:47:46.752 DEBUG:boto:Data: 2022-09-03T06:47:46.753 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.753 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.754 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.754 DEBUG:boto:Params: {} 2022-09-03T06:47:46.755 DEBUG:boto:Token: None 2022-09-03T06:47:46.756 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj58 2022-09-03T06:47:46.756 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QN3tUHbJHoX4EUAlLtrX9cM7J3g= 2022-09-03T06:47:46.757 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QN3tUHbJHoX4EUAlLtrX9cM7J3g=', 'Content-Length': '0'} 2022-09-03T06:47:46.759 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d766037577458de5-006312f892-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.760 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:47:46.761 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:47:46.761 DEBUG:boto:Method: GET 2022-09-03T06:47:46.762 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:47:46.763 DEBUG:boto:Data: 2022-09-03T06:47:46.763 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.764 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.765 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.766 DEBUG:boto:Params: {} 2022-09-03T06:47:46.767 DEBUG:boto:Token: None 2022-09-03T06:47:46.769 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj58 2022-09-03T06:47:46.770 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QN3tUHbJHoX4EUAlLtrX9cM7J3g= 2022-09-03T06:47:46.771 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QN3tUHbJHoX4EUAlLtrX9cM7J3g=', 'Content-Length': '0'} 2022-09-03T06:47:46.774 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008546a1492af54e59-006312f892-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.775 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:46.776 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:46.776 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:46.777 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:46.777 DEBUG:boto:Method: GET 2022-09-03T06:47:46.778 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:47:46.778 DEBUG:boto:Data: 2022-09-03T06:47:46.779 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.780 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.780 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.781 DEBUG:boto:Params: {} 2022-09-03T06:47:46.781 DEBUG:boto:Token: None 2022-09-03T06:47:46.782 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/ 2022-09-03T06:47:46.783 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g= 2022-09-03T06:47:46.784 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g=', 'Content-Length': '0'} 2022-09-03T06:47:46.787 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000092547c5a8810a34c-006312f892-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.788 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:46.790 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:46.790 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:46.791 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:46.792 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:47:46.792 DEBUG:boto:Method: GET 2022-09-03T06:47:46.793 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:47:46.794 DEBUG:boto:Data: 2022-09-03T06:47:46.794 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.795 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.795 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.796 DEBUG:boto:Params: {} 2022-09-03T06:47:46.797 DEBUG:boto:Token: None 2022-09-03T06:47:46.797 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/ 2022-09-03T06:47:46.798 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g= 2022-09-03T06:47:46.799 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g=', 'Content-Length': '0'} 2022-09-03T06:47:46.801 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000015f765a3f288712-006312f892-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.803 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:46.804 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2022-09-03T06:47:46.805 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:46.806 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:46.807 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.808 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.809 DEBUG:boto:Method: GET 2022-09-03T06:47:46.810 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.810 DEBUG:boto:Data: 2022-09-03T06:47:46.811 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.811 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.812 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.812 DEBUG:boto:Params: {} 2022-09-03T06:47:46.813 DEBUG:boto:Token: None 2022-09-03T06:47:46.813 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.814 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ls/ByXojF4mWU/o2LItIP7sHAiY= 2022-09-03T06:47:46.815 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ls/ByXojF4mWU/o2LItIP7sHAiY=', 'Content-Length': '0'} 2022-09-03T06:47:46.817 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a14abe79df15ee40-006312f892-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.862 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:46.863 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:46.864 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.864 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.865 DEBUG:boto:Method: GET 2022-09-03T06:47:46.866 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.867 DEBUG:boto:Data: 2022-09-03T06:47:46.867 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.868 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.869 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.869 DEBUG:boto:Params: {} 2022-09-03T06:47:46.870 DEBUG:boto:Token: None 2022-09-03T06:47:46.872 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.873 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ls/ByXojF4mWU/o2LItIP7sHAiY= 2022-09-03T06:47:46.874 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ls/ByXojF4mWU/o2LItIP7sHAiY=', 'Content-Length': '0'} 2022-09-03T06:47:46.876 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000648bad2c09ff4218-006312f892-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.878 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:46.879 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:46.880 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.880 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.881 DEBUG:boto:Method: HEAD 2022-09-03T06:47:46.881 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.884 DEBUG:boto:Data: 2022-09-03T06:47:46.884 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.885 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.886 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.886 DEBUG:boto:Params: {} 2022-09-03T06:47:46.887 DEBUG:boto:Token: None 2022-09-03T06:47:46.888 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.888 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mJ2G+IV2coNTNh5KAFg4Ome0vMA= 2022-09-03T06:47:46.889 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mJ2G+IV2coNTNh5KAFg4Ome0vMA=', 'Content-Length': '0'} 2022-09-03T06:47:46.891 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d2d4ef07e7e322f-006312f892-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.892 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:46.893 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:46.894 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.895 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.895 DEBUG:boto:Method: HEAD 2022-09-03T06:47:46.896 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.896 DEBUG:boto:Data: 2022-09-03T06:47:46.897 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.898 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.898 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.899 DEBUG:boto:Params: {} 2022-09-03T06:47:46.900 DEBUG:boto:Token: None 2022-09-03T06:47:46.900 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.901 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mJ2G+IV2coNTNh5KAFg4Ome0vMA= 2022-09-03T06:47:46.902 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mJ2G+IV2coNTNh5KAFg4Ome0vMA=', 'Content-Length': '0'} 2022-09-03T06:47:46.904 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d9048b940ec083df-006312f892-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.906 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2022-09-03T06:47:46.907 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:46.908 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:46.908 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.909 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.910 DEBUG:boto:Method: GET 2022-09-03T06:47:46.911 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.912 DEBUG:boto:Data: 2022-09-03T06:47:46.913 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.916 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.917 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.918 DEBUG:boto:Params: {} 2022-09-03T06:47:46.919 DEBUG:boto:Token: None 2022-09-03T06:47:46.920 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.921 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ls/ByXojF4mWU/o2LItIP7sHAiY= 2022-09-03T06:47:46.922 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ls/ByXojF4mWU/o2LItIP7sHAiY=', 'Content-Length': '0'} 2022-09-03T06:47:46.926 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000029a3a8f2c8c5ea7-006312f892-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.928 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:46.929 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:46.929 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.930 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.930 DEBUG:boto:Method: GET 2022-09-03T06:47:46.931 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.932 DEBUG:boto:Data: 2022-09-03T06:47:46.932 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.933 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.935 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.935 DEBUG:boto:Params: {} 2022-09-03T06:47:46.936 DEBUG:boto:Token: None 2022-09-03T06:47:46.937 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:47:46.937 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ls/ByXojF4mWU/o2LItIP7sHAiY= 2022-09-03T06:47:46.938 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ls/ByXojF4mWU/o2LItIP7sHAiY=', 'Content-Length': '0'} 2022-09-03T06:47:46.940 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000041b2c2827736098f-006312f892-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.942 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:46.942 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:46.943 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:46.944 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:46.944 DEBUG:boto:Method: GET 2022-09-03T06:47:46.945 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:47:46.946 DEBUG:boto:Data: 2022-09-03T06:47:46.946 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.947 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.948 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.948 DEBUG:boto:Params: {} 2022-09-03T06:47:46.949 DEBUG:boto:Token: None 2022-09-03T06:47:46.950 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/ 2022-09-03T06:47:46.951 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g= 2022-09-03T06:47:46.952 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g=', 'Content-Length': '0'} 2022-09-03T06:47:46.955 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000602fa026acb9a988-006312f892-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.956 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:46.958 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:46.958 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:46.959 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:46.960 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:46.960 DEBUG:boto:Method: GET 2022-09-03T06:47:46.961 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:47:46.962 DEBUG:boto:Data: 2022-09-03T06:47:46.962 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.963 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.964 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.965 DEBUG:boto:Params: {} 2022-09-03T06:47:46.966 DEBUG:boto:Token: None 2022-09-03T06:47:46.967 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/ 2022-09-03T06:47:46.969 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g= 2022-09-03T06:47:46.970 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j9WLKqnT1b2Ld6ZmQJwgFLP1a0g=', 'Content-Length': '0'} 2022-09-03T06:47:46.974 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000086911cc927af82ac-006312f892-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.976 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:46.977 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:46.980 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:46.981 DEBUG:boto:Method: HEAD 2022-09-03T06:47:46.982 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:47:46.983 DEBUG:boto:Data: 2022-09-03T06:47:46.985 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.987 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:46.988 DEBUG:boto:Port: 8001 2022-09-03T06:47:46.988 DEBUG:boto:Params: {} 2022-09-03T06:47:46.989 DEBUG:boto:Token: None 2022-09-03T06:47:46.989 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj62 2022-09-03T06:47:46.990 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TnUk6UW1omuGtHHy0bdrFZswg8c= 2022-09-03T06:47:46.991 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TnUk6UW1omuGtHHy0bdrFZswg8c=', 'Content-Length': '0'} 2022-09-03T06:47:46.993 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f95c61d85ac9e3ff-006312f892-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:46.993 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:46.994 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:46.995 DEBUG:boto:Method: HEAD 2022-09-03T06:47:46.995 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:47:46.996 DEBUG:boto:Data: 2022-09-03T06:47:46.996 DEBUG:boto:Headers: {} 2022-09-03T06:47:46.997 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:46.997 DEBUG:boto:Port: 8000 2022-09-03T06:47:46.998 DEBUG:boto:Params: {} 2022-09-03T06:47:46.998 DEBUG:boto:Token: None 2022-09-03T06:47:46.999 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:46 GMT /dsgvcy-75/obj62 2022-09-03T06:47:46.999 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TnUk6UW1omuGtHHy0bdrFZswg8c= 2022-09-03T06:47:47.000 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TnUk6UW1omuGtHHy0bdrFZswg8c=', 'Content-Length': '0'} 2022-09-03T06:47:47.002 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001e93c662f53c62a1-006312f893-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.004 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2022-09-03T06:47:47.005 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:47.006 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:47.007 DEBUG:boto:Method: GET 2022-09-03T06:47:47.008 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:47:47.009 DEBUG:boto:Data: 2022-09-03T06:47:47.011 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.011 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.012 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.013 DEBUG:boto:Params: {} 2022-09-03T06:47:47.015 DEBUG:boto:Token: None 2022-09-03T06:47:47.016 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj62 2022-09-03T06:47:47.017 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ciM71gmc6c8K01xAIBkttZUzCC0= 2022-09-03T06:47:47.018 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ciM71gmc6c8K01xAIBkttZUzCC0=', 'Content-Length': '0'} 2022-09-03T06:47:47.021 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f7c7f5d1729b135a-006312f893-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.062 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:47:47.063 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:47:47.064 DEBUG:boto:Method: GET 2022-09-03T06:47:47.064 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:47:47.065 DEBUG:boto:Data: 2022-09-03T06:47:47.065 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.066 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:47.066 DEBUG:boto:Port: 8000 2022-09-03T06:47:47.067 DEBUG:boto:Params: {} 2022-09-03T06:47:47.068 DEBUG:boto:Token: None 2022-09-03T06:47:47.068 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj62 2022-09-03T06:47:47.069 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ciM71gmc6c8K01xAIBkttZUzCC0= 2022-09-03T06:47:47.069 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ciM71gmc6c8K01xAIBkttZUzCC0=', 'Content-Length': '0'} 2022-09-03T06:47:47.072 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000229962d77deb7f94-006312f893-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.073 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:47.073 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:47.074 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:47.074 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:47.075 DEBUG:boto:Method: GET 2022-09-03T06:47:47.075 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:47:47.076 DEBUG:boto:Data: 2022-09-03T06:47:47.076 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.077 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.078 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.078 DEBUG:boto:Params: {} 2022-09-03T06:47:47.079 DEBUG:boto:Token: None 2022-09-03T06:47:47.079 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/ 2022-09-03T06:47:47.080 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A= 2022-09-03T06:47:47.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A=', 'Content-Length': '0'} 2022-09-03T06:47:47.085 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000699265046e043555-006312f893-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.087 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:47.088 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:47.089 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:47.089 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:47.090 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:47:47.090 DEBUG:boto:Method: GET 2022-09-03T06:47:47.091 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:47:47.091 DEBUG:boto:Data: 2022-09-03T06:47:47.092 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.092 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:47.093 DEBUG:boto:Port: 8000 2022-09-03T06:47:47.093 DEBUG:boto:Params: {} 2022-09-03T06:47:47.094 DEBUG:boto:Token: None 2022-09-03T06:47:47.095 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/ 2022-09-03T06:47:47.096 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A= 2022-09-03T06:47:47.096 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A=', 'Content-Length': '0'} 2022-09-03T06:47:47.099 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a1f5cdf49974597e-006312f893-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.100 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:47.101 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:47:47.102 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:47.102 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:47.103 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.103 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.104 DEBUG:boto:Method: GET 2022-09-03T06:47:47.105 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.105 DEBUG:boto:Data: 2022-09-03T06:47:47.106 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.106 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.107 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.107 DEBUG:boto:Params: {} 2022-09-03T06:47:47.108 DEBUG:boto:Token: None 2022-09-03T06:47:47.109 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.109 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D1h/+YipVmZ1f5Ws9Gukm/KiANw= 2022-09-03T06:47:47.110 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D1h/+YipVmZ1f5Ws9Gukm/KiANw=', 'Content-Length': '0'} 2022-09-03T06:47:47.112 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e2f60be2833811e9-006312f893-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.113 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:47.114 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:47.114 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.115 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.115 DEBUG:boto:Method: GET 2022-09-03T06:47:47.116 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.116 DEBUG:boto:Data: 2022-09-03T06:47:47.117 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.117 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:47.118 DEBUG:boto:Port: 8000 2022-09-03T06:47:47.119 DEBUG:boto:Params: {} 2022-09-03T06:47:47.119 DEBUG:boto:Token: None 2022-09-03T06:47:47.120 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.120 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D1h/+YipVmZ1f5Ws9Gukm/KiANw= 2022-09-03T06:47:47.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D1h/+YipVmZ1f5Ws9Gukm/KiANw=', 'Content-Length': '0'} 2022-09-03T06:47:47.123 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001bc6c6723f0252bd-006312f893-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.124 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:47.125 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:47.125 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.126 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.126 DEBUG:boto:Method: HEAD 2022-09-03T06:47:47.127 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.127 DEBUG:boto:Data: 2022-09-03T06:47:47.128 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.128 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.129 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.129 DEBUG:boto:Params: {} 2022-09-03T06:47:47.130 DEBUG:boto:Token: None 2022-09-03T06:47:47.130 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.131 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Cf8orD2P7/aKlo7geomemq7y7/4= 2022-09-03T06:47:47.131 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Cf8orD2P7/aKlo7geomemq7y7/4=', 'Content-Length': '0'} 2022-09-03T06:47:47.133 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007a071d1569f22e20-006312f893-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.134 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:47.135 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:47.135 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.136 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.136 DEBUG:boto:Method: HEAD 2022-09-03T06:47:47.137 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.137 DEBUG:boto:Data: 2022-09-03T06:47:47.138 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.138 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:47.139 DEBUG:boto:Port: 8000 2022-09-03T06:47:47.139 DEBUG:boto:Params: {} 2022-09-03T06:47:47.140 DEBUG:boto:Token: None 2022-09-03T06:47:47.140 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.141 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Cf8orD2P7/aKlo7geomemq7y7/4= 2022-09-03T06:47:47.142 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Cf8orD2P7/aKlo7geomemq7y7/4=', 'Content-Length': '0'} 2022-09-03T06:47:47.144 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cc0cd8758e7e1441-006312f893-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.145 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:47:47.145 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:47.146 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:47.146 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.147 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.147 DEBUG:boto:Method: GET 2022-09-03T06:47:47.148 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.148 DEBUG:boto:Data: 2022-09-03T06:47:47.149 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.150 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.150 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.151 DEBUG:boto:Params: {} 2022-09-03T06:47:47.151 DEBUG:boto:Token: None 2022-09-03T06:47:47.152 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.152 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D1h/+YipVmZ1f5Ws9Gukm/KiANw= 2022-09-03T06:47:47.153 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D1h/+YipVmZ1f5Ws9Gukm/KiANw=', 'Content-Length': '0'} 2022-09-03T06:47:47.155 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000102e42f8d9a51939-006312f893-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.156 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:47.157 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:47.157 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.158 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.158 DEBUG:boto:Method: GET 2022-09-03T06:47:47.159 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.159 DEBUG:boto:Data: 2022-09-03T06:47:47.160 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.160 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:47.161 DEBUG:boto:Port: 8000 2022-09-03T06:47:47.161 DEBUG:boto:Params: {} 2022-09-03T06:47:47.162 DEBUG:boto:Token: None 2022-09-03T06:47:47.163 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:47:47.163 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D1h/+YipVmZ1f5Ws9Gukm/KiANw= 2022-09-03T06:47:47.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D1h/+YipVmZ1f5Ws9Gukm/KiANw=', 'Content-Length': '0'} 2022-09-03T06:47:47.166 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000087f6f68d2d8fc2c3-006312f893-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.167 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:47.168 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:47.168 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:47.169 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:47.169 DEBUG:boto:Method: GET 2022-09-03T06:47:47.170 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:47:47.170 DEBUG:boto:Data: 2022-09-03T06:47:47.171 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.171 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.172 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.172 DEBUG:boto:Params: {} 2022-09-03T06:47:47.173 DEBUG:boto:Token: None 2022-09-03T06:47:47.174 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/ 2022-09-03T06:47:47.175 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A= 2022-09-03T06:47:47.176 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A=', 'Content-Length': '0'} 2022-09-03T06:47:47.179 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000aad6c0254668b2d-006312f893-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.180 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:47.181 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:47.181 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:47.182 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:47.182 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:47.183 DEBUG:boto:Method: GET 2022-09-03T06:47:47.184 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:47:47.184 DEBUG:boto:Data: 2022-09-03T06:47:47.185 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.185 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:47.186 DEBUG:boto:Port: 8000 2022-09-03T06:47:47.186 DEBUG:boto:Params: {} 2022-09-03T06:47:47.187 DEBUG:boto:Token: None 2022-09-03T06:47:47.187 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/ 2022-09-03T06:47:47.188 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A= 2022-09-03T06:47:47.189 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A=', 'Content-Length': '0'} 2022-09-03T06:47:47.191 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000966f9ae9188a6833-006312f893-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.238 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:47.240 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:47.240 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:47.241 DEBUG:boto:Method: HEAD 2022-09-03T06:47:47.242 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:47:47.242 DEBUG:boto:Data: 2022-09-03T06:47:47.243 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.243 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.244 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.244 DEBUG:boto:Params: {} 2022-09-03T06:47:47.245 DEBUG:boto:Token: None 2022-09-03T06:47:47.246 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj68 2022-09-03T06:47:47.246 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S/CEOJiS+xy8Fn+Rdr8cWjqmYKc= 2022-09-03T06:47:47.247 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S/CEOJiS+xy8Fn+Rdr8cWjqmYKc=', 'Content-Length': '0'} 2022-09-03T06:47:47.249 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ff3536358022e3e0-006312f893-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.251 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:47.252 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:47.253 DEBUG:boto:Method: HEAD 2022-09-03T06:47:47.254 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:47:47.254 DEBUG:boto:Data: 2022-09-03T06:47:47.255 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.255 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:47.256 DEBUG:boto:Port: 8000 2022-09-03T06:47:47.256 DEBUG:boto:Params: {} 2022-09-03T06:47:47.257 DEBUG:boto:Token: None 2022-09-03T06:47:47.258 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj68 2022-09-03T06:47:47.259 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S/CEOJiS+xy8Fn+Rdr8cWjqmYKc= 2022-09-03T06:47:47.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S/CEOJiS+xy8Fn+Rdr8cWjqmYKc=', 'Content-Length': '0'} 2022-09-03T06:47:47.262 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000013e51eaa7eec72d7-006312f893-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.263 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:47:47.263 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:47.264 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:47.264 DEBUG:boto:Method: GET 2022-09-03T06:47:47.265 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:47:47.265 DEBUG:boto:Data: 2022-09-03T06:47:47.266 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.267 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.267 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.268 DEBUG:boto:Params: {} 2022-09-03T06:47:47.269 DEBUG:boto:Token: None 2022-09-03T06:47:47.269 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj68 2022-09-03T06:47:47.270 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yhu3F9RzkMwAmc5xD4V8pJGxxcQ= 2022-09-03T06:47:47.271 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yhu3F9RzkMwAmc5xD4V8pJGxxcQ=', 'Content-Length': '0'} 2022-09-03T06:47:47.278 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ae5416326bad021a-006312f893-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.280 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:47:47.281 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:47:47.281 DEBUG:boto:Method: GET 2022-09-03T06:47:47.283 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:47:47.283 DEBUG:boto:Data: 2022-09-03T06:47:47.284 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.285 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:47.286 DEBUG:boto:Port: 8000 2022-09-03T06:47:47.287 DEBUG:boto:Params: {} 2022-09-03T06:47:47.288 DEBUG:boto:Token: None 2022-09-03T06:47:47.288 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj68 2022-09-03T06:47:47.289 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yhu3F9RzkMwAmc5xD4V8pJGxxcQ= 2022-09-03T06:47:47.291 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yhu3F9RzkMwAmc5xD4V8pJGxxcQ=', 'Content-Length': '0'} 2022-09-03T06:47:47.295 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b52599cf1a9629d5-006312f893-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.296 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:47.297 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:47.297 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:47.298 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:47.298 DEBUG:boto:Method: GET 2022-09-03T06:47:47.299 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:47:47.299 DEBUG:boto:Data: 2022-09-03T06:47:47.300 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.300 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.301 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.301 DEBUG:boto:Params: {} 2022-09-03T06:47:47.302 DEBUG:boto:Token: None 2022-09-03T06:47:47.303 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/ 2022-09-03T06:47:47.303 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A= 2022-09-03T06:47:47.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A=', 'Content-Length': '0'} 2022-09-03T06:47:47.307 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000024e7867024b8d586-006312f893-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.354 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:47.356 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:47.357 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:47.357 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:47.358 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:47:47.359 DEBUG:boto:Method: GET 2022-09-03T06:47:47.360 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:47:47.361 DEBUG:boto:Data: 2022-09-03T06:47:47.362 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.363 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:47.364 DEBUG:boto:Port: 8000 2022-09-03T06:47:47.365 DEBUG:boto:Params: {} 2022-09-03T06:47:47.365 DEBUG:boto:Token: None 2022-09-03T06:47:47.366 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/ 2022-09-03T06:47:47.366 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A= 2022-09-03T06:47:47.367 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A=', 'Content-Length': '0'} 2022-09-03T06:47:47.370 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000009523582c3258a6e-006312f893-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.371 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:47.372 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:47:47.372 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:47.373 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:47.373 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.374 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.374 DEBUG:boto:Method: GET 2022-09-03T06:47:47.375 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.375 DEBUG:boto:Data: 2022-09-03T06:47:47.376 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.377 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.377 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.378 DEBUG:boto:Params: {} 2022-09-03T06:47:47.379 DEBUG:boto:Token: None 2022-09-03T06:47:47.379 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.380 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VLebg51Nl5JA1z43/A0EDGWXVlk= 2022-09-03T06:47:47.381 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VLebg51Nl5JA1z43/A0EDGWXVlk=', 'Content-Length': '0'} 2022-09-03T06:47:47.383 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004bbf13932c4135c8-006312f893-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.384 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:47.384 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:47.385 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.385 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.386 DEBUG:boto:Method: GET 2022-09-03T06:47:47.387 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.388 DEBUG:boto:Data: 2022-09-03T06:47:47.389 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.390 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:47.391 DEBUG:boto:Port: 8000 2022-09-03T06:47:47.392 DEBUG:boto:Params: {} 2022-09-03T06:47:47.393 DEBUG:boto:Token: None 2022-09-03T06:47:47.394 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.395 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VLebg51Nl5JA1z43/A0EDGWXVlk= 2022-09-03T06:47:47.396 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VLebg51Nl5JA1z43/A0EDGWXVlk=', 'Content-Length': '0'} 2022-09-03T06:47:47.398 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c1dde22714e0ab9e-006312f893-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.399 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:47.400 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:47.401 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.401 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.402 DEBUG:boto:Method: HEAD 2022-09-03T06:47:47.403 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.404 DEBUG:boto:Data: 2022-09-03T06:47:47.405 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.406 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.406 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.407 DEBUG:boto:Params: {} 2022-09-03T06:47:47.408 DEBUG:boto:Token: None 2022-09-03T06:47:47.408 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.409 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7rESxJwSdb5XtCjMO8YnQdvgw58= 2022-09-03T06:47:47.409 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7rESxJwSdb5XtCjMO8YnQdvgw58=', 'Content-Length': '0'} 2022-09-03T06:47:47.412 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e9ebea75d72fa88a-006312f893-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.413 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:47.414 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:47.414 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.415 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.416 DEBUG:boto:Method: HEAD 2022-09-03T06:47:47.416 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.417 DEBUG:boto:Data: 2022-09-03T06:47:47.418 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.419 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:47.420 DEBUG:boto:Port: 8000 2022-09-03T06:47:47.420 DEBUG:boto:Params: {} 2022-09-03T06:47:47.422 DEBUG:boto:Token: None 2022-09-03T06:47:47.423 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.424 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7rESxJwSdb5XtCjMO8YnQdvgw58= 2022-09-03T06:47:47.425 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7rESxJwSdb5XtCjMO8YnQdvgw58=', 'Content-Length': '0'} 2022-09-03T06:47:47.427 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000287c66945c740f9a-006312f893-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.428 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:47:47.428 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:47.429 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:47.430 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.430 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.431 DEBUG:boto:Method: GET 2022-09-03T06:47:47.431 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.432 DEBUG:boto:Data: 2022-09-03T06:47:47.432 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.433 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.434 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.435 DEBUG:boto:Params: {} 2022-09-03T06:47:47.436 DEBUG:boto:Token: None 2022-09-03T06:47:47.437 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.438 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VLebg51Nl5JA1z43/A0EDGWXVlk= 2022-09-03T06:47:47.439 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VLebg51Nl5JA1z43/A0EDGWXVlk=', 'Content-Length': '0'} 2022-09-03T06:47:47.441 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000073c830b2e8cb6bbf-006312f893-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.442 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:47.443 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:47.444 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.444 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.445 DEBUG:boto:Method: GET 2022-09-03T06:47:47.445 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.446 DEBUG:boto:Data: 2022-09-03T06:47:47.446 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.447 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:47.447 DEBUG:boto:Port: 8000 2022-09-03T06:47:47.448 DEBUG:boto:Params: {} 2022-09-03T06:47:47.449 DEBUG:boto:Token: None 2022-09-03T06:47:47.449 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:47:47.450 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VLebg51Nl5JA1z43/A0EDGWXVlk= 2022-09-03T06:47:47.451 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VLebg51Nl5JA1z43/A0EDGWXVlk=', 'Content-Length': '0'} 2022-09-03T06:47:47.453 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e425f0415d0e09e-006312f893-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.494 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:47.496 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:47.497 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:47.498 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:47.499 DEBUG:boto:Method: GET 2022-09-03T06:47:47.500 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:47:47.501 DEBUG:boto:Data: 2022-09-03T06:47:47.501 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.502 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.503 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.504 DEBUG:boto:Params: {} 2022-09-03T06:47:47.505 DEBUG:boto:Token: None 2022-09-03T06:47:47.506 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/ 2022-09-03T06:47:47.507 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A= 2022-09-03T06:47:47.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A=', 'Content-Length': '0'} 2022-09-03T06:47:47.512 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000acfbb4f54e5eae17-006312f893-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.513 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:47.515 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:47.516 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:47.517 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:47.518 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:47.519 DEBUG:boto:Method: GET 2022-09-03T06:47:47.520 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:47:47.521 DEBUG:boto:Data: 2022-09-03T06:47:47.523 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.524 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:47.524 DEBUG:boto:Port: 8000 2022-09-03T06:47:47.530 DEBUG:boto:Params: {} 2022-09-03T06:47:47.532 DEBUG:boto:Token: None 2022-09-03T06:47:47.534 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/ 2022-09-03T06:47:47.536 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A= 2022-09-03T06:47:47.537 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A=', 'Content-Length': '0'} 2022-09-03T06:47:47.541 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000034b38b751c43e82c-006312f893-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.542 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:47.544 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:47.545 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:47.545 DEBUG:boto:Method: HEAD 2022-09-03T06:47:47.546 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:47:47.547 DEBUG:boto:Data: 2022-09-03T06:47:47.549 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.550 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.551 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.552 DEBUG:boto:Params: {} 2022-09-03T06:47:47.553 DEBUG:boto:Token: None 2022-09-03T06:47:47.554 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj69 2022-09-03T06:47:47.555 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/Uc4br5LibP3/BXNkpfeSEzCBz8= 2022-09-03T06:47:47.556 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/Uc4br5LibP3/BXNkpfeSEzCBz8=', 'Content-Length': '0'} 2022-09-03T06:47:47.559 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000976548861765ef91-006312f893-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.560 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:47.561 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:47.563 DEBUG:boto:Method: HEAD 2022-09-03T06:47:47.564 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:47:47.565 DEBUG:boto:Data: 2022-09-03T06:47:47.565 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.566 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:47.567 DEBUG:boto:Port: 8000 2022-09-03T06:47:47.568 DEBUG:boto:Params: {} 2022-09-03T06:47:47.569 DEBUG:boto:Token: None 2022-09-03T06:47:47.570 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj69 2022-09-03T06:47:47.571 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/Uc4br5LibP3/BXNkpfeSEzCBz8= 2022-09-03T06:47:47.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/Uc4br5LibP3/BXNkpfeSEzCBz8=', 'Content-Length': '0'} 2022-09-03T06:47:47.574 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000055e93001a8fa454c-006312f893-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.575 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:47:47.576 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:47.577 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:47.578 DEBUG:boto:Method: GET 2022-09-03T06:47:47.579 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:47:47.580 DEBUG:boto:Data: 2022-09-03T06:47:47.581 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.581 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.582 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.583 DEBUG:boto:Params: {} 2022-09-03T06:47:47.584 DEBUG:boto:Token: None 2022-09-03T06:47:47.585 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj69 2022-09-03T06:47:47.586 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4+gD7yFy2bgelfl45ESbH4fZKnM= 2022-09-03T06:47:47.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4+gD7yFy2bgelfl45ESbH4fZKnM=', 'Content-Length': '0'} 2022-09-03T06:47:47.591 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000257d7ea58e15f617-006312f893-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.634 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:47:47.636 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:47:47.636 DEBUG:boto:Method: GET 2022-09-03T06:47:47.637 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:47:47.638 DEBUG:boto:Data: 2022-09-03T06:47:47.639 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.639 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:47.640 DEBUG:boto:Port: 8000 2022-09-03T06:47:47.641 DEBUG:boto:Params: {} 2022-09-03T06:47:47.641 DEBUG:boto:Token: None 2022-09-03T06:47:47.642 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj69 2022-09-03T06:47:47.643 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4+gD7yFy2bgelfl45ESbH4fZKnM= 2022-09-03T06:47:47.644 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4+gD7yFy2bgelfl45ESbH4fZKnM=', 'Content-Length': '0'} 2022-09-03T06:47:47.646 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000027e5b0359190fc2f-006312f893-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.647 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:47.648 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:47.648 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:47.649 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:47.650 DEBUG:boto:Method: GET 2022-09-03T06:47:47.652 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:47:47.653 DEBUG:boto:Data: 2022-09-03T06:47:47.655 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.656 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.657 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.658 DEBUG:boto:Params: {} 2022-09-03T06:47:47.659 DEBUG:boto:Token: None 2022-09-03T06:47:47.660 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/ 2022-09-03T06:47:47.662 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A= 2022-09-03T06:47:47.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A=', 'Content-Length': '0'} 2022-09-03T06:47:47.666 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000018058b7d47fbe511-006312f893-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.667 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:47.668 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:47.669 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:47.670 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:47.670 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:47:47.671 DEBUG:boto:Method: GET 2022-09-03T06:47:47.673 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:47:47.674 DEBUG:boto:Data: 2022-09-03T06:47:47.675 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.676 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:47.677 DEBUG:boto:Port: 8000 2022-09-03T06:47:47.679 DEBUG:boto:Params: {} 2022-09-03T06:47:47.680 DEBUG:boto:Token: None 2022-09-03T06:47:47.681 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/ 2022-09-03T06:47:47.683 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A= 2022-09-03T06:47:47.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A=', 'Content-Length': '0'} 2022-09-03T06:47:47.687 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ee1c5b43d6e9f408-006312f893-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.689 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:47.690 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:47:47.691 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:47.692 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:47.693 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.694 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.695 DEBUG:boto:Method: GET 2022-09-03T06:47:47.696 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.697 DEBUG:boto:Data: 2022-09-03T06:47:47.698 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.698 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.699 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.700 DEBUG:boto:Params: {} 2022-09-03T06:47:47.701 DEBUG:boto:Token: None 2022-09-03T06:47:47.702 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.703 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:StXJZoOT+nuvouSX7IISdbN0j4g= 2022-09-03T06:47:47.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:StXJZoOT+nuvouSX7IISdbN0j4g=', 'Content-Length': '0'} 2022-09-03T06:47:47.707 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1285cc0bd2403a3-006312f893-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.709 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:47.710 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:47.711 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.711 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.712 DEBUG:boto:Method: GET 2022-09-03T06:47:47.713 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.713 DEBUG:boto:Data: 2022-09-03T06:47:47.714 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.714 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:47.715 DEBUG:boto:Port: 8000 2022-09-03T06:47:47.715 DEBUG:boto:Params: {} 2022-09-03T06:47:47.716 DEBUG:boto:Token: None 2022-09-03T06:47:47.717 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.717 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:StXJZoOT+nuvouSX7IISdbN0j4g= 2022-09-03T06:47:47.718 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:StXJZoOT+nuvouSX7IISdbN0j4g=', 'Content-Length': '0'} 2022-09-03T06:47:47.720 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b5c1f8c04c5f30c5-006312f893-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.721 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:47.721 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:47.722 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.723 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.724 DEBUG:boto:Method: HEAD 2022-09-03T06:47:47.724 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.725 DEBUG:boto:Data: 2022-09-03T06:47:47.726 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.726 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.727 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.727 DEBUG:boto:Params: {} 2022-09-03T06:47:47.728 DEBUG:boto:Token: None 2022-09-03T06:47:47.728 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.729 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fSVzc+eipCSKY+tb5Pybq4zysBA= 2022-09-03T06:47:47.730 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fSVzc+eipCSKY+tb5Pybq4zysBA=', 'Content-Length': '0'} 2022-09-03T06:47:47.732 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000071766008df5e1a3a-006312f893-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.733 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:47.734 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:47.734 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.735 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.735 DEBUG:boto:Method: HEAD 2022-09-03T06:47:47.736 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.736 DEBUG:boto:Data: 2022-09-03T06:47:47.737 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.737 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:47.738 DEBUG:boto:Port: 8000 2022-09-03T06:47:47.739 DEBUG:boto:Params: {} 2022-09-03T06:47:47.739 DEBUG:boto:Token: None 2022-09-03T06:47:47.740 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.740 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fSVzc+eipCSKY+tb5Pybq4zysBA= 2022-09-03T06:47:47.741 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fSVzc+eipCSKY+tb5Pybq4zysBA=', 'Content-Length': '0'} 2022-09-03T06:47:47.743 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c008d91609abeee-006312f893-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.744 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:47:47.745 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:47.745 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:47.746 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.747 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.747 DEBUG:boto:Method: GET 2022-09-03T06:47:47.748 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.748 DEBUG:boto:Data: 2022-09-03T06:47:47.749 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.750 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.751 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.751 DEBUG:boto:Params: {} 2022-09-03T06:47:47.752 DEBUG:boto:Token: None 2022-09-03T06:47:47.753 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.754 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:StXJZoOT+nuvouSX7IISdbN0j4g= 2022-09-03T06:47:47.754 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:StXJZoOT+nuvouSX7IISdbN0j4g=', 'Content-Length': '0'} 2022-09-03T06:47:47.757 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a375d6fc0d15cdb3-006312f893-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.798 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:47.799 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:47.800 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.800 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.801 DEBUG:boto:Method: GET 2022-09-03T06:47:47.801 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.802 DEBUG:boto:Data: 2022-09-03T06:47:47.803 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.803 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:47.804 DEBUG:boto:Port: 8000 2022-09-03T06:47:47.805 DEBUG:boto:Params: {} 2022-09-03T06:47:47.805 DEBUG:boto:Token: None 2022-09-03T06:47:47.806 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:47:47.807 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:StXJZoOT+nuvouSX7IISdbN0j4g= 2022-09-03T06:47:47.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:StXJZoOT+nuvouSX7IISdbN0j4g=', 'Content-Length': '0'} 2022-09-03T06:47:47.810 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cb21e9b8126a1809-006312f893-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.811 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:47.812 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:47.813 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:47.813 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:47.814 DEBUG:boto:Method: GET 2022-09-03T06:47:47.814 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:47:47.815 DEBUG:boto:Data: 2022-09-03T06:47:47.816 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.816 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.817 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.818 DEBUG:boto:Params: {} 2022-09-03T06:47:47.818 DEBUG:boto:Token: None 2022-09-03T06:47:47.819 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/ 2022-09-03T06:47:47.820 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A= 2022-09-03T06:47:47.820 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A=', 'Content-Length': '0'} 2022-09-03T06:47:47.825 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005713db5cd4ff5fcc-006312f893-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.827 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:47.829 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:47.830 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:47.831 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:47.832 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:47.833 DEBUG:boto:Method: GET 2022-09-03T06:47:47.834 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:47:47.835 DEBUG:boto:Data: 2022-09-03T06:47:47.836 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.837 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:47.838 DEBUG:boto:Port: 8000 2022-09-03T06:47:47.839 DEBUG:boto:Params: {} 2022-09-03T06:47:47.841 DEBUG:boto:Token: None 2022-09-03T06:47:47.842 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/ 2022-09-03T06:47:47.843 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A= 2022-09-03T06:47:47.845 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A=', 'Content-Length': '0'} 2022-09-03T06:47:47.847 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fb23c21664ab71be-006312f893-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.894 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:47.895 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:47.896 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:47.897 DEBUG:boto:Method: HEAD 2022-09-03T06:47:47.898 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:47:47.899 DEBUG:boto:Data: 2022-09-03T06:47:47.899 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.900 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.901 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.901 DEBUG:boto:Params: {} 2022-09-03T06:47:47.902 DEBUG:boto:Token: None 2022-09-03T06:47:47.903 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj76 2022-09-03T06:47:47.903 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ytt544alQpEXqdqKovrJqnZ4Cq4= 2022-09-03T06:47:47.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ytt544alQpEXqdqKovrJqnZ4Cq4=', 'Content-Length': '0'} 2022-09-03T06:47:47.906 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000287890bee4ec4105-006312f893-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.907 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:47.908 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:47.908 DEBUG:boto:Method: HEAD 2022-09-03T06:47:47.909 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:47:47.909 DEBUG:boto:Data: 2022-09-03T06:47:47.910 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.910 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:47.911 DEBUG:boto:Port: 8000 2022-09-03T06:47:47.912 DEBUG:boto:Params: {} 2022-09-03T06:47:47.913 DEBUG:boto:Token: None 2022-09-03T06:47:47.914 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj76 2022-09-03T06:47:47.916 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ytt544alQpEXqdqKovrJqnZ4Cq4= 2022-09-03T06:47:47.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ytt544alQpEXqdqKovrJqnZ4Cq4=', 'Content-Length': '0'} 2022-09-03T06:47:47.919 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aebc8931fcb6fee0-006312f893-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.920 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:47:47.921 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:47.922 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:47.923 DEBUG:boto:Method: GET 2022-09-03T06:47:47.923 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:47:47.924 DEBUG:boto:Data: 2022-09-03T06:47:47.925 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.925 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.926 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.927 DEBUG:boto:Params: {} 2022-09-03T06:47:47.927 DEBUG:boto:Token: None 2022-09-03T06:47:47.928 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj76 2022-09-03T06:47:47.929 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3DjiavnCAoqoFFpx89+SVHLGdbo= 2022-09-03T06:47:47.930 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3DjiavnCAoqoFFpx89+SVHLGdbo=', 'Content-Length': '0'} 2022-09-03T06:47:47.932 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a5af7fdcb32b8260-006312f893-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.933 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:47:47.934 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:47:47.935 DEBUG:boto:Method: GET 2022-09-03T06:47:47.935 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:47:47.936 DEBUG:boto:Data: 2022-09-03T06:47:47.936 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.937 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:47.938 DEBUG:boto:Port: 8000 2022-09-03T06:47:47.938 DEBUG:boto:Params: {} 2022-09-03T06:47:47.941 DEBUG:boto:Token: None 2022-09-03T06:47:47.942 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj76 2022-09-03T06:47:47.943 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3DjiavnCAoqoFFpx89+SVHLGdbo= 2022-09-03T06:47:47.943 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3DjiavnCAoqoFFpx89+SVHLGdbo=', 'Content-Length': '0'} 2022-09-03T06:47:47.946 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b4d3d198297e249b-006312f893-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.947 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:47.949 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:47.950 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:47.951 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:47.952 DEBUG:boto:Method: GET 2022-09-03T06:47:47.953 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:47:47.954 DEBUG:boto:Data: 2022-09-03T06:47:47.955 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.956 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.957 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.958 DEBUG:boto:Params: {} 2022-09-03T06:47:47.960 DEBUG:boto:Token: None 2022-09-03T06:47:47.961 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/ 2022-09-03T06:47:47.962 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A= 2022-09-03T06:47:47.963 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A=', 'Content-Length': '0'} 2022-09-03T06:47:47.967 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000feeb01c571da6b34-006312f893-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.968 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:47.969 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:47.970 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:47.971 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:47.972 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:47:47.973 DEBUG:boto:Method: GET 2022-09-03T06:47:47.974 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:47:47.975 DEBUG:boto:Data: 2022-09-03T06:47:47.976 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.977 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:47.978 DEBUG:boto:Port: 8000 2022-09-03T06:47:47.979 DEBUG:boto:Params: {} 2022-09-03T06:47:47.980 DEBUG:boto:Token: None 2022-09-03T06:47:47.980 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/ 2022-09-03T06:47:47.981 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A= 2022-09-03T06:47:47.982 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lc0etIjNHgUHwfA13ABA/6NQe4A=', 'Content-Length': '0'} 2022-09-03T06:47:47.987 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b05f7f8a59e17fa3-006312f893-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:47.988 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:47.989 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:47:47.990 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:47.990 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:47.991 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:47.991 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:47.992 DEBUG:boto:Method: GET 2022-09-03T06:47:47.993 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:47.993 DEBUG:boto:Data: 2022-09-03T06:47:47.994 DEBUG:boto:Headers: {} 2022-09-03T06:47:47.994 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:47.995 DEBUG:boto:Port: 8001 2022-09-03T06:47:47.996 DEBUG:boto:Params: {} 2022-09-03T06:47:47.996 DEBUG:boto:Token: None 2022-09-03T06:47:47.997 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:47 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:47.998 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/JqYgAQEQ1tgbKssY/g3yocYsyQ= 2022-09-03T06:47:47.998 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:47 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/JqYgAQEQ1tgbKssY/g3yocYsyQ=', 'Content-Length': '0'} 2022-09-03T06:47:48.000 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002030746819938260-006312f893-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:47 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.042 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:48.043 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:48.044 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:48.044 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:48.045 DEBUG:boto:Method: GET 2022-09-03T06:47:48.046 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:48.046 DEBUG:boto:Data: 2022-09-03T06:47:48.047 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.048 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.048 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.049 DEBUG:boto:Params: {} 2022-09-03T06:47:48.049 DEBUG:boto:Token: None 2022-09-03T06:47:48.050 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:48.051 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JxgacL2IqRn+M8Z2zG5/DIQYyKU= 2022-09-03T06:47:48.052 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JxgacL2IqRn+M8Z2zG5/DIQYyKU=', 'Content-Length': '0'} 2022-09-03T06:47:48.054 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000468efc879092fa1-006312f894-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.056 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:48.056 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:48.057 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:48.057 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:48.058 DEBUG:boto:Method: HEAD 2022-09-03T06:47:48.058 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:48.059 DEBUG:boto:Data: 2022-09-03T06:47:48.060 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.060 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.061 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.061 DEBUG:boto:Params: {} 2022-09-03T06:47:48.062 DEBUG:boto:Token: None 2022-09-03T06:47:48.063 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:48.064 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4gNneETPKpaONH7mSb+05Gpp4N8= 2022-09-03T06:47:48.065 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4gNneETPKpaONH7mSb+05Gpp4N8=', 'Content-Length': '0'} 2022-09-03T06:47:48.068 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000853e539468cb38fe-006312f894-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.069 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:48.070 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:48.071 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:48.072 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:48.073 DEBUG:boto:Method: HEAD 2022-09-03T06:47:48.075 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:48.076 DEBUG:boto:Data: 2022-09-03T06:47:48.077 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.078 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.078 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.079 DEBUG:boto:Params: {} 2022-09-03T06:47:48.080 DEBUG:boto:Token: None 2022-09-03T06:47:48.080 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:48.081 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4gNneETPKpaONH7mSb+05Gpp4N8= 2022-09-03T06:47:48.082 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4gNneETPKpaONH7mSb+05Gpp4N8=', 'Content-Length': '0'} 2022-09-03T06:47:48.084 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000372604e96581a3dc-006312f894-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.084 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:47:48.085 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:48.086 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:48.086 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:48.087 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:48.087 DEBUG:boto:Method: GET 2022-09-03T06:47:48.088 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:48.089 DEBUG:boto:Data: 2022-09-03T06:47:48.089 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.090 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.090 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.091 DEBUG:boto:Params: {} 2022-09-03T06:47:48.092 DEBUG:boto:Token: None 2022-09-03T06:47:48.093 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:48.094 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JxgacL2IqRn+M8Z2zG5/DIQYyKU= 2022-09-03T06:47:48.094 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JxgacL2IqRn+M8Z2zG5/DIQYyKU=', 'Content-Length': '0'} 2022-09-03T06:47:48.097 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e53b48479923386f-006312f894-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.098 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:48.098 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:48.099 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:48.099 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:48.100 DEBUG:boto:Method: GET 2022-09-03T06:47:48.100 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:48.101 DEBUG:boto:Data: 2022-09-03T06:47:48.102 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.102 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.103 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.104 DEBUG:boto:Params: {} 2022-09-03T06:47:48.105 DEBUG:boto:Token: None 2022-09-03T06:47:48.105 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:47:48.106 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JxgacL2IqRn+M8Z2zG5/DIQYyKU= 2022-09-03T06:47:48.107 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JxgacL2IqRn+M8Z2zG5/DIQYyKU=', 'Content-Length': '0'} 2022-09-03T06:47:48.109 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001df2ca5ce5bc7640-006312f894-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.150 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:48.151 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:48.152 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:48.153 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:48.154 DEBUG:boto:Method: GET 2022-09-03T06:47:48.156 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:47:48.158 DEBUG:boto:Data: 2022-09-03T06:47:48.159 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.160 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.160 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.161 DEBUG:boto:Params: {} 2022-09-03T06:47:48.162 DEBUG:boto:Token: None 2022-09-03T06:47:48.162 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/ 2022-09-03T06:47:48.163 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg= 2022-09-03T06:47:48.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg=', 'Content-Length': '0'} 2022-09-03T06:47:48.167 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001d00b505bed6c522-006312f894-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.168 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:48.169 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:48.169 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:48.170 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:48.171 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:48.171 DEBUG:boto:Method: GET 2022-09-03T06:47:48.172 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:47:48.172 DEBUG:boto:Data: 2022-09-03T06:47:48.173 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.173 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.174 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.175 DEBUG:boto:Params: {} 2022-09-03T06:47:48.175 DEBUG:boto:Token: None 2022-09-03T06:47:48.176 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/ 2022-09-03T06:47:48.177 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg= 2022-09-03T06:47:48.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg=', 'Content-Length': '0'} 2022-09-03T06:47:48.180 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000cfaf853ad9ed199-006312f894-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.181 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:48.182 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:48.183 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:48.183 DEBUG:boto:Method: HEAD 2022-09-03T06:47:48.184 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:47:48.184 DEBUG:boto:Data: 2022-09-03T06:47:48.185 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.186 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.186 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.187 DEBUG:boto:Params: {} 2022-09-03T06:47:48.188 DEBUG:boto:Token: None 2022-09-03T06:47:48.189 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj85 2022-09-03T06:47:48.189 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TbB7HoJhrKKUstsKswYKEze9AnQ= 2022-09-03T06:47:48.190 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TbB7HoJhrKKUstsKswYKEze9AnQ=', 'Content-Length': '0'} 2022-09-03T06:47:48.193 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000028ee75a85b23a7d7-006312f894-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.194 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:48.195 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:48.196 DEBUG:boto:Method: HEAD 2022-09-03T06:47:48.197 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:47:48.198 DEBUG:boto:Data: 2022-09-03T06:47:48.199 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.200 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.201 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.202 DEBUG:boto:Params: {} 2022-09-03T06:47:48.203 DEBUG:boto:Token: None 2022-09-03T06:47:48.204 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj85 2022-09-03T06:47:48.205 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TbB7HoJhrKKUstsKswYKEze9AnQ= 2022-09-03T06:47:48.206 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TbB7HoJhrKKUstsKswYKEze9AnQ=', 'Content-Length': '0'} 2022-09-03T06:47:48.208 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e091d8b10c0eeea6-006312f894-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.209 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:47:48.210 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:48.211 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:48.212 DEBUG:boto:Method: GET 2022-09-03T06:47:48.213 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:47:48.213 DEBUG:boto:Data: 2022-09-03T06:47:48.214 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.215 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.216 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.216 DEBUG:boto:Params: {} 2022-09-03T06:47:48.217 DEBUG:boto:Token: None 2022-09-03T06:47:48.218 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj85 2022-09-03T06:47:48.219 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oT7N7d8pJXJzZ4nyfKdWqQfAFfI= 2022-09-03T06:47:48.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oT7N7d8pJXJzZ4nyfKdWqQfAFfI=', 'Content-Length': '0'} 2022-09-03T06:47:48.222 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eb63e6c829fcd165-006312f894-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.223 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:47:48.224 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:47:48.225 DEBUG:boto:Method: GET 2022-09-03T06:47:48.225 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:47:48.226 DEBUG:boto:Data: 2022-09-03T06:47:48.227 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.227 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.228 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.228 DEBUG:boto:Params: {} 2022-09-03T06:47:48.229 DEBUG:boto:Token: None 2022-09-03T06:47:48.230 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj85 2022-09-03T06:47:48.231 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oT7N7d8pJXJzZ4nyfKdWqQfAFfI= 2022-09-03T06:47:48.231 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oT7N7d8pJXJzZ4nyfKdWqQfAFfI=', 'Content-Length': '0'} 2022-09-03T06:47:48.234 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000063f7f7f0c98f6bcd-006312f894-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.235 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:48.236 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:48.236 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:48.237 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:48.238 DEBUG:boto:Method: GET 2022-09-03T06:47:48.239 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:47:48.240 DEBUG:boto:Data: 2022-09-03T06:47:48.241 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.242 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.243 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.244 DEBUG:boto:Params: {} 2022-09-03T06:47:48.245 DEBUG:boto:Token: None 2022-09-03T06:47:48.246 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/ 2022-09-03T06:47:48.247 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg= 2022-09-03T06:47:48.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg=', 'Content-Length': '0'} 2022-09-03T06:47:48.251 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002acb58fd6d5220f8-006312f894-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.298 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:48.300 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:48.301 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:48.302 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:48.305 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:47:48.305 DEBUG:boto:Method: GET 2022-09-03T06:47:48.306 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:47:48.306 DEBUG:boto:Data: 2022-09-03T06:47:48.307 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.308 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.308 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.309 DEBUG:boto:Params: {} 2022-09-03T06:47:48.309 DEBUG:boto:Token: None 2022-09-03T06:47:48.310 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/ 2022-09-03T06:47:48.311 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg= 2022-09-03T06:47:48.312 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg=', 'Content-Length': '0'} 2022-09-03T06:47:48.315 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000099d60634fd0d53a8-006312f894-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.316 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:48.317 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:47:48.317 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:48.318 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:48.319 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.319 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.320 DEBUG:boto:Method: GET 2022-09-03T06:47:48.320 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.321 DEBUG:boto:Data: 2022-09-03T06:47:48.322 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.323 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.324 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.325 DEBUG:boto:Params: {} 2022-09-03T06:47:48.325 DEBUG:boto:Token: None 2022-09-03T06:47:48.326 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.327 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JcXPT/i8oksF3bot0R3sswge6Lk= 2022-09-03T06:47:48.329 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JcXPT/i8oksF3bot0R3sswge6Lk=', 'Content-Length': '0'} 2022-09-03T06:47:48.332 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b03ae6d8152a0464-006312f894-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.333 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:48.333 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:48.334 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.335 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.335 DEBUG:boto:Method: GET 2022-09-03T06:47:48.336 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.336 DEBUG:boto:Data: 2022-09-03T06:47:48.337 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.337 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.338 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.339 DEBUG:boto:Params: {} 2022-09-03T06:47:48.339 DEBUG:boto:Token: None 2022-09-03T06:47:48.340 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.340 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JcXPT/i8oksF3bot0R3sswge6Lk= 2022-09-03T06:47:48.341 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JcXPT/i8oksF3bot0R3sswge6Lk=', 'Content-Length': '0'} 2022-09-03T06:47:48.344 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002220c81aa3e00d76-006312f894-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.344 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:48.345 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:48.346 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.346 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.347 DEBUG:boto:Method: HEAD 2022-09-03T06:47:48.347 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.348 DEBUG:boto:Data: 2022-09-03T06:47:48.348 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.349 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.349 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.350 DEBUG:boto:Params: {} 2022-09-03T06:47:48.351 DEBUG:boto:Token: None 2022-09-03T06:47:48.351 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.352 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kb4z2YdsEfKCgmer+URo7ncVe1o= 2022-09-03T06:47:48.353 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kb4z2YdsEfKCgmer+URo7ncVe1o=', 'Content-Length': '0'} 2022-09-03T06:47:48.355 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c85f738643b854c0-006312f894-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.356 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:48.356 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:48.357 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.357 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.358 DEBUG:boto:Method: HEAD 2022-09-03T06:47:48.359 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.359 DEBUG:boto:Data: 2022-09-03T06:47:48.360 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.360 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.361 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.361 DEBUG:boto:Params: {} 2022-09-03T06:47:48.362 DEBUG:boto:Token: None 2022-09-03T06:47:48.362 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.363 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kb4z2YdsEfKCgmer+URo7ncVe1o= 2022-09-03T06:47:48.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kb4z2YdsEfKCgmer+URo7ncVe1o=', 'Content-Length': '0'} 2022-09-03T06:47:48.366 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001336447bd2bb9533-006312f894-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.367 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:47:48.367 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:48.368 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:48.369 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.370 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.370 DEBUG:boto:Method: GET 2022-09-03T06:47:48.371 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.372 DEBUG:boto:Data: 2022-09-03T06:47:48.373 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.373 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.374 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.375 DEBUG:boto:Params: {} 2022-09-03T06:47:48.376 DEBUG:boto:Token: None 2022-09-03T06:47:48.377 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.377 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JcXPT/i8oksF3bot0R3sswge6Lk= 2022-09-03T06:47:48.378 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JcXPT/i8oksF3bot0R3sswge6Lk=', 'Content-Length': '0'} 2022-09-03T06:47:48.381 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ac45342ab5b591ad-006312f894-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.382 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:48.382 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:48.383 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.383 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.384 DEBUG:boto:Method: GET 2022-09-03T06:47:48.385 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.385 DEBUG:boto:Data: 2022-09-03T06:47:48.386 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.386 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.387 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.387 DEBUG:boto:Params: {} 2022-09-03T06:47:48.388 DEBUG:boto:Token: None 2022-09-03T06:47:48.389 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:47:48.389 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JcXPT/i8oksF3bot0R3sswge6Lk= 2022-09-03T06:47:48.390 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JcXPT/i8oksF3bot0R3sswge6Lk=', 'Content-Length': '0'} 2022-09-03T06:47:48.392 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002af662607f9080c2-006312f894-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.393 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:48.394 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:48.394 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:48.395 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:48.395 DEBUG:boto:Method: GET 2022-09-03T06:47:48.396 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:47:48.396 DEBUG:boto:Data: 2022-09-03T06:47:48.397 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.397 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.398 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.399 DEBUG:boto:Params: {} 2022-09-03T06:47:48.399 DEBUG:boto:Token: None 2022-09-03T06:47:48.400 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/ 2022-09-03T06:47:48.400 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg= 2022-09-03T06:47:48.401 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg=', 'Content-Length': '0'} 2022-09-03T06:47:48.404 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006532de02c8512444-006312f894-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.405 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:48.406 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:48.407 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:48.407 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:48.408 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:48.409 DEBUG:boto:Method: GET 2022-09-03T06:47:48.410 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:47:48.411 DEBUG:boto:Data: 2022-09-03T06:47:48.412 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.413 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.414 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.415 DEBUG:boto:Params: {} 2022-09-03T06:47:48.416 DEBUG:boto:Token: None 2022-09-03T06:47:48.417 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/ 2022-09-03T06:47:48.418 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg= 2022-09-03T06:47:48.419 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg=', 'Content-Length': '0'} 2022-09-03T06:47:48.423 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b012e30b0fa703f9-006312f894-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.424 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:48.425 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:48.426 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:48.427 DEBUG:boto:Method: HEAD 2022-09-03T06:47:48.428 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:47:48.429 DEBUG:boto:Data: 2022-09-03T06:47:48.430 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.431 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.432 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.432 DEBUG:boto:Params: {} 2022-09-03T06:47:48.433 DEBUG:boto:Token: None 2022-09-03T06:47:48.433 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj87 2022-09-03T06:47:48.434 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cMNq9Z0yzLdycggDZ0IDcVHQC3k= 2022-09-03T06:47:48.435 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cMNq9Z0yzLdycggDZ0IDcVHQC3k=', 'Content-Length': '0'} 2022-09-03T06:47:48.438 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003812c00a4db19f59-006312f894-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.439 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:48.440 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:48.440 DEBUG:boto:Method: HEAD 2022-09-03T06:47:48.441 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:47:48.441 DEBUG:boto:Data: 2022-09-03T06:47:48.442 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.443 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.443 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.444 DEBUG:boto:Params: {} 2022-09-03T06:47:48.445 DEBUG:boto:Token: None 2022-09-03T06:47:48.445 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj87 2022-09-03T06:47:48.446 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cMNq9Z0yzLdycggDZ0IDcVHQC3k= 2022-09-03T06:47:48.447 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cMNq9Z0yzLdycggDZ0IDcVHQC3k=', 'Content-Length': '0'} 2022-09-03T06:47:48.449 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000648f03a9c0387ae2-006312f894-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.450 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:47:48.451 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:48.451 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:48.452 DEBUG:boto:Method: GET 2022-09-03T06:47:48.453 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:47:48.454 DEBUG:boto:Data: 2022-09-03T06:47:48.454 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.455 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.455 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.456 DEBUG:boto:Params: {} 2022-09-03T06:47:48.457 DEBUG:boto:Token: None 2022-09-03T06:47:48.458 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj87 2022-09-03T06:47:48.459 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pFSBXJeUob3SFz6EqoND8PIy1Go= 2022-09-03T06:47:48.460 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pFSBXJeUob3SFz6EqoND8PIy1Go=', 'Content-Length': '0'} 2022-09-03T06:47:48.463 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000475576a4da6bc8d0-006312f894-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.465 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:47:48.466 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:47:48.467 DEBUG:boto:Method: GET 2022-09-03T06:47:48.468 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:47:48.468 DEBUG:boto:Data: 2022-09-03T06:47:48.469 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.469 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.470 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.470 DEBUG:boto:Params: {} 2022-09-03T06:47:48.471 DEBUG:boto:Token: None 2022-09-03T06:47:48.472 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj87 2022-09-03T06:47:48.473 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pFSBXJeUob3SFz6EqoND8PIy1Go= 2022-09-03T06:47:48.474 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pFSBXJeUob3SFz6EqoND8PIy1Go=', 'Content-Length': '0'} 2022-09-03T06:47:48.476 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d78c8266b55bd45c-006312f894-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.518 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:48.519 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:48.519 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:48.520 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:48.520 DEBUG:boto:Method: GET 2022-09-03T06:47:48.521 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:47:48.521 DEBUG:boto:Data: 2022-09-03T06:47:48.522 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.523 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.523 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.524 DEBUG:boto:Params: {} 2022-09-03T06:47:48.524 DEBUG:boto:Token: None 2022-09-03T06:47:48.525 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/ 2022-09-03T06:47:48.526 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg= 2022-09-03T06:47:48.527 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg=', 'Content-Length': '0'} 2022-09-03T06:47:48.530 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000731e67b355dc92b6-006312f894-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.530 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:48.531 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:48.532 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:48.533 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:48.533 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:47:48.534 DEBUG:boto:Method: GET 2022-09-03T06:47:48.534 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:47:48.535 DEBUG:boto:Data: 2022-09-03T06:47:48.536 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.536 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.537 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.537 DEBUG:boto:Params: {} 2022-09-03T06:47:48.538 DEBUG:boto:Token: None 2022-09-03T06:47:48.539 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/ 2022-09-03T06:47:48.539 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg= 2022-09-03T06:47:48.540 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg=', 'Content-Length': '0'} 2022-09-03T06:47:48.543 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e9b8f53534f4c231-006312f894-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.544 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:48.545 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:47:48.546 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:48.546 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:48.547 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.548 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.548 DEBUG:boto:Method: GET 2022-09-03T06:47:48.549 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.549 DEBUG:boto:Data: 2022-09-03T06:47:48.550 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.550 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.551 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.552 DEBUG:boto:Params: {} 2022-09-03T06:47:48.552 DEBUG:boto:Token: None 2022-09-03T06:47:48.553 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.554 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fz3sGltfm2vwNkaoBoltT8qjfh4= 2022-09-03T06:47:48.555 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fz3sGltfm2vwNkaoBoltT8qjfh4=', 'Content-Length': '0'} 2022-09-03T06:47:48.557 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c0b128d7db1f6aee-006312f894-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.558 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:48.559 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:48.559 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.560 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.560 DEBUG:boto:Method: GET 2022-09-03T06:47:48.561 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.562 DEBUG:boto:Data: 2022-09-03T06:47:48.563 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.564 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.565 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.565 DEBUG:boto:Params: {} 2022-09-03T06:47:48.566 DEBUG:boto:Token: None 2022-09-03T06:47:48.567 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.568 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fz3sGltfm2vwNkaoBoltT8qjfh4= 2022-09-03T06:47:48.569 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fz3sGltfm2vwNkaoBoltT8qjfh4=', 'Content-Length': '0'} 2022-09-03T06:47:48.571 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000538819e7ca9a9157-006312f894-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.572 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:48.572 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:48.573 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.574 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.574 DEBUG:boto:Method: HEAD 2022-09-03T06:47:48.575 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.575 DEBUG:boto:Data: 2022-09-03T06:47:48.576 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.576 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.577 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.578 DEBUG:boto:Params: {} 2022-09-03T06:47:48.578 DEBUG:boto:Token: None 2022-09-03T06:47:48.579 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.580 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vw09RKmMTLiLI4nmSqBO7S6Oe5U= 2022-09-03T06:47:48.581 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vw09RKmMTLiLI4nmSqBO7S6Oe5U=', 'Content-Length': '0'} 2022-09-03T06:47:48.583 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000694a1581d86d302c-006312f894-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.583 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:48.584 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:48.585 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.585 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.586 DEBUG:boto:Method: HEAD 2022-09-03T06:47:48.586 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.587 DEBUG:boto:Data: 2022-09-03T06:47:48.587 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.588 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.589 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.589 DEBUG:boto:Params: {} 2022-09-03T06:47:48.590 DEBUG:boto:Token: None 2022-09-03T06:47:48.590 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.591 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vw09RKmMTLiLI4nmSqBO7S6Oe5U= 2022-09-03T06:47:48.592 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vw09RKmMTLiLI4nmSqBO7S6Oe5U=', 'Content-Length': '0'} 2022-09-03T06:47:48.594 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c3ec7f7fa79e72cf-006312f894-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.595 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:47:48.595 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:48.596 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:48.596 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.597 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.597 DEBUG:boto:Method: GET 2022-09-03T06:47:48.598 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.599 DEBUG:boto:Data: 2022-09-03T06:47:48.599 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.600 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.600 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.601 DEBUG:boto:Params: {} 2022-09-03T06:47:48.601 DEBUG:boto:Token: None 2022-09-03T06:47:48.602 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.603 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fz3sGltfm2vwNkaoBoltT8qjfh4= 2022-09-03T06:47:48.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fz3sGltfm2vwNkaoBoltT8qjfh4=', 'Content-Length': '0'} 2022-09-03T06:47:48.606 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a471f25dd80c5c5-006312f894-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.607 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:48.607 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:48.608 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.608 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.609 DEBUG:boto:Method: GET 2022-09-03T06:47:48.609 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.610 DEBUG:boto:Data: 2022-09-03T06:47:48.611 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.611 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.612 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.612 DEBUG:boto:Params: {} 2022-09-03T06:47:48.613 DEBUG:boto:Token: None 2022-09-03T06:47:48.614 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:47:48.614 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fz3sGltfm2vwNkaoBoltT8qjfh4= 2022-09-03T06:47:48.615 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fz3sGltfm2vwNkaoBoltT8qjfh4=', 'Content-Length': '0'} 2022-09-03T06:47:48.617 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a2e3f1dc3aaa31dd-006312f894-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.658 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:48.659 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:48.659 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:48.660 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:48.661 DEBUG:boto:Method: GET 2022-09-03T06:47:48.661 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:47:48.662 DEBUG:boto:Data: 2022-09-03T06:47:48.663 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.664 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.664 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.665 DEBUG:boto:Params: {} 2022-09-03T06:47:48.666 DEBUG:boto:Token: None 2022-09-03T06:47:48.667 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/ 2022-09-03T06:47:48.668 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg= 2022-09-03T06:47:48.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg=', 'Content-Length': '0'} 2022-09-03T06:47:48.673 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000083068a18389084e0-006312f894-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.673 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:48.674 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:48.675 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:48.676 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:48.677 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:48.678 DEBUG:boto:Method: GET 2022-09-03T06:47:48.679 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:47:48.680 DEBUG:boto:Data: 2022-09-03T06:47:48.681 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.681 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.682 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.683 DEBUG:boto:Params: {} 2022-09-03T06:47:48.684 DEBUG:boto:Token: None 2022-09-03T06:47:48.684 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/ 2022-09-03T06:47:48.685 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg= 2022-09-03T06:47:48.686 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg=', 'Content-Length': '0'} 2022-09-03T06:47:48.689 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d6c88c9435ec0d3a-006312f894-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.690 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:48.691 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:48.691 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:48.692 DEBUG:boto:Method: HEAD 2022-09-03T06:47:48.692 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:47:48.693 DEBUG:boto:Data: 2022-09-03T06:47:48.693 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.694 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.694 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.695 DEBUG:boto:Params: {} 2022-09-03T06:47:48.695 DEBUG:boto:Token: None 2022-09-03T06:47:48.696 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj92 2022-09-03T06:47:48.697 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z0xTtiCHmK+mwfRH0sYongjj0GM= 2022-09-03T06:47:48.698 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z0xTtiCHmK+mwfRH0sYongjj0GM=', 'Content-Length': '0'} 2022-09-03T06:47:48.700 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009a70a9e24b826cee-006312f894-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.701 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:48.702 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:48.702 DEBUG:boto:Method: HEAD 2022-09-03T06:47:48.703 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:47:48.704 DEBUG:boto:Data: 2022-09-03T06:47:48.704 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.705 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.705 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.706 DEBUG:boto:Params: {} 2022-09-03T06:47:48.707 DEBUG:boto:Token: None 2022-09-03T06:47:48.707 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj92 2022-09-03T06:47:48.708 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z0xTtiCHmK+mwfRH0sYongjj0GM= 2022-09-03T06:47:48.709 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z0xTtiCHmK+mwfRH0sYongjj0GM=', 'Content-Length': '0'} 2022-09-03T06:47:48.711 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e130a14cfac0658-006312f894-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.712 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:47:48.713 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:48.713 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:48.714 DEBUG:boto:Method: GET 2022-09-03T06:47:48.715 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:47:48.715 DEBUG:boto:Data: 2022-09-03T06:47:48.716 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.716 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.717 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.717 DEBUG:boto:Params: {} 2022-09-03T06:47:48.719 DEBUG:boto:Token: None 2022-09-03T06:47:48.719 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj92 2022-09-03T06:47:48.720 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BM0wF3+UUCqxLe5vco7DsMSXAnE= 2022-09-03T06:47:48.721 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BM0wF3+UUCqxLe5vco7DsMSXAnE=', 'Content-Length': '0'} 2022-09-03T06:47:48.723 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bbf6ddf728f49e09-006312f894-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.723 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:47:48.724 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:47:48.725 DEBUG:boto:Method: GET 2022-09-03T06:47:48.725 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:47:48.726 DEBUG:boto:Data: 2022-09-03T06:47:48.727 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.727 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.728 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.729 DEBUG:boto:Params: {} 2022-09-03T06:47:48.729 DEBUG:boto:Token: None 2022-09-03T06:47:48.730 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj92 2022-09-03T06:47:48.730 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BM0wF3+UUCqxLe5vco7DsMSXAnE= 2022-09-03T06:47:48.731 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BM0wF3+UUCqxLe5vco7DsMSXAnE=', 'Content-Length': '0'} 2022-09-03T06:47:48.733 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000184060eb7434e119-006312f894-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.735 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:48.735 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:48.736 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:48.736 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:48.737 DEBUG:boto:Method: GET 2022-09-03T06:47:48.737 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:47:48.738 DEBUG:boto:Data: 2022-09-03T06:47:48.738 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.739 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.740 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.740 DEBUG:boto:Params: {} 2022-09-03T06:47:48.741 DEBUG:boto:Token: None 2022-09-03T06:47:48.741 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/ 2022-09-03T06:47:48.742 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg= 2022-09-03T06:47:48.743 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg=', 'Content-Length': '0'} 2022-09-03T06:47:48.746 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b455dd8e9134bcdb-006312f894-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.786 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:48.787 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:48.788 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:48.788 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:48.789 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:47:48.789 DEBUG:boto:Method: GET 2022-09-03T06:47:48.790 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:47:48.791 DEBUG:boto:Data: 2022-09-03T06:47:48.791 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.792 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.793 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.793 DEBUG:boto:Params: {} 2022-09-03T06:47:48.794 DEBUG:boto:Token: None 2022-09-03T06:47:48.795 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/ 2022-09-03T06:47:48.796 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg= 2022-09-03T06:47:48.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg=', 'Content-Length': '0'} 2022-09-03T06:47:48.800 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006ed2798dbd0c4551-006312f894-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.801 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:48.802 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2022-09-03T06:47:48.803 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:48.804 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:48.804 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.805 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.805 DEBUG:boto:Method: GET 2022-09-03T06:47:48.806 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.807 DEBUG:boto:Data: 2022-09-03T06:47:48.808 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.808 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.809 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.810 DEBUG:boto:Params: {} 2022-09-03T06:47:48.811 DEBUG:boto:Token: None 2022-09-03T06:47:48.811 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.812 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uFTuAGTruTrtlzXwzxaACkeFolM= 2022-09-03T06:47:48.813 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uFTuAGTruTrtlzXwzxaACkeFolM=', 'Content-Length': '0'} 2022-09-03T06:47:48.815 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000045c82caf9abc0767-006312f894-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.816 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:48.817 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:48.818 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.818 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.819 DEBUG:boto:Method: GET 2022-09-03T06:47:48.820 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.820 DEBUG:boto:Data: 2022-09-03T06:47:48.821 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.821 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.822 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.823 DEBUG:boto:Params: {} 2022-09-03T06:47:48.824 DEBUG:boto:Token: None 2022-09-03T06:47:48.825 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.826 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uFTuAGTruTrtlzXwzxaACkeFolM= 2022-09-03T06:47:48.827 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uFTuAGTruTrtlzXwzxaACkeFolM=', 'Content-Length': '0'} 2022-09-03T06:47:48.830 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000081b3e8aa0e128fb9-006312f894-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.831 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:48.832 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:48.833 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.834 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.835 DEBUG:boto:Method: HEAD 2022-09-03T06:47:48.836 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.836 DEBUG:boto:Data: 2022-09-03T06:47:48.837 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.837 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.838 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.839 DEBUG:boto:Params: {} 2022-09-03T06:47:48.839 DEBUG:boto:Token: None 2022-09-03T06:47:48.840 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.841 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vgw9HHcMMT/DFjh69DbfE08zScY= 2022-09-03T06:47:48.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vgw9HHcMMT/DFjh69DbfE08zScY=', 'Content-Length': '0'} 2022-09-03T06:47:48.844 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002edb5e41cca5b870-006312f894-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.845 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:48.846 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:48.847 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.848 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.849 DEBUG:boto:Method: HEAD 2022-09-03T06:47:48.850 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.850 DEBUG:boto:Data: 2022-09-03T06:47:48.851 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.852 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.852 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.853 DEBUG:boto:Params: {} 2022-09-03T06:47:48.854 DEBUG:boto:Token: None 2022-09-03T06:47:48.854 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.855 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vgw9HHcMMT/DFjh69DbfE08zScY= 2022-09-03T06:47:48.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vgw9HHcMMT/DFjh69DbfE08zScY=', 'Content-Length': '0'} 2022-09-03T06:47:48.858 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008faf377cb20d1e69-006312f894-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.869 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2022-09-03T06:47:48.870 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:48.871 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:48.871 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.872 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.873 DEBUG:boto:Method: GET 2022-09-03T06:47:48.873 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.874 DEBUG:boto:Data: 2022-09-03T06:47:48.875 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.875 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.876 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.876 DEBUG:boto:Params: {} 2022-09-03T06:47:48.877 DEBUG:boto:Token: None 2022-09-03T06:47:48.878 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.878 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uFTuAGTruTrtlzXwzxaACkeFolM= 2022-09-03T06:47:48.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uFTuAGTruTrtlzXwzxaACkeFolM=', 'Content-Length': '0'} 2022-09-03T06:47:48.881 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065da6a8635a70d73-006312f894-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.882 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:48.883 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:48.884 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.884 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.885 DEBUG:boto:Method: GET 2022-09-03T06:47:48.886 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.886 DEBUG:boto:Data: 2022-09-03T06:47:48.887 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.888 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.888 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.889 DEBUG:boto:Params: {} 2022-09-03T06:47:48.889 DEBUG:boto:Token: None 2022-09-03T06:47:48.890 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:47:48.891 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uFTuAGTruTrtlzXwzxaACkeFolM= 2022-09-03T06:47:48.891 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uFTuAGTruTrtlzXwzxaACkeFolM=', 'Content-Length': '0'} 2022-09-03T06:47:48.894 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b9ff75a17d1901e-006312f894-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.895 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:48.895 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:48.896 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:48.896 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:48.897 DEBUG:boto:Method: GET 2022-09-03T06:47:48.897 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:47:48.898 DEBUG:boto:Data: 2022-09-03T06:47:48.898 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.899 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.899 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.900 DEBUG:boto:Params: {} 2022-09-03T06:47:48.900 DEBUG:boto:Token: None 2022-09-03T06:47:48.901 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/ 2022-09-03T06:47:48.902 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg= 2022-09-03T06:47:48.902 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg=', 'Content-Length': '0'} 2022-09-03T06:47:48.905 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000798adc2030ca07cf-006312f894-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.906 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:48.907 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:48.907 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:48.908 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:48.908 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:48.909 DEBUG:boto:Method: GET 2022-09-03T06:47:48.909 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:47:48.910 DEBUG:boto:Data: 2022-09-03T06:47:48.910 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.911 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.911 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.912 DEBUG:boto:Params: {} 2022-09-03T06:47:48.912 DEBUG:boto:Token: None 2022-09-03T06:47:48.913 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/ 2022-09-03T06:47:48.914 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg= 2022-09-03T06:47:48.914 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:099MxYp5kwTgo9vIlz0/LLx/Wcg=', 'Content-Length': '0'} 2022-09-03T06:47:48.917 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000097edde502387dc27-006312f894-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.918 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:48.919 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:48.919 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:48.920 DEBUG:boto:Method: HEAD 2022-09-03T06:47:48.920 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:47:48.921 DEBUG:boto:Data: 2022-09-03T06:47:48.921 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.922 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.922 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.923 DEBUG:boto:Params: {} 2022-09-03T06:47:48.924 DEBUG:boto:Token: None 2022-09-03T06:47:48.924 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj93 2022-09-03T06:47:48.925 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BTcLSFLcjaZzYmYypQlS1g3rbDk= 2022-09-03T06:47:48.926 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BTcLSFLcjaZzYmYypQlS1g3rbDk=', 'Content-Length': '0'} 2022-09-03T06:47:48.928 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004c29998ecd9ba368-006312f894-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.929 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:48.929 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:48.930 DEBUG:boto:Method: HEAD 2022-09-03T06:47:48.931 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:47:48.931 DEBUG:boto:Data: 2022-09-03T06:47:48.932 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.932 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.933 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.933 DEBUG:boto:Params: {} 2022-09-03T06:47:48.934 DEBUG:boto:Token: None 2022-09-03T06:47:48.935 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj93 2022-09-03T06:47:48.935 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BTcLSFLcjaZzYmYypQlS1g3rbDk= 2022-09-03T06:47:48.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BTcLSFLcjaZzYmYypQlS1g3rbDk=', 'Content-Length': '0'} 2022-09-03T06:47:48.938 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003753ef8ff1daabac-006312f894-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.939 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2022-09-03T06:47:48.939 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:48.940 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:48.941 DEBUG:boto:Method: GET 2022-09-03T06:47:48.941 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:47:48.942 DEBUG:boto:Data: 2022-09-03T06:47:48.942 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.943 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:48.943 DEBUG:boto:Port: 8001 2022-09-03T06:47:48.944 DEBUG:boto:Params: {} 2022-09-03T06:47:48.945 DEBUG:boto:Token: None 2022-09-03T06:47:48.945 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj93 2022-09-03T06:47:48.946 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WwPqneGDn8FjjTnwg/Wyyr3H+NA= 2022-09-03T06:47:48.947 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WwPqneGDn8FjjTnwg/Wyyr3H+NA=', 'Content-Length': '0'} 2022-09-03T06:47:48.949 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001df9e8868c3aac6c-006312f894-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:48.950 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:47:48.950 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:47:48.951 DEBUG:boto:Method: GET 2022-09-03T06:47:48.951 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:47:48.952 DEBUG:boto:Data: 2022-09-03T06:47:48.952 DEBUG:boto:Headers: {} 2022-09-03T06:47:48.953 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:48.953 DEBUG:boto:Port: 8000 2022-09-03T06:47:48.954 DEBUG:boto:Params: {} 2022-09-03T06:47:48.955 DEBUG:boto:Token: None 2022-09-03T06:47:48.955 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:48 GMT /dsgvcy-75/obj93 2022-09-03T06:47:48.956 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WwPqneGDn8FjjTnwg/Wyyr3H+NA= 2022-09-03T06:47:48.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WwPqneGDn8FjjTnwg/Wyyr3H+NA=', 'Content-Length': '0'} 2022-09-03T06:47:48.959 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e1b02d1ee4075ff2-006312f894-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.010 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:49.014 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:49.015 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:49.016 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:49.017 DEBUG:boto:Method: GET 2022-09-03T06:47:49.018 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:47:49.018 DEBUG:boto:Data: 2022-09-03T06:47:49.019 DEBUG:boto:Headers: {} 2022-09-03T06:47:49.020 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:49.021 DEBUG:boto:Port: 8001 2022-09-03T06:47:49.022 DEBUG:boto:Params: {} 2022-09-03T06:47:49.023 DEBUG:boto:Token: None 2022-09-03T06:47:49.024 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/ 2022-09-03T06:47:49.025 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QRhUxJoYI5nTnTZiG5imOSTQ57U= 2022-09-03T06:47:49.027 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QRhUxJoYI5nTnTZiG5imOSTQ57U=', 'Content-Length': '0'} 2022-09-03T06:47:49.030 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004704adb36c3e5fd4-006312f895-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.031 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:49.033 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:49.033 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:49.034 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:49.035 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:47:49.036 DEBUG:boto:Method: GET 2022-09-03T06:47:49.038 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:47:49.039 DEBUG:boto:Data: 2022-09-03T06:47:49.040 DEBUG:boto:Headers: {} 2022-09-03T06:47:49.041 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:49.042 DEBUG:boto:Port: 8000 2022-09-03T06:47:49.043 DEBUG:boto:Params: {} 2022-09-03T06:47:49.044 DEBUG:boto:Token: None 2022-09-03T06:47:49.045 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/ 2022-09-03T06:47:49.046 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QRhUxJoYI5nTnTZiG5imOSTQ57U= 2022-09-03T06:47:49.048 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QRhUxJoYI5nTnTZiG5imOSTQ57U=', 'Content-Length': '0'} 2022-09-03T06:47:49.051 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002820a2924ca060a8-006312f895-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.053 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:49.054 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:47:49.055 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:49.056 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:49.057 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.058 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.058 DEBUG:boto:Method: GET 2022-09-03T06:47:49.059 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.059 DEBUG:boto:Data: 2022-09-03T06:47:49.060 DEBUG:boto:Headers: {} 2022-09-03T06:47:49.061 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:49.061 DEBUG:boto:Port: 8001 2022-09-03T06:47:49.062 DEBUG:boto:Params: {} 2022-09-03T06:47:49.062 DEBUG:boto:Token: None 2022-09-03T06:47:49.063 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.064 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:foO7jKPaLNwCanpYF+RXN0LA0e4= 2022-09-03T06:47:49.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:foO7jKPaLNwCanpYF+RXN0LA0e4=', 'Content-Length': '0'} 2022-09-03T06:47:49.067 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000047f5aa51832c4077-006312f895-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.110 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:49.111 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:49.112 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.112 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.113 DEBUG:boto:Method: GET 2022-09-03T06:47:49.113 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.114 DEBUG:boto:Data: 2022-09-03T06:47:49.115 DEBUG:boto:Headers: {} 2022-09-03T06:47:49.115 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:49.116 DEBUG:boto:Port: 8000 2022-09-03T06:47:49.116 DEBUG:boto:Params: {} 2022-09-03T06:47:49.117 DEBUG:boto:Token: None 2022-09-03T06:47:49.118 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.119 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:foO7jKPaLNwCanpYF+RXN0LA0e4= 2022-09-03T06:47:49.120 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:foO7jKPaLNwCanpYF+RXN0LA0e4=', 'Content-Length': '0'} 2022-09-03T06:47:49.122 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004bdf1d369a72a9ac-006312f895-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.123 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:49.124 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:49.124 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.125 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.125 DEBUG:boto:Method: HEAD 2022-09-03T06:47:49.126 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.126 DEBUG:boto:Data: 2022-09-03T06:47:49.127 DEBUG:boto:Headers: {} 2022-09-03T06:47:49.128 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:49.129 DEBUG:boto:Port: 8001 2022-09-03T06:47:49.130 DEBUG:boto:Params: {} 2022-09-03T06:47:49.131 DEBUG:boto:Token: None 2022-09-03T06:47:49.132 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.133 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p2VR0Z+t+jJtu3c8yV7/FktJ2qQ= 2022-09-03T06:47:49.134 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p2VR0Z+t+jJtu3c8yV7/FktJ2qQ=', 'Content-Length': '0'} 2022-09-03T06:47:49.136 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004b24aafbfc19b484-006312f895-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.137 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:49.138 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:49.139 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.139 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.140 DEBUG:boto:Method: HEAD 2022-09-03T06:47:49.140 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.141 DEBUG:boto:Data: 2022-09-03T06:47:49.141 DEBUG:boto:Headers: {} 2022-09-03T06:47:49.142 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:49.142 DEBUG:boto:Port: 8000 2022-09-03T06:47:49.143 DEBUG:boto:Params: {} 2022-09-03T06:47:49.144 DEBUG:boto:Token: None 2022-09-03T06:47:49.144 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.145 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:p2VR0Z+t+jJtu3c8yV7/FktJ2qQ= 2022-09-03T06:47:49.146 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:p2VR0Z+t+jJtu3c8yV7/FktJ2qQ=', 'Content-Length': '0'} 2022-09-03T06:47:49.148 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009b8fc463090fd8b9-006312f895-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.150 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:47:49.151 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:49.152 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:49.153 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.154 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.155 DEBUG:boto:Method: GET 2022-09-03T06:47:49.155 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.156 DEBUG:boto:Data: 2022-09-03T06:47:49.157 DEBUG:boto:Headers: {} 2022-09-03T06:47:49.158 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:49.159 DEBUG:boto:Port: 8001 2022-09-03T06:47:49.160 DEBUG:boto:Params: {} 2022-09-03T06:47:49.160 DEBUG:boto:Token: None 2022-09-03T06:47:49.161 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.162 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:foO7jKPaLNwCanpYF+RXN0LA0e4= 2022-09-03T06:47:49.162 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:foO7jKPaLNwCanpYF+RXN0LA0e4=', 'Content-Length': '0'} 2022-09-03T06:47:49.164 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004628e83636a8102b-006312f895-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.165 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:49.166 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:49.167 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.167 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.168 DEBUG:boto:Method: GET 2022-09-03T06:47:49.168 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.169 DEBUG:boto:Data: 2022-09-03T06:47:49.170 DEBUG:boto:Headers: {} 2022-09-03T06:47:49.171 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:49.172 DEBUG:boto:Port: 8000 2022-09-03T06:47:49.172 DEBUG:boto:Params: {} 2022-09-03T06:47:49.173 DEBUG:boto:Token: None 2022-09-03T06:47:49.174 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:47:49.176 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:foO7jKPaLNwCanpYF+RXN0LA0e4= 2022-09-03T06:47:49.176 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:foO7jKPaLNwCanpYF+RXN0LA0e4=', 'Content-Length': '0'} 2022-09-03T06:47:49.179 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000881460c043688e9f-006312f895-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.180 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:49.181 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:49.182 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:49.182 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:49.183 DEBUG:boto:Method: GET 2022-09-03T06:47:49.184 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:47:49.184 DEBUG:boto:Data: 2022-09-03T06:47:49.185 DEBUG:boto:Headers: {} 2022-09-03T06:47:49.186 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:49.186 DEBUG:boto:Port: 8001 2022-09-03T06:47:49.187 DEBUG:boto:Params: {} 2022-09-03T06:47:49.187 DEBUG:boto:Token: None 2022-09-03T06:47:49.195 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/ 2022-09-03T06:47:49.196 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QRhUxJoYI5nTnTZiG5imOSTQ57U= 2022-09-03T06:47:49.197 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QRhUxJoYI5nTnTZiG5imOSTQ57U=', 'Content-Length': '0'} 2022-09-03T06:47:49.199 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007641c614576180f8-006312f895-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.200 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:49.201 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:49.202 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:49.202 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:49.203 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:49.204 DEBUG:boto:Method: GET 2022-09-03T06:47:49.204 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:47:49.205 DEBUG:boto:Data: 2022-09-03T06:47:49.205 DEBUG:boto:Headers: {} 2022-09-03T06:47:49.206 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:49.207 DEBUG:boto:Port: 8000 2022-09-03T06:47:49.207 DEBUG:boto:Params: {} 2022-09-03T06:47:49.208 DEBUG:boto:Token: None 2022-09-03T06:47:49.209 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/ 2022-09-03T06:47:49.209 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QRhUxJoYI5nTnTZiG5imOSTQ57U= 2022-09-03T06:47:49.210 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QRhUxJoYI5nTnTZiG5imOSTQ57U=', 'Content-Length': '0'} 2022-09-03T06:47:49.213 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eeee4b9281919710-006312f895-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.214 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:49.215 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:49.215 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:49.216 DEBUG:boto:Method: HEAD 2022-09-03T06:47:49.217 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:47:49.217 DEBUG:boto:Data: 2022-09-03T06:47:49.218 DEBUG:boto:Headers: {} 2022-09-03T06:47:49.219 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:49.219 DEBUG:boto:Port: 8001 2022-09-03T06:47:49.220 DEBUG:boto:Params: {} 2022-09-03T06:47:49.220 DEBUG:boto:Token: None 2022-09-03T06:47:49.221 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/obj95 2022-09-03T06:47:49.222 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1GEQ77hkaTAsVlhbHnmvaUT54uA= 2022-09-03T06:47:49.223 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1GEQ77hkaTAsVlhbHnmvaUT54uA=', 'Content-Length': '0'} 2022-09-03T06:47:49.225 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b0dce51ca91a9257-006312f895-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.226 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:49.227 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:49.228 DEBUG:boto:Method: HEAD 2022-09-03T06:47:49.228 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:47:49.229 DEBUG:boto:Data: 2022-09-03T06:47:49.229 DEBUG:boto:Headers: {} 2022-09-03T06:47:49.230 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:49.230 DEBUG:boto:Port: 8000 2022-09-03T06:47:49.231 DEBUG:boto:Params: {} 2022-09-03T06:47:49.231 DEBUG:boto:Token: None 2022-09-03T06:47:49.232 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/obj95 2022-09-03T06:47:49.233 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1GEQ77hkaTAsVlhbHnmvaUT54uA= 2022-09-03T06:47:49.233 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1GEQ77hkaTAsVlhbHnmvaUT54uA=', 'Content-Length': '0'} 2022-09-03T06:47:49.236 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000943c675f425e23b3-006312f895-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.237 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:47:49.237 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:49.238 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:49.239 DEBUG:boto:Method: GET 2022-09-03T06:47:49.239 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:47:49.240 DEBUG:boto:Data: 2022-09-03T06:47:49.241 DEBUG:boto:Headers: {} 2022-09-03T06:47:49.241 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:49.242 DEBUG:boto:Port: 8001 2022-09-03T06:47:49.243 DEBUG:boto:Params: {} 2022-09-03T06:47:49.243 DEBUG:boto:Token: None 2022-09-03T06:47:49.244 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/obj95 2022-09-03T06:47:49.245 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dhe3E5Ua5AfWwnxkNkvAI8P0bkI= 2022-09-03T06:47:49.246 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dhe3E5Ua5AfWwnxkNkvAI8P0bkI=', 'Content-Length': '0'} 2022-09-03T06:47:49.248 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a7e074b062acfaf-006312f895-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.249 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:47:49.250 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:47:49.251 DEBUG:boto:Method: GET 2022-09-03T06:47:49.251 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:47:49.252 DEBUG:boto:Data: 2022-09-03T06:47:49.253 DEBUG:boto:Headers: {} 2022-09-03T06:47:49.253 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:49.254 DEBUG:boto:Port: 8000 2022-09-03T06:47:49.255 DEBUG:boto:Params: {} 2022-09-03T06:47:49.255 DEBUG:boto:Token: None 2022-09-03T06:47:49.256 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/obj95 2022-09-03T06:47:49.257 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dhe3E5Ua5AfWwnxkNkvAI8P0bkI= 2022-09-03T06:47:49.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dhe3E5Ua5AfWwnxkNkvAI8P0bkI=', 'Content-Length': '0'} 2022-09-03T06:47:49.260 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b22c3f8d255249b4-006312f895-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.262 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:49.262 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:49.263 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:49.263 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:49.264 DEBUG:boto:Method: GET 2022-09-03T06:47:49.265 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:47:49.265 DEBUG:boto:Data: 2022-09-03T06:47:49.266 DEBUG:boto:Headers: {} 2022-09-03T06:47:49.266 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:47:49.267 DEBUG:boto:Port: 8001 2022-09-03T06:47:49.268 DEBUG:boto:Params: {} 2022-09-03T06:47:49.269 DEBUG:boto:Token: None 2022-09-03T06:47:49.270 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/ 2022-09-03T06:47:49.271 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QRhUxJoYI5nTnTZiG5imOSTQ57U= 2022-09-03T06:47:49.271 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QRhUxJoYI5nTnTZiG5imOSTQ57U=', 'Content-Length': '0'} 2022-09-03T06:47:49.274 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c9ea6ac1d2674008-006312f895-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.275 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:49.277 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:49.277 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:49.278 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:49.279 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:47:49.279 DEBUG:boto:Method: GET 2022-09-03T06:47:49.280 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:47:49.280 DEBUG:boto:Data: 2022-09-03T06:47:49.281 DEBUG:boto:Headers: {} 2022-09-03T06:47:49.281 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:47:49.282 DEBUG:boto:Port: 8000 2022-09-03T06:47:49.282 DEBUG:boto:Params: {} 2022-09-03T06:47:49.283 DEBUG:boto:Token: None 2022-09-03T06:47:49.283 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/ 2022-09-03T06:47:49.284 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QRhUxJoYI5nTnTZiG5imOSTQ57U= 2022-09-03T06:47:49.285 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QRhUxJoYI5nTnTZiG5imOSTQ57U=', 'Content-Length': '0'} 2022-09-03T06:47:49.287 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007df3bf6023657cd5-006312f895-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.288 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:47:49.289 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-75 zones={test-zone3, test-zone2} 2022-09-03T06:47:49.290 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--num-shards', '37', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:47:49.291 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'reshard', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--num-shards', '37', '--yes-i-really-mean-it', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:47:49.291 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket reshard --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --num-shards 37 --yes-i-really-mean-it --cluster c1 --debug-rgw 0 --debug-ms 0 2022-09-03T06:47:49.351 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:47:49.352 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:47:49.416 INFO:teuthology.orchestra.run.smithi181.stdout:tenant: 2022-09-03T06:47:49.417 INFO:teuthology.orchestra.run.smithi181.stdout:bucket name: dsgvcy-75 2022-09-03T06:47:49.429 INFO:teuthology.orchestra.run.smithi181.stdout:total entries: 30 2022-09-03T06:47:49.497 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:49.498 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:49.498 DEBUG:boto:Method: HEAD 2022-09-03T06:47:49.499 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:47:49.499 DEBUG:boto:Data: 2022-09-03T06:47:49.500 DEBUG:boto:Headers: {} 2022-09-03T06:47:49.500 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:49.501 DEBUG:boto:Port: 8000 2022-09-03T06:47:49.501 DEBUG:boto:Params: {} 2022-09-03T06:47:49.502 DEBUG:boto:Token: None 2022-09-03T06:47:49.503 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/ 2022-09-03T06:47:49.503 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Nt4gqULYdz68lmva8e/z63Z5ACw= 2022-09-03T06:47:49.504 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Nt4gqULYdz68lmva8e/z63Z5ACw=', 'Content-Length': '0'} 2022-09-03T06:47:49.508 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '30'), ('X-RGW-Bytes-Used', '90'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000006ca549cffc8e7dcf-006312f895-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.509 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:47:49.510 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:47:49.510 DEBUG:boto:Method: PUT 2022-09-03T06:47:49.511 DEBUG:boto:Path: /dsgvcy-75/obj37 2022-09-03T06:47:49.511 DEBUG:boto:Data: 2022-09-03T06:47:49.512 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:47:49.512 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:49.513 DEBUG:boto:Port: 8000 2022-09-03T06:47:49.513 DEBUG:boto:Params: {} 2022-09-03T06:47:49.514 DEBUG:boto:Token: None 2022-09-03T06:47:49.515 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/obj37 2022-09-03T06:47:49.515 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8HAZU6Rl/hnqznG51I/YOLLnFBs= 2022-09-03T06:47:49.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8HAZU6Rl/hnqznG51I/YOLLnFBs='} 2022-09-03T06:47:49.562 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000003e56cc60efad7a47-006312f895-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.564 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:49.568 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:49.569 DEBUG:boto:Method: HEAD 2022-09-03T06:47:49.570 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:47:49.571 DEBUG:boto:Data: 2022-09-03T06:47:49.572 DEBUG:boto:Headers: {} 2022-09-03T06:47:49.573 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:49.574 DEBUG:boto:Port: 8000 2022-09-03T06:47:49.575 DEBUG:boto:Params: {} 2022-09-03T06:47:49.575 DEBUG:boto:Token: None 2022-09-03T06:47:49.576 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/ 2022-09-03T06:47:49.577 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Nt4gqULYdz68lmva8e/z63Z5ACw= 2022-09-03T06:47:49.578 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Nt4gqULYdz68lmva8e/z63Z5ACw=', 'Content-Length': '0'} 2022-09-03T06:47:49.583 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '31'), ('X-RGW-Bytes-Used', '93'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000acf4cec3c4948d9b-006312f895-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.584 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:47:49.584 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:47:49.585 DEBUG:boto:Method: PUT 2022-09-03T06:47:49.586 DEBUG:boto:Path: /dsgvcy-75/obj74 2022-09-03T06:47:49.586 DEBUG:boto:Data: 2022-09-03T06:47:49.587 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:47:49.587 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:49.588 DEBUG:boto:Port: 8000 2022-09-03T06:47:49.589 DEBUG:boto:Params: {} 2022-09-03T06:47:49.590 DEBUG:boto:Token: None 2022-09-03T06:47:49.591 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/obj74 2022-09-03T06:47:49.592 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:shQJy82w7EuG5R+GzVRRN5o+C+g= 2022-09-03T06:47:49.593 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:shQJy82w7EuG5R+GzVRRN5o+C+g='} 2022-09-03T06:47:49.603 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000007c362e81a177bf3b-006312f895-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.605 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:49.606 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:49.607 DEBUG:boto:Method: HEAD 2022-09-03T06:47:49.608 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:47:49.609 DEBUG:boto:Data: 2022-09-03T06:47:49.610 DEBUG:boto:Headers: {} 2022-09-03T06:47:49.611 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:49.612 DEBUG:boto:Port: 8000 2022-09-03T06:47:49.612 DEBUG:boto:Params: {} 2022-09-03T06:47:49.613 DEBUG:boto:Token: None 2022-09-03T06:47:49.614 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/ 2022-09-03T06:47:49.615 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Nt4gqULYdz68lmva8e/z63Z5ACw= 2022-09-03T06:47:49.617 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Nt4gqULYdz68lmva8e/z63Z5ACw=', 'Content-Length': '0'} 2022-09-03T06:47:49.621 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '32'), ('X-RGW-Bytes-Used', '96'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000c6804657851677e4-006312f895-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.622 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:47:49.623 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:47:49.624 DEBUG:boto:Method: PUT 2022-09-03T06:47:49.625 DEBUG:boto:Path: /dsgvcy-75/obj111 2022-09-03T06:47:49.626 DEBUG:boto:Data: 2022-09-03T06:47:49.627 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:47:49.628 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:49.628 DEBUG:boto:Port: 8000 2022-09-03T06:47:49.629 DEBUG:boto:Params: {} 2022-09-03T06:47:49.630 DEBUG:boto:Token: None 2022-09-03T06:47:49.631 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/obj111 2022-09-03T06:47:49.632 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j4fngimvdv0EqjyAFDoJm3zI9aY= 2022-09-03T06:47:49.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j4fngimvdv0EqjyAFDoJm3zI9aY='} 2022-09-03T06:47:49.678 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000dc0e62edf8cbd501-006312f895-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.680 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:49.680 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:49.681 DEBUG:boto:Method: HEAD 2022-09-03T06:47:49.681 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:47:49.682 DEBUG:boto:Data: 2022-09-03T06:47:49.682 DEBUG:boto:Headers: {} 2022-09-03T06:47:49.683 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:49.683 DEBUG:boto:Port: 8000 2022-09-03T06:47:49.684 DEBUG:boto:Params: {} 2022-09-03T06:47:49.685 DEBUG:boto:Token: None 2022-09-03T06:47:49.686 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/ 2022-09-03T06:47:49.686 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Nt4gqULYdz68lmva8e/z63Z5ACw= 2022-09-03T06:47:49.687 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Nt4gqULYdz68lmva8e/z63Z5ACw=', 'Content-Length': '0'} 2022-09-03T06:47:49.691 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '33'), ('X-RGW-Bytes-Used', '99'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000385697f498cac742-006312f895-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.691 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:47:49.692 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:47:49.692 DEBUG:boto:Method: PUT 2022-09-03T06:47:49.693 DEBUG:boto:Path: /dsgvcy-75/obj148 2022-09-03T06:47:49.693 DEBUG:boto:Data: 2022-09-03T06:47:49.694 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:47:49.695 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:49.697 DEBUG:boto:Port: 8000 2022-09-03T06:47:49.697 DEBUG:boto:Params: {} 2022-09-03T06:47:49.698 DEBUG:boto:Token: None 2022-09-03T06:47:49.699 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/obj148 2022-09-03T06:47:49.700 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xlm6hMaIipo84DxhVST/AZwl7QA= 2022-09-03T06:47:49.701 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xlm6hMaIipo84DxhVST/AZwl7QA='} 2022-09-03T06:47:49.710 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000094b0c526effe5655-006312f895-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.711 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:47:49.712 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:47:49.712 DEBUG:boto:Method: HEAD 2022-09-03T06:47:49.713 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:47:49.713 DEBUG:boto:Data: 2022-09-03T06:47:49.714 DEBUG:boto:Headers: {} 2022-09-03T06:47:49.714 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:49.715 DEBUG:boto:Port: 8000 2022-09-03T06:47:49.715 DEBUG:boto:Params: {} 2022-09-03T06:47:49.716 DEBUG:boto:Token: None 2022-09-03T06:47:49.717 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/ 2022-09-03T06:47:49.718 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Nt4gqULYdz68lmva8e/z63Z5ACw= 2022-09-03T06:47:49.719 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Nt4gqULYdz68lmva8e/z63Z5ACw=', 'Content-Length': '0'} 2022-09-03T06:47:49.722 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '34'), ('X-RGW-Bytes-Used', '102'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000070f9405333f8cd1d-006312f895-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.723 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:47:49.724 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:47:49.724 DEBUG:boto:Method: PUT 2022-09-03T06:47:49.725 DEBUG:boto:Path: /dsgvcy-75/obj185 2022-09-03T06:47:49.725 DEBUG:boto:Data: 2022-09-03T06:47:49.726 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:47:49.726 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:47:49.727 DEBUG:boto:Port: 8000 2022-09-03T06:47:49.727 DEBUG:boto:Params: {} 2022-09-03T06:47:49.728 DEBUG:boto:Token: None 2022-09-03T06:47:49.729 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:47:49 GMT /dsgvcy-75/obj185 2022-09-03T06:47:49.729 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uzSF8sUyjCH4SrhhUZ+nK/FRoAk= 2022-09-03T06:47:49.730 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:47:49 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uzSF8sUyjCH4SrhhUZ+nK/FRoAk='} 2022-09-03T06:47:49.778 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000063da54d1968567b9-006312f895-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:47:49.780 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-75 2022-09-03T06:47:49.781 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:47:49.782 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:47:49.783 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:47:49.848 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:47:49.848 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:47:49.888 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:47:49.886+0000 7fbff5dc6600 1 waiting to reach latest gen 6, on 5.. 2022-09-03T06:47:54.889 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:47:54.886+0000 7fbff5dc6600 1 waiting to reach latest gen 6, on 5.. 2022-09-03T06:47:59.891 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:47:59.890+0000 7fbff5dc6600 1 waiting for incremental sync to catch up: 2022-09-03T06:47:59.892 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2022-09-03T06:47:59.892 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:48:04.895 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:48:04.893+0000 7fbff5dc6600 1 waiting for incremental sync to catch up: 2022-09-03T06:48:04.896 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2022-09-03T06:48:04.896 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:48:09.897 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:48:09.895+0000 7fbff5dc6600 1 waiting for incremental sync to catch up: 2022-09-03T06:48:09.898 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2022-09-03T06:48:09.898 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:48:14.900 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:48:14.897+0000 7fbff5dc6600 1 waiting for incremental sync to catch up: 2022-09-03T06:48:14.901 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2022-09-03T06:48:14.901 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:48:19.904 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:48:19.901+0000 7fbff5dc6600 1 waiting for incremental sync to catch up: 2022-09-03T06:48:19.905 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , 00000000002.45.5, , , , , , , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:48:19.905 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:48:24.906 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:48:24.904+0000 7fbff5dc6600 1 waiting for incremental sync to catch up: 2022-09-03T06:48:24.907 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , 00000000002.45.5, , , , , , , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:48:24.907 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:48:29.909 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:48:29.907+0000 7fbff5dc6600 1 waiting for incremental sync to catch up: 2022-09-03T06:48:29.909 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , 00000000002.45.5, , , , , , , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:48:29.910 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:48:34.912 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:48:34.910+0000 7fbff5dc6600 1 waiting for incremental sync to catch up: 2022-09-03T06:48:34.912 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , 00000000002.45.5, , , , , , , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:48:34.913 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:48:39.915 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:48:39.913+0000 7fbff5dc6600 1 waiting for incremental sync to catch up: 2022-09-03T06:48:39.915 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:48:39.915 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:48:44.917 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:48:44.915+0000 7fbff5dc6600 1 waiting for incremental sync to catch up: 2022-09-03T06:48:44.918 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:48:44.918 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:48:49.920 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:48:49.918+0000 7fbff5dc6600 1 waiting for incremental sync to catch up: 2022-09-03T06:48:49.921 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:48:49.921 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:48:54.923 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:48:54.921+0000 7fbff5dc6600 1 waiting for incremental sync to catch up: 2022-09-03T06:48:54.924 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , , , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:48:54.924 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:48:59.926 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:48:59.924+0000 7fbff5dc6600 1 bucket sync caught up with source: 2022-09-03T06:48:59.927 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:48:59.927 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:48:59.927 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:48:59.924+0000 7fbff5dc6600 0 bucket checkpoint complete 2022-09-03T06:48:59.945 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone3 bucket=dsgvcy-75 2022-09-03T06:48:59.947 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:48:59.947 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:48:59.949 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:49:00.013 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:49:00.013 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:49:00.057 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:49:00.055+0000 7f6227b64600 1 waiting for incremental sync to catch up: 2022-09-03T06:49:00.058 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , 00000000002.81.5, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ] 2022-09-03T06:49:00.058 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:49:05.060 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:49:05.059+0000 7f6227b64600 1 bucket sync caught up with source: 2022-09-03T06:49:05.061 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:49:05.061 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:49:05.061 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:49:05.059+0000 7f6227b64600 0 bucket checkpoint complete 2022-09-03T06:49:05.079 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone4 bucket=dsgvcy-75 2022-09-03T06:49:05.080 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:49:05.081 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:49:05.081 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:49:05.146 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:49:05.147 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:49:05.195 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:49:05.191+0000 7feae4fcc600 1 waiting for incremental sync to catch up: 2022-09-03T06:49:05.196 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000002.81.5, , , , , , , , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , ] 2022-09-03T06:49:05.196 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:49:10.196 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:49:10.195+0000 7feae4fcc600 1 waiting for incremental sync to catch up: 2022-09-03T06:49:10.197 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000002.81.5, , , , , , , , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , ] 2022-09-03T06:49:10.197 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:49:15.199 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:49:15.197+0000 7feae4fcc600 1 waiting for incremental sync to catch up: 2022-09-03T06:49:15.200 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000002.81.5, , , , , , , , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , ] 2022-09-03T06:49:15.200 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:49:20.202 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:49:20.200+0000 7feae4fcc600 1 waiting for incremental sync to catch up: 2022-09-03T06:49:20.203 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000002.81.5, , , , , , , , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , ] 2022-09-03T06:49:20.203 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:49:25.205 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:49:25.203+0000 7feae4fcc600 1 waiting for incremental sync to catch up: 2022-09-03T06:49:25.206 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000002.81.5, , , , , , , , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , ] 2022-09-03T06:49:25.207 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:49:30.208 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:49:30.206+0000 7feae4fcc600 1 waiting for incremental sync to catch up: 2022-09-03T06:49:30.210 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000002.81.5, , , , , , , , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , ] 2022-09-03T06:49:30.210 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:49:35.211 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:49:35.209+0000 7feae4fcc600 1 waiting for incremental sync to catch up: 2022-09-03T06:49:35.212 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000002.81.5, , , , , , , , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , ] 2022-09-03T06:49:35.212 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:49:40.214 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:49:40.212+0000 7feae4fcc600 1 waiting for incremental sync to catch up: 2022-09-03T06:49:40.214 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000002.81.5, , , , , , , , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , ] 2022-09-03T06:49:40.215 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:49:45.217 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:49:45.215+0000 7feae4fcc600 1 waiting for incremental sync to catch up: 2022-09-03T06:49:45.218 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000002.81.5, , , , , , , , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , ] 2022-09-03T06:49:45.218 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:49:50.219 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:49:50.218+0000 7feae4fcc600 1 waiting for incremental sync to catch up: 2022-09-03T06:49:50.220 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000002.81.5, , , , , , , , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , ] 2022-09-03T06:49:50.220 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:49:55.222 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:49:55.219+0000 7feae4fcc600 1 waiting for incremental sync to catch up: 2022-09-03T06:49:55.223 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000002.81.5, , , , , , , , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , ] 2022-09-03T06:49:55.223 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:50:00.225 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:00.222+0000 7feae4fcc600 1 bucket sync caught up with source: 2022-09-03T06:50:00.226 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:50:00.226 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, , , , , , 00000000002.81.5, , , , , , 00000000002.45.5, , , , , 00000000002.69.5, , , , , , , , , , , , , , , 00000000001.60.5, , , , 00000000001.76.5] 2022-09-03T06:50:00.226 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:00.222+0000 7feae4fcc600 0 bucket checkpoint complete 2022-09-03T06:50:00.247 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone1 bucket=dsgvcy-75 2022-09-03T06:50:00.248 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:50:00.249 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:50:00.250 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:50:00.316 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:00.316 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:00.369 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:50:00.368+0000 7f282bb29600 1 bucket sync caught up with source: 2022-09-03T06:50:00.370 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000003.590.5, 00000000002.739.5, 00000000002.386.5, 00000000005.719.5, 00000000004.380.5, 00000000001.364.5, 00000000001.865.5, 00000000004.877.5, 00000000007.322.5, 00000000006.597.5] 2022-09-03T06:50:00.371 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000003.590.5, 00000000002.739.5, 00000000002.386.5, 00000000005.719.5, 00000000004.380.5, 00000000001.364.5, 00000000001.865.5, 00000000004.877.5, 00000000007.322.5, 00000000006.597.5] 2022-09-03T06:50:00.371 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:50:00.368+0000 7f282bb29600 0 bucket checkpoint complete 2022-09-03T06:50:00.390 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone3 bucket=dsgvcy-75 2022-09-03T06:50:00.391 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:50:00.392 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone2', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:50:00.393 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone2 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:50:00.451 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:00.455 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:00.491 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:50:00.486+0000 7ff7ebc6d600 1 bucket sync caught up with source: 2022-09-03T06:50:00.491 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000003.590.5, 00000000002.739.5, 00000000002.386.5, 00000000005.719.5, 00000000004.380.5, 00000000001.364.5, 00000000001.865.5, 00000000004.877.5, 00000000007.322.5, 00000000006.597.5] 2022-09-03T06:50:00.492 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000003.590.5, 00000000002.739.5, 00000000002.386.5, 00000000005.719.5, 00000000004.380.5, 00000000001.364.5, 00000000001.865.5, 00000000004.877.5, 00000000007.322.5, 00000000006.597.5] 2022-09-03T06:50:00.493 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:50:00.486+0000 7ff7ebc6d600 0 bucket checkpoint complete 2022-09-03T06:50:00.502 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone2 target=test-zone4 bucket=dsgvcy-75 2022-09-03T06:50:00.503 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone1 bucket=dsgvcy-75 2022-09-03T06:50:00.504 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:50:00.505 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:50:00.507 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 1 --debug-ms 0 2022-09-03T06:50:00.565 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:00.566 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:00.607 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:50:00.605+0000 7f7e96e55600 1 bucket sync caught up with source: 2022-09-03T06:50:00.608 INFO:teuthology.orchestra.run.smithi181.stderr: local status: [, 00000000003.210.5, 00000000003.158.5, 00000000002.291.5, 00000000005.281.5, 00000000004.285.5, 00000000001.249.5, 00000000001.327.5, 00000000004.341.5, 00000000007.258.5, 00000000006.301.5] 2022-09-03T06:50:00.608 INFO:teuthology.orchestra.run.smithi181.stderr: remote markers: [, 00000000003.210.5, 00000000003.158.5, 00000000002.291.5, 00000000005.281.5, 00000000004.285.5, 00000000001.249.5, 00000000001.327.5, 00000000004.341.5, 00000000007.258.5, 00000000006.301.5] 2022-09-03T06:50:00.608 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:50:00.605+0000 7f7e96e55600 0 bucket checkpoint complete 2022-09-03T06:50:00.627 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone2 bucket=dsgvcy-75 2022-09-03T06:50:00.628 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:50:00.629 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone3', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:50:00.630 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-75 --source-zone test-zone3 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:50:00.694 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:00.695 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:00.746 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:00.744+0000 7f5aab96d600 1 bucket sync caught up with source: 2022-09-03T06:50:00.746 INFO:teuthology.orchestra.run.smithi189.stderr: local status: [, 00000000003.210.5, 00000000003.158.5, 00000000002.291.5, 00000000005.281.5, 00000000004.285.5, 00000000001.249.5, 00000000001.327.5, 00000000004.341.5, 00000000007.258.5, 00000000006.301.5] 2022-09-03T06:50:00.747 INFO:teuthology.orchestra.run.smithi189.stderr: remote markers: [, 00000000003.210.5, 00000000003.158.5, 00000000002.291.5, 00000000005.281.5, 00000000004.285.5, 00000000001.249.5, 00000000001.327.5, 00000000004.341.5, 00000000007.258.5, 00000000006.301.5] 2022-09-03T06:50:00.747 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:00.744+0000 7f5aab96d600 0 bucket checkpoint complete 2022-09-03T06:50:00.761 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone3 target=test-zone4 bucket=dsgvcy-75 2022-09-03T06:50:00.762 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-75 zones={test-zone2, test-zone1} 2022-09-03T06:50:00.763 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:00.764 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:00.765 DEBUG:boto:Method: HEAD 2022-09-03T06:50:00.765 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:50:00.766 DEBUG:boto:Data: 2022-09-03T06:50:00.767 DEBUG:boto:Headers: {} 2022-09-03T06:50:00.767 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:00.768 DEBUG:boto:Port: 8000 2022-09-03T06:50:00.769 DEBUG:boto:Params: {} 2022-09-03T06:50:00.770 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:50:00.771 DEBUG:boto:Token: None 2022-09-03T06:50:00.772 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:00 GMT /dsgvcy-75/ 2022-09-03T06:50:00.773 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DBWCC0myftCTqheYq0+YWodJm0E= 2022-09-03T06:50:00.774 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DBWCC0myftCTqheYq0+YWodJm0E=', 'Content-Length': '0'} 2022-09-03T06:50:00.780 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '35'), ('X-RGW-Bytes-Used', '105'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b9ea0d11025a45cd-006312f918-188c-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:50:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:00.781 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:00.782 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:00.782 DEBUG:boto:Method: HEAD 2022-09-03T06:50:00.783 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:50:00.783 DEBUG:boto:Data: 2022-09-03T06:50:00.784 DEBUG:boto:Headers: {} 2022-09-03T06:50:00.785 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:00.786 DEBUG:boto:Port: 8000 2022-09-03T06:50:00.787 DEBUG:boto:Params: {} 2022-09-03T06:50:00.788 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:50:00.789 DEBUG:boto:Token: None 2022-09-03T06:50:00.790 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:00 GMT /dsgvcy-75/ 2022-09-03T06:50:00.791 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DBWCC0myftCTqheYq0+YWodJm0E= 2022-09-03T06:50:00.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DBWCC0myftCTqheYq0+YWodJm0E=', 'Content-Length': '0'} 2022-09-03T06:50:00.798 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '35'), ('X-RGW-Bytes-Used', '105'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000132342e893723d1-006312f918-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:50:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:00.799 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:50:00.801 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:00.801 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:00.802 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:00.803 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:00.804 DEBUG:boto:Method: GET 2022-09-03T06:50:00.806 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:00.807 DEBUG:boto:Data: 2022-09-03T06:50:00.808 DEBUG:boto:Headers: {} 2022-09-03T06:50:00.809 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:00.810 DEBUG:boto:Port: 8000 2022-09-03T06:50:00.811 DEBUG:boto:Params: {} 2022-09-03T06:50:00.812 DEBUG:boto:Token: None 2022-09-03T06:50:00.813 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:00 GMT /dsgvcy-75/?versions 2022-09-03T06:50:00.814 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5P4O1ObmUuOJrNq67vECLCYatHY= 2022-09-03T06:50:00.815 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5P4O1ObmUuOJrNq67vECLCYatHY=', 'Content-Length': '0'} 2022-09-03T06:50:00.821 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000012c8c3dbaa4244dd-006312f918-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:00.822 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj111nulltrue2022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj148nulltrue2022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj185nulltrue2022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj37nulltrue2022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj74nulltrue2022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:00.827 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:50:00.828 DEBUG:tasks.rgw_multi.zone_rados:o=obj111 2022-09-03T06:50:00.829 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:50:00.829 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2022-09-03T06:50:00.830 DEBUG:tasks.rgw_multi.zone_rados:o=obj124 2022-09-03T06:50:00.831 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2022-09-03T06:50:00.831 DEBUG:tasks.rgw_multi.zone_rados:o=obj148 2022-09-03T06:50:00.832 DEBUG:tasks.rgw_multi.zone_rados:o=obj155 2022-09-03T06:50:00.833 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:50:00.834 DEBUG:tasks.rgw_multi.zone_rados:o=obj185 2022-09-03T06:50:00.835 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:50:00.835 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:50:00.836 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:50:00.837 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2022-09-03T06:50:00.838 DEBUG:tasks.rgw_multi.zone_rados:o=obj31 2022-09-03T06:50:00.838 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:50:00.840 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:50:00.841 DEBUG:tasks.rgw_multi.zone_rados:o=obj37 2022-09-03T06:50:00.841 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:50:00.842 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:50:00.843 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:50:00.844 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:50:00.845 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:50:00.846 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:50:00.847 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2022-09-03T06:50:00.847 DEBUG:tasks.rgw_multi.zone_rados:o=obj62 2022-09-03T06:50:00.848 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:50:00.849 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:50:00.850 DEBUG:tasks.rgw_multi.zone_rados:o=obj74 2022-09-03T06:50:00.851 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:50:00.852 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:50:00.853 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2022-09-03T06:50:00.853 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:50:00.854 DEBUG:tasks.rgw_multi.zone_rados:o=obj93 2022-09-03T06:50:00.855 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:50:00.856 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:50:00.857 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:00.858 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:00.859 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:00.860 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:00.860 DEBUG:boto:Method: GET 2022-09-03T06:50:00.861 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:00.862 DEBUG:boto:Data: 2022-09-03T06:50:00.863 DEBUG:boto:Headers: {} 2022-09-03T06:50:00.864 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:00.865 DEBUG:boto:Port: 8000 2022-09-03T06:50:00.866 DEBUG:boto:Params: {} 2022-09-03T06:50:00.867 DEBUG:boto:Token: None 2022-09-03T06:50:00.869 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:00 GMT /dsgvcy-75/?versions 2022-09-03T06:50:00.870 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5P4O1ObmUuOJrNq67vECLCYatHY= 2022-09-03T06:50:00.871 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5P4O1ObmUuOJrNq67vECLCYatHY=', 'Content-Length': '0'} 2022-09-03T06:50:00.876 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f3051cebdd4aeba1-006312f918-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:00.877 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj111nulltrue2022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj148nulltrue2022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj185nulltrue2022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj37nulltrue2022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj74nulltrue2022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:00.882 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:50:00.884 DEBUG:tasks.rgw_multi.zone_rados:o=obj111 2022-09-03T06:50:00.884 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:50:00.885 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2022-09-03T06:50:00.885 DEBUG:tasks.rgw_multi.zone_rados:o=obj124 2022-09-03T06:50:00.886 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2022-09-03T06:50:00.886 DEBUG:tasks.rgw_multi.zone_rados:o=obj148 2022-09-03T06:50:00.887 DEBUG:tasks.rgw_multi.zone_rados:o=obj155 2022-09-03T06:50:00.887 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:50:00.888 DEBUG:tasks.rgw_multi.zone_rados:o=obj185 2022-09-03T06:50:00.888 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:50:00.889 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:50:00.889 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:50:00.890 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2022-09-03T06:50:00.891 DEBUG:tasks.rgw_multi.zone_rados:o=obj31 2022-09-03T06:50:00.892 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:50:00.892 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:50:00.893 DEBUG:tasks.rgw_multi.zone_rados:o=obj37 2022-09-03T06:50:00.893 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:50:00.894 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:50:00.894 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:50:00.895 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:50:00.896 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:50:00.896 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:50:00.897 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2022-09-03T06:50:00.897 DEBUG:tasks.rgw_multi.zone_rados:o=obj62 2022-09-03T06:50:00.898 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:50:00.898 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:50:00.899 DEBUG:tasks.rgw_multi.zone_rados:o=obj74 2022-09-03T06:50:00.900 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:50:00.900 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:50:00.901 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2022-09-03T06:50:00.901 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:50:00.902 DEBUG:tasks.rgw_multi.zone_rados:o=obj93 2022-09-03T06:50:00.902 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:50:00.903 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:00.903 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:00.904 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:00.905 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:00.905 DEBUG:boto:Method: GET 2022-09-03T06:50:00.906 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:00.906 DEBUG:boto:Data: 2022-09-03T06:50:00.907 DEBUG:boto:Headers: {} 2022-09-03T06:50:00.908 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:00.908 DEBUG:boto:Port: 8000 2022-09-03T06:50:00.909 DEBUG:boto:Params: {} 2022-09-03T06:50:00.909 DEBUG:boto:Token: None 2022-09-03T06:50:00.910 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:00 GMT /dsgvcy-75/?versions 2022-09-03T06:50:00.911 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5P4O1ObmUuOJrNq67vECLCYatHY= 2022-09-03T06:50:00.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5P4O1ObmUuOJrNq67vECLCYatHY=', 'Content-Length': '0'} 2022-09-03T06:50:00.914 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d397615e2098a8b2-006312f918-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:00.915 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj111nulltrue2022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj148nulltrue2022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj185nulltrue2022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj37nulltrue2022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj74nulltrue2022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:00.919 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:00.920 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:00.921 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:00.922 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:00.923 DEBUG:boto:Method: GET 2022-09-03T06:50:00.924 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:00.925 DEBUG:boto:Data: 2022-09-03T06:50:00.926 DEBUG:boto:Headers: {} 2022-09-03T06:50:00.927 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:00.928 DEBUG:boto:Port: 8000 2022-09-03T06:50:00.928 DEBUG:boto:Params: {} 2022-09-03T06:50:00.930 DEBUG:boto:Token: None 2022-09-03T06:50:00.931 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:00 GMT /dsgvcy-75/?versions 2022-09-03T06:50:00.932 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5P4O1ObmUuOJrNq67vECLCYatHY= 2022-09-03T06:50:00.932 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5P4O1ObmUuOJrNq67vECLCYatHY=', 'Content-Length': '0'} 2022-09-03T06:50:00.937 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000097a9c584b7288e8e-006312f918-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:00.938 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj111nulltrue2022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj148nulltrue2022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj185nulltrue2022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj37nulltrue2022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj74nulltrue2022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:00.944 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:50:00.945 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:00.946 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:00.947 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:00.948 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:00.949 DEBUG:boto:Method: GET 2022-09-03T06:50:00.950 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:00.951 DEBUG:boto:Data: 2022-09-03T06:50:00.952 DEBUG:boto:Headers: {} 2022-09-03T06:50:00.953 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:00.954 DEBUG:boto:Port: 8000 2022-09-03T06:50:00.955 DEBUG:boto:Params: {} 2022-09-03T06:50:00.957 DEBUG:boto:Token: None 2022-09-03T06:50:00.958 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:00 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:00.959 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sMjbLjEgOAD2fNeX7pBx1K8yJmQ= 2022-09-03T06:50:00.960 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sMjbLjEgOAD2fNeX7pBx1K8yJmQ=', 'Content-Length': '0'} 2022-09-03T06:50:00.963 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b7e62b74f0c838a4-006312f918-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:00.964 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:00.964 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:00.965 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:00.965 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:00.966 DEBUG:boto:Method: GET 2022-09-03T06:50:00.967 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:00.967 DEBUG:boto:Data: 2022-09-03T06:50:00.968 DEBUG:boto:Headers: {} 2022-09-03T06:50:00.968 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:00.969 DEBUG:boto:Port: 8000 2022-09-03T06:50:00.969 DEBUG:boto:Params: {} 2022-09-03T06:50:00.970 DEBUG:boto:Token: None 2022-09-03T06:50:00.971 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:00 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:00.971 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sMjbLjEgOAD2fNeX7pBx1K8yJmQ= 2022-09-03T06:50:00.972 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sMjbLjEgOAD2fNeX7pBx1K8yJmQ=', 'Content-Length': '0'} 2022-09-03T06:50:00.979 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000886620c4b04c5dc0-006312f918-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:00 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:00.987 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:00.988 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:00.989 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:00.989 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:00.990 DEBUG:boto:Method: HEAD 2022-09-03T06:50:00.990 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:00.991 DEBUG:boto:Data: 2022-09-03T06:50:00.991 DEBUG:boto:Headers: {} 2022-09-03T06:50:00.992 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:00.992 DEBUG:boto:Port: 8000 2022-09-03T06:50:00.993 DEBUG:boto:Params: {} 2022-09-03T06:50:00.993 DEBUG:boto:Token: None 2022-09-03T06:50:00.996 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:00 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:01.003 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YJBV4MSdkpr20fsUF93hY5FdxyU= 2022-09-03T06:50:01.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:00 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YJBV4MSdkpr20fsUF93hY5FdxyU=', 'Content-Length': '0'} 2022-09-03T06:50:01.007 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fa8be222812ad89a-006312f919-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.009 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:01.010 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:01.011 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:01.012 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:01.013 DEBUG:boto:Method: HEAD 2022-09-03T06:50:01.013 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:01.014 DEBUG:boto:Data: 2022-09-03T06:50:01.015 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.016 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:01.017 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.018 DEBUG:boto:Params: {} 2022-09-03T06:50:01.019 DEBUG:boto:Token: None 2022-09-03T06:50:01.020 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:01.022 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bjyPIGxdy9QlC3y6vmhGODQ9Rok= 2022-09-03T06:50:01.023 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bjyPIGxdy9QlC3y6vmhGODQ9Rok=', 'Content-Length': '0'} 2022-09-03T06:50:01.030 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000073091c4f6a209bb8-006312f919-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.032 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:50:01.033 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:01.034 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:01.035 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:01.036 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:01.037 DEBUG:boto:Method: GET 2022-09-03T06:50:01.038 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:01.039 DEBUG:boto:Data: 2022-09-03T06:50:01.040 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.042 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:01.043 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.044 DEBUG:boto:Params: {} 2022-09-03T06:50:01.045 DEBUG:boto:Token: None 2022-09-03T06:50:01.046 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:01.048 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rBhHAmDomYeag3/DkLuN8C+ZsrI= 2022-09-03T06:50:01.050 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rBhHAmDomYeag3/DkLuN8C+ZsrI=', 'Content-Length': '0'} 2022-09-03T06:50:01.060 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000028dcf16b5f6878c9-006312f919-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.064 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:01.066 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:01.067 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:01.068 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:01.069 DEBUG:boto:Method: GET 2022-09-03T06:50:01.071 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:01.072 DEBUG:boto:Data: 2022-09-03T06:50:01.073 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.074 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:01.075 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.076 DEBUG:boto:Params: {} 2022-09-03T06:50:01.077 DEBUG:boto:Token: None 2022-09-03T06:50:01.079 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:01.080 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rBhHAmDomYeag3/DkLuN8C+ZsrI= 2022-09-03T06:50:01.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rBhHAmDomYeag3/DkLuN8C+ZsrI=', 'Content-Length': '0'} 2022-09-03T06:50:01.091 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002fc08001cf26badb-006312f919-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.093 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:01.094 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:01.095 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:01.096 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:01.097 DEBUG:boto:Method: GET 2022-09-03T06:50:01.098 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:50:01.099 DEBUG:boto:Data: 2022-09-03T06:50:01.100 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.101 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:01.102 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.103 DEBUG:boto:Params: {} 2022-09-03T06:50:01.105 DEBUG:boto:Token: None 2022-09-03T06:50:01.106 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/ 2022-09-03T06:50:01.107 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto= 2022-09-03T06:50:01.108 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto=', 'Content-Length': '0'} 2022-09-03T06:50:01.111 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001358a1b8f260bff8-006312f919-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.112 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:01.113 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:01.113 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:01.114 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:01.115 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:01.116 DEBUG:boto:Method: GET 2022-09-03T06:50:01.117 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:50:01.118 DEBUG:boto:Data: 2022-09-03T06:50:01.119 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.120 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:01.121 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.122 DEBUG:boto:Params: {} 2022-09-03T06:50:01.123 DEBUG:boto:Token: None 2022-09-03T06:50:01.124 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/ 2022-09-03T06:50:01.125 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto= 2022-09-03T06:50:01.127 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto=', 'Content-Length': '0'} 2022-09-03T06:50:01.131 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b9a67aecdb099b20-006312f919-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.132 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:01.133 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:01.134 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:01.135 DEBUG:boto:Method: HEAD 2022-09-03T06:50:01.136 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:50:01.136 DEBUG:boto:Data: 2022-09-03T06:50:01.137 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.137 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:01.138 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.139 DEBUG:boto:Params: {} 2022-09-03T06:50:01.139 DEBUG:boto:Token: None 2022-09-03T06:50:01.140 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj11 2022-09-03T06:50:01.141 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MKkYHdRRf2tsrTloMu6qz8dr4+g= 2022-09-03T06:50:01.142 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MKkYHdRRf2tsrTloMu6qz8dr4+g=', 'Content-Length': '0'} 2022-09-03T06:50:01.144 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000028c5afd1721db42d-006312f919-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.145 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:01.145 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:01.146 DEBUG:boto:Method: HEAD 2022-09-03T06:50:01.146 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:50:01.147 DEBUG:boto:Data: 2022-09-03T06:50:01.147 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.148 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:01.149 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.149 DEBUG:boto:Params: {} 2022-09-03T06:50:01.150 DEBUG:boto:Token: None 2022-09-03T06:50:01.150 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj11 2022-09-03T06:50:01.151 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MKkYHdRRf2tsrTloMu6qz8dr4+g= 2022-09-03T06:50:01.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MKkYHdRRf2tsrTloMu6qz8dr4+g=', 'Content-Length': '0'} 2022-09-03T06:50:01.154 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d8a754144067753-006312f919-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.155 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:50:01.155 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:01.156 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:01.158 DEBUG:boto:Method: GET 2022-09-03T06:50:01.158 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:50:01.159 DEBUG:boto:Data: 2022-09-03T06:50:01.159 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.160 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:01.161 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.161 DEBUG:boto:Params: {} 2022-09-03T06:50:01.162 DEBUG:boto:Token: None 2022-09-03T06:50:01.163 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj11 2022-09-03T06:50:01.164 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CO67LGHyBE8XlmZM0vM40dw94Gk= 2022-09-03T06:50:01.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CO67LGHyBE8XlmZM0vM40dw94Gk=', 'Content-Length': '0'} 2022-09-03T06:50:01.166 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000088a96c17334b5b5e-006312f919-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.175 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:01.176 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:01.177 DEBUG:boto:Method: GET 2022-09-03T06:50:01.178 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:50:01.178 DEBUG:boto:Data: 2022-09-03T06:50:01.180 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.181 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:01.181 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.182 DEBUG:boto:Params: {} 2022-09-03T06:50:01.183 DEBUG:boto:Token: None 2022-09-03T06:50:01.183 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj11 2022-09-03T06:50:01.184 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CO67LGHyBE8XlmZM0vM40dw94Gk= 2022-09-03T06:50:01.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CO67LGHyBE8XlmZM0vM40dw94Gk=', 'Content-Length': '0'} 2022-09-03T06:50:01.187 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000012c5c8dfa2f6c402-006312f919-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.234 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:01.236 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:01.236 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:01.237 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:01.237 DEBUG:boto:Method: GET 2022-09-03T06:50:01.238 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:50:01.238 DEBUG:boto:Data: 2022-09-03T06:50:01.239 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.240 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:01.240 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.242 DEBUG:boto:Params: {} 2022-09-03T06:50:01.243 DEBUG:boto:Token: None 2022-09-03T06:50:01.243 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/ 2022-09-03T06:50:01.244 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto= 2022-09-03T06:50:01.245 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto=', 'Content-Length': '0'} 2022-09-03T06:50:01.248 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c1052e7672492695-006312f919-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.249 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:01.250 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:01.251 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:01.252 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:01.252 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:01.253 DEBUG:boto:Method: GET 2022-09-03T06:50:01.253 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:50:01.254 DEBUG:boto:Data: 2022-09-03T06:50:01.255 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.255 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:01.256 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.257 DEBUG:boto:Params: {} 2022-09-03T06:50:01.266 DEBUG:boto:Token: None 2022-09-03T06:50:01.267 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/ 2022-09-03T06:50:01.268 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto= 2022-09-03T06:50:01.269 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto=', 'Content-Length': '0'} 2022-09-03T06:50:01.273 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e700d09430fbc5dd-006312f919-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.275 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:01.277 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj111 2022-09-03T06:50:01.278 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:01.279 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:01.280 DEBUG:boto:path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.280 DEBUG:boto:auth_path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.281 DEBUG:boto:Method: GET 2022-09-03T06:50:01.283 DEBUG:boto:Path: /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.284 DEBUG:boto:Data: 2022-09-03T06:50:01.284 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.285 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:01.286 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.287 DEBUG:boto:Params: {} 2022-09-03T06:50:01.288 DEBUG:boto:Token: None 2022-09-03T06:50:01.289 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.290 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dJ49n3MHkwN6LBDVBt613Tw9JOA= 2022-09-03T06:50:01.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dJ49n3MHkwN6LBDVBt613Tw9JOA=', 'Content-Length': '0'} 2022-09-03T06:50:01.295 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b8baf79412bf1774-006312f919-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.308 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:01.310 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:01.311 DEBUG:boto:path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.312 DEBUG:boto:auth_path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.313 DEBUG:boto:Method: GET 2022-09-03T06:50:01.314 DEBUG:boto:Path: /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.314 DEBUG:boto:Data: 2022-09-03T06:50:01.315 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.315 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:01.316 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.317 DEBUG:boto:Params: {} 2022-09-03T06:50:01.317 DEBUG:boto:Token: None 2022-09-03T06:50:01.318 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.319 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dJ49n3MHkwN6LBDVBt613Tw9JOA= 2022-09-03T06:50:01.319 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dJ49n3MHkwN6LBDVBt613Tw9JOA=', 'Content-Length': '0'} 2022-09-03T06:50:01.322 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee6e2e904db12df9-006312f919-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.324 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:01.325 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:01.326 DEBUG:boto:path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.327 DEBUG:boto:auth_path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.327 DEBUG:boto:Method: HEAD 2022-09-03T06:50:01.328 DEBUG:boto:Path: /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.328 DEBUG:boto:Data: 2022-09-03T06:50:01.330 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.331 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:01.332 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.334 DEBUG:boto:Params: {} 2022-09-03T06:50:01.335 DEBUG:boto:Token: None 2022-09-03T06:50:01.337 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.338 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ncgYDn28cPkceXwuarlsmfhO7Xc= 2022-09-03T06:50:01.339 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ncgYDn28cPkceXwuarlsmfhO7Xc=', 'Content-Length': '0'} 2022-09-03T06:50:01.353 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001af793c4fc2ee341-006312f919-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.354 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:01.355 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:01.356 DEBUG:boto:path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.357 DEBUG:boto:auth_path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.358 DEBUG:boto:Method: HEAD 2022-09-03T06:50:01.359 DEBUG:boto:Path: /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.360 DEBUG:boto:Data: 2022-09-03T06:50:01.361 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.362 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:01.363 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.364 DEBUG:boto:Params: {} 2022-09-03T06:50:01.365 DEBUG:boto:Token: None 2022-09-03T06:50:01.366 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.367 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ncgYDn28cPkceXwuarlsmfhO7Xc= 2022-09-03T06:50:01.368 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ncgYDn28cPkceXwuarlsmfhO7Xc=', 'Content-Length': '0'} 2022-09-03T06:50:01.371 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009bb65e93d51f7e0b-006312f919-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.372 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj111 2022-09-03T06:50:01.373 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:01.375 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:01.376 DEBUG:boto:path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.376 DEBUG:boto:auth_path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.377 DEBUG:boto:Method: GET 2022-09-03T06:50:01.378 DEBUG:boto:Path: /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.379 DEBUG:boto:Data: 2022-09-03T06:50:01.380 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.381 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:01.384 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.386 DEBUG:boto:Params: {} 2022-09-03T06:50:01.387 DEBUG:boto:Token: None 2022-09-03T06:50:01.388 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.389 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dJ49n3MHkwN6LBDVBt613Tw9JOA= 2022-09-03T06:50:01.391 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dJ49n3MHkwN6LBDVBt613Tw9JOA=', 'Content-Length': '0'} 2022-09-03T06:50:01.394 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000030a8a99c5f8cd1f2-006312f919-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.396 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:01.397 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:01.398 DEBUG:boto:path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.399 DEBUG:boto:auth_path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.400 DEBUG:boto:Method: GET 2022-09-03T06:50:01.401 DEBUG:boto:Path: /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.402 DEBUG:boto:Data: 2022-09-03T06:50:01.403 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.404 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:01.405 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.406 DEBUG:boto:Params: {} 2022-09-03T06:50:01.406 DEBUG:boto:Token: None 2022-09-03T06:50:01.407 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:01.408 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dJ49n3MHkwN6LBDVBt613Tw9JOA= 2022-09-03T06:50:01.409 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dJ49n3MHkwN6LBDVBt613Tw9JOA=', 'Content-Length': '0'} 2022-09-03T06:50:01.411 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ff695f0fc01e82bf-006312f919-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.412 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:01.413 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:01.413 DEBUG:boto:path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:01.414 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:01.415 DEBUG:boto:Method: GET 2022-09-03T06:50:01.415 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj111 2022-09-03T06:50:01.416 DEBUG:boto:Data: 2022-09-03T06:50:01.417 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.417 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:01.418 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.418 DEBUG:boto:Params: {} 2022-09-03T06:50:01.419 DEBUG:boto:Token: None 2022-09-03T06:50:01.420 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/ 2022-09-03T06:50:01.421 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto= 2022-09-03T06:50:01.421 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto=', 'Content-Length': '0'} 2022-09-03T06:50:01.424 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d853fb3d01da4504-006312f919-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.425 DEBUG:boto:b'dsgvcy-75obj1111000falseobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:01.426 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:01.427 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:01.427 DEBUG:boto:path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:01.428 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:01.428 DEBUG:boto:Method: GET 2022-09-03T06:50:01.429 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj111 2022-09-03T06:50:01.429 DEBUG:boto:Data: 2022-09-03T06:50:01.430 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.431 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:01.431 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.432 DEBUG:boto:Params: {} 2022-09-03T06:50:01.433 DEBUG:boto:Token: None 2022-09-03T06:50:01.433 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/ 2022-09-03T06:50:01.434 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto= 2022-09-03T06:50:01.435 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto=', 'Content-Length': '0'} 2022-09-03T06:50:01.439 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b4a486ad687c7d22-006312f919-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.486 DEBUG:boto:b'dsgvcy-75obj1111000falseobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:01.489 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:01.490 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:01.490 DEBUG:boto:Method: HEAD 2022-09-03T06:50:01.491 DEBUG:boto:Path: /dsgvcy-75/obj111 2022-09-03T06:50:01.492 DEBUG:boto:Data: 2022-09-03T06:50:01.492 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.493 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:01.494 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.495 DEBUG:boto:Params: {} 2022-09-03T06:50:01.496 DEBUG:boto:Token: None 2022-09-03T06:50:01.497 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj111 2022-09-03T06:50:01.498 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xpyFSOf2c39Toe6Kz73evdmgJ2Q= 2022-09-03T06:50:01.500 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xpyFSOf2c39Toe6Kz73evdmgJ2Q=', 'Content-Length': '0'} 2022-09-03T06:50:01.503 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007f7068005bb81c9b-006312f919-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.504 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:01.505 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:01.505 DEBUG:boto:Method: HEAD 2022-09-03T06:50:01.506 DEBUG:boto:Path: /dsgvcy-75/obj111 2022-09-03T06:50:01.507 DEBUG:boto:Data: 2022-09-03T06:50:01.508 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.509 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:01.510 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.511 DEBUG:boto:Params: {} 2022-09-03T06:50:01.511 DEBUG:boto:Token: None 2022-09-03T06:50:01.512 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj111 2022-09-03T06:50:01.513 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xpyFSOf2c39Toe6Kz73evdmgJ2Q= 2022-09-03T06:50:01.515 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xpyFSOf2c39Toe6Kz73evdmgJ2Q=', 'Content-Length': '0'} 2022-09-03T06:50:01.519 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cd81bdf44bb86976-006312f919-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.520 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj111 2022-09-03T06:50:01.521 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:01.521 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:01.522 DEBUG:boto:Method: GET 2022-09-03T06:50:01.523 DEBUG:boto:Path: /dsgvcy-75/obj111 2022-09-03T06:50:01.524 DEBUG:boto:Data: 2022-09-03T06:50:01.525 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.525 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:01.526 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.527 DEBUG:boto:Params: {} 2022-09-03T06:50:01.528 DEBUG:boto:Token: None 2022-09-03T06:50:01.528 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj111 2022-09-03T06:50:01.530 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ligkElSdqlHsoe3xnHar1sSfPs0= 2022-09-03T06:50:01.531 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ligkElSdqlHsoe3xnHar1sSfPs0=', 'Content-Length': '0'} 2022-09-03T06:50:01.534 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000022cd09d5fb4e00da-006312f919-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.574 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:01.575 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:01.576 DEBUG:boto:Method: GET 2022-09-03T06:50:01.577 DEBUG:boto:Path: /dsgvcy-75/obj111 2022-09-03T06:50:01.577 DEBUG:boto:Data: 2022-09-03T06:50:01.578 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.579 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:01.579 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.580 DEBUG:boto:Params: {} 2022-09-03T06:50:01.581 DEBUG:boto:Token: None 2022-09-03T06:50:01.581 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj111 2022-09-03T06:50:01.582 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ligkElSdqlHsoe3xnHar1sSfPs0= 2022-09-03T06:50:01.583 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ligkElSdqlHsoe3xnHar1sSfPs0=', 'Content-Length': '0'} 2022-09-03T06:50:01.585 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a058cf98cbe9bd3e-006312f919-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.586 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:01.587 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:01.588 DEBUG:boto:path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:01.588 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:01.589 DEBUG:boto:Method: GET 2022-09-03T06:50:01.590 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj111 2022-09-03T06:50:01.590 DEBUG:boto:Data: 2022-09-03T06:50:01.591 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.592 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:01.592 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.593 DEBUG:boto:Params: {} 2022-09-03T06:50:01.594 DEBUG:boto:Token: None 2022-09-03T06:50:01.595 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/ 2022-09-03T06:50:01.595 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto= 2022-09-03T06:50:01.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto=', 'Content-Length': '0'} 2022-09-03T06:50:01.599 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008ffb138a9995e4c3-006312f919-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.600 DEBUG:boto:b'dsgvcy-75obj1111000falseobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:01.600 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:01.601 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:01.602 DEBUG:boto:path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:01.602 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:01.603 DEBUG:boto:Method: GET 2022-09-03T06:50:01.603 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj111 2022-09-03T06:50:01.604 DEBUG:boto:Data: 2022-09-03T06:50:01.604 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.605 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:01.606 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.606 DEBUG:boto:Params: {} 2022-09-03T06:50:01.607 DEBUG:boto:Token: None 2022-09-03T06:50:01.607 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/ 2022-09-03T06:50:01.608 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto= 2022-09-03T06:50:01.609 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto=', 'Content-Length': '0'} 2022-09-03T06:50:01.612 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f23724733b9b6e79-006312f919-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.613 DEBUG:boto:b'dsgvcy-75obj1111000falseobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:01.614 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:50:01.615 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:01.615 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:01.616 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.617 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.617 DEBUG:boto:Method: GET 2022-09-03T06:50:01.618 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.618 DEBUG:boto:Data: 2022-09-03T06:50:01.619 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.619 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:01.620 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.621 DEBUG:boto:Params: {} 2022-09-03T06:50:01.621 DEBUG:boto:Token: None 2022-09-03T06:50:01.622 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.622 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Uv8RTQ9aj7xuMf0TotPJKD0EnY4= 2022-09-03T06:50:01.623 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Uv8RTQ9aj7xuMf0TotPJKD0EnY4=', 'Content-Length': '0'} 2022-09-03T06:50:01.625 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006de457c49ceca543-006312f919-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.626 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:01.627 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:01.627 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.628 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.629 DEBUG:boto:Method: GET 2022-09-03T06:50:01.629 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.630 DEBUG:boto:Data: 2022-09-03T06:50:01.630 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.631 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:01.631 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.632 DEBUG:boto:Params: {} 2022-09-03T06:50:01.633 DEBUG:boto:Token: None 2022-09-03T06:50:01.633 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.634 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Uv8RTQ9aj7xuMf0TotPJKD0EnY4= 2022-09-03T06:50:01.635 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Uv8RTQ9aj7xuMf0TotPJKD0EnY4=', 'Content-Length': '0'} 2022-09-03T06:50:01.637 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b896c0eb004ecf75-006312f919-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.638 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:01.638 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:01.639 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.639 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.640 DEBUG:boto:Method: HEAD 2022-09-03T06:50:01.641 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.641 DEBUG:boto:Data: 2022-09-03T06:50:01.642 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.642 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:01.643 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.644 DEBUG:boto:Params: {} 2022-09-03T06:50:01.644 DEBUG:boto:Token: None 2022-09-03T06:50:01.645 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.645 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ibuCc+2slLsJ9hW8UYIBUzEtjvk= 2022-09-03T06:50:01.646 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ibuCc+2slLsJ9hW8UYIBUzEtjvk=', 'Content-Length': '0'} 2022-09-03T06:50:01.648 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000989b0061ee02aa67-006312f919-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.649 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:01.649 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:01.650 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.650 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.651 DEBUG:boto:Method: HEAD 2022-09-03T06:50:01.651 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.652 DEBUG:boto:Data: 2022-09-03T06:50:01.652 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.653 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:01.654 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.654 DEBUG:boto:Params: {} 2022-09-03T06:50:01.655 DEBUG:boto:Token: None 2022-09-03T06:50:01.655 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.656 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ibuCc+2slLsJ9hW8UYIBUzEtjvk= 2022-09-03T06:50:01.657 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ibuCc+2slLsJ9hW8UYIBUzEtjvk=', 'Content-Length': '0'} 2022-09-03T06:50:01.659 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c027b30ddc20d8b-006312f919-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.660 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:50:01.660 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:01.661 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:01.662 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.662 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.663 DEBUG:boto:Method: GET 2022-09-03T06:50:01.663 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.664 DEBUG:boto:Data: 2022-09-03T06:50:01.664 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.665 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:01.666 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.666 DEBUG:boto:Params: {} 2022-09-03T06:50:01.667 DEBUG:boto:Token: None 2022-09-03T06:50:01.668 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.668 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Uv8RTQ9aj7xuMf0TotPJKD0EnY4= 2022-09-03T06:50:01.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Uv8RTQ9aj7xuMf0TotPJKD0EnY4=', 'Content-Length': '0'} 2022-09-03T06:50:01.671 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d205bd5825902b5a-006312f919-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.672 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:01.673 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:01.674 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.674 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.675 DEBUG:boto:Method: GET 2022-09-03T06:50:01.676 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.676 DEBUG:boto:Data: 2022-09-03T06:50:01.677 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.677 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:01.678 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.679 DEBUG:boto:Params: {} 2022-09-03T06:50:01.679 DEBUG:boto:Token: None 2022-09-03T06:50:01.680 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:01.681 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Uv8RTQ9aj7xuMf0TotPJKD0EnY4= 2022-09-03T06:50:01.681 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Uv8RTQ9aj7xuMf0TotPJKD0EnY4=', 'Content-Length': '0'} 2022-09-03T06:50:01.684 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000068ec45eb3c152acb-006312f919-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.726 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:01.727 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:01.728 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:01.730 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:01.731 DEBUG:boto:Method: GET 2022-09-03T06:50:01.732 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:50:01.733 DEBUG:boto:Data: 2022-09-03T06:50:01.734 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.735 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:01.736 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.737 DEBUG:boto:Params: {} 2022-09-03T06:50:01.738 DEBUG:boto:Token: None 2022-09-03T06:50:01.739 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/ 2022-09-03T06:50:01.740 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto= 2022-09-03T06:50:01.741 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto=', 'Content-Length': '0'} 2022-09-03T06:50:01.744 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000183b3e78d218547b-006312f919-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.745 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:01.746 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:01.747 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:01.747 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:01.748 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:01.748 DEBUG:boto:Method: GET 2022-09-03T06:50:01.749 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:50:01.750 DEBUG:boto:Data: 2022-09-03T06:50:01.751 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.752 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:01.753 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.754 DEBUG:boto:Params: {} 2022-09-03T06:50:01.755 DEBUG:boto:Token: None 2022-09-03T06:50:01.756 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/ 2022-09-03T06:50:01.757 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto= 2022-09-03T06:50:01.759 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto=', 'Content-Length': '0'} 2022-09-03T06:50:01.764 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003d83af3a2abc58ce-006312f919-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.765 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:01.767 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:01.768 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:01.768 DEBUG:boto:Method: HEAD 2022-09-03T06:50:01.769 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:50:01.770 DEBUG:boto:Data: 2022-09-03T06:50:01.771 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.771 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:01.772 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.773 DEBUG:boto:Params: {} 2022-09-03T06:50:01.774 DEBUG:boto:Token: None 2022-09-03T06:50:01.775 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj115 2022-09-03T06:50:01.776 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nhWYZflaqQeo7rlSqLqbd/cogZ4= 2022-09-03T06:50:01.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nhWYZflaqQeo7rlSqLqbd/cogZ4=', 'Content-Length': '0'} 2022-09-03T06:50:01.784 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007c9b54db8154a201-006312f919-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.786 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:01.787 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:01.788 DEBUG:boto:Method: HEAD 2022-09-03T06:50:01.788 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:50:01.789 DEBUG:boto:Data: 2022-09-03T06:50:01.790 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.791 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:01.792 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.793 DEBUG:boto:Params: {} 2022-09-03T06:50:01.793 DEBUG:boto:Token: None 2022-09-03T06:50:01.794 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj115 2022-09-03T06:50:01.795 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nhWYZflaqQeo7rlSqLqbd/cogZ4= 2022-09-03T06:50:01.795 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nhWYZflaqQeo7rlSqLqbd/cogZ4=', 'Content-Length': '0'} 2022-09-03T06:50:01.798 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d769cf5369d6c86-006312f919-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.801 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:50:01.802 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:01.803 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:01.804 DEBUG:boto:Method: GET 2022-09-03T06:50:01.805 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:50:01.806 DEBUG:boto:Data: 2022-09-03T06:50:01.807 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.808 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:01.810 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.811 DEBUG:boto:Params: {} 2022-09-03T06:50:01.812 DEBUG:boto:Token: None 2022-09-03T06:50:01.813 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj115 2022-09-03T06:50:01.814 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bMm+UVCVn/6GWv9unYZGsdOG47c= 2022-09-03T06:50:01.815 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bMm+UVCVn/6GWv9unYZGsdOG47c=', 'Content-Length': '0'} 2022-09-03T06:50:01.821 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000089451cab8ba8646b-006312f919-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.870 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:01.871 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:01.872 DEBUG:boto:Method: GET 2022-09-03T06:50:01.873 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:50:01.874 DEBUG:boto:Data: 2022-09-03T06:50:01.875 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.875 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:01.876 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.877 DEBUG:boto:Params: {} 2022-09-03T06:50:01.878 DEBUG:boto:Token: None 2022-09-03T06:50:01.879 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj115 2022-09-03T06:50:01.880 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bMm+UVCVn/6GWv9unYZGsdOG47c= 2022-09-03T06:50:01.881 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bMm+UVCVn/6GWv9unYZGsdOG47c=', 'Content-Length': '0'} 2022-09-03T06:50:01.884 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b5916ee0e7c9f778-006312f919-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.885 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:01.885 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:01.886 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:01.887 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:01.887 DEBUG:boto:Method: GET 2022-09-03T06:50:01.888 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:50:01.889 DEBUG:boto:Data: 2022-09-03T06:50:01.889 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.890 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:01.891 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.891 DEBUG:boto:Params: {} 2022-09-03T06:50:01.892 DEBUG:boto:Token: None 2022-09-03T06:50:01.893 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/ 2022-09-03T06:50:01.894 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto= 2022-09-03T06:50:01.895 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto=', 'Content-Length': '0'} 2022-09-03T06:50:01.898 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005bb244c43ab9d2f4-006312f919-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.900 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:01.901 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:01.902 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:01.904 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:01.905 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:01.906 DEBUG:boto:Method: GET 2022-09-03T06:50:01.907 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:50:01.908 DEBUG:boto:Data: 2022-09-03T06:50:01.908 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.909 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:01.910 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.911 DEBUG:boto:Params: {} 2022-09-03T06:50:01.912 DEBUG:boto:Token: None 2022-09-03T06:50:01.913 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/ 2022-09-03T06:50:01.914 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto= 2022-09-03T06:50:01.915 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TBuo/6mt4d/VVwnIKRLIdV2Omto=', 'Content-Length': '0'} 2022-09-03T06:50:01.920 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002fe72f4026021cee-006312f919-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.921 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:01.923 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:50:01.924 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:01.925 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:01.927 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:01.928 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:01.929 DEBUG:boto:Method: GET 2022-09-03T06:50:01.930 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:01.931 DEBUG:boto:Data: 2022-09-03T06:50:01.932 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.933 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:01.934 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.935 DEBUG:boto:Params: {} 2022-09-03T06:50:01.936 DEBUG:boto:Token: None 2022-09-03T06:50:01.938 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:01.938 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kO2hLhWN4HKf5/1FLOg9u35RqVU= 2022-09-03T06:50:01.939 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kO2hLhWN4HKf5/1FLOg9u35RqVU=', 'Content-Length': '0'} 2022-09-03T06:50:01.941 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000006b2a2e758ff944a-006312f919-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.942 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:01.943 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:01.943 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:01.944 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:01.944 DEBUG:boto:Method: GET 2022-09-03T06:50:01.945 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:01.946 DEBUG:boto:Data: 2022-09-03T06:50:01.946 DEBUG:boto:Headers: {} 2022-09-03T06:50:01.947 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:01.948 DEBUG:boto:Port: 8000 2022-09-03T06:50:01.949 DEBUG:boto:Params: {} 2022-09-03T06:50:01.950 DEBUG:boto:Token: None 2022-09-03T06:50:01.950 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:01 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:01.951 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kO2hLhWN4HKf5/1FLOg9u35RqVU= 2022-09-03T06:50:01.952 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:01 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kO2hLhWN4HKf5/1FLOg9u35RqVU=', 'Content-Length': '0'} 2022-09-03T06:50:01.955 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000030787b4fa9ee087e-006312f919-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:01 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:01.998 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:02.000 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:02.000 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:02.001 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:02.002 DEBUG:boto:Method: HEAD 2022-09-03T06:50:02.003 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:02.003 DEBUG:boto:Data: 2022-09-03T06:50:02.004 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.005 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:02.006 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.008 DEBUG:boto:Params: {} 2022-09-03T06:50:02.009 DEBUG:boto:Token: None 2022-09-03T06:50:02.010 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:02.011 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZMb4Rl3b7crm9gJ3ObRbcdT9TKc= 2022-09-03T06:50:02.012 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZMb4Rl3b7crm9gJ3ObRbcdT9TKc=', 'Content-Length': '0'} 2022-09-03T06:50:02.015 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000197f297270f7ee66-006312f91a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.017 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:02.018 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:02.019 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:02.020 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:02.021 DEBUG:boto:Method: HEAD 2022-09-03T06:50:02.022 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:02.022 DEBUG:boto:Data: 2022-09-03T06:50:02.023 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.024 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:02.025 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.026 DEBUG:boto:Params: {} 2022-09-03T06:50:02.027 DEBUG:boto:Token: None 2022-09-03T06:50:02.028 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:02.029 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZMb4Rl3b7crm9gJ3ObRbcdT9TKc= 2022-09-03T06:50:02.030 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZMb4Rl3b7crm9gJ3ObRbcdT9TKc=', 'Content-Length': '0'} 2022-09-03T06:50:02.033 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e5b42567955de7a-006312f91a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.034 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:50:02.034 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:02.035 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:02.035 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:02.036 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:02.036 DEBUG:boto:Method: GET 2022-09-03T06:50:02.037 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:02.037 DEBUG:boto:Data: 2022-09-03T06:50:02.038 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.039 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:02.039 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.040 DEBUG:boto:Params: {} 2022-09-03T06:50:02.040 DEBUG:boto:Token: None 2022-09-03T06:50:02.041 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:02.042 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XvXbHp2PbrsJIIyrU8n0SqXJncM= 2022-09-03T06:50:02.042 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XvXbHp2PbrsJIIyrU8n0SqXJncM=', 'Content-Length': '0'} 2022-09-03T06:50:02.044 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad1a9ce6a7fe0ce0-006312f91a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.086 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:02.089 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:02.091 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:02.092 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:02.094 DEBUG:boto:Method: GET 2022-09-03T06:50:02.095 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:02.096 DEBUG:boto:Data: 2022-09-03T06:50:02.097 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.098 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:02.099 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.100 DEBUG:boto:Params: {} 2022-09-03T06:50:02.101 DEBUG:boto:Token: None 2022-09-03T06:50:02.102 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:02.103 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XvXbHp2PbrsJIIyrU8n0SqXJncM= 2022-09-03T06:50:02.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XvXbHp2PbrsJIIyrU8n0SqXJncM=', 'Content-Length': '0'} 2022-09-03T06:50:02.106 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000236fb80ce97a40c1-006312f91a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.107 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:02.108 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:02.108 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:02.109 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:02.110 DEBUG:boto:Method: GET 2022-09-03T06:50:02.111 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:50:02.111 DEBUG:boto:Data: 2022-09-03T06:50:02.112 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.113 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:02.113 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.114 DEBUG:boto:Params: {} 2022-09-03T06:50:02.114 DEBUG:boto:Token: None 2022-09-03T06:50:02.115 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/ 2022-09-03T06:50:02.116 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI= 2022-09-03T06:50:02.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI=', 'Content-Length': '0'} 2022-09-03T06:50:02.119 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000044d57ca0247d3373-006312f91a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.120 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:02.121 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:02.122 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:02.122 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:02.123 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:02.124 DEBUG:boto:Method: GET 2022-09-03T06:50:02.124 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:50:02.125 DEBUG:boto:Data: 2022-09-03T06:50:02.125 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.126 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:02.127 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.128 DEBUG:boto:Params: {} 2022-09-03T06:50:02.129 DEBUG:boto:Token: None 2022-09-03T06:50:02.129 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/ 2022-09-03T06:50:02.130 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI= 2022-09-03T06:50:02.131 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI=', 'Content-Length': '0'} 2022-09-03T06:50:02.135 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005355afa9efaa2137-006312f91a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.136 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:02.137 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:02.137 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:02.138 DEBUG:boto:Method: HEAD 2022-09-03T06:50:02.138 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:50:02.139 DEBUG:boto:Data: 2022-09-03T06:50:02.139 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.140 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:02.140 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.141 DEBUG:boto:Params: {} 2022-09-03T06:50:02.141 DEBUG:boto:Token: None 2022-09-03T06:50:02.151 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj116 2022-09-03T06:50:02.152 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i3dtCcSA/Kxu6aZkRv14lCxM/TU= 2022-09-03T06:50:02.153 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i3dtCcSA/Kxu6aZkRv14lCxM/TU=', 'Content-Length': '0'} 2022-09-03T06:50:02.155 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000356d852b7cce7b2c-006312f91a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.156 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:02.157 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:02.157 DEBUG:boto:Method: HEAD 2022-09-03T06:50:02.158 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:50:02.158 DEBUG:boto:Data: 2022-09-03T06:50:02.159 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.159 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:02.160 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.160 DEBUG:boto:Params: {} 2022-09-03T06:50:02.161 DEBUG:boto:Token: None 2022-09-03T06:50:02.162 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj116 2022-09-03T06:50:02.162 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:i3dtCcSA/Kxu6aZkRv14lCxM/TU= 2022-09-03T06:50:02.163 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:i3dtCcSA/Kxu6aZkRv14lCxM/TU=', 'Content-Length': '0'} 2022-09-03T06:50:02.165 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ac081a3bdfc706fe-006312f91a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.166 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:50:02.167 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:02.167 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:02.168 DEBUG:boto:Method: GET 2022-09-03T06:50:02.169 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:50:02.169 DEBUG:boto:Data: 2022-09-03T06:50:02.170 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.170 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:02.171 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.172 DEBUG:boto:Params: {} 2022-09-03T06:50:02.172 DEBUG:boto:Token: None 2022-09-03T06:50:02.173 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj116 2022-09-03T06:50:02.174 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mjyaBH0pGCxIZPb9U/hXtFlfSGU= 2022-09-03T06:50:02.174 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mjyaBH0pGCxIZPb9U/hXtFlfSGU=', 'Content-Length': '0'} 2022-09-03T06:50:02.176 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000880a0a4076284f3a-006312f91a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.177 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:02.178 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:02.178 DEBUG:boto:Method: GET 2022-09-03T06:50:02.179 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:50:02.179 DEBUG:boto:Data: 2022-09-03T06:50:02.180 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.180 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:02.181 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.182 DEBUG:boto:Params: {} 2022-09-03T06:50:02.182 DEBUG:boto:Token: None 2022-09-03T06:50:02.183 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj116 2022-09-03T06:50:02.184 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mjyaBH0pGCxIZPb9U/hXtFlfSGU= 2022-09-03T06:50:02.184 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mjyaBH0pGCxIZPb9U/hXtFlfSGU=', 'Content-Length': '0'} 2022-09-03T06:50:02.187 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006eefb04eb882dc39-006312f91a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.230 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:02.231 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:02.232 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:02.232 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:02.233 DEBUG:boto:Method: GET 2022-09-03T06:50:02.233 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:50:02.234 DEBUG:boto:Data: 2022-09-03T06:50:02.235 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.235 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:02.236 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.237 DEBUG:boto:Params: {} 2022-09-03T06:50:02.238 DEBUG:boto:Token: None 2022-09-03T06:50:02.239 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/ 2022-09-03T06:50:02.239 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI= 2022-09-03T06:50:02.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI=', 'Content-Length': '0'} 2022-09-03T06:50:02.243 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e2924ff35d382fc7-006312f91a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.244 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:02.245 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:02.246 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:02.246 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:02.247 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:02.248 DEBUG:boto:Method: GET 2022-09-03T06:50:02.248 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:50:02.249 DEBUG:boto:Data: 2022-09-03T06:50:02.249 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.250 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:02.251 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.251 DEBUG:boto:Params: {} 2022-09-03T06:50:02.252 DEBUG:boto:Token: None 2022-09-03T06:50:02.252 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/ 2022-09-03T06:50:02.253 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI= 2022-09-03T06:50:02.253 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI=', 'Content-Length': '0'} 2022-09-03T06:50:02.258 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000053d45938650b106a-006312f91a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.260 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:02.260 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2022-09-03T06:50:02.261 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:02.262 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:02.263 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.263 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.264 DEBUG:boto:Method: GET 2022-09-03T06:50:02.264 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.265 DEBUG:boto:Data: 2022-09-03T06:50:02.265 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.266 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:02.266 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.267 DEBUG:boto:Params: {} 2022-09-03T06:50:02.268 DEBUG:boto:Token: None 2022-09-03T06:50:02.268 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.269 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nodcUZ5vjulR8LzFa93Qks+vBJ4= 2022-09-03T06:50:02.270 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nodcUZ5vjulR8LzFa93Qks+vBJ4=', 'Content-Length': '0'} 2022-09-03T06:50:02.272 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000be84a84f747c2617-006312f91a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.315 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:02.316 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:02.317 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.318 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.320 DEBUG:boto:Method: GET 2022-09-03T06:50:02.321 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.321 DEBUG:boto:Data: 2022-09-03T06:50:02.322 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.323 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:02.323 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.325 DEBUG:boto:Params: {} 2022-09-03T06:50:02.326 DEBUG:boto:Token: None 2022-09-03T06:50:02.327 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.329 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nodcUZ5vjulR8LzFa93Qks+vBJ4= 2022-09-03T06:50:02.330 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nodcUZ5vjulR8LzFa93Qks+vBJ4=', 'Content-Length': '0'} 2022-09-03T06:50:02.333 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000bfe3ae48fa04180-006312f91a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.334 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:02.335 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:02.336 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.337 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.338 DEBUG:boto:Method: HEAD 2022-09-03T06:50:02.339 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.339 DEBUG:boto:Data: 2022-09-03T06:50:02.340 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.341 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:02.341 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.342 DEBUG:boto:Params: {} 2022-09-03T06:50:02.343 DEBUG:boto:Token: None 2022-09-03T06:50:02.343 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.344 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FI9ePd/6eBrDpbXDPG9SyO7WfRM= 2022-09-03T06:50:02.345 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FI9ePd/6eBrDpbXDPG9SyO7WfRM=', 'Content-Length': '0'} 2022-09-03T06:50:02.347 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a9145fdb4cf380ec-006312f91a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.348 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:02.349 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:02.350 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.351 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.352 DEBUG:boto:Method: HEAD 2022-09-03T06:50:02.353 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.354 DEBUG:boto:Data: 2022-09-03T06:50:02.355 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.357 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:02.358 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.359 DEBUG:boto:Params: {} 2022-09-03T06:50:02.360 DEBUG:boto:Token: None 2022-09-03T06:50:02.361 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.361 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FI9ePd/6eBrDpbXDPG9SyO7WfRM= 2022-09-03T06:50:02.362 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FI9ePd/6eBrDpbXDPG9SyO7WfRM=', 'Content-Length': '0'} 2022-09-03T06:50:02.374 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000faf13402775a23c2-006312f91a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.376 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2022-09-03T06:50:02.377 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:02.378 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:02.379 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.380 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.381 DEBUG:boto:Method: GET 2022-09-03T06:50:02.381 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.382 DEBUG:boto:Data: 2022-09-03T06:50:02.383 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.383 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:02.384 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.384 DEBUG:boto:Params: {} 2022-09-03T06:50:02.385 DEBUG:boto:Token: None 2022-09-03T06:50:02.386 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.387 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nodcUZ5vjulR8LzFa93Qks+vBJ4= 2022-09-03T06:50:02.387 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nodcUZ5vjulR8LzFa93Qks+vBJ4=', 'Content-Length': '0'} 2022-09-03T06:50:02.389 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b2d6581f61337e1-006312f91a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.391 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:02.392 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:02.393 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.395 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.396 DEBUG:boto:Method: GET 2022-09-03T06:50:02.397 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.399 DEBUG:boto:Data: 2022-09-03T06:50:02.400 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.401 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:02.401 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.402 DEBUG:boto:Params: {} 2022-09-03T06:50:02.403 DEBUG:boto:Token: None 2022-09-03T06:50:02.404 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:02.405 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nodcUZ5vjulR8LzFa93Qks+vBJ4= 2022-09-03T06:50:02.405 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nodcUZ5vjulR8LzFa93Qks+vBJ4=', 'Content-Length': '0'} 2022-09-03T06:50:02.409 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a7a595243131418e-006312f91a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.411 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:02.412 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:02.413 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:02.414 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:02.415 DEBUG:boto:Method: GET 2022-09-03T06:50:02.416 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:50:02.416 DEBUG:boto:Data: 2022-09-03T06:50:02.417 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.417 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:02.418 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.419 DEBUG:boto:Params: {} 2022-09-03T06:50:02.420 DEBUG:boto:Token: None 2022-09-03T06:50:02.420 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/ 2022-09-03T06:50:02.421 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI= 2022-09-03T06:50:02.422 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI=', 'Content-Length': '0'} 2022-09-03T06:50:02.426 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a9a19e8abcdb6eb8-006312f91a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.427 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:02.428 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:02.429 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:02.430 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:02.430 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:02.431 DEBUG:boto:Method: GET 2022-09-03T06:50:02.432 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:50:02.432 DEBUG:boto:Data: 2022-09-03T06:50:02.433 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.433 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:02.434 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.435 DEBUG:boto:Params: {} 2022-09-03T06:50:02.436 DEBUG:boto:Token: None 2022-09-03T06:50:02.437 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/ 2022-09-03T06:50:02.438 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI= 2022-09-03T06:50:02.439 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI=', 'Content-Length': '0'} 2022-09-03T06:50:02.443 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000013be469d932357ed-006312f91a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.486 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:02.488 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:02.490 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:02.491 DEBUG:boto:Method: HEAD 2022-09-03T06:50:02.492 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:50:02.493 DEBUG:boto:Data: 2022-09-03T06:50:02.494 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.495 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:02.496 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.497 DEBUG:boto:Params: {} 2022-09-03T06:50:02.498 DEBUG:boto:Token: None 2022-09-03T06:50:02.500 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj124 2022-09-03T06:50:02.501 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rBT5IRgXA1gT3Vuf6CAuTxP736o= 2022-09-03T06:50:02.502 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rBT5IRgXA1gT3Vuf6CAuTxP736o=', 'Content-Length': '0'} 2022-09-03T06:50:02.504 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000097a7bb1fab3fde5a-006312f91a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.505 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:02.505 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:02.516 DEBUG:boto:Method: HEAD 2022-09-03T06:50:02.517 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:50:02.517 DEBUG:boto:Data: 2022-09-03T06:50:02.518 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.518 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:02.519 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.520 DEBUG:boto:Params: {} 2022-09-03T06:50:02.520 DEBUG:boto:Token: None 2022-09-03T06:50:02.521 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj124 2022-09-03T06:50:02.522 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rBT5IRgXA1gT3Vuf6CAuTxP736o= 2022-09-03T06:50:02.522 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rBT5IRgXA1gT3Vuf6CAuTxP736o=', 'Content-Length': '0'} 2022-09-03T06:50:02.524 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006bc6f2c575bef462-006312f91a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.525 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2022-09-03T06:50:02.526 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:02.526 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:02.527 DEBUG:boto:Method: GET 2022-09-03T06:50:02.528 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:50:02.528 DEBUG:boto:Data: 2022-09-03T06:50:02.529 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.529 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:02.530 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.531 DEBUG:boto:Params: {} 2022-09-03T06:50:02.532 DEBUG:boto:Token: None 2022-09-03T06:50:02.532 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj124 2022-09-03T06:50:02.533 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xNCBUvspBKshcCOcnWzVyoIkW3Y= 2022-09-03T06:50:02.534 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xNCBUvspBKshcCOcnWzVyoIkW3Y=', 'Content-Length': '0'} 2022-09-03T06:50:02.542 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fa07e6c0516107bf-006312f91a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.543 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:02.544 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:02.544 DEBUG:boto:Method: GET 2022-09-03T06:50:02.545 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:50:02.546 DEBUG:boto:Data: 2022-09-03T06:50:02.547 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.548 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:02.549 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.549 DEBUG:boto:Params: {} 2022-09-03T06:50:02.550 DEBUG:boto:Token: None 2022-09-03T06:50:02.551 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj124 2022-09-03T06:50:02.552 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xNCBUvspBKshcCOcnWzVyoIkW3Y= 2022-09-03T06:50:02.553 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xNCBUvspBKshcCOcnWzVyoIkW3Y=', 'Content-Length': '0'} 2022-09-03T06:50:02.564 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006c9924d4421ed8ea-006312f91a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.566 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:02.567 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:02.568 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:02.568 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:02.569 DEBUG:boto:Method: GET 2022-09-03T06:50:02.569 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:50:02.570 DEBUG:boto:Data: 2022-09-03T06:50:02.571 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.571 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:02.572 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.572 DEBUG:boto:Params: {} 2022-09-03T06:50:02.573 DEBUG:boto:Token: None 2022-09-03T06:50:02.574 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/ 2022-09-03T06:50:02.574 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI= 2022-09-03T06:50:02.575 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI=', 'Content-Length': '0'} 2022-09-03T06:50:02.578 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004e648fe79eb6666c-006312f91a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.578 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:02.579 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:02.580 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:02.580 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:02.582 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:02.582 DEBUG:boto:Method: GET 2022-09-03T06:50:02.583 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:50:02.583 DEBUG:boto:Data: 2022-09-03T06:50:02.584 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.585 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:02.585 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.587 DEBUG:boto:Params: {} 2022-09-03T06:50:02.588 DEBUG:boto:Token: None 2022-09-03T06:50:02.589 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/ 2022-09-03T06:50:02.589 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI= 2022-09-03T06:50:02.590 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI=', 'Content-Length': '0'} 2022-09-03T06:50:02.595 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000065e0cdd27c4fe717-006312f91a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.596 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:02.597 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:50:02.598 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:02.599 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:02.602 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.603 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.604 DEBUG:boto:Method: GET 2022-09-03T06:50:02.606 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.607 DEBUG:boto:Data: 2022-09-03T06:50:02.608 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.610 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:02.611 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.612 DEBUG:boto:Params: {} 2022-09-03T06:50:02.615 DEBUG:boto:Token: None 2022-09-03T06:50:02.618 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.619 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5q0zcRskBe3PNIGLyyz82rJ4Q8U= 2022-09-03T06:50:02.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5q0zcRskBe3PNIGLyyz82rJ4Q8U=', 'Content-Length': '0'} 2022-09-03T06:50:02.624 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a5cdb758b24c0057-006312f91a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.627 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:02.629 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:02.630 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.631 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.632 DEBUG:boto:Method: GET 2022-09-03T06:50:02.633 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.634 DEBUG:boto:Data: 2022-09-03T06:50:02.635 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.636 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:02.636 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.638 DEBUG:boto:Params: {} 2022-09-03T06:50:02.639 DEBUG:boto:Token: None 2022-09-03T06:50:02.640 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.641 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5q0zcRskBe3PNIGLyyz82rJ4Q8U= 2022-09-03T06:50:02.643 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5q0zcRskBe3PNIGLyyz82rJ4Q8U=', 'Content-Length': '0'} 2022-09-03T06:50:02.646 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006953eb6e63bd4db9-006312f91a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.647 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:02.648 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:02.648 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.649 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.649 DEBUG:boto:Method: HEAD 2022-09-03T06:50:02.650 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.651 DEBUG:boto:Data: 2022-09-03T06:50:02.651 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.652 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:02.652 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.653 DEBUG:boto:Params: {} 2022-09-03T06:50:02.653 DEBUG:boto:Token: None 2022-09-03T06:50:02.654 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.655 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BghWa/+YQgfY9v0tSfJgwr5orvs= 2022-09-03T06:50:02.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BghWa/+YQgfY9v0tSfJgwr5orvs=', 'Content-Length': '0'} 2022-09-03T06:50:02.658 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ff02667a7126269e-006312f91a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.659 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:02.660 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:02.661 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.662 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.663 DEBUG:boto:Method: HEAD 2022-09-03T06:50:02.664 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.665 DEBUG:boto:Data: 2022-09-03T06:50:02.666 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.666 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:02.667 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.668 DEBUG:boto:Params: {} 2022-09-03T06:50:02.668 DEBUG:boto:Token: None 2022-09-03T06:50:02.669 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.669 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BghWa/+YQgfY9v0tSfJgwr5orvs= 2022-09-03T06:50:02.670 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BghWa/+YQgfY9v0tSfJgwr5orvs=', 'Content-Length': '0'} 2022-09-03T06:50:02.673 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000059bf1ddb3d87569a-006312f91a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.674 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:50:02.675 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:02.676 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:02.677 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.678 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.679 DEBUG:boto:Method: GET 2022-09-03T06:50:02.680 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.681 DEBUG:boto:Data: 2022-09-03T06:50:02.682 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.683 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:02.684 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.685 DEBUG:boto:Params: {} 2022-09-03T06:50:02.686 DEBUG:boto:Token: None 2022-09-03T06:50:02.687 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.696 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5q0zcRskBe3PNIGLyyz82rJ4Q8U= 2022-09-03T06:50:02.697 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5q0zcRskBe3PNIGLyyz82rJ4Q8U=', 'Content-Length': '0'} 2022-09-03T06:50:02.700 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f13b89d832c32a03-006312f91a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.746 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:02.748 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:02.750 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.751 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.751 DEBUG:boto:Method: GET 2022-09-03T06:50:02.752 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.753 DEBUG:boto:Data: 2022-09-03T06:50:02.753 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.754 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:02.754 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.755 DEBUG:boto:Params: {} 2022-09-03T06:50:02.756 DEBUG:boto:Token: None 2022-09-03T06:50:02.756 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:02.757 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5q0zcRskBe3PNIGLyyz82rJ4Q8U= 2022-09-03T06:50:02.758 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5q0zcRskBe3PNIGLyyz82rJ4Q8U=', 'Content-Length': '0'} 2022-09-03T06:50:02.760 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007d52be131130b4ee-006312f91a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.761 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:02.762 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:02.762 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:02.763 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:02.763 DEBUG:boto:Method: GET 2022-09-03T06:50:02.764 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:50:02.764 DEBUG:boto:Data: 2022-09-03T06:50:02.765 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.766 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:02.766 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.767 DEBUG:boto:Params: {} 2022-09-03T06:50:02.768 DEBUG:boto:Token: None 2022-09-03T06:50:02.768 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/ 2022-09-03T06:50:02.769 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI= 2022-09-03T06:50:02.769 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI=', 'Content-Length': '0'} 2022-09-03T06:50:02.775 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000edd31861c482b6c3-006312f91a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.776 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:02.777 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:02.778 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:02.779 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:02.779 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:02.780 DEBUG:boto:Method: GET 2022-09-03T06:50:02.780 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:50:02.782 DEBUG:boto:Data: 2022-09-03T06:50:02.783 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.784 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:02.784 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.785 DEBUG:boto:Params: {} 2022-09-03T06:50:02.786 DEBUG:boto:Token: None 2022-09-03T06:50:02.787 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/ 2022-09-03T06:50:02.787 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI= 2022-09-03T06:50:02.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI=', 'Content-Length': '0'} 2022-09-03T06:50:02.795 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e9a9d52cf33167d2-006312f91a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.796 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:02.797 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:02.797 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:02.798 DEBUG:boto:Method: HEAD 2022-09-03T06:50:02.798 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:50:02.799 DEBUG:boto:Data: 2022-09-03T06:50:02.800 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.800 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:02.801 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.801 DEBUG:boto:Params: {} 2022-09-03T06:50:02.802 DEBUG:boto:Token: None 2022-09-03T06:50:02.803 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj145 2022-09-03T06:50:02.804 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pOArlhB0V/CgpzXA4606SR4bIZ0= 2022-09-03T06:50:02.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pOArlhB0V/CgpzXA4606SR4bIZ0=', 'Content-Length': '0'} 2022-09-03T06:50:02.808 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048c8bcffc2beb19b-006312f91a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.808 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:02.809 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:02.809 DEBUG:boto:Method: HEAD 2022-09-03T06:50:02.810 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:50:02.810 DEBUG:boto:Data: 2022-09-03T06:50:02.811 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.811 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:02.812 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.812 DEBUG:boto:Params: {} 2022-09-03T06:50:02.813 DEBUG:boto:Token: None 2022-09-03T06:50:02.814 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj145 2022-09-03T06:50:02.814 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pOArlhB0V/CgpzXA4606SR4bIZ0= 2022-09-03T06:50:02.815 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pOArlhB0V/CgpzXA4606SR4bIZ0=', 'Content-Length': '0'} 2022-09-03T06:50:02.817 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004493978ed93907a0-006312f91a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.818 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:50:02.818 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:02.819 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:02.819 DEBUG:boto:Method: GET 2022-09-03T06:50:02.820 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:50:02.820 DEBUG:boto:Data: 2022-09-03T06:50:02.821 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.822 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:02.822 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.823 DEBUG:boto:Params: {} 2022-09-03T06:50:02.823 DEBUG:boto:Token: None 2022-09-03T06:50:02.824 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj145 2022-09-03T06:50:02.824 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vwr406HcZh9n97ATrr5ZF57hTpk= 2022-09-03T06:50:02.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vwr406HcZh9n97ATrr5ZF57hTpk=', 'Content-Length': '0'} 2022-09-03T06:50:02.827 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2877865edc8ca64-006312f91a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.828 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:02.829 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:02.829 DEBUG:boto:Method: GET 2022-09-03T06:50:02.830 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:50:02.830 DEBUG:boto:Data: 2022-09-03T06:50:02.831 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.831 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:02.832 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.832 DEBUG:boto:Params: {} 2022-09-03T06:50:02.833 DEBUG:boto:Token: None 2022-09-03T06:50:02.834 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj145 2022-09-03T06:50:02.834 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vwr406HcZh9n97ATrr5ZF57hTpk= 2022-09-03T06:50:02.835 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vwr406HcZh9n97ATrr5ZF57hTpk=', 'Content-Length': '0'} 2022-09-03T06:50:02.837 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c187907b50220b8d-006312f91a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.878 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:02.879 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:02.880 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:02.881 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:02.882 DEBUG:boto:Method: GET 2022-09-03T06:50:02.884 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:50:02.885 DEBUG:boto:Data: 2022-09-03T06:50:02.886 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.887 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:02.887 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.888 DEBUG:boto:Params: {} 2022-09-03T06:50:02.889 DEBUG:boto:Token: None 2022-09-03T06:50:02.889 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/ 2022-09-03T06:50:02.891 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI= 2022-09-03T06:50:02.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI=', 'Content-Length': '0'} 2022-09-03T06:50:02.897 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000af19510491c937f-006312f91a-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.899 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:02.904 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:02.905 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:02.906 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:02.906 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:02.907 DEBUG:boto:Method: GET 2022-09-03T06:50:02.908 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:50:02.908 DEBUG:boto:Data: 2022-09-03T06:50:02.909 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.909 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:02.910 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.910 DEBUG:boto:Params: {} 2022-09-03T06:50:02.911 DEBUG:boto:Token: None 2022-09-03T06:50:02.912 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/ 2022-09-03T06:50:02.912 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI= 2022-09-03T06:50:02.913 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UeWu5986HoZKoQ0guWvK8XOCOXI=', 'Content-Length': '0'} 2022-09-03T06:50:02.918 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007376bdd3a377d850-006312f91a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.919 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:02.920 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj148 2022-09-03T06:50:02.921 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:02.921 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:02.922 DEBUG:boto:path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:02.923 DEBUG:boto:auth_path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:02.923 DEBUG:boto:Method: GET 2022-09-03T06:50:02.924 DEBUG:boto:Path: /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:02.924 DEBUG:boto:Data: 2022-09-03T06:50:02.925 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.926 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:02.926 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.927 DEBUG:boto:Params: {} 2022-09-03T06:50:02.928 DEBUG:boto:Token: None 2022-09-03T06:50:02.929 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:02.929 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JGu3NCEtQETeOPaEaxQw4Wp0yhM= 2022-09-03T06:50:02.930 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JGu3NCEtQETeOPaEaxQw4Wp0yhM=', 'Content-Length': '0'} 2022-09-03T06:50:02.933 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a7bc6393e36c3e06-006312f91a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.934 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:02.935 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:02.936 DEBUG:boto:path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:02.936 DEBUG:boto:auth_path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:02.937 DEBUG:boto:Method: GET 2022-09-03T06:50:02.938 DEBUG:boto:Path: /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:02.938 DEBUG:boto:Data: 2022-09-03T06:50:02.939 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.939 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:02.940 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.941 DEBUG:boto:Params: {} 2022-09-03T06:50:02.943 DEBUG:boto:Token: None 2022-09-03T06:50:02.944 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:02.945 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JGu3NCEtQETeOPaEaxQw4Wp0yhM= 2022-09-03T06:50:02.945 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JGu3NCEtQETeOPaEaxQw4Wp0yhM=', 'Content-Length': '0'} 2022-09-03T06:50:02.948 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000785d39aa7f76ae7d-006312f91a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.949 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:02.949 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:02.950 DEBUG:boto:path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:02.952 DEBUG:boto:auth_path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:02.952 DEBUG:boto:Method: HEAD 2022-09-03T06:50:02.953 DEBUG:boto:Path: /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:02.954 DEBUG:boto:Data: 2022-09-03T06:50:02.955 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.955 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:02.956 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.956 DEBUG:boto:Params: {} 2022-09-03T06:50:02.957 DEBUG:boto:Token: None 2022-09-03T06:50:02.963 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:02.964 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xh1RBAwC9HYEpXt6uYC02a8JCVY= 2022-09-03T06:50:02.965 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xh1RBAwC9HYEpXt6uYC02a8JCVY=', 'Content-Length': '0'} 2022-09-03T06:50:02.968 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c95109362926aa98-006312f91a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.969 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:02.970 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:02.970 DEBUG:boto:path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:02.971 DEBUG:boto:auth_path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:02.971 DEBUG:boto:Method: HEAD 2022-09-03T06:50:02.973 DEBUG:boto:Path: /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:02.973 DEBUG:boto:Data: 2022-09-03T06:50:02.974 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.974 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:02.975 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.975 DEBUG:boto:Params: {} 2022-09-03T06:50:02.976 DEBUG:boto:Token: None 2022-09-03T06:50:02.977 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:02.978 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xh1RBAwC9HYEpXt6uYC02a8JCVY= 2022-09-03T06:50:02.978 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xh1RBAwC9HYEpXt6uYC02a8JCVY=', 'Content-Length': '0'} 2022-09-03T06:50:02.981 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009244a0f7e637d143-006312f91a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.981 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj148 2022-09-03T06:50:02.983 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:02.983 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:02.984 DEBUG:boto:path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:02.985 DEBUG:boto:auth_path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:02.985 DEBUG:boto:Method: GET 2022-09-03T06:50:02.986 DEBUG:boto:Path: /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:02.986 DEBUG:boto:Data: 2022-09-03T06:50:02.987 DEBUG:boto:Headers: {} 2022-09-03T06:50:02.987 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:02.988 DEBUG:boto:Port: 8000 2022-09-03T06:50:02.989 DEBUG:boto:Params: {} 2022-09-03T06:50:02.990 DEBUG:boto:Token: None 2022-09-03T06:50:02.990 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:02 GMT /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:02.991 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JGu3NCEtQETeOPaEaxQw4Wp0yhM= 2022-09-03T06:50:02.992 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:02 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JGu3NCEtQETeOPaEaxQw4Wp0yhM=', 'Content-Length': '0'} 2022-09-03T06:50:02.994 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002c524fafee92c9a2-006312f91a-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:02 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:02.995 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:02.995 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:02.996 DEBUG:boto:path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:02.997 DEBUG:boto:auth_path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:02.997 DEBUG:boto:Method: GET 2022-09-03T06:50:02.998 DEBUG:boto:Path: /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:02.999 DEBUG:boto:Data: 2022-09-03T06:50:02.999 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.000 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.001 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.002 DEBUG:boto:Params: {} 2022-09-03T06:50:03.004 DEBUG:boto:Token: None 2022-09-03T06:50:03.005 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:03.028 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4eqRG/nYBVmUPwybfD2pNYh+9M4= 2022-09-03T06:50:03.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4eqRG/nYBVmUPwybfD2pNYh+9M4=', 'Content-Length': '0'} 2022-09-03T06:50:03.031 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a70ad9e156a3d691-006312f91b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.032 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:03.033 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:03.033 DEBUG:boto:path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:03.034 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:03.035 DEBUG:boto:Method: GET 2022-09-03T06:50:03.037 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj148 2022-09-03T06:50:03.037 DEBUG:boto:Data: 2022-09-03T06:50:03.038 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.039 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:03.039 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.040 DEBUG:boto:Params: {} 2022-09-03T06:50:03.041 DEBUG:boto:Token: None 2022-09-03T06:50:03.042 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/ 2022-09-03T06:50:03.043 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U= 2022-09-03T06:50:03.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U=', 'Content-Length': '0'} 2022-09-03T06:50:03.052 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004c7448bb9ad64db7-006312f91b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.054 DEBUG:boto:b'dsgvcy-75obj1481000falseobj1482022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:03.055 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:03.056 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:03.060 DEBUG:boto:path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:03.061 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:03.061 DEBUG:boto:Method: GET 2022-09-03T06:50:03.062 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj148 2022-09-03T06:50:03.062 DEBUG:boto:Data: 2022-09-03T06:50:03.063 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.063 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.064 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.065 DEBUG:boto:Params: {} 2022-09-03T06:50:03.065 DEBUG:boto:Token: None 2022-09-03T06:50:03.066 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/ 2022-09-03T06:50:03.067 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U= 2022-09-03T06:50:03.067 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U=', 'Content-Length': '0'} 2022-09-03T06:50:03.072 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005bccf0f6688d2f83-006312f91b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.073 DEBUG:boto:b'dsgvcy-75obj1481000falseobj1482022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:03.074 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:03.074 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:03.075 DEBUG:boto:Method: HEAD 2022-09-03T06:50:03.075 DEBUG:boto:Path: /dsgvcy-75/obj148 2022-09-03T06:50:03.076 DEBUG:boto:Data: 2022-09-03T06:50:03.077 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.077 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:03.078 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.078 DEBUG:boto:Params: {} 2022-09-03T06:50:03.079 DEBUG:boto:Token: None 2022-09-03T06:50:03.080 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj148 2022-09-03T06:50:03.080 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7M6iBVGxFC3cwCabCF5IQmeXSMA= 2022-09-03T06:50:03.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7M6iBVGxFC3cwCabCF5IQmeXSMA=', 'Content-Length': '0'} 2022-09-03T06:50:03.083 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000005c3a993ca55c6b2-006312f91b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.084 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:03.084 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:03.085 DEBUG:boto:Method: HEAD 2022-09-03T06:50:03.086 DEBUG:boto:Path: /dsgvcy-75/obj148 2022-09-03T06:50:03.086 DEBUG:boto:Data: 2022-09-03T06:50:03.087 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.087 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.088 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.088 DEBUG:boto:Params: {} 2022-09-03T06:50:03.089 DEBUG:boto:Token: None 2022-09-03T06:50:03.089 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj148 2022-09-03T06:50:03.090 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7M6iBVGxFC3cwCabCF5IQmeXSMA= 2022-09-03T06:50:03.091 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7M6iBVGxFC3cwCabCF5IQmeXSMA=', 'Content-Length': '0'} 2022-09-03T06:50:03.093 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000254d431bd1ad2201-006312f91b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.094 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj148 2022-09-03T06:50:03.095 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:03.095 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:03.096 DEBUG:boto:Method: GET 2022-09-03T06:50:03.097 DEBUG:boto:Path: /dsgvcy-75/obj148 2022-09-03T06:50:03.097 DEBUG:boto:Data: 2022-09-03T06:50:03.098 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.099 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:03.099 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.100 DEBUG:boto:Params: {} 2022-09-03T06:50:03.101 DEBUG:boto:Token: None 2022-09-03T06:50:03.102 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj148 2022-09-03T06:50:03.105 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NBiIcRPDpKqhM5DUCqwx2Jsr9Xc= 2022-09-03T06:50:03.105 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NBiIcRPDpKqhM5DUCqwx2Jsr9Xc=', 'Content-Length': '0'} 2022-09-03T06:50:03.108 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b8ed6313ee54aaa8-006312f91b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.150 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:03.151 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:03.151 DEBUG:boto:Method: GET 2022-09-03T06:50:03.152 DEBUG:boto:Path: /dsgvcy-75/obj148 2022-09-03T06:50:03.153 DEBUG:boto:Data: 2022-09-03T06:50:03.153 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.154 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.154 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.155 DEBUG:boto:Params: {} 2022-09-03T06:50:03.155 DEBUG:boto:Token: None 2022-09-03T06:50:03.156 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj148 2022-09-03T06:50:03.157 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NBiIcRPDpKqhM5DUCqwx2Jsr9Xc= 2022-09-03T06:50:03.157 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NBiIcRPDpKqhM5DUCqwx2Jsr9Xc=', 'Content-Length': '0'} 2022-09-03T06:50:03.159 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b3489d0d7b5e137d-006312f91b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.160 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:03.161 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:03.161 DEBUG:boto:path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:03.162 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:03.162 DEBUG:boto:Method: GET 2022-09-03T06:50:03.163 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj148 2022-09-03T06:50:03.164 DEBUG:boto:Data: 2022-09-03T06:50:03.164 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.165 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:03.165 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.166 DEBUG:boto:Params: {} 2022-09-03T06:50:03.167 DEBUG:boto:Token: None 2022-09-03T06:50:03.168 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/ 2022-09-03T06:50:03.169 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U= 2022-09-03T06:50:03.169 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U=', 'Content-Length': '0'} 2022-09-03T06:50:03.172 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000047cdfdeb2a7bf72d-006312f91b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.173 DEBUG:boto:b'dsgvcy-75obj1481000falseobj1482022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:03.174 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:03.174 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:03.175 DEBUG:boto:path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:03.175 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:03.176 DEBUG:boto:Method: GET 2022-09-03T06:50:03.176 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj148 2022-09-03T06:50:03.179 DEBUG:boto:Data: 2022-09-03T06:50:03.179 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.180 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.180 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.181 DEBUG:boto:Params: {} 2022-09-03T06:50:03.182 DEBUG:boto:Token: None 2022-09-03T06:50:03.182 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/ 2022-09-03T06:50:03.183 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U= 2022-09-03T06:50:03.183 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U=', 'Content-Length': '0'} 2022-09-03T06:50:03.188 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c746652ea02367d5-006312f91b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.189 DEBUG:boto:b'dsgvcy-75obj1481000falseobj1482022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:03.190 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2022-09-03T06:50:03.190 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:03.191 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:03.191 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.192 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.192 DEBUG:boto:Method: GET 2022-09-03T06:50:03.193 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.193 DEBUG:boto:Data: 2022-09-03T06:50:03.194 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.194 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:03.195 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.195 DEBUG:boto:Params: {} 2022-09-03T06:50:03.196 DEBUG:boto:Token: None 2022-09-03T06:50:03.197 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.197 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EZa9Au/iZj//cmLnbKS39voV+PQ= 2022-09-03T06:50:03.198 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EZa9Au/iZj//cmLnbKS39voV+PQ=', 'Content-Length': '0'} 2022-09-03T06:50:03.200 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d767df38eb441eea-006312f91b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.201 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:03.202 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:03.203 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.203 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.204 DEBUG:boto:Method: GET 2022-09-03T06:50:03.204 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.205 DEBUG:boto:Data: 2022-09-03T06:50:03.205 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.206 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.207 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.207 DEBUG:boto:Params: {} 2022-09-03T06:50:03.208 DEBUG:boto:Token: None 2022-09-03T06:50:03.208 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.209 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EZa9Au/iZj//cmLnbKS39voV+PQ= 2022-09-03T06:50:03.210 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EZa9Au/iZj//cmLnbKS39voV+PQ=', 'Content-Length': '0'} 2022-09-03T06:50:03.212 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f671579d3cb0bb8-006312f91b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.213 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:03.213 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:03.214 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.214 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.215 DEBUG:boto:Method: HEAD 2022-09-03T06:50:03.216 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.216 DEBUG:boto:Data: 2022-09-03T06:50:03.217 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.217 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:03.218 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.218 DEBUG:boto:Params: {} 2022-09-03T06:50:03.219 DEBUG:boto:Token: None 2022-09-03T06:50:03.219 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.220 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:23mS5/YVQctlumX3jD7jO31hblo= 2022-09-03T06:50:03.221 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:23mS5/YVQctlumX3jD7jO31hblo=', 'Content-Length': '0'} 2022-09-03T06:50:03.223 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000acb69d1f5f3347f4-006312f91b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.225 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:03.225 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:03.226 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.227 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.227 DEBUG:boto:Method: HEAD 2022-09-03T06:50:03.228 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.228 DEBUG:boto:Data: 2022-09-03T06:50:03.229 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.231 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.231 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.232 DEBUG:boto:Params: {} 2022-09-03T06:50:03.232 DEBUG:boto:Token: None 2022-09-03T06:50:03.233 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.234 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:23mS5/YVQctlumX3jD7jO31hblo= 2022-09-03T06:50:03.234 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:23mS5/YVQctlumX3jD7jO31hblo=', 'Content-Length': '0'} 2022-09-03T06:50:03.238 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000904dab07f4fae7df-006312f91b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.239 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2022-09-03T06:50:03.239 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:03.240 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:03.240 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.241 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.241 DEBUG:boto:Method: GET 2022-09-03T06:50:03.242 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.243 DEBUG:boto:Data: 2022-09-03T06:50:03.243 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.244 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:03.244 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.245 DEBUG:boto:Params: {} 2022-09-03T06:50:03.245 DEBUG:boto:Token: None 2022-09-03T06:50:03.246 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.247 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EZa9Au/iZj//cmLnbKS39voV+PQ= 2022-09-03T06:50:03.247 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EZa9Au/iZj//cmLnbKS39voV+PQ=', 'Content-Length': '0'} 2022-09-03T06:50:03.250 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e82a429f3c525f3c-006312f91b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.251 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:03.251 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:03.252 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.253 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.253 DEBUG:boto:Method: GET 2022-09-03T06:50:03.254 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.254 DEBUG:boto:Data: 2022-09-03T06:50:03.255 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.255 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.256 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.257 DEBUG:boto:Params: {} 2022-09-03T06:50:03.257 DEBUG:boto:Token: None 2022-09-03T06:50:03.258 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:03.259 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EZa9Au/iZj//cmLnbKS39voV+PQ= 2022-09-03T06:50:03.259 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EZa9Au/iZj//cmLnbKS39voV+PQ=', 'Content-Length': '0'} 2022-09-03T06:50:03.261 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009c4b4817b3a98acc-006312f91b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.262 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:03.263 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:03.264 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:03.264 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:03.265 DEBUG:boto:Method: GET 2022-09-03T06:50:03.265 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:50:03.266 DEBUG:boto:Data: 2022-09-03T06:50:03.266 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.267 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:03.268 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.268 DEBUG:boto:Params: {} 2022-09-03T06:50:03.269 DEBUG:boto:Token: None 2022-09-03T06:50:03.269 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/ 2022-09-03T06:50:03.270 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U= 2022-09-03T06:50:03.271 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U=', 'Content-Length': '0'} 2022-09-03T06:50:03.274 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000092ba1072a035baa4-006312f91b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.275 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:03.275 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:03.276 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:03.277 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:03.277 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:03.278 DEBUG:boto:Method: GET 2022-09-03T06:50:03.278 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:50:03.279 DEBUG:boto:Data: 2022-09-03T06:50:03.279 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.280 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.280 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.281 DEBUG:boto:Params: {} 2022-09-03T06:50:03.282 DEBUG:boto:Token: None 2022-09-03T06:50:03.283 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/ 2022-09-03T06:50:03.283 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U= 2022-09-03T06:50:03.284 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U=', 'Content-Length': '0'} 2022-09-03T06:50:03.288 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a0375c74a45308b4-006312f91b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.289 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:03.290 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:03.291 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:03.291 DEBUG:boto:Method: HEAD 2022-09-03T06:50:03.292 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:50:03.293 DEBUG:boto:Data: 2022-09-03T06:50:03.293 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.294 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:03.295 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.295 DEBUG:boto:Params: {} 2022-09-03T06:50:03.296 DEBUG:boto:Token: None 2022-09-03T06:50:03.297 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj155 2022-09-03T06:50:03.298 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Cv4YU3XSaAtOmhU+7HAfoxOhUkk= 2022-09-03T06:50:03.298 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Cv4YU3XSaAtOmhU+7HAfoxOhUkk=', 'Content-Length': '0'} 2022-09-03T06:50:03.300 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad13bc5421162b00-006312f91b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.301 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:03.302 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:03.302 DEBUG:boto:Method: HEAD 2022-09-03T06:50:03.303 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:50:03.304 DEBUG:boto:Data: 2022-09-03T06:50:03.304 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.305 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.305 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.306 DEBUG:boto:Params: {} 2022-09-03T06:50:03.306 DEBUG:boto:Token: None 2022-09-03T06:50:03.307 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj155 2022-09-03T06:50:03.308 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Cv4YU3XSaAtOmhU+7HAfoxOhUkk= 2022-09-03T06:50:03.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Cv4YU3XSaAtOmhU+7HAfoxOhUkk=', 'Content-Length': '0'} 2022-09-03T06:50:03.311 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000097405df528b23027-006312f91b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.312 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2022-09-03T06:50:03.312 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:03.313 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:03.314 DEBUG:boto:Method: GET 2022-09-03T06:50:03.315 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:50:03.316 DEBUG:boto:Data: 2022-09-03T06:50:03.316 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.317 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:03.317 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.318 DEBUG:boto:Params: {} 2022-09-03T06:50:03.319 DEBUG:boto:Token: None 2022-09-03T06:50:03.319 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj155 2022-09-03T06:50:03.320 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ejzN+gBqTQhnGJELgcCwRbZA5wE= 2022-09-03T06:50:03.321 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ejzN+gBqTQhnGJELgcCwRbZA5wE=', 'Content-Length': '0'} 2022-09-03T06:50:03.323 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2c3ee8e6fb9abd7-006312f91b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.369 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:03.370 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:03.371 DEBUG:boto:Method: GET 2022-09-03T06:50:03.371 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:50:03.372 DEBUG:boto:Data: 2022-09-03T06:50:03.372 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.373 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.373 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.374 DEBUG:boto:Params: {} 2022-09-03T06:50:03.375 DEBUG:boto:Token: None 2022-09-03T06:50:03.375 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj155 2022-09-03T06:50:03.376 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ejzN+gBqTQhnGJELgcCwRbZA5wE= 2022-09-03T06:50:03.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ejzN+gBqTQhnGJELgcCwRbZA5wE=', 'Content-Length': '0'} 2022-09-03T06:50:03.379 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009ed2ac04b6eeb6e6-006312f91b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.380 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:03.380 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:03.381 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:03.381 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:03.382 DEBUG:boto:Method: GET 2022-09-03T06:50:03.383 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:50:03.383 DEBUG:boto:Data: 2022-09-03T06:50:03.384 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.384 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:03.385 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.385 DEBUG:boto:Params: {} 2022-09-03T06:50:03.387 DEBUG:boto:Token: None 2022-09-03T06:50:03.387 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/ 2022-09-03T06:50:03.388 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U= 2022-09-03T06:50:03.388 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U=', 'Content-Length': '0'} 2022-09-03T06:50:03.391 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a452e6abec0325d0-006312f91b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.392 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:03.393 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:03.393 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:03.394 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:03.394 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:03.395 DEBUG:boto:Method: GET 2022-09-03T06:50:03.396 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:50:03.397 DEBUG:boto:Data: 2022-09-03T06:50:03.397 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.398 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.398 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.399 DEBUG:boto:Params: {} 2022-09-03T06:50:03.399 DEBUG:boto:Token: None 2022-09-03T06:50:03.400 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/ 2022-09-03T06:50:03.401 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U= 2022-09-03T06:50:03.402 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U=', 'Content-Length': '0'} 2022-09-03T06:50:03.406 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cbabb7053e4dd7d8-006312f91b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.454 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:03.455 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:50:03.456 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:03.456 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:03.457 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.457 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.458 DEBUG:boto:Method: GET 2022-09-03T06:50:03.458 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.459 DEBUG:boto:Data: 2022-09-03T06:50:03.459 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.460 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:03.461 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.461 DEBUG:boto:Params: {} 2022-09-03T06:50:03.462 DEBUG:boto:Token: None 2022-09-03T06:50:03.463 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.463 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NcL3Xg4SLw2p+Z6QuoBk225zHCA= 2022-09-03T06:50:03.464 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NcL3Xg4SLw2p+Z6QuoBk225zHCA=', 'Content-Length': '0'} 2022-09-03T06:50:03.467 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fe46cd9e3bb36246-006312f91b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.467 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:03.468 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:03.469 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.469 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.470 DEBUG:boto:Method: GET 2022-09-03T06:50:03.470 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.471 DEBUG:boto:Data: 2022-09-03T06:50:03.471 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.472 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.472 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.473 DEBUG:boto:Params: {} 2022-09-03T06:50:03.474 DEBUG:boto:Token: None 2022-09-03T06:50:03.474 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.475 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NcL3Xg4SLw2p+Z6QuoBk225zHCA= 2022-09-03T06:50:03.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NcL3Xg4SLw2p+Z6QuoBk225zHCA=', 'Content-Length': '0'} 2022-09-03T06:50:03.479 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f778c61d564e2d01-006312f91b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.480 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:03.481 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:03.481 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.482 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.483 DEBUG:boto:Method: HEAD 2022-09-03T06:50:03.483 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.484 DEBUG:boto:Data: 2022-09-03T06:50:03.484 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.485 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:03.485 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.486 DEBUG:boto:Params: {} 2022-09-03T06:50:03.486 DEBUG:boto:Token: None 2022-09-03T06:50:03.487 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.488 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZpyQgaaaXCJ57Q4VHTUgflFodsY= 2022-09-03T06:50:03.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZpyQgaaaXCJ57Q4VHTUgflFodsY=', 'Content-Length': '0'} 2022-09-03T06:50:03.490 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e8d059f9ea48b135-006312f91b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.491 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:03.492 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:03.493 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.493 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.494 DEBUG:boto:Method: HEAD 2022-09-03T06:50:03.494 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.495 DEBUG:boto:Data: 2022-09-03T06:50:03.496 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.496 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.497 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.497 DEBUG:boto:Params: {} 2022-09-03T06:50:03.498 DEBUG:boto:Token: None 2022-09-03T06:50:03.499 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.500 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZpyQgaaaXCJ57Q4VHTUgflFodsY= 2022-09-03T06:50:03.501 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZpyQgaaaXCJ57Q4VHTUgflFodsY=', 'Content-Length': '0'} 2022-09-03T06:50:03.504 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000edfde4eb378c332e-006312f91b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.505 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:50:03.506 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:03.507 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:03.508 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.510 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.511 DEBUG:boto:Method: GET 2022-09-03T06:50:03.512 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.513 DEBUG:boto:Data: 2022-09-03T06:50:03.514 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.515 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:03.516 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.516 DEBUG:boto:Params: {} 2022-09-03T06:50:03.517 DEBUG:boto:Token: None 2022-09-03T06:50:03.517 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.518 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NcL3Xg4SLw2p+Z6QuoBk225zHCA= 2022-09-03T06:50:03.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NcL3Xg4SLw2p+Z6QuoBk225zHCA=', 'Content-Length': '0'} 2022-09-03T06:50:03.522 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ff25934c7b409176-006312f91b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.562 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:03.563 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:03.565 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.566 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.567 DEBUG:boto:Method: GET 2022-09-03T06:50:03.568 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.569 DEBUG:boto:Data: 2022-09-03T06:50:03.570 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.571 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.572 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.573 DEBUG:boto:Params: {} 2022-09-03T06:50:03.574 DEBUG:boto:Token: None 2022-09-03T06:50:03.575 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:03.577 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NcL3Xg4SLw2p+Z6QuoBk225zHCA= 2022-09-03T06:50:03.578 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NcL3Xg4SLw2p+Z6QuoBk225zHCA=', 'Content-Length': '0'} 2022-09-03T06:50:03.584 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000df2326b27cf9fc77-006312f91b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.585 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:03.586 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:03.587 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:03.587 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:03.588 DEBUG:boto:Method: GET 2022-09-03T06:50:03.588 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:50:03.589 DEBUG:boto:Data: 2022-09-03T06:50:03.589 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.590 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:03.591 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.591 DEBUG:boto:Params: {} 2022-09-03T06:50:03.592 DEBUG:boto:Token: None 2022-09-03T06:50:03.592 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/ 2022-09-03T06:50:03.593 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U= 2022-09-03T06:50:03.594 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U=', 'Content-Length': '0'} 2022-09-03T06:50:03.596 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000037dce942b13972f7-006312f91b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.597 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:03.598 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:03.599 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:03.599 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:03.600 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:03.601 DEBUG:boto:Method: GET 2022-09-03T06:50:03.601 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:50:03.602 DEBUG:boto:Data: 2022-09-03T06:50:03.602 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.603 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.603 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.604 DEBUG:boto:Params: {} 2022-09-03T06:50:03.605 DEBUG:boto:Token: None 2022-09-03T06:50:03.605 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/ 2022-09-03T06:50:03.606 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U= 2022-09-03T06:50:03.607 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U=', 'Content-Length': '0'} 2022-09-03T06:50:03.615 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000698203278733d462-006312f91b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.662 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:03.663 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:03.664 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:03.665 DEBUG:boto:Method: HEAD 2022-09-03T06:50:03.665 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:50:03.666 DEBUG:boto:Data: 2022-09-03T06:50:03.666 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.667 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:03.668 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.668 DEBUG:boto:Params: {} 2022-09-03T06:50:03.669 DEBUG:boto:Token: None 2022-09-03T06:50:03.669 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj17 2022-09-03T06:50:03.670 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cmQzuZ++1Ir0ifhyREEB5C9pooU= 2022-09-03T06:50:03.671 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cmQzuZ++1Ir0ifhyREEB5C9pooU=', 'Content-Length': '0'} 2022-09-03T06:50:03.674 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002701bcaeaa0e813f-006312f91b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.676 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:03.676 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:03.677 DEBUG:boto:Method: HEAD 2022-09-03T06:50:03.678 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:50:03.679 DEBUG:boto:Data: 2022-09-03T06:50:03.679 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.680 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.681 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.683 DEBUG:boto:Params: {} 2022-09-03T06:50:03.685 DEBUG:boto:Token: None 2022-09-03T06:50:03.686 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj17 2022-09-03T06:50:03.687 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cmQzuZ++1Ir0ifhyREEB5C9pooU= 2022-09-03T06:50:03.689 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cmQzuZ++1Ir0ifhyREEB5C9pooU=', 'Content-Length': '0'} 2022-09-03T06:50:03.693 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d01ebf0064a3dd9c-006312f91b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.694 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:50:03.696 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:03.697 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:03.698 DEBUG:boto:Method: GET 2022-09-03T06:50:03.698 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:50:03.699 DEBUG:boto:Data: 2022-09-03T06:50:03.701 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.702 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:03.704 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.706 DEBUG:boto:Params: {} 2022-09-03T06:50:03.708 DEBUG:boto:Token: None 2022-09-03T06:50:03.709 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj17 2022-09-03T06:50:03.710 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oUbfBdXYTlf3eq/iVKQuJS466II= 2022-09-03T06:50:03.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oUbfBdXYTlf3eq/iVKQuJS466II=', 'Content-Length': '0'} 2022-09-03T06:50:03.715 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003338b6b453e3e583-006312f91b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.718 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:03.720 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:03.722 DEBUG:boto:Method: GET 2022-09-03T06:50:03.723 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:50:03.724 DEBUG:boto:Data: 2022-09-03T06:50:03.725 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.727 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.728 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.729 DEBUG:boto:Params: {} 2022-09-03T06:50:03.730 DEBUG:boto:Token: None 2022-09-03T06:50:03.731 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj17 2022-09-03T06:50:03.733 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oUbfBdXYTlf3eq/iVKQuJS466II= 2022-09-03T06:50:03.735 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oUbfBdXYTlf3eq/iVKQuJS466II=', 'Content-Length': '0'} 2022-09-03T06:50:03.738 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b7b81a1e5097f1f4-006312f91b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.740 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:03.741 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:03.742 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:03.744 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:03.745 DEBUG:boto:Method: GET 2022-09-03T06:50:03.746 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:50:03.747 DEBUG:boto:Data: 2022-09-03T06:50:03.748 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.749 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:03.750 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.751 DEBUG:boto:Params: {} 2022-09-03T06:50:03.752 DEBUG:boto:Token: None 2022-09-03T06:50:03.755 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/ 2022-09-03T06:50:03.757 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U= 2022-09-03T06:50:03.758 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U=', 'Content-Length': '0'} 2022-09-03T06:50:03.763 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e4d60c55ae330e13-006312f91b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.764 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:03.765 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:03.765 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:03.766 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:03.767 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:03.767 DEBUG:boto:Method: GET 2022-09-03T06:50:03.768 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:50:03.768 DEBUG:boto:Data: 2022-09-03T06:50:03.769 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.769 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.770 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.770 DEBUG:boto:Params: {} 2022-09-03T06:50:03.771 DEBUG:boto:Token: None 2022-09-03T06:50:03.771 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/ 2022-09-03T06:50:03.772 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U= 2022-09-03T06:50:03.773 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U=', 'Content-Length': '0'} 2022-09-03T06:50:03.777 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a521bc8f7f40a68d-006312f91b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.778 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:03.779 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj185 2022-09-03T06:50:03.779 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:03.780 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:03.782 DEBUG:boto:path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.783 DEBUG:boto:auth_path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.783 DEBUG:boto:Method: GET 2022-09-03T06:50:03.784 DEBUG:boto:Path: /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.784 DEBUG:boto:Data: 2022-09-03T06:50:03.785 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.785 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:03.787 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.787 DEBUG:boto:Params: {} 2022-09-03T06:50:03.788 DEBUG:boto:Token: None 2022-09-03T06:50:03.788 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.789 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nukCUzxG18Ua8vtYC/KugDjibao= 2022-09-03T06:50:03.791 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nukCUzxG18Ua8vtYC/KugDjibao=', 'Content-Length': '0'} 2022-09-03T06:50:03.793 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2b4d256afe9d4ff-006312f91b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.794 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:03.795 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:03.795 DEBUG:boto:path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.796 DEBUG:boto:auth_path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.797 DEBUG:boto:Method: GET 2022-09-03T06:50:03.797 DEBUG:boto:Path: /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.798 DEBUG:boto:Data: 2022-09-03T06:50:03.798 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.799 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.799 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.800 DEBUG:boto:Params: {} 2022-09-03T06:50:03.801 DEBUG:boto:Token: None 2022-09-03T06:50:03.801 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.802 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nukCUzxG18Ua8vtYC/KugDjibao= 2022-09-03T06:50:03.803 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nukCUzxG18Ua8vtYC/KugDjibao=', 'Content-Length': '0'} 2022-09-03T06:50:03.806 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000782fe30cbf090c48-006312f91b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.847 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:03.847 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:03.848 DEBUG:boto:path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.849 DEBUG:boto:auth_path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.849 DEBUG:boto:Method: HEAD 2022-09-03T06:50:03.850 DEBUG:boto:Path: /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.850 DEBUG:boto:Data: 2022-09-03T06:50:03.851 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.851 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:03.852 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.852 DEBUG:boto:Params: {} 2022-09-03T06:50:03.853 DEBUG:boto:Token: None 2022-09-03T06:50:03.853 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.854 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3c5oAjb+aOluoU+C7ChE9cupAwE= 2022-09-03T06:50:03.855 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3c5oAjb+aOluoU+C7ChE9cupAwE=', 'Content-Length': '0'} 2022-09-03T06:50:03.857 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000020b6449639c93a35-006312f91b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.858 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:03.859 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:03.859 DEBUG:boto:path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.860 DEBUG:boto:auth_path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.860 DEBUG:boto:Method: HEAD 2022-09-03T06:50:03.861 DEBUG:boto:Path: /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.861 DEBUG:boto:Data: 2022-09-03T06:50:03.862 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.862 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.863 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.863 DEBUG:boto:Params: {} 2022-09-03T06:50:03.864 DEBUG:boto:Token: None 2022-09-03T06:50:03.864 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.865 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3c5oAjb+aOluoU+C7ChE9cupAwE= 2022-09-03T06:50:03.866 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3c5oAjb+aOluoU+C7ChE9cupAwE=', 'Content-Length': '0'} 2022-09-03T06:50:03.868 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000022cbead86822a88-006312f91b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.869 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj185 2022-09-03T06:50:03.869 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:03.870 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:03.870 DEBUG:boto:path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.871 DEBUG:boto:auth_path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.871 DEBUG:boto:Method: GET 2022-09-03T06:50:03.872 DEBUG:boto:Path: /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.873 DEBUG:boto:Data: 2022-09-03T06:50:03.873 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.874 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:03.874 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.875 DEBUG:boto:Params: {} 2022-09-03T06:50:03.883 DEBUG:boto:Token: None 2022-09-03T06:50:03.884 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.885 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nukCUzxG18Ua8vtYC/KugDjibao= 2022-09-03T06:50:03.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nukCUzxG18Ua8vtYC/KugDjibao=', 'Content-Length': '0'} 2022-09-03T06:50:03.890 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000361af2316dba119a-006312f91b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.891 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:03.892 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:03.892 DEBUG:boto:path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.893 DEBUG:boto:auth_path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.893 DEBUG:boto:Method: GET 2022-09-03T06:50:03.894 DEBUG:boto:Path: /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.894 DEBUG:boto:Data: 2022-09-03T06:50:03.895 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.896 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.896 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.897 DEBUG:boto:Params: {} 2022-09-03T06:50:03.897 DEBUG:boto:Token: None 2022-09-03T06:50:03.898 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:03.899 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nukCUzxG18Ua8vtYC/KugDjibao= 2022-09-03T06:50:03.900 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nukCUzxG18Ua8vtYC/KugDjibao=', 'Content-Length': '0'} 2022-09-03T06:50:03.902 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a999dc54c240886-006312f91b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.903 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:03.904 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:03.904 DEBUG:boto:path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:03.905 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:03.905 DEBUG:boto:Method: GET 2022-09-03T06:50:03.906 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj185 2022-09-03T06:50:03.907 DEBUG:boto:Data: 2022-09-03T06:50:03.907 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.908 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:03.908 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.909 DEBUG:boto:Params: {} 2022-09-03T06:50:03.909 DEBUG:boto:Token: None 2022-09-03T06:50:03.910 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/ 2022-09-03T06:50:03.911 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U= 2022-09-03T06:50:03.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U=', 'Content-Length': '0'} 2022-09-03T06:50:03.915 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e8fae1eb3a24500a-006312f91b-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.958 DEBUG:boto:b'dsgvcy-75obj1851000falseobj1852022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:03.959 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:03.960 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:03.961 DEBUG:boto:path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:03.961 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:03.962 DEBUG:boto:Method: GET 2022-09-03T06:50:03.962 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj185 2022-09-03T06:50:03.963 DEBUG:boto:Data: 2022-09-03T06:50:03.964 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.964 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.965 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.966 DEBUG:boto:Params: {} 2022-09-03T06:50:03.967 DEBUG:boto:Token: None 2022-09-03T06:50:03.967 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/ 2022-09-03T06:50:03.968 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U= 2022-09-03T06:50:03.969 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pEci5LOUOXjxc8VyW+pvJRDol1U=', 'Content-Length': '0'} 2022-09-03T06:50:03.974 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000017b2e81366796a18-006312f91b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.975 DEBUG:boto:b'dsgvcy-75obj1851000falseobj1852022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:03.976 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:03.976 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:03.977 DEBUG:boto:Method: HEAD 2022-09-03T06:50:03.978 DEBUG:boto:Path: /dsgvcy-75/obj185 2022-09-03T06:50:03.978 DEBUG:boto:Data: 2022-09-03T06:50:03.979 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.980 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:03.980 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.981 DEBUG:boto:Params: {} 2022-09-03T06:50:03.981 DEBUG:boto:Token: None 2022-09-03T06:50:03.982 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj185 2022-09-03T06:50:03.983 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IMMRF2wlhICZmkguKU5QItRFZsE= 2022-09-03T06:50:03.987 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IMMRF2wlhICZmkguKU5QItRFZsE=', 'Content-Length': '0'} 2022-09-03T06:50:03.989 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e2adb6b06ab69985-006312f91b-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:03 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:03.991 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:03.992 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:03.993 DEBUG:boto:Method: HEAD 2022-09-03T06:50:03.993 DEBUG:boto:Path: /dsgvcy-75/obj185 2022-09-03T06:50:03.994 DEBUG:boto:Data: 2022-09-03T06:50:03.995 DEBUG:boto:Headers: {} 2022-09-03T06:50:03.995 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:03.996 DEBUG:boto:Port: 8000 2022-09-03T06:50:03.996 DEBUG:boto:Params: {} 2022-09-03T06:50:03.997 DEBUG:boto:Token: None 2022-09-03T06:50:03.998 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:03 GMT /dsgvcy-75/obj185 2022-09-03T06:50:03.999 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IMMRF2wlhICZmkguKU5QItRFZsE= 2022-09-03T06:50:04.000 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:03 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IMMRF2wlhICZmkguKU5QItRFZsE=', 'Content-Length': '0'} 2022-09-03T06:50:04.003 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bf30f558d1b64807-006312f91c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.004 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj185 2022-09-03T06:50:04.005 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:04.006 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:04.007 DEBUG:boto:Method: GET 2022-09-03T06:50:04.008 DEBUG:boto:Path: /dsgvcy-75/obj185 2022-09-03T06:50:04.009 DEBUG:boto:Data: 2022-09-03T06:50:04.010 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.011 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:04.012 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.013 DEBUG:boto:Params: {} 2022-09-03T06:50:04.014 DEBUG:boto:Token: None 2022-09-03T06:50:04.015 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj185 2022-09-03T06:50:04.015 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B0PNnuJ7M7mtavKwbIJ6LA+KwUE= 2022-09-03T06:50:04.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B0PNnuJ7M7mtavKwbIJ6LA+KwUE=', 'Content-Length': '0'} 2022-09-03T06:50:04.022 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eba3e521fba465ab-006312f91c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.023 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:04.025 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:04.025 DEBUG:boto:Method: GET 2022-09-03T06:50:04.026 DEBUG:boto:Path: /dsgvcy-75/obj185 2022-09-03T06:50:04.026 DEBUG:boto:Data: 2022-09-03T06:50:04.027 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.028 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:04.028 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.029 DEBUG:boto:Params: {} 2022-09-03T06:50:04.030 DEBUG:boto:Token: None 2022-09-03T06:50:04.030 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj185 2022-09-03T06:50:04.032 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B0PNnuJ7M7mtavKwbIJ6LA+KwUE= 2022-09-03T06:50:04.033 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B0PNnuJ7M7mtavKwbIJ6LA+KwUE=', 'Content-Length': '0'} 2022-09-03T06:50:04.039 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003d19cc13dce3fbc0-006312f91c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.041 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:04.042 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:04.042 DEBUG:boto:path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:04.043 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:04.043 DEBUG:boto:Method: GET 2022-09-03T06:50:04.044 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj185 2022-09-03T06:50:04.045 DEBUG:boto:Data: 2022-09-03T06:50:04.045 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.046 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:04.047 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.047 DEBUG:boto:Params: {} 2022-09-03T06:50:04.048 DEBUG:boto:Token: None 2022-09-03T06:50:04.049 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/ 2022-09-03T06:50:04.049 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mo1kCmUkXme481OHSNCFa9qqQLc= 2022-09-03T06:50:04.050 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mo1kCmUkXme481OHSNCFa9qqQLc=', 'Content-Length': '0'} 2022-09-03T06:50:04.053 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b162dcbb5d8a83d7-006312f91c-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.054 DEBUG:boto:b'dsgvcy-75obj1851000falseobj1852022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:04.055 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:04.055 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:04.056 DEBUG:boto:path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:04.057 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:04.057 DEBUG:boto:Method: GET 2022-09-03T06:50:04.058 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj185 2022-09-03T06:50:04.059 DEBUG:boto:Data: 2022-09-03T06:50:04.059 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.060 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:04.061 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.061 DEBUG:boto:Params: {} 2022-09-03T06:50:04.062 DEBUG:boto:Token: None 2022-09-03T06:50:04.063 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/ 2022-09-03T06:50:04.063 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mo1kCmUkXme481OHSNCFa9qqQLc= 2022-09-03T06:50:04.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mo1kCmUkXme481OHSNCFa9qqQLc=', 'Content-Length': '0'} 2022-09-03T06:50:04.069 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002b8e958d8dbf5360-006312f91c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.110 DEBUG:boto:b'dsgvcy-75obj1851000falseobj1852022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:04.112 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:50:04.113 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:04.114 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:04.115 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.115 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.117 DEBUG:boto:Method: GET 2022-09-03T06:50:04.117 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.119 DEBUG:boto:Data: 2022-09-03T06:50:04.120 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.121 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:04.122 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.122 DEBUG:boto:Params: {} 2022-09-03T06:50:04.123 DEBUG:boto:Token: None 2022-09-03T06:50:04.124 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.124 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TUiU6D3yDv6VUBCK+j9kwbZiIKs= 2022-09-03T06:50:04.125 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TUiU6D3yDv6VUBCK+j9kwbZiIKs=', 'Content-Length': '0'} 2022-09-03T06:50:04.139 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000092b28ec846b04179-006312f91c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.140 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:04.143 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:04.144 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.146 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.147 DEBUG:boto:Method: GET 2022-09-03T06:50:04.148 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.149 DEBUG:boto:Data: 2022-09-03T06:50:04.150 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.151 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:04.152 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.153 DEBUG:boto:Params: {} 2022-09-03T06:50:04.161 DEBUG:boto:Token: None 2022-09-03T06:50:04.162 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.163 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TUiU6D3yDv6VUBCK+j9kwbZiIKs= 2022-09-03T06:50:04.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TUiU6D3yDv6VUBCK+j9kwbZiIKs=', 'Content-Length': '0'} 2022-09-03T06:50:04.168 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d6d44bf73c4108e4-006312f91c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.170 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:04.171 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:04.172 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.173 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.174 DEBUG:boto:Method: HEAD 2022-09-03T06:50:04.175 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.175 DEBUG:boto:Data: 2022-09-03T06:50:04.176 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.177 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:04.178 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.179 DEBUG:boto:Params: {} 2022-09-03T06:50:04.180 DEBUG:boto:Token: None 2022-09-03T06:50:04.181 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.181 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3+SBJmZbeI8Q6ZEMVxdQUoXW/fI= 2022-09-03T06:50:04.182 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3+SBJmZbeI8Q6ZEMVxdQUoXW/fI=', 'Content-Length': '0'} 2022-09-03T06:50:04.186 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a5b3ac44d71e6fc-006312f91c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.187 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:04.188 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:04.189 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.190 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.191 DEBUG:boto:Method: HEAD 2022-09-03T06:50:04.192 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.193 DEBUG:boto:Data: 2022-09-03T06:50:04.194 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.195 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:04.195 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.196 DEBUG:boto:Params: {} 2022-09-03T06:50:04.197 DEBUG:boto:Token: None 2022-09-03T06:50:04.198 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.200 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3+SBJmZbeI8Q6ZEMVxdQUoXW/fI= 2022-09-03T06:50:04.201 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3+SBJmZbeI8Q6ZEMVxdQUoXW/fI=', 'Content-Length': '0'} 2022-09-03T06:50:04.203 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cbf404f153146400-006312f91c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.205 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:50:04.209 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:04.211 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:04.213 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.214 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.215 DEBUG:boto:Method: GET 2022-09-03T06:50:04.215 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.216 DEBUG:boto:Data: 2022-09-03T06:50:04.216 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.217 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:04.218 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.220 DEBUG:boto:Params: {} 2022-09-03T06:50:04.221 DEBUG:boto:Token: None 2022-09-03T06:50:04.223 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.225 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TUiU6D3yDv6VUBCK+j9kwbZiIKs= 2022-09-03T06:50:04.226 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TUiU6D3yDv6VUBCK+j9kwbZiIKs=', 'Content-Length': '0'} 2022-09-03T06:50:04.234 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cfc0a266a0cccab7-006312f91c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.235 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:04.235 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:04.236 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.237 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.238 DEBUG:boto:Method: GET 2022-09-03T06:50:04.240 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.241 DEBUG:boto:Data: 2022-09-03T06:50:04.242 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.243 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:04.244 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.245 DEBUG:boto:Params: {} 2022-09-03T06:50:04.246 DEBUG:boto:Token: None 2022-09-03T06:50:04.247 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:04.249 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TUiU6D3yDv6VUBCK+j9kwbZiIKs= 2022-09-03T06:50:04.250 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TUiU6D3yDv6VUBCK+j9kwbZiIKs=', 'Content-Length': '0'} 2022-09-03T06:50:04.252 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004de00eb4aa4bf426-006312f91c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.253 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:04.254 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:04.255 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:04.255 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:04.256 DEBUG:boto:Method: GET 2022-09-03T06:50:04.256 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:50:04.257 DEBUG:boto:Data: 2022-09-03T06:50:04.258 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.259 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:04.259 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.260 DEBUG:boto:Params: {} 2022-09-03T06:50:04.260 DEBUG:boto:Token: None 2022-09-03T06:50:04.261 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/ 2022-09-03T06:50:04.262 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mo1kCmUkXme481OHSNCFa9qqQLc= 2022-09-03T06:50:04.262 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mo1kCmUkXme481OHSNCFa9qqQLc=', 'Content-Length': '0'} 2022-09-03T06:50:04.265 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000041301363f1d2ceb8-006312f91c-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.310 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:04.312 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:04.313 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:04.314 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:04.315 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:04.315 DEBUG:boto:Method: GET 2022-09-03T06:50:04.316 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:50:04.316 DEBUG:boto:Data: 2022-09-03T06:50:04.317 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.318 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:04.319 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.321 DEBUG:boto:Params: {} 2022-09-03T06:50:04.322 DEBUG:boto:Token: None 2022-09-03T06:50:04.323 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/ 2022-09-03T06:50:04.325 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mo1kCmUkXme481OHSNCFa9qqQLc= 2022-09-03T06:50:04.326 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mo1kCmUkXme481OHSNCFa9qqQLc=', 'Content-Length': '0'} 2022-09-03T06:50:04.331 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000acbe2c30bf9855e5-006312f91c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.332 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:04.334 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:04.334 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:04.335 DEBUG:boto:Method: HEAD 2022-09-03T06:50:04.341 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:50:04.356 DEBUG:boto:Data: 2022-09-03T06:50:04.358 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.365 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:04.366 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.367 DEBUG:boto:Params: {} 2022-09-03T06:50:04.368 DEBUG:boto:Token: None 2022-09-03T06:50:04.369 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj19 2022-09-03T06:50:04.370 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nfiFjdqOesigms1bvmMI4evmgC8= 2022-09-03T06:50:04.373 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nfiFjdqOesigms1bvmMI4evmgC8=', 'Content-Length': '0'} 2022-09-03T06:50:04.376 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003ef5ad03ea8dd308-006312f91c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.377 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:04.377 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:04.378 DEBUG:boto:Method: HEAD 2022-09-03T06:50:04.378 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:50:04.379 DEBUG:boto:Data: 2022-09-03T06:50:04.380 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.380 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:04.381 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.381 DEBUG:boto:Params: {} 2022-09-03T06:50:04.382 DEBUG:boto:Token: None 2022-09-03T06:50:04.383 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj19 2022-09-03T06:50:04.383 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nfiFjdqOesigms1bvmMI4evmgC8= 2022-09-03T06:50:04.384 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nfiFjdqOesigms1bvmMI4evmgC8=', 'Content-Length': '0'} 2022-09-03T06:50:04.387 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000687c5ed4d87bda57-006312f91c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.388 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:50:04.389 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:04.390 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:04.392 DEBUG:boto:Method: GET 2022-09-03T06:50:04.393 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:50:04.394 DEBUG:boto:Data: 2022-09-03T06:50:04.396 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.397 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:04.398 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.398 DEBUG:boto:Params: {} 2022-09-03T06:50:04.399 DEBUG:boto:Token: None 2022-09-03T06:50:04.401 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj19 2022-09-03T06:50:04.402 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FcrgKDhF6vfdHyxt7kqzxn2IrvM= 2022-09-03T06:50:04.402 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FcrgKDhF6vfdHyxt7kqzxn2IrvM=', 'Content-Length': '0'} 2022-09-03T06:50:04.408 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d674eecfd40c960-006312f91c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.409 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:04.410 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:04.411 DEBUG:boto:Method: GET 2022-09-03T06:50:04.411 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:50:04.412 DEBUG:boto:Data: 2022-09-03T06:50:04.412 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.413 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:04.414 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.414 DEBUG:boto:Params: {} 2022-09-03T06:50:04.415 DEBUG:boto:Token: None 2022-09-03T06:50:04.416 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj19 2022-09-03T06:50:04.416 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FcrgKDhF6vfdHyxt7kqzxn2IrvM= 2022-09-03T06:50:04.417 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FcrgKDhF6vfdHyxt7kqzxn2IrvM=', 'Content-Length': '0'} 2022-09-03T06:50:04.428 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000383e27669021941d-006312f91c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.430 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:04.431 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:04.432 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:04.433 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:04.435 DEBUG:boto:Method: GET 2022-09-03T06:50:04.436 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:50:04.437 DEBUG:boto:Data: 2022-09-03T06:50:04.437 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.438 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:04.440 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.441 DEBUG:boto:Params: {} 2022-09-03T06:50:04.442 DEBUG:boto:Token: None 2022-09-03T06:50:04.443 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/ 2022-09-03T06:50:04.444 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mo1kCmUkXme481OHSNCFa9qqQLc= 2022-09-03T06:50:04.445 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mo1kCmUkXme481OHSNCFa9qqQLc=', 'Content-Length': '0'} 2022-09-03T06:50:04.451 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000063319a358e41a1f4-006312f91c-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.452 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:04.457 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:04.458 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:04.459 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:04.459 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:04.460 DEBUG:boto:Method: GET 2022-09-03T06:50:04.461 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:50:04.461 DEBUG:boto:Data: 2022-09-03T06:50:04.462 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.462 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:04.463 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.463 DEBUG:boto:Params: {} 2022-09-03T06:50:04.465 DEBUG:boto:Token: None 2022-09-03T06:50:04.467 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/ 2022-09-03T06:50:04.469 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mo1kCmUkXme481OHSNCFa9qqQLc= 2022-09-03T06:50:04.470 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mo1kCmUkXme481OHSNCFa9qqQLc=', 'Content-Length': '0'} 2022-09-03T06:50:04.481 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bc2472e389566e6f-006312f91c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.482 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:04.484 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:50:04.484 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:04.486 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:04.489 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.490 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.491 DEBUG:boto:Method: GET 2022-09-03T06:50:04.492 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.493 DEBUG:boto:Data: 2022-09-03T06:50:04.494 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.495 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:04.496 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.497 DEBUG:boto:Params: {} 2022-09-03T06:50:04.498 DEBUG:boto:Token: None 2022-09-03T06:50:04.499 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.500 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PlEJ0tvCsggitJ8RWydvkqlTBcQ= 2022-09-03T06:50:04.501 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PlEJ0tvCsggitJ8RWydvkqlTBcQ=', 'Content-Length': '0'} 2022-09-03T06:50:04.504 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aadf2d8998d342b4-006312f91c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.505 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:04.507 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:04.508 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.508 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.509 DEBUG:boto:Method: GET 2022-09-03T06:50:04.509 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.510 DEBUG:boto:Data: 2022-09-03T06:50:04.511 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.511 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:04.512 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.512 DEBUG:boto:Params: {} 2022-09-03T06:50:04.513 DEBUG:boto:Token: None 2022-09-03T06:50:04.513 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.514 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PlEJ0tvCsggitJ8RWydvkqlTBcQ= 2022-09-03T06:50:04.515 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PlEJ0tvCsggitJ8RWydvkqlTBcQ=', 'Content-Length': '0'} 2022-09-03T06:50:04.517 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dae7656fafcf1367-006312f91c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.563 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:04.564 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:04.565 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.565 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.566 DEBUG:boto:Method: HEAD 2022-09-03T06:50:04.566 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.567 DEBUG:boto:Data: 2022-09-03T06:50:04.567 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.568 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:04.568 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.569 DEBUG:boto:Params: {} 2022-09-03T06:50:04.569 DEBUG:boto:Token: None 2022-09-03T06:50:04.570 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.571 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IRsJJ/WMmSM5InohdHs7mg7KNc8= 2022-09-03T06:50:04.572 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IRsJJ/WMmSM5InohdHs7mg7KNc8=', 'Content-Length': '0'} 2022-09-03T06:50:04.575 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000013182dd36470cedf-006312f91c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.578 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:04.579 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:04.582 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.584 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.585 DEBUG:boto:Method: HEAD 2022-09-03T06:50:04.585 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.586 DEBUG:boto:Data: 2022-09-03T06:50:04.586 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.587 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:04.588 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.588 DEBUG:boto:Params: {} 2022-09-03T06:50:04.589 DEBUG:boto:Token: None 2022-09-03T06:50:04.589 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.590 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IRsJJ/WMmSM5InohdHs7mg7KNc8= 2022-09-03T06:50:04.591 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IRsJJ/WMmSM5InohdHs7mg7KNc8=', 'Content-Length': '0'} 2022-09-03T06:50:04.593 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d17a7eb1bc411cd0-006312f91c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.593 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:50:04.594 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:04.595 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:04.595 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.596 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.596 DEBUG:boto:Method: GET 2022-09-03T06:50:04.597 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.598 DEBUG:boto:Data: 2022-09-03T06:50:04.598 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.599 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:04.603 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.605 DEBUG:boto:Params: {} 2022-09-03T06:50:04.606 DEBUG:boto:Token: None 2022-09-03T06:50:04.607 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.608 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PlEJ0tvCsggitJ8RWydvkqlTBcQ= 2022-09-03T06:50:04.609 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PlEJ0tvCsggitJ8RWydvkqlTBcQ=', 'Content-Length': '0'} 2022-09-03T06:50:04.612 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000089982f3026258c8d-006312f91c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.614 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:04.615 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:04.616 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.617 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.618 DEBUG:boto:Method: GET 2022-09-03T06:50:04.619 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.620 DEBUG:boto:Data: 2022-09-03T06:50:04.621 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.623 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:04.624 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.626 DEBUG:boto:Params: {} 2022-09-03T06:50:04.628 DEBUG:boto:Token: None 2022-09-03T06:50:04.629 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:04.630 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PlEJ0tvCsggitJ8RWydvkqlTBcQ= 2022-09-03T06:50:04.631 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PlEJ0tvCsggitJ8RWydvkqlTBcQ=', 'Content-Length': '0'} 2022-09-03T06:50:04.642 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d50d73d1f82b5bba-006312f91c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.644 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:04.645 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:04.646 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:04.647 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:04.648 DEBUG:boto:Method: GET 2022-09-03T06:50:04.650 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:50:04.651 DEBUG:boto:Data: 2022-09-03T06:50:04.652 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.653 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:04.655 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.656 DEBUG:boto:Params: {} 2022-09-03T06:50:04.657 DEBUG:boto:Token: None 2022-09-03T06:50:04.659 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/ 2022-09-03T06:50:04.660 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mo1kCmUkXme481OHSNCFa9qqQLc= 2022-09-03T06:50:04.661 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mo1kCmUkXme481OHSNCFa9qqQLc=', 'Content-Length': '0'} 2022-09-03T06:50:04.666 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005b378013c496a5f1-006312f91c-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.670 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:04.671 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:04.672 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:04.672 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:04.673 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:04.673 DEBUG:boto:Method: GET 2022-09-03T06:50:04.674 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:50:04.674 DEBUG:boto:Data: 2022-09-03T06:50:04.675 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.676 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:04.676 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.677 DEBUG:boto:Params: {} 2022-09-03T06:50:04.677 DEBUG:boto:Token: None 2022-09-03T06:50:04.678 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/ 2022-09-03T06:50:04.679 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mo1kCmUkXme481OHSNCFa9qqQLc= 2022-09-03T06:50:04.679 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mo1kCmUkXme481OHSNCFa9qqQLc=', 'Content-Length': '0'} 2022-09-03T06:50:04.684 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005d41c9909f2add2b-006312f91c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.689 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:04.692 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:04.693 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:04.694 DEBUG:boto:Method: HEAD 2022-09-03T06:50:04.695 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:50:04.696 DEBUG:boto:Data: 2022-09-03T06:50:04.696 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.697 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:04.698 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.699 DEBUG:boto:Params: {} 2022-09-03T06:50:04.700 DEBUG:boto:Token: None 2022-09-03T06:50:04.701 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj22 2022-09-03T06:50:04.701 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z2W2FEzGOae7XWEM1v8r9EVk8ko= 2022-09-03T06:50:04.702 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z2W2FEzGOae7XWEM1v8r9EVk8ko=', 'Content-Length': '0'} 2022-09-03T06:50:04.705 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ffb7c9d4f76c2f01-006312f91c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.706 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:04.707 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:04.707 DEBUG:boto:Method: HEAD 2022-09-03T06:50:04.708 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:50:04.709 DEBUG:boto:Data: 2022-09-03T06:50:04.711 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.712 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:04.712 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.713 DEBUG:boto:Params: {} 2022-09-03T06:50:04.713 DEBUG:boto:Token: None 2022-09-03T06:50:04.714 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj22 2022-09-03T06:50:04.715 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z2W2FEzGOae7XWEM1v8r9EVk8ko= 2022-09-03T06:50:04.715 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z2W2FEzGOae7XWEM1v8r9EVk8ko=', 'Content-Length': '0'} 2022-09-03T06:50:04.719 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000090a0ca8e162396f0-006312f91c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.720 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:50:04.721 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:04.723 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:04.724 DEBUG:boto:Method: GET 2022-09-03T06:50:04.725 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:50:04.726 DEBUG:boto:Data: 2022-09-03T06:50:04.728 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.728 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:04.729 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.733 DEBUG:boto:Params: {} 2022-09-03T06:50:04.734 DEBUG:boto:Token: None 2022-09-03T06:50:04.735 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj22 2022-09-03T06:50:04.737 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rkTWAEEmngng6VTLVrUmkPFPirU= 2022-09-03T06:50:04.739 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rkTWAEEmngng6VTLVrUmkPFPirU=', 'Content-Length': '0'} 2022-09-03T06:50:04.745 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ed6fc9c1d6925dc3-006312f91c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.746 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:04.747 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:04.748 DEBUG:boto:Method: GET 2022-09-03T06:50:04.749 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:50:04.750 DEBUG:boto:Data: 2022-09-03T06:50:04.751 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.753 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:04.754 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.755 DEBUG:boto:Params: {} 2022-09-03T06:50:04.756 DEBUG:boto:Token: None 2022-09-03T06:50:04.756 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj22 2022-09-03T06:50:04.757 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rkTWAEEmngng6VTLVrUmkPFPirU= 2022-09-03T06:50:04.758 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rkTWAEEmngng6VTLVrUmkPFPirU=', 'Content-Length': '0'} 2022-09-03T06:50:04.760 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f194efd10bcffbd-006312f91c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.761 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:04.761 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:04.762 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:04.762 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:04.763 DEBUG:boto:Method: GET 2022-09-03T06:50:04.763 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:50:04.764 DEBUG:boto:Data: 2022-09-03T06:50:04.764 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.765 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:04.765 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.766 DEBUG:boto:Params: {} 2022-09-03T06:50:04.767 DEBUG:boto:Token: None 2022-09-03T06:50:04.767 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/ 2022-09-03T06:50:04.768 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mo1kCmUkXme481OHSNCFa9qqQLc= 2022-09-03T06:50:04.769 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mo1kCmUkXme481OHSNCFa9qqQLc=', 'Content-Length': '0'} 2022-09-03T06:50:04.771 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002f15487d803fe05e-006312f91c-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.818 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:04.820 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:04.821 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:04.822 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:04.823 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:04.824 DEBUG:boto:Method: GET 2022-09-03T06:50:04.825 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:50:04.826 DEBUG:boto:Data: 2022-09-03T06:50:04.829 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.830 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:04.831 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.832 DEBUG:boto:Params: {} 2022-09-03T06:50:04.833 DEBUG:boto:Token: None 2022-09-03T06:50:04.835 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/ 2022-09-03T06:50:04.836 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mo1kCmUkXme481OHSNCFa9qqQLc= 2022-09-03T06:50:04.837 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mo1kCmUkXme481OHSNCFa9qqQLc=', 'Content-Length': '0'} 2022-09-03T06:50:04.843 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000075df17643b76f1b3-006312f91c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.844 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:04.845 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:50:04.846 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:04.846 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:04.847 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.847 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.848 DEBUG:boto:Method: GET 2022-09-03T06:50:04.849 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.849 DEBUG:boto:Data: 2022-09-03T06:50:04.850 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.851 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:04.852 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.852 DEBUG:boto:Params: {} 2022-09-03T06:50:04.853 DEBUG:boto:Token: None 2022-09-03T06:50:04.855 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.856 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I5d/jyd7Pg2do+PW6yFrphQxv9c= 2022-09-03T06:50:04.857 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I5d/jyd7Pg2do+PW6yFrphQxv9c=', 'Content-Length': '0'} 2022-09-03T06:50:04.859 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008bd450d210bd2dac-006312f91c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.861 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:04.862 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:04.863 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.863 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.864 DEBUG:boto:Method: GET 2022-09-03T06:50:04.865 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.866 DEBUG:boto:Data: 2022-09-03T06:50:04.868 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.869 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:04.870 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.871 DEBUG:boto:Params: {} 2022-09-03T06:50:04.872 DEBUG:boto:Token: None 2022-09-03T06:50:04.875 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.877 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I5d/jyd7Pg2do+PW6yFrphQxv9c= 2022-09-03T06:50:04.878 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I5d/jyd7Pg2do+PW6yFrphQxv9c=', 'Content-Length': '0'} 2022-09-03T06:50:04.881 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000883374d9b3aef48-006312f91c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.922 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:04.923 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:04.923 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.924 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.924 DEBUG:boto:Method: HEAD 2022-09-03T06:50:04.925 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.925 DEBUG:boto:Data: 2022-09-03T06:50:04.926 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.927 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:04.927 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.928 DEBUG:boto:Params: {} 2022-09-03T06:50:04.929 DEBUG:boto:Token: None 2022-09-03T06:50:04.930 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.931 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lsR2oY4SxBFDbwFDJ8G3PuqOUpg= 2022-09-03T06:50:04.931 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lsR2oY4SxBFDbwFDJ8G3PuqOUpg=', 'Content-Length': '0'} 2022-09-03T06:50:04.933 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a10911fe30d6dfca-006312f91c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.934 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:04.935 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:04.935 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.936 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.940 DEBUG:boto:Method: HEAD 2022-09-03T06:50:04.942 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.943 DEBUG:boto:Data: 2022-09-03T06:50:04.944 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.945 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:04.947 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.948 DEBUG:boto:Params: {} 2022-09-03T06:50:04.949 DEBUG:boto:Token: None 2022-09-03T06:50:04.950 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.951 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lsR2oY4SxBFDbwFDJ8G3PuqOUpg= 2022-09-03T06:50:04.952 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lsR2oY4SxBFDbwFDJ8G3PuqOUpg=', 'Content-Length': '0'} 2022-09-03T06:50:04.955 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000010d9be2ecb435985-006312f91c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.958 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:50:04.959 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:04.960 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:04.960 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.961 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.962 DEBUG:boto:Method: GET 2022-09-03T06:50:04.962 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.963 DEBUG:boto:Data: 2022-09-03T06:50:04.963 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.964 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:04.965 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.965 DEBUG:boto:Params: {} 2022-09-03T06:50:04.966 DEBUG:boto:Token: None 2022-09-03T06:50:04.967 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.968 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I5d/jyd7Pg2do+PW6yFrphQxv9c= 2022-09-03T06:50:04.968 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I5d/jyd7Pg2do+PW6yFrphQxv9c=', 'Content-Length': '0'} 2022-09-03T06:50:04.973 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d677b810c1cf5464-006312f91c-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:04.977 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:04.979 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:04.980 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.980 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.981 DEBUG:boto:Method: GET 2022-09-03T06:50:04.982 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.983 DEBUG:boto:Data: 2022-09-03T06:50:04.984 DEBUG:boto:Headers: {} 2022-09-03T06:50:04.985 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:04.986 DEBUG:boto:Port: 8000 2022-09-03T06:50:04.988 DEBUG:boto:Params: {} 2022-09-03T06:50:04.989 DEBUG:boto:Token: None 2022-09-03T06:50:04.992 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:04 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:04.993 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I5d/jyd7Pg2do+PW6yFrphQxv9c= 2022-09-03T06:50:04.994 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:04 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I5d/jyd7Pg2do+PW6yFrphQxv9c=', 'Content-Length': '0'} 2022-09-03T06:50:04.997 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004453020bba0bc680-006312f91c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:04 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.000 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:05.001 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:05.002 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:05.003 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:05.004 DEBUG:boto:Method: GET 2022-09-03T06:50:05.006 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:50:05.007 DEBUG:boto:Data: 2022-09-03T06:50:05.009 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.011 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:05.012 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.015 DEBUG:boto:Params: {} 2022-09-03T06:50:05.016 DEBUG:boto:Token: None 2022-09-03T06:50:05.017 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/ 2022-09-03T06:50:05.019 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI= 2022-09-03T06:50:05.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI=', 'Content-Length': '0'} 2022-09-03T06:50:05.023 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eafe89c72e186bb3-006312f91d-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.025 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:05.027 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:05.028 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:05.029 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:05.029 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:05.030 DEBUG:boto:Method: GET 2022-09-03T06:50:05.031 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:50:05.032 DEBUG:boto:Data: 2022-09-03T06:50:05.033 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.034 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:05.035 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.036 DEBUG:boto:Params: {} 2022-09-03T06:50:05.037 DEBUG:boto:Token: None 2022-09-03T06:50:05.038 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/ 2022-09-03T06:50:05.039 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI= 2022-09-03T06:50:05.041 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI=', 'Content-Length': '0'} 2022-09-03T06:50:05.046 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005f5b708adb071196-006312f91d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.047 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:05.048 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:05.049 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:05.049 DEBUG:boto:Method: HEAD 2022-09-03T06:50:05.050 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:50:05.050 DEBUG:boto:Data: 2022-09-03T06:50:05.051 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.051 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:05.052 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.052 DEBUG:boto:Params: {} 2022-09-03T06:50:05.053 DEBUG:boto:Token: None 2022-09-03T06:50:05.053 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj23 2022-09-03T06:50:05.054 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xBc5KoN8mkNSk9swh9+hyecypAs= 2022-09-03T06:50:05.055 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xBc5KoN8mkNSk9swh9+hyecypAs=', 'Content-Length': '0'} 2022-09-03T06:50:05.057 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009eb41f838c34167d-006312f91d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.058 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:05.058 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:05.059 DEBUG:boto:Method: HEAD 2022-09-03T06:50:05.059 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:50:05.060 DEBUG:boto:Data: 2022-09-03T06:50:05.060 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.061 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:05.061 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.062 DEBUG:boto:Params: {} 2022-09-03T06:50:05.062 DEBUG:boto:Token: None 2022-09-03T06:50:05.063 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj23 2022-09-03T06:50:05.064 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xBc5KoN8mkNSk9swh9+hyecypAs= 2022-09-03T06:50:05.064 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xBc5KoN8mkNSk9swh9+hyecypAs=', 'Content-Length': '0'} 2022-09-03T06:50:05.077 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000050c530aa40e8aa3b-006312f91d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.081 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:50:05.082 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:05.083 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:05.085 DEBUG:boto:Method: GET 2022-09-03T06:50:05.087 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:50:05.088 DEBUG:boto:Data: 2022-09-03T06:50:05.090 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.091 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:05.091 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.092 DEBUG:boto:Params: {} 2022-09-03T06:50:05.093 DEBUG:boto:Token: None 2022-09-03T06:50:05.093 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj23 2022-09-03T06:50:05.094 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HRQpK5xdKsO1pF98WLxgDsrsirw= 2022-09-03T06:50:05.095 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HRQpK5xdKsO1pF98WLxgDsrsirw=', 'Content-Length': '0'} 2022-09-03T06:50:05.097 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ddfed4ec12a20610-006312f91d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.098 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:05.099 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:05.100 DEBUG:boto:Method: GET 2022-09-03T06:50:05.101 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:50:05.102 DEBUG:boto:Data: 2022-09-03T06:50:05.102 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.103 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:05.104 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.104 DEBUG:boto:Params: {} 2022-09-03T06:50:05.105 DEBUG:boto:Token: None 2022-09-03T06:50:05.106 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj23 2022-09-03T06:50:05.106 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:HRQpK5xdKsO1pF98WLxgDsrsirw= 2022-09-03T06:50:05.107 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:HRQpK5xdKsO1pF98WLxgDsrsirw=', 'Content-Length': '0'} 2022-09-03T06:50:05.111 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e85c378d5b18f63b-006312f91d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.113 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:05.114 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:05.115 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:05.116 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:05.116 DEBUG:boto:Method: GET 2022-09-03T06:50:05.117 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:50:05.117 DEBUG:boto:Data: 2022-09-03T06:50:05.118 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.119 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:05.120 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.121 DEBUG:boto:Params: {} 2022-09-03T06:50:05.123 DEBUG:boto:Token: None 2022-09-03T06:50:05.124 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/ 2022-09-03T06:50:05.125 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI= 2022-09-03T06:50:05.127 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI=', 'Content-Length': '0'} 2022-09-03T06:50:05.131 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007ccc41329869b9ff-006312f91d-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.132 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:05.134 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:05.136 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:05.137 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:05.138 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:05.139 DEBUG:boto:Method: GET 2022-09-03T06:50:05.140 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:50:05.141 DEBUG:boto:Data: 2022-09-03T06:50:05.142 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.144 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:05.145 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.145 DEBUG:boto:Params: {} 2022-09-03T06:50:05.146 DEBUG:boto:Token: None 2022-09-03T06:50:05.149 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/ 2022-09-03T06:50:05.149 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI= 2022-09-03T06:50:05.150 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI=', 'Content-Length': '0'} 2022-09-03T06:50:05.156 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001c4ad3c8f80171c5-006312f91d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.157 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:05.158 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:50:05.159 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:05.160 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:05.161 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.162 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.163 DEBUG:boto:Method: GET 2022-09-03T06:50:05.164 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.165 DEBUG:boto:Data: 2022-09-03T06:50:05.166 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.167 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:05.168 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.169 DEBUG:boto:Params: {} 2022-09-03T06:50:05.170 DEBUG:boto:Token: None 2022-09-03T06:50:05.171 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.172 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:upo4um7QNxLyOvX6zJqw+QT9pio= 2022-09-03T06:50:05.173 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:upo4um7QNxLyOvX6zJqw+QT9pio=', 'Content-Length': '0'} 2022-09-03T06:50:05.175 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c019e18ec5ea7439-006312f91d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.177 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:05.178 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:05.179 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.180 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.181 DEBUG:boto:Method: GET 2022-09-03T06:50:05.182 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.183 DEBUG:boto:Data: 2022-09-03T06:50:05.184 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.185 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:05.186 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.187 DEBUG:boto:Params: {} 2022-09-03T06:50:05.189 DEBUG:boto:Token: None 2022-09-03T06:50:05.190 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.191 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:upo4um7QNxLyOvX6zJqw+QT9pio= 2022-09-03T06:50:05.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:upo4um7QNxLyOvX6zJqw+QT9pio=', 'Content-Length': '0'} 2022-09-03T06:50:05.196 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2ed0c9f85ff0feb-006312f91d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.238 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:05.239 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:05.240 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.241 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.241 DEBUG:boto:Method: HEAD 2022-09-03T06:50:05.242 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.243 DEBUG:boto:Data: 2022-09-03T06:50:05.244 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.245 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:05.245 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.246 DEBUG:boto:Params: {} 2022-09-03T06:50:05.247 DEBUG:boto:Token: None 2022-09-03T06:50:05.248 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.250 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xQJAol5qLY/7PNvvoFbxN2PhoUE= 2022-09-03T06:50:05.254 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xQJAol5qLY/7PNvvoFbxN2PhoUE=', 'Content-Length': '0'} 2022-09-03T06:50:05.256 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000024fea88e81de4def-006312f91d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.257 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:05.258 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:05.258 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.259 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.260 DEBUG:boto:Method: HEAD 2022-09-03T06:50:05.260 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.261 DEBUG:boto:Data: 2022-09-03T06:50:05.263 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.263 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:05.266 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.267 DEBUG:boto:Params: {} 2022-09-03T06:50:05.267 DEBUG:boto:Token: None 2022-09-03T06:50:05.268 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.268 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xQJAol5qLY/7PNvvoFbxN2PhoUE= 2022-09-03T06:50:05.269 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xQJAol5qLY/7PNvvoFbxN2PhoUE=', 'Content-Length': '0'} 2022-09-03T06:50:05.272 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000054999d110fc87749-006312f91d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.272 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:50:05.273 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:05.273 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:05.274 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.275 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.275 DEBUG:boto:Method: GET 2022-09-03T06:50:05.276 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.277 DEBUG:boto:Data: 2022-09-03T06:50:05.277 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.278 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:05.278 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.283 DEBUG:boto:Params: {} 2022-09-03T06:50:05.286 DEBUG:boto:Token: None 2022-09-03T06:50:05.287 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.288 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:upo4um7QNxLyOvX6zJqw+QT9pio= 2022-09-03T06:50:05.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:upo4um7QNxLyOvX6zJqw+QT9pio=', 'Content-Length': '0'} 2022-09-03T06:50:05.291 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee26f72dc7c3c3b7-006312f91d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.295 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:05.296 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:05.297 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.297 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.298 DEBUG:boto:Method: GET 2022-09-03T06:50:05.298 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.299 DEBUG:boto:Data: 2022-09-03T06:50:05.299 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.300 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:05.301 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.301 DEBUG:boto:Params: {} 2022-09-03T06:50:05.302 DEBUG:boto:Token: None 2022-09-03T06:50:05.303 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:05.304 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:upo4um7QNxLyOvX6zJqw+QT9pio= 2022-09-03T06:50:05.304 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:upo4um7QNxLyOvX6zJqw+QT9pio=', 'Content-Length': '0'} 2022-09-03T06:50:05.307 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000841e569c37193027-006312f91d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.308 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:05.308 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:05.309 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:05.310 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:05.310 DEBUG:boto:Method: GET 2022-09-03T06:50:05.311 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:50:05.312 DEBUG:boto:Data: 2022-09-03T06:50:05.312 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.313 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:05.313 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.314 DEBUG:boto:Params: {} 2022-09-03T06:50:05.315 DEBUG:boto:Token: None 2022-09-03T06:50:05.316 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/ 2022-09-03T06:50:05.317 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI= 2022-09-03T06:50:05.318 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI=', 'Content-Length': '0'} 2022-09-03T06:50:05.324 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dd6fba9d2ecdfda7-006312f91d-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.362 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:05.364 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:05.365 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:05.366 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:05.370 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:05.371 DEBUG:boto:Method: GET 2022-09-03T06:50:05.372 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:50:05.372 DEBUG:boto:Data: 2022-09-03T06:50:05.373 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.374 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:05.374 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.375 DEBUG:boto:Params: {} 2022-09-03T06:50:05.376 DEBUG:boto:Token: None 2022-09-03T06:50:05.377 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/ 2022-09-03T06:50:05.377 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI= 2022-09-03T06:50:05.378 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI=', 'Content-Length': '0'} 2022-09-03T06:50:05.385 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001ed99fd9c7b5244f-006312f91d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.386 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:05.387 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:05.388 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:05.389 DEBUG:boto:Method: HEAD 2022-09-03T06:50:05.390 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:50:05.391 DEBUG:boto:Data: 2022-09-03T06:50:05.392 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.393 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:05.395 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.395 DEBUG:boto:Params: {} 2022-09-03T06:50:05.396 DEBUG:boto:Token: None 2022-09-03T06:50:05.397 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj29 2022-09-03T06:50:05.398 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R/BhtrCTP5exm0WfMiq6DD8Mt8o= 2022-09-03T06:50:05.399 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R/BhtrCTP5exm0WfMiq6DD8Mt8o=', 'Content-Length': '0'} 2022-09-03T06:50:05.401 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000015f1332744e65813-006312f91d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.407 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:05.408 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:05.409 DEBUG:boto:Method: HEAD 2022-09-03T06:50:05.409 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:50:05.410 DEBUG:boto:Data: 2022-09-03T06:50:05.410 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.411 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:05.411 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.412 DEBUG:boto:Params: {} 2022-09-03T06:50:05.412 DEBUG:boto:Token: None 2022-09-03T06:50:05.413 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj29 2022-09-03T06:50:05.413 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R/BhtrCTP5exm0WfMiq6DD8Mt8o= 2022-09-03T06:50:05.414 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R/BhtrCTP5exm0WfMiq6DD8Mt8o=', 'Content-Length': '0'} 2022-09-03T06:50:05.417 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001da825ce1df3f943-006312f91d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.418 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:50:05.418 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:05.419 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:05.420 DEBUG:boto:Method: GET 2022-09-03T06:50:05.421 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:50:05.422 DEBUG:boto:Data: 2022-09-03T06:50:05.422 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.423 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:05.424 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.424 DEBUG:boto:Params: {} 2022-09-03T06:50:05.425 DEBUG:boto:Token: None 2022-09-03T06:50:05.426 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj29 2022-09-03T06:50:05.426 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mb9uAHT+1NVMJd24MBJKqXdg32o= 2022-09-03T06:50:05.427 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mb9uAHT+1NVMJd24MBJKqXdg32o=', 'Content-Length': '0'} 2022-09-03T06:50:05.429 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008f26bc603399f220-006312f91d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.430 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:05.430 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:05.431 DEBUG:boto:Method: GET 2022-09-03T06:50:05.432 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:50:05.432 DEBUG:boto:Data: 2022-09-03T06:50:05.433 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.433 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:05.434 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.434 DEBUG:boto:Params: {} 2022-09-03T06:50:05.435 DEBUG:boto:Token: None 2022-09-03T06:50:05.436 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj29 2022-09-03T06:50:05.436 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Mb9uAHT+1NVMJd24MBJKqXdg32o= 2022-09-03T06:50:05.437 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Mb9uAHT+1NVMJd24MBJKqXdg32o=', 'Content-Length': '0'} 2022-09-03T06:50:05.439 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000015901416f5edeb6a-006312f91d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.440 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:05.441 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:05.441 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:05.442 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:05.443 DEBUG:boto:Method: GET 2022-09-03T06:50:05.443 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:50:05.444 DEBUG:boto:Data: 2022-09-03T06:50:05.445 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.445 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:05.446 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.446 DEBUG:boto:Params: {} 2022-09-03T06:50:05.447 DEBUG:boto:Token: None 2022-09-03T06:50:05.448 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/ 2022-09-03T06:50:05.449 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI= 2022-09-03T06:50:05.449 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI=', 'Content-Length': '0'} 2022-09-03T06:50:05.455 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008782bf24939d4497-006312f91d-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.456 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:05.456 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:05.457 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:05.458 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:05.458 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:05.459 DEBUG:boto:Method: GET 2022-09-03T06:50:05.459 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:50:05.460 DEBUG:boto:Data: 2022-09-03T06:50:05.461 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.461 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:05.462 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.462 DEBUG:boto:Params: {} 2022-09-03T06:50:05.463 DEBUG:boto:Token: None 2022-09-03T06:50:05.463 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/ 2022-09-03T06:50:05.464 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI= 2022-09-03T06:50:05.465 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI=', 'Content-Length': '0'} 2022-09-03T06:50:05.470 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cbcd7e0610e6e145-006312f91d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.471 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:05.473 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2022-09-03T06:50:05.474 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:05.475 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:05.476 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.477 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.478 DEBUG:boto:Method: GET 2022-09-03T06:50:05.480 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.481 DEBUG:boto:Data: 2022-09-03T06:50:05.482 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.483 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:05.483 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.484 DEBUG:boto:Params: {} 2022-09-03T06:50:05.484 DEBUG:boto:Token: None 2022-09-03T06:50:05.489 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.491 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S3cCfYPPuGmYc8g9+OUO2z5KfC8= 2022-09-03T06:50:05.493 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S3cCfYPPuGmYc8g9+OUO2z5KfC8=', 'Content-Length': '0'} 2022-09-03T06:50:05.495 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000946a2fffbd4443c7-006312f91d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.496 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:05.497 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:05.498 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.498 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.499 DEBUG:boto:Method: GET 2022-09-03T06:50:05.500 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.500 DEBUG:boto:Data: 2022-09-03T06:50:05.501 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.502 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:05.502 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.503 DEBUG:boto:Params: {} 2022-09-03T06:50:05.503 DEBUG:boto:Token: None 2022-09-03T06:50:05.504 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.505 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S3cCfYPPuGmYc8g9+OUO2z5KfC8= 2022-09-03T06:50:05.505 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S3cCfYPPuGmYc8g9+OUO2z5KfC8=', 'Content-Length': '0'} 2022-09-03T06:50:05.508 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000688f15947168edeb-006312f91d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.550 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:05.551 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:05.552 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.553 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.554 DEBUG:boto:Method: HEAD 2022-09-03T06:50:05.555 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.558 DEBUG:boto:Data: 2022-09-03T06:50:05.559 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.561 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:05.562 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.564 DEBUG:boto:Params: {} 2022-09-03T06:50:05.567 DEBUG:boto:Token: None 2022-09-03T06:50:05.569 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.570 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wwu4vQQC/G92pg0bItIoZ5ZBcow= 2022-09-03T06:50:05.570 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wwu4vQQC/G92pg0bItIoZ5ZBcow=', 'Content-Length': '0'} 2022-09-03T06:50:05.573 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ec57393b7383c242-006312f91d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.573 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:05.574 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:05.575 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.575 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.576 DEBUG:boto:Method: HEAD 2022-09-03T06:50:05.576 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.577 DEBUG:boto:Data: 2022-09-03T06:50:05.577 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.578 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:05.578 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.579 DEBUG:boto:Params: {} 2022-09-03T06:50:05.579 DEBUG:boto:Token: None 2022-09-03T06:50:05.580 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.581 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wwu4vQQC/G92pg0bItIoZ5ZBcow= 2022-09-03T06:50:05.581 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wwu4vQQC/G92pg0bItIoZ5ZBcow=', 'Content-Length': '0'} 2022-09-03T06:50:05.584 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ce3ef1df53faad26-006312f91d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.584 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2022-09-03T06:50:05.585 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:05.586 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:05.586 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.587 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.587 DEBUG:boto:Method: GET 2022-09-03T06:50:05.588 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.589 DEBUG:boto:Data: 2022-09-03T06:50:05.590 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.590 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:05.591 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.592 DEBUG:boto:Params: {} 2022-09-03T06:50:05.592 DEBUG:boto:Token: None 2022-09-03T06:50:05.594 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.595 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S3cCfYPPuGmYc8g9+OUO2z5KfC8= 2022-09-03T06:50:05.599 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S3cCfYPPuGmYc8g9+OUO2z5KfC8=', 'Content-Length': '0'} 2022-09-03T06:50:05.606 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002c24f0b329d9dd51-006312f91d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.607 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:05.607 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:05.608 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.609 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.609 DEBUG:boto:Method: GET 2022-09-03T06:50:05.610 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.611 DEBUG:boto:Data: 2022-09-03T06:50:05.611 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.612 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:05.612 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.613 DEBUG:boto:Params: {} 2022-09-03T06:50:05.613 DEBUG:boto:Token: None 2022-09-03T06:50:05.614 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:05.615 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S3cCfYPPuGmYc8g9+OUO2z5KfC8= 2022-09-03T06:50:05.615 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S3cCfYPPuGmYc8g9+OUO2z5KfC8=', 'Content-Length': '0'} 2022-09-03T06:50:05.618 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b593eb44ae291863-006312f91d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.620 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:05.621 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:05.621 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:05.622 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:05.623 DEBUG:boto:Method: GET 2022-09-03T06:50:05.624 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:50:05.625 DEBUG:boto:Data: 2022-09-03T06:50:05.625 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.628 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:05.629 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.630 DEBUG:boto:Params: {} 2022-09-03T06:50:05.630 DEBUG:boto:Token: None 2022-09-03T06:50:05.631 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/ 2022-09-03T06:50:05.632 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI= 2022-09-03T06:50:05.633 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI=', 'Content-Length': '0'} 2022-09-03T06:50:05.636 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a55772df0b12083f-006312f91d-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.687 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:05.688 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:05.689 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:05.690 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:05.690 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:05.692 DEBUG:boto:Method: GET 2022-09-03T06:50:05.692 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:50:05.693 DEBUG:boto:Data: 2022-09-03T06:50:05.693 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.694 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:05.694 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.695 DEBUG:boto:Params: {} 2022-09-03T06:50:05.696 DEBUG:boto:Token: None 2022-09-03T06:50:05.697 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/ 2022-09-03T06:50:05.698 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI= 2022-09-03T06:50:05.698 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI=', 'Content-Length': '0'} 2022-09-03T06:50:05.703 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f21a03ebcb8ab0b0-006312f91d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.704 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:05.705 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:05.706 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:05.707 DEBUG:boto:Method: HEAD 2022-09-03T06:50:05.707 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:50:05.708 DEBUG:boto:Data: 2022-09-03T06:50:05.708 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.709 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:05.709 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.710 DEBUG:boto:Params: {} 2022-09-03T06:50:05.711 DEBUG:boto:Token: None 2022-09-03T06:50:05.711 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj31 2022-09-03T06:50:05.713 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Qx6zBWssWG12QqYjR8nOs9iarVc= 2022-09-03T06:50:05.714 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Qx6zBWssWG12QqYjR8nOs9iarVc=', 'Content-Length': '0'} 2022-09-03T06:50:05.716 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f927a5394f0600d3-006312f91d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.717 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:05.717 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:05.718 DEBUG:boto:Method: HEAD 2022-09-03T06:50:05.719 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:50:05.719 DEBUG:boto:Data: 2022-09-03T06:50:05.720 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.721 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:05.721 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.722 DEBUG:boto:Params: {} 2022-09-03T06:50:05.723 DEBUG:boto:Token: None 2022-09-03T06:50:05.724 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj31 2022-09-03T06:50:05.725 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Qx6zBWssWG12QqYjR8nOs9iarVc= 2022-09-03T06:50:05.725 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Qx6zBWssWG12QqYjR8nOs9iarVc=', 'Content-Length': '0'} 2022-09-03T06:50:05.728 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000e4f8088d1c192ee-006312f91d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.729 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2022-09-03T06:50:05.730 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:05.732 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:05.734 DEBUG:boto:Method: GET 2022-09-03T06:50:05.738 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:50:05.739 DEBUG:boto:Data: 2022-09-03T06:50:05.740 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.741 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:05.742 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.743 DEBUG:boto:Params: {} 2022-09-03T06:50:05.745 DEBUG:boto:Token: None 2022-09-03T06:50:05.747 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj31 2022-09-03T06:50:05.748 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/iPeMn7PEtG4aIuIDV6Vnl3hwrg= 2022-09-03T06:50:05.749 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/iPeMn7PEtG4aIuIDV6Vnl3hwrg=', 'Content-Length': '0'} 2022-09-03T06:50:05.753 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d2b554ed32e33c20-006312f91d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.756 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:05.757 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:05.757 DEBUG:boto:Method: GET 2022-09-03T06:50:05.758 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:50:05.759 DEBUG:boto:Data: 2022-09-03T06:50:05.760 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.760 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:05.761 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.761 DEBUG:boto:Params: {} 2022-09-03T06:50:05.762 DEBUG:boto:Token: None 2022-09-03T06:50:05.763 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj31 2022-09-03T06:50:05.764 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/iPeMn7PEtG4aIuIDV6Vnl3hwrg= 2022-09-03T06:50:05.766 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/iPeMn7PEtG4aIuIDV6Vnl3hwrg=', 'Content-Length': '0'} 2022-09-03T06:50:05.769 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cb511c3946e6b483-006312f91d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.773 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:05.782 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:05.782 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:05.783 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:05.784 DEBUG:boto:Method: GET 2022-09-03T06:50:05.784 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:50:05.785 DEBUG:boto:Data: 2022-09-03T06:50:05.785 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.786 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:05.787 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.787 DEBUG:boto:Params: {} 2022-09-03T06:50:05.788 DEBUG:boto:Token: None 2022-09-03T06:50:05.789 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/ 2022-09-03T06:50:05.789 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI= 2022-09-03T06:50:05.790 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI=', 'Content-Length': '0'} 2022-09-03T06:50:05.797 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002954c2c6522c4cec-006312f91d-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.799 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:05.801 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:05.802 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:05.803 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:05.804 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:05.806 DEBUG:boto:Method: GET 2022-09-03T06:50:05.807 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:50:05.808 DEBUG:boto:Data: 2022-09-03T06:50:05.809 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.810 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:05.810 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.811 DEBUG:boto:Params: {} 2022-09-03T06:50:05.813 DEBUG:boto:Token: None 2022-09-03T06:50:05.814 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/ 2022-09-03T06:50:05.815 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI= 2022-09-03T06:50:05.816 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI=', 'Content-Length': '0'} 2022-09-03T06:50:05.822 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005676871ba5677fbb-006312f91d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.823 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:05.825 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:50:05.826 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:05.827 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:05.828 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.829 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.831 DEBUG:boto:Method: GET 2022-09-03T06:50:05.832 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.833 DEBUG:boto:Data: 2022-09-03T06:50:05.833 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.834 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:05.836 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.839 DEBUG:boto:Params: {} 2022-09-03T06:50:05.839 DEBUG:boto:Token: None 2022-09-03T06:50:05.840 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.841 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fCdeyLTTIAA2JUrJZbIXzHwzwo4= 2022-09-03T06:50:05.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fCdeyLTTIAA2JUrJZbIXzHwzwo4=', 'Content-Length': '0'} 2022-09-03T06:50:05.844 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000772da677cf63492d-006312f91d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.845 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:05.846 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:05.847 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.848 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.849 DEBUG:boto:Method: GET 2022-09-03T06:50:05.850 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.851 DEBUG:boto:Data: 2022-09-03T06:50:05.852 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.853 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:05.854 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.855 DEBUG:boto:Params: {} 2022-09-03T06:50:05.856 DEBUG:boto:Token: None 2022-09-03T06:50:05.857 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.858 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fCdeyLTTIAA2JUrJZbIXzHwzwo4= 2022-09-03T06:50:05.859 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fCdeyLTTIAA2JUrJZbIXzHwzwo4=', 'Content-Length': '0'} 2022-09-03T06:50:05.863 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dff135a52a30a889-006312f91d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.865 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:05.866 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:05.867 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.868 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.869 DEBUG:boto:Method: HEAD 2022-09-03T06:50:05.870 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.871 DEBUG:boto:Data: 2022-09-03T06:50:05.872 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.873 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:05.874 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.875 DEBUG:boto:Params: {} 2022-09-03T06:50:05.876 DEBUG:boto:Token: None 2022-09-03T06:50:05.877 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.877 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:28RTG62SrnSy6K3zaksfh0+tZKI= 2022-09-03T06:50:05.878 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:28RTG62SrnSy6K3zaksfh0+tZKI=', 'Content-Length': '0'} 2022-09-03T06:50:05.881 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004009abb7f5abd1d2-006312f91d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.883 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:05.883 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:05.884 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.885 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.885 DEBUG:boto:Method: HEAD 2022-09-03T06:50:05.886 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.887 DEBUG:boto:Data: 2022-09-03T06:50:05.887 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.888 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:05.889 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.889 DEBUG:boto:Params: {} 2022-09-03T06:50:05.890 DEBUG:boto:Token: None 2022-09-03T06:50:05.891 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.892 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:28RTG62SrnSy6K3zaksfh0+tZKI= 2022-09-03T06:50:05.893 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:28RTG62SrnSy6K3zaksfh0+tZKI=', 'Content-Length': '0'} 2022-09-03T06:50:05.896 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009722d4d982ff39f0-006312f91d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.897 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:50:05.898 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:05.899 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:05.899 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.900 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.900 DEBUG:boto:Method: GET 2022-09-03T06:50:05.901 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.901 DEBUG:boto:Data: 2022-09-03T06:50:05.902 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.903 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:05.904 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.904 DEBUG:boto:Params: {} 2022-09-03T06:50:05.905 DEBUG:boto:Token: None 2022-09-03T06:50:05.906 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.907 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fCdeyLTTIAA2JUrJZbIXzHwzwo4= 2022-09-03T06:50:05.907 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fCdeyLTTIAA2JUrJZbIXzHwzwo4=', 'Content-Length': '0'} 2022-09-03T06:50:05.910 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a9f79b04d5e8d854-006312f91d-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.912 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:05.913 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:05.914 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.915 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.915 DEBUG:boto:Method: GET 2022-09-03T06:50:05.916 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.917 DEBUG:boto:Data: 2022-09-03T06:50:05.919 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.920 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:05.921 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.922 DEBUG:boto:Params: {} 2022-09-03T06:50:05.923 DEBUG:boto:Token: None 2022-09-03T06:50:05.927 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:05.929 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fCdeyLTTIAA2JUrJZbIXzHwzwo4= 2022-09-03T06:50:05.930 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fCdeyLTTIAA2JUrJZbIXzHwzwo4=', 'Content-Length': '0'} 2022-09-03T06:50:05.935 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003370a6b783e0cf9a-006312f91d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.978 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:05.979 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:05.980 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:05.980 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:05.981 DEBUG:boto:Method: GET 2022-09-03T06:50:05.981 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:50:05.983 DEBUG:boto:Data: 2022-09-03T06:50:05.983 DEBUG:boto:Headers: {} 2022-09-03T06:50:05.984 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:05.985 DEBUG:boto:Port: 8000 2022-09-03T06:50:05.985 DEBUG:boto:Params: {} 2022-09-03T06:50:05.986 DEBUG:boto:Token: None 2022-09-03T06:50:05.987 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:05 GMT /dsgvcy-75/ 2022-09-03T06:50:05.988 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI= 2022-09-03T06:50:05.989 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:05 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IxPAmiWBHhGUeuNsQraA9HtErKI=', 'Content-Length': '0'} 2022-09-03T06:50:05.992 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004dfb6cfab373067e-006312f91d-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:05 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:05.993 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:05.996 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:06.000 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:06.001 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:06.002 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:06.002 DEBUG:boto:Method: GET 2022-09-03T06:50:06.003 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:50:06.004 DEBUG:boto:Data: 2022-09-03T06:50:06.004 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.005 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.005 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.006 DEBUG:boto:Params: {} 2022-09-03T06:50:06.006 DEBUG:boto:Token: None 2022-09-03T06:50:06.007 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/ 2022-09-03T06:50:06.007 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs= 2022-09-03T06:50:06.008 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs=', 'Content-Length': '0'} 2022-09-03T06:50:06.013 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bf048b330a8e6883-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.014 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:06.015 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:06.016 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:06.017 DEBUG:boto:Method: HEAD 2022-09-03T06:50:06.017 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:50:06.018 DEBUG:boto:Data: 2022-09-03T06:50:06.018 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.019 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.019 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.020 DEBUG:boto:Params: {} 2022-09-03T06:50:06.021 DEBUG:boto:Token: None 2022-09-03T06:50:06.022 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj33 2022-09-03T06:50:06.022 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Blq/ZBGmetPVryXKX0VexmgdgPY= 2022-09-03T06:50:06.023 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Blq/ZBGmetPVryXKX0VexmgdgPY=', 'Content-Length': '0'} 2022-09-03T06:50:06.025 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000085f08e51d05f2cfc-006312f91e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.026 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:06.026 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:06.027 DEBUG:boto:Method: HEAD 2022-09-03T06:50:06.027 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:50:06.028 DEBUG:boto:Data: 2022-09-03T06:50:06.028 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.029 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.029 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.030 DEBUG:boto:Params: {} 2022-09-03T06:50:06.030 DEBUG:boto:Token: None 2022-09-03T06:50:06.032 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj33 2022-09-03T06:50:06.033 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Blq/ZBGmetPVryXKX0VexmgdgPY= 2022-09-03T06:50:06.033 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Blq/ZBGmetPVryXKX0VexmgdgPY=', 'Content-Length': '0'} 2022-09-03T06:50:06.035 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a872fb3f87d0d8d4-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.036 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:50:06.037 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:06.038 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:06.038 DEBUG:boto:Method: GET 2022-09-03T06:50:06.039 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:50:06.039 DEBUG:boto:Data: 2022-09-03T06:50:06.040 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.040 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.041 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.041 DEBUG:boto:Params: {} 2022-09-03T06:50:06.042 DEBUG:boto:Token: None 2022-09-03T06:50:06.043 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj33 2022-09-03T06:50:06.043 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nk/m7jqmL2JGrVgNZ82aZkUIAMA= 2022-09-03T06:50:06.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nk/m7jqmL2JGrVgNZ82aZkUIAMA=', 'Content-Length': '0'} 2022-09-03T06:50:06.047 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009a7dfad99fca7b07-006312f91e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.048 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:06.049 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:06.050 DEBUG:boto:Method: GET 2022-09-03T06:50:06.051 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:50:06.052 DEBUG:boto:Data: 2022-09-03T06:50:06.053 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.054 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.055 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.056 DEBUG:boto:Params: {} 2022-09-03T06:50:06.058 DEBUG:boto:Token: None 2022-09-03T06:50:06.059 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj33 2022-09-03T06:50:06.059 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nk/m7jqmL2JGrVgNZ82aZkUIAMA= 2022-09-03T06:50:06.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nk/m7jqmL2JGrVgNZ82aZkUIAMA=', 'Content-Length': '0'} 2022-09-03T06:50:06.062 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c001d88e725c987b-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.063 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:06.064 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:06.064 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:06.065 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:06.066 DEBUG:boto:Method: GET 2022-09-03T06:50:06.068 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:50:06.068 DEBUG:boto:Data: 2022-09-03T06:50:06.069 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.069 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.070 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.072 DEBUG:boto:Params: {} 2022-09-03T06:50:06.073 DEBUG:boto:Token: None 2022-09-03T06:50:06.074 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/ 2022-09-03T06:50:06.075 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs= 2022-09-03T06:50:06.075 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs=', 'Content-Length': '0'} 2022-09-03T06:50:06.078 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000037e0232fd8d00f9-006312f91e-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.079 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:06.080 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:06.081 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:06.081 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:06.082 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:06.083 DEBUG:boto:Method: GET 2022-09-03T06:50:06.083 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:50:06.084 DEBUG:boto:Data: 2022-09-03T06:50:06.084 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.085 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.086 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.087 DEBUG:boto:Params: {} 2022-09-03T06:50:06.087 DEBUG:boto:Token: None 2022-09-03T06:50:06.088 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/ 2022-09-03T06:50:06.089 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs= 2022-09-03T06:50:06.089 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs=', 'Content-Length': '0'} 2022-09-03T06:50:06.098 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c5cc876bd66e82f3-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.138 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:06.139 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:50:06.140 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:06.140 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:06.141 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.141 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.142 DEBUG:boto:Method: GET 2022-09-03T06:50:06.143 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.143 DEBUG:boto:Data: 2022-09-03T06:50:06.144 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.144 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.145 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.145 DEBUG:boto:Params: {} 2022-09-03T06:50:06.146 DEBUG:boto:Token: None 2022-09-03T06:50:06.147 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.147 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WgqRQDJusc8qxWjFLpOR8WuCyoE= 2022-09-03T06:50:06.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WgqRQDJusc8qxWjFLpOR8WuCyoE=', 'Content-Length': '0'} 2022-09-03T06:50:06.150 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006bb3856e7f2dd428-006312f91e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.151 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:06.152 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:06.153 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.153 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.154 DEBUG:boto:Method: GET 2022-09-03T06:50:06.154 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.155 DEBUG:boto:Data: 2022-09-03T06:50:06.155 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.156 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.157 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.157 DEBUG:boto:Params: {} 2022-09-03T06:50:06.158 DEBUG:boto:Token: None 2022-09-03T06:50:06.158 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.159 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WgqRQDJusc8qxWjFLpOR8WuCyoE= 2022-09-03T06:50:06.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WgqRQDJusc8qxWjFLpOR8WuCyoE=', 'Content-Length': '0'} 2022-09-03T06:50:06.163 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007d13136eea77339f-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.164 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:06.164 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:06.165 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.165 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.166 DEBUG:boto:Method: HEAD 2022-09-03T06:50:06.167 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.167 DEBUG:boto:Data: 2022-09-03T06:50:06.168 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.168 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.169 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.169 DEBUG:boto:Params: {} 2022-09-03T06:50:06.178 DEBUG:boto:Token: None 2022-09-03T06:50:06.179 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.180 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IUutR6evMA333qMpeOf7ALhkjzg= 2022-09-03T06:50:06.180 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IUutR6evMA333qMpeOf7ALhkjzg=', 'Content-Length': '0'} 2022-09-03T06:50:06.182 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000482a62068c6af9ff-006312f91e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.183 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:06.184 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:06.184 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.186 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.186 DEBUG:boto:Method: HEAD 2022-09-03T06:50:06.187 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.187 DEBUG:boto:Data: 2022-09-03T06:50:06.188 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.189 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.189 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.189 DEBUG:boto:Params: {} 2022-09-03T06:50:06.190 DEBUG:boto:Token: None 2022-09-03T06:50:06.191 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.191 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IUutR6evMA333qMpeOf7ALhkjzg= 2022-09-03T06:50:06.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IUutR6evMA333qMpeOf7ALhkjzg=', 'Content-Length': '0'} 2022-09-03T06:50:06.194 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002ce1f68f1a7d23ab-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.195 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:50:06.196 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:06.196 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:06.197 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.197 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.198 DEBUG:boto:Method: GET 2022-09-03T06:50:06.198 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.199 DEBUG:boto:Data: 2022-09-03T06:50:06.199 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.200 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.201 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.201 DEBUG:boto:Params: {} 2022-09-03T06:50:06.202 DEBUG:boto:Token: None 2022-09-03T06:50:06.202 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.203 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WgqRQDJusc8qxWjFLpOR8WuCyoE= 2022-09-03T06:50:06.204 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WgqRQDJusc8qxWjFLpOR8WuCyoE=', 'Content-Length': '0'} 2022-09-03T06:50:06.206 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b65315ea383e1d7e-006312f91e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.250 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:06.251 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:06.251 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.252 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.252 DEBUG:boto:Method: GET 2022-09-03T06:50:06.253 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.254 DEBUG:boto:Data: 2022-09-03T06:50:06.255 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.256 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.256 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.257 DEBUG:boto:Params: {} 2022-09-03T06:50:06.259 DEBUG:boto:Token: None 2022-09-03T06:50:06.260 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:06.262 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WgqRQDJusc8qxWjFLpOR8WuCyoE= 2022-09-03T06:50:06.263 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WgqRQDJusc8qxWjFLpOR8WuCyoE=', 'Content-Length': '0'} 2022-09-03T06:50:06.267 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c2148c619dc9d03f-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.268 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:06.269 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:06.270 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:06.270 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:06.271 DEBUG:boto:Method: GET 2022-09-03T06:50:06.271 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:50:06.272 DEBUG:boto:Data: 2022-09-03T06:50:06.272 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.273 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.273 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.274 DEBUG:boto:Params: {} 2022-09-03T06:50:06.275 DEBUG:boto:Token: None 2022-09-03T06:50:06.275 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/ 2022-09-03T06:50:06.276 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs= 2022-09-03T06:50:06.277 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs=', 'Content-Length': '0'} 2022-09-03T06:50:06.281 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f28c94038c645e70-006312f91e-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.282 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:06.284 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:06.285 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:06.286 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:06.287 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:06.288 DEBUG:boto:Method: GET 2022-09-03T06:50:06.289 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:50:06.291 DEBUG:boto:Data: 2022-09-03T06:50:06.292 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.293 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.294 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.295 DEBUG:boto:Params: {} 2022-09-03T06:50:06.297 DEBUG:boto:Token: None 2022-09-03T06:50:06.298 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/ 2022-09-03T06:50:06.299 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs= 2022-09-03T06:50:06.301 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs=', 'Content-Length': '0'} 2022-09-03T06:50:06.307 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005e330ce5da5cd3b4-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.351 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:06.353 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:06.354 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:06.355 DEBUG:boto:Method: HEAD 2022-09-03T06:50:06.356 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:50:06.357 DEBUG:boto:Data: 2022-09-03T06:50:06.358 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.359 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.360 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.361 DEBUG:boto:Params: {} 2022-09-03T06:50:06.362 DEBUG:boto:Token: None 2022-09-03T06:50:06.363 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj34 2022-09-03T06:50:06.364 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dpPMmSp8n7pSNCBB1TykeEr7fRk= 2022-09-03T06:50:06.365 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dpPMmSp8n7pSNCBB1TykeEr7fRk=', 'Content-Length': '0'} 2022-09-03T06:50:06.368 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004573d084d72f61a2-006312f91e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.369 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:06.370 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:06.372 DEBUG:boto:Method: HEAD 2022-09-03T06:50:06.373 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:50:06.374 DEBUG:boto:Data: 2022-09-03T06:50:06.375 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.376 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.377 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.377 DEBUG:boto:Params: {} 2022-09-03T06:50:06.379 DEBUG:boto:Token: None 2022-09-03T06:50:06.380 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj34 2022-09-03T06:50:06.381 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dpPMmSp8n7pSNCBB1TykeEr7fRk= 2022-09-03T06:50:06.382 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dpPMmSp8n7pSNCBB1TykeEr7fRk=', 'Content-Length': '0'} 2022-09-03T06:50:06.386 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007acf70f826c7978c-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.388 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:50:06.390 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:06.391 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:06.392 DEBUG:boto:Method: GET 2022-09-03T06:50:06.393 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:50:06.394 DEBUG:boto:Data: 2022-09-03T06:50:06.394 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.396 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.397 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.398 DEBUG:boto:Params: {} 2022-09-03T06:50:06.399 DEBUG:boto:Token: None 2022-09-03T06:50:06.400 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj34 2022-09-03T06:50:06.401 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qPQqRvabIJKr1mI50IAUmzB6pEM= 2022-09-03T06:50:06.402 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qPQqRvabIJKr1mI50IAUmzB6pEM=', 'Content-Length': '0'} 2022-09-03T06:50:06.404 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a0255587659b3e1-006312f91e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.407 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:06.408 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:06.410 DEBUG:boto:Method: GET 2022-09-03T06:50:06.411 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:50:06.412 DEBUG:boto:Data: 2022-09-03T06:50:06.413 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.414 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.415 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.418 DEBUG:boto:Params: {} 2022-09-03T06:50:06.419 DEBUG:boto:Token: None 2022-09-03T06:50:06.420 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj34 2022-09-03T06:50:06.421 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qPQqRvabIJKr1mI50IAUmzB6pEM= 2022-09-03T06:50:06.422 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qPQqRvabIJKr1mI50IAUmzB6pEM=', 'Content-Length': '0'} 2022-09-03T06:50:06.424 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000abd03a873826bd72-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.425 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:06.426 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:06.427 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:06.427 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:06.428 DEBUG:boto:Method: GET 2022-09-03T06:50:06.428 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:50:06.429 DEBUG:boto:Data: 2022-09-03T06:50:06.429 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.430 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.431 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.432 DEBUG:boto:Params: {} 2022-09-03T06:50:06.433 DEBUG:boto:Token: None 2022-09-03T06:50:06.434 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/ 2022-09-03T06:50:06.436 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs= 2022-09-03T06:50:06.437 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs=', 'Content-Length': '0'} 2022-09-03T06:50:06.440 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009a4d96524e5c152a-006312f91e-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.441 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:06.443 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:06.444 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:06.445 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:06.446 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:06.447 DEBUG:boto:Method: GET 2022-09-03T06:50:06.447 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:50:06.448 DEBUG:boto:Data: 2022-09-03T06:50:06.449 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.450 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.451 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.452 DEBUG:boto:Params: {} 2022-09-03T06:50:06.453 DEBUG:boto:Token: None 2022-09-03T06:50:06.454 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/ 2022-09-03T06:50:06.455 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs= 2022-09-03T06:50:06.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs=', 'Content-Length': '0'} 2022-09-03T06:50:06.462 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004243f0076b784196-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.463 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:06.465 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj37 2022-09-03T06:50:06.465 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:06.466 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:06.467 DEBUG:boto:path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.467 DEBUG:boto:auth_path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.468 DEBUG:boto:Method: GET 2022-09-03T06:50:06.469 DEBUG:boto:Path: /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.470 DEBUG:boto:Data: 2022-09-03T06:50:06.470 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.471 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.471 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.472 DEBUG:boto:Params: {} 2022-09-03T06:50:06.473 DEBUG:boto:Token: None 2022-09-03T06:50:06.474 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.474 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:51jWIg42e9ku719vEYHGtQmiZSs= 2022-09-03T06:50:06.475 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:51jWIg42e9ku719vEYHGtQmiZSs=', 'Content-Length': '0'} 2022-09-03T06:50:06.477 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008d0ad4994fee5d16-006312f91e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.479 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:06.479 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:06.480 DEBUG:boto:path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.481 DEBUG:boto:auth_path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.487 DEBUG:boto:Method: GET 2022-09-03T06:50:06.488 DEBUG:boto:Path: /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.488 DEBUG:boto:Data: 2022-09-03T06:50:06.489 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.489 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.490 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.490 DEBUG:boto:Params: {} 2022-09-03T06:50:06.491 DEBUG:boto:Token: None 2022-09-03T06:50:06.492 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.492 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:51jWIg42e9ku719vEYHGtQmiZSs= 2022-09-03T06:50:06.493 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:51jWIg42e9ku719vEYHGtQmiZSs=', 'Content-Length': '0'} 2022-09-03T06:50:06.495 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004b1edd800245daa6-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.497 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:06.499 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:06.500 DEBUG:boto:path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.501 DEBUG:boto:auth_path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.502 DEBUG:boto:Method: HEAD 2022-09-03T06:50:06.503 DEBUG:boto:Path: /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.504 DEBUG:boto:Data: 2022-09-03T06:50:06.505 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.505 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.506 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.506 DEBUG:boto:Params: {} 2022-09-03T06:50:06.507 DEBUG:boto:Token: None 2022-09-03T06:50:06.508 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.508 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:raJBRKQ++uvFJnAGH8kNRjSxF+A= 2022-09-03T06:50:06.509 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:raJBRKQ++uvFJnAGH8kNRjSxF+A=', 'Content-Length': '0'} 2022-09-03T06:50:06.512 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000013f88765f320ab9e-006312f91e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.513 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:06.514 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:06.514 DEBUG:boto:path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.515 DEBUG:boto:auth_path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.516 DEBUG:boto:Method: HEAD 2022-09-03T06:50:06.516 DEBUG:boto:Path: /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.517 DEBUG:boto:Data: 2022-09-03T06:50:06.517 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.518 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.519 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.519 DEBUG:boto:Params: {} 2022-09-03T06:50:06.520 DEBUG:boto:Token: None 2022-09-03T06:50:06.521 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.522 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:raJBRKQ++uvFJnAGH8kNRjSxF+A= 2022-09-03T06:50:06.523 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:raJBRKQ++uvFJnAGH8kNRjSxF+A=', 'Content-Length': '0'} 2022-09-03T06:50:06.526 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cc4db0a44fdc9f1a-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.527 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj37 2022-09-03T06:50:06.528 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:06.528 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:06.529 DEBUG:boto:path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.530 DEBUG:boto:auth_path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.531 DEBUG:boto:Method: GET 2022-09-03T06:50:06.532 DEBUG:boto:Path: /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.533 DEBUG:boto:Data: 2022-09-03T06:50:06.534 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.535 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.536 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.537 DEBUG:boto:Params: {} 2022-09-03T06:50:06.539 DEBUG:boto:Token: None 2022-09-03T06:50:06.540 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.540 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:51jWIg42e9ku719vEYHGtQmiZSs= 2022-09-03T06:50:06.541 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:51jWIg42e9ku719vEYHGtQmiZSs=', 'Content-Length': '0'} 2022-09-03T06:50:06.545 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d1edfed67836d553-006312f91e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.546 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:06.547 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:06.548 DEBUG:boto:path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.548 DEBUG:boto:auth_path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.549 DEBUG:boto:Method: GET 2022-09-03T06:50:06.549 DEBUG:boto:Path: /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.550 DEBUG:boto:Data: 2022-09-03T06:50:06.550 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.551 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.552 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.552 DEBUG:boto:Params: {} 2022-09-03T06:50:06.553 DEBUG:boto:Token: None 2022-09-03T06:50:06.554 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:06.555 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:51jWIg42e9ku719vEYHGtQmiZSs= 2022-09-03T06:50:06.556 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:51jWIg42e9ku719vEYHGtQmiZSs=', 'Content-Length': '0'} 2022-09-03T06:50:06.558 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000098ef289596e8a420-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.606 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:06.607 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:06.608 DEBUG:boto:path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:06.609 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:06.610 DEBUG:boto:Method: GET 2022-09-03T06:50:06.611 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj37 2022-09-03T06:50:06.612 DEBUG:boto:Data: 2022-09-03T06:50:06.613 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.615 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.616 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.617 DEBUG:boto:Params: {} 2022-09-03T06:50:06.618 DEBUG:boto:Token: None 2022-09-03T06:50:06.620 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/ 2022-09-03T06:50:06.621 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs= 2022-09-03T06:50:06.622 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs=', 'Content-Length': '0'} 2022-09-03T06:50:06.625 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eca49f3c25158036-006312f91e-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.626 DEBUG:boto:b'dsgvcy-75obj371000falseobj372022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:06.626 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:06.627 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:06.628 DEBUG:boto:path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:06.628 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:06.629 DEBUG:boto:Method: GET 2022-09-03T06:50:06.630 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj37 2022-09-03T06:50:06.631 DEBUG:boto:Data: 2022-09-03T06:50:06.632 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.632 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.633 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.633 DEBUG:boto:Params: {} 2022-09-03T06:50:06.634 DEBUG:boto:Token: None 2022-09-03T06:50:06.635 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/ 2022-09-03T06:50:06.635 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs= 2022-09-03T06:50:06.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs=', 'Content-Length': '0'} 2022-09-03T06:50:06.640 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004476a6ba819eee3b-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.642 DEBUG:boto:b'dsgvcy-75obj371000falseobj372022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:06.643 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:06.644 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:06.644 DEBUG:boto:Method: HEAD 2022-09-03T06:50:06.645 DEBUG:boto:Path: /dsgvcy-75/obj37 2022-09-03T06:50:06.645 DEBUG:boto:Data: 2022-09-03T06:50:06.646 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.646 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.647 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.647 DEBUG:boto:Params: {} 2022-09-03T06:50:06.648 DEBUG:boto:Token: None 2022-09-03T06:50:06.649 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj37 2022-09-03T06:50:06.649 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VRnZvuteOw9CHIsOJZJQe0LQoKs= 2022-09-03T06:50:06.650 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VRnZvuteOw9CHIsOJZJQe0LQoKs=', 'Content-Length': '0'} 2022-09-03T06:50:06.652 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d33029eeb164938d-006312f91e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.653 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:06.654 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:06.654 DEBUG:boto:Method: HEAD 2022-09-03T06:50:06.655 DEBUG:boto:Path: /dsgvcy-75/obj37 2022-09-03T06:50:06.655 DEBUG:boto:Data: 2022-09-03T06:50:06.656 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.656 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.657 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.657 DEBUG:boto:Params: {} 2022-09-03T06:50:06.658 DEBUG:boto:Token: None 2022-09-03T06:50:06.658 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj37 2022-09-03T06:50:06.659 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VRnZvuteOw9CHIsOJZJQe0LQoKs= 2022-09-03T06:50:06.660 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VRnZvuteOw9CHIsOJZJQe0LQoKs=', 'Content-Length': '0'} 2022-09-03T06:50:06.662 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000014c6d6f0417789fb-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.663 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj37 2022-09-03T06:50:06.663 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:06.664 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:06.664 DEBUG:boto:Method: GET 2022-09-03T06:50:06.665 DEBUG:boto:Path: /dsgvcy-75/obj37 2022-09-03T06:50:06.665 DEBUG:boto:Data: 2022-09-03T06:50:06.666 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.666 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.667 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.668 DEBUG:boto:Params: {} 2022-09-03T06:50:06.668 DEBUG:boto:Token: None 2022-09-03T06:50:06.669 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj37 2022-09-03T06:50:06.669 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gX9jCmQruyQzCYM4cxxPsTtk2RM= 2022-09-03T06:50:06.670 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gX9jCmQruyQzCYM4cxxPsTtk2RM=', 'Content-Length': '0'} 2022-09-03T06:50:06.672 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc9fa53777b49b55-006312f91e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.673 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:06.674 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:06.674 DEBUG:boto:Method: GET 2022-09-03T06:50:06.675 DEBUG:boto:Path: /dsgvcy-75/obj37 2022-09-03T06:50:06.675 DEBUG:boto:Data: 2022-09-03T06:50:06.676 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.676 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.677 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.677 DEBUG:boto:Params: {} 2022-09-03T06:50:06.678 DEBUG:boto:Token: None 2022-09-03T06:50:06.679 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj37 2022-09-03T06:50:06.679 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gX9jCmQruyQzCYM4cxxPsTtk2RM= 2022-09-03T06:50:06.680 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gX9jCmQruyQzCYM4cxxPsTtk2RM=', 'Content-Length': '0'} 2022-09-03T06:50:06.682 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004c753b28972a283a-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.683 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:06.684 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:06.684 DEBUG:boto:path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:06.685 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:06.685 DEBUG:boto:Method: GET 2022-09-03T06:50:06.686 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj37 2022-09-03T06:50:06.687 DEBUG:boto:Data: 2022-09-03T06:50:06.687 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.688 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.688 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.689 DEBUG:boto:Params: {} 2022-09-03T06:50:06.689 DEBUG:boto:Token: None 2022-09-03T06:50:06.690 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/ 2022-09-03T06:50:06.691 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs= 2022-09-03T06:50:06.692 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs=', 'Content-Length': '0'} 2022-09-03T06:50:06.694 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009ed7d7adddc4d0c5-006312f91e-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.695 DEBUG:boto:b'dsgvcy-75obj371000falseobj372022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:06.696 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:06.697 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:06.697 DEBUG:boto:path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:06.698 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:06.699 DEBUG:boto:Method: GET 2022-09-03T06:50:06.699 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj37 2022-09-03T06:50:06.700 DEBUG:boto:Data: 2022-09-03T06:50:06.700 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.701 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.702 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.702 DEBUG:boto:Params: {} 2022-09-03T06:50:06.703 DEBUG:boto:Token: None 2022-09-03T06:50:06.703 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/ 2022-09-03T06:50:06.704 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs= 2022-09-03T06:50:06.705 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs=', 'Content-Length': '0'} 2022-09-03T06:50:06.709 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000484c6a4e807e36f2-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.750 DEBUG:boto:b'dsgvcy-75obj371000falseobj372022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:06.751 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:50:06.752 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:06.752 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:06.753 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.753 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.754 DEBUG:boto:Method: GET 2022-09-03T06:50:06.754 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.755 DEBUG:boto:Data: 2022-09-03T06:50:06.755 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.756 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.756 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.757 DEBUG:boto:Params: {} 2022-09-03T06:50:06.758 DEBUG:boto:Token: None 2022-09-03T06:50:06.758 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.759 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KR7Q77rUAKm97OKU4n18meQVGlA= 2022-09-03T06:50:06.759 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KR7Q77rUAKm97OKU4n18meQVGlA=', 'Content-Length': '0'} 2022-09-03T06:50:06.762 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003019e91d29c7ee04-006312f91e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.763 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:06.763 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:06.764 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.764 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.765 DEBUG:boto:Method: GET 2022-09-03T06:50:06.765 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.766 DEBUG:boto:Data: 2022-09-03T06:50:06.767 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.768 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.768 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.769 DEBUG:boto:Params: {} 2022-09-03T06:50:06.769 DEBUG:boto:Token: None 2022-09-03T06:50:06.770 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.771 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KR7Q77rUAKm97OKU4n18meQVGlA= 2022-09-03T06:50:06.771 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KR7Q77rUAKm97OKU4n18meQVGlA=', 'Content-Length': '0'} 2022-09-03T06:50:06.774 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008ff67a65bde3b10c-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.775 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:06.775 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:06.776 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.776 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.777 DEBUG:boto:Method: HEAD 2022-09-03T06:50:06.778 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.778 DEBUG:boto:Data: 2022-09-03T06:50:06.779 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.779 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.780 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.780 DEBUG:boto:Params: {} 2022-09-03T06:50:06.781 DEBUG:boto:Token: None 2022-09-03T06:50:06.782 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.782 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2vYAdiO8yJLRfkrsZlD3JHUeHqs= 2022-09-03T06:50:06.783 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2vYAdiO8yJLRfkrsZlD3JHUeHqs=', 'Content-Length': '0'} 2022-09-03T06:50:06.785 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006ba479f6c7f61b68-006312f91e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.786 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:06.787 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:06.787 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.788 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.788 DEBUG:boto:Method: HEAD 2022-09-03T06:50:06.789 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.789 DEBUG:boto:Data: 2022-09-03T06:50:06.790 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.790 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.791 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.792 DEBUG:boto:Params: {} 2022-09-03T06:50:06.792 DEBUG:boto:Token: None 2022-09-03T06:50:06.793 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.793 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2vYAdiO8yJLRfkrsZlD3JHUeHqs= 2022-09-03T06:50:06.794 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2vYAdiO8yJLRfkrsZlD3JHUeHqs=', 'Content-Length': '0'} 2022-09-03T06:50:06.796 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007635fad04af4f344-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.797 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:50:06.798 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:06.798 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:06.799 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.800 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.800 DEBUG:boto:Method: GET 2022-09-03T06:50:06.801 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.801 DEBUG:boto:Data: 2022-09-03T06:50:06.802 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.802 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.803 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.803 DEBUG:boto:Params: {} 2022-09-03T06:50:06.804 DEBUG:boto:Token: None 2022-09-03T06:50:06.805 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.805 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KR7Q77rUAKm97OKU4n18meQVGlA= 2022-09-03T06:50:06.806 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KR7Q77rUAKm97OKU4n18meQVGlA=', 'Content-Length': '0'} 2022-09-03T06:50:06.809 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc64dc26d1d30ad4-006312f91e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.811 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:06.812 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:06.812 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.813 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.814 DEBUG:boto:Method: GET 2022-09-03T06:50:06.814 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.815 DEBUG:boto:Data: 2022-09-03T06:50:06.816 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.817 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.817 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.818 DEBUG:boto:Params: {} 2022-09-03T06:50:06.819 DEBUG:boto:Token: None 2022-09-03T06:50:06.819 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:06.820 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KR7Q77rUAKm97OKU4n18meQVGlA= 2022-09-03T06:50:06.821 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KR7Q77rUAKm97OKU4n18meQVGlA=', 'Content-Length': '0'} 2022-09-03T06:50:06.823 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d89b992a3c64dda-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.824 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:06.825 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:06.825 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:06.826 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:06.826 DEBUG:boto:Method: GET 2022-09-03T06:50:06.827 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:50:06.828 DEBUG:boto:Data: 2022-09-03T06:50:06.828 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.829 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.830 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.830 DEBUG:boto:Params: {} 2022-09-03T06:50:06.831 DEBUG:boto:Token: None 2022-09-03T06:50:06.832 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/ 2022-09-03T06:50:06.833 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs= 2022-09-03T06:50:06.834 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs=', 'Content-Length': '0'} 2022-09-03T06:50:06.836 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bc8d9b467766c479-006312f91e-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.838 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:06.840 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:06.841 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:06.842 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:06.843 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:06.845 DEBUG:boto:Method: GET 2022-09-03T06:50:06.846 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:50:06.847 DEBUG:boto:Data: 2022-09-03T06:50:06.847 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.848 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.849 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.850 DEBUG:boto:Params: {} 2022-09-03T06:50:06.851 DEBUG:boto:Token: None 2022-09-03T06:50:06.852 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/ 2022-09-03T06:50:06.852 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs= 2022-09-03T06:50:06.853 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs=', 'Content-Length': '0'} 2022-09-03T06:50:06.859 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002b3214e21d2a5eee-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.902 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:06.904 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:06.905 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:06.907 DEBUG:boto:Method: HEAD 2022-09-03T06:50:06.908 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:50:06.909 DEBUG:boto:Data: 2022-09-03T06:50:06.910 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.911 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.912 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.913 DEBUG:boto:Params: {} 2022-09-03T06:50:06.914 DEBUG:boto:Token: None 2022-09-03T06:50:06.915 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj38 2022-09-03T06:50:06.916 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2cAF3CsV5e+JfMRJu2G+25W+Hd8= 2022-09-03T06:50:06.917 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2cAF3CsV5e+JfMRJu2G+25W+Hd8=', 'Content-Length': '0'} 2022-09-03T06:50:06.919 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007286214e3dd289e8-006312f91e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.920 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:06.921 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:06.921 DEBUG:boto:Method: HEAD 2022-09-03T06:50:06.922 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:50:06.922 DEBUG:boto:Data: 2022-09-03T06:50:06.923 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.923 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.924 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.925 DEBUG:boto:Params: {} 2022-09-03T06:50:06.925 DEBUG:boto:Token: None 2022-09-03T06:50:06.926 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj38 2022-09-03T06:50:06.927 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2cAF3CsV5e+JfMRJu2G+25W+Hd8= 2022-09-03T06:50:06.927 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2cAF3CsV5e+JfMRJu2G+25W+Hd8=', 'Content-Length': '0'} 2022-09-03T06:50:06.934 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001454beffbe3c8c81-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.935 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:50:06.937 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:06.939 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:06.939 DEBUG:boto:Method: GET 2022-09-03T06:50:06.940 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:50:06.941 DEBUG:boto:Data: 2022-09-03T06:50:06.941 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.942 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.942 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.943 DEBUG:boto:Params: {} 2022-09-03T06:50:06.944 DEBUG:boto:Token: None 2022-09-03T06:50:06.944 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj38 2022-09-03T06:50:06.945 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AAC43BePqe/YPsqIZkRaaXFxRrA= 2022-09-03T06:50:06.946 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AAC43BePqe/YPsqIZkRaaXFxRrA=', 'Content-Length': '0'} 2022-09-03T06:50:06.952 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dbc0779502d741f7-006312f91e-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.954 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:06.954 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:06.955 DEBUG:boto:Method: GET 2022-09-03T06:50:06.956 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:50:06.956 DEBUG:boto:Data: 2022-09-03T06:50:06.957 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.957 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.958 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.959 DEBUG:boto:Params: {} 2022-09-03T06:50:06.960 DEBUG:boto:Token: None 2022-09-03T06:50:06.961 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/obj38 2022-09-03T06:50:06.962 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AAC43BePqe/YPsqIZkRaaXFxRrA= 2022-09-03T06:50:06.963 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AAC43BePqe/YPsqIZkRaaXFxRrA=', 'Content-Length': '0'} 2022-09-03T06:50:06.966 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000993b42151df4836f-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.968 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:06.969 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:06.970 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:06.971 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:06.972 DEBUG:boto:Method: GET 2022-09-03T06:50:06.973 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:50:06.973 DEBUG:boto:Data: 2022-09-03T06:50:06.974 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.975 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:06.975 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.976 DEBUG:boto:Params: {} 2022-09-03T06:50:06.977 DEBUG:boto:Token: None 2022-09-03T06:50:06.978 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/ 2022-09-03T06:50:06.979 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs= 2022-09-03T06:50:06.980 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs=', 'Content-Length': '0'} 2022-09-03T06:50:06.986 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000aca58cef9efc0a02-006312f91e-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:06 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:06.987 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:06.989 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:06.990 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:06.991 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:06.992 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:06.992 DEBUG:boto:Method: GET 2022-09-03T06:50:06.993 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:50:06.993 DEBUG:boto:Data: 2022-09-03T06:50:06.994 DEBUG:boto:Headers: {} 2022-09-03T06:50:06.995 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:06.996 DEBUG:boto:Port: 8000 2022-09-03T06:50:06.996 DEBUG:boto:Params: {} 2022-09-03T06:50:06.997 DEBUG:boto:Token: None 2022-09-03T06:50:06.998 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:06 GMT /dsgvcy-75/ 2022-09-03T06:50:06.999 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs= 2022-09-03T06:50:06.999 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:06 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vWgFa0cNVXygWCxopJ+2gtknnEs=', 'Content-Length': '0'} 2022-09-03T06:50:07.004 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fd145db913dcd9da-006312f91e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.005 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:07.006 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:50:07.007 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:07.008 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:07.010 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.011 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.012 DEBUG:boto:Method: GET 2022-09-03T06:50:07.013 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.014 DEBUG:boto:Data: 2022-09-03T06:50:07.015 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.016 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:07.017 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.019 DEBUG:boto:Params: {} 2022-09-03T06:50:07.020 DEBUG:boto:Token: None 2022-09-03T06:50:07.021 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.022 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hyr2WH40Mg9i5lPpCN+UadnDkIk= 2022-09-03T06:50:07.023 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hyr2WH40Mg9i5lPpCN+UadnDkIk=', 'Content-Length': '0'} 2022-09-03T06:50:07.027 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e322e44382e84121-006312f91f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.074 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:07.076 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:07.076 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.077 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.078 DEBUG:boto:Method: GET 2022-09-03T06:50:07.079 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.080 DEBUG:boto:Data: 2022-09-03T06:50:07.080 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.081 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:07.082 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.082 DEBUG:boto:Params: {} 2022-09-03T06:50:07.083 DEBUG:boto:Token: None 2022-09-03T06:50:07.084 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.085 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hyr2WH40Mg9i5lPpCN+UadnDkIk= 2022-09-03T06:50:07.086 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hyr2WH40Mg9i5lPpCN+UadnDkIk=', 'Content-Length': '0'} 2022-09-03T06:50:07.089 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000012a07c518201f8d0-006312f91f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.091 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:07.092 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:07.094 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.095 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.096 DEBUG:boto:Method: HEAD 2022-09-03T06:50:07.096 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.097 DEBUG:boto:Data: 2022-09-03T06:50:07.098 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.099 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:07.102 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.103 DEBUG:boto:Params: {} 2022-09-03T06:50:07.104 DEBUG:boto:Token: None 2022-09-03T06:50:07.105 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.106 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iUBq0k7pgqtdJ2WtAZTwQ/3lvpM= 2022-09-03T06:50:07.107 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iUBq0k7pgqtdJ2WtAZTwQ/3lvpM=', 'Content-Length': '0'} 2022-09-03T06:50:07.110 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e112075ac8fd28a8-006312f91f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.112 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:07.113 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:07.116 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.117 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.118 DEBUG:boto:Method: HEAD 2022-09-03T06:50:07.119 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.120 DEBUG:boto:Data: 2022-09-03T06:50:07.121 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.122 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:07.123 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.124 DEBUG:boto:Params: {} 2022-09-03T06:50:07.126 DEBUG:boto:Token: None 2022-09-03T06:50:07.127 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.128 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iUBq0k7pgqtdJ2WtAZTwQ/3lvpM= 2022-09-03T06:50:07.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iUBq0k7pgqtdJ2WtAZTwQ/3lvpM=', 'Content-Length': '0'} 2022-09-03T06:50:07.132 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000513b905a76176394-006312f91f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.134 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:50:07.135 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:07.137 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:07.142 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.143 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.144 DEBUG:boto:Method: GET 2022-09-03T06:50:07.146 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.147 DEBUG:boto:Data: 2022-09-03T06:50:07.148 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.149 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:07.150 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.151 DEBUG:boto:Params: {} 2022-09-03T06:50:07.152 DEBUG:boto:Token: None 2022-09-03T06:50:07.153 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.159 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hyr2WH40Mg9i5lPpCN+UadnDkIk= 2022-09-03T06:50:07.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hyr2WH40Mg9i5lPpCN+UadnDkIk=', 'Content-Length': '0'} 2022-09-03T06:50:07.162 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007bfce06c3df22c49-006312f91f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.163 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:07.164 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:07.164 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.165 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.165 DEBUG:boto:Method: GET 2022-09-03T06:50:07.166 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.167 DEBUG:boto:Data: 2022-09-03T06:50:07.167 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.168 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:07.169 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.170 DEBUG:boto:Params: {} 2022-09-03T06:50:07.170 DEBUG:boto:Token: None 2022-09-03T06:50:07.171 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:07.172 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hyr2WH40Mg9i5lPpCN+UadnDkIk= 2022-09-03T06:50:07.172 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hyr2WH40Mg9i5lPpCN+UadnDkIk=', 'Content-Length': '0'} 2022-09-03T06:50:07.175 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000053d3932cff79ee96-006312f91f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.222 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:07.223 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:07.224 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:07.224 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:07.225 DEBUG:boto:Method: GET 2022-09-03T06:50:07.225 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:50:07.226 DEBUG:boto:Data: 2022-09-03T06:50:07.227 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.227 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:07.228 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.228 DEBUG:boto:Params: {} 2022-09-03T06:50:07.232 DEBUG:boto:Token: None 2022-09-03T06:50:07.233 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/ 2022-09-03T06:50:07.234 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k= 2022-09-03T06:50:07.235 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k=', 'Content-Length': '0'} 2022-09-03T06:50:07.238 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001f3fae8b9eba7a2c-006312f91f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.240 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:07.241 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:07.242 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:07.242 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:07.243 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:07.244 DEBUG:boto:Method: GET 2022-09-03T06:50:07.245 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:50:07.246 DEBUG:boto:Data: 2022-09-03T06:50:07.246 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.247 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:07.248 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.248 DEBUG:boto:Params: {} 2022-09-03T06:50:07.249 DEBUG:boto:Token: None 2022-09-03T06:50:07.250 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/ 2022-09-03T06:50:07.251 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k= 2022-09-03T06:50:07.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k=', 'Content-Length': '0'} 2022-09-03T06:50:07.256 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000084cda0769c8f5984-006312f91f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.258 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:07.259 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:07.259 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:07.260 DEBUG:boto:Method: HEAD 2022-09-03T06:50:07.261 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:50:07.262 DEBUG:boto:Data: 2022-09-03T06:50:07.263 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.263 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:07.264 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.264 DEBUG:boto:Params: {} 2022-09-03T06:50:07.265 DEBUG:boto:Token: None 2022-09-03T06:50:07.266 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj44 2022-09-03T06:50:07.267 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:45Vz/lKxwIFzmkna8M30PDgaRno= 2022-09-03T06:50:07.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:45Vz/lKxwIFzmkna8M30PDgaRno=', 'Content-Length': '0'} 2022-09-03T06:50:07.271 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000c2aec94fb78f033-006312f91f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.272 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:07.272 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:07.273 DEBUG:boto:Method: HEAD 2022-09-03T06:50:07.273 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:50:07.274 DEBUG:boto:Data: 2022-09-03T06:50:07.274 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.275 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:07.275 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.276 DEBUG:boto:Params: {} 2022-09-03T06:50:07.276 DEBUG:boto:Token: None 2022-09-03T06:50:07.277 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj44 2022-09-03T06:50:07.278 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:45Vz/lKxwIFzmkna8M30PDgaRno= 2022-09-03T06:50:07.278 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:45Vz/lKxwIFzmkna8M30PDgaRno=', 'Content-Length': '0'} 2022-09-03T06:50:07.280 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b1d810b2f91f65a-006312f91f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.281 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:50:07.282 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:07.283 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:07.284 DEBUG:boto:Method: GET 2022-09-03T06:50:07.284 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:50:07.285 DEBUG:boto:Data: 2022-09-03T06:50:07.285 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.286 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:07.286 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.287 DEBUG:boto:Params: {} 2022-09-03T06:50:07.288 DEBUG:boto:Token: None 2022-09-03T06:50:07.288 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj44 2022-09-03T06:50:07.289 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QMq36C3fkg+z7QMYXKkWAWXjZ6c= 2022-09-03T06:50:07.290 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QMq36C3fkg+z7QMYXKkWAWXjZ6c=', 'Content-Length': '0'} 2022-09-03T06:50:07.292 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002fb5aec0c2f1fb06-006312f91f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.293 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:07.293 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:07.294 DEBUG:boto:Method: GET 2022-09-03T06:50:07.295 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:50:07.295 DEBUG:boto:Data: 2022-09-03T06:50:07.296 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.296 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:07.297 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.297 DEBUG:boto:Params: {} 2022-09-03T06:50:07.298 DEBUG:boto:Token: None 2022-09-03T06:50:07.299 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj44 2022-09-03T06:50:07.299 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QMq36C3fkg+z7QMYXKkWAWXjZ6c= 2022-09-03T06:50:07.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QMq36C3fkg+z7QMYXKkWAWXjZ6c=', 'Content-Length': '0'} 2022-09-03T06:50:07.302 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000533bfa1fc0560939-006312f91f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.303 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:07.304 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:07.305 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:07.305 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:07.306 DEBUG:boto:Method: GET 2022-09-03T06:50:07.307 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:50:07.307 DEBUG:boto:Data: 2022-09-03T06:50:07.308 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.309 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:07.309 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.310 DEBUG:boto:Params: {} 2022-09-03T06:50:07.311 DEBUG:boto:Token: None 2022-09-03T06:50:07.313 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/ 2022-09-03T06:50:07.314 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k= 2022-09-03T06:50:07.315 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k=', 'Content-Length': '0'} 2022-09-03T06:50:07.318 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bae69ded097cba68-006312f91f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.319 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:07.320 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:07.321 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:07.321 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:07.322 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:07.323 DEBUG:boto:Method: GET 2022-09-03T06:50:07.323 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:50:07.324 DEBUG:boto:Data: 2022-09-03T06:50:07.324 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.325 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:07.325 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.326 DEBUG:boto:Params: {} 2022-09-03T06:50:07.327 DEBUG:boto:Token: None 2022-09-03T06:50:07.327 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/ 2022-09-03T06:50:07.328 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k= 2022-09-03T06:50:07.329 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k=', 'Content-Length': '0'} 2022-09-03T06:50:07.333 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b3f9471d95756cd2-006312f91f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.378 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:07.380 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:50:07.380 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:07.381 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:07.382 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.382 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.383 DEBUG:boto:Method: GET 2022-09-03T06:50:07.384 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.384 DEBUG:boto:Data: 2022-09-03T06:50:07.385 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.386 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:07.387 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.388 DEBUG:boto:Params: {} 2022-09-03T06:50:07.388 DEBUG:boto:Token: None 2022-09-03T06:50:07.389 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.390 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6nXX06Qu9NmYLcJNuiveua5jdBU= 2022-09-03T06:50:07.390 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6nXX06Qu9NmYLcJNuiveua5jdBU=', 'Content-Length': '0'} 2022-09-03T06:50:07.393 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000684e44dd33deb21d-006312f91f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.394 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:07.394 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:07.395 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.395 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.396 DEBUG:boto:Method: GET 2022-09-03T06:50:07.397 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.397 DEBUG:boto:Data: 2022-09-03T06:50:07.398 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.398 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:07.400 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.401 DEBUG:boto:Params: {} 2022-09-03T06:50:07.402 DEBUG:boto:Token: None 2022-09-03T06:50:07.403 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.404 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6nXX06Qu9NmYLcJNuiveua5jdBU= 2022-09-03T06:50:07.405 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6nXX06Qu9NmYLcJNuiveua5jdBU=', 'Content-Length': '0'} 2022-09-03T06:50:07.409 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ffc36decefc1e5fd-006312f91f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.410 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:07.411 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:07.412 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.413 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.414 DEBUG:boto:Method: HEAD 2022-09-03T06:50:07.415 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.416 DEBUG:boto:Data: 2022-09-03T06:50:07.417 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.417 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:07.418 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.419 DEBUG:boto:Params: {} 2022-09-03T06:50:07.420 DEBUG:boto:Token: None 2022-09-03T06:50:07.421 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.423 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lvFQzGWS0njk8uU355GLde0SB0U= 2022-09-03T06:50:07.424 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lvFQzGWS0njk8uU355GLde0SB0U=', 'Content-Length': '0'} 2022-09-03T06:50:07.426 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001bef3a227de6d3c6-006312f91f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.427 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:07.428 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:07.429 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.430 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.431 DEBUG:boto:Method: HEAD 2022-09-03T06:50:07.432 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.433 DEBUG:boto:Data: 2022-09-03T06:50:07.434 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.435 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:07.436 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.437 DEBUG:boto:Params: {} 2022-09-03T06:50:07.438 DEBUG:boto:Token: None 2022-09-03T06:50:07.439 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.441 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lvFQzGWS0njk8uU355GLde0SB0U= 2022-09-03T06:50:07.441 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lvFQzGWS0njk8uU355GLde0SB0U=', 'Content-Length': '0'} 2022-09-03T06:50:07.444 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f93f81718b9c511b-006312f91f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.445 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:50:07.445 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:07.446 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:07.446 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.447 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.448 DEBUG:boto:Method: GET 2022-09-03T06:50:07.448 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.449 DEBUG:boto:Data: 2022-09-03T06:50:07.449 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.450 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:07.451 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.451 DEBUG:boto:Params: {} 2022-09-03T06:50:07.452 DEBUG:boto:Token: None 2022-09-03T06:50:07.453 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.454 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6nXX06Qu9NmYLcJNuiveua5jdBU= 2022-09-03T06:50:07.455 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6nXX06Qu9NmYLcJNuiveua5jdBU=', 'Content-Length': '0'} 2022-09-03T06:50:07.457 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000049ff9fbc32a2b2d7-006312f91f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.498 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:07.499 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:07.500 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.504 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.506 DEBUG:boto:Method: GET 2022-09-03T06:50:07.508 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.509 DEBUG:boto:Data: 2022-09-03T06:50:07.510 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.511 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:07.512 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.513 DEBUG:boto:Params: {} 2022-09-03T06:50:07.516 DEBUG:boto:Token: None 2022-09-03T06:50:07.517 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:07.519 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6nXX06Qu9NmYLcJNuiveua5jdBU= 2022-09-03T06:50:07.520 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6nXX06Qu9NmYLcJNuiveua5jdBU=', 'Content-Length': '0'} 2022-09-03T06:50:07.525 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006a98365733cfefba-006312f91f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.527 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:07.528 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:07.529 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:07.530 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:07.531 DEBUG:boto:Method: GET 2022-09-03T06:50:07.532 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:50:07.533 DEBUG:boto:Data: 2022-09-03T06:50:07.534 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.535 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:07.535 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.536 DEBUG:boto:Params: {} 2022-09-03T06:50:07.537 DEBUG:boto:Token: None 2022-09-03T06:50:07.538 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/ 2022-09-03T06:50:07.540 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k= 2022-09-03T06:50:07.541 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k=', 'Content-Length': '0'} 2022-09-03T06:50:07.548 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000061b086c411feed1a-006312f91f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.549 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:07.550 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:07.551 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:07.551 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:07.552 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:07.552 DEBUG:boto:Method: GET 2022-09-03T06:50:07.553 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:50:07.554 DEBUG:boto:Data: 2022-09-03T06:50:07.555 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.556 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:07.557 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.559 DEBUG:boto:Params: {} 2022-09-03T06:50:07.560 DEBUG:boto:Token: None 2022-09-03T06:50:07.561 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/ 2022-09-03T06:50:07.568 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k= 2022-09-03T06:50:07.569 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k=', 'Content-Length': '0'} 2022-09-03T06:50:07.575 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007a2edf22cdbd2eb0-006312f91f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.576 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:07.577 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:07.577 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:07.578 DEBUG:boto:Method: HEAD 2022-09-03T06:50:07.579 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:50:07.579 DEBUG:boto:Data: 2022-09-03T06:50:07.580 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.580 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:07.581 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.582 DEBUG:boto:Params: {} 2022-09-03T06:50:07.582 DEBUG:boto:Token: None 2022-09-03T06:50:07.583 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj46 2022-09-03T06:50:07.584 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B+Uhi2yuC08bPRmBvimzCXZsR+A= 2022-09-03T06:50:07.585 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B+Uhi2yuC08bPRmBvimzCXZsR+A=', 'Content-Length': '0'} 2022-09-03T06:50:07.587 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cef2d2a193fcebf2-006312f91f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.588 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:07.588 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:07.589 DEBUG:boto:Method: HEAD 2022-09-03T06:50:07.589 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:50:07.590 DEBUG:boto:Data: 2022-09-03T06:50:07.591 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.592 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:07.593 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.593 DEBUG:boto:Params: {} 2022-09-03T06:50:07.594 DEBUG:boto:Token: None 2022-09-03T06:50:07.595 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj46 2022-09-03T06:50:07.595 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B+Uhi2yuC08bPRmBvimzCXZsR+A= 2022-09-03T06:50:07.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B+Uhi2yuC08bPRmBvimzCXZsR+A=', 'Content-Length': '0'} 2022-09-03T06:50:07.599 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000062572ced89323244-006312f91f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.599 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:50:07.600 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:07.600 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:07.601 DEBUG:boto:Method: GET 2022-09-03T06:50:07.602 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:50:07.602 DEBUG:boto:Data: 2022-09-03T06:50:07.603 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.603 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:07.604 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.604 DEBUG:boto:Params: {} 2022-09-03T06:50:07.605 DEBUG:boto:Token: None 2022-09-03T06:50:07.606 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj46 2022-09-03T06:50:07.607 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XbQl4j6JrYKSvunQEnLn/zAG0Eg= 2022-09-03T06:50:07.607 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XbQl4j6JrYKSvunQEnLn/zAG0Eg=', 'Content-Length': '0'} 2022-09-03T06:50:07.610 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bf40593c3646925a-006312f91f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.612 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:07.612 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:07.613 DEBUG:boto:Method: GET 2022-09-03T06:50:07.613 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:50:07.614 DEBUG:boto:Data: 2022-09-03T06:50:07.615 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.616 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:07.617 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.618 DEBUG:boto:Params: {} 2022-09-03T06:50:07.620 DEBUG:boto:Token: None 2022-09-03T06:50:07.621 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj46 2022-09-03T06:50:07.623 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XbQl4j6JrYKSvunQEnLn/zAG0Eg= 2022-09-03T06:50:07.624 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XbQl4j6JrYKSvunQEnLn/zAG0Eg=', 'Content-Length': '0'} 2022-09-03T06:50:07.627 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000830973b536f36345-006312f91f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.670 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:07.672 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:07.673 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:07.673 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:07.675 DEBUG:boto:Method: GET 2022-09-03T06:50:07.676 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:50:07.676 DEBUG:boto:Data: 2022-09-03T06:50:07.677 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.678 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:07.679 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.680 DEBUG:boto:Params: {} 2022-09-03T06:50:07.680 DEBUG:boto:Token: None 2022-09-03T06:50:07.681 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/ 2022-09-03T06:50:07.682 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k= 2022-09-03T06:50:07.683 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k=', 'Content-Length': '0'} 2022-09-03T06:50:07.686 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d1028c3a38819da4-006312f91f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.687 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:07.688 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:07.688 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:07.689 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:07.689 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:07.690 DEBUG:boto:Method: GET 2022-09-03T06:50:07.691 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:50:07.691 DEBUG:boto:Data: 2022-09-03T06:50:07.692 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.693 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:07.693 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.694 DEBUG:boto:Params: {} 2022-09-03T06:50:07.694 DEBUG:boto:Token: None 2022-09-03T06:50:07.695 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/ 2022-09-03T06:50:07.696 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k= 2022-09-03T06:50:07.696 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k=', 'Content-Length': '0'} 2022-09-03T06:50:07.701 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000049521f8537df7d3f-006312f91f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.702 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:07.703 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:50:07.704 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:07.704 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:07.705 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.706 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.707 DEBUG:boto:Method: GET 2022-09-03T06:50:07.707 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.708 DEBUG:boto:Data: 2022-09-03T06:50:07.709 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.710 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:07.711 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.712 DEBUG:boto:Params: {} 2022-09-03T06:50:07.714 DEBUG:boto:Token: None 2022-09-03T06:50:07.715 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.716 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8x/FSXHqq0K9xwCfRMjhzhmYjmU= 2022-09-03T06:50:07.717 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8x/FSXHqq0K9xwCfRMjhzhmYjmU=', 'Content-Length': '0'} 2022-09-03T06:50:07.720 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e2cddbd8e70510db-006312f91f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.721 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:07.722 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:07.723 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.724 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.726 DEBUG:boto:Method: GET 2022-09-03T06:50:07.727 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.728 DEBUG:boto:Data: 2022-09-03T06:50:07.729 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.730 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:07.731 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.732 DEBUG:boto:Params: {} 2022-09-03T06:50:07.733 DEBUG:boto:Token: None 2022-09-03T06:50:07.735 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.736 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8x/FSXHqq0K9xwCfRMjhzhmYjmU= 2022-09-03T06:50:07.737 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8x/FSXHqq0K9xwCfRMjhzhmYjmU=', 'Content-Length': '0'} 2022-09-03T06:50:07.740 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000327822ab90787f23-006312f91f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.742 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:07.743 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:07.743 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.744 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.745 DEBUG:boto:Method: HEAD 2022-09-03T06:50:07.746 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.747 DEBUG:boto:Data: 2022-09-03T06:50:07.748 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.749 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:07.750 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.751 DEBUG:boto:Params: {} 2022-09-03T06:50:07.751 DEBUG:boto:Token: None 2022-09-03T06:50:07.752 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.753 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:teiEvrf4fKE1QXcE17PBDPk3SWE= 2022-09-03T06:50:07.753 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:teiEvrf4fKE1QXcE17PBDPk3SWE=', 'Content-Length': '0'} 2022-09-03T06:50:07.756 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dde1a3a219f8e88e-006312f91f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.757 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:07.758 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:07.759 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.760 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.761 DEBUG:boto:Method: HEAD 2022-09-03T06:50:07.761 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.763 DEBUG:boto:Data: 2022-09-03T06:50:07.764 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.765 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:07.766 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.767 DEBUG:boto:Params: {} 2022-09-03T06:50:07.769 DEBUG:boto:Token: None 2022-09-03T06:50:07.771 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.772 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:teiEvrf4fKE1QXcE17PBDPk3SWE= 2022-09-03T06:50:07.773 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:teiEvrf4fKE1QXcE17PBDPk3SWE=', 'Content-Length': '0'} 2022-09-03T06:50:07.776 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000058f3aef168f78825-006312f91f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.778 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:50:07.780 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:07.780 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:07.781 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.781 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.782 DEBUG:boto:Method: GET 2022-09-03T06:50:07.783 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.783 DEBUG:boto:Data: 2022-09-03T06:50:07.784 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.785 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:07.785 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.786 DEBUG:boto:Params: {} 2022-09-03T06:50:07.787 DEBUG:boto:Token: None 2022-09-03T06:50:07.788 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.788 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8x/FSXHqq0K9xwCfRMjhzhmYjmU= 2022-09-03T06:50:07.789 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8x/FSXHqq0K9xwCfRMjhzhmYjmU=', 'Content-Length': '0'} 2022-09-03T06:50:07.791 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a69a5b54a55d5c01-006312f91f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.792 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:07.793 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:07.793 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.794 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.795 DEBUG:boto:Method: GET 2022-09-03T06:50:07.795 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.796 DEBUG:boto:Data: 2022-09-03T06:50:07.796 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.797 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:07.798 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.799 DEBUG:boto:Params: {} 2022-09-03T06:50:07.800 DEBUG:boto:Token: None 2022-09-03T06:50:07.801 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:07.802 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8x/FSXHqq0K9xwCfRMjhzhmYjmU= 2022-09-03T06:50:07.803 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8x/FSXHqq0K9xwCfRMjhzhmYjmU=', 'Content-Length': '0'} 2022-09-03T06:50:07.805 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000002ca41a54e0ead34-006312f91f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.807 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:07.808 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:07.809 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:07.810 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:07.811 DEBUG:boto:Method: GET 2022-09-03T06:50:07.812 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:50:07.813 DEBUG:boto:Data: 2022-09-03T06:50:07.814 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.815 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:07.816 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.817 DEBUG:boto:Params: {} 2022-09-03T06:50:07.818 DEBUG:boto:Token: None 2022-09-03T06:50:07.819 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/ 2022-09-03T06:50:07.820 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k= 2022-09-03T06:50:07.821 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k=', 'Content-Length': '0'} 2022-09-03T06:50:07.825 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000edaeb4fa1b200433-006312f91f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.826 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:07.828 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:07.829 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:07.830 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:07.831 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:07.832 DEBUG:boto:Method: GET 2022-09-03T06:50:07.832 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:50:07.833 DEBUG:boto:Data: 2022-09-03T06:50:07.834 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.835 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:07.836 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.837 DEBUG:boto:Params: {} 2022-09-03T06:50:07.838 DEBUG:boto:Token: None 2022-09-03T06:50:07.839 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/ 2022-09-03T06:50:07.840 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k= 2022-09-03T06:50:07.841 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k=', 'Content-Length': '0'} 2022-09-03T06:50:07.847 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b5d21cba93728519-006312f91f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.848 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:07.849 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:07.850 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:07.851 DEBUG:boto:Method: HEAD 2022-09-03T06:50:07.851 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:50:07.852 DEBUG:boto:Data: 2022-09-03T06:50:07.853 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.853 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:07.854 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.855 DEBUG:boto:Params: {} 2022-09-03T06:50:07.855 DEBUG:boto:Token: None 2022-09-03T06:50:07.857 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj51 2022-09-03T06:50:07.857 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xHwlKF8q1BJHsTomnXiM49kBrlg= 2022-09-03T06:50:07.858 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xHwlKF8q1BJHsTomnXiM49kBrlg=', 'Content-Length': '0'} 2022-09-03T06:50:07.860 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004e066100517ca246-006312f91f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.861 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:07.861 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:07.862 DEBUG:boto:Method: HEAD 2022-09-03T06:50:07.862 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:50:07.863 DEBUG:boto:Data: 2022-09-03T06:50:07.863 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.864 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:07.864 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.865 DEBUG:boto:Params: {} 2022-09-03T06:50:07.865 DEBUG:boto:Token: None 2022-09-03T06:50:07.866 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj51 2022-09-03T06:50:07.867 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xHwlKF8q1BJHsTomnXiM49kBrlg= 2022-09-03T06:50:07.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xHwlKF8q1BJHsTomnXiM49kBrlg=', 'Content-Length': '0'} 2022-09-03T06:50:07.870 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e38c9f233c18a4a8-006312f91f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.871 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:50:07.871 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:07.872 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:07.873 DEBUG:boto:Method: GET 2022-09-03T06:50:07.873 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:50:07.874 DEBUG:boto:Data: 2022-09-03T06:50:07.874 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.875 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:07.875 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.876 DEBUG:boto:Params: {} 2022-09-03T06:50:07.877 DEBUG:boto:Token: None 2022-09-03T06:50:07.877 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj51 2022-09-03T06:50:07.878 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FKS3Pc8rFfwiE9auXBE+iXCNiEs= 2022-09-03T06:50:07.879 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FKS3Pc8rFfwiE9auXBE+iXCNiEs=', 'Content-Length': '0'} 2022-09-03T06:50:07.881 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a056c01ebd6ecfa-006312f91f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.882 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:07.883 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:07.884 DEBUG:boto:Method: GET 2022-09-03T06:50:07.885 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:50:07.885 DEBUG:boto:Data: 2022-09-03T06:50:07.886 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.887 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:07.887 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.888 DEBUG:boto:Params: {} 2022-09-03T06:50:07.889 DEBUG:boto:Token: None 2022-09-03T06:50:07.889 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj51 2022-09-03T06:50:07.890 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FKS3Pc8rFfwiE9auXBE+iXCNiEs= 2022-09-03T06:50:07.891 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FKS3Pc8rFfwiE9auXBE+iXCNiEs=', 'Content-Length': '0'} 2022-09-03T06:50:07.894 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c733c89d14a3b133-006312f91f-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.934 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:07.935 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:07.936 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:07.937 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:07.937 DEBUG:boto:Method: GET 2022-09-03T06:50:07.938 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:50:07.938 DEBUG:boto:Data: 2022-09-03T06:50:07.939 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.939 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:07.940 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.941 DEBUG:boto:Params: {} 2022-09-03T06:50:07.941 DEBUG:boto:Token: None 2022-09-03T06:50:07.943 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/ 2022-09-03T06:50:07.944 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k= 2022-09-03T06:50:07.945 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k=', 'Content-Length': '0'} 2022-09-03T06:50:07.949 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000b8544038653c0fa-006312f91f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.950 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:07.951 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:07.952 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:07.953 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:07.953 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:07.954 DEBUG:boto:Method: GET 2022-09-03T06:50:07.955 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:50:07.955 DEBUG:boto:Data: 2022-09-03T06:50:07.956 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.956 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:07.957 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.957 DEBUG:boto:Params: {} 2022-09-03T06:50:07.958 DEBUG:boto:Token: None 2022-09-03T06:50:07.960 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/ 2022-09-03T06:50:07.962 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k= 2022-09-03T06:50:07.963 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XYS3r1BlN/Qw0yVLQ8+l0YiR82k=', 'Content-Length': '0'} 2022-09-03T06:50:07.968 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c0fd8484041b309a-006312f91f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.970 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:07.972 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:50:07.973 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:07.974 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:07.975 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:07.976 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:07.978 DEBUG:boto:Method: GET 2022-09-03T06:50:07.979 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:07.980 DEBUG:boto:Data: 2022-09-03T06:50:07.981 DEBUG:boto:Headers: {} 2022-09-03T06:50:07.982 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:07.983 DEBUG:boto:Port: 8000 2022-09-03T06:50:07.985 DEBUG:boto:Params: {} 2022-09-03T06:50:07.986 DEBUG:boto:Token: None 2022-09-03T06:50:07.987 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:07 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:07.988 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4o/gyxDjXzCh38UhWYE7d7+Jzrg= 2022-09-03T06:50:07.989 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:07 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4o/gyxDjXzCh38UhWYE7d7+Jzrg=', 'Content-Length': '0'} 2022-09-03T06:50:07.997 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f616a64eb9aa4e65-006312f91f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:07 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:07.998 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:08.000 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:08.001 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:08.002 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:08.003 DEBUG:boto:Method: GET 2022-09-03T06:50:08.004 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:08.004 DEBUG:boto:Data: 2022-09-03T06:50:08.005 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.006 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.006 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.007 DEBUG:boto:Params: {} 2022-09-03T06:50:08.008 DEBUG:boto:Token: None 2022-09-03T06:50:08.008 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:08.009 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RcoMPXPYurTIKo9xQgDhEELSDUA= 2022-09-03T06:50:08.010 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RcoMPXPYurTIKo9xQgDhEELSDUA=', 'Content-Length': '0'} 2022-09-03T06:50:08.012 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000024d3fd431fe334a-006312f920-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.013 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:08.014 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:08.015 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:08.016 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:08.017 DEBUG:boto:Method: HEAD 2022-09-03T06:50:08.018 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:08.019 DEBUG:boto:Data: 2022-09-03T06:50:08.019 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.020 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:08.021 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.022 DEBUG:boto:Params: {} 2022-09-03T06:50:08.023 DEBUG:boto:Token: None 2022-09-03T06:50:08.024 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:08.026 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A8zpbFbveXLXJHU9kb+GkfyFmxE= 2022-09-03T06:50:08.027 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A8zpbFbveXLXJHU9kb+GkfyFmxE=', 'Content-Length': '0'} 2022-09-03T06:50:08.029 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ecdc6d284396caa7-006312f920-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.030 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:08.031 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:08.032 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:08.033 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:08.034 DEBUG:boto:Method: HEAD 2022-09-03T06:50:08.035 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:08.036 DEBUG:boto:Data: 2022-09-03T06:50:08.037 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.038 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.039 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.040 DEBUG:boto:Params: {} 2022-09-03T06:50:08.041 DEBUG:boto:Token: None 2022-09-03T06:50:08.042 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:08.043 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:A8zpbFbveXLXJHU9kb+GkfyFmxE= 2022-09-03T06:50:08.044 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:A8zpbFbveXLXJHU9kb+GkfyFmxE=', 'Content-Length': '0'} 2022-09-03T06:50:08.046 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000089b2b65640fc651-006312f920-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.047 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:50:08.048 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:08.048 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:08.049 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:08.050 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:08.051 DEBUG:boto:Method: GET 2022-09-03T06:50:08.051 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:08.052 DEBUG:boto:Data: 2022-09-03T06:50:08.053 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.054 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:08.055 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.056 DEBUG:boto:Params: {} 2022-09-03T06:50:08.057 DEBUG:boto:Token: None 2022-09-03T06:50:08.058 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:08.059 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RcoMPXPYurTIKo9xQgDhEELSDUA= 2022-09-03T06:50:08.060 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RcoMPXPYurTIKo9xQgDhEELSDUA=', 'Content-Length': '0'} 2022-09-03T06:50:08.063 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000016252b12ebc984db-006312f920-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.110 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:08.111 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:08.112 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:08.113 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:08.114 DEBUG:boto:Method: GET 2022-09-03T06:50:08.115 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:08.116 DEBUG:boto:Data: 2022-09-03T06:50:08.118 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.119 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.119 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.120 DEBUG:boto:Params: {} 2022-09-03T06:50:08.122 DEBUG:boto:Token: None 2022-09-03T06:50:08.123 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:08.123 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RcoMPXPYurTIKo9xQgDhEELSDUA= 2022-09-03T06:50:08.124 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RcoMPXPYurTIKo9xQgDhEELSDUA=', 'Content-Length': '0'} 2022-09-03T06:50:08.126 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fa97e91c2c36caeb-006312f920-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.127 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:08.128 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:08.128 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:08.129 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:08.129 DEBUG:boto:Method: GET 2022-09-03T06:50:08.130 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:50:08.131 DEBUG:boto:Data: 2022-09-03T06:50:08.132 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.132 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:08.133 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.133 DEBUG:boto:Params: {} 2022-09-03T06:50:08.134 DEBUG:boto:Token: None 2022-09-03T06:50:08.135 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/ 2022-09-03T06:50:08.135 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0= 2022-09-03T06:50:08.136 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0=', 'Content-Length': '0'} 2022-09-03T06:50:08.139 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000093acb403100bf4e2-006312f920-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.140 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:08.141 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:08.142 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:08.142 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:08.143 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:08.143 DEBUG:boto:Method: GET 2022-09-03T06:50:08.144 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:50:08.144 DEBUG:boto:Data: 2022-09-03T06:50:08.145 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.146 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.147 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.147 DEBUG:boto:Params: {} 2022-09-03T06:50:08.148 DEBUG:boto:Token: None 2022-09-03T06:50:08.149 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/ 2022-09-03T06:50:08.150 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0= 2022-09-03T06:50:08.152 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0=', 'Content-Length': '0'} 2022-09-03T06:50:08.157 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000771f2434de8958c0-006312f920-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.158 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:08.158 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:08.159 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:08.159 DEBUG:boto:Method: HEAD 2022-09-03T06:50:08.160 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:50:08.160 DEBUG:boto:Data: 2022-09-03T06:50:08.161 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.161 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:08.162 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.162 DEBUG:boto:Params: {} 2022-09-03T06:50:08.163 DEBUG:boto:Token: None 2022-09-03T06:50:08.164 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj55 2022-09-03T06:50:08.164 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w54lD3Jny/j8xkStaa/gqoDxF+8= 2022-09-03T06:50:08.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w54lD3Jny/j8xkStaa/gqoDxF+8=', 'Content-Length': '0'} 2022-09-03T06:50:08.167 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000012812c4ef3039077-006312f920-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.168 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:08.168 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:08.169 DEBUG:boto:Method: HEAD 2022-09-03T06:50:08.169 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:50:08.170 DEBUG:boto:Data: 2022-09-03T06:50:08.170 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.171 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.171 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.172 DEBUG:boto:Params: {} 2022-09-03T06:50:08.172 DEBUG:boto:Token: None 2022-09-03T06:50:08.173 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj55 2022-09-03T06:50:08.173 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w54lD3Jny/j8xkStaa/gqoDxF+8= 2022-09-03T06:50:08.174 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w54lD3Jny/j8xkStaa/gqoDxF+8=', 'Content-Length': '0'} 2022-09-03T06:50:08.176 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000087f239113f3a88e6-006312f920-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.177 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:50:08.177 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:08.178 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:08.179 DEBUG:boto:Method: GET 2022-09-03T06:50:08.179 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:50:08.180 DEBUG:boto:Data: 2022-09-03T06:50:08.181 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.181 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:08.182 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.182 DEBUG:boto:Params: {} 2022-09-03T06:50:08.183 DEBUG:boto:Token: None 2022-09-03T06:50:08.184 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj55 2022-09-03T06:50:08.184 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iJGSPzFrGOhUgS70d9DnwGjWZwE= 2022-09-03T06:50:08.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iJGSPzFrGOhUgS70d9DnwGjWZwE=', 'Content-Length': '0'} 2022-09-03T06:50:08.187 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b9120b30baf41a30-006312f920-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.230 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:08.231 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:08.231 DEBUG:boto:Method: GET 2022-09-03T06:50:08.232 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:50:08.232 DEBUG:boto:Data: 2022-09-03T06:50:08.233 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.234 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.234 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.235 DEBUG:boto:Params: {} 2022-09-03T06:50:08.236 DEBUG:boto:Token: None 2022-09-03T06:50:08.237 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj55 2022-09-03T06:50:08.238 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iJGSPzFrGOhUgS70d9DnwGjWZwE= 2022-09-03T06:50:08.239 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iJGSPzFrGOhUgS70d9DnwGjWZwE=', 'Content-Length': '0'} 2022-09-03T06:50:08.242 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000068519d8eeb396b95-006312f920-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.243 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:08.244 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:08.245 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:08.246 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:08.247 DEBUG:boto:Method: GET 2022-09-03T06:50:08.248 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:50:08.249 DEBUG:boto:Data: 2022-09-03T06:50:08.250 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.251 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:08.251 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.252 DEBUG:boto:Params: {} 2022-09-03T06:50:08.253 DEBUG:boto:Token: None 2022-09-03T06:50:08.253 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/ 2022-09-03T06:50:08.254 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0= 2022-09-03T06:50:08.255 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0=', 'Content-Length': '0'} 2022-09-03T06:50:08.257 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fcb35f6f55b37a12-006312f920-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.258 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:08.259 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:08.260 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:08.261 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:08.261 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:08.262 DEBUG:boto:Method: GET 2022-09-03T06:50:08.263 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:50:08.263 DEBUG:boto:Data: 2022-09-03T06:50:08.264 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.264 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.265 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.265 DEBUG:boto:Params: {} 2022-09-03T06:50:08.266 DEBUG:boto:Token: None 2022-09-03T06:50:08.267 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/ 2022-09-03T06:50:08.268 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0= 2022-09-03T06:50:08.268 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0=', 'Content-Length': '0'} 2022-09-03T06:50:08.273 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000321f749f95c60b50-006312f920-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.274 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:08.275 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:50:08.276 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:08.277 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:08.278 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.279 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.280 DEBUG:boto:Method: GET 2022-09-03T06:50:08.281 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.282 DEBUG:boto:Data: 2022-09-03T06:50:08.283 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.284 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:08.285 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.286 DEBUG:boto:Params: {} 2022-09-03T06:50:08.287 DEBUG:boto:Token: None 2022-09-03T06:50:08.288 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.290 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FjO2K3f2YLMh/v7FJLInv35UFLA= 2022-09-03T06:50:08.291 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FjO2K3f2YLMh/v7FJLInv35UFLA=', 'Content-Length': '0'} 2022-09-03T06:50:08.293 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000039817e1e62f8ce99-006312f920-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.294 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:08.294 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:08.295 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.296 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.296 DEBUG:boto:Method: GET 2022-09-03T06:50:08.297 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.298 DEBUG:boto:Data: 2022-09-03T06:50:08.298 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.299 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.299 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.300 DEBUG:boto:Params: {} 2022-09-03T06:50:08.301 DEBUG:boto:Token: None 2022-09-03T06:50:08.301 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.302 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FjO2K3f2YLMh/v7FJLInv35UFLA= 2022-09-03T06:50:08.303 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FjO2K3f2YLMh/v7FJLInv35UFLA=', 'Content-Length': '0'} 2022-09-03T06:50:08.306 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000091ae7cc9664d8fb8-006312f920-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.308 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:08.309 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:08.310 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.311 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.312 DEBUG:boto:Method: HEAD 2022-09-03T06:50:08.313 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.314 DEBUG:boto:Data: 2022-09-03T06:50:08.315 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.315 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:08.316 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.317 DEBUG:boto:Params: {} 2022-09-03T06:50:08.317 DEBUG:boto:Token: None 2022-09-03T06:50:08.319 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.319 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cb1HJndQIUydLkaDZV1H6uQjgQc= 2022-09-03T06:50:08.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cb1HJndQIUydLkaDZV1H6uQjgQc=', 'Content-Length': '0'} 2022-09-03T06:50:08.322 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cb3fc83ce3ea799a-006312f920-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.323 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:08.323 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:08.324 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.324 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.325 DEBUG:boto:Method: HEAD 2022-09-03T06:50:08.325 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.326 DEBUG:boto:Data: 2022-09-03T06:50:08.326 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.327 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.327 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.328 DEBUG:boto:Params: {} 2022-09-03T06:50:08.329 DEBUG:boto:Token: None 2022-09-03T06:50:08.330 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.331 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cb1HJndQIUydLkaDZV1H6uQjgQc= 2022-09-03T06:50:08.332 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cb1HJndQIUydLkaDZV1H6uQjgQc=', 'Content-Length': '0'} 2022-09-03T06:50:08.338 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003ef4c0cef16a6cb5-006312f920-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.340 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:50:08.341 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:08.342 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:08.343 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.345 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.346 DEBUG:boto:Method: GET 2022-09-03T06:50:08.347 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.348 DEBUG:boto:Data: 2022-09-03T06:50:08.349 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.350 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:08.350 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.351 DEBUG:boto:Params: {} 2022-09-03T06:50:08.352 DEBUG:boto:Token: None 2022-09-03T06:50:08.352 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.353 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FjO2K3f2YLMh/v7FJLInv35UFLA= 2022-09-03T06:50:08.354 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FjO2K3f2YLMh/v7FJLInv35UFLA=', 'Content-Length': '0'} 2022-09-03T06:50:08.356 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d763de705db54b71-006312f920-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.357 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:08.358 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:08.359 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.359 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.360 DEBUG:boto:Method: GET 2022-09-03T06:50:08.360 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.361 DEBUG:boto:Data: 2022-09-03T06:50:08.361 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.362 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.363 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.363 DEBUG:boto:Params: {} 2022-09-03T06:50:08.364 DEBUG:boto:Token: None 2022-09-03T06:50:08.365 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:08.365 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FjO2K3f2YLMh/v7FJLInv35UFLA= 2022-09-03T06:50:08.366 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FjO2K3f2YLMh/v7FJLInv35UFLA=', 'Content-Length': '0'} 2022-09-03T06:50:08.368 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f6a5b73918acf09b-006312f920-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.371 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:08.372 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:08.373 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:08.374 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:08.375 DEBUG:boto:Method: GET 2022-09-03T06:50:08.376 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:50:08.377 DEBUG:boto:Data: 2022-09-03T06:50:08.378 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.379 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:08.380 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.381 DEBUG:boto:Params: {} 2022-09-03T06:50:08.382 DEBUG:boto:Token: None 2022-09-03T06:50:08.383 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/ 2022-09-03T06:50:08.388 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0= 2022-09-03T06:50:08.390 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0=', 'Content-Length': '0'} 2022-09-03T06:50:08.393 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009ac0b94f5a8ba9c2-006312f920-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.394 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:08.396 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:08.397 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:08.398 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:08.399 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:08.400 DEBUG:boto:Method: GET 2022-09-03T06:50:08.401 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:50:08.402 DEBUG:boto:Data: 2022-09-03T06:50:08.403 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.403 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.404 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.405 DEBUG:boto:Params: {} 2022-09-03T06:50:08.405 DEBUG:boto:Token: None 2022-09-03T06:50:08.406 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/ 2022-09-03T06:50:08.407 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0= 2022-09-03T06:50:08.407 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0=', 'Content-Length': '0'} 2022-09-03T06:50:08.417 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ea8a77eca7687293-006312f920-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.418 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:08.419 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:08.419 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:08.420 DEBUG:boto:Method: HEAD 2022-09-03T06:50:08.421 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:50:08.421 DEBUG:boto:Data: 2022-09-03T06:50:08.422 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.423 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:08.428 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.431 DEBUG:boto:Params: {} 2022-09-03T06:50:08.432 DEBUG:boto:Token: None 2022-09-03T06:50:08.433 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj57 2022-09-03T06:50:08.434 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fiQEEyt9FHAMCBGfJi6mol2Xp3g= 2022-09-03T06:50:08.435 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fiQEEyt9FHAMCBGfJi6mol2Xp3g=', 'Content-Length': '0'} 2022-09-03T06:50:08.438 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006e00a9d4231a4c5b-006312f920-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.440 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:08.441 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:08.442 DEBUG:boto:Method: HEAD 2022-09-03T06:50:08.443 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:50:08.444 DEBUG:boto:Data: 2022-09-03T06:50:08.445 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.446 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.447 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.449 DEBUG:boto:Params: {} 2022-09-03T06:50:08.452 DEBUG:boto:Token: None 2022-09-03T06:50:08.454 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj57 2022-09-03T06:50:08.455 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fiQEEyt9FHAMCBGfJi6mol2Xp3g= 2022-09-03T06:50:08.457 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fiQEEyt9FHAMCBGfJi6mol2Xp3g=', 'Content-Length': '0'} 2022-09-03T06:50:08.460 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d9e8949b014ed6c-006312f920-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.461 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:50:08.462 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:08.462 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:08.463 DEBUG:boto:Method: GET 2022-09-03T06:50:08.464 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:50:08.464 DEBUG:boto:Data: 2022-09-03T06:50:08.465 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.466 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:08.466 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.468 DEBUG:boto:Params: {} 2022-09-03T06:50:08.469 DEBUG:boto:Token: None 2022-09-03T06:50:08.470 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj57 2022-09-03T06:50:08.471 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nBaOOX0/0xdPrc6rmTvmdF5Tq6E= 2022-09-03T06:50:08.471 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nBaOOX0/0xdPrc6rmTvmdF5Tq6E=', 'Content-Length': '0'} 2022-09-03T06:50:08.474 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004744f8a9e5302ed7-006312f920-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.514 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:08.515 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:08.516 DEBUG:boto:Method: GET 2022-09-03T06:50:08.517 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:50:08.517 DEBUG:boto:Data: 2022-09-03T06:50:08.518 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.518 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.519 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.520 DEBUG:boto:Params: {} 2022-09-03T06:50:08.520 DEBUG:boto:Token: None 2022-09-03T06:50:08.521 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj57 2022-09-03T06:50:08.522 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nBaOOX0/0xdPrc6rmTvmdF5Tq6E= 2022-09-03T06:50:08.522 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nBaOOX0/0xdPrc6rmTvmdF5Tq6E=', 'Content-Length': '0'} 2022-09-03T06:50:08.525 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006786e7f775b4667a-006312f920-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.526 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:08.527 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:08.527 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:08.528 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:08.528 DEBUG:boto:Method: GET 2022-09-03T06:50:08.529 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:50:08.530 DEBUG:boto:Data: 2022-09-03T06:50:08.530 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.531 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:08.531 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.532 DEBUG:boto:Params: {} 2022-09-03T06:50:08.532 DEBUG:boto:Token: None 2022-09-03T06:50:08.533 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/ 2022-09-03T06:50:08.534 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0= 2022-09-03T06:50:08.534 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0=', 'Content-Length': '0'} 2022-09-03T06:50:08.538 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000070f21e3e0b783404-006312f920-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.539 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:08.541 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:08.542 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:08.542 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:08.543 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:08.544 DEBUG:boto:Method: GET 2022-09-03T06:50:08.545 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:50:08.546 DEBUG:boto:Data: 2022-09-03T06:50:08.547 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.548 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.549 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.550 DEBUG:boto:Params: {} 2022-09-03T06:50:08.551 DEBUG:boto:Token: None 2022-09-03T06:50:08.552 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/ 2022-09-03T06:50:08.553 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0= 2022-09-03T06:50:08.554 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0=', 'Content-Length': '0'} 2022-09-03T06:50:08.559 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000054a899a0bf5b28f9-006312f920-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.560 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:08.561 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:50:08.562 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:08.562 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:08.563 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.564 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.565 DEBUG:boto:Method: GET 2022-09-03T06:50:08.566 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.567 DEBUG:boto:Data: 2022-09-03T06:50:08.568 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.569 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:08.570 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.571 DEBUG:boto:Params: {} 2022-09-03T06:50:08.572 DEBUG:boto:Token: None 2022-09-03T06:50:08.573 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.574 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2SqVlBwZu+rR3ruDVtVSqMHsH2U= 2022-09-03T06:50:08.575 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2SqVlBwZu+rR3ruDVtVSqMHsH2U=', 'Content-Length': '0'} 2022-09-03T06:50:08.578 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000058357667db0cd323-006312f920-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.581 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:08.582 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:08.583 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.585 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.586 DEBUG:boto:Method: GET 2022-09-03T06:50:08.587 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.588 DEBUG:boto:Data: 2022-09-03T06:50:08.589 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.590 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.591 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.592 DEBUG:boto:Params: {} 2022-09-03T06:50:08.593 DEBUG:boto:Token: None 2022-09-03T06:50:08.594 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.595 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2SqVlBwZu+rR3ruDVtVSqMHsH2U= 2022-09-03T06:50:08.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2SqVlBwZu+rR3ruDVtVSqMHsH2U=', 'Content-Length': '0'} 2022-09-03T06:50:08.599 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c22a8ea5e7531cb-006312f920-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.642 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:08.644 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:08.644 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.645 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.646 DEBUG:boto:Method: HEAD 2022-09-03T06:50:08.646 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.647 DEBUG:boto:Data: 2022-09-03T06:50:08.647 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.648 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:08.648 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.649 DEBUG:boto:Params: {} 2022-09-03T06:50:08.649 DEBUG:boto:Token: None 2022-09-03T06:50:08.650 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.651 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hAnCNzGSL4vG4UdHJHHSn6p4wUA= 2022-09-03T06:50:08.651 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hAnCNzGSL4vG4UdHJHHSn6p4wUA=', 'Content-Length': '0'} 2022-09-03T06:50:08.653 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000037ea46280b9f7c36-006312f920-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.654 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:08.655 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:08.655 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.656 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.656 DEBUG:boto:Method: HEAD 2022-09-03T06:50:08.657 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.657 DEBUG:boto:Data: 2022-09-03T06:50:08.658 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.658 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.659 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.659 DEBUG:boto:Params: {} 2022-09-03T06:50:08.660 DEBUG:boto:Token: None 2022-09-03T06:50:08.661 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.662 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hAnCNzGSL4vG4UdHJHHSn6p4wUA= 2022-09-03T06:50:08.662 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hAnCNzGSL4vG4UdHJHHSn6p4wUA=', 'Content-Length': '0'} 2022-09-03T06:50:08.664 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dc225d9337d4f96d-006312f920-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.665 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:50:08.666 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:08.666 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:08.667 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.667 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.668 DEBUG:boto:Method: GET 2022-09-03T06:50:08.668 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.669 DEBUG:boto:Data: 2022-09-03T06:50:08.670 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.670 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:08.671 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.671 DEBUG:boto:Params: {} 2022-09-03T06:50:08.672 DEBUG:boto:Token: None 2022-09-03T06:50:08.673 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.673 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2SqVlBwZu+rR3ruDVtVSqMHsH2U= 2022-09-03T06:50:08.674 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2SqVlBwZu+rR3ruDVtVSqMHsH2U=', 'Content-Length': '0'} 2022-09-03T06:50:08.676 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001da813be14b090a3-006312f920-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.677 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:08.677 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:08.678 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.678 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.679 DEBUG:boto:Method: GET 2022-09-03T06:50:08.680 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.680 DEBUG:boto:Data: 2022-09-03T06:50:08.681 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.681 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.682 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.682 DEBUG:boto:Params: {} 2022-09-03T06:50:08.683 DEBUG:boto:Token: None 2022-09-03T06:50:08.684 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:08.685 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2SqVlBwZu+rR3ruDVtVSqMHsH2U= 2022-09-03T06:50:08.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2SqVlBwZu+rR3ruDVtVSqMHsH2U=', 'Content-Length': '0'} 2022-09-03T06:50:08.688 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000030c93c5d1410c5d-006312f920-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.689 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:08.690 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:08.691 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:08.691 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:08.692 DEBUG:boto:Method: GET 2022-09-03T06:50:08.693 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:50:08.694 DEBUG:boto:Data: 2022-09-03T06:50:08.695 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.696 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:08.696 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.697 DEBUG:boto:Params: {} 2022-09-03T06:50:08.698 DEBUG:boto:Token: None 2022-09-03T06:50:08.698 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/ 2022-09-03T06:50:08.699 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0= 2022-09-03T06:50:08.700 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0=', 'Content-Length': '0'} 2022-09-03T06:50:08.702 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000632c444751bdbf7d-006312f920-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.703 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:08.704 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:08.705 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:08.705 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:08.706 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:08.706 DEBUG:boto:Method: GET 2022-09-03T06:50:08.707 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:50:08.707 DEBUG:boto:Data: 2022-09-03T06:50:08.709 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.710 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.711 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.711 DEBUG:boto:Params: {} 2022-09-03T06:50:08.712 DEBUG:boto:Token: None 2022-09-03T06:50:08.713 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/ 2022-09-03T06:50:08.713 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0= 2022-09-03T06:50:08.714 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0=', 'Content-Length': '0'} 2022-09-03T06:50:08.719 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000186d93d3a25291fd-006312f920-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.720 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:08.721 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:08.721 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:08.722 DEBUG:boto:Method: HEAD 2022-09-03T06:50:08.723 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:50:08.723 DEBUG:boto:Data: 2022-09-03T06:50:08.724 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.724 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:08.725 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.725 DEBUG:boto:Params: {} 2022-09-03T06:50:08.726 DEBUG:boto:Token: None 2022-09-03T06:50:08.726 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj58 2022-09-03T06:50:08.727 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ti/897cRa+1+0OF/tTQoJILql9o= 2022-09-03T06:50:08.728 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ti/897cRa+1+0OF/tTQoJILql9o=', 'Content-Length': '0'} 2022-09-03T06:50:08.730 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f7672cddcfcc6214-006312f920-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.731 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:08.732 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:08.732 DEBUG:boto:Method: HEAD 2022-09-03T06:50:08.733 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:50:08.733 DEBUG:boto:Data: 2022-09-03T06:50:08.734 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.734 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.735 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.735 DEBUG:boto:Params: {} 2022-09-03T06:50:08.736 DEBUG:boto:Token: None 2022-09-03T06:50:08.736 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj58 2022-09-03T06:50:08.737 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ti/897cRa+1+0OF/tTQoJILql9o= 2022-09-03T06:50:08.738 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ti/897cRa+1+0OF/tTQoJILql9o=', 'Content-Length': '0'} 2022-09-03T06:50:08.740 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a332600d73b960fd-006312f920-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.741 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:50:08.742 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:08.742 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:08.743 DEBUG:boto:Method: GET 2022-09-03T06:50:08.744 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:50:08.744 DEBUG:boto:Data: 2022-09-03T06:50:08.745 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.746 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:08.746 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.747 DEBUG:boto:Params: {} 2022-09-03T06:50:08.748 DEBUG:boto:Token: None 2022-09-03T06:50:08.748 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj58 2022-09-03T06:50:08.749 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cC8ovvI9H1PkZcn5jvPxMwk2O20= 2022-09-03T06:50:08.750 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cC8ovvI9H1PkZcn5jvPxMwk2O20=', 'Content-Length': '0'} 2022-09-03T06:50:08.752 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000008c71585698fd3be-006312f920-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.753 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:08.754 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:08.755 DEBUG:boto:Method: GET 2022-09-03T06:50:08.756 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:50:08.757 DEBUG:boto:Data: 2022-09-03T06:50:08.757 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.758 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.759 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.759 DEBUG:boto:Params: {} 2022-09-03T06:50:08.760 DEBUG:boto:Token: None 2022-09-03T06:50:08.761 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj58 2022-09-03T06:50:08.762 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cC8ovvI9H1PkZcn5jvPxMwk2O20= 2022-09-03T06:50:08.762 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cC8ovvI9H1PkZcn5jvPxMwk2O20=', 'Content-Length': '0'} 2022-09-03T06:50:08.765 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000041857ff9c114f46c-006312f920-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.766 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:08.767 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:08.768 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:08.769 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:08.770 DEBUG:boto:Method: GET 2022-09-03T06:50:08.771 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:50:08.772 DEBUG:boto:Data: 2022-09-03T06:50:08.773 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.774 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:08.775 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.776 DEBUG:boto:Params: {} 2022-09-03T06:50:08.777 DEBUG:boto:Token: None 2022-09-03T06:50:08.778 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/ 2022-09-03T06:50:08.779 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0= 2022-09-03T06:50:08.780 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0=', 'Content-Length': '0'} 2022-09-03T06:50:08.783 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b6d7070f62c7c1b7-006312f920-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.784 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:08.785 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:08.786 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:08.787 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:08.788 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:08.789 DEBUG:boto:Method: GET 2022-09-03T06:50:08.790 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:50:08.791 DEBUG:boto:Data: 2022-09-03T06:50:08.792 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.793 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.793 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.794 DEBUG:boto:Params: {} 2022-09-03T06:50:08.795 DEBUG:boto:Token: None 2022-09-03T06:50:08.796 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/ 2022-09-03T06:50:08.797 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0= 2022-09-03T06:50:08.798 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vDoWIaNTU1PkJZSdDKQUW9P/ja0=', 'Content-Length': '0'} 2022-09-03T06:50:08.804 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f264c3fca16ea741-006312f920-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.806 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:08.807 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2022-09-03T06:50:08.808 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:08.809 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:08.809 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.810 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.811 DEBUG:boto:Method: GET 2022-09-03T06:50:08.811 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.812 DEBUG:boto:Data: 2022-09-03T06:50:08.813 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.814 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:08.814 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.815 DEBUG:boto:Params: {} 2022-09-03T06:50:08.815 DEBUG:boto:Token: None 2022-09-03T06:50:08.816 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.817 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3+PhsOWAxw6oWl/FhKQN6Mpijv0= 2022-09-03T06:50:08.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3+PhsOWAxw6oWl/FhKQN6Mpijv0=', 'Content-Length': '0'} 2022-09-03T06:50:08.820 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ce7271de3d5ce86b-006312f920-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.866 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:08.867 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:08.868 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.869 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.872 DEBUG:boto:Method: GET 2022-09-03T06:50:08.873 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.874 DEBUG:boto:Data: 2022-09-03T06:50:08.875 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.875 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.876 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.876 DEBUG:boto:Params: {} 2022-09-03T06:50:08.878 DEBUG:boto:Token: None 2022-09-03T06:50:08.878 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.879 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3+PhsOWAxw6oWl/FhKQN6Mpijv0= 2022-09-03T06:50:08.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3+PhsOWAxw6oWl/FhKQN6Mpijv0=', 'Content-Length': '0'} 2022-09-03T06:50:08.883 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ace4652ed9cf0b1e-006312f920-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.885 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:08.886 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:08.887 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.889 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.889 DEBUG:boto:Method: HEAD 2022-09-03T06:50:08.890 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.891 DEBUG:boto:Data: 2022-09-03T06:50:08.892 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.893 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:08.893 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.894 DEBUG:boto:Params: {} 2022-09-03T06:50:08.895 DEBUG:boto:Token: None 2022-09-03T06:50:08.897 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.898 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jsv2TqupwkNPwXdZ3DbxwZ1Z0VE= 2022-09-03T06:50:08.899 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jsv2TqupwkNPwXdZ3DbxwZ1Z0VE=', 'Content-Length': '0'} 2022-09-03T06:50:08.903 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000056766c5497abed35-006312f920-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.904 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:08.905 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:08.906 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.907 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.908 DEBUG:boto:Method: HEAD 2022-09-03T06:50:08.909 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.910 DEBUG:boto:Data: 2022-09-03T06:50:08.910 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.911 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.912 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.913 DEBUG:boto:Params: {} 2022-09-03T06:50:08.914 DEBUG:boto:Token: None 2022-09-03T06:50:08.916 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.917 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Jsv2TqupwkNPwXdZ3DbxwZ1Z0VE= 2022-09-03T06:50:08.918 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Jsv2TqupwkNPwXdZ3DbxwZ1Z0VE=', 'Content-Length': '0'} 2022-09-03T06:50:08.921 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000045db641a75a5cd69-006312f920-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.923 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2022-09-03T06:50:08.924 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:08.925 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:08.926 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.927 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.927 DEBUG:boto:Method: GET 2022-09-03T06:50:08.928 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.928 DEBUG:boto:Data: 2022-09-03T06:50:08.929 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.930 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:08.931 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.932 DEBUG:boto:Params: {} 2022-09-03T06:50:08.932 DEBUG:boto:Token: None 2022-09-03T06:50:08.933 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.934 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3+PhsOWAxw6oWl/FhKQN6Mpijv0= 2022-09-03T06:50:08.935 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3+PhsOWAxw6oWl/FhKQN6Mpijv0=', 'Content-Length': '0'} 2022-09-03T06:50:08.937 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000502b453050eb2a59-006312f920-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.939 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:08.940 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:08.941 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.942 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.943 DEBUG:boto:Method: GET 2022-09-03T06:50:08.944 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.945 DEBUG:boto:Data: 2022-09-03T06:50:08.945 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.946 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:08.946 DEBUG:boto:Port: 8000 2022-09-03T06:50:08.947 DEBUG:boto:Params: {} 2022-09-03T06:50:08.948 DEBUG:boto:Token: None 2022-09-03T06:50:08.948 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:08 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:08.949 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3+PhsOWAxw6oWl/FhKQN6Mpijv0= 2022-09-03T06:50:08.950 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:08 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3+PhsOWAxw6oWl/FhKQN6Mpijv0=', 'Content-Length': '0'} 2022-09-03T06:50:08.952 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f86232b10c570b13-006312f920-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:08 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:08.994 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:08.995 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:08.996 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:08.996 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:08.997 DEBUG:boto:Method: GET 2022-09-03T06:50:08.997 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:50:08.998 DEBUG:boto:Data: 2022-09-03T06:50:08.999 DEBUG:boto:Headers: {} 2022-09-03T06:50:08.999 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:09.000 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.000 DEBUG:boto:Params: {} 2022-09-03T06:50:09.001 DEBUG:boto:Token: None 2022-09-03T06:50:09.002 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/ 2022-09-03T06:50:09.002 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800= 2022-09-03T06:50:09.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800=', 'Content-Length': '0'} 2022-09-03T06:50:09.006 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fba28921692b9c6e-006312f921-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.007 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:09.007 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:09.008 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:09.009 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:09.009 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:09.010 DEBUG:boto:Method: GET 2022-09-03T06:50:09.010 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:50:09.011 DEBUG:boto:Data: 2022-09-03T06:50:09.012 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.012 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:09.013 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.013 DEBUG:boto:Params: {} 2022-09-03T06:50:09.014 DEBUG:boto:Token: None 2022-09-03T06:50:09.015 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/ 2022-09-03T06:50:09.015 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800= 2022-09-03T06:50:09.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800=', 'Content-Length': '0'} 2022-09-03T06:50:09.021 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008f08cf95dac8219e-006312f921-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.022 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:09.022 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:09.023 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:09.024 DEBUG:boto:Method: HEAD 2022-09-03T06:50:09.024 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:50:09.025 DEBUG:boto:Data: 2022-09-03T06:50:09.025 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.026 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:09.026 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.027 DEBUG:boto:Params: {} 2022-09-03T06:50:09.027 DEBUG:boto:Token: None 2022-09-03T06:50:09.029 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj62 2022-09-03T06:50:09.029 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QKOIfssfMk5cCimtEyQYDbE3k5o= 2022-09-03T06:50:09.030 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QKOIfssfMk5cCimtEyQYDbE3k5o=', 'Content-Length': '0'} 2022-09-03T06:50:09.033 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000045d8e90d2ff652ff-006312f921-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.034 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:09.034 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:09.035 DEBUG:boto:Method: HEAD 2022-09-03T06:50:09.036 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:50:09.036 DEBUG:boto:Data: 2022-09-03T06:50:09.037 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.037 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:09.038 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.038 DEBUG:boto:Params: {} 2022-09-03T06:50:09.039 DEBUG:boto:Token: None 2022-09-03T06:50:09.040 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj62 2022-09-03T06:50:09.040 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QKOIfssfMk5cCimtEyQYDbE3k5o= 2022-09-03T06:50:09.041 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QKOIfssfMk5cCimtEyQYDbE3k5o=', 'Content-Length': '0'} 2022-09-03T06:50:09.043 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b5b9c0dc95655cd4-006312f921-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.044 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2022-09-03T06:50:09.045 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:09.045 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:09.046 DEBUG:boto:Method: GET 2022-09-03T06:50:09.046 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:50:09.047 DEBUG:boto:Data: 2022-09-03T06:50:09.048 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.049 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:09.049 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.050 DEBUG:boto:Params: {} 2022-09-03T06:50:09.050 DEBUG:boto:Token: None 2022-09-03T06:50:09.051 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj62 2022-09-03T06:50:09.053 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Sp4YyZssi5nmLp6qqavI7+vBmeo= 2022-09-03T06:50:09.054 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Sp4YyZssi5nmLp6qqavI7+vBmeo=', 'Content-Length': '0'} 2022-09-03T06:50:09.057 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c8447de1703a2598-006312f921-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.098 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:09.099 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:09.100 DEBUG:boto:Method: GET 2022-09-03T06:50:09.101 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:50:09.102 DEBUG:boto:Data: 2022-09-03T06:50:09.103 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.104 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:09.105 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.106 DEBUG:boto:Params: {} 2022-09-03T06:50:09.107 DEBUG:boto:Token: None 2022-09-03T06:50:09.108 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj62 2022-09-03T06:50:09.110 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Sp4YyZssi5nmLp6qqavI7+vBmeo= 2022-09-03T06:50:09.111 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Sp4YyZssi5nmLp6qqavI7+vBmeo=', 'Content-Length': '0'} 2022-09-03T06:50:09.114 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd891dbdf3dda286-006312f921-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.115 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:09.116 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:09.117 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:09.118 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:09.119 DEBUG:boto:Method: GET 2022-09-03T06:50:09.120 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:50:09.121 DEBUG:boto:Data: 2022-09-03T06:50:09.121 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.122 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:09.123 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.124 DEBUG:boto:Params: {} 2022-09-03T06:50:09.125 DEBUG:boto:Token: None 2022-09-03T06:50:09.125 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/ 2022-09-03T06:50:09.126 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800= 2022-09-03T06:50:09.127 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800=', 'Content-Length': '0'} 2022-09-03T06:50:09.130 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000016054f7d1536495e-006312f921-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.131 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:09.132 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:09.133 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:09.134 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:09.134 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:09.135 DEBUG:boto:Method: GET 2022-09-03T06:50:09.135 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:50:09.136 DEBUG:boto:Data: 2022-09-03T06:50:09.137 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.138 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:09.139 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.140 DEBUG:boto:Params: {} 2022-09-03T06:50:09.143 DEBUG:boto:Token: None 2022-09-03T06:50:09.144 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/ 2022-09-03T06:50:09.145 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800= 2022-09-03T06:50:09.146 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800=', 'Content-Length': '0'} 2022-09-03T06:50:09.151 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000091fb50be4cf933d9-006312f921-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.152 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:09.155 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:50:09.156 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:09.157 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:09.158 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.159 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.160 DEBUG:boto:Method: GET 2022-09-03T06:50:09.161 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.162 DEBUG:boto:Data: 2022-09-03T06:50:09.163 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.164 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:09.165 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.166 DEBUG:boto:Params: {} 2022-09-03T06:50:09.168 DEBUG:boto:Token: None 2022-09-03T06:50:09.169 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.170 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aMi/Z7poUL3/sb+aVIqW1M/Ajaw= 2022-09-03T06:50:09.171 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aMi/Z7poUL3/sb+aVIqW1M/Ajaw=', 'Content-Length': '0'} 2022-09-03T06:50:09.178 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a797aa450dffddb-006312f921-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.179 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:09.180 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:09.181 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.182 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.183 DEBUG:boto:Method: GET 2022-09-03T06:50:09.184 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.187 DEBUG:boto:Data: 2022-09-03T06:50:09.188 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.189 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:09.191 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.192 DEBUG:boto:Params: {} 2022-09-03T06:50:09.193 DEBUG:boto:Token: None 2022-09-03T06:50:09.194 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.195 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aMi/Z7poUL3/sb+aVIqW1M/Ajaw= 2022-09-03T06:50:09.195 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aMi/Z7poUL3/sb+aVIqW1M/Ajaw=', 'Content-Length': '0'} 2022-09-03T06:50:09.198 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a12a2bce2a9dc60c-006312f921-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.199 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:09.200 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:09.202 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.203 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.204 DEBUG:boto:Method: HEAD 2022-09-03T06:50:09.204 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.205 DEBUG:boto:Data: 2022-09-03T06:50:09.206 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.207 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:09.208 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.208 DEBUG:boto:Params: {} 2022-09-03T06:50:09.209 DEBUG:boto:Token: None 2022-09-03T06:50:09.210 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.211 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hAAAm/VMFzoda8NynBh+pQ/HYpY= 2022-09-03T06:50:09.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hAAAm/VMFzoda8NynBh+pQ/HYpY=', 'Content-Length': '0'} 2022-09-03T06:50:09.216 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bbcdcd4d198ecebb-006312f921-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.217 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:09.218 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:09.218 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.219 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.220 DEBUG:boto:Method: HEAD 2022-09-03T06:50:09.220 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.221 DEBUG:boto:Data: 2022-09-03T06:50:09.221 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.222 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:09.223 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.223 DEBUG:boto:Params: {} 2022-09-03T06:50:09.224 DEBUG:boto:Token: None 2022-09-03T06:50:09.224 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.226 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hAAAm/VMFzoda8NynBh+pQ/HYpY= 2022-09-03T06:50:09.227 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hAAAm/VMFzoda8NynBh+pQ/HYpY=', 'Content-Length': '0'} 2022-09-03T06:50:09.231 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002918b05d03633f55-006312f921-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.232 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:50:09.234 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:09.235 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:09.236 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.237 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.238 DEBUG:boto:Method: GET 2022-09-03T06:50:09.239 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.240 DEBUG:boto:Data: 2022-09-03T06:50:09.241 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.243 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:09.244 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.244 DEBUG:boto:Params: {} 2022-09-03T06:50:09.246 DEBUG:boto:Token: None 2022-09-03T06:50:09.247 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.248 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aMi/Z7poUL3/sb+aVIqW1M/Ajaw= 2022-09-03T06:50:09.249 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aMi/Z7poUL3/sb+aVIqW1M/Ajaw=', 'Content-Length': '0'} 2022-09-03T06:50:09.254 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d07dfdc73d4b96de-006312f921-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.256 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:09.257 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:09.258 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.259 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.260 DEBUG:boto:Method: GET 2022-09-03T06:50:09.261 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.261 DEBUG:boto:Data: 2022-09-03T06:50:09.262 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.263 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:09.264 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.265 DEBUG:boto:Params: {} 2022-09-03T06:50:09.266 DEBUG:boto:Token: None 2022-09-03T06:50:09.268 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:09.270 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aMi/Z7poUL3/sb+aVIqW1M/Ajaw= 2022-09-03T06:50:09.271 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aMi/Z7poUL3/sb+aVIqW1M/Ajaw=', 'Content-Length': '0'} 2022-09-03T06:50:09.274 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b01d0bc3c56d747e-006312f921-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.315 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:09.371 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:09.373 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:09.374 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:09.375 DEBUG:boto:Method: GET 2022-09-03T06:50:09.376 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:50:09.377 DEBUG:boto:Data: 2022-09-03T06:50:09.380 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.381 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:09.382 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.383 DEBUG:boto:Params: {} 2022-09-03T06:50:09.384 DEBUG:boto:Token: None 2022-09-03T06:50:09.385 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/ 2022-09-03T06:50:09.386 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800= 2022-09-03T06:50:09.386 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800=', 'Content-Length': '0'} 2022-09-03T06:50:09.389 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bc3233dd97106667-006312f921-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.391 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:09.392 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:09.393 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:09.393 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:09.394 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:09.394 DEBUG:boto:Method: GET 2022-09-03T06:50:09.395 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:50:09.396 DEBUG:boto:Data: 2022-09-03T06:50:09.397 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.397 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:09.400 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.402 DEBUG:boto:Params: {} 2022-09-03T06:50:09.404 DEBUG:boto:Token: None 2022-09-03T06:50:09.404 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/ 2022-09-03T06:50:09.405 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800= 2022-09-03T06:50:09.406 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800=', 'Content-Length': '0'} 2022-09-03T06:50:09.415 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005a50cb41f35eea50-006312f921-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.416 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:09.420 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:09.421 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:09.422 DEBUG:boto:Method: HEAD 2022-09-03T06:50:09.423 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:50:09.426 DEBUG:boto:Data: 2022-09-03T06:50:09.428 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.429 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:09.430 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.431 DEBUG:boto:Params: {} 2022-09-03T06:50:09.432 DEBUG:boto:Token: None 2022-09-03T06:50:09.433 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj68 2022-09-03T06:50:09.435 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4c0naMnVtE+JBlmBGZ8drf+eY8A= 2022-09-03T06:50:09.436 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4c0naMnVtE+JBlmBGZ8drf+eY8A=', 'Content-Length': '0'} 2022-09-03T06:50:09.442 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2a49b0adeef3d34-006312f921-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.444 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:09.445 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:09.446 DEBUG:boto:Method: HEAD 2022-09-03T06:50:09.448 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:50:09.448 DEBUG:boto:Data: 2022-09-03T06:50:09.449 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.449 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:09.450 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.450 DEBUG:boto:Params: {} 2022-09-03T06:50:09.451 DEBUG:boto:Token: None 2022-09-03T06:50:09.451 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj68 2022-09-03T06:50:09.452 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4c0naMnVtE+JBlmBGZ8drf+eY8A= 2022-09-03T06:50:09.453 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4c0naMnVtE+JBlmBGZ8drf+eY8A=', 'Content-Length': '0'} 2022-09-03T06:50:09.455 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000093836b2863d5160f-006312f921-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.456 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:50:09.457 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:09.457 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:09.458 DEBUG:boto:Method: GET 2022-09-03T06:50:09.459 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:50:09.460 DEBUG:boto:Data: 2022-09-03T06:50:09.461 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.466 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:09.467 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.468 DEBUG:boto:Params: {} 2022-09-03T06:50:09.469 DEBUG:boto:Token: None 2022-09-03T06:50:09.471 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj68 2022-09-03T06:50:09.472 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:upbWqlHYg+eSk/3dQNxVBZH/l5M= 2022-09-03T06:50:09.474 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:upbWqlHYg+eSk/3dQNxVBZH/l5M=', 'Content-Length': '0'} 2022-09-03T06:50:09.477 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dbb54d7b49120a0d-006312f921-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.479 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:09.480 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:09.481 DEBUG:boto:Method: GET 2022-09-03T06:50:09.483 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:50:09.484 DEBUG:boto:Data: 2022-09-03T06:50:09.484 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.485 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:09.486 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.487 DEBUG:boto:Params: {} 2022-09-03T06:50:09.488 DEBUG:boto:Token: None 2022-09-03T06:50:09.490 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj68 2022-09-03T06:50:09.491 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:upbWqlHYg+eSk/3dQNxVBZH/l5M= 2022-09-03T06:50:09.492 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:upbWqlHYg+eSk/3dQNxVBZH/l5M=', 'Content-Length': '0'} 2022-09-03T06:50:09.495 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009bc58198986b2c8d-006312f921-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.496 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:09.497 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:09.498 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:09.499 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:09.501 DEBUG:boto:Method: GET 2022-09-03T06:50:09.502 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:50:09.503 DEBUG:boto:Data: 2022-09-03T06:50:09.504 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.505 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:09.505 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.509 DEBUG:boto:Params: {} 2022-09-03T06:50:09.511 DEBUG:boto:Token: None 2022-09-03T06:50:09.512 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/ 2022-09-03T06:50:09.513 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800= 2022-09-03T06:50:09.514 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800=', 'Content-Length': '0'} 2022-09-03T06:50:09.518 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f673607a47bc61d7-006312f921-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.519 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:09.520 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:09.521 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:09.522 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:09.523 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:09.525 DEBUG:boto:Method: GET 2022-09-03T06:50:09.526 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:50:09.527 DEBUG:boto:Data: 2022-09-03T06:50:09.527 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.528 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:09.528 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.529 DEBUG:boto:Params: {} 2022-09-03T06:50:09.529 DEBUG:boto:Token: None 2022-09-03T06:50:09.530 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/ 2022-09-03T06:50:09.531 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800= 2022-09-03T06:50:09.531 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800=', 'Content-Length': '0'} 2022-09-03T06:50:09.536 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000216f83553a711b29-006312f921-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.582 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:09.584 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:50:09.585 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:09.586 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:09.587 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.589 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.590 DEBUG:boto:Method: GET 2022-09-03T06:50:09.591 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.592 DEBUG:boto:Data: 2022-09-03T06:50:09.594 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.595 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:09.596 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.596 DEBUG:boto:Params: {} 2022-09-03T06:50:09.597 DEBUG:boto:Token: None 2022-09-03T06:50:09.597 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.598 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wvKuv4YqyZ+/YOIvHmucoXoiKdg= 2022-09-03T06:50:09.599 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wvKuv4YqyZ+/YOIvHmucoXoiKdg=', 'Content-Length': '0'} 2022-09-03T06:50:09.602 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000083ae83295d923776-006312f921-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.603 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:09.603 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:09.604 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.605 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.605 DEBUG:boto:Method: GET 2022-09-03T06:50:09.606 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.606 DEBUG:boto:Data: 2022-09-03T06:50:09.607 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.608 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:09.609 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.610 DEBUG:boto:Params: {} 2022-09-03T06:50:09.611 DEBUG:boto:Token: None 2022-09-03T06:50:09.612 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.613 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wvKuv4YqyZ+/YOIvHmucoXoiKdg= 2022-09-03T06:50:09.614 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wvKuv4YqyZ+/YOIvHmucoXoiKdg=', 'Content-Length': '0'} 2022-09-03T06:50:09.617 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000098de1e06599469b9-006312f921-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.618 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:09.619 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:09.620 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.622 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.623 DEBUG:boto:Method: HEAD 2022-09-03T06:50:09.624 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.625 DEBUG:boto:Data: 2022-09-03T06:50:09.626 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.627 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:09.628 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.629 DEBUG:boto:Params: {} 2022-09-03T06:50:09.630 DEBUG:boto:Token: None 2022-09-03T06:50:09.631 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.632 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nwB0kXlpASa/v1yH1bvIl4Vit3c= 2022-09-03T06:50:09.634 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nwB0kXlpASa/v1yH1bvIl4Vit3c=', 'Content-Length': '0'} 2022-09-03T06:50:09.637 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021bdf73f766f5b1d-006312f921-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.639 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:09.640 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:09.641 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.642 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.643 DEBUG:boto:Method: HEAD 2022-09-03T06:50:09.644 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.645 DEBUG:boto:Data: 2022-09-03T06:50:09.646 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.647 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:09.648 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.648 DEBUG:boto:Params: {} 2022-09-03T06:50:09.649 DEBUG:boto:Token: None 2022-09-03T06:50:09.650 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.651 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nwB0kXlpASa/v1yH1bvIl4Vit3c= 2022-09-03T06:50:09.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nwB0kXlpASa/v1yH1bvIl4Vit3c=', 'Content-Length': '0'} 2022-09-03T06:50:09.655 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000072b776d10ad70953-006312f921-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.656 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:50:09.656 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:09.657 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:09.658 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.658 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.659 DEBUG:boto:Method: GET 2022-09-03T06:50:09.660 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.661 DEBUG:boto:Data: 2022-09-03T06:50:09.662 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.663 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:09.664 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.665 DEBUG:boto:Params: {} 2022-09-03T06:50:09.666 DEBUG:boto:Token: None 2022-09-03T06:50:09.667 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.668 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wvKuv4YqyZ+/YOIvHmucoXoiKdg= 2022-09-03T06:50:09.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wvKuv4YqyZ+/YOIvHmucoXoiKdg=', 'Content-Length': '0'} 2022-09-03T06:50:09.672 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000106a441a1f915000-006312f921-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.674 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:09.675 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:09.676 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.677 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.678 DEBUG:boto:Method: GET 2022-09-03T06:50:09.679 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.679 DEBUG:boto:Data: 2022-09-03T06:50:09.680 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.681 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:09.682 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.683 DEBUG:boto:Params: {} 2022-09-03T06:50:09.683 DEBUG:boto:Token: None 2022-09-03T06:50:09.684 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:09.685 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wvKuv4YqyZ+/YOIvHmucoXoiKdg= 2022-09-03T06:50:09.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wvKuv4YqyZ+/YOIvHmucoXoiKdg=', 'Content-Length': '0'} 2022-09-03T06:50:09.688 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b368bacb73dee68a-006312f921-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.689 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:09.690 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:09.690 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:09.691 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:09.692 DEBUG:boto:Method: GET 2022-09-03T06:50:09.692 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:50:09.693 DEBUG:boto:Data: 2022-09-03T06:50:09.693 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.694 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:09.694 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.695 DEBUG:boto:Params: {} 2022-09-03T06:50:09.696 DEBUG:boto:Token: None 2022-09-03T06:50:09.696 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/ 2022-09-03T06:50:09.697 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800= 2022-09-03T06:50:09.699 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800=', 'Content-Length': '0'} 2022-09-03T06:50:09.702 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f210885c6e928aab-006312f921-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.703 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:09.705 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:09.706 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:09.707 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:09.708 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:09.709 DEBUG:boto:Method: GET 2022-09-03T06:50:09.710 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:50:09.711 DEBUG:boto:Data: 2022-09-03T06:50:09.712 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.713 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:09.714 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.714 DEBUG:boto:Params: {} 2022-09-03T06:50:09.715 DEBUG:boto:Token: None 2022-09-03T06:50:09.716 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/ 2022-09-03T06:50:09.716 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800= 2022-09-03T06:50:09.717 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800=', 'Content-Length': '0'} 2022-09-03T06:50:09.722 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001e6c3cb8ae5c9955-006312f921-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.723 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:09.724 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:09.724 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:09.725 DEBUG:boto:Method: HEAD 2022-09-03T06:50:09.725 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:50:09.726 DEBUG:boto:Data: 2022-09-03T06:50:09.727 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.727 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:09.728 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.728 DEBUG:boto:Params: {} 2022-09-03T06:50:09.729 DEBUG:boto:Token: None 2022-09-03T06:50:09.730 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj69 2022-09-03T06:50:09.730 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2Akfn/EoI9gVzv4mhSK/WU7X4AU= 2022-09-03T06:50:09.731 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2Akfn/EoI9gVzv4mhSK/WU7X4AU=', 'Content-Length': '0'} 2022-09-03T06:50:09.734 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000111db83a27dd41f8-006312f921-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.735 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:09.735 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:09.736 DEBUG:boto:Method: HEAD 2022-09-03T06:50:09.737 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:50:09.738 DEBUG:boto:Data: 2022-09-03T06:50:09.738 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.739 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:09.739 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.740 DEBUG:boto:Params: {} 2022-09-03T06:50:09.741 DEBUG:boto:Token: None 2022-09-03T06:50:09.742 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj69 2022-09-03T06:50:09.742 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2Akfn/EoI9gVzv4mhSK/WU7X4AU= 2022-09-03T06:50:09.743 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2Akfn/EoI9gVzv4mhSK/WU7X4AU=', 'Content-Length': '0'} 2022-09-03T06:50:09.745 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a0ad2e3c90bb04fd-006312f921-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.747 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:50:09.748 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:09.749 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:09.750 DEBUG:boto:Method: GET 2022-09-03T06:50:09.752 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:50:09.753 DEBUG:boto:Data: 2022-09-03T06:50:09.753 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.755 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:09.762 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.762 DEBUG:boto:Params: {} 2022-09-03T06:50:09.763 DEBUG:boto:Token: None 2022-09-03T06:50:09.764 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj69 2022-09-03T06:50:09.765 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OzOQm9JViHH3c4uCbiBCqTWWNUU= 2022-09-03T06:50:09.766 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OzOQm9JViHH3c4uCbiBCqTWWNUU=', 'Content-Length': '0'} 2022-09-03T06:50:09.768 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006e1cab1bffda8348-006312f921-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.814 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:09.815 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:09.817 DEBUG:boto:Method: GET 2022-09-03T06:50:09.818 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:50:09.819 DEBUG:boto:Data: 2022-09-03T06:50:09.820 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.821 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:09.822 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.822 DEBUG:boto:Params: {} 2022-09-03T06:50:09.823 DEBUG:boto:Token: None 2022-09-03T06:50:09.824 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj69 2022-09-03T06:50:09.824 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OzOQm9JViHH3c4uCbiBCqTWWNUU= 2022-09-03T06:50:09.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OzOQm9JViHH3c4uCbiBCqTWWNUU=', 'Content-Length': '0'} 2022-09-03T06:50:09.827 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000860ec96bd15a909f-006312f921-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.829 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:09.829 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:09.830 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:09.831 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:09.831 DEBUG:boto:Method: GET 2022-09-03T06:50:09.832 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:50:09.833 DEBUG:boto:Data: 2022-09-03T06:50:09.833 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.834 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:09.834 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.835 DEBUG:boto:Params: {} 2022-09-03T06:50:09.836 DEBUG:boto:Token: None 2022-09-03T06:50:09.836 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/ 2022-09-03T06:50:09.837 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800= 2022-09-03T06:50:09.838 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800=', 'Content-Length': '0'} 2022-09-03T06:50:09.841 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c3ca26590a11fa0b-006312f921-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.842 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:09.842 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:09.843 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:09.844 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:09.844 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:09.845 DEBUG:boto:Method: GET 2022-09-03T06:50:09.845 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:50:09.846 DEBUG:boto:Data: 2022-09-03T06:50:09.846 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.847 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:09.848 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.848 DEBUG:boto:Params: {} 2022-09-03T06:50:09.849 DEBUG:boto:Token: None 2022-09-03T06:50:09.849 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/ 2022-09-03T06:50:09.850 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800= 2022-09-03T06:50:09.851 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800=', 'Content-Length': '0'} 2022-09-03T06:50:09.856 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007e29fa55f33e2883-006312f921-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.856 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:09.858 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj74 2022-09-03T06:50:09.858 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:09.859 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:09.860 DEBUG:boto:path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.860 DEBUG:boto:auth_path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.861 DEBUG:boto:Method: GET 2022-09-03T06:50:09.861 DEBUG:boto:Path: /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.862 DEBUG:boto:Data: 2022-09-03T06:50:09.863 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.863 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:09.864 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.864 DEBUG:boto:Params: {} 2022-09-03T06:50:09.867 DEBUG:boto:Token: None 2022-09-03T06:50:09.869 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.870 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2+J9+7UPAmC/gQYunA+TxE6Zdks= 2022-09-03T06:50:09.872 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2+J9+7UPAmC/gQYunA+TxE6Zdks=', 'Content-Length': '0'} 2022-09-03T06:50:09.878 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f91f087ee4554361-006312f921-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.880 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:09.881 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:09.881 DEBUG:boto:path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.883 DEBUG:boto:auth_path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.884 DEBUG:boto:Method: GET 2022-09-03T06:50:09.884 DEBUG:boto:Path: /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.885 DEBUG:boto:Data: 2022-09-03T06:50:09.886 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.886 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:09.887 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.887 DEBUG:boto:Params: {} 2022-09-03T06:50:09.888 DEBUG:boto:Token: None 2022-09-03T06:50:09.889 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.890 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2+J9+7UPAmC/gQYunA+TxE6Zdks= 2022-09-03T06:50:09.891 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2+J9+7UPAmC/gQYunA+TxE6Zdks=', 'Content-Length': '0'} 2022-09-03T06:50:09.893 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e56c2f10ad5ec9bb-006312f921-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.896 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:09.897 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:09.898 DEBUG:boto:path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.900 DEBUG:boto:auth_path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.901 DEBUG:boto:Method: HEAD 2022-09-03T06:50:09.902 DEBUG:boto:Path: /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.904 DEBUG:boto:Data: 2022-09-03T06:50:09.905 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.906 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:09.908 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.908 DEBUG:boto:Params: {} 2022-09-03T06:50:09.909 DEBUG:boto:Token: None 2022-09-03T06:50:09.910 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.911 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DwOULF/dDGXjj60p6EQfjVlWOmo= 2022-09-03T06:50:09.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DwOULF/dDGXjj60p6EQfjVlWOmo=', 'Content-Length': '0'} 2022-09-03T06:50:09.914 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000840758956c762d09-006312f921-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.915 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:09.916 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:09.916 DEBUG:boto:path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.917 DEBUG:boto:auth_path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.917 DEBUG:boto:Method: HEAD 2022-09-03T06:50:09.918 DEBUG:boto:Path: /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.919 DEBUG:boto:Data: 2022-09-03T06:50:09.919 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.920 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:09.921 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.921 DEBUG:boto:Params: {} 2022-09-03T06:50:09.922 DEBUG:boto:Token: None 2022-09-03T06:50:09.923 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.924 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DwOULF/dDGXjj60p6EQfjVlWOmo= 2022-09-03T06:50:09.924 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DwOULF/dDGXjj60p6EQfjVlWOmo=', 'Content-Length': '0'} 2022-09-03T06:50:09.927 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b4bcafd04c854a59-006312f921-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.928 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj74 2022-09-03T06:50:09.928 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:09.929 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:09.930 DEBUG:boto:path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.930 DEBUG:boto:auth_path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.931 DEBUG:boto:Method: GET 2022-09-03T06:50:09.932 DEBUG:boto:Path: /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.933 DEBUG:boto:Data: 2022-09-03T06:50:09.934 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.935 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:09.938 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.939 DEBUG:boto:Params: {} 2022-09-03T06:50:09.939 DEBUG:boto:Token: None 2022-09-03T06:50:09.940 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.942 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2+J9+7UPAmC/gQYunA+TxE6Zdks= 2022-09-03T06:50:09.942 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2+J9+7UPAmC/gQYunA+TxE6Zdks=', 'Content-Length': '0'} 2022-09-03T06:50:09.945 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000079e9396515cb0b7c-006312f921-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.946 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:09.947 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:09.947 DEBUG:boto:path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.948 DEBUG:boto:auth_path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.949 DEBUG:boto:Method: GET 2022-09-03T06:50:09.950 DEBUG:boto:Path: /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.950 DEBUG:boto:Data: 2022-09-03T06:50:09.951 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.952 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:09.953 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.954 DEBUG:boto:Params: {} 2022-09-03T06:50:09.955 DEBUG:boto:Token: None 2022-09-03T06:50:09.956 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:09.957 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2+J9+7UPAmC/gQYunA+TxE6Zdks= 2022-09-03T06:50:09.958 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2+J9+7UPAmC/gQYunA+TxE6Zdks=', 'Content-Length': '0'} 2022-09-03T06:50:09.960 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000480ac33054ff4c5d-006312f921-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.961 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:09.962 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:09.963 DEBUG:boto:path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:09.963 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:09.964 DEBUG:boto:Method: GET 2022-09-03T06:50:09.964 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj74 2022-09-03T06:50:09.965 DEBUG:boto:Data: 2022-09-03T06:50:09.966 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.966 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:09.967 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.967 DEBUG:boto:Params: {} 2022-09-03T06:50:09.968 DEBUG:boto:Token: None 2022-09-03T06:50:09.969 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/ 2022-09-03T06:50:09.970 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800= 2022-09-03T06:50:09.971 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800=', 'Content-Length': '0'} 2022-09-03T06:50:09.974 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007ada4f2d39a0f1e9-006312f921-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.976 DEBUG:boto:b'dsgvcy-75obj741000falseobj742022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:09.977 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:09.977 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:09.978 DEBUG:boto:path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:09.978 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:09.979 DEBUG:boto:Method: GET 2022-09-03T06:50:09.980 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj74 2022-09-03T06:50:09.980 DEBUG:boto:Data: 2022-09-03T06:50:09.981 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.982 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:09.982 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.983 DEBUG:boto:Params: {} 2022-09-03T06:50:09.984 DEBUG:boto:Token: None 2022-09-03T06:50:09.985 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:09 GMT /dsgvcy-75/ 2022-09-03T06:50:09.986 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800= 2022-09-03T06:50:09.987 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:09 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a/0OMq729OFfRXypLcZh0uCK800=', 'Content-Length': '0'} 2022-09-03T06:50:09.991 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000095c9390a445d41b8-006312f921-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:09 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:09.992 DEBUG:boto:b'dsgvcy-75obj741000falseobj742022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:09.994 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:09.994 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:09.995 DEBUG:boto:Method: HEAD 2022-09-03T06:50:09.995 DEBUG:boto:Path: /dsgvcy-75/obj74 2022-09-03T06:50:09.996 DEBUG:boto:Data: 2022-09-03T06:50:09.997 DEBUG:boto:Headers: {} 2022-09-03T06:50:09.997 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:09.998 DEBUG:boto:Port: 8000 2022-09-03T06:50:09.999 DEBUG:boto:Params: {} 2022-09-03T06:50:10.000 DEBUG:boto:Token: None 2022-09-03T06:50:10.001 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj74 2022-09-03T06:50:10.001 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pmlteWcR3jc2RFjqGUkeir163XA= 2022-09-03T06:50:10.002 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pmlteWcR3jc2RFjqGUkeir163XA=', 'Content-Length': '0'} 2022-09-03T06:50:10.004 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000326329ff4f289956-006312f922-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.006 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:10.006 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:10.007 DEBUG:boto:Method: HEAD 2022-09-03T06:50:10.008 DEBUG:boto:Path: /dsgvcy-75/obj74 2022-09-03T06:50:10.008 DEBUG:boto:Data: 2022-09-03T06:50:10.009 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.010 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:10.011 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.012 DEBUG:boto:Params: {} 2022-09-03T06:50:10.012 DEBUG:boto:Token: None 2022-09-03T06:50:10.013 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj74 2022-09-03T06:50:10.014 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pmlteWcR3jc2RFjqGUkeir163XA= 2022-09-03T06:50:10.014 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pmlteWcR3jc2RFjqGUkeir163XA=', 'Content-Length': '0'} 2022-09-03T06:50:10.017 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f1ed6456e8fb5070-006312f922-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.018 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj74 2022-09-03T06:50:10.019 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:10.019 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:10.020 DEBUG:boto:Method: GET 2022-09-03T06:50:10.021 DEBUG:boto:Path: /dsgvcy-75/obj74 2022-09-03T06:50:10.022 DEBUG:boto:Data: 2022-09-03T06:50:10.022 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.023 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:10.024 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.024 DEBUG:boto:Params: {} 2022-09-03T06:50:10.025 DEBUG:boto:Token: None 2022-09-03T06:50:10.026 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj74 2022-09-03T06:50:10.026 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YXln8KzDydeY66csROhx7Wfk9M8= 2022-09-03T06:50:10.027 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YXln8KzDydeY66csROhx7Wfk9M8=', 'Content-Length': '0'} 2022-09-03T06:50:10.030 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000076cb9e6488eb7bf-006312f922-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.074 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:10.075 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:10.077 DEBUG:boto:Method: GET 2022-09-03T06:50:10.078 DEBUG:boto:Path: /dsgvcy-75/obj74 2022-09-03T06:50:10.078 DEBUG:boto:Data: 2022-09-03T06:50:10.080 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.081 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:10.082 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.083 DEBUG:boto:Params: {} 2022-09-03T06:50:10.084 DEBUG:boto:Token: None 2022-09-03T06:50:10.085 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj74 2022-09-03T06:50:10.087 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YXln8KzDydeY66csROhx7Wfk9M8= 2022-09-03T06:50:10.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YXln8KzDydeY66csROhx7Wfk9M8=', 'Content-Length': '0'} 2022-09-03T06:50:10.091 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007a71d21c236d1794-006312f922-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.093 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:10.094 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:10.095 DEBUG:boto:path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:10.096 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:10.097 DEBUG:boto:Method: GET 2022-09-03T06:50:10.098 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj74 2022-09-03T06:50:10.099 DEBUG:boto:Data: 2022-09-03T06:50:10.099 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.100 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:10.101 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.101 DEBUG:boto:Params: {} 2022-09-03T06:50:10.102 DEBUG:boto:Token: None 2022-09-03T06:50:10.103 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/ 2022-09-03T06:50:10.104 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as= 2022-09-03T06:50:10.104 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as=', 'Content-Length': '0'} 2022-09-03T06:50:10.107 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008589662869005413-006312f922-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.108 DEBUG:boto:b'dsgvcy-75obj741000falseobj742022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:10.109 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:10.110 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:10.110 DEBUG:boto:path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:10.111 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:10.112 DEBUG:boto:Method: GET 2022-09-03T06:50:10.112 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj74 2022-09-03T06:50:10.113 DEBUG:boto:Data: 2022-09-03T06:50:10.113 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.114 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:10.115 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.115 DEBUG:boto:Params: {} 2022-09-03T06:50:10.116 DEBUG:boto:Token: None 2022-09-03T06:50:10.116 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/ 2022-09-03T06:50:10.117 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as= 2022-09-03T06:50:10.118 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as=', 'Content-Length': '0'} 2022-09-03T06:50:10.123 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000390e553a53c2503b-006312f922-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.123 DEBUG:boto:b'dsgvcy-75obj741000falseobj742022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:10.124 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:50:10.125 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:10.126 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:10.126 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.127 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.128 DEBUG:boto:Method: GET 2022-09-03T06:50:10.129 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.130 DEBUG:boto:Data: 2022-09-03T06:50:10.131 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.132 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:10.133 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.134 DEBUG:boto:Params: {} 2022-09-03T06:50:10.136 DEBUG:boto:Token: None 2022-09-03T06:50:10.137 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.138 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BMnvjgNTueR26LiH0I55FV3eKx8= 2022-09-03T06:50:10.139 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BMnvjgNTueR26LiH0I55FV3eKx8=', 'Content-Length': '0'} 2022-09-03T06:50:10.142 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000062a9ceffea0441cc-006312f922-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.144 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:10.145 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:10.146 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.147 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.148 DEBUG:boto:Method: GET 2022-09-03T06:50:10.149 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.150 DEBUG:boto:Data: 2022-09-03T06:50:10.151 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.152 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:10.153 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.155 DEBUG:boto:Params: {} 2022-09-03T06:50:10.156 DEBUG:boto:Token: None 2022-09-03T06:50:10.157 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.159 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BMnvjgNTueR26LiH0I55FV3eKx8= 2022-09-03T06:50:10.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BMnvjgNTueR26LiH0I55FV3eKx8=', 'Content-Length': '0'} 2022-09-03T06:50:10.163 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000000fcab8194d02e6-006312f922-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.165 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:10.166 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:10.167 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.168 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.169 DEBUG:boto:Method: HEAD 2022-09-03T06:50:10.170 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.170 DEBUG:boto:Data: 2022-09-03T06:50:10.171 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.172 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:10.174 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.175 DEBUG:boto:Params: {} 2022-09-03T06:50:10.176 DEBUG:boto:Token: None 2022-09-03T06:50:10.177 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.178 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:umVzdlLowMX0KzTqcWuOCf1kBWA= 2022-09-03T06:50:10.178 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:umVzdlLowMX0KzTqcWuOCf1kBWA=', 'Content-Length': '0'} 2022-09-03T06:50:10.181 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eeb2d26b81619c8d-006312f922-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.182 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:10.183 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:10.185 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.185 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.186 DEBUG:boto:Method: HEAD 2022-09-03T06:50:10.187 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.188 DEBUG:boto:Data: 2022-09-03T06:50:10.188 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.189 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:10.190 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.190 DEBUG:boto:Params: {} 2022-09-03T06:50:10.191 DEBUG:boto:Token: None 2022-09-03T06:50:10.192 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.192 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:umVzdlLowMX0KzTqcWuOCf1kBWA= 2022-09-03T06:50:10.193 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:umVzdlLowMX0KzTqcWuOCf1kBWA=', 'Content-Length': '0'} 2022-09-03T06:50:10.195 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006c88d9f9e71ba7d6-006312f922-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.197 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:50:10.197 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:10.198 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:10.198 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.199 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.200 DEBUG:boto:Method: GET 2022-09-03T06:50:10.200 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.201 DEBUG:boto:Data: 2022-09-03T06:50:10.202 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.211 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:10.212 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.212 DEBUG:boto:Params: {} 2022-09-03T06:50:10.213 DEBUG:boto:Token: None 2022-09-03T06:50:10.214 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.215 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BMnvjgNTueR26LiH0I55FV3eKx8= 2022-09-03T06:50:10.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BMnvjgNTueR26LiH0I55FV3eKx8=', 'Content-Length': '0'} 2022-09-03T06:50:10.218 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a1ec4dfeba43740-006312f922-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.220 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:10.221 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:10.222 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.224 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.225 DEBUG:boto:Method: GET 2022-09-03T06:50:10.226 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.228 DEBUG:boto:Data: 2022-09-03T06:50:10.228 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.230 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:10.231 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.231 DEBUG:boto:Params: {} 2022-09-03T06:50:10.232 DEBUG:boto:Token: None 2022-09-03T06:50:10.234 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:10.236 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BMnvjgNTueR26LiH0I55FV3eKx8= 2022-09-03T06:50:10.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BMnvjgNTueR26LiH0I55FV3eKx8=', 'Content-Length': '0'} 2022-09-03T06:50:10.240 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003898d42890beca8e-006312f922-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.241 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:10.241 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:10.242 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:10.243 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:10.243 DEBUG:boto:Method: GET 2022-09-03T06:50:10.244 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:50:10.244 DEBUG:boto:Data: 2022-09-03T06:50:10.245 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.245 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:10.246 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.246 DEBUG:boto:Params: {} 2022-09-03T06:50:10.247 DEBUG:boto:Token: None 2022-09-03T06:50:10.248 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/ 2022-09-03T06:50:10.248 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as= 2022-09-03T06:50:10.249 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as=', 'Content-Length': '0'} 2022-09-03T06:50:10.252 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f4712af711b69d44-006312f922-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.294 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:10.296 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:10.297 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:10.298 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:10.299 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:10.299 DEBUG:boto:Method: GET 2022-09-03T06:50:10.300 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:50:10.300 DEBUG:boto:Data: 2022-09-03T06:50:10.301 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.301 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:10.302 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.302 DEBUG:boto:Params: {} 2022-09-03T06:50:10.303 DEBUG:boto:Token: None 2022-09-03T06:50:10.304 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/ 2022-09-03T06:50:10.304 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as= 2022-09-03T06:50:10.305 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as=', 'Content-Length': '0'} 2022-09-03T06:50:10.310 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000443df2e96d36bef1-006312f922-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.310 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:10.312 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:10.312 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:10.313 DEBUG:boto:Method: HEAD 2022-09-03T06:50:10.314 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:50:10.314 DEBUG:boto:Data: 2022-09-03T06:50:10.315 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.316 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:10.317 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.318 DEBUG:boto:Params: {} 2022-09-03T06:50:10.319 DEBUG:boto:Token: None 2022-09-03T06:50:10.320 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj76 2022-09-03T06:50:10.321 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XqGcXzasUfWI/i9i36BTOUvOLp8= 2022-09-03T06:50:10.322 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XqGcXzasUfWI/i9i36BTOUvOLp8=', 'Content-Length': '0'} 2022-09-03T06:50:10.325 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002c13f873f5e9de0a-006312f922-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.325 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:10.326 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:10.327 DEBUG:boto:Method: HEAD 2022-09-03T06:50:10.328 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:50:10.329 DEBUG:boto:Data: 2022-09-03T06:50:10.334 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.335 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:10.337 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.342 DEBUG:boto:Params: {} 2022-09-03T06:50:10.343 DEBUG:boto:Token: None 2022-09-03T06:50:10.344 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj76 2022-09-03T06:50:10.345 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XqGcXzasUfWI/i9i36BTOUvOLp8= 2022-09-03T06:50:10.346 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XqGcXzasUfWI/i9i36BTOUvOLp8=', 'Content-Length': '0'} 2022-09-03T06:50:10.349 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000079f837e6c65d2320-006312f922-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.350 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:50:10.350 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:10.351 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:10.352 DEBUG:boto:Method: GET 2022-09-03T06:50:10.353 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:50:10.353 DEBUG:boto:Data: 2022-09-03T06:50:10.354 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.354 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:10.355 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.356 DEBUG:boto:Params: {} 2022-09-03T06:50:10.357 DEBUG:boto:Token: None 2022-09-03T06:50:10.358 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj76 2022-09-03T06:50:10.359 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NHqInpOEx4Nh28bR4YgnvB7s7Ho= 2022-09-03T06:50:10.359 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NHqInpOEx4Nh28bR4YgnvB7s7Ho=', 'Content-Length': '0'} 2022-09-03T06:50:10.376 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000079c35cc9aaf5f588-006312f922-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.377 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:10.378 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:10.379 DEBUG:boto:Method: GET 2022-09-03T06:50:10.380 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:50:10.381 DEBUG:boto:Data: 2022-09-03T06:50:10.381 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.382 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:10.383 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.383 DEBUG:boto:Params: {} 2022-09-03T06:50:10.384 DEBUG:boto:Token: None 2022-09-03T06:50:10.385 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj76 2022-09-03T06:50:10.385 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NHqInpOEx4Nh28bR4YgnvB7s7Ho= 2022-09-03T06:50:10.386 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NHqInpOEx4Nh28bR4YgnvB7s7Ho=', 'Content-Length': '0'} 2022-09-03T06:50:10.389 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000622ac934c555b39c-006312f922-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.390 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:10.390 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:10.391 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:10.391 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:10.392 DEBUG:boto:Method: GET 2022-09-03T06:50:10.392 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:50:10.393 DEBUG:boto:Data: 2022-09-03T06:50:10.394 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.394 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:10.395 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.395 DEBUG:boto:Params: {} 2022-09-03T06:50:10.396 DEBUG:boto:Token: None 2022-09-03T06:50:10.397 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/ 2022-09-03T06:50:10.398 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as= 2022-09-03T06:50:10.398 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as=', 'Content-Length': '0'} 2022-09-03T06:50:10.402 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e379360e47b68582-006312f922-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.402 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:10.403 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:10.404 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:10.405 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:10.405 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:10.406 DEBUG:boto:Method: GET 2022-09-03T06:50:10.407 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:50:10.407 DEBUG:boto:Data: 2022-09-03T06:50:10.408 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.408 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:10.409 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.410 DEBUG:boto:Params: {} 2022-09-03T06:50:10.410 DEBUG:boto:Token: None 2022-09-03T06:50:10.411 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/ 2022-09-03T06:50:10.412 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as= 2022-09-03T06:50:10.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as=', 'Content-Length': '0'} 2022-09-03T06:50:10.418 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a98a3752bc45f070-006312f922-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.420 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:10.421 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:50:10.422 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:10.424 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:10.425 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.425 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.427 DEBUG:boto:Method: GET 2022-09-03T06:50:10.427 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.428 DEBUG:boto:Data: 2022-09-03T06:50:10.429 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.433 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:10.433 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.434 DEBUG:boto:Params: {} 2022-09-03T06:50:10.436 DEBUG:boto:Token: None 2022-09-03T06:50:10.437 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.438 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xz65u9WVQY0sksZjGLS4Gu1qPs0= 2022-09-03T06:50:10.439 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xz65u9WVQY0sksZjGLS4Gu1qPs0=', 'Content-Length': '0'} 2022-09-03T06:50:10.444 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002710c779e0ed80fc-006312f922-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.445 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:10.446 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:10.447 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.447 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.448 DEBUG:boto:Method: GET 2022-09-03T06:50:10.448 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.449 DEBUG:boto:Data: 2022-09-03T06:50:10.450 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.450 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:10.451 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.451 DEBUG:boto:Params: {} 2022-09-03T06:50:10.452 DEBUG:boto:Token: None 2022-09-03T06:50:10.452 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.453 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xz65u9WVQY0sksZjGLS4Gu1qPs0= 2022-09-03T06:50:10.454 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xz65u9WVQY0sksZjGLS4Gu1qPs0=', 'Content-Length': '0'} 2022-09-03T06:50:10.456 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007fba213daad2212e-006312f922-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.457 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:10.458 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:10.458 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.459 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.459 DEBUG:boto:Method: HEAD 2022-09-03T06:50:10.460 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.460 DEBUG:boto:Data: 2022-09-03T06:50:10.461 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.461 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:10.462 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.462 DEBUG:boto:Params: {} 2022-09-03T06:50:10.463 DEBUG:boto:Token: None 2022-09-03T06:50:10.463 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.464 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w1tXGS1kK4TiQM+SLc98eWw1nIQ= 2022-09-03T06:50:10.465 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w1tXGS1kK4TiQM+SLc98eWw1nIQ=', 'Content-Length': '0'} 2022-09-03T06:50:10.467 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000be04901bc312cc31-006312f922-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.467 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:10.468 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:10.469 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.469 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.470 DEBUG:boto:Method: HEAD 2022-09-03T06:50:10.470 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.471 DEBUG:boto:Data: 2022-09-03T06:50:10.471 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.472 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:10.472 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.473 DEBUG:boto:Params: {} 2022-09-03T06:50:10.473 DEBUG:boto:Token: None 2022-09-03T06:50:10.474 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.474 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:w1tXGS1kK4TiQM+SLc98eWw1nIQ= 2022-09-03T06:50:10.475 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:w1tXGS1kK4TiQM+SLc98eWw1nIQ=', 'Content-Length': '0'} 2022-09-03T06:50:10.477 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003fb0336a133f20e9-006312f922-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.478 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:50:10.478 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:10.479 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:10.480 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.480 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.481 DEBUG:boto:Method: GET 2022-09-03T06:50:10.481 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.482 DEBUG:boto:Data: 2022-09-03T06:50:10.483 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.483 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:10.484 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.484 DEBUG:boto:Params: {} 2022-09-03T06:50:10.485 DEBUG:boto:Token: None 2022-09-03T06:50:10.485 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.486 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xz65u9WVQY0sksZjGLS4Gu1qPs0= 2022-09-03T06:50:10.487 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xz65u9WVQY0sksZjGLS4Gu1qPs0=', 'Content-Length': '0'} 2022-09-03T06:50:10.489 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008fbe53e971f71cfb-006312f922-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.534 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:10.535 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:10.536 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.537 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.538 DEBUG:boto:Method: GET 2022-09-03T06:50:10.539 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.541 DEBUG:boto:Data: 2022-09-03T06:50:10.542 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.543 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:10.545 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.546 DEBUG:boto:Params: {} 2022-09-03T06:50:10.548 DEBUG:boto:Token: None 2022-09-03T06:50:10.549 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:10.550 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xz65u9WVQY0sksZjGLS4Gu1qPs0= 2022-09-03T06:50:10.551 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xz65u9WVQY0sksZjGLS4Gu1qPs0=', 'Content-Length': '0'} 2022-09-03T06:50:10.554 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000361760e51957ebf9-006312f922-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.556 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:10.557 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:10.558 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:10.559 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:10.560 DEBUG:boto:Method: GET 2022-09-03T06:50:10.560 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:50:10.561 DEBUG:boto:Data: 2022-09-03T06:50:10.562 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.563 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:10.564 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.565 DEBUG:boto:Params: {} 2022-09-03T06:50:10.566 DEBUG:boto:Token: None 2022-09-03T06:50:10.567 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/ 2022-09-03T06:50:10.568 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as= 2022-09-03T06:50:10.569 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as=', 'Content-Length': '0'} 2022-09-03T06:50:10.573 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000065dcfe1307e6db38-006312f922-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.574 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:10.576 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:10.577 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:10.577 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:10.578 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:10.579 DEBUG:boto:Method: GET 2022-09-03T06:50:10.580 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:50:10.581 DEBUG:boto:Data: 2022-09-03T06:50:10.582 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.583 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:10.584 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.585 DEBUG:boto:Params: {} 2022-09-03T06:50:10.586 DEBUG:boto:Token: None 2022-09-03T06:50:10.587 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/ 2022-09-03T06:50:10.589 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as= 2022-09-03T06:50:10.590 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as=', 'Content-Length': '0'} 2022-09-03T06:50:10.596 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000870d1cb6bf7e0652-006312f922-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.597 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:10.599 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:10.599 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:10.600 DEBUG:boto:Method: HEAD 2022-09-03T06:50:10.601 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:50:10.603 DEBUG:boto:Data: 2022-09-03T06:50:10.604 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.605 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:10.606 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.607 DEBUG:boto:Params: {} 2022-09-03T06:50:10.608 DEBUG:boto:Token: None 2022-09-03T06:50:10.609 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj85 2022-09-03T06:50:10.610 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:amwww8xNIR2iaWC3t2MaB/z+RMQ= 2022-09-03T06:50:10.611 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:amwww8xNIR2iaWC3t2MaB/z+RMQ=', 'Content-Length': '0'} 2022-09-03T06:50:10.614 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004d586db29bb36307-006312f922-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.616 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:10.617 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:10.618 DEBUG:boto:Method: HEAD 2022-09-03T06:50:10.619 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:50:10.619 DEBUG:boto:Data: 2022-09-03T06:50:10.620 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.621 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:10.623 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.624 DEBUG:boto:Params: {} 2022-09-03T06:50:10.625 DEBUG:boto:Token: None 2022-09-03T06:50:10.626 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj85 2022-09-03T06:50:10.627 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:amwww8xNIR2iaWC3t2MaB/z+RMQ= 2022-09-03T06:50:10.628 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:amwww8xNIR2iaWC3t2MaB/z+RMQ=', 'Content-Length': '0'} 2022-09-03T06:50:10.631 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c6dab8ece7de17f8-006312f922-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.632 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:50:10.633 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:10.635 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:10.636 DEBUG:boto:Method: GET 2022-09-03T06:50:10.637 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:50:10.638 DEBUG:boto:Data: 2022-09-03T06:50:10.639 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.640 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:10.641 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.641 DEBUG:boto:Params: {} 2022-09-03T06:50:10.642 DEBUG:boto:Token: None 2022-09-03T06:50:10.643 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj85 2022-09-03T06:50:10.643 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XddAPYWOBa22dqtuaRg9niONxE4= 2022-09-03T06:50:10.644 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XddAPYWOBa22dqtuaRg9niONxE4=', 'Content-Length': '0'} 2022-09-03T06:50:10.646 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000058388a3a6f15457f-006312f922-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.647 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:10.647 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:10.648 DEBUG:boto:Method: GET 2022-09-03T06:50:10.649 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:50:10.649 DEBUG:boto:Data: 2022-09-03T06:50:10.650 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.650 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:10.651 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.652 DEBUG:boto:Params: {} 2022-09-03T06:50:10.652 DEBUG:boto:Token: None 2022-09-03T06:50:10.653 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj85 2022-09-03T06:50:10.653 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XddAPYWOBa22dqtuaRg9niONxE4= 2022-09-03T06:50:10.654 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XddAPYWOBa22dqtuaRg9niONxE4=', 'Content-Length': '0'} 2022-09-03T06:50:10.657 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000144a2c9d5245d47d-006312f922-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.698 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:10.699 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:10.700 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:10.701 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:10.702 DEBUG:boto:Method: GET 2022-09-03T06:50:10.702 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:50:10.704 DEBUG:boto:Data: 2022-09-03T06:50:10.704 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.705 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:10.705 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.706 DEBUG:boto:Params: {} 2022-09-03T06:50:10.707 DEBUG:boto:Token: None 2022-09-03T06:50:10.707 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/ 2022-09-03T06:50:10.708 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as= 2022-09-03T06:50:10.709 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as=', 'Content-Length': '0'} 2022-09-03T06:50:10.712 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004f022406ef0f56a9-006312f922-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.713 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:10.714 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:10.715 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:10.716 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:10.716 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:10.717 DEBUG:boto:Method: GET 2022-09-03T06:50:10.718 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:50:10.718 DEBUG:boto:Data: 2022-09-03T06:50:10.719 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.720 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:10.721 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.722 DEBUG:boto:Params: {} 2022-09-03T06:50:10.723 DEBUG:boto:Token: None 2022-09-03T06:50:10.724 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/ 2022-09-03T06:50:10.725 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as= 2022-09-03T06:50:10.727 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as=', 'Content-Length': '0'} 2022-09-03T06:50:10.732 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f3dbd2057a07525c-006312f922-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.733 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:10.734 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:50:10.735 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:10.736 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:10.737 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.738 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.739 DEBUG:boto:Method: GET 2022-09-03T06:50:10.740 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.741 DEBUG:boto:Data: 2022-09-03T06:50:10.741 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.742 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:10.743 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.744 DEBUG:boto:Params: {} 2022-09-03T06:50:10.745 DEBUG:boto:Token: None 2022-09-03T06:50:10.746 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.747 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1x4JROw2Q7cOErh54PC+A9GzJZ0= 2022-09-03T06:50:10.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1x4JROw2Q7cOErh54PC+A9GzJZ0=', 'Content-Length': '0'} 2022-09-03T06:50:10.750 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d48e6e46d39da00a-006312f922-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.752 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:10.753 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:10.757 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.758 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.758 DEBUG:boto:Method: GET 2022-09-03T06:50:10.759 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.759 DEBUG:boto:Data: 2022-09-03T06:50:10.760 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.761 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:10.761 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.762 DEBUG:boto:Params: {} 2022-09-03T06:50:10.763 DEBUG:boto:Token: None 2022-09-03T06:50:10.764 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.764 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1x4JROw2Q7cOErh54PC+A9GzJZ0= 2022-09-03T06:50:10.765 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1x4JROw2Q7cOErh54PC+A9GzJZ0=', 'Content-Length': '0'} 2022-09-03T06:50:10.768 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000c872b984a61b06f-006312f922-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.769 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:10.770 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:10.770 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.771 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.772 DEBUG:boto:Method: HEAD 2022-09-03T06:50:10.772 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.773 DEBUG:boto:Data: 2022-09-03T06:50:10.773 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.774 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:10.774 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.775 DEBUG:boto:Params: {} 2022-09-03T06:50:10.775 DEBUG:boto:Token: None 2022-09-03T06:50:10.776 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.777 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DPnCpXgpVtjYGq03zyELh1ibYXc= 2022-09-03T06:50:10.777 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DPnCpXgpVtjYGq03zyELh1ibYXc=', 'Content-Length': '0'} 2022-09-03T06:50:10.781 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000da2e05be9e41d67f-006312f922-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.783 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:10.783 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:10.784 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.785 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.785 DEBUG:boto:Method: HEAD 2022-09-03T06:50:10.786 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.787 DEBUG:boto:Data: 2022-09-03T06:50:10.787 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.788 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:10.788 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.789 DEBUG:boto:Params: {} 2022-09-03T06:50:10.790 DEBUG:boto:Token: None 2022-09-03T06:50:10.791 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.791 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DPnCpXgpVtjYGq03zyELh1ibYXc= 2022-09-03T06:50:10.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DPnCpXgpVtjYGq03zyELh1ibYXc=', 'Content-Length': '0'} 2022-09-03T06:50:10.795 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009399dde2e707af8f-006312f922-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.796 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:50:10.796 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:10.797 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:10.798 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.798 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.799 DEBUG:boto:Method: GET 2022-09-03T06:50:10.799 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.800 DEBUG:boto:Data: 2022-09-03T06:50:10.801 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.801 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:10.802 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.802 DEBUG:boto:Params: {} 2022-09-03T06:50:10.803 DEBUG:boto:Token: None 2022-09-03T06:50:10.804 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.805 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1x4JROw2Q7cOErh54PC+A9GzJZ0= 2022-09-03T06:50:10.806 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1x4JROw2Q7cOErh54PC+A9GzJZ0=', 'Content-Length': '0'} 2022-09-03T06:50:10.808 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000092a40a19d4989b79-006312f922-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.809 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:10.810 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:10.811 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.812 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.813 DEBUG:boto:Method: GET 2022-09-03T06:50:10.814 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.815 DEBUG:boto:Data: 2022-09-03T06:50:10.815 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.816 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:10.817 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.817 DEBUG:boto:Params: {} 2022-09-03T06:50:10.818 DEBUG:boto:Token: None 2022-09-03T06:50:10.819 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:10.819 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1x4JROw2Q7cOErh54PC+A9GzJZ0= 2022-09-03T06:50:10.820 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1x4JROw2Q7cOErh54PC+A9GzJZ0=', 'Content-Length': '0'} 2022-09-03T06:50:10.836 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f586a0e8dd458c55-006312f922-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.876 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:10.877 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:10.878 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:10.879 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:10.879 DEBUG:boto:Method: GET 2022-09-03T06:50:10.880 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:50:10.880 DEBUG:boto:Data: 2022-09-03T06:50:10.881 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.882 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:10.883 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.884 DEBUG:boto:Params: {} 2022-09-03T06:50:10.884 DEBUG:boto:Token: None 2022-09-03T06:50:10.885 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/ 2022-09-03T06:50:10.886 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as= 2022-09-03T06:50:10.886 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as=', 'Content-Length': '0'} 2022-09-03T06:50:10.893 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000007e1660a9e5f2844-006312f922-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.894 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:10.895 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:10.896 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:10.896 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:10.897 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:10.898 DEBUG:boto:Method: GET 2022-09-03T06:50:10.898 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:50:10.899 DEBUG:boto:Data: 2022-09-03T06:50:10.899 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.900 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:10.901 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.901 DEBUG:boto:Params: {} 2022-09-03T06:50:10.902 DEBUG:boto:Token: None 2022-09-03T06:50:10.902 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/ 2022-09-03T06:50:10.903 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as= 2022-09-03T06:50:10.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:lf/fQQGkg70fQUC85z6yDuYa+as=', 'Content-Length': '0'} 2022-09-03T06:50:10.909 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001f63c2b81f7b14ae-006312f922-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.910 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:10.911 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:10.912 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:10.912 DEBUG:boto:Method: HEAD 2022-09-03T06:50:10.913 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:50:10.913 DEBUG:boto:Data: 2022-09-03T06:50:10.914 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.915 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:10.915 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.916 DEBUG:boto:Params: {} 2022-09-03T06:50:10.917 DEBUG:boto:Token: None 2022-09-03T06:50:10.918 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj87 2022-09-03T06:50:10.919 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:u5mpJUNbKvwABydNidPdPeKvPJg= 2022-09-03T06:50:10.920 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:u5mpJUNbKvwABydNidPdPeKvPJg=', 'Content-Length': '0'} 2022-09-03T06:50:10.922 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000dce04bd69faa716-006312f922-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.923 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:10.923 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:10.924 DEBUG:boto:Method: HEAD 2022-09-03T06:50:10.925 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:50:10.925 DEBUG:boto:Data: 2022-09-03T06:50:10.926 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.926 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:10.927 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.927 DEBUG:boto:Params: {} 2022-09-03T06:50:10.928 DEBUG:boto:Token: None 2022-09-03T06:50:10.929 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj87 2022-09-03T06:50:10.929 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:u5mpJUNbKvwABydNidPdPeKvPJg= 2022-09-03T06:50:10.931 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:u5mpJUNbKvwABydNidPdPeKvPJg=', 'Content-Length': '0'} 2022-09-03T06:50:10.933 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bfb0e411edd9f504-006312f922-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.934 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:50:10.935 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:10.936 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:10.938 DEBUG:boto:Method: GET 2022-09-03T06:50:10.939 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:50:10.940 DEBUG:boto:Data: 2022-09-03T06:50:10.941 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.942 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:10.943 DEBUG:boto:Port: 8000 2022-09-03T06:50:10.944 DEBUG:boto:Params: {} 2022-09-03T06:50:10.945 DEBUG:boto:Token: None 2022-09-03T06:50:10.946 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:10 GMT /dsgvcy-75/obj87 2022-09-03T06:50:10.947 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cTWtTHH0xQDIM2Rqar277j82dvI= 2022-09-03T06:50:10.948 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:10 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cTWtTHH0xQDIM2Rqar277j82dvI=', 'Content-Length': '0'} 2022-09-03T06:50:10.951 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b3be8b53f66307d7-006312f922-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:10 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:10.994 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:10.995 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:10.996 DEBUG:boto:Method: GET 2022-09-03T06:50:10.996 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:50:10.997 DEBUG:boto:Data: 2022-09-03T06:50:10.998 DEBUG:boto:Headers: {} 2022-09-03T06:50:10.998 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:10.999 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.000 DEBUG:boto:Params: {} 2022-09-03T06:50:11.001 DEBUG:boto:Token: None 2022-09-03T06:50:11.001 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj87 2022-09-03T06:50:11.002 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6112efhbqKt2F103Itr0Bd7Jwwo= 2022-09-03T06:50:11.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6112efhbqKt2F103Itr0Bd7Jwwo=', 'Content-Length': '0'} 2022-09-03T06:50:11.008 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fd0859bdea5a009a-006312f923-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.009 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:11.010 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:11.010 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:11.011 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:11.011 DEBUG:boto:Method: GET 2022-09-03T06:50:11.012 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:50:11.013 DEBUG:boto:Data: 2022-09-03T06:50:11.013 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.014 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:11.014 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.015 DEBUG:boto:Params: {} 2022-09-03T06:50:11.016 DEBUG:boto:Token: None 2022-09-03T06:50:11.017 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/ 2022-09-03T06:50:11.017 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI= 2022-09-03T06:50:11.018 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI=', 'Content-Length': '0'} 2022-09-03T06:50:11.021 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000010d5838e200a03f6-006312f923-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.022 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:11.023 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:11.023 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:11.024 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:11.024 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:11.025 DEBUG:boto:Method: GET 2022-09-03T06:50:11.025 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:50:11.026 DEBUG:boto:Data: 2022-09-03T06:50:11.027 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.027 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:11.028 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.028 DEBUG:boto:Params: {} 2022-09-03T06:50:11.029 DEBUG:boto:Token: None 2022-09-03T06:50:11.030 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/ 2022-09-03T06:50:11.030 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI= 2022-09-03T06:50:11.031 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI=', 'Content-Length': '0'} 2022-09-03T06:50:11.036 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006d33ca02115922d5-006312f923-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.078 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:11.079 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:50:11.080 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:11.080 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:11.081 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.081 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.082 DEBUG:boto:Method: GET 2022-09-03T06:50:11.082 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.083 DEBUG:boto:Data: 2022-09-03T06:50:11.084 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.084 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:11.085 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.085 DEBUG:boto:Params: {} 2022-09-03T06:50:11.086 DEBUG:boto:Token: None 2022-09-03T06:50:11.087 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.087 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Byb/7VdWbqxGOn3Pr+0d2pT8/FI= 2022-09-03T06:50:11.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Byb/7VdWbqxGOn3Pr+0d2pT8/FI=', 'Content-Length': '0'} 2022-09-03T06:50:11.090 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a05921e0f61c5e0e-006312f923-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.091 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:11.092 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:11.092 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.093 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.093 DEBUG:boto:Method: GET 2022-09-03T06:50:11.094 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.094 DEBUG:boto:Data: 2022-09-03T06:50:11.095 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.095 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:11.096 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.096 DEBUG:boto:Params: {} 2022-09-03T06:50:11.097 DEBUG:boto:Token: None 2022-09-03T06:50:11.098 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.098 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Byb/7VdWbqxGOn3Pr+0d2pT8/FI= 2022-09-03T06:50:11.099 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Byb/7VdWbqxGOn3Pr+0d2pT8/FI=', 'Content-Length': '0'} 2022-09-03T06:50:11.102 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018033a1597a824fe-006312f923-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.103 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:11.103 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:11.104 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.104 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.105 DEBUG:boto:Method: HEAD 2022-09-03T06:50:11.105 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.106 DEBUG:boto:Data: 2022-09-03T06:50:11.106 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.107 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:11.107 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.108 DEBUG:boto:Params: {} 2022-09-03T06:50:11.109 DEBUG:boto:Token: None 2022-09-03T06:50:11.109 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.110 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wgIlhDJ2J4fXmgF3LrItxzRZVnE= 2022-09-03T06:50:11.110 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wgIlhDJ2J4fXmgF3LrItxzRZVnE=', 'Content-Length': '0'} 2022-09-03T06:50:11.113 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048d1c8ba665cc6b3-006312f923-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.114 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:11.115 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:11.115 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.116 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.116 DEBUG:boto:Method: HEAD 2022-09-03T06:50:11.117 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.118 DEBUG:boto:Data: 2022-09-03T06:50:11.118 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.119 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:11.119 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.120 DEBUG:boto:Params: {} 2022-09-03T06:50:11.120 DEBUG:boto:Token: None 2022-09-03T06:50:11.121 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.122 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wgIlhDJ2J4fXmgF3LrItxzRZVnE= 2022-09-03T06:50:11.122 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wgIlhDJ2J4fXmgF3LrItxzRZVnE=', 'Content-Length': '0'} 2022-09-03T06:50:11.125 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000618f3d0fffa882f0-006312f923-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.126 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:50:11.127 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:11.127 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:11.128 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.128 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.129 DEBUG:boto:Method: GET 2022-09-03T06:50:11.130 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.130 DEBUG:boto:Data: 2022-09-03T06:50:11.131 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.131 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:11.132 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.132 DEBUG:boto:Params: {} 2022-09-03T06:50:11.133 DEBUG:boto:Token: None 2022-09-03T06:50:11.134 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.135 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Byb/7VdWbqxGOn3Pr+0d2pT8/FI= 2022-09-03T06:50:11.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Byb/7VdWbqxGOn3Pr+0d2pT8/FI=', 'Content-Length': '0'} 2022-09-03T06:50:11.138 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f1e57d99db9fe6c6-006312f923-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.139 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:11.140 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:11.140 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.141 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.142 DEBUG:boto:Method: GET 2022-09-03T06:50:11.143 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.143 DEBUG:boto:Data: 2022-09-03T06:50:11.144 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.145 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:11.145 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.146 DEBUG:boto:Params: {} 2022-09-03T06:50:11.147 DEBUG:boto:Token: None 2022-09-03T06:50:11.148 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:11.148 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Byb/7VdWbqxGOn3Pr+0d2pT8/FI= 2022-09-03T06:50:11.149 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Byb/7VdWbqxGOn3Pr+0d2pT8/FI=', 'Content-Length': '0'} 2022-09-03T06:50:11.151 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008cc1768c2f984463-006312f923-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.152 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:11.153 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:11.153 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:11.154 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:11.154 DEBUG:boto:Method: GET 2022-09-03T06:50:11.155 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:50:11.155 DEBUG:boto:Data: 2022-09-03T06:50:11.156 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.157 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:11.157 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.158 DEBUG:boto:Params: {} 2022-09-03T06:50:11.158 DEBUG:boto:Token: None 2022-09-03T06:50:11.159 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/ 2022-09-03T06:50:11.159 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI= 2022-09-03T06:50:11.160 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI=', 'Content-Length': '0'} 2022-09-03T06:50:11.167 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b4f15d77d09c337a-006312f923-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.168 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:11.169 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:11.170 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:11.171 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:11.171 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:11.172 DEBUG:boto:Method: GET 2022-09-03T06:50:11.173 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:50:11.174 DEBUG:boto:Data: 2022-09-03T06:50:11.175 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.175 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:11.176 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.177 DEBUG:boto:Params: {} 2022-09-03T06:50:11.178 DEBUG:boto:Token: None 2022-09-03T06:50:11.179 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/ 2022-09-03T06:50:11.180 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI= 2022-09-03T06:50:11.181 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI=', 'Content-Length': '0'} 2022-09-03T06:50:11.185 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000143d8650d5849cd3-006312f923-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.186 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:11.187 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:11.188 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:11.188 DEBUG:boto:Method: HEAD 2022-09-03T06:50:11.189 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:50:11.189 DEBUG:boto:Data: 2022-09-03T06:50:11.190 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.191 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:11.191 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.192 DEBUG:boto:Params: {} 2022-09-03T06:50:11.193 DEBUG:boto:Token: None 2022-09-03T06:50:11.193 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj92 2022-09-03T06:50:11.194 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q6A0Bx40UGCJllcQIE9ey+mjw5I= 2022-09-03T06:50:11.195 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q6A0Bx40UGCJllcQIE9ey+mjw5I=', 'Content-Length': '0'} 2022-09-03T06:50:11.197 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008021142b76e42465-006312f923-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.198 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:11.198 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:11.199 DEBUG:boto:Method: HEAD 2022-09-03T06:50:11.200 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:50:11.200 DEBUG:boto:Data: 2022-09-03T06:50:11.201 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.201 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:11.202 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.202 DEBUG:boto:Params: {} 2022-09-03T06:50:11.203 DEBUG:boto:Token: None 2022-09-03T06:50:11.204 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj92 2022-09-03T06:50:11.204 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q6A0Bx40UGCJllcQIE9ey+mjw5I= 2022-09-03T06:50:11.205 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q6A0Bx40UGCJllcQIE9ey+mjw5I=', 'Content-Length': '0'} 2022-09-03T06:50:11.208 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006fcea519903980cc-006312f923-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.209 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:50:11.210 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:11.210 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:11.211 DEBUG:boto:Method: GET 2022-09-03T06:50:11.212 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:50:11.212 DEBUG:boto:Data: 2022-09-03T06:50:11.213 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.214 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:11.215 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.216 DEBUG:boto:Params: {} 2022-09-03T06:50:11.217 DEBUG:boto:Token: None 2022-09-03T06:50:11.218 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj92 2022-09-03T06:50:11.219 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oNowLck70+bDEHoH8XR5jVYLk5w= 2022-09-03T06:50:11.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oNowLck70+bDEHoH8XR5jVYLk5w=', 'Content-Length': '0'} 2022-09-03T06:50:11.223 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e51a1dc896f4ed2-006312f923-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.269 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:11.270 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:11.270 DEBUG:boto:Method: GET 2022-09-03T06:50:11.271 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:50:11.272 DEBUG:boto:Data: 2022-09-03T06:50:11.272 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.273 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:11.273 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.274 DEBUG:boto:Params: {} 2022-09-03T06:50:11.275 DEBUG:boto:Token: None 2022-09-03T06:50:11.275 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj92 2022-09-03T06:50:11.276 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oNowLck70+bDEHoH8XR5jVYLk5w= 2022-09-03T06:50:11.277 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oNowLck70+bDEHoH8XR5jVYLk5w=', 'Content-Length': '0'} 2022-09-03T06:50:11.280 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002c5e1b837b48ef51-006312f923-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.281 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:11.282 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:11.282 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:11.283 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:11.283 DEBUG:boto:Method: GET 2022-09-03T06:50:11.284 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:50:11.284 DEBUG:boto:Data: 2022-09-03T06:50:11.285 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.285 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:11.287 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.287 DEBUG:boto:Params: {} 2022-09-03T06:50:11.288 DEBUG:boto:Token: None 2022-09-03T06:50:11.289 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/ 2022-09-03T06:50:11.291 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI= 2022-09-03T06:50:11.292 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI=', 'Content-Length': '0'} 2022-09-03T06:50:11.297 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ffa13bc7e6367828-006312f923-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.299 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:11.300 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:11.301 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:11.302 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:11.303 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:11.304 DEBUG:boto:Method: GET 2022-09-03T06:50:11.305 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:50:11.306 DEBUG:boto:Data: 2022-09-03T06:50:11.307 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.308 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:11.309 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.310 DEBUG:boto:Params: {} 2022-09-03T06:50:11.311 DEBUG:boto:Token: None 2022-09-03T06:50:11.312 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/ 2022-09-03T06:50:11.313 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI= 2022-09-03T06:50:11.314 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI=', 'Content-Length': '0'} 2022-09-03T06:50:11.321 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005053cc6e99d6fc6e-006312f923-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.366 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:11.368 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2022-09-03T06:50:11.369 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:11.370 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:11.371 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.375 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.376 DEBUG:boto:Method: GET 2022-09-03T06:50:11.377 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.377 DEBUG:boto:Data: 2022-09-03T06:50:11.378 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.378 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:11.379 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.381 DEBUG:boto:Params: {} 2022-09-03T06:50:11.382 DEBUG:boto:Token: None 2022-09-03T06:50:11.384 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.385 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OiBYixuK5aOteGsnRL4x77gNWks= 2022-09-03T06:50:11.386 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OiBYixuK5aOteGsnRL4x77gNWks=', 'Content-Length': '0'} 2022-09-03T06:50:11.389 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007578c470c1705c67-006312f923-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.391 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:11.392 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:11.393 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.394 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.396 DEBUG:boto:Method: GET 2022-09-03T06:50:11.398 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.399 DEBUG:boto:Data: 2022-09-03T06:50:11.400 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.401 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:11.403 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.404 DEBUG:boto:Params: {} 2022-09-03T06:50:11.405 DEBUG:boto:Token: None 2022-09-03T06:50:11.406 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.407 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OiBYixuK5aOteGsnRL4x77gNWks= 2022-09-03T06:50:11.409 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OiBYixuK5aOteGsnRL4x77gNWks=', 'Content-Length': '0'} 2022-09-03T06:50:11.412 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004cd85513ea132933-006312f923-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.414 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:11.415 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:11.416 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.417 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.418 DEBUG:boto:Method: HEAD 2022-09-03T06:50:11.419 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.420 DEBUG:boto:Data: 2022-09-03T06:50:11.421 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.422 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:11.423 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.424 DEBUG:boto:Params: {} 2022-09-03T06:50:11.425 DEBUG:boto:Token: None 2022-09-03T06:50:11.426 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.427 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iUm/idiAd4C4Iwu/4gR98OI34iw= 2022-09-03T06:50:11.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iUm/idiAd4C4Iwu/4gR98OI34iw=', 'Content-Length': '0'} 2022-09-03T06:50:11.432 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000038ac66ac9a60b0ca-006312f923-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.434 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:11.435 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:11.436 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.437 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.438 DEBUG:boto:Method: HEAD 2022-09-03T06:50:11.439 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.440 DEBUG:boto:Data: 2022-09-03T06:50:11.441 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.442 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:11.447 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.448 DEBUG:boto:Params: {} 2022-09-03T06:50:11.449 DEBUG:boto:Token: None 2022-09-03T06:50:11.450 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.451 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iUm/idiAd4C4Iwu/4gR98OI34iw= 2022-09-03T06:50:11.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iUm/idiAd4C4Iwu/4gR98OI34iw=', 'Content-Length': '0'} 2022-09-03T06:50:11.455 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a4150b91b303820d-006312f923-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.457 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2022-09-03T06:50:11.457 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:11.458 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:11.459 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.460 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.461 DEBUG:boto:Method: GET 2022-09-03T06:50:11.462 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.463 DEBUG:boto:Data: 2022-09-03T06:50:11.465 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.466 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:11.467 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.468 DEBUG:boto:Params: {} 2022-09-03T06:50:11.470 DEBUG:boto:Token: None 2022-09-03T06:50:11.471 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.472 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OiBYixuK5aOteGsnRL4x77gNWks= 2022-09-03T06:50:11.473 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OiBYixuK5aOteGsnRL4x77gNWks=', 'Content-Length': '0'} 2022-09-03T06:50:11.476 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000052dc04d38a12fb39-006312f923-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.478 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:11.479 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:11.479 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.480 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.481 DEBUG:boto:Method: GET 2022-09-03T06:50:11.481 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.482 DEBUG:boto:Data: 2022-09-03T06:50:11.483 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.483 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:11.484 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.485 DEBUG:boto:Params: {} 2022-09-03T06:50:11.485 DEBUG:boto:Token: None 2022-09-03T06:50:11.486 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:11.487 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OiBYixuK5aOteGsnRL4x77gNWks= 2022-09-03T06:50:11.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OiBYixuK5aOteGsnRL4x77gNWks=', 'Content-Length': '0'} 2022-09-03T06:50:11.493 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e6c4ff9803c068c8-006312f923-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.496 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:11.497 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:11.498 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:11.499 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:11.500 DEBUG:boto:Method: GET 2022-09-03T06:50:11.501 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:50:11.502 DEBUG:boto:Data: 2022-09-03T06:50:11.503 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.504 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:11.505 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.506 DEBUG:boto:Params: {} 2022-09-03T06:50:11.507 DEBUG:boto:Token: None 2022-09-03T06:50:11.508 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/ 2022-09-03T06:50:11.509 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI= 2022-09-03T06:50:11.510 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI=', 'Content-Length': '0'} 2022-09-03T06:50:11.514 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006d02745cc442b04b-006312f923-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.515 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:11.517 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:11.518 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:11.519 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:11.520 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:11.521 DEBUG:boto:Method: GET 2022-09-03T06:50:11.522 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:50:11.523 DEBUG:boto:Data: 2022-09-03T06:50:11.524 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.525 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:11.525 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.526 DEBUG:boto:Params: {} 2022-09-03T06:50:11.528 DEBUG:boto:Token: None 2022-09-03T06:50:11.529 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/ 2022-09-03T06:50:11.530 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI= 2022-09-03T06:50:11.531 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI=', 'Content-Length': '0'} 2022-09-03T06:50:11.536 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ffe9a72e0a1af794-006312f923-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.537 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:11.538 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:11.539 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:11.539 DEBUG:boto:Method: HEAD 2022-09-03T06:50:11.540 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:50:11.541 DEBUG:boto:Data: 2022-09-03T06:50:11.542 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.543 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:11.544 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.545 DEBUG:boto:Params: {} 2022-09-03T06:50:11.546 DEBUG:boto:Token: None 2022-09-03T06:50:11.546 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj93 2022-09-03T06:50:11.547 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uBF551O8qyMYdqfoQHj5edt1E6w= 2022-09-03T06:50:11.548 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uBF551O8qyMYdqfoQHj5edt1E6w=', 'Content-Length': '0'} 2022-09-03T06:50:11.551 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002cf84d020f2efa00-006312f923-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.551 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:11.552 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:11.553 DEBUG:boto:Method: HEAD 2022-09-03T06:50:11.553 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:50:11.554 DEBUG:boto:Data: 2022-09-03T06:50:11.555 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.555 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:11.556 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.556 DEBUG:boto:Params: {} 2022-09-03T06:50:11.557 DEBUG:boto:Token: None 2022-09-03T06:50:11.558 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj93 2022-09-03T06:50:11.559 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uBF551O8qyMYdqfoQHj5edt1E6w= 2022-09-03T06:50:11.560 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uBF551O8qyMYdqfoQHj5edt1E6w=', 'Content-Length': '0'} 2022-09-03T06:50:11.562 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002de338e889ce1f12-006312f923-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.563 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2022-09-03T06:50:11.563 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:11.564 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:11.564 DEBUG:boto:Method: GET 2022-09-03T06:50:11.565 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:50:11.565 DEBUG:boto:Data: 2022-09-03T06:50:11.566 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.567 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:11.567 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.568 DEBUG:boto:Params: {} 2022-09-03T06:50:11.569 DEBUG:boto:Token: None 2022-09-03T06:50:11.569 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj93 2022-09-03T06:50:11.570 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:di27kprm99vzAcZkJ2RRox2SDX8= 2022-09-03T06:50:11.571 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:di27kprm99vzAcZkJ2RRox2SDX8=', 'Content-Length': '0'} 2022-09-03T06:50:11.573 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e25afa9afc8d7c1f-006312f923-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.574 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:11.574 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:11.575 DEBUG:boto:Method: GET 2022-09-03T06:50:11.576 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:50:11.576 DEBUG:boto:Data: 2022-09-03T06:50:11.577 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.577 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:11.578 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.579 DEBUG:boto:Params: {} 2022-09-03T06:50:11.580 DEBUG:boto:Token: None 2022-09-03T06:50:11.581 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj93 2022-09-03T06:50:11.582 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:di27kprm99vzAcZkJ2RRox2SDX8= 2022-09-03T06:50:11.582 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:di27kprm99vzAcZkJ2RRox2SDX8=', 'Content-Length': '0'} 2022-09-03T06:50:11.585 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e17b1e11adfd5efa-006312f923-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.626 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:11.627 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:11.628 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:11.628 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:11.629 DEBUG:boto:Method: GET 2022-09-03T06:50:11.629 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:50:11.630 DEBUG:boto:Data: 2022-09-03T06:50:11.630 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.632 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:11.632 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.633 DEBUG:boto:Params: {} 2022-09-03T06:50:11.634 DEBUG:boto:Token: None 2022-09-03T06:50:11.635 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/ 2022-09-03T06:50:11.636 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI= 2022-09-03T06:50:11.637 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI=', 'Content-Length': '0'} 2022-09-03T06:50:11.639 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d91ee45911cafbec-006312f923-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.640 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:11.641 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:11.642 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:11.643 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:11.643 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:11.644 DEBUG:boto:Method: GET 2022-09-03T06:50:11.645 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:50:11.645 DEBUG:boto:Data: 2022-09-03T06:50:11.646 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.647 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:11.648 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.648 DEBUG:boto:Params: {} 2022-09-03T06:50:11.649 DEBUG:boto:Token: None 2022-09-03T06:50:11.649 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/ 2022-09-03T06:50:11.650 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI= 2022-09-03T06:50:11.651 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI=', 'Content-Length': '0'} 2022-09-03T06:50:11.655 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c1e91cda57293c67-006312f923-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.656 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:11.657 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:50:11.657 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:11.658 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:11.659 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.659 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.660 DEBUG:boto:Method: GET 2022-09-03T06:50:11.660 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.661 DEBUG:boto:Data: 2022-09-03T06:50:11.661 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.662 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:11.663 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.664 DEBUG:boto:Params: {} 2022-09-03T06:50:11.665 DEBUG:boto:Token: None 2022-09-03T06:50:11.665 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.666 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qGqIDxY5iL+sEOT0JvGfwXt+EEQ= 2022-09-03T06:50:11.667 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qGqIDxY5iL+sEOT0JvGfwXt+EEQ=', 'Content-Length': '0'} 2022-09-03T06:50:11.669 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000827b6e5061232481-006312f923-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.715 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:11.716 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:11.717 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.717 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.718 DEBUG:boto:Method: GET 2022-09-03T06:50:11.719 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.719 DEBUG:boto:Data: 2022-09-03T06:50:11.720 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.720 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:11.721 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.722 DEBUG:boto:Params: {} 2022-09-03T06:50:11.723 DEBUG:boto:Token: None 2022-09-03T06:50:11.724 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.724 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qGqIDxY5iL+sEOT0JvGfwXt+EEQ= 2022-09-03T06:50:11.725 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qGqIDxY5iL+sEOT0JvGfwXt+EEQ=', 'Content-Length': '0'} 2022-09-03T06:50:11.729 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a64f239c921c1884-006312f923-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.730 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:11.731 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:11.732 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.732 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.733 DEBUG:boto:Method: HEAD 2022-09-03T06:50:11.734 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.734 DEBUG:boto:Data: 2022-09-03T06:50:11.735 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.735 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:11.736 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.736 DEBUG:boto:Params: {} 2022-09-03T06:50:11.737 DEBUG:boto:Token: None 2022-09-03T06:50:11.738 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.739 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IX/fFb8o5MIGwpkKaoh3xW3dNYs= 2022-09-03T06:50:11.740 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IX/fFb8o5MIGwpkKaoh3xW3dNYs=', 'Content-Length': '0'} 2022-09-03T06:50:11.742 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b9b3d597b44b519c-006312f923-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.744 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:11.745 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:11.746 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.747 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.748 DEBUG:boto:Method: HEAD 2022-09-03T06:50:11.749 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.750 DEBUG:boto:Data: 2022-09-03T06:50:11.751 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.751 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:11.752 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.753 DEBUG:boto:Params: {} 2022-09-03T06:50:11.754 DEBUG:boto:Token: None 2022-09-03T06:50:11.755 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.755 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IX/fFb8o5MIGwpkKaoh3xW3dNYs= 2022-09-03T06:50:11.756 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IX/fFb8o5MIGwpkKaoh3xW3dNYs=', 'Content-Length': '0'} 2022-09-03T06:50:11.759 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000078f3a3cd23414636-006312f923-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.759 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:50:11.760 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:11.760 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:11.761 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.763 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.764 DEBUG:boto:Method: GET 2022-09-03T06:50:11.765 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.766 DEBUG:boto:Data: 2022-09-03T06:50:11.766 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.767 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:11.768 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.769 DEBUG:boto:Params: {} 2022-09-03T06:50:11.771 DEBUG:boto:Token: None 2022-09-03T06:50:11.772 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.773 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qGqIDxY5iL+sEOT0JvGfwXt+EEQ= 2022-09-03T06:50:11.774 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qGqIDxY5iL+sEOT0JvGfwXt+EEQ=', 'Content-Length': '0'} 2022-09-03T06:50:11.776 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003072c2e3bae77540-006312f923-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.777 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:11.778 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:11.779 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.780 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.780 DEBUG:boto:Method: GET 2022-09-03T06:50:11.781 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.781 DEBUG:boto:Data: 2022-09-03T06:50:11.782 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.783 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:11.783 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.784 DEBUG:boto:Params: {} 2022-09-03T06:50:11.784 DEBUG:boto:Token: None 2022-09-03T06:50:11.785 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:11.786 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qGqIDxY5iL+sEOT0JvGfwXt+EEQ= 2022-09-03T06:50:11.786 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qGqIDxY5iL+sEOT0JvGfwXt+EEQ=', 'Content-Length': '0'} 2022-09-03T06:50:11.789 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000975d89af9e810208-006312f923-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.789 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:11.790 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:11.791 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:11.791 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:11.792 DEBUG:boto:Method: GET 2022-09-03T06:50:11.792 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:50:11.793 DEBUG:boto:Data: 2022-09-03T06:50:11.793 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.794 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:11.795 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.795 DEBUG:boto:Params: {} 2022-09-03T06:50:11.796 DEBUG:boto:Token: None 2022-09-03T06:50:11.796 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/ 2022-09-03T06:50:11.797 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI= 2022-09-03T06:50:11.798 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI=', 'Content-Length': '0'} 2022-09-03T06:50:11.801 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a29396f784ecfdfe-006312f923-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.803 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:11.804 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:11.805 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:11.806 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:11.807 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:11.808 DEBUG:boto:Method: GET 2022-09-03T06:50:11.809 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:50:11.810 DEBUG:boto:Data: 2022-09-03T06:50:11.811 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.812 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:11.813 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.814 DEBUG:boto:Params: {} 2022-09-03T06:50:11.815 DEBUG:boto:Token: None 2022-09-03T06:50:11.816 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/ 2022-09-03T06:50:11.817 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI= 2022-09-03T06:50:11.819 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI=', 'Content-Length': '0'} 2022-09-03T06:50:11.824 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000184bde67d8bddca2-006312f923-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.826 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:11.827 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:11.828 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:11.829 DEBUG:boto:Method: HEAD 2022-09-03T06:50:11.829 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:50:11.830 DEBUG:boto:Data: 2022-09-03T06:50:11.830 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.831 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:11.831 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.832 DEBUG:boto:Params: {} 2022-09-03T06:50:11.833 DEBUG:boto:Token: None 2022-09-03T06:50:11.833 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj95 2022-09-03T06:50:11.834 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pZyA7AKquycx2cTHFG3wrhoXaWA= 2022-09-03T06:50:11.835 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pZyA7AKquycx2cTHFG3wrhoXaWA=', 'Content-Length': '0'} 2022-09-03T06:50:11.837 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002584bee1460cf9d1-006312f923-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.838 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:11.838 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:11.839 DEBUG:boto:Method: HEAD 2022-09-03T06:50:11.840 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:50:11.840 DEBUG:boto:Data: 2022-09-03T06:50:11.841 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.841 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:11.842 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.842 DEBUG:boto:Params: {} 2022-09-03T06:50:11.843 DEBUG:boto:Token: None 2022-09-03T06:50:11.844 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj95 2022-09-03T06:50:11.844 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pZyA7AKquycx2cTHFG3wrhoXaWA= 2022-09-03T06:50:11.845 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pZyA7AKquycx2cTHFG3wrhoXaWA=', 'Content-Length': '0'} 2022-09-03T06:50:11.847 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000077cf06c49250456f-006312f923-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.848 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:50:11.848 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:11.849 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:11.850 DEBUG:boto:Method: GET 2022-09-03T06:50:11.850 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:50:11.851 DEBUG:boto:Data: 2022-09-03T06:50:11.851 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.852 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:11.852 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.853 DEBUG:boto:Params: {} 2022-09-03T06:50:11.854 DEBUG:boto:Token: None 2022-09-03T06:50:11.854 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj95 2022-09-03T06:50:11.855 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4OGWAEndHpjoWy+6V9uYlpjYLjM= 2022-09-03T06:50:11.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4OGWAEndHpjoWy+6V9uYlpjYLjM=', 'Content-Length': '0'} 2022-09-03T06:50:11.859 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b9ee02c40a39e48f-006312f923-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.860 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:11.860 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:11.861 DEBUG:boto:Method: GET 2022-09-03T06:50:11.862 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:50:11.862 DEBUG:boto:Data: 2022-09-03T06:50:11.863 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.863 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:11.864 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.865 DEBUG:boto:Params: {} 2022-09-03T06:50:11.865 DEBUG:boto:Token: None 2022-09-03T06:50:11.866 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/obj95 2022-09-03T06:50:11.867 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4OGWAEndHpjoWy+6V9uYlpjYLjM= 2022-09-03T06:50:11.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4OGWAEndHpjoWy+6V9uYlpjYLjM=', 'Content-Length': '0'} 2022-09-03T06:50:11.870 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000385ceaa5d6ef8318-006312f923-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.916 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:11.917 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:11.918 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:11.919 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:11.920 DEBUG:boto:Method: GET 2022-09-03T06:50:11.920 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:50:11.922 DEBUG:boto:Data: 2022-09-03T06:50:11.922 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.923 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:11.924 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.924 DEBUG:boto:Params: {} 2022-09-03T06:50:11.925 DEBUG:boto:Token: None 2022-09-03T06:50:11.926 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/ 2022-09-03T06:50:11.926 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI= 2022-09-03T06:50:11.927 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI=', 'Content-Length': '0'} 2022-09-03T06:50:11.930 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b0e1ee2e3e336791-006312f923-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.931 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:11.932 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:11.932 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:11.933 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:11.933 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:11.934 DEBUG:boto:Method: GET 2022-09-03T06:50:11.935 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:50:11.935 DEBUG:boto:Data: 2022-09-03T06:50:11.936 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.937 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:11.937 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.938 DEBUG:boto:Params: {} 2022-09-03T06:50:11.939 DEBUG:boto:Token: None 2022-09-03T06:50:11.939 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/ 2022-09-03T06:50:11.940 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI= 2022-09-03T06:50:11.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jGDNZT/V9GgcFobryNh/etkPFnI=', 'Content-Length': '0'} 2022-09-03T06:50:11.945 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009f79591931ae3cfe-006312f923-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.946 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:11.947 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-75 zones={test-zone2, test-zone1} 2022-09-03T06:50:11.948 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-75 zones={test-zone3, test-zone1} 2022-09-03T06:50:11.948 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:11.949 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:11.950 DEBUG:boto:Method: HEAD 2022-09-03T06:50:11.950 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:50:11.951 DEBUG:boto:Data: 2022-09-03T06:50:11.952 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.952 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:11.953 DEBUG:boto:Port: 8001 2022-09-03T06:50:11.953 DEBUG:boto:Params: {} 2022-09-03T06:50:11.954 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T06:50:11.955 DEBUG:boto:Token: None 2022-09-03T06:50:11.955 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/ 2022-09-03T06:50:11.956 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vAGC0ZRJGwoTYvJQVIQrwFKsQO0= 2022-09-03T06:50:11.957 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vAGC0ZRJGwoTYvJQVIQrwFKsQO0=', 'Content-Length': '0'} 2022-09-03T06:50:11.961 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '35'), ('X-RGW-Bytes-Used', '105'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000183dd5faa65fa612-006312f923-1851-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.961 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:11.962 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:11.963 DEBUG:boto:Method: HEAD 2022-09-03T06:50:11.963 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:50:11.964 DEBUG:boto:Data: 2022-09-03T06:50:11.964 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.965 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:11.966 DEBUG:boto:Port: 8000 2022-09-03T06:50:11.966 DEBUG:boto:Params: {} 2022-09-03T06:50:11.967 DEBUG:boto:Token: None 2022-09-03T06:50:11.967 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/ 2022-09-03T06:50:11.968 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vAGC0ZRJGwoTYvJQVIQrwFKsQO0= 2022-09-03T06:50:11.969 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vAGC0ZRJGwoTYvJQVIQrwFKsQO0=', 'Content-Length': '0'} 2022-09-03T06:50:11.973 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '35'), ('X-RGW-Bytes-Used', '105'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b4ec86a85549ecfa-006312f923-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.974 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:50:11.975 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:11.975 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:11.976 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:11.976 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:11.977 DEBUG:boto:Method: GET 2022-09-03T06:50:11.978 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:11.978 DEBUG:boto:Data: 2022-09-03T06:50:11.979 DEBUG:boto:Headers: {} 2022-09-03T06:50:11.979 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:11.980 DEBUG:boto:Port: 8001 2022-09-03T06:50:11.980 DEBUG:boto:Params: {} 2022-09-03T06:50:11.981 DEBUG:boto:Token: None 2022-09-03T06:50:11.982 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:11 GMT /dsgvcy-75/?versions 2022-09-03T06:50:11.982 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wZRl0F8tE5qM/Wsn0E7lR8q3HBE= 2022-09-03T06:50:11.983 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:11 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wZRl0F8tE5qM/Wsn0E7lR8q3HBE=', 'Content-Length': '0'} 2022-09-03T06:50:11.986 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000082aa457ce6c91d0e-006312f923-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:11 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:11.988 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj111nulltrue2022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj148nulltrue2022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj185nulltrue2022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj37nulltrue2022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj74nulltrue2022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:11.990 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:50:11.991 DEBUG:tasks.rgw_multi.zone_rados:o=obj111 2022-09-03T06:50:11.991 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:50:11.992 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2022-09-03T06:50:11.992 DEBUG:tasks.rgw_multi.zone_rados:o=obj124 2022-09-03T06:50:11.993 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2022-09-03T06:50:11.993 DEBUG:tasks.rgw_multi.zone_rados:o=obj148 2022-09-03T06:50:11.994 DEBUG:tasks.rgw_multi.zone_rados:o=obj155 2022-09-03T06:50:11.994 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:50:11.995 DEBUG:tasks.rgw_multi.zone_rados:o=obj185 2022-09-03T06:50:11.995 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:50:11.996 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:50:11.996 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:50:11.997 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2022-09-03T06:50:11.997 DEBUG:tasks.rgw_multi.zone_rados:o=obj31 2022-09-03T06:50:11.998 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:50:11.998 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:50:11.999 DEBUG:tasks.rgw_multi.zone_rados:o=obj37 2022-09-03T06:50:11.999 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:50:12.000 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:50:12.000 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:50:12.001 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:50:12.001 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:50:12.001 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:50:12.002 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2022-09-03T06:50:12.003 DEBUG:tasks.rgw_multi.zone_rados:o=obj62 2022-09-03T06:50:12.003 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:50:12.004 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:50:12.004 DEBUG:tasks.rgw_multi.zone_rados:o=obj74 2022-09-03T06:50:12.004 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:50:12.005 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:50:12.005 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2022-09-03T06:50:12.006 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:50:12.006 DEBUG:tasks.rgw_multi.zone_rados:o=obj93 2022-09-03T06:50:12.007 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:50:12.007 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:50:12.008 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:12.009 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:12.010 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:12.010 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:12.011 DEBUG:boto:Method: GET 2022-09-03T06:50:12.011 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:12.012 DEBUG:boto:Data: 2022-09-03T06:50:12.012 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.013 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.013 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.014 DEBUG:boto:Params: {} 2022-09-03T06:50:12.015 DEBUG:boto:Token: None 2022-09-03T06:50:12.015 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/?versions 2022-09-03T06:50:12.016 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1gAieKAIym2/x/YmtfFFeMf3gZo= 2022-09-03T06:50:12.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1gAieKAIym2/x/YmtfFFeMf3gZo=', 'Content-Length': '0'} 2022-09-03T06:50:12.022 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000238c2ed3027c29c0-006312f924-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.023 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj111nulltrue2022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj148nulltrue2022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj185nulltrue2022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj37nulltrue2022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj74nulltrue2022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:12.025 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:50:12.026 DEBUG:tasks.rgw_multi.zone_rados:o=obj111 2022-09-03T06:50:12.027 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:50:12.027 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2022-09-03T06:50:12.028 DEBUG:tasks.rgw_multi.zone_rados:o=obj124 2022-09-03T06:50:12.028 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2022-09-03T06:50:12.029 DEBUG:tasks.rgw_multi.zone_rados:o=obj148 2022-09-03T06:50:12.029 DEBUG:tasks.rgw_multi.zone_rados:o=obj155 2022-09-03T06:50:12.030 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:50:12.030 DEBUG:tasks.rgw_multi.zone_rados:o=obj185 2022-09-03T06:50:12.031 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:50:12.031 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:50:12.032 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:50:12.032 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2022-09-03T06:50:12.033 DEBUG:tasks.rgw_multi.zone_rados:o=obj31 2022-09-03T06:50:12.033 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:50:12.034 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:50:12.034 DEBUG:tasks.rgw_multi.zone_rados:o=obj37 2022-09-03T06:50:12.035 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:50:12.035 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:50:12.036 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:50:12.036 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:50:12.037 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:50:12.037 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:50:12.038 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2022-09-03T06:50:12.038 DEBUG:tasks.rgw_multi.zone_rados:o=obj62 2022-09-03T06:50:12.039 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:50:12.039 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:50:12.040 DEBUG:tasks.rgw_multi.zone_rados:o=obj74 2022-09-03T06:50:12.040 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:50:12.041 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:50:12.041 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2022-09-03T06:50:12.041 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:50:12.042 DEBUG:tasks.rgw_multi.zone_rados:o=obj93 2022-09-03T06:50:12.042 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:50:12.043 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:12.044 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:12.044 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:12.045 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:12.045 DEBUG:boto:Method: GET 2022-09-03T06:50:12.046 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:12.046 DEBUG:boto:Data: 2022-09-03T06:50:12.047 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.047 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.048 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.049 DEBUG:boto:Params: {} 2022-09-03T06:50:12.049 DEBUG:boto:Token: None 2022-09-03T06:50:12.050 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/?versions 2022-09-03T06:50:12.051 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1gAieKAIym2/x/YmtfFFeMf3gZo= 2022-09-03T06:50:12.051 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1gAieKAIym2/x/YmtfFFeMf3gZo=', 'Content-Length': '0'} 2022-09-03T06:50:12.055 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008a4e72831f487c46-006312f924-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.055 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj111nulltrue2022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj148nulltrue2022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj185nulltrue2022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj37nulltrue2022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj74nulltrue2022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:12.060 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:12.061 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:12.062 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:12.063 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:12.064 DEBUG:boto:Method: GET 2022-09-03T06:50:12.065 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:12.066 DEBUG:boto:Data: 2022-09-03T06:50:12.067 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.068 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.069 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.069 DEBUG:boto:Params: {} 2022-09-03T06:50:12.071 DEBUG:boto:Token: None 2022-09-03T06:50:12.072 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/?versions 2022-09-03T06:50:12.072 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1gAieKAIym2/x/YmtfFFeMf3gZo= 2022-09-03T06:50:12.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1gAieKAIym2/x/YmtfFFeMf3gZo=', 'Content-Length': '0'} 2022-09-03T06:50:12.078 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006f5640f0d22bd31c-006312f924-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.079 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj111nulltrue2022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj148nulltrue2022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj185nulltrue2022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj37nulltrue2022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj74nulltrue2022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:12.081 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:50:12.082 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:12.082 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:12.083 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.083 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.084 DEBUG:boto:Method: GET 2022-09-03T06:50:12.084 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.085 DEBUG:boto:Data: 2022-09-03T06:50:12.085 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.086 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.087 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.088 DEBUG:boto:Params: {} 2022-09-03T06:50:12.088 DEBUG:boto:Token: None 2022-09-03T06:50:12.089 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.090 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sCUFr5lGVROT7SilrLVdkEFRmBo= 2022-09-03T06:50:12.090 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sCUFr5lGVROT7SilrLVdkEFRmBo=', 'Content-Length': '0'} 2022-09-03T06:50:12.093 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d3f908bf3579efc6-006312f924-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.093 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:12.094 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:12.095 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.095 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.096 DEBUG:boto:Method: GET 2022-09-03T06:50:12.097 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.098 DEBUG:boto:Data: 2022-09-03T06:50:12.099 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.100 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.101 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.102 DEBUG:boto:Params: {} 2022-09-03T06:50:12.103 DEBUG:boto:Token: None 2022-09-03T06:50:12.104 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.105 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sCUFr5lGVROT7SilrLVdkEFRmBo= 2022-09-03T06:50:12.106 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sCUFr5lGVROT7SilrLVdkEFRmBo=', 'Content-Length': '0'} 2022-09-03T06:50:12.109 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b50c23c78bddb20-006312f924-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.110 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:12.111 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:12.111 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.112 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.113 DEBUG:boto:Method: HEAD 2022-09-03T06:50:12.113 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.114 DEBUG:boto:Data: 2022-09-03T06:50:12.114 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.115 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.116 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.116 DEBUG:boto:Params: {} 2022-09-03T06:50:12.117 DEBUG:boto:Token: None 2022-09-03T06:50:12.118 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.120 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DrlM1+DLmvU7NmrA5xerHGypRkY= 2022-09-03T06:50:12.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DrlM1+DLmvU7NmrA5xerHGypRkY=', 'Content-Length': '0'} 2022-09-03T06:50:12.123 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1cb0ea0ad310f0a-006312f924-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.125 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:12.125 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:12.126 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.127 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.128 DEBUG:boto:Method: HEAD 2022-09-03T06:50:12.129 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.130 DEBUG:boto:Data: 2022-09-03T06:50:12.131 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.132 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.133 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.134 DEBUG:boto:Params: {} 2022-09-03T06:50:12.135 DEBUG:boto:Token: None 2022-09-03T06:50:12.136 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.137 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DrlM1+DLmvU7NmrA5xerHGypRkY= 2022-09-03T06:50:12.138 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DrlM1+DLmvU7NmrA5xerHGypRkY=', 'Content-Length': '0'} 2022-09-03T06:50:12.141 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000004780e2a39f3dc5f-006312f924-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.142 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:50:12.143 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:12.145 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:12.146 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.147 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.147 DEBUG:boto:Method: GET 2022-09-03T06:50:12.148 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.149 DEBUG:boto:Data: 2022-09-03T06:50:12.150 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.151 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.152 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.153 DEBUG:boto:Params: {} 2022-09-03T06:50:12.154 DEBUG:boto:Token: None 2022-09-03T06:50:12.155 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.157 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sCUFr5lGVROT7SilrLVdkEFRmBo= 2022-09-03T06:50:12.158 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sCUFr5lGVROT7SilrLVdkEFRmBo=', 'Content-Length': '0'} 2022-09-03T06:50:12.160 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000085dfea6dc7b22103-006312f924-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.162 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:12.162 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:12.163 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.164 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.165 DEBUG:boto:Method: GET 2022-09-03T06:50:12.166 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.166 DEBUG:boto:Data: 2022-09-03T06:50:12.167 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.168 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.169 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.170 DEBUG:boto:Params: {} 2022-09-03T06:50:12.171 DEBUG:boto:Token: None 2022-09-03T06:50:12.172 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:12.173 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sCUFr5lGVROT7SilrLVdkEFRmBo= 2022-09-03T06:50:12.174 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sCUFr5lGVROT7SilrLVdkEFRmBo=', 'Content-Length': '0'} 2022-09-03T06:50:12.176 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000866e2089367dd808-006312f924-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.177 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:12.178 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:12.178 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:12.179 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:12.179 DEBUG:boto:Method: GET 2022-09-03T06:50:12.180 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:50:12.180 DEBUG:boto:Data: 2022-09-03T06:50:12.181 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.182 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.182 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.183 DEBUG:boto:Params: {} 2022-09-03T06:50:12.183 DEBUG:boto:Token: None 2022-09-03T06:50:12.184 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/ 2022-09-03T06:50:12.185 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4= 2022-09-03T06:50:12.185 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4=', 'Content-Length': '0'} 2022-09-03T06:50:12.189 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000df989368358c8b8c-006312f924-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.189 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:12.191 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:12.191 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:12.192 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:12.192 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:12.193 DEBUG:boto:Method: GET 2022-09-03T06:50:12.193 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:50:12.194 DEBUG:boto:Data: 2022-09-03T06:50:12.195 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.195 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.196 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.196 DEBUG:boto:Params: {} 2022-09-03T06:50:12.197 DEBUG:boto:Token: None 2022-09-03T06:50:12.198 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/ 2022-09-03T06:50:12.199 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4= 2022-09-03T06:50:12.199 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4=', 'Content-Length': '0'} 2022-09-03T06:50:12.204 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c023f91172dd077c-006312f924-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.205 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:12.206 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:12.207 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:12.208 DEBUG:boto:Method: HEAD 2022-09-03T06:50:12.208 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:50:12.209 DEBUG:boto:Data: 2022-09-03T06:50:12.209 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.210 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.211 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.211 DEBUG:boto:Params: {} 2022-09-03T06:50:12.212 DEBUG:boto:Token: None 2022-09-03T06:50:12.213 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj11 2022-09-03T06:50:12.213 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j4YeW/SRDwdYW47L1LtuRhuyzwY= 2022-09-03T06:50:12.214 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j4YeW/SRDwdYW47L1LtuRhuyzwY=', 'Content-Length': '0'} 2022-09-03T06:50:12.216 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000046d8c8456927058-006312f924-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.217 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:12.217 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:12.218 DEBUG:boto:Method: HEAD 2022-09-03T06:50:12.229 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:50:12.230 DEBUG:boto:Data: 2022-09-03T06:50:12.231 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.231 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.232 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.233 DEBUG:boto:Params: {} 2022-09-03T06:50:12.233 DEBUG:boto:Token: None 2022-09-03T06:50:12.235 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj11 2022-09-03T06:50:12.236 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j4YeW/SRDwdYW47L1LtuRhuyzwY= 2022-09-03T06:50:12.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j4YeW/SRDwdYW47L1LtuRhuyzwY=', 'Content-Length': '0'} 2022-09-03T06:50:12.241 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003ab28840d355654b-006312f924-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.243 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:50:12.244 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:12.245 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:12.246 DEBUG:boto:Method: GET 2022-09-03T06:50:12.246 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:50:12.247 DEBUG:boto:Data: 2022-09-03T06:50:12.248 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.248 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.250 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.251 DEBUG:boto:Params: {} 2022-09-03T06:50:12.252 DEBUG:boto:Token: None 2022-09-03T06:50:12.254 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj11 2022-09-03T06:50:12.255 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q2MNLYzYED2m+MJ8tgeyv8OjHWU= 2022-09-03T06:50:12.256 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q2MNLYzYED2m+MJ8tgeyv8OjHWU=', 'Content-Length': '0'} 2022-09-03T06:50:12.259 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096f93edcaf39a949-006312f924-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.260 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:12.261 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:12.262 DEBUG:boto:Method: GET 2022-09-03T06:50:12.263 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:50:12.264 DEBUG:boto:Data: 2022-09-03T06:50:12.265 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.267 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.268 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.268 DEBUG:boto:Params: {} 2022-09-03T06:50:12.270 DEBUG:boto:Token: None 2022-09-03T06:50:12.271 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj11 2022-09-03T06:50:12.271 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q2MNLYzYED2m+MJ8tgeyv8OjHWU= 2022-09-03T06:50:12.272 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q2MNLYzYED2m+MJ8tgeyv8OjHWU=', 'Content-Length': '0'} 2022-09-03T06:50:12.276 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005ab3b4e22d83949b-006312f924-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.322 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:12.323 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:12.326 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:12.327 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:12.328 DEBUG:boto:Method: GET 2022-09-03T06:50:12.329 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:50:12.330 DEBUG:boto:Data: 2022-09-03T06:50:12.331 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.333 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.334 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.335 DEBUG:boto:Params: {} 2022-09-03T06:50:12.337 DEBUG:boto:Token: None 2022-09-03T06:50:12.338 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/ 2022-09-03T06:50:12.339 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4= 2022-09-03T06:50:12.341 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4=', 'Content-Length': '0'} 2022-09-03T06:50:12.345 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001d771a73f6f76167-006312f924-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.347 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:12.348 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:12.349 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:12.351 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:12.352 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:12.353 DEBUG:boto:Method: GET 2022-09-03T06:50:12.354 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:50:12.355 DEBUG:boto:Data: 2022-09-03T06:50:12.356 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.356 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.357 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.357 DEBUG:boto:Params: {} 2022-09-03T06:50:12.358 DEBUG:boto:Token: None 2022-09-03T06:50:12.358 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/ 2022-09-03T06:50:12.359 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4= 2022-09-03T06:50:12.360 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4=', 'Content-Length': '0'} 2022-09-03T06:50:12.365 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001818e820b5fc56e8-006312f924-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.366 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:12.367 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj111 2022-09-03T06:50:12.367 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:12.368 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:12.368 DEBUG:boto:path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.369 DEBUG:boto:auth_path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.369 DEBUG:boto:Method: GET 2022-09-03T06:50:12.370 DEBUG:boto:Path: /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.371 DEBUG:boto:Data: 2022-09-03T06:50:12.371 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.372 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.373 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.374 DEBUG:boto:Params: {} 2022-09-03T06:50:12.374 DEBUG:boto:Token: None 2022-09-03T06:50:12.375 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.375 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+rmQC7/muzGqZ71+ZZ9gaQZ7Uq4= 2022-09-03T06:50:12.376 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+rmQC7/muzGqZ71+ZZ9gaQZ7Uq4=', 'Content-Length': '0'} 2022-09-03T06:50:12.378 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000089b0e062cff59cbf-006312f924-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.379 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:12.380 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:12.381 DEBUG:boto:path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.381 DEBUG:boto:auth_path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.382 DEBUG:boto:Method: GET 2022-09-03T06:50:12.382 DEBUG:boto:Path: /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.384 DEBUG:boto:Data: 2022-09-03T06:50:12.385 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.386 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.387 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.389 DEBUG:boto:Params: {} 2022-09-03T06:50:12.390 DEBUG:boto:Token: None 2022-09-03T06:50:12.391 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.392 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+rmQC7/muzGqZ71+ZZ9gaQZ7Uq4= 2022-09-03T06:50:12.393 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+rmQC7/muzGqZ71+ZZ9gaQZ7Uq4=', 'Content-Length': '0'} 2022-09-03T06:50:12.395 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd606f8996f6ab55-006312f924-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.396 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:12.397 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:12.397 DEBUG:boto:path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.398 DEBUG:boto:auth_path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.398 DEBUG:boto:Method: HEAD 2022-09-03T06:50:12.399 DEBUG:boto:Path: /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.400 DEBUG:boto:Data: 2022-09-03T06:50:12.400 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.400 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.401 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.401 DEBUG:boto:Params: {} 2022-09-03T06:50:12.403 DEBUG:boto:Token: None 2022-09-03T06:50:12.404 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.404 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tT5XrhYKRARFNSrl7YOmGT8eqLw= 2022-09-03T06:50:12.405 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tT5XrhYKRARFNSrl7YOmGT8eqLw=', 'Content-Length': '0'} 2022-09-03T06:50:12.408 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e39413ca0ef1d1f6-006312f924-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.409 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:12.410 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:12.410 DEBUG:boto:path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.411 DEBUG:boto:auth_path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.411 DEBUG:boto:Method: HEAD 2022-09-03T06:50:12.412 DEBUG:boto:Path: /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.412 DEBUG:boto:Data: 2022-09-03T06:50:12.413 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.413 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.414 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.415 DEBUG:boto:Params: {} 2022-09-03T06:50:12.415 DEBUG:boto:Token: None 2022-09-03T06:50:12.416 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.416 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tT5XrhYKRARFNSrl7YOmGT8eqLw= 2022-09-03T06:50:12.417 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tT5XrhYKRARFNSrl7YOmGT8eqLw=', 'Content-Length': '0'} 2022-09-03T06:50:12.420 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006a32e48d3dabecee-006312f924-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.421 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj111 2022-09-03T06:50:12.421 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:12.422 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:12.422 DEBUG:boto:path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.423 DEBUG:boto:auth_path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.424 DEBUG:boto:Method: GET 2022-09-03T06:50:12.424 DEBUG:boto:Path: /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.425 DEBUG:boto:Data: 2022-09-03T06:50:12.425 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.426 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.426 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.427 DEBUG:boto:Params: {} 2022-09-03T06:50:12.427 DEBUG:boto:Token: None 2022-09-03T06:50:12.428 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.429 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+rmQC7/muzGqZ71+ZZ9gaQZ7Uq4= 2022-09-03T06:50:12.430 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+rmQC7/muzGqZ71+ZZ9gaQZ7Uq4=', 'Content-Length': '0'} 2022-09-03T06:50:12.433 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008d97f741b1765a36-006312f924-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.434 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:12.434 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:12.435 DEBUG:boto:path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.435 DEBUG:boto:auth_path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.436 DEBUG:boto:Method: GET 2022-09-03T06:50:12.436 DEBUG:boto:Path: /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.437 DEBUG:boto:Data: 2022-09-03T06:50:12.437 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.438 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.438 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.439 DEBUG:boto:Params: {} 2022-09-03T06:50:12.440 DEBUG:boto:Token: None 2022-09-03T06:50:12.441 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:12.442 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+rmQC7/muzGqZ71+ZZ9gaQZ7Uq4= 2022-09-03T06:50:12.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+rmQC7/muzGqZ71+ZZ9gaQZ7Uq4=', 'Content-Length': '0'} 2022-09-03T06:50:12.445 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b37268ceeab6fe17-006312f924-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.486 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:12.487 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:12.488 DEBUG:boto:path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:12.489 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:12.490 DEBUG:boto:Method: GET 2022-09-03T06:50:12.491 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj111 2022-09-03T06:50:12.492 DEBUG:boto:Data: 2022-09-03T06:50:12.493 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.493 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.494 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.494 DEBUG:boto:Params: {} 2022-09-03T06:50:12.495 DEBUG:boto:Token: None 2022-09-03T06:50:12.496 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/ 2022-09-03T06:50:12.496 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4= 2022-09-03T06:50:12.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4=', 'Content-Length': '0'} 2022-09-03T06:50:12.500 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000071aac5ce3e979607-006312f924-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.501 DEBUG:boto:b'dsgvcy-75obj1111000falseobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:12.502 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:12.503 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:12.504 DEBUG:boto:path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:12.504 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:12.505 DEBUG:boto:Method: GET 2022-09-03T06:50:12.505 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj111 2022-09-03T06:50:12.506 DEBUG:boto:Data: 2022-09-03T06:50:12.507 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.508 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.508 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.509 DEBUG:boto:Params: {} 2022-09-03T06:50:12.510 DEBUG:boto:Token: None 2022-09-03T06:50:12.510 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/ 2022-09-03T06:50:12.511 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4= 2022-09-03T06:50:12.512 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4=', 'Content-Length': '0'} 2022-09-03T06:50:12.517 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007b80f0d6ae856a13-006312f924-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.518 DEBUG:boto:b'dsgvcy-75obj1111000falseobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:12.519 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:12.520 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:12.521 DEBUG:boto:Method: HEAD 2022-09-03T06:50:12.522 DEBUG:boto:Path: /dsgvcy-75/obj111 2022-09-03T06:50:12.524 DEBUG:boto:Data: 2022-09-03T06:50:12.525 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.525 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.526 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.527 DEBUG:boto:Params: {} 2022-09-03T06:50:12.528 DEBUG:boto:Token: None 2022-09-03T06:50:12.529 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj111 2022-09-03T06:50:12.530 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R4yAOY89IUNUE3UJb4F8HPAqSJA= 2022-09-03T06:50:12.531 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R4yAOY89IUNUE3UJb4F8HPAqSJA=', 'Content-Length': '0'} 2022-09-03T06:50:12.533 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e7cb0b9867a4237a-006312f924-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.534 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:12.534 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:12.535 DEBUG:boto:Method: HEAD 2022-09-03T06:50:12.535 DEBUG:boto:Path: /dsgvcy-75/obj111 2022-09-03T06:50:12.536 DEBUG:boto:Data: 2022-09-03T06:50:12.536 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.537 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.538 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.538 DEBUG:boto:Params: {} 2022-09-03T06:50:12.539 DEBUG:boto:Token: None 2022-09-03T06:50:12.540 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj111 2022-09-03T06:50:12.540 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R4yAOY89IUNUE3UJb4F8HPAqSJA= 2022-09-03T06:50:12.541 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R4yAOY89IUNUE3UJb4F8HPAqSJA=', 'Content-Length': '0'} 2022-09-03T06:50:12.543 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000020e4476dd28a532f-006312f924-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.544 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj111 2022-09-03T06:50:12.545 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:12.546 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:12.547 DEBUG:boto:Method: GET 2022-09-03T06:50:12.548 DEBUG:boto:Path: /dsgvcy-75/obj111 2022-09-03T06:50:12.549 DEBUG:boto:Data: 2022-09-03T06:50:12.549 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.550 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.551 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.551 DEBUG:boto:Params: {} 2022-09-03T06:50:12.552 DEBUG:boto:Token: None 2022-09-03T06:50:12.553 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj111 2022-09-03T06:50:12.553 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vLM/jD3GdA11T8ds64mng9FaljU= 2022-09-03T06:50:12.554 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vLM/jD3GdA11T8ds64mng9FaljU=', 'Content-Length': '0'} 2022-09-03T06:50:12.556 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f98ffa1ad09deeed-006312f924-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.557 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:12.559 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:12.559 DEBUG:boto:Method: GET 2022-09-03T06:50:12.560 DEBUG:boto:Path: /dsgvcy-75/obj111 2022-09-03T06:50:12.561 DEBUG:boto:Data: 2022-09-03T06:50:12.561 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.562 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.563 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.563 DEBUG:boto:Params: {} 2022-09-03T06:50:12.564 DEBUG:boto:Token: None 2022-09-03T06:50:12.564 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj111 2022-09-03T06:50:12.565 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vLM/jD3GdA11T8ds64mng9FaljU= 2022-09-03T06:50:12.566 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vLM/jD3GdA11T8ds64mng9FaljU=', 'Content-Length': '0'} 2022-09-03T06:50:12.568 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007f67d3d9f56d95f1-006312f924-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.569 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:12.569 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:12.570 DEBUG:boto:path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:12.571 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:12.571 DEBUG:boto:Method: GET 2022-09-03T06:50:12.572 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj111 2022-09-03T06:50:12.572 DEBUG:boto:Data: 2022-09-03T06:50:12.573 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.573 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.574 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.574 DEBUG:boto:Params: {} 2022-09-03T06:50:12.575 DEBUG:boto:Token: None 2022-09-03T06:50:12.576 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/ 2022-09-03T06:50:12.576 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4= 2022-09-03T06:50:12.577 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4=', 'Content-Length': '0'} 2022-09-03T06:50:12.580 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000029fcf9179697b5a8-006312f924-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.582 DEBUG:boto:b'dsgvcy-75obj1111000falseobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:12.583 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:12.584 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:12.585 DEBUG:boto:path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:12.585 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:12.586 DEBUG:boto:Method: GET 2022-09-03T06:50:12.586 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj111 2022-09-03T06:50:12.587 DEBUG:boto:Data: 2022-09-03T06:50:12.587 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.588 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.589 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.590 DEBUG:boto:Params: {} 2022-09-03T06:50:12.591 DEBUG:boto:Token: None 2022-09-03T06:50:12.591 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/ 2022-09-03T06:50:12.592 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4= 2022-09-03T06:50:12.593 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4=', 'Content-Length': '0'} 2022-09-03T06:50:12.597 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008cfc421c56c20de5-006312f924-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.598 DEBUG:boto:b'dsgvcy-75obj1111000falseobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:12.599 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:50:12.600 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:12.600 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:12.601 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.602 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.603 DEBUG:boto:Method: GET 2022-09-03T06:50:12.603 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.604 DEBUG:boto:Data: 2022-09-03T06:50:12.604 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.605 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.606 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.606 DEBUG:boto:Params: {} 2022-09-03T06:50:12.607 DEBUG:boto:Token: None 2022-09-03T06:50:12.607 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.608 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S41XhKiLG9qjT2ssADxT6ccE/2Q= 2022-09-03T06:50:12.609 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S41XhKiLG9qjT2ssADxT6ccE/2Q=', 'Content-Length': '0'} 2022-09-03T06:50:12.611 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b7260fe36b40bef-006312f924-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.612 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:12.613 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:12.613 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.614 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.615 DEBUG:boto:Method: GET 2022-09-03T06:50:12.615 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.616 DEBUG:boto:Data: 2022-09-03T06:50:12.616 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.617 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.617 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.618 DEBUG:boto:Params: {} 2022-09-03T06:50:12.619 DEBUG:boto:Token: None 2022-09-03T06:50:12.619 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.620 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S41XhKiLG9qjT2ssADxT6ccE/2Q= 2022-09-03T06:50:12.620 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S41XhKiLG9qjT2ssADxT6ccE/2Q=', 'Content-Length': '0'} 2022-09-03T06:50:12.623 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000febecabec4ea2fa6-006312f924-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.676 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:12.677 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:12.677 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.678 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.679 DEBUG:boto:Method: HEAD 2022-09-03T06:50:12.680 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.680 DEBUG:boto:Data: 2022-09-03T06:50:12.681 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.681 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.682 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.682 DEBUG:boto:Params: {} 2022-09-03T06:50:12.683 DEBUG:boto:Token: None 2022-09-03T06:50:12.684 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.684 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JePk1S+/WIlGTjMT1NpZkpsPREQ= 2022-09-03T06:50:12.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JePk1S+/WIlGTjMT1NpZkpsPREQ=', 'Content-Length': '0'} 2022-09-03T06:50:12.687 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dffd54cee1a3e714-006312f924-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.688 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:12.689 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:12.690 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.691 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.691 DEBUG:boto:Method: HEAD 2022-09-03T06:50:12.692 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.692 DEBUG:boto:Data: 2022-09-03T06:50:12.693 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.693 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.694 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.695 DEBUG:boto:Params: {} 2022-09-03T06:50:12.695 DEBUG:boto:Token: None 2022-09-03T06:50:12.696 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.697 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JePk1S+/WIlGTjMT1NpZkpsPREQ= 2022-09-03T06:50:12.697 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JePk1S+/WIlGTjMT1NpZkpsPREQ=', 'Content-Length': '0'} 2022-09-03T06:50:12.700 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000faf82cee8abedc01-006312f924-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.701 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:50:12.702 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:12.702 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:12.703 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.704 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.704 DEBUG:boto:Method: GET 2022-09-03T06:50:12.705 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.706 DEBUG:boto:Data: 2022-09-03T06:50:12.706 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.707 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.707 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.708 DEBUG:boto:Params: {} 2022-09-03T06:50:12.709 DEBUG:boto:Token: None 2022-09-03T06:50:12.709 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.711 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S41XhKiLG9qjT2ssADxT6ccE/2Q= 2022-09-03T06:50:12.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S41XhKiLG9qjT2ssADxT6ccE/2Q=', 'Content-Length': '0'} 2022-09-03T06:50:12.714 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f373098e3d30af4-006312f924-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.715 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:12.716 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:12.716 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.717 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.718 DEBUG:boto:Method: GET 2022-09-03T06:50:12.718 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.719 DEBUG:boto:Data: 2022-09-03T06:50:12.720 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.720 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.721 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.722 DEBUG:boto:Params: {} 2022-09-03T06:50:12.723 DEBUG:boto:Token: None 2022-09-03T06:50:12.723 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:12.724 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:S41XhKiLG9qjT2ssADxT6ccE/2Q= 2022-09-03T06:50:12.725 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:S41XhKiLG9qjT2ssADxT6ccE/2Q=', 'Content-Length': '0'} 2022-09-03T06:50:12.727 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009359451d462e75b6-006312f924-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.728 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:12.728 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:12.729 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:12.730 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:12.730 DEBUG:boto:Method: GET 2022-09-03T06:50:12.731 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:50:12.732 DEBUG:boto:Data: 2022-09-03T06:50:12.733 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.733 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.734 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.735 DEBUG:boto:Params: {} 2022-09-03T06:50:12.735 DEBUG:boto:Token: None 2022-09-03T06:50:12.736 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/ 2022-09-03T06:50:12.737 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4= 2022-09-03T06:50:12.737 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4=', 'Content-Length': '0'} 2022-09-03T06:50:12.740 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f991c362bb21f5bf-006312f924-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.782 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:12.783 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:12.784 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:12.784 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:12.785 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:12.785 DEBUG:boto:Method: GET 2022-09-03T06:50:12.786 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:50:12.787 DEBUG:boto:Data: 2022-09-03T06:50:12.787 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.788 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.789 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.789 DEBUG:boto:Params: {} 2022-09-03T06:50:12.790 DEBUG:boto:Token: None 2022-09-03T06:50:12.790 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/ 2022-09-03T06:50:12.791 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4= 2022-09-03T06:50:12.792 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4=', 'Content-Length': '0'} 2022-09-03T06:50:12.797 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ef7a4e7c4f882d44-006312f924-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.799 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:12.800 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:12.801 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:12.802 DEBUG:boto:Method: HEAD 2022-09-03T06:50:12.803 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:50:12.804 DEBUG:boto:Data: 2022-09-03T06:50:12.805 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.806 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.807 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.808 DEBUG:boto:Params: {} 2022-09-03T06:50:12.809 DEBUG:boto:Token: None 2022-09-03T06:50:12.810 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj115 2022-09-03T06:50:12.811 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:btHoP670fmtgehkIhBQrVyonRjs= 2022-09-03T06:50:12.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:btHoP670fmtgehkIhBQrVyonRjs=', 'Content-Length': '0'} 2022-09-03T06:50:12.815 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000027267e6703cfea3a-006312f924-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.817 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:12.818 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:12.819 DEBUG:boto:Method: HEAD 2022-09-03T06:50:12.820 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:50:12.821 DEBUG:boto:Data: 2022-09-03T06:50:12.822 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.823 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.823 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.824 DEBUG:boto:Params: {} 2022-09-03T06:50:12.825 DEBUG:boto:Token: None 2022-09-03T06:50:12.826 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj115 2022-09-03T06:50:12.828 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:btHoP670fmtgehkIhBQrVyonRjs= 2022-09-03T06:50:12.829 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:btHoP670fmtgehkIhBQrVyonRjs=', 'Content-Length': '0'} 2022-09-03T06:50:12.831 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000008af2e5527592e20-006312f924-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.832 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:50:12.833 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:12.833 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:12.834 DEBUG:boto:Method: GET 2022-09-03T06:50:12.834 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:50:12.835 DEBUG:boto:Data: 2022-09-03T06:50:12.835 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.836 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.837 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.837 DEBUG:boto:Params: {} 2022-09-03T06:50:12.838 DEBUG:boto:Token: None 2022-09-03T06:50:12.838 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj115 2022-09-03T06:50:12.839 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qJYDH63bLvnNc8YA46K5Jx4j7fI= 2022-09-03T06:50:12.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qJYDH63bLvnNc8YA46K5Jx4j7fI=', 'Content-Length': '0'} 2022-09-03T06:50:12.842 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e865dfa24ee64b57-006312f924-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.843 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:12.843 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:12.844 DEBUG:boto:Method: GET 2022-09-03T06:50:12.844 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:50:12.845 DEBUG:boto:Data: 2022-09-03T06:50:12.845 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.846 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.846 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.847 DEBUG:boto:Params: {} 2022-09-03T06:50:12.847 DEBUG:boto:Token: None 2022-09-03T06:50:12.848 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj115 2022-09-03T06:50:12.849 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qJYDH63bLvnNc8YA46K5Jx4j7fI= 2022-09-03T06:50:12.849 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qJYDH63bLvnNc8YA46K5Jx4j7fI=', 'Content-Length': '0'} 2022-09-03T06:50:12.852 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000760e0114ca9bd068-006312f924-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.853 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:12.854 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:12.855 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:12.856 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:12.857 DEBUG:boto:Method: GET 2022-09-03T06:50:12.858 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:50:12.859 DEBUG:boto:Data: 2022-09-03T06:50:12.859 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.860 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.860 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.861 DEBUG:boto:Params: {} 2022-09-03T06:50:12.862 DEBUG:boto:Token: None 2022-09-03T06:50:12.863 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/ 2022-09-03T06:50:12.864 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4= 2022-09-03T06:50:12.865 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4=', 'Content-Length': '0'} 2022-09-03T06:50:12.871 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000380c1c04e5d7653b-006312f924-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.872 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:12.874 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:12.875 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:12.875 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:12.876 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:12.877 DEBUG:boto:Method: GET 2022-09-03T06:50:12.878 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:50:12.879 DEBUG:boto:Data: 2022-09-03T06:50:12.880 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.881 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.881 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.882 DEBUG:boto:Params: {} 2022-09-03T06:50:12.883 DEBUG:boto:Token: None 2022-09-03T06:50:12.883 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/ 2022-09-03T06:50:12.884 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4= 2022-09-03T06:50:12.884 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:C9h62yZVc2IAAebvRkeAPUGnPR4=', 'Content-Length': '0'} 2022-09-03T06:50:12.889 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009eda8513bd462244-006312f924-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.889 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:12.890 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:50:12.891 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:12.892 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:12.892 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:12.893 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:12.893 DEBUG:boto:Method: GET 2022-09-03T06:50:12.894 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:12.894 DEBUG:boto:Data: 2022-09-03T06:50:12.895 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.895 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.896 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.896 DEBUG:boto:Params: {} 2022-09-03T06:50:12.897 DEBUG:boto:Token: None 2022-09-03T06:50:12.897 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:12.898 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GFymuHyN5ZxUbRZuBfjbEJKai/o= 2022-09-03T06:50:12.899 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GFymuHyN5ZxUbRZuBfjbEJKai/o=', 'Content-Length': '0'} 2022-09-03T06:50:12.901 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d6def005b147e0a0-006312f924-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.902 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:12.903 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:12.904 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:12.905 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:12.906 DEBUG:boto:Method: GET 2022-09-03T06:50:12.907 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:12.907 DEBUG:boto:Data: 2022-09-03T06:50:12.908 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.909 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.910 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.911 DEBUG:boto:Params: {} 2022-09-03T06:50:12.912 DEBUG:boto:Token: None 2022-09-03T06:50:12.912 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:12.913 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GFymuHyN5ZxUbRZuBfjbEJKai/o= 2022-09-03T06:50:12.914 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GFymuHyN5ZxUbRZuBfjbEJKai/o=', 'Content-Length': '0'} 2022-09-03T06:50:12.916 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000016686c3f6994217d-006312f924-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.958 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:12.959 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:12.959 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:12.960 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:12.960 DEBUG:boto:Method: HEAD 2022-09-03T06:50:12.961 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:12.961 DEBUG:boto:Data: 2022-09-03T06:50:12.962 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.962 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.963 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.963 DEBUG:boto:Params: {} 2022-09-03T06:50:12.964 DEBUG:boto:Token: None 2022-09-03T06:50:12.965 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:12.965 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8w6zTQ0Wrm7FxfdZ7/2GHAnZLU4= 2022-09-03T06:50:12.966 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8w6zTQ0Wrm7FxfdZ7/2GHAnZLU4=', 'Content-Length': '0'} 2022-09-03T06:50:12.968 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000ee944522d6406a5-006312f924-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.969 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:12.969 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:12.970 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:12.970 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:12.971 DEBUG:boto:Method: HEAD 2022-09-03T06:50:12.971 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:12.972 DEBUG:boto:Data: 2022-09-03T06:50:12.972 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.973 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.973 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.974 DEBUG:boto:Params: {} 2022-09-03T06:50:12.974 DEBUG:boto:Token: None 2022-09-03T06:50:12.975 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:12.976 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8w6zTQ0Wrm7FxfdZ7/2GHAnZLU4= 2022-09-03T06:50:12.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8w6zTQ0Wrm7FxfdZ7/2GHAnZLU4=', 'Content-Length': '0'} 2022-09-03T06:50:12.979 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b7e4fb3754c0ce5c-006312f924-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.979 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:50:12.980 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:12.981 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:12.981 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:12.982 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:12.982 DEBUG:boto:Method: GET 2022-09-03T06:50:12.983 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:12.983 DEBUG:boto:Data: 2022-09-03T06:50:12.984 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.984 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:12.985 DEBUG:boto:Port: 8001 2022-09-03T06:50:12.985 DEBUG:boto:Params: {} 2022-09-03T06:50:12.986 DEBUG:boto:Token: None 2022-09-03T06:50:12.987 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:12.988 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GFymuHyN5ZxUbRZuBfjbEJKai/o= 2022-09-03T06:50:12.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GFymuHyN5ZxUbRZuBfjbEJKai/o=', 'Content-Length': '0'} 2022-09-03T06:50:12.992 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000041bd3fd4d85f7af3-006312f924-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:12 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:12.993 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:12.994 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:12.994 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:12.995 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:12.995 DEBUG:boto:Method: GET 2022-09-03T06:50:12.996 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:12.996 DEBUG:boto:Data: 2022-09-03T06:50:12.997 DEBUG:boto:Headers: {} 2022-09-03T06:50:12.997 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:12.998 DEBUG:boto:Port: 8000 2022-09-03T06:50:12.998 DEBUG:boto:Params: {} 2022-09-03T06:50:12.999 DEBUG:boto:Token: None 2022-09-03T06:50:13.000 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:12 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:13.000 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GFymuHyN5ZxUbRZuBfjbEJKai/o= 2022-09-03T06:50:13.001 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:12 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GFymuHyN5ZxUbRZuBfjbEJKai/o=', 'Content-Length': '0'} 2022-09-03T06:50:13.003 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cdc7d841bf716aff-006312f925-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.004 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:13.005 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:13.005 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:13.006 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:13.007 DEBUG:boto:Method: GET 2022-09-03T06:50:13.007 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:50:13.008 DEBUG:boto:Data: 2022-09-03T06:50:13.008 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.009 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.010 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.010 DEBUG:boto:Params: {} 2022-09-03T06:50:13.011 DEBUG:boto:Token: None 2022-09-03T06:50:13.012 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/ 2022-09-03T06:50:13.012 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA= 2022-09-03T06:50:13.013 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA=', 'Content-Length': '0'} 2022-09-03T06:50:13.016 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e41d153052436bfe-006312f925-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.058 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:13.059 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:13.061 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:13.062 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:13.063 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:13.064 DEBUG:boto:Method: GET 2022-09-03T06:50:13.065 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:50:13.066 DEBUG:boto:Data: 2022-09-03T06:50:13.067 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.068 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:13.069 DEBUG:boto:Port: 8000 2022-09-03T06:50:13.070 DEBUG:boto:Params: {} 2022-09-03T06:50:13.071 DEBUG:boto:Token: None 2022-09-03T06:50:13.072 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/ 2022-09-03T06:50:13.073 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA= 2022-09-03T06:50:13.074 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA=', 'Content-Length': '0'} 2022-09-03T06:50:13.079 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002192e8c6619540f7-006312f925-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.081 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:13.082 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:13.084 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:13.085 DEBUG:boto:Method: HEAD 2022-09-03T06:50:13.085 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:50:13.086 DEBUG:boto:Data: 2022-09-03T06:50:13.087 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.088 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.089 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.090 DEBUG:boto:Params: {} 2022-09-03T06:50:13.090 DEBUG:boto:Token: None 2022-09-03T06:50:13.091 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj116 2022-09-03T06:50:13.092 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:21hNno6Gp8CiJJHmnCWRW4Bp5Jg= 2022-09-03T06:50:13.092 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:21hNno6Gp8CiJJHmnCWRW4Bp5Jg=', 'Content-Length': '0'} 2022-09-03T06:50:13.095 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001756623df33e7982-006312f925-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.096 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:13.096 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:13.097 DEBUG:boto:Method: HEAD 2022-09-03T06:50:13.097 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:50:13.099 DEBUG:boto:Data: 2022-09-03T06:50:13.100 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.101 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:13.102 DEBUG:boto:Port: 8000 2022-09-03T06:50:13.103 DEBUG:boto:Params: {} 2022-09-03T06:50:13.105 DEBUG:boto:Token: None 2022-09-03T06:50:13.105 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj116 2022-09-03T06:50:13.106 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:21hNno6Gp8CiJJHmnCWRW4Bp5Jg= 2022-09-03T06:50:13.107 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:21hNno6Gp8CiJJHmnCWRW4Bp5Jg=', 'Content-Length': '0'} 2022-09-03T06:50:13.109 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad3db0cf2551c7b9-006312f925-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.109 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:50:13.110 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:13.111 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:13.111 DEBUG:boto:Method: GET 2022-09-03T06:50:13.112 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:50:13.113 DEBUG:boto:Data: 2022-09-03T06:50:13.114 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.115 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.116 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.117 DEBUG:boto:Params: {} 2022-09-03T06:50:13.118 DEBUG:boto:Token: None 2022-09-03T06:50:13.120 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj116 2022-09-03T06:50:13.122 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2x3Sth2PXAVFWRRPJuPpFaYMr/c= 2022-09-03T06:50:13.123 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2x3Sth2PXAVFWRRPJuPpFaYMr/c=', 'Content-Length': '0'} 2022-09-03T06:50:13.125 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003f0fa2d2bad7cdf9-006312f925-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.129 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:13.130 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:13.131 DEBUG:boto:Method: GET 2022-09-03T06:50:13.131 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:50:13.132 DEBUG:boto:Data: 2022-09-03T06:50:13.135 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.136 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:13.137 DEBUG:boto:Port: 8000 2022-09-03T06:50:13.139 DEBUG:boto:Params: {} 2022-09-03T06:50:13.140 DEBUG:boto:Token: None 2022-09-03T06:50:13.141 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj116 2022-09-03T06:50:13.142 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2x3Sth2PXAVFWRRPJuPpFaYMr/c= 2022-09-03T06:50:13.143 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2x3Sth2PXAVFWRRPJuPpFaYMr/c=', 'Content-Length': '0'} 2022-09-03T06:50:13.145 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f91d84689d4dd304-006312f925-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.192 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:13.193 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:13.193 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:13.194 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:13.195 DEBUG:boto:Method: GET 2022-09-03T06:50:13.195 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:50:13.196 DEBUG:boto:Data: 2022-09-03T06:50:13.196 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.197 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.197 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.198 DEBUG:boto:Params: {} 2022-09-03T06:50:13.198 DEBUG:boto:Token: None 2022-09-03T06:50:13.199 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/ 2022-09-03T06:50:13.200 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA= 2022-09-03T06:50:13.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA=', 'Content-Length': '0'} 2022-09-03T06:50:13.203 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000057439cddd2d25247-006312f925-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.204 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:13.205 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:13.205 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:13.206 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:13.207 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:13.207 DEBUG:boto:Method: GET 2022-09-03T06:50:13.208 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:50:13.208 DEBUG:boto:Data: 2022-09-03T06:50:13.209 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.209 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:13.210 DEBUG:boto:Port: 8000 2022-09-03T06:50:13.210 DEBUG:boto:Params: {} 2022-09-03T06:50:13.211 DEBUG:boto:Token: None 2022-09-03T06:50:13.212 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/ 2022-09-03T06:50:13.212 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA= 2022-09-03T06:50:13.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA=', 'Content-Length': '0'} 2022-09-03T06:50:13.218 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007e3a09cb2fcbc10b-006312f925-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.219 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:13.220 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2022-09-03T06:50:13.220 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:13.221 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:13.222 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.222 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.223 DEBUG:boto:Method: GET 2022-09-03T06:50:13.224 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.225 DEBUG:boto:Data: 2022-09-03T06:50:13.226 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.227 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.228 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.229 DEBUG:boto:Params: {} 2022-09-03T06:50:13.230 DEBUG:boto:Token: None 2022-09-03T06:50:13.232 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.233 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nAYjPTDGsc8P9CN7ldUtJ4TbJRQ= 2022-09-03T06:50:13.234 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nAYjPTDGsc8P9CN7ldUtJ4TbJRQ=', 'Content-Length': '0'} 2022-09-03T06:50:13.238 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e70f692c2d49a9dc-006312f925-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.278 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:13.279 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:13.280 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.281 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.282 DEBUG:boto:Method: GET 2022-09-03T06:50:13.282 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.283 DEBUG:boto:Data: 2022-09-03T06:50:13.283 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.284 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:13.285 DEBUG:boto:Port: 8000 2022-09-03T06:50:13.285 DEBUG:boto:Params: {} 2022-09-03T06:50:13.286 DEBUG:boto:Token: None 2022-09-03T06:50:13.287 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.287 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nAYjPTDGsc8P9CN7ldUtJ4TbJRQ= 2022-09-03T06:50:13.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nAYjPTDGsc8P9CN7ldUtJ4TbJRQ=', 'Content-Length': '0'} 2022-09-03T06:50:13.290 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000887f4ef01a33f8ed-006312f925-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.292 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:13.292 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:13.293 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.293 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.294 DEBUG:boto:Method: HEAD 2022-09-03T06:50:13.294 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.295 DEBUG:boto:Data: 2022-09-03T06:50:13.295 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.296 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.296 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.297 DEBUG:boto:Params: {} 2022-09-03T06:50:13.297 DEBUG:boto:Token: None 2022-09-03T06:50:13.298 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.299 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gs4YF+ycYzBkeHU3s1ter42Bhoo= 2022-09-03T06:50:13.299 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gs4YF+ycYzBkeHU3s1ter42Bhoo=', 'Content-Length': '0'} 2022-09-03T06:50:13.301 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000033c7af77a6839773-006312f925-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.302 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:13.303 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:13.303 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.304 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.305 DEBUG:boto:Method: HEAD 2022-09-03T06:50:13.305 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.306 DEBUG:boto:Data: 2022-09-03T06:50:13.306 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.307 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:13.307 DEBUG:boto:Port: 8000 2022-09-03T06:50:13.308 DEBUG:boto:Params: {} 2022-09-03T06:50:13.308 DEBUG:boto:Token: None 2022-09-03T06:50:13.309 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.310 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:gs4YF+ycYzBkeHU3s1ter42Bhoo= 2022-09-03T06:50:13.310 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:gs4YF+ycYzBkeHU3s1ter42Bhoo=', 'Content-Length': '0'} 2022-09-03T06:50:13.313 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b40b981d7a33d6e5-006312f925-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.313 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2022-09-03T06:50:13.314 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:13.315 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:13.316 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.317 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.318 DEBUG:boto:Method: GET 2022-09-03T06:50:13.319 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.319 DEBUG:boto:Data: 2022-09-03T06:50:13.321 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.322 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.323 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.324 DEBUG:boto:Params: {} 2022-09-03T06:50:13.325 DEBUG:boto:Token: None 2022-09-03T06:50:13.326 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.326 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nAYjPTDGsc8P9CN7ldUtJ4TbJRQ= 2022-09-03T06:50:13.327 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nAYjPTDGsc8P9CN7ldUtJ4TbJRQ=', 'Content-Length': '0'} 2022-09-03T06:50:13.329 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f62ed0eb1e89bc6-006312f925-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.330 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:13.331 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:13.331 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.332 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.332 DEBUG:boto:Method: GET 2022-09-03T06:50:13.333 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.333 DEBUG:boto:Data: 2022-09-03T06:50:13.334 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.334 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:13.335 DEBUG:boto:Port: 8000 2022-09-03T06:50:13.336 DEBUG:boto:Params: {} 2022-09-03T06:50:13.336 DEBUG:boto:Token: None 2022-09-03T06:50:13.337 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:13.338 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nAYjPTDGsc8P9CN7ldUtJ4TbJRQ= 2022-09-03T06:50:13.338 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nAYjPTDGsc8P9CN7ldUtJ4TbJRQ=', 'Content-Length': '0'} 2022-09-03T06:50:13.340 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000033fba9534e656541-006312f925-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.341 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:13.342 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:13.343 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:13.343 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:13.344 DEBUG:boto:Method: GET 2022-09-03T06:50:13.344 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:50:13.345 DEBUG:boto:Data: 2022-09-03T06:50:13.345 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.346 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.347 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.348 DEBUG:boto:Params: {} 2022-09-03T06:50:13.349 DEBUG:boto:Token: None 2022-09-03T06:50:13.350 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/ 2022-09-03T06:50:13.350 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA= 2022-09-03T06:50:13.351 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA=', 'Content-Length': '0'} 2022-09-03T06:50:13.354 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008f54c6404ff6aeb5-006312f925-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.355 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:13.356 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:13.356 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:13.357 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:13.357 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:13.358 DEBUG:boto:Method: GET 2022-09-03T06:50:13.358 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:50:13.359 DEBUG:boto:Data: 2022-09-03T06:50:13.359 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.360 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:13.360 DEBUG:boto:Port: 8000 2022-09-03T06:50:13.361 DEBUG:boto:Params: {} 2022-09-03T06:50:13.362 DEBUG:boto:Token: None 2022-09-03T06:50:13.362 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/ 2022-09-03T06:50:13.363 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA= 2022-09-03T06:50:13.364 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA=', 'Content-Length': '0'} 2022-09-03T06:50:13.368 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000572b7f18f49de83c-006312f925-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.369 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:13.370 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:13.371 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:13.372 DEBUG:boto:Method: HEAD 2022-09-03T06:50:13.372 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:50:13.373 DEBUG:boto:Data: 2022-09-03T06:50:13.373 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.374 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.374 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.375 DEBUG:boto:Params: {} 2022-09-03T06:50:13.375 DEBUG:boto:Token: None 2022-09-03T06:50:13.376 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj124 2022-09-03T06:50:13.376 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s648VPenF22T0EcVl++WJinXUO0= 2022-09-03T06:50:13.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s648VPenF22T0EcVl++WJinXUO0=', 'Content-Length': '0'} 2022-09-03T06:50:13.379 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b1296adb67cb0d8-006312f925-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.380 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:13.380 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:13.381 DEBUG:boto:Method: HEAD 2022-09-03T06:50:13.381 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:50:13.382 DEBUG:boto:Data: 2022-09-03T06:50:13.382 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.383 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:13.383 DEBUG:boto:Port: 8000 2022-09-03T06:50:13.384 DEBUG:boto:Params: {} 2022-09-03T06:50:13.385 DEBUG:boto:Token: None 2022-09-03T06:50:13.385 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj124 2022-09-03T06:50:13.386 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:s648VPenF22T0EcVl++WJinXUO0= 2022-09-03T06:50:13.386 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:s648VPenF22T0EcVl++WJinXUO0=', 'Content-Length': '0'} 2022-09-03T06:50:13.388 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f263ce1ba3557136-006312f925-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.389 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2022-09-03T06:50:13.390 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:13.390 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:13.391 DEBUG:boto:Method: GET 2022-09-03T06:50:13.392 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:50:13.392 DEBUG:boto:Data: 2022-09-03T06:50:13.393 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.393 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.394 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.395 DEBUG:boto:Params: {} 2022-09-03T06:50:13.395 DEBUG:boto:Token: None 2022-09-03T06:50:13.396 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj124 2022-09-03T06:50:13.396 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Xm4iYhq6LNO8jk7ktPyWSz1FNc4= 2022-09-03T06:50:13.397 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Xm4iYhq6LNO8jk7ktPyWSz1FNc4=', 'Content-Length': '0'} 2022-09-03T06:50:13.399 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000984bdf21591f0bf1-006312f925-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.400 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:13.401 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:13.401 DEBUG:boto:Method: GET 2022-09-03T06:50:13.402 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:50:13.402 DEBUG:boto:Data: 2022-09-03T06:50:13.403 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.403 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:13.404 DEBUG:boto:Port: 8000 2022-09-03T06:50:13.404 DEBUG:boto:Params: {} 2022-09-03T06:50:13.405 DEBUG:boto:Token: None 2022-09-03T06:50:13.406 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj124 2022-09-03T06:50:13.406 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Xm4iYhq6LNO8jk7ktPyWSz1FNc4= 2022-09-03T06:50:13.407 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Xm4iYhq6LNO8jk7ktPyWSz1FNc4=', 'Content-Length': '0'} 2022-09-03T06:50:13.409 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000140f3a9c97655cf6-006312f925-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.454 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:13.455 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:13.456 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:13.456 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:13.457 DEBUG:boto:Method: GET 2022-09-03T06:50:13.457 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:50:13.458 DEBUG:boto:Data: 2022-09-03T06:50:13.458 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.459 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.459 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.460 DEBUG:boto:Params: {} 2022-09-03T06:50:13.460 DEBUG:boto:Token: None 2022-09-03T06:50:13.461 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/ 2022-09-03T06:50:13.462 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA= 2022-09-03T06:50:13.463 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA=', 'Content-Length': '0'} 2022-09-03T06:50:13.466 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009e5c192fe78545da-006312f925-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.467 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:13.468 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:13.468 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:13.469 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:13.469 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:13.470 DEBUG:boto:Method: GET 2022-09-03T06:50:13.470 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:50:13.471 DEBUG:boto:Data: 2022-09-03T06:50:13.471 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.472 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:13.472 DEBUG:boto:Port: 8000 2022-09-03T06:50:13.473 DEBUG:boto:Params: {} 2022-09-03T06:50:13.474 DEBUG:boto:Token: None 2022-09-03T06:50:13.474 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/ 2022-09-03T06:50:13.475 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA= 2022-09-03T06:50:13.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA=', 'Content-Length': '0'} 2022-09-03T06:50:13.480 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004ae214cb55ff78bb-006312f925-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.481 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:13.482 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:50:13.484 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:13.484 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:13.485 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.486 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.486 DEBUG:boto:Method: GET 2022-09-03T06:50:13.487 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.487 DEBUG:boto:Data: 2022-09-03T06:50:13.488 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.488 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.489 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.489 DEBUG:boto:Params: {} 2022-09-03T06:50:13.490 DEBUG:boto:Token: None 2022-09-03T06:50:13.491 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.491 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yUy4MjRJqFs2kBHlxVP2jfwqVPY= 2022-09-03T06:50:13.492 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yUy4MjRJqFs2kBHlxVP2jfwqVPY=', 'Content-Length': '0'} 2022-09-03T06:50:13.494 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002226d6853f00b7c5-006312f925-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.538 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:13.539 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:13.540 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.540 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.541 DEBUG:boto:Method: GET 2022-09-03T06:50:13.541 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.542 DEBUG:boto:Data: 2022-09-03T06:50:13.542 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.543 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:13.544 DEBUG:boto:Port: 8000 2022-09-03T06:50:13.544 DEBUG:boto:Params: {} 2022-09-03T06:50:13.545 DEBUG:boto:Token: None 2022-09-03T06:50:13.545 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.546 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yUy4MjRJqFs2kBHlxVP2jfwqVPY= 2022-09-03T06:50:13.547 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yUy4MjRJqFs2kBHlxVP2jfwqVPY=', 'Content-Length': '0'} 2022-09-03T06:50:13.550 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000008dafff120469965-006312f925-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.552 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:13.552 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:13.553 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.553 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.554 DEBUG:boto:Method: HEAD 2022-09-03T06:50:13.555 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.555 DEBUG:boto:Data: 2022-09-03T06:50:13.556 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.557 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.558 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.559 DEBUG:boto:Params: {} 2022-09-03T06:50:13.559 DEBUG:boto:Token: None 2022-09-03T06:50:13.560 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.561 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mxo1H4Z8R1M2f2lXHvVud2NFrKE= 2022-09-03T06:50:13.562 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mxo1H4Z8R1M2f2lXHvVud2NFrKE=', 'Content-Length': '0'} 2022-09-03T06:50:13.564 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008887428d2e29bb57-006312f925-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.566 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:13.567 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:13.568 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.569 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.570 DEBUG:boto:Method: HEAD 2022-09-03T06:50:13.571 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.577 DEBUG:boto:Data: 2022-09-03T06:50:13.578 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.579 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:13.580 DEBUG:boto:Port: 8000 2022-09-03T06:50:13.581 DEBUG:boto:Params: {} 2022-09-03T06:50:13.582 DEBUG:boto:Token: None 2022-09-03T06:50:13.583 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.584 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mxo1H4Z8R1M2f2lXHvVud2NFrKE= 2022-09-03T06:50:13.585 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mxo1H4Z8R1M2f2lXHvVud2NFrKE=', 'Content-Length': '0'} 2022-09-03T06:50:13.589 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008e87ad98a10f9552-006312f925-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.590 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:50:13.591 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:13.591 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:13.592 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.592 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.593 DEBUG:boto:Method: GET 2022-09-03T06:50:13.594 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.595 DEBUG:boto:Data: 2022-09-03T06:50:13.595 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.596 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.596 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.597 DEBUG:boto:Params: {} 2022-09-03T06:50:13.598 DEBUG:boto:Token: None 2022-09-03T06:50:13.598 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.599 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yUy4MjRJqFs2kBHlxVP2jfwqVPY= 2022-09-03T06:50:13.600 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yUy4MjRJqFs2kBHlxVP2jfwqVPY=', 'Content-Length': '0'} 2022-09-03T06:50:13.603 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004caa84e5eb4d71b8-006312f925-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.604 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:13.605 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:13.606 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.607 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.607 DEBUG:boto:Method: GET 2022-09-03T06:50:13.608 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.608 DEBUG:boto:Data: 2022-09-03T06:50:13.609 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.610 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:13.611 DEBUG:boto:Port: 8000 2022-09-03T06:50:13.612 DEBUG:boto:Params: {} 2022-09-03T06:50:13.613 DEBUG:boto:Token: None 2022-09-03T06:50:13.614 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:13.615 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yUy4MjRJqFs2kBHlxVP2jfwqVPY= 2022-09-03T06:50:13.616 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yUy4MjRJqFs2kBHlxVP2jfwqVPY=', 'Content-Length': '0'} 2022-09-03T06:50:13.619 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000010cd00a942994061-006312f925-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.621 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:13.623 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:13.623 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:13.626 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:13.627 DEBUG:boto:Method: GET 2022-09-03T06:50:13.628 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:50:13.628 DEBUG:boto:Data: 2022-09-03T06:50:13.629 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.630 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.631 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.632 DEBUG:boto:Params: {} 2022-09-03T06:50:13.633 DEBUG:boto:Token: None 2022-09-03T06:50:13.634 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/ 2022-09-03T06:50:13.635 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA= 2022-09-03T06:50:13.636 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA=', 'Content-Length': '0'} 2022-09-03T06:50:13.639 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a3fdf87f5d95871c-006312f925-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.641 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:13.642 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:13.643 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:13.644 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:13.644 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:13.645 DEBUG:boto:Method: GET 2022-09-03T06:50:13.646 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:50:13.647 DEBUG:boto:Data: 2022-09-03T06:50:13.647 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.648 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:13.649 DEBUG:boto:Port: 8000 2022-09-03T06:50:13.650 DEBUG:boto:Params: {} 2022-09-03T06:50:13.650 DEBUG:boto:Token: None 2022-09-03T06:50:13.651 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/ 2022-09-03T06:50:13.652 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA= 2022-09-03T06:50:13.652 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA=', 'Content-Length': '0'} 2022-09-03T06:50:13.658 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007b870fbf521b52ce-006312f925-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.702 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:13.703 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:13.704 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:13.704 DEBUG:boto:Method: HEAD 2022-09-03T06:50:13.705 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:50:13.705 DEBUG:boto:Data: 2022-09-03T06:50:13.706 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.706 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.707 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.708 DEBUG:boto:Params: {} 2022-09-03T06:50:13.708 DEBUG:boto:Token: None 2022-09-03T06:50:13.709 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj145 2022-09-03T06:50:13.710 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FevZyRHPc1/irNkOVksEZm4PdKQ= 2022-09-03T06:50:13.711 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FevZyRHPc1/irNkOVksEZm4PdKQ=', 'Content-Length': '0'} 2022-09-03T06:50:13.713 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000058382a313dd4e7e5-006312f925-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.714 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:13.715 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:13.715 DEBUG:boto:Method: HEAD 2022-09-03T06:50:13.716 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:50:13.716 DEBUG:boto:Data: 2022-09-03T06:50:13.717 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.717 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:13.718 DEBUG:boto:Port: 8000 2022-09-03T06:50:13.718 DEBUG:boto:Params: {} 2022-09-03T06:50:13.719 DEBUG:boto:Token: None 2022-09-03T06:50:13.720 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj145 2022-09-03T06:50:13.720 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FevZyRHPc1/irNkOVksEZm4PdKQ= 2022-09-03T06:50:13.721 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FevZyRHPc1/irNkOVksEZm4PdKQ=', 'Content-Length': '0'} 2022-09-03T06:50:13.723 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000011caea06fa698eec-006312f925-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.724 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:50:13.724 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:13.725 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:13.725 DEBUG:boto:Method: GET 2022-09-03T06:50:13.726 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:50:13.727 DEBUG:boto:Data: 2022-09-03T06:50:13.727 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.728 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.728 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.729 DEBUG:boto:Params: {} 2022-09-03T06:50:13.729 DEBUG:boto:Token: None 2022-09-03T06:50:13.730 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj145 2022-09-03T06:50:13.731 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EUg8Ii/vpngTGQ/NT4wBykXxlHs= 2022-09-03T06:50:13.731 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EUg8Ii/vpngTGQ/NT4wBykXxlHs=', 'Content-Length': '0'} 2022-09-03T06:50:13.734 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026c18585bb449423-006312f925-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.734 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:13.735 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:13.736 DEBUG:boto:Method: GET 2022-09-03T06:50:13.736 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:50:13.737 DEBUG:boto:Data: 2022-09-03T06:50:13.737 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.738 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:13.739 DEBUG:boto:Port: 8000 2022-09-03T06:50:13.739 DEBUG:boto:Params: {} 2022-09-03T06:50:13.740 DEBUG:boto:Token: None 2022-09-03T06:50:13.741 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj145 2022-09-03T06:50:13.741 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EUg8Ii/vpngTGQ/NT4wBykXxlHs= 2022-09-03T06:50:13.742 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EUg8Ii/vpngTGQ/NT4wBykXxlHs=', 'Content-Length': '0'} 2022-09-03T06:50:13.744 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000877932592dce6f3a-006312f925-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.745 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:13.746 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:13.747 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:13.747 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:13.748 DEBUG:boto:Method: GET 2022-09-03T06:50:13.748 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:50:13.749 DEBUG:boto:Data: 2022-09-03T06:50:13.750 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.750 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.751 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.751 DEBUG:boto:Params: {} 2022-09-03T06:50:13.752 DEBUG:boto:Token: None 2022-09-03T06:50:13.753 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/ 2022-09-03T06:50:13.754 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA= 2022-09-03T06:50:13.754 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA=', 'Content-Length': '0'} 2022-09-03T06:50:13.757 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009bda474b3f96a9bb-006312f925-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.758 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:13.759 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:13.759 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:13.760 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:13.760 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:13.761 DEBUG:boto:Method: GET 2022-09-03T06:50:13.762 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:50:13.762 DEBUG:boto:Data: 2022-09-03T06:50:13.763 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.764 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:13.764 DEBUG:boto:Port: 8000 2022-09-03T06:50:13.765 DEBUG:boto:Params: {} 2022-09-03T06:50:13.765 DEBUG:boto:Token: None 2022-09-03T06:50:13.766 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/ 2022-09-03T06:50:13.767 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA= 2022-09-03T06:50:13.767 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA=', 'Content-Length': '0'} 2022-09-03T06:50:13.772 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000317425153a8d6ebd-006312f925-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.773 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:13.774 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj148 2022-09-03T06:50:13.774 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:13.775 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:13.775 DEBUG:boto:path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.776 DEBUG:boto:auth_path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.777 DEBUG:boto:Method: GET 2022-09-03T06:50:13.777 DEBUG:boto:Path: /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.778 DEBUG:boto:Data: 2022-09-03T06:50:13.778 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.779 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.779 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.780 DEBUG:boto:Params: {} 2022-09-03T06:50:13.780 DEBUG:boto:Token: None 2022-09-03T06:50:13.781 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.782 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ECksNizH/CEA0DY0yOQaj2WwdBI= 2022-09-03T06:50:13.782 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ECksNizH/CEA0DY0yOQaj2WwdBI=', 'Content-Length': '0'} 2022-09-03T06:50:13.785 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048fd6febc9892e09-006312f925-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.786 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:13.787 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:13.787 DEBUG:boto:path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.788 DEBUG:boto:auth_path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.788 DEBUG:boto:Method: GET 2022-09-03T06:50:13.789 DEBUG:boto:Path: /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.789 DEBUG:boto:Data: 2022-09-03T06:50:13.790 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.790 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:13.791 DEBUG:boto:Port: 8000 2022-09-03T06:50:13.791 DEBUG:boto:Params: {} 2022-09-03T06:50:13.792 DEBUG:boto:Token: None 2022-09-03T06:50:13.793 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.793 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ECksNizH/CEA0DY0yOQaj2WwdBI= 2022-09-03T06:50:13.794 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ECksNizH/CEA0DY0yOQaj2WwdBI=', 'Content-Length': '0'} 2022-09-03T06:50:13.796 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b730f30d008d4f8-006312f925-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.842 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:13.843 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:13.843 DEBUG:boto:path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.844 DEBUG:boto:auth_path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.844 DEBUG:boto:Method: HEAD 2022-09-03T06:50:13.845 DEBUG:boto:Path: /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.845 DEBUG:boto:Data: 2022-09-03T06:50:13.846 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.848 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.848 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.849 DEBUG:boto:Params: {} 2022-09-03T06:50:13.850 DEBUG:boto:Token: None 2022-09-03T06:50:13.850 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.851 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1u+m6mQWfZQwsgxCcfIF4/mnAO4= 2022-09-03T06:50:13.852 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1u+m6mQWfZQwsgxCcfIF4/mnAO4=', 'Content-Length': '0'} 2022-09-03T06:50:13.854 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008cc68d9acbdd5a19-006312f925-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.855 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:13.856 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:13.856 DEBUG:boto:path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.857 DEBUG:boto:auth_path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.857 DEBUG:boto:Method: HEAD 2022-09-03T06:50:13.858 DEBUG:boto:Path: /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.858 DEBUG:boto:Data: 2022-09-03T06:50:13.859 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.859 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:13.860 DEBUG:boto:Port: 8000 2022-09-03T06:50:13.860 DEBUG:boto:Params: {} 2022-09-03T06:50:13.861 DEBUG:boto:Token: None 2022-09-03T06:50:13.861 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.862 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1u+m6mQWfZQwsgxCcfIF4/mnAO4= 2022-09-03T06:50:13.862 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1u+m6mQWfZQwsgxCcfIF4/mnAO4=', 'Content-Length': '0'} 2022-09-03T06:50:13.865 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b1700dc9a17b452-006312f925-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.866 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj148 2022-09-03T06:50:13.867 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:13.867 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:13.868 DEBUG:boto:path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.868 DEBUG:boto:auth_path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.869 DEBUG:boto:Method: GET 2022-09-03T06:50:13.870 DEBUG:boto:Path: /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.870 DEBUG:boto:Data: 2022-09-03T06:50:13.871 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.872 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.872 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.873 DEBUG:boto:Params: {} 2022-09-03T06:50:13.873 DEBUG:boto:Token: None 2022-09-03T06:50:13.874 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.875 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ECksNizH/CEA0DY0yOQaj2WwdBI= 2022-09-03T06:50:13.876 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ECksNizH/CEA0DY0yOQaj2WwdBI=', 'Content-Length': '0'} 2022-09-03T06:50:13.878 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad2fb9d27be52d2e-006312f925-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.918 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:13.919 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:13.920 DEBUG:boto:path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.920 DEBUG:boto:auth_path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.921 DEBUG:boto:Method: GET 2022-09-03T06:50:13.922 DEBUG:boto:Path: /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.922 DEBUG:boto:Data: 2022-09-03T06:50:13.923 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.923 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:13.924 DEBUG:boto:Port: 8000 2022-09-03T06:50:13.925 DEBUG:boto:Params: {} 2022-09-03T06:50:13.925 DEBUG:boto:Token: None 2022-09-03T06:50:13.926 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:13.927 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ECksNizH/CEA0DY0yOQaj2WwdBI= 2022-09-03T06:50:13.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ECksNizH/CEA0DY0yOQaj2WwdBI=', 'Content-Length': '0'} 2022-09-03T06:50:13.930 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021209656405a1c22-006312f925-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.931 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:13.931 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:13.932 DEBUG:boto:path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:13.933 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:13.933 DEBUG:boto:Method: GET 2022-09-03T06:50:13.934 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj148 2022-09-03T06:50:13.935 DEBUG:boto:Data: 2022-09-03T06:50:13.935 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.936 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.936 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.937 DEBUG:boto:Params: {} 2022-09-03T06:50:13.937 DEBUG:boto:Token: None 2022-09-03T06:50:13.938 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/ 2022-09-03T06:50:13.939 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA= 2022-09-03T06:50:13.940 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA=', 'Content-Length': '0'} 2022-09-03T06:50:13.942 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000af0393e16bb7e464-006312f925-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.943 DEBUG:boto:b'dsgvcy-75obj1481000falseobj1482022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:13.945 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:13.946 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:13.946 DEBUG:boto:path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:13.947 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:13.948 DEBUG:boto:Method: GET 2022-09-03T06:50:13.949 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj148 2022-09-03T06:50:13.950 DEBUG:boto:Data: 2022-09-03T06:50:13.951 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.952 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:13.952 DEBUG:boto:Port: 8000 2022-09-03T06:50:13.953 DEBUG:boto:Params: {} 2022-09-03T06:50:13.954 DEBUG:boto:Token: None 2022-09-03T06:50:13.955 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/ 2022-09-03T06:50:13.956 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA= 2022-09-03T06:50:13.957 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FXQtBOACw9og8iAbI0VwsKM8YwA=', 'Content-Length': '0'} 2022-09-03T06:50:13.961 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cfa0d15d4fa2c2fd-006312f925-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.962 DEBUG:boto:b'dsgvcy-75obj1481000falseobj1482022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:13.964 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:13.964 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:13.965 DEBUG:boto:Method: HEAD 2022-09-03T06:50:13.965 DEBUG:boto:Path: /dsgvcy-75/obj148 2022-09-03T06:50:13.966 DEBUG:boto:Data: 2022-09-03T06:50:13.967 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.967 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.968 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.968 DEBUG:boto:Params: {} 2022-09-03T06:50:13.969 DEBUG:boto:Token: None 2022-09-03T06:50:13.970 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj148 2022-09-03T06:50:13.970 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2lisXdjmfYOGflyGbYKmuHFb3dM= 2022-09-03T06:50:13.971 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2lisXdjmfYOGflyGbYKmuHFb3dM=', 'Content-Length': '0'} 2022-09-03T06:50:13.973 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d304b82ce99facf2-006312f925-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.974 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:13.975 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:13.975 DEBUG:boto:Method: HEAD 2022-09-03T06:50:13.976 DEBUG:boto:Path: /dsgvcy-75/obj148 2022-09-03T06:50:13.976 DEBUG:boto:Data: 2022-09-03T06:50:13.977 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.977 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:13.978 DEBUG:boto:Port: 8000 2022-09-03T06:50:13.979 DEBUG:boto:Params: {} 2022-09-03T06:50:13.979 DEBUG:boto:Token: None 2022-09-03T06:50:13.980 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj148 2022-09-03T06:50:13.980 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2lisXdjmfYOGflyGbYKmuHFb3dM= 2022-09-03T06:50:13.981 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2lisXdjmfYOGflyGbYKmuHFb3dM=', 'Content-Length': '0'} 2022-09-03T06:50:13.983 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000044e1ba0eeb7680f0-006312f925-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:13.984 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj148 2022-09-03T06:50:13.985 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:13.985 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:13.986 DEBUG:boto:Method: GET 2022-09-03T06:50:13.987 DEBUG:boto:Path: /dsgvcy-75/obj148 2022-09-03T06:50:13.987 DEBUG:boto:Data: 2022-09-03T06:50:13.988 DEBUG:boto:Headers: {} 2022-09-03T06:50:13.989 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:13.989 DEBUG:boto:Port: 8001 2022-09-03T06:50:13.991 DEBUG:boto:Params: {} 2022-09-03T06:50:13.993 DEBUG:boto:Token: None 2022-09-03T06:50:13.994 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:13 GMT /dsgvcy-75/obj148 2022-09-03T06:50:13.995 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OCiEkcT5+WK0U/H11YZ3CmBo9WQ= 2022-09-03T06:50:13.996 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:13 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OCiEkcT5+WK0U/H11YZ3CmBo9WQ=', 'Content-Length': '0'} 2022-09-03T06:50:14.001 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001753ec21b0180168-006312f925-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:13 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.002 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:14.004 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:14.005 DEBUG:boto:Method: GET 2022-09-03T06:50:14.006 DEBUG:boto:Path: /dsgvcy-75/obj148 2022-09-03T06:50:14.007 DEBUG:boto:Data: 2022-09-03T06:50:14.008 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.009 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:14.010 DEBUG:boto:Port: 8000 2022-09-03T06:50:14.011 DEBUG:boto:Params: {} 2022-09-03T06:50:14.012 DEBUG:boto:Token: None 2022-09-03T06:50:14.013 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj148 2022-09-03T06:50:14.014 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ecHDI64/mNMUm8WIPyRtdj4fKC4= 2022-09-03T06:50:14.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ecHDI64/mNMUm8WIPyRtdj4fKC4=', 'Content-Length': '0'} 2022-09-03T06:50:14.018 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000093a0b62869adc42d-006312f926-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.062 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:14.063 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:14.064 DEBUG:boto:path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:14.064 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:14.065 DEBUG:boto:Method: GET 2022-09-03T06:50:14.065 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj148 2022-09-03T06:50:14.066 DEBUG:boto:Data: 2022-09-03T06:50:14.066 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.067 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:14.068 DEBUG:boto:Port: 8001 2022-09-03T06:50:14.068 DEBUG:boto:Params: {} 2022-09-03T06:50:14.069 DEBUG:boto:Token: None 2022-09-03T06:50:14.070 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/ 2022-09-03T06:50:14.071 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs= 2022-09-03T06:50:14.072 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs=', 'Content-Length': '0'} 2022-09-03T06:50:14.075 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000038f9555f8ec349d7-006312f926-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.077 DEBUG:boto:b'dsgvcy-75obj1481000falseobj1482022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:14.079 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:14.080 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:14.081 DEBUG:boto:path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:14.082 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:14.083 DEBUG:boto:Method: GET 2022-09-03T06:50:14.084 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj148 2022-09-03T06:50:14.085 DEBUG:boto:Data: 2022-09-03T06:50:14.086 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.087 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:14.088 DEBUG:boto:Port: 8000 2022-09-03T06:50:14.089 DEBUG:boto:Params: {} 2022-09-03T06:50:14.090 DEBUG:boto:Token: None 2022-09-03T06:50:14.091 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/ 2022-09-03T06:50:14.092 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs= 2022-09-03T06:50:14.093 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs=', 'Content-Length': '0'} 2022-09-03T06:50:14.099 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003b2adede0a70c628-006312f926-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.100 DEBUG:boto:b'dsgvcy-75obj1481000falseobj1482022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:14.102 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2022-09-03T06:50:14.103 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:14.103 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:14.104 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.105 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.106 DEBUG:boto:Method: GET 2022-09-03T06:50:14.107 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.107 DEBUG:boto:Data: 2022-09-03T06:50:14.108 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.108 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:14.109 DEBUG:boto:Port: 8001 2022-09-03T06:50:14.109 DEBUG:boto:Params: {} 2022-09-03T06:50:14.110 DEBUG:boto:Token: None 2022-09-03T06:50:14.111 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.111 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xg2qCvLGt5ZA+02hTNJeieEreBo= 2022-09-03T06:50:14.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xg2qCvLGt5ZA+02hTNJeieEreBo=', 'Content-Length': '0'} 2022-09-03T06:50:14.114 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004901c60182d7526a-006312f926-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.158 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:14.159 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:14.160 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.160 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.161 DEBUG:boto:Method: GET 2022-09-03T06:50:14.161 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.162 DEBUG:boto:Data: 2022-09-03T06:50:14.163 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.163 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:14.164 DEBUG:boto:Port: 8000 2022-09-03T06:50:14.164 DEBUG:boto:Params: {} 2022-09-03T06:50:14.165 DEBUG:boto:Token: None 2022-09-03T06:50:14.166 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.167 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xg2qCvLGt5ZA+02hTNJeieEreBo= 2022-09-03T06:50:14.168 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xg2qCvLGt5ZA+02hTNJeieEreBo=', 'Content-Length': '0'} 2022-09-03T06:50:14.171 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af47b2255ae57052-006312f926-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.172 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:14.174 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:14.176 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.176 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.187 DEBUG:boto:Method: HEAD 2022-09-03T06:50:14.188 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.189 DEBUG:boto:Data: 2022-09-03T06:50:14.190 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.191 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:14.192 DEBUG:boto:Port: 8001 2022-09-03T06:50:14.192 DEBUG:boto:Params: {} 2022-09-03T06:50:14.193 DEBUG:boto:Token: None 2022-09-03T06:50:14.195 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.196 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VcxbnVeyQKP5oaCu70acs2iro7o= 2022-09-03T06:50:14.196 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VcxbnVeyQKP5oaCu70acs2iro7o=', 'Content-Length': '0'} 2022-09-03T06:50:14.201 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bcdd4c8956254b29-006312f926-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.203 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:14.203 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:14.204 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.204 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.205 DEBUG:boto:Method: HEAD 2022-09-03T06:50:14.205 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.206 DEBUG:boto:Data: 2022-09-03T06:50:14.207 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.207 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:14.210 DEBUG:boto:Port: 8000 2022-09-03T06:50:14.211 DEBUG:boto:Params: {} 2022-09-03T06:50:14.212 DEBUG:boto:Token: None 2022-09-03T06:50:14.214 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.215 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VcxbnVeyQKP5oaCu70acs2iro7o= 2022-09-03T06:50:14.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VcxbnVeyQKP5oaCu70acs2iro7o=', 'Content-Length': '0'} 2022-09-03T06:50:14.219 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000016bed8353bce38f2-006312f926-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.221 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2022-09-03T06:50:14.222 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:14.224 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:14.225 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.226 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.227 DEBUG:boto:Method: GET 2022-09-03T06:50:14.228 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.229 DEBUG:boto:Data: 2022-09-03T06:50:14.230 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.235 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:14.236 DEBUG:boto:Port: 8001 2022-09-03T06:50:14.237 DEBUG:boto:Params: {} 2022-09-03T06:50:14.238 DEBUG:boto:Token: None 2022-09-03T06:50:14.239 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.240 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xg2qCvLGt5ZA+02hTNJeieEreBo= 2022-09-03T06:50:14.241 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xg2qCvLGt5ZA+02hTNJeieEreBo=', 'Content-Length': '0'} 2022-09-03T06:50:14.249 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006834cec9a51695e0-006312f926-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.250 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:14.251 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:14.252 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.253 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.255 DEBUG:boto:Method: GET 2022-09-03T06:50:14.256 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.257 DEBUG:boto:Data: 2022-09-03T06:50:14.258 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.259 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:14.260 DEBUG:boto:Port: 8000 2022-09-03T06:50:14.261 DEBUG:boto:Params: {} 2022-09-03T06:50:14.262 DEBUG:boto:Token: None 2022-09-03T06:50:14.263 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:14.264 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xg2qCvLGt5ZA+02hTNJeieEreBo= 2022-09-03T06:50:14.266 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xg2qCvLGt5ZA+02hTNJeieEreBo=', 'Content-Length': '0'} 2022-09-03T06:50:14.268 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000252a0996c10f726b-006312f926-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.269 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:14.270 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:14.270 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:14.271 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:14.271 DEBUG:boto:Method: GET 2022-09-03T06:50:14.272 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:50:14.272 DEBUG:boto:Data: 2022-09-03T06:50:14.274 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.274 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:14.275 DEBUG:boto:Port: 8001 2022-09-03T06:50:14.276 DEBUG:boto:Params: {} 2022-09-03T06:50:14.276 DEBUG:boto:Token: None 2022-09-03T06:50:14.277 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/ 2022-09-03T06:50:14.278 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs= 2022-09-03T06:50:14.279 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs=', 'Content-Length': '0'} 2022-09-03T06:50:14.284 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000041435ac054ecd8d-006312f926-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.287 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:14.288 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:14.289 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:14.291 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:14.292 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:14.293 DEBUG:boto:Method: GET 2022-09-03T06:50:14.294 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:50:14.295 DEBUG:boto:Data: 2022-09-03T06:50:14.295 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.296 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:14.297 DEBUG:boto:Port: 8000 2022-09-03T06:50:14.299 DEBUG:boto:Params: {} 2022-09-03T06:50:14.300 DEBUG:boto:Token: None 2022-09-03T06:50:14.302 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/ 2022-09-03T06:50:14.303 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs= 2022-09-03T06:50:14.303 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs=', 'Content-Length': '0'} 2022-09-03T06:50:14.308 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d629f6eee1811a40-006312f926-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.309 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:14.310 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:14.311 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:14.311 DEBUG:boto:Method: HEAD 2022-09-03T06:50:14.312 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:50:14.313 DEBUG:boto:Data: 2022-09-03T06:50:14.314 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.319 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:14.320 DEBUG:boto:Port: 8001 2022-09-03T06:50:14.323 DEBUG:boto:Params: {} 2022-09-03T06:50:14.324 DEBUG:boto:Token: None 2022-09-03T06:50:14.325 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj155 2022-09-03T06:50:14.328 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7TN4UiTrrqVn/515oJ6qgPBIm4A= 2022-09-03T06:50:14.333 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7TN4UiTrrqVn/515oJ6qgPBIm4A=', 'Content-Length': '0'} 2022-09-03T06:50:14.336 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000065480cef2573ba37-006312f926-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.337 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:14.337 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:14.338 DEBUG:boto:Method: HEAD 2022-09-03T06:50:14.339 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:50:14.339 DEBUG:boto:Data: 2022-09-03T06:50:14.340 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.340 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:14.341 DEBUG:boto:Port: 8000 2022-09-03T06:50:14.342 DEBUG:boto:Params: {} 2022-09-03T06:50:14.343 DEBUG:boto:Token: None 2022-09-03T06:50:14.344 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj155 2022-09-03T06:50:14.345 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7TN4UiTrrqVn/515oJ6qgPBIm4A= 2022-09-03T06:50:14.346 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7TN4UiTrrqVn/515oJ6qgPBIm4A=', 'Content-Length': '0'} 2022-09-03T06:50:14.349 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ab10ca8d0a880e1-006312f926-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.351 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2022-09-03T06:50:14.352 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:14.353 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:14.354 DEBUG:boto:Method: GET 2022-09-03T06:50:14.355 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:50:14.356 DEBUG:boto:Data: 2022-09-03T06:50:14.357 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.358 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:14.359 DEBUG:boto:Port: 8001 2022-09-03T06:50:14.360 DEBUG:boto:Params: {} 2022-09-03T06:50:14.361 DEBUG:boto:Token: None 2022-09-03T06:50:14.362 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj155 2022-09-03T06:50:14.363 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y/nqZPVzU1abfmhkzOpAXgZGVX8= 2022-09-03T06:50:14.363 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y/nqZPVzU1abfmhkzOpAXgZGVX8=', 'Content-Length': '0'} 2022-09-03T06:50:14.367 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ba43c0a9832f39be-006312f926-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.368 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:14.369 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:14.373 DEBUG:boto:Method: GET 2022-09-03T06:50:14.374 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:50:14.375 DEBUG:boto:Data: 2022-09-03T06:50:14.376 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.377 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:14.378 DEBUG:boto:Port: 8000 2022-09-03T06:50:14.379 DEBUG:boto:Params: {} 2022-09-03T06:50:14.380 DEBUG:boto:Token: None 2022-09-03T06:50:14.381 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj155 2022-09-03T06:50:14.383 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Y/nqZPVzU1abfmhkzOpAXgZGVX8= 2022-09-03T06:50:14.383 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Y/nqZPVzU1abfmhkzOpAXgZGVX8=', 'Content-Length': '0'} 2022-09-03T06:50:14.386 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008726578e2fe54f37-006312f926-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.387 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:14.388 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:14.389 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:14.390 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:14.391 DEBUG:boto:Method: GET 2022-09-03T06:50:14.392 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:50:14.392 DEBUG:boto:Data: 2022-09-03T06:50:14.393 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.393 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:14.394 DEBUG:boto:Port: 8001 2022-09-03T06:50:14.394 DEBUG:boto:Params: {} 2022-09-03T06:50:14.395 DEBUG:boto:Token: None 2022-09-03T06:50:14.396 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/ 2022-09-03T06:50:14.397 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs= 2022-09-03T06:50:14.398 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs=', 'Content-Length': '0'} 2022-09-03T06:50:14.401 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000677accf7c063986d-006312f926-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.448 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:14.450 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:14.451 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:14.452 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:14.453 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:14.454 DEBUG:boto:Method: GET 2022-09-03T06:50:14.456 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:50:14.458 DEBUG:boto:Data: 2022-09-03T06:50:14.459 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.460 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:14.461 DEBUG:boto:Port: 8000 2022-09-03T06:50:14.462 DEBUG:boto:Params: {} 2022-09-03T06:50:14.463 DEBUG:boto:Token: None 2022-09-03T06:50:14.464 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/ 2022-09-03T06:50:14.465 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs= 2022-09-03T06:50:14.467 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs=', 'Content-Length': '0'} 2022-09-03T06:50:14.473 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a0dfda958d9261eb-006312f926-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.474 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:14.475 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:50:14.476 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:14.477 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:14.478 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.479 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.480 DEBUG:boto:Method: GET 2022-09-03T06:50:14.481 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.482 DEBUG:boto:Data: 2022-09-03T06:50:14.483 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.484 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:14.485 DEBUG:boto:Port: 8001 2022-09-03T06:50:14.485 DEBUG:boto:Params: {} 2022-09-03T06:50:14.486 DEBUG:boto:Token: None 2022-09-03T06:50:14.487 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.487 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rtrq5Cm1Oq+nYYo52UYLKH92kk8= 2022-09-03T06:50:14.488 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rtrq5Cm1Oq+nYYo52UYLKH92kk8=', 'Content-Length': '0'} 2022-09-03T06:50:14.491 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fcdb7fcc69dc6c1d-006312f926-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.493 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:14.494 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:14.495 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.497 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.498 DEBUG:boto:Method: GET 2022-09-03T06:50:14.500 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.502 DEBUG:boto:Data: 2022-09-03T06:50:14.503 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.504 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:14.505 DEBUG:boto:Port: 8000 2022-09-03T06:50:14.506 DEBUG:boto:Params: {} 2022-09-03T06:50:14.507 DEBUG:boto:Token: None 2022-09-03T06:50:14.508 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.509 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rtrq5Cm1Oq+nYYo52UYLKH92kk8= 2022-09-03T06:50:14.510 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rtrq5Cm1Oq+nYYo52UYLKH92kk8=', 'Content-Length': '0'} 2022-09-03T06:50:14.523 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000546301e10e960b31-006312f926-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.530 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:14.531 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:14.534 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.535 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.536 DEBUG:boto:Method: HEAD 2022-09-03T06:50:14.540 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.544 DEBUG:boto:Data: 2022-09-03T06:50:14.548 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.550 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:14.553 DEBUG:boto:Port: 8001 2022-09-03T06:50:14.554 DEBUG:boto:Params: {} 2022-09-03T06:50:14.555 DEBUG:boto:Token: None 2022-09-03T06:50:14.557 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.557 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bLood9I8E02LV9rDZyebiMXJ1WY= 2022-09-03T06:50:14.558 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bLood9I8E02LV9rDZyebiMXJ1WY=', 'Content-Length': '0'} 2022-09-03T06:50:14.561 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e9a3eb9deaf7b776-006312f926-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.562 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:14.563 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:14.563 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.564 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.565 DEBUG:boto:Method: HEAD 2022-09-03T06:50:14.565 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.566 DEBUG:boto:Data: 2022-09-03T06:50:14.567 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.567 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:14.568 DEBUG:boto:Port: 8000 2022-09-03T06:50:14.569 DEBUG:boto:Params: {} 2022-09-03T06:50:14.570 DEBUG:boto:Token: None 2022-09-03T06:50:14.571 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.572 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bLood9I8E02LV9rDZyebiMXJ1WY= 2022-09-03T06:50:14.573 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bLood9I8E02LV9rDZyebiMXJ1WY=', 'Content-Length': '0'} 2022-09-03T06:50:14.588 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000929ce8692c4b4b47-006312f926-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.590 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:50:14.601 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:14.602 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:14.603 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.604 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.605 DEBUG:boto:Method: GET 2022-09-03T06:50:14.606 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.608 DEBUG:boto:Data: 2022-09-03T06:50:14.610 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.611 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:14.612 DEBUG:boto:Port: 8001 2022-09-03T06:50:14.613 DEBUG:boto:Params: {} 2022-09-03T06:50:14.614 DEBUG:boto:Token: None 2022-09-03T06:50:14.615 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.616 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rtrq5Cm1Oq+nYYo52UYLKH92kk8= 2022-09-03T06:50:14.617 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rtrq5Cm1Oq+nYYo52UYLKH92kk8=', 'Content-Length': '0'} 2022-09-03T06:50:14.621 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000206670ef4967969d-006312f926-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.629 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:14.631 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:14.632 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.632 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.633 DEBUG:boto:Method: GET 2022-09-03T06:50:14.634 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.635 DEBUG:boto:Data: 2022-09-03T06:50:14.636 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.637 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:14.638 DEBUG:boto:Port: 8000 2022-09-03T06:50:14.640 DEBUG:boto:Params: {} 2022-09-03T06:50:14.640 DEBUG:boto:Token: None 2022-09-03T06:50:14.641 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:14.643 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Rtrq5Cm1Oq+nYYo52UYLKH92kk8= 2022-09-03T06:50:14.644 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Rtrq5Cm1Oq+nYYo52UYLKH92kk8=', 'Content-Length': '0'} 2022-09-03T06:50:14.649 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000008df3a184b415daf-006312f926-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.651 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:14.651 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:14.652 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:14.654 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:14.655 DEBUG:boto:Method: GET 2022-09-03T06:50:14.655 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:50:14.656 DEBUG:boto:Data: 2022-09-03T06:50:14.657 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.657 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:14.658 DEBUG:boto:Port: 8001 2022-09-03T06:50:14.658 DEBUG:boto:Params: {} 2022-09-03T06:50:14.659 DEBUG:boto:Token: None 2022-09-03T06:50:14.661 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/ 2022-09-03T06:50:14.662 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs= 2022-09-03T06:50:14.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs=', 'Content-Length': '0'} 2022-09-03T06:50:14.669 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000089ccdf7c2b9766bb-006312f926-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.671 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:14.672 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:14.673 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:14.674 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:14.675 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:14.676 DEBUG:boto:Method: GET 2022-09-03T06:50:14.677 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:50:14.678 DEBUG:boto:Data: 2022-09-03T06:50:14.679 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.680 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:14.681 DEBUG:boto:Port: 8000 2022-09-03T06:50:14.682 DEBUG:boto:Params: {} 2022-09-03T06:50:14.684 DEBUG:boto:Token: None 2022-09-03T06:50:14.686 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/ 2022-09-03T06:50:14.688 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs= 2022-09-03T06:50:14.690 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs=', 'Content-Length': '0'} 2022-09-03T06:50:14.702 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003a6387b3cf371538-006312f926-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.704 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:14.705 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:14.706 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:14.707 DEBUG:boto:Method: HEAD 2022-09-03T06:50:14.708 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:50:14.709 DEBUG:boto:Data: 2022-09-03T06:50:14.710 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.710 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:14.711 DEBUG:boto:Port: 8001 2022-09-03T06:50:14.711 DEBUG:boto:Params: {} 2022-09-03T06:50:14.712 DEBUG:boto:Token: None 2022-09-03T06:50:14.713 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj17 2022-09-03T06:50:14.713 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3b5TTh1JYCZbiMfFuLZbqoGp8u8= 2022-09-03T06:50:14.714 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3b5TTh1JYCZbiMfFuLZbqoGp8u8=', 'Content-Length': '0'} 2022-09-03T06:50:14.716 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004094ae624037d72a-006312f926-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.717 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:14.718 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:14.718 DEBUG:boto:Method: HEAD 2022-09-03T06:50:14.719 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:50:14.719 DEBUG:boto:Data: 2022-09-03T06:50:14.720 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.721 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:14.721 DEBUG:boto:Port: 8000 2022-09-03T06:50:14.722 DEBUG:boto:Params: {} 2022-09-03T06:50:14.722 DEBUG:boto:Token: None 2022-09-03T06:50:14.723 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj17 2022-09-03T06:50:14.723 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3b5TTh1JYCZbiMfFuLZbqoGp8u8= 2022-09-03T06:50:14.724 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3b5TTh1JYCZbiMfFuLZbqoGp8u8=', 'Content-Length': '0'} 2022-09-03T06:50:14.727 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000099bc3dc9c40aa3a0-006312f926-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.727 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:50:14.728 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:14.729 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:14.729 DEBUG:boto:Method: GET 2022-09-03T06:50:14.730 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:50:14.730 DEBUG:boto:Data: 2022-09-03T06:50:14.731 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.731 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:14.732 DEBUG:boto:Port: 8001 2022-09-03T06:50:14.732 DEBUG:boto:Params: {} 2022-09-03T06:50:14.733 DEBUG:boto:Token: None 2022-09-03T06:50:14.734 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj17 2022-09-03T06:50:14.734 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6Y51gXMjMGYx11vNbnVfgueTZ60= 2022-09-03T06:50:14.735 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6Y51gXMjMGYx11vNbnVfgueTZ60=', 'Content-Length': '0'} 2022-09-03T06:50:14.738 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003fcb0a9a571ee4e4-006312f926-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.739 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:14.739 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:14.740 DEBUG:boto:Method: GET 2022-09-03T06:50:14.740 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:50:14.741 DEBUG:boto:Data: 2022-09-03T06:50:14.741 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.742 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:14.742 DEBUG:boto:Port: 8000 2022-09-03T06:50:14.743 DEBUG:boto:Params: {} 2022-09-03T06:50:14.744 DEBUG:boto:Token: None 2022-09-03T06:50:14.744 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj17 2022-09-03T06:50:14.745 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6Y51gXMjMGYx11vNbnVfgueTZ60= 2022-09-03T06:50:14.746 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6Y51gXMjMGYx11vNbnVfgueTZ60=', 'Content-Length': '0'} 2022-09-03T06:50:14.748 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000c5b37df3ff591c8-006312f926-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.790 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:14.792 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:14.793 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:14.794 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:14.795 DEBUG:boto:Method: GET 2022-09-03T06:50:14.796 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:50:14.797 DEBUG:boto:Data: 2022-09-03T06:50:14.798 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.799 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:14.800 DEBUG:boto:Port: 8001 2022-09-03T06:50:14.800 DEBUG:boto:Params: {} 2022-09-03T06:50:14.801 DEBUG:boto:Token: None 2022-09-03T06:50:14.802 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/ 2022-09-03T06:50:14.803 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs= 2022-09-03T06:50:14.805 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs=', 'Content-Length': '0'} 2022-09-03T06:50:14.808 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e9e7166cdd4a0491-006312f926-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.810 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:14.812 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:14.812 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:14.813 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:14.813 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:14.814 DEBUG:boto:Method: GET 2022-09-03T06:50:14.815 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:50:14.816 DEBUG:boto:Data: 2022-09-03T06:50:14.817 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.818 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:14.819 DEBUG:boto:Port: 8000 2022-09-03T06:50:14.820 DEBUG:boto:Params: {} 2022-09-03T06:50:14.821 DEBUG:boto:Token: None 2022-09-03T06:50:14.823 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/ 2022-09-03T06:50:14.825 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs= 2022-09-03T06:50:14.826 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs=', 'Content-Length': '0'} 2022-09-03T06:50:14.831 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c2587c28751fb365-006312f926-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.832 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:14.834 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj185 2022-09-03T06:50:14.835 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:14.836 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:14.837 DEBUG:boto:path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.837 DEBUG:boto:auth_path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.838 DEBUG:boto:Method: GET 2022-09-03T06:50:14.838 DEBUG:boto:Path: /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.839 DEBUG:boto:Data: 2022-09-03T06:50:14.839 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.840 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:14.841 DEBUG:boto:Port: 8001 2022-09-03T06:50:14.841 DEBUG:boto:Params: {} 2022-09-03T06:50:14.842 DEBUG:boto:Token: None 2022-09-03T06:50:14.842 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.843 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wM5Ia5VqTXh23P/SkPn2JJdhhWg= 2022-09-03T06:50:14.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wM5Ia5VqTXh23P/SkPn2JJdhhWg=', 'Content-Length': '0'} 2022-09-03T06:50:14.846 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001f86c1fa9237e58b-006312f926-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.847 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:14.848 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:14.848 DEBUG:boto:path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.849 DEBUG:boto:auth_path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.850 DEBUG:boto:Method: GET 2022-09-03T06:50:14.851 DEBUG:boto:Path: /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.852 DEBUG:boto:Data: 2022-09-03T06:50:14.853 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.854 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:14.854 DEBUG:boto:Port: 8000 2022-09-03T06:50:14.855 DEBUG:boto:Params: {} 2022-09-03T06:50:14.856 DEBUG:boto:Token: None 2022-09-03T06:50:14.856 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.857 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wM5Ia5VqTXh23P/SkPn2JJdhhWg= 2022-09-03T06:50:14.858 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wM5Ia5VqTXh23P/SkPn2JJdhhWg=', 'Content-Length': '0'} 2022-09-03T06:50:14.860 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af92e9aaee0ccdb6-006312f926-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.861 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:14.862 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:14.862 DEBUG:boto:path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.863 DEBUG:boto:auth_path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.863 DEBUG:boto:Method: HEAD 2022-09-03T06:50:14.864 DEBUG:boto:Path: /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.864 DEBUG:boto:Data: 2022-09-03T06:50:14.865 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.865 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:14.866 DEBUG:boto:Port: 8001 2022-09-03T06:50:14.866 DEBUG:boto:Params: {} 2022-09-03T06:50:14.867 DEBUG:boto:Token: None 2022-09-03T06:50:14.868 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.868 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e4hUI1+oBUjW19hAwEJJaSbUnvQ= 2022-09-03T06:50:14.869 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e4hUI1+oBUjW19hAwEJJaSbUnvQ=', 'Content-Length': '0'} 2022-09-03T06:50:14.871 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000072483ef8811d1acc-006312f926-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.872 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:14.873 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:14.873 DEBUG:boto:path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.874 DEBUG:boto:auth_path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.874 DEBUG:boto:Method: HEAD 2022-09-03T06:50:14.875 DEBUG:boto:Path: /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.875 DEBUG:boto:Data: 2022-09-03T06:50:14.876 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.876 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:14.877 DEBUG:boto:Port: 8000 2022-09-03T06:50:14.877 DEBUG:boto:Params: {} 2022-09-03T06:50:14.878 DEBUG:boto:Token: None 2022-09-03T06:50:14.878 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.879 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e4hUI1+oBUjW19hAwEJJaSbUnvQ= 2022-09-03T06:50:14.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e4hUI1+oBUjW19hAwEJJaSbUnvQ=', 'Content-Length': '0'} 2022-09-03T06:50:14.882 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aaa34b89e2de301c-006312f926-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.882 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj185 2022-09-03T06:50:14.883 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:14.884 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:14.884 DEBUG:boto:path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.885 DEBUG:boto:auth_path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.885 DEBUG:boto:Method: GET 2022-09-03T06:50:14.886 DEBUG:boto:Path: /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.886 DEBUG:boto:Data: 2022-09-03T06:50:14.887 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.887 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:14.888 DEBUG:boto:Port: 8001 2022-09-03T06:50:14.888 DEBUG:boto:Params: {} 2022-09-03T06:50:14.889 DEBUG:boto:Token: None 2022-09-03T06:50:14.889 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.890 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wM5Ia5VqTXh23P/SkPn2JJdhhWg= 2022-09-03T06:50:14.891 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wM5Ia5VqTXh23P/SkPn2JJdhhWg=', 'Content-Length': '0'} 2022-09-03T06:50:14.893 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000085f0f0b102d446a-006312f926-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.894 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:14.894 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:14.895 DEBUG:boto:path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.896 DEBUG:boto:auth_path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.896 DEBUG:boto:Method: GET 2022-09-03T06:50:14.897 DEBUG:boto:Path: /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.897 DEBUG:boto:Data: 2022-09-03T06:50:14.898 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.898 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:14.899 DEBUG:boto:Port: 8000 2022-09-03T06:50:14.899 DEBUG:boto:Params: {} 2022-09-03T06:50:14.900 DEBUG:boto:Token: None 2022-09-03T06:50:14.900 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:14.901 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wM5Ia5VqTXh23P/SkPn2JJdhhWg= 2022-09-03T06:50:14.902 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wM5Ia5VqTXh23P/SkPn2JJdhhWg=', 'Content-Length': '0'} 2022-09-03T06:50:14.904 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000416ec0be96a63aa-006312f926-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.905 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:14.906 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:14.906 DEBUG:boto:path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:14.907 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:14.907 DEBUG:boto:Method: GET 2022-09-03T06:50:14.908 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj185 2022-09-03T06:50:14.908 DEBUG:boto:Data: 2022-09-03T06:50:14.909 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.910 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:14.910 DEBUG:boto:Port: 8001 2022-09-03T06:50:14.911 DEBUG:boto:Params: {} 2022-09-03T06:50:14.912 DEBUG:boto:Token: None 2022-09-03T06:50:14.912 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/ 2022-09-03T06:50:14.913 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs= 2022-09-03T06:50:14.913 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs=', 'Content-Length': '0'} 2022-09-03T06:50:14.916 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000048cad1b3d3f0c9a5-006312f926-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.917 DEBUG:boto:b'dsgvcy-75obj1851000falseobj1852022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:14.918 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:14.918 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:14.919 DEBUG:boto:path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:14.919 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:14.920 DEBUG:boto:Method: GET 2022-09-03T06:50:14.921 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj185 2022-09-03T06:50:14.921 DEBUG:boto:Data: 2022-09-03T06:50:14.922 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.922 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:14.923 DEBUG:boto:Port: 8000 2022-09-03T06:50:14.923 DEBUG:boto:Params: {} 2022-09-03T06:50:14.924 DEBUG:boto:Token: None 2022-09-03T06:50:14.925 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/ 2022-09-03T06:50:14.925 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs= 2022-09-03T06:50:14.926 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:en4G1DE1Ip3KJcbFC0CDUEvliFs=', 'Content-Length': '0'} 2022-09-03T06:50:14.930 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c904fba2a8384327-006312f926-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.932 DEBUG:boto:b'dsgvcy-75obj1851000falseobj1852022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:14.933 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:14.934 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:14.935 DEBUG:boto:Method: HEAD 2022-09-03T06:50:14.936 DEBUG:boto:Path: /dsgvcy-75/obj185 2022-09-03T06:50:14.937 DEBUG:boto:Data: 2022-09-03T06:50:14.938 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.939 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:14.940 DEBUG:boto:Port: 8001 2022-09-03T06:50:14.941 DEBUG:boto:Params: {} 2022-09-03T06:50:14.942 DEBUG:boto:Token: None 2022-09-03T06:50:14.943 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj185 2022-09-03T06:50:14.944 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IZu/4urG/3eXaV/fDDSfqP8J9q4= 2022-09-03T06:50:14.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IZu/4urG/3eXaV/fDDSfqP8J9q4=', 'Content-Length': '0'} 2022-09-03T06:50:14.947 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003f6d1f7a2ca78d44-006312f926-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.947 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:14.948 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:14.949 DEBUG:boto:Method: HEAD 2022-09-03T06:50:14.949 DEBUG:boto:Path: /dsgvcy-75/obj185 2022-09-03T06:50:14.950 DEBUG:boto:Data: 2022-09-03T06:50:14.950 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.951 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:14.951 DEBUG:boto:Port: 8000 2022-09-03T06:50:14.952 DEBUG:boto:Params: {} 2022-09-03T06:50:14.952 DEBUG:boto:Token: None 2022-09-03T06:50:14.953 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj185 2022-09-03T06:50:14.953 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IZu/4urG/3eXaV/fDDSfqP8J9q4= 2022-09-03T06:50:14.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IZu/4urG/3eXaV/fDDSfqP8J9q4=', 'Content-Length': '0'} 2022-09-03T06:50:14.956 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003c1773f85f207044-006312f926-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:14.957 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj185 2022-09-03T06:50:14.958 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:14.958 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:14.959 DEBUG:boto:Method: GET 2022-09-03T06:50:14.961 DEBUG:boto:Path: /dsgvcy-75/obj185 2022-09-03T06:50:14.961 DEBUG:boto:Data: 2022-09-03T06:50:14.962 DEBUG:boto:Headers: {} 2022-09-03T06:50:14.962 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:14.963 DEBUG:boto:Port: 8001 2022-09-03T06:50:14.964 DEBUG:boto:Params: {} 2022-09-03T06:50:14.964 DEBUG:boto:Token: None 2022-09-03T06:50:14.965 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:14 GMT /dsgvcy-75/obj185 2022-09-03T06:50:14.966 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XAI5P7ILHyJuxVxqI9zH8+1hEZM= 2022-09-03T06:50:14.967 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:14 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XAI5P7ILHyJuxVxqI9zH8+1hEZM=', 'Content-Length': '0'} 2022-09-03T06:50:14.969 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f9e31f10f490805e-006312f926-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:14 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.014 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:15.015 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:15.015 DEBUG:boto:Method: GET 2022-09-03T06:50:15.016 DEBUG:boto:Path: /dsgvcy-75/obj185 2022-09-03T06:50:15.018 DEBUG:boto:Data: 2022-09-03T06:50:15.020 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.021 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:15.021 DEBUG:boto:Port: 8000 2022-09-03T06:50:15.023 DEBUG:boto:Params: {} 2022-09-03T06:50:15.024 DEBUG:boto:Token: None 2022-09-03T06:50:15.025 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj185 2022-09-03T06:50:15.026 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UnAw0WTXOu74ytHuVVDxpJEj7kM= 2022-09-03T06:50:15.027 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UnAw0WTXOu74ytHuVVDxpJEj7kM=', 'Content-Length': '0'} 2022-09-03T06:50:15.030 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000654e86ea77beaf71-006312f927-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.032 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:15.033 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:15.036 DEBUG:boto:path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:15.037 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:15.038 DEBUG:boto:Method: GET 2022-09-03T06:50:15.039 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj185 2022-09-03T06:50:15.040 DEBUG:boto:Data: 2022-09-03T06:50:15.041 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.042 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:15.043 DEBUG:boto:Port: 8001 2022-09-03T06:50:15.044 DEBUG:boto:Params: {} 2022-09-03T06:50:15.046 DEBUG:boto:Token: None 2022-09-03T06:50:15.047 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/ 2022-09-03T06:50:15.048 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E= 2022-09-03T06:50:15.049 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E=', 'Content-Length': '0'} 2022-09-03T06:50:15.053 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000059e621d43fd6e7bc-006312f927-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.054 DEBUG:boto:b'dsgvcy-75obj1851000falseobj1852022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:15.056 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:15.058 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:15.059 DEBUG:boto:path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:15.060 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:15.061 DEBUG:boto:Method: GET 2022-09-03T06:50:15.063 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj185 2022-09-03T06:50:15.064 DEBUG:boto:Data: 2022-09-03T06:50:15.065 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.066 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:15.067 DEBUG:boto:Port: 8000 2022-09-03T06:50:15.068 DEBUG:boto:Params: {} 2022-09-03T06:50:15.069 DEBUG:boto:Token: None 2022-09-03T06:50:15.071 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/ 2022-09-03T06:50:15.073 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E= 2022-09-03T06:50:15.074 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E=', 'Content-Length': '0'} 2022-09-03T06:50:15.080 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000031d6e9ebe1d418a3-006312f927-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.082 DEBUG:boto:b'dsgvcy-75obj1851000falseobj1852022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:15.083 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:50:15.084 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:15.085 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:15.086 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.087 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.089 DEBUG:boto:Method: GET 2022-09-03T06:50:15.091 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.092 DEBUG:boto:Data: 2022-09-03T06:50:15.093 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.094 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:15.095 DEBUG:boto:Port: 8001 2022-09-03T06:50:15.096 DEBUG:boto:Params: {} 2022-09-03T06:50:15.097 DEBUG:boto:Token: None 2022-09-03T06:50:15.098 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.099 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1A+lP4SUD5vlsgIVQEZBybZlW5A= 2022-09-03T06:50:15.101 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1A+lP4SUD5vlsgIVQEZBybZlW5A=', 'Content-Length': '0'} 2022-09-03T06:50:15.105 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001f99f6fa6be3f5f9-006312f927-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.106 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:15.107 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:15.108 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.108 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.109 DEBUG:boto:Method: GET 2022-09-03T06:50:15.109 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.110 DEBUG:boto:Data: 2022-09-03T06:50:15.110 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.111 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:15.111 DEBUG:boto:Port: 8000 2022-09-03T06:50:15.112 DEBUG:boto:Params: {} 2022-09-03T06:50:15.112 DEBUG:boto:Token: None 2022-09-03T06:50:15.113 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.114 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1A+lP4SUD5vlsgIVQEZBybZlW5A= 2022-09-03T06:50:15.115 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1A+lP4SUD5vlsgIVQEZBybZlW5A=', 'Content-Length': '0'} 2022-09-03T06:50:15.117 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ba91ed099a7d6f74-006312f927-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.119 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:15.119 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:15.120 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.125 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.126 DEBUG:boto:Method: HEAD 2022-09-03T06:50:15.127 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.128 DEBUG:boto:Data: 2022-09-03T06:50:15.129 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.130 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:15.131 DEBUG:boto:Port: 8001 2022-09-03T06:50:15.132 DEBUG:boto:Params: {} 2022-09-03T06:50:15.133 DEBUG:boto:Token: None 2022-09-03T06:50:15.134 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.135 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:x0vgLYfpIusEKjNz0KhGUsL1Uoo= 2022-09-03T06:50:15.136 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:x0vgLYfpIusEKjNz0KhGUsL1Uoo=', 'Content-Length': '0'} 2022-09-03T06:50:15.139 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000038962fb335d361ba-006312f927-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.140 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:15.141 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:15.142 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.143 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.144 DEBUG:boto:Method: HEAD 2022-09-03T06:50:15.145 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.146 DEBUG:boto:Data: 2022-09-03T06:50:15.146 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.147 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:15.148 DEBUG:boto:Port: 8000 2022-09-03T06:50:15.149 DEBUG:boto:Params: {} 2022-09-03T06:50:15.150 DEBUG:boto:Token: None 2022-09-03T06:50:15.151 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.152 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:x0vgLYfpIusEKjNz0KhGUsL1Uoo= 2022-09-03T06:50:15.153 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:x0vgLYfpIusEKjNz0KhGUsL1Uoo=', 'Content-Length': '0'} 2022-09-03T06:50:15.156 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000899f01ad25d45bb6-006312f927-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.158 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:50:15.159 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:15.160 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:15.162 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.163 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.164 DEBUG:boto:Method: GET 2022-09-03T06:50:15.165 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.166 DEBUG:boto:Data: 2022-09-03T06:50:15.167 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.168 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:15.169 DEBUG:boto:Port: 8001 2022-09-03T06:50:15.169 DEBUG:boto:Params: {} 2022-09-03T06:50:15.170 DEBUG:boto:Token: None 2022-09-03T06:50:15.171 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.172 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1A+lP4SUD5vlsgIVQEZBybZlW5A= 2022-09-03T06:50:15.174 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1A+lP4SUD5vlsgIVQEZBybZlW5A=', 'Content-Length': '0'} 2022-09-03T06:50:15.176 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cfef504a1146792a-006312f927-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.177 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:15.178 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:15.179 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.179 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.180 DEBUG:boto:Method: GET 2022-09-03T06:50:15.181 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.181 DEBUG:boto:Data: 2022-09-03T06:50:15.182 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.182 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:15.183 DEBUG:boto:Port: 8000 2022-09-03T06:50:15.183 DEBUG:boto:Params: {} 2022-09-03T06:50:15.184 DEBUG:boto:Token: None 2022-09-03T06:50:15.185 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:15.185 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1A+lP4SUD5vlsgIVQEZBybZlW5A= 2022-09-03T06:50:15.186 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1A+lP4SUD5vlsgIVQEZBybZlW5A=', 'Content-Length': '0'} 2022-09-03T06:50:15.190 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a4295fef200fbda9-006312f927-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.192 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:15.192 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:15.193 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:15.194 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:15.194 DEBUG:boto:Method: GET 2022-09-03T06:50:15.195 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:50:15.195 DEBUG:boto:Data: 2022-09-03T06:50:15.196 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.196 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:15.197 DEBUG:boto:Port: 8001 2022-09-03T06:50:15.197 DEBUG:boto:Params: {} 2022-09-03T06:50:15.198 DEBUG:boto:Token: None 2022-09-03T06:50:15.199 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/ 2022-09-03T06:50:15.199 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E= 2022-09-03T06:50:15.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E=', 'Content-Length': '0'} 2022-09-03T06:50:15.203 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002df145da3e7d716f-006312f927-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.204 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:15.205 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:15.206 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:15.206 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:15.207 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:15.208 DEBUG:boto:Method: GET 2022-09-03T06:50:15.208 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:50:15.209 DEBUG:boto:Data: 2022-09-03T06:50:15.213 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.215 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:15.216 DEBUG:boto:Port: 8000 2022-09-03T06:50:15.217 DEBUG:boto:Params: {} 2022-09-03T06:50:15.218 DEBUG:boto:Token: None 2022-09-03T06:50:15.219 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/ 2022-09-03T06:50:15.219 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E= 2022-09-03T06:50:15.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E=', 'Content-Length': '0'} 2022-09-03T06:50:15.226 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000092eb1cd047b76803-006312f927-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.227 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:15.228 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:15.228 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:15.229 DEBUG:boto:Method: HEAD 2022-09-03T06:50:15.229 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:50:15.230 DEBUG:boto:Data: 2022-09-03T06:50:15.230 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.231 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:15.231 DEBUG:boto:Port: 8001 2022-09-03T06:50:15.232 DEBUG:boto:Params: {} 2022-09-03T06:50:15.232 DEBUG:boto:Token: None 2022-09-03T06:50:15.233 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj19 2022-09-03T06:50:15.234 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5vg6daRAepyy4GCviC50nyOTgEc= 2022-09-03T06:50:15.234 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5vg6daRAepyy4GCviC50nyOTgEc=', 'Content-Length': '0'} 2022-09-03T06:50:15.237 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c05facb9110dee4a-006312f927-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.239 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:15.239 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:15.241 DEBUG:boto:Method: HEAD 2022-09-03T06:50:15.241 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:50:15.242 DEBUG:boto:Data: 2022-09-03T06:50:15.243 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.244 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:15.244 DEBUG:boto:Port: 8000 2022-09-03T06:50:15.245 DEBUG:boto:Params: {} 2022-09-03T06:50:15.246 DEBUG:boto:Token: None 2022-09-03T06:50:15.247 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj19 2022-09-03T06:50:15.247 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5vg6daRAepyy4GCviC50nyOTgEc= 2022-09-03T06:50:15.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5vg6daRAepyy4GCviC50nyOTgEc=', 'Content-Length': '0'} 2022-09-03T06:50:15.250 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000de9f41e958c6fed6-006312f927-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.251 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:50:15.252 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:15.252 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:15.253 DEBUG:boto:Method: GET 2022-09-03T06:50:15.253 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:50:15.254 DEBUG:boto:Data: 2022-09-03T06:50:15.254 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.255 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:15.256 DEBUG:boto:Port: 8001 2022-09-03T06:50:15.256 DEBUG:boto:Params: {} 2022-09-03T06:50:15.257 DEBUG:boto:Token: None 2022-09-03T06:50:15.257 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj19 2022-09-03T06:50:15.258 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/bP4nNU5UgCn+2HvBmYYlPK4bZU= 2022-09-03T06:50:15.259 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/bP4nNU5UgCn+2HvBmYYlPK4bZU=', 'Content-Length': '0'} 2022-09-03T06:50:15.261 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007a64f118088ebeb9-006312f927-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.262 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:15.263 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:15.264 DEBUG:boto:Method: GET 2022-09-03T06:50:15.265 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:50:15.265 DEBUG:boto:Data: 2022-09-03T06:50:15.266 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.266 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:15.267 DEBUG:boto:Port: 8000 2022-09-03T06:50:15.268 DEBUG:boto:Params: {} 2022-09-03T06:50:15.268 DEBUG:boto:Token: None 2022-09-03T06:50:15.269 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj19 2022-09-03T06:50:15.269 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/bP4nNU5UgCn+2HvBmYYlPK4bZU= 2022-09-03T06:50:15.270 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/bP4nNU5UgCn+2HvBmYYlPK4bZU=', 'Content-Length': '0'} 2022-09-03T06:50:15.273 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db6f97170895e338-006312f927-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.314 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:15.315 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:15.316 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:15.316 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:15.317 DEBUG:boto:Method: GET 2022-09-03T06:50:15.317 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:50:15.318 DEBUG:boto:Data: 2022-09-03T06:50:15.318 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.319 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:15.319 DEBUG:boto:Port: 8001 2022-09-03T06:50:15.320 DEBUG:boto:Params: {} 2022-09-03T06:50:15.321 DEBUG:boto:Token: None 2022-09-03T06:50:15.321 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/ 2022-09-03T06:50:15.322 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E= 2022-09-03T06:50:15.323 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E=', 'Content-Length': '0'} 2022-09-03T06:50:15.326 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008d38581da762ed6c-006312f927-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.328 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:15.329 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:15.329 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:15.330 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:15.330 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:15.331 DEBUG:boto:Method: GET 2022-09-03T06:50:15.332 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:50:15.332 DEBUG:boto:Data: 2022-09-03T06:50:15.333 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.333 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:15.334 DEBUG:boto:Port: 8000 2022-09-03T06:50:15.335 DEBUG:boto:Params: {} 2022-09-03T06:50:15.335 DEBUG:boto:Token: None 2022-09-03T06:50:15.336 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/ 2022-09-03T06:50:15.337 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E= 2022-09-03T06:50:15.337 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E=', 'Content-Length': '0'} 2022-09-03T06:50:15.342 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000586f46ef01782f64-006312f927-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.343 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:15.343 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:50:15.344 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:15.345 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:15.345 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.346 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.346 DEBUG:boto:Method: GET 2022-09-03T06:50:15.347 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.348 DEBUG:boto:Data: 2022-09-03T06:50:15.348 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.349 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:15.349 DEBUG:boto:Port: 8001 2022-09-03T06:50:15.350 DEBUG:boto:Params: {} 2022-09-03T06:50:15.351 DEBUG:boto:Token: None 2022-09-03T06:50:15.351 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.352 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/BJCBJIde41IQuoFgAQvWid4u4= 2022-09-03T06:50:15.353 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/BJCBJIde41IQuoFgAQvWid4u4=', 'Content-Length': '0'} 2022-09-03T06:50:15.357 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003553714a9d189907-006312f927-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.402 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:15.403 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:15.405 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.406 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.407 DEBUG:boto:Method: GET 2022-09-03T06:50:15.409 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.410 DEBUG:boto:Data: 2022-09-03T06:50:15.418 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.419 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:15.420 DEBUG:boto:Port: 8000 2022-09-03T06:50:15.421 DEBUG:boto:Params: {} 2022-09-03T06:50:15.422 DEBUG:boto:Token: None 2022-09-03T06:50:15.423 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.424 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/BJCBJIde41IQuoFgAQvWid4u4= 2022-09-03T06:50:15.426 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/BJCBJIde41IQuoFgAQvWid4u4=', 'Content-Length': '0'} 2022-09-03T06:50:15.429 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026d9fc04c6f9d1f2-006312f927-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.430 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:15.431 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:15.432 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.433 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.434 DEBUG:boto:Method: HEAD 2022-09-03T06:50:15.435 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.436 DEBUG:boto:Data: 2022-09-03T06:50:15.437 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.438 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:15.439 DEBUG:boto:Port: 8001 2022-09-03T06:50:15.440 DEBUG:boto:Params: {} 2022-09-03T06:50:15.441 DEBUG:boto:Token: None 2022-09-03T06:50:15.442 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.443 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1gDmv3uptNMG3NGEMPF8HbLbREs= 2022-09-03T06:50:15.444 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1gDmv3uptNMG3NGEMPF8HbLbREs=', 'Content-Length': '0'} 2022-09-03T06:50:15.447 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004743c62e480222fd-006312f927-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.448 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:15.449 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:15.450 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.451 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.452 DEBUG:boto:Method: HEAD 2022-09-03T06:50:15.452 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.453 DEBUG:boto:Data: 2022-09-03T06:50:15.454 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.455 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:15.456 DEBUG:boto:Port: 8000 2022-09-03T06:50:15.457 DEBUG:boto:Params: {} 2022-09-03T06:50:15.458 DEBUG:boto:Token: None 2022-09-03T06:50:15.459 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.461 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1gDmv3uptNMG3NGEMPF8HbLbREs= 2022-09-03T06:50:15.462 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1gDmv3uptNMG3NGEMPF8HbLbREs=', 'Content-Length': '0'} 2022-09-03T06:50:15.465 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009dc813eb7e09b1f5-006312f927-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.466 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:50:15.467 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:15.468 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:15.469 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.470 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.471 DEBUG:boto:Method: GET 2022-09-03T06:50:15.472 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.473 DEBUG:boto:Data: 2022-09-03T06:50:15.474 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.475 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:15.477 DEBUG:boto:Port: 8001 2022-09-03T06:50:15.479 DEBUG:boto:Params: {} 2022-09-03T06:50:15.480 DEBUG:boto:Token: None 2022-09-03T06:50:15.481 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.482 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/BJCBJIde41IQuoFgAQvWid4u4= 2022-09-03T06:50:15.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/BJCBJIde41IQuoFgAQvWid4u4=', 'Content-Length': '0'} 2022-09-03T06:50:15.487 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000de8c0951fd431c77-006312f927-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.488 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:15.489 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:15.490 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.492 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.493 DEBUG:boto:Method: GET 2022-09-03T06:50:15.494 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.495 DEBUG:boto:Data: 2022-09-03T06:50:15.496 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.497 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:15.498 DEBUG:boto:Port: 8000 2022-09-03T06:50:15.499 DEBUG:boto:Params: {} 2022-09-03T06:50:15.500 DEBUG:boto:Token: None 2022-09-03T06:50:15.502 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:15.503 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/BJCBJIde41IQuoFgAQvWid4u4= 2022-09-03T06:50:15.504 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/BJCBJIde41IQuoFgAQvWid4u4=', 'Content-Length': '0'} 2022-09-03T06:50:15.507 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ce827cea77ace23-006312f927-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.509 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:15.510 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:15.511 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:15.512 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:15.513 DEBUG:boto:Method: GET 2022-09-03T06:50:15.514 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:50:15.515 DEBUG:boto:Data: 2022-09-03T06:50:15.516 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.518 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:15.519 DEBUG:boto:Port: 8001 2022-09-03T06:50:15.520 DEBUG:boto:Params: {} 2022-09-03T06:50:15.521 DEBUG:boto:Token: None 2022-09-03T06:50:15.522 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/ 2022-09-03T06:50:15.523 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E= 2022-09-03T06:50:15.524 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E=', 'Content-Length': '0'} 2022-09-03T06:50:15.528 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000940b98233a368829-006312f927-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.530 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:15.532 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:15.533 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:15.534 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:15.535 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:15.537 DEBUG:boto:Method: GET 2022-09-03T06:50:15.537 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:50:15.538 DEBUG:boto:Data: 2022-09-03T06:50:15.539 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.540 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:15.541 DEBUG:boto:Port: 8000 2022-09-03T06:50:15.542 DEBUG:boto:Params: {} 2022-09-03T06:50:15.543 DEBUG:boto:Token: None 2022-09-03T06:50:15.544 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/ 2022-09-03T06:50:15.545 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E= 2022-09-03T06:50:15.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E=', 'Content-Length': '0'} 2022-09-03T06:50:15.552 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000532e41fd372527b2-006312f927-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.554 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:15.556 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:15.557 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:15.559 DEBUG:boto:Method: HEAD 2022-09-03T06:50:15.560 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:50:15.562 DEBUG:boto:Data: 2022-09-03T06:50:15.562 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.563 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:15.563 DEBUG:boto:Port: 8001 2022-09-03T06:50:15.564 DEBUG:boto:Params: {} 2022-09-03T06:50:15.565 DEBUG:boto:Token: None 2022-09-03T06:50:15.566 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj22 2022-09-03T06:50:15.567 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wMvrPdgVh3dAs2jl17kQn8VnyCI= 2022-09-03T06:50:15.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wMvrPdgVh3dAs2jl17kQn8VnyCI=', 'Content-Length': '0'} 2022-09-03T06:50:15.571 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004f21f16dbe5c29ea-006312f927-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.572 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:15.573 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:15.574 DEBUG:boto:Method: HEAD 2022-09-03T06:50:15.574 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:50:15.575 DEBUG:boto:Data: 2022-09-03T06:50:15.576 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.578 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:15.579 DEBUG:boto:Port: 8000 2022-09-03T06:50:15.579 DEBUG:boto:Params: {} 2022-09-03T06:50:15.580 DEBUG:boto:Token: None 2022-09-03T06:50:15.581 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj22 2022-09-03T06:50:15.582 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wMvrPdgVh3dAs2jl17kQn8VnyCI= 2022-09-03T06:50:15.583 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wMvrPdgVh3dAs2jl17kQn8VnyCI=', 'Content-Length': '0'} 2022-09-03T06:50:15.586 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004c4e6b15af784994-006312f927-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.587 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:50:15.587 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:15.588 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:15.589 DEBUG:boto:Method: GET 2022-09-03T06:50:15.590 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:50:15.591 DEBUG:boto:Data: 2022-09-03T06:50:15.592 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.593 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:15.594 DEBUG:boto:Port: 8001 2022-09-03T06:50:15.595 DEBUG:boto:Params: {} 2022-09-03T06:50:15.595 DEBUG:boto:Token: None 2022-09-03T06:50:15.596 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj22 2022-09-03T06:50:15.597 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zRpKQZljV0mC1eYlkjkYEIR0cOE= 2022-09-03T06:50:15.598 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zRpKQZljV0mC1eYlkjkYEIR0cOE=', 'Content-Length': '0'} 2022-09-03T06:50:15.605 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000325b1bda2d9751f6-006312f927-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.607 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:15.608 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:15.609 DEBUG:boto:Method: GET 2022-09-03T06:50:15.610 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:50:15.611 DEBUG:boto:Data: 2022-09-03T06:50:15.612 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.613 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:15.615 DEBUG:boto:Port: 8000 2022-09-03T06:50:15.616 DEBUG:boto:Params: {} 2022-09-03T06:50:15.617 DEBUG:boto:Token: None 2022-09-03T06:50:15.618 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj22 2022-09-03T06:50:15.619 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zRpKQZljV0mC1eYlkjkYEIR0cOE= 2022-09-03T06:50:15.621 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zRpKQZljV0mC1eYlkjkYEIR0cOE=', 'Content-Length': '0'} 2022-09-03T06:50:15.625 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000016188a24f1c93602-006312f927-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.672 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:15.674 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:15.675 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:15.676 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:15.677 DEBUG:boto:Method: GET 2022-09-03T06:50:15.678 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:50:15.679 DEBUG:boto:Data: 2022-09-03T06:50:15.680 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.682 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:15.683 DEBUG:boto:Port: 8001 2022-09-03T06:50:15.684 DEBUG:boto:Params: {} 2022-09-03T06:50:15.685 DEBUG:boto:Token: None 2022-09-03T06:50:15.687 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/ 2022-09-03T06:50:15.688 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E= 2022-09-03T06:50:15.691 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E=', 'Content-Length': '0'} 2022-09-03T06:50:15.705 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009d9034f8d6cd2823-006312f927-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.706 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:15.707 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:15.707 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:15.708 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:15.708 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:15.709 DEBUG:boto:Method: GET 2022-09-03T06:50:15.709 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:50:15.710 DEBUG:boto:Data: 2022-09-03T06:50:15.711 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.711 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:15.712 DEBUG:boto:Port: 8000 2022-09-03T06:50:15.712 DEBUG:boto:Params: {} 2022-09-03T06:50:15.713 DEBUG:boto:Token: None 2022-09-03T06:50:15.714 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/ 2022-09-03T06:50:15.714 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E= 2022-09-03T06:50:15.715 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E=', 'Content-Length': '0'} 2022-09-03T06:50:15.725 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c824a4be9ae87859-006312f927-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.735 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:15.737 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:50:15.738 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:15.739 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:15.741 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.742 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.743 DEBUG:boto:Method: GET 2022-09-03T06:50:15.744 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.745 DEBUG:boto:Data: 2022-09-03T06:50:15.751 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.753 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:15.754 DEBUG:boto:Port: 8001 2022-09-03T06:50:15.755 DEBUG:boto:Params: {} 2022-09-03T06:50:15.756 DEBUG:boto:Token: None 2022-09-03T06:50:15.757 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.762 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QjjhTDF5U2SFBFV0/Ij3TQDIjgc= 2022-09-03T06:50:15.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QjjhTDF5U2SFBFV0/Ij3TQDIjgc=', 'Content-Length': '0'} 2022-09-03T06:50:15.766 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000014fb1cf210668b1f-006312f927-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.767 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:15.768 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:15.769 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.770 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.770 DEBUG:boto:Method: GET 2022-09-03T06:50:15.771 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.771 DEBUG:boto:Data: 2022-09-03T06:50:15.772 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.773 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:15.773 DEBUG:boto:Port: 8000 2022-09-03T06:50:15.774 DEBUG:boto:Params: {} 2022-09-03T06:50:15.775 DEBUG:boto:Token: None 2022-09-03T06:50:15.775 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.776 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QjjhTDF5U2SFBFV0/Ij3TQDIjgc= 2022-09-03T06:50:15.776 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QjjhTDF5U2SFBFV0/Ij3TQDIjgc=', 'Content-Length': '0'} 2022-09-03T06:50:15.779 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b607a54207c952fb-006312f927-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.783 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:15.784 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:15.785 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.786 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.788 DEBUG:boto:Method: HEAD 2022-09-03T06:50:15.789 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.792 DEBUG:boto:Data: 2022-09-03T06:50:15.793 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.794 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:15.795 DEBUG:boto:Port: 8001 2022-09-03T06:50:15.796 DEBUG:boto:Params: {} 2022-09-03T06:50:15.797 DEBUG:boto:Token: None 2022-09-03T06:50:15.798 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.799 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NFmaIOfcEps12MJfyMXl51vPztY= 2022-09-03T06:50:15.801 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NFmaIOfcEps12MJfyMXl51vPztY=', 'Content-Length': '0'} 2022-09-03T06:50:15.809 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000061e45f3128bb43d1-006312f927-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.811 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:15.812 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:15.813 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.813 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.815 DEBUG:boto:Method: HEAD 2022-09-03T06:50:15.816 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.818 DEBUG:boto:Data: 2022-09-03T06:50:15.819 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.820 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:15.821 DEBUG:boto:Port: 8000 2022-09-03T06:50:15.822 DEBUG:boto:Params: {} 2022-09-03T06:50:15.823 DEBUG:boto:Token: None 2022-09-03T06:50:15.825 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.826 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NFmaIOfcEps12MJfyMXl51vPztY= 2022-09-03T06:50:15.827 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NFmaIOfcEps12MJfyMXl51vPztY=', 'Content-Length': '0'} 2022-09-03T06:50:15.834 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e82b5ae74c1935b-006312f927-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.837 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:50:15.838 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:15.839 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:15.840 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.841 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.842 DEBUG:boto:Method: GET 2022-09-03T06:50:15.843 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.844 DEBUG:boto:Data: 2022-09-03T06:50:15.845 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.846 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:15.846 DEBUG:boto:Port: 8001 2022-09-03T06:50:15.847 DEBUG:boto:Params: {} 2022-09-03T06:50:15.848 DEBUG:boto:Token: None 2022-09-03T06:50:15.848 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.849 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QjjhTDF5U2SFBFV0/Ij3TQDIjgc= 2022-09-03T06:50:15.850 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QjjhTDF5U2SFBFV0/Ij3TQDIjgc=', 'Content-Length': '0'} 2022-09-03T06:50:15.857 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000522c4752694d303b-006312f927-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.899 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:15.900 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:15.900 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.901 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.904 DEBUG:boto:Method: GET 2022-09-03T06:50:15.905 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.905 DEBUG:boto:Data: 2022-09-03T06:50:15.906 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.907 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:15.908 DEBUG:boto:Port: 8000 2022-09-03T06:50:15.910 DEBUG:boto:Params: {} 2022-09-03T06:50:15.914 DEBUG:boto:Token: None 2022-09-03T06:50:15.916 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:15.917 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QjjhTDF5U2SFBFV0/Ij3TQDIjgc= 2022-09-03T06:50:15.917 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QjjhTDF5U2SFBFV0/Ij3TQDIjgc=', 'Content-Length': '0'} 2022-09-03T06:50:15.921 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004cd220b63b7bec00-006312f927-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.922 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:15.923 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:15.924 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:15.925 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:15.926 DEBUG:boto:Method: GET 2022-09-03T06:50:15.927 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:50:15.928 DEBUG:boto:Data: 2022-09-03T06:50:15.929 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.930 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:15.931 DEBUG:boto:Port: 8001 2022-09-03T06:50:15.931 DEBUG:boto:Params: {} 2022-09-03T06:50:15.935 DEBUG:boto:Token: None 2022-09-03T06:50:15.936 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/ 2022-09-03T06:50:15.937 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E= 2022-09-03T06:50:15.937 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E=', 'Content-Length': '0'} 2022-09-03T06:50:15.941 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000416f96b98463c7d6-006312f927-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.942 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:15.943 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:15.944 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:15.945 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:15.945 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:15.946 DEBUG:boto:Method: GET 2022-09-03T06:50:15.947 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:50:15.947 DEBUG:boto:Data: 2022-09-03T06:50:15.948 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.949 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:15.950 DEBUG:boto:Port: 8000 2022-09-03T06:50:15.951 DEBUG:boto:Params: {} 2022-09-03T06:50:15.952 DEBUG:boto:Token: None 2022-09-03T06:50:15.953 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/ 2022-09-03T06:50:15.954 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E= 2022-09-03T06:50:15.955 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e3IWASdDK2d9IVckwtFq/Yr2x2E=', 'Content-Length': '0'} 2022-09-03T06:50:15.960 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a5b247399ffdba45-006312f927-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.962 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:15.963 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:15.964 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:15.965 DEBUG:boto:Method: HEAD 2022-09-03T06:50:15.966 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:50:15.967 DEBUG:boto:Data: 2022-09-03T06:50:15.968 DEBUG:boto:Headers: {} 2022-09-03T06:50:15.969 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:15.971 DEBUG:boto:Port: 8001 2022-09-03T06:50:15.972 DEBUG:boto:Params: {} 2022-09-03T06:50:15.973 DEBUG:boto:Token: None 2022-09-03T06:50:15.974 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:15 GMT /dsgvcy-75/obj23 2022-09-03T06:50:15.975 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:dL53C+ifOsRZpq+VdfH6BZfsUzI= 2022-09-03T06:50:15.976 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:15 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:dL53C+ifOsRZpq+VdfH6BZfsUzI=', 'Content-Length': '0'} 2022-09-03T06:50:15.986 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fe5b2343a967d4b2-006312f927-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:15 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:15.995 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:15.996 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:15.997 DEBUG:boto:Method: HEAD 2022-09-03T06:50:15.998 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:50:15.999 DEBUG:boto:Data: 2022-09-03T06:50:16.001 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.001 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:16.002 DEBUG:boto:Port: 8000 2022-09-03T06:50:16.003 DEBUG:boto:Params: {} 2022-09-03T06:50:16.010 DEBUG:boto:Token: None 2022-09-03T06:50:16.011 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj23 2022-09-03T06:50:16.011 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h90xOkBQqw4oiFtCJK1i7/rsPIM= 2022-09-03T06:50:16.012 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h90xOkBQqw4oiFtCJK1i7/rsPIM=', 'Content-Length': '0'} 2022-09-03T06:50:16.014 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000006edf0af42f16857-006312f928-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.015 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:50:16.016 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:16.016 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:16.017 DEBUG:boto:Method: GET 2022-09-03T06:50:16.018 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:50:16.018 DEBUG:boto:Data: 2022-09-03T06:50:16.019 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.019 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:16.020 DEBUG:boto:Port: 8001 2022-09-03T06:50:16.021 DEBUG:boto:Params: {} 2022-09-03T06:50:16.022 DEBUG:boto:Token: None 2022-09-03T06:50:16.023 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj23 2022-09-03T06:50:16.024 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EHiDhG61YSkmEA5cB1CxpjGY8nk= 2022-09-03T06:50:16.025 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EHiDhG61YSkmEA5cB1CxpjGY8nk=', 'Content-Length': '0'} 2022-09-03T06:50:16.035 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000240c0d0816278495-006312f928-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.038 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:16.039 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:16.040 DEBUG:boto:Method: GET 2022-09-03T06:50:16.041 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:50:16.041 DEBUG:boto:Data: 2022-09-03T06:50:16.042 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.043 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:16.044 DEBUG:boto:Port: 8000 2022-09-03T06:50:16.045 DEBUG:boto:Params: {} 2022-09-03T06:50:16.046 DEBUG:boto:Token: None 2022-09-03T06:50:16.047 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj23 2022-09-03T06:50:16.049 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EHiDhG61YSkmEA5cB1CxpjGY8nk= 2022-09-03T06:50:16.050 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EHiDhG61YSkmEA5cB1CxpjGY8nk=', 'Content-Length': '0'} 2022-09-03T06:50:16.054 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004edb80d99e63e2da-006312f928-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.056 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:16.057 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:16.057 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:16.058 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:16.059 DEBUG:boto:Method: GET 2022-09-03T06:50:16.060 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:50:16.061 DEBUG:boto:Data: 2022-09-03T06:50:16.062 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.063 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:16.064 DEBUG:boto:Port: 8001 2022-09-03T06:50:16.065 DEBUG:boto:Params: {} 2022-09-03T06:50:16.066 DEBUG:boto:Token: None 2022-09-03T06:50:16.067 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/ 2022-09-03T06:50:16.068 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sOo3A7inbLk2imqiirszYcz/dhs= 2022-09-03T06:50:16.068 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sOo3A7inbLk2imqiirszYcz/dhs=', 'Content-Length': '0'} 2022-09-03T06:50:16.071 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f088d2860b4b3598-006312f928-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.072 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:16.073 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:16.074 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:16.075 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:16.076 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:16.077 DEBUG:boto:Method: GET 2022-09-03T06:50:16.078 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:50:16.079 DEBUG:boto:Data: 2022-09-03T06:50:16.080 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.081 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:16.082 DEBUG:boto:Port: 8000 2022-09-03T06:50:16.083 DEBUG:boto:Params: {} 2022-09-03T06:50:16.084 DEBUG:boto:Token: None 2022-09-03T06:50:16.085 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/ 2022-09-03T06:50:16.088 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sOo3A7inbLk2imqiirszYcz/dhs= 2022-09-03T06:50:16.090 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sOo3A7inbLk2imqiirszYcz/dhs=', 'Content-Length': '0'} 2022-09-03T06:50:16.097 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007896e0619070849e-006312f928-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.100 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:16.102 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:50:16.103 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:16.104 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:16.105 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.106 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.107 DEBUG:boto:Method: GET 2022-09-03T06:50:16.108 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.109 DEBUG:boto:Data: 2022-09-03T06:50:16.110 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.111 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:16.113 DEBUG:boto:Port: 8001 2022-09-03T06:50:16.114 DEBUG:boto:Params: {} 2022-09-03T06:50:16.115 DEBUG:boto:Token: None 2022-09-03T06:50:16.117 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.118 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BOlm6LinRNUIVqO1LR18PJqwcrI= 2022-09-03T06:50:16.120 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BOlm6LinRNUIVqO1LR18PJqwcrI=', 'Content-Length': '0'} 2022-09-03T06:50:16.123 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004f247178869ace1b-006312f928-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.124 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:16.125 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:16.126 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.127 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.128 DEBUG:boto:Method: GET 2022-09-03T06:50:16.130 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.131 DEBUG:boto:Data: 2022-09-03T06:50:16.132 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.138 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:16.140 DEBUG:boto:Port: 8000 2022-09-03T06:50:16.141 DEBUG:boto:Params: {} 2022-09-03T06:50:16.142 DEBUG:boto:Token: None 2022-09-03T06:50:16.143 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.144 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BOlm6LinRNUIVqO1LR18PJqwcrI= 2022-09-03T06:50:16.145 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BOlm6LinRNUIVqO1LR18PJqwcrI=', 'Content-Length': '0'} 2022-09-03T06:50:16.152 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a683119db3e96803-006312f928-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.160 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:16.162 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:16.163 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.164 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.165 DEBUG:boto:Method: HEAD 2022-09-03T06:50:16.166 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.166 DEBUG:boto:Data: 2022-09-03T06:50:16.167 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.168 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:16.173 DEBUG:boto:Port: 8001 2022-09-03T06:50:16.175 DEBUG:boto:Params: {} 2022-09-03T06:50:16.178 DEBUG:boto:Token: None 2022-09-03T06:50:16.179 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.180 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:evwdLXG+3k5JRVB62wQdH03vpuY= 2022-09-03T06:50:16.182 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:evwdLXG+3k5JRVB62wQdH03vpuY=', 'Content-Length': '0'} 2022-09-03T06:50:16.189 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000363b77ca1b7db950-006312f928-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.190 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:16.192 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:16.193 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.194 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.195 DEBUG:boto:Method: HEAD 2022-09-03T06:50:16.196 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.199 DEBUG:boto:Data: 2022-09-03T06:50:16.201 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.202 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:16.203 DEBUG:boto:Port: 8000 2022-09-03T06:50:16.204 DEBUG:boto:Params: {} 2022-09-03T06:50:16.205 DEBUG:boto:Token: None 2022-09-03T06:50:16.206 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.207 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:evwdLXG+3k5JRVB62wQdH03vpuY= 2022-09-03T06:50:16.211 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:evwdLXG+3k5JRVB62wQdH03vpuY=', 'Content-Length': '0'} 2022-09-03T06:50:16.214 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000035f37842aeebc701-006312f928-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.216 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:50:16.216 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:16.217 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:16.218 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.222 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.223 DEBUG:boto:Method: GET 2022-09-03T06:50:16.224 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.224 DEBUG:boto:Data: 2022-09-03T06:50:16.225 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.225 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:16.226 DEBUG:boto:Port: 8001 2022-09-03T06:50:16.227 DEBUG:boto:Params: {} 2022-09-03T06:50:16.227 DEBUG:boto:Token: None 2022-09-03T06:50:16.228 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.231 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BOlm6LinRNUIVqO1LR18PJqwcrI= 2022-09-03T06:50:16.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BOlm6LinRNUIVqO1LR18PJqwcrI=', 'Content-Length': '0'} 2022-09-03T06:50:16.247 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000da1d719a9da89806-006312f928-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.249 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:16.250 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:16.252 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.253 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.254 DEBUG:boto:Method: GET 2022-09-03T06:50:16.255 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.268 DEBUG:boto:Data: 2022-09-03T06:50:16.270 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.271 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:16.271 DEBUG:boto:Port: 8000 2022-09-03T06:50:16.272 DEBUG:boto:Params: {} 2022-09-03T06:50:16.278 DEBUG:boto:Token: None 2022-09-03T06:50:16.279 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:16.281 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:BOlm6LinRNUIVqO1LR18PJqwcrI= 2022-09-03T06:50:16.287 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:BOlm6LinRNUIVqO1LR18PJqwcrI=', 'Content-Length': '0'} 2022-09-03T06:50:16.306 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a7af938f51458d86-006312f928-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.308 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:16.308 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:16.309 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:16.309 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:16.310 DEBUG:boto:Method: GET 2022-09-03T06:50:16.311 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:50:16.311 DEBUG:boto:Data: 2022-09-03T06:50:16.312 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.312 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:16.313 DEBUG:boto:Port: 8001 2022-09-03T06:50:16.314 DEBUG:boto:Params: {} 2022-09-03T06:50:16.315 DEBUG:boto:Token: None 2022-09-03T06:50:16.315 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/ 2022-09-03T06:50:16.317 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sOo3A7inbLk2imqiirszYcz/dhs= 2022-09-03T06:50:16.317 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sOo3A7inbLk2imqiirszYcz/dhs=', 'Content-Length': '0'} 2022-09-03T06:50:16.320 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004edf778c26c689f7-006312f928-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.321 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:16.323 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:16.323 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:16.324 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:16.324 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:16.325 DEBUG:boto:Method: GET 2022-09-03T06:50:16.325 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:50:16.326 DEBUG:boto:Data: 2022-09-03T06:50:16.330 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.331 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:16.332 DEBUG:boto:Port: 8000 2022-09-03T06:50:16.333 DEBUG:boto:Params: {} 2022-09-03T06:50:16.334 DEBUG:boto:Token: None 2022-09-03T06:50:16.335 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/ 2022-09-03T06:50:16.336 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sOo3A7inbLk2imqiirszYcz/dhs= 2022-09-03T06:50:16.337 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sOo3A7inbLk2imqiirszYcz/dhs=', 'Content-Length': '0'} 2022-09-03T06:50:16.351 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008280d62ea6da0658-006312f928-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.361 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:16.363 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:16.364 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:16.366 DEBUG:boto:Method: HEAD 2022-09-03T06:50:16.367 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:50:16.368 DEBUG:boto:Data: 2022-09-03T06:50:16.369 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.370 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:16.371 DEBUG:boto:Port: 8001 2022-09-03T06:50:16.372 DEBUG:boto:Params: {} 2022-09-03T06:50:16.374 DEBUG:boto:Token: None 2022-09-03T06:50:16.375 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj29 2022-09-03T06:50:16.376 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kyelrPatScWQWmcIkRXMtc4uzEc= 2022-09-03T06:50:16.376 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kyelrPatScWQWmcIkRXMtc4uzEc=', 'Content-Length': '0'} 2022-09-03T06:50:16.379 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000100bbefaa622468-006312f928-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.379 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:16.380 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:16.381 DEBUG:boto:Method: HEAD 2022-09-03T06:50:16.381 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:50:16.382 DEBUG:boto:Data: 2022-09-03T06:50:16.382 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.383 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:16.383 DEBUG:boto:Port: 8000 2022-09-03T06:50:16.384 DEBUG:boto:Params: {} 2022-09-03T06:50:16.384 DEBUG:boto:Token: None 2022-09-03T06:50:16.385 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj29 2022-09-03T06:50:16.386 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kyelrPatScWQWmcIkRXMtc4uzEc= 2022-09-03T06:50:16.387 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kyelrPatScWQWmcIkRXMtc4uzEc=', 'Content-Length': '0'} 2022-09-03T06:50:16.394 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d2d6fb3174b25990-006312f928-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.395 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:50:16.396 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:16.396 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:16.397 DEBUG:boto:Method: GET 2022-09-03T06:50:16.398 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:50:16.399 DEBUG:boto:Data: 2022-09-03T06:50:16.399 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.400 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:16.401 DEBUG:boto:Port: 8001 2022-09-03T06:50:16.402 DEBUG:boto:Params: {} 2022-09-03T06:50:16.402 DEBUG:boto:Token: None 2022-09-03T06:50:16.403 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj29 2022-09-03T06:50:16.404 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZvQOqnVAYLEiSkHNVlfU5ZQQVBc= 2022-09-03T06:50:16.404 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZvQOqnVAYLEiSkHNVlfU5ZQQVBc=', 'Content-Length': '0'} 2022-09-03T06:50:16.408 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f4cde52f9fcf5b78-006312f928-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.409 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:16.410 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:16.411 DEBUG:boto:Method: GET 2022-09-03T06:50:16.412 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:50:16.413 DEBUG:boto:Data: 2022-09-03T06:50:16.413 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.414 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:16.415 DEBUG:boto:Port: 8000 2022-09-03T06:50:16.416 DEBUG:boto:Params: {} 2022-09-03T06:50:16.417 DEBUG:boto:Token: None 2022-09-03T06:50:16.418 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj29 2022-09-03T06:50:16.420 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZvQOqnVAYLEiSkHNVlfU5ZQQVBc= 2022-09-03T06:50:16.421 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZvQOqnVAYLEiSkHNVlfU5ZQQVBc=', 'Content-Length': '0'} 2022-09-03T06:50:16.424 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c846d83a25238caf-006312f928-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.425 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:16.425 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:16.426 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:16.427 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:16.427 DEBUG:boto:Method: GET 2022-09-03T06:50:16.428 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:50:16.428 DEBUG:boto:Data: 2022-09-03T06:50:16.429 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.429 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:16.430 DEBUG:boto:Port: 8001 2022-09-03T06:50:16.430 DEBUG:boto:Params: {} 2022-09-03T06:50:16.431 DEBUG:boto:Token: None 2022-09-03T06:50:16.432 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/ 2022-09-03T06:50:16.432 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sOo3A7inbLk2imqiirszYcz/dhs= 2022-09-03T06:50:16.433 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sOo3A7inbLk2imqiirszYcz/dhs=', 'Content-Length': '0'} 2022-09-03T06:50:16.436 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000089069d2e7e5a9309-006312f928-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.437 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:16.438 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:16.446 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:16.446 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:16.447 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:16.448 DEBUG:boto:Method: GET 2022-09-03T06:50:16.448 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:50:16.449 DEBUG:boto:Data: 2022-09-03T06:50:16.449 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.450 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:16.451 DEBUG:boto:Port: 8000 2022-09-03T06:50:16.451 DEBUG:boto:Params: {} 2022-09-03T06:50:16.452 DEBUG:boto:Token: None 2022-09-03T06:50:16.452 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/ 2022-09-03T06:50:16.453 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sOo3A7inbLk2imqiirszYcz/dhs= 2022-09-03T06:50:16.454 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sOo3A7inbLk2imqiirszYcz/dhs=', 'Content-Length': '0'} 2022-09-03T06:50:16.458 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a018dd25005c8d1a-006312f928-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.459 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:16.460 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2022-09-03T06:50:16.460 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:16.461 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:16.461 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.462 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.467 DEBUG:boto:Method: GET 2022-09-03T06:50:16.467 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.468 DEBUG:boto:Data: 2022-09-03T06:50:16.469 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.469 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:16.470 DEBUG:boto:Port: 8001 2022-09-03T06:50:16.474 DEBUG:boto:Params: {} 2022-09-03T06:50:16.474 DEBUG:boto:Token: None 2022-09-03T06:50:16.476 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.478 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5IyyN+aB2iwiPPDj154rbvpzvRA= 2022-09-03T06:50:16.479 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5IyyN+aB2iwiPPDj154rbvpzvRA=', 'Content-Length': '0'} 2022-09-03T06:50:16.482 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ec08283d66f6de85-006312f928-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.484 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:16.485 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:16.485 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.486 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.487 DEBUG:boto:Method: GET 2022-09-03T06:50:16.488 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.489 DEBUG:boto:Data: 2022-09-03T06:50:16.490 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.491 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:16.492 DEBUG:boto:Port: 8000 2022-09-03T06:50:16.493 DEBUG:boto:Params: {} 2022-09-03T06:50:16.494 DEBUG:boto:Token: None 2022-09-03T06:50:16.495 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.496 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5IyyN+aB2iwiPPDj154rbvpzvRA= 2022-09-03T06:50:16.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5IyyN+aB2iwiPPDj154rbvpzvRA=', 'Content-Length': '0'} 2022-09-03T06:50:16.500 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fce8ea0fcb1dae2e-006312f928-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.543 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:16.544 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:16.545 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.546 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.547 DEBUG:boto:Method: HEAD 2022-09-03T06:50:16.548 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.550 DEBUG:boto:Data: 2022-09-03T06:50:16.551 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.552 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:16.553 DEBUG:boto:Port: 8001 2022-09-03T06:50:16.553 DEBUG:boto:Params: {} 2022-09-03T06:50:16.554 DEBUG:boto:Token: None 2022-09-03T06:50:16.555 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.556 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q9Wn7HnGP2Q72E/795zrcO56vck= 2022-09-03T06:50:16.557 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q9Wn7HnGP2Q72E/795zrcO56vck=', 'Content-Length': '0'} 2022-09-03T06:50:16.561 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b7d6daf61644a24-006312f928-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.563 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:16.564 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:16.565 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.566 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.567 DEBUG:boto:Method: HEAD 2022-09-03T06:50:16.568 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.569 DEBUG:boto:Data: 2022-09-03T06:50:16.571 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.572 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:16.573 DEBUG:boto:Port: 8000 2022-09-03T06:50:16.573 DEBUG:boto:Params: {} 2022-09-03T06:50:16.574 DEBUG:boto:Token: None 2022-09-03T06:50:16.575 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.575 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Q9Wn7HnGP2Q72E/795zrcO56vck= 2022-09-03T06:50:16.576 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Q9Wn7HnGP2Q72E/795zrcO56vck=', 'Content-Length': '0'} 2022-09-03T06:50:16.579 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002b0663b534df20bf-006312f928-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.579 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2022-09-03T06:50:16.580 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:16.580 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:16.581 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.581 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.582 DEBUG:boto:Method: GET 2022-09-03T06:50:16.583 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.583 DEBUG:boto:Data: 2022-09-03T06:50:16.584 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.585 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:16.585 DEBUG:boto:Port: 8001 2022-09-03T06:50:16.586 DEBUG:boto:Params: {} 2022-09-03T06:50:16.587 DEBUG:boto:Token: None 2022-09-03T06:50:16.588 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.589 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5IyyN+aB2iwiPPDj154rbvpzvRA= 2022-09-03T06:50:16.590 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5IyyN+aB2iwiPPDj154rbvpzvRA=', 'Content-Length': '0'} 2022-09-03T06:50:16.593 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000af46cea46638aa07-006312f928-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.594 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:16.595 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:16.596 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.598 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.599 DEBUG:boto:Method: GET 2022-09-03T06:50:16.601 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.602 DEBUG:boto:Data: 2022-09-03T06:50:16.605 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.606 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:16.607 DEBUG:boto:Port: 8000 2022-09-03T06:50:16.608 DEBUG:boto:Params: {} 2022-09-03T06:50:16.610 DEBUG:boto:Token: None 2022-09-03T06:50:16.611 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:16.613 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5IyyN+aB2iwiPPDj154rbvpzvRA= 2022-09-03T06:50:16.614 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5IyyN+aB2iwiPPDj154rbvpzvRA=', 'Content-Length': '0'} 2022-09-03T06:50:16.618 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006c0d13566cae9194-006312f928-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.620 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:16.622 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:16.629 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:16.629 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:16.630 DEBUG:boto:Method: GET 2022-09-03T06:50:16.630 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:50:16.632 DEBUG:boto:Data: 2022-09-03T06:50:16.634 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.635 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:16.636 DEBUG:boto:Port: 8001 2022-09-03T06:50:16.637 DEBUG:boto:Params: {} 2022-09-03T06:50:16.638 DEBUG:boto:Token: None 2022-09-03T06:50:16.640 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/ 2022-09-03T06:50:16.641 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sOo3A7inbLk2imqiirszYcz/dhs= 2022-09-03T06:50:16.642 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sOo3A7inbLk2imqiirszYcz/dhs=', 'Content-Length': '0'} 2022-09-03T06:50:16.647 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004a03e81f64ea8a1f-006312f928-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.648 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:16.650 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:16.653 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:16.654 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:16.655 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:16.655 DEBUG:boto:Method: GET 2022-09-03T06:50:16.656 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:50:16.657 DEBUG:boto:Data: 2022-09-03T06:50:16.657 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.658 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:16.659 DEBUG:boto:Port: 8000 2022-09-03T06:50:16.659 DEBUG:boto:Params: {} 2022-09-03T06:50:16.660 DEBUG:boto:Token: None 2022-09-03T06:50:16.660 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/ 2022-09-03T06:50:16.661 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sOo3A7inbLk2imqiirszYcz/dhs= 2022-09-03T06:50:16.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sOo3A7inbLk2imqiirszYcz/dhs=', 'Content-Length': '0'} 2022-09-03T06:50:16.668 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000de0966a7fe4318cd-006312f928-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.710 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:16.711 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:16.712 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:16.712 DEBUG:boto:Method: HEAD 2022-09-03T06:50:16.713 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:50:16.713 DEBUG:boto:Data: 2022-09-03T06:50:16.714 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.714 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:16.715 DEBUG:boto:Port: 8001 2022-09-03T06:50:16.715 DEBUG:boto:Params: {} 2022-09-03T06:50:16.716 DEBUG:boto:Token: None 2022-09-03T06:50:16.717 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj31 2022-09-03T06:50:16.717 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:W5Afggrtf49jSLOBJuIRe2i4vms= 2022-09-03T06:50:16.718 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:W5Afggrtf49jSLOBJuIRe2i4vms=', 'Content-Length': '0'} 2022-09-03T06:50:16.720 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b359f0155ddd2bdf-006312f928-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.721 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:16.721 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:16.722 DEBUG:boto:Method: HEAD 2022-09-03T06:50:16.722 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:50:16.723 DEBUG:boto:Data: 2022-09-03T06:50:16.723 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.724 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:16.725 DEBUG:boto:Port: 8000 2022-09-03T06:50:16.725 DEBUG:boto:Params: {} 2022-09-03T06:50:16.726 DEBUG:boto:Token: None 2022-09-03T06:50:16.727 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj31 2022-09-03T06:50:16.727 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:W5Afggrtf49jSLOBJuIRe2i4vms= 2022-09-03T06:50:16.728 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:W5Afggrtf49jSLOBJuIRe2i4vms=', 'Content-Length': '0'} 2022-09-03T06:50:16.730 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000091dab84d958b500d-006312f928-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.731 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2022-09-03T06:50:16.732 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:16.732 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:16.733 DEBUG:boto:Method: GET 2022-09-03T06:50:16.734 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:50:16.735 DEBUG:boto:Data: 2022-09-03T06:50:16.735 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.736 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:16.736 DEBUG:boto:Port: 8001 2022-09-03T06:50:16.737 DEBUG:boto:Params: {} 2022-09-03T06:50:16.738 DEBUG:boto:Token: None 2022-09-03T06:50:16.739 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj31 2022-09-03T06:50:16.740 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tBSAttV9mqWGweZInj3FZ5UoS0w= 2022-09-03T06:50:16.741 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tBSAttV9mqWGweZInj3FZ5UoS0w=', 'Content-Length': '0'} 2022-09-03T06:50:16.743 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048436c9fe41496f8-006312f928-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.745 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:16.746 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:16.747 DEBUG:boto:Method: GET 2022-09-03T06:50:16.748 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:50:16.749 DEBUG:boto:Data: 2022-09-03T06:50:16.750 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.750 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:16.751 DEBUG:boto:Port: 8000 2022-09-03T06:50:16.752 DEBUG:boto:Params: {} 2022-09-03T06:50:16.753 DEBUG:boto:Token: None 2022-09-03T06:50:16.754 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj31 2022-09-03T06:50:16.754 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tBSAttV9mqWGweZInj3FZ5UoS0w= 2022-09-03T06:50:16.755 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tBSAttV9mqWGweZInj3FZ5UoS0w=', 'Content-Length': '0'} 2022-09-03T06:50:16.757 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ebd688a173f1e73a-006312f928-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.758 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:16.759 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:16.760 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:16.760 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:16.761 DEBUG:boto:Method: GET 2022-09-03T06:50:16.761 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:50:16.762 DEBUG:boto:Data: 2022-09-03T06:50:16.763 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.764 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:16.766 DEBUG:boto:Port: 8001 2022-09-03T06:50:16.768 DEBUG:boto:Params: {} 2022-09-03T06:50:16.769 DEBUG:boto:Token: None 2022-09-03T06:50:16.771 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/ 2022-09-03T06:50:16.772 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sOo3A7inbLk2imqiirszYcz/dhs= 2022-09-03T06:50:16.773 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sOo3A7inbLk2imqiirszYcz/dhs=', 'Content-Length': '0'} 2022-09-03T06:50:16.780 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000545168a608168f1-006312f928-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.818 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:16.820 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:16.821 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:16.822 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:16.823 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:16.824 DEBUG:boto:Method: GET 2022-09-03T06:50:16.827 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:50:16.828 DEBUG:boto:Data: 2022-09-03T06:50:16.830 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.831 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:16.832 DEBUG:boto:Port: 8000 2022-09-03T06:50:16.833 DEBUG:boto:Params: {} 2022-09-03T06:50:16.834 DEBUG:boto:Token: None 2022-09-03T06:50:16.836 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/ 2022-09-03T06:50:16.837 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sOo3A7inbLk2imqiirszYcz/dhs= 2022-09-03T06:50:16.838 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sOo3A7inbLk2imqiirszYcz/dhs=', 'Content-Length': '0'} 2022-09-03T06:50:16.844 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000033a79c133351d06b-006312f928-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.845 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:16.847 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:50:16.848 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:16.849 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:16.850 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.851 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.853 DEBUG:boto:Method: GET 2022-09-03T06:50:16.854 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.855 DEBUG:boto:Data: 2022-09-03T06:50:16.856 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.857 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:16.858 DEBUG:boto:Port: 8001 2022-09-03T06:50:16.859 DEBUG:boto:Params: {} 2022-09-03T06:50:16.860 DEBUG:boto:Token: None 2022-09-03T06:50:16.861 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.862 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xPXaDKjDDtAG8Z01Z0elAEN8Aug= 2022-09-03T06:50:16.863 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xPXaDKjDDtAG8Z01Z0elAEN8Aug=', 'Content-Length': '0'} 2022-09-03T06:50:16.872 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000017e0a7cedccfc8fa-006312f928-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.873 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:16.874 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:16.874 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.875 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.876 DEBUG:boto:Method: GET 2022-09-03T06:50:16.876 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.877 DEBUG:boto:Data: 2022-09-03T06:50:16.877 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.878 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:16.879 DEBUG:boto:Port: 8000 2022-09-03T06:50:16.879 DEBUG:boto:Params: {} 2022-09-03T06:50:16.880 DEBUG:boto:Token: None 2022-09-03T06:50:16.881 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.882 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xPXaDKjDDtAG8Z01Z0elAEN8Aug= 2022-09-03T06:50:16.883 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xPXaDKjDDtAG8Z01Z0elAEN8Aug=', 'Content-Length': '0'} 2022-09-03T06:50:16.888 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000034e27948ad8823a5-006312f928-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.889 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:16.890 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:16.890 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.891 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.892 DEBUG:boto:Method: HEAD 2022-09-03T06:50:16.892 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.893 DEBUG:boto:Data: 2022-09-03T06:50:16.894 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.894 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:16.895 DEBUG:boto:Port: 8001 2022-09-03T06:50:16.895 DEBUG:boto:Params: {} 2022-09-03T06:50:16.896 DEBUG:boto:Token: None 2022-09-03T06:50:16.896 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.897 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/jZyfYrgzuajvwMDi5wiTv+uPvI= 2022-09-03T06:50:16.898 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/jZyfYrgzuajvwMDi5wiTv+uPvI=', 'Content-Length': '0'} 2022-09-03T06:50:16.901 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008731570853ce9686-006312f928-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.903 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:16.903 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:16.904 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.906 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.907 DEBUG:boto:Method: HEAD 2022-09-03T06:50:16.908 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.908 DEBUG:boto:Data: 2022-09-03T06:50:16.909 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.910 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:16.911 DEBUG:boto:Port: 8000 2022-09-03T06:50:16.912 DEBUG:boto:Params: {} 2022-09-03T06:50:16.913 DEBUG:boto:Token: None 2022-09-03T06:50:16.914 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.915 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/jZyfYrgzuajvwMDi5wiTv+uPvI= 2022-09-03T06:50:16.917 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/jZyfYrgzuajvwMDi5wiTv+uPvI=', 'Content-Length': '0'} 2022-09-03T06:50:16.920 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fe84f12f99dd1f15-006312f928-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.921 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:50:16.922 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:16.923 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:16.923 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.924 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.924 DEBUG:boto:Method: GET 2022-09-03T06:50:16.925 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.925 DEBUG:boto:Data: 2022-09-03T06:50:16.926 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.931 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:16.932 DEBUG:boto:Port: 8001 2022-09-03T06:50:16.933 DEBUG:boto:Params: {} 2022-09-03T06:50:16.935 DEBUG:boto:Token: None 2022-09-03T06:50:16.936 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.938 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xPXaDKjDDtAG8Z01Z0elAEN8Aug= 2022-09-03T06:50:16.939 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xPXaDKjDDtAG8Z01Z0elAEN8Aug=', 'Content-Length': '0'} 2022-09-03T06:50:16.942 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b9965e16979f818e-006312f928-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:16.943 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:16.944 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:16.946 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.947 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.948 DEBUG:boto:Method: GET 2022-09-03T06:50:16.949 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.950 DEBUG:boto:Data: 2022-09-03T06:50:16.951 DEBUG:boto:Headers: {} 2022-09-03T06:50:16.952 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:16.953 DEBUG:boto:Port: 8000 2022-09-03T06:50:16.954 DEBUG:boto:Params: {} 2022-09-03T06:50:16.956 DEBUG:boto:Token: None 2022-09-03T06:50:16.957 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:16 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:16.958 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xPXaDKjDDtAG8Z01Z0elAEN8Aug= 2022-09-03T06:50:16.959 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:16 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xPXaDKjDDtAG8Z01Z0elAEN8Aug=', 'Content-Length': '0'} 2022-09-03T06:50:16.962 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b4ec7f155167837c-006312f928-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.010 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:17.011 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:17.012 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:17.013 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:17.014 DEBUG:boto:Method: GET 2022-09-03T06:50:17.015 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:50:17.015 DEBUG:boto:Data: 2022-09-03T06:50:17.016 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.018 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.018 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.019 DEBUG:boto:Params: {} 2022-09-03T06:50:17.020 DEBUG:boto:Token: None 2022-09-03T06:50:17.021 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/ 2022-09-03T06:50:17.022 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8= 2022-09-03T06:50:17.023 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8=', 'Content-Length': '0'} 2022-09-03T06:50:17.026 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000023ffa308b853222d-006312f929-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.027 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:17.028 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:17.028 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:17.029 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:17.029 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:17.030 DEBUG:boto:Method: GET 2022-09-03T06:50:17.030 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:50:17.031 DEBUG:boto:Data: 2022-09-03T06:50:17.032 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.032 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.033 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.033 DEBUG:boto:Params: {} 2022-09-03T06:50:17.034 DEBUG:boto:Token: None 2022-09-03T06:50:17.035 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/ 2022-09-03T06:50:17.035 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8= 2022-09-03T06:50:17.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8=', 'Content-Length': '0'} 2022-09-03T06:50:17.043 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ed465fcb996098d7-006312f929-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.048 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:17.049 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:17.051 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:17.052 DEBUG:boto:Method: HEAD 2022-09-03T06:50:17.052 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:50:17.053 DEBUG:boto:Data: 2022-09-03T06:50:17.054 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.055 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.056 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.059 DEBUG:boto:Params: {} 2022-09-03T06:50:17.060 DEBUG:boto:Token: None 2022-09-03T06:50:17.061 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj33 2022-09-03T06:50:17.061 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RD7yYcPvePl9yP7UzOCjrBxM5Tg= 2022-09-03T06:50:17.062 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RD7yYcPvePl9yP7UzOCjrBxM5Tg=', 'Content-Length': '0'} 2022-09-03T06:50:17.064 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d398ca2bd9a25e40-006312f929-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.065 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:17.065 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:17.066 DEBUG:boto:Method: HEAD 2022-09-03T06:50:17.066 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:50:17.067 DEBUG:boto:Data: 2022-09-03T06:50:17.067 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.068 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.069 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.069 DEBUG:boto:Params: {} 2022-09-03T06:50:17.070 DEBUG:boto:Token: None 2022-09-03T06:50:17.071 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj33 2022-09-03T06:50:17.072 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:RD7yYcPvePl9yP7UzOCjrBxM5Tg= 2022-09-03T06:50:17.072 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:RD7yYcPvePl9yP7UzOCjrBxM5Tg=', 'Content-Length': '0'} 2022-09-03T06:50:17.075 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c1ecec22cb5540e0-006312f929-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.076 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:50:17.076 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:17.077 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:17.078 DEBUG:boto:Method: GET 2022-09-03T06:50:17.078 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:50:17.079 DEBUG:boto:Data: 2022-09-03T06:50:17.079 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.080 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.081 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.082 DEBUG:boto:Params: {} 2022-09-03T06:50:17.082 DEBUG:boto:Token: None 2022-09-03T06:50:17.083 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj33 2022-09-03T06:50:17.084 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H7Hx4rgRHS0zPdJUMgjRmt6Q8E8= 2022-09-03T06:50:17.085 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H7Hx4rgRHS0zPdJUMgjRmt6Q8E8=', 'Content-Length': '0'} 2022-09-03T06:50:17.087 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fc666ccfdf4db928-006312f929-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.088 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:17.088 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:17.089 DEBUG:boto:Method: GET 2022-09-03T06:50:17.090 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:50:17.091 DEBUG:boto:Data: 2022-09-03T06:50:17.091 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.092 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.092 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.093 DEBUG:boto:Params: {} 2022-09-03T06:50:17.094 DEBUG:boto:Token: None 2022-09-03T06:50:17.095 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj33 2022-09-03T06:50:17.096 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H7Hx4rgRHS0zPdJUMgjRmt6Q8E8= 2022-09-03T06:50:17.097 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H7Hx4rgRHS0zPdJUMgjRmt6Q8E8=', 'Content-Length': '0'} 2022-09-03T06:50:17.103 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000075817ef4bb67d298-006312f929-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.142 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:17.144 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:17.145 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:17.146 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:17.147 DEBUG:boto:Method: GET 2022-09-03T06:50:17.148 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:50:17.150 DEBUG:boto:Data: 2022-09-03T06:50:17.151 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.152 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.162 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.163 DEBUG:boto:Params: {} 2022-09-03T06:50:17.164 DEBUG:boto:Token: None 2022-09-03T06:50:17.166 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/ 2022-09-03T06:50:17.167 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8= 2022-09-03T06:50:17.168 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8=', 'Content-Length': '0'} 2022-09-03T06:50:17.172 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000084fa83e2a1e95aea-006312f929-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.174 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:17.176 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:17.177 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:17.178 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:17.179 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:17.180 DEBUG:boto:Method: GET 2022-09-03T06:50:17.181 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:50:17.182 DEBUG:boto:Data: 2022-09-03T06:50:17.183 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.184 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.185 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.186 DEBUG:boto:Params: {} 2022-09-03T06:50:17.187 DEBUG:boto:Token: None 2022-09-03T06:50:17.188 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/ 2022-09-03T06:50:17.189 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8= 2022-09-03T06:50:17.190 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8=', 'Content-Length': '0'} 2022-09-03T06:50:17.195 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c08ad1e4318bade4-006312f929-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.196 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:17.197 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:50:17.197 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:17.198 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:17.198 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.199 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.199 DEBUG:boto:Method: GET 2022-09-03T06:50:17.200 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.201 DEBUG:boto:Data: 2022-09-03T06:50:17.201 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.202 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.203 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.204 DEBUG:boto:Params: {} 2022-09-03T06:50:17.205 DEBUG:boto:Token: None 2022-09-03T06:50:17.206 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.206 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CdnHc50jKxrIsxSi1TOBpF/ctRo= 2022-09-03T06:50:17.207 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CdnHc50jKxrIsxSi1TOBpF/ctRo=', 'Content-Length': '0'} 2022-09-03T06:50:17.209 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009ff03a19a92e891d-006312f929-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.210 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:17.211 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:17.211 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.212 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.213 DEBUG:boto:Method: GET 2022-09-03T06:50:17.213 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.214 DEBUG:boto:Data: 2022-09-03T06:50:17.215 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.215 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.216 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.216 DEBUG:boto:Params: {} 2022-09-03T06:50:17.217 DEBUG:boto:Token: None 2022-09-03T06:50:17.218 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.218 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CdnHc50jKxrIsxSi1TOBpF/ctRo= 2022-09-03T06:50:17.219 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CdnHc50jKxrIsxSi1TOBpF/ctRo=', 'Content-Length': '0'} 2022-09-03T06:50:17.222 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c90ebee8a9f7876d-006312f929-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.222 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:17.223 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:17.224 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.225 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.226 DEBUG:boto:Method: HEAD 2022-09-03T06:50:17.227 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.228 DEBUG:boto:Data: 2022-09-03T06:50:17.229 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.230 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.231 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.231 DEBUG:boto:Params: {} 2022-09-03T06:50:17.232 DEBUG:boto:Token: None 2022-09-03T06:50:17.233 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.233 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uH9ozDvG59n4duUcodkURtwNJDs= 2022-09-03T06:50:17.234 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uH9ozDvG59n4duUcodkURtwNJDs=', 'Content-Length': '0'} 2022-09-03T06:50:17.236 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d242552938c442a8-006312f929-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.238 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:17.238 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:17.239 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.239 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.240 DEBUG:boto:Method: HEAD 2022-09-03T06:50:17.240 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.241 DEBUG:boto:Data: 2022-09-03T06:50:17.242 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.242 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.243 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.244 DEBUG:boto:Params: {} 2022-09-03T06:50:17.245 DEBUG:boto:Token: None 2022-09-03T06:50:17.246 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.247 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:uH9ozDvG59n4duUcodkURtwNJDs= 2022-09-03T06:50:17.248 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:uH9ozDvG59n4duUcodkURtwNJDs=', 'Content-Length': '0'} 2022-09-03T06:50:17.251 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000da0fd22921216ca9-006312f929-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.252 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:50:17.253 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:17.254 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:17.256 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.257 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.257 DEBUG:boto:Method: GET 2022-09-03T06:50:17.258 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.259 DEBUG:boto:Data: 2022-09-03T06:50:17.259 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.260 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.260 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.261 DEBUG:boto:Params: {} 2022-09-03T06:50:17.261 DEBUG:boto:Token: None 2022-09-03T06:50:17.262 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.263 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CdnHc50jKxrIsxSi1TOBpF/ctRo= 2022-09-03T06:50:17.263 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CdnHc50jKxrIsxSi1TOBpF/ctRo=', 'Content-Length': '0'} 2022-09-03T06:50:17.265 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000063da77dca87e29e7-006312f929-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.266 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:17.267 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:17.267 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.268 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.268 DEBUG:boto:Method: GET 2022-09-03T06:50:17.269 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.270 DEBUG:boto:Data: 2022-09-03T06:50:17.270 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.271 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.271 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.272 DEBUG:boto:Params: {} 2022-09-03T06:50:17.272 DEBUG:boto:Token: None 2022-09-03T06:50:17.273 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:17.273 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CdnHc50jKxrIsxSi1TOBpF/ctRo= 2022-09-03T06:50:17.274 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CdnHc50jKxrIsxSi1TOBpF/ctRo=', 'Content-Length': '0'} 2022-09-03T06:50:17.276 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000044144ffa12cf65e-006312f929-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.277 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:17.278 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:17.279 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:17.279 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:17.280 DEBUG:boto:Method: GET 2022-09-03T06:50:17.280 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:50:17.281 DEBUG:boto:Data: 2022-09-03T06:50:17.281 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.282 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.282 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.283 DEBUG:boto:Params: {} 2022-09-03T06:50:17.283 DEBUG:boto:Token: None 2022-09-03T06:50:17.284 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/ 2022-09-03T06:50:17.285 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8= 2022-09-03T06:50:17.285 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8=', 'Content-Length': '0'} 2022-09-03T06:50:17.288 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ed3e7b6e8d3cfeed-006312f929-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.289 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:17.290 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:17.290 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:17.291 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:17.292 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:17.292 DEBUG:boto:Method: GET 2022-09-03T06:50:17.293 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:50:17.293 DEBUG:boto:Data: 2022-09-03T06:50:17.294 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.294 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.295 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.295 DEBUG:boto:Params: {} 2022-09-03T06:50:17.296 DEBUG:boto:Token: None 2022-09-03T06:50:17.297 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/ 2022-09-03T06:50:17.297 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8= 2022-09-03T06:50:17.298 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8=', 'Content-Length': '0'} 2022-09-03T06:50:17.302 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007fc95423f64289b9-006312f929-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.303 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:17.304 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:17.305 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:17.305 DEBUG:boto:Method: HEAD 2022-09-03T06:50:17.306 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:50:17.306 DEBUG:boto:Data: 2022-09-03T06:50:17.307 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.307 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.308 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.309 DEBUG:boto:Params: {} 2022-09-03T06:50:17.309 DEBUG:boto:Token: None 2022-09-03T06:50:17.310 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj34 2022-09-03T06:50:17.310 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xmE19IlvCjdGuVGz3eP9Xl4sXNM= 2022-09-03T06:50:17.311 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xmE19IlvCjdGuVGz3eP9Xl4sXNM=', 'Content-Length': '0'} 2022-09-03T06:50:17.313 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a210568034ff346c-006312f929-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.314 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:17.315 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:17.316 DEBUG:boto:Method: HEAD 2022-09-03T06:50:17.316 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:50:17.317 DEBUG:boto:Data: 2022-09-03T06:50:17.318 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.318 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.319 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.319 DEBUG:boto:Params: {} 2022-09-03T06:50:17.320 DEBUG:boto:Token: None 2022-09-03T06:50:17.320 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj34 2022-09-03T06:50:17.321 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xmE19IlvCjdGuVGz3eP9Xl4sXNM= 2022-09-03T06:50:17.322 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xmE19IlvCjdGuVGz3eP9Xl4sXNM=', 'Content-Length': '0'} 2022-09-03T06:50:17.325 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c5ed58ae2b0007ce-006312f929-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.325 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:50:17.326 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:17.327 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:17.328 DEBUG:boto:Method: GET 2022-09-03T06:50:17.328 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:50:17.329 DEBUG:boto:Data: 2022-09-03T06:50:17.330 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.330 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.331 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.331 DEBUG:boto:Params: {} 2022-09-03T06:50:17.332 DEBUG:boto:Token: None 2022-09-03T06:50:17.333 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj34 2022-09-03T06:50:17.333 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hyixbt00/4YoxVBYMvSRQ9rOg/k= 2022-09-03T06:50:17.334 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hyixbt00/4YoxVBYMvSRQ9rOg/k=', 'Content-Length': '0'} 2022-09-03T06:50:17.336 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f38d95d3da9c973d-006312f929-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.337 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:17.338 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:17.338 DEBUG:boto:Method: GET 2022-09-03T06:50:17.339 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:50:17.339 DEBUG:boto:Data: 2022-09-03T06:50:17.340 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.342 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.342 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.343 DEBUG:boto:Params: {} 2022-09-03T06:50:17.343 DEBUG:boto:Token: None 2022-09-03T06:50:17.344 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj34 2022-09-03T06:50:17.345 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hyixbt00/4YoxVBYMvSRQ9rOg/k= 2022-09-03T06:50:17.345 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hyixbt00/4YoxVBYMvSRQ9rOg/k=', 'Content-Length': '0'} 2022-09-03T06:50:17.347 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f741985809eee131-006312f929-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.390 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:17.391 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:17.391 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:17.392 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:17.393 DEBUG:boto:Method: GET 2022-09-03T06:50:17.393 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:50:17.394 DEBUG:boto:Data: 2022-09-03T06:50:17.394 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.395 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.396 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.396 DEBUG:boto:Params: {} 2022-09-03T06:50:17.397 DEBUG:boto:Token: None 2022-09-03T06:50:17.397 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/ 2022-09-03T06:50:17.398 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8= 2022-09-03T06:50:17.399 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8=', 'Content-Length': '0'} 2022-09-03T06:50:17.402 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006de0d565503258ca-006312f929-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.402 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:17.403 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:17.404 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:17.405 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:17.405 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:17.406 DEBUG:boto:Method: GET 2022-09-03T06:50:17.406 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:50:17.407 DEBUG:boto:Data: 2022-09-03T06:50:17.407 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.408 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.408 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.409 DEBUG:boto:Params: {} 2022-09-03T06:50:17.410 DEBUG:boto:Token: None 2022-09-03T06:50:17.410 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/ 2022-09-03T06:50:17.411 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8= 2022-09-03T06:50:17.411 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8=', 'Content-Length': '0'} 2022-09-03T06:50:17.416 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001b2e97b8b31be402-006312f929-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.417 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:17.418 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj37 2022-09-03T06:50:17.418 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:17.419 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:17.419 DEBUG:boto:path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.420 DEBUG:boto:auth_path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.420 DEBUG:boto:Method: GET 2022-09-03T06:50:17.421 DEBUG:boto:Path: /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.421 DEBUG:boto:Data: 2022-09-03T06:50:17.422 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.423 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.423 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.425 DEBUG:boto:Params: {} 2022-09-03T06:50:17.425 DEBUG:boto:Token: None 2022-09-03T06:50:17.426 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.427 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rmov8lv7eUpCglllxxQpwKHh1KM= 2022-09-03T06:50:17.427 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rmov8lv7eUpCglllxxQpwKHh1KM=', 'Content-Length': '0'} 2022-09-03T06:50:17.430 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000efe030c78fc7f85a-006312f929-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.431 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:17.431 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:17.432 DEBUG:boto:path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.432 DEBUG:boto:auth_path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.433 DEBUG:boto:Method: GET 2022-09-03T06:50:17.434 DEBUG:boto:Path: /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.435 DEBUG:boto:Data: 2022-09-03T06:50:17.436 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.437 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.438 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.439 DEBUG:boto:Params: {} 2022-09-03T06:50:17.440 DEBUG:boto:Token: None 2022-09-03T06:50:17.441 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.443 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rmov8lv7eUpCglllxxQpwKHh1KM= 2022-09-03T06:50:17.444 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rmov8lv7eUpCglllxxQpwKHh1KM=', 'Content-Length': '0'} 2022-09-03T06:50:17.446 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a0206086dac36bd0-006312f929-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.447 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:17.448 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:17.448 DEBUG:boto:path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.449 DEBUG:boto:auth_path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.449 DEBUG:boto:Method: HEAD 2022-09-03T06:50:17.450 DEBUG:boto:Path: /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.450 DEBUG:boto:Data: 2022-09-03T06:50:17.451 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.451 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.452 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.453 DEBUG:boto:Params: {} 2022-09-03T06:50:17.453 DEBUG:boto:Token: None 2022-09-03T06:50:17.454 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.455 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mKYAtZAjxMObStTie3SAJgukteM= 2022-09-03T06:50:17.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mKYAtZAjxMObStTie3SAJgukteM=', 'Content-Length': '0'} 2022-09-03T06:50:17.459 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000265b8296ea47af4a-006312f929-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.460 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:17.461 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:17.461 DEBUG:boto:path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.462 DEBUG:boto:auth_path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.462 DEBUG:boto:Method: HEAD 2022-09-03T06:50:17.463 DEBUG:boto:Path: /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.464 DEBUG:boto:Data: 2022-09-03T06:50:17.465 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.466 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.466 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.467 DEBUG:boto:Params: {} 2022-09-03T06:50:17.467 DEBUG:boto:Token: None 2022-09-03T06:50:17.468 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.469 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mKYAtZAjxMObStTie3SAJgukteM= 2022-09-03T06:50:17.470 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mKYAtZAjxMObStTie3SAJgukteM=', 'Content-Length': '0'} 2022-09-03T06:50:17.473 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eb8067ed48501534-006312f929-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.473 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj37 2022-09-03T06:50:17.474 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:17.475 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:17.475 DEBUG:boto:path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.476 DEBUG:boto:auth_path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.477 DEBUG:boto:Method: GET 2022-09-03T06:50:17.478 DEBUG:boto:Path: /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.479 DEBUG:boto:Data: 2022-09-03T06:50:17.480 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.481 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.482 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.483 DEBUG:boto:Params: {} 2022-09-03T06:50:17.484 DEBUG:boto:Token: None 2022-09-03T06:50:17.485 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.486 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rmov8lv7eUpCglllxxQpwKHh1KM= 2022-09-03T06:50:17.487 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rmov8lv7eUpCglllxxQpwKHh1KM=', 'Content-Length': '0'} 2022-09-03T06:50:17.493 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003728c5ea4f10f129-006312f929-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.536 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:17.537 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:17.538 DEBUG:boto:path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.540 DEBUG:boto:auth_path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.541 DEBUG:boto:Method: GET 2022-09-03T06:50:17.541 DEBUG:boto:Path: /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.542 DEBUG:boto:Data: 2022-09-03T06:50:17.543 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.543 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.544 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.545 DEBUG:boto:Params: {} 2022-09-03T06:50:17.546 DEBUG:boto:Token: None 2022-09-03T06:50:17.546 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:17.547 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:rmov8lv7eUpCglllxxQpwKHh1KM= 2022-09-03T06:50:17.548 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:rmov8lv7eUpCglllxxQpwKHh1KM=', 'Content-Length': '0'} 2022-09-03T06:50:17.550 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000007637dee8c33bbab-006312f929-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.552 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:17.553 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:17.554 DEBUG:boto:path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:17.555 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:17.556 DEBUG:boto:Method: GET 2022-09-03T06:50:17.556 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj37 2022-09-03T06:50:17.557 DEBUG:boto:Data: 2022-09-03T06:50:17.558 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.558 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.559 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.559 DEBUG:boto:Params: {} 2022-09-03T06:50:17.560 DEBUG:boto:Token: None 2022-09-03T06:50:17.561 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/ 2022-09-03T06:50:17.562 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8= 2022-09-03T06:50:17.564 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8=', 'Content-Length': '0'} 2022-09-03T06:50:17.570 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004e8b0baadb2b3797-006312f929-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.571 DEBUG:boto:b'dsgvcy-75obj371000falseobj372022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:17.572 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:17.574 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:17.574 DEBUG:boto:path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:17.575 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:17.576 DEBUG:boto:Method: GET 2022-09-03T06:50:17.576 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj37 2022-09-03T06:50:17.577 DEBUG:boto:Data: 2022-09-03T06:50:17.577 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.578 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.579 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.579 DEBUG:boto:Params: {} 2022-09-03T06:50:17.580 DEBUG:boto:Token: None 2022-09-03T06:50:17.580 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/ 2022-09-03T06:50:17.581 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8= 2022-09-03T06:50:17.582 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8=', 'Content-Length': '0'} 2022-09-03T06:50:17.587 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b5e1605fe779c380-006312f929-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.588 DEBUG:boto:b'dsgvcy-75obj371000falseobj372022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:17.588 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:17.589 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:17.590 DEBUG:boto:Method: HEAD 2022-09-03T06:50:17.590 DEBUG:boto:Path: /dsgvcy-75/obj37 2022-09-03T06:50:17.591 DEBUG:boto:Data: 2022-09-03T06:50:17.591 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.592 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.592 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.593 DEBUG:boto:Params: {} 2022-09-03T06:50:17.593 DEBUG:boto:Token: None 2022-09-03T06:50:17.594 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj37 2022-09-03T06:50:17.595 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AI1zUtocRcLXGE54hdUtajxI6sI= 2022-09-03T06:50:17.595 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AI1zUtocRcLXGE54hdUtajxI6sI=', 'Content-Length': '0'} 2022-09-03T06:50:17.597 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b7fb03dd218c406f-006312f929-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.598 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:17.599 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:17.599 DEBUG:boto:Method: HEAD 2022-09-03T06:50:17.600 DEBUG:boto:Path: /dsgvcy-75/obj37 2022-09-03T06:50:17.601 DEBUG:boto:Data: 2022-09-03T06:50:17.601 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.602 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.602 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.603 DEBUG:boto:Params: {} 2022-09-03T06:50:17.603 DEBUG:boto:Token: None 2022-09-03T06:50:17.604 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj37 2022-09-03T06:50:17.604 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:AI1zUtocRcLXGE54hdUtajxI6sI= 2022-09-03T06:50:17.605 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:AI1zUtocRcLXGE54hdUtajxI6sI=', 'Content-Length': '0'} 2022-09-03T06:50:17.607 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000005da92438a0719aa-006312f929-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.608 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj37 2022-09-03T06:50:17.609 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:17.610 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:17.610 DEBUG:boto:Method: GET 2022-09-03T06:50:17.611 DEBUG:boto:Path: /dsgvcy-75/obj37 2022-09-03T06:50:17.611 DEBUG:boto:Data: 2022-09-03T06:50:17.612 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.613 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.613 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.614 DEBUG:boto:Params: {} 2022-09-03T06:50:17.614 DEBUG:boto:Token: None 2022-09-03T06:50:17.615 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj37 2022-09-03T06:50:17.616 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zCR20AjtZCkSTqdfZd41ie8hS1A= 2022-09-03T06:50:17.616 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zCR20AjtZCkSTqdfZd41ie8hS1A=', 'Content-Length': '0'} 2022-09-03T06:50:17.622 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000022678c4d6eba9bd7-006312f929-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.624 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:17.625 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:17.626 DEBUG:boto:Method: GET 2022-09-03T06:50:17.627 DEBUG:boto:Path: /dsgvcy-75/obj37 2022-09-03T06:50:17.628 DEBUG:boto:Data: 2022-09-03T06:50:17.629 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.630 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.630 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.631 DEBUG:boto:Params: {} 2022-09-03T06:50:17.631 DEBUG:boto:Token: None 2022-09-03T06:50:17.632 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj37 2022-09-03T06:50:17.633 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:zCR20AjtZCkSTqdfZd41ie8hS1A= 2022-09-03T06:50:17.634 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:zCR20AjtZCkSTqdfZd41ie8hS1A=', 'Content-Length': '0'} 2022-09-03T06:50:17.636 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000550dbaed9ca0eefd-006312f929-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.637 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:17.638 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:17.639 DEBUG:boto:path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:17.639 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:17.640 DEBUG:boto:Method: GET 2022-09-03T06:50:17.641 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj37 2022-09-03T06:50:17.641 DEBUG:boto:Data: 2022-09-03T06:50:17.642 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.643 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.643 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.644 DEBUG:boto:Params: {} 2022-09-03T06:50:17.645 DEBUG:boto:Token: None 2022-09-03T06:50:17.645 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/ 2022-09-03T06:50:17.646 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8= 2022-09-03T06:50:17.647 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8=', 'Content-Length': '0'} 2022-09-03T06:50:17.650 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ad41ecd4d142337a-006312f929-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.694 DEBUG:boto:b'dsgvcy-75obj371000falseobj372022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:17.695 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:17.696 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:17.697 DEBUG:boto:path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:17.697 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:17.698 DEBUG:boto:Method: GET 2022-09-03T06:50:17.699 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj37 2022-09-03T06:50:17.700 DEBUG:boto:Data: 2022-09-03T06:50:17.701 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.702 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.703 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.704 DEBUG:boto:Params: {} 2022-09-03T06:50:17.705 DEBUG:boto:Token: None 2022-09-03T06:50:17.706 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/ 2022-09-03T06:50:17.707 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8= 2022-09-03T06:50:17.708 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8=', 'Content-Length': '0'} 2022-09-03T06:50:17.714 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004b728070e6d447a6-006312f929-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.715 DEBUG:boto:b'dsgvcy-75obj371000falseobj372022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:17.716 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:50:17.717 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:17.717 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:17.718 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.718 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.719 DEBUG:boto:Method: GET 2022-09-03T06:50:17.720 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.720 DEBUG:boto:Data: 2022-09-03T06:50:17.721 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.721 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.722 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.723 DEBUG:boto:Params: {} 2022-09-03T06:50:17.723 DEBUG:boto:Token: None 2022-09-03T06:50:17.724 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.725 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LPaWknIJkpUWlnkqMWjBfC3C7F4= 2022-09-03T06:50:17.726 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LPaWknIJkpUWlnkqMWjBfC3C7F4=', 'Content-Length': '0'} 2022-09-03T06:50:17.728 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007227ffc9f7760bba-006312f929-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.729 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:17.730 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:17.730 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.731 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.731 DEBUG:boto:Method: GET 2022-09-03T06:50:17.732 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.732 DEBUG:boto:Data: 2022-09-03T06:50:17.733 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.733 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.734 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.734 DEBUG:boto:Params: {} 2022-09-03T06:50:17.735 DEBUG:boto:Token: None 2022-09-03T06:50:17.736 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.736 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LPaWknIJkpUWlnkqMWjBfC3C7F4= 2022-09-03T06:50:17.737 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LPaWknIJkpUWlnkqMWjBfC3C7F4=', 'Content-Length': '0'} 2022-09-03T06:50:17.739 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009edb26b0902830a7-006312f929-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.786 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:17.787 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:17.787 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.788 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.788 DEBUG:boto:Method: HEAD 2022-09-03T06:50:17.789 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.789 DEBUG:boto:Data: 2022-09-03T06:50:17.790 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.791 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.791 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.792 DEBUG:boto:Params: {} 2022-09-03T06:50:17.792 DEBUG:boto:Token: None 2022-09-03T06:50:17.793 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.794 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PV5Zm9DOrWVkZ6OaKAC3R5S1Tws= 2022-09-03T06:50:17.794 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PV5Zm9DOrWVkZ6OaKAC3R5S1Tws=', 'Content-Length': '0'} 2022-09-03T06:50:17.797 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007053be78d2aeb570-006312f929-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.798 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:17.798 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:17.799 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.799 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.800 DEBUG:boto:Method: HEAD 2022-09-03T06:50:17.800 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.801 DEBUG:boto:Data: 2022-09-03T06:50:17.801 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.802 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.802 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.803 DEBUG:boto:Params: {} 2022-09-03T06:50:17.803 DEBUG:boto:Token: None 2022-09-03T06:50:17.804 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.805 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PV5Zm9DOrWVkZ6OaKAC3R5S1Tws= 2022-09-03T06:50:17.805 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PV5Zm9DOrWVkZ6OaKAC3R5S1Tws=', 'Content-Length': '0'} 2022-09-03T06:50:17.808 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000089069af87d384d6d-006312f929-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.809 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:50:17.809 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:17.810 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:17.811 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.811 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.812 DEBUG:boto:Method: GET 2022-09-03T06:50:17.812 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.813 DEBUG:boto:Data: 2022-09-03T06:50:17.813 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.814 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.814 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.815 DEBUG:boto:Params: {} 2022-09-03T06:50:17.816 DEBUG:boto:Token: None 2022-09-03T06:50:17.816 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.817 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LPaWknIJkpUWlnkqMWjBfC3C7F4= 2022-09-03T06:50:17.818 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LPaWknIJkpUWlnkqMWjBfC3C7F4=', 'Content-Length': '0'} 2022-09-03T06:50:17.820 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000088b6d3f9135408c0-006312f929-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.821 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:17.822 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:17.822 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.823 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.824 DEBUG:boto:Method: GET 2022-09-03T06:50:17.824 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.825 DEBUG:boto:Data: 2022-09-03T06:50:17.825 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.826 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.827 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.827 DEBUG:boto:Params: {} 2022-09-03T06:50:17.828 DEBUG:boto:Token: None 2022-09-03T06:50:17.828 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:17.829 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LPaWknIJkpUWlnkqMWjBfC3C7F4= 2022-09-03T06:50:17.830 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LPaWknIJkpUWlnkqMWjBfC3C7F4=', 'Content-Length': '0'} 2022-09-03T06:50:17.832 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ac3f1f70975865e3-006312f929-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.833 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:17.833 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:17.834 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:17.834 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:17.835 DEBUG:boto:Method: GET 2022-09-03T06:50:17.836 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:50:17.836 DEBUG:boto:Data: 2022-09-03T06:50:17.837 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.837 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.838 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.838 DEBUG:boto:Params: {} 2022-09-03T06:50:17.839 DEBUG:boto:Token: None 2022-09-03T06:50:17.840 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/ 2022-09-03T06:50:17.840 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8= 2022-09-03T06:50:17.841 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8=', 'Content-Length': '0'} 2022-09-03T06:50:17.844 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f1e9961df55a88f2-006312f929-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.845 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:17.846 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:17.846 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:17.847 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:17.847 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:17.848 DEBUG:boto:Method: GET 2022-09-03T06:50:17.848 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:50:17.849 DEBUG:boto:Data: 2022-09-03T06:50:17.849 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.850 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.851 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.851 DEBUG:boto:Params: {} 2022-09-03T06:50:17.852 DEBUG:boto:Token: None 2022-09-03T06:50:17.853 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/ 2022-09-03T06:50:17.853 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8= 2022-09-03T06:50:17.854 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8=', 'Content-Length': '0'} 2022-09-03T06:50:17.859 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000018228604248f6535-006312f929-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.859 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:17.860 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:17.861 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:17.862 DEBUG:boto:Method: HEAD 2022-09-03T06:50:17.862 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:50:17.863 DEBUG:boto:Data: 2022-09-03T06:50:17.863 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.864 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.864 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.865 DEBUG:boto:Params: {} 2022-09-03T06:50:17.865 DEBUG:boto:Token: None 2022-09-03T06:50:17.866 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj38 2022-09-03T06:50:17.866 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KSY9RUk/XDpDSriaYzqSOua1ACc= 2022-09-03T06:50:17.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KSY9RUk/XDpDSriaYzqSOua1ACc=', 'Content-Length': '0'} 2022-09-03T06:50:17.869 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000076f4bf22208f6816-006312f929-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.870 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:17.871 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:17.871 DEBUG:boto:Method: HEAD 2022-09-03T06:50:17.872 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:50:17.872 DEBUG:boto:Data: 2022-09-03T06:50:17.873 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.874 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.874 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.875 DEBUG:boto:Params: {} 2022-09-03T06:50:17.875 DEBUG:boto:Token: None 2022-09-03T06:50:17.876 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj38 2022-09-03T06:50:17.876 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KSY9RUk/XDpDSriaYzqSOua1ACc= 2022-09-03T06:50:17.877 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KSY9RUk/XDpDSriaYzqSOua1ACc=', 'Content-Length': '0'} 2022-09-03T06:50:17.879 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000041e4d90ea73a73e0-006312f929-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.880 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:50:17.881 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:17.881 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:17.882 DEBUG:boto:Method: GET 2022-09-03T06:50:17.883 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:50:17.883 DEBUG:boto:Data: 2022-09-03T06:50:17.884 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.884 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.885 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.885 DEBUG:boto:Params: {} 2022-09-03T06:50:17.886 DEBUG:boto:Token: None 2022-09-03T06:50:17.887 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj38 2022-09-03T06:50:17.887 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v1w5XOytaIeMG/2ma8vMMBMvQY8= 2022-09-03T06:50:17.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v1w5XOytaIeMG/2ma8vMMBMvQY8=', 'Content-Length': '0'} 2022-09-03T06:50:17.890 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a50522fb54654402-006312f929-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.891 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:17.891 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:17.892 DEBUG:boto:Method: GET 2022-09-03T06:50:17.893 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:50:17.893 DEBUG:boto:Data: 2022-09-03T06:50:17.894 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.894 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.895 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.896 DEBUG:boto:Params: {} 2022-09-03T06:50:17.896 DEBUG:boto:Token: None 2022-09-03T06:50:17.897 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj38 2022-09-03T06:50:17.897 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v1w5XOytaIeMG/2ma8vMMBMvQY8= 2022-09-03T06:50:17.898 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v1w5XOytaIeMG/2ma8vMMBMvQY8=', 'Content-Length': '0'} 2022-09-03T06:50:17.902 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006c1751352d4d4f6f-006312f929-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.904 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:17.905 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:17.905 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:17.906 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:17.907 DEBUG:boto:Method: GET 2022-09-03T06:50:17.907 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:50:17.908 DEBUG:boto:Data: 2022-09-03T06:50:17.908 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.909 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.910 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.910 DEBUG:boto:Params: {} 2022-09-03T06:50:17.911 DEBUG:boto:Token: None 2022-09-03T06:50:17.911 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/ 2022-09-03T06:50:17.912 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8= 2022-09-03T06:50:17.913 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8=', 'Content-Length': '0'} 2022-09-03T06:50:17.916 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000003c1df064b2acf3c-006312f929-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.917 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:17.918 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:17.919 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:17.920 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:17.921 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:17.921 DEBUG:boto:Method: GET 2022-09-03T06:50:17.922 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:50:17.923 DEBUG:boto:Data: 2022-09-03T06:50:17.924 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.924 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:17.925 DEBUG:boto:Port: 8000 2022-09-03T06:50:17.926 DEBUG:boto:Params: {} 2022-09-03T06:50:17.927 DEBUG:boto:Token: None 2022-09-03T06:50:17.927 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/ 2022-09-03T06:50:17.928 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8= 2022-09-03T06:50:17.929 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/00WxGIUQB7USMAvImWtV3hRKm8=', 'Content-Length': '0'} 2022-09-03T06:50:17.934 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000512a203e84aea76a-006312f929-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.976 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:17.977 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:50:17.978 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:17.979 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:17.979 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:17.980 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:17.980 DEBUG:boto:Method: GET 2022-09-03T06:50:17.981 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:17.981 DEBUG:boto:Data: 2022-09-03T06:50:17.982 DEBUG:boto:Headers: {} 2022-09-03T06:50:17.983 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:17.983 DEBUG:boto:Port: 8001 2022-09-03T06:50:17.984 DEBUG:boto:Params: {} 2022-09-03T06:50:17.991 DEBUG:boto:Token: None 2022-09-03T06:50:17.993 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:17 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:17.994 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5hMpW0w2wCn9OngkdJbbnH3taEE= 2022-09-03T06:50:17.994 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:17 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5hMpW0w2wCn9OngkdJbbnH3taEE=', 'Content-Length': '0'} 2022-09-03T06:50:17.997 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008dea52936dffb63f-006312f929-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:17 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:17.998 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:17.999 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:18.000 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:18.000 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:18.001 DEBUG:boto:Method: GET 2022-09-03T06:50:18.001 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:18.002 DEBUG:boto:Data: 2022-09-03T06:50:18.002 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.003 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.003 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.004 DEBUG:boto:Params: {} 2022-09-03T06:50:18.005 DEBUG:boto:Token: None 2022-09-03T06:50:18.005 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:18.006 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JhrqhxnWfriQZlytjsnV7H+Vj5M= 2022-09-03T06:50:18.007 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JhrqhxnWfriQZlytjsnV7H+Vj5M=', 'Content-Length': '0'} 2022-09-03T06:50:18.009 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008606bb6eafde0e75-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.010 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:18.011 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:18.011 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:18.012 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:18.012 DEBUG:boto:Method: HEAD 2022-09-03T06:50:18.013 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:18.014 DEBUG:boto:Data: 2022-09-03T06:50:18.014 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.015 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.015 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.016 DEBUG:boto:Params: {} 2022-09-03T06:50:18.017 DEBUG:boto:Token: None 2022-09-03T06:50:18.017 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:18.018 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XjfAZFLOpGshT8mdf896uYfvtDQ= 2022-09-03T06:50:18.018 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XjfAZFLOpGshT8mdf896uYfvtDQ=', 'Content-Length': '0'} 2022-09-03T06:50:18.021 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000024c640c41473adac-006312f92a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.022 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:18.023 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:18.024 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:18.024 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:18.025 DEBUG:boto:Method: HEAD 2022-09-03T06:50:18.027 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:18.028 DEBUG:boto:Data: 2022-09-03T06:50:18.029 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.030 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.031 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.032 DEBUG:boto:Params: {} 2022-09-03T06:50:18.033 DEBUG:boto:Token: None 2022-09-03T06:50:18.034 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:18.035 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:XjfAZFLOpGshT8mdf896uYfvtDQ= 2022-09-03T06:50:18.036 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:XjfAZFLOpGshT8mdf896uYfvtDQ=', 'Content-Length': '0'} 2022-09-03T06:50:18.039 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005fc25c68a379da2c-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.040 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:50:18.041 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:18.042 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:18.043 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:18.044 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:18.044 DEBUG:boto:Method: GET 2022-09-03T06:50:18.045 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:18.047 DEBUG:boto:Data: 2022-09-03T06:50:18.048 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.049 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.049 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.050 DEBUG:boto:Params: {} 2022-09-03T06:50:18.051 DEBUG:boto:Token: None 2022-09-03T06:50:18.051 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:18.052 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JhrqhxnWfriQZlytjsnV7H+Vj5M= 2022-09-03T06:50:18.053 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JhrqhxnWfriQZlytjsnV7H+Vj5M=', 'Content-Length': '0'} 2022-09-03T06:50:18.056 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018ef67f0b764b9e4-006312f92a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.057 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:18.058 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:18.059 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:18.060 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:18.061 DEBUG:boto:Method: GET 2022-09-03T06:50:18.062 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:18.063 DEBUG:boto:Data: 2022-09-03T06:50:18.064 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.065 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.066 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.067 DEBUG:boto:Params: {} 2022-09-03T06:50:18.068 DEBUG:boto:Token: None 2022-09-03T06:50:18.069 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:18.070 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JhrqhxnWfriQZlytjsnV7H+Vj5M= 2022-09-03T06:50:18.071 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JhrqhxnWfriQZlytjsnV7H+Vj5M=', 'Content-Length': '0'} 2022-09-03T06:50:18.073 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000395d8e4374d86826-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.076 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:18.077 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:18.079 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:18.080 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:18.081 DEBUG:boto:Method: GET 2022-09-03T06:50:18.082 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:50:18.083 DEBUG:boto:Data: 2022-09-03T06:50:18.085 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.087 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.087 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.088 DEBUG:boto:Params: {} 2022-09-03T06:50:18.089 DEBUG:boto:Token: None 2022-09-03T06:50:18.089 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/ 2022-09-03T06:50:18.090 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM= 2022-09-03T06:50:18.090 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM=', 'Content-Length': '0'} 2022-09-03T06:50:18.099 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ffbc278ab4a367a2-006312f92a-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.099 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:18.100 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:18.101 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:18.102 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:18.102 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:18.103 DEBUG:boto:Method: GET 2022-09-03T06:50:18.103 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:50:18.104 DEBUG:boto:Data: 2022-09-03T06:50:18.104 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.105 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.105 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.106 DEBUG:boto:Params: {} 2022-09-03T06:50:18.107 DEBUG:boto:Token: None 2022-09-03T06:50:18.107 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/ 2022-09-03T06:50:18.109 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM= 2022-09-03T06:50:18.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM=', 'Content-Length': '0'} 2022-09-03T06:50:18.114 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000267705bb065703ce-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.159 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:18.160 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:18.161 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:18.161 DEBUG:boto:Method: HEAD 2022-09-03T06:50:18.162 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:50:18.163 DEBUG:boto:Data: 2022-09-03T06:50:18.164 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.164 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.165 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.166 DEBUG:boto:Params: {} 2022-09-03T06:50:18.169 DEBUG:boto:Token: None 2022-09-03T06:50:18.170 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj44 2022-09-03T06:50:18.172 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:38UhG4grYAZ3pE5zXTX3tzY6Ws8= 2022-09-03T06:50:18.173 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:38UhG4grYAZ3pE5zXTX3tzY6Ws8=', 'Content-Length': '0'} 2022-09-03T06:50:18.178 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000022b9067eb7cf84a-006312f92a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.180 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:18.181 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:18.181 DEBUG:boto:Method: HEAD 2022-09-03T06:50:18.182 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:50:18.183 DEBUG:boto:Data: 2022-09-03T06:50:18.183 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.184 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.185 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.187 DEBUG:boto:Params: {} 2022-09-03T06:50:18.187 DEBUG:boto:Token: None 2022-09-03T06:50:18.188 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj44 2022-09-03T06:50:18.189 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:38UhG4grYAZ3pE5zXTX3tzY6Ws8= 2022-09-03T06:50:18.190 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:38UhG4grYAZ3pE5zXTX3tzY6Ws8=', 'Content-Length': '0'} 2022-09-03T06:50:18.193 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f4c0a9ef56cdde6-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.193 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:50:18.194 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:18.195 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:18.195 DEBUG:boto:Method: GET 2022-09-03T06:50:18.196 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:50:18.197 DEBUG:boto:Data: 2022-09-03T06:50:18.197 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.198 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.199 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.199 DEBUG:boto:Params: {} 2022-09-03T06:50:18.200 DEBUG:boto:Token: None 2022-09-03T06:50:18.201 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj44 2022-09-03T06:50:18.201 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZwuDePuUKuCkwC7GtlFS33dvzr8= 2022-09-03T06:50:18.202 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZwuDePuUKuCkwC7GtlFS33dvzr8=', 'Content-Length': '0'} 2022-09-03T06:50:18.204 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000066524292abd9e70b-006312f92a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.206 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:18.207 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:18.207 DEBUG:boto:Method: GET 2022-09-03T06:50:18.208 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:50:18.208 DEBUG:boto:Data: 2022-09-03T06:50:18.209 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.209 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.210 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.210 DEBUG:boto:Params: {} 2022-09-03T06:50:18.211 DEBUG:boto:Token: None 2022-09-03T06:50:18.212 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj44 2022-09-03T06:50:18.212 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZwuDePuUKuCkwC7GtlFS33dvzr8= 2022-09-03T06:50:18.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZwuDePuUKuCkwC7GtlFS33dvzr8=', 'Content-Length': '0'} 2022-09-03T06:50:18.217 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d1d73485c81bb8f-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.218 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:18.219 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:18.220 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:18.220 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:18.221 DEBUG:boto:Method: GET 2022-09-03T06:50:18.221 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:50:18.222 DEBUG:boto:Data: 2022-09-03T06:50:18.222 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.223 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.224 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.224 DEBUG:boto:Params: {} 2022-09-03T06:50:18.225 DEBUG:boto:Token: None 2022-09-03T06:50:18.225 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/ 2022-09-03T06:50:18.226 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM= 2022-09-03T06:50:18.227 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM=', 'Content-Length': '0'} 2022-09-03T06:50:18.230 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000765a065747465d6f-006312f92a-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.230 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:18.231 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:18.232 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:18.233 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:18.233 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:18.234 DEBUG:boto:Method: GET 2022-09-03T06:50:18.234 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:50:18.235 DEBUG:boto:Data: 2022-09-03T06:50:18.235 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.236 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.237 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.237 DEBUG:boto:Params: {} 2022-09-03T06:50:18.238 DEBUG:boto:Token: None 2022-09-03T06:50:18.239 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/ 2022-09-03T06:50:18.239 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM= 2022-09-03T06:50:18.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM=', 'Content-Length': '0'} 2022-09-03T06:50:18.245 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004513c15093a36038-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.246 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:18.246 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:50:18.247 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:18.248 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:18.248 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.249 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.249 DEBUG:boto:Method: GET 2022-09-03T06:50:18.250 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.251 DEBUG:boto:Data: 2022-09-03T06:50:18.251 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.252 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.252 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.253 DEBUG:boto:Params: {} 2022-09-03T06:50:18.253 DEBUG:boto:Token: None 2022-09-03T06:50:18.254 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.255 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qd51FL0BnoR3t+6Kc3Rdq5DDGf4= 2022-09-03T06:50:18.255 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qd51FL0BnoR3t+6Kc3Rdq5DDGf4=', 'Content-Length': '0'} 2022-09-03T06:50:18.257 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2e3e5b954721f66-006312f92a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.298 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:18.306 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:18.307 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.308 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.310 DEBUG:boto:Method: GET 2022-09-03T06:50:18.310 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.311 DEBUG:boto:Data: 2022-09-03T06:50:18.311 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.312 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.312 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.313 DEBUG:boto:Params: {} 2022-09-03T06:50:18.314 DEBUG:boto:Token: None 2022-09-03T06:50:18.314 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.315 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qd51FL0BnoR3t+6Kc3Rdq5DDGf4= 2022-09-03T06:50:18.316 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qd51FL0BnoR3t+6Kc3Rdq5DDGf4=', 'Content-Length': '0'} 2022-09-03T06:50:18.320 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b272fc47ec58890b-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.321 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:18.322 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:18.323 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.323 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.324 DEBUG:boto:Method: HEAD 2022-09-03T06:50:18.324 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.325 DEBUG:boto:Data: 2022-09-03T06:50:18.325 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.326 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.326 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.327 DEBUG:boto:Params: {} 2022-09-03T06:50:18.329 DEBUG:boto:Token: None 2022-09-03T06:50:18.329 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.330 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vxPGRhLfKvF8aSdKDRuAgA/UGkE= 2022-09-03T06:50:18.331 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vxPGRhLfKvF8aSdKDRuAgA/UGkE=', 'Content-Length': '0'} 2022-09-03T06:50:18.333 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007f4bc2f190a9fac5-006312f92a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.334 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:18.335 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:18.336 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.336 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.337 DEBUG:boto:Method: HEAD 2022-09-03T06:50:18.337 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.338 DEBUG:boto:Data: 2022-09-03T06:50:18.339 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.340 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.340 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.341 DEBUG:boto:Params: {} 2022-09-03T06:50:18.342 DEBUG:boto:Token: None 2022-09-03T06:50:18.342 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.343 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:vxPGRhLfKvF8aSdKDRuAgA/UGkE= 2022-09-03T06:50:18.343 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:vxPGRhLfKvF8aSdKDRuAgA/UGkE=', 'Content-Length': '0'} 2022-09-03T06:50:18.345 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000786bf2508c350088-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.346 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:50:18.346 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:18.347 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:18.348 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.348 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.350 DEBUG:boto:Method: GET 2022-09-03T06:50:18.351 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.351 DEBUG:boto:Data: 2022-09-03T06:50:18.352 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.352 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.353 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.354 DEBUG:boto:Params: {} 2022-09-03T06:50:18.354 DEBUG:boto:Token: None 2022-09-03T06:50:18.355 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.356 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qd51FL0BnoR3t+6Kc3Rdq5DDGf4= 2022-09-03T06:50:18.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qd51FL0BnoR3t+6Kc3Rdq5DDGf4=', 'Content-Length': '0'} 2022-09-03T06:50:18.370 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000374d3bd601f289e1-006312f92a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.371 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:18.372 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:18.372 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.373 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.373 DEBUG:boto:Method: GET 2022-09-03T06:50:18.374 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.375 DEBUG:boto:Data: 2022-09-03T06:50:18.376 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.377 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.377 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.378 DEBUG:boto:Params: {} 2022-09-03T06:50:18.379 DEBUG:boto:Token: None 2022-09-03T06:50:18.380 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:18.380 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qd51FL0BnoR3t+6Kc3Rdq5DDGf4= 2022-09-03T06:50:18.381 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qd51FL0BnoR3t+6Kc3Rdq5DDGf4=', 'Content-Length': '0'} 2022-09-03T06:50:18.384 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006154f4f31be8d352-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.385 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:18.385 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:18.386 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:18.386 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:18.387 DEBUG:boto:Method: GET 2022-09-03T06:50:18.388 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:50:18.388 DEBUG:boto:Data: 2022-09-03T06:50:18.389 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.390 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.390 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.391 DEBUG:boto:Params: {} 2022-09-03T06:50:18.391 DEBUG:boto:Token: None 2022-09-03T06:50:18.392 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/ 2022-09-03T06:50:18.393 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM= 2022-09-03T06:50:18.393 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM=', 'Content-Length': '0'} 2022-09-03T06:50:18.397 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008840b082fd08662b-006312f92a-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.399 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:18.400 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:18.400 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:18.401 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:18.402 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:18.402 DEBUG:boto:Method: GET 2022-09-03T06:50:18.403 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:50:18.403 DEBUG:boto:Data: 2022-09-03T06:50:18.404 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.404 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.405 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.406 DEBUG:boto:Params: {} 2022-09-03T06:50:18.406 DEBUG:boto:Token: None 2022-09-03T06:50:18.407 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/ 2022-09-03T06:50:18.408 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM= 2022-09-03T06:50:18.408 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM=', 'Content-Length': '0'} 2022-09-03T06:50:18.413 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ee5cf4d5f82c0e7e-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.414 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:18.415 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:18.416 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:18.416 DEBUG:boto:Method: HEAD 2022-09-03T06:50:18.417 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:50:18.417 DEBUG:boto:Data: 2022-09-03T06:50:18.418 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.419 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.419 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.420 DEBUG:boto:Params: {} 2022-09-03T06:50:18.420 DEBUG:boto:Token: None 2022-09-03T06:50:18.421 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj46 2022-09-03T06:50:18.422 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:W7oYmxmHOSJk9iuLRmX+GdbafTM= 2022-09-03T06:50:18.422 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:W7oYmxmHOSJk9iuLRmX+GdbafTM=', 'Content-Length': '0'} 2022-09-03T06:50:18.425 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c732fbef2bab7ce6-006312f92a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.425 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:18.426 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:18.427 DEBUG:boto:Method: HEAD 2022-09-03T06:50:18.427 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:50:18.428 DEBUG:boto:Data: 2022-09-03T06:50:18.428 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.429 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.429 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.430 DEBUG:boto:Params: {} 2022-09-03T06:50:18.431 DEBUG:boto:Token: None 2022-09-03T06:50:18.431 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj46 2022-09-03T06:50:18.432 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:W7oYmxmHOSJk9iuLRmX+GdbafTM= 2022-09-03T06:50:18.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:W7oYmxmHOSJk9iuLRmX+GdbafTM=', 'Content-Length': '0'} 2022-09-03T06:50:18.435 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003dd0aa1aa55dbf82-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.435 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:50:18.436 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:18.437 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:18.437 DEBUG:boto:Method: GET 2022-09-03T06:50:18.438 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:50:18.438 DEBUG:boto:Data: 2022-09-03T06:50:18.439 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.439 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.440 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.440 DEBUG:boto:Params: {} 2022-09-03T06:50:18.441 DEBUG:boto:Token: None 2022-09-03T06:50:18.441 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj46 2022-09-03T06:50:18.442 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tgugORGPVbdw+G/UVR0Oy+M8tAM= 2022-09-03T06:50:18.443 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tgugORGPVbdw+G/UVR0Oy+M8tAM=', 'Content-Length': '0'} 2022-09-03T06:50:18.445 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005563a6edad3b68fb-006312f92a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.446 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:18.446 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:18.447 DEBUG:boto:Method: GET 2022-09-03T06:50:18.447 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:50:18.448 DEBUG:boto:Data: 2022-09-03T06:50:18.448 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.449 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.449 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.450 DEBUG:boto:Params: {} 2022-09-03T06:50:18.451 DEBUG:boto:Token: None 2022-09-03T06:50:18.451 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj46 2022-09-03T06:50:18.452 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tgugORGPVbdw+G/UVR0Oy+M8tAM= 2022-09-03T06:50:18.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tgugORGPVbdw+G/UVR0Oy+M8tAM=', 'Content-Length': '0'} 2022-09-03T06:50:18.455 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009ffd50a7df81b836-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.498 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:18.500 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:18.501 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:18.502 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:18.502 DEBUG:boto:Method: GET 2022-09-03T06:50:18.503 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:50:18.504 DEBUG:boto:Data: 2022-09-03T06:50:18.505 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.505 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.506 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.506 DEBUG:boto:Params: {} 2022-09-03T06:50:18.507 DEBUG:boto:Token: None 2022-09-03T06:50:18.508 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/ 2022-09-03T06:50:18.509 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM= 2022-09-03T06:50:18.510 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM=', 'Content-Length': '0'} 2022-09-03T06:50:18.513 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000906ec8e870d76c66-006312f92a-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.514 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:18.516 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:18.517 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:18.518 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:18.519 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:18.520 DEBUG:boto:Method: GET 2022-09-03T06:50:18.521 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:50:18.521 DEBUG:boto:Data: 2022-09-03T06:50:18.522 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.523 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.524 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.525 DEBUG:boto:Params: {} 2022-09-03T06:50:18.526 DEBUG:boto:Token: None 2022-09-03T06:50:18.527 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/ 2022-09-03T06:50:18.528 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM= 2022-09-03T06:50:18.529 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM=', 'Content-Length': '0'} 2022-09-03T06:50:18.533 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c2ecb4d2f8f7966c-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.535 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:18.536 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:50:18.536 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:18.537 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:18.537 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.538 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.539 DEBUG:boto:Method: GET 2022-09-03T06:50:18.540 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.540 DEBUG:boto:Data: 2022-09-03T06:50:18.541 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.542 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.542 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.543 DEBUG:boto:Params: {} 2022-09-03T06:50:18.543 DEBUG:boto:Token: None 2022-09-03T06:50:18.544 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.545 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3hcGITsLa9BiYmoOcNen9i7fTSY= 2022-09-03T06:50:18.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3hcGITsLa9BiYmoOcNen9i7fTSY=', 'Content-Length': '0'} 2022-09-03T06:50:18.550 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000acf7d7fd532de663-006312f92a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.551 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:18.552 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:18.553 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.554 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.555 DEBUG:boto:Method: GET 2022-09-03T06:50:18.556 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.557 DEBUG:boto:Data: 2022-09-03T06:50:18.558 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.558 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.559 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.560 DEBUG:boto:Params: {} 2022-09-03T06:50:18.561 DEBUG:boto:Token: None 2022-09-03T06:50:18.561 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.562 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3hcGITsLa9BiYmoOcNen9i7fTSY= 2022-09-03T06:50:18.563 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3hcGITsLa9BiYmoOcNen9i7fTSY=', 'Content-Length': '0'} 2022-09-03T06:50:18.566 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e028024f44d60b8-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.567 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:18.567 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:18.568 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.568 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.569 DEBUG:boto:Method: HEAD 2022-09-03T06:50:18.570 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.570 DEBUG:boto:Data: 2022-09-03T06:50:18.571 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.571 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.572 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.572 DEBUG:boto:Params: {} 2022-09-03T06:50:18.573 DEBUG:boto:Token: None 2022-09-03T06:50:18.574 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.575 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:raT8ySO0m05funfJ53mJi3+mrZw= 2022-09-03T06:50:18.575 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:raT8ySO0m05funfJ53mJi3+mrZw=', 'Content-Length': '0'} 2022-09-03T06:50:18.578 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000062efeb55dec874f0-006312f92a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.579 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:18.580 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:18.580 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.581 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.582 DEBUG:boto:Method: HEAD 2022-09-03T06:50:18.582 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.583 DEBUG:boto:Data: 2022-09-03T06:50:18.583 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.584 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.584 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.587 DEBUG:boto:Params: {} 2022-09-03T06:50:18.588 DEBUG:boto:Token: None 2022-09-03T06:50:18.588 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.589 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:raT8ySO0m05funfJ53mJi3+mrZw= 2022-09-03T06:50:18.589 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:raT8ySO0m05funfJ53mJi3+mrZw=', 'Content-Length': '0'} 2022-09-03T06:50:18.595 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2c1db70df580cce-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.597 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:50:18.598 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:18.599 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:18.599 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.600 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.600 DEBUG:boto:Method: GET 2022-09-03T06:50:18.601 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.601 DEBUG:boto:Data: 2022-09-03T06:50:18.602 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.604 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.606 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.607 DEBUG:boto:Params: {} 2022-09-03T06:50:18.608 DEBUG:boto:Token: None 2022-09-03T06:50:18.610 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.611 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3hcGITsLa9BiYmoOcNen9i7fTSY= 2022-09-03T06:50:18.612 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3hcGITsLa9BiYmoOcNen9i7fTSY=', 'Content-Length': '0'} 2022-09-03T06:50:18.614 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ae4cf872c5652892-006312f92a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.658 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:18.659 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:18.660 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.661 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.662 DEBUG:boto:Method: GET 2022-09-03T06:50:18.663 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.664 DEBUG:boto:Data: 2022-09-03T06:50:18.666 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.667 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.668 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.668 DEBUG:boto:Params: {} 2022-09-03T06:50:18.669 DEBUG:boto:Token: None 2022-09-03T06:50:18.670 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:18.670 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3hcGITsLa9BiYmoOcNen9i7fTSY= 2022-09-03T06:50:18.671 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3hcGITsLa9BiYmoOcNen9i7fTSY=', 'Content-Length': '0'} 2022-09-03T06:50:18.674 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000da45400307c97a3e-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.675 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:18.676 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:18.677 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:18.677 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:18.678 DEBUG:boto:Method: GET 2022-09-03T06:50:18.679 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:50:18.679 DEBUG:boto:Data: 2022-09-03T06:50:18.680 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.681 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.682 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.682 DEBUG:boto:Params: {} 2022-09-03T06:50:18.683 DEBUG:boto:Token: None 2022-09-03T06:50:18.684 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/ 2022-09-03T06:50:18.684 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM= 2022-09-03T06:50:18.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM=', 'Content-Length': '0'} 2022-09-03T06:50:18.689 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dd775bf99894f7ec-006312f92a-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.689 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:18.690 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:18.691 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:18.692 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:18.692 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:18.693 DEBUG:boto:Method: GET 2022-09-03T06:50:18.693 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:50:18.694 DEBUG:boto:Data: 2022-09-03T06:50:18.694 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.695 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.695 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.696 DEBUG:boto:Params: {} 2022-09-03T06:50:18.697 DEBUG:boto:Token: None 2022-09-03T06:50:18.697 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/ 2022-09-03T06:50:18.698 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM= 2022-09-03T06:50:18.699 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM=', 'Content-Length': '0'} 2022-09-03T06:50:18.703 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003ed10ea5a2edf1b7-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.746 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:18.748 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:18.749 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:18.750 DEBUG:boto:Method: HEAD 2022-09-03T06:50:18.751 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:50:18.752 DEBUG:boto:Data: 2022-09-03T06:50:18.753 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.755 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.756 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.757 DEBUG:boto:Params: {} 2022-09-03T06:50:18.757 DEBUG:boto:Token: None 2022-09-03T06:50:18.758 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj51 2022-09-03T06:50:18.758 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3mpesErBcNVP1erh+bpgH84UCGo= 2022-09-03T06:50:18.759 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3mpesErBcNVP1erh+bpgH84UCGo=', 'Content-Length': '0'} 2022-09-03T06:50:18.762 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000665bcd0b1077976a-006312f92a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.763 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:18.764 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:18.765 DEBUG:boto:Method: HEAD 2022-09-03T06:50:18.766 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:50:18.767 DEBUG:boto:Data: 2022-09-03T06:50:18.768 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.770 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.770 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.771 DEBUG:boto:Params: {} 2022-09-03T06:50:18.771 DEBUG:boto:Token: None 2022-09-03T06:50:18.772 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj51 2022-09-03T06:50:18.773 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3mpesErBcNVP1erh+bpgH84UCGo= 2022-09-03T06:50:18.773 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3mpesErBcNVP1erh+bpgH84UCGo=', 'Content-Length': '0'} 2022-09-03T06:50:18.775 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004efc1d6957eed49c-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.776 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:50:18.777 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:18.777 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:18.778 DEBUG:boto:Method: GET 2022-09-03T06:50:18.778 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:50:18.779 DEBUG:boto:Data: 2022-09-03T06:50:18.779 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.780 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.781 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.781 DEBUG:boto:Params: {} 2022-09-03T06:50:18.782 DEBUG:boto:Token: None 2022-09-03T06:50:18.782 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj51 2022-09-03T06:50:18.783 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TV/PsftO75uVbP02JV9DxrmraBo= 2022-09-03T06:50:18.784 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TV/PsftO75uVbP02JV9DxrmraBo=', 'Content-Length': '0'} 2022-09-03T06:50:18.786 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f005d4e75da5c9c1-006312f92a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.787 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:18.788 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:18.788 DEBUG:boto:Method: GET 2022-09-03T06:50:18.789 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:50:18.790 DEBUG:boto:Data: 2022-09-03T06:50:18.790 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.791 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.791 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.792 DEBUG:boto:Params: {} 2022-09-03T06:50:18.793 DEBUG:boto:Token: None 2022-09-03T06:50:18.793 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj51 2022-09-03T06:50:18.794 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TV/PsftO75uVbP02JV9DxrmraBo= 2022-09-03T06:50:18.795 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TV/PsftO75uVbP02JV9DxrmraBo=', 'Content-Length': '0'} 2022-09-03T06:50:18.797 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000336d206bb5235b44-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.798 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:18.798 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:18.799 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:18.799 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:18.800 DEBUG:boto:Method: GET 2022-09-03T06:50:18.801 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:50:18.801 DEBUG:boto:Data: 2022-09-03T06:50:18.802 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.802 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.803 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.803 DEBUG:boto:Params: {} 2022-09-03T06:50:18.804 DEBUG:boto:Token: None 2022-09-03T06:50:18.804 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/ 2022-09-03T06:50:18.805 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM= 2022-09-03T06:50:18.806 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM=', 'Content-Length': '0'} 2022-09-03T06:50:18.808 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000040e673fc2ff8c402-006312f92a-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.809 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:18.810 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:18.811 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:18.811 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:18.812 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:18.812 DEBUG:boto:Method: GET 2022-09-03T06:50:18.813 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:50:18.813 DEBUG:boto:Data: 2022-09-03T06:50:18.814 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.814 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.815 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.816 DEBUG:boto:Params: {} 2022-09-03T06:50:18.817 DEBUG:boto:Token: None 2022-09-03T06:50:18.817 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/ 2022-09-03T06:50:18.818 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM= 2022-09-03T06:50:18.818 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM=', 'Content-Length': '0'} 2022-09-03T06:50:18.823 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c218cb4ecb3747c5-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.824 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:18.825 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:50:18.825 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:18.826 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:18.826 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.827 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.828 DEBUG:boto:Method: GET 2022-09-03T06:50:18.828 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.829 DEBUG:boto:Data: 2022-09-03T06:50:18.829 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.830 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.830 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.831 DEBUG:boto:Params: {} 2022-09-03T06:50:18.831 DEBUG:boto:Token: None 2022-09-03T06:50:18.832 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.833 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yknsz7/Ds0P4CjZiBnDoc3Zm2aU= 2022-09-03T06:50:18.833 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yknsz7/Ds0P4CjZiBnDoc3Zm2aU=', 'Content-Length': '0'} 2022-09-03T06:50:18.835 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d4ddc91e860a2926-006312f92a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.836 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:18.837 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:18.837 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.838 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.839 DEBUG:boto:Method: GET 2022-09-03T06:50:18.839 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.840 DEBUG:boto:Data: 2022-09-03T06:50:18.840 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.841 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.841 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.842 DEBUG:boto:Params: {} 2022-09-03T06:50:18.842 DEBUG:boto:Token: None 2022-09-03T06:50:18.843 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.844 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yknsz7/Ds0P4CjZiBnDoc3Zm2aU= 2022-09-03T06:50:18.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yknsz7/Ds0P4CjZiBnDoc3Zm2aU=', 'Content-Length': '0'} 2022-09-03T06:50:18.847 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c545bd53360d8dd4-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.847 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:18.848 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:18.849 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.849 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.850 DEBUG:boto:Method: HEAD 2022-09-03T06:50:18.850 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.851 DEBUG:boto:Data: 2022-09-03T06:50:18.851 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.852 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.852 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.853 DEBUG:boto:Params: {} 2022-09-03T06:50:18.854 DEBUG:boto:Token: None 2022-09-03T06:50:18.854 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.855 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:skYofm9RK0iJdwdu4AnggBhsIMY= 2022-09-03T06:50:18.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:skYofm9RK0iJdwdu4AnggBhsIMY=', 'Content-Length': '0'} 2022-09-03T06:50:18.858 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000031c6cfa71fbb31a0-006312f92a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.859 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:18.860 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:18.860 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.861 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.861 DEBUG:boto:Method: HEAD 2022-09-03T06:50:18.862 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.862 DEBUG:boto:Data: 2022-09-03T06:50:18.863 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.863 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.864 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.864 DEBUG:boto:Params: {} 2022-09-03T06:50:18.865 DEBUG:boto:Token: None 2022-09-03T06:50:18.866 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.866 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:skYofm9RK0iJdwdu4AnggBhsIMY= 2022-09-03T06:50:18.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:skYofm9RK0iJdwdu4AnggBhsIMY=', 'Content-Length': '0'} 2022-09-03T06:50:18.870 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001088714919696cb5-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.871 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:50:18.872 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:18.872 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:18.873 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.874 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.874 DEBUG:boto:Method: GET 2022-09-03T06:50:18.875 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.875 DEBUG:boto:Data: 2022-09-03T06:50:18.876 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.877 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.877 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.878 DEBUG:boto:Params: {} 2022-09-03T06:50:18.878 DEBUG:boto:Token: None 2022-09-03T06:50:18.879 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.880 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yknsz7/Ds0P4CjZiBnDoc3Zm2aU= 2022-09-03T06:50:18.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yknsz7/Ds0P4CjZiBnDoc3Zm2aU=', 'Content-Length': '0'} 2022-09-03T06:50:18.882 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d45895b6b63d636-006312f92a-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.883 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:18.884 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:18.885 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.885 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.886 DEBUG:boto:Method: GET 2022-09-03T06:50:18.887 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.887 DEBUG:boto:Data: 2022-09-03T06:50:18.888 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.888 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.889 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.889 DEBUG:boto:Params: {} 2022-09-03T06:50:18.890 DEBUG:boto:Token: None 2022-09-03T06:50:18.890 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:18.891 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Yknsz7/Ds0P4CjZiBnDoc3Zm2aU= 2022-09-03T06:50:18.892 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Yknsz7/Ds0P4CjZiBnDoc3Zm2aU=', 'Content-Length': '0'} 2022-09-03T06:50:18.895 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1f34d18d0ca85e3-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.945 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:18.946 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:18.947 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:18.948 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:18.948 DEBUG:boto:Method: GET 2022-09-03T06:50:18.949 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:50:18.950 DEBUG:boto:Data: 2022-09-03T06:50:18.952 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.953 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.954 DEBUG:boto:Port: 8001 2022-09-03T06:50:18.955 DEBUG:boto:Params: {} 2022-09-03T06:50:18.958 DEBUG:boto:Token: None 2022-09-03T06:50:18.959 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/ 2022-09-03T06:50:18.960 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM= 2022-09-03T06:50:18.961 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM=', 'Content-Length': '0'} 2022-09-03T06:50:18.964 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000be8f78c9804ad766-006312f92a-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.966 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:18.967 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:18.968 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:18.969 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:18.970 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:18.971 DEBUG:boto:Method: GET 2022-09-03T06:50:18.973 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:50:18.975 DEBUG:boto:Data: 2022-09-03T06:50:18.975 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.976 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:18.977 DEBUG:boto:Port: 8000 2022-09-03T06:50:18.979 DEBUG:boto:Params: {} 2022-09-03T06:50:18.980 DEBUG:boto:Token: None 2022-09-03T06:50:18.982 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:18 GMT /dsgvcy-75/ 2022-09-03T06:50:18.983 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM= 2022-09-03T06:50:18.984 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:18 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:z9Yfd0S18YxwBCU0EmPI9ix33EM=', 'Content-Length': '0'} 2022-09-03T06:50:18.989 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f000d8b54f274d48-006312f92a-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:18 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:18.991 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:18.992 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:18.993 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:18.994 DEBUG:boto:Method: HEAD 2022-09-03T06:50:18.995 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:50:18.996 DEBUG:boto:Data: 2022-09-03T06:50:18.997 DEBUG:boto:Headers: {} 2022-09-03T06:50:18.998 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:18.999 DEBUG:boto:Port: 8001 2022-09-03T06:50:19.000 DEBUG:boto:Params: {} 2022-09-03T06:50:19.001 DEBUG:boto:Token: None 2022-09-03T06:50:19.002 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj55 2022-09-03T06:50:19.003 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R5blDydms1dEHR2EuyuHui4KaXg= 2022-09-03T06:50:19.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R5blDydms1dEHR2EuyuHui4KaXg=', 'Content-Length': '0'} 2022-09-03T06:50:19.007 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000abaa6e08d59526e0-006312f92b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.008 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:19.009 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:19.010 DEBUG:boto:Method: HEAD 2022-09-03T06:50:19.011 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:50:19.012 DEBUG:boto:Data: 2022-09-03T06:50:19.013 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.014 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:19.015 DEBUG:boto:Port: 8000 2022-09-03T06:50:19.016 DEBUG:boto:Params: {} 2022-09-03T06:50:19.017 DEBUG:boto:Token: None 2022-09-03T06:50:19.018 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj55 2022-09-03T06:50:19.019 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:R5blDydms1dEHR2EuyuHui4KaXg= 2022-09-03T06:50:19.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:R5blDydms1dEHR2EuyuHui4KaXg=', 'Content-Length': '0'} 2022-09-03T06:50:19.023 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005bf8cae265bdf1e8-006312f92b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.025 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:50:19.025 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:19.026 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:19.027 DEBUG:boto:Method: GET 2022-09-03T06:50:19.028 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:50:19.030 DEBUG:boto:Data: 2022-09-03T06:50:19.031 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.032 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:19.033 DEBUG:boto:Port: 8001 2022-09-03T06:50:19.034 DEBUG:boto:Params: {} 2022-09-03T06:50:19.035 DEBUG:boto:Token: None 2022-09-03T06:50:19.036 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj55 2022-09-03T06:50:19.037 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KEzC/0apTdwlnFbJiV2KjFxFSlo= 2022-09-03T06:50:19.038 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KEzC/0apTdwlnFbJiV2KjFxFSlo=', 'Content-Length': '0'} 2022-09-03T06:50:19.041 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c5e4d6424dcd108d-006312f92b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.083 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:19.085 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:19.085 DEBUG:boto:Method: GET 2022-09-03T06:50:19.086 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:50:19.087 DEBUG:boto:Data: 2022-09-03T06:50:19.088 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.089 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:19.090 DEBUG:boto:Port: 8000 2022-09-03T06:50:19.091 DEBUG:boto:Params: {} 2022-09-03T06:50:19.091 DEBUG:boto:Token: None 2022-09-03T06:50:19.092 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj55 2022-09-03T06:50:19.092 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KEzC/0apTdwlnFbJiV2KjFxFSlo= 2022-09-03T06:50:19.093 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KEzC/0apTdwlnFbJiV2KjFxFSlo=', 'Content-Length': '0'} 2022-09-03T06:50:19.096 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000089556c80e939b8b3-006312f92b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.097 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:19.097 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:19.098 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:19.098 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:19.099 DEBUG:boto:Method: GET 2022-09-03T06:50:19.100 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:50:19.101 DEBUG:boto:Data: 2022-09-03T06:50:19.103 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.104 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:19.104 DEBUG:boto:Port: 8001 2022-09-03T06:50:19.105 DEBUG:boto:Params: {} 2022-09-03T06:50:19.105 DEBUG:boto:Token: None 2022-09-03T06:50:19.106 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/ 2022-09-03T06:50:19.107 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H+cg3iF0OPwtpX0ObtXleACnfD8= 2022-09-03T06:50:19.108 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H+cg3iF0OPwtpX0ObtXleACnfD8=', 'Content-Length': '0'} 2022-09-03T06:50:19.111 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e890c65ecf5f7380-006312f92b-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.113 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:19.114 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:19.115 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:19.116 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:19.117 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:19.118 DEBUG:boto:Method: GET 2022-09-03T06:50:19.119 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:50:19.119 DEBUG:boto:Data: 2022-09-03T06:50:19.120 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.121 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:19.122 DEBUG:boto:Port: 8000 2022-09-03T06:50:19.123 DEBUG:boto:Params: {} 2022-09-03T06:50:19.124 DEBUG:boto:Token: None 2022-09-03T06:50:19.124 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/ 2022-09-03T06:50:19.125 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H+cg3iF0OPwtpX0ObtXleACnfD8= 2022-09-03T06:50:19.126 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H+cg3iF0OPwtpX0ObtXleACnfD8=', 'Content-Length': '0'} 2022-09-03T06:50:19.131 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bbcad52c93ba73d3-006312f92b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.132 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:19.133 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:50:19.134 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:19.135 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:19.136 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.137 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.139 DEBUG:boto:Method: GET 2022-09-03T06:50:19.140 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.141 DEBUG:boto:Data: 2022-09-03T06:50:19.141 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.143 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:19.143 DEBUG:boto:Port: 8001 2022-09-03T06:50:19.144 DEBUG:boto:Params: {} 2022-09-03T06:50:19.145 DEBUG:boto:Token: None 2022-09-03T06:50:19.145 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.146 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:71CrzqtG5hKqX63CFv0EJ5DKoaE= 2022-09-03T06:50:19.147 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:71CrzqtG5hKqX63CFv0EJ5DKoaE=', 'Content-Length': '0'} 2022-09-03T06:50:19.149 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b8d8c0a4595f393-006312f92b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.150 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:19.151 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:19.152 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.152 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.153 DEBUG:boto:Method: GET 2022-09-03T06:50:19.154 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.155 DEBUG:boto:Data: 2022-09-03T06:50:19.156 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.157 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:19.157 DEBUG:boto:Port: 8000 2022-09-03T06:50:19.158 DEBUG:boto:Params: {} 2022-09-03T06:50:19.159 DEBUG:boto:Token: None 2022-09-03T06:50:19.159 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.160 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:71CrzqtG5hKqX63CFv0EJ5DKoaE= 2022-09-03T06:50:19.161 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:71CrzqtG5hKqX63CFv0EJ5DKoaE=', 'Content-Length': '0'} 2022-09-03T06:50:19.164 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a1bd1aee057445e3-006312f92b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.165 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:19.166 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:19.167 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.168 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.169 DEBUG:boto:Method: HEAD 2022-09-03T06:50:19.170 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.171 DEBUG:boto:Data: 2022-09-03T06:50:19.172 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.173 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:19.173 DEBUG:boto:Port: 8001 2022-09-03T06:50:19.174 DEBUG:boto:Params: {} 2022-09-03T06:50:19.175 DEBUG:boto:Token: None 2022-09-03T06:50:19.175 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.176 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PZxQ7l0gG17IzWfRwIQ3gLKaz10= 2022-09-03T06:50:19.177 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PZxQ7l0gG17IzWfRwIQ3gLKaz10=', 'Content-Length': '0'} 2022-09-03T06:50:19.179 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d49788c57ece0424-006312f92b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.179 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:19.180 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:19.181 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.181 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.182 DEBUG:boto:Method: HEAD 2022-09-03T06:50:19.182 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.183 DEBUG:boto:Data: 2022-09-03T06:50:19.183 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.192 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:19.193 DEBUG:boto:Port: 8000 2022-09-03T06:50:19.194 DEBUG:boto:Params: {} 2022-09-03T06:50:19.195 DEBUG:boto:Token: None 2022-09-03T06:50:19.196 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.197 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PZxQ7l0gG17IzWfRwIQ3gLKaz10= 2022-09-03T06:50:19.198 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PZxQ7l0gG17IzWfRwIQ3gLKaz10=', 'Content-Length': '0'} 2022-09-03T06:50:19.201 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000035d5929359be93a7-006312f92b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.202 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:50:19.203 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:19.203 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:19.204 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.204 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.205 DEBUG:boto:Method: GET 2022-09-03T06:50:19.206 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.206 DEBUG:boto:Data: 2022-09-03T06:50:19.207 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.207 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:19.208 DEBUG:boto:Port: 8001 2022-09-03T06:50:19.209 DEBUG:boto:Params: {} 2022-09-03T06:50:19.209 DEBUG:boto:Token: None 2022-09-03T06:50:19.210 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.212 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:71CrzqtG5hKqX63CFv0EJ5DKoaE= 2022-09-03T06:50:19.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:71CrzqtG5hKqX63CFv0EJ5DKoaE=', 'Content-Length': '0'} 2022-09-03T06:50:19.216 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2ff169cc652abc5-006312f92b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.263 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:19.264 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:19.265 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.265 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.266 DEBUG:boto:Method: GET 2022-09-03T06:50:19.266 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.267 DEBUG:boto:Data: 2022-09-03T06:50:19.268 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.268 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:19.269 DEBUG:boto:Port: 8000 2022-09-03T06:50:19.270 DEBUG:boto:Params: {} 2022-09-03T06:50:19.271 DEBUG:boto:Token: None 2022-09-03T06:50:19.271 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:19.272 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:71CrzqtG5hKqX63CFv0EJ5DKoaE= 2022-09-03T06:50:19.273 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:71CrzqtG5hKqX63CFv0EJ5DKoaE=', 'Content-Length': '0'} 2022-09-03T06:50:19.276 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a8d75802804b7299-006312f92b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.278 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:19.279 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:19.279 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:19.280 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:19.281 DEBUG:boto:Method: GET 2022-09-03T06:50:19.281 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:50:19.282 DEBUG:boto:Data: 2022-09-03T06:50:19.283 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.283 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:19.284 DEBUG:boto:Port: 8001 2022-09-03T06:50:19.285 DEBUG:boto:Params: {} 2022-09-03T06:50:19.285 DEBUG:boto:Token: None 2022-09-03T06:50:19.286 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/ 2022-09-03T06:50:19.287 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H+cg3iF0OPwtpX0ObtXleACnfD8= 2022-09-03T06:50:19.287 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H+cg3iF0OPwtpX0ObtXleACnfD8=', 'Content-Length': '0'} 2022-09-03T06:50:19.294 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003711e69c942f7a85-006312f92b-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.296 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:19.298 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:19.299 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:19.299 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:19.300 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:19.301 DEBUG:boto:Method: GET 2022-09-03T06:50:19.302 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:50:19.302 DEBUG:boto:Data: 2022-09-03T06:50:19.303 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.303 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:19.304 DEBUG:boto:Port: 8000 2022-09-03T06:50:19.305 DEBUG:boto:Params: {} 2022-09-03T06:50:19.306 DEBUG:boto:Token: None 2022-09-03T06:50:19.307 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/ 2022-09-03T06:50:19.308 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H+cg3iF0OPwtpX0ObtXleACnfD8= 2022-09-03T06:50:19.309 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H+cg3iF0OPwtpX0ObtXleACnfD8=', 'Content-Length': '0'} 2022-09-03T06:50:19.314 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c591708f08a7117a-006312f92b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.316 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:19.317 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:19.318 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:19.319 DEBUG:boto:Method: HEAD 2022-09-03T06:50:19.320 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:50:19.321 DEBUG:boto:Data: 2022-09-03T06:50:19.322 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.323 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:19.323 DEBUG:boto:Port: 8001 2022-09-03T06:50:19.324 DEBUG:boto:Params: {} 2022-09-03T06:50:19.325 DEBUG:boto:Token: None 2022-09-03T06:50:19.325 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj57 2022-09-03T06:50:19.326 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZLq3gIuSbBgfomQ9JxkTgeNC4Cw= 2022-09-03T06:50:19.327 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZLq3gIuSbBgfomQ9JxkTgeNC4Cw=', 'Content-Length': '0'} 2022-09-03T06:50:19.329 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000a10851edce5a3ef-006312f92b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.330 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:19.330 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:19.331 DEBUG:boto:Method: HEAD 2022-09-03T06:50:19.332 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:50:19.333 DEBUG:boto:Data: 2022-09-03T06:50:19.333 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.334 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:19.335 DEBUG:boto:Port: 8000 2022-09-03T06:50:19.336 DEBUG:boto:Params: {} 2022-09-03T06:50:19.337 DEBUG:boto:Token: None 2022-09-03T06:50:19.338 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj57 2022-09-03T06:50:19.339 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZLq3gIuSbBgfomQ9JxkTgeNC4Cw= 2022-09-03T06:50:19.341 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZLq3gIuSbBgfomQ9JxkTgeNC4Cw=', 'Content-Length': '0'} 2022-09-03T06:50:19.350 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007293b157327285ea-006312f92b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.351 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:50:19.352 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:19.354 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:19.355 DEBUG:boto:Method: GET 2022-09-03T06:50:19.356 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:50:19.357 DEBUG:boto:Data: 2022-09-03T06:50:19.358 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.360 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:19.362 DEBUG:boto:Port: 8001 2022-09-03T06:50:19.363 DEBUG:boto:Params: {} 2022-09-03T06:50:19.364 DEBUG:boto:Token: None 2022-09-03T06:50:19.366 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj57 2022-09-03T06:50:19.367 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ehNnv5c/bGWLoB5fhoK7rlrGzEQ= 2022-09-03T06:50:19.368 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ehNnv5c/bGWLoB5fhoK7rlrGzEQ=', 'Content-Length': '0'} 2022-09-03T06:50:19.371 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000535511059e4e1829-006312f92b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.372 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:19.373 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:19.374 DEBUG:boto:Method: GET 2022-09-03T06:50:19.375 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:50:19.376 DEBUG:boto:Data: 2022-09-03T06:50:19.377 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.379 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:19.383 DEBUG:boto:Port: 8000 2022-09-03T06:50:19.384 DEBUG:boto:Params: {} 2022-09-03T06:50:19.387 DEBUG:boto:Token: None 2022-09-03T06:50:19.388 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj57 2022-09-03T06:50:19.390 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ehNnv5c/bGWLoB5fhoK7rlrGzEQ= 2022-09-03T06:50:19.391 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ehNnv5c/bGWLoB5fhoK7rlrGzEQ=', 'Content-Length': '0'} 2022-09-03T06:50:19.397 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006da9e5d223b89d6e-006312f92b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.398 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:19.399 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:19.400 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:19.401 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:19.402 DEBUG:boto:Method: GET 2022-09-03T06:50:19.403 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:50:19.405 DEBUG:boto:Data: 2022-09-03T06:50:19.405 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.406 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:19.407 DEBUG:boto:Port: 8001 2022-09-03T06:50:19.407 DEBUG:boto:Params: {} 2022-09-03T06:50:19.408 DEBUG:boto:Token: None 2022-09-03T06:50:19.408 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/ 2022-09-03T06:50:19.409 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H+cg3iF0OPwtpX0ObtXleACnfD8= 2022-09-03T06:50:19.410 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H+cg3iF0OPwtpX0ObtXleACnfD8=', 'Content-Length': '0'} 2022-09-03T06:50:19.416 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003eb73c01e4ad2234-006312f92b-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.418 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:19.419 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:19.420 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:19.420 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:19.421 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:19.421 DEBUG:boto:Method: GET 2022-09-03T06:50:19.422 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:50:19.422 DEBUG:boto:Data: 2022-09-03T06:50:19.425 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.426 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:19.427 DEBUG:boto:Port: 8000 2022-09-03T06:50:19.428 DEBUG:boto:Params: {} 2022-09-03T06:50:19.430 DEBUG:boto:Token: None 2022-09-03T06:50:19.431 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/ 2022-09-03T06:50:19.432 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H+cg3iF0OPwtpX0ObtXleACnfD8= 2022-09-03T06:50:19.433 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H+cg3iF0OPwtpX0ObtXleACnfD8=', 'Content-Length': '0'} 2022-09-03T06:50:19.445 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000048cff34375992d4b-006312f92b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.446 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:19.448 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:50:19.449 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:19.450 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:19.452 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.453 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.454 DEBUG:boto:Method: GET 2022-09-03T06:50:19.455 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.456 DEBUG:boto:Data: 2022-09-03T06:50:19.458 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.459 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:19.460 DEBUG:boto:Port: 8001 2022-09-03T06:50:19.463 DEBUG:boto:Params: {} 2022-09-03T06:50:19.464 DEBUG:boto:Token: None 2022-09-03T06:50:19.465 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.466 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9hlP+2i2d8t/UdjQ9fwtxzTifR8= 2022-09-03T06:50:19.467 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9hlP+2i2d8t/UdjQ9fwtxzTifR8=', 'Content-Length': '0'} 2022-09-03T06:50:19.476 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006ab36ed6a9978331-006312f92b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.477 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:19.478 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:19.479 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.480 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.481 DEBUG:boto:Method: GET 2022-09-03T06:50:19.482 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.483 DEBUG:boto:Data: 2022-09-03T06:50:19.484 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.485 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:19.486 DEBUG:boto:Port: 8000 2022-09-03T06:50:19.487 DEBUG:boto:Params: {} 2022-09-03T06:50:19.488 DEBUG:boto:Token: None 2022-09-03T06:50:19.489 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.490 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9hlP+2i2d8t/UdjQ9fwtxzTifR8= 2022-09-03T06:50:19.492 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9hlP+2i2d8t/UdjQ9fwtxzTifR8=', 'Content-Length': '0'} 2022-09-03T06:50:19.495 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c0f7096db13f1892-006312f92b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.550 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:19.555 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:19.556 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.557 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.558 DEBUG:boto:Method: HEAD 2022-09-03T06:50:19.559 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.560 DEBUG:boto:Data: 2022-09-03T06:50:19.561 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.561 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:19.565 DEBUG:boto:Port: 8001 2022-09-03T06:50:19.570 DEBUG:boto:Params: {} 2022-09-03T06:50:19.571 DEBUG:boto:Token: None 2022-09-03T06:50:19.578 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.581 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UinpemNTni2Pm7tB8NeltGUoCJQ= 2022-09-03T06:50:19.583 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UinpemNTni2Pm7tB8NeltGUoCJQ=', 'Content-Length': '0'} 2022-09-03T06:50:19.604 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007f4bc1a81000ff06-006312f92b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.606 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:19.608 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:19.609 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.610 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.611 DEBUG:boto:Method: HEAD 2022-09-03T06:50:19.612 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.612 DEBUG:boto:Data: 2022-09-03T06:50:19.613 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.614 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:19.615 DEBUG:boto:Port: 8000 2022-09-03T06:50:19.616 DEBUG:boto:Params: {} 2022-09-03T06:50:19.617 DEBUG:boto:Token: None 2022-09-03T06:50:19.621 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.623 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UinpemNTni2Pm7tB8NeltGUoCJQ= 2022-09-03T06:50:19.624 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UinpemNTni2Pm7tB8NeltGUoCJQ=', 'Content-Length': '0'} 2022-09-03T06:50:19.629 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003fb8eadd0ba40130-006312f92b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.630 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:50:19.631 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:19.632 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:19.633 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.634 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.635 DEBUG:boto:Method: GET 2022-09-03T06:50:19.636 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.638 DEBUG:boto:Data: 2022-09-03T06:50:19.639 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.639 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:19.640 DEBUG:boto:Port: 8001 2022-09-03T06:50:19.641 DEBUG:boto:Params: {} 2022-09-03T06:50:19.641 DEBUG:boto:Token: None 2022-09-03T06:50:19.642 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.643 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9hlP+2i2d8t/UdjQ9fwtxzTifR8= 2022-09-03T06:50:19.643 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9hlP+2i2d8t/UdjQ9fwtxzTifR8=', 'Content-Length': '0'} 2022-09-03T06:50:19.646 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000354227473f39ab18-006312f92b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.648 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:19.649 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:19.650 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.651 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.652 DEBUG:boto:Method: GET 2022-09-03T06:50:19.653 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.654 DEBUG:boto:Data: 2022-09-03T06:50:19.655 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.656 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:19.657 DEBUG:boto:Port: 8000 2022-09-03T06:50:19.658 DEBUG:boto:Params: {} 2022-09-03T06:50:19.659 DEBUG:boto:Token: None 2022-09-03T06:50:19.660 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:19.661 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9hlP+2i2d8t/UdjQ9fwtxzTifR8= 2022-09-03T06:50:19.662 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9hlP+2i2d8t/UdjQ9fwtxzTifR8=', 'Content-Length': '0'} 2022-09-03T06:50:19.665 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000030371869a16bbfbf-006312f92b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.667 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:19.668 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:19.669 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:19.670 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:19.671 DEBUG:boto:Method: GET 2022-09-03T06:50:19.672 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:50:19.673 DEBUG:boto:Data: 2022-09-03T06:50:19.674 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.677 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:19.678 DEBUG:boto:Port: 8001 2022-09-03T06:50:19.679 DEBUG:boto:Params: {} 2022-09-03T06:50:19.680 DEBUG:boto:Token: None 2022-09-03T06:50:19.681 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/ 2022-09-03T06:50:19.682 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H+cg3iF0OPwtpX0ObtXleACnfD8= 2022-09-03T06:50:19.683 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H+cg3iF0OPwtpX0ObtXleACnfD8=', 'Content-Length': '0'} 2022-09-03T06:50:19.687 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000da1dd6c251890bf1-006312f92b-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.688 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:19.690 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:19.691 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:19.692 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:19.693 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:19.694 DEBUG:boto:Method: GET 2022-09-03T06:50:19.695 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:50:19.696 DEBUG:boto:Data: 2022-09-03T06:50:19.697 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.698 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:19.699 DEBUG:boto:Port: 8000 2022-09-03T06:50:19.700 DEBUG:boto:Params: {} 2022-09-03T06:50:19.701 DEBUG:boto:Token: None 2022-09-03T06:50:19.702 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/ 2022-09-03T06:50:19.703 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H+cg3iF0OPwtpX0ObtXleACnfD8= 2022-09-03T06:50:19.703 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H+cg3iF0OPwtpX0ObtXleACnfD8=', 'Content-Length': '0'} 2022-09-03T06:50:19.709 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ef231ac016d9c2ed-006312f92b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.710 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:19.712 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:19.713 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:19.714 DEBUG:boto:Method: HEAD 2022-09-03T06:50:19.715 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:50:19.715 DEBUG:boto:Data: 2022-09-03T06:50:19.716 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.716 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:19.717 DEBUG:boto:Port: 8001 2022-09-03T06:50:19.717 DEBUG:boto:Params: {} 2022-09-03T06:50:19.718 DEBUG:boto:Token: None 2022-09-03T06:50:19.719 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj58 2022-09-03T06:50:19.720 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GtjEGF4n3tflWuciKcFjZsXReEQ= 2022-09-03T06:50:19.722 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GtjEGF4n3tflWuciKcFjZsXReEQ=', 'Content-Length': '0'} 2022-09-03T06:50:19.725 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d3c4566f2eff32ca-006312f92b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.727 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:19.728 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:19.729 DEBUG:boto:Method: HEAD 2022-09-03T06:50:19.730 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:50:19.732 DEBUG:boto:Data: 2022-09-03T06:50:19.733 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.734 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:19.735 DEBUG:boto:Port: 8000 2022-09-03T06:50:19.736 DEBUG:boto:Params: {} 2022-09-03T06:50:19.737 DEBUG:boto:Token: None 2022-09-03T06:50:19.739 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj58 2022-09-03T06:50:19.740 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GtjEGF4n3tflWuciKcFjZsXReEQ= 2022-09-03T06:50:19.741 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GtjEGF4n3tflWuciKcFjZsXReEQ=', 'Content-Length': '0'} 2022-09-03T06:50:19.744 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d4c1f69941453f98-006312f92b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.745 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:50:19.747 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:19.748 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:19.749 DEBUG:boto:Method: GET 2022-09-03T06:50:19.750 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:50:19.751 DEBUG:boto:Data: 2022-09-03T06:50:19.751 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.752 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:19.753 DEBUG:boto:Port: 8001 2022-09-03T06:50:19.753 DEBUG:boto:Params: {} 2022-09-03T06:50:19.754 DEBUG:boto:Token: None 2022-09-03T06:50:19.755 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj58 2022-09-03T06:50:19.755 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UZePief9bQxts1XqOimwuNAfYDI= 2022-09-03T06:50:19.756 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UZePief9bQxts1XqOimwuNAfYDI=', 'Content-Length': '0'} 2022-09-03T06:50:19.758 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aa47120996c8cf18-006312f92b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.814 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:19.815 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:19.815 DEBUG:boto:Method: GET 2022-09-03T06:50:19.816 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:50:19.816 DEBUG:boto:Data: 2022-09-03T06:50:19.817 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.819 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:19.819 DEBUG:boto:Port: 8000 2022-09-03T06:50:19.820 DEBUG:boto:Params: {} 2022-09-03T06:50:19.821 DEBUG:boto:Token: None 2022-09-03T06:50:19.821 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj58 2022-09-03T06:50:19.823 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UZePief9bQxts1XqOimwuNAfYDI= 2022-09-03T06:50:19.823 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UZePief9bQxts1XqOimwuNAfYDI=', 'Content-Length': '0'} 2022-09-03T06:50:19.826 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000069ad30a6abf6f844-006312f92b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.827 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:19.827 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:19.828 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:19.828 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:19.829 DEBUG:boto:Method: GET 2022-09-03T06:50:19.831 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:50:19.832 DEBUG:boto:Data: 2022-09-03T06:50:19.833 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.835 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:19.836 DEBUG:boto:Port: 8001 2022-09-03T06:50:19.837 DEBUG:boto:Params: {} 2022-09-03T06:50:19.838 DEBUG:boto:Token: None 2022-09-03T06:50:19.840 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/ 2022-09-03T06:50:19.841 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H+cg3iF0OPwtpX0ObtXleACnfD8= 2022-09-03T06:50:19.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H+cg3iF0OPwtpX0ObtXleACnfD8=', 'Content-Length': '0'} 2022-09-03T06:50:19.851 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000985cf0b81a0e2503-006312f92b-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.852 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:19.854 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:19.855 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:19.856 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:19.857 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:19.858 DEBUG:boto:Method: GET 2022-09-03T06:50:19.859 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:50:19.861 DEBUG:boto:Data: 2022-09-03T06:50:19.861 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.862 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:19.863 DEBUG:boto:Port: 8000 2022-09-03T06:50:19.863 DEBUG:boto:Params: {} 2022-09-03T06:50:19.864 DEBUG:boto:Token: None 2022-09-03T06:50:19.865 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/ 2022-09-03T06:50:19.866 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H+cg3iF0OPwtpX0ObtXleACnfD8= 2022-09-03T06:50:19.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H+cg3iF0OPwtpX0ObtXleACnfD8=', 'Content-Length': '0'} 2022-09-03T06:50:19.873 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000018f191335321ae9f-006312f92b-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.874 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:19.875 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2022-09-03T06:50:19.876 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:19.877 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:19.877 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:19.878 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:19.879 DEBUG:boto:Method: GET 2022-09-03T06:50:19.879 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:19.880 DEBUG:boto:Data: 2022-09-03T06:50:19.881 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.882 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:19.883 DEBUG:boto:Port: 8001 2022-09-03T06:50:19.883 DEBUG:boto:Params: {} 2022-09-03T06:50:19.885 DEBUG:boto:Token: None 2022-09-03T06:50:19.886 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:19.888 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZxvxefO//Bkmt3V0/4h+gd6Y7H4= 2022-09-03T06:50:19.889 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZxvxefO//Bkmt3V0/4h+gd6Y7H4=', 'Content-Length': '0'} 2022-09-03T06:50:19.891 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000047c13fa4a16fa83-006312f92b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.892 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:19.892 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:19.893 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:19.894 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:19.894 DEBUG:boto:Method: GET 2022-09-03T06:50:19.895 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:19.895 DEBUG:boto:Data: 2022-09-03T06:50:19.896 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.897 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:19.897 DEBUG:boto:Port: 8000 2022-09-03T06:50:19.898 DEBUG:boto:Params: {} 2022-09-03T06:50:19.898 DEBUG:boto:Token: None 2022-09-03T06:50:19.899 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:19.900 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZxvxefO//Bkmt3V0/4h+gd6Y7H4= 2022-09-03T06:50:19.900 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZxvxefO//Bkmt3V0/4h+gd6Y7H4=', 'Content-Length': '0'} 2022-09-03T06:50:19.904 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001952401268905878-006312f92b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.947 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:19.951 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:19.952 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:19.953 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:19.953 DEBUG:boto:Method: HEAD 2022-09-03T06:50:19.954 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:19.954 DEBUG:boto:Data: 2022-09-03T06:50:19.955 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.955 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:19.956 DEBUG:boto:Port: 8001 2022-09-03T06:50:19.956 DEBUG:boto:Params: {} 2022-09-03T06:50:19.957 DEBUG:boto:Token: None 2022-09-03T06:50:19.964 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:19.965 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GmcTRMf7SutXOQFOic0FhWXX8mY= 2022-09-03T06:50:19.966 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GmcTRMf7SutXOQFOic0FhWXX8mY=', 'Content-Length': '0'} 2022-09-03T06:50:19.969 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000df9df739792f58fb-006312f92b-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.971 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:19.973 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:19.973 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:19.974 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:19.975 DEBUG:boto:Method: HEAD 2022-09-03T06:50:19.976 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:19.977 DEBUG:boto:Data: 2022-09-03T06:50:19.977 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.978 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:19.979 DEBUG:boto:Port: 8000 2022-09-03T06:50:19.980 DEBUG:boto:Params: {} 2022-09-03T06:50:19.981 DEBUG:boto:Token: None 2022-09-03T06:50:19.982 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:19 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:19.983 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GmcTRMf7SutXOQFOic0FhWXX8mY= 2022-09-03T06:50:19.985 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:19 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GmcTRMf7SutXOQFOic0FhWXX8mY=', 'Content-Length': '0'} 2022-09-03T06:50:19.987 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009120b76831519c77-006312f92b-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:19 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:19.989 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2022-09-03T06:50:19.990 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:19.991 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:19.992 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:19.993 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:19.993 DEBUG:boto:Method: GET 2022-09-03T06:50:19.994 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:19.995 DEBUG:boto:Data: 2022-09-03T06:50:19.996 DEBUG:boto:Headers: {} 2022-09-03T06:50:19.997 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:19.998 DEBUG:boto:Port: 8001 2022-09-03T06:50:19.999 DEBUG:boto:Params: {} 2022-09-03T06:50:20.000 DEBUG:boto:Token: None 2022-09-03T06:50:20.001 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:20.002 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WmrhGbj8EWEnoO00Od0KhzyOSXo= 2022-09-03T06:50:20.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WmrhGbj8EWEnoO00Od0KhzyOSXo=', 'Content-Length': '0'} 2022-09-03T06:50:20.005 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c7d55e2172dbb7a6-006312f92c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.006 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:20.006 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:20.008 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:20.009 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:20.011 DEBUG:boto:Method: GET 2022-09-03T06:50:20.012 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:20.013 DEBUG:boto:Data: 2022-09-03T06:50:20.013 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.015 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:20.016 DEBUG:boto:Port: 8000 2022-09-03T06:50:20.017 DEBUG:boto:Params: {} 2022-09-03T06:50:20.018 DEBUG:boto:Token: None 2022-09-03T06:50:20.022 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:20.023 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WmrhGbj8EWEnoO00Od0KhzyOSXo= 2022-09-03T06:50:20.024 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WmrhGbj8EWEnoO00Od0KhzyOSXo=', 'Content-Length': '0'} 2022-09-03T06:50:20.026 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000020d3537da0cfffbe-006312f92c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.027 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:20.027 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:20.028 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:20.029 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:20.029 DEBUG:boto:Method: GET 2022-09-03T06:50:20.030 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:50:20.030 DEBUG:boto:Data: 2022-09-03T06:50:20.031 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.032 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:20.033 DEBUG:boto:Port: 8001 2022-09-03T06:50:20.034 DEBUG:boto:Params: {} 2022-09-03T06:50:20.035 DEBUG:boto:Token: None 2022-09-03T06:50:20.036 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/ 2022-09-03T06:50:20.037 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY= 2022-09-03T06:50:20.039 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY=', 'Content-Length': '0'} 2022-09-03T06:50:20.042 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002d029a7f57d098d2-006312f92c-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.043 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:20.045 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:20.046 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:20.047 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:20.048 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:20.054 DEBUG:boto:Method: GET 2022-09-03T06:50:20.056 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:50:20.057 DEBUG:boto:Data: 2022-09-03T06:50:20.058 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.059 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:20.060 DEBUG:boto:Port: 8000 2022-09-03T06:50:20.062 DEBUG:boto:Params: {} 2022-09-03T06:50:20.063 DEBUG:boto:Token: None 2022-09-03T06:50:20.064 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/ 2022-09-03T06:50:20.065 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY= 2022-09-03T06:50:20.065 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY=', 'Content-Length': '0'} 2022-09-03T06:50:20.071 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000352ebfc9cba8883d-006312f92c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.072 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:20.075 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:20.076 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:20.077 DEBUG:boto:Method: HEAD 2022-09-03T06:50:20.078 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:50:20.079 DEBUG:boto:Data: 2022-09-03T06:50:20.080 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.081 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:20.082 DEBUG:boto:Port: 8001 2022-09-03T06:50:20.083 DEBUG:boto:Params: {} 2022-09-03T06:50:20.084 DEBUG:boto:Token: None 2022-09-03T06:50:20.085 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj62 2022-09-03T06:50:20.086 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0tQIZ1wNGBkckEZoabLbC2uA+nM= 2022-09-03T06:50:20.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0tQIZ1wNGBkckEZoabLbC2uA+nM=', 'Content-Length': '0'} 2022-09-03T06:50:20.093 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000708cb4c2892d8369-006312f92c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.095 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:20.096 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:20.097 DEBUG:boto:Method: HEAD 2022-09-03T06:50:20.097 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:50:20.098 DEBUG:boto:Data: 2022-09-03T06:50:20.099 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.100 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:20.103 DEBUG:boto:Port: 8000 2022-09-03T06:50:20.106 DEBUG:boto:Params: {} 2022-09-03T06:50:20.107 DEBUG:boto:Token: None 2022-09-03T06:50:20.107 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj62 2022-09-03T06:50:20.108 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0tQIZ1wNGBkckEZoabLbC2uA+nM= 2022-09-03T06:50:20.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0tQIZ1wNGBkckEZoabLbC2uA+nM=', 'Content-Length': '0'} 2022-09-03T06:50:20.111 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee21d886a79930c6-006312f92c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.113 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2022-09-03T06:50:20.114 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:20.115 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:20.116 DEBUG:boto:Method: GET 2022-09-03T06:50:20.118 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:50:20.119 DEBUG:boto:Data: 2022-09-03T06:50:20.120 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.121 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:20.123 DEBUG:boto:Port: 8001 2022-09-03T06:50:20.124 DEBUG:boto:Params: {} 2022-09-03T06:50:20.125 DEBUG:boto:Token: None 2022-09-03T06:50:20.126 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj62 2022-09-03T06:50:20.127 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aRhVw0BhJNGS1miPTFE4UvXSZhE= 2022-09-03T06:50:20.129 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aRhVw0BhJNGS1miPTFE4UvXSZhE=', 'Content-Length': '0'} 2022-09-03T06:50:20.132 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007093d258af5fd2cc-006312f92c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.133 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:20.134 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:20.135 DEBUG:boto:Method: GET 2022-09-03T06:50:20.136 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:50:20.137 DEBUG:boto:Data: 2022-09-03T06:50:20.138 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.139 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:20.142 DEBUG:boto:Port: 8000 2022-09-03T06:50:20.142 DEBUG:boto:Params: {} 2022-09-03T06:50:20.143 DEBUG:boto:Token: None 2022-09-03T06:50:20.144 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj62 2022-09-03T06:50:20.144 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aRhVw0BhJNGS1miPTFE4UvXSZhE= 2022-09-03T06:50:20.145 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aRhVw0BhJNGS1miPTFE4UvXSZhE=', 'Content-Length': '0'} 2022-09-03T06:50:20.148 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f8a06f7b77703d5c-006312f92c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.194 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:20.195 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:20.195 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:20.196 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:20.197 DEBUG:boto:Method: GET 2022-09-03T06:50:20.197 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:50:20.198 DEBUG:boto:Data: 2022-09-03T06:50:20.198 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.199 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:20.200 DEBUG:boto:Port: 8001 2022-09-03T06:50:20.201 DEBUG:boto:Params: {} 2022-09-03T06:50:20.202 DEBUG:boto:Token: None 2022-09-03T06:50:20.205 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/ 2022-09-03T06:50:20.208 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY= 2022-09-03T06:50:20.209 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY=', 'Content-Length': '0'} 2022-09-03T06:50:20.213 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a3e322e41a857733-006312f92c-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.215 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:20.217 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:20.218 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:20.219 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:20.221 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:20.222 DEBUG:boto:Method: GET 2022-09-03T06:50:20.223 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:50:20.224 DEBUG:boto:Data: 2022-09-03T06:50:20.226 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.228 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:20.228 DEBUG:boto:Port: 8000 2022-09-03T06:50:20.229 DEBUG:boto:Params: {} 2022-09-03T06:50:20.230 DEBUG:boto:Token: None 2022-09-03T06:50:20.230 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/ 2022-09-03T06:50:20.231 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY= 2022-09-03T06:50:20.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY=', 'Content-Length': '0'} 2022-09-03T06:50:20.236 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b2c77d4442c707df-006312f92c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.238 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:20.239 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:50:20.240 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:20.240 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:20.241 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.241 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.242 DEBUG:boto:Method: GET 2022-09-03T06:50:20.243 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.244 DEBUG:boto:Data: 2022-09-03T06:50:20.245 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.246 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:20.247 DEBUG:boto:Port: 8001 2022-09-03T06:50:20.248 DEBUG:boto:Params: {} 2022-09-03T06:50:20.249 DEBUG:boto:Token: None 2022-09-03T06:50:20.250 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.251 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DzSe+MQRXiLm3kHYxfqE1ewsvPI= 2022-09-03T06:50:20.252 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DzSe+MQRXiLm3kHYxfqE1ewsvPI=', 'Content-Length': '0'} 2022-09-03T06:50:20.256 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007e91b617264e735d-006312f92c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.257 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:20.258 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:20.259 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.260 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.261 DEBUG:boto:Method: GET 2022-09-03T06:50:20.262 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.265 DEBUG:boto:Data: 2022-09-03T06:50:20.266 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.267 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:20.269 DEBUG:boto:Port: 8000 2022-09-03T06:50:20.270 DEBUG:boto:Params: {} 2022-09-03T06:50:20.271 DEBUG:boto:Token: None 2022-09-03T06:50:20.272 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.273 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DzSe+MQRXiLm3kHYxfqE1ewsvPI= 2022-09-03T06:50:20.274 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DzSe+MQRXiLm3kHYxfqE1ewsvPI=', 'Content-Length': '0'} 2022-09-03T06:50:20.279 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c391d19fa66e8fb2-006312f92c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.281 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:20.282 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:20.283 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.284 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.285 DEBUG:boto:Method: HEAD 2022-09-03T06:50:20.286 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.287 DEBUG:boto:Data: 2022-09-03T06:50:20.288 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.289 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:20.290 DEBUG:boto:Port: 8001 2022-09-03T06:50:20.290 DEBUG:boto:Params: {} 2022-09-03T06:50:20.291 DEBUG:boto:Token: None 2022-09-03T06:50:20.291 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.292 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aHCV5YiV4GS3QbC4Q5VtGG1Dvp4= 2022-09-03T06:50:20.293 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aHCV5YiV4GS3QbC4Q5VtGG1Dvp4=', 'Content-Length': '0'} 2022-09-03T06:50:20.295 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002173b7db6288dbe9-006312f92c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.296 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:20.296 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:20.297 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.297 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.298 DEBUG:boto:Method: HEAD 2022-09-03T06:50:20.298 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.299 DEBUG:boto:Data: 2022-09-03T06:50:20.300 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.300 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:20.301 DEBUG:boto:Port: 8000 2022-09-03T06:50:20.303 DEBUG:boto:Params: {} 2022-09-03T06:50:20.307 DEBUG:boto:Token: None 2022-09-03T06:50:20.308 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.309 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:aHCV5YiV4GS3QbC4Q5VtGG1Dvp4= 2022-09-03T06:50:20.311 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:aHCV5YiV4GS3QbC4Q5VtGG1Dvp4=', 'Content-Length': '0'} 2022-09-03T06:50:20.313 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000066c08f4e3d00082f-006312f92c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.324 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:50:20.326 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:20.326 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:20.327 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.327 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.328 DEBUG:boto:Method: GET 2022-09-03T06:50:20.329 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.329 DEBUG:boto:Data: 2022-09-03T06:50:20.330 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.331 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:20.331 DEBUG:boto:Port: 8001 2022-09-03T06:50:20.332 DEBUG:boto:Params: {} 2022-09-03T06:50:20.333 DEBUG:boto:Token: None 2022-09-03T06:50:20.333 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.334 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DzSe+MQRXiLm3kHYxfqE1ewsvPI= 2022-09-03T06:50:20.335 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DzSe+MQRXiLm3kHYxfqE1ewsvPI=', 'Content-Length': '0'} 2022-09-03T06:50:20.337 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b67b28ffc5ab1fa-006312f92c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.383 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:20.384 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:20.385 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.386 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.386 DEBUG:boto:Method: GET 2022-09-03T06:50:20.387 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.388 DEBUG:boto:Data: 2022-09-03T06:50:20.389 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.390 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:20.390 DEBUG:boto:Port: 8000 2022-09-03T06:50:20.391 DEBUG:boto:Params: {} 2022-09-03T06:50:20.392 DEBUG:boto:Token: None 2022-09-03T06:50:20.394 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:20.394 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DzSe+MQRXiLm3kHYxfqE1ewsvPI= 2022-09-03T06:50:20.395 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DzSe+MQRXiLm3kHYxfqE1ewsvPI=', 'Content-Length': '0'} 2022-09-03T06:50:20.398 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a8475b21284ca5c-006312f92c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.399 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:20.400 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:20.401 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:20.402 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:20.403 DEBUG:boto:Method: GET 2022-09-03T06:50:20.405 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:50:20.406 DEBUG:boto:Data: 2022-09-03T06:50:20.407 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.408 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:20.409 DEBUG:boto:Port: 8001 2022-09-03T06:50:20.411 DEBUG:boto:Params: {} 2022-09-03T06:50:20.412 DEBUG:boto:Token: None 2022-09-03T06:50:20.413 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/ 2022-09-03T06:50:20.420 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY= 2022-09-03T06:50:20.421 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY=', 'Content-Length': '0'} 2022-09-03T06:50:20.425 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cd19c9d3fbbf3626-006312f92c-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.426 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:20.429 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:20.431 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:20.432 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:20.434 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:20.435 DEBUG:boto:Method: GET 2022-09-03T06:50:20.436 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:50:20.437 DEBUG:boto:Data: 2022-09-03T06:50:20.438 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.439 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:20.440 DEBUG:boto:Port: 8000 2022-09-03T06:50:20.441 DEBUG:boto:Params: {} 2022-09-03T06:50:20.442 DEBUG:boto:Token: None 2022-09-03T06:50:20.443 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/ 2022-09-03T06:50:20.444 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY= 2022-09-03T06:50:20.446 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY=', 'Content-Length': '0'} 2022-09-03T06:50:20.454 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f464eee64b70cef3-006312f92c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.456 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:20.457 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:20.458 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:20.459 DEBUG:boto:Method: HEAD 2022-09-03T06:50:20.460 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:50:20.462 DEBUG:boto:Data: 2022-09-03T06:50:20.462 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.463 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:20.464 DEBUG:boto:Port: 8001 2022-09-03T06:50:20.464 DEBUG:boto:Params: {} 2022-09-03T06:50:20.465 DEBUG:boto:Token: None 2022-09-03T06:50:20.466 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj68 2022-09-03T06:50:20.466 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2syg2q3FtnYYJ7k0nOB7Myd0qgU= 2022-09-03T06:50:20.467 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2syg2q3FtnYYJ7k0nOB7Myd0qgU=', 'Content-Length': '0'} 2022-09-03T06:50:20.470 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000daacf59037ab5e10-006312f92c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.471 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:20.472 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:20.473 DEBUG:boto:Method: HEAD 2022-09-03T06:50:20.474 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:50:20.475 DEBUG:boto:Data: 2022-09-03T06:50:20.476 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.477 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:20.478 DEBUG:boto:Port: 8000 2022-09-03T06:50:20.479 DEBUG:boto:Params: {} 2022-09-03T06:50:20.480 DEBUG:boto:Token: None 2022-09-03T06:50:20.482 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj68 2022-09-03T06:50:20.483 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2syg2q3FtnYYJ7k0nOB7Myd0qgU= 2022-09-03T06:50:20.484 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2syg2q3FtnYYJ7k0nOB7Myd0qgU=', 'Content-Length': '0'} 2022-09-03T06:50:20.487 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008fa793f54729c45c-006312f92c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.488 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:50:20.503 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:20.515 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:20.516 DEBUG:boto:Method: GET 2022-09-03T06:50:20.517 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:50:20.518 DEBUG:boto:Data: 2022-09-03T06:50:20.519 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.520 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:20.521 DEBUG:boto:Port: 8001 2022-09-03T06:50:20.522 DEBUG:boto:Params: {} 2022-09-03T06:50:20.524 DEBUG:boto:Token: None 2022-09-03T06:50:20.525 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj68 2022-09-03T06:50:20.526 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vwy1y+5D6FOu13+kGoN/PXO4ui4= 2022-09-03T06:50:20.527 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vwy1y+5D6FOu13+kGoN/PXO4ui4=', 'Content-Length': '0'} 2022-09-03T06:50:20.530 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000072573391ac9a1a15-006312f92c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.531 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:20.532 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:20.532 DEBUG:boto:Method: GET 2022-09-03T06:50:20.533 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:50:20.533 DEBUG:boto:Data: 2022-09-03T06:50:20.534 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.537 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:20.538 DEBUG:boto:Port: 8000 2022-09-03T06:50:20.539 DEBUG:boto:Params: {} 2022-09-03T06:50:20.540 DEBUG:boto:Token: None 2022-09-03T06:50:20.541 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj68 2022-09-03T06:50:20.543 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vwy1y+5D6FOu13+kGoN/PXO4ui4= 2022-09-03T06:50:20.544 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vwy1y+5D6FOu13+kGoN/PXO4ui4=', 'Content-Length': '0'} 2022-09-03T06:50:20.547 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bdd7f8216469ae1a-006312f92c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.548 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:20.549 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:20.550 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:20.550 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:20.551 DEBUG:boto:Method: GET 2022-09-03T06:50:20.552 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:50:20.552 DEBUG:boto:Data: 2022-09-03T06:50:20.553 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.553 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:20.554 DEBUG:boto:Port: 8001 2022-09-03T06:50:20.554 DEBUG:boto:Params: {} 2022-09-03T06:50:20.556 DEBUG:boto:Token: None 2022-09-03T06:50:20.557 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/ 2022-09-03T06:50:20.558 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY= 2022-09-03T06:50:20.559 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY=', 'Content-Length': '0'} 2022-09-03T06:50:20.563 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000961d2cafd24f6065-006312f92c-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.565 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:20.567 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:20.568 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:20.569 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:20.570 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:20.571 DEBUG:boto:Method: GET 2022-09-03T06:50:20.572 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:50:20.573 DEBUG:boto:Data: 2022-09-03T06:50:20.573 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.574 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:20.574 DEBUG:boto:Port: 8000 2022-09-03T06:50:20.575 DEBUG:boto:Params: {} 2022-09-03T06:50:20.579 DEBUG:boto:Token: None 2022-09-03T06:50:20.580 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/ 2022-09-03T06:50:20.580 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY= 2022-09-03T06:50:20.581 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY=', 'Content-Length': '0'} 2022-09-03T06:50:20.586 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004b172a150f3c3cb0-006312f92c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.587 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:20.589 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:50:20.589 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:20.591 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:20.592 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.593 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.594 DEBUG:boto:Method: GET 2022-09-03T06:50:20.595 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.596 DEBUG:boto:Data: 2022-09-03T06:50:20.597 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.599 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:20.599 DEBUG:boto:Port: 8001 2022-09-03T06:50:20.600 DEBUG:boto:Params: {} 2022-09-03T06:50:20.601 DEBUG:boto:Token: None 2022-09-03T06:50:20.602 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.608 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FwJz3HBuncxAsuSb88W6wG3mLw0= 2022-09-03T06:50:20.609 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FwJz3HBuncxAsuSb88W6wG3mLw0=', 'Content-Length': '0'} 2022-09-03T06:50:20.612 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e19861fa3032f9cd-006312f92c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.613 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:20.614 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:20.615 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.616 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.616 DEBUG:boto:Method: GET 2022-09-03T06:50:20.617 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.618 DEBUG:boto:Data: 2022-09-03T06:50:20.619 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.620 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:20.621 DEBUG:boto:Port: 8000 2022-09-03T06:50:20.623 DEBUG:boto:Params: {} 2022-09-03T06:50:20.624 DEBUG:boto:Token: None 2022-09-03T06:50:20.625 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.626 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FwJz3HBuncxAsuSb88W6wG3mLw0= 2022-09-03T06:50:20.627 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FwJz3HBuncxAsuSb88W6wG3mLw0=', 'Content-Length': '0'} 2022-09-03T06:50:20.631 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ce3e5b3686b6a54b-006312f92c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.633 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:20.634 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:20.634 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.635 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.636 DEBUG:boto:Method: HEAD 2022-09-03T06:50:20.637 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.638 DEBUG:boto:Data: 2022-09-03T06:50:20.639 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.639 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:20.640 DEBUG:boto:Port: 8001 2022-09-03T06:50:20.640 DEBUG:boto:Params: {} 2022-09-03T06:50:20.641 DEBUG:boto:Token: None 2022-09-03T06:50:20.642 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.643 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wuzAyYCXCqhGR7bIOTE4q5pu4NQ= 2022-09-03T06:50:20.644 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wuzAyYCXCqhGR7bIOTE4q5pu4NQ=', 'Content-Length': '0'} 2022-09-03T06:50:20.647 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000015267538cb1a64d2-006312f92c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.648 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:20.650 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:20.651 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.651 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.652 DEBUG:boto:Method: HEAD 2022-09-03T06:50:20.653 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.654 DEBUG:boto:Data: 2022-09-03T06:50:20.655 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.657 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:20.657 DEBUG:boto:Port: 8000 2022-09-03T06:50:20.658 DEBUG:boto:Params: {} 2022-09-03T06:50:20.659 DEBUG:boto:Token: None 2022-09-03T06:50:20.661 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.662 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wuzAyYCXCqhGR7bIOTE4q5pu4NQ= 2022-09-03T06:50:20.662 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wuzAyYCXCqhGR7bIOTE4q5pu4NQ=', 'Content-Length': '0'} 2022-09-03T06:50:20.665 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048b6aed30045d32c-006312f92c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.666 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:50:20.667 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:20.667 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:20.668 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.669 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.669 DEBUG:boto:Method: GET 2022-09-03T06:50:20.670 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.671 DEBUG:boto:Data: 2022-09-03T06:50:20.671 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.672 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:20.673 DEBUG:boto:Port: 8001 2022-09-03T06:50:20.674 DEBUG:boto:Params: {} 2022-09-03T06:50:20.674 DEBUG:boto:Token: None 2022-09-03T06:50:20.675 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.675 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FwJz3HBuncxAsuSb88W6wG3mLw0= 2022-09-03T06:50:20.676 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FwJz3HBuncxAsuSb88W6wG3mLw0=', 'Content-Length': '0'} 2022-09-03T06:50:20.679 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a2be29a05a4ab02-006312f92c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.680 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:20.680 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:20.681 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.682 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.682 DEBUG:boto:Method: GET 2022-09-03T06:50:20.683 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.684 DEBUG:boto:Data: 2022-09-03T06:50:20.684 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.685 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:20.686 DEBUG:boto:Port: 8000 2022-09-03T06:50:20.686 DEBUG:boto:Params: {} 2022-09-03T06:50:20.687 DEBUG:boto:Token: None 2022-09-03T06:50:20.688 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:20.688 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FwJz3HBuncxAsuSb88W6wG3mLw0= 2022-09-03T06:50:20.689 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FwJz3HBuncxAsuSb88W6wG3mLw0=', 'Content-Length': '0'} 2022-09-03T06:50:20.691 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002073bc359ac46c8c-006312f92c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.734 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:20.735 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:20.736 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:20.736 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:20.737 DEBUG:boto:Method: GET 2022-09-03T06:50:20.738 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:50:20.739 DEBUG:boto:Data: 2022-09-03T06:50:20.739 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.740 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:20.740 DEBUG:boto:Port: 8001 2022-09-03T06:50:20.741 DEBUG:boto:Params: {} 2022-09-03T06:50:20.742 DEBUG:boto:Token: None 2022-09-03T06:50:20.742 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/ 2022-09-03T06:50:20.743 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY= 2022-09-03T06:50:20.744 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY=', 'Content-Length': '0'} 2022-09-03T06:50:20.747 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002834f56238e0c0dd-006312f92c-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.747 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:20.748 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:20.749 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:20.750 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:20.751 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:20.751 DEBUG:boto:Method: GET 2022-09-03T06:50:20.752 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:50:20.752 DEBUG:boto:Data: 2022-09-03T06:50:20.753 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.753 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:20.754 DEBUG:boto:Port: 8000 2022-09-03T06:50:20.755 DEBUG:boto:Params: {} 2022-09-03T06:50:20.755 DEBUG:boto:Token: None 2022-09-03T06:50:20.756 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/ 2022-09-03T06:50:20.757 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY= 2022-09-03T06:50:20.757 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY=', 'Content-Length': '0'} 2022-09-03T06:50:20.762 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004effd02dbd7acd47-006312f92c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.762 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:20.763 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:20.764 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:20.764 DEBUG:boto:Method: HEAD 2022-09-03T06:50:20.765 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:50:20.765 DEBUG:boto:Data: 2022-09-03T06:50:20.766 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.767 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:20.768 DEBUG:boto:Port: 8001 2022-09-03T06:50:20.769 DEBUG:boto:Params: {} 2022-09-03T06:50:20.769 DEBUG:boto:Token: None 2022-09-03T06:50:20.770 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj69 2022-09-03T06:50:20.771 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q4xrQ6aRK+6RaIYI3pMYNtQY6gg= 2022-09-03T06:50:20.771 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q4xrQ6aRK+6RaIYI3pMYNtQY6gg=', 'Content-Length': '0'} 2022-09-03T06:50:20.774 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000913bcdd615a0625f-006312f92c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.775 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:20.775 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:20.776 DEBUG:boto:Method: HEAD 2022-09-03T06:50:20.776 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:50:20.777 DEBUG:boto:Data: 2022-09-03T06:50:20.778 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.778 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:20.779 DEBUG:boto:Port: 8000 2022-09-03T06:50:20.779 DEBUG:boto:Params: {} 2022-09-03T06:50:20.780 DEBUG:boto:Token: None 2022-09-03T06:50:20.782 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj69 2022-09-03T06:50:20.784 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:q4xrQ6aRK+6RaIYI3pMYNtQY6gg= 2022-09-03T06:50:20.785 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:q4xrQ6aRK+6RaIYI3pMYNtQY6gg=', 'Content-Length': '0'} 2022-09-03T06:50:20.788 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c48abc684e870826-006312f92c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.790 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:50:20.791 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:20.793 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:20.793 DEBUG:boto:Method: GET 2022-09-03T06:50:20.794 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:50:20.795 DEBUG:boto:Data: 2022-09-03T06:50:20.795 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.796 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:20.796 DEBUG:boto:Port: 8001 2022-09-03T06:50:20.797 DEBUG:boto:Params: {} 2022-09-03T06:50:20.798 DEBUG:boto:Token: None 2022-09-03T06:50:20.798 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj69 2022-09-03T06:50:20.799 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OaBHPEhbMtvMv7FWguaYKrPeS8g= 2022-09-03T06:50:20.801 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OaBHPEhbMtvMv7FWguaYKrPeS8g=', 'Content-Length': '0'} 2022-09-03T06:50:20.805 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000066a568af5063811d-006312f92c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.846 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:20.848 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:20.848 DEBUG:boto:Method: GET 2022-09-03T06:50:20.849 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:50:20.849 DEBUG:boto:Data: 2022-09-03T06:50:20.850 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.850 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:20.851 DEBUG:boto:Port: 8000 2022-09-03T06:50:20.851 DEBUG:boto:Params: {} 2022-09-03T06:50:20.852 DEBUG:boto:Token: None 2022-09-03T06:50:20.853 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj69 2022-09-03T06:50:20.854 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OaBHPEhbMtvMv7FWguaYKrPeS8g= 2022-09-03T06:50:20.855 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OaBHPEhbMtvMv7FWguaYKrPeS8g=', 'Content-Length': '0'} 2022-09-03T06:50:20.858 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000746590d69e7b6123-006312f92c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.859 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:20.860 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:20.861 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:20.862 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:20.863 DEBUG:boto:Method: GET 2022-09-03T06:50:20.864 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:50:20.865 DEBUG:boto:Data: 2022-09-03T06:50:20.866 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.867 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:20.868 DEBUG:boto:Port: 8001 2022-09-03T06:50:20.869 DEBUG:boto:Params: {} 2022-09-03T06:50:20.869 DEBUG:boto:Token: None 2022-09-03T06:50:20.870 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/ 2022-09-03T06:50:20.871 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY= 2022-09-03T06:50:20.871 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY=', 'Content-Length': '0'} 2022-09-03T06:50:20.874 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d6f6bdf6669e321-006312f92c-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.875 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:20.876 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:20.877 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:20.878 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:20.878 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:20.879 DEBUG:boto:Method: GET 2022-09-03T06:50:20.879 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:50:20.880 DEBUG:boto:Data: 2022-09-03T06:50:20.880 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.881 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:20.881 DEBUG:boto:Port: 8000 2022-09-03T06:50:20.882 DEBUG:boto:Params: {} 2022-09-03T06:50:20.883 DEBUG:boto:Token: None 2022-09-03T06:50:20.884 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/ 2022-09-03T06:50:20.884 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY= 2022-09-03T06:50:20.885 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0o8YDTllTpVGOkaCQzh5xdXdQUY=', 'Content-Length': '0'} 2022-09-03T06:50:20.890 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000090c7bdd30fe2380a-006312f92c-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.891 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:20.892 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj74 2022-09-03T06:50:20.892 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:20.893 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:20.893 DEBUG:boto:path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:20.894 DEBUG:boto:auth_path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:20.895 DEBUG:boto:Method: GET 2022-09-03T06:50:20.896 DEBUG:boto:Path: /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:20.896 DEBUG:boto:Data: 2022-09-03T06:50:20.897 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.898 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:20.899 DEBUG:boto:Port: 8001 2022-09-03T06:50:20.900 DEBUG:boto:Params: {} 2022-09-03T06:50:20.901 DEBUG:boto:Token: None 2022-09-03T06:50:20.902 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:20.903 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ypDLNpZdtkYwQEwgHe84DmyLCVA= 2022-09-03T06:50:20.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ypDLNpZdtkYwQEwgHe84DmyLCVA=', 'Content-Length': '0'} 2022-09-03T06:50:20.907 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000db64ef341197af4-006312f92c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.908 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:20.909 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:20.909 DEBUG:boto:path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:20.910 DEBUG:boto:auth_path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:20.911 DEBUG:boto:Method: GET 2022-09-03T06:50:20.911 DEBUG:boto:Path: /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:20.912 DEBUG:boto:Data: 2022-09-03T06:50:20.912 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.913 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:20.913 DEBUG:boto:Port: 8000 2022-09-03T06:50:20.914 DEBUG:boto:Params: {} 2022-09-03T06:50:20.915 DEBUG:boto:Token: None 2022-09-03T06:50:20.915 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:20.916 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ypDLNpZdtkYwQEwgHe84DmyLCVA= 2022-09-03T06:50:20.917 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ypDLNpZdtkYwQEwgHe84DmyLCVA=', 'Content-Length': '0'} 2022-09-03T06:50:20.919 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f6b6062075eb30a-006312f92c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.920 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:20.920 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:20.921 DEBUG:boto:path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:20.921 DEBUG:boto:auth_path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:20.922 DEBUG:boto:Method: HEAD 2022-09-03T06:50:20.922 DEBUG:boto:Path: /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:20.923 DEBUG:boto:Data: 2022-09-03T06:50:20.923 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.924 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:20.924 DEBUG:boto:Port: 8001 2022-09-03T06:50:20.925 DEBUG:boto:Params: {} 2022-09-03T06:50:20.925 DEBUG:boto:Token: None 2022-09-03T06:50:20.926 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:20.926 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VtPl9k7WbBZCJdiglwCc6u+lk2A= 2022-09-03T06:50:20.927 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VtPl9k7WbBZCJdiglwCc6u+lk2A=', 'Content-Length': '0'} 2022-09-03T06:50:20.930 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000429ff5901ce6f68f-006312f92c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.933 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:20.934 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:20.934 DEBUG:boto:path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:20.935 DEBUG:boto:auth_path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:20.936 DEBUG:boto:Method: HEAD 2022-09-03T06:50:20.937 DEBUG:boto:Path: /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:20.937 DEBUG:boto:Data: 2022-09-03T06:50:20.938 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.938 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:20.939 DEBUG:boto:Port: 8000 2022-09-03T06:50:20.939 DEBUG:boto:Params: {} 2022-09-03T06:50:20.940 DEBUG:boto:Token: None 2022-09-03T06:50:20.941 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:20.942 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VtPl9k7WbBZCJdiglwCc6u+lk2A= 2022-09-03T06:50:20.943 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VtPl9k7WbBZCJdiglwCc6u+lk2A=', 'Content-Length': '0'} 2022-09-03T06:50:20.945 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000897106e59135c75c-006312f92c-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:20.946 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj74 2022-09-03T06:50:20.946 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:20.947 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:20.947 DEBUG:boto:path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:20.948 DEBUG:boto:auth_path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:20.949 DEBUG:boto:Method: GET 2022-09-03T06:50:20.949 DEBUG:boto:Path: /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:20.950 DEBUG:boto:Data: 2022-09-03T06:50:20.951 DEBUG:boto:Headers: {} 2022-09-03T06:50:20.952 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:20.952 DEBUG:boto:Port: 8001 2022-09-03T06:50:20.953 DEBUG:boto:Params: {} 2022-09-03T06:50:20.953 DEBUG:boto:Token: None 2022-09-03T06:50:20.954 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:20 GMT /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:20.955 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ypDLNpZdtkYwQEwgHe84DmyLCVA= 2022-09-03T06:50:20.956 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:20 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ypDLNpZdtkYwQEwgHe84DmyLCVA=', 'Content-Length': '0'} 2022-09-03T06:50:20.958 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006df70cd734fd742f-006312f92c-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:20 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.006 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:21.007 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:21.008 DEBUG:boto:path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:21.008 DEBUG:boto:auth_path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:21.009 DEBUG:boto:Method: GET 2022-09-03T06:50:21.009 DEBUG:boto:Path: /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:21.010 DEBUG:boto:Data: 2022-09-03T06:50:21.011 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.012 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.012 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.013 DEBUG:boto:Params: {} 2022-09-03T06:50:21.014 DEBUG:boto:Token: None 2022-09-03T06:50:21.015 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:21.016 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9YEKQFjiXCpdhpU6dWwICXP5aYY= 2022-09-03T06:50:21.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9YEKQFjiXCpdhpU6dWwICXP5aYY=', 'Content-Length': '0'} 2022-09-03T06:50:21.019 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a8c5e56c0d9d4f86-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.020 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:21.020 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:21.021 DEBUG:boto:path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:21.022 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:21.023 DEBUG:boto:Method: GET 2022-09-03T06:50:21.023 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj74 2022-09-03T06:50:21.024 DEBUG:boto:Data: 2022-09-03T06:50:21.024 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.025 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.026 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.026 DEBUG:boto:Params: {} 2022-09-03T06:50:21.027 DEBUG:boto:Token: None 2022-09-03T06:50:21.028 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/ 2022-09-03T06:50:21.028 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE= 2022-09-03T06:50:21.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE=', 'Content-Length': '0'} 2022-09-03T06:50:21.032 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006650ce2aecdd4a9f-006312f92d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.033 DEBUG:boto:b'dsgvcy-75obj741000falseobj742022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:21.034 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:21.034 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:21.035 DEBUG:boto:path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:21.035 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:21.036 DEBUG:boto:Method: GET 2022-09-03T06:50:21.037 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj74 2022-09-03T06:50:21.037 DEBUG:boto:Data: 2022-09-03T06:50:21.038 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.039 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.039 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.040 DEBUG:boto:Params: {} 2022-09-03T06:50:21.040 DEBUG:boto:Token: None 2022-09-03T06:50:21.041 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/ 2022-09-03T06:50:21.042 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE= 2022-09-03T06:50:21.042 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE=', 'Content-Length': '0'} 2022-09-03T06:50:21.047 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d7f4d68bb6931679-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.048 DEBUG:boto:b'dsgvcy-75obj741000falseobj742022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:21.049 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:21.049 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:21.050 DEBUG:boto:Method: HEAD 2022-09-03T06:50:21.051 DEBUG:boto:Path: /dsgvcy-75/obj74 2022-09-03T06:50:21.051 DEBUG:boto:Data: 2022-09-03T06:50:21.052 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.052 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.053 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.054 DEBUG:boto:Params: {} 2022-09-03T06:50:21.054 DEBUG:boto:Token: None 2022-09-03T06:50:21.055 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj74 2022-09-03T06:50:21.056 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r/t1vr2k5zTdPoeqs7b7oOtn/3M= 2022-09-03T06:50:21.057 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r/t1vr2k5zTdPoeqs7b7oOtn/3M=', 'Content-Length': '0'} 2022-09-03T06:50:21.059 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003a417b685914e4c2-006312f92d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.060 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:21.060 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:21.061 DEBUG:boto:Method: HEAD 2022-09-03T06:50:21.062 DEBUG:boto:Path: /dsgvcy-75/obj74 2022-09-03T06:50:21.062 DEBUG:boto:Data: 2022-09-03T06:50:21.063 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.063 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.064 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.064 DEBUG:boto:Params: {} 2022-09-03T06:50:21.065 DEBUG:boto:Token: None 2022-09-03T06:50:21.066 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj74 2022-09-03T06:50:21.066 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r/t1vr2k5zTdPoeqs7b7oOtn/3M= 2022-09-03T06:50:21.067 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r/t1vr2k5zTdPoeqs7b7oOtn/3M=', 'Content-Length': '0'} 2022-09-03T06:50:21.070 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000090f35732ebf8adf7-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.071 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj74 2022-09-03T06:50:21.072 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:21.073 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:21.074 DEBUG:boto:Method: GET 2022-09-03T06:50:21.074 DEBUG:boto:Path: /dsgvcy-75/obj74 2022-09-03T06:50:21.075 DEBUG:boto:Data: 2022-09-03T06:50:21.076 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.076 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.077 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.077 DEBUG:boto:Params: {} 2022-09-03T06:50:21.078 DEBUG:boto:Token: None 2022-09-03T06:50:21.079 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj74 2022-09-03T06:50:21.080 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l//TbpBXZIflq+vr9HzoljidLis= 2022-09-03T06:50:21.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l//TbpBXZIflq+vr9HzoljidLis=', 'Content-Length': '0'} 2022-09-03T06:50:21.083 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000de227d095bf77596-006312f92d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.084 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:21.085 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:21.086 DEBUG:boto:Method: GET 2022-09-03T06:50:21.086 DEBUG:boto:Path: /dsgvcy-75/obj74 2022-09-03T06:50:21.087 DEBUG:boto:Data: 2022-09-03T06:50:21.088 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.088 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.089 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.091 DEBUG:boto:Params: {} 2022-09-03T06:50:21.091 DEBUG:boto:Token: None 2022-09-03T06:50:21.092 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj74 2022-09-03T06:50:21.093 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l//TbpBXZIflq+vr9HzoljidLis= 2022-09-03T06:50:21.093 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l//TbpBXZIflq+vr9HzoljidLis=', 'Content-Length': '0'} 2022-09-03T06:50:21.096 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000078a9aca23464d27c-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.096 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:21.097 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:21.098 DEBUG:boto:path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:21.098 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:21.099 DEBUG:boto:Method: GET 2022-09-03T06:50:21.100 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj74 2022-09-03T06:50:21.101 DEBUG:boto:Data: 2022-09-03T06:50:21.101 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.102 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.103 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.103 DEBUG:boto:Params: {} 2022-09-03T06:50:21.104 DEBUG:boto:Token: None 2022-09-03T06:50:21.105 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/ 2022-09-03T06:50:21.106 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE= 2022-09-03T06:50:21.107 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE=', 'Content-Length': '0'} 2022-09-03T06:50:21.110 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002e08d97a55d93f6d-006312f92d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.111 DEBUG:boto:b'dsgvcy-75obj741000falseobj742022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:21.112 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:21.112 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:21.113 DEBUG:boto:path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:21.114 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:21.114 DEBUG:boto:Method: GET 2022-09-03T06:50:21.115 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj74 2022-09-03T06:50:21.115 DEBUG:boto:Data: 2022-09-03T06:50:21.116 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.117 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.117 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.118 DEBUG:boto:Params: {} 2022-09-03T06:50:21.119 DEBUG:boto:Token: None 2022-09-03T06:50:21.119 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/ 2022-09-03T06:50:21.120 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE= 2022-09-03T06:50:21.121 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE=', 'Content-Length': '0'} 2022-09-03T06:50:21.126 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008a6a897a8727c66e-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.127 DEBUG:boto:b'dsgvcy-75obj741000falseobj742022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:21.128 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:50:21.128 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:21.129 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:21.130 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.130 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.131 DEBUG:boto:Method: GET 2022-09-03T06:50:21.132 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.133 DEBUG:boto:Data: 2022-09-03T06:50:21.134 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.134 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.135 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.136 DEBUG:boto:Params: {} 2022-09-03T06:50:21.137 DEBUG:boto:Token: None 2022-09-03T06:50:21.139 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.140 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N9ClXsdgIfUz2oYYsxM3lXKuuIM= 2022-09-03T06:50:21.141 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N9ClXsdgIfUz2oYYsxM3lXKuuIM=', 'Content-Length': '0'} 2022-09-03T06:50:21.143 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000922c94da91113823-006312f92d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.144 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:21.145 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:21.146 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.146 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.147 DEBUG:boto:Method: GET 2022-09-03T06:50:21.148 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.149 DEBUG:boto:Data: 2022-09-03T06:50:21.149 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.150 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.151 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.152 DEBUG:boto:Params: {} 2022-09-03T06:50:21.153 DEBUG:boto:Token: None 2022-09-03T06:50:21.154 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.154 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N9ClXsdgIfUz2oYYsxM3lXKuuIM= 2022-09-03T06:50:21.155 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N9ClXsdgIfUz2oYYsxM3lXKuuIM=', 'Content-Length': '0'} 2022-09-03T06:50:21.157 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002882291ed0ab2b29-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.204 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:21.205 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:21.206 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.206 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.207 DEBUG:boto:Method: HEAD 2022-09-03T06:50:21.207 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.208 DEBUG:boto:Data: 2022-09-03T06:50:21.208 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.209 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.210 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.210 DEBUG:boto:Params: {} 2022-09-03T06:50:21.211 DEBUG:boto:Token: None 2022-09-03T06:50:21.211 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.212 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:orVlbe3+aXxDoEnnjXrkFdPT2W8= 2022-09-03T06:50:21.213 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:orVlbe3+aXxDoEnnjXrkFdPT2W8=', 'Content-Length': '0'} 2022-09-03T06:50:21.215 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000018676fb6fc114a94-006312f92d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.216 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:21.216 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:21.217 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.217 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.218 DEBUG:boto:Method: HEAD 2022-09-03T06:50:21.218 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.219 DEBUG:boto:Data: 2022-09-03T06:50:21.220 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.220 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.221 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.221 DEBUG:boto:Params: {} 2022-09-03T06:50:21.222 DEBUG:boto:Token: None 2022-09-03T06:50:21.222 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.223 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:orVlbe3+aXxDoEnnjXrkFdPT2W8= 2022-09-03T06:50:21.224 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:orVlbe3+aXxDoEnnjXrkFdPT2W8=', 'Content-Length': '0'} 2022-09-03T06:50:21.226 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000baa59b31324bfd73-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.228 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:50:21.228 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:21.229 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:21.229 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.230 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.231 DEBUG:boto:Method: GET 2022-09-03T06:50:21.231 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.232 DEBUG:boto:Data: 2022-09-03T06:50:21.233 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.233 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.234 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.235 DEBUG:boto:Params: {} 2022-09-03T06:50:21.235 DEBUG:boto:Token: None 2022-09-03T06:50:21.236 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.237 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N9ClXsdgIfUz2oYYsxM3lXKuuIM= 2022-09-03T06:50:21.237 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N9ClXsdgIfUz2oYYsxM3lXKuuIM=', 'Content-Length': '0'} 2022-09-03T06:50:21.240 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d0204e8bb18b7ccb-006312f92d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.287 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:21.288 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:21.289 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.289 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.290 DEBUG:boto:Method: GET 2022-09-03T06:50:21.291 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.291 DEBUG:boto:Data: 2022-09-03T06:50:21.292 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.293 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.303 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.304 DEBUG:boto:Params: {} 2022-09-03T06:50:21.305 DEBUG:boto:Token: None 2022-09-03T06:50:21.306 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:21.307 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N9ClXsdgIfUz2oYYsxM3lXKuuIM= 2022-09-03T06:50:21.308 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N9ClXsdgIfUz2oYYsxM3lXKuuIM=', 'Content-Length': '0'} 2022-09-03T06:50:21.310 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eeeacacfb5aa95ae-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.312 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:21.312 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:21.313 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:21.314 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:21.314 DEBUG:boto:Method: GET 2022-09-03T06:50:21.315 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:50:21.316 DEBUG:boto:Data: 2022-09-03T06:50:21.316 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.318 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.319 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.320 DEBUG:boto:Params: {} 2022-09-03T06:50:21.322 DEBUG:boto:Token: None 2022-09-03T06:50:21.323 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/ 2022-09-03T06:50:21.324 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE= 2022-09-03T06:50:21.325 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE=', 'Content-Length': '0'} 2022-09-03T06:50:21.329 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000077f28bd949ae7ca6-006312f92d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.330 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:21.332 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:21.333 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:21.335 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:21.336 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:21.337 DEBUG:boto:Method: GET 2022-09-03T06:50:21.338 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:50:21.339 DEBUG:boto:Data: 2022-09-03T06:50:21.340 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.341 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.342 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.343 DEBUG:boto:Params: {} 2022-09-03T06:50:21.344 DEBUG:boto:Token: None 2022-09-03T06:50:21.346 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/ 2022-09-03T06:50:21.347 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE= 2022-09-03T06:50:21.348 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE=', 'Content-Length': '0'} 2022-09-03T06:50:21.353 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000598217a071b3584d-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.355 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:21.356 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:21.357 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:21.358 DEBUG:boto:Method: HEAD 2022-09-03T06:50:21.359 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:50:21.359 DEBUG:boto:Data: 2022-09-03T06:50:21.360 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.361 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.362 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.363 DEBUG:boto:Params: {} 2022-09-03T06:50:21.365 DEBUG:boto:Token: None 2022-09-03T06:50:21.366 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj76 2022-09-03T06:50:21.367 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fHR74kB3uyMJGuyQjN9p5LQqmLc= 2022-09-03T06:50:21.368 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fHR74kB3uyMJGuyQjN9p5LQqmLc=', 'Content-Length': '0'} 2022-09-03T06:50:21.371 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000009129849a34decb3-006312f92d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.372 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:21.374 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:21.375 DEBUG:boto:Method: HEAD 2022-09-03T06:50:21.375 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:50:21.376 DEBUG:boto:Data: 2022-09-03T06:50:21.377 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.378 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.379 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.380 DEBUG:boto:Params: {} 2022-09-03T06:50:21.380 DEBUG:boto:Token: None 2022-09-03T06:50:21.381 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj76 2022-09-03T06:50:21.383 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fHR74kB3uyMJGuyQjN9p5LQqmLc= 2022-09-03T06:50:21.384 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fHR74kB3uyMJGuyQjN9p5LQqmLc=', 'Content-Length': '0'} 2022-09-03T06:50:21.387 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007960d9f2a5881dfd-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.388 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:50:21.389 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:21.390 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:21.391 DEBUG:boto:Method: GET 2022-09-03T06:50:21.393 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:50:21.393 DEBUG:boto:Data: 2022-09-03T06:50:21.403 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.405 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.406 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.406 DEBUG:boto:Params: {} 2022-09-03T06:50:21.407 DEBUG:boto:Token: None 2022-09-03T06:50:21.408 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj76 2022-09-03T06:50:21.408 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pTBIkCL7pafgmHpK8Vs3Vehh3E8= 2022-09-03T06:50:21.409 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pTBIkCL7pafgmHpK8Vs3Vehh3E8=', 'Content-Length': '0'} 2022-09-03T06:50:21.412 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000018d5fe14aee1d47-006312f92d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.412 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:21.413 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:21.415 DEBUG:boto:Method: GET 2022-09-03T06:50:21.416 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:50:21.417 DEBUG:boto:Data: 2022-09-03T06:50:21.418 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.419 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.420 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.421 DEBUG:boto:Params: {} 2022-09-03T06:50:21.423 DEBUG:boto:Token: None 2022-09-03T06:50:21.424 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj76 2022-09-03T06:50:21.425 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pTBIkCL7pafgmHpK8Vs3Vehh3E8= 2022-09-03T06:50:21.427 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pTBIkCL7pafgmHpK8Vs3Vehh3E8=', 'Content-Length': '0'} 2022-09-03T06:50:21.430 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f29e5e94ac311f6c-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.473 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:21.474 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:21.475 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:21.475 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:21.476 DEBUG:boto:Method: GET 2022-09-03T06:50:21.476 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:50:21.477 DEBUG:boto:Data: 2022-09-03T06:50:21.477 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.478 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.478 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.479 DEBUG:boto:Params: {} 2022-09-03T06:50:21.480 DEBUG:boto:Token: None 2022-09-03T06:50:21.481 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/ 2022-09-03T06:50:21.482 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE= 2022-09-03T06:50:21.482 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE=', 'Content-Length': '0'} 2022-09-03T06:50:21.485 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003d09639072aa7b2c-006312f92d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.486 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:21.488 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:21.488 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:21.489 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:21.490 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:21.490 DEBUG:boto:Method: GET 2022-09-03T06:50:21.491 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:50:21.492 DEBUG:boto:Data: 2022-09-03T06:50:21.493 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.493 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.494 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.494 DEBUG:boto:Params: {} 2022-09-03T06:50:21.495 DEBUG:boto:Token: None 2022-09-03T06:50:21.496 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/ 2022-09-03T06:50:21.496 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE= 2022-09-03T06:50:21.497 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE=', 'Content-Length': '0'} 2022-09-03T06:50:21.502 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000082c858c75359d036-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.502 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:21.503 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:50:21.504 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:21.505 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:21.505 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.506 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.506 DEBUG:boto:Method: GET 2022-09-03T06:50:21.507 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.508 DEBUG:boto:Data: 2022-09-03T06:50:21.508 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.509 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.509 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.510 DEBUG:boto:Params: {} 2022-09-03T06:50:21.510 DEBUG:boto:Token: None 2022-09-03T06:50:21.512 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.513 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6RQljlaU4CNtSWYTdLSA4FzMibg= 2022-09-03T06:50:21.515 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6RQljlaU4CNtSWYTdLSA4FzMibg=', 'Content-Length': '0'} 2022-09-03T06:50:21.522 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005d6c3f61e5badec0-006312f92d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.524 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:21.525 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:21.526 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.527 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.528 DEBUG:boto:Method: GET 2022-09-03T06:50:21.529 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.530 DEBUG:boto:Data: 2022-09-03T06:50:21.531 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.532 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.534 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.535 DEBUG:boto:Params: {} 2022-09-03T06:50:21.536 DEBUG:boto:Token: None 2022-09-03T06:50:21.537 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.538 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6RQljlaU4CNtSWYTdLSA4FzMibg= 2022-09-03T06:50:21.539 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6RQljlaU4CNtSWYTdLSA4FzMibg=', 'Content-Length': '0'} 2022-09-03T06:50:21.542 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000028fd00f3509f4c94-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.543 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:21.544 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:21.544 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.545 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.546 DEBUG:boto:Method: HEAD 2022-09-03T06:50:21.546 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.547 DEBUG:boto:Data: 2022-09-03T06:50:21.547 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.548 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.548 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.549 DEBUG:boto:Params: {} 2022-09-03T06:50:21.550 DEBUG:boto:Token: None 2022-09-03T06:50:21.551 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.552 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bYhIGTRSuW8W1hukrxqzA5k1GYg= 2022-09-03T06:50:21.552 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bYhIGTRSuW8W1hukrxqzA5k1GYg=', 'Content-Length': '0'} 2022-09-03T06:50:21.555 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bb63034955505d25-006312f92d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.556 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:21.557 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:21.558 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.559 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.560 DEBUG:boto:Method: HEAD 2022-09-03T06:50:21.561 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.561 DEBUG:boto:Data: 2022-09-03T06:50:21.562 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.563 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.563 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.564 DEBUG:boto:Params: {} 2022-09-03T06:50:21.564 DEBUG:boto:Token: None 2022-09-03T06:50:21.565 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.565 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bYhIGTRSuW8W1hukrxqzA5k1GYg= 2022-09-03T06:50:21.566 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bYhIGTRSuW8W1hukrxqzA5k1GYg=', 'Content-Length': '0'} 2022-09-03T06:50:21.568 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002928540e88ac6534-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.569 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:50:21.569 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:21.570 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:21.571 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.571 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.572 DEBUG:boto:Method: GET 2022-09-03T06:50:21.572 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.573 DEBUG:boto:Data: 2022-09-03T06:50:21.575 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.576 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.577 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.578 DEBUG:boto:Params: {} 2022-09-03T06:50:21.580 DEBUG:boto:Token: None 2022-09-03T06:50:21.582 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.584 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6RQljlaU4CNtSWYTdLSA4FzMibg= 2022-09-03T06:50:21.585 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6RQljlaU4CNtSWYTdLSA4FzMibg=', 'Content-Length': '0'} 2022-09-03T06:50:21.589 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002e299c62a30e4e9e-006312f92d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.590 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:21.591 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:21.592 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.592 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.593 DEBUG:boto:Method: GET 2022-09-03T06:50:21.594 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.595 DEBUG:boto:Data: 2022-09-03T06:50:21.595 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.596 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.597 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.597 DEBUG:boto:Params: {} 2022-09-03T06:50:21.599 DEBUG:boto:Token: None 2022-09-03T06:50:21.600 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:21.601 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6RQljlaU4CNtSWYTdLSA4FzMibg= 2022-09-03T06:50:21.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6RQljlaU4CNtSWYTdLSA4FzMibg=', 'Content-Length': '0'} 2022-09-03T06:50:21.606 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002fbc449d2388e0a4-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.608 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:21.609 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:21.609 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:21.610 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:21.611 DEBUG:boto:Method: GET 2022-09-03T06:50:21.613 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:50:21.614 DEBUG:boto:Data: 2022-09-03T06:50:21.614 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.615 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.616 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.618 DEBUG:boto:Params: {} 2022-09-03T06:50:21.619 DEBUG:boto:Token: None 2022-09-03T06:50:21.621 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/ 2022-09-03T06:50:21.622 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE= 2022-09-03T06:50:21.623 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE=', 'Content-Length': '0'} 2022-09-03T06:50:21.626 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ace74d26d75a00d1-006312f92d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.627 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:21.629 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:21.630 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:21.631 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:21.632 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:21.633 DEBUG:boto:Method: GET 2022-09-03T06:50:21.634 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:50:21.635 DEBUG:boto:Data: 2022-09-03T06:50:21.636 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.637 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.638 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.639 DEBUG:boto:Params: {} 2022-09-03T06:50:21.640 DEBUG:boto:Token: None 2022-09-03T06:50:21.641 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/ 2022-09-03T06:50:21.642 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE= 2022-09-03T06:50:21.643 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE=', 'Content-Length': '0'} 2022-09-03T06:50:21.649 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000072317141cf414074-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.650 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:21.651 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:21.652 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:21.654 DEBUG:boto:Method: HEAD 2022-09-03T06:50:21.655 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:50:21.656 DEBUG:boto:Data: 2022-09-03T06:50:21.657 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.658 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.658 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.659 DEBUG:boto:Params: {} 2022-09-03T06:50:21.660 DEBUG:boto:Token: None 2022-09-03T06:50:21.661 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj85 2022-09-03T06:50:21.662 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qComj/ovD9JoWEwgPHfHZNeustc= 2022-09-03T06:50:21.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qComj/ovD9JoWEwgPHfHZNeustc=', 'Content-Length': '0'} 2022-09-03T06:50:21.669 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b8bd9c38edc31e29-006312f92d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.670 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:21.670 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:21.671 DEBUG:boto:Method: HEAD 2022-09-03T06:50:21.672 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:50:21.672 DEBUG:boto:Data: 2022-09-03T06:50:21.673 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.673 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.674 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.674 DEBUG:boto:Params: {} 2022-09-03T06:50:21.675 DEBUG:boto:Token: None 2022-09-03T06:50:21.675 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj85 2022-09-03T06:50:21.676 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qComj/ovD9JoWEwgPHfHZNeustc= 2022-09-03T06:50:21.677 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qComj/ovD9JoWEwgPHfHZNeustc=', 'Content-Length': '0'} 2022-09-03T06:50:21.679 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000869f8a5a8deda9e-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.680 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:50:21.680 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:21.681 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:21.681 DEBUG:boto:Method: GET 2022-09-03T06:50:21.682 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:50:21.682 DEBUG:boto:Data: 2022-09-03T06:50:21.683 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.683 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.684 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.685 DEBUG:boto:Params: {} 2022-09-03T06:50:21.685 DEBUG:boto:Token: None 2022-09-03T06:50:21.686 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj85 2022-09-03T06:50:21.687 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2qg67gtQbDW7rFuIKQpUUbW7U7U= 2022-09-03T06:50:21.687 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2qg67gtQbDW7rFuIKQpUUbW7U7U=', 'Content-Length': '0'} 2022-09-03T06:50:21.690 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004bf58fe11d9d0e8c-006312f92d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.730 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:21.731 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:21.732 DEBUG:boto:Method: GET 2022-09-03T06:50:21.733 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:50:21.734 DEBUG:boto:Data: 2022-09-03T06:50:21.735 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.736 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.736 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.737 DEBUG:boto:Params: {} 2022-09-03T06:50:21.738 DEBUG:boto:Token: None 2022-09-03T06:50:21.739 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj85 2022-09-03T06:50:21.741 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2qg67gtQbDW7rFuIKQpUUbW7U7U= 2022-09-03T06:50:21.742 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2qg67gtQbDW7rFuIKQpUUbW7U7U=', 'Content-Length': '0'} 2022-09-03T06:50:21.746 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000560ecf74f47235b0-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.747 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:21.748 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:21.749 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:21.750 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:21.750 DEBUG:boto:Method: GET 2022-09-03T06:50:21.752 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:50:21.752 DEBUG:boto:Data: 2022-09-03T06:50:21.753 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.754 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.754 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.755 DEBUG:boto:Params: {} 2022-09-03T06:50:21.756 DEBUG:boto:Token: None 2022-09-03T06:50:21.756 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/ 2022-09-03T06:50:21.757 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE= 2022-09-03T06:50:21.757 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE=', 'Content-Length': '0'} 2022-09-03T06:50:21.762 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008f0baea153bf7d74-006312f92d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.762 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:21.763 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:21.764 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:21.765 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:21.766 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:21.767 DEBUG:boto:Method: GET 2022-09-03T06:50:21.767 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:50:21.768 DEBUG:boto:Data: 2022-09-03T06:50:21.770 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.770 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.771 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.772 DEBUG:boto:Params: {} 2022-09-03T06:50:21.772 DEBUG:boto:Token: None 2022-09-03T06:50:21.774 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/ 2022-09-03T06:50:21.774 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE= 2022-09-03T06:50:21.775 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE=', 'Content-Length': '0'} 2022-09-03T06:50:21.780 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008e08e3dc1b2dced3-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.781 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:21.782 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:50:21.783 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:21.784 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:21.785 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.786 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.787 DEBUG:boto:Method: GET 2022-09-03T06:50:21.788 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.789 DEBUG:boto:Data: 2022-09-03T06:50:21.790 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.791 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.792 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.793 DEBUG:boto:Params: {} 2022-09-03T06:50:21.794 DEBUG:boto:Token: None 2022-09-03T06:50:21.795 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.796 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UmHA6R9mmNySYC6yPF/Z46UV9Jk= 2022-09-03T06:50:21.797 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UmHA6R9mmNySYC6yPF/Z46UV9Jk=', 'Content-Length': '0'} 2022-09-03T06:50:21.799 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000254505cd97ec7f6d-006312f92d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.800 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:21.800 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:21.801 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.801 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.802 DEBUG:boto:Method: GET 2022-09-03T06:50:21.803 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.803 DEBUG:boto:Data: 2022-09-03T06:50:21.804 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.804 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.805 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.805 DEBUG:boto:Params: {} 2022-09-03T06:50:21.806 DEBUG:boto:Token: None 2022-09-03T06:50:21.806 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.807 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UmHA6R9mmNySYC6yPF/Z46UV9Jk= 2022-09-03T06:50:21.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UmHA6R9mmNySYC6yPF/Z46UV9Jk=', 'Content-Length': '0'} 2022-09-03T06:50:21.810 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000872bd82c024d56e9-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.811 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:21.812 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:21.813 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.815 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.816 DEBUG:boto:Method: HEAD 2022-09-03T06:50:21.817 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.818 DEBUG:boto:Data: 2022-09-03T06:50:21.819 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.820 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.821 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.822 DEBUG:boto:Params: {} 2022-09-03T06:50:21.823 DEBUG:boto:Token: None 2022-09-03T06:50:21.823 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.824 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j9RS78a/q/yJLgaq9tYHmkk1iqQ= 2022-09-03T06:50:21.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j9RS78a/q/yJLgaq9tYHmkk1iqQ=', 'Content-Length': '0'} 2022-09-03T06:50:21.827 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a613c0051cfac7fc-006312f92d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.828 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:21.828 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:21.829 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.829 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.830 DEBUG:boto:Method: HEAD 2022-09-03T06:50:21.831 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.831 DEBUG:boto:Data: 2022-09-03T06:50:21.832 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.832 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.833 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.834 DEBUG:boto:Params: {} 2022-09-03T06:50:21.834 DEBUG:boto:Token: None 2022-09-03T06:50:21.835 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.836 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:j9RS78a/q/yJLgaq9tYHmkk1iqQ= 2022-09-03T06:50:21.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:j9RS78a/q/yJLgaq9tYHmkk1iqQ=', 'Content-Length': '0'} 2022-09-03T06:50:21.839 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004fa83d539e04ccf0-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.840 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:50:21.840 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:21.841 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:21.842 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.843 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.843 DEBUG:boto:Method: GET 2022-09-03T06:50:21.844 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.844 DEBUG:boto:Data: 2022-09-03T06:50:21.845 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.845 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.846 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.847 DEBUG:boto:Params: {} 2022-09-03T06:50:21.848 DEBUG:boto:Token: None 2022-09-03T06:50:21.848 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.849 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UmHA6R9mmNySYC6yPF/Z46UV9Jk= 2022-09-03T06:50:21.850 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UmHA6R9mmNySYC6yPF/Z46UV9Jk=', 'Content-Length': '0'} 2022-09-03T06:50:21.852 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a3a962eafc978dfe-006312f92d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.894 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:21.895 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:21.896 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.896 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.897 DEBUG:boto:Method: GET 2022-09-03T06:50:21.898 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.898 DEBUG:boto:Data: 2022-09-03T06:50:21.899 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.899 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.900 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.901 DEBUG:boto:Params: {} 2022-09-03T06:50:21.901 DEBUG:boto:Token: None 2022-09-03T06:50:21.902 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:21.903 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UmHA6R9mmNySYC6yPF/Z46UV9Jk= 2022-09-03T06:50:21.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UmHA6R9mmNySYC6yPF/Z46UV9Jk=', 'Content-Length': '0'} 2022-09-03T06:50:21.906 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000024f3cda5f0b60d2f-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.907 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:21.908 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:21.909 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:21.909 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:21.910 DEBUG:boto:Method: GET 2022-09-03T06:50:21.911 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:50:21.911 DEBUG:boto:Data: 2022-09-03T06:50:21.912 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.912 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.913 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.913 DEBUG:boto:Params: {} 2022-09-03T06:50:21.914 DEBUG:boto:Token: None 2022-09-03T06:50:21.915 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/ 2022-09-03T06:50:21.915 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE= 2022-09-03T06:50:21.916 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE=', 'Content-Length': '0'} 2022-09-03T06:50:21.919 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003dd5502e87b1bba9-006312f92d-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.921 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:21.922 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:21.923 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:21.924 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:21.925 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:21.926 DEBUG:boto:Method: GET 2022-09-03T06:50:21.926 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:50:21.927 DEBUG:boto:Data: 2022-09-03T06:50:21.928 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.928 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.929 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.930 DEBUG:boto:Params: {} 2022-09-03T06:50:21.931 DEBUG:boto:Token: None 2022-09-03T06:50:21.932 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/ 2022-09-03T06:50:21.933 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE= 2022-09-03T06:50:21.934 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:n/k5slJSkf3YHUC/mJutdJ9FVcE=', 'Content-Length': '0'} 2022-09-03T06:50:21.939 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dfc9578bb8036881-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.940 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:21.942 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:21.943 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:21.944 DEBUG:boto:Method: HEAD 2022-09-03T06:50:21.945 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:50:21.945 DEBUG:boto:Data: 2022-09-03T06:50:21.946 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.947 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.949 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.949 DEBUG:boto:Params: {} 2022-09-03T06:50:21.950 DEBUG:boto:Token: None 2022-09-03T06:50:21.951 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj87 2022-09-03T06:50:21.952 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JIpa/WCq2vqvNZcmHR45u/8qJOE= 2022-09-03T06:50:21.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JIpa/WCq2vqvNZcmHR45u/8qJOE=', 'Content-Length': '0'} 2022-09-03T06:50:21.956 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000935627cdad480c92-006312f92d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.957 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:21.958 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:21.958 DEBUG:boto:Method: HEAD 2022-09-03T06:50:21.959 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:50:21.959 DEBUG:boto:Data: 2022-09-03T06:50:21.960 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.960 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.961 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.962 DEBUG:boto:Params: {} 2022-09-03T06:50:21.962 DEBUG:boto:Token: None 2022-09-03T06:50:21.964 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj87 2022-09-03T06:50:21.964 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JIpa/WCq2vqvNZcmHR45u/8qJOE= 2022-09-03T06:50:21.965 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JIpa/WCq2vqvNZcmHR45u/8qJOE=', 'Content-Length': '0'} 2022-09-03T06:50:21.967 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000021772d5428a5bbe7-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.968 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:50:21.969 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:21.969 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:21.970 DEBUG:boto:Method: GET 2022-09-03T06:50:21.970 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:50:21.971 DEBUG:boto:Data: 2022-09-03T06:50:21.972 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.973 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:21.974 DEBUG:boto:Port: 8001 2022-09-03T06:50:21.975 DEBUG:boto:Params: {} 2022-09-03T06:50:21.976 DEBUG:boto:Token: None 2022-09-03T06:50:21.978 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj87 2022-09-03T06:50:21.979 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YfL4zP4+1+UgCHUbwkv7NqBkDZE= 2022-09-03T06:50:21.980 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YfL4zP4+1+UgCHUbwkv7NqBkDZE=', 'Content-Length': '0'} 2022-09-03T06:50:21.983 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000da342400418219f4-006312f92d-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.984 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:21.985 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:21.986 DEBUG:boto:Method: GET 2022-09-03T06:50:21.987 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:50:21.987 DEBUG:boto:Data: 2022-09-03T06:50:21.988 DEBUG:boto:Headers: {} 2022-09-03T06:50:21.988 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:21.989 DEBUG:boto:Port: 8000 2022-09-03T06:50:21.989 DEBUG:boto:Params: {} 2022-09-03T06:50:21.990 DEBUG:boto:Token: None 2022-09-03T06:50:21.991 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:21 GMT /dsgvcy-75/obj87 2022-09-03T06:50:21.992 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YfL4zP4+1+UgCHUbwkv7NqBkDZE= 2022-09-03T06:50:21.993 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:21 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YfL4zP4+1+UgCHUbwkv7NqBkDZE=', 'Content-Length': '0'} 2022-09-03T06:50:21.996 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008808030aebe1d941-006312f92d-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:21 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:21.998 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:21.999 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:21.999 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:22.000 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:22.001 DEBUG:boto:Method: GET 2022-09-03T06:50:22.002 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:50:22.003 DEBUG:boto:Data: 2022-09-03T06:50:22.004 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.005 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:22.006 DEBUG:boto:Port: 8001 2022-09-03T06:50:22.007 DEBUG:boto:Params: {} 2022-09-03T06:50:22.008 DEBUG:boto:Token: None 2022-09-03T06:50:22.009 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/ 2022-09-03T06:50:22.010 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q= 2022-09-03T06:50:22.010 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q=', 'Content-Length': '0'} 2022-09-03T06:50:22.014 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c9739508e0b00eba-006312f92e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.058 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:22.059 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:22.060 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:22.061 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:22.061 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:22.062 DEBUG:boto:Method: GET 2022-09-03T06:50:22.062 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:50:22.063 DEBUG:boto:Data: 2022-09-03T06:50:22.063 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.064 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:22.065 DEBUG:boto:Port: 8000 2022-09-03T06:50:22.066 DEBUG:boto:Params: {} 2022-09-03T06:50:22.066 DEBUG:boto:Token: None 2022-09-03T06:50:22.067 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/ 2022-09-03T06:50:22.068 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q= 2022-09-03T06:50:22.069 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q=', 'Content-Length': '0'} 2022-09-03T06:50:22.074 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c0a6847865ef08c8-006312f92e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.076 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:22.077 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:50:22.078 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:22.079 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:22.080 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.081 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.081 DEBUG:boto:Method: GET 2022-09-03T06:50:22.082 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.083 DEBUG:boto:Data: 2022-09-03T06:50:22.083 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.084 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:22.084 DEBUG:boto:Port: 8001 2022-09-03T06:50:22.085 DEBUG:boto:Params: {} 2022-09-03T06:50:22.085 DEBUG:boto:Token: None 2022-09-03T06:50:22.086 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.087 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6Ghc0mSxAmDkIUv7Gny6wdP1zlQ= 2022-09-03T06:50:22.087 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6Ghc0mSxAmDkIUv7Gny6wdP1zlQ=', 'Content-Length': '0'} 2022-09-03T06:50:22.090 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004142850be750a15b-006312f92e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.092 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:22.092 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:22.093 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.094 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.095 DEBUG:boto:Method: GET 2022-09-03T06:50:22.096 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.096 DEBUG:boto:Data: 2022-09-03T06:50:22.097 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.097 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:22.098 DEBUG:boto:Port: 8000 2022-09-03T06:50:22.098 DEBUG:boto:Params: {} 2022-09-03T06:50:22.099 DEBUG:boto:Token: None 2022-09-03T06:50:22.100 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.100 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6Ghc0mSxAmDkIUv7Gny6wdP1zlQ= 2022-09-03T06:50:22.101 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6Ghc0mSxAmDkIUv7Gny6wdP1zlQ=', 'Content-Length': '0'} 2022-09-03T06:50:22.104 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007d20b8d2ea0b2e95-006312f92e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.146 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:22.147 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:22.148 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.149 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.149 DEBUG:boto:Method: HEAD 2022-09-03T06:50:22.150 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.151 DEBUG:boto:Data: 2022-09-03T06:50:22.151 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.152 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:22.152 DEBUG:boto:Port: 8001 2022-09-03T06:50:22.153 DEBUG:boto:Params: {} 2022-09-03T06:50:22.153 DEBUG:boto:Token: None 2022-09-03T06:50:22.154 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.156 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h8sJXK7DAwmqJt3/YD5YMIdiGHc= 2022-09-03T06:50:22.157 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h8sJXK7DAwmqJt3/YD5YMIdiGHc=', 'Content-Length': '0'} 2022-09-03T06:50:22.159 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000067bcdf5a22e550aa-006312f92e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.160 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:22.161 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:22.162 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.162 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.163 DEBUG:boto:Method: HEAD 2022-09-03T06:50:22.163 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.164 DEBUG:boto:Data: 2022-09-03T06:50:22.165 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.165 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:22.166 DEBUG:boto:Port: 8000 2022-09-03T06:50:22.167 DEBUG:boto:Params: {} 2022-09-03T06:50:22.168 DEBUG:boto:Token: None 2022-09-03T06:50:22.169 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.169 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:h8sJXK7DAwmqJt3/YD5YMIdiGHc= 2022-09-03T06:50:22.170 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:h8sJXK7DAwmqJt3/YD5YMIdiGHc=', 'Content-Length': '0'} 2022-09-03T06:50:22.172 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c484d8bec9537ca1-006312f92e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.173 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:50:22.174 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:22.174 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:22.175 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.176 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.176 DEBUG:boto:Method: GET 2022-09-03T06:50:22.177 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.177 DEBUG:boto:Data: 2022-09-03T06:50:22.178 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.179 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:22.179 DEBUG:boto:Port: 8001 2022-09-03T06:50:22.180 DEBUG:boto:Params: {} 2022-09-03T06:50:22.180 DEBUG:boto:Token: None 2022-09-03T06:50:22.181 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.181 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6Ghc0mSxAmDkIUv7Gny6wdP1zlQ= 2022-09-03T06:50:22.182 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6Ghc0mSxAmDkIUv7Gny6wdP1zlQ=', 'Content-Length': '0'} 2022-09-03T06:50:22.184 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d3a2f420100a5ad2-006312f92e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.185 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:22.186 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:22.186 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.187 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.188 DEBUG:boto:Method: GET 2022-09-03T06:50:22.190 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.191 DEBUG:boto:Data: 2022-09-03T06:50:22.191 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.192 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:22.192 DEBUG:boto:Port: 8000 2022-09-03T06:50:22.193 DEBUG:boto:Params: {} 2022-09-03T06:50:22.193 DEBUG:boto:Token: None 2022-09-03T06:50:22.194 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:22.195 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6Ghc0mSxAmDkIUv7Gny6wdP1zlQ= 2022-09-03T06:50:22.195 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6Ghc0mSxAmDkIUv7Gny6wdP1zlQ=', 'Content-Length': '0'} 2022-09-03T06:50:22.197 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002294bd9f1c07d3ba-006312f92e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.199 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:22.199 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:22.200 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:22.201 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:22.202 DEBUG:boto:Method: GET 2022-09-03T06:50:22.203 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:50:22.204 DEBUG:boto:Data: 2022-09-03T06:50:22.205 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.206 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:22.207 DEBUG:boto:Port: 8001 2022-09-03T06:50:22.208 DEBUG:boto:Params: {} 2022-09-03T06:50:22.209 DEBUG:boto:Token: None 2022-09-03T06:50:22.211 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/ 2022-09-03T06:50:22.211 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q= 2022-09-03T06:50:22.212 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q=', 'Content-Length': '0'} 2022-09-03T06:50:22.215 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000088f380f8b164c959-006312f92e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.215 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:22.216 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:22.217 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:22.217 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:22.218 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:22.218 DEBUG:boto:Method: GET 2022-09-03T06:50:22.219 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:50:22.220 DEBUG:boto:Data: 2022-09-03T06:50:22.220 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.221 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:22.221 DEBUG:boto:Port: 8000 2022-09-03T06:50:22.222 DEBUG:boto:Params: {} 2022-09-03T06:50:22.222 DEBUG:boto:Token: None 2022-09-03T06:50:22.223 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/ 2022-09-03T06:50:22.224 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q= 2022-09-03T06:50:22.224 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q=', 'Content-Length': '0'} 2022-09-03T06:50:22.229 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007002bf9748d7a36e-006312f92e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.230 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:22.231 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:22.231 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:22.232 DEBUG:boto:Method: HEAD 2022-09-03T06:50:22.233 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:50:22.233 DEBUG:boto:Data: 2022-09-03T06:50:22.234 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.235 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:22.236 DEBUG:boto:Port: 8001 2022-09-03T06:50:22.237 DEBUG:boto:Params: {} 2022-09-03T06:50:22.238 DEBUG:boto:Token: None 2022-09-03T06:50:22.239 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj92 2022-09-03T06:50:22.240 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pCnPNNQt495/xH8I6eQV5E2hobM= 2022-09-03T06:50:22.241 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pCnPNNQt495/xH8I6eQV5E2hobM=', 'Content-Length': '0'} 2022-09-03T06:50:22.243 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002123ae13d664f16c-006312f92e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.245 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:22.245 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:22.246 DEBUG:boto:Method: HEAD 2022-09-03T06:50:22.247 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:50:22.248 DEBUG:boto:Data: 2022-09-03T06:50:22.249 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.250 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:22.251 DEBUG:boto:Port: 8000 2022-09-03T06:50:22.252 DEBUG:boto:Params: {} 2022-09-03T06:50:22.253 DEBUG:boto:Token: None 2022-09-03T06:50:22.254 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj92 2022-09-03T06:50:22.255 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pCnPNNQt495/xH8I6eQV5E2hobM= 2022-09-03T06:50:22.256 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pCnPNNQt495/xH8I6eQV5E2hobM=', 'Content-Length': '0'} 2022-09-03T06:50:22.259 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000070d28c645ab641d-006312f92e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.260 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:50:22.261 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:22.262 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:22.263 DEBUG:boto:Method: GET 2022-09-03T06:50:22.264 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:50:22.265 DEBUG:boto:Data: 2022-09-03T06:50:22.266 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.267 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:22.268 DEBUG:boto:Port: 8001 2022-09-03T06:50:22.269 DEBUG:boto:Params: {} 2022-09-03T06:50:22.270 DEBUG:boto:Token: None 2022-09-03T06:50:22.273 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj92 2022-09-03T06:50:22.274 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6qFvIEUstJykDrQ0nmnoHH1PqcI= 2022-09-03T06:50:22.275 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6qFvIEUstJykDrQ0nmnoHH1PqcI=', 'Content-Length': '0'} 2022-09-03T06:50:22.278 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000030f6febd8bc150cc-006312f92e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.318 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:22.319 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:22.320 DEBUG:boto:Method: GET 2022-09-03T06:50:22.320 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:50:22.321 DEBUG:boto:Data: 2022-09-03T06:50:22.321 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.322 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:22.325 DEBUG:boto:Port: 8000 2022-09-03T06:50:22.326 DEBUG:boto:Params: {} 2022-09-03T06:50:22.328 DEBUG:boto:Token: None 2022-09-03T06:50:22.329 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj92 2022-09-03T06:50:22.330 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6qFvIEUstJykDrQ0nmnoHH1PqcI= 2022-09-03T06:50:22.338 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6qFvIEUstJykDrQ0nmnoHH1PqcI=', 'Content-Length': '0'} 2022-09-03T06:50:22.341 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f88e56262f3e6bb8-006312f92e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.343 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:22.346 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:22.347 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:22.348 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:22.349 DEBUG:boto:Method: GET 2022-09-03T06:50:22.350 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:50:22.351 DEBUG:boto:Data: 2022-09-03T06:50:22.352 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.353 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:22.353 DEBUG:boto:Port: 8001 2022-09-03T06:50:22.354 DEBUG:boto:Params: {} 2022-09-03T06:50:22.355 DEBUG:boto:Token: None 2022-09-03T06:50:22.355 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/ 2022-09-03T06:50:22.356 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q= 2022-09-03T06:50:22.357 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q=', 'Content-Length': '0'} 2022-09-03T06:50:22.360 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a4d4ea899992de82-006312f92e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.360 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:22.362 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:22.363 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:22.365 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:22.365 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:22.366 DEBUG:boto:Method: GET 2022-09-03T06:50:22.367 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:50:22.368 DEBUG:boto:Data: 2022-09-03T06:50:22.369 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.370 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:22.371 DEBUG:boto:Port: 8000 2022-09-03T06:50:22.372 DEBUG:boto:Params: {} 2022-09-03T06:50:22.373 DEBUG:boto:Token: None 2022-09-03T06:50:22.374 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/ 2022-09-03T06:50:22.376 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q= 2022-09-03T06:50:22.377 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q=', 'Content-Length': '0'} 2022-09-03T06:50:22.382 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003d737b905b9d6b99-006312f92e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.383 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:22.385 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2022-09-03T06:50:22.385 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:22.386 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:22.387 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.388 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.390 DEBUG:boto:Method: GET 2022-09-03T06:50:22.391 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.392 DEBUG:boto:Data: 2022-09-03T06:50:22.393 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.393 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:22.394 DEBUG:boto:Port: 8001 2022-09-03T06:50:22.395 DEBUG:boto:Params: {} 2022-09-03T06:50:22.396 DEBUG:boto:Token: None 2022-09-03T06:50:22.398 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.399 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TgTs2Sw30/bgCOE1/p6qmW+y0EY= 2022-09-03T06:50:22.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TgTs2Sw30/bgCOE1/p6qmW+y0EY=', 'Content-Length': '0'} 2022-09-03T06:50:22.403 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007c15bbd77d695a50-006312f92e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.404 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:22.405 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:22.406 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.406 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.407 DEBUG:boto:Method: GET 2022-09-03T06:50:22.407 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.408 DEBUG:boto:Data: 2022-09-03T06:50:22.408 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.409 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:22.410 DEBUG:boto:Port: 8000 2022-09-03T06:50:22.411 DEBUG:boto:Params: {} 2022-09-03T06:50:22.411 DEBUG:boto:Token: None 2022-09-03T06:50:22.413 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.413 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TgTs2Sw30/bgCOE1/p6qmW+y0EY= 2022-09-03T06:50:22.414 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TgTs2Sw30/bgCOE1/p6qmW+y0EY=', 'Content-Length': '0'} 2022-09-03T06:50:22.417 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a0b4ed9e09e73bb4-006312f92e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.418 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:22.418 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:22.419 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.419 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.420 DEBUG:boto:Method: HEAD 2022-09-03T06:50:22.421 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.422 DEBUG:boto:Data: 2022-09-03T06:50:22.423 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.424 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:22.424 DEBUG:boto:Port: 8001 2022-09-03T06:50:22.425 DEBUG:boto:Params: {} 2022-09-03T06:50:22.426 DEBUG:boto:Token: None 2022-09-03T06:50:22.427 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.428 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OQW+RwHrPZt7NzPbjvdy/NFNlqc= 2022-09-03T06:50:22.429 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OQW+RwHrPZt7NzPbjvdy/NFNlqc=', 'Content-Length': '0'} 2022-09-03T06:50:22.431 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003b88ddfe3f2a276e-006312f92e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.432 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:22.433 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:22.433 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.434 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.435 DEBUG:boto:Method: HEAD 2022-09-03T06:50:22.436 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.437 DEBUG:boto:Data: 2022-09-03T06:50:22.437 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.439 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:22.440 DEBUG:boto:Port: 8000 2022-09-03T06:50:22.440 DEBUG:boto:Params: {} 2022-09-03T06:50:22.441 DEBUG:boto:Token: None 2022-09-03T06:50:22.442 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.442 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OQW+RwHrPZt7NzPbjvdy/NFNlqc= 2022-09-03T06:50:22.443 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OQW+RwHrPZt7NzPbjvdy/NFNlqc=', 'Content-Length': '0'} 2022-09-03T06:50:22.445 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003e18d07925db329d-006312f92e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.448 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2022-09-03T06:50:22.448 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:22.449 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:22.449 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.450 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.450 DEBUG:boto:Method: GET 2022-09-03T06:50:22.451 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.452 DEBUG:boto:Data: 2022-09-03T06:50:22.453 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.453 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:22.454 DEBUG:boto:Port: 8001 2022-09-03T06:50:22.454 DEBUG:boto:Params: {} 2022-09-03T06:50:22.455 DEBUG:boto:Token: None 2022-09-03T06:50:22.456 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.456 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TgTs2Sw30/bgCOE1/p6qmW+y0EY= 2022-09-03T06:50:22.458 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TgTs2Sw30/bgCOE1/p6qmW+y0EY=', 'Content-Length': '0'} 2022-09-03T06:50:22.460 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a6da7bb308771314-006312f92e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.506 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:22.507 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:22.508 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.508 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.509 DEBUG:boto:Method: GET 2022-09-03T06:50:22.510 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.511 DEBUG:boto:Data: 2022-09-03T06:50:22.512 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.512 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:22.513 DEBUG:boto:Port: 8000 2022-09-03T06:50:22.513 DEBUG:boto:Params: {} 2022-09-03T06:50:22.514 DEBUG:boto:Token: None 2022-09-03T06:50:22.515 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:22.515 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TgTs2Sw30/bgCOE1/p6qmW+y0EY= 2022-09-03T06:50:22.516 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TgTs2Sw30/bgCOE1/p6qmW+y0EY=', 'Content-Length': '0'} 2022-09-03T06:50:22.518 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001e9fc7eb1d7dada3-006312f92e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.519 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:22.520 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:22.521 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:22.521 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:22.522 DEBUG:boto:Method: GET 2022-09-03T06:50:22.522 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:50:22.523 DEBUG:boto:Data: 2022-09-03T06:50:22.524 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.524 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:22.525 DEBUG:boto:Port: 8001 2022-09-03T06:50:22.525 DEBUG:boto:Params: {} 2022-09-03T06:50:22.526 DEBUG:boto:Token: None 2022-09-03T06:50:22.527 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/ 2022-09-03T06:50:22.527 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q= 2022-09-03T06:50:22.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q=', 'Content-Length': '0'} 2022-09-03T06:50:22.531 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d3b73ae9761feda6-006312f92e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.532 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:22.533 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:22.533 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:22.534 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:22.535 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:22.535 DEBUG:boto:Method: GET 2022-09-03T06:50:22.536 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:50:22.536 DEBUG:boto:Data: 2022-09-03T06:50:22.537 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.538 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:22.538 DEBUG:boto:Port: 8000 2022-09-03T06:50:22.539 DEBUG:boto:Params: {} 2022-09-03T06:50:22.540 DEBUG:boto:Token: None 2022-09-03T06:50:22.541 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/ 2022-09-03T06:50:22.542 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q= 2022-09-03T06:50:22.542 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q=', 'Content-Length': '0'} 2022-09-03T06:50:22.547 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e31147a5faac6308-006312f92e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.590 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:22.592 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:22.593 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:22.594 DEBUG:boto:Method: HEAD 2022-09-03T06:50:22.595 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:50:22.596 DEBUG:boto:Data: 2022-09-03T06:50:22.597 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.598 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:22.599 DEBUG:boto:Port: 8001 2022-09-03T06:50:22.600 DEBUG:boto:Params: {} 2022-09-03T06:50:22.601 DEBUG:boto:Token: None 2022-09-03T06:50:22.601 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj93 2022-09-03T06:50:22.602 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H1e/I98m4//V7THZdf4Cw5Vn+uI= 2022-09-03T06:50:22.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H1e/I98m4//V7THZdf4Cw5Vn+uI=', 'Content-Length': '0'} 2022-09-03T06:50:22.608 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004c99c672550149c4-006312f92e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.610 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:22.611 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:22.612 DEBUG:boto:Method: HEAD 2022-09-03T06:50:22.613 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:50:22.615 DEBUG:boto:Data: 2022-09-03T06:50:22.616 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.617 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:22.617 DEBUG:boto:Port: 8000 2022-09-03T06:50:22.618 DEBUG:boto:Params: {} 2022-09-03T06:50:22.619 DEBUG:boto:Token: None 2022-09-03T06:50:22.620 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj93 2022-09-03T06:50:22.622 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H1e/I98m4//V7THZdf4Cw5Vn+uI= 2022-09-03T06:50:22.624 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H1e/I98m4//V7THZdf4Cw5Vn+uI=', 'Content-Length': '0'} 2022-09-03T06:50:22.626 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000048e1c9799b5ebaf9-006312f92e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.627 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2022-09-03T06:50:22.628 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:22.629 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:22.630 DEBUG:boto:Method: GET 2022-09-03T06:50:22.631 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:50:22.632 DEBUG:boto:Data: 2022-09-03T06:50:22.632 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.633 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:22.634 DEBUG:boto:Port: 8001 2022-09-03T06:50:22.634 DEBUG:boto:Params: {} 2022-09-03T06:50:22.635 DEBUG:boto:Token: None 2022-09-03T06:50:22.635 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj93 2022-09-03T06:50:22.636 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DAYv91WvWqOID8PjwZgiWws26QE= 2022-09-03T06:50:22.637 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DAYv91WvWqOID8PjwZgiWws26QE=', 'Content-Length': '0'} 2022-09-03T06:50:22.639 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e735ca92dc394524-006312f92e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.640 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:22.641 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:22.642 DEBUG:boto:Method: GET 2022-09-03T06:50:22.642 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:50:22.643 DEBUG:boto:Data: 2022-09-03T06:50:22.644 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.645 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:22.645 DEBUG:boto:Port: 8000 2022-09-03T06:50:22.646 DEBUG:boto:Params: {} 2022-09-03T06:50:22.648 DEBUG:boto:Token: None 2022-09-03T06:50:22.649 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj93 2022-09-03T06:50:22.650 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DAYv91WvWqOID8PjwZgiWws26QE= 2022-09-03T06:50:22.651 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DAYv91WvWqOID8PjwZgiWws26QE=', 'Content-Length': '0'} 2022-09-03T06:50:22.654 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e787516b2bcb098d-006312f92e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.656 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:22.657 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:22.658 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:22.658 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:22.659 DEBUG:boto:Method: GET 2022-09-03T06:50:22.659 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:50:22.660 DEBUG:boto:Data: 2022-09-03T06:50:22.660 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.661 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:22.662 DEBUG:boto:Port: 8001 2022-09-03T06:50:22.663 DEBUG:boto:Params: {} 2022-09-03T06:50:22.665 DEBUG:boto:Token: None 2022-09-03T06:50:22.666 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/ 2022-09-03T06:50:22.667 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q= 2022-09-03T06:50:22.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q=', 'Content-Length': '0'} 2022-09-03T06:50:22.672 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000062bf5b50770c444d-006312f92e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.674 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:22.676 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:22.677 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:22.678 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:22.679 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:22.680 DEBUG:boto:Method: GET 2022-09-03T06:50:22.681 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:50:22.682 DEBUG:boto:Data: 2022-09-03T06:50:22.682 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.683 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:22.683 DEBUG:boto:Port: 8000 2022-09-03T06:50:22.684 DEBUG:boto:Params: {} 2022-09-03T06:50:22.685 DEBUG:boto:Token: None 2022-09-03T06:50:22.686 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/ 2022-09-03T06:50:22.686 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q= 2022-09-03T06:50:22.687 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q=', 'Content-Length': '0'} 2022-09-03T06:50:22.699 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005e1d61bbbd82db7a-006312f92e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.742 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:22.744 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:50:22.745 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:22.746 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:22.747 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.748 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.749 DEBUG:boto:Method: GET 2022-09-03T06:50:22.750 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.751 DEBUG:boto:Data: 2022-09-03T06:50:22.752 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.753 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:22.754 DEBUG:boto:Port: 8001 2022-09-03T06:50:22.755 DEBUG:boto:Params: {} 2022-09-03T06:50:22.757 DEBUG:boto:Token: None 2022-09-03T06:50:22.758 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.759 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:saj6kaSSisFQ8hodRJdEudolNks= 2022-09-03T06:50:22.760 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:saj6kaSSisFQ8hodRJdEudolNks=', 'Content-Length': '0'} 2022-09-03T06:50:22.763 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f7500fa8752b9005-006312f92e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.765 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:22.766 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:22.767 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.768 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.769 DEBUG:boto:Method: GET 2022-09-03T06:50:22.770 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.771 DEBUG:boto:Data: 2022-09-03T06:50:22.772 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.773 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:22.774 DEBUG:boto:Port: 8000 2022-09-03T06:50:22.775 DEBUG:boto:Params: {} 2022-09-03T06:50:22.776 DEBUG:boto:Token: None 2022-09-03T06:50:22.777 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.778 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:saj6kaSSisFQ8hodRJdEudolNks= 2022-09-03T06:50:22.779 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:saj6kaSSisFQ8hodRJdEudolNks=', 'Content-Length': '0'} 2022-09-03T06:50:22.784 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f89830f53d0f9898-006312f92e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.788 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:22.789 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:22.790 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.791 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.792 DEBUG:boto:Method: HEAD 2022-09-03T06:50:22.793 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.795 DEBUG:boto:Data: 2022-09-03T06:50:22.796 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.796 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:22.797 DEBUG:boto:Port: 8001 2022-09-03T06:50:22.798 DEBUG:boto:Params: {} 2022-09-03T06:50:22.798 DEBUG:boto:Token: None 2022-09-03T06:50:22.799 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.800 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fJZ1y1jEL+tJZ3KNImnD56l49cM= 2022-09-03T06:50:22.800 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fJZ1y1jEL+tJZ3KNImnD56l49cM=', 'Content-Length': '0'} 2022-09-03T06:50:22.802 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000162786b6a15caca0-006312f92e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.803 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:22.804 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:22.805 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.805 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.806 DEBUG:boto:Method: HEAD 2022-09-03T06:50:22.806 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.807 DEBUG:boto:Data: 2022-09-03T06:50:22.807 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.808 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:22.808 DEBUG:boto:Port: 8000 2022-09-03T06:50:22.809 DEBUG:boto:Params: {} 2022-09-03T06:50:22.810 DEBUG:boto:Token: None 2022-09-03T06:50:22.810 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.811 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fJZ1y1jEL+tJZ3KNImnD56l49cM= 2022-09-03T06:50:22.812 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fJZ1y1jEL+tJZ3KNImnD56l49cM=', 'Content-Length': '0'} 2022-09-03T06:50:22.814 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a696e2895d8c4eaa-006312f92e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.815 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:50:22.815 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:22.816 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:22.817 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.817 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.818 DEBUG:boto:Method: GET 2022-09-03T06:50:22.818 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.819 DEBUG:boto:Data: 2022-09-03T06:50:22.819 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.820 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:22.820 DEBUG:boto:Port: 8001 2022-09-03T06:50:22.821 DEBUG:boto:Params: {} 2022-09-03T06:50:22.822 DEBUG:boto:Token: None 2022-09-03T06:50:22.823 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.823 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:saj6kaSSisFQ8hodRJdEudolNks= 2022-09-03T06:50:22.824 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:saj6kaSSisFQ8hodRJdEudolNks=', 'Content-Length': '0'} 2022-09-03T06:50:22.827 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000033bb6b66dc237922-006312f92e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.870 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:22.871 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:22.872 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.872 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.873 DEBUG:boto:Method: GET 2022-09-03T06:50:22.873 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.874 DEBUG:boto:Data: 2022-09-03T06:50:22.874 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.875 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:22.876 DEBUG:boto:Port: 8000 2022-09-03T06:50:22.876 DEBUG:boto:Params: {} 2022-09-03T06:50:22.877 DEBUG:boto:Token: None 2022-09-03T06:50:22.878 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:22.880 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:saj6kaSSisFQ8hodRJdEudolNks= 2022-09-03T06:50:22.883 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:saj6kaSSisFQ8hodRJdEudolNks=', 'Content-Length': '0'} 2022-09-03T06:50:22.885 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cfa493cebd38caaa-006312f92e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.886 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:22.887 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:22.888 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:22.889 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:22.890 DEBUG:boto:Method: GET 2022-09-03T06:50:22.892 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:50:22.893 DEBUG:boto:Data: 2022-09-03T06:50:22.894 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.894 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:22.897 DEBUG:boto:Port: 8001 2022-09-03T06:50:22.898 DEBUG:boto:Params: {} 2022-09-03T06:50:22.898 DEBUG:boto:Token: None 2022-09-03T06:50:22.899 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/ 2022-09-03T06:50:22.902 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q= 2022-09-03T06:50:22.903 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q=', 'Content-Length': '0'} 2022-09-03T06:50:22.907 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d65707d77e2ef723-006312f92e-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.909 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:22.911 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:22.912 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:22.913 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:22.916 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:22.916 DEBUG:boto:Method: GET 2022-09-03T06:50:22.917 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:50:22.918 DEBUG:boto:Data: 2022-09-03T06:50:22.918 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.919 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:22.919 DEBUG:boto:Port: 8000 2022-09-03T06:50:22.920 DEBUG:boto:Params: {} 2022-09-03T06:50:22.920 DEBUG:boto:Token: None 2022-09-03T06:50:22.921 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/ 2022-09-03T06:50:22.922 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q= 2022-09-03T06:50:22.922 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nLcFa3zZd0x1bT57fpvKbrtwD5Q=', 'Content-Length': '0'} 2022-09-03T06:50:22.926 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005c52ac7ffe2cc24b-006312f92e-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.927 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:22.928 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:22.929 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:22.929 DEBUG:boto:Method: HEAD 2022-09-03T06:50:22.930 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:50:22.930 DEBUG:boto:Data: 2022-09-03T06:50:22.931 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.932 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:22.932 DEBUG:boto:Port: 8001 2022-09-03T06:50:22.933 DEBUG:boto:Params: {} 2022-09-03T06:50:22.933 DEBUG:boto:Token: None 2022-09-03T06:50:22.934 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj95 2022-09-03T06:50:22.935 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OCN3ra+P+3T9lsfGd4CE0HeSCY4= 2022-09-03T06:50:22.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OCN3ra+P+3T9lsfGd4CE0HeSCY4=', 'Content-Length': '0'} 2022-09-03T06:50:22.938 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c51e86519997c8a-006312f92e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.939 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:22.939 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:22.940 DEBUG:boto:Method: HEAD 2022-09-03T06:50:22.940 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:50:22.941 DEBUG:boto:Data: 2022-09-03T06:50:22.941 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.942 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:22.943 DEBUG:boto:Port: 8000 2022-09-03T06:50:22.943 DEBUG:boto:Params: {} 2022-09-03T06:50:22.945 DEBUG:boto:Token: None 2022-09-03T06:50:22.946 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj95 2022-09-03T06:50:22.946 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OCN3ra+P+3T9lsfGd4CE0HeSCY4= 2022-09-03T06:50:22.947 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OCN3ra+P+3T9lsfGd4CE0HeSCY4=', 'Content-Length': '0'} 2022-09-03T06:50:22.949 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000028e82e8ea2f10df2-006312f92e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.950 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:50:22.951 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:22.952 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:22.953 DEBUG:boto:Method: GET 2022-09-03T06:50:22.954 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:50:22.954 DEBUG:boto:Data: 2022-09-03T06:50:22.955 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.955 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:22.956 DEBUG:boto:Port: 8001 2022-09-03T06:50:22.957 DEBUG:boto:Params: {} 2022-09-03T06:50:22.957 DEBUG:boto:Token: None 2022-09-03T06:50:22.958 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj95 2022-09-03T06:50:22.958 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IHHAmhiQ+35naccGnixQLTbyLrE= 2022-09-03T06:50:22.959 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IHHAmhiQ+35naccGnixQLTbyLrE=', 'Content-Length': '0'} 2022-09-03T06:50:22.962 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e401e6eb61babcf0-006312f92e-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:22.963 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:22.964 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:22.965 DEBUG:boto:Method: GET 2022-09-03T06:50:22.965 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:50:22.967 DEBUG:boto:Data: 2022-09-03T06:50:22.968 DEBUG:boto:Headers: {} 2022-09-03T06:50:22.968 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:22.969 DEBUG:boto:Port: 8000 2022-09-03T06:50:22.970 DEBUG:boto:Params: {} 2022-09-03T06:50:22.972 DEBUG:boto:Token: None 2022-09-03T06:50:22.972 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:22 GMT /dsgvcy-75/obj95 2022-09-03T06:50:22.973 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:IHHAmhiQ+35naccGnixQLTbyLrE= 2022-09-03T06:50:22.974 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:22 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:IHHAmhiQ+35naccGnixQLTbyLrE=', 'Content-Length': '0'} 2022-09-03T06:50:22.976 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007840a4a356f1055a-006312f92e-15d7-test-zone1'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:22 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.019 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:23.020 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:23.021 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:23.022 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:23.022 DEBUG:boto:Method: GET 2022-09-03T06:50:23.023 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:50:23.024 DEBUG:boto:Data: 2022-09-03T06:50:23.024 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.025 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:23.026 DEBUG:boto:Port: 8001 2022-09-03T06:50:23.027 DEBUG:boto:Params: {} 2022-09-03T06:50:23.028 DEBUG:boto:Token: None 2022-09-03T06:50:23.029 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/ 2022-09-03T06:50:23.030 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw= 2022-09-03T06:50:23.032 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw=', 'Content-Length': '0'} 2022-09-03T06:50:23.036 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000859d9bafc63c23d3-006312f92f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.037 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:23.038 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:23.039 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:23.040 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:23.040 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:23.041 DEBUG:boto:Method: GET 2022-09-03T06:50:23.042 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:50:23.043 DEBUG:boto:Data: 2022-09-03T06:50:23.043 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.044 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:50:23.045 DEBUG:boto:Port: 8000 2022-09-03T06:50:23.046 DEBUG:boto:Params: {} 2022-09-03T06:50:23.047 DEBUG:boto:Token: None 2022-09-03T06:50:23.048 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/ 2022-09-03T06:50:23.048 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw= 2022-09-03T06:50:23.049 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw=', 'Content-Length': '0'} 2022-09-03T06:50:23.054 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000b3907db6d81e51c-006312f92f-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.055 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:23.056 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-75 zones={test-zone3, test-zone1} 2022-09-03T06:50:23.057 INFO:tasks.rgw_multi.zone_rados:comparing bucket=dsgvcy-75 zones={test-zone3, test-zone2} 2022-09-03T06:50:23.058 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:23.058 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:23.059 DEBUG:boto:Method: HEAD 2022-09-03T06:50:23.060 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:50:23.061 DEBUG:boto:Data: 2022-09-03T06:50:23.062 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.063 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:23.064 DEBUG:boto:Port: 8001 2022-09-03T06:50:23.064 DEBUG:boto:Params: {} 2022-09-03T06:50:23.065 DEBUG:boto:Token: None 2022-09-03T06:50:23.066 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/ 2022-09-03T06:50:23.066 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ymPopnpltkHIqKV0LMGeiV7zlPE= 2022-09-03T06:50:23.067 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ymPopnpltkHIqKV0LMGeiV7zlPE=', 'Content-Length': '0'} 2022-09-03T06:50:23.070 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '35'), ('X-RGW-Bytes-Used', '105'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000002eae032189e7bf5c-006312f92f-1851-test-zone3'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.071 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:23.072 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:23.073 DEBUG:boto:Method: HEAD 2022-09-03T06:50:23.073 DEBUG:boto:Path: /dsgvcy-75/ 2022-09-03T06:50:23.074 DEBUG:boto:Data: 2022-09-03T06:50:23.075 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.076 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:23.077 DEBUG:boto:Port: 8000 2022-09-03T06:50:23.077 DEBUG:boto:Params: {} 2022-09-03T06:50:23.078 DEBUG:boto:Token: None 2022-09-03T06:50:23.079 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/ 2022-09-03T06:50:23.080 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ymPopnpltkHIqKV0LMGeiV7zlPE= 2022-09-03T06:50:23.081 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ymPopnpltkHIqKV0LMGeiV7zlPE=', 'Content-Length': '0'} 2022-09-03T06:50:23.086 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '35'), ('X-RGW-Bytes-Used', '105'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000000e54aa5177a14068-006312f92f-188c-test-zone2'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.087 DEBUG:tasks.rgw_multi.zone_rados:bucket1 objects: 2022-09-03T06:50:23.087 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:23.088 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:23.088 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:23.089 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:23.090 DEBUG:boto:Method: GET 2022-09-03T06:50:23.090 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:23.091 DEBUG:boto:Data: 2022-09-03T06:50:23.091 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.092 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:23.092 DEBUG:boto:Port: 8001 2022-09-03T06:50:23.093 DEBUG:boto:Params: {} 2022-09-03T06:50:23.093 DEBUG:boto:Token: None 2022-09-03T06:50:23.094 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/?versions 2022-09-03T06:50:23.095 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3u6AyI6/ojs5lZ18oOwlwou0TMI= 2022-09-03T06:50:23.095 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3u6AyI6/ojs5lZ18oOwlwou0TMI=', 'Content-Length': '0'} 2022-09-03T06:50:23.098 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000026a896f071f95df8-006312f92f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.099 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj111nulltrue2022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj148nulltrue2022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj185nulltrue2022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj37nulltrue2022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj74nulltrue2022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:23.101 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:50:23.102 DEBUG:tasks.rgw_multi.zone_rados:o=obj111 2022-09-03T06:50:23.103 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:50:23.103 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2022-09-03T06:50:23.104 DEBUG:tasks.rgw_multi.zone_rados:o=obj124 2022-09-03T06:50:23.104 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2022-09-03T06:50:23.105 DEBUG:tasks.rgw_multi.zone_rados:o=obj148 2022-09-03T06:50:23.105 DEBUG:tasks.rgw_multi.zone_rados:o=obj155 2022-09-03T06:50:23.106 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:50:23.106 DEBUG:tasks.rgw_multi.zone_rados:o=obj185 2022-09-03T06:50:23.107 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:50:23.107 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:50:23.108 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:50:23.108 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2022-09-03T06:50:23.109 DEBUG:tasks.rgw_multi.zone_rados:o=obj31 2022-09-03T06:50:23.109 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:50:23.110 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:50:23.110 DEBUG:tasks.rgw_multi.zone_rados:o=obj37 2022-09-03T06:50:23.111 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:50:23.112 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:50:23.113 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:50:23.114 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:50:23.115 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:50:23.115 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:50:23.116 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2022-09-03T06:50:23.116 DEBUG:tasks.rgw_multi.zone_rados:o=obj62 2022-09-03T06:50:23.117 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:50:23.117 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:50:23.118 DEBUG:tasks.rgw_multi.zone_rados:o=obj74 2022-09-03T06:50:23.118 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:50:23.119 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:50:23.120 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2022-09-03T06:50:23.121 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:50:23.121 DEBUG:tasks.rgw_multi.zone_rados:o=obj93 2022-09-03T06:50:23.122 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:50:23.123 DEBUG:tasks.rgw_multi.zone_rados:bucket2 objects: 2022-09-03T06:50:23.125 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:23.125 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:23.126 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:23.126 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:23.127 DEBUG:boto:Method: GET 2022-09-03T06:50:23.127 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:23.128 DEBUG:boto:Data: 2022-09-03T06:50:23.129 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.129 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:23.130 DEBUG:boto:Port: 8000 2022-09-03T06:50:23.130 DEBUG:boto:Params: {} 2022-09-03T06:50:23.131 DEBUG:boto:Token: None 2022-09-03T06:50:23.131 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/?versions 2022-09-03T06:50:23.132 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3u6AyI6/ojs5lZ18oOwlwou0TMI= 2022-09-03T06:50:23.133 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3u6AyI6/ojs5lZ18oOwlwou0TMI=', 'Content-Length': '0'} 2022-09-03T06:50:23.136 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c438bc51c0c77dd1-006312f92f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.137 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj111nulltrue2022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj148nulltrue2022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj185nulltrue2022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj37nulltrue2022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj74nulltrue2022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:23.140 DEBUG:tasks.rgw_multi.zone_rados:o=obj11 2022-09-03T06:50:23.141 DEBUG:tasks.rgw_multi.zone_rados:o=obj111 2022-09-03T06:50:23.141 DEBUG:tasks.rgw_multi.zone_rados:o=obj115 2022-09-03T06:50:23.142 DEBUG:tasks.rgw_multi.zone_rados:o=obj116 2022-09-03T06:50:23.142 DEBUG:tasks.rgw_multi.zone_rados:o=obj124 2022-09-03T06:50:23.143 DEBUG:tasks.rgw_multi.zone_rados:o=obj145 2022-09-03T06:50:23.143 DEBUG:tasks.rgw_multi.zone_rados:o=obj148 2022-09-03T06:50:23.143 DEBUG:tasks.rgw_multi.zone_rados:o=obj155 2022-09-03T06:50:23.144 DEBUG:tasks.rgw_multi.zone_rados:o=obj17 2022-09-03T06:50:23.144 DEBUG:tasks.rgw_multi.zone_rados:o=obj185 2022-09-03T06:50:23.145 DEBUG:tasks.rgw_multi.zone_rados:o=obj19 2022-09-03T06:50:23.145 DEBUG:tasks.rgw_multi.zone_rados:o=obj22 2022-09-03T06:50:23.146 DEBUG:tasks.rgw_multi.zone_rados:o=obj23 2022-09-03T06:50:23.146 DEBUG:tasks.rgw_multi.zone_rados:o=obj29 2022-09-03T06:50:23.147 DEBUG:tasks.rgw_multi.zone_rados:o=obj31 2022-09-03T06:50:23.147 DEBUG:tasks.rgw_multi.zone_rados:o=obj33 2022-09-03T06:50:23.148 DEBUG:tasks.rgw_multi.zone_rados:o=obj34 2022-09-03T06:50:23.148 DEBUG:tasks.rgw_multi.zone_rados:o=obj37 2022-09-03T06:50:23.149 DEBUG:tasks.rgw_multi.zone_rados:o=obj38 2022-09-03T06:50:23.149 DEBUG:tasks.rgw_multi.zone_rados:o=obj44 2022-09-03T06:50:23.150 DEBUG:tasks.rgw_multi.zone_rados:o=obj46 2022-09-03T06:50:23.150 DEBUG:tasks.rgw_multi.zone_rados:o=obj51 2022-09-03T06:50:23.151 DEBUG:tasks.rgw_multi.zone_rados:o=obj55 2022-09-03T06:50:23.151 DEBUG:tasks.rgw_multi.zone_rados:o=obj57 2022-09-03T06:50:23.151 DEBUG:tasks.rgw_multi.zone_rados:o=obj58 2022-09-03T06:50:23.152 DEBUG:tasks.rgw_multi.zone_rados:o=obj62 2022-09-03T06:50:23.152 DEBUG:tasks.rgw_multi.zone_rados:o=obj68 2022-09-03T06:50:23.153 DEBUG:tasks.rgw_multi.zone_rados:o=obj69 2022-09-03T06:50:23.153 DEBUG:tasks.rgw_multi.zone_rados:o=obj74 2022-09-03T06:50:23.154 DEBUG:tasks.rgw_multi.zone_rados:o=obj76 2022-09-03T06:50:23.154 DEBUG:tasks.rgw_multi.zone_rados:o=obj85 2022-09-03T06:50:23.155 DEBUG:tasks.rgw_multi.zone_rados:o=obj87 2022-09-03T06:50:23.155 DEBUG:tasks.rgw_multi.zone_rados:o=obj92 2022-09-03T06:50:23.156 DEBUG:tasks.rgw_multi.zone_rados:o=obj93 2022-09-03T06:50:23.156 DEBUG:tasks.rgw_multi.zone_rados:o=obj95 2022-09-03T06:50:23.157 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:23.157 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:23.158 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:23.159 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:23.159 DEBUG:boto:Method: GET 2022-09-03T06:50:23.160 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:23.160 DEBUG:boto:Data: 2022-09-03T06:50:23.161 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.161 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:23.162 DEBUG:boto:Port: 8001 2022-09-03T06:50:23.162 DEBUG:boto:Params: {} 2022-09-03T06:50:23.163 DEBUG:boto:Token: None 2022-09-03T06:50:23.163 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/?versions 2022-09-03T06:50:23.164 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3u6AyI6/ojs5lZ18oOwlwou0TMI= 2022-09-03T06:50:23.165 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3u6AyI6/ojs5lZ18oOwlwou0TMI=', 'Content-Length': '0'} 2022-09-03T06:50:23.168 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d8ee5b60adbdec2-006312f92f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.168 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj111nulltrue2022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj148nulltrue2022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj185nulltrue2022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj37nulltrue2022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj74nulltrue2022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:23.172 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:23.172 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:23.173 DEBUG:boto:path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:23.173 DEBUG:boto:auth_path=/dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:23.174 DEBUG:boto:Method: GET 2022-09-03T06:50:23.175 DEBUG:boto:Path: /dsgvcy-75/?versions&max-keys=999 2022-09-03T06:50:23.176 DEBUG:boto:Data: 2022-09-03T06:50:23.177 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.177 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:23.178 DEBUG:boto:Port: 8000 2022-09-03T06:50:23.178 DEBUG:boto:Params: {} 2022-09-03T06:50:23.179 DEBUG:boto:Token: None 2022-09-03T06:50:23.179 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/?versions 2022-09-03T06:50:23.180 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3u6AyI6/ojs5lZ18oOwlwou0TMI= 2022-09-03T06:50:23.181 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3u6AyI6/ojs5lZ18oOwlwou0TMI=', 'Content-Length': '0'} 2022-09-03T06:50:23.184 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000034085c7fd1c0f5ab-006312f92f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.185 DEBUG:boto:b'dsgvcy-75999falseobj11nulltrue2022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj111nulltrue2022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj115nulltrue2022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj116nulltrue2022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj124nulltrue2022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj145nulltrue2022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj148nulltrue2022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj155nulltrue2022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj17nulltrue2022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj185nulltrue2022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj19nulltrue2022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj22nulltrue2022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj23nulltrue2022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj29nulltrue2022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj31nulltrue2022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj33nulltrue2022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj34nulltrue2022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj37nulltrue2022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj38nulltrue2022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj44nulltrue2022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj46nulltrue2022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj51nulltrue2022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj55nulltrue2022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj57nulltrue2022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj58nulltrue2022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj62nulltrue2022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj68nulltrue2022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj69nulltrue2022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj74nulltrue2022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj76nulltrue2022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj85nulltrue2022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj87nulltrue2022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj92nulltrue2022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj93nulltrue2022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj95nulltrue2022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:23.188 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:50:23.188 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:23.189 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:23.189 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.190 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.190 DEBUG:boto:Method: GET 2022-09-03T06:50:23.191 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.191 DEBUG:boto:Data: 2022-09-03T06:50:23.192 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.193 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:23.194 DEBUG:boto:Port: 8001 2022-09-03T06:50:23.195 DEBUG:boto:Params: {} 2022-09-03T06:50:23.196 DEBUG:boto:Token: None 2022-09-03T06:50:23.197 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.199 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5S4GsQIdcBSXQ5HOoCI+OfWkyT8= 2022-09-03T06:50:23.200 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5S4GsQIdcBSXQ5HOoCI+OfWkyT8=', 'Content-Length': '0'} 2022-09-03T06:50:23.203 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002d5a152a34df4df4-006312f92f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.246 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:23.247 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:23.248 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.249 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.250 DEBUG:boto:Method: GET 2022-09-03T06:50:23.251 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.252 DEBUG:boto:Data: 2022-09-03T06:50:23.253 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.254 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:23.255 DEBUG:boto:Port: 8000 2022-09-03T06:50:23.256 DEBUG:boto:Params: {} 2022-09-03T06:50:23.257 DEBUG:boto:Token: None 2022-09-03T06:50:23.258 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.260 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5S4GsQIdcBSXQ5HOoCI+OfWkyT8= 2022-09-03T06:50:23.261 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5S4GsQIdcBSXQ5HOoCI+OfWkyT8=', 'Content-Length': '0'} 2022-09-03T06:50:23.264 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e11c5bc5952c50a1-006312f92f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.265 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:23.266 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:23.267 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.268 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.269 DEBUG:boto:Method: HEAD 2022-09-03T06:50:23.269 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.270 DEBUG:boto:Data: 2022-09-03T06:50:23.270 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.271 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:23.271 DEBUG:boto:Port: 8001 2022-09-03T06:50:23.272 DEBUG:boto:Params: {} 2022-09-03T06:50:23.272 DEBUG:boto:Token: None 2022-09-03T06:50:23.273 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.274 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LcXV5C4J91OQxFXd/elOtN5q2RY= 2022-09-03T06:50:23.275 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LcXV5C4J91OQxFXd/elOtN5q2RY=', 'Content-Length': '0'} 2022-09-03T06:50:23.277 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d969295b28933672-006312f92f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.278 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:23.279 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:23.280 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.280 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.281 DEBUG:boto:Method: HEAD 2022-09-03T06:50:23.281 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.282 DEBUG:boto:Data: 2022-09-03T06:50:23.282 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.283 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:23.283 DEBUG:boto:Port: 8000 2022-09-03T06:50:23.284 DEBUG:boto:Params: {} 2022-09-03T06:50:23.284 DEBUG:boto:Token: None 2022-09-03T06:50:23.285 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.286 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LcXV5C4J91OQxFXd/elOtN5q2RY= 2022-09-03T06:50:23.286 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LcXV5C4J91OQxFXd/elOtN5q2RY=', 'Content-Length': '0'} 2022-09-03T06:50:23.289 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bd0be8fc3dad5d97-006312f92f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.290 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:50:23.293 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:23.294 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:23.296 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.296 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.297 DEBUG:boto:Method: GET 2022-09-03T06:50:23.298 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.298 DEBUG:boto:Data: 2022-09-03T06:50:23.299 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.300 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:23.301 DEBUG:boto:Port: 8001 2022-09-03T06:50:23.302 DEBUG:boto:Params: {} 2022-09-03T06:50:23.303 DEBUG:boto:Token: None 2022-09-03T06:50:23.304 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.305 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5S4GsQIdcBSXQ5HOoCI+OfWkyT8= 2022-09-03T06:50:23.307 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5S4GsQIdcBSXQ5HOoCI+OfWkyT8=', 'Content-Length': '0'} 2022-09-03T06:50:23.310 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c5c6a04fcf7660d5-006312f92f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.311 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:23.312 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:23.313 DEBUG:boto:path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.314 DEBUG:boto:auth_path=/dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.315 DEBUG:boto:Method: GET 2022-09-03T06:50:23.317 DEBUG:boto:Path: /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.317 DEBUG:boto:Data: 2022-09-03T06:50:23.319 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.320 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:23.322 DEBUG:boto:Port: 8000 2022-09-03T06:50:23.323 DEBUG:boto:Params: {} 2022-09-03T06:50:23.324 DEBUG:boto:Token: None 2022-09-03T06:50:23.325 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj11?versionId=null 2022-09-03T06:50:23.326 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5S4GsQIdcBSXQ5HOoCI+OfWkyT8= 2022-09-03T06:50:23.327 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5S4GsQIdcBSXQ5HOoCI+OfWkyT8=', 'Content-Length': '0'} 2022-09-03T06:50:23.330 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000c22cf7bd23cbbf1-006312f92f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.332 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:23.333 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:23.333 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:23.334 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:23.335 DEBUG:boto:Method: GET 2022-09-03T06:50:23.336 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:50:23.337 DEBUG:boto:Data: 2022-09-03T06:50:23.339 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.339 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:23.340 DEBUG:boto:Port: 8001 2022-09-03T06:50:23.340 DEBUG:boto:Params: {} 2022-09-03T06:50:23.341 DEBUG:boto:Token: None 2022-09-03T06:50:23.342 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/ 2022-09-03T06:50:23.342 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw= 2022-09-03T06:50:23.343 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw=', 'Content-Length': '0'} 2022-09-03T06:50:23.346 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001c617904151aa790-006312f92f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.347 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:23.348 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:23.348 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:23.349 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:23.350 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:23.351 DEBUG:boto:Method: GET 2022-09-03T06:50:23.352 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:50:23.353 DEBUG:boto:Data: 2022-09-03T06:50:23.353 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.354 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:23.355 DEBUG:boto:Port: 8000 2022-09-03T06:50:23.355 DEBUG:boto:Params: {} 2022-09-03T06:50:23.356 DEBUG:boto:Token: None 2022-09-03T06:50:23.357 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/ 2022-09-03T06:50:23.357 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw= 2022-09-03T06:50:23.358 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw=', 'Content-Length': '0'} 2022-09-03T06:50:23.361 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008eda3b26f75c6c58-006312f92f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.362 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:23.363 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:23.363 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:23.364 DEBUG:boto:Method: HEAD 2022-09-03T06:50:23.364 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:50:23.365 DEBUG:boto:Data: 2022-09-03T06:50:23.365 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.366 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:23.366 DEBUG:boto:Port: 8001 2022-09-03T06:50:23.367 DEBUG:boto:Params: {} 2022-09-03T06:50:23.368 DEBUG:boto:Token: None 2022-09-03T06:50:23.368 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj11 2022-09-03T06:50:23.369 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c1DSaJSPwhhu8/OYhQAW5Rzr+hU= 2022-09-03T06:50:23.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c1DSaJSPwhhu8/OYhQAW5Rzr+hU=', 'Content-Length': '0'} 2022-09-03T06:50:23.372 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c320dc0d381d9b65-006312f92f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.373 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:23.374 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:23.375 DEBUG:boto:Method: HEAD 2022-09-03T06:50:23.376 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:50:23.377 DEBUG:boto:Data: 2022-09-03T06:50:23.377 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.378 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:23.379 DEBUG:boto:Port: 8000 2022-09-03T06:50:23.380 DEBUG:boto:Params: {} 2022-09-03T06:50:23.381 DEBUG:boto:Token: None 2022-09-03T06:50:23.381 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj11 2022-09-03T06:50:23.382 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c1DSaJSPwhhu8/OYhQAW5Rzr+hU= 2022-09-03T06:50:23.382 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c1DSaJSPwhhu8/OYhQAW5Rzr+hU=', 'Content-Length': '0'} 2022-09-03T06:50:23.385 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000955892ff7ed8319-006312f92f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.386 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj11 2022-09-03T06:50:23.387 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:23.387 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:23.388 DEBUG:boto:Method: GET 2022-09-03T06:50:23.389 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:50:23.389 DEBUG:boto:Data: 2022-09-03T06:50:23.390 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.390 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:23.391 DEBUG:boto:Port: 8001 2022-09-03T06:50:23.391 DEBUG:boto:Params: {} 2022-09-03T06:50:23.392 DEBUG:boto:Token: None 2022-09-03T06:50:23.393 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj11 2022-09-03T06:50:23.394 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FEcbenaGYvA2ci4kbBSgYi62cxE= 2022-09-03T06:50:23.395 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FEcbenaGYvA2ci4kbBSgYi62cxE=', 'Content-Length': '0'} 2022-09-03T06:50:23.397 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000711a985ecdaa64b8-006312f92f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.398 DEBUG:boto:path=/dsgvcy-75/obj11 2022-09-03T06:50:23.399 DEBUG:boto:auth_path=/dsgvcy-75/obj11 2022-09-03T06:50:23.400 DEBUG:boto:Method: GET 2022-09-03T06:50:23.401 DEBUG:boto:Path: /dsgvcy-75/obj11 2022-09-03T06:50:23.402 DEBUG:boto:Data: 2022-09-03T06:50:23.403 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.404 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:23.405 DEBUG:boto:Port: 8000 2022-09-03T06:50:23.406 DEBUG:boto:Params: {} 2022-09-03T06:50:23.407 DEBUG:boto:Token: None 2022-09-03T06:50:23.408 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj11 2022-09-03T06:50:23.408 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FEcbenaGYvA2ci4kbBSgYi62cxE= 2022-09-03T06:50:23.410 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FEcbenaGYvA2ci4kbBSgYi62cxE=', 'Content-Length': '0'} 2022-09-03T06:50:23.413 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c84dc91bb46d71b-006312f92f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.459 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:23.460 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:23.461 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:23.462 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:23.463 DEBUG:boto:Method: GET 2022-09-03T06:50:23.464 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:50:23.465 DEBUG:boto:Data: 2022-09-03T06:50:23.466 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.467 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:23.468 DEBUG:boto:Port: 8001 2022-09-03T06:50:23.469 DEBUG:boto:Params: {} 2022-09-03T06:50:23.470 DEBUG:boto:Token: None 2022-09-03T06:50:23.471 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/ 2022-09-03T06:50:23.472 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw= 2022-09-03T06:50:23.473 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw=', 'Content-Length': '0'} 2022-09-03T06:50:23.476 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006edeb571033affa1-006312f92f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.478 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:23.479 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:23.479 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:23.480 DEBUG:boto:path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:23.481 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj11 2022-09-03T06:50:23.481 DEBUG:boto:Method: GET 2022-09-03T06:50:23.482 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj11 2022-09-03T06:50:23.482 DEBUG:boto:Data: 2022-09-03T06:50:23.483 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.483 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:23.484 DEBUG:boto:Port: 8000 2022-09-03T06:50:23.486 DEBUG:boto:Params: {} 2022-09-03T06:50:23.487 DEBUG:boto:Token: None 2022-09-03T06:50:23.487 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/ 2022-09-03T06:50:23.488 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw= 2022-09-03T06:50:23.489 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw=', 'Content-Length': '0'} 2022-09-03T06:50:23.491 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000037e37e81918264b-006312f92f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.492 DEBUG:boto:b'dsgvcy-75obj111000falseobj112022-09-03T06:36:05.688Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormalobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:23.493 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj111 2022-09-03T06:50:23.493 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:23.494 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:23.495 DEBUG:boto:path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.496 DEBUG:boto:auth_path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.497 DEBUG:boto:Method: GET 2022-09-03T06:50:23.498 DEBUG:boto:Path: /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.498 DEBUG:boto:Data: 2022-09-03T06:50:23.499 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.499 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:23.500 DEBUG:boto:Port: 8001 2022-09-03T06:50:23.500 DEBUG:boto:Params: {} 2022-09-03T06:50:23.501 DEBUG:boto:Token: None 2022-09-03T06:50:23.502 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.503 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yxDDmsIWbRzrwVlhoNM48p4HAFk= 2022-09-03T06:50:23.503 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yxDDmsIWbRzrwVlhoNM48p4HAFk=', 'Content-Length': '0'} 2022-09-03T06:50:23.506 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004951406ac5747015-006312f92f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.508 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:23.508 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:23.509 DEBUG:boto:path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.510 DEBUG:boto:auth_path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.511 DEBUG:boto:Method: GET 2022-09-03T06:50:23.512 DEBUG:boto:Path: /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.513 DEBUG:boto:Data: 2022-09-03T06:50:23.514 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.515 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:23.516 DEBUG:boto:Port: 8000 2022-09-03T06:50:23.517 DEBUG:boto:Params: {} 2022-09-03T06:50:23.519 DEBUG:boto:Token: None 2022-09-03T06:50:23.520 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.520 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yxDDmsIWbRzrwVlhoNM48p4HAFk= 2022-09-03T06:50:23.521 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yxDDmsIWbRzrwVlhoNM48p4HAFk=', 'Content-Length': '0'} 2022-09-03T06:50:23.524 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000011603c4a44abe082-006312f92f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.525 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:23.525 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:23.526 DEBUG:boto:path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.526 DEBUG:boto:auth_path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.527 DEBUG:boto:Method: HEAD 2022-09-03T06:50:23.527 DEBUG:boto:Path: /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.528 DEBUG:boto:Data: 2022-09-03T06:50:23.528 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.529 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:23.529 DEBUG:boto:Port: 8001 2022-09-03T06:50:23.530 DEBUG:boto:Params: {} 2022-09-03T06:50:23.530 DEBUG:boto:Token: None 2022-09-03T06:50:23.531 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.531 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MgSER/tVpV/hksxBZnVB3AaG4Vw= 2022-09-03T06:50:23.532 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MgSER/tVpV/hksxBZnVB3AaG4Vw=', 'Content-Length': '0'} 2022-09-03T06:50:23.534 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000172385c55db914f9-006312f92f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.535 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:23.535 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:23.536 DEBUG:boto:path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.537 DEBUG:boto:auth_path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.537 DEBUG:boto:Method: HEAD 2022-09-03T06:50:23.538 DEBUG:boto:Path: /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.539 DEBUG:boto:Data: 2022-09-03T06:50:23.539 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.540 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:23.541 DEBUG:boto:Port: 8000 2022-09-03T06:50:23.541 DEBUG:boto:Params: {} 2022-09-03T06:50:23.542 DEBUG:boto:Token: None 2022-09-03T06:50:23.542 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.543 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:MgSER/tVpV/hksxBZnVB3AaG4Vw= 2022-09-03T06:50:23.543 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:MgSER/tVpV/hksxBZnVB3AaG4Vw=', 'Content-Length': '0'} 2022-09-03T06:50:23.546 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a38473bc0b95c6c-006312f92f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.547 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj111 2022-09-03T06:50:23.547 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:23.548 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:23.549 DEBUG:boto:path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.549 DEBUG:boto:auth_path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.550 DEBUG:boto:Method: GET 2022-09-03T06:50:23.550 DEBUG:boto:Path: /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.551 DEBUG:boto:Data: 2022-09-03T06:50:23.551 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.552 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:23.552 DEBUG:boto:Port: 8001 2022-09-03T06:50:23.553 DEBUG:boto:Params: {} 2022-09-03T06:50:23.553 DEBUG:boto:Token: None 2022-09-03T06:50:23.554 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.555 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yxDDmsIWbRzrwVlhoNM48p4HAFk= 2022-09-03T06:50:23.555 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yxDDmsIWbRzrwVlhoNM48p4HAFk=', 'Content-Length': '0'} 2022-09-03T06:50:23.558 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002dd1a429859b62ca-006312f92f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.606 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:23.607 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:23.608 DEBUG:boto:path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.609 DEBUG:boto:auth_path=/dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.609 DEBUG:boto:Method: GET 2022-09-03T06:50:23.610 DEBUG:boto:Path: /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.611 DEBUG:boto:Data: 2022-09-03T06:50:23.611 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.612 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:23.613 DEBUG:boto:Port: 8000 2022-09-03T06:50:23.613 DEBUG:boto:Params: {} 2022-09-03T06:50:23.614 DEBUG:boto:Token: None 2022-09-03T06:50:23.615 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj111?versionId=null 2022-09-03T06:50:23.616 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yxDDmsIWbRzrwVlhoNM48p4HAFk= 2022-09-03T06:50:23.617 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yxDDmsIWbRzrwVlhoNM48p4HAFk=', 'Content-Length': '0'} 2022-09-03T06:50:23.619 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d1aa8b2be0872645-006312f92f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.620 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:23.621 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:23.622 DEBUG:boto:path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:23.622 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:23.623 DEBUG:boto:Method: GET 2022-09-03T06:50:23.623 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj111 2022-09-03T06:50:23.624 DEBUG:boto:Data: 2022-09-03T06:50:23.624 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.625 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:23.626 DEBUG:boto:Port: 8001 2022-09-03T06:50:23.626 DEBUG:boto:Params: {} 2022-09-03T06:50:23.627 DEBUG:boto:Token: None 2022-09-03T06:50:23.627 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/ 2022-09-03T06:50:23.628 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw= 2022-09-03T06:50:23.629 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw=', 'Content-Length': '0'} 2022-09-03T06:50:23.631 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000945a5d719ff27b05-006312f92f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.632 DEBUG:boto:b'dsgvcy-75obj1111000falseobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:23.633 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:23.634 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:23.634 DEBUG:boto:path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:23.635 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:23.635 DEBUG:boto:Method: GET 2022-09-03T06:50:23.636 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj111 2022-09-03T06:50:23.636 DEBUG:boto:Data: 2022-09-03T06:50:23.637 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.637 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:23.638 DEBUG:boto:Port: 8000 2022-09-03T06:50:23.638 DEBUG:boto:Params: {} 2022-09-03T06:50:23.639 DEBUG:boto:Token: None 2022-09-03T06:50:23.640 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/ 2022-09-03T06:50:23.641 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw= 2022-09-03T06:50:23.641 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw=', 'Content-Length': '0'} 2022-09-03T06:50:23.644 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d7c663c8e46c20eb-006312f92f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.645 DEBUG:boto:b'dsgvcy-75obj1111000falseobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:23.646 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:23.647 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:23.648 DEBUG:boto:Method: HEAD 2022-09-03T06:50:23.648 DEBUG:boto:Path: /dsgvcy-75/obj111 2022-09-03T06:50:23.649 DEBUG:boto:Data: 2022-09-03T06:50:23.650 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.650 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:23.651 DEBUG:boto:Port: 8001 2022-09-03T06:50:23.652 DEBUG:boto:Params: {} 2022-09-03T06:50:23.653 DEBUG:boto:Token: None 2022-09-03T06:50:23.654 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj111 2022-09-03T06:50:23.655 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nrzOF9e2GIMCMC3c2i2fAAfEj08= 2022-09-03T06:50:23.656 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nrzOF9e2GIMCMC3c2i2fAAfEj08=', 'Content-Length': '0'} 2022-09-03T06:50:23.659 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006f57c3c0988bd198-006312f92f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.660 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:23.661 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:23.662 DEBUG:boto:Method: HEAD 2022-09-03T06:50:23.664 DEBUG:boto:Path: /dsgvcy-75/obj111 2022-09-03T06:50:23.665 DEBUG:boto:Data: 2022-09-03T06:50:23.666 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.667 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:23.668 DEBUG:boto:Port: 8000 2022-09-03T06:50:23.669 DEBUG:boto:Params: {} 2022-09-03T06:50:23.670 DEBUG:boto:Token: None 2022-09-03T06:50:23.671 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj111 2022-09-03T06:50:23.672 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nrzOF9e2GIMCMC3c2i2fAAfEj08= 2022-09-03T06:50:23.673 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nrzOF9e2GIMCMC3c2i2fAAfEj08=', 'Content-Length': '0'} 2022-09-03T06:50:23.676 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c894d0382196c6e-006312f92f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.677 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj111 2022-09-03T06:50:23.679 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:23.680 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:23.680 DEBUG:boto:Method: GET 2022-09-03T06:50:23.681 DEBUG:boto:Path: /dsgvcy-75/obj111 2022-09-03T06:50:23.681 DEBUG:boto:Data: 2022-09-03T06:50:23.682 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.683 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:23.683 DEBUG:boto:Port: 8001 2022-09-03T06:50:23.684 DEBUG:boto:Params: {} 2022-09-03T06:50:23.684 DEBUG:boto:Token: None 2022-09-03T06:50:23.685 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj111 2022-09-03T06:50:23.686 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sZXMGjY1o/boTscd50TieiB/O9c= 2022-09-03T06:50:23.686 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sZXMGjY1o/boTscd50TieiB/O9c=', 'Content-Length': '0'} 2022-09-03T06:50:23.688 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c3d33c3732e2e640-006312f92f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.734 DEBUG:boto:path=/dsgvcy-75/obj111 2022-09-03T06:50:23.736 DEBUG:boto:auth_path=/dsgvcy-75/obj111 2022-09-03T06:50:23.737 DEBUG:boto:Method: GET 2022-09-03T06:50:23.739 DEBUG:boto:Path: /dsgvcy-75/obj111 2022-09-03T06:50:23.741 DEBUG:boto:Data: 2022-09-03T06:50:23.742 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.743 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:23.744 DEBUG:boto:Port: 8000 2022-09-03T06:50:23.745 DEBUG:boto:Params: {} 2022-09-03T06:50:23.746 DEBUG:boto:Token: None 2022-09-03T06:50:23.747 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj111 2022-09-03T06:50:23.747 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:sZXMGjY1o/boTscd50TieiB/O9c= 2022-09-03T06:50:23.748 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:sZXMGjY1o/boTscd50TieiB/O9c=', 'Content-Length': '0'} 2022-09-03T06:50:23.750 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000177eb16f9f6bb27a-006312f92f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.751 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:23.752 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:23.752 DEBUG:boto:path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:23.753 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:23.753 DEBUG:boto:Method: GET 2022-09-03T06:50:23.754 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj111 2022-09-03T06:50:23.755 DEBUG:boto:Data: 2022-09-03T06:50:23.756 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.756 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:23.757 DEBUG:boto:Port: 8001 2022-09-03T06:50:23.758 DEBUG:boto:Params: {} 2022-09-03T06:50:23.759 DEBUG:boto:Token: None 2022-09-03T06:50:23.759 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/ 2022-09-03T06:50:23.760 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw= 2022-09-03T06:50:23.761 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw=', 'Content-Length': '0'} 2022-09-03T06:50:23.763 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003b98905ecbfb18e3-006312f92f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.764 DEBUG:boto:b'dsgvcy-75obj1111000falseobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:23.765 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:23.766 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:23.767 DEBUG:boto:path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:23.767 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj111 2022-09-03T06:50:23.768 DEBUG:boto:Method: GET 2022-09-03T06:50:23.769 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj111 2022-09-03T06:50:23.769 DEBUG:boto:Data: 2022-09-03T06:50:23.770 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.771 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:23.771 DEBUG:boto:Port: 8000 2022-09-03T06:50:23.772 DEBUG:boto:Params: {} 2022-09-03T06:50:23.772 DEBUG:boto:Token: None 2022-09-03T06:50:23.773 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/ 2022-09-03T06:50:23.774 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw= 2022-09-03T06:50:23.774 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw=', 'Content-Length': '0'} 2022-09-03T06:50:23.777 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a0d4e9f4f1467055-006312f92f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.831 DEBUG:boto:b'dsgvcy-75obj1111000falseobj1112022-09-03T06:47:49.634Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:23.833 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:50:23.833 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:23.834 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:23.835 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.835 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.836 DEBUG:boto:Method: GET 2022-09-03T06:50:23.837 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.837 DEBUG:boto:Data: 2022-09-03T06:50:23.838 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.839 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:23.840 DEBUG:boto:Port: 8001 2022-09-03T06:50:23.840 DEBUG:boto:Params: {} 2022-09-03T06:50:23.842 DEBUG:boto:Token: None 2022-09-03T06:50:23.842 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.843 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2wCawvTcJELQyLhYhj8+HPip5AE= 2022-09-03T06:50:23.844 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2wCawvTcJELQyLhYhj8+HPip5AE=', 'Content-Length': '0'} 2022-09-03T06:50:23.846 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f71cb8a1a0cbd2ca-006312f92f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.847 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:23.847 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:23.848 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.849 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.849 DEBUG:boto:Method: GET 2022-09-03T06:50:23.850 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.850 DEBUG:boto:Data: 2022-09-03T06:50:23.851 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.852 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:23.852 DEBUG:boto:Port: 8000 2022-09-03T06:50:23.853 DEBUG:boto:Params: {} 2022-09-03T06:50:23.854 DEBUG:boto:Token: None 2022-09-03T06:50:23.854 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.855 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2wCawvTcJELQyLhYhj8+HPip5AE= 2022-09-03T06:50:23.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2wCawvTcJELQyLhYhj8+HPip5AE=', 'Content-Length': '0'} 2022-09-03T06:50:23.858 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d6072fc58badf75-006312f92f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.859 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:23.860 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:23.861 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.861 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.862 DEBUG:boto:Method: HEAD 2022-09-03T06:50:23.862 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.863 DEBUG:boto:Data: 2022-09-03T06:50:23.863 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.864 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:23.864 DEBUG:boto:Port: 8001 2022-09-03T06:50:23.865 DEBUG:boto:Params: {} 2022-09-03T06:50:23.866 DEBUG:boto:Token: None 2022-09-03T06:50:23.866 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.867 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qxEXDrpgzU1mjNq10p0MsF2QbPo= 2022-09-03T06:50:23.867 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qxEXDrpgzU1mjNq10p0MsF2QbPo=', 'Content-Length': '0'} 2022-09-03T06:50:23.869 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d7b692f2c82234f-006312f92f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.870 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:23.871 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:23.871 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.872 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.874 DEBUG:boto:Method: HEAD 2022-09-03T06:50:23.874 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.875 DEBUG:boto:Data: 2022-09-03T06:50:23.875 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.876 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:23.877 DEBUG:boto:Port: 8000 2022-09-03T06:50:23.877 DEBUG:boto:Params: {} 2022-09-03T06:50:23.878 DEBUG:boto:Token: None 2022-09-03T06:50:23.878 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.879 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qxEXDrpgzU1mjNq10p0MsF2QbPo= 2022-09-03T06:50:23.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qxEXDrpgzU1mjNq10p0MsF2QbPo=', 'Content-Length': '0'} 2022-09-03T06:50:23.883 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f9849908faa7a9ad-006312f92f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.884 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:50:23.885 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:23.885 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:23.886 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.886 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.887 DEBUG:boto:Method: GET 2022-09-03T06:50:23.888 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.889 DEBUG:boto:Data: 2022-09-03T06:50:23.890 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.891 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:23.892 DEBUG:boto:Port: 8001 2022-09-03T06:50:23.893 DEBUG:boto:Params: {} 2022-09-03T06:50:23.894 DEBUG:boto:Token: None 2022-09-03T06:50:23.895 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.897 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2wCawvTcJELQyLhYhj8+HPip5AE= 2022-09-03T06:50:23.898 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2wCawvTcJELQyLhYhj8+HPip5AE=', 'Content-Length': '0'} 2022-09-03T06:50:23.900 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006878c71000c86e07-006312f92f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.902 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:23.903 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:23.903 DEBUG:boto:path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.904 DEBUG:boto:auth_path=/dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.904 DEBUG:boto:Method: GET 2022-09-03T06:50:23.905 DEBUG:boto:Path: /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.906 DEBUG:boto:Data: 2022-09-03T06:50:23.906 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.907 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:23.907 DEBUG:boto:Port: 8000 2022-09-03T06:50:23.908 DEBUG:boto:Params: {} 2022-09-03T06:50:23.909 DEBUG:boto:Token: None 2022-09-03T06:50:23.909 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj115?versionId=null 2022-09-03T06:50:23.910 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2wCawvTcJELQyLhYhj8+HPip5AE= 2022-09-03T06:50:23.911 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2wCawvTcJELQyLhYhj8+HPip5AE=', 'Content-Length': '0'} 2022-09-03T06:50:23.913 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004fb5ff06ac1d63bc-006312f92f-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.958 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:23.959 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:23.959 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:23.960 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:23.960 DEBUG:boto:Method: GET 2022-09-03T06:50:23.961 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:50:23.962 DEBUG:boto:Data: 2022-09-03T06:50:23.962 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.963 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:23.963 DEBUG:boto:Port: 8001 2022-09-03T06:50:23.964 DEBUG:boto:Params: {} 2022-09-03T06:50:23.965 DEBUG:boto:Token: None 2022-09-03T06:50:23.965 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/ 2022-09-03T06:50:23.966 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw= 2022-09-03T06:50:23.967 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw=', 'Content-Length': '0'} 2022-09-03T06:50:23.970 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e8d22a6e587a4ebe-006312f92f-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.971 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:23.971 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:23.972 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:23.973 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:23.973 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:23.974 DEBUG:boto:Method: GET 2022-09-03T06:50:23.975 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:50:23.975 DEBUG:boto:Data: 2022-09-03T06:50:23.976 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.976 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:23.977 DEBUG:boto:Port: 8000 2022-09-03T06:50:23.977 DEBUG:boto:Params: {} 2022-09-03T06:50:23.978 DEBUG:boto:Token: None 2022-09-03T06:50:23.978 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/ 2022-09-03T06:50:23.979 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw= 2022-09-03T06:50:23.980 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LkbW6T1+FJxjYdYvcrsw5oqFzyw=', 'Content-Length': '0'} 2022-09-03T06:50:23.983 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a01ac23f7925149f-006312f92f-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.983 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:23.984 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:23.985 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:23.985 DEBUG:boto:Method: HEAD 2022-09-03T06:50:23.986 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:50:23.986 DEBUG:boto:Data: 2022-09-03T06:50:23.987 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.987 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:23.988 DEBUG:boto:Port: 8001 2022-09-03T06:50:23.989 DEBUG:boto:Params: {} 2022-09-03T06:50:23.989 DEBUG:boto:Token: None 2022-09-03T06:50:23.990 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:23 GMT /dsgvcy-75/obj115 2022-09-03T06:50:23.991 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4KwWrhRb8NGc/NZ+tfzzx7lPdAQ= 2022-09-03T06:50:23.991 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:23 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4KwWrhRb8NGc/NZ+tfzzx7lPdAQ=', 'Content-Length': '0'} 2022-09-03T06:50:23.993 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f0aa109c17ac6a16-006312f92f-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:23 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:23.994 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:23.995 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:23.996 DEBUG:boto:Method: HEAD 2022-09-03T06:50:23.996 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:50:23.997 DEBUG:boto:Data: 2022-09-03T06:50:23.997 DEBUG:boto:Headers: {} 2022-09-03T06:50:23.998 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:23.998 DEBUG:boto:Port: 8000 2022-09-03T06:50:23.999 DEBUG:boto:Params: {} 2022-09-03T06:50:24.000 DEBUG:boto:Token: None 2022-09-03T06:50:24.001 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj115 2022-09-03T06:50:24.002 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ntTiQ5IpDkJwmZLmc/eECaw/hVw= 2022-09-03T06:50:24.003 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ntTiQ5IpDkJwmZLmc/eECaw/hVw=', 'Content-Length': '0'} 2022-09-03T06:50:24.005 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d56a5ab91d2e4fdf-006312f930-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.006 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj115 2022-09-03T06:50:24.007 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:24.007 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:24.008 DEBUG:boto:Method: GET 2022-09-03T06:50:24.009 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:50:24.010 DEBUG:boto:Data: 2022-09-03T06:50:24.010 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.011 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:24.012 DEBUG:boto:Port: 8001 2022-09-03T06:50:24.012 DEBUG:boto:Params: {} 2022-09-03T06:50:24.013 DEBUG:boto:Token: None 2022-09-03T06:50:24.015 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj115 2022-09-03T06:50:24.016 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZfoYyLcNgXDNLsg1b9gNSYMvixc= 2022-09-03T06:50:24.016 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZfoYyLcNgXDNLsg1b9gNSYMvixc=', 'Content-Length': '0'} 2022-09-03T06:50:24.019 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d05883eb051ac920-006312f930-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.019 DEBUG:boto:path=/dsgvcy-75/obj115 2022-09-03T06:50:24.020 DEBUG:boto:auth_path=/dsgvcy-75/obj115 2022-09-03T06:50:24.021 DEBUG:boto:Method: GET 2022-09-03T06:50:24.021 DEBUG:boto:Path: /dsgvcy-75/obj115 2022-09-03T06:50:24.022 DEBUG:boto:Data: 2022-09-03T06:50:24.023 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.023 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:24.024 DEBUG:boto:Port: 8000 2022-09-03T06:50:24.024 DEBUG:boto:Params: {} 2022-09-03T06:50:24.025 DEBUG:boto:Token: None 2022-09-03T06:50:24.026 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj115 2022-09-03T06:50:24.026 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZfoYyLcNgXDNLsg1b9gNSYMvixc= 2022-09-03T06:50:24.027 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZfoYyLcNgXDNLsg1b9gNSYMvixc=', 'Content-Length': '0'} 2022-09-03T06:50:24.029 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000027c439a36837ffd9-006312f930-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.031 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:24.032 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:24.033 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:24.034 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:24.035 DEBUG:boto:Method: GET 2022-09-03T06:50:24.036 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:50:24.037 DEBUG:boto:Data: 2022-09-03T06:50:24.038 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.039 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:24.040 DEBUG:boto:Port: 8001 2022-09-03T06:50:24.040 DEBUG:boto:Params: {} 2022-09-03T06:50:24.041 DEBUG:boto:Token: None 2022-09-03T06:50:24.042 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/ 2022-09-03T06:50:24.043 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo= 2022-09-03T06:50:24.045 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo=', 'Content-Length': '0'} 2022-09-03T06:50:24.048 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bfbca3a97449dd5b-006312f930-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.094 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:24.095 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:24.096 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:24.096 DEBUG:boto:path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:24.097 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj115 2022-09-03T06:50:24.097 DEBUG:boto:Method: GET 2022-09-03T06:50:24.098 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj115 2022-09-03T06:50:24.099 DEBUG:boto:Data: 2022-09-03T06:50:24.100 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.100 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:24.102 DEBUG:boto:Port: 8000 2022-09-03T06:50:24.103 DEBUG:boto:Params: {} 2022-09-03T06:50:24.104 DEBUG:boto:Token: None 2022-09-03T06:50:24.107 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/ 2022-09-03T06:50:24.108 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo= 2022-09-03T06:50:24.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo=', 'Content-Length': '0'} 2022-09-03T06:50:24.114 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004f15937e66b7b5c8-006312f930-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.116 DEBUG:boto:b'dsgvcy-75obj1151000falseobj1152022-09-03T06:41:59.934Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:24.118 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:50:24.119 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:24.120 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:24.121 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.122 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.122 DEBUG:boto:Method: GET 2022-09-03T06:50:24.123 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.123 DEBUG:boto:Data: 2022-09-03T06:50:24.124 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.124 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:24.125 DEBUG:boto:Port: 8001 2022-09-03T06:50:24.125 DEBUG:boto:Params: {} 2022-09-03T06:50:24.126 DEBUG:boto:Token: None 2022-09-03T06:50:24.127 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.127 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DXqCNfkgicFE/oip5Bd4qBuRllM= 2022-09-03T06:50:24.129 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DXqCNfkgicFE/oip5Bd4qBuRllM=', 'Content-Length': '0'} 2022-09-03T06:50:24.131 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000072dc4dd4ef843ca1-006312f930-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.132 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:24.133 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:24.134 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.134 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.135 DEBUG:boto:Method: GET 2022-09-03T06:50:24.136 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.137 DEBUG:boto:Data: 2022-09-03T06:50:24.138 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.139 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:24.140 DEBUG:boto:Port: 8000 2022-09-03T06:50:24.141 DEBUG:boto:Params: {} 2022-09-03T06:50:24.142 DEBUG:boto:Token: None 2022-09-03T06:50:24.143 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.144 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DXqCNfkgicFE/oip5Bd4qBuRllM= 2022-09-03T06:50:24.145 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DXqCNfkgicFE/oip5Bd4qBuRllM=', 'Content-Length': '0'} 2022-09-03T06:50:24.148 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009da517510ca9df4e-006312f930-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.202 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:24.203 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:24.203 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.204 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.204 DEBUG:boto:Method: HEAD 2022-09-03T06:50:24.205 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.205 DEBUG:boto:Data: 2022-09-03T06:50:24.206 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.207 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:24.207 DEBUG:boto:Port: 8001 2022-09-03T06:50:24.208 DEBUG:boto:Params: {} 2022-09-03T06:50:24.208 DEBUG:boto:Token: None 2022-09-03T06:50:24.209 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.211 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:21rPAgdWUC9UBfP1aG/n/Pmj1eo= 2022-09-03T06:50:24.211 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:21rPAgdWUC9UBfP1aG/n/Pmj1eo=', 'Content-Length': '0'} 2022-09-03T06:50:24.213 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e979fb4ee8f30f1-006312f930-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.214 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:24.215 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:24.215 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.216 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.216 DEBUG:boto:Method: HEAD 2022-09-03T06:50:24.217 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.217 DEBUG:boto:Data: 2022-09-03T06:50:24.218 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.218 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:24.219 DEBUG:boto:Port: 8000 2022-09-03T06:50:24.220 DEBUG:boto:Params: {} 2022-09-03T06:50:24.221 DEBUG:boto:Token: None 2022-09-03T06:50:24.222 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.222 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:21rPAgdWUC9UBfP1aG/n/Pmj1eo= 2022-09-03T06:50:24.223 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:21rPAgdWUC9UBfP1aG/n/Pmj1eo=', 'Content-Length': '0'} 2022-09-03T06:50:24.225 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005adfeacdd9e2ad5b-006312f930-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.226 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:50:24.226 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:24.227 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:24.227 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.228 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.228 DEBUG:boto:Method: GET 2022-09-03T06:50:24.229 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.230 DEBUG:boto:Data: 2022-09-03T06:50:24.230 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.231 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:24.232 DEBUG:boto:Port: 8001 2022-09-03T06:50:24.232 DEBUG:boto:Params: {} 2022-09-03T06:50:24.233 DEBUG:boto:Token: None 2022-09-03T06:50:24.233 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.235 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DXqCNfkgicFE/oip5Bd4qBuRllM= 2022-09-03T06:50:24.236 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DXqCNfkgicFE/oip5Bd4qBuRllM=', 'Content-Length': '0'} 2022-09-03T06:50:24.240 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db1dacae85f4a72f-006312f930-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.241 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:24.241 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:24.242 DEBUG:boto:path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.243 DEBUG:boto:auth_path=/dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.244 DEBUG:boto:Method: GET 2022-09-03T06:50:24.244 DEBUG:boto:Path: /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.245 DEBUG:boto:Data: 2022-09-03T06:50:24.246 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.246 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:24.247 DEBUG:boto:Port: 8000 2022-09-03T06:50:24.247 DEBUG:boto:Params: {} 2022-09-03T06:50:24.248 DEBUG:boto:Token: None 2022-09-03T06:50:24.249 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj116?versionId=null 2022-09-03T06:50:24.249 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DXqCNfkgicFE/oip5Bd4qBuRllM= 2022-09-03T06:50:24.250 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DXqCNfkgicFE/oip5Bd4qBuRllM=', 'Content-Length': '0'} 2022-09-03T06:50:24.253 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cae65d14883bc609-006312f930-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.256 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:24.257 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:24.257 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:24.262 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:24.263 DEBUG:boto:Method: GET 2022-09-03T06:50:24.264 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:50:24.264 DEBUG:boto:Data: 2022-09-03T06:50:24.265 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.265 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:24.266 DEBUG:boto:Port: 8001 2022-09-03T06:50:24.267 DEBUG:boto:Params: {} 2022-09-03T06:50:24.267 DEBUG:boto:Token: None 2022-09-03T06:50:24.268 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/ 2022-09-03T06:50:24.269 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo= 2022-09-03T06:50:24.270 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo=', 'Content-Length': '0'} 2022-09-03T06:50:24.275 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000018bf7c642401bd96-006312f930-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.276 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:24.278 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:24.279 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:24.279 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:24.280 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:24.281 DEBUG:boto:Method: GET 2022-09-03T06:50:24.281 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:50:24.282 DEBUG:boto:Data: 2022-09-03T06:50:24.282 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.283 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:24.283 DEBUG:boto:Port: 8000 2022-09-03T06:50:24.284 DEBUG:boto:Params: {} 2022-09-03T06:50:24.285 DEBUG:boto:Token: None 2022-09-03T06:50:24.286 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/ 2022-09-03T06:50:24.287 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo= 2022-09-03T06:50:24.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo=', 'Content-Length': '0'} 2022-09-03T06:50:24.291 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dfa5e012079fb2be-006312f930-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.291 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:24.292 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:24.293 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:24.293 DEBUG:boto:Method: HEAD 2022-09-03T06:50:24.294 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:50:24.295 DEBUG:boto:Data: 2022-09-03T06:50:24.296 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.296 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:24.297 DEBUG:boto:Port: 8001 2022-09-03T06:50:24.297 DEBUG:boto:Params: {} 2022-09-03T06:50:24.298 DEBUG:boto:Token: None 2022-09-03T06:50:24.298 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj116 2022-09-03T06:50:24.299 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r096aLAk2YYlJ2eVB3MuL780sQ8= 2022-09-03T06:50:24.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r096aLAk2YYlJ2eVB3MuL780sQ8=', 'Content-Length': '0'} 2022-09-03T06:50:24.308 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000061c51f1a21e6f80b-006312f930-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.309 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:24.309 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:24.310 DEBUG:boto:Method: HEAD 2022-09-03T06:50:24.311 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:50:24.311 DEBUG:boto:Data: 2022-09-03T06:50:24.312 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.312 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:24.313 DEBUG:boto:Port: 8000 2022-09-03T06:50:24.313 DEBUG:boto:Params: {} 2022-09-03T06:50:24.315 DEBUG:boto:Token: None 2022-09-03T06:50:24.315 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj116 2022-09-03T06:50:24.319 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r096aLAk2YYlJ2eVB3MuL780sQ8= 2022-09-03T06:50:24.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r096aLAk2YYlJ2eVB3MuL780sQ8=', 'Content-Length': '0'} 2022-09-03T06:50:24.324 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005b1d82dd70809095-006312f930-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.326 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj116 2022-09-03T06:50:24.327 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:24.327 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:24.329 DEBUG:boto:Method: GET 2022-09-03T06:50:24.330 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:50:24.331 DEBUG:boto:Data: 2022-09-03T06:50:24.332 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.334 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:24.335 DEBUG:boto:Port: 8001 2022-09-03T06:50:24.336 DEBUG:boto:Params: {} 2022-09-03T06:50:24.337 DEBUG:boto:Token: None 2022-09-03T06:50:24.338 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj116 2022-09-03T06:50:24.341 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ie2A+b+YzgQAcY9Rv0qs3m5SuQk= 2022-09-03T06:50:24.342 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ie2A+b+YzgQAcY9Rv0qs3m5SuQk=', 'Content-Length': '0'} 2022-09-03T06:50:24.345 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f14905c91d1550bb-006312f930-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.387 DEBUG:boto:path=/dsgvcy-75/obj116 2022-09-03T06:50:24.391 DEBUG:boto:auth_path=/dsgvcy-75/obj116 2022-09-03T06:50:24.392 DEBUG:boto:Method: GET 2022-09-03T06:50:24.393 DEBUG:boto:Path: /dsgvcy-75/obj116 2022-09-03T06:50:24.394 DEBUG:boto:Data: 2022-09-03T06:50:24.395 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.396 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:24.397 DEBUG:boto:Port: 8000 2022-09-03T06:50:24.398 DEBUG:boto:Params: {} 2022-09-03T06:50:24.399 DEBUG:boto:Token: None 2022-09-03T06:50:24.401 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj116 2022-09-03T06:50:24.402 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ie2A+b+YzgQAcY9Rv0qs3m5SuQk= 2022-09-03T06:50:24.403 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ie2A+b+YzgQAcY9Rv0qs3m5SuQk=', 'Content-Length': '0'} 2022-09-03T06:50:24.406 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000452b20290fed9465-006312f930-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.407 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:24.408 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:24.409 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:24.409 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:24.410 DEBUG:boto:Method: GET 2022-09-03T06:50:24.411 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:50:24.412 DEBUG:boto:Data: 2022-09-03T06:50:24.412 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.413 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:24.413 DEBUG:boto:Port: 8001 2022-09-03T06:50:24.414 DEBUG:boto:Params: {} 2022-09-03T06:50:24.415 DEBUG:boto:Token: None 2022-09-03T06:50:24.416 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/ 2022-09-03T06:50:24.417 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo= 2022-09-03T06:50:24.418 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo=', 'Content-Length': '0'} 2022-09-03T06:50:24.421 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008eb2e1e4c94fd91c-006312f930-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.421 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:24.422 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:24.423 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:24.424 DEBUG:boto:path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:24.424 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj116 2022-09-03T06:50:24.425 DEBUG:boto:Method: GET 2022-09-03T06:50:24.425 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj116 2022-09-03T06:50:24.426 DEBUG:boto:Data: 2022-09-03T06:50:24.427 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.427 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:24.428 DEBUG:boto:Port: 8000 2022-09-03T06:50:24.429 DEBUG:boto:Params: {} 2022-09-03T06:50:24.429 DEBUG:boto:Token: None 2022-09-03T06:50:24.430 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/ 2022-09-03T06:50:24.431 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo= 2022-09-03T06:50:24.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo=', 'Content-Length': '0'} 2022-09-03T06:50:24.435 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009ee401c99ad550ea-006312f930-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.478 DEBUG:boto:b'dsgvcy-75obj1161000falseobj1162022-09-03T06:44:18.363Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:24.479 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2022-09-03T06:50:24.480 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:24.480 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:24.481 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.482 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.483 DEBUG:boto:Method: GET 2022-09-03T06:50:24.485 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.486 DEBUG:boto:Data: 2022-09-03T06:50:24.487 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.488 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:24.489 DEBUG:boto:Port: 8001 2022-09-03T06:50:24.490 DEBUG:boto:Params: {} 2022-09-03T06:50:24.491 DEBUG:boto:Token: None 2022-09-03T06:50:24.492 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.493 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y94PKu+0I5ggqoeMz9LR6aDR+8U= 2022-09-03T06:50:24.494 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y94PKu+0I5ggqoeMz9LR6aDR+8U=', 'Content-Length': '0'} 2022-09-03T06:50:24.497 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c80e4e932670d7a-006312f930-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.499 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:24.500 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:24.501 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.502 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.503 DEBUG:boto:Method: GET 2022-09-03T06:50:24.505 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.506 DEBUG:boto:Data: 2022-09-03T06:50:24.507 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.508 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:24.509 DEBUG:boto:Port: 8000 2022-09-03T06:50:24.510 DEBUG:boto:Params: {} 2022-09-03T06:50:24.511 DEBUG:boto:Token: None 2022-09-03T06:50:24.513 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.513 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y94PKu+0I5ggqoeMz9LR6aDR+8U= 2022-09-03T06:50:24.514 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y94PKu+0I5ggqoeMz9LR6aDR+8U=', 'Content-Length': '0'} 2022-09-03T06:50:24.517 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000253ba6f549ae4b4e-006312f930-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.518 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:24.519 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:24.520 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.520 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.521 DEBUG:boto:Method: HEAD 2022-09-03T06:50:24.522 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.523 DEBUG:boto:Data: 2022-09-03T06:50:24.524 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.524 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:24.525 DEBUG:boto:Port: 8001 2022-09-03T06:50:24.526 DEBUG:boto:Params: {} 2022-09-03T06:50:24.527 DEBUG:boto:Token: None 2022-09-03T06:50:24.528 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.529 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tbk1ysxfDfuYSFtvEQwj3Ir2izc= 2022-09-03T06:50:24.530 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tbk1ysxfDfuYSFtvEQwj3Ir2izc=', 'Content-Length': '0'} 2022-09-03T06:50:24.532 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ef1be903e6522e8e-006312f930-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.534 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:24.535 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:24.535 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.536 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.537 DEBUG:boto:Method: HEAD 2022-09-03T06:50:24.538 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.539 DEBUG:boto:Data: 2022-09-03T06:50:24.540 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.541 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:24.542 DEBUG:boto:Port: 8000 2022-09-03T06:50:24.543 DEBUG:boto:Params: {} 2022-09-03T06:50:24.544 DEBUG:boto:Token: None 2022-09-03T06:50:24.545 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.546 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tbk1ysxfDfuYSFtvEQwj3Ir2izc= 2022-09-03T06:50:24.548 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tbk1ysxfDfuYSFtvEQwj3Ir2izc=', 'Content-Length': '0'} 2022-09-03T06:50:24.550 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000377260d78e9f00f6-006312f930-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.552 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2022-09-03T06:50:24.553 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:24.554 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:24.555 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.556 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.557 DEBUG:boto:Method: GET 2022-09-03T06:50:24.557 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.559 DEBUG:boto:Data: 2022-09-03T06:50:24.559 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.560 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:24.560 DEBUG:boto:Port: 8001 2022-09-03T06:50:24.561 DEBUG:boto:Params: {} 2022-09-03T06:50:24.562 DEBUG:boto:Token: None 2022-09-03T06:50:24.562 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.563 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y94PKu+0I5ggqoeMz9LR6aDR+8U= 2022-09-03T06:50:24.563 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y94PKu+0I5ggqoeMz9LR6aDR+8U=', 'Content-Length': '0'} 2022-09-03T06:50:24.566 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f282c6e18596ac92-006312f930-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.566 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:24.567 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:24.568 DEBUG:boto:path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.568 DEBUG:boto:auth_path=/dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.569 DEBUG:boto:Method: GET 2022-09-03T06:50:24.569 DEBUG:boto:Path: /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.570 DEBUG:boto:Data: 2022-09-03T06:50:24.570 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.571 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:24.571 DEBUG:boto:Port: 8000 2022-09-03T06:50:24.572 DEBUG:boto:Params: {} 2022-09-03T06:50:24.573 DEBUG:boto:Token: None 2022-09-03T06:50:24.573 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj124?versionId=null 2022-09-03T06:50:24.574 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:y94PKu+0I5ggqoeMz9LR6aDR+8U= 2022-09-03T06:50:24.575 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:y94PKu+0I5ggqoeMz9LR6aDR+8U=', 'Content-Length': '0'} 2022-09-03T06:50:24.577 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000464e72ee777e518-006312f930-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.578 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:24.579 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:24.579 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:24.580 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:24.580 DEBUG:boto:Method: GET 2022-09-03T06:50:24.581 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:50:24.582 DEBUG:boto:Data: 2022-09-03T06:50:24.582 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.583 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:24.583 DEBUG:boto:Port: 8001 2022-09-03T06:50:24.584 DEBUG:boto:Params: {} 2022-09-03T06:50:24.585 DEBUG:boto:Token: None 2022-09-03T06:50:24.585 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/ 2022-09-03T06:50:24.586 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo= 2022-09-03T06:50:24.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo=', 'Content-Length': '0'} 2022-09-03T06:50:24.590 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000077c3a232ba8ebed5-006312f930-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.590 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:24.591 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:24.592 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:24.592 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:24.593 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:24.593 DEBUG:boto:Method: GET 2022-09-03T06:50:24.594 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:50:24.594 DEBUG:boto:Data: 2022-09-03T06:50:24.595 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.596 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:24.597 DEBUG:boto:Port: 8000 2022-09-03T06:50:24.598 DEBUG:boto:Params: {} 2022-09-03T06:50:24.599 DEBUG:boto:Token: None 2022-09-03T06:50:24.600 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/ 2022-09-03T06:50:24.601 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo= 2022-09-03T06:50:24.602 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo=', 'Content-Length': '0'} 2022-09-03T06:50:24.605 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004400f7b8270c0f89-006312f930-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.606 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:24.608 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:24.609 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:24.609 DEBUG:boto:Method: HEAD 2022-09-03T06:50:24.611 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:50:24.612 DEBUG:boto:Data: 2022-09-03T06:50:24.613 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.613 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:24.614 DEBUG:boto:Port: 8001 2022-09-03T06:50:24.615 DEBUG:boto:Params: {} 2022-09-03T06:50:24.616 DEBUG:boto:Token: None 2022-09-03T06:50:24.617 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj124 2022-09-03T06:50:24.618 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZWKnU+kw7i4XMxiBdQyZC2/Jo2I= 2022-09-03T06:50:24.619 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZWKnU+kw7i4XMxiBdQyZC2/Jo2I=', 'Content-Length': '0'} 2022-09-03T06:50:24.621 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bc8a2ff5b67f8db1-006312f930-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.622 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:24.623 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:24.623 DEBUG:boto:Method: HEAD 2022-09-03T06:50:24.624 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:50:24.624 DEBUG:boto:Data: 2022-09-03T06:50:24.625 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.626 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:24.628 DEBUG:boto:Port: 8000 2022-09-03T06:50:24.629 DEBUG:boto:Params: {} 2022-09-03T06:50:24.630 DEBUG:boto:Token: None 2022-09-03T06:50:24.631 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj124 2022-09-03T06:50:24.632 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZWKnU+kw7i4XMxiBdQyZC2/Jo2I= 2022-09-03T06:50:24.633 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZWKnU+kw7i4XMxiBdQyZC2/Jo2I=', 'Content-Length': '0'} 2022-09-03T06:50:24.638 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d3c2acc654b2ee94-006312f930-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.640 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj124 2022-09-03T06:50:24.641 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:24.642 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:24.643 DEBUG:boto:Method: GET 2022-09-03T06:50:24.643 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:50:24.644 DEBUG:boto:Data: 2022-09-03T06:50:24.644 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.645 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:24.646 DEBUG:boto:Port: 8001 2022-09-03T06:50:24.647 DEBUG:boto:Params: {} 2022-09-03T06:50:24.648 DEBUG:boto:Token: None 2022-09-03T06:50:24.649 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj124 2022-09-03T06:50:24.650 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V9CwTRv5dSRQlEr+ywiicpcQEJQ= 2022-09-03T06:50:24.651 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V9CwTRv5dSRQlEr+ywiicpcQEJQ=', 'Content-Length': '0'} 2022-09-03T06:50:24.655 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008f5a9725cb8a8192-006312f930-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.656 DEBUG:boto:path=/dsgvcy-75/obj124 2022-09-03T06:50:24.657 DEBUG:boto:auth_path=/dsgvcy-75/obj124 2022-09-03T06:50:24.658 DEBUG:boto:Method: GET 2022-09-03T06:50:24.659 DEBUG:boto:Path: /dsgvcy-75/obj124 2022-09-03T06:50:24.661 DEBUG:boto:Data: 2022-09-03T06:50:24.661 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.662 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:24.663 DEBUG:boto:Port: 8000 2022-09-03T06:50:24.664 DEBUG:boto:Params: {} 2022-09-03T06:50:24.665 DEBUG:boto:Token: None 2022-09-03T06:50:24.667 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj124 2022-09-03T06:50:24.668 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V9CwTRv5dSRQlEr+ywiicpcQEJQ= 2022-09-03T06:50:24.669 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V9CwTRv5dSRQlEr+ywiicpcQEJQ=', 'Content-Length': '0'} 2022-09-03T06:50:24.672 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f624694248674356-006312f930-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.673 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:24.674 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:24.675 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:24.678 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:24.679 DEBUG:boto:Method: GET 2022-09-03T06:50:24.680 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:50:24.681 DEBUG:boto:Data: 2022-09-03T06:50:24.682 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.684 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:24.685 DEBUG:boto:Port: 8001 2022-09-03T06:50:24.686 DEBUG:boto:Params: {} 2022-09-03T06:50:24.687 DEBUG:boto:Token: None 2022-09-03T06:50:24.688 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/ 2022-09-03T06:50:24.689 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo= 2022-09-03T06:50:24.689 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo=', 'Content-Length': '0'} 2022-09-03T06:50:24.693 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bed12b15b1858747-006312f930-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.740 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:24.742 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:24.743 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:24.744 DEBUG:boto:path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:24.745 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj124 2022-09-03T06:50:24.746 DEBUG:boto:Method: GET 2022-09-03T06:50:24.747 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj124 2022-09-03T06:50:24.748 DEBUG:boto:Data: 2022-09-03T06:50:24.749 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.750 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:24.751 DEBUG:boto:Port: 8000 2022-09-03T06:50:24.752 DEBUG:boto:Params: {} 2022-09-03T06:50:24.757 DEBUG:boto:Token: None 2022-09-03T06:50:24.758 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/ 2022-09-03T06:50:24.760 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo= 2022-09-03T06:50:24.761 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo=', 'Content-Length': '0'} 2022-09-03T06:50:24.765 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003f0ec927cb36618a-006312f930-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.766 DEBUG:boto:b'dsgvcy-75obj1241000falseobj1242022-09-03T06:45:46.963Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:24.767 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:50:24.767 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:24.770 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:24.771 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.772 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.773 DEBUG:boto:Method: GET 2022-09-03T06:50:24.774 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.775 DEBUG:boto:Data: 2022-09-03T06:50:24.776 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.777 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:24.778 DEBUG:boto:Port: 8001 2022-09-03T06:50:24.779 DEBUG:boto:Params: {} 2022-09-03T06:50:24.781 DEBUG:boto:Token: None 2022-09-03T06:50:24.782 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.783 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l5PoOefUIW0s6NIxjA0J1D1FC5Q= 2022-09-03T06:50:24.784 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l5PoOefUIW0s6NIxjA0J1D1FC5Q=', 'Content-Length': '0'} 2022-09-03T06:50:24.789 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001bde75fee4aff6f2-006312f930-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.791 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:24.792 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:24.793 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.794 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.795 DEBUG:boto:Method: GET 2022-09-03T06:50:24.796 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.797 DEBUG:boto:Data: 2022-09-03T06:50:24.798 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.799 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:24.800 DEBUG:boto:Port: 8000 2022-09-03T06:50:24.801 DEBUG:boto:Params: {} 2022-09-03T06:50:24.802 DEBUG:boto:Token: None 2022-09-03T06:50:24.803 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.805 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l5PoOefUIW0s6NIxjA0J1D1FC5Q= 2022-09-03T06:50:24.806 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l5PoOefUIW0s6NIxjA0J1D1FC5Q=', 'Content-Length': '0'} 2022-09-03T06:50:24.809 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000285d3a2edfb98b49-006312f930-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.810 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:24.812 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:24.813 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.814 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.815 DEBUG:boto:Method: HEAD 2022-09-03T06:50:24.816 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.816 DEBUG:boto:Data: 2022-09-03T06:50:24.817 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.818 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:24.819 DEBUG:boto:Port: 8001 2022-09-03T06:50:24.820 DEBUG:boto:Params: {} 2022-09-03T06:50:24.821 DEBUG:boto:Token: None 2022-09-03T06:50:24.823 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.825 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FTp7AaNPZbVvcyO+jUXcj719vNQ= 2022-09-03T06:50:24.826 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FTp7AaNPZbVvcyO+jUXcj719vNQ=', 'Content-Length': '0'} 2022-09-03T06:50:24.829 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000085b4fbebc33ca943-006312f930-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.831 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:24.831 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:24.832 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.832 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.833 DEBUG:boto:Method: HEAD 2022-09-03T06:50:24.834 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.834 DEBUG:boto:Data: 2022-09-03T06:50:24.835 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.835 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:24.836 DEBUG:boto:Port: 8000 2022-09-03T06:50:24.836 DEBUG:boto:Params: {} 2022-09-03T06:50:24.837 DEBUG:boto:Token: None 2022-09-03T06:50:24.838 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.839 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FTp7AaNPZbVvcyO+jUXcj719vNQ= 2022-09-03T06:50:24.840 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FTp7AaNPZbVvcyO+jUXcj719vNQ=', 'Content-Length': '0'} 2022-09-03T06:50:24.843 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000012efc6be2c03c6f-006312f930-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.844 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:50:24.845 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:24.847 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:24.848 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.849 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.850 DEBUG:boto:Method: GET 2022-09-03T06:50:24.850 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.851 DEBUG:boto:Data: 2022-09-03T06:50:24.852 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.856 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:24.857 DEBUG:boto:Port: 8001 2022-09-03T06:50:24.857 DEBUG:boto:Params: {} 2022-09-03T06:50:24.858 DEBUG:boto:Token: None 2022-09-03T06:50:24.858 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.859 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l5PoOefUIW0s6NIxjA0J1D1FC5Q= 2022-09-03T06:50:24.860 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l5PoOefUIW0s6NIxjA0J1D1FC5Q=', 'Content-Length': '0'} 2022-09-03T06:50:24.865 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dcc42669b05ecb94-006312f930-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.866 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:24.867 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:24.867 DEBUG:boto:path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.868 DEBUG:boto:auth_path=/dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.868 DEBUG:boto:Method: GET 2022-09-03T06:50:24.869 DEBUG:boto:Path: /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.869 DEBUG:boto:Data: 2022-09-03T06:50:24.870 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.870 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:24.871 DEBUG:boto:Port: 8000 2022-09-03T06:50:24.873 DEBUG:boto:Params: {} 2022-09-03T06:50:24.874 DEBUG:boto:Token: None 2022-09-03T06:50:24.874 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj145?versionId=null 2022-09-03T06:50:24.875 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:l5PoOefUIW0s6NIxjA0J1D1FC5Q= 2022-09-03T06:50:24.876 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:l5PoOefUIW0s6NIxjA0J1D1FC5Q=', 'Content-Length': '0'} 2022-09-03T06:50:24.879 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005c08a89388c59a22-006312f930-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.931 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:24.933 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:24.934 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:24.935 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:24.936 DEBUG:boto:Method: GET 2022-09-03T06:50:24.936 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:50:24.937 DEBUG:boto:Data: 2022-09-03T06:50:24.939 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.940 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:24.940 DEBUG:boto:Port: 8001 2022-09-03T06:50:24.941 DEBUG:boto:Params: {} 2022-09-03T06:50:24.942 DEBUG:boto:Token: None 2022-09-03T06:50:24.942 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/ 2022-09-03T06:50:24.943 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo= 2022-09-03T06:50:24.944 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo=', 'Content-Length': '0'} 2022-09-03T06:50:24.947 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f2d4366fc936d797-006312f930-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.948 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:24.949 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:24.950 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:24.951 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:24.952 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:24.953 DEBUG:boto:Method: GET 2022-09-03T06:50:24.954 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:50:24.954 DEBUG:boto:Data: 2022-09-03T06:50:24.956 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.956 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:24.957 DEBUG:boto:Port: 8000 2022-09-03T06:50:24.958 DEBUG:boto:Params: {} 2022-09-03T06:50:24.959 DEBUG:boto:Token: None 2022-09-03T06:50:24.960 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/ 2022-09-03T06:50:24.961 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo= 2022-09-03T06:50:24.962 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bsamuapdC7Aq9PuALHKTd9weQDo=', 'Content-Length': '0'} 2022-09-03T06:50:24.965 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000f97146ed12106ba-006312f930-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.966 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:24.968 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:24.969 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:24.970 DEBUG:boto:Method: HEAD 2022-09-03T06:50:24.971 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:50:24.973 DEBUG:boto:Data: 2022-09-03T06:50:24.973 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.974 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:24.974 DEBUG:boto:Port: 8001 2022-09-03T06:50:24.975 DEBUG:boto:Params: {} 2022-09-03T06:50:24.976 DEBUG:boto:Token: None 2022-09-03T06:50:24.976 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj145 2022-09-03T06:50:24.977 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cx4JIgAiZk6+BVn+Kkqnz5TQ45E= 2022-09-03T06:50:24.977 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cx4JIgAiZk6+BVn+Kkqnz5TQ45E=', 'Content-Length': '0'} 2022-09-03T06:50:24.980 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ee1dbff70dd8a8c7-006312f930-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.980 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:24.981 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:24.982 DEBUG:boto:Method: HEAD 2022-09-03T06:50:24.982 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:50:24.983 DEBUG:boto:Data: 2022-09-03T06:50:24.983 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.984 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:24.984 DEBUG:boto:Port: 8000 2022-09-03T06:50:24.985 DEBUG:boto:Params: {} 2022-09-03T06:50:24.985 DEBUG:boto:Token: None 2022-09-03T06:50:24.986 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj145 2022-09-03T06:50:24.987 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cx4JIgAiZk6+BVn+Kkqnz5TQ45E= 2022-09-03T06:50:24.987 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cx4JIgAiZk6+BVn+Kkqnz5TQ45E=', 'Content-Length': '0'} 2022-09-03T06:50:24.989 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002713c4db6973e554-006312f930-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:24.990 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj145 2022-09-03T06:50:24.991 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:24.992 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:24.992 DEBUG:boto:Method: GET 2022-09-03T06:50:24.993 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:50:24.994 DEBUG:boto:Data: 2022-09-03T06:50:24.995 DEBUG:boto:Headers: {} 2022-09-03T06:50:24.996 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:24.996 DEBUG:boto:Port: 8001 2022-09-03T06:50:24.997 DEBUG:boto:Params: {} 2022-09-03T06:50:24.997 DEBUG:boto:Token: None 2022-09-03T06:50:24.998 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:24 GMT /dsgvcy-75/obj145 2022-09-03T06:50:24.999 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:95D5VFyM7AvAKji9ojJvywkn0Cw= 2022-09-03T06:50:24.999 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:24 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:95D5VFyM7AvAKji9ojJvywkn0Cw=', 'Content-Length': '0'} 2022-09-03T06:50:25.002 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008ade42d7e076346c-006312f930-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:24 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.046 DEBUG:boto:path=/dsgvcy-75/obj145 2022-09-03T06:50:25.047 DEBUG:boto:auth_path=/dsgvcy-75/obj145 2022-09-03T06:50:25.048 DEBUG:boto:Method: GET 2022-09-03T06:50:25.049 DEBUG:boto:Path: /dsgvcy-75/obj145 2022-09-03T06:50:25.050 DEBUG:boto:Data: 2022-09-03T06:50:25.052 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.053 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:25.055 DEBUG:boto:Port: 8000 2022-09-03T06:50:25.056 DEBUG:boto:Params: {} 2022-09-03T06:50:25.056 DEBUG:boto:Token: None 2022-09-03T06:50:25.057 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj145 2022-09-03T06:50:25.057 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:X4Bl3jUbZC/zcfTEs2Daa555fIQ= 2022-09-03T06:50:25.058 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:X4Bl3jUbZC/zcfTEs2Daa555fIQ=', 'Content-Length': '0'} 2022-09-03T06:50:25.060 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000264ba160c571b1f8-006312f931-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.061 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:25.063 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:25.063 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:25.064 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:25.064 DEBUG:boto:Method: GET 2022-09-03T06:50:25.065 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:50:25.067 DEBUG:boto:Data: 2022-09-03T06:50:25.068 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.068 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:25.069 DEBUG:boto:Port: 8001 2022-09-03T06:50:25.069 DEBUG:boto:Params: {} 2022-09-03T06:50:25.070 DEBUG:boto:Token: None 2022-09-03T06:50:25.071 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/ 2022-09-03T06:50:25.074 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM= 2022-09-03T06:50:25.075 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM=', 'Content-Length': '0'} 2022-09-03T06:50:25.078 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000858cea293690ff61-006312f931-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.079 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:25.080 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:25.080 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:25.081 DEBUG:boto:path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:25.081 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj145 2022-09-03T06:50:25.082 DEBUG:boto:Method: GET 2022-09-03T06:50:25.083 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj145 2022-09-03T06:50:25.083 DEBUG:boto:Data: 2022-09-03T06:50:25.084 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.085 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:25.085 DEBUG:boto:Port: 8000 2022-09-03T06:50:25.086 DEBUG:boto:Params: {} 2022-09-03T06:50:25.087 DEBUG:boto:Token: None 2022-09-03T06:50:25.087 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/ 2022-09-03T06:50:25.088 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM= 2022-09-03T06:50:25.088 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM=', 'Content-Length': '0'} 2022-09-03T06:50:25.092 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f1722540b4128ff0-006312f931-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.094 DEBUG:boto:b'dsgvcy-75obj1451000falseobj1452022-09-03T06:44:18.391Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:25.095 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj148 2022-09-03T06:50:25.096 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:25.097 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:25.098 DEBUG:boto:path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.098 DEBUG:boto:auth_path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.099 DEBUG:boto:Method: GET 2022-09-03T06:50:25.100 DEBUG:boto:Path: /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.100 DEBUG:boto:Data: 2022-09-03T06:50:25.101 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.102 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:25.102 DEBUG:boto:Port: 8001 2022-09-03T06:50:25.103 DEBUG:boto:Params: {} 2022-09-03T06:50:25.104 DEBUG:boto:Token: None 2022-09-03T06:50:25.107 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.108 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LqMNds47M4vtAJUgg2ao1TlZmjw= 2022-09-03T06:50:25.109 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LqMNds47M4vtAJUgg2ao1TlZmjw=', 'Content-Length': '0'} 2022-09-03T06:50:25.112 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000895efa8f4fadf2c8-006312f931-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.114 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:25.115 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:25.116 DEBUG:boto:path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.117 DEBUG:boto:auth_path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.118 DEBUG:boto:Method: GET 2022-09-03T06:50:25.119 DEBUG:boto:Path: /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.120 DEBUG:boto:Data: 2022-09-03T06:50:25.121 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.122 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:25.123 DEBUG:boto:Port: 8000 2022-09-03T06:50:25.123 DEBUG:boto:Params: {} 2022-09-03T06:50:25.125 DEBUG:boto:Token: None 2022-09-03T06:50:25.126 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.127 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LqMNds47M4vtAJUgg2ao1TlZmjw= 2022-09-03T06:50:25.128 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LqMNds47M4vtAJUgg2ao1TlZmjw=', 'Content-Length': '0'} 2022-09-03T06:50:25.131 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c43a5728137a0354-006312f931-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.132 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:25.133 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:25.134 DEBUG:boto:path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.135 DEBUG:boto:auth_path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.136 DEBUG:boto:Method: HEAD 2022-09-03T06:50:25.137 DEBUG:boto:Path: /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.138 DEBUG:boto:Data: 2022-09-03T06:50:25.139 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.139 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:25.140 DEBUG:boto:Port: 8001 2022-09-03T06:50:25.141 DEBUG:boto:Params: {} 2022-09-03T06:50:25.142 DEBUG:boto:Token: None 2022-09-03T06:50:25.143 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.144 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fK8sbSe6H9RsryByd0Hq3Qd0O9s= 2022-09-03T06:50:25.145 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fK8sbSe6H9RsryByd0Hq3Qd0O9s=', 'Content-Length': '0'} 2022-09-03T06:50:25.147 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000006b646eab6fac7d1-006312f931-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.149 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:25.149 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:25.150 DEBUG:boto:path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.151 DEBUG:boto:auth_path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.152 DEBUG:boto:Method: HEAD 2022-09-03T06:50:25.153 DEBUG:boto:Path: /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.154 DEBUG:boto:Data: 2022-09-03T06:50:25.154 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.156 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:25.157 DEBUG:boto:Port: 8000 2022-09-03T06:50:25.158 DEBUG:boto:Params: {} 2022-09-03T06:50:25.159 DEBUG:boto:Token: None 2022-09-03T06:50:25.160 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.161 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fK8sbSe6H9RsryByd0Hq3Qd0O9s= 2022-09-03T06:50:25.162 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fK8sbSe6H9RsryByd0Hq3Qd0O9s=', 'Content-Length': '0'} 2022-09-03T06:50:25.165 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007e1bf19940b72c78-006312f931-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.166 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj148 2022-09-03T06:50:25.167 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:25.168 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:25.169 DEBUG:boto:path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.169 DEBUG:boto:auth_path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.171 DEBUG:boto:Method: GET 2022-09-03T06:50:25.171 DEBUG:boto:Path: /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.173 DEBUG:boto:Data: 2022-09-03T06:50:25.174 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.174 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:25.175 DEBUG:boto:Port: 8001 2022-09-03T06:50:25.175 DEBUG:boto:Params: {} 2022-09-03T06:50:25.176 DEBUG:boto:Token: None 2022-09-03T06:50:25.177 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.177 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LqMNds47M4vtAJUgg2ao1TlZmjw= 2022-09-03T06:50:25.178 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LqMNds47M4vtAJUgg2ao1TlZmjw=', 'Content-Length': '0'} 2022-09-03T06:50:25.180 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad25159a06c8cb4e-006312f931-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.181 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:25.182 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:25.184 DEBUG:boto:path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.184 DEBUG:boto:auth_path=/dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.185 DEBUG:boto:Method: GET 2022-09-03T06:50:25.186 DEBUG:boto:Path: /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.186 DEBUG:boto:Data: 2022-09-03T06:50:25.187 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.187 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:25.188 DEBUG:boto:Port: 8000 2022-09-03T06:50:25.188 DEBUG:boto:Params: {} 2022-09-03T06:50:25.189 DEBUG:boto:Token: None 2022-09-03T06:50:25.190 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj148?versionId=null 2022-09-03T06:50:25.190 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LqMNds47M4vtAJUgg2ao1TlZmjw= 2022-09-03T06:50:25.191 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LqMNds47M4vtAJUgg2ao1TlZmjw=', 'Content-Length': '0'} 2022-09-03T06:50:25.195 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000084be7c28589aaf43-006312f931-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.238 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:25.240 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:25.241 DEBUG:boto:path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:25.241 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:25.242 DEBUG:boto:Method: GET 2022-09-03T06:50:25.243 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj148 2022-09-03T06:50:25.244 DEBUG:boto:Data: 2022-09-03T06:50:25.245 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.252 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:25.253 DEBUG:boto:Port: 8001 2022-09-03T06:50:25.254 DEBUG:boto:Params: {} 2022-09-03T06:50:25.255 DEBUG:boto:Token: None 2022-09-03T06:50:25.256 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/ 2022-09-03T06:50:25.257 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM= 2022-09-03T06:50:25.258 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM=', 'Content-Length': '0'} 2022-09-03T06:50:25.262 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000501a09447a46140f-006312f931-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.263 DEBUG:boto:b'dsgvcy-75obj1481000falseobj1482022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:25.264 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:25.265 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:25.266 DEBUG:boto:path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:25.267 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:25.268 DEBUG:boto:Method: GET 2022-09-03T06:50:25.269 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj148 2022-09-03T06:50:25.270 DEBUG:boto:Data: 2022-09-03T06:50:25.271 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.272 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:25.273 DEBUG:boto:Port: 8000 2022-09-03T06:50:25.274 DEBUG:boto:Params: {} 2022-09-03T06:50:25.275 DEBUG:boto:Token: None 2022-09-03T06:50:25.286 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/ 2022-09-03T06:50:25.288 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM= 2022-09-03T06:50:25.289 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM=', 'Content-Length': '0'} 2022-09-03T06:50:25.301 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bb80ea13ab0eb34d-006312f931-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.308 DEBUG:boto:b'dsgvcy-75obj1481000falseobj1482022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:25.310 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:25.311 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:25.312 DEBUG:boto:Method: HEAD 2022-09-03T06:50:25.313 DEBUG:boto:Path: /dsgvcy-75/obj148 2022-09-03T06:50:25.314 DEBUG:boto:Data: 2022-09-03T06:50:25.315 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.317 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:25.318 DEBUG:boto:Port: 8001 2022-09-03T06:50:25.319 DEBUG:boto:Params: {} 2022-09-03T06:50:25.320 DEBUG:boto:Token: None 2022-09-03T06:50:25.321 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj148 2022-09-03T06:50:25.322 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z3BuuKWXSAW9FMtb9IJDkQlBU2Y= 2022-09-03T06:50:25.323 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z3BuuKWXSAW9FMtb9IJDkQlBU2Y=', 'Content-Length': '0'} 2022-09-03T06:50:25.326 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004194778e6e1982e8-006312f931-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.327 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:25.328 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:25.328 DEBUG:boto:Method: HEAD 2022-09-03T06:50:25.329 DEBUG:boto:Path: /dsgvcy-75/obj148 2022-09-03T06:50:25.330 DEBUG:boto:Data: 2022-09-03T06:50:25.331 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.331 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:25.332 DEBUG:boto:Port: 8000 2022-09-03T06:50:25.334 DEBUG:boto:Params: {} 2022-09-03T06:50:25.335 DEBUG:boto:Token: None 2022-09-03T06:50:25.336 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj148 2022-09-03T06:50:25.336 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z3BuuKWXSAW9FMtb9IJDkQlBU2Y= 2022-09-03T06:50:25.337 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z3BuuKWXSAW9FMtb9IJDkQlBU2Y=', 'Content-Length': '0'} 2022-09-03T06:50:25.340 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b13e3c2d94cb70a5-006312f931-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.350 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj148 2022-09-03T06:50:25.352 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:25.352 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:25.353 DEBUG:boto:Method: GET 2022-09-03T06:50:25.353 DEBUG:boto:Path: /dsgvcy-75/obj148 2022-09-03T06:50:25.354 DEBUG:boto:Data: 2022-09-03T06:50:25.354 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.355 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:25.356 DEBUG:boto:Port: 8001 2022-09-03T06:50:25.356 DEBUG:boto:Params: {} 2022-09-03T06:50:25.357 DEBUG:boto:Token: None 2022-09-03T06:50:25.357 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj148 2022-09-03T06:50:25.358 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JK73g6Io4kHIB8hgZnJan6/DByk= 2022-09-03T06:50:25.359 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JK73g6Io4kHIB8hgZnJan6/DByk=', 'Content-Length': '0'} 2022-09-03T06:50:25.361 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b6dd0a987853e257-006312f931-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.362 DEBUG:boto:path=/dsgvcy-75/obj148 2022-09-03T06:50:25.362 DEBUG:boto:auth_path=/dsgvcy-75/obj148 2022-09-03T06:50:25.363 DEBUG:boto:Method: GET 2022-09-03T06:50:25.363 DEBUG:boto:Path: /dsgvcy-75/obj148 2022-09-03T06:50:25.364 DEBUG:boto:Data: 2022-09-03T06:50:25.364 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.365 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:25.366 DEBUG:boto:Port: 8000 2022-09-03T06:50:25.366 DEBUG:boto:Params: {} 2022-09-03T06:50:25.367 DEBUG:boto:Token: None 2022-09-03T06:50:25.367 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj148 2022-09-03T06:50:25.368 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:JK73g6Io4kHIB8hgZnJan6/DByk= 2022-09-03T06:50:25.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:JK73g6Io4kHIB8hgZnJan6/DByk=', 'Content-Length': '0'} 2022-09-03T06:50:25.371 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000072162e9a5445cf5-006312f931-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.372 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:25.373 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:25.373 DEBUG:boto:path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:25.374 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:25.375 DEBUG:boto:Method: GET 2022-09-03T06:50:25.375 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj148 2022-09-03T06:50:25.376 DEBUG:boto:Data: 2022-09-03T06:50:25.376 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.377 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:25.377 DEBUG:boto:Port: 8001 2022-09-03T06:50:25.378 DEBUG:boto:Params: {} 2022-09-03T06:50:25.378 DEBUG:boto:Token: None 2022-09-03T06:50:25.379 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/ 2022-09-03T06:50:25.380 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM= 2022-09-03T06:50:25.380 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM=', 'Content-Length': '0'} 2022-09-03T06:50:25.385 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b0c56ae21ec54dd6-006312f931-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.385 DEBUG:boto:b'dsgvcy-75obj1481000falseobj1482022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:25.386 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:25.387 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:25.389 DEBUG:boto:path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:25.390 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj148 2022-09-03T06:50:25.390 DEBUG:boto:Method: GET 2022-09-03T06:50:25.391 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj148 2022-09-03T06:50:25.391 DEBUG:boto:Data: 2022-09-03T06:50:25.392 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.393 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:25.394 DEBUG:boto:Port: 8000 2022-09-03T06:50:25.395 DEBUG:boto:Params: {} 2022-09-03T06:50:25.396 DEBUG:boto:Token: None 2022-09-03T06:50:25.397 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/ 2022-09-03T06:50:25.398 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM= 2022-09-03T06:50:25.400 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM=', 'Content-Length': '0'} 2022-09-03T06:50:25.403 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000148460f72be2c317-006312f931-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.405 DEBUG:boto:b'dsgvcy-75obj1481000falseobj1482022-09-03T06:47:49.702Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:25.407 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2022-09-03T06:50:25.408 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:25.408 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:25.410 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.411 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.411 DEBUG:boto:Method: GET 2022-09-03T06:50:25.412 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.412 DEBUG:boto:Data: 2022-09-03T06:50:25.413 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.413 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:25.414 DEBUG:boto:Port: 8001 2022-09-03T06:50:25.415 DEBUG:boto:Params: {} 2022-09-03T06:50:25.415 DEBUG:boto:Token: None 2022-09-03T06:50:25.416 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.417 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/4jVqliSBE5PkyhhJ9goiA377E= 2022-09-03T06:50:25.418 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/4jVqliSBE5PkyhhJ9goiA377E=', 'Content-Length': '0'} 2022-09-03T06:50:25.421 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c950ed6b9ae6028f-006312f931-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.465 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:25.467 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:25.468 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.469 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.470 DEBUG:boto:Method: GET 2022-09-03T06:50:25.470 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.471 DEBUG:boto:Data: 2022-09-03T06:50:25.472 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.473 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:25.474 DEBUG:boto:Port: 8000 2022-09-03T06:50:25.474 DEBUG:boto:Params: {} 2022-09-03T06:50:25.475 DEBUG:boto:Token: None 2022-09-03T06:50:25.476 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.477 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/4jVqliSBE5PkyhhJ9goiA377E= 2022-09-03T06:50:25.477 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/4jVqliSBE5PkyhhJ9goiA377E=', 'Content-Length': '0'} 2022-09-03T06:50:25.480 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000020c504f0ca7b23ed-006312f931-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.481 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:25.481 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:25.482 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.482 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.483 DEBUG:boto:Method: HEAD 2022-09-03T06:50:25.483 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.484 DEBUG:boto:Data: 2022-09-03T06:50:25.484 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.485 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:25.485 DEBUG:boto:Port: 8001 2022-09-03T06:50:25.486 DEBUG:boto:Params: {} 2022-09-03T06:50:25.487 DEBUG:boto:Token: None 2022-09-03T06:50:25.489 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.490 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vlh3qRCeaNwh33GL9cOeKb88Efk= 2022-09-03T06:50:25.491 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vlh3qRCeaNwh33GL9cOeKb88Efk=', 'Content-Length': '0'} 2022-09-03T06:50:25.494 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000783bd2485de7a607-006312f931-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.495 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:25.495 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:25.496 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.497 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.498 DEBUG:boto:Method: HEAD 2022-09-03T06:50:25.498 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.499 DEBUG:boto:Data: 2022-09-03T06:50:25.499 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.500 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:25.500 DEBUG:boto:Port: 8000 2022-09-03T06:50:25.501 DEBUG:boto:Params: {} 2022-09-03T06:50:25.502 DEBUG:boto:Token: None 2022-09-03T06:50:25.502 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.503 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vlh3qRCeaNwh33GL9cOeKb88Efk= 2022-09-03T06:50:25.504 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vlh3qRCeaNwh33GL9cOeKb88Efk=', 'Content-Length': '0'} 2022-09-03T06:50:25.507 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d750a6b702a3433e-006312f931-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.508 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2022-09-03T06:50:25.509 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:25.511 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:25.511 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.512 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.513 DEBUG:boto:Method: GET 2022-09-03T06:50:25.513 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.514 DEBUG:boto:Data: 2022-09-03T06:50:25.515 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.515 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:25.516 DEBUG:boto:Port: 8001 2022-09-03T06:50:25.517 DEBUG:boto:Params: {} 2022-09-03T06:50:25.518 DEBUG:boto:Token: None 2022-09-03T06:50:25.519 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.520 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/4jVqliSBE5PkyhhJ9goiA377E= 2022-09-03T06:50:25.522 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/4jVqliSBE5PkyhhJ9goiA377E=', 'Content-Length': '0'} 2022-09-03T06:50:25.525 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007776651b9df80f3a-006312f931-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.526 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:25.527 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:25.528 DEBUG:boto:path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.528 DEBUG:boto:auth_path=/dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.529 DEBUG:boto:Method: GET 2022-09-03T06:50:25.529 DEBUG:boto:Path: /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.530 DEBUG:boto:Data: 2022-09-03T06:50:25.530 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.531 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:25.532 DEBUG:boto:Port: 8000 2022-09-03T06:50:25.532 DEBUG:boto:Params: {} 2022-09-03T06:50:25.533 DEBUG:boto:Token: None 2022-09-03T06:50:25.534 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj155?versionId=null 2022-09-03T06:50:25.535 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F/4jVqliSBE5PkyhhJ9goiA377E= 2022-09-03T06:50:25.535 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F/4jVqliSBE5PkyhhJ9goiA377E=', 'Content-Length': '0'} 2022-09-03T06:50:25.537 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009e0ede4a4c0eceb4-006312f931-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.538 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:25.539 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:25.539 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:25.540 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:25.540 DEBUG:boto:Method: GET 2022-09-03T06:50:25.541 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:50:25.541 DEBUG:boto:Data: 2022-09-03T06:50:25.542 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.542 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:25.543 DEBUG:boto:Port: 8001 2022-09-03T06:50:25.543 DEBUG:boto:Params: {} 2022-09-03T06:50:25.544 DEBUG:boto:Token: None 2022-09-03T06:50:25.545 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/ 2022-09-03T06:50:25.545 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM= 2022-09-03T06:50:25.546 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM=', 'Content-Length': '0'} 2022-09-03T06:50:25.549 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000536921d768b66af2-006312f931-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.550 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:25.551 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:25.552 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:25.553 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:25.554 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:25.555 DEBUG:boto:Method: GET 2022-09-03T06:50:25.556 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:50:25.557 DEBUG:boto:Data: 2022-09-03T06:50:25.558 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.559 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:25.560 DEBUG:boto:Port: 8000 2022-09-03T06:50:25.561 DEBUG:boto:Params: {} 2022-09-03T06:50:25.562 DEBUG:boto:Token: None 2022-09-03T06:50:25.563 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/ 2022-09-03T06:50:25.564 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM= 2022-09-03T06:50:25.565 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM=', 'Content-Length': '0'} 2022-09-03T06:50:25.570 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f9a21c0f1d3bdd2d-006312f931-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.571 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:25.573 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:25.574 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:25.575 DEBUG:boto:Method: HEAD 2022-09-03T06:50:25.576 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:50:25.577 DEBUG:boto:Data: 2022-09-03T06:50:25.578 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.578 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:25.579 DEBUG:boto:Port: 8001 2022-09-03T06:50:25.580 DEBUG:boto:Params: {} 2022-09-03T06:50:25.582 DEBUG:boto:Token: None 2022-09-03T06:50:25.583 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj155 2022-09-03T06:50:25.584 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xruPfPbjAV1a4ZnLlDrKlyEmXLo= 2022-09-03T06:50:25.585 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xruPfPbjAV1a4ZnLlDrKlyEmXLo=', 'Content-Length': '0'} 2022-09-03T06:50:25.588 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007a34550d9aa885a4-006312f931-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.589 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:25.590 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:25.590 DEBUG:boto:Method: HEAD 2022-09-03T06:50:25.591 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:50:25.591 DEBUG:boto:Data: 2022-09-03T06:50:25.592 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.593 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:25.593 DEBUG:boto:Port: 8000 2022-09-03T06:50:25.594 DEBUG:boto:Params: {} 2022-09-03T06:50:25.594 DEBUG:boto:Token: None 2022-09-03T06:50:25.595 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj155 2022-09-03T06:50:25.595 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xruPfPbjAV1a4ZnLlDrKlyEmXLo= 2022-09-03T06:50:25.596 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xruPfPbjAV1a4ZnLlDrKlyEmXLo=', 'Content-Length': '0'} 2022-09-03T06:50:25.598 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006476a53bc0dae4fd-006312f931-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.600 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj155 2022-09-03T06:50:25.601 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:25.601 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:25.602 DEBUG:boto:Method: GET 2022-09-03T06:50:25.603 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:50:25.604 DEBUG:boto:Data: 2022-09-03T06:50:25.605 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.606 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:25.607 DEBUG:boto:Port: 8001 2022-09-03T06:50:25.608 DEBUG:boto:Params: {} 2022-09-03T06:50:25.609 DEBUG:boto:Token: None 2022-09-03T06:50:25.610 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj155 2022-09-03T06:50:25.612 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:km5TE/VTP9E2xgJstcPeL1aiaFY= 2022-09-03T06:50:25.613 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:km5TE/VTP9E2xgJstcPeL1aiaFY=', 'Content-Length': '0'} 2022-09-03T06:50:25.616 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005f0548c72bc6883b-006312f931-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.619 DEBUG:boto:path=/dsgvcy-75/obj155 2022-09-03T06:50:25.620 DEBUG:boto:auth_path=/dsgvcy-75/obj155 2022-09-03T06:50:25.621 DEBUG:boto:Method: GET 2022-09-03T06:50:25.622 DEBUG:boto:Path: /dsgvcy-75/obj155 2022-09-03T06:50:25.623 DEBUG:boto:Data: 2022-09-03T06:50:25.624 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.625 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:25.625 DEBUG:boto:Port: 8000 2022-09-03T06:50:25.626 DEBUG:boto:Params: {} 2022-09-03T06:50:25.627 DEBUG:boto:Token: None 2022-09-03T06:50:25.628 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj155 2022-09-03T06:50:25.629 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:km5TE/VTP9E2xgJstcPeL1aiaFY= 2022-09-03T06:50:25.629 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:km5TE/VTP9E2xgJstcPeL1aiaFY=', 'Content-Length': '0'} 2022-09-03T06:50:25.633 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:47 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000019c1603c6625230e-006312f931-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.674 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:25.677 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:25.678 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:25.679 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:25.679 DEBUG:boto:Method: GET 2022-09-03T06:50:25.680 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:50:25.680 DEBUG:boto:Data: 2022-09-03T06:50:25.681 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.682 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:25.683 DEBUG:boto:Port: 8001 2022-09-03T06:50:25.683 DEBUG:boto:Params: {} 2022-09-03T06:50:25.684 DEBUG:boto:Token: None 2022-09-03T06:50:25.685 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/ 2022-09-03T06:50:25.687 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM= 2022-09-03T06:50:25.687 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM=', 'Content-Length': '0'} 2022-09-03T06:50:25.701 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003608091bbafcb0d0-006312f931-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.702 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:25.703 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:25.704 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:25.705 DEBUG:boto:path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:25.705 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj155 2022-09-03T06:50:25.706 DEBUG:boto:Method: GET 2022-09-03T06:50:25.706 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj155 2022-09-03T06:50:25.707 DEBUG:boto:Data: 2022-09-03T06:50:25.708 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.708 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:25.709 DEBUG:boto:Port: 8000 2022-09-03T06:50:25.709 DEBUG:boto:Params: {} 2022-09-03T06:50:25.710 DEBUG:boto:Token: None 2022-09-03T06:50:25.712 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/ 2022-09-03T06:50:25.713 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM= 2022-09-03T06:50:25.714 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM=', 'Content-Length': '0'} 2022-09-03T06:50:25.718 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000eb510687fbc73ed2-006312f931-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.719 DEBUG:boto:b'dsgvcy-75obj1551000falseobj1552022-09-03T06:45:47.037Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:25.719 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:50:25.720 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:25.721 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:25.721 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.723 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.723 DEBUG:boto:Method: GET 2022-09-03T06:50:25.724 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.724 DEBUG:boto:Data: 2022-09-03T06:50:25.725 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.725 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:25.726 DEBUG:boto:Port: 8001 2022-09-03T06:50:25.726 DEBUG:boto:Params: {} 2022-09-03T06:50:25.727 DEBUG:boto:Token: None 2022-09-03T06:50:25.728 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.728 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0OlVtLvWKcTtpvS8UxhyGn1eQHw= 2022-09-03T06:50:25.729 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0OlVtLvWKcTtpvS8UxhyGn1eQHw=', 'Content-Length': '0'} 2022-09-03T06:50:25.731 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b21e27706ae8bbe9-006312f931-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.774 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:25.775 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:25.776 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.777 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.778 DEBUG:boto:Method: GET 2022-09-03T06:50:25.779 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.780 DEBUG:boto:Data: 2022-09-03T06:50:25.781 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.782 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:25.783 DEBUG:boto:Port: 8000 2022-09-03T06:50:25.784 DEBUG:boto:Params: {} 2022-09-03T06:50:25.785 DEBUG:boto:Token: None 2022-09-03T06:50:25.786 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.787 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0OlVtLvWKcTtpvS8UxhyGn1eQHw= 2022-09-03T06:50:25.788 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0OlVtLvWKcTtpvS8UxhyGn1eQHw=', 'Content-Length': '0'} 2022-09-03T06:50:25.793 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d88ade1da7d38dc-006312f931-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.794 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:25.795 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:25.796 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.796 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.797 DEBUG:boto:Method: HEAD 2022-09-03T06:50:25.797 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.798 DEBUG:boto:Data: 2022-09-03T06:50:25.799 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.800 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:25.800 DEBUG:boto:Port: 8001 2022-09-03T06:50:25.801 DEBUG:boto:Params: {} 2022-09-03T06:50:25.801 DEBUG:boto:Token: None 2022-09-03T06:50:25.802 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.803 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CKzyIcbDr+RU/gs9O6hBt+oHE6o= 2022-09-03T06:50:25.804 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CKzyIcbDr+RU/gs9O6hBt+oHE6o=', 'Content-Length': '0'} 2022-09-03T06:50:25.807 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e656e3260b5dd963-006312f931-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.808 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:25.810 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:25.811 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.812 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.813 DEBUG:boto:Method: HEAD 2022-09-03T06:50:25.815 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.816 DEBUG:boto:Data: 2022-09-03T06:50:25.817 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.820 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:25.821 DEBUG:boto:Port: 8000 2022-09-03T06:50:25.822 DEBUG:boto:Params: {} 2022-09-03T06:50:25.823 DEBUG:boto:Token: None 2022-09-03T06:50:25.824 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.825 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:CKzyIcbDr+RU/gs9O6hBt+oHE6o= 2022-09-03T06:50:25.827 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:CKzyIcbDr+RU/gs9O6hBt+oHE6o=', 'Content-Length': '0'} 2022-09-03T06:50:25.830 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007aa34b2582a9af60-006312f931-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.831 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:50:25.832 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:25.832 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:25.833 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.833 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.834 DEBUG:boto:Method: GET 2022-09-03T06:50:25.835 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.835 DEBUG:boto:Data: 2022-09-03T06:50:25.836 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.836 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:25.837 DEBUG:boto:Port: 8001 2022-09-03T06:50:25.837 DEBUG:boto:Params: {} 2022-09-03T06:50:25.838 DEBUG:boto:Token: None 2022-09-03T06:50:25.842 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.842 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0OlVtLvWKcTtpvS8UxhyGn1eQHw= 2022-09-03T06:50:25.843 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0OlVtLvWKcTtpvS8UxhyGn1eQHw=', 'Content-Length': '0'} 2022-09-03T06:50:25.845 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000683f4866845cab6-006312f931-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.846 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:25.847 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:25.848 DEBUG:boto:path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.849 DEBUG:boto:auth_path=/dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.849 DEBUG:boto:Method: GET 2022-09-03T06:50:25.850 DEBUG:boto:Path: /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.850 DEBUG:boto:Data: 2022-09-03T06:50:25.851 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.852 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:25.852 DEBUG:boto:Port: 8000 2022-09-03T06:50:25.853 DEBUG:boto:Params: {} 2022-09-03T06:50:25.853 DEBUG:boto:Token: None 2022-09-03T06:50:25.854 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj17?versionId=null 2022-09-03T06:50:25.855 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0OlVtLvWKcTtpvS8UxhyGn1eQHw= 2022-09-03T06:50:25.856 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0OlVtLvWKcTtpvS8UxhyGn1eQHw=', 'Content-Length': '0'} 2022-09-03T06:50:25.863 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ac2e8530139ccdad-006312f931-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.864 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:25.864 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:25.865 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:25.865 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:25.866 DEBUG:boto:Method: GET 2022-09-03T06:50:25.867 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:50:25.869 DEBUG:boto:Data: 2022-09-03T06:50:25.870 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.871 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:25.872 DEBUG:boto:Port: 8001 2022-09-03T06:50:25.872 DEBUG:boto:Params: {} 2022-09-03T06:50:25.873 DEBUG:boto:Token: None 2022-09-03T06:50:25.874 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/ 2022-09-03T06:50:25.875 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM= 2022-09-03T06:50:25.876 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM=', 'Content-Length': '0'} 2022-09-03T06:50:25.879 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000bec28962b08c307a-006312f931-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.881 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:25.883 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:25.883 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:25.884 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:25.885 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:25.885 DEBUG:boto:Method: GET 2022-09-03T06:50:25.887 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:50:25.887 DEBUG:boto:Data: 2022-09-03T06:50:25.888 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.889 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:25.889 DEBUG:boto:Port: 8000 2022-09-03T06:50:25.890 DEBUG:boto:Params: {} 2022-09-03T06:50:25.891 DEBUG:boto:Token: None 2022-09-03T06:50:25.892 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/ 2022-09-03T06:50:25.892 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM= 2022-09-03T06:50:25.893 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kk4T3L4r0k3GqFRvBvVdOjG/sTM=', 'Content-Length': '0'} 2022-09-03T06:50:25.897 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b9e146b5024fdb2c-006312f931-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.900 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:25.903 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:25.904 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:25.905 DEBUG:boto:Method: HEAD 2022-09-03T06:50:25.905 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:50:25.907 DEBUG:boto:Data: 2022-09-03T06:50:25.908 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.908 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:25.909 DEBUG:boto:Port: 8001 2022-09-03T06:50:25.909 DEBUG:boto:Params: {} 2022-09-03T06:50:25.910 DEBUG:boto:Token: None 2022-09-03T06:50:25.911 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj17 2022-09-03T06:50:25.911 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mPArJt9SKc6hHYbAH8r9rVjSIO0= 2022-09-03T06:50:25.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mPArJt9SKc6hHYbAH8r9rVjSIO0=', 'Content-Length': '0'} 2022-09-03T06:50:25.914 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a0d7e16a76c318cf-006312f931-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.915 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:25.915 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:25.916 DEBUG:boto:Method: HEAD 2022-09-03T06:50:25.917 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:50:25.917 DEBUG:boto:Data: 2022-09-03T06:50:25.918 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.918 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:25.919 DEBUG:boto:Port: 8000 2022-09-03T06:50:25.919 DEBUG:boto:Params: {} 2022-09-03T06:50:25.920 DEBUG:boto:Token: None 2022-09-03T06:50:25.921 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj17 2022-09-03T06:50:25.921 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:mPArJt9SKc6hHYbAH8r9rVjSIO0= 2022-09-03T06:50:25.922 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:mPArJt9SKc6hHYbAH8r9rVjSIO0=', 'Content-Length': '0'} 2022-09-03T06:50:25.924 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f0c0df35c9fcf0b2-006312f931-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.925 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj17 2022-09-03T06:50:25.926 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:25.927 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:25.928 DEBUG:boto:Method: GET 2022-09-03T06:50:25.930 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:50:25.931 DEBUG:boto:Data: 2022-09-03T06:50:25.931 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.932 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:25.932 DEBUG:boto:Port: 8001 2022-09-03T06:50:25.933 DEBUG:boto:Params: {} 2022-09-03T06:50:25.934 DEBUG:boto:Token: None 2022-09-03T06:50:25.936 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:25 GMT /dsgvcy-75/obj17 2022-09-03T06:50:25.938 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:werPeq2p5+4EJBFM6G7pmNpp4LU= 2022-09-03T06:50:25.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:25 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:werPeq2p5+4EJBFM6G7pmNpp4LU=', 'Content-Length': '0'} 2022-09-03T06:50:25.944 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ceaf84820f8e3812-006312f931-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:25 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:25.988 DEBUG:boto:path=/dsgvcy-75/obj17 2022-09-03T06:50:25.991 DEBUG:boto:auth_path=/dsgvcy-75/obj17 2022-09-03T06:50:25.992 DEBUG:boto:Method: GET 2022-09-03T06:50:25.992 DEBUG:boto:Path: /dsgvcy-75/obj17 2022-09-03T06:50:25.993 DEBUG:boto:Data: 2022-09-03T06:50:25.994 DEBUG:boto:Headers: {} 2022-09-03T06:50:25.995 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:25.997 DEBUG:boto:Port: 8000 2022-09-03T06:50:25.998 DEBUG:boto:Params: {} 2022-09-03T06:50:26.000 DEBUG:boto:Token: None 2022-09-03T06:50:26.001 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj17 2022-09-03T06:50:26.002 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Gd78WGI8sXFSe5omk+lysu83dsU= 2022-09-03T06:50:26.004 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Gd78WGI8sXFSe5omk+lysu83dsU=', 'Content-Length': '0'} 2022-09-03T06:50:26.007 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000080f60c6eaf7e7343-006312f932-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.009 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:26.009 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:26.010 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:26.011 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:26.011 DEBUG:boto:Method: GET 2022-09-03T06:50:26.012 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:50:26.012 DEBUG:boto:Data: 2022-09-03T06:50:26.013 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.014 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:26.014 DEBUG:boto:Port: 8001 2022-09-03T06:50:26.015 DEBUG:boto:Params: {} 2022-09-03T06:50:26.015 DEBUG:boto:Token: None 2022-09-03T06:50:26.016 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/ 2022-09-03T06:50:26.017 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc= 2022-09-03T06:50:26.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc=', 'Content-Length': '0'} 2022-09-03T06:50:26.020 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e60e3bbe48b33d08-006312f932-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.022 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:26.023 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:26.024 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:26.025 DEBUG:boto:path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:26.026 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj17 2022-09-03T06:50:26.027 DEBUG:boto:Method: GET 2022-09-03T06:50:26.028 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj17 2022-09-03T06:50:26.029 DEBUG:boto:Data: 2022-09-03T06:50:26.029 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.030 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:26.031 DEBUG:boto:Port: 8000 2022-09-03T06:50:26.031 DEBUG:boto:Params: {} 2022-09-03T06:50:26.032 DEBUG:boto:Token: None 2022-09-03T06:50:26.032 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/ 2022-09-03T06:50:26.033 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc= 2022-09-03T06:50:26.034 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc=', 'Content-Length': '0'} 2022-09-03T06:50:26.039 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007433d8429ea84a28-006312f932-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.040 DEBUG:boto:b'dsgvcy-75obj171000falseobj172022-09-03T06:37:27.042Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:26.041 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj185 2022-09-03T06:50:26.042 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:26.043 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:26.043 DEBUG:boto:path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.044 DEBUG:boto:auth_path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.044 DEBUG:boto:Method: GET 2022-09-03T06:50:26.045 DEBUG:boto:Path: /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.045 DEBUG:boto:Data: 2022-09-03T06:50:26.046 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.046 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:26.047 DEBUG:boto:Port: 8001 2022-09-03T06:50:26.047 DEBUG:boto:Params: {} 2022-09-03T06:50:26.048 DEBUG:boto:Token: None 2022-09-03T06:50:26.049 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.049 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bCSRK+IfoZFIEyYzDlnB5Gp6He0= 2022-09-03T06:50:26.050 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bCSRK+IfoZFIEyYzDlnB5Gp6He0=', 'Content-Length': '0'} 2022-09-03T06:50:26.052 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b89012fb1375cc6-006312f932-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.053 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:26.054 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:26.054 DEBUG:boto:path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.055 DEBUG:boto:auth_path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.056 DEBUG:boto:Method: GET 2022-09-03T06:50:26.056 DEBUG:boto:Path: /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.057 DEBUG:boto:Data: 2022-09-03T06:50:26.058 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.058 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:26.059 DEBUG:boto:Port: 8000 2022-09-03T06:50:26.059 DEBUG:boto:Params: {} 2022-09-03T06:50:26.060 DEBUG:boto:Token: None 2022-09-03T06:50:26.061 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.061 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bCSRK+IfoZFIEyYzDlnB5Gp6He0= 2022-09-03T06:50:26.062 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bCSRK+IfoZFIEyYzDlnB5Gp6He0=', 'Content-Length': '0'} 2022-09-03T06:50:26.065 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e78c7bead20890ed-006312f932-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.106 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:26.107 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:26.108 DEBUG:boto:path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.108 DEBUG:boto:auth_path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.109 DEBUG:boto:Method: HEAD 2022-09-03T06:50:26.110 DEBUG:boto:Path: /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.110 DEBUG:boto:Data: 2022-09-03T06:50:26.111 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.111 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:26.112 DEBUG:boto:Port: 8001 2022-09-03T06:50:26.113 DEBUG:boto:Params: {} 2022-09-03T06:50:26.113 DEBUG:boto:Token: None 2022-09-03T06:50:26.114 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.114 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3zC6r405Ui7NV5C2rRchYnVuU2M= 2022-09-03T06:50:26.115 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3zC6r405Ui7NV5C2rRchYnVuU2M=', 'Content-Length': '0'} 2022-09-03T06:50:26.117 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000026c66d703f5c9d71-006312f932-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.119 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:26.121 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:26.123 DEBUG:boto:path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.124 DEBUG:boto:auth_path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.125 DEBUG:boto:Method: HEAD 2022-09-03T06:50:26.126 DEBUG:boto:Path: /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.127 DEBUG:boto:Data: 2022-09-03T06:50:26.128 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.128 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:26.129 DEBUG:boto:Port: 8000 2022-09-03T06:50:26.130 DEBUG:boto:Params: {} 2022-09-03T06:50:26.131 DEBUG:boto:Token: None 2022-09-03T06:50:26.133 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.134 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3zC6r405Ui7NV5C2rRchYnVuU2M= 2022-09-03T06:50:26.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3zC6r405Ui7NV5C2rRchYnVuU2M=', 'Content-Length': '0'} 2022-09-03T06:50:26.138 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000042fa48ee6fac2a17-006312f932-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.139 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj185 2022-09-03T06:50:26.140 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:26.141 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:26.143 DEBUG:boto:path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.145 DEBUG:boto:auth_path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.146 DEBUG:boto:Method: GET 2022-09-03T06:50:26.147 DEBUG:boto:Path: /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.148 DEBUG:boto:Data: 2022-09-03T06:50:26.149 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.150 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:26.151 DEBUG:boto:Port: 8001 2022-09-03T06:50:26.152 DEBUG:boto:Params: {} 2022-09-03T06:50:26.153 DEBUG:boto:Token: None 2022-09-03T06:50:26.155 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.156 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bCSRK+IfoZFIEyYzDlnB5Gp6He0= 2022-09-03T06:50:26.157 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bCSRK+IfoZFIEyYzDlnB5Gp6He0=', 'Content-Length': '0'} 2022-09-03T06:50:26.160 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000000391934486726e8-006312f932-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.162 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:26.163 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:26.163 DEBUG:boto:path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.164 DEBUG:boto:auth_path=/dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.164 DEBUG:boto:Method: GET 2022-09-03T06:50:26.165 DEBUG:boto:Path: /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.166 DEBUG:boto:Data: 2022-09-03T06:50:26.167 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.167 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:26.168 DEBUG:boto:Port: 8000 2022-09-03T06:50:26.170 DEBUG:boto:Params: {} 2022-09-03T06:50:26.171 DEBUG:boto:Token: None 2022-09-03T06:50:26.171 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj185?versionId=null 2022-09-03T06:50:26.172 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bCSRK+IfoZFIEyYzDlnB5Gp6He0= 2022-09-03T06:50:26.173 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bCSRK+IfoZFIEyYzDlnB5Gp6He0=', 'Content-Length': '0'} 2022-09-03T06:50:26.177 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dff38aeadc7fdf22-006312f932-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.178 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:26.179 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:26.179 DEBUG:boto:path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:26.180 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:26.181 DEBUG:boto:Method: GET 2022-09-03T06:50:26.181 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj185 2022-09-03T06:50:26.182 DEBUG:boto:Data: 2022-09-03T06:50:26.183 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.183 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:26.184 DEBUG:boto:Port: 8001 2022-09-03T06:50:26.184 DEBUG:boto:Params: {} 2022-09-03T06:50:26.185 DEBUG:boto:Token: None 2022-09-03T06:50:26.186 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/ 2022-09-03T06:50:26.186 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc= 2022-09-03T06:50:26.187 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc=', 'Content-Length': '0'} 2022-09-03T06:50:26.190 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000052bfdc41992c8ae8-006312f932-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.191 DEBUG:boto:b'dsgvcy-75obj1851000falseobj1852022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:26.192 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:26.193 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:26.194 DEBUG:boto:path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:26.194 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:26.195 DEBUG:boto:Method: GET 2022-09-03T06:50:26.196 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj185 2022-09-03T06:50:26.196 DEBUG:boto:Data: 2022-09-03T06:50:26.197 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.198 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:26.199 DEBUG:boto:Port: 8000 2022-09-03T06:50:26.200 DEBUG:boto:Params: {} 2022-09-03T06:50:26.201 DEBUG:boto:Token: None 2022-09-03T06:50:26.202 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/ 2022-09-03T06:50:26.203 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc= 2022-09-03T06:50:26.203 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc=', 'Content-Length': '0'} 2022-09-03T06:50:26.217 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003a063bb8765d6ba5-006312f932-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.218 DEBUG:boto:b'dsgvcy-75obj1851000falseobj1852022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:26.219 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:26.220 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:26.220 DEBUG:boto:Method: HEAD 2022-09-03T06:50:26.221 DEBUG:boto:Path: /dsgvcy-75/obj185 2022-09-03T06:50:26.222 DEBUG:boto:Data: 2022-09-03T06:50:26.222 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.223 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:26.224 DEBUG:boto:Port: 8001 2022-09-03T06:50:26.225 DEBUG:boto:Params: {} 2022-09-03T06:50:26.226 DEBUG:boto:Token: None 2022-09-03T06:50:26.227 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj185 2022-09-03T06:50:26.228 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ETOr2oL8MHmXT6BoAg4TWbOCYbQ= 2022-09-03T06:50:26.228 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ETOr2oL8MHmXT6BoAg4TWbOCYbQ=', 'Content-Length': '0'} 2022-09-03T06:50:26.231 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000049e625b0a1d188ea-006312f932-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.231 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:26.232 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:26.233 DEBUG:boto:Method: HEAD 2022-09-03T06:50:26.233 DEBUG:boto:Path: /dsgvcy-75/obj185 2022-09-03T06:50:26.234 DEBUG:boto:Data: 2022-09-03T06:50:26.235 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.235 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:26.238 DEBUG:boto:Port: 8000 2022-09-03T06:50:26.238 DEBUG:boto:Params: {} 2022-09-03T06:50:26.240 DEBUG:boto:Token: None 2022-09-03T06:50:26.241 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj185 2022-09-03T06:50:26.243 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ETOr2oL8MHmXT6BoAg4TWbOCYbQ= 2022-09-03T06:50:26.244 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ETOr2oL8MHmXT6BoAg4TWbOCYbQ=', 'Content-Length': '0'} 2022-09-03T06:50:26.249 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000012906dbcc5e7c453-006312f932-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.251 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj185 2022-09-03T06:50:26.252 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:26.253 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:26.254 DEBUG:boto:Method: GET 2022-09-03T06:50:26.255 DEBUG:boto:Path: /dsgvcy-75/obj185 2022-09-03T06:50:26.256 DEBUG:boto:Data: 2022-09-03T06:50:26.257 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.258 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:26.259 DEBUG:boto:Port: 8001 2022-09-03T06:50:26.260 DEBUG:boto:Params: {} 2022-09-03T06:50:26.261 DEBUG:boto:Token: None 2022-09-03T06:50:26.263 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj185 2022-09-03T06:50:26.264 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E6FRdBL/4GhBU+kxgqSId/JsBg0= 2022-09-03T06:50:26.265 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E6FRdBL/4GhBU+kxgqSId/JsBg0=', 'Content-Length': '0'} 2022-09-03T06:50:26.271 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ffa7ab63332f4da2-006312f932-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.273 DEBUG:boto:path=/dsgvcy-75/obj185 2022-09-03T06:50:26.274 DEBUG:boto:auth_path=/dsgvcy-75/obj185 2022-09-03T06:50:26.275 DEBUG:boto:Method: GET 2022-09-03T06:50:26.276 DEBUG:boto:Path: /dsgvcy-75/obj185 2022-09-03T06:50:26.277 DEBUG:boto:Data: 2022-09-03T06:50:26.278 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.279 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:26.280 DEBUG:boto:Port: 8000 2022-09-03T06:50:26.281 DEBUG:boto:Params: {} 2022-09-03T06:50:26.282 DEBUG:boto:Token: None 2022-09-03T06:50:26.283 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj185 2022-09-03T06:50:26.284 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:E6FRdBL/4GhBU+kxgqSId/JsBg0= 2022-09-03T06:50:26.285 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:E6FRdBL/4GhBU+kxgqSId/JsBg0=', 'Content-Length': '0'} 2022-09-03T06:50:26.289 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000544d753d8163b958-006312f932-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.292 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:26.295 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:26.296 DEBUG:boto:path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:26.297 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:26.298 DEBUG:boto:Method: GET 2022-09-03T06:50:26.299 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj185 2022-09-03T06:50:26.300 DEBUG:boto:Data: 2022-09-03T06:50:26.301 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.302 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:26.303 DEBUG:boto:Port: 8001 2022-09-03T06:50:26.304 DEBUG:boto:Params: {} 2022-09-03T06:50:26.305 DEBUG:boto:Token: None 2022-09-03T06:50:26.306 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/ 2022-09-03T06:50:26.307 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc= 2022-09-03T06:50:26.309 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc=', 'Content-Length': '0'} 2022-09-03T06:50:26.315 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b4782f3f2aedab83-006312f932-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.354 DEBUG:boto:b'dsgvcy-75obj1851000falseobj1852022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:26.356 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:26.357 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:26.357 DEBUG:boto:path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:26.359 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj185 2022-09-03T06:50:26.359 DEBUG:boto:Method: GET 2022-09-03T06:50:26.360 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj185 2022-09-03T06:50:26.361 DEBUG:boto:Data: 2022-09-03T06:50:26.361 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.362 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:26.362 DEBUG:boto:Port: 8000 2022-09-03T06:50:26.363 DEBUG:boto:Params: {} 2022-09-03T06:50:26.363 DEBUG:boto:Token: None 2022-09-03T06:50:26.364 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/ 2022-09-03T06:50:26.365 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc= 2022-09-03T06:50:26.365 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc=', 'Content-Length': '0'} 2022-09-03T06:50:26.371 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007ef2aa2cdf3c0277-006312f932-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.372 DEBUG:boto:b'dsgvcy-75obj1851000falseobj1852022-09-03T06:47:49.731Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:26.373 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:50:26.374 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:26.375 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:26.375 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.376 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.377 DEBUG:boto:Method: GET 2022-09-03T06:50:26.377 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.378 DEBUG:boto:Data: 2022-09-03T06:50:26.379 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.379 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:26.380 DEBUG:boto:Port: 8001 2022-09-03T06:50:26.380 DEBUG:boto:Params: {} 2022-09-03T06:50:26.381 DEBUG:boto:Token: None 2022-09-03T06:50:26.382 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.383 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bb6NMqQmJOoWAn2SA5fHWxW/7SU= 2022-09-03T06:50:26.383 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bb6NMqQmJOoWAn2SA5fHWxW/7SU=', 'Content-Length': '0'} 2022-09-03T06:50:26.386 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000063bb2a9d1aea56ec-006312f932-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.387 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:26.388 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:26.388 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.389 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.391 DEBUG:boto:Method: GET 2022-09-03T06:50:26.391 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.392 DEBUG:boto:Data: 2022-09-03T06:50:26.393 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.393 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:26.407 DEBUG:boto:Port: 8000 2022-09-03T06:50:26.408 DEBUG:boto:Params: {} 2022-09-03T06:50:26.409 DEBUG:boto:Token: None 2022-09-03T06:50:26.410 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.411 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bb6NMqQmJOoWAn2SA5fHWxW/7SU= 2022-09-03T06:50:26.412 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bb6NMqQmJOoWAn2SA5fHWxW/7SU=', 'Content-Length': '0'} 2022-09-03T06:50:26.417 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005338c92f1d5e908b-006312f932-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.460 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:26.462 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:26.463 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.463 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.464 DEBUG:boto:Method: HEAD 2022-09-03T06:50:26.464 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.465 DEBUG:boto:Data: 2022-09-03T06:50:26.465 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.466 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:26.467 DEBUG:boto:Port: 8001 2022-09-03T06:50:26.467 DEBUG:boto:Params: {} 2022-09-03T06:50:26.468 DEBUG:boto:Token: None 2022-09-03T06:50:26.468 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.469 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kmE7c3sG0z6NYKAAF3Z7KCDYWxk= 2022-09-03T06:50:26.469 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kmE7c3sG0z6NYKAAF3Z7KCDYWxk=', 'Content-Length': '0'} 2022-09-03T06:50:26.472 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009acc6c20da6e8edd-006312f932-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.473 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:26.473 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:26.474 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.474 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.475 DEBUG:boto:Method: HEAD 2022-09-03T06:50:26.475 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.476 DEBUG:boto:Data: 2022-09-03T06:50:26.476 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.477 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:26.477 DEBUG:boto:Port: 8000 2022-09-03T06:50:26.478 DEBUG:boto:Params: {} 2022-09-03T06:50:26.478 DEBUG:boto:Token: None 2022-09-03T06:50:26.479 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.479 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kmE7c3sG0z6NYKAAF3Z7KCDYWxk= 2022-09-03T06:50:26.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kmE7c3sG0z6NYKAAF3Z7KCDYWxk=', 'Content-Length': '0'} 2022-09-03T06:50:26.482 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000068d711325bf9423b-006312f932-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.483 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:50:26.483 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:26.484 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:26.485 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.485 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.486 DEBUG:boto:Method: GET 2022-09-03T06:50:26.486 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.487 DEBUG:boto:Data: 2022-09-03T06:50:26.487 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.488 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:26.488 DEBUG:boto:Port: 8001 2022-09-03T06:50:26.489 DEBUG:boto:Params: {} 2022-09-03T06:50:26.490 DEBUG:boto:Token: None 2022-09-03T06:50:26.490 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.491 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bb6NMqQmJOoWAn2SA5fHWxW/7SU= 2022-09-03T06:50:26.492 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bb6NMqQmJOoWAn2SA5fHWxW/7SU=', 'Content-Length': '0'} 2022-09-03T06:50:26.495 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007c0fbd89ea8c2f63-006312f932-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.496 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:26.497 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:26.498 DEBUG:boto:path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.499 DEBUG:boto:auth_path=/dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.500 DEBUG:boto:Method: GET 2022-09-03T06:50:26.501 DEBUG:boto:Path: /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.502 DEBUG:boto:Data: 2022-09-03T06:50:26.503 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.504 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:26.505 DEBUG:boto:Port: 8000 2022-09-03T06:50:26.507 DEBUG:boto:Params: {} 2022-09-03T06:50:26.508 DEBUG:boto:Token: None 2022-09-03T06:50:26.509 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj19?versionId=null 2022-09-03T06:50:26.510 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bb6NMqQmJOoWAn2SA5fHWxW/7SU= 2022-09-03T06:50:26.511 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bb6NMqQmJOoWAn2SA5fHWxW/7SU=', 'Content-Length': '0'} 2022-09-03T06:50:26.514 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000dab70457e023ba45-006312f932-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.516 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:26.517 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:26.518 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:26.519 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:26.520 DEBUG:boto:Method: GET 2022-09-03T06:50:26.522 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:50:26.523 DEBUG:boto:Data: 2022-09-03T06:50:26.524 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.525 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:26.526 DEBUG:boto:Port: 8001 2022-09-03T06:50:26.527 DEBUG:boto:Params: {} 2022-09-03T06:50:26.529 DEBUG:boto:Token: None 2022-09-03T06:50:26.530 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/ 2022-09-03T06:50:26.530 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc= 2022-09-03T06:50:26.531 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc=', 'Content-Length': '0'} 2022-09-03T06:50:26.534 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000015137e3011356124-006312f932-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.535 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:26.538 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:26.539 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:26.540 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:26.541 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:26.542 DEBUG:boto:Method: GET 2022-09-03T06:50:26.542 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:50:26.543 DEBUG:boto:Data: 2022-09-03T06:50:26.543 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.544 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:26.545 DEBUG:boto:Port: 8000 2022-09-03T06:50:26.545 DEBUG:boto:Params: {} 2022-09-03T06:50:26.546 DEBUG:boto:Token: None 2022-09-03T06:50:26.546 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/ 2022-09-03T06:50:26.547 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc= 2022-09-03T06:50:26.548 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc=', 'Content-Length': '0'} 2022-09-03T06:50:26.551 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f7ed8d22b9e78590-006312f932-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.594 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:26.595 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:26.596 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:26.596 DEBUG:boto:Method: HEAD 2022-09-03T06:50:26.597 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:50:26.598 DEBUG:boto:Data: 2022-09-03T06:50:26.598 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.599 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:26.599 DEBUG:boto:Port: 8001 2022-09-03T06:50:26.600 DEBUG:boto:Params: {} 2022-09-03T06:50:26.600 DEBUG:boto:Token: None 2022-09-03T06:50:26.601 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj19 2022-09-03T06:50:26.601 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GFXixtiyRRXvbqoZ7kZle/spOf8= 2022-09-03T06:50:26.602 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GFXixtiyRRXvbqoZ7kZle/spOf8=', 'Content-Length': '0'} 2022-09-03T06:50:26.604 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d863099a3a8e449b-006312f932-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.605 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:26.605 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:26.606 DEBUG:boto:Method: HEAD 2022-09-03T06:50:26.606 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:50:26.607 DEBUG:boto:Data: 2022-09-03T06:50:26.607 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.608 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:26.608 DEBUG:boto:Port: 8000 2022-09-03T06:50:26.609 DEBUG:boto:Params: {} 2022-09-03T06:50:26.609 DEBUG:boto:Token: None 2022-09-03T06:50:26.610 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj19 2022-09-03T06:50:26.611 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GFXixtiyRRXvbqoZ7kZle/spOf8= 2022-09-03T06:50:26.611 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GFXixtiyRRXvbqoZ7kZle/spOf8=', 'Content-Length': '0'} 2022-09-03T06:50:26.613 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c1570cff6ed39583-006312f932-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.614 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj19 2022-09-03T06:50:26.614 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:26.615 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:26.616 DEBUG:boto:Method: GET 2022-09-03T06:50:26.616 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:50:26.617 DEBUG:boto:Data: 2022-09-03T06:50:26.617 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.618 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:26.619 DEBUG:boto:Port: 8001 2022-09-03T06:50:26.619 DEBUG:boto:Params: {} 2022-09-03T06:50:26.620 DEBUG:boto:Token: None 2022-09-03T06:50:26.620 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj19 2022-09-03T06:50:26.621 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8omm7fgl4l1bXkYB1P0LS6jRHtI= 2022-09-03T06:50:26.622 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8omm7fgl4l1bXkYB1P0LS6jRHtI=', 'Content-Length': '0'} 2022-09-03T06:50:26.624 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000093b873f496aa053d-006312f932-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.625 DEBUG:boto:path=/dsgvcy-75/obj19 2022-09-03T06:50:26.625 DEBUG:boto:auth_path=/dsgvcy-75/obj19 2022-09-03T06:50:26.626 DEBUG:boto:Method: GET 2022-09-03T06:50:26.626 DEBUG:boto:Path: /dsgvcy-75/obj19 2022-09-03T06:50:26.627 DEBUG:boto:Data: 2022-09-03T06:50:26.627 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.628 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:26.628 DEBUG:boto:Port: 8000 2022-09-03T06:50:26.629 DEBUG:boto:Params: {} 2022-09-03T06:50:26.630 DEBUG:boto:Token: None 2022-09-03T06:50:26.630 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj19 2022-09-03T06:50:26.631 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8omm7fgl4l1bXkYB1P0LS6jRHtI= 2022-09-03T06:50:26.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8omm7fgl4l1bXkYB1P0LS6jRHtI=', 'Content-Length': '0'} 2022-09-03T06:50:26.634 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008bc9ab661b031869-006312f932-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.635 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:26.636 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:26.636 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:26.637 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:26.637 DEBUG:boto:Method: GET 2022-09-03T06:50:26.638 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:50:26.639 DEBUG:boto:Data: 2022-09-03T06:50:26.639 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.640 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:26.640 DEBUG:boto:Port: 8001 2022-09-03T06:50:26.641 DEBUG:boto:Params: {} 2022-09-03T06:50:26.641 DEBUG:boto:Token: None 2022-09-03T06:50:26.642 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/ 2022-09-03T06:50:26.643 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc= 2022-09-03T06:50:26.643 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc=', 'Content-Length': '0'} 2022-09-03T06:50:26.646 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000548d866b5cb33ae6-006312f932-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.647 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:26.648 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:26.649 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:26.649 DEBUG:boto:path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:26.650 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj19 2022-09-03T06:50:26.650 DEBUG:boto:Method: GET 2022-09-03T06:50:26.651 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj19 2022-09-03T06:50:26.651 DEBUG:boto:Data: 2022-09-03T06:50:26.652 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.653 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:26.653 DEBUG:boto:Port: 8000 2022-09-03T06:50:26.654 DEBUG:boto:Params: {} 2022-09-03T06:50:26.655 DEBUG:boto:Token: None 2022-09-03T06:50:26.655 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/ 2022-09-03T06:50:26.656 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc= 2022-09-03T06:50:26.657 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc=', 'Content-Length': '0'} 2022-09-03T06:50:26.660 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003b469a0143380908-006312f932-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.661 DEBUG:boto:b'dsgvcy-75obj191000falseobj192022-09-03T06:40:53.130Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:26.662 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:50:26.662 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:26.663 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:26.664 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.665 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.666 DEBUG:boto:Method: GET 2022-09-03T06:50:26.667 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.668 DEBUG:boto:Data: 2022-09-03T06:50:26.669 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.670 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:26.672 DEBUG:boto:Port: 8001 2022-09-03T06:50:26.673 DEBUG:boto:Params: {} 2022-09-03T06:50:26.673 DEBUG:boto:Token: None 2022-09-03T06:50:26.674 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.675 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+r6h0hz7UCF6KpiRxCrf5Vj/DZw= 2022-09-03T06:50:26.676 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+r6h0hz7UCF6KpiRxCrf5Vj/DZw=', 'Content-Length': '0'} 2022-09-03T06:50:26.678 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f72b9318bee21fe5-006312f932-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.726 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:26.728 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:26.729 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.730 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.731 DEBUG:boto:Method: GET 2022-09-03T06:50:26.732 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.733 DEBUG:boto:Data: 2022-09-03T06:50:26.734 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.735 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:26.736 DEBUG:boto:Port: 8000 2022-09-03T06:50:26.737 DEBUG:boto:Params: {} 2022-09-03T06:50:26.738 DEBUG:boto:Token: None 2022-09-03T06:50:26.739 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.740 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+r6h0hz7UCF6KpiRxCrf5Vj/DZw= 2022-09-03T06:50:26.741 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+r6h0hz7UCF6KpiRxCrf5Vj/DZw=', 'Content-Length': '0'} 2022-09-03T06:50:26.744 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000edd658556f2d0f4f-006312f932-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.746 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:26.746 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:26.747 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.747 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.748 DEBUG:boto:Method: HEAD 2022-09-03T06:50:26.749 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.749 DEBUG:boto:Data: 2022-09-03T06:50:26.750 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.750 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:26.751 DEBUG:boto:Port: 8001 2022-09-03T06:50:26.751 DEBUG:boto:Params: {} 2022-09-03T06:50:26.752 DEBUG:boto:Token: None 2022-09-03T06:50:26.753 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.753 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/DJ23FDvAXGbtVc5ZMET1eK1Jh8= 2022-09-03T06:50:26.754 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/DJ23FDvAXGbtVc5ZMET1eK1Jh8=', 'Content-Length': '0'} 2022-09-03T06:50:26.759 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b88264c18f42eee1-006312f932-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.760 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:26.761 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:26.761 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.762 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.762 DEBUG:boto:Method: HEAD 2022-09-03T06:50:26.763 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.763 DEBUG:boto:Data: 2022-09-03T06:50:26.764 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.764 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:26.765 DEBUG:boto:Port: 8000 2022-09-03T06:50:26.765 DEBUG:boto:Params: {} 2022-09-03T06:50:26.766 DEBUG:boto:Token: None 2022-09-03T06:50:26.766 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.767 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/DJ23FDvAXGbtVc5ZMET1eK1Jh8= 2022-09-03T06:50:26.768 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/DJ23FDvAXGbtVc5ZMET1eK1Jh8=', 'Content-Length': '0'} 2022-09-03T06:50:26.770 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b2bc9a022d564f1a-006312f932-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.770 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:50:26.771 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:26.772 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:26.772 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.773 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.773 DEBUG:boto:Method: GET 2022-09-03T06:50:26.774 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.774 DEBUG:boto:Data: 2022-09-03T06:50:26.775 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.775 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:26.776 DEBUG:boto:Port: 8001 2022-09-03T06:50:26.776 DEBUG:boto:Params: {} 2022-09-03T06:50:26.777 DEBUG:boto:Token: None 2022-09-03T06:50:26.778 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.778 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+r6h0hz7UCF6KpiRxCrf5Vj/DZw= 2022-09-03T06:50:26.779 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+r6h0hz7UCF6KpiRxCrf5Vj/DZw=', 'Content-Length': '0'} 2022-09-03T06:50:26.781 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002e37615f32d67529-006312f932-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.782 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:26.783 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:26.783 DEBUG:boto:path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.784 DEBUG:boto:auth_path=/dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.784 DEBUG:boto:Method: GET 2022-09-03T06:50:26.785 DEBUG:boto:Path: /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.785 DEBUG:boto:Data: 2022-09-03T06:50:26.786 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.786 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:26.787 DEBUG:boto:Port: 8000 2022-09-03T06:50:26.787 DEBUG:boto:Params: {} 2022-09-03T06:50:26.788 DEBUG:boto:Token: None 2022-09-03T06:50:26.789 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj22?versionId=null 2022-09-03T06:50:26.789 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:+r6h0hz7UCF6KpiRxCrf5Vj/DZw= 2022-09-03T06:50:26.790 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:+r6h0hz7UCF6KpiRxCrf5Vj/DZw=', 'Content-Length': '0'} 2022-09-03T06:50:26.792 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000090d0a7c2a22b65ce-006312f932-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.793 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:26.794 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:26.794 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:26.795 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:26.795 DEBUG:boto:Method: GET 2022-09-03T06:50:26.796 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:50:26.796 DEBUG:boto:Data: 2022-09-03T06:50:26.797 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.797 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:26.798 DEBUG:boto:Port: 8001 2022-09-03T06:50:26.798 DEBUG:boto:Params: {} 2022-09-03T06:50:26.799 DEBUG:boto:Token: None 2022-09-03T06:50:26.800 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/ 2022-09-03T06:50:26.800 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc= 2022-09-03T06:50:26.801 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc=', 'Content-Length': '0'} 2022-09-03T06:50:26.804 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000083c2a8ff32c1a967-006312f932-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.805 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:26.806 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:26.806 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:26.807 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:26.807 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:26.808 DEBUG:boto:Method: GET 2022-09-03T06:50:26.808 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:50:26.809 DEBUG:boto:Data: 2022-09-03T06:50:26.809 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.810 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:26.811 DEBUG:boto:Port: 8000 2022-09-03T06:50:26.811 DEBUG:boto:Params: {} 2022-09-03T06:50:26.812 DEBUG:boto:Token: None 2022-09-03T06:50:26.812 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/ 2022-09-03T06:50:26.813 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc= 2022-09-03T06:50:26.814 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc=', 'Content-Length': '0'} 2022-09-03T06:50:26.816 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000312c70f774a803c7-006312f932-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.817 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:26.818 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:26.819 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:26.819 DEBUG:boto:Method: HEAD 2022-09-03T06:50:26.820 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:50:26.820 DEBUG:boto:Data: 2022-09-03T06:50:26.821 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.821 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:26.822 DEBUG:boto:Port: 8001 2022-09-03T06:50:26.823 DEBUG:boto:Params: {} 2022-09-03T06:50:26.823 DEBUG:boto:Token: None 2022-09-03T06:50:26.824 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj22 2022-09-03T06:50:26.825 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vrjr9pfs5qeGLFp3PWnXDUBCMA8= 2022-09-03T06:50:26.825 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vrjr9pfs5qeGLFp3PWnXDUBCMA8=', 'Content-Length': '0'} 2022-09-03T06:50:26.827 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c1fadd2871b2bd05-006312f932-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.828 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:26.829 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:26.829 DEBUG:boto:Method: HEAD 2022-09-03T06:50:26.830 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:50:26.830 DEBUG:boto:Data: 2022-09-03T06:50:26.831 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.832 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:26.832 DEBUG:boto:Port: 8000 2022-09-03T06:50:26.833 DEBUG:boto:Params: {} 2022-09-03T06:50:26.833 DEBUG:boto:Token: None 2022-09-03T06:50:26.834 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj22 2022-09-03T06:50:26.835 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Vrjr9pfs5qeGLFp3PWnXDUBCMA8= 2022-09-03T06:50:26.836 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Vrjr9pfs5qeGLFp3PWnXDUBCMA8=', 'Content-Length': '0'} 2022-09-03T06:50:26.838 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e74a60a3eaf79ac9-006312f932-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.839 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj22 2022-09-03T06:50:26.841 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:26.842 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:26.843 DEBUG:boto:Method: GET 2022-09-03T06:50:26.844 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:50:26.845 DEBUG:boto:Data: 2022-09-03T06:50:26.846 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.847 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:26.848 DEBUG:boto:Port: 8001 2022-09-03T06:50:26.849 DEBUG:boto:Params: {} 2022-09-03T06:50:26.851 DEBUG:boto:Token: None 2022-09-03T06:50:26.852 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj22 2022-09-03T06:50:26.853 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B6xpue2td3+EP0v9YC+DfO0Hm9g= 2022-09-03T06:50:26.854 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B6xpue2td3+EP0v9YC+DfO0Hm9g=', 'Content-Length': '0'} 2022-09-03T06:50:26.857 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f902166c0c1790c5-006312f932-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.858 DEBUG:boto:path=/dsgvcy-75/obj22 2022-09-03T06:50:26.859 DEBUG:boto:auth_path=/dsgvcy-75/obj22 2022-09-03T06:50:26.860 DEBUG:boto:Method: GET 2022-09-03T06:50:26.861 DEBUG:boto:Path: /dsgvcy-75/obj22 2022-09-03T06:50:26.862 DEBUG:boto:Data: 2022-09-03T06:50:26.863 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.864 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:26.866 DEBUG:boto:Port: 8000 2022-09-03T06:50:26.866 DEBUG:boto:Params: {} 2022-09-03T06:50:26.867 DEBUG:boto:Token: None 2022-09-03T06:50:26.868 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj22 2022-09-03T06:50:26.868 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:B6xpue2td3+EP0v9YC+DfO0Hm9g= 2022-09-03T06:50:26.869 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:B6xpue2td3+EP0v9YC+DfO0Hm9g=', 'Content-Length': '0'} 2022-09-03T06:50:26.871 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000027dd5ea1cd6d7d8-006312f932-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.918 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:26.919 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:26.920 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:26.920 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:26.921 DEBUG:boto:Method: GET 2022-09-03T06:50:26.921 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:50:26.922 DEBUG:boto:Data: 2022-09-03T06:50:26.923 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.923 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:26.924 DEBUG:boto:Port: 8001 2022-09-03T06:50:26.924 DEBUG:boto:Params: {} 2022-09-03T06:50:26.925 DEBUG:boto:Token: None 2022-09-03T06:50:26.926 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/ 2022-09-03T06:50:26.927 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc= 2022-09-03T06:50:26.927 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc=', 'Content-Length': '0'} 2022-09-03T06:50:26.930 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ae1de82022703909-006312f932-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.931 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:26.932 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:26.933 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:26.933 DEBUG:boto:path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:26.934 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj22 2022-09-03T06:50:26.934 DEBUG:boto:Method: GET 2022-09-03T06:50:26.935 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj22 2022-09-03T06:50:26.935 DEBUG:boto:Data: 2022-09-03T06:50:26.936 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.937 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:26.937 DEBUG:boto:Port: 8000 2022-09-03T06:50:26.938 DEBUG:boto:Params: {} 2022-09-03T06:50:26.939 DEBUG:boto:Token: None 2022-09-03T06:50:26.939 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/ 2022-09-03T06:50:26.940 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc= 2022-09-03T06:50:26.941 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ikz0SfeyErJeQvHbCut+/0+9Hoc=', 'Content-Length': '0'} 2022-09-03T06:50:26.943 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008e140af5536928be-006312f932-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.944 DEBUG:boto:b'dsgvcy-75obj221000falseobj222022-09-03T06:36:05.723Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:26.945 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:50:26.945 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:26.946 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:26.947 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:26.947 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:26.948 DEBUG:boto:Method: GET 2022-09-03T06:50:26.948 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:26.949 DEBUG:boto:Data: 2022-09-03T06:50:26.949 DEBUG:boto:Headers: {} 2022-09-03T06:50:26.950 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:26.951 DEBUG:boto:Port: 8001 2022-09-03T06:50:26.951 DEBUG:boto:Params: {} 2022-09-03T06:50:26.952 DEBUG:boto:Token: None 2022-09-03T06:50:26.952 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:26 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:26.953 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bTMwY7e5gF9/h37eCgsfcIqAYAg= 2022-09-03T06:50:26.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:26 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bTMwY7e5gF9/h37eCgsfcIqAYAg=', 'Content-Length': '0'} 2022-09-03T06:50:26.956 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000035a0e319d4b8add2-006312f932-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:26 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:26.998 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:26.999 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:27.000 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:27.000 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:27.001 DEBUG:boto:Method: GET 2022-09-03T06:50:27.001 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:27.002 DEBUG:boto:Data: 2022-09-03T06:50:27.003 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.003 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.004 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.004 DEBUG:boto:Params: {} 2022-09-03T06:50:27.005 DEBUG:boto:Token: None 2022-09-03T06:50:27.006 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:27.006 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4MmrgzW1HI/iwl0G0jAcG4tKmfQ= 2022-09-03T06:50:27.007 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4MmrgzW1HI/iwl0G0jAcG4tKmfQ=', 'Content-Length': '0'} 2022-09-03T06:50:27.010 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ed0467cbc466e893-006312f933-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.010 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:27.011 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:27.012 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:27.012 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:27.013 DEBUG:boto:Method: HEAD 2022-09-03T06:50:27.013 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:27.014 DEBUG:boto:Data: 2022-09-03T06:50:27.014 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.015 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.015 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.016 DEBUG:boto:Params: {} 2022-09-03T06:50:27.016 DEBUG:boto:Token: None 2022-09-03T06:50:27.017 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:27.018 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pBolt74EbWmh3lLWWfe7Um+qVNA= 2022-09-03T06:50:27.018 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pBolt74EbWmh3lLWWfe7Um+qVNA=', 'Content-Length': '0'} 2022-09-03T06:50:27.021 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bfe298156b55c8ba-006312f933-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.022 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:27.022 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:27.023 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:27.023 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:27.024 DEBUG:boto:Method: HEAD 2022-09-03T06:50:27.024 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:27.025 DEBUG:boto:Data: 2022-09-03T06:50:27.026 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.026 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.027 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.027 DEBUG:boto:Params: {} 2022-09-03T06:50:27.028 DEBUG:boto:Token: None 2022-09-03T06:50:27.028 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:27.029 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pBolt74EbWmh3lLWWfe7Um+qVNA= 2022-09-03T06:50:27.030 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pBolt74EbWmh3lLWWfe7Um+qVNA=', 'Content-Length': '0'} 2022-09-03T06:50:27.032 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b9f9e62423040001-006312f933-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.033 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:50:27.033 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:27.034 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:27.034 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:27.035 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:27.036 DEBUG:boto:Method: GET 2022-09-03T06:50:27.036 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:27.037 DEBUG:boto:Data: 2022-09-03T06:50:27.037 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.038 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.039 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.041 DEBUG:boto:Params: {} 2022-09-03T06:50:27.042 DEBUG:boto:Token: None 2022-09-03T06:50:27.043 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:27.044 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4MmrgzW1HI/iwl0G0jAcG4tKmfQ= 2022-09-03T06:50:27.046 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4MmrgzW1HI/iwl0G0jAcG4tKmfQ=', 'Content-Length': '0'} 2022-09-03T06:50:27.049 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000032b394170cb14af3-006312f933-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.050 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:27.051 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:27.052 DEBUG:boto:path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:27.053 DEBUG:boto:auth_path=/dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:27.055 DEBUG:boto:Method: GET 2022-09-03T06:50:27.056 DEBUG:boto:Path: /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:27.056 DEBUG:boto:Data: 2022-09-03T06:50:27.057 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.058 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.058 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.059 DEBUG:boto:Params: {} 2022-09-03T06:50:27.059 DEBUG:boto:Token: None 2022-09-03T06:50:27.060 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj23?versionId=null 2022-09-03T06:50:27.061 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4MmrgzW1HI/iwl0G0jAcG4tKmfQ= 2022-09-03T06:50:27.062 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4MmrgzW1HI/iwl0G0jAcG4tKmfQ=', 'Content-Length': '0'} 2022-09-03T06:50:27.064 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006265bef382217a0f-006312f933-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.065 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:27.065 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:27.066 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:27.066 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:27.067 DEBUG:boto:Method: GET 2022-09-03T06:50:27.067 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:50:27.068 DEBUG:boto:Data: 2022-09-03T06:50:27.069 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.069 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.070 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.070 DEBUG:boto:Params: {} 2022-09-03T06:50:27.071 DEBUG:boto:Token: None 2022-09-03T06:50:27.072 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/ 2022-09-03T06:50:27.072 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic= 2022-09-03T06:50:27.073 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic=', 'Content-Length': '0'} 2022-09-03T06:50:27.076 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000014aa4086183b8f94-006312f933-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.077 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:27.078 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:27.078 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:27.079 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:27.080 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:27.080 DEBUG:boto:Method: GET 2022-09-03T06:50:27.081 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:50:27.082 DEBUG:boto:Data: 2022-09-03T06:50:27.082 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.083 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.084 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.084 DEBUG:boto:Params: {} 2022-09-03T06:50:27.085 DEBUG:boto:Token: None 2022-09-03T06:50:27.086 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/ 2022-09-03T06:50:27.086 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic= 2022-09-03T06:50:27.087 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic=', 'Content-Length': '0'} 2022-09-03T06:50:27.090 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000659db17011634ad9-006312f933-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.091 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:27.091 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:27.092 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:27.093 DEBUG:boto:Method: HEAD 2022-09-03T06:50:27.093 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:50:27.094 DEBUG:boto:Data: 2022-09-03T06:50:27.095 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.095 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.096 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.096 DEBUG:boto:Params: {} 2022-09-03T06:50:27.097 DEBUG:boto:Token: None 2022-09-03T06:50:27.098 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj23 2022-09-03T06:50:27.099 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LRSkHmYyaq7Lzdvw5AV7RapsZLw= 2022-09-03T06:50:27.099 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LRSkHmYyaq7Lzdvw5AV7RapsZLw=', 'Content-Length': '0'} 2022-09-03T06:50:27.101 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009245cb7e7f50efa5-006312f933-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.102 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:27.103 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:27.104 DEBUG:boto:Method: HEAD 2022-09-03T06:50:27.104 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:50:27.105 DEBUG:boto:Data: 2022-09-03T06:50:27.105 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.106 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.107 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.107 DEBUG:boto:Params: {} 2022-09-03T06:50:27.108 DEBUG:boto:Token: None 2022-09-03T06:50:27.109 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj23 2022-09-03T06:50:27.110 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LRSkHmYyaq7Lzdvw5AV7RapsZLw= 2022-09-03T06:50:27.110 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LRSkHmYyaq7Lzdvw5AV7RapsZLw=', 'Content-Length': '0'} 2022-09-03T06:50:27.112 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003977a4dbd5f2f9d6-006312f933-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.113 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj23 2022-09-03T06:50:27.114 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:27.115 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:27.115 DEBUG:boto:Method: GET 2022-09-03T06:50:27.116 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:50:27.117 DEBUG:boto:Data: 2022-09-03T06:50:27.117 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.118 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.119 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.119 DEBUG:boto:Params: {} 2022-09-03T06:50:27.120 DEBUG:boto:Token: None 2022-09-03T06:50:27.121 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj23 2022-09-03T06:50:27.122 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:otbU4aAMKdJaH37mQvhgDOgK+2c= 2022-09-03T06:50:27.123 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:otbU4aAMKdJaH37mQvhgDOgK+2c=', 'Content-Length': '0'} 2022-09-03T06:50:27.126 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001f4733df7a20bed6-006312f933-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.127 DEBUG:boto:path=/dsgvcy-75/obj23 2022-09-03T06:50:27.128 DEBUG:boto:auth_path=/dsgvcy-75/obj23 2022-09-03T06:50:27.129 DEBUG:boto:Method: GET 2022-09-03T06:50:27.130 DEBUG:boto:Path: /dsgvcy-75/obj23 2022-09-03T06:50:27.131 DEBUG:boto:Data: 2022-09-03T06:50:27.132 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.133 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.134 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.135 DEBUG:boto:Params: {} 2022-09-03T06:50:27.136 DEBUG:boto:Token: None 2022-09-03T06:50:27.137 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj23 2022-09-03T06:50:27.138 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:otbU4aAMKdJaH37mQvhgDOgK+2c= 2022-09-03T06:50:27.140 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:otbU4aAMKdJaH37mQvhgDOgK+2c=', 'Content-Length': '0'} 2022-09-03T06:50:27.142 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b269249ee024b727-006312f933-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.143 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:27.144 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:27.145 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:27.145 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:27.146 DEBUG:boto:Method: GET 2022-09-03T06:50:27.147 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:50:27.148 DEBUG:boto:Data: 2022-09-03T06:50:27.149 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.150 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.151 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.152 DEBUG:boto:Params: {} 2022-09-03T06:50:27.153 DEBUG:boto:Token: None 2022-09-03T06:50:27.154 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/ 2022-09-03T06:50:27.155 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic= 2022-09-03T06:50:27.156 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic=', 'Content-Length': '0'} 2022-09-03T06:50:27.160 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ca5139588bd63d18-006312f933-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.161 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:27.162 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:27.163 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:27.164 DEBUG:boto:path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:27.164 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj23 2022-09-03T06:50:27.166 DEBUG:boto:Method: GET 2022-09-03T06:50:27.166 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj23 2022-09-03T06:50:27.167 DEBUG:boto:Data: 2022-09-03T06:50:27.168 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.169 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.170 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.171 DEBUG:boto:Params: {} 2022-09-03T06:50:27.172 DEBUG:boto:Token: None 2022-09-03T06:50:27.173 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/ 2022-09-03T06:50:27.173 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic= 2022-09-03T06:50:27.174 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic=', 'Content-Length': '0'} 2022-09-03T06:50:27.177 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000403f6174813211ac-006312f933-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.179 DEBUG:boto:b'dsgvcy-75obj231000falseobj232022-09-03T06:41:59.633Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:27.180 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:50:27.181 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:27.182 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:27.182 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.183 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.184 DEBUG:boto:Method: GET 2022-09-03T06:50:27.184 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.185 DEBUG:boto:Data: 2022-09-03T06:50:27.188 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.188 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.189 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.190 DEBUG:boto:Params: {} 2022-09-03T06:50:27.191 DEBUG:boto:Token: None 2022-09-03T06:50:27.191 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.192 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FvaX7D6gLbl5LZx3U4BHzB5kXk4= 2022-09-03T06:50:27.193 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FvaX7D6gLbl5LZx3U4BHzB5kXk4=', 'Content-Length': '0'} 2022-09-03T06:50:27.195 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000db30d085d6d44a0e-006312f933-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.196 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:27.197 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:27.197 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.198 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.199 DEBUG:boto:Method: GET 2022-09-03T06:50:27.199 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.200 DEBUG:boto:Data: 2022-09-03T06:50:27.201 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.201 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.202 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.202 DEBUG:boto:Params: {} 2022-09-03T06:50:27.203 DEBUG:boto:Token: None 2022-09-03T06:50:27.204 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.204 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FvaX7D6gLbl5LZx3U4BHzB5kXk4= 2022-09-03T06:50:27.205 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FvaX7D6gLbl5LZx3U4BHzB5kXk4=', 'Content-Length': '0'} 2022-09-03T06:50:27.207 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f50ffd1d85291088-006312f933-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.208 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:27.209 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:27.209 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.210 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.210 DEBUG:boto:Method: HEAD 2022-09-03T06:50:27.211 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.211 DEBUG:boto:Data: 2022-09-03T06:50:27.212 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.213 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.213 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.214 DEBUG:boto:Params: {} 2022-09-03T06:50:27.214 DEBUG:boto:Token: None 2022-09-03T06:50:27.215 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.215 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:szxzqW3bR2UcO/YwR6RRBUXp7tc= 2022-09-03T06:50:27.216 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:szxzqW3bR2UcO/YwR6RRBUXp7tc=', 'Content-Length': '0'} 2022-09-03T06:50:27.219 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c62842a9e1c35d3d-006312f933-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.220 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:27.221 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:27.222 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.222 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.223 DEBUG:boto:Method: HEAD 2022-09-03T06:50:27.223 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.224 DEBUG:boto:Data: 2022-09-03T06:50:27.224 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.225 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.226 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.226 DEBUG:boto:Params: {} 2022-09-03T06:50:27.227 DEBUG:boto:Token: None 2022-09-03T06:50:27.227 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.228 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:szxzqW3bR2UcO/YwR6RRBUXp7tc= 2022-09-03T06:50:27.229 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:szxzqW3bR2UcO/YwR6RRBUXp7tc=', 'Content-Length': '0'} 2022-09-03T06:50:27.231 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000032644d6b649d1a8f-006312f933-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.232 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:50:27.232 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:27.233 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:27.233 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.234 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.234 DEBUG:boto:Method: GET 2022-09-03T06:50:27.235 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.235 DEBUG:boto:Data: 2022-09-03T06:50:27.236 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.237 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.237 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.238 DEBUG:boto:Params: {} 2022-09-03T06:50:27.238 DEBUG:boto:Token: None 2022-09-03T06:50:27.239 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.240 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FvaX7D6gLbl5LZx3U4BHzB5kXk4= 2022-09-03T06:50:27.240 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FvaX7D6gLbl5LZx3U4BHzB5kXk4=', 'Content-Length': '0'} 2022-09-03T06:50:27.243 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d0f5822c8897cd30-006312f933-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.243 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:27.244 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:27.245 DEBUG:boto:path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.246 DEBUG:boto:auth_path=/dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.246 DEBUG:boto:Method: GET 2022-09-03T06:50:27.247 DEBUG:boto:Path: /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.248 DEBUG:boto:Data: 2022-09-03T06:50:27.248 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.249 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.249 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.250 DEBUG:boto:Params: {} 2022-09-03T06:50:27.251 DEBUG:boto:Token: None 2022-09-03T06:50:27.251 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj29?versionId=null 2022-09-03T06:50:27.252 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FvaX7D6gLbl5LZx3U4BHzB5kXk4= 2022-09-03T06:50:27.253 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FvaX7D6gLbl5LZx3U4BHzB5kXk4=', 'Content-Length': '0'} 2022-09-03T06:50:27.255 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ef6d15960797c715-006312f933-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.256 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:27.257 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:27.257 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:27.258 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:27.258 DEBUG:boto:Method: GET 2022-09-03T06:50:27.259 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:50:27.259 DEBUG:boto:Data: 2022-09-03T06:50:27.260 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.260 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.261 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.262 DEBUG:boto:Params: {} 2022-09-03T06:50:27.262 DEBUG:boto:Token: None 2022-09-03T06:50:27.263 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/ 2022-09-03T06:50:27.264 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic= 2022-09-03T06:50:27.264 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic=', 'Content-Length': '0'} 2022-09-03T06:50:27.267 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000003f3c5aee29fb5a4-006312f933-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.310 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:27.311 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:27.313 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:27.313 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:27.314 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:27.315 DEBUG:boto:Method: GET 2022-09-03T06:50:27.316 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:50:27.317 DEBUG:boto:Data: 2022-09-03T06:50:27.317 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.318 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.319 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.320 DEBUG:boto:Params: {} 2022-09-03T06:50:27.321 DEBUG:boto:Token: None 2022-09-03T06:50:27.323 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/ 2022-09-03T06:50:27.324 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic= 2022-09-03T06:50:27.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic=', 'Content-Length': '0'} 2022-09-03T06:50:27.328 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000800faef48b11ff13-006312f933-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.329 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:27.329 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:27.330 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:27.331 DEBUG:boto:Method: HEAD 2022-09-03T06:50:27.331 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:50:27.332 DEBUG:boto:Data: 2022-09-03T06:50:27.332 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.333 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.333 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.334 DEBUG:boto:Params: {} 2022-09-03T06:50:27.334 DEBUG:boto:Token: None 2022-09-03T06:50:27.335 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj29 2022-09-03T06:50:27.335 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tHqxxv1OdliPl62LT83jbfjFHEo= 2022-09-03T06:50:27.336 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tHqxxv1OdliPl62LT83jbfjFHEo=', 'Content-Length': '0'} 2022-09-03T06:50:27.338 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a52184a1e2981af-006312f933-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.339 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:27.340 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:27.343 DEBUG:boto:Method: HEAD 2022-09-03T06:50:27.344 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:50:27.344 DEBUG:boto:Data: 2022-09-03T06:50:27.345 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.345 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.346 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.346 DEBUG:boto:Params: {} 2022-09-03T06:50:27.347 DEBUG:boto:Token: None 2022-09-03T06:50:27.348 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj29 2022-09-03T06:50:27.348 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tHqxxv1OdliPl62LT83jbfjFHEo= 2022-09-03T06:50:27.349 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tHqxxv1OdliPl62LT83jbfjFHEo=', 'Content-Length': '0'} 2022-09-03T06:50:27.351 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000024cd7196c483820c-006312f933-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.352 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj29 2022-09-03T06:50:27.353 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:27.353 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:27.354 DEBUG:boto:Method: GET 2022-09-03T06:50:27.354 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:50:27.355 DEBUG:boto:Data: 2022-09-03T06:50:27.355 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.356 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.356 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.357 DEBUG:boto:Params: {} 2022-09-03T06:50:27.357 DEBUG:boto:Token: None 2022-09-03T06:50:27.358 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj29 2022-09-03T06:50:27.359 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/Ugu0Ng1dZEqDWUiVrgReTHSoMY= 2022-09-03T06:50:27.359 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/Ugu0Ng1dZEqDWUiVrgReTHSoMY=', 'Content-Length': '0'} 2022-09-03T06:50:27.361 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009477f6cccdfaa217-006312f933-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.362 DEBUG:boto:path=/dsgvcy-75/obj29 2022-09-03T06:50:27.363 DEBUG:boto:auth_path=/dsgvcy-75/obj29 2022-09-03T06:50:27.363 DEBUG:boto:Method: GET 2022-09-03T06:50:27.364 DEBUG:boto:Path: /dsgvcy-75/obj29 2022-09-03T06:50:27.365 DEBUG:boto:Data: 2022-09-03T06:50:27.365 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.366 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.366 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.367 DEBUG:boto:Params: {} 2022-09-03T06:50:27.367 DEBUG:boto:Token: None 2022-09-03T06:50:27.368 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj29 2022-09-03T06:50:27.369 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/Ugu0Ng1dZEqDWUiVrgReTHSoMY= 2022-09-03T06:50:27.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/Ugu0Ng1dZEqDWUiVrgReTHSoMY=', 'Content-Length': '0'} 2022-09-03T06:50:27.372 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003eb90b96526c20a3-006312f933-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.418 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:27.419 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:27.420 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:27.421 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:27.421 DEBUG:boto:Method: GET 2022-09-03T06:50:27.422 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:50:27.423 DEBUG:boto:Data: 2022-09-03T06:50:27.423 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.424 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.425 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.426 DEBUG:boto:Params: {} 2022-09-03T06:50:27.427 DEBUG:boto:Token: None 2022-09-03T06:50:27.428 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/ 2022-09-03T06:50:27.429 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic= 2022-09-03T06:50:27.430 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic=', 'Content-Length': '0'} 2022-09-03T06:50:27.433 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000016b93098d566b02f-006312f933-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.433 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:27.435 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:27.435 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:27.436 DEBUG:boto:path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:27.436 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj29 2022-09-03T06:50:27.437 DEBUG:boto:Method: GET 2022-09-03T06:50:27.437 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj29 2022-09-03T06:50:27.438 DEBUG:boto:Data: 2022-09-03T06:50:27.439 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.439 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.440 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.440 DEBUG:boto:Params: {} 2022-09-03T06:50:27.441 DEBUG:boto:Token: None 2022-09-03T06:50:27.442 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/ 2022-09-03T06:50:27.442 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic= 2022-09-03T06:50:27.443 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic=', 'Content-Length': '0'} 2022-09-03T06:50:27.446 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b25dafe200184179-006312f933-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.447 DEBUG:boto:b'dsgvcy-75obj291000falseobj292022-09-03T06:44:18.231Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:27.447 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2022-09-03T06:50:27.448 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:27.449 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:27.449 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.450 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.450 DEBUG:boto:Method: GET 2022-09-03T06:50:27.451 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.451 DEBUG:boto:Data: 2022-09-03T06:50:27.452 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.453 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.453 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.454 DEBUG:boto:Params: {} 2022-09-03T06:50:27.454 DEBUG:boto:Token: None 2022-09-03T06:50:27.455 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.456 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hv4CVI6G/R7n0S9rVL8HfMam2pQ= 2022-09-03T06:50:27.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hv4CVI6G/R7n0S9rVL8HfMam2pQ=', 'Content-Length': '0'} 2022-09-03T06:50:27.459 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000c8291e06e645ae0-006312f933-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.459 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:27.460 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:27.461 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.461 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.462 DEBUG:boto:Method: GET 2022-09-03T06:50:27.462 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.463 DEBUG:boto:Data: 2022-09-03T06:50:27.463 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.464 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.465 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.466 DEBUG:boto:Params: {} 2022-09-03T06:50:27.466 DEBUG:boto:Token: None 2022-09-03T06:50:27.467 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.468 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hv4CVI6G/R7n0S9rVL8HfMam2pQ= 2022-09-03T06:50:27.468 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hv4CVI6G/R7n0S9rVL8HfMam2pQ=', 'Content-Length': '0'} 2022-09-03T06:50:27.471 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad98bc535687e9e4-006312f933-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.472 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:27.472 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:27.473 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.473 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.474 DEBUG:boto:Method: HEAD 2022-09-03T06:50:27.474 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.475 DEBUG:boto:Data: 2022-09-03T06:50:27.475 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.476 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.476 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.477 DEBUG:boto:Params: {} 2022-09-03T06:50:27.478 DEBUG:boto:Token: None 2022-09-03T06:50:27.478 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.479 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c8pfJiUPEE/7CVRoADAtarRzQxg= 2022-09-03T06:50:27.480 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c8pfJiUPEE/7CVRoADAtarRzQxg=', 'Content-Length': '0'} 2022-09-03T06:50:27.482 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004ad9ad53869f2802-006312f933-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.483 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:27.484 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:27.485 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.485 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.486 DEBUG:boto:Method: HEAD 2022-09-03T06:50:27.487 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.488 DEBUG:boto:Data: 2022-09-03T06:50:27.489 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.490 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.490 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.491 DEBUG:boto:Params: {} 2022-09-03T06:50:27.492 DEBUG:boto:Token: None 2022-09-03T06:50:27.494 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.495 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:c8pfJiUPEE/7CVRoADAtarRzQxg= 2022-09-03T06:50:27.496 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:c8pfJiUPEE/7CVRoADAtarRzQxg=', 'Content-Length': '0'} 2022-09-03T06:50:27.498 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000082b4b7574210d5c2-006312f933-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.499 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2022-09-03T06:50:27.500 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:27.500 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:27.501 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.502 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.503 DEBUG:boto:Method: GET 2022-09-03T06:50:27.503 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.505 DEBUG:boto:Data: 2022-09-03T06:50:27.505 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.506 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.506 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.507 DEBUG:boto:Params: {} 2022-09-03T06:50:27.507 DEBUG:boto:Token: None 2022-09-03T06:50:27.508 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.509 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hv4CVI6G/R7n0S9rVL8HfMam2pQ= 2022-09-03T06:50:27.509 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hv4CVI6G/R7n0S9rVL8HfMam2pQ=', 'Content-Length': '0'} 2022-09-03T06:50:27.511 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d6428ba3fcda265-006312f933-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.554 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:27.555 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:27.556 DEBUG:boto:path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.558 DEBUG:boto:auth_path=/dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.559 DEBUG:boto:Method: GET 2022-09-03T06:50:27.559 DEBUG:boto:Path: /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.560 DEBUG:boto:Data: 2022-09-03T06:50:27.560 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.561 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.562 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.563 DEBUG:boto:Params: {} 2022-09-03T06:50:27.564 DEBUG:boto:Token: None 2022-09-03T06:50:27.565 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj31?versionId=null 2022-09-03T06:50:27.566 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Hv4CVI6G/R7n0S9rVL8HfMam2pQ= 2022-09-03T06:50:27.567 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Hv4CVI6G/R7n0S9rVL8HfMam2pQ=', 'Content-Length': '0'} 2022-09-03T06:50:27.570 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000902313229cf08edc-006312f933-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.571 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:27.573 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:27.574 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:27.574 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:27.575 DEBUG:boto:Method: GET 2022-09-03T06:50:27.576 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:50:27.577 DEBUG:boto:Data: 2022-09-03T06:50:27.578 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.579 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.580 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.581 DEBUG:boto:Params: {} 2022-09-03T06:50:27.583 DEBUG:boto:Token: None 2022-09-03T06:50:27.585 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/ 2022-09-03T06:50:27.586 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic= 2022-09-03T06:50:27.587 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic=', 'Content-Length': '0'} 2022-09-03T06:50:27.590 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000001eb1b95dd13bd54-006312f933-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.591 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:27.592 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:27.592 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:27.593 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:27.593 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:27.594 DEBUG:boto:Method: GET 2022-09-03T06:50:27.595 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:50:27.595 DEBUG:boto:Data: 2022-09-03T06:50:27.596 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.597 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.598 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.599 DEBUG:boto:Params: {} 2022-09-03T06:50:27.601 DEBUG:boto:Token: None 2022-09-03T06:50:27.602 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/ 2022-09-03T06:50:27.604 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic= 2022-09-03T06:50:27.605 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic=', 'Content-Length': '0'} 2022-09-03T06:50:27.609 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000440f975da6f7e5ef-006312f933-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.611 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:27.612 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:27.613 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:27.614 DEBUG:boto:Method: HEAD 2022-09-03T06:50:27.615 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:50:27.616 DEBUG:boto:Data: 2022-09-03T06:50:27.617 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.618 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.619 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.620 DEBUG:boto:Params: {} 2022-09-03T06:50:27.621 DEBUG:boto:Token: None 2022-09-03T06:50:27.622 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj31 2022-09-03T06:50:27.623 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FZTttP5XUIunkfqbxzflENb2Mjo= 2022-09-03T06:50:27.624 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FZTttP5XUIunkfqbxzflENb2Mjo=', 'Content-Length': '0'} 2022-09-03T06:50:27.627 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009aa41d632e7cd36d-006312f933-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.629 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:27.630 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:27.631 DEBUG:boto:Method: HEAD 2022-09-03T06:50:27.632 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:50:27.635 DEBUG:boto:Data: 2022-09-03T06:50:27.635 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.636 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.636 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.637 DEBUG:boto:Params: {} 2022-09-03T06:50:27.638 DEBUG:boto:Token: None 2022-09-03T06:50:27.638 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj31 2022-09-03T06:50:27.639 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:FZTttP5XUIunkfqbxzflENb2Mjo= 2022-09-03T06:50:27.641 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:FZTttP5XUIunkfqbxzflENb2Mjo=', 'Content-Length': '0'} 2022-09-03T06:50:27.644 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000340a71f1e3547ea1-006312f933-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.646 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj31 2022-09-03T06:50:27.647 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:27.648 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:27.649 DEBUG:boto:Method: GET 2022-09-03T06:50:27.650 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:50:27.651 DEBUG:boto:Data: 2022-09-03T06:50:27.652 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.653 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.654 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.655 DEBUG:boto:Params: {} 2022-09-03T06:50:27.656 DEBUG:boto:Token: None 2022-09-03T06:50:27.657 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj31 2022-09-03T06:50:27.657 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oWVvgVtTAaLw6cW+k1KfXYUhGFk= 2022-09-03T06:50:27.658 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oWVvgVtTAaLw6cW+k1KfXYUhGFk=', 'Content-Length': '0'} 2022-09-03T06:50:27.660 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000007d7afeba35ea644-006312f933-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.702 DEBUG:boto:path=/dsgvcy-75/obj31 2022-09-03T06:50:27.703 DEBUG:boto:auth_path=/dsgvcy-75/obj31 2022-09-03T06:50:27.703 DEBUG:boto:Method: GET 2022-09-03T06:50:27.704 DEBUG:boto:Path: /dsgvcy-75/obj31 2022-09-03T06:50:27.705 DEBUG:boto:Data: 2022-09-03T06:50:27.705 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.706 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.707 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.707 DEBUG:boto:Params: {} 2022-09-03T06:50:27.708 DEBUG:boto:Token: None 2022-09-03T06:50:27.709 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj31 2022-09-03T06:50:27.710 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oWVvgVtTAaLw6cW+k1KfXYUhGFk= 2022-09-03T06:50:27.710 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oWVvgVtTAaLw6cW+k1KfXYUhGFk=', 'Content-Length': '0'} 2022-09-03T06:50:27.713 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000144620e23f19bf6b-006312f933-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.714 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:27.715 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:27.716 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:27.716 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:27.717 DEBUG:boto:Method: GET 2022-09-03T06:50:27.718 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:50:27.719 DEBUG:boto:Data: 2022-09-03T06:50:27.720 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.720 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.722 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.724 DEBUG:boto:Params: {} 2022-09-03T06:50:27.725 DEBUG:boto:Token: None 2022-09-03T06:50:27.726 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/ 2022-09-03T06:50:27.727 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic= 2022-09-03T06:50:27.727 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic=', 'Content-Length': '0'} 2022-09-03T06:50:27.731 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000cd61cc7369e7970b-006312f933-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.732 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:27.732 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:27.733 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:27.734 DEBUG:boto:path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:27.734 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj31 2022-09-03T06:50:27.735 DEBUG:boto:Method: GET 2022-09-03T06:50:27.735 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj31 2022-09-03T06:50:27.736 DEBUG:boto:Data: 2022-09-03T06:50:27.736 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.737 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.737 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.739 DEBUG:boto:Params: {} 2022-09-03T06:50:27.739 DEBUG:boto:Token: None 2022-09-03T06:50:27.740 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/ 2022-09-03T06:50:27.740 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic= 2022-09-03T06:50:27.741 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic=', 'Content-Length': '0'} 2022-09-03T06:50:27.744 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e1c9684a5499bd1f-006312f933-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.744 DEBUG:boto:b'dsgvcy-75obj311000falseobj312022-09-03T06:45:46.812Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:27.745 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:50:27.746 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:27.746 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:27.747 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.747 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.748 DEBUG:boto:Method: GET 2022-09-03T06:50:27.749 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.749 DEBUG:boto:Data: 2022-09-03T06:50:27.750 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.750 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.751 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.751 DEBUG:boto:Params: {} 2022-09-03T06:50:27.752 DEBUG:boto:Token: None 2022-09-03T06:50:27.752 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.753 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QV4jQWtfXBk+FmzVwelXKq5owXc= 2022-09-03T06:50:27.754 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QV4jQWtfXBk+FmzVwelXKq5owXc=', 'Content-Length': '0'} 2022-09-03T06:50:27.756 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000537bc44cf73b817c-006312f933-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.757 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:27.757 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:27.758 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.758 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.759 DEBUG:boto:Method: GET 2022-09-03T06:50:27.759 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.760 DEBUG:boto:Data: 2022-09-03T06:50:27.760 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.761 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.761 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.762 DEBUG:boto:Params: {} 2022-09-03T06:50:27.763 DEBUG:boto:Token: None 2022-09-03T06:50:27.763 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.764 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QV4jQWtfXBk+FmzVwelXKq5owXc= 2022-09-03T06:50:27.764 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QV4jQWtfXBk+FmzVwelXKq5owXc=', 'Content-Length': '0'} 2022-09-03T06:50:27.766 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ceb83723057a3da5-006312f933-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.767 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:27.768 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:27.768 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.769 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.769 DEBUG:boto:Method: HEAD 2022-09-03T06:50:27.770 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.771 DEBUG:boto:Data: 2022-09-03T06:50:27.771 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.772 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.772 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.773 DEBUG:boto:Params: {} 2022-09-03T06:50:27.773 DEBUG:boto:Token: None 2022-09-03T06:50:27.774 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.774 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6S910CSuJZHHFpCeKoG8hF2ry7c= 2022-09-03T06:50:27.775 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6S910CSuJZHHFpCeKoG8hF2ry7c=', 'Content-Length': '0'} 2022-09-03T06:50:27.777 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000085dad0f4aa2fd904-006312f933-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.778 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:27.778 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:27.779 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.779 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.780 DEBUG:boto:Method: HEAD 2022-09-03T06:50:27.780 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.781 DEBUG:boto:Data: 2022-09-03T06:50:27.781 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.782 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.782 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.783 DEBUG:boto:Params: {} 2022-09-03T06:50:27.783 DEBUG:boto:Token: None 2022-09-03T06:50:27.784 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.785 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6S910CSuJZHHFpCeKoG8hF2ry7c= 2022-09-03T06:50:27.785 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6S910CSuJZHHFpCeKoG8hF2ry7c=', 'Content-Length': '0'} 2022-09-03T06:50:27.787 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000074365bd6e8b75098-006312f933-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.788 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:50:27.789 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:27.790 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:27.791 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.792 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.793 DEBUG:boto:Method: GET 2022-09-03T06:50:27.794 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.795 DEBUG:boto:Data: 2022-09-03T06:50:27.796 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.797 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.798 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.799 DEBUG:boto:Params: {} 2022-09-03T06:50:27.800 DEBUG:boto:Token: None 2022-09-03T06:50:27.801 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.802 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QV4jQWtfXBk+FmzVwelXKq5owXc= 2022-09-03T06:50:27.803 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QV4jQWtfXBk+FmzVwelXKq5owXc=', 'Content-Length': '0'} 2022-09-03T06:50:27.805 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a0af15320a7d900-006312f933-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.806 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:27.806 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:27.807 DEBUG:boto:path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.807 DEBUG:boto:auth_path=/dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.808 DEBUG:boto:Method: GET 2022-09-03T06:50:27.809 DEBUG:boto:Path: /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.809 DEBUG:boto:Data: 2022-09-03T06:50:27.810 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.810 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.811 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.812 DEBUG:boto:Params: {} 2022-09-03T06:50:27.812 DEBUG:boto:Token: None 2022-09-03T06:50:27.813 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj33?versionId=null 2022-09-03T06:50:27.814 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:QV4jQWtfXBk+FmzVwelXKq5owXc= 2022-09-03T06:50:27.814 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:QV4jQWtfXBk+FmzVwelXKq5owXc=', 'Content-Length': '0'} 2022-09-03T06:50:27.817 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000089435350a11a9f1b-006312f933-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.818 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:27.818 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:27.819 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:27.819 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:27.820 DEBUG:boto:Method: GET 2022-09-03T06:50:27.820 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:50:27.821 DEBUG:boto:Data: 2022-09-03T06:50:27.821 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.822 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.822 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.823 DEBUG:boto:Params: {} 2022-09-03T06:50:27.824 DEBUG:boto:Token: None 2022-09-03T06:50:27.824 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/ 2022-09-03T06:50:27.825 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic= 2022-09-03T06:50:27.826 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic=', 'Content-Length': '0'} 2022-09-03T06:50:27.829 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004b9d49f65851d8e9-006312f933-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.829 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:27.830 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:27.831 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:27.831 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:27.832 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:27.833 DEBUG:boto:Method: GET 2022-09-03T06:50:27.833 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:50:27.834 DEBUG:boto:Data: 2022-09-03T06:50:27.836 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.836 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.837 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.837 DEBUG:boto:Params: {} 2022-09-03T06:50:27.838 DEBUG:boto:Token: None 2022-09-03T06:50:27.839 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/ 2022-09-03T06:50:27.840 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic= 2022-09-03T06:50:27.841 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic=', 'Content-Length': '0'} 2022-09-03T06:50:27.844 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000d11ffcafd1920bc-006312f933-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.846 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:27.847 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:27.847 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:27.848 DEBUG:boto:Method: HEAD 2022-09-03T06:50:27.848 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:50:27.849 DEBUG:boto:Data: 2022-09-03T06:50:27.849 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.850 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.851 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.851 DEBUG:boto:Params: {} 2022-09-03T06:50:27.852 DEBUG:boto:Token: None 2022-09-03T06:50:27.853 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj33 2022-09-03T06:50:27.853 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O3nzWlTyVlBf+u39rbg32tSBOX0= 2022-09-03T06:50:27.854 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O3nzWlTyVlBf+u39rbg32tSBOX0=', 'Content-Length': '0'} 2022-09-03T06:50:27.857 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000056d439e7b2fee88-006312f933-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.858 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:27.858 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:27.859 DEBUG:boto:Method: HEAD 2022-09-03T06:50:27.859 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:50:27.860 DEBUG:boto:Data: 2022-09-03T06:50:27.861 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.861 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.862 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.863 DEBUG:boto:Params: {} 2022-09-03T06:50:27.864 DEBUG:boto:Token: None 2022-09-03T06:50:27.865 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj33 2022-09-03T06:50:27.866 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:O3nzWlTyVlBf+u39rbg32tSBOX0= 2022-09-03T06:50:27.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:O3nzWlTyVlBf+u39rbg32tSBOX0=', 'Content-Length': '0'} 2022-09-03T06:50:27.870 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000893a0044d8e97999-006312f933-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.871 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj33 2022-09-03T06:50:27.871 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:27.872 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:27.873 DEBUG:boto:Method: GET 2022-09-03T06:50:27.874 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:50:27.874 DEBUG:boto:Data: 2022-09-03T06:50:27.875 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.875 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.876 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.877 DEBUG:boto:Params: {} 2022-09-03T06:50:27.878 DEBUG:boto:Token: None 2022-09-03T06:50:27.879 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj33 2022-09-03T06:50:27.879 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NB6Ujoqa0FUTTdI45p4BQ1bH/mo= 2022-09-03T06:50:27.880 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NB6Ujoqa0FUTTdI45p4BQ1bH/mo=', 'Content-Length': '0'} 2022-09-03T06:50:27.882 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000077ed64df253f4a5d-006312f933-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.883 DEBUG:boto:path=/dsgvcy-75/obj33 2022-09-03T06:50:27.883 DEBUG:boto:auth_path=/dsgvcy-75/obj33 2022-09-03T06:50:27.884 DEBUG:boto:Method: GET 2022-09-03T06:50:27.885 DEBUG:boto:Path: /dsgvcy-75/obj33 2022-09-03T06:50:27.885 DEBUG:boto:Data: 2022-09-03T06:50:27.886 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.886 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.887 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.887 DEBUG:boto:Params: {} 2022-09-03T06:50:27.888 DEBUG:boto:Token: None 2022-09-03T06:50:27.889 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj33 2022-09-03T06:50:27.889 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:NB6Ujoqa0FUTTdI45p4BQ1bH/mo= 2022-09-03T06:50:27.890 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:NB6Ujoqa0FUTTdI45p4BQ1bH/mo=', 'Content-Length': '0'} 2022-09-03T06:50:27.895 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000049b9c1ca5dddf641-006312f933-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.938 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:27.939 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:27.940 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:27.941 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:27.941 DEBUG:boto:Method: GET 2022-09-03T06:50:27.942 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:50:27.943 DEBUG:boto:Data: 2022-09-03T06:50:27.943 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.944 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.944 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.945 DEBUG:boto:Params: {} 2022-09-03T06:50:27.947 DEBUG:boto:Token: None 2022-09-03T06:50:27.948 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/ 2022-09-03T06:50:27.949 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic= 2022-09-03T06:50:27.949 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic=', 'Content-Length': '0'} 2022-09-03T06:50:27.952 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f32e3fb2f5face51-006312f933-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.953 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:27.954 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:27.955 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:27.955 DEBUG:boto:path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:27.956 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj33 2022-09-03T06:50:27.956 DEBUG:boto:Method: GET 2022-09-03T06:50:27.957 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj33 2022-09-03T06:50:27.958 DEBUG:boto:Data: 2022-09-03T06:50:27.959 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.959 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.960 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.961 DEBUG:boto:Params: {} 2022-09-03T06:50:27.961 DEBUG:boto:Token: None 2022-09-03T06:50:27.962 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/ 2022-09-03T06:50:27.963 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic= 2022-09-03T06:50:27.964 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hPN/rolbj09d3kuBIc3TsgAdbic=', 'Content-Length': '0'} 2022-09-03T06:50:27.966 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000026614fba63ede46d-006312f933-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.967 DEBUG:boto:b'dsgvcy-75obj331000falseobj332022-09-03T06:36:05.796Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:27.968 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:50:27.969 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:27.969 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:27.970 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:27.971 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:27.971 DEBUG:boto:Method: GET 2022-09-03T06:50:27.972 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:27.972 DEBUG:boto:Data: 2022-09-03T06:50:27.973 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.973 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.974 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.974 DEBUG:boto:Params: {} 2022-09-03T06:50:27.975 DEBUG:boto:Token: None 2022-09-03T06:50:27.975 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:27.976 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WeooEwyeFkD6ecaFvnMsfDrldno= 2022-09-03T06:50:27.977 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WeooEwyeFkD6ecaFvnMsfDrldno=', 'Content-Length': '0'} 2022-09-03T06:50:27.979 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000f6632753f5cc643-006312f933-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.980 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:27.980 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:27.981 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:27.981 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:27.982 DEBUG:boto:Method: GET 2022-09-03T06:50:27.983 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:27.983 DEBUG:boto:Data: 2022-09-03T06:50:27.984 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.984 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:27.985 DEBUG:boto:Port: 8000 2022-09-03T06:50:27.985 DEBUG:boto:Params: {} 2022-09-03T06:50:27.986 DEBUG:boto:Token: None 2022-09-03T06:50:27.987 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:27.987 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:WeooEwyeFkD6ecaFvnMsfDrldno= 2022-09-03T06:50:27.988 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:WeooEwyeFkD6ecaFvnMsfDrldno=', 'Content-Length': '0'} 2022-09-03T06:50:27.990 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000146dcd1617afe0f6-006312f933-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:27.991 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:27.991 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:27.992 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:27.992 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:27.993 DEBUG:boto:Method: HEAD 2022-09-03T06:50:27.994 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:27.994 DEBUG:boto:Data: 2022-09-03T06:50:27.995 DEBUG:boto:Headers: {} 2022-09-03T06:50:27.995 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:27.996 DEBUG:boto:Port: 8001 2022-09-03T06:50:27.996 DEBUG:boto:Params: {} 2022-09-03T06:50:27.997 DEBUG:boto:Token: None 2022-09-03T06:50:27.997 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:27 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:27.998 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KCRR4T93ADll4vZ/Cdl1pbbsbko= 2022-09-03T06:50:27.998 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:27 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KCRR4T93ADll4vZ/Cdl1pbbsbko=', 'Content-Length': '0'} 2022-09-03T06:50:28.000 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000036a9fce0498ab5c3-006312f933-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:27 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.001 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:28.002 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:28.002 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:28.003 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:28.003 DEBUG:boto:Method: HEAD 2022-09-03T06:50:28.004 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:28.004 DEBUG:boto:Data: 2022-09-03T06:50:28.005 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.005 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.006 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.006 DEBUG:boto:Params: {} 2022-09-03T06:50:28.007 DEBUG:boto:Token: None 2022-09-03T06:50:28.008 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:28.008 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ighbQAfDKJrxCy2+ZLhUjNq5WCA= 2022-09-03T06:50:28.009 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ighbQAfDKJrxCy2+ZLhUjNq5WCA=', 'Content-Length': '0'} 2022-09-03T06:50:28.011 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b436b749179e1c24-006312f934-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.012 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:50:28.013 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:28.013 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:28.014 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:28.015 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:28.016 DEBUG:boto:Method: GET 2022-09-03T06:50:28.016 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:28.017 DEBUG:boto:Data: 2022-09-03T06:50:28.018 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.019 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.020 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.021 DEBUG:boto:Params: {} 2022-09-03T06:50:28.024 DEBUG:boto:Token: None 2022-09-03T06:50:28.025 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:28.028 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e0LOcp14I3JPtYF1W4s4VeDDXb0= 2022-09-03T06:50:28.029 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e0LOcp14I3JPtYF1W4s4VeDDXb0=', 'Content-Length': '0'} 2022-09-03T06:50:28.031 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009668c3a01665d212-006312f934-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.074 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:28.075 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:28.075 DEBUG:boto:path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:28.076 DEBUG:boto:auth_path=/dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:28.076 DEBUG:boto:Method: GET 2022-09-03T06:50:28.077 DEBUG:boto:Path: /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:28.078 DEBUG:boto:Data: 2022-09-03T06:50:28.078 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.079 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.080 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.080 DEBUG:boto:Params: {} 2022-09-03T06:50:28.081 DEBUG:boto:Token: None 2022-09-03T06:50:28.082 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj34?versionId=null 2022-09-03T06:50:28.082 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:e0LOcp14I3JPtYF1W4s4VeDDXb0= 2022-09-03T06:50:28.083 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:e0LOcp14I3JPtYF1W4s4VeDDXb0=', 'Content-Length': '0'} 2022-09-03T06:50:28.085 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c5d0c8f5853312dc-006312f934-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.086 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:28.087 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:28.087 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:28.088 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:28.089 DEBUG:boto:Method: GET 2022-09-03T06:50:28.089 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:50:28.090 DEBUG:boto:Data: 2022-09-03T06:50:28.090 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.091 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.092 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.092 DEBUG:boto:Params: {} 2022-09-03T06:50:28.093 DEBUG:boto:Token: None 2022-09-03T06:50:28.093 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/ 2022-09-03T06:50:28.094 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg= 2022-09-03T06:50:28.095 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg=', 'Content-Length': '0'} 2022-09-03T06:50:28.098 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006ff4c3271c9f195a-006312f934-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.099 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:28.100 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:28.100 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:28.101 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:28.102 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:28.102 DEBUG:boto:Method: GET 2022-09-03T06:50:28.103 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:50:28.103 DEBUG:boto:Data: 2022-09-03T06:50:28.104 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.104 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.105 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.106 DEBUG:boto:Params: {} 2022-09-03T06:50:28.106 DEBUG:boto:Token: None 2022-09-03T06:50:28.107 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/ 2022-09-03T06:50:28.107 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg= 2022-09-03T06:50:28.108 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg=', 'Content-Length': '0'} 2022-09-03T06:50:28.111 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004cdbb24aa3befe77-006312f934-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.113 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:28.114 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:28.114 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:28.115 DEBUG:boto:Method: HEAD 2022-09-03T06:50:28.115 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:50:28.116 DEBUG:boto:Data: 2022-09-03T06:50:28.116 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.117 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.117 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.118 DEBUG:boto:Params: {} 2022-09-03T06:50:28.119 DEBUG:boto:Token: None 2022-09-03T06:50:28.119 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj34 2022-09-03T06:50:28.120 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jnxU4msdoM0bOOXD43MsAAZ4GUA= 2022-09-03T06:50:28.120 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jnxU4msdoM0bOOXD43MsAAZ4GUA=', 'Content-Length': '0'} 2022-09-03T06:50:28.122 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005016fffd6ba61086-006312f934-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.124 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:28.125 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:28.125 DEBUG:boto:Method: HEAD 2022-09-03T06:50:28.126 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:50:28.126 DEBUG:boto:Data: 2022-09-03T06:50:28.127 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.127 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.128 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.128 DEBUG:boto:Params: {} 2022-09-03T06:50:28.129 DEBUG:boto:Token: None 2022-09-03T06:50:28.130 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj34 2022-09-03T06:50:28.130 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:jnxU4msdoM0bOOXD43MsAAZ4GUA= 2022-09-03T06:50:28.131 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:jnxU4msdoM0bOOXD43MsAAZ4GUA=', 'Content-Length': '0'} 2022-09-03T06:50:28.133 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008796470d17ce8760-006312f934-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.134 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj34 2022-09-03T06:50:28.135 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:28.136 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:28.136 DEBUG:boto:Method: GET 2022-09-03T06:50:28.137 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:50:28.137 DEBUG:boto:Data: 2022-09-03T06:50:28.138 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.138 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.139 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.140 DEBUG:boto:Params: {} 2022-09-03T06:50:28.141 DEBUG:boto:Token: None 2022-09-03T06:50:28.141 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj34 2022-09-03T06:50:28.142 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PDy+e8G7DN9kVZJKQJHsFEvOzUU= 2022-09-03T06:50:28.144 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PDy+e8G7DN9kVZJKQJHsFEvOzUU=', 'Content-Length': '0'} 2022-09-03T06:50:28.146 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fdc49ad1f6120d23-006312f934-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.186 DEBUG:boto:path=/dsgvcy-75/obj34 2022-09-03T06:50:28.187 DEBUG:boto:auth_path=/dsgvcy-75/obj34 2022-09-03T06:50:28.188 DEBUG:boto:Method: GET 2022-09-03T06:50:28.188 DEBUG:boto:Path: /dsgvcy-75/obj34 2022-09-03T06:50:28.189 DEBUG:boto:Data: 2022-09-03T06:50:28.189 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.190 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.190 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.191 DEBUG:boto:Params: {} 2022-09-03T06:50:28.192 DEBUG:boto:Token: None 2022-09-03T06:50:28.193 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj34 2022-09-03T06:50:28.194 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PDy+e8G7DN9kVZJKQJHsFEvOzUU= 2022-09-03T06:50:28.194 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PDy+e8G7DN9kVZJKQJHsFEvOzUU=', 'Content-Length': '0'} 2022-09-03T06:50:28.197 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f200885b39f20751-006312f934-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.198 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:28.199 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:28.199 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:28.200 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:28.200 DEBUG:boto:Method: GET 2022-09-03T06:50:28.201 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:50:28.201 DEBUG:boto:Data: 2022-09-03T06:50:28.202 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.202 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.203 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.203 DEBUG:boto:Params: {} 2022-09-03T06:50:28.204 DEBUG:boto:Token: None 2022-09-03T06:50:28.205 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/ 2022-09-03T06:50:28.205 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg= 2022-09-03T06:50:28.206 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg=', 'Content-Length': '0'} 2022-09-03T06:50:28.209 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000014b6a07c70ea7062-006312f934-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.209 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:28.211 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:28.211 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:28.212 DEBUG:boto:path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:28.213 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj34 2022-09-03T06:50:28.214 DEBUG:boto:Method: GET 2022-09-03T06:50:28.214 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj34 2022-09-03T06:50:28.215 DEBUG:boto:Data: 2022-09-03T06:50:28.215 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.216 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.216 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.217 DEBUG:boto:Params: {} 2022-09-03T06:50:28.217 DEBUG:boto:Token: None 2022-09-03T06:50:28.218 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/ 2022-09-03T06:50:28.219 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg= 2022-09-03T06:50:28.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg=', 'Content-Length': '0'} 2022-09-03T06:50:28.222 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000098ec21408948068b-006312f934-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.223 DEBUG:boto:b'dsgvcy-75obj341000falseobj342022-09-03T06:37:27.080Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:28.224 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj37 2022-09-03T06:50:28.225 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:28.225 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:28.226 DEBUG:boto:path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.227 DEBUG:boto:auth_path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.227 DEBUG:boto:Method: GET 2022-09-03T06:50:28.228 DEBUG:boto:Path: /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.229 DEBUG:boto:Data: 2022-09-03T06:50:28.229 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.230 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.230 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.231 DEBUG:boto:Params: {} 2022-09-03T06:50:28.231 DEBUG:boto:Token: None 2022-09-03T06:50:28.232 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.233 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v1Y7IVWffKubPO+vt5CJeG5qESI= 2022-09-03T06:50:28.233 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v1Y7IVWffKubPO+vt5CJeG5qESI=', 'Content-Length': '0'} 2022-09-03T06:50:28.236 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000063838a4c13cc8c49-006312f934-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.237 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:28.237 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:28.238 DEBUG:boto:path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.238 DEBUG:boto:auth_path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.239 DEBUG:boto:Method: GET 2022-09-03T06:50:28.239 DEBUG:boto:Path: /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.240 DEBUG:boto:Data: 2022-09-03T06:50:28.240 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.241 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.241 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.242 DEBUG:boto:Params: {} 2022-09-03T06:50:28.243 DEBUG:boto:Token: None 2022-09-03T06:50:28.244 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.244 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v1Y7IVWffKubPO+vt5CJeG5qESI= 2022-09-03T06:50:28.245 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v1Y7IVWffKubPO+vt5CJeG5qESI=', 'Content-Length': '0'} 2022-09-03T06:50:28.248 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005df0e35bebfb338c-006312f934-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.249 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:28.250 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:28.251 DEBUG:boto:path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.252 DEBUG:boto:auth_path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.253 DEBUG:boto:Method: HEAD 2022-09-03T06:50:28.254 DEBUG:boto:Path: /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.255 DEBUG:boto:Data: 2022-09-03T06:50:28.256 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.257 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.258 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.258 DEBUG:boto:Params: {} 2022-09-03T06:50:28.259 DEBUG:boto:Token: None 2022-09-03T06:50:28.260 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.261 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8bUvguFmNLpqmfOVW5lxhCkEJ6A= 2022-09-03T06:50:28.261 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8bUvguFmNLpqmfOVW5lxhCkEJ6A=', 'Content-Length': '0'} 2022-09-03T06:50:28.263 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c4a38fb209d2f9a4-006312f934-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.264 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:28.265 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:28.265 DEBUG:boto:path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.266 DEBUG:boto:auth_path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.266 DEBUG:boto:Method: HEAD 2022-09-03T06:50:28.267 DEBUG:boto:Path: /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.267 DEBUG:boto:Data: 2022-09-03T06:50:28.268 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.268 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.269 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.270 DEBUG:boto:Params: {} 2022-09-03T06:50:28.270 DEBUG:boto:Token: None 2022-09-03T06:50:28.271 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.272 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8bUvguFmNLpqmfOVW5lxhCkEJ6A= 2022-09-03T06:50:28.272 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8bUvguFmNLpqmfOVW5lxhCkEJ6A=', 'Content-Length': '0'} 2022-09-03T06:50:28.274 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a99c19706826a72f-006312f934-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.275 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj37 2022-09-03T06:50:28.276 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:28.276 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:28.277 DEBUG:boto:path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.278 DEBUG:boto:auth_path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.278 DEBUG:boto:Method: GET 2022-09-03T06:50:28.279 DEBUG:boto:Path: /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.279 DEBUG:boto:Data: 2022-09-03T06:50:28.280 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.280 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.281 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.282 DEBUG:boto:Params: {} 2022-09-03T06:50:28.283 DEBUG:boto:Token: None 2022-09-03T06:50:28.283 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.284 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v1Y7IVWffKubPO+vt5CJeG5qESI= 2022-09-03T06:50:28.285 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v1Y7IVWffKubPO+vt5CJeG5qESI=', 'Content-Length': '0'} 2022-09-03T06:50:28.287 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b1dfd0c27288a35d-006312f934-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.288 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:28.288 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:28.289 DEBUG:boto:path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.289 DEBUG:boto:auth_path=/dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.290 DEBUG:boto:Method: GET 2022-09-03T06:50:28.290 DEBUG:boto:Path: /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.291 DEBUG:boto:Data: 2022-09-03T06:50:28.292 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.292 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.293 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.294 DEBUG:boto:Params: {} 2022-09-03T06:50:28.294 DEBUG:boto:Token: None 2022-09-03T06:50:28.295 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj37?versionId=null 2022-09-03T06:50:28.296 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:v1Y7IVWffKubPO+vt5CJeG5qESI= 2022-09-03T06:50:28.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:v1Y7IVWffKubPO+vt5CJeG5qESI=', 'Content-Length': '0'} 2022-09-03T06:50:28.298 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b336bdb2d0edd80-006312f934-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.342 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:28.343 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:28.344 DEBUG:boto:path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:28.344 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:28.345 DEBUG:boto:Method: GET 2022-09-03T06:50:28.345 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj37 2022-09-03T06:50:28.347 DEBUG:boto:Data: 2022-09-03T06:50:28.348 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.349 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.349 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.350 DEBUG:boto:Params: {} 2022-09-03T06:50:28.352 DEBUG:boto:Token: None 2022-09-03T06:50:28.353 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/ 2022-09-03T06:50:28.354 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg= 2022-09-03T06:50:28.355 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg=', 'Content-Length': '0'} 2022-09-03T06:50:28.359 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002c46a467728d2d21-006312f934-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.360 DEBUG:boto:b'dsgvcy-75obj371000falseobj372022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:28.361 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:28.361 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:28.362 DEBUG:boto:path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:28.362 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:28.363 DEBUG:boto:Method: GET 2022-09-03T06:50:28.364 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj37 2022-09-03T06:50:28.365 DEBUG:boto:Data: 2022-09-03T06:50:28.367 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.368 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.369 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.370 DEBUG:boto:Params: {} 2022-09-03T06:50:28.372 DEBUG:boto:Token: None 2022-09-03T06:50:28.373 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/ 2022-09-03T06:50:28.374 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg= 2022-09-03T06:50:28.375 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg=', 'Content-Length': '0'} 2022-09-03T06:50:28.378 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000001a0a9233efdf3b64-006312f934-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.380 DEBUG:boto:b'dsgvcy-75obj371000falseobj372022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:28.381 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:28.382 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:28.383 DEBUG:boto:Method: HEAD 2022-09-03T06:50:28.384 DEBUG:boto:Path: /dsgvcy-75/obj37 2022-09-03T06:50:28.385 DEBUG:boto:Data: 2022-09-03T06:50:28.385 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.386 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.387 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.388 DEBUG:boto:Params: {} 2022-09-03T06:50:28.390 DEBUG:boto:Token: None 2022-09-03T06:50:28.391 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj37 2022-09-03T06:50:28.392 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3Lsrs0iMpLesQs7tytaVRiJupqg= 2022-09-03T06:50:28.393 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3Lsrs0iMpLesQs7tytaVRiJupqg=', 'Content-Length': '0'} 2022-09-03T06:50:28.395 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005e806dc3094c2c8b-006312f934-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.396 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:28.397 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:28.397 DEBUG:boto:Method: HEAD 2022-09-03T06:50:28.398 DEBUG:boto:Path: /dsgvcy-75/obj37 2022-09-03T06:50:28.398 DEBUG:boto:Data: 2022-09-03T06:50:28.399 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.400 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.400 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.401 DEBUG:boto:Params: {} 2022-09-03T06:50:28.402 DEBUG:boto:Token: None 2022-09-03T06:50:28.402 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj37 2022-09-03T06:50:28.403 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3Lsrs0iMpLesQs7tytaVRiJupqg= 2022-09-03T06:50:28.403 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3Lsrs0iMpLesQs7tytaVRiJupqg=', 'Content-Length': '0'} 2022-09-03T06:50:28.411 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007ae5afa00f456a8d-006312f934-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.412 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj37 2022-09-03T06:50:28.413 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:28.414 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:28.423 DEBUG:boto:Method: GET 2022-09-03T06:50:28.424 DEBUG:boto:Path: /dsgvcy-75/obj37 2022-09-03T06:50:28.424 DEBUG:boto:Data: 2022-09-03T06:50:28.425 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.425 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.426 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.426 DEBUG:boto:Params: {} 2022-09-03T06:50:28.427 DEBUG:boto:Token: None 2022-09-03T06:50:28.428 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj37 2022-09-03T06:50:28.428 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2YGLK357pcxZgijO+WSd9FwPRUc= 2022-09-03T06:50:28.429 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2YGLK357pcxZgijO+WSd9FwPRUc=', 'Content-Length': '0'} 2022-09-03T06:50:28.431 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b826b1bf0826e84a-006312f934-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.432 DEBUG:boto:path=/dsgvcy-75/obj37 2022-09-03T06:50:28.433 DEBUG:boto:auth_path=/dsgvcy-75/obj37 2022-09-03T06:50:28.434 DEBUG:boto:Method: GET 2022-09-03T06:50:28.434 DEBUG:boto:Path: /dsgvcy-75/obj37 2022-09-03T06:50:28.435 DEBUG:boto:Data: 2022-09-03T06:50:28.436 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.436 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.437 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.437 DEBUG:boto:Params: {} 2022-09-03T06:50:28.438 DEBUG:boto:Token: None 2022-09-03T06:50:28.439 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj37 2022-09-03T06:50:28.439 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2YGLK357pcxZgijO+WSd9FwPRUc= 2022-09-03T06:50:28.440 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2YGLK357pcxZgijO+WSd9FwPRUc=', 'Content-Length': '0'} 2022-09-03T06:50:28.442 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000abc90e420a6ca279-006312f934-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.490 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:28.491 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:28.491 DEBUG:boto:path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:28.492 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:28.493 DEBUG:boto:Method: GET 2022-09-03T06:50:28.493 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj37 2022-09-03T06:50:28.494 DEBUG:boto:Data: 2022-09-03T06:50:28.494 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.495 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.495 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.496 DEBUG:boto:Params: {} 2022-09-03T06:50:28.496 DEBUG:boto:Token: None 2022-09-03T06:50:28.497 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/ 2022-09-03T06:50:28.498 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg= 2022-09-03T06:50:28.499 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg=', 'Content-Length': '0'} 2022-09-03T06:50:28.502 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004cc8c4c06eb8b680-006312f934-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.503 DEBUG:boto:b'dsgvcy-75obj371000falseobj372022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:28.504 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:28.505 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:28.505 DEBUG:boto:path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:28.506 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj37 2022-09-03T06:50:28.506 DEBUG:boto:Method: GET 2022-09-03T06:50:28.507 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj37 2022-09-03T06:50:28.507 DEBUG:boto:Data: 2022-09-03T06:50:28.508 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.509 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.509 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.510 DEBUG:boto:Params: {} 2022-09-03T06:50:28.510 DEBUG:boto:Token: None 2022-09-03T06:50:28.511 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/ 2022-09-03T06:50:28.512 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg= 2022-09-03T06:50:28.512 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg=', 'Content-Length': '0'} 2022-09-03T06:50:28.515 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004d1283e940dbb200-006312f934-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.516 DEBUG:boto:b'dsgvcy-75obj371000falseobj372022-09-03T06:47:49.518Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:28.517 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:50:28.517 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:28.518 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:28.519 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.519 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.520 DEBUG:boto:Method: GET 2022-09-03T06:50:28.520 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.521 DEBUG:boto:Data: 2022-09-03T06:50:28.521 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.522 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.523 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.523 DEBUG:boto:Params: {} 2022-09-03T06:50:28.524 DEBUG:boto:Token: None 2022-09-03T06:50:28.524 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.525 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k5dqB3lGyHllpj0KhrdLnsTDpTg= 2022-09-03T06:50:28.526 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k5dqB3lGyHllpj0KhrdLnsTDpTg=', 'Content-Length': '0'} 2022-09-03T06:50:28.528 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009fef1a2d1585493f-006312f934-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.574 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:28.575 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:28.576 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.578 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.579 DEBUG:boto:Method: GET 2022-09-03T06:50:28.580 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.580 DEBUG:boto:Data: 2022-09-03T06:50:28.581 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.582 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.582 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.583 DEBUG:boto:Params: {} 2022-09-03T06:50:28.584 DEBUG:boto:Token: None 2022-09-03T06:50:28.584 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.585 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k5dqB3lGyHllpj0KhrdLnsTDpTg= 2022-09-03T06:50:28.586 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k5dqB3lGyHllpj0KhrdLnsTDpTg=', 'Content-Length': '0'} 2022-09-03T06:50:28.589 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ddf4efc37f81cd2b-006312f934-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.590 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:28.591 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:28.591 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.592 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.593 DEBUG:boto:Method: HEAD 2022-09-03T06:50:28.593 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.594 DEBUG:boto:Data: 2022-09-03T06:50:28.595 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.595 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.596 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.596 DEBUG:boto:Params: {} 2022-09-03T06:50:28.597 DEBUG:boto:Token: None 2022-09-03T06:50:28.598 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.598 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8kg7jqOLonwQdxJA8t15lBNOL0Q= 2022-09-03T06:50:28.599 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8kg7jqOLonwQdxJA8t15lBNOL0Q=', 'Content-Length': '0'} 2022-09-03T06:50:28.603 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002fc7833f67d62329-006312f934-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.605 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:28.605 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:28.606 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.606 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.607 DEBUG:boto:Method: HEAD 2022-09-03T06:50:28.607 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.608 DEBUG:boto:Data: 2022-09-03T06:50:28.608 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.609 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.609 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.610 DEBUG:boto:Params: {} 2022-09-03T06:50:28.610 DEBUG:boto:Token: None 2022-09-03T06:50:28.611 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.612 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:8kg7jqOLonwQdxJA8t15lBNOL0Q= 2022-09-03T06:50:28.612 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:8kg7jqOLonwQdxJA8t15lBNOL0Q=', 'Content-Length': '0'} 2022-09-03T06:50:28.615 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000073fc7132b4e790f5-006312f934-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.616 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:50:28.616 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:28.617 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:28.617 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.618 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.619 DEBUG:boto:Method: GET 2022-09-03T06:50:28.619 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.620 DEBUG:boto:Data: 2022-09-03T06:50:28.620 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.621 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.622 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.622 DEBUG:boto:Params: {} 2022-09-03T06:50:28.623 DEBUG:boto:Token: None 2022-09-03T06:50:28.624 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.626 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k5dqB3lGyHllpj0KhrdLnsTDpTg= 2022-09-03T06:50:28.627 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k5dqB3lGyHllpj0KhrdLnsTDpTg=', 'Content-Length': '0'} 2022-09-03T06:50:28.629 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fd16f350e7289e27-006312f934-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.630 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:28.630 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:28.631 DEBUG:boto:path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.631 DEBUG:boto:auth_path=/dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.632 DEBUG:boto:Method: GET 2022-09-03T06:50:28.633 DEBUG:boto:Path: /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.633 DEBUG:boto:Data: 2022-09-03T06:50:28.634 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.635 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.635 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.636 DEBUG:boto:Params: {} 2022-09-03T06:50:28.637 DEBUG:boto:Token: None 2022-09-03T06:50:28.638 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj38?versionId=null 2022-09-03T06:50:28.638 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:k5dqB3lGyHllpj0KhrdLnsTDpTg= 2022-09-03T06:50:28.639 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:k5dqB3lGyHllpj0KhrdLnsTDpTg=', 'Content-Length': '0'} 2022-09-03T06:50:28.641 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e6249f67442b86f9-006312f934-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.642 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:28.643 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:28.643 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:28.644 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:28.645 DEBUG:boto:Method: GET 2022-09-03T06:50:28.645 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:50:28.646 DEBUG:boto:Data: 2022-09-03T06:50:28.646 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.647 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.647 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.648 DEBUG:boto:Params: {} 2022-09-03T06:50:28.648 DEBUG:boto:Token: None 2022-09-03T06:50:28.649 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/ 2022-09-03T06:50:28.650 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg= 2022-09-03T06:50:28.650 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg=', 'Content-Length': '0'} 2022-09-03T06:50:28.653 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006d431d65ed7369cd-006312f934-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.654 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:28.655 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:28.655 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:28.656 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:28.656 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:28.657 DEBUG:boto:Method: GET 2022-09-03T06:50:28.658 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:50:28.658 DEBUG:boto:Data: 2022-09-03T06:50:28.659 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.659 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.660 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.660 DEBUG:boto:Params: {} 2022-09-03T06:50:28.661 DEBUG:boto:Token: None 2022-09-03T06:50:28.661 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/ 2022-09-03T06:50:28.662 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg= 2022-09-03T06:50:28.663 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg=', 'Content-Length': '0'} 2022-09-03T06:50:28.666 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004be32a412e4ace39-006312f934-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.667 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:28.668 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:28.668 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:28.669 DEBUG:boto:Method: HEAD 2022-09-03T06:50:28.669 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:50:28.670 DEBUG:boto:Data: 2022-09-03T06:50:28.671 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.671 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.672 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.672 DEBUG:boto:Params: {} 2022-09-03T06:50:28.673 DEBUG:boto:Token: None 2022-09-03T06:50:28.673 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj38 2022-09-03T06:50:28.674 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F3hTNz4i7kGII6sl19O5m+QQjBw= 2022-09-03T06:50:28.675 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F3hTNz4i7kGII6sl19O5m+QQjBw=', 'Content-Length': '0'} 2022-09-03T06:50:28.677 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000939294912a6787d3-006312f934-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.678 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:28.679 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:28.679 DEBUG:boto:Method: HEAD 2022-09-03T06:50:28.680 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:50:28.680 DEBUG:boto:Data: 2022-09-03T06:50:28.681 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.681 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.682 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.682 DEBUG:boto:Params: {} 2022-09-03T06:50:28.683 DEBUG:boto:Token: None 2022-09-03T06:50:28.683 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj38 2022-09-03T06:50:28.684 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:F3hTNz4i7kGII6sl19O5m+QQjBw= 2022-09-03T06:50:28.685 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:F3hTNz4i7kGII6sl19O5m+QQjBw=', 'Content-Length': '0'} 2022-09-03T06:50:28.687 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e2d42dc7a4c60cf8-006312f934-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.687 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj38 2022-09-03T06:50:28.688 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:28.688 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:28.689 DEBUG:boto:Method: GET 2022-09-03T06:50:28.689 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:50:28.690 DEBUG:boto:Data: 2022-09-03T06:50:28.691 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.691 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.692 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.692 DEBUG:boto:Params: {} 2022-09-03T06:50:28.693 DEBUG:boto:Token: None 2022-09-03T06:50:28.693 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj38 2022-09-03T06:50:28.694 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9bUUSPi92Vi0vAf7mNaxO8YURpU= 2022-09-03T06:50:28.695 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9bUUSPi92Vi0vAf7mNaxO8YURpU=', 'Content-Length': '0'} 2022-09-03T06:50:28.697 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a55c5d2a1069511-006312f934-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.697 DEBUG:boto:path=/dsgvcy-75/obj38 2022-09-03T06:50:28.698 DEBUG:boto:auth_path=/dsgvcy-75/obj38 2022-09-03T06:50:28.698 DEBUG:boto:Method: GET 2022-09-03T06:50:28.699 DEBUG:boto:Path: /dsgvcy-75/obj38 2022-09-03T06:50:28.700 DEBUG:boto:Data: 2022-09-03T06:50:28.700 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.701 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.701 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.702 DEBUG:boto:Params: {} 2022-09-03T06:50:28.702 DEBUG:boto:Token: None 2022-09-03T06:50:28.703 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj38 2022-09-03T06:50:28.704 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9bUUSPi92Vi0vAf7mNaxO8YURpU= 2022-09-03T06:50:28.704 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9bUUSPi92Vi0vAf7mNaxO8YURpU=', 'Content-Length': '0'} 2022-09-03T06:50:28.706 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000200d20e37ba04ec9-006312f934-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.750 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:28.751 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:28.751 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:28.752 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:28.753 DEBUG:boto:Method: GET 2022-09-03T06:50:28.753 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:50:28.754 DEBUG:boto:Data: 2022-09-03T06:50:28.754 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.755 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.755 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.756 DEBUG:boto:Params: {} 2022-09-03T06:50:28.756 DEBUG:boto:Token: None 2022-09-03T06:50:28.757 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/ 2022-09-03T06:50:28.757 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg= 2022-09-03T06:50:28.758 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg=', 'Content-Length': '0'} 2022-09-03T06:50:28.761 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009ec24e555242d39a-006312f934-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.762 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:28.763 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:28.763 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:28.764 DEBUG:boto:path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:28.764 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj38 2022-09-03T06:50:28.765 DEBUG:boto:Method: GET 2022-09-03T06:50:28.765 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj38 2022-09-03T06:50:28.766 DEBUG:boto:Data: 2022-09-03T06:50:28.766 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.767 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.767 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.768 DEBUG:boto:Params: {} 2022-09-03T06:50:28.768 DEBUG:boto:Token: None 2022-09-03T06:50:28.769 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/ 2022-09-03T06:50:28.770 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg= 2022-09-03T06:50:28.770 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg=', 'Content-Length': '0'} 2022-09-03T06:50:28.773 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000336e142a32240a30-006312f934-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.774 DEBUG:boto:b'dsgvcy-75obj381000falseobj382022-09-03T06:40:53.174Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:28.775 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:50:28.775 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:28.776 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:28.776 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.777 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.777 DEBUG:boto:Method: GET 2022-09-03T06:50:28.778 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.778 DEBUG:boto:Data: 2022-09-03T06:50:28.779 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.780 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.780 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.781 DEBUG:boto:Params: {} 2022-09-03T06:50:28.782 DEBUG:boto:Token: None 2022-09-03T06:50:28.782 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.783 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3Qtpz9R8bpC+eYGqx3OdDzgx7Zc= 2022-09-03T06:50:28.784 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3Qtpz9R8bpC+eYGqx3OdDzgx7Zc=', 'Content-Length': '0'} 2022-09-03T06:50:28.786 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000935bdf641be12199-006312f934-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.787 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:28.787 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:28.789 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.789 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.790 DEBUG:boto:Method: GET 2022-09-03T06:50:28.790 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.791 DEBUG:boto:Data: 2022-09-03T06:50:28.791 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.792 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.792 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.793 DEBUG:boto:Params: {} 2022-09-03T06:50:28.794 DEBUG:boto:Token: None 2022-09-03T06:50:28.794 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.795 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3Qtpz9R8bpC+eYGqx3OdDzgx7Zc= 2022-09-03T06:50:28.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3Qtpz9R8bpC+eYGqx3OdDzgx7Zc=', 'Content-Length': '0'} 2022-09-03T06:50:28.798 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004a2ca16d4782c32f-006312f934-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.799 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:28.799 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:28.800 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.800 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.801 DEBUG:boto:Method: HEAD 2022-09-03T06:50:28.801 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.802 DEBUG:boto:Data: 2022-09-03T06:50:28.802 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.803 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.803 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.804 DEBUG:boto:Params: {} 2022-09-03T06:50:28.804 DEBUG:boto:Token: None 2022-09-03T06:50:28.805 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.806 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H+9OuGz5j7evgT1hjr/dvasHTPQ= 2022-09-03T06:50:28.806 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H+9OuGz5j7evgT1hjr/dvasHTPQ=', 'Content-Length': '0'} 2022-09-03T06:50:28.808 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000876de7b2e357e933-006312f934-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.809 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:28.809 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:28.810 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.811 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.811 DEBUG:boto:Method: HEAD 2022-09-03T06:50:28.812 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.812 DEBUG:boto:Data: 2022-09-03T06:50:28.813 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.813 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.814 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.814 DEBUG:boto:Params: {} 2022-09-03T06:50:28.815 DEBUG:boto:Token: None 2022-09-03T06:50:28.816 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.816 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:H+9OuGz5j7evgT1hjr/dvasHTPQ= 2022-09-03T06:50:28.817 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:H+9OuGz5j7evgT1hjr/dvasHTPQ=', 'Content-Length': '0'} 2022-09-03T06:50:28.819 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000728d39fce180f7e0-006312f934-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.820 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:50:28.821 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:28.821 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:28.822 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.823 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.824 DEBUG:boto:Method: GET 2022-09-03T06:50:28.825 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.826 DEBUG:boto:Data: 2022-09-03T06:50:28.827 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.828 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.829 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.830 DEBUG:boto:Params: {} 2022-09-03T06:50:28.831 DEBUG:boto:Token: None 2022-09-03T06:50:28.832 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.833 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3Qtpz9R8bpC+eYGqx3OdDzgx7Zc= 2022-09-03T06:50:28.834 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3Qtpz9R8bpC+eYGqx3OdDzgx7Zc=', 'Content-Length': '0'} 2022-09-03T06:50:28.838 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a7b4ad41eee5944a-006312f934-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.878 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:28.879 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:28.880 DEBUG:boto:path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.881 DEBUG:boto:auth_path=/dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.881 DEBUG:boto:Method: GET 2022-09-03T06:50:28.882 DEBUG:boto:Path: /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.882 DEBUG:boto:Data: 2022-09-03T06:50:28.883 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.884 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.884 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.885 DEBUG:boto:Params: {} 2022-09-03T06:50:28.886 DEBUG:boto:Token: None 2022-09-03T06:50:28.886 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj44?versionId=null 2022-09-03T06:50:28.887 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3Qtpz9R8bpC+eYGqx3OdDzgx7Zc= 2022-09-03T06:50:28.888 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3Qtpz9R8bpC+eYGqx3OdDzgx7Zc=', 'Content-Length': '0'} 2022-09-03T06:50:28.890 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000096b095d46bb97309-006312f934-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.891 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:28.891 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:28.892 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:28.892 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:28.893 DEBUG:boto:Method: GET 2022-09-03T06:50:28.894 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:50:28.894 DEBUG:boto:Data: 2022-09-03T06:50:28.895 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.895 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.896 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.896 DEBUG:boto:Params: {} 2022-09-03T06:50:28.897 DEBUG:boto:Token: None 2022-09-03T06:50:28.897 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/ 2022-09-03T06:50:28.898 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg= 2022-09-03T06:50:28.899 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg=', 'Content-Length': '0'} 2022-09-03T06:50:28.902 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f54448b8d6f77e31-006312f934-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.903 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:28.904 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:28.904 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:28.905 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:28.905 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:28.906 DEBUG:boto:Method: GET 2022-09-03T06:50:28.906 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:50:28.907 DEBUG:boto:Data: 2022-09-03T06:50:28.908 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.908 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.909 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.909 DEBUG:boto:Params: {} 2022-09-03T06:50:28.910 DEBUG:boto:Token: None 2022-09-03T06:50:28.911 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/ 2022-09-03T06:50:28.911 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg= 2022-09-03T06:50:28.912 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:r1PdCiKzHYu+ELVaNsCydhx/gkg=', 'Content-Length': '0'} 2022-09-03T06:50:28.914 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007e00bff1ce721b71-006312f934-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.958 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:28.973 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:28.974 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:28.975 DEBUG:boto:Method: HEAD 2022-09-03T06:50:28.976 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:50:28.976 DEBUG:boto:Data: 2022-09-03T06:50:28.977 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.978 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:28.979 DEBUG:boto:Port: 8001 2022-09-03T06:50:28.980 DEBUG:boto:Params: {} 2022-09-03T06:50:28.981 DEBUG:boto:Token: None 2022-09-03T06:50:28.982 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj44 2022-09-03T06:50:28.984 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0FiQJx7GLFJqHmFilcoJys86/aA= 2022-09-03T06:50:28.985 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0FiQJx7GLFJqHmFilcoJys86/aA=', 'Content-Length': '0'} 2022-09-03T06:50:28.988 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000067b01cb6179b2135-006312f934-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:28 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:28.989 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:28.990 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:28.991 DEBUG:boto:Method: HEAD 2022-09-03T06:50:28.992 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:50:28.993 DEBUG:boto:Data: 2022-09-03T06:50:28.994 DEBUG:boto:Headers: {} 2022-09-03T06:50:28.995 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:28.996 DEBUG:boto:Port: 8000 2022-09-03T06:50:28.997 DEBUG:boto:Params: {} 2022-09-03T06:50:28.998 DEBUG:boto:Token: None 2022-09-03T06:50:28.999 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:28 GMT /dsgvcy-75/obj44 2022-09-03T06:50:29.001 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:0FiQJx7GLFJqHmFilcoJys86/aA= 2022-09-03T06:50:29.002 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:28 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:0FiQJx7GLFJqHmFilcoJys86/aA=', 'Content-Length': '0'} 2022-09-03T06:50:29.004 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007646b3845e4e735d-006312f935-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.006 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj44 2022-09-03T06:50:29.007 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:29.008 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:29.009 DEBUG:boto:Method: GET 2022-09-03T06:50:29.010 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:50:29.011 DEBUG:boto:Data: 2022-09-03T06:50:29.012 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.013 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.014 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.015 DEBUG:boto:Params: {} 2022-09-03T06:50:29.016 DEBUG:boto:Token: None 2022-09-03T06:50:29.017 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj44 2022-09-03T06:50:29.018 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a6dT6K7g0MZLdnLo0aRLjyyZ8TU= 2022-09-03T06:50:29.020 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a6dT6K7g0MZLdnLo0aRLjyyZ8TU=', 'Content-Length': '0'} 2022-09-03T06:50:29.022 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000317a4e0263468a45-006312f935-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.024 DEBUG:boto:path=/dsgvcy-75/obj44 2022-09-03T06:50:29.025 DEBUG:boto:auth_path=/dsgvcy-75/obj44 2022-09-03T06:50:29.026 DEBUG:boto:Method: GET 2022-09-03T06:50:29.026 DEBUG:boto:Path: /dsgvcy-75/obj44 2022-09-03T06:50:29.027 DEBUG:boto:Data: 2022-09-03T06:50:29.027 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.028 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:29.028 DEBUG:boto:Port: 8000 2022-09-03T06:50:29.029 DEBUG:boto:Params: {} 2022-09-03T06:50:29.030 DEBUG:boto:Token: None 2022-09-03T06:50:29.030 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj44 2022-09-03T06:50:29.031 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a6dT6K7g0MZLdnLo0aRLjyyZ8TU= 2022-09-03T06:50:29.031 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a6dT6K7g0MZLdnLo0aRLjyyZ8TU=', 'Content-Length': '0'} 2022-09-03T06:50:29.034 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001ff9a3227ca431eb-006312f935-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.034 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:29.035 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:29.036 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:29.036 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:29.037 DEBUG:boto:Method: GET 2022-09-03T06:50:29.037 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:50:29.038 DEBUG:boto:Data: 2022-09-03T06:50:29.038 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.039 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.040 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.040 DEBUG:boto:Params: {} 2022-09-03T06:50:29.041 DEBUG:boto:Token: None 2022-09-03T06:50:29.041 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/ 2022-09-03T06:50:29.042 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I= 2022-09-03T06:50:29.043 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I=', 'Content-Length': '0'} 2022-09-03T06:50:29.046 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006139ef47c0326d23-006312f935-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.048 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:29.049 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:29.050 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:29.051 DEBUG:boto:path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:29.051 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj44 2022-09-03T06:50:29.052 DEBUG:boto:Method: GET 2022-09-03T06:50:29.052 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj44 2022-09-03T06:50:29.053 DEBUG:boto:Data: 2022-09-03T06:50:29.054 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.054 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:29.055 DEBUG:boto:Port: 8000 2022-09-03T06:50:29.055 DEBUG:boto:Params: {} 2022-09-03T06:50:29.056 DEBUG:boto:Token: None 2022-09-03T06:50:29.057 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/ 2022-09-03T06:50:29.057 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I= 2022-09-03T06:50:29.058 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I=', 'Content-Length': '0'} 2022-09-03T06:50:29.061 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b027ffcfe37a2401-006312f935-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.061 DEBUG:boto:b'dsgvcy-75obj441000falseobj442022-09-03T06:36:05.834Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:29.062 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:50:29.063 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:29.063 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:29.064 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.065 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.065 DEBUG:boto:Method: GET 2022-09-03T06:50:29.066 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.066 DEBUG:boto:Data: 2022-09-03T06:50:29.067 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.068 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.068 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.069 DEBUG:boto:Params: {} 2022-09-03T06:50:29.069 DEBUG:boto:Token: None 2022-09-03T06:50:29.070 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.071 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:T5pnOuxRHnYyQgRKgUy3L5YU91A= 2022-09-03T06:50:29.071 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:T5pnOuxRHnYyQgRKgUy3L5YU91A=', 'Content-Length': '0'} 2022-09-03T06:50:29.074 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d2d5c5c0d600558a-006312f935-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.118 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:29.120 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:29.121 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.122 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.122 DEBUG:boto:Method: GET 2022-09-03T06:50:29.123 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.124 DEBUG:boto:Data: 2022-09-03T06:50:29.125 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.126 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:29.127 DEBUG:boto:Port: 8000 2022-09-03T06:50:29.127 DEBUG:boto:Params: {} 2022-09-03T06:50:29.128 DEBUG:boto:Token: None 2022-09-03T06:50:29.129 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.130 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:T5pnOuxRHnYyQgRKgUy3L5YU91A= 2022-09-03T06:50:29.131 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:T5pnOuxRHnYyQgRKgUy3L5YU91A=', 'Content-Length': '0'} 2022-09-03T06:50:29.134 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fb6a6c3d222a964d-006312f935-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.135 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:29.135 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:29.136 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.137 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.138 DEBUG:boto:Method: HEAD 2022-09-03T06:50:29.138 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.139 DEBUG:boto:Data: 2022-09-03T06:50:29.139 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.140 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.141 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.141 DEBUG:boto:Params: {} 2022-09-03T06:50:29.142 DEBUG:boto:Token: None 2022-09-03T06:50:29.143 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.144 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Aeb77bgArPa650nlFxJe7wh4wy4= 2022-09-03T06:50:29.145 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Aeb77bgArPa650nlFxJe7wh4wy4=', 'Content-Length': '0'} 2022-09-03T06:50:29.149 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009589a570a974c8f2-006312f935-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.151 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:29.152 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:29.153 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.154 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.155 DEBUG:boto:Method: HEAD 2022-09-03T06:50:29.155 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.156 DEBUG:boto:Data: 2022-09-03T06:50:29.157 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.158 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:29.159 DEBUG:boto:Port: 8000 2022-09-03T06:50:29.159 DEBUG:boto:Params: {} 2022-09-03T06:50:29.160 DEBUG:boto:Token: None 2022-09-03T06:50:29.161 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.162 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Aeb77bgArPa650nlFxJe7wh4wy4= 2022-09-03T06:50:29.163 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Aeb77bgArPa650nlFxJe7wh4wy4=', 'Content-Length': '0'} 2022-09-03T06:50:29.166 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000746b55e293fb7aa1-006312f935-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.167 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:50:29.167 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:29.168 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:29.169 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.170 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.171 DEBUG:boto:Method: GET 2022-09-03T06:50:29.172 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.173 DEBUG:boto:Data: 2022-09-03T06:50:29.174 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.174 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.175 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.176 DEBUG:boto:Params: {} 2022-09-03T06:50:29.177 DEBUG:boto:Token: None 2022-09-03T06:50:29.178 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.178 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:T5pnOuxRHnYyQgRKgUy3L5YU91A= 2022-09-03T06:50:29.179 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:T5pnOuxRHnYyQgRKgUy3L5YU91A=', 'Content-Length': '0'} 2022-09-03T06:50:29.181 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000000c0d0cae2943c74-006312f935-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.182 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:29.183 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:29.184 DEBUG:boto:path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.184 DEBUG:boto:auth_path=/dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.185 DEBUG:boto:Method: GET 2022-09-03T06:50:29.185 DEBUG:boto:Path: /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.186 DEBUG:boto:Data: 2022-09-03T06:50:29.187 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.187 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:29.188 DEBUG:boto:Port: 8000 2022-09-03T06:50:29.189 DEBUG:boto:Params: {} 2022-09-03T06:50:29.191 DEBUG:boto:Token: None 2022-09-03T06:50:29.192 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj46?versionId=null 2022-09-03T06:50:29.193 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:T5pnOuxRHnYyQgRKgUy3L5YU91A= 2022-09-03T06:50:29.194 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:T5pnOuxRHnYyQgRKgUy3L5YU91A=', 'Content-Length': '0'} 2022-09-03T06:50:29.197 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e45e95c469fe5309-006312f935-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.198 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:29.199 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:29.199 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:29.210 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:29.211 DEBUG:boto:Method: GET 2022-09-03T06:50:29.212 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:50:29.213 DEBUG:boto:Data: 2022-09-03T06:50:29.214 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.214 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.215 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.216 DEBUG:boto:Params: {} 2022-09-03T06:50:29.216 DEBUG:boto:Token: None 2022-09-03T06:50:29.217 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/ 2022-09-03T06:50:29.219 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I= 2022-09-03T06:50:29.220 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I=', 'Content-Length': '0'} 2022-09-03T06:50:29.224 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000026c2a398cdfe541-006312f935-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.226 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:29.227 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:29.228 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:29.229 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:29.230 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:29.231 DEBUG:boto:Method: GET 2022-09-03T06:50:29.232 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:50:29.233 DEBUG:boto:Data: 2022-09-03T06:50:29.234 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.235 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:29.236 DEBUG:boto:Port: 8000 2022-09-03T06:50:29.237 DEBUG:boto:Params: {} 2022-09-03T06:50:29.238 DEBUG:boto:Token: None 2022-09-03T06:50:29.239 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/ 2022-09-03T06:50:29.240 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I= 2022-09-03T06:50:29.243 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I=', 'Content-Length': '0'} 2022-09-03T06:50:29.245 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000029e93c8977668a88-006312f935-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.247 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:29.250 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:29.251 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:29.251 DEBUG:boto:Method: HEAD 2022-09-03T06:50:29.252 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:50:29.253 DEBUG:boto:Data: 2022-09-03T06:50:29.254 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.255 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.256 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.257 DEBUG:boto:Params: {} 2022-09-03T06:50:29.258 DEBUG:boto:Token: None 2022-09-03T06:50:29.259 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj46 2022-09-03T06:50:29.259 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/jlp6MUPqXtNtja9tvdXajHbNwE= 2022-09-03T06:50:29.260 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/jlp6MUPqXtNtja9tvdXajHbNwE=', 'Content-Length': '0'} 2022-09-03T06:50:29.263 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000082bed1d5708af838-006312f935-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.264 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:29.265 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:29.266 DEBUG:boto:Method: HEAD 2022-09-03T06:50:29.267 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:50:29.267 DEBUG:boto:Data: 2022-09-03T06:50:29.268 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.268 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:29.269 DEBUG:boto:Port: 8000 2022-09-03T06:50:29.269 DEBUG:boto:Params: {} 2022-09-03T06:50:29.270 DEBUG:boto:Token: None 2022-09-03T06:50:29.271 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj46 2022-09-03T06:50:29.272 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/jlp6MUPqXtNtja9tvdXajHbNwE= 2022-09-03T06:50:29.274 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/jlp6MUPqXtNtja9tvdXajHbNwE=', 'Content-Length': '0'} 2022-09-03T06:50:29.277 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000088404d6eca3844bd-006312f935-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.278 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj46 2022-09-03T06:50:29.278 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:29.279 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:29.279 DEBUG:boto:Method: GET 2022-09-03T06:50:29.280 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:50:29.280 DEBUG:boto:Data: 2022-09-03T06:50:29.281 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.282 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.282 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.283 DEBUG:boto:Params: {} 2022-09-03T06:50:29.283 DEBUG:boto:Token: None 2022-09-03T06:50:29.284 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj46 2022-09-03T06:50:29.285 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1pTUAD47U2ThOMPK6miXUO3/PTg= 2022-09-03T06:50:29.286 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1pTUAD47U2ThOMPK6miXUO3/PTg=', 'Content-Length': '0'} 2022-09-03T06:50:29.288 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000072f7f6401eb1acab-006312f935-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.289 DEBUG:boto:path=/dsgvcy-75/obj46 2022-09-03T06:50:29.290 DEBUG:boto:auth_path=/dsgvcy-75/obj46 2022-09-03T06:50:29.291 DEBUG:boto:Method: GET 2022-09-03T06:50:29.293 DEBUG:boto:Path: /dsgvcy-75/obj46 2022-09-03T06:50:29.294 DEBUG:boto:Data: 2022-09-03T06:50:29.295 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.296 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:29.297 DEBUG:boto:Port: 8000 2022-09-03T06:50:29.298 DEBUG:boto:Params: {} 2022-09-03T06:50:29.299 DEBUG:boto:Token: None 2022-09-03T06:50:29.300 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj46 2022-09-03T06:50:29.301 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:1pTUAD47U2ThOMPK6miXUO3/PTg= 2022-09-03T06:50:29.302 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:1pTUAD47U2ThOMPK6miXUO3/PTg=', 'Content-Length': '0'} 2022-09-03T06:50:29.305 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c5617df3f1aa3d2-006312f935-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.306 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:29.307 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:29.308 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:29.310 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:29.311 DEBUG:boto:Method: GET 2022-09-03T06:50:29.312 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:50:29.313 DEBUG:boto:Data: 2022-09-03T06:50:29.314 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.315 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.316 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.317 DEBUG:boto:Params: {} 2022-09-03T06:50:29.318 DEBUG:boto:Token: None 2022-09-03T06:50:29.319 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/ 2022-09-03T06:50:29.320 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I= 2022-09-03T06:50:29.320 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I=', 'Content-Length': '0'} 2022-09-03T06:50:29.324 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000085fbb4ae8839daea-006312f935-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.328 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:29.330 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:29.331 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:29.331 DEBUG:boto:path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:29.332 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj46 2022-09-03T06:50:29.333 DEBUG:boto:Method: GET 2022-09-03T06:50:29.333 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj46 2022-09-03T06:50:29.334 DEBUG:boto:Data: 2022-09-03T06:50:29.335 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.335 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:29.336 DEBUG:boto:Port: 8000 2022-09-03T06:50:29.336 DEBUG:boto:Params: {} 2022-09-03T06:50:29.337 DEBUG:boto:Token: None 2022-09-03T06:50:29.337 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/ 2022-09-03T06:50:29.338 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I= 2022-09-03T06:50:29.339 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I=', 'Content-Length': '0'} 2022-09-03T06:50:29.341 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fd8bfe27ed89c5c5-006312f935-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.342 DEBUG:boto:b'dsgvcy-75obj461000falseobj462022-09-03T06:41:59.675Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:29.343 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:50:29.344 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:29.345 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:29.345 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.346 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.346 DEBUG:boto:Method: GET 2022-09-03T06:50:29.347 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.348 DEBUG:boto:Data: 2022-09-03T06:50:29.348 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.349 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.350 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.350 DEBUG:boto:Params: {} 2022-09-03T06:50:29.351 DEBUG:boto:Token: None 2022-09-03T06:50:29.352 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.352 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ot5z6d4+IyueCpvWXq2kBYXEwdo= 2022-09-03T06:50:29.353 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ot5z6d4+IyueCpvWXq2kBYXEwdo=', 'Content-Length': '0'} 2022-09-03T06:50:29.355 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000d52d89e3ecbc6fc-006312f935-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.402 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:29.403 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:29.405 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.406 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.407 DEBUG:boto:Method: GET 2022-09-03T06:50:29.407 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.408 DEBUG:boto:Data: 2022-09-03T06:50:29.409 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.410 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:29.411 DEBUG:boto:Port: 8000 2022-09-03T06:50:29.412 DEBUG:boto:Params: {} 2022-09-03T06:50:29.413 DEBUG:boto:Token: None 2022-09-03T06:50:29.415 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.416 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ot5z6d4+IyueCpvWXq2kBYXEwdo= 2022-09-03T06:50:29.417 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ot5z6d4+IyueCpvWXq2kBYXEwdo=', 'Content-Length': '0'} 2022-09-03T06:50:29.421 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000047d91310d98516b-006312f935-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.422 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:29.423 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:29.423 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.424 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.424 DEBUG:boto:Method: HEAD 2022-09-03T06:50:29.425 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.425 DEBUG:boto:Data: 2022-09-03T06:50:29.426 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.426 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.427 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.427 DEBUG:boto:Params: {} 2022-09-03T06:50:29.428 DEBUG:boto:Token: None 2022-09-03T06:50:29.428 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.429 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bwWhLJ6LViK9eE2tzeH6JBoNum8= 2022-09-03T06:50:29.430 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bwWhLJ6LViK9eE2tzeH6JBoNum8=', 'Content-Length': '0'} 2022-09-03T06:50:29.432 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000084d6fc7ae959feb5-006312f935-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.433 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:29.433 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:29.434 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.435 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.435 DEBUG:boto:Method: HEAD 2022-09-03T06:50:29.436 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.436 DEBUG:boto:Data: 2022-09-03T06:50:29.437 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.437 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:29.438 DEBUG:boto:Port: 8000 2022-09-03T06:50:29.439 DEBUG:boto:Params: {} 2022-09-03T06:50:29.440 DEBUG:boto:Token: None 2022-09-03T06:50:29.441 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.442 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bwWhLJ6LViK9eE2tzeH6JBoNum8= 2022-09-03T06:50:29.442 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bwWhLJ6LViK9eE2tzeH6JBoNum8=', 'Content-Length': '0'} 2022-09-03T06:50:29.446 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004e0a6b86640a3f06-006312f935-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.447 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:50:29.448 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:29.449 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:29.450 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.451 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.452 DEBUG:boto:Method: GET 2022-09-03T06:50:29.453 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.455 DEBUG:boto:Data: 2022-09-03T06:50:29.456 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.457 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.459 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.460 DEBUG:boto:Params: {} 2022-09-03T06:50:29.461 DEBUG:boto:Token: None 2022-09-03T06:50:29.463 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.464 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ot5z6d4+IyueCpvWXq2kBYXEwdo= 2022-09-03T06:50:29.465 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ot5z6d4+IyueCpvWXq2kBYXEwdo=', 'Content-Length': '0'} 2022-09-03T06:50:29.469 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000190412f6d0e23bb5-006312f935-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.471 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:29.473 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:29.474 DEBUG:boto:path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.475 DEBUG:boto:auth_path=/dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.476 DEBUG:boto:Method: GET 2022-09-03T06:50:29.477 DEBUG:boto:Path: /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.478 DEBUG:boto:Data: 2022-09-03T06:50:29.480 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.481 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:29.482 DEBUG:boto:Port: 8000 2022-09-03T06:50:29.483 DEBUG:boto:Params: {} 2022-09-03T06:50:29.485 DEBUG:boto:Token: None 2022-09-03T06:50:29.486 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj51?versionId=null 2022-09-03T06:50:29.487 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Ot5z6d4+IyueCpvWXq2kBYXEwdo= 2022-09-03T06:50:29.489 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Ot5z6d4+IyueCpvWXq2kBYXEwdo=', 'Content-Length': '0'} 2022-09-03T06:50:29.491 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000594d4433da3db53a-006312f935-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.493 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:29.494 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:29.494 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:29.495 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:29.496 DEBUG:boto:Method: GET 2022-09-03T06:50:29.496 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:50:29.497 DEBUG:boto:Data: 2022-09-03T06:50:29.498 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.498 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.499 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.500 DEBUG:boto:Params: {} 2022-09-03T06:50:29.501 DEBUG:boto:Token: None 2022-09-03T06:50:29.502 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/ 2022-09-03T06:50:29.503 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I= 2022-09-03T06:50:29.503 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I=', 'Content-Length': '0'} 2022-09-03T06:50:29.506 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000da0c9fc4fd6b6e18-006312f935-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.507 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:29.508 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:29.509 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:29.509 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:29.510 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:29.511 DEBUG:boto:Method: GET 2022-09-03T06:50:29.512 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:50:29.513 DEBUG:boto:Data: 2022-09-03T06:50:29.514 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.515 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:29.516 DEBUG:boto:Port: 8000 2022-09-03T06:50:29.517 DEBUG:boto:Params: {} 2022-09-03T06:50:29.519 DEBUG:boto:Token: None 2022-09-03T06:50:29.520 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/ 2022-09-03T06:50:29.520 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I= 2022-09-03T06:50:29.521 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I=', 'Content-Length': '0'} 2022-09-03T06:50:29.524 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006e9ceb3bc5bd6d0c-006312f935-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.525 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:29.526 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:29.526 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:29.527 DEBUG:boto:Method: HEAD 2022-09-03T06:50:29.528 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:50:29.528 DEBUG:boto:Data: 2022-09-03T06:50:29.529 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.529 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.530 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.530 DEBUG:boto:Params: {} 2022-09-03T06:50:29.531 DEBUG:boto:Token: None 2022-09-03T06:50:29.532 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj51 2022-09-03T06:50:29.532 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kdPVRSq6yTp3kA0+c5v9SlqFtkw= 2022-09-03T06:50:29.533 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kdPVRSq6yTp3kA0+c5v9SlqFtkw=', 'Content-Length': '0'} 2022-09-03T06:50:29.535 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a6fad23b9e17f02a-006312f935-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.536 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:29.537 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:29.537 DEBUG:boto:Method: HEAD 2022-09-03T06:50:29.538 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:50:29.539 DEBUG:boto:Data: 2022-09-03T06:50:29.539 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.540 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:29.540 DEBUG:boto:Port: 8000 2022-09-03T06:50:29.541 DEBUG:boto:Params: {} 2022-09-03T06:50:29.542 DEBUG:boto:Token: None 2022-09-03T06:50:29.543 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj51 2022-09-03T06:50:29.544 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:kdPVRSq6yTp3kA0+c5v9SlqFtkw= 2022-09-03T06:50:29.545 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:kdPVRSq6yTp3kA0+c5v9SlqFtkw=', 'Content-Length': '0'} 2022-09-03T06:50:29.548 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000689145d490bba06d-006312f935-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.549 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj51 2022-09-03T06:50:29.550 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:29.551 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:29.552 DEBUG:boto:Method: GET 2022-09-03T06:50:29.553 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:50:29.554 DEBUG:boto:Data: 2022-09-03T06:50:29.555 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.556 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.557 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.558 DEBUG:boto:Params: {} 2022-09-03T06:50:29.559 DEBUG:boto:Token: None 2022-09-03T06:50:29.560 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj51 2022-09-03T06:50:29.560 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xAHlCX9x3Qx27dlaNLlJGVC8saU= 2022-09-03T06:50:29.561 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xAHlCX9x3Qx27dlaNLlJGVC8saU=', 'Content-Length': '0'} 2022-09-03T06:50:29.564 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2bdfa33e1f6ba28-006312f935-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.564 DEBUG:boto:path=/dsgvcy-75/obj51 2022-09-03T06:50:29.565 DEBUG:boto:auth_path=/dsgvcy-75/obj51 2022-09-03T06:50:29.566 DEBUG:boto:Method: GET 2022-09-03T06:50:29.566 DEBUG:boto:Path: /dsgvcy-75/obj51 2022-09-03T06:50:29.567 DEBUG:boto:Data: 2022-09-03T06:50:29.568 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.568 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:29.569 DEBUG:boto:Port: 8000 2022-09-03T06:50:29.570 DEBUG:boto:Params: {} 2022-09-03T06:50:29.571 DEBUG:boto:Token: None 2022-09-03T06:50:29.572 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj51 2022-09-03T06:50:29.573 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xAHlCX9x3Qx27dlaNLlJGVC8saU= 2022-09-03T06:50:29.574 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xAHlCX9x3Qx27dlaNLlJGVC8saU=', 'Content-Length': '0'} 2022-09-03T06:50:29.579 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000c72ff16f1c6d011-006312f935-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.622 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:29.623 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:29.624 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:29.624 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:29.625 DEBUG:boto:Method: GET 2022-09-03T06:50:29.625 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:50:29.626 DEBUG:boto:Data: 2022-09-03T06:50:29.626 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.627 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.627 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.628 DEBUG:boto:Params: {} 2022-09-03T06:50:29.629 DEBUG:boto:Token: None 2022-09-03T06:50:29.629 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/ 2022-09-03T06:50:29.630 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I= 2022-09-03T06:50:29.631 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I=', 'Content-Length': '0'} 2022-09-03T06:50:29.633 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000dd9053f0e6b00a6b-006312f935-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.634 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:29.635 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:29.636 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:29.637 DEBUG:boto:path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:29.637 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj51 2022-09-03T06:50:29.638 DEBUG:boto:Method: GET 2022-09-03T06:50:29.639 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj51 2022-09-03T06:50:29.640 DEBUG:boto:Data: 2022-09-03T06:50:29.641 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.642 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:29.643 DEBUG:boto:Port: 8000 2022-09-03T06:50:29.644 DEBUG:boto:Params: {} 2022-09-03T06:50:29.645 DEBUG:boto:Token: None 2022-09-03T06:50:29.645 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/ 2022-09-03T06:50:29.647 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I= 2022-09-03T06:50:29.648 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I=', 'Content-Length': '0'} 2022-09-03T06:50:29.651 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b8bb9848e52a2ac7-006312f935-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.652 DEBUG:boto:b'dsgvcy-75obj511000falseobj512022-09-03T06:37:27.160Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:29.653 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:50:29.653 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:29.654 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:29.655 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.655 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.656 DEBUG:boto:Method: GET 2022-09-03T06:50:29.656 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.657 DEBUG:boto:Data: 2022-09-03T06:50:29.657 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.658 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.659 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.659 DEBUG:boto:Params: {} 2022-09-03T06:50:29.661 DEBUG:boto:Token: None 2022-09-03T06:50:29.662 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.663 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pTJfYxs5o4PycuBvOXETQuTA10c= 2022-09-03T06:50:29.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pTJfYxs5o4PycuBvOXETQuTA10c=', 'Content-Length': '0'} 2022-09-03T06:50:29.668 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001cc4cd8806106078-006312f935-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.670 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:29.671 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:29.672 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.672 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.673 DEBUG:boto:Method: GET 2022-09-03T06:50:29.675 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.677 DEBUG:boto:Data: 2022-09-03T06:50:29.678 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.680 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:29.681 DEBUG:boto:Port: 8000 2022-09-03T06:50:29.683 DEBUG:boto:Params: {} 2022-09-03T06:50:29.684 DEBUG:boto:Token: None 2022-09-03T06:50:29.686 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.688 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pTJfYxs5o4PycuBvOXETQuTA10c= 2022-09-03T06:50:29.689 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pTJfYxs5o4PycuBvOXETQuTA10c=', 'Content-Length': '0'} 2022-09-03T06:50:29.691 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a54d1f0e3f31daca-006312f935-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.693 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:29.693 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:29.694 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.695 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.695 DEBUG:boto:Method: HEAD 2022-09-03T06:50:29.696 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.696 DEBUG:boto:Data: 2022-09-03T06:50:29.697 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.698 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.699 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.700 DEBUG:boto:Params: {} 2022-09-03T06:50:29.701 DEBUG:boto:Token: None 2022-09-03T06:50:29.702 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.703 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N1vjLUC0GLIDlmpgMSQ02tFbpFc= 2022-09-03T06:50:29.705 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N1vjLUC0GLIDlmpgMSQ02tFbpFc=', 'Content-Length': '0'} 2022-09-03T06:50:29.707 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003334f2e0e5980fdc-006312f935-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.708 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:29.708 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:29.709 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.709 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.710 DEBUG:boto:Method: HEAD 2022-09-03T06:50:29.710 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.711 DEBUG:boto:Data: 2022-09-03T06:50:29.711 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.712 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:29.712 DEBUG:boto:Port: 8000 2022-09-03T06:50:29.713 DEBUG:boto:Params: {} 2022-09-03T06:50:29.713 DEBUG:boto:Token: None 2022-09-03T06:50:29.714 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.715 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:N1vjLUC0GLIDlmpgMSQ02tFbpFc= 2022-09-03T06:50:29.715 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:N1vjLUC0GLIDlmpgMSQ02tFbpFc=', 'Content-Length': '0'} 2022-09-03T06:50:29.717 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008fb4eb669ec3de13-006312f935-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.718 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:50:29.719 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:29.719 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:29.720 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.721 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.721 DEBUG:boto:Method: GET 2022-09-03T06:50:29.722 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.722 DEBUG:boto:Data: 2022-09-03T06:50:29.723 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.724 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.724 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.725 DEBUG:boto:Params: {} 2022-09-03T06:50:29.726 DEBUG:boto:Token: None 2022-09-03T06:50:29.726 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.727 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pTJfYxs5o4PycuBvOXETQuTA10c= 2022-09-03T06:50:29.728 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pTJfYxs5o4PycuBvOXETQuTA10c=', 'Content-Length': '0'} 2022-09-03T06:50:29.730 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000535a078229df0dab-006312f935-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.775 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:29.776 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:29.778 DEBUG:boto:path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.779 DEBUG:boto:auth_path=/dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.780 DEBUG:boto:Method: GET 2022-09-03T06:50:29.781 DEBUG:boto:Path: /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.782 DEBUG:boto:Data: 2022-09-03T06:50:29.783 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.784 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:29.785 DEBUG:boto:Port: 8000 2022-09-03T06:50:29.786 DEBUG:boto:Params: {} 2022-09-03T06:50:29.787 DEBUG:boto:Token: None 2022-09-03T06:50:29.788 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj55?versionId=null 2022-09-03T06:50:29.789 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pTJfYxs5o4PycuBvOXETQuTA10c= 2022-09-03T06:50:29.790 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pTJfYxs5o4PycuBvOXETQuTA10c=', 'Content-Length': '0'} 2022-09-03T06:50:29.793 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000e5e9d9411fc992b-006312f935-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.795 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:29.796 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:29.797 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:29.798 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:29.799 DEBUG:boto:Method: GET 2022-09-03T06:50:29.800 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:50:29.801 DEBUG:boto:Data: 2022-09-03T06:50:29.802 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.803 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.804 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.805 DEBUG:boto:Params: {} 2022-09-03T06:50:29.806 DEBUG:boto:Token: None 2022-09-03T06:50:29.807 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/ 2022-09-03T06:50:29.809 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I= 2022-09-03T06:50:29.810 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I=', 'Content-Length': '0'} 2022-09-03T06:50:29.813 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000002899a19a5a1b3174-006312f935-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.815 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:29.816 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:29.817 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:29.818 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:29.819 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:29.820 DEBUG:boto:Method: GET 2022-09-03T06:50:29.822 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:50:29.824 DEBUG:boto:Data: 2022-09-03T06:50:29.825 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.827 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:29.828 DEBUG:boto:Port: 8000 2022-09-03T06:50:29.829 DEBUG:boto:Params: {} 2022-09-03T06:50:29.830 DEBUG:boto:Token: None 2022-09-03T06:50:29.831 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/ 2022-09-03T06:50:29.832 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I= 2022-09-03T06:50:29.833 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I=', 'Content-Length': '0'} 2022-09-03T06:50:29.836 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007b9f97a066babcf1-006312f935-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.837 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:29.838 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:29.838 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:29.839 DEBUG:boto:Method: HEAD 2022-09-03T06:50:29.839 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:50:29.840 DEBUG:boto:Data: 2022-09-03T06:50:29.840 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.841 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.841 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.842 DEBUG:boto:Params: {} 2022-09-03T06:50:29.843 DEBUG:boto:Token: None 2022-09-03T06:50:29.843 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj55 2022-09-03T06:50:29.844 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/f1CYZ/xRKUOq76WYXx7B1VMmNc= 2022-09-03T06:50:29.845 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/f1CYZ/xRKUOq76WYXx7B1VMmNc=', 'Content-Length': '0'} 2022-09-03T06:50:29.847 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007dcfe2053ec1ef62-006312f935-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.848 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:29.848 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:29.849 DEBUG:boto:Method: HEAD 2022-09-03T06:50:29.850 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:50:29.850 DEBUG:boto:Data: 2022-09-03T06:50:29.851 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.851 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:29.852 DEBUG:boto:Port: 8000 2022-09-03T06:50:29.853 DEBUG:boto:Params: {} 2022-09-03T06:50:29.853 DEBUG:boto:Token: None 2022-09-03T06:50:29.854 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj55 2022-09-03T06:50:29.855 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:/f1CYZ/xRKUOq76WYXx7B1VMmNc= 2022-09-03T06:50:29.855 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:/f1CYZ/xRKUOq76WYXx7B1VMmNc=', 'Content-Length': '0'} 2022-09-03T06:50:29.858 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002068116b1eff6e77-006312f935-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.859 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj55 2022-09-03T06:50:29.859 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:29.860 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:29.860 DEBUG:boto:Method: GET 2022-09-03T06:50:29.861 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:50:29.861 DEBUG:boto:Data: 2022-09-03T06:50:29.862 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.862 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.863 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.864 DEBUG:boto:Params: {} 2022-09-03T06:50:29.865 DEBUG:boto:Token: None 2022-09-03T06:50:29.866 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj55 2022-09-03T06:50:29.867 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TO/wOhLIrWUnNfevkq98BJRAgZU= 2022-09-03T06:50:29.868 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TO/wOhLIrWUnNfevkq98BJRAgZU=', 'Content-Length': '0'} 2022-09-03T06:50:29.871 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b45d095f3eb35a4d-006312f935-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.914 DEBUG:boto:path=/dsgvcy-75/obj55 2022-09-03T06:50:29.915 DEBUG:boto:auth_path=/dsgvcy-75/obj55 2022-09-03T06:50:29.916 DEBUG:boto:Method: GET 2022-09-03T06:50:29.917 DEBUG:boto:Path: /dsgvcy-75/obj55 2022-09-03T06:50:29.919 DEBUG:boto:Data: 2022-09-03T06:50:29.919 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.920 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:29.921 DEBUG:boto:Port: 8000 2022-09-03T06:50:29.922 DEBUG:boto:Params: {} 2022-09-03T06:50:29.924 DEBUG:boto:Token: None 2022-09-03T06:50:29.925 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj55 2022-09-03T06:50:29.927 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TO/wOhLIrWUnNfevkq98BJRAgZU= 2022-09-03T06:50:29.928 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TO/wOhLIrWUnNfevkq98BJRAgZU=', 'Content-Length': '0'} 2022-09-03T06:50:29.931 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:36:05 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000aef394c4b78c4ef1-006312f935-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.933 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:29.936 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:29.938 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:29.940 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:29.942 DEBUG:boto:Method: GET 2022-09-03T06:50:29.943 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:50:29.944 DEBUG:boto:Data: 2022-09-03T06:50:29.945 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.946 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.947 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.948 DEBUG:boto:Params: {} 2022-09-03T06:50:29.949 DEBUG:boto:Token: None 2022-09-03T06:50:29.951 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/ 2022-09-03T06:50:29.952 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I= 2022-09-03T06:50:29.954 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I=', 'Content-Length': '0'} 2022-09-03T06:50:29.957 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e786b735dcb51595-006312f935-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.958 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:29.960 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:29.961 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:29.962 DEBUG:boto:path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:29.963 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj55 2022-09-03T06:50:29.964 DEBUG:boto:Method: GET 2022-09-03T06:50:29.965 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj55 2022-09-03T06:50:29.966 DEBUG:boto:Data: 2022-09-03T06:50:29.967 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.968 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:29.969 DEBUG:boto:Port: 8000 2022-09-03T06:50:29.970 DEBUG:boto:Params: {} 2022-09-03T06:50:29.971 DEBUG:boto:Token: None 2022-09-03T06:50:29.972 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/ 2022-09-03T06:50:29.973 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I= 2022-09-03T06:50:29.975 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:xICeh0CeY1qJ3XUGomb9/pLau+I=', 'Content-Length': '0'} 2022-09-03T06:50:29.978 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000008fb65e3a5cf18b5d-006312f935-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:29.979 DEBUG:boto:b'dsgvcy-75obj551000falseobj552022-09-03T06:36:05.866Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:29.981 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:50:29.982 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:29.983 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:29.984 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:29.985 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:29.986 DEBUG:boto:Method: GET 2022-09-03T06:50:29.987 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:29.988 DEBUG:boto:Data: 2022-09-03T06:50:29.989 DEBUG:boto:Headers: {} 2022-09-03T06:50:29.990 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:29.991 DEBUG:boto:Port: 8001 2022-09-03T06:50:29.992 DEBUG:boto:Params: {} 2022-09-03T06:50:29.993 DEBUG:boto:Token: None 2022-09-03T06:50:29.994 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:29 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:29.995 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:5PvcCsL8ND4eu1PIKoIz2wWdlO4= 2022-09-03T06:50:29.997 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:29 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:5PvcCsL8ND4eu1PIKoIz2wWdlO4=', 'Content-Length': '0'} 2022-09-03T06:50:30.000 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008586737ee5f97291-006312f935-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:29 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.001 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:30.002 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:30.003 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:30.004 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:30.005 DEBUG:boto:Method: GET 2022-09-03T06:50:30.007 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:30.007 DEBUG:boto:Data: 2022-09-03T06:50:30.008 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.009 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.010 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.011 DEBUG:boto:Params: {} 2022-09-03T06:50:30.013 DEBUG:boto:Token: None 2022-09-03T06:50:30.014 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:30.016 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m0BIAKjCFk3cY6Eh4yGj1br4QBE= 2022-09-03T06:50:30.017 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m0BIAKjCFk3cY6Eh4yGj1br4QBE=', 'Content-Length': '0'} 2022-09-03T06:50:30.020 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b91bd6adc0e9e35c-006312f936-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.022 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:30.023 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:30.024 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:30.024 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:30.025 DEBUG:boto:Method: HEAD 2022-09-03T06:50:30.025 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:30.026 DEBUG:boto:Data: 2022-09-03T06:50:30.026 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.027 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.027 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.028 DEBUG:boto:Params: {} 2022-09-03T06:50:30.029 DEBUG:boto:Token: None 2022-09-03T06:50:30.029 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:30.030 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:452ornVx5SBk71flwLxmqAL1rD0= 2022-09-03T06:50:30.031 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:452ornVx5SBk71flwLxmqAL1rD0=', 'Content-Length': '0'} 2022-09-03T06:50:30.033 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fc037eea22f883ec-006312f936-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.033 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:30.034 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:30.035 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:30.035 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:30.036 DEBUG:boto:Method: HEAD 2022-09-03T06:50:30.036 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:30.037 DEBUG:boto:Data: 2022-09-03T06:50:30.037 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.038 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.039 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.039 DEBUG:boto:Params: {} 2022-09-03T06:50:30.040 DEBUG:boto:Token: None 2022-09-03T06:50:30.041 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:30.041 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:452ornVx5SBk71flwLxmqAL1rD0= 2022-09-03T06:50:30.042 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:452ornVx5SBk71flwLxmqAL1rD0=', 'Content-Length': '0'} 2022-09-03T06:50:30.044 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002f53eb451fa704c9-006312f936-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.045 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:50:30.045 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:30.046 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:30.047 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:30.047 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:30.048 DEBUG:boto:Method: GET 2022-09-03T06:50:30.048 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:30.049 DEBUG:boto:Data: 2022-09-03T06:50:30.049 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.050 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.051 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.051 DEBUG:boto:Params: {} 2022-09-03T06:50:30.052 DEBUG:boto:Token: None 2022-09-03T06:50:30.053 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:30.053 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m0BIAKjCFk3cY6Eh4yGj1br4QBE= 2022-09-03T06:50:30.054 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m0BIAKjCFk3cY6Eh4yGj1br4QBE=', 'Content-Length': '0'} 2022-09-03T06:50:30.056 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000097ef11a7c12679f3-006312f936-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.057 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:30.058 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:30.059 DEBUG:boto:path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:30.059 DEBUG:boto:auth_path=/dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:30.060 DEBUG:boto:Method: GET 2022-09-03T06:50:30.060 DEBUG:boto:Path: /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:30.061 DEBUG:boto:Data: 2022-09-03T06:50:30.062 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.062 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.063 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.063 DEBUG:boto:Params: {} 2022-09-03T06:50:30.064 DEBUG:boto:Token: None 2022-09-03T06:50:30.064 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj57?versionId=null 2022-09-03T06:50:30.065 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:m0BIAKjCFk3cY6Eh4yGj1br4QBE= 2022-09-03T06:50:30.066 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:m0BIAKjCFk3cY6Eh4yGj1br4QBE=', 'Content-Length': '0'} 2022-09-03T06:50:30.068 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000049fb4a1e79925acc-006312f936-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.069 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:30.070 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:30.070 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:30.071 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:30.072 DEBUG:boto:Method: GET 2022-09-03T06:50:30.072 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:50:30.073 DEBUG:boto:Data: 2022-09-03T06:50:30.073 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.074 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.074 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.075 DEBUG:boto:Params: {} 2022-09-03T06:50:30.076 DEBUG:boto:Token: None 2022-09-03T06:50:30.076 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/ 2022-09-03T06:50:30.077 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc= 2022-09-03T06:50:30.078 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc=', 'Content-Length': '0'} 2022-09-03T06:50:30.081 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009ef3ed61c4fe32ea-006312f936-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.082 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:30.083 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:30.084 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:30.084 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:30.085 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:30.085 DEBUG:boto:Method: GET 2022-09-03T06:50:30.086 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:50:30.086 DEBUG:boto:Data: 2022-09-03T06:50:30.087 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.087 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.088 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.089 DEBUG:boto:Params: {} 2022-09-03T06:50:30.089 DEBUG:boto:Token: None 2022-09-03T06:50:30.090 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/ 2022-09-03T06:50:30.091 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc= 2022-09-03T06:50:30.091 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc=', 'Content-Length': '0'} 2022-09-03T06:50:30.094 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004ce9f8c775f155a8-006312f936-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.095 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:30.096 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:30.096 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:30.097 DEBUG:boto:Method: HEAD 2022-09-03T06:50:30.098 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:50:30.098 DEBUG:boto:Data: 2022-09-03T06:50:30.099 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.099 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.100 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.100 DEBUG:boto:Params: {} 2022-09-03T06:50:30.101 DEBUG:boto:Token: None 2022-09-03T06:50:30.102 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj57 2022-09-03T06:50:30.102 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Dkt/rdsY8vbCwJzIUV8EAa4CTbg= 2022-09-03T06:50:30.103 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Dkt/rdsY8vbCwJzIUV8EAa4CTbg=', 'Content-Length': '0'} 2022-09-03T06:50:30.105 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000073865e68272840cd-006312f936-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.108 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:30.109 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:30.110 DEBUG:boto:Method: HEAD 2022-09-03T06:50:30.110 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:50:30.111 DEBUG:boto:Data: 2022-09-03T06:50:30.112 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.113 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.114 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.115 DEBUG:boto:Params: {} 2022-09-03T06:50:30.116 DEBUG:boto:Token: None 2022-09-03T06:50:30.117 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj57 2022-09-03T06:50:30.118 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Dkt/rdsY8vbCwJzIUV8EAa4CTbg= 2022-09-03T06:50:30.119 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Dkt/rdsY8vbCwJzIUV8EAa4CTbg=', 'Content-Length': '0'} 2022-09-03T06:50:30.122 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005854ce52ee197e31-006312f936-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.123 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj57 2022-09-03T06:50:30.123 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:30.124 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:30.125 DEBUG:boto:Method: GET 2022-09-03T06:50:30.126 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:50:30.127 DEBUG:boto:Data: 2022-09-03T06:50:30.128 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.128 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.129 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.130 DEBUG:boto:Params: {} 2022-09-03T06:50:30.132 DEBUG:boto:Token: None 2022-09-03T06:50:30.133 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj57 2022-09-03T06:50:30.134 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TLqzkRNq9rc+/gealt/fLOtwigs= 2022-09-03T06:50:30.135 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TLqzkRNq9rc+/gealt/fLOtwigs=', 'Content-Length': '0'} 2022-09-03T06:50:30.138 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fd4e2e7a3a65f4df-006312f936-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.139 DEBUG:boto:path=/dsgvcy-75/obj57 2022-09-03T06:50:30.140 DEBUG:boto:auth_path=/dsgvcy-75/obj57 2022-09-03T06:50:30.141 DEBUG:boto:Method: GET 2022-09-03T06:50:30.141 DEBUG:boto:Path: /dsgvcy-75/obj57 2022-09-03T06:50:30.142 DEBUG:boto:Data: 2022-09-03T06:50:30.143 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.143 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.144 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.145 DEBUG:boto:Params: {} 2022-09-03T06:50:30.145 DEBUG:boto:Token: None 2022-09-03T06:50:30.146 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj57 2022-09-03T06:50:30.147 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:TLqzkRNq9rc+/gealt/fLOtwigs= 2022-09-03T06:50:30.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:TLqzkRNq9rc+/gealt/fLOtwigs=', 'Content-Length': '0'} 2022-09-03T06:50:30.151 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002a5909c64dbdd3f7-006312f936-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.194 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:30.196 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:30.197 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:30.198 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:30.199 DEBUG:boto:Method: GET 2022-09-03T06:50:30.200 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:50:30.201 DEBUG:boto:Data: 2022-09-03T06:50:30.202 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.203 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.204 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.205 DEBUG:boto:Params: {} 2022-09-03T06:50:30.207 DEBUG:boto:Token: None 2022-09-03T06:50:30.208 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/ 2022-09-03T06:50:30.209 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc= 2022-09-03T06:50:30.210 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc=', 'Content-Length': '0'} 2022-09-03T06:50:30.214 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c9b176999d00343e-006312f936-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.216 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:30.217 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:30.217 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:30.218 DEBUG:boto:path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:30.219 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj57 2022-09-03T06:50:30.219 DEBUG:boto:Method: GET 2022-09-03T06:50:30.220 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj57 2022-09-03T06:50:30.221 DEBUG:boto:Data: 2022-09-03T06:50:30.221 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.222 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.223 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.224 DEBUG:boto:Params: {} 2022-09-03T06:50:30.224 DEBUG:boto:Token: None 2022-09-03T06:50:30.225 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/ 2022-09-03T06:50:30.226 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc= 2022-09-03T06:50:30.227 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc=', 'Content-Length': '0'} 2022-09-03T06:50:30.230 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000499fc7f08b812f22-006312f936-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.231 DEBUG:boto:b'dsgvcy-75obj571000falseobj572022-09-03T06:40:53.294Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:30.232 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:50:30.233 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:30.234 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:30.235 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.235 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.236 DEBUG:boto:Method: GET 2022-09-03T06:50:30.236 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.237 DEBUG:boto:Data: 2022-09-03T06:50:30.237 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.238 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.239 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.239 DEBUG:boto:Params: {} 2022-09-03T06:50:30.240 DEBUG:boto:Token: None 2022-09-03T06:50:30.240 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.241 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EWk3z73dy/cObCz6FiA0z0hGoq0= 2022-09-03T06:50:30.242 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EWk3z73dy/cObCz6FiA0z0hGoq0=', 'Content-Length': '0'} 2022-09-03T06:50:30.244 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007b2b83ab5ef698a8-006312f936-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.245 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:30.246 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:30.246 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.247 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.248 DEBUG:boto:Method: GET 2022-09-03T06:50:30.248 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.249 DEBUG:boto:Data: 2022-09-03T06:50:30.250 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.250 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.251 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.251 DEBUG:boto:Params: {} 2022-09-03T06:50:30.252 DEBUG:boto:Token: None 2022-09-03T06:50:30.253 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.254 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EWk3z73dy/cObCz6FiA0z0hGoq0= 2022-09-03T06:50:30.254 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EWk3z73dy/cObCz6FiA0z0hGoq0=', 'Content-Length': '0'} 2022-09-03T06:50:30.257 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001d47cf73b5eab337-006312f936-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.258 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:30.258 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:30.259 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.259 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.260 DEBUG:boto:Method: HEAD 2022-09-03T06:50:30.260 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.261 DEBUG:boto:Data: 2022-09-03T06:50:30.261 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.262 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.262 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.263 DEBUG:boto:Params: {} 2022-09-03T06:50:30.263 DEBUG:boto:Token: None 2022-09-03T06:50:30.264 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.264 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tx5tmB4vm601ShAr8p31iYj6PDc= 2022-09-03T06:50:30.265 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tx5tmB4vm601ShAr8p31iYj6PDc=', 'Content-Length': '0'} 2022-09-03T06:50:30.267 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e41d3d8ce6f66c3b-006312f936-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.268 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:30.269 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:30.269 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.270 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.270 DEBUG:boto:Method: HEAD 2022-09-03T06:50:30.271 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.271 DEBUG:boto:Data: 2022-09-03T06:50:30.272 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.273 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.273 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.274 DEBUG:boto:Params: {} 2022-09-03T06:50:30.274 DEBUG:boto:Token: None 2022-09-03T06:50:30.275 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.275 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:tx5tmB4vm601ShAr8p31iYj6PDc= 2022-09-03T06:50:30.276 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:tx5tmB4vm601ShAr8p31iYj6PDc=', 'Content-Length': '0'} 2022-09-03T06:50:30.278 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f0f005b1f84f84ba-006312f936-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.279 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:50:30.280 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:30.280 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:30.281 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.281 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.282 DEBUG:boto:Method: GET 2022-09-03T06:50:30.282 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.283 DEBUG:boto:Data: 2022-09-03T06:50:30.283 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.284 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.284 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.285 DEBUG:boto:Params: {} 2022-09-03T06:50:30.286 DEBUG:boto:Token: None 2022-09-03T06:50:30.286 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.287 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EWk3z73dy/cObCz6FiA0z0hGoq0= 2022-09-03T06:50:30.288 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EWk3z73dy/cObCz6FiA0z0hGoq0=', 'Content-Length': '0'} 2022-09-03T06:50:30.290 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ea727e205ba6d983-006312f936-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.291 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:30.292 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:30.292 DEBUG:boto:path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.293 DEBUG:boto:auth_path=/dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.294 DEBUG:boto:Method: GET 2022-09-03T06:50:30.294 DEBUG:boto:Path: /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.295 DEBUG:boto:Data: 2022-09-03T06:50:30.295 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.296 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.296 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.297 DEBUG:boto:Params: {} 2022-09-03T06:50:30.298 DEBUG:boto:Token: None 2022-09-03T06:50:30.299 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj58?versionId=null 2022-09-03T06:50:30.300 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EWk3z73dy/cObCz6FiA0z0hGoq0= 2022-09-03T06:50:30.300 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EWk3z73dy/cObCz6FiA0z0hGoq0=', 'Content-Length': '0'} 2022-09-03T06:50:30.302 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000019b9f94d9a555696-006312f936-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.346 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:30.347 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:30.348 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:30.348 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:30.349 DEBUG:boto:Method: GET 2022-09-03T06:50:30.350 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:50:30.350 DEBUG:boto:Data: 2022-09-03T06:50:30.351 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.351 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.352 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.353 DEBUG:boto:Params: {} 2022-09-03T06:50:30.354 DEBUG:boto:Token: None 2022-09-03T06:50:30.355 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/ 2022-09-03T06:50:30.356 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc= 2022-09-03T06:50:30.356 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc=', 'Content-Length': '0'} 2022-09-03T06:50:30.359 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f69da4c9f4e8cf7d-006312f936-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.360 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:30.361 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:30.362 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:30.363 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:30.363 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:30.364 DEBUG:boto:Method: GET 2022-09-03T06:50:30.365 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:50:30.366 DEBUG:boto:Data: 2022-09-03T06:50:30.366 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.367 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.368 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.368 DEBUG:boto:Params: {} 2022-09-03T06:50:30.369 DEBUG:boto:Token: None 2022-09-03T06:50:30.370 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/ 2022-09-03T06:50:30.371 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc= 2022-09-03T06:50:30.372 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc=', 'Content-Length': '0'} 2022-09-03T06:50:30.374 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000adae6687f2e3faa7-006312f936-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.375 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:30.377 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:30.377 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:30.378 DEBUG:boto:Method: HEAD 2022-09-03T06:50:30.379 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:50:30.379 DEBUG:boto:Data: 2022-09-03T06:50:30.380 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.380 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.381 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.381 DEBUG:boto:Params: {} 2022-09-03T06:50:30.382 DEBUG:boto:Token: None 2022-09-03T06:50:30.383 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj58 2022-09-03T06:50:30.384 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3Fw93RpXLgsckt50BeB44Y7zvgg= 2022-09-03T06:50:30.384 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3Fw93RpXLgsckt50BeB44Y7zvgg=', 'Content-Length': '0'} 2022-09-03T06:50:30.387 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009f95b5ecbd614cbe-006312f936-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.388 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:30.389 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:30.390 DEBUG:boto:Method: HEAD 2022-09-03T06:50:30.391 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:50:30.392 DEBUG:boto:Data: 2022-09-03T06:50:30.393 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.395 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.396 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.397 DEBUG:boto:Params: {} 2022-09-03T06:50:30.397 DEBUG:boto:Token: None 2022-09-03T06:50:30.399 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj58 2022-09-03T06:50:30.400 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3Fw93RpXLgsckt50BeB44Y7zvgg= 2022-09-03T06:50:30.401 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3Fw93RpXLgsckt50BeB44Y7zvgg=', 'Content-Length': '0'} 2022-09-03T06:50:30.404 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000df0907fc97b63b66-006312f936-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.405 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj58 2022-09-03T06:50:30.406 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:30.407 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:30.408 DEBUG:boto:Method: GET 2022-09-03T06:50:30.409 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:50:30.410 DEBUG:boto:Data: 2022-09-03T06:50:30.411 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.412 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.413 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.414 DEBUG:boto:Params: {} 2022-09-03T06:50:30.416 DEBUG:boto:Token: None 2022-09-03T06:50:30.417 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj58 2022-09-03T06:50:30.418 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nit5Y/XIdAinOyGF00tMpUD1jHw= 2022-09-03T06:50:30.419 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nit5Y/XIdAinOyGF00tMpUD1jHw=', 'Content-Length': '0'} 2022-09-03T06:50:30.423 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e068ebe48f0dd5df-006312f936-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.466 DEBUG:boto:path=/dsgvcy-75/obj58 2022-09-03T06:50:30.467 DEBUG:boto:auth_path=/dsgvcy-75/obj58 2022-09-03T06:50:30.467 DEBUG:boto:Method: GET 2022-09-03T06:50:30.469 DEBUG:boto:Path: /dsgvcy-75/obj58 2022-09-03T06:50:30.469 DEBUG:boto:Data: 2022-09-03T06:50:30.471 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.472 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.473 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.474 DEBUG:boto:Params: {} 2022-09-03T06:50:30.475 DEBUG:boto:Token: None 2022-09-03T06:50:30.475 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj58 2022-09-03T06:50:30.476 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nit5Y/XIdAinOyGF00tMpUD1jHw= 2022-09-03T06:50:30.477 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nit5Y/XIdAinOyGF00tMpUD1jHw=', 'Content-Length': '0'} 2022-09-03T06:50:30.479 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a05a0a74ed6354d7-006312f936-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.480 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:30.481 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:30.482 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:30.482 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:30.483 DEBUG:boto:Method: GET 2022-09-03T06:50:30.484 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:50:30.485 DEBUG:boto:Data: 2022-09-03T06:50:30.485 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.486 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.487 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.487 DEBUG:boto:Params: {} 2022-09-03T06:50:30.488 DEBUG:boto:Token: None 2022-09-03T06:50:30.489 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/ 2022-09-03T06:50:30.490 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc= 2022-09-03T06:50:30.490 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc=', 'Content-Length': '0'} 2022-09-03T06:50:30.493 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000000925f2fe10508274-006312f936-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.494 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:30.495 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:30.496 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:30.496 DEBUG:boto:path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:30.497 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj58 2022-09-03T06:50:30.497 DEBUG:boto:Method: GET 2022-09-03T06:50:30.498 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj58 2022-09-03T06:50:30.498 DEBUG:boto:Data: 2022-09-03T06:50:30.499 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.500 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.501 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.502 DEBUG:boto:Params: {} 2022-09-03T06:50:30.502 DEBUG:boto:Token: None 2022-09-03T06:50:30.503 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/ 2022-09-03T06:50:30.504 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc= 2022-09-03T06:50:30.505 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc=', 'Content-Length': '0'} 2022-09-03T06:50:30.507 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e3215c0739bfea22-006312f936-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.554 DEBUG:boto:b'dsgvcy-75obj581000falseobj582022-09-03T06:44:18.271Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:30.555 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2022-09-03T06:50:30.556 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:30.557 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:30.557 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.558 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.558 DEBUG:boto:Method: GET 2022-09-03T06:50:30.559 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.560 DEBUG:boto:Data: 2022-09-03T06:50:30.561 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.561 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.562 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.563 DEBUG:boto:Params: {} 2022-09-03T06:50:30.563 DEBUG:boto:Token: None 2022-09-03T06:50:30.564 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.565 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oWJ9FOzAOIOWPKaHcB4sGqn0Tf0= 2022-09-03T06:50:30.566 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oWJ9FOzAOIOWPKaHcB4sGqn0Tf0=', 'Content-Length': '0'} 2022-09-03T06:50:30.568 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e0af4c27a6a7f285-006312f936-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.569 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:30.570 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:30.570 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.571 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.571 DEBUG:boto:Method: GET 2022-09-03T06:50:30.572 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.572 DEBUG:boto:Data: 2022-09-03T06:50:30.573 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.574 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.574 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.575 DEBUG:boto:Params: {} 2022-09-03T06:50:30.575 DEBUG:boto:Token: None 2022-09-03T06:50:30.576 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.577 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oWJ9FOzAOIOWPKaHcB4sGqn0Tf0= 2022-09-03T06:50:30.577 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oWJ9FOzAOIOWPKaHcB4sGqn0Tf0=', 'Content-Length': '0'} 2022-09-03T06:50:30.580 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004757d17d0f96d4e8-006312f936-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.581 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:30.581 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:30.582 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.582 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.583 DEBUG:boto:Method: HEAD 2022-09-03T06:50:30.583 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.584 DEBUG:boto:Data: 2022-09-03T06:50:30.584 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.585 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.586 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.586 DEBUG:boto:Params: {} 2022-09-03T06:50:30.587 DEBUG:boto:Token: None 2022-09-03T06:50:30.587 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.588 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ikEojNrGDvJk9NG4pHhE/cITagY= 2022-09-03T06:50:30.589 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ikEojNrGDvJk9NG4pHhE/cITagY=', 'Content-Length': '0'} 2022-09-03T06:50:30.591 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009122aa44b128a5b6-006312f936-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.592 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:30.593 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:30.594 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.595 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.595 DEBUG:boto:Method: HEAD 2022-09-03T06:50:30.596 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.597 DEBUG:boto:Data: 2022-09-03T06:50:30.598 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.598 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.599 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.600 DEBUG:boto:Params: {} 2022-09-03T06:50:30.601 DEBUG:boto:Token: None 2022-09-03T06:50:30.601 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.602 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ikEojNrGDvJk9NG4pHhE/cITagY= 2022-09-03T06:50:30.602 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ikEojNrGDvJk9NG4pHhE/cITagY=', 'Content-Length': '0'} 2022-09-03T06:50:30.606 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a29c61ec875307d9-006312f936-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.606 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2022-09-03T06:50:30.607 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:30.608 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:30.608 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.609 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.609 DEBUG:boto:Method: GET 2022-09-03T06:50:30.610 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.610 DEBUG:boto:Data: 2022-09-03T06:50:30.611 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.611 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.612 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.612 DEBUG:boto:Params: {} 2022-09-03T06:50:30.613 DEBUG:boto:Token: None 2022-09-03T06:50:30.614 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.614 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oWJ9FOzAOIOWPKaHcB4sGqn0Tf0= 2022-09-03T06:50:30.615 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oWJ9FOzAOIOWPKaHcB4sGqn0Tf0=', 'Content-Length': '0'} 2022-09-03T06:50:30.617 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ba735402e9569e4e-006312f936-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.618 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:30.619 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:30.619 DEBUG:boto:path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.620 DEBUG:boto:auth_path=/dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.620 DEBUG:boto:Method: GET 2022-09-03T06:50:30.621 DEBUG:boto:Path: /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.621 DEBUG:boto:Data: 2022-09-03T06:50:30.622 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.623 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.623 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.624 DEBUG:boto:Params: {} 2022-09-03T06:50:30.624 DEBUG:boto:Token: None 2022-09-03T06:50:30.625 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj62?versionId=null 2022-09-03T06:50:30.626 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:oWJ9FOzAOIOWPKaHcB4sGqn0Tf0= 2022-09-03T06:50:30.626 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:oWJ9FOzAOIOWPKaHcB4sGqn0Tf0=', 'Content-Length': '0'} 2022-09-03T06:50:30.628 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000551b63589dc25ef9-006312f936-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.629 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:30.630 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:30.630 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:30.631 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:30.631 DEBUG:boto:Method: GET 2022-09-03T06:50:30.632 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:50:30.632 DEBUG:boto:Data: 2022-09-03T06:50:30.633 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.633 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.634 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.635 DEBUG:boto:Params: {} 2022-09-03T06:50:30.635 DEBUG:boto:Token: None 2022-09-03T06:50:30.636 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/ 2022-09-03T06:50:30.636 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc= 2022-09-03T06:50:30.637 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc=', 'Content-Length': '0'} 2022-09-03T06:50:30.640 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f4e2f262eba686f1-006312f936-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.640 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:30.641 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:30.642 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:30.643 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:30.644 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:30.644 DEBUG:boto:Method: GET 2022-09-03T06:50:30.645 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:50:30.645 DEBUG:boto:Data: 2022-09-03T06:50:30.646 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.647 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.648 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.648 DEBUG:boto:Params: {} 2022-09-03T06:50:30.649 DEBUG:boto:Token: None 2022-09-03T06:50:30.650 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/ 2022-09-03T06:50:30.651 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc= 2022-09-03T06:50:30.651 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc=', 'Content-Length': '0'} 2022-09-03T06:50:30.654 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e20d1ea80e7e8e91-006312f936-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.655 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:30.656 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:30.657 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:30.658 DEBUG:boto:Method: HEAD 2022-09-03T06:50:30.658 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:50:30.659 DEBUG:boto:Data: 2022-09-03T06:50:30.659 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.660 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.660 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.661 DEBUG:boto:Params: {} 2022-09-03T06:50:30.662 DEBUG:boto:Token: None 2022-09-03T06:50:30.663 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj62 2022-09-03T06:50:30.663 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4gAf7qTXQqoyGd9b4F115cCULY4= 2022-09-03T06:50:30.664 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4gAf7qTXQqoyGd9b4F115cCULY4=', 'Content-Length': '0'} 2022-09-03T06:50:30.666 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000003d7b31544fdb4249-006312f936-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.667 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:30.667 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:30.668 DEBUG:boto:Method: HEAD 2022-09-03T06:50:30.668 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:50:30.669 DEBUG:boto:Data: 2022-09-03T06:50:30.669 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.670 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.670 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.671 DEBUG:boto:Params: {} 2022-09-03T06:50:30.671 DEBUG:boto:Token: None 2022-09-03T06:50:30.672 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj62 2022-09-03T06:50:30.673 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4gAf7qTXQqoyGd9b4F115cCULY4= 2022-09-03T06:50:30.673 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4gAf7qTXQqoyGd9b4F115cCULY4=', 'Content-Length': '0'} 2022-09-03T06:50:30.676 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000856682be848b9af1-006312f936-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.676 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj62 2022-09-03T06:50:30.677 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:30.678 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:30.678 DEBUG:boto:Method: GET 2022-09-03T06:50:30.679 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:50:30.679 DEBUG:boto:Data: 2022-09-03T06:50:30.680 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.680 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.681 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.681 DEBUG:boto:Params: {} 2022-09-03T06:50:30.682 DEBUG:boto:Token: None 2022-09-03T06:50:30.683 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj62 2022-09-03T06:50:30.683 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fMAV5u3ea4r4N5aUf+YzSeLtigc= 2022-09-03T06:50:30.684 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fMAV5u3ea4r4N5aUf+YzSeLtigc=', 'Content-Length': '0'} 2022-09-03T06:50:30.686 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a8b40033e0256544-006312f936-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.734 DEBUG:boto:path=/dsgvcy-75/obj62 2022-09-03T06:50:30.735 DEBUG:boto:auth_path=/dsgvcy-75/obj62 2022-09-03T06:50:30.736 DEBUG:boto:Method: GET 2022-09-03T06:50:30.736 DEBUG:boto:Path: /dsgvcy-75/obj62 2022-09-03T06:50:30.737 DEBUG:boto:Data: 2022-09-03T06:50:30.737 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.738 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.738 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.739 DEBUG:boto:Params: {} 2022-09-03T06:50:30.740 DEBUG:boto:Token: None 2022-09-03T06:50:30.740 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj62 2022-09-03T06:50:30.741 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fMAV5u3ea4r4N5aUf+YzSeLtigc= 2022-09-03T06:50:30.742 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fMAV5u3ea4r4N5aUf+YzSeLtigc=', 'Content-Length': '0'} 2022-09-03T06:50:30.744 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000390ff00aa671c202-006312f936-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.745 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:30.746 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:30.746 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:30.747 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:30.748 DEBUG:boto:Method: GET 2022-09-03T06:50:30.748 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:50:30.749 DEBUG:boto:Data: 2022-09-03T06:50:30.750 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.750 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.751 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.752 DEBUG:boto:Params: {} 2022-09-03T06:50:30.753 DEBUG:boto:Token: None 2022-09-03T06:50:30.755 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/ 2022-09-03T06:50:30.756 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc= 2022-09-03T06:50:30.759 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc=', 'Content-Length': '0'} 2022-09-03T06:50:30.762 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000f507f8f0e8b51623-006312f936-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.762 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:30.763 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:30.764 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:30.765 DEBUG:boto:path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:30.765 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj62 2022-09-03T06:50:30.766 DEBUG:boto:Method: GET 2022-09-03T06:50:30.766 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj62 2022-09-03T06:50:30.767 DEBUG:boto:Data: 2022-09-03T06:50:30.768 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.768 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.769 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.769 DEBUG:boto:Params: {} 2022-09-03T06:50:30.770 DEBUG:boto:Token: None 2022-09-03T06:50:30.771 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/ 2022-09-03T06:50:30.772 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc= 2022-09-03T06:50:30.773 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc=', 'Content-Length': '0'} 2022-09-03T06:50:30.778 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006844ff1df27fcbde-006312f936-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.781 DEBUG:boto:b'dsgvcy-75obj621000falseobj622022-09-03T06:45:46.850Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:30.782 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:50:30.783 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:30.784 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:30.785 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.785 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.786 DEBUG:boto:Method: GET 2022-09-03T06:50:30.787 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.787 DEBUG:boto:Data: 2022-09-03T06:50:30.788 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.789 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.789 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.790 DEBUG:boto:Params: {} 2022-09-03T06:50:30.791 DEBUG:boto:Token: None 2022-09-03T06:50:30.791 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.792 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SJzXtHk4XnS2L9DFkeHJ39gNdHM= 2022-09-03T06:50:30.793 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SJzXtHk4XnS2L9DFkeHJ39gNdHM=', 'Content-Length': '0'} 2022-09-03T06:50:30.795 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e7c969aa914e8aac-006312f936-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.797 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:30.798 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:30.799 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.800 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.801 DEBUG:boto:Method: GET 2022-09-03T06:50:30.802 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.803 DEBUG:boto:Data: 2022-09-03T06:50:30.803 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.804 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.805 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.805 DEBUG:boto:Params: {} 2022-09-03T06:50:30.806 DEBUG:boto:Token: None 2022-09-03T06:50:30.806 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.807 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SJzXtHk4XnS2L9DFkeHJ39gNdHM= 2022-09-03T06:50:30.808 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SJzXtHk4XnS2L9DFkeHJ39gNdHM=', 'Content-Length': '0'} 2022-09-03T06:50:30.810 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fcc901316dba823c-006312f936-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.854 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:30.855 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:30.856 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.856 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.857 DEBUG:boto:Method: HEAD 2022-09-03T06:50:30.857 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.858 DEBUG:boto:Data: 2022-09-03T06:50:30.858 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.859 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.859 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.860 DEBUG:boto:Params: {} 2022-09-03T06:50:30.861 DEBUG:boto:Token: None 2022-09-03T06:50:30.861 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.862 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9LvA2TjgdiTznMMQFCBbBvIJmw0= 2022-09-03T06:50:30.862 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9LvA2TjgdiTznMMQFCBbBvIJmw0=', 'Content-Length': '0'} 2022-09-03T06:50:30.864 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b96795af9990340-006312f936-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.865 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:30.866 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:30.867 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.867 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.868 DEBUG:boto:Method: HEAD 2022-09-03T06:50:30.869 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.869 DEBUG:boto:Data: 2022-09-03T06:50:30.870 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.870 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.871 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.871 DEBUG:boto:Params: {} 2022-09-03T06:50:30.872 DEBUG:boto:Token: None 2022-09-03T06:50:30.872 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.873 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:9LvA2TjgdiTznMMQFCBbBvIJmw0= 2022-09-03T06:50:30.873 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:9LvA2TjgdiTznMMQFCBbBvIJmw0=', 'Content-Length': '0'} 2022-09-03T06:50:30.875 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000049cc41365a6b0f8c-006312f936-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.876 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:50:30.877 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:30.877 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:30.878 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.878 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.879 DEBUG:boto:Method: GET 2022-09-03T06:50:30.879 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.880 DEBUG:boto:Data: 2022-09-03T06:50:30.881 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.881 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.882 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.882 DEBUG:boto:Params: {} 2022-09-03T06:50:30.883 DEBUG:boto:Token: None 2022-09-03T06:50:30.883 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.884 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SJzXtHk4XnS2L9DFkeHJ39gNdHM= 2022-09-03T06:50:30.885 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SJzXtHk4XnS2L9DFkeHJ39gNdHM=', 'Content-Length': '0'} 2022-09-03T06:50:30.889 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000080ab8d89fb0695c4-006312f936-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.890 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:30.891 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:30.892 DEBUG:boto:path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.893 DEBUG:boto:auth_path=/dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.894 DEBUG:boto:Method: GET 2022-09-03T06:50:30.895 DEBUG:boto:Path: /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.896 DEBUG:boto:Data: 2022-09-03T06:50:30.897 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.898 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.899 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.900 DEBUG:boto:Params: {} 2022-09-03T06:50:30.901 DEBUG:boto:Token: None 2022-09-03T06:50:30.902 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj68?versionId=null 2022-09-03T06:50:30.903 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SJzXtHk4XnS2L9DFkeHJ39gNdHM= 2022-09-03T06:50:30.904 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SJzXtHk4XnS2L9DFkeHJ39gNdHM=', 'Content-Length': '0'} 2022-09-03T06:50:30.907 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b6ed08c22c08be7-006312f936-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.909 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:30.910 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:30.911 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:30.911 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:30.912 DEBUG:boto:Method: GET 2022-09-03T06:50:30.913 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:50:30.914 DEBUG:boto:Data: 2022-09-03T06:50:30.914 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.915 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.916 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.916 DEBUG:boto:Params: {} 2022-09-03T06:50:30.917 DEBUG:boto:Token: None 2022-09-03T06:50:30.918 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/ 2022-09-03T06:50:30.918 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc= 2022-09-03T06:50:30.919 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc=', 'Content-Length': '0'} 2022-09-03T06:50:30.922 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000015dc04f4e82325e4-006312f936-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.923 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:30.925 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:30.926 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:30.927 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:30.928 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:30.929 DEBUG:boto:Method: GET 2022-09-03T06:50:30.929 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:50:30.930 DEBUG:boto:Data: 2022-09-03T06:50:30.931 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.932 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.933 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.933 DEBUG:boto:Params: {} 2022-09-03T06:50:30.934 DEBUG:boto:Token: None 2022-09-03T06:50:30.934 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/ 2022-09-03T06:50:30.935 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc= 2022-09-03T06:50:30.936 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:EbDYgRNfl1S4+mehiiSJfVw5kcc=', 'Content-Length': '0'} 2022-09-03T06:50:30.939 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000d16ca0da9fb9b5f6-006312f936-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.940 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:30.941 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:30.942 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:30.943 DEBUG:boto:Method: HEAD 2022-09-03T06:50:30.944 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:50:30.945 DEBUG:boto:Data: 2022-09-03T06:50:30.946 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.947 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.947 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.948 DEBUG:boto:Params: {} 2022-09-03T06:50:30.949 DEBUG:boto:Token: None 2022-09-03T06:50:30.949 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj68 2022-09-03T06:50:30.950 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nO2cycm+lGFSijOsxjiVzW6RgBg= 2022-09-03T06:50:30.950 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nO2cycm+lGFSijOsxjiVzW6RgBg=', 'Content-Length': '0'} 2022-09-03T06:50:30.953 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fe92690a821693cb-006312f936-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.955 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:30.956 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:30.957 DEBUG:boto:Method: HEAD 2022-09-03T06:50:30.957 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:50:30.958 DEBUG:boto:Data: 2022-09-03T06:50:30.959 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.961 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:30.962 DEBUG:boto:Port: 8000 2022-09-03T06:50:30.964 DEBUG:boto:Params: {} 2022-09-03T06:50:30.965 DEBUG:boto:Token: None 2022-09-03T06:50:30.967 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj68 2022-09-03T06:50:30.968 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:nO2cycm+lGFSijOsxjiVzW6RgBg= 2022-09-03T06:50:30.969 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:nO2cycm+lGFSijOsxjiVzW6RgBg=', 'Content-Length': '0'} 2022-09-03T06:50:30.973 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006e8d3608b779b198-006312f936-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:30.975 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj68 2022-09-03T06:50:30.976 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:30.977 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:30.978 DEBUG:boto:Method: GET 2022-09-03T06:50:30.978 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:50:30.979 DEBUG:boto:Data: 2022-09-03T06:50:30.980 DEBUG:boto:Headers: {} 2022-09-03T06:50:30.981 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:30.982 DEBUG:boto:Port: 8001 2022-09-03T06:50:30.984 DEBUG:boto:Params: {} 2022-09-03T06:50:30.985 DEBUG:boto:Token: None 2022-09-03T06:50:30.986 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:30 GMT /dsgvcy-75/obj68 2022-09-03T06:50:30.988 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:niLlo+B83BhInvvNsRPiOmjUDkU= 2022-09-03T06:50:30.990 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:30 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:niLlo+B83BhInvvNsRPiOmjUDkU=', 'Content-Length': '0'} 2022-09-03T06:50:30.999 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ccc06fb6a8ce10c3-006312f936-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:30 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.043 DEBUG:boto:path=/dsgvcy-75/obj68 2022-09-03T06:50:31.044 DEBUG:boto:auth_path=/dsgvcy-75/obj68 2022-09-03T06:50:31.045 DEBUG:boto:Method: GET 2022-09-03T06:50:31.045 DEBUG:boto:Path: /dsgvcy-75/obj68 2022-09-03T06:50:31.047 DEBUG:boto:Data: 2022-09-03T06:50:31.047 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.048 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:31.049 DEBUG:boto:Port: 8000 2022-09-03T06:50:31.050 DEBUG:boto:Params: {} 2022-09-03T06:50:31.051 DEBUG:boto:Token: None 2022-09-03T06:50:31.052 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj68 2022-09-03T06:50:31.053 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:2onBp8AkmNLdOOr/dZPCaMc2AyQ= 2022-09-03T06:50:31.054 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:2onBp8AkmNLdOOr/dZPCaMc2AyQ=', 'Content-Length': '0'} 2022-09-03T06:50:31.057 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000066b469c40248553-006312f937-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.058 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:31.059 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:31.060 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:31.060 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:31.061 DEBUG:boto:Method: GET 2022-09-03T06:50:31.062 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:50:31.063 DEBUG:boto:Data: 2022-09-03T06:50:31.064 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.065 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:31.065 DEBUG:boto:Port: 8001 2022-09-03T06:50:31.066 DEBUG:boto:Params: {} 2022-09-03T06:50:31.068 DEBUG:boto:Token: None 2022-09-03T06:50:31.069 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/ 2022-09-03T06:50:31.070 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI= 2022-09-03T06:50:31.071 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI=', 'Content-Length': '0'} 2022-09-03T06:50:31.075 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000006f281a45270aa134-006312f937-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.077 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:31.078 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:31.079 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:31.079 DEBUG:boto:path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:31.080 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj68 2022-09-03T06:50:31.080 DEBUG:boto:Method: GET 2022-09-03T06:50:31.081 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj68 2022-09-03T06:50:31.082 DEBUG:boto:Data: 2022-09-03T06:50:31.082 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.091 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:31.091 DEBUG:boto:Port: 8000 2022-09-03T06:50:31.092 DEBUG:boto:Params: {} 2022-09-03T06:50:31.093 DEBUG:boto:Token: None 2022-09-03T06:50:31.094 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/ 2022-09-03T06:50:31.094 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI= 2022-09-03T06:50:31.095 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI=', 'Content-Length': '0'} 2022-09-03T06:50:31.098 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000350dac49551e2376-006312f937-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.099 DEBUG:boto:b'dsgvcy-75obj681000falseobj682022-09-03T06:37:27.190Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:31.100 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:50:31.101 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:31.102 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:31.103 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.103 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.104 DEBUG:boto:Method: GET 2022-09-03T06:50:31.105 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.106 DEBUG:boto:Data: 2022-09-03T06:50:31.107 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.107 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:31.108 DEBUG:boto:Port: 8001 2022-09-03T06:50:31.108 DEBUG:boto:Params: {} 2022-09-03T06:50:31.110 DEBUG:boto:Token: None 2022-09-03T06:50:31.110 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.111 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G1TQmJUrGeDA95paV28UQ9fScqY= 2022-09-03T06:50:31.112 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G1TQmJUrGeDA95paV28UQ9fScqY=', 'Content-Length': '0'} 2022-09-03T06:50:31.114 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000016a0007fea81fddb-006312f937-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.115 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:31.116 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:31.116 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.117 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.117 DEBUG:boto:Method: GET 2022-09-03T06:50:31.118 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.119 DEBUG:boto:Data: 2022-09-03T06:50:31.119 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.120 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:31.120 DEBUG:boto:Port: 8000 2022-09-03T06:50:31.121 DEBUG:boto:Params: {} 2022-09-03T06:50:31.121 DEBUG:boto:Token: None 2022-09-03T06:50:31.122 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.123 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G1TQmJUrGeDA95paV28UQ9fScqY= 2022-09-03T06:50:31.123 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G1TQmJUrGeDA95paV28UQ9fScqY=', 'Content-Length': '0'} 2022-09-03T06:50:31.127 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bc21f886c401ba30-006312f937-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.170 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:31.171 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:31.171 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.172 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.173 DEBUG:boto:Method: HEAD 2022-09-03T06:50:31.173 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.174 DEBUG:boto:Data: 2022-09-03T06:50:31.174 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.175 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:31.176 DEBUG:boto:Port: 8001 2022-09-03T06:50:31.176 DEBUG:boto:Params: {} 2022-09-03T06:50:31.177 DEBUG:boto:Token: None 2022-09-03T06:50:31.177 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.178 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YrLzpcOlmHur4n5wFzT8yywzNY0= 2022-09-03T06:50:31.178 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YrLzpcOlmHur4n5wFzT8yywzNY0=', 'Content-Length': '0'} 2022-09-03T06:50:31.181 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000768b0376e6f6a73c-006312f937-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.182 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:31.183 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:31.183 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.184 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.185 DEBUG:boto:Method: HEAD 2022-09-03T06:50:31.185 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.186 DEBUG:boto:Data: 2022-09-03T06:50:31.186 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.187 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:31.188 DEBUG:boto:Port: 8000 2022-09-03T06:50:31.188 DEBUG:boto:Params: {} 2022-09-03T06:50:31.189 DEBUG:boto:Token: None 2022-09-03T06:50:31.190 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.191 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:YrLzpcOlmHur4n5wFzT8yywzNY0= 2022-09-03T06:50:31.192 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:YrLzpcOlmHur4n5wFzT8yywzNY0=', 'Content-Length': '0'} 2022-09-03T06:50:31.195 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000007c0a963145ec8649-006312f937-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.196 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:50:31.197 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:31.198 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:31.198 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.199 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.200 DEBUG:boto:Method: GET 2022-09-03T06:50:31.201 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.202 DEBUG:boto:Data: 2022-09-03T06:50:31.202 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.203 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:31.204 DEBUG:boto:Port: 8001 2022-09-03T06:50:31.204 DEBUG:boto:Params: {} 2022-09-03T06:50:31.205 DEBUG:boto:Token: None 2022-09-03T06:50:31.205 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.206 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G1TQmJUrGeDA95paV28UQ9fScqY= 2022-09-03T06:50:31.207 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G1TQmJUrGeDA95paV28UQ9fScqY=', 'Content-Length': '0'} 2022-09-03T06:50:31.209 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004e875400f31c226f-006312f937-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.210 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:31.210 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:31.211 DEBUG:boto:path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.211 DEBUG:boto:auth_path=/dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.212 DEBUG:boto:Method: GET 2022-09-03T06:50:31.212 DEBUG:boto:Path: /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.213 DEBUG:boto:Data: 2022-09-03T06:50:31.213 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.214 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:31.214 DEBUG:boto:Port: 8000 2022-09-03T06:50:31.215 DEBUG:boto:Params: {} 2022-09-03T06:50:31.216 DEBUG:boto:Token: None 2022-09-03T06:50:31.216 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj69?versionId=null 2022-09-03T06:50:31.217 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:G1TQmJUrGeDA95paV28UQ9fScqY= 2022-09-03T06:50:31.218 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:G1TQmJUrGeDA95paV28UQ9fScqY=', 'Content-Length': '0'} 2022-09-03T06:50:31.220 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d98e6b0f43f750a0-006312f937-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.221 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:31.221 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:31.222 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:31.223 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:31.223 DEBUG:boto:Method: GET 2022-09-03T06:50:31.224 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:50:31.225 DEBUG:boto:Data: 2022-09-03T06:50:31.225 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.226 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:31.226 DEBUG:boto:Port: 8001 2022-09-03T06:50:31.227 DEBUG:boto:Params: {} 2022-09-03T06:50:31.228 DEBUG:boto:Token: None 2022-09-03T06:50:31.228 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/ 2022-09-03T06:50:31.229 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI= 2022-09-03T06:50:31.230 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI=', 'Content-Length': '0'} 2022-09-03T06:50:31.233 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000163c792d5f7da81c-006312f937-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.234 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:31.235 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:31.236 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:31.236 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:31.237 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:31.238 DEBUG:boto:Method: GET 2022-09-03T06:50:31.238 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:50:31.239 DEBUG:boto:Data: 2022-09-03T06:50:31.239 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.240 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:31.240 DEBUG:boto:Port: 8000 2022-09-03T06:50:31.241 DEBUG:boto:Params: {} 2022-09-03T06:50:31.242 DEBUG:boto:Token: None 2022-09-03T06:50:31.242 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/ 2022-09-03T06:50:31.243 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI= 2022-09-03T06:50:31.243 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI=', 'Content-Length': '0'} 2022-09-03T06:50:31.246 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000088bd48fb63122300-006312f937-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.247 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:31.248 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:31.249 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:31.250 DEBUG:boto:Method: HEAD 2022-09-03T06:50:31.250 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:50:31.251 DEBUG:boto:Data: 2022-09-03T06:50:31.252 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.253 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:31.253 DEBUG:boto:Port: 8001 2022-09-03T06:50:31.254 DEBUG:boto:Params: {} 2022-09-03T06:50:31.254 DEBUG:boto:Token: None 2022-09-03T06:50:31.255 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj69 2022-09-03T06:50:31.256 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V9GdSlrsII2CNa7Pf4iq2C6dO+Y= 2022-09-03T06:50:31.256 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V9GdSlrsII2CNa7Pf4iq2C6dO+Y=', 'Content-Length': '0'} 2022-09-03T06:50:31.258 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000036f37f697c968ca8-006312f937-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.259 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:31.260 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:31.260 DEBUG:boto:Method: HEAD 2022-09-03T06:50:31.261 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:50:31.262 DEBUG:boto:Data: 2022-09-03T06:50:31.263 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.263 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:31.264 DEBUG:boto:Port: 8000 2022-09-03T06:50:31.264 DEBUG:boto:Params: {} 2022-09-03T06:50:31.265 DEBUG:boto:Token: None 2022-09-03T06:50:31.265 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj69 2022-09-03T06:50:31.266 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:V9GdSlrsII2CNa7Pf4iq2C6dO+Y= 2022-09-03T06:50:31.267 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:V9GdSlrsII2CNa7Pf4iq2C6dO+Y=', 'Content-Length': '0'} 2022-09-03T06:50:31.269 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004f3cfee21e10ab33-006312f937-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.270 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj69 2022-09-03T06:50:31.270 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:31.271 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:31.272 DEBUG:boto:Method: GET 2022-09-03T06:50:31.273 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:50:31.273 DEBUG:boto:Data: 2022-09-03T06:50:31.274 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.274 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:31.275 DEBUG:boto:Port: 8001 2022-09-03T06:50:31.275 DEBUG:boto:Params: {} 2022-09-03T06:50:31.276 DEBUG:boto:Token: None 2022-09-03T06:50:31.277 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj69 2022-09-03T06:50:31.278 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hFKJo0hw0Ex3vBbc1YjfA7lGIZE= 2022-09-03T06:50:31.278 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hFKJo0hw0Ex3vBbc1YjfA7lGIZE=', 'Content-Length': '0'} 2022-09-03T06:50:31.281 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000166f8dd991d8b9d-006312f937-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.323 DEBUG:boto:path=/dsgvcy-75/obj69 2022-09-03T06:50:31.324 DEBUG:boto:auth_path=/dsgvcy-75/obj69 2022-09-03T06:50:31.326 DEBUG:boto:Method: GET 2022-09-03T06:50:31.327 DEBUG:boto:Path: /dsgvcy-75/obj69 2022-09-03T06:50:31.328 DEBUG:boto:Data: 2022-09-03T06:50:31.329 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.330 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:31.331 DEBUG:boto:Port: 8000 2022-09-03T06:50:31.331 DEBUG:boto:Params: {} 2022-09-03T06:50:31.332 DEBUG:boto:Token: None 2022-09-03T06:50:31.332 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj69 2022-09-03T06:50:31.333 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:hFKJo0hw0Ex3vBbc1YjfA7lGIZE= 2022-09-03T06:50:31.334 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:hFKJo0hw0Ex3vBbc1YjfA7lGIZE=', 'Content-Length': '0'} 2022-09-03T06:50:31.342 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e7818b8cb5defbb6-006312f937-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.343 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:31.344 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:31.344 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:31.345 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:31.346 DEBUG:boto:Method: GET 2022-09-03T06:50:31.347 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:50:31.347 DEBUG:boto:Data: 2022-09-03T06:50:31.348 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.348 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:31.349 DEBUG:boto:Port: 8001 2022-09-03T06:50:31.350 DEBUG:boto:Params: {} 2022-09-03T06:50:31.350 DEBUG:boto:Token: None 2022-09-03T06:50:31.351 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/ 2022-09-03T06:50:31.352 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI= 2022-09-03T06:50:31.352 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI=', 'Content-Length': '0'} 2022-09-03T06:50:31.355 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004ffc4f143b67706b-006312f937-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.356 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:31.358 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:31.359 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:31.359 DEBUG:boto:path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:31.360 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj69 2022-09-03T06:50:31.361 DEBUG:boto:Method: GET 2022-09-03T06:50:31.361 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj69 2022-09-03T06:50:31.362 DEBUG:boto:Data: 2022-09-03T06:50:31.363 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.363 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:31.364 DEBUG:boto:Port: 8000 2022-09-03T06:50:31.364 DEBUG:boto:Params: {} 2022-09-03T06:50:31.365 DEBUG:boto:Token: None 2022-09-03T06:50:31.366 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/ 2022-09-03T06:50:31.367 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI= 2022-09-03T06:50:31.368 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI=', 'Content-Length': '0'} 2022-09-03T06:50:31.371 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a3a2bb2c12d453c1-006312f937-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.415 DEBUG:boto:b'dsgvcy-75obj691000falseobj692022-09-03T06:41:59.821Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:31.420 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj74 2022-09-03T06:50:31.421 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:31.422 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:31.423 DEBUG:boto:path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.424 DEBUG:boto:auth_path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.426 DEBUG:boto:Method: GET 2022-09-03T06:50:31.427 DEBUG:boto:Path: /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.428 DEBUG:boto:Data: 2022-09-03T06:50:31.429 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.430 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:31.431 DEBUG:boto:Port: 8001 2022-09-03T06:50:31.432 DEBUG:boto:Params: {} 2022-09-03T06:50:31.433 DEBUG:boto:Token: None 2022-09-03T06:50:31.434 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.435 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4w7eo9Ts3U4UTRy8/d088SMCGqE= 2022-09-03T06:50:31.435 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4w7eo9Ts3U4UTRy8/d088SMCGqE=', 'Content-Length': '0'} 2022-09-03T06:50:31.438 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000004898c6a329d1af1-006312f937-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.439 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:31.439 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:31.440 DEBUG:boto:path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.440 DEBUG:boto:auth_path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.441 DEBUG:boto:Method: GET 2022-09-03T06:50:31.441 DEBUG:boto:Path: /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.442 DEBUG:boto:Data: 2022-09-03T06:50:31.442 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.443 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:31.443 DEBUG:boto:Port: 8000 2022-09-03T06:50:31.444 DEBUG:boto:Params: {} 2022-09-03T06:50:31.445 DEBUG:boto:Token: None 2022-09-03T06:50:31.445 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.446 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4w7eo9Ts3U4UTRy8/d088SMCGqE= 2022-09-03T06:50:31.446 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4w7eo9Ts3U4UTRy8/d088SMCGqE=', 'Content-Length': '0'} 2022-09-03T06:50:31.449 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004728926654a5d311-006312f937-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.450 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:31.450 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:31.451 DEBUG:boto:path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.451 DEBUG:boto:auth_path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.452 DEBUG:boto:Method: HEAD 2022-09-03T06:50:31.452 DEBUG:boto:Path: /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.453 DEBUG:boto:Data: 2022-09-03T06:50:31.453 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.454 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:31.454 DEBUG:boto:Port: 8001 2022-09-03T06:50:31.455 DEBUG:boto:Params: {} 2022-09-03T06:50:31.455 DEBUG:boto:Token: None 2022-09-03T06:50:31.456 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.457 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ksLbHX3vsGtprjNtgkDFI77Elcs= 2022-09-03T06:50:31.457 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ksLbHX3vsGtprjNtgkDFI77Elcs=', 'Content-Length': '0'} 2022-09-03T06:50:31.459 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c69cbbca9d6a443f-006312f937-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.461 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:31.461 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:31.462 DEBUG:boto:path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.462 DEBUG:boto:auth_path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.463 DEBUG:boto:Method: HEAD 2022-09-03T06:50:31.463 DEBUG:boto:Path: /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.464 DEBUG:boto:Data: 2022-09-03T06:50:31.465 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.465 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:31.466 DEBUG:boto:Port: 8000 2022-09-03T06:50:31.466 DEBUG:boto:Params: {} 2022-09-03T06:50:31.467 DEBUG:boto:Token: None 2022-09-03T06:50:31.468 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.468 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ksLbHX3vsGtprjNtgkDFI77Elcs= 2022-09-03T06:50:31.469 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ksLbHX3vsGtprjNtgkDFI77Elcs=', 'Content-Length': '0'} 2022-09-03T06:50:31.471 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000008d35f5736e35ed9-006312f937-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.472 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj74 2022-09-03T06:50:31.473 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:31.473 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:31.474 DEBUG:boto:path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.474 DEBUG:boto:auth_path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.475 DEBUG:boto:Method: GET 2022-09-03T06:50:31.475 DEBUG:boto:Path: /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.476 DEBUG:boto:Data: 2022-09-03T06:50:31.476 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.477 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:31.478 DEBUG:boto:Port: 8001 2022-09-03T06:50:31.478 DEBUG:boto:Params: {} 2022-09-03T06:50:31.479 DEBUG:boto:Token: None 2022-09-03T06:50:31.480 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.480 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4w7eo9Ts3U4UTRy8/d088SMCGqE= 2022-09-03T06:50:31.481 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4w7eo9Ts3U4UTRy8/d088SMCGqE=', 'Content-Length': '0'} 2022-09-03T06:50:31.501 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000031aee01ad93b4f10-006312f937-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.503 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:31.504 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:31.505 DEBUG:boto:path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.506 DEBUG:boto:auth_path=/dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.507 DEBUG:boto:Method: GET 2022-09-03T06:50:31.509 DEBUG:boto:Path: /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.510 DEBUG:boto:Data: 2022-09-03T06:50:31.511 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.512 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:31.519 DEBUG:boto:Port: 8000 2022-09-03T06:50:31.522 DEBUG:boto:Params: {} 2022-09-03T06:50:31.523 DEBUG:boto:Token: None 2022-09-03T06:50:31.524 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj74?versionId=null 2022-09-03T06:50:31.526 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:4w7eo9Ts3U4UTRy8/d088SMCGqE= 2022-09-03T06:50:31.528 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:4w7eo9Ts3U4UTRy8/d088SMCGqE=', 'Content-Length': '0'} 2022-09-03T06:50:31.530 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000029d39bd6dfeabf59-006312f937-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.531 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:31.532 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:31.533 DEBUG:boto:path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:31.533 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:31.534 DEBUG:boto:Method: GET 2022-09-03T06:50:31.535 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj74 2022-09-03T06:50:31.536 DEBUG:boto:Data: 2022-09-03T06:50:31.536 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.537 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:31.537 DEBUG:boto:Port: 8001 2022-09-03T06:50:31.538 DEBUG:boto:Params: {} 2022-09-03T06:50:31.539 DEBUG:boto:Token: None 2022-09-03T06:50:31.539 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/ 2022-09-03T06:50:31.540 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI= 2022-09-03T06:50:31.541 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI=', 'Content-Length': '0'} 2022-09-03T06:50:31.544 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ac91c504f9f740a5-006312f937-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.545 DEBUG:boto:b'dsgvcy-75obj741000falseobj742022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:31.546 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:31.547 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:31.548 DEBUG:boto:path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:31.548 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:31.549 DEBUG:boto:Method: GET 2022-09-03T06:50:31.549 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj74 2022-09-03T06:50:31.550 DEBUG:boto:Data: 2022-09-03T06:50:31.550 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.551 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:31.552 DEBUG:boto:Port: 8000 2022-09-03T06:50:31.553 DEBUG:boto:Params: {} 2022-09-03T06:50:31.553 DEBUG:boto:Token: None 2022-09-03T06:50:31.554 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/ 2022-09-03T06:50:31.555 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI= 2022-09-03T06:50:31.556 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI=', 'Content-Length': '0'} 2022-09-03T06:50:31.559 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000020f364a892ddf74d-006312f937-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.560 DEBUG:boto:b'dsgvcy-75obj741000falseobj742022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:31.561 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:31.561 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:31.562 DEBUG:boto:Method: HEAD 2022-09-03T06:50:31.563 DEBUG:boto:Path: /dsgvcy-75/obj74 2022-09-03T06:50:31.563 DEBUG:boto:Data: 2022-09-03T06:50:31.564 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.564 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:31.565 DEBUG:boto:Port: 8001 2022-09-03T06:50:31.565 DEBUG:boto:Params: {} 2022-09-03T06:50:31.566 DEBUG:boto:Token: None 2022-09-03T06:50:31.566 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj74 2022-09-03T06:50:31.567 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t+CrnjblppyDa1N6mFkDqgApD5k= 2022-09-03T06:50:31.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t+CrnjblppyDa1N6mFkDqgApD5k=', 'Content-Length': '0'} 2022-09-03T06:50:31.570 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c6aac107ce25feef-006312f937-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.570 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:31.571 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:31.572 DEBUG:boto:Method: HEAD 2022-09-03T06:50:31.572 DEBUG:boto:Path: /dsgvcy-75/obj74 2022-09-03T06:50:31.573 DEBUG:boto:Data: 2022-09-03T06:50:31.573 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.574 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:31.574 DEBUG:boto:Port: 8000 2022-09-03T06:50:31.575 DEBUG:boto:Params: {} 2022-09-03T06:50:31.575 DEBUG:boto:Token: None 2022-09-03T06:50:31.576 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj74 2022-09-03T06:50:31.577 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:t+CrnjblppyDa1N6mFkDqgApD5k= 2022-09-03T06:50:31.577 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:t+CrnjblppyDa1N6mFkDqgApD5k=', 'Content-Length': '0'} 2022-09-03T06:50:31.580 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bb6e8906be14fe78-006312f937-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.581 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj74 2022-09-03T06:50:31.582 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:31.583 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:31.583 DEBUG:boto:Method: GET 2022-09-03T06:50:31.584 DEBUG:boto:Path: /dsgvcy-75/obj74 2022-09-03T06:50:31.584 DEBUG:boto:Data: 2022-09-03T06:50:31.585 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.586 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:31.586 DEBUG:boto:Port: 8001 2022-09-03T06:50:31.587 DEBUG:boto:Params: {} 2022-09-03T06:50:31.588 DEBUG:boto:Token: None 2022-09-03T06:50:31.588 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj74 2022-09-03T06:50:31.589 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z6FN9xS9lK3W9SRAh6XXF9vKgys= 2022-09-03T06:50:31.590 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z6FN9xS9lK3W9SRAh6XXF9vKgys=', 'Content-Length': '0'} 2022-09-03T06:50:31.593 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006db2711c0577bedb-006312f937-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.594 DEBUG:boto:path=/dsgvcy-75/obj74 2022-09-03T06:50:31.594 DEBUG:boto:auth_path=/dsgvcy-75/obj74 2022-09-03T06:50:31.595 DEBUG:boto:Method: GET 2022-09-03T06:50:31.595 DEBUG:boto:Path: /dsgvcy-75/obj74 2022-09-03T06:50:31.596 DEBUG:boto:Data: 2022-09-03T06:50:31.596 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.597 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:31.598 DEBUG:boto:Port: 8000 2022-09-03T06:50:31.598 DEBUG:boto:Params: {} 2022-09-03T06:50:31.599 DEBUG:boto:Token: None 2022-09-03T06:50:31.599 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj74 2022-09-03T06:50:31.600 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Z6FN9xS9lK3W9SRAh6XXF9vKgys= 2022-09-03T06:50:31.601 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Z6FN9xS9lK3W9SRAh6XXF9vKgys=', 'Content-Length': '0'} 2022-09-03T06:50:31.603 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:47:49 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000563f5eb5dbc9e1ac-006312f937-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.646 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:31.647 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:31.648 DEBUG:boto:path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:31.649 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:31.651 DEBUG:boto:Method: GET 2022-09-03T06:50:31.652 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj74 2022-09-03T06:50:31.652 DEBUG:boto:Data: 2022-09-03T06:50:31.653 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.654 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:31.655 DEBUG:boto:Port: 8001 2022-09-03T06:50:31.656 DEBUG:boto:Params: {} 2022-09-03T06:50:31.657 DEBUG:boto:Token: None 2022-09-03T06:50:31.659 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/ 2022-09-03T06:50:31.660 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI= 2022-09-03T06:50:31.661 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI=', 'Content-Length': '0'} 2022-09-03T06:50:31.669 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000fa6450afb64d35d9-006312f937-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.671 DEBUG:boto:b'dsgvcy-75obj741000falseobj742022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:31.672 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:31.673 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:31.675 DEBUG:boto:path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:31.676 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj74 2022-09-03T06:50:31.677 DEBUG:boto:Method: GET 2022-09-03T06:50:31.678 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj74 2022-09-03T06:50:31.679 DEBUG:boto:Data: 2022-09-03T06:50:31.680 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.681 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:31.682 DEBUG:boto:Port: 8000 2022-09-03T06:50:31.683 DEBUG:boto:Params: {} 2022-09-03T06:50:31.685 DEBUG:boto:Token: None 2022-09-03T06:50:31.686 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/ 2022-09-03T06:50:31.687 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI= 2022-09-03T06:50:31.688 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI=', 'Content-Length': '0'} 2022-09-03T06:50:31.692 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000007a420d4a23e6bcaf-006312f937-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.693 DEBUG:boto:b'dsgvcy-75obj741000falseobj742022-09-03T06:47:49.594Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:31.695 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:50:31.696 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:31.697 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:31.698 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.699 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.700 DEBUG:boto:Method: GET 2022-09-03T06:50:31.702 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.703 DEBUG:boto:Data: 2022-09-03T06:50:31.704 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.705 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:31.706 DEBUG:boto:Port: 8001 2022-09-03T06:50:31.707 DEBUG:boto:Params: {} 2022-09-03T06:50:31.708 DEBUG:boto:Token: None 2022-09-03T06:50:31.709 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.710 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:coFa2SmfWipjGHI2Zb8IDf4TA6k= 2022-09-03T06:50:31.712 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:coFa2SmfWipjGHI2Zb8IDf4TA6k=', 'Content-Length': '0'} 2022-09-03T06:50:31.715 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a0146bc8b91172d8-006312f937-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.716 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:31.717 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:31.717 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.718 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.718 DEBUG:boto:Method: GET 2022-09-03T06:50:31.719 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.720 DEBUG:boto:Data: 2022-09-03T06:50:31.721 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.722 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:31.723 DEBUG:boto:Port: 8000 2022-09-03T06:50:31.724 DEBUG:boto:Params: {} 2022-09-03T06:50:31.725 DEBUG:boto:Token: None 2022-09-03T06:50:31.726 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.727 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:coFa2SmfWipjGHI2Zb8IDf4TA6k= 2022-09-03T06:50:31.728 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:coFa2SmfWipjGHI2Zb8IDf4TA6k=', 'Content-Length': '0'} 2022-09-03T06:50:31.732 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000220479099f3b7f6f-006312f937-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.733 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:31.734 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:31.735 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.736 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.737 DEBUG:boto:Method: HEAD 2022-09-03T06:50:31.738 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.740 DEBUG:boto:Data: 2022-09-03T06:50:31.741 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.742 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:31.743 DEBUG:boto:Port: 8001 2022-09-03T06:50:31.744 DEBUG:boto:Params: {} 2022-09-03T06:50:31.745 DEBUG:boto:Token: None 2022-09-03T06:50:31.746 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.747 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cE8I5megZA8KLG/Eqdiepn7kNyU= 2022-09-03T06:50:31.749 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cE8I5megZA8KLG/Eqdiepn7kNyU=', 'Content-Length': '0'} 2022-09-03T06:50:31.760 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d122f4e9d771f0a6-006312f937-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.762 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:31.763 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:31.765 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.765 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.766 DEBUG:boto:Method: HEAD 2022-09-03T06:50:31.770 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.771 DEBUG:boto:Data: 2022-09-03T06:50:31.772 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.773 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:31.775 DEBUG:boto:Port: 8000 2022-09-03T06:50:31.775 DEBUG:boto:Params: {} 2022-09-03T06:50:31.776 DEBUG:boto:Token: None 2022-09-03T06:50:31.777 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.777 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:cE8I5megZA8KLG/Eqdiepn7kNyU= 2022-09-03T06:50:31.778 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:cE8I5megZA8KLG/Eqdiepn7kNyU=', 'Content-Length': '0'} 2022-09-03T06:50:31.781 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000015e6ea917ec89aac-006312f937-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.782 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:50:31.783 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:31.785 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:31.786 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.786 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.787 DEBUG:boto:Method: GET 2022-09-03T06:50:31.788 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.789 DEBUG:boto:Data: 2022-09-03T06:50:31.790 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.791 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:31.792 DEBUG:boto:Port: 8001 2022-09-03T06:50:31.792 DEBUG:boto:Params: {} 2022-09-03T06:50:31.793 DEBUG:boto:Token: None 2022-09-03T06:50:31.794 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.795 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:coFa2SmfWipjGHI2Zb8IDf4TA6k= 2022-09-03T06:50:31.796 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:coFa2SmfWipjGHI2Zb8IDf4TA6k=', 'Content-Length': '0'} 2022-09-03T06:50:31.799 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000043dc6b6668ad89af-006312f937-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.845 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:31.846 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:31.847 DEBUG:boto:path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.847 DEBUG:boto:auth_path=/dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.848 DEBUG:boto:Method: GET 2022-09-03T06:50:31.849 DEBUG:boto:Path: /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.849 DEBUG:boto:Data: 2022-09-03T06:50:31.850 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.850 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:31.852 DEBUG:boto:Port: 8000 2022-09-03T06:50:31.853 DEBUG:boto:Params: {} 2022-09-03T06:50:31.854 DEBUG:boto:Token: None 2022-09-03T06:50:31.856 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj76?versionId=null 2022-09-03T06:50:31.858 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:coFa2SmfWipjGHI2Zb8IDf4TA6k= 2022-09-03T06:50:31.859 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:coFa2SmfWipjGHI2Zb8IDf4TA6k=', 'Content-Length': '0'} 2022-09-03T06:50:31.861 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e9f07313aa68003b-006312f937-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.863 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:31.863 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:31.864 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:31.865 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:31.866 DEBUG:boto:Method: GET 2022-09-03T06:50:31.867 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:50:31.868 DEBUG:boto:Data: 2022-09-03T06:50:31.869 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.870 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:31.871 DEBUG:boto:Port: 8001 2022-09-03T06:50:31.872 DEBUG:boto:Params: {} 2022-09-03T06:50:31.872 DEBUG:boto:Token: None 2022-09-03T06:50:31.873 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/ 2022-09-03T06:50:31.874 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI= 2022-09-03T06:50:31.875 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI=', 'Content-Length': '0'} 2022-09-03T06:50:31.881 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000067489c0c82d9334c-006312f937-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.883 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:31.885 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:31.885 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:31.886 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:31.887 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:31.888 DEBUG:boto:Method: GET 2022-09-03T06:50:31.889 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:50:31.893 DEBUG:boto:Data: 2022-09-03T06:50:31.896 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.897 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:31.898 DEBUG:boto:Port: 8000 2022-09-03T06:50:31.898 DEBUG:boto:Params: {} 2022-09-03T06:50:31.900 DEBUG:boto:Token: None 2022-09-03T06:50:31.901 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/ 2022-09-03T06:50:31.902 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI= 2022-09-03T06:50:31.903 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:glotSylJxRZmbG4eIJAGK1UPEdI=', 'Content-Length': '0'} 2022-09-03T06:50:31.906 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c57571b42d25550f-006312f937-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.908 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:31.908 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:31.909 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:31.910 DEBUG:boto:Method: HEAD 2022-09-03T06:50:31.910 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:50:31.911 DEBUG:boto:Data: 2022-09-03T06:50:31.911 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.912 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:31.912 DEBUG:boto:Port: 8001 2022-09-03T06:50:31.913 DEBUG:boto:Params: {} 2022-09-03T06:50:31.913 DEBUG:boto:Token: None 2022-09-03T06:50:31.914 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj76 2022-09-03T06:50:31.915 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qMuWYwVw0HJzO98TGwGYpAFlTLk= 2022-09-03T06:50:31.915 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qMuWYwVw0HJzO98TGwGYpAFlTLk=', 'Content-Length': '0'} 2022-09-03T06:50:31.918 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000e528f65233e6e6e5-006312f937-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.919 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:31.920 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:31.920 DEBUG:boto:Method: HEAD 2022-09-03T06:50:31.921 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:50:31.921 DEBUG:boto:Data: 2022-09-03T06:50:31.922 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.922 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:31.923 DEBUG:boto:Port: 8000 2022-09-03T06:50:31.924 DEBUG:boto:Params: {} 2022-09-03T06:50:31.924 DEBUG:boto:Token: None 2022-09-03T06:50:31.925 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj76 2022-09-03T06:50:31.925 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qMuWYwVw0HJzO98TGwGYpAFlTLk= 2022-09-03T06:50:31.926 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qMuWYwVw0HJzO98TGwGYpAFlTLk=', 'Content-Length': '0'} 2022-09-03T06:50:31.928 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ad2d655cca00610f-006312f937-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.930 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj76 2022-09-03T06:50:31.931 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:31.932 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:31.933 DEBUG:boto:Method: GET 2022-09-03T06:50:31.934 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:50:31.935 DEBUG:boto:Data: 2022-09-03T06:50:31.936 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.937 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:31.938 DEBUG:boto:Port: 8001 2022-09-03T06:50:31.939 DEBUG:boto:Params: {} 2022-09-03T06:50:31.940 DEBUG:boto:Token: None 2022-09-03T06:50:31.941 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj76 2022-09-03T06:50:31.942 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b9maramNmodUEWDfxJOzEJ2jox4= 2022-09-03T06:50:31.943 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b9maramNmodUEWDfxJOzEJ2jox4=', 'Content-Length': '0'} 2022-09-03T06:50:31.945 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000304c9d24aa8fcee3-006312f937-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.951 DEBUG:boto:path=/dsgvcy-75/obj76 2022-09-03T06:50:31.953 DEBUG:boto:auth_path=/dsgvcy-75/obj76 2022-09-03T06:50:31.954 DEBUG:boto:Method: GET 2022-09-03T06:50:31.955 DEBUG:boto:Path: /dsgvcy-75/obj76 2022-09-03T06:50:31.956 DEBUG:boto:Data: 2022-09-03T06:50:31.958 DEBUG:boto:Headers: {} 2022-09-03T06:50:31.959 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:31.960 DEBUG:boto:Port: 8000 2022-09-03T06:50:31.961 DEBUG:boto:Params: {} 2022-09-03T06:50:31.962 DEBUG:boto:Token: None 2022-09-03T06:50:31.963 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:31 GMT /dsgvcy-75/obj76 2022-09-03T06:50:31.965 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:b9maramNmodUEWDfxJOzEJ2jox4= 2022-09-03T06:50:31.966 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:31 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:b9maramNmodUEWDfxJOzEJ2jox4=', 'Content-Length': '0'} 2022-09-03T06:50:31.969 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008eb31de4e815311c-006312f937-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:31 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:31.990 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:31.992 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:31.993 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:31.995 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:31.996 DEBUG:boto:Method: GET 2022-09-03T06:50:31.998 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:50:31.999 DEBUG:boto:Data: 2022-09-03T06:50:32.001 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.004 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:32.006 DEBUG:boto:Port: 8001 2022-09-03T06:50:32.007 DEBUG:boto:Params: {} 2022-09-03T06:50:32.008 DEBUG:boto:Token: None 2022-09-03T06:50:32.010 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/ 2022-09-03T06:50:32.013 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM= 2022-09-03T06:50:32.014 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM=', 'Content-Length': '0'} 2022-09-03T06:50:32.019 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000a8d77ef4144448e8-006312f938-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.020 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:32.022 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:32.024 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:32.024 DEBUG:boto:path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:32.026 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj76 2022-09-03T06:50:32.027 DEBUG:boto:Method: GET 2022-09-03T06:50:32.028 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj76 2022-09-03T06:50:32.029 DEBUG:boto:Data: 2022-09-03T06:50:32.030 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.031 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:32.032 DEBUG:boto:Port: 8000 2022-09-03T06:50:32.033 DEBUG:boto:Params: {} 2022-09-03T06:50:32.034 DEBUG:boto:Token: None 2022-09-03T06:50:32.035 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/ 2022-09-03T06:50:32.036 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM= 2022-09-03T06:50:32.037 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM=', 'Content-Length': '0'} 2022-09-03T06:50:32.041 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000e03b5cbfb1a285fe-006312f938-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.042 DEBUG:boto:b'dsgvcy-75obj761000falseobj762022-09-03T06:40:53.336Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:32.043 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:50:32.044 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:32.045 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:32.046 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.048 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.049 DEBUG:boto:Method: GET 2022-09-03T06:50:32.049 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.050 DEBUG:boto:Data: 2022-09-03T06:50:32.051 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.051 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:32.052 DEBUG:boto:Port: 8001 2022-09-03T06:50:32.052 DEBUG:boto:Params: {} 2022-09-03T06:50:32.053 DEBUG:boto:Token: None 2022-09-03T06:50:32.054 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.054 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wZi7QVo5FZCco3bU9J97kgQqFgE= 2022-09-03T06:50:32.055 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wZi7QVo5FZCco3bU9J97kgQqFgE=', 'Content-Length': '0'} 2022-09-03T06:50:32.065 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001bedf67857e946a1-006312f938-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.107 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:32.115 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:32.118 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.131 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.136 DEBUG:boto:Method: GET 2022-09-03T06:50:32.137 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.138 DEBUG:boto:Data: 2022-09-03T06:50:32.139 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.141 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:32.142 DEBUG:boto:Port: 8000 2022-09-03T06:50:32.143 DEBUG:boto:Params: {} 2022-09-03T06:50:32.144 DEBUG:boto:Token: None 2022-09-03T06:50:32.145 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.146 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wZi7QVo5FZCco3bU9J97kgQqFgE= 2022-09-03T06:50:32.147 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wZi7QVo5FZCco3bU9J97kgQqFgE=', 'Content-Length': '0'} 2022-09-03T06:50:32.162 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000016a5cd7d6414f7a8-006312f938-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.168 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:32.170 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:32.171 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.172 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.173 DEBUG:boto:Method: HEAD 2022-09-03T06:50:32.174 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.175 DEBUG:boto:Data: 2022-09-03T06:50:32.176 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.177 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:32.178 DEBUG:boto:Port: 8001 2022-09-03T06:50:32.179 DEBUG:boto:Params: {} 2022-09-03T06:50:32.180 DEBUG:boto:Token: None 2022-09-03T06:50:32.181 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.182 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZbhXO/cGbi6e0vAFwrnYP+d+iho= 2022-09-03T06:50:32.183 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZbhXO/cGbi6e0vAFwrnYP+d+iho=', 'Content-Length': '0'} 2022-09-03T06:50:32.187 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009d0bd623f402bf09-006312f938-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.189 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:32.190 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:32.191 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.192 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.193 DEBUG:boto:Method: HEAD 2022-09-03T06:50:32.194 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.195 DEBUG:boto:Data: 2022-09-03T06:50:32.196 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.197 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:32.198 DEBUG:boto:Port: 8000 2022-09-03T06:50:32.199 DEBUG:boto:Params: {} 2022-09-03T06:50:32.200 DEBUG:boto:Token: None 2022-09-03T06:50:32.201 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.202 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZbhXO/cGbi6e0vAFwrnYP+d+iho= 2022-09-03T06:50:32.203 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZbhXO/cGbi6e0vAFwrnYP+d+iho=', 'Content-Length': '0'} 2022-09-03T06:50:32.206 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000009eac844525903ae4-006312f938-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.208 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:50:32.211 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:32.218 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:32.219 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.220 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.221 DEBUG:boto:Method: GET 2022-09-03T06:50:32.223 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.224 DEBUG:boto:Data: 2022-09-03T06:50:32.225 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.226 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:32.227 DEBUG:boto:Port: 8001 2022-09-03T06:50:32.228 DEBUG:boto:Params: {} 2022-09-03T06:50:32.230 DEBUG:boto:Token: None 2022-09-03T06:50:32.231 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.233 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wZi7QVo5FZCco3bU9J97kgQqFgE= 2022-09-03T06:50:32.234 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wZi7QVo5FZCco3bU9J97kgQqFgE=', 'Content-Length': '0'} 2022-09-03T06:50:32.241 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000897be276a2b47251-006312f938-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.243 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:32.244 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:32.245 DEBUG:boto:path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.245 DEBUG:boto:auth_path=/dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.246 DEBUG:boto:Method: GET 2022-09-03T06:50:32.247 DEBUG:boto:Path: /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.248 DEBUG:boto:Data: 2022-09-03T06:50:32.248 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.249 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:32.251 DEBUG:boto:Port: 8000 2022-09-03T06:50:32.252 DEBUG:boto:Params: {} 2022-09-03T06:50:32.253 DEBUG:boto:Token: None 2022-09-03T06:50:32.254 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj85?versionId=null 2022-09-03T06:50:32.256 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:wZi7QVo5FZCco3bU9J97kgQqFgE= 2022-09-03T06:50:32.257 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:wZi7QVo5FZCco3bU9J97kgQqFgE=', 'Content-Length': '0'} 2022-09-03T06:50:32.260 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a62b87604eddf962-006312f938-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.262 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:32.263 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:32.264 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:32.265 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:32.267 DEBUG:boto:Method: GET 2022-09-03T06:50:32.270 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:50:32.271 DEBUG:boto:Data: 2022-09-03T06:50:32.272 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.273 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:32.275 DEBUG:boto:Port: 8001 2022-09-03T06:50:32.276 DEBUG:boto:Params: {} 2022-09-03T06:50:32.277 DEBUG:boto:Token: None 2022-09-03T06:50:32.279 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/ 2022-09-03T06:50:32.280 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM= 2022-09-03T06:50:32.281 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM=', 'Content-Length': '0'} 2022-09-03T06:50:32.290 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000076fd31dc75b60188-006312f938-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.292 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:32.295 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:32.296 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:32.297 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:32.299 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:32.300 DEBUG:boto:Method: GET 2022-09-03T06:50:32.302 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:50:32.303 DEBUG:boto:Data: 2022-09-03T06:50:32.305 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.306 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:32.307 DEBUG:boto:Port: 8000 2022-09-03T06:50:32.307 DEBUG:boto:Params: {} 2022-09-03T06:50:32.308 DEBUG:boto:Token: None 2022-09-03T06:50:32.310 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/ 2022-09-03T06:50:32.310 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM= 2022-09-03T06:50:32.311 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM=', 'Content-Length': '0'} 2022-09-03T06:50:32.314 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003be3e6dfef061d7a-006312f938-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.315 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:32.316 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:32.317 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:32.317 DEBUG:boto:Method: HEAD 2022-09-03T06:50:32.318 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:50:32.319 DEBUG:boto:Data: 2022-09-03T06:50:32.319 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.320 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:32.320 DEBUG:boto:Port: 8001 2022-09-03T06:50:32.321 DEBUG:boto:Params: {} 2022-09-03T06:50:32.321 DEBUG:boto:Token: None 2022-09-03T06:50:32.322 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj85 2022-09-03T06:50:32.323 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZoUni9KhCQxgzQtv6SLAKdZQMT0= 2022-09-03T06:50:32.323 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZoUni9KhCQxgzQtv6SLAKdZQMT0=', 'Content-Length': '0'} 2022-09-03T06:50:32.326 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000054bbeeb4a97abb2f-006312f938-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.327 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:32.328 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:32.329 DEBUG:boto:Method: HEAD 2022-09-03T06:50:32.329 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:50:32.330 DEBUG:boto:Data: 2022-09-03T06:50:32.331 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.332 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:32.333 DEBUG:boto:Port: 8000 2022-09-03T06:50:32.333 DEBUG:boto:Params: {} 2022-09-03T06:50:32.334 DEBUG:boto:Token: None 2022-09-03T06:50:32.335 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj85 2022-09-03T06:50:32.335 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ZoUni9KhCQxgzQtv6SLAKdZQMT0= 2022-09-03T06:50:32.336 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ZoUni9KhCQxgzQtv6SLAKdZQMT0=', 'Content-Length': '0'} 2022-09-03T06:50:32.339 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001b546be8199f8864-006312f938-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.341 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj85 2022-09-03T06:50:32.342 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:32.343 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:32.344 DEBUG:boto:Method: GET 2022-09-03T06:50:32.345 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:50:32.346 DEBUG:boto:Data: 2022-09-03T06:50:32.347 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.348 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:32.349 DEBUG:boto:Port: 8001 2022-09-03T06:50:32.350 DEBUG:boto:Params: {} 2022-09-03T06:50:32.352 DEBUG:boto:Token: None 2022-09-03T06:50:32.353 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj85 2022-09-03T06:50:32.354 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LJWRE/GGcJBDZUX0gUfAHMJMIkw= 2022-09-03T06:50:32.355 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LJWRE/GGcJBDZUX0gUfAHMJMIkw=', 'Content-Length': '0'} 2022-09-03T06:50:32.363 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000bb2b3d5d3ae962b4-006312f938-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.365 DEBUG:boto:path=/dsgvcy-75/obj85 2022-09-03T06:50:32.366 DEBUG:boto:auth_path=/dsgvcy-75/obj85 2022-09-03T06:50:32.367 DEBUG:boto:Method: GET 2022-09-03T06:50:32.368 DEBUG:boto:Path: /dsgvcy-75/obj85 2022-09-03T06:50:32.369 DEBUG:boto:Data: 2022-09-03T06:50:32.370 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.371 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:32.371 DEBUG:boto:Port: 8000 2022-09-03T06:50:32.372 DEBUG:boto:Params: {} 2022-09-03T06:50:32.373 DEBUG:boto:Token: None 2022-09-03T06:50:32.374 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj85 2022-09-03T06:50:32.375 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LJWRE/GGcJBDZUX0gUfAHMJMIkw= 2022-09-03T06:50:32.375 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LJWRE/GGcJBDZUX0gUfAHMJMIkw=', 'Content-Length': '0'} 2022-09-03T06:50:32.378 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:37:27 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008f57e806eb270599-006312f938-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.422 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:32.423 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:32.424 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:32.424 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:32.425 DEBUG:boto:Method: GET 2022-09-03T06:50:32.425 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:50:32.426 DEBUG:boto:Data: 2022-09-03T06:50:32.427 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.428 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:32.428 DEBUG:boto:Port: 8001 2022-09-03T06:50:32.429 DEBUG:boto:Params: {} 2022-09-03T06:50:32.430 DEBUG:boto:Token: None 2022-09-03T06:50:32.431 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/ 2022-09-03T06:50:32.432 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM= 2022-09-03T06:50:32.432 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM=', 'Content-Length': '0'} 2022-09-03T06:50:32.435 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000009ee072c4c79e6a74-006312f938-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.436 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:32.437 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:32.437 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:32.438 DEBUG:boto:path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:32.439 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj85 2022-09-03T06:50:32.439 DEBUG:boto:Method: GET 2022-09-03T06:50:32.440 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj85 2022-09-03T06:50:32.441 DEBUG:boto:Data: 2022-09-03T06:50:32.441 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.442 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:32.443 DEBUG:boto:Port: 8000 2022-09-03T06:50:32.443 DEBUG:boto:Params: {} 2022-09-03T06:50:32.445 DEBUG:boto:Token: None 2022-09-03T06:50:32.445 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/ 2022-09-03T06:50:32.446 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM= 2022-09-03T06:50:32.456 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM=', 'Content-Length': '0'} 2022-09-03T06:50:32.460 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000006e19b5c6fb786f2-006312f938-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.464 DEBUG:boto:b'dsgvcy-75obj851000falseobj852022-09-03T06:37:27.217Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:32.465 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:50:32.466 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:32.467 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:32.468 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.468 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.469 DEBUG:boto:Method: GET 2022-09-03T06:50:32.470 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.471 DEBUG:boto:Data: 2022-09-03T06:50:32.471 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.472 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:32.472 DEBUG:boto:Port: 8001 2022-09-03T06:50:32.473 DEBUG:boto:Params: {} 2022-09-03T06:50:32.474 DEBUG:boto:Token: None 2022-09-03T06:50:32.474 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.475 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KlcGBJ02I7LZ6FkeREUDFDv+9Rg= 2022-09-03T06:50:32.476 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KlcGBJ02I7LZ6FkeREUDFDv+9Rg=', 'Content-Length': '0'} 2022-09-03T06:50:32.479 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000015c5f79c18eebf13-006312f938-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.481 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:32.482 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:32.482 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.483 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.483 DEBUG:boto:Method: GET 2022-09-03T06:50:32.484 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.484 DEBUG:boto:Data: 2022-09-03T06:50:32.485 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.485 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:32.486 DEBUG:boto:Port: 8000 2022-09-03T06:50:32.486 DEBUG:boto:Params: {} 2022-09-03T06:50:32.488 DEBUG:boto:Token: None 2022-09-03T06:50:32.489 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.489 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KlcGBJ02I7LZ6FkeREUDFDv+9Rg= 2022-09-03T06:50:32.490 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KlcGBJ02I7LZ6FkeREUDFDv+9Rg=', 'Content-Length': '0'} 2022-09-03T06:50:32.492 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fcb2f71da0339e03-006312f938-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.493 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:32.494 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:32.494 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.495 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.495 DEBUG:boto:Method: HEAD 2022-09-03T06:50:32.496 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.496 DEBUG:boto:Data: 2022-09-03T06:50:32.497 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.498 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:32.499 DEBUG:boto:Port: 8001 2022-09-03T06:50:32.499 DEBUG:boto:Params: {} 2022-09-03T06:50:32.500 DEBUG:boto:Token: None 2022-09-03T06:50:32.500 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.501 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o10sUU3gPR229/bETR7tbVH9xno= 2022-09-03T06:50:32.502 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o10sUU3gPR229/bETR7tbVH9xno=', 'Content-Length': '0'} 2022-09-03T06:50:32.504 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000962c62ce9f66a0c5-006312f938-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.504 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:32.505 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:32.505 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.506 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.507 DEBUG:boto:Method: HEAD 2022-09-03T06:50:32.507 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.508 DEBUG:boto:Data: 2022-09-03T06:50:32.509 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.510 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:32.510 DEBUG:boto:Port: 8000 2022-09-03T06:50:32.511 DEBUG:boto:Params: {} 2022-09-03T06:50:32.511 DEBUG:boto:Token: None 2022-09-03T06:50:32.512 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.513 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:o10sUU3gPR229/bETR7tbVH9xno= 2022-09-03T06:50:32.513 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:o10sUU3gPR229/bETR7tbVH9xno=', 'Content-Length': '0'} 2022-09-03T06:50:32.516 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000eac68664742c0981-006312f938-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.518 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:50:32.519 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:32.520 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:32.521 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.522 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.524 DEBUG:boto:Method: GET 2022-09-03T06:50:32.525 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.526 DEBUG:boto:Data: 2022-09-03T06:50:32.527 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.528 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:32.529 DEBUG:boto:Port: 8001 2022-09-03T06:50:32.530 DEBUG:boto:Params: {} 2022-09-03T06:50:32.531 DEBUG:boto:Token: None 2022-09-03T06:50:32.531 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.532 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KlcGBJ02I7LZ6FkeREUDFDv+9Rg= 2022-09-03T06:50:32.533 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KlcGBJ02I7LZ6FkeREUDFDv+9Rg=', 'Content-Length': '0'} 2022-09-03T06:50:32.539 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ed1a08e95c19d556-006312f938-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.541 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:32.542 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:32.542 DEBUG:boto:path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.543 DEBUG:boto:auth_path=/dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.544 DEBUG:boto:Method: GET 2022-09-03T06:50:32.545 DEBUG:boto:Path: /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.546 DEBUG:boto:Data: 2022-09-03T06:50:32.546 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.548 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:32.548 DEBUG:boto:Port: 8000 2022-09-03T06:50:32.550 DEBUG:boto:Params: {} 2022-09-03T06:50:32.551 DEBUG:boto:Token: None 2022-09-03T06:50:32.552 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj87?versionId=null 2022-09-03T06:50:32.553 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KlcGBJ02I7LZ6FkeREUDFDv+9Rg= 2022-09-03T06:50:32.554 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KlcGBJ02I7LZ6FkeREUDFDv+9Rg=', 'Content-Length': '0'} 2022-09-03T06:50:32.557 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f2c101b76faf30ae-006312f938-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.559 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:32.559 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:32.560 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:32.560 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:32.561 DEBUG:boto:Method: GET 2022-09-03T06:50:32.561 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:50:32.562 DEBUG:boto:Data: 2022-09-03T06:50:32.563 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.563 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:32.564 DEBUG:boto:Port: 8001 2022-09-03T06:50:32.565 DEBUG:boto:Params: {} 2022-09-03T06:50:32.565 DEBUG:boto:Token: None 2022-09-03T06:50:32.566 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/ 2022-09-03T06:50:32.567 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM= 2022-09-03T06:50:32.568 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM=', 'Content-Length': '0'} 2022-09-03T06:50:32.576 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000b11c501a2f2f385f-006312f938-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.622 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:32.624 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:32.625 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:32.626 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:32.627 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:32.628 DEBUG:boto:Method: GET 2022-09-03T06:50:32.629 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:50:32.630 DEBUG:boto:Data: 2022-09-03T06:50:32.631 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.631 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:32.632 DEBUG:boto:Port: 8000 2022-09-03T06:50:32.634 DEBUG:boto:Params: {} 2022-09-03T06:50:32.635 DEBUG:boto:Token: None 2022-09-03T06:50:32.636 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/ 2022-09-03T06:50:32.636 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM= 2022-09-03T06:50:32.637 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM=', 'Content-Length': '0'} 2022-09-03T06:50:32.641 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000809b33dc1868e10d-006312f938-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.643 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:32.644 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:32.645 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:32.646 DEBUG:boto:Method: HEAD 2022-09-03T06:50:32.646 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:50:32.647 DEBUG:boto:Data: 2022-09-03T06:50:32.648 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.648 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:32.649 DEBUG:boto:Port: 8001 2022-09-03T06:50:32.650 DEBUG:boto:Params: {} 2022-09-03T06:50:32.651 DEBUG:boto:Token: None 2022-09-03T06:50:32.652 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj87 2022-09-03T06:50:32.653 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6bBoMc4rjVO0/Ol12OBqu4pGWrg= 2022-09-03T06:50:32.654 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6bBoMc4rjVO0/Ol12OBqu4pGWrg=', 'Content-Length': '0'} 2022-09-03T06:50:32.657 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d60445620cc0a057-006312f938-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.658 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:32.659 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:32.660 DEBUG:boto:Method: HEAD 2022-09-03T06:50:32.661 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:50:32.662 DEBUG:boto:Data: 2022-09-03T06:50:32.663 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.664 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:32.665 DEBUG:boto:Port: 8000 2022-09-03T06:50:32.666 DEBUG:boto:Params: {} 2022-09-03T06:50:32.667 DEBUG:boto:Token: None 2022-09-03T06:50:32.670 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj87 2022-09-03T06:50:32.671 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:6bBoMc4rjVO0/Ol12OBqu4pGWrg= 2022-09-03T06:50:32.672 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:6bBoMc4rjVO0/Ol12OBqu4pGWrg=', 'Content-Length': '0'} 2022-09-03T06:50:32.675 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c9443dd599617f26-006312f938-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.676 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj87 2022-09-03T06:50:32.678 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:32.679 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:32.680 DEBUG:boto:Method: GET 2022-09-03T06:50:32.681 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:50:32.682 DEBUG:boto:Data: 2022-09-03T06:50:32.683 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.684 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:32.685 DEBUG:boto:Port: 8001 2022-09-03T06:50:32.686 DEBUG:boto:Params: {} 2022-09-03T06:50:32.687 DEBUG:boto:Token: None 2022-09-03T06:50:32.689 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj87 2022-09-03T06:50:32.690 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eQZRPzQYsjnNRFxT7hjTokyBleQ= 2022-09-03T06:50:32.691 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eQZRPzQYsjnNRFxT7hjTokyBleQ=', 'Content-Length': '0'} 2022-09-03T06:50:32.694 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000002e3df90594587ed1-006312f938-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.695 DEBUG:boto:path=/dsgvcy-75/obj87 2022-09-03T06:50:32.696 DEBUG:boto:auth_path=/dsgvcy-75/obj87 2022-09-03T06:50:32.697 DEBUG:boto:Method: GET 2022-09-03T06:50:32.699 DEBUG:boto:Path: /dsgvcy-75/obj87 2022-09-03T06:50:32.700 DEBUG:boto:Data: 2022-09-03T06:50:32.701 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.702 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:32.703 DEBUG:boto:Port: 8000 2022-09-03T06:50:32.703 DEBUG:boto:Params: {} 2022-09-03T06:50:32.704 DEBUG:boto:Token: None 2022-09-03T06:50:32.705 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj87 2022-09-03T06:50:32.705 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:eQZRPzQYsjnNRFxT7hjTokyBleQ= 2022-09-03T06:50:32.706 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:eQZRPzQYsjnNRFxT7hjTokyBleQ=', 'Content-Length': '0'} 2022-09-03T06:50:32.708 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:44:18 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ddbfceb9403e2211-006312f938-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.751 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:32.752 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:32.754 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:32.755 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:32.756 DEBUG:boto:Method: GET 2022-09-03T06:50:32.757 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:50:32.757 DEBUG:boto:Data: 2022-09-03T06:50:32.758 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.758 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:32.759 DEBUG:boto:Port: 8001 2022-09-03T06:50:32.759 DEBUG:boto:Params: {} 2022-09-03T06:50:32.760 DEBUG:boto:Token: None 2022-09-03T06:50:32.761 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/ 2022-09-03T06:50:32.761 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM= 2022-09-03T06:50:32.762 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM=', 'Content-Length': '0'} 2022-09-03T06:50:32.765 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c786f5133c6eeda4-006312f938-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.768 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:32.770 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:32.771 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:32.772 DEBUG:boto:path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:32.773 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj87 2022-09-03T06:50:32.774 DEBUG:boto:Method: GET 2022-09-03T06:50:32.776 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj87 2022-09-03T06:50:32.776 DEBUG:boto:Data: 2022-09-03T06:50:32.777 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.778 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:32.779 DEBUG:boto:Port: 8000 2022-09-03T06:50:32.779 DEBUG:boto:Params: {} 2022-09-03T06:50:32.780 DEBUG:boto:Token: None 2022-09-03T06:50:32.780 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/ 2022-09-03T06:50:32.781 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM= 2022-09-03T06:50:32.782 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM=', 'Content-Length': '0'} 2022-09-03T06:50:32.787 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c05990c7eefd45f4-006312f938-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.789 DEBUG:boto:b'dsgvcy-75obj871000falseobj872022-09-03T06:44:18.298Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:32.790 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:50:32.792 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:32.793 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:32.794 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.794 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.796 DEBUG:boto:Method: GET 2022-09-03T06:50:32.797 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.798 DEBUG:boto:Data: 2022-09-03T06:50:32.799 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.800 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:32.801 DEBUG:boto:Port: 8001 2022-09-03T06:50:32.802 DEBUG:boto:Params: {} 2022-09-03T06:50:32.803 DEBUG:boto:Token: None 2022-09-03T06:50:32.804 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.806 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SExt/3KA/uNoxxN6V2qZeCLXRJk= 2022-09-03T06:50:32.807 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SExt/3KA/uNoxxN6V2qZeCLXRJk=', 'Content-Length': '0'} 2022-09-03T06:50:32.810 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001c8139285d9a9e18-006312f938-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.811 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:32.812 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:32.812 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.813 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.813 DEBUG:boto:Method: GET 2022-09-03T06:50:32.814 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.814 DEBUG:boto:Data: 2022-09-03T06:50:32.815 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.816 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:32.816 DEBUG:boto:Port: 8000 2022-09-03T06:50:32.817 DEBUG:boto:Params: {} 2022-09-03T06:50:32.817 DEBUG:boto:Token: None 2022-09-03T06:50:32.818 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.819 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SExt/3KA/uNoxxN6V2qZeCLXRJk= 2022-09-03T06:50:32.819 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SExt/3KA/uNoxxN6V2qZeCLXRJk=', 'Content-Length': '0'} 2022-09-03T06:50:32.822 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f0430cdc42fa55d2-006312f938-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.823 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:32.823 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:32.824 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.824 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.825 DEBUG:boto:Method: HEAD 2022-09-03T06:50:32.825 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.826 DEBUG:boto:Data: 2022-09-03T06:50:32.827 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.827 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:32.828 DEBUG:boto:Port: 8001 2022-09-03T06:50:32.829 DEBUG:boto:Params: {} 2022-09-03T06:50:32.829 DEBUG:boto:Token: None 2022-09-03T06:50:32.830 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.831 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UAItRoAHteG4GQ8o59XPZoVVHgk= 2022-09-03T06:50:32.832 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UAItRoAHteG4GQ8o59XPZoVVHgk=', 'Content-Length': '0'} 2022-09-03T06:50:32.834 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000f9202f3d7b7df1f3-006312f938-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.835 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:32.835 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:32.836 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.836 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.837 DEBUG:boto:Method: HEAD 2022-09-03T06:50:32.837 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.838 DEBUG:boto:Data: 2022-09-03T06:50:32.838 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.839 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:32.839 DEBUG:boto:Port: 8000 2022-09-03T06:50:32.840 DEBUG:boto:Params: {} 2022-09-03T06:50:32.841 DEBUG:boto:Token: None 2022-09-03T06:50:32.841 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.842 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:UAItRoAHteG4GQ8o59XPZoVVHgk= 2022-09-03T06:50:32.842 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:UAItRoAHteG4GQ8o59XPZoVVHgk=', 'Content-Length': '0'} 2022-09-03T06:50:32.845 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000060d51b728aa7322a-006312f938-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.846 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:50:32.847 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:32.847 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:32.848 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.848 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.849 DEBUG:boto:Method: GET 2022-09-03T06:50:32.849 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.850 DEBUG:boto:Data: 2022-09-03T06:50:32.850 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.851 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:32.852 DEBUG:boto:Port: 8001 2022-09-03T06:50:32.852 DEBUG:boto:Params: {} 2022-09-03T06:50:32.853 DEBUG:boto:Token: None 2022-09-03T06:50:32.853 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.854 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SExt/3KA/uNoxxN6V2qZeCLXRJk= 2022-09-03T06:50:32.855 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SExt/3KA/uNoxxN6V2qZeCLXRJk=', 'Content-Length': '0'} 2022-09-03T06:50:32.857 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000000facd8a9fad8c30-006312f938-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.858 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:32.858 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:32.859 DEBUG:boto:path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.859 DEBUG:boto:auth_path=/dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.860 DEBUG:boto:Method: GET 2022-09-03T06:50:32.861 DEBUG:boto:Path: /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.861 DEBUG:boto:Data: 2022-09-03T06:50:32.862 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.862 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:32.863 DEBUG:boto:Port: 8000 2022-09-03T06:50:32.863 DEBUG:boto:Params: {} 2022-09-03T06:50:32.864 DEBUG:boto:Token: None 2022-09-03T06:50:32.865 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj92?versionId=null 2022-09-03T06:50:32.865 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:SExt/3KA/uNoxxN6V2qZeCLXRJk= 2022-09-03T06:50:32.866 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:SExt/3KA/uNoxxN6V2qZeCLXRJk=', 'Content-Length': '0'} 2022-09-03T06:50:32.868 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000c8fbf49a870322ce-006312f938-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.869 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:32.869 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:32.870 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:32.870 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:32.871 DEBUG:boto:Method: GET 2022-09-03T06:50:32.872 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:50:32.872 DEBUG:boto:Data: 2022-09-03T06:50:32.873 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.873 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:32.874 DEBUG:boto:Port: 8001 2022-09-03T06:50:32.874 DEBUG:boto:Params: {} 2022-09-03T06:50:32.875 DEBUG:boto:Token: None 2022-09-03T06:50:32.875 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/ 2022-09-03T06:50:32.876 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM= 2022-09-03T06:50:32.877 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM=', 'Content-Length': '0'} 2022-09-03T06:50:32.880 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000730671540bdc7c82-006312f938-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.880 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:32.881 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:32.882 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:32.882 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:32.883 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:32.883 DEBUG:boto:Method: GET 2022-09-03T06:50:32.884 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:50:32.884 DEBUG:boto:Data: 2022-09-03T06:50:32.885 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.885 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:32.886 DEBUG:boto:Port: 8000 2022-09-03T06:50:32.886 DEBUG:boto:Params: {} 2022-09-03T06:50:32.887 DEBUG:boto:Token: None 2022-09-03T06:50:32.888 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/ 2022-09-03T06:50:32.888 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM= 2022-09-03T06:50:32.889 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:VW6HKG+px9vOtNAsLwfXV70r6ZM=', 'Content-Length': '0'} 2022-09-03T06:50:32.892 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000099861ff58322467b-006312f938-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.892 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:32.893 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:32.894 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:32.895 DEBUG:boto:Method: HEAD 2022-09-03T06:50:32.895 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:50:32.896 DEBUG:boto:Data: 2022-09-03T06:50:32.896 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.897 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:32.897 DEBUG:boto:Port: 8001 2022-09-03T06:50:32.898 DEBUG:boto:Params: {} 2022-09-03T06:50:32.898 DEBUG:boto:Token: None 2022-09-03T06:50:32.899 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj92 2022-09-03T06:50:32.900 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iAx+GcQ+BunpynuqoskuSFQngoc= 2022-09-03T06:50:32.900 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iAx+GcQ+BunpynuqoskuSFQngoc=', 'Content-Length': '0'} 2022-09-03T06:50:32.902 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000cc8ad463aad3b18e-006312f938-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.903 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:32.903 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:32.904 DEBUG:boto:Method: HEAD 2022-09-03T06:50:32.904 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:50:32.905 DEBUG:boto:Data: 2022-09-03T06:50:32.905 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.906 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:32.906 DEBUG:boto:Port: 8000 2022-09-03T06:50:32.907 DEBUG:boto:Params: {} 2022-09-03T06:50:32.908 DEBUG:boto:Token: None 2022-09-03T06:50:32.908 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj92 2022-09-03T06:50:32.909 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:iAx+GcQ+BunpynuqoskuSFQngoc= 2022-09-03T06:50:32.910 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:iAx+GcQ+BunpynuqoskuSFQngoc=', 'Content-Length': '0'} 2022-09-03T06:50:32.912 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000004caf18614418044a-006312f938-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.913 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj92 2022-09-03T06:50:32.913 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:32.914 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:32.914 DEBUG:boto:Method: GET 2022-09-03T06:50:32.915 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:50:32.916 DEBUG:boto:Data: 2022-09-03T06:50:32.916 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.917 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:32.917 DEBUG:boto:Port: 8001 2022-09-03T06:50:32.918 DEBUG:boto:Params: {} 2022-09-03T06:50:32.919 DEBUG:boto:Token: None 2022-09-03T06:50:32.919 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj92 2022-09-03T06:50:32.920 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yMHpofcMRZhG62HvGRFrgig303c= 2022-09-03T06:50:32.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yMHpofcMRZhG62HvGRFrgig303c=', 'Content-Length': '0'} 2022-09-03T06:50:32.924 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000d0b22bfd000694b0-006312f938-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.966 DEBUG:boto:path=/dsgvcy-75/obj92 2022-09-03T06:50:32.967 DEBUG:boto:auth_path=/dsgvcy-75/obj92 2022-09-03T06:50:32.968 DEBUG:boto:Method: GET 2022-09-03T06:50:32.969 DEBUG:boto:Path: /dsgvcy-75/obj92 2022-09-03T06:50:32.970 DEBUG:boto:Data: 2022-09-03T06:50:32.971 DEBUG:boto:Headers: {} 2022-09-03T06:50:32.972 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:32.973 DEBUG:boto:Port: 8000 2022-09-03T06:50:32.974 DEBUG:boto:Params: {} 2022-09-03T06:50:32.975 DEBUG:boto:Token: None 2022-09-03T06:50:32.976 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:32 GMT /dsgvcy-75/obj92 2022-09-03T06:50:32.977 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:yMHpofcMRZhG62HvGRFrgig303c= 2022-09-03T06:50:32.978 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:32 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:yMHpofcMRZhG62HvGRFrgig303c=', 'Content-Length': '0'} 2022-09-03T06:50:32.992 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:41:59 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000001c0043dc3adfc5-006312f938-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:32 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:32.993 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:32.994 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:32.994 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:32.995 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:32.997 DEBUG:boto:Method: GET 2022-09-03T06:50:32.998 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:50:32.998 DEBUG:boto:Data: 2022-09-03T06:50:33.001 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.002 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:33.004 DEBUG:boto:Port: 8001 2022-09-03T06:50:33.004 DEBUG:boto:Params: {} 2022-09-03T06:50:33.005 DEBUG:boto:Token: None 2022-09-03T06:50:33.007 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/ 2022-09-03T06:50:33.008 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GI/yiBhUwC9fv/okY+G37bGiGrY= 2022-09-03T06:50:33.010 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GI/yiBhUwC9fv/okY+G37bGiGrY=', 'Content-Length': '0'} 2022-09-03T06:50:33.016 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000ea60308ff8ab7c2b-006312f939-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.018 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:33.022 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:33.023 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:33.023 DEBUG:boto:path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:33.024 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj92 2022-09-03T06:50:33.025 DEBUG:boto:Method: GET 2022-09-03T06:50:33.025 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj92 2022-09-03T06:50:33.026 DEBUG:boto:Data: 2022-09-03T06:50:33.026 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.027 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:33.027 DEBUG:boto:Port: 8000 2022-09-03T06:50:33.028 DEBUG:boto:Params: {} 2022-09-03T06:50:33.029 DEBUG:boto:Token: None 2022-09-03T06:50:33.030 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/ 2022-09-03T06:50:33.031 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GI/yiBhUwC9fv/okY+G37bGiGrY= 2022-09-03T06:50:33.031 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GI/yiBhUwC9fv/okY+G37bGiGrY=', 'Content-Length': '0'} 2022-09-03T06:50:33.034 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000214d6111b90aa903-006312f939-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.035 DEBUG:boto:b'dsgvcy-75obj921000falseobj922022-09-03T06:41:59.891Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:33.036 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2022-09-03T06:50:33.037 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:33.037 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:33.038 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.039 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.039 DEBUG:boto:Method: GET 2022-09-03T06:50:33.040 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.040 DEBUG:boto:Data: 2022-09-03T06:50:33.041 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.042 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:33.042 DEBUG:boto:Port: 8001 2022-09-03T06:50:33.043 DEBUG:boto:Params: {} 2022-09-03T06:50:33.044 DEBUG:boto:Token: None 2022-09-03T06:50:33.045 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.045 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D7GWCd0JB/mwKIgDe9gl9zpZuOE= 2022-09-03T06:50:33.046 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D7GWCd0JB/mwKIgDe9gl9zpZuOE=', 'Content-Length': '0'} 2022-09-03T06:50:33.049 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000084e7cf758a2d6c55-006312f939-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.049 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:33.050 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:33.051 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.051 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.052 DEBUG:boto:Method: GET 2022-09-03T06:50:33.053 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.053 DEBUG:boto:Data: 2022-09-03T06:50:33.054 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.054 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:33.055 DEBUG:boto:Port: 8000 2022-09-03T06:50:33.055 DEBUG:boto:Params: {} 2022-09-03T06:50:33.056 DEBUG:boto:Token: None 2022-09-03T06:50:33.057 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.057 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D7GWCd0JB/mwKIgDe9gl9zpZuOE= 2022-09-03T06:50:33.058 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D7GWCd0JB/mwKIgDe9gl9zpZuOE=', 'Content-Length': '0'} 2022-09-03T06:50:33.060 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000171bbf57e0efb0ca-006312f939-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.106 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:33.107 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:33.107 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.108 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.109 DEBUG:boto:Method: HEAD 2022-09-03T06:50:33.109 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.110 DEBUG:boto:Data: 2022-09-03T06:50:33.110 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.111 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:33.112 DEBUG:boto:Port: 8001 2022-09-03T06:50:33.112 DEBUG:boto:Params: {} 2022-09-03T06:50:33.113 DEBUG:boto:Token: None 2022-09-03T06:50:33.114 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.115 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DcKvR0b8PX5lEcaKs0NdBmwvqhk= 2022-09-03T06:50:33.116 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DcKvR0b8PX5lEcaKs0NdBmwvqhk=', 'Content-Length': '0'} 2022-09-03T06:50:33.118 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fc48def2372a5e83-006312f939-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.119 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:33.120 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:33.121 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.126 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.132 DEBUG:boto:Method: HEAD 2022-09-03T06:50:33.133 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.134 DEBUG:boto:Data: 2022-09-03T06:50:33.137 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.139 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:33.143 DEBUG:boto:Port: 8000 2022-09-03T06:50:33.144 DEBUG:boto:Params: {} 2022-09-03T06:50:33.145 DEBUG:boto:Token: None 2022-09-03T06:50:33.146 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.147 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:DcKvR0b8PX5lEcaKs0NdBmwvqhk= 2022-09-03T06:50:33.148 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:DcKvR0b8PX5lEcaKs0NdBmwvqhk=', 'Content-Length': '0'} 2022-09-03T06:50:33.152 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000fb6bb6d1cbb422bb-006312f939-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.153 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2022-09-03T06:50:33.153 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:33.154 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:33.154 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.155 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.156 DEBUG:boto:Method: GET 2022-09-03T06:50:33.156 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.157 DEBUG:boto:Data: 2022-09-03T06:50:33.157 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.158 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:33.159 DEBUG:boto:Port: 8001 2022-09-03T06:50:33.160 DEBUG:boto:Params: {} 2022-09-03T06:50:33.161 DEBUG:boto:Token: None 2022-09-03T06:50:33.162 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.163 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D7GWCd0JB/mwKIgDe9gl9zpZuOE= 2022-09-03T06:50:33.163 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D7GWCd0JB/mwKIgDe9gl9zpZuOE=', 'Content-Length': '0'} 2022-09-03T06:50:33.166 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000ae3e854b5c4aea3e-006312f939-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.167 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:33.168 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:33.169 DEBUG:boto:path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.169 DEBUG:boto:auth_path=/dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.170 DEBUG:boto:Method: GET 2022-09-03T06:50:33.170 DEBUG:boto:Path: /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.171 DEBUG:boto:Data: 2022-09-03T06:50:33.172 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.172 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:33.173 DEBUG:boto:Port: 8000 2022-09-03T06:50:33.173 DEBUG:boto:Params: {} 2022-09-03T06:50:33.174 DEBUG:boto:Token: None 2022-09-03T06:50:33.175 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/obj93?versionId=null 2022-09-03T06:50:33.175 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:D7GWCd0JB/mwKIgDe9gl9zpZuOE= 2022-09-03T06:50:33.176 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:D7GWCd0JB/mwKIgDe9gl9zpZuOE=', 'Content-Length': '0'} 2022-09-03T06:50:33.179 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000022553eb50bbb65d5-006312f939-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.180 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:33.180 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:33.181 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:33.182 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:33.182 DEBUG:boto:Method: GET 2022-09-03T06:50:33.183 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:50:33.184 DEBUG:boto:Data: 2022-09-03T06:50:33.187 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.188 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:33.189 DEBUG:boto:Port: 8001 2022-09-03T06:50:33.190 DEBUG:boto:Params: {} 2022-09-03T06:50:33.191 DEBUG:boto:Token: None 2022-09-03T06:50:33.192 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/ 2022-09-03T06:50:33.193 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GI/yiBhUwC9fv/okY+G37bGiGrY= 2022-09-03T06:50:33.194 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GI/yiBhUwC9fv/okY+G37bGiGrY=', 'Content-Length': '0'} 2022-09-03T06:50:33.197 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000004817e19c8f5b9d48-006312f939-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.199 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:33.201 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:33.202 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:33.203 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:33.203 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:33.204 DEBUG:boto:Method: GET 2022-09-03T06:50:33.205 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:50:33.206 DEBUG:boto:Data: 2022-09-03T06:50:33.207 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.208 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:33.209 DEBUG:boto:Port: 8000 2022-09-03T06:50:33.210 DEBUG:boto:Params: {} 2022-09-03T06:50:33.211 DEBUG:boto:Token: None 2022-09-03T06:50:33.212 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/ 2022-09-03T06:50:33.214 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GI/yiBhUwC9fv/okY+G37bGiGrY= 2022-09-03T06:50:33.215 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GI/yiBhUwC9fv/okY+G37bGiGrY=', 'Content-Length': '0'} 2022-09-03T06:50:33.218 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000071ee6d85f0f8b63-006312f939-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.219 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:33.221 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:33.221 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:33.222 DEBUG:boto:Method: HEAD 2022-09-03T06:50:33.223 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:50:33.224 DEBUG:boto:Data: 2022-09-03T06:50:33.225 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.226 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:33.227 DEBUG:boto:Port: 8001 2022-09-03T06:50:33.228 DEBUG:boto:Params: {} 2022-09-03T06:50:33.229 DEBUG:boto:Token: None 2022-09-03T06:50:33.230 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/obj93 2022-09-03T06:50:33.231 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pSjWebCrnI8DJF0E7LkvP9D6Ngs= 2022-09-03T06:50:33.232 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pSjWebCrnI8DJF0E7LkvP9D6Ngs=', 'Content-Length': '0'} 2022-09-03T06:50:33.234 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001a0f989203952c95-006312f939-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.235 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:33.235 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:33.236 DEBUG:boto:Method: HEAD 2022-09-03T06:50:33.237 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:50:33.237 DEBUG:boto:Data: 2022-09-03T06:50:33.238 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.239 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:33.239 DEBUG:boto:Port: 8000 2022-09-03T06:50:33.240 DEBUG:boto:Params: {} 2022-09-03T06:50:33.241 DEBUG:boto:Token: None 2022-09-03T06:50:33.241 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/obj93 2022-09-03T06:50:33.242 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:pSjWebCrnI8DJF0E7LkvP9D6Ngs= 2022-09-03T06:50:33.243 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:pSjWebCrnI8DJF0E7LkvP9D6Ngs=', 'Content-Length': '0'} 2022-09-03T06:50:33.247 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b654032731e8e821-006312f939-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.249 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj93 2022-09-03T06:50:33.250 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:33.251 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:33.252 DEBUG:boto:Method: GET 2022-09-03T06:50:33.253 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:50:33.253 DEBUG:boto:Data: 2022-09-03T06:50:33.254 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.255 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:33.255 DEBUG:boto:Port: 8001 2022-09-03T06:50:33.256 DEBUG:boto:Params: {} 2022-09-03T06:50:33.256 DEBUG:boto:Token: None 2022-09-03T06:50:33.257 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/obj93 2022-09-03T06:50:33.258 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LnPovgoCKa6cZrBY37Azo3+N+UA= 2022-09-03T06:50:33.259 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LnPovgoCKa6cZrBY37Azo3+N+UA=', 'Content-Length': '0'} 2022-09-03T06:50:33.261 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000000aeaba9ac5574d01-006312f939-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.262 DEBUG:boto:path=/dsgvcy-75/obj93 2022-09-03T06:50:33.263 DEBUG:boto:auth_path=/dsgvcy-75/obj93 2022-09-03T06:50:33.264 DEBUG:boto:Method: GET 2022-09-03T06:50:33.265 DEBUG:boto:Path: /dsgvcy-75/obj93 2022-09-03T06:50:33.266 DEBUG:boto:Data: 2022-09-03T06:50:33.267 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.268 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:33.269 DEBUG:boto:Port: 8000 2022-09-03T06:50:33.270 DEBUG:boto:Params: {} 2022-09-03T06:50:33.270 DEBUG:boto:Token: None 2022-09-03T06:50:33.272 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/obj93 2022-09-03T06:50:33.272 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:LnPovgoCKa6cZrBY37Azo3+N+UA= 2022-09-03T06:50:33.273 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:LnPovgoCKa6cZrBY37Azo3+N+UA=', 'Content-Length': '0'} 2022-09-03T06:50:33.276 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:45:46 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a5e5c38a077cdb3f-006312f939-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.277 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:33.278 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:33.278 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:33.279 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:33.280 DEBUG:boto:Method: GET 2022-09-03T06:50:33.281 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:50:33.283 DEBUG:boto:Data: 2022-09-03T06:50:33.285 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.285 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:33.286 DEBUG:boto:Port: 8001 2022-09-03T06:50:33.287 DEBUG:boto:Params: {} 2022-09-03T06:50:33.288 DEBUG:boto:Token: None 2022-09-03T06:50:33.289 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/ 2022-09-03T06:50:33.290 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GI/yiBhUwC9fv/okY+G37bGiGrY= 2022-09-03T06:50:33.291 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GI/yiBhUwC9fv/okY+G37bGiGrY=', 'Content-Length': '0'} 2022-09-03T06:50:33.295 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx0000030f92c182997bc77-006312f939-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.296 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:33.297 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:33.298 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:33.299 DEBUG:boto:path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:33.300 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj93 2022-09-03T06:50:33.302 DEBUG:boto:Method: GET 2022-09-03T06:50:33.302 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj93 2022-09-03T06:50:33.303 DEBUG:boto:Data: 2022-09-03T06:50:33.303 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.304 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:33.304 DEBUG:boto:Port: 8000 2022-09-03T06:50:33.305 DEBUG:boto:Params: {} 2022-09-03T06:50:33.305 DEBUG:boto:Token: None 2022-09-03T06:50:33.306 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/ 2022-09-03T06:50:33.307 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GI/yiBhUwC9fv/okY+G37bGiGrY= 2022-09-03T06:50:33.307 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GI/yiBhUwC9fv/okY+G37bGiGrY=', 'Content-Length': '0'} 2022-09-03T06:50:33.312 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000c755e758943a8609-006312f939-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.313 DEBUG:boto:b'dsgvcy-75obj931000falseobj932022-09-03T06:45:46.926Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:33.314 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:50:33.315 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:33.316 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:33.316 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.317 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.317 DEBUG:boto:Method: GET 2022-09-03T06:50:33.318 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.318 DEBUG:boto:Data: 2022-09-03T06:50:33.319 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.320 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:33.321 DEBUG:boto:Port: 8001 2022-09-03T06:50:33.321 DEBUG:boto:Params: {} 2022-09-03T06:50:33.322 DEBUG:boto:Token: None 2022-09-03T06:50:33.323 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.323 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:g9eqabeuOBt2PMaHu12PXHBQIlM= 2022-09-03T06:50:33.324 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:g9eqabeuOBt2PMaHu12PXHBQIlM=', 'Content-Length': '0'} 2022-09-03T06:50:33.327 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000020a3514201cae7da-006312f939-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.328 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:33.329 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:33.330 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.331 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.332 DEBUG:boto:Method: GET 2022-09-03T06:50:33.333 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.333 DEBUG:boto:Data: 2022-09-03T06:50:33.334 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.335 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:33.336 DEBUG:boto:Port: 8000 2022-09-03T06:50:33.337 DEBUG:boto:Params: {} 2022-09-03T06:50:33.338 DEBUG:boto:Token: None 2022-09-03T06:50:33.340 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.341 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:g9eqabeuOBt2PMaHu12PXHBQIlM= 2022-09-03T06:50:33.342 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:g9eqabeuOBt2PMaHu12PXHBQIlM=', 'Content-Length': '0'} 2022-09-03T06:50:33.361 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000a54d82d4f25d8751-006312f939-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.402 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:33.403 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:33.404 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.405 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.405 DEBUG:boto:Method: HEAD 2022-09-03T06:50:33.406 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.407 DEBUG:boto:Data: 2022-09-03T06:50:33.408 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.409 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:33.409 DEBUG:boto:Port: 8001 2022-09-03T06:50:33.411 DEBUG:boto:Params: {} 2022-09-03T06:50:33.414 DEBUG:boto:Token: None 2022-09-03T06:50:33.416 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.417 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I5VN54+xHO4RckmshT1SfrVo97o= 2022-09-03T06:50:33.418 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I5VN54+xHO4RckmshT1SfrVo97o=', 'Content-Length': '0'} 2022-09-03T06:50:33.424 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000006b5ce35991323abd-006312f939-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.426 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:33.427 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:33.428 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.429 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.429 DEBUG:boto:Method: HEAD 2022-09-03T06:50:33.430 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.431 DEBUG:boto:Data: 2022-09-03T06:50:33.431 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.432 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:33.432 DEBUG:boto:Port: 8000 2022-09-03T06:50:33.433 DEBUG:boto:Params: {} 2022-09-03T06:50:33.435 DEBUG:boto:Token: None 2022-09-03T06:50:33.435 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.436 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:I5VN54+xHO4RckmshT1SfrVo97o= 2022-09-03T06:50:33.437 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:I5VN54+xHO4RckmshT1SfrVo97o=', 'Content-Length': '0'} 2022-09-03T06:50:33.447 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b9585920c92871d8-006312f939-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.448 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:50:33.449 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:33.450 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:33.451 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.453 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.454 DEBUG:boto:Method: GET 2022-09-03T06:50:33.456 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.458 DEBUG:boto:Data: 2022-09-03T06:50:33.459 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.460 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:33.462 DEBUG:boto:Port: 8001 2022-09-03T06:50:33.463 DEBUG:boto:Params: {} 2022-09-03T06:50:33.465 DEBUG:boto:Token: None 2022-09-03T06:50:33.466 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.467 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:g9eqabeuOBt2PMaHu12PXHBQIlM= 2022-09-03T06:50:33.469 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:g9eqabeuOBt2PMaHu12PXHBQIlM=', 'Content-Length': '0'} 2022-09-03T06:50:33.472 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000001e0130bdc849b159-006312f939-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.473 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:33.474 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:33.475 DEBUG:boto:path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.479 DEBUG:boto:auth_path=/dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.480 DEBUG:boto:Method: GET 2022-09-03T06:50:33.481 DEBUG:boto:Path: /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.482 DEBUG:boto:Data: 2022-09-03T06:50:33.483 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.484 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:33.485 DEBUG:boto:Port: 8000 2022-09-03T06:50:33.486 DEBUG:boto:Params: {} 2022-09-03T06:50:33.487 DEBUG:boto:Token: None 2022-09-03T06:50:33.488 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/obj95?versionId=null 2022-09-03T06:50:33.490 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:g9eqabeuOBt2PMaHu12PXHBQIlM= 2022-09-03T06:50:33.491 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:g9eqabeuOBt2PMaHu12PXHBQIlM=', 'Content-Length': '0'} 2022-09-03T06:50:33.495 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-amz-version-id', 'null'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000008b1962273292ae7e-006312f939-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.502 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:33.503 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:33.504 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:33.505 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:33.506 DEBUG:boto:Method: GET 2022-09-03T06:50:33.507 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:50:33.508 DEBUG:boto:Data: 2022-09-03T06:50:33.509 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.511 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:33.512 DEBUG:boto:Port: 8001 2022-09-03T06:50:33.513 DEBUG:boto:Params: {} 2022-09-03T06:50:33.514 DEBUG:boto:Token: None 2022-09-03T06:50:33.515 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/ 2022-09-03T06:50:33.516 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GI/yiBhUwC9fv/okY+G37bGiGrY= 2022-09-03T06:50:33.517 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GI/yiBhUwC9fv/okY+G37bGiGrY=', 'Content-Length': '0'} 2022-09-03T06:50:33.526 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx00000473f59ee83565a1e-006312f939-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.532 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:33.534 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:33.535 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:33.536 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:33.537 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:33.538 DEBUG:boto:Method: GET 2022-09-03T06:50:33.539 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:50:33.540 DEBUG:boto:Data: 2022-09-03T06:50:33.541 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.542 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:33.543 DEBUG:boto:Port: 8000 2022-09-03T06:50:33.544 DEBUG:boto:Params: {} 2022-09-03T06:50:33.545 DEBUG:boto:Token: None 2022-09-03T06:50:33.546 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/ 2022-09-03T06:50:33.548 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GI/yiBhUwC9fv/okY+G37bGiGrY= 2022-09-03T06:50:33.549 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GI/yiBhUwC9fv/okY+G37bGiGrY=', 'Content-Length': '0'} 2022-09-03T06:50:33.552 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000003aba983575d41f80-006312f939-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.554 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:33.555 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:33.556 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:33.557 DEBUG:boto:Method: HEAD 2022-09-03T06:50:33.558 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:50:33.559 DEBUG:boto:Data: 2022-09-03T06:50:33.560 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.560 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:33.561 DEBUG:boto:Port: 8001 2022-09-03T06:50:33.562 DEBUG:boto:Params: {} 2022-09-03T06:50:33.564 DEBUG:boto:Token: None 2022-09-03T06:50:33.565 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/obj95 2022-09-03T06:50:33.566 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qplBbtuTtRS42cEbN75LROk6K5g= 2022-09-03T06:50:33.567 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qplBbtuTtRS42cEbN75LROk6K5g=', 'Content-Length': '0'} 2022-09-03T06:50:33.571 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx000005102e7eb6785e644-006312f939-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.573 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:33.574 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:33.575 DEBUG:boto:Method: HEAD 2022-09-03T06:50:33.576 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:50:33.577 DEBUG:boto:Data: 2022-09-03T06:50:33.578 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.579 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:33.580 DEBUG:boto:Port: 8000 2022-09-03T06:50:33.581 DEBUG:boto:Params: {} 2022-09-03T06:50:33.583 DEBUG:boto:Token: None 2022-09-03T06:50:33.584 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/obj95 2022-09-03T06:50:33.585 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:qplBbtuTtRS42cEbN75LROk6K5g= 2022-09-03T06:50:33.586 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:qplBbtuTtRS42cEbN75LROk6K5g=', 'Content-Length': '0'} 2022-09-03T06:50:33.588 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx00000b4eab5b5e6d76f5f-006312f939-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.590 DEBUG:tasks.rgw_multi.zone_rados:comparing key name=obj95 2022-09-03T06:50:33.591 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:33.592 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:33.593 DEBUG:boto:Method: GET 2022-09-03T06:50:33.594 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:50:33.595 DEBUG:boto:Data: 2022-09-03T06:50:33.596 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.597 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:33.598 DEBUG:boto:Port: 8001 2022-09-03T06:50:33.599 DEBUG:boto:Params: {} 2022-09-03T06:50:33.600 DEBUG:boto:Token: None 2022-09-03T06:50:33.601 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/obj95 2022-09-03T06:50:33.602 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PbuJQ9nbOBgVBZHjufsGmxDAwLQ= 2022-09-03T06:50:33.603 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PbuJQ9nbOBgVBZHjufsGmxDAwLQ=', 'Content-Length': '0'} 2022-09-03T06:50:33.606 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000055ddc3734d1ff88f-006312f939-1851-test-zone3'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.607 DEBUG:boto:path=/dsgvcy-75/obj95 2022-09-03T06:50:33.608 DEBUG:boto:auth_path=/dsgvcy-75/obj95 2022-09-03T06:50:33.609 DEBUG:boto:Method: GET 2022-09-03T06:50:33.610 DEBUG:boto:Path: /dsgvcy-75/obj95 2022-09-03T06:50:33.611 DEBUG:boto:Data: 2022-09-03T06:50:33.612 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.613 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:33.614 DEBUG:boto:Port: 8000 2022-09-03T06:50:33.615 DEBUG:boto:Params: {} 2022-09-03T06:50:33.616 DEBUG:boto:Token: None 2022-09-03T06:50:33.617 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/obj95 2022-09-03T06:50:33.618 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:PbuJQ9nbOBgVBZHjufsGmxDAwLQ= 2022-09-03T06:50:33.619 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:PbuJQ9nbOBgVBZHjufsGmxDAwLQ=', 'Content-Length': '0'} 2022-09-03T06:50:33.622 DEBUG:boto:Response headers: [('Content-Length', '3'), ('Accept-Ranges', 'bytes'), ('Last-Modified', 'Sat, 03 Sep 2022 06:40:53 GMT'), ('x-rgw-object-type', 'Normal'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('x-amz-request-id', 'tx0000071a4e3fc3a55d94d-006312f939-188c-test-zone2'), ('Content-Type', 'application/octet-stream'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.623 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:33.624 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:33.625 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:33.625 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:33.626 DEBUG:boto:Method: GET 2022-09-03T06:50:33.627 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:50:33.627 DEBUG:boto:Data: 2022-09-03T06:50:33.628 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.628 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8001 2022-09-03T06:50:33.629 DEBUG:boto:Port: 8001 2022-09-03T06:50:33.629 DEBUG:boto:Params: {} 2022-09-03T06:50:33.630 DEBUG:boto:Token: None 2022-09-03T06:50:33.631 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/ 2022-09-03T06:50:33.631 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GI/yiBhUwC9fv/okY+G37bGiGrY= 2022-09-03T06:50:33.632 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GI/yiBhUwC9fv/okY+G37bGiGrY=', 'Content-Length': '0'} 2022-09-03T06:50:33.636 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005071aa056add4f33-006312f939-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.683 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:33.685 DEBUG:boto:path=/dsgvcy-75/ 2022-09-03T06:50:33.686 DEBUG:boto:auth_path=/dsgvcy-75/ 2022-09-03T06:50:33.687 DEBUG:boto:path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:33.688 DEBUG:boto:auth_path=/dsgvcy-75/?prefix=obj95 2022-09-03T06:50:33.689 DEBUG:boto:Method: GET 2022-09-03T06:50:33.690 DEBUG:boto:Path: /dsgvcy-75/?prefix=obj95 2022-09-03T06:50:33.691 DEBUG:boto:Data: 2022-09-03T06:50:33.692 DEBUG:boto:Headers: {} 2022-09-03T06:50:33.693 DEBUG:boto:Host: smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:33.694 DEBUG:boto:Port: 8000 2022-09-03T06:50:33.695 DEBUG:boto:Params: {} 2022-09-03T06:50:33.697 DEBUG:boto:Token: None 2022-09-03T06:50:33.698 DEBUG:boto:StringToSign: GET Sat, 03 Sep 2022 06:50:33 GMT /dsgvcy-75/ 2022-09-03T06:50:33.699 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GI/yiBhUwC9fv/okY+G37bGiGrY= 2022-09-03T06:50:33.700 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:50:33 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GI/yiBhUwC9fv/okY+G37bGiGrY=', 'Content-Length': '0'} 2022-09-03T06:50:33.706 DEBUG:boto:Response headers: [('Transfer-Encoding', 'chunked'), ('x-amz-request-id', 'tx000005ca4975e010a40cb-006312f939-188c-test-zone2'), ('Content-Type', 'application/xml'), ('Date', 'Sat, 03 Sep 2022 06:50:33 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:33.713 DEBUG:boto:b'dsgvcy-75obj951000falseobj952022-09-03T06:40:53.415Z"acbd18db4cc2f85cedef654fccc4a4d8"3STANDARDrgw-multisite-test-userMultisite Test UserNormal' 2022-09-03T06:50:33.715 INFO:tasks.rgw_multi.zone_rados:success, bucket identical: bucket=dsgvcy-75 zones={test-zone3, test-zone2} 2022-09-03T06:50:33.717 DEBUG:tasks.rgw.c2.client.0:waiting for process to exit 2022-09-03T06:50:33.721 INFO:teuthology.orchestra.run:waiting for 300 2022-09-03T06:50:33.723 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:50:33.716+0000 7fae00ef1700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term radosgw --rgw-frontends beast port=8000 -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm (PID: 146916) UID: 0 2022-09-03T06:50:33.724 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:50:33.716+0000 7fae0515c600 -1 shutting down 2022-09-03T06:50:35.242 INFO:tasks.rgw.c1.client.0.smithi181.stdout:2022-09-03T06:50:35.241+0000 7f7eab425700 -1 ERROR: failed operation: ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-5)ERROR: collect() returned error (ret=-5)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-5)ERROR: collect() returned error (ret=-5)ERROR: collect() returned error (ret=-16) 2022-09-03T06:50:35.317 INFO:tasks.rgw.c1.client.0.smithi181.stdout:2022-09-03T06:50:35.315+0000 7f7eab425700 -1 ERROR: failed operation: ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-5)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-5)ERROR: collect() returned error (ret=-16) 2022-09-03T06:50:35.337 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~0s 2022-09-03T06:50:39.815 INFO:tasks.rgw.c2.client.0:Stopped 2022-09-03T06:50:39.817 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'list', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--max-entries', '10000', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:50:39.819 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'list', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--max-entries', '10000', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:50:39.819 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket list --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --max-entries 10000 --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:50:39.880 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:39.881 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:39.912 INFO:teuthology.orchestra.run.smithi189.stdout:[ 2022-09-03T06:50:39.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:39.913 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj11", 2022-09-03T06:50:39.913 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:39.913 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:39.914 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:39.914 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 4 2022-09-03T06:50:39.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.915 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:39.915 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:39.916 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:39.916 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:39.916 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 6, 2022-09-03T06:50:39.916 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:36:05.688419Z", 2022-09-03T06:50:39.917 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:39.917 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:39.917 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:39.917 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:39.918 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:39.918 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:39.918 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:39.919 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:39.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.919 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_hBdj1dVZceofeDZTrznX9LJdZK-Xpoe", 2022-09-03T06:50:39.920 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:39.920 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:39.920 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:39.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.921 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:39.921 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj111", 2022-09-03T06:50:39.922 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:39.924 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:39.924 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:39.925 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 4 2022-09-03T06:50:39.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.925 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:39.925 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:39.926 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:39.926 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:39.926 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 5, 2022-09-03T06:50:39.927 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:47:49.634221Z", 2022-09-03T06:50:39.927 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:39.927 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:39.928 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:39.928 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:39.928 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:39.929 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:39.929 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:39.930 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:39.930 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.930 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_uqzpldzHMuhBpki9mhyKBBfPBqP-oi3", 2022-09-03T06:50:39.931 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:39.931 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:39.931 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:39.932 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.932 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:39.932 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj115", 2022-09-03T06:50:39.934 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:39.934 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:39.935 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:39.935 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 9 2022-09-03T06:50:39.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.936 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:39.938 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:39.939 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:39.939 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:39.939 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 16, 2022-09-03T06:50:39.940 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:41:59.934437Z", 2022-09-03T06:50:39.940 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:39.940 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:39.940 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:39.941 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:39.941 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:39.941 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:39.941 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:39.942 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:39.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.942 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_gSVPN1IYJa_REgdOpZ6Z2GyzFkCS0rL", 2022-09-03T06:50:39.942 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:39.943 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:39.943 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:39.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.944 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:39.944 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj116", 2022-09-03T06:50:39.944 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:39.945 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:39.945 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:39.945 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 1 2022-09-03T06:50:39.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.946 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:39.946 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:39.946 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:39.947 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:39.947 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 16, 2022-09-03T06:50:39.947 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:44:18.363978Z", 2022-09-03T06:50:39.947 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:39.948 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:39.948 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:39.948 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:39.948 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:39.949 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:39.949 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:39.949 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:39.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.950 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_WuKckUWDCm6A-nmVdeZAM9WHZEPsgev", 2022-09-03T06:50:39.950 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:39.950 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:39.950 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:39.951 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.951 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:39.951 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj124", 2022-09-03T06:50:39.951 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:39.952 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:39.952 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:39.952 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 6 2022-09-03T06:50:39.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.953 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:39.953 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:39.953 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:39.954 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:39.954 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 6, 2022-09-03T06:50:39.954 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:45:46.963849Z", 2022-09-03T06:50:39.954 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:39.955 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:39.955 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:39.955 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:39.955 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:39.956 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:39.956 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:39.956 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:39.957 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.957 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_yOJZdmL60qiZQB4-IWuo00DgV6xeypQ", 2022-09-03T06:50:39.957 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:39.957 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:39.958 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:39.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.958 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:39.958 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj145", 2022-09-03T06:50:39.959 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:39.959 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:39.959 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:39.959 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 1 2022-09-03T06:50:39.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.960 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:39.960 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:39.961 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:39.961 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:39.961 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 16, 2022-09-03T06:50:39.961 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:44:18.391303Z", 2022-09-03T06:50:39.962 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:39.962 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:39.963 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:39.963 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:39.963 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:39.964 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:39.964 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:39.964 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:39.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.965 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_L-p6ksvEJv_sSYmc0nMsyT6rQYaV65g", 2022-09-03T06:50:39.965 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:39.966 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:39.966 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:39.966 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.967 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:39.967 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj148", 2022-09-03T06:50:39.967 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:39.968 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:39.968 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:39.968 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 11 2022-09-03T06:50:39.969 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.969 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:39.969 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:39.970 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:39.970 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:39.970 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 5, 2022-09-03T06:50:39.971 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:47:49.702472Z", 2022-09-03T06:50:39.971 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:39.971 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:39.971 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:39.972 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:39.972 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:39.972 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:39.972 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:39.973 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:39.973 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.973 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Y6hwl5W9SryzFJp4k03vTlDQMg-n88o", 2022-09-03T06:50:39.974 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:39.974 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:39.974 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:39.975 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.975 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:39.975 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj155", 2022-09-03T06:50:39.975 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:39.976 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:39.976 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:39.977 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 1 2022-09-03T06:50:39.977 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.977 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:39.978 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:39.978 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:39.979 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:39.979 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 16, 2022-09-03T06:50:39.980 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:45:47.037786Z", 2022-09-03T06:50:39.980 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:39.981 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:39.981 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:39.982 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:39.982 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:39.983 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:39.983 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:39.984 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:39.984 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.985 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_pLpmzC3RfTwVItpqySPWzenR8-LFa-M", 2022-09-03T06:50:39.985 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:39.986 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:39.986 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:39.987 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.987 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:39.988 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj17", 2022-09-03T06:50:39.988 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:39.988 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:39.989 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:39.989 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 13 2022-09-03T06:50:39.990 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.990 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:39.991 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:39.991 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:39.992 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:39.992 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 16, 2022-09-03T06:50:39.992 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:37:27.042294Z", 2022-09-03T06:50:39.993 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:39.993 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:39.993 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:39.994 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:39.994 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:39.994 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:39.994 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:39.995 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:39.995 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.996 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_OHA3IRfh0xDiIhCzu9l3BVVMcJg39gF", 2022-09-03T06:50:39.997 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:39.997 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:39.997 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:39.998 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:39.998 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:39.998 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj185", 2022-09-03T06:50:39.999 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:39.999 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:39.999 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.000 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 3 2022-09-03T06:50:40.000 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.000 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.001 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.001 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.002 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.002 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 16, 2022-09-03T06:50:40.004 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:47:49.731694Z", 2022-09-03T06:50:40.006 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.006 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.006 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.007 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.007 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.007 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.008 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.008 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.009 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.009 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_sW0K2p1vQErK_mFQSvg8leRJUhIsmek", 2022-09-03T06:50:40.009 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.010 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.010 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.012 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj19", 2022-09-03T06:50:40.012 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.013 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.014 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.015 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 1 2022-09-03T06:50:40.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.015 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.015 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.017 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.017 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.017 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 6, 2022-09-03T06:50:40.018 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:40:53.130958Z", 2022-09-03T06:50:40.018 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.018 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.019 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.019 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.019 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.019 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.020 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.020 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.020 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.021 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_xYWGwr2vu_dlR8MexPH412yv7GUi4ac", 2022-09-03T06:50:40.021 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.021 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.021 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.022 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.022 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.022 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj22", 2022-09-03T06:50:40.023 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.023 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.024 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.024 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 10 2022-09-03T06:50:40.024 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.025 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.025 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.025 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.025 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.026 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 5, 2022-09-03T06:50:40.026 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:36:05.723857Z", 2022-09-03T06:50:40.026 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.027 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.028 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.028 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.028 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.028 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.029 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.029 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.029 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.030 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_DuplGLHbbukNcJ5okBAe9HYMaudCm4x", 2022-09-03T06:50:40.030 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.030 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.030 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.031 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.031 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.031 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj23", 2022-09-03T06:50:40.032 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.032 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.032 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.032 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 3 2022-09-03T06:50:40.033 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.033 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.033 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.034 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.034 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.034 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 16, 2022-09-03T06:50:40.035 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:41:59.633592Z", 2022-09-03T06:50:40.035 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.035 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.036 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.036 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.036 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.036 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.037 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.037 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.038 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.038 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_vs5YbpFPNEbkk080QnIGHzWCODXKc6A", 2022-09-03T06:50:40.038 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.039 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.039 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.039 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.039 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.040 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj29", 2022-09-03T06:50:40.040 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.040 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.040 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.041 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 7 2022-09-03T06:50:40.041 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.041 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.041 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.042 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.042 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.042 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 16, 2022-09-03T06:50:40.043 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:44:18.231449Z", 2022-09-03T06:50:40.043 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.043 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.043 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.044 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.044 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.044 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.044 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.045 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.045 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.045 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_Arrvl-inwiRiYuzLihoBYE2EJ-t_8E_", 2022-09-03T06:50:40.046 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.046 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.046 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.046 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.047 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.047 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj31", 2022-09-03T06:50:40.047 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.047 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.048 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.048 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 2 2022-09-03T06:50:40.049 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.049 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.049 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.049 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.050 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.050 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 5, 2022-09-03T06:50:40.050 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:45:46.812991Z", 2022-09-03T06:50:40.051 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.051 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.051 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.052 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.052 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.052 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.053 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.053 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.053 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.053 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_CDiCImA0LNZ4pLoRTx4dqNq0aTg5Kjn", 2022-09-03T06:50:40.054 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.054 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.054 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.054 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.055 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.055 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj33", 2022-09-03T06:50:40.055 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.056 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.056 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.056 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 5 2022-09-03T06:50:40.057 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.057 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.058 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.058 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.059 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.059 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 16, 2022-09-03T06:50:40.060 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:36:05.796207Z", 2022-09-03T06:50:40.060 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.060 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.060 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.061 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.061 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.061 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.061 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.062 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.062 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.062 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_dagkVIjUq-TQI5NGk-NXUkqZlHo--LT", 2022-09-03T06:50:40.063 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.063 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.064 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.064 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.064 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.064 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj34", 2022-09-03T06:50:40.065 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.065 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.065 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.065 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 2 2022-09-03T06:50:40.066 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.066 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.066 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.066 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.067 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.067 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 3, 2022-09-03T06:50:40.068 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:37:27.080034Z", 2022-09-03T06:50:40.068 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.068 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.068 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.069 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.070 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.070 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.070 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.070 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.071 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.071 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_PcGp3nDHtJFAKoV7HlqPer5L22e1jV4", 2022-09-03T06:50:40.071 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.072 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.072 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.072 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.072 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.073 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj37", 2022-09-03T06:50:40.073 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.073 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.073 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.074 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 7 2022-09-03T06:50:40.074 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.074 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.075 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.075 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.075 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.075 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 16, 2022-09-03T06:50:40.076 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:47:49.518311Z", 2022-09-03T06:50:40.076 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.076 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.076 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.077 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.077 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.077 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.078 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.078 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.078 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.078 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_4WvpdO1YHt8oOmib64yMlZRk1Np9JvL", 2022-09-03T06:50:40.079 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.079 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.080 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.080 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.081 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.081 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj38", 2022-09-03T06:50:40.081 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.081 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.082 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.082 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 9 2022-09-03T06:50:40.082 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.082 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.083 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.083 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.084 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.084 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 6, 2022-09-03T06:50:40.084 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:40:53.174242Z", 2022-09-03T06:50:40.084 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.085 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.085 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.085 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.085 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.086 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.086 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.086 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.086 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.087 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_bNDYoz0_t2lVQQ6pJePOtHOXsh-O3yG", 2022-09-03T06:50:40.087 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.087 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.088 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.089 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.089 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj44", 2022-09-03T06:50:40.089 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.089 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.090 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.090 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 8 2022-09-03T06:50:40.091 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.091 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.091 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.091 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.092 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.092 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 16, 2022-09-03T06:50:40.092 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:36:05.834183Z", 2022-09-03T06:50:40.093 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.093 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.093 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.093 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.094 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.094 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.094 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.095 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.095 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.095 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_ufRyA14Kbara9fD_aUU0jbVwVBlKSms", 2022-09-03T06:50:40.095 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.096 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.096 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.097 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.097 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.097 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj46", 2022-09-03T06:50:40.098 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.098 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.098 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.098 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 8 2022-09-03T06:50:40.099 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.099 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.099 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.099 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.100 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.100 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 16, 2022-09-03T06:50:40.100 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:41:59.675906Z", 2022-09-03T06:50:40.102 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.102 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.103 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.103 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.104 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.104 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.104 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.104 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.105 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.105 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_HPm32g9ecitx8UW8lqShpGpDeMxiYyg", 2022-09-03T06:50:40.105 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.106 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.106 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.106 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.107 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.107 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj51", 2022-09-03T06:50:40.107 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.108 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.108 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.108 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 5 2022-09-03T06:50:40.108 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.109 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.109 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.109 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.109 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.110 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 6, 2022-09-03T06:50:40.110 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:37:27.160417Z", 2022-09-03T06:50:40.111 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.111 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.111 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.112 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.113 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.113 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.113 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.114 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.114 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.114 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_zqO5R3OJkJf6vmB6-ZzXsOYM6d7Mbya", 2022-09-03T06:50:40.114 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.115 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.115 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.116 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj55", 2022-09-03T06:50:40.116 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.116 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.117 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.117 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 4 2022-09-03T06:50:40.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.117 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.118 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.118 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.118 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.119 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 5, 2022-09-03T06:50:40.119 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:36:05.866918Z", 2022-09-03T06:50:40.119 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.120 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.120 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.120 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.120 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.121 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.121 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.121 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.121 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.122 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_CetXbRj-LPLwhRZIOIl-x2yrL1zucSW", 2022-09-03T06:50:40.122 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.123 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.123 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.124 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj57", 2022-09-03T06:50:40.125 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.125 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.125 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.126 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 5 2022-09-03T06:50:40.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.126 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.126 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.127 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.127 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.127 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 16, 2022-09-03T06:50:40.128 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:40:53.294309Z", 2022-09-03T06:50:40.128 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.128 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.128 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.129 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.129 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.129 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.129 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.130 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.130 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_lTfYdwtR3DEmu-sfFbpO3bXebM1PxF8", 2022-09-03T06:50:40.131 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.131 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.131 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.132 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj58", 2022-09-03T06:50:40.132 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.133 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.133 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.133 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 3 2022-09-03T06:50:40.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.134 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.134 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.134 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.135 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.135 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 3, 2022-09-03T06:50:40.135 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:44:18.271014Z", 2022-09-03T06:50:40.136 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.136 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.136 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.137 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.137 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.137 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.138 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.138 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.139 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_qIAO2vufXhPQ8qut41BIBx2f6JYf6my", 2022-09-03T06:50:40.139 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.139 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.139 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.140 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.140 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.140 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj62", 2022-09-03T06:50:40.140 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.141 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.141 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.141 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 14 2022-09-03T06:50:40.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.142 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.142 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.142 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.143 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.143 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 16, 2022-09-03T06:50:40.143 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:45:46.850126Z", 2022-09-03T06:50:40.144 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.144 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.144 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.144 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.145 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.145 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.145 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.145 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.146 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_NnnJZPXbYMDEPhtzUxxt6yvcFf6oOxG", 2022-09-03T06:50:40.146 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.147 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.147 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.148 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj68", 2022-09-03T06:50:40.148 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.148 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.148 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.149 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 16 2022-09-03T06:50:40.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.149 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.149 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.150 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.150 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.150 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 16, 2022-09-03T06:50:40.150 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:37:27.190144Z", 2022-09-03T06:50:40.151 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.151 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.151 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.152 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.152 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.152 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.153 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.153 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.154 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_QzM2ENCkNvErl9xjuTtLwan-9CB6__V", 2022-09-03T06:50:40.154 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.154 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.155 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.155 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.156 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj69", 2022-09-03T06:50:40.156 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.156 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.157 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.157 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 3 2022-09-03T06:50:40.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.158 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.159 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.159 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.160 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.160 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 6, 2022-09-03T06:50:40.161 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:41:59.821528Z", 2022-09-03T06:50:40.161 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.162 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.162 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.162 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.163 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.164 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.165 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.166 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.167 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_zzUYq2TI7-C2NkLrIUypvkwhMUfRYtR", 2022-09-03T06:50:40.167 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.168 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.169 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.170 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj74", 2022-09-03T06:50:40.171 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.171 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.172 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.172 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 8 2022-09-03T06:50:40.173 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.173 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.174 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.174 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.175 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.175 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 16, 2022-09-03T06:50:40.176 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:47:49.594713Z", 2022-09-03T06:50:40.177 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.177 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.178 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.179 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.179 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.180 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.180 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.181 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.182 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_J6Hg1f_fyNcpMbuABk2YgsFzhGeXXl-", 2022-09-03T06:50:40.182 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.183 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.183 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.184 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.185 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj76", 2022-09-03T06:50:40.185 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.186 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.186 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.187 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 2 2022-09-03T06:50:40.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.188 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.188 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.189 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.190 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.190 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 6, 2022-09-03T06:50:40.191 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:40:53.336929Z", 2022-09-03T06:50:40.191 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.192 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.192 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.193 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.193 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.194 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.194 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.195 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.196 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_LlsgAIieKSkvioq0e97LN0g_uyxuyoV", 2022-09-03T06:50:40.196 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.197 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.197 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.198 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj85", 2022-09-03T06:50:40.199 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.199 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.200 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.200 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 1 2022-09-03T06:50:40.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.201 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.201 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.202 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.202 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.203 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 3, 2022-09-03T06:50:40.203 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:37:27.217677Z", 2022-09-03T06:50:40.204 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.204 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.205 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.205 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.206 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.206 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.206 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.207 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.207 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_HTvYB6XMwi3WbpNTePqP_8p3Vcglpvr", 2022-09-03T06:50:40.208 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.208 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.208 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.209 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.209 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.209 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj87", 2022-09-03T06:50:40.209 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.210 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.211 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.211 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 13 2022-09-03T06:50:40.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.212 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.213 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.213 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.215 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.216 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 6, 2022-09-03T06:50:40.216 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:44:18.298005Z", 2022-09-03T06:50:40.216 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.217 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.217 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.217 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.218 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.218 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.218 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.218 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.219 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_E2i_5g4RgXQtc3pkBk4upctKK-MWrTW", 2022-09-03T06:50:40.219 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.220 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.220 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.220 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.220 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.221 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj92", 2022-09-03T06:50:40.221 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.221 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.221 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.222 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 12 2022-09-03T06:50:40.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.222 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.222 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.223 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.223 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.223 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 16, 2022-09-03T06:50:40.223 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:41:59.891668Z", 2022-09-03T06:50:40.224 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.224 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.225 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.225 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.226 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.226 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.226 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.227 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.228 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_qGJq5FXeNYia2PqqdioM9lCHlxrmITb", 2022-09-03T06:50:40.228 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.228 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.228 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.229 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.229 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj93", 2022-09-03T06:50:40.229 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.230 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.230 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.230 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 9 2022-09-03T06:50:40.230 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.231 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.231 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.231 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.231 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.232 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 6, 2022-09-03T06:50:40.232 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:45:46.926180Z", 2022-09-03T06:50:40.232 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.232 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.233 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.233 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.233 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.233 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.234 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.234 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.235 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_GSPRRHvQTC6RzIfhLWawyZTW-SnHLfi", 2022-09-03T06:50:40.235 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.236 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.236 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:50:40.237 INFO:teuthology.orchestra.run.smithi189.stdout: "name": "obj95", 2022-09-03T06:50:40.238 INFO:teuthology.orchestra.run.smithi189.stdout: "instance": "", 2022-09-03T06:50:40.238 INFO:teuthology.orchestra.run.smithi189.stdout: "ver": { 2022-09-03T06:50:40.239 INFO:teuthology.orchestra.run.smithi189.stdout: "pool": 9, 2022-09-03T06:50:40.239 INFO:teuthology.orchestra.run.smithi189.stdout: "epoch": 2 2022-09-03T06:50:40.240 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.240 INFO:teuthology.orchestra.run.smithi189.stdout: "locator": "", 2022-09-03T06:50:40.241 INFO:teuthology.orchestra.run.smithi189.stdout: "exists": true, 2022-09-03T06:50:40.241 INFO:teuthology.orchestra.run.smithi189.stdout: "meta": { 2022-09-03T06:50:40.242 INFO:teuthology.orchestra.run.smithi189.stdout: "category": 1, 2022-09-03T06:50:40.242 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 3, 2022-09-03T06:50:40.243 INFO:teuthology.orchestra.run.smithi189.stdout: "mtime": "2022-09-03T06:40:53.415040Z", 2022-09-03T06:50:40.243 INFO:teuthology.orchestra.run.smithi189.stdout: "etag": "acbd18db4cc2f85cedef654fccc4a4d8", 2022-09-03T06:50:40.244 INFO:teuthology.orchestra.run.smithi189.stdout: "storage_class": "", 2022-09-03T06:50:40.244 INFO:teuthology.orchestra.run.smithi189.stdout: "owner": "rgw-multisite-test-user", 2022-09-03T06:50:40.245 INFO:teuthology.orchestra.run.smithi189.stdout: "owner_display_name": "Multisite Test User", 2022-09-03T06:50:40.245 INFO:teuthology.orchestra.run.smithi189.stdout: "content_type": "application/octet-stream", 2022-09-03T06:50:40.246 INFO:teuthology.orchestra.run.smithi189.stdout: "accounted_size": 3, 2022-09-03T06:50:40.246 INFO:teuthology.orchestra.run.smithi189.stdout: "user_data": "", 2022-09-03T06:50:40.246 INFO:teuthology.orchestra.run.smithi189.stdout: "appendable": false 2022-09-03T06:50:40.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:50:40.247 INFO:teuthology.orchestra.run.smithi189.stdout: "tag": "_EtC04GLkLcchK97OB4of6eyeKrBRByf", 2022-09-03T06:50:40.248 INFO:teuthology.orchestra.run.smithi189.stdout: "flags": 0, 2022-09-03T06:50:40.248 INFO:teuthology.orchestra.run.smithi189.stdout: "pending_map": [], 2022-09-03T06:50:40.249 INFO:teuthology.orchestra.run.smithi189.stdout: "versioned_epoch": 0 2022-09-03T06:50:40.249 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:50:40.250 INFO:teuthology.orchestra.run.smithi189.stdout:] 2022-09-03T06:50:40.251 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:40.252 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj11', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:40.253 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj11 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:40.311 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:40.311 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:40.418 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj111', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:40.421 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj111', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:40.422 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj111 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:40.480 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:40.481 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:40.586 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj115', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:40.587 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj115', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:40.590 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj115 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:40.644 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:40.645 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:40.736 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj116', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:40.737 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj116', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:40.738 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj116 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:40.794 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:40.795 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:40.907 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj124', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:40.908 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj124', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:40.908 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj124 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:40.944 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 has been restored 2022-09-03T06:50:40.963 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:40.964 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:41.067 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj145', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:41.068 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj145', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:41.069 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj145 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:41.121 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:41.122 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:41.223 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj148', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:41.223 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj148', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:41.224 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj148 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:41.280 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:41.290 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:41.382 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj155', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:41.383 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj155', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:41.385 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj155 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:41.436 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:41.436 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:41.533 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:41.533 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj17', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:41.534 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj17 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:41.587 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:41.587 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:41.676 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj185', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:41.677 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj185', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:41.679 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj185 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:41.733 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:41.734 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:41.845 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:41.845 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj19', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:41.846 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj19 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:41.897 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:41.898 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:41.999 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj22', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:42.000 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj22', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:42.000 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj22 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:42.053 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:42.053 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:42.098 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:50:42.096+0000 7fd12f781700 -1 ERROR: failed operation: ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-5)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-5)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16) 2022-09-03T06:50:42.144 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:50:42.142+0000 7fd12f781700 -1 ERROR: failed operation: ERROR: collect() returned error (ret=-5)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16) 2022-09-03T06:50:42.164 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj23', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:42.165 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj23', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:42.165 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj23 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:42.217 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:42.218 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:42.263 INFO:tasks.rgw.c1.client.1.smithi181.stdout:2022-09-03T06:50:42.261+0000 7fd12f781700 -1 ERROR: failed operation: ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-16)ERROR: collect() returned error (ret=-5) 2022-09-03T06:50:42.311 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj29', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:42.312 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj29', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:42.314 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj29 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:42.368 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:42.369 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:42.467 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj31', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:42.468 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj31', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:42.469 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj31 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:42.521 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:42.521 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:42.618 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj33', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:42.619 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj33', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:42.620 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj33 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:42.672 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:42.673 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:42.793 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj34', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:42.794 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj34', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:42.795 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj34 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:42.847 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:42.848 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:42.939 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj37', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:42.940 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj37', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:42.941 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj37 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:42.991 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:42.992 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:43.105 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj38', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:43.106 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj38', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:43.107 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj38 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:43.159 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:43.161 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:43.265 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:43.267 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj44', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:43.268 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj44 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:43.322 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:43.322 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:43.419 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj46', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:43.420 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj46', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:43.421 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj46 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:43.474 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:43.475 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:43.587 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj51', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:43.588 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj51', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:43.588 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj51 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:43.640 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:43.641 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:43.758 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj55', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:43.759 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj55', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:43.762 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj55 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:43.815 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:43.816 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:43.917 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj57', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:43.918 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj57', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:43.919 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj57 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:43.974 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:43.975 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:44.071 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj58', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:44.072 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj58', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:44.073 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj58 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:44.129 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:44.130 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:44.242 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:44.243 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj62', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:44.245 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj62 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:44.297 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:44.298 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:44.393 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj68', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:44.393 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj68', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:44.394 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj68 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:44.446 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:44.447 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:44.547 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:44.549 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj69', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:44.549 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj69 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:44.602 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:44.603 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:44.718 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj74', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:44.718 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj74', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:44.719 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj74 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:44.772 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:44.772 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:44.874 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj76', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:44.875 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj76', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:44.876 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj76 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:44.928 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:44.929 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:45.045 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj85', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:45.046 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj85', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:45.047 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj85 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:45.099 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:45.099 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:45.203 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj87', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:45.204 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj87', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:45.205 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj87 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:45.257 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:45.258 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:45.364 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj92', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:45.365 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj92', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:45.366 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj92 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:45.420 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:45.421 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:45.532 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj93', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:45.534 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj93', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:45.535 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj93 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:45.590 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:45.591 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:45.691 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj95', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:45.692 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'object', 'rm', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--object', 'obj95', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:45.693 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 object rm --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --object obj95 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:45.745 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:45.746 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:45.843 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bilog', 'trim', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:45.843 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bilog', 'trim', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:45.844 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bilog trim --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:45.895 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:45.896 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:45.950 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'init', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:45.951 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'init', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:45.952 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync init --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:46.009 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:46.009 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:46.063 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'run', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:46.064 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'run', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--bucket', 'dsgvcy-75', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:50:46.066 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync run --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --bucket dsgvcy-75 --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:50:46.120 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:50:46.120 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:50:46.200 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.197+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: sync status for source bucket: init. lease is: not taken. stop indications is: 0 2022-09-03T06:50:46.204 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.201+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: waiting on bucket lease 2022-09-03T06:50:46.206 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.203+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: status after acquiring the lock is: init 2022-09-03T06:50:46.226 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.223+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: listing bucket for full sync 2022-09-03T06:50:46.276 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj11 2022-09-03T06:50:46.276 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj11]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj11 op=0 op_state=1 2022-09-03T06:50:46.276 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj11]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj11[0] 2022-09-03T06:50:46.277 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj111 2022-09-03T06:50:46.277 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj111]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj111 op=0 op_state=1 2022-09-03T06:50:46.278 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 0 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj111]: : injecting data sync error on key=obj111 2022-09-03T06:50:46.278 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 10 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj111]: failed, retcode=-5 ((5) Input/output error) 2022-09-03T06:50:46.279 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 0 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj111]: ERROR: failed to sync object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj111 2022-09-03T06:50:46.279 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj115 2022-09-03T06:50:46.279 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj115]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj115 op=0 op_state=1 2022-09-03T06:50:46.280 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj115]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj115[0] 2022-09-03T06:50:46.280 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj116 2022-09-03T06:50:46.280 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj116]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj116 op=0 op_state=1 2022-09-03T06:50:46.281 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj116]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj116[0] 2022-09-03T06:50:46.281 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj124 2022-09-03T06:50:46.284 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj124]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj124 op=0 op_state=1 2022-09-03T06:50:46.284 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj124]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj124[0] 2022-09-03T06:50:46.286 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj145 2022-09-03T06:50:46.286 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj145]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj145 op=0 op_state=1 2022-09-03T06:50:46.286 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj145]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj145[0] 2022-09-03T06:50:46.287 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj148 2022-09-03T06:50:46.287 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj148]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj148 op=0 op_state=1 2022-09-03T06:50:46.287 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj148]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj148[0] 2022-09-03T06:50:46.289 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj155 2022-09-03T06:50:46.289 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj155]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj155 op=0 op_state=1 2022-09-03T06:50:46.289 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 0 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj155]: : injecting data sync error on key=obj155 2022-09-03T06:50:46.289 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 10 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj155]: failed, retcode=-5 ((5) Input/output error) 2022-09-03T06:50:46.290 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 0 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj155]: ERROR: failed to sync object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj155 2022-09-03T06:50:46.290 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj17 2022-09-03T06:50:46.290 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj17]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj17 op=0 op_state=1 2022-09-03T06:50:46.291 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.269+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj17]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj17[0] 2022-09-03T06:50:46.291 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj185 2022-09-03T06:50:46.291 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj185]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj185 op=0 op_state=1 2022-09-03T06:50:46.292 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj185]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj185[0] 2022-09-03T06:50:46.292 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj19 2022-09-03T06:50:46.292 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj19]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj19 op=0 op_state=1 2022-09-03T06:50:46.293 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj19]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj19[0] 2022-09-03T06:50:46.293 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj22 2022-09-03T06:50:46.293 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj22]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj22 op=0 op_state=1 2022-09-03T06:50:46.294 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 0 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj22]: : injecting data sync error on key=obj22 2022-09-03T06:50:46.295 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 10 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj22]: failed, retcode=-5 ((5) Input/output error) 2022-09-03T06:50:46.295 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 0 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj22]: ERROR: failed to sync object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj22 2022-09-03T06:50:46.296 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj23 2022-09-03T06:50:46.296 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj23]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj23 op=0 op_state=1 2022-09-03T06:50:46.298 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj23]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj23[0] 2022-09-03T06:50:46.298 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj29 2022-09-03T06:50:46.299 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj29]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj29 op=0 op_state=1 2022-09-03T06:50:46.302 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 0 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj29]: : injecting data sync error on key=obj29 2022-09-03T06:50:46.302 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 10 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj29]: failed, retcode=-5 ((5) Input/output error) 2022-09-03T06:50:46.303 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 0 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj29]: ERROR: failed to sync object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj29 2022-09-03T06:50:46.305 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj31 2022-09-03T06:50:46.305 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj31]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj31 op=0 op_state=1 2022-09-03T06:50:46.306 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj31]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj31[0] 2022-09-03T06:50:46.306 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj33 2022-09-03T06:50:46.307 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj33]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj33 op=0 op_state=1 2022-09-03T06:50:46.308 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj33]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj33[0] 2022-09-03T06:50:46.310 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj34 2022-09-03T06:50:46.311 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj34]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj34 op=0 op_state=1 2022-09-03T06:50:46.311 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 0 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj34]: : injecting data sync error on key=obj34 2022-09-03T06:50:46.311 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 10 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj34]: failed, retcode=-5 ((5) Input/output error) 2022-09-03T06:50:46.312 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 0 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj34]: ERROR: failed to sync object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj34 2022-09-03T06:50:46.312 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj37 2022-09-03T06:50:46.313 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj37]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj37 op=0 op_state=1 2022-09-03T06:50:46.313 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj37]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj37[0] 2022-09-03T06:50:46.314 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj38 2022-09-03T06:50:46.314 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj38]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj38 op=0 op_state=1 2022-09-03T06:50:46.315 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj38]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj38[0] 2022-09-03T06:50:46.315 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj44 2022-09-03T06:50:46.316 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj44]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj44 op=0 op_state=1 2022-09-03T06:50:46.316 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj44]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj44[0] 2022-09-03T06:50:46.317 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj46 2022-09-03T06:50:46.317 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj46]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj46 op=0 op_state=1 2022-09-03T06:50:46.322 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.270+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj46]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj46[0] 2022-09-03T06:50:46.323 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.272+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: ERROR: full sync failed. error: -5 2022-09-03T06:50:46.325 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.290+0000 7fdd98754600 10 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj148]: success 2022-09-03T06:50:46.325 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.290+0000 7fdd98754600 10 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj44]: success 2022-09-03T06:50:46.325 INFO:teuthology.orchestra.run.smithi189.stderr:*** Caught signal (Segmentation fault) ** 2022-09-03T06:50:46.326 INFO:teuthology.orchestra.run.smithi189.stderr: in thread 7fdd98754600 thread_name:radosgw-admin 2022-09-03T06:50:46.326 INFO:teuthology.orchestra.run.smithi189.stderr: ceph version 17.0.0-14661-g4d487742 (4d487742eda56d4badbc9bdb33761243365bb37a) quincy (dev) 2022-09-03T06:50:46.326 INFO:teuthology.orchestra.run.smithi189.stderr: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fdd9ff17cf0] 2022-09-03T06:50:46.327 INFO:teuthology.orchestra.run.smithi189.stderr: 2: (tcmalloc::ThreadCache::ReleaseToCentralCache(tcmalloc::ThreadCache::FreeList*, unsigned int, int)+0x93) [0x7fdda0f65523] 2022-09-03T06:50:46.327 INFO:teuthology.orchestra.run.smithi189.stderr: 3: (tcmalloc::ThreadCache::Scavenge()+0x4e) [0x7fdda0f6586e] 2022-09-03T06:50:46.328 INFO:teuthology.orchestra.run.smithi189.stderr: 4: (RGWFetchRemoteObjCR::~RGWFetchRemoteObjCR()+0x43) [0x56538f21a593] 2022-09-03T06:50:46.328 INFO:teuthology.orchestra.run.smithi189.stderr: 5: (ceph::common::RefCountedObject::put() const+0x1ba) [0x7fdd9f3464da] 2022-09-03T06:50:46.329 INFO:teuthology.orchestra.run.smithi189.stderr: 6: (RGWCoroutinesStack::operate(DoutPrefixProvider const*, RGWCoroutinesEnv*)+0x360) [0x56538f359190] 2022-09-03T06:50:46.329 INFO:teuthology.orchestra.run.smithi189.stderr: 7: (RGWCoroutinesManager::run(DoutPrefixProvider const*, std::__cxx11::list >&)+0x521) [0x56538f359791] 2022-09-03T06:50:46.330 INFO:teuthology.orchestra.run.smithi189.stderr: 8: (RGWBucketPipeSyncStatusManager::run(DoutPrefixProvider const*)+0xaca) [0x56538f20985a] 2022-09-03T06:50:46.331 INFO:teuthology.orchestra.run.smithi189.stderr: 9: main() 2022-09-03T06:50:46.331 INFO:teuthology.orchestra.run.smithi189.stderr: 10: __libc_start_main() 2022-09-03T06:50:46.332 INFO:teuthology.orchestra.run.smithi189.stderr: 11: _start() 2022-09-03T06:50:46.332 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:50:46.295+0000 7fdd98754600 -1 *** Caught signal (Segmentation fault) ** 2022-09-03T06:50:46.333 INFO:teuthology.orchestra.run.smithi189.stderr: in thread 7fdd98754600 thread_name:radosgw-admin 2022-09-03T06:50:46.333 INFO:teuthology.orchestra.run.smithi189.stderr: 2022-09-03T06:50:46.334 INFO:teuthology.orchestra.run.smithi189.stderr: ceph version 17.0.0-14661-g4d487742 (4d487742eda56d4badbc9bdb33761243365bb37a) quincy (dev) 2022-09-03T06:50:46.334 INFO:teuthology.orchestra.run.smithi189.stderr: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fdd9ff17cf0] 2022-09-03T06:50:46.335 INFO:teuthology.orchestra.run.smithi189.stderr: 2: (tcmalloc::ThreadCache::ReleaseToCentralCache(tcmalloc::ThreadCache::FreeList*, unsigned int, int)+0x93) [0x7fdda0f65523] 2022-09-03T06:50:46.338 INFO:teuthology.orchestra.run.smithi189.stderr: 3: (tcmalloc::ThreadCache::Scavenge()+0x4e) [0x7fdda0f6586e] 2022-09-03T06:50:46.339 INFO:teuthology.orchestra.run.smithi189.stderr: 4: (RGWFetchRemoteObjCR::~RGWFetchRemoteObjCR()+0x43) [0x56538f21a593] 2022-09-03T06:50:46.340 INFO:teuthology.orchestra.run.smithi189.stderr: 5: (ceph::common::RefCountedObject::put() const+0x1ba) [0x7fdd9f3464da] 2022-09-03T06:50:46.340 INFO:teuthology.orchestra.run.smithi189.stderr: 6: (RGWCoroutinesStack::operate(DoutPrefixProvider const*, RGWCoroutinesEnv*)+0x360) [0x56538f359190] 2022-09-03T06:50:46.340 INFO:teuthology.orchestra.run.smithi189.stderr: 7: (RGWCoroutinesManager::run(DoutPrefixProvider const*, std::__cxx11::list >&)+0x521) [0x56538f359791] 2022-09-03T06:50:46.341 INFO:teuthology.orchestra.run.smithi189.stderr: 8: (RGWBucketPipeSyncStatusManager::run(DoutPrefixProvider const*)+0xaca) [0x56538f20985a] 2022-09-03T06:50:46.342 INFO:teuthology.orchestra.run.smithi189.stderr: 9: main() 2022-09-03T06:50:46.342 INFO:teuthology.orchestra.run.smithi189.stderr: 10: __libc_start_main() 2022-09-03T06:50:46.342 INFO:teuthology.orchestra.run.smithi189.stderr: 11: _start() 2022-09-03T06:50:46.342 INFO:teuthology.orchestra.run.smithi189.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2022-09-03T06:50:46.343 INFO:teuthology.orchestra.run.smithi189.stderr: 2022-09-03T06:50:46.343 INFO:teuthology.orchestra.run.smithi189.stderr:--- begin dump of recent events --- 2022-09-03T06:50:46.343 INFO:teuthology.orchestra.run.smithi189.stderr: -499> 2022-09-03T06:50:46.117+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: secure 2022-09-03T06:50:46.343 INFO:teuthology.orchestra.run.smithi189.stderr: -498> 2022-09-03T06:50:46.117+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: crc 2022-09-03T06:50:46.344 INFO:teuthology.orchestra.run.smithi189.stderr: -497> 2022-09-03T06:50:46.117+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: secure 2022-09-03T06:50:46.344 INFO:teuthology.orchestra.run.smithi189.stderr: -496> 2022-09-03T06:50:46.117+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: crc 2022-09-03T06:50:46.344 INFO:teuthology.orchestra.run.smithi189.stderr: -495> 2022-09-03T06:50:46.117+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: secure 2022-09-03T06:50:46.345 INFO:teuthology.orchestra.run.smithi189.stderr: -494> 2022-09-03T06:50:46.117+0000 7fdd98754600 2 auth: KeyRing::load: loaded key file /etc/ceph/c2.client.0.keyring 2022-09-03T06:50:46.345 INFO:teuthology.orchestra.run.smithi189.stderr: -493> 2022-09-03T06:50:46.117+0000 7fdd98754600 10 monclient: init 2022-09-03T06:50:46.346 INFO:teuthology.orchestra.run.smithi189.stderr: -492> 2022-09-03T06:50:46.117+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ece690) adding auth protocol: cephx 2022-09-03T06:50:46.346 INFO:teuthology.orchestra.run.smithi189.stderr: -491> 2022-09-03T06:50:46.117+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ece690) adding auth protocol: cephx 2022-09-03T06:50:46.349 INFO:teuthology.orchestra.run.smithi189.stderr: -490> 2022-09-03T06:50:46.117+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ece690) adding auth protocol: cephx 2022-09-03T06:50:46.349 INFO:teuthology.orchestra.run.smithi189.stderr: -489> 2022-09-03T06:50:46.117+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ece690) adding con mode: secure 2022-09-03T06:50:46.349 INFO:teuthology.orchestra.run.smithi189.stderr: -488> 2022-09-03T06:50:46.117+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ece690) adding con mode: crc 2022-09-03T06:50:46.350 INFO:teuthology.orchestra.run.smithi189.stderr: -487> 2022-09-03T06:50:46.117+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ece690) adding con mode: secure 2022-09-03T06:50:46.350 INFO:teuthology.orchestra.run.smithi189.stderr: -486> 2022-09-03T06:50:46.117+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ece690) adding con mode: crc 2022-09-03T06:50:46.350 INFO:teuthology.orchestra.run.smithi189.stderr: -485> 2022-09-03T06:50:46.117+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ece690) adding con mode: secure 2022-09-03T06:50:46.351 INFO:teuthology.orchestra.run.smithi189.stderr: -484> 2022-09-03T06:50:46.117+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ece690) adding con mode: crc 2022-09-03T06:50:46.351 INFO:teuthology.orchestra.run.smithi189.stderr: -483> 2022-09-03T06:50:46.117+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ece690) adding con mode: crc 2022-09-03T06:50:46.351 INFO:teuthology.orchestra.run.smithi189.stderr: -482> 2022-09-03T06:50:46.117+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ece690) adding con mode: secure 2022-09-03T06:50:46.351 INFO:teuthology.orchestra.run.smithi189.stderr: -481> 2022-09-03T06:50:46.117+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ece690) adding con mode: crc 2022-09-03T06:50:46.352 INFO:teuthology.orchestra.run.smithi189.stderr: -480> 2022-09-03T06:50:46.117+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ece690) adding con mode: secure 2022-09-03T06:50:46.352 INFO:teuthology.orchestra.run.smithi189.stderr: -479> 2022-09-03T06:50:46.117+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ece690) adding con mode: crc 2022-09-03T06:50:46.352 INFO:teuthology.orchestra.run.smithi189.stderr: -478> 2022-09-03T06:50:46.117+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ece690) adding con mode: secure 2022-09-03T06:50:46.353 INFO:teuthology.orchestra.run.smithi189.stderr: -477> 2022-09-03T06:50:46.117+0000 7fdd98754600 2 auth: KeyRing::load: loaded key file /etc/ceph/c2.client.0.keyring 2022-09-03T06:50:46.353 INFO:teuthology.orchestra.run.smithi189.stderr: -476> 2022-09-03T06:50:46.117+0000 7fdd98754600 2 auth: KeyRing::load: loaded key file /etc/ceph/c2.client.0.keyring 2022-09-03T06:50:46.353 INFO:teuthology.orchestra.run.smithi189.stderr: -475> 2022-09-03T06:50:46.117+0000 7fdd98754600 10 monclient: _reopen_session rank -1 2022-09-03T06:50:46.354 INFO:teuthology.orchestra.run.smithi189.stderr: -474> 2022-09-03T06:50:46.117+0000 7fdd98754600 10 monclient: _add_conns ranks=[0,1] 2022-09-03T06:50:46.354 INFO:teuthology.orchestra.run.smithi189.stderr: -473> 2022-09-03T06:50:46.117+0000 7fdd98754600 10 monclient(hunting): picked mon.noname-a-legacy con 0x565390bb6800 addr v1:172.21.15.189:6789/0 2022-09-03T06:50:46.354 INFO:teuthology.orchestra.run.smithi189.stderr: -472> 2022-09-03T06:50:46.117+0000 7fdd98754600 10 monclient(hunting): picked mon.noname-a con 0x565390bb6c00 addr v2:172.21.15.189:3300/0 2022-09-03T06:50:46.354 INFO:teuthology.orchestra.run.smithi189.stderr: -471> 2022-09-03T06:50:46.117+0000 7fdd98754600 10 monclient(hunting): start opening mon connection 2022-09-03T06:50:46.355 INFO:teuthology.orchestra.run.smithi189.stderr: -470> 2022-09-03T06:50:46.117+0000 7fdd98754600 10 monclient(hunting): _renew_subs 2022-09-03T06:50:46.355 INFO:teuthology.orchestra.run.smithi189.stderr: -469> 2022-09-03T06:50:46.117+0000 7fdd98754600 10 monclient(hunting): authenticate will time out at 6734.533794s 2022-09-03T06:50:46.355 INFO:teuthology.orchestra.run.smithi189.stderr: -468> 2022-09-03T06:50:46.117+0000 7fdd95eff700 10 monclient(hunting): get_auth_request con 0x565390bb6c00 auth_method 0 2022-09-03T06:50:46.355 INFO:teuthology.orchestra.run.smithi189.stderr: -467> 2022-09-03T06:50:46.117+0000 7fdd95eff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2022-09-03T06:50:46.356 INFO:teuthology.orchestra.run.smithi189.stderr: -466> 2022-09-03T06:50:46.117+0000 7fdd95eff700 10 monclient(hunting): _init_auth method 2 2022-09-03T06:50:46.356 INFO:teuthology.orchestra.run.smithi189.stderr: -465> 2022-09-03T06:50:46.117+0000 7fdd95eff700 10 monclient(hunting): _init_auth creating new auth 2022-09-03T06:50:46.356 INFO:teuthology.orchestra.run.smithi189.stderr: -464> 2022-09-03T06:50:46.118+0000 7fdd95eff700 10 monclient(hunting): handle_auth_reply_more payload 9 2022-09-03T06:50:46.357 INFO:teuthology.orchestra.run.smithi189.stderr: -463> 2022-09-03T06:50:46.118+0000 7fdd95eff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2022-09-03T06:50:46.357 INFO:teuthology.orchestra.run.smithi189.stderr: -462> 2022-09-03T06:50:46.118+0000 7fdd95eff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2022-09-03T06:50:46.357 INFO:teuthology.orchestra.run.smithi189.stderr: -461> 2022-09-03T06:50:46.118+0000 7fdd95eff700 10 monclient(hunting): handle_auth_done global_id 6688 payload 274 2022-09-03T06:50:46.357 INFO:teuthology.orchestra.run.smithi189.stderr: -460> 2022-09-03T06:50:46.118+0000 7fdd95eff700 10 monclient: _finish_hunting 0 2022-09-03T06:50:46.358 INFO:teuthology.orchestra.run.smithi189.stderr: -459> 2022-09-03T06:50:46.118+0000 7fdd95eff700 1 monclient: found mon.noname-a 2022-09-03T06:50:46.358 INFO:teuthology.orchestra.run.smithi189.stderr: -458> 2022-09-03T06:50:46.118+0000 7fdd95eff700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.189:3300/0 2022-09-03T06:50:46.358 INFO:teuthology.orchestra.run.smithi189.stderr: -457> 2022-09-03T06:50:46.118+0000 7fdd956fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2022-09-03T06:50:46.359 INFO:teuthology.orchestra.run.smithi189.stderr: -456> 2022-09-03T06:50:46.118+0000 7fdd956fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2022-09-03T06:50:46.359 INFO:teuthology.orchestra.run.smithi189.stderr: -455> 2022-09-03T06:50:46.118+0000 7fdd956fe700 10 monclient: dump: 2022-09-03T06:50:46.360 INFO:teuthology.orchestra.run.smithi189.stderr:epoch 1 2022-09-03T06:50:46.360 INFO:teuthology.orchestra.run.smithi189.stderr:fsid 08f0e03a-4e9e-4009-81f6-ef837e373516 2022-09-03T06:50:46.361 INFO:teuthology.orchestra.run.smithi189.stderr:last_changed 2022-09-03T05:16:00.054991+0000 2022-09-03T06:50:46.361 INFO:teuthology.orchestra.run.smithi189.stderr:created 2022-09-03T05:16:00.054991+0000 2022-09-03T06:50:46.362 INFO:teuthology.orchestra.run.smithi189.stderr:min_mon_release 17 (quincy) 2022-09-03T06:50:46.362 INFO:teuthology.orchestra.run.smithi189.stderr:election_strategy: 1 2022-09-03T06:50:46.363 INFO:teuthology.orchestra.run.smithi189.stderr:0: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.a 2022-09-03T06:50:46.363 INFO:teuthology.orchestra.run.smithi189.stderr: 2022-09-03T06:50:46.364 INFO:teuthology.orchestra.run.smithi189.stderr: -454> 2022-09-03T06:50:46.118+0000 7fdd956fe700 10 monclient: _finish_auth 0 2022-09-03T06:50:46.364 INFO:teuthology.orchestra.run.smithi189.stderr: -453> 2022-09-03T06:50:46.118+0000 7fdd956fe700 10 monclient: _check_auth_tickets 2022-09-03T06:50:46.364 INFO:teuthology.orchestra.run.smithi189.stderr: -452> 2022-09-03T06:50:46.118+0000 7fdd98754600 5 monclient: authenticate success, global_id 6688 2022-09-03T06:50:46.364 INFO:teuthology.orchestra.run.smithi189.stderr: -451> 2022-09-03T06:50:46.118+0000 7fdd956fe700 10 monclient: handle_config config(0 keys) v1 2022-09-03T06:50:46.365 INFO:teuthology.orchestra.run.smithi189.stderr: -450> 2022-09-03T06:50:46.118+0000 7fdd956fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2022-09-03T06:50:46.365 INFO:teuthology.orchestra.run.smithi189.stderr: -449> 2022-09-03T06:50:46.118+0000 7fdd956fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2022-09-03T06:50:46.365 INFO:teuthology.orchestra.run.smithi189.stderr: -448> 2022-09-03T06:50:46.118+0000 7fdd956fe700 10 monclient: dump: 2022-09-03T06:50:46.365 INFO:teuthology.orchestra.run.smithi189.stderr:epoch 1 2022-09-03T06:50:46.366 INFO:teuthology.orchestra.run.smithi189.stderr:fsid 08f0e03a-4e9e-4009-81f6-ef837e373516 2022-09-03T06:50:46.366 INFO:teuthology.orchestra.run.smithi189.stderr:last_changed 2022-09-03T05:16:00.054991+0000 2022-09-03T06:50:46.366 INFO:teuthology.orchestra.run.smithi189.stderr:created 2022-09-03T05:16:00.054991+0000 2022-09-03T06:50:46.367 INFO:teuthology.orchestra.run.smithi189.stderr:min_mon_release 17 (quincy) 2022-09-03T06:50:46.367 INFO:teuthology.orchestra.run.smithi189.stderr:election_strategy: 1 2022-09-03T06:50:46.367 INFO:teuthology.orchestra.run.smithi189.stderr:0: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.a 2022-09-03T06:50:46.368 INFO:teuthology.orchestra.run.smithi189.stderr: 2022-09-03T06:50:46.368 INFO:teuthology.orchestra.run.smithi189.stderr: -447> 2022-09-03T06:50:46.118+0000 7fdd98754600 10 monclient: get_monmap_and_config success 2022-09-03T06:50:46.368 INFO:teuthology.orchestra.run.smithi189.stderr: -446> 2022-09-03T06:50:46.118+0000 7fdd98754600 4 set_mon_vals no callback set 2022-09-03T06:50:46.369 INFO:teuthology.orchestra.run.smithi189.stderr: -445> 2022-09-03T06:50:46.118+0000 7fdd98754600 10 monclient: shutdown 2022-09-03T06:50:46.369 INFO:teuthology.orchestra.run.smithi189.stderr: -444> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 asok(0x565390c4a000) init /var/run/ceph/c2-client.0.154806.asok 2022-09-03T06:50:46.369 INFO:teuthology.orchestra.run.smithi189.stderr: -443> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 asok(0x565390c4a000) bind_and_listen /var/run/ceph/c2-client.0.154806.asok 2022-09-03T06:50:46.369 INFO:teuthology.orchestra.run.smithi189.stderr: -442> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 asok(0x565390c4a000) register_command 0 hook 0x565390c07378 2022-09-03T06:50:46.370 INFO:teuthology.orchestra.run.smithi189.stderr: -441> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 asok(0x565390c4a000) register_command version hook 0x565390c07378 2022-09-03T06:50:46.370 INFO:teuthology.orchestra.run.smithi189.stderr: -440> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 asok(0x565390c4a000) register_command git_version hook 0x565390c07378 2022-09-03T06:50:46.371 INFO:teuthology.orchestra.run.smithi189.stderr: -439> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 asok(0x565390c4a000) register_command help hook 0x565390baabf0 2022-09-03T06:50:46.371 INFO:teuthology.orchestra.run.smithi189.stderr: -438> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 asok(0x565390c4a000) register_command get_command_descriptions hook 0x565390baab50 2022-09-03T06:50:46.371 INFO:teuthology.orchestra.run.smithi189.stderr: -437> 2022-09-03T06:50:46.119+0000 7fdd956fe700 5 asok(0x565390c4a000) entry start 2022-09-03T06:50:46.371 INFO:teuthology.orchestra.run.smithi189.stderr: -436> 2022-09-03T06:50:46.119+0000 7fdd98754600 10 monclient: get_monmap_and_config 2022-09-03T06:50:46.372 INFO:teuthology.orchestra.run.smithi189.stderr: -435> 2022-09-03T06:50:46.119+0000 7fdd98754600 10 monclient: build_initial_monmap 2022-09-03T06:50:46.372 INFO:teuthology.orchestra.run.smithi189.stderr: -434> 2022-09-03T06:50:46.119+0000 7fdd98754600 1 build_initial for_mkfs: 0 2022-09-03T06:50:46.372 INFO:teuthology.orchestra.run.smithi189.stderr: -433> 2022-09-03T06:50:46.119+0000 7fdd98754600 10 monclient: monmap: 2022-09-03T06:50:46.372 INFO:teuthology.orchestra.run.smithi189.stderr:epoch 0 2022-09-03T06:50:46.373 INFO:teuthology.orchestra.run.smithi189.stderr:fsid 00000000-0000-0000-0000-000000000000 2022-09-03T06:50:46.373 INFO:teuthology.orchestra.run.smithi189.stderr:last_changed 0.000000 2022-09-03T06:50:46.373 INFO:teuthology.orchestra.run.smithi189.stderr:created 0.000000 2022-09-03T06:50:46.374 INFO:teuthology.orchestra.run.smithi189.stderr:min_mon_release 0 (unknown) 2022-09-03T06:50:46.374 INFO:teuthology.orchestra.run.smithi189.stderr:election_strategy: 1 2022-09-03T06:50:46.374 INFO:teuthology.orchestra.run.smithi189.stderr:0: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.noname-a 2022-09-03T06:50:46.375 INFO:teuthology.orchestra.run.smithi189.stderr: 2022-09-03T06:50:46.375 INFO:teuthology.orchestra.run.smithi189.stderr: -432> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding auth protocol: cephx 2022-09-03T06:50:46.375 INFO:teuthology.orchestra.run.smithi189.stderr: -431> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding auth protocol: cephx 2022-09-03T06:50:46.375 INFO:teuthology.orchestra.run.smithi189.stderr: -430> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding auth protocol: cephx 2022-09-03T06:50:46.376 INFO:teuthology.orchestra.run.smithi189.stderr: -429> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: secure 2022-09-03T06:50:46.376 INFO:teuthology.orchestra.run.smithi189.stderr: -428> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: crc 2022-09-03T06:50:46.376 INFO:teuthology.orchestra.run.smithi189.stderr: -427> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: secure 2022-09-03T06:50:46.376 INFO:teuthology.orchestra.run.smithi189.stderr: -426> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: crc 2022-09-03T06:50:46.377 INFO:teuthology.orchestra.run.smithi189.stderr: -425> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: secure 2022-09-03T06:50:46.377 INFO:teuthology.orchestra.run.smithi189.stderr: -424> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: crc 2022-09-03T06:50:46.377 INFO:teuthology.orchestra.run.smithi189.stderr: -423> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: crc 2022-09-03T06:50:46.377 INFO:teuthology.orchestra.run.smithi189.stderr: -422> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: secure 2022-09-03T06:50:46.378 INFO:teuthology.orchestra.run.smithi189.stderr: -421> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: crc 2022-09-03T06:50:46.378 INFO:teuthology.orchestra.run.smithi189.stderr: -420> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: secure 2022-09-03T06:50:46.378 INFO:teuthology.orchestra.run.smithi189.stderr: -419> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: crc 2022-09-03T06:50:46.379 INFO:teuthology.orchestra.run.smithi189.stderr: -418> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: secure 2022-09-03T06:50:46.379 INFO:teuthology.orchestra.run.smithi189.stderr: -417> 2022-09-03T06:50:46.119+0000 7fdd98754600 2 auth: KeyRing::load: loaded key file /etc/ceph/c2.client.0.keyring 2022-09-03T06:50:46.379 INFO:teuthology.orchestra.run.smithi189.stderr: -416> 2022-09-03T06:50:46.119+0000 7fdd98754600 10 monclient: init 2022-09-03T06:50:46.379 INFO:teuthology.orchestra.run.smithi189.stderr: -415> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecefd0) adding auth protocol: cephx 2022-09-03T06:50:46.380 INFO:teuthology.orchestra.run.smithi189.stderr: -414> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecefd0) adding auth protocol: cephx 2022-09-03T06:50:46.380 INFO:teuthology.orchestra.run.smithi189.stderr: -413> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecefd0) adding auth protocol: cephx 2022-09-03T06:50:46.380 INFO:teuthology.orchestra.run.smithi189.stderr: -412> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecefd0) adding con mode: secure 2022-09-03T06:50:46.380 INFO:teuthology.orchestra.run.smithi189.stderr: -411> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecefd0) adding con mode: crc 2022-09-03T06:50:46.381 INFO:teuthology.orchestra.run.smithi189.stderr: -410> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecefd0) adding con mode: secure 2022-09-03T06:50:46.381 INFO:teuthology.orchestra.run.smithi189.stderr: -409> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecefd0) adding con mode: crc 2022-09-03T06:50:46.381 INFO:teuthology.orchestra.run.smithi189.stderr: -408> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecefd0) adding con mode: secure 2022-09-03T06:50:46.381 INFO:teuthology.orchestra.run.smithi189.stderr: -407> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecefd0) adding con mode: crc 2022-09-03T06:50:46.382 INFO:teuthology.orchestra.run.smithi189.stderr: -406> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecefd0) adding con mode: crc 2022-09-03T06:50:46.382 INFO:teuthology.orchestra.run.smithi189.stderr: -405> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecefd0) adding con mode: secure 2022-09-03T06:50:46.382 INFO:teuthology.orchestra.run.smithi189.stderr: -404> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecefd0) adding con mode: crc 2022-09-03T06:50:46.382 INFO:teuthology.orchestra.run.smithi189.stderr: -403> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecefd0) adding con mode: secure 2022-09-03T06:50:46.383 INFO:teuthology.orchestra.run.smithi189.stderr: -402> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecefd0) adding con mode: crc 2022-09-03T06:50:46.383 INFO:teuthology.orchestra.run.smithi189.stderr: -401> 2022-09-03T06:50:46.119+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecefd0) adding con mode: secure 2022-09-03T06:50:46.383 INFO:teuthology.orchestra.run.smithi189.stderr: -400> 2022-09-03T06:50:46.119+0000 7fdd98754600 2 auth: KeyRing::load: loaded key file /etc/ceph/c2.client.0.keyring 2022-09-03T06:50:46.383 INFO:teuthology.orchestra.run.smithi189.stderr: -399> 2022-09-03T06:50:46.119+0000 7fdd98754600 2 auth: KeyRing::load: loaded key file /etc/ceph/c2.client.0.keyring 2022-09-03T06:50:46.384 INFO:teuthology.orchestra.run.smithi189.stderr: -398> 2022-09-03T06:50:46.119+0000 7fdd98754600 10 monclient: _reopen_session rank -1 2022-09-03T06:50:46.384 INFO:teuthology.orchestra.run.smithi189.stderr: -397> 2022-09-03T06:50:46.119+0000 7fdd98754600 10 monclient: _add_conns ranks=[0] 2022-09-03T06:50:46.384 INFO:teuthology.orchestra.run.smithi189.stderr: -396> 2022-09-03T06:50:46.119+0000 7fdd98754600 10 monclient(hunting): picked mon.noname-a con 0x565390bb6800 addr [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] 2022-09-03T06:50:46.384 INFO:teuthology.orchestra.run.smithi189.stderr: -395> 2022-09-03T06:50:46.119+0000 7fdd98754600 10 monclient(hunting): start opening mon connection 2022-09-03T06:50:46.385 INFO:teuthology.orchestra.run.smithi189.stderr: -394> 2022-09-03T06:50:46.119+0000 7fdd98754600 10 monclient(hunting): _renew_subs 2022-09-03T06:50:46.385 INFO:teuthology.orchestra.run.smithi189.stderr: -393> 2022-09-03T06:50:46.119+0000 7fdd98754600 10 monclient(hunting): authenticate will time out at 6734.535801s 2022-09-03T06:50:46.385 INFO:teuthology.orchestra.run.smithi189.stderr: -392> 2022-09-03T06:50:46.119+0000 7fdd96700700 10 monclient(hunting): get_auth_request con 0x565390bb6800 auth_method 0 2022-09-03T06:50:46.385 INFO:teuthology.orchestra.run.smithi189.stderr: -391> 2022-09-03T06:50:46.119+0000 7fdd96700700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2022-09-03T06:50:46.386 INFO:teuthology.orchestra.run.smithi189.stderr: -390> 2022-09-03T06:50:46.119+0000 7fdd96700700 10 monclient(hunting): _init_auth method 2 2022-09-03T06:50:46.386 INFO:teuthology.orchestra.run.smithi189.stderr: -389> 2022-09-03T06:50:46.119+0000 7fdd96700700 10 monclient(hunting): _init_auth creating new auth 2022-09-03T06:50:46.386 INFO:teuthology.orchestra.run.smithi189.stderr: -388> 2022-09-03T06:50:46.119+0000 7fdd96700700 10 monclient(hunting): handle_auth_reply_more payload 9 2022-09-03T06:50:46.387 INFO:teuthology.orchestra.run.smithi189.stderr: -387> 2022-09-03T06:50:46.119+0000 7fdd96700700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2022-09-03T06:50:46.387 INFO:teuthology.orchestra.run.smithi189.stderr: -386> 2022-09-03T06:50:46.119+0000 7fdd96700700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2022-09-03T06:50:46.387 INFO:teuthology.orchestra.run.smithi189.stderr: -385> 2022-09-03T06:50:46.120+0000 7fdd96700700 10 monclient(hunting): handle_auth_done global_id 6689 payload 274 2022-09-03T06:50:46.387 INFO:teuthology.orchestra.run.smithi189.stderr: -384> 2022-09-03T06:50:46.120+0000 7fdd96700700 10 monclient: _finish_hunting 0 2022-09-03T06:50:46.388 INFO:teuthology.orchestra.run.smithi189.stderr: -383> 2022-09-03T06:50:46.120+0000 7fdd96700700 1 monclient: found mon.noname-a 2022-09-03T06:50:46.388 INFO:teuthology.orchestra.run.smithi189.stderr: -382> 2022-09-03T06:50:46.120+0000 7fdd96700700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.189:3300/0 2022-09-03T06:50:46.388 INFO:teuthology.orchestra.run.smithi189.stderr: -381> 2022-09-03T06:50:46.120+0000 7fdd946fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2022-09-03T06:50:46.388 INFO:teuthology.orchestra.run.smithi189.stderr: -380> 2022-09-03T06:50:46.120+0000 7fdd946fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2022-09-03T06:50:46.389 INFO:teuthology.orchestra.run.smithi189.stderr: -379> 2022-09-03T06:50:46.120+0000 7fdd946fc700 10 monclient: dump: 2022-09-03T06:50:46.389 INFO:teuthology.orchestra.run.smithi189.stderr:epoch 1 2022-09-03T06:50:46.389 INFO:teuthology.orchestra.run.smithi189.stderr:fsid 08f0e03a-4e9e-4009-81f6-ef837e373516 2022-09-03T06:50:46.390 INFO:teuthology.orchestra.run.smithi189.stderr:last_changed 2022-09-03T05:16:00.054991+0000 2022-09-03T06:50:46.390 INFO:teuthology.orchestra.run.smithi189.stderr:created 2022-09-03T05:16:00.054991+0000 2022-09-03T06:50:46.390 INFO:teuthology.orchestra.run.smithi189.stderr:min_mon_release 17 (quincy) 2022-09-03T06:50:46.390 INFO:teuthology.orchestra.run.smithi189.stderr:election_strategy: 1 2022-09-03T06:50:46.391 INFO:teuthology.orchestra.run.smithi189.stderr:0: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.a 2022-09-03T06:50:46.391 INFO:teuthology.orchestra.run.smithi189.stderr: 2022-09-03T06:50:46.391 INFO:teuthology.orchestra.run.smithi189.stderr: -378> 2022-09-03T06:50:46.120+0000 7fdd946fc700 10 monclient: _finish_auth 0 2022-09-03T06:50:46.391 INFO:teuthology.orchestra.run.smithi189.stderr: -377> 2022-09-03T06:50:46.120+0000 7fdd946fc700 10 monclient: _check_auth_tickets 2022-09-03T06:50:46.392 INFO:teuthology.orchestra.run.smithi189.stderr: -376> 2022-09-03T06:50:46.120+0000 7fdd946fc700 10 monclient: handle_config config(0 keys) v1 2022-09-03T06:50:46.392 INFO:teuthology.orchestra.run.smithi189.stderr: -375> 2022-09-03T06:50:46.120+0000 7fdd946fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2022-09-03T06:50:46.392 INFO:teuthology.orchestra.run.smithi189.stderr: -374> 2022-09-03T06:50:46.120+0000 7fdd946fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2022-09-03T06:50:46.392 INFO:teuthology.orchestra.run.smithi189.stderr: -373> 2022-09-03T06:50:46.120+0000 7fdd946fc700 10 monclient: dump: 2022-09-03T06:50:46.393 INFO:teuthology.orchestra.run.smithi189.stderr:epoch 1 2022-09-03T06:50:46.393 INFO:teuthology.orchestra.run.smithi189.stderr:fsid 08f0e03a-4e9e-4009-81f6-ef837e373516 2022-09-03T06:50:46.393 INFO:teuthology.orchestra.run.smithi189.stderr:last_changed 2022-09-03T05:16:00.054991+0000 2022-09-03T06:50:46.393 INFO:teuthology.orchestra.run.smithi189.stderr:created 2022-09-03T05:16:00.054991+0000 2022-09-03T06:50:46.394 INFO:teuthology.orchestra.run.smithi189.stderr:min_mon_release 17 (quincy) 2022-09-03T06:50:46.394 INFO:teuthology.orchestra.run.smithi189.stderr:election_strategy: 1 2022-09-03T06:50:46.401 INFO:teuthology.orchestra.run.smithi189.stderr:0: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.a 2022-09-03T06:50:46.401 INFO:teuthology.orchestra.run.smithi189.stderr: 2022-09-03T06:50:46.401 INFO:teuthology.orchestra.run.smithi189.stderr: -372> 2022-09-03T06:50:46.120+0000 7fdd98754600 5 monclient: authenticate success, global_id 6689 2022-09-03T06:50:46.402 INFO:teuthology.orchestra.run.smithi189.stderr: -371> 2022-09-03T06:50:46.120+0000 7fdd98754600 10 monclient: get_monmap_and_config success 2022-09-03T06:50:46.402 INFO:teuthology.orchestra.run.smithi189.stderr: -370> 2022-09-03T06:50:46.120+0000 7fdd98754600 4 set_mon_vals no callback set 2022-09-03T06:50:46.402 INFO:teuthology.orchestra.run.smithi189.stderr: -369> 2022-09-03T06:50:46.120+0000 7fdd98754600 10 monclient: shutdown 2022-09-03T06:50:46.402 INFO:teuthology.orchestra.run.smithi189.stderr: -368> 2022-09-03T06:50:46.120+0000 7fdd98754600 10 monclient: build_initial_monmap 2022-09-03T06:50:46.403 INFO:teuthology.orchestra.run.smithi189.stderr: -367> 2022-09-03T06:50:46.120+0000 7fdd98754600 1 build_initial for_mkfs: 0 2022-09-03T06:50:46.403 INFO:teuthology.orchestra.run.smithi189.stderr: -366> 2022-09-03T06:50:46.120+0000 7fdd98754600 10 monclient: monmap: 2022-09-03T06:50:46.403 INFO:teuthology.orchestra.run.smithi189.stderr:epoch 0 2022-09-03T06:50:46.403 INFO:teuthology.orchestra.run.smithi189.stderr:fsid 00000000-0000-0000-0000-000000000000 2022-09-03T06:50:46.404 INFO:teuthology.orchestra.run.smithi189.stderr:last_changed 0.000000 2022-09-03T06:50:46.404 INFO:teuthology.orchestra.run.smithi189.stderr:created 0.000000 2022-09-03T06:50:46.404 INFO:teuthology.orchestra.run.smithi189.stderr:min_mon_release 0 (unknown) 2022-09-03T06:50:46.404 INFO:teuthology.orchestra.run.smithi189.stderr:election_strategy: 1 2022-09-03T06:50:46.405 INFO:teuthology.orchestra.run.smithi189.stderr:0: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.noname-a 2022-09-03T06:50:46.405 INFO:teuthology.orchestra.run.smithi189.stderr: 2022-09-03T06:50:46.405 INFO:teuthology.orchestra.run.smithi189.stderr: -365> 2022-09-03T06:50:46.120+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding auth protocol: cephx 2022-09-03T06:50:46.405 INFO:teuthology.orchestra.run.smithi189.stderr: -364> 2022-09-03T06:50:46.120+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding auth protocol: cephx 2022-09-03T06:50:46.406 INFO:teuthology.orchestra.run.smithi189.stderr: -363> 2022-09-03T06:50:46.120+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding auth protocol: cephx 2022-09-03T06:50:46.406 INFO:teuthology.orchestra.run.smithi189.stderr: -362> 2022-09-03T06:50:46.120+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: secure 2022-09-03T06:50:46.406 INFO:teuthology.orchestra.run.smithi189.stderr: -361> 2022-09-03T06:50:46.120+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: crc 2022-09-03T06:50:46.407 INFO:teuthology.orchestra.run.smithi189.stderr: -360> 2022-09-03T06:50:46.120+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: secure 2022-09-03T06:50:46.407 INFO:teuthology.orchestra.run.smithi189.stderr: -359> 2022-09-03T06:50:46.120+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: crc 2022-09-03T06:50:46.407 INFO:teuthology.orchestra.run.smithi189.stderr: -358> 2022-09-03T06:50:46.120+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: secure 2022-09-03T06:50:46.408 INFO:teuthology.orchestra.run.smithi189.stderr: -357> 2022-09-03T06:50:46.120+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: crc 2022-09-03T06:50:46.408 INFO:teuthology.orchestra.run.smithi189.stderr: -356> 2022-09-03T06:50:46.120+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: crc 2022-09-03T06:50:46.408 INFO:teuthology.orchestra.run.smithi189.stderr: -355> 2022-09-03T06:50:46.120+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: secure 2022-09-03T06:50:46.408 INFO:teuthology.orchestra.run.smithi189.stderr: -354> 2022-09-03T06:50:46.120+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: crc 2022-09-03T06:50:46.409 INFO:teuthology.orchestra.run.smithi189.stderr: -353> 2022-09-03T06:50:46.120+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: secure 2022-09-03T06:50:46.409 INFO:teuthology.orchestra.run.smithi189.stderr: -352> 2022-09-03T06:50:46.120+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: crc 2022-09-03T06:50:46.409 INFO:teuthology.orchestra.run.smithi189.stderr: -351> 2022-09-03T06:50:46.120+0000 7fdd98754600 5 AuthRegistry(0x565391916a40) adding con mode: secure 2022-09-03T06:50:46.410 INFO:teuthology.orchestra.run.smithi189.stderr: -350> 2022-09-03T06:50:46.120+0000 7fdd98754600 2 auth: KeyRing::load: loaded key file /etc/ceph/c2.client.0.keyring 2022-09-03T06:50:46.410 INFO:teuthology.orchestra.run.smithi189.stderr: -349> 2022-09-03T06:50:46.120+0000 7fdd98754600 1 librados: starting msgr at 2022-09-03T06:50:46.410 INFO:teuthology.orchestra.run.smithi189.stderr: -348> 2022-09-03T06:50:46.120+0000 7fdd98754600 1 librados: starting objecter 2022-09-03T06:50:46.410 INFO:teuthology.orchestra.run.smithi189.stderr: -347> 2022-09-03T06:50:46.121+0000 7fdd98754600 5 asok(0x565390c4a000) register_command objecter_requests hook 0x565390baab90 2022-09-03T06:50:46.411 INFO:teuthology.orchestra.run.smithi189.stderr: -346> 2022-09-03T06:50:46.121+0000 7fdd98754600 1 librados: setting wanted keys 2022-09-03T06:50:46.411 INFO:teuthology.orchestra.run.smithi189.stderr: -345> 2022-09-03T06:50:46.121+0000 7fdd98754600 1 librados: calling monclient init 2022-09-03T06:50:46.411 INFO:teuthology.orchestra.run.smithi189.stderr: -344> 2022-09-03T06:50:46.121+0000 7fdd98754600 10 monclient: init 2022-09-03T06:50:46.411 INFO:teuthology.orchestra.run.smithi189.stderr: -343> 2022-09-03T06:50:46.121+0000 7fdd98754600 5 AuthRegistry(0x565390c120d8) adding auth protocol: cephx 2022-09-03T06:50:46.412 INFO:teuthology.orchestra.run.smithi189.stderr: -342> 2022-09-03T06:50:46.121+0000 7fdd98754600 5 AuthRegistry(0x565390c120d8) adding auth protocol: cephx 2022-09-03T06:50:46.412 INFO:teuthology.orchestra.run.smithi189.stderr: -341> 2022-09-03T06:50:46.121+0000 7fdd98754600 5 AuthRegistry(0x565390c120d8) adding auth protocol: cephx 2022-09-03T06:50:46.412 INFO:teuthology.orchestra.run.smithi189.stderr: -340> 2022-09-03T06:50:46.121+0000 7fdd98754600 5 AuthRegistry(0x565390c120d8) adding con mode: secure 2022-09-03T06:50:46.412 INFO:teuthology.orchestra.run.smithi189.stderr: -339> 2022-09-03T06:50:46.121+0000 7fdd98754600 5 AuthRegistry(0x565390c120d8) adding con mode: crc 2022-09-03T06:50:46.413 INFO:teuthology.orchestra.run.smithi189.stderr: -338> 2022-09-03T06:50:46.121+0000 7fdd98754600 5 AuthRegistry(0x565390c120d8) adding con mode: secure 2022-09-03T06:50:46.413 INFO:teuthology.orchestra.run.smithi189.stderr: -337> 2022-09-03T06:50:46.121+0000 7fdd98754600 5 AuthRegistry(0x565390c120d8) adding con mode: crc 2022-09-03T06:50:46.413 INFO:teuthology.orchestra.run.smithi189.stderr: -336> 2022-09-03T06:50:46.121+0000 7fdd98754600 5 AuthRegistry(0x565390c120d8) adding con mode: secure 2022-09-03T06:50:46.414 INFO:teuthology.orchestra.run.smithi189.stderr: -335> 2022-09-03T06:50:46.121+0000 7fdd98754600 5 AuthRegistry(0x565390c120d8) adding con mode: crc 2022-09-03T06:50:46.414 INFO:teuthology.orchestra.run.smithi189.stderr: -334> 2022-09-03T06:50:46.121+0000 7fdd98754600 5 AuthRegistry(0x565390c120d8) adding con mode: crc 2022-09-03T06:50:46.414 INFO:teuthology.orchestra.run.smithi189.stderr: -333> 2022-09-03T06:50:46.121+0000 7fdd98754600 5 AuthRegistry(0x565390c120d8) adding con mode: secure 2022-09-03T06:50:46.415 INFO:teuthology.orchestra.run.smithi189.stderr: -332> 2022-09-03T06:50:46.121+0000 7fdd98754600 5 AuthRegistry(0x565390c120d8) adding con mode: crc 2022-09-03T06:50:46.415 INFO:teuthology.orchestra.run.smithi189.stderr: -331> 2022-09-03T06:50:46.121+0000 7fdd98754600 5 AuthRegistry(0x565390c120d8) adding con mode: secure 2022-09-03T06:50:46.415 INFO:teuthology.orchestra.run.smithi189.stderr: -330> 2022-09-03T06:50:46.121+0000 7fdd98754600 5 AuthRegistry(0x565390c120d8) adding con mode: crc 2022-09-03T06:50:46.416 INFO:teuthology.orchestra.run.smithi189.stderr: -329> 2022-09-03T06:50:46.121+0000 7fdd98754600 5 AuthRegistry(0x565390c120d8) adding con mode: secure 2022-09-03T06:50:46.416 INFO:teuthology.orchestra.run.smithi189.stderr: -328> 2022-09-03T06:50:46.121+0000 7fdd98754600 2 auth: KeyRing::load: loaded key file /etc/ceph/c2.client.0.keyring 2022-09-03T06:50:46.416 INFO:teuthology.orchestra.run.smithi189.stderr: -327> 2022-09-03T06:50:46.121+0000 7fdd98754600 2 auth: KeyRing::load: loaded key file /etc/ceph/c2.client.0.keyring 2022-09-03T06:50:46.416 INFO:teuthology.orchestra.run.smithi189.stderr: -326> 2022-09-03T06:50:46.121+0000 7fdd98754600 10 monclient: _reopen_session rank -1 2022-09-03T06:50:46.417 INFO:teuthology.orchestra.run.smithi189.stderr: -325> 2022-09-03T06:50:46.121+0000 7fdd98754600 10 monclient: _add_conns ranks=[0] 2022-09-03T06:50:46.417 INFO:teuthology.orchestra.run.smithi189.stderr: -324> 2022-09-03T06:50:46.121+0000 7fdd98754600 10 monclient(hunting): picked mon.noname-a con 0x565390bb6800 addr [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] 2022-09-03T06:50:46.417 INFO:teuthology.orchestra.run.smithi189.stderr: -323> 2022-09-03T06:50:46.121+0000 7fdd98754600 10 monclient(hunting): start opening mon connection 2022-09-03T06:50:46.417 INFO:teuthology.orchestra.run.smithi189.stderr: -322> 2022-09-03T06:50:46.121+0000 7fdd98754600 10 monclient(hunting): _renew_subs 2022-09-03T06:50:46.418 INFO:teuthology.orchestra.run.smithi189.stderr: -321> 2022-09-03T06:50:46.121+0000 7fdd98754600 10 monclient(hunting): authenticate will time out at 6734.537222s 2022-09-03T06:50:46.418 INFO:teuthology.orchestra.run.smithi189.stderr: -320> 2022-09-03T06:50:46.121+0000 7fdd96700700 10 monclient(hunting): get_auth_request con 0x565390bb6800 auth_method 0 2022-09-03T06:50:46.418 INFO:teuthology.orchestra.run.smithi189.stderr: -319> 2022-09-03T06:50:46.121+0000 7fdd96700700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2022-09-03T06:50:46.418 INFO:teuthology.orchestra.run.smithi189.stderr: -318> 2022-09-03T06:50:46.121+0000 7fdd96700700 10 monclient(hunting): _init_auth method 2 2022-09-03T06:50:46.419 INFO:teuthology.orchestra.run.smithi189.stderr: -317> 2022-09-03T06:50:46.121+0000 7fdd96700700 10 monclient(hunting): _init_auth creating new auth 2022-09-03T06:50:46.419 INFO:teuthology.orchestra.run.smithi189.stderr: -316> 2022-09-03T06:50:46.121+0000 7fdd96700700 10 monclient(hunting): handle_auth_reply_more payload 9 2022-09-03T06:50:46.419 INFO:teuthology.orchestra.run.smithi189.stderr: -315> 2022-09-03T06:50:46.121+0000 7fdd96700700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2022-09-03T06:50:46.420 INFO:teuthology.orchestra.run.smithi189.stderr: -314> 2022-09-03T06:50:46.121+0000 7fdd96700700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2022-09-03T06:50:46.420 INFO:teuthology.orchestra.run.smithi189.stderr: -313> 2022-09-03T06:50:46.121+0000 7fdd96700700 10 monclient(hunting): handle_auth_done global_id 6690 payload 824 2022-09-03T06:50:46.420 INFO:teuthology.orchestra.run.smithi189.stderr: -312> 2022-09-03T06:50:46.121+0000 7fdd96700700 10 monclient: _finish_hunting 0 2022-09-03T06:50:46.420 INFO:teuthology.orchestra.run.smithi189.stderr: -311> 2022-09-03T06:50:46.121+0000 7fdd96700700 1 monclient: found mon.noname-a 2022-09-03T06:50:46.421 INFO:teuthology.orchestra.run.smithi189.stderr: -310> 2022-09-03T06:50:46.121+0000 7fdd96700700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.189:3300/0 2022-09-03T06:50:46.421 INFO:teuthology.orchestra.run.smithi189.stderr: -309> 2022-09-03T06:50:46.121+0000 7fdd92ef9700 10 monclient: handle_monmap mon_map magic: 0 v1 2022-09-03T06:50:46.421 INFO:teuthology.orchestra.run.smithi189.stderr: -308> 2022-09-03T06:50:46.121+0000 7fdd92ef9700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2022-09-03T06:50:46.421 INFO:teuthology.orchestra.run.smithi189.stderr: -307> 2022-09-03T06:50:46.121+0000 7fdd92ef9700 10 monclient: dump: 2022-09-03T06:50:46.422 INFO:teuthology.orchestra.run.smithi189.stderr:epoch 1 2022-09-03T06:50:46.422 INFO:teuthology.orchestra.run.smithi189.stderr:fsid 08f0e03a-4e9e-4009-81f6-ef837e373516 2022-09-03T06:50:46.422 INFO:teuthology.orchestra.run.smithi189.stderr:last_changed 2022-09-03T05:16:00.054991+0000 2022-09-03T06:50:46.422 INFO:teuthology.orchestra.run.smithi189.stderr:created 2022-09-03T05:16:00.054991+0000 2022-09-03T06:50:46.423 INFO:teuthology.orchestra.run.smithi189.stderr:min_mon_release 17 (quincy) 2022-09-03T06:50:46.423 INFO:teuthology.orchestra.run.smithi189.stderr:election_strategy: 1 2022-09-03T06:50:46.423 INFO:teuthology.orchestra.run.smithi189.stderr:0: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.a 2022-09-03T06:50:46.423 INFO:teuthology.orchestra.run.smithi189.stderr: 2022-09-03T06:50:46.424 INFO:teuthology.orchestra.run.smithi189.stderr: -306> 2022-09-03T06:50:46.121+0000 7fdd92ef9700 10 monclient: _finish_auth 0 2022-09-03T06:50:46.424 INFO:teuthology.orchestra.run.smithi189.stderr: -305> 2022-09-03T06:50:46.121+0000 7fdd92ef9700 10 monclient: _check_auth_tickets 2022-09-03T06:50:46.424 INFO:teuthology.orchestra.run.smithi189.stderr: -304> 2022-09-03T06:50:46.121+0000 7fdd98754600 5 monclient: authenticate success, global_id 6690 2022-09-03T06:50:46.424 INFO:teuthology.orchestra.run.smithi189.stderr: -303> 2022-09-03T06:50:46.121+0000 7fdd98754600 10 monclient: _renew_subs 2022-09-03T06:50:46.425 INFO:teuthology.orchestra.run.smithi189.stderr: -302> 2022-09-03T06:50:46.121+0000 7fdd98754600 10 monclient: _send_mon_message to mon.a at v2:172.21.15.189:3300/0 2022-09-03T06:50:46.425 INFO:teuthology.orchestra.run.smithi189.stderr: -301> 2022-09-03T06:50:46.121+0000 7fdd92ef9700 10 monclient: handle_config config(0 keys) v1 2022-09-03T06:50:46.425 INFO:teuthology.orchestra.run.smithi189.stderr: -300> 2022-09-03T06:50:46.121+0000 7fdd92ef9700 10 monclient: handle_monmap mon_map magic: 0 v1 2022-09-03T06:50:46.426 INFO:teuthology.orchestra.run.smithi189.stderr: -299> 2022-09-03T06:50:46.121+0000 7fdd92ef9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2022-09-03T06:50:46.426 INFO:teuthology.orchestra.run.smithi189.stderr: -298> 2022-09-03T06:50:46.121+0000 7fdd946fc700 4 set_mon_vals no callback set 2022-09-03T06:50:46.426 INFO:teuthology.orchestra.run.smithi189.stderr: -297> 2022-09-03T06:50:46.121+0000 7fdd92ef9700 10 monclient: dump: 2022-09-03T06:50:46.427 INFO:teuthology.orchestra.run.smithi189.stderr:epoch 1 2022-09-03T06:50:46.427 INFO:teuthology.orchestra.run.smithi189.stderr:fsid 08f0e03a-4e9e-4009-81f6-ef837e373516 2022-09-03T06:50:46.427 INFO:teuthology.orchestra.run.smithi189.stderr:last_changed 2022-09-03T05:16:00.054991+0000 2022-09-03T06:50:46.427 INFO:teuthology.orchestra.run.smithi189.stderr:created 2022-09-03T05:16:00.054991+0000 2022-09-03T06:50:46.428 INFO:teuthology.orchestra.run.smithi189.stderr:min_mon_release 17 (quincy) 2022-09-03T06:50:46.428 INFO:teuthology.orchestra.run.smithi189.stderr:election_strategy: 1 2022-09-03T06:50:46.428 INFO:teuthology.orchestra.run.smithi189.stderr:0: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.a 2022-09-03T06:50:46.428 INFO:teuthology.orchestra.run.smithi189.stderr: 2022-09-03T06:50:46.429 INFO:teuthology.orchestra.run.smithi189.stderr: -296> 2022-09-03T06:50:46.121+0000 7fdd98754600 10 monclient: _renew_subs 2022-09-03T06:50:46.429 INFO:teuthology.orchestra.run.smithi189.stderr: -295> 2022-09-03T06:50:46.121+0000 7fdd98754600 10 monclient: _send_mon_message to mon.a at v2:172.21.15.189:3300/0 2022-09-03T06:50:46.429 INFO:teuthology.orchestra.run.smithi189.stderr: -294> 2022-09-03T06:50:46.121+0000 7fdd98754600 1 librados: init done 2022-09-03T06:50:46.429 INFO:teuthology.orchestra.run.smithi189.stderr: -293> 2022-09-03T06:50:46.122+0000 7fdd92ef9700 4 mgrc handle_mgr_map Got map version 5 2022-09-03T06:50:46.430 INFO:teuthology.orchestra.run.smithi189.stderr: -292> 2022-09-03T06:50:46.122+0000 7fdd92ef9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.189:6824/104033,v1:172.21.15.189:6825/104033] 2022-09-03T06:50:46.430 INFO:teuthology.orchestra.run.smithi189.stderr: -291> 2022-09-03T06:50:46.122+0000 7fdd92ef9700 4 mgrc reconnect Starting new session with [v2:172.21.15.189:6824/104033,v1:172.21.15.189:6825/104033] 2022-09-03T06:50:46.430 INFO:teuthology.orchestra.run.smithi189.stderr: -290> 2022-09-03T06:50:46.122+0000 7fdd95eff700 10 monclient: get_auth_request con 0x565390bb7400 auth_method 0 2022-09-03T06:50:46.431 INFO:teuthology.orchestra.run.smithi189.stderr: -289> 2022-09-03T06:50:46.122+0000 7fdd96f01700 10 monclient: get_auth_request con 0x565390bb6c00 auth_method 0 2022-09-03T06:50:46.431 INFO:teuthology.orchestra.run.smithi189.stderr: -288> 2022-09-03T06:50:46.123+0000 7fdd96700700 10 monclient: get_auth_request con 0x565390bb7c00 auth_method 0 2022-09-03T06:50:46.431 INFO:teuthology.orchestra.run.smithi189.stderr: -287> 2022-09-03T06:50:46.124+0000 7fdd95eff700 10 monclient: get_auth_request con 0x565390ede800 auth_method 0 2022-09-03T06:50:46.431 INFO:teuthology.orchestra.run.smithi189.stderr: -286> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf9e0) adding auth protocol: cephx 2022-09-03T06:50:46.432 INFO:teuthology.orchestra.run.smithi189.stderr: -285> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf9e0) adding auth protocol: cephx 2022-09-03T06:50:46.432 INFO:teuthology.orchestra.run.smithi189.stderr: -284> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf9e0) adding auth protocol: cephx 2022-09-03T06:50:46.432 INFO:teuthology.orchestra.run.smithi189.stderr: -283> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf9e0) adding con mode: secure 2022-09-03T06:50:46.432 INFO:teuthology.orchestra.run.smithi189.stderr: -282> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf9e0) adding con mode: crc 2022-09-03T06:50:46.432 INFO:teuthology.orchestra.run.smithi189.stderr: -281> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf9e0) adding con mode: secure 2022-09-03T06:50:46.433 INFO:teuthology.orchestra.run.smithi189.stderr: -280> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf9e0) adding con mode: crc 2022-09-03T06:50:46.433 INFO:teuthology.orchestra.run.smithi189.stderr: -279> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf9e0) adding con mode: secure 2022-09-03T06:50:46.433 INFO:teuthology.orchestra.run.smithi189.stderr: -278> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf9e0) adding con mode: crc 2022-09-03T06:50:46.433 INFO:teuthology.orchestra.run.smithi189.stderr: -277> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf9e0) adding con mode: crc 2022-09-03T06:50:46.434 INFO:teuthology.orchestra.run.smithi189.stderr: -276> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf9e0) adding con mode: secure 2022-09-03T06:50:46.434 INFO:teuthology.orchestra.run.smithi189.stderr: -275> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf9e0) adding con mode: crc 2022-09-03T06:50:46.434 INFO:teuthology.orchestra.run.smithi189.stderr: -274> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf9e0) adding con mode: secure 2022-09-03T06:50:46.434 INFO:teuthology.orchestra.run.smithi189.stderr: -273> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf9e0) adding con mode: crc 2022-09-03T06:50:46.435 INFO:teuthology.orchestra.run.smithi189.stderr: -272> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf9e0) adding con mode: secure 2022-09-03T06:50:46.435 INFO:teuthology.orchestra.run.smithi189.stderr: -271> 2022-09-03T06:50:46.150+0000 7fdd98754600 2 auth: KeyRing::load: loaded key file /etc/ceph/c2.client.0.keyring 2022-09-03T06:50:46.435 INFO:teuthology.orchestra.run.smithi189.stderr: -270> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 asok(0x565390c4a000) register_command cache list hook 0x565390baab60 2022-09-03T06:50:46.435 INFO:teuthology.orchestra.run.smithi189.stderr: -269> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 asok(0x565390c4a000) register_command cache inspect hook 0x565390baab60 2022-09-03T06:50:46.436 INFO:teuthology.orchestra.run.smithi189.stderr: -268> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 asok(0x565390c4a000) register_command cache erase hook 0x565390baab60 2022-09-03T06:50:46.436 INFO:teuthology.orchestra.run.smithi189.stderr: -267> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 asok(0x565390c4a000) register_command cache zap hook 0x565390baab60 2022-09-03T06:50:46.436 INFO:teuthology.orchestra.run.smithi189.stderr: -266> 2022-09-03T06:50:46.150+0000 7fdd98754600 10 monclient: get_monmap_and_config 2022-09-03T06:50:46.436 INFO:teuthology.orchestra.run.smithi189.stderr: -265> 2022-09-03T06:50:46.150+0000 7fdd98754600 10 monclient: build_initial_monmap 2022-09-03T06:50:46.437 INFO:teuthology.orchestra.run.smithi189.stderr: -264> 2022-09-03T06:50:46.150+0000 7fdd98754600 1 build_initial for_mkfs: 0 2022-09-03T06:50:46.437 INFO:teuthology.orchestra.run.smithi189.stderr: -263> 2022-09-03T06:50:46.150+0000 7fdd98754600 10 monclient: monmap: 2022-09-03T06:50:46.437 INFO:teuthology.orchestra.run.smithi189.stderr:epoch 0 2022-09-03T06:50:46.438 INFO:teuthology.orchestra.run.smithi189.stderr:fsid 00000000-0000-0000-0000-000000000000 2022-09-03T06:50:46.438 INFO:teuthology.orchestra.run.smithi189.stderr:last_changed 0.000000 2022-09-03T06:50:46.438 INFO:teuthology.orchestra.run.smithi189.stderr:created 0.000000 2022-09-03T06:50:46.438 INFO:teuthology.orchestra.run.smithi189.stderr:min_mon_release 0 (unknown) 2022-09-03T06:50:46.439 INFO:teuthology.orchestra.run.smithi189.stderr:election_strategy: 1 2022-09-03T06:50:46.439 INFO:teuthology.orchestra.run.smithi189.stderr:0: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.noname-a 2022-09-03T06:50:46.439 INFO:teuthology.orchestra.run.smithi189.stderr: 2022-09-03T06:50:46.439 INFO:teuthology.orchestra.run.smithi189.stderr: -262> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding auth protocol: cephx 2022-09-03T06:50:46.440 INFO:teuthology.orchestra.run.smithi189.stderr: -261> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding auth protocol: cephx 2022-09-03T06:50:46.440 INFO:teuthology.orchestra.run.smithi189.stderr: -260> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding auth protocol: cephx 2022-09-03T06:50:46.440 INFO:teuthology.orchestra.run.smithi189.stderr: -259> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: secure 2022-09-03T06:50:46.440 INFO:teuthology.orchestra.run.smithi189.stderr: -258> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: crc 2022-09-03T06:50:46.441 INFO:teuthology.orchestra.run.smithi189.stderr: -257> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: secure 2022-09-03T06:50:46.441 INFO:teuthology.orchestra.run.smithi189.stderr: -256> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: crc 2022-09-03T06:50:46.441 INFO:teuthology.orchestra.run.smithi189.stderr: -255> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: secure 2022-09-03T06:50:46.442 INFO:teuthology.orchestra.run.smithi189.stderr: -254> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: crc 2022-09-03T06:50:46.443 INFO:teuthology.orchestra.run.smithi189.stderr: -253> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: crc 2022-09-03T06:50:46.443 INFO:teuthology.orchestra.run.smithi189.stderr: -252> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: secure 2022-09-03T06:50:46.444 INFO:teuthology.orchestra.run.smithi189.stderr: -251> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: crc 2022-09-03T06:50:46.444 INFO:teuthology.orchestra.run.smithi189.stderr: -250> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: secure 2022-09-03T06:50:46.445 INFO:teuthology.orchestra.run.smithi189.stderr: -249> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: crc 2022-09-03T06:50:46.445 INFO:teuthology.orchestra.run.smithi189.stderr: -248> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: secure 2022-09-03T06:50:46.446 INFO:teuthology.orchestra.run.smithi189.stderr: -247> 2022-09-03T06:50:46.150+0000 7fdd98754600 2 auth: KeyRing::load: loaded key file /etc/ceph/c2.client.0.keyring 2022-09-03T06:50:46.446 INFO:teuthology.orchestra.run.smithi189.stderr: -246> 2022-09-03T06:50:46.150+0000 7fdd98754600 10 monclient: init 2022-09-03T06:50:46.447 INFO:teuthology.orchestra.run.smithi189.stderr: -245> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf610) adding auth protocol: cephx 2022-09-03T06:50:46.447 INFO:teuthology.orchestra.run.smithi189.stderr: -244> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf610) adding auth protocol: cephx 2022-09-03T06:50:46.448 INFO:teuthology.orchestra.run.smithi189.stderr: -243> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf610) adding auth protocol: cephx 2022-09-03T06:50:46.448 INFO:teuthology.orchestra.run.smithi189.stderr: -242> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf610) adding con mode: secure 2022-09-03T06:50:46.450 INFO:teuthology.orchestra.run.smithi189.stderr: -241> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf610) adding con mode: crc 2022-09-03T06:50:46.450 INFO:teuthology.orchestra.run.smithi189.stderr: -240> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf610) adding con mode: secure 2022-09-03T06:50:46.451 INFO:teuthology.orchestra.run.smithi189.stderr: -239> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf610) adding con mode: crc 2022-09-03T06:50:46.451 INFO:teuthology.orchestra.run.smithi189.stderr: -238> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf610) adding con mode: secure 2022-09-03T06:50:46.452 INFO:teuthology.orchestra.run.smithi189.stderr: -237> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf610) adding con mode: crc 2022-09-03T06:50:46.454 INFO:teuthology.orchestra.run.smithi189.stderr: -236> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf610) adding con mode: crc 2022-09-03T06:50:46.455 INFO:teuthology.orchestra.run.smithi189.stderr: -235> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf610) adding con mode: secure 2022-09-03T06:50:46.455 INFO:teuthology.orchestra.run.smithi189.stderr: -234> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf610) adding con mode: crc 2022-09-03T06:50:46.456 INFO:teuthology.orchestra.run.smithi189.stderr: -233> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf610) adding con mode: secure 2022-09-03T06:50:46.456 INFO:teuthology.orchestra.run.smithi189.stderr: -232> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf610) adding con mode: crc 2022-09-03T06:50:46.457 INFO:teuthology.orchestra.run.smithi189.stderr: -231> 2022-09-03T06:50:46.150+0000 7fdd98754600 5 AuthRegistry(0x7fffe8ecf610) adding con mode: secure 2022-09-03T06:50:46.457 INFO:teuthology.orchestra.run.smithi189.stderr: -230> 2022-09-03T06:50:46.150+0000 7fdd98754600 2 auth: KeyRing::load: loaded key file /etc/ceph/c2.client.0.keyring 2022-09-03T06:50:46.458 INFO:teuthology.orchestra.run.smithi189.stderr: -229> 2022-09-03T06:50:46.150+0000 7fdd98754600 2 auth: KeyRing::load: loaded key file /etc/ceph/c2.client.0.keyring 2022-09-03T06:50:46.458 INFO:teuthology.orchestra.run.smithi189.stderr: -228> 2022-09-03T06:50:46.150+0000 7fdd98754600 10 monclient: _reopen_session rank -1 2022-09-03T06:50:46.459 INFO:teuthology.orchestra.run.smithi189.stderr: -227> 2022-09-03T06:50:46.150+0000 7fdd98754600 10 monclient: _add_conns ranks=[0] 2022-09-03T06:50:46.460 INFO:teuthology.orchestra.run.smithi189.stderr: -226> 2022-09-03T06:50:46.150+0000 7fdd98754600 10 monclient(hunting): picked mon.noname-a con 0x565390f08800 addr [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] 2022-09-03T06:50:46.460 INFO:teuthology.orchestra.run.smithi189.stderr: -225> 2022-09-03T06:50:46.150+0000 7fdd98754600 10 monclient(hunting): start opening mon connection 2022-09-03T06:50:46.461 INFO:teuthology.orchestra.run.smithi189.stderr: -224> 2022-09-03T06:50:46.150+0000 7fdd98754600 10 monclient(hunting): _renew_subs 2022-09-03T06:50:46.461 INFO:teuthology.orchestra.run.smithi189.stderr: -223> 2022-09-03T06:50:46.150+0000 7fdd98754600 10 monclient(hunting): authenticate will time out at 6734.566987s 2022-09-03T06:50:46.462 INFO:teuthology.orchestra.run.smithi189.stderr: -222> 2022-09-03T06:50:46.151+0000 7fdd96700700 10 monclient(hunting): get_auth_request con 0x565390f08800 auth_method 0 2022-09-03T06:50:46.465 INFO:teuthology.orchestra.run.smithi189.stderr: -221> 2022-09-03T06:50:46.151+0000 7fdd96700700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2022-09-03T06:50:46.465 INFO:teuthology.orchestra.run.smithi189.stderr: -220> 2022-09-03T06:50:46.151+0000 7fdd96700700 10 monclient(hunting): _init_auth method 2 2022-09-03T06:50:46.466 INFO:teuthology.orchestra.run.smithi189.stderr: -219> 2022-09-03T06:50:46.151+0000 7fdd96700700 10 monclient(hunting): _init_auth creating new auth 2022-09-03T06:50:46.466 INFO:teuthology.orchestra.run.smithi189.stderr: -218> 2022-09-03T06:50:46.151+0000 7fdd96700700 10 monclient(hunting): handle_auth_reply_more payload 9 2022-09-03T06:50:46.467 INFO:teuthology.orchestra.run.smithi189.stderr: -217> 2022-09-03T06:50:46.151+0000 7fdd96700700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2022-09-03T06:50:46.467 INFO:teuthology.orchestra.run.smithi189.stderr: -216> 2022-09-03T06:50:46.151+0000 7fdd96700700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2022-09-03T06:50:46.467 INFO:teuthology.orchestra.run.smithi189.stderr: -215> 2022-09-03T06:50:46.151+0000 7fdd96700700 10 monclient(hunting): handle_auth_done global_id 6691 payload 274 2022-09-03T06:50:46.468 INFO:teuthology.orchestra.run.smithi189.stderr: -214> 2022-09-03T06:50:46.151+0000 7fdd96700700 10 monclient: _finish_hunting 0 2022-09-03T06:50:46.468 INFO:teuthology.orchestra.run.smithi189.stderr: -213> 2022-09-03T06:50:46.151+0000 7fdd96700700 1 monclient: found mon.noname-a 2022-09-03T06:50:46.468 INFO:teuthology.orchestra.run.smithi189.stderr: -212> 2022-09-03T06:50:46.151+0000 7fdd96700700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.189:3300/0 2022-09-03T06:50:46.468 INFO:teuthology.orchestra.run.smithi189.stderr: -211> 2022-09-03T06:50:46.151+0000 7fdd806d4700 10 monclient: handle_monmap mon_map magic: 0 v1 2022-09-03T06:50:46.469 INFO:teuthology.orchestra.run.smithi189.stderr: -210> 2022-09-03T06:50:46.151+0000 7fdd806d4700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2022-09-03T06:50:46.469 INFO:teuthology.orchestra.run.smithi189.stderr: -209> 2022-09-03T06:50:46.151+0000 7fdd806d4700 10 monclient: dump: 2022-09-03T06:50:46.469 INFO:teuthology.orchestra.run.smithi189.stderr:epoch 1 2022-09-03T06:50:46.470 INFO:teuthology.orchestra.run.smithi189.stderr:fsid 08f0e03a-4e9e-4009-81f6-ef837e373516 2022-09-03T06:50:46.470 INFO:teuthology.orchestra.run.smithi189.stderr:last_changed 2022-09-03T05:16:00.054991+0000 2022-09-03T06:50:46.471 INFO:teuthology.orchestra.run.smithi189.stderr:created 2022-09-03T05:16:00.054991+0000 2022-09-03T06:50:46.474 INFO:teuthology.orchestra.run.smithi189.stderr:min_mon_release 17 (quincy) 2022-09-03T06:50:46.475 INFO:teuthology.orchestra.run.smithi189.stderr:election_strategy: 1 2022-09-03T06:50:46.475 INFO:teuthology.orchestra.run.smithi189.stderr:0: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.a 2022-09-03T06:50:46.475 INFO:teuthology.orchestra.run.smithi189.stderr: 2022-09-03T06:50:46.476 INFO:teuthology.orchestra.run.smithi189.stderr: -208> 2022-09-03T06:50:46.151+0000 7fdd806d4700 10 monclient: _finish_auth 0 2022-09-03T06:50:46.476 INFO:teuthology.orchestra.run.smithi189.stderr: -207> 2022-09-03T06:50:46.151+0000 7fdd806d4700 10 monclient: _check_auth_tickets 2022-09-03T06:50:46.477 INFO:teuthology.orchestra.run.smithi189.stderr: -206> 2022-09-03T06:50:46.151+0000 7fdd98754600 5 monclient: authenticate success, global_id 6691 2022-09-03T06:50:46.477 INFO:teuthology.orchestra.run.smithi189.stderr: -205> 2022-09-03T06:50:46.151+0000 7fdd806d4700 10 monclient: handle_config config(0 keys) v1 2022-09-03T06:50:46.477 INFO:teuthology.orchestra.run.smithi189.stderr: -204> 2022-09-03T06:50:46.151+0000 7fdd806d4700 10 monclient: handle_monmap mon_map magic: 0 v1 2022-09-03T06:50:46.478 INFO:teuthology.orchestra.run.smithi189.stderr: -203> 2022-09-03T06:50:46.151+0000 7fdd806d4700 10 monclient: got monmap 1 from mon.a (according to old e1) 2022-09-03T06:50:46.479 INFO:teuthology.orchestra.run.smithi189.stderr: -202> 2022-09-03T06:50:46.151+0000 7fdd806d4700 10 monclient: dump: 2022-09-03T06:50:46.479 INFO:teuthology.orchestra.run.smithi189.stderr:epoch 1 2022-09-03T06:50:46.480 INFO:teuthology.orchestra.run.smithi189.stderr:fsid 08f0e03a-4e9e-4009-81f6-ef837e373516 2022-09-03T06:50:46.481 INFO:teuthology.orchestra.run.smithi189.stderr:last_changed 2022-09-03T05:16:00.054991+0000 2022-09-03T06:50:46.481 INFO:teuthology.orchestra.run.smithi189.stderr:created 2022-09-03T05:16:00.054991+0000 2022-09-03T06:50:46.482 INFO:teuthology.orchestra.run.smithi189.stderr:min_mon_release 17 (quincy) 2022-09-03T06:50:46.483 INFO:teuthology.orchestra.run.smithi189.stderr:election_strategy: 1 2022-09-03T06:50:46.484 INFO:teuthology.orchestra.run.smithi189.stderr:0: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.a 2022-09-03T06:50:46.485 INFO:teuthology.orchestra.run.smithi189.stderr: 2022-09-03T06:50:46.485 INFO:teuthology.orchestra.run.smithi189.stderr: -201> 2022-09-03T06:50:46.151+0000 7fdd98754600 10 monclient: get_monmap_and_config success 2022-09-03T06:50:46.486 INFO:teuthology.orchestra.run.smithi189.stderr: -200> 2022-09-03T06:50:46.151+0000 7fdd98754600 4 set_mon_vals no callback set 2022-09-03T06:50:46.486 INFO:teuthology.orchestra.run.smithi189.stderr: -199> 2022-09-03T06:50:46.152+0000 7fdd98754600 10 monclient: shutdown 2022-09-03T06:50:46.487 INFO:teuthology.orchestra.run.smithi189.stderr: -198> 2022-09-03T06:50:46.152+0000 7fdd98754600 10 monclient: build_initial_monmap 2022-09-03T06:50:46.488 INFO:teuthology.orchestra.run.smithi189.stderr: -197> 2022-09-03T06:50:46.152+0000 7fdd98754600 1 build_initial for_mkfs: 0 2022-09-03T06:50:46.489 INFO:teuthology.orchestra.run.smithi189.stderr: -196> 2022-09-03T06:50:46.152+0000 7fdd98754600 10 monclient: monmap: 2022-09-03T06:50:46.490 INFO:teuthology.orchestra.run.smithi189.stderr:epoch 0 2022-09-03T06:50:46.491 INFO:teuthology.orchestra.run.smithi189.stderr:fsid 00000000-0000-0000-0000-000000000000 2022-09-03T06:50:46.491 INFO:teuthology.orchestra.run.smithi189.stderr:last_changed 0.000000 2022-09-03T06:50:46.492 INFO:teuthology.orchestra.run.smithi189.stderr:created 0.000000 2022-09-03T06:50:46.492 INFO:teuthology.orchestra.run.smithi189.stderr:min_mon_release 0 (unknown) 2022-09-03T06:50:46.493 INFO:teuthology.orchestra.run.smithi189.stderr:election_strategy: 1 2022-09-03T06:50:46.493 INFO:teuthology.orchestra.run.smithi189.stderr:0: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.noname-a 2022-09-03T06:50:46.493 INFO:teuthology.orchestra.run.smithi189.stderr: 2022-09-03T06:50:46.493 INFO:teuthology.orchestra.run.smithi189.stderr: -195> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding auth protocol: cephx 2022-09-03T06:50:46.494 INFO:teuthology.orchestra.run.smithi189.stderr: -194> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding auth protocol: cephx 2022-09-03T06:50:46.494 INFO:teuthology.orchestra.run.smithi189.stderr: -193> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding auth protocol: cephx 2022-09-03T06:50:46.495 INFO:teuthology.orchestra.run.smithi189.stderr: -192> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: secure 2022-09-03T06:50:46.495 INFO:teuthology.orchestra.run.smithi189.stderr: -191> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: crc 2022-09-03T06:50:46.495 INFO:teuthology.orchestra.run.smithi189.stderr: -190> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: secure 2022-09-03T06:50:46.496 INFO:teuthology.orchestra.run.smithi189.stderr: -189> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: crc 2022-09-03T06:50:46.496 INFO:teuthology.orchestra.run.smithi189.stderr: -188> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: secure 2022-09-03T06:50:46.496 INFO:teuthology.orchestra.run.smithi189.stderr: -187> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: crc 2022-09-03T06:50:46.497 INFO:teuthology.orchestra.run.smithi189.stderr: -186> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: crc 2022-09-03T06:50:46.497 INFO:teuthology.orchestra.run.smithi189.stderr: -185> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: secure 2022-09-03T06:50:46.498 INFO:teuthology.orchestra.run.smithi189.stderr: -184> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: crc 2022-09-03T06:50:46.498 INFO:teuthology.orchestra.run.smithi189.stderr: -183> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: secure 2022-09-03T06:50:46.498 INFO:teuthology.orchestra.run.smithi189.stderr: -182> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: crc 2022-09-03T06:50:46.498 INFO:teuthology.orchestra.run.smithi189.stderr: -181> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565391916140) adding con mode: secure 2022-09-03T06:50:46.499 INFO:teuthology.orchestra.run.smithi189.stderr: -180> 2022-09-03T06:50:46.152+0000 7fdd98754600 2 auth: KeyRing::load: loaded key file /etc/ceph/c2.client.0.keyring 2022-09-03T06:50:46.499 INFO:teuthology.orchestra.run.smithi189.stderr: -179> 2022-09-03T06:50:46.152+0000 7fdd98754600 1 librados: starting msgr at 2022-09-03T06:50:46.499 INFO:teuthology.orchestra.run.smithi189.stderr: -178> 2022-09-03T06:50:46.152+0000 7fdd98754600 1 librados: starting objecter 2022-09-03T06:50:46.500 INFO:teuthology.orchestra.run.smithi189.stderr: -177> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 asok(0x565390c4a000) register_command objecter_requests cmddesc objecter_requests hook 0x565390baa730 EEXIST 2022-09-03T06:50:46.500 INFO:teuthology.orchestra.run.smithi189.stderr: -176> 2022-09-03T06:50:46.152+0000 7fdd98754600 1 librados: setting wanted keys 2022-09-03T06:50:46.500 INFO:teuthology.orchestra.run.smithi189.stderr: -175> 2022-09-03T06:50:46.152+0000 7fdd98754600 1 librados: calling monclient init 2022-09-03T06:50:46.501 INFO:teuthology.orchestra.run.smithi189.stderr: -174> 2022-09-03T06:50:46.152+0000 7fdd98754600 10 monclient: init 2022-09-03T06:50:46.501 INFO:teuthology.orchestra.run.smithi189.stderr: -173> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565390ed20d8) adding auth protocol: cephx 2022-09-03T06:50:46.502 INFO:teuthology.orchestra.run.smithi189.stderr: -172> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565390ed20d8) adding auth protocol: cephx 2022-09-03T06:50:46.502 INFO:teuthology.orchestra.run.smithi189.stderr: -171> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565390ed20d8) adding auth protocol: cephx 2022-09-03T06:50:46.502 INFO:teuthology.orchestra.run.smithi189.stderr: -170> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565390ed20d8) adding con mode: secure 2022-09-03T06:50:46.503 INFO:teuthology.orchestra.run.smithi189.stderr: -169> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565390ed20d8) adding con mode: crc 2022-09-03T06:50:46.503 INFO:teuthology.orchestra.run.smithi189.stderr: -168> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565390ed20d8) adding con mode: secure 2022-09-03T06:50:46.503 INFO:teuthology.orchestra.run.smithi189.stderr: -167> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565390ed20d8) adding con mode: crc 2022-09-03T06:50:46.504 INFO:teuthology.orchestra.run.smithi189.stderr: -166> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565390ed20d8) adding con mode: secure 2022-09-03T06:50:46.504 INFO:teuthology.orchestra.run.smithi189.stderr: -165> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565390ed20d8) adding con mode: crc 2022-09-03T06:50:46.504 INFO:teuthology.orchestra.run.smithi189.stderr: -164> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565390ed20d8) adding con mode: crc 2022-09-03T06:50:46.504 INFO:teuthology.orchestra.run.smithi189.stderr: -163> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565390ed20d8) adding con mode: secure 2022-09-03T06:50:46.505 INFO:teuthology.orchestra.run.smithi189.stderr: -162> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565390ed20d8) adding con mode: crc 2022-09-03T06:50:46.505 INFO:teuthology.orchestra.run.smithi189.stderr: -161> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565390ed20d8) adding con mode: secure 2022-09-03T06:50:46.506 INFO:teuthology.orchestra.run.smithi189.stderr: -160> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565390ed20d8) adding con mode: crc 2022-09-03T06:50:46.506 INFO:teuthology.orchestra.run.smithi189.stderr: -159> 2022-09-03T06:50:46.152+0000 7fdd98754600 5 AuthRegistry(0x565390ed20d8) adding con mode: secure 2022-09-03T06:50:46.506 INFO:teuthology.orchestra.run.smithi189.stderr: -158> 2022-09-03T06:50:46.152+0000 7fdd98754600 2 auth: KeyRing::load: loaded key file /etc/ceph/c2.client.0.keyring 2022-09-03T06:50:46.507 INFO:teuthology.orchestra.run.smithi189.stderr: -157> 2022-09-03T06:50:46.152+0000 7fdd98754600 2 auth: KeyRing::load: loaded key file /etc/ceph/c2.client.0.keyring 2022-09-03T06:50:46.507 INFO:teuthology.orchestra.run.smithi189.stderr: -156> 2022-09-03T06:50:46.152+0000 7fdd98754600 10 monclient: _reopen_session rank -1 2022-09-03T06:50:46.507 INFO:teuthology.orchestra.run.smithi189.stderr: -155> 2022-09-03T06:50:46.152+0000 7fdd98754600 10 monclient: _add_conns ranks=[0] 2022-09-03T06:50:46.507 INFO:teuthology.orchestra.run.smithi189.stderr: -154> 2022-09-03T06:50:46.152+0000 7fdd98754600 10 monclient(hunting): picked mon.noname-a con 0x565390f08800 addr [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] 2022-09-03T06:50:46.508 INFO:teuthology.orchestra.run.smithi189.stderr: -153> 2022-09-03T06:50:46.152+0000 7fdd98754600 10 monclient(hunting): start opening mon connection 2022-09-03T06:50:46.508 INFO:teuthology.orchestra.run.smithi189.stderr: -152> 2022-09-03T06:50:46.152+0000 7fdd98754600 10 monclient(hunting): _renew_subs 2022-09-03T06:50:46.509 INFO:teuthology.orchestra.run.smithi189.stderr: -151> 2022-09-03T06:50:46.152+0000 7fdd98754600 10 monclient(hunting): authenticate will time out at 6734.568693s 2022-09-03T06:50:46.509 INFO:teuthology.orchestra.run.smithi189.stderr: -150> 2022-09-03T06:50:46.152+0000 7fdd96700700 10 monclient(hunting): get_auth_request con 0x565390f08800 auth_method 0 2022-09-03T06:50:46.509 INFO:teuthology.orchestra.run.smithi189.stderr: -149> 2022-09-03T06:50:46.152+0000 7fdd96700700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2022-09-03T06:50:46.510 INFO:teuthology.orchestra.run.smithi189.stderr: -148> 2022-09-03T06:50:46.152+0000 7fdd96700700 10 monclient(hunting): _init_auth method 2 2022-09-03T06:50:46.510 INFO:teuthology.orchestra.run.smithi189.stderr: -147> 2022-09-03T06:50:46.152+0000 7fdd96700700 10 monclient(hunting): _init_auth creating new auth 2022-09-03T06:50:46.511 INFO:teuthology.orchestra.run.smithi189.stderr: -146> 2022-09-03T06:50:46.152+0000 7fdd96700700 10 monclient(hunting): handle_auth_reply_more payload 9 2022-09-03T06:50:46.511 INFO:teuthology.orchestra.run.smithi189.stderr: -145> 2022-09-03T06:50:46.152+0000 7fdd96700700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2022-09-03T06:50:46.511 INFO:teuthology.orchestra.run.smithi189.stderr: -144> 2022-09-03T06:50:46.152+0000 7fdd96700700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2022-09-03T06:50:46.512 INFO:teuthology.orchestra.run.smithi189.stderr: -143> 2022-09-03T06:50:46.153+0000 7fdd96700700 10 monclient(hunting): handle_auth_done global_id 6692 payload 824 2022-09-03T06:50:46.512 INFO:teuthology.orchestra.run.smithi189.stderr: -142> 2022-09-03T06:50:46.153+0000 7fdd96700700 10 monclient: _finish_hunting 0 2022-09-03T06:50:46.513 INFO:teuthology.orchestra.run.smithi189.stderr: -141> 2022-09-03T06:50:46.153+0000 7fdd96700700 1 monclient: found mon.noname-a 2022-09-03T06:50:46.513 INFO:teuthology.orchestra.run.smithi189.stderr: -140> 2022-09-03T06:50:46.153+0000 7fdd96700700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.189:3300/0 2022-09-03T06:50:46.514 INFO:teuthology.orchestra.run.smithi189.stderr: -139> 2022-09-03T06:50:46.153+0000 7fdd7eed1700 10 monclient: handle_monmap mon_map magic: 0 v1 2022-09-03T06:50:46.514 INFO:teuthology.orchestra.run.smithi189.stderr: -138> 2022-09-03T06:50:46.153+0000 7fdd7eed1700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2022-09-03T06:50:46.515 INFO:teuthology.orchestra.run.smithi189.stderr: -137> 2022-09-03T06:50:46.153+0000 7fdd7eed1700 10 monclient: dump: 2022-09-03T06:50:46.515 INFO:teuthology.orchestra.run.smithi189.stderr:epoch 1 2022-09-03T06:50:46.516 INFO:teuthology.orchestra.run.smithi189.stderr:fsid 08f0e03a-4e9e-4009-81f6-ef837e373516 2022-09-03T06:50:46.516 INFO:teuthology.orchestra.run.smithi189.stderr:last_changed 2022-09-03T05:16:00.054991+0000 2022-09-03T06:50:46.517 INFO:teuthology.orchestra.run.smithi189.stderr:created 2022-09-03T05:16:00.054991+0000 2022-09-03T06:50:46.518 INFO:teuthology.orchestra.run.smithi189.stderr:min_mon_release 17 (quincy) 2022-09-03T06:50:46.520 INFO:teuthology.orchestra.run.smithi189.stderr:election_strategy: 1 2022-09-03T06:50:46.521 INFO:teuthology.orchestra.run.smithi189.stderr:0: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.a 2022-09-03T06:50:46.522 INFO:teuthology.orchestra.run.smithi189.stderr: 2022-09-03T06:50:46.523 INFO:teuthology.orchestra.run.smithi189.stderr: -136> 2022-09-03T06:50:46.153+0000 7fdd7eed1700 10 monclient: _finish_auth 0 2022-09-03T06:50:46.524 INFO:teuthology.orchestra.run.smithi189.stderr: -135> 2022-09-03T06:50:46.153+0000 7fdd7eed1700 10 monclient: _check_auth_tickets 2022-09-03T06:50:46.524 INFO:teuthology.orchestra.run.smithi189.stderr: -134> 2022-09-03T06:50:46.153+0000 7fdd98754600 5 monclient: authenticate success, global_id 6692 2022-09-03T06:50:46.525 INFO:teuthology.orchestra.run.smithi189.stderr: -133> 2022-09-03T06:50:46.153+0000 7fdd98754600 10 monclient: _renew_subs 2022-09-03T06:50:46.525 INFO:teuthology.orchestra.run.smithi189.stderr: -132> 2022-09-03T06:50:46.153+0000 7fdd98754600 10 monclient: _send_mon_message to mon.a at v2:172.21.15.189:3300/0 2022-09-03T06:50:46.526 INFO:teuthology.orchestra.run.smithi189.stderr: -131> 2022-09-03T06:50:46.153+0000 7fdd7eed1700 10 monclient: handle_config config(0 keys) v1 2022-09-03T06:50:46.526 INFO:teuthology.orchestra.run.smithi189.stderr: -130> 2022-09-03T06:50:46.153+0000 7fdd7eed1700 10 monclient: handle_monmap mon_map magic: 0 v1 2022-09-03T06:50:46.527 INFO:teuthology.orchestra.run.smithi189.stderr: -129> 2022-09-03T06:50:46.153+0000 7fdd7eed1700 10 monclient: got monmap 1 from mon.a (according to old e1) 2022-09-03T06:50:46.527 INFO:teuthology.orchestra.run.smithi189.stderr: -128> 2022-09-03T06:50:46.153+0000 7fdd7eed1700 10 monclient: dump: 2022-09-03T06:50:46.528 INFO:teuthology.orchestra.run.smithi189.stderr:epoch 1 2022-09-03T06:50:46.528 INFO:teuthology.orchestra.run.smithi189.stderr:fsid 08f0e03a-4e9e-4009-81f6-ef837e373516 2022-09-03T06:50:46.528 INFO:teuthology.orchestra.run.smithi189.stderr:last_changed 2022-09-03T05:16:00.054991+0000 2022-09-03T06:50:46.529 INFO:teuthology.orchestra.run.smithi189.stderr:created 2022-09-03T05:16:00.054991+0000 2022-09-03T06:50:46.529 INFO:teuthology.orchestra.run.smithi189.stderr:min_mon_release 17 (quincy) 2022-09-03T06:50:46.530 INFO:teuthology.orchestra.run.smithi189.stderr:election_strategy: 1 2022-09-03T06:50:46.531 INFO:teuthology.orchestra.run.smithi189.stderr:0: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.a 2022-09-03T06:50:46.532 INFO:teuthology.orchestra.run.smithi189.stderr: 2022-09-03T06:50:46.533 INFO:teuthology.orchestra.run.smithi189.stderr: -127> 2022-09-03T06:50:46.153+0000 7fdd806d4700 4 set_mon_vals no callback set 2022-09-03T06:50:46.533 INFO:teuthology.orchestra.run.smithi189.stderr: -126> 2022-09-03T06:50:46.153+0000 7fdd98754600 10 monclient: _renew_subs 2022-09-03T06:50:46.534 INFO:teuthology.orchestra.run.smithi189.stderr: -125> 2022-09-03T06:50:46.153+0000 7fdd98754600 10 monclient: _send_mon_message to mon.a at v2:172.21.15.189:3300/0 2022-09-03T06:50:46.535 INFO:teuthology.orchestra.run.smithi189.stderr: -124> 2022-09-03T06:50:46.153+0000 7fdd98754600 1 librados: init done 2022-09-03T06:50:46.536 INFO:teuthology.orchestra.run.smithi189.stderr: -123> 2022-09-03T06:50:46.153+0000 7fdd98754600 5 asok(0x565390c4a000) register_command cr dump hook 0x56539192add8 2022-09-03T06:50:46.536 INFO:teuthology.orchestra.run.smithi189.stderr: -122> 2022-09-03T06:50:46.154+0000 7fdd7eed1700 4 mgrc handle_mgr_map Got map version 5 2022-09-03T06:50:46.537 INFO:teuthology.orchestra.run.smithi189.stderr: -121> 2022-09-03T06:50:46.154+0000 7fdd7eed1700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.189:6824/104033,v1:172.21.15.189:6825/104033] 2022-09-03T06:50:46.538 INFO:teuthology.orchestra.run.smithi189.stderr: -120> 2022-09-03T06:50:46.154+0000 7fdd7eed1700 4 mgrc reconnect Starting new session with [v2:172.21.15.189:6824/104033,v1:172.21.15.189:6825/104033] 2022-09-03T06:50:46.539 INFO:teuthology.orchestra.run.smithi189.stderr: -119> 2022-09-03T06:50:46.154+0000 7fdd95eff700 10 monclient: get_auth_request con 0x565390f6b400 auth_method 0 2022-09-03T06:50:46.539 INFO:teuthology.orchestra.run.smithi189.stderr: -118> 2022-09-03T06:50:46.154+0000 7fdd96f01700 10 monclient: get_auth_request con 0x565390f09400 auth_method 0 2022-09-03T06:50:46.540 INFO:teuthology.orchestra.run.smithi189.stderr: -117> 2022-09-03T06:50:46.156+0000 7fdd96700700 10 monclient: get_auth_request con 0x565390edf800 auth_method 0 2022-09-03T06:50:46.540 INFO:teuthology.orchestra.run.smithi189.stderr: -116> 2022-09-03T06:50:46.159+0000 7fdd95eff700 10 monclient: get_auth_request con 0x565390ede400 auth_method 0 2022-09-03T06:50:46.540 INFO:teuthology.orchestra.run.smithi189.stderr: -115> 2022-09-03T06:50:46.194+0000 7fdd98754600 5 asok(0x565390c4a000) register_command sync trace show hook 0x565390bdf420 2022-09-03T06:50:46.541 INFO:teuthology.orchestra.run.smithi189.stderr: -114> 2022-09-03T06:50:46.194+0000 7fdd98754600 5 asok(0x565390c4a000) register_command sync trace history hook 0x565390bdf420 2022-09-03T06:50:46.542 INFO:teuthology.orchestra.run.smithi189.stderr: -113> 2022-09-03T06:50:46.194+0000 7fdd98754600 5 asok(0x565390c4a000) register_command sync trace active hook 0x565390bdf420 2022-09-03T06:50:46.542 INFO:teuthology.orchestra.run.smithi189.stderr: -112> 2022-09-03T06:50:46.194+0000 7fdd98754600 5 asok(0x565390c4a000) register_command sync trace active_short hook 0x565390bdf420 2022-09-03T06:50:46.543 INFO:teuthology.orchestra.run.smithi189.stderr: -111> 2022-09-03T06:50:46.197+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: sync status for source bucket: init. lease is: not taken. stop indications is: 0 2022-09-03T06:50:46.543 INFO:teuthology.orchestra.run.smithi189.stderr: -110> 2022-09-03T06:50:46.201+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: waiting on bucket lease 2022-09-03T06:50:46.544 INFO:teuthology.orchestra.run.smithi189.stderr: -109> 2022-09-03T06:50:46.203+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: status after acquiring the lock is: init 2022-09-03T06:50:46.544 INFO:teuthology.orchestra.run.smithi189.stderr: -108> 2022-09-03T06:50:46.223+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: listing bucket for full sync 2022-09-03T06:50:46.545 INFO:teuthology.orchestra.run.smithi189.stderr: -107> 2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj11 2022-09-03T06:50:46.545 INFO:teuthology.orchestra.run.smithi189.stderr: -106> 2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj11]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj11 op=0 op_state=1 2022-09-03T06:50:46.545 INFO:teuthology.orchestra.run.smithi189.stderr: -105> 2022-09-03T06:50:46.269+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj11]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj11[0] 2022-09-03T06:50:46.546 INFO:teuthology.orchestra.run.smithi189.stderr: -104> 2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj111 2022-09-03T06:50:46.546 INFO:teuthology.orchestra.run.smithi189.stderr: -103> 2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj111]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj111 op=0 op_state=1 2022-09-03T06:50:46.546 INFO:teuthology.orchestra.run.smithi189.stderr: -102> 2022-09-03T06:50:46.269+0000 7fdd98754600 0 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj111]: : injecting data sync error on key=obj111 2022-09-03T06:50:46.546 INFO:teuthology.orchestra.run.smithi189.stderr: -101> 2022-09-03T06:50:46.269+0000 7fdd98754600 10 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj111]: failed, retcode=-5 ((5) Input/output error) 2022-09-03T06:50:46.547 INFO:teuthology.orchestra.run.smithi189.stderr: -100> 2022-09-03T06:50:46.269+0000 7fdd98754600 0 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj111]: ERROR: failed to sync object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj111 2022-09-03T06:50:46.547 INFO:teuthology.orchestra.run.smithi189.stderr: -99> 2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj115 2022-09-03T06:50:46.547 INFO:teuthology.orchestra.run.smithi189.stderr: -98> 2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj115]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj115 op=0 op_state=1 2022-09-03T06:50:46.548 INFO:teuthology.orchestra.run.smithi189.stderr: -97> 2022-09-03T06:50:46.269+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj115]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj115[0] 2022-09-03T06:50:46.548 INFO:teuthology.orchestra.run.smithi189.stderr: -96> 2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj116 2022-09-03T06:50:46.548 INFO:teuthology.orchestra.run.smithi189.stderr: -95> 2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj116]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj116 op=0 op_state=1 2022-09-03T06:50:46.549 INFO:teuthology.orchestra.run.smithi189.stderr: -94> 2022-09-03T06:50:46.269+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj116]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj116[0] 2022-09-03T06:50:46.549 INFO:teuthology.orchestra.run.smithi189.stderr: -93> 2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj124 2022-09-03T06:50:46.549 INFO:teuthology.orchestra.run.smithi189.stderr: -92> 2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj124]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj124 op=0 op_state=1 2022-09-03T06:50:46.550 INFO:teuthology.orchestra.run.smithi189.stderr: -91> 2022-09-03T06:50:46.269+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj124]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj124[0] 2022-09-03T06:50:46.550 INFO:teuthology.orchestra.run.smithi189.stderr: -90> 2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj145 2022-09-03T06:50:46.550 INFO:teuthology.orchestra.run.smithi189.stderr: -89> 2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj145]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj145 op=0 op_state=1 2022-09-03T06:50:46.550 INFO:teuthology.orchestra.run.smithi189.stderr: -88> 2022-09-03T06:50:46.269+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj145]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj145[0] 2022-09-03T06:50:46.551 INFO:teuthology.orchestra.run.smithi189.stderr: -87> 2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj148 2022-09-03T06:50:46.551 INFO:teuthology.orchestra.run.smithi189.stderr: -86> 2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj148]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj148 op=0 op_state=1 2022-09-03T06:50:46.551 INFO:teuthology.orchestra.run.smithi189.stderr: -85> 2022-09-03T06:50:46.269+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj148]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj148[0] 2022-09-03T06:50:46.552 INFO:teuthology.orchestra.run.smithi189.stderr: -84> 2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj155 2022-09-03T06:50:46.552 INFO:teuthology.orchestra.run.smithi189.stderr: -83> 2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj155]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj155 op=0 op_state=1 2022-09-03T06:50:46.552 INFO:teuthology.orchestra.run.smithi189.stderr: -82> 2022-09-03T06:50:46.269+0000 7fdd98754600 0 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj155]: : injecting data sync error on key=obj155 2022-09-03T06:50:46.552 INFO:teuthology.orchestra.run.smithi189.stderr: -81> 2022-09-03T06:50:46.269+0000 7fdd98754600 10 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj155]: failed, retcode=-5 ((5) Input/output error) 2022-09-03T06:50:46.553 INFO:teuthology.orchestra.run.smithi189.stderr: -80> 2022-09-03T06:50:46.269+0000 7fdd98754600 0 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj155]: ERROR: failed to sync object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj155 2022-09-03T06:50:46.553 INFO:teuthology.orchestra.run.smithi189.stderr: -79> 2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj17 2022-09-03T06:50:46.553 INFO:teuthology.orchestra.run.smithi189.stderr: -78> 2022-09-03T06:50:46.269+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj17]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj17 op=0 op_state=1 2022-09-03T06:50:46.554 INFO:teuthology.orchestra.run.smithi189.stderr: -77> 2022-09-03T06:50:46.269+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj17]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj17[0] 2022-09-03T06:50:46.554 INFO:teuthology.orchestra.run.smithi189.stderr: -76> 2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj185 2022-09-03T06:50:46.564 INFO:teuthology.orchestra.run.smithi189.stderr: -75> 2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj185]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj185 op=0 op_state=1 2022-09-03T06:50:46.565 INFO:teuthology.orchestra.run.smithi189.stderr: -74> 2022-09-03T06:50:46.270+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj185]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj185[0] 2022-09-03T06:50:46.565 INFO:teuthology.orchestra.run.smithi189.stderr: -73> 2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj19 2022-09-03T06:50:46.566 INFO:teuthology.orchestra.run.smithi189.stderr: -72> 2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj19]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj19 op=0 op_state=1 2022-09-03T06:50:46.566 INFO:teuthology.orchestra.run.smithi189.stderr: -71> 2022-09-03T06:50:46.270+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj19]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj19[0] 2022-09-03T06:50:46.566 INFO:teuthology.orchestra.run.smithi189.stderr: -70> 2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj22 2022-09-03T06:50:46.567 INFO:teuthology.orchestra.run.smithi189.stderr: -69> 2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj22]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj22 op=0 op_state=1 2022-09-03T06:50:46.567 INFO:teuthology.orchestra.run.smithi189.stderr: -68> 2022-09-03T06:50:46.270+0000 7fdd98754600 0 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj22]: : injecting data sync error on key=obj22 2022-09-03T06:50:46.567 INFO:teuthology.orchestra.run.smithi189.stderr: -67> 2022-09-03T06:50:46.270+0000 7fdd98754600 10 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj22]: failed, retcode=-5 ((5) Input/output error) 2022-09-03T06:50:46.568 INFO:teuthology.orchestra.run.smithi189.stderr: -66> 2022-09-03T06:50:46.270+0000 7fdd98754600 0 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj22]: ERROR: failed to sync object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj22 2022-09-03T06:50:46.568 INFO:teuthology.orchestra.run.smithi189.stderr: -65> 2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj23 2022-09-03T06:50:46.568 INFO:teuthology.orchestra.run.smithi189.stderr: -64> 2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj23]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj23 op=0 op_state=1 2022-09-03T06:50:46.569 INFO:teuthology.orchestra.run.smithi189.stderr: -63> 2022-09-03T06:50:46.270+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj23]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj23[0] 2022-09-03T06:50:46.569 INFO:teuthology.orchestra.run.smithi189.stderr: -62> 2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj29 2022-09-03T06:50:46.570 INFO:teuthology.orchestra.run.smithi189.stderr: -61> 2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj29]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj29 op=0 op_state=1 2022-09-03T06:50:46.570 INFO:teuthology.orchestra.run.smithi189.stderr: -60> 2022-09-03T06:50:46.270+0000 7fdd98754600 0 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj29]: : injecting data sync error on key=obj29 2022-09-03T06:50:46.570 INFO:teuthology.orchestra.run.smithi189.stderr: -59> 2022-09-03T06:50:46.270+0000 7fdd98754600 10 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj29]: failed, retcode=-5 ((5) Input/output error) 2022-09-03T06:50:46.571 INFO:teuthology.orchestra.run.smithi189.stderr: -58> 2022-09-03T06:50:46.270+0000 7fdd98754600 0 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj29]: ERROR: failed to sync object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj29 2022-09-03T06:50:46.571 INFO:teuthology.orchestra.run.smithi189.stderr: -57> 2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj31 2022-09-03T06:50:46.571 INFO:teuthology.orchestra.run.smithi189.stderr: -56> 2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj31]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj31 op=0 op_state=1 2022-09-03T06:50:46.572 INFO:teuthology.orchestra.run.smithi189.stderr: -55> 2022-09-03T06:50:46.270+0000 7fdd98754600 5 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj31]: bucket sync: sync obj: 91786ef1-cdb4-4061-8a3e-2a28cc497285/:dsgvcy-75[91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26])/obj31[0] 2022-09-03T06:50:46.572 INFO:teuthology.orchestra.run.smithi189.stderr: -54> 2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]: [full sync] syncing object: dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0/obj33 2022-09-03T06:50:46.572 INFO:teuthology.orchestra.run.smithi189.stderr: -53> 2022-09-03T06:50:46.270+0000 7fdd98754600 20 RGW-SYNC:bucket[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26<-dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:full_sync[dsgvcy-75:91786ef1-cdb4-4061-8a3e-2a28cc497285.5593.26:0]:entry[obj33]: bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b=bucket sync single entry (source_zone=91786ef1-cdb4-4061-8a3e-2a28cc497285) b= log_entry=obj33 op=0 op_state=1 2022-09-03T06:50:46.575 DEBUG:teuthology.orchestra.run:got remote process result: None 2022-09-03T06:50:46.576 INFO:tasks.rgw.c2.client.0:Restarting daemon 2022-09-03T06:50:46.577 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c2.client.0.stdout 2>&1 2022-09-03T06:50:46.586 INFO:tasks.rgw.c2.client.0:Started 2022-09-03T06:50:46.587 DEBUG:teuthology.orchestra.run.smithi189:> curl http://smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:46.612 INFO:teuthology.orchestra.run.smithi189.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T06:50:46.613 INFO:teuthology.orchestra.run.smithi189.stderr: Dload Upload Total Spent Left Speed 2022-09-03T06:50:46.614 INFO:teuthology.orchestra.run.smithi189.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi189.front.sepia.ceph.com port 8000: Connection refused 2022-09-03T06:50:46.619 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T06:50:47.622 DEBUG:teuthology.orchestra.run.smithi189:> curl http://smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:47.650 INFO:teuthology.orchestra.run.smithi189.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T06:50:47.651 INFO:teuthology.orchestra.run.smithi189.stderr: Dload Upload Total Spent Left Speed 2022-09-03T06:50:47.652 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T06:50:47.653 INFO:teuthology.orchestra.run.smithi189.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi189.front.sepia.ceph.com port 8000: Connection refused 2022-09-03T06:50:49.654 DEBUG:teuthology.orchestra.run.smithi189:> curl http://smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:49.685 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T06:50:49.687 INFO:teuthology.orchestra.run.smithi189.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T06:50:49.688 INFO:teuthology.orchestra.run.smithi189.stderr: Dload Upload Total Spent Left Speed 2022-09-03T06:50:49.688 INFO:teuthology.orchestra.run.smithi189.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi189.front.sepia.ceph.com port 8000: Connection refused 2022-09-03T06:50:53.689 DEBUG:teuthology.orchestra.run.smithi189:> curl http://smithi189.front.sepia.ceph.com:8000 2022-09-03T06:50:53.719 INFO:teuthology.orchestra.run.smithi189.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T06:50:53.720 INFO:teuthology.orchestra.run.smithi189.stderr: Dload Upload Total Spent Left Speed 2022-09-03T06:50:53.720 INFO:teuthology.orchestra.run.smithi189.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 208k 0 --:--:-- --:--:-- --:--:-- 208k 2022-09-03T06:50:53.722 INFO:teuthology.orchestra.run.smithi189.stdout:anonymous 2022-09-03T06:50:53.868 INFO:tasks.rgw_multisite_tests:Create several generations of objects, trash them, then run bucket sync init ... FAIL 2022-09-03T06:50:53.872 INFO:rgw_multi.tests:create role zone=test-zone1 name=roles-dsgvcy-1 2022-09-03T06:50:53.872 DEBUG:boto:Method: POST 2022-09-03T06:50:53.883 DEBUG:boto:Path: / 2022-09-03T06:50:53.884 DEBUG:boto:Data: 2022-09-03T06:50:53.885 DEBUG:boto:Headers: {} 2022-09-03T06:50:53.886 DEBUG:boto:Host: smithi181.front.sepia.ceph.com 2022-09-03T06:50:53.887 DEBUG:boto:Port: 8000 2022-09-03T06:50:53.888 DEBUG:boto:Params: {'RoleName': 'roles-dsgvcy-1', 'AssumeRolePolicyDocument': '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["arn:aws:iam:::user/testuser"]},"Action":["sts:AssumeRole"]}]}', 'Path': '', 'Action': 'CreateRole', 'Version': '2010-05-08'} 2022-09-03T06:50:53.889 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:50:53.890 DEBUG:boto:Token: None 2022-09-03T06:50:53.892 DEBUG:boto:CanonicalRequest: POST / host:smithi181.front.sepia.ceph.com:8000 x-amz-date:20220903T065053Z host;x-amz-date 5ddb570acc490aff424c554b43b4b3dbe1962b42f84bd5b6cdd4f29d85c25f43 2022-09-03T06:50:53.893 DEBUG:boto:StringToSign: AWS4-HMAC-SHA256 20220903T065053Z 20220903/front/smithi181/aws4_request bf550047df4e85d042eb9c8a5d6dc1575697d9dc8750f2ee8984a6b0e42b547d 2022-09-03T06:50:53.894 DEBUG:boto:Signature: bf823544dacb8df58f875fcaf7af7984adb234cf9d5e6b5b38b6f9d20e4ec23c 2022-09-03T06:50:53.896 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'X-Amz-Date': '20220903T065053Z', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'Content-Length': '325', 'Authorization': 'AWS4-HMAC-SHA256 Credential=SA4D9959RL1I1FMQ4Y2O/20220903/front/smithi181/aws4_request,SignedHeaders=host;x-amz-date,Signature=bf823544dacb8df58f875fcaf7af7984adb234cf9d5e6b5b38b6f9d20e4ec23c', 'Host': 'smithi181.front.sepia.ceph.com:8000'} 2022-09-03T06:50:53.911 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000a12c1912279627b6-006312f94d-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '731'), ('Date', 'Sat, 03 Sep 2022 06:50:53 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:50:53.912 DEBUG:boto:b'0d0f2c36-7c4b-46d9-b89d-1f549bd2dfa1roles-dsgvcy-1/arn:aws:iam:::role/roles-dsgvcy-12022-09-03T06:50:53.900Z3600{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["arn:aws:iam:::user/testuser"]},"Action":["sts:AssumeRole"]}]}tx00000a12c1912279627b6-006312f94d-15d7-test-zone1' 2022-09-03T06:50:53.914 INFO:rgw_multi.tests:create role zone=test-zone2 name=roles-dsgvcy-2 2022-09-03T06:50:53.915 DEBUG:boto:Method: POST 2022-09-03T06:50:53.915 DEBUG:boto:Path: / 2022-09-03T06:50:53.916 DEBUG:boto:Data: 2022-09-03T06:50:53.916 DEBUG:boto:Headers: {} 2022-09-03T06:50:53.917 DEBUG:boto:Host: smithi189.front.sepia.ceph.com 2022-09-03T06:50:53.917 DEBUG:boto:Port: 8000 2022-09-03T06:50:53.918 DEBUG:boto:Params: {'RoleName': 'roles-dsgvcy-2', 'AssumeRolePolicyDocument': '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["arn:aws:iam:::user/testuser"]},"Action":["sts:AssumeRole"]}]}', 'Path': '', 'Action': 'CreateRole', 'Version': '2010-05-08'} 2022-09-03T06:50:53.918 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:50:53.919 DEBUG:boto:Token: None 2022-09-03T06:50:53.920 DEBUG:boto:CanonicalRequest: POST / host:smithi189.front.sepia.ceph.com:8000 x-amz-date:20220903T065053Z host;x-amz-date d2c55ed2197d2347c7f35c3392f508c952bb60e5c469aa5d1c21f86bd57547d0 2022-09-03T06:50:53.920 DEBUG:boto:StringToSign: AWS4-HMAC-SHA256 20220903T065053Z 20220903/front/smithi189/aws4_request eb74e1ffa6f4f09ead82996f383ae1c6ef2f8c6043a1940fe119b7ecae5bd8ea 2022-09-03T06:50:53.921 DEBUG:boto:Signature: 9eb6217d1f412a834abff9cba54eb83b702371cd0fc4a688ccbd0b5c8e9a9515 2022-09-03T06:50:53.921 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'X-Amz-Date': '20220903T065053Z', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'Content-Length': '325', 'Authorization': 'AWS4-HMAC-SHA256 Credential=SA4D9959RL1I1FMQ4Y2O/20220903/front/smithi189/aws4_request,SignedHeaders=host;x-amz-date,Signature=9eb6217d1f412a834abff9cba54eb83b702371cd0fc4a688ccbd0b5c8e9a9515', 'Host': 'smithi189.front.sepia.ceph.com:8000'} 2022-09-03T06:51:16.150 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000fc3f2482b9f36bbe-006312f94d-1a27-test-zone2'), ('Content-Type', 'application/xml'), ('Content-Length', '731'), ('Date', 'Sat, 03 Sep 2022 06:51:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:51:16.152 DEBUG:boto:b'90db15e6-6472-4f84-b30f-37ff9bde26b9roles-dsgvcy-2/arn:aws:iam:::role/roles-dsgvcy-22022-09-03T06:50:53.939Z3600{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["arn:aws:iam:::user/testuser"]},"Action":["sts:AssumeRole"]}]}tx00000fc3f2482b9f36bbe-006312f94d-1a27-test-zone2' 2022-09-03T06:51:16.154 INFO:rgw_multi.tests:create role zone=test-zone3 name=roles-dsgvcy-3 2022-09-03T06:51:16.154 DEBUG:boto:Method: POST 2022-09-03T06:51:16.156 DEBUG:boto:Path: / 2022-09-03T06:51:16.156 DEBUG:boto:Data: 2022-09-03T06:51:16.157 DEBUG:boto:Headers: {} 2022-09-03T06:51:16.158 DEBUG:boto:Host: smithi181.front.sepia.ceph.com 2022-09-03T06:51:16.158 DEBUG:boto:Port: 8001 2022-09-03T06:51:16.159 DEBUG:boto:Params: {'RoleName': 'roles-dsgvcy-3', 'AssumeRolePolicyDocument': '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["arn:aws:iam:::user/testuser"]},"Action":["sts:AssumeRole"]}]}', 'Path': '', 'Action': 'CreateRole', 'Version': '2010-05-08'} 2022-09-03T06:51:16.159 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T06:51:16.160 DEBUG:boto:Token: None 2022-09-03T06:51:16.161 DEBUG:boto:CanonicalRequest: POST / host:smithi181.front.sepia.ceph.com:8001 x-amz-date:20220903T065116Z host;x-amz-date 31f79516dd8f5e9fa583d6e6596cc60c585f08f04e179e31d3dcab568b6d839b 2022-09-03T06:51:16.162 DEBUG:boto:StringToSign: AWS4-HMAC-SHA256 20220903T065116Z 20220903/front/smithi181/aws4_request 51c60ae7c5e6183a439deb422054a183f459e50f68465e9e5137411ef42685ff 2022-09-03T06:51:16.163 DEBUG:boto:Signature: 9ee6648a8208941c3596e6a04e6211ad086bb071956b37eeee1339083dd99976 2022-09-03T06:51:16.164 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'X-Amz-Date': '20220903T065116Z', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'Content-Length': '325', 'Authorization': 'AWS4-HMAC-SHA256 Credential=SA4D9959RL1I1FMQ4Y2O/20220903/front/smithi181/aws4_request,SignedHeaders=host;x-amz-date,Signature=9ee6648a8208941c3596e6a04e6211ad086bb071956b37eeee1339083dd99976', 'Host': 'smithi181.front.sepia.ceph.com:8001'} 2022-09-03T06:51:16.187 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000e9827446c1397188-006312f964-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Content-Length', '731'), ('Date', 'Sat, 03 Sep 2022 06:51:16 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:51:16.188 DEBUG:boto:b'6765c729-46a2-41ec-89f5-2d6b47ba00cbroles-dsgvcy-3/arn:aws:iam:::role/roles-dsgvcy-32022-09-03T06:51:16.180Z3600{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["arn:aws:iam:::user/testuser"]},"Action":["sts:AssumeRole"]}]}tx00000e9827446c1397188-006312f964-1851-test-zone3' 2022-09-03T06:51:16.191 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:51:16.193 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:51:16.194 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:51:16.250 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:51:16.251 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:51:16.301 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T06:51:16.322 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T06:51:16.323 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:51:16.323 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:51:16.324 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:51:16.173767Z" 2022-09-03T06:51:16.324 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:51:16.325 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:51:16.325 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:51:16.326 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:50:53.931976Z" 2022-09-03T06:51:16.327 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:51:16.327 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:51:16.329 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187669.440315_550.1", 2022-09-03T06:51:16.329 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:47:49.440315Z" 2022-09-03T06:51:16.330 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:51:16.331 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:51:16.332 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:51:16.332 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:51:16.333 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:51:16.333 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T06:51:16.336 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:51:16.338 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T06:51:16.339 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:51:16.340 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:51:16.341 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:51:16.402 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:51:16.406 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:51:16.435 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:51:16.432+0000 7f6740031600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:51:16.436 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:51:16.433+0000 7f6740031600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:51:16.453 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:51:16.453 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:51:16.454 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:51:16.454 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:51:16.455 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:51:16.455 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:51:16.456 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:51:16.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:16.457 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:51:16.458 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:16.458 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:51:16.459 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:16.460 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:16.461 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:51:16.461 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:16.462 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:51:16.462 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:16.463 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:51:16.463 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:16.464 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:16.464 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:16.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:16.465 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:51:16.466 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:16.466 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:16.467 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:51:16.467 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:16.468 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:16.469 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:16.469 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:51:16.470 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:16.471 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:16.472 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:16.472 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:16.474 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:51:16.474 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:16.475 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:16.476 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187669.440315_550.1", 2022-09-03T06:51:16.476 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:16.477 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:16.477 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:16.478 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:47:49.440315Z", 2022-09-03T06:51:16.478 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:51:16.479 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:16.479 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:16.481 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:16.481 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:51:16.482 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:16.482 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:16.483 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:51:16.483 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:16.484 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:16.484 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:16.485 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:51:16.485 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:16.486 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:16.486 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:16.487 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:51:16.487 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:16.487 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:51:16.487 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:51:16.488 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:51:16.488 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:16.489 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:51:16Z" 2022-09-03T06:51:16.491 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:51:16.493 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187669.440315_550.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:47:49.440315Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:51:16Z" } 2022-09-03T06:51:16.495 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187669.440315_550.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:47:49.440315Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:51:16.496 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:51:16.497 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:51:16.498 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662187876.173767_261.1 target=, shard=1 master=1_1662187853.931976_390.1 target= 2022-09-03T06:51:21.500 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:51:21.501 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:51:21.502 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:51:21.563 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:51:21.564 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:51:21.605 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:51:21.602+0000 7eff476f3600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:51:21.605 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:51:21.603+0000 7eff476f3600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:51:21.618 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:51:21.619 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:51:21.619 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:51:21.619 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:51:21.620 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:51:21.620 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:51:21.621 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:51:21.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:21.623 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:51:21.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:21.624 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:51:21.624 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:21.624 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:21.625 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:51:21.626 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:21.626 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:51:21.627 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:21.627 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:51:21.628 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:21.628 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:21.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:21.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:21.630 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:51:21.631 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:21.631 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:21.632 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:51:21.632 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:21.632 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:21.633 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:21.633 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:51:21.634 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:21.634 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:21.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:21.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:21.636 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:51:21.636 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:21.636 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:21.637 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187669.440315_550.1", 2022-09-03T06:51:21.637 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:21.637 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:21.638 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:21.638 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:47:49.440315Z", 2022-09-03T06:51:21.639 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:51:21.639 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:21.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:21.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:21.640 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:51:21.641 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:21.641 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:21.641 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:51:21.642 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:21.642 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:21.642 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:21.643 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:51:21.643 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:21.643 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:21.644 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:21.644 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:51:21.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:21.644 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:51:21.645 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:51:21.645 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:51:21.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:21.646 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:51:21Z" 2022-09-03T06:51:21.647 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:51:21.649 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187669.440315_550.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:47:49.440315Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:51:21Z" } 2022-09-03T06:51:21.650 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187669.440315_550.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:47:49.440315Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:51:21.651 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:51:21.652 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:51:21.653 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662187876.173767_261.1 target=, shard=1 master=1_1662187853.931976_390.1 target= 2022-09-03T06:51:26.655 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:51:26.656 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:51:26.657 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:51:26.716 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:51:26.717 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:51:26.749 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:51:26.747+0000 7fe450792600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:51:26.750 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:51:26.748+0000 7fe450792600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:51:26.764 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:51:26.765 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:51:26.766 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:51:26.767 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:51:26.767 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:51:26.768 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:51:26.768 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:51:26.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:26.770 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:51:26.771 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:26.771 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:51:26.772 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:26.772 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:26.773 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:51:26.773 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:26.773 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:51:26.774 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:26.775 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:51:26.775 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:26.776 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:26.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:26.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:26.779 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:51:26.779 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:26.780 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:26.781 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:51:26.782 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:26.783 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:26.783 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:26.784 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:51:26.784 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:26.785 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:26.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:26.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:26.786 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:51:26.787 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:26.787 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:26.787 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187669.440315_550.1", 2022-09-03T06:51:26.788 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:26.788 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:26.788 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:26.789 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:47:49.440315Z", 2022-09-03T06:51:26.789 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:51:26.789 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:26.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:26.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:26.791 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:51:26.791 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:26.791 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:26.792 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:51:26.792 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:26.792 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:26.793 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:26.793 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:51:26.794 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:26.794 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:26.794 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:26.795 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:51:26.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:26.795 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:51:26.796 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:51:26.796 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:51:26.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:26.796 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:51:26Z" 2022-09-03T06:51:26.797 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:51:26.798 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187669.440315_550.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:47:49.440315Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:51:26Z" } 2022-09-03T06:51:26.799 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187669.440315_550.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:47:49.440315Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:51:26.799 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:51:26.800 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:51:26.800 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662187876.173767_261.1 target=, shard=1 master=1_1662187853.931976_390.1 target= 2022-09-03T06:51:31.801 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:51:31.802 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:51:31.803 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:51:31.861 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:51:31.861 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:51:31.893 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:51:31.891+0000 7ff5ebeea600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:51:31.894 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:51:31.892+0000 7ff5ebeea600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:51:31.914 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:51:31.914 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:51:31.915 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:51:31.915 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:51:31.916 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:51:31.916 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:51:31.917 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:51:31.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:31.918 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:51:31.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:31.918 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:51:31.919 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:31.919 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:31.919 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:51:31.920 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:31.920 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:51:31.920 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:31.921 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:51:31.921 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:31.923 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:31.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:31.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:31.925 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:51:31.925 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:31.926 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:31.926 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:51:31.927 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:31.927 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:31.928 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:31.929 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:51:31.929 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:31.932 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:31.932 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:31.933 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:31.934 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:51:31.934 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:31.936 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:31.937 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187669.440315_550.1", 2022-09-03T06:51:31.937 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:31.938 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:31.938 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:31.939 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:47:49.440315Z", 2022-09-03T06:51:31.939 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:51:31.940 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:31.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:31.941 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:31.941 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:51:31.942 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:31.942 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:31.943 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:51:31.943 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:31.944 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:31.944 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:31.945 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:51:31.945 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:31.946 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:31.946 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:31.947 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:51:31.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:31.948 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:51:31.949 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:51:31.949 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:51:31.950 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:31.950 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:51:31Z" 2022-09-03T06:51:31.951 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:51:31.952 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187669.440315_550.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:47:49.440315Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:51:31Z" } 2022-09-03T06:51:31.953 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187669.440315_550.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:47:49.440315Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:51:31.954 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:51:31.955 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:51:31.956 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662187876.173767_261.1 target=, shard=1 master=1_1662187853.931976_390.1 target= 2022-09-03T06:51:36.960 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:51:36.965 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:51:36.965 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:51:37.045 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:51:37.045 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:51:37.091 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:51:37.084+0000 7f720e8c5600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:51:37.094 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:51:37.085+0000 7f720e8c5600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:51:37.105 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:51:37.106 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:51:37.106 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:51:37.106 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:51:37.107 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:51:37.107 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:51:37.107 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:51:37.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:37.108 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:51:37.108 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:37.109 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:51:37.109 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:37.109 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:37.110 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:51:37.110 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:37.110 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:51:37.111 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:37.111 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:51:37.111 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:37.112 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:37.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:37.113 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:37.114 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:51:37.115 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:37.115 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:37.115 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:51:37.116 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:37.116 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:37.116 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:37.117 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:51:37.117 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:37.118 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:37.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:37.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:37.119 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:51:37.119 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:37.120 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:37.120 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187669.440315_550.1", 2022-09-03T06:51:37.120 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:37.120 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:37.121 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:37.121 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:47:49.440315Z", 2022-09-03T06:51:37.121 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:51:37.122 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:37.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:37.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:37.123 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:51:37.123 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:37.123 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:37.123 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:51:37.124 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:37.124 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:37.124 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:37.125 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:51:37.125 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:37.125 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:37.125 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:37.126 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:51:37.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:37.127 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:51:37.127 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:51:37.128 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:51:37.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:37.129 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:51:37Z" 2022-09-03T06:51:37.130 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:51:37.132 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187669.440315_550.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:47:49.440315Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:51:37Z" } 2022-09-03T06:51:37.133 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187669.440315_550.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:47:49.440315Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:51:37.136 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:51:37.137 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:51:37.139 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662187876.173767_261.1 target=, shard=1 master=1_1662187853.931976_390.1 target= 2022-09-03T06:51:42.141 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:51:42.142 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:51:42.143 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:51:42.201 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:51:42.202 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:51:42.232 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:51:42.229+0000 7f5e9327e600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:51:42.233 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:51:42.231+0000 7f5e9327e600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:51:42.248 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:51:42.249 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:51:42.250 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:51:42.250 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:51:42.251 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:51:42.252 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:51:42.252 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:51:42.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:42.253 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:51:42.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:42.254 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:51:42.255 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:42.255 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:42.256 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:51:42.256 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:42.257 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:51:42.257 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:42.258 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:51:42.258 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:42.259 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:42.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:42.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:42.261 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:51:42.261 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:42.262 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:42.262 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:51:42.263 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:42.263 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:42.264 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:42.264 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:51:42.265 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:42.265 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:42.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:42.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:42.267 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:51:42.268 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:42.268 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:42.269 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187669.440315_550.1", 2022-09-03T06:51:42.269 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:42.270 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:42.270 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:42.271 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:47:49.440315Z", 2022-09-03T06:51:42.271 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:51:42.272 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:42.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:42.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:42.274 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:51:42.274 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:42.275 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:42.275 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:51:42.276 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:42.276 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:42.277 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:42.277 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:51:42.278 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:42.279 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:42.279 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:42.280 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:51:42.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:42.281 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:51:42.281 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:51:42.282 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:51:42.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:42.283 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:51:42Z" 2022-09-03T06:51:42.283 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:51:42.285 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187669.440315_550.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:47:49.440315Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:51:42Z" } 2022-09-03T06:51:42.286 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187669.440315_550.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:47:49.440315Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:51:42.287 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:51:42.287 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:51:42.288 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662187876.173767_261.1 target=, shard=1 master=1_1662187853.931976_390.1 target= 2022-09-03T06:51:47.290 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:51:47.291 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:51:47.292 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:51:47.352 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:51:47.353 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:51:47.382 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:51:47.379+0000 7f0958917600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:51:47.383 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:51:47.380+0000 7f0958917600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:51:47.396 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:51:47.396 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:51:47.396 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:51:47.397 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:51:47.397 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:51:47.397 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:51:47.397 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:51:47.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:47.398 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:51:47.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:47.399 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:51:47.399 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:47.399 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:47.400 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:51:47.400 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:47.400 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:51:47.400 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:47.401 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:51:47.401 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:47.401 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:47.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:47.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:47.402 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:51:47.403 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:47.403 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:47.403 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:51:47.405 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:47.405 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:47.406 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:47.406 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:51:47.406 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:47.407 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:47.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:47.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:47.408 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:51:47.409 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:47.409 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:47.410 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187669.440315_550.1", 2022-09-03T06:51:47.410 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:47.411 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:47.412 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:47.412 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:47:49.440315Z", 2022-09-03T06:51:47.413 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:51:47.413 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:47.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:47.415 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:47.416 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:51:47.416 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:47.417 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:47.419 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:51:47.419 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:47.420 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:47.421 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:47.421 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:51:47.422 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:47.423 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:47.423 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:47.424 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:51:47.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:47.425 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:51:47.425 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:51:47.426 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:51:47.426 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:47.427 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:51:47Z" 2022-09-03T06:51:47.427 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:51:47.429 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187669.440315_550.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:47:49.440315Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:51:47Z" } 2022-09-03T06:51:47.430 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187669.440315_550.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:47:49.440315Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:51:47.431 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:51:47.432 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:51:47.433 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662187876.173767_261.1 target=, shard=1 master=1_1662187853.931976_390.1 target= 2022-09-03T06:51:52.434 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:51:52.435 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:51:52.436 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:51:52.494 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:51:52.495 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:51:52.533 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:51:52.525+0000 7f4a5cab2600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:51:52.533 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:51:52.526+0000 7f4a5cab2600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:51:52.548 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:51:52.549 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:51:52.550 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:51:52.551 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:51:52.552 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:51:52.552 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:51:52.553 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:51:52.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:52.554 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:51:52.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:52.554 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:51:52.555 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:52.555 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:52.556 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:51:52.556 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:52.557 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:51:52.557 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:52.558 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:51:52.558 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:52.559 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:52.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:52.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:52.560 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:51:52.561 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:52.561 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:52.561 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:51:52.562 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:52.562 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:52.564 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:52.565 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:51:52.565 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:52.565 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:52.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:52.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:52.566 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:51:52.566 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:52.567 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:52.567 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187669.440315_550.1", 2022-09-03T06:51:52.567 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:52.568 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:52.568 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:52.568 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:47:49.440315Z", 2022-09-03T06:51:52.569 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:51:52.569 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:52.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:52.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:52.570 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:51:52.571 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:52.572 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:52.572 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:51:52.572 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:52.573 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:52.573 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:52.573 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:51:52.573 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:52.574 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:52.574 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:52.574 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:51:52.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:52.575 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:51:52.576 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:51:52.576 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:51:52.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:52.577 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:51:52Z" 2022-09-03T06:51:52.577 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:51:52.577 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187669.440315_550.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:47:49.440315Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:51:52Z" } 2022-09-03T06:51:52.578 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187669.440315_550.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:47:49.440315Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:51:52.579 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:51:52.579 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:51:52.580 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662187876.173767_261.1 target=, shard=1 master=1_1662187853.931976_390.1 target= 2022-09-03T06:51:57.581 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:51:57.601 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:51:57.604 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:51:57.662 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:51:57.662 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:51:57.707 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:51:57.702+0000 7f3b32a99600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:51:57.708 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:51:57.703+0000 7f3b32a99600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:51:57.728 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:51:57.729 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:51:57.729 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:51:57.730 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:51:57.730 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:51:57.731 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:51:57.732 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:51:57.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:57.732 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:51:57.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:57.734 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:51:57.735 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:57.735 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:57.735 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:51:57.736 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:57.737 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:51:57.737 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:57.738 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:51:57.738 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:57.739 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:57.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:57.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:57.740 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:51:57.741 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:57.741 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:57.742 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:51:57.743 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:57.744 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:57.744 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:57.745 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:51:57.745 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:57.746 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:57.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:57.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:57.747 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:51:57.747 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:57.748 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:57.748 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187669.440315_550.1", 2022-09-03T06:51:57.749 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:57.749 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:57.750 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:57.750 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:47:49.440315Z", 2022-09-03T06:51:57.751 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:51:57.751 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:57.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:57.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:51:57.753 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:51:57.753 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:51:57.754 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:51:57.754 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:51:57.755 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:51:57.755 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:51:57.755 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:51:57.756 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:51:57.756 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:51:57.756 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:57.757 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:51:57.757 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:51:57.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:57.758 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:51:57.758 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:51:57.759 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:51:57.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:51:57.760 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:51:57Z" 2022-09-03T06:51:57.760 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:51:57.761 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187669.440315_550.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:47:49.440315Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:51:57Z" } 2022-09-03T06:51:57.762 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187669.440315_550.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:47:49.440315Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:51:57.763 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:51:57.764 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:51:57.765 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662187876.173767_261.1 target=, shard=1 master=1_1662187853.931976_390.1 target= 2022-09-03T06:52:02.766 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:02.767 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:02.768 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:52:02.828 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:52:02.829 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:52:02.861 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:02.857+0000 7f1516b03600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:02.862 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:02.858+0000 7f1516b03600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:02.876 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:52:02.880 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:52:02.881 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:52:02.881 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:52:02.882 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:52:02.882 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:52:02.883 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:02.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:02.886 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:52:02.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:02.888 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:52:02.889 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:02.889 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:02.890 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:52:02.891 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:02.891 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:52:02.892 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:02.892 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:52:02.893 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:52:02.893 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:02.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:02.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:02.897 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:52:02.898 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:02.899 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:02.901 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:52:02.902 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:02.902 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:02.903 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:02.904 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:52:02.904 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:52:02.905 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:02.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:02.906 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:02.906 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:52:02.907 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:02.908 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:02.908 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187669.440315_550.1", 2022-09-03T06:52:02.909 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:02.910 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:02.910 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:02.911 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:47:49.440315Z", 2022-09-03T06:52:02.912 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:02.912 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:02.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:02.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:02.914 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:52:02.914 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:02.915 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:02.916 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:52:02.916 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:02.917 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:02.917 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:02.918 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:52:02.918 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:52:02.919 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:02.919 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:02.920 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:52:02.920 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:02.921 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:52:02.922 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:52:02.922 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:52:02.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:02.923 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:52:02Z" 2022-09-03T06:52:02.924 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:52:02.925 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187669.440315_550.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:47:49.440315Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:52:02Z" } 2022-09-03T06:52:02.926 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187669.440315_550.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:47:49.440315Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:52:02.927 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:52:02.928 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:52:02.929 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662187876.173767_261.1 target=, shard=1 master=1_1662187853.931976_390.1 target= 2022-09-03T06:52:07.932 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:07.934 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:07.935 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:52:07.994 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:52:07.995 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:52:08.027 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:08.025+0000 7fa6bf0f3600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:08.028 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:08.026+0000 7fa6bf0f3600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:08.052 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:52:08.053 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:52:08.053 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:52:08.053 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:52:08.053 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:52:08.054 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:52:08.054 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:08.054 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:08.055 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:52:08.055 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:08.055 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:52:08.055 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:08.056 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:08.056 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:52:08.056 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:08.056 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:52:08.057 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:08.057 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:52:08.057 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:52:08.058 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:08.058 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:08.058 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:08.058 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:52:08.059 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:08.059 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:08.059 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:52:08.059 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:08.060 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:08.060 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:08.060 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:52:08.061 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:52:08.061 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:08.061 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:08.062 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:08.062 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:52:08.063 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:08.063 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:08.063 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187669.440315_550.1", 2022-09-03T06:52:08.064 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:08.064 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:08.064 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:08.065 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:47:49.440315Z", 2022-09-03T06:52:08.065 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:08.065 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:08.065 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:08.066 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:08.066 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:52:08.066 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:08.067 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:08.067 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:52:08.067 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:08.067 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:08.068 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:08.068 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:52:08.068 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:52:08.068 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:08.069 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:08.069 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:52:08.069 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:08.069 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:52:08.070 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:52:08.070 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:52:08.070 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:08.071 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:52:08Z" 2022-09-03T06:52:08.071 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:52:08.071 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187669.440315_550.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:47:49.440315Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:52:08Z" } 2022-09-03T06:52:08.072 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187669.440315_550.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:47:49.440315Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:52:08.073 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:52:08.073 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:52:08.074 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662187876.173767_261.1 target=, shard=1 master=1_1662187853.931976_390.1 target= 2022-09-03T06:52:13.079 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:13.081 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:13.082 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:52:13.141 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:52:13.142 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:52:13.184 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:13.178+0000 7f904fb37600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:13.185 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:13.179+0000 7f904fb37600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:13.199 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:52:13.199 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:52:13.200 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:52:13.200 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:52:13.200 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:52:13.201 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:52:13.201 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:13.201 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:13.201 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:52:13.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:13.202 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:52:13.202 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:13.202 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:13.203 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182298.906852_69.1", 2022-09-03T06:52:13.203 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:13.204 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:52:13.204 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:13.204 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:18:18.906852Z", 2022-09-03T06:52:13.205 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:52:13.205 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:13.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:13.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:13.206 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:52:13.206 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:13.206 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:13.207 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:52:13.207 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:13.207 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:13.207 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:13.208 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:52:13.208 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:52:13.208 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:13.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:13.209 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:13.209 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:52:13.209 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:13.209 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:13.210 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187669.440315_550.1", 2022-09-03T06:52:13.210 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:13.210 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:13.210 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:13.211 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:47:49.440315Z", 2022-09-03T06:52:13.211 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:13.211 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:13.211 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:13.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:13.212 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:52:13.212 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:13.213 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:13.213 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:52:13.213 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:13.214 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:13.215 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:13.215 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:52:13.215 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:52:13.216 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:13.216 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:13.216 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:52:13.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:13.217 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:52:13.217 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:52:13.217 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:52:13.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:13.218 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:52:13Z" 2022-09-03T06:52:13.218 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:52:13.218 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662182298.906852_69.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T05:18:18.906852Z", "realm_epoch": 2 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187669.440315_550.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:47:49.440315Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:52:13Z" } 2022-09-03T06:52:13.219 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662182298.906852_69.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T05:18:18.906852Z', 'realm_epoch': 2}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187669.440315_550.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:47:49.440315Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:52:13.220 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:52:13.220 DEBUG:rgw_multi.tests:sync_status={0: '', 1: '', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:52:13.221 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=0 master=1_1662187876.173767_261.1 target=, shard=1 master=1_1662187853.931976_390.1 target= 2022-09-03T06:52:18.223 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:18.224 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:18.225 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:52:18.314 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:52:18.315 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:52:18.332 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:18.330+0000 7f3e5d3c8600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:18.333 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:18.331+0000 7f3e5d3c8600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:18.347 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:52:18.349 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:52:18.349 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:52:18.350 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:52:18.351 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:52:18.351 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:52:18.352 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:18.352 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:18.353 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:52:18.353 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:18.354 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:52:18.354 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:18.355 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:18.355 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:52:18.355 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:18.356 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:52:18.356 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:18.357 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:52:18.357 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:18.358 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:18.358 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:18.359 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:18.359 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:52:18.360 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:18.360 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:18.361 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:52:18.361 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:18.362 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:18.363 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:18.363 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:52:18.364 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:52:18.364 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:18.365 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:18.365 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:18.365 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:52:18.366 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:18.366 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:18.366 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187669.440315_550.1", 2022-09-03T06:52:18.367 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:18.367 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:18.367 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:18.368 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:47:49.440315Z", 2022-09-03T06:52:18.368 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:18.368 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:18.369 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:18.369 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:18.369 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:52:18.369 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:18.370 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:18.370 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:52:18.371 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:18.371 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:18.371 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:18.371 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:52:18.372 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:52:18.372 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:18.372 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:18.373 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:52:18.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:18.373 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:52:18.374 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:52:18.374 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:52:18.374 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:18.374 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:52:18Z" 2022-09-03T06:52:18.375 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:52:18.376 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187669.440315_550.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:47:49.440315Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:52:18Z" } 2022-09-03T06:52:18.377 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187669.440315_550.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:47:49.440315Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:52:18.378 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:52:18.379 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:52:18.379 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=1 master=1_1662187853.931976_390.1 target= 2022-09-03T06:52:23.380 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:23.382 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:23.384 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:52:23.465 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:52:23.465 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:52:23.495 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:23.492+0000 7fee8be51600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:23.495 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:23.493+0000 7fee8be51600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:23.514 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:52:23.515 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:52:23.515 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:52:23.516 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:52:23.516 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:52:23.517 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:52:23.517 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:23.517 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:23.518 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:52:23.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:23.519 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:52:23.519 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:23.520 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:23.520 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:52:23.521 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:23.521 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:52:23.522 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:23.522 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:52:23.523 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:23.523 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:23.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:23.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:23.525 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:52:23.525 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:23.525 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:23.526 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:52:23.526 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:23.527 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:23.527 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:23.528 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:52:23.528 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:52:23.528 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:23.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:23.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:23.529 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:52:23.529 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:23.530 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:23.530 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187669.440315_550.1", 2022-09-03T06:52:23.530 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:23.531 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:23.531 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:23.531 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:47:49.440315Z", 2022-09-03T06:52:23.531 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:23.532 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:23.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:23.532 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:23.532 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:52:23.533 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:23.533 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:23.533 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:52:23.533 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:23.534 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:23.534 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:23.535 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:52:23.535 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:52:23.535 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:23.535 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:23.536 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:52:23.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:23.536 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:52:23.537 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:52:23.537 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:52:23.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:23.538 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:52:23Z" 2022-09-03T06:52:23.538 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:52:23.539 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187669.440315_550.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:47:49.440315Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:52:23Z" } 2022-09-03T06:52:23.540 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187669.440315_550.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:47:49.440315Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:52:23.540 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:52:23.541 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:52:23.542 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=1 master=1_1662187853.931976_390.1 target= 2022-09-03T06:52:28.544 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:28.545 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:28.546 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:52:28.607 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:52:28.608 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:52:28.640 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:28.637+0000 7ff181daf600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:28.640 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:28.638+0000 7ff181daf600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:28.656 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:52:28.657 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:52:28.657 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:52:28.657 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:52:28.658 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:52:28.659 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:52:28.659 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:28.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:28.660 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:52:28.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:28.661 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:52:28.661 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:28.662 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:28.662 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:52:28.662 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:28.662 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:52:28.663 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:28.663 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:52:28.663 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:28.664 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:28.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:28.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:28.665 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:52:28.665 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:28.665 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:28.666 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "", 2022-09-03T06:52:28.667 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:28.667 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:28.668 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:28.669 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "0.000000", 2022-09-03T06:52:28.669 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:52:28.670 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:28.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:28.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:28.672 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:52:28.673 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:28.673 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:28.674 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187669.440315_550.1", 2022-09-03T06:52:28.674 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:28.675 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:28.675 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:28.676 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:47:49.440315Z", 2022-09-03T06:52:28.676 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:28.676 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:28.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:28.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:28.678 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:52:28.678 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:28.679 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:28.679 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:52:28.680 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:28.680 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:28.680 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:28.680 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:52:28.681 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:52:28.681 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:28.681 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:28.682 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:52:28.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:28.683 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:52:28.683 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:52:28.684 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:52:28.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:28.685 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:52:28Z" 2022-09-03T06:52:28.685 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:52:28.688 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "0.000000", "realm_epoch": 2 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187669.440315_550.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:47:49.440315Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:52:28Z" } 2022-09-03T06:52:28.689 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '0.000000', 'realm_epoch': 2}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187669.440315_550.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:47:49.440315Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:52:28.690 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:52:28.691 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:52:28.692 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=1 master=1_1662187853.931976_390.1 target= 2022-09-03T06:52:33.694 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:33.695 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:33.696 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:52:33.753 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:52:33.754 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:52:33.799 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:33.784+0000 7fc2a6a2c600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:33.800 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:33.785+0000 7fc2a6a2c600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:33.802 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:52:33.803 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:52:33.803 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:52:33.803 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:52:33.804 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:52:33.804 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:52:33.805 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:33.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:33.806 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:52:33.812 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:33.813 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:52:33.813 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:33.814 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:33.814 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:52:33.815 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:33.816 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:52:33.816 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:33.817 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:52:33.817 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:33.818 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:33.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:33.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:33.819 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:52:33.819 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:33.820 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:33.820 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.900961_387.1", 2022-09-03T06:52:33.821 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:33.821 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:33.821 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:33.821 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.900961Z", 2022-09-03T06:52:33.822 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:33.823 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:33.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:33.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:33.824 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:52:33.824 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:33.824 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:33.825 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187669.440315_550.1", 2022-09-03T06:52:33.825 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:33.825 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:33.826 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:33.826 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:47:49.440315Z", 2022-09-03T06:52:33.826 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:33.826 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:33.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:33.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:33.827 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:52:33.828 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:33.828 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:33.828 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:52:33.828 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:33.829 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:33.829 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:33.829 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:52:33.829 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:52:33.830 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:33.830 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:33.830 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:52:33.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:33.831 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:52:33.831 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:52:33.831 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:52:33.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:33.832 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:52:33Z" 2022-09-03T06:52:33.833 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:52:33.834 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.900961_387.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.900961Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187669.440315_550.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:47:49.440315Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:52:33Z" } 2022-09-03T06:52:33.835 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.900961_387.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.900961Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187669.440315_550.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:47:49.440315Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:52:33.836 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:52:33.836 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.900961_387.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:52:33.837 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=1 master=1_1662187853.931976_390.1 target=1_1662187853.900961_387.1 2022-09-03T06:52:38.838 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:38.839 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:38.840 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:52:38.898 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:52:38.899 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:52:38.931 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:38.929+0000 7f689470f600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:38.932 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:38.930+0000 7f689470f600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:38.946 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:52:38.947 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:52:38.947 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:52:38.947 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:52:38.947 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:52:38.948 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:52:38.948 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:38.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:38.949 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:52:38.949 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:38.949 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:52:38.949 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:38.950 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:38.950 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:52:38.950 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:38.950 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:52:38.951 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:38.951 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:52:38.951 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:38.951 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:38.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:38.952 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:38.952 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:52:38.953 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:38.953 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:38.953 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:52:38.953 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:38.954 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:38.954 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:38.954 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:52:38.955 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:38.955 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:38.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:38.955 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:38.956 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:52:38.956 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:38.956 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:38.956 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187669.440315_550.1", 2022-09-03T06:52:38.957 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:38.957 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:38.958 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:38.958 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:47:49.440315Z", 2022-09-03T06:52:38.958 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:38.959 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:38.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:38.959 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:38.960 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:52:38.960 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:38.960 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:38.960 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:52:38.961 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:38.961 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:38.961 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:38.962 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:52:38.962 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:52:38.962 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:38.962 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:38.963 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:52:38.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:38.963 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:52:38.963 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:52:38.964 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:52:38.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:38.964 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:52:38Z" 2022-09-03T06:52:38.964 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:52:38.966 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187669.440315_550.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:47:49.440315Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:52:38Z" } 2022-09-03T06:52:38.966 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187669.440315_550.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:47:49.440315Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:52:38.967 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:52:38.968 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:52:38.968 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T06:52:38.969 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T06:52:38.969 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:38.970 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:38.970 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:52:39.027 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:52:39.028 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:52:39.074 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:52:39.072+0000 7fd95e357600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:39.076 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:52:39.073+0000 7fd95e357600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:39.100 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T06:52:39.101 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T06:52:39.101 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T06:52:39.102 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T06:52:39.102 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T06:52:39.103 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:52:39.103 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:52:39.104 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:52:39.104 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T06:52:39.105 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:52:39.105 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T06:52:39.106 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:52:39.106 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:52:39.106 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:52:39.107 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:52:39.107 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T06:52:39.107 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:52:39.108 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:52:39.108 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:52:39.108 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:52:39.108 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:52:39.109 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:52:39.109 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T06:52:39.109 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:52:39.109 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:52:39.110 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:52:39.110 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:52:39.110 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:52:39.111 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:52:39.111 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:52:39.111 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:52:39.111 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:52:39.112 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:52:39.112 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:52:39.112 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T06:52:39.112 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:52:39.113 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:52:39.113 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187669.440315_550.1", 2022-09-03T06:52:39.114 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:52:39.114 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:52:39.114 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:52:39.115 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:47:49.440315Z", 2022-09-03T06:52:39.115 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:52:39.116 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:52:39.116 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:52:39.116 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:52:39.117 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T06:52:39.117 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:52:39.117 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:52:39.117 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:52:39.118 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:52:39.118 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:52:39.118 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:52:39.119 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:52:39.119 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:52:39.119 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:52:39.120 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:52:39.120 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:52:39.120 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:52:39.120 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T06:52:39.121 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T06:52:39.121 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T06:52:39.121 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:52:39.122 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T06:52:39Z" 2022-09-03T06:52:39.122 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T06:52:39.123 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187669.440315_550.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:47:49.440315Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:52:39Z" } 2022-09-03T06:52:39.124 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187669.440315_550.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:47:49.440315Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:52:39.125 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:52:39.125 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:52:39.126 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T06:52:39.127 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T06:52:39.127 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:39.128 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:39.128 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:52:39.181 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:52:39.182 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:52:39.225 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:39.222+0000 7f4ede81c600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:39.226 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:39.223+0000 7f4ede81c600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:39.246 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:52:39.247 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:52:39.247 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:52:39.249 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:52:39.249 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:52:39.250 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:52:39.250 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:39.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:39.251 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:52:39.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:39.253 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:52:39.253 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:39.253 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:39.254 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:52:39.254 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:39.255 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:52:39.255 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:39.256 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:52:39.256 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:39.256 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:39.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:39.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:39.257 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:52:39.257 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:39.258 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:39.258 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:52:39.258 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:39.258 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:39.259 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:39.260 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:52:39.260 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:39.261 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:39.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:39.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:39.261 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:52:39.262 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:39.262 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:39.263 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187669.440315_550.1", 2022-09-03T06:52:39.263 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:39.263 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:39.263 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:39.264 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:47:49.440315Z", 2022-09-03T06:52:39.264 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:39.265 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:39.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:39.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:39.266 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:52:39.266 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:39.267 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:39.267 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:52:39.267 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:39.267 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:39.268 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:39.268 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:52:39.268 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:52:39.268 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:39.269 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:39.269 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:52:39.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:39.270 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:52:39.270 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:52:39.270 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:52:39.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:39.271 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:52:39Z" 2022-09-03T06:52:39.271 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:52:39.274 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187669.440315_550.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:47:49.440315Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:52:39Z" } 2022-09-03T06:52:39.275 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187669.440315_550.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:47:49.440315Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:52:39.276 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:52:39.277 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187669.440315_550.1', 3: ''} 2022-09-03T06:52:39.278 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T06:52:39.279 INFO:tasks.rgw_multi.zone_rados:comparing role=roles-dsgvcy-1 zones={test-zone2, test-zone1} 2022-09-03T06:52:39.280 DEBUG:boto:Method: POST 2022-09-03T06:52:39.281 DEBUG:boto:Path: / 2022-09-03T06:52:39.282 DEBUG:boto:Data: 2022-09-03T06:52:39.283 DEBUG:boto:Headers: {} 2022-09-03T06:52:39.284 DEBUG:boto:Host: smithi189.front.sepia.ceph.com 2022-09-03T06:52:39.285 DEBUG:boto:Port: 8000 2022-09-03T06:52:39.285 DEBUG:boto:Params: {'RoleName': 'roles-dsgvcy-1', 'Action': 'GetRole', 'Version': '2010-05-08'} 2022-09-03T06:52:39.287 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:52:39.288 DEBUG:boto:Token: None 2022-09-03T06:52:39.290 DEBUG:boto:CanonicalRequest: POST / host:smithi189.front.sepia.ceph.com:8000 x-amz-date:20220903T065239Z host;x-amz-date 0c2342aba61bdba62099dca5c0dedaefbcd87d676cc1ef5580e1c08e004ee6c9 2022-09-03T06:52:39.291 DEBUG:boto:StringToSign: AWS4-HMAC-SHA256 20220903T065239Z 20220903/front/smithi189/aws4_request cc449d0cfd95ad817a3e9a4ece4931cc651c1d60fa44e79173af063ccfbe8ecb 2022-09-03T06:52:39.292 DEBUG:boto:Signature: ac72d8aa2735dd7c0dacceeafc50c39175d560f8cb247812e275c90f02a6708c 2022-09-03T06:52:39.293 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'X-Amz-Date': '20220903T065239Z', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'Content-Length': '57', 'Authorization': 'AWS4-HMAC-SHA256 Credential=SA4D9959RL1I1FMQ4Y2O/20220903/front/smithi189/aws4_request,SignedHeaders=host;x-amz-date,Signature=ac72d8aa2735dd7c0dacceeafc50c39175d560f8cb247812e275c90f02a6708c', 'Host': 'smithi189.front.sepia.ceph.com:8000'} 2022-09-03T06:52:39.298 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000e4148e1c72ad65d3-006312f9b7-1a27-test-zone2'), ('Content-Type', 'application/xml'), ('Content-Length', '719'), ('Date', 'Sat, 03 Sep 2022 06:52:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:52:39.299 DEBUG:boto:b'tx00000e4148e1c72ad65d3-006312f9b7-1a27-test-zone20d0f2c36-7c4b-46d9-b89d-1f549bd2dfa1roles-dsgvcy-1/arn:aws:iam:::role/roles-dsgvcy-12022-09-03T06:50:53.900Z3600{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["arn:aws:iam:::user/testuser"]},"Action":["sts:AssumeRole"]}]}' 2022-09-03T06:52:39.300 DEBUG:boto:Method: POST 2022-09-03T06:52:39.301 DEBUG:boto:Path: / 2022-09-03T06:52:39.301 DEBUG:boto:Data: 2022-09-03T06:52:39.303 DEBUG:boto:Headers: {} 2022-09-03T06:52:39.304 DEBUG:boto:Host: smithi181.front.sepia.ceph.com 2022-09-03T06:52:39.304 DEBUG:boto:Port: 8000 2022-09-03T06:52:39.305 DEBUG:boto:Params: {'RoleName': 'roles-dsgvcy-1', 'Action': 'GetRole', 'Version': '2010-05-08'} 2022-09-03T06:52:39.306 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:52:39.307 DEBUG:boto:Token: None 2022-09-03T06:52:39.311 DEBUG:boto:CanonicalRequest: POST / host:smithi181.front.sepia.ceph.com:8000 x-amz-date:20220903T065239Z host;x-amz-date 0c2342aba61bdba62099dca5c0dedaefbcd87d676cc1ef5580e1c08e004ee6c9 2022-09-03T06:52:39.312 DEBUG:boto:StringToSign: AWS4-HMAC-SHA256 20220903T065239Z 20220903/front/smithi181/aws4_request 28ab68c0231ab5099f11227722cd8b99152a146ce3d4e5ab35e3a63ec413c435 2022-09-03T06:52:39.313 DEBUG:boto:Signature: c108f5589effc2757c1fd7c64c56e60d3f6893286e28a9d98f5686fb9a05f238 2022-09-03T06:52:39.315 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'X-Amz-Date': '20220903T065239Z', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'Content-Length': '57', 'Authorization': 'AWS4-HMAC-SHA256 Credential=SA4D9959RL1I1FMQ4Y2O/20220903/front/smithi181/aws4_request,SignedHeaders=host;x-amz-date,Signature=c108f5589effc2757c1fd7c64c56e60d3f6893286e28a9d98f5686fb9a05f238', 'Host': 'smithi181.front.sepia.ceph.com:8000'} 2022-09-03T06:52:39.325 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000150968da656075e3-006312f9b7-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '719'), ('Date', 'Sat, 03 Sep 2022 06:52:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:52:39.325 DEBUG:boto:b'tx00000150968da656075e3-006312f9b7-15d7-test-zone10d0f2c36-7c4b-46d9-b89d-1f549bd2dfa1roles-dsgvcy-1/arn:aws:iam:::role/roles-dsgvcy-12022-09-03T06:50:53.900Z3600{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["arn:aws:iam:::user/testuser"]},"Action":["sts:AssumeRole"]}]}' 2022-09-03T06:52:39.327 DEBUG:tasks.rgw_multi.zone_rados:comparing role name=roles-dsgvcy-1 2022-09-03T06:52:39.327 INFO:tasks.rgw_multi.zone_rados:success, role identical: role=roles-dsgvcy-1 zones={test-zone2, test-zone1} 2022-09-03T06:52:39.328 INFO:tasks.rgw_multi.zone_rados:comparing role=roles-dsgvcy-1 zones={test-zone3, test-zone1} 2022-09-03T06:52:39.329 DEBUG:boto:Method: POST 2022-09-03T06:52:39.329 DEBUG:boto:Path: / 2022-09-03T06:52:39.330 DEBUG:boto:Data: 2022-09-03T06:52:39.331 DEBUG:boto:Headers: {} 2022-09-03T06:52:39.332 DEBUG:boto:Host: smithi181.front.sepia.ceph.com 2022-09-03T06:52:39.332 DEBUG:boto:Port: 8001 2022-09-03T06:52:39.333 DEBUG:boto:Params: {'RoleName': 'roles-dsgvcy-1', 'Action': 'GetRole', 'Version': '2010-05-08'} 2022-09-03T06:52:39.334 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8001} 2022-09-03T06:52:39.335 DEBUG:boto:Token: None 2022-09-03T06:52:39.335 DEBUG:boto:CanonicalRequest: POST / host:smithi181.front.sepia.ceph.com:8001 x-amz-date:20220903T065239Z host;x-amz-date 0c2342aba61bdba62099dca5c0dedaefbcd87d676cc1ef5580e1c08e004ee6c9 2022-09-03T06:52:39.336 DEBUG:boto:StringToSign: AWS4-HMAC-SHA256 20220903T065239Z 20220903/front/smithi181/aws4_request c1592b2d2e2713da7d012c43b86a4be9e155ef71d1dd724e1559606e99a61a52 2022-09-03T06:52:39.337 DEBUG:boto:Signature: 2729a7f659ba6bcc4dc233e5cabe40c9e1ea9fc9a2bfce3756913b45f68a4fb5 2022-09-03T06:52:39.337 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'X-Amz-Date': '20220903T065239Z', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'Content-Length': '57', 'Authorization': 'AWS4-HMAC-SHA256 Credential=SA4D9959RL1I1FMQ4Y2O/20220903/front/smithi181/aws4_request,SignedHeaders=host;x-amz-date,Signature=2729a7f659ba6bcc4dc233e5cabe40c9e1ea9fc9a2bfce3756913b45f68a4fb5', 'Host': 'smithi181.front.sepia.ceph.com:8001'} 2022-09-03T06:52:39.340 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000067b2b71aed5d716a-006312f9b7-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Content-Length', '719'), ('Date', 'Sat, 03 Sep 2022 06:52:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:52:39.341 DEBUG:boto:b'tx0000067b2b71aed5d716a-006312f9b7-1851-test-zone30d0f2c36-7c4b-46d9-b89d-1f549bd2dfa1roles-dsgvcy-1/arn:aws:iam:::role/roles-dsgvcy-12022-09-03T06:50:53.985Z3600{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["arn:aws:iam:::user/testuser"]},"Action":["sts:AssumeRole"]}]}' 2022-09-03T06:52:39.342 DEBUG:boto:Method: POST 2022-09-03T06:52:39.343 DEBUG:boto:Path: / 2022-09-03T06:52:39.344 DEBUG:boto:Data: 2022-09-03T06:52:39.344 DEBUG:boto:Headers: {} 2022-09-03T06:52:39.345 DEBUG:boto:Host: smithi181.front.sepia.ceph.com 2022-09-03T06:52:39.345 DEBUG:boto:Port: 8000 2022-09-03T06:52:39.346 DEBUG:boto:Params: {'RoleName': 'roles-dsgvcy-1', 'Action': 'GetRole', 'Version': '2010-05-08'} 2022-09-03T06:52:39.346 DEBUG:boto:Token: None 2022-09-03T06:52:39.347 DEBUG:boto:CanonicalRequest: POST / host:smithi181.front.sepia.ceph.com:8000 x-amz-date:20220903T065239Z host;x-amz-date 0c2342aba61bdba62099dca5c0dedaefbcd87d676cc1ef5580e1c08e004ee6c9 2022-09-03T06:52:39.348 DEBUG:boto:StringToSign: AWS4-HMAC-SHA256 20220903T065239Z 20220903/front/smithi181/aws4_request 28ab68c0231ab5099f11227722cd8b99152a146ce3d4e5ab35e3a63ec413c435 2022-09-03T06:52:39.349 DEBUG:boto:Signature: c108f5589effc2757c1fd7c64c56e60d3f6893286e28a9d98f5686fb9a05f238 2022-09-03T06:52:39.349 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'X-Amz-Date': '20220903T065239Z', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'Content-Length': '57', 'Authorization': 'AWS4-HMAC-SHA256 Credential=SA4D9959RL1I1FMQ4Y2O/20220903/front/smithi181/aws4_request,SignedHeaders=host;x-amz-date,Signature=c108f5589effc2757c1fd7c64c56e60d3f6893286e28a9d98f5686fb9a05f238', 'Host': 'smithi181.front.sepia.ceph.com:8000'} 2022-09-03T06:52:39.352 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000009d3631582cecf9e2-006312f9b7-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '719'), ('Date', 'Sat, 03 Sep 2022 06:52:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:52:39.353 DEBUG:boto:b'tx000009d3631582cecf9e2-006312f9b7-15d7-test-zone10d0f2c36-7c4b-46d9-b89d-1f549bd2dfa1roles-dsgvcy-1/arn:aws:iam:::role/roles-dsgvcy-12022-09-03T06:50:53.900Z3600{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["arn:aws:iam:::user/testuser"]},"Action":["sts:AssumeRole"]}]}' 2022-09-03T06:52:39.354 DEBUG:tasks.rgw_multi.zone_rados:comparing role name=roles-dsgvcy-1 2022-09-03T06:52:39.354 INFO:tasks.rgw_multi.zone_rados:success, role identical: role=roles-dsgvcy-1 zones={test-zone3, test-zone1} 2022-09-03T06:52:39.355 INFO:tasks.rgw_multi.zone_rados:comparing role=roles-dsgvcy-2 zones={test-zone1, test-zone2} 2022-09-03T06:52:39.355 DEBUG:boto:Method: POST 2022-09-03T06:52:39.356 DEBUG:boto:Path: / 2022-09-03T06:52:39.357 DEBUG:boto:Data: 2022-09-03T06:52:39.358 DEBUG:boto:Headers: {} 2022-09-03T06:52:39.359 DEBUG:boto:Host: smithi181.front.sepia.ceph.com 2022-09-03T06:52:39.360 DEBUG:boto:Port: 8000 2022-09-03T06:52:39.360 DEBUG:boto:Params: {'RoleName': 'roles-dsgvcy-2', 'Action': 'GetRole', 'Version': '2010-05-08'} 2022-09-03T06:52:39.362 DEBUG:boto:Token: None 2022-09-03T06:52:39.366 DEBUG:boto:CanonicalRequest: POST / host:smithi181.front.sepia.ceph.com:8000 x-amz-date:20220903T065239Z host;x-amz-date efc09ad2e77071df674d9987b9ba775760f7530993dec59449a862961fa77f08 2022-09-03T06:52:39.367 DEBUG:boto:StringToSign: AWS4-HMAC-SHA256 20220903T065239Z 20220903/front/smithi181/aws4_request 8db9aaa425200988f0e0ed886c2feef35115b25273c3dc6ec6bccf3ad90df723 2022-09-03T06:52:39.368 DEBUG:boto:Signature: 619c94a662d21b8c5b03cbe8f91f27e3e4ae91398b4e323c5f1cd65878a9a9d1 2022-09-03T06:52:39.369 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'X-Amz-Date': '20220903T065239Z', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'Content-Length': '57', 'Authorization': 'AWS4-HMAC-SHA256 Credential=SA4D9959RL1I1FMQ4Y2O/20220903/front/smithi181/aws4_request,SignedHeaders=host;x-amz-date,Signature=619c94a662d21b8c5b03cbe8f91f27e3e4ae91398b4e323c5f1cd65878a9a9d1', 'Host': 'smithi181.front.sepia.ceph.com:8000'} 2022-09-03T06:52:39.371 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000004568dad2384a047e-006312f9b7-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '719'), ('Date', 'Sat, 03 Sep 2022 06:52:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:52:39.374 DEBUG:boto:b'tx000004568dad2384a047e-006312f9b7-15d7-test-zone190db15e6-6472-4f84-b30f-37ff9bde26b9roles-dsgvcy-2/arn:aws:iam:::role/roles-dsgvcy-22022-09-03T06:50:53.927Z3600{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["arn:aws:iam:::user/testuser"]},"Action":["sts:AssumeRole"]}]}' 2022-09-03T06:52:39.375 DEBUG:boto:Method: POST 2022-09-03T06:52:39.376 DEBUG:boto:Path: / 2022-09-03T06:52:39.376 DEBUG:boto:Data: 2022-09-03T06:52:39.377 DEBUG:boto:Headers: {} 2022-09-03T06:52:39.377 DEBUG:boto:Host: smithi189.front.sepia.ceph.com 2022-09-03T06:52:39.378 DEBUG:boto:Port: 8000 2022-09-03T06:52:39.379 DEBUG:boto:Params: {'RoleName': 'roles-dsgvcy-2', 'Action': 'GetRole', 'Version': '2010-05-08'} 2022-09-03T06:52:39.380 DEBUG:boto:Token: None 2022-09-03T06:52:39.382 DEBUG:boto:CanonicalRequest: POST / host:smithi189.front.sepia.ceph.com:8000 x-amz-date:20220903T065239Z host;x-amz-date efc09ad2e77071df674d9987b9ba775760f7530993dec59449a862961fa77f08 2022-09-03T06:52:39.383 DEBUG:boto:StringToSign: AWS4-HMAC-SHA256 20220903T065239Z 20220903/front/smithi189/aws4_request 979e28c9ce3cde6e14db6caadbcf05d4813524f097c900281fb26aff50e1c98f 2022-09-03T06:52:39.384 DEBUG:boto:Signature: 908dd9aa7f4994720170af30f5daeb7c22d2f929b04cf04f2647a654b13eda2c 2022-09-03T06:52:39.385 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'X-Amz-Date': '20220903T065239Z', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'Content-Length': '57', 'Authorization': 'AWS4-HMAC-SHA256 Credential=SA4D9959RL1I1FMQ4Y2O/20220903/front/smithi189/aws4_request,SignedHeaders=host;x-amz-date,Signature=908dd9aa7f4994720170af30f5daeb7c22d2f929b04cf04f2647a654b13eda2c', 'Host': 'smithi189.front.sepia.ceph.com:8000'} 2022-09-03T06:52:39.388 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000b66f948955461491-006312f9b7-1a27-test-zone2'), ('Content-Type', 'application/xml'), ('Content-Length', '719'), ('Date', 'Sat, 03 Sep 2022 06:52:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:52:39.389 DEBUG:boto:b'tx00000b66f948955461491-006312f9b7-1a27-test-zone290db15e6-6472-4f84-b30f-37ff9bde26b9roles-dsgvcy-2/arn:aws:iam:::role/roles-dsgvcy-22022-09-03T06:50:53.927Z3600{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["arn:aws:iam:::user/testuser"]},"Action":["sts:AssumeRole"]}]}' 2022-09-03T06:52:39.391 DEBUG:tasks.rgw_multi.zone_rados:comparing role name=roles-dsgvcy-2 2022-09-03T06:52:39.392 INFO:tasks.rgw_multi.zone_rados:success, role identical: role=roles-dsgvcy-2 zones={test-zone1, test-zone2} 2022-09-03T06:52:39.393 INFO:tasks.rgw_multi.zone_rados:comparing role=roles-dsgvcy-2 zones={test-zone3, test-zone2} 2022-09-03T06:52:39.394 DEBUG:boto:Method: POST 2022-09-03T06:52:39.395 DEBUG:boto:Path: / 2022-09-03T06:52:39.395 DEBUG:boto:Data: 2022-09-03T06:52:39.396 DEBUG:boto:Headers: {} 2022-09-03T06:52:39.397 DEBUG:boto:Host: smithi181.front.sepia.ceph.com 2022-09-03T06:52:39.398 DEBUG:boto:Port: 8001 2022-09-03T06:52:39.398 DEBUG:boto:Params: {'RoleName': 'roles-dsgvcy-2', 'Action': 'GetRole', 'Version': '2010-05-08'} 2022-09-03T06:52:39.399 DEBUG:boto:Token: None 2022-09-03T06:52:39.400 DEBUG:boto:CanonicalRequest: POST / host:smithi181.front.sepia.ceph.com:8001 x-amz-date:20220903T065239Z host;x-amz-date efc09ad2e77071df674d9987b9ba775760f7530993dec59449a862961fa77f08 2022-09-03T06:52:39.401 DEBUG:boto:StringToSign: AWS4-HMAC-SHA256 20220903T065239Z 20220903/front/smithi181/aws4_request ede601fca9a13b7951193bc57dc10082b70518f6ab6ba1275b582149788a13ae 2022-09-03T06:52:39.401 DEBUG:boto:Signature: 7f8ad520bbf3572b749740bf5771da570c8d30852881d5245065726a53e9d358 2022-09-03T06:52:39.403 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'X-Amz-Date': '20220903T065239Z', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'Content-Length': '57', 'Authorization': 'AWS4-HMAC-SHA256 Credential=SA4D9959RL1I1FMQ4Y2O/20220903/front/smithi181/aws4_request,SignedHeaders=host;x-amz-date,Signature=7f8ad520bbf3572b749740bf5771da570c8d30852881d5245065726a53e9d358', 'Host': 'smithi181.front.sepia.ceph.com:8001'} 2022-09-03T06:52:39.408 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000004a5917209497a3c0-006312f9b7-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Content-Length', '719'), ('Date', 'Sat, 03 Sep 2022 06:52:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:52:39.409 DEBUG:boto:b'tx000004a5917209497a3c0-006312f9b7-1851-test-zone390db15e6-6472-4f84-b30f-37ff9bde26b9roles-dsgvcy-2/arn:aws:iam:::role/roles-dsgvcy-22022-09-03T06:50:53.945Z3600{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["arn:aws:iam:::user/testuser"]},"Action":["sts:AssumeRole"]}]}' 2022-09-03T06:52:39.423 DEBUG:boto:Method: POST 2022-09-03T06:52:39.424 DEBUG:boto:Path: / 2022-09-03T06:52:39.425 DEBUG:boto:Data: 2022-09-03T06:52:39.426 DEBUG:boto:Headers: {} 2022-09-03T06:52:39.427 DEBUG:boto:Host: smithi189.front.sepia.ceph.com 2022-09-03T06:52:39.428 DEBUG:boto:Port: 8000 2022-09-03T06:52:39.428 DEBUG:boto:Params: {'RoleName': 'roles-dsgvcy-2', 'Action': 'GetRole', 'Version': '2010-05-08'} 2022-09-03T06:52:39.429 DEBUG:boto:Token: None 2022-09-03T06:52:39.431 DEBUG:boto:CanonicalRequest: POST / host:smithi189.front.sepia.ceph.com:8000 x-amz-date:20220903T065239Z host;x-amz-date efc09ad2e77071df674d9987b9ba775760f7530993dec59449a862961fa77f08 2022-09-03T06:52:39.432 DEBUG:boto:StringToSign: AWS4-HMAC-SHA256 20220903T065239Z 20220903/front/smithi189/aws4_request 979e28c9ce3cde6e14db6caadbcf05d4813524f097c900281fb26aff50e1c98f 2022-09-03T06:52:39.433 DEBUG:boto:Signature: 908dd9aa7f4994720170af30f5daeb7c22d2f929b04cf04f2647a654b13eda2c 2022-09-03T06:52:39.435 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'X-Amz-Date': '20220903T065239Z', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'Content-Length': '57', 'Authorization': 'AWS4-HMAC-SHA256 Credential=SA4D9959RL1I1FMQ4Y2O/20220903/front/smithi189/aws4_request,SignedHeaders=host;x-amz-date,Signature=908dd9aa7f4994720170af30f5daeb7c22d2f929b04cf04f2647a654b13eda2c', 'Host': 'smithi189.front.sepia.ceph.com:8000'} 2022-09-03T06:52:39.437 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000554b2d469fe787f6-006312f9b7-1a27-test-zone2'), ('Content-Type', 'application/xml'), ('Content-Length', '719'), ('Date', 'Sat, 03 Sep 2022 06:52:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:52:39.438 DEBUG:boto:b'tx00000554b2d469fe787f6-006312f9b7-1a27-test-zone290db15e6-6472-4f84-b30f-37ff9bde26b9roles-dsgvcy-2/arn:aws:iam:::role/roles-dsgvcy-22022-09-03T06:50:53.927Z3600{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["arn:aws:iam:::user/testuser"]},"Action":["sts:AssumeRole"]}]}' 2022-09-03T06:52:39.440 DEBUG:tasks.rgw_multi.zone_rados:comparing role name=roles-dsgvcy-2 2022-09-03T06:52:39.441 INFO:tasks.rgw_multi.zone_rados:success, role identical: role=roles-dsgvcy-2 zones={test-zone3, test-zone2} 2022-09-03T06:52:39.443 INFO:tasks.rgw_multi.zone_rados:comparing role=roles-dsgvcy-3 zones={test-zone1, test-zone3} 2022-09-03T06:52:39.444 DEBUG:boto:Method: POST 2022-09-03T06:52:39.445 DEBUG:boto:Path: / 2022-09-03T06:52:39.445 DEBUG:boto:Data: 2022-09-03T06:52:39.446 DEBUG:boto:Headers: {} 2022-09-03T06:52:39.447 DEBUG:boto:Host: smithi181.front.sepia.ceph.com 2022-09-03T06:52:39.448 DEBUG:boto:Port: 8000 2022-09-03T06:52:39.448 DEBUG:boto:Params: {'RoleName': 'roles-dsgvcy-3', 'Action': 'GetRole', 'Version': '2010-05-08'} 2022-09-03T06:52:39.449 DEBUG:boto:Token: None 2022-09-03T06:52:39.450 DEBUG:boto:CanonicalRequest: POST / host:smithi181.front.sepia.ceph.com:8000 x-amz-date:20220903T065239Z host;x-amz-date 2d1623dd44a0a6afd0390499543e1b1478aa4853bd7290659563cb318f01ce26 2022-09-03T06:52:39.451 DEBUG:boto:StringToSign: AWS4-HMAC-SHA256 20220903T065239Z 20220903/front/smithi181/aws4_request baf53e129b7e07474a1537e353ef33cf471c83a77b8f9d90142657d67c931579 2022-09-03T06:52:39.451 DEBUG:boto:Signature: 5d99daa00575e999440e6ceec919b059f37d994b9b4361c3501a94d9f1e04520 2022-09-03T06:52:39.452 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'X-Amz-Date': '20220903T065239Z', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'Content-Length': '57', 'Authorization': 'AWS4-HMAC-SHA256 Credential=SA4D9959RL1I1FMQ4Y2O/20220903/front/smithi181/aws4_request,SignedHeaders=host;x-amz-date,Signature=5d99daa00575e999440e6ceec919b059f37d994b9b4361c3501a94d9f1e04520', 'Host': 'smithi181.front.sepia.ceph.com:8000'} 2022-09-03T06:52:39.453 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000da3fc79d3998bc62-006312f9b7-15d7-test-zone1'), ('Content-Type', 'application/xml'), ('Content-Length', '719'), ('Date', 'Sat, 03 Sep 2022 06:52:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:52:39.454 DEBUG:boto:b'tx00000da3fc79d3998bc62-006312f9b7-15d7-test-zone16765c729-46a2-41ec-89f5-2d6b47ba00cbroles-dsgvcy-3/arn:aws:iam:::role/roles-dsgvcy-32022-09-03T06:51:16.169Z3600{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["arn:aws:iam:::user/testuser"]},"Action":["sts:AssumeRole"]}]}' 2022-09-03T06:52:39.456 DEBUG:boto:Method: POST 2022-09-03T06:52:39.457 DEBUG:boto:Path: / 2022-09-03T06:52:39.457 DEBUG:boto:Data: 2022-09-03T06:52:39.458 DEBUG:boto:Headers: {} 2022-09-03T06:52:39.458 DEBUG:boto:Host: smithi181.front.sepia.ceph.com 2022-09-03T06:52:39.459 DEBUG:boto:Port: 8001 2022-09-03T06:52:39.459 DEBUG:boto:Params: {'RoleName': 'roles-dsgvcy-3', 'Action': 'GetRole', 'Version': '2010-05-08'} 2022-09-03T06:52:39.460 DEBUG:boto:Token: None 2022-09-03T06:52:39.460 DEBUG:boto:CanonicalRequest: POST / host:smithi181.front.sepia.ceph.com:8001 x-amz-date:20220903T065239Z host;x-amz-date 2d1623dd44a0a6afd0390499543e1b1478aa4853bd7290659563cb318f01ce26 2022-09-03T06:52:39.461 DEBUG:boto:StringToSign: AWS4-HMAC-SHA256 20220903T065239Z 20220903/front/smithi181/aws4_request a392e06dc66260b6731be6b1541ad9f5edb73721de0db3d3c78051590698b7ae 2022-09-03T06:52:39.462 DEBUG:boto:Signature: 611155b319b5375a5f9f20d0fe35f6718c27e8dc7af9785c0e1e0c4e230e7a91 2022-09-03T06:52:39.462 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'X-Amz-Date': '20220903T065239Z', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'Content-Length': '57', 'Authorization': 'AWS4-HMAC-SHA256 Credential=SA4D9959RL1I1FMQ4Y2O/20220903/front/smithi181/aws4_request,SignedHeaders=host;x-amz-date,Signature=611155b319b5375a5f9f20d0fe35f6718c27e8dc7af9785c0e1e0c4e230e7a91', 'Host': 'smithi181.front.sepia.ceph.com:8001'} 2022-09-03T06:52:39.465 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx0000065553399e40a270d-006312f9b7-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Content-Length', '719'), ('Date', 'Sat, 03 Sep 2022 06:52:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:52:39.465 DEBUG:boto:b'tx0000065553399e40a270d-006312f9b7-1851-test-zone36765c729-46a2-41ec-89f5-2d6b47ba00cbroles-dsgvcy-3/arn:aws:iam:::role/roles-dsgvcy-32022-09-03T06:51:16.169Z3600{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["arn:aws:iam:::user/testuser"]},"Action":["sts:AssumeRole"]}]}' 2022-09-03T06:52:39.467 DEBUG:tasks.rgw_multi.zone_rados:comparing role name=roles-dsgvcy-3 2022-09-03T06:52:39.467 INFO:tasks.rgw_multi.zone_rados:success, role identical: role=roles-dsgvcy-3 zones={test-zone1, test-zone3} 2022-09-03T06:52:39.468 INFO:tasks.rgw_multi.zone_rados:comparing role=roles-dsgvcy-3 zones={test-zone2, test-zone3} 2022-09-03T06:52:39.468 DEBUG:boto:Method: POST 2022-09-03T06:52:39.469 DEBUG:boto:Path: / 2022-09-03T06:52:39.469 DEBUG:boto:Data: 2022-09-03T06:52:39.470 DEBUG:boto:Headers: {} 2022-09-03T06:52:39.471 DEBUG:boto:Host: smithi189.front.sepia.ceph.com 2022-09-03T06:52:39.472 DEBUG:boto:Port: 8000 2022-09-03T06:52:39.472 DEBUG:boto:Params: {'RoleName': 'roles-dsgvcy-3', 'Action': 'GetRole', 'Version': '2010-05-08'} 2022-09-03T06:52:39.473 DEBUG:boto:Token: None 2022-09-03T06:52:39.474 DEBUG:boto:CanonicalRequest: POST / host:smithi189.front.sepia.ceph.com:8000 x-amz-date:20220903T065239Z host;x-amz-date 2d1623dd44a0a6afd0390499543e1b1478aa4853bd7290659563cb318f01ce26 2022-09-03T06:52:39.474 DEBUG:boto:StringToSign: AWS4-HMAC-SHA256 20220903T065239Z 20220903/front/smithi189/aws4_request fa9215a18e703f91e3bfd7b17fff69bde7aea277c91007ac6eeb605a38cfbb9f 2022-09-03T06:52:39.475 DEBUG:boto:Signature: 4a82ceabb63512a3c78095928c91a3052f51a3e2f5436c93e0b7002e3b602307 2022-09-03T06:52:39.475 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'X-Amz-Date': '20220903T065239Z', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'Content-Length': '57', 'Authorization': 'AWS4-HMAC-SHA256 Credential=SA4D9959RL1I1FMQ4Y2O/20220903/front/smithi189/aws4_request,SignedHeaders=host;x-amz-date,Signature=4a82ceabb63512a3c78095928c91a3052f51a3e2f5436c93e0b7002e3b602307', 'Host': 'smithi189.front.sepia.ceph.com:8000'} 2022-09-03T06:52:39.478 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000b5c0320e632eab4f-006312f9b7-1a27-test-zone2'), ('Content-Type', 'application/xml'), ('Content-Length', '719'), ('Date', 'Sat, 03 Sep 2022 06:52:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:52:39.480 DEBUG:boto:b'tx00000b5c0320e632eab4f-006312f9b7-1a27-test-zone26765c729-46a2-41ec-89f5-2d6b47ba00cbroles-dsgvcy-3/arn:aws:iam:::role/roles-dsgvcy-32022-09-03T06:52:17.412Z3600{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["arn:aws:iam:::user/testuser"]},"Action":["sts:AssumeRole"]}]}' 2022-09-03T06:52:39.481 DEBUG:boto:Method: POST 2022-09-03T06:52:39.483 DEBUG:boto:Path: / 2022-09-03T06:52:39.483 DEBUG:boto:Data: 2022-09-03T06:52:39.484 DEBUG:boto:Headers: {} 2022-09-03T06:52:39.485 DEBUG:boto:Host: smithi181.front.sepia.ceph.com 2022-09-03T06:52:39.487 DEBUG:boto:Port: 8001 2022-09-03T06:52:39.488 DEBUG:boto:Params: {'RoleName': 'roles-dsgvcy-3', 'Action': 'GetRole', 'Version': '2010-05-08'} 2022-09-03T06:52:39.489 DEBUG:boto:Token: None 2022-09-03T06:52:39.490 DEBUG:boto:CanonicalRequest: POST / host:smithi181.front.sepia.ceph.com:8001 x-amz-date:20220903T065239Z host;x-amz-date 2d1623dd44a0a6afd0390499543e1b1478aa4853bd7290659563cb318f01ce26 2022-09-03T06:52:39.491 DEBUG:boto:StringToSign: AWS4-HMAC-SHA256 20220903T065239Z 20220903/front/smithi181/aws4_request a392e06dc66260b6731be6b1541ad9f5edb73721de0db3d3c78051590698b7ae 2022-09-03T06:52:39.492 DEBUG:boto:Signature: 611155b319b5375a5f9f20d0fe35f6718c27e8dc7af9785c0e1e0c4e230e7a91 2022-09-03T06:52:39.493 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'X-Amz-Date': '20220903T065239Z', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'Content-Length': '57', 'Authorization': 'AWS4-HMAC-SHA256 Credential=SA4D9959RL1I1FMQ4Y2O/20220903/front/smithi181/aws4_request,SignedHeaders=host;x-amz-date,Signature=611155b319b5375a5f9f20d0fe35f6718c27e8dc7af9785c0e1e0c4e230e7a91', 'Host': 'smithi181.front.sepia.ceph.com:8001'} 2022-09-03T06:52:39.494 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000003965856f8aa952-006312f9b7-1851-test-zone3'), ('Content-Type', 'application/xml'), ('Content-Length', '719'), ('Date', 'Sat, 03 Sep 2022 06:52:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:52:39.495 DEBUG:boto:b'tx00000003965856f8aa952-006312f9b7-1851-test-zone36765c729-46a2-41ec-89f5-2d6b47ba00cbroles-dsgvcy-3/arn:aws:iam:::role/roles-dsgvcy-32022-09-03T06:51:16.169Z3600{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":["arn:aws:iam:::user/testuser"]},"Action":["sts:AssumeRole"]}]}' 2022-09-03T06:52:39.496 DEBUG:tasks.rgw_multi.zone_rados:comparing role name=roles-dsgvcy-3 2022-09-03T06:52:39.497 INFO:tasks.rgw_multi.zone_rados:success, role identical: role=roles-dsgvcy-3 zones={test-zone2, test-zone3} 2022-09-03T06:52:39.499 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_role_sync ... ok 2022-09-03T06:52:39.500 DEBUG:boto:path=/dsgvcy-76/ 2022-09-03T06:52:39.501 DEBUG:boto:auth_path=/dsgvcy-76/ 2022-09-03T06:52:39.501 DEBUG:boto:Method: PUT 2022-09-03T06:52:39.502 DEBUG:boto:Path: /dsgvcy-76/ 2022-09-03T06:52:39.503 DEBUG:boto:Data: 2022-09-03T06:52:39.503 DEBUG:boto:Headers: {} 2022-09-03T06:52:39.504 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:52:39.504 DEBUG:boto:Port: 8000 2022-09-03T06:52:39.504 DEBUG:boto:Params: {} 2022-09-03T06:52:39.505 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:52:39.506 DEBUG:boto:Token: None 2022-09-03T06:52:39.507 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 06:52:39 GMT /dsgvcy-76/ 2022-09-03T06:52:39.507 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:GjkfxwyFDVM9UD64e4qHtch1mYo= 2022-09-03T06:52:39.508 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:52:39 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:GjkfxwyFDVM9UD64e4qHtch1mYo=', 'Content-Length': '0'} 2022-09-03T06:52:39.535 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000fb3575acf8af5ed1-006312f9b7-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:52:39 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:52:39.536 DEBUG:rgw_multi.tests:created bucket=dsgvcy-76 2022-09-03T06:52:39.537 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:39.538 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:39.539 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:52:39.589 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:52:39.590 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:52:39.624 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T06:52:39.642 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T06:52:39.642 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:52:39.643 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:52:39.643 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:51:16.173767Z" 2022-09-03T06:52:39.644 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:52:39.644 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:52:39.644 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:52:39.645 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:50:53.931976Z" 2022-09-03T06:52:39.645 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:52:39.645 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:52:39.646 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:52:39.646 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:52:39.530444Z" 2022-09-03T06:52:39.646 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:52:39.646 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:52:39.647 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:52:39.647 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:52:39.647 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:52:39.648 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T06:52:39.648 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:52:39.649 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T06:52:39.649 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:39.650 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:39.651 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:52:39.701 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:52:39.701 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:52:39.729 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:39.727+0000 7fa5861fa600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:39.730 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:39.728+0000 7fa5861fa600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:39.743 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:52:39.744 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:52:39.745 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:52:39.746 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:52:39.746 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:52:39.747 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:52:39.747 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:39.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:39.748 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:52:39.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:39.748 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:52:39.748 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:39.749 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:39.749 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:52:39.749 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:39.750 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:52:39.750 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:39.750 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:52:39.751 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:39.751 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:39.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:39.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:39.752 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:52:39.752 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:39.752 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:39.753 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:52:39.753 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:39.753 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:39.754 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:39.754 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:52:39.754 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:39.755 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:39.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:39.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:39.756 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:52:39.756 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:39.757 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:39.757 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:52:39.758 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:39.758 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:39.758 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:39.759 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:52:39.759 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:39.760 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:39.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:39.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:39.761 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:52:39.762 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:39.762 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:39.763 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:52:39.764 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:39.764 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:39.764 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:39.765 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:52:39.765 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:52:39.766 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:39.766 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:39.766 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:52:39.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:39.767 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:52:39.768 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:52:39.768 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:52:39.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:39.769 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:52:39Z" 2022-09-03T06:52:39.769 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:52:39.771 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:52:39Z" } 2022-09-03T06:52:39.772 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:52:39.773 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:52:39.774 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:52:39.775 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone2 2022-09-03T06:52:39.776 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone3 2022-09-03T06:52:39.777 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:39.778 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone3', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:39.779 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 metadata sync status --rgw-zone test-zone3 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:52:39.832 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:52:39.833 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:52:39.892 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:52:39.880+0000 7f9241a1d600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:39.893 INFO:teuthology.orchestra.run.smithi181.stderr:2022-09-03T06:52:39.881+0000 7f9241a1d600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:39.899 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2022-09-03T06:52:39.899 INFO:teuthology.orchestra.run.smithi181.stdout: "sync_status": { 2022-09-03T06:52:39.900 INFO:teuthology.orchestra.run.smithi181.stdout: "info": { 2022-09-03T06:52:39.900 INFO:teuthology.orchestra.run.smithi181.stdout: "status": "sync", 2022-09-03T06:52:39.901 INFO:teuthology.orchestra.run.smithi181.stdout: "num_shards": 4, 2022-09-03T06:52:39.901 INFO:teuthology.orchestra.run.smithi181.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:52:39.901 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:52:39.902 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:52:39.902 INFO:teuthology.orchestra.run.smithi181.stdout: "markers": [ 2022-09-03T06:52:39.903 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:52:39.903 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 0, 2022-09-03T06:52:39.918 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:52:39.919 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:52:39.928 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:52:39.928 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:52:39.929 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 1, 2022-09-03T06:52:39.929 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:52:39.929 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:52:39.930 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:52:39.930 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:52:39.931 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:52:39.931 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:52:39.932 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 1, 2022-09-03T06:52:39.932 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:52:39.933 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:52:39.933 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:52:39.935 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:52:39.935 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:52:39.935 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:52:39.936 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:52:39.937 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:52:39.937 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:52:39.938 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:52:39.938 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:52:39.939 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 2, 2022-09-03T06:52:39.939 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:52:39.945 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:52:39.946 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:52:39.946 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:52:39.948 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:52:39.948 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:52:39.949 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:52:39.950 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 4 2022-09-03T06:52:39.950 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:52:39.951 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:52:39.951 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:52:39.951 INFO:teuthology.orchestra.run.smithi181.stdout: "key": 3, 2022-09-03T06:52:39.952 INFO:teuthology.orchestra.run.smithi181.stdout: "val": { 2022-09-03T06:52:39.952 INFO:teuthology.orchestra.run.smithi181.stdout: "state": 1, 2022-09-03T06:52:39.953 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:52:39.953 INFO:teuthology.orchestra.run.smithi181.stdout: "next_step_marker": "", 2022-09-03T06:52:39.954 INFO:teuthology.orchestra.run.smithi181.stdout: "total_entries": 0, 2022-09-03T06:52:39.954 INFO:teuthology.orchestra.run.smithi181.stdout: "pos": 0, 2022-09-03T06:52:39.955 INFO:teuthology.orchestra.run.smithi181.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:52:39.955 INFO:teuthology.orchestra.run.smithi181.stdout: "realm_epoch": 2 2022-09-03T06:52:39.956 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:52:39.956 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:52:39.957 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2022-09-03T06:52:39.957 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:52:39.958 INFO:teuthology.orchestra.run.smithi181.stdout: "full_sync": { 2022-09-03T06:52:39.960 INFO:teuthology.orchestra.run.smithi181.stdout: "total": 1, 2022-09-03T06:52:39.961 INFO:teuthology.orchestra.run.smithi181.stdout: "complete": 1 2022-09-03T06:52:39.961 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:52:39.962 INFO:teuthology.orchestra.run.smithi181.stdout: "current_time": "2022-09-03T06:52:39Z" 2022-09-03T06:52:39.962 INFO:teuthology.orchestra.run.smithi181.stdout:} 2022-09-03T06:52:39.965 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:52:39Z" } 2022-09-03T06:52:39.967 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:52:39.968 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:52:39.969 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:52:39.969 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone3 2022-09-03T06:52:39.970 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone4 2022-09-03T06:52:39.971 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:39.972 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone4', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:52:39.973 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:52:40.026 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:52:40.026 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:52:40.062 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:40.060+0000 7f432c66e600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:40.063 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:40.061+0000 7f432c66e600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:52:40.083 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:52:40.084 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:52:40.084 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:52:40.085 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:52:40.085 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:52:40.085 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:52:40.086 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:40.086 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:40.086 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:52:40.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:40.087 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:52:40.087 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:40.088 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:40.088 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:52:40.088 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:40.088 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:52:40.089 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:40.089 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:52:40.089 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:40.090 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:40.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:40.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:40.091 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:52:40.091 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:40.091 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:40.092 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:52:40.092 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:40.092 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:40.092 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:40.093 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:52:40.093 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:40.094 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:40.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:40.095 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:40.095 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:52:40.095 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:40.095 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:40.096 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:52:40.096 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:40.096 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:40.097 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:40.097 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:52:40.097 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:52:40.098 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:40.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:40.099 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:52:40.099 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:52:40.100 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:52:40.100 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:52:40.100 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:52:40.101 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:52:40.101 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:52:40.101 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:52:40.101 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:52:40.102 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:52:40.103 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:40.103 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:52:40.104 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:52:40.104 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:40.104 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:52:40.105 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:52:40.105 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:52:40.105 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:52:40.106 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:52:40Z" 2022-09-03T06:52:40.106 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:52:40.108 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:52:40Z" } 2022-09-03T06:52:40.109 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:52:40.110 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:52:40.110 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:52:40.111 INFO:rgw_multi.tests:finish meta checkpoint for zone=test-zone4 2022-09-03T06:52:40.112 DEBUG:tasks.rgw.c2.client.0:waiting for process to exit 2022-09-03T06:52:40.113 INFO:teuthology.orchestra.run:waiting for 300 2022-09-03T06:52:40.114 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:52:40.111+0000 7fcd34517700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term radosgw --rgw-frontends beast port=8000 -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm (PID: 154961) UID: 0 2022-09-03T06:52:40.115 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:52:40.111+0000 7fcd38782600 -1 shutting down 2022-09-03T06:52:40.790 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~0s 2022-09-03T06:52:46.216 INFO:tasks.rgw.c2.client.0:Stopped 2022-09-03T06:52:46.217 DEBUG:boto:path=/dsgvcy-76/ 2022-09-03T06:52:46.217 DEBUG:boto:auth_path=/dsgvcy-76/ 2022-09-03T06:52:46.218 DEBUG:boto:Method: HEAD 2022-09-03T06:52:46.218 DEBUG:boto:Path: /dsgvcy-76/ 2022-09-03T06:52:46.219 DEBUG:boto:Data: 2022-09-03T06:52:46.220 DEBUG:boto:Headers: {} 2022-09-03T06:52:46.221 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:52:46.222 DEBUG:boto:Port: 8000 2022-09-03T06:52:46.222 DEBUG:boto:Params: {} 2022-09-03T06:52:46.223 DEBUG:boto:Token: None 2022-09-03T06:52:46.224 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:52:46 GMT /dsgvcy-76/ 2022-09-03T06:52:46.225 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a1Fqy42Af7GhZMFeupHPvGtxGgM= 2022-09-03T06:52:46.225 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:52:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a1Fqy42Af7GhZMFeupHPvGtxGgM=', 'Content-Length': '0'} 2022-09-03T06:52:46.230 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '0'), ('X-RGW-Bytes-Used', '0'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ff007295f0c7b90f-006312f9be-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:52:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:52:46.232 DEBUG:boto:path=/dsgvcy-76/obj11 2022-09-03T06:52:46.233 DEBUG:boto:auth_path=/dsgvcy-76/obj11 2022-09-03T06:52:46.234 DEBUG:boto:Method: PUT 2022-09-03T06:52:46.236 DEBUG:boto:Path: /dsgvcy-76/obj11 2022-09-03T06:52:46.237 DEBUG:boto:Data: 2022-09-03T06:52:46.238 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:52:46.239 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:52:46.240 DEBUG:boto:Port: 8000 2022-09-03T06:52:46.241 DEBUG:boto:Params: {} 2022-09-03T06:52:46.241 DEBUG:boto:Token: None 2022-09-03T06:52:46.242 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:52:46 GMT /dsgvcy-76/obj11 2022-09-03T06:52:46.242 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:3mKrm2fNH1tBOwg1LXAYLosdHWU= 2022-09-03T06:52:46.243 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:52:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:3mKrm2fNH1tBOwg1LXAYLosdHWU='} 2022-09-03T06:52:46.256 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000682258202d078c4e-006312f9be-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:52:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:52:46.257 DEBUG:boto:path=/dsgvcy-76/ 2022-09-03T06:52:46.258 DEBUG:boto:auth_path=/dsgvcy-76/ 2022-09-03T06:52:46.259 DEBUG:boto:Method: HEAD 2022-09-03T06:52:46.260 DEBUG:boto:Path: /dsgvcy-76/ 2022-09-03T06:52:46.261 DEBUG:boto:Data: 2022-09-03T06:52:46.262 DEBUG:boto:Headers: {} 2022-09-03T06:52:46.263 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:52:46.264 DEBUG:boto:Port: 8000 2022-09-03T06:52:46.265 DEBUG:boto:Params: {} 2022-09-03T06:52:46.266 DEBUG:boto:Token: None 2022-09-03T06:52:46.267 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:52:46 GMT /dsgvcy-76/ 2022-09-03T06:52:46.269 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a1Fqy42Af7GhZMFeupHPvGtxGgM= 2022-09-03T06:52:46.270 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:52:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a1Fqy42Af7GhZMFeupHPvGtxGgM=', 'Content-Length': '0'} 2022-09-03T06:52:46.274 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '1'), ('X-RGW-Bytes-Used', '3'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx0000044bfe92493c85f2a-006312f9be-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:52:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:52:46.275 DEBUG:boto:path=/dsgvcy-76/obj22 2022-09-03T06:52:46.275 DEBUG:boto:auth_path=/dsgvcy-76/obj22 2022-09-03T06:52:46.276 DEBUG:boto:Method: PUT 2022-09-03T06:52:46.276 DEBUG:boto:Path: /dsgvcy-76/obj22 2022-09-03T06:52:46.277 DEBUG:boto:Data: 2022-09-03T06:52:46.277 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:52:46.278 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:52:46.278 DEBUG:boto:Port: 8000 2022-09-03T06:52:46.279 DEBUG:boto:Params: {} 2022-09-03T06:52:46.280 DEBUG:boto:Token: None 2022-09-03T06:52:46.280 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:52:46 GMT /dsgvcy-76/obj22 2022-09-03T06:52:46.281 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:bA/M6YKswlWVpdjk0V7XSG3hYHQ= 2022-09-03T06:52:46.281 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:52:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:bA/M6YKswlWVpdjk0V7XSG3hYHQ='} 2022-09-03T06:52:46.288 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000b02025e5dcb26855-006312f9be-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:52:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:52:46.289 DEBUG:boto:path=/dsgvcy-76/ 2022-09-03T06:52:46.289 DEBUG:boto:auth_path=/dsgvcy-76/ 2022-09-03T06:52:46.290 DEBUG:boto:Method: HEAD 2022-09-03T06:52:46.290 DEBUG:boto:Path: /dsgvcy-76/ 2022-09-03T06:52:46.291 DEBUG:boto:Data: 2022-09-03T06:52:46.291 DEBUG:boto:Headers: {} 2022-09-03T06:52:46.292 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:52:46.293 DEBUG:boto:Port: 8000 2022-09-03T06:52:46.293 DEBUG:boto:Params: {} 2022-09-03T06:52:46.294 DEBUG:boto:Token: None 2022-09-03T06:52:46.294 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:52:46 GMT /dsgvcy-76/ 2022-09-03T06:52:46.295 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a1Fqy42Af7GhZMFeupHPvGtxGgM= 2022-09-03T06:52:46.296 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:52:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a1Fqy42Af7GhZMFeupHPvGtxGgM=', 'Content-Length': '0'} 2022-09-03T06:52:46.301 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '2'), ('X-RGW-Bytes-Used', '6'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000b3adacadcf93b9ca-006312f9be-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:52:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:52:46.305 DEBUG:boto:path=/dsgvcy-76/obj33 2022-09-03T06:52:46.306 DEBUG:boto:auth_path=/dsgvcy-76/obj33 2022-09-03T06:52:46.306 DEBUG:boto:Method: PUT 2022-09-03T06:52:46.307 DEBUG:boto:Path: /dsgvcy-76/obj33 2022-09-03T06:52:46.307 DEBUG:boto:Data: 2022-09-03T06:52:46.308 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:52:46.308 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:52:46.309 DEBUG:boto:Port: 8000 2022-09-03T06:52:46.309 DEBUG:boto:Params: {} 2022-09-03T06:52:46.311 DEBUG:boto:Token: None 2022-09-03T06:52:46.311 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:52:46 GMT /dsgvcy-76/obj33 2022-09-03T06:52:46.312 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7dz5SBvE6S68Ak2iKeI9ubuXYgM= 2022-09-03T06:52:46.313 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:52:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7dz5SBvE6S68Ak2iKeI9ubuXYgM='} 2022-09-03T06:52:46.362 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx00000999ccd05243f7d33-006312f9be-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:52:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:52:46.365 DEBUG:boto:path=/dsgvcy-76/ 2022-09-03T06:52:46.366 DEBUG:boto:auth_path=/dsgvcy-76/ 2022-09-03T06:52:46.368 DEBUG:boto:Method: HEAD 2022-09-03T06:52:46.368 DEBUG:boto:Path: /dsgvcy-76/ 2022-09-03T06:52:46.369 DEBUG:boto:Data: 2022-09-03T06:52:46.369 DEBUG:boto:Headers: {} 2022-09-03T06:52:46.370 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:52:46.371 DEBUG:boto:Port: 8000 2022-09-03T06:52:46.371 DEBUG:boto:Params: {} 2022-09-03T06:52:46.372 DEBUG:boto:Token: None 2022-09-03T06:52:46.372 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:52:46 GMT /dsgvcy-76/ 2022-09-03T06:52:46.375 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a1Fqy42Af7GhZMFeupHPvGtxGgM= 2022-09-03T06:52:46.376 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:52:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a1Fqy42Af7GhZMFeupHPvGtxGgM=', 'Content-Length': '0'} 2022-09-03T06:52:46.380 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '3'), ('X-RGW-Bytes-Used', '9'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx000003f059d0496f9b663-006312f9be-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:52:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:52:46.381 DEBUG:boto:path=/dsgvcy-76/obj44 2022-09-03T06:52:46.381 DEBUG:boto:auth_path=/dsgvcy-76/obj44 2022-09-03T06:52:46.382 DEBUG:boto:Method: PUT 2022-09-03T06:52:46.383 DEBUG:boto:Path: /dsgvcy-76/obj44 2022-09-03T06:52:46.383 DEBUG:boto:Data: 2022-09-03T06:52:46.384 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:52:46.384 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:52:46.385 DEBUG:boto:Port: 8000 2022-09-03T06:52:46.386 DEBUG:boto:Params: {} 2022-09-03T06:52:46.387 DEBUG:boto:Token: None 2022-09-03T06:52:46.389 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:52:46 GMT /dsgvcy-76/obj44 2022-09-03T06:52:46.390 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:Js4EBbtXblkxCe5FIi//0pKsjK8= 2022-09-03T06:52:46.391 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:52:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:Js4EBbtXblkxCe5FIi//0pKsjK8='} 2022-09-03T06:52:46.397 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~6s 2022-09-03T06:52:46.399 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx0000073a2d758c1468995-006312f9be-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:52:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:52:46.399 DEBUG:boto:path=/dsgvcy-76/ 2022-09-03T06:52:46.400 DEBUG:boto:auth_path=/dsgvcy-76/ 2022-09-03T06:52:46.401 DEBUG:boto:Method: HEAD 2022-09-03T06:52:46.401 DEBUG:boto:Path: /dsgvcy-76/ 2022-09-03T06:52:46.402 DEBUG:boto:Data: 2022-09-03T06:52:46.402 DEBUG:boto:Headers: {} 2022-09-03T06:52:46.404 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:52:46.406 DEBUG:boto:Port: 8000 2022-09-03T06:52:46.407 DEBUG:boto:Params: {} 2022-09-03T06:52:46.409 DEBUG:boto:Token: None 2022-09-03T06:52:46.410 DEBUG:boto:StringToSign: HEAD Sat, 03 Sep 2022 06:52:46 GMT /dsgvcy-76/ 2022-09-03T06:52:46.411 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:a1Fqy42Af7GhZMFeupHPvGtxGgM= 2022-09-03T06:52:46.411 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:52:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:a1Fqy42Af7GhZMFeupHPvGtxGgM=', 'Content-Length': '0'} 2022-09-03T06:52:46.418 DEBUG:boto:Response headers: [('X-RGW-Object-Count', '4'), ('X-RGW-Bytes-Used', '12'), ('X-RGW-Quota-User-Size', '-1'), ('X-RGW-Quota-User-Objects', '-1'), ('X-RGW-Quota-Max-Buckets', '1000'), ('X-RGW-Quota-Bucket-Size', '-1'), ('X-RGW-Quota-Bucket-Objects', '-1'), ('x-amz-request-id', 'tx00000ea60b122ba9b9e04-006312f9be-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:52:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:52:46.419 DEBUG:boto:path=/dsgvcy-76/obj55 2022-09-03T06:52:46.419 DEBUG:boto:auth_path=/dsgvcy-76/obj55 2022-09-03T06:52:46.420 DEBUG:boto:Method: PUT 2022-09-03T06:52:46.421 DEBUG:boto:Path: /dsgvcy-76/obj55 2022-09-03T06:52:46.421 DEBUG:boto:Data: 2022-09-03T06:52:46.422 DEBUG:boto:Headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue'} 2022-09-03T06:52:46.423 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:52:46.423 DEBUG:boto:Port: 8000 2022-09-03T06:52:46.425 DEBUG:boto:Params: {} 2022-09-03T06:52:46.426 DEBUG:boto:Token: None 2022-09-03T06:52:46.427 DEBUG:boto:StringToSign: PUT rL0Y20zC+Fzt72VPzMSk2A== application/octet-stream Sat, 03 Sep 2022 06:52:46 GMT /dsgvcy-76/obj55 2022-09-03T06:52:46.427 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:OPy3ipz449xQO1zuP+xiEzT3GB0= 2022-09-03T06:52:46.428 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Content-Type': 'application/octet-stream', 'Content-MD5': 'rL0Y20zC+Fzt72VPzMSk2A==', 'Content-Length': '3', 'Expect': '100-Continue', 'Date': 'Sat, 03 Sep 2022 06:52:46 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:OPy3ipz449xQO1zuP+xiEzT3GB0='} 2022-09-03T06:52:46.471 DEBUG:boto:Response headers: [('Content-Length', '0'), ('ETag', '"acbd18db4cc2f85cedef654fccc4a4d8"'), ('Accept-Ranges', 'bytes'), ('x-amz-request-id', 'tx000001d654ee656a921ee-006312f9be-15d7-test-zone1'), ('Date', 'Sat, 03 Sep 2022 06:52:46 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:52:46.472 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['data', 'sync', 'init', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:52:46.475 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'data', 'sync', 'init', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--source-zone', 'test-zone1', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0'] 2022-09-03T06:52:46.476 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 data sync init --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --source-zone test-zone1 --cluster c2 --debug-rgw 0 --debug-ms 0 2022-09-03T06:52:46.539 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:52:46.539 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:52:46.577 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:46.573+0000 7fe01cfa9600 10 RGW-SYNC:data:init_data_sync_status: took lease 2022-09-03T06:52:46.599 INFO:tasks.rgw.c2.client.0:Restarting daemon 2022-09-03T06:52:46.601 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=8000' -n client.0 --cluster c2 -k /etc/ceph/c2.client.0.keyring --log-file /var/log/ceph/rgw.c2.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.0.sock --foreground --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm | sudo tee /var/log/ceph/rgw.c2.client.0.stdout 2>&1 2022-09-03T06:52:46.604 INFO:tasks.rgw.c2.client.0:Started 2022-09-03T06:52:46.605 DEBUG:teuthology.orchestra.run.smithi189:> curl http://smithi189.front.sepia.ceph.com:8000 2022-09-03T06:52:46.628 INFO:teuthology.orchestra.run.smithi189.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T06:52:46.628 INFO:teuthology.orchestra.run.smithi189.stderr: Dload Upload Total Spent Left Speed 2022-09-03T06:52:46.629 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T06:52:46.632 INFO:teuthology.orchestra.run.smithi189.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi189.front.sepia.ceph.com port 8000: Connection refused 2022-09-03T06:52:47.634 DEBUG:teuthology.orchestra.run.smithi189:> curl http://smithi189.front.sepia.ceph.com:8000 2022-09-03T06:52:47.660 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T06:52:47.662 INFO:teuthology.orchestra.run.smithi189.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T06:52:47.662 INFO:teuthology.orchestra.run.smithi189.stderr: Dload Upload Total Spent Left Speed 2022-09-03T06:52:47.662 INFO:teuthology.orchestra.run.smithi189.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi189.front.sepia.ceph.com port 8000: Connection refused 2022-09-03T06:52:49.664 DEBUG:teuthology.orchestra.run.smithi189:> curl http://smithi189.front.sepia.ceph.com:8000 2022-09-03T06:52:49.695 DEBUG:teuthology.orchestra.run:got remote process result: 7 2022-09-03T06:52:49.697 INFO:teuthology.orchestra.run.smithi189.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T06:52:49.698 INFO:teuthology.orchestra.run.smithi189.stderr: Dload Upload Total Spent Left Speed 2022-09-03T06:52:49.699 INFO:teuthology.orchestra.run.smithi189.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to smithi189.front.sepia.ceph.com port 8000: Connection refused 2022-09-03T06:52:52.107 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 has been restored 2022-09-03T06:52:53.700 DEBUG:teuthology.orchestra.run.smithi189:> curl http://smithi189.front.sepia.ceph.com:8000 2022-09-03T06:52:53.726 INFO:teuthology.orchestra.run.smithi189.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2022-09-03T06:52:53.726 INFO:teuthology.orchestra.run.smithi189.stderr: Dload Upload Total Spent Left Speed 2022-09-03T06:52:53.727 INFO:teuthology.orchestra.run.smithi189.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 208k 0 --:--:-- --:--:-- --:--:-- 208k 2022-09-03T06:52:53.728 INFO:teuthology.orchestra.run.smithi189.stdout:anonymous 2022-09-03T06:52:53.728 DEBUG:rgw_multi.tests:bucket checkpoint: source=test-zone1 target=test-zone2 bucket=dsgvcy-76 2022-09-03T06:52:53.729 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-76', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:52:53.729 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'bucket', 'sync', 'checkpoint', '--bucket', 'dsgvcy-76', '--source-zone', 'test-zone1', '--retry-delay-ms', '5000', '--timeout-sec', '300', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '1', '--debug-ms', '0'] 2022-09-03T06:52:53.730 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 bucket sync checkpoint --bucket dsgvcy-76 --source-zone test-zone1 --retry-delay-ms 5000 --timeout-sec 300 --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 1 --debug-ms 0 2022-09-03T06:52:53.828 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:52:53.828 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:52:53.871 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:53.870+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:52:58.872 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:52:58.870+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:53:03.874 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:53:03.871+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:53:08.873 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:53:08.871+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:53:13.874 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:53:13.872+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:53:18.875 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:53:18.872+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:53:23.875 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:53:23.873+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:53:28.877 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:53:28.874+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:53:33.878 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:53:33.874+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:53:38.877 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:53:38.875+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:53:43.878 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:53:43.875+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:53:48.878 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:53:48.876+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:53:53.891 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:53:53.877+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:53:58.880 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:53:58.877+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:54:03.880 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:54:03.878+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:54:08.891 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:54:08.879+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:54:13.881 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:54:13.879+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:54:18.881 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:54:18.880+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:54:23.886 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:54:23.880+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:54:27.869 INFO:tasks.rgw.c2.client.0.smithi189.stdout:src/tcmalloc.cc:332] Attempt to free invalid pointer 0x55b300000000 2022-09-03T06:54:27.869 INFO:tasks.rgw.c2.client.0.smithi189.stdout:*** Caught signal (Aborted) ** 2022-09-03T06:54:27.870 INFO:tasks.rgw.c2.client.0.smithi189.stdout: in thread 7fb605cca700 thread_name:data-sync 2022-09-03T06:54:27.873 INFO:tasks.rgw.c2.client.0.smithi189.stdout: ceph version 17.0.0-14661-g4d487742 (4d487742eda56d4badbc9bdb33761243365bb37a) quincy (dev) 2022-09-03T06:54:27.874 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fb62d7cfcf0] 2022-09-03T06:54:27.874 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 2: gsignal() 2022-09-03T06:54:27.874 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 3: abort() 2022-09-03T06:54:27.875 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 4: /lib64/libtcmalloc.so.4(+0x18499) [0x7fb62e576499] 2022-09-03T06:54:27.875 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 5: /lib64/libtcmalloc.so.4(+0x19d79) [0x7fb62e577d79] 2022-09-03T06:54:27.876 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 6: (std::_Rb_tree::marker_entry>, std::_Select1st::marker_entry> >, std::less, std::allocator::marker_entry> > >::_M_erase_aux(std::_Rb_tree_const_iterator::marker_entry> >, std::_Rb_tree_const_iterator::marker_entry> >)+0x80) [0x55b3d4e15ff0] 2022-09-03T06:54:27.877 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 7: (RGWBucketSyncSingleEntryCR::operate(DoutPrefixProvider const*)+0x8fe) [0x55b3d4e3abfe] 2022-09-03T06:54:27.877 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 8: (RGWCoroutinesStack::operate(DoutPrefixProvider const*, RGWCoroutinesEnv*)+0x18f) [0x55b3d497b2af] 2022-09-03T06:54:27.877 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 9: (RGWCoroutinesManager::run(DoutPrefixProvider const*, std::__cxx11::list >&)+0x521) [0x55b3d497ba81] 2022-09-03T06:54:27.878 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 10: (RGWCoroutinesManager::run(DoutPrefixProvider const*, RGWCoroutine*)+0x91) [0x55b3d497ca91] 2022-09-03T06:54:27.878 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 11: (RGWRemoteDataLog::run_sync(DoutPrefixProvider const*, int)+0x20a) [0x55b3d4de0a4a] 2022-09-03T06:54:27.879 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 12: (RGWDataSyncProcessorThread::process(DoutPrefixProvider const*)+0x58) [0x55b3d4a58ce8] 2022-09-03T06:54:27.879 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 13: (RGWRadosThread::Worker::entry()+0xb3) [0x55b3d4a22f03] 2022-09-03T06:54:27.879 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 14: /lib64/libpthread.so.0(+0x81ca) [0x7fb62d7c51ca] 2022-09-03T06:54:27.879 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 15: clone() 2022-09-03T06:54:27.880 INFO:tasks.rgw.c2.client.0.smithi189.stdout:2022-09-03T06:54:27.871+0000 7fb605cca700 -1 *** Caught signal (Aborted) ** 2022-09-03T06:54:27.880 INFO:tasks.rgw.c2.client.0.smithi189.stdout: in thread 7fb605cca700 thread_name:data-sync 2022-09-03T06:54:27.881 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 2022-09-03T06:54:27.881 INFO:tasks.rgw.c2.client.0.smithi189.stdout: ceph version 17.0.0-14661-g4d487742 (4d487742eda56d4badbc9bdb33761243365bb37a) quincy (dev) 2022-09-03T06:54:27.881 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fb62d7cfcf0] 2022-09-03T06:54:27.882 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 2: gsignal() 2022-09-03T06:54:27.882 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 3: abort() 2022-09-03T06:54:27.882 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 4: /lib64/libtcmalloc.so.4(+0x18499) [0x7fb62e576499] 2022-09-03T06:54:27.883 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 5: /lib64/libtcmalloc.so.4(+0x19d79) [0x7fb62e577d79] 2022-09-03T06:54:27.885 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 6: (std::_Rb_tree::marker_entry>, std::_Select1st::marker_entry> >, std::less, std::allocator::marker_entry> > >::_M_erase_aux(std::_Rb_tree_const_iterator::marker_entry> >, std::_Rb_tree_const_iterator::marker_entry> >)+0x80) [0x55b3d4e15ff0] 2022-09-03T06:54:27.885 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 7: (RGWBucketSyncSingleEntryCR::operate(DoutPrefixProvider const*)+0x8fe) [0x55b3d4e3abfe] 2022-09-03T06:54:27.886 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 8: (RGWCoroutinesStack::operate(DoutPrefixProvider const*, RGWCoroutinesEnv*)+0x18f) [0x55b3d497b2af] 2022-09-03T06:54:27.887 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 9: (RGWCoroutinesManager::run(DoutPrefixProvider const*, std::__cxx11::list >&)+0x521) [0x55b3d497ba81] 2022-09-03T06:54:27.887 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 10: (RGWCoroutinesManager::run(DoutPrefixProvider const*, RGWCoroutine*)+0x91) [0x55b3d497ca91] 2022-09-03T06:54:27.887 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 11: (RGWRemoteDataLog::run_sync(DoutPrefixProvider const*, int)+0x20a) [0x55b3d4de0a4a] 2022-09-03T06:54:27.887 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 12: (RGWDataSyncProcessorThread::process(DoutPrefixProvider const*)+0x58) [0x55b3d4a58ce8] 2022-09-03T06:54:27.888 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 13: (RGWRadosThread::Worker::entry()+0xb3) [0x55b3d4a22f03] 2022-09-03T06:54:27.888 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 14: /lib64/libpthread.so.0(+0x81ca) [0x7fb62d7c51ca] 2022-09-03T06:54:27.888 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 15: clone() 2022-09-03T06:54:27.889 INFO:tasks.rgw.c2.client.0.smithi189.stdout: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2022-09-03T06:54:27.889 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 2022-09-03T06:54:27.890 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 0> 2022-09-03T06:54:27.871+0000 7fb605cca700 -1 *** Caught signal (Aborted) ** 2022-09-03T06:54:27.890 INFO:tasks.rgw.c2.client.0.smithi189.stdout: in thread 7fb605cca700 thread_name:data-sync 2022-09-03T06:54:27.890 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 2022-09-03T06:54:27.891 INFO:tasks.rgw.c2.client.0.smithi189.stdout: ceph version 17.0.0-14661-g4d487742 (4d487742eda56d4badbc9bdb33761243365bb37a) quincy (dev) 2022-09-03T06:54:27.891 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fb62d7cfcf0] 2022-09-03T06:54:27.891 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 2: gsignal() 2022-09-03T06:54:27.891 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 3: abort() 2022-09-03T06:54:27.892 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 4: /lib64/libtcmalloc.so.4(+0x18499) [0x7fb62e576499] 2022-09-03T06:54:27.892 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 5: /lib64/libtcmalloc.so.4(+0x19d79) [0x7fb62e577d79] 2022-09-03T06:54:27.892 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 6: (std::_Rb_tree::marker_entry>, std::_Select1st::marker_entry> >, std::less, std::allocator::marker_entry> > >::_M_erase_aux(std::_Rb_tree_const_iterator::marker_entry> >, std::_Rb_tree_const_iterator::marker_entry> >)+0x80) [0x55b3d4e15ff0] 2022-09-03T06:54:27.892 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 7: (RGWBucketSyncSingleEntryCR::operate(DoutPrefixProvider const*)+0x8fe) [0x55b3d4e3abfe] 2022-09-03T06:54:27.893 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 8: (RGWCoroutinesStack::operate(DoutPrefixProvider const*, RGWCoroutinesEnv*)+0x18f) [0x55b3d497b2af] 2022-09-03T06:54:27.893 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 9: (RGWCoroutinesManager::run(DoutPrefixProvider const*, std::__cxx11::list >&)+0x521) [0x55b3d497ba81] 2022-09-03T06:54:27.893 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 10: (RGWCoroutinesManager::run(DoutPrefixProvider const*, RGWCoroutine*)+0x91) [0x55b3d497ca91] 2022-09-03T06:54:27.893 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 11: (RGWRemoteDataLog::run_sync(DoutPrefixProvider const*, int)+0x20a) [0x55b3d4de0a4a] 2022-09-03T06:54:27.894 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 12: (RGWDataSyncProcessorThread::process(DoutPrefixProvider const*)+0x58) [0x55b3d4a58ce8] 2022-09-03T06:54:27.894 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 13: (RGWRadosThread::Worker::entry()+0xb3) [0x55b3d4a22f03] 2022-09-03T06:54:27.894 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 14: /lib64/libpthread.so.0(+0x81ca) [0x7fb62d7c51ca] 2022-09-03T06:54:27.895 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 15: clone() 2022-09-03T06:54:27.895 INFO:tasks.rgw.c2.client.0.smithi189.stdout: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2022-09-03T06:54:27.896 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 2022-09-03T06:54:27.901 INFO:tasks.rgw.c2.client.0.smithi189.stdout: -91> 2022-09-03T06:54:27.871+0000 7fb605cca700 -1 *** Caught signal (Aborted) ** 2022-09-03T06:54:27.901 INFO:tasks.rgw.c2.client.0.smithi189.stdout: in thread 7fb605cca700 thread_name:data-sync 2022-09-03T06:54:27.901 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 2022-09-03T06:54:27.902 INFO:tasks.rgw.c2.client.0.smithi189.stdout: ceph version 17.0.0-14661-g4d487742 (4d487742eda56d4badbc9bdb33761243365bb37a) quincy (dev) 2022-09-03T06:54:27.902 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fb62d7cfcf0] 2022-09-03T06:54:27.902 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 2: gsignal() 2022-09-03T06:54:27.903 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 3: abort() 2022-09-03T06:54:27.903 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 4: /lib64/libtcmalloc.so.4(+0x18499) [0x7fb62e576499] 2022-09-03T06:54:27.903 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 5: /lib64/libtcmalloc.so.4(+0x19d79) [0x7fb62e577d79] 2022-09-03T06:54:27.904 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 6: (std::_Rb_tree::marker_entry>, std::_Select1st::marker_entry> >, std::less, std::allocator::marker_entry> > >::_M_erase_aux(std::_Rb_tree_const_iterator::marker_entry> >, std::_Rb_tree_const_iterator::marker_entry> >)+0x80) [0x55b3d4e15ff0] 2022-09-03T06:54:27.904 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 7: (RGWBucketSyncSingleEntryCR::operate(DoutPrefixProvider const*)+0x8fe) [0x55b3d4e3abfe] 2022-09-03T06:54:27.905 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 8: (RGWCoroutinesStack::operate(DoutPrefixProvider const*, RGWCoroutinesEnv*)+0x18f) [0x55b3d497b2af] 2022-09-03T06:54:27.905 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 9: (RGWCoroutinesManager::run(DoutPrefixProvider const*, std::__cxx11::list >&)+0x521) [0x55b3d497ba81] 2022-09-03T06:54:27.906 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 10: (RGWCoroutinesManager::run(DoutPrefixProvider const*, RGWCoroutine*)+0x91) [0x55b3d497ca91] 2022-09-03T06:54:27.906 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 11: (RGWRemoteDataLog::run_sync(DoutPrefixProvider const*, int)+0x20a) [0x55b3d4de0a4a] 2022-09-03T06:54:27.907 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 12: (RGWDataSyncProcessorThread::process(DoutPrefixProvider const*)+0x58) [0x55b3d4a58ce8] 2022-09-03T06:54:27.907 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 13: (RGWRadosThread::Worker::entry()+0xb3) [0x55b3d4a22f03] 2022-09-03T06:54:27.907 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 14: /lib64/libpthread.so.0(+0x81ca) [0x7fb62d7c51ca] 2022-09-03T06:54:27.908 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 15: clone() 2022-09-03T06:54:27.908 INFO:tasks.rgw.c2.client.0.smithi189.stdout: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2022-09-03T06:54:27.909 INFO:tasks.rgw.c2.client.0.smithi189.stdout: 2022-09-03T06:54:28.001 INFO:tasks.rgw.c2.client.0.smithi189.stderr:daemon-helper: command crashed with signal 6 2022-09-03T06:54:28.888 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:54:28.881+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:54:29.066 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~0s 2022-09-03T06:54:33.884 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:54:33.881+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:54:34.673 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~6s 2022-09-03T06:54:38.884 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:54:38.882+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:54:40.278 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~11s 2022-09-03T06:54:43.885 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:54:43.883+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:54:45.891 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~17s 2022-09-03T06:54:48.885 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:54:48.883+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:54:51.498 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~22s 2022-09-03T06:54:53.886 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:54:53.884+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:54:57.108 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~28s 2022-09-03T06:54:58.887 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:54:58.884+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:55:02.713 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~34s 2022-09-03T06:55:03.889 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:55:03.884+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:55:08.322 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~39s 2022-09-03T06:55:08.887 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:55:08.885+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:55:13.888 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:55:13.886+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:55:13.929 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~45s 2022-09-03T06:55:18.889 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:55:18.887+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:55:19.536 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~50s 2022-09-03T06:55:23.889 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:55:23.887+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:55:25.143 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~56s 2022-09-03T06:55:28.889 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:55:28.888+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:55:30.751 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~62s 2022-09-03T06:55:33.891 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:55:33.889+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:55:36.357 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~67s 2022-09-03T06:55:38.891 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:55:38.889+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:55:41.964 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~73s 2022-09-03T06:55:43.891 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:55:43.890+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:55:47.570 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~79s 2022-09-03T06:55:48.892 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:55:48.890+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:55:53.176 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~84s 2022-09-03T06:55:53.893 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:55:53.890+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:55:58.782 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~90s 2022-09-03T06:55:58.894 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:55:58.892+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:56:03.894 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:56:03.893+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:56:04.388 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~95s 2022-09-03T06:56:08.895 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:56:08.893+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:56:09.998 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~101s 2022-09-03T06:56:13.896 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:56:13.894+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:56:15.605 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~107s 2022-09-03T06:56:18.896 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:56:18.894+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:56:21.212 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~112s 2022-09-03T06:56:23.897 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:56:23.895+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:56:26.820 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~118s 2022-09-03T06:56:28.897 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:56:28.896+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:56:32.427 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~123s 2022-09-03T06:56:33.898 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:56:33.896+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:56:38.033 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~129s 2022-09-03T06:56:38.898 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:56:38.897+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:56:43.640 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~135s 2022-09-03T06:56:43.899 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:56:43.897+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:56:48.912 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:56:48.897+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:56:49.247 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~140s 2022-09-03T06:56:53.900 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:56:53.898+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:56:54.854 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~146s 2022-09-03T06:56:58.901 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:56:58.899+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:57:00.461 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~151s 2022-09-03T06:57:03.902 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:57:03.900+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:57:06.068 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~157s 2022-09-03T06:57:08.903 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:57:08.901+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:57:11.675 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~163s 2022-09-03T06:57:13.903 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:57:13.901+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:57:17.280 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~168s 2022-09-03T06:57:18.904 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:57:18.902+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:57:22.886 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~174s 2022-09-03T06:57:23.904 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:57:23.903+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:57:28.494 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~179s 2022-09-03T06:57:28.905 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:57:28.903+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:57:33.906 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:57:33.904+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:57:34.102 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~185s 2022-09-03T06:57:38.906 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:57:38.904+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:57:39.713 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~191s 2022-09-03T06:57:43.907 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:57:43.905+0000 7f8975dd7600 1 waiting to reach incremental sync.. 2022-09-03T06:57:45.318 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~196s 2022-09-03T06:57:48.908 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:57:48.906+0000 7f8975dd7600 -1 bucket checkpoint timed out waiting to reach incremental sync 2022-09-03T06:57:48.908 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:57:48.906+0000 7f8975dd7600 0 bucket sync checkpoint failed: (110) Connection timed out 2022-09-03T06:57:48.909 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:57:48.906+0000 7f8975dd7600 -1 bucket sync checkpoint failed: (110) Connection timed out 2022-09-03T06:57:48.925 DEBUG:teuthology.orchestra.run:got remote process result: 110 2022-09-03T06:57:48.931 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_full_sync_after_data_sync_init ... FAIL 2022-09-03T06:57:48.934 DEBUG:boto:path=/dsgvcy-77/ 2022-09-03T06:57:48.938 DEBUG:boto:auth_path=/dsgvcy-77/ 2022-09-03T06:57:48.939 DEBUG:boto:Method: PUT 2022-09-03T06:57:48.940 DEBUG:boto:Path: /dsgvcy-77/ 2022-09-03T06:57:48.940 DEBUG:boto:Data: 2022-09-03T06:57:48.941 DEBUG:boto:Headers: {} 2022-09-03T06:57:48.941 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T06:57:48.942 DEBUG:boto:Port: 8000 2022-09-03T06:57:48.942 DEBUG:boto:Params: {} 2022-09-03T06:57:48.943 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T06:57:48.944 DEBUG:boto:Token: None 2022-09-03T06:57:48.945 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 06:57:48 GMT /dsgvcy-77/ 2022-09-03T06:57:48.946 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:ceZ508x6nkf6laKTR+6wzce2/S0= 2022-09-03T06:57:48.947 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 06:57:48 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:ceZ508x6nkf6laKTR+6wzce2/S0=', 'Content-Length': '0'} 2022-09-03T06:57:48.972 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000fa8f4582bdfff403-006312faec-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 06:57:48 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T06:57:48.972 DEBUG:rgw_multi.tests:created bucket=dsgvcy-77 2022-09-03T06:57:48.974 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:57:48.974 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:57:48.976 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:57:49.030 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:57:49.031 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:57:49.066 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T06:57:49.087 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T06:57:49.088 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:57:49.088 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:57:49.089 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:51:16.173767Z" 2022-09-03T06:57:49.089 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:57:49.089 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:57:49.089 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:57:49.090 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:50:53.931976Z" 2022-09-03T06:57:49.090 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:57:49.090 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:57:49.090 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662188268.968250_567.1", 2022-09-03T06:57:49.091 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:57:48.968250Z" 2022-09-03T06:57:49.091 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T06:57:49.091 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T06:57:49.092 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T06:57:49.092 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T06:57:49.092 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T06:57:49.093 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T06:57:49.095 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662188268.968250_567.1', 3: ''} 2022-09-03T06:57:49.095 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T06:57:49.096 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:57:49.096 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:57:49.097 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:57:49.155 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:57:49.156 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:57:49.192 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:57:49.190+0000 7ff9c1f6e600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:57:49.193 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:57:49.192+0000 7ff9c1f6e600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:57:49.207 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:57:49.208 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:57:49.208 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:57:49.208 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:57:49.209 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:57:49.209 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:57:49.210 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:57:49.210 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:57:49.210 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:57:49.210 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:57:49.211 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:57:49.211 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:57:49.211 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:57:49.212 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:57:49.212 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:57:49.212 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:57:49.212 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:57:49.213 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:57:49.213 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:57:49.213 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:57:49.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:57:49.214 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:57:49.214 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:57:49.214 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:57:49.215 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:57:49.215 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:57:49.215 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:57:49.216 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:57:49.216 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:57:49.216 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:57:49.216 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:57:49.217 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:57:49.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:57:49.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:57:49.218 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:57:49.218 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:57:49.218 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:57:49.219 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:57:49.219 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:57:49.219 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:57:49.219 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:57:49.220 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:57:49.220 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:57:49.220 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:57:49.220 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:57:49.222 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:57:49.222 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:57:49.222 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:57:49.222 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:57:49.223 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:57:49.223 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:57:49.223 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:57:49.224 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:57:49.224 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:57:49.224 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:57:49.224 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:57:49.225 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:57:49.225 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:57:49.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:57:49.226 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:57:49.226 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:57:49.226 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:57:49.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:57:49.227 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:57:49Z" 2022-09-03T06:57:49.227 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:57:49.228 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:57:49Z" } 2022-09-03T06:57:49.229 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:57:49.230 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662188268.968250_567.1', 3: ''} 2022-09-03T06:57:49.231 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:57:49.232 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662188268.968250_567.1 target=1_1662187959.530444_555.1 2022-09-03T06:57:50.924 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~202s 2022-09-03T06:57:54.234 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:57:54.235 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:57:54.235 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:57:54.291 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:57:54.292 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:57:54.331 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:57:54.330+0000 7f7563f80600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:57:54.332 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:57:54.331+0000 7f7563f80600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:57:54.348 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:57:54.348 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:57:54.349 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:57:54.349 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:57:54.349 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:57:54.350 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:57:54.350 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:57:54.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:57:54.351 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:57:54.351 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:57:54.351 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:57:54.351 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:57:54.352 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:57:54.352 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:57:54.352 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:57:54.352 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:57:54.353 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:57:54.353 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:57:54.353 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:57:54.353 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:57:54.354 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:57:54.354 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:57:54.354 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:57:54.354 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:57:54.355 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:57:54.355 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:57:54.355 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:57:54.355 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:57:54.356 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:57:54.356 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:57:54.356 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:57:54.356 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:57:54.357 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:57:54.357 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:57:54.357 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:57:54.357 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:57:54.358 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:57:54.358 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:57:54.358 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:57:54.358 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:57:54.359 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:57:54.359 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:57:54.360 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:57:54.360 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:57:54.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:57:54.361 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:57:54.361 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:57:54.362 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:57:54.362 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:57:54.363 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:57:54.363 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:57:54.363 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:57:54.364 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:57:54.365 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:57:54.365 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:57:54.365 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:57:54.366 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:57:54.366 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:57:54.366 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:57:54.367 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:57:54.367 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:57:54.368 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:57:54.368 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:57:54.369 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:57:54Z" 2022-09-03T06:57:54.369 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:57:54.371 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:57:54Z" } 2022-09-03T06:57:54.372 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:57:54.372 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662188268.968250_567.1', 3: ''} 2022-09-03T06:57:54.373 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:57:54.373 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662188268.968250_567.1 target=1_1662187959.530444_555.1 2022-09-03T06:57:56.530 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~207s 2022-09-03T06:57:59.375 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:57:59.376 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:57:59.376 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:57:59.434 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:57:59.435 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:57:59.466 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:57:59.464+0000 7f1ad94a3600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:57:59.468 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:57:59.466+0000 7f1ad94a3600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:57:59.481 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:57:59.483 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:57:59.483 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:57:59.484 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:57:59.484 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:57:59.486 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:57:59.486 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:57:59.487 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:57:59.487 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:57:59.488 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:57:59.489 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:57:59.489 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:57:59.490 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:57:59.490 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:57:59.490 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:57:59.491 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:57:59.491 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:57:59.491 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:57:59.491 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:57:59.492 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:57:59.492 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:57:59.493 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:57:59.493 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:57:59.494 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:57:59.494 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:57:59.494 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:57:59.495 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:57:59.495 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:57:59.496 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:57:59.496 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:57:59.497 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:57:59.497 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:57:59.498 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:57:59.498 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:57:59.498 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:57:59.499 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:57:59.499 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:57:59.500 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:57:59.500 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:57:59.500 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:57:59.501 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:57:59.501 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:57:59.502 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:57:59.502 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:57:59.502 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:57:59.503 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:57:59.503 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:57:59.503 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:57:59.503 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:57:59.504 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:57:59.505 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:57:59.506 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:57:59.507 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:57:59.507 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:57:59.508 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:57:59.508 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:57:59.509 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:57:59.509 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:57:59.510 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:57:59.510 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:57:59.511 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:57:59.511 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:57:59.512 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:57:59.512 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:57:59Z" 2022-09-03T06:57:59.513 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:57:59.514 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:57:59Z" } 2022-09-03T06:57:59.515 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:57:59.516 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662188268.968250_567.1', 3: ''} 2022-09-03T06:57:59.516 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:57:59.517 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662188268.968250_567.1 target=1_1662187959.530444_555.1 2022-09-03T06:58:02.140 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~213s 2022-09-03T06:58:04.518 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:58:04.519 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:58:04.520 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:58:04.579 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:58:04.579 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:58:04.622 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:58:04.620+0000 7f9d4f55d600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:58:04.623 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:58:04.621+0000 7f9d4f55d600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:58:04.637 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:58:04.638 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:58:04.639 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:58:04.639 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:58:04.639 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:58:04.640 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:58:04.640 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:04.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:04.641 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:58:04.641 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:04.641 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:58:04.641 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:04.642 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:04.642 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:58:04.642 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:04.642 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:58:04.643 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:04.643 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:58:04.643 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:04.643 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:04.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:04.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:04.644 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:58:04.644 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:04.645 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:04.645 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:58:04.645 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:04.645 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:04.646 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:04.646 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:58:04.646 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:04.647 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:04.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:04.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:04.647 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:58:04.648 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:04.649 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:04.649 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:58:04.650 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:04.650 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:04.651 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:04.651 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:58:04.652 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:04.652 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:04.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:04.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:04.654 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:58:04.654 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:04.655 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:04.655 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:58:04.656 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:04.656 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:04.656 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:04.656 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:58:04.657 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:58:04.657 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:04.657 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:04.658 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:58:04.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:04.658 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:58:04.658 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:58:04.659 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:58:04.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:04.660 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:58:04Z" 2022-09-03T06:58:04.660 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:58:04.662 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:58:04Z" } 2022-09-03T06:58:04.662 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:58:04.663 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662188268.968250_567.1', 3: ''} 2022-09-03T06:58:04.663 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:58:04.664 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662188268.968250_567.1 target=1_1662187959.530444_555.1 2022-09-03T06:58:07.749 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~219s 2022-09-03T06:58:09.665 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:58:09.667 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:58:09.668 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:58:09.727 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:58:09.728 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:58:09.760 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:58:09.758+0000 7f1241125600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:58:09.761 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:58:09.759+0000 7f1241125600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:58:09.778 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:58:09.778 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:58:09.779 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:58:09.779 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:58:09.779 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:58:09.779 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:58:09.780 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:09.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:09.780 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:58:09.781 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:09.781 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:58:09.781 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:09.784 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:09.784 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:58:09.784 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:09.785 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:58:09.785 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:09.785 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:58:09.785 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:09.786 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:09.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:09.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:09.787 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:58:09.787 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:09.787 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:09.788 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:58:09.788 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:09.788 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:09.788 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:09.789 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:58:09.789 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:09.789 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:09.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:09.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:09.792 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:58:09.792 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:09.792 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:09.792 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:58:09.793 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:09.793 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:09.793 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:09.793 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:58:09.794 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:09.794 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:09.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:09.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:09.795 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:58:09.795 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:09.795 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:09.796 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:58:09.796 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:09.796 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:09.796 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:09.797 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:58:09.797 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:58:09.797 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:09.797 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:09.798 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:58:09.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:09.798 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:58:09.798 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:58:09.799 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:58:09.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:09.799 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:58:09Z" 2022-09-03T06:58:09.799 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:58:09.800 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:58:09Z" } 2022-09-03T06:58:09.801 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:58:09.801 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662188268.968250_567.1', 3: ''} 2022-09-03T06:58:09.802 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:58:09.802 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662188268.968250_567.1 target=1_1662187959.530444_555.1 2022-09-03T06:58:13.356 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~224s 2022-09-03T06:58:14.804 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:58:14.805 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:58:14.805 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:58:14.862 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:58:14.863 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:58:14.908 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:58:14.906+0000 7f1d13662600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:58:14.909 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:58:14.907+0000 7f1d13662600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:58:14.922 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:58:14.923 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:58:14.923 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:58:14.923 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:58:14.923 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:58:14.924 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:58:14.924 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:14.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:14.924 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:58:14.925 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:14.925 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:58:14.926 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:14.926 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:14.927 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:58:14.927 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:14.928 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:58:14.928 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:14.929 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:58:14.929 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:14.930 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:14.930 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:14.931 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:14.931 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:58:14.932 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:14.932 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:14.933 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:58:14.933 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:14.934 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:14.934 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:14.935 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:58:14.935 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:14.936 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:14.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:14.937 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:14.937 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:58:14.938 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:14.939 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:14.939 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:58:14.939 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:14.940 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:14.940 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:14.941 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:58:14.941 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:14.942 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:14.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:14.942 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:14.943 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:58:14.943 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:14.943 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:14.943 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:58:14.944 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:14.944 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:14.944 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:14.944 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:58:14.945 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:58:14.945 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:14.945 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:14.945 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:58:14.946 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:14.946 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:58:14.946 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:58:14.946 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:58:14.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:14.947 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:58:14Z" 2022-09-03T06:58:14.947 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:58:14.949 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:58:14Z" } 2022-09-03T06:58:14.950 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:58:14.950 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662188268.968250_567.1', 3: ''} 2022-09-03T06:58:14.951 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:58:14.951 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662188268.968250_567.1 target=1_1662187959.530444_555.1 2022-09-03T06:58:18.961 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~230s 2022-09-03T06:58:19.952 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:58:19.954 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:58:19.956 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:58:20.014 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:58:20.015 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:58:20.044 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:58:20.043+0000 7f00839a0600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:58:20.045 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:58:20.044+0000 7f00839a0600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:58:20.062 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:58:20.064 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:58:20.064 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:58:20.064 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:58:20.064 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:58:20.065 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:58:20.065 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:20.065 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:20.066 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:58:20.066 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:20.066 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:58:20.066 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:20.067 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:20.067 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:58:20.067 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:20.067 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:58:20.068 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:20.068 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:58:20.068 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:20.068 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:20.069 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:20.069 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:20.069 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:58:20.069 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:20.070 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:20.070 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:58:20.070 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:20.070 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:20.071 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:20.071 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:58:20.071 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:20.072 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:20.072 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:20.072 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:20.073 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:58:20.073 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:20.074 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:20.074 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:58:20.075 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:20.075 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:20.076 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:20.076 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:58:20.077 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:20.077 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:20.077 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:20.078 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:20.078 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:58:20.079 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:20.079 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:20.080 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:58:20.080 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:20.081 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:20.081 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:20.082 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:58:20.082 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:58:20.083 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:20.083 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:20.084 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:58:20.084 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:20.085 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:58:20.085 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:58:20.086 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:58:20.086 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:20.087 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:58:20Z" 2022-09-03T06:58:20.087 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:58:20.088 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:58:20Z" } 2022-09-03T06:58:20.089 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:58:20.090 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662188268.968250_567.1', 3: ''} 2022-09-03T06:58:20.091 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:58:20.092 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662188268.968250_567.1 target=1_1662187959.530444_555.1 2022-09-03T06:58:24.569 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~236s 2022-09-03T06:58:25.093 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:58:25.095 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:58:25.095 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:58:25.154 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:58:25.154 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:58:25.187 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:58:25.185+0000 7f8d43d6f600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:58:25.188 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:58:25.186+0000 7f8d43d6f600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:58:25.202 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:58:25.202 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:58:25.203 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:58:25.203 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:58:25.203 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:58:25.204 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:58:25.204 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:25.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:25.204 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:58:25.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:25.205 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:58:25.205 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:25.206 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:25.206 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:58:25.206 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:25.207 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:58:25.207 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:25.207 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:58:25.207 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:25.208 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:25.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:25.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:25.209 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:58:25.209 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:25.209 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:25.209 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:58:25.210 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:25.210 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:25.210 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:25.211 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:58:25.211 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:25.211 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:25.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:25.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:25.212 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:58:25.212 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:25.213 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:25.213 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:58:25.213 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:25.213 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:25.214 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:25.214 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:58:25.214 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:25.215 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:25.215 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:25.215 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:25.215 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:58:25.216 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:25.216 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:25.216 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:58:25.217 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:25.217 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:25.217 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:25.217 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:58:25.218 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:58:25.218 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:25.218 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:25.219 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:58:25.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:25.219 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:58:25.219 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:58:25.220 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:58:25.220 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:25.220 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:58:25Z" 2022-09-03T06:58:25.220 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:58:25.222 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:58:25Z" } 2022-09-03T06:58:25.223 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:58:25.223 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662188268.968250_567.1', 3: ''} 2022-09-03T06:58:25.224 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:58:25.225 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662188268.968250_567.1 target=1_1662187959.530444_555.1 2022-09-03T06:58:30.175 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~241s 2022-09-03T06:58:30.226 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:58:30.228 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:58:30.229 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:58:30.287 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:58:30.287 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:58:30.323 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:58:30.319+0000 7f1e9f8dd600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:58:30.324 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:58:30.320+0000 7f1e9f8dd600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:58:30.340 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:58:30.341 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:58:30.342 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:58:30.342 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:58:30.343 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:58:30.343 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:58:30.344 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:30.344 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:30.345 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:58:30.347 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:30.347 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:58:30.348 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:30.348 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:30.348 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:58:30.349 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:30.349 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:58:30.349 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:30.350 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:58:30.350 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:30.350 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:30.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:30.351 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:30.351 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:58:30.351 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:30.351 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:30.352 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:58:30.352 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:30.352 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:30.352 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:30.353 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:58:30.353 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:30.353 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:30.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:30.354 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:30.354 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:58:30.355 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:30.355 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:30.358 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:58:30.359 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:30.359 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:30.359 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:30.359 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:58:30.360 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:30.360 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:30.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:30.361 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:30.361 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:58:30.362 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:30.362 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:30.363 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:58:30.363 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:30.364 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:30.364 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:30.366 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:58:30.366 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:58:30.367 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:30.370 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:30.371 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:58:30.371 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:30.372 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:58:30.372 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:58:30.373 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:58:30.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:30.374 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:58:30Z" 2022-09-03T06:58:30.374 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:58:30.377 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:58:30Z" } 2022-09-03T06:58:30.378 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:58:30.379 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662188268.968250_567.1', 3: ''} 2022-09-03T06:58:30.380 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:58:30.381 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662188268.968250_567.1 target=1_1662187959.530444_555.1 2022-09-03T06:58:35.383 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:58:35.385 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:58:35.387 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:58:35.448 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:58:35.449 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:58:35.478 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:58:35.476+0000 7f29da6f0600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:58:35.479 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:58:35.477+0000 7f29da6f0600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:58:35.493 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:58:35.494 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:58:35.494 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:58:35.494 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:58:35.495 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:58:35.495 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:58:35.498 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:35.499 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:35.499 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:58:35.500 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:35.500 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:58:35.500 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:35.501 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:35.501 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:58:35.502 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:35.502 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:58:35.502 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:35.502 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:58:35.503 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:35.503 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:35.503 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:35.504 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:35.504 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:58:35.504 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:35.505 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:35.505 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:58:35.505 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:35.506 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:35.506 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:35.506 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:58:35.506 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:35.507 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:35.507 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:35.507 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:35.507 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:58:35.508 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:35.508 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:35.508 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:58:35.508 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:35.509 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:35.509 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:35.509 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:58:35.509 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:35.510 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:35.510 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:35.510 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:35.511 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:58:35.511 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:35.511 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:35.511 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:58:35.512 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:35.512 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:35.512 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:35.512 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:58:35.513 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:58:35.513 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:35.513 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:35.513 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:58:35.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:35.514 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:58:35.514 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:58:35.514 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:58:35.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:35.515 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:58:35Z" 2022-09-03T06:58:35.515 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:58:35.517 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:58:35Z" } 2022-09-03T06:58:35.517 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:58:35.518 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662188268.968250_567.1', 3: ''} 2022-09-03T06:58:35.518 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:58:35.519 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662188268.968250_567.1 target=1_1662187959.530444_555.1 2022-09-03T06:58:35.783 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~247s 2022-09-03T06:58:40.520 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:58:40.522 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:58:40.523 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:58:40.581 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:58:40.582 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:58:40.611 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:58:40.609+0000 7f8075775600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:58:40.612 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:58:40.611+0000 7f8075775600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:58:40.626 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:58:40.627 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:58:40.627 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:58:40.628 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:58:40.628 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:58:40.629 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:58:40.630 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:40.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:40.631 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:58:40.631 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:40.632 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:58:40.632 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:40.632 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:40.632 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:58:40.633 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:40.633 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:58:40.633 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:40.634 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:58:40.634 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:40.634 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:40.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:40.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:40.635 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:58:40.635 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:40.636 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:40.637 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:58:40.637 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:40.638 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:40.639 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:40.639 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:58:40.640 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:40.640 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:40.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:40.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:40.641 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:58:40.641 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:40.641 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:40.642 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:58:40.642 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:40.642 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:40.643 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:40.643 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:58:40.643 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:40.643 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:40.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:40.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:40.644 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:58:40.645 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:40.645 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:40.645 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:58:40.645 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:40.646 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:40.646 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:40.646 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:58:40.647 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:58:40.647 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:40.647 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:40.647 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:58:40.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:40.648 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:58:40.648 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:58:40.648 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:58:40.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:40.649 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:58:40Z" 2022-09-03T06:58:40.649 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:58:40.651 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:58:40Z" } 2022-09-03T06:58:40.651 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:58:40.652 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662188268.968250_567.1', 3: ''} 2022-09-03T06:58:40.652 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:58:40.653 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662188268.968250_567.1 target=1_1662187959.530444_555.1 2022-09-03T06:58:41.390 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~252s 2022-09-03T06:58:45.654 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:58:45.655 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:58:45.656 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:58:45.715 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:58:45.715 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:58:45.747 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:58:45.745+0000 7f2b16caa600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:58:45.748 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:58:45.746+0000 7f2b16caa600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:58:45.762 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:58:45.762 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:58:45.763 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:58:45.763 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:58:45.763 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:58:45.763 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:58:45.764 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:45.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:45.764 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:58:45.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:45.765 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:58:45.765 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:45.765 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:45.765 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:58:45.766 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:45.766 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:58:45.766 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:45.767 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:58:45.767 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:45.767 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:45.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:45.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:45.768 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:58:45.768 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:45.768 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:45.769 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:58:45.769 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:45.769 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:45.769 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:45.770 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:58:45.770 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:45.770 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:45.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:45.771 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:45.771 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:58:45.771 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:45.771 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:45.772 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:58:45.772 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:45.772 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:45.772 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:45.773 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:58:45.773 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:45.773 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:45.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:45.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:45.774 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:58:45.774 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:45.774 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:45.775 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:58:45.775 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:45.775 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:45.775 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:45.776 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:58:45.776 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:58:45.776 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:45.776 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:45.777 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:58:45.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:45.777 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:58:45.777 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:58:45.778 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:58:45.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:45.778 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:58:45Z" 2022-09-03T06:58:45.778 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:58:45.780 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:58:45Z" } 2022-09-03T06:58:45.780 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:58:45.781 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662188268.968250_567.1', 3: ''} 2022-09-03T06:58:45.781 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:58:45.782 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662188268.968250_567.1 target=1_1662187959.530444_555.1 2022-09-03T06:58:46.996 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~258s 2022-09-03T06:58:50.787 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:58:50.789 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:58:50.790 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:58:50.852 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:58:50.854 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:58:50.880 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:58:50.877+0000 7fb8cefc2600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:58:50.881 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:58:50.879+0000 7fb8cefc2600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:58:50.894 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:58:50.895 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:58:50.895 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:58:50.895 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:58:50.896 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:58:50.896 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:58:50.897 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:50.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:50.898 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:58:50.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:50.899 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:58:50.899 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:50.899 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:50.900 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:58:50.900 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:50.900 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:58:50.901 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:50.901 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:58:50.901 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:50.901 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:50.902 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:50.902 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:50.902 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:58:50.903 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:50.904 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:50.904 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:58:50.904 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:50.905 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:50.905 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:50.906 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:58:50.907 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:50.907 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:50.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:50.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:50.909 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:58:50.909 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:50.911 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:50.912 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:58:50.912 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:50.913 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:50.913 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:50.914 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:58:50.914 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:50.915 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:50.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:50.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:50.916 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:58:50.917 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:50.917 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:50.918 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:58:50.919 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:50.919 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:50.920 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:50.920 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:58:50.921 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:58:50.921 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:50.922 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:50.922 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:58:50.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:50.923 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:58:50.923 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:58:50.924 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:58:50.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:50.925 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:58:50Z" 2022-09-03T06:58:50.925 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:58:50.927 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:58:50Z" } 2022-09-03T06:58:50.928 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:58:50.928 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662188268.968250_567.1', 3: ''} 2022-09-03T06:58:50.929 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:58:50.929 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662188268.968250_567.1 target=1_1662187959.530444_555.1 2022-09-03T06:58:52.604 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~264s 2022-09-03T06:58:55.931 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:58:55.932 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:58:55.934 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:58:55.993 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:58:55.994 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:58:56.020 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:58:56.019+0000 7f2373f87600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:58:56.022 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:58:56.020+0000 7f2373f87600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:58:56.040 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:58:56.041 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:58:56.042 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:58:56.043 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:58:56.043 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:58:56.044 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:58:56.044 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:56.045 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:56.045 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:58:56.046 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:56.046 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:58:56.047 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:56.047 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:56.047 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:58:56.048 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:56.048 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:58:56.048 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:56.049 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:58:56.049 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:56.051 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:56.051 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:56.052 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:56.052 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:58:56.053 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:56.053 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:56.059 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:58:56.059 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:56.059 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:56.059 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:56.060 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:58:56.060 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:56.061 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:56.061 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:56.062 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:56.062 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:58:56.063 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:56.063 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:56.064 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:58:56.064 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:56.065 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:56.066 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:56.067 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:58:56.067 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:58:56.067 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:56.068 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:56.068 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:58:56.069 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:58:56.069 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:58:56.070 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:58:56.070 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:58:56.071 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:58:56.071 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:58:56.072 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:58:56.072 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:58:56.073 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:58:56.073 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:56.074 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:58:56.074 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:58:56.075 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:56.075 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:58:56.075 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:58:56.076 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:58:56.076 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:58:56.077 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:58:56Z" 2022-09-03T06:58:56.077 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:58:56.079 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:58:56Z" } 2022-09-03T06:58:56.080 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:58:56.081 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662188268.968250_567.1', 3: ''} 2022-09-03T06:58:56.082 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:58:56.083 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662188268.968250_567.1 target=1_1662187959.530444_555.1 2022-09-03T06:58:58.213 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~269s 2022-09-03T06:59:01.084 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:59:01.086 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:59:01.088 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:59:01.147 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:59:01.147 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:59:01.187 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:59:01.184+0000 7f6fec6aa600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:59:01.188 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:59:01.185+0000 7f6fec6aa600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:59:01.200 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:59:01.201 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:59:01.202 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:59:01.202 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:59:01.202 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:59:01.203 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:59:01.203 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:01.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:01.204 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:59:01.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:01.205 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:59:01.205 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:01.206 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:01.207 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:59:01.207 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:01.208 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:59:01.208 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:01.209 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:59:01.209 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:01.210 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:01.211 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:01.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:01.212 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:59:01.212 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:01.214 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:01.214 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:59:01.215 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:01.215 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:59:01.215 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:01.216 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:59:01.216 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:01.217 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:01.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:01.218 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:01.218 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:59:01.218 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:01.219 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:01.219 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:59:01.220 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:01.220 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:59:01.221 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:01.221 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:59:01.222 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:01.222 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:01.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:01.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:01.224 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:59:01.224 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:01.225 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:01.225 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:59:01.226 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:01.226 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:59:01.227 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:01.227 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:59:01.227 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:59:01.228 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:01.228 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:01.228 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:59:01.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:01.229 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:59:01.229 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:59:01.230 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:59:01.230 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:01.231 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:59:01Z" 2022-09-03T06:59:01.231 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:59:01.233 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:59:01Z" } 2022-09-03T06:59:01.234 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:59:01.235 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662188268.968250_567.1', 3: ''} 2022-09-03T06:59:01.236 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:59:01.237 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662188268.968250_567.1 target=1_1662187959.530444_555.1 2022-09-03T06:59:03.821 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~275s 2022-09-03T06:59:06.238 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:59:06.239 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:59:06.240 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:59:06.297 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:59:06.298 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:59:06.328 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:59:06.326+0000 7fe2cfd09600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:59:06.329 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:59:06.327+0000 7fe2cfd09600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:59:06.347 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:59:06.347 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:59:06.348 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:59:06.348 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:59:06.348 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:59:06.349 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:59:06.349 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:06.349 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:06.349 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:59:06.350 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:06.350 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:59:06.350 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:06.350 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:06.351 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:59:06.351 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:06.351 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:59:06.351 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:06.352 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:59:06.352 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:06.352 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:06.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:06.354 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:06.355 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:59:06.356 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:06.356 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:06.356 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:59:06.357 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:06.357 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:59:06.357 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:06.358 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:59:06.358 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:06.358 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:06.359 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:06.359 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:06.359 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:59:06.360 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:06.360 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:06.360 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:59:06.361 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:06.361 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:59:06.361 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:06.362 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:59:06.362 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:06.362 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:06.363 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:06.363 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:06.363 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:59:06.364 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:06.364 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:06.364 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:59:06.365 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:06.365 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:59:06.366 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:06.366 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:59:06.366 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:59:06.367 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:06.367 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:06.367 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:59:06.368 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:06.369 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:59:06.369 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:59:06.369 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:59:06.369 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:06.370 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:59:06Z" 2022-09-03T06:59:06.370 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:59:06.372 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:59:06Z" } 2022-09-03T06:59:06.373 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:59:06.374 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662188268.968250_567.1', 3: ''} 2022-09-03T06:59:06.374 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:59:06.375 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662188268.968250_567.1 target=1_1662187959.530444_555.1 2022-09-03T06:59:09.427 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~280s 2022-09-03T06:59:11.378 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:59:11.379 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:59:11.380 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:59:11.438 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:59:11.439 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:59:11.480 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:59:11.478+0000 7fc939781600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:59:11.481 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:59:11.479+0000 7fc939781600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:59:11.499 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:59:11.499 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:59:11.500 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:59:11.500 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:59:11.500 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:59:11.501 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:59:11.501 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:11.501 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:11.501 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:59:11.502 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:11.502 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:59:11.502 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:11.503 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:11.503 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:59:11.503 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:11.503 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:59:11.504 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:11.504 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:59:11.504 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:11.504 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:11.505 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:11.505 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:11.505 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:59:11.506 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:11.506 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:11.506 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:59:11.506 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:11.507 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:59:11.507 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:11.507 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:59:11.507 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:11.508 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:11.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:11.508 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:11.508 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:59:11.509 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:11.509 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:11.510 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:59:11.510 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:11.510 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:59:11.510 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:11.511 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:59:11.511 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:11.511 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:11.511 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:11.512 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:11.512 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:59:11.512 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:11.513 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:11.513 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:59:11.513 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:11.513 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:59:11.514 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:11.514 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:59:11.514 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:59:11.514 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:11.515 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:11.515 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:59:11.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:11.515 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:59:11.516 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:59:11.516 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:59:11.516 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:11.516 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:59:11Z" 2022-09-03T06:59:11.517 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:59:11.518 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:59:11Z" } 2022-09-03T06:59:11.519 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:59:11.519 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662188268.968250_567.1', 3: ''} 2022-09-03T06:59:11.520 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:59:11.521 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662188268.968250_567.1 target=1_1662187959.530444_555.1 2022-09-03T06:59:15.033 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~286s 2022-09-03T06:59:16.522 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:59:16.523 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:59:16.524 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:59:16.585 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:59:16.585 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:59:16.617 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:59:16.613+0000 7f06fcbf7600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:59:16.617 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:59:16.614+0000 7f06fcbf7600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:59:16.631 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:59:16.632 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:59:16.633 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:59:16.633 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:59:16.634 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:59:16.634 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:59:16.635 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:16.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:16.636 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:59:16.636 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:16.637 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:59:16.637 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:16.637 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:16.637 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:59:16.638 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:16.638 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:59:16.638 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:16.639 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:59:16.639 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:16.639 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:16.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:16.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:16.641 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:59:16.641 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:16.641 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:16.642 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:59:16.643 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:16.644 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:59:16.644 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:16.645 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:59:16.645 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:16.646 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:16.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:16.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:16.648 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:59:16.648 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:16.649 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:16.649 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:59:16.649 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:16.650 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:59:16.650 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:16.651 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:59:16.651 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:16.652 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:16.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:16.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:16.653 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:59:16.654 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:16.654 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:16.654 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:59:16.655 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:16.655 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:59:16.655 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:16.656 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:59:16.656 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:59:16.656 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:16.656 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:16.657 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:59:16.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:16.657 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:59:16.658 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:59:16.658 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:59:16.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:16.658 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:59:16Z" 2022-09-03T06:59:16.659 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:59:16.660 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:59:16Z" } 2022-09-03T06:59:16.661 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:59:16.661 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662188268.968250_567.1', 3: ''} 2022-09-03T06:59:16.662 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:59:16.662 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662188268.968250_567.1 target=1_1662187959.530444_555.1 2022-09-03T06:59:20.641 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~292s 2022-09-03T06:59:21.664 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:59:21.665 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:59:21.666 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:59:21.725 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:59:21.726 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:59:21.766 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:59:21.763+0000 7fc87220a600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:59:21.767 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:59:21.765+0000 7fc87220a600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:59:21.783 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:59:21.784 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:59:21.784 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:59:21.785 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:59:21.785 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:59:21.785 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:59:21.785 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:21.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:21.786 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:59:21.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:21.786 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:59:21.787 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:21.787 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:21.787 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:59:21.788 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:21.788 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:59:21.788 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:21.789 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:59:21.789 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:21.789 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:21.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:21.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:21.790 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:59:21.790 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:21.791 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:21.791 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:59:21.791 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:21.791 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:59:21.792 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:21.792 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:59:21.792 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:21.793 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:21.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:21.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:21.795 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:59:21.795 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:21.795 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:21.795 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:59:21.796 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:21.796 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:59:21.796 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:21.796 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:59:21.797 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:21.797 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:21.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:21.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:21.798 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:59:21.798 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:21.799 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:21.799 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:59:21.799 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:21.799 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:59:21.800 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:21.800 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:59:21.800 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:59:21.800 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:21.801 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:21.801 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:59:21.801 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:21.801 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:59:21.802 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:59:21.802 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:59:21.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:21.803 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:59:21Z" 2022-09-03T06:59:21.803 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:59:21.805 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:59:21Z" } 2022-09-03T06:59:21.806 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:59:21.807 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662188268.968250_567.1', 3: ''} 2022-09-03T06:59:21.808 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:59:21.810 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662188268.968250_567.1 target=1_1662187959.530444_555.1 2022-09-03T06:59:26.247 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~297s 2022-09-03T06:59:26.812 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:59:26.813 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:59:26.815 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:59:26.871 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:59:26.872 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:59:26.904 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:59:26.901+0000 7f34cc131600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:59:26.905 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:59:26.902+0000 7f34cc131600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:59:26.920 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:59:26.921 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:59:26.921 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:59:26.921 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:59:26.922 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:59:26.922 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:59:26.923 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:26.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:26.923 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:59:26.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:26.924 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:59:26.924 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:26.925 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:26.925 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:59:26.925 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:26.926 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:59:26.926 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:26.926 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:59:26.927 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:26.927 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:26.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:26.928 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:26.928 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:59:26.929 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:26.929 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:26.930 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:59:26.930 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:26.931 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:59:26.932 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:26.932 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:59:26.933 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:26.933 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:26.934 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:26.934 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:26.935 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:59:26.935 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:26.936 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:26.936 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:59:26.937 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:26.937 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:59:26.938 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:26.938 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:59:26.939 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:26.940 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:26.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:26.941 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:26.942 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:59:26.942 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:26.943 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:26.943 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:59:26.944 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:26.944 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:59:26.945 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:26.946 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:59:26.946 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:59:26.947 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:26.947 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:26.948 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:59:26.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:26.949 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:59:26.949 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:59:26.950 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:59:26.951 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:26.951 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:59:26Z" 2022-09-03T06:59:26.952 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:59:26.953 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:59:26Z" } 2022-09-03T06:59:26.954 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:59:26.954 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662188268.968250_567.1', 3: ''} 2022-09-03T06:59:26.955 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:59:26.955 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662188268.968250_567.1 target=1_1662187959.530444_555.1 2022-09-03T06:59:31.878 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon c2.rgw.client.0 is failed for ~303s 2022-09-03T06:59:31.879 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2022-09-03T06:59:31.967 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:59:31.968 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:59:31.969 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:59:32.032 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:59:32.033 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T06:59:32.067 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:59:32.065+0000 7f6989953600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:59:32.069 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T06:59:32.066+0000 7f6989953600 20 RGW-SYNC:meta: read sync status 2022-09-03T06:59:32.087 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2022-09-03T06:59:32.089 INFO:teuthology.orchestra.run.smithi189.stdout: "sync_status": { 2022-09-03T06:59:32.092 INFO:teuthology.orchestra.run.smithi189.stdout: "info": { 2022-09-03T06:59:32.094 INFO:teuthology.orchestra.run.smithi189.stdout: "status": "sync", 2022-09-03T06:59:32.094 INFO:teuthology.orchestra.run.smithi189.stdout: "num_shards": 4, 2022-09-03T06:59:32.095 INFO:teuthology.orchestra.run.smithi189.stdout: "period": "229c309b-f405-4ead-a3a8-baa670c6da51", 2022-09-03T06:59:32.095 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:32.096 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:32.097 INFO:teuthology.orchestra.run.smithi189.stdout: "markers": [ 2022-09-03T06:59:32.097 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:32.098 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 0, 2022-09-03T06:59:32.098 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:32.098 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:32.099 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T06:59:32.101 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:32.102 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 1, 2022-09-03T06:59:32.103 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:32.103 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:51:16.173767Z", 2022-09-03T06:59:32.103 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:32.104 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:32.104 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:32.104 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:32.104 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 1, 2022-09-03T06:59:32.105 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:32.105 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:32.105 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T06:59:32.106 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:32.106 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:59:32.106 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:32.106 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:50:53.931976Z", 2022-09-03T06:59:32.107 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:32.107 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:32.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:32.108 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:32.108 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 2, 2022-09-03T06:59:32.109 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:32.109 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:32.109 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662187959.530444_555.1", 2022-09-03T06:59:32.109 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:32.110 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:59:32.110 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:32.110 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T06:52:39.530444Z", 2022-09-03T06:59:32.111 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 4 2022-09-03T06:59:32.111 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:32.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:32.113 INFO:teuthology.orchestra.run.smithi189.stdout: { 2022-09-03T06:59:32.113 INFO:teuthology.orchestra.run.smithi189.stdout: "key": 3, 2022-09-03T06:59:32.114 INFO:teuthology.orchestra.run.smithi189.stdout: "val": { 2022-09-03T06:59:32.114 INFO:teuthology.orchestra.run.smithi189.stdout: "state": 1, 2022-09-03T06:59:32.115 INFO:teuthology.orchestra.run.smithi189.stdout: "marker": "1_1662182430.928498_66.1", 2022-09-03T06:59:32.115 INFO:teuthology.orchestra.run.smithi189.stdout: "next_step_marker": "", 2022-09-03T06:59:32.116 INFO:teuthology.orchestra.run.smithi189.stdout: "total_entries": 0, 2022-09-03T06:59:32.116 INFO:teuthology.orchestra.run.smithi189.stdout: "pos": 0, 2022-09-03T06:59:32.117 INFO:teuthology.orchestra.run.smithi189.stdout: "timestamp": "2022-09-03T05:20:30.928498Z", 2022-09-03T06:59:32.117 INFO:teuthology.orchestra.run.smithi189.stdout: "realm_epoch": 2 2022-09-03T06:59:32.118 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:32.119 INFO:teuthology.orchestra.run.smithi189.stdout: } 2022-09-03T06:59:32.120 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2022-09-03T06:59:32.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:32.120 INFO:teuthology.orchestra.run.smithi189.stdout: "full_sync": { 2022-09-03T06:59:32.121 INFO:teuthology.orchestra.run.smithi189.stdout: "total": 1, 2022-09-03T06:59:32.121 INFO:teuthology.orchestra.run.smithi189.stdout: "complete": 1 2022-09-03T06:59:32.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2022-09-03T06:59:32.122 INFO:teuthology.orchestra.run.smithi189.stdout: "current_time": "2022-09-03T06:59:32Z" 2022-09-03T06:59:32.123 INFO:teuthology.orchestra.run.smithi189.stdout:} 2022-09-03T06:59:32.124 DEBUG:rgw_multi.tests:current meta sync status={ "sync_status": { "info": { "status": "sync", "num_shards": 4, "period": "229c309b-f405-4ead-a3a8-baa670c6da51", "realm_epoch": 4 }, "markers": [ { "key": 0, "val": { "state": 1, "marker": "1_1662187876.173767_261.1", "next_step_marker": "", "total_entries": 1, "pos": 0, "timestamp": "2022-09-03T06:51:16.173767Z", "realm_epoch": 4 } }, { "key": 1, "val": { "state": 1, "marker": "1_1662187853.931976_390.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:50:53.931976Z", "realm_epoch": 4 } }, { "key": 2, "val": { "state": 1, "marker": "1_1662187959.530444_555.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T06:52:39.530444Z", "realm_epoch": 4 } }, { "key": 3, "val": { "state": 1, "marker": "1_1662182430.928498_66.1", "next_step_marker": "", "total_entries": 0, "pos": 0, "timestamp": "2022-09-03T05:20:30.928498Z", "realm_epoch": 2 } } ] }, "full_sync": { "total": 1, "complete": 1 }, "current_time": "2022-09-03T06:59:32Z" } 2022-09-03T06:59:32.125 DEBUG:rgw_multi.tests:sync_markers=[{'key': 0, 'val': {'state': 1, 'marker': '1_1662187876.173767_261.1', 'next_step_marker': '', 'total_entries': 1, 'pos': 0, 'timestamp': '2022-09-03T06:51:16.173767Z', 'realm_epoch': 4}}, {'key': 1, 'val': {'state': 1, 'marker': '1_1662187853.931976_390.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:50:53.931976Z', 'realm_epoch': 4}}, {'key': 2, 'val': {'state': 1, 'marker': '1_1662187959.530444_555.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T06:52:39.530444Z', 'realm_epoch': 4}}, {'key': 3, 'val': {'state': 1, 'marker': '1_1662182430.928498_66.1', 'next_step_marker': '', 'total_entries': 0, 'pos': 0, 'timestamp': '2022-09-03T05:20:30.928498Z', 'realm_epoch': 2}}] 2022-09-03T06:59:32.126 DEBUG:rgw_multi.tests:log_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662188268.968250_567.1', 3: ''} 2022-09-03T06:59:32.127 DEBUG:rgw_multi.tests:sync_status={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662187959.530444_555.1', 3: ''} 2022-09-03T06:59:32.127 WARNING:rgw_multi.tests:zone test-zone2 behind master: shard=2 master=1_1662188268.968250_567.1 target=1_1662187959.530444_555.1 2022-09-03T06:59:32.491 INFO:tasks.ceph.c2.osd.0:Sent signal 15 2022-09-03T06:59:32.498 INFO:tasks.ceph.c2.osd.1:Sent signal 15 2022-09-03T06:59:32.499 INFO:tasks.ceph.c2.osd.2:Sent signal 15 2022-09-03T06:59:32.500 INFO:tasks.ceph.c2.mon.a:Sent signal 15 2022-09-03T06:59:32.501 INFO:tasks.rgw.c2.client.1:Sent signal 15 2022-09-03T06:59:32.501 INFO:tasks.ceph.c2.mgr.x:Sent signal 15 2022-09-03T06:59:32.503 INFO:tasks.ceph.c2.osd.0.smithi189.stderr:2022-09-03T06:59:32.491+0000 7f6512d78700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster c2 -i 0 (PID: 104422) UID: 0 2022-09-03T06:59:32.503 INFO:tasks.ceph.c2.osd.0.smithi189.stderr:2022-09-03T06:59:32.491+0000 7f6512d78700 -1 osd.0 63 *** Got signal Terminated *** 2022-09-03T06:59:32.503 INFO:tasks.ceph.c2.osd.0.smithi189.stderr:2022-09-03T06:59:32.491+0000 7f6512d78700 -1 osd.0 63 *** Immediate shutdown (osd_fast_shutdown=true) *** 2022-09-03T06:59:32.504 INFO:tasks.ceph.c2.osd.1.smithi189.stderr:2022-09-03T06:59:32.498+0000 7f7c14f6b700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster c2 -i 1 (PID: 104431) UID: 0 2022-09-03T06:59:32.505 INFO:tasks.ceph.c2.osd.1.smithi189.stderr:2022-09-03T06:59:32.498+0000 7f7c14f6b700 -1 osd.1 63 *** Got signal Terminated *** 2022-09-03T06:59:32.505 INFO:tasks.ceph.c2.osd.1.smithi189.stderr:2022-09-03T06:59:32.498+0000 7f7c14f6b700 -1 osd.1 63 *** Immediate shutdown (osd_fast_shutdown=true) *** 2022-09-03T06:59:32.506 INFO:tasks.ceph.c2.osd.2.smithi189.stderr:2022-09-03T06:59:32.498+0000 7f7ca11b8700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster c2 -i 2 (PID: 104434) UID: 0 2022-09-03T06:59:32.507 INFO:tasks.ceph.c2.osd.2.smithi189.stderr:2022-09-03T06:59:32.498+0000 7f7ca11b8700 -1 osd.2 63 *** Got signal Terminated *** 2022-09-03T06:59:32.507 INFO:tasks.ceph.c2.osd.2.smithi189.stderr:2022-09-03T06:59:32.498+0000 7f7ca11b8700 -1 osd.2 63 *** Immediate shutdown (osd_fast_shutdown=true) *** 2022-09-03T06:59:32.508 INFO:tasks.ceph.c2.mon.a.smithi189.stderr:2022-09-03T06:59:32.499+0000 7f77e1951700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster c2 -i a (PID: 104025) UID: 0 2022-09-03T06:59:32.508 INFO:tasks.ceph.c2.mon.a.smithi189.stderr:2022-09-03T06:59:32.499+0000 7f77e1951700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2022-09-03T06:59:32.509 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:59:32.500+0000 7fc8c3a87700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term radosgw --rgw-frontends beast port=8001 -n client.1 --cluster c2 -k /etc/ceph/c2.client.1.keyring --log-file /var/log/ceph/rgw.c2.client.1.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.c2.client.1.sock --foreground --rgw-zone test-zone4 --rgw-zonegroup test-zonegroup --rgw-realm test-realm (PID: 113467) UID: 0 2022-09-03T06:59:32.509 INFO:tasks.rgw.c2.client.1.smithi189.stdout:2022-09-03T06:59:32.500+0000 7fc8c7cf2600 -1 shutting down 2022-09-03T06:59:37.129 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:59:37.130 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T06:59:37.131 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T06:59:37.188 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T06:59:37.189 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T07:04:37.192 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T07:04:37.187+0000 7f3c21a76600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T07:09:37.202 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T07:09:37.188+0000 7f3c21a76600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T07:14:37.197 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T07:14:37.189+0000 7f3c21a76600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T07:19:37.194 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T07:19:37.190+0000 7f3c21a76600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T07:24:37.194 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T07:24:37.191+0000 7f3c21a76600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T07:29:37.196 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T07:29:37.191+0000 7f3c21a76600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T07:34:37.199 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T07:34:37.190+0000 7f3c21a76600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T07:39:37.196 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T07:39:37.191+0000 7f3c21a76600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T07:44:37.196 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T07:44:37.192+0000 7f3c21a76600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T07:49:37.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-09-03T07:49:37.202 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T07:49:37.192+0000 7f3c21a76600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T07:49:37.203 INFO:teuthology.orchestra.run.smithi189.stderr:failed to fetch mon config (--no-mon-config to skip) 2022-09-03T07:49:37.208 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_resharded_bucket_full_sync_after_data_sync_init ... FAIL 2022-09-03T07:49:37.211 DEBUG:boto:path=/dsgvcy-78/ 2022-09-03T07:49:37.213 DEBUG:boto:auth_path=/dsgvcy-78/ 2022-09-03T07:49:37.214 DEBUG:boto:Method: PUT 2022-09-03T07:49:37.214 DEBUG:boto:Path: /dsgvcy-78/ 2022-09-03T07:49:37.215 DEBUG:boto:Data: 2022-09-03T07:49:37.215 DEBUG:boto:Headers: {} 2022-09-03T07:49:37.216 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T07:49:37.216 DEBUG:boto:Port: 8000 2022-09-03T07:49:37.217 DEBUG:boto:Params: {} 2022-09-03T07:49:37.218 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T07:49:37.219 DEBUG:boto:Token: None 2022-09-03T07:49:37.220 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 07:49:37 GMT /dsgvcy-78/ 2022-09-03T07:49:37.221 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:KxMJGZ20RJoKTqccBI5b1dLa8xE= 2022-09-03T07:49:37.223 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 07:49:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:KxMJGZ20RJoKTqccBI5b1dLa8xE=', 'Content-Length': '0'} 2022-09-03T07:49:37.254 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000e621fb84698b3ffe-0063130711-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 07:49:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T07:49:37.255 DEBUG:rgw_multi.tests:created bucket=dsgvcy-78 2022-09-03T07:49:37.255 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T07:49:37.256 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T07:49:37.258 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T07:49:37.314 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T07:49:37.315 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T07:49:37.346 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T07:49:37.359 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T07:49:37.360 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T07:49:37.360 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T07:49:37.360 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:51:16.173767Z" 2022-09-03T07:49:37.360 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T07:49:37.361 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T07:49:37.361 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T07:49:37.362 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:50:53.931976Z" 2022-09-03T07:49:37.362 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T07:49:37.362 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T07:49:37.363 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662191377.250121_601.1", 2022-09-03T07:49:37.363 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T07:49:37.250121Z" 2022-09-03T07:49:37.364 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T07:49:37.364 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T07:49:37.364 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T07:49:37.365 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T07:49:37.365 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T07:49:37.365 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T07:49:37.367 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662191377.250121_601.1', 3: ''} 2022-09-03T07:49:37.368 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T07:49:37.369 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T07:49:37.369 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T07:49:37.370 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T07:49:37.429 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T07:49:37.431 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T07:54:37.433 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T07:54:37.429+0000 7fa0983f8600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T07:59:37.438 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T07:59:37.429+0000 7fa0983f8600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T08:04:37.441 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T08:04:37.430+0000 7fa0983f8600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T08:09:37.434 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T08:09:37.430+0000 7fa0983f8600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T08:14:37.451 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T08:14:37.431+0000 7fa0983f8600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T08:19:37.437 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T08:19:37.432+0000 7fa0983f8600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T08:24:37.435 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T08:24:37.432+0000 7fa0983f8600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T08:29:37.436 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T08:29:37.433+0000 7fa0983f8600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T08:34:37.437 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T08:34:37.432+0000 7fa0983f8600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T08:39:37.439 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T08:39:37.433+0000 7fa0983f8600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T08:39:37.442 INFO:teuthology.orchestra.run.smithi189.stderr:failed to fetch mon config (--no-mon-config to skip) 2022-09-03T08:39:37.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-09-03T08:39:37.455 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_bucket_incremental_sync_after_data_sync_init ... FAIL 2022-09-03T08:39:37.458 DEBUG:boto:path=/dsgvcy-79/ 2022-09-03T08:39:37.459 DEBUG:boto:auth_path=/dsgvcy-79/ 2022-09-03T08:39:37.460 DEBUG:boto:Method: PUT 2022-09-03T08:39:37.461 DEBUG:boto:Path: /dsgvcy-79/ 2022-09-03T08:39:37.462 DEBUG:boto:Data: 2022-09-03T08:39:37.463 DEBUG:boto:Headers: {} 2022-09-03T08:39:37.464 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T08:39:37.466 DEBUG:boto:Port: 8000 2022-09-03T08:39:37.467 DEBUG:boto:Params: {} 2022-09-03T08:39:37.469 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T08:39:37.470 DEBUG:boto:Token: None 2022-09-03T08:39:37.472 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 08:39:37 GMT /dsgvcy-79/ 2022-09-03T08:39:37.473 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:fAjv+I5sVRs+ARDWKq098kAjokY= 2022-09-03T08:39:37.474 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 08:39:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:fAjv+I5sVRs+ARDWKq098kAjokY=', 'Content-Length': '0'} 2022-09-03T08:39:37.506 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx000002a7bbb12987f39b8-00631312c9-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 08:39:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T08:39:37.507 DEBUG:rgw_multi.tests:created bucket=dsgvcy-79 2022-09-03T08:39:37.507 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T08:39:37.508 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T08:39:37.510 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T08:39:37.569 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T08:39:37.570 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T08:39:37.610 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T08:39:37.626 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T08:39:37.627 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T08:39:37.627 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T08:39:37.628 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:51:16.173767Z" 2022-09-03T08:39:37.629 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T08:39:37.629 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T08:39:37.629 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T08:39:37.629 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:50:53.931976Z" 2022-09-03T08:39:37.630 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T08:39:37.630 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T08:39:37.630 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662194377.503243_637.1", 2022-09-03T08:39:37.630 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T08:39:37.503243Z" 2022-09-03T08:39:37.631 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T08:39:37.631 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T08:39:37.631 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T08:39:37.632 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T08:39:37.632 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T08:39:37.632 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T08:39:37.634 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662194377.503243_637.1', 3: ''} 2022-09-03T08:39:37.634 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T08:39:37.635 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T08:39:37.635 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T08:39:37.636 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T08:39:37.691 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T08:39:37.692 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T08:44:37.707 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T08:44:37.692+0000 7f3212437600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T08:49:37.696 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T08:49:37.692+0000 7f3212437600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T08:54:37.696 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T08:54:37.692+0000 7f3212437600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T08:59:37.696 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T08:59:37.693+0000 7f3212437600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T09:04:37.698 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T09:04:37.694+0000 7f3212437600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T09:09:37.705 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T09:09:37.695+0000 7f3212437600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T09:14:37.699 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T09:14:37.696+0000 7f3212437600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T09:19:37.699 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T09:19:37.696+0000 7f3212437600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T09:24:37.700 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T09:24:37.696+0000 7f3212437600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T09:29:37.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-09-03T09:29:37.715 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T09:29:37.697+0000 7f3212437600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T09:29:37.715 INFO:teuthology.orchestra.run.smithi189.stderr:failed to fetch mon config (--no-mon-config to skip) 2022-09-03T09:29:37.718 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_resharded_bucket_incremental_sync_latest_after_data_sync_init ... FAIL 2022-09-03T09:29:37.720 DEBUG:boto:path=/dsgvcy-80/ 2022-09-03T09:29:37.721 DEBUG:boto:auth_path=/dsgvcy-80/ 2022-09-03T09:29:37.722 DEBUG:boto:Method: PUT 2022-09-03T09:29:37.723 DEBUG:boto:Path: /dsgvcy-80/ 2022-09-03T09:29:37.723 DEBUG:boto:Data: 2022-09-03T09:29:37.724 DEBUG:boto:Headers: {} 2022-09-03T09:29:37.724 DEBUG:boto:Host: smithi181.front.sepia.ceph.com:8000 2022-09-03T09:29:37.725 DEBUG:boto:Port: 8000 2022-09-03T09:29:37.725 DEBUG:boto:Params: {} 2022-09-03T09:29:37.726 DEBUG:boto:establishing HTTP connection: kwargs={'timeout': 70, 'port': 8000} 2022-09-03T09:29:37.727 DEBUG:boto:Token: None 2022-09-03T09:29:37.728 DEBUG:boto:StringToSign: PUT Sat, 03 Sep 2022 09:29:37 GMT /dsgvcy-80/ 2022-09-03T09:29:37.729 DEBUG:boto:Signature: AWS SA4D9959RL1I1FMQ4Y2O:7mLTGM5eVtEiVJYVLzfgtzHBnyM= 2022-09-03T09:29:37.730 DEBUG:boto:Final headers: {'User-Agent': 'Boto/2.49.0 Python/3.6.9 Linux/5.4.0-113-generic', 'Date': 'Sat, 03 Sep 2022 09:29:37 GMT', 'Authorization': 'AWS SA4D9959RL1I1FMQ4Y2O:7mLTGM5eVtEiVJYVLzfgtzHBnyM=', 'Content-Length': '0'} 2022-09-03T09:29:37.855 DEBUG:boto:Response headers: [('x-amz-request-id', 'tx00000bd79371bcc4011a2-0063131e81-15d7-test-zone1'), ('Content-Length', '0'), ('Date', 'Sat, 03 Sep 2022 09:29:37 GMT'), ('Connection', 'Keep-Alive')] 2022-09-03T09:29:37.856 DEBUG:rgw_multi.tests:created bucket=dsgvcy-80 2022-09-03T09:29:37.857 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T09:29:37.858 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T09:29:37.859 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T09:29:38.130 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T09:29:38.131 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T09:29:38.172 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T09:29:38.189 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T09:29:38.190 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T09:29:38.190 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T09:29:38.191 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:51:16.173767Z" 2022-09-03T09:29:38.191 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T09:29:38.192 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T09:29:38.192 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T09:29:38.193 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:50:53.931976Z" 2022-09-03T09:29:38.193 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T09:29:38.194 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T09:29:38.194 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662197377.849842_673.1", 2022-09-03T09:29:38.195 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T09:29:37.849842Z" 2022-09-03T09:29:38.195 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T09:29:38.196 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T09:29:38.196 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T09:29:38.197 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T09:29:38.197 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T09:29:38.198 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T09:29:38.200 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662197377.849842_673.1', 3: ''} 2022-09-03T09:29:38.201 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T09:29:38.202 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T09:29:38.203 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T09:29:38.204 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T09:29:38.266 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T09:29:38.267 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T09:34:38.271 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T09:34:38.266+0000 7fd963d0d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T09:39:38.270 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T09:39:38.267+0000 7fd963d0d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T09:44:38.271 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T09:44:38.267+0000 7fd963d0d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T09:49:38.271 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T09:49:38.269+0000 7fd963d0d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T09:54:38.275 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T09:54:38.269+0000 7fd963d0d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T09:59:38.275 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T09:59:38.270+0000 7fd963d0d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T10:04:38.275 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T10:04:38.270+0000 7fd963d0d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T10:09:38.275 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T10:09:38.271+0000 7fd963d0d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T10:14:38.276 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T10:14:38.271+0000 7fd963d0d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T10:19:38.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-09-03T10:19:38.323 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T10:19:38.271+0000 7fd963d0d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T10:19:38.324 INFO:teuthology.orchestra.run.smithi189.stderr:failed to fetch mon config (--no-mon-config to skip) 2022-09-03T10:19:38.327 INFO:tasks.rgw_multisite_tests:rgw_multi.tests.test_resharded_bucket_incremental_sync_oldest_after_data_sync_init ... FAIL 2022-09-03T10:19:38.329 INFO:tasks.rgw_multisite_tests:log information for manual testing ... ok 2022-09-03T10:19:38.330 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T10:19:38.336 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T10:19:38.338 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T10:19:38.505 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T10:19:38.506 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T10:19:38.554 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T10:19:38.575 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T10:19:38.575 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T10:19:38.575 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T10:19:38.576 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:51:16.173767Z" 2022-09-03T10:19:38.576 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T10:19:38.576 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T10:19:38.577 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T10:19:38.577 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:50:53.931976Z" 2022-09-03T10:19:38.577 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T10:19:38.577 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T10:19:38.578 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662197377.849842_673.1", 2022-09-03T10:19:38.578 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T09:29:37.849842Z" 2022-09-03T10:19:38.578 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T10:19:38.579 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T10:19:38.579 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T10:19:38.579 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T10:19:38.579 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T10:19:38.580 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T10:19:38.581 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662197377.849842_673.1', 3: ''} 2022-09-03T10:19:38.582 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T10:19:38.582 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T10:19:38.583 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T10:19:38.584 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T10:19:38.643 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T10:19:38.644 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T10:24:38.648 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T10:24:38.644+0000 7fe5af994600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T10:29:38.648 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T10:29:38.644+0000 7fe5af994600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T10:34:38.648 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T10:34:38.644+0000 7fe5af994600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T10:39:38.648 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T10:39:38.644+0000 7fe5af994600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T10:44:38.650 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T10:44:38.645+0000 7fe5af994600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T10:49:38.651 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T10:49:38.644+0000 7fe5af994600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T10:54:38.650 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T10:54:38.646+0000 7fe5af994600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T10:59:38.651 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T10:59:38.647+0000 7fe5af994600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T11:04:38.652 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T11:04:38.647+0000 7fe5af994600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T11:09:38.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-09-03T11:09:38.659 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T11:09:38.648+0000 7fe5af994600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T11:09:38.659 INFO:teuthology.orchestra.run.smithi189.stderr:failed to fetch mon config (--no-mon-config to skip) 2022-09-03T11:09:38.663 INFO:tasks.rgw_multisite_tests:test low level implementation of s3 notifications ... FAIL 2022-09-03T11:09:38.666 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T11:09:38.668 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T11:09:38.670 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T11:09:38.749 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T11:09:38.749 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T11:09:38.802 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T11:09:38.816 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T11:09:38.817 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T11:09:38.817 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T11:09:38.817 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:51:16.173767Z" 2022-09-03T11:09:38.818 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T11:09:38.818 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T11:09:38.818 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T11:09:38.819 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:50:53.931976Z" 2022-09-03T11:09:38.819 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T11:09:38.819 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T11:09:38.819 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662197377.849842_673.1", 2022-09-03T11:09:38.820 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T09:29:37.849842Z" 2022-09-03T11:09:38.820 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T11:09:38.820 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T11:09:38.821 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T11:09:38.821 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T11:09:38.821 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T11:09:38.821 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T11:09:38.824 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662197377.849842_673.1', 3: ''} 2022-09-03T11:09:38.825 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T11:09:38.826 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T11:09:38.826 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T11:09:38.827 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T11:09:38.886 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T11:09:38.887 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T11:14:38.893 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T11:14:38.885+0000 7ff5f9f5b600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T11:19:38.890 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T11:19:38.886+0000 7ff5f9f5b600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T11:24:38.892 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T11:24:38.886+0000 7ff5f9f5b600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T11:29:38.897 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T11:29:38.887+0000 7ff5f9f5b600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T11:34:38.892 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T11:34:38.888+0000 7ff5f9f5b600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T11:39:38.892 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T11:39:38.888+0000 7ff5f9f5b600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T11:44:38.892 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T11:44:38.889+0000 7ff5f9f5b600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T11:49:38.893 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T11:49:38.889+0000 7ff5f9f5b600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T11:54:38.894 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T11:54:38.889+0000 7ff5f9f5b600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T11:59:38.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-09-03T11:59:38.898 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T11:59:38.889+0000 7ff5f9f5b600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T11:59:38.898 INFO:teuthology.orchestra.run.smithi189.stderr:failed to fetch mon config (--no-mon-config to skip) 2022-09-03T11:59:38.901 INFO:tasks.rgw_multisite_tests:test s3 records fetching ... FAIL 2022-09-03T11:59:38.903 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T11:59:38.904 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T11:59:38.905 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T11:59:39.045 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T11:59:39.045 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T11:59:39.091 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T11:59:39.112 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T11:59:39.112 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T11:59:39.113 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T11:59:39.113 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:51:16.173767Z" 2022-09-03T11:59:39.113 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T11:59:39.113 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T11:59:39.114 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T11:59:39.114 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:50:53.931976Z" 2022-09-03T11:59:39.114 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T11:59:39.114 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T11:59:39.115 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662197377.849842_673.1", 2022-09-03T11:59:39.115 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T09:29:37.849842Z" 2022-09-03T11:59:39.115 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T11:59:39.115 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T11:59:39.116 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T11:59:39.116 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T11:59:39.116 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T11:59:39.116 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T11:59:39.118 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662197377.849842_673.1', 3: ''} 2022-09-03T11:59:39.119 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T11:59:39.119 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T11:59:39.120 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T11:59:39.120 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T11:59:39.173 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T11:59:39.173 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T12:04:39.177 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T12:04:39.172+0000 7fd702501600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T12:09:39.182 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T12:09:39.172+0000 7fd702501600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T12:14:39.178 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T12:14:39.172+0000 7fd702501600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T12:19:39.177 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T12:19:39.174+0000 7fd702501600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T12:24:39.178 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T12:24:39.174+0000 7fd702501600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T12:29:39.180 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T12:29:39.175+0000 7fd702501600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T12:34:39.179 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T12:34:39.175+0000 7fd702501600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T12:39:39.181 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T12:39:39.175+0000 7fd702501600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T12:44:39.181 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T12:44:39.176+0000 7fd702501600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T12:49:39.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-09-03T12:49:39.187 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T12:49:39.177+0000 7fd702501600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T12:49:39.187 INFO:teuthology.orchestra.run.smithi189.stderr:failed to fetch mon config (--no-mon-config to skip) 2022-09-03T12:49:39.191 INFO:tasks.rgw_multisite_tests:test s3 notification set/get/delete ... FAIL 2022-09-03T12:49:39.193 INFO:tasks.rgw_multisite_tests:test s3 notification filter on master ... ok 2022-09-03T12:49:39.194 INFO:tasks.rgw_multisite_tests:rgw_multi.tests_ps.test_object_timing ... ok 2022-09-03T12:49:39.195 INFO:tasks.rgw_multisite_tests:test that opaque id set in topic, is sent in notification ... ok 2022-09-03T12:49:39.196 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T12:49:39.197 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T12:49:39.199 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T12:49:39.289 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T12:49:39.289 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T12:49:39.331 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T12:49:39.347 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T12:49:39.348 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T12:49:39.348 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T12:49:39.349 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:51:16.173767Z" 2022-09-03T12:49:39.349 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T12:49:39.350 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T12:49:39.350 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T12:49:39.351 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:50:53.931976Z" 2022-09-03T12:49:39.351 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T12:49:39.352 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T12:49:39.352 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662197377.849842_673.1", 2022-09-03T12:49:39.352 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T09:29:37.849842Z" 2022-09-03T12:49:39.353 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T12:49:39.353 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T12:49:39.353 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T12:49:39.354 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T12:49:39.354 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T12:49:39.354 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T12:49:39.356 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662197377.849842_673.1', 3: ''} 2022-09-03T12:49:39.357 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T12:49:39.358 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T12:49:39.359 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T12:49:39.359 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T12:49:39.415 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T12:49:39.415 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T12:54:39.419 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T12:54:39.415+0000 7fb2e935d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T12:59:39.419 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T12:59:39.415+0000 7fb2e935d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T13:04:39.421 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T13:04:39.415+0000 7fb2e935d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T13:09:39.421 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T13:09:39.416+0000 7fb2e935d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T13:14:39.422 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T13:14:39.417+0000 7fb2e935d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T13:19:39.428 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T13:19:39.417+0000 7fb2e935d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T13:24:39.423 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T13:24:39.418+0000 7fb2e935d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T13:29:39.422 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T13:29:39.419+0000 7fb2e935d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T13:34:39.423 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T13:34:39.418+0000 7fb2e935d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T13:39:39.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-09-03T13:39:39.428 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T13:39:39.419+0000 7fb2e935d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T13:39:39.428 INFO:teuthology.orchestra.run.smithi189.stderr:failed to fetch mon config (--no-mon-config to skip) 2022-09-03T13:39:39.431 INFO:tasks.rgw_multisite_tests:test set/get/delete of topic ... FAIL 2022-09-03T13:39:39.433 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T13:39:39.434 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T13:39:39.435 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T13:39:39.646 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T13:39:39.647 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T13:39:39.681 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T13:39:39.696 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T13:39:39.697 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T13:39:39.697 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T13:39:39.697 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:51:16.173767Z" 2022-09-03T13:39:39.697 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T13:39:39.698 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T13:39:39.698 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T13:39:39.698 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:50:53.931976Z" 2022-09-03T13:39:39.699 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T13:39:39.699 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T13:39:39.699 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662197377.849842_673.1", 2022-09-03T13:39:39.699 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T09:29:37.849842Z" 2022-09-03T13:39:39.700 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T13:39:39.700 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T13:39:39.700 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T13:39:39.700 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T13:39:39.701 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T13:39:39.701 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T13:39:39.702 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662197377.849842_673.1', 3: ''} 2022-09-03T13:39:39.703 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T13:39:39.704 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T13:39:39.704 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T13:39:39.705 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T13:39:39.759 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T13:39:39.760 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T13:44:39.762 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T13:44:39.758+0000 7f46f4a6d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T13:49:39.763 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T13:49:39.758+0000 7f46f4a6d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T13:54:39.763 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T13:54:39.759+0000 7f46f4a6d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T13:59:39.763 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T13:59:39.759+0000 7f46f4a6d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T14:04:39.765 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T14:04:39.760+0000 7f46f4a6d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T14:09:39.765 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T14:09:39.761+0000 7f46f4a6d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T14:14:39.766 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T14:14:39.761+0000 7f46f4a6d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T14:19:39.767 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T14:19:39.762+0000 7f46f4a6d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T14:24:39.766 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T14:24:39.762+0000 7f46f4a6d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T14:29:39.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-09-03T14:29:39.772 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T14:29:39.763+0000 7f46f4a6d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T14:29:39.773 INFO:teuthology.orchestra.run.smithi189.stderr:failed to fetch mon config (--no-mon-config to skip) 2022-09-03T14:29:39.776 INFO:tasks.rgw_multisite_tests:test set topic with endpoint ... FAIL 2022-09-03T14:29:39.778 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T14:29:39.779 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T14:29:39.780 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T14:29:39.969 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T14:29:39.970 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T14:29:40.014 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T14:29:40.029 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T14:29:40.029 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T14:29:40.030 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T14:29:40.030 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:51:16.173767Z" 2022-09-03T14:29:40.030 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T14:29:40.031 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T14:29:40.031 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T14:29:40.031 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:50:53.931976Z" 2022-09-03T14:29:40.032 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T14:29:40.032 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T14:29:40.033 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662197377.849842_673.1", 2022-09-03T14:29:40.033 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T09:29:37.849842Z" 2022-09-03T14:29:40.034 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T14:29:40.034 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T14:29:40.034 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T14:29:40.035 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T14:29:40.035 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T14:29:40.035 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T14:29:40.037 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662197377.849842_673.1', 3: ''} 2022-09-03T14:29:40.037 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T14:29:40.038 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T14:29:40.039 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T14:29:40.040 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T14:29:40.091 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T14:29:40.092 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T14:34:40.095 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T14:34:40.091+0000 7f9295033600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T14:39:40.097 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T14:39:40.092+0000 7f9295033600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T14:44:40.096 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T14:44:40.092+0000 7f9295033600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T14:49:40.096 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T14:49:40.092+0000 7f9295033600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T14:54:40.097 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T14:54:40.093+0000 7f9295033600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T14:59:40.097 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T14:59:40.094+0000 7f9295033600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T15:04:40.098 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T15:04:40.094+0000 7f9295033600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T15:09:40.099 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T15:09:40.095+0000 7f9295033600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T15:14:40.099 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T15:14:40.096+0000 7f9295033600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T15:19:40.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-09-03T15:19:40.106 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T15:19:40.096+0000 7f9295033600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T15:19:40.106 INFO:teuthology.orchestra.run.smithi189.stderr:failed to fetch mon config (--no-mon-config to skip) 2022-09-03T15:19:40.110 INFO:tasks.rgw_multisite_tests:test set/get/delete of notification ... FAIL 2022-09-03T15:19:40.113 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T15:19:40.115 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T15:19:40.116 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T15:19:40.195 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T15:19:40.196 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T15:19:40.239 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T15:19:40.256 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T15:19:40.257 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T15:19:40.258 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T15:19:40.258 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:51:16.173767Z" 2022-09-03T15:19:40.258 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T15:19:40.259 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T15:19:40.259 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T15:19:40.259 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:50:53.931976Z" 2022-09-03T15:19:40.260 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T15:19:40.260 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T15:19:40.260 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662197377.849842_673.1", 2022-09-03T15:19:40.260 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T09:29:37.849842Z" 2022-09-03T15:19:40.261 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T15:19:40.261 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T15:19:40.261 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T15:19:40.262 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T15:19:40.262 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T15:19:40.262 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T15:19:40.264 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662197377.849842_673.1', 3: ''} 2022-09-03T15:19:40.264 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T15:19:40.265 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T15:19:40.266 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T15:19:40.266 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T15:19:40.324 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T15:19:40.325 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T15:24:40.328 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T15:24:40.324+0000 7f9da233d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T15:29:40.329 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T15:29:40.325+0000 7f9da233d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T15:34:40.329 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T15:34:40.325+0000 7f9da233d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T15:39:40.329 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T15:39:40.325+0000 7f9da233d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T15:44:40.331 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T15:44:40.326+0000 7f9da233d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T15:49:40.337 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T15:49:40.327+0000 7f9da233d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T15:54:40.341 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T15:54:40.327+0000 7f9da233d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T15:59:40.345 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T15:59:40.327+0000 7f9da233d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T16:04:40.347 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T16:04:40.328+0000 7f9da233d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T16:09:40.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-09-03T16:09:40.354 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T16:09:40.328+0000 7f9da233d600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T16:09:40.354 INFO:teuthology.orchestra.run.smithi189.stderr:failed to fetch mon config (--no-mon-config to skip) 2022-09-03T16:09:40.357 INFO:tasks.rgw_multisite_tests:test set/get/delete of notification on specific events ... FAIL 2022-09-03T16:09:40.359 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T16:09:40.360 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T16:09:40.361 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T16:09:40.423 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T16:09:40.423 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T16:09:40.483 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T16:09:40.500 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T16:09:40.500 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T16:09:40.501 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T16:09:40.501 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:51:16.173767Z" 2022-09-03T16:09:40.501 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T16:09:40.501 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T16:09:40.502 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T16:09:40.502 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:50:53.931976Z" 2022-09-03T16:09:40.502 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T16:09:40.502 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T16:09:40.503 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662197377.849842_673.1", 2022-09-03T16:09:40.503 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T09:29:37.849842Z" 2022-09-03T16:09:40.503 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T16:09:40.504 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T16:09:40.504 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T16:09:40.504 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T16:09:40.504 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T16:09:40.505 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T16:09:40.506 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662197377.849842_673.1', 3: ''} 2022-09-03T16:09:40.507 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T16:09:40.508 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T16:09:40.508 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T16:09:40.509 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T16:09:40.563 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T16:09:40.564 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T16:14:40.568 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T16:14:40.547+0000 7f281343b600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T16:19:40.570 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T16:19:40.547+0000 7f281343b600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T16:24:40.572 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T16:24:40.548+0000 7f281343b600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T16:29:40.573 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T16:29:40.548+0000 7f281343b600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T16:34:40.574 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T16:34:40.548+0000 7f281343b600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T16:39:40.576 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T16:39:40.548+0000 7f281343b600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T16:44:40.579 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T16:44:40.549+0000 7f281343b600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T16:49:40.579 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T16:49:40.550+0000 7f281343b600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T16:54:40.575 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T16:54:40.550+0000 7f281343b600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T16:59:40.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-09-03T16:59:40.577 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T16:59:40.551+0000 7f281343b600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T16:59:40.577 INFO:teuthology.orchestra.run.smithi189.stderr:failed to fetch mon config (--no-mon-config to skip) 2022-09-03T16:59:40.580 INFO:tasks.rgw_multisite_tests:test set/get/delete of subscription ... FAIL 2022-09-03T16:59:40.582 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T16:59:40.583 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'mdlog', 'status', '--rgw-zone', 'test-zone1', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T16:59:40.584 DEBUG:teuthology.orchestra.run.smithi181:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 mdlog status --rgw-zone test-zone1 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T16:59:40.749 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setuser ceph since I am not root 2022-09-03T16:59:40.750 INFO:teuthology.orchestra.run.smithi181.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T16:59:40.782 INFO:teuthology.orchestra.run.smithi181.stderr:No --period given, using current period=229c309b-f405-4ead-a3a8-baa670c6da51 2022-09-03T16:59:40.801 INFO:teuthology.orchestra.run.smithi181.stdout:[ 2022-09-03T16:59:40.801 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T16:59:40.802 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187876.173767_261.1", 2022-09-03T16:59:40.802 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:51:16.173767Z" 2022-09-03T16:59:40.802 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T16:59:40.802 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T16:59:40.803 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662187853.931976_390.1", 2022-09-03T16:59:40.803 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T06:50:53.931976Z" 2022-09-03T16:59:40.803 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T16:59:40.804 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T16:59:40.804 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "1_1662197377.849842_673.1", 2022-09-03T16:59:40.804 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "2022-09-03T09:29:37.849842Z" 2022-09-03T16:59:40.804 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2022-09-03T16:59:40.805 INFO:teuthology.orchestra.run.smithi181.stdout: { 2022-09-03T16:59:40.805 INFO:teuthology.orchestra.run.smithi181.stdout: "marker": "", 2022-09-03T16:59:40.805 INFO:teuthology.orchestra.run.smithi181.stdout: "last_update": "0.000000" 2022-09-03T16:59:40.805 INFO:teuthology.orchestra.run.smithi181.stdout: } 2022-09-03T16:59:40.806 INFO:teuthology.orchestra.run.smithi181.stdout:] 2022-09-03T16:59:40.807 DEBUG:rgw_multi.tests:master meta markers={0: '1_1662187876.173767_261.1', 1: '1_1662187853.931976_390.1', 2: '1_1662197377.849842_673.1', 3: ''} 2022-09-03T16:59:40.808 INFO:rgw_multi.tests:starting meta checkpoint for zone=test-zone2 2022-09-03T16:59:40.809 INFO:tasks.util.rgw:rgwadmin: c2.client.0 : ['metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T16:59:40.809 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c2', 'metadata', 'sync', 'status', '--rgw-zone', 'test-zone2', '--rgw-zonegroup', 'test-zonegroup', '--rgw-realm', 'test-realm', '--cluster', 'c2', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false'] 2022-09-03T16:59:40.810 DEBUG:teuthology.orchestra.run.smithi189:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c2 metadata sync status --rgw-zone test-zone2 --rgw-zonegroup test-zonegroup --rgw-realm test-realm --cluster c2 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false 2022-09-03T16:59:40.862 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setuser ceph since I am not root 2022-09-03T16:59:40.862 INFO:teuthology.orchestra.run.smithi189.stderr:ignoring --setgroup ceph since I am not root 2022-09-03T17:04:40.861 INFO:teuthology.orchestra.run.smithi189.stderr:2022-09-03T17:04:40.847+0000 7f289f534600 0 monclient(hunting): authenticate timed out after 300 2022-09-03T17:05:22.312 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2022-09-03T17:05:22.317 DEBUG:teuthology.task.console_log:Killing console logger for smithi181 2022-09-03T17:05:22.319 DEBUG:teuthology.task.console_log:Killing console logger for smithi189 2022-09-03T17:05:22.320 DEBUG:teuthology.exit:Finished running handlers